2023-12-19T16:25:19.531 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-19T16:25:19.531 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112 --owner scheduled_vshankar@teuthology --verbose --name vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} -- /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112/orig.config.yaml 2023-12-19T16:25:19.577 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T16:25:19.694 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112 branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: vshankar@redhat.com first_in_suite: false job_id: '7497112' kernel: client: branch: testing flavor: default kdb: true sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e owner: scheduled_vshankar@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 430 sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e sleep_before_teardown: 0 subset: 301/1024 suite: fs suite_branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_568df92b7e50b2108bce4c77c393ff294d6a5e5e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEThGRU3mLKvXf/qg5JsL9om6klrL7d2pnzrm5Fi5uTyVqXIF4g8S5zX5tYXJppael5Gt/C+TDpmOvRWe3aZ47c= smithi160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF897hQ5wuG30hBord1CoC4OmMeyPjDjjNM2QCswEqM6Iw6w8vkW9m7Db/dfhBMAlZZSpd+l6Ah3H5U7ILP4eCY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73 timestamp: 2023-12-19_13:22:41 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1485078 2023-12-19T16:25:19.695 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_568df92b7e50b2108bce4c77c393ff294d6a5e5e/qa; will attempt to use it 2023-12-19T16:25:19.695 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_568df92b7e50b2108bce4c77c393ff294d6a5e5e/qa/tasks 2023-12-19T16:25:19.695 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-19T16:25:19.697 INFO:teuthology.task.internal:Checking packages... 2023-12-19T16:25:19.720 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '568df92b7e50b2108bce4c77c393ff294d6a5e5e' 2023-12-19T16:25:19.720 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-19T16:25:19.720 INFO:teuthology.packaging:ref: None 2023-12-19T16:25:19.720 INFO:teuthology.packaging:tag: None 2023-12-19T16:25:19.720 INFO:teuthology.packaging:branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 2023-12-19T16:25:19.720 INFO:teuthology.packaging:sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-19T16:25:19.720 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-vshankar-testing1-2023-12-18-1207-reef-2 2023-12-19T16:25:19.938 INFO:teuthology.task.internal:Found packages for ceph version 18.2.1-132-g568df92b-1jammy 2023-12-19T16:25:19.939 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-19T16:25:19.945 INFO:teuthology.task.internal:no buildpackages task found 2023-12-19T16:25:19.945 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-19T16:25:19.963 INFO:teuthology.task.internal:Saving configuration 2023-12-19T16:25:19.979 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-19T16:25:19.985 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-19T16:25:20.009 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 16:19:32.316466', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEYGB633wuxw52Rh0kJ1YVSSrka0+oICMRjgeCGU4pAX'} 2023-12-19T16:25:20.032 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 16:19:32.315056', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIO/yVr7uRFZaXgWDc0FbydFiX01ePfJOXgShCC672Eae'} 2023-12-19T16:25:20.033 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-19T16:25:20.039 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-12-19T16:25:20.039 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-12-19T16:25:20.039 INFO:teuthology.run_tasks:Running task console_log... 2023-12-19T16:25:20.113 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff7a9f31b80>, signals=[15]) 2023-12-19T16:25:20.114 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-19T16:25:20.119 INFO:teuthology.task.internal:Opening connections... 2023-12-19T16:25:20.119 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T16:25:20.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T16:25:20.198 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2023-12-19T16:25:20.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T16:25:20.274 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-19T16:25:20.280 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2023-12-19T16:25:20.285 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2023-12-19T16:25:20.285 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2023-12-19T16:25:20.331 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-19T16:25:20.331 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="Ubuntu" 2023-12-19T16:25:20.331 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="22.04" 2023-12-19T16:25:20.332 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-19T16:25:20.332 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_CODENAME=jammy 2023-12-19T16:25:20.332 INFO:teuthology.orchestra.run.smithi026.stdout:ID=ubuntu 2023-12-19T16:25:20.332 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE=debian 2023-12-19T16:25:20.332 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-19T16:25:20.332 INFO:teuthology.orchestra.run.smithi026.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-19T16:25:20.332 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-19T16:25:20.332 INFO:teuthology.orchestra.run.smithi026.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-19T16:25:20.332 INFO:teuthology.orchestra.run.smithi026.stdout:UBUNTU_CODENAME=jammy 2023-12-19T16:25:20.333 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2023-12-19T16:25:20.353 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2023-12-19T16:25:20.358 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2023-12-19T16:25:20.358 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2023-12-19T16:25:20.402 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-19T16:25:20.402 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="Ubuntu" 2023-12-19T16:25:20.402 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="22.04" 2023-12-19T16:25:20.403 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-19T16:25:20.403 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_CODENAME=jammy 2023-12-19T16:25:20.403 INFO:teuthology.orchestra.run.smithi160.stdout:ID=ubuntu 2023-12-19T16:25:20.403 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE=debian 2023-12-19T16:25:20.403 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-19T16:25:20.403 INFO:teuthology.orchestra.run.smithi160.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-19T16:25:20.403 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-19T16:25:20.403 INFO:teuthology.orchestra.run.smithi160.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-19T16:25:20.403 INFO:teuthology.orchestra.run.smithi160.stdout:UBUNTU_CODENAME=jammy 2023-12-19T16:25:20.403 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2023-12-19T16:25:20.428 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-19T16:25:20.436 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-19T16:25:20.507 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-19T16:25:20.508 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2023-12-19T16:25:20.510 DEBUG:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2023-12-19T16:25:20.514 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-19T16:25:20.587 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-19T16:25:20.587 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2023-12-19T16:25:20.589 DEBUG:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2023-12-19T16:25:20.605 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-19T16:25:20.684 INFO:teuthology.run_tasks:Running task kernel... 2023-12-19T16:25:20.697 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'} 2023-12-19T16:25:20.697 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-12-19T16:25:20.697 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-12-19T16:25:20.698 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-19T16:25:20.698 DEBUG:teuthology.orchestra.run.smithi160:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-19T16:25:20.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:25:20.703 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2023-12-19T16:25:20.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:25:20.707 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2023-12-19T16:25:20.885 DEBUG:teuthology.task.kernel:sha1 for client.1 is cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T16:25:20.885 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "cd68252d7b4d7802f3cae7f5945b38e9b20dad4c"... 2023-12-19T16:25:20.886 DEBUG:teuthology.orchestra.run.smithi160:> uname -r 2023-12-19T16:25:20.891 INFO:teuthology.orchestra.run.smithi160.stdout:5.15.0-56-generic 2023-12-19T16:25:20.891 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T16:25:20.892 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-19T16:25:20.892 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2023-12-19T16:25:20.892 DEBUG:teuthology.orchestra.run.smithi160:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-12-19T16:25:20.940 DEBUG:teuthology.task.kernel:sha1 for client.0 is cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T16:25:20.941 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "cd68252d7b4d7802f3cae7f5945b38e9b20dad4c"... 2023-12-19T16:25:20.941 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2023-12-19T16:25:20.944 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-12-19T16:25:20.946 INFO:teuthology.orchestra.run.smithi026.stdout:5.15.0-56-generic 2023-12-19T16:25:20.947 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T16:25:20.947 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-19T16:25:20.947 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2023-12-19T16:25:20.947 DEBUG:teuthology.orchestra.run.smithi026:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-12-19T16:25:21.002 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-12-19T16:25:21.109 INFO:teuthology.orchestra.run.smithi160.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-12-19T16:25:21.109 INFO:teuthology.orchestra.run.smithi160.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-12-19T16:25:21.109 INFO:teuthology.orchestra.run.smithi160.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-12-19T16:25:21.109 INFO:teuthology.orchestra.run.smithi160.stderr:hint: 2023-12-19T16:25:21.109 INFO:teuthology.orchestra.run.smithi160.stderr:hint: git config --global init.defaultBranch 2023-12-19T16:25:21.110 INFO:teuthology.orchestra.run.smithi160.stderr:hint: 2023-12-19T16:25:21.110 INFO:teuthology.orchestra.run.smithi160.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-12-19T16:25:21.110 INFO:teuthology.orchestra.run.smithi160.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-12-19T16:25:21.110 INFO:teuthology.orchestra.run.smithi160.stderr:hint: 2023-12-19T16:25:21.110 INFO:teuthology.orchestra.run.smithi160.stderr:hint: git branch -m 2023-12-19T16:25:21.110 INFO:teuthology.orchestra.run.smithi160.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-12-19T16:25:21.112 DEBUG:teuthology.orchestra.run.smithi160:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-12-19T16:25:21.129 INFO:teuthology.orchestra.run.smithi026.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-12-19T16:25:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-12-19T16:25:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-12-19T16:25:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:hint: 2023-12-19T16:25:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:hint: git config --global init.defaultBranch 2023-12-19T16:25:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:hint: 2023-12-19T16:25:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-12-19T16:25:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-12-19T16:25:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:hint: 2023-12-19T16:25:21.130 INFO:teuthology.orchestra.run.smithi026.stderr:hint: git branch -m 2023-12-19T16:25:21.130 INFO:teuthology.orchestra.run.smithi026.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-12-19T16:25:21.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-12-19T16:25:21.183 DEBUG:teuthology.orchestra.run.smithi160:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-12-19T16:25:21.203 DEBUG:teuthology.orchestra.run.smithi026:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-12-19T16:25:21.241 DEBUG:teuthology.orchestra.run.smithi160:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-12-19T16:25:21.261 DEBUG:teuthology.orchestra.run.smithi026:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-12-19T16:26:37.451 INFO:teuthology.orchestra.run.smithi160.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-12-19T16:26:37.452 INFO:teuthology.orchestra.run.smithi160.stderr: * [new branch] main -> origin/main 2023-12-19T16:26:37.483 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20190312 -> 20190312 2023-12-19T16:26:37.483 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20190416 -> 20190416 2023-12-19T16:26:37.483 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20190514 -> 20190514 2023-12-19T16:26:37.483 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20190618 -> 20190618 2023-12-19T16:26:37.484 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20190717 -> 20190717 2023-12-19T16:26:37.484 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20190815 -> 20190815 2023-12-19T16:26:37.484 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20190923 -> 20190923 2023-12-19T16:26:37.484 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20191022 -> 20191022 2023-12-19T16:26:37.484 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20191215 -> 20191215 2023-12-19T16:26:37.485 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20200122 -> 20200122 2023-12-19T16:26:37.485 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20200316 -> 20200316 2023-12-19T16:26:37.485 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20200421 -> 20200421 2023-12-19T16:26:37.485 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20200519 -> 20200519 2023-12-19T16:26:37.485 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20200619 -> 20200619 2023-12-19T16:26:37.485 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20200721 -> 20200721 2023-12-19T16:26:37.486 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20200817 -> 20200817 2023-12-19T16:26:37.486 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20200918 -> 20200918 2023-12-19T16:26:37.486 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20201022 -> 20201022 2023-12-19T16:26:37.486 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20201118 -> 20201118 2023-12-19T16:26:37.487 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20201218 -> 20201218 2023-12-19T16:26:37.487 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20210208 -> 20210208 2023-12-19T16:26:37.487 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20210315 -> 20210315 2023-12-19T16:26:37.487 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20210511 -> 20210511 2023-12-19T16:26:37.488 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20210716 -> 20210716 2023-12-19T16:26:37.488 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20210818 -> 20210818 2023-12-19T16:26:37.488 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20210919 -> 20210919 2023-12-19T16:26:37.488 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20211027 -> 20211027 2023-12-19T16:26:37.488 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20211216 -> 20211216 2023-12-19T16:26:37.489 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20220209 -> 20220209 2023-12-19T16:26:37.489 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20220310 -> 20220310 2023-12-19T16:26:37.489 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20220411 -> 20220411 2023-12-19T16:26:37.489 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20220509 -> 20220509 2023-12-19T16:26:37.489 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20220610 -> 20220610 2023-12-19T16:26:37.489 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20220708 -> 20220708 2023-12-19T16:26:37.490 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20220815 -> 20220815 2023-12-19T16:26:37.490 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20220913 -> 20220913 2023-12-19T16:26:37.490 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20221012 -> 20221012 2023-12-19T16:26:37.490 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20221109 -> 20221109 2023-12-19T16:26:37.490 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20221214 -> 20221214 2023-12-19T16:26:37.490 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20230117 -> 20230117 2023-12-19T16:26:37.491 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20230210 -> 20230210 2023-12-19T16:26:37.491 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20230310 -> 20230310 2023-12-19T16:26:37.491 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20230404 -> 20230404 2023-12-19T16:26:37.492 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20230515 -> 20230515 2023-12-19T16:26:37.492 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20230625 -> 20230625 2023-12-19T16:26:37.492 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20230804 -> 20230804 2023-12-19T16:26:37.492 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20230919 -> 20230919 2023-12-19T16:26:37.492 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20231030 -> 20231030 2023-12-19T16:26:37.492 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20231111 -> 20231111 2023-12-19T16:26:37.492 INFO:teuthology.orchestra.run.smithi160.stderr: * [new tag] 20231211 -> 20231211 2023-12-19T16:26:41.483 INFO:teuthology.orchestra.run.smithi160.stderr:Updating files: 51% (1673/3256) Updating files: 52% (1694/3256) Updating files: 53% (1726/3256) Updating files: 54% (1759/3256) Updating files: 55% (1791/3256) Updating files: 56% (1824/3256) Updating files: 57% (1856/3256) Updating files: 58% (1889/3256) Updating files: 59% (1922/3256) Updating files: 60% (1954/3256) Updating files: 61% (1987/3256) Updating files: 62% (2019/3256) Updating files: 63% (2052/3256) Updating files: 64% (2084/3256) Updating files: 65% (2117/3256) Updating files: 66% (2149/3256) Updating files: 67% (2182/3256) Updating files: 68% (2215/3256) Updating files: 69% (2247/3256) Updating files: 70% (2280/3256) Updating files: 71% (2312/3256) Updating files: 72% (2345/3256) Updating files: 73% (2377/3256) Updating files: 74% (2410/3256) Updating files: 75% (2442/3256) Updating files: 76% (2475/3256) Updating files: 77% (2508/3256) Updating files: 78% (2540/3256) Updating files: 79% (2573/3256) Updating files: 80% (2605/3256) Updating files: 81% (2638/3256) Updating files: 81% (2653/3256) Updating files: 82% (2670/3256) Updating files: 83% (2703/3256) Updating files: 84% (2736/3256) Updating files: 85% (2768/3256) Updating files: 86% (2801/3256) Updating files: 87% (2833/3256) Updating files: 88% (2866/3256) Updating files: 89% (2898/3256) Updating files: 90% (2931/3256) Updating files: 91% (2963/3256) Updating files: 92% (2996/3256) Updating files: 93% (3029/3256) Updating files: 94% (3061/3256) Updating files: 95% (3094/3256) Updating files: 96% (3126/3256) Updating files: 97% (3159/3256) Updating files: 98% (3191/3256) Updating files: 99% (3224/3256) Updating files: 100% (3256/3256) Updating files: 100% (3256/3256), done. 2023-12-19T16:26:41.487 INFO:teuthology.orchestra.run.smithi160.stdout:HEAD is now at c156e6be Merge branch 'robot/pr-0-1702625169' into 'main' 2023-12-19T16:26:41.508 INFO:teuthology.task.kernel:Downloading kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on client.1... 2023-12-19T16:26:41.509 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T16:26:42.155 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-gcd68252d7b4d/ 2023-12-19T16:26:42.156 DEBUG:teuthology.orchestra.run.smithi160:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-gcd68252d7b4d_6.6.0-gcd68252d7b4d-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-gcd68252d7b4d/ --input-file=- 2023-12-19T16:26:42.158 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi160.front.sepia.ceph.com to complete... 2023-12-19T16:26:44.804 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19 16:26:45 URL:https://2.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-gcd68252d7b4d/linux-image-6.6.0-gcd68252d7b4d_6.6.0-gcd68252d7b4d-1_amd64.deb [38930180/38930180] -> "/tmp/linux-image.deb" [1] 2023-12-19T16:26:44.804 INFO:teuthology.orchestra.run.smithi160.stderr:FINISHED --2023-12-19 16:26:45-- 2023-12-19T16:26:44.804 INFO:teuthology.orchestra.run.smithi160.stderr:Total wall clock time: 2.6s 2023-12-19T16:26:44.804 INFO:teuthology.orchestra.run.smithi160.stderr:Downloaded: 1 files, 37M in 2.4s (15.5 MB/s) 2023-12-19T16:26:44.808 INFO:teuthology.task.kernel:Installing kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on client.1... 2023-12-19T16:26:44.808 DEBUG:teuthology.orchestra.run.smithi160:> sudo dpkg -i /tmp/linux-image.deb 2023-12-19T16:26:45.017 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package linux-image-6.6.0-gcd68252d7b4d. 2023-12-19T16:26:46.954 INFO:teuthology.orchestra.run.smithi160.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-12-19T16:26:46.959 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-12-19T16:26:47.088 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking linux-image-6.6.0-gcd68252d7b4d (6.6.0-gcd68252d7b4d-1) ... 2023-12-19T16:26:47.286 INFO:teuthology.orchestra.run.smithi026.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-12-19T16:26:47.286 INFO:teuthology.orchestra.run.smithi026.stderr: * [new branch] main -> origin/main 2023-12-19T16:26:47.319 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20190312 -> 20190312 2023-12-19T16:26:47.319 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20190416 -> 20190416 2023-12-19T16:26:47.319 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20190514 -> 20190514 2023-12-19T16:26:47.319 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20190618 -> 20190618 2023-12-19T16:26:47.319 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20190717 -> 20190717 2023-12-19T16:26:47.319 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20190815 -> 20190815 2023-12-19T16:26:47.319 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20190923 -> 20190923 2023-12-19T16:26:47.319 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20191022 -> 20191022 2023-12-19T16:26:47.320 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20191215 -> 20191215 2023-12-19T16:26:47.320 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20200122 -> 20200122 2023-12-19T16:26:47.322 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20200316 -> 20200316 2023-12-19T16:26:47.322 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20200421 -> 20200421 2023-12-19T16:26:47.322 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20200519 -> 20200519 2023-12-19T16:26:47.322 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20200619 -> 20200619 2023-12-19T16:26:47.322 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20200721 -> 20200721 2023-12-19T16:26:47.322 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20200817 -> 20200817 2023-12-19T16:26:47.322 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20200918 -> 20200918 2023-12-19T16:26:47.322 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20201022 -> 20201022 2023-12-19T16:26:47.322 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20201118 -> 20201118 2023-12-19T16:26:47.322 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20201218 -> 20201218 2023-12-19T16:26:47.322 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20210208 -> 20210208 2023-12-19T16:26:47.322 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20210315 -> 20210315 2023-12-19T16:26:47.322 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20210511 -> 20210511 2023-12-19T16:26:47.323 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20210716 -> 20210716 2023-12-19T16:26:47.323 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20210818 -> 20210818 2023-12-19T16:26:47.323 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20210919 -> 20210919 2023-12-19T16:26:47.323 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20211027 -> 20211027 2023-12-19T16:26:47.323 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20211216 -> 20211216 2023-12-19T16:26:47.325 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20220209 -> 20220209 2023-12-19T16:26:47.325 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20220310 -> 20220310 2023-12-19T16:26:47.325 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20220411 -> 20220411 2023-12-19T16:26:47.325 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20220509 -> 20220509 2023-12-19T16:26:47.325 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20220610 -> 20220610 2023-12-19T16:26:47.325 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20220708 -> 20220708 2023-12-19T16:26:47.325 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20220815 -> 20220815 2023-12-19T16:26:47.326 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20220913 -> 20220913 2023-12-19T16:26:47.326 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20221012 -> 20221012 2023-12-19T16:26:47.326 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20221109 -> 20221109 2023-12-19T16:26:47.326 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20221214 -> 20221214 2023-12-19T16:26:47.326 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20230117 -> 20230117 2023-12-19T16:26:47.326 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20230210 -> 20230210 2023-12-19T16:26:47.326 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20230310 -> 20230310 2023-12-19T16:26:47.326 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20230404 -> 20230404 2023-12-19T16:26:47.326 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20230515 -> 20230515 2023-12-19T16:26:47.326 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20230625 -> 20230625 2023-12-19T16:26:47.326 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20230804 -> 20230804 2023-12-19T16:26:47.326 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20230919 -> 20230919 2023-12-19T16:26:47.326 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20231030 -> 20231030 2023-12-19T16:26:47.327 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20231111 -> 20231111 2023-12-19T16:26:47.327 INFO:teuthology.orchestra.run.smithi026.stderr: * [new tag] 20231211 -> 20231211 2023-12-19T16:26:50.643 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up linux-image-6.6.0-gcd68252d7b4d (6.6.0-gcd68252d7b4d-1) ... 2023-12-19T16:26:50.751 INFO:teuthology.orchestra.run.smithi160.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-gcd68252d7b4d 2023-12-19T16:26:51.532 INFO:teuthology.orchestra.run.smithi026.stderr:Updating files: 48% (1591/3256) Updating files: 49% (1596/3256) Updating files: 50% (1628/3256) Updating files: 51% (1661/3256) Updating files: 52% (1694/3256) Updating files: 53% (1726/3256) Updating files: 54% (1759/3256) Updating files: 55% (1791/3256) Updating files: 56% (1824/3256) Updating files: 57% (1856/3256) Updating files: 58% (1889/3256) Updating files: 59% (1922/3256) Updating files: 60% (1954/3256) Updating files: 61% (1987/3256) Updating files: 62% (2019/3256) Updating files: 63% (2052/3256) Updating files: 63% (2076/3256) Updating files: 64% (2084/3256) Updating files: 65% (2117/3256) Updating files: 66% (2149/3256) Updating files: 67% (2182/3256) Updating files: 68% (2215/3256) Updating files: 69% (2247/3256) Updating files: 70% (2280/3256) Updating files: 71% (2312/3256) Updating files: 72% (2345/3256) Updating files: 73% (2377/3256) Updating files: 74% (2410/3256) Updating files: 75% (2442/3256) Updating files: 76% (2475/3256) Updating files: 77% (2508/3256) Updating files: 78% (2540/3256) Updating files: 79% (2573/3256) Updating files: 79% (2584/3256) Updating files: 80% (2605/3256) Updating files: 81% (2638/3256) Updating files: 82% (2670/3256) Updating files: 83% (2703/3256) Updating files: 84% (2736/3256) Updating files: 85% (2768/3256) Updating files: 85% (2769/3256) Updating files: 86% (2801/3256) Updating files: 87% (2833/3256) Updating files: 88% (2866/3256) Updating files: 89% (2898/3256) Updating files: 90% (2931/3256) Updating files: 91% (2963/3256) Updating files: 92% (2996/3256) Updating files: 93% (3029/3256) Updating files: 94% (3061/3256) Updating files: 95% (3094/3256) Updating files: 96% (3126/3256) Updating files: 97% (3159/3256) Updating files: 98% (3191/3256) Updating files: 99% (3224/3256) Updating files: 100% (3256/3256) Updating files: 100% (3256/3256), done. 2023-12-19T16:26:51.537 INFO:teuthology.orchestra.run.smithi026.stdout:HEAD is now at c156e6be Merge branch 'robot/pr-0-1702625169' into 'main' 2023-12-19T16:26:51.559 INFO:teuthology.task.kernel:Downloading kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on client.0... 2023-12-19T16:26:51.560 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T16:26:51.779 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-gcd68252d7b4d/ 2023-12-19T16:26:51.779 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-gcd68252d7b4d_6.6.0-gcd68252d7b4d-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-gcd68252d7b4d/ --input-file=- 2023-12-19T16:26:51.782 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi026.front.sepia.ceph.com to complete... 2023-12-19T16:26:53.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19 16:26:53 URL:https://2.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-gcd68252d7b4d/linux-image-6.6.0-gcd68252d7b4d_6.6.0-gcd68252d7b4d-1_amd64.deb [38930180/38930180] -> "/tmp/linux-image.deb" [1] 2023-12-19T16:26:53.434 INFO:teuthology.orchestra.run.smithi026.stderr:FINISHED --2023-12-19 16:26:53-- 2023-12-19T16:26:53.434 INFO:teuthology.orchestra.run.smithi026.stderr:Total wall clock time: 1.6s 2023-12-19T16:26:53.434 INFO:teuthology.orchestra.run.smithi026.stderr:Downloaded: 1 files, 37M in 1.4s (26.0 MB/s) 2023-12-19T16:26:53.439 INFO:teuthology.task.kernel:Installing kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on client.0... 2023-12-19T16:26:53.439 DEBUG:teuthology.orchestra.run.smithi026:> sudo dpkg -i /tmp/linux-image.deb 2023-12-19T16:26:53.656 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package linux-image-6.6.0-gcd68252d7b4d. 2023-12-19T16:26:55.635 INFO:teuthology.orchestra.run.smithi026.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-12-19T16:26:55.639 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-12-19T16:26:55.756 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking linux-image-6.6.0-gcd68252d7b4d (6.6.0-gcd68252d7b4d-1) ... 2023-12-19T16:26:59.456 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up linux-image-6.6.0-gcd68252d7b4d (6.6.0-gcd68252d7b4d-1) ... 2023-12-19T16:26:59.567 INFO:teuthology.orchestra.run.smithi026.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-gcd68252d7b4d 2023-12-19T16:27:11.661 INFO:teuthology.orchestra.run.smithi160.stderr:Sourcing file `/etc/default/grub' 2023-12-19T16:27:11.685 INFO:teuthology.orchestra.run.smithi160.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-19T16:27:11.714 INFO:teuthology.orchestra.run.smithi160.stderr:Generating grub configuration file ... 2023-12-19T16:27:11.991 INFO:teuthology.orchestra.run.smithi160.stderr:Found linux image: /boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-19T16:27:12.009 INFO:teuthology.orchestra.run.smithi160.stderr:Found initrd image: /boot/initrd.img-6.6.0-gcd68252d7b4d 2023-12-19T16:27:12.319 INFO:teuthology.orchestra.run.smithi160.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-19T16:27:12.334 INFO:teuthology.orchestra.run.smithi160.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-19T16:27:12.401 INFO:teuthology.orchestra.run.smithi160.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-19T16:27:12.407 INFO:teuthology.orchestra.run.smithi160.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-19T16:27:12.550 INFO:teuthology.orchestra.run.smithi160.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-19T16:27:12.550 INFO:teuthology.orchestra.run.smithi160.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-12-19T16:27:12.550 INFO:teuthology.orchestra.run.smithi160.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-19T16:27:12.596 INFO:teuthology.orchestra.run.smithi160.stderr:done 2023-12-19T16:27:12.765 DEBUG:teuthology.orchestra.run.smithi160:> dpkg-deb -c /tmp/linux-image.deb 2023-12-19T16:27:12.777 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./ 2023-12-19T16:27:12.777 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./boot/ 2023-12-19T16:27:12.777 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 4415617 2023-12-08 17:49 ./boot/System.map-6.6.0-gcd68252d7b4d 2023-12-19T16:27:12.789 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 169214 2023-12-08 17:49 ./boot/config-6.6.0-gcd68252d7b4d 2023-12-19T16:27:12.789 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11352864 2023-12-08 17:49 ./boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-19T16:27:12.808 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./etc/ 2023-12-19T16:27:12.808 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./etc/kernel/ 2023-12-19T16:27:12.808 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./etc/kernel/postinst.d/ 2023-12-19T16:27:12.808 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./etc/kernel/postrm.d/ 2023-12-19T16:27:12.809 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./etc/kernel/preinst.d/ 2023-12-19T16:27:12.809 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./etc/kernel/prerm.d/ 2023-12-19T16:27:12.809 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/ 2023-12-19T16:27:12.809 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/ 2023-12-19T16:27:12.809 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/ 2023-12-19T16:27:12.809 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/ 2023-12-19T16:27:12.809 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/ 2023-12-19T16:27:12.809 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/ 2023-12-19T16:27:12.809 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ 2023-12-19T16:27:12.809 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 490233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/aesni-intel.ko 2023-12-19T16:27:12.811 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-19T16:27:12.811 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-19T16:27:12.811 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-19T16:27:12.811 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 93113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-19T16:27:12.811 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 106729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 152241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 83889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 40529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 58585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 50497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-19T16:27:12.812 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-19T16:27:12.813 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/ 2023-12-19T16:27:12.813 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/ 2023-12-19T16:27:12.813 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/ 2023-12-19T16:27:12.813 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 40113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-19T16:27:12.813 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpuid.ko 2023-12-19T16:27:12.813 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/msr.ko 2023-12-19T16:27:12.813 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/ 2023-12-19T16:27:12.813 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 354433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-19T16:27:12.813 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 881625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-19T16:27:12.813 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 2275865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm.ko 2023-12-19T16:27:12.824 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ 2023-12-19T16:27:12.825 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 53697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/af_alg.ko 2023-12-19T16:27:12.825 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/algif_hash.ko 2023-12-19T16:27:12.825 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/algif_skcipher.ko 2023-12-19T16:27:12.825 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ansi_cprng.ko 2023-12-19T16:27:12.825 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/anubis.ko 2023-12-19T16:27:12.825 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/arc4.ko 2023-12-19T16:27:12.825 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/asymmetric_keys/ 2023-12-19T16:27:12.825 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-12-19T16:27:12.825 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/ 2023-12-19T16:27:12.825 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_memcpy.ko 2023-12-19T16:27:12.825 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_pq.ko 2023-12-19T16:27:12.825 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-19T16:27:12.825 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_tx.ko 2023-12-19T16:27:12.826 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_xor.ko 2023-12-19T16:27:12.826 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/raid6test.ko 2023-12-19T16:27:12.826 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authenc.ko 2023-12-19T16:27:12.826 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authencesn.ko 2023-12-19T16:27:12.826 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blake2b_generic.ko 2023-12-19T16:27:12.826 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_common.ko 2023-12-19T16:27:12.826 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_generic.ko 2023-12-19T16:27:12.826 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/camellia_generic.ko 2023-12-19T16:27:12.826 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast5_generic.ko 2023-12-19T16:27:12.826 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast6_generic.ko 2023-12-19T16:27:12.826 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast_common.ko 2023-12-19T16:27:12.826 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ccm.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cmac.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crc32_generic.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cryptd.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_engine.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_simd.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_user.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cts.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/des_generic.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/dh_generic.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/echainiv.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/essiv.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/fcrypt.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/khazad.ko 2023-12-19T16:27:12.827 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lrw.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4hc.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/md4.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/michael_mic.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcbc.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcrypt.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/rmd160.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/seed.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/serpent_generic.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 149193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tcrypt.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tea.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_common.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_generic.ko 2023-12-19T16:27:12.828 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/vmac.ko 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/wp512.ko 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xcbc.ko 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xor.ko 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xts.ko 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xxhash_generic.ko 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_extlog.ko 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_pad.ko 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/apei/ 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/apei/einj.ko 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ec_sys.ko 2023-12-19T16:27:12.829 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/platform_profile.ko 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbs.ko 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbshc.ko 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 103025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/video.ko 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/acard-ahci.ko 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 88865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci.ko 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci_platform.ko 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 88289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci.ko 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci_platform.ko 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_acpi.ko 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ali.ko 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_amd.ko 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_artop.ko 2023-12-19T16:27:12.830 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atiixp.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atp867x.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd640.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd64x.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cypress.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_efar.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt366.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt37x.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it8213.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it821x.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_jmicron.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_legacy.ko 2023-12-19T16:27:12.831 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_marvell.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_mpiix.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_netcell.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ninja32.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87410.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87415.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_oldpiix.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_opti.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_optidma.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pcmcia.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_piccolo.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_radisys.ko 2023-12-19T16:27:12.832 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rdc.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rz1000.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sch.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_serverworks.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sil680.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sl82c105.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_triflex.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_via.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pdc_adma.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_inic162x.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 83345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_mv.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 70393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_nv.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_promise.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_qstor.ko 2023-12-19T16:27:12.833 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil.ko 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil24.ko 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sis.ko 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_svw.ko 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 44793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sx4.ko 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_uli.ko 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_via.ko 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_vsc.ko 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ks0108.ko 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/ 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 124473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/bcma.ko 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/ 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/ 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 113665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/aoe.ko 2023-12-19T16:27:12.834 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/ 2023-12-19T16:27:12.835 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 965585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/drbd.ko 2023-12-19T16:27:12.868 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 173225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/floppy.ko 2023-12-19T16:27:12.868 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/ 2023-12-19T16:27:12.868 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 94297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-19T16:27:12.868 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 115793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/nbd.ko 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/ 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 110385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/null_blk.ko 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 98193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/pktcdvd.ko 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 263769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/rbd.ko 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/ 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 98145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/ 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 60217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/zram.ko 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/applicom.ko 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hangcheck-timer.ko 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/ 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-19T16:27:12.869 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/via-rng.ko 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 148601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 139313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 70921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 53513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/lp.ko 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/ 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 84097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/mwave.ko 2023-12-19T16:27:12.870 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/nvram.ko 2023-12-19T16:27:12.871 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ppdev.ko 2023-12-19T16:27:12.871 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tlclk.ko 2023-12-19T16:27:12.871 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/ 2023-12-19T16:27:12.871 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-19T16:27:12.871 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-12-19T16:27:12.871 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-12-19T16:27:12.871 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-12-19T16:27:12.871 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-19T16:27:12.871 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-19T16:27:12.871 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-12-19T16:27:12.871 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/xillybus/ 2023-12-19T16:27:12.871 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/xillybus/xillybus_class.ko 2023-12-19T16:27:12.871 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 50945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/xillybus/xillybus_core.ko 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/clk/ 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/clk/clk-palmas.ko 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/clk/clk-si5351.ko 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/clk/clk-twl6040.ko 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/clk/clk-wm831x.ko 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/ 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 70801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 243025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp.ko 2023-12-19T16:27:12.872 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/ 2023-12-19T16:27:12.873 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/ 2023-12-19T16:27:12.873 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/ 2023-12-19T16:27:12.873 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 438761 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-19T16:27:12.873 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-12-19T16:27:12.873 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-19T16:27:12.873 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-aes.ko 2023-12-19T16:27:12.873 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-sha.ko 2023-12-19T16:27:12.873 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/ 2023-12-19T16:27:12.873 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 70273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-19T16:27:12.873 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/ 2023-12-19T16:27:12.873 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/dca.ko 2023-12-19T16:27:12.873 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ 2023-12-19T16:27:12.873 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/ 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 122569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 79953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/amd64_edac.ko 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/e752x_edac.ko 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 63617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/edac_mce_amd.ko 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3000_edac.ko 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3200_edac.ko 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5100_edac.ko 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5400_edac.ko 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7300_edac.ko 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 52041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7core_edac.ko 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i82975x_edac.ko 2023-12-19T16:27:12.874 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ie31200_edac.ko 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 56825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/sb_edac.ko 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/x38_edac.ko 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/ 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-gpio.ko 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-max14577.ko 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-max77693.ko 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-max8997.ko 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-palmas.ko 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-rt8973a.ko 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-sm5502.ko 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/ 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/dmi-sysfs.ko 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/efi/ 2023-12-19T16:27:12.875 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/efi/efi-pstore.ko 2023-12-19T16:27:12.876 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-19T16:27:12.876 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/ 2023-12-19T16:27:12.876 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-adp5520.ko 2023-12-19T16:27:12.876 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-amd8111.ko 2023-12-19T16:27:12.876 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-arizona.ko 2023-12-19T16:27:12.876 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-da9052.ko 2023-12-19T16:27:12.876 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-da9055.ko 2023-12-19T16:27:12.876 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-dln2.ko 2023-12-19T16:27:12.876 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 77849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-f7188x.ko 2023-12-19T16:27:12.876 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-generic.ko 2023-12-19T16:27:12.876 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ich.ko 2023-12-19T16:27:12.876 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-12-19T16:27:12.876 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-kempld.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-lp3943.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-max7300.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-max730x.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-max732x.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-pca953x.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-pcf857x.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-rdc321x.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-sch.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-sch311x.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-tps65912.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-twl4030.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-twl6040.ko 2023-12-19T16:27:12.877 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-vx855.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-wm831x.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-wm8350.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-wm8994.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/ 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-a4tech.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-apple.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-appleir.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-aureal.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-axff.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-belkin.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-cherry.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-chicony.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 60649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-cp2112.ko 2023-12-19T16:27:12.878 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-cypress.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-dr.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elecom.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elo.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-emsff.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ezkey.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gaff.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-generic.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gt683r.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gyration.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtekff.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-hyperv.ko 2023-12-19T16:27:12.879 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-icade.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kensington.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-keytouch.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kye.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lcpower.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-led.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 42297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lenovo.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lg-g15.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 58185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 101481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 80993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-magicmouse.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-microsoft.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-monterey.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 50161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-multitouch.ko 2023-12-19T16:27:12.880 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ntrig.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ortek.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-penmount.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-petalynx.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 72937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-picolcd.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-pl.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-plantronics.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-primax.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-rmi.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-common.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-19T16:27:12.881 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-saitek.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-samsung.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sjoy.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sony.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-speedlink.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-steelseries.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sunplus.ko 2023-12-19T16:27:12.882 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tivo.ko 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tmff.ko 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-topseed.ko 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-twinhan.ko 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 71073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-uclogic.ko 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-waltop.ko 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 143473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-wiimote.ko 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-xinmo.ko 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zpff.ko 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zydacron.ko 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 267825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid.ko 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/uhid.ko 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/usbhid/ 2023-12-19T16:27:12.883 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 125721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/usbhid/usbhid.ko 2023-12-19T16:27:12.884 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/usbhid/usbkbd.ko 2023-12-19T16:27:12.884 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/usbhid/usbmouse.ko 2023-12-19T16:27:12.884 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 217873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/wacom.ko 2023-12-19T16:27:12.884 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/ 2023-12-19T16:27:12.884 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 64201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_balloon.ko 2023-12-19T16:27:12.884 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 89625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_utils.ko 2023-12-19T16:27:12.884 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 305465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_vmbus.ko 2023-12-19T16:27:12.884 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ 2023-12-19T16:27:12.884 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 66521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru.ko 2023-12-19T16:27:12.884 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 61393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru3.ko 2023-12-19T16:27:12.884 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-19T16:27:12.884 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7414.ko 2023-12-19T16:27:12.884 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7418.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adc128d818.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1025.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 97977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1026.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1029.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 45553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1031.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm9240.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ads7828.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7410.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7411.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 74433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7462.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7470.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 67929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7475.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7x10.ko 2023-12-19T16:27:12.885 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/amc6821.ko 2023-12-19T16:27:12.886 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 55201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/applesmc.ko 2023-12-19T16:27:12.886 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 49913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asb100.ko 2023-12-19T16:27:12.886 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 58617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asc7621.ko 2023-12-19T16:27:12.886 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 40825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-19T16:27:12.886 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/atxp1.ko 2023-12-19T16:27:12.886 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/coretemp.ko 2023-12-19T16:27:12.886 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/da9052-hwmon.ko 2023-12-19T16:27:12.886 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/da9055-hwmon.ko 2023-12-19T16:27:12.886 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 44465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-19T16:27:12.886 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 99601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dme1737.ko 2023-12-19T16:27:12.886 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds1621.ko 2023-12-19T16:27:12.886 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds620.ko 2023-12-19T16:27:12.886 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc1403.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc2103.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc6w201.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 66985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71805f.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 94457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71882fg.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 41401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f75375s.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fam15h_power.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 55449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fschmd.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g760a.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g762.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 46801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl518sm.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 45529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl520sm.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hih6130.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5500_temp.ko 2023-12-19T16:27:12.887 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5k_amb.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmaem.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmpex.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina209.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina2xx.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 130449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/it87.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/jc42.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k10temp.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k8temp.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lineage-pem.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 48601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm63.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm73.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm75.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm77.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm78.ko 2023-12-19T16:27:12.888 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm80.ko 2023-12-19T16:27:12.889 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm83.ko 2023-12-19T16:27:12.889 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 72833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm85.ko 2023-12-19T16:27:12.889 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 46153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm87.ko 2023-12-19T16:27:12.889 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm90.ko 2023-12-19T16:27:12.889 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm92.ko 2023-12-19T16:27:12.889 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 108841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm93.ko 2023-12-19T16:27:12.889 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95234.ko 2023-12-19T16:27:12.889 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95241.ko 2023-12-19T16:27:12.889 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95245.ko 2023-12-19T16:27:12.889 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc2945.ko 2023-12-19T16:27:12.889 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4151.ko 2023-12-19T16:27:12.889 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4215.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4222.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4245.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4260.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4261.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max16065.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1619.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1668.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max197.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6639.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6650.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6697.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mc13783-adc.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mcp3021.ko 2023-12-19T16:27:12.890 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 44129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6683.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 119817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775-core.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 60705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 59737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct7802.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 82337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87360.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 54433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87427.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pcf8591.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-19T16:27:12.891 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 82577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/powr1220.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5627.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5636.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht15.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht21.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/shtc1.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 40105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sis5595.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47b397.ko 2023-12-19T16:27:12.892 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m1.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m192.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/thmc50.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp102.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp103.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp401.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp421.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via-cputemp.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 41945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via686a.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 52457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt1211.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt8231.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 57449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627ehf.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 78873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627hf.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 71345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83781d.ko 2023-12-19T16:27:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 63225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83791d.ko 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 71321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83792d.ko 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 101473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83793.ko 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 119505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83795.ko 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l785ts.ko 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l786ng.ko 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/ 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/ 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/ 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-12-19T16:27:12.894 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 6929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 50201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 62817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-19T16:27:12.895 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-19T16:27:12.896 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-12-19T16:27:12.896 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-19T16:27:12.896 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-19T16:27:12.896 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-12-19T16:27:12.896 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-19T16:27:12.896 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-19T16:27:12.896 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-19T16:27:12.896 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-12-19T16:27:12.896 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-19T16:27:12.896 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-19T16:27:12.896 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-12-19T16:27:12.896 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-12-19T16:27:12.896 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-19T16:27:12.897 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-12-19T16:27:12.897 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-19T16:27:12.897 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-19T16:27:12.897 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-19T16:27:12.897 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-19T16:27:12.897 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-12-19T16:27:12.897 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-mux.ko 2023-12-19T16:27:12.897 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-smbus.ko 2023-12-19T16:27:12.897 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-stub.ko 2023-12-19T16:27:12.897 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/muxes/ 2023-12-19T16:27:12.897 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-12-19T16:27:12.897 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-12-19T16:27:12.897 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 273609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 879969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_core.ko 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 69273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 324217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 97985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 282273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 66201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/ 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/ 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 493073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/ 2023-12-19T16:27:12.898 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 419737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/ 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 739025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 296625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/ocrdma/ 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 153697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/usnic/ 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 168969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 263081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 100241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-19T16:27:12.899 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 111425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 128753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 129529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/ 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/evbug.ko 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/ff-memless.ko 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/ 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 41241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/gameport.ko 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 41801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/joydev.ko 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/ 2023-12-19T16:27:12.900 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24761 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/lkkbd.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/lm8323.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/lm8333.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/newtonkbd.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-12-19T16:27:12.901 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/qt1070.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/qt2160.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/stowaway.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/sunkbd.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/xtkbd.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/matrix-keymap.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/ 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/appletouch.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/bcm5974.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 123681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/cyapatp.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 89449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-19T16:27:12.902 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/gpio_mouse.ko 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 335929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/psmouse.ko 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/sermouse.ko 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/ 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 109873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/ 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/altera_ps2.ko 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/arc_ps2.ko 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/ct82c710.ko 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/parkbd.ko 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/pcips2.ko 2023-12-19T16:27:12.903 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/ps2mult.ko 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serio_raw.ko 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serport.ko 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/sparse-keymap.ko 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/ 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/ 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/ 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-88pm860x.ko 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-adp5520.ko 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 59753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-bd2802.ko 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-blinkm.ko 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-da903x.ko 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-da9052.ko 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-gpio.ko 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3530.ko 2023-12-19T16:27:12.904 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3533.ko 2023-12-19T16:27:12.905 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm355x.ko 2023-12-19T16:27:12.905 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3642.ko 2023-12-19T16:27:12.905 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp3944.ko 2023-12-19T16:27:12.905 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp8788.ko 2023-12-19T16:27:12.905 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-max8997.ko 2023-12-19T16:27:12.905 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-mc13783.ko 2023-12-19T16:27:12.905 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-menf21bmc.ko 2023-12-19T16:27:12.905 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-pca9532.ko 2023-12-19T16:27:12.905 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-pca955x.ko 2023-12-19T16:27:12.905 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-pca963x.ko 2023-12-19T16:27:12.905 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-ss4200.ko 2023-12-19T16:27:12.905 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-tca6507.ko 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-wm831x-status.ko 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-wm8350.ko 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/ 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/ 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 654969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/bcache.ko 2023-12-19T16:27:12.906 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 40577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bio-prison.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 89601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bufio.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache-smq.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 156009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 106897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-crypt.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-delay.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 53857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-era.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-flakey.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 41529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log-userspace.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 49545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-mirror.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 78033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-multipath.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-queue-length.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 70825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-raid.ko 2023-12-19T16:27:12.907 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-region-hash.ko 2023-12-19T16:27:12.908 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-round-robin.ko 2023-12-19T16:27:12.908 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-service-time.ko 2023-12-19T16:27:12.908 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 109529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-snapshot.ko 2023-12-19T16:27:12.908 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-switch.ko 2023-12-19T16:27:12.908 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 181865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-thin-pool.ko 2023-12-19T16:27:12.948 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-verity.ko 2023-12-19T16:27:12.948 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-zero.ko 2023-12-19T16:27:12.948 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/faulty.ko 2023-12-19T16:27:12.948 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/linear.ko 2023-12-19T16:27:12.948 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/multipath.ko 2023-12-19T16:27:12.948 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/ 2023-12-19T16:27:12.948 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 199937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-19T16:27:12.948 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid0.ko 2023-12-19T16:27:12.948 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 113073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid1.ko 2023-12-19T16:27:12.948 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 158681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid10.ko 2023-12-19T16:27:12.948 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 395721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid456.ko 2023-12-19T16:27:12.948 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/ 2023-12-19T16:27:12.949 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/ 2023-12-19T16:27:12.949 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 178137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptbase.ko 2023-12-19T16:27:12.949 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 63385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptctl.ko 2023-12-19T16:27:12.949 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 44665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptfc.ko 2023-12-19T16:27:12.949 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptlan.ko 2023-12-19T16:27:12.949 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 123321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptsas.ko 2023-12-19T16:27:12.949 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 79041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptscsih.ko 2023-12-19T16:27:12.949 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 53017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptspi.ko 2023-12-19T16:27:12.949 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/ 2023-12-19T16:27:12.949 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/88pm800.ko 2023-12-19T16:27:12.949 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12761 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/88pm805.ko 2023-12-19T16:27:12.949 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/88pm80x.ko 2023-12-19T16:27:12.949 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/arizona-i2c.ko 2023-12-19T16:27:12.950 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 123721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/arizona.ko 2023-12-19T16:27:12.950 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/bcm590xx.ko 2023-12-19T16:27:12.950 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/cros_ec_dev.ko 2023-12-19T16:27:12.950 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/dln2.ko 2023-12-19T16:27:12.950 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/janz-cmodio.ko 2023-12-19T16:27:12.950 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 53585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/kempld-core.ko 2023-12-19T16:27:12.950 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lm3533-core.ko 2023-12-19T16:27:12.950 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-12-19T16:27:12.950 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lp3943.ko 2023-12-19T16:27:12.950 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 54569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_ich.ko 2023-12-19T16:27:12.950 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_sch.ko 2023-12-19T16:27:12.950 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/max8907.ko 2023-12-19T16:27:12.950 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mc13xxx-core.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/menf21bmc.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/pcf50633-adc.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/pcf50633-gpio.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/pcf50633.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/retu-mfd.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 80425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/si476x-core.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 48457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/sm501.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/tps65010.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/tps6507x.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/viperboard.ko 2023-12-19T16:27:12.951 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/vx855.ko 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wl1273-core.ko 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 44345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wm8994.ko 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 48489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ad525x_dpot.ko 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/ 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 46585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds9802als.ko 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds990x.ko 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 45593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/bh1770glc.ko 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/c2port/ 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/c2port/core.ko 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/ 2023-12-19T16:27:12.952 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/cb710.ko 2023-12-19T16:27:12.953 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ds1682.ko 2023-12-19T16:27:12.953 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/dummy-irq.ko 2023-12-19T16:27:12.953 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/echo/ 2023-12-19T16:27:12.953 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/echo/echo.ko 2023-12-19T16:27:12.953 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/ 2023-12-19T16:27:12.953 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/at24.ko 2023-12-19T16:27:12.953 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-19T16:27:12.953 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-19T16:27:12.953 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/max6875.ko 2023-12-19T16:27:12.953 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/enclosure.ko 2023-12-19T16:27:12.953 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/genwqe/ 2023-12-19T16:27:12.953 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 152201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-12-19T16:27:12.953 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hmc6352.ko 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hpilo.ko 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ibmasm/ 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 72129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ics932s401.ko 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29003.ko 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29020.ko 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/ 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 42513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/ 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 125209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-me.ko 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 56521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-txe.ko 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 310113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei.ko 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/phantom.ko 2023-12-19T16:27:12.954 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ti-st/ 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 74265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ti-st/st_drv.ko 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_7xx1.ko 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_core.ko 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tsl2550.ko 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 48697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_balloon.ko 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/ 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 183609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/ 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 490129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/bonding.ko 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/ 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/ 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 106033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 40417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-19T16:27:12.955 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/ 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 453737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dummy.ko 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/eql.ko 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/ 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 45785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 42105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 130553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 68849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 59689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-19T16:27:12.956 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 99049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/ 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 54521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/ 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 69137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/ 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 59585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/ 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 41409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/ 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 54609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 49289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 87297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/ 2023-12-19T16:27:12.957 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/ 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 97337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 104673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 90257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/ 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 81569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 69729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/ 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 78401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 176761 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 1421497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-19T16:27:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 128569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/ 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 132089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 312081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/ 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/ 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 323513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/ 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 193497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 347417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 1044497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-12-19T16:27:12.959 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 166193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/ 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/ 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 204345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/ 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/ 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 69449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 61489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 141225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 62817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/ 2023-12-19T16:27:12.960 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 60873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 62833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 45721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dnet.ko 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ec_bhf.ko 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/ 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/ 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 281137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 49457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ethoc.ko 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fealnx.ko 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fujitsu/ 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 45345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 90465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/ 2023-12-19T16:27:12.961 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 295577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/ 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 584497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/ 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 246209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/ 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 925825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/ 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 302937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/ 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 514641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/ 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 93593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-12-19T16:27:12.962 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 734305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 151769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 86921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/jme.ko 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/ 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 102289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 122649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/ 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 727601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 278937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 1113681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-19T16:27:12.963 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 53041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 40993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 95665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/ 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 108649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 73441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/ 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 150905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/ 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 136721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-19T16:27:12.964 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/ 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 50481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 43145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/ 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 227897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 87081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 672049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/ 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 43017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/ 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 67385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 83985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-19T16:27:12.965 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 189801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/samsung/ 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 104241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/ 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 1052497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/ 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/ 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 54817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 63337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/ 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 61537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 67193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-19T16:27:12.966 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 73409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 68177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/ 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 487025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/ 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 103337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 166097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 66329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 78801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/ 2023-12-19T16:27:12.967 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 50057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/ 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 86529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/ 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 80273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 83633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/ 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 46241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/ 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 59993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/fddi/ 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 41001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/fddi/defxx.ko 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/fddi/skfp/ 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 216985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/fddi/skfp/skfp.ko 2023-12-19T16:27:12.968 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 77345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/geneve.ko 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/ 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 235521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/ 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ifb.ko 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 83769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 62145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvlan.ko 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvtap.ko 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/ 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-gpio.ko 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-19T16:27:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio.ko 2023-12-19T16:27:12.970 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mii.ko 2023-12-19T16:27:12.970 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 43953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/netconsole.ko 2023-12-19T16:27:12.972 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/nlmon.ko 2023-12-19T16:27:12.972 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/ 2023-12-19T16:27:12.972 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-19T16:27:12.972 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-19T16:27:12.972 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/amd.ko 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 50033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 44721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm87xx.ko 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 42241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/broadcom.ko 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/cicada.ko 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/davicom.ko 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/et1011c.ko 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/icplus.ko 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/lxt.ko 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 82049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/marvell.ko 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 76113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/micrel.ko 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/national.ko 2023-12-19T16:27:12.973 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 97857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/phylink.ko 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/qsemi.ko 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 50065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/realtek.ko 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/smsc.ko 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ste10Xp.ko 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/vitesse.ko 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/plip/ 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 43505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/plip/plip.ko 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_async.ko 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-19T16:27:12.974 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 42545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppoe.ko 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppox.ko 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pptp.ko 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sb1000.ko 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/ 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 42465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slip.ko 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sungem_phy.ko 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 64449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tap.ko 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/ 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 106169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team.ko 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_random.ko 2023-12-19T16:27:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 87017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/veth.ko 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/ 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 144297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/ 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 297329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/vxlan.ko 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/ 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 144585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ntb.ko 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/ 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/ 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 265121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme-core.ko 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 102209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme.ko 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/ 2023-12-19T16:27:12.976 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 133841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport.ko 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_cs.ko 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 84465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_pc.ko 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_serial.ko 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/ 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/ 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pci-stub.ko 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 46465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/xen-pcifront.ko 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/ 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/i82092.ko 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 145809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pcmcia.ko 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 62137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pcmcia_core.ko 2023-12-19T16:27:12.977 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pd6729.ko 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 112889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/phy/ 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/phy/broadcom/ 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/phy/samsung/ 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/ 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/ 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 70153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/ 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/ 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 50889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros_ec.ko 2023-12-19T16:27:12.978 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/ 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 67097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acerhdf.ko 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 59665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 127441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 49121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-19T16:27:12.979 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 40265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/hdaps.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/ibm_rtl.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/ 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 43497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel_ips.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 42329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-19T16:27:12.980 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-19T16:27:12.981 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-19T16:27:12.981 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-19T16:27:12.981 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 57569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi.ko 2023-12-19T16:27:12.981 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/ 2023-12-19T16:27:12.981 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/ 2023-12-19T16:27:12.981 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/88pm860x_battery.ko 2023-12-19T16:27:12.981 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/88pm860x_charger.ko 2023-12-19T16:27:12.981 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 55057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/bq2415x_charger.ko 2023-12-19T16:27:12.981 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 54081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/bq24190_charger.ko 2023-12-19T16:27:12.981 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/bq24735-charger.ko 2023-12-19T16:27:12.981 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-12-19T16:27:12.981 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/da9030_battery.ko 2023-12-19T16:27:12.981 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/da9052-battery.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/ds2760_battery.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/ds2780_battery.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/ds2781_battery.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/ds2782_battery.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/gpio-charger.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/isp1704_charger.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/lp8727_charger.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/max14577_charger.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/max17040_battery.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/max17042_battery.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/max8903_charger.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/max8925_power.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/pcf50633-charger.ko 2023-12-19T16:27:12.982 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/sbs-battery.ko 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/test_power.ko 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/tps65090-charger.ko 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/wm831x_backup.ko 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/wm831x_power.ko 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/wm8350_power.ko 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/ 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 57729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/ 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/ 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps_parport.ko 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 41873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/pps_core.ko 2023-12-19T16:27:12.983 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 66385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp.ko 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp_kvm.ko 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/ 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-88pm80x.ko 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-88pm860x.ko 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-da9052.ko 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-da9055.ko 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-da9063.ko 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 68577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-19T16:27:12.984 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-em3027.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-lp8788.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t86.ko 2023-12-19T16:27:12.985 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max6900.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max8907.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max8925.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max8997.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max8998.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-palmas.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 46633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf50633.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-19T16:27:12.986 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rc5t583.ko 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-s35390a.ko 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-tps6586x.ko 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-tps65910.ko 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-wm831x.ko 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-wm8350.ko 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-x1205.ko 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ 2023-12-19T16:27:12.987 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 77617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-9xxx.ko 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 57665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-sas.ko 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 55105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-xxxx.ko 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 135185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/BusLogic.ko 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/a100u2w.ko 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/ 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 272305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 100617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/advansys.ko 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/ 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 356201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 295801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic94xx/ 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 189185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/am53c974.ko 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/ 2023-12-19T16:27:12.988 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 106881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/atp870u.ko 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/ 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 246985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/ 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 1171945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/bfa.ko 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/ 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 237273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/ 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 115369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 40585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ch.ko 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/ 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 308921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/ 2023-12-19T16:27:12.989 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 76145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 101449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 146689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 75553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dc395x.ko 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dmx3191d.ko 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/ 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 212425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 60505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esp_scsi.ko 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/ 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 78209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 158505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fdomain.ko 2023-12-19T16:27:12.990 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/ 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 263489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/fnic.ko 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 201817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hpsa.ko 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hptiop.ko 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 49041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hv_storvsc.ko 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/imm.ko 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 44057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/initio.ko 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 266537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ipr.ko 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 101881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ips.ko 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/ 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 317153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/isci.ko 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 52961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/ 2023-12-19T16:27:12.991 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 314865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/libfc.ko 2023-12-19T16:27:12.992 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 145585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi.ko 2023-12-19T16:27:12.992 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 53521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-19T16:27:12.992 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/ 2023-12-19T16:27:12.992 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 216665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/libsas.ko 2023-12-19T16:27:12.992 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/ 2023-12-19T16:27:12.992 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 2197249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-19T16:27:12.992 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/ 2023-12-19T16:27:12.992 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 73569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-19T16:27:12.992 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-19T16:27:12.992 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 310617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-19T16:27:13.030 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 84721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid.ko 2023-12-19T16:27:13.030 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/ 2023-12-19T16:27:13.030 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 659825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-19T16:27:13.030 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/ 2023-12-19T16:27:13.030 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 119561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-19T16:27:13.030 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 78129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvumi.ko 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pcmcia/ 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 81625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/ 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 364105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 123617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pmcraid.ko 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ppa.ko 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 55361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla1280.ko 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/ 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 1876689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 85289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/ 2023-12-19T16:27:13.031 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 559033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-19T16:27:13.032 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qlogicfas408.ko 2023-12-19T16:27:13.032 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/raid_class.ko 2023-12-19T16:27:13.032 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 238761 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_debug.ko 2023-12-19T16:27:13.032 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 173937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-19T16:27:13.032 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 279313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-19T16:27:13.032 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 97729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-19T16:27:13.032 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 73881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-19T16:27:13.032 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 44993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-19T16:27:13.032 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ses.ko 2023-12-19T16:27:13.032 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 115889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/st.ko 2023-12-19T16:27:13.032 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 46825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/stex.ko 2023-12-19T16:27:13.032 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/ 2023-12-19T16:27:13.032 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 162121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 42297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/virtio_scsi.ko 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 50321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/wd719x.ko 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 50025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/xen-scsifront.ko 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 129441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ssb.ko 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/ 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/ 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 736601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/ 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 49985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_file.ko 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 43025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_iblock.ko 2023-12-19T16:27:13.033 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 930257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_mod.ko 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 44305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_pscsi.ko 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 122521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_user.ko 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/ 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 75593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/ 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/ 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/ 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-19T16:27:13.034 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 78425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 57865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/moxa.ko 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 50953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/mxser.ko 2023-12-19T16:27:13.035 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 108601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_gsm.ko 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_hdlc.ko 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 67929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/nozomi.ko 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/ 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/ 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 59497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/altera_uart.ko 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/arc_uart.ko 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 72425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/ 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 103073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/rp2.ko 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 52817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/sc16is7xx.ko 2023-12-19T16:27:13.036 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/ 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 531265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/ 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 44513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio.ko 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_aec.ko 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_cif.ko 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_dmem_genirq.ko 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_mf624.ko 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_netx.ko 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pci_generic.ko 2023-12-19T16:27:13.037 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_sercos3.ko 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/ 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/c67x00/ 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 73297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/c67x00/c67x00.ko 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/chipidea/ 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 144377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/ 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 87089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-acm.ko 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usblp.ko 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 89857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usbtmc.ko 2023-12-19T16:27:13.038 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/common/ 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/common/ulpi.ko 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/dwc2/ 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/dwc3/ 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 448505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/dwc3/dwc3.ko 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/ 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/ 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 60177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/u_ether.ko 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/u_serial.ko 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 42449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-12-19T16:27:13.039 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 92489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 122081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 57681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 57729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 85609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 58113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 67337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/ 2023-12-19T16:27:13.040 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 62185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 161225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/libcomposite.ko 2023-12-19T16:27:13.041 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/ 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/bdc/ 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 157681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 52721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 69705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 56033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 76057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/net2272.ko 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 75073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/net2280.ko 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 73169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 65809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 66097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 81505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 156073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-12-19T16:27:13.042 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/ 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/bcma-hcd.ko 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 80113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/isp116x-hcd.ko 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 87353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 78873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811_cs.ko 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/ssb-hcd.ko 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/ 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/mdc800.ko 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/microtek.ko 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 41721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/adutux.ko 2023-12-19T16:27:13.043 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/appledisplay.ko 2023-12-19T16:27:13.044 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-12-19T16:27:13.044 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/cytherm.ko 2023-12-19T16:27:13.044 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ehset.ko 2023-12-19T16:27:13.044 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi26.ko 2023-12-19T16:27:13.044 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi62.ko 2023-12-19T16:27:13.044 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ezusb.ko 2023-12-19T16:27:13.044 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/idmouse.ko 2023-12-19T16:27:13.044 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/iowarrior.ko 2023-12-19T16:27:13.044 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-19T16:27:13.044 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ldusb.ko 2023-12-19T16:27:13.044 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/legousbtower.ko 2023-12-19T16:27:13.044 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/lvstest.ko 2023-12-19T16:27:13.044 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/ 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 64841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usb3503.ko 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usblcd.ko 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 70809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbtest.ko 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 41905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/uss720.ko 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/yurex.ko 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/mon/ 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 72953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/mon/usbmon.ko 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/musb/ 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 301697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/musb/musb_hdrc.ko 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/ 2023-12-19T16:27:13.045 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-generic.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-isp1301.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-tahvo.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/aircable.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ark3116.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ch341.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 70257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cp210x.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cyberjack.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 58217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 49817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/empeg.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 44273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/f81232.ko 2023-12-19T16:27:13.046 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 175137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 53113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 137081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 124833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_ti.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 67769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipaq.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipw.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ir-usb.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 58233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 107521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mct_u232.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/metro-usb.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 66449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7720.ko 2023-12-19T16:27:13.047 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 78217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7840.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 56337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mxuport.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/navman.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/omninet.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/opticon.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 191137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/option.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/oti6858.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 54353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/pl2303.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcaux.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcserial.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/quatech2.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/safe_serial.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 51625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/sierra.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ssu100.ko 2023-12-19T16:27:13.048 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/symbolserial.ko 2023-12-19T16:27:13.049 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 61489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-19T16:27:13.049 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-19T16:27:13.049 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_debug.ko 2023-12-19T16:27:13.049 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-19T16:27:13.049 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 125609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usbserial.ko 2023-12-19T16:27:13.049 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/visor.ko 2023-12-19T16:27:13.049 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/whiteheat.ko 2023-12-19T16:27:13.049 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/wishbone-serial.ko 2023-12-19T16:27:13.049 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-19T16:27:13.049 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ 2023-12-19T16:27:13.049 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 52913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/uas.ko 2023-12-19T16:27:13.049 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-19T16:27:13.049 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-karma.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 179969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/usb-storage.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/ 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 67481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-19T16:27:13.050 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 72177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 107001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/ 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/ 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 154297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 108737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio.ko 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 77609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/ 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 113785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost.ko 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 58089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_net.ko 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 89249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_scsi.ko 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/ 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/ 2023-12-19T16:27:13.051 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/aat2870_bl.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/adp5520_bl.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/adp8860_bl.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/adp8870_bl.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/apple_bl.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/as3711_bl.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/bd6107.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/da903x_bl.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/da9052_bl.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/gpio_backlight.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/kb3886_bl.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lcd.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lm3533_bl.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lm3639_bl.ko 2023-12-19T16:27:13.052 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lv5207lp.ko 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/max8925_bl.ko 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/pandora_bl.ko 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/wm831x_bl.ko 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/ 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2482.ko 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2490.ko 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/matrox_w1.ko 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/w1-gpio.ko 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/ 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-19T16:27:13.053 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 56425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 102673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/wire.ko 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/acquirewdt.ko 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/advantechwdt.ko 2023-12-19T16:27:13.054 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/cpu5wdt.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/da9052_wdt.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/da9055_wdt.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/da9063_wdt.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/dw_wdt.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/eurotechwdt.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/hpwdt.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/i6300esb.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ib700wdt.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ibmasr.ko 2023-12-19T16:27:13.055 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it87_wdt.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/kempld_wdt.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/machzwd.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/mena21_wdt.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/nv_tco.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pc87413_wdt.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/retu_wdt.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-12-19T16:27:13.056 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sc1200wdt.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/softdog.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/twl4030_wdt.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/via_wdt.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wafer5823wdt.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wdt_pci.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wm831x_wdt.ko 2023-12-19T16:27:13.057 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wm8350_wdt.ko 2023-12-19T16:27:13.058 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/xen_wdt.ko 2023-12-19T16:27:13.058 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/ 2023-12-19T16:27:13.058 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-evtchn.ko 2023-12-19T16:27:13.058 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntalloc.ko 2023-12-19T16:27:13.058 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntdev.ko 2023-12-19T16:27:13.058 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/ 2023-12-19T16:27:13.058 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 176377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-19T16:27:13.058 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 40121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-privcmd.ko 2023-12-19T16:27:13.058 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 71249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-scsiback.ko 2023-12-19T16:27:13.058 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/ 2023-12-19T16:27:13.058 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-19T16:27:13.058 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ 2023-12-19T16:27:13.058 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/ 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 166425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/9p.ko 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/adfs/ 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 61073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/adfs/adfs.ko 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/ 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 163593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/affs.ko 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/afs/ 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 932817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/afs/kafs.ko 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/autofs/ 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 101057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/autofs/autofs4.ko 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/ 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 64545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/befs.ko 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/bfs/ 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 41257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/bfs/bfs.ko 2023-12-19T16:27:13.059 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/binfmt_misc.ko 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/ 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 3402729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/btrfs.ko 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/ 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 306009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/cachefiles.ko 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 1333017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ceph.ko 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/ 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 100801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/coda.ko 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/configfs/ 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 108393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/configfs/configfs.ko 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/ 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 29625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/cramfs.ko 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/ 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 642513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/dlm.ko 2023-12-19T16:27:13.060 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/efs/ 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/efs/efs.ko 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/ 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 1505025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/f2fs.ko 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/ 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/msdos.ko 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/freevxfs/ 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 42545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/freevxfs/freevxfs.ko 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/ 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 226481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/fscache.ko 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/ 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/cuse.ko 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/ 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 1231529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/gfs2.ko 2023-12-19T16:27:13.061 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/ 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 141209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/hfs.ko 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/ 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 222817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/hfsplus.ko 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hpfs/ 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 192329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hpfs/hpfs.ko 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/ 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 74065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/isofs.ko 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/ 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 419497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/jfs.ko 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/ 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 280313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/lockd.ko 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/ 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 89193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/minix.ko 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/ 2023-12-19T16:27:13.062 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 124649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/netfs.ko 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/ 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/ 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 81449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/ 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 59449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/ 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 117465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 1055721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfs.ko 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 66441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv2.ko 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 120177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv3.ko 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 2042137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv4.ko 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/ 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/grace.ko 2023-12-19T16:27:13.063 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/nfs_acl.ko 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/ 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 1516873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/nfsd.ko 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/ 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 592833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/nilfs2.ko 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/ 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-celtic.ko 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-centeuro.ko 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-croatian.ko 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-cyrillic.ko 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-gaelic.ko 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-greek.ko 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-iceland.ko 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-inuit.ko 2023-12-19T16:27:13.064 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-roman.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-romanian.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-turkish.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ascii.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1250.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1251.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1255.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp737.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp775.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp850.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp852.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp855.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp857.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp860.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp861.ko 2023-12-19T16:27:13.065 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp862.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp863.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp864.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp865.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp866.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp869.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp874.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 94137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp932.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 142673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp936.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 165745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp949.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 118121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp950.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_euc-jp.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-1.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-13.ko 2023-12-19T16:27:13.066 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-14.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-15.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-2.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-3.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-4.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-5.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-6.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-7.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-9.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-r.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-ru.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-u.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_utf8.ko 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ntfs/ 2023-12-19T16:27:13.067 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 219417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ntfs/ntfs.ko 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 244409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 440737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 54049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 2592433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2.ko 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/omfs/ 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 53433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/omfs/omfs.ko 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/ 2023-12-19T16:27:13.068 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 352161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/overlay.ko 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 54809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ramoops.ko 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/qnx4/ 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/qnx4/qnx4.ko 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/qnx6/ 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 46689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/qnx6/qnx6.ko 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/quota/ 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/quota/quota_tree.ko 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/quota/quota_v1.ko 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/quota/quota_v2.ko 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/ 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 532673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/reiserfs.ko 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/ 2023-12-19T16:27:13.069 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/romfs.ko 2023-12-19T16:27:13.070 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/ 2023-12-19T16:27:13.070 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/ 2023-12-19T16:27:13.070 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 2668401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/cifs.ko 2023-12-19T16:27:13.103 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/ 2023-12-19T16:27:13.103 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_arc4.ko 2023-12-19T16:27:13.103 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_md4.ko 2023-12-19T16:27:13.103 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/ 2023-12-19T16:27:13.103 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 134641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/squashfs.ko 2023-12-19T16:27:13.103 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/ 2023-12-19T16:27:13.103 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 90721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/sysv.ko 2023-12-19T16:27:13.103 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/ 2023-12-19T16:27:13.103 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 247737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/udf.ko 2023-12-19T16:27:13.103 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ 2023-12-19T16:27:13.103 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 162497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ufs.ko 2023-12-19T16:27:13.103 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/ 2023-12-19T16:27:13.103 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 4011017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/xfs.ko 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/ 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/locking/ 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 75489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/locking/locktorture.ko 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/time/ 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/time/test_udelay.ko 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 60553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/torture.ko 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 6833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc-itu-t.ko 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 6529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc7.ko 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc8.ko 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/ 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libarc4.ko 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libdes.ko 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/interval_tree_test.ko 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/libcrc32c.ko 2023-12-19T16:27:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lru_cache.ko 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/ 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4_compress.ko 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4hc_compress.ko 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/ 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 6617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/cordic.ko 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/memory-notifier-error-inject.ko 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/notifier-error-inject.ko 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/percpu_test.ko 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/pm-notifier-error-inject.ko 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/ 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 143433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/raid6_pq.ko 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/rbtree_test.ko 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/ 2023-12-19T16:27:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/test-kstrtox.ko 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/test-string_helpers.ko 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 4760425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/test_bpf.ko 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 49089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/test_firmware.ko 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 6369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/test_module.ko 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/test_user_copy.ko 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_bm.ko 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_fsm.ko 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_kmp.ko 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/xz/ 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/xz/xz_dec_test.ko 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/ 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/hwpoison-inject.ko 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ 2023-12-19T16:27:13.106 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/ 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 55737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/6lowpan.ko 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_dest.ko 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_fragment.ko 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_hop.ko 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 6985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_mobility.ko 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_routing.ko 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_udp.ko 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/ 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/garp.ko 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/mrp.ko 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/p8022.ko 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/psnap.ko 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/stp.ko 2023-12-19T16:27:13.107 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/ 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 81737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/8021q.ko 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/ 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 198337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet.ko 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_fd.ko 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_rdma.ko 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_virtio.ko 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/ 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 649297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/batman-adv.ko 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/ 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 63809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/br_netfilter.ko 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 777689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/bridge.ko 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-19T16:27:13.108 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-19T16:27:13.109 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-19T16:27:13.110 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-19T16:27:13.110 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-19T16:27:13.110 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-19T16:27:13.110 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-19T16:27:13.110 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 55929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtables.ko 2023-12-19T16:27:13.110 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-19T16:27:13.110 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-19T16:27:13.110 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/ 2023-12-19T16:27:13.110 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 1046681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/libceph.ko 2023-12-19T16:27:13.110 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/ 2023-12-19T16:27:13.110 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 125993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/pktgen.ko 2023-12-19T16:27:13.110 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/ 2023-12-19T16:27:13.110 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 194209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp.ko 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_diag.ko 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 57881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv4.ko 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 60337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv6.ko 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/ 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 390473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/dsa_core.ko 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_brcm.ko 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_dsa.ko 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_none.ko 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_trailer.ko 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/hsr/ 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 99729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/hsr/hsr.ko 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ 2023-12-19T16:27:13.111 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 45233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 254625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154.ko 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ah4.ko 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/esp4.ko 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 48337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/fou.ko 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/gre.ko 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 48657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/inet_diag.ko 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 60633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_gre.ko 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 61225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_tunnel.ko 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_vti.ko 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipcomp.ko 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipip.ko 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ 2023-12-19T16:27:13.112 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 41609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ip_tables.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-19T16:27:13.113 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_bic.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_dctcp.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_diag.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_highspeed.ko 2023-12-19T16:27:13.114 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_htcp.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_hybla.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_illinois.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_lp.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_scalable.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_vegas.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_veno.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_westwood.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_yeah.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tunnel4.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_diag.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 41665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_tunnel.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ah6.ko 2023-12-19T16:27:13.115 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/esp6.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/fou6.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 80897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_gre.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 93577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_tunnel.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 53273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_vti.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ipcomp6.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/mip6.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 42169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-19T16:27:13.116 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-19T16:27:13.117 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 74073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/sit.ko 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/tunnel6.ko 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/ 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 82833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/af_key.ko 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/ 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 125281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_core.ko 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 25953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_eth.ko 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip.ko 2023-12-19T16:27:13.118 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip6.ko 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_netlink.ko 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 53241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ppp.ko 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/lapb/ 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 47977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/lapb/lapb.ko 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/ 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc.ko 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 267913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc2.ko 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/ 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 290417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/mac802154.ko 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/ 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/mpls_gso.ko 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ 2023-12-19T16:27:13.119 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 119585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 100297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 101441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 102817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 103417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 112849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 60329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 109593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 113129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 112585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 111585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-19T16:27:13.120 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 114465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 411729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-19T16:27:13.121 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 49257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conncount.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 408257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33761 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 118145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 107233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 66641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-19T16:27:13.122 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 89425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 37129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_log_syslog.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 95353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_irc.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_sip.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 44065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 615649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_tables.ko 2023-12-19T16:27:13.123 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 42561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 39785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_log.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 49377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_chain_nat.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_compat.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 40721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_ct.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fib.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_hash.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_limit.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_log.ko 2023-12-19T16:27:13.124 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_masq.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_nat.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_numgen.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_osf.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_queue.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_redir.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject_inet.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_socket.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_synproxy.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_tproxy.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 89313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/x_tables.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_AUDIT.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-19T16:27:13.125 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 18169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CT.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_DSCP.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HL.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HMARK.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LED.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LOG.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NETMAP.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFLOG.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_RATEEST.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 14337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_SECMARK.ko 2023-12-19T16:27:13.126 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TEE.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TPROXY.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TRACE.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_addrtype.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_bpf.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cgroup.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cluster.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_comment.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connbytes.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlabel.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlimit.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connmark.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_conntrack.ko 2023-12-19T16:27:13.127 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cpu.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dccp.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_devgroup.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dscp.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ecn.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_esp.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 43289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hashlimit.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_helper.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hl.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipcomp.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_iprange.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipvs.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_l2tp.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_length.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_limit.ko 2023-12-19T16:27:13.128 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mac.ko 2023-12-19T16:27:13.129 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mark.ko 2023-12-19T16:27:13.129 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_multiport.ko 2023-12-19T16:27:13.129 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nat.ko 2023-12-19T16:27:13.129 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nfacct.ko 2023-12-19T16:27:13.129 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_osf.ko 2023-12-19T16:27:13.129 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_owner.ko 2023-12-19T16:27:13.129 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_physdev.ko 2023-12-19T16:27:13.129 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_pkttype.ko 2023-12-19T16:27:13.129 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_policy.ko 2023-12-19T16:27:13.129 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_quota.ko 2023-12-19T16:27:13.129 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_rateest.ko 2023-12-19T16:27:13.129 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_realm.ko 2023-12-19T16:27:13.129 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_recent.ko 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_sctp.ko 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_set.ko 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 16089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_socket.ko 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_state.ko 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_statistic.ko 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_string.ko 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 8553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpmss.ko 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpudp.ko 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_time.ko 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_u32.ko 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/ 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/netlink_diag.ko 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/ 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 12425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/nsh.ko 2023-12-19T16:27:13.130 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/ 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 380033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/openvswitch.ko 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-geneve.ko 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-gre.ko 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 11793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-vxlan.ko 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/ 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/af_packet_diag.ko 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/phonet/ 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 110201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/phonet/phonet.ko 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 54721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/phonet/pn_pep.ko 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/ 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 251481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds.ko 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 164937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_rdma.ko 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 67377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_tcp.ko 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rxrpc/ 2023-12-19T16:27:13.131 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 819657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rxrpc/rxrpc.ko 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/ 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_csum.ko 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_gact.ko 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_ipt.ko 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_mirred.ko 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_nat.ko 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 43449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_pedit.ko 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_police.ko 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_simple.ko 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_skbedit.ko 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_vlan.ko 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_basic.ko 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_bpf.ko 2023-12-19T16:27:13.132 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_cgroup.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_flow.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_fw.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 38425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_route.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 61409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_u32.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_cmp.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 13409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_ipset.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_meta.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_nbyte.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_text.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_u32.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 21649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_choke.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_codel.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 34633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_drr.ko 2023-12-19T16:27:13.133 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 33609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq_codel.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 35433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_gred.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 49817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hfsc.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hhf.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 66393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_htb.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_ingress.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 30273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio_lib.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_multiq.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 36137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_netem.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 27497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_pie.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_plug.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 24729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_prio.ko 2023-12-19T16:27:13.134 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 48729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_qfq.ko 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 32905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_red.ko 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 26881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfb.ko 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 31353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfq.ko 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_tbf.ko 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 28105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_teql.ko 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/ 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 929745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp.ko 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 19033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp_diag.ko 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/ 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/ 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 336529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 54737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 1520097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/sunrpc.ko 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/ 2023-12-19T16:27:13.135 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 707657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/ 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/diag.ko 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 862665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/tipc.ko 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/ 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/unix_diag.ko 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/ 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 85409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 68049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 96193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock.ko 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 10705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 17521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/x25/ 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 125177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/x25/x25.ko 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/ 2023-12-19T16:27:13.136 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 22657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_algo.ko 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 20649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 102257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_user.ko 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/ 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/ 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 15537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/irqbypass.ko 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 640081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.alias 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 624107 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.alias.bin 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 7663 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.builtin 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 23428 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.alias.bin 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 9821 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.bin 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 70004 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.modinfo 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 146313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.dep 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 232548 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.dep.bin 2023-12-19T16:27:13.137 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 210 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.devname 2023-12-19T16:27:13.138 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 72606 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.order 2023-12-19T16:27:13.138 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 879 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.softdep 2023-12-19T16:27:13.138 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 254438 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.symbols 2023-12-19T16:27:13.138 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 309363 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.symbols.bin 2023-12-19T16:27:13.138 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./usr/ 2023-12-19T16:27:13.138 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./usr/share/ 2023-12-19T16:27:13.138 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./usr/share/doc/ 2023-12-19T16:27:13.138 INFO:teuthology.orchestra.run.smithi160.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./usr/share/doc/linux-image-6.6.0-gcd68252d7b4d/ 2023-12-19T16:27:13.138 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 170 2023-12-08 17:49 ./usr/share/doc/linux-image-6.6.0-gcd68252d7b4d/changelog.Debian.gz 2023-12-19T16:27:13.138 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-r--r-- root/root 691 2023-12-08 17:49 ./usr/share/doc/linux-image-6.6.0-gcd68252d7b4d/copyright 2023-12-19T16:27:13.139 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-gcd68252d7b4d 2023-12-19T16:27:13.139 INFO:teuthology.task.kernel:searching for kernel 6.6.0-gcd68252d7b4d 2023-12-19T16:27:13.139 DEBUG:teuthology.orchestra.run.smithi160:> egrep '(submenu|menuentry.*6.6.0-gcd68252d7b4d).*{' /boot/grub/grub.cfg 2023-12-19T16:27:13.152 INFO:teuthology.orchestra.run.smithi160.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-12-19T16:27:13.152 INFO:teuthology.orchestra.run.smithi160.stdout: menuentry 'Ubuntu, with Linux 6.6.0-gcd68252d7b4d' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-gcd68252d7b4d-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-12-19T16:27:13.152 INFO:teuthology.orchestra.run.smithi160.stdout: menuentry 'Ubuntu, with Linux 6.6.0-gcd68252d7b4d (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-gcd68252d7b4d-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-12-19T16:27:13.152 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-12-19T16:27:13.153 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-gcd68252d7b4d 2023-12-19T16:27:13.153 DEBUG:teuthology.orchestra.run.smithi160:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-12-19T16:27:13.195 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi160.front.sepia.ceph.com to complete... 2023-12-19T16:27:13.614 INFO:teuthology.orchestra.run.smithi160.stderr:Sourcing file `/etc/default/grub' 2023-12-19T16:27:13.615 INFO:teuthology.orchestra.run.smithi160.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-19T16:27:13.644 INFO:teuthology.orchestra.run.smithi160.stderr:Generating grub configuration file ... 2023-12-19T16:27:13.861 INFO:teuthology.orchestra.run.smithi160.stderr:Found linux image: /boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-19T16:27:13.867 INFO:teuthology.orchestra.run.smithi160.stderr:Found initrd image: /boot/initrd.img-6.6.0-gcd68252d7b4d 2023-12-19T16:27:14.119 INFO:teuthology.orchestra.run.smithi160.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-19T16:27:14.124 INFO:teuthology.orchestra.run.smithi160.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-19T16:27:14.166 INFO:teuthology.orchestra.run.smithi160.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-19T16:27:14.171 INFO:teuthology.orchestra.run.smithi160.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-19T16:27:14.320 INFO:teuthology.orchestra.run.smithi160.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-19T16:27:14.320 INFO:teuthology.orchestra.run.smithi160.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-12-19T16:27:14.320 INFO:teuthology.orchestra.run.smithi160.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-19T16:27:14.331 INFO:teuthology.orchestra.run.smithi160.stderr:done 2023-12-19T16:27:21.073 INFO:teuthology.orchestra.run.smithi026.stderr:Sourcing file `/etc/default/grub' 2023-12-19T16:27:21.085 INFO:teuthology.orchestra.run.smithi026.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-19T16:27:21.143 INFO:teuthology.orchestra.run.smithi026.stderr:Generating grub configuration file ... 2023-12-19T16:27:21.556 INFO:teuthology.orchestra.run.smithi026.stderr:Found linux image: /boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-19T16:27:21.571 INFO:teuthology.orchestra.run.smithi026.stderr:Found initrd image: /boot/initrd.img-6.6.0-gcd68252d7b4d 2023-12-19T16:27:21.960 INFO:teuthology.orchestra.run.smithi026.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-19T16:27:21.967 INFO:teuthology.orchestra.run.smithi026.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-19T16:27:22.037 INFO:teuthology.orchestra.run.smithi026.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-19T16:27:22.043 INFO:teuthology.orchestra.run.smithi026.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-19T16:27:22.200 INFO:teuthology.orchestra.run.smithi026.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-19T16:27:22.200 INFO:teuthology.orchestra.run.smithi026.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-12-19T16:27:22.200 INFO:teuthology.orchestra.run.smithi026.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-19T16:27:22.251 INFO:teuthology.orchestra.run.smithi026.stderr:done 2023-12-19T16:27:22.446 DEBUG:teuthology.orchestra.run.smithi026:> dpkg-deb -c /tmp/linux-image.deb 2023-12-19T16:27:22.454 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./ 2023-12-19T16:27:22.454 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./boot/ 2023-12-19T16:27:22.454 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 4415617 2023-12-08 17:49 ./boot/System.map-6.6.0-gcd68252d7b4d 2023-12-19T16:27:22.467 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 169214 2023-12-08 17:49 ./boot/config-6.6.0-gcd68252d7b4d 2023-12-19T16:27:22.468 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11352864 2023-12-08 17:49 ./boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-19T16:27:22.486 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./etc/ 2023-12-19T16:27:22.487 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./etc/kernel/ 2023-12-19T16:27:22.487 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./etc/kernel/postinst.d/ 2023-12-19T16:27:22.487 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./etc/kernel/postrm.d/ 2023-12-19T16:27:22.487 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./etc/kernel/preinst.d/ 2023-12-19T16:27:22.487 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./etc/kernel/prerm.d/ 2023-12-19T16:27:22.487 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/ 2023-12-19T16:27:22.487 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/ 2023-12-19T16:27:22.487 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/ 2023-12-19T16:27:22.487 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/ 2023-12-19T16:27:22.487 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/ 2023-12-19T16:27:22.487 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/ 2023-12-19T16:27:22.487 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ 2023-12-19T16:27:22.488 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 490233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/aesni-intel.ko 2023-12-19T16:27:22.489 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-19T16:27:22.489 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 93113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 106729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 152241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 83889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 40529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-19T16:27:22.490 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 58585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-19T16:27:22.491 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 50497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-19T16:27:22.491 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-19T16:27:22.491 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-19T16:27:22.491 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/ 2023-12-19T16:27:22.491 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/ 2023-12-19T16:27:22.491 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/ 2023-12-19T16:27:22.491 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 40113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-19T16:27:22.491 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpuid.ko 2023-12-19T16:27:22.491 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/msr.ko 2023-12-19T16:27:22.491 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/ 2023-12-19T16:27:22.491 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 354433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-19T16:27:22.492 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 881625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-19T16:27:22.492 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 2275865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm.ko 2023-12-19T16:27:22.504 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ 2023-12-19T16:27:22.504 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 53697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/af_alg.ko 2023-12-19T16:27:22.504 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/algif_hash.ko 2023-12-19T16:27:22.504 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/algif_skcipher.ko 2023-12-19T16:27:22.504 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ansi_cprng.ko 2023-12-19T16:27:22.504 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/anubis.ko 2023-12-19T16:27:22.504 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/arc4.ko 2023-12-19T16:27:22.504 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/asymmetric_keys/ 2023-12-19T16:27:22.504 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-12-19T16:27:22.504 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/ 2023-12-19T16:27:22.504 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_memcpy.ko 2023-12-19T16:27:22.504 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_pq.ko 2023-12-19T16:27:22.505 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-19T16:27:22.505 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_tx.ko 2023-12-19T16:27:22.505 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_xor.ko 2023-12-19T16:27:22.505 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/raid6test.ko 2023-12-19T16:27:22.505 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authenc.ko 2023-12-19T16:27:22.505 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authencesn.ko 2023-12-19T16:27:22.505 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blake2b_generic.ko 2023-12-19T16:27:22.505 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_common.ko 2023-12-19T16:27:22.505 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_generic.ko 2023-12-19T16:27:22.505 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/camellia_generic.ko 2023-12-19T16:27:22.505 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast5_generic.ko 2023-12-19T16:27:22.505 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast6_generic.ko 2023-12-19T16:27:22.505 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast_common.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ccm.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cmac.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crc32_generic.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cryptd.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_engine.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_simd.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_user.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cts.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/des_generic.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/dh_generic.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/echainiv.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/essiv.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/fcrypt.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/khazad.ko 2023-12-19T16:27:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lrw.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4hc.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/md4.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/michael_mic.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcbc.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcrypt.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/rmd160.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/seed.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/serpent_generic.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 149193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tcrypt.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tea.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_common.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_generic.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/vmac.ko 2023-12-19T16:27:22.507 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/wp512.ko 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xcbc.ko 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xor.ko 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xts.ko 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xxhash_generic.ko 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_extlog.ko 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_pad.ko 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/apei/ 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/apei/einj.ko 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ec_sys.ko 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/platform_profile.ko 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbs.ko 2023-12-19T16:27:22.508 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbshc.ko 2023-12-19T16:27:22.509 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 103025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/video.ko 2023-12-19T16:27:22.509 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ 2023-12-19T16:27:22.509 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/acard-ahci.ko 2023-12-19T16:27:22.509 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 88865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci.ko 2023-12-19T16:27:22.509 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci_platform.ko 2023-12-19T16:27:22.509 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 88289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci.ko 2023-12-19T16:27:22.509 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci_platform.ko 2023-12-19T16:27:22.509 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_acpi.ko 2023-12-19T16:27:22.509 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ali.ko 2023-12-19T16:27:22.509 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_amd.ko 2023-12-19T16:27:22.509 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_artop.ko 2023-12-19T16:27:22.509 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atiixp.ko 2023-12-19T16:27:22.509 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atp867x.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd640.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd64x.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cypress.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_efar.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt366.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt37x.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it8213.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it821x.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_jmicron.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_legacy.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_marvell.ko 2023-12-19T16:27:22.510 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_mpiix.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_netcell.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ninja32.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87410.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87415.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_oldpiix.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_opti.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_optidma.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pcmcia.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_piccolo.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_radisys.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rdc.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rz1000.ko 2023-12-19T16:27:22.511 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sch.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_serverworks.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sil680.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sl82c105.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_triflex.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_via.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pdc_adma.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_inic162x.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 83345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_mv.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 70393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_nv.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_promise.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_qstor.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil24.ko 2023-12-19T16:27:22.512 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sis.ko 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_svw.ko 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 44793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sx4.ko 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_uli.ko 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_via.ko 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_vsc.ko 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ks0108.ko 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/ 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 124473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/bcma.ko 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/ 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/ 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 113665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/aoe.ko 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/ 2023-12-19T16:27:22.513 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 965585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/drbd.ko 2023-12-19T16:27:22.540 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 173225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/floppy.ko 2023-12-19T16:27:22.540 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/ 2023-12-19T16:27:22.540 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 94297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-19T16:27:22.540 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 115793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/nbd.ko 2023-12-19T16:27:22.540 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/ 2023-12-19T16:27:22.540 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 110385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/null_blk.ko 2023-12-19T16:27:22.541 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 98193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/pktcdvd.ko 2023-12-19T16:27:22.541 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 263769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/rbd.ko 2023-12-19T16:27:22.541 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/ 2023-12-19T16:27:22.541 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 98145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-19T16:27:22.541 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/ 2023-12-19T16:27:22.541 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 60217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/zram.ko 2023-12-19T16:27:22.541 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ 2023-12-19T16:27:22.541 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/applicom.ko 2023-12-19T16:27:22.541 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hangcheck-timer.ko 2023-12-19T16:27:22.541 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/ 2023-12-19T16:27:22.541 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-19T16:27:22.541 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-19T16:27:22.541 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/via-rng.ko 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 148601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 139313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 70921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 53513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/lp.ko 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/ 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 84097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/mwave.ko 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/nvram.ko 2023-12-19T16:27:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ppdev.ko 2023-12-19T16:27:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tlclk.ko 2023-12-19T16:27:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/ 2023-12-19T16:27:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-19T16:27:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-12-19T16:27:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-12-19T16:27:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-12-19T16:27:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-19T16:27:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-19T16:27:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-12-19T16:27:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/xillybus/ 2023-12-19T16:27:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/xillybus/xillybus_class.ko 2023-12-19T16:27:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 50945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/xillybus/xillybus_core.ko 2023-12-19T16:27:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/clk/ 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/clk/clk-palmas.ko 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/clk/clk-si5351.ko 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/clk/clk-twl6040.ko 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/clk/clk-wm831x.ko 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/ 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 70801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 243025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp.ko 2023-12-19T16:27:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/ 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/ 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/ 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 438761 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-aes.ko 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-sha.ko 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/ 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 70273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/ 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/dca.ko 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/ 2023-12-19T16:27:22.545 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 122569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 79953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/amd64_edac.ko 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/e752x_edac.ko 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 63617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/edac_mce_amd.ko 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3000_edac.ko 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3200_edac.ko 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5100_edac.ko 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5400_edac.ko 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7300_edac.ko 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 52041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7core_edac.ko 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i82975x_edac.ko 2023-12-19T16:27:22.546 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ie31200_edac.ko 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 56825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/sb_edac.ko 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/x38_edac.ko 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/ 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-gpio.ko 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-max14577.ko 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-max77693.ko 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-max8997.ko 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-palmas.ko 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-rt8973a.ko 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/extcon/extcon-sm5502.ko 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/ 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/dmi-sysfs.ko 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/efi/ 2023-12-19T16:27:22.547 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/efi/efi-pstore.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/ 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-adp5520.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-amd8111.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-arizona.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-da9052.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-da9055.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-dln2.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 77849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-f7188x.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-generic.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ich.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-kempld.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-lp3943.ko 2023-12-19T16:27:22.548 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-max7300.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-max730x.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-max732x.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-pca953x.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-pcf857x.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-rdc321x.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-sch.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-sch311x.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-tps65912.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-twl4030.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-twl6040.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-vx855.ko 2023-12-19T16:27:22.549 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-wm831x.ko 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-wm8350.ko 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-wm8994.ko 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/ 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-a4tech.ko 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-apple.ko 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-appleir.ko 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-aureal.ko 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-axff.ko 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-belkin.ko 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-cherry.ko 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-chicony.ko 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 60649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-cp2112.ko 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-cypress.ko 2023-12-19T16:27:22.550 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-dr.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elecom.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elo.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-emsff.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ezkey.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gaff.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-generic.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gt683r.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gyration.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtekff.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-hyperv.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-icade.ko 2023-12-19T16:27:22.551 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kensington.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-keytouch.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kye.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lcpower.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-led.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 42297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lenovo.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lg-g15.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 58185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 101481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 80993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-magicmouse.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-microsoft.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-monterey.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 50161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-multitouch.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ntrig.ko 2023-12-19T16:27:22.552 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ortek.ko 2023-12-19T16:27:22.553 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-penmount.ko 2023-12-19T16:27:22.553 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-petalynx.ko 2023-12-19T16:27:22.553 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 72937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-picolcd.ko 2023-12-19T16:27:22.553 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-pl.ko 2023-12-19T16:27:22.553 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-plantronics.ko 2023-12-19T16:27:22.553 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-primax.ko 2023-12-19T16:27:22.553 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-rmi.ko 2023-12-19T16:27:22.553 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-19T16:27:22.553 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-common.ko 2023-12-19T16:27:22.553 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-19T16:27:22.553 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-19T16:27:22.553 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-19T16:27:22.553 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-saitek.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-samsung.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sjoy.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sony.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-speedlink.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-steelseries.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sunplus.ko 2023-12-19T16:27:22.554 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tivo.ko 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tmff.ko 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-topseed.ko 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-twinhan.ko 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 71073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-uclogic.ko 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-waltop.ko 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 143473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-wiimote.ko 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-xinmo.ko 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zpff.ko 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zydacron.ko 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 267825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid.ko 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/uhid.ko 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/usbhid/ 2023-12-19T16:27:22.555 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 125721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/usbhid/usbhid.ko 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/usbhid/usbkbd.ko 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/usbhid/usbmouse.ko 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 217873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/wacom.ko 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/ 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 64201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_balloon.ko 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 89625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_utils.ko 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 305465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_vmbus.ko 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 66521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru.ko 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 61393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru3.ko 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7414.ko 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7418.ko 2023-12-19T16:27:22.556 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adc128d818.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1025.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 97977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1026.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1029.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 45553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1031.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm9240.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ads7828.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7410.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7411.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 74433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7462.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7470.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 67929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7475.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7x10.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/amc6821.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 55201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/applesmc.ko 2023-12-19T16:27:22.557 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 49913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asb100.ko 2023-12-19T16:27:22.558 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 58617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asc7621.ko 2023-12-19T16:27:22.558 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 40825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-19T16:27:22.558 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/atxp1.ko 2023-12-19T16:27:22.558 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/coretemp.ko 2023-12-19T16:27:22.558 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/da9052-hwmon.ko 2023-12-19T16:27:22.558 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/da9055-hwmon.ko 2023-12-19T16:27:22.558 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 44465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-19T16:27:22.558 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 99601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dme1737.ko 2023-12-19T16:27:22.558 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds1621.ko 2023-12-19T16:27:22.558 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds620.ko 2023-12-19T16:27:22.558 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc1403.ko 2023-12-19T16:27:22.558 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc2103.ko 2023-12-19T16:27:22.558 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc6w201.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 66985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71805f.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 94457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71882fg.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 41401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f75375s.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fam15h_power.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 55449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fschmd.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g760a.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g762.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 46801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl518sm.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 45529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl520sm.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hih6130.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5500_temp.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5k_amb.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmaem.ko 2023-12-19T16:27:22.559 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmpex.ko 2023-12-19T16:27:22.560 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina209.ko 2023-12-19T16:27:22.560 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina2xx.ko 2023-12-19T16:27:22.560 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 130449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/it87.ko 2023-12-19T16:27:22.560 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/jc42.ko 2023-12-19T16:27:22.560 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k10temp.ko 2023-12-19T16:27:22.560 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k8temp.ko 2023-12-19T16:27:22.560 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lineage-pem.ko 2023-12-19T16:27:22.560 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 48601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm63.ko 2023-12-19T16:27:22.560 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm73.ko 2023-12-19T16:27:22.560 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm75.ko 2023-12-19T16:27:22.560 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm77.ko 2023-12-19T16:27:22.560 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm78.ko 2023-12-19T16:27:22.560 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm80.ko 2023-12-19T16:27:22.561 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm83.ko 2023-12-19T16:27:22.561 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 72833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm85.ko 2023-12-19T16:27:22.561 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 46153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm87.ko 2023-12-19T16:27:22.561 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm90.ko 2023-12-19T16:27:22.561 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm92.ko 2023-12-19T16:27:22.561 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 108841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm93.ko 2023-12-19T16:27:22.561 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95234.ko 2023-12-19T16:27:22.561 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95241.ko 2023-12-19T16:27:22.561 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95245.ko 2023-12-19T16:27:22.561 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc2945.ko 2023-12-19T16:27:22.561 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4151.ko 2023-12-19T16:27:22.561 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4215.ko 2023-12-19T16:27:22.561 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4222.ko 2023-12-19T16:27:22.562 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4245.ko 2023-12-19T16:27:22.562 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4260.ko 2023-12-19T16:27:22.562 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4261.ko 2023-12-19T16:27:22.562 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max16065.ko 2023-12-19T16:27:22.562 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1619.ko 2023-12-19T16:27:22.562 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1668.ko 2023-12-19T16:27:22.562 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max197.ko 2023-12-19T16:27:22.562 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6639.ko 2023-12-19T16:27:22.562 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6650.ko 2023-12-19T16:27:22.562 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6697.ko 2023-12-19T16:27:22.562 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mc13783-adc.ko 2023-12-19T16:27:22.562 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mcp3021.ko 2023-12-19T16:27:22.562 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 44129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6683.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 119817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775-core.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 60705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 59737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct7802.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 82337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87360.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 54433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87427.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pcf8591.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-19T16:27:22.563 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 82577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/powr1220.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5627.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5636.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht15.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht21.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/shtc1.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 40105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sis5595.ko 2023-12-19T16:27:22.564 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47b397.ko 2023-12-19T16:27:22.565 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m1.ko 2023-12-19T16:27:22.565 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m192.ko 2023-12-19T16:27:22.565 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/thmc50.ko 2023-12-19T16:27:22.565 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp102.ko 2023-12-19T16:27:22.565 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp103.ko 2023-12-19T16:27:22.565 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp401.ko 2023-12-19T16:27:22.565 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp421.ko 2023-12-19T16:27:22.565 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via-cputemp.ko 2023-12-19T16:27:22.565 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 41945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via686a.ko 2023-12-19T16:27:22.565 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 52457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt1211.ko 2023-12-19T16:27:22.565 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt8231.ko 2023-12-19T16:27:22.565 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 57449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627ehf.ko 2023-12-19T16:27:22.565 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 78873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627hf.ko 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 71345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83781d.ko 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 63225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83791d.ko 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 71321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83792d.ko 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 101473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83793.ko 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 119505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83795.ko 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l785ts.ko 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l786ng.ko 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/ 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/ 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/ 2023-12-19T16:27:22.566 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-12-19T16:27:22.567 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-12-19T16:27:22.567 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-12-19T16:27:22.567 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-19T16:27:22.567 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-19T16:27:22.567 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-19T16:27:22.567 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-12-19T16:27:22.567 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 6929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-19T16:27:22.567 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-12-19T16:27:22.567 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 50201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-19T16:27:22.567 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-19T16:27:22.567 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 62817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-12-19T16:27:22.568 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-mux.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-smbus.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-stub.ko 2023-12-19T16:27:22.569 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/muxes/ 2023-12-19T16:27:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-12-19T16:27:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-12-19T16:27:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-12-19T16:27:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ 2023-12-19T16:27:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ 2023-12-19T16:27:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 273609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-19T16:27:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 879969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_core.ko 2023-12-19T16:27:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 69273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-19T16:27:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 324217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-19T16:27:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 97985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-19T16:27:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 282273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-19T16:27:22.570 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 66201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-19T16:27:22.571 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/ 2023-12-19T16:27:22.571 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/ 2023-12-19T16:27:22.571 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 493073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-19T16:27:22.571 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/ 2023-12-19T16:27:22.571 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 419737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-19T16:27:22.595 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/ 2023-12-19T16:27:22.595 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 739025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-19T16:27:22.595 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ 2023-12-19T16:27:22.595 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 296625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-19T16:27:22.595 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/ocrdma/ 2023-12-19T16:27:22.595 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 153697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-12-19T16:27:22.595 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/usnic/ 2023-12-19T16:27:22.595 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 168969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-12-19T16:27:22.595 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ 2023-12-19T16:27:22.595 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ 2023-12-19T16:27:22.595 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 263081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-19T16:27:22.595 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 100241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 111425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 128753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 129529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/ 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/evbug.ko 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/ff-memless.ko 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/ 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-19T16:27:22.596 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 41241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/gameport.ko 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 41801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/joydev.ko 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/ 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24761 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/lkkbd.ko 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/lm8323.ko 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/lm8333.ko 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-12-19T16:27:22.597 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/newtonkbd.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/qt1070.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/qt2160.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/stowaway.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/sunkbd.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/xtkbd.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/matrix-keymap.ko 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/ 2023-12-19T16:27:22.598 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/appletouch.ko 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/bcm5974.ko 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 123681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/cyapatp.ko 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 89449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/gpio_mouse.ko 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 335929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/psmouse.ko 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/sermouse.ko 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/ 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 109873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/ 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/altera_ps2.ko 2023-12-19T16:27:22.599 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/arc_ps2.ko 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/ct82c710.ko 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/parkbd.ko 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/pcips2.ko 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/ps2mult.ko 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serio_raw.ko 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serport.ko 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/sparse-keymap.ko 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/ 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/ 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/ 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-88pm860x.ko 2023-12-19T16:27:22.600 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-adp5520.ko 2023-12-19T16:27:22.601 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 59753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-bd2802.ko 2023-12-19T16:27:22.601 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-blinkm.ko 2023-12-19T16:27:22.601 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-da903x.ko 2023-12-19T16:27:22.601 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-da9052.ko 2023-12-19T16:27:22.601 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-gpio.ko 2023-12-19T16:27:22.601 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3530.ko 2023-12-19T16:27:22.601 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3533.ko 2023-12-19T16:27:22.601 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm355x.ko 2023-12-19T16:27:22.601 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3642.ko 2023-12-19T16:27:22.601 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp3944.ko 2023-12-19T16:27:22.601 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp8788.ko 2023-12-19T16:27:22.601 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-max8997.ko 2023-12-19T16:27:22.601 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-mc13783.ko 2023-12-19T16:27:22.604 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-menf21bmc.ko 2023-12-19T16:27:22.604 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-pca9532.ko 2023-12-19T16:27:22.604 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-pca955x.ko 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-pca963x.ko 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-ss4200.ko 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-tca6507.ko 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-wm831x-status.ko 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-wm8350.ko 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-19T16:27:22.605 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/ 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/ 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 654969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/bcache.ko 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 40577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bio-prison.ko 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 89601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bufio.ko 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache-smq.ko 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 156009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache.ko 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 106897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-crypt.ko 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-delay.ko 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 53857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-era.ko 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-flakey.ko 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 41529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log-userspace.ko 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log.ko 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 49545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-mirror.ko 2023-12-19T16:27:22.606 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 78033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-multipath.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-queue-length.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 70825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-raid.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-region-hash.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-round-robin.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-service-time.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 109529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-snapshot.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-switch.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 181865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-thin-pool.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-verity.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-zero.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/faulty.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/linear.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/multipath.ko 2023-12-19T16:27:22.607 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/ 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 199937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid0.ko 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 113073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid1.ko 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 158681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid10.ko 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 395721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid456.ko 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/ 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/ 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 178137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptbase.ko 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 63385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptctl.ko 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 44665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptfc.ko 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptlan.ko 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 123321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptsas.ko 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 79041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptscsih.ko 2023-12-19T16:27:22.608 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 53017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptspi.ko 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/ 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/88pm800.ko 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12761 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/88pm805.ko 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/88pm80x.ko 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/arizona-i2c.ko 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 123721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/arizona.ko 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/bcm590xx.ko 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/cros_ec_dev.ko 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/dln2.ko 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/janz-cmodio.ko 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 53585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/kempld-core.ko 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lm3533-core.ko 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-12-19T16:27:22.609 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lp3943.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 54569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_ich.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_sch.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/max8907.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mc13xxx-core.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/menf21bmc.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/pcf50633-adc.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/pcf50633-gpio.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/pcf50633.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/retu-mfd.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 80425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/si476x-core.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 48457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/sm501.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-12-19T16:27:22.610 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/tps65010.ko 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/tps6507x.ko 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/viperboard.ko 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/vx855.ko 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wl1273-core.ko 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 44345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wm8994.ko 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 48489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ad525x_dpot.ko 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/ 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 46585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds9802als.ko 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds990x.ko 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 45593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/bh1770glc.ko 2023-12-19T16:27:22.611 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/c2port/ 2023-12-19T16:27:22.612 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-12-19T16:27:22.612 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/c2port/core.ko 2023-12-19T16:27:22.612 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/ 2023-12-19T16:27:22.612 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/cb710.ko 2023-12-19T16:27:22.612 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ds1682.ko 2023-12-19T16:27:22.612 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/dummy-irq.ko 2023-12-19T16:27:22.612 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/echo/ 2023-12-19T16:27:22.612 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/echo/echo.ko 2023-12-19T16:27:22.612 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/ 2023-12-19T16:27:22.612 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/at24.ko 2023-12-19T16:27:22.612 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-19T16:27:22.612 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-19T16:27:22.612 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/max6875.ko 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/enclosure.ko 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/genwqe/ 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 152201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hmc6352.ko 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hpilo.ko 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ibmasm/ 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 72129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ics932s401.ko 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29003.ko 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29020.ko 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/ 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 42513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/ 2023-12-19T16:27:22.613 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 125209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-me.ko 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 56521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-txe.ko 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 310113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei.ko 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/phantom.ko 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ti-st/ 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 74265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/ti-st/st_drv.ko 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_7xx1.ko 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_core.ko 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tsl2550.ko 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 48697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_balloon.ko 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/ 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 183609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/ 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 490129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/bonding.ko 2023-12-19T16:27:22.614 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/ 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/ 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 106033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 40417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/ 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 453737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dummy.ko 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/eql.ko 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/ 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 45785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 42105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 130553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-19T16:27:22.615 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 68849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 59689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 99049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/ 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 54521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/ 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 69137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/ 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 59585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/ 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 41409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/ 2023-12-19T16:27:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 54609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-19T16:27:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 49289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-19T16:27:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 87297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-19T16:27:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/ 2023-12-19T16:27:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/ 2023-12-19T16:27:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 97337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-19T16:27:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-12-19T16:27:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 104673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-19T16:27:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-12-19T16:27:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 90257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-19T16:27:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/ 2023-12-19T16:27:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 81569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-19T16:27:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 69729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-19T16:27:22.617 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/ 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 78401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 176761 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 1421497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 128569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/ 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 132089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 312081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/ 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/ 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 323513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/ 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-12-19T16:27:22.618 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 193497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 347417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 1044497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 166193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/ 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/ 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 204345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/ 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/ 2023-12-19T16:27:22.619 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 69449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 61489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 141225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 62817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/ 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 60873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 62833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 45721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dnet.ko 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ec_bhf.ko 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/ 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/ 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 281137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-19T16:27:22.620 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 49457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ethoc.ko 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fealnx.ko 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fujitsu/ 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 45345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 90465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/ 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 295577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/ 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 584497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/ 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 246209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/ 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 925825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-19T16:27:22.621 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/ 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 302937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/ 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 514641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/ 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 93593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 734305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 151769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 86921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/jme.ko 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/ 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 102289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-19T16:27:22.622 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 122649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-19T16:27:22.623 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/ 2023-12-19T16:27:22.623 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-12-19T16:27:22.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 727601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-19T16:27:22.623 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 278937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-19T16:27:22.650 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-12-19T16:27:22.651 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-12-19T16:27:22.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 1113681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-19T16:27:22.651 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ 2023-12-19T16:27:22.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 53041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-12-19T16:27:22.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 40993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-12-19T16:27:22.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-12-19T16:27:22.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 95665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-19T16:27:22.651 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/ 2023-12-19T16:27:22.651 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-12-19T16:27:22.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 108649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-19T16:27:22.651 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ 2023-12-19T16:27:22.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 73441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-19T16:27:22.651 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/ 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 150905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/ 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 136721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/ 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 50481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 43145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/ 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 227897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 87081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 672049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-19T16:27:22.652 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/ 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 43017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/ 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 67385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 83985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 189801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/samsung/ 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 104241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/ 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 1052497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/ 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-19T16:27:22.653 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/ 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 54817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 63337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/ 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 61537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 67193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 73409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 68177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/ 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 487025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-19T16:27:22.654 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/ 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 103337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 166097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 66329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 78801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/ 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 50057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/ 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 86529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/ 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 80273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 83633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/ 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 46241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-19T16:27:22.655 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/ 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 59993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/fddi/ 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 41001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/fddi/defxx.ko 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/fddi/skfp/ 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 216985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/fddi/skfp/skfp.ko 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 77345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/geneve.ko 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/ 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 235521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/ 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ifb.ko 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ 2023-12-19T16:27:22.656 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 83769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-19T16:27:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 62145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvlan.ko 2023-12-19T16:27:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvtap.ko 2023-12-19T16:27:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/ 2023-12-19T16:27:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-19T16:27:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-19T16:27:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-gpio.ko 2023-12-19T16:27:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-19T16:27:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio.ko 2023-12-19T16:27:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mii.ko 2023-12-19T16:27:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 43953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/netconsole.ko 2023-12-19T16:27:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/nlmon.ko 2023-12-19T16:27:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/ 2023-12-19T16:27:22.657 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/amd.ko 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 50033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 44721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm87xx.ko 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 42241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/broadcom.ko 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/cicada.ko 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/davicom.ko 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/et1011c.ko 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/icplus.ko 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/lxt.ko 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 82049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/marvell.ko 2023-12-19T16:27:22.658 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 76113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/micrel.ko 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/national.ko 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 97857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/phylink.ko 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/qsemi.ko 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 50065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/realtek.ko 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/smsc.ko 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ste10Xp.ko 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/vitesse.ko 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/plip/ 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 43505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/plip/plip.ko 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_async.ko 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-19T16:27:22.659 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 42545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppoe.ko 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppox.ko 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pptp.ko 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sb1000.ko 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/ 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 42465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slip.ko 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sungem_phy.ko 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 64449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tap.ko 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/ 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 106169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team.ko 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-19T16:27:22.660 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_random.ko 2023-12-19T16:27:22.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-19T16:27:22.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 87017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/veth.ko 2023-12-19T16:27:22.661 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/ 2023-12-19T16:27:22.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 144297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-19T16:27:22.661 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/ 2023-12-19T16:27:22.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 297329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/vxlan.ko 2023-12-19T16:27:22.661 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/ 2023-12-19T16:27:22.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 144585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-19T16:27:22.661 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ 2023-12-19T16:27:22.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ntb.ko 2023-12-19T16:27:22.661 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/ 2023-12-19T16:27:22.661 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/ 2023-12-19T16:27:22.661 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 265121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme-core.ko 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 102209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme.ko 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/ 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 133841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport.ko 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_cs.ko 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 84465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_pc.ko 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_serial.ko 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/ 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/ 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pci-stub.ko 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 46465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/xen-pcifront.ko 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/ 2023-12-19T16:27:22.662 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/i82092.ko 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 145809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pcmcia.ko 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 62137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pcmcia_core.ko 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pd6729.ko 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 112889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/phy/ 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/phy/broadcom/ 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/phy/samsung/ 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/ 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/ 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 70153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/ 2023-12-19T16:27:22.663 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/ 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 50889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros_ec.ko 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/ 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 67097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acerhdf.ko 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 59665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-19T16:27:22.664 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 127441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 49121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 40265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/hdaps.ko 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/ibm_rtl.ko 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/ 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 43497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel_ips.ko 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-19T16:27:22.665 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 42329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 57569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi.ko 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/ 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/ 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/88pm860x_battery.ko 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/88pm860x_charger.ko 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 55057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/bq2415x_charger.ko 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 54081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/bq24190_charger.ko 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/bq24735-charger.ko 2023-12-19T16:27:22.666 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/da9030_battery.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/da9052-battery.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/ds2760_battery.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/ds2780_battery.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/ds2781_battery.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/ds2782_battery.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/gpio-charger.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/isp1704_charger.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/lp8727_charger.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/max14577_charger.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/max17040_battery.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/max17042_battery.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/max8903_charger.ko 2023-12-19T16:27:22.667 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/max8925_power.ko 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/pcf50633-charger.ko 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/sbs-battery.ko 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/test_power.ko 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/tps65090-charger.ko 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/wm831x_backup.ko 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/wm831x_power.ko 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/power/supply/wm8350_power.ko 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/ 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 57729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/ 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/ 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps_parport.ko 2023-12-19T16:27:22.668 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 41873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/pps_core.ko 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 66385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp.ko 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp_kvm.ko 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/ 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-88pm80x.ko 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-88pm860x.ko 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-da9052.ko 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-da9055.ko 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-da9063.ko 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 68577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-19T16:27:22.669 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-em3027.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-lp8788.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t86.ko 2023-12-19T16:27:22.670 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max6900.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max8907.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max8925.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max8997.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max8998.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-palmas.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 46633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf50633.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rc5t583.ko 2023-12-19T16:27:22.671 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-s35390a.ko 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-tps6586x.ko 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-tps65910.ko 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-wm831x.ko 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-wm8350.ko 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-x1205.ko 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 77617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-9xxx.ko 2023-12-19T16:27:22.672 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 57665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-sas.ko 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 55105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-xxxx.ko 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 135185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/BusLogic.ko 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/a100u2w.ko 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/ 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 272305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 100617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/advansys.ko 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/ 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 356201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 295801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic94xx/ 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 189185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/am53c974.ko 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/ 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 106881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-19T16:27:22.673 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/atp870u.ko 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/ 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 246985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/ 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 1171945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/bfa.ko 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/ 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 237273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/ 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 115369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 40585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ch.ko 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/ 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 308921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/ 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-12-19T16:27:22.674 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 76145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 101449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 146689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 75553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dc395x.ko 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dmx3191d.ko 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/ 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 212425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 60505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esp_scsi.ko 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/ 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 78209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 158505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fdomain.ko 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/ 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 263489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/fnic.ko 2023-12-19T16:27:22.675 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 201817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hpsa.ko 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hptiop.ko 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 49041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hv_storvsc.ko 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/imm.ko 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 44057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/initio.ko 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 266537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ipr.ko 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 101881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ips.ko 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/ 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 317153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/isci.ko 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 52961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/ 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 314865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/libfc.ko 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 145585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi.ko 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 53521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-19T16:27:22.676 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/ 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 216665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/libsas.ko 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/ 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 2197249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/ 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 73569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 310617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 84721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid.ko 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/ 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 659825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/ 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 119561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 78129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvumi.ko 2023-12-19T16:27:22.677 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pcmcia/ 2023-12-19T16:27:22.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 81625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-12-19T16:27:22.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-12-19T16:27:22.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-12-19T16:27:22.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-12-19T16:27:22.678 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/ 2023-12-19T16:27:22.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 364105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-19T16:27:22.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 123617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pmcraid.ko 2023-12-19T16:27:22.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ppa.ko 2023-12-19T16:27:22.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 55361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla1280.ko 2023-12-19T16:27:22.678 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/ 2023-12-19T16:27:22.678 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 1876689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-19T16:27:22.709 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 85289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-19T16:27:22.709 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/ 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 559033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qlogicfas408.ko 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/raid_class.ko 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 238761 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_debug.ko 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 173937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 279313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 97729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 73881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 44993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ses.ko 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 115889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/st.ko 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 46825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/stex.ko 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/ 2023-12-19T16:27:22.710 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 162121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 42297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/virtio_scsi.ko 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 50321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/wd719x.ko 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 50025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/xen-scsifront.ko 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 129441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ssb.ko 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/ 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/ 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 736601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/ 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 49985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_file.ko 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 43025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_iblock.ko 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 930257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_mod.ko 2023-12-19T16:27:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 44305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_pscsi.ko 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 122521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_user.ko 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/ 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 75593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/ 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/ 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/ 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-19T16:27:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 78425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 57865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/moxa.ko 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 50953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/mxser.ko 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 108601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_gsm.ko 2023-12-19T16:27:22.713 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_hdlc.ko 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 67929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/nozomi.ko 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/ 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/ 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 59497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/altera_uart.ko 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/arc_uart.ko 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 72425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/ 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 103073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/rp2.ko 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 52817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/sc16is7xx.ko 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/ 2023-12-19T16:27:22.714 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 531265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/ 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 44513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio.ko 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_aec.ko 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_cif.ko 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_dmem_genirq.ko 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_mf624.ko 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_netx.ko 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pci_generic.ko 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-12-19T16:27:22.715 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_sercos3.ko 2023-12-19T16:27:22.716 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/ 2023-12-19T16:27:22.716 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/c67x00/ 2023-12-19T16:27:22.716 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 73297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/c67x00/c67x00.ko 2023-12-19T16:27:22.716 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/chipidea/ 2023-12-19T16:27:22.716 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 144377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-12-19T16:27:22.716 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-12-19T16:27:22.716 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-12-19T16:27:22.716 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-12-19T16:27:22.716 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/ 2023-12-19T16:27:22.716 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 87089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-acm.ko 2023-12-19T16:27:22.716 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-19T16:27:22.716 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usblp.ko 2023-12-19T16:27:22.716 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 89857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usbtmc.ko 2023-12-19T16:27:22.717 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/common/ 2023-12-19T16:27:22.717 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/common/ulpi.ko 2023-12-19T16:27:22.717 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/dwc2/ 2023-12-19T16:27:22.717 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-12-19T16:27:22.717 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/dwc3/ 2023-12-19T16:27:22.717 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-12-19T16:27:22.717 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-12-19T16:27:22.717 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 448505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/dwc3/dwc3.ko 2023-12-19T16:27:22.717 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/ 2023-12-19T16:27:22.717 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/ 2023-12-19T16:27:22.717 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 60177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/u_ether.ko 2023-12-19T16:27:22.717 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/u_serial.ko 2023-12-19T16:27:22.717 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 42449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 92489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 122081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 57681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 57729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 85609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 58113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 67337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-12-19T16:27:22.718 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/ 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 62185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-12-19T16:27:22.719 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 161225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/libcomposite.ko 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/ 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/bdc/ 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 157681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 52721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 69705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 56033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 76057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/net2272.ko 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 75073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/net2280.ko 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 73169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 65809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 66097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-12-19T16:27:22.720 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 81505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 156073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/ 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/bcma-hcd.ko 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 80113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/isp116x-hcd.ko 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 87353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 78873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811_cs.ko 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/ssb-hcd.ko 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/ 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/mdc800.ko 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/microtek.ko 2023-12-19T16:27:22.721 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 41721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/adutux.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/appledisplay.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/cytherm.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ehset.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi26.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi62.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ezusb.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/idmouse.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/iowarrior.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ldusb.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/legousbtower.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/lvstest.ko 2023-12-19T16:27:22.722 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/ 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 64841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usb3503.ko 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usblcd.ko 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 70809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbtest.ko 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 41905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/uss720.ko 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/yurex.ko 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/mon/ 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 72953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/mon/usbmon.ko 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/musb/ 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 301697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/musb/musb_hdrc.ko 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/ 2023-12-19T16:27:22.723 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-generic.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-isp1301.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-tahvo.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/aircable.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ark3116.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ch341.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 70257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cp210x.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cyberjack.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 58217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 49817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/empeg.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 44273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/f81232.ko 2023-12-19T16:27:22.724 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 175137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 53113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 137081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 124833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_ti.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 67769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipaq.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipw.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ir-usb.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 58233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 107521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mct_u232.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/metro-usb.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 66449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7720.ko 2023-12-19T16:27:22.725 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 78217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7840.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 56337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mxuport.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/navman.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/omninet.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/opticon.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 191137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/option.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/oti6858.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 54353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/pl2303.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcaux.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcserial.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/quatech2.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/safe_serial.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 51625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/sierra.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-19T16:27:22.726 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ssu100.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/symbolserial.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 61489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_debug.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 125609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usbserial.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/visor.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/whiteheat.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/wishbone-serial.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 52913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/uas.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-19T16:27:22.727 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-karma.ko 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 179969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/usb-storage.ko 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/ 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 67481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-19T16:27:22.728 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 72177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 107001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/ 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/ 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 154297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 108737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio.ko 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 77609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/ 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 113785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost.ko 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 58089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_net.ko 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 89249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_scsi.ko 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/ 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/ 2023-12-19T16:27:22.729 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/aat2870_bl.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/adp5520_bl.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/adp8860_bl.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/adp8870_bl.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/apple_bl.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/as3711_bl.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/bd6107.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/da903x_bl.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/da9052_bl.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/gpio_backlight.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/kb3886_bl.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lcd.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lm3533_bl.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lm3639_bl.ko 2023-12-19T16:27:22.730 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lv5207lp.ko 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/max8925_bl.ko 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/pandora_bl.ko 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/wm831x_bl.ko 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/ 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2482.ko 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2490.ko 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/matrox_w1.ko 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/w1-gpio.ko 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/ 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-19T16:27:22.731 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14153 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 56425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 102673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/wire.ko 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/acquirewdt.ko 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/advantechwdt.ko 2023-12-19T16:27:22.732 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-19T16:27:22.733 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-19T16:27:22.733 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/cpu5wdt.ko 2023-12-19T16:27:22.733 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/da9052_wdt.ko 2023-12-19T16:27:22.733 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/da9055_wdt.ko 2023-12-19T16:27:22.733 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/da9063_wdt.ko 2023-12-19T16:27:22.733 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/dw_wdt.ko 2023-12-19T16:27:22.733 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/eurotechwdt.ko 2023-12-19T16:27:22.733 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-19T16:27:22.733 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/hpwdt.ko 2023-12-19T16:27:22.733 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/i6300esb.ko 2023-12-19T16:27:22.734 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-19T16:27:22.734 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-19T16:27:22.734 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ib700wdt.ko 2023-12-19T16:27:22.734 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ibmasr.ko 2023-12-19T16:27:22.734 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-19T16:27:22.734 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-19T16:27:22.734 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it87_wdt.ko 2023-12-19T16:27:22.734 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/kempld_wdt.ko 2023-12-19T16:27:22.734 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/machzwd.ko 2023-12-19T16:27:22.734 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/mena21_wdt.ko 2023-12-19T16:27:22.734 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-12-19T16:27:22.734 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/nv_tco.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22265 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pc87413_wdt.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/retu_wdt.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sc1200wdt.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/softdog.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/twl4030_wdt.ko 2023-12-19T16:27:22.735 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/via_wdt.ko 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wafer5823wdt.ko 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wdt_pci.ko 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wm831x_wdt.ko 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wm8350_wdt.ko 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/xen_wdt.ko 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/ 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-evtchn.ko 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntalloc.ko 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntdev.ko 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/ 2023-12-19T16:27:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 176377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 40121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-privcmd.ko 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 71249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-scsiback.ko 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/ 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/ 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 166425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/9p.ko 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/adfs/ 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 61073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/adfs/adfs.ko 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/ 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 163593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/affs.ko 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/afs/ 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 932817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/afs/kafs.ko 2023-12-19T16:27:22.737 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/autofs/ 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 101057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/autofs/autofs4.ko 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/ 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 64545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/befs.ko 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/bfs/ 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 41257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/bfs/bfs.ko 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/binfmt_misc.ko 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/ 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 3402729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/btrfs.ko 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/ 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 306009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/cachefiles.ko 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 1333017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ceph.ko 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/ 2023-12-19T16:27:22.738 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 100801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/coda.ko 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/configfs/ 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 108393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/configfs/configfs.ko 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/ 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 29625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/cramfs.ko 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/ 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 642513 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/dlm.ko 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/efs/ 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/efs/efs.ko 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/ 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 1505025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/f2fs.ko 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/ 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/msdos.ko 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/freevxfs/ 2023-12-19T16:27:22.739 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 42545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/freevxfs/freevxfs.ko 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/ 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 226481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/fscache.ko 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/ 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/cuse.ko 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/ 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 1231529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/gfs2.ko 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/ 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 141209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/hfs.ko 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/ 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 222817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/hfsplus.ko 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hpfs/ 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 192329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hpfs/hpfs.ko 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/ 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 74065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/isofs.ko 2023-12-19T16:27:22.740 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/ 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 419497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/jfs.ko 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/ 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 280313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/lockd.ko 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/ 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 89193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/minix.ko 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/ 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 124649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/netfs.ko 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/ 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/ 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 81449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/ 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 59449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/ 2023-12-19T16:27:22.741 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 117465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-19T16:27:22.742 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 1055721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfs.ko 2023-12-19T16:27:22.742 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 66441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv2.ko 2023-12-19T16:27:22.742 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 120177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv3.ko 2023-12-19T16:27:22.742 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 2042137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv4.ko 2023-12-19T16:27:22.774 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/ 2023-12-19T16:27:22.774 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/grace.ko 2023-12-19T16:27:22.774 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/nfs_acl.ko 2023-12-19T16:27:22.774 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/ 2023-12-19T16:27:22.774 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 1516873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/nfsd.ko 2023-12-19T16:27:22.774 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/ 2023-12-19T16:27:22.774 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 592833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/nilfs2.ko 2023-12-19T16:27:22.774 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/ 2023-12-19T16:27:22.774 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-celtic.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-centeuro.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-croatian.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-cyrillic.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-gaelic.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-greek.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-iceland.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-inuit.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-roman.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-romanian.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-turkish.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ascii.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1250.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1251.ko 2023-12-19T16:27:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1255.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp737.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp775.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp850.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp852.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp855.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp857.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp860.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp861.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp862.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp863.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp864.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp865.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp866.ko 2023-12-19T16:27:22.776 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp869.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp874.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 94137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp932.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 142673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp936.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 165745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp949.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 118121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp950.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_euc-jp.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-1.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-13.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-14.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-15.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-2.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-3.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-4.ko 2023-12-19T16:27:22.777 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-5.ko 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-6.ko 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-7.ko 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-9.ko 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-r.ko 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-ru.ko 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-u.ko 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_utf8.ko 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ntfs/ 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 219417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ntfs/ntfs.ko 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 244409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-19T16:27:22.778 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 440737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 54049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 2592433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2.ko 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/omfs/ 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 53433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/omfs/omfs.ko 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/ 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 352161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/overlay.ko 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 54809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ramoops.ko 2023-12-19T16:27:22.779 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/qnx4/ 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/qnx4/qnx4.ko 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/qnx6/ 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 46689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/qnx6/qnx6.ko 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/quota/ 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/quota/quota_tree.ko 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/quota/quota_v1.ko 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/quota/quota_v2.ko 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/ 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 532673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/reiserfs.ko 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/ 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/romfs.ko 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/ 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/ 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 2668401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/cifs.ko 2023-12-19T16:27:22.780 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/ 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_arc4.ko 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_md4.ko 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/ 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 134641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/squashfs.ko 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/ 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 90721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/sysv.ko 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/ 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 247737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/udf.ko 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 162497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ufs.ko 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/ 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 4011017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/xfs.ko 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/ 2023-12-19T16:27:22.781 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/locking/ 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 75489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/locking/locktorture.ko 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/time/ 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/time/test_udelay.ko 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 60553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/torture.ko 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 6833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc-itu-t.ko 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 6529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc7.ko 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc8.ko 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/ 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libarc4.ko 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libdes.ko 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/interval_tree_test.ko 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/libcrc32c.ko 2023-12-19T16:27:22.782 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lru_cache.ko 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/ 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4_compress.ko 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4hc_compress.ko 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/ 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 6617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/cordic.ko 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/memory-notifier-error-inject.ko 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/notifier-error-inject.ko 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/percpu_test.ko 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/pm-notifier-error-inject.ko 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/ 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 143433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/raid6_pq.ko 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/rbtree_test.ko 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/ 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-19T16:27:22.783 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/test-kstrtox.ko 2023-12-19T16:27:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/test-string_helpers.ko 2023-12-19T16:27:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 4760425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/test_bpf.ko 2023-12-19T16:27:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 49089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/test_firmware.ko 2023-12-19T16:27:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 6369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/test_module.ko 2023-12-19T16:27:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/test_user_copy.ko 2023-12-19T16:27:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_bm.ko 2023-12-19T16:27:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_fsm.ko 2023-12-19T16:27:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_kmp.ko 2023-12-19T16:27:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/xz/ 2023-12-19T16:27:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/xz/xz_dec_test.ko 2023-12-19T16:27:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/ 2023-12-19T16:27:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/hwpoison-inject.ko 2023-12-19T16:27:22.784 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ 2023-12-19T16:27:22.785 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/ 2023-12-19T16:27:22.785 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 55737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/6lowpan.ko 2023-12-19T16:27:22.785 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_dest.ko 2023-12-19T16:27:22.785 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_fragment.ko 2023-12-19T16:27:22.785 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_hop.ko 2023-12-19T16:27:22.785 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 6985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-19T16:27:22.785 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_mobility.ko 2023-12-19T16:27:22.785 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_routing.ko 2023-12-19T16:27:22.785 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_udp.ko 2023-12-19T16:27:22.785 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/ 2023-12-19T16:27:22.785 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/garp.ko 2023-12-19T16:27:22.785 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/mrp.ko 2023-12-19T16:27:22.785 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/p8022.ko 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/psnap.ko 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/stp.ko 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/ 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 81737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/8021q.ko 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/ 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 198337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet.ko 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_fd.ko 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26345 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_rdma.ko 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_virtio.ko 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/ 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 649297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/batman-adv.ko 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/ 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 63809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/br_netfilter.ko 2023-12-19T16:27:22.786 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 777689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/bridge.ko 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13217 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8385 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-19T16:27:22.787 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 55929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtables.ko 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/ 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 1046681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/libceph.ko 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/ 2023-12-19T16:27:22.788 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 125993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/pktgen.ko 2023-12-19T16:27:22.789 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/ 2023-12-19T16:27:22.789 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 194209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp.ko 2023-12-19T16:27:22.789 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_diag.ko 2023-12-19T16:27:22.789 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 57881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv4.ko 2023-12-19T16:27:22.789 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 60337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv6.ko 2023-12-19T16:27:22.789 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/ 2023-12-19T16:27:22.789 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 390473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/dsa_core.ko 2023-12-19T16:27:22.789 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_brcm.ko 2023-12-19T16:27:22.789 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_dsa.ko 2023-12-19T16:27:22.789 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_none.ko 2023-12-19T16:27:22.789 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_trailer.ko 2023-12-19T16:27:22.789 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/hsr/ 2023-12-19T16:27:22.789 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 99729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/hsr/hsr.ko 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 45233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 254625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154.ko 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ah4.ko 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/esp4.ko 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 48337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/fou.ko 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/gre.ko 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 48657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/inet_diag.ko 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 60633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_gre.ko 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 61225 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_tunnel.ko 2023-12-19T16:27:22.790 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_vti.ko 2023-12-19T16:27:22.791 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipcomp.ko 2023-12-19T16:27:22.791 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipip.ko 2023-12-19T16:27:22.791 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ 2023-12-19T16:27:22.791 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-19T16:27:22.791 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-19T16:27:22.791 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-19T16:27:22.791 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 41609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ip_tables.ko 2023-12-19T16:27:22.791 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-19T16:27:22.791 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-12-19T16:27:22.791 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-19T16:27:22.791 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-19T16:27:22.791 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-19T16:27:22.791 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11305 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-12-19T16:27:22.792 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-19T16:27:22.792 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-19T16:27:22.792 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-19T16:27:22.792 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-19T16:27:22.792 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-19T16:27:22.792 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-19T16:27:22.792 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-19T16:27:22.792 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18161 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-19T16:27:22.793 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-19T16:27:22.793 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-12-19T16:27:22.793 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-19T16:27:22.793 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-19T16:27:22.793 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-19T16:27:22.793 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-19T16:27:22.793 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-19T16:27:22.793 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_bic.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_dctcp.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_diag.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_highspeed.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_htcp.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_hybla.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_illinois.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_lp.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_scalable.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_vegas.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_veno.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_westwood.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_yeah.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tunnel4.ko 2023-12-19T16:27:22.794 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_diag.ko 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 41665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_tunnel.ko 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ah6.ko 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/esp6.ko 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/fou6.ko 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 80897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_gre.ko 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 93577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_tunnel.ko 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 53273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_vti.ko 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ipcomp6.ko 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19017 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/mip6.ko 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 42169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-19T16:27:22.795 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11121 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-19T16:27:22.796 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25945 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9009 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 74073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/sit.ko 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/tunnel6.ko 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/ 2023-12-19T16:27:22.797 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 82833 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/af_key.ko 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/ 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 125281 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_core.ko 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 25953 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_eth.ko 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip.ko 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36713 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip6.ko 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_netlink.ko 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 53241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ppp.ko 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/lapb/ 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 47977 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/lapb/lapb.ko 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/ 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc.ko 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 267913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc2.ko 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/ 2023-12-19T16:27:22.798 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 290417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/mac802154.ko 2023-12-19T16:27:22.799 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/ 2023-12-19T16:27:22.799 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/mpls_gso.ko 2023-12-19T16:27:22.799 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ 2023-12-19T16:27:22.799 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ 2023-12-19T16:27:22.799 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 119585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set.ko 2023-12-19T16:27:22.799 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-19T16:27:22.799 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26913 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-19T16:27:22.799 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-19T16:27:22.799 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 100297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-19T16:27:22.799 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 101441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-19T16:27:22.799 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 102817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-19T16:27:22.799 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 103417 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 112849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 60329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 109593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 113129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 112585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 111585 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 114465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 411729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-19T16:27:22.800 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-19T16:27:22.801 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-19T16:27:22.801 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-19T16:27:22.801 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7689 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-19T16:27:22.801 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10681 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-19T16:27:22.801 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-19T16:27:22.801 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-19T16:27:22.801 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-19T16:27:22.801 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7697 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-19T16:27:22.801 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10993 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-19T16:27:22.801 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 49257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conncount.ko 2023-12-19T16:27:22.801 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 408257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack.ko 2023-12-19T16:27:22.801 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-19T16:27:22.802 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-19T16:27:22.802 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33761 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-19T16:27:22.802 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 118145 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-19T16:27:22.802 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-19T16:27:22.802 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-19T16:27:22.802 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 107233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-19T16:27:22.802 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-19T16:27:22.802 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-19T16:27:22.802 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 66641 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-19T16:27:22.802 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-19T16:27:22.802 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-19T16:27:22.802 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10113 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-19T16:27:22.802 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 89425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9593 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 37129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_log_syslog.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 95353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11857 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_irc.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28929 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_sip.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 44065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 615649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_tables.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 42561 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30233 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30569 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-12-19T16:27:22.803 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 39785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_log.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 49377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10457 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_chain_nat.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_compat.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 40721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_ct.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9601 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fib.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14185 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_hash.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_limit.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_log.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_masq.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_nat.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13673 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_numgen.ko 2023-12-19T16:27:22.804 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11841 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_osf.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_queue.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_redir.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject_inet.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_socket.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_synproxy.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_tproxy.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 89313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/x_tables.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_AUDIT.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 18169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CT.ko 2023-12-19T16:27:22.805 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_DSCP.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HL.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13369 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HMARK.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13889 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LED.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11489 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LOG.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11753 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NETMAP.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFLOG.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_RATEEST.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 14337 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_SECMARK.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-19T16:27:22.806 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-19T16:27:22.807 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17361 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TEE.ko 2023-12-19T16:27:22.807 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TPROXY.ko 2023-12-19T16:27:22.807 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8321 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TRACE.ko 2023-12-19T16:27:22.807 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_addrtype.ko 2023-12-19T16:27:22.807 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_bpf.ko 2023-12-19T16:27:22.807 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cgroup.ko 2023-12-19T16:27:22.807 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cluster.ko 2023-12-19T16:27:22.807 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_comment.ko 2023-12-19T16:27:22.807 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10465 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connbytes.ko 2023-12-19T16:27:22.807 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlabel.ko 2023-12-19T16:27:22.807 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlimit.ko 2023-12-19T16:27:22.807 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13129 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connmark.ko 2023-12-19T16:27:22.807 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_conntrack.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cpu.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dccp.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_devgroup.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8801 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dscp.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ecn.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_esp.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 43289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hashlimit.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_helper.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7865 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hl.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11921 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipcomp.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_iprange.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11785 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipvs.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16041 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_l2tp.ko 2023-12-19T16:27:22.808 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8001 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_length.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10209 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_limit.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7545 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mac.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mark.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11825 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_multiport.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nat.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nfacct.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_osf.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_owner.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_physdev.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_pkttype.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11985 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_policy.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_quota.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_rateest.ko 2023-12-19T16:27:22.809 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_realm.ko 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_recent.ko 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13329 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_sctp.ko 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_set.ko 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 16089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_socket.ko 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9505 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_state.ko 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9473 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_statistic.ko 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8809 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_string.ko 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 8553 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpmss.ko 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15777 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpudp.ko 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_time.ko 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_u32.ko 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/ 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13969 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/netlink_diag.ko 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/ 2023-12-19T16:27:22.810 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 12425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/nsh.ko 2023-12-19T16:27:22.811 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/ 2023-12-19T16:27:22.811 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 380033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/openvswitch.ko 2023-12-19T16:27:22.811 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-geneve.ko 2023-12-19T16:27:22.812 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-gre.ko 2023-12-19T16:27:22.812 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 11793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-vxlan.ko 2023-12-19T16:27:22.812 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/ 2023-12-19T16:27:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13057 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/af_packet_diag.ko 2023-12-19T16:27:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/phonet/ 2023-12-19T16:27:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 110201 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/phonet/phonet.ko 2023-12-19T16:27:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 54721 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/phonet/pn_pep.ko 2023-12-19T16:27:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/ 2023-12-19T16:27:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 251481 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds.ko 2023-12-19T16:27:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 164937 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_rdma.ko 2023-12-19T16:27:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 67377 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_tcp.ko 2023-12-19T16:27:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rxrpc/ 2023-12-19T16:27:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 819657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rxrpc/rxrpc.ko 2023-12-19T16:27:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/ 2023-12-19T16:27:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32241 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_csum.ko 2023-12-19T16:27:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23577 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_gact.ko 2023-12-19T16:27:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26873 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_ipt.ko 2023-12-19T16:27:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_mirred.ko 2023-12-19T16:27:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_nat.ko 2023-12-19T16:27:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 43449 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_pedit.ko 2023-12-19T16:27:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_police.ko 2023-12-19T16:27:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_simple.ko 2023-12-19T16:27:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35297 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_skbedit.ko 2023-12-19T16:27:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_vlan.ko 2023-12-19T16:27:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_basic.ko 2023-12-19T16:27:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33961 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_bpf.ko 2023-12-19T16:27:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19169 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_cgroup.ko 2023-12-19T16:27:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31401 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_flow.ko 2023-12-19T16:27:22.814 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31769 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_fw.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 38425 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_route.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 61409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_u32.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7441 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_cmp.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 13409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_ipset.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33289 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_meta.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_nbyte.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_text.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7249 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_u32.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 21649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_choke.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20849 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_codel.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 34633 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_drr.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 33609 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31617 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq_codel.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 35433 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_gred.ko 2023-12-19T16:27:22.815 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 49817 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hfsc.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22089 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hhf.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 66393 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_htb.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26073 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_ingress.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 30273 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10793 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio_lib.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26065 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_multiq.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 36137 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_netem.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 27497 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_pie.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10625 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_plug.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 24729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_prio.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 48729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_qfq.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 32905 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_red.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 26881 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfb.ko 2023-12-19T16:27:22.816 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 31353 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfq.ko 2023-12-19T16:27:22.817 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28897 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_tbf.ko 2023-12-19T16:27:22.817 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 28105 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_teql.ko 2023-12-19T16:27:22.817 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/ 2023-12-19T16:27:22.817 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 929745 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp.ko 2023-12-19T16:27:22.817 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 19033 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp_diag.ko 2023-12-19T16:27:22.817 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/ 2023-12-19T16:27:22.817 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/ 2023-12-19T16:27:22.817 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 336529 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-19T16:27:22.820 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 54737 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-19T16:27:22.821 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 1520097 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/sunrpc.ko 2023-12-19T16:27:22.821 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/ 2023-12-19T16:27:22.821 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 707657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-19T16:27:22.821 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/ 2023-12-19T16:27:22.821 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10025 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/diag.ko 2023-12-19T16:27:22.821 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 862665 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/tipc.ko 2023-12-19T16:27:22.821 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/ 2023-12-19T16:27:22.821 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15729 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/unix_diag.ko 2023-12-19T16:27:22.821 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/ 2023-12-19T16:27:22.821 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 85409 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-19T16:27:22.821 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 68049 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-19T16:27:22.821 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 96193 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock.ko 2023-12-19T16:27:22.821 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 10705 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 17521 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/x25/ 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 125177 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/x25/x25.ko 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/ 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 22657 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_algo.ko 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 20649 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 102257 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_user.ko 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/ 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/ 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 15537 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/irqbypass.ko 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 640081 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.alias 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 624107 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.alias.bin 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 7663 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.builtin 2023-12-19T16:27:22.822 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 23428 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.alias.bin 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 9821 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.bin 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 70004 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.modinfo 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 146313 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.dep 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 232548 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.dep.bin 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 210 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.devname 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 72606 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.order 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 879 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.softdep 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 254438 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.symbols 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 309363 2023-12-08 17:49 ./lib/modules/6.6.0-gcd68252d7b4d/modules.symbols.bin 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./usr/ 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./usr/share/ 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./usr/share/doc/ 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:drwxr-xr-x root/root 0 2023-12-08 17:49 ./usr/share/doc/linux-image-6.6.0-gcd68252d7b4d/ 2023-12-19T16:27:22.823 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 170 2023-12-08 17:49 ./usr/share/doc/linux-image-6.6.0-gcd68252d7b4d/changelog.Debian.gz 2023-12-19T16:27:22.824 INFO:teuthology.orchestra.run.smithi026.stdout:-rw-r--r-- root/root 691 2023-12-08 17:49 ./usr/share/doc/linux-image-6.6.0-gcd68252d7b4d/copyright 2023-12-19T16:27:22.824 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-gcd68252d7b4d 2023-12-19T16:27:22.824 INFO:teuthology.task.kernel:searching for kernel 6.6.0-gcd68252d7b4d 2023-12-19T16:27:22.825 DEBUG:teuthology.orchestra.run.smithi026:> egrep '(submenu|menuentry.*6.6.0-gcd68252d7b4d).*{' /boot/grub/grub.cfg 2023-12-19T16:27:22.869 INFO:teuthology.orchestra.run.smithi026.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-12-19T16:27:22.869 INFO:teuthology.orchestra.run.smithi026.stdout: menuentry 'Ubuntu, with Linux 6.6.0-gcd68252d7b4d' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-gcd68252d7b4d-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-12-19T16:27:22.869 INFO:teuthology.orchestra.run.smithi026.stdout: menuentry 'Ubuntu, with Linux 6.6.0-gcd68252d7b4d (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-gcd68252d7b4d-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-12-19T16:27:22.870 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-12-19T16:27:22.870 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-gcd68252d7b4d 2023-12-19T16:27:22.870 DEBUG:teuthology.orchestra.run.smithi026:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-12-19T16:27:22.913 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi026.front.sepia.ceph.com to complete... 2023-12-19T16:27:23.318 INFO:teuthology.orchestra.run.smithi026.stderr:Sourcing file `/etc/default/grub' 2023-12-19T16:27:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-19T16:27:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:Generating grub configuration file ... 2023-12-19T16:27:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:Found linux image: /boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-19T16:27:23.657 INFO:teuthology.orchestra.run.smithi026.stderr:Found initrd image: /boot/initrd.img-6.6.0-gcd68252d7b4d 2023-12-19T16:27:23.949 INFO:teuthology.orchestra.run.smithi026.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-19T16:27:23.955 INFO:teuthology.orchestra.run.smithi026.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-19T16:27:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-19T16:27:24.003 INFO:teuthology.orchestra.run.smithi026.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-19T16:27:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-19T16:27:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-12-19T16:27:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-19T16:27:24.142 INFO:teuthology.orchestra.run.smithi026.stderr:done 2023-12-19T16:27:45.457 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-12-19T16:27:45.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-12-19T16:27:45.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T16:27:55.245 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-12-19T16:27:55.245 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-12-19T16:27:55.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T16:28:20.114 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2023-12-19T16:28:29.123 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-12-19T16:28:29.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T16:28:29.841 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2023-12-19T16:28:38.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-12-19T16:28:38.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T16:28:41.909 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2023-12-19T16:28:53.922 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-12-19T16:28:53.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T16:28:56.981 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2023-12-19T16:29:11.993 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-12-19T16:29:11.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T16:29:12.203 DEBUG:teuthology.orchestra.run.smithi026:> true 2023-12-19T16:29:12.841 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-12-19T16:29:12.841 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-gcd68252d7b4d"... 2023-12-19T16:29:12.841 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2023-12-19T16:29:12.892 INFO:teuthology.orchestra.run.smithi026.stdout:6.6.0-gcd68252d7b4d 2023-12-19T16:29:12.892 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-gcd68252d7b4d vs 6.6.0-gcd68252d7b4d 2023-12-19T16:29:12.892 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-19T16:29:12.892 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-12-19T16:29:13.893 DEBUG:teuthology.parallel:result is None 2023-12-19T16:29:29.126 DEBUG:teuthology.orchestra.remote:timed out 2023-12-19T16:29:41.127 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-12-19T16:29:41.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T16:29:41.483 DEBUG:teuthology.orchestra.run.smithi160:> true 2023-12-19T16:29:42.252 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-12-19T16:29:42.252 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-gcd68252d7b4d"... 2023-12-19T16:29:42.252 DEBUG:teuthology.orchestra.run.smithi160:> uname -r 2023-12-19T16:29:42.304 INFO:teuthology.orchestra.run.smithi160.stdout:6.6.0-gcd68252d7b4d 2023-12-19T16:29:42.304 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-gcd68252d7b4d vs 6.6.0-gcd68252d7b4d 2023-12-19T16:29:42.304 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-19T16:29:42.304 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-12-19T16:29:43.305 DEBUG:teuthology.parallel:result is None 2023-12-19T16:29:43.305 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-19T16:29:43.317 INFO:teuthology.task.internal:Creating test directory... 2023-12-19T16:29:43.318 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-19T16:29:43.322 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-19T16:29:43.331 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-19T16:29:43.351 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-19T16:29:43.359 INFO:teuthology.task.internal:Creating archive directory... 2023-12-19T16:29:43.359 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-19T16:29:43.378 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-19T16:29:43.404 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-19T16:29:43.411 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-19T16:29:43.411 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-19T16:29:43.443 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-19T16:29:43.521 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-19T16:29:43.538 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-19T16:29:43.555 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-19T16:29:43.580 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-19T16:29:43.582 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-19T16:29:43.590 INFO:teuthology.task.internal:Configuring sudo... 2023-12-19T16:29:43.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-19T16:29:43.594 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-19T16:29:43.616 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-19T16:29:43.658 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-19T16:29:43.659 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-19T16:29:43.662 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-19T16:29:43.671 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-19T16:29:43.720 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-19T16:29:43.773 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:29:43.773 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-19T16:29:43.840 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-19T16:29:43.849 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-19T16:29:43.905 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:29:43.905 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-19T16:29:43.979 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2023-12-19T16:29:43.983 DEBUG:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2023-12-19T16:29:44.101 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-19T16:29:44.110 INFO:teuthology.task.internal:Starting timer... 2023-12-19T16:29:44.110 INFO:teuthology.run_tasks:Running task pcp... 2023-12-19T16:29:44.127 INFO:teuthology.run_tasks:Running task selinux... 2023-12-19T16:29:44.137 DEBUG:teuthology.task.selinux:Excluding smithi026: OS 'ubuntu' does not support SELinux 2023-12-19T16:29:44.138 DEBUG:teuthology.task.selinux:Excluding smithi160: OS 'ubuntu' does not support SELinux 2023-12-19T16:29:44.138 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-19T16:29:44.138 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-12-19T16:29:44.138 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-19T16:29:44.138 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-19T16:29:44.153 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-12-19T16:29:44.158 INFO:teuthology.repo_utils:Fetching main from origin 2023-12-19T16:29:44.273 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-19T16:29:44.290 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-19T16:29:44.291 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi026.front.sepia.ceph.com,smithi160.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-19T16:38:38.796 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi160.front.sepia.ceph.com')] 2023-12-19T16:38:38.797 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-12-19T16:38:38.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T16:38:38.924 DEBUG:teuthology.orchestra.run.smithi026:> true 2023-12-19T16:38:39.045 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2023-12-19T16:38:39.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-12-19T16:38:39.045 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T16:38:39.121 DEBUG:teuthology.orchestra.run.smithi160:> true 2023-12-19T16:38:39.256 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-12-19T16:38:39.257 INFO:teuthology.run_tasks:Running task clock... 2023-12-19T16:38:39.269 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-19T16:38:39.269 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-19T16:38:39.270 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-19T16:38:39.273 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-19T16:38:39.273 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-19T16:38:39.311 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-12-19T16:38:39.311 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: Command line: ntpd -gq 2023-12-19T16:38:39.311 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: ---------------------------------------------------- 2023-12-19T16:38:39.311 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: ntp-4 is maintained by Network Time Foundation, 2023-12-19T16:38:39.311 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-12-19T16:38:39.311 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: corporation. Support and training for ntp-4 are 2023-12-19T16:38:39.311 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: available at https://www.nwtime.org/support 2023-12-19T16:38:39.312 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: ---------------------------------------------------- 2023-12-19T16:38:39.312 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: proto: precision = 0.043 usec (-24) 2023-12-19T16:38:39.312 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: basedate set to 2022-02-04 2023-12-19T16:38:39.312 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: gps base set to 2022-02-06 (week 2196) 2023-12-19T16:38:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-19T16:38:39.313 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-19T16:38:39.313 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: restrict ::: KOD does nothing without LIMITED. 2023-12-19T16:38:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:restrict ::: KOD does nothing without LIMITED. 2023-12-19T16:38:39.314 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: Listen and drop on 0 v6wildcard [::]:123 2023-12-19T16:38:39.314 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-12-19T16:38:39.315 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: Listen normally on 2 lo 127.0.0.1:123 2023-12-19T16:38:39.315 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: Listen normally on 3 ens1f0 172.21.15.26:123 2023-12-19T16:38:39.315 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: Listen normally on 4 lo [::1]:123 2023-12-19T16:38:39.315 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fea5:cac0%4]:123 2023-12-19T16:38:39.315 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:39 ntpd[27875]: Listening on routing socket on fd #22 for interface updates 2023-12-19T16:38:39.337 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-12-19T16:38:39.337 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: Command line: ntpd -gq 2023-12-19T16:38:39.337 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: ---------------------------------------------------- 2023-12-19T16:38:39.337 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: ntp-4 is maintained by Network Time Foundation, 2023-12-19T16:38:39.337 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-12-19T16:38:39.337 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: corporation. Support and training for ntp-4 are 2023-12-19T16:38:39.337 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: available at https://www.nwtime.org/support 2023-12-19T16:38:39.337 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: ---------------------------------------------------- 2023-12-19T16:38:39.338 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: proto: precision = 0.045 usec (-24) 2023-12-19T16:38:39.338 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: basedate set to 2022-02-04 2023-12-19T16:38:39.338 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: gps base set to 2022-02-06 (week 2196) 2023-12-19T16:38:39.338 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-19T16:38:39.339 INFO:teuthology.orchestra.run.smithi160.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-19T16:38:39.339 INFO:teuthology.orchestra.run.smithi160.stderr:restrict ::: KOD does nothing without LIMITED. 2023-12-19T16:38:39.339 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: restrict ::: KOD does nothing without LIMITED. 2023-12-19T16:38:39.340 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: Listen and drop on 0 v6wildcard [::]:123 2023-12-19T16:38:39.340 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-12-19T16:38:39.340 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: Listen normally on 2 lo 127.0.0.1:123 2023-12-19T16:38:39.340 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: Listen normally on 3 enp3s0f1 172.21.15.160:123 2023-12-19T16:38:39.340 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: Listen normally on 4 lo [::1]:123 2023-12-19T16:38:39.340 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f5b%5]:123 2023-12-19T16:38:39.340 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:39 ntpd[27991]: Listening on routing socket on fd #22 for interface updates 2023-12-19T16:38:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:40 ntpd[27875]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:40.337 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:40 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:40.337 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:40 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:40.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:40 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:40.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:40 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:40.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:40 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:40.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:40 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:40.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:40 ntpd[27991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:40.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:40 ntpd[27991]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:42.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:42 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:42.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:42 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:42.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:42 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:42.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:42 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:42.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:42 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:42.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:42 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:42.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:42 ntpd[27875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:42.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:42 ntpd[27875]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:42.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:42 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:42.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:42 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:42.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:42 ntpd[27875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:42.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:42 ntpd[27875]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:42.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:42 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:42.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:42 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:42.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:42 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:42.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:42 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:42.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:42 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:42.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:42 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:42.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:42 ntpd[27991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:42.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:42 ntpd[27991]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:42.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:42 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:42.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:42 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:42.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:42 ntpd[27991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:42.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:42 ntpd[27991]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:44.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:44.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:44.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:44.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:44.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:44.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:44.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:44.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:44.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:44.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:44.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:44.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:44.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:44.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:44.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:44.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:44.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:44.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:44 ntpd[27875]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:44.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:44.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:44.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:44.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:44.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:44.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:44.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:44.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:44.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:44.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:44.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:44.340 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:44.340 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:44.340 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:44.340 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:44.340 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:44.340 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:44.340 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:44 ntpd[27991]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:46.312 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:46 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:46.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:46 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:46.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:46 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:46.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:46 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:46.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:46 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:46.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:46 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:46.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:46 ntpd[27875]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:46.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:46 ntpd[27875]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:46.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:46 ntpd[27875]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:46.313 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:46 ntpd[27875]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:46.314 INFO:teuthology.orchestra.run.smithi026.stdout:19 Dec 16:38:46 ntpd[27875]: ntpd: time slew -0.001220 s 2023-12-19T16:38:46.314 INFO:teuthology.orchestra.run.smithi026.stdout:ntpd: time slew -0.001220s 2023-12-19T16:38:46.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:46 ntpd[27875]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-12-19T16:38:46.314 INFO:teuthology.orchestra.run.smithi026.stderr:19 Dec 16:38:46 ntpd[27875]: can't open /var/log/ntpstats/loopstats.20231219: Permission denied 2023-12-19T16:38:46.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:46.338 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:46.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:46.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:46.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:46.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:46.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:46.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:46.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:46.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:46.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-19T16:38:46.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: can't open /var/log/ntpstats/rawstats.20231219: Permission denied 2023-12-19T16:38:46.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-19T16:38:46.339 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: can't open /var/log/ntpstats/peerstats.20231219: Permission denied 2023-12-19T16:38:46.340 INFO:teuthology.orchestra.run.smithi160.stdout:19 Dec 16:38:46 ntpd[27991]: ntpd: time slew -0.000797 s 2023-12-19T16:38:46.340 INFO:teuthology.orchestra.run.smithi160.stdout:ntpd: time slew -0.000797s 2023-12-19T16:38:46.340 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-12-19T16:38:46.340 INFO:teuthology.orchestra.run.smithi160.stderr:19 Dec 16:38:46 ntpd[27991]: can't open /var/log/ntpstats/loopstats.20231219: Permission denied 2023-12-19T16:38:46.445 INFO:teuthology.orchestra.run.smithi026.stdout: remote refid st t when poll reach delay offset jitter 2023-12-19T16:38:46.446 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================== 2023-12-19T16:38:46.446 INFO:teuthology.orchestra.run.smithi026.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-19T16:38:46.446 INFO:teuthology.orchestra.run.smithi026.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-19T16:38:46.446 INFO:teuthology.orchestra.run.smithi026.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-19T16:38:46.446 INFO:teuthology.orchestra.run.smithi026.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-19T16:38:46.449 INFO:teuthology.orchestra.run.smithi160.stdout: remote refid st t when poll reach delay offset jitter 2023-12-19T16:38:46.449 INFO:teuthology.orchestra.run.smithi160.stdout:============================================================================== 2023-12-19T16:38:46.449 INFO:teuthology.orchestra.run.smithi160.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-19T16:38:46.449 INFO:teuthology.orchestra.run.smithi160.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-19T16:38:46.449 INFO:teuthology.orchestra.run.smithi160.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-19T16:38:46.449 INFO:teuthology.orchestra.run.smithi160.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-19T16:38:46.451 INFO:teuthology.run_tasks:Running task install... 2023-12-19T16:38:46.460 DEBUG:teuthology.task.install:project ceph 2023-12-19T16:38:46.461 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}} 2023-12-19T16:38:46.461 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'} 2023-12-19T16:38:46.461 INFO:teuthology.task.install:Using flavor: default 2023-12-19T16:38:46.467 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-19T16:38:46.467 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2023-12-19T16:38:46.468 DEBUG:teuthology.orchestra.run.smithi026:> sudo apt-key list | grep Ceph 2023-12-19T16:38:46.468 DEBUG:teuthology.orchestra.run.smithi160:> sudo apt-key list | grep Ceph 2023-12-19T16:38:46.600 INFO:teuthology.orchestra.run.smithi026.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-12-19T16:38:46.603 INFO:teuthology.orchestra.run.smithi160.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-12-19T16:38:46.654 INFO:teuthology.orchestra.run.smithi026.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-12-19T16:38:46.654 INFO:teuthology.orchestra.run.smithi026.stdout:uid [ unknown] Ceph.com (release key) 2023-12-19T16:38:46.655 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-12-19T16:38:46.655 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-12-19T16:38:46.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-19T16:38:46.661 INFO:teuthology.orchestra.run.smithi160.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-12-19T16:38:46.661 INFO:teuthology.orchestra.run.smithi160.stdout:uid [ unknown] Ceph.com (release key) 2023-12-19T16:38:46.661 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-12-19T16:38:46.661 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-12-19T16:38:46.662 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-19T16:38:46.793 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default/ 2023-12-19T16:38:46.794 INFO:teuthology.task.install.deb:Package version is 18.2.1-132-g568df92b-1jammy 2023-12-19T16:38:46.872 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default/ 2023-12-19T16:38:46.872 INFO:teuthology.task.install.deb:Package version is 18.2.1-132-g568df92b-1jammy 2023-12-19T16:38:46.968 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:38:46.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-12-19T16:38:46.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo apt-get update 2023-12-19T16:38:47.040 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:38:47.040 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-12-19T16:38:47.061 DEBUG:teuthology.orchestra.run.smithi160:> sudo apt-get update 2023-12-19T16:38:47.188 INFO:teuthology.orchestra.run.smithi026.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-12-19T16:38:47.248 INFO:teuthology.orchestra.run.smithi160.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-12-19T16:38:47.295 INFO:teuthology.orchestra.run.smithi160.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy InRelease 2023-12-19T16:38:47.302 INFO:teuthology.orchestra.run.smithi026.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-12-19T16:38:47.308 INFO:teuthology.orchestra.run.smithi026.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy InRelease 2023-12-19T16:38:47.316 INFO:teuthology.orchestra.run.smithi160.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-12-19T16:38:47.338 INFO:teuthology.orchestra.run.smithi160.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy Release.gpg 2023-12-19T16:38:47.344 INFO:teuthology.orchestra.run.smithi026.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-12-19T16:38:47.359 INFO:teuthology.orchestra.run.smithi160.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2023-12-19T16:38:47.373 INFO:teuthology.orchestra.run.smithi160.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-12-19T16:38:47.382 INFO:teuthology.orchestra.run.smithi026.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy Release.gpg 2023-12-19T16:38:47.384 INFO:teuthology.orchestra.run.smithi026.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-12-19T16:38:47.394 INFO:teuthology.orchestra.run.smithi160.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2023-12-19T16:38:47.418 INFO:teuthology.orchestra.run.smithi026.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2023-12-19T16:38:47.444 INFO:teuthology.orchestra.run.smithi160.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-12-19T16:38:47.466 INFO:teuthology.orchestra.run.smithi026.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-12-19T16:38:47.466 INFO:teuthology.orchestra.run.smithi026.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2023-12-19T16:38:47.526 INFO:teuthology.orchestra.run.smithi160.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-12-19T16:38:47.819 INFO:teuthology.orchestra.run.smithi026.stdout:Fetched 28.5 kB in 1s (47.8 kB/s) 2023-12-19T16:38:47.858 INFO:teuthology.orchestra.run.smithi160.stdout:Fetched 28.5 kB in 1s (49.4 kB/s) 2023-12-19T16:38:48.785 INFO:teuthology.orchestra.run.smithi026.stdout:Reading package lists... 2023-12-19T16:38:48.788 INFO:teuthology.orchestra.run.smithi160.stdout:Reading package lists... 2023-12-19T16:38:48.799 DEBUG:teuthology.orchestra.run.smithi026:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.1-132-g568df92b-1jammy cephadm=18.2.1-132-g568df92b-1jammy ceph-mds=18.2.1-132-g568df92b-1jammy ceph-mgr=18.2.1-132-g568df92b-1jammy ceph-common=18.2.1-132-g568df92b-1jammy ceph-fuse=18.2.1-132-g568df92b-1jammy ceph-test=18.2.1-132-g568df92b-1jammy ceph-volume=18.2.1-132-g568df92b-1jammy radosgw=18.2.1-132-g568df92b-1jammy python3-rados=18.2.1-132-g568df92b-1jammy python3-rgw=18.2.1-132-g568df92b-1jammy python3-cephfs=18.2.1-132-g568df92b-1jammy python3-rbd=18.2.1-132-g568df92b-1jammy libcephfs2=18.2.1-132-g568df92b-1jammy libcephfs-dev=18.2.1-132-g568df92b-1jammy librados2=18.2.1-132-g568df92b-1jammy librbd1=18.2.1-132-g568df92b-1jammy rbd-fuse=18.2.1-132-g568df92b-1jammy python3-cephfs=18.2.1-132-g568df92b-1jammy cephfs-shell=18.2.1-132-g568df92b-1jammy cephfs-top=18.2.1-132-g568df92b-1jammy cephfs-mirror=18.2.1-132-g568df92b-1jammy 2023-12-19T16:38:48.803 DEBUG:teuthology.orchestra.run.smithi160:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.1-132-g568df92b-1jammy cephadm=18.2.1-132-g568df92b-1jammy ceph-mds=18.2.1-132-g568df92b-1jammy ceph-mgr=18.2.1-132-g568df92b-1jammy ceph-common=18.2.1-132-g568df92b-1jammy ceph-fuse=18.2.1-132-g568df92b-1jammy ceph-test=18.2.1-132-g568df92b-1jammy ceph-volume=18.2.1-132-g568df92b-1jammy radosgw=18.2.1-132-g568df92b-1jammy python3-rados=18.2.1-132-g568df92b-1jammy python3-rgw=18.2.1-132-g568df92b-1jammy python3-cephfs=18.2.1-132-g568df92b-1jammy python3-rbd=18.2.1-132-g568df92b-1jammy libcephfs2=18.2.1-132-g568df92b-1jammy libcephfs-dev=18.2.1-132-g568df92b-1jammy librados2=18.2.1-132-g568df92b-1jammy librbd1=18.2.1-132-g568df92b-1jammy rbd-fuse=18.2.1-132-g568df92b-1jammy python3-cephfs=18.2.1-132-g568df92b-1jammy cephfs-shell=18.2.1-132-g568df92b-1jammy cephfs-top=18.2.1-132-g568df92b-1jammy cephfs-mirror=18.2.1-132-g568df92b-1jammy 2023-12-19T16:38:48.877 INFO:teuthology.orchestra.run.smithi026.stdout:Reading package lists... 2023-12-19T16:38:48.878 INFO:teuthology.orchestra.run.smithi160.stdout:Reading package lists... 2023-12-19T16:38:49.045 INFO:teuthology.orchestra.run.smithi160.stdout:Building dependency tree... 2023-12-19T16:38:49.046 INFO:teuthology.orchestra.run.smithi160.stdout:Reading state information... 2023-12-19T16:38:49.047 INFO:teuthology.orchestra.run.smithi026.stdout:Building dependency tree... 2023-12-19T16:38:49.048 INFO:teuthology.orchestra.run.smithi026.stdout:Reading state information... 2023-12-19T16:38:49.200 INFO:teuthology.orchestra.run.smithi160.stdout:The following packages were automatically installed and are no longer required: 2023-12-19T16:38:49.200 INFO:teuthology.orchestra.run.smithi160.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-19T16:38:49.200 INFO:teuthology.orchestra.run.smithi160.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-19T16:38:49.200 INFO:teuthology.orchestra.run.smithi160.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-19T16:38:49.200 INFO:teuthology.orchestra.run.smithi160.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-19T16:38:49.200 INFO:teuthology.orchestra.run.smithi160.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-19T16:38:49.201 INFO:teuthology.orchestra.run.smithi160.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-19T16:38:49.202 INFO:teuthology.orchestra.run.smithi160.stdout:The following additional packages will be installed: 2023-12-19T16:38:49.202 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-12-19T16:38:49.202 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-12-19T16:38:49.202 INFO:teuthology.orchestra.run.smithi160.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-12-19T16:38:49.202 INFO:teuthology.orchestra.run.smithi160.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-12-19T16:38:49.202 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-12-19T16:38:49.202 INFO:teuthology.orchestra.run.smithi160.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-12-19T16:38:49.203 INFO:teuthology.orchestra.run.smithi160.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-12-19T16:38:49.204 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-12-19T16:38:49.204 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-12-19T16:38:49.204 INFO:teuthology.orchestra.run.smithi160.stdout: qttranslations5-l10n socat xmlstarlet 2023-12-19T16:38:49.204 INFO:teuthology.orchestra.run.smithi160.stdout:Suggested packages: 2023-12-19T16:38:49.204 INFO:teuthology.orchestra.run.smithi160.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-12-19T16:38:49.204 INFO:teuthology.orchestra.run.smithi160.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-12-19T16:38:49.204 INFO:teuthology.orchestra.run.smithi160.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-12-19T16:38:49.204 INFO:teuthology.orchestra.run.smithi160.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-12-19T16:38:49.204 INFO:teuthology.orchestra.run.smithi160.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-12-19T16:38:49.204 INFO:teuthology.orchestra.run.smithi160.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-12-19T16:38:49.204 INFO:teuthology.orchestra.run.smithi160.stdout: python3-watchdog 2023-12-19T16:38:49.204 INFO:teuthology.orchestra.run.smithi160.stdout:Recommended packages: 2023-12-19T16:38:49.204 INFO:teuthology.orchestra.run.smithi160.stdout: btrfs-tools 2023-12-19T16:38:49.211 INFO:teuthology.orchestra.run.smithi026.stdout:The following packages were automatically installed and are no longer required: 2023-12-19T16:38:49.211 INFO:teuthology.orchestra.run.smithi026.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-19T16:38:49.211 INFO:teuthology.orchestra.run.smithi026.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-19T16:38:49.211 INFO:teuthology.orchestra.run.smithi026.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-19T16:38:49.212 INFO:teuthology.orchestra.run.smithi026.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-19T16:38:49.212 INFO:teuthology.orchestra.run.smithi026.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-19T16:38:49.212 INFO:teuthology.orchestra.run.smithi026.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-19T16:38:49.213 INFO:teuthology.orchestra.run.smithi026.stdout:The following additional packages will be installed: 2023-12-19T16:38:49.213 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-12-19T16:38:49.213 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-12-19T16:38:49.213 INFO:teuthology.orchestra.run.smithi026.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-12-19T16:38:49.213 INFO:teuthology.orchestra.run.smithi026.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-12-19T16:38:49.214 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-12-19T16:38:49.214 INFO:teuthology.orchestra.run.smithi026.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-12-19T16:38:49.214 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-12-19T16:38:49.214 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-12-19T16:38:49.214 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-12-19T16:38:49.214 INFO:teuthology.orchestra.run.smithi026.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-12-19T16:38:49.214 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-12-19T16:38:49.214 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-12-19T16:38:49.214 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-12-19T16:38:49.215 INFO:teuthology.orchestra.run.smithi026.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-12-19T16:38:49.215 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-12-19T16:38:49.215 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-12-19T16:38:49.215 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-12-19T16:38:49.215 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-12-19T16:38:49.215 INFO:teuthology.orchestra.run.smithi026.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-12-19T16:38:49.215 INFO:teuthology.orchestra.run.smithi026.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-12-19T16:38:49.215 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-12-19T16:38:49.215 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-12-19T16:38:49.215 INFO:teuthology.orchestra.run.smithi026.stdout: qttranslations5-l10n socat xmlstarlet 2023-12-19T16:38:49.216 INFO:teuthology.orchestra.run.smithi026.stdout:Suggested packages: 2023-12-19T16:38:49.216 INFO:teuthology.orchestra.run.smithi026.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-12-19T16:38:49.216 INFO:teuthology.orchestra.run.smithi026.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-12-19T16:38:49.216 INFO:teuthology.orchestra.run.smithi026.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-12-19T16:38:49.216 INFO:teuthology.orchestra.run.smithi026.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-12-19T16:38:49.216 INFO:teuthology.orchestra.run.smithi026.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-12-19T16:38:49.216 INFO:teuthology.orchestra.run.smithi026.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-12-19T16:38:49.216 INFO:teuthology.orchestra.run.smithi026.stdout: python3-watchdog 2023-12-19T16:38:49.216 INFO:teuthology.orchestra.run.smithi026.stdout:Recommended packages: 2023-12-19T16:38:49.216 INFO:teuthology.orchestra.run.smithi026.stdout: btrfs-tools 2023-12-19T16:38:49.347 INFO:teuthology.orchestra.run.smithi160.stdout:The following NEW packages will be installed: 2023-12-19T16:38:49.348 INFO:teuthology.orchestra.run.smithi160.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-12-19T16:38:49.348 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-12-19T16:38:49.348 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-12-19T16:38:49.348 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-12-19T16:38:49.349 INFO:teuthology.orchestra.run.smithi160.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-12-19T16:38:49.349 INFO:teuthology.orchestra.run.smithi160.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-12-19T16:38:49.349 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-12-19T16:38:49.350 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-12-19T16:38:49.351 INFO:teuthology.orchestra.run.smithi160.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-12-19T16:38:49.351 INFO:teuthology.orchestra.run.smithi160.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-12-19T16:38:49.351 INFO:teuthology.orchestra.run.smithi160.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-12-19T16:38:49.351 INFO:teuthology.orchestra.run.smithi160.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-12-19T16:38:49.351 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-12-19T16:38:49.351 INFO:teuthology.orchestra.run.smithi160.stdout: socat xmlstarlet 2023-12-19T16:38:49.365 INFO:teuthology.orchestra.run.smithi026.stdout:The following NEW packages will be installed: 2023-12-19T16:38:49.365 INFO:teuthology.orchestra.run.smithi026.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-12-19T16:38:49.365 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-12-19T16:38:49.365 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-12-19T16:38:49.365 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-12-19T16:38:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-12-19T16:38:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-12-19T16:38:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-12-19T16:38:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-12-19T16:38:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-12-19T16:38:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-12-19T16:38:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-12-19T16:38:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-12-19T16:38:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-12-19T16:38:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-12-19T16:38:49.367 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-12-19T16:38:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-12-19T16:38:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-12-19T16:38:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-12-19T16:38:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-12-19T16:38:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-12-19T16:38:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-12-19T16:38:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-12-19T16:38:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-12-19T16:38:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-12-19T16:38:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-12-19T16:38:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-12-19T16:38:49.368 INFO:teuthology.orchestra.run.smithi026.stdout: socat xmlstarlet 2023-12-19T16:38:49.416 INFO:teuthology.orchestra.run.smithi160.stdout:0 upgraded, 115 newly installed, 0 to remove and 268 not upgraded. 2023-12-19T16:38:49.416 INFO:teuthology.orchestra.run.smithi160.stdout:Need to get 181 MB of archives. 2023-12-19T16:38:49.416 INFO:teuthology.orchestra.run.smithi160.stdout:After this operation, 812 MB of additional disk space will be used. 2023-12-19T16:38:49.416 INFO:teuthology.orchestra.run.smithi160.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-12-19T16:38:49.433 INFO:teuthology.orchestra.run.smithi160.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-12-19T16:38:49.443 INFO:teuthology.orchestra.run.smithi160.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-12-19T16:38:49.485 INFO:teuthology.orchestra.run.smithi160.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-12-19T16:38:49.493 INFO:teuthology.orchestra.run.smithi160.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-12-19T16:38:49.509 INFO:teuthology.orchestra.run.smithi160.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-12-19T16:38:49.512 INFO:teuthology.orchestra.run.smithi160.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-12-19T16:38:49.522 INFO:teuthology.orchestra.run.smithi160.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-12-19T16:38:49.557 INFO:teuthology.orchestra.run.smithi160.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-12-19T16:38:49.559 INFO:teuthology.orchestra.run.smithi160.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-12-19T16:38:49.563 INFO:teuthology.orchestra.run.smithi026.stdout:0 upgraded, 115 newly installed, 0 to remove and 268 not upgraded. 2023-12-19T16:38:49.563 INFO:teuthology.orchestra.run.smithi026.stdout:Need to get 181 MB of archives. 2023-12-19T16:38:49.563 INFO:teuthology.orchestra.run.smithi026.stdout:After this operation, 812 MB of additional disk space will be used. 2023-12-19T16:38:49.563 INFO:teuthology.orchestra.run.smithi026.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-12-19T16:38:49.566 INFO:teuthology.orchestra.run.smithi160.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-12-19T16:38:49.569 INFO:teuthology.orchestra.run.smithi160.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-12-19T16:38:49.569 INFO:teuthology.orchestra.run.smithi160.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-12-19T16:38:49.573 INFO:teuthology.orchestra.run.smithi160.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.1-132-g568df92b-1jammy [3,418 kB] 2023-12-19T16:38:49.575 INFO:teuthology.orchestra.run.smithi160.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-12-19T16:38:49.576 INFO:teuthology.orchestra.run.smithi160.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-12-19T16:38:49.576 INFO:teuthology.orchestra.run.smithi160.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-12-19T16:38:49.577 INFO:teuthology.orchestra.run.smithi160.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-12-19T16:38:49.577 INFO:teuthology.orchestra.run.smithi160.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-12-19T16:38:49.592 INFO:teuthology.orchestra.run.smithi026.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.1-132-g568df92b-1jammy [3,418 kB] 2023-12-19T16:38:49.593 INFO:teuthology.orchestra.run.smithi160.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-12-19T16:38:49.593 INFO:teuthology.orchestra.run.smithi160.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-12-19T16:38:49.593 INFO:teuthology.orchestra.run.smithi160.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-12-19T16:38:49.593 INFO:teuthology.orchestra.run.smithi160.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-12-19T16:38:49.609 INFO:teuthology.orchestra.run.smithi160.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-12-19T16:38:49.611 INFO:teuthology.orchestra.run.smithi160.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-12-19T16:38:49.611 INFO:teuthology.orchestra.run.smithi160.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-12-19T16:38:49.612 INFO:teuthology.orchestra.run.smithi160.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-12-19T16:38:49.612 INFO:teuthology.orchestra.run.smithi160.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-12-19T16:38:49.613 INFO:teuthology.orchestra.run.smithi160.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-12-19T16:38:49.626 INFO:teuthology.orchestra.run.smithi160.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-12-19T16:38:49.626 INFO:teuthology.orchestra.run.smithi160.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-12-19T16:38:49.627 INFO:teuthology.orchestra.run.smithi160.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-12-19T16:38:49.627 INFO:teuthology.orchestra.run.smithi160.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-12-19T16:38:49.642 INFO:teuthology.orchestra.run.smithi160.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-12-19T16:38:49.646 INFO:teuthology.orchestra.run.smithi160.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-12-19T16:38:49.647 INFO:teuthology.orchestra.run.smithi160.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-12-19T16:38:49.647 INFO:teuthology.orchestra.run.smithi026.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-12-19T16:38:49.648 INFO:teuthology.orchestra.run.smithi160.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-12-19T16:38:49.648 INFO:teuthology.orchestra.run.smithi160.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-12-19T16:38:49.649 INFO:teuthology.orchestra.run.smithi160.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-12-19T16:38:49.659 INFO:teuthology.orchestra.run.smithi160.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-12-19T16:38:49.659 INFO:teuthology.orchestra.run.smithi160.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-12-19T16:38:49.661 INFO:teuthology.orchestra.run.smithi160.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-12-19T16:38:49.661 INFO:teuthology.orchestra.run.smithi160.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-12-19T16:38:49.676 INFO:teuthology.orchestra.run.smithi160.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-12-19T16:38:49.677 INFO:teuthology.orchestra.run.smithi160.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-12-19T16:38:49.678 INFO:teuthology.orchestra.run.smithi160.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-12-19T16:38:49.691 INFO:teuthology.orchestra.run.smithi026.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-12-19T16:38:49.722 INFO:teuthology.orchestra.run.smithi160.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-12-19T16:38:49.723 INFO:teuthology.orchestra.run.smithi160.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-12-19T16:38:49.724 INFO:teuthology.orchestra.run.smithi160.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-12-19T16:38:49.725 INFO:teuthology.orchestra.run.smithi160.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-12-19T16:38:49.725 INFO:teuthology.orchestra.run.smithi160.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-12-19T16:38:49.726 INFO:teuthology.orchestra.run.smithi160.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-12-19T16:38:49.727 INFO:teuthology.orchestra.run.smithi160.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-12-19T16:38:49.727 INFO:teuthology.orchestra.run.smithi160.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-12-19T16:38:49.729 INFO:teuthology.orchestra.run.smithi160.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-12-19T16:38:49.741 INFO:teuthology.orchestra.run.smithi160.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-12-19T16:38:49.747 INFO:teuthology.orchestra.run.smithi160.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-12-19T16:38:49.766 INFO:teuthology.orchestra.run.smithi160.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-12-19T16:38:49.768 INFO:teuthology.orchestra.run.smithi160.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-12-19T16:38:49.768 INFO:teuthology.orchestra.run.smithi160.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-12-19T16:38:49.785 INFO:teuthology.orchestra.run.smithi160.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-12-19T16:38:49.785 INFO:teuthology.orchestra.run.smithi160.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-12-19T16:38:49.785 INFO:teuthology.orchestra.run.smithi160.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-12-19T16:38:49.786 INFO:teuthology.orchestra.run.smithi160.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-12-19T16:38:49.786 INFO:teuthology.orchestra.run.smithi160.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-12-19T16:38:49.787 INFO:teuthology.orchestra.run.smithi160.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-12-19T16:38:49.791 INFO:teuthology.orchestra.run.smithi160.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-12-19T16:38:49.791 INFO:teuthology.orchestra.run.smithi160.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-12-19T16:38:49.795 INFO:teuthology.orchestra.run.smithi160.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-12-19T16:38:49.807 INFO:teuthology.orchestra.run.smithi160.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-12-19T16:38:49.809 INFO:teuthology.orchestra.run.smithi160.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-12-19T16:38:49.810 INFO:teuthology.orchestra.run.smithi160.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-12-19T16:38:49.813 INFO:teuthology.orchestra.run.smithi160.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-12-19T16:38:49.816 INFO:teuthology.orchestra.run.smithi160.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-12-19T16:38:49.816 INFO:teuthology.orchestra.run.smithi160.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-12-19T16:38:49.820 INFO:teuthology.orchestra.run.smithi160.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-12-19T16:38:49.824 INFO:teuthology.orchestra.run.smithi160.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-12-19T16:38:49.824 INFO:teuthology.orchestra.run.smithi160.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-12-19T16:38:49.826 INFO:teuthology.orchestra.run.smithi160.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-12-19T16:38:49.842 INFO:teuthology.orchestra.run.smithi160.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-12-19T16:38:49.842 INFO:teuthology.orchestra.run.smithi160.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-12-19T16:38:49.842 INFO:teuthology.orchestra.run.smithi160.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-12-19T16:38:49.899 INFO:teuthology.orchestra.run.smithi026.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-12-19T16:38:49.914 INFO:teuthology.orchestra.run.smithi160.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.1-132-g568df92b-1jammy [3,104 kB] 2023-12-19T16:38:49.943 INFO:teuthology.orchestra.run.smithi026.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-12-19T16:38:50.019 INFO:teuthology.orchestra.run.smithi026.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-12-19T16:38:50.030 INFO:teuthology.orchestra.run.smithi026.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-12-19T16:38:50.043 INFO:teuthology.orchestra.run.smithi160.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.1-132-g568df92b-1jammy [670 kB] 2023-12-19T16:38:50.065 INFO:teuthology.orchestra.run.smithi160.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.1-132-g568df92b-1jammy [356 kB] 2023-12-19T16:38:50.067 INFO:teuthology.orchestra.run.smithi026.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.1-132-g568df92b-1jammy [3,104 kB] 2023-12-19T16:38:50.077 INFO:teuthology.orchestra.run.smithi026.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-12-19T16:38:50.106 INFO:teuthology.orchestra.run.smithi160.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.1-132-g568df92b-1jammy [32.8 kB] 2023-12-19T16:38:50.107 INFO:teuthology.orchestra.run.smithi160.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.1-132-g568df92b-1jammy [180 kB] 2023-12-19T16:38:50.113 INFO:teuthology.orchestra.run.smithi160.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.1-132-g568df92b-1jammy [61.5 kB] 2023-12-19T16:38:50.115 INFO:teuthology.orchestra.run.smithi160.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.1-132-g568df92b-1jammy [330 kB] 2023-12-19T16:38:50.125 INFO:teuthology.orchestra.run.smithi160.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.1-132-g568df92b-1jammy [5,884 kB] 2023-12-19T16:38:50.243 INFO:teuthology.orchestra.run.smithi026.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-12-19T16:38:50.251 INFO:teuthology.orchestra.run.smithi026.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-12-19T16:38:50.278 INFO:teuthology.orchestra.run.smithi026.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-12-19T16:38:50.316 INFO:teuthology.orchestra.run.smithi026.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.1-132-g568df92b-1jammy [670 kB] 2023-12-19T16:38:50.341 INFO:teuthology.orchestra.run.smithi026.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.1-132-g568df92b-1jammy [356 kB] 2023-12-19T16:38:50.402 INFO:teuthology.orchestra.run.smithi026.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.1-132-g568df92b-1jammy [32.8 kB] 2023-12-19T16:38:50.403 INFO:teuthology.orchestra.run.smithi026.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.1-132-g568df92b-1jammy [180 kB] 2023-12-19T16:38:50.409 INFO:teuthology.orchestra.run.smithi026.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.1-132-g568df92b-1jammy [61.5 kB] 2023-12-19T16:38:50.411 INFO:teuthology.orchestra.run.smithi026.stdout:Get:19 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.1-132-g568df92b-1jammy [330 kB] 2023-12-19T16:38:50.425 INFO:teuthology.orchestra.run.smithi026.stdout:Get:20 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.1-132-g568df92b-1jammy [5,884 kB] 2023-12-19T16:38:50.451 INFO:teuthology.orchestra.run.smithi026.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-12-19T16:38:50.456 INFO:teuthology.orchestra.run.smithi160.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.1-132-g568df92b-1jammy [112 kB] 2023-12-19T16:38:50.457 INFO:teuthology.orchestra.run.smithi160.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.1-132-g568df92b-1jammy [442 kB] 2023-12-19T16:38:50.472 INFO:teuthology.orchestra.run.smithi160.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.1-132-g568df92b-1jammy [22.0 MB] 2023-12-19T16:38:50.643 INFO:teuthology.orchestra.run.smithi026.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-12-19T16:38:50.873 INFO:teuthology.orchestra.run.smithi026.stdout:Get:23 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.1-132-g568df92b-1jammy [112 kB] 2023-12-19T16:38:50.877 INFO:teuthology.orchestra.run.smithi026.stdout:Get:24 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.1-132-g568df92b-1jammy [442 kB] 2023-12-19T16:38:50.922 INFO:teuthology.orchestra.run.smithi026.stdout:Get:25 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.1-132-g568df92b-1jammy [22.0 MB] 2023-12-19T16:38:51.061 INFO:teuthology.orchestra.run.smithi026.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-12-19T16:38:51.069 INFO:teuthology.orchestra.run.smithi026.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-12-19T16:38:51.070 INFO:teuthology.orchestra.run.smithi026.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-12-19T16:38:51.081 INFO:teuthology.orchestra.run.smithi026.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-12-19T16:38:51.082 INFO:teuthology.orchestra.run.smithi026.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-12-19T16:38:51.084 INFO:teuthology.orchestra.run.smithi026.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-12-19T16:38:51.086 INFO:teuthology.orchestra.run.smithi026.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-12-19T16:38:51.088 INFO:teuthology.orchestra.run.smithi026.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-12-19T16:38:51.088 INFO:teuthology.orchestra.run.smithi026.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-12-19T16:38:51.254 INFO:teuthology.orchestra.run.smithi026.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-12-19T16:38:51.699 INFO:teuthology.orchestra.run.smithi026.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-12-19T16:38:51.709 INFO:teuthology.orchestra.run.smithi026.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-12-19T16:38:51.715 INFO:teuthology.orchestra.run.smithi026.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-12-19T16:38:51.719 INFO:teuthology.orchestra.run.smithi026.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-12-19T16:38:51.738 INFO:teuthology.orchestra.run.smithi026.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-12-19T16:38:51.741 INFO:teuthology.orchestra.run.smithi026.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-12-19T16:38:51.743 INFO:teuthology.orchestra.run.smithi026.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-12-19T16:38:51.784 INFO:teuthology.orchestra.run.smithi026.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-12-19T16:38:51.798 INFO:teuthology.orchestra.run.smithi026.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-12-19T16:38:51.803 INFO:teuthology.orchestra.run.smithi026.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-12-19T16:38:51.848 INFO:teuthology.orchestra.run.smithi160.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.1-132-g568df92b-1jammy [6,835 kB] 2023-12-19T16:38:52.057 INFO:teuthology.orchestra.run.smithi026.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-12-19T16:38:52.141 INFO:teuthology.orchestra.run.smithi026.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-12-19T16:38:52.211 INFO:teuthology.orchestra.run.smithi160.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.1-132-g568df92b-1jammy [237 kB] 2023-12-19T16:38:52.217 INFO:teuthology.orchestra.run.smithi160.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.1-132-g568df92b-1jammy [130 kB] 2023-12-19T16:38:52.233 INFO:teuthology.orchestra.run.smithi026.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-12-19T16:38:52.242 INFO:teuthology.orchestra.run.smithi160.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.1-132-g568df92b-1jammy [1,484 kB] 2023-12-19T16:38:52.305 INFO:teuthology.orchestra.run.smithi026.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-12-19T16:38:52.339 INFO:teuthology.orchestra.run.smithi160.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.1-132-g568df92b-1jammy [9,877 kB] 2023-12-19T16:38:52.412 INFO:teuthology.orchestra.run.smithi026.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-12-19T16:38:52.494 INFO:teuthology.orchestra.run.smithi026.stdout:Get:51 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.1-132-g568df92b-1jammy [6,835 kB] 2023-12-19T16:38:52.497 INFO:teuthology.orchestra.run.smithi026.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-12-19T16:38:52.516 INFO:teuthology.orchestra.run.smithi026.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-12-19T16:38:52.546 INFO:teuthology.orchestra.run.smithi026.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-12-19T16:38:52.558 INFO:teuthology.orchestra.run.smithi026.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-12-19T16:38:52.594 INFO:teuthology.orchestra.run.smithi026.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-12-19T16:38:52.603 INFO:teuthology.orchestra.run.smithi026.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-12-19T16:38:52.793 INFO:teuthology.orchestra.run.smithi026.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-12-19T16:38:52.930 INFO:teuthology.orchestra.run.smithi160.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.1-132-g568df92b-1jammy [31.5 MB] 2023-12-19T16:38:53.015 INFO:teuthology.orchestra.run.smithi026.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.1-132-g568df92b-1jammy [237 kB] 2023-12-19T16:38:53.017 INFO:teuthology.orchestra.run.smithi026.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.1-132-g568df92b-1jammy [130 kB] 2023-12-19T16:38:53.022 INFO:teuthology.orchestra.run.smithi026.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.1-132-g568df92b-1jammy [1,484 kB] 2023-12-19T16:38:53.122 INFO:teuthology.orchestra.run.smithi026.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.1-132-g568df92b-1jammy [9,877 kB] 2023-12-19T16:38:53.622 INFO:teuthology.orchestra.run.smithi026.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-12-19T16:38:53.623 INFO:teuthology.orchestra.run.smithi026.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-12-19T16:38:53.624 INFO:teuthology.orchestra.run.smithi026.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-12-19T16:38:53.625 INFO:teuthology.orchestra.run.smithi026.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-12-19T16:38:53.625 INFO:teuthology.orchestra.run.smithi026.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-12-19T16:38:53.626 INFO:teuthology.orchestra.run.smithi026.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-12-19T16:38:53.626 INFO:teuthology.orchestra.run.smithi026.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-12-19T16:38:53.627 INFO:teuthology.orchestra.run.smithi026.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-12-19T16:38:53.629 INFO:teuthology.orchestra.run.smithi026.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-12-19T16:38:53.703 INFO:teuthology.orchestra.run.smithi026.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-12-19T16:38:53.868 INFO:teuthology.orchestra.run.smithi026.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-12-19T16:38:53.872 INFO:teuthology.orchestra.run.smithi026.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.1-132-g568df92b-1jammy [31.5 MB] 2023-12-19T16:38:54.591 INFO:teuthology.orchestra.run.smithi026.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-12-19T16:38:54.596 INFO:teuthology.orchestra.run.smithi026.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-12-19T16:38:54.596 INFO:teuthology.orchestra.run.smithi026.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-12-19T16:38:54.673 INFO:teuthology.orchestra.run.smithi026.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-12-19T16:38:54.673 INFO:teuthology.orchestra.run.smithi026.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-12-19T16:38:54.674 INFO:teuthology.orchestra.run.smithi026.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-12-19T16:38:54.675 INFO:teuthology.orchestra.run.smithi026.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-12-19T16:38:54.676 INFO:teuthology.orchestra.run.smithi026.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-12-19T16:38:54.676 INFO:teuthology.orchestra.run.smithi026.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-12-19T16:38:54.677 INFO:teuthology.orchestra.run.smithi026.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-12-19T16:38:54.731 INFO:teuthology.orchestra.run.smithi160.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.1-132-g568df92b-1jammy [14.1 kB] 2023-12-19T16:38:54.731 INFO:teuthology.orchestra.run.smithi160.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.1-132-g568df92b-1jammy [824 kB] 2023-12-19T16:38:54.784 INFO:teuthology.orchestra.run.smithi160.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.1-132-g568df92b-1jammy [2,173 kB] 2023-12-19T16:38:54.841 INFO:teuthology.orchestra.run.smithi026.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-12-19T16:38:54.895 INFO:teuthology.orchestra.run.smithi160.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.1-132-g568df92b-1jammy [235 kB] 2023-12-19T16:38:54.908 INFO:teuthology.orchestra.run.smithi160.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.1-132-g568df92b-1jammy [138 kB] 2023-12-19T16:38:54.934 INFO:teuthology.orchestra.run.smithi160.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.1-132-g568df92b-1jammy [1,898 kB] 2023-12-19T16:38:55.047 INFO:teuthology.orchestra.run.smithi160.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.1-132-g568df92b-1jammy [8,625 kB] 2023-12-19T16:38:55.354 INFO:teuthology.orchestra.run.smithi026.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-12-19T16:38:55.382 INFO:teuthology.orchestra.run.smithi026.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-12-19T16:38:55.404 INFO:teuthology.orchestra.run.smithi026.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-12-19T16:38:55.413 INFO:teuthology.orchestra.run.smithi026.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-12-19T16:38:55.490 INFO:teuthology.orchestra.run.smithi026.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-12-19T16:38:55.523 INFO:teuthology.orchestra.run.smithi026.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-12-19T16:38:55.525 INFO:teuthology.orchestra.run.smithi026.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-12-19T16:38:55.529 INFO:teuthology.orchestra.run.smithi160.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.1-132-g568df92b-1jammy [14.2 kB] 2023-12-19T16:38:55.530 INFO:teuthology.orchestra.run.smithi160.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.1-132-g568df92b-1jammy [46.4 MB] 2023-12-19T16:38:55.561 INFO:teuthology.orchestra.run.smithi026.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-12-19T16:38:55.585 INFO:teuthology.orchestra.run.smithi026.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-12-19T16:38:55.589 INFO:teuthology.orchestra.run.smithi026.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-12-19T16:38:55.763 INFO:teuthology.orchestra.run.smithi026.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-12-19T16:38:56.369 INFO:teuthology.orchestra.run.smithi026.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-12-19T16:38:56.371 INFO:teuthology.orchestra.run.smithi026.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-12-19T16:38:56.374 INFO:teuthology.orchestra.run.smithi026.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-12-19T16:38:56.443 INFO:teuthology.orchestra.run.smithi026.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.1-132-g568df92b-1jammy [14.1 kB] 2023-12-19T16:38:56.443 INFO:teuthology.orchestra.run.smithi026.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.1-132-g568df92b-1jammy [824 kB] 2023-12-19T16:38:56.507 INFO:teuthology.orchestra.run.smithi026.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.1-132-g568df92b-1jammy [2,173 kB] 2023-12-19T16:38:56.708 INFO:teuthology.orchestra.run.smithi026.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.1-132-g568df92b-1jammy [235 kB] 2023-12-19T16:38:56.716 INFO:teuthology.orchestra.run.smithi026.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.1-132-g568df92b-1jammy [138 kB] 2023-12-19T16:38:56.722 INFO:teuthology.orchestra.run.smithi026.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.1-132-g568df92b-1jammy [1,898 kB] 2023-12-19T16:38:56.900 INFO:teuthology.orchestra.run.smithi026.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.1-132-g568df92b-1jammy [8,625 kB] 2023-12-19T16:38:57.759 INFO:teuthology.orchestra.run.smithi026.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.1-132-g568df92b-1jammy [14.2 kB] 2023-12-19T16:38:57.760 INFO:teuthology.orchestra.run.smithi026.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.1-132-g568df92b-1jammy [46.4 MB] 2023-12-19T16:38:57.910 INFO:teuthology.orchestra.run.smithi160.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.1-132-g568df92b-1jammy [121 kB] 2023-12-19T16:38:57.910 INFO:teuthology.orchestra.run.smithi160.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.1-132-g568df92b-1jammy [228 kB] 2023-12-19T16:38:57.912 INFO:teuthology.orchestra.run.smithi160.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.1-132-g568df92b-1jammy [34.0 kB] 2023-12-19T16:38:57.913 INFO:teuthology.orchestra.run.smithi160.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.1-132-g568df92b-1jammy [23.3 kB] 2023-12-19T16:38:57.913 INFO:teuthology.orchestra.run.smithi160.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.1-132-g568df92b-1jammy [38.3 kB] 2023-12-19T16:38:57.914 INFO:teuthology.orchestra.run.smithi160.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.1-132-g568df92b-1jammy [11.6 MB] 2023-12-19T16:38:58.515 INFO:teuthology.orchestra.run.smithi160.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.1-132-g568df92b-1jammy [92.9 kB] 2023-12-19T16:38:59.081 INFO:teuthology.orchestra.run.smithi160.stdout:Fetched 181 MB in 9s (19.8 MB/s) 2023-12-19T16:38:59.287 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pyperclip. 2023-12-19T16:38:59.357 INFO:teuthology.orchestra.run.smithi160.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132930 files and directories currently installed.) 2023-12-19T16:38:59.360 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-12-19T16:38:59.411 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-12-19T16:38:59.763 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-wcwidth. 2023-12-19T16:38:59.777 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-12-19T16:38:59.830 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-19T16:38:59.907 INFO:teuthology.orchestra.run.smithi026.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.1-132-g568df92b-1jammy [121 kB] 2023-12-19T16:38:59.909 INFO:teuthology.orchestra.run.smithi026.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.1-132-g568df92b-1jammy [228 kB] 2023-12-19T16:38:59.910 INFO:teuthology.orchestra.run.smithi026.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.1-132-g568df92b-1jammy [34.0 kB] 2023-12-19T16:38:59.911 INFO:teuthology.orchestra.run.smithi026.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.1-132-g568df92b-1jammy [23.3 kB] 2023-12-19T16:38:59.911 INFO:teuthology.orchestra.run.smithi026.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.1-132-g568df92b-1jammy [38.3 kB] 2023-12-19T16:38:59.912 INFO:teuthology.orchestra.run.smithi026.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.1-132-g568df92b-1jammy [11.6 MB] 2023-12-19T16:39:00.249 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-cmd2. 2023-12-19T16:39:00.263 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-12-19T16:39:00.308 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-12-19T16:39:00.352 INFO:teuthology.orchestra.run.smithi026.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.1-132-g568df92b-1jammy [92.9 kB] 2023-12-19T16:39:00.677 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libfmt8:amd64. 2023-12-19T16:39:00.691 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-12-19T16:39:00.777 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-12-19T16:39:00.945 INFO:teuthology.orchestra.run.smithi026.stdout:Fetched 181 MB in 11s (16.5 MB/s) 2023-12-19T16:39:01.180 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-12-19T16:39:01.195 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-12-19T16:39:01.235 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pyperclip. 2023-12-19T16:39:01.247 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-19T16:39:01.307 INFO:teuthology.orchestra.run.smithi026.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132930 files and directories currently installed.) 2023-12-19T16:39:01.311 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-12-19T16:39:01.350 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-12-19T16:39:01.652 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-wcwidth. 2023-12-19T16:39:01.667 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-12-19T16:39:01.667 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-12-19T16:39:01.682 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-12-19T16:39:01.711 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-19T16:39:01.733 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-19T16:39:02.030 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-cmd2. 2023-12-19T16:39:02.046 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-12-19T16:39:02.089 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-12-19T16:39:02.378 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-12-19T16:39:02.393 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-12-19T16:39:02.425 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libfmt8:amd64. 2023-12-19T16:39:02.440 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-12-19T16:39:02.445 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-19T16:39:02.550 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-12-19T16:39:02.811 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-12-19T16:39:02.827 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-12-19T16:39:02.869 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-19T16:39:02.990 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-12-19T16:39:03.005 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-19T16:39:03.056 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-19T16:39:03.198 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-12-19T16:39:03.213 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-12-19T16:39:03.256 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-19T16:39:03.543 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-12-19T16:39:03.558 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-19T16:39:03.601 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-12-19T16:39:03.609 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-19T16:39:03.616 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-12-19T16:39:03.659 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-19T16:39:04.021 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-12-19T16:39:04.036 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-19T16:39:04.046 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-12-19T16:39:04.062 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-19T16:39:04.087 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-19T16:39:04.105 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-19T16:39:04.490 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-12-19T16:39:04.506 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-19T16:39:04.532 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-12-19T16:39:04.547 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-12-19T16:39:04.574 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-19T16:39:04.590 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-19T16:39:04.910 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-12-19T16:39:04.925 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-19T16:39:04.968 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package librados2. 2023-12-19T16:39:04.968 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-19T16:39:04.983 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../011-librados2_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:05.035 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking librados2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:05.330 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-12-19T16:39:05.345 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-12-19T16:39:05.380 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-19T16:39:05.529 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package librbd1. 2023-12-19T16:39:05.544 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../012-librbd1_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:05.632 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking librbd1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:05.682 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package librados2. 2023-12-19T16:39:05.698 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../011-librados2_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:05.741 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking librados2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:06.040 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libcephfs2. 2023-12-19T16:39:06.055 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../013-libcephfs2_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:06.098 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libcephfs2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:06.310 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package librbd1. 2023-12-19T16:39:06.326 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../012-librbd1_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:06.342 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-rados. 2023-12-19T16:39:06.357 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../014-python3-rados_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:06.392 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-rados (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:06.393 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking librbd1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:06.695 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-12-19T16:39:06.709 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:06.753 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-ceph-argparse (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:06.813 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libcephfs2. 2023-12-19T16:39:06.829 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../013-libcephfs2_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:06.871 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libcephfs2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:07.039 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-cephfs. 2023-12-19T16:39:07.054 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../016-python3-cephfs_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:07.097 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-cephfs (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:07.149 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-rados. 2023-12-19T16:39:07.164 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../014-python3-rados_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:07.216 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-rados (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:07.375 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-ceph-common. 2023-12-19T16:39:07.390 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:07.425 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:07.502 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-12-19T16:39:07.517 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:07.551 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-ceph-argparse (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:07.770 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-prettytable. 2023-12-19T16:39:07.785 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-12-19T16:39:07.795 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-cephfs. 2023-12-19T16:39:07.811 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../016-python3-cephfs_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:07.819 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-12-19T16:39:07.854 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-cephfs (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:08.080 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-rbd. 2023-12-19T16:39:08.095 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../019-python3-rbd_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:08.139 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-rbd (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:08.165 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-ceph-common. 2023-12-19T16:39:08.180 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:08.223 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:08.500 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-12-19T16:39:08.515 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-12-19T16:39:08.543 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-prettytable. 2023-12-19T16:39:08.558 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-19T16:39:08.558 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-12-19T16:39:08.601 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-12-19T16:39:08.837 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-rbd. 2023-12-19T16:39:08.852 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../019-python3-rbd_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:08.861 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package librgw2. 2023-12-19T16:39:08.876 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../021-librgw2_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:08.887 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-rbd (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:08.910 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking librgw2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:09.206 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-12-19T16:39:09.222 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-12-19T16:39:09.265 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-19T16:39:09.346 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-rgw. 2023-12-19T16:39:09.361 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../022-python3-rgw_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:09.404 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-rgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:09.593 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package librgw2. 2023-12-19T16:39:09.608 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../021-librgw2_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:09.651 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking librgw2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:09.741 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package liboath0:amd64. 2023-12-19T16:39:09.755 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-12-19T16:39:09.799 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-12-19T16:39:10.095 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-rgw. 2023-12-19T16:39:10.110 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../022-python3-rgw_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:10.135 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libradosstriper1. 2023-12-19T16:39:10.149 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../024-libradosstriper1_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:10.153 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-rgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:10.193 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libradosstriper1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:10.464 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package liboath0:amd64. 2023-12-19T16:39:10.480 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-12-19T16:39:10.514 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-12-19T16:39:10.590 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-common. 2023-12-19T16:39:10.605 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../025-ceph-common_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:10.648 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:10.826 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libradosstriper1. 2023-12-19T16:39:10.841 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../024-libradosstriper1_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:10.884 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libradosstriper1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:11.304 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-common. 2023-12-19T16:39:11.319 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../025-ceph-common_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:11.362 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:11.720 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-base. 2023-12-19T16:39:11.735 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../026-ceph-base_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:11.788 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-base (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:12.526 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-base. 2023-12-19T16:39:12.541 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../026-ceph-base_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:12.586 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-base (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:12.624 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-12-19T16:39:12.641 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-12-19T16:39:12.683 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-12-19T16:39:12.944 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-cheroot. 2023-12-19T16:39:12.958 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-12-19T16:39:13.002 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-19T16:39:13.239 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-12-19T16:39:13.253 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-12-19T16:39:13.287 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-12-19T16:39:13.297 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-12-19T16:39:13.302 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-12-19T16:39:13.346 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-12-19T16:39:13.600 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-cheroot. 2023-12-19T16:39:13.615 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-12-19T16:39:13.658 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-19T16:39:13.682 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-jaraco.text. 2023-12-19T16:39:13.697 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-12-19T16:39:13.741 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-12-19T16:39:13.986 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-12-19T16:39:14.002 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-12-19T16:39:14.044 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-12-19T16:39:14.044 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-12-19T16:39:14.058 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-12-19T16:39:14.093 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-12-19T16:39:14.354 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-tempora. 2023-12-19T16:39:14.355 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-jaraco.text. 2023-12-19T16:39:14.369 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-12-19T16:39:14.371 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-12-19T16:39:14.404 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-12-19T16:39:14.414 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-12-19T16:39:14.690 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-portend. 2023-12-19T16:39:14.692 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-12-19T16:39:14.705 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-12-19T16:39:14.707 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-12-19T16:39:14.749 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-12-19T16:39:14.750 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-12-19T16:39:15.036 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-tempora. 2023-12-19T16:39:15.051 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-12-19T16:39:15.076 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-12-19T16:39:15.091 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-12-19T16:39:15.096 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-12-19T16:39:15.135 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-12-19T16:39:15.397 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-portend. 2023-12-19T16:39:15.413 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-12-19T16:39:15.429 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-cherrypy3. 2023-12-19T16:39:15.444 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-12-19T16:39:15.480 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-12-19T16:39:15.487 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-12-19T16:39:15.742 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-12-19T16:39:15.757 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-12-19T16:39:15.792 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-12-19T16:39:15.849 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-natsort. 2023-12-19T16:39:15.864 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-12-19T16:39:15.932 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-12-19T16:39:16.094 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-cherrypy3. 2023-12-19T16:39:16.110 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-12-19T16:39:16.153 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-12-19T16:39:16.243 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-logutils. 2023-12-19T16:39:16.258 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-12-19T16:39:16.301 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-12-19T16:39:16.473 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-natsort. 2023-12-19T16:39:16.488 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-12-19T16:39:16.523 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-12-19T16:39:16.604 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-markupsafe. 2023-12-19T16:39:16.619 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-12-19T16:39:16.662 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-12-19T16:39:16.914 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-mako. 2023-12-19T16:39:16.929 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-12-19T16:39:16.965 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-19T16:39:17.051 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-logutils. 2023-12-19T16:39:17.067 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-12-19T16:39:17.120 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-12-19T16:39:17.343 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-simplegeneric. 2023-12-19T16:39:17.358 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-12-19T16:39:17.420 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-markupsafe. 2023-12-19T16:39:17.435 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-12-19T16:39:17.479 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-12-19T16:39:17.502 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-12-19T16:39:17.781 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-mako. 2023-12-19T16:39:17.797 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-12-19T16:39:17.829 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-singledispatch. 2023-12-19T16:39:17.840 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-19T16:39:17.844 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-12-19T16:39:17.879 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-12-19T16:39:18.148 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-webob. 2023-12-19T16:39:18.163 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-12-19T16:39:18.193 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-simplegeneric. 2023-12-19T16:39:18.206 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-12-19T16:39:18.209 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-12-19T16:39:18.251 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-12-19T16:39:18.504 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-singledispatch. 2023-12-19T16:39:18.520 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-12-19T16:39:18.562 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-12-19T16:39:18.584 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-waitress. 2023-12-19T16:39:18.599 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-12-19T16:39:18.645 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-19T16:39:18.823 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-webob. 2023-12-19T16:39:18.839 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-12-19T16:39:18.881 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-12-19T16:39:18.937 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-tempita. 2023-12-19T16:39:18.951 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-12-19T16:39:18.998 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-12-19T16:39:19.243 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-waitress. 2023-12-19T16:39:19.259 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-12-19T16:39:19.323 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-paste. 2023-12-19T16:39:19.329 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-19T16:39:19.338 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-12-19T16:39:19.381 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-12-19T16:39:19.637 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-tempita. 2023-12-19T16:39:19.653 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-12-19T16:39:19.696 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-12-19T16:39:19.701 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-12-19T16:39:19.715 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-12-19T16:39:19.759 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-12-19T16:39:19.991 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-paste. 2023-12-19T16:39:20.006 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-12-19T16:39:20.040 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-12-19T16:39:20.087 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pastedeploy. 2023-12-19T16:39:20.102 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-12-19T16:39:20.145 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-12-19T16:39:20.352 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-12-19T16:39:20.368 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-12-19T16:39:20.410 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-12-19T16:39:20.414 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-webtest. 2023-12-19T16:39:20.430 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-12-19T16:39:20.464 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-12-19T16:39:20.730 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pastedeploy. 2023-12-19T16:39:20.746 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-12-19T16:39:20.784 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pecan. 2023-12-19T16:39:20.788 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-12-19T16:39:20.799 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-12-19T16:39:20.842 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-12-19T16:39:21.041 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-webtest. 2023-12-19T16:39:21.057 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-12-19T16:39:21.099 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-12-19T16:39:21.188 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-certifi. 2023-12-19T16:39:21.203 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-12-19T16:39:21.246 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-12-19T16:39:21.535 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pecan. 2023-12-19T16:39:21.551 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-12-19T16:39:21.591 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-idna. 2023-12-19T16:39:21.593 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-12-19T16:39:21.606 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-12-19T16:39:21.649 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-idna (3.3-1) ... 2023-12-19T16:39:21.956 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-certifi. 2023-12-19T16:39:21.960 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-urllib3. 2023-12-19T16:39:21.972 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-12-19T16:39:21.975 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-12-19T16:39:22.035 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-19T16:39:22.123 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-12-19T16:39:22.972 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-requests. 2023-12-19T16:39:22.987 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-12-19T16:39:23.021 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-19T16:39:23.035 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-idna. 2023-12-19T16:39:23.050 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-12-19T16:39:23.093 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-idna (3.3-1) ... 2023-12-19T16:39:23.283 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-werkzeug. 2023-12-19T16:39:23.298 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-12-19T16:39:23.333 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-19T16:39:23.437 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-urllib3. 2023-12-19T16:39:23.453 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-12-19T16:39:23.496 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-19T16:39:23.635 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-12-19T16:39:23.651 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:23.694 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mgr-modules-core (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:23.824 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-requests. 2023-12-19T16:39:23.839 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-12-19T16:39:23.882 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-19T16:39:24.065 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-12-19T16:39:24.079 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:24.153 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libsqlite3-mod-ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:24.185 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-werkzeug. 2023-12-19T16:39:24.201 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-12-19T16:39:24.235 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-19T16:39:24.417 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-openssl. 2023-12-19T16:39:24.432 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-12-19T16:39:24.467 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-12-19T16:39:24.563 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-12-19T16:39:24.579 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:24.621 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mgr-modules-core (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:24.795 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mgr. 2023-12-19T16:39:24.810 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../058-ceph-mgr_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:24.878 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mgr (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:25.026 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-12-19T16:39:25.041 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:25.084 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libsqlite3-mod-ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:25.265 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mon. 2023-12-19T16:39:25.279 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../059-ceph-mon_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:25.323 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mon (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:25.387 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-openssl. 2023-12-19T16:39:25.402 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-12-19T16:39:25.445 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-12-19T16:39:25.798 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mgr. 2023-12-19T16:39:25.813 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../058-ceph-mgr_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:25.856 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mgr (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:26.035 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-osd. 2023-12-19T16:39:26.049 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../060-ceph-osd_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:26.084 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-osd (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:26.201 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mon. 2023-12-19T16:39:26.217 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../059-ceph-mon_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:26.251 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mon (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:27.063 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-osd. 2023-12-19T16:39:27.079 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../060-ceph-osd_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:27.121 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-osd (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:27.212 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph. 2023-12-19T16:39:27.227 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../061-ceph_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:27.271 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:27.573 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-fuse. 2023-12-19T16:39:27.588 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../062-ceph-fuse_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:27.639 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:28.127 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mds. 2023-12-19T16:39:28.141 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../063-ceph-mds_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:28.176 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mds (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:28.267 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph. 2023-12-19T16:39:28.282 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../061-ceph_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:28.341 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:28.512 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package cephadm. 2023-12-19T16:39:28.527 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../064-cephadm_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:28.571 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:28.744 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-fuse. 2023-12-19T16:39:28.760 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../062-ceph-fuse_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:28.810 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:28.898 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-asyncssh. 2023-12-19T16:39:28.914 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-12-19T16:39:28.965 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-12-19T16:39:29.268 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python-babel-localedata. 2023-12-19T16:39:29.272 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mds. 2023-12-19T16:39:29.282 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-12-19T16:39:29.288 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../063-ceph-mds_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:29.326 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-19T16:39:29.338 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mds (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:29.758 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package cephadm. 2023-12-19T16:39:29.774 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../064-cephadm_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:29.825 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:30.112 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-asyncssh. 2023-12-19T16:39:30.127 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-12-19T16:39:30.169 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-12-19T16:39:30.372 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-babel. 2023-12-19T16:39:30.386 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-12-19T16:39:30.430 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-12-19T16:39:30.473 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python-babel-localedata. 2023-12-19T16:39:30.488 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-12-19T16:39:30.530 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-19T16:39:30.742 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-jinja2. 2023-12-19T16:39:30.756 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-12-19T16:39:30.825 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-12-19T16:39:31.170 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-12-19T16:39:31.184 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:31.228 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mgr-cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:31.527 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-babel. 2023-12-19T16:39:31.542 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-12-19T16:39:31.581 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-bcrypt. 2023-12-19T16:39:31.585 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-12-19T16:39:31.595 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-12-19T16:39:31.639 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-12-19T16:39:31.884 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pluggy. 2023-12-19T16:39:31.897 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-jinja2. 2023-12-19T16:39:31.897 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2023-12-19T16:39:31.911 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-12-19T16:39:31.934 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-12-19T16:39:31.955 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-12-19T16:39:32.279 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-py. 2023-12-19T16:39:32.293 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2023-12-19T16:39:32.336 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-py (1.10.0-1) ... 2023-12-19T16:39:32.517 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-12-19T16:39:32.533 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:32.575 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mgr-cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:32.665 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-iniconfig. 2023-12-19T16:39:32.679 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2023-12-19T16:39:32.723 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-12-19T16:39:32.904 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-bcrypt. 2023-12-19T16:39:32.920 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-12-19T16:39:32.962 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-12-19T16:39:33.084 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-toml. 2023-12-19T16:39:33.099 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2023-12-19T16:39:33.142 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-12-19T16:39:33.248 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pluggy. 2023-12-19T16:39:33.264 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2023-12-19T16:39:33.306 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-12-19T16:39:33.403 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pytest. 2023-12-19T16:39:33.419 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-12-19T16:39:33.453 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-12-19T16:39:33.643 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-py. 2023-12-19T16:39:33.659 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2023-12-19T16:39:33.693 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-py (1.10.0-1) ... 2023-12-19T16:39:33.832 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-repoze.lru. 2023-12-19T16:39:33.847 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2023-12-19T16:39:33.890 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-12-19T16:39:33.988 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-iniconfig. 2023-12-19T16:39:34.006 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2023-12-19T16:39:34.038 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-12-19T16:39:34.151 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-routes. 2023-12-19T16:39:34.167 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-12-19T16:39:34.201 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-12-19T16:39:34.323 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-toml. 2023-12-19T16:39:34.339 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2023-12-19T16:39:34.382 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-12-19T16:39:34.573 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-12-19T16:39:34.586 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:34.629 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mgr-dashboard (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:34.693 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pytest. 2023-12-19T16:39:34.709 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-12-19T16:39:34.777 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-12-19T16:39:35.082 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-12-19T16:39:35.097 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-12-19T16:39:35.139 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-repoze.lru. 2023-12-19T16:39:35.149 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-19T16:39:35.155 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2023-12-19T16:39:35.197 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-12-19T16:39:35.541 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-routes. 2023-12-19T16:39:35.557 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-12-19T16:39:35.599 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-12-19T16:39:35.627 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-joblib. 2023-12-19T16:39:35.642 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-12-19T16:39:35.685 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-12-19T16:39:35.919 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-12-19T16:39:35.935 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:35.978 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mgr-dashboard (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:36.047 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-12-19T16:39:36.061 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-12-19T16:39:36.105 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-12-19T16:39:36.398 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-12-19T16:39:36.413 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-12-19T16:39:36.441 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-sklearn. 2023-12-19T16:39:36.456 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-19T16:39:36.456 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-12-19T16:39:36.499 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-19T16:39:36.968 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-joblib. 2023-12-19T16:39:36.984 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-12-19T16:39:37.026 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-12-19T16:39:37.225 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-12-19T16:39:37.240 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:37.284 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:37.404 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-12-19T16:39:37.420 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-12-19T16:39:37.462 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-12-19T16:39:37.757 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-sklearn. 2023-12-19T16:39:37.773 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-12-19T16:39:37.815 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-19T16:39:38.462 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-cachetools. 2023-12-19T16:39:38.478 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2023-12-19T16:39:38.537 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-12-19T16:39:38.592 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-12-19T16:39:38.607 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:38.652 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:38.823 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pyasn1. 2023-12-19T16:39:38.839 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2023-12-19T16:39:38.873 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-12-19T16:39:39.143 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-12-19T16:39:39.159 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-12-19T16:39:39.201 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-12-19T16:39:39.512 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-rsa. 2023-12-19T16:39:39.528 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2023-12-19T16:39:39.570 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-rsa (4.8-1) ... 2023-12-19T16:39:39.606 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-cachetools. 2023-12-19T16:39:39.622 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2023-12-19T16:39:39.656 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-12-19T16:39:39.891 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-google-auth. 2023-12-19T16:39:39.908 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2023-12-19T16:39:39.949 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-12-19T16:39:39.967 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pyasn1. 2023-12-19T16:39:39.983 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2023-12-19T16:39:40.025 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-12-19T16:39:40.277 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-12-19T16:39:40.292 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-12-19T16:39:40.327 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-19T16:39:40.353 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-12-19T16:39:40.370 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-12-19T16:39:40.412 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-12-19T16:39:40.571 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-websocket. 2023-12-19T16:39:40.586 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2023-12-19T16:39:40.629 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-12-19T16:39:40.740 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-rsa. 2023-12-19T16:39:40.756 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2023-12-19T16:39:40.798 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-rsa (4.8-1) ... 2023-12-19T16:39:40.974 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-kubernetes. 2023-12-19T16:39:40.989 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-12-19T16:39:41.041 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-19T16:39:41.068 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-google-auth. 2023-12-19T16:39:41.084 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2023-12-19T16:39:41.160 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-12-19T16:39:41.471 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-12-19T16:39:41.487 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-12-19T16:39:41.529 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-19T16:39:41.755 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-12-19T16:39:41.769 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:41.804 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-mgr-k8sevents (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:41.824 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-websocket. 2023-12-19T16:39:41.840 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2023-12-19T16:39:41.882 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-12-19T16:39:42.123 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libonig5:amd64. 2023-12-19T16:39:42.137 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-12-19T16:39:42.181 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-19T16:39:42.202 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-kubernetes. 2023-12-19T16:39:42.218 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-12-19T16:39:42.260 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-19T16:39:42.534 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libjq1:amd64. 2023-12-19T16:39:42.549 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-12-19T16:39:42.593 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-19T16:39:42.837 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package jq. 2023-12-19T16:39:42.851 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-12-19T16:39:42.887 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-12-19T16:39:43.082 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-12-19T16:39:43.097 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:43.114 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package socat. 2023-12-19T16:39:43.128 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-12-19T16:39:43.141 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-mgr-k8sevents (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:43.164 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-12-19T16:39:43.426 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libonig5:amd64. 2023-12-19T16:39:43.441 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-12-19T16:39:43.459 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package xmlstarlet. 2023-12-19T16:39:43.474 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-12-19T16:39:43.610 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-19T16:39:43.648 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-12-19T16:39:43.954 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-test. 2023-12-19T16:39:43.968 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../098-ceph-test_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:43.997 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libjq1:amd64. 2023-12-19T16:39:44.005 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-test (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:44.013 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-12-19T16:39:44.054 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-19T16:39:44.315 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package jq. 2023-12-19T16:39:44.331 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-12-19T16:39:44.374 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-12-19T16:39:44.710 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package socat. 2023-12-19T16:39:44.726 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-12-19T16:39:44.784 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-12-19T16:39:45.238 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package xmlstarlet. 2023-12-19T16:39:45.254 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-12-19T16:39:45.313 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-12-19T16:39:45.701 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ceph-volume. 2023-12-19T16:39:45.716 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../099-ceph-volume_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:45.758 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ceph-volume (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:45.809 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-test. 2023-12-19T16:39:45.824 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../098-ceph-test_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:45.883 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-test (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:46.162 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package cephfs-mirror. 2023-12-19T16:39:46.178 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:46.221 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking cephfs-mirror (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:46.532 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-colorama. 2023-12-19T16:39:46.548 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2023-12-19T16:39:46.590 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-12-19T16:39:46.860 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package cephfs-shell. 2023-12-19T16:39:46.877 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../102-cephfs-shell_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:46.918 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking cephfs-shell (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:47.170 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package cephfs-top. 2023-12-19T16:39:47.186 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../103-cephfs-top_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:47.228 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking cephfs-top (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:47.481 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libcephfs-dev. 2023-12-19T16:39:47.497 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:47.539 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libcephfs-dev (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:47.921 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ceph-volume. 2023-12-19T16:39:47.927 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package nvme-cli. 2023-12-19T16:39:47.937 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../099-ceph-volume_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:47.943 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-12-19T16:39:47.986 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-12-19T16:39:47.996 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ceph-volume (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:48.332 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-12-19T16:39:48.347 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-12-19T16:39:48.390 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-12-19T16:39:48.400 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package cephfs-mirror. 2023-12-19T16:39:48.416 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:48.458 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking cephfs-mirror (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:48.753 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-colorama. 2023-12-19T16:39:48.770 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2023-12-19T16:39:48.793 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pastescript. 2023-12-19T16:39:48.803 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-12-19T16:39:48.808 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2023-12-19T16:39:48.851 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-12-19T16:39:49.089 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package cephfs-shell. 2023-12-19T16:39:49.105 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../102-cephfs-shell_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:49.147 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking cephfs-shell (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:49.213 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-psutil. 2023-12-19T16:39:49.227 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-12-19T16:39:49.271 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-12-19T16:39:49.408 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package cephfs-top. 2023-12-19T16:39:49.424 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../103-cephfs-top_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-19T16:39:49.475 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking cephfs-top (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:49.590 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pygments. 2023-12-19T16:39:49.632 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-12-19T16:39:49.648 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-12-19T16:39:49.786 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libcephfs-dev. 2023-12-19T16:39:49.802 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:49.852 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libcephfs-dev (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:50.128 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-pyinotify. 2023-12-19T16:39:50.142 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-12-19T16:39:50.177 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-12-19T16:39:50.289 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package nvme-cli. 2023-12-19T16:39:50.304 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-12-19T16:39:50.347 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-12-19T16:39:50.463 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package python3-simplejson. 2023-12-19T16:39:50.477 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-12-19T16:39:50.521 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-12-19T16:39:50.677 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-12-19T16:39:50.692 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-12-19T16:39:50.735 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-12-19T16:39:50.774 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-12-19T16:39:50.789 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-12-19T16:39:50.832 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-12-19T16:39:51.163 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pastescript. 2023-12-19T16:39:51.182 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2023-12-19T16:39:51.221 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-12-19T16:39:51.550 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-psutil. 2023-12-19T16:39:51.565 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-12-19T16:39:51.608 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-12-19T16:39:51.611 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package radosgw. 2023-12-19T16:39:51.625 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../113-radosgw_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:51.669 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking radosgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:51.927 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pygments. 2023-12-19T16:39:51.943 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-12-19T16:39:51.985 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-12-19T16:39:52.222 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package rbd-fuse. 2023-12-19T16:39:52.237 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../114-rbd-fuse_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:52.280 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking rbd-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:52.439 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-pyinotify. 2023-12-19T16:39:52.455 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-12-19T16:39:52.498 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-12-19T16:39:52.565 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-12-19T16:39:52.758 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package python3-simplejson. 2023-12-19T16:39:52.774 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-12-19T16:39:52.842 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-12-19T16:39:52.877 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-19T16:39:53.003 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-12-19T16:39:53.257 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-12-19T16:39:53.344 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-12-19T16:39:53.361 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-12-19T16:39:53.394 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-12-19T16:39:53.636 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-12-19T16:39:54.123 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package radosgw. 2023-12-19T16:39:54.138 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../113-radosgw_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:54.190 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking radosgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:54.216 INFO:teuthology.orchestra.run.smithi160.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-12-19T16:39:54.289 INFO:teuthology.orchestra.run.smithi160.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-12-19T16:39:54.329 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:54.834 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package rbd-fuse. 2023-12-19T16:39:54.851 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../114-rbd-fuse_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-19T16:39:54.892 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking rbd-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:54.916 INFO:teuthology.orchestra.run.smithi160.stdout:Adding system user cephadm....done 2023-12-19T16:39:55.077 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-19T16:39:55.197 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-12-19T16:39:55.353 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-12-19T16:39:55.522 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-19T16:39:55.623 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-12-19T16:39:55.637 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-12-19T16:39:55.763 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-12-19T16:39:55.893 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-12-19T16:39:56.031 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-12-19T16:39:56.299 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-12-19T16:39:56.324 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-12-19T16:39:56.450 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-py (1.10.0-1) ... 2023-12-19T16:39:56.768 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-12-19T16:39:56.945 INFO:teuthology.orchestra.run.smithi026.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-12-19T16:39:56.993 INFO:teuthology.orchestra.run.smithi026.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-12-19T16:39:57.042 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:57.153 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-12-19T16:39:57.413 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-12-19T16:39:57.587 INFO:teuthology.orchestra.run.smithi026.stdout:Adding system user cephadm....done 2023-12-19T16:39:57.681 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-12-19T16:39:57.741 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-19T16:39:57.974 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-12-19T16:39:58.016 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-12-19T16:39:58.257 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-12-19T16:39:58.259 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-12-19T16:39:58.384 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-12-19T16:39:58.542 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-12-19T16:39:58.652 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-12-19T16:39:58.929 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-12-19T16:39:59.055 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-py (1.10.0-1) ... 2023-12-19T16:39:59.060 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-ceph-argparse (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:39:59.320 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-19T16:39:59.373 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-12-19T16:39:59.437 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-12-19T16:39:59.705 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-12-19T16:39:59.742 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-12-19T16:39:59.964 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-19T16:40:00.009 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-12-19T16:40:00.090 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-12-19T16:40:00.277 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-12-19T16:40:00.458 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-19T16:40:00.579 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-12-19T16:40:00.584 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-12-19T16:40:00.846 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-12-19T16:40:00.868 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-12-19T16:40:01.114 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-12-19T16:40:01.136 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-12-19T16:40:01.446 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-12-19T16:40:01.523 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-ceph-argparse (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:01.791 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-19T16:40:01.934 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-12-19T16:40:02.131 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-12-19T16:40:02.202 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-12-19T16:40:02.399 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-12-19T16:40:02.478 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-19T16:40:02.566 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-12-19T16:40:02.604 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-12-19T16:40:02.692 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-19T16:40:02.980 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-19T16:40:03.018 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-12-19T16:40:03.118 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-12-19T16:40:03.399 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-12-19T16:40:03.411 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-12-19T16:40:03.671 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-12-19T16:40:03.674 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-12-19T16:40:03.992 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-12-19T16:40:04.063 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-19T16:40:04.348 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-12-19T16:40:04.674 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-19T16:40:04.711 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-12-19T16:40:04.996 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-12-19T16:40:05.034 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-12-19T16:40:05.113 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-12-19T16:40:05.231 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-19T16:40:05.326 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-12-19T16:40:05.427 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:05.532 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-12-19T16:40:05.712 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-idna (3.3-1) ... 2023-12-19T16:40:05.925 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-12-19T16:40:06.063 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-19T16:40:06.193 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-12-19T16:40:06.603 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-19T16:40:06.912 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-12-19T16:40:07.260 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-19T16:40:07.263 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-19T16:40:07.396 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-toml (0.10.2-1) ... 2023-12-19T16:40:07.632 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-12-19T16:40:07.687 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-19T16:40:07.813 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-12-19T16:40:07.890 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-12-19T16:40:07.939 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-19T16:40:08.000 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:08.241 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-12-19T16:40:08.302 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-idna (3.3-1) ... 2023-12-19T16:40:08.508 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-12-19T16:40:08.776 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-19T16:40:08.869 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-19T16:40:08.944 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-12-19T16:40:09.044 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-12-19T16:40:09.362 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-12-19T16:40:09.622 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-12-19T16:40:09.881 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-12-19T16:40:10.150 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-19T16:40:10.157 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-12-19T16:40:10.284 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-toml (0.10.2-1) ... 2023-12-19T16:40:10.450 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-12-19T16:40:10.560 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-19T16:40:10.686 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-12-19T16:40:10.743 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-19T16:40:10.837 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-19T16:40:10.868 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-12-19T16:40:11.139 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-12-19T16:40:11.186 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-19T16:40:11.312 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-19T16:40:11.390 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-12-19T16:40:11.605 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-12-19T16:40:11.657 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-19T16:40:11.800 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-12-19T16:40:11.926 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-12-19T16:40:11.931 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-12-19T16:40:12.207 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-12-19T16:40:12.253 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-12-19T16:40:12.520 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-12-19T16:40:12.628 INFO:teuthology.orchestra.run.smithi160.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-12-19T16:40:12.788 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-12-19T16:40:12.893 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-19T16:40:12.994 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-12-19T16:40:13.047 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-12-19T16:40:13.307 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-12-19T16:40:13.429 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-12-19T16:40:13.608 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-19T16:40:13.697 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-portend (3.0.0-1) ... 2023-12-19T16:40:13.725 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-12-19T16:40:13.948 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-19T16:40:14.026 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-19T16:40:14.157 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-19T16:40:14.282 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-12-19T16:40:14.462 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-12-19T16:40:14.617 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-12-19T16:40:14.788 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-12-19T16:40:14.935 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-19T16:40:15.056 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-12-19T16:40:15.235 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-12-19T16:40:15.345 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-rsa (4.8-1) ... 2023-12-19T16:40:15.448 INFO:teuthology.orchestra.run.smithi026.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-12-19T16:40:15.621 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-12-19T16:40:15.701 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-19T16:40:15.826 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-12-19T16:40:15.880 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-19T16:40:16.173 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-12-19T16:40:16.237 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-12-19T16:40:16.512 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-portend (3.0.0-1) ... 2023-12-19T16:40:16.558 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-12-19T16:40:16.772 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-19T16:40:16.892 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-12-19T16:40:16.897 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-12-19T16:40:17.249 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-12-19T16:40:17.253 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-19T16:40:17.403 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up librados2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:17.513 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up librgw2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:17.551 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-19T16:40:17.707 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libsqlite3-mod-ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:17.833 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libcephfs2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:17.834 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-12-19T16:40:17.960 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-rsa (4.8-1) ... 2023-12-19T16:40:17.968 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libradosstriper1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:18.144 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-12-19T16:40:18.253 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-12-19T16:40:18.429 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up librbd1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:18.546 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-19T16:40:18.555 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mgr-modules-core (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:18.681 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:18.822 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-12-19T16:40:18.916 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-19T16:40:18.916 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-19T16:40:19.215 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-12-19T16:40:19.508 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-12-19T16:40:19.557 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libcephfs-dev (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:19.683 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-rados (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:19.793 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up cephfs-top (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:19.868 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-19T16:40:19.894 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-19T16:40:20.003 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up librados2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:20.121 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up librgw2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:20.332 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libsqlite3-mod-ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:20.458 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libcephfs2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:20.584 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libradosstriper1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:20.710 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-12-19T16:40:21.012 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up librbd1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:21.156 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-rbd (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:21.163 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mgr-modules-core (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:21.282 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up rbd-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:21.289 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:21.408 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-rgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:21.511 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-19T16:40:21.511 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-19T16:40:21.568 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up python3-cephfs (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:21.694 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up cephfs-shell (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:21.820 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:22.165 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libcephfs-dev (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:22.291 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-rados (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:22.331 INFO:teuthology.orchestra.run.smithi160.stdout:Adding group ceph....done 2023-12-19T16:40:22.451 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up cephfs-top (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:22.577 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-19T16:40:22.707 INFO:teuthology.orchestra.run.smithi160.stdout:Adding system user ceph....done 2023-12-19T16:40:22.893 INFO:teuthology.orchestra.run.smithi160.stdout:Setting system user ceph properties....done 2023-12-19T16:40:22.912 INFO:teuthology.orchestra.run.smithi160.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-12-19T16:40:23.052 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-12-19T16:40:23.422 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-12-19T16:40:23.915 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-rbd (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:24.040 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up rbd-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:24.104 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up cephfs-mirror (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:24.166 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-rgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:24.292 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up python3-cephfs (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:24.312 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-12-19T16:40:24.312 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-12-19T16:40:24.418 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up cephfs-shell (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:24.553 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:24.922 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-test (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:25.070 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up radosgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:25.132 INFO:teuthology.orchestra.run.smithi026.stdout:Adding group ceph....done 2023-12-19T16:40:25.508 INFO:teuthology.orchestra.run.smithi026.stdout:Adding system user ceph....done 2023-12-19T16:40:25.610 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-19T16:40:25.610 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-19T16:40:25.744 INFO:teuthology.orchestra.run.smithi026.stdout:Setting system user ceph properties....done 2023-12-19T16:40:25.761 INFO:teuthology.orchestra.run.smithi026.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-12-19T16:40:25.911 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-12-19T16:40:26.200 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-base (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:26.323 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-12-19T16:40:26.570 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-12-19T16:40:26.971 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up cephfs-mirror (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:27.187 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-12-19T16:40:27.187 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-12-19T16:40:27.235 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mds (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:27.466 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-19T16:40:27.466 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-19T16:40:27.806 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-test (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:27.932 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up radosgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:28.078 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mgr (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:28.292 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-19T16:40:28.292 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-19T16:40:28.462 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-19T16:40:28.462 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-19T16:40:28.871 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-osd (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:29.092 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-base (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:29.201 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-19T16:40:29.201 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-19T16:40:29.478 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-12-19T16:40:29.857 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mgr-k8sevents (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:30.004 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mgr-diskprediction-local (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:30.152 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mon (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:30.160 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mds (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:30.359 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-19T16:40:30.359 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-19T16:40:30.390 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-19T16:40:30.390 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-19T16:40:30.993 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mgr-cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:31.021 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mgr (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:31.136 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:31.242 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-19T16:40:31.242 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-19T16:40:31.262 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-mgr-dashboard (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:31.430 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ceph-volume (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:31.783 INFO:teuthology.orchestra.run.smithi160.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-12-19T16:40:31.888 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-osd (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:32.215 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-19T16:40:32.216 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-19T16:40:32.848 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mgr-k8sevents (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:32.923 INFO:teuthology.orchestra.run.smithi160.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-12-19T16:40:32.949 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mgr-diskprediction-local (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:33.084 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mon (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:33.299 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-19T16:40:33.299 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-19T16:40:33.384 INFO:teuthology.orchestra.run.smithi160.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-12-19T16:40:33.389 DEBUG:teuthology.orchestra.run.smithi160:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-12-19T16:40:33.464 INFO:teuthology.orchestra.run.smithi160.stdout:Reading package lists... 2023-12-19T16:40:33.632 INFO:teuthology.orchestra.run.smithi160.stdout:Building dependency tree... 2023-12-19T16:40:33.633 INFO:teuthology.orchestra.run.smithi160.stdout:Reading state information... 2023-12-19T16:40:33.785 INFO:teuthology.orchestra.run.smithi160.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-12-19T16:40:33.786 INFO:teuthology.orchestra.run.smithi160.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-12-19T16:40:33.786 INFO:teuthology.orchestra.run.smithi160.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-12-19T16:40:33.786 INFO:teuthology.orchestra.run.smithi160.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-12-19T16:40:33.786 INFO:teuthology.orchestra.run.smithi160.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-12-19T16:40:33.786 INFO:teuthology.orchestra.run.smithi160.stdout:The following packages were automatically installed and are no longer required: 2023-12-19T16:40:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-19T16:40:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-19T16:40:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-19T16:40:33.786 INFO:teuthology.orchestra.run.smithi160.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-19T16:40:33.787 INFO:teuthology.orchestra.run.smithi160.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-19T16:40:33.787 INFO:teuthology.orchestra.run.smithi160.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-19T16:40:33.788 INFO:teuthology.orchestra.run.smithi160.stdout:The following additional packages will be installed: 2023-12-19T16:40:33.788 INFO:teuthology.orchestra.run.smithi160.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-12-19T16:40:33.788 INFO:teuthology.orchestra.run.smithi160.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-12-19T16:40:33.788 INFO:teuthology.orchestra.run.smithi160.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-12-19T16:40:33.788 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-12-19T16:40:33.788 INFO:teuthology.orchestra.run.smithi160.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-12-19T16:40:33.788 INFO:teuthology.orchestra.run.smithi160.stdout:Suggested packages: 2023-12-19T16:40:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-12-19T16:40:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-12-19T16:40:33.789 INFO:teuthology.orchestra.run.smithi160.stdout: libengine-pkcs11-openssl 2023-12-19T16:40:33.838 INFO:teuthology.orchestra.run.smithi160.stdout:The following NEW packages will be installed: 2023-12-19T16:40:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-12-19T16:40:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-12-19T16:40:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-12-19T16:40:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-12-19T16:40:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-12-19T16:40:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-12-19T16:40:33.839 INFO:teuthology.orchestra.run.smithi160.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-12-19T16:40:33.840 INFO:teuthology.orchestra.run.smithi160.stdout:The following packages will be upgraded: 2023-12-19T16:40:33.840 INFO:teuthology.orchestra.run.smithi160.stdout: libssl3 2023-12-19T16:40:33.901 INFO:teuthology.orchestra.run.smithi160.stdout:1 upgraded, 39 newly installed, 0 to remove and 267 not upgraded. 2023-12-19T16:40:33.901 INFO:teuthology.orchestra.run.smithi160.stdout:Need to get 54.6 MB of archives. 2023-12-19T16:40:33.902 INFO:teuthology.orchestra.run.smithi160.stdout:After this operation, 202 MB of additional disk space will be used. 2023-12-19T16:40:33.902 INFO:teuthology.orchestra.run.smithi160.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-12-19T16:40:33.935 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mgr-cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:34.046 INFO:teuthology.orchestra.run.smithi160.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-12-19T16:40:34.096 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:34.121 INFO:teuthology.orchestra.run.smithi160.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-12-19T16:40:34.132 INFO:teuthology.orchestra.run.smithi160.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-12-19T16:40:34.133 INFO:teuthology.orchestra.run.smithi160.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-12-19T16:40:34.134 INFO:teuthology.orchestra.run.smithi160.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-12-19T16:40:34.136 INFO:teuthology.orchestra.run.smithi160.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-12-19T16:40:34.137 INFO:teuthology.orchestra.run.smithi160.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2023-12-19T16:40:34.138 INFO:teuthology.orchestra.run.smithi160.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-12-19T16:40:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-12-19T16:40:34.139 INFO:teuthology.orchestra.run.smithi160.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-12-19T16:40:34.148 INFO:teuthology.orchestra.run.smithi160.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-12-19T16:40:34.160 INFO:teuthology.orchestra.run.smithi160.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-12-19T16:40:34.307 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-mgr-dashboard (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:34.441 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ceph-volume (18.2.1-132-g568df92b-1jammy) ... 2023-12-19T16:40:34.539 INFO:teuthology.orchestra.run.smithi160.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-12-19T16:40:34.541 INFO:teuthology.orchestra.run.smithi160.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-12-19T16:40:34.541 INFO:teuthology.orchestra.run.smithi160.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-12-19T16:40:34.544 INFO:teuthology.orchestra.run.smithi160.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-12-19T16:40:34.544 INFO:teuthology.orchestra.run.smithi160.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-12-19T16:40:34.548 INFO:teuthology.orchestra.run.smithi160.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.10-0ubuntu0.22.04.1 [143 kB] 2023-12-19T16:40:34.550 INFO:teuthology.orchestra.run.smithi160.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-12-19T16:40:34.558 INFO:teuthology.orchestra.run.smithi160.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-12-19T16:40:34.558 INFO:teuthology.orchestra.run.smithi160.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-12-19T16:40:34.600 INFO:teuthology.orchestra.run.smithi160.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-12-19T16:40:34.603 INFO:teuthology.orchestra.run.smithi160.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-12-19T16:40:34.617 INFO:teuthology.orchestra.run.smithi160.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-12-19T16:40:34.652 INFO:teuthology.orchestra.run.smithi160.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-12-19T16:40:34.669 INFO:teuthology.orchestra.run.smithi160.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-12-19T16:40:34.670 INFO:teuthology.orchestra.run.smithi160.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-12-19T16:40:34.670 INFO:teuthology.orchestra.run.smithi160.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-12-19T16:40:34.671 INFO:teuthology.orchestra.run.smithi160.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.10-0ubuntu0.22.04.1 [1,223 kB] 2023-12-19T16:40:34.743 INFO:teuthology.orchestra.run.smithi160.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-12-19T16:40:34.744 INFO:teuthology.orchestra.run.smithi160.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.10-0ubuntu0.22.04.1 [16.1 MB] 2023-12-19T16:40:34.803 INFO:teuthology.orchestra.run.smithi026.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-12-19T16:40:34.942 INFO:teuthology.orchestra.run.smithi160.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-12-19T16:40:34.942 INFO:teuthology.orchestra.run.smithi160.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-12-19T16:40:34.942 INFO:teuthology.orchestra.run.smithi160.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-12-19T16:40:34.942 INFO:teuthology.orchestra.run.smithi160.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-12-19T16:40:34.942 INFO:teuthology.orchestra.run.smithi160.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-12-19T16:40:34.942 INFO:teuthology.orchestra.run.smithi160.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-12-19T16:40:34.955 INFO:teuthology.orchestra.run.smithi160.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-12-19T16:40:34.956 INFO:teuthology.orchestra.run.smithi160.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-12-19T16:40:35.334 INFO:teuthology.orchestra.run.smithi160.stdout:Preconfiguring packages ... 2023-12-19T16:40:35.481 INFO:teuthology.orchestra.run.smithi160.stdout:Fetched 54.6 MB in 1s (49.3 MB/s) 2023-12-19T16:40:35.647 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package flex. 2023-12-19T16:40:35.720 INFO:teuthology.orchestra.run.smithi160.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139847 files and directories currently installed.) 2023-12-19T16:40:35.724 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-12-19T16:40:35.771 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking flex (2.6.4-8build2) ... 2023-12-19T16:40:36.212 INFO:teuthology.orchestra.run.smithi026.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-12-19T16:40:36.366 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-12-19T16:40:36.518 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-12-19T16:40:36.687 INFO:teuthology.orchestra.run.smithi026.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-12-19T16:40:36.692 DEBUG:teuthology.orchestra.run.smithi026:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-12-19T16:40:36.769 INFO:teuthology.orchestra.run.smithi026.stdout:Reading package lists... 2023-12-19T16:40:36.856 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-19T16:40:36.941 INFO:teuthology.orchestra.run.smithi026.stdout:Building dependency tree... 2023-12-19T16:40:36.942 INFO:teuthology.orchestra.run.smithi026.stdout:Reading state information... 2023-12-19T16:40:37.101 INFO:teuthology.orchestra.run.smithi026.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-12-19T16:40:37.101 INFO:teuthology.orchestra.run.smithi026.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-12-19T16:40:37.101 INFO:teuthology.orchestra.run.smithi026.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-12-19T16:40:37.101 INFO:teuthology.orchestra.run.smithi026.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-12-19T16:40:37.101 INFO:teuthology.orchestra.run.smithi026.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-12-19T16:40:37.101 INFO:teuthology.orchestra.run.smithi026.stdout:The following packages were automatically installed and are no longer required: 2023-12-19T16:40:37.101 INFO:teuthology.orchestra.run.smithi026.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-19T16:40:37.101 INFO:teuthology.orchestra.run.smithi026.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-19T16:40:37.101 INFO:teuthology.orchestra.run.smithi026.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-19T16:40:37.101 INFO:teuthology.orchestra.run.smithi026.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-19T16:40:37.103 INFO:teuthology.orchestra.run.smithi026.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-19T16:40:37.103 INFO:teuthology.orchestra.run.smithi026.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-19T16:40:37.103 INFO:teuthology.orchestra.run.smithi026.stdout:The following additional packages will be installed: 2023-12-19T16:40:37.103 INFO:teuthology.orchestra.run.smithi026.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-12-19T16:40:37.103 INFO:teuthology.orchestra.run.smithi026.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-12-19T16:40:37.103 INFO:teuthology.orchestra.run.smithi026.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-12-19T16:40:37.103 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-12-19T16:40:37.103 INFO:teuthology.orchestra.run.smithi026.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-12-19T16:40:37.104 INFO:teuthology.orchestra.run.smithi026.stdout:Suggested packages: 2023-12-19T16:40:37.104 INFO:teuthology.orchestra.run.smithi026.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-12-19T16:40:37.104 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-12-19T16:40:37.104 INFO:teuthology.orchestra.run.smithi026.stdout: libengine-pkcs11-openssl 2023-12-19T16:40:37.156 INFO:teuthology.orchestra.run.smithi026.stdout:The following NEW packages will be installed: 2023-12-19T16:40:37.156 INFO:teuthology.orchestra.run.smithi026.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-12-19T16:40:37.157 INFO:teuthology.orchestra.run.smithi026.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-12-19T16:40:37.157 INFO:teuthology.orchestra.run.smithi026.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-12-19T16:40:37.157 INFO:teuthology.orchestra.run.smithi026.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-12-19T16:40:37.157 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-12-19T16:40:37.157 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-12-19T16:40:37.157 INFO:teuthology.orchestra.run.smithi026.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-12-19T16:40:37.158 INFO:teuthology.orchestra.run.smithi026.stdout:The following packages will be upgraded: 2023-12-19T16:40:37.159 INFO:teuthology.orchestra.run.smithi026.stdout: libssl3 2023-12-19T16:40:37.345 INFO:teuthology.orchestra.run.smithi026.stdout:1 upgraded, 39 newly installed, 0 to remove and 267 not upgraded. 2023-12-19T16:40:37.345 INFO:teuthology.orchestra.run.smithi026.stdout:Need to get 54.6 MB of archives. 2023-12-19T16:40:37.345 INFO:teuthology.orchestra.run.smithi026.stdout:After this operation, 202 MB of additional disk space will be used. 2023-12-19T16:40:37.345 INFO:teuthology.orchestra.run.smithi026.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-12-19T16:40:37.394 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package bison. 2023-12-19T16:40:37.466 INFO:teuthology.orchestra.run.smithi160.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139920 files and directories currently installed.) 2023-12-19T16:40:37.470 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-12-19T16:40:37.513 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-12-19T16:40:37.846 INFO:teuthology.orchestra.run.smithi026.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-12-19T16:40:37.955 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-12-19T16:40:37.969 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-12-19T16:40:38.013 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-12-19T16:40:38.103 INFO:teuthology.orchestra.run.smithi026.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-12-19T16:40:38.138 INFO:teuthology.orchestra.run.smithi026.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-12-19T16:40:38.140 INFO:teuthology.orchestra.run.smithi026.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-12-19T16:40:38.145 INFO:teuthology.orchestra.run.smithi026.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-12-19T16:40:38.149 INFO:teuthology.orchestra.run.smithi026.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-12-19T16:40:38.149 INFO:teuthology.orchestra.run.smithi026.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2023-12-19T16:40:38.153 INFO:teuthology.orchestra.run.smithi026.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-12-19T16:40:38.155 INFO:teuthology.orchestra.run.smithi026.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-12-19T16:40:38.308 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package dump. 2023-12-19T16:40:38.319 INFO:teuthology.orchestra.run.smithi026.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-12-19T16:40:38.320 INFO:teuthology.orchestra.run.smithi026.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-12-19T16:40:38.322 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-12-19T16:40:38.358 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking dump (0.4b47-3) ... 2023-12-19T16:40:38.578 INFO:teuthology.orchestra.run.smithi026.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-12-19T16:40:38.653 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package indent. 2023-12-19T16:40:38.667 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-12-19T16:40:38.711 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-12-19T16:40:38.947 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-12-19T16:40:38.961 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-12-19T16:40:39.005 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-12-19T16:40:39.434 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-12-19T16:40:39.449 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2023-12-19T16:40:39.492 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2023-12-19T16:40:39.608 INFO:teuthology.orchestra.run.smithi026.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-12-19T16:40:39.610 INFO:teuthology.orchestra.run.smithi026.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-12-19T16:40:39.610 INFO:teuthology.orchestra.run.smithi026.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-12-19T16:40:39.613 INFO:teuthology.orchestra.run.smithi026.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-12-19T16:40:39.613 INFO:teuthology.orchestra.run.smithi026.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-12-19T16:40:39.618 INFO:teuthology.orchestra.run.smithi026.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.10-0ubuntu0.22.04.1 [143 kB] 2023-12-19T16:40:39.619 INFO:teuthology.orchestra.run.smithi026.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-12-19T16:40:39.641 INFO:teuthology.orchestra.run.smithi026.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-12-19T16:40:39.728 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-12-19T16:40:39.743 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-12-19T16:40:39.786 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-12-19T16:40:39.805 INFO:teuthology.orchestra.run.smithi026.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-12-19T16:40:40.274 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libfl2:amd64. 2023-12-19T16:40:40.289 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-12-19T16:40:40.332 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-12-19T16:40:40.545 INFO:teuthology.orchestra.run.smithi026.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-12-19T16:40:40.545 INFO:teuthology.orchestra.run.smithi026.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-12-19T16:40:40.572 INFO:teuthology.orchestra.run.smithi026.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-12-19T16:40:40.643 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-12-19T16:40:40.649 INFO:teuthology.orchestra.run.smithi026.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-12-19T16:40:40.657 INFO:teuthology.orchestra.run.smithi026.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-12-19T16:40:40.658 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-12-19T16:40:40.659 INFO:teuthology.orchestra.run.smithi026.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-12-19T16:40:40.659 INFO:teuthology.orchestra.run.smithi026.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-12-19T16:40:40.660 INFO:teuthology.orchestra.run.smithi026.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.10-0ubuntu0.22.04.1 [1,223 kB] 2023-12-19T16:40:40.688 INFO:teuthology.orchestra.run.smithi026.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-12-19T16:40:40.691 INFO:teuthology.orchestra.run.smithi026.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.10-0ubuntu0.22.04.1 [16.1 MB] 2023-12-19T16:40:40.701 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-12-19T16:40:40.912 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libjson-perl. 2023-12-19T16:40:40.928 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-12-19T16:40:40.971 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-12-19T16:40:41.297 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libllvm14:amd64. 2023-12-19T16:40:41.312 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-12-19T16:40:41.318 INFO:teuthology.orchestra.run.smithi026.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-12-19T16:40:41.357 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-12-19T16:40:41.400 INFO:teuthology.orchestra.run.smithi026.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-12-19T16:40:41.401 INFO:teuthology.orchestra.run.smithi026.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-12-19T16:40:41.401 INFO:teuthology.orchestra.run.smithi026.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-12-19T16:40:41.493 INFO:teuthology.orchestra.run.smithi026.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-12-19T16:40:41.508 INFO:teuthology.orchestra.run.smithi026.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-12-19T16:40:41.959 INFO:teuthology.orchestra.run.smithi026.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-12-19T16:40:41.961 INFO:teuthology.orchestra.run.smithi026.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-12-19T16:40:42.373 INFO:teuthology.orchestra.run.smithi026.stdout:Preconfiguring packages ... 2023-12-19T16:40:42.529 INFO:teuthology.orchestra.run.smithi026.stdout:Fetched 54.6 MB in 5s (11.4 MB/s) 2023-12-19T16:40:42.686 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package flex. 2023-12-19T16:40:42.745 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-12-19T16:40:42.759 INFO:teuthology.orchestra.run.smithi026.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139847 files and directories currently installed.) 2023-12-19T16:40:42.760 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-12-19T16:40:42.763 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-12-19T16:40:42.795 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-12-19T16:40:42.809 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking flex (2.6.4-8build2) ... 2023-12-19T16:40:43.047 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libmbim-proxy. 2023-12-19T16:40:43.063 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-12-19T16:40:43.139 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-12-19T16:40:43.458 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-12-19T16:40:43.471 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-12-19T16:40:43.473 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-12-19T16:40:43.516 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-12-19T16:40:43.674 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-12-19T16:40:43.894 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libndp0:amd64. 2023-12-19T16:40:43.910 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-12-19T16:40:43.961 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-12-19T16:40:44.089 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-19T16:40:44.247 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libnm0:amd64. 2023-12-19T16:40:44.262 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-12-19T16:40:44.305 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-12-19T16:40:44.650 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libpq5:amd64. 2023-12-19T16:40:44.665 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../16-libpq5_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-19T16:40:44.707 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2023-12-19T16:40:45.077 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-12-19T16:40:45.093 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-12-19T16:40:45.134 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package bison. 2023-12-19T16:40:45.136 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-12-19T16:40:45.213 INFO:teuthology.orchestra.run.smithi026.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139920 files and directories currently installed.) 2023-12-19T16:40:45.216 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-12-19T16:40:45.278 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-12-19T16:40:45.455 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libqmi-proxy. 2023-12-19T16:40:45.470 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-12-19T16:40:45.514 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-12-19T16:40:45.730 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-12-19T16:40:45.744 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-12-19T16:40:45.782 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-12-19T16:40:45.787 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-12-19T16:40:45.798 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-12-19T16:40:45.840 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-19T16:40:46.124 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package dump. 2023-12-19T16:40:46.139 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-12-19T16:40:46.182 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking dump (0.4b47-3) ... 2023-12-19T16:40:46.319 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-12-19T16:40:46.334 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-12-19T16:40:46.369 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-12-19T16:40:46.452 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package indent. 2023-12-19T16:40:46.466 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-12-19T16:40:46.502 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-12-19T16:40:46.739 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package modemmanager. 2023-12-19T16:40:46.746 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-12-19T16:40:46.754 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-12-19T16:40:46.760 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-12-19T16:40:46.797 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-12-19T16:40:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-12-19T16:40:47.141 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-12-19T16:40:47.156 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2023-12-19T16:40:47.200 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2023-12-19T16:40:47.444 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package network-manager. 2023-12-19T16:40:47.452 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-12-19T16:40:47.459 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-12-19T16:40:47.467 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-12-19T16:40:47.503 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-12-19T16:40:47.510 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-12-19T16:40:47.813 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libfl2:amd64. 2023-12-19T16:40:47.828 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-12-19T16:40:47.863 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-12-19T16:40:48.101 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package ppp. 2023-12-19T16:40:48.116 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-12-19T16:40:48.141 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-12-19T16:40:48.156 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-12-19T16:40:48.200 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-12-19T16:40:48.466 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-12-19T16:40:48.468 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libjson-perl. 2023-12-19T16:40:48.484 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-12-19T16:40:48.527 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-12-19T16:40:48.822 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package pptp-linux. 2023-12-19T16:40:48.836 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-12-19T16:40:48.838 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libllvm14:amd64. 2023-12-19T16:40:48.854 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-12-19T16:40:48.880 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-12-19T16:40:48.896 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-12-19T16:40:49.125 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package network-manager-pptp. 2023-12-19T16:40:49.139 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-12-19T16:40:49.175 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-12-19T16:40:49.444 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package postgresql-client-common. 2023-12-19T16:40:49.458 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-12-19T16:40:49.519 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking postgresql-client-common (238) ... 2023-12-19T16:40:49.840 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package postgresql-client-14. 2023-12-19T16:40:49.854 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../27-postgresql-client-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-19T16:40:49.898 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-19T16:40:50.292 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-12-19T16:40:50.307 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-12-19T16:40:50.350 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-12-19T16:40:50.512 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package postgresql-common. 2023-12-19T16:40:50.526 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-12-19T16:40:50.588 INFO:teuthology.orchestra.run.smithi160.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-12-19T16:40:50.611 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libmbim-proxy. 2023-12-19T16:40:50.618 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking postgresql-common (238) ... 2023-12-19T16:40:50.626 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-12-19T16:40:50.669 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-12-19T16:40:50.997 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-12-19T16:40:51.013 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-12-19T16:40:51.055 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-12-19T16:40:51.119 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package postgresql-14. 2023-12-19T16:40:51.133 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../29-postgresql-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-19T16:40:51.179 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-19T16:40:51.366 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libndp0:amd64. 2023-12-19T16:40:51.382 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-12-19T16:40:51.416 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-12-19T16:40:51.744 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libnm0:amd64. 2023-12-19T16:40:51.760 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-12-19T16:40:51.802 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-12-19T16:40:52.147 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libpq5:amd64. 2023-12-19T16:40:52.163 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../16-libpq5_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-19T16:40:52.205 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2023-12-19T16:40:52.330 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package postgresql. 2023-12-19T16:40:52.345 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-12-19T16:40:52.380 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking postgresql (14+238) ... 2023-12-19T16:40:52.524 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-12-19T16:40:52.540 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-12-19T16:40:52.599 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-12-19T16:40:52.615 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package postgresql-client. 2023-12-19T16:40:52.629 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-12-19T16:40:52.673 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking postgresql-client (14+238) ... 2023-12-19T16:40:52.868 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libqmi-proxy. 2023-12-19T16:40:52.884 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-12-19T16:40:52.917 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package postgresql-contrib. 2023-12-19T16:40:52.926 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-12-19T16:40:52.932 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-12-19T16:40:52.975 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking postgresql-contrib (14+238) ... 2023-12-19T16:40:53.171 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-12-19T16:40:53.187 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-12-19T16:40:53.229 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-19T16:40:53.320 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package tcl8.6. 2023-12-19T16:40:53.334 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-12-19T16:40:53.378 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-12-19T16:40:53.647 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package tcl. 2023-12-19T16:40:53.662 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-12-19T16:40:53.736 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-12-19T16:40:53.740 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-12-19T16:40:53.756 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-12-19T16:40:53.799 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-12-19T16:40:54.058 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package wpasupplicant. 2023-12-19T16:40:54.073 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-12-19T16:40:54.116 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-12-19T16:40:54.185 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package modemmanager. 2023-12-19T16:40:54.201 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-12-19T16:40:54.243 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-12-19T16:40:54.472 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-12-19T16:40:54.486 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-12-19T16:40:54.529 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-12-19T16:40:54.841 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package network-manager. 2023-12-19T16:40:54.857 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-12-19T16:40:54.899 INFO:teuthology.orchestra.run.smithi160.stdout:Selecting previously unselected package usb-modeswitch. 2023-12-19T16:40:54.908 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-12-19T16:40:54.914 INFO:teuthology.orchestra.run.smithi160.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-12-19T16:40:54.957 INFO:teuthology.orchestra.run.smithi160.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-12-19T16:40:55.249 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up postgresql-client-common (238) ... 2023-12-19T16:40:55.463 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up flex (2.6.4-8build2) ... 2023-12-19T16:40:55.580 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-12-19T16:40:55.658 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package ppp. 2023-12-19T16:40:55.674 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-12-19T16:40:55.690 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2023-12-19T16:40:55.823 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-12-19T16:40:55.950 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-12-19T16:40:56.027 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-12-19T16:40:56.220 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-19T16:40:56.355 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package pptp-linux. 2023-12-19T16:40:56.369 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-12-19T16:40:56.412 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-12-19T16:40:56.682 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package network-manager-pptp. 2023-12-19T16:40:56.696 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-12-19T16:40:56.732 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-12-19T16:40:56.985 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package postgresql-client-common. 2023-12-19T16:40:56.999 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-12-19T16:40:57.043 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking postgresql-client-common (238) ... 2023-12-19T16:40:57.235 INFO:teuthology.orchestra.run.smithi160.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-12-19T16:40:57.333 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-12-19T16:40:57.505 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package postgresql-client-14. 2023-12-19T16:40:57.520 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../27-postgresql-client-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-19T16:40:57.580 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-19T16:40:57.711 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-12-19T16:40:57.711 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-12-19T16:40:58.387 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-12-19T16:40:58.435 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package postgresql-common. 2023-12-19T16:40:58.450 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-12-19T16:40:58.520 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-19T16:40:58.528 INFO:teuthology.orchestra.run.smithi026.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-12-19T16:40:58.550 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking postgresql-common (238) ... 2023-12-19T16:40:58.673 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-12-19T16:40:58.789 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-12-19T16:40:58.898 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-12-19T16:40:58.999 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-12-19T16:40:59.025 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package postgresql-14. 2023-12-19T16:40:59.041 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../29-postgresql-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-19T16:40:59.103 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-19T16:40:59.125 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-12-19T16:40:59.864 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-12-19T16:40:59.989 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up dump (0.4b47-3) ... 2023-12-19T16:41:00.109 INFO:teuthology.orchestra.run.smithi160.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-12-19T16:41:00.174 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-12-19T16:41:00.300 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2023-12-19T16:41:00.387 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package postgresql. 2023-12-19T16:41:00.402 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-12-19T16:41:00.426 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up tcl (8.6.11+1build2) ... 2023-12-19T16:41:00.445 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking postgresql (14+238) ... 2023-12-19T16:41:00.544 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libjson-perl (4.04000-1) ... 2023-12-19T16:41:00.645 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-12-19T16:41:00.689 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package postgresql-client. 2023-12-19T16:41:00.704 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-12-19T16:41:00.743 INFO:teuthology.orchestra.run.smithi160.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-12-19T16:41:00.747 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking postgresql-client (14+238) ... 2023-12-19T16:41:00.846 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-12-19T16:41:00.972 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-12-19T16:41:00.991 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package postgresql-contrib. 2023-12-19T16:41:01.006 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-12-19T16:41:01.049 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking postgresql-contrib (14+238) ... 2023-12-19T16:41:01.310 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package tcl8.6. 2023-12-19T16:41:01.326 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-12-19T16:41:01.368 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-12-19T16:41:01.574 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-12-19T16:41:01.688 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package tcl. 2023-12-19T16:41:01.704 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-12-19T16:41:01.716 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-12-19T16:41:01.751 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-12-19T16:41:01.842 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-12-19T16:41:02.002 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up postgresql-client (14+238) ... 2023-12-19T16:41:02.082 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package wpasupplicant. 2023-12-19T16:41:02.099 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-12-19T16:41:02.111 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-12-19T16:41:02.141 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-12-19T16:41:02.478 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-12-19T16:41:02.495 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-12-19T16:41:02.528 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-12-19T16:41:02.712 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:02.712 INFO:teuthology.orchestra.run.smithi160.stdout:The following network interfaces were found in /etc/network/interfaces 2023-12-19T16:41:02.712 INFO:teuthology.orchestra.run.smithi160.stdout:which means they are currently configured by ifupdown: 2023-12-19T16:41:02.712 INFO:teuthology.orchestra.run.smithi160.stdout:- enp3s0f1 2023-12-19T16:41:02.712 INFO:teuthology.orchestra.run.smithi160.stdout:If you want to manage those interfaces with NetworkManager instead 2023-12-19T16:41:02.712 INFO:teuthology.orchestra.run.smithi160.stdout:remove their configuration from /etc/network/interfaces. 2023-12-19T16:41:02.712 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:02.844 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-12-19T16:41:02.931 INFO:teuthology.orchestra.run.smithi026.stdout:Selecting previously unselected package usb-modeswitch. 2023-12-19T16:41:02.948 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-12-19T16:41:02.999 INFO:teuthology.orchestra.run.smithi026.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-12-19T16:41:03.234 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-12-19T16:41:03.283 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up postgresql-client-common (238) ... 2023-12-19T16:41:03.537 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up flex (2.6.4-8build2) ... 2023-12-19T16:41:03.670 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-12-19T16:41:03.705 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-12-19T16:41:03.856 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2023-12-19T16:41:04.007 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-12-19T16:41:04.150 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-12-19T16:41:04.309 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-19T16:41:04.414 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-12-19T16:41:04.540 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-12-19T16:41:04.657 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-12-19T16:41:04.767 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up postgresql-common (238) ... 2023-12-19T16:41:05.339 INFO:teuthology.orchestra.run.smithi026.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-12-19T16:41:05.457 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-12-19T16:41:05.668 INFO:teuthology.orchestra.run.smithi160.stdout:Adding user postgres to group ssl-cert 2023-12-19T16:41:05.919 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-12-19T16:41:05.919 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-12-19T16:41:05.957 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:05.957 INFO:teuthology.orchestra.run.smithi160.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-12-19T16:41:06.213 INFO:teuthology.orchestra.run.smithi160.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-12-19T16:41:06.213 INFO:teuthology.orchestra.run.smithi160.stdout:Removing obsolete dictionary files: 2023-12-19T16:41:06.710 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-12-19T16:41:06.886 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-19T16:41:06.937 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-12-19T16:41:07.037 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-12-19T16:41:07.249 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-12-19T16:41:07.393 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-19T16:41:07.448 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-12-19T16:41:07.599 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-12-19T16:41:07.733 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-12-19T16:41:07.758 INFO:teuthology.orchestra.run.smithi160.stdout:Creating new PostgreSQL cluster 14/main ... 2023-12-19T16:41:07.775 INFO:teuthology.orchestra.run.smithi160.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-12-19T16:41:07.817 INFO:teuthology.orchestra.run.smithi160.stdout:The files belonging to this database system will be owned by user "postgres". 2023-12-19T16:41:07.818 INFO:teuthology.orchestra.run.smithi160.stdout:This user must also own the server process. 2023-12-19T16:41:07.818 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:07.818 INFO:teuthology.orchestra.run.smithi160.stdout:The database cluster will be initialized with locale "en_US". 2023-12-19T16:41:07.818 INFO:teuthology.orchestra.run.smithi160.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-12-19T16:41:07.818 INFO:teuthology.orchestra.run.smithi160.stdout:The default text search configuration will be set to "english". 2023-12-19T16:41:07.818 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:07.818 INFO:teuthology.orchestra.run.smithi160.stdout:Data page checksums are disabled. 2023-12-19T16:41:07.818 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:07.818 INFO:teuthology.orchestra.run.smithi160.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-12-19T16:41:07.819 INFO:teuthology.orchestra.run.smithi160.stdout:creating subdirectories ... ok 2023-12-19T16:41:07.819 INFO:teuthology.orchestra.run.smithi160.stdout:selecting dynamic shared memory implementation ... posix 2023-12-19T16:41:07.843 INFO:teuthology.orchestra.run.smithi160.stdout:selecting default max_connections ... 100 2023-12-19T16:41:07.879 INFO:teuthology.orchestra.run.smithi160.stdout:selecting default shared_buffers ... 128MB 2023-12-19T16:41:07.891 INFO:teuthology.orchestra.run.smithi160.stdout:selecting default time zone ... Etc/UTC 2023-12-19T16:41:07.892 INFO:teuthology.orchestra.run.smithi160.stdout:creating configuration files ... ok 2023-12-19T16:41:08.060 INFO:teuthology.orchestra.run.smithi160.stdout:running bootstrap script ... ok 2023-12-19T16:41:08.514 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-12-19T16:41:08.655 INFO:teuthology.orchestra.run.smithi160.stdout:performing post-bootstrap initialization ... ok 2023-12-19T16:41:08.665 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up dump (0.4b47-3) ... 2023-12-19T16:41:08.777 INFO:teuthology.orchestra.run.smithi026.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-12-19T16:41:08.850 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-12-19T16:41:08.976 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2023-12-19T16:41:09.102 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up tcl (8.6.11+1build2) ... 2023-12-19T16:41:09.154 INFO:teuthology.orchestra.run.smithi160.stdout:syncing data to disk ... ok 2023-12-19T16:41:09.228 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libjson-perl (4.04000-1) ... 2023-12-19T16:41:09.379 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-12-19T16:41:09.477 INFO:teuthology.orchestra.run.smithi026.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-12-19T16:41:09.547 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-12-19T16:41:09.656 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-12-19T16:41:09.660 INFO:teuthology.orchestra.run.smithi160.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-12-19T16:41:09.816 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-12-19T16:41:09.942 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-12-19T16:41:10.068 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-12-19T16:41:10.236 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up postgresql-client (14+238) ... 2023-12-19T16:41:10.737 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-12-19T16:41:11.353 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:11.353 INFO:teuthology.orchestra.run.smithi026.stdout:The following network interfaces were found in /etc/network/interfaces 2023-12-19T16:41:11.353 INFO:teuthology.orchestra.run.smithi026.stdout:which means they are currently configured by ifupdown: 2023-12-19T16:41:11.353 INFO:teuthology.orchestra.run.smithi026.stdout:- ens1f0 2023-12-19T16:41:11.353 INFO:teuthology.orchestra.run.smithi026.stdout:If you want to manage those interfaces with NetworkManager instead 2023-12-19T16:41:11.353 INFO:teuthology.orchestra.run.smithi026.stdout:remove their configuration from /etc/network/interfaces. 2023-12-19T16:41:11.353 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:11.493 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-12-19T16:41:11.912 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-12-19T16:41:12.324 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-12-19T16:41:12.726 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-12-19T16:41:12.868 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up postgresql-contrib (14+238) ... 2023-12-19T16:41:12.969 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up postgresql (14+238) ... 2023-12-19T16:41:13.032 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-12-19T16:41:13.095 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-12-19T16:41:13.175 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-12-19T16:41:13.221 INFO:teuthology.orchestra.run.smithi160.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-12-19T16:41:13.326 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-12-19T16:41:13.452 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up postgresql-common (238) ... 2023-12-19T16:41:13.484 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-12-19T16:41:13.484 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-12-19T16:41:14.249 INFO:teuthology.orchestra.run.smithi160.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-12-19T16:41:14.296 INFO:teuthology.orchestra.run.smithi026.stdout:Adding user postgres to group ssl-cert 2023-12-19T16:41:14.527 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:14.527 INFO:teuthology.orchestra.run.smithi026.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-12-19T16:41:14.567 INFO:teuthology.orchestra.run.smithi160.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-12-19T16:41:14.719 INFO:teuthology.orchestra.run.smithi160.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-12-19T16:41:14.807 INFO:teuthology.orchestra.run.smithi026.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-12-19T16:41:14.807 INFO:teuthology.orchestra.run.smithi026.stdout:Removing obsolete dictionary files: 2023-12-19T16:41:15.524 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-12-19T16:41:16.004 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-19T16:41:16.146 INFO:teuthology.orchestra.run.smithi160.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-12-19T16:41:16.246 INFO:teuthology.orchestra.run.smithi160.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-12-19T16:41:16.413 INFO:teuthology.orchestra.run.smithi026.stdout:Creating new PostgreSQL cluster 14/main ... 2023-12-19T16:41:16.433 INFO:teuthology.orchestra.run.smithi026.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-12-19T16:41:16.489 INFO:teuthology.orchestra.run.smithi026.stdout:The files belonging to this database system will be owned by user "postgres". 2023-12-19T16:41:16.489 INFO:teuthology.orchestra.run.smithi026.stdout:This user must also own the server process. 2023-12-19T16:41:16.489 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:16.489 INFO:teuthology.orchestra.run.smithi026.stdout:The database cluster will be initialized with locale "en_US". 2023-12-19T16:41:16.489 INFO:teuthology.orchestra.run.smithi026.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-12-19T16:41:16.489 INFO:teuthology.orchestra.run.smithi026.stdout:The default text search configuration will be set to "english". 2023-12-19T16:41:16.490 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:16.490 INFO:teuthology.orchestra.run.smithi026.stdout:Data page checksums are disabled. 2023-12-19T16:41:16.490 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:16.490 INFO:teuthology.orchestra.run.smithi026.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-12-19T16:41:16.490 INFO:teuthology.orchestra.run.smithi026.stdout:creating subdirectories ... ok 2023-12-19T16:41:16.490 INFO:teuthology.orchestra.run.smithi026.stdout:selecting dynamic shared memory implementation ... posix 2023-12-19T16:41:16.515 INFO:teuthology.orchestra.run.smithi026.stdout:selecting default max_connections ... 100 2023-12-19T16:41:16.551 INFO:teuthology.orchestra.run.smithi026.stdout:selecting default shared_buffers ... 128MB 2023-12-19T16:41:16.565 INFO:teuthology.orchestra.run.smithi026.stdout:selecting default time zone ... Etc/UTC 2023-12-19T16:41:16.566 INFO:teuthology.orchestra.run.smithi026.stdout:creating configuration files ... ok 2023-12-19T16:41:16.745 INFO:teuthology.orchestra.run.smithi026.stdout:running bootstrap script ... ok 2023-12-19T16:41:16.996 INFO:teuthology.orchestra.run.smithi160.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-12-19T16:41:17.003 DEBUG:teuthology.parallel:result is None 2023-12-19T16:41:17.387 INFO:teuthology.orchestra.run.smithi026.stdout:performing post-bootstrap initialization ... ok 2023-12-19T16:41:17.923 INFO:teuthology.orchestra.run.smithi026.stdout:syncing data to disk ... ok 2023-12-19T16:41:18.466 INFO:teuthology.orchestra.run.smithi026.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-12-19T16:41:21.528 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-12-19T16:41:21.662 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up postgresql-contrib (14+238) ... 2023-12-19T16:41:21.788 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up postgresql (14+238) ... 2023-12-19T16:41:21.914 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-12-19T16:41:22.015 INFO:teuthology.orchestra.run.smithi026.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-12-19T16:41:22.276 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-12-19T16:41:22.276 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-12-19T16:41:23.042 INFO:teuthology.orchestra.run.smithi026.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-12-19T16:41:23.352 INFO:teuthology.orchestra.run.smithi026.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-12-19T16:41:23.504 INFO:teuthology.orchestra.run.smithi026.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-12-19T16:41:24.941 INFO:teuthology.orchestra.run.smithi026.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-12-19T16:41:25.025 INFO:teuthology.orchestra.run.smithi026.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-12-19T16:41:25.683 INFO:teuthology.orchestra.run.smithi026.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-12-19T16:41:25.690 DEBUG:teuthology.parallel:result is None 2023-12-19T16:41:25.691 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-19T16:41:25.839 DEBUG:teuthology.orchestra.run.smithi026:> dpkg-query -W -f '${Version}' ceph 2023-12-19T16:41:25.862 INFO:teuthology.orchestra.run.smithi026.stdout:18.2.1-132-g568df92b-1jammy 2023-12-19T16:41:25.862 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-132-g568df92b-1jammy 2023-12-19T16:41:25.863 INFO:teuthology.task.install:The correct ceph version 18.2.1-132-g568df92b-1jammy is installed. 2023-12-19T16:41:25.864 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-19T16:41:26.142 DEBUG:teuthology.orchestra.run.smithi160:> dpkg-query -W -f '${Version}' ceph 2023-12-19T16:41:26.166 INFO:teuthology.orchestra.run.smithi160.stdout:18.2.1-132-g568df92b-1jammy 2023-12-19T16:41:26.166 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-132-g568df92b-1jammy 2023-12-19T16:41:26.166 INFO:teuthology.task.install:The correct ceph version 18.2.1-132-g568df92b-1jammy is installed. 2023-12-19T16:41:26.168 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-19T16:41:26.169 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:26.169 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-19T16:41:26.190 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:41:26.190 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-19T16:41:26.235 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-19T16:41:26.236 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:26.236 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2023-12-19T16:41:26.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-19T16:41:26.317 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:41:26.318 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/daemon-helper 2023-12-19T16:41:26.339 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-19T16:41:26.403 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-19T16:41:26.403 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:26.404 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-19T16:41:26.424 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-19T16:41:26.486 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:41:26.486 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-19T16:41:26.507 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-19T16:41:26.570 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-19T16:41:26.570 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:26.570 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2023-12-19T16:41:26.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-19T16:41:26.654 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:41:26.654 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/stdin-killer 2023-12-19T16:41:26.675 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-19T16:41:26.738 INFO:teuthology.run_tasks:Running task ceph... 2023-12-19T16:41:26.805 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-12-19T16:41:26.805 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /var/log/ceph 2023-12-19T16:41:26.808 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 777 /var/log/ceph 2023-12-19T16:41:26.829 INFO:tasks.ceph:Disabling ceph logrotate... 2023-12-19T16:41:26.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-19T16:41:26.873 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-19T16:41:26.893 INFO:tasks.ceph:Creating extra log directories... 2023-12-19T16:41:26.894 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-19T16:41:26.937 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-19T16:41:26.970 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-12-19T16:41:26.970 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-12-19T16:41:26.970 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-12-19T16:41:26.970 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:26.970 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-19T16:41:27.006 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-19T16:41:27.103 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:41:27.103 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-19T16:41:27.112 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-19T16:41:27.209 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-12-19T16:41:27.209 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-12-19T16:41:27.209 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112', 'branch': 'wip-vshankar-testing1-2023-12-18-1207-reef-2', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7497112', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing1-2023-12-18-1207-reef-2'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing1-2023-12-18-1207-reef-2', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 430, 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e', 'sleep_before_teardown': 0, 'subset': '301/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing1-2023-12-18-1207-reef-2', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_568df92b7e50b2108bce4c77c393ff294d6a5e5e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi026.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEThGRU3mLKvXf/qg5JsL9om6klrL7d2pnzrm5Fi5uTyVqXIF4g8S5zX5tYXJppael5Gt/C+TDpmOvRWe3aZ47c=', 'smithi160.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF897hQ5wuG30hBord1CoC4OmMeyPjDjjNM2QCswEqM6Iw6w8vkW9m7Db/dfhBMAlZZSpd+l6Ah3H5U7ILP4eCY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73', 'timestamp': '2023-12-19_13:22:41', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1485078'} 2023-12-19T16:41:27.210 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-19T16:41:27.213 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-19T16:41:27.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/run/ceph 2023-12-19T16:41:27.264 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m0777 -- /var/run/ceph 2023-12-19T16:41:27.289 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:27.290 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2023-12-19T16:41:27.334 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-19T16:41:27.334 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2023-12-19T16:41:27.386 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-19T16:41:27.386 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T16:41:27.386 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 903 Links: 1 2023-12-19T16:41:27.387 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T16:41:27.387 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-12-19 16:34:58.347359905 +0000 2023-12-19T16:41:27.387 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-12-19 16:34:57.959363369 +0000 2023-12-19T16:41:27.387 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-12-19 16:34:57.959363369 +0000 2023-12-19T16:41:27.387 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2023-12-19 16:34:57.959363369 +0000 2023-12-19T16:41:27.387 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-19T16:41:27.448 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-12-19T16:41:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-12-19T16:41:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000240859 s, 2.1 MB/s 2023-12-19T16:41:27.450 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-19T16:41:27.504 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2023-12-19T16:41:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-19T16:41:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T16:41:27.559 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 921 Links: 1 2023-12-19T16:41:27.560 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T16:41:27.560 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-12-19 16:34:59.039353723 +0000 2023-12-19T16:41:27.560 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-12-19 16:34:58.619357474 +0000 2023-12-19T16:41:27.560 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-12-19 16:34:58.619357474 +0000 2023-12-19T16:41:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2023-12-19 16:34:58.619357474 +0000 2023-12-19T16:41:27.561 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-19T16:41:27.621 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-12-19T16:41:27.621 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-12-19T16:41:27.621 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000237973 s, 2.2 MB/s 2023-12-19T16:41:27.623 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-19T16:41:27.679 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2023-12-19T16:41:27.731 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-19T16:41:27.732 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T16:41:27.732 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 938 Links: 1 2023-12-19T16:41:27.732 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T16:41:27.732 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-12-19 16:34:59.747347399 +0000 2023-12-19T16:41:27.732 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-12-19 16:34:59.319351222 +0000 2023-12-19T16:41:27.732 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-12-19 16:34:59.319351222 +0000 2023-12-19T16:41:27.733 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2023-12-19 16:34:59.319351222 +0000 2023-12-19T16:41:27.733 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-19T16:41:27.793 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-12-19T16:41:27.793 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-12-19T16:41:27.793 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000248158 s, 2.1 MB/s 2023-12-19T16:41:27.795 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-19T16:41:27.848 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2023-12-19T16:41:27.903 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-19T16:41:27.903 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T16:41:27.903 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 952 Links: 1 2023-12-19T16:41:27.903 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T16:41:27.903 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2023-12-19 16:35:00.431341290 +0000 2023-12-19T16:41:27.903 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2023-12-19 16:35:00.011345041 +0000 2023-12-19T16:41:27.903 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2023-12-19 16:35:00.011345041 +0000 2023-12-19T16:41:27.903 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2023-12-19 16:35:00.011345041 +0000 2023-12-19T16:41:27.904 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-19T16:41:27.965 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2023-12-19T16:41:27.965 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2023-12-19T16:41:27.965 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000242117 s, 2.1 MB/s 2023-12-19T16:41:27.967 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-19T16:41:28.020 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-19T16:41:28.021 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:41:28.021 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/scratch_devs of=/dev/stdout 2023-12-19T16:41:28.030 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-19T16:41:28.030 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 2023-12-19T16:41:28.083 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-19T16:41:28.083 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T16:41:28.083 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 887 Links: 1 2023-12-19T16:41:28.083 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T16:41:28.083 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2023-12-19 16:35:01.857269702 +0000 2023-12-19T16:41:28.083 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2023-12-19 16:35:01.853269738 +0000 2023-12-19T16:41:28.083 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2023-12-19 16:35:01.853269738 +0000 2023-12-19T16:41:28.083 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2023-12-19 16:35:01.853269738 +0000 2023-12-19T16:41:28.084 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-19T16:41:28.146 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2023-12-19T16:41:28.146 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2023-12-19T16:41:28.146 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000400304 s, 1.3 MB/s 2023-12-19T16:41:28.148 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-19T16:41:28.205 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 2023-12-19T16:41:28.259 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-19T16:41:28.259 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T16:41:28.259 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 901 Links: 1 2023-12-19T16:41:28.259 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T16:41:28.259 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2023-12-19 16:35:02.933260091 +0000 2023-12-19T16:41:28.259 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2023-12-19 16:35:02.529263699 +0000 2023-12-19T16:41:28.259 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2023-12-19 16:35:02.529263699 +0000 2023-12-19T16:41:28.260 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2023-12-19 16:35:02.529263699 +0000 2023-12-19T16:41:28.260 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-19T16:41:28.326 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2023-12-19T16:41:28.326 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2023-12-19T16:41:28.326 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000385638 s, 1.3 MB/s 2023-12-19T16:41:28.328 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-19T16:41:28.381 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 2023-12-19T16:41:28.435 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-19T16:41:28.435 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T16:41:28.435 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 919 Links: 1 2023-12-19T16:41:28.435 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T16:41:28.435 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2023-12-19 16:35:03.617253981 +0000 2023-12-19T16:41:28.435 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2023-12-19 16:35:03.221257518 +0000 2023-12-19T16:41:28.436 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2023-12-19 16:35:03.221257518 +0000 2023-12-19T16:41:28.436 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2023-12-19 16:35:03.221257518 +0000 2023-12-19T16:41:28.436 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-19T16:41:28.497 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2023-12-19T16:41:28.497 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2023-12-19T16:41:28.498 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000385311 s, 1.3 MB/s 2023-12-19T16:41:28.499 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-19T16:41:28.553 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 2023-12-19T16:41:28.607 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-19T16:41:28.607 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T16:41:28.607 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 935 Links: 1 2023-12-19T16:41:28.607 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T16:41:28.607 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2023-12-19 16:35:04.381247157 +0000 2023-12-19T16:41:28.607 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2023-12-19 16:35:03.989250659 +0000 2023-12-19T16:41:28.607 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2023-12-19 16:35:03.989250659 +0000 2023-12-19T16:41:28.607 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2023-12-19 16:35:03.989250659 +0000 2023-12-19T16:41:28.607 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-19T16:41:28.673 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2023-12-19T16:41:28.673 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2023-12-19T16:41:28.674 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000386048 s, 1.3 MB/s 2023-12-19T16:41:28.675 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-19T16:41:28.732 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-19T16:41:28.733 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi160.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-19T16:41:28.733 INFO:tasks.ceph:Generating config... 2023-12-19T16:41:28.734 INFO:tasks.ceph:[client] client mount timeout = 600 2023-12-19T16:41:28.734 INFO:tasks.ceph:[client] debug client = 20 2023-12-19T16:41:28.735 INFO:tasks.ceph:[client] debug ms = 1 2023-12-19T16:41:28.735 INFO:tasks.ceph:[client] pid file = 2023-12-19T16:41:28.735 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-12-19T16:41:28.735 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-12-19T16:41:28.735 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-12-19T16:41:28.735 INFO:tasks.ceph:[mds] debug mds = 20 2023-12-19T16:41:28.735 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-12-19T16:41:28.736 INFO:tasks.ceph:[mds] debug ms = 1 2023-12-19T16:41:28.736 INFO:tasks.ceph:[mds] mds debug frag = True 2023-12-19T16:41:28.736 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-12-19T16:41:28.736 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-12-19T16:41:28.736 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-12-19T16:41:28.736 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-12-19T16:41:28.736 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-12-19T16:41:28.737 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-12-19T16:41:28.737 INFO:tasks.ceph:[mgr] debug client = 20 2023-12-19T16:41:28.737 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-12-19T16:41:28.737 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-12-19T16:41:28.737 INFO:tasks.ceph:[mgr] debug ms = 1 2023-12-19T16:41:28.737 INFO:tasks.ceph:[mon] debug mon = 20 2023-12-19T16:41:28.737 INFO:tasks.ceph:[mon] debug ms = 1 2023-12-19T16:41:28.738 INFO:tasks.ceph:[mon] debug paxos = 20 2023-12-19T16:41:28.738 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-12-19T16:41:28.738 INFO:tasks.ceph:[osd] bdev async discard = True 2023-12-19T16:41:28.738 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-12-19T16:41:28.738 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-12-19T16:41:28.738 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-12-19T16:41:28.739 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-12-19T16:41:28.739 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-12-19T16:41:28.739 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-12-19T16:41:28.739 INFO:tasks.ceph:[osd] debug ms = 1 2023-12-19T16:41:28.739 INFO:tasks.ceph:[osd] debug osd = 20 2023-12-19T16:41:28.739 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-12-19T16:41:28.739 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-12-19T16:41:28.740 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-12-19T16:41:28.740 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-12-19T16:41:28.740 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-12-19T16:41:28.740 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-12-19T16:41:28.740 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-12-19T16:41:28.741 INFO:tasks.ceph:Setting up mon.a... 2023-12-19T16:41:28.741 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-12-19T16:41:28.791 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.keyring 2023-12-19T16:41:28.811 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-12-19T16:41:28.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-19T16:41:28.886 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.26'), ('mon.b', '172.21.15.160'), ('mon.c', '[v2:172.21.15.160:3301,v1:172.21.15.160:6790]')] 2023-12-19T16:41:28.887 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.26,172.21.15.160,[v2:172.21.15.160:3301,v1:172.21.15.160:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-12-19T16:41:28.888 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:28.889 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-12-19T16:41:28.939 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.26 --add b 172.21.15.160 --addv c '[v2:172.21.15.160:3301,v1:172.21.15.160:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:41:29.034 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:41:29.034 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: generated fsid 2264810b-2623-438e-8790-ac9d6fbbbfd5 2023-12-19T16:41:29.034 INFO:teuthology.orchestra.run.smithi026.stdout:setting min_mon_release = pacific 2023-12-19T16:41:29.034 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 0 2023-12-19T16:41:29.034 INFO:teuthology.orchestra.run.smithi026.stdout:fsid 2264810b-2623-438e-8790-ac9d6fbbbfd5 2023-12-19T16:41:29.034 INFO:teuthology.orchestra.run.smithi026.stdout:last_changed 2023-12-19T16:41:29.032530+0000 2023-12-19T16:41:29.034 INFO:teuthology.orchestra.run.smithi026.stdout:created 2023-12-19T16:41:29.032530+0000 2023-12-19T16:41:29.034 INFO:teuthology.orchestra.run.smithi026.stdout:min_mon_release 16 (pacific) 2023-12-19T16:41:29.034 INFO:teuthology.orchestra.run.smithi026.stdout:election_strategy: 1 2023-12-19T16:41:29.034 INFO:teuthology.orchestra.run.smithi026.stdout:0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2023-12-19T16:41:29.034 INFO:teuthology.orchestra.run.smithi026.stdout:1: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.b 2023-12-19T16:41:29.035 INFO:teuthology.orchestra.run.smithi026.stdout:2: [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] mon.c 2023-12-19T16:41:29.035 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-12-19T16:41:29.037 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-12-19T16:41:29.045 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2264810b-2623-438e-8790-ac9d6fbbbfd5... 2023-12-19T16:41:29.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-19T16:41:29.097 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-19T16:41:29.139 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2023-12-19T16:41:29.140 INFO:teuthology.orchestra.run.smithi026.stdout: chdir = "" 2023-12-19T16:41:29.140 INFO:teuthology.orchestra.run.smithi026.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-19T16:41:29.140 INFO:teuthology.orchestra.run.smithi026.stdout: auth supported = cephx 2023-12-19T16:41:29.140 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.140 INFO:teuthology.orchestra.run.smithi026.stdout: filestore xattr use omap = true 2023-12-19T16:41:29.140 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.140 INFO:teuthology.orchestra.run.smithi026.stdout: mon clock drift allowed = 1.000 2023-12-19T16:41:29.140 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.140 INFO:teuthology.orchestra.run.smithi026.stdout: osd crush chooseleaf type = 0 2023-12-19T16:41:29.140 INFO:teuthology.orchestra.run.smithi026.stdout: auth debug = true 2023-12-19T16:41:29.140 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.140 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on old message = true 2023-12-19T16:41:29.141 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on bug = true 2023-12-19T16:41:29.141 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.141 INFO:teuthology.orchestra.run.smithi026.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-19T16:41:29.141 INFO:teuthology.orchestra.run.smithi026.stdout: mon pg warn max object skew = 0 2023-12-19T16:41:29.141 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.141 INFO:teuthology.orchestra.run.smithi026.stdout: # disable pg_autoscaler by default for new pools 2023-12-19T16:41:29.141 INFO:teuthology.orchestra.run.smithi026.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-19T16:41:29.141 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.141 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default size = 2 2023-12-19T16:41:29.141 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.141 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow primary affinity = true 2023-12-19T16:41:29.141 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow pg remap = true 2023-12-19T16:41:29.141 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on legacy crush tunables = false 2023-12-19T16:41:29.142 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on crush straw calc version zero = false 2023-12-19T16:41:29.142 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on no sortbitwise = false 2023-12-19T16:41:29.142 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on osd down out interval zero = false 2023-12-19T16:41:29.142 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on too few osds = false 2023-12-19T16:41:29.142 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-19T16:41:29.142 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-19T16:41:29.142 INFO:teuthology.orchestra.run.smithi026.stdout: mon_allow_pool_size_one = true 2023-12-19T16:41:29.142 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.142 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-19T16:41:29.142 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.142 INFO:teuthology.orchestra.run.smithi026.stdout: osd default data pool replay window = 5 2023-12-19T16:41:29.142 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.143 INFO:teuthology.orchestra.run.smithi026.stdout: mon allow pool delete = true 2023-12-19T16:41:29.143 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.143 INFO:teuthology.orchestra.run.smithi026.stdout: mon cluster log file level = debug 2023-12-19T16:41:29.143 INFO:teuthology.orchestra.run.smithi026.stdout: debug asserts on shutdown = true 2023-12-19T16:41:29.143 INFO:teuthology.orchestra.run.smithi026.stdout: mon health detail to clog = false 2023-12-19T16:41:29.143 INFO:teuthology.orchestra.run.smithi026.stdout: mon host = "172.21.15.26,172.21.15.160,[v2:172.21.15.160:3301,v1:172.21.15.160:6790]" 2023-12-19T16:41:29.143 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on skipped message = False 2023-12-19T16:41:29.143 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = 2264810b-2623-438e-8790-ac9d6fbbbfd5 2023-12-19T16:41:29.143 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.143 INFO:teuthology.orchestra.run.smithi026.stdout:[osd] 2023-12-19T16:41:29.143 INFO:teuthology.orchestra.run.smithi026.stdout: osd journal size = 100 2023-12-19T16:41:29.143 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.143 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub load threshold = 5.0 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub max interval = 600 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: osd mclock profile = high_recovery_ops 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: osd recover clone overlap = true 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: osd recovery max chunk = 1048576 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug shutdown = true 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug op order = true 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug verify stray on activate = true 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: osd open classes on start = true 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug pg log writeout = true 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.144 INFO:teuthology.orchestra.run.smithi026.stdout: osd deep scrub update digest min age = 30 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: osd map max advance = 10 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: journal zero on create = true 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: filestore ondisk finisher threads = 3 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: filestore apply finisher threads = 3 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: bdev debug aio = true 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug misdirected ops = true 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: bdev async discard = True 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: bdev enable discard = True 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore allocator = bitmap 2023-12-19T16:41:29.145 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore block size = 96636764160 2023-12-19T16:41:29.146 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore fsck on mount = True 2023-12-19T16:41:29.146 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluefs = 1/20 2023-12-19T16:41:29.146 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluestore = 1/20 2023-12-19T16:41:29.146 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2023-12-19T16:41:29.146 INFO:teuthology.orchestra.run.smithi026.stdout: debug osd = 20 2023-12-19T16:41:29.146 INFO:teuthology.orchestra.run.smithi026.stdout: debug rocksdb = 4/10 2023-12-19T16:41:29.146 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd backfillfull_ratio = 0.85 2023-12-19T16:41:29.146 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd full ratio = 0.9 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd nearfull ratio = 0.8 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout: osd failsafe full ratio = 0.95 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout: osd objectstore = bluestore 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout: osd op complaint time = 180 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr] 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout: debug mgr = 20 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout: mgr/telemetry/nag = false 2023-12-19T16:41:29.147 INFO:teuthology.orchestra.run.smithi026.stdout: debug client = 20 2023-12-19T16:41:29.148 INFO:teuthology.orchestra.run.smithi026.stdout: debug finisher = 20 2023-12-19T16:41:29.148 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.148 INFO:teuthology.orchestra.run.smithi026.stdout:[mon] 2023-12-19T16:41:29.148 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2023-12-19T16:41:29.148 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2023-12-19T16:41:29.148 INFO:teuthology.orchestra.run.smithi026.stdout: debug paxos = 20 2023-12-19T16:41:29.148 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2023-12-19T16:41:29.149 INFO:teuthology.orchestra.run.smithi026.stdout: mon data avail warn = 5 2023-12-19T16:41:29.149 INFO:teuthology.orchestra.run.smithi026.stdout: mon mgr mkfs grace = 240 2023-12-19T16:41:29.149 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2023-12-19T16:41:29.149 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd reporter subtree level = osd 2023-12-19T16:41:29.149 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd prime pg temp = true 2023-12-19T16:41:29.149 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2023-12-19T16:41:29.149 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.149 INFO:teuthology.orchestra.run.smithi026.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-19T16:41:29.149 INFO:teuthology.orchestra.run.smithi026.stdout: auth mon ticket ttl = 660 # 11m 2023-12-19T16:41:29.149 INFO:teuthology.orchestra.run.smithi026.stdout: auth service ticket ttl = 240 # 4m 2023-12-19T16:41:29.149 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.149 INFO:teuthology.orchestra.run.smithi026.stdout: # don't complain about insecure global_id in the test suite 2023-12-19T16:41:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-19T16:41:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-19T16:41:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: # 1m isn't quite enough 2023-12-19T16:41:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: mon_down_mkfs_grace = 2m 2023-12-19T16:41:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_filestore_osds = false 2023-12-19T16:41:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: mon op complaint time = 120 2023-12-19T16:41:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:41:29.150 INFO:teuthology.orchestra.run.smithi026.stdout:[client] 2023-12-19T16:41:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: rgw cache enabled = true 2023-12-19T16:41:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable ops log = true 2023-12-19T16:41:29.150 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable usage log = true 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: client mount timeout = 600 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: debug client = 20 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: pid file = "" 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: rados mon op timeout = 900 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout: rados osd op timeout = 900 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.a] 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.b] 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.c] 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a] 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c] 2023-12-19T16:41:29.151 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b] 2023-12-19T16:41:29.152 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d] 2023-12-19T16:41:29.152 INFO:teuthology.orchestra.run.smithi026.stdout:[mds] 2023-12-19T16:41:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: debug mds = 20 2023-12-19T16:41:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: debug mds balancer = 20 2023-12-19T16:41:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2023-12-19T16:41:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: mds debug frag = True 2023-12-19T16:41:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: mds debug scatterstat = True 2023-12-19T16:41:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: mds op complaint time = 180 2023-12-19T16:41:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: mds verify scatter = True 2023-12-19T16:41:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: osd op complaint time = 180 2023-12-19T16:41:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: rados mon op timeout = 900 2023-12-19T16:41:29.152 INFO:teuthology.orchestra.run.smithi026.stdout: rados osd op timeout = 900 2023-12-19T16:41:29.153 INFO:teuthology.orchestra.run.smithi160.stdout:[global] 2023-12-19T16:41:29.153 INFO:teuthology.orchestra.run.smithi160.stdout: chdir = "" 2023-12-19T16:41:29.153 INFO:teuthology.orchestra.run.smithi160.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-19T16:41:29.153 INFO:teuthology.orchestra.run.smithi160.stdout: auth supported = cephx 2023-12-19T16:41:29.153 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.153 INFO:teuthology.orchestra.run.smithi160.stdout: filestore xattr use omap = true 2023-12-19T16:41:29.153 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.153 INFO:teuthology.orchestra.run.smithi160.stdout: mon clock drift allowed = 1.000 2023-12-19T16:41:29.154 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.154 INFO:teuthology.orchestra.run.smithi160.stdout: osd crush chooseleaf type = 0 2023-12-19T16:41:29.154 INFO:teuthology.orchestra.run.smithi160.stdout: auth debug = true 2023-12-19T16:41:29.154 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.154 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on old message = true 2023-12-19T16:41:29.154 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on bug = true 2023-12-19T16:41:29.154 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.154 INFO:teuthology.orchestra.run.smithi160.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-19T16:41:29.154 INFO:teuthology.orchestra.run.smithi160.stdout: mon pg warn max object skew = 0 2023-12-19T16:41:29.154 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.154 INFO:teuthology.orchestra.run.smithi160.stdout: # disable pg_autoscaler by default for new pools 2023-12-19T16:41:29.154 INFO:teuthology.orchestra.run.smithi160.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: osd pool default size = 2 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd allow primary affinity = true 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd allow pg remap = true 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on legacy crush tunables = false 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on crush straw calc version zero = false 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on no sortbitwise = false 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on osd down out interval zero = false 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on too few osds = false 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: mon_allow_pool_size_one = true 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.155 INFO:teuthology.orchestra.run.smithi160.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-19T16:41:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: osd default data pool replay window = 5 2023-12-19T16:41:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: mon allow pool delete = true 2023-12-19T16:41:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: mon cluster log file level = debug 2023-12-19T16:41:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: debug asserts on shutdown = true 2023-12-19T16:41:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: mon health detail to clog = false 2023-12-19T16:41:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: mon host = "172.21.15.26,172.21.15.160,[v2:172.21.15.160:3301,v1:172.21.15.160:6790]" 2023-12-19T16:41:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on skipped message = False 2023-12-19T16:41:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: fsid = 2264810b-2623-438e-8790-ac9d6fbbbfd5 2023-12-19T16:41:29.156 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.156 INFO:teuthology.orchestra.run.smithi160.stdout:[osd] 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: osd journal size = 100 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: osd scrub load threshold = 5.0 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: osd scrub max interval = 600 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: osd mclock profile = high_recovery_ops 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: osd recover clone overlap = true 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: osd recovery max chunk = 1048576 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug shutdown = true 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug op order = true 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug verify stray on activate = true 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.157 INFO:teuthology.orchestra.run.smithi160.stdout: osd open classes on start = true 2023-12-19T16:41:29.158 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug pg log writeout = true 2023-12-19T16:41:29.158 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.158 INFO:teuthology.orchestra.run.smithi160.stdout: osd deep scrub update digest min age = 30 2023-12-19T16:41:29.158 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.158 INFO:teuthology.orchestra.run.smithi160.stdout: osd map max advance = 10 2023-12-19T16:41:29.158 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.158 INFO:teuthology.orchestra.run.smithi160.stdout: journal zero on create = true 2023-12-19T16:41:29.158 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.158 INFO:teuthology.orchestra.run.smithi160.stdout: filestore ondisk finisher threads = 3 2023-12-19T16:41:29.158 INFO:teuthology.orchestra.run.smithi160.stdout: filestore apply finisher threads = 3 2023-12-19T16:41:29.158 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.158 INFO:teuthology.orchestra.run.smithi160.stdout: bdev debug aio = true 2023-12-19T16:41:29.158 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug misdirected ops = true 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: bdev async discard = True 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: bdev enable discard = True 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: bluestore allocator = bitmap 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: bluestore block size = 96636764160 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: bluestore fsck on mount = True 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: debug bluefs = 1/20 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: debug bluestore = 1/20 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: debug osd = 20 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: debug rocksdb = 4/10 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd backfillfull_ratio = 0.85 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd full ratio = 0.9 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd nearfull ratio = 0.8 2023-12-19T16:41:29.159 INFO:teuthology.orchestra.run.smithi160.stdout: osd failsafe full ratio = 0.95 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout: osd objectstore = bluestore 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout: osd op complaint time = 180 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr] 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout: debug mgr = 20 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout: debug mon = 20 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout: debug auth = 20 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min pgs per osd = 4 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min bytes per osd = 10 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout: mgr/telemetry/nag = false 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout: debug client = 20 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout: debug finisher = 20 2023-12-19T16:41:29.160 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout:[mon] 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout: debug mon = 20 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout: debug paxos = 20 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout: debug auth = 20 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout: mon data avail warn = 5 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout: mon mgr mkfs grace = 240 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min pgs per osd = 4 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd reporter subtree level = osd 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd prime pg temp = true 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min bytes per osd = 10 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-19T16:41:29.161 INFO:teuthology.orchestra.run.smithi160.stdout: auth mon ticket ttl = 660 # 11m 2023-12-19T16:41:29.162 INFO:teuthology.orchestra.run.smithi160.stdout: auth service ticket ttl = 240 # 4m 2023-12-19T16:41:29.162 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.162 INFO:teuthology.orchestra.run.smithi160.stdout: # don't complain about insecure global_id in the test suite 2023-12-19T16:41:29.162 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-19T16:41:29.162 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-19T16:41:29.162 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.162 INFO:teuthology.orchestra.run.smithi160.stdout: # 1m isn't quite enough 2023-12-19T16:41:29.162 INFO:teuthology.orchestra.run.smithi160.stdout: mon_down_mkfs_grace = 2m 2023-12-19T16:41:29.162 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.162 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_filestore_osds = false 2023-12-19T16:41:29.162 INFO:teuthology.orchestra.run.smithi160.stdout: mon op complaint time = 120 2023-12-19T16:41:29.162 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-19T16:41:29.162 INFO:teuthology.orchestra.run.smithi160.stdout:[client] 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout: rgw cache enabled = true 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout: rgw enable ops log = true 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout: rgw enable usage log = true 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout: client mount timeout = 600 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout: debug client = 20 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout: pid file = "" 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout: rados mon op timeout = 900 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout: rados osd op timeout = 900 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.a] 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.b] 2023-12-19T16:41:29.163 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.c] 2023-12-19T16:41:29.164 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.a] 2023-12-19T16:41:29.164 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.c] 2023-12-19T16:41:29.164 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.b] 2023-12-19T16:41:29.164 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.d] 2023-12-19T16:41:29.164 INFO:teuthology.orchestra.run.smithi160.stdout:[mds] 2023-12-19T16:41:29.164 INFO:teuthology.orchestra.run.smithi160.stdout: debug mds = 20 2023-12-19T16:41:29.164 INFO:teuthology.orchestra.run.smithi160.stdout: debug mds balancer = 20 2023-12-19T16:41:29.164 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2023-12-19T16:41:29.164 INFO:teuthology.orchestra.run.smithi160.stdout: mds debug frag = True 2023-12-19T16:41:29.164 INFO:teuthology.orchestra.run.smithi160.stdout: mds debug scatterstat = True 2023-12-19T16:41:29.164 INFO:teuthology.orchestra.run.smithi160.stdout: mds op complaint time = 180 2023-12-19T16:41:29.164 INFO:teuthology.orchestra.run.smithi160.stdout: mds verify scatter = True 2023-12-19T16:41:29.164 INFO:teuthology.orchestra.run.smithi160.stdout: osd op complaint time = 180 2023-12-19T16:41:29.165 INFO:teuthology.orchestra.run.smithi160.stdout: rados mon op timeout = 900 2023-12-19T16:41:29.165 INFO:teuthology.orchestra.run.smithi160.stdout: rados osd op timeout = 900 2023-12-19T16:41:29.166 INFO:tasks.ceph:Creating admin key on mon.a... 2023-12-19T16:41:29.166 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-12-19T16:41:29.268 INFO:tasks.ceph:Copying monmap to all nodes... 2023-12-19T16:41:29.268 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:29.269 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-12-19T16:41:29.278 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:29.278 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-12-19T16:41:29.330 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T16:41:29.330 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:29.330 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-19T16:41:29.330 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-19T16:41:29.411 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:29.411 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:41:29.462 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi160.front.sepia.ceph.com 2023-12-19T16:41:29.463 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:41:29.463 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-19T16:41:29.463 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-19T16:41:29.498 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:41:29.498 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:41:29.550 INFO:tasks.ceph:Setting up mon nodes... 2023-12-19T16:41:29.550 INFO:tasks.ceph:Setting up mgr nodes... 2023-12-19T16:41:29.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-12-19T16:41:29.608 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-12-19T16:41:29.613 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-12-19T16:41:29.673 INFO:teuthology.orchestra.run.smithi160.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-12-19T16:41:29.685 INFO:tasks.ceph:Setting up mds nodes... 2023-12-19T16:41:29.685 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-12-19T16:41:29.744 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-12-19T16:41:29.749 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-12-19T16:41:29.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-12-19T16:41:29.871 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-12-19T16:41:29.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-12-19T16:41:29.896 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-12-19T16:41:29.954 INFO:teuthology.orchestra.run.smithi160.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-12-19T16:41:29.959 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-12-19T16:41:29.979 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-12-19T16:41:30.076 INFO:teuthology.orchestra.run.smithi160.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-12-19T16:41:30.081 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-12-19T16:41:30.101 INFO:tasks.ceph_client:Setting up client nodes... 2023-12-19T16:41:30.101 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-19T16:41:30.147 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-12-19T16:41:30.165 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-12-19T16:41:30.209 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-12-19T16:41:30.227 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-12-19T16:41:30.270 INFO:teuthology.orchestra.run.smithi160.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-12-19T16:41:30.287 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-12-19T16:41:30.374 INFO:teuthology.orchestra.run.smithi160.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-12-19T16:41:30.390 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-12-19T16:41:30.391 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi160.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-19T16:41:30.391 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-12-19T16:41:30.412 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-19T16:41:30.412 INFO:tasks.ceph:role: osd.0 2023-12-19T16:41:30.412 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T16:41:30.412 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-19T16:41:30.478 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T16:41:30.478 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T16:41:30.478 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T16:41:30.478 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-19T16:41:30.478 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T16:41:30.478 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2023-12-19T16:41:30.478 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T16:41:30.478 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-19T16:41:30.479 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T16:41:30.479 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T16:41:30.503 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2023-12-19T16:41:30.506 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2023-12-19T16:41:30.506 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-12-19T16:41:30.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-12-19T16:41:30.638 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: /sbin/restorecon: command not found 2023-12-19T16:41:30.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:41:30.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-12-19T16:41:30.702 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-19T16:41:30.702 INFO:tasks.ceph:role: osd.1 2023-12-19T16:41:30.702 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T16:41:30.702 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-19T16:41:30.770 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T16:41:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T16:41:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T16:41:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-19T16:41:30.770 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T16:41:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2023-12-19T16:41:30.770 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T16:41:30.770 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-19T16:41:30.770 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T16:41:30.771 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T16:41:30.795 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2023-12-19T16:41:30.798 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2023-12-19T16:41:30.798 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-12-19T16:41:30.868 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-12-19T16:41:30.926 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: /sbin/restorecon: command not found 2023-12-19T16:41:30.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:41:30.927 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-12-19T16:41:30.990 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-19T16:41:30.990 INFO:tasks.ceph:role: osd.2 2023-12-19T16:41:30.990 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T16:41:30.990 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-19T16:41:31.054 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T16:41:31.054 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T16:41:31.055 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T16:41:31.055 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-19T16:41:31.055 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T16:41:31.055 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2023-12-19T16:41:31.055 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T16:41:31.055 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-19T16:41:31.055 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T16:41:31.055 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T16:41:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2023-12-19T16:41:31.082 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2023-12-19T16:41:31.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-12-19T16:41:31.152 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-12-19T16:41:31.210 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: /sbin/restorecon: command not found 2023-12-19T16:41:31.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:41:31.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-12-19T16:41:31.278 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-19T16:41:31.278 INFO:tasks.ceph:role: osd.3 2023-12-19T16:41:31.278 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T16:41:31.278 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-19T16:41:31.343 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T16:41:31.343 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T16:41:31.343 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T16:41:31.343 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-19T16:41:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T16:41:31.344 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2023-12-19T16:41:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T16:41:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-19T16:41:31.344 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T16:41:31.344 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T16:41:31.370 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2023-12-19T16:41:31.372 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2023-12-19T16:41:31.373 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-12-19T16:41:31.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-12-19T16:41:31.502 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: /sbin/restorecon: command not found 2023-12-19T16:41:31.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:41:31.503 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:41:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:31.625+0000 7fc58b701640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-12-19T16:41:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:31.625+0000 7fc58b701640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-12-19T16:41:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:31.625+0000 7fc58b701640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-19T16:41:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:31.625+0000 7fc58b701640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-19T16:41:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:31.625+0000 7fc58b701640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-19T16:41:31.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:31.625+0000 7fc58b701640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-12-19T16:41:34.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-12-19T16:41:34.656 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:41:34.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:34.745+0000 7f8ff9ab8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-12-19T16:41:34.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:34.745+0000 7f8ff9ab8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-12-19T16:41:34.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:34.745+0000 7f8ff9ab8640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-19T16:41:34.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:34.745+0000 7f8ff9ab8640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-19T16:41:34.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:34.745+0000 7f8ff9ab8640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-19T16:41:34.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:34.749+0000 7f8ff9ab8640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-12-19T16:41:37.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-12-19T16:41:37.760 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:41:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:37.857+0000 7f2f5e4e2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-12-19T16:41:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:37.857+0000 7f2f5e4e2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-12-19T16:41:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:37.857+0000 7f2f5e4e2640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-19T16:41:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:37.857+0000 7f2f5e4e2640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-19T16:41:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:37.857+0000 7f2f5e4e2640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-19T16:41:37.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:37.857+0000 7f2f5e4e2640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-12-19T16:41:40.845 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-12-19T16:41:40.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:41:40.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:40.957+0000 7f4f5b828640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-12-19T16:41:40.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:40.957+0000 7f4f5b828640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-12-19T16:41:40.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:40.957+0000 7f4f5b828640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-19T16:41:40.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:40.957+0000 7f4f5b828640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-19T16:41:40.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:40.957+0000 7f4f5b828640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-19T16:41:40.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:41:40.961+0000 7f4f5b828640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-12-19T16:41:43.953 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-12-19T16:41:43.975 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-12-19T16:41:43.998 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-19T16:41:43.998 INFO:tasks.ceph:role: osd.4 2023-12-19T16:41:43.998 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi160.front.sepia.ceph.com 2023-12-19T16:41:43.998 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-19T16:41:44.071 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T16:41:44.071 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T16:41:44.071 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T16:41:44.071 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-19T16:41:44.071 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T16:41:44.071 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2023-12-19T16:41:44.071 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T16:41:44.071 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-19T16:41:44.071 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T16:41:44.071 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T16:41:44.529 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2023-12-19T16:41:44.532 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2023-12-19T16:41:44.532 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-12-19T16:41:44.649 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-12-19T16:41:44.666 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: /sbin/restorecon: command not found 2023-12-19T16:41:44.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:41:44.667 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-12-19T16:41:44.730 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-19T16:41:44.730 INFO:tasks.ceph:role: osd.5 2023-12-19T16:41:44.730 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi160.front.sepia.ceph.com 2023-12-19T16:41:44.731 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-19T16:41:44.803 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T16:41:44.803 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T16:41:44.803 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T16:41:44.803 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-19T16:41:44.803 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T16:41:44.803 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2023-12-19T16:41:44.803 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T16:41:44.803 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-19T16:41:44.804 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T16:41:44.804 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T16:41:45.299 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2023-12-19T16:41:45.302 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2023-12-19T16:41:45.302 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-12-19T16:41:45.445 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-12-19T16:41:45.463 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: /sbin/restorecon: command not found 2023-12-19T16:41:45.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:41:45.464 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-12-19T16:41:45.531 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-19T16:41:45.531 INFO:tasks.ceph:role: osd.6 2023-12-19T16:41:45.531 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi160.front.sepia.ceph.com 2023-12-19T16:41:45.531 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-19T16:41:45.602 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T16:41:45.603 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T16:41:45.603 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T16:41:45.603 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-19T16:41:45.603 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T16:41:45.603 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2023-12-19T16:41:45.603 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T16:41:45.603 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-19T16:41:45.604 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T16:41:45.604 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T16:41:46.084 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2023-12-19T16:41:46.087 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2023-12-19T16:41:46.087 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-12-19T16:41:46.210 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-12-19T16:41:46.228 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: /sbin/restorecon: command not found 2023-12-19T16:41:46.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:41:46.229 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-12-19T16:41:46.295 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-19T16:41:46.295 INFO:tasks.ceph:role: osd.7 2023-12-19T16:41:46.296 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi160.front.sepia.ceph.com 2023-12-19T16:41:46.296 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-19T16:41:46.367 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T16:41:46.367 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T16:41:46.367 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T16:41:46.367 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-19T16:41:46.368 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T16:41:46.368 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2023-12-19T16:41:46.368 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T16:41:46.368 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-19T16:41:46.368 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T16:41:46.368 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T16:41:46.818 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2023-12-19T16:41:46.820 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2023-12-19T16:41:46.820 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-12-19T16:41:46.933 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-12-19T16:41:46.950 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: /sbin/restorecon: command not found 2023-12-19T16:41:46.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:41:46.951 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:41:47.084 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:47.079+0000 7f844ed55640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-12-19T16:41:47.087 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:47.083+0000 7f844ed55640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-12-19T16:41:47.087 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:47.083+0000 7f844ed55640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-19T16:41:47.087 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:47.083+0000 7f844ed55640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-19T16:41:47.087 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:47.083+0000 7f844ed55640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-19T16:41:47.089 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:47.083+0000 7f844ed55640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-12-19T16:41:50.098 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-12-19T16:41:50.118 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:41:50.214 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:50.211+0000 7f5821472640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-12-19T16:41:50.215 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:50.211+0000 7f5821472640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-12-19T16:41:50.215 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:50.211+0000 7f5821472640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-19T16:41:50.216 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:50.211+0000 7f5821472640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-19T16:41:50.216 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:50.211+0000 7f5821472640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-19T16:41:50.217 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:50.211+0000 7f5821472640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-12-19T16:41:53.222 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-12-19T16:41:53.243 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:41:53.339 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:53.335+0000 7f751d457640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-12-19T16:41:53.340 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:53.335+0000 7f751d457640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-12-19T16:41:53.340 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:53.335+0000 7f751d457640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-19T16:41:53.340 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:53.335+0000 7f751d457640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-19T16:41:53.340 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:53.335+0000 7f751d457640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-19T16:41:53.341 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:53.335+0000 7f751d457640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-12-19T16:41:56.344 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-12-19T16:41:56.365 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:41:56.460 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:56.455+0000 7fdc95161640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-12-19T16:41:56.461 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:56.455+0000 7fdc95161640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-12-19T16:41:56.461 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:56.455+0000 7fdc95161640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-19T16:41:56.461 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:56.455+0000 7fdc95161640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-19T16:41:56.461 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:56.455+0000 7fdc95161640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-19T16:41:56.462 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:41:56.459+0000 7fdc95161640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-12-19T16:41:57.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:41:57.215 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:41:59.460 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-12-19T16:41:59.483 INFO:tasks.ceph:Reading keys from all nodes... 2023-12-19T16:41:59.483 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:59.483 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-12-19T16:41:59.507 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:59.508 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-12-19T16:41:59.571 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:59.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-12-19T16:41:59.635 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:59.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-12-19T16:41:59.700 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:59.700 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-12-19T16:41:59.763 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:59.763 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-12-19T16:41:59.827 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:41:59.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-12-19T16:41:59.892 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:41:59.892 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-12-19T16:41:59.914 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:41:59.914 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-12-19T16:41:59.978 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:41:59.978 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-12-19T16:42:00.041 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:00.041 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-12-19T16:42:00.101 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:00.102 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-12-19T16:42:00.165 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:00.165 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-12-19T16:42:00.225 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:00.225 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-12-19T16:42:00.285 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:42:00.285 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-12-19T16:42:00.295 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:42:00.295 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-12-19T16:42:00.346 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:00.347 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-12-19T16:42:00.355 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:00.355 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-12-19T16:42:00.406 INFO:tasks.ceph:Adding keys to all mons... 2023-12-19T16:42:00.406 DEBUG:teuthology.orchestra.run.smithi026:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-19T16:42:00.409 DEBUG:teuthology.orchestra.run.smithi160:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-19T16:42:00.458 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.y] 2023-12-19T16:42:00.458 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg== 2023-12-19T16:42:00.458 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a] 2023-12-19T16:42:00.458 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg== 2023-12-19T16:42:00.458 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c] 2023-12-19T16:42:00.458 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg== 2023-12-19T16:42:00.458 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.0] 2023-12-19T16:42:00.458 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig== 2023-12-19T16:42:00.458 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.1] 2023-12-19T16:42:00.458 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA== 2023-12-19T16:42:00.458 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.2] 2023-12-19T16:42:00.458 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ== 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.3] 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ== 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.x] 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA== 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b] 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg== 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d] 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw== 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.4] 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A== 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.5] 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ== 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.6] 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg== 2023-12-19T16:42:00.459 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.7] 2023-12-19T16:42:00.460 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg== 2023-12-19T16:42:00.460 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2023-12-19T16:42:00.460 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ== 2023-12-19T16:42:00.460 INFO:teuthology.orchestra.run.smithi026.stdout:[client.2] 2023-12-19T16:42:00.460 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ== 2023-12-19T16:42:00.460 INFO:teuthology.orchestra.run.smithi026.stdout:[client.1] 2023-12-19T16:42:00.460 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ== 2023-12-19T16:42:00.460 INFO:teuthology.orchestra.run.smithi026.stdout:[client.3] 2023-12-19T16:42:00.460 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ== 2023-12-19T16:42:00.471 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr.y] 2023-12-19T16:42:00.471 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg== 2023-12-19T16:42:00.471 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.a] 2023-12-19T16:42:00.471 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg== 2023-12-19T16:42:00.471 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.c] 2023-12-19T16:42:00.471 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg== 2023-12-19T16:42:00.471 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.0] 2023-12-19T16:42:00.472 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig== 2023-12-19T16:42:00.472 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.1] 2023-12-19T16:42:00.472 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA== 2023-12-19T16:42:00.472 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.2] 2023-12-19T16:42:00.472 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ== 2023-12-19T16:42:00.472 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.3] 2023-12-19T16:42:00.472 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ== 2023-12-19T16:42:00.472 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr.x] 2023-12-19T16:42:00.473 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA== 2023-12-19T16:42:00.473 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.b] 2023-12-19T16:42:00.473 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg== 2023-12-19T16:42:00.473 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.d] 2023-12-19T16:42:00.473 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw== 2023-12-19T16:42:00.473 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.4] 2023-12-19T16:42:00.473 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A== 2023-12-19T16:42:00.473 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.5] 2023-12-19T16:42:00.473 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ== 2023-12-19T16:42:00.473 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.6] 2023-12-19T16:42:00.473 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg== 2023-12-19T16:42:00.474 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.7] 2023-12-19T16:42:00.474 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg== 2023-12-19T16:42:00.474 INFO:teuthology.orchestra.run.smithi160.stdout:[client.0] 2023-12-19T16:42:00.474 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ== 2023-12-19T16:42:00.474 INFO:teuthology.orchestra.run.smithi160.stdout:[client.2] 2023-12-19T16:42:00.474 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ== 2023-12-19T16:42:00.474 INFO:teuthology.orchestra.run.smithi160.stdout:[client.1] 2023-12-19T16:42:00.474 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ== 2023-12-19T16:42:00.474 INFO:teuthology.orchestra.run.smithi160.stdout:[client.3] 2023-12-19T16:42:00.474 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ== 2023-12-19T16:42:00.475 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-19T16:42:00.505 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-19T16:42:00.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T16:42:00.576 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T16:42:00.634 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T16:42:00.638 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T16:42:00.694 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:00.698 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:00.752 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:00.756 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:00.809 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:00.813 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:00.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:00.870 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:00.924 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-19T16:42:00.928 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-19T16:42:00.981 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T16:42:00.985 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T16:42:01.042 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T16:42:01.046 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T16:42:01.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:01.106 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:01.162 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:01.165 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:01.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:01.224 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:01.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:01.283 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T16:42:01.337 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T16:42:01.341 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T16:42:01.398 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T16:42:01.402 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T16:42:01.459 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T16:42:01.463 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T16:42:01.521 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T16:42:01.525 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T16:42:01.583 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-12-19T16:42:01.583 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-12-19T16:42:01.604 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-19T16:42:01.719 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-12-19T16:42:01.740 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-12-19T16:42:01.760 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-19T16:42:01.874 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-12-19T16:42:01.896 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-12-19T16:42:01.957 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-19T16:42:02.077 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-12-19T16:42:02.098 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:42:02.102 DEBUG:teuthology.orchestra.run.smithi160:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-19T16:42:02.150 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-12-19T16:42:02.150 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-19T16:42:02.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-12-19T16:42:02.154 INFO:tasks.ceph.mon.a:Started 2023-12-19T16:42:02.154 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-19T16:42:02.154 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-12-19T16:42:02.201 INFO:tasks.ceph.mon.b:Started 2023-12-19T16:42:02.201 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-19T16:42:02.201 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-12-19T16:42:02.204 INFO:tasks.ceph.mon.c:Started 2023-12-19T16:42:02.204 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-12-19T16:42:02.205 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-19T16:42:02.205 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-12-19T16:42:02.208 INFO:tasks.ceph.mgr.y:Started 2023-12-19T16:42:02.209 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-12-19T16:42:02.209 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-12-19T16:42:02.212 INFO:tasks.ceph.mgr.x:Started 2023-12-19T16:42:02.212 INFO:tasks.ceph:Setting crush tunables to default 2023-12-19T16:42:02.212 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph osd crush tunables default 2023-12-19T16:42:02.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.345+0000 7f208c2f9640 1 Processor -- start 2023-12-19T16:42:02.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.345+0000 7f208c2f9640 1 -- start start 2023-12-19T16:42:02.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.345+0000 7f208c2f9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2084160d00 0x7f20840ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:02.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f208c2f9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20841610e0 0x7f2084161540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:02.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f208c2f9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20840cd100 con 0x7f20840d0df0 2023-12-19T16:42:02.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f208c2f9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20840cd330 con 0x7f2084160d00 2023-12-19T16:42:02.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f208c2f9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20840cd610 con 0x7f20841610e0 2023-12-19T16:42:02.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f208a095640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2084160d00 0x7f20840ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:02.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f208a095640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2084160d00 0x7f20840ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:56472/0 (socket says 172.21.15.26:56472) 2023-12-19T16:42:02.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f208a095640 1 -- 172.21.15.26:0/585838308 learned_addr learned my addr 172.21.15.26:0/585838308 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f2089894640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20841610e0 0x7f2084161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f208a095640 1 -- 172.21.15.26:0/585838308 >> v2:172.21.15.160:3300/0 conn(0x7f2084160d00 msgr2=0x7f20840ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-19T16:42:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f208a095640 1 -- 172.21.15.26:0/585838308 >> v2:172.21.15.160:3300/0 conn(0x7f2084160d00 msgr2=0x7f20840ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-19T16:42:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f208a095640 1 --2- 172.21.15.26:0/585838308 >> v2:172.21.15.160:3300/0 conn(0x7f2084160d00 0x7f20840ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-19T16:42:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f208a095640 1 --2- 172.21.15.26:0/585838308 >> v2:172.21.15.160:3300/0 conn(0x7f2084160d00 0x7f20840ccbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:42:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f2089894640 1 -- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20841610e0 msgr2=0x7f2084161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-19T16:42:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f2089894640 1 -- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20841610e0 msgr2=0x7f2084161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-19T16:42:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f2089894640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20841610e0 0x7f2084161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-19T16:42:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.349+0000 7f2089894640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20841610e0 0x7f2084161540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:42:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825251786 0 0) 0x7f20840cd100 con 0x7f20840d0df0 2023-12-19T16:42:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2070003530 con 0x7f20840d0df0 2023-12-19T16:42:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1165343645 0 0) 0x7f2070003530 con 0x7f20840d0df0 2023-12-19T16:42:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20841610e0 msgr2=0x7f2084161540 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-19T16:42:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20841610e0 0x7f2084161540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 >> v2:172.21.15.160:3300/0 conn(0x7f2084160d00 msgr2=0x7f20840ccbc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 --2- 172.21.15.26:0/585838308 >> v2:172.21.15.160:3300/0 conn(0x7f2084160d00 0x7f20840ccbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20840cd8f0 con 0x7f20840d0df0 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f207c001a20 con 0x7f20840d0df0 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 >> v1:172.21.15.26:6789/0 conn(0x7f20840d0df0 legacy=0x7f20840d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2070003980 0x7f2070003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20700047d0 0x7f2070004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20841610e0 0x7f2070005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20840cd330 con 0x7f2070003980 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20840cd610 con 0x7f20700047d0 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207000ba40 con 0x7f20841610e0 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f208a095640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2070003980 0x7f2070003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f208a896640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20841610e0 0x7f2070005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f2089894640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20700047d0 0x7f2070004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f208a095640 1 -- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20841610e0 msgr2=0x7f2070005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f208a095640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20841610e0 0x7f2070005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f208a095640 1 -- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20700047d0 msgr2=0x7f2070004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f208a095640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20700047d0 0x7f2070004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.505+0000 7f208a095640 1 -- 172.21.15.26:0/585838308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2070003530 con 0x7f2070003980 2023-12-19T16:42:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.509+0000 7f208a095640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2070003980 0x7f2070003da0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2080009100 tx=0x7f208001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8c324e3445141a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:02.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.509+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800a9650 con 0x7f2070003980 2023-12-19T16:42:02.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.509+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20800a97b0 con 0x7f2070003980 2023-12-19T16:42:02.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.509+0000 7f2089093640 1 -- 172.21.15.26:0/585838308 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800a9b60 con 0x7f2070003980 2023-12-19T16:42:02.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.509+0000 7f208c2f9640 1 -- 172.21.15.26:0/585838308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2070003980 msgr2=0x7f2070003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:02.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.509+0000 7f208c2f9640 1 --2- 172.21.15.26:0/585838308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2070003980 0x7f2070003da0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2080009100 tx=0x7f208001c560 comp rx=0 tx=0).stop 2023-12-19T16:42:02.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.509+0000 7f208a896640 1 -- 172.21.15.26:0/585838308 reap_dead start 2023-12-19T16:42:02.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.509+0000 7f208c2f9640 1 -- 172.21.15.26:0/585838308 shutdown_connections 2023-12-19T16:42:02.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.509+0000 7f208c2f9640 1 -- 172.21.15.26:0/585838308 >> 172.21.15.26:0/585838308 conn(0x7f208406ef60 msgr2=0x7f208406f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:02.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.509+0000 7f208c2f9640 1 -- 172.21.15.26:0/585838308 shutdown_connections 2023-12-19T16:42:02.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.509+0000 7f208c2f9640 1 -- 172.21.15.26:0/585838308 wait complete. 2023-12-19T16:42:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208c2f9640 1 Processor -- start 2023-12-19T16:42:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208c2f9640 1 -- start start 2023-12-19T16:42:02.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208c2f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2070003980 0x7f20840af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:02.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208c2f9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20840c4e90 0x7f208415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:02.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208c2f9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20840c5310 0x7f208415f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:02.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208c2f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20840c4c40 con 0x7f2070003980 2023-12-19T16:42:02.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208c2f9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20840c4960 con 0x7f20840c4e90 2023-12-19T16:42:02.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208c2f9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208415e170 con 0x7f20840c5310 2023-12-19T16:42:02.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208a095640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2070003980 0x7f20840af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:02.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f2089894640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20840c4e90 0x7f208415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:02.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208a095640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2070003980 0x7f20840af0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40444/0 (socket says 172.21.15.26:40444) 2023-12-19T16:42:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208a095640 1 -- 172.21.15.26:0/2986317139 learned_addr learned my addr 172.21.15.26:0/2986317139 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208a896640 1 --2- 172.21.15.26:0/2986317139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20840c5310 0x7f208415f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208a095640 1 -- 172.21.15.26:0/2986317139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20840c5310 msgr2=0x7f208415f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208a095640 1 --2- 172.21.15.26:0/2986317139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20840c5310 0x7f208415f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208a095640 1 -- 172.21.15.26:0/2986317139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20840c4e90 msgr2=0x7f208415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208a095640 1 --2- 172.21.15.26:0/2986317139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20840c4e90 0x7f208415ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208a095640 1 -- 172.21.15.26:0/2986317139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208415e450 con 0x7f2070003980 2023-12-19T16:42:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208a095640 1 --2- 172.21.15.26:0/2986317139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2070003980 0x7f20840af0d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f208001c390 tx=0x7f2070005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f207affd640 1 -- 172.21.15.26:0/2986317139 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800b7070 con 0x7f2070003980 2023-12-19T16:42:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f207affd640 1 -- 172.21.15.26:0/2986317139 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20800a9450 con 0x7f2070003980 2023-12-19T16:42:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.513+0000 7f208c2f9640 1 -- 172.21.15.26:0/2986317139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20840c4c40 con 0x7f2070003980 2023-12-19T16:42:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.517+0000 7f207affd640 1 -- 172.21.15.26:0/2986317139 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20800bc620 con 0x7f2070003980 2023-12-19T16:42:02.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.517+0000 7f208c2f9640 1 -- 172.21.15.26:0/2986317139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20840af610 con 0x7f2070003980 2023-12-19T16:42:02.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.517+0000 7f207affd640 1 -- 172.21.15.26:0/2986317139 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f20800bc780 con 0x7f2070003980 2023-12-19T16:42:02.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.517+0000 7f207affd640 1 -- 172.21.15.26:0/2986317139 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f20800b1820 con 0x7f2070003980 2023-12-19T16:42:02.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.517+0000 7f208c2f9640 1 -- 172.21.15.26:0/2986317139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2084074270 con 0x7f2070003980 2023-12-19T16:42:02.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.617+0000 7f207affd640 1 -- 172.21.15.26:0/2986317139 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f208000f040 con 0x7f2070003980 2023-12-19T16:42:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.677+0000 7f208c2f9640 1 -- 172.21.15.26:0/2986317139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f20840b2680 con 0x7f2070003980 2023-12-19T16:42:02.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.681+0000 7f207affd640 1 -- 172.21.15.26:0/2986317139 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f20800cc090 con 0x7f2070003980 2023-12-19T16:42:02.688 INFO:teuthology.orchestra.run.smithi026.stderr:adjusted tunables profile to default 2023-12-19T16:42:02.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.681+0000 7f208c2f9640 1 -- 172.21.15.26:0/2986317139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2070003980 msgr2=0x7f20840af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:02.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.681+0000 7f208c2f9640 1 --2- 172.21.15.26:0/2986317139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2070003980 0x7f20840af0d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f208001c390 tx=0x7f2070005800 comp rx=0 tx=0).stop 2023-12-19T16:42:02.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.681+0000 7f208c2f9640 1 -- 172.21.15.26:0/2986317139 shutdown_connections 2023-12-19T16:42:02.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.681+0000 7f208c2f9640 1 --2- 172.21.15.26:0/2986317139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20840c5310 0x7f208415f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:02.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.681+0000 7f208c2f9640 1 --2- 172.21.15.26:0/2986317139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20840c4e90 0x7f208415ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:02.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.681+0000 7f208c2f9640 1 --2- 172.21.15.26:0/2986317139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2070003980 0x7f20840af0d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:02.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.681+0000 7f208c2f9640 1 -- 172.21.15.26:0/2986317139 >> 172.21.15.26:0/2986317139 conn(0x7f208406ef60 msgr2=0x7f20840cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:02.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.681+0000 7f208c2f9640 1 -- 172.21.15.26:0/2986317139 shutdown_connections 2023-12-19T16:42:02.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:02.681+0000 7f208c2f9640 1 -- 172.21.15.26:0/2986317139 wait complete. 2023-12-19T16:42:02.709 INFO:tasks.ceph:check_enable_crimson: False 2023-12-19T16:42:02.709 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-12-19T16:42:02.709 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:42:02.709 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-12-19T16:42:02.730 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:42:02.730 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-12-19T16:42:02.754 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:02.747+0000 7f1a90249180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:42:02.794 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:42:02.794 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-12-19T16:42:02.800 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:02.793+0000 7fbfae177180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:42:02.821 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:42:02.821 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-12-19T16:42:02.882 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:02.882 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-12-19T16:42:02.903 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:02.903 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-12-19T16:42:02.966 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:02.966 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-12-19T16:42:03.029 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:03.029 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-12-19T16:42:03.090 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new 17f9dbf7-85f6-4164-aa85-d87685bda626 0 2023-12-19T16:42:03.270 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.263+0000 7f02cb725640 1 Processor -- start 2023-12-19T16:42:03.270 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.263+0000 7f02cb725640 1 -- start start 2023-12-19T16:42:03.272 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.267+0000 7f02cb725640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f02c40b4600 0x7f02c40b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.272 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.267+0000 7f02cb725640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02c40b8940 0x7f02c40d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.272 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.267+0000 7f02cb725640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c40d21d0 con 0x7f02c40d1120 2023-12-19T16:42:03.272 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.267+0000 7f02cb725640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c40b4f20 con 0x7f02c40b4600 2023-12-19T16:42:03.272 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.267+0000 7f02cb725640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c40d15c0 con 0x7f02c40b8940 2023-12-19T16:42:03.272 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.267+0000 7f02c8cc0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02c40b8940 0x7f02c40d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:03.272 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.267+0000 7f02c8cc0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02c40b8940 0x7f02c40d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:37858/0 (socket says 172.21.15.160:37858) 2023-12-19T16:42:03.272 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.267+0000 7f02c8cc0640 1 -- 172.21.15.160:0/2867217998 learned_addr learned my addr 172.21.15.160:0/2867217998 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:03.272 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.267+0000 7f02c8cc0640 1 -- 172.21.15.160:0/2867217998 >> v2:172.21.15.26:3300/0 conn(0x7f02c40b4600 msgr2=0x7f02c40b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:03.272 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.267+0000 7f02c8cc0640 1 --2- 172.21.15.160:0/2867217998 >> v2:172.21.15.26:3300/0 conn(0x7f02c40b4600 0x7f02c40b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.272 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.267+0000 7f02c8cc0640 1 -- 172.21.15.160:0/2867217998 >> v1:172.21.15.160:6789/0 conn(0x7f02c40d1120 legacy=0x7f02c4069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:03.272 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.267+0000 7f02c8cc0640 1 -- 172.21.15.160:0/2867217998 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c40d18a0 con 0x7f02c40b8940 2023-12-19T16:42:03.274 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02c8cc0640 1 --2- 172.21.15.160:0/2867217998 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02c40b8940 0x7f02c40d0be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009870 tx=0x7f02b401ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76340c8fcd7b131e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:03.275 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02bbfff640 1 -- 172.21.15.160:0/2867217998 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40a89b0 con 0x7f02c40b8940 2023-12-19T16:42:03.275 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02bbfff640 1 -- 172.21.15.160:0/2867217998 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b40a8b10 con 0x7f02c40b8940 2023-12-19T16:42:03.275 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02bbfff640 1 -- 172.21.15.160:0/2867217998 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40a8ea0 con 0x7f02c40b8940 2023-12-19T16:42:03.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02cb725640 1 -- 172.21.15.160:0/2867217998 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02c40b8940 msgr2=0x7f02c40d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02cb725640 1 --2- 172.21.15.160:0/2867217998 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02c40b8940 0x7f02c40d0be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009870 tx=0x7f02b401ecf0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02cb725640 1 -- 172.21.15.160:0/2867217998 shutdown_connections 2023-12-19T16:42:03.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02cb725640 1 --2- 172.21.15.160:0/2867217998 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02c40b8940 0x7f02c40d0be0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02cb725640 1 --2- 172.21.15.160:0/2867217998 >> v2:172.21.15.26:3300/0 conn(0x7f02c40b4600 0x7f02c40b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02cb725640 1 -- 172.21.15.160:0/2867217998 >> 172.21.15.160:0/2867217998 conn(0x7f02c4065ee0 msgr2=0x7f02c40cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:03.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02cb725640 1 -- 172.21.15.160:0/2867217998 shutdown_connections 2023-12-19T16:42:03.276 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02cb725640 1 -- 172.21.15.160:0/2867217998 wait complete. 2023-12-19T16:42:03.277 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02cb725640 1 Processor -- start 2023-12-19T16:42:03.278 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.271+0000 7f02cb725640 1 -- start start 2023-12-19T16:42:03.279 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02cb725640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c40b4600 0x7f02c414e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.279 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02cb725640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02c40b8940 0x7f02c415d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.279 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02cb725640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02c40d1120 0x7f02c415f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.280 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02cb725640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c415dd10 con 0x7f02c40b4600 2023-12-19T16:42:03.280 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02cb725640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c415dff0 con 0x7f02c40d1120 2023-12-19T16:42:03.280 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02cb725640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c415e2d0 con 0x7f02c40b8940 2023-12-19T16:42:03.280 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02c8cc0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02c40b8940 0x7f02c415d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:03.280 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02c9cc2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02c40d1120 0x7f02c415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:03.280 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02c9cc2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02c40d1120 0x7f02c415f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60770/0 (socket says 172.21.15.160:60770) 2023-12-19T16:42:03.280 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02c9cc2640 1 -- 172.21.15.160:0/3492098149 learned_addr learned my addr 172.21.15.160:0/3492098149 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:03.280 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02c94c1640 1 --2- 172.21.15.160:0/3492098149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c40b4600 0x7f02c414e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:03.280 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02c8cc0640 1 -- 172.21.15.160:0/3492098149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02c40d1120 msgr2=0x7f02c415f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.280 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02c8cc0640 1 --2- 172.21.15.160:0/3492098149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02c40d1120 0x7f02c415f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.280 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02c8cc0640 1 -- 172.21.15.160:0/3492098149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c40b4600 msgr2=0x7f02c414e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.280 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02c8cc0640 1 --2- 172.21.15.160:0/3492098149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c40b4600 0x7f02c414e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.280 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02c8cc0640 1 -- 172.21.15.160:0/3492098149 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b40094d0 con 0x7f02c40b8940 2023-12-19T16:42:03.281 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02c9cc2640 1 --2- 172.21.15.160:0/3492098149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02c40d1120 0x7f02c415f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:03.281 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02c8cc0640 1 --2- 172.21.15.160:0/3492098149 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02c40b8940 0x7f02c415d3a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f02b40af000 tx=0x7f02b401ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:03.282 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02b9ffb640 1 -- 172.21.15.160:0/3492098149 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40bf070 con 0x7f02c40b8940 2023-12-19T16:42:03.282 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02b9ffb640 1 -- 172.21.15.160:0/3492098149 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b40b0c80 con 0x7f02c40b8940 2023-12-19T16:42:03.282 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02cb725640 1 -- 172.21.15.160:0/3492098149 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c415e5b0 con 0x7f02c40b8940 2023-12-19T16:42:03.282 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02b9ffb640 1 -- 172.21.15.160:0/3492098149 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40bb470 con 0x7f02c40b8940 2023-12-19T16:42:03.282 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02cb725640 1 -- 172.21.15.160:0/3492098149 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c415edb0 con 0x7f02c40b8940 2023-12-19T16:42:03.282 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02b9ffb640 1 -- 172.21.15.160:0/3492098149 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f02b40b7070 con 0x7f02c40b8940 2023-12-19T16:42:03.282 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02b9ffb640 1 -- 172.21.15.160:0/3492098149 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f02b40b98f0 con 0x7f02c40b8940 2023-12-19T16:42:03.282 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.275+0000 7f02cb725640 1 -- 172.21.15.160:0/3492098149 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c4074270 con 0x7f02c40b8940 2023-12-19T16:42:03.304 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.299+0000 7f02b9ffb640 1 -- 172.21.15.160:0/3492098149 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f02b40b5070 con 0x7f02c40b8940 2023-12-19T16:42:03.308 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:03.303+0000 7f1a90249180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:42:03.350 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:03.341+0000 7fbfae177180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:42:03.369 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.363+0000 7f02cb725640 1 -- 172.21.15.160:0/3492098149 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd new", "uuid": "17f9dbf7-85f6-4164-aa85-d87685bda626", "id": 0} v 0) v1 -- 0x7f02c415e410 con 0x7f02c40b8940 2023-12-19T16:42:03.374 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.371+0000 7f02b9ffb640 1 -- 172.21.15.160:0/3492098149 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "17f9dbf7-85f6-4164-aa85-d87685bda626", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f02c415e410 con 0x7f02c40b8940 2023-12-19T16:42:03.374 INFO:teuthology.orchestra.run.smithi160.stdout:0 2023-12-19T16:42:03.376 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.371+0000 7f02cb725640 1 -- 172.21.15.160:0/3492098149 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02c40b8940 msgr2=0x7f02c415d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.376 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.371+0000 7f02cb725640 1 --2- 172.21.15.160:0/3492098149 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02c40b8940 0x7f02c415d3a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f02b40af000 tx=0x7f02b401ba90 comp rx=0 tx=0).stop 2023-12-19T16:42:03.376 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.371+0000 7f02cb725640 1 -- 172.21.15.160:0/3492098149 shutdown_connections 2023-12-19T16:42:03.376 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.371+0000 7f02cb725640 1 --2- 172.21.15.160:0/3492098149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02c40d1120 0x7f02c415f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.376 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.371+0000 7f02cb725640 1 --2- 172.21.15.160:0/3492098149 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02c40b8940 0x7f02c415d3a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.376 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.371+0000 7f02cb725640 1 --2- 172.21.15.160:0/3492098149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02c40b4600 0x7f02c414e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.376 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.371+0000 7f02cb725640 1 -- 172.21.15.160:0/3492098149 >> 172.21.15.160:0/3492098149 conn(0x7f02c4065ee0 msgr2=0x7f02c40b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:03.376 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.371+0000 7f02cb725640 1 -- 172.21.15.160:0/3492098149 shutdown_connections 2023-12-19T16:42:03.376 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.371+0000 7f02cb725640 1 -- 172.21.15.160:0/3492098149 wait complete. 2023-12-19T16:42:03.395 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new 2b12025e-9cb7-43d2-a3a0-5563968aec5a 1 2023-12-19T16:42:03.509 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:03.503+0000 7f1a90249180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:42:03.533 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.527+0000 7f3608eb6640 1 Processor -- start 2023-12-19T16:42:03.534 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.527+0000 7f3608eb6640 1 -- start start 2023-12-19T16:42:03.535 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f3608eb6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f36040b4600 0x7f36040b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.535 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f3608eb6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36040d21d0 con 0x7f36040d2660 2023-12-19T16:42:03.535 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f3608eb6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36040b4f20 con 0x7f36040b8940 2023-12-19T16:42:03.535 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f3608eb6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36040d12f0 con 0x7f36040b4600 2023-12-19T16:42:03.535 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f3601d9b640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f36040b8940 0x7f36040d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:58224/0 (socket says 172.21.15.160:58224) 2023-12-19T16:42:03.535 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f3601d9b640 1 -- 172.21.15.160:0/1917295463 learned_addr learned my addr 172.21.15.160:0/1917295463 (peer_addr_for_me v1:172.21.15.160:0/0) 2023-12-19T16:42:03.535 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f360259c640 1 --2- 172.21.15.160:0/1917295463 >> v2:172.21.15.26:3300/0 conn(0x7f36040b4600 0x7f36040b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:03.536 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f360159a640 1 -- 172.21.15.160:0/1917295463 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92847781 0 0) 0x7f36040b4f20 con 0x7f36040b8940 2023-12-19T16:42:03.536 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f360159a640 1 -- 172.21.15.160:0/1917295463 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35ec003530 con 0x7f36040b8940 2023-12-19T16:42:03.536 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f360159a640 1 -- 172.21.15.160:0/1917295463 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 41473331 0 0) 0x7f36040d21d0 con 0x7f36040d2660 2023-12-19T16:42:03.536 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f360159a640 1 -- 172.21.15.160:0/1917295463 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36040b4f20 con 0x7f36040d2660 2023-12-19T16:42:03.536 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f360259c640 1 -- 172.21.15.160:0/1917295463 >> v1:172.21.15.160:6789/0 conn(0x7f36040b8940 legacy=0x7f36040d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.536 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f360259c640 1 -- 172.21.15.160:0/1917295463 >> v1:172.21.15.26:6789/0 conn(0x7f36040d2660 legacy=0x7f36040b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.536 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.531+0000 7f360259c640 1 -- 172.21.15.160:0/1917295463 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36040d15c0 con 0x7f36040b4600 2023-12-19T16:42:03.538 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.535+0000 7f360259c640 1 --2- 172.21.15.160:0/1917295463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36040b4600 0x7f36040b49e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f35f40097c0 tx=0x7f35f401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb7d8f5212b190c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:03.539 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.535+0000 7f360159a640 1 -- 172.21.15.160:0/1917295463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40a8970 con 0x7f36040b4600 2023-12-19T16:42:03.539 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.535+0000 7f360159a640 1 -- 172.21.15.160:0/1917295463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f40a8ad0 con 0x7f36040b4600 2023-12-19T16:42:03.539 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.535+0000 7f360159a640 1 -- 172.21.15.160:0/1917295463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40a8e60 con 0x7f36040b4600 2023-12-19T16:42:03.539 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.535+0000 7f3608eb6640 1 -- 172.21.15.160:0/1917295463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36040b4600 msgr2=0x7f36040b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.539 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.535+0000 7f3608eb6640 1 --2- 172.21.15.160:0/1917295463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36040b4600 0x7f36040b49e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f35f40097c0 tx=0x7f35f401ec70 comp rx=0 tx=0).stop 2023-12-19T16:42:03.540 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.535+0000 7f3608eb6640 1 -- 172.21.15.160:0/1917295463 shutdown_connections 2023-12-19T16:42:03.540 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.535+0000 7f3608eb6640 1 --2- 172.21.15.160:0/1917295463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36040b4600 0x7f36040b49e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.540 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.535+0000 7f3608eb6640 1 -- 172.21.15.160:0/1917295463 >> 172.21.15.160:0/1917295463 conn(0x7f3604065ee0 msgr2=0x7f36040cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:03.540 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.535+0000 7f3608eb6640 1 -- 172.21.15.160:0/1917295463 shutdown_connections 2023-12-19T16:42:03.540 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.535+0000 7f3608eb6640 1 -- 172.21.15.160:0/1917295463 wait complete. 2023-12-19T16:42:03.542 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.535+0000 7f3608eb6640 1 Processor -- start 2023-12-19T16:42:03.542 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3608eb6640 1 -- start start 2023-12-19T16:42:03.543 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3608eb6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36040b4600 0x7f3604155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.543 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3608eb6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36040b8940 0x7f3604151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.544 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3608eb6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36040d2660 0x7f3604158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.544 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3608eb6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36040b2f20 con 0x7f36040d2660 2023-12-19T16:42:03.544 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3608eb6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36040b3200 con 0x7f36040b4600 2023-12-19T16:42:03.544 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3608eb6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36040b34e0 con 0x7f36040b8940 2023-12-19T16:42:03.544 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3601d9b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36040b8940 0x7f3604151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:03.544 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f360259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36040b4600 0x7f3604155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:03.544 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3601d9b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36040b8940 0x7f3604151f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:37882/0 (socket says 172.21.15.160:37882) 2023-12-19T16:42:03.544 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3601d9b640 1 -- 172.21.15.160:0/612384596 learned_addr learned my addr 172.21.15.160:0/612384596 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:03.544 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3602d9d640 1 --2- 172.21.15.160:0/612384596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36040d2660 0x7f3604158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:03.547 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3601d9b640 1 -- 172.21.15.160:0/612384596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36040b4600 msgr2=0x7f3604155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.547 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3601d9b640 1 --2- 172.21.15.160:0/612384596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36040b4600 0x7f3604155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.547 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3601d9b640 1 -- 172.21.15.160:0/612384596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36040d2660 msgr2=0x7f3604158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.547 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3601d9b640 1 --2- 172.21.15.160:0/612384596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36040d2660 0x7f3604158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.547 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3601d9b640 1 -- 172.21.15.160:0/612384596 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f4009420 con 0x7f36040b8940 2023-12-19T16:42:03.547 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f360259c640 1 --2- 172.21.15.160:0/612384596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36040b4600 0x7f3604155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:03.547 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3601d9b640 1 --2- 172.21.15.160:0/612384596 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36040b8940 0x7f3604151f70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f35f800c4d0 tx=0x7f35f800c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:03.547 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f35f2ffd640 1 -- 172.21.15.160:0/612384596 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8005cf0 con 0x7f36040b8940 2023-12-19T16:42:03.547 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f35f2ffd640 1 -- 172.21.15.160:0/612384596 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f8005e50 con 0x7f36040b8940 2023-12-19T16:42:03.547 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3608eb6640 1 -- 172.21.15.160:0/612384596 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36040b37c0 con 0x7f36040b8940 2023-12-19T16:42:03.548 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f35f2ffd640 1 -- 172.21.15.160:0/612384596 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f8007270 con 0x7f36040b8940 2023-12-19T16:42:03.548 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3608eb6640 1 -- 172.21.15.160:0/612384596 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36041524b0 con 0x7f36040b8940 2023-12-19T16:42:03.548 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f35f2ffd640 1 -- 172.21.15.160:0/612384596 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f35f801c440 con 0x7f36040b8940 2023-12-19T16:42:03.548 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f35f2ffd640 1 -- 172.21.15.160:0/612384596 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f35f80127b0 con 0x7f36040b8940 2023-12-19T16:42:03.548 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.539+0000 7f3608eb6640 1 -- 172.21.15.160:0/612384596 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3604074270 con 0x7f36040b8940 2023-12-19T16:42:03.548 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.543+0000 7f35f2ffd640 1 -- 172.21.15.160:0/612384596 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f35f8021060 con 0x7f36040b8940 2023-12-19T16:42:03.556 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:03.549+0000 7fbfae177180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:42:03.614 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.607+0000 7f3608eb6640 1 -- 172.21.15.160:0/612384596 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2b12025e-9cb7-43d2-a3a0-5563968aec5a", "id": 1} v 0) v1 -- 0x7f36040b3e70 con 0x7f36040b8940 2023-12-19T16:42:03.619 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.615+0000 7f35f2ffd640 1 -- 172.21.15.160:0/612384596 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2b12025e-9cb7-43d2-a3a0-5563968aec5a", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f35f8017070 con 0x7f36040b8940 2023-12-19T16:42:03.619 INFO:teuthology.orchestra.run.smithi160.stdout:1 2023-12-19T16:42:03.620 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.615+0000 7f3608eb6640 1 -- 172.21.15.160:0/612384596 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36040b8940 msgr2=0x7f3604151f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.620 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.615+0000 7f3608eb6640 1 --2- 172.21.15.160:0/612384596 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36040b8940 0x7f3604151f70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f35f800c4d0 tx=0x7f35f800c990 comp rx=0 tx=0).stop 2023-12-19T16:42:03.620 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.615+0000 7f3608eb6640 1 -- 172.21.15.160:0/612384596 shutdown_connections 2023-12-19T16:42:03.620 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.615+0000 7f3608eb6640 1 --2- 172.21.15.160:0/612384596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36040d2660 0x7f3604158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.620 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.615+0000 7f3608eb6640 1 --2- 172.21.15.160:0/612384596 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36040b8940 0x7f3604151f70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.620 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.615+0000 7f3608eb6640 1 --2- 172.21.15.160:0/612384596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36040b4600 0x7f3604155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.620 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.615+0000 7f3608eb6640 1 -- 172.21.15.160:0/612384596 >> 172.21.15.160:0/612384596 conn(0x7f3604065ee0 msgr2=0x7f36040b78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:03.620 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.615+0000 7f3608eb6640 1 -- 172.21.15.160:0/612384596 shutdown_connections 2023-12-19T16:42:03.620 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.615+0000 7f3608eb6640 1 -- 172.21.15.160:0/612384596 wait complete. 2023-12-19T16:42:03.638 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new a7110e7e-9060-4ad4-9ea8-93108111a6c9 2 2023-12-19T16:42:03.773 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.767+0000 7f529498f640 1 Processor -- start 2023-12-19T16:42:03.773 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.767+0000 7f529498f640 1 -- start start 2023-12-19T16:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f529498f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f52900b4600 0x7f52900b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f529498f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52900d2660 0x7f52900b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f529498f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52900d21d0 con 0x7f52900b8940 2023-12-19T16:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f529498f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52900b4f20 con 0x7f52900b4600 2023-12-19T16:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f529498f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52900d12f0 con 0x7f52900d2660 2023-12-19T16:42:03.775 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f528e59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f52900b4600 0x7f52900b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:03.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f528ed9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52900d2660 0x7f52900b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:03.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f528e59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f52900b4600 0x7f52900b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60782/0 (socket says 172.21.15.160:60782) 2023-12-19T16:42:03.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f528e59c640 1 -- 172.21.15.160:0/3613073894 learned_addr learned my addr 172.21.15.160:0/3613073894 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:03.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f528e59c640 1 -- 172.21.15.160:0/3613073894 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52900d2660 msgr2=0x7f52900b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:03.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f528e59c640 1 --2- 172.21.15.160:0/3613073894 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52900d2660 0x7f52900b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f528e59c640 1 -- 172.21.15.160:0/3613073894 >> v1:172.21.15.160:6789/0 conn(0x7f52900b8940 legacy=0x7f52900d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f528e59c640 1 -- 172.21.15.160:0/3613073894 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52900d15c0 con 0x7f52900b4600 2023-12-19T16:42:03.776 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.771+0000 7f528ed9d640 1 --2- 172.21.15.160:0/3613073894 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52900d2660 0x7f52900b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:03.778 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f528e59c640 1 --2- 172.21.15.160:0/3613073894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52900b4600 0x7f52900b49e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5280009870 tx=0x7f528001ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ddccef47fbc46a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:03.778 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f528d59a640 1 -- 172.21.15.160:0/3613073894 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52800a8a30 con 0x7f52900b4600 2023-12-19T16:42:03.778 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f528d59a640 1 -- 172.21.15.160:0/3613073894 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52800a8b90 con 0x7f52900b4600 2023-12-19T16:42:03.778 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f528d59a640 1 -- 172.21.15.160:0/3613073894 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52800b0440 con 0x7f52900b4600 2023-12-19T16:42:03.779 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f529498f640 1 -- 172.21.15.160:0/3613073894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52900b4600 msgr2=0x7f52900b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.779 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f529498f640 1 --2- 172.21.15.160:0/3613073894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52900b4600 0x7f52900b49e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5280009870 tx=0x7f528001ed30 comp rx=0 tx=0).stop 2023-12-19T16:42:03.780 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f529498f640 1 -- 172.21.15.160:0/3613073894 shutdown_connections 2023-12-19T16:42:03.780 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f529498f640 1 --2- 172.21.15.160:0/3613073894 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52900d2660 0x7f52900b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.780 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f529498f640 1 --2- 172.21.15.160:0/3613073894 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52900b4600 0x7f52900b49e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.780 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f529498f640 1 -- 172.21.15.160:0/3613073894 >> 172.21.15.160:0/3613073894 conn(0x7f5290065ee0 msgr2=0x7f52900cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:03.780 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f529498f640 1 -- 172.21.15.160:0/3613073894 shutdown_connections 2023-12-19T16:42:03.780 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f529498f640 1 -- 172.21.15.160:0/3613073894 wait complete. 2023-12-19T16:42:03.781 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f529498f640 1 Processor -- start 2023-12-19T16:42:03.781 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.775+0000 7f529498f640 1 -- start start 2023-12-19T16:42:03.783 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f529498f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52900b4600 0x7f52901574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.783 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f529498f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52900b8940 0x7f52901563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.783 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f529498f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52900d2660 0x7f5290159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:03.783 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f529498f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5290156930 con 0x7f52900b4600 2023-12-19T16:42:03.783 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f529498f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5290159ea0 con 0x7f52900d2660 2023-12-19T16:42:03.783 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f529498f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5290157ba0 con 0x7f52900b8940 2023-12-19T16:42:03.784 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f528dd9b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52900b8940 0x7f52901563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:03.784 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f528ed9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52900d2660 0x7f5290159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:03.784 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f528dd9b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52900b8940 0x7f52901563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:37896/0 (socket says 172.21.15.160:37896) 2023-12-19T16:42:03.784 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f528dd9b640 1 -- 172.21.15.160:0/3606058584 learned_addr learned my addr 172.21.15.160:0/3606058584 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:03.784 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f528e59c640 1 --2- 172.21.15.160:0/3606058584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52900b4600 0x7f52901574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:03.784 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f528dd9b640 1 -- 172.21.15.160:0/3606058584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52900d2660 msgr2=0x7f5290159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.784 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f528dd9b640 1 --2- 172.21.15.160:0/3606058584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52900d2660 0x7f5290159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.784 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f528dd9b640 1 -- 172.21.15.160:0/3606058584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52900b4600 msgr2=0x7f52901574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.784 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f528dd9b640 1 --2- 172.21.15.160:0/3606058584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52900b4600 0x7f52901574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.784 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f528dd9b640 1 -- 172.21.15.160:0/3606058584 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5288009a70 con 0x7f52900b8940 2023-12-19T16:42:03.784 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f528ed9d640 1 --2- 172.21.15.160:0/3606058584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52900d2660 0x7f5290159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:03.784 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f528dd9b640 1 --2- 172.21.15.160:0/3606058584 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52900b8940 0x7f52901563f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f528400c500 tx=0x7f528400c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:03.785 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f527effd640 1 -- 172.21.15.160:0/3606058584 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52840052d0 con 0x7f52900b8940 2023-12-19T16:42:03.785 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f527effd640 1 -- 172.21.15.160:0/3606058584 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284005430 con 0x7f52900b8940 2023-12-19T16:42:03.785 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f527effd640 1 -- 172.21.15.160:0/3606058584 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284003840 con 0x7f52900b8940 2023-12-19T16:42:03.785 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f529498f640 1 -- 172.21.15.160:0/3606058584 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52800094d0 con 0x7f52900b8940 2023-12-19T16:42:03.786 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f529498f640 1 -- 172.21.15.160:0/3606058584 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52900cb100 con 0x7f52900b8940 2023-12-19T16:42:03.786 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f527effd640 1 -- 172.21.15.160:0/3606058584 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5284003ca0 con 0x7f52900b8940 2023-12-19T16:42:03.786 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f527effd640 1 -- 172.21.15.160:0/3606058584 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f52840119b0 con 0x7f52900b8940 2023-12-19T16:42:03.786 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.779+0000 7f529498f640 1 -- 172.21.15.160:0/3606058584 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52900d1b30 con 0x7f52900b8940 2023-12-19T16:42:03.787 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.783+0000 7f527effd640 1 -- 172.21.15.160:0/3606058584 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f5284017070 con 0x7f52900b8940 2023-12-19T16:42:03.852 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.847+0000 7f529498f640 1 -- 172.21.15.160:0/3606058584 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a7110e7e-9060-4ad4-9ea8-93108111a6c9", "id": 2} v 0) v1 -- 0x7f5290074270 con 0x7f52900b8940 2023-12-19T16:42:03.858 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.851+0000 7f527effd640 1 -- 172.21.15.160:0/3606058584 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a7110e7e-9060-4ad4-9ea8-93108111a6c9", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5284011e50 con 0x7f52900b8940 2023-12-19T16:42:03.858 INFO:teuthology.orchestra.run.smithi160.stdout:2 2023-12-19T16:42:03.859 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.855+0000 7f529498f640 1 -- 172.21.15.160:0/3606058584 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52900b8940 msgr2=0x7f52901563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:03.859 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.855+0000 7f529498f640 1 --2- 172.21.15.160:0/3606058584 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52900b8940 0x7f52901563f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f528400c500 tx=0x7f528400c900 comp rx=0 tx=0).stop 2023-12-19T16:42:03.859 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.855+0000 7f529498f640 1 -- 172.21.15.160:0/3606058584 shutdown_connections 2023-12-19T16:42:03.859 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.855+0000 7f529498f640 1 --2- 172.21.15.160:0/3606058584 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52900d2660 0x7f5290159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.859 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.855+0000 7f529498f640 1 --2- 172.21.15.160:0/3606058584 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52900b8940 0x7f52901563f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.859 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.855+0000 7f529498f640 1 --2- 172.21.15.160:0/3606058584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52900b4600 0x7f52901574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:03.859 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.855+0000 7f529498f640 1 -- 172.21.15.160:0/3606058584 >> 172.21.15.160:0/3606058584 conn(0x7f5290065ee0 msgr2=0x7f5290069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:03.859 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.855+0000 7f529498f640 1 -- 172.21.15.160:0/3606058584 shutdown_connections 2023-12-19T16:42:03.859 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:03.855+0000 7f529498f640 1 -- 172.21.15.160:0/3606058584 wait complete. 2023-12-19T16:42:03.877 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new cfa611d8-cd56-4ce2-b72b-7e3a70ed1886 3 2023-12-19T16:42:04.013 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.007+0000 7f34914fc640 1 Processor -- start 2023-12-19T16:42:04.014 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.007+0000 7f34914fc640 1 -- start start 2023-12-19T16:42:04.015 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.011+0000 7f34914fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f348c0b8940 0x7f348c0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.015 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.011+0000 7f34914fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f348c0d1120 0x7f348c0b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.015 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.011+0000 7f34914fc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348c0d21d0 con 0x7f348c0b4600 2023-12-19T16:42:04.015 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.011+0000 7f34914fc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c0d15c0 con 0x7f348c0d1120 2023-12-19T16:42:04.015 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.011+0000 7f34914fc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c0d18a0 con 0x7f348c0b8940 2023-12-19T16:42:04.015 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.011+0000 7f348a7fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f348c0b8940 0x7f348c0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.015 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.011+0000 7f348a7fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f348c0b8940 0x7f348c0d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60814/0 (socket says 172.21.15.160:60814) 2023-12-19T16:42:04.015 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.011+0000 7f348a7fc640 1 -- 172.21.15.160:0/1740581947 learned_addr learned my addr 172.21.15.160:0/1740581947 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:04.015 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.011+0000 7f348b7fe640 1 --2- 172.21.15.160:0/1740581947 >> v2:172.21.15.26:3300/0 conn(0x7f348c0d1120 0x7f348c0b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.016 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.011+0000 7f348a7fc640 1 -- 172.21.15.160:0/1740581947 >> v2:172.21.15.26:3300/0 conn(0x7f348c0d1120 msgr2=0x7f348c0b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:04.016 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.011+0000 7f348a7fc640 1 --2- 172.21.15.160:0/1740581947 >> v2:172.21.15.26:3300/0 conn(0x7f348c0d1120 0x7f348c0b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.016 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.011+0000 7f348a7fc640 1 -- 172.21.15.160:0/1740581947 >> v1:172.21.15.26:6789/0 conn(0x7f348c0b4600 legacy=0x7f348c0b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.016 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.011+0000 7f348a7fc640 1 -- 172.21.15.160:0/1740581947 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c0d1b80 con 0x7f348c0b8940 2023-12-19T16:42:04.018 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f348a7fc640 1 --2- 172.21.15.160:0/1740581947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f348c0b8940 0x7f348c0d0be0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f348c0d3c20 tx=0x7f348001ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=decb844c0b8ad8a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:04.018 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f3489ffb640 1 -- 172.21.15.160:0/1740581947 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34800a89f0 con 0x7f348c0b8940 2023-12-19T16:42:04.018 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f3489ffb640 1 -- 172.21.15.160:0/1740581947 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34800a8b50 con 0x7f348c0b8940 2023-12-19T16:42:04.019 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f3489ffb640 1 -- 172.21.15.160:0/1740581947 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34800b0440 con 0x7f348c0b8940 2023-12-19T16:42:04.019 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f34914fc640 1 -- 172.21.15.160:0/1740581947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f348c0b8940 msgr2=0x7f348c0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.020 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f34914fc640 1 --2- 172.21.15.160:0/1740581947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f348c0b8940 0x7f348c0d0be0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f348c0d3c20 tx=0x7f348001ecf0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.020 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f34914fc640 1 -- 172.21.15.160:0/1740581947 shutdown_connections 2023-12-19T16:42:04.020 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f34914fc640 1 --2- 172.21.15.160:0/1740581947 >> v2:172.21.15.26:3300/0 conn(0x7f348c0d1120 0x7f348c0b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.020 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f34914fc640 1 --2- 172.21.15.160:0/1740581947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f348c0b8940 0x7f348c0d0be0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.020 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f34914fc640 1 -- 172.21.15.160:0/1740581947 >> 172.21.15.160:0/1740581947 conn(0x7f348c065ee0 msgr2=0x7f348c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:04.020 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f34914fc640 1 -- 172.21.15.160:0/1740581947 shutdown_connections 2023-12-19T16:42:04.020 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f34914fc640 1 -- 172.21.15.160:0/1740581947 wait complete. 2023-12-19T16:42:04.021 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f34914fc640 1 Processor -- start 2023-12-19T16:42:04.021 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.015+0000 7f34914fc640 1 -- start start 2023-12-19T16:42:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34914fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f348c0b4600 0x7f348c15f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34914fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f348c0b8940 0x7f348c0b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34914fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f348c0d1120 0x7f348c1f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34914fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c15ff10 con 0x7f348c0d1120 2023-12-19T16:42:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34914fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0cad90 con 0x7f348c0b8940 2023-12-19T16:42:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34914fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0ca210 con 0x7f348c0b4600 2023-12-19T16:42:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348affd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f348c0b4600 0x7f348c15f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f348c0b8940 0x7f348c0b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f348c0b8940 0x7f348c0b6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60818/0 (socket says 172.21.15.160:60818) 2023-12-19T16:42:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f348c0d1120 0x7f348c1f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348a7fc640 1 -- 172.21.15.160:0/770050287 learned_addr learned my addr 172.21.15.160:0/770050287 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348affd640 1 --2- 172.21.15.160:0/770050287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f348c0b4600 0x7f348c15f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:37912/0 (socket says 172.21.15.160:37912) 2023-12-19T16:42:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348affd640 1 -- 172.21.15.160:0/770050287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f348c0b8940 msgr2=0x7f348c0b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348affd640 1 --2- 172.21.15.160:0/770050287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f348c0b8940 0x7f348c0b6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348affd640 1 -- 172.21.15.160:0/770050287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f348c0d1120 msgr2=0x7f348c1f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348affd640 1 --2- 172.21.15.160:0/770050287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f348c0d1120 0x7f348c1f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348affd640 1 -- 172.21.15.160:0/770050287 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800094d0 con 0x7f348c0b4600 2023-12-19T16:42:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348a7fc640 1 --2- 172.21.15.160:0/770050287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f348c0b8940 0x7f348c0b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348b7fe640 1 --2- 172.21.15.160:0/770050287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f348c0d1120 0x7f348c1f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f348affd640 1 --2- 172.21.15.160:0/770050287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f348c0b4600 0x7f348c15f5d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f347c000f70 tx=0x7f347c00c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34737fe640 1 -- 172.21.15.160:0/770050287 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c0052d0 con 0x7f348c0b4600 2023-12-19T16:42:04.026 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34737fe640 1 -- 172.21.15.160:0/770050287 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c005430 con 0x7f348c0b4600 2023-12-19T16:42:04.026 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34914fc640 1 -- 172.21.15.160:0/770050287 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348c0ca4f0 con 0x7f348c0b4600 2023-12-19T16:42:04.026 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34737fe640 1 -- 172.21.15.160:0/770050287 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c003840 con 0x7f348c0b4600 2023-12-19T16:42:04.026 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34914fc640 1 -- 172.21.15.160:0/770050287 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c0caab0 con 0x7f348c0b4600 2023-12-19T16:42:04.026 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34737fe640 1 -- 172.21.15.160:0/770050287 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f347c003ca0 con 0x7f348c0b4600 2023-12-19T16:42:04.026 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34737fe640 1 -- 172.21.15.160:0/770050287 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f347c011a20 con 0x7f348c0b4600 2023-12-19T16:42:04.026 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.019+0000 7f34914fc640 1 -- 172.21.15.160:0/770050287 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348c074270 con 0x7f348c0b4600 2023-12-19T16:42:04.027 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.023+0000 7f34737fe640 1 -- 172.21.15.160:0/770050287 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f347c021060 con 0x7f348c0b4600 2023-12-19T16:42:04.093 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.087+0000 7f34914fc640 1 -- 172.21.15.160:0/770050287 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cfa611d8-cd56-4ce2-b72b-7e3a70ed1886", "id": 3} v 0) v1 -- 0x7f348c0cef00 con 0x7f348c0b4600 2023-12-19T16:42:04.098 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.091+0000 7f34737fe640 1 -- 172.21.15.160:0/770050287 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cfa611d8-cd56-4ce2-b72b-7e3a70ed1886", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f347c017070 con 0x7f348c0b4600 2023-12-19T16:42:04.098 INFO:teuthology.orchestra.run.smithi160.stdout:3 2023-12-19T16:42:04.099 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.095+0000 7f34914fc640 1 -- 172.21.15.160:0/770050287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f348c0b4600 msgr2=0x7f348c15f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.099 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.095+0000 7f34914fc640 1 --2- 172.21.15.160:0/770050287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f348c0b4600 0x7f348c15f5d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f347c000f70 tx=0x7f347c00c890 comp rx=0 tx=0).stop 2023-12-19T16:42:04.099 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.095+0000 7f34914fc640 1 -- 172.21.15.160:0/770050287 shutdown_connections 2023-12-19T16:42:04.099 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.095+0000 7f34914fc640 1 --2- 172.21.15.160:0/770050287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f348c0d1120 0x7f348c1f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.099 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.095+0000 7f34914fc640 1 --2- 172.21.15.160:0/770050287 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f348c0b8940 0x7f348c0b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.099 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.095+0000 7f34914fc640 1 --2- 172.21.15.160:0/770050287 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f348c0b4600 0x7f348c15f5d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.099 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.095+0000 7f34914fc640 1 -- 172.21.15.160:0/770050287 >> 172.21.15.160:0/770050287 conn(0x7f348c065ee0 msgr2=0x7f348c0cd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:04.100 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.095+0000 7f34914fc640 1 -- 172.21.15.160:0/770050287 shutdown_connections 2023-12-19T16:42:04.100 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.095+0000 7f34914fc640 1 -- 172.21.15.160:0/770050287 wait complete. 2023-12-19T16:42:04.118 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new 5e877a40-25b0-4de4-8f3f-7a8b8c3e4992 4 2023-12-19T16:42:04.168 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:04.163+0000 7f1a90249180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:42:04.216 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:04.209+0000 7fbfae177180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:42:04.231 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:04.227+0000 7f1a90249180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:42:04.255 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.251+0000 7f7da3a2d640 1 Processor -- start 2023-12-19T16:42:04.255 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.251+0000 7f7da3a2d640 1 -- start start 2023-12-19T16:42:04.256 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.251+0000 7f7da3a2d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7d9c0b4600 0x7f7d9c0b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.256 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.251+0000 7f7da3a2d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c0d21d0 con 0x7f7d9c0d2660 2023-12-19T16:42:04.256 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.251+0000 7f7da3a2d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d9c0b4f20 con 0x7f7d9c0b8940 2023-12-19T16:42:04.256 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.251+0000 7f7da3a2d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9c0d12f0 con 0x7f7d9c0b4600 2023-12-19T16:42:04.257 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.251+0000 7f7da17c9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7d9c0b4600 0x7f7d9c0b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.257 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.251+0000 7f7da0fc8640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f7d9c0b8940 0x7f7d9c0d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:58244/0 (socket says 172.21.15.160:58244) 2023-12-19T16:42:04.257 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.251+0000 7f7da0fc8640 1 -- 172.21.15.160:0/2793428699 learned_addr learned my addr 172.21.15.160:0/2793428699 (peer_addr_for_me v1:172.21.15.160:0/0) 2023-12-19T16:42:04.257 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.251+0000 7f7da17c9640 1 -- 172.21.15.160:0/2793428699 >> v1:172.21.15.160:6789/0 conn(0x7f7d9c0b8940 legacy=0x7f7d9c0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.257 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.251+0000 7f7da17c9640 1 -- 172.21.15.160:0/2793428699 >> v1:172.21.15.26:6789/0 conn(0x7f7d9c0d2660 legacy=0x7f7d9c0b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.257 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.251+0000 7f7d93fff640 1 -- 172.21.15.160:0/2793428699 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2973967619 0 0) 0x7f7d9c0d21d0 con 0x7f7d9c0d2660 2023-12-19T16:42:04.257 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.251+0000 7f7da17c9640 1 -- 172.21.15.160:0/2793428699 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c0d15c0 con 0x7f7d9c0b4600 2023-12-19T16:42:04.260 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.255+0000 7f7da17c9640 1 --2- 172.21.15.160:0/2793428699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d9c0b4600 0x7f7d9c0b49e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7d980097c0 tx=0x7f7d9801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f9eec9016fec5c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:04.260 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.255+0000 7f7d93fff640 1 -- 172.21.15.160:0/2793428699 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d980a8970 con 0x7f7d9c0b4600 2023-12-19T16:42:04.260 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.255+0000 7f7d93fff640 1 -- 172.21.15.160:0/2793428699 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d980a8ad0 con 0x7f7d9c0b4600 2023-12-19T16:42:04.260 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.255+0000 7f7d93fff640 1 -- 172.21.15.160:0/2793428699 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d980a8e60 con 0x7f7d9c0b4600 2023-12-19T16:42:04.261 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.255+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2793428699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d9c0b4600 msgr2=0x7f7d9c0b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.261 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.255+0000 7f7da3a2d640 1 --2- 172.21.15.160:0/2793428699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d9c0b4600 0x7f7d9c0b49e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7d980097c0 tx=0x7f7d9801ec70 comp rx=0 tx=0).stop 2023-12-19T16:42:04.261 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.255+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2793428699 shutdown_connections 2023-12-19T16:42:04.261 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.255+0000 7f7da3a2d640 1 --2- 172.21.15.160:0/2793428699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d9c0b4600 0x7f7d9c0b49e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.261 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.255+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2793428699 >> 172.21.15.160:0/2793428699 conn(0x7f7d9c065ee0 msgr2=0x7f7d9c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:04.261 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.255+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2793428699 shutdown_connections 2023-12-19T16:42:04.262 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.255+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2793428699 wait complete. 2023-12-19T16:42:04.263 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da3a2d640 1 Processor -- start 2023-12-19T16:42:04.263 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da3a2d640 1 -- start start 2023-12-19T16:42:04.264 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da3a2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d9c0b4600 0x7f7d9c151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.265 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da3a2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d9c0b8940 0x7f7d9c0caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.265 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da3a2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d9c0d2660 0x7f7d9c1551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.265 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da3a2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0cb990 con 0x7f7d9c0d2660 2023-12-19T16:42:04.265 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da3a2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0cbc70 con 0x7f7d9c0b4600 2023-12-19T16:42:04.265 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da3a2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9c0cbf50 con 0x7f7d9c0b8940 2023-12-19T16:42:04.265 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da17c9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d9c0b4600 0x7f7d9c151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.265 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da0fc8640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d9c0b8940 0x7f7d9c0caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.265 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da17c9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d9c0b4600 0x7f7d9c151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60834/0 (socket says 172.21.15.160:60834) 2023-12-19T16:42:04.265 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da17c9640 1 -- 172.21.15.160:0/2739755946 learned_addr learned my addr 172.21.15.160:0/2739755946 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:04.265 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da1fca640 1 --2- 172.21.15.160:0/2739755946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d9c0d2660 0x7f7d9c1551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.265 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da17c9640 1 -- 172.21.15.160:0/2739755946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d9c0b8940 msgr2=0x7f7d9c0caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.265 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da17c9640 1 --2- 172.21.15.160:0/2739755946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d9c0b8940 0x7f7d9c0caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.265 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da17c9640 1 -- 172.21.15.160:0/2739755946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d9c0d2660 msgr2=0x7f7d9c1551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.266 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da17c9640 1 --2- 172.21.15.160:0/2739755946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d9c0d2660 0x7f7d9c1551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.266 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da17c9640 1 -- 172.21.15.160:0/2739755946 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d98009420 con 0x7f7d9c0b4600 2023-12-19T16:42:04.266 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da17c9640 1 --2- 172.21.15.160:0/2739755946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d9c0b4600 0x7f7d9c151d60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7d9801c8a0 tx=0x7f7d9801d4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:04.267 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7d91ffb640 1 -- 172.21.15.160:0/2739755946 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d980b1e10 con 0x7f7d9c0b4600 2023-12-19T16:42:04.267 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7d91ffb640 1 -- 172.21.15.160:0/2739755946 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d98004030 con 0x7f7d9c0b4600 2023-12-19T16:42:04.267 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7d91ffb640 1 -- 172.21.15.160:0/2739755946 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d980043a0 con 0x7f7d9c0b4600 2023-12-19T16:42:04.267 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2739755946 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d9c0cc230 con 0x7f7d9c0b4600 2023-12-19T16:42:04.267 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2739755946 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d9c15ffa0 con 0x7f7d9c0b4600 2023-12-19T16:42:04.267 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.259+0000 7f7d91ffb640 1 -- 172.21.15.160:0/2739755946 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7d98003920 con 0x7f7d9c0b4600 2023-12-19T16:42:04.267 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.263+0000 7f7d91ffb640 1 -- 172.21.15.160:0/2739755946 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f7d980bccd0 con 0x7f7d9c0b4600 2023-12-19T16:42:04.267 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.263+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2739755946 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9c0cc090 con 0x7f7d9c0b4600 2023-12-19T16:42:04.271 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.267+0000 7f7d91ffb640 1 -- 172.21.15.160:0/2739755946 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f7d9c0cc090 con 0x7f7d9c0b4600 2023-12-19T16:42:04.282 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:04.277+0000 7fbfae177180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:42:04.297 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:04.291+0000 7f1a90249180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:42:04.338 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.335+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2739755946 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5e877a40-25b0-4de4-8f3f-7a8b8c3e4992", "id": 4} v 0) v1 -- 0x7f7d9c15c8a0 con 0x7f7d9c0b4600 2023-12-19T16:42:04.344 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.339+0000 7f7d91ffb640 1 -- 172.21.15.160:0/2739755946 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5e877a40-25b0-4de4-8f3f-7a8b8c3e4992", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7d980c2070 con 0x7f7d9c0b4600 2023-12-19T16:42:04.344 INFO:teuthology.orchestra.run.smithi160.stdout:4 2023-12-19T16:42:04.346 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.339+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2739755946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d9c0b4600 msgr2=0x7f7d9c151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.346 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.339+0000 7f7da3a2d640 1 --2- 172.21.15.160:0/2739755946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d9c0b4600 0x7f7d9c151d60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7d9801c8a0 tx=0x7f7d9801d4b0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.346 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.339+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2739755946 shutdown_connections 2023-12-19T16:42:04.346 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.339+0000 7f7da3a2d640 1 --2- 172.21.15.160:0/2739755946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d9c0d2660 0x7f7d9c1551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.346 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.339+0000 7f7da3a2d640 1 --2- 172.21.15.160:0/2739755946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d9c0b8940 0x7f7d9c0caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.346 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.339+0000 7f7da3a2d640 1 --2- 172.21.15.160:0/2739755946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d9c0b4600 0x7f7d9c151d60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.346 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.339+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2739755946 >> 172.21.15.160:0/2739755946 conn(0x7f7d9c065ee0 msgr2=0x7f7d9c0cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:04.346 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.339+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2739755946 shutdown_connections 2023-12-19T16:42:04.346 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.339+0000 7f7da3a2d640 1 -- 172.21.15.160:0/2739755946 wait complete. 2023-12-19T16:42:04.347 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:04.341+0000 7fbfae177180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:42:04.364 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new 32858744-828a-4628-be89-edb9f3a19ee9 5 2023-12-19T16:42:04.391 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:04.387+0000 7f1a90249180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:42:04.439 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:04.433+0000 7fbfae177180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:42:04.452 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:04.447+0000 7f1a90249180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:42:04.501 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.495+0000 7f655c57b640 1 Processor -- start 2023-12-19T16:42:04.501 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.495+0000 7f655c57b640 1 -- start start 2023-12-19T16:42:04.502 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.499+0000 7f655c57b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f65541599e0 0x7f655406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.502 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.499+0000 7f655c57b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f655415dc40 0x7f65540cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.502 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.499+0000 7f655c57b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655415e240 con 0x7f65540cb150 2023-12-19T16:42:04.503 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.499+0000 7f655c57b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65540cbb20 con 0x7f65541599e0 2023-12-19T16:42:04.503 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.499+0000 7f655c57b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65540c9520 con 0x7f655415dc40 2023-12-19T16:42:04.503 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.499+0000 7f655a317640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f65541599e0 0x7f655406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.503 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.499+0000 7f655ab18640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f655415dc40 0x7f65540cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.503 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.499+0000 7f655a317640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f65541599e0 0x7f655406f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60848/0 (socket says 172.21.15.160:60848) 2023-12-19T16:42:04.503 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.499+0000 7f655a317640 1 -- 172.21.15.160:0/2994826469 learned_addr learned my addr 172.21.15.160:0/2994826469 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:04.503 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:04.497+0000 7fbfae177180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:42:04.504 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.499+0000 7f655ab18640 1 -- 172.21.15.160:0/2994826469 >> v2:172.21.15.160:3300/0 conn(0x7f65541599e0 msgr2=0x7f655406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:04.504 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.499+0000 7f655ab18640 1 --2- 172.21.15.160:0/2994826469 >> v2:172.21.15.160:3300/0 conn(0x7f65541599e0 0x7f655406f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.504 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.499+0000 7f655ab18640 1 -- 172.21.15.160:0/2994826469 >> v1:172.21.15.160:6789/0 conn(0x7f65540cb150 legacy=0x7f655415d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.504 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.499+0000 7f655ab18640 1 -- 172.21.15.160:0/2994826469 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65540c9800 con 0x7f655415dc40 2023-12-19T16:42:04.506 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.503+0000 7f655ab18640 1 --2- 172.21.15.160:0/2994826469 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f655415dc40 0x7f65540cb5e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f654c009d90 tx=0x7f654c01c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc3f6da87f764e34 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:04.508 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.503+0000 7f6559315640 1 -- 172.21.15.160:0/2994826469 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0a5920 con 0x7f655415dc40 2023-12-19T16:42:04.508 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.503+0000 7f6559315640 1 -- 172.21.15.160:0/2994826469 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c0a5a80 con 0x7f655415dc40 2023-12-19T16:42:04.508 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.503+0000 7f6559315640 1 -- 172.21.15.160:0/2994826469 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0a5e10 con 0x7f655415dc40 2023-12-19T16:42:04.509 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.503+0000 7f655c57b640 1 -- 172.21.15.160:0/2994826469 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f655415dc40 msgr2=0x7f65540cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.509 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.503+0000 7f655c57b640 1 --2- 172.21.15.160:0/2994826469 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f655415dc40 0x7f65540cb5e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f654c009d90 tx=0x7f654c01c950 comp rx=0 tx=0).stop 2023-12-19T16:42:04.509 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.503+0000 7f655c57b640 1 -- 172.21.15.160:0/2994826469 shutdown_connections 2023-12-19T16:42:04.509 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.503+0000 7f655c57b640 1 --2- 172.21.15.160:0/2994826469 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f655415dc40 0x7f65540cb5e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.510 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.503+0000 7f655c57b640 1 --2- 172.21.15.160:0/2994826469 >> v2:172.21.15.160:3300/0 conn(0x7f65541599e0 0x7f655406f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.510 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.503+0000 7f655c57b640 1 -- 172.21.15.160:0/2994826469 >> 172.21.15.160:0/2994826469 conn(0x7f6554066160 msgr2=0x7f65540b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:04.510 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.503+0000 7f655c57b640 1 -- 172.21.15.160:0/2994826469 shutdown_connections 2023-12-19T16:42:04.510 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.503+0000 7f655c57b640 1 -- 172.21.15.160:0/2994826469 wait complete. 2023-12-19T16:42:04.510 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655c57b640 1 Processor -- start 2023-12-19T16:42:04.510 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655c57b640 1 -- start start 2023-12-19T16:42:04.512 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655c57b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65540cb150 0x7f6554152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.512 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655c57b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65541599e0 0x7f65540c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.512 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655c57b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f655415dc40 0x7f655415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.512 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655c57b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6554153cd0 con 0x7f655415dc40 2023-12-19T16:42:04.512 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655c57b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655414b470 con 0x7f65540cb150 2023-12-19T16:42:04.512 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655c57b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65540c47e0 con 0x7f65541599e0 2023-12-19T16:42:04.512 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655a317640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65540cb150 0x7f6554152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.513 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655a317640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65540cb150 0x7f6554152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60860/0 (socket says 172.21.15.160:60860) 2023-12-19T16:42:04.513 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f6559b16640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65541599e0 0x7f65540c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.513 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655a317640 1 -- 172.21.15.160:0/3158745440 learned_addr learned my addr 172.21.15.160:0/3158745440 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:04.513 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655ab18640 1 --2- 172.21.15.160:0/3158745440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f655415dc40 0x7f655415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.513 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655a317640 1 -- 172.21.15.160:0/3158745440 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65541599e0 msgr2=0x7f65540c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.513 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655a317640 1 --2- 172.21.15.160:0/3158745440 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65541599e0 0x7f65540c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.513 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655a317640 1 -- 172.21.15.160:0/3158745440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f655415dc40 msgr2=0x7f655415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.513 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655a317640 1 --2- 172.21.15.160:0/3158745440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f655415dc40 0x7f655415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.513 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655a317640 1 -- 172.21.15.160:0/3158745440 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c009a70 con 0x7f65540cb150 2023-12-19T16:42:04.514 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655a317640 1 --2- 172.21.15.160:0/3158745440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65540cb150 0x7f6554152e90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f655000b240 tx=0x7f655000d350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:04.515 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f654affd640 1 -- 172.21.15.160:0/3158745440 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550010440 con 0x7f65540cb150 2023-12-19T16:42:04.515 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f654affd640 1 -- 172.21.15.160:0/3158745440 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65500105a0 con 0x7f65540cb150 2023-12-19T16:42:04.515 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f654affd640 1 -- 172.21.15.160:0/3158745440 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6550010950 con 0x7f65540cb150 2023-12-19T16:42:04.515 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655c57b640 1 -- 172.21.15.160:0/3158745440 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65541539f0 con 0x7f65540cb150 2023-12-19T16:42:04.515 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f655c57b640 1 -- 172.21.15.160:0/3158745440 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6554152690 con 0x7f65540cb150 2023-12-19T16:42:04.515 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f654affd640 1 -- 172.21.15.160:0/3158745440 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f65500059a0 con 0x7f65540cb150 2023-12-19T16:42:04.515 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.507+0000 7f654affd640 1 -- 172.21.15.160:0/3158745440 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f6550010db0 con 0x7f65540cb150 2023-12-19T16:42:04.515 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.511+0000 7f655c57b640 1 -- 172.21.15.160:0/3158745440 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6554152210 con 0x7f65540cb150 2023-12-19T16:42:04.516 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.511+0000 7f654affd640 1 -- 172.21.15.160:0/3158745440 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f6550037b40 con 0x7f65540cb150 2023-12-19T16:42:04.546 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:04.543+0000 7f1a90249180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:42:04.583 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.579+0000 7f655c57b640 1 -- 172.21.15.160:0/3158745440 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd new", "uuid": "32858744-828a-4628-be89-edb9f3a19ee9", "id": 5} v 0) v1 -- 0x7f65540c2050 con 0x7f65540cb150 2023-12-19T16:42:04.587 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.583+0000 7f654affd640 1 -- 172.21.15.160:0/3158745440 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "32858744-828a-4628-be89-edb9f3a19ee9", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6550026030 con 0x7f65540cb150 2023-12-19T16:42:04.588 INFO:teuthology.orchestra.run.smithi160.stdout:5 2023-12-19T16:42:04.589 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.583+0000 7f655c57b640 1 -- 172.21.15.160:0/3158745440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65540cb150 msgr2=0x7f6554152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.589 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.583+0000 7f655c57b640 1 --2- 172.21.15.160:0/3158745440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65540cb150 0x7f6554152e90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f655000b240 tx=0x7f655000d350 comp rx=0 tx=0).stop 2023-12-19T16:42:04.589 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.583+0000 7f655c57b640 1 -- 172.21.15.160:0/3158745440 shutdown_connections 2023-12-19T16:42:04.589 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.583+0000 7f655c57b640 1 --2- 172.21.15.160:0/3158745440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f655415dc40 0x7f655415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.590 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.583+0000 7f655c57b640 1 --2- 172.21.15.160:0/3158745440 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65541599e0 0x7f65540c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.590 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.583+0000 7f655c57b640 1 --2- 172.21.15.160:0/3158745440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65540cb150 0x7f6554152e90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.590 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.583+0000 7f655c57b640 1 -- 172.21.15.160:0/3158745440 >> 172.21.15.160:0/3158745440 conn(0x7f6554066160 msgr2=0x7f655406fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:04.590 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.583+0000 7f655c57b640 1 -- 172.21.15.160:0/3158745440 shutdown_connections 2023-12-19T16:42:04.590 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.583+0000 7f655c57b640 1 -- 172.21.15.160:0/3158745440 wait complete. 2023-12-19T16:42:04.598 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:04.593+0000 7fbfae177180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:42:04.607 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new 896666f0-45ff-4a17-aa09-eb6947ad2a1a 6 2023-12-19T16:42:04.608 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:04.603+0000 7f1a90249180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:42:04.661 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:04.653+0000 7fbfae177180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:42:04.746 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6c21ce640 1 Processor -- start 2023-12-19T16:42:04.746 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6c21ce640 1 -- start start 2023-12-19T16:42:04.746 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6c21ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6bc0b4600 0x7fb6bc0b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.746 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6c21ce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb6bc0b8940 0x7fb6bc0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.746 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6c21ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb6bc0d1120 0x7fb6bc069d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6c21ce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6bc0d21d0 con 0x7fb6bc0d1120 2023-12-19T16:42:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6c21ce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6bc0b4f20 con 0x7fb6bc0b8940 2023-12-19T16:42:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6c21ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc0d15c0 con 0x7fb6bc0b4600 2023-12-19T16:42:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6bb7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6bc0b4600 0x7fb6bc0b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6baffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb6bc0b8940 0x7fb6bc0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6baffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb6bc0b8940 0x7fb6bc0d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:60868/0 (socket says 172.21.15.160:60868) 2023-12-19T16:42:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6baffd640 1 -- 172.21.15.160:0/4273607712 learned_addr learned my addr 172.21.15.160:0/4273607712 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6bbfff640 1 --2- 172.21.15.160:0/4273607712 >> v2:172.21.15.26:3300/0 conn(0x7fb6bc0d1120 0x7fb6bc069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6bb7fe640 1 -- 172.21.15.160:0/4273607712 >> v2:172.21.15.160:3300/0 conn(0x7fb6bc0b8940 msgr2=0x7fb6bc0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6bb7fe640 1 --2- 172.21.15.160:0/4273607712 >> v2:172.21.15.160:3300/0 conn(0x7fb6bc0b8940 0x7fb6bc0d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6bb7fe640 1 -- 172.21.15.160:0/4273607712 >> v2:172.21.15.26:3300/0 conn(0x7fb6bc0d1120 msgr2=0x7fb6bc069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6bb7fe640 1 --2- 172.21.15.160:0/4273607712 >> v2:172.21.15.26:3300/0 conn(0x7fb6bc0d1120 0x7fb6bc069d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.747 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6bb7fe640 1 -- 172.21.15.160:0/4273607712 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc0d18a0 con 0x7fb6bc0b4600 2023-12-19T16:42:04.748 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.739+0000 7fb6baffd640 1 --2- 172.21.15.160:0/4273607712 >> v2:172.21.15.160:3300/0 conn(0x7fb6bc0b8940 0x7fb6bc0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:04.748 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.743+0000 7fb6bb7fe640 1 --2- 172.21.15.160:0/4273607712 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6bc0b4600 0x7fb6bc0b49e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0d0700 tx=0x7fb6ac01ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9425396a5f267bca server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:04.748 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.743+0000 7fb6ba7fc640 1 -- 172.21.15.160:0/4273607712 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0a8910 con 0x7fb6bc0b4600 2023-12-19T16:42:04.748 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.743+0000 7fb6ba7fc640 1 -- 172.21.15.160:0/4273607712 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac0a8a70 con 0x7fb6bc0b4600 2023-12-19T16:42:04.748 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.743+0000 7fb6ba7fc640 1 -- 172.21.15.160:0/4273607712 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0a8e00 con 0x7fb6bc0b4600 2023-12-19T16:42:04.749 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.743+0000 7fb6c21ce640 1 -- 172.21.15.160:0/4273607712 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6bc0b4600 msgr2=0x7fb6bc0b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.749 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.743+0000 7fb6c21ce640 1 --2- 172.21.15.160:0/4273607712 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6bc0b4600 0x7fb6bc0b49e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0d0700 tx=0x7fb6ac01ec50 comp rx=0 tx=0).stop 2023-12-19T16:42:04.749 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.743+0000 7fb6c21ce640 1 -- 172.21.15.160:0/4273607712 shutdown_connections 2023-12-19T16:42:04.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.743+0000 7fb6c21ce640 1 --2- 172.21.15.160:0/4273607712 >> v2:172.21.15.26:3300/0 conn(0x7fb6bc0d1120 0x7fb6bc069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.743+0000 7fb6c21ce640 1 --2- 172.21.15.160:0/4273607712 >> v2:172.21.15.160:3300/0 conn(0x7fb6bc0b8940 0x7fb6bc0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.743+0000 7fb6c21ce640 1 --2- 172.21.15.160:0/4273607712 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6bc0b4600 0x7fb6bc0b49e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.743+0000 7fb6c21ce640 1 -- 172.21.15.160:0/4273607712 >> 172.21.15.160:0/4273607712 conn(0x7fb6bc065ee0 msgr2=0x7fb6bc0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:04.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.743+0000 7fb6c21ce640 1 -- 172.21.15.160:0/4273607712 shutdown_connections 2023-12-19T16:42:04.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.743+0000 7fb6c21ce640 1 -- 172.21.15.160:0/4273607712 wait complete. 2023-12-19T16:42:04.750 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6c21ce640 1 Processor -- start 2023-12-19T16:42:04.751 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6c21ce640 1 -- start start 2023-12-19T16:42:04.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6c21ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6bc0b4600 0x7fb6bc151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6c21ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6bc0b8940 0x7fb6bc1536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6c21ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6bc0d1120 0x7fb6bc1516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6c21ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc155fc0 con 0x7fb6bc0b4600 2023-12-19T16:42:04.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6c21ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc151c10 con 0x7fb6bc0b8940 2023-12-19T16:42:04.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6c21ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc1553b0 con 0x7fb6bc0d1120 2023-12-19T16:42:04.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6baffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6bc0b8940 0x7fb6bc1536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.754 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6bbfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6bc0d1120 0x7fb6bc1516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6bbfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6bc0d1120 0x7fb6bc1516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:37958/0 (socket says 172.21.15.160:37958) 2023-12-19T16:42:04.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6bbfff640 1 -- 172.21.15.160:0/3971299042 learned_addr learned my addr 172.21.15.160:0/3971299042 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:04.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6bb7fe640 1 --2- 172.21.15.160:0/3971299042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6bc0b4600 0x7fb6bc151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6baffd640 1 -- 172.21.15.160:0/3971299042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6bc0d1120 msgr2=0x7fb6bc1516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6baffd640 1 --2- 172.21.15.160:0/3971299042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6bc0d1120 0x7fb6bc1516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6baffd640 1 -- 172.21.15.160:0/3971299042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6bc0b4600 msgr2=0x7fb6bc151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6baffd640 1 --2- 172.21.15.160:0/3971299042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6bc0b4600 0x7fb6bc151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6baffd640 1 -- 172.21.15.160:0/3971299042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b00094d0 con 0x7fb6bc0b8940 2023-12-19T16:42:04.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6bbfff640 1 --2- 172.21.15.160:0/3971299042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6bc0d1120 0x7fb6bc1516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:04.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6baffd640 1 --2- 172.21.15.160:0/3971299042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6bc0b8940 0x7fb6bc1536e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000e8e0 tx=0x7fb6b000eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:04.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6a3fff640 1 -- 172.21.15.160:0/3971299042 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b000ca00 con 0x7fb6bc0b8940 2023-12-19T16:42:04.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6a3fff640 1 -- 172.21.15.160:0/3971299042 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b000cb60 con 0x7fb6bc0b8940 2023-12-19T16:42:04.755 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6a3fff640 1 -- 172.21.15.160:0/3971299042 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0013470 con 0x7fb6bc0b8940 2023-12-19T16:42:04.756 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6c21ce640 1 -- 172.21.15.160:0/3971299042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac009420 con 0x7fb6bc0b8940 2023-12-19T16:42:04.756 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6c21ce640 1 -- 172.21.15.160:0/3971299042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6bc155970 con 0x7fb6bc0b8940 2023-12-19T16:42:04.756 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6a3fff640 1 -- 172.21.15.160:0/3971299042 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb6b0005ca0 con 0x7fb6bc0b8940 2023-12-19T16:42:04.756 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.747+0000 7fb6a3fff640 1 -- 172.21.15.160:0/3971299042 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fb6b00135d0 con 0x7fb6bc0b8940 2023-12-19T16:42:04.756 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.751+0000 7fb6c21ce640 1 -- 172.21.15.160:0/3971299042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6bc074270 con 0x7fb6bc0b8940 2023-12-19T16:42:04.756 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.751+0000 7fb6a3fff640 1 -- 172.21.15.160:0/3971299042 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7fb6b0021060 con 0x7fb6bc0b8940 2023-12-19T16:42:04.771 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:04.767+0000 7f1a90249180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:42:04.822 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.819+0000 7fb6c21ce640 1 -- 172.21.15.160:0/3971299042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd new", "uuid": "896666f0-45ff-4a17-aa09-eb6947ad2a1a", "id": 6} v 0) v1 -- 0x7fb6bc0ccc20 con 0x7fb6bc0b8940 2023-12-19T16:42:04.824 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:04.817+0000 7fbfae177180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:42:04.827 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.823+0000 7fb6a3fff640 1 -- 172.21.15.160:0/3971299042 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "896666f0-45ff-4a17-aa09-eb6947ad2a1a", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb6b0029020 con 0x7fb6bc0b8940 2023-12-19T16:42:04.827 INFO:teuthology.orchestra.run.smithi160.stdout:6 2023-12-19T16:42:04.828 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.823+0000 7fb6c21ce640 1 -- 172.21.15.160:0/3971299042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6bc0b8940 msgr2=0x7fb6bc1536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.828 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.823+0000 7fb6c21ce640 1 --2- 172.21.15.160:0/3971299042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6bc0b8940 0x7fb6bc1536e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000e8e0 tx=0x7fb6b000eda0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.828 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.823+0000 7fb6c21ce640 1 -- 172.21.15.160:0/3971299042 shutdown_connections 2023-12-19T16:42:04.828 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.823+0000 7fb6c21ce640 1 --2- 172.21.15.160:0/3971299042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6bc0d1120 0x7fb6bc1516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.829 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.823+0000 7fb6c21ce640 1 --2- 172.21.15.160:0/3971299042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6bc0b8940 0x7fb6bc1536e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.829 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.823+0000 7fb6c21ce640 1 --2- 172.21.15.160:0/3971299042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6bc0b4600 0x7fb6bc151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.829 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.823+0000 7fb6c21ce640 1 -- 172.21.15.160:0/3971299042 >> 172.21.15.160:0/3971299042 conn(0x7fb6bc065ee0 msgr2=0x7fb6bc0d33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:04.829 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.823+0000 7fb6c21ce640 1 -- 172.21.15.160:0/3971299042 shutdown_connections 2023-12-19T16:42:04.829 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.823+0000 7fb6c21ce640 1 -- 172.21.15.160:0/3971299042 wait complete. 2023-12-19T16:42:04.841 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:04.835+0000 7f1a90249180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:42:04.847 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd new 6de47e3b-3a62-43f4-bce9-8a18c07b306f 7 2023-12-19T16:42:04.897 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:04.889+0000 7fbfae177180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:42:04.983 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48d5bc0640 1 Processor -- start 2023-12-19T16:42:04.983 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48d5bc0640 1 -- start start 2023-12-19T16:42:04.984 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48d5bc0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f48d00b4600 0x7f48d00b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.984 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48d5bc0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48d00b8940 0x7f48d00d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48d5bc0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f48d00d1120 0x7f48d0069d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48d5bc0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d00d21d0 con 0x7f48d00b4600 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48d5bc0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d00b4f20 con 0x7f48d00d1120 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48d5bc0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00d15c0 con 0x7f48d00b8940 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48ceffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48d00b8940 0x7f48d00d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48ceffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48d00b8940 0x7f48d00d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:37962/0 (socket says 172.21.15.160:37962) 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48cffff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f48d00d1120 0x7f48d0069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48ceffd640 1 -- 172.21.15.160:0/3742065216 learned_addr learned my addr 172.21.15.160:0/3742065216 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48cf7fe640 1 --2- 172.21.15.160:0/3742065216 >> v2:172.21.15.26:3300/0 conn(0x7f48d00b4600 0x7f48d00b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48ceffd640 1 -- 172.21.15.160:0/3742065216 >> v2:172.21.15.160:3300/0 conn(0x7f48d00d1120 msgr2=0x7f48d0069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48ceffd640 1 --2- 172.21.15.160:0/3742065216 >> v2:172.21.15.160:3300/0 conn(0x7f48d00d1120 0x7f48d0069d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48ceffd640 1 -- 172.21.15.160:0/3742065216 >> v2:172.21.15.26:3300/0 conn(0x7f48d00b4600 msgr2=0x7f48d00b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48ceffd640 1 --2- 172.21.15.160:0/3742065216 >> v2:172.21.15.26:3300/0 conn(0x7f48d00b4600 0x7f48d00b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.985 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.979+0000 7f48ceffd640 1 -- 172.21.15.160:0/3742065216 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d00d18a0 con 0x7f48d00b8940 2023-12-19T16:42:04.987 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.983+0000 7f48ceffd640 1 --2- 172.21.15.160:0/3742065216 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48d00b8940 0x7f48d00d0be0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f48d008bee0 tx=0x7f48c401ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72848d1cee885f18 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:04.988 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.983+0000 7f48ce7fc640 1 -- 172.21.15.160:0/3742065216 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c40a89b0 con 0x7f48d00b8940 2023-12-19T16:42:04.988 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.983+0000 7f48ce7fc640 1 -- 172.21.15.160:0/3742065216 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c40a8b10 con 0x7f48d00b8940 2023-12-19T16:42:04.988 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.983+0000 7f48ce7fc640 1 -- 172.21.15.160:0/3742065216 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c40a8ea0 con 0x7f48d00b8940 2023-12-19T16:42:04.989 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.983+0000 7f48d5bc0640 1 -- 172.21.15.160:0/3742065216 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48d00b8940 msgr2=0x7f48d00d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.989 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.983+0000 7f48d5bc0640 1 --2- 172.21.15.160:0/3742065216 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48d00b8940 0x7f48d00d0be0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f48d008bee0 tx=0x7f48c401ecf0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.989 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.983+0000 7f48d5bc0640 1 -- 172.21.15.160:0/3742065216 shutdown_connections 2023-12-19T16:42:04.989 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.983+0000 7f48d5bc0640 1 --2- 172.21.15.160:0/3742065216 >> v2:172.21.15.160:3300/0 conn(0x7f48d00d1120 0x7f48d0069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.989 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.983+0000 7f48d5bc0640 1 --2- 172.21.15.160:0/3742065216 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48d00b8940 0x7f48d00d0be0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.989 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.983+0000 7f48d5bc0640 1 --2- 172.21.15.160:0/3742065216 >> v2:172.21.15.26:3300/0 conn(0x7f48d00b4600 0x7f48d00b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.989 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.983+0000 7f48d5bc0640 1 -- 172.21.15.160:0/3742065216 >> 172.21.15.160:0/3742065216 conn(0x7f48d0065ee0 msgr2=0x7f48d00cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:04.989 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.983+0000 7f48d5bc0640 1 -- 172.21.15.160:0/3742065216 shutdown_connections 2023-12-19T16:42:04.989 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.983+0000 7f48d5bc0640 1 -- 172.21.15.160:0/3742065216 wait complete. 2023-12-19T16:42:04.990 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48d5bc0640 1 Processor -- start 2023-12-19T16:42:04.990 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48d5bc0640 1 -- start start 2023-12-19T16:42:04.992 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48d5bc0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48d00b4600 0x7f48d00a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.992 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48d5bc0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48d00b8940 0x7f48d015f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.992 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48d5bc0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48d00d1120 0x7f48d015fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:04.992 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48d5bc0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00aaa40 con 0x7f48d00b4600 2023-12-19T16:42:04.992 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48d5bc0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00aad20 con 0x7f48d00b8940 2023-12-19T16:42:04.992 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48d5bc0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00ab000 con 0x7f48d00d1120 2023-12-19T16:42:04.993 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48cffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48d00d1120 0x7f48d015fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.993 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48ceffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48d00b8940 0x7f48d015f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.993 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48cffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48d00d1120 0x7f48d015fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.160:37966/0 (socket says 172.21.15.160:37966) 2023-12-19T16:42:04.993 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48cffff640 1 -- 172.21.15.160:0/4190353599 learned_addr learned my addr 172.21.15.160:0/4190353599 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-19T16:42:04.993 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48cf7fe640 1 --2- 172.21.15.160:0/4190353599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48d00b4600 0x7f48d00a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:04.993 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48ceffd640 1 -- 172.21.15.160:0/4190353599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48d00d1120 msgr2=0x7f48d015fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.993 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48ceffd640 1 --2- 172.21.15.160:0/4190353599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48d00d1120 0x7f48d015fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.993 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48ceffd640 1 -- 172.21.15.160:0/4190353599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48d00b4600 msgr2=0x7f48d00a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:04.993 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48ceffd640 1 --2- 172.21.15.160:0/4190353599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48d00b4600 0x7f48d00a9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:04.993 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48ceffd640 1 -- 172.21.15.160:0/4190353599 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c40094d0 con 0x7f48d00b8940 2023-12-19T16:42:04.993 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48ceffd640 1 --2- 172.21.15.160:0/4190353599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48d00b8940 0x7f48d015f5c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f48c40af000 tx=0x7f48c401ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:04.994 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48b7fff640 1 -- 172.21.15.160:0/4190353599 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c40b0da0 con 0x7f48d00b8940 2023-12-19T16:42:04.994 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48b7fff640 1 -- 172.21.15.160:0/4190353599 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c40bb460 con 0x7f48d00b8940 2023-12-19T16:42:04.994 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48d5bc0640 1 -- 172.21.15.160:0/4190353599 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d00ab2e0 con 0x7f48d00b8940 2023-12-19T16:42:04.994 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48b7fff640 1 -- 172.21.15.160:0/4190353599 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c40bb810 con 0x7f48d00b8940 2023-12-19T16:42:04.995 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48d5bc0640 1 -- 172.21.15.160:0/4190353599 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d00cc700 con 0x7f48d00b8940 2023-12-19T16:42:04.995 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48b7fff640 1 -- 172.21.15.160:0/4190353599 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f48c40bbc70 con 0x7f48d00b8940 2023-12-19T16:42:04.995 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48b7fff640 1 -- 172.21.15.160:0/4190353599 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f48c40c5440 con 0x7f48d00b8940 2023-12-19T16:42:04.995 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.987+0000 7f48d5bc0640 1 -- 172.21.15.160:0/4190353599 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d0074270 con 0x7f48d00b8940 2023-12-19T16:42:04.996 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:04.991+0000 7f48b7fff640 1 -- 172.21.15.160:0/4190353599 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f48c40c0070 con 0x7f48d00b8940 2023-12-19T16:42:05.061 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:05.055+0000 7f48d5bc0640 1 -- 172.21.15.160:0/4190353599 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6de47e3b-3a62-43f4-bce9-8a18c07b306f", "id": 7} v 0) v1 -- 0x7f48d00aae60 con 0x7f48d00b8940 2023-12-19T16:42:05.066 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:05.063+0000 7f48b7fff640 1 -- 172.21.15.160:0/4190353599 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6de47e3b-3a62-43f4-bce9-8a18c07b306f", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f48d00aae60 con 0x7f48d00b8940 2023-12-19T16:42:05.066 INFO:teuthology.orchestra.run.smithi160.stdout:7 2023-12-19T16:42:05.067 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:05.063+0000 7f48d5bc0640 1 -- 172.21.15.160:0/4190353599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48d00b8940 msgr2=0x7f48d015f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:05.067 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:05.063+0000 7f48d5bc0640 1 --2- 172.21.15.160:0/4190353599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48d00b8940 0x7f48d015f5c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f48c40af000 tx=0x7f48c401ba90 comp rx=0 tx=0).stop 2023-12-19T16:42:05.067 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:05.063+0000 7f48d5bc0640 1 -- 172.21.15.160:0/4190353599 shutdown_connections 2023-12-19T16:42:05.067 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:05.063+0000 7f48d5bc0640 1 --2- 172.21.15.160:0/4190353599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48d00d1120 0x7f48d015fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:05.067 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:05.063+0000 7f48d5bc0640 1 --2- 172.21.15.160:0/4190353599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48d00b8940 0x7f48d015f5c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:05.067 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:05.063+0000 7f48d5bc0640 1 --2- 172.21.15.160:0/4190353599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48d00b4600 0x7f48d00a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:05.068 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:05.063+0000 7f48d5bc0640 1 -- 172.21.15.160:0/4190353599 >> 172.21.15.160:0/4190353599 conn(0x7f48d0065ee0 msgr2=0x7f48d00b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:05.068 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:05.063+0000 7f48d5bc0640 1 -- 172.21.15.160:0/4190353599 shutdown_connections 2023-12-19T16:42:05.068 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-19T16:42:05.063+0000 7f48d5bc0640 1 -- 172.21.15.160:0/4190353599 wait complete. 2023-12-19T16:42:05.086 INFO:tasks.ceph.osd.0:Restarting daemon 2023-12-19T16:42:05.086 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-12-19T16:42:05.090 INFO:tasks.ceph.osd.0:Started 2023-12-19T16:42:05.090 INFO:tasks.ceph.osd.1:Restarting daemon 2023-12-19T16:42:05.090 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-12-19T16:42:05.093 INFO:tasks.ceph.osd.1:Started 2023-12-19T16:42:05.093 INFO:tasks.ceph.osd.2:Restarting daemon 2023-12-19T16:42:05.093 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-12-19T16:42:05.097 INFO:tasks.ceph.osd.2:Started 2023-12-19T16:42:05.097 INFO:tasks.ceph.osd.3:Restarting daemon 2023-12-19T16:42:05.097 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-12-19T16:42:05.100 INFO:tasks.ceph.osd.3:Started 2023-12-19T16:42:05.100 INFO:tasks.ceph.osd.4:Restarting daemon 2023-12-19T16:42:05.100 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-12-19T16:42:05.104 INFO:tasks.ceph.osd.4:Started 2023-12-19T16:42:05.104 INFO:tasks.ceph.osd.5:Restarting daemon 2023-12-19T16:42:05.104 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-12-19T16:42:05.107 INFO:tasks.ceph.osd.5:Started 2023-12-19T16:42:05.107 INFO:tasks.ceph.osd.6:Restarting daemon 2023-12-19T16:42:05.108 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-12-19T16:42:05.111 INFO:tasks.ceph.osd.6:Started 2023-12-19T16:42:05.111 INFO:tasks.ceph.osd.7:Restarting daemon 2023-12-19T16:42:05.111 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-12-19T16:42:05.114 INFO:tasks.ceph.osd.7:Started 2023-12-19T16:42:05.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:05.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.281+0000 7f3333d2c640 1 Processor -- start 2023-12-19T16:42:05.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.281+0000 7f3333d2c640 1 -- start start 2023-12-19T16:42:05.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.281+0000 7f3333d2c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f332c0ce040 0x7f332c0d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:05.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.281+0000 7f3333d2c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f332c0d3130 0x7f332c0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:05.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.281+0000 7f3333d2c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f332c0d29d0 con 0x7f332c15cd40 2023-12-19T16:42:05.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.281+0000 7f3333d2c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f332c0d3570 con 0x7f332c0ce040 2023-12-19T16:42:05.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.281+0000 7f3333d2c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c15d680 con 0x7f332c0d3130 2023-12-19T16:42:05.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.285+0000 7f33312c7640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f332c0ce040 0x7f332c0d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:05.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.285+0000 7f3331ac8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f332c15cd40 0x7f332c160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45942/0 (socket says 172.21.15.26:45942) 2023-12-19T16:42:05.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.285+0000 7f3331ac8640 1 -- 172.21.15.26:0/2365705090 learned_addr learned my addr 172.21.15.26:0/2365705090 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:05.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.285+0000 7f33322c9640 1 --2- 172.21.15.26:0/2365705090 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f332c0d3130 0x7f332c0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.285+0000 7f33322c9640 1 -- 172.21.15.26:0/2365705090 >> v2:172.21.15.160:3300/0 conn(0x7f332c0ce040 msgr2=0x7f332c0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.285+0000 7f33322c9640 1 --2- 172.21.15.26:0/2365705090 >> v2:172.21.15.160:3300/0 conn(0x7f332c0ce040 0x7f332c0d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.285+0000 7f33322c9640 1 -- 172.21.15.26:0/2365705090 >> v1:172.21.15.26:6789/0 conn(0x7f332c15cd40 legacy=0x7f332c160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.285+0000 7f3330ac6640 1 -- 172.21.15.26:0/2365705090 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010792489 0 0) 0x7f3328004b40 con 0x7f332c15cd40 2023-12-19T16:42:05.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.285+0000 7f33322c9640 1 -- 172.21.15.26:0/2365705090 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c0ce4b0 con 0x7f332c0d3130 2023-12-19T16:42:05.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.289+0000 7f33322c9640 1 --2- 172.21.15.26:0/2365705090 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f332c0d3130 0x7f332c0d0e30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3324009d90 tx=0x7f332401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df4ec314da9b989e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.289+0000 7f3330ac6640 1 -- 172.21.15.26:0/2365705090 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33240a5a10 con 0x7f332c0d3130 2023-12-19T16:42:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.289+0000 7f3330ac6640 1 -- 172.21.15.26:0/2365705090 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33240a5b70 con 0x7f332c0d3130 2023-12-19T16:42:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.289+0000 7f3330ac6640 1 -- 172.21.15.26:0/2365705090 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33240ad440 con 0x7f332c0d3130 2023-12-19T16:42:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.289+0000 7f3333d2c640 1 -- 172.21.15.26:0/2365705090 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f332c0d3130 msgr2=0x7f332c0d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.289+0000 7f3333d2c640 1 --2- 172.21.15.26:0/2365705090 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f332c0d3130 0x7f332c0d0e30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3324009d90 tx=0x7f332401c950 comp rx=0 tx=0).stop 2023-12-19T16:42:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.293+0000 7f3333d2c640 1 -- 172.21.15.26:0/2365705090 shutdown_connections 2023-12-19T16:42:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.293+0000 7f3333d2c640 1 --2- 172.21.15.26:0/2365705090 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f332c0d3130 0x7f332c0d0e30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.293+0000 7f3333d2c640 1 --2- 172.21.15.26:0/2365705090 >> v2:172.21.15.160:3300/0 conn(0x7f332c0ce040 0x7f332c0d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.293+0000 7f3333d2c640 1 -- 172.21.15.26:0/2365705090 >> 172.21.15.26:0/2365705090 conn(0x7f332c06a120 msgr2=0x7f332c06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:05.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.293+0000 7f3333d2c640 1 -- 172.21.15.26:0/2365705090 shutdown_connections 2023-12-19T16:42:05.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.293+0000 7f3333d2c640 1 -- 172.21.15.26:0/2365705090 wait complete. 2023-12-19T16:42:05.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.293+0000 7f3333d2c640 1 Processor -- start 2023-12-19T16:42:05.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.297+0000 7f3333d2c640 1 -- start start 2023-12-19T16:42:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.297+0000 7f3333d2c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f332c0ce040 0x7f332c153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.297+0000 7f3333d2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f332c0d3130 0x7f332c0b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:05.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.297+0000 7f3331ac8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f332c0ce040 0x7f332c153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.297+0000 7f3331ac8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f332c0ce040 0x7f332c153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:56558/0 (socket says 172.21.15.26:56558) 2023-12-19T16:42:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.297+0000 7f3333d2c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f332c15cd40 0x7f332c1520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.297+0000 7f3333d2c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c15fbe0 con 0x7f332c0d3130 2023-12-19T16:42:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.297+0000 7f3333d2c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332c1586a0 con 0x7f332c0ce040 2023-12-19T16:42:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.297+0000 7f3333d2c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332c159e20 con 0x7f332c15cd40 2023-12-19T16:42:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.297+0000 7f33322c9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f332c15cd40 0x7f332c1520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.297+0000 7f33312c7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f332c0d3130 0x7f332c0b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.297+0000 7f33322c9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f332c15cd40 0x7f332c1520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:35484/0 (socket says 172.21.15.26:35484) 2023-12-19T16:42:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.297+0000 7f33322c9640 1 -- 172.21.15.26:0/2655727400 learned_addr learned my addr 172.21.15.26:0/2655727400 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f33322c9640 1 -- 172.21.15.26:0/2655727400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f332c0ce040 msgr2=0x7f332c153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f33322c9640 1 --2- 172.21.15.26:0/2655727400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f332c0ce040 0x7f332c153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f33322c9640 1 -- 172.21.15.26:0/2655727400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f332c0d3130 msgr2=0x7f332c0b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f33322c9640 1 --2- 172.21.15.26:0/2655727400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f332c0d3130 0x7f332c0b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f33322c9640 1 -- 172.21.15.26:0/2655727400 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3324009a70 con 0x7f332c15cd40 2023-12-19T16:42:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f33312c7640 1 --2- 172.21.15.26:0/2655727400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f332c0d3130 0x7f332c0b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f3331ac8640 1 --2- 172.21.15.26:0/2655727400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f332c0ce040 0x7f332c153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2023-12-19T16:42:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f33322c9640 1 --2- 172.21.15.26:0/2655727400 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f332c15cd40 0x7f332c1520e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f332401ce30 tx=0x7f33240196f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f33227fc640 1 -- 172.21.15.26:0/2655727400 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33240ba380 con 0x7f332c15cd40 2023-12-19T16:42:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f33227fc640 1 -- 172.21.15.26:0/2655727400 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33240ba4e0 con 0x7f332c15cd40 2023-12-19T16:42:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f33227fc640 1 -- 172.21.15.26:0/2655727400 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33240ba870 con 0x7f332c15cd40 2023-12-19T16:42:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f3333d2c640 1 -- 172.21.15.26:0/2655727400 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332c15a0c0 con 0x7f332c15cd40 2023-12-19T16:42:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f3333d2c640 1 -- 172.21.15.26:0/2655727400 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332c1582c0 con 0x7f332c15cd40 2023-12-19T16:42:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f33227fc640 1 -- 172.21.15.26:0/2655727400 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f33240b4070 con 0x7f332c15cd40 2023-12-19T16:42:05.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f33227fc640 1 -- 172.21.15.26:0/2655727400 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f33240ad950 con 0x7f332c15cd40 2023-12-19T16:42:05.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.301+0000 7f3333d2c640 1 -- 172.21.15.26:0/2655727400 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332c1587e0 con 0x7f332c15cd40 2023-12-19T16:42:05.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.305+0000 7f33227fc640 1 -- 172.21.15.26:0/2655727400 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f332c1587e0 con 0x7f332c15cd40 2023-12-19T16:42:05.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.389+0000 7f3333d2c640 1 -- 172.21.15.26:0/2655727400 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f332c074270 con 0x7f332c15cd40 2023-12-19T16:42:05.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.393+0000 7f33227fc640 1 -- 172.21.15.26:0/2655727400 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f33240da060 con 0x7f332c15cd40 2023-12-19T16:42:05.398 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:05.398 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":10,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:05.061321+0000","last_up_change":"0.000000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.393+0000 7f3333d2c640 1 -- 172.21.15.26:0/2655727400 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f332c15cd40 msgr2=0x7f332c1520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.393+0000 7f3333d2c640 1 --2- 172.21.15.26:0/2655727400 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f332c15cd40 0x7f332c1520e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f332401ce30 tx=0x7f33240196f0 comp rx=0 tx=0).stop 2023-12-19T16:42:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.393+0000 7f3333d2c640 1 -- 172.21.15.26:0/2655727400 shutdown_connections 2023-12-19T16:42:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.393+0000 7f3333d2c640 1 --2- 172.21.15.26:0/2655727400 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f332c15cd40 0x7f332c1520e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.393+0000 7f3333d2c640 1 --2- 172.21.15.26:0/2655727400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f332c0d3130 0x7f332c0b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.393+0000 7f3333d2c640 1 --2- 172.21.15.26:0/2655727400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f332c0ce040 0x7f332c153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.393+0000 7f3333d2c640 1 -- 172.21.15.26:0/2655727400 >> 172.21.15.26:0/2655727400 conn(0x7f332c06a120 msgr2=0x7f332c06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.393+0000 7f3333d2c640 1 -- 172.21.15.26:0/2655727400 shutdown_connections 2023-12-19T16:42:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:05.393+0000 7f3333d2c640 1 -- 172.21.15.26:0/2655727400 wait complete. 2023-12-19T16:42:05.418 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-12-19T16:42:05.418 INFO:tasks.ceph:Waiting for OSDs to come up 2023-12-19T16:42:05.521 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:42:05.522 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:42:05.522 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:42:05.587 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:05.583+0000 7f1a90249180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:42:05.634 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:42:05.634 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:42:05.634 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:42:05.687 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:05.681+0000 7fbfae177180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:42:05.721 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:05.715+0000 7f1a90249180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:42:05.755 INFO:tasks.ceph.osd.0.smithi026.stderr:2023-12-19T16:42:05.749+0000 7f7b17680640 -1 Falling back to public interface 2023-12-19T16:42:05.764 INFO:tasks.ceph.osd.1.smithi026.stderr:2023-12-19T16:42:05.757+0000 7feff4e69640 -1 Falling back to public interface 2023-12-19T16:42:05.764 INFO:tasks.ceph.osd.2.smithi026.stderr:2023-12-19T16:42:05.757+0000 7fc720f36640 -1 Falling back to public interface 2023-12-19T16:42:05.766 INFO:tasks.ceph.osd.2.smithi026.stderr:2023-12-19T16:42:05.761+0000 7fc71fef7640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6814/0: (98) Address already in use 2023-12-19T16:42:05.775 INFO:tasks.ceph.osd.3.smithi026.stderr:2023-12-19T16:42:05.769+0000 7f2e1b4e4640 -1 Falling back to public interface 2023-12-19T16:42:05.803 INFO:tasks.ceph.osd.4.smithi160.stderr:2023-12-19T16:42:05.799+0000 7f7fb8d05640 -1 Falling back to public interface 2023-12-19T16:42:05.804 INFO:tasks.ceph.osd.5.smithi160.stderr:2023-12-19T16:42:05.799+0000 7f42e6c93640 -1 Falling back to public interface 2023-12-19T16:42:05.804 INFO:tasks.ceph.osd.7.smithi160.stderr:2023-12-19T16:42:05.799+0000 7f760b895640 -1 Falling back to public interface 2023-12-19T16:42:05.835 INFO:tasks.ceph.osd.6.smithi160.stderr:2023-12-19T16:42:05.831+0000 7f1d204a9640 -1 Falling back to public interface 2023-12-19T16:42:05.838 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:05.833+0000 7fbfae177180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:42:05.984 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:05.979+0000 7f1a90249180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:42:06.025 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:06.017+0000 7fbfae177180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:42:06.056 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:06.051+0000 7f1a90249180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:42:06.102 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:06.093+0000 7fbfae177180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:42:06.195 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:06.191+0000 7f1a90249180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:42:06.222 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:06.246 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:06.241+0000 7fbfae177180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:42:06.355 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f788a641640 1 Processor -- start 2023-12-19T16:42:06.356 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f788a641640 1 -- start start 2023-12-19T16:42:06.357 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f788a641640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f78840ce040 0x7f78840d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:06.357 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f788a641640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78840d29d0 con 0x7f788415cd40 2023-12-19T16:42:06.357 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f788a641640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78840d3570 con 0x7f78840d3130 2023-12-19T16:42:06.357 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f788a641640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788415d680 con 0x7f78840ce040 2023-12-19T16:42:06.357 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f7883fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f788415cd40 0x7f7884160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60350/0 (socket says 172.21.15.26:60350) 2023-12-19T16:42:06.357 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f7883fff640 1 -- 172.21.15.26:0/783965754 learned_addr learned my addr 172.21.15.26:0/783965754 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:06.357 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f78837fe640 1 --2- 172.21.15.26:0/783965754 >> v2:172.21.15.160:3300/0 conn(0x7f78840ce040 0x7f78840d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:06.358 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f7882ffd640 1 -- 172.21.15.26:0/783965754 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198411325 0 0) 0x7f78740012a0 con 0x7f788415cd40 2023-12-19T16:42:06.358 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f7882ffd640 1 -- 172.21.15.26:0/783965754 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f786c003530 con 0x7f788415cd40 2023-12-19T16:42:06.358 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f78837fe640 1 -- 172.21.15.26:0/783965754 >> v1:172.21.15.160:6789/0 conn(0x7f78840d3130 legacy=0x7f78840d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:06.358 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f78837fe640 1 -- 172.21.15.26:0/783965754 >> v1:172.21.15.26:6789/0 conn(0x7f788415cd40 legacy=0x7f7884160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:06.358 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.349+0000 7f78837fe640 1 -- 172.21.15.26:0/783965754 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78840ce4b0 con 0x7f78840ce040 2023-12-19T16:42:06.361 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.353+0000 7f78837fe640 1 --2- 172.21.15.26:0/783965754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78840ce040 0x7f78840d2bf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f78780097c0 tx=0x7f787801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c595f1d5d81b8360 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:06.361 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.353+0000 7f7882ffd640 1 -- 172.21.15.26:0/783965754 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78780ab680 con 0x7f78840ce040 2023-12-19T16:42:06.361 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.353+0000 7f7882ffd640 1 -- 172.21.15.26:0/783965754 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78780ab7e0 con 0x7f78840ce040 2023-12-19T16:42:06.362 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.353+0000 7f7882ffd640 1 -- 172.21.15.26:0/783965754 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78780abb70 con 0x7f78840ce040 2023-12-19T16:42:06.362 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 -- 172.21.15.26:0/783965754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78840ce040 msgr2=0x7f78840d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:06.362 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 --2- 172.21.15.26:0/783965754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78840ce040 0x7f78840d2bf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f78780097c0 tx=0x7f787801c480 comp rx=0 tx=0).stop 2023-12-19T16:42:06.362 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 -- 172.21.15.26:0/783965754 shutdown_connections 2023-12-19T16:42:06.363 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 --2- 172.21.15.26:0/783965754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78840ce040 0x7f78840d2bf0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:06.363 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 -- 172.21.15.26:0/783965754 >> 172.21.15.26:0/783965754 conn(0x7f788406a120 msgr2=0x7f788406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:06.363 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 -- 172.21.15.26:0/783965754 shutdown_connections 2023-12-19T16:42:06.363 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 -- 172.21.15.26:0/783965754 wait complete. 2023-12-19T16:42:06.364 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 Processor -- start 2023-12-19T16:42:06.364 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 -- start start 2023-12-19T16:42:06.366 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78840ce040 0x7f7884167ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:06.366 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78840d3130 0x7f78840c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:06.366 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f788415cd40 0x7f788414bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:06.366 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78840c71e0 con 0x7f78840ce040 2023-12-19T16:42:06.366 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78840c74c0 con 0x7f78840d3130 2023-12-19T16:42:06.366 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f788a641640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78840c77a0 con 0x7f788415cd40 2023-12-19T16:42:06.367 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f7883fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78840ce040 0x7f7884167ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:06.367 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f7883fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78840ce040 0x7f7884167ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53790/0 (socket says 172.21.15.26:53790) 2023-12-19T16:42:06.367 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f7883fff640 1 -- 172.21.15.26:0/3355588475 learned_addr learned my addr 172.21.15.26:0/3355588475 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:06.367 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f78837fe640 1 --2- 172.21.15.26:0/3355588475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78840d3130 0x7f78840c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:06.367 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.357+0000 7f7888bde640 1 --2- 172.21.15.26:0/3355588475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f788415cd40 0x7f788414bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:06.367 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f7883fff640 1 -- 172.21.15.26:0/3355588475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f788415cd40 msgr2=0x7f788414bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:06.367 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f7883fff640 1 --2- 172.21.15.26:0/3355588475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f788415cd40 0x7f788414bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:06.367 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f7883fff640 1 -- 172.21.15.26:0/3355588475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78840d3130 msgr2=0x7f78840c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:06.367 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f7883fff640 1 --2- 172.21.15.26:0/3355588475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78840d3130 0x7f78840c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:06.367 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f7883fff640 1 -- 172.21.15.26:0/3355588475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7878009420 con 0x7f78840ce040 2023-12-19T16:42:06.367 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f7883fff640 1 --2- 172.21.15.26:0/3355588475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78840ce040 0x7f7884167ef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f787400c4d0 tx=0x7f787400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:06.368 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f7880ff9640 1 -- 172.21.15.26:0/3355588475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874007480 con 0x7f78840ce040 2023-12-19T16:42:06.368 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f7880ff9640 1 -- 172.21.15.26:0/3355588475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78740075e0 con 0x7f78840ce040 2023-12-19T16:42:06.368 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f7880ff9640 1 -- 172.21.15.26:0/3355588475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78740059d0 con 0x7f78840ce040 2023-12-19T16:42:06.368 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f788a641640 1 -- 172.21.15.26:0/3355588475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78840c7a80 con 0x7f78840ce040 2023-12-19T16:42:06.368 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f7880ff9640 1 -- 172.21.15.26:0/3355588475 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7874012400 con 0x7f78840ce040 2023-12-19T16:42:06.369 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f788a641640 1 -- 172.21.15.26:0/3355588475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78840c8280 con 0x7f78840ce040 2023-12-19T16:42:06.369 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f7880ff9640 1 -- 172.21.15.26:0/3355588475 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f787401b400 con 0x7f78840ce040 2023-12-19T16:42:06.369 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.361+0000 7f788a641640 1 -- 172.21.15.26:0/3355588475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78840cdad0 con 0x7f78840ce040 2023-12-19T16:42:06.370 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.365+0000 7f7880ff9640 1 -- 172.21.15.26:0/3355588475 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f7874017070 con 0x7f78840ce040 2023-12-19T16:42:06.429 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.421+0000 7f788a641640 1 -- 172.21.15.26:0/3355588475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78840cd810 con 0x7f78840ce040 2023-12-19T16:42:06.429 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.421+0000 7f7880ff9640 1 -- 172.21.15.26:0/3355588475 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f78840d29d0 con 0x7f78840ce040 2023-12-19T16:42:06.429 INFO:teuthology.misc.health.smithi026.stdout: 2023-12-19T16:42:06.430 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":10,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:05.061321+0000","last_up_change":"0.000000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:06.430 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.425+0000 7f788a641640 1 -- 172.21.15.26:0/3355588475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78840ce040 msgr2=0x7f7884167ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:06.430 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.425+0000 7f788a641640 1 --2- 172.21.15.26:0/3355588475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78840ce040 0x7f7884167ef0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f787400c4d0 tx=0x7f787400c990 comp rx=0 tx=0).stop 2023-12-19T16:42:06.431 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.425+0000 7f788a641640 1 -- 172.21.15.26:0/3355588475 shutdown_connections 2023-12-19T16:42:06.431 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.425+0000 7f788a641640 1 --2- 172.21.15.26:0/3355588475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f788415cd40 0x7f788414bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:06.431 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.425+0000 7f788a641640 1 --2- 172.21.15.26:0/3355588475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78840d3130 0x7f78840c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:06.431 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.425+0000 7f788a641640 1 --2- 172.21.15.26:0/3355588475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78840ce040 0x7f7884167ef0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:06.431 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.425+0000 7f788a641640 1 -- 172.21.15.26:0/3355588475 >> 172.21.15.26:0/3355588475 conn(0x7f788406a120 msgr2=0x7f7884160890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:06.431 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.425+0000 7f788a641640 1 -- 172.21.15.26:0/3355588475 shutdown_connections 2023-12-19T16:42:06.431 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:06.425+0000 7f788a641640 1 -- 172.21.15.26:0/3355588475 wait complete. 2023-12-19T16:42:06.448 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-19T16:42:06.862 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:06.855+0000 7f1a90249180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:42:06.948 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:06.943+0000 7f1a90249180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:42:06.990 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:06.985+0000 7fbfae177180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:42:07.081 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:07.073+0000 7fbfae177180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:42:07.139 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:07.135+0000 7f1a90249180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:42:07.222 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:07.219+0000 7f1a90249180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:42:07.280 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:07.273+0000 7fbfae177180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:42:07.369 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:07.361+0000 7fbfae177180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:42:07.551 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:07.547+0000 7f1a90249180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:42:07.621 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:07.615+0000 7f1a90249180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:42:07.717 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:07.709+0000 7fbfae177180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:42:07.788 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:42:07.781+0000 7fbfae177180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:42:08.644 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:08.639+0000 7f1a36f5f640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-12-19T16:42:08.644 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:42:08.639+0000 7f1a36f5f640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-12-19T16:42:09.576 INFO:tasks.ceph.osd.0.smithi026.stderr:2023-12-19T16:42:09.569+0000 7f7b17680640 -1 osd.0 0 log_to_monitors true 2023-12-19T16:42:09.649 INFO:tasks.ceph.osd.1.smithi026.stderr:2023-12-19T16:42:09.641+0000 7feff4e69640 -1 osd.1 0 log_to_monitors true 2023-12-19T16:42:09.717 INFO:tasks.ceph.osd.2.smithi026.stderr:2023-12-19T16:42:09.709+0000 7fc720f36640 -1 osd.2 0 log_to_monitors true 2023-12-19T16:42:09.733 INFO:tasks.ceph.osd.3.smithi026.stderr:2023-12-19T16:42:09.725+0000 7f2e1b4e4640 -1 osd.3 0 log_to_monitors true 2023-12-19T16:42:09.738 INFO:tasks.ceph.osd.4.smithi160.stderr:2023-12-19T16:42:09.735+0000 7f7fb8d05640 -1 osd.4 0 log_to_monitors true 2023-12-19T16:42:09.767 INFO:tasks.ceph.osd.6.smithi160.stderr:2023-12-19T16:42:09.763+0000 7f1d204a9640 -1 osd.6 0 log_to_monitors true 2023-12-19T16:42:09.830 INFO:tasks.ceph.osd.5.smithi160.stderr:2023-12-19T16:42:09.827+0000 7f42e6c93640 -1 osd.5 0 log_to_monitors true 2023-12-19T16:42:09.942 INFO:tasks.ceph.osd.7.smithi160.stderr:2023-12-19T16:42:09.935+0000 7f760b895640 -1 osd.7 0 log_to_monitors true 2023-12-19T16:42:11.635 INFO:tasks.ceph.osd.0.smithi026.stderr:2023-12-19T16:42:11.633+0000 7f7b12e04640 -1 osd.0 0 waiting for initial osdmap 2023-12-19T16:42:11.675 INFO:tasks.ceph.osd.0.smithi026.stderr:2023-12-19T16:42:11.673+0000 7f7b0b449640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T16:42:13.169 INFO:tasks.ceph.osd.1.smithi026.stderr:2023-12-19T16:42:13.160+0000 7feff05eb640 -1 osd.1 0 waiting for initial osdmap 2023-12-19T16:42:13.169 INFO:tasks.ceph.osd.3.smithi026.stderr:2023-12-19T16:42:13.164+0000 7f2e16c66640 -1 osd.3 0 waiting for initial osdmap 2023-12-19T16:42:13.178 INFO:tasks.ceph.osd.2.smithi026.stderr:2023-12-19T16:42:13.176+0000 7fc71c6b8640 -1 osd.2 0 waiting for initial osdmap 2023-12-19T16:42:13.191 INFO:tasks.ceph.osd.1.smithi026.stderr:2023-12-19T16:42:13.188+0000 7fefe8c30640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T16:42:13.191 INFO:tasks.ceph.osd.3.smithi026.stderr:2023-12-19T16:42:13.188+0000 7f2e0f2ab640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T16:42:13.198 INFO:tasks.ceph.osd.2.smithi026.stderr:2023-12-19T16:42:13.196+0000 7fc714cfd640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T16:42:13.255 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:13.387 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ffab09640 1 Processor -- start 2023-12-19T16:42:13.387 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ffab09640 1 -- start start 2023-12-19T16:42:13.388 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ffab09640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0ff415cf60 0x7f0ff40d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:13.389 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ffab09640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ff40d1420 0x7f0ff40b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:13.389 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ffab09640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ff40b88e0 0x7f0ff40b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:13.389 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ffab09640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff40d2960 con 0x7f0ff40b88e0 2023-12-19T16:42:13.389 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ffab09640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff40d18c0 con 0x7f0ff415cf60 2023-12-19T16:42:13.389 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ffab09640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40d1ba0 con 0x7f0ff40d1420 2023-12-19T16:42:13.389 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ff90a6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ff40b88e0 0x7f0ff40b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:13.389 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ff90a6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ff40b88e0 0x7f0ff40b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53854/0 (socket says 172.21.15.26:53854) 2023-12-19T16:42:13.389 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ff90a6640 1 -- 172.21.15.26:0/1626007569 learned_addr learned my addr 172.21.15.26:0/1626007569 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:13.389 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ff3fff640 1 --2- 172.21.15.26:0/1626007569 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ff40d1420 0x7f0ff40b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:13.389 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ff88a5640 1 --2- 172.21.15.26:0/1626007569 >> v2:172.21.15.160:3300/0 conn(0x7f0ff415cf60 0x7f0ff40d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:13.389 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ff90a6640 1 -- 172.21.15.26:0/1626007569 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ff40d1420 msgr2=0x7f0ff40b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:13.389 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ff90a6640 1 --2- 172.21.15.26:0/1626007569 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ff40d1420 0x7f0ff40b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:13.390 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ff90a6640 1 -- 172.21.15.26:0/1626007569 >> v2:172.21.15.160:3300/0 conn(0x7f0ff415cf60 msgr2=0x7f0ff40d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:13.390 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ff90a6640 1 --2- 172.21.15.26:0/1626007569 >> v2:172.21.15.160:3300/0 conn(0x7f0ff415cf60 0x7f0ff40d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:13.390 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ff90a6640 1 -- 172.21.15.26:0/1626007569 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff40d1e80 con 0x7f0ff40b88e0 2023-12-19T16:42:13.390 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.384+0000 7f0ff88a5640 1 --2- 172.21.15.26:0/1626007569 >> v2:172.21.15.160:3300/0 conn(0x7f0ff415cf60 0x7f0ff40d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:13.391 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.388+0000 7f0ff90a6640 1 --2- 172.21.15.26:0/1626007569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ff40b88e0 0x7f0ff40b4b40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0ff406ae20 tx=0x7f0fec01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6119a73131fa1302 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:13.392 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.388+0000 7f0ff37fe640 1 -- 172.21.15.26:0/1626007569 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0af400 con 0x7f0ff40b88e0 2023-12-19T16:42:13.392 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.388+0000 7f0ff37fe640 1 -- 172.21.15.26:0/1626007569 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fec0ab050 con 0x7f0ff40b88e0 2023-12-19T16:42:13.392 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.388+0000 7f0ff37fe640 1 -- 172.21.15.26:0/1626007569 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0af780 con 0x7f0ff40b88e0 2023-12-19T16:42:13.393 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.388+0000 7f0ffab09640 1 -- 172.21.15.26:0/1626007569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ff40b88e0 msgr2=0x7f0ff40b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:13.393 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.388+0000 7f0ffab09640 1 --2- 172.21.15.26:0/1626007569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ff40b88e0 0x7f0ff40b4b40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0ff406ae20 tx=0x7f0fec01f1c0 comp rx=0 tx=0).stop 2023-12-19T16:42:13.393 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.388+0000 7f0ffab09640 1 -- 172.21.15.26:0/1626007569 shutdown_connections 2023-12-19T16:42:13.393 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.388+0000 7f0ffab09640 1 --2- 172.21.15.26:0/1626007569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ff40b88e0 0x7f0ff40b4b40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:13.393 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.388+0000 7f0ffab09640 1 --2- 172.21.15.26:0/1626007569 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ff40d1420 0x7f0ff40b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:13.393 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.388+0000 7f0ffab09640 1 --2- 172.21.15.26:0/1626007569 >> v2:172.21.15.160:3300/0 conn(0x7f0ff415cf60 0x7f0ff40d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:13.393 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.388+0000 7f0ffab09640 1 -- 172.21.15.26:0/1626007569 >> 172.21.15.26:0/1626007569 conn(0x7f0ff4065ee0 msgr2=0x7f0ff40cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:13.393 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.388+0000 7f0ffab09640 1 -- 172.21.15.26:0/1626007569 shutdown_connections 2023-12-19T16:42:13.393 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.388+0000 7f0ffab09640 1 -- 172.21.15.26:0/1626007569 wait complete. 2023-12-19T16:42:13.394 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ffab09640 1 Processor -- start 2023-12-19T16:42:13.394 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ffab09640 1 -- start start 2023-12-19T16:42:13.397 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ffab09640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ff40b88e0 0x7f0ff4153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:13.397 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ffab09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ff40d1420 0x7f0ff4152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:13.397 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ffab09640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ff415cf60 0x7f0ff4155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:13.397 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ffab09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff4144c40 con 0x7f0ff40d1420 2023-12-19T16:42:13.397 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ffab09640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40cade0 con 0x7f0ff415cf60 2023-12-19T16:42:13.397 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ffab09640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40cb050 con 0x7f0ff40b88e0 2023-12-19T16:42:13.397 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ff40d1420 0x7f0ff4152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:13.397 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ff40d1420 0x7f0ff4152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53866/0 (socket says 172.21.15.26:53866) 2023-12-19T16:42:13.397 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff3fff640 1 -- 172.21.15.26:0/190408369 learned_addr learned my addr 172.21.15.26:0/190408369 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:13.397 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff88a5640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ff40b88e0 0x7f0ff4153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:13.397 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff90a6640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ff415cf60 0x7f0ff4155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:13.398 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff3fff640 1 -- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ff40b88e0 msgr2=0x7f0ff4153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:13.398 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff3fff640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ff40b88e0 0x7f0ff4153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:13.398 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff3fff640 1 -- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ff415cf60 msgr2=0x7f0ff4155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:13.398 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff3fff640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ff415cf60 0x7f0ff4155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:13.398 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff3fff640 1 -- 172.21.15.26:0/190408369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec009b30 con 0x7f0ff40d1420 2023-12-19T16:42:13.398 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff88a5640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ff40b88e0 0x7f0ff4153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:13.398 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff3fff640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ff40d1420 0x7f0ff4152a80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000d830 tx=0x7f0fe000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:13.398 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff17fa640 1 -- 172.21.15.26:0/190408369 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00096a0 con 0x7f0ff40d1420 2023-12-19T16:42:13.398 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff17fa640 1 -- 172.21.15.26:0/190408369 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe0009800 con 0x7f0ff40d1420 2023-12-19T16:42:13.398 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ffab09640 1 -- 172.21.15.26:0/190408369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff40cb330 con 0x7f0ff40d1420 2023-12-19T16:42:13.398 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ff17fa640 1 -- 172.21.15.26:0/190408369 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0009b70 con 0x7f0ff40d1420 2023-12-19T16:42:13.398 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ffab09640 1 -- 172.21.15.26:0/190408369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff40cb8f0 con 0x7f0ff40d1420 2023-12-19T16:42:13.399 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.392+0000 7f0ffab09640 1 -- 172.21.15.26:0/190408369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff4074270 con 0x7f0ff40d1420 2023-12-19T16:42:13.399 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.396+0000 7f0ff17fa640 1 -- 172.21.15.26:0/190408369 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 85172+0+0 (secure 0 0 0) 0x7f0fe0014410 con 0x7f0ff40d1420 2023-12-19T16:42:13.399 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.396+0000 7f0ff17fa640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0fd80659d0 0x7f0fd8067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:13.399 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.396+0000 7f0ff17fa640 1 -- 172.21.15.26:0/190408369 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2511+0+0 (secure 0 0 0) 0x7f0fe0082d50 con 0x7f0ff40d1420 2023-12-19T16:42:13.400 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.396+0000 7f0ff88a5640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0fd80659d0 0x7f0fd8067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:13.404 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.400+0000 7f0ff88a5640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0fd80659d0 0x7f0fd8067e90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4160100 tx=0x7f0fe8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:13.404 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.400+0000 7f0ff17fa640 1 -- 172.21.15.26:0/190408369 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0fe0051460 con 0x7f0ff40d1420 2023-12-19T16:42:13.563 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ffab09640 1 -- 172.21.15.26:0/190408369 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ff40d0bc0 con 0x7f0ff40d1420 2023-12-19T16:42:13.564 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ff17fa640 1 -- 172.21.15.26:0/190408369 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7494 (secure 0 0 0) 0x7f0fe0054a80 con 0x7f0ff40d1420 2023-12-19T16:42:13.564 INFO:teuthology.misc.health.smithi026.stdout: 2023-12-19T16:42:13.564 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":14,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:12.650084+0000","last_up_change":"2023-12-19T16:42:12.650084+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:13.566 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ffab09640 1 -- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0fd80659d0 msgr2=0x7f0fd8067e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:13.566 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ffab09640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0fd80659d0 0x7f0fd8067e90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4160100 tx=0x7f0fe8008040 comp rx=0 tx=0).stop 2023-12-19T16:42:13.566 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ffab09640 1 -- 172.21.15.26:0/190408369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ff40d1420 msgr2=0x7f0ff4152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:13.566 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ffab09640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ff40d1420 0x7f0ff4152a80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0fe000d830 tx=0x7f0fe000dcf0 comp rx=0 tx=0).stop 2023-12-19T16:42:13.566 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ffab09640 1 -- 172.21.15.26:0/190408369 shutdown_connections 2023-12-19T16:42:13.566 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ffab09640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ff415cf60 0x7f0ff4155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:13.566 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ffab09640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ff40d1420 0x7f0ff4152a80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:13.566 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ffab09640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0fd80659d0 0x7f0fd8067e90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:13.566 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ffab09640 1 --2- 172.21.15.26:0/190408369 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ff40b88e0 0x7f0ff4153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:13.566 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ffab09640 1 -- 172.21.15.26:0/190408369 >> 172.21.15.26:0/190408369 conn(0x7f0ff4065ee0 msgr2=0x7f0ff40b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:13.567 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ffab09640 1 -- 172.21.15.26:0/190408369 shutdown_connections 2023-12-19T16:42:13.567 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:13.560+0000 7f0ffab09640 1 -- 172.21.15.26:0/190408369 wait complete. 2023-12-19T16:42:13.584 DEBUG:teuthology.misc:1 of 8 OSDs are up 2023-12-19T16:42:13.876 INFO:tasks.ceph.osd.4.smithi160.stderr:2023-12-19T16:42:13.871+0000 7f7fb4487640 -1 osd.4 0 waiting for initial osdmap 2023-12-19T16:42:13.915 INFO:tasks.ceph.osd.4.smithi160.stderr:2023-12-19T16:42:13.911+0000 7f7facacc640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T16:42:13.969 INFO:tasks.ceph.osd.7.smithi160.stderr:2023-12-19T16:42:13.963+0000 7f7607017640 -1 osd.7 0 waiting for initial osdmap 2023-12-19T16:42:14.011 INFO:tasks.ceph.osd.7.smithi160.stderr:2023-12-19T16:42:14.007+0000 7f75ff65c640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T16:42:14.064 INFO:tasks.ceph.osd.6.smithi160.stderr:2023-12-19T16:42:14.059+0000 7f1d1bc2b640 -1 osd.6 0 waiting for initial osdmap 2023-12-19T16:42:14.091 INFO:tasks.ceph.osd.6.smithi160.stderr:2023-12-19T16:42:14.087+0000 7f1d14270640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T16:42:14.123 INFO:tasks.ceph.osd.5.smithi160.stderr:2023-12-19T16:42:14.119+0000 7f42e2415640 -1 osd.5 0 waiting for initial osdmap 2023-12-19T16:42:14.147 INFO:tasks.ceph.osd.5.smithi160.stderr:2023-12-19T16:42:14.143+0000 7f42daa5a640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T16:42:20.391 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:20.534 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.528+0000 7fe9135e9640 1 Processor -- start 2023-12-19T16:42:20.534 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe9135e9640 1 -- start start 2023-12-19T16:42:20.535 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe9135e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe90c154220 0x7fe90c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:20.536 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe9135e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90c06bc30 con 0x7fe90c0cee10 2023-12-19T16:42:20.536 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe9135e9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90c0cc6e0 con 0x7fe90c06b3d0 2023-12-19T16:42:20.536 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe9135e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c15afc0 con 0x7fe90c154220 2023-12-19T16:42:20.536 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe911b86640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe90c0cee10 0x7fe90c15b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38782/0 (socket says 172.21.15.26:38782) 2023-12-19T16:42:20.536 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe911b86640 1 -- 172.21.15.26:0/1497232734 learned_addr learned my addr 172.21.15.26:0/1497232734 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:20.536 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe911385640 1 --2- 172.21.15.26:0/1497232734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe90c154220 0x7fe90c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:20.537 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe903fff640 1 -- 172.21.15.26:0/1497232734 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466636695 0 0) 0x7fe90c06bc30 con 0x7fe90c0cee10 2023-12-19T16:42:20.537 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe903fff640 1 -- 172.21.15.26:0/1497232734 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f8003530 con 0x7fe90c0cee10 2023-12-19T16:42:20.537 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe903fff640 1 -- 172.21.15.26:0/1497232734 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421583330 0 0) 0x7fe90c0cc6e0 con 0x7fe90c06b3d0 2023-12-19T16:42:20.537 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe903fff640 1 -- 172.21.15.26:0/1497232734 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe90c06bc30 con 0x7fe90c06b3d0 2023-12-19T16:42:20.537 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe911385640 1 -- 172.21.15.26:0/1497232734 >> v1:172.21.15.160:6789/0 conn(0x7fe90c06b3d0 legacy=0x7fe90c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:20.537 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe911385640 1 -- 172.21.15.26:0/1497232734 >> v1:172.21.15.26:6789/0 conn(0x7fe90c0cee10 legacy=0x7fe90c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:20.537 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.532+0000 7fe911385640 1 -- 172.21.15.26:0/1497232734 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c06b950 con 0x7fe90c154220 2023-12-19T16:42:20.541 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.536+0000 7fe911385640 1 --2- 172.21.15.26:0/1497232734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe90c154220 0x7fe90c15aa80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe9080097f0 tx=0x7fe90801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc6578cf222a8d0b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:20.542 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.536+0000 7fe903fff640 1 -- 172.21.15.26:0/1497232734 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9080aa5f0 con 0x7fe90c154220 2023-12-19T16:42:20.542 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.536+0000 7fe903fff640 1 -- 172.21.15.26:0/1497232734 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9080aa750 con 0x7fe90c154220 2023-12-19T16:42:20.542 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.536+0000 7fe903fff640 1 -- 172.21.15.26:0/1497232734 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9080aaae0 con 0x7fe90c154220 2023-12-19T16:42:20.542 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.536+0000 7fe9135e9640 1 -- 172.21.15.26:0/1497232734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe90c154220 msgr2=0x7fe90c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:20.542 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.536+0000 7fe9135e9640 1 --2- 172.21.15.26:0/1497232734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe90c154220 0x7fe90c15aa80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe9080097f0 tx=0x7fe90801c490 comp rx=0 tx=0).stop 2023-12-19T16:42:20.542 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.536+0000 7fe9135e9640 1 -- 172.21.15.26:0/1497232734 shutdown_connections 2023-12-19T16:42:20.542 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.536+0000 7fe9135e9640 1 --2- 172.21.15.26:0/1497232734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe90c154220 0x7fe90c15aa80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.542 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.536+0000 7fe9135e9640 1 -- 172.21.15.26:0/1497232734 >> 172.21.15.26:0/1497232734 conn(0x7fe90c0658e0 msgr2=0x7fe90c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:20.542 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.536+0000 7fe9135e9640 1 -- 172.21.15.26:0/1497232734 shutdown_connections 2023-12-19T16:42:20.542 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.536+0000 7fe9135e9640 1 -- 172.21.15.26:0/1497232734 wait complete. 2023-12-19T16:42:20.545 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe9135e9640 1 Processor -- start 2023-12-19T16:42:20.545 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe9135e9640 1 -- start start 2023-12-19T16:42:20.546 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe9135e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe90c06b3d0 0x7fe90c14cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:20.546 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe9135e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe90c0cee10 0x7fe90c14e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:20.546 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe9135e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe90c154220 0x7fe90c158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:20.546 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe9135e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c14d710 con 0x7fe90c0cee10 2023-12-19T16:42:20.546 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe9135e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c150300 con 0x7fe90c06b3d0 2023-12-19T16:42:20.546 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe9135e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c1505e0 con 0x7fe90c154220 2023-12-19T16:42:20.546 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe910b84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe90c0cee10 0x7fe90c14e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:20.546 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe910b84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe90c0cee10 0x7fe90c14e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44050/0 (socket says 172.21.15.26:44050) 2023-12-19T16:42:20.546 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe910b84640 1 -- 172.21.15.26:0/3477667579 learned_addr learned my addr 172.21.15.26:0/3477667579 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:20.546 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe911385640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe90c06b3d0 0x7fe90c14cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:20.546 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe911b86640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe90c154220 0x7fe90c158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:20.546 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe910b84640 1 -- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe90c154220 msgr2=0x7fe90c158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:20.546 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe910b84640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe90c154220 0x7fe90c158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.547 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe910b84640 1 -- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe90c06b3d0 msgr2=0x7fe90c14cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:20.547 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe910b84640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe90c06b3d0 0x7fe90c14cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.547 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe910b84640 1 -- 172.21.15.26:0/3477667579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe908009420 con 0x7fe90c0cee10 2023-12-19T16:42:20.547 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe910b84640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe90c0cee10 0x7fe90c14e4b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00c430 tx=0x7fe8fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:20.547 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe901ffb640 1 -- 172.21.15.26:0/3477667579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0052d0 con 0x7fe90c0cee10 2023-12-19T16:42:20.547 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe901ffb640 1 -- 172.21.15.26:0/3477667579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc005450 con 0x7fe90c0cee10 2023-12-19T16:42:20.547 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe9135e9640 1 -- 172.21.15.26:0/3477667579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90c1508c0 con 0x7fe90c0cee10 2023-12-19T16:42:20.547 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe901ffb640 1 -- 172.21.15.26:0/3477667579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc003840 con 0x7fe90c0cee10 2023-12-19T16:42:20.548 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe9135e9640 1 -- 172.21.15.26:0/3477667579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90c151220 con 0x7fe90c0cee10 2023-12-19T16:42:20.548 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.540+0000 7fe9135e9640 1 -- 172.21.15.26:0/3477667579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90c074270 con 0x7fe90c0cee10 2023-12-19T16:42:20.549 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.544+0000 7fe901ffb640 1 -- 172.21.15.26:0/3477667579 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe8fc0039a0 con 0x7fe90c0cee10 2023-12-19T16:42:20.549 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.544+0000 7fe901ffb640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe8f0065a60 0x7fe8f0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:20.549 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.544+0000 7fe901ffb640 1 -- 172.21.15.26:0/3477667579 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe8fc0825d0 con 0x7fe90c0cee10 2023-12-19T16:42:20.549 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.544+0000 7fe911385640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe8f0065a60 0x7fe8f0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:20.549 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.544+0000 7fe911385640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe8f0065a60 0x7fe8f0067f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9080ae000 tx=0x7fe9080a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:20.554 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.548+0000 7fe901ffb640 1 -- 172.21.15.26:0/3477667579 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe8fc0501b0 con 0x7fe90c0cee10 2023-12-19T16:42:20.711 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe9135e9640 1 -- 172.21.15.26:0/3477667579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe90c0cdf60 con 0x7fe90c0cee10 2023-12-19T16:42:20.711 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe901ffb640 1 -- 172.21.15.26:0/3477667579 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12382 (secure 0 0 0) 0x7fe8fc017090 con 0x7fe90c0cee10 2023-12-19T16:42:20.712 INFO:teuthology.misc.health.smithi026.stdout: 2023-12-19T16:42:20.712 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":19,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:18.207464+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:20.713 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe9135e9640 1 -- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe8f0065a60 msgr2=0x7fe8f0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:20.714 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe9135e9640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe8f0065a60 0x7fe8f0067f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9080ae000 tx=0x7fe9080a7000 comp rx=0 tx=0).stop 2023-12-19T16:42:20.714 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe9135e9640 1 -- 172.21.15.26:0/3477667579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe90c0cee10 msgr2=0x7fe90c14e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:20.714 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe9135e9640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe90c0cee10 0x7fe90c14e4b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00c430 tx=0x7fe8fc00c8f0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.714 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe9135e9640 1 -- 172.21.15.26:0/3477667579 shutdown_connections 2023-12-19T16:42:20.714 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe9135e9640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe8f0065a60 0x7fe8f0067f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.714 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe9135e9640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe90c154220 0x7fe90c158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.714 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe9135e9640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe90c0cee10 0x7fe90c14e4b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.714 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe9135e9640 1 --2- 172.21.15.26:0/3477667579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe90c06b3d0 0x7fe90c14cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.714 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe9135e9640 1 -- 172.21.15.26:0/3477667579 >> 172.21.15.26:0/3477667579 conn(0x7fe90c0658e0 msgr2=0x7fe90c065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:20.714 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe9135e9640 1 -- 172.21.15.26:0/3477667579 shutdown_connections 2023-12-19T16:42:20.714 INFO:teuthology.misc.health.smithi026.stderr:2023-12-19T16:42:20.708+0000 7fe9135e9640 1 -- 172.21.15.26:0/3477667579 wait complete. 2023-12-19T16:42:20.732 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-12-19T16:42:20.732 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-12-19T16:42:20.732 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T16:42:20.733 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T16:42:20.736 INFO:tasks.ceph.mds.a:Started 2023-12-19T16:42:20.736 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T16:42:20.736 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T16:42:20.739 INFO:tasks.ceph.mds.c:Started 2023-12-19T16:42:20.740 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T16:42:20.740 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T16:42:20.745 INFO:tasks.ceph.mds.b:Started 2023-12-19T16:42:20.746 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T16:42:20.746 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T16:42:20.750 INFO:tasks.ceph.mds.d:Started 2023-12-19T16:42:20.750 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-12-19T16:42:20.750 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:20.840 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T16:42:20.840 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T16:42:20.853 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T16:42:20.853 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T16:42:20.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.896+0000 7fb7593b3640 1 Processor -- start 2023-12-19T16:42:20.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.896+0000 7fb7593b3640 1 -- start start 2023-12-19T16:42:20.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.896+0000 7fb7593b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb7540cc9b0 0x7fb7540cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:20.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.896+0000 7fb7593b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb7540afd70 0x7fb7540cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:20.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.896+0000 7fb7593b3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb75406b730 con 0x7fb7540ce520 2023-12-19T16:42:20.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.896+0000 7fb7593b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7541617d0 con 0x7fb7540cc9b0 2023-12-19T16:42:20.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.896+0000 7fb7593b3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7540b3020 con 0x7fb7540afd70 2023-12-19T16:42:20.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.900+0000 7fb7527fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb7540afd70 0x7fb7540cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:20.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.900+0000 7fb752ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb7540cc9b0 0x7fb7540cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:20.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.900+0000 7fb7527fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb7540afd70 0x7fb7540cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:38806/0 (socket says 172.21.15.26:38806) 2023-12-19T16:42:20.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.900+0000 7fb7527fc640 1 -- 172.21.15.26:0/3838755067 learned_addr learned my addr 172.21.15.26:0/3838755067 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:20.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.900+0000 7fb7537fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb7540ce520 0x7fb754069d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38798/0 (socket says 172.21.15.26:38798) 2023-12-19T16:42:20.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.900+0000 7fb752ffd640 1 -- 172.21.15.26:0/3838755067 >> v2:172.21.15.160:3300/0 conn(0x7fb7540afd70 msgr2=0x7fb7540cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:20.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.900+0000 7fb752ffd640 1 --2- 172.21.15.26:0/3838755067 >> v2:172.21.15.160:3300/0 conn(0x7fb7540afd70 0x7fb7540cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.900+0000 7fb752ffd640 1 -- 172.21.15.26:0/3838755067 >> v1:172.21.15.26:6789/0 conn(0x7fb7540ce520 legacy=0x7fb754069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:20.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.900+0000 7fb752ffd640 1 -- 172.21.15.26:0/3838755067 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb754161a40 con 0x7fb7540cc9b0 2023-12-19T16:42:20.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.900+0000 7fb751ffb640 1 -- 172.21.15.26:0/3838755067 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546252430 0 0) 0x7fb75406b730 con 0x7fb7540ce520 2023-12-19T16:42:20.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.900+0000 7fb752ffd640 1 --2- 172.21.15.26:0/3838755067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cc9b0 0x7fb7540cd1d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb75406ef30 tx=0x7fb74401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2da6b30d82847e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:20.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb751ffb640 1 -- 172.21.15.26:0/3838755067 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7440ab600 con 0x7fb7540cc9b0 2023-12-19T16:42:20.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb751ffb640 1 -- 172.21.15.26:0/3838755067 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7440a7050 con 0x7fb7540cc9b0 2023-12-19T16:42:20.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb751ffb640 1 -- 172.21.15.26:0/3838755067 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7440ab980 con 0x7fb7540cc9b0 2023-12-19T16:42:20.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 -- 172.21.15.26:0/3838755067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cc9b0 msgr2=0x7fb7540cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:20.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 --2- 172.21.15.26:0/3838755067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cc9b0 0x7fb7540cd1d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb75406ef30 tx=0x7fb74401c3d0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 -- 172.21.15.26:0/3838755067 shutdown_connections 2023-12-19T16:42:20.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 --2- 172.21.15.26:0/3838755067 >> v2:172.21.15.160:3300/0 conn(0x7fb7540afd70 0x7fb7540cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 --2- 172.21.15.26:0/3838755067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cc9b0 0x7fb7540cd1d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 -- 172.21.15.26:0/3838755067 >> 172.21.15.26:0/3838755067 conn(0x7fb75406d570 msgr2=0x7fb75406d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:20.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 -- 172.21.15.26:0/3838755067 shutdown_connections 2023-12-19T16:42:20.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 -- 172.21.15.26:0/3838755067 wait complete. 2023-12-19T16:42:20.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 Processor -- start 2023-12-19T16:42:20.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 -- start start 2023-12-19T16:42:20.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540afd70 0x7fb7540ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:20.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cc9b0 0x7fb7541600a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:20.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7540ce520 0x7fb754148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:20.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7540c7a20 con 0x7fb7540cc9b0 2023-12-19T16:42:20.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7540ca680 con 0x7fb7540ce520 2023-12-19T16:42:20.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.904+0000 7fb7593b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7540c7c70 con 0x7fb7540afd70 2023-12-19T16:42:20.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb7527fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cc9b0 0x7fb7541600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:20.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb7527fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cc9b0 0x7fb7541600a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44116/0 (socket says 172.21.15.26:44116) 2023-12-19T16:42:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb7527fc640 1 -- 172.21.15.26:0/2064881555 learned_addr learned my addr 172.21.15.26:0/2064881555 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb752ffd640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540afd70 0x7fb7540ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb7537fe640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7540ce520 0x7fb754148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb7527fc640 1 -- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540afd70 msgr2=0x7fb7540ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb7527fc640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540afd70 0x7fb7540ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb7527fc640 1 -- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7540ce520 msgr2=0x7fb754148c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb7527fc640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7540ce520 0x7fb754148c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb7527fc640 1 -- 172.21.15.26:0/2064881555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb744009420 con 0x7fb7540cc9b0 2023-12-19T16:42:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb7527fc640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cc9b0 0x7fb7541600a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb74800e7f0 tx=0x7fb74800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb73b7fe640 1 -- 172.21.15.26:0/2064881555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb748009580 con 0x7fb7540cc9b0 2023-12-19T16:42:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb73b7fe640 1 -- 172.21.15.26:0/2064881555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74800c940 con 0x7fb7540cc9b0 2023-12-19T16:42:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb7593b3640 1 -- 172.21.15.26:0/2064881555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7540c7f50 con 0x7fb7540cc9b0 2023-12-19T16:42:20.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb73b7fe640 1 -- 172.21.15.26:0/2064881555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74800ccf0 con 0x7fb7540cc9b0 2023-12-19T16:42:20.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb7593b3640 1 -- 172.21.15.26:0/2064881555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7541605e0 con 0x7fb7540cc9b0 2023-12-19T16:42:20.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb7593b3640 1 -- 172.21.15.26:0/2064881555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb754074270 con 0x7fb7540cc9b0 2023-12-19T16:42:20.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.908+0000 7fb73b7fe640 1 -- 172.21.15.26:0/2064881555 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb748013470 con 0x7fb7540cc9b0 2023-12-19T16:42:20.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.912+0000 7fb73b7fe640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb740065a60 0x7fb740067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:20.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.912+0000 7fb73b7fe640 1 -- 172.21.15.26:0/2064881555 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb748082ec0 con 0x7fb7540cc9b0 2023-12-19T16:42:20.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.912+0000 7fb752ffd640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb740065a60 0x7fb740067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:20.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.912+0000 7fb752ffd640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb740065a60 0x7fb740067f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb7440ad000 tx=0x7fb7440a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:20.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:20.912+0000 7fb73b7fe640 1 -- 172.21.15.26:0/2064881555 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb748050be0 con 0x7fb7540cc9b0 2023-12-19T16:42:21.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.072+0000 7fb7593b3640 1 -- 172.21.15.26:0/2064881555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7540b27c0 con 0x7fb7540cc9b0 2023-12-19T16:42:21.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.076+0000 7fb73b7fe640 1 -- 172.21.15.26:0/2064881555 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12382 (secure 0 0 0) 0x7fb748017090 con 0x7fb7540cc9b0 2023-12-19T16:42:21.078 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:21.078 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":19,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:18.207464+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.076+0000 7fb7593b3640 1 -- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb740065a60 msgr2=0x7fb740067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.076+0000 7fb7593b3640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb740065a60 0x7fb740067f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb7440ad000 tx=0x7fb7440a6000 comp rx=0 tx=0).stop 2023-12-19T16:42:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.076+0000 7fb7593b3640 1 -- 172.21.15.26:0/2064881555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cc9b0 msgr2=0x7fb7541600a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.076+0000 7fb7593b3640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cc9b0 0x7fb7541600a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb74800e7f0 tx=0x7fb74800ecb0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.076+0000 7fb7593b3640 1 -- 172.21.15.26:0/2064881555 shutdown_connections 2023-12-19T16:42:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.076+0000 7fb7593b3640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7540ce520 0x7fb754148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.076+0000 7fb7593b3640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cc9b0 0x7fb7541600a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.076+0000 7fb7593b3640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb740065a60 0x7fb740067f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.076+0000 7fb7593b3640 1 --2- 172.21.15.26:0/2064881555 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540afd70 0x7fb7540ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.076+0000 7fb7593b3640 1 -- 172.21.15.26:0/2064881555 >> 172.21.15.26:0/2064881555 conn(0x7fb75406d570 msgr2=0x7fb75406d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.076+0000 7fb7593b3640 1 -- 172.21.15.26:0/2064881555 shutdown_connections 2023-12-19T16:42:21.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.076+0000 7fb7593b3640 1 -- 172.21.15.26:0/2064881555 wait complete. 2023-12-19T16:42:21.100 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:42:21.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:42:21.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.244+0000 7fdf7df98640 1 Processor -- start 2023-12-19T16:42:21.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.244+0000 7fdf7df98640 1 -- start start 2023-12-19T16:42:21.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.244+0000 7fdf7df98640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf780ca060 0x7fdf7806d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:21.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.244+0000 7fdf7df98640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdf780658e0 0x7fdf78065d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:21.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.244+0000 7fdf7df98640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf780cd940 con 0x7fdf7815cdb0 2023-12-19T16:42:21.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.244+0000 7fdf7df98640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7815d190 con 0x7fdf780ca060 2023-12-19T16:42:21.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.244+0000 7fdf7df98640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf78066290 con 0x7fdf780658e0 2023-12-19T16:42:21.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.248+0000 7fdf76ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf780ca060 0x7fdf7806d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:21.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.248+0000 7fdf777fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdf7815cdb0 0x7fdf78161280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38812/0 (socket says 172.21.15.26:38812) 2023-12-19T16:42:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.248+0000 7fdf777fe640 1 -- 172.21.15.26:0/2551076790 learned_addr learned my addr 172.21.15.26:0/2551076790 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.248+0000 7fdf77fff640 1 --2- 172.21.15.26:0/2551076790 >> v2:172.21.15.160:3300/0 conn(0x7fdf780658e0 0x7fdf78065d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.248+0000 7fdf76ffd640 1 -- 172.21.15.26:0/2551076790 >> v2:172.21.15.160:3300/0 conn(0x7fdf780658e0 msgr2=0x7fdf78065d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.248+0000 7fdf76ffd640 1 --2- 172.21.15.26:0/2551076790 >> v2:172.21.15.160:3300/0 conn(0x7fdf780658e0 0x7fdf78065d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.248+0000 7fdf76ffd640 1 -- 172.21.15.26:0/2551076790 >> v1:172.21.15.26:6789/0 conn(0x7fdf7815cdb0 legacy=0x7fdf78161280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.248+0000 7fdf76ffd640 1 -- 172.21.15.26:0/2551076790 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf780cdf50 con 0x7fdf780ca060 2023-12-19T16:42:21.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.248+0000 7fdf767fc640 1 -- 172.21.15.26:0/2551076790 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908953558 0 0) 0x7fdf780cd940 con 0x7fdf7815cdb0 2023-12-19T16:42:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.248+0000 7fdf76ffd640 1 --2- 172.21.15.26:0/2551076790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf780ca060 0x7fdf7806d610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf780cd3e0 tx=0x7fdf6c01c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc0fe03e7c93d1e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:21.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.248+0000 7fdf767fc640 1 -- 172.21.15.26:0/2551076790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0ab600 con 0x7fdf780ca060 2023-12-19T16:42:21.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.248+0000 7fdf767fc640 1 -- 172.21.15.26:0/2551076790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c0a7050 con 0x7fdf780ca060 2023-12-19T16:42:21.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.248+0000 7fdf767fc640 1 -- 172.21.15.26:0/2551076790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0ab980 con 0x7fdf780ca060 2023-12-19T16:42:21.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 -- 172.21.15.26:0/2551076790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf780ca060 msgr2=0x7fdf7806d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 --2- 172.21.15.26:0/2551076790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf780ca060 0x7fdf7806d610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdf780cd3e0 tx=0x7fdf6c01c2f0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 -- 172.21.15.26:0/2551076790 shutdown_connections 2023-12-19T16:42:21.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 --2- 172.21.15.26:0/2551076790 >> v2:172.21.15.160:3300/0 conn(0x7fdf780658e0 0x7fdf78065d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 --2- 172.21.15.26:0/2551076790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf780ca060 0x7fdf7806d610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 -- 172.21.15.26:0/2551076790 >> 172.21.15.26:0/2551076790 conn(0x7fdf78069d20 msgr2=0x7fdf7806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:21.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 -- 172.21.15.26:0/2551076790 shutdown_connections 2023-12-19T16:42:21.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 -- 172.21.15.26:0/2551076790 wait complete. 2023-12-19T16:42:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 Processor -- start 2023-12-19T16:42:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 -- start start 2023-12-19T16:42:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf780658e0 0x7fdf780c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf780ca060 0x7fdf780ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf7815cdb0 0x7fdf7806ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf780cadb0 con 0x7fdf780ca060 2023-12-19T16:42:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf780cb090 con 0x7fdf7815cdb0 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf7df98640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf780cb370 con 0x7fdf780658e0 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf76ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf780ca060 0x7fdf780ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf76ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf780ca060 0x7fdf780ca460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44140/0 (socket says 172.21.15.26:44140) 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf76ffd640 1 -- 172.21.15.26:0/906909479 learned_addr learned my addr 172.21.15.26:0/906909479 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf777fe640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf780658e0 0x7fdf780c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf77fff640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf7815cdb0 0x7fdf7806ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf76ffd640 1 -- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf780658e0 msgr2=0x7fdf780c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf76ffd640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf780658e0 0x7fdf780c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf76ffd640 1 -- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf7815cdb0 msgr2=0x7fdf7806ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf76ffd640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf7815cdb0 0x7fdf7806ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.252+0000 7fdf76ffd640 1 -- 172.21.15.26:0/906909479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c009420 con 0x7fdf780ca060 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.256+0000 7fdf76ffd640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf780ca060 0x7fdf780ca460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c018f00 tx=0x7fdf6c01ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.256+0000 7fdf5ffff640 1 -- 172.21.15.26:0/906909479 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0b7450 con 0x7fdf780ca060 2023-12-19T16:42:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.256+0000 7fdf5ffff640 1 -- 172.21.15.26:0/906909479 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c0b75b0 con 0x7fdf780ca060 2023-12-19T16:42:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.256+0000 7fdf5ffff640 1 -- 172.21.15.26:0/906909479 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0b7960 con 0x7fdf780ca060 2023-12-19T16:42:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.256+0000 7fdf7df98640 1 -- 172.21.15.26:0/906909479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7815f150 con 0x7fdf780ca060 2023-12-19T16:42:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.256+0000 7fdf7df98640 1 -- 172.21.15.26:0/906909479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf780c7070 con 0x7fdf780ca060 2023-12-19T16:42:21.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.256+0000 7fdf7df98640 1 -- 172.21.15.26:0/906909479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf78074270 con 0x7fdf780ca060 2023-12-19T16:42:21.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.256+0000 7fdf5ffff640 1 -- 172.21.15.26:0/906909479 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdf6c0b7ac0 con 0x7fdf780ca060 2023-12-19T16:42:21.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.256+0000 7fdf5ffff640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf64065a60 0x7fdf64067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:21.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.256+0000 7fdf5ffff640 1 -- 172.21.15.26:0/906909479 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdf6c11d670 con 0x7fdf780ca060 2023-12-19T16:42:21.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.256+0000 7fdf777fe640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf64065a60 0x7fdf64067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:21.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.260+0000 7fdf777fe640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf64065a60 0x7fdf64067f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdf68000f70 tx=0x7fdf68007c10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:21.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.260+0000 7fdf5ffff640 1 -- 172.21.15.26:0/906909479 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdf6c0eb370 con 0x7fdf780ca060 2023-12-19T16:42:21.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf7df98640 1 -- 172.21.15.26:0/906909479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf78070c80 con 0x7fdf780ca060 2023-12-19T16:42:21.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf5ffff640 1 -- 172.21.15.26:0/906909479 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf6c0bb020 con 0x7fdf780ca060 2023-12-19T16:42:21.424 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:42:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf7df98640 1 -- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf64065a60 msgr2=0x7fdf64067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf7df98640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf64065a60 0x7fdf64067f20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdf68000f70 tx=0x7fdf68007c10 comp rx=0 tx=0).stop 2023-12-19T16:42:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf7df98640 1 -- 172.21.15.26:0/906909479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf780ca060 msgr2=0x7fdf780ca460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf7df98640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf780ca060 0x7fdf780ca460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c018f00 tx=0x7fdf6c01ab30 comp rx=0 tx=0).stop 2023-12-19T16:42:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf7df98640 1 -- 172.21.15.26:0/906909479 shutdown_connections 2023-12-19T16:42:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf7df98640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf7815cdb0 0x7fdf7806ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf7df98640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf780ca060 0x7fdf780ca460 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf7df98640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf64065a60 0x7fdf64067f20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf7df98640 1 --2- 172.21.15.26:0/906909479 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf780658e0 0x7fdf780c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf7df98640 1 -- 172.21.15.26:0/906909479 >> 172.21.15.26:0/906909479 conn(0x7fdf78069d20 msgr2=0x7fdf7815bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf7df98640 1 -- 172.21.15.26:0/906909479 shutdown_connections 2023-12-19T16:42:21.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.420+0000 7fdf7df98640 1 -- 172.21.15.26:0/906909479 wait complete. 2023-12-19T16:42:21.446 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:42:21.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.584+0000 7f543c16d640 1 Processor -- start 2023-12-19T16:42:21.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.584+0000 7f543c16d640 1 -- start start 2023-12-19T16:42:21.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.584+0000 7f543c16d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5434154220 0x7f543415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:21.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.584+0000 7f543c16d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543406bc30 con 0x7f54340cee10 2023-12-19T16:42:21.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.584+0000 7f543c16d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54340cc6e0 con 0x7f543406b3d0 2023-12-19T16:42:21.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.584+0000 7f543c16d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543415afc0 con 0x7f5434154220 2023-12-19T16:42:21.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.588+0000 7f5439f09640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5434154220 0x7f543415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:21.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.588+0000 7f543a70a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f54340cee10 0x7f543415b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38814/0 (socket says 172.21.15.26:38814) 2023-12-19T16:42:21.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.588+0000 7f543a70a640 1 -- 172.21.15.26:0/1107184196 learned_addr learned my addr 172.21.15.26:0/1107184196 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:21.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.588+0000 7f5439f09640 1 -- 172.21.15.26:0/1107184196 >> v1:172.21.15.160:6789/0 conn(0x7f543406b3d0 legacy=0x7f54340ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:21.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.588+0000 7f5439f09640 1 -- 172.21.15.26:0/1107184196 >> v1:172.21.15.26:6789/0 conn(0x7f54340cee10 legacy=0x7f543415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.588+0000 7f5439f09640 1 -- 172.21.15.26:0/1107184196 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543406b950 con 0x7f5434154220 2023-12-19T16:42:21.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.588+0000 7f5439f09640 1 --2- 172.21.15.26:0/1107184196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434154220 0x7f543415aa80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f54300097c0 tx=0x7f543001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea911fcb913c1611 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.588+0000 7f5438f07640 1 -- 172.21.15.26:0/1107184196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300ab610 con 0x7f5434154220 2023-12-19T16:42:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.588+0000 7f5438f07640 1 -- 172.21.15.26:0/1107184196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54300ab770 con 0x7f5434154220 2023-12-19T16:42:21.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.588+0000 7f5438f07640 1 -- 172.21.15.26:0/1107184196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300abb00 con 0x7f5434154220 2023-12-19T16:42:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 -- 172.21.15.26:0/1107184196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434154220 msgr2=0x7f543415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 --2- 172.21.15.26:0/1107184196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434154220 0x7f543415aa80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f54300097c0 tx=0x7f543001c410 comp rx=0 tx=0).stop 2023-12-19T16:42:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 -- 172.21.15.26:0/1107184196 shutdown_connections 2023-12-19T16:42:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 --2- 172.21.15.26:0/1107184196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434154220 0x7f543415aa80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 -- 172.21.15.26:0/1107184196 >> 172.21.15.26:0/1107184196 conn(0x7f54340658e0 msgr2=0x7f5434065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 -- 172.21.15.26:0/1107184196 shutdown_connections 2023-12-19T16:42:21.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 -- 172.21.15.26:0/1107184196 wait complete. 2023-12-19T16:42:21.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 Processor -- start 2023-12-19T16:42:21.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 -- start start 2023-12-19T16:42:21.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543406b3d0 0x7f54340cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:21.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54340cee10 0x7f543414bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:21.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5434154220 0x7f543414b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:21.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5434156d80 con 0x7f543406b3d0 2023-12-19T16:42:21.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5434156200 con 0x7f54340cee10 2023-12-19T16:42:21.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f543c16d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54341564e0 con 0x7f5434154220 2023-12-19T16:42:21.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f5439f09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543406b3d0 0x7f54340cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:21.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f5439f09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543406b3d0 0x7f54340cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44158/0 (socket says 172.21.15.26:44158) 2023-12-19T16:42:21.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f5439f09640 1 -- 172.21.15.26:0/175366374 learned_addr learned my addr 172.21.15.26:0/175366374 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:21.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f5439f09640 1 -- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5434154220 msgr2=0x7f543414b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f5439f09640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5434154220 0x7f543414b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f5439708640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54340cee10 0x7f543414bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:21.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f5439f09640 1 -- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54340cee10 msgr2=0x7f543414bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f5439f09640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54340cee10 0x7f543414bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f5439f09640 1 -- 172.21.15.26:0/175366374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5430009420 con 0x7f543406b3d0 2023-12-19T16:42:21.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.592+0000 7f5439708640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54340cee10 0x7f543414bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:21.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.596+0000 7f5439f09640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543406b3d0 0x7f54340cb4c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54300ad040 tx=0x7f543001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:21.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.596+0000 7f542a7fc640 1 -- 172.21.15.26:0/175366374 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300b5810 con 0x7f543406b3d0 2023-12-19T16:42:21.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.596+0000 7f542a7fc640 1 -- 172.21.15.26:0/175366374 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54300b5990 con 0x7f543406b3d0 2023-12-19T16:42:21.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.596+0000 7f542a7fc640 1 -- 172.21.15.26:0/175366374 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300b5d40 con 0x7f543406b3d0 2023-12-19T16:42:21.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.596+0000 7f543c16d640 1 -- 172.21.15.26:0/175366374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54341567c0 con 0x7f543406b3d0 2023-12-19T16:42:21.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.596+0000 7f543c16d640 1 -- 172.21.15.26:0/175366374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54340cba00 con 0x7f543406b3d0 2023-12-19T16:42:21.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.596+0000 7f543c16d640 1 -- 172.21.15.26:0/175366374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54340cbbf0 con 0x7f543406b3d0 2023-12-19T16:42:21.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.596+0000 7f542a7fc640 1 -- 172.21.15.26:0/175366374 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f54300bf410 con 0x7f543406b3d0 2023-12-19T16:42:21.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.596+0000 7f542a7fc640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5418065a60 0x7f5418067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:21.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.596+0000 7f542a7fc640 1 -- 172.21.15.26:0/175366374 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f543011de30 con 0x7f543406b3d0 2023-12-19T16:42:21.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.596+0000 7f5439708640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5418065a60 0x7f5418067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:21.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.600+0000 7f5439708640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5418065a60 0x7f5418067f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5434155d30 tx=0x7f5424009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:21.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.600+0000 7f542a7fc640 1 -- 172.21.15.26:0/175366374 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f54340cbbf0 con 0x7f543406b3d0 2023-12-19T16:42:21.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f543c16d640 1 -- 172.21.15.26:0/175366374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54340cbbf0 con 0x7f543406b3d0 2023-12-19T16:42:21.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f542a7fc640 1 -- 172.21.15.26:0/175366374 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1253 (secure 0 0 0) 0x7f54340cbbf0 con 0x7f543406b3d0 2023-12-19T16:42:21.796 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:21.796 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/837780144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":837780144},{"type":"v1","addr":"172.21.15.26:6834","nonce":837780144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-12-19T16:42:21.796 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 2 2023-12-19T16:42:21.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f543c16d640 1 -- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5418065a60 msgr2=0x7f5418067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f543c16d640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5418065a60 0x7f5418067f20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5434155d30 tx=0x7f5424009040 comp rx=0 tx=0).stop 2023-12-19T16:42:21.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f543c16d640 1 -- 172.21.15.26:0/175366374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543406b3d0 msgr2=0x7f54340cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:21.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f543c16d640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543406b3d0 0x7f54340cb4c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54300ad040 tx=0x7f543001ac50 comp rx=0 tx=0).stop 2023-12-19T16:42:21.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f543c16d640 1 -- 172.21.15.26:0/175366374 shutdown_connections 2023-12-19T16:42:21.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f543c16d640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5418065a60 0x7f5418067f20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f543c16d640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5434154220 0x7f543414b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f543c16d640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54340cee10 0x7f543414bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f543c16d640 1 --2- 172.21.15.26:0/175366374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f543406b3d0 0x7f54340cb4c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:21.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f543c16d640 1 -- 172.21.15.26:0/175366374 >> 172.21.15.26:0/175366374 conn(0x7f54340658e0 msgr2=0x7f5434065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:21.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f543c16d640 1 -- 172.21.15.26:0/175366374 shutdown_connections 2023-12-19T16:42:21.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:21.792+0000 7f543c16d640 1 -- 172.21.15.26:0/175366374 wait complete. 2023-12-19T16:42:23.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:42:23.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.968+0000 7fc4a936d640 1 Processor -- start 2023-12-19T16:42:23.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.968+0000 7fc4a936d640 1 -- start start 2023-12-19T16:42:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.968+0000 7fc4a936d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc4a40d1290 0x7fc4a40b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.968+0000 7fc4a936d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4a406afd0 0x7fc4a40b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.968+0000 7fc4a936d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a406b790 con 0x7fc4a40cd330 2023-12-19T16:42:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.968+0000 7fc4a936d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a406abb0 con 0x7fc4a40d1290 2023-12-19T16:42:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.968+0000 7fc4a936d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a406bce0 con 0x7fc4a406afd0 2023-12-19T16:42:23.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.968+0000 7fc4a2ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc4a40d1290 0x7fc4a40b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:23.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.968+0000 7fc49bfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4a406afd0 0x7fc4a40b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:23.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.968+0000 7fc4a37fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fc4a40cd330 0x7fc4a40cfe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:52508/0 (socket says 172.21.15.26:52508) 2023-12-19T16:42:23.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.968+0000 7fc4a37fe640 1 -- 172.21.15.26:0/848201388 learned_addr learned my addr 172.21.15.26:0/848201388 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.972+0000 7fc49bfff640 1 -- 172.21.15.26:0/848201388 >> v2:172.21.15.160:3300/0 conn(0x7fc4a40d1290 msgr2=0x7fc4a40b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.972+0000 7fc49bfff640 1 --2- 172.21.15.26:0/848201388 >> v2:172.21.15.160:3300/0 conn(0x7fc4a40d1290 0x7fc4a40b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.972+0000 7fc49bfff640 1 -- 172.21.15.26:0/848201388 >> v1:172.21.15.160:6789/0 conn(0x7fc4a40cd330 legacy=0x7fc4a40cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.972+0000 7fc49bfff640 1 -- 172.21.15.26:0/848201388 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a406ba00 con 0x7fc4a406afd0 2023-12-19T16:42:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.972+0000 7fc4a2ffd640 1 --2- 172.21.15.26:0/848201388 >> v2:172.21.15.160:3300/0 conn(0x7fc4a40d1290 0x7fc4a40b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.972+0000 7fc49bfff640 1 --2- 172.21.15.26:0/848201388 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4a406afd0 0x7fc4a40b2dc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0097c0 tx=0x7fc48c01c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f5fe5a02475623 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.972+0000 7fc4a27fc640 1 -- 172.21.15.26:0/848201388 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c0ad680 con 0x7fc4a406afd0 2023-12-19T16:42:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.972+0000 7fc4a27fc640 1 -- 172.21.15.26:0/848201388 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c0ad7e0 con 0x7fc4a406afd0 2023-12-19T16:42:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.972+0000 7fc4a27fc640 1 -- 172.21.15.26:0/848201388 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c0adb70 con 0x7fc4a406afd0 2023-12-19T16:42:23.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 -- 172.21.15.26:0/848201388 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4a406afd0 msgr2=0x7fc4a40b2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:23.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 --2- 172.21.15.26:0/848201388 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4a406afd0 0x7fc4a40b2dc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0097c0 tx=0x7fc48c01c380 comp rx=0 tx=0).stop 2023-12-19T16:42:23.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 -- 172.21.15.26:0/848201388 shutdown_connections 2023-12-19T16:42:23.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 --2- 172.21.15.26:0/848201388 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4a406afd0 0x7fc4a40b2dc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:23.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 --2- 172.21.15.26:0/848201388 >> v2:172.21.15.160:3300/0 conn(0x7fc4a40d1290 0x7fc4a40b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:23.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 -- 172.21.15.26:0/848201388 >> 172.21.15.26:0/848201388 conn(0x7fc4a406d570 msgr2=0x7fc4a406d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:23.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 -- 172.21.15.26:0/848201388 shutdown_connections 2023-12-19T16:42:23.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 -- 172.21.15.26:0/848201388 wait complete. 2023-12-19T16:42:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 Processor -- start 2023-12-19T16:42:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 -- start start 2023-12-19T16:42:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a406afd0 0x7fc4a415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a40cd330 0x7fc4a415fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4a40d1290 0x7fc4a4159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a415dbc0 con 0x7fc4a406afd0 2023-12-19T16:42:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a415dea0 con 0x7fc4a40cd330 2023-12-19T16:42:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a936d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a415e180 con 0x7fc4a40d1290 2023-12-19T16:42:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a406afd0 0x7fc4a415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a406afd0 0x7fc4a415b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44160/0 (socket says 172.21.15.26:44160) 2023-12-19T16:42:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a2ffd640 1 -- 172.21.15.26:0/1095133276 learned_addr learned my addr 172.21.15.26:0/1095133276 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc49bfff640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a40cd330 0x7fc4a415fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a37fe640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4a40d1290 0x7fc4a4159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a2ffd640 1 -- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4a40d1290 msgr2=0x7fc4a4159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a2ffd640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4a40d1290 0x7fc4a4159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a2ffd640 1 -- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a40cd330 msgr2=0x7fc4a415fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a2ffd640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a40cd330 0x7fc4a415fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a2ffd640 1 -- 172.21.15.26:0/1095133276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc494009420 con 0x7fc4a406afd0 2023-12-19T16:42:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.976+0000 7fc4a2ffd640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a406afd0 0x7fc4a415b970 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc49400e980 tx=0x7fc49400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.980+0000 7fc49b7fe640 1 -- 172.21.15.26:0/1095133276 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49400ca00 con 0x7fc4a406afd0 2023-12-19T16:42:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.980+0000 7fc49b7fe640 1 -- 172.21.15.26:0/1095133276 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49400cb60 con 0x7fc4a406afd0 2023-12-19T16:42:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.980+0000 7fc4a936d640 1 -- 172.21.15.26:0/1095133276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48c009420 con 0x7fc4a406afd0 2023-12-19T16:42:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.980+0000 7fc49b7fe640 1 -- 172.21.15.26:0/1095133276 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494013470 con 0x7fc4a406afd0 2023-12-19T16:42:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.980+0000 7fc4a936d640 1 -- 172.21.15.26:0/1095133276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a40cdcd0 con 0x7fc4a406afd0 2023-12-19T16:42:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.980+0000 7fc4a936d640 1 -- 172.21.15.26:0/1095133276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a4074270 con 0x7fc4a406afd0 2023-12-19T16:42:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.980+0000 7fc49b7fe640 1 -- 172.21.15.26:0/1095133276 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc4940135d0 con 0x7fc4a406afd0 2023-12-19T16:42:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.980+0000 7fc49b7fe640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc484065a60 0x7fc484067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.980+0000 7fc49b7fe640 1 -- 172.21.15.26:0/1095133276 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc494082ff0 con 0x7fc4a406afd0 2023-12-19T16:42:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.980+0000 7fc49bfff640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc484065a60 0x7fc484067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.984+0000 7fc49bfff640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc484065a60 0x7fc484067f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0af000 tx=0x7fc48c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:23.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:23.984+0000 7fc49b7fe640 1 -- 172.21.15.26:0/1095133276 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc4940b40a0 con 0x7fc4a406afd0 2023-12-19T16:42:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.176+0000 7fc4a936d640 1 -- 172.21.15.26:0/1095133276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4a40ce4c0 con 0x7fc4a406afd0 2023-12-19T16:42:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.180+0000 7fc49b7fe640 1 -- 172.21.15.26:0/1095133276 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7fc494017de0 con 0x7fc4a406afd0 2023-12-19T16:42:24.183 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:24.183 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2850479611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2850479611},{"type":"v1","addr":"172.21.15.160:6836","nonce":2850479611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/837780144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":837780144},{"type":"v1","addr":"172.21.15.26:6834","nonce":837780144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2784408519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2784408519},{"type":"v1","addr":"172.21.15.26:6835","nonce":2784408519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3427345273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3427345273},{"type":"v1","addr":"172.21.15.160:6837","nonce":3427345273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-12-19T16:42:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 3 2023-12-19T16:42:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.180+0000 7fc4a936d640 1 -- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc484065a60 msgr2=0x7fc484067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.180+0000 7fc4a936d640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc484065a60 0x7fc484067f20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0af000 tx=0x7fc48c0a7000 comp rx=0 tx=0).stop 2023-12-19T16:42:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.180+0000 7fc4a936d640 1 -- 172.21.15.26:0/1095133276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a406afd0 msgr2=0x7fc4a415b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.180+0000 7fc4a936d640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a406afd0 0x7fc4a415b970 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc49400e980 tx=0x7fc49400ee40 comp rx=0 tx=0).stop 2023-12-19T16:42:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.180+0000 7fc4a936d640 1 -- 172.21.15.26:0/1095133276 shutdown_connections 2023-12-19T16:42:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.180+0000 7fc4a936d640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc484065a60 0x7fc484067f20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.180+0000 7fc4a936d640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4a40d1290 0x7fc4a4159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.180+0000 7fc4a936d640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4a40cd330 0x7fc4a415fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.180+0000 7fc4a936d640 1 --2- 172.21.15.26:0/1095133276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4a406afd0 0x7fc4a415b970 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.180+0000 7fc4a936d640 1 -- 172.21.15.26:0/1095133276 >> 172.21.15.26:0/1095133276 conn(0x7fc4a406d570 msgr2=0x7fc4a40ad060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:24.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.180+0000 7fc4a936d640 1 -- 172.21.15.26:0/1095133276 shutdown_connections 2023-12-19T16:42:24.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.180+0000 7fc4a936d640 1 -- 172.21.15.26:0/1095133276 wait complete. 2023-12-19T16:42:24.204 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:24.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd63344640 1 Processor -- start 2023-12-19T16:42:24.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd63344640 1 -- start start 2023-12-19T16:42:24.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd63344640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efd5c15cca0 0x7efd5c0b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:24.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd63344640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd5c0b6b20 0x7efd5c0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:24.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd63344640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd5c0b6fb0 con 0x7efd5c0b8760 2023-12-19T16:42:24.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd63344640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5c0b7c30 con 0x7efd5c0b6b20 2023-12-19T16:42:24.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd63344640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5c0b7290 con 0x7efd5c15cca0 2023-12-19T16:42:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd62b43640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd5c0b6b20 0x7efd5c0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd62b43640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd5c0b6b20 0x7efd5c0d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44168/0 (socket says 172.21.15.26:44168) 2023-12-19T16:42:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd62b43640 1 -- 172.21.15.26:0/208509659 learned_addr learned my addr 172.21.15.26:0/208509659 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd62b43640 1 -- 172.21.15.26:0/208509659 >> v2:172.21.15.160:3300/0 conn(0x7efd5c15cca0 msgr2=0x7efd5c0b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd62b43640 1 --2- 172.21.15.26:0/208509659 >> v2:172.21.15.160:3300/0 conn(0x7efd5c15cca0 0x7efd5c0b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd62b43640 1 -- 172.21.15.26:0/208509659 >> v1:172.21.15.160:6789/0 conn(0x7efd5c0b8760 legacy=0x7efd5c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.348+0000 7efd62b43640 1 -- 172.21.15.26:0/208509659 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5c0b7570 con 0x7efd5c0b6b20 2023-12-19T16:42:24.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.352+0000 7efd62b43640 1 --2- 172.21.15.26:0/208509659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5c0b6b20 0x7efd5c0d09e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0b5500 tx=0x7efd5401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da76f35a050e2cf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:24.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.352+0000 7efd61340640 1 -- 172.21.15.26:0/208509659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd540a5840 con 0x7efd5c0b6b20 2023-12-19T16:42:24.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.352+0000 7efd61340640 1 -- 172.21.15.26:0/208509659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd540a59a0 con 0x7efd5c0b6b20 2023-12-19T16:42:24.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.352+0000 7efd61340640 1 -- 172.21.15.26:0/208509659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd540a5d30 con 0x7efd5c0b6b20 2023-12-19T16:42:24.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.352+0000 7efd63344640 1 -- 172.21.15.26:0/208509659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5c0b6b20 msgr2=0x7efd5c0d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:24.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.352+0000 7efd63344640 1 --2- 172.21.15.26:0/208509659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5c0b6b20 0x7efd5c0d09e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0b5500 tx=0x7efd5401c960 comp rx=0 tx=0).stop 2023-12-19T16:42:24.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.352+0000 7efd63344640 1 -- 172.21.15.26:0/208509659 shutdown_connections 2023-12-19T16:42:24.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.352+0000 7efd63344640 1 --2- 172.21.15.26:0/208509659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5c0b6b20 0x7efd5c0d09e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.352+0000 7efd63344640 1 --2- 172.21.15.26:0/208509659 >> v2:172.21.15.160:3300/0 conn(0x7efd5c15cca0 0x7efd5c0b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.352+0000 7efd63344640 1 -- 172.21.15.26:0/208509659 >> 172.21.15.26:0/208509659 conn(0x7efd5c06a120 msgr2=0x7efd5c06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:24.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.352+0000 7efd63344640 1 -- 172.21.15.26:0/208509659 shutdown_connections 2023-12-19T16:42:24.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.352+0000 7efd63344640 1 -- 172.21.15.26:0/208509659 wait complete. 2023-12-19T16:42:24.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.352+0000 7efd63344640 1 Processor -- start 2023-12-19T16:42:24.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd63344640 1 -- start start 2023-12-19T16:42:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd63344640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5c0b8760 0x7efd5c14e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd63344640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd5c15cca0 0x7efd5c0b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd63344640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5c0b3080 0x7efd5c15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd63344640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c15b810 con 0x7efd5c0b8760 2023-12-19T16:42:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd63344640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c0b3520 con 0x7efd5c15cca0 2023-12-19T16:42:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd63344640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5c15b1c0 con 0x7efd5c0b3080 2023-12-19T16:42:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd62342640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5c0b8760 0x7efd5c14e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd62342640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5c0b8760 0x7efd5c14e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44180/0 (socket says 172.21.15.26:44180) 2023-12-19T16:42:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd62342640 1 -- 172.21.15.26:0/1102557378 learned_addr learned my addr 172.21.15.26:0/1102557378 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd61b41640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd5c15cca0 0x7efd5c0b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd62b43640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5c0b3080 0x7efd5c15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd62342640 1 -- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5c0b3080 msgr2=0x7efd5c15bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd62342640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5c0b3080 0x7efd5c15bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd62342640 1 -- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd5c15cca0 msgr2=0x7efd5c0b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd62342640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd5c15cca0 0x7efd5c0b2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd62342640 1 -- 172.21.15.26:0/1102557378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd54009b30 con 0x7efd5c0b8760 2023-12-19T16:42:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd62b43640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5c0b3080 0x7efd5c15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd62342640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5c0b8760 0x7efd5c14e230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd5800b410 tx=0x7efd5800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd52ffd640 1 -- 172.21.15.26:0/1102557378 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58011830 con 0x7efd5c0b8760 2023-12-19T16:42:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd52ffd640 1 -- 172.21.15.26:0/1102557378 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd58011990 con 0x7efd5c0b8760 2023-12-19T16:42:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd63344640 1 -- 172.21.15.26:0/1102557378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5c15b4a0 con 0x7efd5c0b8760 2023-12-19T16:42:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd52ffd640 1 -- 172.21.15.26:0/1102557378 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd58011d40 con 0x7efd5c0b8760 2023-12-19T16:42:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd63344640 1 -- 172.21.15.26:0/1102557378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5c14f7f0 con 0x7efd5c0b8760 2023-12-19T16:42:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.356+0000 7efd63344640 1 -- 172.21.15.26:0/1102557378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5c15afd0 con 0x7efd5c0b8760 2023-12-19T16:42:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.360+0000 7efd52ffd640 1 -- 172.21.15.26:0/1102557378 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efd5801a400 con 0x7efd5c0b8760 2023-12-19T16:42:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.360+0000 7efd52ffd640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd40065a60 0x7efd40067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.360+0000 7efd52ffd640 1 -- 172.21.15.26:0/1102557378 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efd58083070 con 0x7efd5c0b8760 2023-12-19T16:42:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.360+0000 7efd61b41640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd40065a60 0x7efd40067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:24.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.360+0000 7efd61b41640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd40065a60 0x7efd40067f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efd5c1541d0 tx=0x7efd4c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:24.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.364+0000 7efd52ffd640 1 -- 172.21.15.26:0/1102557378 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efd580b40a0 con 0x7efd5c0b8760 2023-12-19T16:42:24.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.520+0000 7efd63344640 1 -- 172.21.15.26:0/1102557378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd5c067400 con 0x7efd5c0b8760 2023-12-19T16:42:24.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.520+0000 7efd52ffd640 1 -- 172.21.15.26:0/1102557378 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12382 (secure 0 0 0) 0x7efd58050ce0 con 0x7efd5c0b8760 2023-12-19T16:42:24.525 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:24.526 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":19,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:18.207464+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.524+0000 7efd63344640 1 -- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd40065a60 msgr2=0x7efd40067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.524+0000 7efd63344640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd40065a60 0x7efd40067f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efd5c1541d0 tx=0x7efd4c009040 comp rx=0 tx=0).stop 2023-12-19T16:42:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.524+0000 7efd63344640 1 -- 172.21.15.26:0/1102557378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5c0b8760 msgr2=0x7efd5c14e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.524+0000 7efd63344640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5c0b8760 0x7efd5c14e230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd5800b410 tx=0x7efd5800b8d0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.524+0000 7efd63344640 1 -- 172.21.15.26:0/1102557378 shutdown_connections 2023-12-19T16:42:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.524+0000 7efd63344640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd40065a60 0x7efd40067f20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.524+0000 7efd63344640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5c0b3080 0x7efd5c15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.524+0000 7efd63344640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd5c15cca0 0x7efd5c0b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.524+0000 7efd63344640 1 --2- 172.21.15.26:0/1102557378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd5c0b8760 0x7efd5c14e230 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.524+0000 7efd63344640 1 -- 172.21.15.26:0/1102557378 >> 172.21.15.26:0/1102557378 conn(0x7efd5c06a120 msgr2=0x7efd5c158c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.524+0000 7efd63344640 1 -- 172.21.15.26:0/1102557378 shutdown_connections 2023-12-19T16:42:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.524+0000 7efd63344640 1 -- 172.21.15.26:0/1102557378 wait complete. 2023-12-19T16:42:24.547 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:42:24.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:42:24.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.688+0000 7f1dbb49a640 1 Processor -- start 2023-12-19T16:42:24.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.688+0000 7f1dbb49a640 1 -- start start 2023-12-19T16:42:24.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.688+0000 7f1dbb49a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1db415a770 0x7f1db40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:24.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.688+0000 7f1dbb49a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1db40cfae0 0x7f1db4160a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:24.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.688+0000 7f1dbb49a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db40cff30 con 0x7f1db4160f80 2023-12-19T16:42:24.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.688+0000 7f1dbb49a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db40cd490 con 0x7f1db415a770 2023-12-19T16:42:24.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.688+0000 7f1dbb49a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db40cd770 con 0x7f1db40cfae0 2023-12-19T16:42:24.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.692+0000 7f1dba498640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1db415a770 0x7f1db40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:24.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.692+0000 7f1dba498640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1db415a770 0x7f1db40cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44190/0 (socket says 172.21.15.26:44190) 2023-12-19T16:42:24.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.692+0000 7f1dba498640 1 -- 172.21.15.26:0/2323659149 learned_addr learned my addr 172.21.15.26:0/2323659149 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:24.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.692+0000 7f1db9c97640 1 --2- 172.21.15.26:0/2323659149 >> v2:172.21.15.160:3300/0 conn(0x7f1db40cfae0 0x7f1db4160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:24.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.692+0000 7f1dba498640 1 -- 172.21.15.26:0/2323659149 >> v2:172.21.15.160:3300/0 conn(0x7f1db40cfae0 msgr2=0x7f1db4160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:24.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.692+0000 7f1dba498640 1 --2- 172.21.15.26:0/2323659149 >> v2:172.21.15.160:3300/0 conn(0x7f1db40cfae0 0x7f1db4160a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.692+0000 7f1dba498640 1 -- 172.21.15.26:0/2323659149 >> v1:172.21.15.160:6789/0 conn(0x7f1db4160f80 legacy=0x7f1db40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:24.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.692+0000 7f1dba498640 1 -- 172.21.15.26:0/2323659149 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db40cda50 con 0x7f1db415a770 2023-12-19T16:42:24.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.692+0000 7f1dba498640 1 --2- 172.21.15.26:0/2323659149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db415a770 0x7f1db40cf5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1db40b8510 tx=0x7f1db001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ef3e3511ffdd00 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:24.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.692+0000 7f1db9496640 1 -- 172.21.15.26:0/2323659149 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db00a8050 con 0x7f1db415a770 2023-12-19T16:42:24.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.692+0000 7f1db9496640 1 -- 172.21.15.26:0/2323659149 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db00ac5f0 con 0x7f1db415a770 2023-12-19T16:42:24.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.692+0000 7f1db9496640 1 -- 172.21.15.26:0/2323659149 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db00ac980 con 0x7f1db415a770 2023-12-19T16:42:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 -- 172.21.15.26:0/2323659149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db415a770 msgr2=0x7f1db40cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 --2- 172.21.15.26:0/2323659149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db415a770 0x7f1db40cf5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1db40b8510 tx=0x7f1db001c4d0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 -- 172.21.15.26:0/2323659149 shutdown_connections 2023-12-19T16:42:24.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 --2- 172.21.15.26:0/2323659149 >> v2:172.21.15.160:3300/0 conn(0x7f1db40cfae0 0x7f1db4160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 --2- 172.21.15.26:0/2323659149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db415a770 0x7f1db40cf5a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 -- 172.21.15.26:0/2323659149 >> 172.21.15.26:0/2323659149 conn(0x7f1db406a150 msgr2=0x7f1db406a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 -- 172.21.15.26:0/2323659149 shutdown_connections 2023-12-19T16:42:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 -- 172.21.15.26:0/2323659149 wait complete. 2023-12-19T16:42:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 Processor -- start 2023-12-19T16:42:24.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 -- start start 2023-12-19T16:42:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40cfae0 0x7f1db4163dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1db415a770 0x7f1db4166640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1db4160f80 0x7f1db4157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db4166e00 con 0x7f1db40cfae0 2023-12-19T16:42:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db4166b80 con 0x7f1db415a770 2023-12-19T16:42:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbb49a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db4162ec0 con 0x7f1db4160f80 2023-12-19T16:42:24.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dba498640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40cfae0 0x7f1db4163dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dba498640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40cfae0 0x7f1db4163dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44194/0 (socket says 172.21.15.26:44194) 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dba498640 1 -- 172.21.15.26:0/3521708220 learned_addr learned my addr 172.21.15.26:0/3521708220 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1db9c97640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1db415a770 0x7f1db4166640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dbac99640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1db4160f80 0x7f1db4157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dba498640 1 -- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1db4160f80 msgr2=0x7f1db4157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dba498640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1db4160f80 0x7f1db4157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dba498640 1 -- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1db415a770 msgr2=0x7f1db4166640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dba498640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1db415a770 0x7f1db4166640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.696+0000 7f1dba498640 1 -- 172.21.15.26:0/3521708220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db00094d0 con 0x7f1db40cfae0 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.700+0000 7f1dba498640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40cfae0 0x7f1db4163dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1db001ab80 tx=0x7f1db0019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.700+0000 7f1daaffd640 1 -- 172.21.15.26:0/3521708220 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db00a8040 con 0x7f1db40cfae0 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.700+0000 7f1daaffd640 1 -- 172.21.15.26:0/3521708220 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db00b45c0 con 0x7f1db40cfae0 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.700+0000 7f1daaffd640 1 -- 172.21.15.26:0/3521708220 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db00b4950 con 0x7f1db40cfae0 2023-12-19T16:42:24.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.700+0000 7f1dbb49a640 1 -- 172.21.15.26:0/3521708220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db41631a0 con 0x7f1db40cfae0 2023-12-19T16:42:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.700+0000 7f1dbb49a640 1 -- 172.21.15.26:0/3521708220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db40cb6d0 con 0x7f1db40cfae0 2023-12-19T16:42:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.700+0000 7f1dbb49a640 1 -- 172.21.15.26:0/3521708220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db40cb8c0 con 0x7f1db40cfae0 2023-12-19T16:42:24.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.700+0000 7f1daaffd640 1 -- 172.21.15.26:0/3521708220 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1db00b4ab0 con 0x7f1db40cfae0 2023-12-19T16:42:24.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.700+0000 7f1daaffd640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1d98065a90 0x7f1d98067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:24.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.700+0000 7f1daaffd640 1 -- 172.21.15.26:0/3521708220 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1db011d260 con 0x7f1db40cfae0 2023-12-19T16:42:24.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.704+0000 7f1db9c97640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1d98065a90 0x7f1d98067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:24.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.704+0000 7f1db9c97640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1d98065a90 0x7f1d98067f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1db40b4cf0 tx=0x7f1da4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:24.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.704+0000 7f1daaffd640 1 -- 172.21.15.26:0/3521708220 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1db00eaed0 con 0x7f1db40cfae0 2023-12-19T16:42:24.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.860+0000 7f1dbb49a640 1 -- 172.21.15.26:0/3521708220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1db4061930 con 0x7f1db40cfae0 2023-12-19T16:42:24.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.864+0000 7f1daaffd640 1 -- 172.21.15.26:0/3521708220 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1db011be60 con 0x7f1db40cfae0 2023-12-19T16:42:24.867 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:42:24.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.864+0000 7f1dbb49a640 1 -- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1d98065a90 msgr2=0x7f1d98067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:24.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.864+0000 7f1dbb49a640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1d98065a90 0x7f1d98067f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1db40b4cf0 tx=0x7f1da4008040 comp rx=0 tx=0).stop 2023-12-19T16:42:24.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.864+0000 7f1dbb49a640 1 -- 172.21.15.26:0/3521708220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40cfae0 msgr2=0x7f1db4163dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:24.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.864+0000 7f1dbb49a640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40cfae0 0x7f1db4163dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1db001ab80 tx=0x7f1db0019270 comp rx=0 tx=0).stop 2023-12-19T16:42:24.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.864+0000 7f1dbb49a640 1 -- 172.21.15.26:0/3521708220 shutdown_connections 2023-12-19T16:42:24.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.864+0000 7f1dbb49a640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1d98065a90 0x7f1d98067f50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.864+0000 7f1dbb49a640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1db4160f80 0x7f1db4157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.864+0000 7f1dbb49a640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1db415a770 0x7f1db4166640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.864+0000 7f1dbb49a640 1 --2- 172.21.15.26:0/3521708220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1db40cfae0 0x7f1db4163dc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:24.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.864+0000 7f1dbb49a640 1 -- 172.21.15.26:0/3521708220 >> 172.21.15.26:0/3521708220 conn(0x7f1db406a150 msgr2=0x7f1db415fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:24.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.864+0000 7f1dbb49a640 1 -- 172.21.15.26:0/3521708220 shutdown_connections 2023-12-19T16:42:24.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:24.864+0000 7f1dbb49a640 1 -- 172.21.15.26:0/3521708220 wait complete. 2023-12-19T16:42:24.888 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T16:42:25.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8ab1095640 1 Processor -- start 2023-12-19T16:42:25.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8ab1095640 1 -- start start 2023-12-19T16:42:25.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8ab1095640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8aac0d3300 0x7f8aac0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8ab1095640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aac0d29d0 con 0x7f8aac0ce040 2023-12-19T16:42:25.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8ab1095640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aac15d680 con 0x7f8aac15cd40 2023-12-19T16:42:25.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8ab1095640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac0ce4b0 con 0x7f8aac0d3300 2023-12-19T16:42:25.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8aa3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8aac0ce040 0x7f8aac0d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38830/0 (socket says 172.21.15.26:38830) 2023-12-19T16:42:25.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8aa3fff640 1 -- 172.21.15.26:0/1120220954 learned_addr learned my addr 172.21.15.26:0/1120220954 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:25.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8aab59e640 1 --2- 172.21.15.26:0/1120220954 >> v2:172.21.15.160:3300/0 conn(0x7f8aac0d3300 0x7f8aac0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8aaa59c640 1 -- 172.21.15.26:0/1120220954 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3293611599 0 0) 0x7f8a94001120 con 0x7f8aac0ce040 2023-12-19T16:42:25.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8aaa59c640 1 -- 172.21.15.26:0/1120220954 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a98003530 con 0x7f8aac0ce040 2023-12-19T16:42:25.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8aaa59c640 1 -- 172.21.15.26:0/1120220954 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440722283 0 0) 0x7f8aac15d680 con 0x7f8aac15cd40 2023-12-19T16:42:25.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8aaa59c640 1 -- 172.21.15.26:0/1120220954 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a94001120 con 0x7f8aac15cd40 2023-12-19T16:42:25.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8aab59e640 1 -- 172.21.15.26:0/1120220954 >> v1:172.21.15.160:6789/0 conn(0x7f8aac15cd40 legacy=0x7f8aac160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:25.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8aab59e640 1 -- 172.21.15.26:0/1120220954 >> v1:172.21.15.26:6789/0 conn(0x7f8aac0ce040 legacy=0x7f8aac0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:25.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8aaa59c640 1 -- 172.21.15.26:0/1120220954 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 753995657 0 0) 0x7f8a98003530 con 0x7f8aac0ce040 2023-12-19T16:42:25.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.028+0000 7f8aab59e640 1 -- 172.21.15.26:0/1120220954 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac161750 con 0x7f8aac0d3300 2023-12-19T16:42:25.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.032+0000 7f8aab59e640 1 --2- 172.21.15.26:0/1120220954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8aac0d3300 0x7f8aac0d0e30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009d90 tx=0x7f8aa401c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=911a6ef978702101 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:25.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.032+0000 7f8aaa59c640 1 -- 172.21.15.26:0/1120220954 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94001810 con 0x7f8aac0d3300 2023-12-19T16:42:25.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.032+0000 7f8aaa59c640 1 -- 172.21.15.26:0/1120220954 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa40a5680 con 0x7f8aac0d3300 2023-12-19T16:42:25.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.032+0000 7f8aaa59c640 1 -- 172.21.15.26:0/1120220954 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40a59f0 con 0x7f8aac0d3300 2023-12-19T16:42:25.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.032+0000 7f8ab1095640 1 -- 172.21.15.26:0/1120220954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8aac0d3300 msgr2=0x7f8aac0d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:25.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.032+0000 7f8ab1095640 1 --2- 172.21.15.26:0/1120220954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8aac0d3300 0x7f8aac0d0e30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009d90 tx=0x7f8aa401c840 comp rx=0 tx=0).stop 2023-12-19T16:42:25.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.032+0000 7f8ab1095640 1 -- 172.21.15.26:0/1120220954 shutdown_connections 2023-12-19T16:42:25.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.032+0000 7f8ab1095640 1 --2- 172.21.15.26:0/1120220954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8aac0d3300 0x7f8aac0d0e30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:25.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.032+0000 7f8ab1095640 1 -- 172.21.15.26:0/1120220954 >> 172.21.15.26:0/1120220954 conn(0x7f8aac06a120 msgr2=0x7f8aac06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:25.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.032+0000 7f8ab1095640 1 -- 172.21.15.26:0/1120220954 shutdown_connections 2023-12-19T16:42:25.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.032+0000 7f8ab1095640 1 -- 172.21.15.26:0/1120220954 wait complete. 2023-12-19T16:42:25.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8ab1095640 1 Processor -- start 2023-12-19T16:42:25.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8ab1095640 1 -- start start 2023-12-19T16:42:25.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8ab1095640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8aac0ce040 0x7f8aac14b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8ab1095640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac0d3300 0x7f8aac15ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8ab1095640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8aac15cd40 0x7f8aac1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8ab1095640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac14be20 con 0x7f8aac0d3300 2023-12-19T16:42:25.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8ab1095640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac15a390 con 0x7f8aac0ce040 2023-12-19T16:42:25.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8ab1095640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac154da0 con 0x7f8aac15cd40 2023-12-19T16:42:25.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aa3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac0d3300 0x7f8aac15ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aa3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac0d3300 0x7f8aac15ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44204/0 (socket says 172.21.15.26:44204) 2023-12-19T16:42:25.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aa3fff640 1 -- 172.21.15.26:0/736942444 learned_addr learned my addr 172.21.15.26:0/736942444 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:25.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aaad9d640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8aac0ce040 0x7f8aac14b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aab59e640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8aac15cd40 0x7f8aac1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aa3fff640 1 -- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8aac15cd40 msgr2=0x7f8aac1561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:25.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aa3fff640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8aac15cd40 0x7f8aac1561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:25.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aa3fff640 1 -- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8aac0ce040 msgr2=0x7f8aac14b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:25.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aa3fff640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8aac0ce040 0x7f8aac14b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:25.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aa3fff640 1 -- 172.21.15.26:0/736942444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa4009a70 con 0x7f8aac0d3300 2023-12-19T16:42:25.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aa3fff640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac0d3300 0x7f8aac15ade0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400c750 tx=0x7f8a9400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:25.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aa37fe640 1 -- 172.21.15.26:0/736942444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94005cf0 con 0x7f8aac0d3300 2023-12-19T16:42:25.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aa37fe640 1 -- 172.21.15.26:0/736942444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a94005e50 con 0x7f8aac0d3300 2023-12-19T16:42:25.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8aa37fe640 1 -- 172.21.15.26:0/736942444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94007270 con 0x7f8aac0d3300 2023-12-19T16:42:25.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8ab1095640 1 -- 172.21.15.26:0/736942444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac155080 con 0x7f8aac0d3300 2023-12-19T16:42:25.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.036+0000 7f8ab1095640 1 -- 172.21.15.26:0/736942444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac15a8c0 con 0x7f8aac0d3300 2023-12-19T16:42:25.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.040+0000 7f8ab1095640 1 -- 172.21.15.26:0/736942444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac15aab0 con 0x7f8aac0d3300 2023-12-19T16:42:25.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.040+0000 7f8aa37fe640 1 -- 172.21.15.26:0/736942444 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8a940073d0 con 0x7f8aac0d3300 2023-12-19T16:42:25.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.040+0000 7f8aa37fe640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8a8c065a60 0x7f8a8c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.040+0000 7f8aa37fe640 1 -- 172.21.15.26:0/736942444 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8a940824b0 con 0x7f8aac0d3300 2023-12-19T16:42:25.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.040+0000 7f8aaad9d640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8a8c065a60 0x7f8a8c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.040+0000 7f8aaad9d640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8a8c065a60 0x7f8a8c067f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c001240 tx=0x7f8a9c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.044+0000 7f8aa37fe640 1 -- 172.21.15.26:0/736942444 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8a940501d0 con 0x7f8aac0d3300 2023-12-19T16:42:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8ab1095640 1 -- 172.21.15.26:0/736942444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8aac074270 con 0x7f8aac0d3300 2023-12-19T16:42:25.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8aa37fe640 1 -- 172.21.15.26:0/736942444 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8a94017090 con 0x7f8aac0d3300 2023-12-19T16:42:25.208 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:25.208 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T16:42:25.208 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:42:25.208 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T16:42:25.208 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T16:42:25.208 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:42:25.208 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T16:42:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8ab1095640 1 -- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8a8c065a60 msgr2=0x7f8a8c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8ab1095640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8a8c065a60 0x7f8a8c067f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c001240 tx=0x7f8a9c009040 comp rx=0 tx=0).stop 2023-12-19T16:42:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8ab1095640 1 -- 172.21.15.26:0/736942444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac0d3300 msgr2=0x7f8aac15ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8ab1095640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac0d3300 0x7f8aac15ade0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400c750 tx=0x7f8a9400cc10 comp rx=0 tx=0).stop 2023-12-19T16:42:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8ab1095640 1 -- 172.21.15.26:0/736942444 shutdown_connections 2023-12-19T16:42:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8ab1095640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8a8c065a60 0x7f8a8c067f20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8ab1095640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8aac15cd40 0x7f8aac1561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8ab1095640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aac0d3300 0x7f8aac15ade0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:25.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8ab1095640 1 --2- 172.21.15.26:0/736942444 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8aac0ce040 0x7f8aac14b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8ab1095640 1 -- 172.21.15.26:0/736942444 >> 172.21.15.26:0/736942444 conn(0x7f8aac06a120 msgr2=0x7f8aac15d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8ab1095640 1 -- 172.21.15.26:0/736942444 shutdown_connections 2023-12-19T16:42:25.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.204+0000 7f8ab1095640 1 -- 172.21.15.26:0/736942444 wait complete. 2023-12-19T16:42:25.229 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T16:42:25.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T16:42:25.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.372+0000 7fe7a759e640 1 Processor -- start 2023-12-19T16:42:25.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.372+0000 7fe7a759e640 1 -- start start 2023-12-19T16:42:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.372+0000 7fe7a759e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe7a80cfcb0 0x7fe7a8160a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.372+0000 7fe7a759e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7a8160f80 0x7fe7a80d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.372+0000 7fe7a759e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a80cd490 con 0x7fe7a815a770 2023-12-19T16:42:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.372+0000 7fe7a759e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a80cd770 con 0x7fe7a80cfcb0 2023-12-19T16:42:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.372+0000 7fe7a759e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80cda50 con 0x7fe7a8160f80 2023-12-19T16:42:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.372+0000 7fe7a659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe7a815a770 0x7fe7a80cf5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38844/0 (socket says 172.21.15.26:38844) 2023-12-19T16:42:25.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.372+0000 7fe7a659c640 1 -- 172.21.15.26:0/2190724257 learned_addr learned my addr 172.21.15.26:0/2190724257 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe79ffff640 1 --2- 172.21.15.26:0/2190724257 >> v2:172.21.15.160:3300/0 conn(0x7fe7a80cfcb0 0x7fe7a8160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a6d9d640 1 --2- 172.21.15.26:0/2190724257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7a8160f80 0x7fe7a80d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651962371 0 0) 0x7fe7a80cd490 con 0x7fe7a815a770 2023-12-19T16:42:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe794003530 con 0x7fe7a815a770 2023-12-19T16:42:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1698952168 0 0) 0x7fe794003530 con 0x7fe7a815a770 2023-12-19T16:42:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7a8160f80 msgr2=0x7fe7a80d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 --2- 172.21.15.26:0/2190724257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7a8160f80 0x7fe7a80d0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 >> v2:172.21.15.160:3300/0 conn(0x7fe7a80cfcb0 msgr2=0x7fe7a8160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 --2- 172.21.15.26:0/2190724257 >> v2:172.21.15.160:3300/0 conn(0x7fe7a80cfcb0 0x7fe7a8160a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a80cdd30 con 0x7fe7a815a770 2023-12-19T16:42:25.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fe798001680 con 0x7fe7a815a770 2023-12-19T16:42:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 >> v1:172.21.15.26:6789/0 conn(0x7fe7a815a770 legacy=0x7fe7a80cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 --2- 172.21.15.26:0/2190724257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe794003980 0x7fe794003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 --2- 172.21.15.26:0/2190724257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7a8160f80 0x7fe7940047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 --2- 172.21.15.26:0/2190724257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe794004d10 0x7fe794005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80cd770 con 0x7fe794004d10 2023-12-19T16:42:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80cda50 con 0x7fe794003980 2023-12-19T16:42:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79400b990 con 0x7fe7a8160f80 2023-12-19T16:42:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe79ffff640 1 --2- 172.21.15.26:0/2190724257 >> v2:172.21.15.160:3300/0 conn(0x7fe7a80cfcb0 0x7fe7a8160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:25.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a6d9d640 1 --2- 172.21.15.26:0/2190724257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe794004d10 0x7fe794005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a659c640 1 --2- 172.21.15.26:0/2190724257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe794003980 0x7fe794003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe79ffff640 1 --2- 172.21.15.26:0/2190724257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7a8160f80 0x7fe7940047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a6d9d640 1 -- 172.21.15.26:0/2190724257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7a8160f80 msgr2=0x7fe7940047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a6d9d640 1 --2- 172.21.15.26:0/2190724257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7a8160f80 0x7fe7940047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a6d9d640 1 -- 172.21.15.26:0/2190724257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe794003980 msgr2=0x7fe794003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a6d9d640 1 --2- 172.21.15.26:0/2190724257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe794003980 0x7fe794003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:25.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.376+0000 7fe7a6d9d640 1 -- 172.21.15.26:0/2190724257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe794003530 con 0x7fe794004d10 2023-12-19T16:42:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a6d9d640 1 --2- 172.21.15.26:0/2190724257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe794004d10 0x7fe794005190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009fc0 tx=0x7fe7a001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8f736fe78b0695 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00a5c70 con 0x7fe794004d10 2023-12-19T16:42:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a000f040 con 0x7fe794004d10 2023-12-19T16:42:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a5d9b640 1 -- 172.21.15.26:0/2190724257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00ad440 con 0x7fe794004d10 2023-12-19T16:42:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a759e640 1 -- 172.21.15.26:0/2190724257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe794004d10 msgr2=0x7fe794005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:25.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a759e640 1 --2- 172.21.15.26:0/2190724257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe794004d10 0x7fe794005190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009fc0 tx=0x7fe7a001cc60 comp rx=0 tx=0).stop 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a6d9d640 1 -- 172.21.15.26:0/2190724257 reap_dead start 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a759e640 1 -- 172.21.15.26:0/2190724257 shutdown_connections 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a759e640 1 -- 172.21.15.26:0/2190724257 >> 172.21.15.26:0/2190724257 conn(0x7fe7a806a150 msgr2=0x7fe7a806a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a759e640 1 -- 172.21.15.26:0/2190724257 shutdown_connections 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a759e640 1 -- 172.21.15.26:0/2190724257 wait complete. 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a759e640 1 Processor -- start 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a759e640 1 -- start start 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.380+0000 7fe7a759e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe794004d10 0x7fe7a80c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe7a759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7a815d3a0 0x7fe7a815d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe7a759e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7a8155200 0x7fe7a8155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe7a759e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a815dd60 con 0x7fe7a815d3a0 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe7a759e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80c8790 con 0x7fe794004d10 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe7a759e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80ca2a0 con 0x7fe7a8155200 2023-12-19T16:42:25.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe79ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7a815d3a0 0x7fe7a815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe79ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7a815d3a0 0x7fe7a815d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44234/0 (socket says 172.21.15.26:44234) 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe79ffff640 1 -- 172.21.15.26:0/1635786671 learned_addr learned my addr 172.21.15.26:0/1635786671 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe7a659c640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe794004d10 0x7fe7a80c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe7a6d9d640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7a8155200 0x7fe7a8155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe79ffff640 1 -- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7a8155200 msgr2=0x7fe7a8155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe79ffff640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7a8155200 0x7fe7a8155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe79ffff640 1 -- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe794004d10 msgr2=0x7fe7a80c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe79ffff640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe794004d10 0x7fe7a80c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe79ffff640 1 -- 172.21.15.26:0/1635786671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a80ca580 con 0x7fe7a815d3a0 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe79ffff640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7a815d3a0 0x7fe7a815d820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe790011900 tx=0x7fe790011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe79effd640 1 -- 172.21.15.26:0/1635786671 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79000fbd0 con 0x7fe7a815d3a0 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe79effd640 1 -- 172.21.15.26:0/1635786671 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79000fd30 con 0x7fe7a815d3a0 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe79effd640 1 -- 172.21.15.26:0/1635786671 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790015660 con 0x7fe7a815d3a0 2023-12-19T16:42:25.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe7a759e640 1 -- 172.21.15.26:0/1635786671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a80ca860 con 0x7fe7a815d3a0 2023-12-19T16:42:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe7a759e640 1 -- 172.21.15.26:0/1635786671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a80cae20 con 0x7fe7a815d3a0 2023-12-19T16:42:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe7a759e640 1 -- 172.21.15.26:0/1635786671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a804e850 con 0x7fe7a815d3a0 2023-12-19T16:42:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.384+0000 7fe79effd640 1 -- 172.21.15.26:0/1635786671 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe7900157c0 con 0x7fe7a815d3a0 2023-12-19T16:42:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.388+0000 7fe79effd640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe788065a60 0x7fe788067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.388+0000 7fe79effd640 1 -- 172.21.15.26:0/1635786671 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe790083800 con 0x7fe7a815d3a0 2023-12-19T16:42:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.388+0000 7fe7a659c640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe788065a60 0x7fe788067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:25.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.388+0000 7fe7a659c640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe788065a60 0x7fe788067f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe798000c10 tx=0x7fe798007900 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:25.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.392+0000 7fe79effd640 1 -- 172.21.15.26:0/1635786671 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe7900515a0 con 0x7fe7a815d3a0 2023-12-19T16:42:25.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:25.552+0000 7fe7a759e640 1 -- 172.21.15.26:0/1635786671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe7a80d2230 con 0x7fe7a815d3a0 2023-12-19T16:42:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.284+0000 7fe79effd640 1 -- 172.21.15.26:0/1635786671 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe79000fea0 con 0x7fe7a815d3a0 2023-12-19T16:42:26.289 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T16:42:26.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.288+0000 7fe7a759e640 1 -- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe788065a60 msgr2=0x7fe788067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:26.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.288+0000 7fe7a759e640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe788065a60 0x7fe788067f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe798000c10 tx=0x7fe798007900 comp rx=0 tx=0).stop 2023-12-19T16:42:26.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.288+0000 7fe7a759e640 1 -- 172.21.15.26:0/1635786671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7a815d3a0 msgr2=0x7fe7a815d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:26.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.288+0000 7fe7a759e640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7a815d3a0 0x7fe7a815d820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe790011900 tx=0x7fe790011dc0 comp rx=0 tx=0).stop 2023-12-19T16:42:26.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.288+0000 7fe7a759e640 1 -- 172.21.15.26:0/1635786671 shutdown_connections 2023-12-19T16:42:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.288+0000 7fe7a759e640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe788065a60 0x7fe788067f20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.288+0000 7fe7a759e640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7a8155200 0x7fe7a8155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.288+0000 7fe7a759e640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7a815d3a0 0x7fe7a815d820 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.288+0000 7fe7a759e640 1 --2- 172.21.15.26:0/1635786671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe794004d10 0x7fe7a80c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.288+0000 7fe7a759e640 1 -- 172.21.15.26:0/1635786671 >> 172.21.15.26:0/1635786671 conn(0x7fe7a806a150 msgr2=0x7fe7a8156ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.288+0000 7fe7a759e640 1 -- 172.21.15.26:0/1635786671 shutdown_connections 2023-12-19T16:42:26.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.288+0000 7fe7a759e640 1 -- 172.21.15.26:0/1635786671 wait complete. 2023-12-19T16:42:26.322 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T16:42:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f810e339640 1 Processor -- start 2023-12-19T16:42:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f810e339640 1 -- start start 2023-12-19T16:42:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f810e339640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8108070db0 0x7f81080ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f810e339640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f810806b500 con 0x7f810815ada0 2023-12-19T16:42:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f810e339640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8108070b60 con 0x7f81080b0c30 2023-12-19T16:42:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f810e339640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81080715b0 con 0x7f8108070db0 2023-12-19T16:42:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8107fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f810815ada0 0x7f81080b0520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50180/0 (socket says 172.21.15.26:50180) 2023-12-19T16:42:26.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8107fff640 1 -- 172.21.15.26:0/1388519103 learned_addr learned my addr 172.21.15.26:0/1388519103 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605857051 0 0) 0x7f810806b500 con 0x7f810815ada0 2023-12-19T16:42:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80f0003530 con 0x7f810815ada0 2023-12-19T16:42:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f810c8d6640 1 --2- 172.21.15.26:0/1388519103 >> v2:172.21.15.160:3300/0 conn(0x7f8108070db0 0x7f81080ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:26.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1378241638 0 0) 0x7f80f0003530 con 0x7f810815ada0 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 >> v2:172.21.15.160:3300/0 conn(0x7f8108070db0 msgr2=0x7f81080ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 --2- 172.21.15.26:0/1388519103 >> v2:172.21.15.160:3300/0 conn(0x7f8108070db0 0x7f81080ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 >> v1:172.21.15.160:6789/0 conn(0x7f81080b0c30 legacy=0x7f810806b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81080702c0 con 0x7f810815ada0 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f80f8001680 con 0x7f810815ada0 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 >> v1:172.21.15.26:6789/0 conn(0x7f810815ada0 legacy=0x7f81080b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 --2- 172.21.15.26:0/1388519103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f00039b0 0x7f80f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 --2- 172.21.15.26:0/1388519103 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80f00047d0 0x7f80f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 --2- 172.21.15.26:0/1388519103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f00051e0 0x7f80f000fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81080715b0 con 0x7f80f00039b0 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f0010520 con 0x7f80f00051e0 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.472+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00106e0 con 0x7f80f00047d0 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.476+0000 7f8107fff640 1 --2- 172.21.15.26:0/1388519103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f00039b0 0x7f80f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:26.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.476+0000 7f810c8d6640 1 --2- 172.21.15.26:0/1388519103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f00051e0 0x7f80f000fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.476+0000 7f81077fe640 1 --2- 172.21.15.26:0/1388519103 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80f00047d0 0x7f80f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.476+0000 7f8107fff640 1 -- 172.21.15.26:0/1388519103 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80f00047d0 msgr2=0x7f80f0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.476+0000 7f8107fff640 1 --2- 172.21.15.26:0/1388519103 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80f00047d0 0x7f80f0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.476+0000 7f8107fff640 1 -- 172.21.15.26:0/1388519103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f00051e0 msgr2=0x7f80f000fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.476+0000 7f8107fff640 1 --2- 172.21.15.26:0/1388519103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80f00051e0 0x7f80f000fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.476+0000 7f8107fff640 1 -- 172.21.15.26:0/1388519103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f0003530 con 0x7f80f00039b0 2023-12-19T16:42:26.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.476+0000 7f810c8d6640 1 -- 172.21.15.26:0/1388519103 reap_dead start 2023-12-19T16:42:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.476+0000 7f8107fff640 1 --2- 172.21.15.26:0/1388519103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f00039b0 0x7f80f0003dd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f80f800c190 tx=0x7f80f801cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cfb27bae5f8097 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.476+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f8005ae0 con 0x7f80f00039b0 2023-12-19T16:42:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.476+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f8005c40 con 0x7f80f00039b0 2023-12-19T16:42:26.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.476+0000 7f8106ffd640 1 -- 172.21.15.26:0/1388519103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f80072b0 con 0x7f80f00039b0 2023-12-19T16:42:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 -- 172.21.15.26:0/1388519103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f00039b0 msgr2=0x7f80f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 --2- 172.21.15.26:0/1388519103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f00039b0 0x7f80f0003dd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f80f800c190 tx=0x7f80f801cb90 comp rx=0 tx=0).stop 2023-12-19T16:42:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 -- 172.21.15.26:0/1388519103 shutdown_connections 2023-12-19T16:42:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 --2- 172.21.15.26:0/1388519103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f00039b0 0x7f80f0003dd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 -- 172.21.15.26:0/1388519103 >> 172.21.15.26:0/1388519103 conn(0x7f81080660d0 msgr2=0x7f81080ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 -- 172.21.15.26:0/1388519103 shutdown_connections 2023-12-19T16:42:26.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 -- 172.21.15.26:0/1388519103 wait complete. 2023-12-19T16:42:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 Processor -- start 2023-12-19T16:42:26.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 -- start start 2023-12-19T16:42:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f00039b0 0x7f8108154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f810814f660 0x7f81081549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8108151ba0 0x7f810814fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81081520d0 con 0x7f80f00039b0 2023-12-19T16:42:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8108154f00 con 0x7f8108151ba0 2023-12-19T16:42:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810e339640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8108152550 con 0x7f810814f660 2023-12-19T16:42:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f8107fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f00039b0 0x7f8108154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f8107fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f00039b0 0x7f8108154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49010/0 (socket says 172.21.15.26:49010) 2023-12-19T16:42:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f8107fff640 1 -- 172.21.15.26:0/1493655457 learned_addr learned my addr 172.21.15.26:0/1493655457 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f81077fe640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f810814f660 0x7f81081549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:26.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.480+0000 7f810c8d6640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8108151ba0 0x7f810814fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.484+0000 7f8107fff640 1 -- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f810814f660 msgr2=0x7f81081549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.484+0000 7f8107fff640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f810814f660 0x7f81081549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.484+0000 7f8107fff640 1 -- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8108151ba0 msgr2=0x7f810814fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.484+0000 7f8107fff640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8108151ba0 0x7f810814fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.484+0000 7f8107fff640 1 -- 172.21.15.26:0/1493655457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8108152830 con 0x7f80f00039b0 2023-12-19T16:42:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.484+0000 7f8107fff640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f00039b0 0x7f8108154480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80f801c9c0 tx=0x7f80f0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.484+0000 7f8104ff9640 1 -- 172.21.15.26:0/1493655457 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f80b4070 con 0x7f80f00039b0 2023-12-19T16:42:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.484+0000 7f8104ff9640 1 -- 172.21.15.26:0/1493655457 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f8001690 con 0x7f80f00039b0 2023-12-19T16:42:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.484+0000 7f810e339640 1 -- 172.21.15.26:0/1493655457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8108152b10 con 0x7f80f00039b0 2023-12-19T16:42:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.484+0000 7f8104ff9640 1 -- 172.21.15.26:0/1493655457 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f80072a0 con 0x7f80f00039b0 2023-12-19T16:42:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.484+0000 7f810e339640 1 -- 172.21.15.26:0/1493655457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8108158360 con 0x7f80f00039b0 2023-12-19T16:42:26.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.484+0000 7f810e339640 1 -- 172.21.15.26:0/1493655457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8108070860 con 0x7f80f00039b0 2023-12-19T16:42:26.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.484+0000 7f8104ff9640 1 -- 172.21.15.26:0/1493655457 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f80f8007400 con 0x7f80f00039b0 2023-12-19T16:42:26.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.488+0000 7f8104ff9640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f80f4065a60 0x7f80f4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:26.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.488+0000 7f8104ff9640 1 -- 172.21.15.26:0/1493655457 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f80f811cbb0 con 0x7f80f00039b0 2023-12-19T16:42:26.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.488+0000 7f81077fe640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f80f4065a60 0x7f80f4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:26.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.488+0000 7f81077fe640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f80f4065a60 0x7f80f4067f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8108157a60 tx=0x7f80fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:26.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.492+0000 7f8104ff9640 1 -- 172.21.15.26:0/1493655457 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f80f80ea7e0 con 0x7f80f00039b0 2023-12-19T16:42:26.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:26.648+0000 7f810e339640 1 -- 172.21.15.26:0/1493655457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8108070860 con 0x7f80f00039b0 2023-12-19T16:42:27.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.288+0000 7f8104ff9640 1 -- 172.21.15.26:0/1493655457 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8108070860 con 0x7f80f00039b0 2023-12-19T16:42:27.294 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T16:42:27.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:42:27.308 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:42:27.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.296+0000 7f810e339640 1 -- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f80f4065a60 msgr2=0x7f80f4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:27.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.296+0000 7f810e339640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f80f4065a60 0x7f80f4067f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8108157a60 tx=0x7f80fc009040 comp rx=0 tx=0).stop 2023-12-19T16:42:27.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.296+0000 7f810e339640 1 -- 172.21.15.26:0/1493655457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f00039b0 msgr2=0x7f8108154480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:27.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.296+0000 7f810e339640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f00039b0 0x7f8108154480 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80f801c9c0 tx=0x7f80f0005800 comp rx=0 tx=0).stop 2023-12-19T16:42:27.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.300+0000 7f810e339640 1 -- 172.21.15.26:0/1493655457 shutdown_connections 2023-12-19T16:42:27.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.300+0000 7f810e339640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8108151ba0 0x7f810814fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.300+0000 7f810e339640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f80f4065a60 0x7f80f4067f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.300+0000 7f810e339640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f810814f660 0x7f81081549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.300+0000 7f810e339640 1 --2- 172.21.15.26:0/1493655457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80f00039b0 0x7f8108154480 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.300+0000 7f810e339640 1 -- 172.21.15.26:0/1493655457 >> 172.21.15.26:0/1493655457 conn(0x7f81080660d0 msgr2=0x7f810815ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:27.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.304+0000 7f810e339640 1 -- 172.21.15.26:0/1493655457 shutdown_connections 2023-12-19T16:42:27.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.304+0000 7f810e339640 1 -- 172.21.15.26:0/1493655457 wait complete. 2023-12-19T16:42:27.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T16:42:27.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f932f1ce640 1 Processor -- start 2023-12-19T16:42:27.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f932f1ce640 1 -- start start 2023-12-19T16:42:27.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f932f1ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f932815ada0 0x7f93280706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f932f1ce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f932806ac80 0x7f932806b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f932f1ce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9328070d90 con 0x7f932815a9c0 2023-12-19T16:42:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f932f1ce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93280cc270 con 0x7f932815ada0 2023-12-19T16:42:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f932f1ce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9328071070 con 0x7f932806ac80 2023-12-19T16:42:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f9327fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f932815ada0 0x7f93280706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f9327fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f932815ada0 0x7f93280706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49018/0 (socket says 172.21.15.26:49018) 2023-12-19T16:42:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f9327fff640 1 -- 172.21.15.26:0/4270043425 learned_addr learned my addr 172.21.15.26:0/4270043425 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f932d76b640 1 --2- 172.21.15.26:0/4270043425 >> v2:172.21.15.160:3300/0 conn(0x7f932806ac80 0x7f932806b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f9327fff640 1 -- 172.21.15.26:0/4270043425 >> v2:172.21.15.160:3300/0 conn(0x7f932806ac80 msgr2=0x7f932806b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f9327fff640 1 --2- 172.21.15.26:0/4270043425 >> v2:172.21.15.160:3300/0 conn(0x7f932806ac80 0x7f932806b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f9327fff640 1 -- 172.21.15.26:0/4270043425 >> v1:172.21.15.160:6789/0 conn(0x7f932815a9c0 legacy=0x7f93280b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.568+0000 7f9327fff640 1 -- 172.21.15.26:0/4270043425 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9328071350 con 0x7f932815ada0 2023-12-19T16:42:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f9327fff640 1 --2- 172.21.15.26:0/4270043425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932815ada0 0x7f93280706d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f93280cc3b0 tx=0x7f931401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc41d3dcbaab146 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f93277fe640 1 -- 172.21.15.26:0/4270043425 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93140ab660 con 0x7f932815ada0 2023-12-19T16:42:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f93277fe640 1 -- 172.21.15.26:0/4270043425 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93140ab7c0 con 0x7f932815ada0 2023-12-19T16:42:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f93277fe640 1 -- 172.21.15.26:0/4270043425 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93140abb50 con 0x7f932815ada0 2023-12-19T16:42:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f932f1ce640 1 -- 172.21.15.26:0/4270043425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932815ada0 msgr2=0x7f93280706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f932f1ce640 1 --2- 172.21.15.26:0/4270043425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932815ada0 0x7f93280706d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f93280cc3b0 tx=0x7f931401c440 comp rx=0 tx=0).stop 2023-12-19T16:42:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f932f1ce640 1 -- 172.21.15.26:0/4270043425 shutdown_connections 2023-12-19T16:42:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f932f1ce640 1 --2- 172.21.15.26:0/4270043425 >> v2:172.21.15.160:3300/0 conn(0x7f932806ac80 0x7f932806b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f932f1ce640 1 --2- 172.21.15.26:0/4270043425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932815ada0 0x7f93280706d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f932f1ce640 1 -- 172.21.15.26:0/4270043425 >> 172.21.15.26:0/4270043425 conn(0x7f93280ce730 msgr2=0x7f93280ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f932f1ce640 1 -- 172.21.15.26:0/4270043425 shutdown_connections 2023-12-19T16:42:27.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f932f1ce640 1 -- 172.21.15.26:0/4270043425 wait complete. 2023-12-19T16:42:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f932f1ce640 1 Processor -- start 2023-12-19T16:42:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.572+0000 7f932f1ce640 1 -- start start 2023-12-19T16:42:27.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932f1ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932806ac80 0x7f9328150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:27.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932f1ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932815a9c0 0x7f932814f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:27.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932f1ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932815ada0 0x7f93280c4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:27.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932f1ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281508b0 con 0x7f932806ac80 2023-12-19T16:42:27.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932f1ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9328151210 con 0x7f932815ada0 2023-12-19T16:42:27.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932f1ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932815b520 con 0x7f932815a9c0 2023-12-19T16:42:27.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932cf6a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932806ac80 0x7f9328150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:27.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932cf6a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932806ac80 0x7f9328150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49032/0 (socket says 172.21.15.26:49032) 2023-12-19T16:42:27.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932cf6a640 1 -- 172.21.15.26:0/3580053315 learned_addr learned my addr 172.21.15.26:0/3580053315 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:27.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932d76b640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932815ada0 0x7f93280c4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:27.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f9327fff640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932815a9c0 0x7f932814f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932cf6a640 1 -- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932815a9c0 msgr2=0x7f932814f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932cf6a640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932815a9c0 0x7f932814f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932cf6a640 1 -- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932815ada0 msgr2=0x7f93280c4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932cf6a640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932815ada0 0x7f93280c4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932cf6a640 1 -- 172.21.15.26:0/3580053315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93140094d0 con 0x7f932806ac80 2023-12-19T16:42:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932cf6a640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932806ac80 0x7f9328150cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f931c00c450 tx=0x7f931c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f93257fa640 1 -- 172.21.15.26:0/3580053315 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c0052d0 con 0x7f932806ac80 2023-12-19T16:42:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f93257fa640 1 -- 172.21.15.26:0/3580053315 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c005430 con 0x7f932806ac80 2023-12-19T16:42:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932f1ce640 1 -- 172.21.15.26:0/3580053315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932814ff50 con 0x7f932806ac80 2023-12-19T16:42:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f93257fa640 1 -- 172.21.15.26:0/3580053315 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c007260 con 0x7f932806ac80 2023-12-19T16:42:27.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932f1ce640 1 -- 172.21.15.26:0/3580053315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93281504d0 con 0x7f932806ac80 2023-12-19T16:42:27.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.576+0000 7f932f1ce640 1 -- 172.21.15.26:0/3580053315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9328074270 con 0x7f932806ac80 2023-12-19T16:42:27.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.580+0000 7f93257fa640 1 -- 172.21.15.26:0/3580053315 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f931c0073c0 con 0x7f932806ac80 2023-12-19T16:42:27.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.580+0000 7f93257fa640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f930c065a60 0x7f930c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:27.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.580+0000 7f93257fa640 1 -- 172.21.15.26:0/3580053315 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f931c0836c0 con 0x7f932806ac80 2023-12-19T16:42:27.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.580+0000 7f9327fff640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f930c065a60 0x7f930c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:27.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.580+0000 7f9327fff640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f930c065a60 0x7f930c067f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93140af000 tx=0x7f93140b6040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:27.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.584+0000 7f93257fa640 1 -- 172.21.15.26:0/3580053315 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f931c051200 con 0x7f932806ac80 2023-12-19T16:42:27.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.772+0000 7f932f1ce640 1 -- 172.21.15.26:0/3580053315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f93280cd7b0 con 0x7f932806ac80 2023-12-19T16:42:27.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.780+0000 7f93257fa640 1 -- 172.21.15.26:0/3580053315 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f931c054820 con 0x7f932806ac80 2023-12-19T16:42:27.786 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 2 and data pool 3 2023-12-19T16:42:27.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.784+0000 7f932f1ce640 1 -- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f930c065a60 msgr2=0x7f930c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:27.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.784+0000 7f932f1ce640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f930c065a60 0x7f930c067f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93140af000 tx=0x7f93140b6040 comp rx=0 tx=0).stop 2023-12-19T16:42:27.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.784+0000 7f932f1ce640 1 -- 172.21.15.26:0/3580053315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932806ac80 msgr2=0x7f9328150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:27.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.784+0000 7f932f1ce640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932806ac80 0x7f9328150cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f931c00c450 tx=0x7f931c00c910 comp rx=0 tx=0).stop 2023-12-19T16:42:27.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.788+0000 7f932f1ce640 1 -- 172.21.15.26:0/3580053315 shutdown_connections 2023-12-19T16:42:27.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.788+0000 7f932f1ce640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932815ada0 0x7f93280c4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.788+0000 7f932f1ce640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f930c065a60 0x7f930c067f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.788+0000 7f932f1ce640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932815a9c0 0x7f932814f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.788+0000 7f932f1ce640 1 --2- 172.21.15.26:0/3580053315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932806ac80 0x7f9328150cd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.788+0000 7f932f1ce640 1 -- 172.21.15.26:0/3580053315 >> 172.21.15.26:0/3580053315 conn(0x7f93280ce730 msgr2=0x7f932806c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.788+0000 7f932f1ce640 1 -- 172.21.15.26:0/3580053315 shutdown_connections 2023-12-19T16:42:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.792+0000 7f932f1ce640 1 -- 172.21.15.26:0/3580053315 wait complete. 2023-12-19T16:42:27.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:27.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e8f396640 1 Processor -- start 2023-12-19T16:42:27.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e8f396640 1 -- start start 2023-12-19T16:42:27.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e8f396640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3e8806b3d0 0x7f3e880ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:27.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e8f396640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8806bc30 con 0x7f3e880cec40 2023-12-19T16:42:27.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e8f396640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e880cc6e0 con 0x7f3e88154220 2023-12-19T16:42:27.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e8f396640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e880cf0d0 con 0x7f3e8806b3d0 2023-12-19T16:42:27.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e8d132640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f3e88154220 0x7f3e8815aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46880/0 (socket says 172.21.15.26:46880) 2023-12-19T16:42:27.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e8d132640 1 -- 172.21.15.26:0/2403532587 learned_addr learned my addr 172.21.15.26:0/2403532587 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:27.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e8c931640 1 --2- 172.21.15.26:0/2403532587 >> v2:172.21.15.160:3300/0 conn(0x7f3e8806b3d0 0x7f3e880ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:27.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e7ffff640 1 -- 172.21.15.26:0/2403532587 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441443438 0 0) 0x7f3e880cc6e0 con 0x7f3e88154220 2023-12-19T16:42:27.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e7ffff640 1 -- 172.21.15.26:0/2403532587 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e74003530 con 0x7f3e88154220 2023-12-19T16:42:27.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e7ffff640 1 -- 172.21.15.26:0/2403532587 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496285056 0 0) 0x7f3e8806bc30 con 0x7f3e880cec40 2023-12-19T16:42:27.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e7ffff640 1 -- 172.21.15.26:0/2403532587 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e880cc6e0 con 0x7f3e880cec40 2023-12-19T16:42:27.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e8c931640 1 -- 172.21.15.26:0/2403532587 >> v1:172.21.15.160:6789/0 conn(0x7f3e88154220 legacy=0x7f3e8815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:27.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e8c931640 1 -- 172.21.15.26:0/2403532587 >> v1:172.21.15.26:6789/0 conn(0x7f3e880cec40 legacy=0x7f3e8815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:27.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.968+0000 7f3e8c931640 1 -- 172.21.15.26:0/2403532587 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8806b950 con 0x7f3e8806b3d0 2023-12-19T16:42:27.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.972+0000 7f3e8c931640 1 --2- 172.21.15.26:0/2403532587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e8806b3d0 0x7f3e880ce700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3e780097c0 tx=0x7f3e7801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51c8cc2fe488f05d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:27.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.972+0000 7f3e7ffff640 1 -- 172.21.15.26:0/2403532587 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e780ab650 con 0x7f3e8806b3d0 2023-12-19T16:42:27.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.972+0000 7f3e7ffff640 1 -- 172.21.15.26:0/2403532587 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e780ab7b0 con 0x7f3e8806b3d0 2023-12-19T16:42:27.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.972+0000 7f3e7ffff640 1 -- 172.21.15.26:0/2403532587 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e780abb40 con 0x7f3e8806b3d0 2023-12-19T16:42:27.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.972+0000 7f3e8f396640 1 -- 172.21.15.26:0/2403532587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e8806b3d0 msgr2=0x7f3e880ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:27.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.972+0000 7f3e8f396640 1 --2- 172.21.15.26:0/2403532587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e8806b3d0 0x7f3e880ce700 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3e780097c0 tx=0x7f3e7801c450 comp rx=0 tx=0).stop 2023-12-19T16:42:27.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.972+0000 7f3e8f396640 1 -- 172.21.15.26:0/2403532587 shutdown_connections 2023-12-19T16:42:27.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.972+0000 7f3e8f396640 1 --2- 172.21.15.26:0/2403532587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e8806b3d0 0x7f3e880ce700 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.972+0000 7f3e8f396640 1 -- 172.21.15.26:0/2403532587 >> 172.21.15.26:0/2403532587 conn(0x7f3e880658e0 msgr2=0x7f3e88065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:27.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.972+0000 7f3e8f396640 1 -- 172.21.15.26:0/2403532587 shutdown_connections 2023-12-19T16:42:27.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.972+0000 7f3e8f396640 1 -- 172.21.15.26:0/2403532587 wait complete. 2023-12-19T16:42:27.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8f396640 1 Processor -- start 2023-12-19T16:42:27.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8f396640 1 -- start start 2023-12-19T16:42:27.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8f396640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e8806b3d0 0x7f3e8814e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8f396640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e880cec40 0x7f3e880cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8f396640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e88154220 0x7f3e880cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8f396640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e88159ac0 con 0x7f3e8806b3d0 2023-12-19T16:42:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8f396640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e88158070 con 0x7f3e88154220 2023-12-19T16:42:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8f396640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e88158350 con 0x7f3e880cec40 2023-12-19T16:42:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8d132640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e8806b3d0 0x7f3e8814e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8d132640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e8806b3d0 0x7f3e8814e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49034/0 (socket says 172.21.15.26:49034) 2023-12-19T16:42:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8d132640 1 -- 172.21.15.26:0/3306224527 learned_addr learned my addr 172.21.15.26:0/3306224527 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8d132640 1 -- 172.21.15.26:0/3306224527 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e880cec40 msgr2=0x7f3e880cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8d132640 1 --2- 172.21.15.26:0/3306224527 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e880cec40 0x7f3e880cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8d132640 1 -- 172.21.15.26:0/3306224527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e88154220 msgr2=0x7f3e880cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8d132640 1 --2- 172.21.15.26:0/3306224527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e88154220 0x7f3e880cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8d132640 1 -- 172.21.15.26:0/3306224527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e78009420 con 0x7f3e8806b3d0 2023-12-19T16:42:27.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8d132640 1 --2- 172.21.15.26:0/3306224527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e8806b3d0 0x7f3e8814e290 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400c4f0 tx=0x7f3e8400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:27.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e7dffb640 1 -- 172.21.15.26:0/3306224527 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84005cf0 con 0x7f3e8806b3d0 2023-12-19T16:42:27.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e7dffb640 1 -- 172.21.15.26:0/3306224527 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e84005e50 con 0x7f3e8806b3d0 2023-12-19T16:42:27.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e7dffb640 1 -- 172.21.15.26:0/3306224527 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84007270 con 0x7f3e8806b3d0 2023-12-19T16:42:27.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8f396640 1 -- 172.21.15.26:0/3306224527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e88158630 con 0x7f3e8806b3d0 2023-12-19T16:42:27.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.976+0000 7f3e8f396640 1 -- 172.21.15.26:0/3306224527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e88158bf0 con 0x7f3e8806b3d0 2023-12-19T16:42:27.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.980+0000 7f3e8f396640 1 -- 172.21.15.26:0/3306224527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e881581b0 con 0x7f3e8806b3d0 2023-12-19T16:42:27.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.980+0000 7f3e7dffb640 1 -- 172.21.15.26:0/3306224527 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3e840073d0 con 0x7f3e8806b3d0 2023-12-19T16:42:27.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.980+0000 7f3e7dffb640 1 --2- 172.21.15.26:0/3306224527 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3e6c065a60 0x7f3e6c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:27.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.980+0000 7f3e7dffb640 1 -- 172.21.15.26:0/3306224527 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e84082710 con 0x7f3e8806b3d0 2023-12-19T16:42:27.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.980+0000 7f3e8c931640 1 --2- 172.21.15.26:0/3306224527 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3e6c065a60 0x7f3e6c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.980+0000 7f3e8c931640 1 --2- 172.21.15.26:0/3306224527 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3e6c065a60 0x7f3e6c067f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e780af000 tx=0x7f3e78008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:27.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:27.984+0000 7f3e7dffb640 1 -- 172.21.15.26:0/3306224527 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3e881581b0 con 0x7f3e8806b3d0 2023-12-19T16:42:28.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.144+0000 7f3e8f396640 1 -- 172.21.15.26:0/3306224527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e881581b0 con 0x7f3e8806b3d0 2023-12-19T16:42:28.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.144+0000 7f3e7dffb640 1 -- 172.21.15.26:0/3306224527 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16050 (secure 0 0 0) 0x7f3e881581b0 con 0x7f3e8806b3d0 2023-12-19T16:42:28.149 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:28.149 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:27.776757+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.144+0000 7f3e8f396640 1 -- 172.21.15.26:0/3306224527 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3e6c065a60 msgr2=0x7f3e6c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.144+0000 7f3e8f396640 1 --2- 172.21.15.26:0/3306224527 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3e6c065a60 0x7f3e6c067f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e780af000 tx=0x7f3e78008040 comp rx=0 tx=0).stop 2023-12-19T16:42:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.148+0000 7f3e8f396640 1 -- 172.21.15.26:0/3306224527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e8806b3d0 msgr2=0x7f3e8814e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.148+0000 7f3e8f396640 1 --2- 172.21.15.26:0/3306224527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e8806b3d0 0x7f3e8814e290 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400c4f0 tx=0x7f3e8400c9b0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.148+0000 7f3e8f396640 1 -- 172.21.15.26:0/3306224527 shutdown_connections 2023-12-19T16:42:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.148+0000 7f3e8f396640 1 --2- 172.21.15.26:0/3306224527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3e88154220 0x7f3e880cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.148+0000 7f3e8f396640 1 --2- 172.21.15.26:0/3306224527 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3e6c065a60 0x7f3e6c067f20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.148+0000 7f3e8f396640 1 --2- 172.21.15.26:0/3306224527 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3e880cec40 0x7f3e880cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.148+0000 7f3e8f396640 1 --2- 172.21.15.26:0/3306224527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e8806b3d0 0x7f3e8814e290 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.148+0000 7f3e8f396640 1 -- 172.21.15.26:0/3306224527 >> 172.21.15.26:0/3306224527 conn(0x7f3e880658e0 msgr2=0x7f3e880ac850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.148+0000 7f3e8f396640 1 -- 172.21.15.26:0/3306224527 shutdown_connections 2023-12-19T16:42:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.148+0000 7f3e8f396640 1 -- 172.21.15.26:0/3306224527 wait complete. 2023-12-19T16:42:28.170 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:28.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.308+0000 7f98be49a640 1 Processor -- start 2023-12-19T16:42:28.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.308+0000 7f98be49a640 1 -- start start 2023-12-19T16:42:28.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.308+0000 7f98be49a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98b80b4070 0x7f98b8071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.308+0000 7f98be49a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b815e7c0 con 0x7f98b80b2030 2023-12-19T16:42:28.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.308+0000 7f98be49a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b80b04c0 con 0x7f98b80affe0 2023-12-19T16:42:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.308+0000 7f98be49a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80c97a0 con 0x7f98b80b4070 2023-12-19T16:42:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.308+0000 7f98bd498640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f98b80b2030 0x7f98b80c9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50208/0 (socket says 172.21.15.26:50208) 2023-12-19T16:42:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.308+0000 7f98bd498640 1 -- 172.21.15.26:0/4162036973 learned_addr learned my addr 172.21.15.26:0/4162036973 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98bdc99640 1 --2- 172.21.15.26:0/4162036973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98b80b4070 0x7f98b8071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3054166520 0 0) 0x7f98b815e7c0 con 0x7f98b80b2030 2023-12-19T16:42:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98a4003530 con 0x7f98b80b2030 2023-12-19T16:42:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1100979471 0 0) 0x7f98a4003530 con 0x7f98b80b2030 2023-12-19T16:42:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98b80b4070 msgr2=0x7f98b8071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 --2- 172.21.15.26:0/4162036973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98b80b4070 0x7f98b8071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 >> v1:172.21.15.160:6789/0 conn(0x7f98b80affe0 legacy=0x7f98b80b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b80b07a0 con 0x7f98b80b2030 2023-12-19T16:42:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f98b4001680 con 0x7f98b80b2030 2023-12-19T16:42:28.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98bdc99640 1 --2- 172.21.15.26:0/4162036973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98b80b4070 0x7f98b8071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 >> v1:172.21.15.26:6789/0 conn(0x7f98b80b2030 legacy=0x7f98b80c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 --2- 172.21.15.26:0/4162036973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a4003a20 0x7f98a4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 --2- 172.21.15.26:0/4162036973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98a40047d0 0x7f98a4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 --2- 172.21.15.26:0/4162036973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a4005190 0x7f98a400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80c97a0 con 0x7f98a4005190 2023-12-19T16:42:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a400c000 con 0x7f98a40047d0 2023-12-19T16:42:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a400c1c0 con 0x7f98a4003a20 2023-12-19T16:42:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98bdc99640 1 --2- 172.21.15.26:0/4162036973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a4005190 0x7f98a400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98bcc97640 1 --2- 172.21.15.26:0/4162036973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98a40047d0 0x7f98a4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98bd498640 1 --2- 172.21.15.26:0/4162036973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a4003a20 0x7f98a4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98bdc99640 1 -- 172.21.15.26:0/4162036973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a4003a20 msgr2=0x7f98a4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:28.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98bdc99640 1 --2- 172.21.15.26:0/4162036973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a4003a20 0x7f98a4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98bdc99640 1 -- 172.21.15.26:0/4162036973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98a40047d0 msgr2=0x7f98a4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98bdc99640 1 --2- 172.21.15.26:0/4162036973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98a40047d0 0x7f98a4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.312+0000 7f98bdc99640 1 -- 172.21.15.26:0/4162036973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a4003530 con 0x7f98a4005190 2023-12-19T16:42:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.316+0000 7f98bdc99640 1 --2- 172.21.15.26:0/4162036973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a4005190 0x7f98a400b990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98b000e1f0 tx=0x7f98b001cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9917faa106313e1e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.316+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b00a5d70 con 0x7f98a4005190 2023-12-19T16:42:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.316+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b000f040 con 0x7f98a4005190 2023-12-19T16:42:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.316+0000 7f98affff640 1 -- 172.21.15.26:0/4162036973 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0003bc0 con 0x7f98a4005190 2023-12-19T16:42:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.316+0000 7f98be49a640 1 -- 172.21.15.26:0/4162036973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a4005190 msgr2=0x7f98a400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:28.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.316+0000 7f98be49a640 1 --2- 172.21.15.26:0/4162036973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a4005190 0x7f98a400b990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f98b000e1f0 tx=0x7f98b001cd20 comp rx=0 tx=0).stop 2023-12-19T16:42:28.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.316+0000 7f98bdc99640 1 -- 172.21.15.26:0/4162036973 reap_dead start 2023-12-19T16:42:28.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.316+0000 7f98be49a640 1 -- 172.21.15.26:0/4162036973 shutdown_connections 2023-12-19T16:42:28.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.316+0000 7f98be49a640 1 -- 172.21.15.26:0/4162036973 >> 172.21.15.26:0/4162036973 conn(0x7f98b806b0f0 msgr2=0x7f98b806b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:28.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.316+0000 7f98be49a640 1 -- 172.21.15.26:0/4162036973 shutdown_connections 2023-12-19T16:42:28.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.316+0000 7f98be49a640 1 -- 172.21.15.26:0/4162036973 wait complete. 2023-12-19T16:42:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.316+0000 7f98be49a640 1 Processor -- start 2023-12-19T16:42:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98be49a640 1 -- start start 2023-12-19T16:42:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98be49a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a4005190 0x7f98b81f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98be49a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98b80c7880 0x7f98b80c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98be49a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98b80c6560 0x7f98b80ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98be49a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80c7630 con 0x7f98a4005190 2023-12-19T16:42:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98be49a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b81f2820 con 0x7f98b80c7880 2023-12-19T16:42:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98be49a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80c6a20 con 0x7f98b80c6560 2023-12-19T16:42:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98bd498640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a4005190 0x7f98b81f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98bd498640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a4005190 0x7f98b81f22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49062/0 (socket says 172.21.15.26:49062) 2023-12-19T16:42:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98bd498640 1 -- 172.21.15.26:0/1416581715 learned_addr learned my addr 172.21.15.26:0/1416581715 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98bcc97640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98b80c7880 0x7f98b80c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98bdc99640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98b80c6560 0x7f98b80ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98bd498640 1 -- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98b80c6560 msgr2=0x7f98b80ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98bd498640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98b80c6560 0x7f98b80ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98bd498640 1 -- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98b80c7880 msgr2=0x7f98b80c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98bd498640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98b80c7880 0x7f98b80c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98bd498640 1 -- 172.21.15.26:0/1416581715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b80c6d00 con 0x7f98a4005190 2023-12-19T16:42:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98bd498640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a4005190 0x7f98b81f22e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f98b4007290 tx=0x7f98b4007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98adffb640 1 -- 172.21.15.26:0/1416581715 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b40014a0 con 0x7f98a4005190 2023-12-19T16:42:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98adffb640 1 -- 172.21.15.26:0/1416581715 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b400f940 con 0x7f98a4005190 2023-12-19T16:42:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98adffb640 1 -- 172.21.15.26:0/1416581715 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b400fcf0 con 0x7f98a4005190 2023-12-19T16:42:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98be49a640 1 -- 172.21.15.26:0/1416581715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b80c6fe0 con 0x7f98a4005190 2023-12-19T16:42:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98be49a640 1 -- 172.21.15.26:0/1416581715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b80acda0 con 0x7f98a4005190 2023-12-19T16:42:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.320+0000 7f98be49a640 1 -- 172.21.15.26:0/1416581715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b8067400 con 0x7f98a4005190 2023-12-19T16:42:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.324+0000 7f98adffb640 1 -- 172.21.15.26:0/1416581715 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f98b401d460 con 0x7f98a4005190 2023-12-19T16:42:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.324+0000 7f98adffb640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f989c065a60 0x7f989c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.324+0000 7f98adffb640 1 -- 172.21.15.26:0/1416581715 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98b4083280 con 0x7f98a4005190 2023-12-19T16:42:28.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.324+0000 7f98bcc97640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f989c065a60 0x7f989c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.324+0000 7f98bcc97640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f989c065a60 0x7f989c067f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98b80aff10 tx=0x7f98a8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:28.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.328+0000 7f98adffb640 1 -- 172.21.15.26:0/1416581715 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f98b4050c40 con 0x7f98a4005190 2023-12-19T16:42:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.488+0000 7f98be49a640 1 -- 172.21.15.26:0/1416581715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98b815cad0 con 0x7f98a4005190 2023-12-19T16:42:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.488+0000 7f98adffb640 1 -- 172.21.15.26:0/1416581715 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16050 (secure 0 0 0) 0x7f98b4054260 con 0x7f98a4005190 2023-12-19T16:42:28.494 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:28.494 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:27.776757+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:28.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.492+0000 7f98be49a640 1 -- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f989c065a60 msgr2=0x7f989c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.492+0000 7f98be49a640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f989c065a60 0x7f989c067f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98b80aff10 tx=0x7f98a8009040 comp rx=0 tx=0).stop 2023-12-19T16:42:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.492+0000 7f98be49a640 1 -- 172.21.15.26:0/1416581715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a4005190 msgr2=0x7f98b81f22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.492+0000 7f98be49a640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a4005190 0x7f98b81f22e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f98b4007290 tx=0x7f98b4007690 comp rx=0 tx=0).stop 2023-12-19T16:42:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.492+0000 7f98be49a640 1 -- 172.21.15.26:0/1416581715 shutdown_connections 2023-12-19T16:42:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.492+0000 7f98be49a640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f989c065a60 0x7f989c067f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.492+0000 7f98be49a640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98b80c6560 0x7f98b80ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.492+0000 7f98be49a640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98b80c7880 0x7f98b80c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.492+0000 7f98be49a640 1 --2- 172.21.15.26:0/1416581715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a4005190 0x7f98b81f22e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.492+0000 7f98be49a640 1 -- 172.21.15.26:0/1416581715 >> 172.21.15.26:0/1416581715 conn(0x7f98b806b0f0 msgr2=0x7f98b815d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.492+0000 7f98be49a640 1 -- 172.21.15.26:0/1416581715 shutdown_connections 2023-12-19T16:42:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.492+0000 7f98be49a640 1 -- 172.21.15.26:0/1416581715 wait complete. 2023-12-19T16:42:28.518 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T16:42:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.660+0000 7fc91d9db640 1 Processor -- start 2023-12-19T16:42:28.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.660+0000 7fc91d9db640 1 -- start start 2023-12-19T16:42:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.660+0000 7fc91d9db640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc91815cf80 0x7fc9180cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.660+0000 7fc91d9db640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc9180ccbd0 0x7fc9180cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.660+0000 7fc91d9db640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9180ce750 con 0x7fc9180cd910 2023-12-19T16:42:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.660+0000 7fc91d9db640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9180cdd40 con 0x7fc9180ccbd0 2023-12-19T16:42:28.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.660+0000 7fc91d9db640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9180afaa0 con 0x7fc91815cf80 2023-12-19T16:42:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.660+0000 7fc90ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc9180ccbd0 0x7fc9180cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.660+0000 7fc9177fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc9180cd910 0x7fc91806a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50216/0 (socket says 172.21.15.26:50216) 2023-12-19T16:42:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.660+0000 7fc9177fe640 1 -- 172.21.15.26:0/4130534431 learned_addr learned my addr 172.21.15.26:0/4130534431 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.660+0000 7fc916ffd640 1 --2- 172.21.15.26:0/4130534431 >> v2:172.21.15.160:3300/0 conn(0x7fc91815cf80 0x7fc9180cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc9167fc640 1 -- 172.21.15.26:0/4130534431 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881375489 0 0) 0x7fc9180ce750 con 0x7fc9180cd910 2023-12-19T16:42:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc9167fc640 1 -- 172.21.15.26:0/4130534431 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc904003530 con 0x7fc9180cd910 2023-12-19T16:42:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc90ffff640 1 -- 172.21.15.26:0/4130534431 >> v2:172.21.15.160:3300/0 conn(0x7fc91815cf80 msgr2=0x7fc9180cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc90ffff640 1 --2- 172.21.15.26:0/4130534431 >> v2:172.21.15.160:3300/0 conn(0x7fc91815cf80 0x7fc9180cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc90ffff640 1 -- 172.21.15.26:0/4130534431 >> v1:172.21.15.26:6789/0 conn(0x7fc9180cd910 legacy=0x7fc91806a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc90ffff640 1 -- 172.21.15.26:0/4130534431 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9180afd80 con 0x7fc9180ccbd0 2023-12-19T16:42:28.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc90ffff640 1 --2- 172.21.15.26:0/4130534431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9180ccbd0 0x7fc9180cd3d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc91806c990 tx=0x7fc90001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56665a84d9e998ab server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:28.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc9167fc640 1 -- 172.21.15.26:0/4130534431 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000ab610 con 0x7fc9180ccbd0 2023-12-19T16:42:28.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc9167fc640 1 -- 172.21.15.26:0/4130534431 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9000ab770 con 0x7fc9180ccbd0 2023-12-19T16:42:28.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc9167fc640 1 -- 172.21.15.26:0/4130534431 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000abae0 con 0x7fc9180ccbd0 2023-12-19T16:42:28.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc91d9db640 1 -- 172.21.15.26:0/4130534431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9180ccbd0 msgr2=0x7fc9180cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:28.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc91d9db640 1 --2- 172.21.15.26:0/4130534431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9180ccbd0 0x7fc9180cd3d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc91806c990 tx=0x7fc90001c410 comp rx=0 tx=0).stop 2023-12-19T16:42:28.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc91d9db640 1 -- 172.21.15.26:0/4130534431 shutdown_connections 2023-12-19T16:42:28.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc91d9db640 1 --2- 172.21.15.26:0/4130534431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc9180ccbd0 0x7fc9180cd3d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc91d9db640 1 --2- 172.21.15.26:0/4130534431 >> v2:172.21.15.160:3300/0 conn(0x7fc91815cf80 0x7fc9180cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.664+0000 7fc91d9db640 1 -- 172.21.15.26:0/4130534431 >> 172.21.15.26:0/4130534431 conn(0x7fc918071490 msgr2=0x7fc9180718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc91d9db640 1 -- 172.21.15.26:0/4130534431 shutdown_connections 2023-12-19T16:42:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc91d9db640 1 -- 172.21.15.26:0/4130534431 wait complete. 2023-12-19T16:42:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc91d9db640 1 Processor -- start 2023-12-19T16:42:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc91d9db640 1 -- start start 2023-12-19T16:42:28.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc91d9db640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9180ccbd0 0x7fc91814fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc91d9db640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc9180cd910 0x7fc918157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc91d9db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91815cf80 0x7fc918157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc91d9db640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180b4950 con 0x7fc91815cf80 2023-12-19T16:42:28.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc91d9db640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180b4ba0 con 0x7fc9180ccbd0 2023-12-19T16:42:28.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc91d9db640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180b4e80 con 0x7fc9180cd910 2023-12-19T16:42:28.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc9177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91815cf80 0x7fc918157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc9177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91815cf80 0x7fc918157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49074/0 (socket says 172.21.15.26:49074) 2023-12-19T16:42:28.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc9177fe640 1 -- 172.21.15.26:0/3968995911 learned_addr learned my addr 172.21.15.26:0/3968995911 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:28.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc916ffd640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9180ccbd0 0x7fc91814fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc90ffff640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc9180cd910 0x7fc918157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc9177fe640 1 -- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc9180cd910 msgr2=0x7fc918157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc9177fe640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc9180cd910 0x7fc918157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc9177fe640 1 -- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9180ccbd0 msgr2=0x7fc91814fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc9177fe640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9180ccbd0 0x7fc91814fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc9177fe640 1 -- 172.21.15.26:0/3968995911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9000093b0 con 0x7fc91815cf80 2023-12-19T16:42:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.668+0000 7fc9177fe640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91815cf80 0x7fc918157fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc91000cb40 tx=0x7fc91000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:28.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.672+0000 7fc914ff9640 1 -- 172.21.15.26:0/3968995911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91000aca0 con 0x7fc91815cf80 2023-12-19T16:42:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.672+0000 7fc914ff9640 1 -- 172.21.15.26:0/3968995911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91000ae00 con 0x7fc91815cf80 2023-12-19T16:42:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.672+0000 7fc914ff9640 1 -- 172.21.15.26:0/3968995911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910012620 con 0x7fc91815cf80 2023-12-19T16:42:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.672+0000 7fc91d9db640 1 -- 172.21.15.26:0/3968995911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9180b5160 con 0x7fc91815cf80 2023-12-19T16:42:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.672+0000 7fc91d9db640 1 -- 172.21.15.26:0/3968995911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9180b5720 con 0x7fc91815cf80 2023-12-19T16:42:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.672+0000 7fc91d9db640 1 -- 172.21.15.26:0/3968995911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9180b5910 con 0x7fc91815cf80 2023-12-19T16:42:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.672+0000 7fc914ff9640 1 -- 172.21.15.26:0/3968995911 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc910007070 con 0x7fc91815cf80 2023-12-19T16:42:28.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.672+0000 7fc914ff9640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc8f8065a60 0x7fc8f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:28.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.672+0000 7fc914ff9640 1 -- 172.21.15.26:0/3968995911 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc910083140 con 0x7fc91815cf80 2023-12-19T16:42:28.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.672+0000 7fc916ffd640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc8f8065a60 0x7fc8f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:28.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.676+0000 7fc916ffd640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc8f8065a60 0x7fc8f8067f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc908009ac0 tx=0x7fc908009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:28.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.676+0000 7fc914ff9640 1 -- 172.21.15.26:0/3968995911 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc910050ba0 con 0x7fc91815cf80 2023-12-19T16:42:28.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:28.868+0000 7fc91d9db640 1 -- 172.21.15.26:0/3968995911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc918074270 con 0x7fc91815cf80 2023-12-19T16:42:29.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.796+0000 7fc914ff9640 1 -- 172.21.15.26:0/3968995911 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc9100541c0 con 0x7fc91815cf80 2023-12-19T16:42:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.800+0000 7fc91d9db640 1 -- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc8f8065a60 msgr2=0x7fc8f8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.800+0000 7fc91d9db640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc8f8065a60 0x7fc8f8067f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc908009ac0 tx=0x7fc908009190 comp rx=0 tx=0).stop 2023-12-19T16:42:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.800+0000 7fc91d9db640 1 -- 172.21.15.26:0/3968995911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91815cf80 msgr2=0x7fc918157fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.800+0000 7fc91d9db640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91815cf80 0x7fc918157fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc91000cb40 tx=0x7fc91000a060 comp rx=0 tx=0).stop 2023-12-19T16:42:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.800+0000 7fc91d9db640 1 -- 172.21.15.26:0/3968995911 shutdown_connections 2023-12-19T16:42:29.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.800+0000 7fc91d9db640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc91815cf80 0x7fc918157fe0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:29.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.800+0000 7fc91d9db640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc8f8065a60 0x7fc8f8067f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:29.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.800+0000 7fc91d9db640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc9180cd910 0x7fc918157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:29.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.800+0000 7fc91d9db640 1 --2- 172.21.15.26:0/3968995911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9180ccbd0 0x7fc91814fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:29.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.800+0000 7fc91d9db640 1 -- 172.21.15.26:0/3968995911 >> 172.21.15.26:0/3968995911 conn(0x7fc918071490 msgr2=0x7fc9180b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:29.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.800+0000 7fc91d9db640 1 -- 172.21.15.26:0/3968995911 shutdown_connections 2023-12-19T16:42:29.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.800+0000 7fc91d9db640 1 -- 172.21.15.26:0/3968995911 wait complete. 2023-12-19T16:42:29.827 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T16:42:29.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T16:42:29.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.972+0000 7f7376598640 1 Processor -- start 2023-12-19T16:42:29.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.972+0000 7f7376598640 1 -- start start 2023-12-19T16:42:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f7376598640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f737006f6c0 0x7f737015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:29.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f7376598640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7370159c00 0x7f7370070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f7376598640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f737006fb60 con 0x7f7370159820 2023-12-19T16:42:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f7376598640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737015e470 con 0x7f7370159c00 2023-12-19T16:42:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f7376598640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737006fe40 con 0x7f737006f6c0 2023-12-19T16:42:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f7374b35640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7370159c00 0x7f7370070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f736ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7370159820 0x7f73700c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50222/0 (socket says 172.21.15.26:50222) 2023-12-19T16:42:29.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f736ffff640 1 -- 172.21.15.26:0/4237834533 learned_addr learned my addr 172.21.15.26:0/4237834533 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f736f7fe640 1 --2- 172.21.15.26:0/4237834533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f737006f6c0 0x7f737015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f736effd640 1 -- 172.21.15.26:0/4237834533 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614141035 0 0) 0x7f737006fb60 con 0x7f7370159820 2023-12-19T16:42:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f7374b35640 1 -- 172.21.15.26:0/4237834533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f737006f6c0 msgr2=0x7f737015df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f7374b35640 1 --2- 172.21.15.26:0/4237834533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f737006f6c0 0x7f737015df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f7374b35640 1 -- 172.21.15.26:0/4237834533 >> v1:172.21.15.26:6789/0 conn(0x7f7370159820 legacy=0x7f73700c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:29.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.976+0000 7f7374b35640 1 -- 172.21.15.26:0/4237834533 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7370070120 con 0x7f7370159c00 2023-12-19T16:42:29.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.980+0000 7f7374b35640 1 --2- 172.21.15.26:0/4237834533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7370159c00 0x7f7370070c00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7368009d90 tx=0x7f736801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef4db767f16de39 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:29.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.980+0000 7f736effd640 1 -- 172.21.15.26:0/4237834533 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73680a58f0 con 0x7f7370159c00 2023-12-19T16:42:29.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.980+0000 7f736effd640 1 -- 172.21.15.26:0/4237834533 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73680a5a50 con 0x7f7370159c00 2023-12-19T16:42:29.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.980+0000 7f736effd640 1 -- 172.21.15.26:0/4237834533 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73680a5de0 con 0x7f7370159c00 2023-12-19T16:42:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.980+0000 7f7376598640 1 -- 172.21.15.26:0/4237834533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7370159c00 msgr2=0x7f7370070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.980+0000 7f7376598640 1 --2- 172.21.15.26:0/4237834533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7370159c00 0x7f7370070c00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7368009d90 tx=0x7f736801c920 comp rx=0 tx=0).stop 2023-12-19T16:42:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.980+0000 7f7376598640 1 -- 172.21.15.26:0/4237834533 shutdown_connections 2023-12-19T16:42:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.980+0000 7f7376598640 1 --2- 172.21.15.26:0/4237834533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7370159c00 0x7f7370070c00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.980+0000 7f7376598640 1 --2- 172.21.15.26:0/4237834533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f737006f6c0 0x7f737015df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:29.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.980+0000 7f7376598640 1 -- 172.21.15.26:0/4237834533 >> 172.21.15.26:0/4237834533 conn(0x7f73700ce730 msgr2=0x7f73700ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:29.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.980+0000 7f7376598640 1 -- 172.21.15.26:0/4237834533 shutdown_connections 2023-12-19T16:42:29.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.980+0000 7f7376598640 1 -- 172.21.15.26:0/4237834533 wait complete. 2023-12-19T16:42:29.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.980+0000 7f7376598640 1 Processor -- start 2023-12-19T16:42:29.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f7376598640 1 -- start start 2023-12-19T16:42:29.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f7376598640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737006f6c0 0x7f737015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:29.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f7376598640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7370159820 0x7f7370152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:29.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f7376598640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7370159c00 0x7f73700ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:29.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f7376598640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737015b260 con 0x7f737006f6c0 2023-12-19T16:42:29.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f7376598640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73700b2b10 con 0x7f7370159c00 2023-12-19T16:42:29.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f7376598640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73700b2df0 con 0x7f7370159820 2023-12-19T16:42:29.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f736ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737006f6c0 0x7f737015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:29.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f736ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737006f6c0 0x7f737015ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49092/0 (socket says 172.21.15.26:49092) 2023-12-19T16:42:29.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f736ffff640 1 -- 172.21.15.26:0/1346510949 learned_addr learned my addr 172.21.15.26:0/1346510949 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:29.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f736f7fe640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7370159820 0x7f7370152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:29.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f7374b35640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7370159c00 0x7f73700ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:29.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f736ffff640 1 -- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7370159820 msgr2=0x7f7370152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:29.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f736ffff640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7370159820 0x7f7370152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:29.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f736ffff640 1 -- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7370159c00 msgr2=0x7f73700ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:29.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f736ffff640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7370159c00 0x7f73700ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:29.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f736ffff640 1 -- 172.21.15.26:0/1346510949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7368009a70 con 0x7f737006f6c0 2023-12-19T16:42:29.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f7374b35640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7370159c00 0x7f73700ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:29.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f736ffff640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737006f6c0 0x7f737015ad20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73600011f0 tx=0x7f736000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:29.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f736cff9640 1 -- 172.21.15.26:0/1346510949 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73600074e0 con 0x7f737006f6c0 2023-12-19T16:42:29.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f736cff9640 1 -- 172.21.15.26:0/1346510949 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7360007640 con 0x7f737006f6c0 2023-12-19T16:42:29.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f7376598640 1 -- 172.21.15.26:0/1346510949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737006b880 con 0x7f737006f6c0 2023-12-19T16:42:29.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f736cff9640 1 -- 172.21.15.26:0/1346510949 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360004a10 con 0x7f737006f6c0 2023-12-19T16:42:29.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f7376598640 1 -- 172.21.15.26:0/1346510949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73700b33b0 con 0x7f737006f6c0 2023-12-19T16:42:29.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.984+0000 7f7376598640 1 -- 172.21.15.26:0/1346510949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7370074270 con 0x7f737006f6c0 2023-12-19T16:42:29.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.988+0000 7f736cff9640 1 -- 172.21.15.26:0/1346510949 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7360004b70 con 0x7f737006f6c0 2023-12-19T16:42:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.988+0000 7f736cff9640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f735c065a60 0x7f735c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.988+0000 7f736cff9640 1 -- 172.21.15.26:0/1346510949 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73600823e0 con 0x7f737006f6c0 2023-12-19T16:42:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.988+0000 7f736f7fe640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f735c065a60 0x7f735c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.988+0000 7f736f7fe640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f735c065a60 0x7f735c067f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73700adce0 tx=0x7f7364008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:29.992+0000 7f736cff9640 1 -- 172.21.15.26:0/1346510949 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f736004fe40 con 0x7f737006f6c0 2023-12-19T16:42:30.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.184+0000 7f7376598640 1 -- 172.21.15.26:0/1346510949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f73700b46b0 con 0x7f737006f6c0 2023-12-19T16:42:30.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.804+0000 7f736cff9640 1 -- 172.21.15.26:0/1346510949 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f73600180c0 con 0x7f737006f6c0 2023-12-19T16:42:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.808+0000 7f7376598640 1 -- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f735c065a60 msgr2=0x7f735c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.808+0000 7f7376598640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f735c065a60 0x7f735c067f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f73700adce0 tx=0x7f7364008040 comp rx=0 tx=0).stop 2023-12-19T16:42:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.808+0000 7f7376598640 1 -- 172.21.15.26:0/1346510949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737006f6c0 msgr2=0x7f737015ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.808+0000 7f7376598640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737006f6c0 0x7f737015ad20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f73600011f0 tx=0x7f736000c9b0 comp rx=0 tx=0).stop 2023-12-19T16:42:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.808+0000 7f7376598640 1 -- 172.21.15.26:0/1346510949 shutdown_connections 2023-12-19T16:42:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.808+0000 7f7376598640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7370159c00 0x7f73700ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.808+0000 7f7376598640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f735c065a60 0x7f735c067f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.808+0000 7f7376598640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7370159820 0x7f7370152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.808+0000 7f7376598640 1 --2- 172.21.15.26:0/1346510949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737006f6c0 0x7f737015ad20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.808+0000 7f7376598640 1 -- 172.21.15.26:0/1346510949 >> 172.21.15.26:0/1346510949 conn(0x7f73700ce730 msgr2=0x7f73700ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.808+0000 7f7376598640 1 -- 172.21.15.26:0/1346510949 shutdown_connections 2023-12-19T16:42:30.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.808+0000 7f7376598640 1 -- 172.21.15.26:0/1346510949 wait complete. 2023-12-19T16:42:30.835 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T16:42:30.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:42:30.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd4176d640 1 Processor -- start 2023-12-19T16:42:30.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd4176d640 1 -- start start 2023-12-19T16:42:30.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd4176d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcd3c0d1290 0x7fcd3c0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:30.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd4176d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd3c06afd0 0x7fcd3c0b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:30.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd4176d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3c06b790 con 0x7fcd3c0cd330 2023-12-19T16:42:30.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd4176d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c06abb0 con 0x7fcd3c0d1290 2023-12-19T16:42:30.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd4176d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c06bce0 con 0x7fcd3c06afd0 2023-12-19T16:42:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd3affd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcd3c0d1290 0x7fcd3c0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd33fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd3c06afd0 0x7fcd3c0b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd3affd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcd3c0d1290 0x7fcd3c0b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:33740/0 (socket says 172.21.15.26:33740) 2023-12-19T16:42:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd3affd640 1 -- 172.21.15.26:0/901892059 learned_addr learned my addr 172.21.15.26:0/901892059 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd3b7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fcd3c0cd330 0x7fcd3c0cfe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46904/0 (socket says 172.21.15.26:46904) 2023-12-19T16:42:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd3affd640 1 -- 172.21.15.26:0/901892059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd3c06afd0 msgr2=0x7fcd3c0b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd3affd640 1 --2- 172.21.15.26:0/901892059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd3c06afd0 0x7fcd3c0b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd3affd640 1 -- 172.21.15.26:0/901892059 >> v1:172.21.15.160:6789/0 conn(0x7fcd3c0cd330 legacy=0x7fcd3c0cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd3affd640 1 -- 172.21.15.26:0/901892059 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c06ba00 con 0x7fcd3c0d1290 2023-12-19T16:42:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.980+0000 7fcd33fff640 1 --2- 172.21.15.26:0/901892059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd3c06afd0 0x7fcd3c0b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:30.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.984+0000 7fcd3affd640 1 --2- 172.21.15.26:0/901892059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd3c0d1290 0x7fcd3c0b25d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009870 tx=0x7fcd2c01c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b292ab62ac1bb655 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.984+0000 7fcd3a7fc640 1 -- 172.21.15.26:0/901892059 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0ab640 con 0x7fcd3c0d1290 2023-12-19T16:42:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.984+0000 7fcd3a7fc640 1 -- 172.21.15.26:0/901892059 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c0a7070 con 0x7fcd3c0d1290 2023-12-19T16:42:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.984+0000 7fcd3a7fc640 1 -- 172.21.15.26:0/901892059 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0ab9c0 con 0x7fcd3c0d1290 2023-12-19T16:42:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.984+0000 7fcd4176d640 1 -- 172.21.15.26:0/901892059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd3c0d1290 msgr2=0x7fcd3c0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.984+0000 7fcd4176d640 1 --2- 172.21.15.26:0/901892059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd3c0d1290 0x7fcd3c0b25d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009870 tx=0x7fcd2c01c440 comp rx=0 tx=0).stop 2023-12-19T16:42:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.984+0000 7fcd4176d640 1 -- 172.21.15.26:0/901892059 shutdown_connections 2023-12-19T16:42:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.984+0000 7fcd4176d640 1 --2- 172.21.15.26:0/901892059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd3c06afd0 0x7fcd3c0b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.984+0000 7fcd4176d640 1 --2- 172.21.15.26:0/901892059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd3c0d1290 0x7fcd3c0b25d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.984+0000 7fcd4176d640 1 -- 172.21.15.26:0/901892059 >> 172.21.15.26:0/901892059 conn(0x7fcd3c06d570 msgr2=0x7fcd3c06d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.984+0000 7fcd4176d640 1 -- 172.21.15.26:0/901892059 shutdown_connections 2023-12-19T16:42:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.984+0000 7fcd4176d640 1 -- 172.21.15.26:0/901892059 wait complete. 2023-12-19T16:42:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd4176d640 1 Processor -- start 2023-12-19T16:42:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd4176d640 1 -- start start 2023-12-19T16:42:30.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd4176d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd3c06afd0 0x7fcd3c156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:30.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd4176d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd3c0cd330 0x7fcd3c0b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:30.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd4176d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd3c0d1290 0x7fcd3c0b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:30.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd4176d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0b49f0 con 0x7fcd3c0d1290 2023-12-19T16:42:30.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd4176d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c15b590 con 0x7fcd3c0cd330 2023-12-19T16:42:30.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd4176d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c15b870 con 0x7fcd3c06afd0 2023-12-19T16:42:30.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd3b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd3c0d1290 0x7fcd3c0b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:30.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd3b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd3c0d1290 0x7fcd3c0b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49096/0 (socket says 172.21.15.26:49096) 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd3b7fe640 1 -- 172.21.15.26:0/2262517784 learned_addr learned my addr 172.21.15.26:0/2262517784 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd3affd640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd3c06afd0 0x7fcd3c156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd33fff640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd3c0cd330 0x7fcd3c0b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd3b7fe640 1 -- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd3c06afd0 msgr2=0x7fcd3c156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd3b7fe640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd3c06afd0 0x7fcd3c156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd3b7fe640 1 -- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd3c0cd330 msgr2=0x7fcd3c0b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd3b7fe640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd3c0cd330 0x7fcd3c0b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd3b7fe640 1 -- 172.21.15.26:0/2262517784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd24009420 con 0x7fcd3c0d1290 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd33fff640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd3c0cd330 0x7fcd3c0b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd3b7fe640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd3c0d1290 0x7fcd3c0b5c20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c7a0 tx=0x7fcd3400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd337fe640 1 -- 172.21.15.26:0/2262517784 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400a640 con 0x7fcd3c0d1290 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd337fe640 1 -- 172.21.15.26:0/2262517784 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3400a7a0 con 0x7fcd3c0d1290 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd337fe640 1 -- 172.21.15.26:0/2262517784 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400ab10 con 0x7fcd3c0d1290 2023-12-19T16:42:30.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd4176d640 1 -- 172.21.15.26:0/2262517784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2c0094d0 con 0x7fcd3c0d1290 2023-12-19T16:42:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd4176d640 1 -- 172.21.15.26:0/2262517784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3c0cdcd0 con 0x7fcd3c0d1290 2023-12-19T16:42:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.988+0000 7fcd4176d640 1 -- 172.21.15.26:0/2262517784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3c15bd50 con 0x7fcd3c0d1290 2023-12-19T16:42:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.992+0000 7fcd337fe640 1 -- 172.21.15.26:0/2262517784 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcd3400ac70 con 0x7fcd3c0d1290 2023-12-19T16:42:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.992+0000 7fcd337fe640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd1c065a90 0x7fcd1c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.992+0000 7fcd337fe640 1 -- 172.21.15.26:0/2262517784 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd34082820 con 0x7fcd3c0d1290 2023-12-19T16:42:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.992+0000 7fcd3affd640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd1c065a90 0x7fcd1c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.992+0000 7fcd3affd640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd1c065a90 0x7fcd1c067f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0af000 tx=0x7fcd2c0b6040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:30.996+0000 7fcd337fe640 1 -- 172.21.15.26:0/2262517784 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcd34050320 con 0x7fcd3c0d1290 2023-12-19T16:42:31.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd4176d640 1 -- 172.21.15.26:0/2262517784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd3c074270 con 0x7fcd3c0d1290 2023-12-19T16:42:31.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd337fe640 1 -- 172.21.15.26:0/2262517784 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fcd34017020 con 0x7fcd3c0d1290 2023-12-19T16:42:31.191 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:31.192 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2850479611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2850479611},{"type":"v1","addr":"172.21.15.160:6836","nonce":2850479611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/837780144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":837780144},{"type":"v1","addr":"172.21.15.26:6834","nonce":837780144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2784408519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2784408519},{"type":"v1","addr":"172.21.15.26:6835","nonce":2784408519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:42:27.777197+0000","modified":"2023-12-19T16:42:30.798296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6837/3427345273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3427345273},{"type":"v1","addr":"172.21.15.160:6837","nonce":3427345273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T16:42:31.192 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2023-12-19T16:42:31.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd4176d640 1 -- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd1c065a90 msgr2=0x7fcd1c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd4176d640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd1c065a90 0x7fcd1c067f50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0af000 tx=0x7fcd2c0b6040 comp rx=0 tx=0).stop 2023-12-19T16:42:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd4176d640 1 -- 172.21.15.26:0/2262517784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd3c0d1290 msgr2=0x7fcd3c0b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd4176d640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd3c0d1290 0x7fcd3c0b5c20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c7a0 tx=0x7fcd3400cc60 comp rx=0 tx=0).stop 2023-12-19T16:42:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd4176d640 1 -- 172.21.15.26:0/2262517784 shutdown_connections 2023-12-19T16:42:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd4176d640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd3c0d1290 0x7fcd3c0b5c20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd4176d640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd3c0cd330 0x7fcd3c0b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd4176d640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd1c065a90 0x7fcd1c067f50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd4176d640 1 --2- 172.21.15.26:0/2262517784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd3c06afd0 0x7fcd3c156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd4176d640 1 -- 172.21.15.26:0/2262517784 >> 172.21.15.26:0/2262517784 conn(0x7fcd3c06d570 msgr2=0x7fcd3c0c6470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd4176d640 1 -- 172.21.15.26:0/2262517784 shutdown_connections 2023-12-19T16:42:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.188+0000 7fcd4176d640 1 -- 172.21.15.26:0/2262517784 wait complete. 2023-12-19T16:42:31.213 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:31.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa50a5a2640 1 Processor -- start 2023-12-19T16:42:31.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa50a5a2640 1 -- start start 2023-12-19T16:42:31.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa50a5a2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5040ada30 0x7fa5040c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:31.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa50a5a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa504164f00 0x7fa5040cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:31.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa50a5a2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa5040cb3a0 0x7fa5040affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:31.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa50a5a2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5040b16e0 con 0x7fa5040ada30 2023-12-19T16:42:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa50a5a2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5040cb810 con 0x7fa5040cb3a0 2023-12-19T16:42:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa50a5a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5040ade10 con 0x7fa504164f00 2023-12-19T16:42:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa5095a0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5040ada30 0x7fa5040c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa5095a0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa5040ada30 0x7fa5040c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49106/0 (socket says 172.21.15.26:49106) 2023-12-19T16:42:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa5095a0640 1 -- 172.21.15.26:0/267664071 learned_addr learned my addr 172.21.15.26:0/267664071 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa508d9f640 1 --2- 172.21.15.26:0/267664071 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa504164f00 0x7fa5040cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa509da1640 1 --2- 172.21.15.26:0/267664071 >> v2:172.21.15.160:3300/0 conn(0x7fa5040cb3a0 0x7fa5040affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa5095a0640 1 -- 172.21.15.26:0/267664071 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa504164f00 msgr2=0x7fa5040cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa5095a0640 1 --2- 172.21.15.26:0/267664071 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa504164f00 0x7fa5040cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa5095a0640 1 -- 172.21.15.26:0/267664071 >> v2:172.21.15.160:3300/0 conn(0x7fa5040cb3a0 msgr2=0x7fa5040affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa5095a0640 1 --2- 172.21.15.26:0/267664071 >> v2:172.21.15.160:3300/0 conn(0x7fa5040cb3a0 0x7fa5040affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.352+0000 7fa5095a0640 1 -- 172.21.15.26:0/267664071 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5040cc9d0 con 0x7fa5040ada30 2023-12-19T16:42:31.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.356+0000 7fa5095a0640 1 --2- 172.21.15.26:0/267664071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5040ada30 0x7fa5040c9860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa504071fe0 tx=0x7fa50001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67bc54f15c13e35 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.356+0000 7fa4fbfff640 1 -- 172.21.15.26:0/267664071 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5000ab600 con 0x7fa5040ada30 2023-12-19T16:42:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.356+0000 7fa4fbfff640 1 -- 172.21.15.26:0/267664071 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5000a7050 con 0x7fa5040ada30 2023-12-19T16:42:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.356+0000 7fa4fbfff640 1 -- 172.21.15.26:0/267664071 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5000ab980 con 0x7fa5040ada30 2023-12-19T16:42:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.356+0000 7fa50a5a2640 1 -- 172.21.15.26:0/267664071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5040ada30 msgr2=0x7fa5040c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.356+0000 7fa50a5a2640 1 --2- 172.21.15.26:0/267664071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5040ada30 0x7fa5040c9860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa504071fe0 tx=0x7fa50001c240 comp rx=0 tx=0).stop 2023-12-19T16:42:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.356+0000 7fa50a5a2640 1 -- 172.21.15.26:0/267664071 shutdown_connections 2023-12-19T16:42:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.356+0000 7fa50a5a2640 1 --2- 172.21.15.26:0/267664071 >> v2:172.21.15.160:3300/0 conn(0x7fa5040cb3a0 0x7fa5040affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.356+0000 7fa50a5a2640 1 --2- 172.21.15.26:0/267664071 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa504164f00 0x7fa5040cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.356+0000 7fa50a5a2640 1 --2- 172.21.15.26:0/267664071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5040ada30 0x7fa5040c9860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.356+0000 7fa50a5a2640 1 -- 172.21.15.26:0/267664071 >> 172.21.15.26:0/267664071 conn(0x7fa50406b0f0 msgr2=0x7fa50406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:31.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.356+0000 7fa50a5a2640 1 -- 172.21.15.26:0/267664071 shutdown_connections 2023-12-19T16:42:31.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.356+0000 7fa50a5a2640 1 -- 172.21.15.26:0/267664071 wait complete. 2023-12-19T16:42:31.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa50a5a2640 1 Processor -- start 2023-12-19T16:42:31.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa50a5a2640 1 -- start start 2023-12-19T16:42:31.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa50a5a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5040ada30 0x7fa504070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa50a5a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5040cb3a0 0x7fa5040b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa50a5a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa504164f00 0x7fa5040b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa50a5a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5041626c0 con 0x7fa5040ada30 2023-12-19T16:42:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa50a5a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5041629a0 con 0x7fa5040cb3a0 2023-12-19T16:42:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa50a5a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa504162c80 con 0x7fa504164f00 2023-12-19T16:42:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa5095a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5040ada30 0x7fa504070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa5095a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5040ada30 0x7fa504070340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49116/0 (socket says 172.21.15.26:49116) 2023-12-19T16:42:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa5095a0640 1 -- 172.21.15.26:0/207830474 learned_addr learned my addr 172.21.15.26:0/207830474 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa509da1640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa504164f00 0x7fa5040b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa508d9f640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5040cb3a0 0x7fa5040b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa5095a0640 1 -- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa504164f00 msgr2=0x7fa5040b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:31.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa5095a0640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa504164f00 0x7fa5040b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa5095a0640 1 -- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5040cb3a0 msgr2=0x7fa5040b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:31.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa5095a0640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5040cb3a0 0x7fa5040b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa5095a0640 1 -- 172.21.15.26:0/207830474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5000094d0 con 0x7fa5040ada30 2023-12-19T16:42:31.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa509da1640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa504164f00 0x7fa5040b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:31.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa508d9f640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5040cb3a0 0x7fa5040b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:31.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa5095a0640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5040ada30 0x7fa504070340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa50001a8f0 tx=0x7fa500018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:31.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa4f9ffb640 1 -- 172.21.15.26:0/207830474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5000b4840 con 0x7fa5040ada30 2023-12-19T16:42:31.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa4f9ffb640 1 -- 172.21.15.26:0/207830474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5000b49a0 con 0x7fa5040ada30 2023-12-19T16:42:31.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa50a5a2640 1 -- 172.21.15.26:0/207830474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa504163450 con 0x7fa5040ada30 2023-12-19T16:42:31.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa4f9ffb640 1 -- 172.21.15.26:0/207830474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5000b4d50 con 0x7fa5040ada30 2023-12-19T16:42:31.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.360+0000 7fa50a5a2640 1 -- 172.21.15.26:0/207830474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa504070880 con 0x7fa5040ada30 2023-12-19T16:42:31.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.364+0000 7fa50a5a2640 1 -- 172.21.15.26:0/207830474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa504067400 con 0x7fa5040ada30 2023-12-19T16:42:31.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.364+0000 7fa4f9ffb640 1 -- 172.21.15.26:0/207830474 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa5000b2040 con 0x7fa5040ada30 2023-12-19T16:42:31.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.364+0000 7fa4f9ffb640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4e8065a60 0x7fa4e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:31.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.364+0000 7fa4f9ffb640 1 -- 172.21.15.26:0/207830474 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa50011c860 con 0x7fa5040ada30 2023-12-19T16:42:31.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.364+0000 7fa508d9f640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4e8065a60 0x7fa4e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:31.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.364+0000 7fa508d9f640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4e8065a60 0x7fa4e8067f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5041fcea0 tx=0x7fa4f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:31.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.368+0000 7fa4f9ffb640 1 -- 172.21.15.26:0/207830474 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa5000ea180 con 0x7fa5040ada30 2023-12-19T16:42:31.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.528+0000 7fa50a5a2640 1 -- 172.21.15.26:0/207830474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5040cc2a0 con 0x7fa5040ada30 2023-12-19T16:42:31.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.528+0000 7fa4f9ffb640 1 -- 172.21.15.26:0/207830474 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fa5000ed7a0 con 0x7fa5040ada30 2023-12-19T16:42:31.534 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:31.534 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:31.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.532+0000 7fa50a5a2640 1 -- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4e8065a60 msgr2=0x7fa4e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:31.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.532+0000 7fa50a5a2640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4e8065a60 0x7fa4e8067f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5041fcea0 tx=0x7fa4f4008040 comp rx=0 tx=0).stop 2023-12-19T16:42:31.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.532+0000 7fa50a5a2640 1 -- 172.21.15.26:0/207830474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5040ada30 msgr2=0x7fa504070340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:31.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.532+0000 7fa50a5a2640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5040ada30 0x7fa504070340 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa50001a8f0 tx=0x7fa500018fe0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.532+0000 7fa50a5a2640 1 -- 172.21.15.26:0/207830474 shutdown_connections 2023-12-19T16:42:31.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.532+0000 7fa50a5a2640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4e8065a60 0x7fa4e8067f20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.532+0000 7fa50a5a2640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa504164f00 0x7fa5040b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.532+0000 7fa50a5a2640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5040cb3a0 0x7fa5040b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.532+0000 7fa50a5a2640 1 --2- 172.21.15.26:0/207830474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5040ada30 0x7fa504070340 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.532+0000 7fa50a5a2640 1 -- 172.21.15.26:0/207830474 >> 172.21.15.26:0/207830474 conn(0x7fa50406b0f0 msgr2=0x7fa5040c8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.532+0000 7fa50a5a2640 1 -- 172.21.15.26:0/207830474 shutdown_connections 2023-12-19T16:42:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.532+0000 7fa50a5a2640 1 -- 172.21.15.26:0/207830474 wait complete. 2023-12-19T16:42:31.555 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T16:42:31.556 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:42:31.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b3d6d640 1 Processor -- start 2023-12-19T16:42:31.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b3d6d640 1 -- start start 2023-12-19T16:42:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b3d6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9ac154220 0x7fd9ac15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b3d6d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd9ac0cee10 0x7fd9ac15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b3d6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ac06bc30 con 0x7fd9ac06b3d0 2023-12-19T16:42:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b3d6d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ac0cc6e0 con 0x7fd9ac154220 2023-12-19T16:42:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b3d6d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ac15afc0 con 0x7fd9ac0cee10 2023-12-19T16:42:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b1308640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd9ac06b3d0 0x7fd9ac0ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50236/0 (socket says 172.21.15.26:50236) 2023-12-19T16:42:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b1308640 1 -- 172.21.15.26:0/3705655264 learned_addr learned my addr 172.21.15.26:0/3705655264 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b1b09640 1 --2- 172.21.15.26:0/3705655264 >> v2:172.21.15.26:3300/0 conn(0x7fd9ac154220 0x7fd9ac15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b1b09640 1 -- 172.21.15.26:0/3705655264 >> v2:172.21.15.160:3300/0 conn(0x7fd9ac0cee10 msgr2=0x7fd9ac15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:31.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b1b09640 1 --2- 172.21.15.26:0/3705655264 >> v2:172.21.15.160:3300/0 conn(0x7fd9ac0cee10 0x7fd9ac15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b1b09640 1 -- 172.21.15.26:0/3705655264 >> v1:172.21.15.26:6789/0 conn(0x7fd9ac06b3d0 legacy=0x7fd9ac0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:31.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b1b09640 1 -- 172.21.15.26:0/3705655264 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac06b950 con 0x7fd9ac154220 2023-12-19T16:42:31.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.696+0000 7fd9b0b07640 1 -- 172.21.15.26:0/3705655264 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617724560 0 0) 0x7fd9ac06bc30 con 0x7fd9ac06b3d0 2023-12-19T16:42:31.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.700+0000 7fd9b1b09640 1 --2- 172.21.15.26:0/3705655264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac154220 0x7fd9ac15aa80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0cdf30 tx=0x7fd9a801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50efe1e925f2e1f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.700+0000 7fd9b0b07640 1 -- 172.21.15.26:0/3705655264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80ab650 con 0x7fd9ac154220 2023-12-19T16:42:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.700+0000 7fd9b0b07640 1 -- 172.21.15.26:0/3705655264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a80ab7b0 con 0x7fd9ac154220 2023-12-19T16:42:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.700+0000 7fd9b0b07640 1 -- 172.21.15.26:0/3705655264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a80abb40 con 0x7fd9ac154220 2023-12-19T16:42:31.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.700+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/3705655264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac154220 msgr2=0x7fd9ac15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:31.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.700+0000 7fd9b3d6d640 1 --2- 172.21.15.26:0/3705655264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac154220 0x7fd9ac15aa80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0cdf30 tx=0x7fd9a801c450 comp rx=0 tx=0).stop 2023-12-19T16:42:31.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.700+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/3705655264 shutdown_connections 2023-12-19T16:42:31.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.700+0000 7fd9b3d6d640 1 --2- 172.21.15.26:0/3705655264 >> v2:172.21.15.160:3300/0 conn(0x7fd9ac0cee10 0x7fd9ac15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.700+0000 7fd9b3d6d640 1 --2- 172.21.15.26:0/3705655264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac154220 0x7fd9ac15aa80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.700+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/3705655264 >> 172.21.15.26:0/3705655264 conn(0x7fd9ac0658e0 msgr2=0x7fd9ac065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:31.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.700+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/3705655264 shutdown_connections 2023-12-19T16:42:31.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.700+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/3705655264 wait complete. 2023-12-19T16:42:31.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b3d6d640 1 Processor -- start 2023-12-19T16:42:31.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b3d6d640 1 -- start start 2023-12-19T16:42:31.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b3d6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9ac06b3d0 0x7fd9ac151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:31.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b3d6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9ac0cee10 0x7fd9ac145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:31.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b3d6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac154220 0x7fd9ac14e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:31.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b3d6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac0712a0 con 0x7fd9ac154220 2023-12-19T16:42:31.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b3d6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac070c70 con 0x7fd9ac0cee10 2023-12-19T16:42:31.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b3d6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac148100 con 0x7fd9ac06b3d0 2023-12-19T16:42:31.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b230a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac154220 0x7fd9ac14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:31.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b230a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac154220 0x7fd9ac14e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49146/0 (socket says 172.21.15.26:49146) 2023-12-19T16:42:31.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b230a640 1 -- 172.21.15.26:0/1451741103 learned_addr learned my addr 172.21.15.26:0/1451741103 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:31.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b1b09640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9ac06b3d0 0x7fd9ac151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:31.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b1308640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9ac0cee10 0x7fd9ac145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:31.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b230a640 1 -- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9ac06b3d0 msgr2=0x7fd9ac151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:31.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b230a640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9ac06b3d0 0x7fd9ac151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b230a640 1 -- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9ac0cee10 msgr2=0x7fd9ac145fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:31.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b230a640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9ac0cee10 0x7fd9ac145fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b230a640 1 -- 172.21.15.26:0/1451741103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a8009420 con 0x7fd9ac154220 2023-12-19T16:42:31.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b230a640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac154220 0x7fd9ac14e160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400b9e0 tx=0x7fd9a400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:31.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9a27fc640 1 -- 172.21.15.26:0/1451741103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a400cb20 con 0x7fd9ac154220 2023-12-19T16:42:31.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9a27fc640 1 -- 172.21.15.26:0/1451741103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a400cca0 con 0x7fd9ac154220 2023-12-19T16:42:31.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9a27fc640 1 -- 172.21.15.26:0/1451741103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4014680 con 0x7fd9ac154220 2023-12-19T16:42:31.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/1451741103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ac150540 con 0x7fd9ac154220 2023-12-19T16:42:31.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/1451741103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ac070ec0 con 0x7fd9ac154220 2023-12-19T16:42:31.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.704+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/1451741103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ac0715c0 con 0x7fd9ac154220 2023-12-19T16:42:31.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.708+0000 7fd9a27fc640 1 -- 172.21.15.26:0/1451741103 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd9a401d3b0 con 0x7fd9ac154220 2023-12-19T16:42:31.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.708+0000 7fd9a27fc640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd990065a60 0x7fd990067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:31.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.708+0000 7fd9a27fc640 1 -- 172.21.15.26:0/1451741103 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9a4083070 con 0x7fd9ac154220 2023-12-19T16:42:31.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.708+0000 7fd9b1b09640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd990065a60 0x7fd990067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:31.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.708+0000 7fd9b1b09640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd990065a60 0x7fd990067f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80af000 tx=0x7fd9a80a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:31.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.712+0000 7fd9a27fc640 1 -- 172.21.15.26:0/1451741103 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd9a40b40a0 con 0x7fd9ac154220 2023-12-19T16:42:31.867 INFO:tasks.ceph.mds.d.smithi160.stderr:2023-12-19T16:42:31.862+0000 7f0fa7506640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T16:42:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.872+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/1451741103 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9ac0ccef0 con 0x7fd990065a60 2023-12-19T16:42:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.880+0000 7fd9a27fc640 1 -- 172.21.15.26:0/1451741103 <== mgr.4106 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264547 (secure 0 0 0) 0x7fd9a80abaa0 con 0x7fd990065a60 2023-12-19T16:42:31.889 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:31.891 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:42:31.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.884+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd990065a60 msgr2=0x7fd990067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:31.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.884+0000 7fd9b3d6d640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd990065a60 0x7fd990067f20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd9a80af000 tx=0x7fd9a80a7000 comp rx=0 tx=0).stop 2023-12-19T16:42:31.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.884+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/1451741103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac154220 msgr2=0x7fd9ac14e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:31.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.884+0000 7fd9b3d6d640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac154220 0x7fd9ac14e160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400b9e0 tx=0x7fd9a400bea0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.884+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/1451741103 shutdown_connections 2023-12-19T16:42:31.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.884+0000 7fd9b3d6d640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ac154220 0x7fd9ac14e160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.884+0000 7fd9b3d6d640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9ac0cee10 0x7fd9ac145fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.884+0000 7fd9b3d6d640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd990065a60 0x7fd990067f20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.884+0000 7fd9b3d6d640 1 --2- 172.21.15.26:0/1451741103 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9ac06b3d0 0x7fd9ac151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:31.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.884+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/1451741103 >> 172.21.15.26:0/1451741103 conn(0x7fd9ac0658e0 msgr2=0x7fd9ac0ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:31.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.884+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/1451741103 shutdown_connections 2023-12-19T16:42:31.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:31.884+0000 7fd9b3d6d640 1 -- 172.21.15.26:0/1451741103 wait complete. 2023-12-19T16:42:31.907 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-19T16:42:31.636952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234964,"kb_used_data":2964,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739756,"statfs":{"total":773094113280,"available":772853510144,"internally_reserved":0,"allocated":3035136,"data_stored":1793835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.347344"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830677+0000","last_change":"2023-12-19T16:42:27.824402+0000","last_active":"2023-12-19T16:42:27.830677+0000","last_peered":"2023-12-19T16:42:27.830677+0000","last_clean":"2023-12-19T16:42:27.830677+0000","last_became_active":"2023-12-19T16:42:27.818342+0000","last_became_peered":"2023-12-19T16:42:27.818342+0000","last_unstale":"2023-12-19T16:42:27.830677+0000","last_undegraded":"2023-12-19T16:42:27.830677+0000","last_fullsized":"2023-12-19T16:42:27.830677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:53.498191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.956000+0000","last_change":"2023-12-19T16:42:27.802326+0000","last_active":"2023-12-19T16:42:28.956000+0000","last_peered":"2023-12-19T16:42:28.956000+0000","last_clean":"2023-12-19T16:42:28.956000+0000","last_became_active":"2023-12-19T16:42:27.802056+0000","last_became_peered":"2023-12-19T16:42:27.802056+0000","last_unstale":"2023-12-19T16:42:28.956000+0000","last_undegraded":"2023-12-19T16:42:28.956000+0000","last_fullsized":"2023-12-19T16:42:28.956000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:30.545767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.807788+0000","last_change":"2023-12-19T16:42:27.798150+0000","last_active":"2023-12-19T16:42:27.807788+0000","last_peered":"2023-12-19T16:42:27.807788+0000","last_clean":"2023-12-19T16:42:27.807788+0000","last_became_active":"2023-12-19T16:42:27.796339+0000","last_became_peered":"2023-12-19T16:42:27.796339+0000","last_unstale":"2023-12-19T16:42:27.807788+0000","last_undegraded":"2023-12-19T16:42:27.807788+0000","last_fullsized":"2023-12-19T16:42:27.807788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:42:19.277925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790632+0000","last_change":"2023-12-19T16:42:27.814794+0000","last_active":"2023-12-19T16:42:28.790632+0000","last_peered":"2023-12-19T16:42:28.790632+0000","last_clean":"2023-12-19T16:42:28.790632+0000","last_became_active":"2023-12-19T16:42:27.814461+0000","last_became_peered":"2023-12-19T16:42:27.814461+0000","last_unstale":"2023-12-19T16:42:28.790632+0000","last_undegraded":"2023-12-19T16:42:28.790632+0000","last_fullsized":"2023-12-19T16:42:28.790632+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:18.209972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830892+0000","last_change":"2023-12-19T16:42:27.825817+0000","last_active":"2023-12-19T16:42:27.830892+0000","last_peered":"2023-12-19T16:42:27.830892+0000","last_clean":"2023-12-19T16:42:27.830892+0000","last_became_active":"2023-12-19T16:42:27.825532+0000","last_became_peered":"2023-12-19T16:42:27.825532+0000","last_unstale":"2023-12-19T16:42:27.830892+0000","last_undegraded":"2023-12-19T16:42:27.830892+0000","last_fullsized":"2023-12-19T16:42:27.830892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:49.764257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790441+0000","last_change":"2023-12-19T16:42:27.801183+0000","last_active":"2023-12-19T16:42:28.790441+0000","last_peered":"2023-12-19T16:42:28.790441+0000","last_clean":"2023-12-19T16:42:28.790441+0000","last_became_active":"2023-12-19T16:42:27.800866+0000","last_became_peered":"2023-12-19T16:42:27.800866+0000","last_unstale":"2023-12-19T16:42:28.790441+0000","last_undegraded":"2023-12-19T16:42:28.790441+0000","last_fullsized":"2023-12-19T16:42:28.790441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:36.473614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.827704+0000","last_change":"2023-12-19T16:42:27.817242+0000","last_active":"2023-12-19T16:42:27.827704+0000","last_peered":"2023-12-19T16:42:27.827704+0000","last_clean":"2023-12-19T16:42:27.827704+0000","last_became_active":"2023-12-19T16:42:27.816801+0000","last_became_peered":"2023-12-19T16:42:27.816801+0000","last_unstale":"2023-12-19T16:42:27.827704+0000","last_undegraded":"2023-12-19T16:42:27.827704+0000","last_fullsized":"2023-12-19T16:42:27.827704+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.813815+0000","last_change":"2023-12-19T16:42:27.801939+0000","last_active":"2023-12-19T16:42:27.813815+0000","last_peered":"2023-12-19T16:42:27.813815+0000","last_clean":"2023-12-19T16:42:27.813815+0000","last_became_active":"2023-12-19T16:42:27.801672+0000","last_became_peered":"2023-12-19T16:42:27.801672+0000","last_unstale":"2023-12-19T16:42:27.813815+0000","last_undegraded":"2023-12-19T16:42:27.813815+0000","last_fullsized":"2023-12-19T16:42:27.813815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.831759+0000","last_change":"2023-12-19T16:42:27.829667+0000","last_active":"2023-12-19T16:42:27.831759+0000","last_peered":"2023-12-19T16:42:27.831759+0000","last_clean":"2023-12-19T16:42:27.831759+0000","last_became_active":"2023-12-19T16:42:27.829357+0000","last_became_peered":"2023-12-19T16:42:27.829357+0000","last_unstale":"2023-12-19T16:42:27.831759+0000","last_undegraded":"2023-12-19T16:42:27.831759+0000","last_fullsized":"2023-12-19T16:42:27.831759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:06.914435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.816645+0000","last_change":"2023-12-19T16:42:27.812001+0000","last_active":"2023-12-19T16:42:27.816645+0000","last_peered":"2023-12-19T16:42:27.816645+0000","last_clean":"2023-12-19T16:42:27.816645+0000","last_became_active":"2023-12-19T16:42:27.811749+0000","last_became_peered":"2023-12-19T16:42:27.811749+0000","last_unstale":"2023-12-19T16:42:27.816645+0000","last_undegraded":"2023-12-19T16:42:27.816645+0000","last_fullsized":"2023-12-19T16:42:27.816645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:29.376069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.818425+0000","last_change":"2023-12-19T16:42:27.809484+0000","last_active":"2023-12-19T16:42:27.818425+0000","last_peered":"2023-12-19T16:42:27.818425+0000","last_clean":"2023-12-19T16:42:27.818425+0000","last_became_active":"2023-12-19T16:42:27.809189+0000","last_became_peered":"2023-12-19T16:42:27.809189+0000","last_unstale":"2023-12-19T16:42:27.818425+0000","last_undegraded":"2023-12-19T16:42:27.818425+0000","last_fullsized":"2023-12-19T16:42:27.818425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:03.390272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789997+0000","last_change":"2023-12-19T16:42:27.812088+0000","last_active":"2023-12-19T16:42:28.789997+0000","last_peered":"2023-12-19T16:42:28.789997+0000","last_clean":"2023-12-19T16:42:28.789997+0000","last_became_active":"2023-12-19T16:42:27.811768+0000","last_became_peered":"2023-12-19T16:42:27.811768+0000","last_unstale":"2023-12-19T16:42:28.789997+0000","last_undegraded":"2023-12-19T16:42:28.789997+0000","last_fullsized":"2023-12-19T16:42:28.789997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:47:07.567597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.814597+0000","last_change":"2023-12-19T16:42:27.805169+0000","last_active":"2023-12-19T16:42:27.814597+0000","last_peered":"2023-12-19T16:42:27.814597+0000","last_clean":"2023-12-19T16:42:27.814597+0000","last_became_active":"2023-12-19T16:42:27.804906+0000","last_became_peered":"2023-12-19T16:42:27.804906+0000","last_unstale":"2023-12-19T16:42:27.814597+0000","last_undegraded":"2023-12-19T16:42:27.814597+0000","last_fullsized":"2023-12-19T16:42:27.814597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:57:49.461118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.808101+0000","last_change":"2023-12-19T16:42:27.796100+0000","last_active":"2023-12-19T16:42:27.808101+0000","last_peered":"2023-12-19T16:42:27.808101+0000","last_clean":"2023-12-19T16:42:27.808101+0000","last_became_active":"2023-12-19T16:42:27.795802+0000","last_became_peered":"2023-12-19T16:42:27.795802+0000","last_unstale":"2023-12-19T16:42:27.808101+0000","last_undegraded":"2023-12-19T16:42:27.808101+0000","last_fullsized":"2023-12-19T16:42:27.808101+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:30:13.550890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955716+0000","last_change":"2023-12-19T16:42:27.815496+0000","last_active":"2023-12-19T16:42:28.955716+0000","last_peered":"2023-12-19T16:42:28.955716+0000","last_clean":"2023-12-19T16:42:28.955716+0000","last_became_active":"2023-12-19T16:42:27.812535+0000","last_became_peered":"2023-12-19T16:42:27.812535+0000","last_unstale":"2023-12-19T16:42:28.955716+0000","last_undegraded":"2023-12-19T16:42:28.955716+0000","last_fullsized":"2023-12-19T16:42:28.955716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:37.400723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790179+0000","last_change":"2023-12-19T16:42:27.804674+0000","last_active":"2023-12-19T16:42:28.790179+0000","last_peered":"2023-12-19T16:42:28.790179+0000","last_clean":"2023-12-19T16:42:28.790179+0000","last_became_active":"2023-12-19T16:42:27.804057+0000","last_became_peered":"2023-12-19T16:42:27.804057+0000","last_unstale":"2023-12-19T16:42:28.790179+0000","last_undegraded":"2023-12-19T16:42:28.790179+0000","last_fullsized":"2023-12-19T16:42:28.790179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:57:49.461118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790884+0000","last_change":"2023-12-19T16:42:27.829970+0000","last_active":"2023-12-19T16:42:28.790884+0000","last_peered":"2023-12-19T16:42:28.790884+0000","last_clean":"2023-12-19T16:42:28.790884+0000","last_became_active":"2023-12-19T16:42:27.829616+0000","last_became_peered":"2023-12-19T16:42:27.829616+0000","last_unstale":"2023-12-19T16:42:28.790884+0000","last_undegraded":"2023-12-19T16:42:28.790884+0000","last_fullsized":"2023-12-19T16:42:28.790884+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:50.303035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784083+0000","last_change":"2023-12-19T16:42:27.804309+0000","last_active":"2023-12-19T16:42:28.784083+0000","last_peered":"2023-12-19T16:42:28.784083+0000","last_clean":"2023-12-19T16:42:28.784083+0000","last_became_active":"2023-12-19T16:42:27.803973+0000","last_became_peered":"2023-12-19T16:42:27.803973+0000","last_unstale":"2023-12-19T16:42:28.784083+0000","last_undegraded":"2023-12-19T16:42:28.784083+0000","last_fullsized":"2023-12-19T16:42:28.784083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:47:07.567597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.831180+0000","last_change":"2023-12-19T16:42:27.827321+0000","last_active":"2023-12-19T16:42:27.831180+0000","last_peered":"2023-12-19T16:42:27.831180+0000","last_clean":"2023-12-19T16:42:27.831180+0000","last_became_active":"2023-12-19T16:42:27.826329+0000","last_became_peered":"2023-12-19T16:42:27.826329+0000","last_unstale":"2023-12-19T16:42:27.831180+0000","last_undegraded":"2023-12-19T16:42:27.831180+0000","last_fullsized":"2023-12-19T16:42:27.831180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:01.122709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.822288+0000","last_change":"2023-12-19T16:42:27.816854+0000","last_active":"2023-12-19T16:42:27.822288+0000","last_peered":"2023-12-19T16:42:27.822288+0000","last_clean":"2023-12-19T16:42:27.822288+0000","last_became_active":"2023-12-19T16:42:27.816538+0000","last_became_peered":"2023-12-19T16:42:27.816538+0000","last_unstale":"2023-12-19T16:42:27.822288+0000","last_undegraded":"2023-12-19T16:42:27.822288+0000","last_fullsized":"2023-12-19T16:42:27.822288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:30.545767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.819162+0000","last_change":"2023-12-19T16:42:27.817144+0000","last_active":"2023-12-19T16:42:27.819162+0000","last_peered":"2023-12-19T16:42:27.819162+0000","last_clean":"2023-12-19T16:42:27.819162+0000","last_became_active":"2023-12-19T16:42:27.816891+0000","last_became_peered":"2023-12-19T16:42:27.816891+0000","last_unstale":"2023-12-19T16:42:27.819162+0000","last_undegraded":"2023-12-19T16:42:27.819162+0000","last_fullsized":"2023-12-19T16:42:27.819162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:51.134371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790438+0000","last_change":"2023-12-19T16:42:27.826273+0000","last_active":"2023-12-19T16:42:28.790438+0000","last_peered":"2023-12-19T16:42:28.790438+0000","last_clean":"2023-12-19T16:42:28.790438+0000","last_became_active":"2023-12-19T16:42:27.825992+0000","last_became_peered":"2023-12-19T16:42:27.825992+0000","last_unstale":"2023-12-19T16:42:28.790438+0000","last_undegraded":"2023-12-19T16:42:28.790438+0000","last_fullsized":"2023-12-19T16:42:28.790438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:36.031318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.817466+0000","last_change":"2023-12-19T16:42:27.813323+0000","last_active":"2023-12-19T16:42:27.817466+0000","last_peered":"2023-12-19T16:42:27.817466+0000","last_clean":"2023-12-19T16:42:27.817466+0000","last_became_active":"2023-12-19T16:42:27.812808+0000","last_became_peered":"2023-12-19T16:42:27.812808+0000","last_unstale":"2023-12-19T16:42:27.817466+0000","last_undegraded":"2023-12-19T16:42:27.817466+0000","last_fullsized":"2023-12-19T16:42:27.817466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:36.473614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790317+0000","last_change":"2023-12-19T16:42:27.810294+0000","last_active":"2023-12-19T16:42:28.790317+0000","last_peered":"2023-12-19T16:42:28.790317+0000","last_clean":"2023-12-19T16:42:28.790317+0000","last_became_active":"2023-12-19T16:42:27.810031+0000","last_became_peered":"2023-12-19T16:42:27.810031+0000","last_unstale":"2023-12-19T16:42:28.790317+0000","last_undegraded":"2023-12-19T16:42:28.790317+0000","last_fullsized":"2023-12-19T16:42:28.790317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:01.122709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.818055+0000","last_change":"2023-12-19T16:42:27.813230+0000","last_active":"2023-12-19T16:42:27.818055+0000","last_peered":"2023-12-19T16:42:27.818055+0000","last_clean":"2023-12-19T16:42:27.818055+0000","last_became_active":"2023-12-19T16:42:27.812914+0000","last_became_peered":"2023-12-19T16:42:27.812914+0000","last_unstale":"2023-12-19T16:42:27.818055+0000","last_undegraded":"2023-12-19T16:42:27.818055+0000","last_fullsized":"2023-12-19T16:42:27.818055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:50.303035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.823553+0000","last_change":"2023-12-19T16:42:27.819653+0000","last_active":"2023-12-19T16:42:27.823553+0000","last_peered":"2023-12-19T16:42:27.823553+0000","last_clean":"2023-12-19T16:42:27.823553+0000","last_became_active":"2023-12-19T16:42:27.819325+0000","last_became_peered":"2023-12-19T16:42:27.819325+0000","last_unstale":"2023-12-19T16:42:27.823553+0000","last_undegraded":"2023-12-19T16:42:27.823553+0000","last_fullsized":"2023-12-19T16:42:27.823553+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:40:46.397053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955842+0000","last_change":"2023-12-19T16:42:27.813010+0000","last_active":"2023-12-19T16:42:28.955842+0000","last_peered":"2023-12-19T16:42:28.955842+0000","last_clean":"2023-12-19T16:42:28.955842+0000","last_became_active":"2023-12-19T16:42:27.812336+0000","last_became_peered":"2023-12-19T16:42:27.812336+0000","last_unstale":"2023-12-19T16:42:28.955842+0000","last_undegraded":"2023-12-19T16:42:28.955842+0000","last_fullsized":"2023-12-19T16:42:28.955842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.818940+0000","last_change":"2023-12-19T16:42:27.813584+0000","last_active":"2023-12-19T16:42:27.818940+0000","last_peered":"2023-12-19T16:42:27.818940+0000","last_clean":"2023-12-19T16:42:27.818940+0000","last_became_active":"2023-12-19T16:42:27.813276+0000","last_became_peered":"2023-12-19T16:42:27.813276+0000","last_unstale":"2023-12-19T16:42:27.818940+0000","last_undegraded":"2023-12-19T16:42:27.818940+0000","last_fullsized":"2023-12-19T16:42:27.818940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:36.031318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.811791+0000","last_change":"2023-12-19T16:42:27.805390+0000","last_active":"2023-12-19T16:42:27.811791+0000","last_peered":"2023-12-19T16:42:27.811791+0000","last_clean":"2023-12-19T16:42:27.811791+0000","last_became_active":"2023-12-19T16:42:27.804956+0000","last_became_peered":"2023-12-19T16:42:27.804956+0000","last_unstale":"2023-12-19T16:42:27.811791+0000","last_undegraded":"2023-12-19T16:42:27.811791+0000","last_fullsized":"2023-12-19T16:42:27.811791+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:58.680381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.791132+0000","last_change":"2023-12-19T16:42:27.809669+0000","last_active":"2023-12-19T16:42:28.791132+0000","last_peered":"2023-12-19T16:42:28.791132+0000","last_clean":"2023-12-19T16:42:28.791132+0000","last_became_active":"2023-12-19T16:42:27.809374+0000","last_became_peered":"2023-12-19T16:42:27.809374+0000","last_unstale":"2023-12-19T16:42:28.791132+0000","last_undegraded":"2023-12-19T16:42:28.791132+0000","last_fullsized":"2023-12-19T16:42:28.791132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:53.498191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784039+0000","last_change":"2023-12-19T16:42:27.799272+0000","last_active":"2023-12-19T16:42:28.784039+0000","last_peered":"2023-12-19T16:42:28.784039+0000","last_clean":"2023-12-19T16:42:28.784039+0000","last_became_active":"2023-12-19T16:42:27.799006+0000","last_became_peered":"2023-12-19T16:42:27.799006+0000","last_unstale":"2023-12-19T16:42:28.784039+0000","last_undegraded":"2023-12-19T16:42:28.784039+0000","last_fullsized":"2023-12-19T16:42:28.784039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:37.286916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.816353+0000","last_change":"2023-12-19T16:42:27.809714+0000","last_active":"2023-12-19T16:42:27.816353+0000","last_peered":"2023-12-19T16:42:27.816353+0000","last_clean":"2023-12-19T16:42:27.816353+0000","last_became_active":"2023-12-19T16:42:27.809384+0000","last_became_peered":"2023-12-19T16:42:27.809384+0000","last_unstale":"2023-12-19T16:42:27.816353+0000","last_undegraded":"2023-12-19T16:42:27.816353+0000","last_fullsized":"2023-12-19T16:42:27.816353+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:37.400723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789634+0000","last_change":"2023-12-19T16:42:27.811964+0000","last_active":"2023-12-19T16:42:28.789634+0000","last_peered":"2023-12-19T16:42:28.789634+0000","last_clean":"2023-12-19T16:42:28.789634+0000","last_became_active":"2023-12-19T16:42:27.811777+0000","last_became_peered":"2023-12-19T16:42:27.811777+0000","last_unstale":"2023-12-19T16:42:28.789634+0000","last_undegraded":"2023-12-19T16:42:28.789634+0000","last_fullsized":"2023-12-19T16:42:28.789634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:50.714712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.825802+0000","last_change":"2023-12-19T16:42:27.324768+0000","last_active":"2023-12-19T16:42:27.825802+0000","last_peered":"2023-12-19T16:42:27.825802+0000","last_clean":"2023-12-19T16:42:27.825802+0000","last_became_active":"2023-12-19T16:42:27.324380+0000","last_became_peered":"2023-12-19T16:42:27.324380+0000","last_unstale":"2023-12-19T16:42:27.825802+0000","last_undegraded":"2023-12-19T16:42:27.825802+0000","last_fullsized":"2023-12-19T16:42:27.825802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.819143+0000","last_change":"2023-12-19T16:42:27.816607+0000","last_active":"2023-12-19T16:42:27.819143+0000","last_peered":"2023-12-19T16:42:27.819143+0000","last_clean":"2023-12-19T16:42:27.819143+0000","last_became_active":"2023-12-19T16:42:27.815987+0000","last_became_peered":"2023-12-19T16:42:27.815987+0000","last_unstale":"2023-12-19T16:42:27.819143+0000","last_undegraded":"2023-12-19T16:42:27.819143+0000","last_fullsized":"2023-12-19T16:42:27.819143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:32:26.852410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.816394+0000","last_change":"2023-12-19T16:42:27.327298+0000","last_active":"2023-12-19T16:42:27.816394+0000","last_peered":"2023-12-19T16:42:27.816394+0000","last_clean":"2023-12-19T16:42:27.816394+0000","last_became_active":"2023-12-19T16:42:27.326988+0000","last_became_peered":"2023-12-19T16:42:27.326988+0000","last_unstale":"2023-12-19T16:42:27.816394+0000","last_undegraded":"2023-12-19T16:42:27.816394+0000","last_fullsized":"2023-12-19T16:42:27.816394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.791085+0000","last_change":"2023-12-19T16:42:27.320910+0000","last_active":"2023-12-19T16:42:27.791085+0000","last_peered":"2023-12-19T16:42:27.791085+0000","last_clean":"2023-12-19T16:42:27.791085+0000","last_became_active":"2023-12-19T16:42:27.320629+0000","last_became_peered":"2023-12-19T16:42:27.320629+0000","last_unstale":"2023-12-19T16:42:27.791085+0000","last_undegraded":"2023-12-19T16:42:27.791085+0000","last_fullsized":"2023-12-19T16:42:27.791085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.814014+0000","last_change":"2023-12-19T16:42:27.803836+0000","last_active":"2023-12-19T16:42:27.814014+0000","last_peered":"2023-12-19T16:42:27.814014+0000","last_clean":"2023-12-19T16:42:27.814014+0000","last_became_active":"2023-12-19T16:42:27.803527+0000","last_became_peered":"2023-12-19T16:42:27.803527+0000","last_unstale":"2023-12-19T16:42:27.814014+0000","last_undegraded":"2023-12-19T16:42:27.814014+0000","last_fullsized":"2023-12-19T16:42:27.814014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:37.400723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790294+0000","last_change":"2023-12-19T16:42:27.322383+0000","last_active":"2023-12-19T16:42:28.790294+0000","last_peered":"2023-12-19T16:42:28.790294+0000","last_clean":"2023-12-19T16:42:28.790294+0000","last_became_active":"2023-12-19T16:42:27.321439+0000","last_became_peered":"2023-12-19T16:42:27.321439+0000","last_unstale":"2023-12-19T16:42:28.790294+0000","last_undegraded":"2023-12-19T16:42:28.790294+0000","last_fullsized":"2023-12-19T16:42:28.790294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955715+0000","last_change":"2023-12-19T16:42:27.800626+0000","last_active":"2023-12-19T16:42:28.955715+0000","last_peered":"2023-12-19T16:42:28.955715+0000","last_clean":"2023-12-19T16:42:28.955715+0000","last_became_active":"2023-12-19T16:42:27.800338+0000","last_became_peered":"2023-12-19T16:42:27.800338+0000","last_unstale":"2023-12-19T16:42:28.955715+0000","last_undegraded":"2023-12-19T16:42:28.955715+0000","last_fullsized":"2023-12-19T16:42:28.955715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:29.376069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.797659+0000","last_change":"2023-12-19T16:42:27.324962+0000","last_active":"2023-12-19T16:42:27.797659+0000","last_peered":"2023-12-19T16:42:27.797659+0000","last_clean":"2023-12-19T16:42:27.797659+0000","last_became_active":"2023-12-19T16:42:27.324627+0000","last_became_peered":"2023-12-19T16:42:27.324627+0000","last_unstale":"2023-12-19T16:42:27.797659+0000","last_undegraded":"2023-12-19T16:42:27.797659+0000","last_fullsized":"2023-12-19T16:42:27.797659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790825+0000","last_change":"2023-12-19T16:42:27.829894+0000","last_active":"2023-12-19T16:42:28.790825+0000","last_peered":"2023-12-19T16:42:28.790825+0000","last_clean":"2023-12-19T16:42:28.790825+0000","last_became_active":"2023-12-19T16:42:27.829581+0000","last_became_peered":"2023-12-19T16:42:27.829581+0000","last_unstale":"2023-12-19T16:42:28.790825+0000","last_undegraded":"2023-12-19T16:42:28.790825+0000","last_fullsized":"2023-12-19T16:42:28.790825+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:00.499277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790905+0000","last_change":"2023-12-19T16:42:27.322335+0000","last_active":"2023-12-19T16:42:28.790905+0000","last_peered":"2023-12-19T16:42:28.790905+0000","last_clean":"2023-12-19T16:42:28.790905+0000","last_became_active":"2023-12-19T16:42:27.319501+0000","last_became_peered":"2023-12-19T16:42:27.319501+0000","last_unstale":"2023-12-19T16:42:28.790905+0000","last_undegraded":"2023-12-19T16:42:28.790905+0000","last_fullsized":"2023-12-19T16:42:28.790905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.808144+0000","last_change":"2023-12-19T16:42:27.798069+0000","last_active":"2023-12-19T16:42:27.808144+0000","last_peered":"2023-12-19T16:42:27.808144+0000","last_clean":"2023-12-19T16:42:27.808144+0000","last_became_active":"2023-12-19T16:42:27.795622+0000","last_became_peered":"2023-12-19T16:42:27.795622+0000","last_unstale":"2023-12-19T16:42:27.808144+0000","last_undegraded":"2023-12-19T16:42:27.808144+0000","last_fullsized":"2023-12-19T16:42:27.808144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:15:14.536450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955857+0000","last_change":"2023-12-19T16:42:27.310215+0000","last_active":"2023-12-19T16:42:28.955857+0000","last_peered":"2023-12-19T16:42:28.955857+0000","last_clean":"2023-12-19T16:42:28.955857+0000","last_became_active":"2023-12-19T16:42:27.308663+0000","last_became_peered":"2023-12-19T16:42:27.308663+0000","last_unstale":"2023-12-19T16:42:28.955857+0000","last_undegraded":"2023-12-19T16:42:28.955857+0000","last_fullsized":"2023-12-19T16:42:28.955857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:41.116483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790149+0000","last_change":"2023-12-19T16:42:27.825472+0000","last_active":"2023-12-19T16:42:28.790149+0000","last_peered":"2023-12-19T16:42:28.790149+0000","last_clean":"2023-12-19T16:42:28.790149+0000","last_became_active":"2023-12-19T16:42:27.825201+0000","last_became_peered":"2023-12-19T16:42:27.825201+0000","last_unstale":"2023-12-19T16:42:28.790149+0000","last_undegraded":"2023-12-19T16:42:28.790149+0000","last_fullsized":"2023-12-19T16:42:28.790149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:42.122812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.828695+0000","last_change":"2023-12-19T16:42:27.321076+0000","last_active":"2023-12-19T16:42:27.828695+0000","last_peered":"2023-12-19T16:42:27.828695+0000","last_clean":"2023-12-19T16:42:27.828695+0000","last_became_active":"2023-12-19T16:42:27.320592+0000","last_became_peered":"2023-12-19T16:42:27.320592+0000","last_unstale":"2023-12-19T16:42:27.828695+0000","last_undegraded":"2023-12-19T16:42:27.828695+0000","last_fullsized":"2023-12-19T16:42:27.828695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.708383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.827744+0000","last_change":"2023-12-19T16:42:27.818972+0000","last_active":"2023-12-19T16:42:27.827744+0000","last_peered":"2023-12-19T16:42:27.827744+0000","last_clean":"2023-12-19T16:42:27.827744+0000","last_became_active":"2023-12-19T16:42:27.818689+0000","last_became_peered":"2023-12-19T16:42:27.818689+0000","last_unstale":"2023-12-19T16:42:27.827744+0000","last_undegraded":"2023-12-19T16:42:27.827744+0000","last_fullsized":"2023-12-19T16:42:27.827744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:48:49.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.823238+0000","last_change":"2023-12-19T16:42:27.306416+0000","last_active":"2023-12-19T16:42:27.823238+0000","last_peered":"2023-12-19T16:42:27.823238+0000","last_clean":"2023-12-19T16:42:27.823238+0000","last_became_active":"2023-12-19T16:42:27.306033+0000","last_became_peered":"2023-12-19T16:42:27.306033+0000","last_unstale":"2023-12-19T16:42:27.823238+0000","last_undegraded":"2023-12-19T16:42:27.823238+0000","last_fullsized":"2023-12-19T16:42:27.823238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:29.172833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.813217+0000","last_change":"2023-12-19T16:42:27.809290+0000","last_active":"2023-12-19T16:42:27.813217+0000","last_peered":"2023-12-19T16:42:27.813217+0000","last_clean":"2023-12-19T16:42:27.813217+0000","last_became_active":"2023-12-19T16:42:27.806028+0000","last_became_peered":"2023-12-19T16:42:27.806028+0000","last_unstale":"2023-12-19T16:42:27.813217+0000","last_undegraded":"2023-12-19T16:42:27.813217+0000","last_fullsized":"2023-12-19T16:42:27.813217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:48:43.490883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.791044+0000","last_change":"2023-12-19T16:42:27.317606+0000","last_active":"2023-12-19T16:42:28.791044+0000","last_peered":"2023-12-19T16:42:28.791044+0000","last_clean":"2023-12-19T16:42:28.791044+0000","last_became_active":"2023-12-19T16:42:27.316871+0000","last_became_peered":"2023-12-19T16:42:27.316871+0000","last_unstale":"2023-12-19T16:42:28.791044+0000","last_undegraded":"2023-12-19T16:42:28.791044+0000","last_fullsized":"2023-12-19T16:42:28.791044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.708383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830963+0000","last_change":"2023-12-19T16:42:27.826100+0000","last_active":"2023-12-19T16:42:27.830963+0000","last_peered":"2023-12-19T16:42:27.830963+0000","last_clean":"2023-12-19T16:42:27.830963+0000","last_became_active":"2023-12-19T16:42:27.825764+0000","last_became_peered":"2023-12-19T16:42:27.825764+0000","last_unstale":"2023-12-19T16:42:27.830963+0000","last_undegraded":"2023-12-19T16:42:27.830963+0000","last_fullsized":"2023-12-19T16:42:27.830963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:00.499277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.797604+0000","last_change":"2023-12-19T16:42:27.327321+0000","last_active":"2023-12-19T16:42:27.797604+0000","last_peered":"2023-12-19T16:42:27.797604+0000","last_clean":"2023-12-19T16:42:27.797604+0000","last_became_active":"2023-12-19T16:42:27.327010+0000","last_became_peered":"2023-12-19T16:42:27.327010+0000","last_unstale":"2023-12-19T16:42:27.797604+0000","last_undegraded":"2023-12-19T16:42:27.797604+0000","last_fullsized":"2023-12-19T16:42:27.797604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.810813+0000","last_change":"2023-12-19T16:42:27.802521+0000","last_active":"2023-12-19T16:42:27.810813+0000","last_peered":"2023-12-19T16:42:27.810813+0000","last_clean":"2023-12-19T16:42:27.810813+0000","last_became_active":"2023-12-19T16:42:27.801786+0000","last_became_peered":"2023-12-19T16:42:27.801786+0000","last_unstale":"2023-12-19T16:42:27.810813+0000","last_undegraded":"2023-12-19T16:42:27.810813+0000","last_fullsized":"2023-12-19T16:42:27.810813+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:16.399823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784006+0000","last_change":"2023-12-19T16:42:27.311025+0000","last_active":"2023-12-19T16:42:28.784006+0000","last_peered":"2023-12-19T16:42:28.784006+0000","last_clean":"2023-12-19T16:42:28.784006+0000","last_became_active":"2023-12-19T16:42:27.310676+0000","last_became_peered":"2023-12-19T16:42:27.310676+0000","last_unstale":"2023-12-19T16:42:28.784006+0000","last_undegraded":"2023-12-19T16:42:28.784006+0000","last_fullsized":"2023-12-19T16:42:28.784006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830676+0000","last_change":"2023-12-19T16:42:27.821468+0000","last_active":"2023-12-19T16:42:27.830676+0000","last_peered":"2023-12-19T16:42:27.830676+0000","last_clean":"2023-12-19T16:42:27.830676+0000","last_became_active":"2023-12-19T16:42:27.821113+0000","last_became_peered":"2023-12-19T16:42:27.821113+0000","last_unstale":"2023-12-19T16:42:27.830676+0000","last_undegraded":"2023-12-19T16:42:27.830676+0000","last_fullsized":"2023-12-19T16:42:27.830676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:03.390272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790606+0000","last_change":"2023-12-19T16:42:27.323313+0000","last_active":"2023-12-19T16:42:28.790606+0000","last_peered":"2023-12-19T16:42:28.790606+0000","last_clean":"2023-12-19T16:42:28.790606+0000","last_became_active":"2023-12-19T16:42:27.319464+0000","last_became_peered":"2023-12-19T16:42:27.319464+0000","last_unstale":"2023-12-19T16:42:28.790606+0000","last_undegraded":"2023-12-19T16:42:28.790606+0000","last_fullsized":"2023-12-19T16:42:28.790606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:22:50.990280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830635+0000","last_change":"2023-12-19T16:42:27.821587+0000","last_active":"2023-12-19T16:42:27.830635+0000","last_peered":"2023-12-19T16:42:27.830635+0000","last_clean":"2023-12-19T16:42:27.830635+0000","last_became_active":"2023-12-19T16:42:27.821251+0000","last_became_peered":"2023-12-19T16:42:27.821251+0000","last_unstale":"2023-12-19T16:42:27.830635+0000","last_undegraded":"2023-12-19T16:42:27.830635+0000","last_fullsized":"2023-12-19T16:42:27.830635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:18.209972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784184+0000","last_change":"2023-12-19T16:42:27.319966+0000","last_active":"2023-12-19T16:42:28.784184+0000","last_peered":"2023-12-19T16:42:28.784184+0000","last_clean":"2023-12-19T16:42:28.784184+0000","last_became_active":"2023-12-19T16:42:27.319141+0000","last_became_peered":"2023-12-19T16:42:27.319141+0000","last_unstale":"2023-12-19T16:42:28.784184+0000","last_undegraded":"2023-12-19T16:42:28.784184+0000","last_fullsized":"2023-12-19T16:42:28.784184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830698+0000","last_change":"2023-12-19T16:42:27.824936+0000","last_active":"2023-12-19T16:42:27.830698+0000","last_peered":"2023-12-19T16:42:27.830698+0000","last_clean":"2023-12-19T16:42:27.830698+0000","last_became_active":"2023-12-19T16:42:27.824643+0000","last_became_peered":"2023-12-19T16:42:27.824643+0000","last_unstale":"2023-12-19T16:42:27.830698+0000","last_undegraded":"2023-12-19T16:42:27.830698+0000","last_fullsized":"2023-12-19T16:42:27.830698+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:23.006980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955939+0000","last_change":"2023-12-19T16:42:27.332207+0000","last_active":"2023-12-19T16:42:28.955939+0000","last_peered":"2023-12-19T16:42:28.955939+0000","last_clean":"2023-12-19T16:42:28.955939+0000","last_became_active":"2023-12-19T16:42:27.331920+0000","last_became_peered":"2023-12-19T16:42:27.331920+0000","last_unstale":"2023-12-19T16:42:28.955939+0000","last_undegraded":"2023-12-19T16:42:28.955939+0000","last_fullsized":"2023-12-19T16:42:28.955939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.708383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790779+0000","last_change":"2023-12-19T16:42:27.803564+0000","last_active":"2023-12-19T16:42:28.790779+0000","last_peered":"2023-12-19T16:42:28.790779+0000","last_clean":"2023-12-19T16:42:28.790779+0000","last_became_active":"2023-12-19T16:42:27.803291+0000","last_became_peered":"2023-12-19T16:42:27.803291+0000","last_unstale":"2023-12-19T16:42:28.790779+0000","last_undegraded":"2023-12-19T16:42:28.790779+0000","last_fullsized":"2023-12-19T16:42:28.790779+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:49.764257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784115+0000","last_change":"2023-12-19T16:42:27.316666+0000","last_active":"2023-12-19T16:42:28.784115+0000","last_peered":"2023-12-19T16:42:28.784115+0000","last_clean":"2023-12-19T16:42:28.784115+0000","last_became_active":"2023-12-19T16:42:27.316322+0000","last_became_peered":"2023-12-19T16:42:27.316322+0000","last_unstale":"2023-12-19T16:42:28.784115+0000","last_undegraded":"2023-12-19T16:42:28.784115+0000","last_fullsized":"2023-12-19T16:42:28.784115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.831678+0000","last_change":"2023-12-19T16:42:27.828580+0000","last_active":"2023-12-19T16:42:27.831678+0000","last_peered":"2023-12-19T16:42:27.831678+0000","last_clean":"2023-12-19T16:42:27.831678+0000","last_became_active":"2023-12-19T16:42:27.828263+0000","last_became_peered":"2023-12-19T16:42:27.828263+0000","last_unstale":"2023-12-19T16:42:27.831678+0000","last_undegraded":"2023-12-19T16:42:27.831678+0000","last_fullsized":"2023-12-19T16:42:27.831678+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:15.012739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.318071+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:42:27.318071+0000","last_peered":"2023-12-19T16:42:27.318071+0000","last_clean":"2023-12-19T16:42:27.318071+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:42:27.318071+0000","last_undegraded":"2023-12-19T16:42:27.318071+0000","last_fullsized":"2023-12-19T16:42:27.318071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790309+0000","last_change":"2023-12-19T16:42:27.322949+0000","last_active":"2023-12-19T16:42:28.790309+0000","last_peered":"2023-12-19T16:42:28.790309+0000","last_clean":"2023-12-19T16:42:28.790309+0000","last_became_active":"2023-12-19T16:42:27.321633+0000","last_became_peered":"2023-12-19T16:42:27.321633+0000","last_unstale":"2023-12-19T16:42:28.790309+0000","last_undegraded":"2023-12-19T16:42:28.790309+0000","last_fullsized":"2023-12-19T16:42:28.790309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.811675+0000","last_change":"2023-12-19T16:42:27.798689+0000","last_active":"2023-12-19T16:42:27.811675+0000","last_peered":"2023-12-19T16:42:27.811675+0000","last_clean":"2023-12-19T16:42:27.811675+0000","last_became_active":"2023-12-19T16:42:27.798430+0000","last_became_peered":"2023-12-19T16:42:27.798430+0000","last_unstale":"2023-12-19T16:42:27.811675+0000","last_undegraded":"2023-12-19T16:42:27.811675+0000","last_fullsized":"2023-12-19T16:42:27.811675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:16.399823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.801228+0000","last_change":"2023-12-19T16:42:27.326882+0000","last_active":"2023-12-19T16:42:27.801228+0000","last_peered":"2023-12-19T16:42:27.801228+0000","last_clean":"2023-12-19T16:42:27.801228+0000","last_became_active":"2023-12-19T16:42:27.326616+0000","last_became_peered":"2023-12-19T16:42:27.326616+0000","last_unstale":"2023-12-19T16:42:27.801228+0000","last_undegraded":"2023-12-19T16:42:27.801228+0000","last_fullsized":"2023-12-19T16:42:27.801228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:36.280588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.819374+0000","last_change":"2023-12-19T16:42:27.816279+0000","last_active":"2023-12-19T16:42:27.819374+0000","last_peered":"2023-12-19T16:42:27.819374+0000","last_clean":"2023-12-19T16:42:27.819374+0000","last_became_active":"2023-12-19T16:42:27.815940+0000","last_became_peered":"2023-12-19T16:42:27.815940+0000","last_unstale":"2023-12-19T16:42:27.819374+0000","last_undegraded":"2023-12-19T16:42:27.819374+0000","last_fullsized":"2023-12-19T16:42:27.819374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:51:00.190122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784142+0000","last_change":"2023-12-19T16:42:27.317414+0000","last_active":"2023-12-19T16:42:28.784142+0000","last_peered":"2023-12-19T16:42:28.784142+0000","last_clean":"2023-12-19T16:42:28.784142+0000","last_became_active":"2023-12-19T16:42:27.317065+0000","last_became_peered":"2023-12-19T16:42:27.317065+0000","last_unstale":"2023-12-19T16:42:28.784142+0000","last_undegraded":"2023-12-19T16:42:28.784142+0000","last_fullsized":"2023-12-19T16:42:28.784142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955832+0000","last_change":"2023-12-19T16:42:27.794867+0000","last_active":"2023-12-19T16:42:28.955832+0000","last_peered":"2023-12-19T16:42:28.955832+0000","last_clean":"2023-12-19T16:42:28.955832+0000","last_became_active":"2023-12-19T16:42:27.793788+0000","last_became_peered":"2023-12-19T16:42:27.793788+0000","last_unstale":"2023-12-19T16:42:28.955832+0000","last_undegraded":"2023-12-19T16:42:28.955832+0000","last_fullsized":"2023-12-19T16:42:28.955832+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:58:03.549430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.791093+0000","last_change":"2023-12-19T16:42:27.312696+0000","last_active":"2023-12-19T16:42:28.791093+0000","last_peered":"2023-12-19T16:42:28.791093+0000","last_clean":"2023-12-19T16:42:28.791093+0000","last_became_active":"2023-12-19T16:42:27.312406+0000","last_became_peered":"2023-12-19T16:42:27.312406+0000","last_unstale":"2023-12-19T16:42:28.791093+0000","last_undegraded":"2023-12-19T16:42:28.791093+0000","last_fullsized":"2023-12-19T16:42:28.791093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:41.116483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789955+0000","last_change":"2023-12-19T16:42:27.813887+0000","last_active":"2023-12-19T16:42:28.789955+0000","last_peered":"2023-12-19T16:42:28.789955+0000","last_clean":"2023-12-19T16:42:28.789955+0000","last_became_active":"2023-12-19T16:42:27.813476+0000","last_became_peered":"2023-12-19T16:42:27.813476+0000","last_unstale":"2023-12-19T16:42:28.789955+0000","last_undegraded":"2023-12-19T16:42:28.789955+0000","last_fullsized":"2023-12-19T16:42:28.789955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:36.975084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955856+0000","last_change":"2023-12-19T16:42:27.302950+0000","last_active":"2023-12-19T16:42:28.955856+0000","last_peered":"2023-12-19T16:42:28.955856+0000","last_clean":"2023-12-19T16:42:28.955856+0000","last_became_active":"2023-12-19T16:42:27.302614+0000","last_became_peered":"2023-12-19T16:42:27.302614+0000","last_unstale":"2023-12-19T16:42:28.955856+0000","last_undegraded":"2023-12-19T16:42:28.955856+0000","last_fullsized":"2023-12-19T16:42:28.955856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790926+0000","last_change":"2023-12-19T16:42:27.803672+0000","last_active":"2023-12-19T16:42:28.790926+0000","last_peered":"2023-12-19T16:42:28.790926+0000","last_clean":"2023-12-19T16:42:28.790926+0000","last_became_active":"2023-12-19T16:42:27.803402+0000","last_became_peered":"2023-12-19T16:42:27.803402+0000","last_unstale":"2023-12-19T16:42:28.790926+0000","last_undegraded":"2023-12-19T16:42:28.790926+0000","last_fullsized":"2023-12-19T16:42:28.790926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:48:49.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.804106+0000","last_change":"2023-12-19T16:42:27.330785+0000","last_active":"2023-12-19T16:42:27.804106+0000","last_peered":"2023-12-19T16:42:27.804106+0000","last_clean":"2023-12-19T16:42:27.804106+0000","last_became_active":"2023-12-19T16:42:27.329823+0000","last_became_peered":"2023-12-19T16:42:27.329823+0000","last_unstale":"2023-12-19T16:42:27.804106+0000","last_undegraded":"2023-12-19T16:42:27.804106+0000","last_fullsized":"2023-12-19T16:42:27.804106+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955776+0000","last_change":"2023-12-19T16:42:27.794649+0000","last_active":"2023-12-19T16:42:28.955776+0000","last_peered":"2023-12-19T16:42:28.955776+0000","last_clean":"2023-12-19T16:42:28.955776+0000","last_became_active":"2023-12-19T16:42:27.793710+0000","last_became_peered":"2023-12-19T16:42:27.793710+0000","last_unstale":"2023-12-19T16:42:28.955776+0000","last_undegraded":"2023-12-19T16:42:28.955776+0000","last_fullsized":"2023-12-19T16:42:28.955776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:16.399823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955987+0000","last_change":"2023-12-19T16:42:27.321060+0000","last_active":"2023-12-19T16:42:28.955987+0000","last_peered":"2023-12-19T16:42:28.955987+0000","last_clean":"2023-12-19T16:42:28.955987+0000","last_became_active":"2023-12-19T16:42:27.320705+0000","last_became_peered":"2023-12-19T16:42:27.320705+0000","last_unstale":"2023-12-19T16:42:28.955987+0000","last_undegraded":"2023-12-19T16:42:28.955987+0000","last_fullsized":"2023-12-19T16:42:28.955987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:35.968756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.819267+0000","last_change":"2023-12-19T16:42:27.818162+0000","last_active":"2023-12-19T16:42:27.819267+0000","last_peered":"2023-12-19T16:42:27.819267+0000","last_clean":"2023-12-19T16:42:27.819267+0000","last_became_active":"2023-12-19T16:42:27.817894+0000","last_became_peered":"2023-12-19T16:42:27.817894+0000","last_unstale":"2023-12-19T16:42:27.819267+0000","last_undegraded":"2023-12-19T16:42:27.819267+0000","last_fullsized":"2023-12-19T16:42:27.819267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:29.376069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.825018+0000","last_change":"2023-12-19T16:42:27.330277+0000","last_active":"2023-12-19T16:42:27.825018+0000","last_peered":"2023-12-19T16:42:27.825018+0000","last_clean":"2023-12-19T16:42:27.825018+0000","last_became_active":"2023-12-19T16:42:27.330006+0000","last_became_peered":"2023-12-19T16:42:27.330006+0000","last_unstale":"2023-12-19T16:42:27.825018+0000","last_undegraded":"2023-12-19T16:42:27.825018+0000","last_fullsized":"2023-12-19T16:42:27.825018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.799477+0000","last_change":"2023-12-19T16:42:27.791761+0000","last_active":"2023-12-19T16:42:27.799477+0000","last_peered":"2023-12-19T16:42:27.799477+0000","last_clean":"2023-12-19T16:42:27.799477+0000","last_became_active":"2023-12-19T16:42:27.791490+0000","last_became_peered":"2023-12-19T16:42:27.791490+0000","last_unstale":"2023-12-19T16:42:27.799477+0000","last_undegraded":"2023-12-19T16:42:27.799477+0000","last_fullsized":"2023-12-19T16:42:27.799477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:48:40.695901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.794541+0000","last_change":"2023-12-19T16:42:27.326363+0000","last_active":"2023-12-19T16:42:27.794541+0000","last_peered":"2023-12-19T16:42:27.794541+0000","last_clean":"2023-12-19T16:42:27.794541+0000","last_became_active":"2023-12-19T16:42:27.326046+0000","last_became_peered":"2023-12-19T16:42:27.326046+0000","last_unstale":"2023-12-19T16:42:27.794541+0000","last_undegraded":"2023-12-19T16:42:27.794541+0000","last_fullsized":"2023-12-19T16:42:27.794541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:41.116483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789733+0000","last_change":"2023-12-19T16:42:27.808612+0000","last_active":"2023-12-19T16:42:28.789733+0000","last_peered":"2023-12-19T16:42:28.789733+0000","last_clean":"2023-12-19T16:42:28.789733+0000","last_became_active":"2023-12-19T16:42:27.808219+0000","last_became_peered":"2023-12-19T16:42:27.808219+0000","last_unstale":"2023-12-19T16:42:28.789733+0000","last_undegraded":"2023-12-19T16:42:28.789733+0000","last_fullsized":"2023-12-19T16:42:28.789733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:37.286916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955811+0000","last_change":"2023-12-19T16:42:27.308462+0000","last_active":"2023-12-19T16:42:28.955811+0000","last_peered":"2023-12-19T16:42:28.955811+0000","last_clean":"2023-12-19T16:42:28.955811+0000","last_became_active":"2023-12-19T16:42:27.307879+0000","last_became_peered":"2023-12-19T16:42:27.307879+0000","last_unstale":"2023-12-19T16:42:28.955811+0000","last_undegraded":"2023-12-19T16:42:28.955811+0000","last_fullsized":"2023-12-19T16:42:28.955811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.811504+0000","last_change":"2023-12-19T16:42:27.798674+0000","last_active":"2023-12-19T16:42:27.811504+0000","last_peered":"2023-12-19T16:42:27.811504+0000","last_clean":"2023-12-19T16:42:27.811504+0000","last_became_active":"2023-12-19T16:42:27.798442+0000","last_became_peered":"2023-12-19T16:42:27.798442+0000","last_unstale":"2023-12-19T16:42:27.811504+0000","last_undegraded":"2023-12-19T16:42:27.811504+0000","last_fullsized":"2023-12-19T16:42:27.811504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:01.122709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955925+0000","last_change":"2023-12-19T16:42:27.310942+0000","last_active":"2023-12-19T16:42:28.955925+0000","last_peered":"2023-12-19T16:42:28.955925+0000","last_clean":"2023-12-19T16:42:28.955925+0000","last_became_active":"2023-12-19T16:42:27.310605+0000","last_became_peered":"2023-12-19T16:42:27.310605+0000","last_unstale":"2023-12-19T16:42:28.955925+0000","last_undegraded":"2023-12-19T16:42:28.955925+0000","last_fullsized":"2023-12-19T16:42:28.955925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.814364+0000","last_change":"2023-12-19T16:42:27.802676+0000","last_active":"2023-12-19T16:42:27.814364+0000","last_peered":"2023-12-19T16:42:27.814364+0000","last_clean":"2023-12-19T16:42:27.814364+0000","last_became_active":"2023-12-19T16:42:27.801627+0000","last_became_peered":"2023-12-19T16:42:27.801627+0000","last_unstale":"2023-12-19T16:42:27.814364+0000","last_undegraded":"2023-12-19T16:42:27.814364+0000","last_fullsized":"2023-12-19T16:42:27.814364+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:30.545767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.834491+0000","last_change":"2023-12-19T16:42:27.334084+0000","last_active":"2023-12-19T16:42:27.834491+0000","last_peered":"2023-12-19T16:42:27.834491+0000","last_clean":"2023-12-19T16:42:27.834491+0000","last_became_active":"2023-12-19T16:42:27.333748+0000","last_became_peered":"2023-12-19T16:42:27.333748+0000","last_unstale":"2023-12-19T16:42:27.834491+0000","last_undegraded":"2023-12-19T16:42:27.834491+0000","last_fullsized":"2023-12-19T16:42:27.834491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.822317+0000","last_change":"2023-12-19T16:42:27.817078+0000","last_active":"2023-12-19T16:42:27.822317+0000","last_peered":"2023-12-19T16:42:27.822317+0000","last_clean":"2023-12-19T16:42:27.822317+0000","last_became_active":"2023-12-19T16:42:27.816835+0000","last_became_peered":"2023-12-19T16:42:27.816835+0000","last_unstale":"2023-12-19T16:42:27.822317+0000","last_undegraded":"2023-12-19T16:42:27.822317+0000","last_fullsized":"2023-12-19T16:42:27.822317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790847+0000","last_change":"2023-12-19T16:42:27.820348+0000","last_active":"2023-12-19T16:42:28.790847+0000","last_peered":"2023-12-19T16:42:28.790847+0000","last_clean":"2023-12-19T16:42:28.790847+0000","last_became_active":"2023-12-19T16:42:27.820051+0000","last_became_peered":"2023-12-19T16:42:27.820051+0000","last_unstale":"2023-12-19T16:42:28.790847+0000","last_undegraded":"2023-12-19T16:42:28.790847+0000","last_fullsized":"2023-12-19T16:42:28.790847+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:23.006980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.829846+0000","last_change":"2023-12-19T16:42:27.328852+0000","last_active":"2023-12-19T16:42:27.829846+0000","last_peered":"2023-12-19T16:42:27.829846+0000","last_clean":"2023-12-19T16:42:27.829846+0000","last_became_active":"2023-12-19T16:42:27.328521+0000","last_became_peered":"2023-12-19T16:42:27.328521+0000","last_unstale":"2023-12-19T16:42:27.829846+0000","last_undegraded":"2023-12-19T16:42:27.829846+0000","last_fullsized":"2023-12-19T16:42:27.829846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:35.968756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790575+0000","last_change":"2023-12-19T16:42:27.820348+0000","last_active":"2023-12-19T16:42:28.790575+0000","last_peered":"2023-12-19T16:42:28.790575+0000","last_clean":"2023-12-19T16:42:28.790575+0000","last_became_active":"2023-12-19T16:42:27.820018+0000","last_became_peered":"2023-12-19T16:42:27.820018+0000","last_unstale":"2023-12-19T16:42:28.790575+0000","last_undegraded":"2023-12-19T16:42:28.790575+0000","last_fullsized":"2023-12-19T16:42:28.790575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830692+0000","last_change":"2023-12-19T16:42:27.331867+0000","last_active":"2023-12-19T16:42:27.830692+0000","last_peered":"2023-12-19T16:42:27.830692+0000","last_clean":"2023-12-19T16:42:27.830692+0000","last_became_active":"2023-12-19T16:42:27.331354+0000","last_became_peered":"2023-12-19T16:42:27.331354+0000","last_unstale":"2023-12-19T16:42:27.830692+0000","last_undegraded":"2023-12-19T16:42:27.830692+0000","last_fullsized":"2023-12-19T16:42:27.830692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:47:06.561268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.810018+0000","last_change":"2023-12-19T16:42:27.800529+0000","last_active":"2023-12-19T16:42:27.810018+0000","last_peered":"2023-12-19T16:42:27.810018+0000","last_clean":"2023-12-19T16:42:27.810018+0000","last_became_active":"2023-12-19T16:42:27.800250+0000","last_became_peered":"2023-12-19T16:42:27.800250+0000","last_unstale":"2023-12-19T16:42:27.810018+0000","last_undegraded":"2023-12-19T16:42:27.810018+0000","last_fullsized":"2023-12-19T16:42:27.810018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:27.376917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.814552+0000","last_change":"2023-12-19T16:42:27.314367+0000","last_active":"2023-12-19T16:42:27.814552+0000","last_peered":"2023-12-19T16:42:27.814552+0000","last_clean":"2023-12-19T16:42:27.814552+0000","last_became_active":"2023-12-19T16:42:27.314128+0000","last_became_peered":"2023-12-19T16:42:27.314128+0000","last_unstale":"2023-12-19T16:42:27.814552+0000","last_undegraded":"2023-12-19T16:42:27.814552+0000","last_fullsized":"2023-12-19T16:42:27.814552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:34:59.521223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.814525+0000","last_change":"2023-12-19T16:42:27.804285+0000","last_active":"2023-12-19T16:42:27.814525+0000","last_peered":"2023-12-19T16:42:27.814525+0000","last_clean":"2023-12-19T16:42:27.814525+0000","last_became_active":"2023-12-19T16:42:27.804027+0000","last_became_peered":"2023-12-19T16:42:27.804027+0000","last_unstale":"2023-12-19T16:42:27.814525+0000","last_undegraded":"2023-12-19T16:42:27.814525+0000","last_fullsized":"2023-12-19T16:42:27.814525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:53.498191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.827895+0000","last_change":"2023-12-19T16:42:27.325512+0000","last_active":"2023-12-19T16:42:27.827895+0000","last_peered":"2023-12-19T16:42:27.827895+0000","last_clean":"2023-12-19T16:42:27.827895+0000","last_became_active":"2023-12-19T16:42:27.325175+0000","last_became_peered":"2023-12-19T16:42:27.325175+0000","last_unstale":"2023-12-19T16:42:27.827895+0000","last_undegraded":"2023-12-19T16:42:27.827895+0000","last_fullsized":"2023-12-19T16:42:27.827895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:31:30.386546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":819200,"data_stored":665561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":233472,"data_stored":76175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":81001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":81037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31276,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340564,"statfs":{"total":96636764160,"available":96604737536,"internally_reserved":0,"allocated":233472,"data_stored":76175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31276,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340564,"statfs":{"total":96636764160,"available":96604737536,"internally_reserved":0,"allocated":233472,"data_stored":80911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31272,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340568,"statfs":{"total":96636764160,"available":96604741632,"internally_reserved":0,"allocated":229376,"data_stored":74771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":802816,"data_stored":658204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:42:31.914 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:42:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.088+0000 7f81dbf96640 1 Processor -- start 2023-12-19T16:42:32.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.088+0000 7f81dbf96640 1 -- start start 2023-12-19T16:42:32.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.088+0000 7f81dbf96640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f81d4154220 0x7f81d415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.088+0000 7f81dbf96640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f81d406b3d0 0x7f81d40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.088+0000 7f81dbf96640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d406bc30 con 0x7f81d40cec40 2023-12-19T16:42:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.088+0000 7f81dbf96640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d40cf0d0 con 0x7f81d4154220 2023-12-19T16:42:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.088+0000 7f81dbf96640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d40cc6e0 con 0x7f81d406b3d0 2023-12-19T16:42:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.088+0000 7f81d9d32640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f81d4154220 0x7f81d415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.088+0000 7f81da533640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f81d40cec40 0x7f81d415d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50238/0 (socket says 172.21.15.26:50238) 2023-12-19T16:42:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.088+0000 7f81d9d32640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f81d4154220 0x7f81d415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49154/0 (socket says 172.21.15.26:49154) 2023-12-19T16:42:32.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.092+0000 7f81da533640 1 -- 172.21.15.26:0/2726957486 learned_addr learned my addr 172.21.15.26:0/2726957486 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:32.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.092+0000 7f81d9531640 1 --2- 172.21.15.26:0/2726957486 >> v2:172.21.15.160:3300/0 conn(0x7f81d406b3d0 0x7f81d40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.092+0000 7f81d9d32640 1 -- 172.21.15.26:0/2726957486 >> v2:172.21.15.160:3300/0 conn(0x7f81d406b3d0 msgr2=0x7f81d40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:32.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.092+0000 7f81d9d32640 1 --2- 172.21.15.26:0/2726957486 >> v2:172.21.15.160:3300/0 conn(0x7f81d406b3d0 0x7f81d40ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.092+0000 7f81d9d32640 1 -- 172.21.15.26:0/2726957486 >> v1:172.21.15.26:6789/0 conn(0x7f81d40cec40 legacy=0x7f81d415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.092+0000 7f81d9d32640 1 -- 172.21.15.26:0/2726957486 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d415afc0 con 0x7f81d4154220 2023-12-19T16:42:32.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.092+0000 7f81d9531640 1 --2- 172.21.15.26:0/2726957486 >> v2:172.21.15.160:3300/0 conn(0x7f81d406b3d0 0x7f81d40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.092+0000 7f81d9d32640 1 --2- 172.21.15.26:0/2726957486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81d4154220 0x7f81d415aa80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f81d40cf880 tx=0x7f81d001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ba8bb35b69f9e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.092+0000 7f81d8d30640 1 -- 172.21.15.26:0/2726957486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0004230 con 0x7f81d4154220 2023-12-19T16:42:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.092+0000 7f81d8d30640 1 -- 172.21.15.26:0/2726957486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d00a7050 con 0x7f81d4154220 2023-12-19T16:42:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.092+0000 7f81d8d30640 1 -- 172.21.15.26:0/2726957486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0003620 con 0x7f81d4154220 2023-12-19T16:42:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 -- 172.21.15.26:0/2726957486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81d4154220 msgr2=0x7f81d415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 --2- 172.21.15.26:0/2726957486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81d4154220 0x7f81d415aa80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f81d40cf880 tx=0x7f81d001c430 comp rx=0 tx=0).stop 2023-12-19T16:42:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 -- 172.21.15.26:0/2726957486 shutdown_connections 2023-12-19T16:42:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 --2- 172.21.15.26:0/2726957486 >> v2:172.21.15.160:3300/0 conn(0x7f81d406b3d0 0x7f81d40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 --2- 172.21.15.26:0/2726957486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81d4154220 0x7f81d415aa80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 -- 172.21.15.26:0/2726957486 >> 172.21.15.26:0/2726957486 conn(0x7f81d40658e0 msgr2=0x7f81d4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 -- 172.21.15.26:0/2726957486 shutdown_connections 2023-12-19T16:42:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 -- 172.21.15.26:0/2726957486 wait complete. 2023-12-19T16:42:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 Processor -- start 2023-12-19T16:42:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 -- start start 2023-12-19T16:42:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f81d406b3d0 0x7f81d413dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81d40cec40 0x7f81d40cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81d4154220 0x7f81d415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d414f280 con 0x7f81d4154220 2023-12-19T16:42:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d41572e0 con 0x7f81d40cec40 2023-12-19T16:42:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81dbf96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d41522d0 con 0x7f81d406b3d0 2023-12-19T16:42:32.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81da533640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81d4154220 0x7f81d415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81da533640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81d4154220 0x7f81d415c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49162/0 (socket says 172.21.15.26:49162) 2023-12-19T16:42:32.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81da533640 1 -- 172.21.15.26:0/1094205781 learned_addr learned my addr 172.21.15.26:0/1094205781 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:32.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81d9531640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81d40cec40 0x7f81d40cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81d9d32640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f81d406b3d0 0x7f81d413dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81da533640 1 -- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f81d406b3d0 msgr2=0x7f81d413dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81da533640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f81d406b3d0 0x7f81d413dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81da533640 1 -- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81d40cec40 msgr2=0x7f81d40cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81da533640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81d40cec40 0x7f81d40cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81da533640 1 -- 172.21.15.26:0/1094205781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d0009420 con 0x7f81d4154220 2023-12-19T16:42:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81d9531640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81d40cec40 0x7f81d40cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.096+0000 7f81da533640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81d4154220 0x7f81d415c4d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00cab0 tx=0x7f81cc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.100+0000 7f81ca7fc640 1 -- 172.21.15.26:0/1094205781 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00ac30 con 0x7f81d4154220 2023-12-19T16:42:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.100+0000 7f81ca7fc640 1 -- 172.21.15.26:0/1094205781 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc00ad90 con 0x7f81d4154220 2023-12-19T16:42:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.100+0000 7f81ca7fc640 1 -- 172.21.15.26:0/1094205781 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc012600 con 0x7f81d4154220 2023-12-19T16:42:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.100+0000 7f81dbf96640 1 -- 172.21.15.26:0/1094205781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d41556e0 con 0x7f81d4154220 2023-12-19T16:42:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.100+0000 7f81dbf96640 1 -- 172.21.15.26:0/1094205781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d4155ca0 con 0x7f81d4154220 2023-12-19T16:42:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.100+0000 7f81dbf96640 1 -- 172.21.15.26:0/1094205781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d4155ec0 con 0x7f81d4154220 2023-12-19T16:42:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.100+0000 7f81ca7fc640 1 -- 172.21.15.26:0/1094205781 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f81cc012760 con 0x7f81d4154220 2023-12-19T16:42:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.100+0000 7f81ca7fc640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f81b8065a60 0x7f81b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.100+0000 7f81ca7fc640 1 -- 172.21.15.26:0/1094205781 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81cc082ee0 con 0x7f81d4154220 2023-12-19T16:42:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.100+0000 7f81d9d32640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f81b8065a60 0x7f81b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.104+0000 7f81d9d32640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f81b8065a60 0x7f81b8067f20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f81d00ad000 tx=0x7f81d00a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:32.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.104+0000 7f81ca7fc640 1 -- 172.21.15.26:0/1094205781 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f81d4155ec0 con 0x7f81d4154220 2023-12-19T16:42:32.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.260+0000 7f81dbf96640 1 -- 172.21.15.26:0/1094205781 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f81d40ccef0 con 0x7f81b8065a60 2023-12-19T16:42:32.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.264+0000 7f81ca7fc640 1 -- 172.21.15.26:0/1094205781 <== mgr.4106 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264547 (secure 0 0 0) 0x7f81d00bb090 con 0x7f81b8065a60 2023-12-19T16:42:32.273 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:32.274 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:42:32.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.268+0000 7f81dbf96640 1 -- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f81b8065a60 msgr2=0x7f81b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.268+0000 7f81dbf96640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f81b8065a60 0x7f81b8067f20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f81d00ad000 tx=0x7f81d00a6000 comp rx=0 tx=0).stop 2023-12-19T16:42:32.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.268+0000 7f81dbf96640 1 -- 172.21.15.26:0/1094205781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81d4154220 msgr2=0x7f81d415c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.268+0000 7f81dbf96640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81d4154220 0x7f81d415c4d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00cab0 tx=0x7f81cc00cf70 comp rx=0 tx=0).stop 2023-12-19T16:42:32.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.268+0000 7f81dbf96640 1 -- 172.21.15.26:0/1094205781 shutdown_connections 2023-12-19T16:42:32.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.268+0000 7f81dbf96640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81d4154220 0x7f81d415c4d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.268+0000 7f81dbf96640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81d40cec40 0x7f81d40cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.268+0000 7f81dbf96640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f81b8065a60 0x7f81b8067f20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.268+0000 7f81dbf96640 1 --2- 172.21.15.26:0/1094205781 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f81d406b3d0 0x7f81d413dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.268+0000 7f81dbf96640 1 -- 172.21.15.26:0/1094205781 >> 172.21.15.26:0/1094205781 conn(0x7f81d40658e0 msgr2=0x7f81d4065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:32.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.268+0000 7f81dbf96640 1 -- 172.21.15.26:0/1094205781 shutdown_connections 2023-12-19T16:42:32.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.268+0000 7f81dbf96640 1 -- 172.21.15.26:0/1094205781 wait complete. 2023-12-19T16:42:32.293 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-19T16:42:31.636952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234964,"kb_used_data":2964,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739756,"statfs":{"total":773094113280,"available":772853510144,"internally_reserved":0,"allocated":3035136,"data_stored":1793835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.347344"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830677+0000","last_change":"2023-12-19T16:42:27.824402+0000","last_active":"2023-12-19T16:42:27.830677+0000","last_peered":"2023-12-19T16:42:27.830677+0000","last_clean":"2023-12-19T16:42:27.830677+0000","last_became_active":"2023-12-19T16:42:27.818342+0000","last_became_peered":"2023-12-19T16:42:27.818342+0000","last_unstale":"2023-12-19T16:42:27.830677+0000","last_undegraded":"2023-12-19T16:42:27.830677+0000","last_fullsized":"2023-12-19T16:42:27.830677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:53.498191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.956000+0000","last_change":"2023-12-19T16:42:27.802326+0000","last_active":"2023-12-19T16:42:28.956000+0000","last_peered":"2023-12-19T16:42:28.956000+0000","last_clean":"2023-12-19T16:42:28.956000+0000","last_became_active":"2023-12-19T16:42:27.802056+0000","last_became_peered":"2023-12-19T16:42:27.802056+0000","last_unstale":"2023-12-19T16:42:28.956000+0000","last_undegraded":"2023-12-19T16:42:28.956000+0000","last_fullsized":"2023-12-19T16:42:28.956000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:30.545767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.807788+0000","last_change":"2023-12-19T16:42:27.798150+0000","last_active":"2023-12-19T16:42:27.807788+0000","last_peered":"2023-12-19T16:42:27.807788+0000","last_clean":"2023-12-19T16:42:27.807788+0000","last_became_active":"2023-12-19T16:42:27.796339+0000","last_became_peered":"2023-12-19T16:42:27.796339+0000","last_unstale":"2023-12-19T16:42:27.807788+0000","last_undegraded":"2023-12-19T16:42:27.807788+0000","last_fullsized":"2023-12-19T16:42:27.807788+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:42:19.277925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790632+0000","last_change":"2023-12-19T16:42:27.814794+0000","last_active":"2023-12-19T16:42:28.790632+0000","last_peered":"2023-12-19T16:42:28.790632+0000","last_clean":"2023-12-19T16:42:28.790632+0000","last_became_active":"2023-12-19T16:42:27.814461+0000","last_became_peered":"2023-12-19T16:42:27.814461+0000","last_unstale":"2023-12-19T16:42:28.790632+0000","last_undegraded":"2023-12-19T16:42:28.790632+0000","last_fullsized":"2023-12-19T16:42:28.790632+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:18.209972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830892+0000","last_change":"2023-12-19T16:42:27.825817+0000","last_active":"2023-12-19T16:42:27.830892+0000","last_peered":"2023-12-19T16:42:27.830892+0000","last_clean":"2023-12-19T16:42:27.830892+0000","last_became_active":"2023-12-19T16:42:27.825532+0000","last_became_peered":"2023-12-19T16:42:27.825532+0000","last_unstale":"2023-12-19T16:42:27.830892+0000","last_undegraded":"2023-12-19T16:42:27.830892+0000","last_fullsized":"2023-12-19T16:42:27.830892+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:49.764257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790441+0000","last_change":"2023-12-19T16:42:27.801183+0000","last_active":"2023-12-19T16:42:28.790441+0000","last_peered":"2023-12-19T16:42:28.790441+0000","last_clean":"2023-12-19T16:42:28.790441+0000","last_became_active":"2023-12-19T16:42:27.800866+0000","last_became_peered":"2023-12-19T16:42:27.800866+0000","last_unstale":"2023-12-19T16:42:28.790441+0000","last_undegraded":"2023-12-19T16:42:28.790441+0000","last_fullsized":"2023-12-19T16:42:28.790441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:36.473614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.827704+0000","last_change":"2023-12-19T16:42:27.817242+0000","last_active":"2023-12-19T16:42:27.827704+0000","last_peered":"2023-12-19T16:42:27.827704+0000","last_clean":"2023-12-19T16:42:27.827704+0000","last_became_active":"2023-12-19T16:42:27.816801+0000","last_became_peered":"2023-12-19T16:42:27.816801+0000","last_unstale":"2023-12-19T16:42:27.827704+0000","last_undegraded":"2023-12-19T16:42:27.827704+0000","last_fullsized":"2023-12-19T16:42:27.827704+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.813815+0000","last_change":"2023-12-19T16:42:27.801939+0000","last_active":"2023-12-19T16:42:27.813815+0000","last_peered":"2023-12-19T16:42:27.813815+0000","last_clean":"2023-12-19T16:42:27.813815+0000","last_became_active":"2023-12-19T16:42:27.801672+0000","last_became_peered":"2023-12-19T16:42:27.801672+0000","last_unstale":"2023-12-19T16:42:27.813815+0000","last_undegraded":"2023-12-19T16:42:27.813815+0000","last_fullsized":"2023-12-19T16:42:27.813815+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.831759+0000","last_change":"2023-12-19T16:42:27.829667+0000","last_active":"2023-12-19T16:42:27.831759+0000","last_peered":"2023-12-19T16:42:27.831759+0000","last_clean":"2023-12-19T16:42:27.831759+0000","last_became_active":"2023-12-19T16:42:27.829357+0000","last_became_peered":"2023-12-19T16:42:27.829357+0000","last_unstale":"2023-12-19T16:42:27.831759+0000","last_undegraded":"2023-12-19T16:42:27.831759+0000","last_fullsized":"2023-12-19T16:42:27.831759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:06.914435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.816645+0000","last_change":"2023-12-19T16:42:27.812001+0000","last_active":"2023-12-19T16:42:27.816645+0000","last_peered":"2023-12-19T16:42:27.816645+0000","last_clean":"2023-12-19T16:42:27.816645+0000","last_became_active":"2023-12-19T16:42:27.811749+0000","last_became_peered":"2023-12-19T16:42:27.811749+0000","last_unstale":"2023-12-19T16:42:27.816645+0000","last_undegraded":"2023-12-19T16:42:27.816645+0000","last_fullsized":"2023-12-19T16:42:27.816645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:29.376069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.818425+0000","last_change":"2023-12-19T16:42:27.809484+0000","last_active":"2023-12-19T16:42:27.818425+0000","last_peered":"2023-12-19T16:42:27.818425+0000","last_clean":"2023-12-19T16:42:27.818425+0000","last_became_active":"2023-12-19T16:42:27.809189+0000","last_became_peered":"2023-12-19T16:42:27.809189+0000","last_unstale":"2023-12-19T16:42:27.818425+0000","last_undegraded":"2023-12-19T16:42:27.818425+0000","last_fullsized":"2023-12-19T16:42:27.818425+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:03.390272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789997+0000","last_change":"2023-12-19T16:42:27.812088+0000","last_active":"2023-12-19T16:42:28.789997+0000","last_peered":"2023-12-19T16:42:28.789997+0000","last_clean":"2023-12-19T16:42:28.789997+0000","last_became_active":"2023-12-19T16:42:27.811768+0000","last_became_peered":"2023-12-19T16:42:27.811768+0000","last_unstale":"2023-12-19T16:42:28.789997+0000","last_undegraded":"2023-12-19T16:42:28.789997+0000","last_fullsized":"2023-12-19T16:42:28.789997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:47:07.567597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.814597+0000","last_change":"2023-12-19T16:42:27.805169+0000","last_active":"2023-12-19T16:42:27.814597+0000","last_peered":"2023-12-19T16:42:27.814597+0000","last_clean":"2023-12-19T16:42:27.814597+0000","last_became_active":"2023-12-19T16:42:27.804906+0000","last_became_peered":"2023-12-19T16:42:27.804906+0000","last_unstale":"2023-12-19T16:42:27.814597+0000","last_undegraded":"2023-12-19T16:42:27.814597+0000","last_fullsized":"2023-12-19T16:42:27.814597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:57:49.461118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.808101+0000","last_change":"2023-12-19T16:42:27.796100+0000","last_active":"2023-12-19T16:42:27.808101+0000","last_peered":"2023-12-19T16:42:27.808101+0000","last_clean":"2023-12-19T16:42:27.808101+0000","last_became_active":"2023-12-19T16:42:27.795802+0000","last_became_peered":"2023-12-19T16:42:27.795802+0000","last_unstale":"2023-12-19T16:42:27.808101+0000","last_undegraded":"2023-12-19T16:42:27.808101+0000","last_fullsized":"2023-12-19T16:42:27.808101+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:30:13.550890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955716+0000","last_change":"2023-12-19T16:42:27.815496+0000","last_active":"2023-12-19T16:42:28.955716+0000","last_peered":"2023-12-19T16:42:28.955716+0000","last_clean":"2023-12-19T16:42:28.955716+0000","last_became_active":"2023-12-19T16:42:27.812535+0000","last_became_peered":"2023-12-19T16:42:27.812535+0000","last_unstale":"2023-12-19T16:42:28.955716+0000","last_undegraded":"2023-12-19T16:42:28.955716+0000","last_fullsized":"2023-12-19T16:42:28.955716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:37.400723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790179+0000","last_change":"2023-12-19T16:42:27.804674+0000","last_active":"2023-12-19T16:42:28.790179+0000","last_peered":"2023-12-19T16:42:28.790179+0000","last_clean":"2023-12-19T16:42:28.790179+0000","last_became_active":"2023-12-19T16:42:27.804057+0000","last_became_peered":"2023-12-19T16:42:27.804057+0000","last_unstale":"2023-12-19T16:42:28.790179+0000","last_undegraded":"2023-12-19T16:42:28.790179+0000","last_fullsized":"2023-12-19T16:42:28.790179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:57:49.461118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790884+0000","last_change":"2023-12-19T16:42:27.829970+0000","last_active":"2023-12-19T16:42:28.790884+0000","last_peered":"2023-12-19T16:42:28.790884+0000","last_clean":"2023-12-19T16:42:28.790884+0000","last_became_active":"2023-12-19T16:42:27.829616+0000","last_became_peered":"2023-12-19T16:42:27.829616+0000","last_unstale":"2023-12-19T16:42:28.790884+0000","last_undegraded":"2023-12-19T16:42:28.790884+0000","last_fullsized":"2023-12-19T16:42:28.790884+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:50.303035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784083+0000","last_change":"2023-12-19T16:42:27.804309+0000","last_active":"2023-12-19T16:42:28.784083+0000","last_peered":"2023-12-19T16:42:28.784083+0000","last_clean":"2023-12-19T16:42:28.784083+0000","last_became_active":"2023-12-19T16:42:27.803973+0000","last_became_peered":"2023-12-19T16:42:27.803973+0000","last_unstale":"2023-12-19T16:42:28.784083+0000","last_undegraded":"2023-12-19T16:42:28.784083+0000","last_fullsized":"2023-12-19T16:42:28.784083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:47:07.567597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.831180+0000","last_change":"2023-12-19T16:42:27.827321+0000","last_active":"2023-12-19T16:42:27.831180+0000","last_peered":"2023-12-19T16:42:27.831180+0000","last_clean":"2023-12-19T16:42:27.831180+0000","last_became_active":"2023-12-19T16:42:27.826329+0000","last_became_peered":"2023-12-19T16:42:27.826329+0000","last_unstale":"2023-12-19T16:42:27.831180+0000","last_undegraded":"2023-12-19T16:42:27.831180+0000","last_fullsized":"2023-12-19T16:42:27.831180+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:01.122709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.822288+0000","last_change":"2023-12-19T16:42:27.816854+0000","last_active":"2023-12-19T16:42:27.822288+0000","last_peered":"2023-12-19T16:42:27.822288+0000","last_clean":"2023-12-19T16:42:27.822288+0000","last_became_active":"2023-12-19T16:42:27.816538+0000","last_became_peered":"2023-12-19T16:42:27.816538+0000","last_unstale":"2023-12-19T16:42:27.822288+0000","last_undegraded":"2023-12-19T16:42:27.822288+0000","last_fullsized":"2023-12-19T16:42:27.822288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:30.545767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.819162+0000","last_change":"2023-12-19T16:42:27.817144+0000","last_active":"2023-12-19T16:42:27.819162+0000","last_peered":"2023-12-19T16:42:27.819162+0000","last_clean":"2023-12-19T16:42:27.819162+0000","last_became_active":"2023-12-19T16:42:27.816891+0000","last_became_peered":"2023-12-19T16:42:27.816891+0000","last_unstale":"2023-12-19T16:42:27.819162+0000","last_undegraded":"2023-12-19T16:42:27.819162+0000","last_fullsized":"2023-12-19T16:42:27.819162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:51.134371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790438+0000","last_change":"2023-12-19T16:42:27.826273+0000","last_active":"2023-12-19T16:42:28.790438+0000","last_peered":"2023-12-19T16:42:28.790438+0000","last_clean":"2023-12-19T16:42:28.790438+0000","last_became_active":"2023-12-19T16:42:27.825992+0000","last_became_peered":"2023-12-19T16:42:27.825992+0000","last_unstale":"2023-12-19T16:42:28.790438+0000","last_undegraded":"2023-12-19T16:42:28.790438+0000","last_fullsized":"2023-12-19T16:42:28.790438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:36.031318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.817466+0000","last_change":"2023-12-19T16:42:27.813323+0000","last_active":"2023-12-19T16:42:27.817466+0000","last_peered":"2023-12-19T16:42:27.817466+0000","last_clean":"2023-12-19T16:42:27.817466+0000","last_became_active":"2023-12-19T16:42:27.812808+0000","last_became_peered":"2023-12-19T16:42:27.812808+0000","last_unstale":"2023-12-19T16:42:27.817466+0000","last_undegraded":"2023-12-19T16:42:27.817466+0000","last_fullsized":"2023-12-19T16:42:27.817466+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:36.473614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790317+0000","last_change":"2023-12-19T16:42:27.810294+0000","last_active":"2023-12-19T16:42:28.790317+0000","last_peered":"2023-12-19T16:42:28.790317+0000","last_clean":"2023-12-19T16:42:28.790317+0000","last_became_active":"2023-12-19T16:42:27.810031+0000","last_became_peered":"2023-12-19T16:42:27.810031+0000","last_unstale":"2023-12-19T16:42:28.790317+0000","last_undegraded":"2023-12-19T16:42:28.790317+0000","last_fullsized":"2023-12-19T16:42:28.790317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:01.122709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.818055+0000","last_change":"2023-12-19T16:42:27.813230+0000","last_active":"2023-12-19T16:42:27.818055+0000","last_peered":"2023-12-19T16:42:27.818055+0000","last_clean":"2023-12-19T16:42:27.818055+0000","last_became_active":"2023-12-19T16:42:27.812914+0000","last_became_peered":"2023-12-19T16:42:27.812914+0000","last_unstale":"2023-12-19T16:42:27.818055+0000","last_undegraded":"2023-12-19T16:42:27.818055+0000","last_fullsized":"2023-12-19T16:42:27.818055+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:50.303035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.823553+0000","last_change":"2023-12-19T16:42:27.819653+0000","last_active":"2023-12-19T16:42:27.823553+0000","last_peered":"2023-12-19T16:42:27.823553+0000","last_clean":"2023-12-19T16:42:27.823553+0000","last_became_active":"2023-12-19T16:42:27.819325+0000","last_became_peered":"2023-12-19T16:42:27.819325+0000","last_unstale":"2023-12-19T16:42:27.823553+0000","last_undegraded":"2023-12-19T16:42:27.823553+0000","last_fullsized":"2023-12-19T16:42:27.823553+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:40:46.397053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955842+0000","last_change":"2023-12-19T16:42:27.813010+0000","last_active":"2023-12-19T16:42:28.955842+0000","last_peered":"2023-12-19T16:42:28.955842+0000","last_clean":"2023-12-19T16:42:28.955842+0000","last_became_active":"2023-12-19T16:42:27.812336+0000","last_became_peered":"2023-12-19T16:42:27.812336+0000","last_unstale":"2023-12-19T16:42:28.955842+0000","last_undegraded":"2023-12-19T16:42:28.955842+0000","last_fullsized":"2023-12-19T16:42:28.955842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.818940+0000","last_change":"2023-12-19T16:42:27.813584+0000","last_active":"2023-12-19T16:42:27.818940+0000","last_peered":"2023-12-19T16:42:27.818940+0000","last_clean":"2023-12-19T16:42:27.818940+0000","last_became_active":"2023-12-19T16:42:27.813276+0000","last_became_peered":"2023-12-19T16:42:27.813276+0000","last_unstale":"2023-12-19T16:42:27.818940+0000","last_undegraded":"2023-12-19T16:42:27.818940+0000","last_fullsized":"2023-12-19T16:42:27.818940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:36.031318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.811791+0000","last_change":"2023-12-19T16:42:27.805390+0000","last_active":"2023-12-19T16:42:27.811791+0000","last_peered":"2023-12-19T16:42:27.811791+0000","last_clean":"2023-12-19T16:42:27.811791+0000","last_became_active":"2023-12-19T16:42:27.804956+0000","last_became_peered":"2023-12-19T16:42:27.804956+0000","last_unstale":"2023-12-19T16:42:27.811791+0000","last_undegraded":"2023-12-19T16:42:27.811791+0000","last_fullsized":"2023-12-19T16:42:27.811791+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:58.680381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.791132+0000","last_change":"2023-12-19T16:42:27.809669+0000","last_active":"2023-12-19T16:42:28.791132+0000","last_peered":"2023-12-19T16:42:28.791132+0000","last_clean":"2023-12-19T16:42:28.791132+0000","last_became_active":"2023-12-19T16:42:27.809374+0000","last_became_peered":"2023-12-19T16:42:27.809374+0000","last_unstale":"2023-12-19T16:42:28.791132+0000","last_undegraded":"2023-12-19T16:42:28.791132+0000","last_fullsized":"2023-12-19T16:42:28.791132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:53.498191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784039+0000","last_change":"2023-12-19T16:42:27.799272+0000","last_active":"2023-12-19T16:42:28.784039+0000","last_peered":"2023-12-19T16:42:28.784039+0000","last_clean":"2023-12-19T16:42:28.784039+0000","last_became_active":"2023-12-19T16:42:27.799006+0000","last_became_peered":"2023-12-19T16:42:27.799006+0000","last_unstale":"2023-12-19T16:42:28.784039+0000","last_undegraded":"2023-12-19T16:42:28.784039+0000","last_fullsized":"2023-12-19T16:42:28.784039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:37.286916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.816353+0000","last_change":"2023-12-19T16:42:27.809714+0000","last_active":"2023-12-19T16:42:27.816353+0000","last_peered":"2023-12-19T16:42:27.816353+0000","last_clean":"2023-12-19T16:42:27.816353+0000","last_became_active":"2023-12-19T16:42:27.809384+0000","last_became_peered":"2023-12-19T16:42:27.809384+0000","last_unstale":"2023-12-19T16:42:27.816353+0000","last_undegraded":"2023-12-19T16:42:27.816353+0000","last_fullsized":"2023-12-19T16:42:27.816353+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:37.400723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789634+0000","last_change":"2023-12-19T16:42:27.811964+0000","last_active":"2023-12-19T16:42:28.789634+0000","last_peered":"2023-12-19T16:42:28.789634+0000","last_clean":"2023-12-19T16:42:28.789634+0000","last_became_active":"2023-12-19T16:42:27.811777+0000","last_became_peered":"2023-12-19T16:42:27.811777+0000","last_unstale":"2023-12-19T16:42:28.789634+0000","last_undegraded":"2023-12-19T16:42:28.789634+0000","last_fullsized":"2023-12-19T16:42:28.789634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:50.714712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.825802+0000","last_change":"2023-12-19T16:42:27.324768+0000","last_active":"2023-12-19T16:42:27.825802+0000","last_peered":"2023-12-19T16:42:27.825802+0000","last_clean":"2023-12-19T16:42:27.825802+0000","last_became_active":"2023-12-19T16:42:27.324380+0000","last_became_peered":"2023-12-19T16:42:27.324380+0000","last_unstale":"2023-12-19T16:42:27.825802+0000","last_undegraded":"2023-12-19T16:42:27.825802+0000","last_fullsized":"2023-12-19T16:42:27.825802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.819143+0000","last_change":"2023-12-19T16:42:27.816607+0000","last_active":"2023-12-19T16:42:27.819143+0000","last_peered":"2023-12-19T16:42:27.819143+0000","last_clean":"2023-12-19T16:42:27.819143+0000","last_became_active":"2023-12-19T16:42:27.815987+0000","last_became_peered":"2023-12-19T16:42:27.815987+0000","last_unstale":"2023-12-19T16:42:27.819143+0000","last_undegraded":"2023-12-19T16:42:27.819143+0000","last_fullsized":"2023-12-19T16:42:27.819143+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:32:26.852410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.816394+0000","last_change":"2023-12-19T16:42:27.327298+0000","last_active":"2023-12-19T16:42:27.816394+0000","last_peered":"2023-12-19T16:42:27.816394+0000","last_clean":"2023-12-19T16:42:27.816394+0000","last_became_active":"2023-12-19T16:42:27.326988+0000","last_became_peered":"2023-12-19T16:42:27.326988+0000","last_unstale":"2023-12-19T16:42:27.816394+0000","last_undegraded":"2023-12-19T16:42:27.816394+0000","last_fullsized":"2023-12-19T16:42:27.816394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.791085+0000","last_change":"2023-12-19T16:42:27.320910+0000","last_active":"2023-12-19T16:42:27.791085+0000","last_peered":"2023-12-19T16:42:27.791085+0000","last_clean":"2023-12-19T16:42:27.791085+0000","last_became_active":"2023-12-19T16:42:27.320629+0000","last_became_peered":"2023-12-19T16:42:27.320629+0000","last_unstale":"2023-12-19T16:42:27.791085+0000","last_undegraded":"2023-12-19T16:42:27.791085+0000","last_fullsized":"2023-12-19T16:42:27.791085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.814014+0000","last_change":"2023-12-19T16:42:27.803836+0000","last_active":"2023-12-19T16:42:27.814014+0000","last_peered":"2023-12-19T16:42:27.814014+0000","last_clean":"2023-12-19T16:42:27.814014+0000","last_became_active":"2023-12-19T16:42:27.803527+0000","last_became_peered":"2023-12-19T16:42:27.803527+0000","last_unstale":"2023-12-19T16:42:27.814014+0000","last_undegraded":"2023-12-19T16:42:27.814014+0000","last_fullsized":"2023-12-19T16:42:27.814014+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:37.400723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790294+0000","last_change":"2023-12-19T16:42:27.322383+0000","last_active":"2023-12-19T16:42:28.790294+0000","last_peered":"2023-12-19T16:42:28.790294+0000","last_clean":"2023-12-19T16:42:28.790294+0000","last_became_active":"2023-12-19T16:42:27.321439+0000","last_became_peered":"2023-12-19T16:42:27.321439+0000","last_unstale":"2023-12-19T16:42:28.790294+0000","last_undegraded":"2023-12-19T16:42:28.790294+0000","last_fullsized":"2023-12-19T16:42:28.790294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955715+0000","last_change":"2023-12-19T16:42:27.800626+0000","last_active":"2023-12-19T16:42:28.955715+0000","last_peered":"2023-12-19T16:42:28.955715+0000","last_clean":"2023-12-19T16:42:28.955715+0000","last_became_active":"2023-12-19T16:42:27.800338+0000","last_became_peered":"2023-12-19T16:42:27.800338+0000","last_unstale":"2023-12-19T16:42:28.955715+0000","last_undegraded":"2023-12-19T16:42:28.955715+0000","last_fullsized":"2023-12-19T16:42:28.955715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:29.376069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.797659+0000","last_change":"2023-12-19T16:42:27.324962+0000","last_active":"2023-12-19T16:42:27.797659+0000","last_peered":"2023-12-19T16:42:27.797659+0000","last_clean":"2023-12-19T16:42:27.797659+0000","last_became_active":"2023-12-19T16:42:27.324627+0000","last_became_peered":"2023-12-19T16:42:27.324627+0000","last_unstale":"2023-12-19T16:42:27.797659+0000","last_undegraded":"2023-12-19T16:42:27.797659+0000","last_fullsized":"2023-12-19T16:42:27.797659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790825+0000","last_change":"2023-12-19T16:42:27.829894+0000","last_active":"2023-12-19T16:42:28.790825+0000","last_peered":"2023-12-19T16:42:28.790825+0000","last_clean":"2023-12-19T16:42:28.790825+0000","last_became_active":"2023-12-19T16:42:27.829581+0000","last_became_peered":"2023-12-19T16:42:27.829581+0000","last_unstale":"2023-12-19T16:42:28.790825+0000","last_undegraded":"2023-12-19T16:42:28.790825+0000","last_fullsized":"2023-12-19T16:42:28.790825+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:00.499277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790905+0000","last_change":"2023-12-19T16:42:27.322335+0000","last_active":"2023-12-19T16:42:28.790905+0000","last_peered":"2023-12-19T16:42:28.790905+0000","last_clean":"2023-12-19T16:42:28.790905+0000","last_became_active":"2023-12-19T16:42:27.319501+0000","last_became_peered":"2023-12-19T16:42:27.319501+0000","last_unstale":"2023-12-19T16:42:28.790905+0000","last_undegraded":"2023-12-19T16:42:28.790905+0000","last_fullsized":"2023-12-19T16:42:28.790905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.808144+0000","last_change":"2023-12-19T16:42:27.798069+0000","last_active":"2023-12-19T16:42:27.808144+0000","last_peered":"2023-12-19T16:42:27.808144+0000","last_clean":"2023-12-19T16:42:27.808144+0000","last_became_active":"2023-12-19T16:42:27.795622+0000","last_became_peered":"2023-12-19T16:42:27.795622+0000","last_unstale":"2023-12-19T16:42:27.808144+0000","last_undegraded":"2023-12-19T16:42:27.808144+0000","last_fullsized":"2023-12-19T16:42:27.808144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:15:14.536450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955857+0000","last_change":"2023-12-19T16:42:27.310215+0000","last_active":"2023-12-19T16:42:28.955857+0000","last_peered":"2023-12-19T16:42:28.955857+0000","last_clean":"2023-12-19T16:42:28.955857+0000","last_became_active":"2023-12-19T16:42:27.308663+0000","last_became_peered":"2023-12-19T16:42:27.308663+0000","last_unstale":"2023-12-19T16:42:28.955857+0000","last_undegraded":"2023-12-19T16:42:28.955857+0000","last_fullsized":"2023-12-19T16:42:28.955857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:41.116483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790149+0000","last_change":"2023-12-19T16:42:27.825472+0000","last_active":"2023-12-19T16:42:28.790149+0000","last_peered":"2023-12-19T16:42:28.790149+0000","last_clean":"2023-12-19T16:42:28.790149+0000","last_became_active":"2023-12-19T16:42:27.825201+0000","last_became_peered":"2023-12-19T16:42:27.825201+0000","last_unstale":"2023-12-19T16:42:28.790149+0000","last_undegraded":"2023-12-19T16:42:28.790149+0000","last_fullsized":"2023-12-19T16:42:28.790149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:42.122812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.828695+0000","last_change":"2023-12-19T16:42:27.321076+0000","last_active":"2023-12-19T16:42:27.828695+0000","last_peered":"2023-12-19T16:42:27.828695+0000","last_clean":"2023-12-19T16:42:27.828695+0000","last_became_active":"2023-12-19T16:42:27.320592+0000","last_became_peered":"2023-12-19T16:42:27.320592+0000","last_unstale":"2023-12-19T16:42:27.828695+0000","last_undegraded":"2023-12-19T16:42:27.828695+0000","last_fullsized":"2023-12-19T16:42:27.828695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.708383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.827744+0000","last_change":"2023-12-19T16:42:27.818972+0000","last_active":"2023-12-19T16:42:27.827744+0000","last_peered":"2023-12-19T16:42:27.827744+0000","last_clean":"2023-12-19T16:42:27.827744+0000","last_became_active":"2023-12-19T16:42:27.818689+0000","last_became_peered":"2023-12-19T16:42:27.818689+0000","last_unstale":"2023-12-19T16:42:27.827744+0000","last_undegraded":"2023-12-19T16:42:27.827744+0000","last_fullsized":"2023-12-19T16:42:27.827744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:48:49.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.823238+0000","last_change":"2023-12-19T16:42:27.306416+0000","last_active":"2023-12-19T16:42:27.823238+0000","last_peered":"2023-12-19T16:42:27.823238+0000","last_clean":"2023-12-19T16:42:27.823238+0000","last_became_active":"2023-12-19T16:42:27.306033+0000","last_became_peered":"2023-12-19T16:42:27.306033+0000","last_unstale":"2023-12-19T16:42:27.823238+0000","last_undegraded":"2023-12-19T16:42:27.823238+0000","last_fullsized":"2023-12-19T16:42:27.823238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:29.172833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.813217+0000","last_change":"2023-12-19T16:42:27.809290+0000","last_active":"2023-12-19T16:42:27.813217+0000","last_peered":"2023-12-19T16:42:27.813217+0000","last_clean":"2023-12-19T16:42:27.813217+0000","last_became_active":"2023-12-19T16:42:27.806028+0000","last_became_peered":"2023-12-19T16:42:27.806028+0000","last_unstale":"2023-12-19T16:42:27.813217+0000","last_undegraded":"2023-12-19T16:42:27.813217+0000","last_fullsized":"2023-12-19T16:42:27.813217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:48:43.490883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.791044+0000","last_change":"2023-12-19T16:42:27.317606+0000","last_active":"2023-12-19T16:42:28.791044+0000","last_peered":"2023-12-19T16:42:28.791044+0000","last_clean":"2023-12-19T16:42:28.791044+0000","last_became_active":"2023-12-19T16:42:27.316871+0000","last_became_peered":"2023-12-19T16:42:27.316871+0000","last_unstale":"2023-12-19T16:42:28.791044+0000","last_undegraded":"2023-12-19T16:42:28.791044+0000","last_fullsized":"2023-12-19T16:42:28.791044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.708383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830963+0000","last_change":"2023-12-19T16:42:27.826100+0000","last_active":"2023-12-19T16:42:27.830963+0000","last_peered":"2023-12-19T16:42:27.830963+0000","last_clean":"2023-12-19T16:42:27.830963+0000","last_became_active":"2023-12-19T16:42:27.825764+0000","last_became_peered":"2023-12-19T16:42:27.825764+0000","last_unstale":"2023-12-19T16:42:27.830963+0000","last_undegraded":"2023-12-19T16:42:27.830963+0000","last_fullsized":"2023-12-19T16:42:27.830963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:00.499277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.797604+0000","last_change":"2023-12-19T16:42:27.327321+0000","last_active":"2023-12-19T16:42:27.797604+0000","last_peered":"2023-12-19T16:42:27.797604+0000","last_clean":"2023-12-19T16:42:27.797604+0000","last_became_active":"2023-12-19T16:42:27.327010+0000","last_became_peered":"2023-12-19T16:42:27.327010+0000","last_unstale":"2023-12-19T16:42:27.797604+0000","last_undegraded":"2023-12-19T16:42:27.797604+0000","last_fullsized":"2023-12-19T16:42:27.797604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.810813+0000","last_change":"2023-12-19T16:42:27.802521+0000","last_active":"2023-12-19T16:42:27.810813+0000","last_peered":"2023-12-19T16:42:27.810813+0000","last_clean":"2023-12-19T16:42:27.810813+0000","last_became_active":"2023-12-19T16:42:27.801786+0000","last_became_peered":"2023-12-19T16:42:27.801786+0000","last_unstale":"2023-12-19T16:42:27.810813+0000","last_undegraded":"2023-12-19T16:42:27.810813+0000","last_fullsized":"2023-12-19T16:42:27.810813+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:16.399823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784006+0000","last_change":"2023-12-19T16:42:27.311025+0000","last_active":"2023-12-19T16:42:28.784006+0000","last_peered":"2023-12-19T16:42:28.784006+0000","last_clean":"2023-12-19T16:42:28.784006+0000","last_became_active":"2023-12-19T16:42:27.310676+0000","last_became_peered":"2023-12-19T16:42:27.310676+0000","last_unstale":"2023-12-19T16:42:28.784006+0000","last_undegraded":"2023-12-19T16:42:28.784006+0000","last_fullsized":"2023-12-19T16:42:28.784006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830676+0000","last_change":"2023-12-19T16:42:27.821468+0000","last_active":"2023-12-19T16:42:27.830676+0000","last_peered":"2023-12-19T16:42:27.830676+0000","last_clean":"2023-12-19T16:42:27.830676+0000","last_became_active":"2023-12-19T16:42:27.821113+0000","last_became_peered":"2023-12-19T16:42:27.821113+0000","last_unstale":"2023-12-19T16:42:27.830676+0000","last_undegraded":"2023-12-19T16:42:27.830676+0000","last_fullsized":"2023-12-19T16:42:27.830676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:03.390272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790606+0000","last_change":"2023-12-19T16:42:27.323313+0000","last_active":"2023-12-19T16:42:28.790606+0000","last_peered":"2023-12-19T16:42:28.790606+0000","last_clean":"2023-12-19T16:42:28.790606+0000","last_became_active":"2023-12-19T16:42:27.319464+0000","last_became_peered":"2023-12-19T16:42:27.319464+0000","last_unstale":"2023-12-19T16:42:28.790606+0000","last_undegraded":"2023-12-19T16:42:28.790606+0000","last_fullsized":"2023-12-19T16:42:28.790606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:22:50.990280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830635+0000","last_change":"2023-12-19T16:42:27.821587+0000","last_active":"2023-12-19T16:42:27.830635+0000","last_peered":"2023-12-19T16:42:27.830635+0000","last_clean":"2023-12-19T16:42:27.830635+0000","last_became_active":"2023-12-19T16:42:27.821251+0000","last_became_peered":"2023-12-19T16:42:27.821251+0000","last_unstale":"2023-12-19T16:42:27.830635+0000","last_undegraded":"2023-12-19T16:42:27.830635+0000","last_fullsized":"2023-12-19T16:42:27.830635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:18.209972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784184+0000","last_change":"2023-12-19T16:42:27.319966+0000","last_active":"2023-12-19T16:42:28.784184+0000","last_peered":"2023-12-19T16:42:28.784184+0000","last_clean":"2023-12-19T16:42:28.784184+0000","last_became_active":"2023-12-19T16:42:27.319141+0000","last_became_peered":"2023-12-19T16:42:27.319141+0000","last_unstale":"2023-12-19T16:42:28.784184+0000","last_undegraded":"2023-12-19T16:42:28.784184+0000","last_fullsized":"2023-12-19T16:42:28.784184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830698+0000","last_change":"2023-12-19T16:42:27.824936+0000","last_active":"2023-12-19T16:42:27.830698+0000","last_peered":"2023-12-19T16:42:27.830698+0000","last_clean":"2023-12-19T16:42:27.830698+0000","last_became_active":"2023-12-19T16:42:27.824643+0000","last_became_peered":"2023-12-19T16:42:27.824643+0000","last_unstale":"2023-12-19T16:42:27.830698+0000","last_undegraded":"2023-12-19T16:42:27.830698+0000","last_fullsized":"2023-12-19T16:42:27.830698+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:23.006980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955939+0000","last_change":"2023-12-19T16:42:27.332207+0000","last_active":"2023-12-19T16:42:28.955939+0000","last_peered":"2023-12-19T16:42:28.955939+0000","last_clean":"2023-12-19T16:42:28.955939+0000","last_became_active":"2023-12-19T16:42:27.331920+0000","last_became_peered":"2023-12-19T16:42:27.331920+0000","last_unstale":"2023-12-19T16:42:28.955939+0000","last_undegraded":"2023-12-19T16:42:28.955939+0000","last_fullsized":"2023-12-19T16:42:28.955939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.708383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790779+0000","last_change":"2023-12-19T16:42:27.803564+0000","last_active":"2023-12-19T16:42:28.790779+0000","last_peered":"2023-12-19T16:42:28.790779+0000","last_clean":"2023-12-19T16:42:28.790779+0000","last_became_active":"2023-12-19T16:42:27.803291+0000","last_became_peered":"2023-12-19T16:42:27.803291+0000","last_unstale":"2023-12-19T16:42:28.790779+0000","last_undegraded":"2023-12-19T16:42:28.790779+0000","last_fullsized":"2023-12-19T16:42:28.790779+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:49.764257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784115+0000","last_change":"2023-12-19T16:42:27.316666+0000","last_active":"2023-12-19T16:42:28.784115+0000","last_peered":"2023-12-19T16:42:28.784115+0000","last_clean":"2023-12-19T16:42:28.784115+0000","last_became_active":"2023-12-19T16:42:27.316322+0000","last_became_peered":"2023-12-19T16:42:27.316322+0000","last_unstale":"2023-12-19T16:42:28.784115+0000","last_undegraded":"2023-12-19T16:42:28.784115+0000","last_fullsized":"2023-12-19T16:42:28.784115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.831678+0000","last_change":"2023-12-19T16:42:27.828580+0000","last_active":"2023-12-19T16:42:27.831678+0000","last_peered":"2023-12-19T16:42:27.831678+0000","last_clean":"2023-12-19T16:42:27.831678+0000","last_became_active":"2023-12-19T16:42:27.828263+0000","last_became_peered":"2023-12-19T16:42:27.828263+0000","last_unstale":"2023-12-19T16:42:27.831678+0000","last_undegraded":"2023-12-19T16:42:27.831678+0000","last_fullsized":"2023-12-19T16:42:27.831678+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:15.012739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":163,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.318071+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:42:27.318071+0000","last_peered":"2023-12-19T16:42:27.318071+0000","last_clean":"2023-12-19T16:42:27.318071+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:42:27.318071+0000","last_undegraded":"2023-12-19T16:42:27.318071+0000","last_fullsized":"2023-12-19T16:42:27.318071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790309+0000","last_change":"2023-12-19T16:42:27.322949+0000","last_active":"2023-12-19T16:42:28.790309+0000","last_peered":"2023-12-19T16:42:28.790309+0000","last_clean":"2023-12-19T16:42:28.790309+0000","last_became_active":"2023-12-19T16:42:27.321633+0000","last_became_peered":"2023-12-19T16:42:27.321633+0000","last_unstale":"2023-12-19T16:42:28.790309+0000","last_undegraded":"2023-12-19T16:42:28.790309+0000","last_fullsized":"2023-12-19T16:42:28.790309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.811675+0000","last_change":"2023-12-19T16:42:27.798689+0000","last_active":"2023-12-19T16:42:27.811675+0000","last_peered":"2023-12-19T16:42:27.811675+0000","last_clean":"2023-12-19T16:42:27.811675+0000","last_became_active":"2023-12-19T16:42:27.798430+0000","last_became_peered":"2023-12-19T16:42:27.798430+0000","last_unstale":"2023-12-19T16:42:27.811675+0000","last_undegraded":"2023-12-19T16:42:27.811675+0000","last_fullsized":"2023-12-19T16:42:27.811675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:16.399823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.801228+0000","last_change":"2023-12-19T16:42:27.326882+0000","last_active":"2023-12-19T16:42:27.801228+0000","last_peered":"2023-12-19T16:42:27.801228+0000","last_clean":"2023-12-19T16:42:27.801228+0000","last_became_active":"2023-12-19T16:42:27.326616+0000","last_became_peered":"2023-12-19T16:42:27.326616+0000","last_unstale":"2023-12-19T16:42:27.801228+0000","last_undegraded":"2023-12-19T16:42:27.801228+0000","last_fullsized":"2023-12-19T16:42:27.801228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:36.280588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.819374+0000","last_change":"2023-12-19T16:42:27.816279+0000","last_active":"2023-12-19T16:42:27.819374+0000","last_peered":"2023-12-19T16:42:27.819374+0000","last_clean":"2023-12-19T16:42:27.819374+0000","last_became_active":"2023-12-19T16:42:27.815940+0000","last_became_peered":"2023-12-19T16:42:27.815940+0000","last_unstale":"2023-12-19T16:42:27.819374+0000","last_undegraded":"2023-12-19T16:42:27.819374+0000","last_fullsized":"2023-12-19T16:42:27.819374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:51:00.190122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784142+0000","last_change":"2023-12-19T16:42:27.317414+0000","last_active":"2023-12-19T16:42:28.784142+0000","last_peered":"2023-12-19T16:42:28.784142+0000","last_clean":"2023-12-19T16:42:28.784142+0000","last_became_active":"2023-12-19T16:42:27.317065+0000","last_became_peered":"2023-12-19T16:42:27.317065+0000","last_unstale":"2023-12-19T16:42:28.784142+0000","last_undegraded":"2023-12-19T16:42:28.784142+0000","last_fullsized":"2023-12-19T16:42:28.784142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955832+0000","last_change":"2023-12-19T16:42:27.794867+0000","last_active":"2023-12-19T16:42:28.955832+0000","last_peered":"2023-12-19T16:42:28.955832+0000","last_clean":"2023-12-19T16:42:28.955832+0000","last_became_active":"2023-12-19T16:42:27.793788+0000","last_became_peered":"2023-12-19T16:42:27.793788+0000","last_unstale":"2023-12-19T16:42:28.955832+0000","last_undegraded":"2023-12-19T16:42:28.955832+0000","last_fullsized":"2023-12-19T16:42:28.955832+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:58:03.549430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.791093+0000","last_change":"2023-12-19T16:42:27.312696+0000","last_active":"2023-12-19T16:42:28.791093+0000","last_peered":"2023-12-19T16:42:28.791093+0000","last_clean":"2023-12-19T16:42:28.791093+0000","last_became_active":"2023-12-19T16:42:27.312406+0000","last_became_peered":"2023-12-19T16:42:27.312406+0000","last_unstale":"2023-12-19T16:42:28.791093+0000","last_undegraded":"2023-12-19T16:42:28.791093+0000","last_fullsized":"2023-12-19T16:42:28.791093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:41.116483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789955+0000","last_change":"2023-12-19T16:42:27.813887+0000","last_active":"2023-12-19T16:42:28.789955+0000","last_peered":"2023-12-19T16:42:28.789955+0000","last_clean":"2023-12-19T16:42:28.789955+0000","last_became_active":"2023-12-19T16:42:27.813476+0000","last_became_peered":"2023-12-19T16:42:27.813476+0000","last_unstale":"2023-12-19T16:42:28.789955+0000","last_undegraded":"2023-12-19T16:42:28.789955+0000","last_fullsized":"2023-12-19T16:42:28.789955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:36.975084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955856+0000","last_change":"2023-12-19T16:42:27.302950+0000","last_active":"2023-12-19T16:42:28.955856+0000","last_peered":"2023-12-19T16:42:28.955856+0000","last_clean":"2023-12-19T16:42:28.955856+0000","last_became_active":"2023-12-19T16:42:27.302614+0000","last_became_peered":"2023-12-19T16:42:27.302614+0000","last_unstale":"2023-12-19T16:42:28.955856+0000","last_undegraded":"2023-12-19T16:42:28.955856+0000","last_fullsized":"2023-12-19T16:42:28.955856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790926+0000","last_change":"2023-12-19T16:42:27.803672+0000","last_active":"2023-12-19T16:42:28.790926+0000","last_peered":"2023-12-19T16:42:28.790926+0000","last_clean":"2023-12-19T16:42:28.790926+0000","last_became_active":"2023-12-19T16:42:27.803402+0000","last_became_peered":"2023-12-19T16:42:27.803402+0000","last_unstale":"2023-12-19T16:42:28.790926+0000","last_undegraded":"2023-12-19T16:42:28.790926+0000","last_fullsized":"2023-12-19T16:42:28.790926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:48:49.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.804106+0000","last_change":"2023-12-19T16:42:27.330785+0000","last_active":"2023-12-19T16:42:27.804106+0000","last_peered":"2023-12-19T16:42:27.804106+0000","last_clean":"2023-12-19T16:42:27.804106+0000","last_became_active":"2023-12-19T16:42:27.329823+0000","last_became_peered":"2023-12-19T16:42:27.329823+0000","last_unstale":"2023-12-19T16:42:27.804106+0000","last_undegraded":"2023-12-19T16:42:27.804106+0000","last_fullsized":"2023-12-19T16:42:27.804106+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955776+0000","last_change":"2023-12-19T16:42:27.794649+0000","last_active":"2023-12-19T16:42:28.955776+0000","last_peered":"2023-12-19T16:42:28.955776+0000","last_clean":"2023-12-19T16:42:28.955776+0000","last_became_active":"2023-12-19T16:42:27.793710+0000","last_became_peered":"2023-12-19T16:42:27.793710+0000","last_unstale":"2023-12-19T16:42:28.955776+0000","last_undegraded":"2023-12-19T16:42:28.955776+0000","last_fullsized":"2023-12-19T16:42:28.955776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:16.399823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955987+0000","last_change":"2023-12-19T16:42:27.321060+0000","last_active":"2023-12-19T16:42:28.955987+0000","last_peered":"2023-12-19T16:42:28.955987+0000","last_clean":"2023-12-19T16:42:28.955987+0000","last_became_active":"2023-12-19T16:42:27.320705+0000","last_became_peered":"2023-12-19T16:42:27.320705+0000","last_unstale":"2023-12-19T16:42:28.955987+0000","last_undegraded":"2023-12-19T16:42:28.955987+0000","last_fullsized":"2023-12-19T16:42:28.955987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:35.968756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.819267+0000","last_change":"2023-12-19T16:42:27.818162+0000","last_active":"2023-12-19T16:42:27.819267+0000","last_peered":"2023-12-19T16:42:27.819267+0000","last_clean":"2023-12-19T16:42:27.819267+0000","last_became_active":"2023-12-19T16:42:27.817894+0000","last_became_peered":"2023-12-19T16:42:27.817894+0000","last_unstale":"2023-12-19T16:42:27.819267+0000","last_undegraded":"2023-12-19T16:42:27.819267+0000","last_fullsized":"2023-12-19T16:42:27.819267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:29.376069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.825018+0000","last_change":"2023-12-19T16:42:27.330277+0000","last_active":"2023-12-19T16:42:27.825018+0000","last_peered":"2023-12-19T16:42:27.825018+0000","last_clean":"2023-12-19T16:42:27.825018+0000","last_became_active":"2023-12-19T16:42:27.330006+0000","last_became_peered":"2023-12-19T16:42:27.330006+0000","last_unstale":"2023-12-19T16:42:27.825018+0000","last_undegraded":"2023-12-19T16:42:27.825018+0000","last_fullsized":"2023-12-19T16:42:27.825018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.799477+0000","last_change":"2023-12-19T16:42:27.791761+0000","last_active":"2023-12-19T16:42:27.799477+0000","last_peered":"2023-12-19T16:42:27.799477+0000","last_clean":"2023-12-19T16:42:27.799477+0000","last_became_active":"2023-12-19T16:42:27.791490+0000","last_became_peered":"2023-12-19T16:42:27.791490+0000","last_unstale":"2023-12-19T16:42:27.799477+0000","last_undegraded":"2023-12-19T16:42:27.799477+0000","last_fullsized":"2023-12-19T16:42:27.799477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:48:40.695901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.794541+0000","last_change":"2023-12-19T16:42:27.326363+0000","last_active":"2023-12-19T16:42:27.794541+0000","last_peered":"2023-12-19T16:42:27.794541+0000","last_clean":"2023-12-19T16:42:27.794541+0000","last_became_active":"2023-12-19T16:42:27.326046+0000","last_became_peered":"2023-12-19T16:42:27.326046+0000","last_unstale":"2023-12-19T16:42:27.794541+0000","last_undegraded":"2023-12-19T16:42:27.794541+0000","last_fullsized":"2023-12-19T16:42:27.794541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:41.116483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789733+0000","last_change":"2023-12-19T16:42:27.808612+0000","last_active":"2023-12-19T16:42:28.789733+0000","last_peered":"2023-12-19T16:42:28.789733+0000","last_clean":"2023-12-19T16:42:28.789733+0000","last_became_active":"2023-12-19T16:42:27.808219+0000","last_became_peered":"2023-12-19T16:42:27.808219+0000","last_unstale":"2023-12-19T16:42:28.789733+0000","last_undegraded":"2023-12-19T16:42:28.789733+0000","last_fullsized":"2023-12-19T16:42:28.789733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:37.286916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955811+0000","last_change":"2023-12-19T16:42:27.308462+0000","last_active":"2023-12-19T16:42:28.955811+0000","last_peered":"2023-12-19T16:42:28.955811+0000","last_clean":"2023-12-19T16:42:28.955811+0000","last_became_active":"2023-12-19T16:42:27.307879+0000","last_became_peered":"2023-12-19T16:42:27.307879+0000","last_unstale":"2023-12-19T16:42:28.955811+0000","last_undegraded":"2023-12-19T16:42:28.955811+0000","last_fullsized":"2023-12-19T16:42:28.955811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.811504+0000","last_change":"2023-12-19T16:42:27.798674+0000","last_active":"2023-12-19T16:42:27.811504+0000","last_peered":"2023-12-19T16:42:27.811504+0000","last_clean":"2023-12-19T16:42:27.811504+0000","last_became_active":"2023-12-19T16:42:27.798442+0000","last_became_peered":"2023-12-19T16:42:27.798442+0000","last_unstale":"2023-12-19T16:42:27.811504+0000","last_undegraded":"2023-12-19T16:42:27.811504+0000","last_fullsized":"2023-12-19T16:42:27.811504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:01.122709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955925+0000","last_change":"2023-12-19T16:42:27.310942+0000","last_active":"2023-12-19T16:42:28.955925+0000","last_peered":"2023-12-19T16:42:28.955925+0000","last_clean":"2023-12-19T16:42:28.955925+0000","last_became_active":"2023-12-19T16:42:27.310605+0000","last_became_peered":"2023-12-19T16:42:27.310605+0000","last_unstale":"2023-12-19T16:42:28.955925+0000","last_undegraded":"2023-12-19T16:42:28.955925+0000","last_fullsized":"2023-12-19T16:42:28.955925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.814364+0000","last_change":"2023-12-19T16:42:27.802676+0000","last_active":"2023-12-19T16:42:27.814364+0000","last_peered":"2023-12-19T16:42:27.814364+0000","last_clean":"2023-12-19T16:42:27.814364+0000","last_became_active":"2023-12-19T16:42:27.801627+0000","last_became_peered":"2023-12-19T16:42:27.801627+0000","last_unstale":"2023-12-19T16:42:27.814364+0000","last_undegraded":"2023-12-19T16:42:27.814364+0000","last_fullsized":"2023-12-19T16:42:27.814364+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:30.545767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.834491+0000","last_change":"2023-12-19T16:42:27.334084+0000","last_active":"2023-12-19T16:42:27.834491+0000","last_peered":"2023-12-19T16:42:27.834491+0000","last_clean":"2023-12-19T16:42:27.834491+0000","last_became_active":"2023-12-19T16:42:27.333748+0000","last_became_peered":"2023-12-19T16:42:27.333748+0000","last_unstale":"2023-12-19T16:42:27.834491+0000","last_undegraded":"2023-12-19T16:42:27.834491+0000","last_fullsized":"2023-12-19T16:42:27.834491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.822317+0000","last_change":"2023-12-19T16:42:27.817078+0000","last_active":"2023-12-19T16:42:27.822317+0000","last_peered":"2023-12-19T16:42:27.822317+0000","last_clean":"2023-12-19T16:42:27.822317+0000","last_became_active":"2023-12-19T16:42:27.816835+0000","last_became_peered":"2023-12-19T16:42:27.816835+0000","last_unstale":"2023-12-19T16:42:27.822317+0000","last_undegraded":"2023-12-19T16:42:27.822317+0000","last_fullsized":"2023-12-19T16:42:27.822317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790847+0000","last_change":"2023-12-19T16:42:27.820348+0000","last_active":"2023-12-19T16:42:28.790847+0000","last_peered":"2023-12-19T16:42:28.790847+0000","last_clean":"2023-12-19T16:42:28.790847+0000","last_became_active":"2023-12-19T16:42:27.820051+0000","last_became_peered":"2023-12-19T16:42:27.820051+0000","last_unstale":"2023-12-19T16:42:28.790847+0000","last_undegraded":"2023-12-19T16:42:28.790847+0000","last_fullsized":"2023-12-19T16:42:28.790847+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:23.006980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.829846+0000","last_change":"2023-12-19T16:42:27.328852+0000","last_active":"2023-12-19T16:42:27.829846+0000","last_peered":"2023-12-19T16:42:27.829846+0000","last_clean":"2023-12-19T16:42:27.829846+0000","last_became_active":"2023-12-19T16:42:27.328521+0000","last_became_peered":"2023-12-19T16:42:27.328521+0000","last_unstale":"2023-12-19T16:42:27.829846+0000","last_undegraded":"2023-12-19T16:42:27.829846+0000","last_fullsized":"2023-12-19T16:42:27.829846+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:35.968756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790575+0000","last_change":"2023-12-19T16:42:27.820348+0000","last_active":"2023-12-19T16:42:28.790575+0000","last_peered":"2023-12-19T16:42:28.790575+0000","last_clean":"2023-12-19T16:42:28.790575+0000","last_became_active":"2023-12-19T16:42:27.820018+0000","last_became_peered":"2023-12-19T16:42:27.820018+0000","last_unstale":"2023-12-19T16:42:28.790575+0000","last_undegraded":"2023-12-19T16:42:28.790575+0000","last_fullsized":"2023-12-19T16:42:28.790575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.830692+0000","last_change":"2023-12-19T16:42:27.331867+0000","last_active":"2023-12-19T16:42:27.830692+0000","last_peered":"2023-12-19T16:42:27.830692+0000","last_clean":"2023-12-19T16:42:27.830692+0000","last_became_active":"2023-12-19T16:42:27.331354+0000","last_became_peered":"2023-12-19T16:42:27.331354+0000","last_unstale":"2023-12-19T16:42:27.830692+0000","last_undegraded":"2023-12-19T16:42:27.830692+0000","last_fullsized":"2023-12-19T16:42:27.830692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:47:06.561268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.810018+0000","last_change":"2023-12-19T16:42:27.800529+0000","last_active":"2023-12-19T16:42:27.810018+0000","last_peered":"2023-12-19T16:42:27.810018+0000","last_clean":"2023-12-19T16:42:27.810018+0000","last_became_active":"2023-12-19T16:42:27.800250+0000","last_became_peered":"2023-12-19T16:42:27.800250+0000","last_unstale":"2023-12-19T16:42:27.810018+0000","last_undegraded":"2023-12-19T16:42:27.810018+0000","last_fullsized":"2023-12-19T16:42:27.810018+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:27.376917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.814552+0000","last_change":"2023-12-19T16:42:27.314367+0000","last_active":"2023-12-19T16:42:27.814552+0000","last_peered":"2023-12-19T16:42:27.814552+0000","last_clean":"2023-12-19T16:42:27.814552+0000","last_became_active":"2023-12-19T16:42:27.314128+0000","last_became_peered":"2023-12-19T16:42:27.314128+0000","last_unstale":"2023-12-19T16:42:27.814552+0000","last_undegraded":"2023-12-19T16:42:27.814552+0000","last_fullsized":"2023-12-19T16:42:27.814552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:34:59.521223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.814525+0000","last_change":"2023-12-19T16:42:27.804285+0000","last_active":"2023-12-19T16:42:27.814525+0000","last_peered":"2023-12-19T16:42:27.814525+0000","last_clean":"2023-12-19T16:42:27.814525+0000","last_became_active":"2023-12-19T16:42:27.804027+0000","last_became_peered":"2023-12-19T16:42:27.804027+0000","last_unstale":"2023-12-19T16:42:27.814525+0000","last_undegraded":"2023-12-19T16:42:27.814525+0000","last_fullsized":"2023-12-19T16:42:27.814525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:53.498191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.827895+0000","last_change":"2023-12-19T16:42:27.325512+0000","last_active":"2023-12-19T16:42:27.827895+0000","last_peered":"2023-12-19T16:42:27.827895+0000","last_clean":"2023-12-19T16:42:27.827895+0000","last_became_active":"2023-12-19T16:42:27.325175+0000","last_became_peered":"2023-12-19T16:42:27.325175+0000","last_unstale":"2023-12-19T16:42:27.827895+0000","last_undegraded":"2023-12-19T16:42:27.827895+0000","last_fullsized":"2023-12-19T16:42:27.827895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:31:30.386546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":800,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":819200,"data_stored":665561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":233472,"data_stored":76175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":81001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":81037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31276,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340564,"statfs":{"total":96636764160,"available":96604737536,"internally_reserved":0,"allocated":233472,"data_stored":76175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31276,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340564,"statfs":{"total":96636764160,"available":96604737536,"internally_reserved":0,"allocated":233472,"data_stored":80911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31272,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340568,"statfs":{"total":96636764160,"available":96604741632,"internally_reserved":0,"allocated":229376,"data_stored":74771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":802816,"data_stored":658204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:42:32.297 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T16:42:32.298 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-19T16:42:32.298 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-12-19T16:42:32.298 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-19T16:42:32.299 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-12-19T16:42:32.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.472+0000 7f6d6202d640 1 Processor -- start 2023-12-19T16:42:32.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.472+0000 7f6d6202d640 1 -- start start 2023-12-19T16:42:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.476+0000 7f6d6202d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d5c06cb30 0x7f6d5c0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.476+0000 7f6d6202d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d5c0b34e0 0x7f6d5c15d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.476+0000 7f6d6202d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5c15ddc0 con 0x7f6d5c15d2b0 2023-12-19T16:42:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.476+0000 7f6d6202d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5c15e0a0 con 0x7f6d5c06cb30 2023-12-19T16:42:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.476+0000 7f6d6202d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0cb450 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.476+0000 7f6d5affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6d5c15d2b0 0x7f6d5c0b2dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50240/0 (socket says 172.21.15.26:50240) 2023-12-19T16:42:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.476+0000 7f6d5b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d5c06cb30 0x7f6d5c0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.476+0000 7f6d5affd640 1 -- 172.21.15.26:0/340814656 learned_addr learned my addr 172.21.15.26:0/340814656 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.476+0000 7f6d5b7fe640 1 -- 172.21.15.26:0/340814656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d5c0b34e0 msgr2=0x7f6d5c15d730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-19T16:42:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.476+0000 7f6d5b7fe640 1 --2- 172.21.15.26:0/340814656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d5c0b34e0 0x7f6d5c15d730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.476+0000 7f6d5b7fe640 1 -- 172.21.15.26:0/340814656 >> v1:172.21.15.26:6789/0 conn(0x7f6d5c15d2b0 legacy=0x7f6d5c0b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.476+0000 7f6d5b7fe640 1 -- 172.21.15.26:0/340814656 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c0cb730 con 0x7f6d5c06cb30 2023-12-19T16:42:32.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.476+0000 7f6d5a7fc640 1 -- 172.21.15.26:0/340814656 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868717580 0 0) 0x7f6d5c15ddc0 con 0x7f6d5c15d2b0 2023-12-19T16:42:32.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d5b7fe640 1 --2- 172.21.15.26:0/340814656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c06cb30 0x7f6d5c0cbc50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0097c0 tx=0x7f6d4c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab255ecf9857a8b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:32.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d5a7fc640 1 -- 172.21.15.26:0/340814656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0ab5f0 con 0x7f6d5c06cb30 2023-12-19T16:42:32.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d5a7fc640 1 -- 172.21.15.26:0/340814656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c0ab750 con 0x7f6d5c06cb30 2023-12-19T16:42:32.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d5a7fc640 1 -- 172.21.15.26:0/340814656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0abae0 con 0x7f6d5c06cb30 2023-12-19T16:42:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d6202d640 1 -- 172.21.15.26:0/340814656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c06cb30 msgr2=0x7f6d5c0cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d6202d640 1 --2- 172.21.15.26:0/340814656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c06cb30 0x7f6d5c0cbc50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0097c0 tx=0x7f6d4c01c4a0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d6202d640 1 -- 172.21.15.26:0/340814656 shutdown_connections 2023-12-19T16:42:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d6202d640 1 --2- 172.21.15.26:0/340814656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d5c0b34e0 0x7f6d5c15d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d6202d640 1 --2- 172.21.15.26:0/340814656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c06cb30 0x7f6d5c0cbc50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d6202d640 1 -- 172.21.15.26:0/340814656 >> 172.21.15.26:0/340814656 conn(0x7f6d5c0658e0 msgr2=0x7f6d5c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d6202d640 1 -- 172.21.15.26:0/340814656 shutdown_connections 2023-12-19T16:42:32.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d6202d640 1 -- 172.21.15.26:0/340814656 wait complete. 2023-12-19T16:42:32.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d6202d640 1 Processor -- start 2023-12-19T16:42:32.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.480+0000 7f6d6202d640 1 -- start start 2023-12-19T16:42:32.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d6202d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d5c06cb30 0x7f6d5c0caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d6202d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c0b34e0 0x7f6d5c0b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d6202d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d5c15d2b0 0x7f6d5c13dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d6202d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0caf40 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d6202d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c143d30 con 0x7f6d5c15d2b0 2023-12-19T16:42:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d6202d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c144010 con 0x7f6d5c06cb30 2023-12-19T16:42:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d5affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c0b34e0 0x7f6d5c0b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d5affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c0b34e0 0x7f6d5c0b0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49186/0 (socket says 172.21.15.26:49186) 2023-12-19T16:42:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d5affd640 1 -- 172.21.15.26:0/906318596 learned_addr learned my addr 172.21.15.26:0/906318596 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d5b7fe640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d5c06cb30 0x7f6d5c0caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d5bfff640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d5c15d2b0 0x7f6d5c13dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d5affd640 1 -- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d5c06cb30 msgr2=0x7f6d5c0caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d5affd640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d5c06cb30 0x7f6d5c0caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d5affd640 1 -- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d5c15d2b0 msgr2=0x7f6d5c13dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d5affd640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d5c15d2b0 0x7f6d5c13dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d5affd640 1 -- 172.21.15.26:0/906318596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c009420 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d5affd640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c0b34e0 0x7f6d5c0b0250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c480 tx=0x7f6d5000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:32.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d43fff640 1 -- 172.21.15.26:0/906318596 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d500074e0 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d43fff640 1 -- 172.21.15.26:0/906318596 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d50007660 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d43fff640 1 -- 172.21.15.26:0/906318596 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50004a30 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d6202d640 1 -- 172.21.15.26:0/906318596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5c1442f0 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d6202d640 1 -- 172.21.15.26:0/906318596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5c1448b0 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.484+0000 7f6d6202d640 1 -- 172.21.15.26:0/906318596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5c144ad0 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.488+0000 7f6d43fff640 1 -- 172.21.15.26:0/906318596 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6d50004b90 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.488+0000 7f6d43fff640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6d48065a60 0x7f6d48067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.488+0000 7f6d43fff640 1 -- 172.21.15.26:0/906318596 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d500822b0 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.488+0000 7f6d5b7fe640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6d48065a60 0x7f6d48067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.488+0000 7f6d5b7fe640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6d48065a60 0x7f6d48067f20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0af000 tx=0x7f6d4c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:32.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.492+0000 7f6d43fff640 1 -- 172.21.15.26:0/906318596 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6d5004fd10 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.688+0000 7f6d6202d640 1 -- 172.21.15.26:0/906318596 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6d5c161490 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.688+0000 7f6d43fff640 1 -- 172.21.15.26:0/906318596 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163368 (secure 0 0 0) 0x7f6d50053330 con 0x7f6d5c0b34e0 2023-12-19T16:42:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:32.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.692+0000 7f6d6202d640 1 -- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6d48065a60 msgr2=0x7f6d48067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.692+0000 7f6d6202d640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6d48065a60 0x7f6d48067f20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c0af000 tx=0x7f6d4c0a7000 comp rx=0 tx=0).stop 2023-12-19T16:42:32.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.692+0000 7f6d6202d640 1 -- 172.21.15.26:0/906318596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c0b34e0 msgr2=0x7f6d5c0b0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.692+0000 7f6d6202d640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c0b34e0 0x7f6d5c0b0250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c480 tx=0x7f6d5000c940 comp rx=0 tx=0).stop 2023-12-19T16:42:32.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.692+0000 7f6d6202d640 1 -- 172.21.15.26:0/906318596 shutdown_connections 2023-12-19T16:42:32.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.692+0000 7f6d6202d640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d5c15d2b0 0x7f6d5c13dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.692+0000 7f6d6202d640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d5c0b34e0 0x7f6d5c0b0250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.692+0000 7f6d6202d640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6d48065a60 0x7f6d48067f20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.692+0000 7f6d6202d640 1 --2- 172.21.15.26:0/906318596 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d5c06cb30 0x7f6d5c0caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.692+0000 7f6d6202d640 1 -- 172.21.15.26:0/906318596 >> 172.21.15.26:0/906318596 conn(0x7f6d5c0658e0 msgr2=0x7f6d5c160d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:32.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.692+0000 7f6d6202d640 1 -- 172.21.15.26:0/906318596 shutdown_connections 2023-12-19T16:42:32.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.692+0000 7f6d6202d640 1 -- 172.21.15.26:0/906318596 wait complete. 2023-12-19T16:42:32.716 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":5,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":38098},{"type":"v1","addr":"172.21.15.160:6833","nonce":38098}]},"active_addr":"172.21.15.160:6833/38098","active_change":"2023-12-19T16:42:07.621290+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":1318901201}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":713602764}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":810298028}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":1246358820}]}]} 2023-12-19T16:42:32.721 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-12-19T16:42:32.721 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-12-19T16:42:32.721 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.860+0000 7fe994e66640 1 Processor -- start 2023-12-19T16:42:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.860+0000 7fe994e66640 1 -- start start 2023-12-19T16:42:32.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.860+0000 7fe994e66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe99006c7f0 0x7fe9900ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.860+0000 7fe994e66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe9900cc6e0 0x7fe9900ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.860+0000 7fe994e66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99006cc80 con 0x7fe99015c7f0 2023-12-19T16:42:32.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.860+0000 7fe994e66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99006d770 con 0x7fe9900cc6e0 2023-12-19T16:42:32.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.860+0000 7fe994e66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99006ced0 con 0x7fe99006c7f0 2023-12-19T16:42:32.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.860+0000 7fe98e59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe99015c7f0 0x7fe9900adb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50252/0 (socket says 172.21.15.26:50252) 2023-12-19T16:42:32.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.860+0000 7fe98e59c640 1 -- 172.21.15.26:0/1440907508 learned_addr learned my addr 172.21.15.26:0/1440907508 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:32.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.860+0000 7fe98ed9d640 1 --2- 172.21.15.26:0/1440907508 >> v2:172.21.15.26:3300/0 conn(0x7fe9900cc6e0 0x7fe9900ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.864+0000 7fe987fff640 1 --2- 172.21.15.26:0/1440907508 >> v2:172.21.15.160:3300/0 conn(0x7fe99006c7f0 0x7fe9900ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.864+0000 7fe98dd9b640 1 -- 172.21.15.26:0/1440907508 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 405467631 0 0) 0x7fe99006cc80 con 0x7fe99015c7f0 2023-12-19T16:42:32.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.864+0000 7fe98dd9b640 1 -- 172.21.15.26:0/1440907508 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe97c003530 con 0x7fe99015c7f0 2023-12-19T16:42:32.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.864+0000 7fe98ed9d640 1 -- 172.21.15.26:0/1440907508 >> v2:172.21.15.160:3300/0 conn(0x7fe99006c7f0 msgr2=0x7fe9900ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:32.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.864+0000 7fe98ed9d640 1 --2- 172.21.15.26:0/1440907508 >> v2:172.21.15.160:3300/0 conn(0x7fe99006c7f0 0x7fe9900ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.864+0000 7fe98ed9d640 1 -- 172.21.15.26:0/1440907508 >> v1:172.21.15.26:6789/0 conn(0x7fe99015c7f0 legacy=0x7fe9900adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.864+0000 7fe98ed9d640 1 -- 172.21.15.26:0/1440907508 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99006d1b0 con 0x7fe9900cc6e0 2023-12-19T16:42:32.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.864+0000 7fe98ed9d640 1 --2- 172.21.15.26:0/1440907508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9900cc6e0 0x7fe9900ccaf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe9900667b0 tx=0x7fe98801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663970b371481f7d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:32.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.864+0000 7fe98dd9b640 1 -- 172.21.15.26:0/1440907508 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9880a58c0 con 0x7fe9900cc6e0 2023-12-19T16:42:32.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.864+0000 7fe98dd9b640 1 -- 172.21.15.26:0/1440907508 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9880a5a20 con 0x7fe9900cc6e0 2023-12-19T16:42:32.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.864+0000 7fe98dd9b640 1 -- 172.21.15.26:0/1440907508 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9880a5db0 con 0x7fe9900cc6e0 2023-12-19T16:42:32.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.864+0000 7fe994e66640 1 -- 172.21.15.26:0/1440907508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9900cc6e0 msgr2=0x7fe9900ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.864+0000 7fe994e66640 1 --2- 172.21.15.26:0/1440907508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9900cc6e0 0x7fe9900ccaf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe9900667b0 tx=0x7fe98801c910 comp rx=0 tx=0).stop 2023-12-19T16:42:32.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 -- 172.21.15.26:0/1440907508 shutdown_connections 2023-12-19T16:42:32.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 --2- 172.21.15.26:0/1440907508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9900cc6e0 0x7fe9900ccaf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 --2- 172.21.15.26:0/1440907508 >> v2:172.21.15.160:3300/0 conn(0x7fe99006c7f0 0x7fe9900ce7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 -- 172.21.15.26:0/1440907508 >> 172.21.15.26:0/1440907508 conn(0x7fe9900658e0 msgr2=0x7fe990065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:32.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 -- 172.21.15.26:0/1440907508 shutdown_connections 2023-12-19T16:42:32.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 -- 172.21.15.26:0/1440907508 wait complete. 2023-12-19T16:42:32.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 Processor -- start 2023-12-19T16:42:32.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 -- start start 2023-12-19T16:42:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99006c7f0 0x7fe9901660a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9900cc6e0 0x7fe990144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99015c7f0 0x7fe99006f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe990167600 con 0x7fe99015c7f0 2023-12-19T16:42:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe990166980 con 0x7fe9900cc6e0 2023-12-19T16:42:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe994e66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe990166c60 con 0x7fe99006c7f0 2023-12-19T16:42:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe98ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99015c7f0 0x7fe99006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe98ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99015c7f0 0x7fe99006f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49214/0 (socket says 172.21.15.26:49214) 2023-12-19T16:42:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe98ed9d640 1 -- 172.21.15.26:0/648433733 learned_addr learned my addr 172.21.15.26:0/648433733 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe98e59c640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99006c7f0 0x7fe9901660a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe987fff640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9900cc6e0 0x7fe990144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe98ed9d640 1 -- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99006c7f0 msgr2=0x7fe9901660a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe98ed9d640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99006c7f0 0x7fe9901660a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe98ed9d640 1 -- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9900cc6e0 msgr2=0x7fe990144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:32.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe98ed9d640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9900cc6e0 0x7fe990144fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:32.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe98ed9d640 1 -- 172.21.15.26:0/648433733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe988009a70 con 0x7fe99015c7f0 2023-12-19T16:42:32.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe98e59c640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99006c7f0 0x7fe9901660a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:32.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe987fff640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9900cc6e0 0x7fe990144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:32.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.868+0000 7fe98ed9d640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99015c7f0 0x7fe99006f910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe988019520 tx=0x7fe9880196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:32.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.872+0000 7fe986ffd640 1 -- 172.21.15.26:0/648433733 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9880b9860 con 0x7fe99015c7f0 2023-12-19T16:42:32.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.872+0000 7fe986ffd640 1 -- 172.21.15.26:0/648433733 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9880b99c0 con 0x7fe99015c7f0 2023-12-19T16:42:32.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.872+0000 7fe986ffd640 1 -- 172.21.15.26:0/648433733 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9880b9d70 con 0x7fe99015c7f0 2023-12-19T16:42:32.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.872+0000 7fe994e66640 1 -- 172.21.15.26:0/648433733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe990166f40 con 0x7fe99015c7f0 2023-12-19T16:42:32.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.872+0000 7fe994e66640 1 -- 172.21.15.26:0/648433733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9901665e0 con 0x7fe99015c7f0 2023-12-19T16:42:32.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.872+0000 7fe994e66640 1 -- 172.21.15.26:0/648433733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe990074270 con 0x7fe99015c7f0 2023-12-19T16:42:32.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.872+0000 7fe986ffd640 1 -- 172.21.15.26:0/648433733 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe9880ad440 con 0x7fe99015c7f0 2023-12-19T16:42:32.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.872+0000 7fe986ffd640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe970065a60 0x7fe970067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:32.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.872+0000 7fe986ffd640 1 -- 172.21.15.26:0/648433733 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe98811e690 con 0x7fe99015c7f0 2023-12-19T16:42:32.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.872+0000 7fe98e59c640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe970065a60 0x7fe970067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:32.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.876+0000 7fe98e59c640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe970065a60 0x7fe970067f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe9800011f0 tx=0x7fe980009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:32.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:32.876+0000 7fe986ffd640 1 -- 172.21.15.26:0/648433733 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe9880ec160 con 0x7fe99015c7f0 2023-12-19T16:42:33.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.036+0000 7fe994e66640 1 -- 172.21.15.26:0/648433733 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe990070fd0 con 0x7fe99015c7f0 2023-12-19T16:42:33.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.036+0000 7fe986ffd640 1 -- 172.21.15.26:0/648433733 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fe9880ef780 con 0x7fe99015c7f0 2023-12-19T16:42:33.040 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:33.040 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:33.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.036+0000 7fe994e66640 1 -- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe970065a60 msgr2=0x7fe970067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.036+0000 7fe994e66640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe970065a60 0x7fe970067f20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe9800011f0 tx=0x7fe980009040 comp rx=0 tx=0).stop 2023-12-19T16:42:33.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.036+0000 7fe994e66640 1 -- 172.21.15.26:0/648433733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99015c7f0 msgr2=0x7fe99006f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.036+0000 7fe994e66640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99015c7f0 0x7fe99006f910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe988019520 tx=0x7fe9880196b0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.036+0000 7fe994e66640 1 -- 172.21.15.26:0/648433733 shutdown_connections 2023-12-19T16:42:33.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.040+0000 7fe994e66640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99015c7f0 0x7fe99006f910 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.040+0000 7fe994e66640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9900cc6e0 0x7fe990144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.040+0000 7fe994e66640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe970065a60 0x7fe970067f20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.040+0000 7fe994e66640 1 --2- 172.21.15.26:0/648433733 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99006c7f0 0x7fe9901660a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.040+0000 7fe994e66640 1 -- 172.21.15.26:0/648433733 >> 172.21.15.26:0/648433733 conn(0x7fe9900658e0 msgr2=0x7fe9900cb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.040+0000 7fe994e66640 1 -- 172.21.15.26:0/648433733 shutdown_connections 2023-12-19T16:42:33.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.040+0000 7fe994e66640 1 -- 172.21.15.26:0/648433733 wait complete. 2023-12-19T16:42:33.062 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-12-19T16:42:33.062 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:33.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.200+0000 7ffbaaf39640 1 Processor -- start 2023-12-19T16:42:33.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.200+0000 7ffbaaf39640 1 -- start start 2023-12-19T16:42:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.200+0000 7ffbaaf39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffba40cb1d0 0x7ffba40cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffbaaf39640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffba40b1d40 0x7ffba40b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffbaaf39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba406d4b0 con 0x7ffba406bee0 2023-12-19T16:42:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffbaaf39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba40cc030 con 0x7ffba40cb1d0 2023-12-19T16:42:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffbaaf39640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba40cbaf0 con 0x7ffba40b1d40 2023-12-19T16:42:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffba8cd5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffba40cb1d0 0x7ffba40cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffba8cd5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffba40cb1d0 0x7ffba40cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49222/0 (socket says 172.21.15.26:49222) 2023-12-19T16:42:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffba8cd5640 1 -- 172.21.15.26:0/1252575035 learned_addr learned my addr 172.21.15.26:0/1252575035 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffba94d6640 1 --2- 172.21.15.26:0/1252575035 >> v2:172.21.15.160:3300/0 conn(0x7ffba40b1d40 0x7ffba40b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffba8cd5640 1 -- 172.21.15.26:0/1252575035 >> v2:172.21.15.160:3300/0 conn(0x7ffba40b1d40 msgr2=0x7ffba40b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffba8cd5640 1 --2- 172.21.15.26:0/1252575035 >> v2:172.21.15.160:3300/0 conn(0x7ffba40b1d40 0x7ffba40b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffba8cd5640 1 -- 172.21.15.26:0/1252575035 >> v1:172.21.15.160:6789/0 conn(0x7ffba406bee0 legacy=0x7ffba406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffba8cd5640 1 -- 172.21.15.26:0/1252575035 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba406cc10 con 0x7ffba40cb1d0 2023-12-19T16:42:33.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffba8cd5640 1 --2- 172.21.15.26:0/1252575035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba40cb1d0 0x7ffba40cb5b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffba40cca50 tx=0x7ffb9801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dced7e7b646e3c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.204+0000 7ffba37fe640 1 -- 172.21.15.26:0/1252575035 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb980ab670 con 0x7ffba40cb1d0 2023-12-19T16:42:33.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffba37fe640 1 -- 172.21.15.26:0/1252575035 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb980ab7d0 con 0x7ffba40cb1d0 2023-12-19T16:42:33.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffba37fe640 1 -- 172.21.15.26:0/1252575035 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb980abb60 con 0x7ffba40cb1d0 2023-12-19T16:42:33.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 -- 172.21.15.26:0/1252575035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba40cb1d0 msgr2=0x7ffba40cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 --2- 172.21.15.26:0/1252575035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba40cb1d0 0x7ffba40cb5b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffba40cca50 tx=0x7ffb9801c470 comp rx=0 tx=0).stop 2023-12-19T16:42:33.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 -- 172.21.15.26:0/1252575035 shutdown_connections 2023-12-19T16:42:33.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 --2- 172.21.15.26:0/1252575035 >> v2:172.21.15.160:3300/0 conn(0x7ffba40b1d40 0x7ffba40b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 --2- 172.21.15.26:0/1252575035 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba40cb1d0 0x7ffba40cb5b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 -- 172.21.15.26:0/1252575035 >> 172.21.15.26:0/1252575035 conn(0x7ffba40658e0 msgr2=0x7ffba4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 -- 172.21.15.26:0/1252575035 shutdown_connections 2023-12-19T16:42:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 -- 172.21.15.26:0/1252575035 wait complete. 2023-12-19T16:42:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 Processor -- start 2023-12-19T16:42:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 -- start start 2023-12-19T16:42:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba406bee0 0x7ffba4140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba40b1d40 0x7ffba4158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffba40cb1d0 0x7ffba4158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba415ffb0 con 0x7ffba406bee0 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba4160290 con 0x7ffba40b1d40 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffbaaf39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba4160570 con 0x7ffba40cb1d0 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffba8cd5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba406bee0 0x7ffba4140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffba8cd5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba406bee0 0x7ffba4140930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49226/0 (socket says 172.21.15.26:49226) 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffba8cd5640 1 -- 172.21.15.26:0/2826422450 learned_addr learned my addr 172.21.15.26:0/2826422450 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.208+0000 7ffba3fff640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba40b1d40 0x7ffba4158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba94d6640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffba40cb1d0 0x7ffba4158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba8cd5640 1 -- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffba40cb1d0 msgr2=0x7ffba4158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba8cd5640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffba40cb1d0 0x7ffba4158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba8cd5640 1 -- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba40b1d40 msgr2=0x7ffba4158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba8cd5640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba40b1d40 0x7ffba4158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba8cd5640 1 -- 172.21.15.26:0/2826422450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb980094d0 con 0x7ffba406bee0 2023-12-19T16:42:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba94d6640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffba40cb1d0 0x7ffba4158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba8cd5640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba406bee0 0x7ffba4140930 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffb980ad040 tx=0x7ffb9801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba17fa640 1 -- 172.21.15.26:0/2826422450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb980b4800 con 0x7ffba406bee0 2023-12-19T16:42:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba17fa640 1 -- 172.21.15.26:0/2826422450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb980b4980 con 0x7ffba406bee0 2023-12-19T16:42:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba17fa640 1 -- 172.21.15.26:0/2826422450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb980b4d30 con 0x7ffba406bee0 2023-12-19T16:42:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffbaaf39640 1 -- 172.21.15.26:0/2826422450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba4160850 con 0x7ffba406bee0 2023-12-19T16:42:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffbaaf39640 1 -- 172.21.15.26:0/2826422450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba4140e70 con 0x7ffba406bee0 2023-12-19T16:42:33.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffbaaf39640 1 -- 172.21.15.26:0/2826422450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba4074270 con 0x7ffba406bee0 2023-12-19T16:42:33.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba17fa640 1 -- 172.21.15.26:0/2826422450 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ffb980b3380 con 0x7ffba406bee0 2023-12-19T16:42:33.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba17fa640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb88065a60 0x7ffb88067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.212+0000 7ffba17fa640 1 -- 172.21.15.26:0/2826422450 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb9811d940 con 0x7ffba406bee0 2023-12-19T16:42:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.216+0000 7ffba3fff640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb88065a60 0x7ffb88067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.216+0000 7ffba3fff640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb88065a60 0x7ffb88067f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffba4158220 tx=0x7ffb90009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.216+0000 7ffba17fa640 1 -- 172.21.15.26:0/2826422450 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffb980eb430 con 0x7ffba406bee0 2023-12-19T16:42:33.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.376+0000 7ffbaaf39640 1 -- 172.21.15.26:0/2826422450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffba40cf710 con 0x7ffba406bee0 2023-12-19T16:42:33.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.376+0000 7ffba17fa640 1 -- 172.21.15.26:0/2826422450 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7ffb980eea50 con 0x7ffba406bee0 2023-12-19T16:42:33.381 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:33.382 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:33.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.380+0000 7ffbaaf39640 1 -- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb88065a60 msgr2=0x7ffb88067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.380+0000 7ffbaaf39640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb88065a60 0x7ffb88067f20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffba4158220 tx=0x7ffb90009040 comp rx=0 tx=0).stop 2023-12-19T16:42:33.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.380+0000 7ffbaaf39640 1 -- 172.21.15.26:0/2826422450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba406bee0 msgr2=0x7ffba4140930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.380+0000 7ffbaaf39640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba406bee0 0x7ffba4140930 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffb980ad040 tx=0x7ffb9801acb0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.380+0000 7ffbaaf39640 1 -- 172.21.15.26:0/2826422450 shutdown_connections 2023-12-19T16:42:33.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.380+0000 7ffbaaf39640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb88065a60 0x7ffb88067f20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.380+0000 7ffbaaf39640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffba40cb1d0 0x7ffba4158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.380+0000 7ffbaaf39640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffba40b1d40 0x7ffba4158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.380+0000 7ffbaaf39640 1 --2- 172.21.15.26:0/2826422450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffba406bee0 0x7ffba4140930 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.380+0000 7ffbaaf39640 1 -- 172.21.15.26:0/2826422450 >> 172.21.15.26:0/2826422450 conn(0x7ffba40658e0 msgr2=0x7ffba4163f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.380+0000 7ffbaaf39640 1 -- 172.21.15.26:0/2826422450 shutdown_connections 2023-12-19T16:42:33.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.380+0000 7ffbaaf39640 1 -- 172.21.15.26:0/2826422450 wait complete. 2023-12-19T16:42:33.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-12-19T16:42:33.404 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-12-19T16:42:33.404 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-12-19T16:42:33.404 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-12-19T16:42:33.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-12-19T16:42:33.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-12-19T16:42:33.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-12-19T16:42:33.406 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-12-19T16:42:33.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.644+0000 7fa104a41640 1 Processor -- start 2023-12-19T16:42:33.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.644+0000 7fa104a41640 1 -- start start 2023-12-19T16:42:33.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.648+0000 7fa104a41640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa1001599c0 0x7fa100159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.648+0000 7fa104a41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa1000cb900 0x7fa100152a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.648+0000 7fa104a41640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1000b3200 con 0x7fa1000b2790 2023-12-19T16:42:33.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.648+0000 7fa104a41640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10006baa0 con 0x7fa1000cb900 2023-12-19T16:42:33.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.648+0000 7fa104a41640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1000b3aa0 con 0x7fa1001599c0 2023-12-19T16:42:33.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.648+0000 7fa0fe59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa1001599c0 0x7fa100159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.648+0000 7fa0fe59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa1001599c0 0x7fa100159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:33858/0 (socket says 172.21.15.26:33858) 2023-12-19T16:42:33.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.648+0000 7fa0fe59c640 1 -- 172.21.15.26:0/1369939272 learned_addr learned my addr 172.21.15.26:0/1369939272 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.648+0000 7fa0fed9d640 1 --2- 172.21.15.26:0/1369939272 >> v2:172.21.15.26:3300/0 conn(0x7fa1000cb900 0x7fa100152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.648+0000 7fa0fed9d640 1 -- 172.21.15.26:0/1369939272 >> v2:172.21.15.160:3300/0 conn(0x7fa1001599c0 msgr2=0x7fa100159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:33.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.648+0000 7fa0fed9d640 1 --2- 172.21.15.26:0/1369939272 >> v2:172.21.15.160:3300/0 conn(0x7fa1001599c0 0x7fa100159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.648+0000 7fa0fed9d640 1 -- 172.21.15.26:0/1369939272 >> v1:172.21.15.160:6789/0 conn(0x7fa1000b2790 legacy=0x7fa1000cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.648+0000 7fa0fed9d640 1 -- 172.21.15.26:0/1369939272 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10006ced0 con 0x7fa1000cb900 2023-12-19T16:42:33.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.652+0000 7f9a251a2640 1 Processor -- start 2023-12-19T16:42:33.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.652+0000 7f9a251a2640 1 -- start start 2023-12-19T16:42:33.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7fa0fed9d640 1 --2- 172.21.15.26:0/1369939272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1000cb900 0x7fa100152a60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa1000ccb40 tx=0x7fa0f401e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a49f346d0ee9ef server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7fa0fd59a640 1 -- 172.21.15.26:0/1369939272 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40a7980 con 0x7fa1000cb900 2023-12-19T16:42:33.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7fa0fd59a640 1 -- 172.21.15.26:0/1369939272 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f40ab070 con 0x7fa1000cb900 2023-12-19T16:42:33.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7fa0fd59a640 1 -- 172.21.15.26:0/1369939272 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40a7d00 con 0x7fa1000cb900 2023-12-19T16:42:33.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7f9a251a2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a2006d390 0x7f9a200c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7fa104a41640 1 -- 172.21.15.26:0/1369939272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1000cb900 msgr2=0x7fa100152a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7fa104a41640 1 --2- 172.21.15.26:0/1369939272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1000cb900 0x7fa100152a60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa1000ccb40 tx=0x7fa0f401e8d0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7fa104a41640 1 -- 172.21.15.26:0/1369939272 shutdown_connections 2023-12-19T16:42:33.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7f9a1f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a2006d390 0x7f9a200c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7fa104a41640 1 --2- 172.21.15.26:0/1369939272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1000cb900 0x7fa100152a60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7fa104a41640 1 --2- 172.21.15.26:0/1369939272 >> v2:172.21.15.160:3300/0 conn(0x7fa1001599c0 0x7fa100159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7fa104a41640 1 -- 172.21.15.26:0/1369939272 >> 172.21.15.26:0/1369939272 conn(0x7fa1000658e0 msgr2=0x7fa100065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7f9a251a2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a2015d2a0 con 0x7f9a200ca060 2023-12-19T16:42:33.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7f9a251a2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a20160a60 con 0x7f9a200affe0 2023-12-19T16:42:33.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7f9a251a2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a2006d7f0 con 0x7f9a2006d390 2023-12-19T16:42:33.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7fa104a41640 1 -- 172.21.15.26:0/1369939272 shutdown_connections 2023-12-19T16:42:33.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7f9a1f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a2006d390 0x7f9a200c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49242/0 (socket says 172.21.15.26:49242) 2023-12-19T16:42:33.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7f9a1f7fe640 1 -- 172.21.15.26:0/1182361826 learned_addr learned my addr 172.21.15.26:0/1182361826 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7fa104a41640 1 -- 172.21.15.26:0/1369939272 wait complete. 2023-12-19T16:42:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7f9a1f7fe640 1 -- 172.21.15.26:0/1182361826 >> v1:172.21.15.160:6789/0 conn(0x7f9a200affe0 legacy=0x7f9a200b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.656+0000 7f9a1f7fe640 1 -- 172.21.15.26:0/1182361826 >> v1:172.21.15.26:6789/0 conn(0x7f9a200ca060 legacy=0x7f9a20160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7f9a1f7fe640 1 -- 172.21.15.26:0/1182361826 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a200cd680 con 0x7f9a2006d390 2023-12-19T16:42:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7f600b59e640 1 Processor -- start 2023-12-19T16:42:33.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7f600b59e640 1 -- start start 2023-12-19T16:42:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7fa104a41640 1 Processor -- start 2023-12-19T16:42:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7fa104a41640 1 -- start start 2023-12-19T16:42:33.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7f600b59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f600c1591f0 0x7f600c0b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7f600b59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f600c1595d0 0x7f600c0cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7f600b59e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600c0cdd80 con 0x7f600c0cd800 2023-12-19T16:42:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7fa104a41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1000b2790 0x7fa1001fdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7fa104a41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1000cb900 0x7fa100154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7fa104a41640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1001599c0 0x7fa1000cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7fa104a41640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1001fe320 con 0x7fa1000b2790 2023-12-19T16:42:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7fa104a41640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa100149800 con 0x7fa1001599c0 2023-12-19T16:42:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7fa104a41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa100149ae0 con 0x7fa1000cb900 2023-12-19T16:42:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.660+0000 7f600b59e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600c0ce060 con 0x7f600c1591f0 2023-12-19T16:42:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f600ad9d640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f600c0cd800 0x7f600c0affe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46940/0 (socket says 172.21.15.26:46940) 2023-12-19T16:42:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f600ad9d640 1 -- 172.21.15.26:0/3602263318 learned_addr learned my addr 172.21.15.26:0/3602263318 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:33.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f600b59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0c8cc0 con 0x7f600c1595d0 2023-12-19T16:42:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f6001d9b640 1 --2- 172.21.15.26:0/3602263318 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f600c1595d0 0x7f600c0cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f600a59c640 1 --2- 172.21.15.26:0/3602263318 >> v2:172.21.15.26:3300/0 conn(0x7f600c1591f0 0x7f600c0b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0fed9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1001599c0 0x7fa1000cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0fed9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1001599c0 0x7fa1000cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:33862/0 (socket says 172.21.15.26:33862) 2023-12-19T16:42:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0fed9d640 1 -- 172.21.15.26:0/1085421260 learned_addr learned my addr 172.21.15.26:0/1085421260 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f6009d9b640 1 -- 172.21.15.26:0/3602263318 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755925363 0 0) 0x7f600c0cdd80 con 0x7f600c0cd800 2023-12-19T16:42:33.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f6009d9b640 1 -- 172.21.15.26:0/3602263318 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff8003530 con 0x7f600c0cd800 2023-12-19T16:42:33.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fb8b77ce640 1 Processor -- start 2023-12-19T16:42:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0fdd9b640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1000cb900 0x7fa100154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0fe59c640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1000b2790 0x7fa1001fdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f6001d9b640 1 -- 172.21.15.26:0/3602263318 >> v2:172.21.15.26:3300/0 conn(0x7f600c1591f0 msgr2=0x7f600c0b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f6001d9b640 1 --2- 172.21.15.26:0/3602263318 >> v2:172.21.15.26:3300/0 conn(0x7f600c1591f0 0x7f600c0b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f6001d9b640 1 -- 172.21.15.26:0/3602263318 >> v1:172.21.15.160:6789/0 conn(0x7f600c0cd800 legacy=0x7f600c0affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f6009d9b640 1 -- 172.21.15.26:0/3602263318 <== mon.1 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3483837185 0 0) 0x7f5ff8003530 con 0x7f600c0cd800 2023-12-19T16:42:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0fed9d640 1 -- 172.21.15.26:0/1085421260 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1000cb900 msgr2=0x7fa100154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0fed9d640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1000cb900 0x7fa100154fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0fed9d640 1 -- 172.21.15.26:0/1085421260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1000b2790 msgr2=0x7fa1001fdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0fed9d640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1000b2790 0x7fa1001fdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0fed9d640 1 -- 172.21.15.26:0/1085421260 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f40084a0 con 0x7fa1001599c0 2023-12-19T16:42:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f6001d9b640 1 -- 172.21.15.26:0/3602263318 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c0c8fa0 con 0x7f600c1595d0 2023-12-19T16:42:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f600a59c640 1 --2- 172.21.15.26:0/3602263318 >> v2:172.21.15.26:3300/0 conn(0x7f600c1591f0 0x7f600c0b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:33.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0fed9d640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1001599c0 0x7fa1000cd840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40b3040 tx=0x7fa0f401d110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f9a1f7fe640 1 --2- 172.21.15.26:0/1182361826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2006d390 0x7f9a200c9b20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9a140097f0 tx=0x7f9a1401c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d849cc0748c03f04 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f9a1effd640 1 -- 172.21.15.26:0/1182361826 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a140ab600 con 0x7f9a2006d390 2023-12-19T16:42:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fb8b77ce640 1 -- start start 2023-12-19T16:42:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0eeffd640 1 -- 172.21.15.26:0/1085421260 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40a7d50 con 0x7fa1001599c0 2023-12-19T16:42:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0eeffd640 1 -- 172.21.15.26:0/1085421260 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f400b120 con 0x7fa1001599c0 2023-12-19T16:42:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa0eeffd640 1 -- 172.21.15.26:0/1085421260 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f400b4d0 con 0x7fa1001599c0 2023-12-19T16:42:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa104a41640 1 -- 172.21.15.26:0/1085421260 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa100149dc0 con 0x7fa1001599c0 2023-12-19T16:42:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa104a41640 1 -- 172.21.15.26:0/1085421260 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10014a380 con 0x7fa1001599c0 2023-12-19T16:42:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7f9a1effd640 1 -- 172.21.15.26:0/1182361826 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a140a7050 con 0x7f9a2006d390 2023-12-19T16:42:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f9a251a2640 1 -- 172.21.15.26:0/1182361826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2006d390 msgr2=0x7f9a200c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f9a251a2640 1 --2- 172.21.15.26:0/1182361826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2006d390 0x7f9a200c9b20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9a140097f0 tx=0x7f9a1401c2b0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f9a251a2640 1 -- 172.21.15.26:0/1182361826 shutdown_connections 2023-12-19T16:42:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f9a251a2640 1 --2- 172.21.15.26:0/1182361826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a2006d390 0x7f9a200c9b20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f9a251a2640 1 -- 172.21.15.26:0/1182361826 >> 172.21.15.26:0/1182361826 conn(0x7f9a200ce560 msgr2=0x7f9a200ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.664+0000 7fa104a41640 1 -- 172.21.15.26:0/1085421260 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa1001537b0 con 0x7fa1001599c0 2023-12-19T16:42:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f9a251a2640 1 -- 172.21.15.26:0/1182361826 shutdown_connections 2023-12-19T16:42:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f27385a6640 1 Processor -- start 2023-12-19T16:42:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f9a251a2640 1 -- 172.21.15.26:0/1182361826 wait complete. 2023-12-19T16:42:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f27385a6640 1 -- start start 2023-12-19T16:42:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fa0eeffd640 1 -- 172.21.15.26:0/1085421260 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa0f400b630 con 0x7fa1001599c0 2023-12-19T16:42:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fb8b77ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb8b015aa90 0x7fb8b00ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fb8b77ce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb8b015f140 0x7fb8b006f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fb8b77ce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b015f640 con 0x7fb8b00cb580 2023-12-19T16:42:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fb8b77ce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b006c4a0 con 0x7fb8b015aa90 2023-12-19T16:42:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fb8b77ce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b006c780 con 0x7fb8b015f140 2023-12-19T16:42:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fb8b4d69640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb8b015f140 0x7fb8b006f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fb8b4d69640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb8b015f140 0x7fb8b006f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:33876/0 (socket says 172.21.15.26:33876) 2023-12-19T16:42:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fb8b4d69640 1 -- 172.21.15.26:0/3066784020 learned_addr learned my addr 172.21.15.26:0/3066784020 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f6001d9b640 1 --2- 172.21.15.26:0/3602263318 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f600c1595d0 0x7f600c0cd2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009d90 tx=0x7f5ffc01c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=489d3380eecd375f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fb8b4d69640 1 -- 172.21.15.26:0/3066784020 >> v2:172.21.15.26:3300/0 conn(0x7fb8b015aa90 msgr2=0x7fb8b00ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fb8b4d69640 1 --2- 172.21.15.26:0/3066784020 >> v2:172.21.15.26:3300/0 conn(0x7fb8b015aa90 0x7fb8b00ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fb8b4d69640 1 -- 172.21.15.26:0/3066784020 >> v1:172.21.15.160:6789/0 conn(0x7fb8b00cb580 legacy=0x7fb8b006d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fb8b4d69640 1 -- 172.21.15.26:0/3066784020 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b006ca60 con 0x7fb8b015f140 2023-12-19T16:42:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f6009d9b640 1 -- 172.21.15.26:0/3602263318 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004001570 con 0x7f600c1595d0 2023-12-19T16:42:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f6009d9b640 1 -- 172.21.15.26:0/3602263318 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc0a55d0 con 0x7f600c1595d0 2023-12-19T16:42:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f6009d9b640 1 -- 172.21.15.26:0/3602263318 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc0a5960 con 0x7f600c1595d0 2023-12-19T16:42:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7f27385a6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f273006bee0 0x7f273006c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.668+0000 7fa0eeffd640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa0e0065b60 0x7fa0e0068020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7fa0eeffd640 1 -- 172.21.15.26:0/1085421260 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0f411e5b0 con 0x7fa1001599c0 2023-12-19T16:42:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7fa0fe59c640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa0e0065b60 0x7fa0e0068020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7fa0eeffd640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] conn(0x7fa0e006dd80 0x7fa0e00701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7fa0eeffd640 1 -- 172.21.15.26:0/1085421260 --> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa0e0070850 con 0x7fa0e006dd80 2023-12-19T16:42:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7fa0eeffd640 1 -- 172.21.15.26:0/1085421260 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0f411e960 con 0x7fa1001599c0 2023-12-19T16:42:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735b41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f273006bee0 0x7f273006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735b41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f273006bee0 0x7f273006c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49270/0 (socket says 172.21.15.26:49270) 2023-12-19T16:42:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f9a251a2640 1 Processor -- start 2023-12-19T16:42:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f600b59e640 1 -- 172.21.15.26:0/3602263318 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f600c1595d0 msgr2=0x7f600c0cd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f600b59e640 1 --2- 172.21.15.26:0/3602263318 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f600c1595d0 0x7f600c0cd2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009d90 tx=0x7f5ffc01c7b0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f600b59e640 1 -- 172.21.15.26:0/3602263318 shutdown_connections 2023-12-19T16:42:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f600b59e640 1 --2- 172.21.15.26:0/3602263318 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f600c1595d0 0x7f600c0cd2c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f9a251a2640 1 -- start start 2023-12-19T16:42:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f600b59e640 1 --2- 172.21.15.26:0/3602263318 >> v2:172.21.15.26:3300/0 conn(0x7f600c1591f0 0x7f600c0b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f600b59e640 1 -- 172.21.15.26:0/3602263318 >> 172.21.15.26:0/3602263318 conn(0x7f600c06b0f0 msgr2=0x7f600c06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f600b59e640 1 -- 172.21.15.26:0/3602263318 shutdown_connections 2023-12-19T16:42:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f600b59e640 1 -- 172.21.15.26:0/3602263318 wait complete. 2023-12-19T16:42:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2736342640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f27300cb1d0 0x7f27300cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46952/0 (socket says 172.21.15.26:46952) 2023-12-19T16:42:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7fa0fdd9b640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] conn(0x7fa0e006dd80 0x7fa0e00701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f9a251a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2006d390 0x7f9a200b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f9a251a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a200affe0 0x7f9a2014cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735b41640 1 -- 172.21.15.26:0/3139306385 learned_addr learned my addr 172.21.15.26:0/3139306385 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f27385a6640 1 -- 172.21.15.26:0/3139306385 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f273006d4b0 con 0x7f27300b1d40 2023-12-19T16:42:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f27385a6640 1 -- 172.21.15.26:0/3139306385 --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f273006cc10 con 0x7f27300cb1d0 2023-12-19T16:42:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f27385a6640 1 -- 172.21.15.26:0/3139306385 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27300cc030 con 0x7f273006bee0 2023-12-19T16:42:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 -- 172.21.15.26:0/3139306385 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602207415 0 0) 0x7f273006cc10 con 0x7f27300cb1d0 2023-12-19T16:42:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 -- 172.21.15.26:0/3139306385 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f271c003530 con 0x7f27300cb1d0 2023-12-19T16:42:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 -- 172.21.15.26:0/3139306385 <== mon.1 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4001557348 0 0) 0x7f271c003530 con 0x7f27300cb1d0 2023-12-19T16:42:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 -- 172.21.15.26:0/3139306385 >> v2:172.21.15.26:3300/0 conn(0x7f273006bee0 msgr2=0x7f273006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 --2- 172.21.15.26:0/3139306385 >> v2:172.21.15.26:3300/0 conn(0x7f273006bee0 0x7f273006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 -- 172.21.15.26:0/3139306385 >> v1:172.21.15.26:6789/0 conn(0x7f27300b1d40 legacy=0x7f27300b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 -- 172.21.15.26:0/3139306385 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273006cef0 con 0x7f27300cb1d0 2023-12-19T16:42:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 -- 172.21.15.26:0/3139306385 <== mon.1 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f2728001a00 con 0x7f27300cb1d0 2023-12-19T16:42:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 -- 172.21.15.26:0/3139306385 >> v1:172.21.15.160:6789/0 conn(0x7f27300cb1d0 legacy=0x7f27300cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 --2- 172.21.15.26:0/3139306385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f271c0039d0 0x7f271c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f9a251a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a200ca060 0x7f9a200c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f9a251a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200c4fd0 con 0x7f9a200affe0 2023-12-19T16:42:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f9a251a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200b3b70 con 0x7f9a2006d390 2023-12-19T16:42:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f9a251a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200b3e50 con 0x7f9a200ca060 2023-12-19T16:42:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7fa0fe59c640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa0e0065b60 0x7fa0e0068020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000c900 tx=0x7fa0f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a200affe0 0x7f9a2014cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f9a1f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a200affe0 0x7f9a2014cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49274/0 (socket says 172.21.15.26:49274) 2023-12-19T16:42:33.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f9a1f7fe640 1 -- 172.21.15.26:0/3863727465 learned_addr learned my addr 172.21.15.26:0/3863727465 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7fabcdc12640 1 Processor -- start 2023-12-19T16:42:33.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f9a1ffff640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2006d390 0x7f9a200b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 --2- 172.21.15.26:0/3139306385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f271c0047d0 0x7f271c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 --2- 172.21.15.26:0/3139306385 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f271c005260 0x7f271c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 -- 172.21.15.26:0/3139306385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27300cc030 con 0x7f271c0047d0 2023-12-19T16:42:33.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 -- 172.21.15.26:0/3139306385 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c00c1b0 con 0x7f271c0039d0 2023-12-19T16:42:33.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735340640 1 -- 172.21.15.26:0/3139306385 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c00c370 con 0x7f271c005260 2023-12-19T16:42:33.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2736342640 1 --2- 172.21.15.26:0/3139306385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f271c0039d0 0x7f271c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.672+0000 7f2735b41640 1 --2- 172.21.15.26:0/3139306385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f271c0047d0 0x7f271c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fa0fdd9b640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] conn(0x7fa0e006dd80 0x7fa0e00701a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fabcdc12640 1 -- start start 2023-12-19T16:42:33.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f2736342640 1 -- 172.21.15.26:0/3139306385 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f271c005260 msgr2=0x7f271c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:33.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f2736b43640 1 --2- 172.21.15.26:0/3139306385 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f271c005260 0x7f271c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f2736342640 1 --2- 172.21.15.26:0/3139306385 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f271c005260 0x7f271c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f2736342640 1 -- 172.21.15.26:0/3139306385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f271c0047d0 msgr2=0x7f271c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:33.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f2736342640 1 --2- 172.21.15.26:0/3139306385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f271c0047d0 0x7f271c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f2736342640 1 -- 172.21.15.26:0/3139306385 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c003530 con 0x7f271c0039d0 2023-12-19T16:42:33.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f600b59e640 1 Processor -- start 2023-12-19T16:42:33.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f2735b41640 1 --2- 172.21.15.26:0/3139306385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f271c0047d0 0x7f271c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:33.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a249a1640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a200ca060 0x7f9a200c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a1ffff640 1 -- 172.21.15.26:0/3863727465 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a200ca060 msgr2=0x7f9a200c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a1ffff640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a200ca060 0x7f9a200c4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a1ffff640 1 -- 172.21.15.26:0/3863727465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a200affe0 msgr2=0x7f9a2014cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a1ffff640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a200affe0 0x7f9a2014cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a1ffff640 1 -- 172.21.15.26:0/3863727465 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140094d0 con 0x7f9a2006d390 2023-12-19T16:42:33.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f600b59e640 1 -- start start 2023-12-19T16:42:33.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f2736b43640 1 --2- 172.21.15.26:0/3139306385 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f271c005260 0x7f271c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:33.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f2736b43640 1 -- 172.21.15.26:0/3139306385 reap_dead start 2023-12-19T16:42:33.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fb8b4d69640 1 --2- 172.21.15.26:0/3066784020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8b015f140 0x7fb8b006f370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0071d30 tx=0x7fb8a001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba0ae4eb5a4d7a99 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a249a1640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a200ca060 0x7f9a200c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:33.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fa0eeffd640 1 -- 172.21.15.26:0/1085421260 <== osd.0 v2:172.21.15.26:6800/38117 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa0e0070850 con 0x7fa0e006dd80 2023-12-19T16:42:33.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a1ffff640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2006d390 0x7f9a200b34e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000c500 tx=0x7f9a1000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a1cff9640 1 -- 172.21.15.26:0/3863727465 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10003600 con 0x7f9a2006d390 2023-12-19T16:42:33.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a1cff9640 1 -- 172.21.15.26:0/3863727465 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a10003760 con 0x7f9a2006d390 2023-12-19T16:42:33.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a1cff9640 1 -- 172.21.15.26:0/3863727465 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10003b50 con 0x7f9a2006d390 2023-12-19T16:42:33.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fb8a7fff640 1 -- 172.21.15.26:0/3066784020 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a00aa650 con 0x7fb8b015f140 2023-12-19T16:42:33.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fb8a7fff640 1 -- 172.21.15.26:0/3066784020 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a00aa7b0 con 0x7fb8b015f140 2023-12-19T16:42:33.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fb8a7fff640 1 -- 172.21.15.26:0/3066784020 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a00aab40 con 0x7fb8b015f140 2023-12-19T16:42:33.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f99d30b6640 1 Processor -- start 2023-12-19T16:42:33.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a251a2640 1 -- 172.21.15.26:0/3863727465 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a200b13b0 con 0x7f9a2006d390 2023-12-19T16:42:33.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a251a2640 1 -- 172.21.15.26:0/3863727465 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a200b1970 con 0x7f9a2006d390 2023-12-19T16:42:33.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fb8b77ce640 1 -- 172.21.15.26:0/3066784020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8b015f140 msgr2=0x7fb8b006f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fb8b77ce640 1 --2- 172.21.15.26:0/3066784020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8b015f140 0x7fb8b006f370 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0071d30 tx=0x7fb8a001c4d0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f99d30b6640 1 -- start start 2023-12-19T16:42:33.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fabcdc12640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabc81599c0 0x7fabc8159da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fabcdc12640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fabc80b2790 0x7fabc80b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fabcdc12640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fabc80cb1f0 0x7fabc80cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f7a3868c640 1 Processor -- start 2023-12-19T16:42:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fabcdc12640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc806baa0 con 0x7fabc80b2790 2023-12-19T16:42:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fabcdc12640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc80b3aa0 con 0x7fabc80cb1f0 2023-12-19T16:42:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7fabcdc12640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80cbb70 con 0x7fabc81599c0 2023-12-19T16:42:33.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f9a251a2640 1 -- 172.21.15.26:0/3863727465 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a200cdf20 con 0x7f9a2006d390 2023-12-19T16:42:33.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f600b59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f600c0cd800 0x7f600c0b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f600b59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f600c1591f0 0x7f600c0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f600b59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f600c1595d0 0x7f600c157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f600b59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0ae950 con 0x7f600c1595d0 2023-12-19T16:42:33.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f600b59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0b24c0 con 0x7f600c0cd800 2023-12-19T16:42:33.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f600b59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0ae060 con 0x7f600c1591f0 2023-12-19T16:42:33.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f6001d9b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f600c1591f0 0x7f600c0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f6001d9b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f600c1591f0 0x7f600c0b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:48954/0 (socket says 172.21.15.26:48954) 2023-12-19T16:42:33.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f6001d9b640 1 -- 172.21.15.26:0/4212236722 learned_addr learned my addr 172.21.15.26:0/4212236722 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.676+0000 7f600a59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f600c0cd800 0x7f600c0b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fabc7fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fabc80cb1f0 0x7fabc80cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fabc77fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabc81599c0 0x7fabc8159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fabc7fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fabc80cb1f0 0x7fabc80cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:33914/0 (socket says 172.21.15.26:33914) 2023-12-19T16:42:33.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fabc7fff640 1 -- 172.21.15.26:0/3067964321 learned_addr learned my addr 172.21.15.26:0/3067964321 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f600a59c640 1 -- 172.21.15.26:0/4212236722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f600c1591f0 msgr2=0x7f600c0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f600a59c640 1 --2- 172.21.15.26:0/4212236722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f600c1591f0 0x7f600c0b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f600a59c640 1 -- 172.21.15.26:0/4212236722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f600c1595d0 msgr2=0x7f600c157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f600a59c640 1 --2- 172.21.15.26:0/4212236722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f600c1595d0 0x7f600c157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f600a59c640 1 -- 172.21.15.26:0/4212236722 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff4009420 con 0x7f600c0cd800 2023-12-19T16:42:33.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f99d30b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99cc0ce040 0x7f99cc0d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f99d30b6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f99cc0d3130 0x7f99cc0d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f99d30b6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99cc0d29d0 con 0x7f99cc15cd40 2023-12-19T16:42:33.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f99d30b6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99cc0d3570 con 0x7f99cc0d3130 2023-12-19T16:42:33.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f99d30b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc15d680 con 0x7f99cc0ce040 2023-12-19T16:42:33.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fabc7fff640 1 -- 172.21.15.26:0/3067964321 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabc81599c0 msgr2=0x7fabc8159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:33.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fabc7fff640 1 --2- 172.21.15.26:0/3067964321 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabc81599c0 0x7fabc8159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fabc6ffd640 1 --2- 172.21.15.26:0/3067964321 >> v2:172.21.15.26:3300/0 conn(0x7fabc80b2790 0x7fabc80b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fabc7fff640 1 -- 172.21.15.26:0/3067964321 >> v2:172.21.15.26:3300/0 conn(0x7fabc80b2790 msgr2=0x7fabc80b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:33.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f600a59c640 1 --2- 172.21.15.26:0/4212236722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f600c0cd800 0x7f600c0b1f80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400e9e0 tx=0x7f5ff400eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fabc7fff640 1 --2- 172.21.15.26:0/3067964321 >> v2:172.21.15.26:3300/0 conn(0x7fabc80b2790 0x7fabc80b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fabc7fff640 1 -- 172.21.15.26:0/3067964321 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc80b3200 con 0x7fabc80cb1f0 2023-12-19T16:42:33.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fabc6ffd640 1 --2- 172.21.15.26:0/3067964321 >> v2:172.21.15.26:3300/0 conn(0x7fabc80b2790 0x7fabc80b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:33.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f99cbfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99cc0ce040 0x7f99cc0d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f99cbfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99cc0ce040 0x7f99cc0d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:48972/0 (socket says 172.21.15.26:48972) 2023-12-19T16:42:33.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f99cbfff640 1 -- 172.21.15.26:0/1494750122 learned_addr learned my addr 172.21.15.26:0/1494750122 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f60037fe640 1 -- 172.21.15.26:0/4212236722 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff400c970 con 0x7f600c0cd800 2023-12-19T16:42:33.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f60037fe640 1 -- 172.21.15.26:0/4212236722 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff400cad0 con 0x7f600c0cd800 2023-12-19T16:42:33.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f60037fe640 1 -- 172.21.15.26:0/4212236722 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff400ce60 con 0x7f600c0cd800 2023-12-19T16:42:33.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f7a3868c640 1 -- start start 2023-12-19T16:42:33.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f99d1653640 1 --2- 172.21.15.26:0/1494750122 >> v2:172.21.15.160:3300/0 conn(0x7f99cc0d3130 0x7f99cc0d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f7a3868c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a3006bee0 0x7f7a3006c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f7a3868c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7a300b1d40 0x7f7a300b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f99cbfff640 1 -- 172.21.15.26:0/1494750122 >> v2:172.21.15.160:3300/0 conn(0x7f99cc0d3130 msgr2=0x7f99cc0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:33.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f99cbfff640 1 --2- 172.21.15.26:0/1494750122 >> v2:172.21.15.160:3300/0 conn(0x7f99cc0d3130 0x7f99cc0d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f99cbfff640 1 -- 172.21.15.26:0/1494750122 >> v1:172.21.15.160:6789/0 conn(0x7f99cc15cd40 legacy=0x7f99cc160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f600b59e640 1 -- 172.21.15.26:0/4212236722 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ffc009a70 con 0x7f600c0cd800 2023-12-19T16:42:33.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f600b59e640 1 -- 172.21.15.26:0/4212236722 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c0ae5e0 con 0x7f600c0cd800 2023-12-19T16:42:33.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fb8b77ce640 1 -- 172.21.15.26:0/3066784020 shutdown_connections 2023-12-19T16:42:33.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fb8b77ce640 1 --2- 172.21.15.26:0/3066784020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8b015f140 0x7fb8b006f370 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fb8b77ce640 1 --2- 172.21.15.26:0/3066784020 >> v2:172.21.15.26:3300/0 conn(0x7fb8b015aa90 0x7fb8b00ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fb8b77ce640 1 -- 172.21.15.26:0/3066784020 >> 172.21.15.26:0/3066784020 conn(0x7fb8b00b03f0 msgr2=0x7fb8b00b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f2736342640 1 --2- 172.21.15.26:0/3139306385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f271c0039d0 0x7f271c003dd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f272800c4c0 tx=0x7f272801ce10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8931dcddac7188cc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f7a3868c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3006d4b0 con 0x7f7a300cb1d0 2023-12-19T16:42:33.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f7a3868c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a300cc030 con 0x7f7a300b1d40 2023-12-19T16:42:33.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f7a3868c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3006cc10 con 0x7f7a3006bee0 2023-12-19T16:42:33.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f7a36428640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7a300cb1d0 0x7f7a300cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50274/0 (socket says 172.21.15.26:50274) 2023-12-19T16:42:33.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f7a36428640 1 -- 172.21.15.26:0/2442097234 learned_addr learned my addr 172.21.15.26:0/2442097234 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:33.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f99cbfff640 1 -- 172.21.15.26:0/1494750122 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc161750 con 0x7f99cc0ce040 2023-12-19T16:42:33.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f9a1cff9640 1 -- 172.21.15.26:0/3863727465 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9a10003cb0 con 0x7f9a2006d390 2023-12-19T16:42:33.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fb8b77ce640 1 -- 172.21.15.26:0/3066784020 shutdown_connections 2023-12-19T16:42:33.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f2735340640 1 -- 172.21.15.26:0/3139306385 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728005da0 con 0x7f271c0039d0 2023-12-19T16:42:33.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f2735340640 1 -- 172.21.15.26:0/3139306385 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27280a5af0 con 0x7f271c0039d0 2023-12-19T16:42:33.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7f7a36c29640 1 --2- 172.21.15.26:0/2442097234 >> v2:172.21.15.26:3300/0 conn(0x7f7a300b1d40 0x7f7a300b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f27385a6640 1 -- 172.21.15.26:0/3139306385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f271c0039d0 msgr2=0x7f271c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f27385a6640 1 --2- 172.21.15.26:0/3139306385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f271c0039d0 0x7f271c003dd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f272800c4c0 tx=0x7f272801ce10 comp rx=0 tx=0).stop 2023-12-19T16:42:33.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f27385a6640 1 -- 172.21.15.26:0/3139306385 shutdown_connections 2023-12-19T16:42:33.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f27385a6640 1 --2- 172.21.15.26:0/3139306385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f271c0039d0 0x7f271c003dd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f27385a6640 1 -- 172.21.15.26:0/3139306385 >> 172.21.15.26:0/3139306385 conn(0x7f27300658e0 msgr2=0x7f2730065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f9a1cff9640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9a0c065a60 0x7f9a0c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f9a1cff9640 1 -- 172.21.15.26:0/3863727465 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a100827f0 con 0x7f9a2006d390 2023-12-19T16:42:33.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.680+0000 7fb8b77ce640 1 -- 172.21.15.26:0/3066784020 wait complete. 2023-12-19T16:42:33.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f27385a6640 1 -- 172.21.15.26:0/3139306385 shutdown_connections 2023-12-19T16:42:33.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f7a36c29640 1 -- 172.21.15.26:0/2442097234 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a3006bee0 msgr2=0x7f7a3006c340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-19T16:42:33.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f7a36c29640 1 --2- 172.21.15.26:0/2442097234 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a3006bee0 0x7f7a3006c340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f7a36c29640 1 -- 172.21.15.26:0/2442097234 >> v1:172.21.15.26:6789/0 conn(0x7f7a300cb1d0 legacy=0x7f7a300cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f7a36c29640 1 -- 172.21.15.26:0/2442097234 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3006cef0 con 0x7f7a300b1d40 2023-12-19T16:42:33.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f600b59e640 1 -- 172.21.15.26:0/4212236722 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f600c0c97a0 con 0x7f600c0cd800 2023-12-19T16:42:33.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f9a1cff9640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] conn(0x7f9a0c06dc00 0x7f9a0c070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f9a1cff9640 1 -- 172.21.15.26:0/3863727465 --> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a0c0706d0 con 0x7f9a0c06dc00 2023-12-19T16:42:33.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f9a1cff9640 1 -- 172.21.15.26:0/3863727465 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a10050240 con 0x7f9a2006d390 2023-12-19T16:42:33.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7fb8b77ce640 1 Processor -- start 2023-12-19T16:42:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f9a1f7fe640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9a0c065a60 0x7f9a0c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f9a249a1640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] conn(0x7f9a0c06dc00 0x7f9a0c070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f60037fe640 1 -- 172.21.15.26:0/4212236722 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5ff4013470 con 0x7f600c0cd800 2023-12-19T16:42:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7fabc7fff640 1 --2- 172.21.15.26:0/3067964321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabc80cb1f0 0x7fabc80cb630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fabc80ccae0 tx=0x7fabbc01c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=224fc6843da594a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7fb8b77ce640 1 -- start start 2023-12-19T16:42:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f9a249a1640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] conn(0x7f9a0c06dc00 0x7f9a0c070020 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f9a1f7fe640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9a0c065a60 0x7f9a0c067f20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a14000c10 tx=0x7f9a140a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7fabc67fc640 1 -- 172.21.15.26:0/3067964321 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0ac640 con 0x7fabc80cb1f0 2023-12-19T16:42:33.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7fabc67fc640 1 -- 172.21.15.26:0/3067964321 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc0ac7a0 con 0x7fabc80cb1f0 2023-12-19T16:42:33.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7fabc67fc640 1 -- 172.21.15.26:0/3067964321 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0acb30 con 0x7fabc80cb1f0 2023-12-19T16:42:33.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f60037fe640 1 --2- 172.21.15.26:0/4212236722 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5fec065b30 0x7f5fec067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f60037fe640 1 -- 172.21.15.26:0/4212236722 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ff40832a0 con 0x7f600c0cd800 2023-12-19T16:42:33.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f60037fe640 1 --2- 172.21.15.26:0/4212236722 >> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] conn(0x7f5fec06dd50 0x7f5fec070170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.684+0000 7f27385a6640 1 -- 172.21.15.26:0/3139306385 wait complete. 2023-12-19T16:42:33.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f27385a6640 1 Processor -- start 2023-12-19T16:42:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f9a1cff9640 1 -- 172.21.15.26:0/3863727465 <== osd.1 v2:172.21.15.26:6808/38116 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9a0c0706d0 con 0x7f9a0c06dc00 2023-12-19T16:42:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f60037fe640 1 -- 172.21.15.26:0/4212236722 --> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5fec070820 con 0x7f5fec06dd50 2023-12-19T16:42:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f60037fe640 1 -- 172.21.15.26:0/4212236722 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ff4085800 con 0x7f600c0cd800 2023-12-19T16:42:33.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fabcdc12640 1 -- 172.21.15.26:0/3067964321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabc80cb1f0 msgr2=0x7fabc80cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fabcdc12640 1 --2- 172.21.15.26:0/3067964321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabc80cb1f0 0x7fabc80cb630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fabc80ccae0 tx=0x7fabbc01c440 comp rx=0 tx=0).stop 2023-12-19T16:42:33.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f27385a6640 1 -- start start 2023-12-19T16:42:33.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f600ad9d640 1 --2- 172.21.15.26:0/4212236722 >> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] conn(0x7f5fec06dd50 0x7f5fec070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f6001d9b640 1 --2- 172.21.15.26:0/4212236722 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5fec065b30 0x7f5fec067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f99cbfff640 1 --2- 172.21.15.26:0/1494750122 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99cc0ce040 0x7f99cc0d2bf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009870 tx=0x7f99b801c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb840b7c660f68bc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fabcdc12640 1 -- 172.21.15.26:0/3067964321 shutdown_connections 2023-12-19T16:42:33.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f99cb7fe640 1 -- 172.21.15.26:0/1494750122 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80aa680 con 0x7f99cc0ce040 2023-12-19T16:42:33.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b77ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b00cb580 0x7fb8b02020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f99cb7fe640 1 -- 172.21.15.26:0/1494750122 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b80aa7e0 con 0x7f99cc0ce040 2023-12-19T16:42:33.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f99cb7fe640 1 -- 172.21.15.26:0/1494750122 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80aab70 con 0x7f99cc0ce040 2023-12-19T16:42:33.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fabcdc12640 1 --2- 172.21.15.26:0/3067964321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabc80cb1f0 0x7fabc80cb630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fabcdc12640 1 --2- 172.21.15.26:0/3067964321 >> v2:172.21.15.26:3300/0 conn(0x7fabc80b2790 0x7fabc80b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fabcdc12640 1 --2- 172.21.15.26:0/3067964321 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabc81599c0 0x7fabc8159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fabcdc12640 1 -- 172.21.15.26:0/3067964321 >> 172.21.15.26:0/3067964321 conn(0x7fabc80658e0 msgr2=0x7fabc8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f600ad9d640 1 --2- 172.21.15.26:0/4212236722 >> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] conn(0x7f5fec06dd50 0x7f5fec070170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b77ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8b015aa90 0x7fb8b00b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f6001d9b640 1 --2- 172.21.15.26:0/4212236722 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5fec065b30 0x7f5fec067ff0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009d90 tx=0x7f5ffc0a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b556a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b00cb580 0x7fb8b02020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b77ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8b015f140 0x7fb8b0153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b556a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b00cb580 0x7fb8b02020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49308/0 (socket says 172.21.15.26:49308) 2023-12-19T16:42:33.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b556a640 1 -- 172.21.15.26:0/2674566269 learned_addr learned my addr 172.21.15.26:0/2674566269 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b00b22d0 con 0x7fb8b00cb580 2023-12-19T16:42:33.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b00b25b0 con 0x7fb8b015f140 2023-12-19T16:42:33.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b00b2890 con 0x7fb8b015aa90 2023-12-19T16:42:33.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b5d6b640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8b015f140 0x7fb8b0153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f99d30b6640 1 -- 172.21.15.26:0/1494750122 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99cc0ce040 msgr2=0x7f99cc0d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f99d30b6640 1 --2- 172.21.15.26:0/1494750122 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99cc0ce040 0x7f99cc0d2bf0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009870 tx=0x7f99b801c520 comp rx=0 tx=0).stop 2023-12-19T16:42:33.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b4d69640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8b015aa90 0x7fb8b00b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f27385a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f271c0039d0 0x7f2730159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f60037fe640 1 -- 172.21.15.26:0/4212236722 <== osd.2 v2:172.21.15.26:6809/38119 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5fec070820 con 0x7f5fec06dd50 2023-12-19T16:42:33.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fabcdc12640 1 -- 172.21.15.26:0/3067964321 shutdown_connections 2023-12-19T16:42:33.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fabcdc12640 1 -- 172.21.15.26:0/3067964321 wait complete. 2023-12-19T16:42:33.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f27385a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273015cf40 0x7f2730151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f2736342640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f271c0039d0 0x7f2730159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f2736342640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f271c0039d0 0x7f2730159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49002/0 (socket says 172.21.15.26:49002) 2023-12-19T16:42:33.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f2736342640 1 -- 172.21.15.26:0/2138119303 learned_addr learned my addr 172.21.15.26:0/2138119303 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f27385a6640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27300ce060 0x7f27300ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27300cea30 con 0x7f273015cf40 2023-12-19T16:42:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f2735b41640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273015cf40 0x7f2730151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f2736b43640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27300ce060 0x7f27300ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2730159e90 con 0x7f27300ce060 2023-12-19T16:42:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273015d3d0 con 0x7f271c0039d0 2023-12-19T16:42:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f2736342640 1 -- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27300ce060 msgr2=0x7f27300ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f2736342640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27300ce060 0x7f27300ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f2736342640 1 -- 172.21.15.26:0/2138119303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273015cf40 msgr2=0x7f2730151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f2736342640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f273015cf40 0x7f2730151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f2736342640 1 -- 172.21.15.26:0/2138119303 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273015d6b0 con 0x7f271c0039d0 2023-12-19T16:42:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f7a36c29640 1 --2- 172.21.15.26:0/2442097234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a300b1d40 0x7f7a300b21d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7a280097c0 tx=0x7f7a2801c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96dce6a525e3c2a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f7a35426640 1 -- 172.21.15.26:0/2442097234 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280ab600 con 0x7f7a300b1d40 2023-12-19T16:42:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f7a35426640 1 -- 172.21.15.26:0/2442097234 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a280a7050 con 0x7f7a300b1d40 2023-12-19T16:42:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f7a35426640 1 -- 172.21.15.26:0/2442097234 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a280ab980 con 0x7f7a300b1d40 2023-12-19T16:42:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f99d30b6640 1 -- 172.21.15.26:0/1494750122 shutdown_connections 2023-12-19T16:42:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f99d30b6640 1 --2- 172.21.15.26:0/1494750122 >> v2:172.21.15.160:3300/0 conn(0x7f99cc0d3130 0x7f99cc0d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f99d30b6640 1 --2- 172.21.15.26:0/1494750122 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99cc0ce040 0x7f99cc0d2bf0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f99d30b6640 1 -- 172.21.15.26:0/1494750122 >> 172.21.15.26:0/1494750122 conn(0x7f99cc06a120 msgr2=0x7f99cc06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f99d30b6640 1 -- 172.21.15.26:0/1494750122 shutdown_connections 2023-12-19T16:42:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7f99d30b6640 1 -- 172.21.15.26:0/1494750122 wait complete. 2023-12-19T16:42:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f2736342640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f271c0039d0 0x7f2730159950 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f272801cc40 tx=0x7f271c005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7fabcdc12640 1 Processor -- start 2023-12-19T16:42:33.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7fabcdc12640 1 -- start start 2023-12-19T16:42:33.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b5d6b640 1 -- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8b015aa90 msgr2=0x7fb8b00b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b5d6b640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8b015aa90 0x7fb8b00b1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.688+0000 7fb8b5d6b640 1 -- 172.21.15.26:0/2674566269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b00cb580 msgr2=0x7fb8b02020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7fb8b5d6b640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b00cb580 0x7fb8b02020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7fb8b5d6b640 1 -- 172.21.15.26:0/2674566269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a00094d0 con 0x7fb8b015f140 2023-12-19T16:42:33.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7fb8b556a640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8b00cb580 0x7fb8b02020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:33.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7fb8b5d6b640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8b015f140 0x7fb8b0153530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800c390 tx=0x7fb8a800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7fb8a5ffb640 1 -- 172.21.15.26:0/2674566269 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80052d0 con 0x7fb8b015f140 2023-12-19T16:42:33.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7fb8a5ffb640 1 -- 172.21.15.26:0/2674566269 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a8005430 con 0x7fb8b015f140 2023-12-19T16:42:33.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7fb8a5ffb640 1 -- 172.21.15.26:0/2674566269 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8003840 con 0x7fb8b015f140 2023-12-19T16:42:33.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b00cbf20 con 0x7fb8b015f140 2023-12-19T16:42:33.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b0202610 con 0x7fb8b015f140 2023-12-19T16:42:33.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f2726ffd640 1 -- 172.21.15.26:0/2138119303 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27280b3070 con 0x7f271c0039d0 2023-12-19T16:42:33.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f7a3868c640 1 -- 172.21.15.26:0/2442097234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a300b1d40 msgr2=0x7f7a300b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f7a3868c640 1 --2- 172.21.15.26:0/2442097234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a300b1d40 0x7f7a300b21d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7a280097c0 tx=0x7f7a2801c270 comp rx=0 tx=0).stop 2023-12-19T16:42:33.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f2726ffd640 1 -- 172.21.15.26:0/2138119303 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27280017c0 con 0x7f271c0039d0 2023-12-19T16:42:33.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273015d990 con 0x7f271c0039d0 2023-12-19T16:42:33.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273015df50 con 0x7f271c0039d0 2023-12-19T16:42:33.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f2726ffd640 1 -- 172.21.15.26:0/2138119303 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728007790 con 0x7f271c0039d0 2023-12-19T16:42:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb890000fd0 con 0x7fb8b015f140 2023-12-19T16:42:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabcdc12640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabc80b2790 0x7fabc8157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f99d30b6640 1 Processor -- start 2023-12-19T16:42:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.692+0000 7f99d30b6640 1 -- start start 2023-12-19T16:42:33.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabcdc12640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabc80cb1f0 0x7fabc8153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99d30b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99cc0ce040 0x7f99cc14b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2718000fd0 con 0x7f271c0039d0 2023-12-19T16:42:33.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99d30b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99cc0d3130 0x7f99cc0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99d30b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99cc15cd40 0x7f99cc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99d30b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc148e20 con 0x7f99cc0ce040 2023-12-19T16:42:33.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99d30b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc14f440 con 0x7f99cc15cd40 2023-12-19T16:42:33.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99d30b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0c9d30 con 0x7f99cc0d3130 2023-12-19T16:42:33.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f7a3868c640 1 -- 172.21.15.26:0/2442097234 shutdown_connections 2023-12-19T16:42:33.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f7a3868c640 1 --2- 172.21.15.26:0/2442097234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a300b1d40 0x7f7a300b21d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f7a3868c640 1 --2- 172.21.15.26:0/2442097234 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a3006bee0 0x7f7a3006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f7a3868c640 1 -- 172.21.15.26:0/2442097234 >> 172.21.15.26:0/2442097234 conn(0x7f7a300658e0 msgr2=0x7f7a30065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabcdc12640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc81599c0 0x7fabc8156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabcdc12640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc81688a0 con 0x7fabc81599c0 2023-12-19T16:42:33.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f7a3868c640 1 -- 172.21.15.26:0/2442097234 shutdown_connections 2023-12-19T16:42:33.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabcdc12640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc8167f90 con 0x7fabc80b2790 2023-12-19T16:42:33.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabc77fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabc80b2790 0x7fabc8157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f7a3868c640 1 -- 172.21.15.26:0/2442097234 wait complete. 2023-12-19T16:42:33.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabc77fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabc80b2790 0x7fabc8157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:33950/0 (socket says 172.21.15.26:33950) 2023-12-19T16:42:33.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabc77fe640 1 -- 172.21.15.26:0/2019521113 learned_addr learned my addr 172.21.15.26:0/2019521113 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabc6ffd640 1 --2- 172.21.15.26:0/2019521113 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabc80cb1f0 0x7fabc8153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99cbfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99cc0d3130 0x7f99cc0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99cbfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99cc0d3130 0x7f99cc0b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49004/0 (socket says 172.21.15.26:49004) 2023-12-19T16:42:33.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99cbfff640 1 -- 172.21.15.26:0/2150153596 learned_addr learned my addr 172.21.15.26:0/2150153596 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99cbfff640 1 -- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99cc15cd40 msgr2=0x7f99cc0b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-19T16:42:33.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99cbfff640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99cc15cd40 0x7f99cc0b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99cbfff640 1 -- 172.21.15.26:0/2150153596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99cc0ce040 msgr2=0x7f99cc14b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99d0e52640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99cc0ce040 0x7f99cc14b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99cbfff640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99cc0ce040 0x7f99cc14b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99cbfff640 1 -- 172.21.15.26:0/2150153596 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b80094d0 con 0x7f99cc0d3130 2023-12-19T16:42:33.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabcdc12640 1 -- 172.21.15.26:0/2019521113 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc81583e0 con 0x7fabc80cb1f0 2023-12-19T16:42:33.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabc77fe640 1 -- 172.21.15.26:0/2019521113 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabc80cb1f0 msgr2=0x7fabc8153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabc77fe640 1 --2- 172.21.15.26:0/2019521113 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabc80cb1f0 0x7fabc8153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabc77fe640 1 -- 172.21.15.26:0/2019521113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc81599c0 msgr2=0x7fabc8156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabc77fe640 1 --2- 172.21.15.26:0/2019521113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc81599c0 0x7fabc8156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabc77fe640 1 -- 172.21.15.26:0/2019521113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc0094d0 con 0x7fabc80b2790 2023-12-19T16:42:33.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabc77fe640 1 --2- 172.21.15.26:0/2019521113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabc80b2790 0x7fabc8157a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fabc0007ef0 tx=0x7fabc000e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99d0e52640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99cc0ce040 0x7f99cc14b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:33.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabaffff640 1 -- 172.21.15.26:0/2019521113 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0014620 con 0x7fabc80b2790 2023-12-19T16:42:33.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99cbfff640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99cc0d3130 0x7f99cc0b31d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f99b80ac040 tx=0x7f99b801ad60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabcdc12640 1 -- 172.21.15.26:0/2019521113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc81681e0 con 0x7fabc80b2790 2023-12-19T16:42:33.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabcdc12640 1 -- 172.21.15.26:0/2019521113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc816acd0 con 0x7fabc80b2790 2023-12-19T16:42:33.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f2726ffd640 1 -- 172.21.15.26:0/2138119303 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f27280078f0 con 0x7f271c0039d0 2023-12-19T16:42:33.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f7a3868c640 1 Processor -- start 2023-12-19T16:42:33.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabaffff640 1 -- 172.21.15.26:0/2019521113 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc0014780 con 0x7fabc80b2790 2023-12-19T16:42:33.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7fabaffff640 1 -- 172.21.15.26:0/2019521113 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0014af0 con 0x7fabc80b2790 2023-12-19T16:42:33.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f99c97fa640 1 -- 172.21.15.26:0/2150153596 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80b3800 con 0x7f99cc0d3130 2023-12-19T16:42:33.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.696+0000 7f7a3868c640 1 -- start start 2023-12-19T16:42:33.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f99d30b6640 1 -- 172.21.15.26:0/2150153596 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc0ca010 con 0x7f99cc0d3130 2023-12-19T16:42:33.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7fabcdc12640 1 -- 172.21.15.26:0/2019521113 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faba8000fd0 con 0x7fabc80b2790 2023-12-19T16:42:33.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f99c97fa640 1 -- 172.21.15.26:0/2150153596 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b80b3960 con 0x7f99cc0d3130 2023-12-19T16:42:33.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f99d30b6640 1 -- 172.21.15.26:0/2150153596 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc0ca5d0 con 0x7f99cc0d3130 2023-12-19T16:42:33.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f99c97fa640 1 -- 172.21.15.26:0/2150153596 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80b3800 con 0x7f99cc0d3130 2023-12-19T16:42:33.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7fb8a5ffb640 1 -- 172.21.15.26:0/2674566269 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb8a80039a0 con 0x7fb8b015f140 2023-12-19T16:42:33.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a3868c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3006bee0 0x7f7a3014ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a3868c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a300b1d40 0x7f7a30163940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a3868c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a300cb1d0 0x7f7a30160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a3868c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3014f7c0 con 0x7f7a3006bee0 2023-12-19T16:42:33.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a3868c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3014faa0 con 0x7f7a300cb1d0 2023-12-19T16:42:33.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a3868c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3014fd80 con 0x7f7a300b1d40 2023-12-19T16:42:33.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a36428640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3006bee0 0x7f7a3014ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f99d30b6640 1 -- 172.21.15.26:0/2150153596 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f99ac000fd0 con 0x7f99cc0d3130 2023-12-19T16:42:33.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f2726ffd640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2714065b30 0x7f2714067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a36c29640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a300cb1d0 0x7f7a30160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a36c29640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a300cb1d0 0x7f7a30160170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:33978/0 (socket says 172.21.15.26:33978) 2023-12-19T16:42:33.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a36c29640 1 -- 172.21.15.26:0/3991251293 learned_addr learned my addr 172.21.15.26:0/3991251293 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:33.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a35c27640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a300b1d40 0x7f7a30163940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f2726ffd640 1 -- 172.21.15.26:0/2138119303 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f272811c300 con 0x7f271c0039d0 2023-12-19T16:42:33.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a35c27640 1 -- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a300cb1d0 msgr2=0x7f7a30160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a35c27640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a300cb1d0 0x7f7a30160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a35c27640 1 -- 172.21.15.26:0/3991251293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3006bee0 msgr2=0x7f7a3014ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a35c27640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3006bee0 0x7f7a3014ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a35c27640 1 -- 172.21.15.26:0/3991251293 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a28009420 con 0x7f7a300b1d40 2023-12-19T16:42:33.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a36428640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3006bee0 0x7f7a3014ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:33.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f2735b41640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2714065b30 0x7f2714067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a35c27640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a300b1d40 0x7f7a30163940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00b3c0 tx=0x7f7a1c00b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7fb8a5ffb640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb894065a60 0x7fb894067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f2726ffd640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] conn(0x7f271406dd50 0x7f2714070170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f2726ffd640 1 -- 172.21.15.26:0/2138119303 --> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2714070820 con 0x7f271406dd50 2023-12-19T16:42:33.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a2effd640 1 -- 172.21.15.26:0/3991251293 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c011830 con 0x7f7a300b1d40 2023-12-19T16:42:33.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a2effd640 1 -- 172.21.15.26:0/3991251293 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1c0119b0 con 0x7f7a300b1d40 2023-12-19T16:42:33.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a2effd640 1 -- 172.21.15.26:0/3991251293 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c011d60 con 0x7f7a300b1d40 2023-12-19T16:42:33.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7fb8a5ffb640 1 -- 172.21.15.26:0/2674566269 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8a8082620 con 0x7fb8b015f140 2023-12-19T16:42:33.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f2726ffd640 1 -- 172.21.15.26:0/2138119303 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f272811c6b0 con 0x7f271c0039d0 2023-12-19T16:42:33.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a3868c640 1 -- 172.21.15.26:0/3991251293 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a30150060 con 0x7f7a300b1d40 2023-12-19T16:42:33.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7f7a3868c640 1 -- 172.21.15.26:0/3991251293 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3014efc0 con 0x7f7a300b1d40 2023-12-19T16:42:33.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.700+0000 7fb8b556a640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb894065a60 0x7fb894067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fb8a5ffb640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] conn(0x7fb89406dc00 0x7fb894070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7f2736b43640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] conn(0x7f271406dd50 0x7f2714070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fb8a5ffb640 1 -- 172.21.15.26:0/2674566269 --> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8940706d0 con 0x7fb89406dc00 2023-12-19T16:42:33.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fb8b556a640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb894065a60 0x7fb894067f20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00c530 tx=0x7fb8ac00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7f7a3868c640 1 -- 172.21.15.26:0/3991251293 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7a18000fd0 con 0x7f7a300b1d40 2023-12-19T16:42:33.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7f2735b41640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2714065b30 0x7f2714067ff0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2730205a50 tx=0x7f2720008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fb8a5ffb640 1 -- 172.21.15.26:0/2674566269 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8a8084ce0 con 0x7fb8b015f140 2023-12-19T16:42:33.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fabaffff640 1 -- 172.21.15.26:0/2019521113 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fabc0014c50 con 0x7fabc80b2790 2023-12-19T16:42:33.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7f2736b43640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] conn(0x7f271406dd50 0x7f2714070170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fb8b4d69640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] conn(0x7fb89406dc00 0x7fb894070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7f99c97fa640 1 -- 172.21.15.26:0/2150153596 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f99b80b3ad0 con 0x7f99cc0d3130 2023-12-19T16:42:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fabaffff640 1 --2- 172.21.15.26:0/2019521113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fabb4065ac0 0x7fabb4067f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fabaffff640 1 -- 172.21.15.26:0/2019521113 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabc00848a0 con 0x7fabc80b2790 2023-12-19T16:42:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fabaffff640 1 --2- 172.21.15.26:0/2019521113 >> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] conn(0x7fabb406dc60 0x7fabb4070080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fabaffff640 1 -- 172.21.15.26:0/2019521113 --> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fabb4070730 con 0x7fabb406dc60 2023-12-19T16:42:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fabaffff640 1 -- 172.21.15.26:0/2019521113 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabc0014af0 con 0x7fabc80b2790 2023-12-19T16:42:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fa104a41640 1 -- 172.21.15.26:0/1085421260 --> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa10015a2e0 con 0x7fa0e006dd80 2023-12-19T16:42:33.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7f2726ffd640 1 -- 172.21.15.26:0/2138119303 <== osd.5 v2:172.21.15.160:6802/38419 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2714070820 con 0x7f271406dd50 2023-12-19T16:42:33.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fabc6ffd640 1 --2- 172.21.15.26:0/2019521113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fabb4065ac0 0x7fabb4067f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fb8b4d69640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] conn(0x7fb89406dc00 0x7fb894070020 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.704+0000 7fabc7fff640 1 --2- 172.21.15.26:0/2019521113 >> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] conn(0x7fabb406dc60 0x7fabb4070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f99c97fa640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f99b0065ac0 0x7f99b0067f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f99c97fa640 1 -- 172.21.15.26:0/2150153596 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99b811dae0 con 0x7f99cc0d3130 2023-12-19T16:42:33.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7fa0eeffd640 1 -- 172.21.15.26:0/1085421260 <== osd.0 v2:172.21.15.26:6800/38117 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa0e0070850 con 0x7fa0e006dd80 2023-12-19T16:42:33.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7fabc7fff640 1 --2- 172.21.15.26:0/2019521113 >> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] conn(0x7fabb406dc60 0x7fabb4070080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f7a2effd640 1 -- 172.21.15.26:0/3991251293 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7a1c01b460 con 0x7f7a300b1d40 2023-12-19T16:42:33.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f99c97fa640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] conn(0x7f99b006dc60 0x7f99b0070080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f99c97fa640 1 -- 172.21.15.26:0/2150153596 --> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99b0070730 con 0x7f99b006dc60 2023-12-19T16:42:33.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f99c97fa640 1 -- 172.21.15.26:0/2150153596 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99b81222c0 con 0x7f99cc0d3130 2023-12-19T16:42:33.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f99d0e52640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f99b0065ac0 0x7f99b0067f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7fb8a5ffb640 1 -- 172.21.15.26:0/2674566269 <== osd.4 v2:172.21.15.160:6800/38418 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb8940706d0 con 0x7fb89406dc00 2023-12-19T16:42:33.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7fabc6ffd640 1 --2- 172.21.15.26:0/2019521113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fabb4065ac0 0x7fabb4067f80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fabc8164c50 tx=0x7fabb8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f99d1653640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] conn(0x7f99b006dc60 0x7f99b0070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7fa104a41640 1 -- 172.21.15.26:0/1085421260 >> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] conn(0x7fa0e006dd80 msgr2=0x7fa0e00701a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7fa104a41640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] conn(0x7fa0e006dd80 0x7fa0e00701a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f7a2effd640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a14065ab0 0x7f7a14067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f99d0e52640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f99b0065ac0 0x7f99b0067f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99cc070310 tx=0x7f99c0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7fabaffff640 1 -- 172.21.15.26:0/2019521113 <== osd.3 v2:172.21.15.26:6824/38120 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fabb4070730 con 0x7fabb406dc60 2023-12-19T16:42:33.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f7a2effd640 1 -- 172.21.15.26:0/3991251293 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a1c082dd0 con 0x7f7a300b1d40 2023-12-19T16:42:33.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f99d1653640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] conn(0x7f99b006dc60 0x7f99b0070080 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7fa104a41640 1 -- 172.21.15.26:0/1085421260 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa0e0065b60 msgr2=0x7fa0e0068020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7fa104a41640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa0e0065b60 0x7fa0e0068020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000c900 tx=0x7fa0f0008040 comp rx=0 tx=0).stop 2023-12-19T16:42:33.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7fa104a41640 1 -- 172.21.15.26:0/1085421260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1001599c0 msgr2=0x7fa1000cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7fa104a41640 1 --2- 172.21.15.26:0/1085421260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1001599c0 0x7fa1000cd840 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40b3040 tx=0x7fa0f401d110 comp rx=0 tx=0).stop 2023-12-19T16:42:33.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f7a2effd640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] conn(0x7f7a1406dcd0 0x7f7a140700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:33.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f7a2effd640 1 -- 172.21.15.26:0/3991251293 --> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7a140707a0 con 0x7f7a1406dcd0 2023-12-19T16:42:33.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f7a2effd640 1 -- 172.21.15.26:0/3991251293 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a1c086020 con 0x7f7a300b1d40 2023-12-19T16:42:33.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f7a36c29640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] conn(0x7f7a1406dcd0 0x7f7a140700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.708+0000 7f99c97fa640 1 -- 172.21.15.26:0/2150153596 <== osd.7 v2:172.21.15.160:6803/38420 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f99b0070730 con 0x7f99b006dc60 2023-12-19T16:42:33.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.712+0000 7f7a36428640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a14065ab0 0x7f7a14067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:33.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.712+0000 7fa0fed9d640 1 -- 172.21.15.26:0/1085421260 reap_dead start 2023-12-19T16:42:33.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.712+0000 7fa104a41640 1 -- 172.21.15.26:0/1085421260 shutdown_connections 2023-12-19T16:42:33.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.712+0000 7fa104a41640 1 -- 172.21.15.26:0/1085421260 >> 172.21.15.26:0/1085421260 conn(0x7fa1000658e0 msgr2=0x7fa100065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.712+0000 7f7a36428640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a14065ab0 0x7f7a14067f70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7a240012f0 tx=0x7f7a24005bc0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.712+0000 7fa104a41640 1 -- 172.21.15.26:0/1085421260 shutdown_connections 2023-12-19T16:42:33.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.712+0000 7fa104a41640 1 -- 172.21.15.26:0/1085421260 wait complete. 2023-12-19T16:42:33.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.712+0000 7f7a36c29640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] conn(0x7f7a1406dcd0 0x7f7a140700f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:33.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.712+0000 7f7a2effd640 1 -- 172.21.15.26:0/3991251293 <== osd.6 v2:172.21.15.160:6824/38421 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7a140707a0 con 0x7f7a1406dcd0 2023-12-19T16:42:33.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.720+0000 7f9a251a2640 1 -- 172.21.15.26:0/3863727465 --> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9a20067400 con 0x7f9a0c06dc00 2023-12-19T16:42:33.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f9a1cff9640 1 -- 172.21.15.26:0/3863727465 <== osd.1 v2:172.21.15.26:6808/38116 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9a20067400 con 0x7f9a0c06dc00 2023-12-19T16:42:33.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f9a251a2640 1 -- 172.21.15.26:0/3863727465 >> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] conn(0x7f9a0c06dc00 msgr2=0x7f9a0c070020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f9a251a2640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] conn(0x7f9a0c06dc00 0x7f9a0c070020 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f9a251a2640 1 -- 172.21.15.26:0/3863727465 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9a0c065a60 msgr2=0x7f9a0c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f9a251a2640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9a0c065a60 0x7f9a0c067f20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9a14000c10 tx=0x7f9a140a6000 comp rx=0 tx=0).stop 2023-12-19T16:42:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f9a251a2640 1 -- 172.21.15.26:0/3863727465 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2006d390 msgr2=0x7f9a200b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f9a251a2640 1 --2- 172.21.15.26:0/3863727465 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a2006d390 0x7f9a200b34e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000c500 tx=0x7f9a1000c9c0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f9a249a1640 1 -- 172.21.15.26:0/3863727465 reap_dead start 2023-12-19T16:42:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f9a251a2640 1 -- 172.21.15.26:0/3863727465 shutdown_connections 2023-12-19T16:42:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f9a251a2640 1 -- 172.21.15.26:0/3863727465 >> 172.21.15.26:0/3863727465 conn(0x7f9a200ce560 msgr2=0x7f9a200ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f600b59e640 1 -- 172.21.15.26:0/4212236722 --> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f600c0b3910 con 0x7f5fec06dd50 2023-12-19T16:42:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f9a251a2640 1 -- 172.21.15.26:0/3863727465 shutdown_connections 2023-12-19T16:42:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f9a251a2640 1 -- 172.21.15.26:0/3863727465 wait complete. 2023-12-19T16:42:33.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.724+0000 7f60037fe640 1 -- 172.21.15.26:0/4212236722 <== osd.2 v2:172.21.15.26:6809/38119 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f600c0b3910 con 0x7f5fec06dd50 2023-12-19T16:42:33.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.728+0000 7f600b59e640 1 -- 172.21.15.26:0/4212236722 >> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] conn(0x7f5fec06dd50 msgr2=0x7f5fec070170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.728+0000 7f600b59e640 1 --2- 172.21.15.26:0/4212236722 >> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] conn(0x7f5fec06dd50 0x7f5fec070170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.728+0000 7f600b59e640 1 -- 172.21.15.26:0/4212236722 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5fec065b30 msgr2=0x7f5fec067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.728+0000 7f600b59e640 1 --2- 172.21.15.26:0/4212236722 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5fec065b30 0x7f5fec067ff0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009d90 tx=0x7f5ffc0a7070 comp rx=0 tx=0).stop 2023-12-19T16:42:33.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.728+0000 7f600b59e640 1 -- 172.21.15.26:0/4212236722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f600c0cd800 msgr2=0x7f600c0b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.728+0000 7f600b59e640 1 --2- 172.21.15.26:0/4212236722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f600c0cd800 0x7f600c0b1f80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400e9e0 tx=0x7f5ff400eea0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.728+0000 7f600ad9d640 1 -- 172.21.15.26:0/4212236722 reap_dead start 2023-12-19T16:42:33.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.728+0000 7f600b59e640 1 -- 172.21.15.26:0/4212236722 shutdown_connections 2023-12-19T16:42:33.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.728+0000 7f600b59e640 1 -- 172.21.15.26:0/4212236722 >> 172.21.15.26:0/4212236722 conn(0x7f600c06b0f0 msgr2=0x7f600c156d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.728+0000 7f600b59e640 1 -- 172.21.15.26:0/4212236722 shutdown_connections 2023-12-19T16:42:33.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.728+0000 7f600b59e640 1 -- 172.21.15.26:0/4212236722 wait complete. 2023-12-19T16:42:33.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.732+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 --> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2718002cc0 con 0x7f271406dd50 2023-12-19T16:42:33.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7f2726ffd640 1 -- 172.21.15.26:0/2138119303 <== osd.5 v2:172.21.15.160:6802/38419 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2718002cc0 con 0x7f271406dd50 2023-12-19T16:42:33.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] conn(0x7f271406dd50 msgr2=0x7f2714070170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7f27385a6640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] conn(0x7f271406dd50 0x7f2714070170 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2714065b30 msgr2=0x7f2714067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7f27385a6640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2714065b30 0x7f2714067ff0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2730205a50 tx=0x7f2720008040 comp rx=0 tx=0).stop 2023-12-19T16:42:33.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f271c0039d0 msgr2=0x7f2730159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7f27385a6640 1 --2- 172.21.15.26:0/2138119303 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f271c0039d0 0x7f2730159950 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f272801cc40 tx=0x7f271c005800 comp rx=0 tx=0).stop 2023-12-19T16:42:33.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7f2736b43640 1 -- 172.21.15.26:0/2138119303 reap_dead start 2023-12-19T16:42:33.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 shutdown_connections 2023-12-19T16:42:33.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 >> 172.21.15.26:0/2138119303 conn(0x7f27300658e0 msgr2=0x7f2730160b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 shutdown_connections 2023-12-19T16:42:33.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7fabcdc12640 1 -- 172.21.15.26:0/2019521113 --> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faba8002cc0 con 0x7fabb406dc60 2023-12-19T16:42:33.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7f99d30b6640 1 -- 172.21.15.26:0/2150153596 --> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f99ac002cc0 con 0x7f99b006dc60 2023-12-19T16:42:33.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7f27385a6640 1 -- 172.21.15.26:0/2138119303 wait complete. 2023-12-19T16:42:33.783 INFO:teuthology.orchestra.run.smithi026.stdout:60129542152 2023-12-19T16:42:33.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fabaffff640 1 -- 172.21.15.26:0/2019521113 <== osd.3 v2:172.21.15.26:6824/38120 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faba8002cc0 con 0x7fabb406dc60 2023-12-19T16:42:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fabcdc12640 1 -- 172.21.15.26:0/2019521113 >> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] conn(0x7fabb406dc60 msgr2=0x7fabb4070080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fabcdc12640 1 --2- 172.21.15.26:0/2019521113 >> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] conn(0x7fabb406dc60 0x7fabb4070080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fabcdc12640 1 -- 172.21.15.26:0/2019521113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fabb4065ac0 msgr2=0x7fabb4067f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fabcdc12640 1 --2- 172.21.15.26:0/2019521113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fabb4065ac0 0x7fabb4067f80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fabc8164c50 tx=0x7fabb8008040 comp rx=0 tx=0).stop 2023-12-19T16:42:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fabcdc12640 1 -- 172.21.15.26:0/2019521113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabc80b2790 msgr2=0x7fabc8157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fabcdc12640 1 --2- 172.21.15.26:0/2019521113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabc80b2790 0x7fabc8157a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fabc0007ef0 tx=0x7fabc000e4b0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fabc7fff640 1 -- 172.21.15.26:0/2019521113 reap_dead start 2023-12-19T16:42:33.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.736+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 --> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb890002cc0 con 0x7fb89406dc00 2023-12-19T16:42:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fb8a5ffb640 1 -- 172.21.15.26:0/2674566269 <== osd.4 v2:172.21.15.160:6800/38418 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb890002cc0 con 0x7fb89406dc00 2023-12-19T16:42:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f99c97fa640 1 -- 172.21.15.26:0/2150153596 <== osd.7 v2:172.21.15.160:6803/38420 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f99ac002cc0 con 0x7f99b006dc60 2023-12-19T16:42:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fabcdc12640 1 -- 172.21.15.26:0/2019521113 shutdown_connections 2023-12-19T16:42:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fabcdc12640 1 -- 172.21.15.26:0/2019521113 >> 172.21.15.26:0/2019521113 conn(0x7fabc80658e0 msgr2=0x7fabc815b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fabcdc12640 1 -- 172.21.15.26:0/2019521113 shutdown_connections 2023-12-19T16:42:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fabcdc12640 1 -- 172.21.15.26:0/2019521113 wait complete. 2023-12-19T16:42:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] conn(0x7fb89406dc00 msgr2=0x7fb894070020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fb8b77ce640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] conn(0x7fb89406dc00 0x7fb894070020 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb894065a60 msgr2=0x7fb894067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fb8b77ce640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb894065a60 0x7fb894067f20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00c530 tx=0x7fb8ac00b040 comp rx=0 tx=0).stop 2023-12-19T16:42:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8b015f140 msgr2=0x7fb8b0153530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fb8b77ce640 1 --2- 172.21.15.26:0/2674566269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8b015f140 0x7fb8b0153530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800c390 tx=0x7fb8a800c850 comp rx=0 tx=0).stop 2023-12-19T16:42:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fb8b5d6b640 1 -- 172.21.15.26:0/2674566269 reap_dead start 2023-12-19T16:42:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 shutdown_connections 2023-12-19T16:42:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 >> 172.21.15.26:0/2674566269 conn(0x7fb8b00b03f0 msgr2=0x7fb8b00afac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 shutdown_connections 2023-12-19T16:42:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7fb8b77ce640 1 -- 172.21.15.26:0/2674566269 wait complete. 2023-12-19T16:42:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f7a3868c640 1 -- 172.21.15.26:0/3991251293 --> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7a18002cc0 con 0x7f7a1406dcd0 2023-12-19T16:42:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f99d30b6640 1 -- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] conn(0x7f99b006dc60 msgr2=0x7f99b0070080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f99d30b6640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] conn(0x7f99b006dc60 0x7f99b0070080 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f99d30b6640 1 -- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f99b0065ac0 msgr2=0x7f99b0067f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f99d30b6640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f99b0065ac0 0x7f99b0067f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99cc070310 tx=0x7f99c0009040 comp rx=0 tx=0).stop 2023-12-19T16:42:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f99d30b6640 1 -- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99cc0d3130 msgr2=0x7f99cc0b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f99d30b6640 1 --2- 172.21.15.26:0/2150153596 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99cc0d3130 0x7f99cc0b31d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f99b80ac040 tx=0x7f99b801ad60 comp rx=0 tx=0).stop 2023-12-19T16:42:33.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f99d1653640 1 -- 172.21.15.26:0/2150153596 reap_dead start 2023-12-19T16:42:33.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-19T16:42:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f7a2effd640 1 -- 172.21.15.26:0/3991251293 <== osd.6 v2:172.21.15.160:6824/38421 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7a18002cc0 con 0x7f7a1406dcd0 2023-12-19T16:42:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f7a3868c640 1 -- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] conn(0x7f7a1406dcd0 msgr2=0x7f7a140700f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f7a3868c640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] conn(0x7f7a1406dcd0 0x7f7a140700f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f99d30b6640 1 -- 172.21.15.26:0/2150153596 shutdown_connections 2023-12-19T16:42:33.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f99d30b6640 1 -- 172.21.15.26:0/2150153596 >> 172.21.15.26:0/2150153596 conn(0x7f99cc06a120 msgr2=0x7f99cc06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f99d30b6640 1 -- 172.21.15.26:0/2150153596 shutdown_connections 2023-12-19T16:42:33.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f99d30b6640 1 -- 172.21.15.26:0/2150153596 wait complete. 2023-12-19T16:42:33.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f7a3868c640 1 -- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a14065ab0 msgr2=0x7f7a14067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f7a3868c640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a14065ab0 0x7f7a14067f70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7a240012f0 tx=0x7f7a24005bc0 comp rx=0 tx=0).stop 2023-12-19T16:42:33.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f7a3868c640 1 -- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a300b1d40 msgr2=0x7f7a30163940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:33.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f7a3868c640 1 --2- 172.21.15.26:0/3991251293 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a300b1d40 0x7f7a30163940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00b3c0 tx=0x7f7a1c00b880 comp rx=0 tx=0).stop 2023-12-19T16:42:33.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.740+0000 7f7a36c29640 1 -- 172.21.15.26:0/3991251293 reap_dead start 2023-12-19T16:42:33.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.744+0000 7f7a3868c640 1 -- 172.21.15.26:0/3991251293 shutdown_connections 2023-12-19T16:42:33.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.744+0000 7f7a3868c640 1 -- 172.21.15.26:0/3991251293 >> 172.21.15.26:0/3991251293 conn(0x7f7a300658e0 msgr2=0x7f7a30164c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:33.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.744+0000 7f7a3868c640 1 -- 172.21.15.26:0/3991251293 shutdown_connections 2023-12-19T16:42:33.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:33.744+0000 7f7a3868c640 1 -- 172.21.15.26:0/3991251293 wait complete. 2023-12-19T16:42:33.792 INFO:teuthology.orchestra.run.smithi026.stdout:64424509446 2023-12-19T16:42:33.792 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-19T16:42:33.793 INFO:teuthology.orchestra.run.smithi026.stdout:64424509445 2023-12-19T16:42:33.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-19T16:42:33.794 INFO:teuthology.orchestra.run.smithi026.stdout:64424509446 2023-12-19T16:42:33.794 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-19T16:42:33.795 INFO:teuthology.orchestra.run.smithi026.stdout:64424509446 2023-12-19T16:42:33.795 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-19T16:42:33.796 INFO:teuthology.orchestra.run.smithi026.stdout:64424509446 2023-12-19T16:42:33.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-19T16:42:33.797 INFO:teuthology.orchestra.run.smithi026.stdout:64424509446 2023-12-19T16:42:33.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-19T16:42:33.798 INFO:teuthology.orchestra.run.smithi026.stdout:64424509446 2023-12-19T16:42:33.798 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-19T16:42:34.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.072+0000 7fcbce7b3640 1 Processor -- start 2023-12-19T16:42:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.076+0000 7fcbce7b3640 1 -- start start 2023-12-19T16:42:34.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.076+0000 7f29c110a640 1 Processor -- start 2023-12-19T16:42:34.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.076+0000 7f29c110a640 1 -- start start 2023-12-19T16:42:34.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.076+0000 7fcbce7b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcbc80ad0c0 0x7fcbc8070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.076+0000 7fcbce7b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcbc80b1d40 0x7fcbc815c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7fcbce7b3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc80706d0 con 0x7fcbc80710a0 2023-12-19T16:42:34.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7fcbce7b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc81616e0 con 0x7fcbc80ad0c0 2023-12-19T16:42:34.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7fcbce7b3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc806fe20 con 0x7fcbc80b1d40 2023-12-19T16:42:34.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7fcbc7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcbc80ad0c0 0x7fcbc8070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7f29c110a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f29b4058a30 0x7f29b40e34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7f29c110a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b403bff0 con 0x7f29b40e3100 2023-12-19T16:42:34.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7f29c110a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b4058140 con 0x7f29b40585d0 2023-12-19T16:42:34.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7f29c110a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b4054790 con 0x7f29b4058a30 2023-12-19T16:42:34.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7fcbc7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcbc80ad0c0 0x7fcbc8070b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49366/0 (socket says 172.21.15.26:49366) 2023-12-19T16:42:34.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7fcbc7fff640 1 -- 172.21.15.26:0/714880953 learned_addr learned my addr 172.21.15.26:0/714880953 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:34.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7fcbc7fff640 1 -- 172.21.15.26:0/714880953 >> v2:172.21.15.160:3300/0 conn(0x7fcbc80b1d40 msgr2=0x7fcbc815c590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-19T16:42:34.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7fcbc7fff640 1 --2- 172.21.15.26:0/714880953 >> v2:172.21.15.160:3300/0 conn(0x7fcbc80b1d40 0x7fcbc815c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7fcbc7fff640 1 -- 172.21.15.26:0/714880953 >> v1:172.21.15.26:6789/0 conn(0x7fcbc80710a0 legacy=0x7fcbc80b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:34.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7f29bad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f29b40e3100 0x7f29b403e870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50292/0 (socket says 172.21.15.26:50292) 2023-12-19T16:42:34.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7f29bad9d640 1 -- 172.21.15.26:0/517897382 learned_addr learned my addr 172.21.15.26:0/517897382 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:34.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7fcbc7fff640 1 -- 172.21.15.26:0/714880953 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc80700c0 con 0x7fcbc80ad0c0 2023-12-19T16:42:34.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.080+0000 7f29bb59e640 1 --2- 172.21.15.26:0/517897382 >> v2:172.21.15.26:3300/0 conn(0x7f29b4058a30 0x7f29b40e34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.084+0000 7f29b9d9b640 1 -- 172.21.15.26:0/517897382 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140237300 0 0) 0x7f29b403bff0 con 0x7f29b40e3100 2023-12-19T16:42:34.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.084+0000 7f29b9d9b640 1 -- 172.21.15.26:0/517897382 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29a4003530 con 0x7f29b40e3100 2023-12-19T16:42:34.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.084+0000 7f29bb59e640 1 -- 172.21.15.26:0/517897382 >> v1:172.21.15.160:6789/0 conn(0x7f29b40585d0 legacy=0x7f29b4006290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:34.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.084+0000 7f29bb59e640 1 -- 172.21.15.26:0/517897382 >> v1:172.21.15.26:6789/0 conn(0x7f29b40e3100 legacy=0x7f29b403e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.084+0000 7f29bb59e640 1 -- 172.21.15.26:0/517897382 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b403ef80 con 0x7f29b4058a30 2023-12-19T16:42:34.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.084+0000 7fcbc7fff640 1 --2- 172.21.15.26:0/714880953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc80ad0c0 0x7fcbc8070b60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcbc806f3d0 tx=0x7fcbb801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765e67fa0680bded server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.084+0000 7fcbc6ffd640 1 -- 172.21.15.26:0/714880953 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80aa610 con 0x7fcbc80ad0c0 2023-12-19T16:42:34.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.084+0000 7fcbc6ffd640 1 -- 172.21.15.26:0/714880953 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb80a6040 con 0x7fcbc80ad0c0 2023-12-19T16:42:34.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7fcbc6ffd640 1 -- 172.21.15.26:0/714880953 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb80aa610 con 0x7fcbc80ad0c0 2023-12-19T16:42:34.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7fcbce7b3640 1 -- 172.21.15.26:0/714880953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc80ad0c0 msgr2=0x7fcbc8070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7f29bb59e640 1 --2- 172.21.15.26:0/517897382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29b4058a30 0x7f29b40e34e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0097c0 tx=0x7f29ac01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4436cba8c20f64c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7fcbce7b3640 1 --2- 172.21.15.26:0/714880953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc80ad0c0 0x7fcbc8070b60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcbc806f3d0 tx=0x7fcbb801c4b0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7fcbce7b3640 1 -- 172.21.15.26:0/714880953 shutdown_connections 2023-12-19T16:42:34.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7fcbce7b3640 1 --2- 172.21.15.26:0/714880953 >> v2:172.21.15.160:3300/0 conn(0x7fcbc80b1d40 0x7fcbc815c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7fcbce7b3640 1 --2- 172.21.15.26:0/714880953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc80ad0c0 0x7fcbc8070b60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7fcbce7b3640 1 -- 172.21.15.26:0/714880953 >> 172.21.15.26:0/714880953 conn(0x7fcbc806b3d0 msgr2=0x7fcbc806b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7f31087f0640 1 Processor -- start 2023-12-19T16:42:34.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7f29b9d9b640 1 -- 172.21.15.26:0/517897382 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0a8970 con 0x7f29b4058a30 2023-12-19T16:42:34.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7f31087f0640 1 -- start start 2023-12-19T16:42:34.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7f29b9d9b640 1 -- 172.21.15.26:0/517897382 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac0a8ad0 con 0x7f29b4058a30 2023-12-19T16:42:34.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7f29c110a640 1 -- 172.21.15.26:0/517897382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29b4058a30 msgr2=0x7f29b40e34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7f29b9d9b640 1 -- 172.21.15.26:0/517897382 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac0a8e40 con 0x7f29b4058a30 2023-12-19T16:42:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7f29c110a640 1 --2- 172.21.15.26:0/517897382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29b4058a30 0x7f29b40e34e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0097c0 tx=0x7f29ac01ec70 comp rx=0 tx=0).stop 2023-12-19T16:42:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7f29c110a640 1 -- 172.21.15.26:0/517897382 shutdown_connections 2023-12-19T16:42:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7f29c110a640 1 --2- 172.21.15.26:0/517897382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29b4058a30 0x7f29b40e34e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7f29c110a640 1 -- 172.21.15.26:0/517897382 >> 172.21.15.26:0/517897382 conn(0x7f29b403aa80 msgr2=0x7f29b403ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7fcbce7b3640 1 -- 172.21.15.26:0/714880953 shutdown_connections 2023-12-19T16:42:34.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7f29c110a640 1 -- 172.21.15.26:0/517897382 shutdown_connections 2023-12-19T16:42:34.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7f29c110a640 1 -- 172.21.15.26:0/517897382 wait complete. 2023-12-19T16:42:34.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f31087f0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f31000b09d0 0x7f31000b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f31087f0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31001614d0 0x7f3100161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f31087f0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31001624f0 con 0x7f310015b5d0 2023-12-19T16:42:34.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f31087f0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31000b3100 con 0x7f31001614d0 2023-12-19T16:42:34.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f31087f0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310015b9b0 con 0x7f31000b09d0 2023-12-19T16:42:34.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.088+0000 7fcbce7b3640 1 -- 172.21.15.26:0/714880953 wait complete. 2023-12-19T16:42:34.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f3105d8b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f31000b09d0 0x7f31000b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f3105d8b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f31000b09d0 0x7f31000b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:33984/0 (socket says 172.21.15.26:33984) 2023-12-19T16:42:34.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f3105d8b640 1 -- 172.21.15.26:0/1717504154 learned_addr learned my addr 172.21.15.26:0/1717504154 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:34.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f310658c640 1 --1- 172.21.15.26:0/1717504154 >> v1:172.21.15.160:6789/0 conn(0x7f310015b5d0 0x7f31000b02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46986/0 (socket says 172.21.15.26:46986) 2023-12-19T16:42:34.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f3106d8d640 1 --2- 172.21.15.26:0/1717504154 >> v2:172.21.15.26:3300/0 conn(0x7f31001614d0 0x7f3100161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7fcbce7b3640 1 Processor -- start 2023-12-19T16:42:34.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f310558a640 1 -- 172.21.15.26:0/1717504154 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182426663 0 0) 0x7f31001624f0 con 0x7f310015b5d0 2023-12-19T16:42:34.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f310558a640 1 -- 172.21.15.26:0/1717504154 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30ec003530 con 0x7f310015b5d0 2023-12-19T16:42:34.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7fcbce7b3640 1 -- start start 2023-12-19T16:42:34.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f29c110a640 1 Processor -- start 2023-12-19T16:42:34.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f29c110a640 1 -- start start 2023-12-19T16:42:34.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f3105d8b640 1 -- 172.21.15.26:0/1717504154 >> v2:172.21.15.26:3300/0 conn(0x7f31001614d0 msgr2=0x7f3100161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:34.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f3105d8b640 1 --2- 172.21.15.26:0/1717504154 >> v2:172.21.15.26:3300/0 conn(0x7f31001614d0 0x7f3100161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f3105d8b640 1 -- 172.21.15.26:0/1717504154 >> v1:172.21.15.160:6789/0 conn(0x7f310015b5d0 legacy=0x7f31000b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.092+0000 7f3105d8b640 1 -- 172.21.15.26:0/1717504154 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3100162210 con 0x7f31000b09d0 2023-12-19T16:42:34.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbce7b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc80ad0c0 0x7fcbc814ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29c110a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29b40585d0 0x7f29b404ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbce7b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcbc80b1d40 0x7fcbc81427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbce7b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc814c580 0x7fcbc8148020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29bad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29b40585d0 0x7f29b404ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29bad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29b40585d0 0x7f29b404ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49396/0 (socket says 172.21.15.26:49396) 2023-12-19T16:42:34.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbc7fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc80ad0c0 0x7fcbc814ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fddd89b3640 1 Processor -- start 2023-12-19T16:42:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbce7b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc814c3d0 con 0x7fcbc814c580 2023-12-19T16:42:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbce7b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc814d340 con 0x7fcbc80ad0c0 2023-12-19T16:42:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbce7b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc8147b90 con 0x7fcbc80b1d40 2023-12-19T16:42:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29bad9d640 1 -- 172.21.15.26:0/2984199761 learned_addr learned my addr 172.21.15.26:0/2984199761 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:34.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29c110a640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29b4058a30 0x7f29b40dd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f0f39bbe640 1 Processor -- start 2023-12-19T16:42:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29c110a640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29b40e3100 0x7f29b40d9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbc77fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcbc80b1d40 0x7fcbc81427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbccd50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc814c580 0x7fcbc8148020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbccd50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc814c580 0x7fcbc8148020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49412/0 (socket says 172.21.15.26:49412) 2023-12-19T16:42:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbccd50640 1 -- 172.21.15.26:0/2243327563 learned_addr learned my addr 172.21.15.26:0/2243327563 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29c110a640 1 -- 172.21.15.26:0/2984199761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b40dac70 con 0x7f29b40585d0 2023-12-19T16:42:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29c110a640 1 -- 172.21.15.26:0/2984199761 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b40de630 con 0x7f29b40e3100 2023-12-19T16:42:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29c110a640 1 -- 172.21.15.26:0/2984199761 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b40df490 con 0x7f29b4058a30 2023-12-19T16:42:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29ba59c640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29b4058a30 0x7f29b40dd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fddd89b3640 1 -- start start 2023-12-19T16:42:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29bb59e640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29b40e3100 0x7f29b40d9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f0f39bbe640 1 -- start start 2023-12-19T16:42:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f6a88596640 1 Processor -- start 2023-12-19T16:42:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbccd50640 1 -- 172.21.15.26:0/2243327563 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcbc80b1d40 msgr2=0x7fcbc81427a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbccd50640 1 --2- 172.21.15.26:0/2243327563 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcbc80b1d40 0x7fcbc81427a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbccd50640 1 -- 172.21.15.26:0/2243327563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc80ad0c0 msgr2=0x7fcbc814ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbccd50640 1 --2- 172.21.15.26:0/2243327563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc80ad0c0 0x7fcbc814ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbccd50640 1 -- 172.21.15.26:0/2243327563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb80094d0 con 0x7fcbc814c580 2023-12-19T16:42:34.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7fcbc7fff640 1 --2- 172.21.15.26:0/2243327563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc80ad0c0 0x7fcbc814ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:34.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29bad9d640 1 -- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29b4058a30 msgr2=0x7f29b40dd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29bad9d640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29b4058a30 0x7f29b40dd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29bad9d640 1 -- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29b40e3100 msgr2=0x7f29b40d9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29bad9d640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29b40e3100 0x7f29b40d9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29bad9d640 1 -- 172.21.15.26:0/2984199761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac009420 con 0x7f29b40585d0 2023-12-19T16:42:34.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f29bb59e640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29b40e3100 0x7f29b40d9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f3105d8b640 1 --2- 172.21.15.26:0/1717504154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31000b09d0 0x7f31000b2bc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f31000b1d90 tx=0x7f30f001c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd5e6d134f6368 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.096+0000 7f6a88596640 1 -- start start 2023-12-19T16:42:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fcbccd50640 1 --2- 172.21.15.26:0/2243327563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc814c580 0x7fcbc8148020 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00b410 tx=0x7fcbbc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f29bad9d640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29b40585d0 0x7f29b404ffd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f29b000c450 tx=0x7f29b000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f310558a640 1 -- 172.21.15.26:0/1717504154 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f00ab600 con 0x7f31000b09d0 2023-12-19T16:42:34.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f310558a640 1 -- 172.21.15.26:0/1717504154 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f00ab760 con 0x7f31000b09d0 2023-12-19T16:42:34.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f310558a640 1 -- 172.21.15.26:0/1717504154 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f00abaf0 con 0x7f31000b09d0 2023-12-19T16:42:34.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fddd89b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fddd40cb1d0 0x7fddd40cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fddd89b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fddd406bee0 0x7fddd406c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fddd89b3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd406d4b0 con 0x7fddd4164c90 2023-12-19T16:42:34.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fddd89b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd40cc030 con 0x7fddd406bee0 2023-12-19T16:42:34.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fddd89b3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd40cbaf0 con 0x7fddd40cb1d0 2023-12-19T16:42:34.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f0f39bbe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f3415cd40 0x7f0f340ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f0f39bbe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f340d3300 0x7f0f34160d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f0f39bbe640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f340d29d0 con 0x7f0f34160900 2023-12-19T16:42:34.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f0f39bbe640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f3415d680 con 0x7f0f3415cd40 2023-12-19T16:42:34.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f0f39bbe640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f34161750 con 0x7f0f340d3300 2023-12-19T16:42:34.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fcbc4ff9640 1 -- 172.21.15.26:0/2243327563 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc011830 con 0x7fcbc814c580 2023-12-19T16:42:34.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fcbce7b3640 1 -- 172.21.15.26:0/2243327563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc8142ce0 con 0x7fcbc814c580 2023-12-19T16:42:34.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fcbce7b3640 1 -- 172.21.15.26:0/2243327563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc814d850 con 0x7fcbc814c580 2023-12-19T16:42:34.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f31087f0640 1 -- 172.21.15.26:0/1717504154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31000b09d0 msgr2=0x7f31000b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f31087f0640 1 --2- 172.21.15.26:0/1717504154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31000b09d0 0x7f31000b2bc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f31000b1d90 tx=0x7f30f001c420 comp rx=0 tx=0).stop 2023-12-19T16:42:34.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f6a88596640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6a800ce040 0x7f6a800d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fcbc4ff9640 1 -- 172.21.15.26:0/2243327563 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc011990 con 0x7fcbc814c580 2023-12-19T16:42:34.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fcbc4ff9640 1 -- 172.21.15.26:0/2243327563 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc011d40 con 0x7fcbc814c580 2023-12-19T16:42:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f29a37fe640 1 -- 172.21.15.26:0/2984199761 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0005cf0 con 0x7f29b40585d0 2023-12-19T16:42:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f29a37fe640 1 -- 172.21.15.26:0/2984199761 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b0005e50 con 0x7f29b40585d0 2023-12-19T16:42:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f29a37fe640 1 -- 172.21.15.26:0/2984199761 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b00072d0 con 0x7f29b40585d0 2023-12-19T16:42:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f6a88596640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a800d3130 0x7f6a800d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f6a88596640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a800d29d0 con 0x7f6a8015cd40 2023-12-19T16:42:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f6a88596640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a800d3570 con 0x7f6a800d3130 2023-12-19T16:42:34.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f6a88596640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8015d680 con 0x7f6a800ce040 2023-12-19T16:42:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fddd259c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fddd40cb1d0 0x7fddd40cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f31087f0640 1 -- 172.21.15.26:0/1717504154 shutdown_connections 2023-12-19T16:42:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f31087f0640 1 --2- 172.21.15.26:0/1717504154 >> v2:172.21.15.26:3300/0 conn(0x7f31001614d0 0x7f3100161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f31087f0640 1 --2- 172.21.15.26:0/1717504154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31000b09d0 0x7f31000b2bc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f31087f0640 1 -- 172.21.15.26:0/1717504154 >> 172.21.15.26:0/1717504154 conn(0x7f3100073360 msgr2=0x7f31000658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fddd259c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fddd40cb1d0 0x7fddd40cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:34016/0 (socket says 172.21.15.26:34016) 2023-12-19T16:42:34.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fddd259c640 1 -- 172.21.15.26:0/1906614086 learned_addr learned my addr 172.21.15.26:0/1906614086 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:34.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f29c110a640 1 -- 172.21.15.26:0/2984199761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b40de880 con 0x7f29b40585d0 2023-12-19T16:42:34.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f29c110a640 1 -- 172.21.15.26:0/2984199761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b40dee40 con 0x7f29b40585d0 2023-12-19T16:42:34.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f6a85b31640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6a800ce040 0x7f6a800d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f6a85b31640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6a800ce040 0x7f6a800d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:34018/0 (socket says 172.21.15.26:34018) 2023-12-19T16:42:34.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f6a85b31640 1 -- 172.21.15.26:0/179341278 learned_addr learned my addr 172.21.15.26:0/179341278 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f0f32ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f0f34160900 0x7f0f340d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46992/0 (socket says 172.21.15.26:46992) 2023-12-19T16:42:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f0f32ffd640 1 -- 172.21.15.26:0/2041305076 learned_addr learned my addr 172.21.15.26:0/2041305076 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fddd1d9b640 1 --2- 172.21.15.26:0/1906614086 >> v2:172.21.15.26:3300/0 conn(0x7fddd406bee0 0x7fddd406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fddd259c640 1 -- 172.21.15.26:0/1906614086 >> v2:172.21.15.26:3300/0 conn(0x7fddd406bee0 msgr2=0x7fddd406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f0f33fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f340d3300 0x7f0f34160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f0f337fe640 1 --2- 172.21.15.26:0/2041305076 >> v2:172.21.15.26:3300/0 conn(0x7f0f3415cd40 0x7f0f340ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f6a85b31640 1 -- 172.21.15.26:0/179341278 >> v2:172.21.15.26:3300/0 conn(0x7f6a800d3130 msgr2=0x7f6a800d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f6a85b31640 1 --2- 172.21.15.26:0/179341278 >> v2:172.21.15.26:3300/0 conn(0x7f6a800d3130 0x7f6a800d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f6a85330640 1 -- 172.21.15.26:0/179341278 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029402909 0 0) 0x7f6a7c001170 con 0x7f6a8015cd40 2023-12-19T16:42:34.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f0f327fc640 1 -- 172.21.15.26:0/2041305076 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 415722347 0 0) 0x7f0f28001120 con 0x7f0f34160900 2023-12-19T16:42:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f6a85b31640 1 -- 172.21.15.26:0/179341278 >> v1:172.21.15.160:6789/0 conn(0x7f6a8015cd40 legacy=0x7f6a80160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7fcbce7b3640 1 -- 172.21.15.26:0/2243327563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc804e850 con 0x7fcbc814c580 2023-12-19T16:42:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f6a85b31640 1 -- 172.21.15.26:0/179341278 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a800ce4b0 con 0x7f6a800ce040 2023-12-19T16:42:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f0f327fc640 1 -- 172.21.15.26:0/2041305076 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f1c003530 con 0x7f0f34160900 2023-12-19T16:42:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f0f33fff640 1 -- 172.21.15.26:0/2041305076 >> v2:172.21.15.26:3300/0 conn(0x7f0f3415cd40 msgr2=0x7f0f340ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f0f33fff640 1 --2- 172.21.15.26:0/2041305076 >> v2:172.21.15.26:3300/0 conn(0x7f0f3415cd40 0x7f0f340ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f0f33fff640 1 -- 172.21.15.26:0/2041305076 >> v1:172.21.15.160:6789/0 conn(0x7f0f34160900 legacy=0x7f0f340d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f0f33fff640 1 -- 172.21.15.26:0/2041305076 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f34161470 con 0x7f0f340d3300 2023-12-19T16:42:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f135a939640 1 Processor -- start 2023-12-19T16:42:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7f135a939640 1 -- start start 2023-12-19T16:42:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fddd259c640 1 --2- 172.21.15.26:0/1906614086 >> v2:172.21.15.26:3300/0 conn(0x7fddd406bee0 0x7fddd406c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fddd259c640 1 -- 172.21.15.26:0/1906614086 >> v1:172.21.15.26:6789/0 conn(0x7fddd4164c90 legacy=0x7fddd4165100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.100+0000 7fddd259c640 1 -- 172.21.15.26:0/1906614086 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd406cc10 con 0x7fddd40cb1d0 2023-12-19T16:42:34.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7fddd1d9b640 1 --2- 172.21.15.26:0/1906614086 >> v2:172.21.15.26:3300/0 conn(0x7fddd406bee0 0x7fddd406c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:34.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f135a939640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f135415c180 0x7f13540ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f135a939640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f13540cbd10 0x7f13540b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f135a939640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13540ce3f0 con 0x7f135415d5c0 2023-12-19T16:42:34.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f135a939640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135415cae0 con 0x7f135415c180 2023-12-19T16:42:34.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f135a939640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135415cdc0 con 0x7f13540cbd10 2023-12-19T16:42:34.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f31087f0640 1 -- 172.21.15.26:0/1717504154 shutdown_connections 2023-12-19T16:42:34.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f1358ed6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f13540cbd10 0x7f13540b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f1358ed6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f13540cbd10 0x7f13540b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:34034/0 (socket says 172.21.15.26:34034) 2023-12-19T16:42:34.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f1358ed6640 1 -- 172.21.15.26:0/1855921807 learned_addr learned my addr 172.21.15.26:0/1855921807 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:34.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7fcbc4ff9640 1 -- 172.21.15.26:0/2243327563 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcbbc01a400 con 0x7fcbc814c580 2023-12-19T16:42:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f1353fff640 1 --2- 172.21.15.26:0/1855921807 >> v2:172.21.15.26:3300/0 conn(0x7f135415c180 0x7f13540ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f1358ed6640 1 -- 172.21.15.26:0/1855921807 >> v2:172.21.15.26:3300/0 conn(0x7f135415c180 msgr2=0x7f13540ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f1358ed6640 1 --2- 172.21.15.26:0/1855921807 >> v2:172.21.15.26:3300/0 conn(0x7f135415c180 0x7f13540ad6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f1358ed6640 1 -- 172.21.15.26:0/1855921807 >> v1:172.21.15.26:6789/0 conn(0x7f135415d5c0 legacy=0x7f13540cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f1358ed6640 1 -- 172.21.15.26:0/1855921807 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135415d0a0 con 0x7f13540cbd10 2023-12-19T16:42:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.104+0000 7f29c110a640 1 -- 172.21.15.26:0/2984199761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299c005260 con 0x7f29b40585d0 2023-12-19T16:42:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fcbc4ff9640 1 --2- 172.21.15.26:0/2243327563 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcbb4065a60 0x7fcbb4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f31087f0640 1 -- 172.21.15.26:0/1717504154 wait complete. 2023-12-19T16:42:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fcbc4ff9640 1 -- 172.21.15.26:0/2243327563 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbbc083240 con 0x7fcbc814c580 2023-12-19T16:42:34.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fddd259c640 1 --2- 172.21.15.26:0/1906614086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddd40cb1d0 0x7fddd40cb5b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fddd40ccb20 tx=0x7fddc801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58ee58aa2c46429b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f6a85b31640 1 --2- 172.21.15.26:0/179341278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a800ce040 0x7f6a800d2bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a800d0bf0 tx=0x7f6a7001c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=172fe74b3734d7c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fcbc7fff640 1 --2- 172.21.15.26:0/2243327563 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcbb4065a60 0x7fcbb4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f6a85330640 1 -- 172.21.15.26:0/179341278 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a700aa660 con 0x7f6a800ce040 2023-12-19T16:42:34.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fddd159a640 1 -- 172.21.15.26:0/1906614086 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc80ab640 con 0x7fddd40cb1d0 2023-12-19T16:42:34.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fddd159a640 1 -- 172.21.15.26:0/1906614086 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc80a7070 con 0x7fddd40cb1d0 2023-12-19T16:42:34.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fddd159a640 1 -- 172.21.15.26:0/1906614086 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc80ab640 con 0x7fddd40cb1d0 2023-12-19T16:42:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f6a85330640 1 -- 172.21.15.26:0/179341278 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a700aa7c0 con 0x7f6a800ce040 2023-12-19T16:42:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f6a85330640 1 -- 172.21.15.26:0/179341278 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a700aa660 con 0x7f6a800ce040 2023-12-19T16:42:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f6a88596640 1 -- 172.21.15.26:0/179341278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a800ce040 msgr2=0x7f6a800d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f6a88596640 1 --2- 172.21.15.26:0/179341278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a800ce040 0x7f6a800d2bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a800d0bf0 tx=0x7f6a7001c500 comp rx=0 tx=0).stop 2023-12-19T16:42:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f29a37fe640 1 -- 172.21.15.26:0/2984199761 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f29b0007430 con 0x7f29b40585d0 2023-12-19T16:42:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fddd89b3640 1 -- 172.21.15.26:0/1906614086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddd40cb1d0 msgr2=0x7fddd40cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fddd89b3640 1 --2- 172.21.15.26:0/1906614086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddd40cb1d0 0x7fddd40cb5b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fddd40ccb20 tx=0x7fddc801c480 comp rx=0 tx=0).stop 2023-12-19T16:42:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fe072a95640 1 Processor -- start 2023-12-19T16:42:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f29a37fe640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f29a8065b30 0x7f29a8067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f29a37fe640 1 -- 172.21.15.26:0/2984199761 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29b0082610 con 0x7f29b40585d0 2023-12-19T16:42:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f0f33fff640 1 --2- 172.21.15.26:0/2041305076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f340d3300 0x7f0f34160d70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009d90 tx=0x7f0f2c01c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=368b3b33038af0fd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f31087f0640 1 Processor -- start 2023-12-19T16:42:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fddd89b3640 1 -- 172.21.15.26:0/1906614086 shutdown_connections 2023-12-19T16:42:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fddd89b3640 1 --2- 172.21.15.26:0/1906614086 >> v2:172.21.15.26:3300/0 conn(0x7fddd406bee0 0x7fddd406c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fddd89b3640 1 --2- 172.21.15.26:0/1906614086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddd40cb1d0 0x7fddd40cb5b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fddd89b3640 1 -- 172.21.15.26:0/1906614086 >> 172.21.15.26:0/1906614086 conn(0x7fddd40658e0 msgr2=0x7fddd4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f6a88596640 1 -- 172.21.15.26:0/179341278 shutdown_connections 2023-12-19T16:42:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f6a88596640 1 --2- 172.21.15.26:0/179341278 >> v2:172.21.15.26:3300/0 conn(0x7f6a800d3130 0x7f6a800d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f6a88596640 1 --2- 172.21.15.26:0/179341278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a800ce040 0x7f6a800d2bf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f6a88596640 1 -- 172.21.15.26:0/179341278 >> 172.21.15.26:0/179341278 conn(0x7f6a8006a120 msgr2=0x7f6a8006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f31087f0640 1 -- start start 2023-12-19T16:42:34.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fe072a95640 1 -- start start 2023-12-19T16:42:34.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fcbc7fff640 1 --2- 172.21.15.26:0/2243327563 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcbb4065a60 0x7fcbb4067f20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80ae000 tx=0x7fcbb80b5040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f6a88596640 1 -- 172.21.15.26:0/179341278 shutdown_connections 2023-12-19T16:42:34.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f29ba59c640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f29a8065b30 0x7f29a8067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f6a88596640 1 -- 172.21.15.26:0/179341278 wait complete. 2023-12-19T16:42:34.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f29ba59c640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f29a8065b30 0x7f29a8067ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29bc068e70 tx=0x7f29bc06c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f0f327fc640 1 -- 172.21.15.26:0/2041305076 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0a5920 con 0x7f0f340d3300 2023-12-19T16:42:34.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f0f327fc640 1 -- 172.21.15.26:0/2041305076 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2c0a5a80 con 0x7f0f340d3300 2023-12-19T16:42:34.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7f0f327fc640 1 -- 172.21.15.26:0/2041305076 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0a5e10 con 0x7f0f340d3300 2023-12-19T16:42:34.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f31087f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31000b09d0 0x7f3100160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f31087f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f310015b5d0 0x7f3100159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2041305076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f340d3300 msgr2=0x7f0f34160d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f0f39bbe640 1 --2- 172.21.15.26:0/2041305076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f340d3300 0x7f0f34160d70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009d90 tx=0x7f0f2c01c950 comp rx=0 tx=0).stop 2023-12-19T16:42:34.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2041305076 shutdown_connections 2023-12-19T16:42:34.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f0f39bbe640 1 --2- 172.21.15.26:0/2041305076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f340d3300 0x7f0f34160d70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f0f39bbe640 1 --2- 172.21.15.26:0/2041305076 >> v2:172.21.15.26:3300/0 conn(0x7f0f3415cd40 0x7f0f340ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2041305076 >> 172.21.15.26:0/2041305076 conn(0x7f0f3406a120 msgr2=0x7f0f3406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f1358ed6640 1 --2- 172.21.15.26:0/1855921807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13540cbd10 0x7f13540b16f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1344008870 tx=0x7f134401e900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=675190461a693875 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fddd89b3640 1 -- 172.21.15.26:0/1906614086 shutdown_connections 2023-12-19T16:42:34.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.108+0000 7fddd89b3640 1 -- 172.21.15.26:0/1906614086 wait complete. 2023-12-19T16:42:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f310658c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31000b09d0 0x7f3100160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f31087f0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31001614d0 0x7f310015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f31087f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3100157df0 con 0x7f310015b5d0 2023-12-19T16:42:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f310658c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31000b09d0 0x7f3100160560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49052/0 (socket says 172.21.15.26:49052) 2023-12-19T16:42:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f31087f0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3100160b40 con 0x7f31001614d0 2023-12-19T16:42:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f310658c640 1 -- 172.21.15.26:0/2835533409 learned_addr learned my addr 172.21.15.26:0/2835533409 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f31087f0640 1 -- 172.21.15.26:0/2835533409 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31001574c0 con 0x7f31000b09d0 2023-12-19T16:42:34.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2041305076 shutdown_connections 2023-12-19T16:42:34.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f1352ffd640 1 -- 172.21.15.26:0/1855921807 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13440a7a30 con 0x7f13540cbd10 2023-12-19T16:42:34.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f3105d8b640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f310015b5d0 0x7f3100159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f3106d8d640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31001614d0 0x7f310015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7fe072a95640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe064050fb0 0x7fe064050190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7fe070831640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe0640dff60 0x7fe06403a610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:47004/0 (socket says 172.21.15.26:47004) 2023-12-19T16:42:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f3105d8b640 1 -- 172.21.15.26:0/2835533409 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31000b09d0 msgr2=0x7f3100160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f3105d8b640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31000b09d0 0x7f3100160560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f3105d8b640 1 -- 172.21.15.26:0/2835533409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31001614d0 msgr2=0x7f310015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f3105d8b640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31001614d0 0x7f310015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f3105d8b640 1 -- 172.21.15.26:0/2835533409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f00094d0 con 0x7f310015b5d0 2023-12-19T16:42:34.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f3105d8b640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f310015b5d0 0x7f3100159bb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f30f00ac000 tx=0x7f30f001ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f6a88596640 1 Processor -- start 2023-12-19T16:42:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f6a88596640 1 -- start start 2023-12-19T16:42:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7fe063fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe064038ff0 0x7fe0640508a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50328/0 (socket says 172.21.15.26:50328) 2023-12-19T16:42:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7fe070831640 1 -- 172.21.15.26:0/2253320760 learned_addr learned my addr 172.21.15.26:0/2253320760 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7fe072a95640 1 -- 172.21.15.26:0/2253320760 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0640e0340 con 0x7fe064038ff0 2023-12-19T16:42:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7fe072a95640 1 -- 172.21.15.26:0/2253320760 --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe06404f550 con 0x7fe0640dff60 2023-12-19T16:42:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7fe072a95640 1 -- 172.21.15.26:0/2253320760 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe064039570 con 0x7fe064050fb0 2023-12-19T16:42:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7fddd89b3640 1 Processor -- start 2023-12-19T16:42:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7fe071032640 1 --2- 172.21.15.26:0/2253320760 >> v2:172.21.15.26:3300/0 conn(0x7fe064050fb0 0x7fe064050190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7fddd89b3640 1 -- start start 2023-12-19T16:42:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f1352ffd640 1 -- 172.21.15.26:0/1855921807 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13440ab070 con 0x7f13540cbd10 2023-12-19T16:42:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f1352ffd640 1 -- 172.21.15.26:0/1855921807 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13440a7a30 con 0x7f13540cbd10 2023-12-19T16:42:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2041305076 wait complete. 2023-12-19T16:42:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7fcbc4ff9640 1 -- 172.21.15.26:0/2243327563 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcbbc050c10 con 0x7fcbc814c580 2023-12-19T16:42:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a88596640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a800ce040 0x7f6a8014d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a88596640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a800d3130 0x7f6a80152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a88596640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a8015cd40 0x7f6a8014b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a88596640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8014d000 con 0x7f6a800d3130 2023-12-19T16:42:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a88596640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a80152b80 con 0x7f6a800ce040 2023-12-19T16:42:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a88596640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a80151880 con 0x7f6a8015cd40 2023-12-19T16:42:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd89b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fddd406bee0 0x7fddd4157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd89b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddd40cb1d0 0x7fddd4152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd89b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fddd4164c90 0x7fddd415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd89b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd40ce4e0 con 0x7fddd406bee0 2023-12-19T16:42:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd89b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd41587b0 con 0x7fddd40cb1d0 2023-12-19T16:42:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f0f39bbe640 1 Processor -- start 2023-12-19T16:42:34.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f135a939640 1 -- 172.21.15.26:0/1855921807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13540cbd10 msgr2=0x7f13540b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f135a939640 1 --2- 172.21.15.26:0/1855921807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13540cbd10 0x7f13540b16f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1344008870 tx=0x7f134401e900 comp rx=0 tx=0).stop 2023-12-19T16:42:34.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137986498 0 0) 0x7fe06404f550 con 0x7fe0640dff60 2023-12-19T16:42:34.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.112+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe054003530 con 0x7fe0640dff60 2023-12-19T16:42:34.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd89b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd415a7d0 con 0x7fddd4164c90 2023-12-19T16:42:34.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f0f39bbe640 1 -- start start 2023-12-19T16:42:34.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f135a939640 1 -- 172.21.15.26:0/1855921807 shutdown_connections 2023-12-19T16:42:34.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f135a939640 1 --2- 172.21.15.26:0/1855921807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13540cbd10 0x7f13540b16f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f135a939640 1 --2- 172.21.15.26:0/1855921807 >> v2:172.21.15.26:3300/0 conn(0x7f135415c180 0x7f13540ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a86332640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a800ce040 0x7f6a8014d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f135a939640 1 -- 172.21.15.26:0/1855921807 >> 172.21.15.26:0/1855921807 conn(0x7f13540b03f0 msgr2=0x7f13540b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a86332640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a800ce040 0x7f6a8014d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:34062/0 (socket says 172.21.15.26:34062) 2023-12-19T16:42:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a86332640 1 -- 172.21.15.26:0/3414511461 learned_addr learned my addr 172.21.15.26:0/3414511461 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd1d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddd40cb1d0 0x7fddd4152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f135a939640 1 -- 172.21.15.26:0/1855921807 shutdown_connections 2023-12-19T16:42:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 <== mon.1 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1580334793 0 0) 0x7fe054003530 con 0x7fe0640dff60 2023-12-19T16:42:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 >> v2:172.21.15.26:3300/0 conn(0x7fe064050fb0 msgr2=0x7fe064050190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe0637fe640 1 --2- 172.21.15.26:0/2253320760 >> v2:172.21.15.26:3300/0 conn(0x7fe064050fb0 0x7fe064050190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 >> v1:172.21.15.26:6789/0 conn(0x7fe064038ff0 legacy=0x7fe0640508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe064039850 con 0x7fe0640dff60 2023-12-19T16:42:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a85b31640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a800d3130 0x7f6a80152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd1d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddd40cb1d0 0x7fddd4152700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:34072/0 (socket says 172.21.15.26:34072) 2023-12-19T16:42:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd1d9b640 1 -- 172.21.15.26:0/3096586058 learned_addr learned my addr 172.21.15.26:0/3096586058 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd259c640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fddd406bee0 0x7fddd4157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd2d9d640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fddd4164c90 0x7fddd415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a86332640 1 -- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a8015cd40 msgr2=0x7f6a8014b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a86b33640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a8015cd40 0x7f6a8014b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a86332640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a8015cd40 0x7f6a8014b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a86332640 1 -- 172.21.15.26:0/3414511461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a800d3130 msgr2=0x7f6a80152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a86332640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a800d3130 0x7f6a80152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a86332640 1 -- 172.21.15.26:0/3414511461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a700094d0 con 0x7f6a800ce040 2023-12-19T16:42:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a85b31640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a800d3130 0x7f6a80152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a86b33640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a8015cd40 0x7f6a8014b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd259c640 1 -- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fddd4164c90 msgr2=0x7fddd415a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a86332640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a800ce040 0x7f6a8014d170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00c410 tx=0x7f6a7c00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd259c640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fddd4164c90 0x7fddd415a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd259c640 1 -- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddd40cb1d0 msgr2=0x7fddd4152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd259c640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddd40cb1d0 0x7fddd4152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd259c640 1 -- 172.21.15.26:0/3096586058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc80094d0 con 0x7fddd406bee0 2023-12-19T16:42:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd2d9d640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fddd4164c90 0x7fddd415a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 <== mon.1 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fe068001680 con 0x7fe0640dff60 2023-12-19T16:42:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 >> v1:172.21.15.160:6789/0 conn(0x7fe0640dff60 legacy=0x7fe06403a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe0637fe640 1 --2- 172.21.15.26:0/2253320760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0540039d0 0x7fe054003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe0637fe640 1 --2- 172.21.15.26:0/2253320760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0540047d0 0x7fe054004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe0637fe640 1 --2- 172.21.15.26:0/2253320760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe054005190 0x7fe05400ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe064039570 con 0x7fe0540047d0 2023-12-19T16:42:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe054009b80 con 0x7fe054005190 2023-12-19T16:42:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe054009d40 con 0x7fe0540039d0 2023-12-19T16:42:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe071032640 1 --2- 172.21.15.26:0/2253320760 >> v2:172.21.15.26:3300/0 conn(0x7fe064050fb0 0x7fe064050190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a76ffd640 1 -- 172.21.15.26:0/3414511461 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c0054e0 con 0x7f6a800ce040 2023-12-19T16:42:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a76ffd640 1 -- 172.21.15.26:0/3414511461 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c003650 con 0x7f6a800ce040 2023-12-19T16:42:34.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a76ffd640 1 -- 172.21.15.26:0/3414511461 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c003a00 con 0x7f6a800ce040 2023-12-19T16:42:34.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f135a939640 1 -- 172.21.15.26:0/1855921807 wait complete. 2023-12-19T16:42:34.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fddd259c640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fddd406bee0 0x7fddd4157dd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fddc80ac000 tx=0x7fddc8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe071032640 1 --2- 172.21.15.26:0/2253320760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe054005190 0x7fe05400ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe070831640 1 --2- 172.21.15.26:0/2253320760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0540039d0 0x7fe054003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a88596640 1 -- 172.21.15.26:0/3414511461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a80151b60 con 0x7f6a800ce040 2023-12-19T16:42:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7f6a88596640 1 -- 172.21.15.26:0/3414511461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a80152120 con 0x7f6a800ce040 2023-12-19T16:42:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe063fff640 1 --2- 172.21.15.26:0/2253320760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0540047d0 0x7fe054004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe070831640 1 -- 172.21.15.26:0/2253320760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe054005190 msgr2=0x7fe05400ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe070831640 1 --2- 172.21.15.26:0/2253320760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe054005190 0x7fe05400ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe070831640 1 -- 172.21.15.26:0/2253320760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0540047d0 msgr2=0x7fe054004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe070831640 1 --2- 172.21.15.26:0/2253320760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0540047d0 0x7fe054004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe070831640 1 -- 172.21.15.26:0/2253320760 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe054003530 con 0x7fe0540039d0 2023-12-19T16:42:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.116+0000 7fe063fff640 1 --2- 172.21.15.26:0/2253320760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0540047d0 0x7fe054004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7fe071032640 1 -- 172.21.15.26:0/2253320760 reap_dead start 2023-12-19T16:42:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f39bbe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f340d3300 0x7f0f3406d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f39bbe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f3415cd40 0x7f0f3406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f39bbe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f34160900 0x7f0f3406f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f39bbe640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3406d5d0 con 0x7f0f340d3300 2023-12-19T16:42:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f39bbe640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3406fd10 con 0x7f0f3415cd40 2023-12-19T16:42:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f39bbe640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f34202af0 con 0x7f0f34160900 2023-12-19T16:42:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f33fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f34160900 0x7f0f3406f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f33fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f34160900 0x7f0f3406f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49090/0 (socket says 172.21.15.26:49090) 2023-12-19T16:42:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f33fff640 1 -- 172.21.15.26:0/2299659237 learned_addr learned my addr 172.21.15.26:0/2299659237 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f6a88596640 1 -- 172.21.15.26:0/3414511461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a80074270 con 0x7f6a800ce040 2023-12-19T16:42:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f32ffd640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f3415cd40 0x7f0f3406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f337fe640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f340d3300 0x7f0f3406d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f33fff640 1 -- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f3415cd40 msgr2=0x7f0f3406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f33fff640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f3415cd40 0x7f0f3406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f33fff640 1 -- 172.21.15.26:0/2299659237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f340d3300 msgr2=0x7f0f3406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f135a939640 1 Processor -- start 2023-12-19T16:42:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f33fff640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f340d3300 0x7f0f3406d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f33fff640 1 -- 172.21.15.26:0/2299659237 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c009a70 con 0x7f0f34160900 2023-12-19T16:42:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f33fff640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f34160900 0x7f0f3406f280 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009d90 tx=0x7f0f2c0196f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f135a939640 1 -- start start 2023-12-19T16:42:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f6a76ffd640 1 -- 172.21.15.26:0/3414511461 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6a7c003b60 con 0x7f6a800ce040 2023-12-19T16:42:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f1bfff640 1 -- 172.21.15.26:0/2299659237 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0aec30 con 0x7f0f34160900 2023-12-19T16:42:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f30f6ffd640 1 -- 172.21.15.26:0/2835533409 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f00be5f0 con 0x7f310015b5d0 2023-12-19T16:42:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f30f6ffd640 1 -- 172.21.15.26:0/2835533409 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f00be770 con 0x7f310015b5d0 2023-12-19T16:42:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7fddc2ffd640 1 -- 172.21.15.26:0/3096586058 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8003bf0 con 0x7fddd406bee0 2023-12-19T16:42:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f30f6ffd640 1 -- 172.21.15.26:0/2835533409 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f00beb20 con 0x7f310015b5d0 2023-12-19T16:42:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7fddc2ffd640 1 -- 172.21.15.26:0/3096586058 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc8003d50 con 0x7fddd406bee0 2023-12-19T16:42:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7fddc2ffd640 1 -- 172.21.15.26:0/3096586058 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc80ab660 con 0x7fddd406bee0 2023-12-19T16:42:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f29a37fe640 1 -- 172.21.15.26:0/2984199761 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f29b0050060 con 0x7f29b40585d0 2023-12-19T16:42:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f1bfff640 1 -- 172.21.15.26:0/2299659237 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2c0aed90 con 0x7f0f34160900 2023-12-19T16:42:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f1bfff640 1 -- 172.21.15.26:0/2299659237 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0b7650 con 0x7f0f34160900 2023-12-19T16:42:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2299659237 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f34202dd0 con 0x7f0f34160900 2023-12-19T16:42:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2299659237 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f34203390 con 0x7f0f34160900 2023-12-19T16:42:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f6a76ffd640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6a64065b30 0x7f6a64067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f31087f0640 1 -- 172.21.15.26:0/2835533409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31001577a0 con 0x7f310015b5d0 2023-12-19T16:42:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.120+0000 7f31087f0640 1 -- 172.21.15.26:0/2835533409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310014eff0 con 0x7f310015b5d0 2023-12-19T16:42:34.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f6a76ffd640 1 -- 172.21.15.26:0/3414511461 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a7c082720 con 0x7f6a800ce040 2023-12-19T16:42:34.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fddd89b3640 1 -- 172.21.15.26:0/3096586058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd41584d0 con 0x7fddd406bee0 2023-12-19T16:42:34.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fddd89b3640 1 -- 172.21.15.26:0/3096586058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd40ce9d0 con 0x7fddd406bee0 2023-12-19T16:42:34.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f135a939640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13540cbd10 0x7f135406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fe070831640 1 --2- 172.21.15.26:0/2253320760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0540039d0 0x7fe054003df0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe068001ec0 tx=0x7fe06801c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a10b9d6d7f0829a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f135a939640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f135415c180 0x7f1354159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f135a939640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f135415d5c0 0x7f135406b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f135a939640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135415ade0 con 0x7f135415c180 2023-12-19T16:42:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f135a939640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135415a160 con 0x7f135415d5c0 2023-12-19T16:42:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f135a939640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135415a440 con 0x7f13540cbd10 2023-12-19T16:42:34.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068005ae0 con 0x7fe0540039d0 2023-12-19T16:42:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe068005c40 con 0x7fe0540039d0 2023-12-19T16:42:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fe0637fe640 1 -- 172.21.15.26:0/2253320760 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0680072b0 con 0x7fe0540039d0 2023-12-19T16:42:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f31087f0640 1 -- 172.21.15.26:0/2835533409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31000cc000 con 0x7f310015b5d0 2023-12-19T16:42:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f6a85b31640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6a64065b30 0x7f6a64067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fddd89b3640 1 -- 172.21.15.26:0/3096586058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd4074270 con 0x7fddd406bee0 2023-12-19T16:42:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fe072a95640 1 -- 172.21.15.26:0/2253320760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0540039d0 msgr2=0x7fe054003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fe072a95640 1 --2- 172.21.15.26:0/2253320760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0540039d0 0x7fe054003df0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe068001ec0 tx=0x7fe06801c9b0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2299659237 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f34074270 con 0x7f0f34160900 2023-12-19T16:42:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fe072a95640 1 -- 172.21.15.26:0/2253320760 shutdown_connections 2023-12-19T16:42:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f1353fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13540cbd10 0x7f135406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f1353fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13540cbd10 0x7f135406c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49100/0 (socket says 172.21.15.26:49100) 2023-12-19T16:42:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f1353fff640 1 -- 172.21.15.26:0/1316286025 learned_addr learned my addr 172.21.15.26:0/1316286025 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fe072a95640 1 --2- 172.21.15.26:0/2253320760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0540039d0 0x7fe054003df0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fe072a95640 1 -- 172.21.15.26:0/2253320760 >> 172.21.15.26:0/2253320760 conn(0x7fe064054c50 msgr2=0x7fe064055060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fe072a95640 1 -- 172.21.15.26:0/2253320760 shutdown_connections 2023-12-19T16:42:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f1353fff640 1 -- 172.21.15.26:0/1316286025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f135415d5c0 msgr2=0x7f135406b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f1353fff640 1 --2- 172.21.15.26:0/1316286025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f135415d5c0 0x7f135406b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f1353fff640 1 -- 172.21.15.26:0/1316286025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f135415c180 msgr2=0x7f1354159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f1353fff640 1 --2- 172.21.15.26:0/1316286025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f135415c180 0x7f1354159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f1353fff640 1 -- 172.21.15.26:0/1316286025 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1344008550 con 0x7f13540cbd10 2023-12-19T16:42:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7fe072a95640 1 -- 172.21.15.26:0/2253320760 wait complete. 2023-12-19T16:42:34.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f30f6ffd640 1 -- 172.21.15.26:0/2835533409 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f30f00b43f0 con 0x7f310015b5d0 2023-12-19T16:42:34.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f1353fff640 1 --2- 172.21.15.26:0/1316286025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13540cbd10 0x7f135406c7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f134c007ef0 tx=0x7f134c00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.124+0000 7f0f1bfff640 1 -- 172.21.15.26:0/2299659237 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0f2c0b4050 con 0x7f0f34160900 2023-12-19T16:42:34.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.128+0000 7f6a85b31640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6a64065b30 0x7f6a64067ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6a700ae000 tx=0x7f6a70009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.128+0000 7f0f1bfff640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f20065a60 0x7f0f20067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.128+0000 7f0f1bfff640 1 -- 172.21.15.26:0/2299659237 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f2c11e4b0 con 0x7f0f34160900 2023-12-19T16:42:34.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.128+0000 7f30f6ffd640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f30e4065b30 0x7f30e4067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.128+0000 7f30f6ffd640 1 -- 172.21.15.26:0/2835533409 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30f011e290 con 0x7f310015b5d0 2023-12-19T16:42:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.128+0000 7f0f337fe640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f20065a60 0x7f0f20067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.128+0000 7f6a76ffd640 1 -- 172.21.15.26:0/3414511461 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6a7c050030 con 0x7f6a800ce040 2023-12-19T16:42:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.128+0000 7fddc2ffd640 1 -- 172.21.15.26:0/3096586058 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fddc80ab7c0 con 0x7fddd406bee0 2023-12-19T16:42:34.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.128+0000 7f310658c640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f30e4065b30 0x7f30e4067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.128+0000 7f0f337fe640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f20065a60 0x7f0f20067f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009ad0 tx=0x7f0f24008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.128+0000 7fe072a95640 1 Processor -- start 2023-12-19T16:42:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.128+0000 7fe072a95640 1 -- start start 2023-12-19T16:42:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.128+0000 7fddc2ffd640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fddb4065b30 0x7fddb4067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7f310658c640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f30e4065b30 0x7f30e4067ff0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f30fc000c10 tx=0x7f30fc007e20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fddc2ffd640 1 -- 172.21.15.26:0/3096586058 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddc811dc00 con 0x7fddd406bee0 2023-12-19T16:42:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fddd1d9b640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fddb4065b30 0x7fddb4067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fddd1d9b640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fddb4065b30 0x7fddb4067ff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fddd41577e0 tx=0x7fddc4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe072a95640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0540039d0 0x7fe0640368b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe072a95640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe064036df0 0x7fe0640d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe072a95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0640d8610 0x7fe0640dbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe072a95640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0640d8a90 con 0x7fe0640d8610 2023-12-19T16:42:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe072a95640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0640498c0 con 0x7fe0540039d0 2023-12-19T16:42:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe072a95640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe064049ba0 con 0x7fe064036df0 2023-12-19T16:42:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe070831640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0540039d0 0x7fe0640368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe070831640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0540039d0 0x7fe0640368b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:34118/0 (socket says 172.21.15.26:34118) 2023-12-19T16:42:34.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe070831640 1 -- 172.21.15.26:0/1147781282 learned_addr learned my addr 172.21.15.26:0/1147781282 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:34.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7f0f1bfff640 1 -- 172.21.15.26:0/2299659237 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0f2c0ebe60 con 0x7f0f34160900 2023-12-19T16:42:34.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe071032640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0640d8610 0x7fe0640dbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe063fff640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe064036df0 0x7fe0640d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7f30f6ffd640 1 -- 172.21.15.26:0/2835533409 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f30f00ebd70 con 0x7f310015b5d0 2023-12-19T16:42:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe070831640 1 -- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe064036df0 msgr2=0x7fe0640d80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe070831640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe064036df0 0x7fe0640d80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe070831640 1 -- 172.21.15.26:0/1147781282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0640d8610 msgr2=0x7fe0640dbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe070831640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0640d8610 0x7fe0640dbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7fe070831640 1 -- 172.21.15.26:0/1147781282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe064049e80 con 0x7fe0540039d0 2023-12-19T16:42:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7f1350ff9640 1 -- 172.21.15.26:0/1316286025 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c014620 con 0x7f13540cbd10 2023-12-19T16:42:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7f1350ff9640 1 -- 172.21.15.26:0/1316286025 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134c014780 con 0x7f13540cbd10 2023-12-19T16:42:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.132+0000 7f1350ff9640 1 -- 172.21.15.26:0/1316286025 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c014b10 con 0x7f13540cbd10 2023-12-19T16:42:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.136+0000 7f135a939640 1 -- 172.21.15.26:0/1316286025 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135415a720 con 0x7f13540cbd10 2023-12-19T16:42:34.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.136+0000 7f135a939640 1 -- 172.21.15.26:0/1316286025 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135406cd00 con 0x7f13540cbd10 2023-12-19T16:42:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.136+0000 7fe070831640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0540039d0 0x7fe0640368b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe06801c7e0 tx=0x7fe054005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.136+0000 7fe0617fa640 1 -- 172.21.15.26:0/1147781282 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0680b4070 con 0x7fe0540039d0 2023-12-19T16:42:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.136+0000 7fe0617fa640 1 -- 172.21.15.26:0/1147781282 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe068007070 con 0x7fe0540039d0 2023-12-19T16:42:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.136+0000 7fe0617fa640 1 -- 172.21.15.26:0/1147781282 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068001680 con 0x7fe0540039d0 2023-12-19T16:42:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.136+0000 7fe072a95640 1 -- 172.21.15.26:0/1147781282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0640360b0 con 0x7fe0540039d0 2023-12-19T16:42:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.136+0000 7fe072a95640 1 -- 172.21.15.26:0/1147781282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0640d9190 con 0x7fe0540039d0 2023-12-19T16:42:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.136+0000 7f1350ff9640 1 -- 172.21.15.26:0/1316286025 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f134c014c70 con 0x7f13540cbd10 2023-12-19T16:42:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.140+0000 7f135a939640 1 -- 172.21.15.26:0/1316286025 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f135415af20 con 0x7f13540cbd10 2023-12-19T16:42:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.140+0000 7f1350ff9640 1 --2- 172.21.15.26:0/1316286025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1340065b30 0x7f1340067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.140+0000 7f1350ff9640 1 -- 172.21.15.26:0/1316286025 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f134c083ad0 con 0x7f13540cbd10 2023-12-19T16:42:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.140+0000 7fe072a95640 1 -- 172.21.15.26:0/1147781282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0640054d0 con 0x7fe0540039d0 2023-12-19T16:42:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.140+0000 7fddc2ffd640 1 -- 172.21.15.26:0/3096586058 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fddc80eb6e0 con 0x7fddd406bee0 2023-12-19T16:42:34.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.144+0000 7fe0617fa640 1 -- 172.21.15.26:0/1147781282 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe068007420 con 0x7fe0540039d0 2023-12-19T16:42:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.144+0000 7f13537fe640 1 --2- 172.21.15.26:0/1316286025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1340065b30 0x7f1340067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.144+0000 7f13537fe640 1 --2- 172.21.15.26:0/1316286025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1340065b30 0x7f1340067ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1354071520 tx=0x7f1348006bb0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.144+0000 7fe0617fa640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe058065b60 0x7fe058068020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:34.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.144+0000 7f1350ff9640 1 -- 172.21.15.26:0/1316286025 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f135415af20 con 0x7f13540cbd10 2023-12-19T16:42:34.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.144+0000 7fe0617fa640 1 -- 172.21.15.26:0/1147781282 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe06811cbc0 con 0x7fe0540039d0 2023-12-19T16:42:34.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.148+0000 7fe063fff640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe058065b60 0x7fe058068020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:34.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.148+0000 7fe063fff640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe058065b60 0x7fe058068020 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe05c000c10 tx=0x7fe05c007e10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:34.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.148+0000 7fe0617fa640 1 -- 172.21.15.26:0/1147781282 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe0680ea610 con 0x7fe0540039d0 2023-12-19T16:42:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.448+0000 7fcbce7b3640 1 -- 172.21.15.26:0/2243327563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcbc8147cd0 con 0x7fcbc814c580 2023-12-19T16:42:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.452+0000 7fcbc4ff9640 1 -- 172.21.15.26:0/2243327563 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcbbc054230 con 0x7fcbc814c580 2023-12-19T16:42:34.455 INFO:teuthology.orchestra.run.smithi026.stdout:60129542150 2023-12-19T16:42:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.456+0000 7fcbce7b3640 1 -- 172.21.15.26:0/2243327563 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcbb4065a60 msgr2=0x7fcbb4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.456+0000 7fcbce7b3640 1 --2- 172.21.15.26:0/2243327563 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcbb4065a60 0x7fcbb4067f20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80ae000 tx=0x7fcbb80b5040 comp rx=0 tx=0).stop 2023-12-19T16:42:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.456+0000 7fcbce7b3640 1 -- 172.21.15.26:0/2243327563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc814c580 msgr2=0x7fcbc8148020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.456+0000 7fcbce7b3640 1 --2- 172.21.15.26:0/2243327563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc814c580 0x7fcbc8148020 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc00b410 tx=0x7fcbbc00b8d0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.456+0000 7fcbce7b3640 1 -- 172.21.15.26:0/2243327563 shutdown_connections 2023-12-19T16:42:34.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.456+0000 7fcbce7b3640 1 --2- 172.21.15.26:0/2243327563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbc814c580 0x7fcbc8148020 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.456+0000 7fcbce7b3640 1 --2- 172.21.15.26:0/2243327563 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcbb4065a60 0x7fcbb4067f20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.456+0000 7fcbce7b3640 1 --2- 172.21.15.26:0/2243327563 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcbc80b1d40 0x7fcbc81427a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.456+0000 7fcbce7b3640 1 --2- 172.21.15.26:0/2243327563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc80ad0c0 0x7fcbc814ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.456+0000 7fcbce7b3640 1 -- 172.21.15.26:0/2243327563 >> 172.21.15.26:0/2243327563 conn(0x7fcbc806b3d0 msgr2=0x7fcbc80af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.456+0000 7fcbce7b3640 1 -- 172.21.15.26:0/2243327563 shutdown_connections 2023-12-19T16:42:34.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.456+0000 7fcbce7b3640 1 -- 172.21.15.26:0/2243327563 wait complete. 2023-12-19T16:42:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.460+0000 7f29c110a640 1 -- 172.21.15.26:0/2984199761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f299c005510 con 0x7f29b40585d0 2023-12-19T16:42:34.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.464+0000 7f29a37fe640 1 -- 172.21.15.26:0/2984199761 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f29b0053680 con 0x7f29b40585d0 2023-12-19T16:42:34.468 INFO:teuthology.orchestra.run.smithi026.stdout:64424509445 2023-12-19T16:42:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.464+0000 7f6a88596640 1 -- 172.21.15.26:0/3414511461 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6a801676b0 con 0x7f6a800ce040 2023-12-19T16:42:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.464+0000 7f6a76ffd640 1 -- 172.21.15.26:0/3414511461 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6a7c017090 con 0x7f6a800ce040 2023-12-19T16:42:34.470 INFO:teuthology.orchestra.run.smithi026.stdout:64424509445 2023-12-19T16:42:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f29c110a640 1 -- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f29a8065b30 msgr2=0x7f29a8067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f29c110a640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f29a8065b30 0x7f29a8067ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f29bc068e70 tx=0x7f29bc06c040 comp rx=0 tx=0).stop 2023-12-19T16:42:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f29c110a640 1 -- 172.21.15.26:0/2984199761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29b40585d0 msgr2=0x7f29b404ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f29c110a640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29b40585d0 0x7f29b404ffd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f29b000c450 tx=0x7f29b000c850 comp rx=0 tx=0).stop 2023-12-19T16:42:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f29c110a640 1 -- 172.21.15.26:0/2984199761 shutdown_connections 2023-12-19T16:42:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f29c110a640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29b40e3100 0x7f29b40d9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f29c110a640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f29a8065b30 0x7f29a8067ff0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f29c110a640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29b4058a30 0x7f29b40dd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f29c110a640 1 --2- 172.21.15.26:0/2984199761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29b40585d0 0x7f29b404ffd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f29c110a640 1 -- 172.21.15.26:0/2984199761 >> 172.21.15.26:0/2984199761 conn(0x7f29b403aa80 msgr2=0x7f29b403df90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f29c110a640 1 -- 172.21.15.26:0/2984199761 shutdown_connections 2023-12-19T16:42:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f29c110a640 1 -- 172.21.15.26:0/2984199761 wait complete. 2023-12-19T16:42:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f6a74ff9640 1 -- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6a64065b30 msgr2=0x7f6a64067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f6a74ff9640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6a64065b30 0x7f6a64067ff0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6a700ae000 tx=0x7f6a70009040 comp rx=0 tx=0).stop 2023-12-19T16:42:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f6a74ff9640 1 -- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a800ce040 msgr2=0x7f6a8014d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f6a74ff9640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a800ce040 0x7f6a8014d170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00c410 tx=0x7f6a7c00c8d0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f6a74ff9640 1 -- 172.21.15.26:0/3414511461 shutdown_connections 2023-12-19T16:42:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f6a74ff9640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6a64065b30 0x7f6a64067ff0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f6a74ff9640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a8015cd40 0x7f6a8014b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f6a74ff9640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a800d3130 0x7f6a80152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f6a74ff9640 1 --2- 172.21.15.26:0/3414511461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a800ce040 0x7f6a8014d170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.468+0000 7f6a74ff9640 1 -- 172.21.15.26:0/3414511461 >> 172.21.15.26:0/3414511461 conn(0x7f6a8006a120 msgr2=0x7f6a80158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.472+0000 7f6a74ff9640 1 -- 172.21.15.26:0/3414511461 shutdown_connections 2023-12-19T16:42:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.472+0000 7f6a74ff9640 1 -- 172.21.15.26:0/3414511461 wait complete. 2023-12-19T16:42:34.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.472+0000 7f31087f0640 1 -- 172.21.15.26:0/2835533409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f31000ce3d0 con 0x7f310015b5d0 2023-12-19T16:42:34.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.472+0000 7f30f6ffd640 1 -- 172.21.15.26:0/2835533409 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f30f00ef390 con 0x7f310015b5d0 2023-12-19T16:42:34.476 INFO:teuthology.orchestra.run.smithi026.stdout:64424509444 2023-12-19T16:42:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2299659237 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0f34202c30 con 0x7f0f34160900 2023-12-19T16:42:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f31087f0640 1 -- 172.21.15.26:0/2835533409 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f30e4065b30 msgr2=0x7f30e4067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f31087f0640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f30e4065b30 0x7f30e4067ff0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f30fc000c10 tx=0x7f30fc007e20 comp rx=0 tx=0).stop 2023-12-19T16:42:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f31087f0640 1 -- 172.21.15.26:0/2835533409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f310015b5d0 msgr2=0x7f3100159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f31087f0640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f310015b5d0 0x7f3100159bb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f30f00ac000 tx=0x7f30f001ac60 comp rx=0 tx=0).stop 2023-12-19T16:42:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f0f1bfff640 1 -- 172.21.15.26:0/2299659237 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0f34202c30 con 0x7f0f34160900 2023-12-19T16:42:34.481 INFO:teuthology.orchestra.run.smithi026.stdout:64424509445 2023-12-19T16:42:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f31087f0640 1 -- 172.21.15.26:0/2835533409 shutdown_connections 2023-12-19T16:42:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f31087f0640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31001614d0 0x7f310015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f31087f0640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f310015b5d0 0x7f3100159bb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f31087f0640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f30e4065b30 0x7f30e4067ff0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f31087f0640 1 --2- 172.21.15.26:0/2835533409 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31000b09d0 0x7f3100160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f31087f0640 1 -- 172.21.15.26:0/2835533409 >> 172.21.15.26:0/2835533409 conn(0x7f3100073360 msgr2=0x7f310006bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f31087f0640 1 -- 172.21.15.26:0/2835533409 shutdown_connections 2023-12-19T16:42:34.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.476+0000 7f31087f0640 1 -- 172.21.15.26:0/2835533409 wait complete. 2023-12-19T16:42:34.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f20065a60 msgr2=0x7f0f20067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7f0f39bbe640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f20065a60 0x7f0f20067f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009ad0 tx=0x7f0f24008040 comp rx=0 tx=0).stop 2023-12-19T16:42:34.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f34160900 msgr2=0x7f0f3406f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7f0f39bbe640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f34160900 0x7f0f3406f280 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009d90 tx=0x7f0f2c0196f0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2299659237 shutdown_connections 2023-12-19T16:42:34.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7f0f39bbe640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f20065a60 0x7f0f20067f20 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009ad0 tx=0x7f0f24008040 comp rx=0 tx=0).stop 2023-12-19T16:42:34.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7f0f39bbe640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f34160900 0x7f0f3406f280 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7f0f39bbe640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f3415cd40 0x7f0f3406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7f0f39bbe640 1 --2- 172.21.15.26:0/2299659237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f340d3300 0x7f0f3406d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2299659237 >> 172.21.15.26:0/2299659237 conn(0x7f0f3406a120 msgr2=0x7f0f340b64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7fddd89b3640 1 -- 172.21.15.26:0/3096586058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fddd415a910 con 0x7fddd406bee0 2023-12-19T16:42:34.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7fddc2ffd640 1 -- 172.21.15.26:0/3096586058 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fddc80c4020 con 0x7fddd406bee0 2023-12-19T16:42:34.486 INFO:teuthology.orchestra.run.smithi026.stdout:64424509445 2023-12-19T16:42:34.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2299659237 shutdown_connections 2023-12-19T16:42:34.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.480+0000 7f0f39bbe640 1 -- 172.21.15.26:0/2299659237 wait complete. 2023-12-19T16:42:34.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.484+0000 7f135a939640 1 -- 172.21.15.26:0/1316286025 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f135415af20 con 0x7f13540cbd10 2023-12-19T16:42:34.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.484+0000 7f1350ff9640 1 -- 172.21.15.26:0/1316286025 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f135415af20 con 0x7f13540cbd10 2023-12-19T16:42:34.488 INFO:teuthology.orchestra.run.smithi026.stdout:64424509445 2023-12-19T16:42:34.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.484+0000 7fe072a95640 1 -- 172.21.15.26:0/1147781282 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe064036560 con 0x7fe0540039d0 2023-12-19T16:42:34.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7fe0617fa640 1 -- 172.21.15.26:0/1147781282 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe0680edc30 con 0x7fe0540039d0 2023-12-19T16:42:34.491 INFO:teuthology.orchestra.run.smithi026.stdout:64424509445 2023-12-19T16:42:34.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7f135a939640 1 -- 172.21.15.26:0/1316286025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1340065b30 msgr2=0x7f1340067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7f135a939640 1 --2- 172.21.15.26:0/1316286025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1340065b30 0x7f1340067ff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1354071520 tx=0x7f1348006bb0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7fddd89b3640 1 -- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fddb4065b30 msgr2=0x7fddb4067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7fddd89b3640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fddb4065b30 0x7fddb4067ff0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fddd41577e0 tx=0x7fddc4008040 comp rx=0 tx=0).stop 2023-12-19T16:42:34.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7f135a939640 1 -- 172.21.15.26:0/1316286025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13540cbd10 msgr2=0x7f135406c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7f135a939640 1 --2- 172.21.15.26:0/1316286025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13540cbd10 0x7f135406c7c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f134c007ef0 tx=0x7f134c00e4b0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7fddd89b3640 1 -- 172.21.15.26:0/3096586058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fddd406bee0 msgr2=0x7fddd4157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7fddd89b3640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fddd406bee0 0x7fddd4157dd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fddc80ac000 tx=0x7fddc8019220 comp rx=0 tx=0).stop 2023-12-19T16:42:34.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7fddd89b3640 1 -- 172.21.15.26:0/3096586058 shutdown_connections 2023-12-19T16:42:34.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7fddd89b3640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fddb4065b30 0x7fddb4067ff0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fddd41577e0 tx=0x7fddc4008040 comp rx=0 tx=0).stop 2023-12-19T16:42:34.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7fddd89b3640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fddd4164c90 0x7fddd415a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7fddd89b3640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fddd40cb1d0 0x7fddd4152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7fddd89b3640 1 --2- 172.21.15.26:0/3096586058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fddd406bee0 0x7fddd4157dd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7fddd89b3640 1 -- 172.21.15.26:0/3096586058 >> 172.21.15.26:0/3096586058 conn(0x7fddd40658e0 msgr2=0x7fddd4161530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7f135a939640 1 -- 172.21.15.26:0/1316286025 shutdown_connections 2023-12-19T16:42:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7f135a939640 1 --2- 172.21.15.26:0/1316286025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f135415d5c0 0x7f135406b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7f135a939640 1 --2- 172.21.15.26:0/1316286025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f135415c180 0x7f1354159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7f135a939640 1 --2- 172.21.15.26:0/1316286025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1340065b30 0x7f1340067ff0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7f135a939640 1 --2- 172.21.15.26:0/1316286025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13540cbd10 0x7f135406c7c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7f135a939640 1 -- 172.21.15.26:0/1316286025 >> 172.21.15.26:0/1316286025 conn(0x7f13540b03f0 msgr2=0x7f13540cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7f135a939640 1 -- 172.21.15.26:0/1316286025 shutdown_connections 2023-12-19T16:42:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7f135a939640 1 -- 172.21.15.26:0/1316286025 wait complete. 2023-12-19T16:42:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7fddd89b3640 1 -- 172.21.15.26:0/3096586058 shutdown_connections 2023-12-19T16:42:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.488+0000 7fddd89b3640 1 -- 172.21.15.26:0/3096586058 wait complete. 2023-12-19T16:42:34.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.0 2023-12-19T16:42:34.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.492+0000 7fe072a95640 1 -- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe058065b60 msgr2=0x7fe058068020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.492+0000 7fe072a95640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe058065b60 0x7fe058068020 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe05c000c10 tx=0x7fe05c007e10 comp rx=0 tx=0).stop 2023-12-19T16:42:34.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.492+0000 7fe072a95640 1 -- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0540039d0 msgr2=0x7fe0640368b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:34.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.492+0000 7fe072a95640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0540039d0 0x7fe0640368b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe06801c7e0 tx=0x7fe054005800 comp rx=0 tx=0).stop 2023-12-19T16:42:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.492+0000 7fe072a95640 1 -- 172.21.15.26:0/1147781282 shutdown_connections 2023-12-19T16:42:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.492+0000 7fe072a95640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0640d8610 0x7fe0640dbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.492+0000 7fe072a95640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe058065b60 0x7fe058068020 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.492+0000 7fe072a95640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe064036df0 0x7fe0640d80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.492+0000 7fe072a95640 1 --2- 172.21.15.26:0/1147781282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0540039d0 0x7fe0640368b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.492+0000 7fe072a95640 1 -- 172.21.15.26:0/1147781282 >> 172.21.15.26:0/1147781282 conn(0x7fe064054c50 msgr2=0x7fe064053420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.492+0000 7fe072a95640 1 -- 172.21.15.26:0/1147781282 shutdown_connections 2023-12-19T16:42:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:34.492+0000 7fe072a95640 1 -- 172.21.15.26:0/1147781282 wait complete. 2023-12-19T16:42:34.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2023-12-19T16:42:34.517 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.1 2023-12-19T16:42:34.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.2 2023-12-19T16:42:34.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2023-12-19T16:42:34.528 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.6 2023-12-19T16:42:34.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2023-12-19T16:42:34.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.7 2023-12-19T16:42:35.497 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-19T16:42:35.512 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-19T16:42:35.518 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-19T16:42:35.524 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-19T16:42:35.525 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-19T16:42:35.529 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-19T16:42:35.536 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-19T16:42:35.536 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-19T16:42:35.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.720+0000 7f1a3ae2d640 1 Processor -- start 2023-12-19T16:42:35.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.720+0000 7f1a3ae2d640 1 -- start start 2023-12-19T16:42:35.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.720+0000 7f1a3ae2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a340cd890 0x7f1a34159180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.724+0000 7f1a3ae2d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a34071d60 0x7f1a3406f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.724+0000 7f1a3ae2d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a341596c0 con 0x7f1a340ce5b0 2023-12-19T16:42:35.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.724+0000 7f1a3ae2d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3406faa0 con 0x7f1a34071d60 2023-12-19T16:42:35.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.724+0000 7f1a3ae2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3406fd80 con 0x7f1a340cd890 2023-12-19T16:42:35.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.724+0000 7f1a33fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a34071d60 0x7f1a3406f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.724+0000 7f1a33fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a34071d60 0x7f1a3406f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49516/0 (socket says 172.21.15.26:49516) 2023-12-19T16:42:35.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.724+0000 7f1a33fff640 1 -- 172.21.15.26:0/1278827550 learned_addr learned my addr 172.21.15.26:0/1278827550 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.724+0000 7f1a38bc9640 1 --2- 172.21.15.26:0/1278827550 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a340cd890 0x7f1a34159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.724+0000 7f1a33fff640 1 -- 172.21.15.26:0/1278827550 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a340cd890 msgr2=0x7f1a34159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.724+0000 7f1a33fff640 1 --2- 172.21.15.26:0/1278827550 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a340cd890 0x7f1a34159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.724+0000 7f1a33fff640 1 -- 172.21.15.26:0/1278827550 >> v1:172.21.15.160:6789/0 conn(0x7f1a340ce5b0 legacy=0x7f1a3415ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:35.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.724+0000 7f1a33fff640 1 -- 172.21.15.26:0/1278827550 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a34070060 con 0x7f1a34071d60 2023-12-19T16:42:35.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.732+0000 7f1a33fff640 1 --2- 172.21.15.26:0/1278827550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a34071d60 0x7f1a3406f0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009ad0 tx=0x7f1a2401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c241c351cdfe67c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.732+0000 7f1a337fe640 1 -- 172.21.15.26:0/1278827550 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a240a6650 con 0x7f1a34071d60 2023-12-19T16:42:35.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.732+0000 7f1a337fe640 1 -- 172.21.15.26:0/1278827550 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a240a67b0 con 0x7f1a34071d60 2023-12-19T16:42:35.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.732+0000 7f1a337fe640 1 -- 172.21.15.26:0/1278827550 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a240a6b40 con 0x7f1a34071d60 2023-12-19T16:42:35.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.732+0000 7f1a3ae2d640 1 -- 172.21.15.26:0/1278827550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a34071d60 msgr2=0x7f1a3406f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.732+0000 7f1a3ae2d640 1 --2- 172.21.15.26:0/1278827550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a34071d60 0x7f1a3406f0c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009ad0 tx=0x7f1a2401c730 comp rx=0 tx=0).stop 2023-12-19T16:42:35.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.732+0000 7f1a3ae2d640 1 -- 172.21.15.26:0/1278827550 shutdown_connections 2023-12-19T16:42:35.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.732+0000 7f1a3ae2d640 1 --2- 172.21.15.26:0/1278827550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a34071d60 0x7f1a3406f0c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.732+0000 7f1a3ae2d640 1 --2- 172.21.15.26:0/1278827550 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a340cd890 0x7f1a34159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.732+0000 7f1a3ae2d640 1 -- 172.21.15.26:0/1278827550 >> 172.21.15.26:0/1278827550 conn(0x7f1a3406b3d0 msgr2=0x7f1a3406b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:35.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.732+0000 7f1a3ae2d640 1 -- 172.21.15.26:0/1278827550 shutdown_connections 2023-12-19T16:42:35.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.732+0000 7f1a3ae2d640 1 -- 172.21.15.26:0/1278827550 wait complete. 2023-12-19T16:42:35.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.736+0000 7f1a3ae2d640 1 Processor -- start 2023-12-19T16:42:35.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.736+0000 7f1a3ae2d640 1 -- start start 2023-12-19T16:42:35.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.740+0000 7f1a3ae2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a34071d60 0x7f1a340cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.740+0000 7f1a3ae2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a340cd890 0x7f1a340ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.740+0000 7f1a3ae2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340ce5b0 0x7f1a34148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.740+0000 7f1a33fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a340cd890 0x7f1a340ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.740+0000 7f1a33fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a340cd890 0x7f1a340ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49128/0 (socket says 172.21.15.26:49128) 2023-12-19T16:42:35.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.740+0000 7f1a33fff640 1 -- 172.21.15.26:0/2944544657 learned_addr learned my addr 172.21.15.26:0/2944544657 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.740+0000 7f1a3ae2d640 1 -- 172.21.15.26:0/2944544657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a34153e90 con 0x7f1a34071d60 2023-12-19T16:42:35.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.740+0000 7f1a3ae2d640 1 -- 172.21.15.26:0/2944544657 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a34154170 con 0x7f1a340ce5b0 2023-12-19T16:42:35.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.740+0000 7f1a38bc9640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a34071d60 0x7f1a340cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.740+0000 7f1a393ca640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340ce5b0 0x7f1a34148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.740+0000 7f1a3ae2d640 1 -- 172.21.15.26:0/2944544657 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a34154450 con 0x7f1a340cd890 2023-12-19T16:42:35.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.744+0000 7f1a38bc9640 1 -- 172.21.15.26:0/2944544657 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a340cd890 msgr2=0x7f1a340ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.744+0000 7f1a38bc9640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a340cd890 0x7f1a340ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.744+0000 7f1a38bc9640 1 -- 172.21.15.26:0/2944544657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340ce5b0 msgr2=0x7f1a34148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.744+0000 7f1a38bc9640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340ce5b0 0x7f1a34148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.744+0000 7f1a38bc9640 1 -- 172.21.15.26:0/2944544657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a240094d0 con 0x7f1a34071d60 2023-12-19T16:42:35.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.744+0000 7f1a38bc9640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a34071d60 0x7f1a340cbdd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800c7f0 tx=0x7f1a2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.744+0000 7f1a317fa640 1 -- 172.21.15.26:0/2944544657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28004220 con 0x7f1a34071d60 2023-12-19T16:42:35.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.744+0000 7f1a3ae2d640 1 -- 172.21.15.26:0/2944544657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a34154730 con 0x7f1a34071d60 2023-12-19T16:42:35.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.744+0000 7f1a3ae2d640 1 -- 172.21.15.26:0/2944544657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a340cc310 con 0x7f1a34071d60 2023-12-19T16:42:35.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.744+0000 7f1a317fa640 1 -- 172.21.15.26:0/2944544657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2800a3b0 con 0x7f1a34071d60 2023-12-19T16:42:35.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.744+0000 7f1a317fa640 1 -- 172.21.15.26:0/2944544657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28003820 con 0x7f1a34071d60 2023-12-19T16:42:35.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.748+0000 7f1a317fa640 1 -- 172.21.15.26:0/2944544657 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1a28003980 con 0x7f1a34071d60 2023-12-19T16:42:35.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.752+0000 7f1a1affd640 1 -- 172.21.15.26:0/2944544657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3404e850 con 0x7f1a34071d60 2023-12-19T16:42:35.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.752+0000 7f1a317fa640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1a20065ac0 0x7f1a20067f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.752+0000 7f1a33fff640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1a20065ac0 0x7f1a20067f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.756+0000 7f1a33fff640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1a20065ac0 0x7f1a20067f80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a2401c770 tx=0x7f1a240b3040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.756+0000 7f1a317fa640 1 -- 172.21.15.26:0/2944544657 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a28054720 con 0x7f1a34071d60 2023-12-19T16:42:35.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.764+0000 7f1a317fa640 1 -- 172.21.15.26:0/2944544657 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1a2801e080 con 0x7f1a34071d60 2023-12-19T16:42:35.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.796+0000 7f261a441640 1 Processor -- start 2023-12-19T16:42:35.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.796+0000 7f261a441640 1 -- start start 2023-12-19T16:42:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.796+0000 7f261a441640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26140ce040 0x7f26140d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.796+0000 7f261a441640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140d29d0 con 0x7f261415cd40 2023-12-19T16:42:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.796+0000 7f261a441640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140d3570 con 0x7f26140d3130 2023-12-19T16:42:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.796+0000 7f261a441640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261415d680 con 0x7f26140ce040 2023-12-19T16:42:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.796+0000 7f26189de640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f26140d3130 0x7f26140d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:47030/0 (socket says 172.21.15.26:47030) 2023-12-19T16:42:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.796+0000 7f26189de640 1 -- 172.21.15.26:0/2581153073 learned_addr learned my addr 172.21.15.26:0/2581153073 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:35.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.800+0000 7f26137fe640 1 --2- 172.21.15.26:0/2581153073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26140ce040 0x7f26140d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.800+0000 7f2612ffd640 1 -- 172.21.15.26:0/2581153073 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161952071 0 0) 0x7f26140d3570 con 0x7f26140d3130 2023-12-19T16:42:35.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.800+0000 7f2612ffd640 1 -- 172.21.15.26:0/2581153073 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25fc003530 con 0x7f26140d3130 2023-12-19T16:42:35.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.800+0000 7f26137fe640 1 -- 172.21.15.26:0/2581153073 >> v1:172.21.15.160:6789/0 conn(0x7f26140d3130 legacy=0x7f26140d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.800+0000 7f26137fe640 1 -- 172.21.15.26:0/2581153073 >> v1:172.21.15.26:6789/0 conn(0x7f261415cd40 legacy=0x7f2614160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:35.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.800+0000 7f26137fe640 1 -- 172.21.15.26:0/2581153073 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2614161750 con 0x7f26140ce040 2023-12-19T16:42:35.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.804+0000 7fc7cfe8c640 1 Processor -- start 2023-12-19T16:42:35.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.804+0000 7f5324d43640 1 Processor -- start 2023-12-19T16:42:35.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.804+0000 7fc7cfe8c640 1 -- start start 2023-12-19T16:42:35.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.804+0000 7f5324d43640 1 -- start start 2023-12-19T16:42:35.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.804+0000 7f26137fe640 1 --2- 172.21.15.26:0/2581153073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26140ce040 0x7f26140d2bf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26080097f0 tx=0x7f260801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82097a58f05af802 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.804+0000 7f2612ffd640 1 -- 172.21.15.26:0/2581153073 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26080a6040 con 0x7f26140ce040 2023-12-19T16:42:35.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.804+0000 7f2612ffd640 1 -- 172.21.15.26:0/2581153073 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26080aa600 con 0x7f26140ce040 2023-12-19T16:42:35.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.804+0000 7f2612ffd640 1 -- 172.21.15.26:0/2581153073 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26080aa990 con 0x7f26140ce040 2023-12-19T16:42:35.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7fc7cfe8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7c80cb1d0 0x7fc7c80cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f5324d43640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f532015cca0 0x7f53200b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f5324d43640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f53200b8760 0x7f53200cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f5324d43640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53200b6b20 0x7f53200cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f5324d43640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53200b7c30 con 0x7f532015cca0 2023-12-19T16:42:35.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f261a441640 1 -- 172.21.15.26:0/2581153073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26140ce040 msgr2=0x7f26140d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f261a441640 1 --2- 172.21.15.26:0/2581153073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26140ce040 0x7f26140d2bf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26080097f0 tx=0x7f260801c4a0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f5324d43640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53200b6fb0 con 0x7f53200b8760 2023-12-19T16:42:35.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f5324d43640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53200b7290 con 0x7f53200b6b20 2023-12-19T16:42:35.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f531effd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f53200b8760 0x7f53200cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f531ffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53200b6b20 0x7f53200cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f531effd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f53200b8760 0x7f53200cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:34128/0 (socket says 172.21.15.26:34128) 2023-12-19T16:42:35.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f531effd640 1 -- 172.21.15.26:0/2286071053 learned_addr learned my addr 172.21.15.26:0/2286071053 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7fc7cdc28640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7c80cb1d0 0x7fc7c80cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7fc7cfe8c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc7c80b1d40 0x7fc7c80b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f531f7fe640 1 --2- 172.21.15.26:0/2286071053 >> v2:172.21.15.26:3300/0 conn(0x7f532015cca0 0x7f53200b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7fc7cfe8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c806d4b0 con 0x7fc7c806bee0 2023-12-19T16:42:35.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7fc7cfe8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c80cc030 con 0x7fc7c80cb1d0 2023-12-19T16:42:35.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7fc7cfe8c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c80cbaf0 con 0x7fc7c80b1d40 2023-12-19T16:42:35.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7fc7cdc28640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc7c80cb1d0 0x7fc7c80cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49524/0 (socket says 172.21.15.26:49524) 2023-12-19T16:42:35.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7fc7cdc28640 1 -- 172.21.15.26:0/3640158533 learned_addr learned my addr 172.21.15.26:0/3640158533 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7fc7ce429640 1 --2- 172.21.15.26:0/3640158533 >> v2:172.21.15.160:3300/0 conn(0x7fc7c80b1d40 0x7fc7c80b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f531effd640 1 -- 172.21.15.26:0/2286071053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53200b6b20 msgr2=0x7f53200cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f531effd640 1 --2- 172.21.15.26:0/2286071053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53200b6b20 0x7f53200cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f531effd640 1 -- 172.21.15.26:0/2286071053 >> v2:172.21.15.26:3300/0 conn(0x7f532015cca0 msgr2=0x7f53200b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f531effd640 1 --2- 172.21.15.26:0/2286071053 >> v2:172.21.15.26:3300/0 conn(0x7f532015cca0 0x7f53200b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7f531effd640 1 -- 172.21.15.26:0/2286071053 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53200b7570 con 0x7f53200b8760 2023-12-19T16:42:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7fc7ce429640 1 -- 172.21.15.26:0/3640158533 >> v2:172.21.15.26:3300/0 conn(0x7fc7c80cb1d0 msgr2=0x7fc7c80cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7fc7ce429640 1 --2- 172.21.15.26:0/3640158533 >> v2:172.21.15.26:3300/0 conn(0x7fc7c80cb1d0 0x7fc7c80cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7fc7ce429640 1 -- 172.21.15.26:0/3640158533 >> v1:172.21.15.26:6789/0 conn(0x7fc7c806bee0 legacy=0x7fc7c806c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.808+0000 7fc7ce429640 1 -- 172.21.15.26:0/3640158533 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c806cc10 con 0x7fc7c80b1d40 2023-12-19T16:42:35.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7f261a441640 1 -- 172.21.15.26:0/2581153073 shutdown_connections 2023-12-19T16:42:35.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7f261a441640 1 --2- 172.21.15.26:0/2581153073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26140ce040 0x7f26140d2bf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7f261a441640 1 -- 172.21.15.26:0/2581153073 >> 172.21.15.26:0/2581153073 conn(0x7f261406a120 msgr2=0x7f261406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:35.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7f2d0e387640 1 Processor -- start 2023-12-19T16:42:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7fefefe55640 1 Processor -- start 2023-12-19T16:42:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7fefefe55640 1 -- start start 2023-12-19T16:42:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7f2d0e387640 1 -- start start 2023-12-19T16:42:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7f261a441640 1 -- 172.21.15.26:0/2581153073 shutdown_connections 2023-12-19T16:42:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7f261a441640 1 -- 172.21.15.26:0/2581153073 wait complete. 2023-12-19T16:42:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7f2d0e387640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2d0815ce00 0x7f2d080c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7f2d0e387640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2d080cbb80 0x7f2d080ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7f2d0e387640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080b3c60 0x7f2d080c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7f2d0e387640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d080b45d0 con 0x7f2d0815ce00 2023-12-19T16:42:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7f2d0e387640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d081618c0 con 0x7f2d080cbb80 2023-12-19T16:42:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.812+0000 7f2d0e387640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08160b00 con 0x7f2d080b3c60 2023-12-19T16:42:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefefe55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefe815f370 0x7fefe815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefefe55640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fefe80ce740 0x7fefe80cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefefe55640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe806c970 con 0x7fefe8155030 2023-12-19T16:42:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefefe55640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe806c790 con 0x7fefe80ce740 2023-12-19T16:42:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefefe55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe80cf130 con 0x7fefe815f370 2023-12-19T16:42:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefedbf1640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fefe8155030 0x7fefe806c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:47040/0 (socket says 172.21.15.26:47040) 2023-12-19T16:42:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefedbf1640 1 -- 172.21.15.26:0/1204560213 learned_addr learned my addr 172.21.15.26:0/1204560213 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f2d0cb84640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2d080cbb80 0x7f2d080ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f2d0cb84640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2d080cbb80 0x7f2d080ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:34146/0 (socket says 172.21.15.26:34146) 2023-12-19T16:42:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f2d0cb84640 1 -- 172.21.15.26:0/3860118454 learned_addr learned my addr 172.21.15.26:0/3860118454 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f2d0d385640 1 --2- 172.21.15.26:0/3860118454 >> v2:172.21.15.26:3300/0 conn(0x7f2d0815ce00 0x7f2d080c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f2d0db86640 1 --2- 172.21.15.26:0/3860118454 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080b3c60 0x7f2d080c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f261a441640 1 Processor -- start 2023-12-19T16:42:35.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f2d0cb84640 1 -- 172.21.15.26:0/3860118454 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080b3c60 msgr2=0x7f2d080c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f2d0cb84640 1 --2- 172.21.15.26:0/3860118454 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080b3c60 0x7f2d080c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f2d0cb84640 1 -- 172.21.15.26:0/3860118454 >> v2:172.21.15.26:3300/0 conn(0x7f2d0815ce00 msgr2=0x7f2d080c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefecbef640 1 -- 172.21.15.26:0/1204560213 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075057454 0 0) 0x7fefe806c970 con 0x7fefe8155030 2023-12-19T16:42:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefed3f0640 1 --2- 172.21.15.26:0/1204560213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefe815f370 0x7fefe815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefecbef640 1 -- 172.21.15.26:0/1204560213 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefd4003530 con 0x7fefe8155030 2023-12-19T16:42:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f2d0cb84640 1 --2- 172.21.15.26:0/3860118454 >> v2:172.21.15.26:3300/0 conn(0x7f2d0815ce00 0x7f2d080c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f2d0cb84640 1 -- 172.21.15.26:0/3860118454 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d08160de0 con 0x7f2d080cbb80 2023-12-19T16:42:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f2d0db86640 1 --2- 172.21.15.26:0/3860118454 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080b3c60 0x7f2d080c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f2d0d385640 1 --2- 172.21.15.26:0/3860118454 >> v2:172.21.15.26:3300/0 conn(0x7f2d0815ce00 0x7f2d080c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefee3f2640 1 --2- 172.21.15.26:0/1204560213 >> v2:172.21.15.26:3300/0 conn(0x7fefe80ce740 0x7fefe80cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f261a441640 1 -- start start 2023-12-19T16:42:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefed3f0640 1 -- 172.21.15.26:0/1204560213 >> v2:172.21.15.26:3300/0 conn(0x7fefe80ce740 msgr2=0x7fefe80cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefed3f0640 1 --2- 172.21.15.26:0/1204560213 >> v2:172.21.15.26:3300/0 conn(0x7fefe80ce740 0x7fefe80cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefed3f0640 1 -- 172.21.15.26:0/1204560213 >> v1:172.21.15.160:6789/0 conn(0x7fefe8155030 legacy=0x7fefe806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fefed3f0640 1 -- 172.21.15.26:0/1204560213 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe815b1f0 con 0x7fefe815f370 2023-12-19T16:42:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f531effd640 1 --2- 172.21.15.26:0/2286071053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53200b8760 0x7f53200cd670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53200b8050 tx=0x7f531801c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7a14738bc13dd17 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f01e528c640 1 Processor -- start 2023-12-19T16:42:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f01e528c640 1 -- start start 2023-12-19T16:42:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f531e7fc640 1 -- 172.21.15.26:0/2286071053 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180a5840 con 0x7f53200b8760 2023-12-19T16:42:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7fc7ce429640 1 --2- 172.21.15.26:0/3640158533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7c80b1d40 0x7fc7c80b21b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80cca50 tx=0x7fc7c001c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0f94ce2ba6ae3b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f261a441640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26140ce040 0x7f2614159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f261a441640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26140d3130 0x7f261415e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f261a441640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f261415cd40 0x7f261414bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f261a441640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26140ca060 con 0x7f26140ce040 2023-12-19T16:42:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f261a441640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261415b320 con 0x7f261415cd40 2023-12-19T16:42:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f261a441640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261415a710 con 0x7f26140d3130 2023-12-19T16:42:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f531e7fc640 1 -- 172.21.15.26:0/2286071053 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53180a59a0 con 0x7f53200b8760 2023-12-19T16:42:35.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f531e7fc640 1 -- 172.21.15.26:0/2286071053 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180a5d30 con 0x7f53200b8760 2023-12-19T16:42:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.816+0000 7f26189de640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f261415cd40 0x7f261414bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f26189de640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f261415cd40 0x7f261414bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:34156/0 (socket says 172.21.15.26:34156) 2023-12-19T16:42:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f26189de640 1 -- 172.21.15.26:0/2345157704 learned_addr learned my addr 172.21.15.26:0/2345157704 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f2613fff640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26140ce040 0x7f2614159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f26137fe640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26140d3130 0x7f261415e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f5324d43640 1 -- 172.21.15.26:0/2286071053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53200b8760 msgr2=0x7f53200cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f5324d43640 1 --2- 172.21.15.26:0/2286071053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53200b8760 0x7f53200cd670 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53200b8050 tx=0x7f531801c9a0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f26189de640 1 -- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26140d3130 msgr2=0x7f261415e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f26189de640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26140d3130 0x7f261415e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f26189de640 1 -- 172.21.15.26:0/2345157704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26140ce040 msgr2=0x7f2614159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7fc7ccc26640 1 -- 172.21.15.26:0/3640158533 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00a5940 con 0x7fc7c80b1d40 2023-12-19T16:42:35.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f5324d43640 1 -- 172.21.15.26:0/2286071053 shutdown_connections 2023-12-19T16:42:35.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f5324d43640 1 --2- 172.21.15.26:0/2286071053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53200b6b20 0x7f53200cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f5324d43640 1 --2- 172.21.15.26:0/2286071053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53200b8760 0x7f53200cd670 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f5324d43640 1 --2- 172.21.15.26:0/2286071053 >> v2:172.21.15.26:3300/0 conn(0x7f532015cca0 0x7f53200b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f5324d43640 1 -- 172.21.15.26:0/2286071053 >> 172.21.15.26:0/2286071053 conn(0x7f532006a120 msgr2=0x7f532006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:35.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f5324d43640 1 -- 172.21.15.26:0/2286071053 shutdown_connections 2023-12-19T16:42:35.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f5324d43640 1 -- 172.21.15.26:0/2286071053 wait complete. 2023-12-19T16:42:35.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f01e528c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f01e015d5c0 0x7f01e00ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f01e528c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e015cae0 con 0x7f01e015c630 2023-12-19T16:42:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f01e528c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e015cdc0 con 0x7f01e00b0f10 2023-12-19T16:42:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f01e528c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e015d0a0 con 0x7f01e015d5c0 2023-12-19T16:42:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7fc7ccc26640 1 -- 172.21.15.26:0/3640158533 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c00a5aa0 con 0x7fc7c80b1d40 2023-12-19T16:42:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7fc7ccc26640 1 -- 172.21.15.26:0/3640158533 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00a5940 con 0x7fc7c80b1d40 2023-12-19T16:42:35.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f26189de640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26140ce040 0x7f2614159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f01deffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f01e015d5c0 0x7f01e00ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f26189de640 1 -- 172.21.15.26:0/2345157704 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2608009420 con 0x7f261415cd40 2023-12-19T16:42:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f01deffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f01e015d5c0 0x7f01e00ad6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:34172/0 (socket says 172.21.15.26:34172) 2023-12-19T16:42:35.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f01deffd640 1 -- 172.21.15.26:0/339699280 learned_addr learned my addr 172.21.15.26:0/339699280 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f2613fff640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26140ce040 0x7f2614159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:35.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f2ffbd18640 1 Processor -- start 2023-12-19T16:42:35.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f26189de640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f261415cd40 0x7f261414bc80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f260c00cb70 tx=0x7f260c00a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/3640158533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7c80b1d40 msgr2=0x7fc7c80b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7fc7cfe8c640 1 --2- 172.21.15.26:0/3640158533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7c80b1d40 0x7fc7c80b21b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80cca50 tx=0x7fc7c001c8a0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/3640158533 shutdown_connections 2023-12-19T16:42:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7fc7cfe8c640 1 --2- 172.21.15.26:0/3640158533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7c80b1d40 0x7fc7c80b21b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7fc7cfe8c640 1 --2- 172.21.15.26:0/3640158533 >> v2:172.21.15.26:3300/0 conn(0x7fc7c80cb1d0 0x7fc7c80cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f01deffd640 1 -- 172.21.15.26:0/339699280 >> v1:172.21.15.160:6789/0 conn(0x7f01e00b0f10 legacy=0x7f01e015bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7fefed3f0640 1 --2- 172.21.15.26:0/1204560213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefe815f370 0x7fefe815bcd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fefd80097c0 tx=0x7fefd801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30254e3101cf75b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f01ddffb640 1 -- 172.21.15.26:0/339699280 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105114540 0 0) 0x7f01e015cdc0 con 0x7f01e00b0f10 2023-12-19T16:42:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/3640158533 >> 172.21.15.26:0/3640158533 conn(0x7fc7c80658e0 msgr2=0x7fc7c8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:35.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f2610ff9640 1 -- 172.21.15.26:0/2345157704 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c00ad40 con 0x7f261415cd40 2023-12-19T16:42:35.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f2ffbd18640 1 -- start start 2023-12-19T16:42:35.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/3640158533 shutdown_connections 2023-12-19T16:42:35.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f2610ff9640 1 -- 172.21.15.26:0/2345157704 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c012400 con 0x7f261415cd40 2023-12-19T16:42:35.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f2610ff9640 1 -- 172.21.15.26:0/2345157704 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c00aea0 con 0x7f261415cd40 2023-12-19T16:42:35.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/3640158533 wait complete. 2023-12-19T16:42:35.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f261a441640 1 -- 172.21.15.26:0/2345157704 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261415a9f0 con 0x7f261415cd40 2023-12-19T16:42:35.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7fefecbef640 1 -- 172.21.15.26:0/1204560213 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd80ab650 con 0x7fefe815f370 2023-12-19T16:42:35.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f261a441640 1 -- 172.21.15.26:0/2345157704 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261415afb0 con 0x7f261415cd40 2023-12-19T16:42:35.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7fefecbef640 1 -- 172.21.15.26:0/1204560213 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd80ab7b0 con 0x7fefe815f370 2023-12-19T16:42:35.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7fefecbef640 1 -- 172.21.15.26:0/1204560213 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd80abb40 con 0x7fefe815f370 2023-12-19T16:42:35.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f5324d43640 1 Processor -- start 2023-12-19T16:42:35.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f01deffd640 1 -- 172.21.15.26:0/339699280 >> v1:172.21.15.26:6789/0 conn(0x7f01e015c630 legacy=0x7f01e00cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.820+0000 7f01deffd640 1 -- 172.21.15.26:0/339699280 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e00cbd10 con 0x7f01e015d5c0 2023-12-19T16:42:35.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2d0cb84640 1 --2- 172.21.15.26:0/3860118454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d080cbb80 0x7f2d080ca3b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2d080adf80 tx=0x7f2d0401c210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30511bc578a8cc73 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f5324d43640 1 -- start start 2023-12-19T16:42:35.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ffbd18640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ff40ce040 0x7f2ff40d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ffbd18640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2ff40d3130 0x7f2ff40d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ffbd18640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff40d29d0 con 0x7f2ff415cd40 2023-12-19T16:42:35.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ffbd18640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff40d3570 con 0x7f2ff40d3130 2023-12-19T16:42:35.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ffbd18640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff415d680 con 0x7f2ff40ce040 2023-12-19T16:42:35.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2cfffff640 1 -- 172.21.15.26:0/3860118454 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040ab600 con 0x7f2d080cbb80 2023-12-19T16:42:35.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2cfffff640 1 -- 172.21.15.26:0/3860118454 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d040a7050 con 0x7f2d080cbb80 2023-12-19T16:42:35.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2cfffff640 1 -- 172.21.15.26:0/3860118454 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d040ab980 con 0x7f2d080cbb80 2023-12-19T16:42:35.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ff9ab4640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f2ff415cd40 0x7f2ff4160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:47060/0 (socket says 172.21.15.26:47060) 2023-12-19T16:42:35.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7fc7cfe8c640 1 Processor -- start 2023-12-19T16:42:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7fefefe55640 1 -- 172.21.15.26:0/1204560213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefe815f370 msgr2=0x7fefe815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7fefefe55640 1 --2- 172.21.15.26:0/1204560213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefe815f370 0x7fefe815bcd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fefd80097c0 tx=0x7fefd801c450 comp rx=0 tx=0).stop 2023-12-19T16:42:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2d0e387640 1 -- 172.21.15.26:0/3860118454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d080cbb80 msgr2=0x7f2d080ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2d0e387640 1 --2- 172.21.15.26:0/3860118454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d080cbb80 0x7f2d080ca3b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2d080adf80 tx=0x7f2d0401c210 comp rx=0 tx=0).stop 2023-12-19T16:42:35.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f5324d43640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53200b8760 0x7f5320157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f5324d43640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f532015cca0 0x7f53200d2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f5324d43640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532015a600 0x7f53200d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f5324d43640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5320157cb0 con 0x7f53200b8760 2023-12-19T16:42:35.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f5324d43640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532015aa90 con 0x7f532015cca0 2023-12-19T16:42:35.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f5324d43640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532015ad70 con 0x7f532015a600 2023-12-19T16:42:35.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7fc7cfe8c640 1 -- start start 2023-12-19T16:42:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7fefefe55640 1 -- 172.21.15.26:0/1204560213 shutdown_connections 2023-12-19T16:42:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7fefefe55640 1 --2- 172.21.15.26:0/1204560213 >> v2:172.21.15.26:3300/0 conn(0x7fefe80ce740 0x7fefe80cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7fefefe55640 1 --2- 172.21.15.26:0/1204560213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefe815f370 0x7fefe815bcd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7fefefe55640 1 -- 172.21.15.26:0/1204560213 >> 172.21.15.26:0/1204560213 conn(0x7fefe80658e0 msgr2=0x7fefe8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f531effd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f532015cca0 0x7f53200d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f531effd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f532015cca0 0x7f53200d2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:34190/0 (socket says 172.21.15.26:34190) 2023-12-19T16:42:35.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f531effd640 1 -- 172.21.15.26:0/3662249956 learned_addr learned my addr 172.21.15.26:0/3662249956 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ff92b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ff40ce040 0x7f2ff40d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2d0e387640 1 -- 172.21.15.26:0/3860118454 shutdown_connections 2023-12-19T16:42:35.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2d0e387640 1 --2- 172.21.15.26:0/3860118454 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d080b3c60 0x7f2d080c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2d0e387640 1 --2- 172.21.15.26:0/3860118454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d080cbb80 0x7f2d080ca3b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2d0e387640 1 --2- 172.21.15.26:0/3860118454 >> v2:172.21.15.26:3300/0 conn(0x7f2d0815ce00 0x7f2d080c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2d0e387640 1 -- 172.21.15.26:0/3860118454 >> 172.21.15.26:0/3860118454 conn(0x7f2d080caff0 msgr2=0x7f2d080cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:35.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f531ffff640 1 --2- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532015a600 0x7f53200d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ff92b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ff40ce040 0x7f2ff40d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49174/0 (socket says 172.21.15.26:49174) 2023-12-19T16:42:35.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ff92b3640 1 -- 172.21.15.26:0/1868676496 learned_addr learned my addr 172.21.15.26:0/1868676496 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ffa2b5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2ff40d3130 0x7f2ff40d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f261a441640 1 -- 172.21.15.26:0/2345157704 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261415b490 con 0x7f261415cd40 2023-12-19T16:42:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f531effd640 1 -- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532015a600 msgr2=0x7f53200d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f531effd640 1 --2- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532015a600 0x7f53200d2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f531effd640 1 -- 172.21.15.26:0/3662249956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53200b8760 msgr2=0x7f5320157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f531effd640 1 --2- 172.21.15.26:0/3662249956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53200b8760 0x7f5320157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f531effd640 1 -- 172.21.15.26:0/3662249956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5318009b30 con 0x7f532015cca0 2023-12-19T16:42:35.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2d0e387640 1 -- 172.21.15.26:0/3860118454 shutdown_connections 2023-12-19T16:42:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ffa2b5640 1 -- 172.21.15.26:0/1868676496 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ff40ce040 msgr2=0x7f2ff40d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ffa2b5640 1 --2- 172.21.15.26:0/1868676496 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ff40ce040 0x7f2ff40d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ffa2b5640 1 -- 172.21.15.26:0/1868676496 >> v1:172.21.15.160:6789/0 conn(0x7f2ff415cd40 legacy=0x7f2ff4160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2ffa2b5640 1 -- 172.21.15.26:0/1868676496 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff4161750 con 0x7f2ff40d3130 2023-12-19T16:42:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7f2610ff9640 1 -- 172.21.15.26:0/2345157704 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f260c007070 con 0x7f261415cd40 2023-12-19T16:42:35.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7fefefe55640 1 -- 172.21.15.26:0/1204560213 shutdown_connections 2023-12-19T16:42:35.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.824+0000 7f2d0e387640 1 -- 172.21.15.26:0/3860118454 wait complete. 2023-12-19T16:42:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7cfe8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7c806bee0 0x7fc7c8151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7cfe8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7c80b1d40 0x7fc7c80ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7cfe8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7c80cb1d0 0x7fc7c8070400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7cfe8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c8151ba0 con 0x7fc7c806bee0 2023-12-19T16:42:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7cfe8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80cd010 con 0x7fc7c80b1d40 2023-12-19T16:42:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7cfe8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80cd2f0 con 0x7fc7c80cb1d0 2023-12-19T16:42:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7ce429640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7c80cb1d0 0x7fc7c8070400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7ce429640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7c80cb1d0 0x7fc7c8070400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49194/0 (socket says 172.21.15.26:49194) 2023-12-19T16:42:35.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7ce429640 1 -- 172.21.15.26:0/4083759126 learned_addr learned my addr 172.21.15.26:0/4083759126 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7f2610ff9640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2600065b60 0x7f2600068020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7cd427640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7c80b1d40 0x7fc7c80ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7f2610ff9640 1 -- 172.21.15.26:0/2345157704 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f260c083140 con 0x7f261415cd40 2023-12-19T16:42:35.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fefefe55640 1 -- 172.21.15.26:0/1204560213 wait complete. 2023-12-19T16:42:35.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7ce429640 1 -- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7c80b1d40 msgr2=0x7fc7c80ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7ce429640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7c80b1d40 0x7fc7c80ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7ce429640 1 -- 172.21.15.26:0/4083759126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7c806bee0 msgr2=0x7fc7c8151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7ce429640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7c806bee0 0x7fc7c8151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7ce429640 1 -- 172.21.15.26:0/4083759126 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c0009a70 con 0x7fc7c80cb1d0 2023-12-19T16:42:35.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7f2613fff640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2600065b60 0x7f2600068020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7cd427640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7c80b1d40 0x7fc7c80ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:35.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7f531effd640 1 --2- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f532015cca0 0x7f53200d2620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f531801d240 tx=0x7f5318019740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7ce429640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7c80cb1d0 0x7fc7c8070400 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00194b0 tx=0x7fc7c0019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7f2613fff640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2600065b60 0x7f2600068020 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2604007fd0 tx=0x7f2604009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fefefe55640 1 Processor -- start 2023-12-19T16:42:35.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7be7fc640 1 -- 172.21.15.26:0/4083759126 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0003db0 con 0x7fc7c80cb1d0 2023-12-19T16:42:35.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7be7fc640 1 -- 172.21.15.26:0/4083759126 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c0004040 con 0x7fc7c80cb1d0 2023-12-19T16:42:35.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7be7fc640 1 -- 172.21.15.26:0/4083759126 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00043f0 con 0x7fc7c80cb1d0 2023-12-19T16:42:35.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/4083759126 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c80cd5d0 con 0x7fc7c80cb1d0 2023-12-19T16:42:35.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fefefe55640 1 -- start start 2023-12-19T16:42:35.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7f01deffd640 1 --2- 172.21.15.26:0/339699280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e015d5c0 0x7f01e00ad6a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009870 tx=0x7f01d001c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddb58b0afca44616 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/4083759126 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c80cdb90 con 0x7fc7c80cb1d0 2023-12-19T16:42:35.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.828+0000 7f2ffa2b5640 1 --2- 172.21.15.26:0/1868676496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ff40d3130 0x7f2ff40d0e30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009d90 tx=0x7f2fec01c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72cb4926b533d845 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/4083759126 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c80cddb0 con 0x7fc7c80cb1d0 2023-12-19T16:42:35.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2d0e387640 1 Processor -- start 2023-12-19T16:42:35.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefefe55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe80ce740 0x7fefe8142990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefefe55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefe8155030 0x7fefe815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefefe55640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fefe815f370 0x7fefe80b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2d0e387640 1 -- start start 2023-12-19T16:42:35.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefefe55640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe815ca00 con 0x7fefe80ce740 2023-12-19T16:42:35.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefefe55640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe815d1d0 con 0x7fefe815f370 2023-12-19T16:42:35.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefefe55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe814b640 con 0x7fefe8155030 2023-12-19T16:42:35.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fc7be7fc640 1 -- 172.21.15.26:0/4083759126 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc7c00b4070 con 0x7fc7c80cb1d0 2023-12-19T16:42:35.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f01ddffb640 1 -- 172.21.15.26:0/339699280 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00ab650 con 0x7f01e015d5c0 2023-12-19T16:42:35.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefed3f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefe8155030 0x7fefe815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefed3f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefe8155030 0x7fefe815c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49200/0 (socket says 172.21.15.26:49200) 2023-12-19T16:42:35.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefed3f0640 1 -- 172.21.15.26:0/2772679778 learned_addr learned my addr 172.21.15.26:0/2772679778 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f01ddffb640 1 -- 172.21.15.26:0/339699280 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d00ab7b0 con 0x7f01e015d5c0 2023-12-19T16:42:35.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f5307fff640 1 -- 172.21.15.26:0/3662249956 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180a5840 con 0x7f532015cca0 2023-12-19T16:42:35.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f5307fff640 1 -- 172.21.15.26:0/3662249956 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53180b88a0 con 0x7f532015cca0 2023-12-19T16:42:35.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f5307fff640 1 -- 172.21.15.26:0/3662249956 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53180b8c50 con 0x7f532015cca0 2023-12-19T16:42:35.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f01ddffb640 1 -- 172.21.15.26:0/339699280 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00ab650 con 0x7f01e015d5c0 2023-12-19T16:42:35.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2ff8ab2640 1 -- 172.21.15.26:0/1868676496 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0a57c0 con 0x7f2ff40d3130 2023-12-19T16:42:35.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2ff8ab2640 1 -- 172.21.15.26:0/1868676496 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fec0a5920 con 0x7f2ff40d3130 2023-12-19T16:42:35.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2ff8ab2640 1 -- 172.21.15.26:0/1868676496 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0a5cb0 con 0x7f2ff40d3130 2023-12-19T16:42:35.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefee3f2640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fefe815f370 0x7fefe80b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f5324d43640 1 -- 172.21.15.26:0/3662249956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532015b050 con 0x7f532015cca0 2023-12-19T16:42:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f5324d43640 1 -- 172.21.15.26:0/3662249956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53200d2260 con 0x7f532015cca0 2023-12-19T16:42:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefedbf1640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe80ce740 0x7fefe8142990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2ffbd18640 1 -- 172.21.15.26:0/1868676496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ff40d3130 msgr2=0x7f2ff40d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2ffbd18640 1 --2- 172.21.15.26:0/1868676496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ff40d3130 0x7f2ff40d0e30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009d90 tx=0x7f2fec01c7c0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefedbf1640 1 -- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefe8155030 msgr2=0x7fefe815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefedbf1640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefe8155030 0x7fefe815c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2ffbd18640 1 -- 172.21.15.26:0/1868676496 shutdown_connections 2023-12-19T16:42:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2ffbd18640 1 --2- 172.21.15.26:0/1868676496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ff40d3130 0x7f2ff40d0e30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2ffbd18640 1 --2- 172.21.15.26:0/1868676496 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ff40ce040 0x7f2ff40d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2ffbd18640 1 -- 172.21.15.26:0/1868676496 >> 172.21.15.26:0/1868676496 conn(0x7f2ff406a120 msgr2=0x7f2ff406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fc7be7fc640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc7ac065ab0 0x7fc7ac067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fc7be7fc640 1 -- 172.21.15.26:0/4083759126 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7c011dc90 con 0x7fc7c80cb1d0 2023-12-19T16:42:35.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefedbf1640 1 -- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fefe815f370 msgr2=0x7fefe80b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefedbf1640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fefe815f370 0x7fefe80b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefedbf1640 1 -- 172.21.15.26:0/2772679778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd8009420 con 0x7fefe80ce740 2023-12-19T16:42:35.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefee3f2640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fefe815f370 0x7fefe80b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:35.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2610ff9640 1 -- 172.21.15.26:0/2345157704 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f260c050b90 con 0x7f261415cd40 2023-12-19T16:42:35.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2d0e387640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d080b3c60 0x7f2d08151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2d0e387640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d080cbb80 0x7f2d08142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2d0e387640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d0815ce00 0x7f2d080b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2d0e387640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08151eb0 con 0x7f2d080cbb80 2023-12-19T16:42:35.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2d0e387640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08152190 con 0x7f2d080b3c60 2023-12-19T16:42:35.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2d0e387640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08152470 con 0x7f2d0815ce00 2023-12-19T16:42:35.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefedbf1640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe80ce740 0x7fefe8142990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fefe000cbc0 tx=0x7fefe000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2ffbd18640 1 -- 172.21.15.26:0/1868676496 shutdown_connections 2023-12-19T16:42:35.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2ffbd18640 1 -- 172.21.15.26:0/1868676496 wait complete. 2023-12-19T16:42:35.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7f2d0d385640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d080b3c60 0x7f2d08151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefde7fc640 1 -- 172.21.15.26:0/2772679778 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe000aca0 con 0x7fefe80ce740 2023-12-19T16:42:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefde7fc640 1 -- 172.21.15.26:0/2772679778 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe000ae00 con 0x7fefe80ce740 2023-12-19T16:42:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.832+0000 7fefde7fc640 1 -- 172.21.15.26:0/2772679778 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe0012600 con 0x7fefe80ce740 2023-12-19T16:42:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7fefefe55640 1 -- 172.21.15.26:0/2772679778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe815cc10 con 0x7fefe80ce740 2023-12-19T16:42:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7fefefe55640 1 -- 172.21.15.26:0/2772679778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe814b2d0 con 0x7fefe80ce740 2023-12-19T16:42:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2d0d385640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d080b3c60 0x7f2d08151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:34218/0 (socket says 172.21.15.26:34218) 2023-12-19T16:42:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2d0db86640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d0815ce00 0x7f2d080b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2d0db86640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d0815ce00 0x7f2d080b2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49210/0 (socket says 172.21.15.26:49210) 2023-12-19T16:42:35.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2d0db86640 1 -- 172.21.15.26:0/3929476441 learned_addr learned my addr 172.21.15.26:0/3929476441 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f52fdffb640 1 -- 172.21.15.26:0/3662249956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5320150310 con 0x7f532015cca0 2023-12-19T16:42:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2d0db86640 1 -- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d080b3c60 msgr2=0x7f2d08151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2d0db86640 1 --2- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d080b3c60 0x7f2d08151890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2d0db86640 1 -- 172.21.15.26:0/3929476441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d080cbb80 msgr2=0x7f2d08142750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2d0db86640 1 --2- 172.21.15.26:0/3929476441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d080cbb80 0x7f2d08142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2d0db86640 1 -- 172.21.15.26:0/3929476441 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf8009420 con 0x7f2d0815ce00 2023-12-19T16:42:35.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2d0d385640 1 --2- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d080b3c60 0x7f2d08151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:35.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f01e528c640 1 -- 172.21.15.26:0/339699280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e015d5c0 msgr2=0x7f01e00ad6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f01e528c640 1 --2- 172.21.15.26:0/339699280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e015d5c0 0x7f01e00ad6a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009870 tx=0x7f01d001c540 comp rx=0 tx=0).stop 2023-12-19T16:42:35.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2d0db86640 1 --2- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d0815ce00 0x7f2d080b2410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d00007db0 tx=0x7f2d0000d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7fc7cdc28640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc7ac065ab0 0x7fc7ac067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f01e528c640 1 -- 172.21.15.26:0/339699280 shutdown_connections 2023-12-19T16:42:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f01e528c640 1 --2- 172.21.15.26:0/339699280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e015d5c0 0x7f01e00ad6a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f01e528c640 1 -- 172.21.15.26:0/339699280 >> 172.21.15.26:0/339699280 conn(0x7f01e00b03f0 msgr2=0x7f01e00b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f01e528c640 1 -- 172.21.15.26:0/339699280 shutdown_connections 2023-12-19T16:42:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2ffbd18640 1 Processor -- start 2023-12-19T16:42:35.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f5307fff640 1 -- 172.21.15.26:0/3662249956 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f53180c3460 con 0x7f532015cca0 2023-12-19T16:42:35.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f01e528c640 1 -- 172.21.15.26:0/339699280 wait complete. 2023-12-19T16:42:35.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7fefefe55640 1 -- 172.21.15.26:0/2772679778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe814b130 con 0x7fefe80ce740 2023-12-19T16:42:35.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2cfdffb640 1 -- 172.21.15.26:0/3929476441 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d000039a0 con 0x7f2d0815ce00 2023-12-19T16:42:35.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2cfdffb640 1 -- 172.21.15.26:0/3929476441 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0000f040 con 0x7f2d0815ce00 2023-12-19T16:42:35.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2cfdffb640 1 -- 172.21.15.26:0/3929476441 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00003d60 con 0x7f2d0815ce00 2023-12-19T16:42:35.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7fc7be7fc640 1 -- 172.21.15.26:0/4083759126 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc7c00eb630 con 0x7fc7c80cb1d0 2023-12-19T16:42:35.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7fc7cdc28640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc7ac065ab0 0x7fc7ac067f70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8156370 tx=0x7fc7c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2d0e387640 1 -- 172.21.15.26:0/3929476441 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d040094d0 con 0x7f2d0815ce00 2023-12-19T16:42:35.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.836+0000 7f2d0e387640 1 -- 172.21.15.26:0/3929476441 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d08152a30 con 0x7f2d0815ce00 2023-12-19T16:42:35.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f2d0e387640 1 -- 172.21.15.26:0/3929476441 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d081525b0 con 0x7f2d0815ce00 2023-12-19T16:42:35.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f5307fff640 1 --2- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f530c065b30 0x7f530c067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f5307fff640 1 -- 172.21.15.26:0/3662249956 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f531811e9c0 con 0x7f532015cca0 2023-12-19T16:42:35.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f531f7fe640 1 --2- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f530c065b30 0x7f530c067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f531f7fe640 1 --2- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f530c065b30 0x7f530c067ff0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f531400b850 tx=0x7f5314008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f01e528c640 1 Processor -- start 2023-12-19T16:42:35.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7fefde7fc640 1 -- 172.21.15.26:0/2772679778 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fefe0007070 con 0x7fefe80ce740 2023-12-19T16:42:35.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f01e528c640 1 -- start start 2023-12-19T16:42:35.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7fefde7fc640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fefcc065ab0 0x7fefcc067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7fefed3f0640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fefcc065ab0 0x7fefcc067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f2cfdffb640 1 -- 172.21.15.26:0/3929476441 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2d00004030 con 0x7f2d0815ce00 2023-12-19T16:42:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f5307fff640 1 -- 172.21.15.26:0/3662249956 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5320150310 con 0x7f532015cca0 2023-12-19T16:42:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f01e528c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00b0f10 0x7f01e0153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f01e528c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e015c630 0x7f01e01fcf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f01e528c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01e015d5c0 0x7f01e01fd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f01e528c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e0150e40 con 0x7f01e00b0f10 2023-12-19T16:42:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f01e528c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e01546d0 con 0x7f01e015c630 2023-12-19T16:42:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f01e528c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e01549b0 con 0x7f01e015d5c0 2023-12-19T16:42:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01df7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01e015d5c0 0x7f01e01fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01df7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01e015d5c0 0x7f01e01fd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49222/0 (socket says 172.21.15.26:49222) 2023-12-19T16:42:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f2ffbd18640 1 -- start start 2023-12-19T16:42:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f2ffbd18640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ff40ce040 0x7f2ff414dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f2ffbd18640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ff40d3130 0x7f2ff415d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f2ff9ab4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ff40ce040 0x7f2ff414dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f2ff9ab4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ff40ce040 0x7f2ff414dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49220/0 (socket says 172.21.15.26:49220) 2023-12-19T16:42:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.840+0000 7f2ff9ab4640 1 -- 172.21.15.26:0/1179244411 learned_addr learned my addr 172.21.15.26:0/1179244411 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ff92b3640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ff40d3130 0x7f2ff415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ffbd18640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff415cd40 0x7f2ff415ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ffbd18640 1 -- 172.21.15.26:0/1179244411 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff4151b10 con 0x7f2ff415cd40 2023-12-19T16:42:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ffbd18640 1 -- 172.21.15.26:0/1179244411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff4151df0 con 0x7f2ff40d3130 2023-12-19T16:42:35.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ffbd18640 1 -- 172.21.15.26:0/1179244411 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff41520d0 con 0x7f2ff40ce040 2023-12-19T16:42:35.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7fefed3f0640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fefcc065ab0 0x7fefcc067f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fefd801c490 tx=0x7fefd80b6040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2cfdffb640 1 --2- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2cec065b30 0x7f2cec067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2cfdffb640 1 -- 172.21.15.26:0/3929476441 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d00083400 con 0x7f2d0815ce00 2023-12-19T16:42:35.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ff92b3640 1 -- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ff40ce040 msgr2=0x7f2ff414dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ff92b3640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ff40ce040 0x7f2ff414dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ff92b3640 1 -- 172.21.15.26:0/1179244411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff415cd40 msgr2=0x7f2ff415ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ffa2b5640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff415cd40 0x7f2ff415ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ff92b3640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff415cd40 0x7f2ff415ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ff92b3640 1 -- 172.21.15.26:0/1179244411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec009a70 con 0x7f2ff40d3130 2023-12-19T16:42:35.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ffa2b5640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff415cd40 0x7f2ff415ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:35.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01deffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00b0f10 0x7f01e0153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ff92b3640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ff40d3130 0x7f2ff415d880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400e870 tx=0x7f2fe400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01deffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00b0f10 0x7f01e0153e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49604/0 (socket says 172.21.15.26:49604) 2023-12-19T16:42:35.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2d0d385640 1 --2- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2cec065b30 0x7f2cec067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01deffd640 1 -- 172.21.15.26:0/720150696 learned_addr learned my addr 172.21.15.26:0/720150696 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:35.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2fea7fc640 1 -- 172.21.15.26:0/1179244411 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400c940 con 0x7f2ff40d3130 2023-12-19T16:42:35.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01de7fc640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e015c630 0x7f01e01fcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2fea7fc640 1 -- 172.21.15.26:0/1179244411 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe400caa0 con 0x7f2ff40d3130 2023-12-19T16:42:35.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01deffd640 1 -- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01e015d5c0 msgr2=0x7f01e01fd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2ffbd18640 1 -- 172.21.15.26:0/1179244411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff41523b0 con 0x7f2ff40d3130 2023-12-19T16:42:35.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01deffd640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01e015d5c0 0x7f01e01fd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01deffd640 1 -- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e015c630 msgr2=0x7f01e01fcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:35.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01deffd640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e015c630 0x7f01e01fcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:35.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01deffd640 1 -- 172.21.15.26:0/720150696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d00094d0 con 0x7f01e00b0f10 2023-12-19T16:42:35.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01de7fc640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e015c630 0x7f01e01fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:35.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01deffd640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00b0f10 0x7f01e0153e80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f01d001abf0 tx=0x7f01d00192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f01df7fe640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01e015d5c0 0x7f01e01fd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:35.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7fefde7fc640 1 -- 172.21.15.26:0/2772679778 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefe0054b10 con 0x7fefe80ce740 2023-12-19T16:42:35.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.844+0000 7f2fea7fc640 1 -- 172.21.15.26:0/1179244411 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400c940 con 0x7f2ff40d3130 2023-12-19T16:42:35.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.848+0000 7f2d0d385640 1 --2- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2cec065b30 0x7f2cec067ff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d080cf190 tx=0x7f2cf8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.848+0000 7f2ffbd18640 1 -- 172.21.15.26:0/1179244411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff40c9490 con 0x7f2ff40d3130 2023-12-19T16:42:35.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.848+0000 7f2cfdffb640 1 -- 172.21.15.26:0/3929476441 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2d081525b0 con 0x7f2d0815ce00 2023-12-19T16:42:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.848+0000 7f2fea7fc640 1 -- 172.21.15.26:0/1179244411 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2fe400cc10 con 0x7f2ff40d3130 2023-12-19T16:42:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.848+0000 7f2fea7fc640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2fd8065ac0 0x7f2fd8067f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.848+0000 7f2fea7fc640 1 -- 172.21.15.26:0/1179244411 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fe4053e30 con 0x7f2ff40d3130 2023-12-19T16:42:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.848+0000 7f2ff9ab4640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2fd8065ac0 0x7f2fd8067f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.852+0000 7f2fd7fff640 1 -- 172.21.15.26:0/1179244411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff4074270 con 0x7f2ff40d3130 2023-12-19T16:42:35.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.852+0000 7f2ff9ab4640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2fd8065ac0 0x7f2fd8067f80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0000fc0 tx=0x7f2ff0007f20 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.852+0000 7f01c77fe640 1 -- 172.21.15.26:0/720150696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00abdf0 con 0x7f01e00b0f10 2023-12-19T16:42:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.852+0000 7f01c77fe640 1 -- 172.21.15.26:0/720150696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d0003620 con 0x7f01e00b0f10 2023-12-19T16:42:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.852+0000 7f01c77fe640 1 -- 172.21.15.26:0/720150696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00039d0 con 0x7f01e00b0f10 2023-12-19T16:42:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.852+0000 7f01e528c640 1 -- 172.21.15.26:0/720150696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e01507f0 con 0x7f01e00b0f10 2023-12-19T16:42:35.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.852+0000 7fefde7fc640 1 -- 172.21.15.26:0/2772679778 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fefe0050e50 con 0x7fefe80ce740 2023-12-19T16:42:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.852+0000 7f01e528c640 1 -- 172.21.15.26:0/720150696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e01543c0 con 0x7f01e00b0f10 2023-12-19T16:42:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.856+0000 7f01e528c640 1 -- 172.21.15.26:0/720150696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e0074270 con 0x7f01e00b0f10 2023-12-19T16:42:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.856+0000 7f01c77fe640 1 -- 172.21.15.26:0/720150696 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f01d0003b30 con 0x7f01e00b0f10 2023-12-19T16:42:35.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.856+0000 7f01c77fe640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01cc065a90 0x7f01cc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.856+0000 7f01c77fe640 1 -- 172.21.15.26:0/720150696 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01d011daa0 con 0x7f01e00b0f10 2023-12-19T16:42:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.856+0000 7f01de7fc640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01cc065a90 0x7f01cc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.856+0000 7f2fea7fc640 1 -- 172.21.15.26:0/1179244411 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2fe4050900 con 0x7f2ff40d3130 2023-12-19T16:42:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.860+0000 7f01de7fc640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01cc065a90 0x7f01cc067f50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01e0157c70 tx=0x7f01d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:35.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:35.864+0000 7f01c77fe640 1 -- 172.21.15.26:0/720150696 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f01d00eb3c0 con 0x7f01e00b0f10 2023-12-19T16:42:36.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.108+0000 7f1a1affd640 1 -- 172.21.15.26:0/2944544657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1a34154be0 con 0x7f1a34071d60 2023-12-19T16:42:36.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.112+0000 7f1a317fa640 1 -- 172.21.15.26:0/2944544657 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1a2801e080 con 0x7f1a34071d60 2023-12-19T16:42:36.115 INFO:teuthology.orchestra.run.smithi026.stdout:60129542150 2023-12-19T16:42:36.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.112+0000 7f1a1affd640 1 -- 172.21.15.26:0/2944544657 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1a20065ac0 msgr2=0x7f1a20067f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.112+0000 7f1a1affd640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1a20065ac0 0x7f1a20067f80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a2401c770 tx=0x7f1a240b3040 comp rx=0 tx=0).stop 2023-12-19T16:42:36.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.112+0000 7f1a1affd640 1 -- 172.21.15.26:0/2944544657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a34071d60 msgr2=0x7f1a340cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.112+0000 7f1a1affd640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a34071d60 0x7f1a340cbdd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800c7f0 tx=0x7f1a2800ccb0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.116+0000 7f1a1affd640 1 -- 172.21.15.26:0/2944544657 shutdown_connections 2023-12-19T16:42:36.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.116+0000 7f1a1affd640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a340ce5b0 0x7f1a34148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.116+0000 7f1a1affd640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1a20065ac0 0x7f1a20067f80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.116+0000 7f1a1affd640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a340cd890 0x7f1a340ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.116+0000 7f1a1affd640 1 --2- 172.21.15.26:0/2944544657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a34071d60 0x7f1a340cbdd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.116+0000 7f1a1affd640 1 -- 172.21.15.26:0/2944544657 >> 172.21.15.26:0/2944544657 conn(0x7f1a3406b3d0 msgr2=0x7f1a341561d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:36.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.116+0000 7f1a1affd640 1 -- 172.21.15.26:0/2944544657 shutdown_connections 2023-12-19T16:42:36.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.116+0000 7f1a1affd640 1 -- 172.21.15.26:0/2944544657 wait complete. 2023-12-19T16:42:36.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.0 2023-12-19T16:42:36.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.156+0000 7f261a441640 1 -- 172.21.15.26:0/2345157704 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2614074270 con 0x7f261415cd40 2023-12-19T16:42:36.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.156+0000 7f2610ff9640 1 -- 172.21.15.26:0/2345157704 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f260c0541b0 con 0x7f261415cd40 2023-12-19T16:42:36.162 INFO:teuthology.orchestra.run.smithi026.stdout:64424509445 2023-12-19T16:42:36.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.160+0000 7f261a441640 1 -- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2600065b60 msgr2=0x7f2600068020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.160+0000 7f261a441640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2600065b60 0x7f2600068020 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2604007fd0 tx=0x7f2604009040 comp rx=0 tx=0).stop 2023-12-19T16:42:36.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.160+0000 7f261a441640 1 -- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f261415cd40 msgr2=0x7f261414bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.160+0000 7f261a441640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f261415cd40 0x7f261414bc80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f260c00cb70 tx=0x7f260c00a430 comp rx=0 tx=0).stop 2023-12-19T16:42:36.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.160+0000 7f261a441640 1 -- 172.21.15.26:0/2345157704 shutdown_connections 2023-12-19T16:42:36.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.160+0000 7f261a441640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f261415cd40 0x7f261414bc80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.160+0000 7f261a441640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2600065b60 0x7f2600068020 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2604007fd0 tx=0x7f2604009040 comp rx=0 tx=0).stop 2023-12-19T16:42:36.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.160+0000 7f261a441640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26140d3130 0x7f261415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.160+0000 7f261a441640 1 --2- 172.21.15.26:0/2345157704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26140ce040 0x7f2614159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.160+0000 7f261a441640 1 -- 172.21.15.26:0/2345157704 >> 172.21.15.26:0/2345157704 conn(0x7f261406a120 msgr2=0x7f2614160cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.160+0000 7f261a441640 1 -- 172.21.15.26:0/2345157704 shutdown_connections 2023-12-19T16:42:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.160+0000 7f261a441640 1 -- 172.21.15.26:0/2345157704 wait complete. 2023-12-19T16:42:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.168+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/4083759126 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc7c8074270 con 0x7fc7c80cb1d0 2023-12-19T16:42:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.172+0000 7fc7be7fc640 1 -- 172.21.15.26:0/4083759126 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc7c00f3e60 con 0x7fc7c80cb1d0 2023-12-19T16:42:36.177 INFO:teuthology.orchestra.run.smithi026.stdout:64424509445 2023-12-19T16:42:36.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.172+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc7ac065ab0 msgr2=0x7fc7ac067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.172+0000 7fc7cfe8c640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc7ac065ab0 0x7fc7ac067f70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8156370 tx=0x7fc7c4008040 comp rx=0 tx=0).stop 2023-12-19T16:42:36.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.172+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7c80cb1d0 msgr2=0x7fc7c8070400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.172+0000 7fc7cfe8c640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7c80cb1d0 0x7fc7c8070400 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00194b0 tx=0x7fc7c0019640 comp rx=0 tx=0).stop 2023-12-19T16:42:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.176+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/4083759126 shutdown_connections 2023-12-19T16:42:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.176+0000 7fc7cfe8c640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc7ac065ab0 0x7fc7ac067f70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.176+0000 7fc7cfe8c640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7c80cb1d0 0x7fc7c8070400 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.176+0000 7fc7cfe8c640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7c80b1d40 0x7fc7c80ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.176+0000 7fc7cfe8c640 1 --2- 172.21.15.26:0/4083759126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7c806bee0 0x7fc7c8151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.176+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/4083759126 >> 172.21.15.26:0/4083759126 conn(0x7fc7c80658e0 msgr2=0x7fc7c8163c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.176+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/4083759126 shutdown_connections 2023-12-19T16:42:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.176+0000 7fc7cfe8c640 1 -- 172.21.15.26:0/4083759126 wait complete. 2023-12-19T16:42:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.176+0000 7f52fdffb640 1 -- 172.21.15.26:0/3662249956 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5320150310 con 0x7f532015cca0 2023-12-19T16:42:36.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.176+0000 7f5307fff640 1 -- 172.21.15.26:0/3662249956 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5320150310 con 0x7f532015cca0 2023-12-19T16:42:36.181 INFO:teuthology.orchestra.run.smithi026.stdout:64424509445 2023-12-19T16:42:36.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7fefefe55640 1 -- 172.21.15.26:0/2772679778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fefe814b130 con 0x7fefe80ce740 2023-12-19T16:42:36.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7fefde7fc640 1 -- 172.21.15.26:0/2772679778 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fefe814b130 con 0x7fefe80ce740 2023-12-19T16:42:36.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f52fdffb640 1 -- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f530c065b30 msgr2=0x7f530c067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f52fdffb640 1 --2- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f530c065b30 0x7f530c067ff0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f531400b850 tx=0x7f5314008040 comp rx=0 tx=0).stop 2023-12-19T16:42:36.185 INFO:teuthology.orchestra.run.smithi026.stdout:64424509444 2023-12-19T16:42:36.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f52fdffb640 1 -- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f532015cca0 msgr2=0x7f53200d2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f52fdffb640 1 --2- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f532015cca0 0x7f53200d2620 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f531801d240 tx=0x7f5318019740 comp rx=0 tx=0).stop 2023-12-19T16:42:36.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f2d0e387640 1 -- 172.21.15.26:0/3929476441 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2d081525b0 con 0x7f2d0815ce00 2023-12-19T16:42:36.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f52fdffb640 1 -- 172.21.15.26:0/3662249956 shutdown_connections 2023-12-19T16:42:36.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f52fdffb640 1 --2- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f530c065b30 0x7f530c067ff0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f52fdffb640 1 --2- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532015a600 0x7f53200d2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f52fdffb640 1 --2- 172.21.15.26:0/3662249956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f532015cca0 0x7f53200d2620 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f52fdffb640 1 --2- 172.21.15.26:0/3662249956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53200b8760 0x7f5320157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f52fdffb640 1 -- 172.21.15.26:0/3662249956 >> 172.21.15.26:0/3662249956 conn(0x7f532006a120 msgr2=0x7f532014b570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:36.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f2cfdffb640 1 -- 172.21.15.26:0/3929476441 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d081525b0 con 0x7f2d0815ce00 2023-12-19T16:42:36.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f52fdffb640 1 -- 172.21.15.26:0/3662249956 shutdown_connections 2023-12-19T16:42:36.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.180+0000 7f52fdffb640 1 -- 172.21.15.26:0/3662249956 wait complete. 2023-12-19T16:42:36.188 INFO:teuthology.orchestra.run.smithi026.stdout:64424509445 2023-12-19T16:42:36.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7fefefe55640 1 -- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fefcc065ab0 msgr2=0x7fefcc067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7fefefe55640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fefcc065ab0 0x7fefcc067f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fefd801c490 tx=0x7fefd80b6040 comp rx=0 tx=0).stop 2023-12-19T16:42:36.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7fefefe55640 1 -- 172.21.15.26:0/2772679778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe80ce740 msgr2=0x7fefe8142990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7fefefe55640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe80ce740 0x7fefe8142990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fefe000cbc0 tx=0x7fefe000a060 comp rx=0 tx=0).stop 2023-12-19T16:42:36.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7fefefe55640 1 -- 172.21.15.26:0/2772679778 shutdown_connections 2023-12-19T16:42:36.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7fefefe55640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fefe815f370 0x7fefe80b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7fefefe55640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fefcc065ab0 0x7fefcc067f70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7fefefe55640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefe8155030 0x7fefe815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7fefefe55640 1 --2- 172.21.15.26:0/2772679778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefe80ce740 0x7fefe8142990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7fefefe55640 1 -- 172.21.15.26:0/2772679778 >> 172.21.15.26:0/2772679778 conn(0x7fefe80658e0 msgr2=0x7fefe815d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:36.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7fefefe55640 1 -- 172.21.15.26:0/2772679778 shutdown_connections 2023-12-19T16:42:36.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7fefefe55640 1 -- 172.21.15.26:0/2772679778 wait complete. 2023-12-19T16:42:36.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7f2d0e387640 1 -- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2cec065b30 msgr2=0x7f2cec067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7f2d0e387640 1 --2- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2cec065b30 0x7f2cec067ff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2d080cf190 tx=0x7f2cf8009190 comp rx=0 tx=0).stop 2023-12-19T16:42:36.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7f2d0e387640 1 -- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d0815ce00 msgr2=0x7f2d080b2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7f2d0e387640 1 --2- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d0815ce00 0x7f2d080b2410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d00007db0 tx=0x7f2d0000d4a0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.184+0000 7f2d0e387640 1 -- 172.21.15.26:0/3929476441 shutdown_connections 2023-12-19T16:42:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.188+0000 7f2d0e387640 1 --2- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2cec065b30 0x7f2cec067ff0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.188+0000 7f2d0e387640 1 --2- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d0815ce00 0x7f2d080b2410 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.188+0000 7f2d0e387640 1 --2- 172.21.15.26:0/3929476441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d080cbb80 0x7f2d08142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.188+0000 7f2d0e387640 1 --2- 172.21.15.26:0/3929476441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d080b3c60 0x7f2d08151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.188+0000 7f2d0e387640 1 -- 172.21.15.26:0/3929476441 >> 172.21.15.26:0/3929476441 conn(0x7f2d080caff0 msgr2=0x7f2d08153290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:36.192 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.1 2023-12-19T16:42:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.192+0000 7f2d0e387640 1 -- 172.21.15.26:0/3929476441 shutdown_connections 2023-12-19T16:42:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.192+0000 7f2d0e387640 1 -- 172.21.15.26:0/3929476441 wait complete. 2023-12-19T16:42:36.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.192+0000 7f2fd7fff640 1 -- 172.21.15.26:0/1179244411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2ff4152860 con 0x7f2ff40d3130 2023-12-19T16:42:36.199 INFO:teuthology.orchestra.run.smithi026.stdout:64424509445 2023-12-19T16:42:36.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.196+0000 7f2fea7fc640 1 -- 172.21.15.26:0/1179244411 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2fe4050aa0 con 0x7f2ff40d3130 2023-12-19T16:42:36.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.196+0000 7f01e528c640 1 -- 172.21.15.26:0/720150696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f01e00b1fd0 con 0x7f01e00b0f10 2023-12-19T16:42:36.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.196+0000 7f01c77fe640 1 -- 172.21.15.26:0/720150696 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01d00ba090 con 0x7f01e00b0f10 2023-12-19T16:42:36.202 INFO:teuthology.orchestra.run.smithi026.stdout:64424509445 2023-12-19T16:42:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f2fd7fff640 1 -- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2fd8065ac0 msgr2=0x7f2fd8067f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f2fd7fff640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2fd8065ac0 0x7f2fd8067f80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0000fc0 tx=0x7f2ff0007f20 comp rx=0 tx=0).stop 2023-12-19T16:42:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f2fd7fff640 1 -- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ff40d3130 msgr2=0x7f2ff415d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f2fd7fff640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ff40d3130 0x7f2ff415d880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400e870 tx=0x7f2fe400ed30 comp rx=0 tx=0).stop 2023-12-19T16:42:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f2fd7fff640 1 -- 172.21.15.26:0/1179244411 shutdown_connections 2023-12-19T16:42:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f2fd7fff640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff415cd40 0x7f2ff415ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f2fd7fff640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ff40d3130 0x7f2ff415d880 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f2fd7fff640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2fd8065ac0 0x7f2fd8067f80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f2fd7fff640 1 --2- 172.21.15.26:0/1179244411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ff40ce040 0x7f2ff414dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f2fd7fff640 1 -- 172.21.15.26:0/1179244411 >> 172.21.15.26:0/1179244411 conn(0x7f2ff406a120 msgr2=0x7f2ff406f900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:36.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f2fd7fff640 1 -- 172.21.15.26:0/1179244411 shutdown_connections 2023-12-19T16:42:36.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f2fd7fff640 1 -- 172.21.15.26:0/1179244411 wait complete. 2023-12-19T16:42:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f01e528c640 1 -- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01cc065a90 msgr2=0x7f01cc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f01e528c640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01cc065a90 0x7f01cc067f50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01e0157c70 tx=0x7f01d4009040 comp rx=0 tx=0).stop 2023-12-19T16:42:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f01e528c640 1 -- 172.21.15.26:0/720150696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00b0f10 msgr2=0x7f01e0153e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f01e528c640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00b0f10 0x7f01e0153e80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f01d001abf0 tx=0x7f01d00192e0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f01e528c640 1 -- 172.21.15.26:0/720150696 shutdown_connections 2023-12-19T16:42:36.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f01e528c640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01cc065a90 0x7f01cc067f50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f01e528c640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01e015d5c0 0x7f01e01fd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f01e528c640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01e015c630 0x7f01e01fcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f01e528c640 1 --2- 172.21.15.26:0/720150696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00b0f10 0x7f01e0153e80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:36.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f01e528c640 1 -- 172.21.15.26:0/720150696 >> 172.21.15.26:0/720150696 conn(0x7f01e00b03f0 msgr2=0x7f01e006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:36.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f01e528c640 1 -- 172.21.15.26:0/720150696 shutdown_connections 2023-12-19T16:42:36.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:36.200+0000 7f01e528c640 1 -- 172.21.15.26:0/720150696 wait complete. 2023-12-19T16:42:36.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2023-12-19T16:42:36.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.2 2023-12-19T16:42:36.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.6 2023-12-19T16:42:36.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2023-12-19T16:42:36.234 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.3 2023-12-19T16:42:36.238 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.7 2023-12-19T16:42:37.153 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-19T16:42:37.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-19T16:42:37.212 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-19T16:42:37.220 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-19T16:42:37.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-19T16:42:37.234 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-19T16:42:37.234 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-19T16:42:37.239 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-19T16:42:37.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.344+0000 7f3ff4d2c640 1 Processor -- start 2023-12-19T16:42:37.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.344+0000 7f3ff4d2c640 1 -- start start 2023-12-19T16:42:37.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.348+0000 7f3ff4d2c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3ff0071d60 0x7f3ff006f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.348+0000 7f3ff4d2c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3ff006bef0 0x7f3ff015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.348+0000 7f3ff4d2c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff006faa0 con 0x7f3ff00cd890 2023-12-19T16:42:37.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.348+0000 7f3ff4d2c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff006fd80 con 0x7f3ff006bef0 2023-12-19T16:42:37.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.348+0000 7f3ff4d2c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff0070060 con 0x7f3ff0071d60 2023-12-19T16:42:37.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.348+0000 7f3fee59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3ff00cd890 0x7f3ff0159180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36066/0 (socket says 172.21.15.26:36066) 2023-12-19T16:42:37.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.348+0000 7f3fee59c640 1 -- 172.21.15.26:0/1709704939 learned_addr learned my addr 172.21.15.26:0/1709704939 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:37.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.348+0000 7f3fedd9b640 1 --2- 172.21.15.26:0/1709704939 >> v2:172.21.15.160:3300/0 conn(0x7f3ff0071d60 0x7f3ff006f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.348+0000 7f3feed9d640 1 --2- 172.21.15.26:0/1709704939 >> v2:172.21.15.26:3300/0 conn(0x7f3ff006bef0 0x7f3ff015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.348+0000 7f3fedd9b640 1 -- 172.21.15.26:0/1709704939 >> v2:172.21.15.26:3300/0 conn(0x7f3ff006bef0 msgr2=0x7f3ff015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:37.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.348+0000 7f3fedd9b640 1 --2- 172.21.15.26:0/1709704939 >> v2:172.21.15.26:3300/0 conn(0x7f3ff006bef0 0x7f3ff015ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.348+0000 7f3fedd9b640 1 -- 172.21.15.26:0/1709704939 >> v1:172.21.15.26:6789/0 conn(0x7f3ff00cd890 legacy=0x7f3ff0159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.352+0000 7f3fedd9b640 1 -- 172.21.15.26:0/1709704939 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff0070340 con 0x7f3ff0071d60 2023-12-19T16:42:37.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.356+0000 7f3fedd9b640 1 --2- 172.21.15.26:0/1709704939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff0071d60 0x7f3ff006f0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0071910 tx=0x7f3fe801c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd6076701a809d77 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.356+0000 7f3fed59a640 1 -- 172.21.15.26:0/1709704939 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80a5910 con 0x7f3ff0071d60 2023-12-19T16:42:37.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.356+0000 7f3fed59a640 1 -- 172.21.15.26:0/1709704939 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe80a5a70 con 0x7f3ff0071d60 2023-12-19T16:42:37.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.356+0000 7f3fed59a640 1 -- 172.21.15.26:0/1709704939 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80a5e00 con 0x7f3ff0071d60 2023-12-19T16:42:37.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.356+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/1709704939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff0071d60 msgr2=0x7f3ff006f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.356+0000 7f3ff4d2c640 1 --2- 172.21.15.26:0/1709704939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff0071d60 0x7f3ff006f0c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0071910 tx=0x7f3fe801c910 comp rx=0 tx=0).stop 2023-12-19T16:42:37.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.360+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/1709704939 shutdown_connections 2023-12-19T16:42:37.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.360+0000 7f3ff4d2c640 1 --2- 172.21.15.26:0/1709704939 >> v2:172.21.15.26:3300/0 conn(0x7f3ff006bef0 0x7f3ff015ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.360+0000 7f3ff4d2c640 1 --2- 172.21.15.26:0/1709704939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff0071d60 0x7f3ff006f0c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.360+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/1709704939 >> 172.21.15.26:0/1709704939 conn(0x7f3ff006b3d0 msgr2=0x7f3ff006b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.360+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/1709704939 shutdown_connections 2023-12-19T16:42:37.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.360+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/1709704939 wait complete. 2023-12-19T16:42:37.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.364+0000 7f3ff4d2c640 1 Processor -- start 2023-12-19T16:42:37.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.364+0000 7f3ff4d2c640 1 -- start start 2023-12-19T16:42:37.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.364+0000 7f3ff4d2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ff006bef0 0x7f3ff0150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.364+0000 7f3ff4d2c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff0071d60 0x7f3ff013df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.364+0000 7f3ff4d2c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ff00cd890 0x7f3ff014ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.364+0000 7f3fee59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ff006bef0 0x7f3ff0150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.364+0000 7f3fee59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ff006bef0 0x7f3ff0150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33258/0 (socket says 172.21.15.26:33258) 2023-12-19T16:42:37.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3fedd9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff0071d60 0x7f3ff013df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3fedd9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff0071d60 0x7f3ff013df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:56916/0 (socket says 172.21.15.26:56916) 2023-12-19T16:42:37.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/2200566575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff014eae0 con 0x7f3ff006bef0 2023-12-19T16:42:37.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3fee59c640 1 -- 172.21.15.26:0/2200566575 learned_addr learned my addr 172.21.15.26:0/2200566575 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:37.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/2200566575 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff0150580 con 0x7f3ff0071d60 2023-12-19T16:42:37.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/2200566575 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff0143830 con 0x7f3ff00cd890 2023-12-19T16:42:37.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3feed9d640 1 --2- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ff00cd890 0x7f3ff014ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3feed9d640 1 -- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff0071d60 msgr2=0x7f3ff013df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3feed9d640 1 --2- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff0071d60 0x7f3ff013df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3feed9d640 1 -- 172.21.15.26:0/2200566575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ff006bef0 msgr2=0x7f3ff0150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3feed9d640 1 --2- 172.21.15.26:0/2200566575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ff006bef0 0x7f3ff0150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3feed9d640 1 -- 172.21.15.26:0/2200566575 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe8009a70 con 0x7f3ff00cd890 2023-12-19T16:42:37.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3feed9d640 1 --2- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ff00cd890 0x7f3ff014ef00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c540 tx=0x7f3fe400ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3fdeffd640 1 -- 172.21.15.26:0/2200566575 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4004220 con 0x7f3ff00cd890 2023-12-19T16:42:37.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3fdeffd640 1 -- 172.21.15.26:0/2200566575 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe40043a0 con 0x7f3ff00cd890 2023-12-19T16:42:37.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/2200566575 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff0143550 con 0x7f3ff00cd890 2023-12-19T16:42:37.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3fdeffd640 1 -- 172.21.15.26:0/2200566575 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4004510 con 0x7f3ff00cd890 2023-12-19T16:42:37.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.368+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/2200566575 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff014e700 con 0x7f3ff00cd890 2023-12-19T16:42:37.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.372+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/2200566575 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd4005260 con 0x7f3ff00cd890 2023-12-19T16:42:37.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.376+0000 7f3fdeffd640 1 -- 172.21.15.26:0/2200566575 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3fe4003860 con 0x7f3ff00cd890 2023-12-19T16:42:37.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.376+0000 7f3fdeffd640 1 --2- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3fd0065a60 0x7f3fd0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.376+0000 7f3fdeffd640 1 -- 172.21.15.26:0/2200566575 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fe4082cf0 con 0x7f3ff00cd890 2023-12-19T16:42:37.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.376+0000 7f3fee59c640 1 --2- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3fd0065a60 0x7f3fd0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.376+0000 7f3fee59c640 1 --2- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3fd0065a60 0x7f3fd0067f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0005890 tx=0x7f3fe0016040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.380+0000 7f3fdeffd640 1 -- 172.21.15.26:0/2200566575 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3fe4087070 con 0x7f3ff00cd890 2023-12-19T16:42:37.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.424+0000 7fabb23bc640 1 Processor -- start 2023-12-19T16:42:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.428+0000 7fabb23bc640 1 -- start start 2023-12-19T16:42:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.428+0000 7fabb23bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fabac071d60 0x7fabac06f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.428+0000 7fabb23bc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fabac0ce5b0 0x7fabac0cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.428+0000 7fabb23bc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabac06faa0 con 0x7fabac0cd890 2023-12-19T16:42:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.428+0000 7fababfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fabac0cd890 0x7fabac159180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:60878/0 (socket says 172.21.15.26:60878) 2023-12-19T16:42:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.428+0000 7fababfff640 1 -- 172.21.15.26:0/85016539 learned_addr learned my addr 172.21.15.26:0/85016539 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.428+0000 7fabab7fe640 1 --2- 172.21.15.26:0/85016539 >> v2:172.21.15.26:3300/0 conn(0x7fabac071d60 0x7fabac06f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.428+0000 7fabb23bc640 1 -- 172.21.15.26:0/85016539 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac0cef80 con 0x7fabac071d60 2023-12-19T16:42:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.428+0000 7fabb23bc640 1 -- 172.21.15.26:0/85016539 --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac06fd80 con 0x7fabac0ce5b0 2023-12-19T16:42:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.428+0000 7fabb0959640 1 --2- 172.21.15.26:0/85016539 >> v2:172.21.15.160:3300/0 conn(0x7fabac0ce5b0 0x7fabac0cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.432+0000 7fabaaffd640 1 -- 172.21.15.26:0/85016539 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085389214 0 0) 0x7fabac06faa0 con 0x7fabac0cd890 2023-12-19T16:42:37.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.432+0000 7fabaaffd640 1 -- 172.21.15.26:0/85016539 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab94005740 con 0x7fabac0cd890 2023-12-19T16:42:37.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.432+0000 7fabb0959640 1 -- 172.21.15.26:0/85016539 >> v2:172.21.15.26:3300/0 conn(0x7fabac071d60 msgr2=0x7fabac06f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:37.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.432+0000 7fabb0959640 1 --2- 172.21.15.26:0/85016539 >> v2:172.21.15.26:3300/0 conn(0x7fabac071d60 0x7fabac06f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.432+0000 7fabb0959640 1 -- 172.21.15.26:0/85016539 >> v1:172.21.15.160:6789/0 conn(0x7fabac0cd890 legacy=0x7fabac159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.432+0000 7fabb0959640 1 -- 172.21.15.26:0/85016539 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac070060 con 0x7fabac0ce5b0 2023-12-19T16:42:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.436+0000 7fabb0959640 1 --2- 172.21.15.26:0/85016539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabac0ce5b0 0x7fabac0cea40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fabac15a240 tx=0x7fab9c01c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebea2fe288368319 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.436+0000 7fabaaffd640 1 -- 172.21.15.26:0/85016539 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0ad660 con 0x7fabac0ce5b0 2023-12-19T16:42:37.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.436+0000 7fabaaffd640 1 -- 172.21.15.26:0/85016539 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c0ad7c0 con 0x7fabac0ce5b0 2023-12-19T16:42:37.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.436+0000 7fabaaffd640 1 -- 172.21.15.26:0/85016539 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0adb50 con 0x7fabac0ce5b0 2023-12-19T16:42:37.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.436+0000 7fabb23bc640 1 -- 172.21.15.26:0/85016539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabac0ce5b0 msgr2=0x7fabac0cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.436+0000 7fabb23bc640 1 --2- 172.21.15.26:0/85016539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabac0ce5b0 0x7fabac0cea40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fabac15a240 tx=0x7fab9c01c430 comp rx=0 tx=0).stop 2023-12-19T16:42:37.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.440+0000 7fabb23bc640 1 -- 172.21.15.26:0/85016539 shutdown_connections 2023-12-19T16:42:37.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.440+0000 7fabb23bc640 1 --2- 172.21.15.26:0/85016539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabac0ce5b0 0x7fabac0cea40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.440+0000 7fabb23bc640 1 --2- 172.21.15.26:0/85016539 >> v2:172.21.15.26:3300/0 conn(0x7fabac071d60 0x7fabac06f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.440+0000 7fabb23bc640 1 -- 172.21.15.26:0/85016539 >> 172.21.15.26:0/85016539 conn(0x7fabac06b3d0 msgr2=0x7fabac06b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.440+0000 7fabb23bc640 1 -- 172.21.15.26:0/85016539 shutdown_connections 2023-12-19T16:42:37.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.440+0000 7fabb23bc640 1 -- 172.21.15.26:0/85016539 wait complete. 2023-12-19T16:42:37.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.440+0000 7fabb23bc640 1 Processor -- start 2023-12-19T16:42:37.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.444+0000 7fabb23bc640 1 -- start start 2023-12-19T16:42:37.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.444+0000 7fabb23bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabac071d60 0x7fabac14e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.444+0000 7fabb23bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabac0cd890 0x7fabac0b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.444+0000 7fabb23bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabac0ce5b0 0x7fabac140780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.444+0000 7fababfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabac071d60 0x7fabac14e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fababfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabac071d60 0x7fabac14e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33278/0 (socket says 172.21.15.26:33278) 2023-12-19T16:42:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fabab7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabac0cd890 0x7fabac0b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fababfff640 1 -- 172.21.15.26:0/1786257507 learned_addr learned my addr 172.21.15.26:0/1786257507 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:37.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fabb0959640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabac0ce5b0 0x7fabac140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fabb23bc640 1 -- 172.21.15.26:0/1786257507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0a8440 con 0x7fabac071d60 2023-12-19T16:42:37.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fabb23bc640 1 -- 172.21.15.26:0/1786257507 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac14eb90 con 0x7fabac0ce5b0 2023-12-19T16:42:37.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fabb23bc640 1 -- 172.21.15.26:0/1786257507 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac1577a0 con 0x7fabac0cd890 2023-12-19T16:42:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fabab7fe640 1 -- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabac0ce5b0 msgr2=0x7fabac140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fabab7fe640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabac0ce5b0 0x7fabac140780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fabab7fe640 1 -- 172.21.15.26:0/1786257507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabac071d60 msgr2=0x7fabac14e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fabab7fe640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabac071d60 0x7fabac14e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fabab7fe640 1 -- 172.21.15.26:0/1786257507 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c009420 con 0x7fabac0cd890 2023-12-19T16:42:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fababfff640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabac071d60 0x7fabac14e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fabb0959640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabac0ce5b0 0x7fabac140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.448+0000 7fabab7fe640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabac0cd890 0x7fabac0b06b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faba4007b60 tx=0x7faba400d490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.452+0000 7faba8ff9640 1 -- 172.21.15.26:0/1786257507 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba400f040 con 0x7fabac0cd890 2023-12-19T16:42:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.452+0000 7faba8ff9640 1 -- 172.21.15.26:0/1786257507 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba4013400 con 0x7fabac0cd890 2023-12-19T16:42:37.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.452+0000 7faba8ff9640 1 -- 172.21.15.26:0/1786257507 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba40137b0 con 0x7fabac0cd890 2023-12-19T16:42:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.452+0000 7fabb23bc640 1 -- 172.21.15.26:0/1786257507 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabac0a8650 con 0x7fabac0cd890 2023-12-19T16:42:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.452+0000 7fabb23bc640 1 -- 172.21.15.26:0/1786257507 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac0a8c10 con 0x7fabac0cd890 2023-12-19T16:42:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.452+0000 7faba8ff9640 1 -- 172.21.15.26:0/1786257507 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faba4013910 con 0x7fabac0cd890 2023-12-19T16:42:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.456+0000 7fab927fc640 1 -- 172.21.15.26:0/1786257507 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabac04e850 con 0x7fabac0cd890 2023-12-19T16:42:37.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.456+0000 7faba8ff9640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fab98065a60 0x7fab98067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.456+0000 7faba8ff9640 1 -- 172.21.15.26:0/1786257507 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faba4083c50 con 0x7fabac0cd890 2023-12-19T16:42:37.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.456+0000 7fababfff640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fab98065a60 0x7fab98067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.456+0000 7fababfff640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fab98065a60 0x7fab98067f20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faba0004b90 tx=0x7faba000a290 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.464+0000 7faba8ff9640 1 -- 172.21.15.26:0/1786257507 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faba40516a0 con 0x7fabac0cd890 2023-12-19T16:42:37.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.496+0000 7fa61c4b6640 1 Processor -- start 2023-12-19T16:42:37.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.496+0000 7fa61c4b6640 1 -- start start 2023-12-19T16:42:37.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa61c4b6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa6140b1880 0x7fa6140b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa61c4b6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa614164d50 0x7fa614165180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa61c4b6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6140b2600 con 0x7fa6140cb1f0 2023-12-19T16:42:37.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa61c4b6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa614161700 con 0x7fa614164d50 2023-12-19T16:42:37.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa61c4b6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6141626c0 con 0x7fa6140b1880 2023-12-19T16:42:37.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa619a51640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa6140b1880 0x7fa6140b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa61aa53640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa614164d50 0x7fa614165180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa61aa53640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa614164d50 0x7fa614165180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33294/0 (socket says 172.21.15.26:33294) 2023-12-19T16:42:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa61aa53640 1 -- 172.21.15.26:0/1179669072 learned_addr learned my addr 172.21.15.26:0/1179669072 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa61aa53640 1 -- 172.21.15.26:0/1179669072 >> v2:172.21.15.160:3300/0 conn(0x7fa6140b1880 msgr2=0x7fa6140b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa61aa53640 1 --2- 172.21.15.26:0/1179669072 >> v2:172.21.15.160:3300/0 conn(0x7fa6140b1880 0x7fa6140b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa61aa53640 1 -- 172.21.15.26:0/1179669072 >> v1:172.21.15.160:6789/0 conn(0x7fa6140cb1f0 legacy=0x7fa6140cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa61aa53640 1 -- 172.21.15.26:0/1179669072 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa614161900 con 0x7fa614164d50 2023-12-19T16:42:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.500+0000 7fa619a51640 1 --2- 172.21.15.26:0/1179669072 >> v2:172.21.15.160:3300/0 conn(0x7fa6140b1880 0x7fa6140b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:37.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f1e7417b640 1 Processor -- start 2023-12-19T16:42:37.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f35f7404640 1 Processor -- start 2023-12-19T16:42:37.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f1e7417b640 1 -- start start 2023-12-19T16:42:37.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f35f7404640 1 -- start start 2023-12-19T16:42:37.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f35f7404640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f015cde0 0x7f35f0160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f35f7404640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f35f015d1c0 0x7f35f0161020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f35f7404640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f35f00cf410 0x7f35f00cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f35f7404640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f01619f0 con 0x7f35f015d1c0 2023-12-19T16:42:37.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f35f7404640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f00cfde0 con 0x7f35f00cf410 2023-12-19T16:42:37.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f35f7404640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0161710 con 0x7f35f015cde0 2023-12-19T16:42:37.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f1e7417b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e6c0ad4a0 0x7f1e6c0b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f1e7417b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1e6c0b1b70 0x7f1e6c15c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f1e7417b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e6c0706d0 con 0x7f1e6c0ad0c0 2023-12-19T16:42:37.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f1e7417b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e6c1616e0 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f1e7417b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c0b2000 con 0x7f1e6c0ad4a0 2023-12-19T16:42:37.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7efd371bc640 1 Processor -- start 2023-12-19T16:42:37.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7efd371bc640 1 -- start start 2023-12-19T16:42:37.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f1e71f17640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f1e6c0ad0c0 0x7f1e6c070b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:60892/0 (socket says 172.21.15.26:60892) 2023-12-19T16:42:37.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f1e71f17640 1 -- 172.21.15.26:0/4094303079 learned_addr learned my addr 172.21.15.26:0/4094303079 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:37.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f1e72718640 1 --2- 172.21.15.26:0/4094303079 >> v2:172.21.15.160:3300/0 conn(0x7f1e6c0b1b70 0x7f1e6c15c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f1e71716640 1 --2- 172.21.15.26:0/4094303079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e6c0ad4a0 0x7f1e6c0b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7fa509dfd640 1 Processor -- start 2023-12-19T16:42:37.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.504+0000 7f35f51a0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f015cde0 0x7f35f0160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f35f59a1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f35f00cf410 0x7f35f00cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f35f51a0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f015cde0 0x7f35f0160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:39788/0 (socket says 172.21.15.26:39788) 2023-12-19T16:42:37.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f35f51a0640 1 -- 172.21.15.26:0/641782180 learned_addr learned my addr 172.21.15.26:0/641782180 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:37.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f35f499f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f35f015d1c0 0x7f35f0161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f35f59a1640 1 -- 172.21.15.26:0/641782180 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f015cde0 msgr2=0x7f35f0160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:37.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f35f59a1640 1 --2- 172.21.15.26:0/641782180 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f015cde0 0x7f35f0160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f35f59a1640 1 -- 172.21.15.26:0/641782180 >> v2:172.21.15.26:3300/0 conn(0x7f35f015d1c0 msgr2=0x7f35f0161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:37.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f35f59a1640 1 --2- 172.21.15.26:0/641782180 >> v2:172.21.15.26:3300/0 conn(0x7f35f015d1c0 0x7f35f0161020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f35f59a1640 1 -- 172.21.15.26:0/641782180 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f00d0ee0 con 0x7f35f00cf410 2023-12-19T16:42:37.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f35f499f640 1 --2- 172.21.15.26:0/641782180 >> v2:172.21.15.26:3300/0 conn(0x7f35f015d1c0 0x7f35f0161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:37.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f523b536640 1 Processor -- start 2023-12-19T16:42:37.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f523b536640 1 -- start start 2023-12-19T16:42:37.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f523a534640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f52340b40e0 0x7f52340afff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36094/0 (socket says 172.21.15.26:36094) 2023-12-19T16:42:37.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f523a534640 1 -- 172.21.15.26:0/300134936 learned_addr learned my addr 172.21.15.26:0/300134936 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:37.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f523b536640 1 --2- 172.21.15.26:0/300134936 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52340b0700 0x7f5234164c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7efd371bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd300b1e20 0x7efd300ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7efd371bc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3015d710 con 0x7efd300cd7f0 2023-12-19T16:42:37.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7efd371bc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3015d9f0 con 0x7efd300adbe0 2023-12-19T16:42:37.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7efd371bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3015e390 con 0x7efd300b1e20 2023-12-19T16:42:37.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7efd34f58640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efd300cd7f0 0x7efd300cdbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36082/0 (socket says 172.21.15.26:36082) 2023-12-19T16:42:37.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7efd34f58640 1 -- 172.21.15.26:0/1075244361 learned_addr learned my addr 172.21.15.26:0/1075244361 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7efd2ffff640 1 --2- 172.21.15.26:0/1075244361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd300b1e20 0x7efd300ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7efd2f7fe640 1 -- 172.21.15.26:0/1075244361 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366052289 0 0) 0x7efd3015d9f0 con 0x7efd300adbe0 2023-12-19T16:42:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7efd2f7fe640 1 -- 172.21.15.26:0/1075244361 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd1c003530 con 0x7efd300adbe0 2023-12-19T16:42:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7efd2ffff640 1 -- 172.21.15.26:0/1075244361 >> v1:172.21.15.160:6789/0 conn(0x7efd300adbe0 legacy=0x7efd3015a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7efd2f7fe640 1 -- 172.21.15.26:0/1075244361 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666823224 0 0) 0x7efd3015d710 con 0x7efd300cd7f0 2023-12-19T16:42:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f1e71716640 1 -- 172.21.15.26:0/4094303079 >> v2:172.21.15.160:3300/0 conn(0x7f1e6c0b1b70 msgr2=0x7f1e6c15c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:37.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f1e71716640 1 --2- 172.21.15.26:0/4094303079 >> v2:172.21.15.160:3300/0 conn(0x7f1e6c0b1b70 0x7f1e6c15c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f1e71716640 1 -- 172.21.15.26:0/4094303079 >> v1:172.21.15.160:6789/0 conn(0x7f1e6c0ad0c0 legacy=0x7f1e6c070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f1e71716640 1 -- 172.21.15.26:0/4094303079 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c06fe20 con 0x7f1e6c0ad4a0 2023-12-19T16:42:37.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7f1e72718640 1 --2- 172.21.15.26:0/4094303079 >> v2:172.21.15.160:3300/0 conn(0x7f1e6c0b1b70 0x7f1e6c15c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:37.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7fa509dfd640 1 -- start start 2023-12-19T16:42:37.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7fa509dfd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa5040cbd10 0x7fa5040ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7fa5037fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa50415c180 0x7fa5040cb600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:60902/0 (socket says 172.21.15.26:60902) 2023-12-19T16:42:37.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa509dfd640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5040ce3f0 con 0x7fa50415d5c0 2023-12-19T16:42:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa509dfd640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa50415cae0 con 0x7fa50415c180 2023-12-19T16:42:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa509dfd640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa50415cdc0 con 0x7fa5040cbd10 2023-12-19T16:42:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5037fe640 1 -- 172.21.15.26:0/642088980 learned_addr learned my addr 172.21.15.26:0/642088980 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7efd2ffff640 1 -- 172.21.15.26:0/1075244361 >> v1:172.21.15.26:6789/0 conn(0x7efd300cd7f0 legacy=0x7efd300cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7fa61aa53640 1 --2- 172.21.15.26:0/1179669072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa614164d50 0x7fa614165180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa6140cafa0 tx=0x7fa60c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86d10b4bf168fc0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7fa619250640 1 -- 172.21.15.26:0/1179669072 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0a5860 con 0x7fa614164d50 2023-12-19T16:42:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7fa619250640 1 -- 172.21.15.26:0/1179669072 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c0a59c0 con 0x7fa614164d50 2023-12-19T16:42:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7fa61c4b6640 1 -- 172.21.15.26:0/1179669072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa614164d50 msgr2=0x7fa614165180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7fa619250640 1 -- 172.21.15.26:0/1179669072 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0a5860 con 0x7fa614164d50 2023-12-19T16:42:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.508+0000 7fa61c4b6640 1 --2- 172.21.15.26:0/1179669072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa614164d50 0x7fa614165180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa6140cafa0 tx=0x7fa60c01c9a0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 -- 172.21.15.26:0/642088980 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896863555 0 0) 0x7fa50415cae0 con 0x7fa50415c180 2023-12-19T16:42:37.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 -- 172.21.15.26:0/642088980 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4f0003530 con 0x7fa50415c180 2023-12-19T16:42:37.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7efd2ffff640 1 -- 172.21.15.26:0/1075244361 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3015dcd0 con 0x7efd300b1e20 2023-12-19T16:42:37.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f5239d33640 1 --2- 172.21.15.26:0/300134936 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52340b0700 0x7f5234164c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 -- 172.21.15.26:0/642088980 <== mon.1 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 238500765 0 0) 0x7fa4f0003530 con 0x7fa50415c180 2023-12-19T16:42:37.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 -- 172.21.15.26:0/642088980 >> v2:172.21.15.160:3300/0 conn(0x7fa5040cbd10 msgr2=0x7fa5040ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:37.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa503fff640 1 --2- 172.21.15.26:0/642088980 >> v2:172.21.15.160:3300/0 conn(0x7fa5040cbd10 0x7fa5040ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f523b536640 1 -- 172.21.15.26:0/300134936 --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5234165200 con 0x7f52340b40e0 2023-12-19T16:42:37.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f5239532640 1 -- 172.21.15.26:0/300134936 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074824469 0 0) 0x7f5234165200 con 0x7f52340b40e0 2023-12-19T16:42:37.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 --2- 172.21.15.26:0/642088980 >> v2:172.21.15.160:3300/0 conn(0x7fa5040cbd10 0x7fa5040ad6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 -- 172.21.15.26:0/642088980 >> v1:172.21.15.26:6789/0 conn(0x7fa50415d5c0 legacy=0x7fa5040b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:37.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 -- 172.21.15.26:0/642088980 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa50415d0a0 con 0x7fa50415c180 2023-12-19T16:42:37.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 -- 172.21.15.26:0/642088980 <== mon.1 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fa4fc001a00 con 0x7fa50415c180 2023-12-19T16:42:37.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 -- 172.21.15.26:0/642088980 >> v1:172.21.15.160:6789/0 conn(0x7fa50415c180 legacy=0x7fa5040cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f523b536640 1 -- 172.21.15.26:0/300134936 --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52341654e0 con 0x7f52340ad740 2023-12-19T16:42:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f523b536640 1 -- 172.21.15.26:0/300134936 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52341657c0 con 0x7f52340b0700 2023-12-19T16:42:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa61c4b6640 1 -- 172.21.15.26:0/1179669072 shutdown_connections 2023-12-19T16:42:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa61c4b6640 1 --2- 172.21.15.26:0/1179669072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa614164d50 0x7fa614165180 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa61c4b6640 1 --2- 172.21.15.26:0/1179669072 >> v2:172.21.15.160:3300/0 conn(0x7fa6140b1880 0x7fa6140b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa61c4b6640 1 -- 172.21.15.26:0/1179669072 >> 172.21.15.26:0/1179669072 conn(0x7fa6140b03f0 msgr2=0x7fa6140b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa61c4b6640 1 -- 172.21.15.26:0/1179669072 shutdown_connections 2023-12-19T16:42:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa61c4b6640 1 -- 172.21.15.26:0/1179669072 wait complete. 2023-12-19T16:42:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f35f59a1640 1 --2- 172.21.15.26:0/641782180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f00cf410 0x7f35f00cf8a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f35f008bfe0 tx=0x7f35ec01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71cda5b7214048e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f5239532640 1 -- 172.21.15.26:0/300134936 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5220003530 con 0x7f52340b40e0 2023-12-19T16:42:37.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f5239532640 1 -- 172.21.15.26:0/300134936 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771131760 0 0) 0x7f52341654e0 con 0x7f52340ad740 2023-12-19T16:42:37.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f5239532640 1 -- 172.21.15.26:0/300134936 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5234165200 con 0x7f52340ad740 2023-12-19T16:42:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f1e71716640 1 --2- 172.21.15.26:0/4094303079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e6c0ad4a0 0x7f1e6c0b1630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1e580097c0 tx=0x7f1e5801c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e3e0b695dc8a672 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f1e70f15640 1 -- 172.21.15.26:0/4094303079 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580ab600 con 0x7f1e6c0ad4a0 2023-12-19T16:42:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f1e70f15640 1 -- 172.21.15.26:0/4094303079 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e580a7050 con 0x7f1e6c0ad4a0 2023-12-19T16:42:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f1e70f15640 1 -- 172.21.15.26:0/4094303079 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e580ab980 con 0x7f1e6c0ad4a0 2023-12-19T16:42:37.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 --2- 172.21.15.26:0/642088980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4f0003980 0x7fa4f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 --2- 172.21.15.26:0/642088980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4f00047d0 0x7fa4f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 --2- 172.21.15.26:0/642088980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4f0005390 0x7fa4f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 -- 172.21.15.26:0/642088980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50415cdc0 con 0x7fa4f00047d0 2023-12-19T16:42:37.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 -- 172.21.15.26:0/642088980 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f000c250 con 0x7fa4f0005390 2023-12-19T16:42:37.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5027fc640 1 -- 172.21.15.26:0/642088980 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f000c410 con 0x7fa4f0003980 2023-12-19T16:42:37.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa5037fe640 1 --2- 172.21.15.26:0/642088980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4f0003980 0x7fa4f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa503fff640 1 --2- 172.21.15.26:0/642088980 >> v2:172.21.15.160:3300/0 conn(0x7fa5040cbd10 0x7fa5040ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:37.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa502ffd640 1 --2- 172.21.15.26:0/642088980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4f00047d0 0x7fa4f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa503fff640 1 --2- 172.21.15.26:0/642088980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4f0005390 0x7fa4f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa502ffd640 1 -- 172.21.15.26:0/642088980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4f0003980 msgr2=0x7fa4f0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:37.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa502ffd640 1 --2- 172.21.15.26:0/642088980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4f0003980 0x7fa4f0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa502ffd640 1 -- 172.21.15.26:0/642088980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4f0005390 msgr2=0x7fa4f000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:37.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa502ffd640 1 --2- 172.21.15.26:0/642088980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4f0005390 0x7fa4f000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa502ffd640 1 -- 172.21.15.26:0/642088980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f0003530 con 0x7fa4f00047d0 2023-12-19T16:42:37.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7fa503fff640 1 -- 172.21.15.26:0/642088980 reap_dead start 2023-12-19T16:42:37.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f35e7fff640 1 -- 172.21.15.26:0/641782180 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0ab600 con 0x7f35f00cf410 2023-12-19T16:42:37.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f35e7fff640 1 -- 172.21.15.26:0/641782180 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ec0a7050 con 0x7f35f00cf410 2023-12-19T16:42:37.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f35e7fff640 1 -- 172.21.15.26:0/641782180 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ec0ab980 con 0x7f35f00cf410 2023-12-19T16:42:37.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f1e7417b640 1 -- 172.21.15.26:0/4094303079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e6c0ad4a0 msgr2=0x7f1e6c0b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f1e7417b640 1 --2- 172.21.15.26:0/4094303079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e6c0ad4a0 0x7f1e6c0b1630 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1e580097c0 tx=0x7f1e5801c3f0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f35f7404640 1 -- 172.21.15.26:0/641782180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f00cf410 msgr2=0x7f35f00cf8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f35f7404640 1 --2- 172.21.15.26:0/641782180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f00cf410 0x7f35f00cf8a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f35f008bfe0 tx=0x7f35ec01c410 comp rx=0 tx=0).stop 2023-12-19T16:42:37.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f1e7417b640 1 -- 172.21.15.26:0/4094303079 shutdown_connections 2023-12-19T16:42:37.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f35f7404640 1 -- 172.21.15.26:0/641782180 shutdown_connections 2023-12-19T16:42:37.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f35f7404640 1 --2- 172.21.15.26:0/641782180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f00cf410 0x7f35f00cf8a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f35f7404640 1 --2- 172.21.15.26:0/641782180 >> v2:172.21.15.26:3300/0 conn(0x7f35f015d1c0 0x7f35f0161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f1e7417b640 1 --2- 172.21.15.26:0/4094303079 >> v2:172.21.15.160:3300/0 conn(0x7f1e6c0b1b70 0x7f1e6c15c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f35f7404640 1 --2- 172.21.15.26:0/641782180 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f015cde0 0x7f35f0160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f35f7404640 1 -- 172.21.15.26:0/641782180 >> 172.21.15.26:0/641782180 conn(0x7f35f00658e0 msgr2=0x7f35f0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f1e7417b640 1 --2- 172.21.15.26:0/4094303079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e6c0ad4a0 0x7f1e6c0b1630 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f1e7417b640 1 -- 172.21.15.26:0/4094303079 >> 172.21.15.26:0/4094303079 conn(0x7f1e6c06b3d0 msgr2=0x7f1e6c06b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f5239d33640 1 -- 172.21.15.26:0/300134936 >> v1:172.21.15.160:6789/0 conn(0x7f52340ad740 legacy=0x7f52340cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f5239d33640 1 -- 172.21.15.26:0/300134936 >> v1:172.21.15.26:6789/0 conn(0x7f52340b40e0 legacy=0x7f52340afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.512+0000 7f5239d33640 1 -- 172.21.15.26:0/300134936 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5234165aa0 con 0x7f52340b0700 2023-12-19T16:42:37.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f1e7417b640 1 -- 172.21.15.26:0/4094303079 shutdown_connections 2023-12-19T16:42:37.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7fa61c4b6640 1 Processor -- start 2023-12-19T16:42:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7fa61c4b6640 1 -- start start 2023-12-19T16:42:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f35f7404640 1 -- 172.21.15.26:0/641782180 shutdown_connections 2023-12-19T16:42:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f35f7404640 1 -- 172.21.15.26:0/641782180 wait complete. 2023-12-19T16:42:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7efd2ffff640 1 --2- 172.21.15.26:0/1075244361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd300b1e20 0x7efd300ad6a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efd200097f0 tx=0x7efd2001c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd9d5b4ad56db6d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7efd2f7fe640 1 -- 172.21.15.26:0/1075244361 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd200ab600 con 0x7efd300b1e20 2023-12-19T16:42:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7efd2f7fe640 1 -- 172.21.15.26:0/1075244361 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd200a7050 con 0x7efd300b1e20 2023-12-19T16:42:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7efd2f7fe640 1 -- 172.21.15.26:0/1075244361 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd200ab980 con 0x7efd300b1e20 2023-12-19T16:42:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7efd371bc640 1 -- 172.21.15.26:0/1075244361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd300b1e20 msgr2=0x7efd300ad6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7efd371bc640 1 --2- 172.21.15.26:0/1075244361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd300b1e20 0x7efd300ad6a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efd200097f0 tx=0x7efd2001c430 comp rx=0 tx=0).stop 2023-12-19T16:42:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7efd371bc640 1 -- 172.21.15.26:0/1075244361 shutdown_connections 2023-12-19T16:42:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7efd371bc640 1 --2- 172.21.15.26:0/1075244361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd300b1e20 0x7efd300ad6a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7efd371bc640 1 -- 172.21.15.26:0/1075244361 >> 172.21.15.26:0/1075244361 conn(0x7efd300b03f0 msgr2=0x7efd300b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7efd371bc640 1 -- 172.21.15.26:0/1075244361 shutdown_connections 2023-12-19T16:42:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.516+0000 7f1e7417b640 1 -- 172.21.15.26:0/4094303079 wait complete. 2023-12-19T16:42:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa61c4b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6140b1880 0x7fa614160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7efd371bc640 1 -- 172.21.15.26:0/1075244361 wait complete. 2023-12-19T16:42:37.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa61c4b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6140cb1f0 0x7fa61415f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f5239d33640 1 --2- 172.21.15.26:0/300134936 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52340b0700 0x7f5234164c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f52240097f0 tx=0x7f522401ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79d53aa5d4a7ab33 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa61a252640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6140b1880 0x7fa614160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa61a252640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6140b1880 0x7fa614160b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:56986/0 (socket says 172.21.15.26:56986) 2023-12-19T16:42:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa61a252640 1 -- 172.21.15.26:0/3647380552 learned_addr learned my addr 172.21.15.26:0/3647380552 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa61c4b6640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa614164d50 0x7fa614155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa619a51640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6140cb1f0 0x7fa61415f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f5239532640 1 -- 172.21.15.26:0/300134936 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240a89b0 con 0x7f52340b0700 2023-12-19T16:42:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f5239532640 1 -- 172.21.15.26:0/300134936 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52240a8b10 con 0x7f52340b0700 2023-12-19T16:42:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa61c4b6640 1 -- 172.21.15.26:0/3647380552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa614160350 con 0x7fa6140cb1f0 2023-12-19T16:42:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa61c4b6640 1 -- 172.21.15.26:0/3647380552 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61415cc90 con 0x7fa6140b1880 2023-12-19T16:42:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa61c4b6640 1 -- 172.21.15.26:0/3647380552 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61415c6d0 con 0x7fa614164d50 2023-12-19T16:42:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa61aa53640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa614164d50 0x7fa614155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f5239532640 1 -- 172.21.15.26:0/300134936 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240a89b0 con 0x7f52340b0700 2023-12-19T16:42:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e7417b640 1 Processor -- start 2023-12-19T16:42:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e7417b640 1 -- start start 2023-12-19T16:42:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa619a51640 1 -- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa614164d50 msgr2=0x7fa614155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa502ffd640 1 --2- 172.21.15.26:0/642088980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4f00047d0 0x7fa4f0004c50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec001340 tx=0x7fa4ec0207d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0066af26a986778 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa619a51640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa614164d50 0x7fa614155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa619a51640 1 -- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6140b1880 msgr2=0x7fa614160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa619a51640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6140b1880 0x7fa614160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa619a51640 1 -- 172.21.15.26:0/3647380552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c009b30 con 0x7fa6140cb1f0 2023-12-19T16:42:37.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa61aa53640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa614164d50 0x7fa614155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:37.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7efd371bc640 1 Processor -- start 2023-12-19T16:42:37.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f35f7404640 1 Processor -- start 2023-12-19T16:42:37.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f523b536640 1 -- 172.21.15.26:0/300134936 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52340b0700 msgr2=0x7f5234164c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f523b536640 1 --2- 172.21.15.26:0/300134936 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52340b0700 0x7f5234164c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f52240097f0 tx=0x7f522401ecf0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7efd371bc640 1 -- start start 2023-12-19T16:42:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e7417b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e6c0ad0c0 0x7f1e6c15b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e7417b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e6c0ad4a0 0x7f1e6c14b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e7417b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e6c0b1b70 0x7f1e6c14bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e7417b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c14d500 con 0x7f1e6c0ad4a0 2023-12-19T16:42:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e7417b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c14f570 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e7417b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e6c14ea60 con 0x7f1e6c0ad0c0 2023-12-19T16:42:37.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7fa619a51640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6140cb1f0 0x7fa61415f310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa60400d5f0 tx=0x7fa60400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e71f17640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e6c0ad0c0 0x7f1e6c15b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e72718640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e6c0b1b70 0x7f1e6c14bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e72718640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e6c0b1b70 0x7f1e6c14bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:56998/0 (socket says 172.21.15.26:56998) 2023-12-19T16:42:37.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e72718640 1 -- 172.21.15.26:0/2372212913 learned_addr learned my addr 172.21.15.26:0/2372212913 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f523b536640 1 -- 172.21.15.26:0/300134936 shutdown_connections 2023-12-19T16:42:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f523b536640 1 --2- 172.21.15.26:0/300134936 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52340b0700 0x7f5234164c20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f523b536640 1 -- 172.21.15.26:0/300134936 >> 172.21.15.26:0/300134936 conn(0x7f523406b060 msgr2=0x7f523406b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e72718640 1 -- 172.21.15.26:0/2372212913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e6c0ad0c0 msgr2=0x7f1e6c15b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e72718640 1 --2- 172.21.15.26:0/2372212913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e6c0ad0c0 0x7f1e6c15b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f1e72718640 1 -- 172.21.15.26:0/2372212913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e6c0ad4a0 msgr2=0x7f1e6c14b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f1e72718640 1 --2- 172.21.15.26:0/2372212913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e6c0ad4a0 0x7f1e6c14b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f1e72718640 1 -- 172.21.15.26:0/2372212913 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e58009420 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa5027fc640 1 -- 172.21.15.26:0/642088980 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0052d0 con 0x7fa4f00047d0 2023-12-19T16:42:37.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f523b536640 1 -- 172.21.15.26:0/300134936 shutdown_connections 2023-12-19T16:42:37.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa60affd640 1 -- 172.21.15.26:0/3647380552 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6040125f0 con 0x7fa6140cb1f0 2023-12-19T16:42:37.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f523b536640 1 -- 172.21.15.26:0/300134936 wait complete. 2023-12-19T16:42:37.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.520+0000 7f35f7404640 1 -- start start 2023-12-19T16:42:37.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa60affd640 1 -- 172.21.15.26:0/3647380552 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa604012750 con 0x7fa6140cb1f0 2023-12-19T16:42:37.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa60affd640 1 -- 172.21.15.26:0/3647380552 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6040125f0 con 0x7fa6140cb1f0 2023-12-19T16:42:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f7404640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f00cf410 0x7f35f0149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f7404640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f015cde0 0x7f35f00c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f7404640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f015d1c0 0x7f35f0148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f7404640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00c6730 con 0x7f35f00cf410 2023-12-19T16:42:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f7404640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0149850 con 0x7f35f015d1c0 2023-12-19T16:42:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f7404640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00c6980 con 0x7f35f015cde0 2023-12-19T16:42:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa61c4b6640 1 -- 172.21.15.26:0/3647380552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61415fd90 con 0x7fa6140cb1f0 2023-12-19T16:42:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd371bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd300adbe0 0x7efd300af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd371bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd300b1e20 0x7efd300ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd371bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd300cd7f0 0x7efd3013adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f1e72718640 1 --2- 172.21.15.26:0/2372212913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e6c0b1b70 0x7f1e6c14bc40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400d830 tx=0x7f1e6400dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa61c4b6640 1 -- 172.21.15.26:0/3647380552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61415e620 con 0x7fa6140cb1f0 2023-12-19T16:42:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa61c4b6640 1 -- 172.21.15.26:0/3647380552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61415ce40 con 0x7fa6140cb1f0 2023-12-19T16:42:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f51a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f00cf410 0x7f35f0149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f51a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f00cf410 0x7f35f0149240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33336/0 (socket says 172.21.15.26:33336) 2023-12-19T16:42:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f51a0640 1 -- 172.21.15.26:0/2929809529 learned_addr learned my addr 172.21.15.26:0/2929809529 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd371bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3014c700 con 0x7efd300adbe0 2023-12-19T16:42:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd371bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3014c9e0 con 0x7efd300b1e20 2023-12-19T16:42:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd371bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3014ccc0 con 0x7efd300cd7f0 2023-12-19T16:42:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd35759640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd300cd7f0 0x7efd3013adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd34f58640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd300adbe0 0x7efd300af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd35759640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd300cd7f0 0x7efd3013adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:39850/0 (socket says 172.21.15.26:39850) 2023-12-19T16:42:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa5027fc640 1 -- 172.21.15.26:0/642088980 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec005450 con 0x7fa4f00047d0 2023-12-19T16:42:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa509dfd640 1 -- 172.21.15.26:0/642088980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4f00047d0 msgr2=0x7fa4f0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa509dfd640 1 --2- 172.21.15.26:0/642088980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4f00047d0 0x7fa4f0004c50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec001340 tx=0x7fa4ec0207d0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd35759640 1 -- 172.21.15.26:0/2131445070 learned_addr learned my addr 172.21.15.26:0/2131445070 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:37.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd35759640 1 -- 172.21.15.26:0/2131445070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd300b1e20 msgr2=0x7efd300ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f59a1640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f015d1c0 0x7f35f0148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f499f640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f015cde0 0x7f35f00c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd35759640 1 --2- 172.21.15.26:0/2131445070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd300b1e20 0x7efd300ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd35759640 1 -- 172.21.15.26:0/2131445070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd300adbe0 msgr2=0x7efd300af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd35759640 1 --2- 172.21.15.26:0/2131445070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd300adbe0 0x7efd300af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd35759640 1 -- 172.21.15.26:0/2131445070 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd20009420 con 0x7efd300cd7f0 2023-12-19T16:42:37.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f51a0640 1 -- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f015cde0 msgr2=0x7f35f00c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f51a0640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f015cde0 0x7f35f00c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f51a0640 1 -- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f015d1c0 msgr2=0x7f35f0148040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f51a0640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f015d1c0 0x7f35f0148040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f51a0640 1 -- 172.21.15.26:0/2929809529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ec009420 con 0x7f35f00cf410 2023-12-19T16:42:37.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f499f640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f015cde0 0x7f35f00c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:37.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f59a1640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f015d1c0 0x7f35f0148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:37.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f35f51a0640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f00cf410 0x7f35f0149240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f35e80070b0 tx=0x7f35e800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f523b536640 1 Processor -- start 2023-12-19T16:42:37.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f523b536640 1 -- start start 2023-12-19T16:42:37.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7efd35759640 1 --2- 172.21.15.26:0/2131445070 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd300cd7f0 0x7efd3013adc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efd1800c730 tx=0x7efd1800cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f1e6a7fc640 1 -- 172.21.15.26:0/2372212913 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e64007e10 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7f1e6a7fc640 1 -- 172.21.15.26:0/2372212913 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6400b8d0 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f1e7417b640 1 -- 172.21.15.26:0/2372212913 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e6c14ecd0 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa509dfd640 1 -- 172.21.15.26:0/642088980 shutdown_connections 2023-12-19T16:42:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa509dfd640 1 --2- 172.21.15.26:0/642088980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4f00047d0 0x7fa4f0004c50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa509dfd640 1 -- 172.21.15.26:0/642088980 >> 172.21.15.26:0/642088980 conn(0x7fa5040b03f0 msgr2=0x7fa5040b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa509dfd640 1 -- 172.21.15.26:0/642088980 shutdown_connections 2023-12-19T16:42:37.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f1e6a7fc640 1 -- 172.21.15.26:0/2372212913 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6400bc80 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f1e7417b640 1 -- 172.21.15.26:0/2372212913 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e6c14f290 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.524+0000 7fa509dfd640 1 -- 172.21.15.26:0/642088980 wait complete. 2023-12-19T16:42:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523b536640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52340ad740 0x7f5234071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7fa60affd640 1 -- 172.21.15.26:0/3647380552 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa6040128c0 con 0x7fa6140cb1f0 2023-12-19T16:42:37.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f1e7417b640 1 -- 172.21.15.26:0/2372212913 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e6c04e850 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7fa60affd640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa5f8065ac0 0x7fa5f8067f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7fa60affd640 1 -- 172.21.15.26:0/3647380552 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa604083b40 con 0x7fa6140cb1f0 2023-12-19T16:42:37.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523b536640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52340b0700 0x7f5234159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523b536640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52340b40e0 0x7f52340c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523a534640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52340ad740 0x7f5234071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523a534640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52340ad740 0x7f5234071390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:57028/0 (socket says 172.21.15.26:57028) 2023-12-19T16:42:37.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523a534640 1 -- 172.21.15.26:0/3188226694 learned_addr learned my addr 172.21.15.26:0/3188226694 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:37.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523b536640 1 -- 172.21.15.26:0/3188226694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5234159650 con 0x7f52340b0700 2023-12-19T16:42:37.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523b536640 1 -- 172.21.15.26:0/3188226694 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52340c4e20 con 0x7f52340ad740 2023-12-19T16:42:37.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523b536640 1 -- 172.21.15.26:0/3188226694 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52340c6980 con 0x7f52340b40e0 2023-12-19T16:42:37.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7fa61a252640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa5f8065ac0 0x7fa5f8067f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f5239d33640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52340b0700 0x7f5234159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523ad35640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52340b40e0 0x7f52340c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523a534640 1 -- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52340b40e0 msgr2=0x7f52340c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523a534640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52340b40e0 0x7f52340c48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523a534640 1 -- 172.21.15.26:0/3188226694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52340b0700 msgr2=0x7f5234159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523a534640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52340b0700 0x7f5234159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523a534640 1 -- 172.21.15.26:0/3188226694 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5224009420 con 0x7f52340ad740 2023-12-19T16:42:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f5239d33640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52340b0700 0x7f5234159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7efd2d7fa640 1 -- 172.21.15.26:0/2131445070 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd18003b80 con 0x7efd300cd7f0 2023-12-19T16:42:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7efd2d7fa640 1 -- 172.21.15.26:0/2131445070 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd18003ce0 con 0x7efd300cd7f0 2023-12-19T16:42:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7efd371bc640 1 -- 172.21.15.26:0/2131445070 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3014cfa0 con 0x7efd300cd7f0 2023-12-19T16:42:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7f523a534640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52340ad740 0x7f5234071390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f523000c420 tx=0x7f523000c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.532+0000 7f522affd640 1 -- 172.21.15.26:0/3188226694 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5230005cf0 con 0x7f52340ad740 2023-12-19T16:42:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.532+0000 7f523b536640 1 -- 172.21.15.26:0/3188226694 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52340c7940 con 0x7f52340ad740 2023-12-19T16:42:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.532+0000 7f522affd640 1 -- 172.21.15.26:0/3188226694 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5230005e50 con 0x7f52340ad740 2023-12-19T16:42:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.532+0000 7f522affd640 1 -- 172.21.15.26:0/3188226694 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52300072d0 con 0x7f52340ad740 2023-12-19T16:42:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.532+0000 7f523b536640 1 -- 172.21.15.26:0/3188226694 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52340c66a0 con 0x7f52340ad740 2023-12-19T16:42:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7efd2d7fa640 1 -- 172.21.15.26:0/2131445070 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1800aa90 con 0x7efd300cd7f0 2023-12-19T16:42:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.528+0000 7efd371bc640 1 -- 172.21.15.26:0/2131445070 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd300ae9a0 con 0x7efd300cd7f0 2023-12-19T16:42:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.532+0000 7fa61a252640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa5f8065ac0 0x7fa5f8067f80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa6100016d0 tx=0x7fa610009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.532+0000 7efd371bc640 1 -- 172.21.15.26:0/2131445070 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3014d1e0 con 0x7efd300cd7f0 2023-12-19T16:42:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.532+0000 7f1e6a7fc640 1 -- 172.21.15.26:0/2372212913 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1e6401e460 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.532+0000 7fa509dfd640 1 Processor -- start 2023-12-19T16:42:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.532+0000 7fa509dfd640 1 -- start start 2023-12-19T16:42:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.532+0000 7f1e6a7fc640 1 --2- 172.21.15.26:0/2372212913 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1e50065b60 0x7f1e50068020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.532+0000 7f523b536640 1 -- 172.21.15.26:0/3188226694 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523404e850 con 0x7f52340ad740 2023-12-19T16:42:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.532+0000 7f1e6a7fc640 1 -- 172.21.15.26:0/2372212913 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e640835b0 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7f1e71f17640 1 --2- 172.21.15.26:0/2372212913 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1e50065b60 0x7f1e50068020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7efd2d7fa640 1 -- 172.21.15.26:0/2131445070 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efd1800abf0 con 0x7efd300cd7f0 2023-12-19T16:42:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa509dfd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4f00047d0 0x7fa5040a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7f1e71f17640 1 --2- 172.21.15.26:0/2372212913 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1e50065b60 0x7f1e50068020 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1e60005c10 tx=0x7f1e60009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa509dfd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa504141540 0x7fa5040a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7efd2d7fa640 1 --2- 172.21.15.26:0/2131445070 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd10065ab0 0x7efd10067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7efd2d7fa640 1 -- 172.21.15.26:0/2131445070 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd18082f10 con 0x7efd300cd7f0 2023-12-19T16:42:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa60affd640 1 -- 172.21.15.26:0/3647380552 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa6040514e0 con 0x7fa6140cb1f0 2023-12-19T16:42:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7f35e5ffb640 1 -- 172.21.15.26:0/2929809529 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8009e40 con 0x7f35f00cf410 2023-12-19T16:42:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7f35e5ffb640 1 -- 172.21.15.26:0/2929809529 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e800f040 con 0x7f35f00cf410 2023-12-19T16:42:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7f35e5ffb640 1 -- 172.21.15.26:0/2929809529 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8015620 con 0x7f35f00cf410 2023-12-19T16:42:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa509dfd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5040a9f10 0x7fa5040b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa509dfd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50414ed30 con 0x7fa504141540 2023-12-19T16:42:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa509dfd640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5040aa390 con 0x7fa4f00047d0 2023-12-19T16:42:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa509dfd640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5040b3140 con 0x7fa5040a9f10 2023-12-19T16:42:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa503fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5040a9f10 0x7fa5040b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa5037fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4f00047d0 0x7fa5040a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa5037fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4f00047d0 0x7fa5040a92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:57036/0 (socket says 172.21.15.26:57036) 2023-12-19T16:42:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa5037fe640 1 -- 172.21.15.26:0/690508319 learned_addr learned my addr 172.21.15.26:0/690508319 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7f1e6a7fc640 1 -- 172.21.15.26:0/2372212913 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1e64050f50 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa502ffd640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa504141540 0x7fa5040a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa5037fe640 1 -- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5040a9f10 msgr2=0x7fa5040b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa5037fe640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5040a9f10 0x7fa5040b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa5037fe640 1 -- 172.21.15.26:0/690508319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa504141540 msgr2=0x7fa5040a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7f35f7404640 1 -- 172.21.15.26:0/2929809529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f00c6c60 con 0x7f35f00cf410 2023-12-19T16:42:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7f35f7404640 1 -- 172.21.15.26:0/2929809529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f00c9570 con 0x7f35f00cf410 2023-12-19T16:42:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa5037fe640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa504141540 0x7fa5040a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa5037fe640 1 -- 172.21.15.26:0/690508319 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5040a84e0 con 0x7fa4f00047d0 2023-12-19T16:42:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa502ffd640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa504141540 0x7fa5040a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa5037fe640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4f00047d0 0x7fa5040a92a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0079d0 tx=0x7fa4fc00a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7fa503fff640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5040a9f10 0x7fa5040b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.536+0000 7efd34f58640 1 --2- 172.21.15.26:0/2131445070 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd10065ab0 0x7efd10067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.540+0000 7f35f7404640 1 -- 172.21.15.26:0/2929809529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f00c7180 con 0x7f35f00cf410 2023-12-19T16:42:37.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.540+0000 7f522affd640 1 -- 172.21.15.26:0/3188226694 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5230007430 con 0x7f52340ad740 2023-12-19T16:42:37.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.540+0000 7efd34f58640 1 --2- 172.21.15.26:0/2131445070 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd10065ab0 0x7efd10067f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efd28000f70 tx=0x7efd28007bb0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.540+0000 7efd2d7fa640 1 -- 172.21.15.26:0/2131445070 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efd3014d1e0 con 0x7efd300cd7f0 2023-12-19T16:42:37.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.540+0000 7f522affd640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5218065a60 0x7f5218067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.540+0000 7f35e5ffb640 1 -- 172.21.15.26:0/2929809529 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f35e8015780 con 0x7f35f00cf410 2023-12-19T16:42:37.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.540+0000 7f5239d33640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5218065a60 0x7f5218067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.540+0000 7f522affd640 1 -- 172.21.15.26:0/3188226694 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5230082b80 con 0x7f52340ad740 2023-12-19T16:42:37.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.544+0000 7f522affd640 1 -- 172.21.15.26:0/3188226694 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f52300ad0a0 con 0x7f52340ad740 2023-12-19T16:42:37.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.544+0000 7f5239d33640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5218065a60 0x7f5218067f20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f52240b2000 tx=0x7f5224008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.540+0000 7fa4fbfff640 1 -- 172.21.15.26:0/690508319 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0133f0 con 0x7fa4f00047d0 2023-12-19T16:42:37.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.540+0000 7fa4fbfff640 1 -- 172.21.15.26:0/690508319 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4fc013550 con 0x7fa4f00047d0 2023-12-19T16:42:37.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.540+0000 7fa4fbfff640 1 -- 172.21.15.26:0/690508319 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc0138e0 con 0x7fa4f00047d0 2023-12-19T16:42:37.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.540+0000 7fa509dfd640 1 -- 172.21.15.26:0/690508319 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5040a87c0 con 0x7fa4f00047d0 2023-12-19T16:42:37.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.540+0000 7fa509dfd640 1 -- 172.21.15.26:0/690508319 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5040a8d80 con 0x7fa4f00047d0 2023-12-19T16:42:37.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.544+0000 7fa509dfd640 1 -- 172.21.15.26:0/690508319 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5040a97e0 con 0x7fa4f00047d0 2023-12-19T16:42:37.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.544+0000 7f35e5ffb640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35d4065a60 0x7f35d4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.544+0000 7f35e5ffb640 1 -- 172.21.15.26:0/2929809529 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35e80836e0 con 0x7f35f00cf410 2023-12-19T16:42:37.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.548+0000 7f35f499f640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35d4065a60 0x7f35d4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.548+0000 7f35f499f640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35d4065a60 0x7f35d4067f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f35f0149780 tx=0x7f35e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.548+0000 7fa4fbfff640 1 -- 172.21.15.26:0/690508319 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa4fc013a40 con 0x7fa4f00047d0 2023-12-19T16:42:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.548+0000 7fa4fbfff640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4e4065b60 0x7fa4e4068020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.548+0000 7fa4fbfff640 1 -- 172.21.15.26:0/690508319 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4fc083c90 con 0x7fa4f00047d0 2023-12-19T16:42:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.548+0000 7fa502ffd640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4e4065b60 0x7fa4e4068020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.552+0000 7f35e5ffb640 1 -- 172.21.15.26:0/2929809529 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f35e80511c0 con 0x7f35f00cf410 2023-12-19T16:42:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.552+0000 7fa502ffd640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4e4065b60 0x7fa4e4068020 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec001730 tx=0x7fa4ec0011a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:37.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.556+0000 7fa4fbfff640 1 -- 172.21.15.26:0/690508319 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa4fc0550c0 con 0x7fa4f00047d0 2023-12-19T16:42:37.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.708+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/2200566575 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3fd40057e0 con 0x7f3ff00cd890 2023-12-19T16:42:37.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.708+0000 7f3fdeffd640 1 -- 172.21.15.26:0/2200566575 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3fe401e020 con 0x7f3ff00cd890 2023-12-19T16:42:37.714 INFO:teuthology.orchestra.run.smithi026.stdout:60129542152 2023-12-19T16:42:37.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.712+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3fd0065a60 msgr2=0x7f3fd0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.712+0000 7f3ff4d2c640 1 --2- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3fd0065a60 0x7f3fd0067f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0005890 tx=0x7f3fe0016040 comp rx=0 tx=0).stop 2023-12-19T16:42:37.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.712+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ff00cd890 msgr2=0x7f3ff014ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.712+0000 7f3ff4d2c640 1 --2- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ff00cd890 0x7f3ff014ef00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c540 tx=0x7f3fe400ca00 comp rx=0 tx=0).stop 2023-12-19T16:42:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.716+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/2200566575 shutdown_connections 2023-12-19T16:42:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.716+0000 7f3ff4d2c640 1 --2- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3fd0065a60 0x7f3fd0067f20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.716+0000 7f3ff4d2c640 1 --2- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ff00cd890 0x7f3ff014ef00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.716+0000 7f3ff4d2c640 1 --2- 172.21.15.26:0/2200566575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff0071d60 0x7f3ff013df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.716+0000 7f3ff4d2c640 1 --2- 172.21.15.26:0/2200566575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ff006bef0 0x7f3ff0150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.716+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/2200566575 >> 172.21.15.26:0/2200566575 conn(0x7f3ff006b3d0 msgr2=0x7f3ff006b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.716+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/2200566575 shutdown_connections 2023-12-19T16:42:37.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.716+0000 7f3ff4d2c640 1 -- 172.21.15.26:0/2200566575 wait complete. 2023-12-19T16:42:37.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.0 2023-12-19T16:42:37.752 DEBUG:teuthology.parallel:result is None 2023-12-19T16:42:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.800+0000 7fab927fc640 1 -- 172.21.15.26:0/1786257507 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fabac0ced60 con 0x7fabac0cd890 2023-12-19T16:42:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.800+0000 7faba8ff9640 1 -- 172.21.15.26:0/1786257507 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faba4054cc0 con 0x7fabac0cd890 2023-12-19T16:42:37.805 INFO:teuthology.orchestra.run.smithi026.stdout:64424509447 2023-12-19T16:42:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.804+0000 7fab927fc640 1 -- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fab98065a60 msgr2=0x7fab98067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.804+0000 7fab927fc640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fab98065a60 0x7fab98067f20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faba0004b90 tx=0x7faba000a290 comp rx=0 tx=0).stop 2023-12-19T16:42:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.804+0000 7fab927fc640 1 -- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabac0cd890 msgr2=0x7fabac0b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.804+0000 7fab927fc640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabac0cd890 0x7fabac0b06b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faba4007b60 tx=0x7faba400d490 comp rx=0 tx=0).stop 2023-12-19T16:42:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.804+0000 7fab927fc640 1 -- 172.21.15.26:0/1786257507 shutdown_connections 2023-12-19T16:42:37.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.804+0000 7fab927fc640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fabac0ce5b0 0x7fabac140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.804+0000 7fab927fc640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fab98065a60 0x7fab98067f20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.804+0000 7fab927fc640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fabac0cd890 0x7fabac0b06b0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faba4007b60 tx=0x7faba400d490 comp rx=0 tx=0).stop 2023-12-19T16:42:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.804+0000 7fab927fc640 1 --2- 172.21.15.26:0/1786257507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabac071d60 0x7fabac14e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.804+0000 7fab927fc640 1 -- 172.21.15.26:0/1786257507 >> 172.21.15.26:0/1786257507 conn(0x7fabac06b3d0 msgr2=0x7fabac156cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.804+0000 7fab927fc640 1 -- 172.21.15.26:0/1786257507 shutdown_connections 2023-12-19T16:42:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.804+0000 7fab927fc640 1 -- 172.21.15.26:0/1786257507 wait complete. 2023-12-19T16:42:37.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.812+0000 7f523b536640 1 -- 172.21.15.26:0/3188226694 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5234164940 con 0x7f52340ad740 2023-12-19T16:42:37.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.812+0000 7f522affd640 1 -- 172.21.15.26:0/3188226694 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5230021090 con 0x7f52340ad740 2023-12-19T16:42:37.816 INFO:teuthology.orchestra.run.smithi026.stdout:64424509447 2023-12-19T16:42:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.812+0000 7f523b536640 1 -- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5218065a60 msgr2=0x7f5218067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.812+0000 7f523b536640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5218065a60 0x7f5218067f20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f52240b2000 tx=0x7f5224008040 comp rx=0 tx=0).stop 2023-12-19T16:42:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.812+0000 7f523b536640 1 -- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52340ad740 msgr2=0x7f5234071390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.812+0000 7f523b536640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52340ad740 0x7f5234071390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f523000c420 tx=0x7f523000c8e0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.812+0000 7f523b536640 1 -- 172.21.15.26:0/3188226694 shutdown_connections 2023-12-19T16:42:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.812+0000 7f523b536640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5218065a60 0x7f5218067f20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.812+0000 7f523b536640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52340b40e0 0x7f52340c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.812+0000 7f523b536640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52340b0700 0x7f5234159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.812+0000 7f523b536640 1 --2- 172.21.15.26:0/3188226694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52340ad740 0x7f5234071390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.812+0000 7f523b536640 1 -- 172.21.15.26:0/3188226694 >> 172.21.15.26:0/3188226694 conn(0x7f523406b060 msgr2=0x7f52340b1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.812+0000 7f523b536640 1 -- 172.21.15.26:0/3188226694 shutdown_connections 2023-12-19T16:42:37.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.816+0000 7f523b536640 1 -- 172.21.15.26:0/3188226694 wait complete. 2023-12-19T16:42:37.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.6 2023-12-19T16:42:37.841 DEBUG:teuthology.parallel:result is None 2023-12-19T16:42:37.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.1 2023-12-19T16:42:37.847 DEBUG:teuthology.parallel:result is None 2023-12-19T16:42:37.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.856+0000 7efd371bc640 1 -- 172.21.15.26:0/2131445070 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7efd3014d1e0 con 0x7efd300cd7f0 2023-12-19T16:42:37.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.856+0000 7efd2d7fa640 1 -- 172.21.15.26:0/2131445070 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efd3014d1e0 con 0x7efd300cd7f0 2023-12-19T16:42:37.861 INFO:teuthology.orchestra.run.smithi026.stdout:64424509447 2023-12-19T16:42:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.860+0000 7efd371bc640 1 -- 172.21.15.26:0/2131445070 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd10065ab0 msgr2=0x7efd10067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.860+0000 7efd371bc640 1 --2- 172.21.15.26:0/2131445070 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd10065ab0 0x7efd10067f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efd28000f70 tx=0x7efd28007bb0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.860+0000 7efd371bc640 1 -- 172.21.15.26:0/2131445070 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd300cd7f0 msgr2=0x7efd3013adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.860+0000 7efd371bc640 1 --2- 172.21.15.26:0/2131445070 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd300cd7f0 0x7efd3013adc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efd1800c730 tx=0x7efd1800cbf0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.860+0000 7efd371bc640 1 -- 172.21.15.26:0/2131445070 shutdown_connections 2023-12-19T16:42:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.860+0000 7efd371bc640 1 --2- 172.21.15.26:0/2131445070 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd10065ab0 0x7efd10067f70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.860+0000 7efd371bc640 1 --2- 172.21.15.26:0/2131445070 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd300cd7f0 0x7efd3013adc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.860+0000 7efd371bc640 1 --2- 172.21.15.26:0/2131445070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd300b1e20 0x7efd300ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.860+0000 7efd371bc640 1 --2- 172.21.15.26:0/2131445070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd300adbe0 0x7efd300af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.860+0000 7efd371bc640 1 -- 172.21.15.26:0/2131445070 >> 172.21.15.26:0/2131445070 conn(0x7efd300b03f0 msgr2=0x7efd3006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.860+0000 7efd371bc640 1 -- 172.21.15.26:0/2131445070 shutdown_connections 2023-12-19T16:42:37.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.860+0000 7efd371bc640 1 -- 172.21.15.26:0/2131445070 wait complete. 2023-12-19T16:42:37.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.876+0000 7f1e7417b640 1 -- 172.21.15.26:0/2372212913 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1e6c0acc40 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.4 2023-12-19T16:42:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.876+0000 7fa61c4b6640 1 -- 172.21.15.26:0/3647380552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa61415ce40 con 0x7fa6140cb1f0 2023-12-19T16:42:37.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.876+0000 7f1e6a7fc640 1 -- 172.21.15.26:0/2372212913 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1e64054570 con 0x7f1e6c0b1b70 2023-12-19T16:42:37.884 DEBUG:teuthology.parallel:result is None 2023-12-19T16:42:37.885 INFO:teuthology.orchestra.run.smithi026.stdout:64424509447 2023-12-19T16:42:37.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7fa60affd640 1 -- 172.21.15.26:0/3647380552 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa61415ce40 con 0x7fa6140cb1f0 2023-12-19T16:42:37.886 INFO:teuthology.orchestra.run.smithi026.stdout:64424509447 2023-12-19T16:42:37.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7f1e7417b640 1 -- 172.21.15.26:0/2372212913 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1e50065b60 msgr2=0x7f1e50068020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7f1e7417b640 1 --2- 172.21.15.26:0/2372212913 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1e50065b60 0x7f1e50068020 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1e60005c10 tx=0x7f1e60009040 comp rx=0 tx=0).stop 2023-12-19T16:42:37.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7f1e7417b640 1 -- 172.21.15.26:0/2372212913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e6c0b1b70 msgr2=0x7f1e6c14bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7f1e7417b640 1 --2- 172.21.15.26:0/2372212913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e6c0b1b70 0x7f1e6c14bc40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400d830 tx=0x7f1e6400dcf0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7f1e7417b640 1 -- 172.21.15.26:0/2372212913 shutdown_connections 2023-12-19T16:42:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7f1e7417b640 1 --2- 172.21.15.26:0/2372212913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e6c0b1b70 0x7f1e6c14bc40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7f1e7417b640 1 --2- 172.21.15.26:0/2372212913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e6c0ad4a0 0x7f1e6c14b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7f1e7417b640 1 --2- 172.21.15.26:0/2372212913 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1e50065b60 0x7f1e50068020 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7f1e7417b640 1 --2- 172.21.15.26:0/2372212913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e6c0ad0c0 0x7f1e6c15b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7f1e7417b640 1 -- 172.21.15.26:0/2372212913 >> 172.21.15.26:0/2372212913 conn(0x7f1e6c06b3d0 msgr2=0x7f1e6c0ae1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7f1e7417b640 1 -- 172.21.15.26:0/2372212913 shutdown_connections 2023-12-19T16:42:37.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7f1e7417b640 1 -- 172.21.15.26:0/2372212913 wait complete. 2023-12-19T16:42:37.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7fa61c4b6640 1 -- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa5f8065ac0 msgr2=0x7fa5f8067f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7fa61c4b6640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa5f8065ac0 0x7fa5f8067f80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa6100016d0 tx=0x7fa610009040 comp rx=0 tx=0).stop 2023-12-19T16:42:37.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7fa61c4b6640 1 -- 172.21.15.26:0/3647380552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6140cb1f0 msgr2=0x7fa61415f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7fa61c4b6640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6140cb1f0 0x7fa61415f310 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa60400d5f0 tx=0x7fa60400dab0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7fa61c4b6640 1 -- 172.21.15.26:0/3647380552 shutdown_connections 2023-12-19T16:42:37.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7fa61c4b6640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa5f8065ac0 0x7fa5f8067f80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7fa61c4b6640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa614164d50 0x7fa614155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7fa61c4b6640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6140cb1f0 0x7fa61415f310 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7fa61c4b6640 1 --2- 172.21.15.26:0/3647380552 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6140b1880 0x7fa614160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7fa61c4b6640 1 -- 172.21.15.26:0/3647380552 >> 172.21.15.26:0/3647380552 conn(0x7fa6140b03f0 msgr2=0x7fa6140cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7fa61c4b6640 1 -- 172.21.15.26:0/3647380552 shutdown_connections 2023-12-19T16:42:37.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.880+0000 7fa61c4b6640 1 -- 172.21.15.26:0/3647380552 wait complete. 2023-12-19T16:42:37.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.888+0000 7f35f7404640 1 -- 172.21.15.26:0/2929809529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f35f00c7180 con 0x7f35f00cf410 2023-12-19T16:42:37.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.888+0000 7f35e5ffb640 1 -- 172.21.15.26:0/2929809529 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f35e801a090 con 0x7f35f00cf410 2023-12-19T16:42:37.893 INFO:teuthology.orchestra.run.smithi026.stdout:64424509446 2023-12-19T16:42:37.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7f35f7404640 1 -- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35d4065a60 msgr2=0x7f35d4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7f35f7404640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35d4065a60 0x7f35d4067f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f35f0149780 tx=0x7f35e0008040 comp rx=0 tx=0).stop 2023-12-19T16:42:37.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7f35f7404640 1 -- 172.21.15.26:0/2929809529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f00cf410 msgr2=0x7f35f0149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7f35f7404640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f00cf410 0x7f35f0149240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f35e80070b0 tx=0x7f35e800d4a0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7f35f7404640 1 -- 172.21.15.26:0/2929809529 shutdown_connections 2023-12-19T16:42:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7f35f7404640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f015d1c0 0x7f35f0148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7f35f7404640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35d4065a60 0x7f35d4067f20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7f35f7404640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f015cde0 0x7f35f00c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7f35f7404640 1 --2- 172.21.15.26:0/2929809529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f00cf410 0x7f35f0149240 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7f35f7404640 1 -- 172.21.15.26:0/2929809529 >> 172.21.15.26:0/2929809529 conn(0x7f35f00658e0 msgr2=0x7f35f00d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7f35f7404640 1 -- 172.21.15.26:0/2929809529 shutdown_connections 2023-12-19T16:42:37.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7f35f7404640 1 -- 172.21.15.26:0/2929809529 wait complete. 2023-12-19T16:42:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7fa509dfd640 1 -- 172.21.15.26:0/690508319 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa504074270 con 0x7fa4f00047d0 2023-12-19T16:42:37.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.892+0000 7fa4fbfff640 1 -- 172.21.15.26:0/690508319 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa4fc059e60 con 0x7fa4f00047d0 2023-12-19T16:42:37.898 INFO:teuthology.orchestra.run.smithi026.stdout:64424509447 2023-12-19T16:42:37.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.896+0000 7fa509dfd640 1 -- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4e4065b60 msgr2=0x7fa4e4068020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.896+0000 7fa509dfd640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4e4065b60 0x7fa4e4068020 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec001730 tx=0x7fa4ec0011a0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.896+0000 7fa509dfd640 1 -- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4f00047d0 msgr2=0x7fa5040a92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:37.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.896+0000 7fa509dfd640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4f00047d0 0x7fa5040a92a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc0079d0 tx=0x7fa4fc00a6a0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.896+0000 7fa509dfd640 1 -- 172.21.15.26:0/690508319 shutdown_connections 2023-12-19T16:42:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.896+0000 7fa509dfd640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4e4065b60 0x7fa4e4068020 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.896+0000 7fa509dfd640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5040a9f10 0x7fa5040b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.896+0000 7fa509dfd640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa504141540 0x7fa5040a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.896+0000 7fa509dfd640 1 --2- 172.21.15.26:0/690508319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4f00047d0 0x7fa5040a92a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.896+0000 7fa509dfd640 1 -- 172.21.15.26:0/690508319 >> 172.21.15.26:0/690508319 conn(0x7fa5040b03f0 msgr2=0x7fa5040ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:37.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.896+0000 7fa509dfd640 1 -- 172.21.15.26:0/690508319 shutdown_connections 2023-12-19T16:42:37.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:37.896+0000 7fa509dfd640 1 -- 172.21.15.26:0/690508319 wait complete. 2023-12-19T16:42:37.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.5 2023-12-19T16:42:37.910 DEBUG:teuthology.parallel:result is None 2023-12-19T16:42:37.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.3 2023-12-19T16:42:37.915 DEBUG:teuthology.parallel:result is None 2023-12-19T16:42:37.923 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.7 2023-12-19T16:42:37.924 DEBUG:teuthology.parallel:result is None 2023-12-19T16:42:37.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.2 2023-12-19T16:42:37.925 DEBUG:teuthology.parallel:result is None 2023-12-19T16:42:37.925 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-19T16:42:37.925 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:42:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.108+0000 7f91e23a2640 1 Processor -- start 2023-12-19T16:42:38.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.108+0000 7f91e23a2640 1 -- start start 2023-12-19T16:42:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.108+0000 7f91e23a2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f91dc164f00 0x7f91dc0cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.108+0000 7f91e23a2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91dc0cb3a0 0x7f91dc0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.108+0000 7f91e23a2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91dc0ade10 con 0x7f91dc0ada30 2023-12-19T16:42:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.108+0000 7f91e23a2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91dc0b16e0 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.108+0000 7f91e23a2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91dc0cc9d0 con 0x7f91dc164f00 2023-12-19T16:42:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.108+0000 7f91e1ba1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91dc0cb3a0 0x7f91dc0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.108+0000 7f91e1ba1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91dc0cb3a0 0x7f91dc0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33374/0 (socket says 172.21.15.26:33374) 2023-12-19T16:42:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.108+0000 7f91e1ba1640 1 -- 172.21.15.26:0/448500472 learned_addr learned my addr 172.21.15.26:0/448500472 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.108+0000 7f91e0b9f640 1 --2- 172.21.15.26:0/448500472 >> v2:172.21.15.160:3300/0 conn(0x7f91dc164f00 0x7f91dc0cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91e1ba1640 1 -- 172.21.15.26:0/448500472 >> v2:172.21.15.160:3300/0 conn(0x7f91dc164f00 msgr2=0x7f91dc0cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91e1ba1640 1 --2- 172.21.15.26:0/448500472 >> v2:172.21.15.160:3300/0 conn(0x7f91dc164f00 0x7f91dc0cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91e1ba1640 1 -- 172.21.15.26:0/448500472 >> v1:172.21.15.160:6789/0 conn(0x7f91dc0ada30 legacy=0x7f91dc0c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91e1ba1640 1 -- 172.21.15.26:0/448500472 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc161830 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91e1ba1640 1 --2- 172.21.15.26:0/448500472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91dc0cb3a0 0x7f91dc0cce60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0b10f0 tx=0x7f91d401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d853790d3fc788 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:38.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91d3fff640 1 -- 172.21.15.26:0/448500472 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d40a5890 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91d3fff640 1 -- 172.21.15.26:0/448500472 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d40a59f0 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91d3fff640 1 -- 172.21.15.26:0/448500472 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d40a5d80 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91e23a2640 1 -- 172.21.15.26:0/448500472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91dc0cb3a0 msgr2=0x7f91dc0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91e23a2640 1 --2- 172.21.15.26:0/448500472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91dc0cb3a0 0x7f91dc0cce60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f91dc0b10f0 tx=0x7f91d401c9d0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91e23a2640 1 -- 172.21.15.26:0/448500472 shutdown_connections 2023-12-19T16:42:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91e23a2640 1 --2- 172.21.15.26:0/448500472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91dc0cb3a0 0x7f91dc0cce60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91e23a2640 1 --2- 172.21.15.26:0/448500472 >> v2:172.21.15.160:3300/0 conn(0x7f91dc164f00 0x7f91dc0cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.112+0000 7f91e23a2640 1 -- 172.21.15.26:0/448500472 >> 172.21.15.26:0/448500472 conn(0x7f91dc06b0f0 msgr2=0x7f91dc06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e23a2640 1 -- 172.21.15.26:0/448500472 shutdown_connections 2023-12-19T16:42:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e23a2640 1 -- 172.21.15.26:0/448500472 wait complete. 2023-12-19T16:42:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e23a2640 1 Processor -- start 2023-12-19T16:42:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e23a2640 1 -- start start 2023-12-19T16:42:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e23a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91dc0ada30 0x7f91dc0c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e23a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91dc0cb3a0 0x7f91dc157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e23a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91dc164f00 0x7f91dc15f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e23a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc070c70 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e23a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc15a7a0 con 0x7f91dc0ada30 2023-12-19T16:42:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e23a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc158360 con 0x7f91dc164f00 2023-12-19T16:42:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e0b9f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91dc0cb3a0 0x7f91dc157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e0b9f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91dc0cb3a0 0x7f91dc157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33388/0 (socket says 172.21.15.26:33388) 2023-12-19T16:42:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e0b9f640 1 -- 172.21.15.26:0/250294686 learned_addr learned my addr 172.21.15.26:0/250294686 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e13a0640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91dc0ada30 0x7f91dc0c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e1ba1640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91dc164f00 0x7f91dc15f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e0b9f640 1 -- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91dc164f00 msgr2=0x7f91dc15f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e0b9f640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91dc164f00 0x7f91dc15f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e0b9f640 1 -- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91dc0ada30 msgr2=0x7f91dc0c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e0b9f640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91dc0ada30 0x7f91dc0c1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e0b9f640 1 -- 172.21.15.26:0/250294686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d4009b30 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e0b9f640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91dc0cb3a0 0x7f91dc157e20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00d830 tx=0x7f91cc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:38.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91d1ffb640 1 -- 172.21.15.26:0/250294686 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc007e00 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91d1ffb640 1 -- 172.21.15.26:0/250294686 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc00b8d0 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91d1ffb640 1 -- 172.21.15.26:0/250294686 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00bc80 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.116+0000 7f91e23a2640 1 -- 172.21.15.26:0/250294686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91dc159b90 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.120+0000 7f91e23a2640 1 -- 172.21.15.26:0/250294686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91dc15a150 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.120+0000 7f91e23a2640 1 -- 172.21.15.26:0/250294686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91dc067400 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.120+0000 7f91d1ffb640 1 -- 172.21.15.26:0/250294686 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f91cc01e460 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.120+0000 7f91d1ffb640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91c0065a60 0x7f91c0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.120+0000 7f91d1ffb640 1 -- 172.21.15.26:0/250294686 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91cc083630 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.120+0000 7f91e13a0640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91c0065a60 0x7f91c0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.120+0000 7f91e13a0640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91c0065a60 0x7f91c0067f20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91d8007fd0 tx=0x7f91d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:38.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.124+0000 7f91d1ffb640 1 -- 172.21.15.26:0/250294686 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f91cc050f70 con 0x7f91dc0cb3a0 2023-12-19T16:42:38.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.280+0000 7f91e23a2640 1 -- 172.21.15.26:0/250294686 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91dc1656c0 con 0x7f91c0065a60 2023-12-19T16:42:38.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.288+0000 7f91d1ffb640 1 -- 172.21.15.26:0/250294686 <== mgr.4106 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264447 (secure 0 0 0) 0x7f91d80014f0 con 0x7f91c0065a60 2023-12-19T16:42:38.294 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:38.296 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:42:38.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.292+0000 7f91e23a2640 1 -- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91c0065a60 msgr2=0x7f91c0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.292+0000 7f91e23a2640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91c0065a60 0x7f91c0067f20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f91d8007fd0 tx=0x7f91d8009040 comp rx=0 tx=0).stop 2023-12-19T16:42:38.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.292+0000 7f91e23a2640 1 -- 172.21.15.26:0/250294686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91dc0cb3a0 msgr2=0x7f91dc157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.292+0000 7f91e23a2640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91dc0cb3a0 0x7f91dc157e20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00d830 tx=0x7f91cc00dcf0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.292+0000 7f91e23a2640 1 -- 172.21.15.26:0/250294686 shutdown_connections 2023-12-19T16:42:38.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.292+0000 7f91e23a2640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91c0065a60 0x7f91c0067f20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.292+0000 7f91e23a2640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91dc164f00 0x7f91dc15f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.292+0000 7f91e23a2640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91dc0cb3a0 0x7f91dc157e20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.292+0000 7f91e23a2640 1 --2- 172.21.15.26:0/250294686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91dc0ada30 0x7f91dc0c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.292+0000 7f91e23a2640 1 -- 172.21.15.26:0/250294686 >> 172.21.15.26:0/250294686 conn(0x7f91dc06b0f0 msgr2=0x7f91dc0ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:38.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.292+0000 7f91e23a2640 1 -- 172.21.15.26:0/250294686 shutdown_connections 2023-12-19T16:42:38.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.292+0000 7f91e23a2640 1 -- 172.21.15.26:0/250294686 wait complete. 2023-12-19T16:42:38.315 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-12-19T16:42:37.638794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235048,"kb_used_data":3048,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739672,"statfs":{"total":773094113280,"available":772853424128,"internally_reserved":0,"allocated":3121152,"data_stored":1832748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.854877"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658321+0000","last_change":"2023-12-19T16:42:27.824402+0000","last_active":"2023-12-19T16:42:29.658321+0000","last_peered":"2023-12-19T16:42:29.658321+0000","last_clean":"2023-12-19T16:42:29.658321+0000","last_became_active":"2023-12-19T16:42:27.818342+0000","last_became_peered":"2023-12-19T16:42:27.818342+0000","last_unstale":"2023-12-19T16:42:29.658321+0000","last_undegraded":"2023-12-19T16:42:29.658321+0000","last_fullsized":"2023-12-19T16:42:29.658321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:53.498191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.956000+0000","last_change":"2023-12-19T16:42:27.802326+0000","last_active":"2023-12-19T16:42:28.956000+0000","last_peered":"2023-12-19T16:42:28.956000+0000","last_clean":"2023-12-19T16:42:28.956000+0000","last_became_active":"2023-12-19T16:42:27.802056+0000","last_became_peered":"2023-12-19T16:42:27.802056+0000","last_unstale":"2023-12-19T16:42:28.956000+0000","last_undegraded":"2023-12-19T16:42:28.956000+0000","last_fullsized":"2023-12-19T16:42:28.956000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:30.545767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648509+0000","last_change":"2023-12-19T16:42:27.798150+0000","last_active":"2023-12-19T16:42:28.648509+0000","last_peered":"2023-12-19T16:42:28.648509+0000","last_clean":"2023-12-19T16:42:28.648509+0000","last_became_active":"2023-12-19T16:42:27.796339+0000","last_became_peered":"2023-12-19T16:42:27.796339+0000","last_unstale":"2023-12-19T16:42:28.648509+0000","last_undegraded":"2023-12-19T16:42:28.648509+0000","last_fullsized":"2023-12-19T16:42:28.648509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:42:19.277925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790632+0000","last_change":"2023-12-19T16:42:27.814794+0000","last_active":"2023-12-19T16:42:28.790632+0000","last_peered":"2023-12-19T16:42:28.790632+0000","last_clean":"2023-12-19T16:42:28.790632+0000","last_became_active":"2023-12-19T16:42:27.814461+0000","last_became_peered":"2023-12-19T16:42:27.814461+0000","last_unstale":"2023-12-19T16:42:28.790632+0000","last_undegraded":"2023-12-19T16:42:28.790632+0000","last_fullsized":"2023-12-19T16:42:28.790632+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:18.209972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658113+0000","last_change":"2023-12-19T16:42:27.825817+0000","last_active":"2023-12-19T16:42:29.658113+0000","last_peered":"2023-12-19T16:42:29.658113+0000","last_clean":"2023-12-19T16:42:29.658113+0000","last_became_active":"2023-12-19T16:42:27.825532+0000","last_became_peered":"2023-12-19T16:42:27.825532+0000","last_unstale":"2023-12-19T16:42:29.658113+0000","last_undegraded":"2023-12-19T16:42:29.658113+0000","last_fullsized":"2023-12-19T16:42:29.658113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:49.764257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790441+0000","last_change":"2023-12-19T16:42:27.801183+0000","last_active":"2023-12-19T16:42:28.790441+0000","last_peered":"2023-12-19T16:42:28.790441+0000","last_clean":"2023-12-19T16:42:28.790441+0000","last_became_active":"2023-12-19T16:42:27.800866+0000","last_became_peered":"2023-12-19T16:42:27.800866+0000","last_unstale":"2023-12-19T16:42:28.790441+0000","last_undegraded":"2023-12-19T16:42:28.790441+0000","last_fullsized":"2023-12-19T16:42:28.790441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:36.473614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658078+0000","last_change":"2023-12-19T16:42:27.817242+0000","last_active":"2023-12-19T16:42:29.658078+0000","last_peered":"2023-12-19T16:42:29.658078+0000","last_clean":"2023-12-19T16:42:29.658078+0000","last_became_active":"2023-12-19T16:42:27.816801+0000","last_became_peered":"2023-12-19T16:42:27.816801+0000","last_unstale":"2023-12-19T16:42:29.658078+0000","last_undegraded":"2023-12-19T16:42:29.658078+0000","last_fullsized":"2023-12-19T16:42:29.658078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651814+0000","last_change":"2023-12-19T16:42:27.801939+0000","last_active":"2023-12-19T16:42:29.651814+0000","last_peered":"2023-12-19T16:42:29.651814+0000","last_clean":"2023-12-19T16:42:29.651814+0000","last_became_active":"2023-12-19T16:42:27.801672+0000","last_became_peered":"2023-12-19T16:42:27.801672+0000","last_unstale":"2023-12-19T16:42:29.651814+0000","last_undegraded":"2023-12-19T16:42:29.651814+0000","last_fullsized":"2023-12-19T16:42:29.651814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658359+0000","last_change":"2023-12-19T16:42:27.829667+0000","last_active":"2023-12-19T16:42:29.658359+0000","last_peered":"2023-12-19T16:42:29.658359+0000","last_clean":"2023-12-19T16:42:29.658359+0000","last_became_active":"2023-12-19T16:42:27.829357+0000","last_became_peered":"2023-12-19T16:42:27.829357+0000","last_unstale":"2023-12-19T16:42:29.658359+0000","last_undegraded":"2023-12-19T16:42:29.658359+0000","last_fullsized":"2023-12-19T16:42:29.658359+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:06.914435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654225+0000","last_change":"2023-12-19T16:42:27.812001+0000","last_active":"2023-12-19T16:42:29.654225+0000","last_peered":"2023-12-19T16:42:29.654225+0000","last_clean":"2023-12-19T16:42:29.654225+0000","last_became_active":"2023-12-19T16:42:27.811749+0000","last_became_peered":"2023-12-19T16:42:27.811749+0000","last_unstale":"2023-12-19T16:42:29.654225+0000","last_undegraded":"2023-12-19T16:42:29.654225+0000","last_fullsized":"2023-12-19T16:42:29.654225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:29.376069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654212+0000","last_change":"2023-12-19T16:42:27.809484+0000","last_active":"2023-12-19T16:42:29.654212+0000","last_peered":"2023-12-19T16:42:29.654212+0000","last_clean":"2023-12-19T16:42:29.654212+0000","last_became_active":"2023-12-19T16:42:27.809189+0000","last_became_peered":"2023-12-19T16:42:27.809189+0000","last_unstale":"2023-12-19T16:42:29.654212+0000","last_undegraded":"2023-12-19T16:42:29.654212+0000","last_fullsized":"2023-12-19T16:42:29.654212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:03.390272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789997+0000","last_change":"2023-12-19T16:42:27.812088+0000","last_active":"2023-12-19T16:42:28.789997+0000","last_peered":"2023-12-19T16:42:28.789997+0000","last_clean":"2023-12-19T16:42:28.789997+0000","last_became_active":"2023-12-19T16:42:27.811768+0000","last_became_peered":"2023-12-19T16:42:27.811768+0000","last_unstale":"2023-12-19T16:42:28.789997+0000","last_undegraded":"2023-12-19T16:42:28.789997+0000","last_fullsized":"2023-12-19T16:42:28.789997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:47:07.567597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651980+0000","last_change":"2023-12-19T16:42:27.805169+0000","last_active":"2023-12-19T16:42:29.651980+0000","last_peered":"2023-12-19T16:42:29.651980+0000","last_clean":"2023-12-19T16:42:29.651980+0000","last_became_active":"2023-12-19T16:42:27.804906+0000","last_became_peered":"2023-12-19T16:42:27.804906+0000","last_unstale":"2023-12-19T16:42:29.651980+0000","last_undegraded":"2023-12-19T16:42:29.651980+0000","last_fullsized":"2023-12-19T16:42:29.651980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:57:49.461118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648451+0000","last_change":"2023-12-19T16:42:27.796100+0000","last_active":"2023-12-19T16:42:28.648451+0000","last_peered":"2023-12-19T16:42:28.648451+0000","last_clean":"2023-12-19T16:42:28.648451+0000","last_became_active":"2023-12-19T16:42:27.795802+0000","last_became_peered":"2023-12-19T16:42:27.795802+0000","last_unstale":"2023-12-19T16:42:28.648451+0000","last_undegraded":"2023-12-19T16:42:28.648451+0000","last_fullsized":"2023-12-19T16:42:28.648451+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:30:13.550890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955716+0000","last_change":"2023-12-19T16:42:27.815496+0000","last_active":"2023-12-19T16:42:28.955716+0000","last_peered":"2023-12-19T16:42:28.955716+0000","last_clean":"2023-12-19T16:42:28.955716+0000","last_became_active":"2023-12-19T16:42:27.812535+0000","last_became_peered":"2023-12-19T16:42:27.812535+0000","last_unstale":"2023-12-19T16:42:28.955716+0000","last_undegraded":"2023-12-19T16:42:28.955716+0000","last_fullsized":"2023-12-19T16:42:28.955716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:37.400723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790179+0000","last_change":"2023-12-19T16:42:27.804674+0000","last_active":"2023-12-19T16:42:28.790179+0000","last_peered":"2023-12-19T16:42:28.790179+0000","last_clean":"2023-12-19T16:42:28.790179+0000","last_became_active":"2023-12-19T16:42:27.804057+0000","last_became_peered":"2023-12-19T16:42:27.804057+0000","last_unstale":"2023-12-19T16:42:28.790179+0000","last_undegraded":"2023-12-19T16:42:28.790179+0000","last_fullsized":"2023-12-19T16:42:28.790179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:57:49.461118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790884+0000","last_change":"2023-12-19T16:42:27.829970+0000","last_active":"2023-12-19T16:42:28.790884+0000","last_peered":"2023-12-19T16:42:28.790884+0000","last_clean":"2023-12-19T16:42:28.790884+0000","last_became_active":"2023-12-19T16:42:27.829616+0000","last_became_peered":"2023-12-19T16:42:27.829616+0000","last_unstale":"2023-12-19T16:42:28.790884+0000","last_undegraded":"2023-12-19T16:42:28.790884+0000","last_fullsized":"2023-12-19T16:42:28.790884+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:50.303035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784083+0000","last_change":"2023-12-19T16:42:27.804309+0000","last_active":"2023-12-19T16:42:28.784083+0000","last_peered":"2023-12-19T16:42:28.784083+0000","last_clean":"2023-12-19T16:42:28.784083+0000","last_became_active":"2023-12-19T16:42:27.803973+0000","last_became_peered":"2023-12-19T16:42:27.803973+0000","last_unstale":"2023-12-19T16:42:28.784083+0000","last_undegraded":"2023-12-19T16:42:28.784083+0000","last_fullsized":"2023-12-19T16:42:28.784083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:47:07.567597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658270+0000","last_change":"2023-12-19T16:42:27.827321+0000","last_active":"2023-12-19T16:42:29.658270+0000","last_peered":"2023-12-19T16:42:29.658270+0000","last_clean":"2023-12-19T16:42:29.658270+0000","last_became_active":"2023-12-19T16:42:27.826329+0000","last_became_peered":"2023-12-19T16:42:27.826329+0000","last_unstale":"2023-12-19T16:42:29.658270+0000","last_undegraded":"2023-12-19T16:42:29.658270+0000","last_fullsized":"2023-12-19T16:42:29.658270+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:01.122709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654188+0000","last_change":"2023-12-19T16:42:27.816854+0000","last_active":"2023-12-19T16:42:29.654188+0000","last_peered":"2023-12-19T16:42:29.654188+0000","last_clean":"2023-12-19T16:42:29.654188+0000","last_became_active":"2023-12-19T16:42:27.816538+0000","last_became_peered":"2023-12-19T16:42:27.816538+0000","last_unstale":"2023-12-19T16:42:29.654188+0000","last_undegraded":"2023-12-19T16:42:29.654188+0000","last_fullsized":"2023-12-19T16:42:29.654188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:30.545767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648470+0000","last_change":"2023-12-19T16:42:27.817144+0000","last_active":"2023-12-19T16:42:28.648470+0000","last_peered":"2023-12-19T16:42:28.648470+0000","last_clean":"2023-12-19T16:42:28.648470+0000","last_became_active":"2023-12-19T16:42:27.816891+0000","last_became_peered":"2023-12-19T16:42:27.816891+0000","last_unstale":"2023-12-19T16:42:28.648470+0000","last_undegraded":"2023-12-19T16:42:28.648470+0000","last_fullsized":"2023-12-19T16:42:28.648470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:51.134371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790438+0000","last_change":"2023-12-19T16:42:27.826273+0000","last_active":"2023-12-19T16:42:28.790438+0000","last_peered":"2023-12-19T16:42:28.790438+0000","last_clean":"2023-12-19T16:42:28.790438+0000","last_became_active":"2023-12-19T16:42:27.825992+0000","last_became_peered":"2023-12-19T16:42:27.825992+0000","last_unstale":"2023-12-19T16:42:28.790438+0000","last_undegraded":"2023-12-19T16:42:28.790438+0000","last_fullsized":"2023-12-19T16:42:28.790438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:36.031318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651909+0000","last_change":"2023-12-19T16:42:27.813323+0000","last_active":"2023-12-19T16:42:29.651909+0000","last_peered":"2023-12-19T16:42:29.651909+0000","last_clean":"2023-12-19T16:42:29.651909+0000","last_became_active":"2023-12-19T16:42:27.812808+0000","last_became_peered":"2023-12-19T16:42:27.812808+0000","last_unstale":"2023-12-19T16:42:29.651909+0000","last_undegraded":"2023-12-19T16:42:29.651909+0000","last_fullsized":"2023-12-19T16:42:29.651909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:36.473614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790317+0000","last_change":"2023-12-19T16:42:27.810294+0000","last_active":"2023-12-19T16:42:28.790317+0000","last_peered":"2023-12-19T16:42:28.790317+0000","last_clean":"2023-12-19T16:42:28.790317+0000","last_became_active":"2023-12-19T16:42:27.810031+0000","last_became_peered":"2023-12-19T16:42:27.810031+0000","last_unstale":"2023-12-19T16:42:28.790317+0000","last_undegraded":"2023-12-19T16:42:28.790317+0000","last_fullsized":"2023-12-19T16:42:28.790317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:01.122709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652166+0000","last_change":"2023-12-19T16:42:27.813230+0000","last_active":"2023-12-19T16:42:29.652166+0000","last_peered":"2023-12-19T16:42:29.652166+0000","last_clean":"2023-12-19T16:42:29.652166+0000","last_became_active":"2023-12-19T16:42:27.812914+0000","last_became_peered":"2023-12-19T16:42:27.812914+0000","last_unstale":"2023-12-19T16:42:29.652166+0000","last_undegraded":"2023-12-19T16:42:29.652166+0000","last_fullsized":"2023-12-19T16:42:29.652166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:50.303035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654238+0000","last_change":"2023-12-19T16:42:27.819653+0000","last_active":"2023-12-19T16:42:29.654238+0000","last_peered":"2023-12-19T16:42:29.654238+0000","last_clean":"2023-12-19T16:42:29.654238+0000","last_became_active":"2023-12-19T16:42:27.819325+0000","last_became_peered":"2023-12-19T16:42:27.819325+0000","last_unstale":"2023-12-19T16:42:29.654238+0000","last_undegraded":"2023-12-19T16:42:29.654238+0000","last_fullsized":"2023-12-19T16:42:29.654238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:40:46.397053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955842+0000","last_change":"2023-12-19T16:42:27.813010+0000","last_active":"2023-12-19T16:42:28.955842+0000","last_peered":"2023-12-19T16:42:28.955842+0000","last_clean":"2023-12-19T16:42:28.955842+0000","last_became_active":"2023-12-19T16:42:27.812336+0000","last_became_peered":"2023-12-19T16:42:27.812336+0000","last_unstale":"2023-12-19T16:42:28.955842+0000","last_undegraded":"2023-12-19T16:42:28.955842+0000","last_fullsized":"2023-12-19T16:42:28.955842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652085+0000","last_change":"2023-12-19T16:42:27.813584+0000","last_active":"2023-12-19T16:42:29.652085+0000","last_peered":"2023-12-19T16:42:29.652085+0000","last_clean":"2023-12-19T16:42:29.652085+0000","last_became_active":"2023-12-19T16:42:27.813276+0000","last_became_peered":"2023-12-19T16:42:27.813276+0000","last_unstale":"2023-12-19T16:42:29.652085+0000","last_undegraded":"2023-12-19T16:42:29.652085+0000","last_fullsized":"2023-12-19T16:42:29.652085+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:36.031318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648478+0000","last_change":"2023-12-19T16:42:27.805390+0000","last_active":"2023-12-19T16:42:28.648478+0000","last_peered":"2023-12-19T16:42:28.648478+0000","last_clean":"2023-12-19T16:42:28.648478+0000","last_became_active":"2023-12-19T16:42:27.804956+0000","last_became_peered":"2023-12-19T16:42:27.804956+0000","last_unstale":"2023-12-19T16:42:28.648478+0000","last_undegraded":"2023-12-19T16:42:28.648478+0000","last_fullsized":"2023-12-19T16:42:28.648478+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:58.680381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.791132+0000","last_change":"2023-12-19T16:42:27.809669+0000","last_active":"2023-12-19T16:42:28.791132+0000","last_peered":"2023-12-19T16:42:28.791132+0000","last_clean":"2023-12-19T16:42:28.791132+0000","last_became_active":"2023-12-19T16:42:27.809374+0000","last_became_peered":"2023-12-19T16:42:27.809374+0000","last_unstale":"2023-12-19T16:42:28.791132+0000","last_undegraded":"2023-12-19T16:42:28.791132+0000","last_fullsized":"2023-12-19T16:42:28.791132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:53.498191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784039+0000","last_change":"2023-12-19T16:42:27.799272+0000","last_active":"2023-12-19T16:42:28.784039+0000","last_peered":"2023-12-19T16:42:28.784039+0000","last_clean":"2023-12-19T16:42:28.784039+0000","last_became_active":"2023-12-19T16:42:27.799006+0000","last_became_peered":"2023-12-19T16:42:27.799006+0000","last_unstale":"2023-12-19T16:42:28.784039+0000","last_undegraded":"2023-12-19T16:42:28.784039+0000","last_fullsized":"2023-12-19T16:42:28.784039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:37.286916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654047+0000","last_change":"2023-12-19T16:42:27.809714+0000","last_active":"2023-12-19T16:42:29.654047+0000","last_peered":"2023-12-19T16:42:29.654047+0000","last_clean":"2023-12-19T16:42:29.654047+0000","last_became_active":"2023-12-19T16:42:27.809384+0000","last_became_peered":"2023-12-19T16:42:27.809384+0000","last_unstale":"2023-12-19T16:42:29.654047+0000","last_undegraded":"2023-12-19T16:42:29.654047+0000","last_fullsized":"2023-12-19T16:42:29.654047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:37.400723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789634+0000","last_change":"2023-12-19T16:42:27.811964+0000","last_active":"2023-12-19T16:42:28.789634+0000","last_peered":"2023-12-19T16:42:28.789634+0000","last_clean":"2023-12-19T16:42:28.789634+0000","last_became_active":"2023-12-19T16:42:27.811777+0000","last_became_peered":"2023-12-19T16:42:27.811777+0000","last_unstale":"2023-12-19T16:42:28.789634+0000","last_undegraded":"2023-12-19T16:42:28.789634+0000","last_fullsized":"2023-12-19T16:42:28.789634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:50.714712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652057+0000","last_change":"2023-12-19T16:42:27.324768+0000","last_active":"2023-12-19T16:42:29.652057+0000","last_peered":"2023-12-19T16:42:29.652057+0000","last_clean":"2023-12-19T16:42:29.652057+0000","last_became_active":"2023-12-19T16:42:27.324380+0000","last_became_peered":"2023-12-19T16:42:27.324380+0000","last_unstale":"2023-12-19T16:42:29.652057+0000","last_undegraded":"2023-12-19T16:42:29.652057+0000","last_fullsized":"2023-12-19T16:42:29.652057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648490+0000","last_change":"2023-12-19T16:42:27.816607+0000","last_active":"2023-12-19T16:42:28.648490+0000","last_peered":"2023-12-19T16:42:28.648490+0000","last_clean":"2023-12-19T16:42:28.648490+0000","last_became_active":"2023-12-19T16:42:27.815987+0000","last_became_peered":"2023-12-19T16:42:27.815987+0000","last_unstale":"2023-12-19T16:42:28.648490+0000","last_undegraded":"2023-12-19T16:42:28.648490+0000","last_fullsized":"2023-12-19T16:42:28.648490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:32:26.852410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654158+0000","last_change":"2023-12-19T16:42:27.327298+0000","last_active":"2023-12-19T16:42:29.654158+0000","last_peered":"2023-12-19T16:42:29.654158+0000","last_clean":"2023-12-19T16:42:29.654158+0000","last_became_active":"2023-12-19T16:42:27.326988+0000","last_became_peered":"2023-12-19T16:42:27.326988+0000","last_unstale":"2023-12-19T16:42:29.654158+0000","last_undegraded":"2023-12-19T16:42:29.654158+0000","last_fullsized":"2023-12-19T16:42:29.654158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651933+0000","last_change":"2023-12-19T16:42:27.320910+0000","last_active":"2023-12-19T16:42:29.651933+0000","last_peered":"2023-12-19T16:42:29.651933+0000","last_clean":"2023-12-19T16:42:29.651933+0000","last_became_active":"2023-12-19T16:42:27.320629+0000","last_became_peered":"2023-12-19T16:42:27.320629+0000","last_unstale":"2023-12-19T16:42:29.651933+0000","last_undegraded":"2023-12-19T16:42:29.651933+0000","last_fullsized":"2023-12-19T16:42:29.651933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651846+0000","last_change":"2023-12-19T16:42:27.803836+0000","last_active":"2023-12-19T16:42:29.651846+0000","last_peered":"2023-12-19T16:42:29.651846+0000","last_clean":"2023-12-19T16:42:29.651846+0000","last_became_active":"2023-12-19T16:42:27.803527+0000","last_became_peered":"2023-12-19T16:42:27.803527+0000","last_unstale":"2023-12-19T16:42:29.651846+0000","last_undegraded":"2023-12-19T16:42:29.651846+0000","last_fullsized":"2023-12-19T16:42:29.651846+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:37.400723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790294+0000","last_change":"2023-12-19T16:42:27.322383+0000","last_active":"2023-12-19T16:42:28.790294+0000","last_peered":"2023-12-19T16:42:28.790294+0000","last_clean":"2023-12-19T16:42:28.790294+0000","last_became_active":"2023-12-19T16:42:27.321439+0000","last_became_peered":"2023-12-19T16:42:27.321439+0000","last_unstale":"2023-12-19T16:42:28.790294+0000","last_undegraded":"2023-12-19T16:42:28.790294+0000","last_fullsized":"2023-12-19T16:42:28.790294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955715+0000","last_change":"2023-12-19T16:42:27.800626+0000","last_active":"2023-12-19T16:42:28.955715+0000","last_peered":"2023-12-19T16:42:28.955715+0000","last_clean":"2023-12-19T16:42:28.955715+0000","last_became_active":"2023-12-19T16:42:27.800338+0000","last_became_peered":"2023-12-19T16:42:27.800338+0000","last_unstale":"2023-12-19T16:42:28.955715+0000","last_undegraded":"2023-12-19T16:42:28.955715+0000","last_fullsized":"2023-12-19T16:42:28.955715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:29.376069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654248+0000","last_change":"2023-12-19T16:42:27.324962+0000","last_active":"2023-12-19T16:42:29.654248+0000","last_peered":"2023-12-19T16:42:29.654248+0000","last_clean":"2023-12-19T16:42:29.654248+0000","last_became_active":"2023-12-19T16:42:27.324627+0000","last_became_peered":"2023-12-19T16:42:27.324627+0000","last_unstale":"2023-12-19T16:42:29.654248+0000","last_undegraded":"2023-12-19T16:42:29.654248+0000","last_fullsized":"2023-12-19T16:42:29.654248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790825+0000","last_change":"2023-12-19T16:42:27.829894+0000","last_active":"2023-12-19T16:42:28.790825+0000","last_peered":"2023-12-19T16:42:28.790825+0000","last_clean":"2023-12-19T16:42:28.790825+0000","last_became_active":"2023-12-19T16:42:27.829581+0000","last_became_peered":"2023-12-19T16:42:27.829581+0000","last_unstale":"2023-12-19T16:42:28.790825+0000","last_undegraded":"2023-12-19T16:42:28.790825+0000","last_fullsized":"2023-12-19T16:42:28.790825+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:00.499277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790905+0000","last_change":"2023-12-19T16:42:27.322335+0000","last_active":"2023-12-19T16:42:28.790905+0000","last_peered":"2023-12-19T16:42:28.790905+0000","last_clean":"2023-12-19T16:42:28.790905+0000","last_became_active":"2023-12-19T16:42:27.319501+0000","last_became_peered":"2023-12-19T16:42:27.319501+0000","last_unstale":"2023-12-19T16:42:28.790905+0000","last_undegraded":"2023-12-19T16:42:28.790905+0000","last_fullsized":"2023-12-19T16:42:28.790905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648548+0000","last_change":"2023-12-19T16:42:27.798069+0000","last_active":"2023-12-19T16:42:28.648548+0000","last_peered":"2023-12-19T16:42:28.648548+0000","last_clean":"2023-12-19T16:42:28.648548+0000","last_became_active":"2023-12-19T16:42:27.795622+0000","last_became_peered":"2023-12-19T16:42:27.795622+0000","last_unstale":"2023-12-19T16:42:28.648548+0000","last_undegraded":"2023-12-19T16:42:28.648548+0000","last_fullsized":"2023-12-19T16:42:28.648548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:15:14.536450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955857+0000","last_change":"2023-12-19T16:42:27.310215+0000","last_active":"2023-12-19T16:42:28.955857+0000","last_peered":"2023-12-19T16:42:28.955857+0000","last_clean":"2023-12-19T16:42:28.955857+0000","last_became_active":"2023-12-19T16:42:27.308663+0000","last_became_peered":"2023-12-19T16:42:27.308663+0000","last_unstale":"2023-12-19T16:42:28.955857+0000","last_undegraded":"2023-12-19T16:42:28.955857+0000","last_fullsized":"2023-12-19T16:42:28.955857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:41.116483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790149+0000","last_change":"2023-12-19T16:42:27.825472+0000","last_active":"2023-12-19T16:42:28.790149+0000","last_peered":"2023-12-19T16:42:28.790149+0000","last_clean":"2023-12-19T16:42:28.790149+0000","last_became_active":"2023-12-19T16:42:27.825201+0000","last_became_peered":"2023-12-19T16:42:27.825201+0000","last_unstale":"2023-12-19T16:42:28.790149+0000","last_undegraded":"2023-12-19T16:42:28.790149+0000","last_fullsized":"2023-12-19T16:42:28.790149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:42.122812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654201+0000","last_change":"2023-12-19T16:42:27.321076+0000","last_active":"2023-12-19T16:42:29.654201+0000","last_peered":"2023-12-19T16:42:29.654201+0000","last_clean":"2023-12-19T16:42:29.654201+0000","last_became_active":"2023-12-19T16:42:27.320592+0000","last_became_peered":"2023-12-19T16:42:27.320592+0000","last_unstale":"2023-12-19T16:42:29.654201+0000","last_undegraded":"2023-12-19T16:42:29.654201+0000","last_fullsized":"2023-12-19T16:42:29.654201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.708383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658212+0000","last_change":"2023-12-19T16:42:27.818972+0000","last_active":"2023-12-19T16:42:29.658212+0000","last_peered":"2023-12-19T16:42:29.658212+0000","last_clean":"2023-12-19T16:42:29.658212+0000","last_became_active":"2023-12-19T16:42:27.818689+0000","last_became_peered":"2023-12-19T16:42:27.818689+0000","last_unstale":"2023-12-19T16:42:29.658212+0000","last_undegraded":"2023-12-19T16:42:29.658212+0000","last_fullsized":"2023-12-19T16:42:29.658212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:48:49.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648499+0000","last_change":"2023-12-19T16:42:27.306416+0000","last_active":"2023-12-19T16:42:28.648499+0000","last_peered":"2023-12-19T16:42:28.648499+0000","last_clean":"2023-12-19T16:42:28.648499+0000","last_became_active":"2023-12-19T16:42:27.306033+0000","last_became_peered":"2023-12-19T16:42:27.306033+0000","last_unstale":"2023-12-19T16:42:28.648499+0000","last_undegraded":"2023-12-19T16:42:28.648499+0000","last_fullsized":"2023-12-19T16:42:28.648499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:29.172833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648461+0000","last_change":"2023-12-19T16:42:27.809290+0000","last_active":"2023-12-19T16:42:28.648461+0000","last_peered":"2023-12-19T16:42:28.648461+0000","last_clean":"2023-12-19T16:42:28.648461+0000","last_became_active":"2023-12-19T16:42:27.806028+0000","last_became_peered":"2023-12-19T16:42:27.806028+0000","last_unstale":"2023-12-19T16:42:28.648461+0000","last_undegraded":"2023-12-19T16:42:28.648461+0000","last_fullsized":"2023-12-19T16:42:28.648461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:48:43.490883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.791044+0000","last_change":"2023-12-19T16:42:27.317606+0000","last_active":"2023-12-19T16:42:28.791044+0000","last_peered":"2023-12-19T16:42:28.791044+0000","last_clean":"2023-12-19T16:42:28.791044+0000","last_became_active":"2023-12-19T16:42:27.316871+0000","last_became_peered":"2023-12-19T16:42:27.316871+0000","last_unstale":"2023-12-19T16:42:28.791044+0000","last_undegraded":"2023-12-19T16:42:28.791044+0000","last_fullsized":"2023-12-19T16:42:28.791044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.708383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658130+0000","last_change":"2023-12-19T16:42:27.826100+0000","last_active":"2023-12-19T16:42:29.658130+0000","last_peered":"2023-12-19T16:42:29.658130+0000","last_clean":"2023-12-19T16:42:29.658130+0000","last_became_active":"2023-12-19T16:42:27.825764+0000","last_became_peered":"2023-12-19T16:42:27.825764+0000","last_unstale":"2023-12-19T16:42:29.658130+0000","last_undegraded":"2023-12-19T16:42:29.658130+0000","last_fullsized":"2023-12-19T16:42:29.658130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:00.499277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654128+0000","last_change":"2023-12-19T16:42:27.327321+0000","last_active":"2023-12-19T16:42:29.654128+0000","last_peered":"2023-12-19T16:42:29.654128+0000","last_clean":"2023-12-19T16:42:29.654128+0000","last_became_active":"2023-12-19T16:42:27.327010+0000","last_became_peered":"2023-12-19T16:42:27.327010+0000","last_unstale":"2023-12-19T16:42:29.654128+0000","last_undegraded":"2023-12-19T16:42:29.654128+0000","last_fullsized":"2023-12-19T16:42:29.654128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654100+0000","last_change":"2023-12-19T16:42:27.802521+0000","last_active":"2023-12-19T16:42:29.654100+0000","last_peered":"2023-12-19T16:42:29.654100+0000","last_clean":"2023-12-19T16:42:29.654100+0000","last_became_active":"2023-12-19T16:42:27.801786+0000","last_became_peered":"2023-12-19T16:42:27.801786+0000","last_unstale":"2023-12-19T16:42:29.654100+0000","last_undegraded":"2023-12-19T16:42:29.654100+0000","last_fullsized":"2023-12-19T16:42:29.654100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:16.399823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784006+0000","last_change":"2023-12-19T16:42:27.311025+0000","last_active":"2023-12-19T16:42:28.784006+0000","last_peered":"2023-12-19T16:42:28.784006+0000","last_clean":"2023-12-19T16:42:28.784006+0000","last_became_active":"2023-12-19T16:42:27.310676+0000","last_became_peered":"2023-12-19T16:42:27.310676+0000","last_unstale":"2023-12-19T16:42:28.784006+0000","last_undegraded":"2023-12-19T16:42:28.784006+0000","last_fullsized":"2023-12-19T16:42:28.784006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658095+0000","last_change":"2023-12-19T16:42:27.821468+0000","last_active":"2023-12-19T16:42:29.658095+0000","last_peered":"2023-12-19T16:42:29.658095+0000","last_clean":"2023-12-19T16:42:29.658095+0000","last_became_active":"2023-12-19T16:42:27.821113+0000","last_became_peered":"2023-12-19T16:42:27.821113+0000","last_unstale":"2023-12-19T16:42:29.658095+0000","last_undegraded":"2023-12-19T16:42:29.658095+0000","last_fullsized":"2023-12-19T16:42:29.658095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:03.390272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790606+0000","last_change":"2023-12-19T16:42:27.323313+0000","last_active":"2023-12-19T16:42:28.790606+0000","last_peered":"2023-12-19T16:42:28.790606+0000","last_clean":"2023-12-19T16:42:28.790606+0000","last_became_active":"2023-12-19T16:42:27.319464+0000","last_became_peered":"2023-12-19T16:42:27.319464+0000","last_unstale":"2023-12-19T16:42:28.790606+0000","last_undegraded":"2023-12-19T16:42:28.790606+0000","last_fullsized":"2023-12-19T16:42:28.790606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:22:50.990280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658306+0000","last_change":"2023-12-19T16:42:27.821587+0000","last_active":"2023-12-19T16:42:29.658306+0000","last_peered":"2023-12-19T16:42:29.658306+0000","last_clean":"2023-12-19T16:42:29.658306+0000","last_became_active":"2023-12-19T16:42:27.821251+0000","last_became_peered":"2023-12-19T16:42:27.821251+0000","last_unstale":"2023-12-19T16:42:29.658306+0000","last_undegraded":"2023-12-19T16:42:29.658306+0000","last_fullsized":"2023-12-19T16:42:29.658306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:18.209972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784184+0000","last_change":"2023-12-19T16:42:27.319966+0000","last_active":"2023-12-19T16:42:28.784184+0000","last_peered":"2023-12-19T16:42:28.784184+0000","last_clean":"2023-12-19T16:42:28.784184+0000","last_became_active":"2023-12-19T16:42:27.319141+0000","last_became_peered":"2023-12-19T16:42:27.319141+0000","last_unstale":"2023-12-19T16:42:28.784184+0000","last_undegraded":"2023-12-19T16:42:28.784184+0000","last_fullsized":"2023-12-19T16:42:28.784184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658377+0000","last_change":"2023-12-19T16:42:27.824936+0000","last_active":"2023-12-19T16:42:29.658377+0000","last_peered":"2023-12-19T16:42:29.658377+0000","last_clean":"2023-12-19T16:42:29.658377+0000","last_became_active":"2023-12-19T16:42:27.824643+0000","last_became_peered":"2023-12-19T16:42:27.824643+0000","last_unstale":"2023-12-19T16:42:29.658377+0000","last_undegraded":"2023-12-19T16:42:29.658377+0000","last_fullsized":"2023-12-19T16:42:29.658377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:23.006980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955939+0000","last_change":"2023-12-19T16:42:27.332207+0000","last_active":"2023-12-19T16:42:28.955939+0000","last_peered":"2023-12-19T16:42:28.955939+0000","last_clean":"2023-12-19T16:42:28.955939+0000","last_became_active":"2023-12-19T16:42:27.331920+0000","last_became_peered":"2023-12-19T16:42:27.331920+0000","last_unstale":"2023-12-19T16:42:28.955939+0000","last_undegraded":"2023-12-19T16:42:28.955939+0000","last_fullsized":"2023-12-19T16:42:28.955939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.708383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790779+0000","last_change":"2023-12-19T16:42:27.803564+0000","last_active":"2023-12-19T16:42:28.790779+0000","last_peered":"2023-12-19T16:42:28.790779+0000","last_clean":"2023-12-19T16:42:28.790779+0000","last_became_active":"2023-12-19T16:42:27.803291+0000","last_became_peered":"2023-12-19T16:42:27.803291+0000","last_unstale":"2023-12-19T16:42:28.790779+0000","last_undegraded":"2023-12-19T16:42:28.790779+0000","last_fullsized":"2023-12-19T16:42:28.790779+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:49.764257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784115+0000","last_change":"2023-12-19T16:42:27.316666+0000","last_active":"2023-12-19T16:42:28.784115+0000","last_peered":"2023-12-19T16:42:28.784115+0000","last_clean":"2023-12-19T16:42:28.784115+0000","last_became_active":"2023-12-19T16:42:27.316322+0000","last_became_peered":"2023-12-19T16:42:27.316322+0000","last_unstale":"2023-12-19T16:42:28.784115+0000","last_undegraded":"2023-12-19T16:42:28.784115+0000","last_fullsized":"2023-12-19T16:42:28.784115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658228+0000","last_change":"2023-12-19T16:42:27.828580+0000","last_active":"2023-12-19T16:42:29.658228+0000","last_peered":"2023-12-19T16:42:29.658228+0000","last_clean":"2023-12-19T16:42:29.658228+0000","last_became_active":"2023-12-19T16:42:27.828263+0000","last_became_peered":"2023-12-19T16:42:27.828263+0000","last_unstale":"2023-12-19T16:42:29.658228+0000","last_undegraded":"2023-12-19T16:42:29.658228+0000","last_fullsized":"2023-12-19T16:42:29.658228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:15.012739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":164,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.808382+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:42:27.808382+0000","last_peered":"2023-12-19T16:42:27.808382+0000","last_clean":"2023-12-19T16:42:27.808382+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:42:27.808382+0000","last_undegraded":"2023-12-19T16:42:27.808382+0000","last_fullsized":"2023-12-19T16:42:27.808382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790309+0000","last_change":"2023-12-19T16:42:27.322949+0000","last_active":"2023-12-19T16:42:28.790309+0000","last_peered":"2023-12-19T16:42:28.790309+0000","last_clean":"2023-12-19T16:42:28.790309+0000","last_became_active":"2023-12-19T16:42:27.321633+0000","last_became_peered":"2023-12-19T16:42:27.321633+0000","last_unstale":"2023-12-19T16:42:28.790309+0000","last_undegraded":"2023-12-19T16:42:28.790309+0000","last_fullsized":"2023-12-19T16:42:28.790309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651957+0000","last_change":"2023-12-19T16:42:27.798689+0000","last_active":"2023-12-19T16:42:29.651957+0000","last_peered":"2023-12-19T16:42:29.651957+0000","last_clean":"2023-12-19T16:42:29.651957+0000","last_became_active":"2023-12-19T16:42:27.798430+0000","last_became_peered":"2023-12-19T16:42:27.798430+0000","last_unstale":"2023-12-19T16:42:29.651957+0000","last_undegraded":"2023-12-19T16:42:29.651957+0000","last_fullsized":"2023-12-19T16:42:29.651957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:16.399823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658033+0000","last_change":"2023-12-19T16:42:27.326882+0000","last_active":"2023-12-19T16:42:29.658033+0000","last_peered":"2023-12-19T16:42:29.658033+0000","last_clean":"2023-12-19T16:42:29.658033+0000","last_became_active":"2023-12-19T16:42:27.326616+0000","last_became_peered":"2023-12-19T16:42:27.326616+0000","last_unstale":"2023-12-19T16:42:29.658033+0000","last_undegraded":"2023-12-19T16:42:29.658033+0000","last_fullsized":"2023-12-19T16:42:29.658033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:36.280588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648432+0000","last_change":"2023-12-19T16:42:27.816279+0000","last_active":"2023-12-19T16:42:28.648432+0000","last_peered":"2023-12-19T16:42:28.648432+0000","last_clean":"2023-12-19T16:42:28.648432+0000","last_became_active":"2023-12-19T16:42:27.815940+0000","last_became_peered":"2023-12-19T16:42:27.815940+0000","last_unstale":"2023-12-19T16:42:28.648432+0000","last_undegraded":"2023-12-19T16:42:28.648432+0000","last_fullsized":"2023-12-19T16:42:28.648432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:51:00.190122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784142+0000","last_change":"2023-12-19T16:42:27.317414+0000","last_active":"2023-12-19T16:42:28.784142+0000","last_peered":"2023-12-19T16:42:28.784142+0000","last_clean":"2023-12-19T16:42:28.784142+0000","last_became_active":"2023-12-19T16:42:27.317065+0000","last_became_peered":"2023-12-19T16:42:27.317065+0000","last_unstale":"2023-12-19T16:42:28.784142+0000","last_undegraded":"2023-12-19T16:42:28.784142+0000","last_fullsized":"2023-12-19T16:42:28.784142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955832+0000","last_change":"2023-12-19T16:42:27.794867+0000","last_active":"2023-12-19T16:42:28.955832+0000","last_peered":"2023-12-19T16:42:28.955832+0000","last_clean":"2023-12-19T16:42:28.955832+0000","last_became_active":"2023-12-19T16:42:27.793788+0000","last_became_peered":"2023-12-19T16:42:27.793788+0000","last_unstale":"2023-12-19T16:42:28.955832+0000","last_undegraded":"2023-12-19T16:42:28.955832+0000","last_fullsized":"2023-12-19T16:42:28.955832+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:58:03.549430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.791093+0000","last_change":"2023-12-19T16:42:27.312696+0000","last_active":"2023-12-19T16:42:28.791093+0000","last_peered":"2023-12-19T16:42:28.791093+0000","last_clean":"2023-12-19T16:42:28.791093+0000","last_became_active":"2023-12-19T16:42:27.312406+0000","last_became_peered":"2023-12-19T16:42:27.312406+0000","last_unstale":"2023-12-19T16:42:28.791093+0000","last_undegraded":"2023-12-19T16:42:28.791093+0000","last_fullsized":"2023-12-19T16:42:28.791093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:41.116483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789955+0000","last_change":"2023-12-19T16:42:27.813887+0000","last_active":"2023-12-19T16:42:28.789955+0000","last_peered":"2023-12-19T16:42:28.789955+0000","last_clean":"2023-12-19T16:42:28.789955+0000","last_became_active":"2023-12-19T16:42:27.813476+0000","last_became_peered":"2023-12-19T16:42:27.813476+0000","last_unstale":"2023-12-19T16:42:28.789955+0000","last_undegraded":"2023-12-19T16:42:28.789955+0000","last_fullsized":"2023-12-19T16:42:28.789955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:36.975084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955856+0000","last_change":"2023-12-19T16:42:27.302950+0000","last_active":"2023-12-19T16:42:28.955856+0000","last_peered":"2023-12-19T16:42:28.955856+0000","last_clean":"2023-12-19T16:42:28.955856+0000","last_became_active":"2023-12-19T16:42:27.302614+0000","last_became_peered":"2023-12-19T16:42:27.302614+0000","last_unstale":"2023-12-19T16:42:28.955856+0000","last_undegraded":"2023-12-19T16:42:28.955856+0000","last_fullsized":"2023-12-19T16:42:28.955856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790926+0000","last_change":"2023-12-19T16:42:27.803672+0000","last_active":"2023-12-19T16:42:28.790926+0000","last_peered":"2023-12-19T16:42:28.790926+0000","last_clean":"2023-12-19T16:42:28.790926+0000","last_became_active":"2023-12-19T16:42:27.803402+0000","last_became_peered":"2023-12-19T16:42:27.803402+0000","last_unstale":"2023-12-19T16:42:28.790926+0000","last_undegraded":"2023-12-19T16:42:28.790926+0000","last_fullsized":"2023-12-19T16:42:28.790926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:48:49.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658289+0000","last_change":"2023-12-19T16:42:27.330785+0000","last_active":"2023-12-19T16:42:29.658289+0000","last_peered":"2023-12-19T16:42:29.658289+0000","last_clean":"2023-12-19T16:42:29.658289+0000","last_became_active":"2023-12-19T16:42:27.329823+0000","last_became_peered":"2023-12-19T16:42:27.329823+0000","last_unstale":"2023-12-19T16:42:29.658289+0000","last_undegraded":"2023-12-19T16:42:29.658289+0000","last_fullsized":"2023-12-19T16:42:29.658289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955776+0000","last_change":"2023-12-19T16:42:27.794649+0000","last_active":"2023-12-19T16:42:28.955776+0000","last_peered":"2023-12-19T16:42:28.955776+0000","last_clean":"2023-12-19T16:42:28.955776+0000","last_became_active":"2023-12-19T16:42:27.793710+0000","last_became_peered":"2023-12-19T16:42:27.793710+0000","last_unstale":"2023-12-19T16:42:28.955776+0000","last_undegraded":"2023-12-19T16:42:28.955776+0000","last_fullsized":"2023-12-19T16:42:28.955776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:16.399823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652133+0000","last_change":"2023-12-19T16:42:27.321060+0000","last_active":"2023-12-19T16:42:29.652133+0000","last_peered":"2023-12-19T16:42:29.652133+0000","last_clean":"2023-12-19T16:42:29.652133+0000","last_became_active":"2023-12-19T16:42:27.320705+0000","last_became_peered":"2023-12-19T16:42:27.320705+0000","last_unstale":"2023-12-19T16:42:29.652133+0000","last_undegraded":"2023-12-19T16:42:29.652133+0000","last_fullsized":"2023-12-19T16:42:29.652133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:35.968756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652184+0000","last_change":"2023-12-19T16:42:27.818162+0000","last_active":"2023-12-19T16:42:29.652184+0000","last_peered":"2023-12-19T16:42:29.652184+0000","last_clean":"2023-12-19T16:42:29.652184+0000","last_became_active":"2023-12-19T16:42:27.817894+0000","last_became_peered":"2023-12-19T16:42:27.817894+0000","last_unstale":"2023-12-19T16:42:29.652184+0000","last_undegraded":"2023-12-19T16:42:29.652184+0000","last_fullsized":"2023-12-19T16:42:29.652184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:29.376069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654257+0000","last_change":"2023-12-19T16:42:27.330277+0000","last_active":"2023-12-19T16:42:29.654257+0000","last_peered":"2023-12-19T16:42:29.654257+0000","last_clean":"2023-12-19T16:42:29.654257+0000","last_became_active":"2023-12-19T16:42:27.330006+0000","last_became_peered":"2023-12-19T16:42:27.330006+0000","last_unstale":"2023-12-19T16:42:29.654257+0000","last_undegraded":"2023-12-19T16:42:29.654257+0000","last_fullsized":"2023-12-19T16:42:29.654257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648518+0000","last_change":"2023-12-19T16:42:27.791761+0000","last_active":"2023-12-19T16:42:28.648518+0000","last_peered":"2023-12-19T16:42:28.648518+0000","last_clean":"2023-12-19T16:42:28.648518+0000","last_became_active":"2023-12-19T16:42:27.791490+0000","last_became_peered":"2023-12-19T16:42:27.791490+0000","last_unstale":"2023-12-19T16:42:28.648518+0000","last_undegraded":"2023-12-19T16:42:28.648518+0000","last_fullsized":"2023-12-19T16:42:28.648518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:48:40.695901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658009+0000","last_change":"2023-12-19T16:42:27.326363+0000","last_active":"2023-12-19T16:42:29.658009+0000","last_peered":"2023-12-19T16:42:29.658009+0000","last_clean":"2023-12-19T16:42:29.658009+0000","last_became_active":"2023-12-19T16:42:27.326046+0000","last_became_peered":"2023-12-19T16:42:27.326046+0000","last_unstale":"2023-12-19T16:42:29.658009+0000","last_undegraded":"2023-12-19T16:42:29.658009+0000","last_fullsized":"2023-12-19T16:42:29.658009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:41.116483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789733+0000","last_change":"2023-12-19T16:42:27.808612+0000","last_active":"2023-12-19T16:42:28.789733+0000","last_peered":"2023-12-19T16:42:28.789733+0000","last_clean":"2023-12-19T16:42:28.789733+0000","last_became_active":"2023-12-19T16:42:27.808219+0000","last_became_peered":"2023-12-19T16:42:27.808219+0000","last_unstale":"2023-12-19T16:42:28.789733+0000","last_undegraded":"2023-12-19T16:42:28.789733+0000","last_fullsized":"2023-12-19T16:42:28.789733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:37.286916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955811+0000","last_change":"2023-12-19T16:42:27.308462+0000","last_active":"2023-12-19T16:42:28.955811+0000","last_peered":"2023-12-19T16:42:28.955811+0000","last_clean":"2023-12-19T16:42:28.955811+0000","last_became_active":"2023-12-19T16:42:27.307879+0000","last_became_peered":"2023-12-19T16:42:27.307879+0000","last_unstale":"2023-12-19T16:42:28.955811+0000","last_undegraded":"2023-12-19T16:42:28.955811+0000","last_fullsized":"2023-12-19T16:42:28.955811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651875+0000","last_change":"2023-12-19T16:42:27.798674+0000","last_active":"2023-12-19T16:42:29.651875+0000","last_peered":"2023-12-19T16:42:29.651875+0000","last_clean":"2023-12-19T16:42:29.651875+0000","last_became_active":"2023-12-19T16:42:27.798442+0000","last_became_peered":"2023-12-19T16:42:27.798442+0000","last_unstale":"2023-12-19T16:42:29.651875+0000","last_undegraded":"2023-12-19T16:42:29.651875+0000","last_fullsized":"2023-12-19T16:42:29.651875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:01.122709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955925+0000","last_change":"2023-12-19T16:42:27.310942+0000","last_active":"2023-12-19T16:42:28.955925+0000","last_peered":"2023-12-19T16:42:28.955925+0000","last_clean":"2023-12-19T16:42:28.955925+0000","last_became_active":"2023-12-19T16:42:27.310605+0000","last_became_peered":"2023-12-19T16:42:27.310605+0000","last_unstale":"2023-12-19T16:42:28.955925+0000","last_undegraded":"2023-12-19T16:42:28.955925+0000","last_fullsized":"2023-12-19T16:42:28.955925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652009+0000","last_change":"2023-12-19T16:42:27.802676+0000","last_active":"2023-12-19T16:42:29.652009+0000","last_peered":"2023-12-19T16:42:29.652009+0000","last_clean":"2023-12-19T16:42:29.652009+0000","last_became_active":"2023-12-19T16:42:27.801627+0000","last_became_peered":"2023-12-19T16:42:27.801627+0000","last_unstale":"2023-12-19T16:42:29.652009+0000","last_undegraded":"2023-12-19T16:42:29.652009+0000","last_fullsized":"2023-12-19T16:42:29.652009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:30.545767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658151+0000","last_change":"2023-12-19T16:42:27.334084+0000","last_active":"2023-12-19T16:42:29.658151+0000","last_peered":"2023-12-19T16:42:29.658151+0000","last_clean":"2023-12-19T16:42:29.658151+0000","last_became_active":"2023-12-19T16:42:27.333748+0000","last_became_peered":"2023-12-19T16:42:27.333748+0000","last_unstale":"2023-12-19T16:42:29.658151+0000","last_undegraded":"2023-12-19T16:42:29.658151+0000","last_fullsized":"2023-12-19T16:42:29.658151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654146+0000","last_change":"2023-12-19T16:42:27.817078+0000","last_active":"2023-12-19T16:42:29.654146+0000","last_peered":"2023-12-19T16:42:29.654146+0000","last_clean":"2023-12-19T16:42:29.654146+0000","last_became_active":"2023-12-19T16:42:27.816835+0000","last_became_peered":"2023-12-19T16:42:27.816835+0000","last_unstale":"2023-12-19T16:42:29.654146+0000","last_undegraded":"2023-12-19T16:42:29.654146+0000","last_fullsized":"2023-12-19T16:42:29.654146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790847+0000","last_change":"2023-12-19T16:42:27.820348+0000","last_active":"2023-12-19T16:42:28.790847+0000","last_peered":"2023-12-19T16:42:28.790847+0000","last_clean":"2023-12-19T16:42:28.790847+0000","last_became_active":"2023-12-19T16:42:27.820051+0000","last_became_peered":"2023-12-19T16:42:27.820051+0000","last_unstale":"2023-12-19T16:42:28.790847+0000","last_undegraded":"2023-12-19T16:42:28.790847+0000","last_fullsized":"2023-12-19T16:42:28.790847+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:23.006980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658192+0000","last_change":"2023-12-19T16:42:27.328852+0000","last_active":"2023-12-19T16:42:29.658192+0000","last_peered":"2023-12-19T16:42:29.658192+0000","last_clean":"2023-12-19T16:42:29.658192+0000","last_became_active":"2023-12-19T16:42:27.328521+0000","last_became_peered":"2023-12-19T16:42:27.328521+0000","last_unstale":"2023-12-19T16:42:29.658192+0000","last_undegraded":"2023-12-19T16:42:29.658192+0000","last_fullsized":"2023-12-19T16:42:29.658192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:35.968756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790575+0000","last_change":"2023-12-19T16:42:27.820348+0000","last_active":"2023-12-19T16:42:28.790575+0000","last_peered":"2023-12-19T16:42:28.790575+0000","last_clean":"2023-12-19T16:42:28.790575+0000","last_became_active":"2023-12-19T16:42:27.820018+0000","last_became_peered":"2023-12-19T16:42:27.820018+0000","last_unstale":"2023-12-19T16:42:28.790575+0000","last_undegraded":"2023-12-19T16:42:28.790575+0000","last_fullsized":"2023-12-19T16:42:28.790575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658245+0000","last_change":"2023-12-19T16:42:27.331867+0000","last_active":"2023-12-19T16:42:29.658245+0000","last_peered":"2023-12-19T16:42:29.658245+0000","last_clean":"2023-12-19T16:42:29.658245+0000","last_became_active":"2023-12-19T16:42:27.331354+0000","last_became_peered":"2023-12-19T16:42:27.331354+0000","last_unstale":"2023-12-19T16:42:29.658245+0000","last_undegraded":"2023-12-19T16:42:29.658245+0000","last_fullsized":"2023-12-19T16:42:29.658245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:47:06.561268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648540+0000","last_change":"2023-12-19T16:42:27.800529+0000","last_active":"2023-12-19T16:42:28.648540+0000","last_peered":"2023-12-19T16:42:28.648540+0000","last_clean":"2023-12-19T16:42:28.648540+0000","last_became_active":"2023-12-19T16:42:27.800250+0000","last_became_peered":"2023-12-19T16:42:27.800250+0000","last_unstale":"2023-12-19T16:42:28.648540+0000","last_undegraded":"2023-12-19T16:42:28.648540+0000","last_fullsized":"2023-12-19T16:42:28.648540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:27.376917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652139+0000","last_change":"2023-12-19T16:42:27.314367+0000","last_active":"2023-12-19T16:42:29.652139+0000","last_peered":"2023-12-19T16:42:29.652139+0000","last_clean":"2023-12-19T16:42:29.652139+0000","last_became_active":"2023-12-19T16:42:27.314128+0000","last_became_peered":"2023-12-19T16:42:27.314128+0000","last_unstale":"2023-12-19T16:42:29.652139+0000","last_undegraded":"2023-12-19T16:42:29.652139+0000","last_fullsized":"2023-12-19T16:42:29.652139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:34:59.521223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652110+0000","last_change":"2023-12-19T16:42:27.804285+0000","last_active":"2023-12-19T16:42:29.652110+0000","last_peered":"2023-12-19T16:42:29.652110+0000","last_clean":"2023-12-19T16:42:29.652110+0000","last_became_active":"2023-12-19T16:42:27.804027+0000","last_became_peered":"2023-12-19T16:42:27.804027+0000","last_unstale":"2023-12-19T16:42:29.652110+0000","last_undegraded":"2023-12-19T16:42:29.652110+0000","last_fullsized":"2023-12-19T16:42:29.652110+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:53.498191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648557+0000","last_change":"2023-12-19T16:42:27.325512+0000","last_active":"2023-12-19T16:42:28.648557+0000","last_peered":"2023-12-19T16:42:28.648557+0000","last_clean":"2023-12-19T16:42:28.648557+0000","last_became_active":"2023-12-19T16:42:27.325175+0000","last_became_peered":"2023-12-19T16:42:27.325175+0000","last_unstale":"2023-12-19T16:42:28.648557+0000","last_undegraded":"2023-12-19T16:42:28.648557+0000","last_fullsized":"2023-12-19T16:42:28.648557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:31:30.386546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":831488,"data_stored":671861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6389999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":82475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.5499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":81001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":81037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]}]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":245760,"data_stored":82475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.821}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0179999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5350000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.956}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9279999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31276,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340564,"statfs":{"total":96636764160,"available":96604737536,"internally_reserved":0,"allocated":233472,"data_stored":80911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5150000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.7590000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8940000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1960000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.165}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.234}]}]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":81071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3050000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.992}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.133}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1949999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1269999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.093}]}]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":839680,"data_stored":671917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:42:38.319 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:42:38.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.496+0000 7f7aa5011640 1 Processor -- start 2023-12-19T16:42:38.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.496+0000 7f7aa5011640 1 -- start start 2023-12-19T16:42:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.496+0000 7f7aa5011640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7aa0155030 0x7f7aa015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.496+0000 7f7aa5011640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7aa00ce740 0x7f7aa00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.496+0000 7f7aa5011640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa006c970 con 0x7f7aa015f370 2023-12-19T16:42:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.496+0000 7f7aa5011640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa006c790 con 0x7f7aa0155030 2023-12-19T16:42:38.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.496+0000 7f7aa5011640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa00cf0f0 con 0x7f7aa00ce740 2023-12-19T16:42:38.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.496+0000 7f7a9f59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7aa00ce740 0x7f7aa00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.496+0000 7f7a9ed9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7aa0155030 0x7f7aa015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.496+0000 7f7a9f59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7aa00ce740 0x7f7aa00cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:39870/0 (socket says 172.21.15.26:39870) 2023-12-19T16:42:38.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.496+0000 7f7a97fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f7aa015f370 0x7f7aa006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:60940/0 (socket says 172.21.15.26:60940) 2023-12-19T16:42:38.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.496+0000 7f7a9f59e640 1 -- 172.21.15.26:0/1093867246 learned_addr learned my addr 172.21.15.26:0/1093867246 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:38.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.500+0000 7f7a9ed9d640 1 -- 172.21.15.26:0/1093867246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7aa00ce740 msgr2=0x7f7aa00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:38.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.500+0000 7f7a9ed9d640 1 --2- 172.21.15.26:0/1093867246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7aa00ce740 0x7f7aa00cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.500+0000 7f7a9ed9d640 1 -- 172.21.15.26:0/1093867246 >> v1:172.21.15.160:6789/0 conn(0x7f7aa015f370 legacy=0x7f7aa006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.500+0000 7f7a9ed9d640 1 -- 172.21.15.26:0/1093867246 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa015b1f0 con 0x7f7aa0155030 2023-12-19T16:42:38.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.500+0000 7f7a9f59e640 1 --2- 172.21.15.26:0/1093867246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7aa00ce740 0x7f7aa00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:38.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.500+0000 7f7a9ed9d640 1 --2- 172.21.15.26:0/1093867246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7aa0155030 0x7f7aa015bcd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7a900097c0 tx=0x7f7a9001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4f4115af372003b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:38.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.500+0000 7f7a9e59c640 1 -- 172.21.15.26:0/1093867246 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900ab650 con 0x7f7aa0155030 2023-12-19T16:42:38.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.500+0000 7f7a9e59c640 1 -- 172.21.15.26:0/1093867246 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a900ab7b0 con 0x7f7aa0155030 2023-12-19T16:42:38.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.500+0000 7f7a9e59c640 1 -- 172.21.15.26:0/1093867246 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900abb40 con 0x7f7aa0155030 2023-12-19T16:42:38.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.500+0000 7f7aa5011640 1 -- 172.21.15.26:0/1093867246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7aa0155030 msgr2=0x7f7aa015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.500+0000 7f7aa5011640 1 --2- 172.21.15.26:0/1093867246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7aa0155030 0x7f7aa015bcd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7a900097c0 tx=0x7f7a9001c450 comp rx=0 tx=0).stop 2023-12-19T16:42:38.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 -- 172.21.15.26:0/1093867246 shutdown_connections 2023-12-19T16:42:38.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 --2- 172.21.15.26:0/1093867246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7aa00ce740 0x7f7aa00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 --2- 172.21.15.26:0/1093867246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7aa0155030 0x7f7aa015bcd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 -- 172.21.15.26:0/1093867246 >> 172.21.15.26:0/1093867246 conn(0x7f7aa00658e0 msgr2=0x7f7aa0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:38.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 -- 172.21.15.26:0/1093867246 shutdown_connections 2023-12-19T16:42:38.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 -- 172.21.15.26:0/1093867246 wait complete. 2023-12-19T16:42:38.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 Processor -- start 2023-12-19T16:42:38.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 -- start start 2023-12-19T16:42:38.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00ce740 0x7f7aa00b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7aa0155030 0x7f7aa01537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7aa015f370 0x7f7aa0159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa014d430 con 0x7f7aa00ce740 2023-12-19T16:42:38.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa01462c0 con 0x7f7aa0155030 2023-12-19T16:42:38.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7aa5011640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa0152a20 con 0x7f7aa015f370 2023-12-19T16:42:38.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7a9ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00ce740 0x7f7aa00b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7a9ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00ce740 0x7f7aa00b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33404/0 (socket says 172.21.15.26:33404) 2023-12-19T16:42:38.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7a9ed9d640 1 -- 172.21.15.26:0/63467782 learned_addr learned my addr 172.21.15.26:0/63467782 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:38.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7a9f59e640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7aa015f370 0x7f7aa0159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7a97fff640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7aa0155030 0x7f7aa01537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7a9ed9d640 1 -- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7aa015f370 msgr2=0x7f7aa0159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7a9ed9d640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7aa015f370 0x7f7aa0159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7a9ed9d640 1 -- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7aa0155030 msgr2=0x7f7aa01537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7a9ed9d640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7aa0155030 0x7f7aa01537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.504+0000 7f7a9ed9d640 1 -- 172.21.15.26:0/63467782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a98009b30 con 0x7f7aa00ce740 2023-12-19T16:42:38.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.508+0000 7f7a9ed9d640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00ce740 0x7f7aa00b33e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a900ad040 tx=0x7f7a9001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:38.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.508+0000 7f7a977fe640 1 -- 172.21.15.26:0/63467782 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900b5810 con 0x7f7aa00ce740 2023-12-19T16:42:38.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.508+0000 7f7a977fe640 1 -- 172.21.15.26:0/63467782 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a900b5990 con 0x7f7aa00ce740 2023-12-19T16:42:38.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.508+0000 7f7aa5011640 1 -- 172.21.15.26:0/63467782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a90009420 con 0x7f7aa00ce740 2023-12-19T16:42:38.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.508+0000 7f7a977fe640 1 -- 172.21.15.26:0/63467782 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900b5d40 con 0x7f7aa00ce740 2023-12-19T16:42:38.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.508+0000 7f7aa5011640 1 -- 172.21.15.26:0/63467782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa0152cc0 con 0x7f7aa00ce740 2023-12-19T16:42:38.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.508+0000 7f7aa5011640 1 -- 172.21.15.26:0/63467782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa0074270 con 0x7f7aa00ce740 2023-12-19T16:42:38.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.508+0000 7f7a977fe640 1 -- 172.21.15.26:0/63467782 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7a900c83f0 con 0x7f7aa00ce740 2023-12-19T16:42:38.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.508+0000 7f7a977fe640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a80065a60 0x7f7a80067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.508+0000 7f7a977fe640 1 -- 172.21.15.26:0/63467782 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a9011e060 con 0x7f7aa00ce740 2023-12-19T16:42:38.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.508+0000 7f7a97fff640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a80065a60 0x7f7a80067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.512+0000 7f7a97fff640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a80065a60 0x7f7a80067f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7aa014c8a0 tx=0x7f7a88007cf0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:38.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.512+0000 7f7a977fe640 1 -- 172.21.15.26:0/63467782 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7a900eb980 con 0x7f7aa00ce740 2023-12-19T16:42:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.668+0000 7f7aa5011640 1 -- 172.21.15.26:0/63467782 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7aa006cc50 con 0x7f7a80065a60 2023-12-19T16:42:38.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.672+0000 7f7a977fe640 1 -- 172.21.15.26:0/63467782 <== mgr.4106 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264447 (secure 0 0 0) 0x7f7a880074e0 con 0x7f7a80065a60 2023-12-19T16:42:38.681 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:38.683 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:42:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.676+0000 7f7aa5011640 1 -- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a80065a60 msgr2=0x7f7a80067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.676+0000 7f7aa5011640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a80065a60 0x7f7a80067f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7aa014c8a0 tx=0x7f7a88007cf0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.676+0000 7f7aa5011640 1 -- 172.21.15.26:0/63467782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00ce740 msgr2=0x7f7aa00b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.676+0000 7f7aa5011640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00ce740 0x7f7aa00b33e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7a900ad040 tx=0x7f7a9001ac90 comp rx=0 tx=0).stop 2023-12-19T16:42:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.676+0000 7f7aa5011640 1 -- 172.21.15.26:0/63467782 shutdown_connections 2023-12-19T16:42:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.676+0000 7f7aa5011640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a80065a60 0x7f7a80067f20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.676+0000 7f7aa5011640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7aa015f370 0x7f7aa0159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.676+0000 7f7aa5011640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7aa0155030 0x7f7aa01537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.676+0000 7f7aa5011640 1 --2- 172.21.15.26:0/63467782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7aa00ce740 0x7f7aa00b33e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.676+0000 7f7aa5011640 1 -- 172.21.15.26:0/63467782 >> 172.21.15.26:0/63467782 conn(0x7f7aa00658e0 msgr2=0x7f7aa015ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:38.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.676+0000 7f7aa5011640 1 -- 172.21.15.26:0/63467782 shutdown_connections 2023-12-19T16:42:38.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.676+0000 7f7aa5011640 1 -- 172.21.15.26:0/63467782 wait complete. 2023-12-19T16:42:38.702 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-12-19T16:42:37.638794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235048,"kb_used_data":3048,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739672,"statfs":{"total":773094113280,"available":772853424128,"internally_reserved":0,"allocated":3121152,"data_stored":1832748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.854877"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658321+0000","last_change":"2023-12-19T16:42:27.824402+0000","last_active":"2023-12-19T16:42:29.658321+0000","last_peered":"2023-12-19T16:42:29.658321+0000","last_clean":"2023-12-19T16:42:29.658321+0000","last_became_active":"2023-12-19T16:42:27.818342+0000","last_became_peered":"2023-12-19T16:42:27.818342+0000","last_unstale":"2023-12-19T16:42:29.658321+0000","last_undegraded":"2023-12-19T16:42:29.658321+0000","last_fullsized":"2023-12-19T16:42:29.658321+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:53.498191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.956000+0000","last_change":"2023-12-19T16:42:27.802326+0000","last_active":"2023-12-19T16:42:28.956000+0000","last_peered":"2023-12-19T16:42:28.956000+0000","last_clean":"2023-12-19T16:42:28.956000+0000","last_became_active":"2023-12-19T16:42:27.802056+0000","last_became_peered":"2023-12-19T16:42:27.802056+0000","last_unstale":"2023-12-19T16:42:28.956000+0000","last_undegraded":"2023-12-19T16:42:28.956000+0000","last_fullsized":"2023-12-19T16:42:28.956000+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:30.545767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648509+0000","last_change":"2023-12-19T16:42:27.798150+0000","last_active":"2023-12-19T16:42:28.648509+0000","last_peered":"2023-12-19T16:42:28.648509+0000","last_clean":"2023-12-19T16:42:28.648509+0000","last_became_active":"2023-12-19T16:42:27.796339+0000","last_became_peered":"2023-12-19T16:42:27.796339+0000","last_unstale":"2023-12-19T16:42:28.648509+0000","last_undegraded":"2023-12-19T16:42:28.648509+0000","last_fullsized":"2023-12-19T16:42:28.648509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:42:19.277925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790632+0000","last_change":"2023-12-19T16:42:27.814794+0000","last_active":"2023-12-19T16:42:28.790632+0000","last_peered":"2023-12-19T16:42:28.790632+0000","last_clean":"2023-12-19T16:42:28.790632+0000","last_became_active":"2023-12-19T16:42:27.814461+0000","last_became_peered":"2023-12-19T16:42:27.814461+0000","last_unstale":"2023-12-19T16:42:28.790632+0000","last_undegraded":"2023-12-19T16:42:28.790632+0000","last_fullsized":"2023-12-19T16:42:28.790632+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:18.209972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658113+0000","last_change":"2023-12-19T16:42:27.825817+0000","last_active":"2023-12-19T16:42:29.658113+0000","last_peered":"2023-12-19T16:42:29.658113+0000","last_clean":"2023-12-19T16:42:29.658113+0000","last_became_active":"2023-12-19T16:42:27.825532+0000","last_became_peered":"2023-12-19T16:42:27.825532+0000","last_unstale":"2023-12-19T16:42:29.658113+0000","last_undegraded":"2023-12-19T16:42:29.658113+0000","last_fullsized":"2023-12-19T16:42:29.658113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:49.764257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790441+0000","last_change":"2023-12-19T16:42:27.801183+0000","last_active":"2023-12-19T16:42:28.790441+0000","last_peered":"2023-12-19T16:42:28.790441+0000","last_clean":"2023-12-19T16:42:28.790441+0000","last_became_active":"2023-12-19T16:42:27.800866+0000","last_became_peered":"2023-12-19T16:42:27.800866+0000","last_unstale":"2023-12-19T16:42:28.790441+0000","last_undegraded":"2023-12-19T16:42:28.790441+0000","last_fullsized":"2023-12-19T16:42:28.790441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:36.473614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658078+0000","last_change":"2023-12-19T16:42:27.817242+0000","last_active":"2023-12-19T16:42:29.658078+0000","last_peered":"2023-12-19T16:42:29.658078+0000","last_clean":"2023-12-19T16:42:29.658078+0000","last_became_active":"2023-12-19T16:42:27.816801+0000","last_became_peered":"2023-12-19T16:42:27.816801+0000","last_unstale":"2023-12-19T16:42:29.658078+0000","last_undegraded":"2023-12-19T16:42:29.658078+0000","last_fullsized":"2023-12-19T16:42:29.658078+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651814+0000","last_change":"2023-12-19T16:42:27.801939+0000","last_active":"2023-12-19T16:42:29.651814+0000","last_peered":"2023-12-19T16:42:29.651814+0000","last_clean":"2023-12-19T16:42:29.651814+0000","last_became_active":"2023-12-19T16:42:27.801672+0000","last_became_peered":"2023-12-19T16:42:27.801672+0000","last_unstale":"2023-12-19T16:42:29.651814+0000","last_undegraded":"2023-12-19T16:42:29.651814+0000","last_fullsized":"2023-12-19T16:42:29.651814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658359+0000","last_change":"2023-12-19T16:42:27.829667+0000","last_active":"2023-12-19T16:42:29.658359+0000","last_peered":"2023-12-19T16:42:29.658359+0000","last_clean":"2023-12-19T16:42:29.658359+0000","last_became_active":"2023-12-19T16:42:27.829357+0000","last_became_peered":"2023-12-19T16:42:27.829357+0000","last_unstale":"2023-12-19T16:42:29.658359+0000","last_undegraded":"2023-12-19T16:42:29.658359+0000","last_fullsized":"2023-12-19T16:42:29.658359+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:06.914435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654225+0000","last_change":"2023-12-19T16:42:27.812001+0000","last_active":"2023-12-19T16:42:29.654225+0000","last_peered":"2023-12-19T16:42:29.654225+0000","last_clean":"2023-12-19T16:42:29.654225+0000","last_became_active":"2023-12-19T16:42:27.811749+0000","last_became_peered":"2023-12-19T16:42:27.811749+0000","last_unstale":"2023-12-19T16:42:29.654225+0000","last_undegraded":"2023-12-19T16:42:29.654225+0000","last_fullsized":"2023-12-19T16:42:29.654225+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:29.376069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654212+0000","last_change":"2023-12-19T16:42:27.809484+0000","last_active":"2023-12-19T16:42:29.654212+0000","last_peered":"2023-12-19T16:42:29.654212+0000","last_clean":"2023-12-19T16:42:29.654212+0000","last_became_active":"2023-12-19T16:42:27.809189+0000","last_became_peered":"2023-12-19T16:42:27.809189+0000","last_unstale":"2023-12-19T16:42:29.654212+0000","last_undegraded":"2023-12-19T16:42:29.654212+0000","last_fullsized":"2023-12-19T16:42:29.654212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:03.390272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789997+0000","last_change":"2023-12-19T16:42:27.812088+0000","last_active":"2023-12-19T16:42:28.789997+0000","last_peered":"2023-12-19T16:42:28.789997+0000","last_clean":"2023-12-19T16:42:28.789997+0000","last_became_active":"2023-12-19T16:42:27.811768+0000","last_became_peered":"2023-12-19T16:42:27.811768+0000","last_unstale":"2023-12-19T16:42:28.789997+0000","last_undegraded":"2023-12-19T16:42:28.789997+0000","last_fullsized":"2023-12-19T16:42:28.789997+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:47:07.567597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651980+0000","last_change":"2023-12-19T16:42:27.805169+0000","last_active":"2023-12-19T16:42:29.651980+0000","last_peered":"2023-12-19T16:42:29.651980+0000","last_clean":"2023-12-19T16:42:29.651980+0000","last_became_active":"2023-12-19T16:42:27.804906+0000","last_became_peered":"2023-12-19T16:42:27.804906+0000","last_unstale":"2023-12-19T16:42:29.651980+0000","last_undegraded":"2023-12-19T16:42:29.651980+0000","last_fullsized":"2023-12-19T16:42:29.651980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:57:49.461118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648451+0000","last_change":"2023-12-19T16:42:27.796100+0000","last_active":"2023-12-19T16:42:28.648451+0000","last_peered":"2023-12-19T16:42:28.648451+0000","last_clean":"2023-12-19T16:42:28.648451+0000","last_became_active":"2023-12-19T16:42:27.795802+0000","last_became_peered":"2023-12-19T16:42:27.795802+0000","last_unstale":"2023-12-19T16:42:28.648451+0000","last_undegraded":"2023-12-19T16:42:28.648451+0000","last_fullsized":"2023-12-19T16:42:28.648451+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:30:13.550890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955716+0000","last_change":"2023-12-19T16:42:27.815496+0000","last_active":"2023-12-19T16:42:28.955716+0000","last_peered":"2023-12-19T16:42:28.955716+0000","last_clean":"2023-12-19T16:42:28.955716+0000","last_became_active":"2023-12-19T16:42:27.812535+0000","last_became_peered":"2023-12-19T16:42:27.812535+0000","last_unstale":"2023-12-19T16:42:28.955716+0000","last_undegraded":"2023-12-19T16:42:28.955716+0000","last_fullsized":"2023-12-19T16:42:28.955716+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:37.400723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790179+0000","last_change":"2023-12-19T16:42:27.804674+0000","last_active":"2023-12-19T16:42:28.790179+0000","last_peered":"2023-12-19T16:42:28.790179+0000","last_clean":"2023-12-19T16:42:28.790179+0000","last_became_active":"2023-12-19T16:42:27.804057+0000","last_became_peered":"2023-12-19T16:42:27.804057+0000","last_unstale":"2023-12-19T16:42:28.790179+0000","last_undegraded":"2023-12-19T16:42:28.790179+0000","last_fullsized":"2023-12-19T16:42:28.790179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:57:49.461118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790884+0000","last_change":"2023-12-19T16:42:27.829970+0000","last_active":"2023-12-19T16:42:28.790884+0000","last_peered":"2023-12-19T16:42:28.790884+0000","last_clean":"2023-12-19T16:42:28.790884+0000","last_became_active":"2023-12-19T16:42:27.829616+0000","last_became_peered":"2023-12-19T16:42:27.829616+0000","last_unstale":"2023-12-19T16:42:28.790884+0000","last_undegraded":"2023-12-19T16:42:28.790884+0000","last_fullsized":"2023-12-19T16:42:28.790884+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:50.303035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784083+0000","last_change":"2023-12-19T16:42:27.804309+0000","last_active":"2023-12-19T16:42:28.784083+0000","last_peered":"2023-12-19T16:42:28.784083+0000","last_clean":"2023-12-19T16:42:28.784083+0000","last_became_active":"2023-12-19T16:42:27.803973+0000","last_became_peered":"2023-12-19T16:42:27.803973+0000","last_unstale":"2023-12-19T16:42:28.784083+0000","last_undegraded":"2023-12-19T16:42:28.784083+0000","last_fullsized":"2023-12-19T16:42:28.784083+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:47:07.567597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658270+0000","last_change":"2023-12-19T16:42:27.827321+0000","last_active":"2023-12-19T16:42:29.658270+0000","last_peered":"2023-12-19T16:42:29.658270+0000","last_clean":"2023-12-19T16:42:29.658270+0000","last_became_active":"2023-12-19T16:42:27.826329+0000","last_became_peered":"2023-12-19T16:42:27.826329+0000","last_unstale":"2023-12-19T16:42:29.658270+0000","last_undegraded":"2023-12-19T16:42:29.658270+0000","last_fullsized":"2023-12-19T16:42:29.658270+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:01.122709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654188+0000","last_change":"2023-12-19T16:42:27.816854+0000","last_active":"2023-12-19T16:42:29.654188+0000","last_peered":"2023-12-19T16:42:29.654188+0000","last_clean":"2023-12-19T16:42:29.654188+0000","last_became_active":"2023-12-19T16:42:27.816538+0000","last_became_peered":"2023-12-19T16:42:27.816538+0000","last_unstale":"2023-12-19T16:42:29.654188+0000","last_undegraded":"2023-12-19T16:42:29.654188+0000","last_fullsized":"2023-12-19T16:42:29.654188+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:30.545767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648470+0000","last_change":"2023-12-19T16:42:27.817144+0000","last_active":"2023-12-19T16:42:28.648470+0000","last_peered":"2023-12-19T16:42:28.648470+0000","last_clean":"2023-12-19T16:42:28.648470+0000","last_became_active":"2023-12-19T16:42:27.816891+0000","last_became_peered":"2023-12-19T16:42:27.816891+0000","last_unstale":"2023-12-19T16:42:28.648470+0000","last_undegraded":"2023-12-19T16:42:28.648470+0000","last_fullsized":"2023-12-19T16:42:28.648470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:51.134371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790438+0000","last_change":"2023-12-19T16:42:27.826273+0000","last_active":"2023-12-19T16:42:28.790438+0000","last_peered":"2023-12-19T16:42:28.790438+0000","last_clean":"2023-12-19T16:42:28.790438+0000","last_became_active":"2023-12-19T16:42:27.825992+0000","last_became_peered":"2023-12-19T16:42:27.825992+0000","last_unstale":"2023-12-19T16:42:28.790438+0000","last_undegraded":"2023-12-19T16:42:28.790438+0000","last_fullsized":"2023-12-19T16:42:28.790438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:36.031318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651909+0000","last_change":"2023-12-19T16:42:27.813323+0000","last_active":"2023-12-19T16:42:29.651909+0000","last_peered":"2023-12-19T16:42:29.651909+0000","last_clean":"2023-12-19T16:42:29.651909+0000","last_became_active":"2023-12-19T16:42:27.812808+0000","last_became_peered":"2023-12-19T16:42:27.812808+0000","last_unstale":"2023-12-19T16:42:29.651909+0000","last_undegraded":"2023-12-19T16:42:29.651909+0000","last_fullsized":"2023-12-19T16:42:29.651909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:36.473614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790317+0000","last_change":"2023-12-19T16:42:27.810294+0000","last_active":"2023-12-19T16:42:28.790317+0000","last_peered":"2023-12-19T16:42:28.790317+0000","last_clean":"2023-12-19T16:42:28.790317+0000","last_became_active":"2023-12-19T16:42:27.810031+0000","last_became_peered":"2023-12-19T16:42:27.810031+0000","last_unstale":"2023-12-19T16:42:28.790317+0000","last_undegraded":"2023-12-19T16:42:28.790317+0000","last_fullsized":"2023-12-19T16:42:28.790317+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:01.122709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652166+0000","last_change":"2023-12-19T16:42:27.813230+0000","last_active":"2023-12-19T16:42:29.652166+0000","last_peered":"2023-12-19T16:42:29.652166+0000","last_clean":"2023-12-19T16:42:29.652166+0000","last_became_active":"2023-12-19T16:42:27.812914+0000","last_became_peered":"2023-12-19T16:42:27.812914+0000","last_unstale":"2023-12-19T16:42:29.652166+0000","last_undegraded":"2023-12-19T16:42:29.652166+0000","last_fullsized":"2023-12-19T16:42:29.652166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:50.303035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654238+0000","last_change":"2023-12-19T16:42:27.819653+0000","last_active":"2023-12-19T16:42:29.654238+0000","last_peered":"2023-12-19T16:42:29.654238+0000","last_clean":"2023-12-19T16:42:29.654238+0000","last_became_active":"2023-12-19T16:42:27.819325+0000","last_became_peered":"2023-12-19T16:42:27.819325+0000","last_unstale":"2023-12-19T16:42:29.654238+0000","last_undegraded":"2023-12-19T16:42:29.654238+0000","last_fullsized":"2023-12-19T16:42:29.654238+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:40:46.397053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955842+0000","last_change":"2023-12-19T16:42:27.813010+0000","last_active":"2023-12-19T16:42:28.955842+0000","last_peered":"2023-12-19T16:42:28.955842+0000","last_clean":"2023-12-19T16:42:28.955842+0000","last_became_active":"2023-12-19T16:42:27.812336+0000","last_became_peered":"2023-12-19T16:42:27.812336+0000","last_unstale":"2023-12-19T16:42:28.955842+0000","last_undegraded":"2023-12-19T16:42:28.955842+0000","last_fullsized":"2023-12-19T16:42:28.955842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652085+0000","last_change":"2023-12-19T16:42:27.813584+0000","last_active":"2023-12-19T16:42:29.652085+0000","last_peered":"2023-12-19T16:42:29.652085+0000","last_clean":"2023-12-19T16:42:29.652085+0000","last_became_active":"2023-12-19T16:42:27.813276+0000","last_became_peered":"2023-12-19T16:42:27.813276+0000","last_unstale":"2023-12-19T16:42:29.652085+0000","last_undegraded":"2023-12-19T16:42:29.652085+0000","last_fullsized":"2023-12-19T16:42:29.652085+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:36.031318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648478+0000","last_change":"2023-12-19T16:42:27.805390+0000","last_active":"2023-12-19T16:42:28.648478+0000","last_peered":"2023-12-19T16:42:28.648478+0000","last_clean":"2023-12-19T16:42:28.648478+0000","last_became_active":"2023-12-19T16:42:27.804956+0000","last_became_peered":"2023-12-19T16:42:27.804956+0000","last_unstale":"2023-12-19T16:42:28.648478+0000","last_undegraded":"2023-12-19T16:42:28.648478+0000","last_fullsized":"2023-12-19T16:42:28.648478+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:58.680381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.791132+0000","last_change":"2023-12-19T16:42:27.809669+0000","last_active":"2023-12-19T16:42:28.791132+0000","last_peered":"2023-12-19T16:42:28.791132+0000","last_clean":"2023-12-19T16:42:28.791132+0000","last_became_active":"2023-12-19T16:42:27.809374+0000","last_became_peered":"2023-12-19T16:42:27.809374+0000","last_unstale":"2023-12-19T16:42:28.791132+0000","last_undegraded":"2023-12-19T16:42:28.791132+0000","last_fullsized":"2023-12-19T16:42:28.791132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:53.498191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784039+0000","last_change":"2023-12-19T16:42:27.799272+0000","last_active":"2023-12-19T16:42:28.784039+0000","last_peered":"2023-12-19T16:42:28.784039+0000","last_clean":"2023-12-19T16:42:28.784039+0000","last_became_active":"2023-12-19T16:42:27.799006+0000","last_became_peered":"2023-12-19T16:42:27.799006+0000","last_unstale":"2023-12-19T16:42:28.784039+0000","last_undegraded":"2023-12-19T16:42:28.784039+0000","last_fullsized":"2023-12-19T16:42:28.784039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:37.286916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654047+0000","last_change":"2023-12-19T16:42:27.809714+0000","last_active":"2023-12-19T16:42:29.654047+0000","last_peered":"2023-12-19T16:42:29.654047+0000","last_clean":"2023-12-19T16:42:29.654047+0000","last_became_active":"2023-12-19T16:42:27.809384+0000","last_became_peered":"2023-12-19T16:42:27.809384+0000","last_unstale":"2023-12-19T16:42:29.654047+0000","last_undegraded":"2023-12-19T16:42:29.654047+0000","last_fullsized":"2023-12-19T16:42:29.654047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:37.400723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789634+0000","last_change":"2023-12-19T16:42:27.811964+0000","last_active":"2023-12-19T16:42:28.789634+0000","last_peered":"2023-12-19T16:42:28.789634+0000","last_clean":"2023-12-19T16:42:28.789634+0000","last_became_active":"2023-12-19T16:42:27.811777+0000","last_became_peered":"2023-12-19T16:42:27.811777+0000","last_unstale":"2023-12-19T16:42:28.789634+0000","last_undegraded":"2023-12-19T16:42:28.789634+0000","last_fullsized":"2023-12-19T16:42:28.789634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:50.714712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652057+0000","last_change":"2023-12-19T16:42:27.324768+0000","last_active":"2023-12-19T16:42:29.652057+0000","last_peered":"2023-12-19T16:42:29.652057+0000","last_clean":"2023-12-19T16:42:29.652057+0000","last_became_active":"2023-12-19T16:42:27.324380+0000","last_became_peered":"2023-12-19T16:42:27.324380+0000","last_unstale":"2023-12-19T16:42:29.652057+0000","last_undegraded":"2023-12-19T16:42:29.652057+0000","last_fullsized":"2023-12-19T16:42:29.652057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648490+0000","last_change":"2023-12-19T16:42:27.816607+0000","last_active":"2023-12-19T16:42:28.648490+0000","last_peered":"2023-12-19T16:42:28.648490+0000","last_clean":"2023-12-19T16:42:28.648490+0000","last_became_active":"2023-12-19T16:42:27.815987+0000","last_became_peered":"2023-12-19T16:42:27.815987+0000","last_unstale":"2023-12-19T16:42:28.648490+0000","last_undegraded":"2023-12-19T16:42:28.648490+0000","last_fullsized":"2023-12-19T16:42:28.648490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:32:26.852410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654158+0000","last_change":"2023-12-19T16:42:27.327298+0000","last_active":"2023-12-19T16:42:29.654158+0000","last_peered":"2023-12-19T16:42:29.654158+0000","last_clean":"2023-12-19T16:42:29.654158+0000","last_became_active":"2023-12-19T16:42:27.326988+0000","last_became_peered":"2023-12-19T16:42:27.326988+0000","last_unstale":"2023-12-19T16:42:29.654158+0000","last_undegraded":"2023-12-19T16:42:29.654158+0000","last_fullsized":"2023-12-19T16:42:29.654158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651933+0000","last_change":"2023-12-19T16:42:27.320910+0000","last_active":"2023-12-19T16:42:29.651933+0000","last_peered":"2023-12-19T16:42:29.651933+0000","last_clean":"2023-12-19T16:42:29.651933+0000","last_became_active":"2023-12-19T16:42:27.320629+0000","last_became_peered":"2023-12-19T16:42:27.320629+0000","last_unstale":"2023-12-19T16:42:29.651933+0000","last_undegraded":"2023-12-19T16:42:29.651933+0000","last_fullsized":"2023-12-19T16:42:29.651933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651846+0000","last_change":"2023-12-19T16:42:27.803836+0000","last_active":"2023-12-19T16:42:29.651846+0000","last_peered":"2023-12-19T16:42:29.651846+0000","last_clean":"2023-12-19T16:42:29.651846+0000","last_became_active":"2023-12-19T16:42:27.803527+0000","last_became_peered":"2023-12-19T16:42:27.803527+0000","last_unstale":"2023-12-19T16:42:29.651846+0000","last_undegraded":"2023-12-19T16:42:29.651846+0000","last_fullsized":"2023-12-19T16:42:29.651846+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:37.400723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790294+0000","last_change":"2023-12-19T16:42:27.322383+0000","last_active":"2023-12-19T16:42:28.790294+0000","last_peered":"2023-12-19T16:42:28.790294+0000","last_clean":"2023-12-19T16:42:28.790294+0000","last_became_active":"2023-12-19T16:42:27.321439+0000","last_became_peered":"2023-12-19T16:42:27.321439+0000","last_unstale":"2023-12-19T16:42:28.790294+0000","last_undegraded":"2023-12-19T16:42:28.790294+0000","last_fullsized":"2023-12-19T16:42:28.790294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955715+0000","last_change":"2023-12-19T16:42:27.800626+0000","last_active":"2023-12-19T16:42:28.955715+0000","last_peered":"2023-12-19T16:42:28.955715+0000","last_clean":"2023-12-19T16:42:28.955715+0000","last_became_active":"2023-12-19T16:42:27.800338+0000","last_became_peered":"2023-12-19T16:42:27.800338+0000","last_unstale":"2023-12-19T16:42:28.955715+0000","last_undegraded":"2023-12-19T16:42:28.955715+0000","last_fullsized":"2023-12-19T16:42:28.955715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:29.376069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654248+0000","last_change":"2023-12-19T16:42:27.324962+0000","last_active":"2023-12-19T16:42:29.654248+0000","last_peered":"2023-12-19T16:42:29.654248+0000","last_clean":"2023-12-19T16:42:29.654248+0000","last_became_active":"2023-12-19T16:42:27.324627+0000","last_became_peered":"2023-12-19T16:42:27.324627+0000","last_unstale":"2023-12-19T16:42:29.654248+0000","last_undegraded":"2023-12-19T16:42:29.654248+0000","last_fullsized":"2023-12-19T16:42:29.654248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790825+0000","last_change":"2023-12-19T16:42:27.829894+0000","last_active":"2023-12-19T16:42:28.790825+0000","last_peered":"2023-12-19T16:42:28.790825+0000","last_clean":"2023-12-19T16:42:28.790825+0000","last_became_active":"2023-12-19T16:42:27.829581+0000","last_became_peered":"2023-12-19T16:42:27.829581+0000","last_unstale":"2023-12-19T16:42:28.790825+0000","last_undegraded":"2023-12-19T16:42:28.790825+0000","last_fullsized":"2023-12-19T16:42:28.790825+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:00.499277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790905+0000","last_change":"2023-12-19T16:42:27.322335+0000","last_active":"2023-12-19T16:42:28.790905+0000","last_peered":"2023-12-19T16:42:28.790905+0000","last_clean":"2023-12-19T16:42:28.790905+0000","last_became_active":"2023-12-19T16:42:27.319501+0000","last_became_peered":"2023-12-19T16:42:27.319501+0000","last_unstale":"2023-12-19T16:42:28.790905+0000","last_undegraded":"2023-12-19T16:42:28.790905+0000","last_fullsized":"2023-12-19T16:42:28.790905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648548+0000","last_change":"2023-12-19T16:42:27.798069+0000","last_active":"2023-12-19T16:42:28.648548+0000","last_peered":"2023-12-19T16:42:28.648548+0000","last_clean":"2023-12-19T16:42:28.648548+0000","last_became_active":"2023-12-19T16:42:27.795622+0000","last_became_peered":"2023-12-19T16:42:27.795622+0000","last_unstale":"2023-12-19T16:42:28.648548+0000","last_undegraded":"2023-12-19T16:42:28.648548+0000","last_fullsized":"2023-12-19T16:42:28.648548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:15:14.536450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955857+0000","last_change":"2023-12-19T16:42:27.310215+0000","last_active":"2023-12-19T16:42:28.955857+0000","last_peered":"2023-12-19T16:42:28.955857+0000","last_clean":"2023-12-19T16:42:28.955857+0000","last_became_active":"2023-12-19T16:42:27.308663+0000","last_became_peered":"2023-12-19T16:42:27.308663+0000","last_unstale":"2023-12-19T16:42:28.955857+0000","last_undegraded":"2023-12-19T16:42:28.955857+0000","last_fullsized":"2023-12-19T16:42:28.955857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:41.116483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790149+0000","last_change":"2023-12-19T16:42:27.825472+0000","last_active":"2023-12-19T16:42:28.790149+0000","last_peered":"2023-12-19T16:42:28.790149+0000","last_clean":"2023-12-19T16:42:28.790149+0000","last_became_active":"2023-12-19T16:42:27.825201+0000","last_became_peered":"2023-12-19T16:42:27.825201+0000","last_unstale":"2023-12-19T16:42:28.790149+0000","last_undegraded":"2023-12-19T16:42:28.790149+0000","last_fullsized":"2023-12-19T16:42:28.790149+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:42.122812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654201+0000","last_change":"2023-12-19T16:42:27.321076+0000","last_active":"2023-12-19T16:42:29.654201+0000","last_peered":"2023-12-19T16:42:29.654201+0000","last_clean":"2023-12-19T16:42:29.654201+0000","last_became_active":"2023-12-19T16:42:27.320592+0000","last_became_peered":"2023-12-19T16:42:27.320592+0000","last_unstale":"2023-12-19T16:42:29.654201+0000","last_undegraded":"2023-12-19T16:42:29.654201+0000","last_fullsized":"2023-12-19T16:42:29.654201+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.708383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658212+0000","last_change":"2023-12-19T16:42:27.818972+0000","last_active":"2023-12-19T16:42:29.658212+0000","last_peered":"2023-12-19T16:42:29.658212+0000","last_clean":"2023-12-19T16:42:29.658212+0000","last_became_active":"2023-12-19T16:42:27.818689+0000","last_became_peered":"2023-12-19T16:42:27.818689+0000","last_unstale":"2023-12-19T16:42:29.658212+0000","last_undegraded":"2023-12-19T16:42:29.658212+0000","last_fullsized":"2023-12-19T16:42:29.658212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:48:49.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648499+0000","last_change":"2023-12-19T16:42:27.306416+0000","last_active":"2023-12-19T16:42:28.648499+0000","last_peered":"2023-12-19T16:42:28.648499+0000","last_clean":"2023-12-19T16:42:28.648499+0000","last_became_active":"2023-12-19T16:42:27.306033+0000","last_became_peered":"2023-12-19T16:42:27.306033+0000","last_unstale":"2023-12-19T16:42:28.648499+0000","last_undegraded":"2023-12-19T16:42:28.648499+0000","last_fullsized":"2023-12-19T16:42:28.648499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:29.172833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648461+0000","last_change":"2023-12-19T16:42:27.809290+0000","last_active":"2023-12-19T16:42:28.648461+0000","last_peered":"2023-12-19T16:42:28.648461+0000","last_clean":"2023-12-19T16:42:28.648461+0000","last_became_active":"2023-12-19T16:42:27.806028+0000","last_became_peered":"2023-12-19T16:42:27.806028+0000","last_unstale":"2023-12-19T16:42:28.648461+0000","last_undegraded":"2023-12-19T16:42:28.648461+0000","last_fullsized":"2023-12-19T16:42:28.648461+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:48:43.490883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.791044+0000","last_change":"2023-12-19T16:42:27.317606+0000","last_active":"2023-12-19T16:42:28.791044+0000","last_peered":"2023-12-19T16:42:28.791044+0000","last_clean":"2023-12-19T16:42:28.791044+0000","last_became_active":"2023-12-19T16:42:27.316871+0000","last_became_peered":"2023-12-19T16:42:27.316871+0000","last_unstale":"2023-12-19T16:42:28.791044+0000","last_undegraded":"2023-12-19T16:42:28.791044+0000","last_fullsized":"2023-12-19T16:42:28.791044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.708383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658130+0000","last_change":"2023-12-19T16:42:27.826100+0000","last_active":"2023-12-19T16:42:29.658130+0000","last_peered":"2023-12-19T16:42:29.658130+0000","last_clean":"2023-12-19T16:42:29.658130+0000","last_became_active":"2023-12-19T16:42:27.825764+0000","last_became_peered":"2023-12-19T16:42:27.825764+0000","last_unstale":"2023-12-19T16:42:29.658130+0000","last_undegraded":"2023-12-19T16:42:29.658130+0000","last_fullsized":"2023-12-19T16:42:29.658130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:00.499277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654128+0000","last_change":"2023-12-19T16:42:27.327321+0000","last_active":"2023-12-19T16:42:29.654128+0000","last_peered":"2023-12-19T16:42:29.654128+0000","last_clean":"2023-12-19T16:42:29.654128+0000","last_became_active":"2023-12-19T16:42:27.327010+0000","last_became_peered":"2023-12-19T16:42:27.327010+0000","last_unstale":"2023-12-19T16:42:29.654128+0000","last_undegraded":"2023-12-19T16:42:29.654128+0000","last_fullsized":"2023-12-19T16:42:29.654128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654100+0000","last_change":"2023-12-19T16:42:27.802521+0000","last_active":"2023-12-19T16:42:29.654100+0000","last_peered":"2023-12-19T16:42:29.654100+0000","last_clean":"2023-12-19T16:42:29.654100+0000","last_became_active":"2023-12-19T16:42:27.801786+0000","last_became_peered":"2023-12-19T16:42:27.801786+0000","last_unstale":"2023-12-19T16:42:29.654100+0000","last_undegraded":"2023-12-19T16:42:29.654100+0000","last_fullsized":"2023-12-19T16:42:29.654100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:16.399823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784006+0000","last_change":"2023-12-19T16:42:27.311025+0000","last_active":"2023-12-19T16:42:28.784006+0000","last_peered":"2023-12-19T16:42:28.784006+0000","last_clean":"2023-12-19T16:42:28.784006+0000","last_became_active":"2023-12-19T16:42:27.310676+0000","last_became_peered":"2023-12-19T16:42:27.310676+0000","last_unstale":"2023-12-19T16:42:28.784006+0000","last_undegraded":"2023-12-19T16:42:28.784006+0000","last_fullsized":"2023-12-19T16:42:28.784006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658095+0000","last_change":"2023-12-19T16:42:27.821468+0000","last_active":"2023-12-19T16:42:29.658095+0000","last_peered":"2023-12-19T16:42:29.658095+0000","last_clean":"2023-12-19T16:42:29.658095+0000","last_became_active":"2023-12-19T16:42:27.821113+0000","last_became_peered":"2023-12-19T16:42:27.821113+0000","last_unstale":"2023-12-19T16:42:29.658095+0000","last_undegraded":"2023-12-19T16:42:29.658095+0000","last_fullsized":"2023-12-19T16:42:29.658095+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:03.390272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790606+0000","last_change":"2023-12-19T16:42:27.323313+0000","last_active":"2023-12-19T16:42:28.790606+0000","last_peered":"2023-12-19T16:42:28.790606+0000","last_clean":"2023-12-19T16:42:28.790606+0000","last_became_active":"2023-12-19T16:42:27.319464+0000","last_became_peered":"2023-12-19T16:42:27.319464+0000","last_unstale":"2023-12-19T16:42:28.790606+0000","last_undegraded":"2023-12-19T16:42:28.790606+0000","last_fullsized":"2023-12-19T16:42:28.790606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:22:50.990280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658306+0000","last_change":"2023-12-19T16:42:27.821587+0000","last_active":"2023-12-19T16:42:29.658306+0000","last_peered":"2023-12-19T16:42:29.658306+0000","last_clean":"2023-12-19T16:42:29.658306+0000","last_became_active":"2023-12-19T16:42:27.821251+0000","last_became_peered":"2023-12-19T16:42:27.821251+0000","last_unstale":"2023-12-19T16:42:29.658306+0000","last_undegraded":"2023-12-19T16:42:29.658306+0000","last_fullsized":"2023-12-19T16:42:29.658306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:18.209972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784184+0000","last_change":"2023-12-19T16:42:27.319966+0000","last_active":"2023-12-19T16:42:28.784184+0000","last_peered":"2023-12-19T16:42:28.784184+0000","last_clean":"2023-12-19T16:42:28.784184+0000","last_became_active":"2023-12-19T16:42:27.319141+0000","last_became_peered":"2023-12-19T16:42:27.319141+0000","last_unstale":"2023-12-19T16:42:28.784184+0000","last_undegraded":"2023-12-19T16:42:28.784184+0000","last_fullsized":"2023-12-19T16:42:28.784184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658377+0000","last_change":"2023-12-19T16:42:27.824936+0000","last_active":"2023-12-19T16:42:29.658377+0000","last_peered":"2023-12-19T16:42:29.658377+0000","last_clean":"2023-12-19T16:42:29.658377+0000","last_became_active":"2023-12-19T16:42:27.824643+0000","last_became_peered":"2023-12-19T16:42:27.824643+0000","last_unstale":"2023-12-19T16:42:29.658377+0000","last_undegraded":"2023-12-19T16:42:29.658377+0000","last_fullsized":"2023-12-19T16:42:29.658377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:23.006980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955939+0000","last_change":"2023-12-19T16:42:27.332207+0000","last_active":"2023-12-19T16:42:28.955939+0000","last_peered":"2023-12-19T16:42:28.955939+0000","last_clean":"2023-12-19T16:42:28.955939+0000","last_became_active":"2023-12-19T16:42:27.331920+0000","last_became_peered":"2023-12-19T16:42:27.331920+0000","last_unstale":"2023-12-19T16:42:28.955939+0000","last_undegraded":"2023-12-19T16:42:28.955939+0000","last_fullsized":"2023-12-19T16:42:28.955939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.708383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790779+0000","last_change":"2023-12-19T16:42:27.803564+0000","last_active":"2023-12-19T16:42:28.790779+0000","last_peered":"2023-12-19T16:42:28.790779+0000","last_clean":"2023-12-19T16:42:28.790779+0000","last_became_active":"2023-12-19T16:42:27.803291+0000","last_became_peered":"2023-12-19T16:42:27.803291+0000","last_unstale":"2023-12-19T16:42:28.790779+0000","last_undegraded":"2023-12-19T16:42:28.790779+0000","last_fullsized":"2023-12-19T16:42:28.790779+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:49.764257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784115+0000","last_change":"2023-12-19T16:42:27.316666+0000","last_active":"2023-12-19T16:42:28.784115+0000","last_peered":"2023-12-19T16:42:28.784115+0000","last_clean":"2023-12-19T16:42:28.784115+0000","last_became_active":"2023-12-19T16:42:27.316322+0000","last_became_peered":"2023-12-19T16:42:27.316322+0000","last_unstale":"2023-12-19T16:42:28.784115+0000","last_undegraded":"2023-12-19T16:42:28.784115+0000","last_fullsized":"2023-12-19T16:42:28.784115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658228+0000","last_change":"2023-12-19T16:42:27.828580+0000","last_active":"2023-12-19T16:42:29.658228+0000","last_peered":"2023-12-19T16:42:29.658228+0000","last_clean":"2023-12-19T16:42:29.658228+0000","last_became_active":"2023-12-19T16:42:27.828263+0000","last_became_peered":"2023-12-19T16:42:27.828263+0000","last_unstale":"2023-12-19T16:42:29.658228+0000","last_undegraded":"2023-12-19T16:42:29.658228+0000","last_fullsized":"2023-12-19T16:42:29.658228+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:15.012739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":164,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:27.808382+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:42:27.808382+0000","last_peered":"2023-12-19T16:42:27.808382+0000","last_clean":"2023-12-19T16:42:27.808382+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:42:27.808382+0000","last_undegraded":"2023-12-19T16:42:27.808382+0000","last_fullsized":"2023-12-19T16:42:27.808382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790309+0000","last_change":"2023-12-19T16:42:27.322949+0000","last_active":"2023-12-19T16:42:28.790309+0000","last_peered":"2023-12-19T16:42:28.790309+0000","last_clean":"2023-12-19T16:42:28.790309+0000","last_became_active":"2023-12-19T16:42:27.321633+0000","last_became_peered":"2023-12-19T16:42:27.321633+0000","last_unstale":"2023-12-19T16:42:28.790309+0000","last_undegraded":"2023-12-19T16:42:28.790309+0000","last_fullsized":"2023-12-19T16:42:28.790309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651957+0000","last_change":"2023-12-19T16:42:27.798689+0000","last_active":"2023-12-19T16:42:29.651957+0000","last_peered":"2023-12-19T16:42:29.651957+0000","last_clean":"2023-12-19T16:42:29.651957+0000","last_became_active":"2023-12-19T16:42:27.798430+0000","last_became_peered":"2023-12-19T16:42:27.798430+0000","last_unstale":"2023-12-19T16:42:29.651957+0000","last_undegraded":"2023-12-19T16:42:29.651957+0000","last_fullsized":"2023-12-19T16:42:29.651957+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:16.399823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658033+0000","last_change":"2023-12-19T16:42:27.326882+0000","last_active":"2023-12-19T16:42:29.658033+0000","last_peered":"2023-12-19T16:42:29.658033+0000","last_clean":"2023-12-19T16:42:29.658033+0000","last_became_active":"2023-12-19T16:42:27.326616+0000","last_became_peered":"2023-12-19T16:42:27.326616+0000","last_unstale":"2023-12-19T16:42:29.658033+0000","last_undegraded":"2023-12-19T16:42:29.658033+0000","last_fullsized":"2023-12-19T16:42:29.658033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:36.280588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648432+0000","last_change":"2023-12-19T16:42:27.816279+0000","last_active":"2023-12-19T16:42:28.648432+0000","last_peered":"2023-12-19T16:42:28.648432+0000","last_clean":"2023-12-19T16:42:28.648432+0000","last_became_active":"2023-12-19T16:42:27.815940+0000","last_became_peered":"2023-12-19T16:42:27.815940+0000","last_unstale":"2023-12-19T16:42:28.648432+0000","last_undegraded":"2023-12-19T16:42:28.648432+0000","last_fullsized":"2023-12-19T16:42:28.648432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:51:00.190122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.784142+0000","last_change":"2023-12-19T16:42:27.317414+0000","last_active":"2023-12-19T16:42:28.784142+0000","last_peered":"2023-12-19T16:42:28.784142+0000","last_clean":"2023-12-19T16:42:28.784142+0000","last_became_active":"2023-12-19T16:42:27.317065+0000","last_became_peered":"2023-12-19T16:42:27.317065+0000","last_unstale":"2023-12-19T16:42:28.784142+0000","last_undegraded":"2023-12-19T16:42:28.784142+0000","last_fullsized":"2023-12-19T16:42:28.784142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955832+0000","last_change":"2023-12-19T16:42:27.794867+0000","last_active":"2023-12-19T16:42:28.955832+0000","last_peered":"2023-12-19T16:42:28.955832+0000","last_clean":"2023-12-19T16:42:28.955832+0000","last_became_active":"2023-12-19T16:42:27.793788+0000","last_became_peered":"2023-12-19T16:42:27.793788+0000","last_unstale":"2023-12-19T16:42:28.955832+0000","last_undegraded":"2023-12-19T16:42:28.955832+0000","last_fullsized":"2023-12-19T16:42:28.955832+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:58:03.549430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.791093+0000","last_change":"2023-12-19T16:42:27.312696+0000","last_active":"2023-12-19T16:42:28.791093+0000","last_peered":"2023-12-19T16:42:28.791093+0000","last_clean":"2023-12-19T16:42:28.791093+0000","last_became_active":"2023-12-19T16:42:27.312406+0000","last_became_peered":"2023-12-19T16:42:27.312406+0000","last_unstale":"2023-12-19T16:42:28.791093+0000","last_undegraded":"2023-12-19T16:42:28.791093+0000","last_fullsized":"2023-12-19T16:42:28.791093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:41.116483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789955+0000","last_change":"2023-12-19T16:42:27.813887+0000","last_active":"2023-12-19T16:42:28.789955+0000","last_peered":"2023-12-19T16:42:28.789955+0000","last_clean":"2023-12-19T16:42:28.789955+0000","last_became_active":"2023-12-19T16:42:27.813476+0000","last_became_peered":"2023-12-19T16:42:27.813476+0000","last_unstale":"2023-12-19T16:42:28.789955+0000","last_undegraded":"2023-12-19T16:42:28.789955+0000","last_fullsized":"2023-12-19T16:42:28.789955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:36.975084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955856+0000","last_change":"2023-12-19T16:42:27.302950+0000","last_active":"2023-12-19T16:42:28.955856+0000","last_peered":"2023-12-19T16:42:28.955856+0000","last_clean":"2023-12-19T16:42:28.955856+0000","last_became_active":"2023-12-19T16:42:27.302614+0000","last_became_peered":"2023-12-19T16:42:27.302614+0000","last_unstale":"2023-12-19T16:42:28.955856+0000","last_undegraded":"2023-12-19T16:42:28.955856+0000","last_fullsized":"2023-12-19T16:42:28.955856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:00:14.982943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790926+0000","last_change":"2023-12-19T16:42:27.803672+0000","last_active":"2023-12-19T16:42:28.790926+0000","last_peered":"2023-12-19T16:42:28.790926+0000","last_clean":"2023-12-19T16:42:28.790926+0000","last_became_active":"2023-12-19T16:42:27.803402+0000","last_became_peered":"2023-12-19T16:42:27.803402+0000","last_unstale":"2023-12-19T16:42:28.790926+0000","last_undegraded":"2023-12-19T16:42:28.790926+0000","last_fullsized":"2023-12-19T16:42:28.790926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:48:49.625464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658289+0000","last_change":"2023-12-19T16:42:27.330785+0000","last_active":"2023-12-19T16:42:29.658289+0000","last_peered":"2023-12-19T16:42:29.658289+0000","last_clean":"2023-12-19T16:42:29.658289+0000","last_became_active":"2023-12-19T16:42:27.329823+0000","last_became_peered":"2023-12-19T16:42:27.329823+0000","last_unstale":"2023-12-19T16:42:29.658289+0000","last_undegraded":"2023-12-19T16:42:29.658289+0000","last_fullsized":"2023-12-19T16:42:29.658289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:20.293007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955776+0000","last_change":"2023-12-19T16:42:27.794649+0000","last_active":"2023-12-19T16:42:28.955776+0000","last_peered":"2023-12-19T16:42:28.955776+0000","last_clean":"2023-12-19T16:42:28.955776+0000","last_became_active":"2023-12-19T16:42:27.793710+0000","last_became_peered":"2023-12-19T16:42:27.793710+0000","last_unstale":"2023-12-19T16:42:28.955776+0000","last_undegraded":"2023-12-19T16:42:28.955776+0000","last_fullsized":"2023-12-19T16:42:28.955776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:16.399823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652133+0000","last_change":"2023-12-19T16:42:27.321060+0000","last_active":"2023-12-19T16:42:29.652133+0000","last_peered":"2023-12-19T16:42:29.652133+0000","last_clean":"2023-12-19T16:42:29.652133+0000","last_became_active":"2023-12-19T16:42:27.320705+0000","last_became_peered":"2023-12-19T16:42:27.320705+0000","last_unstale":"2023-12-19T16:42:29.652133+0000","last_undegraded":"2023-12-19T16:42:29.652133+0000","last_fullsized":"2023-12-19T16:42:29.652133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:35.968756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652184+0000","last_change":"2023-12-19T16:42:27.818162+0000","last_active":"2023-12-19T16:42:29.652184+0000","last_peered":"2023-12-19T16:42:29.652184+0000","last_clean":"2023-12-19T16:42:29.652184+0000","last_became_active":"2023-12-19T16:42:27.817894+0000","last_became_peered":"2023-12-19T16:42:27.817894+0000","last_unstale":"2023-12-19T16:42:29.652184+0000","last_undegraded":"2023-12-19T16:42:29.652184+0000","last_fullsized":"2023-12-19T16:42:29.652184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:29.376069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654257+0000","last_change":"2023-12-19T16:42:27.330277+0000","last_active":"2023-12-19T16:42:29.654257+0000","last_peered":"2023-12-19T16:42:29.654257+0000","last_clean":"2023-12-19T16:42:29.654257+0000","last_became_active":"2023-12-19T16:42:27.330006+0000","last_became_peered":"2023-12-19T16:42:27.330006+0000","last_unstale":"2023-12-19T16:42:29.654257+0000","last_undegraded":"2023-12-19T16:42:29.654257+0000","last_fullsized":"2023-12-19T16:42:29.654257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648518+0000","last_change":"2023-12-19T16:42:27.791761+0000","last_active":"2023-12-19T16:42:28.648518+0000","last_peered":"2023-12-19T16:42:28.648518+0000","last_clean":"2023-12-19T16:42:28.648518+0000","last_became_active":"2023-12-19T16:42:27.791490+0000","last_became_peered":"2023-12-19T16:42:27.791490+0000","last_unstale":"2023-12-19T16:42:28.648518+0000","last_undegraded":"2023-12-19T16:42:28.648518+0000","last_fullsized":"2023-12-19T16:42:28.648518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:48:40.695901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658009+0000","last_change":"2023-12-19T16:42:27.326363+0000","last_active":"2023-12-19T16:42:29.658009+0000","last_peered":"2023-12-19T16:42:29.658009+0000","last_clean":"2023-12-19T16:42:29.658009+0000","last_became_active":"2023-12-19T16:42:27.326046+0000","last_became_peered":"2023-12-19T16:42:27.326046+0000","last_unstale":"2023-12-19T16:42:29.658009+0000","last_undegraded":"2023-12-19T16:42:29.658009+0000","last_fullsized":"2023-12-19T16:42:29.658009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:41.116483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.789733+0000","last_change":"2023-12-19T16:42:27.808612+0000","last_active":"2023-12-19T16:42:28.789733+0000","last_peered":"2023-12-19T16:42:28.789733+0000","last_clean":"2023-12-19T16:42:28.789733+0000","last_became_active":"2023-12-19T16:42:27.808219+0000","last_became_peered":"2023-12-19T16:42:27.808219+0000","last_unstale":"2023-12-19T16:42:28.789733+0000","last_undegraded":"2023-12-19T16:42:28.789733+0000","last_fullsized":"2023-12-19T16:42:28.789733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:37.286916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955811+0000","last_change":"2023-12-19T16:42:27.308462+0000","last_active":"2023-12-19T16:42:28.955811+0000","last_peered":"2023-12-19T16:42:28.955811+0000","last_clean":"2023-12-19T16:42:28.955811+0000","last_became_active":"2023-12-19T16:42:27.307879+0000","last_became_peered":"2023-12-19T16:42:27.307879+0000","last_unstale":"2023-12-19T16:42:28.955811+0000","last_undegraded":"2023-12-19T16:42:28.955811+0000","last_fullsized":"2023-12-19T16:42:28.955811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:40.791961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.651875+0000","last_change":"2023-12-19T16:42:27.798674+0000","last_active":"2023-12-19T16:42:29.651875+0000","last_peered":"2023-12-19T16:42:29.651875+0000","last_clean":"2023-12-19T16:42:29.651875+0000","last_became_active":"2023-12-19T16:42:27.798442+0000","last_became_peered":"2023-12-19T16:42:27.798442+0000","last_unstale":"2023-12-19T16:42:29.651875+0000","last_undegraded":"2023-12-19T16:42:29.651875+0000","last_fullsized":"2023-12-19T16:42:29.651875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:01.122709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.955925+0000","last_change":"2023-12-19T16:42:27.310942+0000","last_active":"2023-12-19T16:42:28.955925+0000","last_peered":"2023-12-19T16:42:28.955925+0000","last_clean":"2023-12-19T16:42:28.955925+0000","last_became_active":"2023-12-19T16:42:27.310605+0000","last_became_peered":"2023-12-19T16:42:27.310605+0000","last_unstale":"2023-12-19T16:42:28.955925+0000","last_undegraded":"2023-12-19T16:42:28.955925+0000","last_fullsized":"2023-12-19T16:42:28.955925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652009+0000","last_change":"2023-12-19T16:42:27.802676+0000","last_active":"2023-12-19T16:42:29.652009+0000","last_peered":"2023-12-19T16:42:29.652009+0000","last_clean":"2023-12-19T16:42:29.652009+0000","last_became_active":"2023-12-19T16:42:27.801627+0000","last_became_peered":"2023-12-19T16:42:27.801627+0000","last_unstale":"2023-12-19T16:42:29.652009+0000","last_undegraded":"2023-12-19T16:42:29.652009+0000","last_fullsized":"2023-12-19T16:42:29.652009+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:30.545767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658151+0000","last_change":"2023-12-19T16:42:27.334084+0000","last_active":"2023-12-19T16:42:29.658151+0000","last_peered":"2023-12-19T16:42:29.658151+0000","last_clean":"2023-12-19T16:42:29.658151+0000","last_became_active":"2023-12-19T16:42:27.333748+0000","last_became_peered":"2023-12-19T16:42:27.333748+0000","last_unstale":"2023-12-19T16:42:29.658151+0000","last_undegraded":"2023-12-19T16:42:29.658151+0000","last_fullsized":"2023-12-19T16:42:29.658151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:50.484993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.654146+0000","last_change":"2023-12-19T16:42:27.817078+0000","last_active":"2023-12-19T16:42:29.654146+0000","last_peered":"2023-12-19T16:42:29.654146+0000","last_clean":"2023-12-19T16:42:29.654146+0000","last_became_active":"2023-12-19T16:42:27.816835+0000","last_became_peered":"2023-12-19T16:42:27.816835+0000","last_unstale":"2023-12-19T16:42:29.654146+0000","last_undegraded":"2023-12-19T16:42:29.654146+0000","last_fullsized":"2023-12-19T16:42:29.654146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790847+0000","last_change":"2023-12-19T16:42:27.820348+0000","last_active":"2023-12-19T16:42:28.790847+0000","last_peered":"2023-12-19T16:42:28.790847+0000","last_clean":"2023-12-19T16:42:28.790847+0000","last_became_active":"2023-12-19T16:42:27.820051+0000","last_became_peered":"2023-12-19T16:42:27.820051+0000","last_unstale":"2023-12-19T16:42:28.790847+0000","last_undegraded":"2023-12-19T16:42:28.790847+0000","last_fullsized":"2023-12-19T16:42:28.790847+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:23.006980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658192+0000","last_change":"2023-12-19T16:42:27.328852+0000","last_active":"2023-12-19T16:42:29.658192+0000","last_peered":"2023-12-19T16:42:29.658192+0000","last_clean":"2023-12-19T16:42:29.658192+0000","last_became_active":"2023-12-19T16:42:27.328521+0000","last_became_peered":"2023-12-19T16:42:27.328521+0000","last_unstale":"2023-12-19T16:42:29.658192+0000","last_undegraded":"2023-12-19T16:42:29.658192+0000","last_fullsized":"2023-12-19T16:42:29.658192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:35.968756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.790575+0000","last_change":"2023-12-19T16:42:27.820348+0000","last_active":"2023-12-19T16:42:28.790575+0000","last_peered":"2023-12-19T16:42:28.790575+0000","last_clean":"2023-12-19T16:42:28.790575+0000","last_became_active":"2023-12-19T16:42:27.820018+0000","last_became_peered":"2023-12-19T16:42:27.820018+0000","last_unstale":"2023-12-19T16:42:28.790575+0000","last_undegraded":"2023-12-19T16:42:28.790575+0000","last_fullsized":"2023-12-19T16:42:28.790575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:19:03.534387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.658245+0000","last_change":"2023-12-19T16:42:27.331867+0000","last_active":"2023-12-19T16:42:29.658245+0000","last_peered":"2023-12-19T16:42:29.658245+0000","last_clean":"2023-12-19T16:42:29.658245+0000","last_became_active":"2023-12-19T16:42:27.331354+0000","last_became_peered":"2023-12-19T16:42:27.331354+0000","last_unstale":"2023-12-19T16:42:29.658245+0000","last_undegraded":"2023-12-19T16:42:29.658245+0000","last_fullsized":"2023-12-19T16:42:29.658245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:47:06.561268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648540+0000","last_change":"2023-12-19T16:42:27.800529+0000","last_active":"2023-12-19T16:42:28.648540+0000","last_peered":"2023-12-19T16:42:28.648540+0000","last_clean":"2023-12-19T16:42:28.648540+0000","last_became_active":"2023-12-19T16:42:27.800250+0000","last_became_peered":"2023-12-19T16:42:27.800250+0000","last_unstale":"2023-12-19T16:42:28.648540+0000","last_undegraded":"2023-12-19T16:42:28.648540+0000","last_fullsized":"2023-12-19T16:42:28.648540+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:27.376917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652139+0000","last_change":"2023-12-19T16:42:27.314367+0000","last_active":"2023-12-19T16:42:29.652139+0000","last_peered":"2023-12-19T16:42:29.652139+0000","last_clean":"2023-12-19T16:42:29.652139+0000","last_became_active":"2023-12-19T16:42:27.314128+0000","last_became_peered":"2023-12-19T16:42:27.314128+0000","last_unstale":"2023-12-19T16:42:29.652139+0000","last_undegraded":"2023-12-19T16:42:29.652139+0000","last_fullsized":"2023-12-19T16:42:29.652139+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:34:59.521223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:29.652110+0000","last_change":"2023-12-19T16:42:27.804285+0000","last_active":"2023-12-19T16:42:29.652110+0000","last_peered":"2023-12-19T16:42:29.652110+0000","last_clean":"2023-12-19T16:42:29.652110+0000","last_became_active":"2023-12-19T16:42:27.804027+0000","last_became_peered":"2023-12-19T16:42:27.804027+0000","last_unstale":"2023-12-19T16:42:29.652110+0000","last_undegraded":"2023-12-19T16:42:29.652110+0000","last_fullsized":"2023-12-19T16:42:29.652110+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:27.284808+0000","last_clean_scrub_stamp":"2023-12-19T16:42:27.284808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:53.498191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-12-19T16:42:28.648557+0000","last_change":"2023-12-19T16:42:27.325512+0000","last_active":"2023-12-19T16:42:28.648557+0000","last_peered":"2023-12-19T16:42:28.648557+0000","last_clean":"2023-12-19T16:42:28.648557+0000","last_became_active":"2023-12-19T16:42:27.325175+0000","last_became_peered":"2023-12-19T16:42:27.325175+0000","last_unstale":"2023-12-19T16:42:28.648557+0000","last_undegraded":"2023-12-19T16:42:28.648557+0000","last_fullsized":"2023-12-19T16:42:28.648557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:26.278479+0000","last_clean_scrub_stamp":"2023-12-19T16:42:26.278479+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:31:30.386546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":831488,"data_stored":671861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6389999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":82475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.5499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":81001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1120000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":81037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]}]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":245760,"data_stored":82475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.821}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0179999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5350000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.956}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9279999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31276,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340564,"statfs":{"total":96636764160,"available":96604737536,"internally_reserved":0,"allocated":233472,"data_stored":80911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5150000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.7590000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8940000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1960000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.165}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.234}]}]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31284,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340556,"statfs":{"total":96636764160,"available":96604729344,"internally_reserved":0,"allocated":241664,"data_stored":81071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3050000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.992}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.133}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1949999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1269999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.093}]}]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":839680,"data_stored":671917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:42:38.708 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-19T16:42:38.708 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-19T16:42:38.708 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-12-19T16:42:38.709 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-12-19T16:42:38.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f94370df640 1 Processor -- start 2023-12-19T16:42:38.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f94370df640 1 -- start start 2023-12-19T16:42:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f94370df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9430155030 0x7f943015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f94370df640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f943015f370 0x7f943006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f94370df640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943006c970 con 0x7f94300ce740 2023-12-19T16:42:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f94370df640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943015b1f0 con 0x7f943015f370 2023-12-19T16:42:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f94370df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943015b4d0 con 0x7f9430155030 2023-12-19T16:42:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f943567c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f94300ce740 0x7f94300ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36118/0 (socket says 172.21.15.26:36118) 2023-12-19T16:42:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f943567c640 1 -- 172.21.15.26:0/4265041262 learned_addr learned my addr 172.21.15.26:0/4265041262 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f942ffff640 1 --2- 172.21.15.26:0/4265041262 >> v2:172.21.15.160:3300/0 conn(0x7f943015f370 0x7f943006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f9434e7b640 1 --2- 172.21.15.26:0/4265041262 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9430155030 0x7f943015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f942f7fe640 1 -- 172.21.15.26:0/4265041262 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128425108 0 0) 0x7f943006c970 con 0x7f94300ce740 2023-12-19T16:42:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f942f7fe640 1 -- 172.21.15.26:0/4265041262 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9420003530 con 0x7f94300ce740 2023-12-19T16:42:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f9434e7b640 1 -- 172.21.15.26:0/4265041262 >> v2:172.21.15.160:3300/0 conn(0x7f943015f370 msgr2=0x7f943006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:38.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f9434e7b640 1 --2- 172.21.15.26:0/4265041262 >> v2:172.21.15.160:3300/0 conn(0x7f943015f370 0x7f943006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.888+0000 7f9434e7b640 1 -- 172.21.15.26:0/4265041262 >> v1:172.21.15.26:6789/0 conn(0x7f94300ce740 legacy=0x7f94300ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.892+0000 7f9434e7b640 1 -- 172.21.15.26:0/4265041262 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943015b7b0 con 0x7f9430155030 2023-12-19T16:42:38.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.892+0000 7f942ffff640 1 --2- 172.21.15.26:0/4265041262 >> v2:172.21.15.160:3300/0 conn(0x7f943015f370 0x7f943006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:38.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.892+0000 7f9434e7b640 1 --2- 172.21.15.26:0/4265041262 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9430155030 0x7f943015bcd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f94240097c0 tx=0x7f942401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d679c32a4e2b690 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:38.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.892+0000 7f942f7fe640 1 -- 172.21.15.26:0/4265041262 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240a6040 con 0x7f9430155030 2023-12-19T16:42:38.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.892+0000 7f942f7fe640 1 -- 172.21.15.26:0/4265041262 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94240aa600 con 0x7f9430155030 2023-12-19T16:42:38.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.892+0000 7f942f7fe640 1 -- 172.21.15.26:0/4265041262 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240aa970 con 0x7f9430155030 2023-12-19T16:42:38.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.892+0000 7f94370df640 1 -- 172.21.15.26:0/4265041262 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9430155030 msgr2=0x7f943015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.892+0000 7f94370df640 1 --2- 172.21.15.26:0/4265041262 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9430155030 0x7f943015bcd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f94240097c0 tx=0x7f942401c480 comp rx=0 tx=0).stop 2023-12-19T16:42:38.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.892+0000 7f94370df640 1 -- 172.21.15.26:0/4265041262 shutdown_connections 2023-12-19T16:42:38.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.892+0000 7f94370df640 1 --2- 172.21.15.26:0/4265041262 >> v2:172.21.15.160:3300/0 conn(0x7f943015f370 0x7f943006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.892+0000 7f94370df640 1 --2- 172.21.15.26:0/4265041262 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9430155030 0x7f943015bcd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.892+0000 7f94370df640 1 -- 172.21.15.26:0/4265041262 >> 172.21.15.26:0/4265041262 conn(0x7f94300658e0 msgr2=0x7f9430065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:38.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f94370df640 1 -- 172.21.15.26:0/4265041262 shutdown_connections 2023-12-19T16:42:38.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f94370df640 1 -- 172.21.15.26:0/4265041262 wait complete. 2023-12-19T16:42:38.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f94370df640 1 Processor -- start 2023-12-19T16:42:38.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f94370df640 1 -- start start 2023-12-19T16:42:38.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f94370df640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94300ce740 0x7f943014a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f94370df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9430155030 0x7f9430154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f94370df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943015f370 0x7f943014c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f94370df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94301549a0 con 0x7f943015f370 2023-12-19T16:42:38.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f94370df640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943014acf0 con 0x7f94300ce740 2023-12-19T16:42:38.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f94370df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9430148ac0 con 0x7f9430155030 2023-12-19T16:42:38.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f943567c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943015f370 0x7f943014c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f943567c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943015f370 0x7f943014c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33418/0 (socket says 172.21.15.26:33418) 2023-12-19T16:42:38.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f943567c640 1 -- 172.21.15.26:0/4040183959 learned_addr learned my addr 172.21.15.26:0/4040183959 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:38.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f9434e7b640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94300ce740 0x7f943014a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f942ffff640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9430155030 0x7f9430154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f943567c640 1 -- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9430155030 msgr2=0x7f9430154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f943567c640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9430155030 0x7f9430154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f943567c640 1 -- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94300ce740 msgr2=0x7f943014a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:38.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f943567c640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94300ce740 0x7f943014a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f943567c640 1 -- 172.21.15.26:0/4040183959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c009420 con 0x7f943015f370 2023-12-19T16:42:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.896+0000 7f943567c640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943015f370 0x7f943014c430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f942800caa0 tx=0x7f942800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.900+0000 7f942d7fa640 1 -- 172.21.15.26:0/4040183959 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942800ac00 con 0x7f943015f370 2023-12-19T16:42:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.900+0000 7f942d7fa640 1 -- 172.21.15.26:0/4040183959 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942800ad60 con 0x7f943015f370 2023-12-19T16:42:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.900+0000 7f94370df640 1 -- 172.21.15.26:0/4040183959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9424009420 con 0x7f943015f370 2023-12-19T16:42:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.900+0000 7f942d7fa640 1 -- 172.21.15.26:0/4040183959 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428012620 con 0x7f943015f370 2023-12-19T16:42:38.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.900+0000 7f94370df640 1 -- 172.21.15.26:0/4040183959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943006c740 con 0x7f943015f370 2023-12-19T16:42:38.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.900+0000 7f94370df640 1 -- 172.21.15.26:0/4040183959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9430074270 con 0x7f943015f370 2023-12-19T16:42:38.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.900+0000 7f942d7fa640 1 -- 172.21.15.26:0/4040183959 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9428007070 con 0x7f943015f370 2023-12-19T16:42:38.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.900+0000 7f942d7fa640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9414065a90 0x7f9414067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:38.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.900+0000 7f942d7fa640 1 -- 172.21.15.26:0/4040183959 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9428083040 con 0x7f943015f370 2023-12-19T16:42:38.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.900+0000 7f9434e7b640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9414065a90 0x7f9414067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:38.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.900+0000 7f9434e7b640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9414065a90 0x7f9414067f50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f94240ae000 tx=0x7f94240b5040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:38.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:38.904+0000 7f942d7fa640 1 -- 172.21.15.26:0/4040183959 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f94280b40a0 con 0x7f943015f370 2023-12-19T16:42:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.092+0000 7f94370df640 1 -- 172.21.15.26:0/4040183959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f94300ccc30 con 0x7f943015f370 2023-12-19T16:42:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.092+0000 7f942d7fa640 1 -- 172.21.15.26:0/4040183959 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9428017080 con 0x7f943015f370 2023-12-19T16:42:39.098 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:39.099 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-19T16:42:39.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.096+0000 7f94370df640 1 -- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9414065a90 msgr2=0x7f9414067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.096+0000 7f94370df640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9414065a90 0x7f9414067f50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f94240ae000 tx=0x7f94240b5040 comp rx=0 tx=0).stop 2023-12-19T16:42:39.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.096+0000 7f94370df640 1 -- 172.21.15.26:0/4040183959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943015f370 msgr2=0x7f943014c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.096+0000 7f94370df640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943015f370 0x7f943014c430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f942800caa0 tx=0x7f942800cf60 comp rx=0 tx=0).stop 2023-12-19T16:42:39.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.096+0000 7f94370df640 1 -- 172.21.15.26:0/4040183959 shutdown_connections 2023-12-19T16:42:39.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.096+0000 7f94370df640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943015f370 0x7f943014c430 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.096+0000 7f94370df640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9414065a90 0x7f9414067f50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.096+0000 7f94370df640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9430155030 0x7f9430154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.096+0000 7f94370df640 1 --2- 172.21.15.26:0/4040183959 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94300ce740 0x7f943014a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.096+0000 7f94370df640 1 -- 172.21.15.26:0/4040183959 >> 172.21.15.26:0/4040183959 conn(0x7f94300658e0 msgr2=0x7f94300b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.096+0000 7f94370df640 1 -- 172.21.15.26:0/4040183959 shutdown_connections 2023-12-19T16:42:39.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.096+0000 7f94370df640 1 -- 172.21.15.26:0/4040183959 wait complete. 2023-12-19T16:42:39.120 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-12-19T16:42:39.120 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:39.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.260+0000 7f1adc9e9640 1 Processor -- start 2023-12-19T16:42:39.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.260+0000 7f1adc9e9640 1 -- start start 2023-12-19T16:42:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.260+0000 7f1adc9e9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1ad815c7f0 0x7f1ad806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.260+0000 7f1adc9e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ad80adb80 0x7f1ad80ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.260+0000 7f1adc9e9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad806ced0 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.260+0000 7f1adc9e9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad806d770 con 0x7f1ad815c7f0 2023-12-19T16:42:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.260+0000 7f1adc9e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad806d1b0 con 0x7f1ad80adb80 2023-12-19T16:42:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.260+0000 7f1acffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ad80adb80 0x7f1ad80ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.260+0000 7f1ad659c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1ad815c7f0 0x7f1ad806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.260+0000 7f1acffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ad80adb80 0x7f1ad80ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:39916/0 (socket says 172.21.15.26:39916) 2023-12-19T16:42:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.260+0000 7f1ad659c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1ad815c7f0 0x7f1ad806c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:57110/0 (socket says 172.21.15.26:57110) 2023-12-19T16:42:39.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.260+0000 7f1acffff640 1 -- 172.21.15.26:0/3694089046 learned_addr learned my addr 172.21.15.26:0/3694089046 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.260+0000 7f1ad6d9d640 1 --1- 172.21.15.26:0/3694089046 >> v1:172.21.15.160:6789/0 conn(0x7f1ad80cc6e0 0x7f1ad80b30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:60950/0 (socket says 172.21.15.26:60950) 2023-12-19T16:42:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.264+0000 7f1acffff640 1 -- 172.21.15.26:0/3694089046 >> v2:172.21.15.160:3300/0 conn(0x7f1ad815c7f0 msgr2=0x7f1ad806c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.264+0000 7f1acffff640 1 --2- 172.21.15.26:0/3694089046 >> v2:172.21.15.160:3300/0 conn(0x7f1ad815c7f0 0x7f1ad806c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.264+0000 7f1ad5d9b640 1 -- 172.21.15.26:0/3694089046 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3015133300 0 0) 0x7f1ad806ced0 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.264+0000 7f1acffff640 1 -- 172.21.15.26:0/3694089046 >> v1:172.21.15.160:6789/0 conn(0x7f1ad80cc6e0 legacy=0x7f1ad80b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.264+0000 7f1acffff640 1 -- 172.21.15.26:0/3694089046 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad806d490 con 0x7f1ad80adb80 2023-12-19T16:42:39.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.264+0000 7f1acffff640 1 --2- 172.21.15.26:0/3694089046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ad80adb80 0x7f1ad80ce7e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009870 tx=0x7f1ac001c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d880b60272711ef server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.264+0000 7f1ad5d9b640 1 -- 172.21.15.26:0/3694089046 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00ab650 con 0x7f1ad80adb80 2023-12-19T16:42:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.264+0000 7f1ad5d9b640 1 -- 172.21.15.26:0/3694089046 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac00ab7b0 con 0x7f1ad80adb80 2023-12-19T16:42:39.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.264+0000 7f1ad5d9b640 1 -- 172.21.15.26:0/3694089046 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00abb40 con 0x7f1ad80adb80 2023-12-19T16:42:39.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.264+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3694089046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ad80adb80 msgr2=0x7f1ad80ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.264+0000 7f1adc9e9640 1 --2- 172.21.15.26:0/3694089046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ad80adb80 0x7f1ad80ce7e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009870 tx=0x7f1ac001c540 comp rx=0 tx=0).stop 2023-12-19T16:42:39.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3694089046 shutdown_connections 2023-12-19T16:42:39.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 --2- 172.21.15.26:0/3694089046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ad80adb80 0x7f1ad80ce7e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 --2- 172.21.15.26:0/3694089046 >> v2:172.21.15.160:3300/0 conn(0x7f1ad815c7f0 0x7f1ad806c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3694089046 >> 172.21.15.26:0/3694089046 conn(0x7f1ad80658e0 msgr2=0x7f1ad8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:39.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3694089046 shutdown_connections 2023-12-19T16:42:39.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3694089046 wait complete. 2023-12-19T16:42:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 Processor -- start 2023-12-19T16:42:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 -- start start 2023-12-19T16:42:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ad80adb80 0x7f1ad8145720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad80cc6e0 0x7f1ad814a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ad815c7f0 0x7f1ad8150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8149620 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8148c80 con 0x7f1ad815c7f0 2023-12-19T16:42:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1adc9e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad8148f60 con 0x7f1ad80adb80 2023-12-19T16:42:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1acffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad80cc6e0 0x7f1ad814a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1acffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad80cc6e0 0x7f1ad814a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33424/0 (socket says 172.21.15.26:33424) 2023-12-19T16:42:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1acffff640 1 -- 172.21.15.26:0/3860212745 learned_addr learned my addr 172.21.15.26:0/3860212745 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1ad659c640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ad80adb80 0x7f1ad8145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1ad6d9d640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ad815c7f0 0x7f1ad8150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1acffff640 1 -- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ad80adb80 msgr2=0x7f1ad8145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1acffff640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ad80adb80 0x7f1ad8145720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1acffff640 1 -- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ad815c7f0 msgr2=0x7f1ad8150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1acffff640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ad815c7f0 0x7f1ad8150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.268+0000 7f1acffff640 1 -- 172.21.15.26:0/3860212745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac00094d0 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.272+0000 7f1acffff640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad80cc6e0 0x7f1ad814a8f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00ad040 tx=0x7f1ac001ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.272+0000 7f1aceffd640 1 -- 172.21.15.26:0/3860212745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00be5f0 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.272+0000 7f1aceffd640 1 -- 172.21.15.26:0/3860212745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac00be770 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.272+0000 7f1aceffd640 1 -- 172.21.15.26:0/3860212745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00beb20 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.272+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3860212745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad8149240 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.272+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3860212745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad814b390 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.272+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3860212745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad8149760 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.272+0000 7f1aceffd640 1 -- 172.21.15.26:0/3860212745 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1ac00b4600 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.272+0000 7f1aceffd640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ab8065a60 0x7f1ab8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.272+0000 7f1aceffd640 1 -- 172.21.15.26:0/3860212745 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ac011e1a0 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.276+0000 7f1ad659c640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ab8065a60 0x7f1ab8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.276+0000 7f1ad659c640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ab8065a60 0x7f1ab8067f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80ce6b0 tx=0x7f1ac800f040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.276+0000 7f1aceffd640 1 -- 172.21.15.26:0/3860212745 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1ac00ebb70 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.440+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3860212745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ad8074270 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.440+0000 7f1aceffd640 1 -- 172.21.15.26:0/3860212745 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f1ac00ef190 con 0x7f1ad80cc6e0 2023-12-19T16:42:39.445 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:39.445 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.444+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ab8065a60 msgr2=0x7f1ab8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.444+0000 7f1adc9e9640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ab8065a60 0x7f1ab8067f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80ce6b0 tx=0x7f1ac800f040 comp rx=0 tx=0).stop 2023-12-19T16:42:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.444+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3860212745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad80cc6e0 msgr2=0x7f1ad814a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.444+0000 7f1adc9e9640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad80cc6e0 0x7f1ad814a8f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00ad040 tx=0x7f1ac001ad80 comp rx=0 tx=0).stop 2023-12-19T16:42:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.444+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3860212745 shutdown_connections 2023-12-19T16:42:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.444+0000 7f1adc9e9640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ad815c7f0 0x7f1ad8150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.444+0000 7f1adc9e9640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ad80cc6e0 0x7f1ad814a8f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.444+0000 7f1adc9e9640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ab8065a60 0x7f1ab8067f20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.444+0000 7f1adc9e9640 1 --2- 172.21.15.26:0/3860212745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ad80adb80 0x7f1ad8145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.444+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3860212745 >> 172.21.15.26:0/3860212745 conn(0x7f1ad80658e0 msgr2=0x7f1ad80b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.444+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3860212745 shutdown_connections 2023-12-19T16:42:39.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.444+0000 7f1adc9e9640 1 -- 172.21.15.26:0/3860212745 wait complete. 2023-12-19T16:42:39.466 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:42:25.598550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:42:26.697362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:42:39.467 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:42:39.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.608+0000 7f0f6508c640 1 Processor -- start 2023-12-19T16:42:39.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.608+0000 7f0f6508c640 1 -- start start 2023-12-19T16:42:39.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.608+0000 7f0f6508c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0f6015ada0 0x7f0f600706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.608+0000 7f0f6508c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f6006ac80 0x7f0f6006b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.608+0000 7f0f6508c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f60070d90 con 0x7f0f6015a9c0 2023-12-19T16:42:39.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.608+0000 7f0f6508c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f600cc270 con 0x7f0f6006ac80 2023-12-19T16:42:39.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.608+0000 7f0f6508c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f60071070 con 0x7f0f6015ada0 2023-12-19T16:42:39.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.612+0000 7f0f5f59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f6006ac80 0x7f0f6006b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.612+0000 7f0f5f59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f6006ac80 0x7f0f6006b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33434/0 (socket says 172.21.15.26:33434) 2023-12-19T16:42:39.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.612+0000 7f0f5f59e640 1 -- 172.21.15.26:0/323133374 learned_addr learned my addr 172.21.15.26:0/323133374 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:39.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.612+0000 7f0f5e59c640 1 --2- 172.21.15.26:0/323133374 >> v2:172.21.15.160:3300/0 conn(0x7f0f6015ada0 0x7f0f600706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.612+0000 7f0f5f59e640 1 -- 172.21.15.26:0/323133374 >> v2:172.21.15.160:3300/0 conn(0x7f0f6015ada0 msgr2=0x7f0f600706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:39.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.612+0000 7f0f5f59e640 1 --2- 172.21.15.26:0/323133374 >> v2:172.21.15.160:3300/0 conn(0x7f0f6015ada0 0x7f0f600706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.612+0000 7f0f5f59e640 1 -- 172.21.15.26:0/323133374 >> v1:172.21.15.160:6789/0 conn(0x7f0f6015a9c0 legacy=0x7f0f600b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:39.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.612+0000 7f0f5f59e640 1 -- 172.21.15.26:0/323133374 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f60071350 con 0x7f0f6006ac80 2023-12-19T16:42:39.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.612+0000 7f0f5f59e640 1 --2- 172.21.15.26:0/323133374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f6006ac80 0x7f0f6006b090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0f600cc3b0 tx=0x7f0f5801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7468b0ff3901b66 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:39.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.612+0000 7f0f5dd9b640 1 -- 172.21.15.26:0/323133374 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f580a57f0 con 0x7f0f6006ac80 2023-12-19T16:42:39.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.612+0000 7f0f5dd9b640 1 -- 172.21.15.26:0/323133374 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f580a5950 con 0x7f0f6006ac80 2023-12-19T16:42:39.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.612+0000 7f0f5dd9b640 1 -- 172.21.15.26:0/323133374 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f580a5ce0 con 0x7f0f6006ac80 2023-12-19T16:42:39.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 -- 172.21.15.26:0/323133374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f6006ac80 msgr2=0x7f0f6006b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 --2- 172.21.15.26:0/323133374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f6006ac80 0x7f0f6006b090 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0f600cc3b0 tx=0x7f0f5801c910 comp rx=0 tx=0).stop 2023-12-19T16:42:39.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 -- 172.21.15.26:0/323133374 shutdown_connections 2023-12-19T16:42:39.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 --2- 172.21.15.26:0/323133374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f6006ac80 0x7f0f6006b090 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 --2- 172.21.15.26:0/323133374 >> v2:172.21.15.160:3300/0 conn(0x7f0f6015ada0 0x7f0f600706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 -- 172.21.15.26:0/323133374 >> 172.21.15.26:0/323133374 conn(0x7f0f600ce730 msgr2=0x7f0f600ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:39.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 -- 172.21.15.26:0/323133374 shutdown_connections 2023-12-19T16:42:39.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 -- 172.21.15.26:0/323133374 wait complete. 2023-12-19T16:42:39.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 Processor -- start 2023-12-19T16:42:39.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 -- start start 2023-12-19T16:42:39.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f6006ac80 0x7f0f600c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f6015a9c0 0x7f0f60155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f6015ada0 0x7f0f601564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f601509e0 con 0x7f0f6015a9c0 2023-12-19T16:42:39.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f600c6490 con 0x7f0f6015ada0 2023-12-19T16:42:39.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f6508c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f600c5910 con 0x7f0f6006ac80 2023-12-19T16:42:39.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f5e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f6015a9c0 0x7f0f60155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f5e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f6015a9c0 0x7f0f60155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33442/0 (socket says 172.21.15.26:33442) 2023-12-19T16:42:39.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f5e59c640 1 -- 172.21.15.26:0/1946526868 learned_addr learned my addr 172.21.15.26:0/1946526868 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:39.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f5f59e640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f6015ada0 0x7f0f601564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.616+0000 7f0f5ed9d640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f6006ac80 0x7f0f600c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.620+0000 7f0f5e59c640 1 -- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f6006ac80 msgr2=0x7f0f600c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.620+0000 7f0f5e59c640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f6006ac80 0x7f0f600c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.620+0000 7f0f5e59c640 1 -- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f6015ada0 msgr2=0x7f0f601564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.620+0000 7f0f5e59c640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f6015ada0 0x7f0f601564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.620+0000 7f0f5e59c640 1 -- 172.21.15.26:0/1946526868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f58009b30 con 0x7f0f6015a9c0 2023-12-19T16:42:39.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.620+0000 7f0f5e59c640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f6015a9c0 0x7f0f60155f60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400d800 tx=0x7f0f5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.620+0000 7f0f477fe640 1 -- 172.21.15.26:0/1946526868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400b660 con 0x7f0f6015a9c0 2023-12-19T16:42:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.620+0000 7f0f477fe640 1 -- 172.21.15.26:0/1946526868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5400b7c0 con 0x7f0f6015a9c0 2023-12-19T16:42:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.620+0000 7f0f477fe640 1 -- 172.21.15.26:0/1946526868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400bb30 con 0x7f0f6015a9c0 2023-12-19T16:42:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.620+0000 7f0f6508c640 1 -- 172.21.15.26:0/1946526868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f600c5bf0 con 0x7f0f6015a9c0 2023-12-19T16:42:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.620+0000 7f0f6508c640 1 -- 172.21.15.26:0/1946526868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f600c61b0 con 0x7f0f6015a9c0 2023-12-19T16:42:39.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.620+0000 7f0f6508c640 1 -- 172.21.15.26:0/1946526868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f600b3c90 con 0x7f0f6015a9c0 2023-12-19T16:42:39.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.620+0000 7f0f477fe640 1 -- 172.21.15.26:0/1946526868 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0f5400bc90 con 0x7f0f6015a9c0 2023-12-19T16:42:39.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.624+0000 7f0f477fe640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f4c065a60 0x7f0f4c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.624+0000 7f0f477fe640 1 -- 172.21.15.26:0/1946526868 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f54083550 con 0x7f0f6015a9c0 2023-12-19T16:42:39.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.624+0000 7f0f5ed9d640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f4c065a60 0x7f0f4c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.624+0000 7f0f5ed9d640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f4c065a60 0x7f0f4c067f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0f50007fd0 tx=0x7f0f50009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:39.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.628+0000 7f0f477fe640 1 -- 172.21.15.26:0/1946526868 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0f54050fb0 con 0x7f0f6015a9c0 2023-12-19T16:42:39.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f6508c640 1 -- 172.21.15.26:0/1946526868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f60154660 con 0x7f0f6015a9c0 2023-12-19T16:42:39.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f477fe640 1 -- 172.21.15.26:0/1946526868 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f540545d0 con 0x7f0f6015a9c0 2023-12-19T16:42:39.791 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:42:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f6508c640 1 -- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f4c065a60 msgr2=0x7f0f4c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f6508c640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f4c065a60 0x7f0f4c067f20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0f50007fd0 tx=0x7f0f50009040 comp rx=0 tx=0).stop 2023-12-19T16:42:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f6508c640 1 -- 172.21.15.26:0/1946526868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f6015a9c0 msgr2=0x7f0f60155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f6508c640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f6015a9c0 0x7f0f60155f60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400d800 tx=0x7f0f5400dcc0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f6508c640 1 -- 172.21.15.26:0/1946526868 shutdown_connections 2023-12-19T16:42:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f6508c640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f6015ada0 0x7f0f601564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f6508c640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f6015a9c0 0x7f0f60155f60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f6508c640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f4c065a60 0x7f0f4c067f20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f6508c640 1 --2- 172.21.15.26:0/1946526868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f6006ac80 0x7f0f600c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f6508c640 1 -- 172.21.15.26:0/1946526868 >> 172.21.15.26:0/1946526868 conn(0x7f0f600ce730 msgr2=0x7f0f600af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f6508c640 1 -- 172.21.15.26:0/1946526868 shutdown_connections 2023-12-19T16:42:39.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.788+0000 7f0f6508c640 1 -- 172.21.15.26:0/1946526868 wait complete. 2023-12-19T16:42:39.812 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:42:39.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.952+0000 7f94908fc640 1 Processor -- start 2023-12-19T16:42:39.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.952+0000 7f94908fc640 1 -- start start 2023-12-19T16:42:39.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f94908fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c161230 0x7f948c0ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f94908fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f948c06c3f0 0x7f948c06c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f94908fc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948c0b01b0 con 0x7f948c164e50 2023-12-19T16:42:39.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f94908fc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c0c9e00 con 0x7f948c06c3f0 2023-12-19T16:42:39.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f94908fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0adfb0 con 0x7f948c161230 2023-12-19T16:42:39.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f948ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f948c164e50 0x7f948c1652e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36124/0 (socket says 172.21.15.26:36124) 2023-12-19T16:42:39.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f948ad9d640 1 -- 172.21.15.26:0/1568453954 learned_addr learned my addr 172.21.15.26:0/1568453954 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:39.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f9483fff640 1 --2- 172.21.15.26:0/1568453954 >> v2:172.21.15.160:3300/0 conn(0x7f948c06c3f0 0x7f948c06c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f948a59c640 1 --2- 172.21.15.26:0/1568453954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c161230 0x7f948c0ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f9489d9b640 1 -- 172.21.15.26:0/1568453954 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147466058 0 0) 0x7f948c0b01b0 con 0x7f948c164e50 2023-12-19T16:42:39.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f9489d9b640 1 -- 172.21.15.26:0/1568453954 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9478003530 con 0x7f948c164e50 2023-12-19T16:42:39.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f9483fff640 1 -- 172.21.15.26:0/1568453954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c161230 msgr2=0x7f948c0ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:39.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f9483fff640 1 --2- 172.21.15.26:0/1568453954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c161230 0x7f948c0ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f9489d9b640 1 -- 172.21.15.26:0/1568453954 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 778526295 0 0) 0x7f9478003530 con 0x7f948c164e50 2023-12-19T16:42:39.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f9483fff640 1 -- 172.21.15.26:0/1568453954 >> v1:172.21.15.26:6789/0 conn(0x7f948c164e50 legacy=0x7f948c1652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.956+0000 7f9483fff640 1 -- 172.21.15.26:0/1568453954 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c0ca4e0 con 0x7f948c06c3f0 2023-12-19T16:42:39.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.960+0000 7f9483fff640 1 --2- 172.21.15.26:0/1568453954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c06c3f0 0x7f948c06c850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f94740097c0 tx=0x7f947401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0e2c79b2db4c66f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:39.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.960+0000 7f9489d9b640 1 -- 172.21.15.26:0/1568453954 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94740ac620 con 0x7f948c06c3f0 2023-12-19T16:42:39.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.960+0000 7f9489d9b640 1 -- 172.21.15.26:0/1568453954 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94740ac780 con 0x7f948c06c3f0 2023-12-19T16:42:39.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.960+0000 7f9489d9b640 1 -- 172.21.15.26:0/1568453954 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94740acaf0 con 0x7f948c06c3f0 2023-12-19T16:42:39.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.960+0000 7f94908fc640 1 -- 172.21.15.26:0/1568453954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c06c3f0 msgr2=0x7f948c06c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.960+0000 7f94908fc640 1 --2- 172.21.15.26:0/1568453954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c06c3f0 0x7f948c06c850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f94740097c0 tx=0x7f947401c3c0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.960+0000 7f94908fc640 1 -- 172.21.15.26:0/1568453954 shutdown_connections 2023-12-19T16:42:39.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.960+0000 7f94908fc640 1 --2- 172.21.15.26:0/1568453954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c06c3f0 0x7f948c06c850 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.960+0000 7f94908fc640 1 --2- 172.21.15.26:0/1568453954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c161230 0x7f948c0ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.960+0000 7f94908fc640 1 -- 172.21.15.26:0/1568453954 >> 172.21.15.26:0/1568453954 conn(0x7f948c0ce730 msgr2=0x7f948c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:39.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.960+0000 7f94908fc640 1 -- 172.21.15.26:0/1568453954 shutdown_connections 2023-12-19T16:42:39.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.960+0000 7f94908fc640 1 -- 172.21.15.26:0/1568453954 wait complete. 2023-12-19T16:42:39.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f94908fc640 1 Processor -- start 2023-12-19T16:42:39.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f94908fc640 1 -- start start 2023-12-19T16:42:39.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f94908fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06c3f0 0x7f948c0b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f94908fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c161230 0x7f948c0b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f94908fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c164e50 0x7f948c0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f94908fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c159b30 con 0x7f948c06c3f0 2023-12-19T16:42:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f94908fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0c6090 con 0x7f948c164e50 2023-12-19T16:42:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f94908fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0c6370 con 0x7f948c161230 2023-12-19T16:42:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f948a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06c3f0 0x7f948c0b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f948a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06c3f0 0x7f948c0b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33450/0 (socket says 172.21.15.26:33450) 2023-12-19T16:42:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f948a59c640 1 -- 172.21.15.26:0/4184555027 learned_addr learned my addr 172.21.15.26:0/4184555027 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f9483fff640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c161230 0x7f948c0b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f948ad9d640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c164e50 0x7f948c0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f948a59c640 1 -- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c161230 msgr2=0x7f948c0b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f948a59c640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c161230 0x7f948c0b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f948a59c640 1 -- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c164e50 msgr2=0x7f948c0b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f948a59c640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c164e50 0x7f948c0b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f948a59c640 1 -- 172.21.15.26:0/4184555027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474009420 con 0x7f948c06c3f0 2023-12-19T16:42:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f9483fff640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c161230 0x7f948c0b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f948ad9d640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c164e50 0x7f948c0b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f948a59c640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06c3f0 0x7f948c0b3680 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f947c00e870 tx=0x7f947c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f9482ffd640 1 -- 172.21.15.26:0/4184555027 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c009510 con 0x7f948c06c3f0 2023-12-19T16:42:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f9482ffd640 1 -- 172.21.15.26:0/4184555027 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c00eec0 con 0x7f948c06c3f0 2023-12-19T16:42:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f9482ffd640 1 -- 172.21.15.26:0/4184555027 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c00cb40 con 0x7f948c06c3f0 2023-12-19T16:42:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f94908fc640 1 -- 172.21.15.26:0/4184555027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948c0c6650 con 0x7f948c06c3f0 2023-12-19T16:42:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f94908fc640 1 -- 172.21.15.26:0/4184555027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c15a190 con 0x7f948c06c3f0 2023-12-19T16:42:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.964+0000 7f94908fc640 1 -- 172.21.15.26:0/4184555027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948c15e680 con 0x7f948c06c3f0 2023-12-19T16:42:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.968+0000 7f9482ffd640 1 -- 172.21.15.26:0/4184555027 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f947c00cca0 con 0x7f948c06c3f0 2023-12-19T16:42:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.968+0000 7f9482ffd640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f946c065a60 0x7f946c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.968+0000 7f9482ffd640 1 -- 172.21.15.26:0/4184555027 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f947c083380 con 0x7f948c06c3f0 2023-12-19T16:42:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.968+0000 7f9483fff640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f946c065a60 0x7f946c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.968+0000 7f9483fff640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f946c065a60 0x7f946c067f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f94740ae000 tx=0x7f94740a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:39.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:39.972+0000 7f9482ffd640 1 -- 172.21.15.26:0/4184555027 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f947c050ca0 con 0x7f948c06c3f0 2023-12-19T16:42:40.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f94908fc640 1 -- 172.21.15.26:0/4184555027 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f948c0b0990 con 0x7f948c06c3f0 2023-12-19T16:42:40.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f9482ffd640 1 -- 172.21.15.26:0/4184555027 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f947c0542c0 con 0x7f948c06c3f0 2023-12-19T16:42:40.135 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:42:40.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f94908fc640 1 -- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f946c065a60 msgr2=0x7f946c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:40.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f94908fc640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f946c065a60 0x7f946c067f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f94740ae000 tx=0x7f94740a7000 comp rx=0 tx=0).stop 2023-12-19T16:42:40.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f94908fc640 1 -- 172.21.15.26:0/4184555027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06c3f0 msgr2=0x7f948c0b3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:40.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f94908fc640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06c3f0 0x7f948c0b3680 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f947c00e870 tx=0x7f947c00ec70 comp rx=0 tx=0).stop 2023-12-19T16:42:40.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f94908fc640 1 -- 172.21.15.26:0/4184555027 shutdown_connections 2023-12-19T16:42:40.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f94908fc640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c164e50 0x7f948c0b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f94908fc640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f946c065a60 0x7f946c067f20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f94908fc640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c161230 0x7f948c0b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f94908fc640 1 --2- 172.21.15.26:0/4184555027 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c06c3f0 0x7f948c0b3680 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f94908fc640 1 -- 172.21.15.26:0/4184555027 >> 172.21.15.26:0/4184555027 conn(0x7f948c0ce730 msgr2=0x7f948c165260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:40.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f94908fc640 1 -- 172.21.15.26:0/4184555027 shutdown_connections 2023-12-19T16:42:40.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.132+0000 7f94908fc640 1 -- 172.21.15.26:0/4184555027 wait complete. 2023-12-19T16:42:40.156 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:42:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.296+0000 7fc343b52640 1 Processor -- start 2023-12-19T16:42:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.296+0000 7fc343b52640 1 -- start start 2023-12-19T16:42:40.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.296+0000 7fc343b52640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc33c159140 0x7fc33c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:40.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.296+0000 7fc343b52640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc33c0cd190 0x7fc33c06ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:40.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc343b52640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33c06f490 con 0x7fc33c06b1c0 2023-12-19T16:42:40.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc343b52640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33c06b650 con 0x7fc33c159140 2023-12-19T16:42:40.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc343b52640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c159520 con 0x7fc33c0cd190 2023-12-19T16:42:40.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc3420ef640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc33c06b1c0 0x7fc33c15ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36140/0 (socket says 172.21.15.26:36140) 2023-12-19T16:42:40.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc3420ef640 1 -- 172.21.15.26:0/66611238 learned_addr learned my addr 172.21.15.26:0/66611238 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:40.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc3410ed640 1 --2- 172.21.15.26:0/66611238 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc33c0cd190 0x7fc33c06ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:40.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc3418ee640 1 --2- 172.21.15.26:0/66611238 >> v2:172.21.15.160:3300/0 conn(0x7fc33c159140 0x7fc33c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:40.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc3408ec640 1 -- 172.21.15.26:0/66611238 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228493306 0 0) 0x7fc33c06f490 con 0x7fc33c06b1c0 2023-12-19T16:42:40.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc3408ec640 1 -- 172.21.15.26:0/66611238 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc328005740 con 0x7fc33c06b1c0 2023-12-19T16:42:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc3410ed640 1 -- 172.21.15.26:0/66611238 >> v2:172.21.15.160:3300/0 conn(0x7fc33c159140 msgr2=0x7fc33c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc3410ed640 1 --2- 172.21.15.26:0/66611238 >> v2:172.21.15.160:3300/0 conn(0x7fc33c159140 0x7fc33c0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc3410ed640 1 -- 172.21.15.26:0/66611238 >> v1:172.21.15.26:6789/0 conn(0x7fc33c06b1c0 legacy=0x7fc33c15ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc3410ed640 1 -- 172.21.15.26:0/66611238 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33c0cd690 con 0x7fc33c0cd190 2023-12-19T16:42:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc3418ee640 1 --2- 172.21.15.26:0/66611238 >> v2:172.21.15.160:3300/0 conn(0x7fc33c159140 0x7fc33c0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:40.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.300+0000 7fc3410ed640 1 --2- 172.21.15.26:0/66611238 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc33c0cd190 0x7fc33c06ac80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00a960 tx=0x7fc32c01c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eab2378589f11ff7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:40.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc3408ec640 1 -- 172.21.15.26:0/66611238 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0a5760 con 0x7fc33c0cd190 2023-12-19T16:42:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc3408ec640 1 -- 172.21.15.26:0/66611238 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c0a58c0 con 0x7fc33c0cd190 2023-12-19T16:42:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc3408ec640 1 -- 172.21.15.26:0/66611238 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0a5c50 con 0x7fc33c0cd190 2023-12-19T16:42:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 -- 172.21.15.26:0/66611238 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc33c0cd190 msgr2=0x7fc33c06ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 --2- 172.21.15.26:0/66611238 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc33c0cd190 0x7fc33c06ac80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc32c00a960 tx=0x7fc32c01c720 comp rx=0 tx=0).stop 2023-12-19T16:42:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 -- 172.21.15.26:0/66611238 shutdown_connections 2023-12-19T16:42:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 --2- 172.21.15.26:0/66611238 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc33c0cd190 0x7fc33c06ac80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 --2- 172.21.15.26:0/66611238 >> v2:172.21.15.160:3300/0 conn(0x7fc33c159140 0x7fc33c0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 -- 172.21.15.26:0/66611238 >> 172.21.15.26:0/66611238 conn(0x7fc33c0ce730 msgr2=0x7fc33c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 -- 172.21.15.26:0/66611238 shutdown_connections 2023-12-19T16:42:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 -- 172.21.15.26:0/66611238 wait complete. 2023-12-19T16:42:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 Processor -- start 2023-12-19T16:42:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 -- start start 2023-12-19T16:42:40.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c06b1c0 0x7fc33c148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc33c0cd190 0x7fc33c0af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc33c159140 0x7fc33c0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c0c2f00 con 0x7fc33c06b1c0 2023-12-19T16:42:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c0c31e0 con 0x7fc33c0cd190 2023-12-19T16:42:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc343b52640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33c0c34c0 con 0x7fc33c159140 2023-12-19T16:42:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc3418ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c06b1c0 0x7fc33c148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc3418ee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c06b1c0 0x7fc33c148580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33458/0 (socket says 172.21.15.26:33458) 2023-12-19T16:42:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.304+0000 7fc3418ee640 1 -- 172.21.15.26:0/2843033038 learned_addr learned my addr 172.21.15.26:0/2843033038 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3420ef640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc33c159140 0x7fc33c0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3418ee640 1 -- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc33c159140 msgr2=0x7fc33c0b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3410ed640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc33c0cd190 0x7fc33c0af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3418ee640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc33c159140 0x7fc33c0b17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3418ee640 1 -- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc33c0cd190 msgr2=0x7fc33c0af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3418ee640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc33c0cd190 0x7fc33c0af6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3418ee640 1 -- 172.21.15.26:0/2843033038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338009420 con 0x7fc33c06b1c0 2023-12-19T16:42:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3420ef640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc33c159140 0x7fc33c0b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3410ed640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc33c0cd190 0x7fc33c0af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3418ee640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c06b1c0 0x7fc33c148580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc33800e980 tx=0x7fc33800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3327fc640 1 -- 172.21.15.26:0/2843033038 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33800ca00 con 0x7fc33c06b1c0 2023-12-19T16:42:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3327fc640 1 -- 172.21.15.26:0/2843033038 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33800cb60 con 0x7fc33c06b1c0 2023-12-19T16:42:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc343b52640 1 -- 172.21.15.26:0/2843033038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32c00a5c0 con 0x7fc33c06b1c0 2023-12-19T16:42:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3327fc640 1 -- 172.21.15.26:0/2843033038 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338013470 con 0x7fc33c06b1c0 2023-12-19T16:42:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc343b52640 1 -- 172.21.15.26:0/2843033038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33c0c4fb0 con 0x7fc33c06b1c0 2023-12-19T16:42:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc343b52640 1 -- 172.21.15.26:0/2843033038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33c074270 con 0x7fc33c06b1c0 2023-12-19T16:42:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3327fc640 1 -- 172.21.15.26:0/2843033038 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc3380135d0 con 0x7fc33c06b1c0 2023-12-19T16:42:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.308+0000 7fc3327fc640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc320065a60 0x7fc320067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.312+0000 7fc3327fc640 1 -- 172.21.15.26:0/2843033038 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc338083760 con 0x7fc33c06b1c0 2023-12-19T16:42:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.312+0000 7fc3410ed640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc320065a60 0x7fc320067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.312+0000 7fc3410ed640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc320065a60 0x7fc320067f20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0af000 tx=0x7fc32c0a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.312+0000 7fc3327fc640 1 -- 172.21.15.26:0/2843033038 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc3380511b0 con 0x7fc33c06b1c0 2023-12-19T16:42:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.468+0000 7fc343b52640 1 -- 172.21.15.26:0/2843033038 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc33c0c3040 con 0x7fc33c06b1c0 2023-12-19T16:42:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.472+0000 7fc3327fc640 1 -- 172.21.15.26:0/2843033038 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc33c0c3040 con 0x7fc33c06b1c0 2023-12-19T16:42:40.475 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:42:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.472+0000 7fc343b52640 1 -- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc320065a60 msgr2=0x7fc320067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.472+0000 7fc343b52640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc320065a60 0x7fc320067f20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0af000 tx=0x7fc32c0a7070 comp rx=0 tx=0).stop 2023-12-19T16:42:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.472+0000 7fc343b52640 1 -- 172.21.15.26:0/2843033038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c06b1c0 msgr2=0x7fc33c148580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.472+0000 7fc343b52640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c06b1c0 0x7fc33c148580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc33800e980 tx=0x7fc33800ee40 comp rx=0 tx=0).stop 2023-12-19T16:42:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.472+0000 7fc343b52640 1 -- 172.21.15.26:0/2843033038 shutdown_connections 2023-12-19T16:42:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.472+0000 7fc343b52640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc320065a60 0x7fc320067f20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.472+0000 7fc343b52640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc33c159140 0x7fc33c0b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.472+0000 7fc343b52640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc33c0cd190 0x7fc33c0af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.472+0000 7fc343b52640 1 --2- 172.21.15.26:0/2843033038 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc33c06b1c0 0x7fc33c148580 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.472+0000 7fc343b52640 1 -- 172.21.15.26:0/2843033038 >> 172.21.15.26:0/2843033038 conn(0x7fc33c0ce730 msgr2=0x7fc33c06f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.472+0000 7fc343b52640 1 -- 172.21.15.26:0/2843033038 shutdown_connections 2023-12-19T16:42:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.472+0000 7fc343b52640 1 -- 172.21.15.26:0/2843033038 wait complete. 2023-12-19T16:42:40.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:42:40.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc96cb6640 1 Processor -- start 2023-12-19T16:42:40.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc96cb6640 1 -- start start 2023-12-19T16:42:40.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc96cb6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdc9015cca0 0x7fdc900b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:40.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc96cb6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc900b8760 0x7fdc900cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:40.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc96cb6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdc900b6b20 0x7fdc900cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc96cb6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc900b7c30 con 0x7fdc9015cca0 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc96cb6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc900b6fb0 con 0x7fdc900b6b20 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc96cb6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc900b7290 con 0x7fdc900b8760 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc95cb4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdc9015cca0 0x7fdc900b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc95cb4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdc9015cca0 0x7fdc900b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33472/0 (socket says 172.21.15.26:33472) 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc95cb4640 1 -- 172.21.15.26:0/1355262198 learned_addr learned my addr 172.21.15.26:0/1355262198 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc954b3640 1 --2- 172.21.15.26:0/1355262198 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc900b8760 0x7fdc900cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc964b5640 1 --2- 172.21.15.26:0/1355262198 >> v2:172.21.15.160:3300/0 conn(0x7fdc900b6b20 0x7fdc900cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc95cb4640 1 -- 172.21.15.26:0/1355262198 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc900b8760 msgr2=0x7fdc900cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc95cb4640 1 --2- 172.21.15.26:0/1355262198 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc900b8760 0x7fdc900cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc95cb4640 1 -- 172.21.15.26:0/1355262198 >> v2:172.21.15.160:3300/0 conn(0x7fdc900b6b20 msgr2=0x7fdc900cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc95cb4640 1 --2- 172.21.15.26:0/1355262198 >> v2:172.21.15.160:3300/0 conn(0x7fdc900b6b20 0x7fdc900cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc95cb4640 1 -- 172.21.15.26:0/1355262198 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc900b7570 con 0x7fdc9015cca0 2023-12-19T16:42:40.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc954b3640 1 --2- 172.21.15.26:0/1355262198 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc900b8760 0x7fdc900cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:40.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.636+0000 7fdc964b5640 1 --2- 172.21.15.26:0/1355262198 >> v2:172.21.15.160:3300/0 conn(0x7fdc900b6b20 0x7fdc900cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:40.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc95cb4640 1 --2- 172.21.15.26:0/1355262198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc9015cca0 0x7fdc900b65e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdc900cd2d0 tx=0x7fdc8c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cd222d1bcb9e3e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:40.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc94cb2640 1 -- 172.21.15.26:0/1355262198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c0aa610 con 0x7fdc9015cca0 2023-12-19T16:42:40.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc94cb2640 1 -- 172.21.15.26:0/1355262198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c0a6040 con 0x7fdc9015cca0 2023-12-19T16:42:40.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc94cb2640 1 -- 172.21.15.26:0/1355262198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c0aa990 con 0x7fdc9015cca0 2023-12-19T16:42:40.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1355262198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc9015cca0 msgr2=0x7fdc900b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:40.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc96cb6640 1 --2- 172.21.15.26:0/1355262198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc9015cca0 0x7fdc900b65e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdc900cd2d0 tx=0x7fdc8c01c4d0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1355262198 shutdown_connections 2023-12-19T16:42:40.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc96cb6640 1 --2- 172.21.15.26:0/1355262198 >> v2:172.21.15.160:3300/0 conn(0x7fdc900b6b20 0x7fdc900cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc96cb6640 1 --2- 172.21.15.26:0/1355262198 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc900b8760 0x7fdc900cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc96cb6640 1 --2- 172.21.15.26:0/1355262198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc9015cca0 0x7fdc900b65e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1355262198 >> 172.21.15.26:0/1355262198 conn(0x7fdc9006a120 msgr2=0x7fdc9006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:40.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1355262198 shutdown_connections 2023-12-19T16:42:40.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1355262198 wait complete. 2023-12-19T16:42:40.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc96cb6640 1 Processor -- start 2023-12-19T16:42:40.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.640+0000 7fdc96cb6640 1 -- start start 2023-12-19T16:42:40.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc96cb6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc900b8760 0x7fdc900b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:40.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc96cb6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9015cca0 0x7fdc9014e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:40.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc96cb6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc9014ea90 0x7fdc900cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:40.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc96cb6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9015a160 con 0x7fdc9014ea90 2023-12-19T16:42:40.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc96cb6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc900b32d0 con 0x7fdc9015cca0 2023-12-19T16:42:40.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc96cb6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc900b35b0 con 0x7fdc900b8760 2023-12-19T16:42:40.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc964b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc9014ea90 0x7fdc900cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:40.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc964b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc9014ea90 0x7fdc900cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33486/0 (socket says 172.21.15.26:33486) 2023-12-19T16:42:40.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc964b5640 1 -- 172.21.15.26:0/1002617974 learned_addr learned my addr 172.21.15.26:0/1002617974 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:40.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc95cb4640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc900b8760 0x7fdc900b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:40.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc954b3640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9015cca0 0x7fdc9014e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc964b5640 1 -- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc900b8760 msgr2=0x7fdc900b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc964b5640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc900b8760 0x7fdc900b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc964b5640 1 -- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9015cca0 msgr2=0x7fdc9014e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc964b5640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9015cca0 0x7fdc9014e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc964b5640 1 -- 172.21.15.26:0/1002617974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c0094d0 con 0x7fdc9014ea90 2023-12-19T16:42:40.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc95cb4640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc900b8760 0x7fdc900b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc954b3640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9015cca0 0x7fdc9014e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc964b5640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc9014ea90 0x7fdc900cbde0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdc88007de0 tx=0x7fdc8800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc867fc640 1 -- 172.21.15.26:0/1002617974 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88003ba0 con 0x7fdc9014ea90 2023-12-19T16:42:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc867fc640 1 -- 172.21.15.26:0/1002617974 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8800f040 con 0x7fdc9014ea90 2023-12-19T16:42:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc867fc640 1 -- 172.21.15.26:0/1002617974 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8800de80 con 0x7fdc9014ea90 2023-12-19T16:42:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1002617974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc900b3890 con 0x7fdc9014ea90 2023-12-19T16:42:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1002617974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9014ef20 con 0x7fdc9014ea90 2023-12-19T16:42:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.644+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1002617974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9015a2d0 con 0x7fdc9014ea90 2023-12-19T16:42:40.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.648+0000 7fdc867fc640 1 -- 172.21.15.26:0/1002617974 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdc88004030 con 0x7fdc9014ea90 2023-12-19T16:42:40.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.648+0000 7fdc867fc640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdc74065a90 0x7fdc74067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:40.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.648+0000 7fdc867fc640 1 -- 172.21.15.26:0/1002617974 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc88083540 con 0x7fdc9014ea90 2023-12-19T16:42:40.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.648+0000 7fdc95cb4640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdc74065a90 0x7fdc74067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:40.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.648+0000 7fdc95cb4640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdc74065a90 0x7fdc74067f50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0ae000 tx=0x7fdc8c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:40.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.652+0000 7fdc867fc640 1 -- 172.21.15.26:0/1002617974 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdc88050ef0 con 0x7fdc9014ea90 2023-12-19T16:42:40.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1002617974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc900670f0 con 0x7fdc9014ea90 2023-12-19T16:42:40.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc867fc640 1 -- 172.21.15.26:0/1002617974 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fdc900670f0 con 0x7fdc9014ea90 2023-12-19T16:42:40.844 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:40.844 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2850479611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2850479611},{"type":"v1","addr":"172.21.15.160:6836","nonce":2850479611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/837780144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":837780144},{"type":"v1","addr":"172.21.15.26:6834","nonce":837780144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2784408519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2784408519},{"type":"v1","addr":"172.21.15.26:6835","nonce":2784408519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:42:27.777197+0000","modified":"2023-12-19T16:42:30.798296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6837/3427345273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3427345273},{"type":"v1","addr":"172.21.15.160:6837","nonce":3427345273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T16:42:40.845 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2023-12-19T16:42:40.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdc74065a90 msgr2=0x7fdc74067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:40.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc96cb6640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdc74065a90 0x7fdc74067f50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c0ae000 tx=0x7fdc8c0a7000 comp rx=0 tx=0).stop 2023-12-19T16:42:40.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1002617974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc9014ea90 msgr2=0x7fdc900cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:40.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc96cb6640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc9014ea90 0x7fdc900cbde0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdc88007de0 tx=0x7fdc8800d4a0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1002617974 shutdown_connections 2023-12-19T16:42:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc96cb6640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc9014ea90 0x7fdc900cbde0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc96cb6640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9015cca0 0x7fdc9014e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc96cb6640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdc74065a90 0x7fdc74067f50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc96cb6640 1 --2- 172.21.15.26:0/1002617974 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc900b8760 0x7fdc900b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1002617974 >> 172.21.15.26:0/1002617974 conn(0x7fdc9006a120 msgr2=0x7fdc900c6510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1002617974 shutdown_connections 2023-12-19T16:42:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:40.840+0000 7fdc96cb6640 1 -- 172.21.15.26:0/1002617974 wait complete. 2023-12-19T16:42:40.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:41.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.004+0000 7f1cc7b7b640 1 Processor -- start 2023-12-19T16:42:41.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.004+0000 7f1cc7b7b640 1 -- start start 2023-12-19T16:42:41.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.004+0000 7f1cc7b7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cc0154220 0x7f1cc015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.004+0000 7f1cc7b7b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc006bc30 con 0x7f1cc00cee10 2023-12-19T16:42:41.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.004+0000 7f1cc7b7b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc00cc6e0 con 0x7f1cc006b3d0 2023-12-19T16:42:41.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.004+0000 7f1cc7b7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc015afc0 con 0x7f1cc0154220 2023-12-19T16:42:41.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.004+0000 7f1cc6118640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1cc00cee10 0x7f1cc015b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36154/0 (socket says 172.21.15.26:36154) 2023-12-19T16:42:41.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.004+0000 7f1cc6118640 1 -- 172.21.15.26:0/2114980729 learned_addr learned my addr 172.21.15.26:0/2114980729 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:41.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc5917640 1 --2- 172.21.15.26:0/2114980729 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cc0154220 0x7f1cc015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1013166173 0 0) 0x7f1cc006bc30 con 0x7f1cc00cee10 2023-12-19T16:42:41.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cac003530 con 0x7f1cc00cee10 2023-12-19T16:42:41.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829861308 0 0) 0x7f1cc00cc6e0 con 0x7f1cc006b3d0 2023-12-19T16:42:41.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc006bc30 con 0x7f1cc006b3d0 2023-12-19T16:42:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 550709944 0 0) 0x7f1cac003530 con 0x7f1cc00cee10 2023-12-19T16:42:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cc0154220 msgr2=0x7f1cc015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 --2- 172.21.15.26:0/2114980729 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cc0154220 0x7f1cc015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 >> v1:172.21.15.160:6789/0 conn(0x7f1cc006b3d0 legacy=0x7f1cc00ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc006b950 con 0x7f1cc00cee10 2023-12-19T16:42:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f1cb8001a00 con 0x7f1cc00cee10 2023-12-19T16:42:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 >> v1:172.21.15.26:6789/0 conn(0x7f1cc00cee10 legacy=0x7f1cc015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 --2- 172.21.15.26:0/2114980729 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cc0154220 0x7f1cac003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 --2- 172.21.15.26:0/2114980729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cac0040a0 0x7f1cac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 --2- 172.21.15.26:0/2114980729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cac004d10 0x7f1cac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc015afc0 con 0x7f1cac004d10 2023-12-19T16:42:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac00dc80 con 0x7f1cac0040a0 2023-12-19T16:42:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac00de40 con 0x7f1cc0154220 2023-12-19T16:42:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc6118640 1 --2- 172.21.15.26:0/2114980729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cac004d10 0x7f1cac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc5917640 1 --2- 172.21.15.26:0/2114980729 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cc0154220 0x7f1cac003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc5116640 1 --2- 172.21.15.26:0/2114980729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cac0040a0 0x7f1cac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc6118640 1 -- 172.21.15.26:0/2114980729 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cc0154220 msgr2=0x7f1cac003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc6118640 1 --2- 172.21.15.26:0/2114980729 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cc0154220 0x7f1cac003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc6118640 1 -- 172.21.15.26:0/2114980729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cac0040a0 msgr2=0x7f1cac0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc6118640 1 --2- 172.21.15.26:0/2114980729 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cac0040a0 0x7f1cac0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.008+0000 7f1cc6118640 1 -- 172.21.15.26:0/2114980729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac003530 con 0x7f1cac004d10 2023-12-19T16:42:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc6118640 1 --2- 172.21.15.26:0/2114980729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cac004d10 0x7f1cac005190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800a7a0 tx=0x7f1cb801fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b190060986bb5e92 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80aa630 con 0x7f1cac004d10 2023-12-19T16:42:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb80aa7b0 con 0x7f1cac004d10 2023-12-19T16:42:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc4915640 1 -- 172.21.15.26:0/2114980729 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80aaba0 con 0x7f1cac004d10 2023-12-19T16:42:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/2114980729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cac004d10 msgr2=0x7f1cac005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 --2- 172.21.15.26:0/2114980729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cac004d10 0x7f1cac005190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800a7a0 tx=0x7f1cb801fa00 comp rx=0 tx=0).stop 2023-12-19T16:42:41.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc6118640 1 -- 172.21.15.26:0/2114980729 reap_dead start 2023-12-19T16:42:41.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/2114980729 shutdown_connections 2023-12-19T16:42:41.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/2114980729 >> 172.21.15.26:0/2114980729 conn(0x7f1cc00658e0 msgr2=0x7f1cc0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:41.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/2114980729 shutdown_connections 2023-12-19T16:42:41.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/2114980729 wait complete. 2023-12-19T16:42:41.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 Processor -- start 2023-12-19T16:42:41.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 -- start start 2023-12-19T16:42:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cac004d10 0x7f1cc014d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cc00703c0 0x7f1cc00b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cc00b1460 0x7f1cc00b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc00b1e20 con 0x7f1cc00b1460 2023-12-19T16:42:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc014d9b0 con 0x7f1cc00703c0 2023-12-19T16:42:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.012+0000 7f1cc7b7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc0070bf0 con 0x7f1cac004d10 2023-12-19T16:42:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc6118640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cc00b1460 0x7f1cc00b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc6118640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cc00b1460 0x7f1cc00b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33508/0 (socket says 172.21.15.26:33508) 2023-12-19T16:42:41.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc6118640 1 -- 172.21.15.26:0/1419745713 learned_addr learned my addr 172.21.15.26:0/1419745713 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc5917640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cac004d10 0x7f1cc014d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc5116640 1 --2- 172.21.15.26:0/1419745713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cc00703c0 0x7f1cc00b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc6118640 1 -- 172.21.15.26:0/1419745713 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cac004d10 msgr2=0x7f1cc014d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc6118640 1 --2- 172.21.15.26:0/1419745713 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cac004d10 0x7f1cc014d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc6118640 1 -- 172.21.15.26:0/1419745713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cc00703c0 msgr2=0x7f1cc00b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc6118640 1 --2- 172.21.15.26:0/1419745713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cc00703c0 0x7f1cc00b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc6118640 1 -- 172.21.15.26:0/1419745713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc0070910 con 0x7f1cc00b1460 2023-12-19T16:42:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc6118640 1 --2- 172.21.15.26:0/1419745713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cc00b1460 0x7f1cc00b18e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80ad040 tx=0x7f1cac005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cb67fc640 1 -- 172.21.15.26:0/1419745713 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80b9070 con 0x7f1cc00b1460 2023-12-19T16:42:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cb67fc640 1 -- 172.21.15.26:0/1419745713 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb8001f60 con 0x7f1cc00b1460 2023-12-19T16:42:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cb67fc640 1 -- 172.21.15.26:0/1419745713 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8001210 con 0x7f1cc00b1460 2023-12-19T16:42:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/1419745713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc0155f40 con 0x7f1cc00b1460 2023-12-19T16:42:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/1419745713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc0156500 con 0x7f1cc00b1460 2023-12-19T16:42:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/1419745713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc0155da0 con 0x7f1cc00b1460 2023-12-19T16:42:41.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cb67fc640 1 -- 172.21.15.26:0/1419745713 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1cb80aa430 con 0x7f1cc00b1460 2023-12-19T16:42:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.016+0000 7f1cb67fc640 1 --2- 172.21.15.26:0/1419745713 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ca4065a90 0x7f1ca4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.020+0000 7f1cb67fc640 1 -- 172.21.15.26:0/1419745713 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cb8120b40 con 0x7f1cc00b1460 2023-12-19T16:42:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.020+0000 7f1cc5917640 1 --2- 172.21.15.26:0/1419745713 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ca4065a90 0x7f1ca4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.020+0000 7f1cc5917640 1 --2- 172.21.15.26:0/1419745713 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ca4065a90 0x7f1ca4067f50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1cc01f12d0 tx=0x7f1cbc009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.020+0000 7f1cb67fc640 1 -- 172.21.15.26:0/1419745713 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1cc0155da0 con 0x7f1cc00b1460 2023-12-19T16:42:41.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.180+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/1419745713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cc0155da0 con 0x7f1cc00b1460 2023-12-19T16:42:41.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.180+0000 7f1cb67fc640 1 -- 172.21.15.26:0/1419745713 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f1cc0155da0 con 0x7f1cc00b1460 2023-12-19T16:42:41.185 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:41.186 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:41.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.184+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/1419745713 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ca4065a90 msgr2=0x7f1ca4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.184+0000 7f1cc7b7b640 1 --2- 172.21.15.26:0/1419745713 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ca4065a90 0x7f1ca4067f50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1cc01f12d0 tx=0x7f1cbc009190 comp rx=0 tx=0).stop 2023-12-19T16:42:41.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.184+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/1419745713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cc00b1460 msgr2=0x7f1cc00b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.184+0000 7f1cc7b7b640 1 --2- 172.21.15.26:0/1419745713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cc00b1460 0x7f1cc00b18e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80ad040 tx=0x7f1cac005800 comp rx=0 tx=0).stop 2023-12-19T16:42:41.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.184+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/1419745713 shutdown_connections 2023-12-19T16:42:41.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.184+0000 7f1cc7b7b640 1 --2- 172.21.15.26:0/1419745713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cc00b1460 0x7f1cc00b18e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.184+0000 7f1cc7b7b640 1 --2- 172.21.15.26:0/1419745713 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cc00703c0 0x7f1cc00b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.184+0000 7f1cc7b7b640 1 --2- 172.21.15.26:0/1419745713 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ca4065a90 0x7f1ca4067f50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.184+0000 7f1cc7b7b640 1 --2- 172.21.15.26:0/1419745713 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cac004d10 0x7f1cc014d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.184+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/1419745713 >> 172.21.15.26:0/1419745713 conn(0x7f1cc00658e0 msgr2=0x7f1cc015a750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:41.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.184+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/1419745713 shutdown_connections 2023-12-19T16:42:41.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.184+0000 7f1cc7b7b640 1 -- 172.21.15.26:0/1419745713 wait complete. 2023-12-19T16:42:41.207 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:42:25.598550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:42:26.697362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:42:41.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:42:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.348+0000 7f0e4102d640 1 Processor -- start 2023-12-19T16:42:41.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.348+0000 7f0e4102d640 1 -- start start 2023-12-19T16:42:41.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e4102d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c15ada0 0x7f0e3c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e4102d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3c070d90 con 0x7f0e3c15a9c0 2023-12-19T16:42:41.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e4102d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3c071070 con 0x7f0e3c06ac80 2023-12-19T16:42:41.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e4102d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c0cc270 con 0x7f0e3c15ada0 2023-12-19T16:42:41.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e3ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0e3c15a9c0 0x7f0e3c0b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36160/0 (socket says 172.21.15.26:36160) 2023-12-19T16:42:41.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e3ad9d640 1 -- 172.21.15.26:0/2671707780 learned_addr learned my addr 172.21.15.26:0/2671707780 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:41.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e3a59c640 1 --2- 172.21.15.26:0/2671707780 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c15ada0 0x7f0e3c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e39d9b640 1 -- 172.21.15.26:0/2671707780 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371451372 0 0) 0x7f0e3c070d90 con 0x7f0e3c15a9c0 2023-12-19T16:42:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e39d9b640 1 -- 172.21.15.26:0/2671707780 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e24003530 con 0x7f0e3c15a9c0 2023-12-19T16:42:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e39d9b640 1 -- 172.21.15.26:0/2671707780 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228884666 0 0) 0x7f0e3c071070 con 0x7f0e3c06ac80 2023-12-19T16:42:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e39d9b640 1 -- 172.21.15.26:0/2671707780 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e3c070d90 con 0x7f0e3c06ac80 2023-12-19T16:42:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e3a59c640 1 -- 172.21.15.26:0/2671707780 >> v1:172.21.15.160:6789/0 conn(0x7f0e3c06ac80 legacy=0x7f0e3c0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e3a59c640 1 -- 172.21.15.26:0/2671707780 >> v1:172.21.15.26:6789/0 conn(0x7f0e3c15a9c0 legacy=0x7f0e3c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e3a59c640 1 -- 172.21.15.26:0/2671707780 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c071350 con 0x7f0e3c15ada0 2023-12-19T16:42:41.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.352+0000 7f0e3a59c640 1 --2- 172.21.15.26:0/2671707780 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c15ada0 0x7f0e3c0706d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e300097f0 tx=0x7f0e3001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad7b6f9ccfce5986 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:41.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e39d9b640 1 -- 172.21.15.26:0/2671707780 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e300ab620 con 0x7f0e3c15ada0 2023-12-19T16:42:41.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e39d9b640 1 -- 172.21.15.26:0/2671707780 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e300ab780 con 0x7f0e3c15ada0 2023-12-19T16:42:41.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e39d9b640 1 -- 172.21.15.26:0/2671707780 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e300abaf0 con 0x7f0e3c15ada0 2023-12-19T16:42:41.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 -- 172.21.15.26:0/2671707780 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c15ada0 msgr2=0x7f0e3c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 --2- 172.21.15.26:0/2671707780 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c15ada0 0x7f0e3c0706d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e300097f0 tx=0x7f0e3001c400 comp rx=0 tx=0).stop 2023-12-19T16:42:41.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 -- 172.21.15.26:0/2671707780 shutdown_connections 2023-12-19T16:42:41.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 --2- 172.21.15.26:0/2671707780 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c15ada0 0x7f0e3c0706d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 -- 172.21.15.26:0/2671707780 >> 172.21.15.26:0/2671707780 conn(0x7f0e3c0ce730 msgr2=0x7f0e3c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:41.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 -- 172.21.15.26:0/2671707780 shutdown_connections 2023-12-19T16:42:41.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 -- 172.21.15.26:0/2671707780 wait complete. 2023-12-19T16:42:41.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 Processor -- start 2023-12-19T16:42:41.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 -- start start 2023-12-19T16:42:41.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c06ac80 0x7f0e3c1513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c15a9c0 0x7f0e3c0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c15ada0 0x7f0e3c0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c15d9e0 con 0x7f0e3c06ac80 2023-12-19T16:42:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c151f40 con 0x7f0e3c15a9c0 2023-12-19T16:42:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e4102d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c15ce60 con 0x7f0e3c15ada0 2023-12-19T16:42:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e3ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c06ac80 0x7f0e3c1513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e3ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c06ac80 0x7f0e3c1513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33524/0 (socket says 172.21.15.26:33524) 2023-12-19T16:42:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.356+0000 7f0e3ad9d640 1 -- 172.21.15.26:0/1778528340 learned_addr learned my addr 172.21.15.26:0/1778528340 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e3b59e640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c15ada0 0x7f0e3c0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e3a59c640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c15a9c0 0x7f0e3c0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e3ad9d640 1 -- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c15ada0 msgr2=0x7f0e3c0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e3ad9d640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c15ada0 0x7f0e3c0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e3ad9d640 1 -- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c15a9c0 msgr2=0x7f0e3c0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e3ad9d640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c15a9c0 0x7f0e3c0ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e3ad9d640 1 -- 172.21.15.26:0/1778528340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e30009420 con 0x7f0e3c06ac80 2023-12-19T16:42:41.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e3ad9d640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c06ac80 0x7f0e3c1513e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00c4d0 tx=0x7f0e2c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:41.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e237fe640 1 -- 172.21.15.26:0/1778528340 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c005cf0 con 0x7f0e3c06ac80 2023-12-19T16:42:41.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e237fe640 1 -- 172.21.15.26:0/1778528340 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c005e50 con 0x7f0e3c06ac80 2023-12-19T16:42:41.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e237fe640 1 -- 172.21.15.26:0/1778528340 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0072b0 con 0x7f0e3c06ac80 2023-12-19T16:42:41.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e4102d640 1 -- 172.21.15.26:0/1778528340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c15d140 con 0x7f0e3c06ac80 2023-12-19T16:42:41.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e4102d640 1 -- 172.21.15.26:0/1778528340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c15d700 con 0x7f0e3c06ac80 2023-12-19T16:42:41.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e4102d640 1 -- 172.21.15.26:0/1778528340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3c074270 con 0x7f0e3c06ac80 2023-12-19T16:42:41.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.360+0000 7f0e237fe640 1 -- 172.21.15.26:0/1778528340 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0e2c007410 con 0x7f0e3c06ac80 2023-12-19T16:42:41.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.364+0000 7f0e237fe640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e28065a60 0x7f0e28067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.364+0000 7f0e237fe640 1 -- 172.21.15.26:0/1778528340 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e2c0826f0 con 0x7f0e3c06ac80 2023-12-19T16:42:41.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.364+0000 7f0e3a59c640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e28065a60 0x7f0e28067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.364+0000 7f0e3a59c640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e28065a60 0x7f0e28067f20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0e300af000 tx=0x7f0e300a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:41.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.364+0000 7f0e237fe640 1 -- 172.21.15.26:0/1778528340 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0e2c050160 con 0x7f0e3c06ac80 2023-12-19T16:42:41.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.524+0000 7f0e4102d640 1 -- 172.21.15.26:0/1778528340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e3c0c54e0 con 0x7f0e3c06ac80 2023-12-19T16:42:41.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.524+0000 7f0e237fe640 1 -- 172.21.15.26:0/1778528340 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e2c0170a0 con 0x7f0e3c06ac80 2023-12-19T16:42:41.528 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:42:41.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.524+0000 7f0e4102d640 1 -- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e28065a60 msgr2=0x7f0e28067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.524+0000 7f0e4102d640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e28065a60 0x7f0e28067f20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0e300af000 tx=0x7f0e300a7000 comp rx=0 tx=0).stop 2023-12-19T16:42:41.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.524+0000 7f0e4102d640 1 -- 172.21.15.26:0/1778528340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c06ac80 msgr2=0x7f0e3c1513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.524+0000 7f0e4102d640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c06ac80 0x7f0e3c1513e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00c4d0 tx=0x7f0e2c00c990 comp rx=0 tx=0).stop 2023-12-19T16:42:41.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.524+0000 7f0e4102d640 1 -- 172.21.15.26:0/1778528340 shutdown_connections 2023-12-19T16:42:41.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.524+0000 7f0e4102d640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e28065a60 0x7f0e28067f20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.524+0000 7f0e4102d640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e3c15ada0 0x7f0e3c0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.524+0000 7f0e4102d640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e3c15a9c0 0x7f0e3c0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.524+0000 7f0e4102d640 1 --2- 172.21.15.26:0/1778528340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e3c06ac80 0x7f0e3c1513e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.524+0000 7f0e4102d640 1 -- 172.21.15.26:0/1778528340 >> 172.21.15.26:0/1778528340 conn(0x7f0e3c0ce730 msgr2=0x7f0e3c0afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:41.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.524+0000 7f0e4102d640 1 -- 172.21.15.26:0/1778528340 shutdown_connections 2023-12-19T16:42:41.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.528+0000 7f0e4102d640 1 -- 172.21.15.26:0/1778528340 wait complete. 2023-12-19T16:42:41.550 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:42:41.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d256df640 1 Processor -- start 2023-12-19T16:42:41.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d256df640 1 -- start start 2023-12-19T16:42:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d256df640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8d2015ada0 0x7f8d200706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d256df640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d2006ac80 0x7f8d2006b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d256df640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d20070d90 con 0x7f8d2015a9c0 2023-12-19T16:42:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d256df640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d200cc270 con 0x7f8d2006ac80 2023-12-19T16:42:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d256df640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d20071070 con 0x7f8d2015ada0 2023-12-19T16:42:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d1f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d2006ac80 0x7f8d2006b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d1f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d2006ac80 0x7f8d2006b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33536/0 (socket says 172.21.15.26:33536) 2023-12-19T16:42:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d1f7fe640 1 -- 172.21.15.26:0/1691267461 learned_addr learned my addr 172.21.15.26:0/1691267461 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d17fff640 1 --2- 172.21.15.26:0/1691267461 >> v2:172.21.15.160:3300/0 conn(0x7f8d2015ada0 0x7f8d200706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d1f7fe640 1 -- 172.21.15.26:0/1691267461 >> v2:172.21.15.160:3300/0 conn(0x7f8d2015ada0 msgr2=0x7f8d200706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d1f7fe640 1 --2- 172.21.15.26:0/1691267461 >> v2:172.21.15.160:3300/0 conn(0x7f8d2015ada0 0x7f8d200706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d1f7fe640 1 -- 172.21.15.26:0/1691267461 >> v1:172.21.15.160:6789/0 conn(0x7f8d2015a9c0 legacy=0x7f8d200b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:41.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.692+0000 7f8d1f7fe640 1 -- 172.21.15.26:0/1691267461 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d20071350 con 0x7f8d2006ac80 2023-12-19T16:42:41.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.696+0000 7f8d1f7fe640 1 --2- 172.21.15.26:0/1691267461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d2006ac80 0x7f8d2006b090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8d200cc3b0 tx=0x7f8d1801c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2832f0242f9c82f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:41.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.696+0000 7f8d1e7fc640 1 -- 172.21.15.26:0/1691267461 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d180a57b0 con 0x7f8d2006ac80 2023-12-19T16:42:41.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.696+0000 7f8d1e7fc640 1 -- 172.21.15.26:0/1691267461 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d180a5910 con 0x7f8d2006ac80 2023-12-19T16:42:41.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.696+0000 7f8d1e7fc640 1 -- 172.21.15.26:0/1691267461 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d180a5ca0 con 0x7f8d2006ac80 2023-12-19T16:42:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.696+0000 7f8d256df640 1 -- 172.21.15.26:0/1691267461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d2006ac80 msgr2=0x7f8d2006b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.696+0000 7f8d256df640 1 --2- 172.21.15.26:0/1691267461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d2006ac80 0x7f8d2006b090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8d200cc3b0 tx=0x7f8d1801c8d0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.696+0000 7f8d256df640 1 -- 172.21.15.26:0/1691267461 shutdown_connections 2023-12-19T16:42:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.696+0000 7f8d256df640 1 --2- 172.21.15.26:0/1691267461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d2006ac80 0x7f8d2006b090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.696+0000 7f8d256df640 1 --2- 172.21.15.26:0/1691267461 >> v2:172.21.15.160:3300/0 conn(0x7f8d2015ada0 0x7f8d200706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.696+0000 7f8d256df640 1 -- 172.21.15.26:0/1691267461 >> 172.21.15.26:0/1691267461 conn(0x7f8d200ce730 msgr2=0x7f8d200ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.696+0000 7f8d256df640 1 -- 172.21.15.26:0/1691267461 shutdown_connections 2023-12-19T16:42:41.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.696+0000 7f8d256df640 1 -- 172.21.15.26:0/1691267461 wait complete. 2023-12-19T16:42:41.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d256df640 1 Processor -- start 2023-12-19T16:42:41.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d256df640 1 -- start start 2023-12-19T16:42:41.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d256df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d2006ac80 0x7f8d200b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d256df640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d2015a9c0 0x7f8d200c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d256df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8d2015ada0 0x7f8d20150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d256df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200c72a0 con 0x7f8d2006ac80 2023-12-19T16:42:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d256df640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200c5cc0 con 0x7f8d2015a9c0 2023-12-19T16:42:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d256df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200c5fa0 con 0x7f8d2015ada0 2023-12-19T16:42:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d1effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d2006ac80 0x7f8d200b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d1effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d2006ac80 0x7f8d200b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33552/0 (socket says 172.21.15.26:33552) 2023-12-19T16:42:41.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d1effd640 1 -- 172.21.15.26:0/3168015661 learned_addr learned my addr 172.21.15.26:0/3168015661 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:41.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d1f7fe640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8d2015ada0 0x7f8d20150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d17fff640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d2015a9c0 0x7f8d200c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d1effd640 1 -- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8d2015ada0 msgr2=0x7f8d20150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d1effd640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8d2015ada0 0x7f8d20150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d1effd640 1 -- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d2015a9c0 msgr2=0x7f8d200c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d1effd640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d2015a9c0 0x7f8d200c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d1effd640 1 -- 172.21.15.26:0/3168015661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d18009b30 con 0x7f8d2006ac80 2023-12-19T16:42:41.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d1effd640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d2006ac80 0x7f8d200b2580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000c450 tx=0x7f8d1000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:41.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d177fe640 1 -- 172.21.15.26:0/3168015661 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10012830 con 0x7f8d2006ac80 2023-12-19T16:42:41.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d177fe640 1 -- 172.21.15.26:0/3168015661 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d100129b0 con 0x7f8d2006ac80 2023-12-19T16:42:41.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d177fe640 1 -- 172.21.15.26:0/3168015661 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10012d60 con 0x7f8d2006ac80 2023-12-19T16:42:41.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d256df640 1 -- 172.21.15.26:0/3168015661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d200c6280 con 0x7f8d2006ac80 2023-12-19T16:42:41.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.700+0000 7f8d256df640 1 -- 172.21.15.26:0/3168015661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d200c6840 con 0x7f8d2006ac80 2023-12-19T16:42:41.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.704+0000 7f8d256df640 1 -- 172.21.15.26:0/3168015661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d200c6a60 con 0x7f8d2006ac80 2023-12-19T16:42:41.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.704+0000 7f8d177fe640 1 -- 172.21.15.26:0/3168015661 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8d1001c460 con 0x7f8d2006ac80 2023-12-19T16:42:41.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.704+0000 7f8d177fe640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8d00065a60 0x7f8d00067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:41.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.704+0000 7f8d177fe640 1 -- 172.21.15.26:0/3168015661 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d100830d0 con 0x7f8d2006ac80 2023-12-19T16:42:41.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.704+0000 7f8d17fff640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8d00065a60 0x7f8d00067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:41.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.704+0000 7f8d17fff640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8d00065a60 0x7f8d00067f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8d200c8600 tx=0x7f8d08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:41.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.708+0000 7f8d177fe640 1 -- 172.21.15.26:0/3168015661 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8d10050960 con 0x7f8d2006ac80 2023-12-19T16:42:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.864+0000 7f8d256df640 1 -- 172.21.15.26:0/3168015661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d200ade30 con 0x7f8d2006ac80 2023-12-19T16:42:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.864+0000 7f8d177fe640 1 -- 172.21.15.26:0/3168015661 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d10053f80 con 0x7f8d2006ac80 2023-12-19T16:42:41.869 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:42:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.868+0000 7f8d256df640 1 -- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8d00065a60 msgr2=0x7f8d00067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.868+0000 7f8d256df640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8d00065a60 0x7f8d00067f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8d200c8600 tx=0x7f8d08008040 comp rx=0 tx=0).stop 2023-12-19T16:42:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.868+0000 7f8d256df640 1 -- 172.21.15.26:0/3168015661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d2006ac80 msgr2=0x7f8d200b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.868+0000 7f8d256df640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d2006ac80 0x7f8d200b2580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000c450 tx=0x7f8d1000c910 comp rx=0 tx=0).stop 2023-12-19T16:42:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.868+0000 7f8d256df640 1 -- 172.21.15.26:0/3168015661 shutdown_connections 2023-12-19T16:42:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.868+0000 7f8d256df640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8d00065a60 0x7f8d00067f20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.868+0000 7f8d256df640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8d2015ada0 0x7f8d20150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.868+0000 7f8d256df640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d2015a9c0 0x7f8d200c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.868+0000 7f8d256df640 1 --2- 172.21.15.26:0/3168015661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d2006ac80 0x7f8d200b2580 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.868+0000 7f8d256df640 1 -- 172.21.15.26:0/3168015661 >> 172.21.15.26:0/3168015661 conn(0x7f8d200ce730 msgr2=0x7f8d200af950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.868+0000 7f8d256df640 1 -- 172.21.15.26:0/3168015661 shutdown_connections 2023-12-19T16:42:41.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:41.868+0000 7f8d256df640 1 -- 172.21.15.26:0/3168015661 wait complete. 2023-12-19T16:42:41.890 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:42:42.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae8d4b6640 1 Processor -- start 2023-12-19T16:42:42.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae8d4b6640 1 -- start start 2023-12-19T16:42:42.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae8d4b6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fae8815d0d0 0x7fae880b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae8d4b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae880b4600 0x7fae880b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae8d4b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae880b4f50 con 0x7fae8815ccf0 2023-12-19T16:42:42.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae8d4b6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8815e200 con 0x7fae8815d0d0 2023-12-19T16:42:42.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae8d4b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae880d0aa0 con 0x7fae880b4600 2023-12-19T16:42:42.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae86ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae8815ccf0 0x7fae880d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36164/0 (socket says 172.21.15.26:36164) 2023-12-19T16:42:42.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae86ffd640 1 -- 172.21.15.26:0/3280372697 learned_addr learned my addr 172.21.15.26:0/3280372697 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:42.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae7ffff640 1 --2- 172.21.15.26:0/3280372697 >> v2:172.21.15.160:3300/0 conn(0x7fae8815d0d0 0x7fae880b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae877fe640 1 --2- 172.21.15.26:0/3280372697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae880b4600 0x7fae880b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae867fc640 1 -- 172.21.15.26:0/3280372697 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087655504 0 0) 0x7fae880b4f50 con 0x7fae8815ccf0 2023-12-19T16:42:42.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae867fc640 1 -- 172.21.15.26:0/3280372697 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae74003530 con 0x7fae8815ccf0 2023-12-19T16:42:42.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.032+0000 7fae867fc640 1 -- 172.21.15.26:0/3280372697 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3661842639 0 0) 0x7fae74003530 con 0x7fae8815ccf0 2023-12-19T16:42:42.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae877fe640 1 -- 172.21.15.26:0/3280372697 >> v2:172.21.15.160:3300/0 conn(0x7fae8815d0d0 msgr2=0x7fae880b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:42.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae877fe640 1 --2- 172.21.15.26:0/3280372697 >> v2:172.21.15.160:3300/0 conn(0x7fae8815d0d0 0x7fae880b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae877fe640 1 -- 172.21.15.26:0/3280372697 >> v1:172.21.15.26:6789/0 conn(0x7fae8815ccf0 legacy=0x7fae880d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae877fe640 1 -- 172.21.15.26:0/3280372697 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8815d680 con 0x7fae880b4600 2023-12-19T16:42:42.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae7ffff640 1 --2- 172.21.15.26:0/3280372697 >> v2:172.21.15.160:3300/0 conn(0x7fae8815d0d0 0x7fae880b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:42.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae877fe640 1 --2- 172.21.15.26:0/3280372697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae880b4600 0x7fae880b4a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae80009d90 tx=0x7fae8001c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc59db6dad801ee server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:42.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae867fc640 1 -- 172.21.15.26:0/3280372697 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae78001680 con 0x7fae880b4600 2023-12-19T16:42:42.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae867fc640 1 -- 172.21.15.26:0/3280372697 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae800a5640 con 0x7fae880b4600 2023-12-19T16:42:42.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae867fc640 1 -- 172.21.15.26:0/3280372697 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae800a59b0 con 0x7fae880b4600 2023-12-19T16:42:42.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae8d4b6640 1 -- 172.21.15.26:0/3280372697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae880b4600 msgr2=0x7fae880b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae8d4b6640 1 --2- 172.21.15.26:0/3280372697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae880b4600 0x7fae880b4a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fae80009d90 tx=0x7fae8001c840 comp rx=0 tx=0).stop 2023-12-19T16:42:42.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae8d4b6640 1 -- 172.21.15.26:0/3280372697 shutdown_connections 2023-12-19T16:42:42.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae8d4b6640 1 --2- 172.21.15.26:0/3280372697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae880b4600 0x7fae880b4a10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae8d4b6640 1 --2- 172.21.15.26:0/3280372697 >> v2:172.21.15.160:3300/0 conn(0x7fae8815d0d0 0x7fae880b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.036+0000 7fae8d4b6640 1 -- 172.21.15.26:0/3280372697 >> 172.21.15.26:0/3280372697 conn(0x7fae88065ee0 msgr2=0x7fae880cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:42.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae8d4b6640 1 -- 172.21.15.26:0/3280372697 shutdown_connections 2023-12-19T16:42:42.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae8d4b6640 1 -- 172.21.15.26:0/3280372697 wait complete. 2023-12-19T16:42:42.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae8d4b6640 1 Processor -- start 2023-12-19T16:42:42.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae8d4b6640 1 -- start start 2023-12-19T16:42:42.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae8d4b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae880b4600 0x7fae8815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae8d4b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8815ccf0 0x7fae88153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae8d4b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae8815d0d0 0x7fae880cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae8d4b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae88158190 con 0x7fae8815ccf0 2023-12-19T16:42:42.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae8d4b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae880c8850 con 0x7fae8815d0d0 2023-12-19T16:42:42.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae8d4b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae880c8b30 con 0x7fae880b4600 2023-12-19T16:42:42.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8815ccf0 0x7fae88153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae7ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8815ccf0 0x7fae88153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33564/0 (socket says 172.21.15.26:33564) 2023-12-19T16:42:42.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae7ffff640 1 -- 172.21.15.26:0/2438090745 learned_addr learned my addr 172.21.15.26:0/2438090745 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:42.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae877fe640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae8815d0d0 0x7fae880cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae86ffd640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae880b4600 0x7fae8815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae7ffff640 1 -- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae880b4600 msgr2=0x7fae8815bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae7ffff640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae880b4600 0x7fae8815bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae7ffff640 1 -- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae8815d0d0 msgr2=0x7fae880cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae7ffff640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae8815d0d0 0x7fae880cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae7ffff640 1 -- 172.21.15.26:0/2438090745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae70009420 con 0x7fae8815ccf0 2023-12-19T16:42:42.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae86ffd640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae880b4600 0x7fae8815bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:42.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae7ffff640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8815ccf0 0x7fae88153bd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae7000e980 tx=0x7fae7000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:42.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.040+0000 7fae7f7fe640 1 -- 172.21.15.26:0/2438090745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7000ca00 con 0x7fae8815ccf0 2023-12-19T16:42:42.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.044+0000 7fae7f7fe640 1 -- 172.21.15.26:0/2438090745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7000cb60 con 0x7fae8815ccf0 2023-12-19T16:42:42.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.044+0000 7fae8d4b6640 1 -- 172.21.15.26:0/2438090745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae80009a70 con 0x7fae8815ccf0 2023-12-19T16:42:42.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.044+0000 7fae7f7fe640 1 -- 172.21.15.26:0/2438090745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70013470 con 0x7fae8815ccf0 2023-12-19T16:42:42.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.044+0000 7fae8d4b6640 1 -- 172.21.15.26:0/2438090745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae88157e20 con 0x7fae8815ccf0 2023-12-19T16:42:42.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.044+0000 7fae8d4b6640 1 -- 172.21.15.26:0/2438090745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae88074270 con 0x7fae8815ccf0 2023-12-19T16:42:42.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.044+0000 7fae7f7fe640 1 -- 172.21.15.26:0/2438090745 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fae700135d0 con 0x7fae8815ccf0 2023-12-19T16:42:42.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.044+0000 7fae7f7fe640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae68065a60 0x7fae68067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.044+0000 7fae7f7fe640 1 -- 172.21.15.26:0/2438090745 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae700837e0 con 0x7fae8815ccf0 2023-12-19T16:42:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.044+0000 7fae86ffd640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae68065a60 0x7fae68067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.048+0000 7fae86ffd640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae68065a60 0x7fae68067f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fae78005890 tx=0x7fae78009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.048+0000 7fae7f7fe640 1 -- 172.21.15.26:0/2438090745 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fae700511b0 con 0x7fae8815ccf0 2023-12-19T16:42:42.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.204+0000 7fae8d4b6640 1 -- 172.21.15.26:0/2438090745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae880cff20 con 0x7fae8815ccf0 2023-12-19T16:42:42.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.204+0000 7fae7f7fe640 1 -- 172.21.15.26:0/2438090745 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae700547d0 con 0x7fae8815ccf0 2023-12-19T16:42:42.210 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:42:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.208+0000 7fae8d4b6640 1 -- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae68065a60 msgr2=0x7fae68067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.208+0000 7fae8d4b6640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae68065a60 0x7fae68067f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fae78005890 tx=0x7fae78009040 comp rx=0 tx=0).stop 2023-12-19T16:42:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.208+0000 7fae8d4b6640 1 -- 172.21.15.26:0/2438090745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8815ccf0 msgr2=0x7fae88153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.208+0000 7fae8d4b6640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8815ccf0 0x7fae88153bd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae7000e980 tx=0x7fae7000ee40 comp rx=0 tx=0).stop 2023-12-19T16:42:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.208+0000 7fae8d4b6640 1 -- 172.21.15.26:0/2438090745 shutdown_connections 2023-12-19T16:42:42.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.208+0000 7fae8d4b6640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae8815d0d0 0x7fae880cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.208+0000 7fae8d4b6640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8815ccf0 0x7fae88153bd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.208+0000 7fae8d4b6640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae68065a60 0x7fae68067f20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.208+0000 7fae8d4b6640 1 --2- 172.21.15.26:0/2438090745 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae880b4600 0x7fae8815bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.208+0000 7fae8d4b6640 1 -- 172.21.15.26:0/2438090745 >> 172.21.15.26:0/2438090745 conn(0x7fae88065ee0 msgr2=0x7fae880d1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:42.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.208+0000 7fae8d4b6640 1 -- 172.21.15.26:0/2438090745 shutdown_connections 2023-12-19T16:42:42.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.208+0000 7fae8d4b6640 1 -- 172.21.15.26:0/2438090745 wait complete. 2023-12-19T16:42:42.232 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:42:42.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb990266640 1 Processor -- start 2023-12-19T16:42:42.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb990266640 1 -- start start 2023-12-19T16:42:42.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb990266640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb98815c7f0 0x7fb98806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb990266640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb9880b30d0 0x7fb9880b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb990266640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9880b3a20 con 0x7fb9880adb80 2023-12-19T16:42:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb990266640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98806d770 con 0x7fb98815c7f0 2023-12-19T16:42:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb990266640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98806ced0 con 0x7fb9880b30d0 2023-12-19T16:42:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb98e002640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb98815c7f0 0x7fb98806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb98d801640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb9880adb80 0x7fb9880cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36176/0 (socket says 172.21.15.26:36176) 2023-12-19T16:42:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb98d801640 1 -- 172.21.15.26:0/1889463602 learned_addr learned my addr 172.21.15.26:0/1889463602 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb98e803640 1 --2- 172.21.15.26:0/1889463602 >> v2:172.21.15.160:3300/0 conn(0x7fb9880b30d0 0x7fb9880b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb98e002640 1 -- 172.21.15.26:0/1889463602 >> v2:172.21.15.160:3300/0 conn(0x7fb9880b30d0 msgr2=0x7fb9880b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb98e002640 1 --2- 172.21.15.26:0/1889463602 >> v2:172.21.15.160:3300/0 conn(0x7fb9880b30d0 0x7fb9880b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb98e002640 1 -- 172.21.15.26:0/1889463602 >> v1:172.21.15.26:6789/0 conn(0x7fb9880adb80 legacy=0x7fb9880cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb98e002640 1 -- 172.21.15.26:0/1889463602 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98806d1b0 con 0x7fb98815c7f0 2023-12-19T16:42:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb98e803640 1 --2- 172.21.15.26:0/1889463602 >> v2:172.21.15.160:3300/0 conn(0x7fb9880b30d0 0x7fb9880b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:42.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.372+0000 7fb98d000640 1 -- 172.21.15.26:0/1889463602 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587486170 0 0) 0x7fb9880b3a20 con 0x7fb9880adb80 2023-12-19T16:42:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.376+0000 7fb98e002640 1 --2- 172.21.15.26:0/1889463602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb98815c7f0 0x7fb98806c7f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb9880cc420 tx=0x7fb98401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de6aab3ee0836d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.376+0000 7fb98d000640 1 -- 172.21.15.26:0/1889463602 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9840ab600 con 0x7fb98815c7f0 2023-12-19T16:42:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.376+0000 7fb98d000640 1 -- 172.21.15.26:0/1889463602 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9840a7050 con 0x7fb98815c7f0 2023-12-19T16:42:42.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.376+0000 7fb98d000640 1 -- 172.21.15.26:0/1889463602 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9840ab980 con 0x7fb98815c7f0 2023-12-19T16:42:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.376+0000 7fb990266640 1 -- 172.21.15.26:0/1889463602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb98815c7f0 msgr2=0x7fb98806c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.376+0000 7fb990266640 1 --2- 172.21.15.26:0/1889463602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb98815c7f0 0x7fb98806c7f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb9880cc420 tx=0x7fb98401c460 comp rx=0 tx=0).stop 2023-12-19T16:42:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.376+0000 7fb990266640 1 -- 172.21.15.26:0/1889463602 shutdown_connections 2023-12-19T16:42:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.376+0000 7fb990266640 1 --2- 172.21.15.26:0/1889463602 >> v2:172.21.15.160:3300/0 conn(0x7fb9880b30d0 0x7fb9880b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.376+0000 7fb990266640 1 --2- 172.21.15.26:0/1889463602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb98815c7f0 0x7fb98806c7f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.376+0000 7fb990266640 1 -- 172.21.15.26:0/1889463602 >> 172.21.15.26:0/1889463602 conn(0x7fb9880658e0 msgr2=0x7fb988065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.376+0000 7fb990266640 1 -- 172.21.15.26:0/1889463602 shutdown_connections 2023-12-19T16:42:42.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.376+0000 7fb990266640 1 -- 172.21.15.26:0/1889463602 wait complete. 2023-12-19T16:42:42.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb990266640 1 Processor -- start 2023-12-19T16:42:42.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb990266640 1 -- start start 2023-12-19T16:42:42.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb990266640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9880adb80 0x7fb9881f0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb990266640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9880b30d0 0x7fb98813b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb990266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb98815c7f0 0x7fb98814d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb990266640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9880aed50 con 0x7fb9880adb80 2023-12-19T16:42:42.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb990266640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98813b920 con 0x7fb9880b30d0 2023-12-19T16:42:42.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb990266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb988150770 con 0x7fb98815c7f0 2023-12-19T16:42:42.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb98e002640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9880adb80 0x7fb9881f0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb98e002640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9880adb80 0x7fb9881f0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33580/0 (socket says 172.21.15.26:33580) 2023-12-19T16:42:42.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb98e002640 1 -- 172.21.15.26:0/1456033107 learned_addr learned my addr 172.21.15.26:0/1456033107 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:42.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb98e002640 1 -- 172.21.15.26:0/1456033107 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb98815c7f0 msgr2=0x7fb98814d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb98e002640 1 --2- 172.21.15.26:0/1456033107 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb98815c7f0 0x7fb98814d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb98e002640 1 -- 172.21.15.26:0/1456033107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9880b30d0 msgr2=0x7fb98813b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb98e002640 1 --2- 172.21.15.26:0/1456033107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9880b30d0 0x7fb98813b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb98e002640 1 -- 172.21.15.26:0/1456033107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb984009420 con 0x7fb9880adb80 2023-12-19T16:42:42.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb98e002640 1 --2- 172.21.15.26:0/1456033107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9880adb80 0x7fb9881f0f80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb984019070 tx=0x7fb984019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:42.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb97e7fc640 1 -- 172.21.15.26:0/1456033107 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9840b4610 con 0x7fb9880adb80 2023-12-19T16:42:42.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb97e7fc640 1 -- 172.21.15.26:0/1456033107 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9840b4770 con 0x7fb9880adb80 2023-12-19T16:42:42.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb97e7fc640 1 -- 172.21.15.26:0/1456033107 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9840b4ae0 con 0x7fb9880adb80 2023-12-19T16:42:42.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb990266640 1 -- 172.21.15.26:0/1456033107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98814c900 con 0x7fb9880adb80 2023-12-19T16:42:42.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.380+0000 7fb990266640 1 -- 172.21.15.26:0/1456033107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9880ae970 con 0x7fb9880adb80 2023-12-19T16:42:42.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.384+0000 7fb990266640 1 -- 172.21.15.26:0/1456033107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98814ce20 con 0x7fb9880adb80 2023-12-19T16:42:42.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.384+0000 7fb97e7fc640 1 -- 172.21.15.26:0/1456033107 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb9840b2040 con 0x7fb9880adb80 2023-12-19T16:42:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.384+0000 7fb97e7fc640 1 --2- 172.21.15.26:0/1456033107 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb96c065a60 0x7fb96c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.384+0000 7fb97e7fc640 1 -- 172.21.15.26:0/1456033107 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb98411d110 con 0x7fb9880adb80 2023-12-19T16:42:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.384+0000 7fb98d801640 1 --2- 172.21.15.26:0/1456033107 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb96c065a60 0x7fb96c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.384+0000 7fb98d801640 1 --2- 172.21.15.26:0/1456033107 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb96c065a60 0x7fb96c067f20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb9780011f0 tx=0x7fb978007930 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:42.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.388+0000 7fb97e7fc640 1 -- 172.21.15.26:0/1456033107 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb9840eaac0 con 0x7fb9880adb80 2023-12-19T16:42:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.580+0000 7fb990266640 1 -- 172.21.15.26:0/1456033107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9880b25e0 con 0x7fb9880adb80 2023-12-19T16:42:42.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.580+0000 7fb97e7fc640 1 -- 172.21.15.26:0/1456033107 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fb984006e20 con 0x7fb9880adb80 2023-12-19T16:42:42.586 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:42.586 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2850479611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2850479611},{"type":"v1","addr":"172.21.15.160:6836","nonce":2850479611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/837780144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":837780144},{"type":"v1","addr":"172.21.15.26:6834","nonce":837780144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2784408519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2784408519},{"type":"v1","addr":"172.21.15.26:6835","nonce":2784408519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:42:27.777197+0000","modified":"2023-12-19T16:42:30.798296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6837/3427345273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3427345273},{"type":"v1","addr":"172.21.15.160:6837","nonce":3427345273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T16:42:42.587 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2023-12-19T16:42:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.584+0000 7fb990266640 1 -- 172.21.15.26:0/1456033107 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb96c065a60 msgr2=0x7fb96c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.584+0000 7fb990266640 1 --2- 172.21.15.26:0/1456033107 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb96c065a60 0x7fb96c067f20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb9780011f0 tx=0x7fb978007930 comp rx=0 tx=0).stop 2023-12-19T16:42:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.584+0000 7fb990266640 1 -- 172.21.15.26:0/1456033107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9880adb80 msgr2=0x7fb9881f0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.584+0000 7fb990266640 1 --2- 172.21.15.26:0/1456033107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9880adb80 0x7fb9881f0f80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb984019070 tx=0x7fb984019200 comp rx=0 tx=0).stop 2023-12-19T16:42:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.584+0000 7fb990266640 1 -- 172.21.15.26:0/1456033107 shutdown_connections 2023-12-19T16:42:42.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.584+0000 7fb990266640 1 --2- 172.21.15.26:0/1456033107 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb96c065a60 0x7fb96c067f20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.584+0000 7fb990266640 1 --2- 172.21.15.26:0/1456033107 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb98815c7f0 0x7fb98814d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.584+0000 7fb990266640 1 --2- 172.21.15.26:0/1456033107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9880b30d0 0x7fb98813b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.584+0000 7fb990266640 1 --2- 172.21.15.26:0/1456033107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9880adb80 0x7fb9881f0f80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.584+0000 7fb990266640 1 -- 172.21.15.26:0/1456033107 >> 172.21.15.26:0/1456033107 conn(0x7fb9880658e0 msgr2=0x7fb98806f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:42.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.584+0000 7fb990266640 1 -- 172.21.15.26:0/1456033107 shutdown_connections 2023-12-19T16:42:42.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.584+0000 7fb990266640 1 -- 172.21.15.26:0/1456033107 wait complete. 2023-12-19T16:42:42.607 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:42.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55c56c3640 1 Processor -- start 2023-12-19T16:42:42.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55c56c3640 1 -- start start 2023-12-19T16:42:42.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55c56c3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f55c0158700 0x7f55c00b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55c56c3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c00ca8a0 con 0x7f55c00b0690 2023-12-19T16:42:42.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55c56c3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55c015afe0 con 0x7f55c00b1c30 2023-12-19T16:42:42.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55c56c3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55c015b980 con 0x7f55c0158700 2023-12-19T16:42:42.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55c4ec2640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f55c00b0690 0x7f55c00c8b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36180/0 (socket says 172.21.15.26:36180) 2023-12-19T16:42:42.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55c4ec2640 1 -- 172.21.15.26:0/1079425530 learned_addr learned my addr 172.21.15.26:0/1079425530 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:42.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55bffff640 1 --2- 172.21.15.26:0/1079425530 >> v2:172.21.15.160:3300/0 conn(0x7f55c0158700 0x7f55c00b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552887120 0 0) 0x7f55c00ca8a0 con 0x7f55c00b0690 2023-12-19T16:42:42.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55a8003530 con 0x7f55c00b0690 2023-12-19T16:42:42.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688449868 0 0) 0x7f55c015afe0 con 0x7f55c00b1c30 2023-12-19T16:42:42.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55c00ca8a0 con 0x7f55c00b1c30 2023-12-19T16:42:42.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4074943408 0 0) 0x7f55a8003530 con 0x7f55c00b0690 2023-12-19T16:42:42.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 >> v2:172.21.15.160:3300/0 conn(0x7f55c0158700 msgr2=0x7f55c00b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:42.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 --2- 172.21.15.26:0/1079425530 >> v2:172.21.15.160:3300/0 conn(0x7f55c0158700 0x7f55c00b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 >> v1:172.21.15.160:6789/0 conn(0x7f55c00b1c30 legacy=0x7f55c00aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c015b2c0 con 0x7f55c00b0690 2023-12-19T16:42:42.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f55b8001a00 con 0x7f55c00b0690 2023-12-19T16:42:42.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 >> v1:172.21.15.26:6789/0 conn(0x7f55c00b0690 legacy=0x7f55c00c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 --2- 172.21.15.26:0/1079425530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a8003b80 0x7f55a8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 --2- 172.21.15.26:0/1079425530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80047d0 0x7f55a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 --2- 172.21.15.26:0/1079425530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a8005190 0x7f55a800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c015b980 con 0x7f55a8003b80 2023-12-19T16:42:42.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a800c090 con 0x7f55a80047d0 2023-12-19T16:42:42.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a800c250 con 0x7f55a8005190 2023-12-19T16:42:42.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55bffff640 1 --2- 172.21.15.26:0/1079425530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a8003b80 0x7f55a8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55bf7fe640 1 --2- 172.21.15.26:0/1079425530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80047d0 0x7f55a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55c4ec2640 1 --2- 172.21.15.26:0/1079425530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a8005190 0x7f55a800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55bffff640 1 -- 172.21.15.26:0/1079425530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a8005190 msgr2=0x7f55a800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:42.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55bffff640 1 --2- 172.21.15.26:0/1079425530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a8005190 0x7f55a800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55bffff640 1 -- 172.21.15.26:0/1079425530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80047d0 msgr2=0x7f55a8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:42.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55bffff640 1 --2- 172.21.15.26:0/1079425530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a80047d0 0x7f55a8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55bffff640 1 -- 172.21.15.26:0/1079425530 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a8003530 con 0x7f55a8003b80 2023-12-19T16:42:42.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55c4ec2640 1 --2- 172.21.15.26:0/1079425530 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a8005190 0x7f55a800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:42.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.748+0000 7f55c4ec2640 1 -- 172.21.15.26:0/1079425530 reap_dead start 2023-12-19T16:42:42.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.752+0000 7f55bffff640 1 --2- 172.21.15.26:0/1079425530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a8003b80 0x7f55a8003f80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009ae0 tx=0x7f55b001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb26a37f7f0bea5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:42.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.752+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b00a64c0 con 0x7f55a8003b80 2023-12-19T16:42:42.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.752+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b00a6620 con 0x7f55a8003b80 2023-12-19T16:42:42.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.752+0000 7f55beffd640 1 -- 172.21.15.26:0/1079425530 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b00a69d0 con 0x7f55a8003b80 2023-12-19T16:42:42.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.752+0000 7f55c56c3640 1 -- 172.21.15.26:0/1079425530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a8003b80 msgr2=0x7f55a8003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.752+0000 7f55c56c3640 1 --2- 172.21.15.26:0/1079425530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a8003b80 0x7f55a8003f80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55b0009ae0 tx=0x7f55b001ca00 comp rx=0 tx=0).stop 2023-12-19T16:42:42.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.752+0000 7f55c56c3640 1 -- 172.21.15.26:0/1079425530 shutdown_connections 2023-12-19T16:42:42.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.752+0000 7f55c56c3640 1 --2- 172.21.15.26:0/1079425530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a8003b80 0x7f55a8003f80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.752+0000 7f55c56c3640 1 -- 172.21.15.26:0/1079425530 >> 172.21.15.26:0/1079425530 conn(0x7f55c006b050 msgr2=0x7f55c006b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:42.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.752+0000 7f55c56c3640 1 -- 172.21.15.26:0/1079425530 shutdown_connections 2023-12-19T16:42:42.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.752+0000 7f55c56c3640 1 -- 172.21.15.26:0/1079425530 wait complete. 2023-12-19T16:42:42.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c56c3640 1 Processor -- start 2023-12-19T16:42:42.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c56c3640 1 -- start start 2023-12-19T16:42:42.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c56c3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a8003b80 0x7f55c015d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c56c3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55c01532b0 0x7f55c00c2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c56c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c015d700 0x7f55c00c2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c56c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c015c120 con 0x7f55c015d700 2023-12-19T16:42:42.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c56c3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c015c400 con 0x7f55c01532b0 2023-12-19T16:42:42.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c56c3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c015c6e0 con 0x7f55a8003b80 2023-12-19T16:42:42.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c4ec2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c015d700 0x7f55c00c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c4ec2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c015d700 0x7f55c00c2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33596/0 (socket says 172.21.15.26:33596) 2023-12-19T16:42:42.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c4ec2640 1 -- 172.21.15.26:0/3967610421 learned_addr learned my addr 172.21.15.26:0/3967610421 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:42.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55bffff640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a8003b80 0x7f55c015d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55bf7fe640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55c01532b0 0x7f55c00c2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c4ec2640 1 -- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a8003b80 msgr2=0x7f55c015d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c4ec2640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a8003b80 0x7f55c015d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c4ec2640 1 -- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55c01532b0 msgr2=0x7f55c00c2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c4ec2640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55c01532b0 0x7f55c00c2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c4ec2640 1 -- 172.21.15.26:0/3967610421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c015c9c0 con 0x7f55c015d700 2023-12-19T16:42:42.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c4ec2640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c015d700 0x7f55c00c2570 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f55b8001340 tx=0x7f55b800df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:42.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55bcff9640 1 -- 172.21.15.26:0/3967610421 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800b800 con 0x7f55c015d700 2023-12-19T16:42:42.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55bcff9640 1 -- 172.21.15.26:0/3967610421 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b800b960 con 0x7f55c015d700 2023-12-19T16:42:42.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55bcff9640 1 -- 172.21.15.26:0/3967610421 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b800bd10 con 0x7f55c015d700 2023-12-19T16:42:42.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c56c3640 1 -- 172.21.15.26:0/3967610421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c015cca0 con 0x7f55c015d700 2023-12-19T16:42:42.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.756+0000 7f55c56c3640 1 -- 172.21.15.26:0/3967610421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c01402a0 con 0x7f55c015d700 2023-12-19T16:42:42.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.760+0000 7f55c56c3640 1 -- 172.21.15.26:0/3967610421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c015ce90 con 0x7f55c015d700 2023-12-19T16:42:42.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.760+0000 7f55bcff9640 1 -- 172.21.15.26:0/3967610421 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f55b8007070 con 0x7f55c015d700 2023-12-19T16:42:42.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.760+0000 7f55bcff9640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f55ac065a60 0x7f55ac067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:42.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.760+0000 7f55bcff9640 1 -- 172.21.15.26:0/3967610421 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55b8082ff0 con 0x7f55c015d700 2023-12-19T16:42:42.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.760+0000 7f55bffff640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f55ac065a60 0x7f55ac067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:42.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.764+0000 7f55bffff640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f55ac065a60 0x7f55ac067f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f55b00af000 tx=0x7f55b00046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:42.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.764+0000 7f55bcff9640 1 -- 172.21.15.26:0/3967610421 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f55b80509c0 con 0x7f55c015d700 2023-12-19T16:42:42.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.924+0000 7f55c56c3640 1 -- 172.21.15.26:0/3967610421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55c0067400 con 0x7f55c015d700 2023-12-19T16:42:42.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.924+0000 7f55bcff9640 1 -- 172.21.15.26:0/3967610421 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f55b8053fe0 con 0x7f55c015d700 2023-12-19T16:42:42.930 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:42.930 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.928+0000 7f55c56c3640 1 -- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f55ac065a60 msgr2=0x7f55ac067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.928+0000 7f55c56c3640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f55ac065a60 0x7f55ac067f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f55b00af000 tx=0x7f55b00046d0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.928+0000 7f55c56c3640 1 -- 172.21.15.26:0/3967610421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c015d700 msgr2=0x7f55c00c2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.928+0000 7f55c56c3640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c015d700 0x7f55c00c2570 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f55b8001340 tx=0x7f55b800df20 comp rx=0 tx=0).stop 2023-12-19T16:42:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.928+0000 7f55c56c3640 1 -- 172.21.15.26:0/3967610421 shutdown_connections 2023-12-19T16:42:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.928+0000 7f55c56c3640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55c015d700 0x7f55c00c2570 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.928+0000 7f55c56c3640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55c01532b0 0x7f55c00c2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.928+0000 7f55c56c3640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f55ac065a60 0x7f55ac067f20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.928+0000 7f55c56c3640 1 --2- 172.21.15.26:0/3967610421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a8003b80 0x7f55c015d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:42.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.928+0000 7f55c56c3640 1 -- 172.21.15.26:0/3967610421 >> 172.21.15.26:0/3967610421 conn(0x7f55c006b050 msgr2=0x7f55c0159480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:42.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.928+0000 7f55c56c3640 1 -- 172.21.15.26:0/3967610421 shutdown_connections 2023-12-19T16:42:42.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:42.928+0000 7f55c56c3640 1 -- 172.21.15.26:0/3967610421 wait complete. 2023-12-19T16:42:42.951 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:42:43.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc21d709640 1 Processor -- start 2023-12-19T16:42:43.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc21d709640 1 -- start start 2023-12-19T16:42:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc21d709640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc21815cda0 0x7fc218160920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc21d709640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2180b85e0 0x7fc2180cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc21d709640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180d0d40 con 0x7fc2180cde60 2023-12-19T16:42:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc21d709640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2180ce2f0 con 0x7fc21815cda0 2023-12-19T16:42:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc21d709640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc218161150 con 0x7fc2180b85e0 2023-12-19T16:42:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc216ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc21815cda0 0x7fc218160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc216ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc21815cda0 0x7fc218160920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33606/0 (socket says 172.21.15.26:33606) 2023-12-19T16:42:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc216ffd640 1 -- 172.21.15.26:0/2902868899 learned_addr learned my addr 172.21.15.26:0/2902868899 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc2167fc640 1 --2- 172.21.15.26:0/2902868899 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2180b85e0 0x7fc2180cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc216ffd640 1 -- 172.21.15.26:0/2902868899 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2180b85e0 msgr2=0x7fc2180cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc216ffd640 1 --2- 172.21.15.26:0/2902868899 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2180b85e0 0x7fc2180cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc216ffd640 1 -- 172.21.15.26:0/2902868899 >> v1:172.21.15.160:6789/0 conn(0x7fc2180cde60 legacy=0x7fc2180b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.092+0000 7fc216ffd640 1 -- 172.21.15.26:0/2902868899 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc218161430 con 0x7fc21815cda0 2023-12-19T16:42:43.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.096+0000 7fc216ffd640 1 --2- 172.21.15.26:0/2902868899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21815cda0 0x7fc218160920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc208009870 tx=0x7fc20801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733d4324291292db server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:43.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.096+0000 7fc215ffb640 1 -- 172.21.15.26:0/2902868899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080aa670 con 0x7fc21815cda0 2023-12-19T16:42:43.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.096+0000 7fc215ffb640 1 -- 172.21.15.26:0/2902868899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2080aa7d0 con 0x7fc21815cda0 2023-12-19T16:42:43.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.096+0000 7fc215ffb640 1 -- 172.21.15.26:0/2902868899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080aab60 con 0x7fc21815cda0 2023-12-19T16:42:43.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.096+0000 7fc21d709640 1 -- 172.21.15.26:0/2902868899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21815cda0 msgr2=0x7fc218160920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.096+0000 7fc21d709640 1 --2- 172.21.15.26:0/2902868899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21815cda0 0x7fc218160920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc208009870 tx=0x7fc20801c510 comp rx=0 tx=0).stop 2023-12-19T16:42:43.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.096+0000 7fc21d709640 1 -- 172.21.15.26:0/2902868899 shutdown_connections 2023-12-19T16:42:43.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.096+0000 7fc21d709640 1 --2- 172.21.15.26:0/2902868899 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2180b85e0 0x7fc2180cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.096+0000 7fc21d709640 1 --2- 172.21.15.26:0/2902868899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21815cda0 0x7fc218160920 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.096+0000 7fc21d709640 1 -- 172.21.15.26:0/2902868899 >> 172.21.15.26:0/2902868899 conn(0x7fc2180658e0 msgr2=0x7fc218065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:43.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.096+0000 7fc21d709640 1 -- 172.21.15.26:0/2902868899 shutdown_connections 2023-12-19T16:42:43.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.096+0000 7fc21d709640 1 -- 172.21.15.26:0/2902868899 wait complete. 2023-12-19T16:42:43.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc21d709640 1 Processor -- start 2023-12-19T16:42:43.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc21d709640 1 -- start start 2023-12-19T16:42:43.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc21d709640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2180b85e0 0x7fc21815d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc21d709640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2180cde60 0x7fc218152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc21d709640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21815cda0 0x7fc21814ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc21d709640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b4bb0 con 0x7fc2180b85e0 2023-12-19T16:42:43.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc21d709640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2181601c0 con 0x7fc2180cde60 2023-12-19T16:42:43.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc21d709640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b45f0 con 0x7fc21815cda0 2023-12-19T16:42:43.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc216ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2180b85e0 0x7fc21815d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc216ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2180b85e0 0x7fc21815d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33612/0 (socket says 172.21.15.26:33612) 2023-12-19T16:42:43.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc216ffd640 1 -- 172.21.15.26:0/3021891371 learned_addr learned my addr 172.21.15.26:0/3021891371 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:43.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc2167fc640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2180cde60 0x7fc218152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc2177fe640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21815cda0 0x7fc21814ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc216ffd640 1 -- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21815cda0 msgr2=0x7fc21814ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc216ffd640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21815cda0 0x7fc21814ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc216ffd640 1 -- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2180cde60 msgr2=0x7fc218152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc216ffd640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2180cde60 0x7fc218152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc216ffd640 1 -- 172.21.15.26:0/3021891371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2080094d0 con 0x7fc2180b85e0 2023-12-19T16:42:43.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc216ffd640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2180b85e0 0x7fc21815d5a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc2080ac040 tx=0x7fc20801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc1ff7fe640 1 -- 172.21.15.26:0/3021891371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080b3800 con 0x7fc2180b85e0 2023-12-19T16:42:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc1ff7fe640 1 -- 172.21.15.26:0/3021891371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2080b3980 con 0x7fc2180b85e0 2023-12-19T16:42:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc1ff7fe640 1 -- 172.21.15.26:0/3021891371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080b3d30 con 0x7fc2180b85e0 2023-12-19T16:42:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc21d709640 1 -- 172.21.15.26:0/3021891371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2180b48d0 con 0x7fc2180b85e0 2023-12-19T16:42:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc21d709640 1 -- 172.21.15.26:0/3021891371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21815fca0 con 0x7fc2180b85e0 2023-12-19T16:42:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.100+0000 7fc21d709640 1 -- 172.21.15.26:0/3021891371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21815fe90 con 0x7fc2180b85e0 2023-12-19T16:42:43.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.104+0000 7fc1ff7fe640 1 -- 172.21.15.26:0/3021891371 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc2080b2380 con 0x7fc2180b85e0 2023-12-19T16:42:43.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.104+0000 7fc1ff7fe640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc204065a60 0x7fc204067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.104+0000 7fc1ff7fe640 1 -- 172.21.15.26:0/3021891371 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc20811db00 con 0x7fc2180b85e0 2023-12-19T16:42:43.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.104+0000 7fc2167fc640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc204065a60 0x7fc204067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.104+0000 7fc2167fc640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc204065a60 0x7fc204067f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc20c006fd0 tx=0x7fc20c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:43.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.108+0000 7fc1ff7fe640 1 -- 172.21.15.26:0/3021891371 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc20811fe60 con 0x7fc2180b85e0 2023-12-19T16:42:43.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc21d709640 1 -- 172.21.15.26:0/3021891371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc218074270 con 0x7fc2180b85e0 2023-12-19T16:42:43.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc1ff7fe640 1 -- 172.21.15.26:0/3021891371 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fc2080eb560 con 0x7fc2180b85e0 2023-12-19T16:42:43.304 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:43.304 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2850479611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2850479611},{"type":"v1","addr":"172.21.15.160:6836","nonce":2850479611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/837780144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":837780144},{"type":"v1","addr":"172.21.15.26:6834","nonce":837780144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2784408519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2784408519},{"type":"v1","addr":"172.21.15.26:6835","nonce":2784408519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:42:27.777197+0000","modified":"2023-12-19T16:42:30.798296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6837/3427345273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3427345273},{"type":"v1","addr":"172.21.15.160:6837","nonce":3427345273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T16:42:43.304 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2023-12-19T16:42:43.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc21d709640 1 -- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc204065a60 msgr2=0x7fc204067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc21d709640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc204065a60 0x7fc204067f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc20c006fd0 tx=0x7fc20c008040 comp rx=0 tx=0).stop 2023-12-19T16:42:43.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc21d709640 1 -- 172.21.15.26:0/3021891371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2180b85e0 msgr2=0x7fc21815d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc21d709640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2180b85e0 0x7fc21815d5a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc2080ac040 tx=0x7fc20801ad50 comp rx=0 tx=0).stop 2023-12-19T16:42:43.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc21d709640 1 -- 172.21.15.26:0/3021891371 shutdown_connections 2023-12-19T16:42:43.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc21d709640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc204065a60 0x7fc204067f20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc21d709640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21815cda0 0x7fc21814ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc21d709640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2180cde60 0x7fc218152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc21d709640 1 --2- 172.21.15.26:0/3021891371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2180b85e0 0x7fc21815d5a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc21d709640 1 -- 172.21.15.26:0/3021891371 >> 172.21.15.26:0/3021891371 conn(0x7fc2180658e0 msgr2=0x7fc2180d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:43.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc21d709640 1 -- 172.21.15.26:0/3021891371 shutdown_connections 2023-12-19T16:42:43.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.300+0000 7fc21d709640 1 -- 172.21.15.26:0/3021891371 wait complete. 2023-12-19T16:42:43.729 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T16:42:27.777197+0000', 'modified': '2023-12-19T16:42:30.798296+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4279}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4279': {'gid': 4279, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.160:6837/3427345273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6835', 'nonce': 3427345273}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 3427345273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T16:42:43.729 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T16:42:43.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-19T16:42:43.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc03977b640 1 Processor -- start 2023-12-19T16:42:43.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc03977b640 1 -- start start 2023-12-19T16:42:43.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc03977b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc034153d40 0x7fc0340702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc03977b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc034154120 0x7fc0340b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc03977b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc0340b02c0 0x7fc0340b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc03977b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0340b0cb0 con 0x7fc034153d40 2023-12-19T16:42:43.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc03977b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0340ae150 con 0x7fc0340b02c0 2023-12-19T16:42:43.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc03977b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03415d7a0 con 0x7fc034154120 2023-12-19T16:42:43.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc032ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc034153d40 0x7fc0340702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc032ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc034153d40 0x7fc0340702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33628/0 (socket says 172.21.15.26:33628) 2023-12-19T16:42:43.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc032ffd640 1 -- 172.21.15.26:0/3636388377 learned_addr learned my addr 172.21.15.26:0/3636388377 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:43.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc0337fe640 1 --2- 172.21.15.26:0/3636388377 >> v2:172.21.15.160:3300/0 conn(0x7fc0340b02c0 0x7fc0340b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc02bfff640 1 --2- 172.21.15.26:0/3636388377 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc034154120 0x7fc0340b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc032ffd640 1 -- 172.21.15.26:0/3636388377 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc034154120 msgr2=0x7fc0340b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:43.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc032ffd640 1 --2- 172.21.15.26:0/3636388377 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc034154120 0x7fc0340b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc032ffd640 1 -- 172.21.15.26:0/3636388377 >> v2:172.21.15.160:3300/0 conn(0x7fc0340b02c0 msgr2=0x7fc0340b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:43.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc032ffd640 1 --2- 172.21.15.26:0/3636388377 >> v2:172.21.15.160:3300/0 conn(0x7fc0340b02c0 0x7fc0340b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.876+0000 7fc032ffd640 1 -- 172.21.15.26:0/3636388377 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03415da80 con 0x7fc034153d40 2023-12-19T16:42:43.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.880+0000 7fc032ffd640 1 --2- 172.21.15.26:0/3636388377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc034153d40 0x7fc0340702e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc034071b30 tx=0x7fc02401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ccdbd755b843dd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:43.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.880+0000 7fc0327fc640 1 -- 172.21.15.26:0/3636388377 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240a7070 con 0x7fc034153d40 2023-12-19T16:42:43.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.880+0000 7fc0327fc640 1 -- 172.21.15.26:0/3636388377 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0240ab430 con 0x7fc034153d40 2023-12-19T16:42:43.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.880+0000 7fc0327fc640 1 -- 172.21.15.26:0/3636388377 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240ab7c0 con 0x7fc034153d40 2023-12-19T16:42:43.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.880+0000 7fc03977b640 1 -- 172.21.15.26:0/3636388377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc034153d40 msgr2=0x7fc0340702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.880+0000 7fc03977b640 1 --2- 172.21.15.26:0/3636388377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc034153d40 0x7fc0340702e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc034071b30 tx=0x7fc02401c470 comp rx=0 tx=0).stop 2023-12-19T16:42:43.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.880+0000 7fc03977b640 1 -- 172.21.15.26:0/3636388377 shutdown_connections 2023-12-19T16:42:43.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.880+0000 7fc03977b640 1 --2- 172.21.15.26:0/3636388377 >> v2:172.21.15.160:3300/0 conn(0x7fc0340b02c0 0x7fc0340b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.880+0000 7fc03977b640 1 --2- 172.21.15.26:0/3636388377 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc034154120 0x7fc0340b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.880+0000 7fc03977b640 1 --2- 172.21.15.26:0/3636388377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc034153d40 0x7fc0340702e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.880+0000 7fc03977b640 1 -- 172.21.15.26:0/3636388377 >> 172.21.15.26:0/3636388377 conn(0x7fc034073360 msgr2=0x7fc0340658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:43.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.880+0000 7fc03977b640 1 -- 172.21.15.26:0/3636388377 shutdown_connections 2023-12-19T16:42:43.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.880+0000 7fc03977b640 1 -- 172.21.15.26:0/3636388377 wait complete. 2023-12-19T16:42:43.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc03977b640 1 Processor -- start 2023-12-19T16:42:43.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc03977b640 1 -- start start 2023-12-19T16:42:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc03977b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0340b02c0 0x7fc03414dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc03977b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc034153d40 0x7fc0340cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc03977b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc034154120 0x7fc034156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc03977b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc034155d00 con 0x7fc034154120 2023-12-19T16:42:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc03977b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc034152a60 con 0x7fc0340b02c0 2023-12-19T16:42:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc03977b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc034152d40 con 0x7fc034153d40 2023-12-19T16:42:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc0337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc034154120 0x7fc034156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc0337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc034154120 0x7fc034156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33632/0 (socket says 172.21.15.26:33632) 2023-12-19T16:42:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc0337fe640 1 -- 172.21.15.26:0/4166379625 learned_addr learned my addr 172.21.15.26:0/4166379625 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc032ffd640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0340b02c0 0x7fc03414dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc02bfff640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc034153d40 0x7fc0340cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc0337fe640 1 -- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc034153d40 msgr2=0x7fc0340cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc0337fe640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc034153d40 0x7fc0340cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc0337fe640 1 -- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0340b02c0 msgr2=0x7fc03414dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc0337fe640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0340b02c0 0x7fc03414dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc0337fe640 1 -- 172.21.15.26:0/4166379625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0240094d0 con 0x7fc034154120 2023-12-19T16:42:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc032ffd640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0340b02c0 0x7fc03414dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc02bfff640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc034153d40 0x7fc0340cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc0337fe640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc034154120 0x7fc034156190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc02c007e40 tx=0x7fc02c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc02b7fe640 1 -- 172.21.15.26:0/4166379625 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c014620 con 0x7fc034154120 2023-12-19T16:42:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc02b7fe640 1 -- 172.21.15.26:0/4166379625 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c00ee90 con 0x7fc034154120 2023-12-19T16:42:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc02b7fe640 1 -- 172.21.15.26:0/4166379625 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0149e0 con 0x7fc034154120 2023-12-19T16:42:43.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc03977b640 1 -- 172.21.15.26:0/4166379625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc034155990 con 0x7fc034154120 2023-12-19T16:42:43.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.884+0000 7fc03977b640 1 -- 172.21.15.26:0/4166379625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0340cbb30 con 0x7fc034154120 2023-12-19T16:42:43.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.888+0000 7fc02b7fe640 1 -- 172.21.15.26:0/4166379625 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc02c014b40 con 0x7fc034154120 2023-12-19T16:42:43.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.888+0000 7fc02b7fe640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc014065a60 0x7fc014067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.888+0000 7fc02b7fe640 1 -- 172.21.15.26:0/4166379625 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc02c083b50 con 0x7fc034154120 2023-12-19T16:42:43.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.888+0000 7fc032ffd640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc014065a60 0x7fc014067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.888+0000 7fc03cad9000 1 Processor -- start 2023-12-19T16:42:43.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.888+0000 7fc03cad9000 1 -- start start 2023-12-19T16:42:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.888+0000 7fc032ffd640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc014065a60 0x7fc014067f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc0240af000 tx=0x7fc024008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:43.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.888+0000 7fc03cad9000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33d3070 0x55fad33d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.888+0000 7fc03cad9000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3990 0x55fad3388c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.888+0000 7fc03cad9000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33891c0 0x55fad33895a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.888+0000 7fc03cad9000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad31acc90 con 0x55fad33d3990 2023-12-19T16:42:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.888+0000 7fc03cad9000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad31de220 con 0x55fad33d3070 2023-12-19T16:42:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.888+0000 7fc03cad9000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fad31db7f0 con 0x55fad33891c0 2023-12-19T16:42:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc02bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3990 0x55fad3388c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc02bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3990 0x55fad3388c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33646/0 (socket says 172.21.15.26:33646) 2023-12-19T16:42:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc02bfff640 1 -- 172.21.15.26:0/3975752737 learned_addr learned my addr 172.21.15.26:0/3975752737 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc032ffd640 1 --2- 172.21.15.26:0/3975752737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33d3070 0x55fad33d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc0337fe640 1 --2- 172.21.15.26:0/3975752737 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33891c0 0x55fad33895a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc02bfff640 1 -- 172.21.15.26:0/3975752737 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33891c0 msgr2=0x55fad33895a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc02bfff640 1 --2- 172.21.15.26:0/3975752737 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33891c0 0x55fad33895a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc02bfff640 1 -- 172.21.15.26:0/3975752737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33d3070 msgr2=0x55fad33d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc02bfff640 1 --2- 172.21.15.26:0/3975752737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33d3070 0x55fad33d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc02bfff640 1 -- 172.21.15.26:0/3975752737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fad2ed3890 con 0x55fad33d3990 2023-12-19T16:42:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc0337fe640 1 --2- 172.21.15.26:0/3975752737 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33891c0 0x55fad33895a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc02bfff640 1 --2- 172.21.15.26:0/3975752737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3990 0x55fad3388c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009860 tx=0x7fc01c009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b3dd4ce63e7146 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc028ff9640 1 -- 172.21.15.26:0/3975752737 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c004230 con 0x55fad33d3990 2023-12-19T16:42:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc028ff9640 1 -- 172.21.15.26:0/3975752737 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c004390 con 0x55fad33d3990 2023-12-19T16:42:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc028ff9640 1 -- 172.21.15.26:0/3975752737 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00e950 con 0x55fad33d3990 2023-12-19T16:42:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 -- 172.21.15.26:0/3975752737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3990 msgr2=0x55fad3388c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 --2- 172.21.15.26:0/3975752737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3990 0x55fad3388c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009860 tx=0x7fc01c009d20 comp rx=0 tx=0).stop 2023-12-19T16:42:43.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 -- 172.21.15.26:0/3975752737 shutdown_connections 2023-12-19T16:42:43.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 --2- 172.21.15.26:0/3975752737 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33891c0 0x55fad33895a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 --2- 172.21.15.26:0/3975752737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3990 0x55fad3388c80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 --2- 172.21.15.26:0/3975752737 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33d3070 0x55fad33d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 -- 172.21.15.26:0/3975752737 >> 172.21.15.26:0/3975752737 conn(0x55fad338f950 msgr2=0x55fad3387560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:43.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 -- 172.21.15.26:0/3975752737 shutdown_connections 2023-12-19T16:42:43.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 -- 172.21.15.26:0/3975752737 wait complete. 2023-12-19T16:42:43.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 -- start start 2023-12-19T16:42:43.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 Processor -- start 2023-12-19T16:42:43.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33891c0 0x55fad338e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad3395080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33d3990 0x55fad33955c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad31d70e0 con 0x55fad33d3070 2023-12-19T16:42:43.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad3176310 con 0x55fad33891c0 2023-12-19T16:42:43.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.892+0000 7fc03cad9000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fad3194770 con 0x55fad33d3990 2023-12-19T16:42:43.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc03cad9000 10 client.0 _ll_register_callbacks cb 0x55fad3374b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T16:42:43.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc02bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad3395080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc02bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad3395080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33648/0 (socket says 172.21.15.26:33648) 2023-12-19T16:42:43.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc02bfff640 1 -- 172.21.15.26:0/1019175061 learned_addr learned my addr 172.21.15.26:0/1019175061 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:43.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc032ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33891c0 0x55fad338e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc0337fe640 1 --2- 172.21.15.26:0/1019175061 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33d3990 0x55fad33955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc02bfff640 1 -- 172.21.15.26:0/1019175061 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33d3990 msgr2=0x55fad33955c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:43.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc02bfff640 1 --2- 172.21.15.26:0/1019175061 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33d3990 0x55fad33955c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc02bfff640 1 -- 172.21.15.26:0/1019175061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33891c0 msgr2=0x55fad338e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:43.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc02bfff640 1 --2- 172.21.15.26:0/1019175061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33891c0 0x55fad338e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc02bfff640 1 -- 172.21.15.26:0/1019175061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c0094c0 con 0x55fad33d3070 2023-12-19T16:42:43.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc02bfff640 1 --2- 172.21.15.26:0/1019175061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad3395080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc01c015040 tx=0x7fc01c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd813b7088e0fca server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:43.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T16:42:43.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc011ffb640 1 -- 172.21.15.26:0/1019175061 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00eb60 con 0x55fad33d3070 2023-12-19T16:42:43.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc011ffb640 1 -- 172.21.15.26:0/1019175061 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc01c023070 con 0x55fad33d3070 2023-12-19T16:42:43.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc03cad9000 1 -- 172.21.15.26:0/1019175061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fad31a6c10 con 0x55fad33d3070 2023-12-19T16:42:43.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc011ffb640 1 -- 172.21.15.26:0/1019175061 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc01c00ecc0 con 0x55fad33d3070 2023-12-19T16:42:43.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc03cad9000 10 client.4683 fetch_fsmap learned FSMap version 8 2023-12-19T16:42:43.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc03cad9000 1 -- 172.21.15.26:0/1019175061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55fad319c260 con 0x55fad33d3070 2023-12-19T16:42:43.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc011ffb640 1 -- 172.21.15.26:0/1019175061 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fad31a6c10 con 0x55fad33d3070 2023-12-19T16:42:43.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc011ffb640 1 -- 172.21.15.26:0/1019175061 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc01c003620 con 0x55fad33d3070 2023-12-19T16:42:43.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc03cad9000 10 client.4683 fetch_fsmap finished waiting for FSMap version 8 2023-12-19T16:42:43.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc03cad9000 10 client.4683 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-19T16:42:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc03cad9000 20 client.4683 populate_metadata read hostname 'smithi026' 2023-12-19T16:42:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc03cad9000 1 --2- 172.21.15.26:0/1019175061 >> [v2:172.21.15.160:6835/3427345273,v1:172.21.15.160:6837/3427345273] conn(0x55fad33da8d0 0x55fad33dacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc03cad9000 4 client.4683 mds_command: new command op to 4279 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T16:42:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc03cad9000 1 -- 172.21.15.26:0/1019175061 --> [v2:172.21.15.160:6835/3427345273,v1:172.21.15.160:6837/3427345273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fad31d5a30 con 0x55fad33da8d0 2023-12-19T16:42:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc032ffd640 1 --2- 172.21.15.26:0/1019175061 >> [v2:172.21.15.160:6835/3427345273,v1:172.21.15.160:6837/3427345273] conn(0x55fad33da8d0 0x55fad33dacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc032ffd640 1 --2- 172.21.15.26:0/1019175061 >> [v2:172.21.15.160:6835/3427345273,v1:172.21.15.160:6837/3427345273] conn(0x55fad33da8d0 0x55fad33dacb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec225e35d831186 server_cookie=23407da1b0ee9ca4 in_seq=0 out_seq=0 2023-12-19T16:42:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc011ffb640 10 client.4683 ms_handle_connect on v2:172.21.15.160:6835/3427345273 2023-12-19T16:42:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc011ffb640 1 -- 172.21.15.26:0/1019175061 <== mds.0 v2:172.21.15.160:6835/3427345273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fad31d5a30 con 0x55fad33da8d0 2023-12-19T16:42:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.896+0000 7fc011ffb640 10 client.4683 handle_command_reply: tid=0 2023-12-19T16:42:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.900+0000 7fc03cad9000 1 client.4683 shutdown 2023-12-19T16:42:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.900+0000 7fc03cad9000 1 -- 172.21.15.26:0/1019175061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 msgr2=0x55fad3395080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.900+0000 7fc03cad9000 1 --2- 172.21.15.26:0/1019175061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad3395080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc01c015040 tx=0x7fc01c004740 comp rx=0 tx=0).stop 2023-12-19T16:42:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.900+0000 7fc03cad9000 1 -- 172.21.15.26:0/1019175061 shutdown_connections 2023-12-19T16:42:43.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.900+0000 7fc03cad9000 1 --2- 172.21.15.26:0/1019175061 >> [v2:172.21.15.160:6835/3427345273,v1:172.21.15.160:6837/3427345273] conn(0x55fad33da8d0 0x55fad33dacb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.900+0000 7fc03cad9000 1 --2- 172.21.15.26:0/1019175061 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33d3990 0x55fad33955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.900+0000 7fc03cad9000 1 --2- 172.21.15.26:0/1019175061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad3395080 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.900+0000 7fc03cad9000 1 --2- 172.21.15.26:0/1019175061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33891c0 0x55fad338e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.900+0000 7fc03cad9000 1 -- 172.21.15.26:0/1019175061 >> 172.21.15.26:0/1019175061 conn(0x55fad338f950 msgr2=0x55fad3387c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:43.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.900+0000 7fc011ffb640 0 client.4683 ms_handle_reset on v2:172.21.15.160:6835/3427345273 2023-12-19T16:42:43.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.900+0000 7fc03cad9000 1 -- 172.21.15.26:0/1019175061 shutdown_connections 2023-12-19T16:42:43.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.900+0000 7fc03cad9000 1 -- 172.21.15.26:0/1019175061 wait complete. 2023-12-19T16:42:43.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.900+0000 7fc03cad9000 20 client.4683 trim_cache size 0 max 16384 2023-12-19T16:42:43.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.912+0000 7fc03cad9000 1 Processor -- start 2023-12-19T16:42:43.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.912+0000 7fc03cad9000 1 -- start start 2023-12-19T16:42:43.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33891c0 0x55fad342aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad342b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33d3990 0x55fad342b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad2e94940 con 0x55fad33d3070 2023-12-19T16:42:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad31d2a50 con 0x55fad33d3990 2023-12-19T16:42:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fad317f860 con 0x55fad33891c0 2023-12-19T16:42:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc02bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad342b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc02bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad342b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33656/0 (socket says 172.21.15.26:33656) 2023-12-19T16:42:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc02bfff640 1 -- 172.21.15.26:0/109114486 learned_addr learned my addr 172.21.15.26:0/109114486 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc032ffd640 1 --2- 172.21.15.26:0/109114486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33891c0 0x55fad342aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc0337fe640 1 --2- 172.21.15.26:0/109114486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33d3990 0x55fad342b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc02bfff640 1 -- 172.21.15.26:0/109114486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33891c0 msgr2=0x55fad342aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:43.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc02bfff640 1 --2- 172.21.15.26:0/109114486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33891c0 0x55fad342aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc02bfff640 1 -- 172.21.15.26:0/109114486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33d3990 msgr2=0x55fad342b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:43.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc02bfff640 1 --2- 172.21.15.26:0/109114486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33d3990 0x55fad342b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc02bfff640 1 -- 172.21.15.26:0/109114486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fad31823b0 con 0x55fad33d3070 2023-12-19T16:42:43.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc02bfff640 1 --2- 172.21.15.26:0/109114486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad342b410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc01c000c10 tx=0x7fc01c01f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c97866dcd9e715 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:43.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc0297fa640 1 -- 172.21.15.26:0/109114486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c003bb0 con 0x55fad33d3070 2023-12-19T16:42:43.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc0297fa640 1 -- 172.21.15.26:0/109114486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c021070 con 0x55fad33d3070 2023-12-19T16:42:43.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc0297fa640 1 -- 172.21.15.26:0/109114486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c004350 con 0x55fad33d3070 2023-12-19T16:42:43.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 -- 172.21.15.26:0/109114486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 msgr2=0x55fad342b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 --2- 172.21.15.26:0/109114486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad342b410 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc01c000c10 tx=0x7fc01c01f070 comp rx=0 tx=0).stop 2023-12-19T16:42:43.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 -- 172.21.15.26:0/109114486 shutdown_connections 2023-12-19T16:42:43.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 --2- 172.21.15.26:0/109114486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33d3990 0x55fad342b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 --2- 172.21.15.26:0/109114486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad342b410 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 --2- 172.21.15.26:0/109114486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33891c0 0x55fad342aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 -- 172.21.15.26:0/109114486 >> 172.21.15.26:0/109114486 conn(0x55fad338f950 msgr2=0x55fad3394f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:43.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 -- 172.21.15.26:0/109114486 shutdown_connections 2023-12-19T16:42:43.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 -- 172.21.15.26:0/109114486 wait complete. 2023-12-19T16:42:43.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 -- start start 2023-12-19T16:42:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 Processor -- start 2023-12-19T16:42:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33891c0 0x55fad342f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad342ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33d3990 0x55fad33e5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad338e7d0 con 0x55fad33d3070 2023-12-19T16:42:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fad31a4730 con 0x55fad33891c0 2023-12-19T16:42:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fad336f040 con 0x55fad33d3990 2023-12-19T16:42:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.916+0000 7fc03cad9000 10 client.0 _ll_register_callbacks cb 0x55fad3185640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T16:42:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc02bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad342ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc02bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad342ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33658/0 (socket says 172.21.15.26:33658) 2023-12-19T16:42:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc02bfff640 1 -- 172.21.15.26:0/3328014567 learned_addr learned my addr 172.21.15.26:0/3328014567 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc032ffd640 1 --2- 172.21.15.26:0/3328014567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33891c0 0x55fad342f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc0337fe640 1 --2- 172.21.15.26:0/3328014567 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33d3990 0x55fad33e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc02bfff640 1 -- 172.21.15.26:0/3328014567 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33d3990 msgr2=0x55fad33e5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc02bfff640 1 --2- 172.21.15.26:0/3328014567 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33d3990 0x55fad33e5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc02bfff640 1 -- 172.21.15.26:0/3328014567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33891c0 msgr2=0x55fad342f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc02bfff640 1 --2- 172.21.15.26:0/3328014567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33891c0 0x55fad342f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc02bfff640 1 -- 172.21.15.26:0/3328014567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fad319c260 con 0x55fad33d3070 2023-12-19T16:42:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc02bfff640 1 --2- 172.21.15.26:0/3328014567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad342ff20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc01c028be0 tx=0x7fc01c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c115630fc27d8a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T16:42:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc011ffb640 1 -- 172.21.15.26:0/3328014567 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c03f070 con 0x55fad33d3070 2023-12-19T16:42:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc011ffb640 1 -- 172.21.15.26:0/3328014567 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc01c036070 con 0x55fad33d3070 2023-12-19T16:42:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 -- 172.21.15.26:0/3328014567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fad319ce00 con 0x55fad33d3070 2023-12-19T16:42:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc011ffb640 1 -- 172.21.15.26:0/3328014567 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc01c028480 con 0x55fad33d3070 2023-12-19T16:42:43.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 10 client.4689 fetch_fsmap learned FSMap version 8 2023-12-19T16:42:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 -- 172.21.15.26:0/3328014567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55fad2ed3cc0 con 0x55fad33d3070 2023-12-19T16:42:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc011ffb640 1 -- 172.21.15.26:0/3328014567 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fad319ce00 con 0x55fad33d3070 2023-12-19T16:42:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc011ffb640 1 -- 172.21.15.26:0/3328014567 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc01c0285e0 con 0x55fad33d3070 2023-12-19T16:42:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 10 client.4689 fetch_fsmap finished waiting for FSMap version 8 2023-12-19T16:42:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 10 client.4689 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-19T16:42:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 20 client.4689 populate_metadata read hostname 'smithi026' 2023-12-19T16:42:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 --2- 172.21.15.26:0/3328014567 >> [v2:172.21.15.160:6835/3427345273,v1:172.21.15.160:6837/3427345273] conn(0x55fad33e5c10 0x55fad33e5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 4 client.4689 mds_command: new command op to 4279 tid=0[{"prefix": "status"}] 2023-12-19T16:42:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 -- 172.21.15.26:0/3328014567 --> [v2:172.21.15.160:6835/3427345273,v1:172.21.15.160:6837/3427345273] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fad31d0160 con 0x55fad33e5c10 2023-12-19T16:42:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc032ffd640 1 --2- 172.21.15.26:0/3328014567 >> [v2:172.21.15.160:6835/3427345273,v1:172.21.15.160:6837/3427345273] conn(0x55fad33e5c10 0x55fad33e5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc032ffd640 1 --2- 172.21.15.26:0/3328014567 >> [v2:172.21.15.160:6835/3427345273,v1:172.21.15.160:6837/3427345273] conn(0x55fad33e5c10 0x55fad33e5ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9892a53b252b852 server_cookie=97169fd44056578 in_seq=0 out_seq=0 2023-12-19T16:42:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc011ffb640 10 client.4689 ms_handle_connect on v2:172.21.15.160:6835/3427345273 2023-12-19T16:42:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc011ffb640 1 -- 172.21.15.26:0/3328014567 <== mds.0 v2:172.21.15.160:6835/3427345273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55fad31d0160 con 0x55fad33e5c10 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc011ffb640 10 client.4689 handle_command_reply: tid=0 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 client.4689 shutdown 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 -- 172.21.15.26:0/3328014567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 msgr2=0x55fad342ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 --2- 172.21.15.26:0/3328014567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad342ff20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc01c028be0 tx=0x7fc01c008040 comp rx=0 tx=0).stop 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 -- 172.21.15.26:0/3328014567 shutdown_connections 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 --2- 172.21.15.26:0/3328014567 >> [v2:172.21.15.160:6835/3427345273,v1:172.21.15.160:6837/3427345273] conn(0x55fad33e5c10 0x55fad33e5ff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 --2- 172.21.15.26:0/3328014567 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fad33d3990 0x55fad33e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 --2- 172.21.15.26:0/3328014567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fad33d3070 0x55fad342ff20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 --2- 172.21.15.26:0/3328014567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fad33891c0 0x55fad342f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc011ffb640 0 client.4689 ms_handle_reset on v2:172.21.15.160:6835/3427345273 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 -- 172.21.15.26:0/3328014567 >> 172.21.15.26:0/3328014567 conn(0x55fad338f950 msgr2=0x55fad33d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 -- 172.21.15.26:0/3328014567 shutdown_connections 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 1 -- 172.21.15.26:0/3328014567 wait complete. 2023-12-19T16:42:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.920+0000 7fc03cad9000 20 client.4689 trim_cache size 0 max 16384 2023-12-19T16:42:43.930 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T16:42:43.930 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T16:42:43.930 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T16:42:43.930 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4279, 2023-12-19T16:42:43.930 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T16:42:43.930 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T16:42:43.930 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T16:42:43.930 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 16.142138484, 2023-12-19T16:42:43.930 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 8, 2023-12-19T16:42:43.930 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 23, 2023-12-19T16:42:43.931 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 23, 2023-12-19T16:42:43.931 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 23.067956708000001 2023-12-19T16:42:43.931 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T16:42:43.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.924+0000 7fc03977b640 1 -- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc014065a60 msgr2=0x7fc014067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.924+0000 7fc03977b640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc014065a60 0x7fc014067f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc0240af000 tx=0x7fc024008040 comp rx=0 tx=0).stop 2023-12-19T16:42:43.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.924+0000 7fc03977b640 1 -- 172.21.15.26:0/4166379625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc034154120 msgr2=0x7fc034156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:43.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.924+0000 7fc03977b640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc034154120 0x7fc034156190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc02c007e40 tx=0x7fc02c00e4b0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.924+0000 7fc03977b640 1 -- 172.21.15.26:0/4166379625 shutdown_connections 2023-12-19T16:42:43.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.924+0000 7fc03977b640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc034154120 0x7fc034156190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.924+0000 7fc03977b640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc014065a60 0x7fc014067f20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.924+0000 7fc03977b640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc034153d40 0x7fc0340cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.924+0000 7fc03977b640 1 --2- 172.21.15.26:0/4166379625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0340b02c0 0x7fc03414dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:43.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.924+0000 7fc03977b640 1 -- 172.21.15.26:0/4166379625 >> 172.21.15.26:0/4166379625 conn(0x7fc034073360 msgr2=0x7fc0340afba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:43.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.924+0000 7fc03977b640 1 -- 172.21.15.26:0/4166379625 shutdown_connections 2023-12-19T16:42:43.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:43.924+0000 7fc03977b640 1 -- 172.21.15.26:0/4166379625 wait complete. 2023-12-19T16:42:43.945 INFO:teuthology.run_tasks:Running task kclient... 2023-12-19T16:42:43.970 INFO:tasks.kclient:Mounting kernel clients... 2023-12-19T16:42:43.970 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-12-19T16:42:43.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:43.970 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T16:42:43.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:43.980 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2023-12-19T16:42:44.046 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2023-12-19T16:42:44.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:42:44.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:44.049 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T16:42:44.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:44.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2023-12-19T16:42:44.170 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2023-12-19T16:42:44.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:42:44.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:44.172 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2023-12-19T16:42:44.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:44.183 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link delete ceph-brx 2023-12-19T16:42:44.252 INFO:teuthology.orchestra.run.smithi160.stderr:Cannot find device "ceph-brx" 2023-12-19T16:42:44.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:42:44.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:44.254 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2023-12-19T16:42:44.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:44.304 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link delete ceph-brx 2023-12-19T16:42:44.375 INFO:teuthology.orchestra.run.smithi160.stderr:Cannot find device "ceph-brx" 2023-12-19T16:42:44.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T16:42:44.378 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-12-19T16:42:44.378 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-19T16:42:44.378 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T16:42:44.378 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T16:42:44.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f42e5640 1 Processor -- start 2023-12-19T16:42:44.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f42e5640 1 -- start start 2023-12-19T16:42:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f42e5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f63ec0ce040 0x7f63ec0d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f42e5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ec0d29d0 con 0x7f63ec15cd40 2023-12-19T16:42:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f42e5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ec0d3570 con 0x7f63ec0d3130 2023-12-19T16:42:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f42e5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ec15d680 con 0x7f63ec0ce040 2023-12-19T16:42:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f2081640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f63ec15cd40 0x7f63ec160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36194/0 (socket says 172.21.15.26:36194) 2023-12-19T16:42:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f2081640 1 -- 172.21.15.26:0/1770972882 learned_addr learned my addr 172.21.15.26:0/1770972882 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:44.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f1880640 1 --2- 172.21.15.26:0/1770972882 >> v2:172.21.15.160:3300/0 conn(0x7f63ec0ce040 0x7f63ec0d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:44.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2027338213 0 0) 0x7f63e80012a0 con 0x7f63ec15cd40 2023-12-19T16:42:44.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63d8003530 con 0x7f63ec15cd40 2023-12-19T16:42:44.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2373091497 0 0) 0x7f63d8003530 con 0x7f63ec15cd40 2023-12-19T16:42:44.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 >> v2:172.21.15.160:3300/0 conn(0x7f63ec0ce040 msgr2=0x7f63ec0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:44.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 --2- 172.21.15.26:0/1770972882 >> v2:172.21.15.160:3300/0 conn(0x7f63ec0ce040 0x7f63ec0d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:44.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 >> v1:172.21.15.160:6789/0 conn(0x7f63ec0d3130 legacy=0x7f63ec0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:44.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec0ce4b0 con 0x7f63ec15cd40 2023-12-19T16:42:44.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f63e8001990 con 0x7f63ec15cd40 2023-12-19T16:42:44.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 >> v1:172.21.15.26:6789/0 conn(0x7f63ec15cd40 legacy=0x7f63ec160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:44.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 --2- 172.21.15.26:0/1770972882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63d8003980 0x7f63d8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:44.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 --2- 172.21.15.26:0/1770972882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d80047d0 0x7f63d8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:44.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 --2- 172.21.15.26:0/1770972882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8005190 0x7f63d800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:44.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec15d680 con 0x7f63d8005190 2023-12-19T16:42:44.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d800c030 con 0x7f63d80047d0 2023-12-19T16:42:44.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d800c1f0 con 0x7f63d8003980 2023-12-19T16:42:44.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f2882640 1 --2- 172.21.15.26:0/1770972882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8005190 0x7f63d800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:44.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f1880640 1 --2- 172.21.15.26:0/1770972882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d80047d0 0x7f63d8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:44.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f2081640 1 --2- 172.21.15.26:0/1770972882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63d8003980 0x7f63d8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:44.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f2882640 1 -- 172.21.15.26:0/1770972882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63d8003980 msgr2=0x7f63d8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:44.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f2882640 1 --2- 172.21.15.26:0/1770972882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63d8003980 0x7f63d8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:44.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f2882640 1 -- 172.21.15.26:0/1770972882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d80047d0 msgr2=0x7f63d8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:44.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f2882640 1 --2- 172.21.15.26:0/1770972882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63d80047d0 0x7f63d8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:44.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.520+0000 7f63f2882640 1 -- 172.21.15.26:0/1770972882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d8003530 con 0x7f63d8005190 2023-12-19T16:42:44.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.524+0000 7f63f2882640 1 -- 172.21.15.26:0/1770972882 reap_dead start 2023-12-19T16:42:44.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.524+0000 7f63f2882640 1 --2- 172.21.15.26:0/1770972882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8005190 0x7f63d800b990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f63e4001e70 tx=0x7f63e401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a8bf4766a93a10 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:44.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.524+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4003b80 con 0x7f63d8005190 2023-12-19T16:42:44.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.524+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e4003d20 con 0x7f63d8005190 2023-12-19T16:42:44.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.524+0000 7f63f107f640 1 -- 172.21.15.26:0/1770972882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4005260 con 0x7f63d8005190 2023-12-19T16:42:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.524+0000 7f63f42e5640 1 -- 172.21.15.26:0/1770972882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8005190 msgr2=0x7f63d800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.524+0000 7f63f42e5640 1 --2- 172.21.15.26:0/1770972882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8005190 0x7f63d800b990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f63e4001e70 tx=0x7f63e401c760 comp rx=0 tx=0).stop 2023-12-19T16:42:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.524+0000 7f63f42e5640 1 -- 172.21.15.26:0/1770972882 shutdown_connections 2023-12-19T16:42:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.524+0000 7f63f42e5640 1 --2- 172.21.15.26:0/1770972882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8005190 0x7f63d800b990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.524+0000 7f63f42e5640 1 -- 172.21.15.26:0/1770972882 >> 172.21.15.26:0/1770972882 conn(0x7f63ec06a120 msgr2=0x7f63ec06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.524+0000 7f63f42e5640 1 -- 172.21.15.26:0/1770972882 shutdown_connections 2023-12-19T16:42:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.524+0000 7f63f42e5640 1 -- 172.21.15.26:0/1770972882 wait complete. 2023-12-19T16:42:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f42e5640 1 Processor -- start 2023-12-19T16:42:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f42e5640 1 -- start start 2023-12-19T16:42:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f42e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8005190 0x7f63ec1f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f42e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63ec0aa760 0x7f63ec0a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f42e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63ec1f5f00 0x7f63ec14b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f42e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec1f3210 con 0x7f63d8005190 2023-12-19T16:42:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f42e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec0a9d30 con 0x7f63ec1f5f00 2023-12-19T16:42:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f42e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec0aabe0 con 0x7f63ec0aa760 2023-12-19T16:42:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f2081640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8005190 0x7f63ec1f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f2081640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8005190 0x7f63ec1f2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33674/0 (socket says 172.21.15.26:33674) 2023-12-19T16:42:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f2081640 1 -- 172.21.15.26:0/594221256 learned_addr learned my addr 172.21.15.26:0/594221256 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f1880640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63ec0aa760 0x7f63ec0a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f2882640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63ec1f5f00 0x7f63ec14b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f2081640 1 -- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63ec0aa760 msgr2=0x7f63ec0a97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f2081640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63ec0aa760 0x7f63ec0a97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f2081640 1 -- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63ec1f5f00 msgr2=0x7f63ec14b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f2081640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63ec1f5f00 0x7f63ec14b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f2081640 1 -- 172.21.15.26:0/594221256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec1f2970 con 0x7f63d8005190 2023-12-19T16:42:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f2081640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8005190 0x7f63ec1f2270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f63e8001a30 tx=0x7f63e8006b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63e2ffd640 1 -- 172.21.15.26:0/594221256 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e800f7b0 con 0x7f63d8005190 2023-12-19T16:42:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63e2ffd640 1 -- 172.21.15.26:0/594221256 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e800f910 con 0x7f63d8005190 2023-12-19T16:42:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f42e5640 1 -- 172.21.15.26:0/594221256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ec1f2c50 con 0x7f63d8005190 2023-12-19T16:42:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63e2ffd640 1 -- 172.21.15.26:0/594221256 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e800fcc0 con 0x7f63d8005190 2023-12-19T16:42:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.528+0000 7f63f42e5640 1 -- 172.21.15.26:0/594221256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ec1f59e0 con 0x7f63d8005190 2023-12-19T16:42:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.532+0000 7f63f42e5640 1 -- 172.21.15.26:0/594221256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ec1f5840 con 0x7f63d8005190 2023-12-19T16:42:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.532+0000 7f63e2ffd640 1 -- 172.21.15.26:0/594221256 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f63e80133f0 con 0x7f63d8005190 2023-12-19T16:42:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.532+0000 7f63e2ffd640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f63d0065a90 0x7f63d0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.532+0000 7f63e2ffd640 1 -- 172.21.15.26:0/594221256 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63e8082f70 con 0x7f63d8005190 2023-12-19T16:42:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.532+0000 7f63f1880640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f63d0065a90 0x7f63d0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:44.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.532+0000 7f63f1880640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f63d0065a90 0x7f63d0067f50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0cdcc0 tx=0x7f63dc009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:44.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.536+0000 7f63e2ffd640 1 -- 172.21.15.26:0/594221256 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f63ec1f5840 con 0x7f63d8005190 2023-12-19T16:42:44.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.724+0000 7f63f42e5640 1 -- 172.21.15.26:0/594221256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f63ec1f5840 con 0x7f63d8005190 2023-12-19T16:42:44.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.728+0000 7f63e2ffd640 1 -- 172.21.15.26:0/594221256 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f63ec1f5840 con 0x7f63d8005190 2023-12-19T16:42:44.732 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T16:42:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.728+0000 7f63f42e5640 1 -- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f63d0065a90 msgr2=0x7f63d0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.728+0000 7f63f42e5640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f63d0065a90 0x7f63d0067f50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0cdcc0 tx=0x7f63dc009190 comp rx=0 tx=0).stop 2023-12-19T16:42:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.728+0000 7f63f42e5640 1 -- 172.21.15.26:0/594221256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8005190 msgr2=0x7f63ec1f2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.728+0000 7f63f42e5640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8005190 0x7f63ec1f2270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f63e8001a30 tx=0x7f63e8006b90 comp rx=0 tx=0).stop 2023-12-19T16:42:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.728+0000 7f63f42e5640 1 -- 172.21.15.26:0/594221256 shutdown_connections 2023-12-19T16:42:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.728+0000 7f63f42e5640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63ec1f5f00 0x7f63ec14b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.728+0000 7f63f42e5640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f63d0065a90 0x7f63d0067f50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.728+0000 7f63f42e5640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63ec0aa760 0x7f63ec0a97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.728+0000 7f63f42e5640 1 --2- 172.21.15.26:0/594221256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63d8005190 0x7f63ec1f2270 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:44.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.728+0000 7f63f42e5640 1 -- 172.21.15.26:0/594221256 >> 172.21.15.26:0/594221256 conn(0x7f63ec06a120 msgr2=0x7f63ec06a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:44.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.728+0000 7f63f42e5640 1 -- 172.21.15.26:0/594221256 shutdown_connections 2023-12-19T16:42:44.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:44.728+0000 7f63f42e5640 1 -- 172.21.15.26:0/594221256 wait complete. 2023-12-19T16:42:44.752 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T16:42:44.752 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T16:42:44.752 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T16:42:44.752 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T16:42:44.752 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T16:42:44.752 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T16:42:44.752 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T16:42:44.752 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T16:42:44.753 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:42:44.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:44.753 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T16:42:44.764 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T16:42:44.764 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T16:42:44.764 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T16:42:44.764 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:44.765 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T16:42:44.765 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:44.765 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:42:44.765 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:44.765 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T16:42:44.765 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:42:44.765 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:44.766 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T16:42:44.766 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T16:42:44.766 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:44.766 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T16:42:44.766 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T16:42:44.766 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42376sec preferred_lft 42376sec 2023-12-19T16:42:44.766 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T16:42:44.767 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:44.767 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:42:44.767 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:44.767 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T16:42:44.767 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T16:42:44.767 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:44.768 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T16:42:44.768 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:44.768 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-19T16:42:44.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:44.769 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:42:44.769 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:42:44.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2023-12-19T16:42:44.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2023-12-19T16:42:44.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2023-12-19T16:42:44.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-19T16:42:44.770 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:42:44.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:44.882 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T16:42:44.901 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T16:42:44.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:44.903 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T16:42:44.975 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T16:42:44.975 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T16:42:44.975 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T16:42:44.975 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T16:42:44.975 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T16:42:44.975 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T16:42:44.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:44.976 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:42:44.976 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:42:44.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T16:42:44.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T16:42:44.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T16:42:44.976 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:42:45.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:45.070 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T16:42:45.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:45.081 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2023-12-19T16:42:45.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:45.135 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:42:45.136 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:42:45.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T16:42:45.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-19T16:42:45.136 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:42:45.222 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-19T16:42:45.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:45.222 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:42:45.222 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:42:45.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-19T16:42:45.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-19T16:42:45.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-19T16:42:45.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-19T16:42:45.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-19T16:42:45.223 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:42:45.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:45.397 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:42:45.397 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:42:45.397 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T16:42:45.397 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2023-12-19T16:42:45.397 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:42:45.440 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:42:45.440 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T16:42:45.495 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:42:45.495 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:42:45.495 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T16:42:45.550 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T16:42:45.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:45.550 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:42:45.662 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:42:45.662 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:42:45.768 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:42:45.768 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:42:45.768 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T16:42:45.768 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T16:42:45.768 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T16:42:45.772 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T16:42:45.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:45.773 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T16:42:45.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:45.795 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:42:45.866 INFO:teuthology.orchestra.run.smithi026.stdout:client4532 2023-12-19T16:42:45.866 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:42:45.866 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:42:45.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:45.920 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:42:45.975 INFO:teuthology.orchestra.run.smithi026.stdout:client4532 2023-12-19T16:42:45.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:45.975 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:42:46.030 INFO:teuthology.orchestra.run.smithi026.stdout:client4532 2023-12-19T16:42:46.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:46.030 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/2264810b-2623-438e-8790-ac9d6fbbbfd5.client4532/status') 2023-12-19T16:42:46.095 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.4532 (3)172.21.15.26:0/2603198793 2023-12-19T16:42:46.096 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2023-12-19T16:42:46.096 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2023-12-19T16:42:46.096 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2023-12-19T16:42:46.096 INFO:teuthology.orchestra.run.smithi026.stderr:70 bytes copied, 5.6369e-05 s, 1.2 MB/s 2023-12-19T16:42:46.098 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-12-19T16:42:46.098 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-19T16:42:46.098 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-19T16:42:46.098 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T16:42:46.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bbb6fc640 1 Processor -- start 2023-12-19T16:42:46.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bbb6fc640 1 -- start start 2023-12-19T16:42:46.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bbb6fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9bb4154220 0x7f9bb415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:46.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bbb6fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9bb406b3d0 0x7f9bb40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:46.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bbb6fc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb406bc30 con 0x7f9bb40cec40 2023-12-19T16:42:46.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bbb6fc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb40cf0d0 con 0x7f9bb4154220 2023-12-19T16:42:46.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bbb6fc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb40cc6e0 con 0x7f9bb406b3d0 2023-12-19T16:42:46.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bb9498640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9bb4154220 0x7f9bb415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:46.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bb9498640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9bb4154220 0x7f9bb415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52724/0 (socket says 172.21.15.26:52724) 2023-12-19T16:42:46.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bb9498640 1 -- 172.21.15.26:0/2631775687 learned_addr learned my addr 172.21.15.26:0/2631775687 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bb9498640 1 -- 172.21.15.26:0/2631775687 >> v2:172.21.15.160:3300/0 conn(0x7f9bb406b3d0 msgr2=0x7f9bb40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bb9498640 1 --2- 172.21.15.26:0/2631775687 >> v2:172.21.15.160:3300/0 conn(0x7f9bb406b3d0 0x7f9bb40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bb9498640 1 -- 172.21.15.26:0/2631775687 >> v1:172.21.15.160:6789/0 conn(0x7f9bb40cec40 legacy=0x7f9bb415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.276+0000 7f9bb9498640 1 -- 172.21.15.26:0/2631775687 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb415afc0 con 0x7f9bb4154220 2023-12-19T16:42:46.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.280+0000 7f9bb9498640 1 --2- 172.21.15.26:0/2631775687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb4154220 0x7f9bb415aa80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40cf880 tx=0x7f9bb001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12106d9b20d653fc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:46.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.280+0000 7f9babfff640 1 -- 172.21.15.26:0/2631775687 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00ac410 con 0x7f9bb4154220 2023-12-19T16:42:46.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.280+0000 7f9babfff640 1 -- 172.21.15.26:0/2631775687 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb00ac570 con 0x7f9bb4154220 2023-12-19T16:42:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.280+0000 7f9babfff640 1 -- 172.21.15.26:0/2631775687 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00ac900 con 0x7f9bb4154220 2023-12-19T16:42:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.280+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/2631775687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb4154220 msgr2=0x7f9bb415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.280+0000 7f9bbb6fc640 1 --2- 172.21.15.26:0/2631775687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb4154220 0x7f9bb415aa80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9bb40cf880 tx=0x7f9bb001c490 comp rx=0 tx=0).stop 2023-12-19T16:42:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.280+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/2631775687 shutdown_connections 2023-12-19T16:42:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.280+0000 7f9bbb6fc640 1 --2- 172.21.15.26:0/2631775687 >> v2:172.21.15.160:3300/0 conn(0x7f9bb406b3d0 0x7f9bb40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.280+0000 7f9bbb6fc640 1 --2- 172.21.15.26:0/2631775687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb4154220 0x7f9bb415aa80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.280+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/2631775687 >> 172.21.15.26:0/2631775687 conn(0x7f9bb40658e0 msgr2=0x7f9bb4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.280+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/2631775687 shutdown_connections 2023-12-19T16:42:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.280+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/2631775687 wait complete. 2023-12-19T16:42:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bbb6fc640 1 Processor -- start 2023-12-19T16:42:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bbb6fc640 1 -- start start 2023-12-19T16:42:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bbb6fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9bb406b3d0 0x7f9bb414db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bbb6fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bb40cec40 0x7f9bb414d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bbb6fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb4154220 0x7f9bb40b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bbb6fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb414e760 con 0x7f9bb4154220 2023-12-19T16:42:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bbb6fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb414d890 con 0x7f9bb40cec40 2023-12-19T16:42:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bbb6fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb414ee70 con 0x7f9bb406b3d0 2023-12-19T16:42:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bb9c99640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb4154220 0x7f9bb40b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bb9c99640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb4154220 0x7f9bb40b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52740/0 (socket says 172.21.15.26:52740) 2023-12-19T16:42:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bb9c99640 1 -- 172.21.15.26:0/1356034276 learned_addr learned my addr 172.21.15.26:0/1356034276 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bb9498640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9bb406b3d0 0x7f9bb414db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bb8c97640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bb40cec40 0x7f9bb414d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bb9c99640 1 -- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9bb406b3d0 msgr2=0x7f9bb414db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bb9c99640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9bb406b3d0 0x7f9bb414db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bb9c99640 1 -- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bb40cec40 msgr2=0x7f9bb414d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bb9c99640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bb40cec40 0x7f9bb414d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bb9c99640 1 -- 172.21.15.26:0/1356034276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb00094d0 con 0x7f9bb4154220 2023-12-19T16:42:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bb9c99640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb4154220 0x7f9bb40b1e00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00c780 tx=0x7f9bac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9ba9ffb640 1 -- 172.21.15.26:0/1356034276 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac003d90 con 0x7f9bb4154220 2023-12-19T16:42:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9ba9ffb640 1 -- 172.21.15.26:0/1356034276 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac00ce90 con 0x7f9bb4154220 2023-12-19T16:42:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9ba9ffb640 1 -- 172.21.15.26:0/1356034276 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac0052b0 con 0x7f9bb4154220 2023-12-19T16:42:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/1356034276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb414e1a0 con 0x7f9bb4154220 2023-12-19T16:42:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.284+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/1356034276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb414f110 con 0x7f9bb4154220 2023-12-19T16:42:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.288+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/1356034276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb414ecd0 con 0x7f9bb4154220 2023-12-19T16:42:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.288+0000 7f9ba9ffb640 1 -- 172.21.15.26:0/1356034276 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9bac005410 con 0x7f9bb4154220 2023-12-19T16:42:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.288+0000 7f9ba9ffb640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9b98065a60 0x7f9b98067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.288+0000 7f9ba9ffb640 1 -- 172.21.15.26:0/1356034276 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bac082780 con 0x7f9bb4154220 2023-12-19T16:42:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.288+0000 7f9bb9498640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9b98065a60 0x7f9b98067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.288+0000 7f9bb9498640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9b98065a60 0x7f9b98067f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00ae000 tx=0x7f9bb0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.292+0000 7f9ba9ffb640 1 -- 172.21.15.26:0/1356034276 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9bb414ecd0 con 0x7f9bb4154220 2023-12-19T16:42:46.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.484+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/1356034276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9bb414ecd0 con 0x7f9bb4154220 2023-12-19T16:42:46.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.484+0000 7f9ba9ffb640 1 -- 172.21.15.26:0/1356034276 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9bb414ecd0 con 0x7f9bb4154220 2023-12-19T16:42:46.489 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T16:42:46.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.488+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9b98065a60 msgr2=0x7f9b98067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:46.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.488+0000 7f9bbb6fc640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9b98065a60 0x7f9b98067f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00ae000 tx=0x7f9bb0008040 comp rx=0 tx=0).stop 2023-12-19T16:42:46.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.488+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/1356034276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb4154220 msgr2=0x7f9bb40b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:46.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.488+0000 7f9bbb6fc640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb4154220 0x7f9bb40b1e00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00c780 tx=0x7f9bac00cc40 comp rx=0 tx=0).stop 2023-12-19T16:42:46.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.488+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/1356034276 shutdown_connections 2023-12-19T16:42:46.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.488+0000 7f9bbb6fc640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb4154220 0x7f9bb40b1e00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:46.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.488+0000 7f9bbb6fc640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bb40cec40 0x7f9bb414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:46.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.488+0000 7f9bbb6fc640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9b98065a60 0x7f9b98067f20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:46.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.488+0000 7f9bbb6fc640 1 --2- 172.21.15.26:0/1356034276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9bb406b3d0 0x7f9bb414db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:46.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.488+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/1356034276 >> 172.21.15.26:0/1356034276 conn(0x7f9bb40658e0 msgr2=0x7f9bb4157c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:46.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.488+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/1356034276 shutdown_connections 2023-12-19T16:42:46.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:46.488+0000 7f9bbb6fc640 1 -- 172.21.15.26:0/1356034276 wait complete. 2023-12-19T16:42:46.510 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T16:42:46.510 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T16:42:46.510 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T16:42:46.510 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-12-19T16:42:46.510 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-19T16:42:46.510 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T16:42:46.510 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T16:42:46.510 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T16:42:46.510 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-12-19T16:42:46.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:46.510 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T16:42:46.521 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T16:42:46.521 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T16:42:46.522 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42374sec preferred_lft 42374sec 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T16:42:46.523 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:46.524 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-12-19T16:42:46.524 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:46.524 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T16:42:46.524 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T16:42:46.524 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2023-12-19T16:42:46.524 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:46.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:46.524 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T16:42:46.585 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T16:42:46.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:46.588 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T16:42:46.640 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T16:42:46.640 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T16:42:46.640 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T16:42:46.641 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T16:42:46.641 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T16:42:46.641 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T16:42:46.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:46.641 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:42:46.642 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:42:46.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T16:42:46.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T16:42:46.642 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T16:42:46.642 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:42:46.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:46.736 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T16:42:46.746 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T16:42:46.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:46.749 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2023-12-19T16:42:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-12-19T16:42:46.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:46.800 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:42:46.800 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:42:46.800 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T16:42:46.800 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-12-19T16:42:46.800 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:42:46.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:46.883 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-19T16:42:46.948 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T16:42:46.948 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T16:42:46.948 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T16:42:46.949 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:46.949 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T16:42:46.949 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:46.949 INFO:teuthology.orchestra.run.smithi026.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T16:42:46.949 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5a:1a:3b:0e:35:c0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-19T16:42:46.949 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-19T16:42:46.950 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:46.950 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::581a:3bff:fe0e:35c0/64 scope link tentative 2023-12-19T16:42:46.950 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:46.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:46.954 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-19T16:42:47.015 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T16:42:47.016 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T16:42:47.016 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T16:42:47.016 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:47.016 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T16:42:47.016 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:47.016 INFO:teuthology.orchestra.run.smithi026.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T16:42:47.016 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5a:1a:3b:0e:35:c0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-19T16:42:47.016 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-19T16:42:47.016 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:47.016 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::581a:3bff:fe0e:35c0/64 scope link tentative 2023-12-19T16:42:47.016 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:47.017 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-12-19T16:42:47.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:47.018 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:42:47.018 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:42:47.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-12-19T16:42:47.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-19T16:42:47.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-12-19T16:42:47.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-12-19T16:42:47.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-12-19T16:42:47.018 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:42:47.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:47.162 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:42:47.163 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:42:47.163 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.1 up 2023-12-19T16:42:47.163 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.1 master ceph-brx 2023-12-19T16:42:47.163 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:42:47.203 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:42:47.203 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-12-19T16:42:47.259 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-12-19T16:42:47.259 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:42:47.259 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-12-19T16:42:47.314 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-12-19T16:42:47.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:47.315 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:42:47.410 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:42:47.410 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:42:47.424 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:42:47.424 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:42:47.424 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T16:42:47.425 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 2@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T16:42:47.425 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T16:42:47.429 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T16:42:47.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:47.429 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-12-19T16:42:47.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:47.452 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:42:47.510 INFO:teuthology.orchestra.run.smithi026.stdout:client4538 2023-12-19T16:42:47.511 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:42:47.511 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:42:47.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:47.567 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:42:47.626 INFO:teuthology.orchestra.run.smithi026.stdout:client4538 2023-12-19T16:42:47.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:47.626 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:42:47.685 INFO:teuthology.orchestra.run.smithi026.stdout:client4538 2023-12-19T16:42:47.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:47.686 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/2264810b-2623-438e-8790-ac9d6fbbbfd5.client4538/status') 2023-12-19T16:42:47.747 INFO:teuthology.orchestra.run.smithi026.stdout:instance: client.4538 (3)172.21.15.26:0/3746194676 2023-12-19T16:42:47.747 INFO:teuthology.orchestra.run.smithi026.stdout:blocklisted: false 2023-12-19T16:42:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records in 2023-12-19T16:42:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:0+1 records out 2023-12-19T16:42:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:70 bytes copied, 5.4675e-05 s, 1.3 MB/s 2023-12-19T16:42:47.749 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-12-19T16:42:47.749 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-19T16:42:47.750 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T16:42:47.750 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T16:42:47.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.928+0000 7fe4d4ae5640 1 Processor -- start 2023-12-19T16:42:47.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.928+0000 7fe4d4ae5640 1 -- start start 2023-12-19T16:42:47.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4d4ae5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d00b34e0 0x7fe4d00ad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:47.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4d4ae5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d00cc0d0 con 0x7fe4d006cb30 2023-12-19T16:42:47.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4d4ae5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d015ddc0 con 0x7fe4d00cbc50 2023-12-19T16:42:47.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4d4ae5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d015e0a0 con 0x7fe4d00b34e0 2023-12-19T16:42:47.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4ce59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe4d006cb30 0x7fe4d015d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37198/0 (socket says 172.21.15.26:37198) 2023-12-19T16:42:47.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4ce59c640 1 -- 172.21.15.26:0/1182825435 learned_addr learned my addr 172.21.15.26:0/1182825435 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:47.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4ced9d640 1 --2- 172.21.15.26:0/1182825435 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d00b34e0 0x7fe4d00ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:47.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970063142 0 0) 0x7fe4c0001120 con 0x7fe4d006cb30 2023-12-19T16:42:47.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4bc003530 con 0x7fe4d006cb30 2023-12-19T16:42:47.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3590441747 0 0) 0x7fe4bc003530 con 0x7fe4d006cb30 2023-12-19T16:42:47.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d00b34e0 msgr2=0x7fe4d00ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:47.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 --2- 172.21.15.26:0/1182825435 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d00b34e0 0x7fe4d00ad180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:47.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 >> v1:172.21.15.160:6789/0 conn(0x7fe4d00cbc50 legacy=0x7fe4d00b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:47.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d00cb450 con 0x7fe4d006cb30 2023-12-19T16:42:47.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fe4c0001810 con 0x7fe4d006cb30 2023-12-19T16:42:47.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 >> v1:172.21.15.26:6789/0 conn(0x7fe4d006cb30 legacy=0x7fe4d015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:47.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 --2- 172.21.15.26:0/1182825435 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d00b34e0 0x7fe4bc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:47.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 --2- 172.21.15.26:0/1182825435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4bc003ef0 0x7fe4bc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:47.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 --2- 172.21.15.26:0/1182825435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4bc004d10 0x7fe4bc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:47.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d015e0a0 con 0x7fe4bc003ef0 2023-12-19T16:42:47.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc00b9f0 con 0x7fe4bc004d10 2023-12-19T16:42:47.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc00bbb0 con 0x7fe4d00b34e0 2023-12-19T16:42:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4c7fff640 1 --2- 172.21.15.26:0/1182825435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4bc003ef0 0x7fe4bc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4ced9d640 1 --2- 172.21.15.26:0/1182825435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4bc004d10 0x7fe4bc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4ce59c640 1 --2- 172.21.15.26:0/1182825435 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d00b34e0 0x7fe4bc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4c7fff640 1 -- 172.21.15.26:0/1182825435 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d00b34e0 msgr2=0x7fe4bc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4c7fff640 1 --2- 172.21.15.26:0/1182825435 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d00b34e0 0x7fe4bc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:47.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4c7fff640 1 -- 172.21.15.26:0/1182825435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4bc004d10 msgr2=0x7fe4bc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:47.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4c7fff640 1 --2- 172.21.15.26:0/1182825435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4bc004d10 0x7fe4bc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:47.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.932+0000 7fe4c7fff640 1 -- 172.21.15.26:0/1182825435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc003530 con 0x7fe4bc003ef0 2023-12-19T16:42:47.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.936+0000 7fe4c7fff640 1 --2- 172.21.15.26:0/1182825435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4bc003ef0 0x7fe4bc0047d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8001860 tx=0x7fe4b801eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2cb5637bc22d0a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.936+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b80a87b0 con 0x7fe4bc003ef0 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.936+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b80a8910 con 0x7fe4bc003ef0 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.936+0000 7fe4cdd9b640 1 -- 172.21.15.26:0/1182825435 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b80a8d00 con 0x7fe4bc003ef0 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.936+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/1182825435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4bc003ef0 msgr2=0x7fe4bc0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.936+0000 7fe4d4ae5640 1 --2- 172.21.15.26:0/1182825435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4bc003ef0 0x7fe4bc0047d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8001860 tx=0x7fe4b801eb90 comp rx=0 tx=0).stop 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.936+0000 7fe4ced9d640 1 -- 172.21.15.26:0/1182825435 reap_dead start 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.936+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/1182825435 shutdown_connections 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.936+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/1182825435 >> 172.21.15.26:0/1182825435 conn(0x7fe4d00658e0 msgr2=0x7fe4d0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.936+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/1182825435 shutdown_connections 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.936+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/1182825435 wait complete. 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4d4ae5640 1 Processor -- start 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4d4ae5640 1 -- start start 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4d4ae5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4bc003ef0 0x7fe4d01339c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:47.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4d4ae5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4d00b09b0 0x7fe4d01f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:47.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4d4ae5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d01f5f80 0x7fe4d015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:47.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4d4ae5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d0137150 con 0x7fe4d00b09b0 2023-12-19T16:42:47.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4d4ae5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d01f65b0 con 0x7fe4bc003ef0 2023-12-19T16:42:47.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4d4ae5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d0132ee0 con 0x7fe4d01f5f80 2023-12-19T16:42:47.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4d00b09b0 0x7fe4d01f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:47.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4d00b09b0 0x7fe4d01f5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52770/0 (socket says 172.21.15.26:52770) 2023-12-19T16:42:47.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4c7fff640 1 -- 172.21.15.26:0/816094044 learned_addr learned my addr 172.21.15.26:0/816094044 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:47.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4ce59c640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4bc003ef0 0x7fe4d01339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:47.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4ced9d640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d01f5f80 0x7fe4d015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:47.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4c7fff640 1 -- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d01f5f80 msgr2=0x7fe4d015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4c7fff640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d01f5f80 0x7fe4d015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4c7fff640 1 -- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4bc003ef0 msgr2=0x7fe4d01339c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4c7fff640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4bc003ef0 0x7fe4d01339c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4c7fff640 1 -- 172.21.15.26:0/816094044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d01331c0 con 0x7fe4d00b09b0 2023-12-19T16:42:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4ced9d640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d01f5f80 0x7fe4d015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4c7fff640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4d00b09b0 0x7fe4d01f5a40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80b3040 tx=0x7fe4bc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4c6ffd640 1 -- 172.21.15.26:0/816094044 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8005070 con 0x7fe4d00b09b0 2023-12-19T16:42:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4c6ffd640 1 -- 172.21.15.26:0/816094044 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b80051d0 con 0x7fe4d00b09b0 2023-12-19T16:42:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4c6ffd640 1 -- 172.21.15.26:0/816094044 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b80067e0 con 0x7fe4d00b09b0 2023-12-19T16:42:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/816094044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d01334a0 con 0x7fe4d00b09b0 2023-12-19T16:42:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/816094044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d00b0e30 con 0x7fe4d00b09b0 2023-12-19T16:42:47.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.940+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/816094044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d01f6360 con 0x7fe4d00b09b0 2023-12-19T16:42:47.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.944+0000 7fe4c6ffd640 1 -- 172.21.15.26:0/816094044 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe4b80b7040 con 0x7fe4d00b09b0 2023-12-19T16:42:47.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.944+0000 7fe4c6ffd640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe4b0065a90 0x7fe4b0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:47.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.944+0000 7fe4c6ffd640 1 -- 172.21.15.26:0/816094044 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4b811f650 con 0x7fe4d00b09b0 2023-12-19T16:42:47.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.944+0000 7fe4ce59c640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe4b0065a90 0x7fe4b0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:47.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.944+0000 7fe4ce59c640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe4b0065a90 0x7fe4b0067f50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0000f70 tx=0x7fe4c0000f00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:47.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:47.948+0000 7fe4c6ffd640 1 -- 172.21.15.26:0/816094044 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe4b80ed010 con 0x7fe4d00b09b0 2023-12-19T16:42:48.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.136+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/816094044 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe4d00b1020 con 0x7fe4d00b09b0 2023-12-19T16:42:48.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.140+0000 7fe4c6ffd640 1 -- 172.21.15.26:0/816094044 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe4b80f0630 con 0x7fe4d00b09b0 2023-12-19T16:42:48.143 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T16:42:48.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.140+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe4b0065a90 msgr2=0x7fe4b0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:48.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.140+0000 7fe4d4ae5640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe4b0065a90 0x7fe4b0067f50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0000f70 tx=0x7fe4c0000f00 comp rx=0 tx=0).stop 2023-12-19T16:42:48.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.140+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/816094044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4d00b09b0 msgr2=0x7fe4d01f5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:48.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.140+0000 7fe4d4ae5640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4d00b09b0 0x7fe4d01f5a40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80b3040 tx=0x7fe4bc005800 comp rx=0 tx=0).stop 2023-12-19T16:42:48.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.140+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/816094044 shutdown_connections 2023-12-19T16:42:48.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.140+0000 7fe4d4ae5640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe4b0065a90 0x7fe4b0067f50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:48.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.140+0000 7fe4d4ae5640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4d01f5f80 0x7fe4d015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:48.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.140+0000 7fe4d4ae5640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4d00b09b0 0x7fe4d01f5a40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:48.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.140+0000 7fe4d4ae5640 1 --2- 172.21.15.26:0/816094044 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4bc003ef0 0x7fe4d01339c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:48.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.140+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/816094044 >> 172.21.15.26:0/816094044 conn(0x7fe4d00658e0 msgr2=0x7fe4d0160d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:48.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.140+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/816094044 shutdown_connections 2023-12-19T16:42:48.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:48.140+0000 7fe4d4ae5640 1 -- 172.21.15.26:0/816094044 wait complete. 2023-12-19T16:42:48.164 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T16:42:48.164 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T16:42:48.164 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi160.front.sepia.ceph.com 2023-12-19T16:42:48.164 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-19T16:42:48.164 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T16:42:48.164 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T16:42:48.164 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T16:42:48.164 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T16:42:48.164 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T16:42:48.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:48.165 DEBUG:teuthology.orchestra.run.smithi160:> ip addr 2023-12-19T16:42:48.175 INFO:teuthology.orchestra.run.smithi160.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T16:42:48.175 INFO:teuthology.orchestra.run.smithi160.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T16:42:48.175 INFO:teuthology.orchestra.run.smithi160.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 ::1/128 scope host 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:80 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout: altname enp7s0f0 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:81 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout: altname enp7s0f1 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:48.176 INFO:teuthology.orchestra.run.smithi160.stdout: inet 172.21.15.160/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-12-19T16:42:48.177 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft 42378sec preferred_lft 42378sec 2023-12-19T16:42:48.177 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::ec4:7aff:fe88:7f5b/64 scope link 2023-12-19T16:42:48.177 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:48.177 INFO:teuthology.orchestra.run.smithi160.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T16:42:48.177 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 02:42:51:45:62:2d brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:48.177 INFO:teuthology.orchestra.run.smithi160.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T16:42:48.177 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:48.177 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-19T16:42:48.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:48.178 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-12-19T16:42:48.178 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-12-19T16:42:48.178 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add name ceph-brx type bridge 2023-12-19T16:42:48.178 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip addr flush dev ceph-brx 2023-12-19T16:42:48.178 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set ceph-brx up 2023-12-19T16:42:48.178 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-19T16:42:48.178 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-12-19T16:42:48.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:48.296 DEBUG:teuthology.orchestra.run.smithi160:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T16:42:48.316 INFO:teuthology.orchestra.run.smithi160.stdout:1 2023-12-19T16:42:48.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:48.318 DEBUG:teuthology.orchestra.run.smithi160:> route 2023-12-19T16:42:48.384 INFO:teuthology.orchestra.run.smithi160.stdout:Kernel IP routing table 2023-12-19T16:42:48.385 INFO:teuthology.orchestra.run.smithi160.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T16:42:48.385 INFO:teuthology.orchestra.run.smithi160.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-12-19T16:42:48.385 INFO:teuthology.orchestra.run.smithi160.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T16:42:48.385 INFO:teuthology.orchestra.run.smithi160.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T16:42:48.385 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T16:42:48.385 INFO:teuthology.orchestra.run.smithi160.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-12-19T16:42:48.385 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-12-19T16:42:48.385 INFO:teuthology.orchestra.run.smithi160.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-12-19T16:42:48.385 INFO:teuthology.orchestra.run.smithi160.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T16:42:48.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:48.386 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-12-19T16:42:48.386 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-12-19T16:42:48.386 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T16:42:48.386 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T16:42:48.386 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T16:42:48.386 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-12-19T16:42:48.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:48.479 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2023-12-19T16:42:48.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:48.489 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list-id 2023-12-19T16:42:48.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:48.539 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-12-19T16:42:48.539 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-12-19T16:42:48.539 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T16:42:48.539 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-12-19T16:42:48.539 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-12-19T16:42:48.627 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-12-19T16:42:48.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:48.627 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-12-19T16:42:48.627 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-12-19T16:42:48.627 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-12-19T16:42:48.627 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-19T16:42:48.628 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-12-19T16:42:48.628 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-12-19T16:42:48.628 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-12-19T16:42:48.628 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-12-19T16:42:48.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:48.797 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-12-19T16:42:48.797 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-12-19T16:42:48.798 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set brx.0 up 2023-12-19T16:42:48.798 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set dev brx.0 master ceph-brx 2023-12-19T16:42:48.798 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-12-19T16:42:48.839 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:42:48.839 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-19T16:42:48.895 INFO:teuthology.orchestra.run.smithi160.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-19T16:42:48.896 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:42:48.896 DEBUG:teuthology.orchestra.run.smithi160:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-19T16:42:48.946 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-12-19T16:42:48.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:48.947 DEBUG:teuthology.orchestra.run.smithi160:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:42:49.055 INFO:teuthology.orchestra.run.smithi160.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:42:49.055 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:42:49.154 INFO:teuthology.orchestra.run.smithi160.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:42:49.154 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:42:49.154 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T16:42:49.154 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: trying mount with new device syntax: 1@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T16:42:49.154 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T16:42:49.158 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T16:42:49.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:49.159 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-19T16:42:49.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:49.180 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:42:49.249 INFO:teuthology.orchestra.run.smithi160.stdout:client4547 2023-12-19T16:42:49.249 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:49.249 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:42:49.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:49.299 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:42:49.357 INFO:teuthology.orchestra.run.smithi160.stdout:client4547 2023-12-19T16:42:49.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:49.358 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:42:49.414 INFO:teuthology.orchestra.run.smithi160.stdout:client4547 2023-12-19T16:42:49.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:49.414 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/2264810b-2623-438e-8790-ac9d6fbbbfd5.client4547/status') 2023-12-19T16:42:49.474 INFO:teuthology.orchestra.run.smithi160.stdout:instance: client.4547 (3)192.168.0.1:0/3216450448 2023-12-19T16:42:49.474 INFO:teuthology.orchestra.run.smithi160.stdout:blocklisted: false 2023-12-19T16:42:49.474 INFO:teuthology.orchestra.run.smithi160.stderr:0+1 records in 2023-12-19T16:42:49.474 INFO:teuthology.orchestra.run.smithi160.stderr:0+1 records out 2023-12-19T16:42:49.475 INFO:teuthology.orchestra.run.smithi160.stderr:69 bytes copied, 6.2832e-05 s, 1.1 MB/s 2023-12-19T16:42:49.476 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-12-19T16:42:49.476 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-19T16:42:49.476 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-19T16:42:49.476 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T16:42:49.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a54ae5640 1 Processor -- start 2023-12-19T16:42:49.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a54ae5640 1 -- start start 2023-12-19T16:42:49.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a54ae5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a500ce040 0x7f6a500d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:49.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a54ae5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a500d29d0 con 0x7f6a500d3130 2023-12-19T16:42:49.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a54ae5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a500d3570 con 0x7f6a5015cd40 2023-12-19T16:42:49.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a54ae5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5015d680 con 0x7f6a500ce040 2023-12-19T16:42:49.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a4ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6a500d3130 0x7f6a500d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37204/0 (socket says 172.21.15.26:37204) 2023-12-19T16:42:49.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a4ed9d640 1 -- 172.21.15.26:0/253242622 learned_addr learned my addr 172.21.15.26:0/253242622 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:49.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a4dd9b640 1 --2- 172.21.15.26:0/253242622 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a500ce040 0x7f6a500d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:49.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a4d59a640 1 -- 172.21.15.26:0/253242622 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3822987754 0 0) 0x7f6a480014a0 con 0x7f6a500d3130 2023-12-19T16:42:49.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a4d59a640 1 -- 172.21.15.26:0/253242622 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a38003530 con 0x7f6a500d3130 2023-12-19T16:42:49.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a4d59a640 1 -- 172.21.15.26:0/253242622 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343920329 0 0) 0x7f6a500d3570 con 0x7f6a5015cd40 2023-12-19T16:42:49.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a4d59a640 1 -- 172.21.15.26:0/253242622 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a480014a0 con 0x7f6a5015cd40 2023-12-19T16:42:49.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a4dd9b640 1 -- 172.21.15.26:0/253242622 >> v1:172.21.15.160:6789/0 conn(0x7f6a5015cd40 legacy=0x7f6a50160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:49.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a4dd9b640 1 -- 172.21.15.26:0/253242622 >> v1:172.21.15.26:6789/0 conn(0x7f6a500d3130 legacy=0x7f6a500d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:49.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.628+0000 7f6a4dd9b640 1 -- 172.21.15.26:0/253242622 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a50161750 con 0x7f6a500ce040 2023-12-19T16:42:49.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.632+0000 7f6a4dd9b640 1 --2- 172.21.15.26:0/253242622 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a500ce040 0x7f6a500d2bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a440097f0 tx=0x7f6a4401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e1aec83fad15148 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:49.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.632+0000 7f6a4d59a640 1 -- 172.21.15.26:0/253242622 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a440a6040 con 0x7f6a500ce040 2023-12-19T16:42:49.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.632+0000 7f6a4d59a640 1 -- 172.21.15.26:0/253242622 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a440aa600 con 0x7f6a500ce040 2023-12-19T16:42:49.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.632+0000 7f6a4d59a640 1 -- 172.21.15.26:0/253242622 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a440aa990 con 0x7f6a500ce040 2023-12-19T16:42:49.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.632+0000 7f6a54ae5640 1 -- 172.21.15.26:0/253242622 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a500ce040 msgr2=0x7f6a500d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:49.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.632+0000 7f6a54ae5640 1 --2- 172.21.15.26:0/253242622 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a500ce040 0x7f6a500d2bf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a440097f0 tx=0x7f6a4401c4a0 comp rx=0 tx=0).stop 2023-12-19T16:42:49.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.632+0000 7f6a54ae5640 1 -- 172.21.15.26:0/253242622 shutdown_connections 2023-12-19T16:42:49.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.632+0000 7f6a54ae5640 1 --2- 172.21.15.26:0/253242622 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a500ce040 0x7f6a500d2bf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:49.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.632+0000 7f6a54ae5640 1 -- 172.21.15.26:0/253242622 >> 172.21.15.26:0/253242622 conn(0x7f6a5006a120 msgr2=0x7f6a5006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:49.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.632+0000 7f6a54ae5640 1 -- 172.21.15.26:0/253242622 shutdown_connections 2023-12-19T16:42:49.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.632+0000 7f6a54ae5640 1 -- 172.21.15.26:0/253242622 wait complete. 2023-12-19T16:42:49.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a54ae5640 1 Processor -- start 2023-12-19T16:42:49.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a54ae5640 1 -- start start 2023-12-19T16:42:49.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a54ae5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a500ce040 0x7f6a500c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:49.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a54ae5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a500d3130 0x7f6a500c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:49.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a54ae5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a5015cd40 0x7f6a501549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:49.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a54ae5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500ca270 con 0x7f6a500d3130 2023-12-19T16:42:49.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a54ae5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500ca480 con 0x7f6a5015cd40 2023-12-19T16:42:49.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a54ae5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500ca760 con 0x7f6a500ce040 2023-12-19T16:42:49.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a4dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a500d3130 0x7f6a500c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:49.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a4dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a500d3130 0x7f6a500c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52786/0 (socket says 172.21.15.26:52786) 2023-12-19T16:42:49.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a4dd9b640 1 -- 172.21.15.26:0/3175300178 learned_addr learned my addr 172.21.15.26:0/3175300178 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:49.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a4e59c640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a500ce040 0x7f6a500c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:49.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a4ed9d640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a5015cd40 0x7f6a501549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:49.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a4dd9b640 1 -- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a500ce040 msgr2=0x7f6a500c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:49.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a4dd9b640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a500ce040 0x7f6a500c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:49.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a4dd9b640 1 -- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a5015cd40 msgr2=0x7f6a501549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:49.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a4dd9b640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a5015cd40 0x7f6a501549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:49.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a4dd9b640 1 -- 172.21.15.26:0/3175300178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a44009420 con 0x7f6a500d3130 2023-12-19T16:42:49.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a4dd9b640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a500d3130 0x7f6a500c9d30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6a440190b0 tx=0x7f6a4401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:49.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a3effd640 1 -- 172.21.15.26:0/3175300178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a440b3600 con 0x7f6a500d3130 2023-12-19T16:42:49.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a3effd640 1 -- 172.21.15.26:0/3175300178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a440b3780 con 0x7f6a500d3130 2023-12-19T16:42:49.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a3effd640 1 -- 172.21.15.26:0/3175300178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a440b3b30 con 0x7f6a500d3130 2023-12-19T16:42:49.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a54ae5640 1 -- 172.21.15.26:0/3175300178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a500caa40 con 0x7f6a500d3130 2023-12-19T16:42:49.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.636+0000 7f6a54ae5640 1 -- 172.21.15.26:0/3175300178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a500cb000 con 0x7f6a500d3130 2023-12-19T16:42:49.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.640+0000 7f6a54ae5640 1 -- 172.21.15.26:0/3175300178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a50074270 con 0x7f6a500d3130 2023-12-19T16:42:49.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.640+0000 7f6a3effd640 1 -- 172.21.15.26:0/3175300178 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6a440b3c90 con 0x7f6a500d3130 2023-12-19T16:42:49.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.640+0000 7f6a3effd640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6a30065a60 0x7f6a30067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:49.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.640+0000 7f6a3effd640 1 -- 172.21.15.26:0/3175300178 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a4411d5a0 con 0x7f6a500d3130 2023-12-19T16:42:49.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.640+0000 7f6a4e59c640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6a30065a60 0x7f6a30067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:49.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.640+0000 7f6a4e59c640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6a30065a60 0x7f6a30067f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a40001240 tx=0x7f6a40009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:49.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.644+0000 7f6a3effd640 1 -- 172.21.15.26:0/3175300178 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6a440eb1c0 con 0x7f6a500d3130 2023-12-19T16:42:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.832+0000 7f6a54ae5640 1 -- 172.21.15.26:0/3175300178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6a500cdaf0 con 0x7f6a500d3130 2023-12-19T16:42:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.832+0000 7f6a3effd640 1 -- 172.21.15.26:0/3175300178 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6a440ee7e0 con 0x7f6a500d3130 2023-12-19T16:42:49.838 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T16:42:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.836+0000 7f6a54ae5640 1 -- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6a30065a60 msgr2=0x7f6a30067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.836+0000 7f6a54ae5640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6a30065a60 0x7f6a30067f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a40001240 tx=0x7f6a40009040 comp rx=0 tx=0).stop 2023-12-19T16:42:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.836+0000 7f6a54ae5640 1 -- 172.21.15.26:0/3175300178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a500d3130 msgr2=0x7f6a500c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.836+0000 7f6a54ae5640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a500d3130 0x7f6a500c9d30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6a440190b0 tx=0x7f6a4401ace0 comp rx=0 tx=0).stop 2023-12-19T16:42:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.836+0000 7f6a54ae5640 1 -- 172.21.15.26:0/3175300178 shutdown_connections 2023-12-19T16:42:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.836+0000 7f6a54ae5640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a5015cd40 0x7f6a501549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:49.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.836+0000 7f6a54ae5640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a500d3130 0x7f6a500c9d30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:49.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.836+0000 7f6a54ae5640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6a30065a60 0x7f6a30067f20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:49.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.836+0000 7f6a54ae5640 1 --2- 172.21.15.26:0/3175300178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a500ce040 0x7f6a500c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:49.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.836+0000 7f6a54ae5640 1 -- 172.21.15.26:0/3175300178 >> 172.21.15.26:0/3175300178 conn(0x7f6a5006a120 msgr2=0x7f6a500d1950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:49.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.836+0000 7f6a54ae5640 1 -- 172.21.15.26:0/3175300178 shutdown_connections 2023-12-19T16:42:49.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:49.836+0000 7f6a54ae5640 1 -- 172.21.15.26:0/3175300178 wait complete. 2023-12-19T16:42:49.859 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T16:42:49.859 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T16:42:49.859 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi160.front.sepia.ceph.com 2023-12-19T16:42:49.859 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-12-19T16:42:49.859 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-19T16:42:49.859 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T16:42:49.859 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T16:42:49.859 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T16:42:49.859 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-12-19T16:42:49.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:49.860 DEBUG:teuthology.orchestra.run.smithi160:> ip addr 2023-12-19T16:42:49.870 INFO:teuthology.orchestra.run.smithi160.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T16:42:49.870 INFO:teuthology.orchestra.run.smithi160.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T16:42:49.870 INFO:teuthology.orchestra.run.smithi160.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T16:42:49.870 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:49.870 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 ::1/128 scope host 2023-12-19T16:42:49.870 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:49.870 INFO:teuthology.orchestra.run.smithi160.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:42:49.871 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:80 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:49.871 INFO:teuthology.orchestra.run.smithi160.stdout: altname enp7s0f0 2023-12-19T16:42:49.871 INFO:teuthology.orchestra.run.smithi160.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:42:49.871 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:81 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:49.871 INFO:teuthology.orchestra.run.smithi160.stdout: altname enp7s0f1 2023-12-19T16:42:49.871 INFO:teuthology.orchestra.run.smithi160.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:42:49.872 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:49.872 INFO:teuthology.orchestra.run.smithi160.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T16:42:49.872 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:49.872 INFO:teuthology.orchestra.run.smithi160.stdout: inet 172.21.15.160/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-12-19T16:42:49.872 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft 42376sec preferred_lft 42376sec 2023-12-19T16:42:49.872 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::ec4:7aff:fe88:7f5b/64 scope link 2023-12-19T16:42:49.873 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:49.873 INFO:teuthology.orchestra.run.smithi160.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T16:42:49.873 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 02:42:51:45:62:2d brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:49.873 INFO:teuthology.orchestra.run.smithi160.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T16:42:49.873 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:49.873 INFO:teuthology.orchestra.run.smithi160.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T16:42:49.874 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T16:42:49.874 INFO:teuthology.orchestra.run.smithi160.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T16:42:49.874 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:49.874 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-12-19T16:42:49.874 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:49.874 INFO:teuthology.orchestra.run.smithi160.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T16:42:49.874 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T16:42:49.874 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2023-12-19T16:42:49.875 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:49.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:49.875 DEBUG:teuthology.orchestra.run.smithi160:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T16:42:49.933 INFO:teuthology.orchestra.run.smithi160.stdout:1 2023-12-19T16:42:49.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:49.935 DEBUG:teuthology.orchestra.run.smithi160:> route 2023-12-19T16:42:49.989 INFO:teuthology.orchestra.run.smithi160.stdout:Kernel IP routing table 2023-12-19T16:42:49.989 INFO:teuthology.orchestra.run.smithi160.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T16:42:49.989 INFO:teuthology.orchestra.run.smithi160.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-12-19T16:42:49.989 INFO:teuthology.orchestra.run.smithi160.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T16:42:49.989 INFO:teuthology.orchestra.run.smithi160.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T16:42:49.989 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T16:42:49.989 INFO:teuthology.orchestra.run.smithi160.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-12-19T16:42:49.989 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-12-19T16:42:49.989 INFO:teuthology.orchestra.run.smithi160.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-12-19T16:42:49.989 INFO:teuthology.orchestra.run.smithi160.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T16:42:49.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:49.990 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-12-19T16:42:49.990 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-12-19T16:42:49.990 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T16:42:49.990 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T16:42:49.990 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T16:42:49.990 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-12-19T16:42:50.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:50.086 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2023-12-19T16:42:50.094 INFO:teuthology.orchestra.run.smithi160.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T16:42:50.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:50.096 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list-id 2023-12-19T16:42:50.147 INFO:teuthology.orchestra.run.smithi160.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-12-19T16:42:50.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:50.147 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-12-19T16:42:50.148 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-12-19T16:42:50.148 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T16:42:50.148 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-12-19T16:42:50.148 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-12-19T16:42:50.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:50.232 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-19T16:42:50.296 INFO:teuthology.orchestra.run.smithi160.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T16:42:50.297 INFO:teuthology.orchestra.run.smithi160.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T16:42:50.297 INFO:teuthology.orchestra.run.smithi160.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T16:42:50.297 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:50.297 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 ::1/128 scope host 2023-12-19T16:42:50.297 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:50.297 INFO:teuthology.orchestra.run.smithi160.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T16:42:50.297 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 82:5a:b5:71:d8:7a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-19T16:42:50.297 INFO:teuthology.orchestra.run.smithi160.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-19T16:42:50.297 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:50.297 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::805a:b5ff:fe71:d87a/64 scope link 2023-12-19T16:42:50.297 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:50.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:50.299 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-19T16:42:50.364 INFO:teuthology.orchestra.run.smithi160.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T16:42:50.364 INFO:teuthology.orchestra.run.smithi160.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T16:42:50.364 INFO:teuthology.orchestra.run.smithi160.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T16:42:50.364 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:50.364 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 ::1/128 scope host 2023-12-19T16:42:50.364 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:50.365 INFO:teuthology.orchestra.run.smithi160.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T16:42:50.365 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 82:5a:b5:71:d8:7a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-19T16:42:50.365 INFO:teuthology.orchestra.run.smithi160.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-19T16:42:50.365 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:50.365 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::805a:b5ff:fe71:d87a/64 scope link 2023-12-19T16:42:50.365 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:42:50.366 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-12-19T16:42:50.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:50.367 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-12-19T16:42:50.367 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-12-19T16:42:50.367 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-12-19T16:42:50.367 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-19T16:42:50.367 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-12-19T16:42:50.367 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-12-19T16:42:50.367 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-12-19T16:42:50.367 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-12-19T16:42:50.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:50.509 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec bash -c ' 2023-12-19T16:42:50.509 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2023-12-19T16:42:50.509 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set brx.1 up 2023-12-19T16:42:50.509 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set dev brx.1 master ceph-brx 2023-12-19T16:42:50.509 DEBUG:teuthology.orchestra.run.smithi160:> ') 2023-12-19T16:42:50.550 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:42:50.550 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-12-19T16:42:50.603 INFO:teuthology.orchestra.run.smithi160.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-12-19T16:42:50.603 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:42:50.603 DEBUG:teuthology.orchestra.run.smithi160:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-12-19T16:42:50.654 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-12-19T16:42:50.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:50.655 DEBUG:teuthology.orchestra.run.smithi160:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:42:50.748 INFO:teuthology.orchestra.run.smithi160.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:42:50.750 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:42:50.761 INFO:teuthology.orchestra.run.smithi160.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:42:50.761 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:42:50.761 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T16:42:50.761 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: trying mount with new device syntax: 3@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T16:42:50.761 INFO:teuthology.orchestra.run.smithi160.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T16:42:50.765 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T16:42:50.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:50.766 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-12-19T16:42:50.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:50.786 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:42:50.838 INFO:teuthology.orchestra.run.smithi160.stdout:client4556 2023-12-19T16:42:50.838 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:50.838 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:42:50.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:50.891 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:42:50.945 INFO:teuthology.orchestra.run.smithi160.stdout:client4556 2023-12-19T16:42:50.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:50.946 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:42:50.997 INFO:teuthology.orchestra.run.smithi160.stdout:client4556 2023-12-19T16:42:50.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:50.997 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/2264810b-2623-438e-8790-ac9d6fbbbfd5.client4556/status') 2023-12-19T16:42:51.062 INFO:teuthology.orchestra.run.smithi160.stdout:instance: client.4556 (3)192.168.0.2:0/3699665940 2023-12-19T16:42:51.062 INFO:teuthology.orchestra.run.smithi160.stdout:blocklisted: false 2023-12-19T16:42:51.063 INFO:teuthology.orchestra.run.smithi160.stderr:0+1 records in 2023-12-19T16:42:51.063 INFO:teuthology.orchestra.run.smithi160.stderr:0+1 records out 2023-12-19T16:42:51.063 INFO:teuthology.orchestra.run.smithi160.stderr:69 bytes copied, 5.6014e-05 s, 1.2 MB/s 2023-12-19T16:42:51.064 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-12-19T16:42:51.084 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:51.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.232+0000 7f1ba9873640 1 Processor -- start 2023-12-19T16:42:51.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.232+0000 7f1ba9873640 1 -- start start 2023-12-19T16:42:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.232+0000 7f1ba9873640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba4155030 0x7f1ba415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.232+0000 7f1ba9873640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1ba40ce740 0x7f1ba40cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.232+0000 7f1ba9873640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba406c970 con 0x7f1ba415f370 2023-12-19T16:42:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.232+0000 7f1ba9873640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba406c790 con 0x7f1ba40ce740 2023-12-19T16:42:51.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.232+0000 7f1ba9873640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba40cf130 con 0x7f1ba4155030 2023-12-19T16:42:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1b9bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1ba415f370 0x7f1ba406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37214/0 (socket says 172.21.15.26:37214) 2023-12-19T16:42:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1b9bfff640 1 -- 172.21.15.26:0/617824240 learned_addr learned my addr 172.21.15.26:0/617824240 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:51.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba37fe640 1 --2- 172.21.15.26:0/617824240 >> v2:172.21.15.160:3300/0 conn(0x7f1ba40ce740 0x7f1ba40cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba2ffd640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba4155030 0x7f1ba415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573112867 0 0) 0x7f1ba406c970 con 0x7f1ba415f370 2023-12-19T16:42:51.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b90003530 con 0x7f1ba415f370 2023-12-19T16:42:51.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2806825207 0 0) 0x7f1b90003530 con 0x7f1ba415f370 2023-12-19T16:42:51.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba4155030 msgr2=0x7f1ba415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:51.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba4155030 0x7f1ba415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 >> v2:172.21.15.160:3300/0 conn(0x7f1ba40ce740 msgr2=0x7f1ba40cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:51.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 --2- 172.21.15.26:0/617824240 >> v2:172.21.15.160:3300/0 conn(0x7f1ba40ce740 0x7f1ba40cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba415b1f0 con 0x7f1ba415f370 2023-12-19T16:42:51.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f1b8c001680 con 0x7f1ba415f370 2023-12-19T16:42:51.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 >> v1:172.21.15.26:6789/0 conn(0x7f1ba415f370 legacy=0x7f1ba406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b90003980 0x7f1b90003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b900047d0 0x7f1b90004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90005190 0x7f1b9000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba406c790 con 0x7f1b90005190 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba40cf130 con 0x7f1b90003980 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9000bed0 con 0x7f1b900047d0 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba37fe640 1 --2- 172.21.15.26:0/617824240 >> v2:172.21.15.160:3300/0 conn(0x7f1ba40ce740 0x7f1ba40cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba2ffd640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba4155030 0x7f1ba415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba37fe640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90005190 0x7f1b9000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1b9bfff640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b900047d0 0x7f1b90004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba2ffd640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b90003980 0x7f1b90003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba37fe640 1 -- 172.21.15.26:0/617824240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b900047d0 msgr2=0x7f1b90004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba37fe640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b900047d0 0x7f1b90004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba37fe640 1 -- 172.21.15.26:0/617824240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b90003980 msgr2=0x7f1b90003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba37fe640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b90003980 0x7f1b90003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba37fe640 1 -- 172.21.15.26:0/617824240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b90003530 con 0x7f1b90005190 2023-12-19T16:42:51.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.236+0000 7f1ba2ffd640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b90003980 0x7f1b90003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:51.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.240+0000 7f1ba37fe640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90005190 0x7f1b9000b990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00e380 tx=0x7f1b9c01ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4795daeee05aa477 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:51.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.240+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0a5e30 con 0x7f1b90005190 2023-12-19T16:42:51.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.240+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c00f040 con 0x7f1b90005190 2023-12-19T16:42:51.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.240+0000 7f1ba27fc640 1 -- 172.21.15.26:0/617824240 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c003ba0 con 0x7f1b90005190 2023-12-19T16:42:51.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.240+0000 7f1ba9873640 1 -- 172.21.15.26:0/617824240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90005190 msgr2=0x7f1b9000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.240+0000 7f1ba9873640 1 --2- 172.21.15.26:0/617824240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b90005190 0x7f1b9000b990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00e380 tx=0x7f1b9c01ce20 comp rx=0 tx=0).stop 2023-12-19T16:42:51.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.240+0000 7f1ba37fe640 1 -- 172.21.15.26:0/617824240 reap_dead start 2023-12-19T16:42:51.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.240+0000 7f1ba9873640 1 -- 172.21.15.26:0/617824240 shutdown_connections 2023-12-19T16:42:51.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.240+0000 7f1ba9873640 1 -- 172.21.15.26:0/617824240 >> 172.21.15.26:0/617824240 conn(0x7f1ba40658e0 msgr2=0x7f1ba4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:51.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.240+0000 7f1ba9873640 1 -- 172.21.15.26:0/617824240 shutdown_connections 2023-12-19T16:42:51.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.240+0000 7f1ba9873640 1 -- 172.21.15.26:0/617824240 wait complete. 2023-12-19T16:42:51.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.240+0000 7f1ba9873640 1 Processor -- start 2023-12-19T16:42:51.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.240+0000 7f1ba9873640 1 -- start start 2023-12-19T16:42:51.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba9873640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b90005190 0x7f1ba41524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba9873640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba40c82d0 0x7f1ba40c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba9873640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba40c9fb0 0x7f1ba40ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba9873640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba4152a00 con 0x7f1ba40c9fb0 2023-12-19T16:42:51.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba9873640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba41f5ff0 con 0x7f1b90005190 2023-12-19T16:42:51.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba9873640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba41f62d0 con 0x7f1ba40c82d0 2023-12-19T16:42:51.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba40c9fb0 0x7f1ba40ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba40c9fb0 0x7f1ba40ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52810/0 (socket says 172.21.15.26:52810) 2023-12-19T16:42:51.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba37fe640 1 -- 172.21.15.26:0/2891859526 learned_addr learned my addr 172.21.15.26:0/2891859526 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:51.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1b9bfff640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba40c82d0 0x7f1ba40c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba2ffd640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b90005190 0x7f1ba41524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba37fe640 1 -- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba40c82d0 msgr2=0x7f1ba40c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba37fe640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba40c82d0 0x7f1ba40c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba37fe640 1 -- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b90005190 msgr2=0x7f1ba41524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba37fe640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b90005190 0x7f1ba41524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba37fe640 1 -- 172.21.15.26:0/2891859526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba41f65b0 con 0x7f1ba40c9fb0 2023-12-19T16:42:51.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba37fe640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba40c9fb0 0x7f1ba40ca430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c01cc50 tx=0x7f1b9c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:51.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba0ff9640 1 -- 172.21.15.26:0/2891859526 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0039a0 con 0x7f1ba40c9fb0 2023-12-19T16:42:51.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba0ff9640 1 -- 172.21.15.26:0/2891859526 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c003b00 con 0x7f1ba40c9fb0 2023-12-19T16:42:51.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba0ff9640 1 -- 172.21.15.26:0/2891859526 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c003e70 con 0x7f1ba40c9fb0 2023-12-19T16:42:51.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba9873640 1 -- 172.21.15.26:0/2891859526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba41f6890 con 0x7f1ba40c9fb0 2023-12-19T16:42:51.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba9873640 1 -- 172.21.15.26:0/2891859526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba41f5c30 con 0x7f1ba40c9fb0 2023-12-19T16:42:51.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba9873640 1 -- 172.21.15.26:0/2891859526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba4074270 con 0x7f1ba40c9fb0 2023-12-19T16:42:51.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.244+0000 7f1ba0ff9640 1 -- 172.21.15.26:0/2891859526 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1b9c004020 con 0x7f1ba40c9fb0 2023-12-19T16:42:51.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.248+0000 7f1ba0ff9640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b84065a90 0x7f1b84067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.248+0000 7f1ba0ff9640 1 -- 172.21.15.26:0/2891859526 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b9c11e130 con 0x7f1ba40c9fb0 2023-12-19T16:42:51.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.248+0000 7f1ba2ffd640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b84065a90 0x7f1b84067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.248+0000 7f1ba2ffd640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b84065a90 0x7f1b84067f50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40b0d10 tx=0x7f1b94008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:51.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.252+0000 7f1ba0ff9640 1 -- 172.21.15.26:0/2891859526 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1b9c0ebba0 con 0x7f1ba40c9fb0 2023-12-19T16:42:51.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.412+0000 7f1ba9873640 1 -- 172.21.15.26:0/2891859526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ba40ccc30 con 0x7f1ba40c9fb0 2023-12-19T16:42:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.412+0000 7f1ba0ff9640 1 -- 172.21.15.26:0/2891859526 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f1b9c009dc0 con 0x7f1ba40c9fb0 2023-12-19T16:42:51.418 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:51.418 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.416+0000 7f1ba9873640 1 -- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b84065a90 msgr2=0x7f1b84067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.416+0000 7f1ba9873640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b84065a90 0x7f1b84067f50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40b0d10 tx=0x7f1b94008040 comp rx=0 tx=0).stop 2023-12-19T16:42:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.416+0000 7f1ba9873640 1 -- 172.21.15.26:0/2891859526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba40c9fb0 msgr2=0x7f1ba40ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.416+0000 7f1ba9873640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba40c9fb0 0x7f1ba40ca430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c01cc50 tx=0x7f1b9c009750 comp rx=0 tx=0).stop 2023-12-19T16:42:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.416+0000 7f1ba9873640 1 -- 172.21.15.26:0/2891859526 shutdown_connections 2023-12-19T16:42:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.416+0000 7f1ba9873640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba40c9fb0 0x7f1ba40ca430 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.416+0000 7f1ba9873640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b84065a90 0x7f1b84067f50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.416+0000 7f1ba9873640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba40c82d0 0x7f1ba40c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.416+0000 7f1ba9873640 1 --2- 172.21.15.26:0/2891859526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b90005190 0x7f1ba41524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.416+0000 7f1ba9873640 1 -- 172.21.15.26:0/2891859526 >> 172.21.15.26:0/2891859526 conn(0x7f1ba40658e0 msgr2=0x7f1ba4070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:51.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.416+0000 7f1ba9873640 1 -- 172.21.15.26:0/2891859526 shutdown_connections 2023-12-19T16:42:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.416+0000 7f1ba9873640 1 -- 172.21.15.26:0/2891859526 wait complete. 2023-12-19T16:42:51.439 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:42:25.598550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:42:26.697362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:42:51.440 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:42:51.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.580+0000 7f8baf998640 1 Processor -- start 2023-12-19T16:42:51.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.580+0000 7f8baf998640 1 -- start start 2023-12-19T16:42:51.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8baf998640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ba8159820 0x7f8ba815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8baf998640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba806f6c0 0x7f8ba80c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8baf998640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba806fb60 con 0x7f8ba8159c00 2023-12-19T16:42:51.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8baf998640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba815e470 con 0x7f8ba8159820 2023-12-19T16:42:51.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8baf998640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba806fe40 con 0x7f8ba806f6c0 2023-12-19T16:42:51.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8bad734640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ba8159820 0x7f8ba815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8badf35640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f8ba8159c00 0x7f8ba8070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:38352/0 (socket says 172.21.15.26:38352) 2023-12-19T16:42:51.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8badf35640 1 -- 172.21.15.26:0/336360595 learned_addr learned my addr 172.21.15.26:0/336360595 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:51.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8bacf33640 1 --2- 172.21.15.26:0/336360595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba806f6c0 0x7f8ba80c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8b9ffff640 1 -- 172.21.15.26:0/336360595 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2551241275 0 0) 0x7f8ba806fb60 con 0x7f8ba8159c00 2023-12-19T16:42:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8bad734640 1 -- 172.21.15.26:0/336360595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba806f6c0 msgr2=0x7f8ba80c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8bad734640 1 --2- 172.21.15.26:0/336360595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba806f6c0 0x7f8ba80c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8bad734640 1 -- 172.21.15.26:0/336360595 >> v1:172.21.15.160:6789/0 conn(0x7f8ba8159c00 legacy=0x7f8ba8070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.584+0000 7f8bad734640 1 -- 172.21.15.26:0/336360595 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba8070120 con 0x7f8ba8159820 2023-12-19T16:42:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8bad734640 1 --2- 172.21.15.26:0/336360595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba8159820 0x7f8ba815df30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40097c0 tx=0x7f8ba401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed73f21e786d5e05 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8b9ffff640 1 -- 172.21.15.26:0/336360595 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba40ab650 con 0x7f8ba8159820 2023-12-19T16:42:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8b9ffff640 1 -- 172.21.15.26:0/336360595 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba40ab7b0 con 0x7f8ba8159820 2023-12-19T16:42:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8b9ffff640 1 -- 172.21.15.26:0/336360595 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba40abb40 con 0x7f8ba8159820 2023-12-19T16:42:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8baf998640 1 -- 172.21.15.26:0/336360595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba8159820 msgr2=0x7f8ba815df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8baf998640 1 --2- 172.21.15.26:0/336360595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba8159820 0x7f8ba815df30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40097c0 tx=0x7f8ba401c450 comp rx=0 tx=0).stop 2023-12-19T16:42:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8baf998640 1 -- 172.21.15.26:0/336360595 shutdown_connections 2023-12-19T16:42:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8baf998640 1 --2- 172.21.15.26:0/336360595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba806f6c0 0x7f8ba80c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8baf998640 1 --2- 172.21.15.26:0/336360595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba8159820 0x7f8ba815df30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8baf998640 1 -- 172.21.15.26:0/336360595 >> 172.21.15.26:0/336360595 conn(0x7f8ba80ce730 msgr2=0x7f8ba80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8baf998640 1 -- 172.21.15.26:0/336360595 shutdown_connections 2023-12-19T16:42:51.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8baf998640 1 -- 172.21.15.26:0/336360595 wait complete. 2023-12-19T16:42:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8baf998640 1 Processor -- start 2023-12-19T16:42:51.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.588+0000 7f8baf998640 1 -- start start 2023-12-19T16:42:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8baf998640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba806f6c0 0x7f8ba80c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8baf998640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba8159820 0x7f8ba814b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8baf998640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ba8159c00 0x7f8ba814ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8baf998640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8153ee0 con 0x7f8ba8159820 2023-12-19T16:42:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8baf998640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba80cb560 con 0x7f8ba8159c00 2023-12-19T16:42:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8baf998640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba8153540 con 0x7f8ba806f6c0 2023-12-19T16:42:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8bacf33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba8159820 0x7f8ba814b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8bacf33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba8159820 0x7f8ba814b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52822/0 (socket says 172.21.15.26:52822) 2023-12-19T16:42:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8bacf33640 1 -- 172.21.15.26:0/2582638698 learned_addr learned my addr 172.21.15.26:0/2582638698 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8bad734640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba806f6c0 0x7f8ba80c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8badf35640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ba8159c00 0x7f8ba814ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8bacf33640 1 -- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba806f6c0 msgr2=0x7f8ba80c2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8bacf33640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba806f6c0 0x7f8ba80c2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8bacf33640 1 -- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ba8159c00 msgr2=0x7f8ba814ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8bacf33640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ba8159c00 0x7f8ba814ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8bacf33640 1 -- 172.21.15.26:0/2582638698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba4009420 con 0x7f8ba8159820 2023-12-19T16:42:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8bacf33640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba8159820 0x7f8ba814b540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800e900 tx=0x7f8b9800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8b9dffb640 1 -- 172.21.15.26:0/2582638698 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b98009690 con 0x7f8ba8159820 2023-12-19T16:42:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8b9dffb640 1 -- 172.21.15.26:0/2582638698 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9800ca00 con 0x7f8ba8159820 2023-12-19T16:42:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8b9dffb640 1 -- 172.21.15.26:0/2582638698 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9800cd90 con 0x7f8ba8159820 2023-12-19T16:42:51.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8baf998640 1 -- 172.21.15.26:0/2582638698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba806b880 con 0x7f8ba8159820 2023-12-19T16:42:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8baf998640 1 -- 172.21.15.26:0/2582638698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba8153b00 con 0x7f8ba8159820 2023-12-19T16:42:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8baf998640 1 -- 172.21.15.26:0/2582638698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba8154020 con 0x7f8ba8159820 2023-12-19T16:42:51.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.592+0000 7f8b9dffb640 1 -- 172.21.15.26:0/2582638698 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8b98013470 con 0x7f8ba8159820 2023-12-19T16:42:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.596+0000 7f8b9dffb640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8b8c065a60 0x7f8b8c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.596+0000 7f8b9dffb640 1 -- 172.21.15.26:0/2582638698 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b98083320 con 0x7f8ba8159820 2023-12-19T16:42:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.596+0000 7f8bad734640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8b8c065a60 0x7f8b8c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.596+0000 7f8bad734640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8b8c065a60 0x7f8b8c067f20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40af000 tx=0x7f8ba40a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:51.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.600+0000 7f8b9dffb640 1 -- 172.21.15.26:0/2582638698 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8ba8154020 con 0x7f8ba8159820 2023-12-19T16:42:51.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.756+0000 7f8baf998640 1 -- 172.21.15.26:0/2582638698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ba8154020 con 0x7f8ba8159820 2023-12-19T16:42:51.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.756+0000 7f8b9dffb640 1 -- 172.21.15.26:0/2582638698 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ba8154020 con 0x7f8ba8159820 2023-12-19T16:42:51.761 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:42:51.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.760+0000 7f8baf998640 1 -- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8b8c065a60 msgr2=0x7f8b8c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.760+0000 7f8baf998640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8b8c065a60 0x7f8b8c067f20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40af000 tx=0x7f8ba40a7000 comp rx=0 tx=0).stop 2023-12-19T16:42:51.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.760+0000 7f8baf998640 1 -- 172.21.15.26:0/2582638698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba8159820 msgr2=0x7f8ba814b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.760+0000 7f8baf998640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba8159820 0x7f8ba814b540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800e900 tx=0x7f8b9800edc0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.760+0000 7f8baf998640 1 -- 172.21.15.26:0/2582638698 shutdown_connections 2023-12-19T16:42:51.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.760+0000 7f8baf998640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ba8159c00 0x7f8ba814ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.760+0000 7f8baf998640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba8159820 0x7f8ba814b540 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.760+0000 7f8baf998640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8b8c065a60 0x7f8b8c067f20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.760+0000 7f8baf998640 1 --2- 172.21.15.26:0/2582638698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba806f6c0 0x7f8ba80c2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.760+0000 7f8baf998640 1 -- 172.21.15.26:0/2582638698 >> 172.21.15.26:0/2582638698 conn(0x7f8ba80ce730 msgr2=0x7f8ba80af120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:51.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.760+0000 7f8baf998640 1 -- 172.21.15.26:0/2582638698 shutdown_connections 2023-12-19T16:42:51.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.760+0000 7f8baf998640 1 -- 172.21.15.26:0/2582638698 wait complete. 2023-12-19T16:42:51.782 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:42:51.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.924+0000 7f13fbd98640 1 Processor -- start 2023-12-19T16:42:51.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13fbd98640 1 -- start start 2023-12-19T16:42:51.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13fbd98640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f13f4159820 0x7f13f415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13fbd98640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f13f4159c00 0x7f13f4070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13fbd98640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f406fb60 con 0x7f13f406f6c0 2023-12-19T16:42:51.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13fbd98640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f415e470 con 0x7f13f4159820 2023-12-19T16:42:51.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13fbd98640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f406fe40 con 0x7f13f4159c00 2023-12-19T16:42:51.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13f9b34640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f13f4159820 0x7f13f415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13f9b34640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f13f4159820 0x7f13f415df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52826/0 (socket says 172.21.15.26:52826) 2023-12-19T16:42:51.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13f9b34640 1 -- 172.21.15.26:0/3689614000 learned_addr learned my addr 172.21.15.26:0/3689614000 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:51.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13fa335640 1 --2- 172.21.15.26:0/3689614000 >> v2:172.21.15.160:3300/0 conn(0x7f13f4159c00 0x7f13f4070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13f9b34640 1 -- 172.21.15.26:0/3689614000 >> v2:172.21.15.160:3300/0 conn(0x7f13f4159c00 msgr2=0x7f13f4070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:51.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13f9b34640 1 --2- 172.21.15.26:0/3689614000 >> v2:172.21.15.160:3300/0 conn(0x7f13f4159c00 0x7f13f4070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13f9b34640 1 -- 172.21.15.26:0/3689614000 >> v1:172.21.15.160:6789/0 conn(0x7f13f406f6c0 legacy=0x7f13f40c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.928+0000 7f13f9b34640 1 -- 172.21.15.26:0/3689614000 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f4070120 con 0x7f13f4159820 2023-12-19T16:42:51.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13f9b34640 1 --2- 172.21.15.26:0/3689614000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4159820 0x7f13f415df30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13f406f200 tx=0x7f13f001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3295d16549762f07 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13f8b32640 1 -- 172.21.15.26:0/3689614000 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f00ac640 con 0x7f13f4159820 2023-12-19T16:42:51.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13f8b32640 1 -- 172.21.15.26:0/3689614000 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f00ac7a0 con 0x7f13f4159820 2023-12-19T16:42:51.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13f8b32640 1 -- 172.21.15.26:0/3689614000 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f00acb30 con 0x7f13f4159820 2023-12-19T16:42:51.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13fbd98640 1 -- 172.21.15.26:0/3689614000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4159820 msgr2=0x7f13f415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13fbd98640 1 --2- 172.21.15.26:0/3689614000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4159820 0x7f13f415df30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13f406f200 tx=0x7f13f001c4d0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13fbd98640 1 -- 172.21.15.26:0/3689614000 shutdown_connections 2023-12-19T16:42:51.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13fbd98640 1 --2- 172.21.15.26:0/3689614000 >> v2:172.21.15.160:3300/0 conn(0x7f13f4159c00 0x7f13f4070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13fbd98640 1 --2- 172.21.15.26:0/3689614000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4159820 0x7f13f415df30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13fbd98640 1 -- 172.21.15.26:0/3689614000 >> 172.21.15.26:0/3689614000 conn(0x7f13f40ce730 msgr2=0x7f13f40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:51.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13fbd98640 1 -- 172.21.15.26:0/3689614000 shutdown_connections 2023-12-19T16:42:51.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13fbd98640 1 -- 172.21.15.26:0/3689614000 wait complete. 2023-12-19T16:42:51.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13fbd98640 1 Processor -- start 2023-12-19T16:42:51.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.932+0000 7f13fbd98640 1 -- start start 2023-12-19T16:42:51.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13fbd98640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13f406f6c0 0x7f13f40cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13fbd98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4159820 0x7f13f414de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13fbd98640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13f4159c00 0x7f13f40ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13fbd98640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40c4840 con 0x7f13f4159820 2023-12-19T16:42:51.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13fbd98640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40c2370 con 0x7f13f4159c00 2023-12-19T16:42:51.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13fbd98640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40c2650 con 0x7f13f406f6c0 2023-12-19T16:42:51.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13f9333640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4159820 0x7f13f414de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13f9333640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4159820 0x7f13f414de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52828/0 (socket says 172.21.15.26:52828) 2023-12-19T16:42:51.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13f9333640 1 -- 172.21.15.26:0/1528306978 learned_addr learned my addr 172.21.15.26:0/1528306978 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:51.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13fa335640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13f4159c00 0x7f13f40ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13f9b34640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13f406f6c0 0x7f13f40cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13f9333640 1 -- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13f406f6c0 msgr2=0x7f13f40cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13f9333640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13f406f6c0 0x7f13f40cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13f9333640 1 -- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13f4159c00 msgr2=0x7f13f40ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:51.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13f9333640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13f4159c00 0x7f13f40ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:51.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13f9333640 1 -- 172.21.15.26:0/1528306978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f00094d0 con 0x7f13f4159820 2023-12-19T16:42:51.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13f9333640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4159820 0x7f13f414de60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f13e400c4d0 tx=0x7f13e400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:51.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13ea7fc640 1 -- 172.21.15.26:0/1528306978 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e40052d0 con 0x7f13f4159820 2023-12-19T16:42:51.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13ea7fc640 1 -- 172.21.15.26:0/1528306978 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e4005430 con 0x7f13f4159820 2023-12-19T16:42:51.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13ea7fc640 1 -- 172.21.15.26:0/1528306978 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4012830 con 0x7f13f4159820 2023-12-19T16:42:51.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13fbd98640 1 -- 172.21.15.26:0/1528306978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f406b880 con 0x7f13f4159820 2023-12-19T16:42:51.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13fbd98640 1 -- 172.21.15.26:0/1528306978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f40c4d70 con 0x7f13f4159820 2023-12-19T16:42:51.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13fbd98640 1 -- 172.21.15.26:0/1528306978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f4074270 con 0x7f13f4159820 2023-12-19T16:42:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.936+0000 7f13ea7fc640 1 -- 172.21.15.26:0/1528306978 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f13e4012990 con 0x7f13f4159820 2023-12-19T16:42:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.940+0000 7f13ea7fc640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f13d8065a60 0x7f13d8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:51.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.940+0000 7f13ea7fc640 1 -- 172.21.15.26:0/1528306978 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13e4082b50 con 0x7f13f4159820 2023-12-19T16:42:51.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.940+0000 7f13f9b34640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f13d8065a60 0x7f13d8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:51.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.940+0000 7f13f9b34640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f13d8065a60 0x7f13d8067f20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f13f00ae000 tx=0x7f13f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:51.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:51.944+0000 7f13ea7fc640 1 -- 172.21.15.26:0/1528306978 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f13e40505b0 con 0x7f13f4159820 2023-12-19T16:42:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.100+0000 7f13fbd98640 1 -- 172.21.15.26:0/1528306978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13f40b46b0 con 0x7f13f4159820 2023-12-19T16:42:52.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.100+0000 7f13ea7fc640 1 -- 172.21.15.26:0/1528306978 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f13e4053bd0 con 0x7f13f4159820 2023-12-19T16:42:52.106 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:42:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.104+0000 7f13fbd98640 1 -- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f13d8065a60 msgr2=0x7f13d8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.104+0000 7f13fbd98640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f13d8065a60 0x7f13d8067f20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f13f00ae000 tx=0x7f13f0008040 comp rx=0 tx=0).stop 2023-12-19T16:42:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.104+0000 7f13fbd98640 1 -- 172.21.15.26:0/1528306978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4159820 msgr2=0x7f13f414de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.104+0000 7f13fbd98640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4159820 0x7f13f414de60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f13e400c4d0 tx=0x7f13e400c990 comp rx=0 tx=0).stop 2023-12-19T16:42:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.104+0000 7f13fbd98640 1 -- 172.21.15.26:0/1528306978 shutdown_connections 2023-12-19T16:42:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.104+0000 7f13fbd98640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13f4159c00 0x7f13f40ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.104+0000 7f13fbd98640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13f4159820 0x7f13f414de60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.104+0000 7f13fbd98640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f13d8065a60 0x7f13d8067f20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.104+0000 7f13fbd98640 1 --2- 172.21.15.26:0/1528306978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f13f406f6c0 0x7f13f40cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.104+0000 7f13fbd98640 1 -- 172.21.15.26:0/1528306978 >> 172.21.15.26:0/1528306978 conn(0x7f13f40ce730 msgr2=0x7f13f40aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.104+0000 7f13fbd98640 1 -- 172.21.15.26:0/1528306978 shutdown_connections 2023-12-19T16:42:52.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.104+0000 7f13fbd98640 1 -- 172.21.15.26:0/1528306978 wait complete. 2023-12-19T16:42:52.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:42:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff52708c640 1 Processor -- start 2023-12-19T16:42:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff52708c640 1 -- start start 2023-12-19T16:42:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff52708c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff520159820 0x7ff52015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff52708c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff52006f6c0 0x7ff5200c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff52708c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff520159c00 0x7ff520070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff52708c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52015e470 con 0x7ff52006f6c0 2023-12-19T16:42:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff52708c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52006fb60 con 0x7ff520159820 2023-12-19T16:42:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff52708c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52006fe40 con 0x7ff520159c00 2023-12-19T16:42:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff51ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff52006f6c0 0x7ff5200c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff51ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff52006f6c0 0x7ff5200c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52840/0 (socket says 172.21.15.26:52840) 2023-12-19T16:42:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff51ffff640 1 -- 172.21.15.26:0/1270824012 learned_addr learned my addr 172.21.15.26:0/1270824012 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff525629640 1 --2- 172.21.15.26:0/1270824012 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff520159c00 0x7ff520070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff524e28640 1 --2- 172.21.15.26:0/1270824012 >> v2:172.21.15.160:3300/0 conn(0x7ff520159820 0x7ff52015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff51ffff640 1 -- 172.21.15.26:0/1270824012 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff520159c00 msgr2=0x7ff520070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff51ffff640 1 --2- 172.21.15.26:0/1270824012 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff520159c00 0x7ff520070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff51ffff640 1 -- 172.21.15.26:0/1270824012 >> v2:172.21.15.160:3300/0 conn(0x7ff520159820 msgr2=0x7ff52015df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff51ffff640 1 --2- 172.21.15.26:0/1270824012 >> v2:172.21.15.160:3300/0 conn(0x7ff520159820 0x7ff52015df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.268+0000 7ff51ffff640 1 -- 172.21.15.26:0/1270824012 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff520070120 con 0x7ff52006f6c0 2023-12-19T16:42:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.272+0000 7ff51ffff640 1 --2- 172.21.15.26:0/1270824012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52006f6c0 0x7ff5200c8e80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff5200c8e10 tx=0x7ff50c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d664906feefe7cc7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.272+0000 7ff51f7fe640 1 -- 172.21.15.26:0/1270824012 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0ad680 con 0x7ff52006f6c0 2023-12-19T16:42:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.272+0000 7ff51f7fe640 1 -- 172.21.15.26:0/1270824012 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c0ad7e0 con 0x7ff52006f6c0 2023-12-19T16:42:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.272+0000 7ff51f7fe640 1 -- 172.21.15.26:0/1270824012 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0adb70 con 0x7ff52006f6c0 2023-12-19T16:42:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.272+0000 7ff52708c640 1 -- 172.21.15.26:0/1270824012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52006f6c0 msgr2=0x7ff5200c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.272+0000 7ff52708c640 1 --2- 172.21.15.26:0/1270824012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52006f6c0 0x7ff5200c8e80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff5200c8e10 tx=0x7ff50c01c4d0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.272+0000 7ff52708c640 1 -- 172.21.15.26:0/1270824012 shutdown_connections 2023-12-19T16:42:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.272+0000 7ff52708c640 1 --2- 172.21.15.26:0/1270824012 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff520159c00 0x7ff520070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.272+0000 7ff52708c640 1 --2- 172.21.15.26:0/1270824012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff52006f6c0 0x7ff5200c8e80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.272+0000 7ff52708c640 1 --2- 172.21.15.26:0/1270824012 >> v2:172.21.15.160:3300/0 conn(0x7ff520159820 0x7ff52015df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.272+0000 7ff52708c640 1 -- 172.21.15.26:0/1270824012 >> 172.21.15.26:0/1270824012 conn(0x7ff5200ce730 msgr2=0x7ff5200ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.272+0000 7ff52708c640 1 -- 172.21.15.26:0/1270824012 shutdown_connections 2023-12-19T16:42:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.272+0000 7ff52708c640 1 -- 172.21.15.26:0/1270824012 wait complete. 2023-12-19T16:42:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff52708c640 1 Processor -- start 2023-12-19T16:42:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff52708c640 1 -- start start 2023-12-19T16:42:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff52708c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff52006f6c0 0x7ff52015c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff52708c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff520159820 0x7ff5200acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff52708c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff520159c00 0x7ff5201657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff52708c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200ad250 con 0x7ff520159820 2023-12-19T16:42:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff52708c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff520164a10 con 0x7ff520159c00 2023-12-19T16:42:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff52708c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff520164cf0 con 0x7ff52006f6c0 2023-12-19T16:42:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff51ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff520159820 0x7ff5200acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff51ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff520159820 0x7ff5200acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52844/0 (socket says 172.21.15.26:52844) 2023-12-19T16:42:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff51ffff640 1 -- 172.21.15.26:0/96110454 learned_addr learned my addr 172.21.15.26:0/96110454 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff525629640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff520159c00 0x7ff5201657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff524e28640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff52006f6c0 0x7ff52015c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff51ffff640 1 -- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff52006f6c0 msgr2=0x7ff52015c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff51ffff640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff52006f6c0 0x7ff52015c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff51ffff640 1 -- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff520159c00 msgr2=0x7ff5201657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff51ffff640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff520159c00 0x7ff5201657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff51ffff640 1 -- 172.21.15.26:0/96110454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0094d0 con 0x7ff520159820 2023-12-19T16:42:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff51ffff640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff520159820 0x7ff5200acd10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff50c01ab80 tx=0x7ff50c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:52.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff51d7fa640 1 -- 172.21.15.26:0/96110454 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0bd070 con 0x7ff520159820 2023-12-19T16:42:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.276+0000 7ff51d7fa640 1 -- 172.21.15.26:0/96110454 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c0adbe0 con 0x7ff520159820 2023-12-19T16:42:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.280+0000 7ff52708c640 1 -- 172.21.15.26:0/96110454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52006b880 con 0x7ff520159820 2023-12-19T16:42:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.280+0000 7ff51d7fa640 1 -- 172.21.15.26:0/96110454 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0b8450 con 0x7ff520159820 2023-12-19T16:42:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.280+0000 7ff52708c640 1 -- 172.21.15.26:0/96110454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5201652b0 con 0x7ff520159820 2023-12-19T16:42:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.280+0000 7ff52708c640 1 -- 172.21.15.26:0/96110454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff520070f10 con 0x7ff520159820 2023-12-19T16:42:52.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.280+0000 7ff51d7fa640 1 -- 172.21.15.26:0/96110454 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff50c0b4070 con 0x7ff520159820 2023-12-19T16:42:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.280+0000 7ff51d7fa640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff504065a60 0x7ff504067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.280+0000 7ff51d7fa640 1 -- 172.21.15.26:0/96110454 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff50c11cf60 con 0x7ff520159820 2023-12-19T16:42:52.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.280+0000 7ff524e28640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff504065a60 0x7ff504067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.284+0000 7ff524e28640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff504065a60 0x7ff504067f20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff514009b60 tx=0x7ff514009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.284+0000 7ff51d7fa640 1 -- 172.21.15.26:0/96110454 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff50c0ea9c0 con 0x7ff520159820 2023-12-19T16:42:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff52708c640 1 -- 172.21.15.26:0/96110454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff520074270 con 0x7ff520159820 2023-12-19T16:42:52.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff51d7fa640 1 -- 172.21.15.26:0/96110454 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff50c0edfe0 con 0x7ff520159820 2023-12-19T16:42:52.451 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:42:52.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff52708c640 1 -- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff504065a60 msgr2=0x7ff504067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff52708c640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff504065a60 0x7ff504067f20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff514009b60 tx=0x7ff514009190 comp rx=0 tx=0).stop 2023-12-19T16:42:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff52708c640 1 -- 172.21.15.26:0/96110454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff520159820 msgr2=0x7ff5200acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff52708c640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff520159820 0x7ff5200acd10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff50c01ab80 tx=0x7ff50c01ad10 comp rx=0 tx=0).stop 2023-12-19T16:42:52.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff52708c640 1 -- 172.21.15.26:0/96110454 shutdown_connections 2023-12-19T16:42:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff52708c640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff520159c00 0x7ff5201657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff52708c640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff520159820 0x7ff5200acd10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff52708c640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff504065a60 0x7ff504067f20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff52708c640 1 --2- 172.21.15.26:0/96110454 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff52006f6c0 0x7ff52015c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff52708c640 1 -- 172.21.15.26:0/96110454 >> 172.21.15.26:0/96110454 conn(0x7ff5200ce730 msgr2=0x7ff520158ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff52708c640 1 -- 172.21.15.26:0/96110454 shutdown_connections 2023-12-19T16:42:52.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.448+0000 7ff52708c640 1 -- 172.21.15.26:0/96110454 wait complete. 2023-12-19T16:42:52.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:52.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.612+0000 7f738cef2640 1 Processor -- start 2023-12-19T16:42:52.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.612+0000 7f738cef2640 1 -- start start 2023-12-19T16:42:52.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f738cef2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7388154220 0x7f738815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f738cef2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f738806b3d0 0x7f73880ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f738cef2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f738806bc30 con 0x7f73880cec40 2023-12-19T16:42:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f738cef2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73880cf0d0 con 0x7f738806b3d0 2023-12-19T16:42:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f738cef2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73880cc6e0 con 0x7f7388154220 2023-12-19T16:42:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f737ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f738806b3d0 0x7f73880ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f7386d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f73880cec40 0x7f738815d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37220/0 (socket says 172.21.15.26:37220) 2023-12-19T16:42:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f7386d9d640 1 -- 172.21.15.26:0/1393603864 learned_addr learned my addr 172.21.15.26:0/1393603864 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f738659c640 1 --2- 172.21.15.26:0/1393603864 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7388154220 0x7f738815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f737ffff640 1 -- 172.21.15.26:0/1393603864 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7388154220 msgr2=0x7f738815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f737ffff640 1 --2- 172.21.15.26:0/1393603864 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7388154220 0x7f738815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f737ffff640 1 -- 172.21.15.26:0/1393603864 >> v1:172.21.15.26:6789/0 conn(0x7f73880cec40 legacy=0x7f738815d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f737ffff640 1 -- 172.21.15.26:0/1393603864 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738815afc0 con 0x7f738806b3d0 2023-12-19T16:42:52.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.616+0000 7f7385d9b640 1 -- 172.21.15.26:0/1393603864 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672797404 0 0) 0x7f738806bc30 con 0x7f73880cec40 2023-12-19T16:42:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f737ffff640 1 --2- 172.21.15.26:0/1393603864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f738806b3d0 0x7f73880ce700 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f73700097c0 tx=0x7f737001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42885f1b04401e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f7385d9b640 1 -- 172.21.15.26:0/1393603864 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73700ab600 con 0x7f738806b3d0 2023-12-19T16:42:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f7385d9b640 1 -- 172.21.15.26:0/1393603864 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73700a7050 con 0x7f738806b3d0 2023-12-19T16:42:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f7385d9b640 1 -- 172.21.15.26:0/1393603864 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73700ab980 con 0x7f738806b3d0 2023-12-19T16:42:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f738cef2640 1 -- 172.21.15.26:0/1393603864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f738806b3d0 msgr2=0x7f73880ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f738cef2640 1 --2- 172.21.15.26:0/1393603864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f738806b3d0 0x7f73880ce700 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f73700097c0 tx=0x7f737001c450 comp rx=0 tx=0).stop 2023-12-19T16:42:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f738cef2640 1 -- 172.21.15.26:0/1393603864 shutdown_connections 2023-12-19T16:42:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f738cef2640 1 --2- 172.21.15.26:0/1393603864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f738806b3d0 0x7f73880ce700 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f738cef2640 1 --2- 172.21.15.26:0/1393603864 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7388154220 0x7f738815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f738cef2640 1 -- 172.21.15.26:0/1393603864 >> 172.21.15.26:0/1393603864 conn(0x7f73880658e0 msgr2=0x7f7388065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f738cef2640 1 -- 172.21.15.26:0/1393603864 shutdown_connections 2023-12-19T16:42:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f738cef2640 1 -- 172.21.15.26:0/1393603864 wait complete. 2023-12-19T16:42:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f738cef2640 1 Processor -- start 2023-12-19T16:42:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.620+0000 7f738cef2640 1 -- start start 2023-12-19T16:42:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f738cef2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f738806b3d0 0x7f73880acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f738cef2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73880cec40 0x7f73880b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f738cef2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7388154220 0x7f738813ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f738cef2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73881f9260 con 0x7f73880cec40 2023-12-19T16:42:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f738cef2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73880b2200 con 0x7f738806b3d0 2023-12-19T16:42:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f738cef2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73881f8950 con 0x7f7388154220 2023-12-19T16:42:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f737ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73880cec40 0x7f73880b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f737ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73880cec40 0x7f73880b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52866/0 (socket says 172.21.15.26:52866) 2023-12-19T16:42:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f737ffff640 1 -- 172.21.15.26:0/2745086024 learned_addr learned my addr 172.21.15.26:0/2745086024 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f7386d9d640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7388154220 0x7f738813ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f738659c640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f738806b3d0 0x7f73880acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f737ffff640 1 -- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7388154220 msgr2=0x7f738813ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f737ffff640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7388154220 0x7f738813ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f737ffff640 1 -- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f738806b3d0 msgr2=0x7f73880acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f737ffff640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f738806b3d0 0x7f73880acab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f737ffff640 1 -- 172.21.15.26:0/2745086024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7370009420 con 0x7f73880cec40 2023-12-19T16:42:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f737ffff640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73880cec40 0x7f73880b1460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f73700ae040 tx=0x7f73700191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f737effd640 1 -- 172.21.15.26:0/2745086024 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73700b4840 con 0x7f73880cec40 2023-12-19T16:42:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f737effd640 1 -- 172.21.15.26:0/2745086024 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73700b49c0 con 0x7f73880cec40 2023-12-19T16:42:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f738cef2640 1 -- 172.21.15.26:0/2745086024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73880b1f20 con 0x7f73880cec40 2023-12-19T16:42:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f737effd640 1 -- 172.21.15.26:0/2745086024 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73700b4d70 con 0x7f73880cec40 2023-12-19T16:42:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f738cef2640 1 -- 172.21.15.26:0/2745086024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73881f8e80 con 0x7f73880cec40 2023-12-19T16:42:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f738cef2640 1 -- 172.21.15.26:0/2745086024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73881f93a0 con 0x7f73880cec40 2023-12-19T16:42:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.624+0000 7f737effd640 1 -- 172.21.15.26:0/2745086024 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f73700b2040 con 0x7f73880cec40 2023-12-19T16:42:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.628+0000 7f737effd640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7368065a60 0x7f7368067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.628+0000 7f737effd640 1 -- 172.21.15.26:0/2745086024 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f737011d350 con 0x7f73880cec40 2023-12-19T16:42:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.628+0000 7f738659c640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7368065a60 0x7f7368067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.628+0000 7f738659c640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7368065a60 0x7f7368067f20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7378007a60 tx=0x7f7378008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.632+0000 7f737effd640 1 -- 172.21.15.26:0/2745086024 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f73881f93a0 con 0x7f73880cec40 2023-12-19T16:42:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.804+0000 7f738cef2640 1 -- 172.21.15.26:0/2745086024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73881f93a0 con 0x7f73880cec40 2023-12-19T16:42:52.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.804+0000 7f737effd640 1 -- 172.21.15.26:0/2745086024 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f73881f93a0 con 0x7f73880cec40 2023-12-19T16:42:52.809 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:52.809 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.804+0000 7f738cef2640 1 -- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7368065a60 msgr2=0x7f7368067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.804+0000 7f738cef2640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7368065a60 0x7f7368067f20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7378007a60 tx=0x7f7378008040 comp rx=0 tx=0).stop 2023-12-19T16:42:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.804+0000 7f738cef2640 1 -- 172.21.15.26:0/2745086024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73880cec40 msgr2=0x7f73880b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.804+0000 7f738cef2640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73880cec40 0x7f73880b1460 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f73700ae040 tx=0x7f73700191f0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.804+0000 7f738cef2640 1 -- 172.21.15.26:0/2745086024 shutdown_connections 2023-12-19T16:42:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.804+0000 7f738cef2640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7368065a60 0x7f7368067f20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.804+0000 7f738cef2640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7388154220 0x7f738813ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.804+0000 7f738cef2640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73880cec40 0x7f73880b1460 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.804+0000 7f738cef2640 1 --2- 172.21.15.26:0/2745086024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f738806b3d0 0x7f73880acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.808+0000 7f738cef2640 1 -- 172.21.15.26:0/2745086024 >> 172.21.15.26:0/2745086024 conn(0x7f73880658e0 msgr2=0x7f7388067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.808+0000 7f738cef2640 1 -- 172.21.15.26:0/2745086024 shutdown_connections 2023-12-19T16:42:52.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.808+0000 7f738cef2640 1 -- 172.21.15.26:0/2745086024 wait complete. 2023-12-19T16:42:52.830 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:42:25.598550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:42:26.697362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:42:52.831 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:42:52.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.972+0000 7f87ad5a2640 1 Processor -- start 2023-12-19T16:42:52.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.972+0000 7f87ad5a2640 1 -- start start 2023-12-19T16:42:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87ad5a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87a8161150 0x7f87a8167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87ad5a2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a8161550 con 0x7f87a80cfcb0 2023-12-19T16:42:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87ad5a2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a80cd490 con 0x7f87a815a770 2023-12-19T16:42:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87ad5a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80cd770 con 0x7f87a8161150 2023-12-19T16:42:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87a77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f87a80cfcb0 0x7f87a8160a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37234/0 (socket says 172.21.15.26:37234) 2023-12-19T16:42:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87a77fe640 1 -- 172.21.15.26:0/3174954195 learned_addr learned my addr 172.21.15.26:0/3174954195 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:52.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87acda1640 1 --2- 172.21.15.26:0/3174954195 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87a8161150 0x7f87a8167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87a6ffd640 1 -- 172.21.15.26:0/3174954195 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824985435 0 0) 0x7f879c001120 con 0x7f87a80cfcb0 2023-12-19T16:42:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87a6ffd640 1 -- 172.21.15.26:0/3174954195 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8790003530 con 0x7f87a80cfcb0 2023-12-19T16:42:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87a6ffd640 1 -- 172.21.15.26:0/3174954195 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2910960595 0 0) 0x7f87a80cd490 con 0x7f87a815a770 2023-12-19T16:42:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87a6ffd640 1 -- 172.21.15.26:0/3174954195 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f879c001120 con 0x7f87a815a770 2023-12-19T16:42:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87acda1640 1 -- 172.21.15.26:0/3174954195 >> v1:172.21.15.160:6789/0 conn(0x7f87a815a770 legacy=0x7f87a80cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87acda1640 1 -- 172.21.15.26:0/3174954195 >> v1:172.21.15.26:6789/0 conn(0x7f87a80cfcb0 legacy=0x7f87a8160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.976+0000 7f87acda1640 1 -- 172.21.15.26:0/3174954195 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80cda50 con 0x7f87a8161150 2023-12-19T16:42:52.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.980+0000 7f87acda1640 1 --2- 172.21.15.26:0/3174954195 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87a8161150 0x7f87a8167ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009dc0 tx=0x7f87a001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e60d533b51f809 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:52.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.980+0000 7f87a6ffd640 1 -- 172.21.15.26:0/3174954195 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a00a59a0 con 0x7f87a8161150 2023-12-19T16:42:52.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.980+0000 7f87a6ffd640 1 -- 172.21.15.26:0/3174954195 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a00a5b00 con 0x7f87a8161150 2023-12-19T16:42:52.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.980+0000 7f87a6ffd640 1 -- 172.21.15.26:0/3174954195 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a00a5e90 con 0x7f87a8161150 2023-12-19T16:42:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.980+0000 7f87ad5a2640 1 -- 172.21.15.26:0/3174954195 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87a8161150 msgr2=0x7f87a8167ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.980+0000 7f87ad5a2640 1 --2- 172.21.15.26:0/3174954195 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87a8161150 0x7f87a8167ac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009dc0 tx=0x7f87a001c940 comp rx=0 tx=0).stop 2023-12-19T16:42:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.980+0000 7f87ad5a2640 1 -- 172.21.15.26:0/3174954195 shutdown_connections 2023-12-19T16:42:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.980+0000 7f87ad5a2640 1 --2- 172.21.15.26:0/3174954195 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87a8161150 0x7f87a8167ac0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.980+0000 7f87ad5a2640 1 -- 172.21.15.26:0/3174954195 >> 172.21.15.26:0/3174954195 conn(0x7f87a806a150 msgr2=0x7f87a806a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.980+0000 7f87ad5a2640 1 -- 172.21.15.26:0/3174954195 shutdown_connections 2023-12-19T16:42:52.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.980+0000 7f87ad5a2640 1 -- 172.21.15.26:0/3174954195 wait complete. 2023-12-19T16:42:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.980+0000 7f87ad5a2640 1 Processor -- start 2023-12-19T16:42:52.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.980+0000 7f87ad5a2640 1 -- start start 2023-12-19T16:42:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87ad5a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87a80cfcb0 0x7f87a80c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87ad5a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87a815a770 0x7f87a815e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87ad5a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87a8161150 0x7f87a814bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87ad5a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a814fd70 con 0x7f87a80cfcb0 2023-12-19T16:42:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87ad5a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80c87a0 con 0x7f87a815a770 2023-12-19T16:42:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87ad5a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a8150010 con 0x7f87a8161150 2023-12-19T16:42:52.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87a80cfcb0 0x7f87a80c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87a80cfcb0 0x7f87a80c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52874/0 (socket says 172.21.15.26:52874) 2023-12-19T16:42:52.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a7fff640 1 -- 172.21.15.26:0/676013122 learned_addr learned my addr 172.21.15.26:0/676013122 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:52.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87acda1640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87a8161150 0x7f87a814bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a77fe640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87a815a770 0x7f87a815e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a7fff640 1 -- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87a8161150 msgr2=0x7f87a814bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a7fff640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87a8161150 0x7f87a814bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a7fff640 1 -- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87a815a770 msgr2=0x7f87a815e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a7fff640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87a815a770 0x7f87a815e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:52.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a7fff640 1 -- 172.21.15.26:0/676013122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a0009a70 con 0x7f87a80cfcb0 2023-12-19T16:42:52.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a7fff640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87a80cfcb0 0x7f87a80c8260 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f879800c490 tx=0x7f879800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:52.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a4ff9640 1 -- 172.21.15.26:0/676013122 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87980052d0 con 0x7f87a80cfcb0 2023-12-19T16:42:52.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a4ff9640 1 -- 172.21.15.26:0/676013122 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8798005430 con 0x7f87a80cfcb0 2023-12-19T16:42:52.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87ad5a2640 1 -- 172.21.15.26:0/676013122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a81502f0 con 0x7f87a80cfcb0 2023-12-19T16:42:52.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a4ff9640 1 -- 172.21.15.26:0/676013122 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8798012870 con 0x7f87a80cfcb0 2023-12-19T16:42:52.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87ad5a2640 1 -- 172.21.15.26:0/676013122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a81508b0 con 0x7f87a80cfcb0 2023-12-19T16:42:52.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87ad5a2640 1 -- 172.21.15.26:0/676013122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a8061930 con 0x7f87a80cfcb0 2023-12-19T16:42:52.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.984+0000 7f87a4ff9640 1 -- 172.21.15.26:0/676013122 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f87980129d0 con 0x7f87a80cfcb0 2023-12-19T16:42:52.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.988+0000 7f87a4ff9640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8794065a60 0x7f8794067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:52.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.988+0000 7f87a4ff9640 1 -- 172.21.15.26:0/676013122 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8798082c20 con 0x7f87a80cfcb0 2023-12-19T16:42:52.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.988+0000 7f87a77fe640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8794065a60 0x7f8794067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:52.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.988+0000 7f87a77fe640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8794065a60 0x7f8794067f20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87a814d110 tx=0x7f879c007d40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:52.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:52.992+0000 7f87a4ff9640 1 -- 172.21.15.26:0/676013122 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8798050680 con 0x7f87a80cfcb0 2023-12-19T16:42:53.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.144+0000 7f87ad5a2640 1 -- 172.21.15.26:0/676013122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87a80cf920 con 0x7f87a80cfcb0 2023-12-19T16:42:53.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.148+0000 7f87a4ff9640 1 -- 172.21.15.26:0/676013122 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8798053ca0 con 0x7f87a80cfcb0 2023-12-19T16:42:53.151 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:42:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.148+0000 7f87ad5a2640 1 -- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8794065a60 msgr2=0x7f8794067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.148+0000 7f87ad5a2640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8794065a60 0x7f8794067f20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f87a814d110 tx=0x7f879c007d40 comp rx=0 tx=0).stop 2023-12-19T16:42:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.148+0000 7f87ad5a2640 1 -- 172.21.15.26:0/676013122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87a80cfcb0 msgr2=0x7f87a80c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.148+0000 7f87ad5a2640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87a80cfcb0 0x7f87a80c8260 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f879800c490 tx=0x7f879800c950 comp rx=0 tx=0).stop 2023-12-19T16:42:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.148+0000 7f87ad5a2640 1 -- 172.21.15.26:0/676013122 shutdown_connections 2023-12-19T16:42:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.148+0000 7f87ad5a2640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8794065a60 0x7f8794067f20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.148+0000 7f87ad5a2640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87a8161150 0x7f87a814bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.148+0000 7f87ad5a2640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87a815a770 0x7f87a815e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.148+0000 7f87ad5a2640 1 --2- 172.21.15.26:0/676013122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87a80cfcb0 0x7f87a80c8260 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.148+0000 7f87ad5a2640 1 -- 172.21.15.26:0/676013122 >> 172.21.15.26:0/676013122 conn(0x7f87a806a150 msgr2=0x7f87a80d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.148+0000 7f87ad5a2640 1 -- 172.21.15.26:0/676013122 shutdown_connections 2023-12-19T16:42:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.148+0000 7f87ad5a2640 1 -- 172.21.15.26:0/676013122 wait complete. 2023-12-19T16:42:53.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:42:53.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.312+0000 7fd814109640 1 Processor -- start 2023-12-19T16:42:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.312+0000 7fd814109640 1 -- start start 2023-12-19T16:42:53.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.316+0000 7fd814109640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd80c06f6c0 0x7fd80c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:53.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.316+0000 7fd814109640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd80c06fb60 con 0x7fd80c159c00 2023-12-19T16:42:53.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.316+0000 7fd814109640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd80c06fe40 con 0x7fd80c159820 2023-12-19T16:42:53.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.316+0000 7fd814109640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd80c15e470 con 0x7fd80c06f6c0 2023-12-19T16:42:53.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.316+0000 7fd8116a4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd80c06f6c0 0x7fd80c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:53.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.316+0000 7fd8126a6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd80c159c00 0x7fd80c070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37238/0 (socket says 172.21.15.26:37238) 2023-12-19T16:42:53.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.316+0000 7fd8126a6640 1 -- 172.21.15.26:0/1026678285 learned_addr learned my addr 172.21.15.26:0/1026678285 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:53.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.316+0000 7fd8116a4640 1 -- 172.21.15.26:0/1026678285 >> v1:172.21.15.160:6789/0 conn(0x7fd80c159820 legacy=0x7fd80c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.316+0000 7fd8116a4640 1 -- 172.21.15.26:0/1026678285 >> v1:172.21.15.26:6789/0 conn(0x7fd80c159c00 legacy=0x7fd80c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.316+0000 7fd8116a4640 1 -- 172.21.15.26:0/1026678285 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c070120 con 0x7fd80c06f6c0 2023-12-19T16:42:53.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.316+0000 7fd810ea3640 1 -- 172.21.15.26:0/1026678285 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876112451 0 0) 0x7fd80c06fb60 con 0x7fd80c159c00 2023-12-19T16:42:53.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.316+0000 7fd8116a4640 1 --2- 172.21.15.26:0/1026678285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c06f6c0 0x7fd80c15df30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0097c0 tx=0x7fd7fc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e411bf9d8232b81b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd810ea3640 1 -- 172.21.15.26:0/1026678285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0ab650 con 0x7fd80c06f6c0 2023-12-19T16:42:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd810ea3640 1 -- 172.21.15.26:0/1026678285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7fc0ab7b0 con 0x7fd80c06f6c0 2023-12-19T16:42:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd810ea3640 1 -- 172.21.15.26:0/1026678285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc0abb40 con 0x7fd80c06f6c0 2023-12-19T16:42:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 -- 172.21.15.26:0/1026678285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c06f6c0 msgr2=0x7fd80c15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 --2- 172.21.15.26:0/1026678285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c06f6c0 0x7fd80c15df30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0097c0 tx=0x7fd7fc01c450 comp rx=0 tx=0).stop 2023-12-19T16:42:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 -- 172.21.15.26:0/1026678285 shutdown_connections 2023-12-19T16:42:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 --2- 172.21.15.26:0/1026678285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c06f6c0 0x7fd80c15df30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 -- 172.21.15.26:0/1026678285 >> 172.21.15.26:0/1026678285 conn(0x7fd80c0ce730 msgr2=0x7fd80c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 -- 172.21.15.26:0/1026678285 shutdown_connections 2023-12-19T16:42:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 -- 172.21.15.26:0/1026678285 wait complete. 2023-12-19T16:42:53.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 Processor -- start 2023-12-19T16:42:53.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 -- start start 2023-12-19T16:42:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c06f6c0 0x7fd80c15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd80c159820 0x7fd80c153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd80c159c00 0x7fd80c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c15a810 con 0x7fd80c06f6c0 2023-12-19T16:42:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c0af310 con 0x7fd80c159820 2023-12-19T16:42:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd814109640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c0af560 con 0x7fd80c159c00 2023-12-19T16:42:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd811ea5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c06f6c0 0x7fd80c15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd811ea5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c06f6c0 0x7fd80c15c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52894/0 (socket says 172.21.15.26:52894) 2023-12-19T16:42:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.320+0000 7fd811ea5640 1 -- 172.21.15.26:0/3757225890 learned_addr learned my addr 172.21.15.26:0/3757225890 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd8126a6640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd80c159c00 0x7fd80c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd811ea5640 1 -- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd80c159c00 msgr2=0x7fd80c150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd8116a4640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd80c159820 0x7fd80c153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd811ea5640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd80c159c00 0x7fd80c150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd811ea5640 1 -- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd80c159820 msgr2=0x7fd80c153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd811ea5640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd80c159820 0x7fd80c153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd811ea5640 1 -- 172.21.15.26:0/3757225890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc009420 con 0x7fd80c06f6c0 2023-12-19T16:42:53.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd8116a4640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd80c159820 0x7fd80c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:53.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd8126a6640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd80c159c00 0x7fd80c150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:53.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd811ea5640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c06f6c0 0x7fd80c15c660 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd80800c500 tx=0x7fd80800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:53.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd8027fc640 1 -- 172.21.15.26:0/3757225890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808005220 con 0x7fd80c06f6c0 2023-12-19T16:42:53.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd8027fc640 1 -- 172.21.15.26:0/3757225890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd808005380 con 0x7fd80c06f6c0 2023-12-19T16:42:53.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd8027fc640 1 -- 172.21.15.26:0/3757225890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808007270 con 0x7fd80c06f6c0 2023-12-19T16:42:53.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd814109640 1 -- 172.21.15.26:0/3757225890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80c06b880 con 0x7fd80c06f6c0 2023-12-19T16:42:53.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd814109640 1 -- 172.21.15.26:0/3757225890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80c15ab50 con 0x7fd80c06f6c0 2023-12-19T16:42:53.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd814109640 1 -- 172.21.15.26:0/3757225890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80c0afa80 con 0x7fd80c06f6c0 2023-12-19T16:42:53.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd8027fc640 1 -- 172.21.15.26:0/3757225890 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd8080073d0 con 0x7fd80c06f6c0 2023-12-19T16:42:53.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd8027fc640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd7f0065a60 0x7fd7f0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:53.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.324+0000 7fd8027fc640 1 -- 172.21.15.26:0/3757225890 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd808082590 con 0x7fd80c06f6c0 2023-12-19T16:42:53.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.328+0000 7fd8116a4640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd7f0065a60 0x7fd7f0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:53.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.328+0000 7fd8116a4640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd7f0065a60 0x7fd7f0067f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0af000 tx=0x7fd7fc0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:53.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.328+0000 7fd8027fc640 1 -- 172.21.15.26:0/3757225890 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd80804fec0 con 0x7fd80c06f6c0 2023-12-19T16:42:53.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.488+0000 7fd814109640 1 -- 172.21.15.26:0/3757225890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd80c074270 con 0x7fd80c06f6c0 2023-12-19T16:42:53.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.488+0000 7fd8027fc640 1 -- 172.21.15.26:0/3757225890 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd8080534e0 con 0x7fd80c06f6c0 2023-12-19T16:42:53.493 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:42:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.492+0000 7fd814109640 1 -- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd7f0065a60 msgr2=0x7fd7f0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.492+0000 7fd814109640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd7f0065a60 0x7fd7f0067f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc0af000 tx=0x7fd7fc0a7000 comp rx=0 tx=0).stop 2023-12-19T16:42:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.492+0000 7fd814109640 1 -- 172.21.15.26:0/3757225890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c06f6c0 msgr2=0x7fd80c15c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.492+0000 7fd814109640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c06f6c0 0x7fd80c15c660 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd80800c500 tx=0x7fd80800c9c0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.492+0000 7fd814109640 1 -- 172.21.15.26:0/3757225890 shutdown_connections 2023-12-19T16:42:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.492+0000 7fd814109640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd7f0065a60 0x7fd7f0067f20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.492+0000 7fd814109640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd80c159c00 0x7fd80c150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.492+0000 7fd814109640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd80c159820 0x7fd80c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.492+0000 7fd814109640 1 --2- 172.21.15.26:0/3757225890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd80c06f6c0 0x7fd80c15c660 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.492+0000 7fd814109640 1 -- 172.21.15.26:0/3757225890 >> 172.21.15.26:0/3757225890 conn(0x7fd80c0ce730 msgr2=0x7fd80c06ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.492+0000 7fd814109640 1 -- 172.21.15.26:0/3757225890 shutdown_connections 2023-12-19T16:42:53.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.492+0000 7fd814109640 1 -- 172.21.15.26:0/3757225890 wait complete. 2023-12-19T16:42:53.514 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:42:53.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.656+0000 7f4feb92c640 1 Processor -- start 2023-12-19T16:42:53.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.656+0000 7f4feb92c640 1 -- start start 2023-12-19T16:42:53.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.656+0000 7f4feb92c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4fe4161230 0x7f4fe40ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:53.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.656+0000 7f4feb92c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fe40c8df0 0x7f4fe40c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:53.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.656+0000 7f4feb92c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe40b01b0 con 0x7f4fe406c3f0 2023-12-19T16:42:53.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.656+0000 7f4feb92c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe40adfb0 con 0x7f4fe4161230 2023-12-19T16:42:53.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.656+0000 7f4feb92c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40ca4e0 con 0x7f4fe40c8df0 2023-12-19T16:42:53.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.656+0000 7f4fe96c8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4fe4161230 0x7f4fe40ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:53.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.656+0000 7f4fe8ec7640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4fe406c3f0 0x7f4fe406c850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37246/0 (socket says 172.21.15.26:37246) 2023-12-19T16:42:53.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.656+0000 7f4fe8ec7640 1 -- 172.21.15.26:0/2288842739 learned_addr learned my addr 172.21.15.26:0/2288842739 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:53.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.660+0000 7f4fe9ec9640 1 --2- 172.21.15.26:0/2288842739 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fe40c8df0 0x7f4fe40c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:53.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.660+0000 7f4fe96c8640 1 -- 172.21.15.26:0/2288842739 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fe40c8df0 msgr2=0x7f4fe40c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:53.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.660+0000 7f4fe96c8640 1 --2- 172.21.15.26:0/2288842739 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fe40c8df0 0x7f4fe40c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.660+0000 7f4fe96c8640 1 -- 172.21.15.26:0/2288842739 >> v1:172.21.15.26:6789/0 conn(0x7f4fe406c3f0 legacy=0x7f4fe406c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.660+0000 7f4fe96c8640 1 -- 172.21.15.26:0/2288842739 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe40b0400 con 0x7f4fe4161230 2023-12-19T16:42:53.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.660+0000 7f4fe96c8640 1 --2- 172.21.15.26:0/2288842739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fe4161230 0x7f4fe40ada70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00097c0 tx=0x7f4fe001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72b3c9812851b9b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:53.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.660+0000 7f4fdbfff640 1 -- 172.21.15.26:0/2288842739 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00ac620 con 0x7f4fe4161230 2023-12-19T16:42:53.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.660+0000 7f4fdbfff640 1 -- 172.21.15.26:0/2288842739 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe00ac780 con 0x7f4fe4161230 2023-12-19T16:42:53.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.660+0000 7f4fdbfff640 1 -- 172.21.15.26:0/2288842739 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00acb10 con 0x7f4fe4161230 2023-12-19T16:42:53.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 -- 172.21.15.26:0/2288842739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fe4161230 msgr2=0x7f4fe40ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 --2- 172.21.15.26:0/2288842739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fe4161230 0x7f4fe40ada70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00097c0 tx=0x7f4fe001c3c0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 -- 172.21.15.26:0/2288842739 shutdown_connections 2023-12-19T16:42:53.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 --2- 172.21.15.26:0/2288842739 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fe40c8df0 0x7f4fe40c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 --2- 172.21.15.26:0/2288842739 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fe4161230 0x7f4fe40ada70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 -- 172.21.15.26:0/2288842739 >> 172.21.15.26:0/2288842739 conn(0x7f4fe40ce730 msgr2=0x7f4fe40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:53.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 -- 172.21.15.26:0/2288842739 shutdown_connections 2023-12-19T16:42:53.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 -- 172.21.15.26:0/2288842739 wait complete. 2023-12-19T16:42:53.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 Processor -- start 2023-12-19T16:42:53.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 -- start start 2023-12-19T16:42:53.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fe406c3f0 0x7f4fe40c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:53.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fe40c8df0 0x7f4fe4155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:53.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fe4161230 0x7f4fe40b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:53.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40c8130 con 0x7f4fe40c8df0 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40c7890 con 0x7f4fe406c3f0 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4feb92c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40c7b70 con 0x7f4fe4161230 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4fe8ec7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fe40c8df0 0x7f4fe4155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4fe8ec7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fe40c8df0 0x7f4fe4155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52908/0 (socket says 172.21.15.26:52908) 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4fe8ec7640 1 -- 172.21.15.26:0/1937579841 learned_addr learned my addr 172.21.15.26:0/1937579841 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4fe9ec9640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fe4161230 0x7f4fe40b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4fe96c8640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fe406c3f0 0x7f4fe40c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4fe8ec7640 1 -- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fe4161230 msgr2=0x7f4fe40b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4fe8ec7640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fe4161230 0x7f4fe40b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4fe8ec7640 1 -- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fe406c3f0 msgr2=0x7f4fe40c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4fe8ec7640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fe406c3f0 0x7f4fe40c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4fe8ec7640 1 -- 172.21.15.26:0/1937579841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe0009420 con 0x7f4fe40c8df0 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.664+0000 7f4fe8ec7640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fe40c8df0 0x7f4fe4155c30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c400 tx=0x7f4fd400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:53.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.668+0000 7f4fd9ffb640 1 -- 172.21.15.26:0/1937579841 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4005cf0 con 0x7f4fe40c8df0 2023-12-19T16:42:53.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.668+0000 7f4fd9ffb640 1 -- 172.21.15.26:0/1937579841 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd4005e50 con 0x7f4fe40c8df0 2023-12-19T16:42:53.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.668+0000 7f4fd9ffb640 1 -- 172.21.15.26:0/1937579841 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40072d0 con 0x7f4fe40c8df0 2023-12-19T16:42:53.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.668+0000 7f4feb92c640 1 -- 172.21.15.26:0/1937579841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe40c7e50 con 0x7f4fe40c8df0 2023-12-19T16:42:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.668+0000 7f4feb92c640 1 -- 172.21.15.26:0/1937579841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe4156170 con 0x7f4fe40c8df0 2023-12-19T16:42:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.668+0000 7f4feb92c640 1 -- 172.21.15.26:0/1937579841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe40c1540 con 0x7f4fe40c8df0 2023-12-19T16:42:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.668+0000 7f4fd9ffb640 1 -- 172.21.15.26:0/1937579841 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4fd4007430 con 0x7f4fe40c8df0 2023-12-19T16:42:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.668+0000 7f4fd9ffb640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4fc8065a60 0x7f4fc8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:53.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.668+0000 7f4fd9ffb640 1 -- 172.21.15.26:0/1937579841 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fd4082680 con 0x7f4fe40c8df0 2023-12-19T16:42:53.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.668+0000 7f4fe96c8640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4fc8065a60 0x7f4fc8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:53.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.672+0000 7f4fe96c8640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4fc8065a60 0x7f4fc8067f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00ae000 tx=0x7f4fe0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:53.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.672+0000 7f4fd9ffb640 1 -- 172.21.15.26:0/1937579841 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4fd40500e0 con 0x7f4fe40c8df0 2023-12-19T16:42:53.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.828+0000 7f4feb92c640 1 -- 172.21.15.26:0/1937579841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fe4074270 con 0x7f4fe40c8df0 2023-12-19T16:42:53.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.828+0000 7f4fd9ffb640 1 -- 172.21.15.26:0/1937579841 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fd4053700 con 0x7f4fe40c8df0 2023-12-19T16:42:53.834 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:42:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.832+0000 7f4feb92c640 1 -- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4fc8065a60 msgr2=0x7f4fc8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.832+0000 7f4feb92c640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4fc8065a60 0x7f4fc8067f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4fe00ae000 tx=0x7f4fe0008040 comp rx=0 tx=0).stop 2023-12-19T16:42:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.832+0000 7f4feb92c640 1 -- 172.21.15.26:0/1937579841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fe40c8df0 msgr2=0x7f4fe4155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.832+0000 7f4feb92c640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fe40c8df0 0x7f4fe4155c30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c400 tx=0x7f4fd400c8c0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.832+0000 7f4feb92c640 1 -- 172.21.15.26:0/1937579841 shutdown_connections 2023-12-19T16:42:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.832+0000 7f4feb92c640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4fc8065a60 0x7f4fc8067f20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.832+0000 7f4feb92c640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fe4161230 0x7f4fe40b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.832+0000 7f4feb92c640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fe40c8df0 0x7f4fe4155c30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.832+0000 7f4feb92c640 1 --2- 172.21.15.26:0/1937579841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fe406c3f0 0x7f4fe40c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.832+0000 7f4feb92c640 1 -- 172.21.15.26:0/1937579841 >> 172.21.15.26:0/1937579841 conn(0x7f4fe40ce730 msgr2=0x7f4fe40ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.832+0000 7f4feb92c640 1 -- 172.21.15.26:0/1937579841 shutdown_connections 2023-12-19T16:42:53.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.832+0000 7f4feb92c640 1 -- 172.21.15.26:0/1937579841 wait complete. 2023-12-19T16:42:53.856 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:53.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47ff02d640 1 Processor -- start 2023-12-19T16:42:53.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47ff02d640 1 -- start start 2023-12-19T16:42:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47ff02d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47f80cee50 0x7f47f815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47ff02d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f806c970 con 0x7f47f815f370 2023-12-19T16:42:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47ff02d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f815b1f0 con 0x7f47f8155030 2023-12-19T16:42:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47ff02d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f806c790 con 0x7f47f80cee50 2023-12-19T16:42:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47f7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f47f815f370 0x7f47f80ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37256/0 (socket says 172.21.15.26:37256) 2023-12-19T16:42:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47f7fff640 1 -- 172.21.15.26:0/506219990 learned_addr learned my addr 172.21.15.26:0/506219990 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47fd5ca640 1 --2- 172.21.15.26:0/506219990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47f80cee50 0x7f47f815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47f77fe640 1 -- 172.21.15.26:0/506219990 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241645848 0 0) 0x7f47f806c970 con 0x7f47f815f370 2023-12-19T16:42:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47f77fe640 1 -- 172.21.15.26:0/506219990 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e8003530 con 0x7f47f815f370 2023-12-19T16:42:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47f77fe640 1 -- 172.21.15.26:0/506219990 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 951936647 0 0) 0x7f47f815b1f0 con 0x7f47f8155030 2023-12-19T16:42:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47f77fe640 1 -- 172.21.15.26:0/506219990 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47f806c970 con 0x7f47f8155030 2023-12-19T16:42:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47f77fe640 1 -- 172.21.15.26:0/506219990 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2819242603 0 0) 0x7f47e8003530 con 0x7f47f815f370 2023-12-19T16:42:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47fd5ca640 1 -- 172.21.15.26:0/506219990 >> v1:172.21.15.160:6789/0 conn(0x7f47f8155030 legacy=0x7f47f806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47fd5ca640 1 -- 172.21.15.26:0/506219990 >> v1:172.21.15.26:6789/0 conn(0x7f47f815f370 legacy=0x7f47f80ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:53.996+0000 7f47fd5ca640 1 -- 172.21.15.26:0/506219990 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f815b4d0 con 0x7f47f80cee50 2023-12-19T16:42:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.000+0000 7f47fd5ca640 1 --2- 172.21.15.26:0/506219990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47f80cee50 0x7f47f815bcd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009dc0 tx=0x7f47f001c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=731e55a8cc9bfc2e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.000+0000 7f47f77fe640 1 -- 172.21.15.26:0/506219990 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4001680 con 0x7f47f80cee50 2023-12-19T16:42:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.000+0000 7f47f77fe640 1 -- 172.21.15.26:0/506219990 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f00a5680 con 0x7f47f80cee50 2023-12-19T16:42:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.000+0000 7f47f77fe640 1 -- 172.21.15.26:0/506219990 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f00a59f0 con 0x7f47f80cee50 2023-12-19T16:42:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.000+0000 7f47ff02d640 1 -- 172.21.15.26:0/506219990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47f80cee50 msgr2=0x7f47f815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.000+0000 7f47ff02d640 1 --2- 172.21.15.26:0/506219990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47f80cee50 0x7f47f815bcd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f47f0009dc0 tx=0x7f47f001c820 comp rx=0 tx=0).stop 2023-12-19T16:42:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.000+0000 7f47ff02d640 1 -- 172.21.15.26:0/506219990 shutdown_connections 2023-12-19T16:42:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.000+0000 7f47ff02d640 1 --2- 172.21.15.26:0/506219990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47f80cee50 0x7f47f815bcd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.000+0000 7f47ff02d640 1 -- 172.21.15.26:0/506219990 >> 172.21.15.26:0/506219990 conn(0x7f47f80658e0 msgr2=0x7f47f8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:54.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.000+0000 7f47ff02d640 1 -- 172.21.15.26:0/506219990 shutdown_connections 2023-12-19T16:42:54.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.000+0000 7f47ff02d640 1 -- 172.21.15.26:0/506219990 wait complete. 2023-12-19T16:42:54.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47ff02d640 1 Processor -- start 2023-12-19T16:42:54.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47ff02d640 1 -- start start 2023-12-19T16:42:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47ff02d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47f80cee50 0x7f47f81501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47ff02d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47f8155030 0x7f47f814e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47ff02d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47f815f370 0x7f47f814ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47ff02d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80ad250 con 0x7f47f80cee50 2023-12-19T16:42:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47ff02d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f81560b0 con 0x7f47f8155030 2023-12-19T16:42:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47ff02d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f8155940 con 0x7f47f815f370 2023-12-19T16:42:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47fcdc9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47f80cee50 0x7f47f81501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47fcdc9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47f80cee50 0x7f47f81501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52918/0 (socket says 172.21.15.26:52918) 2023-12-19T16:42:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47fcdc9640 1 -- 172.21.15.26:0/2435903278 learned_addr learned my addr 172.21.15.26:0/2435903278 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47fd5ca640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47f815f370 0x7f47f814ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47f7fff640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47f8155030 0x7f47f814e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47fcdc9640 1 -- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47f815f370 msgr2=0x7f47f814ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47fcdc9640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47f815f370 0x7f47f814ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47fcdc9640 1 -- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47f8155030 msgr2=0x7f47f814e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47fcdc9640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47f8155030 0x7f47f814e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47fcdc9640 1 -- 172.21.15.26:0/2435903278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f0009a70 con 0x7f47f80cee50 2023-12-19T16:42:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47f7fff640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47f8155030 0x7f47f814e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47fcdc9640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47f80cee50 0x7f47f81501f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00c4d0 tx=0x7f47ec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47f57fa640 1 -- 172.21.15.26:0/2435903278 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec005480 con 0x7f47f80cee50 2023-12-19T16:42:54.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47f57fa640 1 -- 172.21.15.26:0/2435903278 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec003600 con 0x7f47f80cee50 2023-12-19T16:42:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47f57fa640 1 -- 172.21.15.26:0/2435903278 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0039b0 con 0x7f47f80cee50 2023-12-19T16:42:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47ff02d640 1 -- 172.21.15.26:0/2435903278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f815aac0 con 0x7f47f80cee50 2023-12-19T16:42:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47ff02d640 1 -- 172.21.15.26:0/2435903278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f8155be0 con 0x7f47f80cee50 2023-12-19T16:42:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.004+0000 7f47ff02d640 1 -- 172.21.15.26:0/2435903278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f81557a0 con 0x7f47f80cee50 2023-12-19T16:42:54.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.008+0000 7f47f57fa640 1 -- 172.21.15.26:0/2435903278 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f47ec003b10 con 0x7f47f80cee50 2023-12-19T16:42:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.008+0000 7f47f57fa640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f47dc065a60 0x7f47dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.008+0000 7f47f57fa640 1 -- 172.21.15.26:0/2435903278 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47ec082770 con 0x7f47f80cee50 2023-12-19T16:42:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.008+0000 7f47f7fff640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f47dc065a60 0x7f47dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.008+0000 7f47f7fff640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f47dc065a60 0x7f47dc067f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f47e40011f0 tx=0x7f47e4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:54.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.012+0000 7f47f57fa640 1 -- 172.21.15.26:0/2435903278 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f47f81557a0 con 0x7f47f80cee50 2023-12-19T16:42:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.168+0000 7f47ff02d640 1 -- 172.21.15.26:0/2435903278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47f81557a0 con 0x7f47f80cee50 2023-12-19T16:42:54.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.172+0000 7f47f57fa640 1 -- 172.21.15.26:0/2435903278 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f47f81557a0 con 0x7f47f80cee50 2023-12-19T16:42:54.175 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:54.175 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.172+0000 7f47ff02d640 1 -- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f47dc065a60 msgr2=0x7f47dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.172+0000 7f47ff02d640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f47dc065a60 0x7f47dc067f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f47e40011f0 tx=0x7f47e4009040 comp rx=0 tx=0).stop 2023-12-19T16:42:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.172+0000 7f47ff02d640 1 -- 172.21.15.26:0/2435903278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47f80cee50 msgr2=0x7f47f81501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.172+0000 7f47ff02d640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47f80cee50 0x7f47f81501f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00c4d0 tx=0x7f47ec00c990 comp rx=0 tx=0).stop 2023-12-19T16:42:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.172+0000 7f47ff02d640 1 -- 172.21.15.26:0/2435903278 shutdown_connections 2023-12-19T16:42:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.172+0000 7f47ff02d640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f47dc065a60 0x7f47dc067f20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.172+0000 7f47ff02d640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47f815f370 0x7f47f814ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.172+0000 7f47ff02d640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47f8155030 0x7f47f814e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.172+0000 7f47ff02d640 1 --2- 172.21.15.26:0/2435903278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47f80cee50 0x7f47f81501f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.172+0000 7f47ff02d640 1 -- 172.21.15.26:0/2435903278 >> 172.21.15.26:0/2435903278 conn(0x7f47f80658e0 msgr2=0x7f47f8065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:54.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.172+0000 7f47ff02d640 1 -- 172.21.15.26:0/2435903278 shutdown_connections 2023-12-19T16:42:54.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.172+0000 7f47ff02d640 1 -- 172.21.15.26:0/2435903278 wait complete. 2023-12-19T16:42:54.196 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:42:25.598550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:42:26.697362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:42:54.197 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:42:54.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.336+0000 7f2706095640 1 Processor -- start 2023-12-19T16:42:54.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.336+0000 7f2706095640 1 -- start start 2023-12-19T16:42:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.340+0000 7f2706095640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2700159820 0x7f270015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.340+0000 7f2706095640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f270006f6c0 0x7f27000c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.340+0000 7f2706095640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f270006fb60 con 0x7f2700159c00 2023-12-19T16:42:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.340+0000 7f2706095640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270015e470 con 0x7f2700159820 2023-12-19T16:42:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.340+0000 7f2706095640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270006fe40 con 0x7f270006f6c0 2023-12-19T16:42:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.340+0000 7f26ff7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2700159820 0x7f270015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.340+0000 7f26ff7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2700159820 0x7f270015df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52928/0 (socket says 172.21.15.26:52928) 2023-12-19T16:42:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.340+0000 7f26ff7fe640 1 -- 172.21.15.26:0/3204332526 learned_addr learned my addr 172.21.15.26:0/3204332526 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.340+0000 7f26ff7fe640 1 -- 172.21.15.26:0/3204332526 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f270006f6c0 msgr2=0x7f27000c8e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-19T16:42:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.340+0000 7f26ff7fe640 1 --2- 172.21.15.26:0/3204332526 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f270006f6c0 0x7f27000c8e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.340+0000 7f26ff7fe640 1 -- 172.21.15.26:0/3204332526 >> v1:172.21.15.160:6789/0 conn(0x7f2700159c00 legacy=0x7f2700070c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-19T16:42:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.340+0000 7f26ff7fe640 1 -- 172.21.15.26:0/3204332526 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2700070120 con 0x7f2700159820 2023-12-19T16:42:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f26ff7fe640 1 --2- 172.21.15.26:0/3204332526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2700159820 0x7f270015df30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009870 tx=0x7f26f001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df60275d45f3af25 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:54.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f26fe7fc640 1 -- 172.21.15.26:0/3204332526 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f00aa610 con 0x7f2700159820 2023-12-19T16:42:54.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f26fe7fc640 1 -- 172.21.15.26:0/3204332526 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f00a6040 con 0x7f2700159820 2023-12-19T16:42:54.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f26fe7fc640 1 -- 172.21.15.26:0/3204332526 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f00aa990 con 0x7f2700159820 2023-12-19T16:42:54.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f2706095640 1 -- 172.21.15.26:0/3204332526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2700159820 msgr2=0x7f270015df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f2706095640 1 --2- 172.21.15.26:0/3204332526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2700159820 0x7f270015df30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009870 tx=0x7f26f001c4f0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f2706095640 1 -- 172.21.15.26:0/3204332526 shutdown_connections 2023-12-19T16:42:54.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f2706095640 1 --2- 172.21.15.26:0/3204332526 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f270006f6c0 0x7f27000c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f2706095640 1 --2- 172.21.15.26:0/3204332526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2700159820 0x7f270015df30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f2706095640 1 -- 172.21.15.26:0/3204332526 >> 172.21.15.26:0/3204332526 conn(0x7f27000ce730 msgr2=0x7f27000ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:54.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f2706095640 1 -- 172.21.15.26:0/3204332526 shutdown_connections 2023-12-19T16:42:54.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f2706095640 1 -- 172.21.15.26:0/3204332526 wait complete. 2023-12-19T16:42:54.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f2706095640 1 Processor -- start 2023-12-19T16:42:54.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.344+0000 7f2706095640 1 -- start start 2023-12-19T16:42:54.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f2706095640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f270006f6c0 0x7f2700151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f2706095640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2700159820 0x7f27001f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f2706095640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2700159c00 0x7f2700157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f2706095640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27001524a0 con 0x7f270006f6c0 2023-12-19T16:42:54.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f2706095640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27000c21d0 con 0x7f2700159c00 2023-12-19T16:42:54.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f2706095640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2700151920 con 0x7f2700159820 2023-12-19T16:42:54.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f270006f6c0 0x7f2700151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f270006f6c0 0x7f2700151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52940/0 (socket says 172.21.15.26:52940) 2023-12-19T16:42:54.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26ff7fe640 1 -- 172.21.15.26:0/3615099985 learned_addr learned my addr 172.21.15.26:0/3615099985 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:54.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26fffff640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2700159c00 0x7f2700157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26feffd640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2700159820 0x7f27001f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26ff7fe640 1 -- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2700159820 msgr2=0x7f27001f9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26ff7fe640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2700159820 0x7f27001f9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26ff7fe640 1 -- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2700159c00 msgr2=0x7f2700157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26ff7fe640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2700159c00 0x7f2700157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26ff7fe640 1 -- 172.21.15.26:0/3615099985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f00094d0 con 0x7f270006f6c0 2023-12-19T16:42:54.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26ff7fe640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f270006f6c0 0x7f2700151040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f26f00ac040 tx=0x7f26f001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:54.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26e7fff640 1 -- 172.21.15.26:0/3615099985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f00bd5f0 con 0x7f270006f6c0 2023-12-19T16:42:54.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26e7fff640 1 -- 172.21.15.26:0/3615099985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f00bd770 con 0x7f270006f6c0 2023-12-19T16:42:54.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26e7fff640 1 -- 172.21.15.26:0/3615099985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f00bdb20 con 0x7f270006f6c0 2023-12-19T16:42:54.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f2706095640 1 -- 172.21.15.26:0/3615099985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270006b880 con 0x7f270006f6c0 2023-12-19T16:42:54.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f2706095640 1 -- 172.21.15.26:0/3615099985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2700151ee0 con 0x7f270006f6c0 2023-12-19T16:42:54.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f2706095640 1 -- 172.21.15.26:0/3615099985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2700074270 con 0x7f270006f6c0 2023-12-19T16:42:54.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.348+0000 7f26e7fff640 1 -- 172.21.15.26:0/3615099985 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f26f00b3600 con 0x7f270006f6c0 2023-12-19T16:42:54.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.352+0000 7f26e7fff640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f26ec065a90 0x7f26ec067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.352+0000 7f26e7fff640 1 -- 172.21.15.26:0/3615099985 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26f011e0a0 con 0x7f270006f6c0 2023-12-19T16:42:54.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.352+0000 7f26feffd640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f26ec065a90 0x7f26ec067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.352+0000 7f26feffd640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f26ec065a90 0x7f26ec067f50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2700150f90 tx=0x7f26f40090e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:54.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.356+0000 7f26e7fff640 1 -- 172.21.15.26:0/3615099985 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f26f00ebba0 con 0x7f270006f6c0 2023-12-19T16:42:54.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.512+0000 7f2706095640 1 -- 172.21.15.26:0/3615099985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27000c1f80 con 0x7f270006f6c0 2023-12-19T16:42:54.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.512+0000 7f26e7fff640 1 -- 172.21.15.26:0/3615099985 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26f00ef1c0 con 0x7f270006f6c0 2023-12-19T16:42:54.517 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:42:54.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.516+0000 7f2706095640 1 -- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f26ec065a90 msgr2=0x7f26ec067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.516+0000 7f2706095640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f26ec065a90 0x7f26ec067f50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2700150f90 tx=0x7f26f40090e0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.516+0000 7f2706095640 1 -- 172.21.15.26:0/3615099985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f270006f6c0 msgr2=0x7f2700151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.516+0000 7f2706095640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f270006f6c0 0x7f2700151040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f26f00ac040 tx=0x7f26f001ad30 comp rx=0 tx=0).stop 2023-12-19T16:42:54.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.516+0000 7f2706095640 1 -- 172.21.15.26:0/3615099985 shutdown_connections 2023-12-19T16:42:54.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.516+0000 7f2706095640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2700159c00 0x7f2700157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.516+0000 7f2706095640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f26ec065a90 0x7f26ec067f50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.516+0000 7f2706095640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2700159820 0x7f27001f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.516+0000 7f2706095640 1 --2- 172.21.15.26:0/3615099985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f270006f6c0 0x7f2700151040 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.516+0000 7f2706095640 1 -- 172.21.15.26:0/3615099985 >> 172.21.15.26:0/3615099985 conn(0x7f27000ce730 msgr2=0x7f27000c97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:54.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.516+0000 7f2706095640 1 -- 172.21.15.26:0/3615099985 shutdown_connections 2023-12-19T16:42:54.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.516+0000 7f2706095640 1 -- 172.21.15.26:0/3615099985 wait complete. 2023-12-19T16:42:54.539 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:42:54.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.684+0000 7f284c82d640 1 Processor -- start 2023-12-19T16:42:54.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.684+0000 7f284c82d640 1 -- start start 2023-12-19T16:42:54.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.684+0000 7f284c82d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f284806f6c0 0x7f284815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.684+0000 7f284c82d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284806fb60 con 0x7f2848159c00 2023-12-19T16:42:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.684+0000 7f284c82d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284806fe40 con 0x7f2848159820 2023-12-19T16:42:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.684+0000 7f284c82d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284815e470 con 0x7f284806f6c0 2023-12-19T16:42:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.684+0000 7f2846d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2848159c00 0x7f2848070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37260/0 (socket says 172.21.15.26:37260) 2023-12-19T16:42:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.684+0000 7f2846d9d640 1 -- 172.21.15.26:0/3995233196 learned_addr learned my addr 172.21.15.26:0/3995233196 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.684+0000 7f2845d9b640 1 --2- 172.21.15.26:0/3995233196 >> v2:172.21.15.26:3300/0 conn(0x7f284806f6c0 0x7f284815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.684+0000 7f284559a640 1 -- 172.21.15.26:0/3995233196 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3370003104 0 0) 0x7f284806fb60 con 0x7f2848159c00 2023-12-19T16:42:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.684+0000 7f2845d9b640 1 -- 172.21.15.26:0/3995233196 >> v1:172.21.15.160:6789/0 conn(0x7f2848159820 legacy=0x7f28480c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.684+0000 7f2845d9b640 1 -- 172.21.15.26:0/3995233196 >> v1:172.21.15.26:6789/0 conn(0x7f2848159c00 legacy=0x7f2848070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.684+0000 7f2845d9b640 1 -- 172.21.15.26:0/3995233196 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2848070120 con 0x7f284806f6c0 2023-12-19T16:42:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.688+0000 7f2845d9b640 1 --2- 172.21.15.26:0/3995233196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f284806f6c0 0x7f284815df30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f283c0097c0 tx=0x7f283c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813f08af9daca159 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:54.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.688+0000 7f284559a640 1 -- 172.21.15.26:0/3995233196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0ab650 con 0x7f284806f6c0 2023-12-19T16:42:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.688+0000 7f284559a640 1 -- 172.21.15.26:0/3995233196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c0ab7b0 con 0x7f284806f6c0 2023-12-19T16:42:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.688+0000 7f284559a640 1 -- 172.21.15.26:0/3995233196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0abb40 con 0x7f284806f6c0 2023-12-19T16:42:54.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.688+0000 7f284c82d640 1 -- 172.21.15.26:0/3995233196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f284806f6c0 msgr2=0x7f284815df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.688+0000 7f284c82d640 1 --2- 172.21.15.26:0/3995233196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f284806f6c0 0x7f284815df30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f283c0097c0 tx=0x7f283c01c450 comp rx=0 tx=0).stop 2023-12-19T16:42:54.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.688+0000 7f284c82d640 1 -- 172.21.15.26:0/3995233196 shutdown_connections 2023-12-19T16:42:54.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.688+0000 7f284c82d640 1 --2- 172.21.15.26:0/3995233196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f284806f6c0 0x7f284815df30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.688+0000 7f284c82d640 1 -- 172.21.15.26:0/3995233196 >> 172.21.15.26:0/3995233196 conn(0x7f28480ce730 msgr2=0x7f28480ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:54.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.688+0000 7f284c82d640 1 -- 172.21.15.26:0/3995233196 shutdown_connections 2023-12-19T16:42:54.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.688+0000 7f284c82d640 1 -- 172.21.15.26:0/3995233196 wait complete. 2023-12-19T16:42:54.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f284c82d640 1 Processor -- start 2023-12-19T16:42:54.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f284c82d640 1 -- start start 2023-12-19T16:42:54.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f284c82d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f284806f6c0 0x7f28480c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f284c82d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2848159820 0x7f28480c3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f284c82d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2848159c00 0x7f28480acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f284c82d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28480c3aa0 con 0x7f2848159820 2023-12-19T16:42:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f284c82d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28480cb140 con 0x7f284806f6c0 2023-12-19T16:42:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f284c82d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28480ca530 con 0x7f2848159c00 2023-12-19T16:42:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f2845d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2848159820 0x7f28480c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f2845d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2848159820 0x7f28480c3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52958/0 (socket says 172.21.15.26:52958) 2023-12-19T16:42:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f2845d9b640 1 -- 172.21.15.26:0/2091203486 learned_addr learned my addr 172.21.15.26:0/2091203486 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f2846d9d640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2848159c00 0x7f28480acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f284659c640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f284806f6c0 0x7f28480c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f2845d9b640 1 -- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2848159c00 msgr2=0x7f28480acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f2845d9b640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2848159c00 0x7f28480acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f2845d9b640 1 -- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f284806f6c0 msgr2=0x7f28480c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f2845d9b640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f284806f6c0 0x7f28480c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f2845d9b640 1 -- 172.21.15.26:0/2091203486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c009420 con 0x7f2848159820 2023-12-19T16:42:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f2845d9b640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2848159820 0x7f28480c3560 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f283c0ad040 tx=0x7f283c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f2836ffd640 1 -- 172.21.15.26:0/2091203486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0b5810 con 0x7f2848159820 2023-12-19T16:42:54.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f2836ffd640 1 -- 172.21.15.26:0/2091203486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c0b5990 con 0x7f2848159820 2023-12-19T16:42:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f2836ffd640 1 -- 172.21.15.26:0/2091203486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c0b5d40 con 0x7f2848159820 2023-12-19T16:42:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f284c82d640 1 -- 172.21.15.26:0/2091203486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284806b880 con 0x7f2848159820 2023-12-19T16:42:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.692+0000 7f284c82d640 1 -- 172.21.15.26:0/2091203486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28480caaf0 con 0x7f2848159820 2023-12-19T16:42:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.696+0000 7f284c82d640 1 -- 172.21.15.26:0/2091203486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2848074270 con 0x7f2848159820 2023-12-19T16:42:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.696+0000 7f2836ffd640 1 -- 172.21.15.26:0/2091203486 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f283c0bf410 con 0x7f2848159820 2023-12-19T16:42:54.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.696+0000 7f2836ffd640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2828065a90 0x7f2828067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:54.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.696+0000 7f2836ffd640 1 -- 172.21.15.26:0/2091203486 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f283c11dc90 con 0x7f2848159820 2023-12-19T16:42:54.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.696+0000 7f284659c640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2828065a90 0x7f2828067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:54.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.696+0000 7f284659c640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2828065a90 0x7f2828067f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2838000f70 tx=0x7f2838009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:54.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.700+0000 7f2836ffd640 1 -- 172.21.15.26:0/2091203486 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f283c0eb770 con 0x7f2848159820 2023-12-19T16:42:54.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.856+0000 7f284c82d640 1 -- 172.21.15.26:0/2091203486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f284815d6d0 con 0x7f2848159820 2023-12-19T16:42:54.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.856+0000 7f2836ffd640 1 -- 172.21.15.26:0/2091203486 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f283c0eed90 con 0x7f2848159820 2023-12-19T16:42:54.862 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:42:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.860+0000 7f284c82d640 1 -- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2828065a90 msgr2=0x7f2828067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.860+0000 7f284c82d640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2828065a90 0x7f2828067f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2838000f70 tx=0x7f2838009040 comp rx=0 tx=0).stop 2023-12-19T16:42:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.860+0000 7f284c82d640 1 -- 172.21.15.26:0/2091203486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2848159820 msgr2=0x7f28480c3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.860+0000 7f284c82d640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2848159820 0x7f28480c3560 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f283c0ad040 tx=0x7f283c01ac90 comp rx=0 tx=0).stop 2023-12-19T16:42:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.860+0000 7f284c82d640 1 -- 172.21.15.26:0/2091203486 shutdown_connections 2023-12-19T16:42:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.860+0000 7f284c82d640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2828065a90 0x7f2828067f50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.860+0000 7f284c82d640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2848159c00 0x7f28480acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.860+0000 7f284c82d640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2848159820 0x7f28480c3560 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.860+0000 7f284c82d640 1 --2- 172.21.15.26:0/2091203486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f284806f6c0 0x7f28480c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.860+0000 7f284c82d640 1 -- 172.21.15.26:0/2091203486 >> 172.21.15.26:0/2091203486 conn(0x7f28480ce730 msgr2=0x7f284806ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:54.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.860+0000 7f284c82d640 1 -- 172.21.15.26:0/2091203486 shutdown_connections 2023-12-19T16:42:54.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:54.860+0000 7f284c82d640 1 -- 172.21.15.26:0/2091203486 wait complete. 2023-12-19T16:42:54.883 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:42:55.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.024+0000 7fa01d7df640 1 Processor -- start 2023-12-19T16:42:55.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.024+0000 7fa01d7df640 1 -- start start 2023-12-19T16:42:55.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01d7df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa01815c620 0x7fa01815a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01d7df640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa0180b37f0 0x7fa0180b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01d7df640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa01815cd10 0x7fa01815d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01d7df640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01815d6c0 con 0x7fa01815cd10 2023-12-19T16:42:55.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01d7df640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01815ad70 con 0x7fa0180b37f0 2023-12-19T16:42:55.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01d7df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180b26f0 con 0x7fa01815c620 2023-12-19T16:42:55.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01cfde640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa01815cd10 0x7fa01815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01cfde640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa01815cd10 0x7fa01815d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52960/0 (socket says 172.21.15.26:52960) 2023-12-19T16:42:55.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01cfde640 1 -- 172.21.15.26:0/3440492829 learned_addr learned my addr 172.21.15.26:0/3440492829 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:55.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa017fff640 1 --2- 172.21.15.26:0/3440492829 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa01815c620 0x7fa01815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa0177fe640 1 --2- 172.21.15.26:0/3440492829 >> v2:172.21.15.160:3300/0 conn(0x7fa0180b37f0 0x7fa0180b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01cfde640 1 -- 172.21.15.26:0/3440492829 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa01815c620 msgr2=0x7fa01815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:55.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01cfde640 1 --2- 172.21.15.26:0/3440492829 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa01815c620 0x7fa01815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01cfde640 1 -- 172.21.15.26:0/3440492829 >> v2:172.21.15.160:3300/0 conn(0x7fa0180b37f0 msgr2=0x7fa0180b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:55.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01cfde640 1 --2- 172.21.15.26:0/3440492829 >> v2:172.21.15.160:3300/0 conn(0x7fa0180b37f0 0x7fa0180b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.028+0000 7fa01cfde640 1 -- 172.21.15.26:0/3440492829 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0180b1e50 con 0x7fa01815cd10 2023-12-19T16:42:55.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa01cfde640 1 --2- 172.21.15.26:0/3440492829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa01815cd10 0x7fa01815d180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa01806c680 tx=0x7fa010020d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94274ac76ea217ae server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:55.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa016ffd640 1 -- 172.21.15.26:0/3440492829 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100a9b60 con 0x7fa01815cd10 2023-12-19T16:42:55.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa016ffd640 1 -- 172.21.15.26:0/3440492829 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0100a9cc0 con 0x7fa01815cd10 2023-12-19T16:42:55.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa016ffd640 1 -- 172.21.15.26:0/3440492829 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100b1610 con 0x7fa01815cd10 2023-12-19T16:42:55.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa01d7df640 1 -- 172.21.15.26:0/3440492829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa01815cd10 msgr2=0x7fa01815d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa01d7df640 1 --2- 172.21.15.26:0/3440492829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa01815cd10 0x7fa01815d180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa01806c680 tx=0x7fa010020d30 comp rx=0 tx=0).stop 2023-12-19T16:42:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa01d7df640 1 -- 172.21.15.26:0/3440492829 shutdown_connections 2023-12-19T16:42:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa01d7df640 1 --2- 172.21.15.26:0/3440492829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa01815cd10 0x7fa01815d180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa01d7df640 1 --2- 172.21.15.26:0/3440492829 >> v2:172.21.15.160:3300/0 conn(0x7fa0180b37f0 0x7fa0180b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa01d7df640 1 --2- 172.21.15.26:0/3440492829 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa01815c620 0x7fa01815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa01d7df640 1 -- 172.21.15.26:0/3440492829 >> 172.21.15.26:0/3440492829 conn(0x7fa0180658e0 msgr2=0x7fa018065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa01d7df640 1 -- 172.21.15.26:0/3440492829 shutdown_connections 2023-12-19T16:42:55.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa01d7df640 1 -- 172.21.15.26:0/3440492829 wait complete. 2023-12-19T16:42:55.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa01d7df640 1 Processor -- start 2023-12-19T16:42:55.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.032+0000 7fa01d7df640 1 -- start start 2023-12-19T16:42:55.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa01d7df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0180b37f0 0x7fa0181482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa01d7df640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01815c620 0x7fa0181526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa01d7df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa01815cd10 0x7fa0180aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa01d7df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180cd5b0 con 0x7fa0180b37f0 2023-12-19T16:42:55.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa01d7df640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa018151a70 con 0x7fa01815c620 2023-12-19T16:42:55.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa01d7df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0180cc930 con 0x7fa01815cd10 2023-12-19T16:42:55.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa017fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0180b37f0 0x7fa0181482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa017fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0180b37f0 0x7fa0181482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52964/0 (socket says 172.21.15.26:52964) 2023-12-19T16:42:55.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa017fff640 1 -- 172.21.15.26:0/3315339105 learned_addr learned my addr 172.21.15.26:0/3315339105 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:55.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa0177fe640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01815c620 0x7fa0181526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa01cfde640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa01815cd10 0x7fa0180aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa017fff640 1 -- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa01815cd10 msgr2=0x7fa0180aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa017fff640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa01815cd10 0x7fa0180aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa017fff640 1 -- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01815c620 msgr2=0x7fa0181526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa017fff640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01815c620 0x7fa0181526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa017fff640 1 -- 172.21.15.26:0/3315339105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa010009b30 con 0x7fa0180b37f0 2023-12-19T16:42:55.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa017fff640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0180b37f0 0x7fa0181482b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa00800e870 tx=0x7fa00800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:55.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa014ff9640 1 -- 172.21.15.26:0/3315339105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa008009510 con 0x7fa0180b37f0 2023-12-19T16:42:55.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa014ff9640 1 -- 172.21.15.26:0/3315339105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00800eec0 con 0x7fa0180b37f0 2023-12-19T16:42:55.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa014ff9640 1 -- 172.21.15.26:0/3315339105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00800cb40 con 0x7fa0180b37f0 2023-12-19T16:42:55.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa01d7df640 1 -- 172.21.15.26:0/3315339105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0180ccc10 con 0x7fa0180b37f0 2023-12-19T16:42:55.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa01d7df640 1 -- 172.21.15.26:0/3315339105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0180cd1d0 con 0x7fa0180b37f0 2023-12-19T16:42:55.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa01d7df640 1 -- 172.21.15.26:0/3315339105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0180cd820 con 0x7fa0180b37f0 2023-12-19T16:42:55.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.036+0000 7fa014ff9640 1 -- 172.21.15.26:0/3315339105 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa00800cca0 con 0x7fa0180b37f0 2023-12-19T16:42:55.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.040+0000 7fa014ff9640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa004065a60 0x7fa004067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.040+0000 7fa014ff9640 1 -- 172.21.15.26:0/3315339105 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa008083340 con 0x7fa0180b37f0 2023-12-19T16:42:55.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.040+0000 7fa0177fe640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa004065a60 0x7fa004067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.040+0000 7fa0177fe640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa004065a60 0x7fa004067f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa0180b0060 tx=0x7fa00c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:55.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.044+0000 7fa014ff9640 1 -- 172.21.15.26:0/3315339105 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa008050c80 con 0x7fa0180b37f0 2023-12-19T16:42:55.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.200+0000 7fa01d7df640 1 -- 172.21.15.26:0/3315339105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0180aeb00 con 0x7fa0180b37f0 2023-12-19T16:42:55.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.200+0000 7fa014ff9640 1 -- 172.21.15.26:0/3315339105 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0080542a0 con 0x7fa0180b37f0 2023-12-19T16:42:55.206 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:42:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.204+0000 7fa01d7df640 1 -- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa004065a60 msgr2=0x7fa004067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.204+0000 7fa01d7df640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa004065a60 0x7fa004067f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa0180b0060 tx=0x7fa00c009190 comp rx=0 tx=0).stop 2023-12-19T16:42:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.204+0000 7fa01d7df640 1 -- 172.21.15.26:0/3315339105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0180b37f0 msgr2=0x7fa0181482b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.204+0000 7fa01d7df640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0180b37f0 0x7fa0181482b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa00800e870 tx=0x7fa00800ec70 comp rx=0 tx=0).stop 2023-12-19T16:42:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.204+0000 7fa01d7df640 1 -- 172.21.15.26:0/3315339105 shutdown_connections 2023-12-19T16:42:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.204+0000 7fa01d7df640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa004065a60 0x7fa004067f20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.204+0000 7fa01d7df640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa01815cd10 0x7fa0180aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.204+0000 7fa01d7df640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01815c620 0x7fa0181526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.204+0000 7fa01d7df640 1 --2- 172.21.15.26:0/3315339105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0180b37f0 0x7fa0181482b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.204+0000 7fa01d7df640 1 -- 172.21.15.26:0/3315339105 >> 172.21.15.26:0/3315339105 conn(0x7fa0180658e0 msgr2=0x7fa0180adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.204+0000 7fa01d7df640 1 -- 172.21.15.26:0/3315339105 shutdown_connections 2023-12-19T16:42:55.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.204+0000 7fa01d7df640 1 -- 172.21.15.26:0/3315339105 wait complete. 2023-12-19T16:42:55.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.364+0000 7f50f57bc640 1 Processor -- start 2023-12-19T16:42:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.364+0000 7f50f57bc640 1 -- start start 2023-12-19T16:42:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.368+0000 7f50f57bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50f00cbc50 0x7f50f00b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.368+0000 7f50f57bc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f00cc0d0 con 0x7f50f00b3310 2023-12-19T16:42:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.368+0000 7f50f57bc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f015ddc0 con 0x7f50f006cb30 2023-12-19T16:42:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.368+0000 7f50f57bc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f015e0a0 con 0x7f50f00cbc50 2023-12-19T16:42:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.368+0000 7f50ef7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f50f00b3310 0x7f50f00ad180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37262/0 (socket says 172.21.15.26:37262) 2023-12-19T16:42:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.368+0000 7f50ef7fe640 1 -- 172.21.15.26:0/2218418672 learned_addr learned my addr 172.21.15.26:0/2218418672 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.368+0000 7f50e7fff640 1 --2- 172.21.15.26:0/2218418672 >> v2:172.21.15.26:3300/0 conn(0x7f50f00cbc50 0x7f50f00b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.368+0000 7f50ee7fc640 1 -- 172.21.15.26:0/2218418672 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249001695 0 0) 0x7f50e80014a0 con 0x7f50f00b3310 2023-12-19T16:42:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.368+0000 7f50ee7fc640 1 -- 172.21.15.26:0/2218418672 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50dc003530 con 0x7f50f00b3310 2023-12-19T16:42:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.368+0000 7f50e7fff640 1 -- 172.21.15.26:0/2218418672 >> v1:172.21.15.160:6789/0 conn(0x7f50f006cb30 legacy=0x7f50f015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.368+0000 7f50e7fff640 1 -- 172.21.15.26:0/2218418672 >> v1:172.21.15.26:6789/0 conn(0x7f50f00b3310 legacy=0x7f50f00ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.368+0000 7f50e7fff640 1 -- 172.21.15.26:0/2218418672 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f00cb450 con 0x7f50f00cbc50 2023-12-19T16:42:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.372+0000 7f50e7fff640 1 --2- 172.21.15.26:0/2218418672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f00cbc50 0x7f50f00b2dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f50d80097c0 tx=0x7f50d801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf898342dbee9fbd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.372+0000 7f50ee7fc640 1 -- 172.21.15.26:0/2218418672 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80ab670 con 0x7f50f00cbc50 2023-12-19T16:42:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.372+0000 7f50ee7fc640 1 -- 172.21.15.26:0/2218418672 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d80ab7d0 con 0x7f50f00cbc50 2023-12-19T16:42:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.372+0000 7f50ee7fc640 1 -- 172.21.15.26:0/2218418672 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80abb60 con 0x7f50f00cbc50 2023-12-19T16:42:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.372+0000 7f50f57bc640 1 -- 172.21.15.26:0/2218418672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f00cbc50 msgr2=0x7f50f00b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.372+0000 7f50f57bc640 1 --2- 172.21.15.26:0/2218418672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f00cbc50 0x7f50f00b2dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f50d80097c0 tx=0x7f50d801c450 comp rx=0 tx=0).stop 2023-12-19T16:42:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.372+0000 7f50f57bc640 1 -- 172.21.15.26:0/2218418672 shutdown_connections 2023-12-19T16:42:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.372+0000 7f50f57bc640 1 --2- 172.21.15.26:0/2218418672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f00cbc50 0x7f50f00b2dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.372+0000 7f50f57bc640 1 -- 172.21.15.26:0/2218418672 >> 172.21.15.26:0/2218418672 conn(0x7f50f00658e0 msgr2=0x7f50f0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.372+0000 7f50f57bc640 1 -- 172.21.15.26:0/2218418672 shutdown_connections 2023-12-19T16:42:55.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.372+0000 7f50f57bc640 1 -- 172.21.15.26:0/2218418672 wait complete. 2023-12-19T16:42:55.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.372+0000 7f50f57bc640 1 Processor -- start 2023-12-19T16:42:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.372+0000 7f50f57bc640 1 -- start start 2023-12-19T16:42:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50f57bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50f006cb30 0x7f50f0139c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50f57bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50f00b3310 0x7f50f00af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50f57bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f00cbc50 0x7f50f0159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50f57bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f00b1290 con 0x7f50f00cbc50 2023-12-19T16:42:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50f57bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f00af680 con 0x7f50f00b3310 2023-12-19T16:42:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50f57bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f00b0fb0 con 0x7f50f006cb30 2023-12-19T16:42:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50ef7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f00cbc50 0x7f50f0159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50ef7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f00cbc50 0x7f50f0159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52982/0 (socket says 172.21.15.26:52982) 2023-12-19T16:42:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50ef7fe640 1 -- 172.21.15.26:0/329041259 learned_addr learned my addr 172.21.15.26:0/329041259 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50e7fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50f00b3310 0x7f50f00af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50eeffd640 1 --2- 172.21.15.26:0/329041259 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50f006cb30 0x7f50f0139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50ef7fe640 1 -- 172.21.15.26:0/329041259 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50f006cb30 msgr2=0x7f50f0139c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50ef7fe640 1 --2- 172.21.15.26:0/329041259 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50f006cb30 0x7f50f0139c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50ef7fe640 1 -- 172.21.15.26:0/329041259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50f00b3310 msgr2=0x7f50f00af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50ef7fe640 1 --2- 172.21.15.26:0/329041259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50f00b3310 0x7f50f00af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50ef7fe640 1 -- 172.21.15.26:0/329041259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d8009420 con 0x7f50f00cbc50 2023-12-19T16:42:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50ef7fe640 1 --2- 172.21.15.26:0/329041259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f00cbc50 0x7f50f0159540 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f50e800cb70 tx=0x7f50e800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:55.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50e77fe640 1 -- 172.21.15.26:0/329041259 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e800ad40 con 0x7f50f00cbc50 2023-12-19T16:42:55.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50e77fe640 1 -- 172.21.15.26:0/329041259 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e8012400 con 0x7f50f00cbc50 2023-12-19T16:42:55.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50e77fe640 1 -- 172.21.15.26:0/329041259 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e800aea0 con 0x7f50f00cbc50 2023-12-19T16:42:55.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50f57bc640 1 -- 172.21.15.26:0/329041259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f015b4e0 con 0x7f50f00cbc50 2023-12-19T16:42:55.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50f57bc640 1 -- 172.21.15.26:0/329041259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f015baa0 con 0x7f50f00cbc50 2023-12-19T16:42:55.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50f57bc640 1 -- 172.21.15.26:0/329041259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f015b340 con 0x7f50f00cbc50 2023-12-19T16:42:55.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.376+0000 7f50e77fe640 1 -- 172.21.15.26:0/329041259 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f50e8007070 con 0x7f50f00cbc50 2023-12-19T16:42:55.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.380+0000 7f50e77fe640 1 --2- 172.21.15.26:0/329041259 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f50d0065a60 0x7f50d0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.380+0000 7f50e77fe640 1 -- 172.21.15.26:0/329041259 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50e8082f50 con 0x7f50f00cbc50 2023-12-19T16:42:55.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.380+0000 7f50eeffd640 1 --2- 172.21.15.26:0/329041259 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f50d0065a60 0x7f50d0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.380+0000 7f50eeffd640 1 --2- 172.21.15.26:0/329041259 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f50d0065a60 0x7f50d0067f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f50e0007fd0 tx=0x7f50e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:55.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.384+0000 7f50e77fe640 1 -- 172.21.15.26:0/329041259 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f50f015b340 con 0x7f50f00cbc50 2023-12-19T16:42:55.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.540+0000 7f50f57bc640 1 -- 172.21.15.26:0/329041259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50f00cc0d0 con 0x7f50f00cbc50 2023-12-19T16:42:55.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.544+0000 7f50e77fe640 1 -- 172.21.15.26:0/329041259 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f50f00cc0d0 con 0x7f50f00cbc50 2023-12-19T16:42:55.547 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:55.547 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.544+0000 7f50f57bc640 1 -- 172.21.15.26:0/329041259 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f50d0065a60 msgr2=0x7f50d0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.544+0000 7f50f57bc640 1 --2- 172.21.15.26:0/329041259 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f50d0065a60 0x7f50d0067f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f50e0007fd0 tx=0x7f50e0009040 comp rx=0 tx=0).stop 2023-12-19T16:42:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.544+0000 7f50f57bc640 1 -- 172.21.15.26:0/329041259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f00cbc50 msgr2=0x7f50f0159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.544+0000 7f50f57bc640 1 --2- 172.21.15.26:0/329041259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f00cbc50 0x7f50f0159540 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f50e800cb70 tx=0x7f50e800a430 comp rx=0 tx=0).stop 2023-12-19T16:42:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.544+0000 7f50f57bc640 1 -- 172.21.15.26:0/329041259 shutdown_connections 2023-12-19T16:42:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.544+0000 7f50f57bc640 1 --2- 172.21.15.26:0/329041259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f00cbc50 0x7f50f0159540 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.544+0000 7f50f57bc640 1 --2- 172.21.15.26:0/329041259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50f00b3310 0x7f50f00af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.544+0000 7f50f57bc640 1 --2- 172.21.15.26:0/329041259 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f50d0065a60 0x7f50d0067f20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.544+0000 7f50f57bc640 1 --2- 172.21.15.26:0/329041259 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f50f006cb30 0x7f50f0139c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.544+0000 7f50f57bc640 1 -- 172.21.15.26:0/329041259 >> 172.21.15.26:0/329041259 conn(0x7f50f00658e0 msgr2=0x7f50f006f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:55.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.544+0000 7f50f57bc640 1 -- 172.21.15.26:0/329041259 shutdown_connections 2023-12-19T16:42:55.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.544+0000 7f50f57bc640 1 -- 172.21.15.26:0/329041259 wait complete. 2023-12-19T16:42:55.570 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:42:25.598550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:42:26.697362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:42:55.570 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:42:55.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.712+0000 7f184cb34640 1 Processor -- start 2023-12-19T16:42:55.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.712+0000 7f184cb34640 1 -- start start 2023-12-19T16:42:55.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.716+0000 7f184cb34640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f184815e5b0 0x7f184815d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.716+0000 7f184cb34640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18480d0dc0 con 0x7f18480cfc30 2023-12-19T16:42:55.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.716+0000 7f184cb34640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f184815e240 con 0x7f1848157410 2023-12-19T16:42:55.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.716+0000 7f184cb34640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18480d1010 con 0x7f184815e5b0 2023-12-19T16:42:55.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.716+0000 7f1846ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f184815e5b0 0x7f184815d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.716+0000 7f1847fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f18480cfc30 0x7f184815da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37274/0 (socket says 172.21.15.26:37274) 2023-12-19T16:42:55.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.716+0000 7f1847fff640 1 -- 172.21.15.26:0/1655929264 learned_addr learned my addr 172.21.15.26:0/1655929264 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:55.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.716+0000 7f1846ffd640 1 -- 172.21.15.26:0/1655929264 >> v1:172.21.15.160:6789/0 conn(0x7f1848157410 legacy=0x7f18480cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:55.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.716+0000 7f1846ffd640 1 -- 172.21.15.26:0/1655929264 >> v1:172.21.15.26:6789/0 conn(0x7f18480cfc30 legacy=0x7f184815da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.716+0000 7f1846ffd640 1 -- 172.21.15.26:0/1655929264 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18480d12f0 con 0x7f184815e5b0 2023-12-19T16:42:55.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.716+0000 7f18467fc640 1 -- 172.21.15.26:0/1655929264 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591539206 0 0) 0x7f18480d0dc0 con 0x7f18480cfc30 2023-12-19T16:42:55.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f1846ffd640 1 --2- 172.21.15.26:0/1655929264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184815e5b0 0x7f184815d4d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f183c0097c0 tx=0x7f183c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c0be72352d042f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:55.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f18467fc640 1 -- 172.21.15.26:0/1655929264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c0ac640 con 0x7f184815e5b0 2023-12-19T16:42:55.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f18467fc640 1 -- 172.21.15.26:0/1655929264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c0ac7a0 con 0x7f184815e5b0 2023-12-19T16:42:55.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f18467fc640 1 -- 172.21.15.26:0/1655929264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c0acb30 con 0x7f184815e5b0 2023-12-19T16:42:55.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 -- 172.21.15.26:0/1655929264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184815e5b0 msgr2=0x7f184815d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 --2- 172.21.15.26:0/1655929264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184815e5b0 0x7f184815d4d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f183c0097c0 tx=0x7f183c01c3c0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 -- 172.21.15.26:0/1655929264 shutdown_connections 2023-12-19T16:42:55.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 --2- 172.21.15.26:0/1655929264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184815e5b0 0x7f184815d4d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 -- 172.21.15.26:0/1655929264 >> 172.21.15.26:0/1655929264 conn(0x7f184806a120 msgr2=0x7f184806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:55.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 -- 172.21.15.26:0/1655929264 shutdown_connections 2023-12-19T16:42:55.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 -- 172.21.15.26:0/1655929264 wait complete. 2023-12-19T16:42:55.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 Processor -- start 2023-12-19T16:42:55.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 -- start start 2023-12-19T16:42:55.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18480cfc30 0x7f184814f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1848157410 0x7f18481585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f184815e5b0 0x7f18480b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1848157b50 con 0x7f1848157410 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184814eed0 con 0x7f18480cfc30 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f184cb34640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1848157df0 con 0x7f184815e5b0 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.720+0000 7f1846ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1848157410 0x7f18481585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f1846ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1848157410 0x7f18481585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52992/0 (socket says 172.21.15.26:52992) 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f1846ffd640 1 -- 172.21.15.26:0/1648552299 learned_addr learned my addr 172.21.15.26:0/1648552299 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f1847fff640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f184815e5b0 0x7f18480b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f18477fe640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18480cfc30 0x7f184814f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f1846ffd640 1 -- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f184815e5b0 msgr2=0x7f18480b71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f1846ffd640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f184815e5b0 0x7f18480b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f1846ffd640 1 -- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18480cfc30 msgr2=0x7f184814f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f1846ffd640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18480cfc30 0x7f184814f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f1846ffd640 1 -- 172.21.15.26:0/1648552299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c009420 con 0x7f1848157410 2023-12-19T16:42:55.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f18477fe640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18480cfc30 0x7f184814f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:55.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f1846ffd640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1848157410 0x7f18481585f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f183c018fd0 tx=0x7f183c019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:55.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f182ffff640 1 -- 172.21.15.26:0/1648552299 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c0b5610 con 0x7f1848157410 2023-12-19T16:42:55.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f182ffff640 1 -- 172.21.15.26:0/1648552299 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c0b5770 con 0x7f1848157410 2023-12-19T16:42:55.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f184cb34640 1 -- 172.21.15.26:0/1648552299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18481580d0 con 0x7f1848157410 2023-12-19T16:42:55.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f182ffff640 1 -- 172.21.15.26:0/1648552299 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c0b5b20 con 0x7f1848157410 2023-12-19T16:42:55.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f184cb34640 1 -- 172.21.15.26:0/1648552299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184814be70 con 0x7f1848157410 2023-12-19T16:42:55.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f184cb34640 1 -- 172.21.15.26:0/1648552299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1848158310 con 0x7f1848157410 2023-12-19T16:42:55.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f182ffff640 1 -- 172.21.15.26:0/1648552299 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f183c0b3040 con 0x7f1848157410 2023-12-19T16:42:55.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.724+0000 7f182ffff640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1834065a90 0x7f1834067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:55.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.728+0000 7f182ffff640 1 -- 172.21.15.26:0/1648552299 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f183c11d8a0 con 0x7f1848157410 2023-12-19T16:42:55.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.728+0000 7f18477fe640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1834065a90 0x7f1834067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:55.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.728+0000 7f18477fe640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1834065a90 0x7f1834067f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1838000f70 tx=0x7f1838009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:55.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.728+0000 7f182ffff640 1 -- 172.21.15.26:0/1648552299 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1848158310 con 0x7f1848157410 2023-12-19T16:42:55.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.888+0000 7f184cb34640 1 -- 172.21.15.26:0/1648552299 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1848158310 con 0x7f1848157410 2023-12-19T16:42:55.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.888+0000 7f182ffff640 1 -- 172.21.15.26:0/1648552299 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1848158310 con 0x7f1848157410 2023-12-19T16:42:55.893 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:42:55.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.892+0000 7f184cb34640 1 -- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1834065a90 msgr2=0x7f1834067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.892+0000 7f184cb34640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1834065a90 0x7f1834067f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1838000f70 tx=0x7f1838009040 comp rx=0 tx=0).stop 2023-12-19T16:42:55.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.892+0000 7f184cb34640 1 -- 172.21.15.26:0/1648552299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1848157410 msgr2=0x7f18481585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:55.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.892+0000 7f184cb34640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1848157410 0x7f18481585f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f183c018fd0 tx=0x7f183c019160 comp rx=0 tx=0).stop 2023-12-19T16:42:55.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.892+0000 7f184cb34640 1 -- 172.21.15.26:0/1648552299 shutdown_connections 2023-12-19T16:42:55.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.892+0000 7f184cb34640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1834065a90 0x7f1834067f50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.892+0000 7f184cb34640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f184815e5b0 0x7f18480b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.892+0000 7f184cb34640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1848157410 0x7f18481585f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.892+0000 7f184cb34640 1 --2- 172.21.15.26:0/1648552299 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18480cfc30 0x7f184814f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:55.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.892+0000 7f184cb34640 1 -- 172.21.15.26:0/1648552299 >> 172.21.15.26:0/1648552299 conn(0x7f184806a120 msgr2=0x7f18480658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:55.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.892+0000 7f184cb34640 1 -- 172.21.15.26:0/1648552299 shutdown_connections 2023-12-19T16:42:55.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:55.892+0000 7f184cb34640 1 -- 172.21.15.26:0/1648552299 wait complete. 2023-12-19T16:42:55.915 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:42:56.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.056+0000 7f9deccd1640 1 Processor -- start 2023-12-19T16:42:56.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.056+0000 7f9deccd1640 1 -- start start 2023-12-19T16:42:56.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.056+0000 7f9deccd1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9de815a9c0 0x7f9de80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.056+0000 7f9deccd1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9de806ac80 0x7f9de80b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.056+0000 7f9deccd1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de8070d90 con 0x7f9de815ada0 2023-12-19T16:42:56.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.056+0000 7f9deccd1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de80cc270 con 0x7f9de815a9c0 2023-12-19T16:42:56.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.056+0000 7f9deccd1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de8071070 con 0x7f9de806ac80 2023-12-19T16:42:56.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.056+0000 7f9de6d9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9de806ac80 0x7f9de80b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.056+0000 7f9de6d9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9de806ac80 0x7f9de80b01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:57248/0 (socket says 172.21.15.26:57248) 2023-12-19T16:42:56.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.056+0000 7f9de6d9d640 1 -- 172.21.15.26:0/750760465 learned_addr learned my addr 172.21.15.26:0/750760465 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:56.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.056+0000 7f9de659c640 1 --2- 172.21.15.26:0/750760465 >> v2:172.21.15.160:3300/0 conn(0x7f9de815a9c0 0x7f9de80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.056+0000 7f9de6d9d640 1 -- 172.21.15.26:0/750760465 >> v2:172.21.15.160:3300/0 conn(0x7f9de815a9c0 msgr2=0x7f9de80706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.056+0000 7f9de6d9d640 1 --2- 172.21.15.26:0/750760465 >> v2:172.21.15.160:3300/0 conn(0x7f9de815a9c0 0x7f9de80706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9de6d9d640 1 -- 172.21.15.26:0/750760465 >> v1:172.21.15.160:6789/0 conn(0x7f9de815ada0 legacy=0x7f9de80b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9de6d9d640 1 -- 172.21.15.26:0/750760465 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de8071350 con 0x7f9de806ac80 2023-12-19T16:42:56.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9de6d9d640 1 --2- 172.21.15.26:0/750760465 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9de806ac80 0x7f9de80b01b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009870 tx=0x7f9de001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b70961d2b5df1d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9de5d9b640 1 -- 172.21.15.26:0/750760465 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00ad640 con 0x7f9de806ac80 2023-12-19T16:42:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9de5d9b640 1 -- 172.21.15.26:0/750760465 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de00ad7a0 con 0x7f9de806ac80 2023-12-19T16:42:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9de5d9b640 1 -- 172.21.15.26:0/750760465 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00adb30 con 0x7f9de806ac80 2023-12-19T16:42:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9deccd1640 1 -- 172.21.15.26:0/750760465 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9de806ac80 msgr2=0x7f9de80b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9deccd1640 1 --2- 172.21.15.26:0/750760465 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9de806ac80 0x7f9de80b01b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009870 tx=0x7f9de001c480 comp rx=0 tx=0).stop 2023-12-19T16:42:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9deccd1640 1 -- 172.21.15.26:0/750760465 shutdown_connections 2023-12-19T16:42:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9deccd1640 1 --2- 172.21.15.26:0/750760465 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9de806ac80 0x7f9de80b01b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9deccd1640 1 --2- 172.21.15.26:0/750760465 >> v2:172.21.15.160:3300/0 conn(0x7f9de815a9c0 0x7f9de80706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9deccd1640 1 -- 172.21.15.26:0/750760465 >> 172.21.15.26:0/750760465 conn(0x7f9de80ce730 msgr2=0x7f9de80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9deccd1640 1 -- 172.21.15.26:0/750760465 shutdown_connections 2023-12-19T16:42:56.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.060+0000 7f9deccd1640 1 -- 172.21.15.26:0/750760465 wait complete. 2023-12-19T16:42:56.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9deccd1640 1 Processor -- start 2023-12-19T16:42:56.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9deccd1640 1 -- start start 2023-12-19T16:42:56.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9deccd1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9de806ac80 0x7f9de8150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9deccd1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9de815a9c0 0x7f9de815d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9deccd1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9de815ada0 0x7f9de80c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9deccd1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de8151240 con 0x7f9de806ac80 2023-12-19T16:42:56.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9deccd1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de815ccf0 con 0x7f9de815ada0 2023-12-19T16:42:56.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9deccd1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de815d6c0 con 0x7f9de815a9c0 2023-12-19T16:42:56.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9de659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9de806ac80 0x7f9de8150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9de659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9de806ac80 0x7f9de8150d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33926/0 (socket says 172.21.15.26:33926) 2023-12-19T16:42:56.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9de659c640 1 -- 172.21.15.26:0/482644339 learned_addr learned my addr 172.21.15.26:0/482644339 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:56.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9dddd9b640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9de815a9c0 0x7f9de815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9de6d9d640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9de815ada0 0x7f9de80c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9de659c640 1 -- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9de815a9c0 msgr2=0x7f9de815d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9de659c640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9de815a9c0 0x7f9de815d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9de659c640 1 -- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9de815ada0 msgr2=0x7f9de80c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9de659c640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9de815ada0 0x7f9de80c69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9de659c640 1 -- 172.21.15.26:0/482644339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00094d0 con 0x7f9de806ac80 2023-12-19T16:42:56.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.064+0000 7f9de6d9d640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9de815ada0 0x7f9de80c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:42:56.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.068+0000 7f9de659c640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9de806ac80 0x7f9de8150d00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800efc0 tx=0x7f9dd800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:56.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.068+0000 7f9ddf7fe640 1 -- 172.21.15.26:0/482644339 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8014480 con 0x7f9de806ac80 2023-12-19T16:42:56.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.068+0000 7f9ddf7fe640 1 -- 172.21.15.26:0/482644339 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd800f040 con 0x7f9de806ac80 2023-12-19T16:42:56.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.068+0000 7f9deccd1640 1 -- 172.21.15.26:0/482644339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de815c0e0 con 0x7f9de806ac80 2023-12-19T16:42:56.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.068+0000 7f9ddf7fe640 1 -- 172.21.15.26:0/482644339 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8014800 con 0x7f9de806ac80 2023-12-19T16:42:56.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.068+0000 7f9deccd1640 1 -- 172.21.15.26:0/482644339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de815c6a0 con 0x7f9de806ac80 2023-12-19T16:42:56.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.068+0000 7f9deccd1640 1 -- 172.21.15.26:0/482644339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de8074270 con 0x7f9de806ac80 2023-12-19T16:42:56.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.068+0000 7f9ddf7fe640 1 -- 172.21.15.26:0/482644339 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9dd8014960 con 0x7f9de806ac80 2023-12-19T16:42:56.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.068+0000 7f9ddf7fe640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9dc8065a60 0x7f9dc8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.068+0000 7f9ddf7fe640 1 -- 172.21.15.26:0/482644339 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9dd8083c50 con 0x7f9de806ac80 2023-12-19T16:42:56.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.068+0000 7f9dddd9b640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9dc8065a60 0x7f9dc8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.072+0000 7f9dddd9b640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9dc8065a60 0x7f9dc8067f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9de81f94f0 tx=0x7f9dd0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:56.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.072+0000 7f9ddf7fe640 1 -- 172.21.15.26:0/482644339 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9dd8089020 con 0x7f9de806ac80 2023-12-19T16:42:56.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.228+0000 7f9deccd1640 1 -- 172.21.15.26:0/482644339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9de8151380 con 0x7f9de806ac80 2023-12-19T16:42:56.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.228+0000 7f9ddf7fe640 1 -- 172.21.15.26:0/482644339 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9dd8055150 con 0x7f9de806ac80 2023-12-19T16:42:56.233 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:42:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.232+0000 7f9deccd1640 1 -- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9dc8065a60 msgr2=0x7f9dc8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.232+0000 7f9deccd1640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9dc8065a60 0x7f9dc8067f20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9de81f94f0 tx=0x7f9dd0009040 comp rx=0 tx=0).stop 2023-12-19T16:42:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.232+0000 7f9deccd1640 1 -- 172.21.15.26:0/482644339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9de806ac80 msgr2=0x7f9de8150d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.232+0000 7f9deccd1640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9de806ac80 0x7f9de8150d00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800efc0 tx=0x7f9dd800c490 comp rx=0 tx=0).stop 2023-12-19T16:42:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.232+0000 7f9deccd1640 1 -- 172.21.15.26:0/482644339 shutdown_connections 2023-12-19T16:42:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.232+0000 7f9deccd1640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9de815ada0 0x7f9de80c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.232+0000 7f9deccd1640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9dc8065a60 0x7f9dc8067f20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.232+0000 7f9deccd1640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9de815a9c0 0x7f9de815d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.232+0000 7f9deccd1640 1 --2- 172.21.15.26:0/482644339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9de806ac80 0x7f9de8150d00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.232+0000 7f9deccd1640 1 -- 172.21.15.26:0/482644339 >> 172.21.15.26:0/482644339 conn(0x7f9de80ce730 msgr2=0x7f9de80c9650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.232+0000 7f9deccd1640 1 -- 172.21.15.26:0/482644339 shutdown_connections 2023-12-19T16:42:56.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.232+0000 7f9deccd1640 1 -- 172.21.15.26:0/482644339 wait complete. 2023-12-19T16:42:56.255 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:42:56.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.396+0000 7f5e51718640 1 Processor -- start 2023-12-19T16:42:56.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.396+0000 7f5e51718640 1 -- start start 2023-12-19T16:42:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.396+0000 7f5e51718640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e4c15ccf0 0x7f5e4c0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.396+0000 7f5e51718640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e4c0b4600 0x7f5e4c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.396+0000 7f5e51718640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4c0b4f50 con 0x7f5e4c15d0d0 2023-12-19T16:42:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.396+0000 7f5e51718640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4c15e200 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.396+0000 7f5e51718640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0d0aa0 con 0x7f5e4c0b4600 2023-12-19T16:42:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.400+0000 7f5e4affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e4c15ccf0 0x7f5e4c0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.400+0000 7f5e43fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5e4c15d0d0 0x7f5e4c0b8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48532/0 (socket says 172.21.15.26:48532) 2023-12-19T16:42:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.400+0000 7f5e43fff640 1 -- 172.21.15.26:0/2252454960 learned_addr learned my addr 172.21.15.26:0/2252454960 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.400+0000 7f5e4b7fe640 1 --2- 172.21.15.26:0/2252454960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e4c0b4600 0x7f5e4c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.400+0000 7f5e4affd640 1 -- 172.21.15.26:0/2252454960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e4c0b4600 msgr2=0x7f5e4c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.400+0000 7f5e4affd640 1 --2- 172.21.15.26:0/2252454960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e4c0b4600 0x7f5e4c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.400+0000 7f5e4affd640 1 -- 172.21.15.26:0/2252454960 >> v1:172.21.15.26:6789/0 conn(0x7f5e4c15d0d0 legacy=0x7f5e4c0b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.400+0000 7f5e4affd640 1 -- 172.21.15.26:0/2252454960 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c0d1380 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.400+0000 7f5e4a7fc640 1 -- 172.21.15.26:0/2252454960 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717329488 0 0) 0x7f5e4c0b4f50 con 0x7f5e4c15d0d0 2023-12-19T16:42:56.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.400+0000 7f5e4affd640 1 --2- 172.21.15.26:0/2252454960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e4c15ccf0 0x7f5e4c0d0e40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0d1570 tx=0x7f5e3c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb805760200cc7f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.400+0000 7f5e4a7fc640 1 -- 172.21.15.26:0/2252454960 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0ab650 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.400+0000 7f5e4a7fc640 1 -- 172.21.15.26:0/2252454960 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c0ab7b0 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.400+0000 7f5e4a7fc640 1 -- 172.21.15.26:0/2252454960 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0abb40 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 -- 172.21.15.26:0/2252454960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e4c15ccf0 msgr2=0x7f5e4c0d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 --2- 172.21.15.26:0/2252454960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e4c15ccf0 0x7f5e4c0d0e40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0d1570 tx=0x7f5e3c01c450 comp rx=0 tx=0).stop 2023-12-19T16:42:56.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 -- 172.21.15.26:0/2252454960 shutdown_connections 2023-12-19T16:42:56.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 --2- 172.21.15.26:0/2252454960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e4c0b4600 0x7f5e4c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 --2- 172.21.15.26:0/2252454960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e4c15ccf0 0x7f5e4c0d0e40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 -- 172.21.15.26:0/2252454960 >> 172.21.15.26:0/2252454960 conn(0x7f5e4c065ee0 msgr2=0x7f5e4c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:56.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 -- 172.21.15.26:0/2252454960 shutdown_connections 2023-12-19T16:42:56.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 -- 172.21.15.26:0/2252454960 wait complete. 2023-12-19T16:42:56.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 Processor -- start 2023-12-19T16:42:56.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 -- start start 2023-12-19T16:42:56.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e4c0b4600 0x7f5e4c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e4c15ccf0 0x7f5e4c1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e4c15d0d0 0x7f5e4c166bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c158860 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0c9350 con 0x7f5e4c0b4600 2023-12-19T16:42:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e51718640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c158ab0 con 0x7f5e4c15d0d0 2023-12-19T16:42:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e43fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e4c15ccf0 0x7f5e4c1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e43fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e4c15ccf0 0x7f5e4c1575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33946/0 (socket says 172.21.15.26:33946) 2023-12-19T16:42:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e43fff640 1 -- 172.21.15.26:0/3661819246 learned_addr learned my addr 172.21.15.26:0/3661819246 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e4affd640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e4c0b4600 0x7f5e4c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e4b7fe640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e4c15d0d0 0x7f5e4c166bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e43fff640 1 -- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e4c15d0d0 msgr2=0x7f5e4c166bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e43fff640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e4c15d0d0 0x7f5e4c166bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e43fff640 1 -- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e4c0b4600 msgr2=0x7f5e4c0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e43fff640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e4c0b4600 0x7f5e4c0c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e43fff640 1 -- 172.21.15.26:0/3661819246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c009420 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.404+0000 7f5e4affd640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e4c0b4600 0x7f5e4c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.408+0000 7f5e43fff640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e4c15ccf0 0x7f5e4c1575a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400c480 tx=0x7f5e3400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.408+0000 7f5e437fe640 1 -- 172.21.15.26:0/3661819246 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e340074e0 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.408+0000 7f5e437fe640 1 -- 172.21.15.26:0/3661819246 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e34007660 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.408+0000 7f5e437fe640 1 -- 172.21.15.26:0/3661819246 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34004a30 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.408+0000 7f5e51718640 1 -- 172.21.15.26:0/3661819246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c158d90 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.408+0000 7f5e51718640 1 -- 172.21.15.26:0/3661819246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c0c8fb0 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.408+0000 7f5e51718640 1 -- 172.21.15.26:0/3661819246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4c074270 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.408+0000 7f5e437fe640 1 -- 172.21.15.26:0/3661819246 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5e34004b90 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.408+0000 7f5e437fe640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5e2c065a60 0x7f5e2c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.408+0000 7f5e437fe640 1 -- 172.21.15.26:0/3661819246 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e340822a0 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.408+0000 7f5e4affd640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5e2c065a60 0x7f5e2c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.412+0000 7f5e4affd640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5e2c065a60 0x7f5e2c067f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0af000 tx=0x7f5e3c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:56.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.412+0000 7f5e437fe640 1 -- 172.21.15.26:0/3661819246 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5e34053790 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.567+0000 7f5e51718640 1 -- 172.21.15.26:0/3661819246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e4c0b25e0 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.567+0000 7f5e437fe640 1 -- 172.21.15.26:0/3661819246 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5e34053320 con 0x7f5e4c15ccf0 2023-12-19T16:42:56.574 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:42:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.571+0000 7f5e51718640 1 -- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5e2c065a60 msgr2=0x7f5e2c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.571+0000 7f5e51718640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5e2c065a60 0x7f5e2c067f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0af000 tx=0x7f5e3c0a7000 comp rx=0 tx=0).stop 2023-12-19T16:42:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.571+0000 7f5e51718640 1 -- 172.21.15.26:0/3661819246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e4c15ccf0 msgr2=0x7f5e4c1575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.571+0000 7f5e51718640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e4c15ccf0 0x7f5e4c1575a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400c480 tx=0x7f5e3400c940 comp rx=0 tx=0).stop 2023-12-19T16:42:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.571+0000 7f5e51718640 1 -- 172.21.15.26:0/3661819246 shutdown_connections 2023-12-19T16:42:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.571+0000 7f5e51718640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5e2c065a60 0x7f5e2c067f20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.571+0000 7f5e51718640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e4c15d0d0 0x7f5e4c166bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.571+0000 7f5e51718640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e4c15ccf0 0x7f5e4c1575a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.571+0000 7f5e51718640 1 --2- 172.21.15.26:0/3661819246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e4c0b4600 0x7f5e4c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.571+0000 7f5e51718640 1 -- 172.21.15.26:0/3661819246 >> 172.21.15.26:0/3661819246 conn(0x7f5e4c065ee0 msgr2=0x7f5e4c0b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.571+0000 7f5e51718640 1 -- 172.21.15.26:0/3661819246 shutdown_connections 2023-12-19T16:42:56.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.571+0000 7f5e51718640 1 -- 172.21.15.26:0/3661819246 wait complete. 2023-12-19T16:42:56.608 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-12-19T16:42:56.608 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-19T16:42:56.609 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-12-19T16:42:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbeb111640 1 Processor -- start 2023-12-19T16:42:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbeb111640 1 -- start start 2023-12-19T16:42:56.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbeb111640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbe4154220 0x7fbbe415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbeb111640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbbe406b3d0 0x7fbbe40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbeb111640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbe40cec40 0x7fbbe415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbeb111640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe40cf0d0 con 0x7fbbe40cec40 2023-12-19T16:42:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbeb111640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe406bc30 con 0x7fbbe406b3d0 2023-12-19T16:42:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbeb111640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe40cc6e0 con 0x7fbbe4154220 2023-12-19T16:42:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbe96ae640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbe40cec40 0x7fbbe415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbe96ae640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbe40cec40 0x7fbbe415d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33952/0 (socket says 172.21.15.26:33952) 2023-12-19T16:42:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbe96ae640 1 -- 172.21.15.26:0/2040115064 learned_addr learned my addr 172.21.15.26:0/2040115064 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbe3fff640 1 --2- 172.21.15.26:0/2040115064 >> v2:172.21.15.160:3300/0 conn(0x7fbbe406b3d0 0x7fbbe40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbe8ead640 1 --2- 172.21.15.26:0/2040115064 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbe4154220 0x7fbbe415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbe96ae640 1 -- 172.21.15.26:0/2040115064 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbe4154220 msgr2=0x7fbbe415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbe96ae640 1 --2- 172.21.15.26:0/2040115064 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbe4154220 0x7fbbe415aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbe96ae640 1 -- 172.21.15.26:0/2040115064 >> v2:172.21.15.160:3300/0 conn(0x7fbbe406b3d0 msgr2=0x7fbbe40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbe96ae640 1 --2- 172.21.15.26:0/2040115064 >> v2:172.21.15.160:3300/0 conn(0x7fbbe406b3d0 0x7fbbe40ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.747+0000 7fbbe96ae640 1 -- 172.21.15.26:0/2040115064 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe415afc0 con 0x7fbbe40cec40 2023-12-19T16:42:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.751+0000 7fbbe96ae640 1 --2- 172.21.15.26:0/2040115064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe40cec40 0x7fbbe415d5f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40cc430 tx=0x7fbbdc01c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d956d81843b27002 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.751+0000 7fbbe37fe640 1 -- 172.21.15.26:0/2040115064 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0a5560 con 0x7fbbe40cec40 2023-12-19T16:42:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.751+0000 7fbbe37fe640 1 -- 172.21.15.26:0/2040115064 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbdc0a56c0 con 0x7fbbe40cec40 2023-12-19T16:42:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.751+0000 7fbbe37fe640 1 -- 172.21.15.26:0/2040115064 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0a5a50 con 0x7fbbe40cec40 2023-12-19T16:42:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.751+0000 7fbbeb111640 1 -- 172.21.15.26:0/2040115064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe40cec40 msgr2=0x7fbbe415d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.751+0000 7fbbeb111640 1 --2- 172.21.15.26:0/2040115064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe40cec40 0x7fbbe415d5f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbbe40cc430 tx=0x7fbbdc01c6e0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.751+0000 7fbbeb111640 1 -- 172.21.15.26:0/2040115064 shutdown_connections 2023-12-19T16:42:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.751+0000 7fbbeb111640 1 --2- 172.21.15.26:0/2040115064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe40cec40 0x7fbbe415d5f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.751+0000 7fbbeb111640 1 --2- 172.21.15.26:0/2040115064 >> v2:172.21.15.160:3300/0 conn(0x7fbbe406b3d0 0x7fbbe40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.751+0000 7fbbeb111640 1 --2- 172.21.15.26:0/2040115064 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbe4154220 0x7fbbe415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.751+0000 7fbbeb111640 1 -- 172.21.15.26:0/2040115064 >> 172.21.15.26:0/2040115064 conn(0x7fbbe40658e0 msgr2=0x7fbbe4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.751+0000 7fbbeb111640 1 -- 172.21.15.26:0/2040115064 shutdown_connections 2023-12-19T16:42:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.751+0000 7fbbeb111640 1 -- 172.21.15.26:0/2040115064 wait complete. 2023-12-19T16:42:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbeb111640 1 Processor -- start 2023-12-19T16:42:56.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbeb111640 1 -- start start 2023-12-19T16:42:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbeb111640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbe406b3d0 0x7fbbe414d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbeb111640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe40cec40 0x7fbbe40b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbeb111640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbe4154220 0x7fbbe4158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbeb111640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe414d890 con 0x7fbbe40cec40 2023-12-19T16:42:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbeb111640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe406f3b0 con 0x7fbbe406b3d0 2023-12-19T16:42:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbeb111640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe40af2c0 con 0x7fbbe4154220 2023-12-19T16:42:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe40cec40 0x7fbbe40b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe40cec40 0x7fbbe40b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33956/0 (socket says 172.21.15.26:33956) 2023-12-19T16:42:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe3fff640 1 -- 172.21.15.26:0/1439214177 learned_addr learned my addr 172.21.15.26:0/1439214177 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:56.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe96ae640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbe4154220 0x7fbbe4158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe8ead640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbe406b3d0 0x7fbbe414d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe3fff640 1 -- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbe4154220 msgr2=0x7fbbe4158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe3fff640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbe4154220 0x7fbbe4158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe3fff640 1 -- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbe406b3d0 msgr2=0x7fbbe414d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe3fff640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbe406b3d0 0x7fbbe414d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe3fff640 1 -- 172.21.15.26:0/1439214177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc009b30 con 0x7fbbe40cec40 2023-12-19T16:42:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe3fff640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe40cec40 0x7fbbe40b1de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000e870 tx=0x7fbbd000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe17fa640 1 -- 172.21.15.26:0/1439214177 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000ee60 con 0x7fbbe40cec40 2023-12-19T16:42:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe17fa640 1 -- 172.21.15.26:0/1439214177 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd000c940 con 0x7fbbe40cec40 2023-12-19T16:42:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbeb111640 1 -- 172.21.15.26:0/1439214177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe40af5a0 con 0x7fbbe40cec40 2023-12-19T16:42:56.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbe17fa640 1 -- 172.21.15.26:0/1439214177 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000ccf0 con 0x7fbbe40cec40 2023-12-19T16:42:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.755+0000 7fbbeb111640 1 -- 172.21.15.26:0/1439214177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe40afb60 con 0x7fbbe40cec40 2023-12-19T16:42:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.759+0000 7fbbeb111640 1 -- 172.21.15.26:0/1439214177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe4074270 con 0x7fbbe40cec40 2023-12-19T16:42:56.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.759+0000 7fbbe17fa640 1 -- 172.21.15.26:0/1439214177 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbbd0013470 con 0x7fbbe40cec40 2023-12-19T16:42:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.759+0000 7fbbe17fa640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbbc8065a60 0x7fbbc8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.759+0000 7fbbe17fa640 1 -- 172.21.15.26:0/1439214177 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbd00831b0 con 0x7fbbe40cec40 2023-12-19T16:42:56.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.759+0000 7fbbe8ead640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbbc8065a60 0x7fbbc8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:56.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.759+0000 7fbbe8ead640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbbc8065a60 0x7fbbc8067f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009b60 tx=0x7fbbd8009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:56.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.763+0000 7fbbe17fa640 1 -- 172.21.15.26:0/1439214177 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbbd0050c10 con 0x7fbbe40cec40 2023-12-19T16:42:56.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:56.975+0000 7fbbeb111640 1 -- 172.21.15.26:0/1439214177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fbbe40cdf60 con 0x7fbbe40cec40 2023-12-19T16:42:57.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.395+0000 7fbbe17fa640 1 -- 172.21.15.26:0/1439214177 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbbd0054230 con 0x7fbbe40cec40 2023-12-19T16:42:57.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.395+0000 7fbbeb111640 1 -- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbbc8065a60 msgr2=0x7fbbc8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:57.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.395+0000 7fbbeb111640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbbc8065a60 0x7fbbc8067f20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009b60 tx=0x7fbbd8009190 comp rx=0 tx=0).stop 2023-12-19T16:42:57.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.395+0000 7fbbeb111640 1 -- 172.21.15.26:0/1439214177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe40cec40 msgr2=0x7fbbe40b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:57.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.395+0000 7fbbeb111640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe40cec40 0x7fbbe40b1de0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000e870 tx=0x7fbbd000ed30 comp rx=0 tx=0).stop 2023-12-19T16:42:57.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.395+0000 7fbbeb111640 1 -- 172.21.15.26:0/1439214177 shutdown_connections 2023-12-19T16:42:57.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.395+0000 7fbbeb111640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbbc8065a60 0x7fbbc8067f20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.395+0000 7fbbeb111640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbe4154220 0x7fbbe4158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.395+0000 7fbbeb111640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbe40cec40 0x7fbbe40b1de0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.395+0000 7fbbeb111640 1 --2- 172.21.15.26:0/1439214177 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbe406b3d0 0x7fbbe414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.395+0000 7fbbeb111640 1 -- 172.21.15.26:0/1439214177 >> 172.21.15.26:0/1439214177 conn(0x7fbbe40658e0 msgr2=0x7fbbe40cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:57.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.395+0000 7fbbeb111640 1 -- 172.21.15.26:0/1439214177 shutdown_connections 2023-12-19T16:42:57.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.395+0000 7fbbeb111640 1 -- 172.21.15.26:0/1439214177 wait complete. 2023-12-19T16:42:57.422 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T16:42:57.463 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:42:57.473 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:42:57.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141fef9640 1 Processor -- start 2023-12-19T16:42:57.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141fef9640 1 -- start start 2023-12-19T16:42:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141fef9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f141815cdb0 0x7f141806d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141fef9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14180658e0 0x7f1418065d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141fef9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14180cd940 con 0x7f14180ca060 2023-12-19T16:42:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141fef9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14180662d0 con 0x7f141815cdb0 2023-12-19T16:42:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141fef9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141815d190 con 0x7f14180658e0 2023-12-19T16:42:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141dc95640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f141815cdb0 0x7f141806d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141dc95640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f141815cdb0 0x7f141806d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33968/0 (socket says 172.21.15.26:33968) 2023-12-19T16:42:57.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141dc95640 1 -- 172.21.15.26:0/3914025548 learned_addr learned my addr 172.21.15.26:0/3914025548 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:57.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141e496640 1 --2- 172.21.15.26:0/3914025548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14180658e0 0x7f1418065d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:57.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141dc95640 1 -- 172.21.15.26:0/3914025548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14180658e0 msgr2=0x7f1418065d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:57.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141dc95640 1 --2- 172.21.15.26:0/3914025548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14180658e0 0x7f1418065d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141dc95640 1 -- 172.21.15.26:0/3914025548 >> v1:172.21.15.160:6789/0 conn(0x7f14180ca060 legacy=0x7f1418161280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:57.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141dc95640 1 -- 172.21.15.26:0/3914025548 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14180cdf50 con 0x7f141815cdb0 2023-12-19T16:42:57.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.567+0000 7f141cc93640 1 -- 172.21.15.26:0/3914025548 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400471819 0 0) 0x7f14180cd940 con 0x7f14180ca060 2023-12-19T16:42:57.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.571+0000 7f141dc95640 1 --2- 172.21.15.26:0/3914025548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f141815cdb0 0x7f141806d610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f14140098a0 tx=0x7f141401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c34bcf503f1d5f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:57.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.571+0000 7f141cc93640 1 -- 172.21.15.26:0/3914025548 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14140ac630 con 0x7f141815cdb0 2023-12-19T16:42:57.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.571+0000 7f141cc93640 1 -- 172.21.15.26:0/3914025548 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14140ac790 con 0x7f141815cdb0 2023-12-19T16:42:57.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.571+0000 7f141cc93640 1 -- 172.21.15.26:0/3914025548 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14140acb20 con 0x7f141815cdb0 2023-12-19T16:42:57.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.571+0000 7f141fef9640 1 -- 172.21.15.26:0/3914025548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f141815cdb0 msgr2=0x7f141806d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:57.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.571+0000 7f141fef9640 1 --2- 172.21.15.26:0/3914025548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f141815cdb0 0x7f141806d610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f14140098a0 tx=0x7f141401c3d0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.571+0000 7f141fef9640 1 -- 172.21.15.26:0/3914025548 shutdown_connections 2023-12-19T16:42:57.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.571+0000 7f141fef9640 1 --2- 172.21.15.26:0/3914025548 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14180658e0 0x7f1418065d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.571+0000 7f141fef9640 1 --2- 172.21.15.26:0/3914025548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f141815cdb0 0x7f141806d610 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.571+0000 7f141fef9640 1 -- 172.21.15.26:0/3914025548 >> 172.21.15.26:0/3914025548 conn(0x7f1418069d20 msgr2=0x7f141806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:57.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.571+0000 7f141fef9640 1 -- 172.21.15.26:0/3914025548 shutdown_connections 2023-12-19T16:42:57.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.571+0000 7f141fef9640 1 -- 172.21.15.26:0/3914025548 wait complete. 2023-12-19T16:42:57.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141fef9640 1 Processor -- start 2023-12-19T16:42:57.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141fef9640 1 -- start start 2023-12-19T16:42:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141fef9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14180658e0 0x7f1418160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141fef9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14180ca060 0x7f141814c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141fef9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f141815cdb0 0x7f141814f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:57.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141fef9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1418160900 con 0x7f14180ca060 2023-12-19T16:42:57.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141fef9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14180c73b0 con 0x7f141815cdb0 2023-12-19T16:42:57.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141fef9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1418151930 con 0x7f14180658e0 2023-12-19T16:42:57.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141d494640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14180ca060 0x7f141814c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:57.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141d494640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14180ca060 0x7f141814c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33984/0 (socket says 172.21.15.26:33984) 2023-12-19T16:42:57.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141d494640 1 -- 172.21.15.26:0/1654228189 learned_addr learned my addr 172.21.15.26:0/1654228189 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:57.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141e496640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f141815cdb0 0x7f141814f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:57.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141dc95640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14180658e0 0x7f1418160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:57.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141d494640 1 -- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14180658e0 msgr2=0x7f1418160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:57.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141d494640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14180658e0 0x7f1418160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141d494640 1 -- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f141815cdb0 msgr2=0x7f141814f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:57.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141d494640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f141815cdb0 0x7f141814f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141d494640 1 -- 172.21.15.26:0/1654228189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14140094d0 con 0x7f14180ca060 2023-12-19T16:42:57.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141d494640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14180ca060 0x7f141814c530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f140800c520 tx=0x7f140800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:57.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f140e7fc640 1 -- 172.21.15.26:0/1654228189 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408003600 con 0x7f14180ca060 2023-12-19T16:42:57.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f140e7fc640 1 -- 172.21.15.26:0/1654228189 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1408003780 con 0x7f14180ca060 2023-12-19T16:42:57.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f140e7fc640 1 -- 172.21.15.26:0/1654228189 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1408003b30 con 0x7f14180ca060 2023-12-19T16:42:57.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141fef9640 1 -- 172.21.15.26:0/1654228189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141814cea0 con 0x7f14180ca060 2023-12-19T16:42:57.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.575+0000 7f141fef9640 1 -- 172.21.15.26:0/1654228189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141814d460 con 0x7f14180ca060 2023-12-19T16:42:57.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.579+0000 7f141fef9640 1 -- 172.21.15.26:0/1654228189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14180c7560 con 0x7f14180ca060 2023-12-19T16:42:57.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.579+0000 7f140e7fc640 1 -- 172.21.15.26:0/1654228189 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1408003c90 con 0x7f14180ca060 2023-12-19T16:42:57.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.579+0000 7f140e7fc640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f13fc065a60 0x7f13fc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:57.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.579+0000 7f140e7fc640 1 -- 172.21.15.26:0/1654228189 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1408082700 con 0x7f14180ca060 2023-12-19T16:42:57.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.579+0000 7f141dc95640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f13fc065a60 0x7f13fc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:57.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.579+0000 7f141dc95640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f13fc065a60 0x7f13fc067f20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f14140ae000 tx=0x7f14140a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:57.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.583+0000 7f140e7fc640 1 -- 172.21.15.26:0/1654228189 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f14180c7560 con 0x7f14180ca060 2023-12-19T16:42:57.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.739+0000 7f141fef9640 1 -- 172.21.15.26:0/1654228189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f141814d680 con 0x7f14180ca060 2023-12-19T16:42:57.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.747+0000 7f140e7fc640 1 -- 172.21.15.26:0/1654228189 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f14080170c0 con 0x7f14180ca060 2023-12-19T16:42:57.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.747+0000 7f141fef9640 1 -- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f13fc065a60 msgr2=0x7f13fc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:57.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.747+0000 7f141fef9640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f13fc065a60 0x7f13fc067f20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f14140ae000 tx=0x7f14140a7000 comp rx=0 tx=0).stop 2023-12-19T16:42:57.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.747+0000 7f141fef9640 1 -- 172.21.15.26:0/1654228189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14180ca060 msgr2=0x7f141814c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:57.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.747+0000 7f141fef9640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14180ca060 0x7f141814c530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f140800c520 tx=0x7f140800c9e0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.747+0000 7f141fef9640 1 -- 172.21.15.26:0/1654228189 shutdown_connections 2023-12-19T16:42:57.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.747+0000 7f141fef9640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f141815cdb0 0x7f141814f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.747+0000 7f141fef9640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14180ca060 0x7f141814c530 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.747+0000 7f141fef9640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f13fc065a60 0x7f13fc067f20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.747+0000 7f141fef9640 1 --2- 172.21.15.26:0/1654228189 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f14180658e0 0x7f1418160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:57.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.747+0000 7f141fef9640 1 -- 172.21.15.26:0/1654228189 >> 172.21.15.26:0/1654228189 conn(0x7f1418069d20 msgr2=0x7f14180cad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:57.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.747+0000 7f141fef9640 1 -- 172.21.15.26:0/1654228189 shutdown_connections 2023-12-19T16:42:57.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:57.747+0000 7f141fef9640 1 -- 172.21.15.26:0/1654228189 wait complete. 2023-12-19T16:42:57.772 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T16:42:57.772 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:42:57.849 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:42:57.930 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T16:42:57.930 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:42:57.930 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:42:57.940 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:42:57.940 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:42:57.991 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:42:57.991 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:42:58.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:42:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07f3c0640 1 Processor -- start 2023-12-19T16:42:58.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07f3c0640 1 -- start start 2023-12-19T16:42:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07f3c0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff07815cd40 0x7ff0780ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07f3c0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff078160900 0x7ff078160d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07f3c0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0780d29d0 con 0x7ff0780d2bf0 2023-12-19T16:42:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07f3c0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff078161280 con 0x7ff07815cd40 2023-12-19T16:42:58.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07f3c0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07815d680 con 0x7ff078160900 2023-12-19T16:42:58.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07c95b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff078160900 0x7ff078160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07d15c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff07815cd40 0x7ff0780ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07c95b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff078160900 0x7ff078160d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:57340/0 (socket says 172.21.15.26:57340) 2023-12-19T16:42:58.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07c95b640 1 -- 172.21.15.26:0/3430610417 learned_addr learned my addr 172.21.15.26:0/3430610417 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:58.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07d15c640 1 -- 172.21.15.26:0/3430610417 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff078160900 msgr2=0x7ff078160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:58.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07d15c640 1 --2- 172.21.15.26:0/3430610417 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff078160900 0x7ff078160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07d15c640 1 -- 172.21.15.26:0/3430610417 >> v1:172.21.15.160:6789/0 conn(0x7ff0780d2bf0 legacy=0x7ff0780d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.227+0000 7ff07d15c640 1 -- 172.21.15.26:0/3430610417 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff078161750 con 0x7ff07815cd40 2023-12-19T16:42:58.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.231+0000 7ff07d15c640 1 --2- 172.21.15.26:0/3430610417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff07815cd40 0x7ff0780ce040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff0740097a0 tx=0x7ff07401c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=619158119418232 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:58.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.231+0000 7ff06ffff640 1 -- 172.21.15.26:0/3430610417 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074004230 con 0x7ff07815cd40 2023-12-19T16:42:58.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.231+0000 7ff06ffff640 1 -- 172.21.15.26:0/3430610417 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0740a7050 con 0x7ff07815cd40 2023-12-19T16:42:58.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.231+0000 7ff06ffff640 1 -- 172.21.15.26:0/3430610417 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074003620 con 0x7ff07815cd40 2023-12-19T16:42:58.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.231+0000 7ff07f3c0640 1 -- 172.21.15.26:0/3430610417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff07815cd40 msgr2=0x7ff0780ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.231+0000 7ff07f3c0640 1 --2- 172.21.15.26:0/3430610417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff07815cd40 0x7ff0780ce040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff0740097a0 tx=0x7ff07401c460 comp rx=0 tx=0).stop 2023-12-19T16:42:58.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.231+0000 7ff07f3c0640 1 -- 172.21.15.26:0/3430610417 shutdown_connections 2023-12-19T16:42:58.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.231+0000 7ff07f3c0640 1 --2- 172.21.15.26:0/3430610417 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff078160900 0x7ff078160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.231+0000 7ff07f3c0640 1 --2- 172.21.15.26:0/3430610417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff07815cd40 0x7ff0780ce040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.231+0000 7ff07f3c0640 1 -- 172.21.15.26:0/3430610417 >> 172.21.15.26:0/3430610417 conn(0x7ff07806a120 msgr2=0x7ff07806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:58.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.231+0000 7ff07f3c0640 1 -- 172.21.15.26:0/3430610417 shutdown_connections 2023-12-19T16:42:58.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07f3c0640 1 -- 172.21.15.26:0/3430610417 wait complete. 2023-12-19T16:42:58.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07f3c0640 1 Processor -- start 2023-12-19T16:42:58.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07f3c0640 1 -- start start 2023-12-19T16:42:58.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07f3c0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0780d2bf0 0x7ff0781565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07f3c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff07815cd40 0x7ff0780c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07f3c0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff078160900 0x7ff078155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07f3c0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0780c9190 con 0x7ff07815cd40 2023-12-19T16:42:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07f3c0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0781549d0 con 0x7ff0780d2bf0 2023-12-19T16:42:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07f3c0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff078154cb0 con 0x7ff078160900 2023-12-19T16:42:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07c95b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff07815cd40 0x7ff0780c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07c95b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff07815cd40 0x7ff0780c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34000/0 (socket says 172.21.15.26:34000) 2023-12-19T16:42:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07c95b640 1 -- 172.21.15.26:0/1899337163 learned_addr learned my addr 172.21.15.26:0/1899337163 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07d15c640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0780d2bf0 0x7ff0781565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07d95d640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff078160900 0x7ff078155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07c95b640 1 -- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff078160900 msgr2=0x7ff078155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07c95b640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff078160900 0x7ff078155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07c95b640 1 -- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0780d2bf0 msgr2=0x7ff0781565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07c95b640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0780d2bf0 0x7ff0781565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07c95b640 1 -- 172.21.15.26:0/1899337163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0740093b0 con 0x7ff07815cd40 2023-12-19T16:42:58.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff07c95b640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff07815cd40 0x7ff0780c8c50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e7f0 tx=0x7ff06800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:58.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.235+0000 7ff06dffb640 1 -- 172.21.15.26:0/1899337163 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068009580 con 0x7ff07815cd40 2023-12-19T16:42:58.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.239+0000 7ff06dffb640 1 -- 172.21.15.26:0/1899337163 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06800c940 con 0x7ff07815cd40 2023-12-19T16:42:58.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.239+0000 7ff06dffb640 1 -- 172.21.15.26:0/1899337163 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06800ccf0 con 0x7ff07815cd40 2023-12-19T16:42:58.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.239+0000 7ff07f3c0640 1 -- 172.21.15.26:0/1899337163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff078154f90 con 0x7ff07815cd40 2023-12-19T16:42:58.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.239+0000 7ff07f3c0640 1 -- 172.21.15.26:0/1899337163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff078155550 con 0x7ff07815cd40 2023-12-19T16:42:58.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.239+0000 7ff07f3c0640 1 -- 172.21.15.26:0/1899337163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff078074270 con 0x7ff07815cd40 2023-12-19T16:42:58.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.239+0000 7ff06dffb640 1 -- 172.21.15.26:0/1899337163 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff068013470 con 0x7ff07815cd40 2023-12-19T16:42:58.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.239+0000 7ff06dffb640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff05c065a90 0x7ff05c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.239+0000 7ff06dffb640 1 -- 172.21.15.26:0/1899337163 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff068083180 con 0x7ff07815cd40 2023-12-19T16:42:58.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.239+0000 7ff07d15c640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff05c065a90 0x7ff05c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.243+0000 7ff07d15c640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff05c065a90 0x7ff05c067f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff0740ad000 tx=0x7ff0740a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:58.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.243+0000 7ff06dffb640 1 -- 172.21.15.26:0/1899337163 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff068050be0 con 0x7ff07815cd40 2023-12-19T16:42:58.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.403+0000 7ff07f3c0640 1 -- 172.21.15.26:0/1899337163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff07815f7c0 con 0x7ff07815cd40 2023-12-19T16:42:58.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.403+0000 7ff06dffb640 1 -- 172.21.15.26:0/1899337163 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7ff068054200 con 0x7ff07815cd40 2023-12-19T16:42:58.409 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:42:58.410 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:42:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.407+0000 7ff07f3c0640 1 -- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff05c065a90 msgr2=0x7ff05c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.407+0000 7ff07f3c0640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff05c065a90 0x7ff05c067f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff0740ad000 tx=0x7ff0740a6000 comp rx=0 tx=0).stop 2023-12-19T16:42:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.407+0000 7ff07f3c0640 1 -- 172.21.15.26:0/1899337163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff07815cd40 msgr2=0x7ff0780c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.407+0000 7ff07f3c0640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff07815cd40 0x7ff0780c8c50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff06800e7f0 tx=0x7ff06800ecb0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.407+0000 7ff07f3c0640 1 -- 172.21.15.26:0/1899337163 shutdown_connections 2023-12-19T16:42:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.407+0000 7ff07f3c0640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff05c065a90 0x7ff05c067f50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.407+0000 7ff07f3c0640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff078160900 0x7ff078155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.407+0000 7ff07f3c0640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff07815cd40 0x7ff0780c8c50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.407+0000 7ff07f3c0640 1 --2- 172.21.15.26:0/1899337163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0780d2bf0 0x7ff0781565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.407+0000 7ff07f3c0640 1 -- 172.21.15.26:0/1899337163 >> 172.21.15.26:0/1899337163 conn(0x7ff07806a120 msgr2=0x7ff07815ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.407+0000 7ff07f3c0640 1 -- 172.21.15.26:0/1899337163 shutdown_connections 2023-12-19T16:42:58.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.407+0000 7ff07f3c0640 1 -- 172.21.15.26:0/1899337163 wait complete. 2023-12-19T16:42:58.432 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:42:25.598550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:42:26.697362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:42:58.432 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:42:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.571+0000 7fa51cc2d640 1 Processor -- start 2023-12-19T16:42:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.571+0000 7fa51cc2d640 1 -- start start 2023-12-19T16:42:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa51cc2d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa51815a9c0 0x7fa5180706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa51cc2d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa51815ada0 0x7fa5180b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa51cc2d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5180b46c0 con 0x7fa51806ac80 2023-12-19T16:42:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa51cc2d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5180cc270 con 0x7fa51815ada0 2023-12-19T16:42:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa51cc2d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa518070d90 con 0x7fa51815a9c0 2023-12-19T16:42:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa516d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa51806ac80 0x7fa5180b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48536/0 (socket says 172.21.15.26:48536) 2023-12-19T16:42:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa50ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa51815ada0 0x7fa5180b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa516d9d640 1 -- 172.21.15.26:0/182287032 learned_addr learned my addr 172.21.15.26:0/182287032 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa51659c640 1 --2- 172.21.15.26:0/182287032 >> v2:172.21.15.160:3300/0 conn(0x7fa51815a9c0 0x7fa5180706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa50ffff640 1 -- 172.21.15.26:0/182287032 >> v2:172.21.15.160:3300/0 conn(0x7fa51815a9c0 msgr2=0x7fa5180706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa50ffff640 1 --2- 172.21.15.26:0/182287032 >> v2:172.21.15.160:3300/0 conn(0x7fa51815a9c0 0x7fa5180706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa50ffff640 1 -- 172.21.15.26:0/182287032 >> v1:172.21.15.26:6789/0 conn(0x7fa51806ac80 legacy=0x7fa5180b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa50ffff640 1 -- 172.21.15.26:0/182287032 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa518071070 con 0x7fa51815ada0 2023-12-19T16:42:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.575+0000 7fa515d9b640 1 -- 172.21.15.26:0/182287032 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019805098 0 0) 0x7fa5180b46c0 con 0x7fa51806ac80 2023-12-19T16:42:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa50ffff640 1 --2- 172.21.15.26:0/182287032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51815ada0 0x7fa5180b4180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa5180cf590 tx=0x7fa50001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9234b7163cefd185 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa515d9b640 1 -- 172.21.15.26:0/182287032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5000ac600 con 0x7fa51815ada0 2023-12-19T16:42:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa515d9b640 1 -- 172.21.15.26:0/182287032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5000a8050 con 0x7fa51815ada0 2023-12-19T16:42:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa515d9b640 1 -- 172.21.15.26:0/182287032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5000ac980 con 0x7fa51815ada0 2023-12-19T16:42:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa51cc2d640 1 -- 172.21.15.26:0/182287032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51815ada0 msgr2=0x7fa5180b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa51cc2d640 1 --2- 172.21.15.26:0/182287032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51815ada0 0x7fa5180b4180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa5180cf590 tx=0x7fa50001c3a0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa51cc2d640 1 -- 172.21.15.26:0/182287032 shutdown_connections 2023-12-19T16:42:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa51cc2d640 1 --2- 172.21.15.26:0/182287032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51815ada0 0x7fa5180b4180 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa51cc2d640 1 --2- 172.21.15.26:0/182287032 >> v2:172.21.15.160:3300/0 conn(0x7fa51815a9c0 0x7fa5180706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa51cc2d640 1 -- 172.21.15.26:0/182287032 >> 172.21.15.26:0/182287032 conn(0x7fa5180ce730 msgr2=0x7fa5180ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa51cc2d640 1 -- 172.21.15.26:0/182287032 shutdown_connections 2023-12-19T16:42:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa51cc2d640 1 -- 172.21.15.26:0/182287032 wait complete. 2023-12-19T16:42:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa51cc2d640 1 Processor -- start 2023-12-19T16:42:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.579+0000 7fa51cc2d640 1 -- start start 2023-12-19T16:42:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51cc2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51806ac80 0x7fa51815d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51cc2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa51815a9c0 0x7fa5180c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51cc2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa51815ada0 0x7fa518158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51cc2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5180c8600 con 0x7fa51806ac80 2023-12-19T16:42:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51cc2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51814add0 con 0x7fa51815ada0 2023-12-19T16:42:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51cc2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51814b0b0 con 0x7fa51815a9c0 2023-12-19T16:42:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51806ac80 0x7fa51815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51806ac80 0x7fa51815d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34020/0 (socket says 172.21.15.26:34020) 2023-12-19T16:42:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51659c640 1 -- 172.21.15.26:0/3701117515 learned_addr learned my addr 172.21.15.26:0/3701117515 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa516d9d640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa51815ada0 0x7fa518158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51659c640 1 -- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa51815a9c0 msgr2=0x7fa5180c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51659c640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa51815a9c0 0x7fa5180c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51659c640 1 -- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa51815ada0 msgr2=0x7fa518158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51659c640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa51815ada0 0x7fa518158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51659c640 1 -- 172.21.15.26:0/3701117515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa500009420 con 0x7fa51806ac80 2023-12-19T16:42:58.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa516d9d640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa51815ada0 0x7fa518158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:58.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51659c640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51806ac80 0x7fa51815d570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa50800d800 tx=0x7fa50800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:58.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa50effd640 1 -- 172.21.15.26:0/3701117515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50800b6c0 con 0x7fa51806ac80 2023-12-19T16:42:58.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa50effd640 1 -- 172.21.15.26:0/3701117515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50800b820 con 0x7fa51806ac80 2023-12-19T16:42:58.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51cc2d640 1 -- 172.21.15.26:0/3701117515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51814b390 con 0x7fa51806ac80 2023-12-19T16:42:58.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa50effd640 1 -- 172.21.15.26:0/3701117515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50800bb90 con 0x7fa51806ac80 2023-12-19T16:42:58.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51cc2d640 1 -- 172.21.15.26:0/3701117515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51814b950 con 0x7fa51806ac80 2023-12-19T16:42:58.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa51cc2d640 1 -- 172.21.15.26:0/3701117515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa518074270 con 0x7fa51806ac80 2023-12-19T16:42:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.583+0000 7fa50effd640 1 -- 172.21.15.26:0/3701117515 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa50800bcf0 con 0x7fa51806ac80 2023-12-19T16:42:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.587+0000 7fa50effd640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4f8065a90 0x7fa4f8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.587+0000 7fa50effd640 1 -- 172.21.15.26:0/3701117515 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa508083780 con 0x7fa51806ac80 2023-12-19T16:42:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.587+0000 7fa50ffff640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4f8065a90 0x7fa4f8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.587+0000 7fa50ffff640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4f8065a90 0x7fa4f8067f50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa5000ae000 tx=0x7fa500008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.591+0000 7fa50effd640 1 -- 172.21.15.26:0/3701117515 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa5080510a0 con 0x7fa51806ac80 2023-12-19T16:42:58.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.747+0000 7fa51cc2d640 1 -- 172.21.15.26:0/3701117515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5180cd7b0 con 0x7fa51806ac80 2023-12-19T16:42:58.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.747+0000 7fa50effd640 1 -- 172.21.15.26:0/3701117515 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5080546c0 con 0x7fa51806ac80 2023-12-19T16:42:58.752 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:42:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.747+0000 7fa51cc2d640 1 -- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4f8065a90 msgr2=0x7fa4f8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.747+0000 7fa51cc2d640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4f8065a90 0x7fa4f8067f50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa5000ae000 tx=0x7fa500008040 comp rx=0 tx=0).stop 2023-12-19T16:42:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.747+0000 7fa51cc2d640 1 -- 172.21.15.26:0/3701117515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51806ac80 msgr2=0x7fa51815d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.747+0000 7fa51cc2d640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51806ac80 0x7fa51815d570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa50800d800 tx=0x7fa50800dcc0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.747+0000 7fa51cc2d640 1 -- 172.21.15.26:0/3701117515 shutdown_connections 2023-12-19T16:42:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.747+0000 7fa51cc2d640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa51815ada0 0x7fa518158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.747+0000 7fa51cc2d640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4f8065a90 0x7fa4f8067f50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.747+0000 7fa51cc2d640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa51815a9c0 0x7fa5180c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.747+0000 7fa51cc2d640 1 --2- 172.21.15.26:0/3701117515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51806ac80 0x7fa51815d570 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.747+0000 7fa51cc2d640 1 -- 172.21.15.26:0/3701117515 >> 172.21.15.26:0/3701117515 conn(0x7fa5180ce730 msgr2=0x7fa518159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.751+0000 7fa51cc2d640 1 -- 172.21.15.26:0/3701117515 shutdown_connections 2023-12-19T16:42:58.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.751+0000 7fa51cc2d640 1 -- 172.21.15.26:0/3701117515 wait complete. 2023-12-19T16:42:58.774 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:42:58.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb6a1398640 1 Processor -- start 2023-12-19T16:42:58.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb6a1398640 1 -- start start 2023-12-19T16:42:58.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb6a1398640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb69c06ac80 0x7fb69c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb6a1398640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb69c06f920 0x7fb69c06fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb6a1398640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69c06f490 con 0x7fb69c159140 2023-12-19T16:42:58.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb6a1398640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c06b650 con 0x7fb69c06ac80 2023-12-19T16:42:58.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb6a1398640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c159520 con 0x7fb69c06f920 2023-12-19T16:42:58.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb69affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb69c159140 0x7fb69c0ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48546/0 (socket says 172.21.15.26:48546) 2023-12-19T16:42:58.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb69affd640 1 -- 172.21.15.26:0/2191135714 learned_addr learned my addr 172.21.15.26:0/2191135714 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:58.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb69a7fc640 1 --2- 172.21.15.26:0/2191135714 >> v2:172.21.15.26:3300/0 conn(0x7fb69c06ac80 0x7fb69c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb699ffb640 1 -- 172.21.15.26:0/2191135714 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352606215 0 0) 0x7fb69c06f490 con 0x7fb69c159140 2023-12-19T16:42:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb69b7fe640 1 --2- 172.21.15.26:0/2191135714 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb69c06f920 0x7fb69c06fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb699ffb640 1 -- 172.21.15.26:0/2191135714 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb684005740 con 0x7fb69c159140 2023-12-19T16:42:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.915+0000 7fb69a7fc640 1 -- 172.21.15.26:0/2191135714 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb69c06f920 msgr2=0x7fb69c06fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb69a7fc640 1 --2- 172.21.15.26:0/2191135714 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb69c06f920 0x7fb69c06fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb69a7fc640 1 -- 172.21.15.26:0/2191135714 >> v1:172.21.15.26:6789/0 conn(0x7fb69c159140 legacy=0x7fb69c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb69a7fc640 1 -- 172.21.15.26:0/2191135714 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c0cd690 con 0x7fb69c06ac80 2023-12-19T16:42:58.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb69a7fc640 1 --2- 172.21.15.26:0/2191135714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb69c06ac80 0x7fb69c06b110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6900097c0 tx=0x7fb69001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f358f1aadb6f2f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:58.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb699ffb640 1 -- 172.21.15.26:0/2191135714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6900ac630 con 0x7fb69c06ac80 2023-12-19T16:42:58.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb699ffb640 1 -- 172.21.15.26:0/2191135714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6900ac790 con 0x7fb69c06ac80 2023-12-19T16:42:58.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb699ffb640 1 -- 172.21.15.26:0/2191135714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6900acb20 con 0x7fb69c06ac80 2023-12-19T16:42:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb6a1398640 1 -- 172.21.15.26:0/2191135714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb69c06ac80 msgr2=0x7fb69c06b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb6a1398640 1 --2- 172.21.15.26:0/2191135714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb69c06ac80 0x7fb69c06b110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb6900097c0 tx=0x7fb69001c450 comp rx=0 tx=0).stop 2023-12-19T16:42:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb6a1398640 1 -- 172.21.15.26:0/2191135714 shutdown_connections 2023-12-19T16:42:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb6a1398640 1 --2- 172.21.15.26:0/2191135714 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb69c06f920 0x7fb69c06fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb6a1398640 1 --2- 172.21.15.26:0/2191135714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb69c06ac80 0x7fb69c06b110 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb6a1398640 1 -- 172.21.15.26:0/2191135714 >> 172.21.15.26:0/2191135714 conn(0x7fb69c0ce730 msgr2=0x7fb69c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb6a1398640 1 -- 172.21.15.26:0/2191135714 shutdown_connections 2023-12-19T16:42:58.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.919+0000 7fb6a1398640 1 -- 172.21.15.26:0/2191135714 wait complete. 2023-12-19T16:42:58.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb6a1398640 1 Processor -- start 2023-12-19T16:42:58.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb6a1398640 1 -- start start 2023-12-19T16:42:58.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb6a1398640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c06ac80 0x7fb69c1f9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb6a1398640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb69c06f920 0x7fb69c0c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb6a1398640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb69c159140 0x7fb69c154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb6a1398640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c1f93f0 con 0x7fb69c06f920 2023-12-19T16:42:58.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb6a1398640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c1550e0 con 0x7fb69c06ac80 2023-12-19T16:42:58.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb6a1398640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c0c65d0 con 0x7fb69c159140 2023-12-19T16:42:58.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb69a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb69c06f920 0x7fb69c0c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb69a7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb69c06f920 0x7fb69c0c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34040/0 (socket says 172.21.15.26:34040) 2023-12-19T16:42:58.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb69a7fc640 1 -- 172.21.15.26:0/1227372098 learned_addr learned my addr 172.21.15.26:0/1227372098 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb69affd640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c06ac80 0x7fb69c1f9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb69b7fe640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb69c159140 0x7fb69c154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb69a7fc640 1 -- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb69c159140 msgr2=0x7fb69c154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb69a7fc640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb69c159140 0x7fb69c154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb69a7fc640 1 -- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c06ac80 msgr2=0x7fb69c1f9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb69a7fc640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c06ac80 0x7fb69c1f9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb69a7fc640 1 -- 172.21.15.26:0/1227372098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb690009420 con 0x7fb69c06f920 2023-12-19T16:42:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb69a7fc640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb69c06f920 0x7fb69c0c5f60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb69001a080 tx=0x7fb6900191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb6837fe640 1 -- 172.21.15.26:0/1227372098 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6900b5610 con 0x7fb69c06f920 2023-12-19T16:42:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb6837fe640 1 -- 172.21.15.26:0/1227372098 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6900b5770 con 0x7fb69c06f920 2023-12-19T16:42:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb6837fe640 1 -- 172.21.15.26:0/1227372098 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6900b5b20 con 0x7fb69c06f920 2023-12-19T16:42:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.923+0000 7fb6a1398640 1 -- 172.21.15.26:0/1227372098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c0c68b0 con 0x7fb69c06f920 2023-12-19T16:42:58.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.927+0000 7fb6a1398640 1 -- 172.21.15.26:0/1227372098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c1f9080 con 0x7fb69c06f920 2023-12-19T16:42:58.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.927+0000 7fb6a1398640 1 -- 172.21.15.26:0/1227372098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c1f95a0 con 0x7fb69c06f920 2023-12-19T16:42:58.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.927+0000 7fb6837fe640 1 -- 172.21.15.26:0/1227372098 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb6900b3040 con 0x7fb69c06f920 2023-12-19T16:42:58.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.927+0000 7fb6837fe640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb688065a60 0x7fb688067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:58.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.927+0000 7fb6837fe640 1 -- 172.21.15.26:0/1227372098 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb69011d3c0 con 0x7fb69c06f920 2023-12-19T16:42:58.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.927+0000 7fb69affd640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb688065a60 0x7fb688067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:58.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.927+0000 7fb69affd640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb688065a60 0x7fb688067f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0011f0 tx=0x7fb68c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:58.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:58.931+0000 7fb6837fe640 1 -- 172.21.15.26:0/1227372098 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb69c1f95a0 con 0x7fb69c06f920 2023-12-19T16:42:59.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.087+0000 7fb6a1398640 1 -- 172.21.15.26:0/1227372098 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb69c1f95a0 con 0x7fb69c06f920 2023-12-19T16:42:59.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.087+0000 7fb6837fe640 1 -- 172.21.15.26:0/1227372098 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb69c1f95a0 con 0x7fb69c06f920 2023-12-19T16:42:59.094 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:42:59.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.091+0000 7fb6a1398640 1 -- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb688065a60 msgr2=0x7fb688067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.091+0000 7fb6a1398640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb688065a60 0x7fb688067f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb68c0011f0 tx=0x7fb68c009040 comp rx=0 tx=0).stop 2023-12-19T16:42:59.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.091+0000 7fb6a1398640 1 -- 172.21.15.26:0/1227372098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb69c06f920 msgr2=0x7fb69c0c5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.091+0000 7fb6a1398640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb69c06f920 0x7fb69c0c5f60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb69001a080 tx=0x7fb6900191f0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.091+0000 7fb6a1398640 1 -- 172.21.15.26:0/1227372098 shutdown_connections 2023-12-19T16:42:59.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.091+0000 7fb6a1398640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb688065a60 0x7fb688067f20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.091+0000 7fb6a1398640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb69c159140 0x7fb69c154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.091+0000 7fb6a1398640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb69c06f920 0x7fb69c0c5f60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.091+0000 7fb6a1398640 1 --2- 172.21.15.26:0/1227372098 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb69c06ac80 0x7fb69c1f9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.091+0000 7fb6a1398640 1 -- 172.21.15.26:0/1227372098 >> 172.21.15.26:0/1227372098 conn(0x7fb69c0ce730 msgr2=0x7fb69c0c8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:59.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.091+0000 7fb6a1398640 1 -- 172.21.15.26:0/1227372098 shutdown_connections 2023-12-19T16:42:59.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.091+0000 7fb6a1398640 1 -- 172.21.15.26:0/1227372098 wait complete. 2023-12-19T16:42:59.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:42:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.255+0000 7f3463666640 1 Processor -- start 2023-12-19T16:42:59.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.255+0000 7f3463666640 1 -- start start 2023-12-19T16:42:59.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3463666640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f345c06f6c0 0x7f345c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3463666640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f345c159c00 0x7f345c070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3463666640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345c06fb60 con 0x7f345c159820 2023-12-19T16:42:59.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3463666640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345c15e470 con 0x7f345c06f6c0 2023-12-19T16:42:59.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3463666640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345c06fe40 con 0x7f345c159c00 2023-12-19T16:42:59.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3461402640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f345c159820 0x7f345c0c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48554/0 (socket says 172.21.15.26:48554) 2023-12-19T16:42:59.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3461402640 1 -- 172.21.15.26:0/3629611665 learned_addr learned my addr 172.21.15.26:0/3629611665 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:59.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3460c01640 1 --2- 172.21.15.26:0/3629611665 >> v2:172.21.15.160:3300/0 conn(0x7f345c06f6c0 0x7f345c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3461c03640 1 --2- 172.21.15.26:0/3629611665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f345c159c00 0x7f345c070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484484039 0 0) 0x7f345c06fb60 con 0x7f345c159820 2023-12-19T16:42:59.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3448003530 con 0x7f345c159820 2023-12-19T16:42:59.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1951356914 0 0) 0x7f3448003530 con 0x7f345c159820 2023-12-19T16:42:59.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f345c159c00 msgr2=0x7f345c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:59.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 --2- 172.21.15.26:0/3629611665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f345c159c00 0x7f345c070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 >> v2:172.21.15.160:3300/0 conn(0x7f345c06f6c0 msgr2=0x7f345c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:59.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 --2- 172.21.15.26:0/3629611665 >> v2:172.21.15.160:3300/0 conn(0x7f345c06f6c0 0x7f345c15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c070120 con 0x7f345c159820 2023-12-19T16:42:59.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f3458001680 con 0x7f345c159820 2023-12-19T16:42:59.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 >> v1:172.21.15.26:6789/0 conn(0x7f345c159820 legacy=0x7f345c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 --2- 172.21.15.26:0/3629611665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f345c159c00 0x7f3448003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 --2- 172.21.15.26:0/3629611665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3448003f00 0x7f34480047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 --2- 172.21.15.26:0/3629611665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3448004d10 0x7f3448005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c15e470 con 0x7f3448003f00 2023-12-19T16:42:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c06fe40 con 0x7f3448004d10 2023-12-19T16:42:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344800b990 con 0x7f345c159c00 2023-12-19T16:42:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3460c01640 1 --2- 172.21.15.26:0/3629611665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3448003f00 0x7f34480047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3461402640 1 --2- 172.21.15.26:0/3629611665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f345c159c00 0x7f3448003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3461c03640 1 --2- 172.21.15.26:0/3629611665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3448004d10 0x7f3448005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3460c01640 1 -- 172.21.15.26:0/3629611665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f345c159c00 msgr2=0x7f3448003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3460c01640 1 --2- 172.21.15.26:0/3629611665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f345c159c00 0x7f3448003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3460c01640 1 -- 172.21.15.26:0/3629611665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3448004d10 msgr2=0x7f3448005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3460c01640 1 --2- 172.21.15.26:0/3629611665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3448004d10 0x7f3448005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.259+0000 7f3460c01640 1 -- 172.21.15.26:0/3629611665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3448003530 con 0x7f3448003f00 2023-12-19T16:42:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.263+0000 7f3460c01640 1 --2- 172.21.15.26:0/3629611665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3448003f00 0x7f34480047d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f344c009670 tx=0x7f344c01ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85a4447bb0c033a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.263+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c0a7880 con 0x7f3448003f00 2023-12-19T16:42:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.263+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c0a79e0 con 0x7f3448003f00 2023-12-19T16:42:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.263+0000 7f3453fff640 1 -- 172.21.15.26:0/3629611665 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c0a7d90 con 0x7f3448003f00 2023-12-19T16:42:59.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.263+0000 7f3463666640 1 -- 172.21.15.26:0/3629611665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3448003f00 msgr2=0x7f34480047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.263+0000 7f3463666640 1 --2- 172.21.15.26:0/3629611665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3448003f00 0x7f34480047d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f344c009670 tx=0x7f344c01ec20 comp rx=0 tx=0).stop 2023-12-19T16:42:59.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.263+0000 7f3461c03640 1 -- 172.21.15.26:0/3629611665 reap_dead start 2023-12-19T16:42:59.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.263+0000 7f3463666640 1 -- 172.21.15.26:0/3629611665 shutdown_connections 2023-12-19T16:42:59.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.263+0000 7f3463666640 1 -- 172.21.15.26:0/3629611665 >> 172.21.15.26:0/3629611665 conn(0x7f345c0ce730 msgr2=0x7f345c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:59.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.263+0000 7f3463666640 1 -- 172.21.15.26:0/3629611665 shutdown_connections 2023-12-19T16:42:59.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.263+0000 7f3463666640 1 -- 172.21.15.26:0/3629611665 wait complete. 2023-12-19T16:42:59.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3463666640 1 Processor -- start 2023-12-19T16:42:59.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3463666640 1 -- start start 2023-12-19T16:42:59.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3463666640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3448003f00 0x7f345c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3463666640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0cd520 0x7f345c152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3463666640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f345c0cd9a0 0x7f345c153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3463666640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c0c39a0 con 0x7f345c0cd520 2023-12-19T16:42:59.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3463666640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c0c2d60 con 0x7f345c0cd9a0 2023-12-19T16:42:59.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3463666640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345c0c3000 con 0x7f3448003f00 2023-12-19T16:42:59.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3460c01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0cd520 0x7f345c152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3460c01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0cd520 0x7f345c152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34058/0 (socket says 172.21.15.26:34058) 2023-12-19T16:42:59.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3460c01640 1 -- 172.21.15.26:0/565899446 learned_addr learned my addr 172.21.15.26:0/565899446 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:59.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3461402640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3448003f00 0x7f345c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3461c03640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f345c0cd9a0 0x7f345c153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3460c01640 1 -- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3448003f00 msgr2=0x7f345c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3460c01640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3448003f00 0x7f345c159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3460c01640 1 -- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f345c0cd9a0 msgr2=0x7f345c153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3460c01640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f345c0cd9a0 0x7f345c153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3460c01640 1 -- 172.21.15.26:0/565899446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c0c32e0 con 0x7f345c0cd520 2023-12-19T16:42:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3460c01640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0cd520 0x7f345c152bc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f344c01ec60 tx=0x7f3448005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:59.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3451ffb640 1 -- 172.21.15.26:0/565899446 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c004230 con 0x7f345c0cd520 2023-12-19T16:42:59.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3451ffb640 1 -- 172.21.15.26:0/565899446 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c0043b0 con 0x7f345c0cd520 2023-12-19T16:42:59.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3451ffb640 1 -- 172.21.15.26:0/565899446 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c003860 con 0x7f345c0cd520 2023-12-19T16:42:59.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3463666640 1 -- 172.21.15.26:0/565899446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345c0c35c0 con 0x7f345c0cd520 2023-12-19T16:42:59.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.267+0000 7f3463666640 1 -- 172.21.15.26:0/565899446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345c0c3dc0 con 0x7f345c0cd520 2023-12-19T16:42:59.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.271+0000 7f3463666640 1 -- 172.21.15.26:0/565899446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345c074270 con 0x7f345c0cd520 2023-12-19T16:42:59.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.271+0000 7f3451ffb640 1 -- 172.21.15.26:0/565899446 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f344c0039c0 con 0x7f345c0cd520 2023-12-19T16:42:59.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.271+0000 7f3451ffb640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3440065a90 0x7f3440067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.271+0000 7f3451ffb640 1 -- 172.21.15.26:0/565899446 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f344c11ce20 con 0x7f345c0cd520 2023-12-19T16:42:59.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.271+0000 7f3461402640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3440065a90 0x7f3440067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.271+0000 7f3461402640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3440065a90 0x7f3440067f50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3458001060 tx=0x7f3458007900 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:59.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.275+0000 7f3451ffb640 1 -- 172.21.15.26:0/565899446 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f344c0ea880 con 0x7f345c0cd520 2023-12-19T16:42:59.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3463666640 1 -- 172.21.15.26:0/565899446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f345c070bf0 con 0x7f345c0cd520 2023-12-19T16:42:59.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3451ffb640 1 -- 172.21.15.26:0/565899446 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f344c0bf0a0 con 0x7f345c0cd520 2023-12-19T16:42:59.439 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:42:59.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3463666640 1 -- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3440065a90 msgr2=0x7f3440067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3463666640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3440065a90 0x7f3440067f50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3458001060 tx=0x7f3458007900 comp rx=0 tx=0).stop 2023-12-19T16:42:59.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3463666640 1 -- 172.21.15.26:0/565899446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0cd520 msgr2=0x7f345c152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3463666640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0cd520 0x7f345c152bc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f344c01ec60 tx=0x7f3448005800 comp rx=0 tx=0).stop 2023-12-19T16:42:59.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3463666640 1 -- 172.21.15.26:0/565899446 shutdown_connections 2023-12-19T16:42:59.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3463666640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f345c0cd9a0 0x7f345c153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3463666640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0cd520 0x7f345c152bc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3463666640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3440065a90 0x7f3440067f50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3463666640 1 --2- 172.21.15.26:0/565899446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3448003f00 0x7f345c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3463666640 1 -- 172.21.15.26:0/565899446 >> 172.21.15.26:0/565899446 conn(0x7f345c0ce730 msgr2=0x7f345c158960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:59.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3463666640 1 -- 172.21.15.26:0/565899446 shutdown_connections 2023-12-19T16:42:59.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.435+0000 7f3463666640 1 -- 172.21.15.26:0/565899446 wait complete. 2023-12-19T16:42:59.461 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T16:42:59.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.599+0000 7fd560539640 1 Processor -- start 2023-12-19T16:42:59.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.599+0000 7fd560539640 1 -- start start 2023-12-19T16:42:59.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.599+0000 7fd560539640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd558155030 0x7fd55815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.599+0000 7fd560539640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd55815f370 0x7fd55806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.599+0000 7fd560539640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55806c970 con 0x7fd5580ce740 2023-12-19T16:42:59.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.599+0000 7fd560539640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55815b1f0 con 0x7fd558155030 2023-12-19T16:42:59.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.599+0000 7fd560539640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55815b4d0 con 0x7fd55815f370 2023-12-19T16:42:59.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.599+0000 7fd55e2d5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd558155030 0x7fd55815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.599+0000 7fd55ead6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd5580ce740 0x7fd5580ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48564/0 (socket says 172.21.15.26:48564) 2023-12-19T16:42:59.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.599+0000 7fd55ead6640 1 -- 172.21.15.26:0/1353654990 learned_addr learned my addr 172.21.15.26:0/1353654990 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:42:59.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.603+0000 7fd55e2d5640 1 -- 172.21.15.26:0/1353654990 >> v2:172.21.15.160:3300/0 conn(0x7fd55815f370 msgr2=0x7fd55806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:42:59.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.603+0000 7fd55dad4640 1 --2- 172.21.15.26:0/1353654990 >> v2:172.21.15.160:3300/0 conn(0x7fd55815f370 0x7fd55806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.603+0000 7fd55d2d3640 1 -- 172.21.15.26:0/1353654990 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228983971 0 0) 0x7fd55806c970 con 0x7fd5580ce740 2023-12-19T16:42:59.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.603+0000 7fd55e2d5640 1 --2- 172.21.15.26:0/1353654990 >> v2:172.21.15.160:3300/0 conn(0x7fd55815f370 0x7fd55806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.603+0000 7fd55e2d5640 1 -- 172.21.15.26:0/1353654990 >> v1:172.21.15.26:6789/0 conn(0x7fd5580ce740 legacy=0x7fd5580ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.603+0000 7fd55e2d5640 1 -- 172.21.15.26:0/1353654990 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55815b7b0 con 0x7fd558155030 2023-12-19T16:42:59.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.603+0000 7fd55dad4640 1 --2- 172.21.15.26:0/1353654990 >> v2:172.21.15.160:3300/0 conn(0x7fd55815f370 0x7fd55806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:42:59.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.603+0000 7fd55e2d5640 1 --2- 172.21.15.26:0/1353654990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd558155030 0x7fd55815bcd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd5580b1780 tx=0x7fd55401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84620fa8725e138b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:59.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.603+0000 7fd55d2d3640 1 -- 172.21.15.26:0/1353654990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540ad680 con 0x7fd558155030 2023-12-19T16:42:59.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.603+0000 7fd55d2d3640 1 -- 172.21.15.26:0/1353654990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5540ad7e0 con 0x7fd558155030 2023-12-19T16:42:59.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.603+0000 7fd55d2d3640 1 -- 172.21.15.26:0/1353654990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540adb70 con 0x7fd558155030 2023-12-19T16:42:59.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 -- 172.21.15.26:0/1353654990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd558155030 msgr2=0x7fd55815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 --2- 172.21.15.26:0/1353654990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd558155030 0x7fd55815bcd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd5580b1780 tx=0x7fd55401c460 comp rx=0 tx=0).stop 2023-12-19T16:42:59.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 -- 172.21.15.26:0/1353654990 shutdown_connections 2023-12-19T16:42:59.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 --2- 172.21.15.26:0/1353654990 >> v2:172.21.15.160:3300/0 conn(0x7fd55815f370 0x7fd55806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 --2- 172.21.15.26:0/1353654990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd558155030 0x7fd55815bcd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 -- 172.21.15.26:0/1353654990 >> 172.21.15.26:0/1353654990 conn(0x7fd5580658e0 msgr2=0x7fd558065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:59.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 -- 172.21.15.26:0/1353654990 shutdown_connections 2023-12-19T16:42:59.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 -- 172.21.15.26:0/1353654990 wait complete. 2023-12-19T16:42:59.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 Processor -- start 2023-12-19T16:42:59.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 -- start start 2023-12-19T16:42:59.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5580ce740 0x7fd5581f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd558155030 0x7fd55814d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd55815f370 0x7fd5580ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5581f6330 con 0x7fd5580ce740 2023-12-19T16:42:59.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55814e340 con 0x7fd55815f370 2023-12-19T16:42:59.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd560539640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55814e140 con 0x7fd558155030 2023-12-19T16:42:59.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd55e2d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5580ce740 0x7fd5581f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd55e2d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5580ce740 0x7fd5581f5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34080/0 (socket says 172.21.15.26:34080) 2023-12-19T16:42:59.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd55e2d5640 1 -- 172.21.15.26:0/127596138 learned_addr learned my addr 172.21.15.26:0/127596138 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:42:59.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd55ead6640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd55815f370 0x7fd5580ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd55dad4640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd558155030 0x7fd55814d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd55e2d5640 1 -- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd558155030 msgr2=0x7fd55814d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd55e2d5640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd558155030 0x7fd55814d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd55e2d5640 1 -- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd55815f370 msgr2=0x7fd5580ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd55e2d5640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd55815f370 0x7fd5580ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd55e2d5640 1 -- 172.21.15.26:0/127596138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd554009420 con 0x7fd5580ce740 2023-12-19T16:42:59.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.607+0000 7fd55dad4640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd558155030 0x7fd55814d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:42:59.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.611+0000 7fd55e2d5640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5580ce740 0x7fd5581f5df0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd554019070 tx=0x7fd554019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:59.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.611+0000 7fd54effd640 1 -- 172.21.15.26:0/127596138 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540adde0 con 0x7fd5580ce740 2023-12-19T16:42:59.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.611+0000 7fd54effd640 1 -- 172.21.15.26:0/127596138 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd554004030 con 0x7fd5580ce740 2023-12-19T16:42:59.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.611+0000 7fd54effd640 1 -- 172.21.15.26:0/127596138 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540043a0 con 0x7fd5580ce740 2023-12-19T16:42:59.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.611+0000 7fd560539640 1 -- 172.21.15.26:0/127596138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5580cf0e0 con 0x7fd5580ce740 2023-12-19T16:42:59.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.611+0000 7fd560539640 1 -- 172.21.15.26:0/127596138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55814e5b0 con 0x7fd5580ce740 2023-12-19T16:42:59.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.611+0000 7fd560539640 1 -- 172.21.15.26:0/127596138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd558074270 con 0x7fd5580ce740 2023-12-19T16:42:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.611+0000 7fd54effd640 1 -- 172.21.15.26:0/127596138 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd5540b4040 con 0x7fd5580ce740 2023-12-19T16:42:59.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.611+0000 7fd54effd640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd53c065a60 0x7fd53c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:42:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.611+0000 7fd54effd640 1 -- 172.21.15.26:0/127596138 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd55411c500 con 0x7fd5580ce740 2023-12-19T16:42:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.611+0000 7fd55dad4640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd53c065a60 0x7fd53c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:42:59.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.615+0000 7fd55dad4640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd53c065a60 0x7fd53c067f20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd55806c2d0 tx=0x7fd548008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:42:59.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.615+0000 7fd54effd640 1 -- 172.21.15.26:0/127596138 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd5540e9f70 con 0x7fd5580ce740 2023-12-19T16:42:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.783+0000 7fd560539640 1 -- 172.21.15.26:0/127596138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd55815af30 con 0x7fd5580ce740 2023-12-19T16:42:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.783+0000 7fd54effd640 1 -- 172.21.15.26:0/127596138 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd5540ed590 con 0x7fd5580ce740 2023-12-19T16:42:59.790 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2023-12-19T16:42:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.787+0000 7fd560539640 1 -- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd53c065a60 msgr2=0x7fd53c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.787+0000 7fd560539640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd53c065a60 0x7fd53c067f20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd55806c2d0 tx=0x7fd548008040 comp rx=0 tx=0).stop 2023-12-19T16:42:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.787+0000 7fd560539640 1 -- 172.21.15.26:0/127596138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5580ce740 msgr2=0x7fd5581f5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:42:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.787+0000 7fd560539640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5580ce740 0x7fd5581f5df0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd554019070 tx=0x7fd554019200 comp rx=0 tx=0).stop 2023-12-19T16:42:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.787+0000 7fd560539640 1 -- 172.21.15.26:0/127596138 shutdown_connections 2023-12-19T16:42:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.787+0000 7fd560539640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd55815f370 0x7fd5580ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.787+0000 7fd560539640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd53c065a60 0x7fd53c067f20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.787+0000 7fd560539640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd558155030 0x7fd55814d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.787+0000 7fd560539640 1 --2- 172.21.15.26:0/127596138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5580ce740 0x7fd5581f5df0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:42:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.787+0000 7fd560539640 1 -- 172.21.15.26:0/127596138 >> 172.21.15.26:0/127596138 conn(0x7fd5580658e0 msgr2=0x7fd5580b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:42:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.787+0000 7fd560539640 1 -- 172.21.15.26:0/127596138 shutdown_connections 2023-12-19T16:42:59.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:42:59.787+0000 7fd560539640 1 -- 172.21.15.26:0/127596138 wait complete. 2023-12-19T16:42:59.813 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T16:42:59.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:59.814 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-12-19T16:42:59.849 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T16:42:59.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:42:59.849 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T16:42:59.908 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:59.908 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:42:59.918 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:59.919 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:42:59.976 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:42:59.976 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:43:00.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:43:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.167+0000 7fbd175ce640 1 Processor -- start 2023-12-19T16:43:00.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.167+0000 7fbd175ce640 1 -- start start 2023-12-19T16:43:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd175ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd100cb1d0 0x7fbd100cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd175ce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbd100b1d40 0x7fbd100b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd175ce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1006d4b0 con 0x7fbd1006bee0 2023-12-19T16:43:00.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd175ce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd100cc030 con 0x7fbd100b1d40 2023-12-19T16:43:00.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd175ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100cbaf0 con 0x7fbd100cb1d0 2023-12-19T16:43:00.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd14b69640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbd1006bee0 0x7fbd1006c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48572/0 (socket says 172.21.15.26:48572) 2023-12-19T16:43:00.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd14b69640 1 -- 172.21.15.26:0/124782902 learned_addr learned my addr 172.21.15.26:0/124782902 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:00.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd1536a640 1 --2- 172.21.15.26:0/124782902 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd100cb1d0 0x7fbd100cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd15b6b640 1 --2- 172.21.15.26:0/124782902 >> v2:172.21.15.160:3300/0 conn(0x7fbd100b1d40 0x7fbd100b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760618514 0 0) 0x7fbd1006d4b0 con 0x7fbd1006bee0 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcfc003530 con 0x7fbd1006bee0 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1211013106 0 0) 0x7fbcfc003530 con 0x7fbd1006bee0 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd100cb1d0 msgr2=0x7fbd100cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 --2- 172.21.15.26:0/124782902 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd100cb1d0 0x7fbd100cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 >> v2:172.21.15.160:3300/0 conn(0x7fbd100b1d40 msgr2=0x7fbd100b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 --2- 172.21.15.26:0/124782902 >> v2:172.21.15.160:3300/0 conn(0x7fbd100b1d40 0x7fbd100b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1006cc10 con 0x7fbd1006bee0 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fbd00001680 con 0x7fbd1006bee0 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 >> v1:172.21.15.26:6789/0 conn(0x7fbd1006bee0 legacy=0x7fbd1006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 --2- 172.21.15.26:0/124782902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcfc003980 0x7fbcfc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 --2- 172.21.15.26:0/124782902 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd100cb1d0 0x7fbcfc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 --2- 172.21.15.26:0/124782902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcfc004d10 0x7fbcfc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100cc030 con 0x7fbcfc003980 2023-12-19T16:43:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd100cbaf0 con 0x7fbcfc004d10 2023-12-19T16:43:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc00b990 con 0x7fbd100cb1d0 2023-12-19T16:43:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd1536a640 1 --2- 172.21.15.26:0/124782902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcfc003980 0x7fbcfc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd14b69640 1 --2- 172.21.15.26:0/124782902 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd100cb1d0 0x7fbcfc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd15b6b640 1 --2- 172.21.15.26:0/124782902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcfc004d10 0x7fbcfc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd1536a640 1 -- 172.21.15.26:0/124782902 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd100cb1d0 msgr2=0x7fbcfc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd1536a640 1 --2- 172.21.15.26:0/124782902 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd100cb1d0 0x7fbcfc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd1536a640 1 -- 172.21.15.26:0/124782902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcfc004d10 msgr2=0x7fbcfc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd1536a640 1 --2- 172.21.15.26:0/124782902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcfc004d10 0x7fbcfc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.171+0000 7fbd1536a640 1 -- 172.21.15.26:0/124782902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc003530 con 0x7fbcfc003980 2023-12-19T16:43:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.175+0000 7fbd1536a640 1 --2- 172.21.15.26:0/124782902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcfc003980 0x7fbcfc003da0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009ae0 tx=0x7fbd0c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81147a8dcaa42559 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.175+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0a5530 con 0x7fbcfc003980 2023-12-19T16:43:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.175+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c0a5690 con 0x7fbcfc003980 2023-12-19T16:43:00.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.175+0000 7fbd07fff640 1 -- 172.21.15.26:0/124782902 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0a5a40 con 0x7fbcfc003980 2023-12-19T16:43:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.175+0000 7fbd175ce640 1 -- 172.21.15.26:0/124782902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcfc003980 msgr2=0x7fbcfc003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.175+0000 7fbd175ce640 1 --2- 172.21.15.26:0/124782902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcfc003980 0x7fbcfc003da0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009ae0 tx=0x7fbd0c01ca50 comp rx=0 tx=0).stop 2023-12-19T16:43:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.175+0000 7fbd15b6b640 1 -- 172.21.15.26:0/124782902 reap_dead start 2023-12-19T16:43:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.175+0000 7fbd175ce640 1 -- 172.21.15.26:0/124782902 shutdown_connections 2023-12-19T16:43:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.175+0000 7fbd175ce640 1 -- 172.21.15.26:0/124782902 >> 172.21.15.26:0/124782902 conn(0x7fbd100658e0 msgr2=0x7fbd10065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.175+0000 7fbd175ce640 1 -- 172.21.15.26:0/124782902 shutdown_connections 2023-12-19T16:43:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.175+0000 7fbd175ce640 1 -- 172.21.15.26:0/124782902 wait complete. 2023-12-19T16:43:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd175ce640 1 Processor -- start 2023-12-19T16:43:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd175ce640 1 -- start start 2023-12-19T16:43:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd175ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcfc003980 0x7fbd1015e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd175ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd10158400 0x7fbd101fcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd175ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd101fd500 0x7fbd100af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd175ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1015ebd0 con 0x7fbcfc003980 2023-12-19T16:43:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd175ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1014f570 con 0x7fbd101fd500 2023-12-19T16:43:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd175ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd101fda80 con 0x7fbd10158400 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd1536a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcfc003980 0x7fbd1015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd1536a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcfc003980 0x7fbd1015e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34106/0 (socket says 172.21.15.26:34106) 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd1536a640 1 -- 172.21.15.26:0/2393447698 learned_addr learned my addr 172.21.15.26:0/2393447698 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd15b6b640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd101fd500 0x7fbd100af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd14b69640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd10158400 0x7fbd101fcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd1536a640 1 -- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd10158400 msgr2=0x7fbd101fcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd1536a640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd10158400 0x7fbd101fcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd1536a640 1 -- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd101fd500 msgr2=0x7fbd100af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd1536a640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd101fd500 0x7fbd100af690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd1536a640 1 -- 172.21.15.26:0/2393447698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd101fdd60 con 0x7fbcfc003980 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd1536a640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcfc003980 0x7fbd1015e690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0b0040 tx=0x7fbcfc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd05ffb640 1 -- 172.21.15.26:0/2393447698 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0be070 con 0x7fbcfc003980 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd05ffb640 1 -- 172.21.15.26:0/2393447698 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c0a5800 con 0x7fbcfc003980 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd05ffb640 1 -- 172.21.15.26:0/2393447698 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c004020 con 0x7fbcfc003980 2023-12-19T16:43:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd175ce640 1 -- 172.21.15.26:0/2393447698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1015dbb0 con 0x7fbcfc003980 2023-12-19T16:43:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.179+0000 7fbd175ce640 1 -- 172.21.15.26:0/2393447698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd1015e170 con 0x7fbcfc003980 2023-12-19T16:43:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.183+0000 7fbd175ce640 1 -- 172.21.15.26:0/2393447698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1015e330 con 0x7fbcfc003980 2023-12-19T16:43:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.183+0000 7fbd05ffb640 1 -- 172.21.15.26:0/2393447698 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbd0c0b4040 con 0x7fbcfc003980 2023-12-19T16:43:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.183+0000 7fbd05ffb640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbcf4065a60 0x7fbcf4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.183+0000 7fbd05ffb640 1 -- 172.21.15.26:0/2393447698 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd0c11cd40 con 0x7fbcfc003980 2023-12-19T16:43:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.183+0000 7fbd14b69640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbcf4065a60 0x7fbcf4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.183+0000 7fbd14b69640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbcf4065a60 0x7fbcf4067f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd10159a70 tx=0x7fbd00009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.187+0000 7fbd05ffb640 1 -- 172.21.15.26:0/2393447698 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbd0c0ea660 con 0x7fbcfc003980 2023-12-19T16:43:00.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.343+0000 7fbd175ce640 1 -- 172.21.15.26:0/2393447698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd10074270 con 0x7fbcfc003980 2023-12-19T16:43:00.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.347+0000 7fbd05ffb640 1 -- 172.21.15.26:0/2393447698 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fbd0c0edc80 con 0x7fbcfc003980 2023-12-19T16:43:00.351 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:00.351 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:43:00.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.347+0000 7fbd175ce640 1 -- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbcf4065a60 msgr2=0x7fbcf4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.347+0000 7fbd175ce640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbcf4065a60 0x7fbcf4067f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbd10159a70 tx=0x7fbd00009040 comp rx=0 tx=0).stop 2023-12-19T16:43:00.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.347+0000 7fbd175ce640 1 -- 172.21.15.26:0/2393447698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcfc003980 msgr2=0x7fbd1015e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.347+0000 7fbd175ce640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcfc003980 0x7fbd1015e690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c0b0040 tx=0x7fbcfc005800 comp rx=0 tx=0).stop 2023-12-19T16:43:00.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.347+0000 7fbd175ce640 1 -- 172.21.15.26:0/2393447698 shutdown_connections 2023-12-19T16:43:00.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.347+0000 7fbd175ce640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd101fd500 0x7fbd100af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.347+0000 7fbd175ce640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbcf4065a60 0x7fbcf4067f20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.347+0000 7fbd175ce640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd10158400 0x7fbd101fcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.347+0000 7fbd175ce640 1 --2- 172.21.15.26:0/2393447698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcfc003980 0x7fbd1015e690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.347+0000 7fbd175ce640 1 -- 172.21.15.26:0/2393447698 >> 172.21.15.26:0/2393447698 conn(0x7fbd100658e0 msgr2=0x7fbd1015f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:00.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.347+0000 7fbd175ce640 1 -- 172.21.15.26:0/2393447698 shutdown_connections 2023-12-19T16:43:00.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.347+0000 7fbd175ce640 1 -- 172.21.15.26:0/2393447698 wait complete. 2023-12-19T16:43:00.372 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:42:25.598550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:42:26.697362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:43:00.373 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:43:00.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.515+0000 7f4eb2996640 1 Processor -- start 2023-12-19T16:43:00.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.515+0000 7f4eb2996640 1 -- start start 2023-12-19T16:43:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.515+0000 7f4eb2996640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4eac06f6c0 0x7f4eac15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.515+0000 7f4eb2996640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eac06fb60 con 0x7f4eac159820 2023-12-19T16:43:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.515+0000 7f4eb2996640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eac06fe40 con 0x7f4eac159c00 2023-12-19T16:43:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.515+0000 7f4eb2996640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac15e470 con 0x7f4eac06f6c0 2023-12-19T16:43:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.515+0000 7f4eabfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4eac159820 0x7f4eac0c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48574/0 (socket says 172.21.15.26:48574) 2023-12-19T16:43:00.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.515+0000 7f4eabfff640 1 -- 172.21.15.26:0/2415387306 learned_addr learned my addr 172.21.15.26:0/2415387306 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.519+0000 7f4eab7fe640 1 --2- 172.21.15.26:0/2415387306 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4eac06f6c0 0x7f4eac15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.519+0000 7f4eaaffd640 1 -- 172.21.15.26:0/2415387306 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149262209 0 0) 0x7f4eac06fb60 con 0x7f4eac159820 2023-12-19T16:43:00.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.519+0000 7f4eaaffd640 1 -- 172.21.15.26:0/2415387306 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e94003530 con 0x7f4eac159820 2023-12-19T16:43:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.519+0000 7f4eaaffd640 1 -- 172.21.15.26:0/2415387306 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999866932 0 0) 0x7f4eac06fe40 con 0x7f4eac159c00 2023-12-19T16:43:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.519+0000 7f4eaaffd640 1 -- 172.21.15.26:0/2415387306 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eac06fb60 con 0x7f4eac159c00 2023-12-19T16:43:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.519+0000 7f4eab7fe640 1 -- 172.21.15.26:0/2415387306 >> v1:172.21.15.160:6789/0 conn(0x7f4eac159c00 legacy=0x7f4eac070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.519+0000 7f4eab7fe640 1 -- 172.21.15.26:0/2415387306 >> v1:172.21.15.26:6789/0 conn(0x7f4eac159820 legacy=0x7f4eac0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.519+0000 7f4eab7fe640 1 -- 172.21.15.26:0/2415387306 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac070120 con 0x7f4eac06f6c0 2023-12-19T16:43:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.519+0000 7f4eab7fe640 1 --2- 172.21.15.26:0/2415387306 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4eac06f6c0 0x7f4eac15df30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00097f0 tx=0x7f4ea001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5306c597f71341b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.519+0000 7f4eaaffd640 1 -- 172.21.15.26:0/2415387306 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00aa5f0 con 0x7f4eac06f6c0 2023-12-19T16:43:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eaaffd640 1 -- 172.21.15.26:0/2415387306 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea00aa750 con 0x7f4eac06f6c0 2023-12-19T16:43:00.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eaaffd640 1 -- 172.21.15.26:0/2415387306 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea00aaae0 con 0x7f4eac06f6c0 2023-12-19T16:43:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 -- 172.21.15.26:0/2415387306 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4eac06f6c0 msgr2=0x7f4eac15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 --2- 172.21.15.26:0/2415387306 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4eac06f6c0 0x7f4eac15df30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00097f0 tx=0x7f4ea001c490 comp rx=0 tx=0).stop 2023-12-19T16:43:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 -- 172.21.15.26:0/2415387306 shutdown_connections 2023-12-19T16:43:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 --2- 172.21.15.26:0/2415387306 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4eac06f6c0 0x7f4eac15df30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 -- 172.21.15.26:0/2415387306 >> 172.21.15.26:0/2415387306 conn(0x7f4eac0ce730 msgr2=0x7f4eac0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 -- 172.21.15.26:0/2415387306 shutdown_connections 2023-12-19T16:43:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 -- 172.21.15.26:0/2415387306 wait complete. 2023-12-19T16:43:00.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 Processor -- start 2023-12-19T16:43:00.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 -- start start 2023-12-19T16:43:00.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac06f6c0 0x7f4eac0c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eac159820 0x7f4eac149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4eac159c00 0x7f4eac0c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac14f9f0 con 0x7f4eac06f6c0 2023-12-19T16:43:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac159fe0 con 0x7f4eac159820 2023-12-19T16:43:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eb2996640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac0c28f0 con 0x7f4eac159c00 2023-12-19T16:43:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.523+0000 7f4eabfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac06f6c0 0x7f4eac0c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4eabfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac06f6c0 0x7f4eac0c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34120/0 (socket says 172.21.15.26:34120) 2023-12-19T16:43:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4eabfff640 1 -- 172.21.15.26:0/4231336186 learned_addr learned my addr 172.21.15.26:0/4231336186 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4eab7fe640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eac159820 0x7f4eac149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4eb0f33640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4eac159c00 0x7f4eac0c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4eabfff640 1 -- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4eac159c00 msgr2=0x7f4eac0c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4eabfff640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4eac159c00 0x7f4eac0c9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4eabfff640 1 -- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eac159820 msgr2=0x7f4eac149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4eabfff640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eac159820 0x7f4eac149700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4eabfff640 1 -- 172.21.15.26:0/4231336186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea0009420 con 0x7f4eac06f6c0 2023-12-19T16:43:00.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4eabfff640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac06f6c0 0x7f4eac0c23b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00c500 tx=0x7f4e9c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4ea8ff9640 1 -- 172.21.15.26:0/4231336186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c005cf0 con 0x7f4eac06f6c0 2023-12-19T16:43:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4ea8ff9640 1 -- 172.21.15.26:0/4231336186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c005e50 con 0x7f4eac06f6c0 2023-12-19T16:43:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4ea8ff9640 1 -- 172.21.15.26:0/4231336186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c007330 con 0x7f4eac06f6c0 2023-12-19T16:43:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4eb2996640 1 -- 172.21.15.26:0/4231336186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eac06b880 con 0x7f4eac06f6c0 2023-12-19T16:43:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4eb2996640 1 -- 172.21.15.26:0/4231336186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eac15a7f0 con 0x7f4eac06f6c0 2023-12-19T16:43:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4eb2996640 1 -- 172.21.15.26:0/4231336186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eac074270 con 0x7f4eac06f6c0 2023-12-19T16:43:00.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4ea8ff9640 1 -- 172.21.15.26:0/4231336186 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4e9c007490 con 0x7f4eac06f6c0 2023-12-19T16:43:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4ea8ff9640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4e98065a60 0x7f4e98067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.527+0000 7f4ea8ff9640 1 -- 172.21.15.26:0/4231336186 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e9c082870 con 0x7f4eac06f6c0 2023-12-19T16:43:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.531+0000 7f4eab7fe640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4e98065a60 0x7f4e98067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.531+0000 7f4eab7fe640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4e98065a60 0x7f4e98067f20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00ae000 tx=0x7f4ea00a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.531+0000 7f4ea8ff9640 1 -- 172.21.15.26:0/4231336186 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4e9c050220 con 0x7f4eac06f6c0 2023-12-19T16:43:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.691+0000 7f4eb2996640 1 -- 172.21.15.26:0/4231336186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4eac0c19a0 con 0x7f4eac06f6c0 2023-12-19T16:43:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.691+0000 7f4ea8ff9640 1 -- 172.21.15.26:0/4231336186 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e9c0170a0 con 0x7f4eac06f6c0 2023-12-19T16:43:00.698 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:43:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.695+0000 7f4eb2996640 1 -- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4e98065a60 msgr2=0x7f4e98067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.695+0000 7f4eb2996640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4e98065a60 0x7f4e98067f20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00ae000 tx=0x7f4ea00a7000 comp rx=0 tx=0).stop 2023-12-19T16:43:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.695+0000 7f4eb2996640 1 -- 172.21.15.26:0/4231336186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac06f6c0 msgr2=0x7f4eac0c23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.695+0000 7f4eb2996640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac06f6c0 0x7f4eac0c23b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00c500 tx=0x7f4e9c00c9c0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.695+0000 7f4eb2996640 1 -- 172.21.15.26:0/4231336186 shutdown_connections 2023-12-19T16:43:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.695+0000 7f4eb2996640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4e98065a60 0x7f4e98067f20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.695+0000 7f4eb2996640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4eac159c00 0x7f4eac0c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.695+0000 7f4eb2996640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eac159820 0x7f4eac149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.695+0000 7f4eb2996640 1 --2- 172.21.15.26:0/4231336186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4eac06f6c0 0x7f4eac0c23b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.695+0000 7f4eb2996640 1 -- 172.21.15.26:0/4231336186 >> 172.21.15.26:0/4231336186 conn(0x7f4eac0ce730 msgr2=0x7f4eac158eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.695+0000 7f4eb2996640 1 -- 172.21.15.26:0/4231336186 shutdown_connections 2023-12-19T16:43:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.695+0000 7f4eb2996640 1 -- 172.21.15.26:0/4231336186 wait complete. 2023-12-19T16:43:00.719 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:43:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.859+0000 7fcb63960640 1 Processor -- start 2023-12-19T16:43:00.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.859+0000 7fcb63960640 1 -- start start 2023-12-19T16:43:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.859+0000 7fcb63960640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcb5c159140 0x7fcb5c0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb63960640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5c06f490 con 0x7fcb5c0ccc50 2023-12-19T16:43:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb63960640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5c159520 con 0x7fcb5c06b390 2023-12-19T16:43:00.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb63960640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5c0cd690 con 0x7fcb5c159140 2023-12-19T16:43:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb60efb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcb5c0ccc50 0x7fcb5c06ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48586/0 (socket says 172.21.15.26:48586) 2023-12-19T16:43:00.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb60efb640 1 -- 172.21.15.26:0/49075764 learned_addr learned my addr 172.21.15.26:0/49075764 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:00.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569591496 0 0) 0x7fcb5c06f490 con 0x7fcb5c0ccc50 2023-12-19T16:43:00.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb48003530 con 0x7fcb5c0ccc50 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb616fc640 1 --2- 172.21.15.26:0/49075764 >> v2:172.21.15.160:3300/0 conn(0x7fcb5c159140 0x7fcb5c0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2112413476 0 0) 0x7fcb48003530 con 0x7fcb5c0ccc50 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 >> v2:172.21.15.160:3300/0 conn(0x7fcb5c159140 msgr2=0x7fcb5c0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 --2- 172.21.15.26:0/49075764 >> v2:172.21.15.160:3300/0 conn(0x7fcb5c159140 0x7fcb5c0b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 >> v1:172.21.15.160:6789/0 conn(0x7fcb5c06b390 legacy=0x7fcb5c164060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c0cd970 con 0x7fcb5c0ccc50 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fcb4c001680 con 0x7fcb5c0ccc50 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 >> v1:172.21.15.26:6789/0 conn(0x7fcb5c0ccc50 legacy=0x7fcb5c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 --2- 172.21.15.26:0/49075764 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb48003980 0x7fcb48003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 --2- 172.21.15.26:0/49075764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb48004380 0x7fcb48004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 --2- 172.21.15.26:0/49075764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb48005050 0x7fcb4800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c0cd690 con 0x7fcb48005050 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4800bfb0 con 0x7fcb48004380 2023-12-19T16:43:00.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4800c170 con 0x7fcb48003980 2023-12-19T16:43:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb60efb640 1 --2- 172.21.15.26:0/49075764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb48004380 0x7fcb48004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb61efd640 1 --2- 172.21.15.26:0/49075764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb48005050 0x7fcb4800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb616fc640 1 --2- 172.21.15.26:0/49075764 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb48003980 0x7fcb48003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb61efd640 1 -- 172.21.15.26:0/49075764 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb48003980 msgr2=0x7fcb48003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb61efd640 1 --2- 172.21.15.26:0/49075764 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb48003980 0x7fcb48003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb61efd640 1 -- 172.21.15.26:0/49075764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb48004380 msgr2=0x7fcb48004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb61efd640 1 --2- 172.21.15.26:0/49075764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb48004380 0x7fcb48004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb61efd640 1 -- 172.21.15.26:0/49075764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb48003530 con 0x7fcb48005050 2023-12-19T16:43:00.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.863+0000 7fcb60efb640 1 --2- 172.21.15.26:0/49075764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb48004380 0x7fcb48004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb61efd640 1 -- 172.21.15.26:0/49075764 reap_dead start 2023-12-19T16:43:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb61efd640 1 --2- 172.21.15.26:0/49075764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb48005050 0x7fcb4800b990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcb54001f60 tx=0x7fcb5401c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d50f0d3e4a48631 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54003b80 con 0x7fcb48005050 2023-12-19T16:43:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54003ce0 con 0x7fcb48005050 2023-12-19T16:43:00.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb53fff640 1 -- 172.21.15.26:0/49075764 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540052f0 con 0x7fcb48005050 2023-12-19T16:43:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb63960640 1 -- 172.21.15.26:0/49075764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb48005050 msgr2=0x7fcb4800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb63960640 1 --2- 172.21.15.26:0/49075764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb48005050 0x7fcb4800b990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcb54001f60 tx=0x7fcb5401c860 comp rx=0 tx=0).stop 2023-12-19T16:43:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb63960640 1 -- 172.21.15.26:0/49075764 shutdown_connections 2023-12-19T16:43:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb63960640 1 --2- 172.21.15.26:0/49075764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb48005050 0x7fcb4800b990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb63960640 1 -- 172.21.15.26:0/49075764 >> 172.21.15.26:0/49075764 conn(0x7fcb5c0ce730 msgr2=0x7fcb5c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb63960640 1 -- 172.21.15.26:0/49075764 shutdown_connections 2023-12-19T16:43:00.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb63960640 1 -- 172.21.15.26:0/49075764 wait complete. 2023-12-19T16:43:00.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb63960640 1 Processor -- start 2023-12-19T16:43:00.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.867+0000 7fcb63960640 1 -- start start 2023-12-19T16:43:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb63960640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb48005050 0x7fcb5c0c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb63960640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb5c14a720 0x7fcb5c0c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb63960640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb5c0acac0 0x7fcb5c0b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb63960640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c14aba0 con 0x7fcb5c0acac0 2023-12-19T16:43:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb63960640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c14afb0 con 0x7fcb48005050 2023-12-19T16:43:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb63960640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c0c6d40 con 0x7fcb5c14a720 2023-12-19T16:43:00.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb61efd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb5c0acac0 0x7fcb5c0b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb61efd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb5c0acac0 0x7fcb5c0b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34140/0 (socket says 172.21.15.26:34140) 2023-12-19T16:43:00.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb61efd640 1 -- 172.21.15.26:0/3761167853 learned_addr learned my addr 172.21.15.26:0/3761167853 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:00.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb60efb640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb5c14a720 0x7fcb5c0c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb616fc640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb48005050 0x7fcb5c0c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb61efd640 1 -- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb5c14a720 msgr2=0x7fcb5c0c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb61efd640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb5c14a720 0x7fcb5c0c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb61efd640 1 -- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb48005050 msgr2=0x7fcb5c0c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb61efd640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb48005050 0x7fcb5c0c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb61efd640 1 -- 172.21.15.26:0/3761167853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c149960 con 0x7fcb5c0acac0 2023-12-19T16:43:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb61efd640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb5c0acac0 0x7fcb5c0b27c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb5401c690 tx=0x7fcb48005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb51ffb640 1 -- 172.21.15.26:0/3761167853 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54003b80 con 0x7fcb5c0acac0 2023-12-19T16:43:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb51ffb640 1 -- 172.21.15.26:0/3761167853 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54003e50 con 0x7fcb5c0acac0 2023-12-19T16:43:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb51ffb640 1 -- 172.21.15.26:0/3761167853 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54006d80 con 0x7fcb5c0acac0 2023-12-19T16:43:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb63960640 1 -- 172.21.15.26:0/3761167853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c149c40 con 0x7fcb5c0acac0 2023-12-19T16:43:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb63960640 1 -- 172.21.15.26:0/3761167853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c14a200 con 0x7fcb5c0acac0 2023-12-19T16:43:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.871+0000 7fcb63960640 1 -- 172.21.15.26:0/3761167853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5c0ce240 con 0x7fcb5c0acac0 2023-12-19T16:43:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.875+0000 7fcb51ffb640 1 -- 172.21.15.26:0/3761167853 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcb540a7070 con 0x7fcb5c0acac0 2023-12-19T16:43:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.875+0000 7fcb51ffb640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcb40065a90 0x7fcb40067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.875+0000 7fcb51ffb640 1 -- 172.21.15.26:0/3761167853 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb5411bfd0 con 0x7fcb5c0acac0 2023-12-19T16:43:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.875+0000 7fcb616fc640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcb40065a90 0x7fcb40067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.875+0000 7fcb616fc640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcb40065a90 0x7fcb40067f50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009b60 tx=0x7fcb58009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:00.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:00.879+0000 7fcb51ffb640 1 -- 172.21.15.26:0/3761167853 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcb540e99b0 con 0x7fcb5c0acac0 2023-12-19T16:43:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.039+0000 7fcb63960640 1 -- 172.21.15.26:0/3761167853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcb5c074270 con 0x7fcb5c0acac0 2023-12-19T16:43:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.039+0000 7fcb51ffb640 1 -- 172.21.15.26:0/3761167853 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcb540ad0a0 con 0x7fcb5c0acac0 2023-12-19T16:43:01.045 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:43:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.043+0000 7fcb63960640 1 -- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcb40065a90 msgr2=0x7fcb40067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.043+0000 7fcb63960640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcb40065a90 0x7fcb40067f50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009b60 tx=0x7fcb58009190 comp rx=0 tx=0).stop 2023-12-19T16:43:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.043+0000 7fcb63960640 1 -- 172.21.15.26:0/3761167853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb5c0acac0 msgr2=0x7fcb5c0b27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.043+0000 7fcb63960640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb5c0acac0 0x7fcb5c0b27c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb5401c690 tx=0x7fcb48005800 comp rx=0 tx=0).stop 2023-12-19T16:43:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.043+0000 7fcb63960640 1 -- 172.21.15.26:0/3761167853 shutdown_connections 2023-12-19T16:43:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.043+0000 7fcb63960640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb5c0acac0 0x7fcb5c0b27c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.043+0000 7fcb63960640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcb40065a90 0x7fcb40067f50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.043+0000 7fcb63960640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcb5c14a720 0x7fcb5c0c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.043+0000 7fcb63960640 1 --2- 172.21.15.26:0/3761167853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcb48005050 0x7fcb5c0c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.043+0000 7fcb63960640 1 -- 172.21.15.26:0/3761167853 >> 172.21.15.26:0/3761167853 conn(0x7fcb5c0ce730 msgr2=0x7fcb5c0cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:01.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.043+0000 7fcb63960640 1 -- 172.21.15.26:0/3761167853 shutdown_connections 2023-12-19T16:43:01.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.043+0000 7fcb63960640 1 -- 172.21.15.26:0/3761167853 wait complete. 2023-12-19T16:43:01.066 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:43:01.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c9d96d640 1 Processor -- start 2023-12-19T16:43:01.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c9d96d640 1 -- start start 2023-12-19T16:43:01.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c9d96d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9c98159820 0x7f9c9815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c9d96d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9c9806f6c0 0x7f9c980c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c9d96d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c9806fb60 con 0x7f9c98159c00 2023-12-19T16:43:01.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c9d96d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9815e470 con 0x7f9c9806f6c0 2023-12-19T16:43:01.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c9d96d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c9806fe40 con 0x7f9c98159820 2023-12-19T16:43:01.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c8e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9c9806f6c0 0x7f9c980c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c977fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9c98159c00 0x7f9c98070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48592/0 (socket says 172.21.15.26:48592) 2023-12-19T16:43:01.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c977fe640 1 -- 172.21.15.26:0/2813268319 learned_addr learned my addr 172.21.15.26:0/2813268319 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:01.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c96ffd640 1 --2- 172.21.15.26:0/2813268319 >> v2:172.21.15.160:3300/0 conn(0x7f9c98159820 0x7f9c9815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c8e7fc640 1 -- 172.21.15.26:0/2813268319 >> v2:172.21.15.160:3300/0 conn(0x7f9c98159820 msgr2=0x7f9c9815df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:01.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c8e7fc640 1 --2- 172.21.15.26:0/2813268319 >> v2:172.21.15.160:3300/0 conn(0x7f9c98159820 0x7f9c9815df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c8e7fc640 1 -- 172.21.15.26:0/2813268319 >> v1:172.21.15.26:6789/0 conn(0x7f9c98159c00 legacy=0x7f9c98070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c8e7fc640 1 -- 172.21.15.26:0/2813268319 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c98070120 con 0x7f9c9806f6c0 2023-12-19T16:43:01.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.207+0000 7f9c967fc640 1 -- 172.21.15.26:0/2813268319 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824497442 0 0) 0x7f9c9806fb60 con 0x7f9c98159c00 2023-12-19T16:43:01.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.211+0000 7f9c8e7fc640 1 --2- 172.21.15.26:0/2813268319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9806f6c0 0x7f9c980c8e80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9c980c8e10 tx=0x7f9c8001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa32c1d04c0fcdb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:01.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.211+0000 7f9c967fc640 1 -- 172.21.15.26:0/2813268319 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c800a7070 con 0x7f9c9806f6c0 2023-12-19T16:43:01.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.211+0000 7f9c967fc640 1 -- 172.21.15.26:0/2813268319 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c800ab430 con 0x7f9c9806f6c0 2023-12-19T16:43:01.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.211+0000 7f9c967fc640 1 -- 172.21.15.26:0/2813268319 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c800ab7c0 con 0x7f9c9806f6c0 2023-12-19T16:43:01.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.211+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2813268319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9806f6c0 msgr2=0x7f9c980c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.211+0000 7f9c9d96d640 1 --2- 172.21.15.26:0/2813268319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9806f6c0 0x7f9c980c8e80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9c980c8e10 tx=0x7f9c8001c430 comp rx=0 tx=0).stop 2023-12-19T16:43:01.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.211+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2813268319 shutdown_connections 2023-12-19T16:43:01.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.211+0000 7f9c9d96d640 1 --2- 172.21.15.26:0/2813268319 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9806f6c0 0x7f9c980c8e80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.211+0000 7f9c9d96d640 1 --2- 172.21.15.26:0/2813268319 >> v2:172.21.15.160:3300/0 conn(0x7f9c98159820 0x7f9c9815df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.211+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2813268319 >> 172.21.15.26:0/2813268319 conn(0x7f9c980ce730 msgr2=0x7f9c980ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.211+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2813268319 shutdown_connections 2023-12-19T16:43:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.211+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2813268319 wait complete. 2023-12-19T16:43:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c9d96d640 1 Processor -- start 2023-12-19T16:43:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c9d96d640 1 -- start start 2023-12-19T16:43:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c9d96d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9806f6c0 0x7f9c9814fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c9d96d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c98159820 0x7f9c98147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c9d96d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c98159c00 0x7f9c98156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c9d96d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c981537b0 con 0x7f9c9806f6c0 2023-12-19T16:43:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c9d96d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c98153a90 con 0x7f9c98159c00 2023-12-19T16:43:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c9d96d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c98153d70 con 0x7f9c98159820 2023-12-19T16:43:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c96ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9806f6c0 0x7f9c9814fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c96ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9806f6c0 0x7f9c9814fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34164/0 (socket says 172.21.15.26:34164) 2023-12-19T16:43:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c96ffd640 1 -- 172.21.15.26:0/2764704760 learned_addr learned my addr 172.21.15.26:0/2764704760 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c8e7fc640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c98159820 0x7f9c98147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c977fe640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c98159c00 0x7f9c98156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c96ffd640 1 -- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c98159820 msgr2=0x7f9c98147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c96ffd640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c98159820 0x7f9c98147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c96ffd640 1 -- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c98159c00 msgr2=0x7f9c98156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c96ffd640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c98159c00 0x7f9c98156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c96ffd640 1 -- 172.21.15.26:0/2764704760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c80009420 con 0x7f9c9806f6c0 2023-12-19T16:43:01.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c977fe640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c98159c00 0x7f9c98156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:01.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c96ffd640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9806f6c0 0x7f9c9814fab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800d800 tx=0x7f9c8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:01.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c94ff9640 1 -- 172.21.15.26:0/2764704760 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800b6c0 con 0x7f9c9806f6c0 2023-12-19T16:43:01.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c94ff9640 1 -- 172.21.15.26:0/2764704760 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8800b820 con 0x7f9c9806f6c0 2023-12-19T16:43:01.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2764704760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9806b880 con 0x7f9c9806f6c0 2023-12-19T16:43:01.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c94ff9640 1 -- 172.21.15.26:0/2764704760 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8800bb90 con 0x7f9c9806f6c0 2023-12-19T16:43:01.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.215+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2764704760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c98154330 con 0x7f9c9806f6c0 2023-12-19T16:43:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.219+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2764704760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c98074270 con 0x7f9c9806f6c0 2023-12-19T16:43:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.219+0000 7f9c94ff9640 1 -- 172.21.15.26:0/2764704760 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9c8800bcf0 con 0x7f9c9806f6c0 2023-12-19T16:43:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.219+0000 7f9c94ff9640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c78065a60 0x7f9c78067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.219+0000 7f9c94ff9640 1 -- 172.21.15.26:0/2764704760 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c88083780 con 0x7f9c9806f6c0 2023-12-19T16:43:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.219+0000 7f9c8e7fc640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c78065a60 0x7f9c78067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.219+0000 7f9c8e7fc640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c78065a60 0x7f9c78067f20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9c800af000 tx=0x7f9c800a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.223+0000 7f9c94ff9640 1 -- 172.21.15.26:0/2764704760 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9c880510b0 con 0x7f9c9806f6c0 2023-12-19T16:43:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2764704760 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c980c1c50 con 0x7f9c9806f6c0 2023-12-19T16:43:01.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c94ff9640 1 -- 172.21.15.26:0/2764704760 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c880546d0 con 0x7f9c9806f6c0 2023-12-19T16:43:01.384 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:43:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c78065a60 msgr2=0x7f9c78067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c9d96d640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c78065a60 0x7f9c78067f20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9c800af000 tx=0x7f9c800a7000 comp rx=0 tx=0).stop 2023-12-19T16:43:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2764704760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9806f6c0 msgr2=0x7f9c9814fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c9d96d640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9806f6c0 0x7f9c9814fab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800d800 tx=0x7f9c8800dcc0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2764704760 shutdown_connections 2023-12-19T16:43:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c9d96d640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c98159c00 0x7f9c98156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c9d96d640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c78065a60 0x7f9c78067f20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c9d96d640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c98159820 0x7f9c98147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c9d96d640 1 --2- 172.21.15.26:0/2764704760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c9806f6c0 0x7f9c9814fab0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2764704760 >> 172.21.15.26:0/2764704760 conn(0x7f9c980ce730 msgr2=0x7f9c9806ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2764704760 shutdown_connections 2023-12-19T16:43:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.379+0000 7f9c9d96d640 1 -- 172.21.15.26:0/2764704760 wait complete. 2023-12-19T16:43:01.406 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T16:43:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4bb17640 1 Processor -- start 2023-12-19T16:43:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4bb17640 1 -- start start 2023-12-19T16:43:01.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4bb17640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7feb440b34e0 0x7feb440ad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4bb17640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb440cc0d0 con 0x7feb440cbc50 2023-12-19T16:43:01.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4bb17640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4415ddc0 con 0x7feb4406cb30 2023-12-19T16:43:01.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4bb17640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4415e0a0 con 0x7feb440b34e0 2023-12-19T16:43:01.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb490b2640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feb440cbc50 0x7feb440b2dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48596/0 (socket says 172.21.15.26:48596) 2023-12-19T16:43:01.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb490b2640 1 -- 172.21.15.26:0/2018474807 learned_addr learned my addr 172.21.15.26:0/2018474807 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:01.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4a0b4640 1 --2- 172.21.15.26:0/2018474807 >> v2:172.21.15.160:3300/0 conn(0x7feb440b34e0 0x7feb440ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130826044 0 0) 0x7feb34001120 con 0x7feb440cbc50 2023-12-19T16:43:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb30003530 con 0x7feb440cbc50 2023-12-19T16:43:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3080870507 0 0) 0x7feb30003530 con 0x7feb440cbc50 2023-12-19T16:43:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 >> v2:172.21.15.160:3300/0 conn(0x7feb440b34e0 msgr2=0x7feb440ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 --2- 172.21.15.26:0/2018474807 >> v2:172.21.15.160:3300/0 conn(0x7feb440b34e0 0x7feb440ad180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 >> v1:172.21.15.160:6789/0 conn(0x7feb4406cb30 legacy=0x7feb4415d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb440cb450 con 0x7feb440cbc50 2023-12-19T16:43:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7feb34001810 con 0x7feb440cbc50 2023-12-19T16:43:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4a0b4640 1 --2- 172.21.15.26:0/2018474807 >> v2:172.21.15.160:3300/0 conn(0x7feb440b34e0 0x7feb440ad180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 >> v1:172.21.15.26:6789/0 conn(0x7feb440cbc50 legacy=0x7feb440b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 --2- 172.21.15.26:0/2018474807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb30003980 0x7feb30003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 --2- 172.21.15.26:0/2018474807 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb30004380 0x7feb30004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 --2- 172.21.15.26:0/2018474807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb30005050 0x7feb3000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4415e0a0 con 0x7feb30005050 2023-12-19T16:43:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3000bfb0 con 0x7feb30003980 2023-12-19T16:43:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3000c170 con 0x7feb30004380 2023-12-19T16:43:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4a0b4640 1 --2- 172.21.15.26:0/2018474807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb30005050 0x7feb3000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb498b3640 1 --2- 172.21.15.26:0/2018474807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb30003980 0x7feb30003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb490b2640 1 --2- 172.21.15.26:0/2018474807 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb30004380 0x7feb30004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4a0b4640 1 -- 172.21.15.26:0/2018474807 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb30004380 msgr2=0x7feb30004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4a0b4640 1 --2- 172.21.15.26:0/2018474807 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb30004380 0x7feb30004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4a0b4640 1 -- 172.21.15.26:0/2018474807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb30003980 msgr2=0x7feb30003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4a0b4640 1 --2- 172.21.15.26:0/2018474807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb30003980 0x7feb30003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.547+0000 7feb4a0b4640 1 -- 172.21.15.26:0/2018474807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb30003530 con 0x7feb30005050 2023-12-19T16:43:01.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.551+0000 7feb4a0b4640 1 -- 172.21.15.26:0/2018474807 reap_dead start 2023-12-19T16:43:01.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.551+0000 7feb4a0b4640 1 --2- 172.21.15.26:0/2018474807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb30005050 0x7feb3000b990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00e350 tx=0x7feb3c01ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e740e3550d3359 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:01.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.551+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c0a5d60 con 0x7feb30005050 2023-12-19T16:43:01.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.551+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c00f040 con 0x7feb30005050 2023-12-19T16:43:01.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.551+0000 7feb488b1640 1 -- 172.21.15.26:0/2018474807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c003be0 con 0x7feb30005050 2023-12-19T16:43:01.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.551+0000 7feb4bb17640 1 -- 172.21.15.26:0/2018474807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb30005050 msgr2=0x7feb3000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.551+0000 7feb4bb17640 1 --2- 172.21.15.26:0/2018474807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb30005050 0x7feb3000b990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00e350 tx=0x7feb3c01ce60 comp rx=0 tx=0).stop 2023-12-19T16:43:01.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.551+0000 7feb4bb17640 1 -- 172.21.15.26:0/2018474807 shutdown_connections 2023-12-19T16:43:01.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.551+0000 7feb4bb17640 1 --2- 172.21.15.26:0/2018474807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb30005050 0x7feb3000b990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.551+0000 7feb4bb17640 1 -- 172.21.15.26:0/2018474807 >> 172.21.15.26:0/2018474807 conn(0x7feb440658e0 msgr2=0x7feb44065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:01.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.551+0000 7feb4bb17640 1 -- 172.21.15.26:0/2018474807 shutdown_connections 2023-12-19T16:43:01.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.551+0000 7feb4bb17640 1 -- 172.21.15.26:0/2018474807 wait complete. 2023-12-19T16:43:01.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb4bb17640 1 Processor -- start 2023-12-19T16:43:01.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb4bb17640 1 -- start start 2023-12-19T16:43:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb4bb17640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb30005050 0x7feb440c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb4bb17640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb44145540 0x7feb44144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb4bb17640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb440c6600 0x7feb440c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb4bb17640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb440c9ec0 con 0x7feb44145540 2023-12-19T16:43:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb4bb17640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb44145350 con 0x7feb30005050 2023-12-19T16:43:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb4bb17640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb440c5b20 con 0x7feb440c6600 2023-12-19T16:43:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb490b2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb44145540 0x7feb44144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb490b2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb44145540 0x7feb44144e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34176/0 (socket says 172.21.15.26:34176) 2023-12-19T16:43:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb490b2640 1 -- 172.21.15.26:0/2951892092 learned_addr learned my addr 172.21.15.26:0/2951892092 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb4a0b4640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb440c6600 0x7feb440c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb498b3640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb30005050 0x7feb440c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb490b2640 1 -- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb440c6600 msgr2=0x7feb440c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb490b2640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb440c6600 0x7feb440c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb490b2640 1 -- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb30005050 msgr2=0x7feb440c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb490b2640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb30005050 0x7feb440c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb490b2640 1 -- 172.21.15.26:0/2951892092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb440c9240 con 0x7feb44145540 2023-12-19T16:43:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb490b2640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb44145540 0x7feb44144e10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feb34007a50 tx=0x7feb34012920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb3a7fc640 1 -- 172.21.15.26:0/2951892092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34014750 con 0x7feb44145540 2023-12-19T16:43:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb3a7fc640 1 -- 172.21.15.26:0/2951892092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb340148d0 con 0x7feb44145540 2023-12-19T16:43:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb3a7fc640 1 -- 172.21.15.26:0/2951892092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34014c80 con 0x7feb44145540 2023-12-19T16:43:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb4bb17640 1 -- 172.21.15.26:0/2951892092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb440c9520 con 0x7feb44145540 2023-12-19T16:43:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.555+0000 7feb4bb17640 1 -- 172.21.15.26:0/2951892092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb440c9ae0 con 0x7feb44145540 2023-12-19T16:43:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.559+0000 7feb4bb17640 1 -- 172.21.15.26:0/2951892092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb440ca000 con 0x7feb44145540 2023-12-19T16:43:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.559+0000 7feb3a7fc640 1 -- 172.21.15.26:0/2951892092 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7feb340153e0 con 0x7feb44145540 2023-12-19T16:43:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.559+0000 7feb3a7fc640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feb28065a60 0x7feb28067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.559+0000 7feb3a7fc640 1 -- 172.21.15.26:0/2951892092 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb340856e0 con 0x7feb44145540 2023-12-19T16:43:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.559+0000 7feb498b3640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feb28065a60 0x7feb28067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.559+0000 7feb498b3640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feb28065a60 0x7feb28067f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feb40007fd0 tx=0x7feb40009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:01.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.563+0000 7feb3a7fc640 1 -- 172.21.15.26:0/2951892092 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feb440ca000 con 0x7feb44145540 2023-12-19T16:43:01.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb4bb17640 1 -- 172.21.15.26:0/2951892092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feb440cc0d0 con 0x7feb44145540 2023-12-19T16:43:01.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb3a7fc640 1 -- 172.21.15.26:0/2951892092 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7feb440cc0d0 con 0x7feb44145540 2023-12-19T16:43:01.740 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2023-12-19T16:43:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb4bb17640 1 -- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feb28065a60 msgr2=0x7feb28067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb4bb17640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feb28065a60 0x7feb28067f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7feb40007fd0 tx=0x7feb40009040 comp rx=0 tx=0).stop 2023-12-19T16:43:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb4bb17640 1 -- 172.21.15.26:0/2951892092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb44145540 msgr2=0x7feb44144e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb4bb17640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb44145540 0x7feb44144e10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feb34007a50 tx=0x7feb34012920 comp rx=0 tx=0).stop 2023-12-19T16:43:01.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb4bb17640 1 -- 172.21.15.26:0/2951892092 shutdown_connections 2023-12-19T16:43:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb4bb17640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feb28065a60 0x7feb28067f20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb4bb17640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb440c6600 0x7feb440c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb4bb17640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb44145540 0x7feb44144e10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb4bb17640 1 --2- 172.21.15.26:0/2951892092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb30005050 0x7feb440c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb4bb17640 1 -- 172.21.15.26:0/2951892092 >> 172.21.15.26:0/2951892092 conn(0x7feb440658e0 msgr2=0x7feb4415d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb4bb17640 1 -- 172.21.15.26:0/2951892092 shutdown_connections 2023-12-19T16:43:01.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:01.735+0000 7feb4bb17640 1 -- 172.21.15.26:0/2951892092 wait complete. 2023-12-19T16:43:01.761 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-12-19T16:43:01.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:01.761 DEBUG:teuthology.orchestra.run.smithi160:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-12-19T16:43:01.801 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi160.front.sepia.ceph.com 2023-12-19T16:43:01.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:01.801 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-19T16:43:01.861 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:43:01.861 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:43:01.871 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:43:01.871 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:43:01.922 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:43:01.922 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:43:01.975 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:43:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.155+0000 7fe5123e9640 1 Processor -- start 2023-12-19T16:43:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.155+0000 7fe5123e9640 1 -- start start 2023-12-19T16:43:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.155+0000 7fe5123e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe50c15c7f0 0x7fe50c06c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe5123e9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe50c0b30d0 0x7fe50c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe5123e9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe50c0b3a60 con 0x7fe50c0adb80 2023-12-19T16:43:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe5123e9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe50c06d770 con 0x7fe50c0b30d0 2023-12-19T16:43:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe5123e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c06ced0 con 0x7fe50c15c7f0 2023-12-19T16:43:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe50c15c7f0 0x7fe50c06c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe50c15c7f0 0x7fe50c06c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:57488/0 (socket says 172.21.15.26:57488) 2023-12-19T16:43:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe50bfff640 1 -- 172.21.15.26:0/1386652255 learned_addr learned my addr 172.21.15.26:0/1386652255 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe50b7fe640 1 --1- 172.21.15.26:0/1386652255 >> v1:172.21.15.160:6789/0 conn(0x7fe50c0adb80 0x7fe50c0cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:41900/0 (socket says 172.21.15.26:41900) 2023-12-19T16:43:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe510986640 1 --2- 172.21.15.26:0/1386652255 >> v2:172.21.15.160:3300/0 conn(0x7fe50c0b30d0 0x7fe50c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe50affd640 1 -- 172.21.15.26:0/1386652255 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773065673 0 0) 0x7fe50c0b3a60 con 0x7fe50c0adb80 2023-12-19T16:43:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe50affd640 1 -- 172.21.15.26:0/1386652255 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4f4003530 con 0x7fe50c0adb80 2023-12-19T16:43:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe50bfff640 1 -- 172.21.15.26:0/1386652255 >> v2:172.21.15.160:3300/0 conn(0x7fe50c0b30d0 msgr2=0x7fe50c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe50bfff640 1 --2- 172.21.15.26:0/1386652255 >> v2:172.21.15.160:3300/0 conn(0x7fe50c0b30d0 0x7fe50c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe50bfff640 1 -- 172.21.15.26:0/1386652255 >> v1:172.21.15.160:6789/0 conn(0x7fe50c0adb80 legacy=0x7fe50c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe50bfff640 1 -- 172.21.15.26:0/1386652255 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c06d1b0 con 0x7fe50c15c7f0 2023-12-19T16:43:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.159+0000 7fe510986640 1 --2- 172.21.15.26:0/1386652255 >> v2:172.21.15.160:3300/0 conn(0x7fe50c0b30d0 0x7fe50c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:02.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe50bfff640 1 --2- 172.21.15.26:0/1386652255 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe50c15c7f0 0x7fe50c06c7f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009870 tx=0x7fe4fc01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=687017060c306d1b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe50affd640 1 -- 172.21.15.26:0/1386652255 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0ac410 con 0x7fe50c15c7f0 2023-12-19T16:43:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe50affd640 1 -- 172.21.15.26:0/1386652255 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4fc0ac570 con 0x7fe50c15c7f0 2023-12-19T16:43:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe50affd640 1 -- 172.21.15.26:0/1386652255 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0ac900 con 0x7fe50c15c7f0 2023-12-19T16:43:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe5123e9640 1 -- 172.21.15.26:0/1386652255 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe50c15c7f0 msgr2=0x7fe50c06c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe5123e9640 1 --2- 172.21.15.26:0/1386652255 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe50c15c7f0 0x7fe50c06c7f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009870 tx=0x7fe4fc01c540 comp rx=0 tx=0).stop 2023-12-19T16:43:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe5123e9640 1 -- 172.21.15.26:0/1386652255 shutdown_connections 2023-12-19T16:43:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe5123e9640 1 --2- 172.21.15.26:0/1386652255 >> v2:172.21.15.160:3300/0 conn(0x7fe50c0b30d0 0x7fe50c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe5123e9640 1 --2- 172.21.15.26:0/1386652255 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe50c15c7f0 0x7fe50c06c7f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe5123e9640 1 -- 172.21.15.26:0/1386652255 >> 172.21.15.26:0/1386652255 conn(0x7fe50c0658e0 msgr2=0x7fe50c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe5123e9640 1 -- 172.21.15.26:0/1386652255 shutdown_connections 2023-12-19T16:43:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe5123e9640 1 -- 172.21.15.26:0/1386652255 wait complete. 2023-12-19T16:43:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe5123e9640 1 Processor -- start 2023-12-19T16:43:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.163+0000 7fe5123e9640 1 -- start start 2023-12-19T16:43:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe5123e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe50c0adb80 0x7fe50c14b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe5123e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe50c0b30d0 0x7fe50c154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe5123e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe50c15c7f0 0x7fe50c149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe5123e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c149470 con 0x7fe50c0adb80 2023-12-19T16:43:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe5123e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c154ff0 con 0x7fe50c0b30d0 2023-12-19T16:43:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe5123e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50c1552d0 con 0x7fe50c15c7f0 2023-12-19T16:43:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe50c0adb80 0x7fe50c14b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe50bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe50c0adb80 0x7fe50c14b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34190/0 (socket says 172.21.15.26:34190) 2023-12-19T16:43:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe50bfff640 1 -- 172.21.15.26:0/1288674521 learned_addr learned my addr 172.21.15.26:0/1288674521 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe50b7fe640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe50c0b30d0 0x7fe50c154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe510986640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe50c15c7f0 0x7fe50c149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe50bfff640 1 -- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe50c15c7f0 msgr2=0x7fe50c149c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe50bfff640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe50c15c7f0 0x7fe50c149c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe50bfff640 1 -- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe50c0b30d0 msgr2=0x7fe50c154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe50bfff640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe50c0b30d0 0x7fe50c154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe50bfff640 1 -- 172.21.15.26:0/1288674521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe504009a70 con 0x7fe50c0adb80 2023-12-19T16:43:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe50bfff640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe50c0adb80 0x7fe50c14b3b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0af040 tx=0x7fe4fc0192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe508ff9640 1 -- 172.21.15.26:0/1288674521 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0ac410 con 0x7fe50c0adb80 2023-12-19T16:43:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe508ff9640 1 -- 172.21.15.26:0/1288674521 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4fc0ace00 con 0x7fe50c0adb80 2023-12-19T16:43:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe508ff9640 1 -- 172.21.15.26:0/1288674521 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0b7890 con 0x7fe50c0adb80 2023-12-19T16:43:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe5123e9640 1 -- 172.21.15.26:0/1288674521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc0094d0 con 0x7fe50c0adb80 2023-12-19T16:43:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe5123e9640 1 -- 172.21.15.26:0/1288674521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50c1555b0 con 0x7fe50c0adb80 2023-12-19T16:43:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe5123e9640 1 -- 172.21.15.26:0/1288674521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe50c074270 con 0x7fe50c0adb80 2023-12-19T16:43:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.167+0000 7fe508ff9640 1 -- 172.21.15.26:0/1288674521 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe4fc0b3040 con 0x7fe50c0adb80 2023-12-19T16:43:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.171+0000 7fe508ff9640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe4f8065a60 0x7fe4f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.171+0000 7fe508ff9640 1 -- 172.21.15.26:0/1288674521 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4fc11d4a0 con 0x7fe50c0adb80 2023-12-19T16:43:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.171+0000 7fe50b7fe640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe4f8065a60 0x7fe4f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.171+0000 7fe50b7fe640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe4f8065a60 0x7fe4f8067f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe50c1545f0 tx=0x7fe500009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.175+0000 7fe508ff9640 1 -- 172.21.15.26:0/1288674521 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe4fc0ead30 con 0x7fe50c0adb80 2023-12-19T16:43:02.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.331+0000 7fe5123e9640 1 -- 172.21.15.26:0/1288674521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe50c1498e0 con 0x7fe50c0adb80 2023-12-19T16:43:02.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.335+0000 7fe508ff9640 1 -- 172.21.15.26:0/1288674521 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7fe4fc0ee350 con 0x7fe50c0adb80 2023-12-19T16:43:02.339 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:02.339 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:43:02.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.335+0000 7fe5123e9640 1 -- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe4f8065a60 msgr2=0x7fe4f8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.335+0000 7fe5123e9640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe4f8065a60 0x7fe4f8067f20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe50c1545f0 tx=0x7fe500009040 comp rx=0 tx=0).stop 2023-12-19T16:43:02.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.335+0000 7fe5123e9640 1 -- 172.21.15.26:0/1288674521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe50c0adb80 msgr2=0x7fe50c14b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.335+0000 7fe5123e9640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe50c0adb80 0x7fe50c14b3b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0af040 tx=0x7fe4fc0192e0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.335+0000 7fe5123e9640 1 -- 172.21.15.26:0/1288674521 shutdown_connections 2023-12-19T16:43:02.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.335+0000 7fe5123e9640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe4f8065a60 0x7fe4f8067f20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.335+0000 7fe5123e9640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe50c15c7f0 0x7fe50c149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.335+0000 7fe5123e9640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe50c0b30d0 0x7fe50c154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.335+0000 7fe5123e9640 1 --2- 172.21.15.26:0/1288674521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe50c0adb80 0x7fe50c14b3b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.335+0000 7fe5123e9640 1 -- 172.21.15.26:0/1288674521 >> 172.21.15.26:0/1288674521 conn(0x7fe50c0658e0 msgr2=0x7fe50c148be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.335+0000 7fe5123e9640 1 -- 172.21.15.26:0/1288674521 shutdown_connections 2023-12-19T16:43:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.335+0000 7fe5123e9640 1 -- 172.21.15.26:0/1288674521 wait complete. 2023-12-19T16:43:02.361 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:42:25.598550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:42:26.697362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:43:02.361 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:43:02.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.503+0000 7f5e1fdc0640 1 Processor -- start 2023-12-19T16:43:02.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.503+0000 7f5e1fdc0640 1 -- start start 2023-12-19T16:43:02.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.503+0000 7f5e1fdc0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e1806c3f0 0x7f5e1806c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.503+0000 7f5e1fdc0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e180c8df0 0x7f5e180c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.503+0000 7f5e1fdc0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e180b01b0 con 0x7f5e18161230 2023-12-19T16:43:02.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.503+0000 7f5e1fdc0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e180ca4e0 con 0x7f5e180c8df0 2023-12-19T16:43:02.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.503+0000 7f5e1fdc0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180b0400 con 0x7f5e1806c3f0 2023-12-19T16:43:02.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.503+0000 7f5e1e35d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5e180c8df0 0x7f5e180c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.503+0000 7f5e1db5c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5e18161230 0x7f5e180ada70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48602/0 (socket says 172.21.15.26:48602) 2023-12-19T16:43:02.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.503+0000 7f5e1db5c640 1 -- 172.21.15.26:0/2621772899 learned_addr learned my addr 172.21.15.26:0/2621772899 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:02.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.503+0000 7f5e1d35b640 1 --2- 172.21.15.26:0/2621772899 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e1806c3f0 0x7f5e1806c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1e35d640 1 -- 172.21.15.26:0/2621772899 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e1806c3f0 msgr2=0x7f5e1806c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:02.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1e35d640 1 --2- 172.21.15.26:0/2621772899 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e1806c3f0 0x7f5e1806c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1e35d640 1 -- 172.21.15.26:0/2621772899 >> v1:172.21.15.26:6789/0 conn(0x7f5e18161230 legacy=0x7f5e180ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1cb5a640 1 -- 172.21.15.26:0/2621772899 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082585366 0 0) 0x7f5e180b01b0 con 0x7f5e18161230 2023-12-19T16:43:02.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1e35d640 1 -- 172.21.15.26:0/2621772899 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e180b06e0 con 0x7f5e180c8df0 2023-12-19T16:43:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1e35d640 1 --2- 172.21.15.26:0/2621772899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e180c8df0 0x7f5e180c9280 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009d90 tx=0x7f5e1001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52b069d22d5ab87 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1cb5a640 1 -- 172.21.15.26:0/2621772899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e100a5970 con 0x7f5e180c8df0 2023-12-19T16:43:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1cb5a640 1 -- 172.21.15.26:0/2621772899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e100a5ad0 con 0x7f5e180c8df0 2023-12-19T16:43:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1cb5a640 1 -- 172.21.15.26:0/2621772899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e100a5e60 con 0x7f5e180c8df0 2023-12-19T16:43:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/2621772899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e180c8df0 msgr2=0x7f5e180c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1fdc0640 1 --2- 172.21.15.26:0/2621772899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e180c8df0 0x7f5e180c9280 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009d90 tx=0x7f5e1001c890 comp rx=0 tx=0).stop 2023-12-19T16:43:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/2621772899 shutdown_connections 2023-12-19T16:43:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1fdc0640 1 --2- 172.21.15.26:0/2621772899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e180c8df0 0x7f5e180c9280 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1fdc0640 1 --2- 172.21.15.26:0/2621772899 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e1806c3f0 0x7f5e1806c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/2621772899 >> 172.21.15.26:0/2621772899 conn(0x7f5e180ce730 msgr2=0x7f5e180ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/2621772899 shutdown_connections 2023-12-19T16:43:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.507+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/2621772899 wait complete. 2023-12-19T16:43:02.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1fdc0640 1 Processor -- start 2023-12-19T16:43:02.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1fdc0640 1 -- start start 2023-12-19T16:43:02.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1fdc0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e1806c3f0 0x7f5e181637c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1fdc0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e180c8df0 0x7f5e18156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1fdc0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e18161230 0x7f5e1815f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1fdc0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1815cc10 con 0x7f5e1806c3f0 2023-12-19T16:43:02.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1fdc0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1815b890 con 0x7f5e18161230 2023-12-19T16:43:02.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1fdc0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1815bb70 con 0x7f5e180c8df0 2023-12-19T16:43:02.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1db5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e1806c3f0 0x7f5e181637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1db5c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e1806c3f0 0x7f5e181637c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34206/0 (socket says 172.21.15.26:34206) 2023-12-19T16:43:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1db5c640 1 -- 172.21.15.26:0/3301690350 learned_addr learned my addr 172.21.15.26:0/3301690350 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1db5c640 1 -- 172.21.15.26:0/3301690350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e180c8df0 msgr2=0x7f5e18156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1db5c640 1 --2- 172.21.15.26:0/3301690350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e180c8df0 0x7f5e18156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1db5c640 1 -- 172.21.15.26:0/3301690350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e18161230 msgr2=0x7f5e1815f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1db5c640 1 --2- 172.21.15.26:0/3301690350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e18161230 0x7f5e1815f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1db5c640 1 -- 172.21.15.26:0/3301690350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e10009a70 con 0x7f5e1806c3f0 2023-12-19T16:43:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1db5c640 1 --2- 172.21.15.26:0/3301690350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e1806c3f0 0x7f5e181637c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5e140011f0 tx=0x7f5e1400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e0e7fc640 1 -- 172.21.15.26:0/3301690350 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e140074e0 con 0x7f5e1806c3f0 2023-12-19T16:43:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e0e7fc640 1 -- 172.21.15.26:0/3301690350 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e14007640 con 0x7f5e1806c3f0 2023-12-19T16:43:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e0e7fc640 1 -- 172.21.15.26:0/3301690350 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e14004a30 con 0x7f5e1806c3f0 2023-12-19T16:43:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/3301690350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1815be50 con 0x7f5e1806c3f0 2023-12-19T16:43:02.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.511+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/3301690350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1815c410 con 0x7f5e1806c3f0 2023-12-19T16:43:02.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.515+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/3301690350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e18074270 con 0x7f5e1806c3f0 2023-12-19T16:43:02.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.515+0000 7f5e0e7fc640 1 -- 172.21.15.26:0/3301690350 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5e14004b90 con 0x7f5e1806c3f0 2023-12-19T16:43:02.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.515+0000 7f5e0e7fc640 1 --2- 172.21.15.26:0/3301690350 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5dfc065a60 0x7f5dfc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.515+0000 7f5e0e7fc640 1 -- 172.21.15.26:0/3301690350 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5e14082540 con 0x7f5e1806c3f0 2023-12-19T16:43:02.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.515+0000 7f5e1d35b640 1 --2- 172.21.15.26:0/3301690350 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5dfc065a60 0x7f5dfc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.515+0000 7f5e1d35b640 1 --2- 172.21.15.26:0/3301690350 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5dfc065a60 0x7f5dfc067f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5e18163d00 tx=0x7f5e08008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:02.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.519+0000 7f5e0e7fc640 1 -- 172.21.15.26:0/3301690350 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5e1404fe60 con 0x7f5e1806c3f0 2023-12-19T16:43:02.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.675+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/3301690350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e180b46b0 con 0x7f5e1806c3f0 2023-12-19T16:43:02.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.679+0000 7f5e0e7fc640 1 -- 172.21.15.26:0/3301690350 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e140180a0 con 0x7f5e1806c3f0 2023-12-19T16:43:02.683 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:43:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.679+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/3301690350 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5dfc065a60 msgr2=0x7f5dfc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.679+0000 7f5e1fdc0640 1 --2- 172.21.15.26:0/3301690350 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5dfc065a60 0x7f5dfc067f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5e18163d00 tx=0x7f5e08008040 comp rx=0 tx=0).stop 2023-12-19T16:43:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.679+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/3301690350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e1806c3f0 msgr2=0x7f5e181637c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.679+0000 7f5e1fdc0640 1 --2- 172.21.15.26:0/3301690350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e1806c3f0 0x7f5e181637c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5e140011f0 tx=0x7f5e1400c9b0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.679+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/3301690350 shutdown_connections 2023-12-19T16:43:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.679+0000 7f5e1fdc0640 1 --2- 172.21.15.26:0/3301690350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e18161230 0x7f5e1815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.679+0000 7f5e1fdc0640 1 --2- 172.21.15.26:0/3301690350 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5dfc065a60 0x7f5dfc067f20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.679+0000 7f5e1fdc0640 1 --2- 172.21.15.26:0/3301690350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5e180c8df0 0x7f5e18156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.679+0000 7f5e1fdc0640 1 --2- 172.21.15.26:0/3301690350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5e1806c3f0 0x7f5e181637c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.679+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/3301690350 >> 172.21.15.26:0/3301690350 conn(0x7f5e180ce730 msgr2=0x7f5e180ccb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.679+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/3301690350 shutdown_connections 2023-12-19T16:43:02.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.679+0000 7f5e1fdc0640 1 -- 172.21.15.26:0/3301690350 wait complete. 2023-12-19T16:43:02.703 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:43:02.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.843+0000 7f765cd19640 1 Processor -- start 2023-12-19T16:43:02.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.843+0000 7f765cd19640 1 -- start start 2023-12-19T16:43:02.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f765cd19640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f76580b2af0 0x7f765806b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f765cd19640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7658071530 0x7f7658160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f765cd19640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765806bd40 con 0x7f76581602e0 2023-12-19T16:43:02.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f765cd19640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765806ef20 con 0x7f76580b2af0 2023-12-19T16:43:02.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f765cd19640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765806f1c0 con 0x7f7658071530 2023-12-19T16:43:02.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f76577fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f76581602e0 0x7f76580b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48610/0 (socket says 172.21.15.26:48610) 2023-12-19T16:43:02.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f76577fe640 1 -- 172.21.15.26:0/923174042 learned_addr learned my addr 172.21.15.26:0/923174042 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:02.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f7657fff640 1 --2- 172.21.15.26:0/923174042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7658071530 0x7f7658160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f7656ffd640 1 --2- 172.21.15.26:0/923174042 >> v2:172.21.15.160:3300/0 conn(0x7f76580b2af0 0x7f765806b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f76567fc640 1 -- 172.21.15.26:0/923174042 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807076225 0 0) 0x7f765806bd40 con 0x7f76581602e0 2023-12-19T16:43:02.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f76567fc640 1 -- 172.21.15.26:0/923174042 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7640003530 con 0x7f76581602e0 2023-12-19T16:43:02.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f7657fff640 1 -- 172.21.15.26:0/923174042 >> v2:172.21.15.160:3300/0 conn(0x7f76580b2af0 msgr2=0x7f765806b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:02.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f76567fc640 1 -- 172.21.15.26:0/923174042 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3846489672 0 0) 0x7f7640003530 con 0x7f76581602e0 2023-12-19T16:43:02.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f7657fff640 1 --2- 172.21.15.26:0/923174042 >> v2:172.21.15.160:3300/0 conn(0x7f76580b2af0 0x7f765806b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f7657fff640 1 -- 172.21.15.26:0/923174042 >> v1:172.21.15.26:6789/0 conn(0x7f76581602e0 legacy=0x7f76580b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f7657fff640 1 -- 172.21.15.26:0/923174042 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765806f4a0 con 0x7f7658071530 2023-12-19T16:43:02.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.847+0000 7f7656ffd640 1 --2- 172.21.15.26:0/923174042 >> v2:172.21.15.160:3300/0 conn(0x7f76580b2af0 0x7f765806b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:02.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f7657fff640 1 --2- 172.21.15.26:0/923174042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7658071530 0x7f7658160ab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7650009d90 tx=0x7f765001c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d511b647a77bede server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:02.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f76567fc640 1 -- 172.21.15.26:0/923174042 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648001680 con 0x7f7658071530 2023-12-19T16:43:02.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f76567fc640 1 -- 172.21.15.26:0/923174042 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76500a6500 con 0x7f7658071530 2023-12-19T16:43:02.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f76567fc640 1 -- 172.21.15.26:0/923174042 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76500a6890 con 0x7f7658071530 2023-12-19T16:43:02.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f765cd19640 1 -- 172.21.15.26:0/923174042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7658071530 msgr2=0x7f7658160ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f765cd19640 1 --2- 172.21.15.26:0/923174042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7658071530 0x7f7658160ab0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7650009d90 tx=0x7f765001c870 comp rx=0 tx=0).stop 2023-12-19T16:43:02.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f765cd19640 1 -- 172.21.15.26:0/923174042 shutdown_connections 2023-12-19T16:43:02.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f765cd19640 1 --2- 172.21.15.26:0/923174042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7658071530 0x7f7658160ab0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f765cd19640 1 --2- 172.21.15.26:0/923174042 >> v2:172.21.15.160:3300/0 conn(0x7f76580b2af0 0x7f765806b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f765cd19640 1 -- 172.21.15.26:0/923174042 >> 172.21.15.26:0/923174042 conn(0x7f76580cd3e0 msgr2=0x7f76580cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:02.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f765cd19640 1 -- 172.21.15.26:0/923174042 shutdown_connections 2023-12-19T16:43:02.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f765cd19640 1 -- 172.21.15.26:0/923174042 wait complete. 2023-12-19T16:43:02.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f765cd19640 1 Processor -- start 2023-12-19T16:43:02.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.851+0000 7f765cd19640 1 -- start start 2023-12-19T16:43:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f765cd19640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7658071530 0x7f76581f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f765cd19640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76580b2af0 0x7f76580c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f765cd19640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76581602e0 0x7f76581f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f765cd19640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76580b24a0 con 0x7f76580b2af0 2023-12-19T16:43:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f765cd19640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76580c5e60 con 0x7f76581602e0 2023-12-19T16:43:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f765cd19640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76580c6120 con 0x7f7658071530 2023-12-19T16:43:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f7656ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76580b2af0 0x7f76580c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f7656ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76580b2af0 0x7f76580c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34218/0 (socket says 172.21.15.26:34218) 2023-12-19T16:43:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f7656ffd640 1 -- 172.21.15.26:0/1389536255 learned_addr learned my addr 172.21.15.26:0/1389536255 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f7657fff640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76581602e0 0x7f76581f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f76577fe640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7658071530 0x7f76581f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f7656ffd640 1 -- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7658071530 msgr2=0x7f76581f5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f7656ffd640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7658071530 0x7f76581f5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f7656ffd640 1 -- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76581602e0 msgr2=0x7f76581f8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f7656ffd640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76581602e0 0x7f76581f8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f7656ffd640 1 -- 172.21.15.26:0/1389536255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c009420 con 0x7f76580b2af0 2023-12-19T16:43:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f7656ffd640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76580b2af0 0x7f76580c5920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f764c00ebe0 tx=0x7f764c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f763ffff640 1 -- 172.21.15.26:0/1389536255 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c00eee0 con 0x7f76580b2af0 2023-12-19T16:43:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f763ffff640 1 -- 172.21.15.26:0/1389536255 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c00cc30 con 0x7f76580b2af0 2023-12-19T16:43:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f763ffff640 1 -- 172.21.15.26:0/1389536255 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c013470 con 0x7f76580b2af0 2023-12-19T16:43:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f765cd19640 1 -- 172.21.15.26:0/1389536255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7650009a70 con 0x7f76580b2af0 2023-12-19T16:43:02.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f765cd19640 1 -- 172.21.15.26:0/1389536255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76580c66e0 con 0x7f76580b2af0 2023-12-19T16:43:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f765cd19640 1 -- 172.21.15.26:0/1389536255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7658061930 con 0x7f76580b2af0 2023-12-19T16:43:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.855+0000 7f763ffff640 1 -- 172.21.15.26:0/1389536255 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f764c0135d0 con 0x7f76580b2af0 2023-12-19T16:43:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.859+0000 7f763ffff640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7644065a60 0x7f7644067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.859+0000 7f763ffff640 1 -- 172.21.15.26:0/1389536255 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f764c0834f0 con 0x7f76580b2af0 2023-12-19T16:43:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.859+0000 7f76577fe640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7644065a60 0x7f7644067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.859+0000 7f76577fe640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7644065a60 0x7f7644067f20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7648001ec0 tx=0x7f76480019c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:02.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:02.863+0000 7f763ffff640 1 -- 172.21.15.26:0/1389536255 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f764c050ec0 con 0x7f76580b2af0 2023-12-19T16:43:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f765cd19640 1 -- 172.21.15.26:0/1389536255 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76580d03f0 con 0x7f76580b2af0 2023-12-19T16:43:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f763ffff640 1 -- 172.21.15.26:0/1389536255 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f764c0544e0 con 0x7f76580b2af0 2023-12-19T16:43:03.023 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:43:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f765cd19640 1 -- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7644065a60 msgr2=0x7f7644067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f765cd19640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7644065a60 0x7f7644067f20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7648001ec0 tx=0x7f76480019c0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f765cd19640 1 -- 172.21.15.26:0/1389536255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76580b2af0 msgr2=0x7f76580c5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f765cd19640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76580b2af0 0x7f76580c5920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f764c00ebe0 tx=0x7f764c00c520 comp rx=0 tx=0).stop 2023-12-19T16:43:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f765cd19640 1 -- 172.21.15.26:0/1389536255 shutdown_connections 2023-12-19T16:43:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f765cd19640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76581602e0 0x7f76581f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f765cd19640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76580b2af0 0x7f76580c5920 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f765cd19640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7644065a60 0x7f7644067f20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f765cd19640 1 --2- 172.21.15.26:0/1389536255 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7658071530 0x7f76581f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f765cd19640 1 -- 172.21.15.26:0/1389536255 >> 172.21.15.26:0/1389536255 conn(0x7f76580cd3e0 msgr2=0x7f7658071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f765cd19640 1 -- 172.21.15.26:0/1389536255 shutdown_connections 2023-12-19T16:43:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.019+0000 7f765cd19640 1 -- 172.21.15.26:0/1389536255 wait complete. 2023-12-19T16:43:03.045 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:43:03.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01eefc640 1 Processor -- start 2023-12-19T16:43:03.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01eefc640 1 -- start start 2023-12-19T16:43:03.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01eefc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa018159820 0x7fa01815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:03.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01eefc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa01806f6c0 0x7fa0180c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:03.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01eefc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01806fb60 con 0x7fa018159c00 2023-12-19T16:43:03.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01eefc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01815e470 con 0x7fa018159820 2023-12-19T16:43:03.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01eefc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01806fe40 con 0x7fa01806f6c0 2023-12-19T16:43:03.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01cc98640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa018159820 0x7fa01815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:03.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01cc98640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa018159820 0x7fa01815df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34224/0 (socket says 172.21.15.26:34224) 2023-12-19T16:43:03.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01cc98640 1 -- 172.21.15.26:0/3051733333 learned_addr learned my addr 172.21.15.26:0/3051733333 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:03.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01cc98640 1 -- 172.21.15.26:0/3051733333 >> v2:172.21.15.160:3300/0 conn(0x7fa01806f6c0 msgr2=0x7fa0180c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:03.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01cc98640 1 --2- 172.21.15.26:0/3051733333 >> v2:172.21.15.160:3300/0 conn(0x7fa01806f6c0 0x7fa0180c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01cc98640 1 -- 172.21.15.26:0/3051733333 >> v1:172.21.15.160:6789/0 conn(0x7fa018159c00 legacy=0x7fa018070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:03.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.187+0000 7fa01cc98640 1 -- 172.21.15.26:0/3051733333 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa018070120 con 0x7fa018159820 2023-12-19T16:43:03.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.191+0000 7fa01cc98640 1 --2- 172.21.15.26:0/3051733333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa018159820 0x7fa01815df30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa0180caf30 tx=0x7fa00c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f33301a368e42f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:03.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.191+0000 7fa0177fe640 1 -- 172.21.15.26:0/3051733333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0aa5f0 con 0x7fa018159820 2023-12-19T16:43:03.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.191+0000 7fa0177fe640 1 -- 172.21.15.26:0/3051733333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c0aa750 con 0x7fa018159820 2023-12-19T16:43:03.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.191+0000 7fa0177fe640 1 -- 172.21.15.26:0/3051733333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0aaae0 con 0x7fa018159820 2023-12-19T16:43:03.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.191+0000 7fa01eefc640 1 -- 172.21.15.26:0/3051733333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa018159820 msgr2=0x7fa01815df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:03.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.191+0000 7fa01eefc640 1 --2- 172.21.15.26:0/3051733333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa018159820 0x7fa01815df30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa0180caf30 tx=0x7fa00c01c490 comp rx=0 tx=0).stop 2023-12-19T16:43:03.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.191+0000 7fa01eefc640 1 -- 172.21.15.26:0/3051733333 shutdown_connections 2023-12-19T16:43:03.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.191+0000 7fa01eefc640 1 --2- 172.21.15.26:0/3051733333 >> v2:172.21.15.160:3300/0 conn(0x7fa01806f6c0 0x7fa0180c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.191+0000 7fa01eefc640 1 --2- 172.21.15.26:0/3051733333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa018159820 0x7fa01815df30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.191+0000 7fa01eefc640 1 -- 172.21.15.26:0/3051733333 >> 172.21.15.26:0/3051733333 conn(0x7fa0180ce730 msgr2=0x7fa0180ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:03.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.191+0000 7fa01eefc640 1 -- 172.21.15.26:0/3051733333 shutdown_connections 2023-12-19T16:43:03.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.191+0000 7fa01eefc640 1 -- 172.21.15.26:0/3051733333 wait complete. 2023-12-19T16:43:03.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa01eefc640 1 Processor -- start 2023-12-19T16:43:03.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa01eefc640 1 -- start start 2023-12-19T16:43:03.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa01eefc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01806f6c0 0x7fa01814cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:03.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa01eefc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa018159820 0x7fa01815c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:03.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa01eefc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa018159c00 0x7fa018158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:03.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa01eefc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01814ebf0 con 0x7fa018159820 2023-12-19T16:43:03.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa01eefc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01814e8a0 con 0x7fa01806f6c0 2023-12-19T16:43:03.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa01eefc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa018152650 con 0x7fa018159c00 2023-12-19T16:43:03.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa017fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa018159820 0x7fa01815c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:03.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa017fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa018159820 0x7fa01815c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34228/0 (socket says 172.21.15.26:34228) 2023-12-19T16:43:03.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa017fff640 1 -- 172.21.15.26:0/3819803057 learned_addr learned my addr 172.21.15.26:0/3819803057 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:03.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa01d499640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa018159c00 0x7fa018158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:03.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa01cc98640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01806f6c0 0x7fa01814cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:03.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa017fff640 1 -- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa018159c00 msgr2=0x7fa018158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:03.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa017fff640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa018159c00 0x7fa018158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa017fff640 1 -- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01806f6c0 msgr2=0x7fa01814cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:03.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa017fff640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01806f6c0 0x7fa01814cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa017fff640 1 -- 172.21.15.26:0/3819803057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c0094d0 con 0x7fa018159820 2023-12-19T16:43:03.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa017fff640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa018159820 0x7fa01815c750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa00400b3f0 tx=0x7fa00400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:03.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa0157fa640 1 -- 172.21.15.26:0/3819803057 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004011830 con 0x7fa018159820 2023-12-19T16:43:03.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa0157fa640 1 -- 172.21.15.26:0/3819803057 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa004011990 con 0x7fa018159820 2023-12-19T16:43:03.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa0157fa640 1 -- 172.21.15.26:0/3819803057 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004011d40 con 0x7fa018159820 2023-12-19T16:43:03.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa01eefc640 1 -- 172.21.15.26:0/3819803057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01806b880 con 0x7fa018159820 2023-12-19T16:43:03.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa01eefc640 1 -- 172.21.15.26:0/3819803057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa018152c10 con 0x7fa018159820 2023-12-19T16:43:03.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.195+0000 7fa01eefc640 1 -- 172.21.15.26:0/3819803057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01814ed60 con 0x7fa018159820 2023-12-19T16:43:03.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.199+0000 7fa0157fa640 1 -- 172.21.15.26:0/3819803057 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa00401a400 con 0x7fa018159820 2023-12-19T16:43:03.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.199+0000 7fa0157fa640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9ffc065a60 0x7f9ffc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:03.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.199+0000 7fa0157fa640 1 -- 172.21.15.26:0/3819803057 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa004083150 con 0x7fa018159820 2023-12-19T16:43:03.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.199+0000 7fa01cc98640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9ffc065a60 0x7f9ffc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:03.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.199+0000 7fa01cc98640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9ffc065a60 0x7f9ffc067f20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0ae000 tx=0x7fa00c0b5040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:03.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.203+0000 7fa0157fa640 1 -- 172.21.15.26:0/3819803057 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa004050bc0 con 0x7fa018159820 2023-12-19T16:43:03.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.355+0000 7fa01eefc640 1 -- 172.21.15.26:0/3819803057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa018074270 con 0x7fa018159820 2023-12-19T16:43:03.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.359+0000 7fa0157fa640 1 -- 172.21.15.26:0/3819803057 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa004015020 con 0x7fa018159820 2023-12-19T16:43:03.363 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:43:03.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.359+0000 7fa01eefc640 1 -- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9ffc065a60 msgr2=0x7f9ffc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:03.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.359+0000 7fa01eefc640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9ffc065a60 0x7f9ffc067f20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0ae000 tx=0x7fa00c0b5040 comp rx=0 tx=0).stop 2023-12-19T16:43:03.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.359+0000 7fa01eefc640 1 -- 172.21.15.26:0/3819803057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa018159820 msgr2=0x7fa01815c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:03.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.359+0000 7fa01eefc640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa018159820 0x7fa01815c750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa00400b3f0 tx=0x7fa00400b8b0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.359+0000 7fa01eefc640 1 -- 172.21.15.26:0/3819803057 shutdown_connections 2023-12-19T16:43:03.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.359+0000 7fa01eefc640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9ffc065a60 0x7f9ffc067f20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.359+0000 7fa01eefc640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa018159c00 0x7fa018158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.359+0000 7fa01eefc640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa018159820 0x7fa01815c750 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.359+0000 7fa01eefc640 1 --2- 172.21.15.26:0/3819803057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa01806f6c0 0x7fa01814cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.359+0000 7fa01eefc640 1 -- 172.21.15.26:0/3819803057 >> 172.21.15.26:0/3819803057 conn(0x7fa0180ce730 msgr2=0x7fa0180af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:03.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.359+0000 7fa01eefc640 1 -- 172.21.15.26:0/3819803057 shutdown_connections 2023-12-19T16:43:03.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.359+0000 7fa01eefc640 1 -- 172.21.15.26:0/3819803057 wait complete. 2023-12-19T16:43:03.385 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T16:43:03.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.523+0000 7fe897917640 1 Processor -- start 2023-12-19T16:43:03.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.523+0000 7fe897917640 1 -- start start 2023-12-19T16:43:03.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe897917640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe890154220 0x7fe89015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:03.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe897917640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe8900cee10 0x7fe89015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:03.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe897917640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89006bc30 con 0x7fe89006b3d0 2023-12-19T16:43:03.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe897917640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8900cc6e0 con 0x7fe890154220 2023-12-19T16:43:03.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe897917640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89015afc0 con 0x7fe8900cee10 2023-12-19T16:43:03.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe894eb2640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe89006b3d0 0x7fe8900ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48616/0 (socket says 172.21.15.26:48616) 2023-12-19T16:43:03.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe894eb2640 1 -- 172.21.15.26:0/2831756211 learned_addr learned my addr 172.21.15.26:0/2831756211 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:03.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe8956b3640 1 --2- 172.21.15.26:0/2831756211 >> v2:172.21.15.26:3300/0 conn(0x7fe890154220 0x7fe89015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:03.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe895eb4640 1 --2- 172.21.15.26:0/2831756211 >> v2:172.21.15.160:3300/0 conn(0x7fe8900cee10 0x7fe89015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:03.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe8956b3640 1 -- 172.21.15.26:0/2831756211 >> v2:172.21.15.160:3300/0 conn(0x7fe8900cee10 msgr2=0x7fe89015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:03.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe8956b3640 1 --2- 172.21.15.26:0/2831756211 >> v2:172.21.15.160:3300/0 conn(0x7fe8900cee10 0x7fe89015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe8956b3640 1 -- 172.21.15.26:0/2831756211 >> v1:172.21.15.26:6789/0 conn(0x7fe89006b3d0 legacy=0x7fe8900ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:03.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe887fff640 1 -- 172.21.15.26:0/2831756211 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638791822 0 0) 0x7fe89006bc30 con 0x7fe89006b3d0 2023-12-19T16:43:03.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe8956b3640 1 -- 172.21.15.26:0/2831756211 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89006b950 con 0x7fe890154220 2023-12-19T16:43:03.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe895eb4640 1 --2- 172.21.15.26:0/2831756211 >> v2:172.21.15.160:3300/0 conn(0x7fe8900cee10 0x7fe89015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:03.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.527+0000 7fe8956b3640 1 --2- 172.21.15.26:0/2831756211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe890154220 0x7fe89015aa80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe8900cdf30 tx=0x7fe88c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f626d617076a33 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:03.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe887fff640 1 -- 172.21.15.26:0/2831756211 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c0a7070 con 0x7fe890154220 2023-12-19T16:43:03.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe887fff640 1 -- 172.21.15.26:0/2831756211 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c0ab430 con 0x7fe890154220 2023-12-19T16:43:03.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe887fff640 1 -- 172.21.15.26:0/2831756211 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c0ab7c0 con 0x7fe890154220 2023-12-19T16:43:03.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 -- 172.21.15.26:0/2831756211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe890154220 msgr2=0x7fe89015aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:03.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 --2- 172.21.15.26:0/2831756211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe890154220 0x7fe89015aa80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe8900cdf30 tx=0x7fe88c01c430 comp rx=0 tx=0).stop 2023-12-19T16:43:03.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 -- 172.21.15.26:0/2831756211 shutdown_connections 2023-12-19T16:43:03.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 --2- 172.21.15.26:0/2831756211 >> v2:172.21.15.160:3300/0 conn(0x7fe8900cee10 0x7fe89015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 --2- 172.21.15.26:0/2831756211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe890154220 0x7fe89015aa80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 -- 172.21.15.26:0/2831756211 >> 172.21.15.26:0/2831756211 conn(0x7fe8900658e0 msgr2=0x7fe890065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:03.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 -- 172.21.15.26:0/2831756211 shutdown_connections 2023-12-19T16:43:03.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 -- 172.21.15.26:0/2831756211 wait complete. 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 Processor -- start 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 -- start start 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89006b3d0 0x7fe890150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe8900cee10 0x7fe89014e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe890154220 0x7fe89014ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8900b39d0 con 0x7fe89006b3d0 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe890150510 con 0x7fe890154220 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe897917640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8900b2c10 con 0x7fe8900cee10 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe8956b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89006b3d0 0x7fe890150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe8956b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89006b3d0 0x7fe890150b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34250/0 (socket says 172.21.15.26:34250) 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.531+0000 7fe8956b3640 1 -- 172.21.15.26:0/1887172026 learned_addr learned my addr 172.21.15.26:0/1887172026 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.535+0000 7fe8956b3640 1 -- 172.21.15.26:0/1887172026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe8900cee10 msgr2=0x7fe89014e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.535+0000 7fe8956b3640 1 --2- 172.21.15.26:0/1887172026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe8900cee10 0x7fe89014e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.535+0000 7fe8956b3640 1 -- 172.21.15.26:0/1887172026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe890154220 msgr2=0x7fe89014ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.535+0000 7fe8956b3640 1 --2- 172.21.15.26:0/1887172026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe890154220 0x7fe89014ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.535+0000 7fe8956b3640 1 -- 172.21.15.26:0/1887172026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c009420 con 0x7fe89006b3d0 2023-12-19T16:43:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.535+0000 7fe8956b3640 1 --2- 172.21.15.26:0/1887172026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89006b3d0 0x7fe890150b50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0ad040 tx=0x7fe88c01ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:03.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.535+0000 7fe885ffb640 1 -- 172.21.15.26:0/1887172026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c0b5810 con 0x7fe89006b3d0 2023-12-19T16:43:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.535+0000 7fe885ffb640 1 -- 172.21.15.26:0/1887172026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c0b5970 con 0x7fe89006b3d0 2023-12-19T16:43:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.535+0000 7fe885ffb640 1 -- 172.21.15.26:0/1887172026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c0b5d20 con 0x7fe89006b3d0 2023-12-19T16:43:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.535+0000 7fe897917640 1 -- 172.21.15.26:0/1887172026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8900b2ef0 con 0x7fe89006b3d0 2023-12-19T16:43:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.535+0000 7fe897917640 1 -- 172.21.15.26:0/1887172026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8900b34b0 con 0x7fe89006b3d0 2023-12-19T16:43:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.535+0000 7fe897917640 1 -- 172.21.15.26:0/1887172026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8900cced0 con 0x7fe89006b3d0 2023-12-19T16:43:03.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.535+0000 7fe885ffb640 1 -- 172.21.15.26:0/1887172026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe88c0bf410 con 0x7fe89006b3d0 2023-12-19T16:43:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.539+0000 7fe885ffb640 1 --2- 172.21.15.26:0/1887172026 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe874065a60 0x7fe874067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:03.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.539+0000 7fe885ffb640 1 -- 172.21.15.26:0/1887172026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe88c11db30 con 0x7fe89006b3d0 2023-12-19T16:43:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.539+0000 7fe894eb2640 1 --2- 172.21.15.26:0/1887172026 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe874065a60 0x7fe874067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:03.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.539+0000 7fe894eb2640 1 --2- 172.21.15.26:0/1887172026 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe874065a60 0x7fe874067f20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe890159020 tx=0x7fe880007970 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:03.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.539+0000 7fe885ffb640 1 -- 172.21.15.26:0/1887172026 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe8900cced0 con 0x7fe89006b3d0 2023-12-19T16:43:03.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.711+0000 7fe897917640 1 -- 172.21.15.26:0/1887172026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe8900cced0 con 0x7fe89006b3d0 2023-12-19T16:43:03.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.711+0000 7fe885ffb640 1 -- 172.21.15.26:0/1887172026 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe8900cced0 con 0x7fe89006b3d0 2023-12-19T16:43:03.716 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2023-12-19T16:43:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.711+0000 7fe897917640 1 -- 172.21.15.26:0/1887172026 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe874065a60 msgr2=0x7fe874067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:03.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.711+0000 7fe897917640 1 --2- 172.21.15.26:0/1887172026 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe874065a60 0x7fe874067f20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe890159020 tx=0x7fe880007970 comp rx=0 tx=0).stop 2023-12-19T16:43:03.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.711+0000 7fe897917640 1 -- 172.21.15.26:0/1887172026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89006b3d0 msgr2=0x7fe890150b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:03.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.711+0000 7fe897917640 1 --2- 172.21.15.26:0/1887172026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89006b3d0 0x7fe890150b50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0ad040 tx=0x7fe88c01ac70 comp rx=0 tx=0).stop 2023-12-19T16:43:03.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.711+0000 7fe897917640 1 -- 172.21.15.26:0/1887172026 shutdown_connections 2023-12-19T16:43:03.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.711+0000 7fe897917640 1 --2- 172.21.15.26:0/1887172026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe890154220 0x7fe89014ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.715+0000 7fe897917640 1 --2- 172.21.15.26:0/1887172026 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe874065a60 0x7fe874067f20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.715+0000 7fe897917640 1 --2- 172.21.15.26:0/1887172026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe8900cee10 0x7fe89014e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.715+0000 7fe897917640 1 --2- 172.21.15.26:0/1887172026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89006b3d0 0x7fe890150b50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:03.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.715+0000 7fe897917640 1 -- 172.21.15.26:0/1887172026 >> 172.21.15.26:0/1887172026 conn(0x7fe8900658e0 msgr2=0x7fe890065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:03.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.715+0000 7fe897917640 1 -- 172.21.15.26:0/1887172026 shutdown_connections 2023-12-19T16:43:03.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:03.715+0000 7fe897917640 1 -- 172.21.15.26:0/1887172026 wait complete. 2023-12-19T16:43:03.738 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-12-19T16:43:03.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:03.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-12-19T16:43:03.762 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T16:43:03.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:03.763 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-19T16:43:03.820 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:43:03.820 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:43:03.831 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:43:03.831 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:43:03.884 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:43:03.884 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:43:03.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:43:04.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f097de8c640 1 Processor -- start 2023-12-19T16:43:04.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f097de8c640 1 -- start start 2023-12-19T16:43:04.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f097de8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09780b25d0 0x7f097806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f097de8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09780cd330 0x7f09780cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f097de8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097806b510 con 0x7f09780d1290 2023-12-19T16:43:04.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f097de8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f097806abb0 con 0x7f09780cd330 2023-12-19T16:43:04.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f097de8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09780cdd00 con 0x7f09780b25d0 2023-12-19T16:43:04.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f0977fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09780cd330 0x7f09780cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f09777fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f09780d1290 0x7f09780b2dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48622/0 (socket says 172.21.15.26:48622) 2023-12-19T16:43:04.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f09777fe640 1 -- 172.21.15.26:0/4190193832 learned_addr learned my addr 172.21.15.26:0/4190193832 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:04.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f0976ffd640 1 --2- 172.21.15.26:0/4190193832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09780b25d0 0x7f097806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f0977fff640 1 -- 172.21.15.26:0/4190193832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09780b25d0 msgr2=0x7f097806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:04.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f0977fff640 1 --2- 172.21.15.26:0/4190193832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09780b25d0 0x7f097806afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f0977fff640 1 -- 172.21.15.26:0/4190193832 >> v1:172.21.15.26:6789/0 conn(0x7f09780d1290 legacy=0x7f09780b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f0977fff640 1 -- 172.21.15.26:0/4190193832 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097806b790 con 0x7f09780cd330 2023-12-19T16:43:04.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.075+0000 7f09767fc640 1 -- 172.21.15.26:0/4190193832 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289387147 0 0) 0x7f097806b510 con 0x7f09780d1290 2023-12-19T16:43:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.079+0000 7f0977fff640 1 --2- 172.21.15.26:0/4190193832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09780cd330 0x7f09780cd7c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0970009d90 tx=0x7f097001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cc70594297a0e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.079+0000 7f09767fc640 1 -- 172.21.15.26:0/4190193832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09700a5870 con 0x7f09780cd330 2023-12-19T16:43:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.079+0000 7f09767fc640 1 -- 172.21.15.26:0/4190193832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09700a59d0 con 0x7f09780cd330 2023-12-19T16:43:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.079+0000 7f09767fc640 1 -- 172.21.15.26:0/4190193832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09700a5d60 con 0x7f09780cd330 2023-12-19T16:43:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.079+0000 7f097de8c640 1 -- 172.21.15.26:0/4190193832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09780cd330 msgr2=0x7f09780cd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.079+0000 7f097de8c640 1 --2- 172.21.15.26:0/4190193832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09780cd330 0x7f09780cd7c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0970009d90 tx=0x7f097001c8c0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.079+0000 7f097de8c640 1 -- 172.21.15.26:0/4190193832 shutdown_connections 2023-12-19T16:43:04.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.079+0000 7f097de8c640 1 --2- 172.21.15.26:0/4190193832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09780cd330 0x7f09780cd7c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.079+0000 7f097de8c640 1 --2- 172.21.15.26:0/4190193832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09780b25d0 0x7f097806afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.079+0000 7f097de8c640 1 -- 172.21.15.26:0/4190193832 >> 172.21.15.26:0/4190193832 conn(0x7f097806d570 msgr2=0x7f097806d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:04.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.079+0000 7f097de8c640 1 -- 172.21.15.26:0/4190193832 shutdown_connections 2023-12-19T16:43:04.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.079+0000 7f097de8c640 1 -- 172.21.15.26:0/4190193832 wait complete. 2023-12-19T16:43:04.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f097de8c640 1 Processor -- start 2023-12-19T16:43:04.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f097de8c640 1 -- start start 2023-12-19T16:43:04.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f097de8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09780b25d0 0x7f09780c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f097de8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09780cd330 0x7f097815cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f097de8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09780d1290 0x7f0978146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f097de8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09780c6c50 con 0x7f09780d1290 2023-12-19T16:43:04.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f097de8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09780c6040 con 0x7f09780b25d0 2023-12-19T16:43:04.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f097de8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09780c6320 con 0x7f09780cd330 2023-12-19T16:43:04.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f0977fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09780d1290 0x7f0978146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f0977fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09780d1290 0x7f0978146c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34272/0 (socket says 172.21.15.26:34272) 2023-12-19T16:43:04.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f0977fff640 1 -- 172.21.15.26:0/2478026691 learned_addr learned my addr 172.21.15.26:0/2478026691 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:04.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f0976ffd640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09780cd330 0x7f097815cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f09777fe640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09780b25d0 0x7f09780c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f0977fff640 1 -- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09780cd330 msgr2=0x7f097815cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f0977fff640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09780cd330 0x7f097815cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f0977fff640 1 -- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09780b25d0 msgr2=0x7f09780c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f0977fff640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09780b25d0 0x7f09780c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f0977fff640 1 -- 172.21.15.26:0/2478026691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0970009a70 con 0x7f09780d1290 2023-12-19T16:43:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f0977fff640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09780d1290 0x7f0978146c10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f097001af70 tx=0x7f0970019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f095ffff640 1 -- 172.21.15.26:0/2478026691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09700ad850 con 0x7f09780d1290 2023-12-19T16:43:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f095ffff640 1 -- 172.21.15.26:0/2478026691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09700ad9b0 con 0x7f09780d1290 2023-12-19T16:43:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f095ffff640 1 -- 172.21.15.26:0/2478026691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09700add20 con 0x7f09780d1290 2023-12-19T16:43:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f097de8c640 1 -- 172.21.15.26:0/2478026691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09780c6600 con 0x7f09780d1290 2023-12-19T16:43:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.083+0000 7f097de8c640 1 -- 172.21.15.26:0/2478026691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097815d1e0 con 0x7f09780d1290 2023-12-19T16:43:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.087+0000 7f097de8c640 1 -- 172.21.15.26:0/2478026691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09780c6dc0 con 0x7f09780d1290 2023-12-19T16:43:04.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.087+0000 7f095ffff640 1 -- 172.21.15.26:0/2478026691 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f09700b4050 con 0x7f09780d1290 2023-12-19T16:43:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.087+0000 7f095ffff640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0964065a60 0x7f0964067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.087+0000 7f095ffff640 1 -- 172.21.15.26:0/2478026691 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f097011e520 con 0x7f09780d1290 2023-12-19T16:43:04.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.087+0000 7f09777fe640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0964065a60 0x7f0964067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.087+0000 7f09777fe640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0964065a60 0x7f0964067f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0968007fd0 tx=0x7f0968007d60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:04.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.091+0000 7f095ffff640 1 -- 172.21.15.26:0/2478026691 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f09700ebef0 con 0x7f09780d1290 2023-12-19T16:43:04.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.247+0000 7f097de8c640 1 -- 172.21.15.26:0/2478026691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0978074270 con 0x7f09780d1290 2023-12-19T16:43:04.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.251+0000 7f095ffff640 1 -- 172.21.15.26:0/2478026691 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f09700ef510 con 0x7f09780d1290 2023-12-19T16:43:04.255 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:04.255 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:43:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.251+0000 7f097de8c640 1 -- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0964065a60 msgr2=0x7f0964067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.251+0000 7f097de8c640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0964065a60 0x7f0964067f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0968007fd0 tx=0x7f0968007d60 comp rx=0 tx=0).stop 2023-12-19T16:43:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.251+0000 7f097de8c640 1 -- 172.21.15.26:0/2478026691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09780d1290 msgr2=0x7f0978146c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.251+0000 7f097de8c640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09780d1290 0x7f0978146c10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f097001af70 tx=0x7f0970019660 comp rx=0 tx=0).stop 2023-12-19T16:43:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.251+0000 7f097de8c640 1 -- 172.21.15.26:0/2478026691 shutdown_connections 2023-12-19T16:43:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.251+0000 7f097de8c640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09780d1290 0x7f0978146c10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.251+0000 7f097de8c640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0964065a60 0x7f0964067f20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.251+0000 7f097de8c640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09780cd330 0x7f097815cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.251+0000 7f097de8c640 1 --2- 172.21.15.26:0/2478026691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09780b25d0 0x7f09780c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.251+0000 7f097de8c640 1 -- 172.21.15.26:0/2478026691 >> 172.21.15.26:0/2478026691 conn(0x7f097806d570 msgr2=0x7f09780b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.251+0000 7f097de8c640 1 -- 172.21.15.26:0/2478026691 shutdown_connections 2023-12-19T16:43:04.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.251+0000 7f097de8c640 1 -- 172.21.15.26:0/2478026691 wait complete. 2023-12-19T16:43:04.276 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:42:25.598550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:42:26.697362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:43:04.277 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:43:04.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f9189739640 1 Processor -- start 2023-12-19T16:43:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f9189739640 1 -- start start 2023-12-19T16:43:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f9189739640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f918415ccf0 0x7f91840d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f9189739640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f918415d0d0 0x7f91840b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f9189739640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f918415e200 con 0x7f91840b4600 2023-12-19T16:43:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f9189739640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91840d0aa0 con 0x7f918415d0d0 2023-12-19T16:43:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f9189739640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91840d1380 con 0x7f918415ccf0 2023-12-19T16:43:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f91827fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f918415d0d0 0x7f91840b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f91827fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f918415d0d0 0x7f91840b8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34282/0 (socket says 172.21.15.26:34282) 2023-12-19T16:43:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f91827fc640 1 -- 172.21.15.26:0/3823100488 learned_addr learned my addr 172.21.15.26:0/3823100488 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f9182ffd640 1 --2- 172.21.15.26:0/3823100488 >> v2:172.21.15.160:3300/0 conn(0x7f918415ccf0 0x7f91840d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f91827fc640 1 -- 172.21.15.26:0/3823100488 >> v2:172.21.15.160:3300/0 conn(0x7f918415ccf0 msgr2=0x7f91840d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f91827fc640 1 --2- 172.21.15.26:0/3823100488 >> v2:172.21.15.160:3300/0 conn(0x7f918415ccf0 0x7f91840d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f91827fc640 1 -- 172.21.15.26:0/3823100488 >> v1:172.21.15.160:6789/0 conn(0x7f91840b4600 legacy=0x7f91840b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.419+0000 7f91827fc640 1 -- 172.21.15.26:0/3823100488 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918415d680 con 0x7f918415d0d0 2023-12-19T16:43:04.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.423+0000 7f91827fc640 1 --2- 172.21.15.26:0/3823100488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f918415d0d0 0x7f91840b8400 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f91840d3c20 tx=0x7f917801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d419bce67b30b09 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:04.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.423+0000 7f9181ffb640 1 -- 172.21.15.26:0/3823100488 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91780aa430 con 0x7f918415d0d0 2023-12-19T16:43:04.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.423+0000 7f9181ffb640 1 -- 172.21.15.26:0/3823100488 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91780aa590 con 0x7f918415d0d0 2023-12-19T16:43:04.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.423+0000 7f9181ffb640 1 -- 172.21.15.26:0/3823100488 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91780aa920 con 0x7f918415d0d0 2023-12-19T16:43:04.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.423+0000 7f9189739640 1 -- 172.21.15.26:0/3823100488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f918415d0d0 msgr2=0x7f91840b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.423+0000 7f9189739640 1 --2- 172.21.15.26:0/3823100488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f918415d0d0 0x7f91840b8400 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f91840d3c20 tx=0x7f917801c4d0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 -- 172.21.15.26:0/3823100488 shutdown_connections 2023-12-19T16:43:04.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 --2- 172.21.15.26:0/3823100488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f918415d0d0 0x7f91840b8400 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 --2- 172.21.15.26:0/3823100488 >> v2:172.21.15.160:3300/0 conn(0x7f918415ccf0 0x7f91840d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 -- 172.21.15.26:0/3823100488 >> 172.21.15.26:0/3823100488 conn(0x7f9184065ee0 msgr2=0x7f91840cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:04.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 -- 172.21.15.26:0/3823100488 shutdown_connections 2023-12-19T16:43:04.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 -- 172.21.15.26:0/3823100488 wait complete. 2023-12-19T16:43:04.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 Processor -- start 2023-12-19T16:43:04.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 -- start start 2023-12-19T16:43:04.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91840b4600 0x7f9184158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f918415ccf0 0x7f91840cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f918415d0d0 0x7f91841562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91841586c0 con 0x7f918415d0d0 2023-12-19T16:43:04.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918415ad40 con 0x7f91840b4600 2023-12-19T16:43:04.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9189739640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918415a130 con 0x7f918415ccf0 2023-12-19T16:43:04.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f91837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f918415d0d0 0x7f91841562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f91837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f918415d0d0 0x7f91841562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34294/0 (socket says 172.21.15.26:34294) 2023-12-19T16:43:04.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f91827fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f918415ccf0 0x7f91840cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f91837fe640 1 -- 172.21.15.26:0/1471346128 learned_addr learned my addr 172.21.15.26:0/1471346128 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:04.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f9182ffd640 1 --2- 172.21.15.26:0/1471346128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91840b4600 0x7f9184158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f91837fe640 1 -- 172.21.15.26:0/1471346128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f918415ccf0 msgr2=0x7f91840cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f91837fe640 1 --2- 172.21.15.26:0/1471346128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f918415ccf0 0x7f91840cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f91837fe640 1 -- 172.21.15.26:0/1471346128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91840b4600 msgr2=0x7f9184158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f91837fe640 1 --2- 172.21.15.26:0/1471346128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91840b4600 0x7f9184158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.427+0000 7f91837fe640 1 -- 172.21.15.26:0/1471346128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91780094d0 con 0x7f918415d0d0 2023-12-19T16:43:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.431+0000 7f91837fe640 1 --2- 172.21.15.26:0/1471346128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f918415d0d0 0x7f91841562f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f917c00c770 tx=0x7f917c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.431+0000 7f916b7fe640 1 -- 172.21.15.26:0/1471346128 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c003b80 con 0x7f918415d0d0 2023-12-19T16:43:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.431+0000 7f916b7fe640 1 -- 172.21.15.26:0/1471346128 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c003ce0 con 0x7f918415d0d0 2023-12-19T16:43:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.431+0000 7f9189739640 1 -- 172.21.15.26:0/1471346128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918415a410 con 0x7f918415d0d0 2023-12-19T16:43:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.431+0000 7f916b7fe640 1 -- 172.21.15.26:0/1471346128 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c00aa90 con 0x7f918415d0d0 2023-12-19T16:43:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.431+0000 7f9189739640 1 -- 172.21.15.26:0/1471346128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918415a9d0 con 0x7f918415d0d0 2023-12-19T16:43:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.431+0000 7f9189739640 1 -- 172.21.15.26:0/1471346128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9184074270 con 0x7f918415d0d0 2023-12-19T16:43:04.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.431+0000 7f916b7fe640 1 -- 172.21.15.26:0/1471346128 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f917c00abf0 con 0x7f918415d0d0 2023-12-19T16:43:04.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.431+0000 7f916b7fe640 1 --2- 172.21.15.26:0/1471346128 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9170065a60 0x7f9170067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.431+0000 7f916b7fe640 1 -- 172.21.15.26:0/1471346128 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f917c082f10 con 0x7f918415d0d0 2023-12-19T16:43:04.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.431+0000 7f9182ffd640 1 --2- 172.21.15.26:0/1471346128 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9170065a60 0x7f9170067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.435+0000 7f9182ffd640 1 --2- 172.21.15.26:0/1471346128 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9170065a60 0x7f9170067f20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9174007a60 tx=0x7f9174008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:04.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.435+0000 7f916b7fe640 1 -- 172.21.15.26:0/1471346128 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f917c0508e0 con 0x7f918415d0d0 2023-12-19T16:43:04.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f9189739640 1 -- 172.21.15.26:0/1471346128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f918415ae80 con 0x7f918415d0d0 2023-12-19T16:43:04.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f916b7fe640 1 -- 172.21.15.26:0/1471346128 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f917c053f00 con 0x7f918415d0d0 2023-12-19T16:43:04.600 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:43:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f9189739640 1 -- 172.21.15.26:0/1471346128 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9170065a60 msgr2=0x7f9170067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f9189739640 1 --2- 172.21.15.26:0/1471346128 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9170065a60 0x7f9170067f20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9174007a60 tx=0x7f9174008040 comp rx=0 tx=0).stop 2023-12-19T16:43:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f9189739640 1 -- 172.21.15.26:0/1471346128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f918415d0d0 msgr2=0x7f91841562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f9189739640 1 --2- 172.21.15.26:0/1471346128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f918415d0d0 0x7f91841562f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f917c00c770 tx=0x7f917c00cc30 comp rx=0 tx=0).stop 2023-12-19T16:43:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f9189739640 1 -- 172.21.15.26:0/1471346128 shutdown_connections 2023-12-19T16:43:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f9189739640 1 --2- 172.21.15.26:0/1471346128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f918415d0d0 0x7f91841562f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f9189739640 1 --2- 172.21.15.26:0/1471346128 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9170065a60 0x7f9170067f20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f9189739640 1 --2- 172.21.15.26:0/1471346128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f918415ccf0 0x7f91840cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f9189739640 1 --2- 172.21.15.26:0/1471346128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91840b4600 0x7f9184158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f9189739640 1 -- 172.21.15.26:0/1471346128 >> 172.21.15.26:0/1471346128 conn(0x7f9184065ee0 msgr2=0x7f91840cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:04.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f9189739640 1 -- 172.21.15.26:0/1471346128 shutdown_connections 2023-12-19T16:43:04.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.595+0000 7f9189739640 1 -- 172.21.15.26:0/1471346128 wait complete. 2023-12-19T16:43:04.622 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:43:04.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd258bbe640 1 Processor -- start 2023-12-19T16:43:04.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd258bbe640 1 -- start start 2023-12-19T16:43:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd258bbe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd25406f6c0 0x7fd25415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd258bbe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd254159c00 0x7fd254070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd258bbe640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25406fb60 con 0x7fd254159820 2023-12-19T16:43:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd258bbe640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25415e470 con 0x7fd25406f6c0 2023-12-19T16:43:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd258bbe640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25406fe40 con 0x7fd254159c00 2023-12-19T16:43:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd25259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd254159820 0x7fd2540c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48636/0 (socket says 172.21.15.26:48636) 2023-12-19T16:43:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd25259c640 1 -- 172.21.15.26:0/3964361628 learned_addr learned my addr 172.21.15.26:0/3964361628 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd251d9b640 1 --2- 172.21.15.26:0/3964361628 >> v2:172.21.15.26:3300/0 conn(0x7fd25406f6c0 0x7fd25415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd252d9d640 1 --2- 172.21.15.26:0/3964361628 >> v2:172.21.15.160:3300/0 conn(0x7fd254159c00 0x7fd254070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd251d9b640 1 -- 172.21.15.26:0/3964361628 >> v2:172.21.15.160:3300/0 conn(0x7fd254159c00 msgr2=0x7fd254070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:04.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd251d9b640 1 --2- 172.21.15.26:0/3964361628 >> v2:172.21.15.160:3300/0 conn(0x7fd254159c00 0x7fd254070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd251d9b640 1 -- 172.21.15.26:0/3964361628 >> v1:172.21.15.26:6789/0 conn(0x7fd254159820 legacy=0x7fd2540c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd251d9b640 1 -- 172.21.15.26:0/3964361628 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd254070120 con 0x7fd25406f6c0 2023-12-19T16:43:04.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.763+0000 7fd25159a640 1 -- 172.21.15.26:0/3964361628 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141225772 0 0) 0x7fd25406fb60 con 0x7fd254159820 2023-12-19T16:43:04.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.767+0000 7fd251d9b640 1 --2- 172.21.15.26:0/3964361628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25406f6c0 0x7fd25415df30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd25406f200 tx=0x7fd24801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d45c9530c4a154 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.767+0000 7fd25159a640 1 -- 172.21.15.26:0/3964361628 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2480ab600 con 0x7fd25406f6c0 2023-12-19T16:43:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.767+0000 7fd25159a640 1 -- 172.21.15.26:0/3964361628 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2480a7050 con 0x7fd25406f6c0 2023-12-19T16:43:04.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.767+0000 7fd25159a640 1 -- 172.21.15.26:0/3964361628 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2480ab980 con 0x7fd25406f6c0 2023-12-19T16:43:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.767+0000 7fd258bbe640 1 -- 172.21.15.26:0/3964361628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25406f6c0 msgr2=0x7fd25415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.767+0000 7fd258bbe640 1 --2- 172.21.15.26:0/3964361628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25406f6c0 0x7fd25415df30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd25406f200 tx=0x7fd24801c430 comp rx=0 tx=0).stop 2023-12-19T16:43:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.767+0000 7fd258bbe640 1 -- 172.21.15.26:0/3964361628 shutdown_connections 2023-12-19T16:43:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.767+0000 7fd258bbe640 1 --2- 172.21.15.26:0/3964361628 >> v2:172.21.15.160:3300/0 conn(0x7fd254159c00 0x7fd254070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.767+0000 7fd258bbe640 1 --2- 172.21.15.26:0/3964361628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd25406f6c0 0x7fd25415df30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.767+0000 7fd258bbe640 1 -- 172.21.15.26:0/3964361628 >> 172.21.15.26:0/3964361628 conn(0x7fd2540ce730 msgr2=0x7fd2540ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:04.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.767+0000 7fd258bbe640 1 -- 172.21.15.26:0/3964361628 shutdown_connections 2023-12-19T16:43:04.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.767+0000 7fd258bbe640 1 -- 172.21.15.26:0/3964361628 wait complete. 2023-12-19T16:43:04.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd258bbe640 1 Processor -- start 2023-12-19T16:43:04.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd258bbe640 1 -- start start 2023-12-19T16:43:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd258bbe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd25406f6c0 0x7fd25414dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd258bbe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd254159820 0x7fd254148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd258bbe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd254159c00 0x7fd2540c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd258bbe640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2540cb380 con 0x7fd254159c00 2023-12-19T16:43:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd258bbe640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd254149170 con 0x7fd25406f6c0 2023-12-19T16:43:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd258bbe640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25414cbd0 con 0x7fd254159820 2023-12-19T16:43:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd252d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd254159c00 0x7fd2540c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd252d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd254159c00 0x7fd2540c37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34322/0 (socket says 172.21.15.26:34322) 2023-12-19T16:43:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd252d9d640 1 -- 172.21.15.26:0/3848140522 learned_addr learned my addr 172.21.15.26:0/3848140522 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd251d9b640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd254159820 0x7fd254148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd25259c640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd25406f6c0 0x7fd25414dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd252d9d640 1 -- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd254159820 msgr2=0x7fd254148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd252d9d640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd254159820 0x7fd254148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd252d9d640 1 -- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd25406f6c0 msgr2=0x7fd25414dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd252d9d640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd25406f6c0 0x7fd25414dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd252d9d640 1 -- 172.21.15.26:0/3848140522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd248009420 con 0x7fd254159c00 2023-12-19T16:43:04.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd25259c640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd25406f6c0 0x7fd25414dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd252d9d640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd254159c00 0x7fd2540c37a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009fd0 tx=0x7fd24c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd242ffd640 1 -- 172.21.15.26:0/3848140522 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c014420 con 0x7fd254159c00 2023-12-19T16:43:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd242ffd640 1 -- 172.21.15.26:0/3848140522 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c014580 con 0x7fd254159c00 2023-12-19T16:43:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd242ffd640 1 -- 172.21.15.26:0/3848140522 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c014930 con 0x7fd254159c00 2023-12-19T16:43:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd258bbe640 1 -- 172.21.15.26:0/3848140522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25406b880 con 0x7fd254159c00 2023-12-19T16:43:04.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd258bbe640 1 -- 172.21.15.26:0/3848140522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25414d190 con 0x7fd254159c00 2023-12-19T16:43:04.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.771+0000 7fd258bbe640 1 -- 172.21.15.26:0/3848140522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd254074270 con 0x7fd254159c00 2023-12-19T16:43:04.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.775+0000 7fd242ffd640 1 -- 172.21.15.26:0/3848140522 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd24c014a90 con 0x7fd254159c00 2023-12-19T16:43:04.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.775+0000 7fd242ffd640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd234065a90 0x7fd234067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:04.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.775+0000 7fd242ffd640 1 -- 172.21.15.26:0/3848140522 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd24c083a10 con 0x7fd254159c00 2023-12-19T16:43:04.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.775+0000 7fd25259c640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd234065a90 0x7fd234067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:04.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.775+0000 7fd25259c640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd234065a90 0x7fd234067f50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd244000f70 tx=0x7fd244007c40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:04.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.779+0000 7fd242ffd640 1 -- 172.21.15.26:0/3848140522 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd24c0513c0 con 0x7fd254159c00 2023-12-19T16:43:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd258bbe640 1 -- 172.21.15.26:0/3848140522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2540cec90 con 0x7fd254159c00 2023-12-19T16:43:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd242ffd640 1 -- 172.21.15.26:0/3848140522 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd24c0549e0 con 0x7fd254159c00 2023-12-19T16:43:04.941 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:43:04.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd258bbe640 1 -- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd234065a90 msgr2=0x7fd234067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd258bbe640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd234065a90 0x7fd234067f50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd244000f70 tx=0x7fd244007c40 comp rx=0 tx=0).stop 2023-12-19T16:43:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd258bbe640 1 -- 172.21.15.26:0/3848140522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd254159c00 msgr2=0x7fd2540c37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd258bbe640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd254159c00 0x7fd2540c37a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd24c009fd0 tx=0x7fd24c00e4b0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd258bbe640 1 -- 172.21.15.26:0/3848140522 shutdown_connections 2023-12-19T16:43:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd258bbe640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd254159c00 0x7fd2540c37a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd258bbe640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd234065a90 0x7fd234067f50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd258bbe640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd254159820 0x7fd254148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd258bbe640 1 --2- 172.21.15.26:0/3848140522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd25406f6c0 0x7fd25414dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd258bbe640 1 -- 172.21.15.26:0/3848140522 >> 172.21.15.26:0/3848140522 conn(0x7fd2540ce730 msgr2=0x7fd25406ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd258bbe640 1 -- 172.21.15.26:0/3848140522 shutdown_connections 2023-12-19T16:43:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:04.935+0000 7fd258bbe640 1 -- 172.21.15.26:0/3848140522 wait complete. 2023-12-19T16:43:04.962 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:43:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe687b6d640 1 Processor -- start 2023-12-19T16:43:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe687b6d640 1 -- start start 2023-12-19T16:43:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe687b6d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe680159820 0x7fe68015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe687b6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68006fb60 con 0x7fe680159c00 2023-12-19T16:43:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe687b6d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68006fe40 con 0x7fe68006f6c0 2023-12-19T16:43:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe687b6d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68015e470 con 0x7fe680159820 2023-12-19T16:43:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe68610a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe680159c00 0x7fe680070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48644/0 (socket says 172.21.15.26:48644) 2023-12-19T16:43:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe68610a640 1 -- 172.21.15.26:0/1076325001 learned_addr learned my addr 172.21.15.26:0/1076325001 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe685909640 1 --2- 172.21.15.26:0/1076325001 >> v2:172.21.15.160:3300/0 conn(0x7fe680159820 0x7fe68015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe684907640 1 -- 172.21.15.26:0/1076325001 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799022684 0 0) 0x7fe68006fb60 con 0x7fe680159c00 2023-12-19T16:43:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe684907640 1 -- 172.21.15.26:0/1076325001 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe66c003530 con 0x7fe680159c00 2023-12-19T16:43:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe684907640 1 -- 172.21.15.26:0/1076325001 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529667618 0 0) 0x7fe68006fe40 con 0x7fe68006f6c0 2023-12-19T16:43:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe684907640 1 -- 172.21.15.26:0/1076325001 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe68006fb60 con 0x7fe68006f6c0 2023-12-19T16:43:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe685909640 1 -- 172.21.15.26:0/1076325001 >> v1:172.21.15.160:6789/0 conn(0x7fe68006f6c0 legacy=0x7fe6800c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe685909640 1 -- 172.21.15.26:0/1076325001 >> v1:172.21.15.26:6789/0 conn(0x7fe680159c00 legacy=0x7fe680070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.103+0000 7fe685909640 1 -- 172.21.15.26:0/1076325001 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe680070120 con 0x7fe680159820 2023-12-19T16:43:05.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.107+0000 7fe685909640 1 --2- 172.21.15.26:0/1076325001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe680159820 0x7fe68015df30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0097c0 tx=0x7fe67c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abe761b897e6665a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:05.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.107+0000 7fe684907640 1 -- 172.21.15.26:0/1076325001 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c0ab650 con 0x7fe680159820 2023-12-19T16:43:05.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.107+0000 7fe684907640 1 -- 172.21.15.26:0/1076325001 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c0ab7b0 con 0x7fe680159820 2023-12-19T16:43:05.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.107+0000 7fe684907640 1 -- 172.21.15.26:0/1076325001 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c0abb40 con 0x7fe680159820 2023-12-19T16:43:05.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.107+0000 7fe687b6d640 1 -- 172.21.15.26:0/1076325001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe680159820 msgr2=0x7fe68015df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.107+0000 7fe687b6d640 1 --2- 172.21.15.26:0/1076325001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe680159820 0x7fe68015df30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0097c0 tx=0x7fe67c01c450 comp rx=0 tx=0).stop 2023-12-19T16:43:05.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.107+0000 7fe687b6d640 1 -- 172.21.15.26:0/1076325001 shutdown_connections 2023-12-19T16:43:05.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.107+0000 7fe687b6d640 1 --2- 172.21.15.26:0/1076325001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe680159820 0x7fe68015df30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.107+0000 7fe687b6d640 1 -- 172.21.15.26:0/1076325001 >> 172.21.15.26:0/1076325001 conn(0x7fe6800ce730 msgr2=0x7fe6800ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.107+0000 7fe687b6d640 1 -- 172.21.15.26:0/1076325001 shutdown_connections 2023-12-19T16:43:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.107+0000 7fe687b6d640 1 -- 172.21.15.26:0/1076325001 wait complete. 2023-12-19T16:43:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe687b6d640 1 Processor -- start 2023-12-19T16:43:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe687b6d640 1 -- start start 2023-12-19T16:43:05.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe687b6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe68006f6c0 0x7fe680153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe687b6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe680159820 0x7fe6800b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe687b6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe680159c00 0x7fe6800c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe687b6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6800c5aa0 con 0x7fe680159820 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe687b6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68015a2b0 con 0x7fe68006f6c0 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe687b6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe680153f90 con 0x7fe680159c00 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe685108640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe680159820 0x7fe6800b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe685108640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe680159820 0x7fe6800b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34334/0 (socket says 172.21.15.26:34334) 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe685108640 1 -- 172.21.15.26:0/1264223529 learned_addr learned my addr 172.21.15.26:0/1264223529 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe685909640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe68006f6c0 0x7fe680153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe68610a640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe680159c00 0x7fe6800c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe685108640 1 -- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe680159c00 msgr2=0x7fe6800c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe685108640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe680159c00 0x7fe6800c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe685108640 1 -- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe68006f6c0 msgr2=0x7fe680153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe685108640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe68006f6c0 0x7fe680153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe685108640 1 -- 172.21.15.26:0/1264223529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c009420 con 0x7fe680159820 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe685108640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe680159820 0x7fe6800b12a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c430 tx=0x7fe67000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:05.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe6767fc640 1 -- 172.21.15.26:0/1264223529 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700052d0 con 0x7fe680159820 2023-12-19T16:43:05.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe6767fc640 1 -- 172.21.15.26:0/1264223529 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe670005450 con 0x7fe680159820 2023-12-19T16:43:05.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe6767fc640 1 -- 172.21.15.26:0/1264223529 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670003840 con 0x7fe680159820 2023-12-19T16:43:05.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe687b6d640 1 -- 172.21.15.26:0/1264223529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68006b880 con 0x7fe680159820 2023-12-19T16:43:05.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.111+0000 7fe687b6d640 1 -- 172.21.15.26:0/1264223529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68015a7e0 con 0x7fe680159820 2023-12-19T16:43:05.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.115+0000 7fe687b6d640 1 -- 172.21.15.26:0/1264223529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe680074270 con 0x7fe680159820 2023-12-19T16:43:05.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.115+0000 7fe6767fc640 1 -- 172.21.15.26:0/1264223529 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe6700039a0 con 0x7fe680159820 2023-12-19T16:43:05.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.115+0000 7fe6767fc640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe664065a90 0x7fe664067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.115+0000 7fe6767fc640 1 -- 172.21.15.26:0/1264223529 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe670082730 con 0x7fe680159820 2023-12-19T16:43:05.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.115+0000 7fe685909640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe664065a90 0x7fe664067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.115+0000 7fe685909640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe664065a90 0x7fe664067f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0af000 tx=0x7fe67c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:05.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.119+0000 7fe6767fc640 1 -- 172.21.15.26:0/1264223529 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe670050100 con 0x7fe680159820 2023-12-19T16:43:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.275+0000 7fe687b6d640 1 -- 172.21.15.26:0/1264223529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe68006c290 con 0x7fe680159820 2023-12-19T16:43:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.275+0000 7fe6767fc640 1 -- 172.21.15.26:0/1264223529 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe670053720 con 0x7fe680159820 2023-12-19T16:43:05.282 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:43:05.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.279+0000 7fe687b6d640 1 -- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe664065a90 msgr2=0x7fe664067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.279+0000 7fe687b6d640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe664065a90 0x7fe664067f50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe67c0af000 tx=0x7fe67c0a7000 comp rx=0 tx=0).stop 2023-12-19T16:43:05.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.279+0000 7fe687b6d640 1 -- 172.21.15.26:0/1264223529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe680159820 msgr2=0x7fe6800b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.279+0000 7fe687b6d640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe680159820 0x7fe6800b12a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c430 tx=0x7fe67000c8f0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.279+0000 7fe687b6d640 1 -- 172.21.15.26:0/1264223529 shutdown_connections 2023-12-19T16:43:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.279+0000 7fe687b6d640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe664065a90 0x7fe664067f50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.279+0000 7fe687b6d640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe680159c00 0x7fe6800c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.279+0000 7fe687b6d640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe680159820 0x7fe6800b12a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.279+0000 7fe687b6d640 1 --2- 172.21.15.26:0/1264223529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe68006f6c0 0x7fe680153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.279+0000 7fe687b6d640 1 -- 172.21.15.26:0/1264223529 >> 172.21.15.26:0/1264223529 conn(0x7fe6800ce730 msgr2=0x7fe680158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.279+0000 7fe687b6d640 1 -- 172.21.15.26:0/1264223529 shutdown_connections 2023-12-19T16:43:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.279+0000 7fe687b6d640 1 -- 172.21.15.26:0/1264223529 wait complete. 2023-12-19T16:43:05.303 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T16:43:05.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5ef96640 1 Processor -- start 2023-12-19T16:43:05.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5ef96640 1 -- start start 2023-12-19T16:43:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5ef96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5815d260 0x7f2b5806e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5ef96640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b580b34f0 0x7f2b580b3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5ef96640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b580b3060 con 0x7f2b580658e0 2023-12-19T16:43:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5ef96640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5815d820 con 0x7f2b580b34f0 2023-12-19T16:43:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5ef96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5815db00 con 0x7f2b5815d260 2023-12-19T16:43:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5cd32640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2b580658e0 0x7f2b58065cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48660/0 (socket says 172.21.15.26:48660) 2023-12-19T16:43:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5cd32640 1 -- 172.21.15.26:0/290944026 learned_addr learned my addr 172.21.15.26:0/290944026 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5d533640 1 --2- 172.21.15.26:0/290944026 >> v2:172.21.15.26:3300/0 conn(0x7f2b580b34f0 0x7f2b580b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b57fff640 1 --2- 172.21.15.26:0/290944026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5815d260 0x7f2b5806e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b577fe640 1 -- 172.21.15.26:0/290944026 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959959111 0 0) 0x7f2b580b3060 con 0x7f2b580658e0 2023-12-19T16:43:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b577fe640 1 -- 172.21.15.26:0/290944026 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b48003530 con 0x7f2b580658e0 2023-12-19T16:43:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5d533640 1 -- 172.21.15.26:0/290944026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5815d260 msgr2=0x7f2b5806e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5d533640 1 --2- 172.21.15.26:0/290944026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5815d260 0x7f2b5806e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5d533640 1 -- 172.21.15.26:0/290944026 >> v1:172.21.15.26:6789/0 conn(0x7f2b580658e0 legacy=0x7f2b58065cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.443+0000 7f2b5d533640 1 -- 172.21.15.26:0/290944026 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5815dde0 con 0x7f2b580b34f0 2023-12-19T16:43:05.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.447+0000 7f2b5d533640 1 --2- 172.21.15.26:0/290944026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b580b34f0 0x7f2b580b3980 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009d90 tx=0x7f2b5001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2104d6650aedc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:05.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.447+0000 7f2b577fe640 1 -- 172.21.15.26:0/290944026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b500a58f0 con 0x7f2b580b34f0 2023-12-19T16:43:05.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.447+0000 7f2b577fe640 1 -- 172.21.15.26:0/290944026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b500a5a50 con 0x7f2b580b34f0 2023-12-19T16:43:05.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.447+0000 7f2b577fe640 1 -- 172.21.15.26:0/290944026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b500a5dc0 con 0x7f2b580b34f0 2023-12-19T16:43:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.447+0000 7f2b5ef96640 1 -- 172.21.15.26:0/290944026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b580b34f0 msgr2=0x7f2b580b3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.447+0000 7f2b5ef96640 1 --2- 172.21.15.26:0/290944026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b580b34f0 0x7f2b580b3980 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009d90 tx=0x7f2b5001c920 comp rx=0 tx=0).stop 2023-12-19T16:43:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.447+0000 7f2b5ef96640 1 -- 172.21.15.26:0/290944026 shutdown_connections 2023-12-19T16:43:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.447+0000 7f2b5ef96640 1 --2- 172.21.15.26:0/290944026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b580b34f0 0x7f2b580b3980 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.447+0000 7f2b5ef96640 1 --2- 172.21.15.26:0/290944026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5815d260 0x7f2b5806e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.447+0000 7f2b5ef96640 1 -- 172.21.15.26:0/290944026 >> 172.21.15.26:0/290944026 conn(0x7f2b5806a320 msgr2=0x7f2b580b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.447+0000 7f2b5ef96640 1 -- 172.21.15.26:0/290944026 shutdown_connections 2023-12-19T16:43:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.447+0000 7f2b5ef96640 1 -- 172.21.15.26:0/290944026 wait complete. 2023-12-19T16:43:05.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5ef96640 1 Processor -- start 2023-12-19T16:43:05.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5ef96640 1 -- start start 2023-12-19T16:43:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5ef96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b580658e0 0x7f2b58070850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5ef96640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b580b34f0 0x7f2b5815f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5ef96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5815d260 0x7f2b5815fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5ef96640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b581447a0 con 0x7f2b5815d260 2023-12-19T16:43:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5ef96640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58144a80 con 0x7f2b580b34f0 2023-12-19T16:43:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5ef96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58144d60 con 0x7f2b580658e0 2023-12-19T16:43:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5d533640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5815d260 0x7f2b5815fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5d533640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5815d260 0x7f2b5815fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34354/0 (socket says 172.21.15.26:34354) 2023-12-19T16:43:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5d533640 1 -- 172.21.15.26:0/3816058923 learned_addr learned my addr 172.21.15.26:0/3816058923 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5cd32640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b580658e0 0x7f2b58070850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b57fff640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b580b34f0 0x7f2b5815f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5d533640 1 -- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b580658e0 msgr2=0x7f2b58070850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5d533640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b580658e0 0x7f2b58070850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5d533640 1 -- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b580b34f0 msgr2=0x7f2b5815f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5d533640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b580b34f0 0x7f2b5815f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5d533640 1 -- 172.21.15.26:0/3816058923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b50009a70 con 0x7f2b5815d260 2023-12-19T16:43:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b57fff640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b580b34f0 0x7f2b5815f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5d533640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5815d260 0x7f2b5815fcd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2b5001a550 tx=0x7f2b500196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b557fa640 1 -- 172.21.15.26:0/3816058923 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b500ad440 con 0x7f2b5815d260 2023-12-19T16:43:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b557fa640 1 -- 172.21.15.26:0/3816058923 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b500ad5a0 con 0x7f2b5815d260 2023-12-19T16:43:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b557fa640 1 -- 172.21.15.26:0/3816058923 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b500ad910 con 0x7f2b5815d260 2023-12-19T16:43:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5ef96640 1 -- 172.21.15.26:0/3816058923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b58145040 con 0x7f2b5815d260 2023-12-19T16:43:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5ef96640 1 -- 172.21.15.26:0/3816058923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b58145840 con 0x7f2b5815d260 2023-12-19T16:43:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.451+0000 7f2b5ef96640 1 -- 172.21.15.26:0/3816058923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b58074270 con 0x7f2b5815d260 2023-12-19T16:43:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.455+0000 7f2b557fa640 1 -- 172.21.15.26:0/3816058923 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2b500ada70 con 0x7f2b5815d260 2023-12-19T16:43:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.455+0000 7f2b557fa640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b3c065a60 0x7f2b3c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.455+0000 7f2b557fa640 1 -- 172.21.15.26:0/3816058923 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b5011e680 con 0x7f2b5815d260 2023-12-19T16:43:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.455+0000 7f2b5cd32640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b3c065a60 0x7f2b3c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.455+0000 7f2b5cd32640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b3c065a60 0x7f2b3c067f20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c005890 tx=0x7f2b4c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:05.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.459+0000 7f2b557fa640 1 -- 172.21.15.26:0/3816058923 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2b500ebfa0 con 0x7f2b5815d260 2023-12-19T16:43:05.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b5ef96640 1 -- 172.21.15.26:0/3816058923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b581454f0 con 0x7f2b5815d260 2023-12-19T16:43:05.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b557fa640 1 -- 172.21.15.26:0/3816058923 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2b500bc080 con 0x7f2b5815d260 2023-12-19T16:43:05.635 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2023-12-19T16:43:05.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b5ef96640 1 -- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b3c065a60 msgr2=0x7f2b3c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b5ef96640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b3c065a60 0x7f2b3c067f20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c005890 tx=0x7f2b4c009040 comp rx=0 tx=0).stop 2023-12-19T16:43:05.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b5ef96640 1 -- 172.21.15.26:0/3816058923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5815d260 msgr2=0x7f2b5815fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b5ef96640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5815d260 0x7f2b5815fcd0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2b5001a550 tx=0x7f2b500196c0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b5ef96640 1 -- 172.21.15.26:0/3816058923 shutdown_connections 2023-12-19T16:43:05.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b5ef96640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5815d260 0x7f2b5815fcd0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b5ef96640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b580b34f0 0x7f2b5815f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b5ef96640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b3c065a60 0x7f2b3c067f20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b5ef96640 1 --2- 172.21.15.26:0/3816058923 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b580658e0 0x7f2b58070850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b5ef96640 1 -- 172.21.15.26:0/3816058923 >> 172.21.15.26:0/3816058923 conn(0x7f2b5806a320 msgr2=0x7f2b580ac850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:05.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b5ef96640 1 -- 172.21.15.26:0/3816058923 shutdown_connections 2023-12-19T16:43:05.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.631+0000 7f2b5ef96640 1 -- 172.21.15.26:0/3816058923 wait complete. 2023-12-19T16:43:05.658 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-12-19T16:43:05.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:05.659 DEBUG:teuthology.orchestra.run.smithi160:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-12-19T16:43:05.686 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi160.front.sepia.ceph.com 2023-12-19T16:43:05.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:05.686 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-19T16:43:05.737 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T16:43:05.737 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:43:05.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.875+0000 7f52633f3640 1 Processor -- start 2023-12-19T16:43:05.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.875+0000 7f52633f3640 1 -- start start 2023-12-19T16:43:05.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f52633f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f525c0b2d50 0x7f525c16c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f52633f3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525c06d4b0 con 0x7f525c0cb1d0 2023-12-19T16:43:05.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f52633f3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525c06cc10 con 0x7f525c06bee0 2023-12-19T16:43:05.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f52633f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0cc030 con 0x7f525c0b2d50 2023-12-19T16:43:05.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f526118f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f525c0cb1d0 0x7f525c0cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48662/0 (socket says 172.21.15.26:48662) 2023-12-19T16:43:05.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f526118f640 1 -- 172.21.15.26:0/2282787310 learned_addr learned my addr 172.21.15.26:0/2282787310 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:05.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5261990640 1 --2- 172.21.15.26:0/2282787310 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f525c0b2d50 0x7f525c16c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725172261 0 0) 0x7f525c06d4b0 con 0x7f525c0cb1d0 2023-12-19T16:43:05.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5248003530 con 0x7f525c0cb1d0 2023-12-19T16:43:05.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544289529 0 0) 0x7f525c06cc10 con 0x7f525c06bee0 2023-12-19T16:43:05.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f525c06d4b0 con 0x7f525c06bee0 2023-12-19T16:43:05.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2515892548 0 0) 0x7f5248003530 con 0x7f525c0cb1d0 2023-12-19T16:43:05.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f525c0b2d50 msgr2=0x7f525c16c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:05.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 --2- 172.21.15.26:0/2282787310 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f525c0b2d50 0x7f525c16c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 >> v1:172.21.15.160:6789/0 conn(0x7f525c06bee0 legacy=0x7f525c06c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c06cef0 con 0x7f525c0cb1d0 2023-12-19T16:43:05.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f5258001680 con 0x7f525c0cb1d0 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 >> v1:172.21.15.26:6789/0 conn(0x7f525c0cb1d0 legacy=0x7f525c0cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 --2- 172.21.15.26:0/2282787310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5248003b40 0x7f5248003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 --2- 172.21.15.26:0/2282787310 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f525c0b2d50 0x7f52480047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 --2- 172.21.15.26:0/2282787310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5248004d10 0x7f5248005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c0cc030 con 0x7f5248004d10 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524800bac0 con 0x7f5248003b40 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524800bc80 con 0x7f525c0b2d50 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5261990640 1 --2- 172.21.15.26:0/2282787310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5248004d10 0x7f5248005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f526118f640 1 --2- 172.21.15.26:0/2282787310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5248003b40 0x7f5248003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f526098e640 1 --2- 172.21.15.26:0/2282787310 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f525c0b2d50 0x7f52480047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5261990640 1 -- 172.21.15.26:0/2282787310 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f525c0b2d50 msgr2=0x7f52480047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5261990640 1 --2- 172.21.15.26:0/2282787310 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f525c0b2d50 0x7f52480047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5261990640 1 -- 172.21.15.26:0/2282787310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5248003b40 msgr2=0x7f5248003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5261990640 1 --2- 172.21.15.26:0/2282787310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5248003b40 0x7f5248003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.879+0000 7f5261990640 1 -- 172.21.15.26:0/2282787310 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5248003530 con 0x7f5248004d10 2023-12-19T16:43:05.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.883+0000 7f5261990640 1 --2- 172.21.15.26:0/2282787310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5248004d10 0x7f5248005190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5254009f90 tx=0x7f525401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fad25dfc2710215 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:05.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.883+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52540a5c30 con 0x7f5248004d10 2023-12-19T16:43:05.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.883+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525400f040 con 0x7f5248004d10 2023-12-19T16:43:05.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.883+0000 7f5253fff640 1 -- 172.21.15.26:0/2282787310 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52540ad440 con 0x7f5248004d10 2023-12-19T16:43:05.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.883+0000 7f52633f3640 1 -- 172.21.15.26:0/2282787310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5248004d10 msgr2=0x7f5248005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.883+0000 7f52633f3640 1 --2- 172.21.15.26:0/2282787310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5248004d10 0x7f5248005190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5254009f90 tx=0x7f525401cc20 comp rx=0 tx=0).stop 2023-12-19T16:43:05.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.883+0000 7f5261990640 1 -- 172.21.15.26:0/2282787310 reap_dead start 2023-12-19T16:43:05.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.883+0000 7f52633f3640 1 -- 172.21.15.26:0/2282787310 shutdown_connections 2023-12-19T16:43:05.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.883+0000 7f52633f3640 1 -- 172.21.15.26:0/2282787310 >> 172.21.15.26:0/2282787310 conn(0x7f525c0658e0 msgr2=0x7f525c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:05.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.883+0000 7f52633f3640 1 -- 172.21.15.26:0/2282787310 shutdown_connections 2023-12-19T16:43:05.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.883+0000 7f52633f3640 1 -- 172.21.15.26:0/2282787310 wait complete. 2023-12-19T16:43:05.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.883+0000 7f52633f3640 1 Processor -- start 2023-12-19T16:43:05.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f52633f3640 1 -- start start 2023-12-19T16:43:05.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f52633f3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5248004d10 0x7f525c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f52633f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f525c164570 0x7f525c151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f52633f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f525c1619f0 0x7f525c161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f52633f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c151970 con 0x7f525c1619f0 2023-12-19T16:43:05.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f52633f3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c15bcc0 con 0x7f5248004d10 2023-12-19T16:43:05.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f52633f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525c15bfa0 con 0x7f525c164570 2023-12-19T16:43:05.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f5261990640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f525c1619f0 0x7f525c161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f5261990640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f525c1619f0 0x7f525c161e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34376/0 (socket says 172.21.15.26:34376) 2023-12-19T16:43:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f5261990640 1 -- 172.21.15.26:0/2833024798 learned_addr learned my addr 172.21.15.26:0/2833024798 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f526118f640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5248004d10 0x7f525c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f526098e640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f525c164570 0x7f525c151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f5261990640 1 -- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f525c164570 msgr2=0x7f525c151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f5261990640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f525c164570 0x7f525c151430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f5261990640 1 -- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5248004d10 msgr2=0x7f525c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f5261990640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5248004d10 0x7f525c0cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f5261990640 1 -- 172.21.15.26:0/2833024798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c1649d0 con 0x7f525c1619f0 2023-12-19T16:43:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f526118f640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5248004d10 0x7f525c0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f5261990640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f525c1619f0 0x7f525c161e70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5254002240 tx=0x7f5254009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:05.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f5251ffb640 1 -- 172.21.15.26:0/2833024798 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52540ae950 con 0x7f525c1619f0 2023-12-19T16:43:05.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f5251ffb640 1 -- 172.21.15.26:0/2833024798 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52540aeab0 con 0x7f525c1619f0 2023-12-19T16:43:05.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f5251ffb640 1 -- 172.21.15.26:0/2833024798 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52540aee40 con 0x7f525c1619f0 2023-12-19T16:43:05.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f52633f3640 1 -- 172.21.15.26:0/2833024798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525c0cd710 con 0x7f525c1619f0 2023-12-19T16:43:05.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f52633f3640 1 -- 172.21.15.26:0/2833024798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f525c0cdcd0 con 0x7f525c1619f0 2023-12-19T16:43:05.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.887+0000 7f52633f3640 1 -- 172.21.15.26:0/2833024798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525c074270 con 0x7f525c1619f0 2023-12-19T16:43:05.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.891+0000 7f5251ffb640 1 -- 172.21.15.26:0/2833024798 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f52540b4040 con 0x7f525c1619f0 2023-12-19T16:43:05.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.891+0000 7f5251ffb640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5240065a60 0x7f5240067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:05.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.891+0000 7f5251ffb640 1 -- 172.21.15.26:0/2833024798 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f524c001d30 con 0x7f525c1619f0 2023-12-19T16:43:05.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.891+0000 7f526118f640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5240065a60 0x7f5240067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:05.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.891+0000 7f526118f640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5240065a60 0x7f5240067f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5258000c10 tx=0x7f5258009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:05.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:05.895+0000 7f5251ffb640 1 -- 172.21.15.26:0/2833024798 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f52540eb530 con 0x7f525c1619f0 2023-12-19T16:43:06.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.083+0000 7f52633f3640 1 -- 172.21.15.26:0/2833024798 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f525c15c0e0 con 0x7f525c1619f0 2023-12-19T16:43:06.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.087+0000 7f5251ffb640 1 -- 172.21.15.26:0/2833024798 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f52540eeb50 con 0x7f525c1619f0 2023-12-19T16:43:06.090 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:06.091 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2850479611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2850479611},{"type":"v1","addr":"172.21.15.160:6836","nonce":2850479611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/837780144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":837780144},{"type":"v1","addr":"172.21.15.26:6834","nonce":837780144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2784408519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2784408519},{"type":"v1","addr":"172.21.15.26:6835","nonce":2784408519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:42:27.777197+0000","modified":"2023-12-19T16:42:30.798296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6837/3427345273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3427345273},{"type":"v1","addr":"172.21.15.160:6837","nonce":3427345273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T16:43:06.091 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2023-12-19T16:43:06.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.087+0000 7f52633f3640 1 -- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5240065a60 msgr2=0x7f5240067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.087+0000 7f52633f3640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5240065a60 0x7f5240067f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5258000c10 tx=0x7f5258009040 comp rx=0 tx=0).stop 2023-12-19T16:43:06.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.087+0000 7f52633f3640 1 -- 172.21.15.26:0/2833024798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f525c1619f0 msgr2=0x7f525c161e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.087+0000 7f52633f3640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f525c1619f0 0x7f525c161e70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5254002240 tx=0x7f5254009750 comp rx=0 tx=0).stop 2023-12-19T16:43:06.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.087+0000 7f52633f3640 1 -- 172.21.15.26:0/2833024798 shutdown_connections 2023-12-19T16:43:06.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.087+0000 7f52633f3640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f525c1619f0 0x7f525c161e70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.087+0000 7f52633f3640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5240065a60 0x7f5240067f20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.087+0000 7f52633f3640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f525c164570 0x7f525c151430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.087+0000 7f52633f3640 1 --2- 172.21.15.26:0/2833024798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5248004d10 0x7f525c0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.087+0000 7f52633f3640 1 -- 172.21.15.26:0/2833024798 >> 172.21.15.26:0/2833024798 conn(0x7f525c0658e0 msgr2=0x7f525c0b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:06.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.087+0000 7f52633f3640 1 -- 172.21.15.26:0/2833024798 shutdown_connections 2023-12-19T16:43:06.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.087+0000 7f52633f3640 1 -- 172.21.15.26:0/2833024798 wait complete. 2023-12-19T16:43:06.112 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:43:06.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f900c4c4640 1 Processor -- start 2023-12-19T16:43:06.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f900c4c4640 1 -- start start 2023-12-19T16:43:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f900c4c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f900406b3d0 0x7f90040ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f900c4c4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900406bc30 con 0x7f90040cec40 2023-12-19T16:43:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f900c4c4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90040cc6e0 con 0x7f9004154220 2023-12-19T16:43:06.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f900c4c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90040cf0d0 con 0x7f900406b3d0 2023-12-19T16:43:06.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f900aa61640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f90040cec40 0x7f900415b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45614/0 (socket says 172.21.15.26:45614) 2023-12-19T16:43:06.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f900aa61640 1 -- 172.21.15.26:0/4181548086 learned_addr learned my addr 172.21.15.26:0/4181548086 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:06.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f9009a5f640 1 --2- 172.21.15.26:0/4181548086 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f900406b3d0 0x7f90040ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f900925e640 1 -- 172.21.15.26:0/4181548086 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2568498481 0 0) 0x7f900406bc30 con 0x7f90040cec40 2023-12-19T16:43:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f900925e640 1 -- 172.21.15.26:0/4181548086 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ff0005740 con 0x7f90040cec40 2023-12-19T16:43:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f900925e640 1 -- 172.21.15.26:0/4181548086 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053147620 0 0) 0x7f90040cc6e0 con 0x7f9004154220 2023-12-19T16:43:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f900925e640 1 -- 172.21.15.26:0/4181548086 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f900406bc30 con 0x7f9004154220 2023-12-19T16:43:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f9009a5f640 1 -- 172.21.15.26:0/4181548086 >> v1:172.21.15.160:6789/0 conn(0x7f9004154220 legacy=0x7f900415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f9009a5f640 1 -- 172.21.15.26:0/4181548086 >> v1:172.21.15.26:6789/0 conn(0x7f90040cec40 legacy=0x7f900415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.251+0000 7f9009a5f640 1 -- 172.21.15.26:0/4181548086 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900406b950 con 0x7f900406b3d0 2023-12-19T16:43:06.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.255+0000 7f9009a5f640 1 --2- 172.21.15.26:0/4181548086 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f900406b3d0 0x7f90040ce700 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40097f0 tx=0x7f8ff401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cb9ba3bd7fce876 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:06.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.255+0000 7f900925e640 1 -- 172.21.15.26:0/4181548086 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40aa5f0 con 0x7f900406b3d0 2023-12-19T16:43:06.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.255+0000 7f900925e640 1 -- 172.21.15.26:0/4181548086 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff40aa750 con 0x7f900406b3d0 2023-12-19T16:43:06.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.255+0000 7f900925e640 1 -- 172.21.15.26:0/4181548086 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40aaae0 con 0x7f900406b3d0 2023-12-19T16:43:06.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.255+0000 7f900c4c4640 1 -- 172.21.15.26:0/4181548086 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f900406b3d0 msgr2=0x7f90040ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.255+0000 7f900c4c4640 1 --2- 172.21.15.26:0/4181548086 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f900406b3d0 0x7f90040ce700 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8ff40097f0 tx=0x7f8ff401c490 comp rx=0 tx=0).stop 2023-12-19T16:43:06.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.255+0000 7f900c4c4640 1 -- 172.21.15.26:0/4181548086 shutdown_connections 2023-12-19T16:43:06.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.255+0000 7f900c4c4640 1 --2- 172.21.15.26:0/4181548086 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f900406b3d0 0x7f90040ce700 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.255+0000 7f900c4c4640 1 -- 172.21.15.26:0/4181548086 >> 172.21.15.26:0/4181548086 conn(0x7f90040658e0 msgr2=0x7f9004065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:06.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.255+0000 7f900c4c4640 1 -- 172.21.15.26:0/4181548086 shutdown_connections 2023-12-19T16:43:06.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.255+0000 7f900c4c4640 1 -- 172.21.15.26:0/4181548086 wait complete. 2023-12-19T16:43:06.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900c4c4640 1 Processor -- start 2023-12-19T16:43:06.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900c4c4640 1 -- start start 2023-12-19T16:43:06.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900c4c4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f900406b3d0 0x7f90040710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900c4c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90040cec40 0x7f90040b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900c4c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9004154220 0x7f9004158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900c4c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900414d9d0 con 0x7f9004154220 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900c4c4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9004071630 con 0x7f900406b3d0 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900c4c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900415d920 con 0x7f90040cec40 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900aa61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9004154220 0x7f9004158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900aa61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9004154220 0x7f9004158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53902/0 (socket says 172.21.15.26:53902) 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900aa61640 1 -- 172.21.15.26:0/1040232428 learned_addr learned my addr 172.21.15.26:0/1040232428 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900a260640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f900406b3d0 0x7f90040710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f9009a5f640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90040cec40 0x7f90040b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900aa61640 1 -- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90040cec40 msgr2=0x7f90040b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900aa61640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90040cec40 0x7f90040b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900aa61640 1 -- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f900406b3d0 msgr2=0x7f90040710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900aa61640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f900406b3d0 0x7f90040710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900aa61640 1 -- 172.21.15.26:0/1040232428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff4009420 con 0x7f9004154220 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f9009a5f640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90040cec40 0x7f90040b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900aa61640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9004154220 0x7f9004158b60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00cbd0 tx=0x7f8ffc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:06.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f8ffaffd640 1 -- 172.21.15.26:0/1040232428 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00ace0 con 0x7f9004154220 2023-12-19T16:43:06.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f8ffaffd640 1 -- 172.21.15.26:0/1040232428 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc00ae40 con 0x7f9004154220 2023-12-19T16:43:06.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f8ffaffd640 1 -- 172.21.15.26:0/1040232428 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc012620 con 0x7f9004154220 2023-12-19T16:43:06.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900c4c4640 1 -- 172.21.15.26:0/1040232428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900415dc00 con 0x7f9004154220 2023-12-19T16:43:06.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.259+0000 7f900c4c4640 1 -- 172.21.15.26:0/1040232428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900415e1c0 con 0x7f9004154220 2023-12-19T16:43:06.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.263+0000 7f900c4c4640 1 -- 172.21.15.26:0/1040232428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900414db80 con 0x7f9004154220 2023-12-19T16:43:06.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.263+0000 7f8ffaffd640 1 -- 172.21.15.26:0/1040232428 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8ffc007070 con 0x7f9004154220 2023-12-19T16:43:06.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.263+0000 7f8ffaffd640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8fe8065a60 0x7f8fe8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.263+0000 7f8ffaffd640 1 -- 172.21.15.26:0/1040232428 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ffc083460 con 0x7f9004154220 2023-12-19T16:43:06.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.263+0000 7f900a260640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8fe8065a60 0x7f8fe8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.263+0000 7f900a260640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8fe8065a60 0x7f8fe8067f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f90000011f0 tx=0x7f9000009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:06.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.267+0000 7f8ffaffd640 1 -- 172.21.15.26:0/1040232428 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f900414db80 con 0x7f9004154220 2023-12-19T16:43:06.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.427+0000 7f900c4c4640 1 -- 172.21.15.26:0/1040232428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f900414db80 con 0x7f9004154220 2023-12-19T16:43:06.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.427+0000 7f8ffaffd640 1 -- 172.21.15.26:0/1040232428 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f900414db80 con 0x7f9004154220 2023-12-19T16:43:06.433 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:06.433 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:43:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.431+0000 7f900c4c4640 1 -- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8fe8065a60 msgr2=0x7f8fe8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.431+0000 7f900c4c4640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8fe8065a60 0x7f8fe8067f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f90000011f0 tx=0x7f9000009040 comp rx=0 tx=0).stop 2023-12-19T16:43:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.431+0000 7f900c4c4640 1 -- 172.21.15.26:0/1040232428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9004154220 msgr2=0x7f9004158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.431+0000 7f900c4c4640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9004154220 0x7f9004158b60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00cbd0 tx=0x7f8ffc00a430 comp rx=0 tx=0).stop 2023-12-19T16:43:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.431+0000 7f900c4c4640 1 -- 172.21.15.26:0/1040232428 shutdown_connections 2023-12-19T16:43:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.431+0000 7f900c4c4640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9004154220 0x7f9004158b60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.431+0000 7f900c4c4640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8fe8065a60 0x7f8fe8067f20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.431+0000 7f900c4c4640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90040cec40 0x7f90040b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.431+0000 7f900c4c4640 1 --2- 172.21.15.26:0/1040232428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f900406b3d0 0x7f90040710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.431+0000 7f900c4c4640 1 -- 172.21.15.26:0/1040232428 >> 172.21.15.26:0/1040232428 conn(0x7f90040658e0 msgr2=0x7f9004065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:06.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.431+0000 7f900c4c4640 1 -- 172.21.15.26:0/1040232428 shutdown_connections 2023-12-19T16:43:06.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.431+0000 7f900c4c4640 1 -- 172.21.15.26:0/1040232428 wait complete. 2023-12-19T16:43:06.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:42:25.598550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:42:26.697362+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:43:06.456 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:43:06.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.595+0000 7f6e11466640 1 Processor -- start 2023-12-19T16:43:06.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.595+0000 7f6e11466640 1 -- start start 2023-12-19T16:43:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.599+0000 7f6e11466640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e0c15cdf0 0x7f6e0c15a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.599+0000 7f6e11466640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0c15ad00 con 0x7f6e0c0b09d0 2023-12-19T16:43:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.599+0000 7f6e11466640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0c15d440 con 0x7f6e0c0b3760 2023-12-19T16:43:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.599+0000 7f6e11466640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e0c15d720 con 0x7f6e0c15cdf0 2023-12-19T16:43:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.599+0000 7f6e0b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6e0c0b09d0 0x7f6e0c15afe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45618/0 (socket says 172.21.15.26:45618) 2023-12-19T16:43:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.599+0000 7f6e0b7fe640 1 -- 172.21.15.26:0/2701107499 learned_addr learned my addr 172.21.15.26:0/2701107499 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.599+0000 7f6e0affd640 1 --2- 172.21.15.26:0/2701107499 >> v2:172.21.15.26:3300/0 conn(0x7f6e0c15cdf0 0x7f6e0c15a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.599+0000 7f6e0affd640 1 -- 172.21.15.26:0/2701107499 >> v1:172.21.15.160:6789/0 conn(0x7f6e0c0b3760 legacy=0x7f6e0c0b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.599+0000 7f6e0affd640 1 -- 172.21.15.26:0/2701107499 >> v1:172.21.15.26:6789/0 conn(0x7f6e0c0b09d0 legacy=0x7f6e0c15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.599+0000 7f6e0affd640 1 -- 172.21.15.26:0/2701107499 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c15da00 con 0x7f6e0c15cdf0 2023-12-19T16:43:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e0affd640 1 --2- 172.21.15.26:0/2701107499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c15cdf0 0x7f6e0c15a7c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0097c0 tx=0x7f6dfc01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45503ace178014c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e0a7fc640 1 -- 172.21.15.26:0/2701107499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0ad680 con 0x7f6e0c15cdf0 2023-12-19T16:43:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e0a7fc640 1 -- 172.21.15.26:0/2701107499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc0ad7e0 con 0x7f6e0c15cdf0 2023-12-19T16:43:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e0a7fc640 1 -- 172.21.15.26:0/2701107499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0adb70 con 0x7f6e0c15cdf0 2023-12-19T16:43:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 -- 172.21.15.26:0/2701107499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c15cdf0 msgr2=0x7f6e0c15a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 --2- 172.21.15.26:0/2701107499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c15cdf0 0x7f6e0c15a7c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0097c0 tx=0x7f6dfc01c380 comp rx=0 tx=0).stop 2023-12-19T16:43:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 -- 172.21.15.26:0/2701107499 shutdown_connections 2023-12-19T16:43:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 --2- 172.21.15.26:0/2701107499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c15cdf0 0x7f6e0c15a7c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 -- 172.21.15.26:0/2701107499 >> 172.21.15.26:0/2701107499 conn(0x7f6e0c0733c0 msgr2=0x7f6e0c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 -- 172.21.15.26:0/2701107499 shutdown_connections 2023-12-19T16:43:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 -- 172.21.15.26:0/2701107499 wait complete. 2023-12-19T16:43:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 Processor -- start 2023-12-19T16:43:06.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 -- start start 2023-12-19T16:43:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0c0b09d0 0x7f6e0c151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b3760 0x7f6e0c1536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e0c15cdf0 0x7f6e0c157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c1532c0 con 0x7f6e0c0b3760 2023-12-19T16:43:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c152640 con 0x7f6e0c15cdf0 2023-12-19T16:43:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.603+0000 7f6e11466640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c152920 con 0x7f6e0c0b09d0 2023-12-19T16:43:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e027fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b3760 0x7f6e0c1536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e027fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b3760 0x7f6e0c1536e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53930/0 (socket says 172.21.15.26:53930) 2023-12-19T16:43:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e027fc640 1 -- 172.21.15.26:0/1045482835 learned_addr learned my addr 172.21.15.26:0/1045482835 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e0b7fe640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e0c15cdf0 0x7f6e0c157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e027fc640 1 -- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0c0b09d0 msgr2=0x7f6e0c151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e027fc640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0c0b09d0 0x7f6e0c151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e027fc640 1 -- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e0c15cdf0 msgr2=0x7f6e0c157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e027fc640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e0c15cdf0 0x7f6e0c157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e027fc640 1 -- 172.21.15.26:0/1045482835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc009420 con 0x7f6e0c0b3760 2023-12-19T16:43:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e0b7fe640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e0c15cdf0 0x7f6e0c157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e027fc640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b3760 0x7f6e0c1536e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6df400c7c0 tx=0x7f6df400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e03fff640 1 -- 172.21.15.26:0/1045482835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df400aa50 con 0x7f6e0c0b3760 2023-12-19T16:43:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e03fff640 1 -- 172.21.15.26:0/1045482835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df400abb0 con 0x7f6e0c0b3760 2023-12-19T16:43:06.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e03fff640 1 -- 172.21.15.26:0/1045482835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df4012420 con 0x7f6e0c0b3760 2023-12-19T16:43:06.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e11466640 1 -- 172.21.15.26:0/1045482835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0c152c00 con 0x7f6e0c0b3760 2023-12-19T16:43:06.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e11466640 1 -- 172.21.15.26:0/1045482835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0c151b60 con 0x7f6e0c0b3760 2023-12-19T16:43:06.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e11466640 1 -- 172.21.15.26:0/1045482835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0c04e850 con 0x7f6e0c0b3760 2023-12-19T16:43:06.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.607+0000 7f6e03fff640 1 -- 172.21.15.26:0/1045482835 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6df4012580 con 0x7f6e0c0b3760 2023-12-19T16:43:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.611+0000 7f6e03fff640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6dec065a60 0x7f6dec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.611+0000 7f6e03fff640 1 -- 172.21.15.26:0/1045482835 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6df40833e0 con 0x7f6e0c0b3760 2023-12-19T16:43:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.611+0000 7f6e0affd640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6dec065a60 0x7f6dec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.611+0000 7f6e0affd640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6dec065a60 0x7f6dec067f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0af000 tx=0x7f6dfc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:06.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.615+0000 7f6e03fff640 1 -- 172.21.15.26:0/1045482835 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6df4050d90 con 0x7f6e0c0b3760 2023-12-19T16:43:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.771+0000 7f6e11466640 1 -- 172.21.15.26:0/1045482835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e0c0cd620 con 0x7f6e0c0b3760 2023-12-19T16:43:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.771+0000 7f6e03fff640 1 -- 172.21.15.26:0/1045482835 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6df40543b0 con 0x7f6e0c0b3760 2023-12-19T16:43:06.778 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:43:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.775+0000 7f6e11466640 1 -- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6dec065a60 msgr2=0x7f6dec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.775+0000 7f6e11466640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6dec065a60 0x7f6dec067f20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0af000 tx=0x7f6dfc008040 comp rx=0 tx=0).stop 2023-12-19T16:43:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.775+0000 7f6e11466640 1 -- 172.21.15.26:0/1045482835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b3760 msgr2=0x7f6e0c1536e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.775+0000 7f6e11466640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b3760 0x7f6e0c1536e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6df400c7c0 tx=0x7f6df400cc80 comp rx=0 tx=0).stop 2023-12-19T16:43:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.775+0000 7f6e11466640 1 -- 172.21.15.26:0/1045482835 shutdown_connections 2023-12-19T16:43:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.775+0000 7f6e11466640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e0c15cdf0 0x7f6e0c157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.775+0000 7f6e11466640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e0c0b3760 0x7f6e0c1536e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.775+0000 7f6e11466640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6dec065a60 0x7f6dec067f20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.775+0000 7f6e11466640 1 --2- 172.21.15.26:0/1045482835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e0c0b09d0 0x7f6e0c151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.775+0000 7f6e11466640 1 -- 172.21.15.26:0/1045482835 >> 172.21.15.26:0/1045482835 conn(0x7f6e0c0733c0 msgr2=0x7f6e0c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.775+0000 7f6e11466640 1 -- 172.21.15.26:0/1045482835 shutdown_connections 2023-12-19T16:43:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.775+0000 7f6e11466640 1 -- 172.21.15.26:0/1045482835 wait complete. 2023-12-19T16:43:06.799 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:43:06.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.939+0000 7f87b4ee5640 1 Processor -- start 2023-12-19T16:43:06.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.939+0000 7f87b4ee5640 1 -- start start 2023-12-19T16:43:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.939+0000 7f87b4ee5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f87b006d450 0x7f87b015a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.939+0000 7f87b4ee5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87b015ab30 0x7f87b015d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.939+0000 7f87b4ee5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b015a160 con 0x7f87b0159940 2023-12-19T16:43:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.939+0000 7f87b4ee5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b006d8d0 con 0x7f87b006d450 2023-12-19T16:43:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.939+0000 7f87b4ee5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00cb910 con 0x7f87b015ab30 2023-12-19T16:43:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.939+0000 7f87ae59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f87b0159940 0x7f87b00cb110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45632/0 (socket says 172.21.15.26:45632) 2023-12-19T16:43:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.939+0000 7f87ae59c640 1 -- 172.21.15.26:0/3617775665 learned_addr learned my addr 172.21.15.26:0/3617775665 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.943+0000 7f87aed9d640 1 --2- 172.21.15.26:0/3617775665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87b015ab30 0x7f87b015d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.943+0000 7f87a7fff640 1 --2- 172.21.15.26:0/3617775665 >> v2:172.21.15.160:3300/0 conn(0x7f87b006d450 0x7f87b015a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.943+0000 7f87add9b640 1 -- 172.21.15.26:0/3617775665 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888949622 0 0) 0x7f87b015a160 con 0x7f87b0159940 2023-12-19T16:43:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.943+0000 7f87add9b640 1 -- 172.21.15.26:0/3617775665 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f879c003530 con 0x7f87b0159940 2023-12-19T16:43:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.943+0000 7f87aed9d640 1 -- 172.21.15.26:0/3617775665 >> v2:172.21.15.160:3300/0 conn(0x7f87b006d450 msgr2=0x7f87b015a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.943+0000 7f87aed9d640 1 --2- 172.21.15.26:0/3617775665 >> v2:172.21.15.160:3300/0 conn(0x7f87b006d450 0x7f87b015a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.943+0000 7f87aed9d640 1 -- 172.21.15.26:0/3617775665 >> v1:172.21.15.26:6789/0 conn(0x7f87b0159940 legacy=0x7f87b00cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.943+0000 7f87add9b640 1 -- 172.21.15.26:0/3617775665 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3031362202 0 0) 0x7f879c003530 con 0x7f87b0159940 2023-12-19T16:43:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.943+0000 7f87aed9d640 1 -- 172.21.15.26:0/3617775665 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b00cbbf0 con 0x7f87b015ab30 2023-12-19T16:43:06.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.943+0000 7f87aed9d640 1 --2- 172.21.15.26:0/3617775665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87b015ab30 0x7f87b015d4d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009d90 tx=0x7f87a801c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dc1cfa5a7490cb2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:06.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.943+0000 7f87add9b640 1 -- 172.21.15.26:0/3617775665 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0001680 con 0x7f87b015ab30 2023-12-19T16:43:06.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.943+0000 7f87add9b640 1 -- 172.21.15.26:0/3617775665 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a80a5550 con 0x7f87b015ab30 2023-12-19T16:43:06.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.943+0000 7f87add9b640 1 -- 172.21.15.26:0/3617775665 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a80a58c0 con 0x7f87b015ab30 2023-12-19T16:43:06.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 -- 172.21.15.26:0/3617775665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87b015ab30 msgr2=0x7f87b015d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 --2- 172.21.15.26:0/3617775665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87b015ab30 0x7f87b015d4d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009d90 tx=0x7f87a801c840 comp rx=0 tx=0).stop 2023-12-19T16:43:06.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 -- 172.21.15.26:0/3617775665 shutdown_connections 2023-12-19T16:43:06.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 --2- 172.21.15.26:0/3617775665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87b015ab30 0x7f87b015d4d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 --2- 172.21.15.26:0/3617775665 >> v2:172.21.15.160:3300/0 conn(0x7f87b006d450 0x7f87b015a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 -- 172.21.15.26:0/3617775665 >> 172.21.15.26:0/3617775665 conn(0x7f87b0069d20 msgr2=0x7f87b006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:06.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 -- 172.21.15.26:0/3617775665 shutdown_connections 2023-12-19T16:43:06.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 -- 172.21.15.26:0/3617775665 wait complete. 2023-12-19T16:43:06.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 Processor -- start 2023-12-19T16:43:06.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 -- start start 2023-12-19T16:43:06.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87b006d450 0x7f87b0148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87b0159940 0x7f87b00cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b015ab30 0x7f87b00cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0153dc0 con 0x7f87b015ab30 2023-12-19T16:43:06.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0148fe0 con 0x7f87b006d450 2023-12-19T16:43:06.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87b4ee5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0152ff0 con 0x7f87b0159940 2023-12-19T16:43:06.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87aed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b015ab30 0x7f87b00cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87aed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b015ab30 0x7f87b00cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53940/0 (socket says 172.21.15.26:53940) 2023-12-19T16:43:06.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87aed9d640 1 -- 172.21.15.26:0/203468886 learned_addr learned my addr 172.21.15.26:0/203468886 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:06.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87ae59c640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87b006d450 0x7f87b0148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.947+0000 7f87a7fff640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87b0159940 0x7f87b00cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87aed9d640 1 -- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87b0159940 msgr2=0x7f87b00cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87aed9d640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87b0159940 0x7f87b00cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87aed9d640 1 -- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87b006d450 msgr2=0x7f87b0148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:06.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87aed9d640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87b006d450 0x7f87b0148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:06.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87aed9d640 1 -- 172.21.15.26:0/203468886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a8009a70 con 0x7f87b015ab30 2023-12-19T16:43:06.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87a7fff640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87b0159940 0x7f87b00cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:06.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87aed9d640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b015ab30 0x7f87b00cd9a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009d90 tx=0x7f87a80195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:06.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87a6ffd640 1 -- 172.21.15.26:0/203468886 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8007000 con 0x7f87b015ab30 2023-12-19T16:43:06.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87a6ffd640 1 -- 172.21.15.26:0/203468886 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a8009450 con 0x7f87b015ab30 2023-12-19T16:43:06.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87b4ee5640 1 -- 172.21.15.26:0/203468886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b01532d0 con 0x7f87b015ab30 2023-12-19T16:43:06.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87a6ffd640 1 -- 172.21.15.26:0/203468886 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a80ad660 con 0x7f87b015ab30 2023-12-19T16:43:06.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87b4ee5640 1 -- 172.21.15.26:0/203468886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b0153890 con 0x7f87b015ab30 2023-12-19T16:43:06.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87b4ee5640 1 -- 172.21.15.26:0/203468886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b0153f00 con 0x7f87b015ab30 2023-12-19T16:43:06.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87a6ffd640 1 -- 172.21.15.26:0/203468886 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f87a80ad7c0 con 0x7f87b015ab30 2023-12-19T16:43:06.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87a6ffd640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8790065a90 0x7f8790067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:06.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87a6ffd640 1 -- 172.21.15.26:0/203468886 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87a811dc80 con 0x7f87b015ab30 2023-12-19T16:43:06.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.951+0000 7f87ae59c640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8790065a90 0x7f8790067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:06.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.955+0000 7f87ae59c640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8790065a90 0x7f8790067f50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f87a0006b60 tx=0x7f87a0001dd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:06.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:06.955+0000 7f87a6ffd640 1 -- 172.21.15.26:0/203468886 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f87b0153f00 con 0x7f87b015ab30 2023-12-19T16:43:07.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.111+0000 7f87b4ee5640 1 -- 172.21.15.26:0/203468886 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87b0153f00 con 0x7f87b015ab30 2023-12-19T16:43:07.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.111+0000 7f87a6ffd640 1 -- 172.21.15.26:0/203468886 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87b0153f00 con 0x7f87b015ab30 2023-12-19T16:43:07.117 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:43:07.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.111+0000 7f87b4ee5640 1 -- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8790065a90 msgr2=0x7f8790067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.111+0000 7f87b4ee5640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8790065a90 0x7f8790067f50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f87a0006b60 tx=0x7f87a0001dd0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.111+0000 7f87b4ee5640 1 -- 172.21.15.26:0/203468886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b015ab30 msgr2=0x7f87b00cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.111+0000 7f87b4ee5640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b015ab30 0x7f87b00cd9a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009d90 tx=0x7f87a80195e0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.111+0000 7f87b4ee5640 1 -- 172.21.15.26:0/203468886 shutdown_connections 2023-12-19T16:43:07.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.111+0000 7f87b4ee5640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87b015ab30 0x7f87b00cd9a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.111+0000 7f87b4ee5640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8790065a90 0x7f8790067f50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.111+0000 7f87b4ee5640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87b0159940 0x7f87b00cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.111+0000 7f87b4ee5640 1 --2- 172.21.15.26:0/203468886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87b006d450 0x7f87b0148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.111+0000 7f87b4ee5640 1 -- 172.21.15.26:0/203468886 >> 172.21.15.26:0/203468886 conn(0x7f87b0069d20 msgr2=0x7f87b0070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:07.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.111+0000 7f87b4ee5640 1 -- 172.21.15.26:0/203468886 shutdown_connections 2023-12-19T16:43:07.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.115+0000 7f87b4ee5640 1 -- 172.21.15.26:0/203468886 wait complete. 2023-12-19T16:43:07.138 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:43:07.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b3b88c640 1 Processor -- start 2023-12-19T16:43:07.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b3b88c640 1 -- start start 2023-12-19T16:43:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b3b88c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b3415cde0 0x7f3b340b8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b3b88c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3b340b4d00 0x7f3b340d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b3b88c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b340b5bd0 con 0x7f3b3415d1c0 2023-12-19T16:43:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b3b88c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3415e380 con 0x7f3b340b4d00 2023-12-19T16:43:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b3b88c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340b8b40 con 0x7f3b3415cde0 2023-12-19T16:43:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b39e29640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3b340b4d00 0x7f3b340d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b39e29640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3b340b4d00 0x7f3b340d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53946/0 (socket says 172.21.15.26:53946) 2023-12-19T16:43:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b39e29640 1 -- 172.21.15.26:0/2810387150 learned_addr learned my addr 172.21.15.26:0/2810387150 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b39628640 1 --2- 172.21.15.26:0/2810387150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b3415cde0 0x7f3b340b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b39e29640 1 -- 172.21.15.26:0/2810387150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b3415cde0 msgr2=0x7f3b340b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b39e29640 1 --2- 172.21.15.26:0/2810387150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b3415cde0 0x7f3b340b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b39e29640 1 -- 172.21.15.26:0/2810387150 >> v1:172.21.15.160:6789/0 conn(0x7f3b3415d1c0 legacy=0x7f3b340b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.279+0000 7f3b39e29640 1 -- 172.21.15.26:0/2810387150 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b340d0ee0 con 0x7f3b340b4d00 2023-12-19T16:43:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b39e29640 1 --2- 172.21.15.26:0/2810387150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b340b4d00 0x7f3b340d1370 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009e80 tx=0x7f3b2c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f9c67672c5c3e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b2bfff640 1 -- 172.21.15.26:0/2810387150 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c0af400 con 0x7f3b340b4d00 2023-12-19T16:43:07.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b2bfff640 1 -- 172.21.15.26:0/2810387150 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c0ab050 con 0x7f3b340b4d00 2023-12-19T16:43:07.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b2bfff640 1 -- 172.21.15.26:0/2810387150 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c0af780 con 0x7f3b340b4d00 2023-12-19T16:43:07.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b3b88c640 1 -- 172.21.15.26:0/2810387150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b340b4d00 msgr2=0x7f3b340d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b3b88c640 1 --2- 172.21.15.26:0/2810387150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b340b4d00 0x7f3b340d1370 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009e80 tx=0x7f3b2c01f1e0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b3b88c640 1 -- 172.21.15.26:0/2810387150 shutdown_connections 2023-12-19T16:43:07.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b3b88c640 1 --2- 172.21.15.26:0/2810387150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b340b4d00 0x7f3b340d1370 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b3b88c640 1 --2- 172.21.15.26:0/2810387150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b3415cde0 0x7f3b340b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b3b88c640 1 -- 172.21.15.26:0/2810387150 >> 172.21.15.26:0/2810387150 conn(0x7f3b34065ee0 msgr2=0x7f3b340d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:07.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b3b88c640 1 -- 172.21.15.26:0/2810387150 shutdown_connections 2023-12-19T16:43:07.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b3b88c640 1 -- 172.21.15.26:0/2810387150 wait complete. 2023-12-19T16:43:07.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b3b88c640 1 Processor -- start 2023-12-19T16:43:07.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.283+0000 7f3b3b88c640 1 -- start start 2023-12-19T16:43:07.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b3b88c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b340b4d00 0x7f3b341f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b3b88c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b3415cde0 0x7f3b3414f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b3b88c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3415d1c0 0x7f3b341f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b3b88c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3414fdc0 con 0x7f3b3415d1c0 2023-12-19T16:43:07.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b3b88c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341500a0 con 0x7f3b340b4d00 2023-12-19T16:43:07.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b3b88c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b34150380 con 0x7f3b3415cde0 2023-12-19T16:43:07.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b39e29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3415d1c0 0x7f3b341f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b39e29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3415d1c0 0x7f3b341f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53960/0 (socket says 172.21.15.26:53960) 2023-12-19T16:43:07.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b39628640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b340b4d00 0x7f3b341f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b39e29640 1 -- 172.21.15.26:0/3558106891 learned_addr learned my addr 172.21.15.26:0/3558106891 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:07.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b38e27640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b3415cde0 0x7f3b3414f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b39e29640 1 -- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b3415cde0 msgr2=0x7f3b3414f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b39e29640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b3415cde0 0x7f3b3414f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b39e29640 1 -- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b340b4d00 msgr2=0x7f3b341f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b39e29640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b340b4d00 0x7f3b341f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b39e29640 1 -- 172.21.15.26:0/3558106891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c009b30 con 0x7f3b3415d1c0 2023-12-19T16:43:07.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b38e27640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b3415cde0 0x7f3b3414f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:07.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b39e29640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3415d1c0 0x7f3b341f5080 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0b2040 tx=0x7f3b2c01bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:07.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b29ffb640 1 -- 172.21.15.26:0/3558106891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c0afd30 con 0x7f3b3415d1c0 2023-12-19T16:43:07.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b29ffb640 1 -- 172.21.15.26:0/3558106891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c009510 con 0x7f3b3415d1c0 2023-12-19T16:43:07.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b29ffb640 1 -- 172.21.15.26:0/3558106891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c0ba660 con 0x7f3b3415d1c0 2023-12-19T16:43:07.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b3b88c640 1 -- 172.21.15.26:0/3558106891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b34150660 con 0x7f3b3415d1c0 2023-12-19T16:43:07.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b3b88c640 1 -- 172.21.15.26:0/3558106891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3414fad0 con 0x7f3b3415d1c0 2023-12-19T16:43:07.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.287+0000 7f3b3b88c640 1 -- 172.21.15.26:0/3558106891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b34074270 con 0x7f3b3415d1c0 2023-12-19T16:43:07.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.291+0000 7f3b29ffb640 1 -- 172.21.15.26:0/3558106891 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3b2c0b6040 con 0x7f3b3415d1c0 2023-12-19T16:43:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.291+0000 7f3b29ffb640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3b18065a60 0x7f3b18067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.291+0000 7f3b29ffb640 1 -- 172.21.15.26:0/3558106891 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b2c1212a0 con 0x7f3b3415d1c0 2023-12-19T16:43:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.291+0000 7f3b39628640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3b18065a60 0x7f3b18067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.291+0000 7f3b39628640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3b18065a60 0x7f3b18067f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009b60 tx=0x7f3b30009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:07.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.295+0000 7f3b29ffb640 1 -- 172.21.15.26:0/3558106891 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3b2c0eec50 con 0x7f3b3415d1c0 2023-12-19T16:43:07.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.455+0000 7f3b3b88c640 1 -- 172.21.15.26:0/3558106891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3b3415c270 con 0x7f3b3415d1c0 2023-12-19T16:43:07.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.455+0000 7f3b29ffb640 1 -- 172.21.15.26:0/3558106891 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3b2c0f2270 con 0x7f3b3415d1c0 2023-12-19T16:43:07.461 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:43:07.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.459+0000 7f3b3b88c640 1 -- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3b18065a60 msgr2=0x7f3b18067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.459+0000 7f3b3b88c640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3b18065a60 0x7f3b18067f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009b60 tx=0x7f3b30009190 comp rx=0 tx=0).stop 2023-12-19T16:43:07.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.459+0000 7f3b3b88c640 1 -- 172.21.15.26:0/3558106891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3415d1c0 msgr2=0x7f3b341f5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.459+0000 7f3b3b88c640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3415d1c0 0x7f3b341f5080 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0b2040 tx=0x7f3b2c01bf80 comp rx=0 tx=0).stop 2023-12-19T16:43:07.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.459+0000 7f3b3b88c640 1 -- 172.21.15.26:0/3558106891 shutdown_connections 2023-12-19T16:43:07.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.459+0000 7f3b3b88c640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b3415d1c0 0x7f3b341f5080 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.459+0000 7f3b3b88c640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3b18065a60 0x7f3b18067f20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.459+0000 7f3b3b88c640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b3415cde0 0x7f3b3414f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.459+0000 7f3b3b88c640 1 --2- 172.21.15.26:0/3558106891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b340b4d00 0x7f3b341f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.459+0000 7f3b3b88c640 1 -- 172.21.15.26:0/3558106891 >> 172.21.15.26:0/3558106891 conn(0x7f3b34065ee0 msgr2=0x7f3b3415fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.459+0000 7f3b3b88c640 1 -- 172.21.15.26:0/3558106891 shutdown_connections 2023-12-19T16:43:07.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.459+0000 7f3b3b88c640 1 -- 172.21.15.26:0/3558106891 wait complete. 2023-12-19T16:43:07.482 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:43:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.619+0000 7fd625a8c640 1 Processor -- start 2023-12-19T16:43:07.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.619+0000 7fd625a8c640 1 -- start start 2023-12-19T16:43:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.619+0000 7fd625a8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd62006b3d0 0x7fd6200ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd625a8c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd6200cec40 0x7fd62015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd625a8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62006bc30 con 0x7fd620154220 2023-12-19T16:43:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd625a8c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6200cf0d0 con 0x7fd6200cec40 2023-12-19T16:43:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd625a8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200cc6e0 con 0x7fd62006b3d0 2023-12-19T16:43:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd620154220 0x7fd62015aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45636/0 (socket says 172.21.15.26:45636) 2023-12-19T16:43:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61f7fe640 1 -- 172.21.15.26:0/2150548703 learned_addr learned my addr 172.21.15.26:0/2150548703 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61ffff640 1 --2- 172.21.15.26:0/2150548703 >> v2:172.21.15.160:3300/0 conn(0x7fd6200cec40 0x7fd62015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740301121 0 0) 0x7fd62006bc30 con 0x7fd620154220 2023-12-19T16:43:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd608003530 con 0x7fd620154220 2023-12-19T16:43:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61effd640 1 --2- 172.21.15.26:0/2150548703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd62006b3d0 0x7fd6200ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2828125465 0 0) 0x7fd608003530 con 0x7fd620154220 2023-12-19T16:43:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd62006b3d0 msgr2=0x7fd6200ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 --2- 172.21.15.26:0/2150548703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd62006b3d0 0x7fd6200ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 >> v2:172.21.15.160:3300/0 conn(0x7fd6200cec40 msgr2=0x7fd62015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 --2- 172.21.15.26:0/2150548703 >> v2:172.21.15.160:3300/0 conn(0x7fd6200cec40 0x7fd62015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62006b950 con 0x7fd620154220 2023-12-19T16:43:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fd610001680 con 0x7fd620154220 2023-12-19T16:43:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 >> v1:172.21.15.26:6789/0 conn(0x7fd620154220 legacy=0x7fd62015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 --2- 172.21.15.26:0/2150548703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd62006b3d0 0x7fd608003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 --2- 172.21.15.26:0/2150548703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd608003f00 0x7fd6080047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 --2- 172.21.15.26:0/2150548703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd608004d10 0x7fd608005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200cf0d0 con 0x7fd608003f00 2023-12-19T16:43:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200cc6e0 con 0x7fd608004d10 2023-12-19T16:43:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd60800dba0 con 0x7fd62006b3d0 2023-12-19T16:43:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61effd640 1 --2- 172.21.15.26:0/2150548703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd608003f00 0x7fd6080047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61f7fe640 1 --2- 172.21.15.26:0/2150548703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd62006b3d0 0x7fd608003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61ffff640 1 --2- 172.21.15.26:0/2150548703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd608004d10 0x7fd608005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61effd640 1 -- 172.21.15.26:0/2150548703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd62006b3d0 msgr2=0x7fd608003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61effd640 1 --2- 172.21.15.26:0/2150548703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd62006b3d0 0x7fd608003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61effd640 1 -- 172.21.15.26:0/2150548703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd608004d10 msgr2=0x7fd608005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61effd640 1 --2- 172.21.15.26:0/2150548703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd608004d10 0x7fd608005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61effd640 1 -- 172.21.15.26:0/2150548703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd608003530 con 0x7fd608003f00 2023-12-19T16:43:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.623+0000 7fd61ffff640 1 --2- 172.21.15.26:0/2150548703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd608004d10 0x7fd608005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.627+0000 7fd61effd640 1 --2- 172.21.15.26:0/2150548703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd608003f00 0x7fd6080047d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd614009ae0 tx=0x7fd61401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bcb81e7cc5ee3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.627+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140a5640 con 0x7fd608003f00 2023-12-19T16:43:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.627+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6140a57a0 con 0x7fd608003f00 2023-12-19T16:43:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.627+0000 7fd61e7fc640 1 -- 172.21.15.26:0/2150548703 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140a5b50 con 0x7fd608003f00 2023-12-19T16:43:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.627+0000 7fd625a8c640 1 -- 172.21.15.26:0/2150548703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd608003f00 msgr2=0x7fd6080047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.627+0000 7fd625a8c640 1 --2- 172.21.15.26:0/2150548703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd608003f00 0x7fd6080047d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd614009ae0 tx=0x7fd61401c9f0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.627+0000 7fd61ffff640 1 -- 172.21.15.26:0/2150548703 reap_dead start 2023-12-19T16:43:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.627+0000 7fd625a8c640 1 -- 172.21.15.26:0/2150548703 shutdown_connections 2023-12-19T16:43:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.627+0000 7fd625a8c640 1 -- 172.21.15.26:0/2150548703 >> 172.21.15.26:0/2150548703 conn(0x7fd6200658e0 msgr2=0x7fd620065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:07.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.627+0000 7fd625a8c640 1 -- 172.21.15.26:0/2150548703 shutdown_connections 2023-12-19T16:43:07.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.627+0000 7fd625a8c640 1 -- 172.21.15.26:0/2150548703 wait complete. 2023-12-19T16:43:07.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.627+0000 7fd625a8c640 1 Processor -- start 2023-12-19T16:43:07.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.627+0000 7fd625a8c640 1 -- start start 2023-12-19T16:43:07.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd625a8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd608003f00 0x7fd62014d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd625a8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6200acda0 0x7fd6200cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd625a8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6200cb730 0x7fd620154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd625a8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200703c0 con 0x7fd6200acda0 2023-12-19T16:43:07.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd625a8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200cbbb0 con 0x7fd6200cb730 2023-12-19T16:43:07.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd625a8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200711e0 con 0x7fd608003f00 2023-12-19T16:43:07.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd61effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6200acda0 0x7fd6200cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd61effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6200acda0 0x7fd6200cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53984/0 (socket says 172.21.15.26:53984) 2023-12-19T16:43:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd61effd640 1 -- 172.21.15.26:0/2823441235 learned_addr learned my addr 172.21.15.26:0/2823441235 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd61f7fe640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd608003f00 0x7fd62014d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd61ffff640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6200cb730 0x7fd620154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd61effd640 1 -- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd608003f00 msgr2=0x7fd62014d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd61effd640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd608003f00 0x7fd62014d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd61effd640 1 -- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6200cb730 msgr2=0x7fd620154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd61effd640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6200cb730 0x7fd620154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd61effd640 1 -- 172.21.15.26:0/2823441235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62014d560 con 0x7fd6200acda0 2023-12-19T16:43:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd61f7fe640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd608003f00 0x7fd62014d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd61ffff640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6200cb730 0x7fd620154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd61effd640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6200acda0 0x7fd6200cb1f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd614009ae0 tx=0x7fd608005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd607fff640 1 -- 172.21.15.26:0/2823441235 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140bc070 con 0x7fd6200acda0 2023-12-19T16:43:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd607fff640 1 -- 172.21.15.26:0/2823441235 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd614004020 con 0x7fd6200acda0 2023-12-19T16:43:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd607fff640 1 -- 172.21.15.26:0/2823441235 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614004390 con 0x7fd6200acda0 2023-12-19T16:43:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd625a8c640 1 -- 172.21.15.26:0/2823441235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6200705d0 con 0x7fd6200acda0 2023-12-19T16:43:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd625a8c640 1 -- 172.21.15.26:0/2823441235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd620070b90 con 0x7fd6200acda0 2023-12-19T16:43:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd625a8c640 1 -- 172.21.15.26:0/2823441235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd620074270 con 0x7fd6200acda0 2023-12-19T16:43:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.631+0000 7fd607fff640 1 -- 172.21.15.26:0/2823441235 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd6140b4040 con 0x7fd6200acda0 2023-12-19T16:43:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.635+0000 7fd607fff640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd60c065a90 0x7fd60c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.635+0000 7fd607fff640 1 -- 172.21.15.26:0/2823441235 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd61411ca50 con 0x7fd6200acda0 2023-12-19T16:43:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.635+0000 7fd61f7fe640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd60c065a90 0x7fd60c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.635+0000 7fd61f7fe640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd60c065a90 0x7fd60c067f50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd610000c10 tx=0x7fd610007900 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:07.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.639+0000 7fd607fff640 1 -- 172.21.15.26:0/2823441235 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd6140ea4b0 con 0x7fd6200acda0 2023-12-19T16:43:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.831+0000 7fd625a8c640 1 -- 172.21.15.26:0/2823441235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6200cdf60 con 0x7fd6200acda0 2023-12-19T16:43:07.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.831+0000 7fd607fff640 1 -- 172.21.15.26:0/2823441235 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fd6140edad0 con 0x7fd6200acda0 2023-12-19T16:43:07.838 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:07.838 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2850479611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2850479611},{"type":"v1","addr":"172.21.15.160:6836","nonce":2850479611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/837780144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":837780144},{"type":"v1","addr":"172.21.15.26:6834","nonce":837780144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2784408519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2784408519},{"type":"v1","addr":"172.21.15.26:6835","nonce":2784408519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:42:27.777197+0000","modified":"2023-12-19T16:42:30.798296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6837/3427345273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3427345273},{"type":"v1","addr":"172.21.15.160:6837","nonce":3427345273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T16:43:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2023-12-19T16:43:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.835+0000 7fd625a8c640 1 -- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd60c065a90 msgr2=0x7fd60c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.835+0000 7fd625a8c640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd60c065a90 0x7fd60c067f50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd610000c10 tx=0x7fd610007900 comp rx=0 tx=0).stop 2023-12-19T16:43:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.835+0000 7fd625a8c640 1 -- 172.21.15.26:0/2823441235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6200acda0 msgr2=0x7fd6200cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.835+0000 7fd625a8c640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6200acda0 0x7fd6200cb1f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd614009ae0 tx=0x7fd608005800 comp rx=0 tx=0).stop 2023-12-19T16:43:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.835+0000 7fd625a8c640 1 -- 172.21.15.26:0/2823441235 shutdown_connections 2023-12-19T16:43:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.835+0000 7fd625a8c640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6200cb730 0x7fd620154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.835+0000 7fd625a8c640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6200acda0 0x7fd6200cb1f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.835+0000 7fd625a8c640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd60c065a90 0x7fd60c067f50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.835+0000 7fd625a8c640 1 --2- 172.21.15.26:0/2823441235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd608003f00 0x7fd62014d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.835+0000 7fd625a8c640 1 -- 172.21.15.26:0/2823441235 >> 172.21.15.26:0/2823441235 conn(0x7fd6200658e0 msgr2=0x7fd620065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.835+0000 7fd625a8c640 1 -- 172.21.15.26:0/2823441235 shutdown_connections 2023-12-19T16:43:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.835+0000 7fd625a8c640 1 -- 172.21.15.26:0/2823441235 wait complete. 2023-12-19T16:43:07.859 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:43:08.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.995+0000 7f8c6fe2d640 1 Processor -- start 2023-12-19T16:43:08.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.995+0000 7f8c6fe2d640 1 -- start start 2023-12-19T16:43:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6fe2d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8c681573f0 0x7f8c6815ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6fe2d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6815e110 con 0x7f8c681577d0 2023-12-19T16:43:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6fe2d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6815e3f0 con 0x7f8c680cea80 2023-12-19T16:43:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6fe2d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c6815a300 con 0x7f8c681573f0 2023-12-19T16:43:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6d3c8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8c681577d0 0x7f8c6815d5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45650/0 (socket says 172.21.15.26:45650) 2023-12-19T16:43:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6d3c8640 1 -- 172.21.15.26:0/2960630413 learned_addr learned my addr 172.21.15.26:0/2960630413 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6dbc9640 1 --2- 172.21.15.26:0/2960630413 >> v2:172.21.15.160:3300/0 conn(0x7f8c681573f0 0x7f8c6815ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6cbc7640 1 -- 172.21.15.26:0/2960630413 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93913833 0 0) 0x7f8c6815e110 con 0x7f8c681577d0 2023-12-19T16:43:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6cbc7640 1 -- 172.21.15.26:0/2960630413 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c54003530 con 0x7f8c681577d0 2023-12-19T16:43:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6cbc7640 1 -- 172.21.15.26:0/2960630413 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720608100 0 0) 0x7f8c6815e3f0 con 0x7f8c680cea80 2023-12-19T16:43:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6cbc7640 1 -- 172.21.15.26:0/2960630413 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c6815e110 con 0x7f8c680cea80 2023-12-19T16:43:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6dbc9640 1 -- 172.21.15.26:0/2960630413 >> v1:172.21.15.160:6789/0 conn(0x7f8c680cea80 legacy=0x7f8c6806f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6dbc9640 1 -- 172.21.15.26:0/2960630413 >> v1:172.21.15.26:6789/0 conn(0x7f8c681577d0 legacy=0x7f8c6815d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6cbc7640 1 -- 172.21.15.26:0/2960630413 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4012950803 0 0) 0x7f8c54003530 con 0x7f8c681577d0 2023-12-19T16:43:08.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:07.999+0000 7f8c6dbc9640 1 -- 172.21.15.26:0/2960630413 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6815a5e0 con 0x7f8c681573f0 2023-12-19T16:43:08.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.003+0000 7f8c6dbc9640 1 --2- 172.21.15.26:0/2960630413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c681573f0 0x7f8c6815ab00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8c640097c0 tx=0x7f8c6401c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa6c26201ac82140 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:08.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.003+0000 7f8c6cbc7640 1 -- 172.21.15.26:0/2960630413 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58001680 con 0x7f8c681573f0 2023-12-19T16:43:08.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.003+0000 7f8c6cbc7640 1 -- 172.21.15.26:0/2960630413 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c640a7050 con 0x7f8c681573f0 2023-12-19T16:43:08.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.003+0000 7f8c6cbc7640 1 -- 172.21.15.26:0/2960630413 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c640ab600 con 0x7f8c681573f0 2023-12-19T16:43:08.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.003+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/2960630413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c681573f0 msgr2=0x7f8c6815ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.003+0000 7f8c6fe2d640 1 --2- 172.21.15.26:0/2960630413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c681573f0 0x7f8c6815ab00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8c640097c0 tx=0x7f8c6401c360 comp rx=0 tx=0).stop 2023-12-19T16:43:08.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.003+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/2960630413 shutdown_connections 2023-12-19T16:43:08.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.003+0000 7f8c6fe2d640 1 --2- 172.21.15.26:0/2960630413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c681573f0 0x7f8c6815ab00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.003+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/2960630413 >> 172.21.15.26:0/2960630413 conn(0x7f8c680b1aa0 msgr2=0x7f8c680b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:08.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.003+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/2960630413 shutdown_connections 2023-12-19T16:43:08.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.003+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/2960630413 wait complete. 2023-12-19T16:43:08.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.003+0000 7f8c6fe2d640 1 Processor -- start 2023-12-19T16:43:08.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.003+0000 7f8c6fe2d640 1 -- start start 2023-12-19T16:43:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6fe2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c680cea80 0x7f8c681532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6fe2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c681573f0 0x7f8c6814aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6fe2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c681577d0 0x7f8c68154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6fe2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6814fdc0 con 0x7f8c681577d0 2023-12-19T16:43:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6fe2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c68151190 con 0x7f8c681573f0 2023-12-19T16:43:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6fe2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c681537f0 con 0x7f8c680cea80 2023-12-19T16:43:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6e3ca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c681577d0 0x7f8c68154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6e3ca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c681577d0 0x7f8c68154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53998/0 (socket says 172.21.15.26:53998) 2023-12-19T16:43:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6e3ca640 1 -- 172.21.15.26:0/1249078189 learned_addr learned my addr 172.21.15.26:0/1249078189 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6dbc9640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c680cea80 0x7f8c681532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6d3c8640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c681573f0 0x7f8c6814aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6e3ca640 1 -- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c680cea80 msgr2=0x7f8c681532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6e3ca640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c680cea80 0x7f8c681532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6e3ca640 1 -- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c681573f0 msgr2=0x7f8c6814aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6e3ca640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c681573f0 0x7f8c6814aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6e3ca640 1 -- 172.21.15.26:0/1249078189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c64009420 con 0x7f8c681577d0 2023-12-19T16:43:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6e3ca640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c681577d0 0x7f8c68154af0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000c760 tx=0x7f8c6000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c5e7fc640 1 -- 172.21.15.26:0/1249078189 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c60003b80 con 0x7f8c681577d0 2023-12-19T16:43:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c5e7fc640 1 -- 172.21.15.26:0/1249078189 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c60003ce0 con 0x7f8c681577d0 2023-12-19T16:43:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/1249078189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c680cf420 con 0x7f8c681577d0 2023-12-19T16:43:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c5e7fc640 1 -- 172.21.15.26:0/1249078189 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6000aa90 con 0x7f8c681577d0 2023-12-19T16:43:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/1249078189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c681516d0 con 0x7f8c681577d0 2023-12-19T16:43:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/1249078189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6815c2e0 con 0x7f8c681577d0 2023-12-19T16:43:08.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.007+0000 7f8c5e7fc640 1 -- 172.21.15.26:0/1249078189 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8c6000abf0 con 0x7f8c681577d0 2023-12-19T16:43:08.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.011+0000 7f8c5e7fc640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c4c065a60 0x7f8c4c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.011+0000 7f8c5e7fc640 1 -- 172.21.15.26:0/1249078189 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c60082f40 con 0x7f8c681577d0 2023-12-19T16:43:08.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.011+0000 7f8c6dbc9640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c4c065a60 0x7f8c4c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.011+0000 7f8c6dbc9640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c4c065a60 0x7f8c4c067f20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8c640ad000 tx=0x7f8c640a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:08.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.015+0000 7f8c5e7fc640 1 -- 172.21.15.26:0/1249078189 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8c60050910 con 0x7f8c681577d0 2023-12-19T16:43:08.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.171+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/1249078189 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c68074270 con 0x7f8c681577d0 2023-12-19T16:43:08.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.175+0000 7f8c5e7fc640 1 -- 172.21.15.26:0/1249078189 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16037 (secure 0 0 0) 0x7f8c60053f30 con 0x7f8c681577d0 2023-12-19T16:43:08.179 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:08.179 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":23,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:42:28.779699+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:43:08.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.175+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c4c065a60 msgr2=0x7f8c4c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.175+0000 7f8c6fe2d640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c4c065a60 0x7f8c4c067f20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8c640ad000 tx=0x7f8c640a6000 comp rx=0 tx=0).stop 2023-12-19T16:43:08.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.175+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/1249078189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c681577d0 msgr2=0x7f8c68154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.175+0000 7f8c6fe2d640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c681577d0 0x7f8c68154af0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000c760 tx=0x7f8c6000cc20 comp rx=0 tx=0).stop 2023-12-19T16:43:08.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.175+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/1249078189 shutdown_connections 2023-12-19T16:43:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.175+0000 7f8c6fe2d640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c681577d0 0x7f8c68154af0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.175+0000 7f8c6fe2d640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c681573f0 0x7f8c6814aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.175+0000 7f8c6fe2d640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c4c065a60 0x7f8c4c067f20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.175+0000 7f8c6fe2d640 1 --2- 172.21.15.26:0/1249078189 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c680cea80 0x7f8c681532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.175+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/1249078189 >> 172.21.15.26:0/1249078189 conn(0x7f8c680b1aa0 msgr2=0x7f8c6806b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.175+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/1249078189 shutdown_connections 2023-12-19T16:43:08.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.175+0000 7f8c6fe2d640 1 -- 172.21.15.26:0/1249078189 wait complete. 2023-12-19T16:43:08.201 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T16:43:08.201 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:43:08.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.339+0000 7f3272666640 1 Processor -- start 2023-12-19T16:43:08.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.339+0000 7f3272666640 1 -- start start 2023-12-19T16:43:08.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.339+0000 7f3272666640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f326c0cf190 0x7f326c0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.339+0000 7f3272666640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326c0c8cd0 con 0x7f326c06b9d0 2023-12-19T16:43:08.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.339+0000 7f3272666640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326c0ad6a0 con 0x7f326c15ab00 2023-12-19T16:43:08.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.339+0000 7f3272666640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326c0ad8f0 con 0x7f326c0cf190 2023-12-19T16:43:08.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.343+0000 7f326b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f326c0cf190 0x7f326c0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.343+0000 7f3270c03640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f326c06b9d0 0x7f326c0c93e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45666/0 (socket says 172.21.15.26:45666) 2023-12-19T16:43:08.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.343+0000 7f3270c03640 1 -- 172.21.15.26:0/1573876017 learned_addr learned my addr 172.21.15.26:0/1573876017 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:08.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.343+0000 7f326b7fe640 1 -- 172.21.15.26:0/1573876017 >> v1:172.21.15.160:6789/0 conn(0x7f326c15ab00 legacy=0x7f326c0cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.343+0000 7f326b7fe640 1 -- 172.21.15.26:0/1573876017 >> v1:172.21.15.26:6789/0 conn(0x7f326c06b9d0 legacy=0x7f326c0c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.343+0000 7f326b7fe640 1 -- 172.21.15.26:0/1573876017 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c0adbd0 con 0x7f326c0cf190 2023-12-19T16:43:08.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.343+0000 7f326affd640 1 -- 172.21.15.26:0/1573876017 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984242584 0 0) 0x7f326c0c8cd0 con 0x7f326c06b9d0 2023-12-19T16:43:08.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.343+0000 7f326b7fe640 1 --2- 172.21.15.26:0/1573876017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f326c0cf190 0x7f326c0c8ea0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f32600097c0 tx=0x7f326001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba272eb0dc3ade4d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:08.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.343+0000 7f326affd640 1 -- 172.21.15.26:0/1573876017 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32600ac620 con 0x7f326c0cf190 2023-12-19T16:43:08.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.343+0000 7f326affd640 1 -- 172.21.15.26:0/1573876017 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32600ac780 con 0x7f326c0cf190 2023-12-19T16:43:08.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.343+0000 7f326affd640 1 -- 172.21.15.26:0/1573876017 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32600acb10 con 0x7f326c0cf190 2023-12-19T16:43:08.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 -- 172.21.15.26:0/1573876017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f326c0cf190 msgr2=0x7f326c0c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 --2- 172.21.15.26:0/1573876017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f326c0cf190 0x7f326c0c8ea0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f32600097c0 tx=0x7f326001c3c0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 -- 172.21.15.26:0/1573876017 shutdown_connections 2023-12-19T16:43:08.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 --2- 172.21.15.26:0/1573876017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f326c0cf190 0x7f326c0c8ea0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 -- 172.21.15.26:0/1573876017 >> 172.21.15.26:0/1573876017 conn(0x7f326c0b1aa0 msgr2=0x7f326c0b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:08.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 -- 172.21.15.26:0/1573876017 shutdown_connections 2023-12-19T16:43:08.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 -- 172.21.15.26:0/1573876017 wait complete. 2023-12-19T16:43:08.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 Processor -- start 2023-12-19T16:43:08.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 -- start start 2023-12-19T16:43:08.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f326c06b9d0 0x7f326c14b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f326c0cf190 0x7f326c1549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f326c15ab00 0x7f326c159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c14c9b0 con 0x7f326c15ab00 2023-12-19T16:43:08.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326c14bda0 con 0x7f326c0cf190 2023-12-19T16:43:08.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3272666640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326c14c080 con 0x7f326c06b9d0 2023-12-19T16:43:08.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3270c03640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f326c15ab00 0x7f326c159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3270c03640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f326c15ab00 0x7f326c159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54008/0 (socket says 172.21.15.26:54008) 2023-12-19T16:43:08.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f3270c03640 1 -- 172.21.15.26:0/1258942447 learned_addr learned my addr 172.21.15.26:0/1258942447 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:08.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f326bfff640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f326c06b9d0 0x7f326c14b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.347+0000 7f326b7fe640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f326c0cf190 0x7f326c1549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3270c03640 1 -- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f326c06b9d0 msgr2=0x7f326c14b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3270c03640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f326c06b9d0 0x7f326c14b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3270c03640 1 -- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f326c0cf190 msgr2=0x7f326c1549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3270c03640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f326c0cf190 0x7f326c1549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3270c03640 1 -- 172.21.15.26:0/1258942447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3260009420 con 0x7f326c15ab00 2023-12-19T16:43:08.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3270c03640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f326c15ab00 0x7f326c159370 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f326400c780 tx=0x7f326400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:08.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3268ff9640 1 -- 172.21.15.26:0/1258942447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326400ce90 con 0x7f326c15ab00 2023-12-19T16:43:08.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3268ff9640 1 -- 172.21.15.26:0/1258942447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326400a640 con 0x7f326c15ab00 2023-12-19T16:43:08.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3268ff9640 1 -- 172.21.15.26:0/1258942447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326400a9d0 con 0x7f326c15ab00 2023-12-19T16:43:08.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3272666640 1 -- 172.21.15.26:0/1258942447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326c14c360 con 0x7f326c15ab00 2023-12-19T16:43:08.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3272666640 1 -- 172.21.15.26:0/1258942447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326c1598b0 con 0x7f326c15ab00 2023-12-19T16:43:08.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3272666640 1 -- 172.21.15.26:0/1258942447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326c14cb20 con 0x7f326c15ab00 2023-12-19T16:43:08.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3268ff9640 1 -- 172.21.15.26:0/1258942447 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f326400ab30 con 0x7f326c15ab00 2023-12-19T16:43:08.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3268ff9640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3258065a60 0x7f3258067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.351+0000 7f3268ff9640 1 -- 172.21.15.26:0/1258942447 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3264082650 con 0x7f326c15ab00 2023-12-19T16:43:08.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.355+0000 7f326bfff640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3258065a60 0x7f3258067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.355+0000 7f326bfff640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3258065a60 0x7f3258067f20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f325c007fd0 tx=0x7f325c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:08.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.355+0000 7f3268ff9640 1 -- 172.21.15.26:0/1258942447 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f32640500b0 con 0x7f326c15ab00 2023-12-19T16:43:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.551+0000 7f3272666640 1 -- 172.21.15.26:0/1258942447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f326c074270 con 0x7f326c15ab00 2023-12-19T16:43:08.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.551+0000 7f3268ff9640 1 -- 172.21.15.26:0/1258942447 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f3264018090 con 0x7f326c15ab00 2023-12-19T16:43:08.557 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:08.557 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2850479611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2850479611},{"type":"v1","addr":"172.21.15.160:6836","nonce":2850479611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/837780144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":837780144},{"type":"v1","addr":"172.21.15.26:6834","nonce":837780144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2784408519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2784408519},{"type":"v1","addr":"172.21.15.26:6835","nonce":2784408519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:42:27.777197+0000","modified":"2023-12-19T16:42:30.798296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6837/3427345273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3427345273},{"type":"v1","addr":"172.21.15.160:6837","nonce":3427345273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T16:43:08.557 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2023-12-19T16:43:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.555+0000 7f3272666640 1 -- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3258065a60 msgr2=0x7f3258067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.555+0000 7f3272666640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3258065a60 0x7f3258067f20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f325c007fd0 tx=0x7f325c009040 comp rx=0 tx=0).stop 2023-12-19T16:43:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.555+0000 7f3272666640 1 -- 172.21.15.26:0/1258942447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f326c15ab00 msgr2=0x7f326c159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.555+0000 7f3272666640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f326c15ab00 0x7f326c159370 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f326400c780 tx=0x7f326400cc40 comp rx=0 tx=0).stop 2023-12-19T16:43:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.555+0000 7f3272666640 1 -- 172.21.15.26:0/1258942447 shutdown_connections 2023-12-19T16:43:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.555+0000 7f3272666640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f326c15ab00 0x7f326c159370 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.555+0000 7f3272666640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f326c0cf190 0x7f326c1549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.555+0000 7f3272666640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3258065a60 0x7f3258067f20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.555+0000 7f3272666640 1 --2- 172.21.15.26:0/1258942447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f326c06b9d0 0x7f326c14b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.555+0000 7f3272666640 1 -- 172.21.15.26:0/1258942447 >> 172.21.15.26:0/1258942447 conn(0x7f326c0b1aa0 msgr2=0x7f326c14df90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:08.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.555+0000 7f3272666640 1 -- 172.21.15.26:0/1258942447 shutdown_connections 2023-12-19T16:43:08.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.555+0000 7f3272666640 1 -- 172.21.15.26:0/1258942447 wait complete. 2023-12-19T16:43:08.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:43:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.715+0000 7f2a43f8f640 1 Processor -- start 2023-12-19T16:43:08.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a43f8f640 1 -- start start 2023-12-19T16:43:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a43f8f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2a3c0cc850 0x7f2a3c0cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a43f8f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a3c0c92d0 0x7f2a3c0ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a43f8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c146140 0x7f2a3c1609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a43f8f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c0cd170 con 0x7f2a3c0c92d0 2023-12-19T16:43:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a43f8f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c15dfd0 con 0x7f2a3c0cc850 2023-12-19T16:43:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a43f8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c15e2b0 con 0x7f2a3c146140 2023-12-19T16:43:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a4152a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a3c0c92d0 0x7f2a3c0ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a4152a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a3c0c92d0 0x7f2a3c0ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54016/0 (socket says 172.21.15.26:54016) 2023-12-19T16:43:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a4152a640 1 -- 172.21.15.26:0/1176745930 learned_addr learned my addr 172.21.15.26:0/1176745930 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a4252c640 1 --2- 172.21.15.26:0/1176745930 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c146140 0x7f2a3c1609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a4152a640 1 -- 172.21.15.26:0/1176745930 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c146140 msgr2=0x7f2a3c1609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a41d2b640 1 --2- 172.21.15.26:0/1176745930 >> v2:172.21.15.160:3300/0 conn(0x7f2a3c0cc850 0x7f2a3c0cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a4152a640 1 --2- 172.21.15.26:0/1176745930 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c146140 0x7f2a3c1609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a4152a640 1 -- 172.21.15.26:0/1176745930 >> v2:172.21.15.160:3300/0 conn(0x7f2a3c0cc850 msgr2=0x7f2a3c0cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a4152a640 1 --2- 172.21.15.26:0/1176745930 >> v2:172.21.15.160:3300/0 conn(0x7f2a3c0cc850 0x7f2a3c0cf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a4152a640 1 -- 172.21.15.26:0/1176745930 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c0ac860 con 0x7f2a3c0c92d0 2023-12-19T16:43:08.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.719+0000 7f2a41d2b640 1 --2- 172.21.15.26:0/1176745930 >> v2:172.21.15.160:3300/0 conn(0x7f2a3c0cc850 0x7f2a3c0cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a4152a640 1 --2- 172.21.15.26:0/1176745930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a3c0c92d0 0x7f2a3c0ccc30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c069d50 tx=0x7f2a2c01c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8089c2c0a67d1c89 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a40d29640 1 -- 172.21.15.26:0/1176745930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0ab400 con 0x7f2a3c0c92d0 2023-12-19T16:43:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a40d29640 1 -- 172.21.15.26:0/1176745930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c0a7050 con 0x7f2a3c0c92d0 2023-12-19T16:43:08.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a40d29640 1 -- 172.21.15.26:0/1176745930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0ab780 con 0x7f2a3c0c92d0 2023-12-19T16:43:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1176745930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a3c0c92d0 msgr2=0x7f2a3c0ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a43f8f640 1 --2- 172.21.15.26:0/1176745930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a3c0c92d0 0x7f2a3c0ccc30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c069d50 tx=0x7f2a2c01c1b0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1176745930 shutdown_connections 2023-12-19T16:43:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a43f8f640 1 --2- 172.21.15.26:0/1176745930 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c146140 0x7f2a3c1609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a43f8f640 1 --2- 172.21.15.26:0/1176745930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a3c0c92d0 0x7f2a3c0ccc30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a43f8f640 1 --2- 172.21.15.26:0/1176745930 >> v2:172.21.15.160:3300/0 conn(0x7f2a3c0cc850 0x7f2a3c0cf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1176745930 >> 172.21.15.26:0/1176745930 conn(0x7f2a3c0cec80 msgr2=0x7f2a3c0cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1176745930 shutdown_connections 2023-12-19T16:43:08.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1176745930 wait complete. 2023-12-19T16:43:08.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a43f8f640 1 Processor -- start 2023-12-19T16:43:08.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.723+0000 7f2a43f8f640 1 -- start start 2023-12-19T16:43:08.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a43f8f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a3c0c92d0 0x7f2a3c1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a43f8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c0cc850 0x7f2a3c06c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a43f8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a3c146140 0x7f2a3c06cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a43f8f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c1602b0 con 0x7f2a3c146140 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a43f8f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c15f6a0 con 0x7f2a3c0c92d0 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a43f8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c15f980 con 0x7f2a3c0cc850 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a4252c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a3c146140 0x7f2a3c06cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a4252c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a3c146140 0x7f2a3c06cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54018/0 (socket says 172.21.15.26:54018) 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a4252c640 1 -- 172.21.15.26:0/1867254 learned_addr learned my addr 172.21.15.26:0/1867254 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a4152a640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c0cc850 0x7f2a3c06c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a41d2b640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a3c0c92d0 0x7f2a3c1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a4252c640 1 -- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c0cc850 msgr2=0x7f2a3c06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a4252c640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c0cc850 0x7f2a3c06c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a4252c640 1 -- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a3c0c92d0 msgr2=0x7f2a3c1562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a4252c640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a3c0c92d0 0x7f2a3c1562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a4252c640 1 -- 172.21.15.26:0/1867254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c0094d0 con 0x7f2a3c146140 2023-12-19T16:43:08.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a4252c640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a3c146140 0x7f2a3c06cb70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2a34007e60 tx=0x7f2a3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:08.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a327fc640 1 -- 172.21.15.26:0/1867254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34014420 con 0x7f2a3c146140 2023-12-19T16:43:08.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a327fc640 1 -- 172.21.15.26:0/1867254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a340145a0 con 0x7f2a3c146140 2023-12-19T16:43:08.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a327fc640 1 -- 172.21.15.26:0/1867254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34014950 con 0x7f2a3c146140 2023-12-19T16:43:08.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1867254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c15fc60 con 0x7f2a3c146140 2023-12-19T16:43:08.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1867254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c1567f0 con 0x7f2a3c146140 2023-12-19T16:43:08.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.727+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1867254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c160420 con 0x7f2a3c146140 2023-12-19T16:43:08.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.731+0000 7f2a327fc640 1 -- 172.21.15.26:0/1867254 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2a34014ab0 con 0x7f2a3c146140 2023-12-19T16:43:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.731+0000 7f2a327fc640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2a20065a60 0x7f2a20067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.731+0000 7f2a327fc640 1 -- 172.21.15.26:0/1867254 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2a34083a80 con 0x7f2a3c146140 2023-12-19T16:43:08.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.731+0000 7f2a41d2b640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2a20065a60 0x7f2a20067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.731+0000 7f2a41d2b640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2a20065a60 0x7f2a20067f20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2a38006b20 tx=0x7f2a38008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:08.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.735+0000 7f2a327fc640 1 -- 172.21.15.26:0/1867254 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2a340513d0 con 0x7f2a3c146140 2023-12-19T16:43:08.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.923+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1867254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a3c074270 con 0x7f2a3c146140 2023-12-19T16:43:08.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.923+0000 7f2a327fc640 1 -- 172.21.15.26:0/1867254 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f2a340549f0 con 0x7f2a3c146140 2023-12-19T16:43:08.929 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:08.929 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2850479611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2850479611},{"type":"v1","addr":"172.21.15.160:6836","nonce":2850479611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/837780144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":837780144},{"type":"v1","addr":"172.21.15.26:6834","nonce":837780144}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2784408519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2784408519},{"type":"v1","addr":"172.21.15.26:6835","nonce":2784408519}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:42:27.777197+0000","modified":"2023-12-19T16:42:30.798296+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4279":{"gid":4279,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.160:6837/3427345273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3427345273},{"type":"v1","addr":"172.21.15.160:6837","nonce":3427345273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T16:43:08.929 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 8 2023-12-19T16:43:08.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.923+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2a20065a60 msgr2=0x7f2a20067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.923+0000 7f2a43f8f640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2a20065a60 0x7f2a20067f20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2a38006b20 tx=0x7f2a38008040 comp rx=0 tx=0).stop 2023-12-19T16:43:08.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.923+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1867254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a3c146140 msgr2=0x7f2a3c06cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:08.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.923+0000 7f2a43f8f640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a3c146140 0x7f2a3c06cb70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2a34007e60 tx=0x7f2a3400e4b0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.927+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1867254 shutdown_connections 2023-12-19T16:43:08.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.927+0000 7f2a43f8f640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a3c146140 0x7f2a3c06cb70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.927+0000 7f2a43f8f640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2a20065a60 0x7f2a20067f20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.927+0000 7f2a43f8f640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a3c0cc850 0x7f2a3c06c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.927+0000 7f2a43f8f640 1 --2- 172.21.15.26:0/1867254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a3c0c92d0 0x7f2a3c1562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:08.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.927+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1867254 >> 172.21.15.26:0/1867254 conn(0x7f2a3c0cec80 msgr2=0x7f2a3c0ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:08.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.927+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1867254 shutdown_connections 2023-12-19T16:43:08.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:08.927+0000 7f2a43f8f640 1 -- 172.21.15.26:0/1867254 wait complete. 2023-12-19T16:43:08.950 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:43:09.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.087+0000 7f4ed7cfc640 1 Processor -- start 2023-12-19T16:43:09.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.087+0000 7f4ed7cfc640 1 -- start start 2023-12-19T16:43:09.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed7cfc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4ed006f9d0 0x7f4ed00ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed7cfc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ed00c4070 0x7f4ed00cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed7cfc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed00c5b40 con 0x7f4ed015a7c0 2023-12-19T16:43:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed7cfc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed00c5e20 con 0x7f4ed00c4070 2023-12-19T16:43:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed7cfc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed00ce770 con 0x7f4ed006f9d0 2023-12-19T16:43:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed6299640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ed00c4070 0x7f4ed00cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed5a98640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4ed015a7c0 0x7f4ed015ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45676/0 (socket says 172.21.15.26:45676) 2023-12-19T16:43:09.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed5a98640 1 -- 172.21.15.26:0/1561084009 learned_addr learned my addr 172.21.15.26:0/1561084009 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:09.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed5297640 1 --2- 172.21.15.26:0/1561084009 >> v2:172.21.15.160:3300/0 conn(0x7f4ed006f9d0 0x7f4ed00ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed4a96640 1 -- 172.21.15.26:0/1561084009 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948603965 0 0) 0x7f4ed00c5b40 con 0x7f4ed015a7c0 2023-12-19T16:43:09.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed4a96640 1 -- 172.21.15.26:0/1561084009 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ebc003530 con 0x7f4ed015a7c0 2023-12-19T16:43:09.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed6299640 1 -- 172.21.15.26:0/1561084009 >> v2:172.21.15.160:3300/0 conn(0x7f4ed006f9d0 msgr2=0x7f4ed00ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:09.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed6299640 1 --2- 172.21.15.26:0/1561084009 >> v2:172.21.15.160:3300/0 conn(0x7f4ed006f9d0 0x7f4ed00ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed6299640 1 -- 172.21.15.26:0/1561084009 >> v1:172.21.15.26:6789/0 conn(0x7f4ed015a7c0 legacy=0x7f4ed015ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed6299640 1 -- 172.21.15.26:0/1561084009 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed00cea50 con 0x7f4ed00c4070 2023-12-19T16:43:09.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed5297640 1 --2- 172.21.15.26:0/1561084009 >> v2:172.21.15.160:3300/0 conn(0x7f4ed006f9d0 0x7f4ed00ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:09.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.091+0000 7f4ed6299640 1 --2- 172.21.15.26:0/1561084009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ed00c4070 0x7f4ed00cf530 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4ed006ab70 tx=0x7f4ec801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384753a3446b94ca server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:09.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed4a96640 1 -- 172.21.15.26:0/1561084009 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80a59c0 con 0x7f4ed00c4070 2023-12-19T16:43:09.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed4a96640 1 -- 172.21.15.26:0/1561084009 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec80a5b20 con 0x7f4ed00c4070 2023-12-19T16:43:09.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed4a96640 1 -- 172.21.15.26:0/1561084009 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80a5e90 con 0x7f4ed00c4070 2023-12-19T16:43:09.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/1561084009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ed00c4070 msgr2=0x7f4ed00cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 --2- 172.21.15.26:0/1561084009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ed00c4070 0x7f4ed00cf530 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4ed006ab70 tx=0x7f4ec801c8e0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/1561084009 shutdown_connections 2023-12-19T16:43:09.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 --2- 172.21.15.26:0/1561084009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ed00c4070 0x7f4ed00cf530 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 --2- 172.21.15.26:0/1561084009 >> v2:172.21.15.160:3300/0 conn(0x7f4ed006f9d0 0x7f4ed00ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/1561084009 >> 172.21.15.26:0/1561084009 conn(0x7f4ed006d5d0 msgr2=0x7f4ed0069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:09.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/1561084009 shutdown_connections 2023-12-19T16:43:09.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/1561084009 wait complete. 2023-12-19T16:43:09.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 Processor -- start 2023-12-19T16:43:09.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 -- start start 2023-12-19T16:43:09.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ed006f9d0 0x7f4ed015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ed00c4070 0x7f4ed015e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ed015a7c0 0x7f4ed0152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0159820 con 0x7f4ed00c4070 2023-12-19T16:43:09.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0158f60 con 0x7f4ed006f9d0 2023-12-19T16:43:09.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed7cfc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed015ea40 con 0x7f4ed015a7c0 2023-12-19T16:43:09.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed5297640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ed00c4070 0x7f4ed015e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed5297640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ed00c4070 0x7f4ed015e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54050/0 (socket says 172.21.15.26:54050) 2023-12-19T16:43:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed5297640 1 -- 172.21.15.26:0/2705570234 learned_addr learned my addr 172.21.15.26:0/2705570234 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed6299640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ed015a7c0 0x7f4ed0152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.095+0000 7f4ed5a98640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ed006f9d0 0x7f4ed015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ed5297640 1 -- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ed015a7c0 msgr2=0x7f4ed0152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ed5297640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ed015a7c0 0x7f4ed0152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ed5297640 1 -- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ed006f9d0 msgr2=0x7f4ed015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ed5297640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ed006f9d0 0x7f4ed015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ed5297640 1 -- 172.21.15.26:0/2705570234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec8009a70 con 0x7f4ed00c4070 2023-12-19T16:43:09.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ed5297640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ed00c4070 0x7f4ed015e500 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000e840 tx=0x7f4ec000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:09.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ec67fc640 1 -- 172.21.15.26:0/2705570234 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000c8d0 con 0x7f4ed00c4070 2023-12-19T16:43:09.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ec67fc640 1 -- 172.21.15.26:0/2705570234 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec000ca30 con 0x7f4ed00c4070 2023-12-19T16:43:09.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/2705570234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed01591d0 con 0x7f4ed00c4070 2023-12-19T16:43:09.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ec67fc640 1 -- 172.21.15.26:0/2705570234 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000cde0 con 0x7f4ed00c4070 2023-12-19T16:43:09.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/2705570234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed015fa20 con 0x7f4ed00c4070 2023-12-19T16:43:09.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/2705570234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed01599d0 con 0x7f4ed00c4070 2023-12-19T16:43:09.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ec67fc640 1 -- 172.21.15.26:0/2705570234 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4ec0013410 con 0x7f4ed00c4070 2023-12-19T16:43:09.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ec67fc640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4eb4065a90 0x7f4eb4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ec67fc640 1 -- 172.21.15.26:0/2705570234 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4ec0082ff0 con 0x7f4ed00c4070 2023-12-19T16:43:09.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.099+0000 7f4ed5a98640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4eb4065a90 0x7f4eb4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.103+0000 7f4ed5a98640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4eb4065a90 0x7f4eb4067f50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc001370 tx=0x7f4ecc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:09.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.103+0000 7f4ec67fc640 1 -- 172.21.15.26:0/2705570234 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4ed01599d0 con 0x7f4ed00c4070 2023-12-19T16:43:09.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/2705570234 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ed01599d0 con 0x7f4ed00c4070 2023-12-19T16:43:09.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ec67fc640 1 -- 172.21.15.26:0/2705570234 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16286 (secure 0 0 0) 0x7f4ed01599d0 con 0x7f4ed00c4070 2023-12-19T16:43:09.268 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:09.268 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:43:08.438162+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:42:25.598550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T16:42:26.697362+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":1}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:43:09.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4eb4065a90 msgr2=0x7f4eb4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ed7cfc640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4eb4065a90 0x7f4eb4067f50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc001370 tx=0x7f4ecc009040 comp rx=0 tx=0).stop 2023-12-19T16:43:09.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/2705570234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ed00c4070 msgr2=0x7f4ed015e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ed7cfc640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ed00c4070 0x7f4ed015e500 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000e840 tx=0x7f4ec000ed00 comp rx=0 tx=0).stop 2023-12-19T16:43:09.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/2705570234 shutdown_connections 2023-12-19T16:43:09.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ed7cfc640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4eb4065a90 0x7f4eb4067f50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ed7cfc640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ed015a7c0 0x7f4ed0152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ed7cfc640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ed00c4070 0x7f4ed015e500 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ed7cfc640 1 --2- 172.21.15.26:0/2705570234 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ed006f9d0 0x7f4ed015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/2705570234 >> 172.21.15.26:0/2705570234 conn(0x7f4ed006d5d0 msgr2=0x7f4ed0153ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:09.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/2705570234 shutdown_connections 2023-12-19T16:43:09.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.263+0000 7f4ed7cfc640 1 -- 172.21.15.26:0/2705570234 wait complete. 2023-12-19T16:43:09.290 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T16:43:09.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f71123b3640 1 Processor -- start 2023-12-19T16:43:09.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f71123b3640 1 -- start start 2023-12-19T16:43:09.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f71123b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c154220 0x7f710c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f71123b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f710c06b3d0 0x7f710c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f71123b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f710c0cec40 0x7f710c15d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f71123b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f710c0cf0d0 con 0x7f710c0cec40 2023-12-19T16:43:09.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f71123b3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f710c06bc30 con 0x7f710c06b3d0 2023-12-19T16:43:09.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f71123b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710c0cc6e0 con 0x7f710c154220 2023-12-19T16:43:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f7110950640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f710c0cec40 0x7f710c15d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f7110950640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f710c0cec40 0x7f710c15d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54066/0 (socket says 172.21.15.26:54066) 2023-12-19T16:43:09.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f7110950640 1 -- 172.21.15.26:0/3811574019 learned_addr learned my addr 172.21.15.26:0/3811574019 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:09.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f710b7fe640 1 --2- 172.21.15.26:0/3811574019 >> v2:172.21.15.160:3300/0 conn(0x7f710c06b3d0 0x7f710c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f7110950640 1 -- 172.21.15.26:0/3811574019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c154220 msgr2=0x7f710c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f710bfff640 1 --2- 172.21.15.26:0/3811574019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c154220 0x7f710c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f7110950640 1 --2- 172.21.15.26:0/3811574019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c154220 0x7f710c15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f7110950640 1 -- 172.21.15.26:0/3811574019 >> v2:172.21.15.160:3300/0 conn(0x7f710c06b3d0 msgr2=0x7f710c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f7110950640 1 --2- 172.21.15.26:0/3811574019 >> v2:172.21.15.160:3300/0 conn(0x7f710c06b3d0 0x7f710c0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f7110950640 1 -- 172.21.15.26:0/3811574019 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710c15afc0 con 0x7f710c0cec40 2023-12-19T16:43:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f710b7fe640 1 --2- 172.21.15.26:0/3811574019 >> v2:172.21.15.160:3300/0 conn(0x7f710c06b3d0 0x7f710c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:09.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.431+0000 7f710bfff640 1 --2- 172.21.15.26:0/3811574019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c154220 0x7f710c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f7110950640 1 --2- 172.21.15.26:0/3811574019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0cec40 0x7f710c15d5f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f710c0cc430 tx=0x7f710401c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e583bba4c1a389 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:09.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f710affd640 1 -- 172.21.15.26:0/3811574019 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040a5560 con 0x7f710c0cec40 2023-12-19T16:43:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f710affd640 1 -- 172.21.15.26:0/3811574019 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71040a56c0 con 0x7f710c0cec40 2023-12-19T16:43:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f710affd640 1 -- 172.21.15.26:0/3811574019 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040a5a50 con 0x7f710c0cec40 2023-12-19T16:43:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f71123b3640 1 -- 172.21.15.26:0/3811574019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0cec40 msgr2=0x7f710c15d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f71123b3640 1 --2- 172.21.15.26:0/3811574019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0cec40 0x7f710c15d5f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f710c0cc430 tx=0x7f710401c6e0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f71123b3640 1 -- 172.21.15.26:0/3811574019 shutdown_connections 2023-12-19T16:43:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f71123b3640 1 --2- 172.21.15.26:0/3811574019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0cec40 0x7f710c15d5f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f71123b3640 1 --2- 172.21.15.26:0/3811574019 >> v2:172.21.15.160:3300/0 conn(0x7f710c06b3d0 0x7f710c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f71123b3640 1 --2- 172.21.15.26:0/3811574019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c154220 0x7f710c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f71123b3640 1 -- 172.21.15.26:0/3811574019 >> 172.21.15.26:0/3811574019 conn(0x7f710c0658e0 msgr2=0x7f710c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f71123b3640 1 -- 172.21.15.26:0/3811574019 shutdown_connections 2023-12-19T16:43:09.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f71123b3640 1 -- 172.21.15.26:0/3811574019 wait complete. 2023-12-19T16:43:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.435+0000 7f71123b3640 1 Processor -- start 2023-12-19T16:43:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f71123b3640 1 -- start start 2023-12-19T16:43:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f71123b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c06b3d0 0x7f710c14a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f71123b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0cec40 0x7f710c070c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f71123b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c154220 0x7f710c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f71123b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c15b340 con 0x7f710c06b3d0 2023-12-19T16:43:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f71123b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c15b620 con 0x7f710c154220 2023-12-19T16:43:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f71123b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710c15b900 con 0x7f710c0cec40 2023-12-19T16:43:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f710bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c06b3d0 0x7f710c14a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f710bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c06b3d0 0x7f710c14a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54068/0 (socket says 172.21.15.26:54068) 2023-12-19T16:43:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f710bfff640 1 -- 172.21.15.26:0/2974381160 learned_addr learned my addr 172.21.15.26:0/2974381160 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f710bfff640 1 -- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0cec40 msgr2=0x7f710c070c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f710b7fe640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0cec40 0x7f710c070c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f7110950640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c154220 0x7f710c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f710bfff640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0cec40 0x7f710c070c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f710bfff640 1 -- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c154220 msgr2=0x7f710c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f710bfff640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c154220 0x7f710c15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f710bfff640 1 -- 172.21.15.26:0/2974381160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7104009b30 con 0x7f710c06b3d0 2023-12-19T16:43:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f7110950640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c154220 0x7f710c15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f710b7fe640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0cec40 0x7f710c070c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f710bfff640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c06b3d0 0x7f710c14a4d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00c7c0 tx=0x7f70fc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f7108ff9640 1 -- 172.21.15.26:0/2974381160 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc007a30 con 0x7f710c06b3d0 2023-12-19T16:43:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f7108ff9640 1 -- 172.21.15.26:0/2974381160 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc007b90 con 0x7f710c06b3d0 2023-12-19T16:43:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f7108ff9640 1 -- 172.21.15.26:0/2974381160 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc00ce70 con 0x7f710c06b3d0 2023-12-19T16:43:09.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f71123b3640 1 -- 172.21.15.26:0/2974381160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710c15bbe0 con 0x7f710c06b3d0 2023-12-19T16:43:09.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f71123b3640 1 -- 172.21.15.26:0/2974381160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710c14aa10 con 0x7f710c06b3d0 2023-12-19T16:43:09.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.439+0000 7f71123b3640 1 -- 172.21.15.26:0/2974381160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710c074270 con 0x7f710c06b3d0 2023-12-19T16:43:09.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.443+0000 7f7108ff9640 1 -- 172.21.15.26:0/2974381160 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f70fc0113f0 con 0x7f710c06b3d0 2023-12-19T16:43:09.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.443+0000 7f7108ff9640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f70f8065a60 0x7f70f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.443+0000 7f7108ff9640 1 -- 172.21.15.26:0/2974381160 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f70fc0836f0 con 0x7f710c06b3d0 2023-12-19T16:43:09.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.443+0000 7f710b7fe640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f70f8065a60 0x7f70f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.443+0000 7f710b7fe640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f70f8065a60 0x7f70f8067f20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7100006fd0 tx=0x7f7100008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:09.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.451+0000 7f7108ff9640 1 -- 172.21.15.26:0/2974381160 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f70fc051020 con 0x7f710c06b3d0 2023-12-19T16:43:09.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.647+0000 7f71123b3640 1 -- 172.21.15.26:0/2974381160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f710c0cdf60 con 0x7f710c06b3d0 2023-12-19T16:43:09.651 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T16:43:09.647+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T16:43:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.655+0000 7f7108ff9640 1 -- 172.21.15.26:0/2974381160 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f70fc054640 con 0x7f710c06b3d0 2023-12-19T16:43:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T16:43:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.655+0000 7f71123b3640 1 -- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f70f8065a60 msgr2=0x7f70f8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.655+0000 7f71123b3640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f70f8065a60 0x7f70f8067f20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7100006fd0 tx=0x7f7100008040 comp rx=0 tx=0).stop 2023-12-19T16:43:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.655+0000 7f71123b3640 1 -- 172.21.15.26:0/2974381160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c06b3d0 msgr2=0x7f710c14a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.655+0000 7f71123b3640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c06b3d0 0x7f710c14a4d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00c7c0 tx=0x7f70fc00cc80 comp rx=0 tx=0).stop 2023-12-19T16:43:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.655+0000 7f71123b3640 1 -- 172.21.15.26:0/2974381160 shutdown_connections 2023-12-19T16:43:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.655+0000 7f71123b3640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c154220 0x7f710c15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.655+0000 7f71123b3640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f70f8065a60 0x7f70f8067f20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.655+0000 7f71123b3640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0cec40 0x7f710c070c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.655+0000 7f71123b3640 1 --2- 172.21.15.26:0/2974381160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c06b3d0 0x7f710c14a4d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.655+0000 7f71123b3640 1 -- 172.21.15.26:0/2974381160 >> 172.21.15.26:0/2974381160 conn(0x7f710c0658e0 msgr2=0x7f710c0aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.659+0000 7f71123b3640 1 -- 172.21.15.26:0/2974381160 shutdown_connections 2023-12-19T16:43:09.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.659+0000 7f71123b3640 1 -- 172.21.15.26:0/2974381160 wait complete. 2023-12-19T16:43:09.670 INFO:tasks.ceph.mds.d.smithi160.stderr: -4092> 2023-12-19T16:42:31.862+0000 7f0fa7506640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T16:43:09.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T16:43:09.721 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T16:43:09.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.823+0000 7fbb1759e640 1 Processor -- start 2023-12-19T16:43:09.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.823+0000 7fbb1759e640 1 -- start start 2023-12-19T16:43:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.823+0000 7fbb1759e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbb180ada30 0x7fbb180c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.823+0000 7fbb1759e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbb18164f00 0x7fbb180cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.823+0000 7fbb1759e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb180cb3a0 0x7fbb180affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.823+0000 7fbb1759e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb180b16e0 con 0x7fbb18164f00 2023-12-19T16:43:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.823+0000 7fbb1759e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb180ade10 con 0x7fbb180ada30 2023-12-19T16:43:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.823+0000 7fbb1759e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180cc9d0 con 0x7fbb180cb3a0 2023-12-19T16:43:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.823+0000 7fbb0ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbb18164f00 0x7fbb180cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.823+0000 7fbb0ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbb18164f00 0x7fbb180cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54070/0 (socket says 172.21.15.26:54070) 2023-12-19T16:43:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.823+0000 7fbb0ffff640 1 -- 172.21.15.26:0/1463073023 learned_addr learned my addr 172.21.15.26:0/1463073023 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:09.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.827+0000 7fbb16d9d640 1 --2- 172.21.15.26:0/1463073023 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb180cb3a0 0x7fbb180affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.827+0000 7fbb1659c640 1 --2- 172.21.15.26:0/1463073023 >> v2:172.21.15.160:3300/0 conn(0x7fbb180ada30 0x7fbb180c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.827+0000 7fbb0ffff640 1 -- 172.21.15.26:0/1463073023 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb180cb3a0 msgr2=0x7fbb180affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:09.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.827+0000 7fbb0ffff640 1 --2- 172.21.15.26:0/1463073023 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb180cb3a0 0x7fbb180affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.827+0000 7fbb0ffff640 1 -- 172.21.15.26:0/1463073023 >> v2:172.21.15.160:3300/0 conn(0x7fbb180ada30 msgr2=0x7fbb180c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:09.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.827+0000 7fbb0ffff640 1 --2- 172.21.15.26:0/1463073023 >> v2:172.21.15.160:3300/0 conn(0x7fbb180ada30 0x7fbb180c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.827+0000 7fbb0ffff640 1 -- 172.21.15.26:0/1463073023 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180cb7f0 con 0x7fbb18164f00 2023-12-19T16:43:09.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.827+0000 7fbb0ffff640 1 --2- 172.21.15.26:0/1463073023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb18164f00 0x7fbb180cae60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbb180cb980 tx=0x7fbb0001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5308a30a6beedc4e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:09.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.827+0000 7fbb15d9b640 1 -- 172.21.15.26:0/1463073023 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb000ab600 con 0x7fbb18164f00 2023-12-19T16:43:09.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.827+0000 7fbb15d9b640 1 -- 172.21.15.26:0/1463073023 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb000a7050 con 0x7fbb18164f00 2023-12-19T16:43:09.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.827+0000 7fbb15d9b640 1 -- 172.21.15.26:0/1463073023 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb000ab980 con 0x7fbb18164f00 2023-12-19T16:43:09.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.827+0000 7fbb1759e640 1 -- 172.21.15.26:0/1463073023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb18164f00 msgr2=0x7fbb180cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.827+0000 7fbb1759e640 1 --2- 172.21.15.26:0/1463073023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb18164f00 0x7fbb180cae60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbb180cb980 tx=0x7fbb0001c240 comp rx=0 tx=0).stop 2023-12-19T16:43:09.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 -- 172.21.15.26:0/1463073023 shutdown_connections 2023-12-19T16:43:09.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 --2- 172.21.15.26:0/1463073023 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb180cb3a0 0x7fbb180affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 --2- 172.21.15.26:0/1463073023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb18164f00 0x7fbb180cae60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 --2- 172.21.15.26:0/1463073023 >> v2:172.21.15.160:3300/0 conn(0x7fbb180ada30 0x7fbb180c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 -- 172.21.15.26:0/1463073023 >> 172.21.15.26:0/1463073023 conn(0x7fbb1806b0f0 msgr2=0x7fbb1806b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:09.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 -- 172.21.15.26:0/1463073023 shutdown_connections 2023-12-19T16:43:09.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 -- 172.21.15.26:0/1463073023 wait complete. 2023-12-19T16:43:09.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 Processor -- start 2023-12-19T16:43:09.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 -- start start 2023-12-19T16:43:09.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb180ada30 0x7fbb1806f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb180cb3a0 0x7fbb18156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb18164f00 0x7fbb180b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18071620 con 0x7fbb180ada30 2023-12-19T16:43:09.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180c4500 con 0x7fbb18164f00 2023-12-19T16:43:09.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1759e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180c3740 con 0x7fbb180cb3a0 2023-12-19T16:43:09.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb180ada30 0x7fbb1806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb180ada30 0x7fbb1806f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54076/0 (socket says 172.21.15.26:54076) 2023-12-19T16:43:09.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1659c640 1 -- 172.21.15.26:0/918190175 learned_addr learned my addr 172.21.15.26:0/918190175 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:09.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb0ffff640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb180cb3a0 0x7fbb18156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb16d9d640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb18164f00 0x7fbb180b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1659c640 1 -- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb180cb3a0 msgr2=0x7fbb18156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1659c640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb180cb3a0 0x7fbb18156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1659c640 1 -- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb18164f00 msgr2=0x7fbb180b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:09.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1659c640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb18164f00 0x7fbb180b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:09.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1659c640 1 -- 172.21.15.26:0/918190175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb000094d0 con 0x7fbb180ada30 2023-12-19T16:43:09.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb16d9d640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb18164f00 0x7fbb180b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:09.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.831+0000 7fbb1659c640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb180ada30 0x7fbb1806f930 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800d800 tx=0x7fbb0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:09.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.835+0000 7fbb0effd640 1 -- 172.21.15.26:0/918190175 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb080096a0 con 0x7fbb180ada30 2023-12-19T16:43:09.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.835+0000 7fbb0effd640 1 -- 172.21.15.26:0/918190175 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb08009800 con 0x7fbb180ada30 2023-12-19T16:43:09.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.835+0000 7fbb1759e640 1 -- 172.21.15.26:0/918190175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb180c3a20 con 0x7fbb180ada30 2023-12-19T16:43:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.835+0000 7fbb0effd640 1 -- 172.21.15.26:0/918190175 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08009990 con 0x7fbb180ada30 2023-12-19T16:43:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.835+0000 7fbb1759e640 1 -- 172.21.15.26:0/918190175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb180c3fe0 con 0x7fbb180ada30 2023-12-19T16:43:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.835+0000 7fbb1759e640 1 -- 172.21.15.26:0/918190175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb180b1230 con 0x7fbb180ada30 2023-12-19T16:43:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.835+0000 7fbb0effd640 1 -- 172.21.15.26:0/918190175 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbb08009af0 con 0x7fbb180ada30 2023-12-19T16:43:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.835+0000 7fbb0effd640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbaf8065a90 0x7fbaf8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.835+0000 7fbb0effd640 1 -- 172.21.15.26:0/918190175 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fbb08083720 con 0x7fbb180ada30 2023-12-19T16:43:09.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.835+0000 7fbb0ffff640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbaf8065a90 0x7fbaf8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:09.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.839+0000 7fbb0ffff640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbaf8065a90 0x7fbaf8067f50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbb000ad000 tx=0x7fbb00008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:09.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:09.839+0000 7fbb0effd640 1 -- 172.21.15.26:0/918190175 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbb08050ee0 con 0x7fbb180ada30 2023-12-19T16:43:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.031+0000 7fbb1759e640 1 -- 172.21.15.26:0/918190175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbb1804e850 con 0x7fbb180ada30 2023-12-19T16:43:10.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.655+0000 7fbb0effd640 1 -- 172.21.15.26:0/918190175 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbb08009da0 con 0x7fbb180ada30 2023-12-19T16:43:10.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.659+0000 7fbb1759e640 1 -- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbaf8065a90 msgr2=0x7fbaf8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:10.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.659+0000 7fbb1759e640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbaf8065a90 0x7fbaf8067f50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbb000ad000 tx=0x7fbb00008040 comp rx=0 tx=0).stop 2023-12-19T16:43:10.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.659+0000 7fbb1759e640 1 -- 172.21.15.26:0/918190175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb180ada30 msgr2=0x7fbb1806f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:10.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.659+0000 7fbb1759e640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb180ada30 0x7fbb1806f930 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800d800 tx=0x7fbb0800dcc0 comp rx=0 tx=0).stop 2023-12-19T16:43:10.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.659+0000 7fbb1759e640 1 -- 172.21.15.26:0/918190175 shutdown_connections 2023-12-19T16:43:10.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.659+0000 7fbb1759e640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb18164f00 0x7fbb180b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:10.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.659+0000 7fbb1759e640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbaf8065a90 0x7fbaf8067f50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:10.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.659+0000 7fbb1759e640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb180cb3a0 0x7fbb18156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:10.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.659+0000 7fbb1759e640 1 --2- 172.21.15.26:0/918190175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb180ada30 0x7fbb1806f930 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:10.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.659+0000 7fbb1759e640 1 -- 172.21.15.26:0/918190175 >> 172.21.15.26:0/918190175 conn(0x7fbb1806b0f0 msgr2=0x7fbb180b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:10.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.659+0000 7fbb1759e640 1 -- 172.21.15.26:0/918190175 shutdown_connections 2023-12-19T16:43:10.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.659+0000 7fbb1759e640 1 -- 172.21.15.26:0/918190175 wait complete. 2023-12-19T16:43:10.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T16:43:10.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.827+0000 7f1ce156d640 1 Processor -- start 2023-12-19T16:43:10.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.827+0000 7f1ce156d640 1 -- start start 2023-12-19T16:43:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.827+0000 7f1ce156d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cdc0c9310 0x7f1cdc0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.827+0000 7f1ce156d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc15e3d0 con 0x7f1cdc159830 2023-12-19T16:43:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.827+0000 7f1ce156d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc15de10 con 0x7f1cdc0cb110 2023-12-19T16:43:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.827+0000 7f1ce156d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc06f7c0 con 0x7f1cdc0c9310 2023-12-19T16:43:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.827+0000 7f1cd3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cdc0c9310 0x7f1cdc0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.827+0000 7f1cdaffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1cdc159830 0x7f1cdc15d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45688/0 (socket says 172.21.15.26:45688) 2023-12-19T16:43:10.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.827+0000 7f1cdaffd640 1 -- 172.21.15.26:0/452008484 learned_addr learned my addr 172.21.15.26:0/452008484 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:10.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.827+0000 7f1cd3fff640 1 -- 172.21.15.26:0/452008484 >> v1:172.21.15.160:6789/0 conn(0x7f1cdc0cb110 legacy=0x7f1cdc0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:10.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.827+0000 7f1cd3fff640 1 -- 172.21.15.26:0/452008484 >> v1:172.21.15.26:6789/0 conn(0x7f1cdc159830 legacy=0x7f1cdc15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:10.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.827+0000 7f1cd3fff640 1 -- 172.21.15.26:0/452008484 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc15e0f0 con 0x7f1cdc0c9310 2023-12-19T16:43:10.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.827+0000 7f1cda7fc640 1 -- 172.21.15.26:0/452008484 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612180762 0 0) 0x7f1cdc15e3d0 con 0x7f1cdc159830 2023-12-19T16:43:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.831+0000 7f1cd3fff640 1 --2- 172.21.15.26:0/452008484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0c9310 0x7f1cdc0cc430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40097c0 tx=0x7f1cc401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5141d92230bdbe server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.831+0000 7f1cda7fc640 1 -- 172.21.15.26:0/452008484 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40ab600 con 0x7f1cdc0c9310 2023-12-19T16:43:10.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.831+0000 7f1cda7fc640 1 -- 172.21.15.26:0/452008484 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc40a7050 con 0x7f1cdc0c9310 2023-12-19T16:43:10.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.831+0000 7f1cda7fc640 1 -- 172.21.15.26:0/452008484 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40ab980 con 0x7f1cdc0c9310 2023-12-19T16:43:10.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.831+0000 7f1ce156d640 1 -- 172.21.15.26:0/452008484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0c9310 msgr2=0x7f1cdc0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:10.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.831+0000 7f1ce156d640 1 --2- 172.21.15.26:0/452008484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0c9310 0x7f1cdc0cc430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40097c0 tx=0x7f1cc401c3f0 comp rx=0 tx=0).stop 2023-12-19T16:43:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.831+0000 7f1ce156d640 1 -- 172.21.15.26:0/452008484 shutdown_connections 2023-12-19T16:43:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.831+0000 7f1ce156d640 1 --2- 172.21.15.26:0/452008484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0c9310 0x7f1cdc0cc430 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.831+0000 7f1ce156d640 1 -- 172.21.15.26:0/452008484 >> 172.21.15.26:0/452008484 conn(0x7f1cdc0b0270 msgr2=0x7f1cdc0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.831+0000 7f1ce156d640 1 -- 172.21.15.26:0/452008484 shutdown_connections 2023-12-19T16:43:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.831+0000 7f1ce156d640 1 -- 172.21.15.26:0/452008484 wait complete. 2023-12-19T16:43:10.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1ce156d640 1 Processor -- start 2023-12-19T16:43:10.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1ce156d640 1 -- start start 2023-12-19T16:43:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1ce156d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0c9310 0x7f1cdc151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1ce156d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cdc0cb110 0x7f1cdc157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1ce156d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc159830 0x7f1cdc0ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1ce156d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc1522d0 con 0x7f1cdc0c9310 2023-12-19T16:43:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1ce156d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc1582a0 con 0x7f1cdc159830 2023-12-19T16:43:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1ce156d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0c2360 con 0x7f1cdc0cb110 2023-12-19T16:43:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cdaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0c9310 0x7f1cdc151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cdaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0c9310 0x7f1cdc151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54102/0 (socket says 172.21.15.26:54102) 2023-12-19T16:43:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cdaffd640 1 -- 172.21.15.26:0/783486511 learned_addr learned my addr 172.21.15.26:0/783486511 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cd3fff640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cdc0cb110 0x7f1cdc157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cdb7fe640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc159830 0x7f1cdc0ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:10.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cdaffd640 1 -- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cdc0cb110 msgr2=0x7f1cdc157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:10.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cdaffd640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cdc0cb110 0x7f1cdc157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:10.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cdaffd640 1 -- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc159830 msgr2=0x7f1cdc0ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:10.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cdaffd640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc159830 0x7f1cdc0ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:10.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cdaffd640 1 -- 172.21.15.26:0/783486511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc4009420 con 0x7f1cdc0c9310 2023-12-19T16:43:10.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cdaffd640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0c9310 0x7f1cdc151d90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00c4d0 tx=0x7f1ccc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:10.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cd37fe640 1 -- 172.21.15.26:0/783486511 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0074e0 con 0x7f1cdc0c9310 2023-12-19T16:43:10.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cd37fe640 1 -- 172.21.15.26:0/783486511 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc007640 con 0x7f1cdc0c9310 2023-12-19T16:43:10.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1ce156d640 1 -- 172.21.15.26:0/783486511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc0cbab0 con 0x7f1cdc0c9310 2023-12-19T16:43:10.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1cd37fe640 1 -- 172.21.15.26:0/783486511 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc004a30 con 0x7f1cdc0c9310 2023-12-19T16:43:10.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.835+0000 7f1ce156d640 1 -- 172.21.15.26:0/783486511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc0aec30 con 0x7f1cdc0c9310 2023-12-19T16:43:10.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.839+0000 7f1ce156d640 1 -- 172.21.15.26:0/783486511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc0ae720 con 0x7f1cdc0c9310 2023-12-19T16:43:10.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.839+0000 7f1cd37fe640 1 -- 172.21.15.26:0/783486511 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1ccc004b90 con 0x7f1cdc0c9310 2023-12-19T16:43:10.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.839+0000 7f1cd37fe640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1cbc065a90 0x7f1cbc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:10.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.839+0000 7f1cd37fe640 1 -- 172.21.15.26:0/783486511 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f1ccc0823a0 con 0x7f1cdc0c9310 2023-12-19T16:43:10.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.839+0000 7f1cd3fff640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1cbc065a90 0x7f1cbc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:10.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.839+0000 7f1cd3fff640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1cbc065a90 0x7f1cbc067f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40ad000 tx=0x7f1cc4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:10.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.843+0000 7f1cd37fe640 1 -- 172.21.15.26:0/783486511 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1ccc04fca0 con 0x7f1cdc0c9310 2023-12-19T16:43:11.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:10.999+0000 7f1ce156d640 1 -- 172.21.15.26:0/783486511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1cdc0cc8f0 con 0x7f1cdc0c9310 2023-12-19T16:43:11.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.659+0000 7f1cd37fe640 1 -- 172.21.15.26:0/783486511 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1ccc0532c0 con 0x7f1cdc0c9310 2023-12-19T16:43:11.666 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T16:43:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.663+0000 7f1ce156d640 1 -- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1cbc065a90 msgr2=0x7f1cbc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.663+0000 7f1ce156d640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1cbc065a90 0x7f1cbc067f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40ad000 tx=0x7f1cc4008040 comp rx=0 tx=0).stop 2023-12-19T16:43:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.663+0000 7f1ce156d640 1 -- 172.21.15.26:0/783486511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0c9310 msgr2=0x7f1cdc151d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.663+0000 7f1ce156d640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0c9310 0x7f1cdc151d90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00c4d0 tx=0x7f1ccc00c990 comp rx=0 tx=0).stop 2023-12-19T16:43:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.663+0000 7f1ce156d640 1 -- 172.21.15.26:0/783486511 shutdown_connections 2023-12-19T16:43:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.663+0000 7f1ce156d640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc159830 0x7f1cdc0ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.663+0000 7f1ce156d640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1cbc065a90 0x7f1cbc067f50 secure :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40ad000 tx=0x7f1cc4008040 comp rx=0 tx=0).stop 2023-12-19T16:43:11.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.663+0000 7f1ce156d640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cdc0cb110 0x7f1cdc157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.663+0000 7f1ce156d640 1 --2- 172.21.15.26:0/783486511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0c9310 0x7f1cdc151d90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.663+0000 7f1ce156d640 1 -- 172.21.15.26:0/783486511 >> 172.21.15.26:0/783486511 conn(0x7f1cdc0b0270 msgr2=0x7f1cdc0b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.667+0000 7f1ce156d640 1 -- 172.21.15.26:0/783486511 shutdown_connections 2023-12-19T16:43:11.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.667+0000 7f1ce156d640 1 -- 172.21.15.26:0/783486511 wait complete. 2023-12-19T16:43:11.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T16:43:11.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefdb6b6640 1 Processor -- start 2023-12-19T16:43:11.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefdb6b6640 1 -- start start 2023-12-19T16:43:11.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefdb6b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefd406c830 0x7fefd40cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:11.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefdb6b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd40cce50 con 0x7fefd40c1a50 2023-12-19T16:43:11.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefdb6b6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd40cc520 con 0x7fefd40ada30 2023-12-19T16:43:11.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefdb6b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd4161800 con 0x7fefd406c830 2023-12-19T16:43:11.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefd9c53640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fefd40c1a50 0x7fefd40ca570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45704/0 (socket says 172.21.15.26:45704) 2023-12-19T16:43:11.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefd9c53640 1 -- 172.21.15.26:0/1636339283 learned_addr learned my addr 172.21.15.26:0/1636339283 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:11.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefd8c51640 1 --2- 172.21.15.26:0/1636339283 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefd406c830 0x7fefd40cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:11.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefcbfff640 1 -- 172.21.15.26:0/1636339283 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085578191 0 0) 0x7fefd40cce50 con 0x7fefd40c1a50 2023-12-19T16:43:11.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefcbfff640 1 -- 172.21.15.26:0/1636339283 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefc0003530 con 0x7fefd40c1a50 2023-12-19T16:43:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefcbfff640 1 -- 172.21.15.26:0/1636339283 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510407387 0 0) 0x7fefd40cc520 con 0x7fefd40ada30 2023-12-19T16:43:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefcbfff640 1 -- 172.21.15.26:0/1636339283 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefd40cce50 con 0x7fefd40ada30 2023-12-19T16:43:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefd8c51640 1 -- 172.21.15.26:0/1636339283 >> v1:172.21.15.160:6789/0 conn(0x7fefd40ada30 legacy=0x7fefd40c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefd8c51640 1 -- 172.21.15.26:0/1636339283 >> v1:172.21.15.26:6789/0 conn(0x7fefd40c1a50 legacy=0x7fefd40ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.855+0000 7fefd8c51640 1 -- 172.21.15.26:0/1636339283 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd40cc800 con 0x7fefd406c830 2023-12-19T16:43:11.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.859+0000 7fefd8c51640 1 --2- 172.21.15.26:0/1636339283 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefd406c830 0x7fefd40cd2e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fefc40097f0 tx=0x7fefc401c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7fdb95605315ec7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:11.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.859+0000 7fefcbfff640 1 -- 172.21.15.26:0/1636339283 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc40ab600 con 0x7fefd406c830 2023-12-19T16:43:11.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.859+0000 7fefcbfff640 1 -- 172.21.15.26:0/1636339283 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc40a7050 con 0x7fefd406c830 2023-12-19T16:43:11.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.859+0000 7fefcbfff640 1 -- 172.21.15.26:0/1636339283 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc40ab980 con 0x7fefd406c830 2023-12-19T16:43:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.859+0000 7fefdb6b6640 1 -- 172.21.15.26:0/1636339283 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefd406c830 msgr2=0x7fefd40cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.859+0000 7fefdb6b6640 1 --2- 172.21.15.26:0/1636339283 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefd406c830 0x7fefd40cd2e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fefc40097f0 tx=0x7fefc401c4c0 comp rx=0 tx=0).stop 2023-12-19T16:43:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.859+0000 7fefdb6b6640 1 -- 172.21.15.26:0/1636339283 shutdown_connections 2023-12-19T16:43:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.859+0000 7fefdb6b6640 1 --2- 172.21.15.26:0/1636339283 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefd406c830 0x7fefd40cd2e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.859+0000 7fefdb6b6640 1 -- 172.21.15.26:0/1636339283 >> 172.21.15.26:0/1636339283 conn(0x7fefd40b0270 msgr2=0x7fefd40b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:11.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefdb6b6640 1 -- 172.21.15.26:0/1636339283 shutdown_connections 2023-12-19T16:43:11.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefdb6b6640 1 -- 172.21.15.26:0/1636339283 wait complete. 2023-12-19T16:43:11.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefdb6b6640 1 Processor -- start 2023-12-19T16:43:11.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefdb6b6640 1 -- start start 2023-12-19T16:43:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefdb6b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefd406c830 0x7fefd415d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefdb6b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefd40ada30 0x7fefd41580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefdb6b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fefd40c1a50 0x7fefd4152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefdb6b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd41635a0 con 0x7fefd406c830 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefdb6b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd4162920 con 0x7fefd40c1a50 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefdb6b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd4162c00 con 0x7fefd40ada30 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefd9452640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefd406c830 0x7fefd415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefd9452640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefd406c830 0x7fefd415d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54116/0 (socket says 172.21.15.26:54116) 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefd9452640 1 -- 172.21.15.26:0/3522719321 learned_addr learned my addr 172.21.15.26:0/3522719321 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefd9c53640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fefd40c1a50 0x7fefd4152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefd8c51640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefd40ada30 0x7fefd41580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefd9452640 1 -- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefd40ada30 msgr2=0x7fefd41580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefd9452640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefd40ada30 0x7fefd41580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefd9452640 1 -- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fefd40c1a50 msgr2=0x7fefd4152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefd9452640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fefd40c1a50 0x7fefd4152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefd9452640 1 -- 172.21.15.26:0/3522719321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc4009420 con 0x7fefd406c830 2023-12-19T16:43:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefd9c53640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fefd40c1a50 0x7fefd4152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefd8c51640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefd40ada30 0x7fefd41580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefd9452640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefd406c830 0x7fefd415d880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c490 tx=0x7fefd000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.863+0000 7fefc9ffb640 1 -- 172.21.15.26:0/3522719321 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd00052d0 con 0x7fefd406c830 2023-12-19T16:43:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.867+0000 7fefc9ffb640 1 -- 172.21.15.26:0/3522719321 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd0005430 con 0x7fefd406c830 2023-12-19T16:43:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.867+0000 7fefc9ffb640 1 -- 172.21.15.26:0/3522719321 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0012870 con 0x7fefd406c830 2023-12-19T16:43:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.867+0000 7fefdb6b6640 1 -- 172.21.15.26:0/3522719321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd4162ee0 con 0x7fefd406c830 2023-12-19T16:43:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.867+0000 7fefdb6b6640 1 -- 172.21.15.26:0/3522719321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd4158620 con 0x7fefd406c830 2023-12-19T16:43:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.867+0000 7fefdb6b6640 1 -- 172.21.15.26:0/3522719321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd4074270 con 0x7fefd406c830 2023-12-19T16:43:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.867+0000 7fefc9ffb640 1 -- 172.21.15.26:0/3522719321 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fefd00129d0 con 0x7fefd406c830 2023-12-19T16:43:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.867+0000 7fefc9ffb640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fefb8065a90 0x7fefb8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.867+0000 7fefc9ffb640 1 -- 172.21.15.26:0/3522719321 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5473+0+0 (secure 0 0 0) 0x7fefd0082ce0 con 0x7fefd406c830 2023-12-19T16:43:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.867+0000 7fefd8c51640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fefb8065a90 0x7fefb8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:11.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.871+0000 7fefd8c51640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fefb8065a90 0x7fefb8067f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fefc40ad000 tx=0x7fefc4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:11.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:11.871+0000 7fefc9ffb640 1 -- 172.21.15.26:0/3522719321 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fefd00506e0 con 0x7fefd406c830 2023-12-19T16:43:12.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.027+0000 7fefdb6b6640 1 -- 172.21.15.26:0/3522719321 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fefd406c130 con 0x7fefd406c830 2023-12-19T16:43:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.663+0000 7fefc9ffb640 1 -- 172.21.15.26:0/3522719321 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7fefd0053d00 con 0x7fefd406c830 2023-12-19T16:43:12.668 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T16:43:12.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.663+0000 7fefdb6b6640 1 -- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fefb8065a90 msgr2=0x7fefb8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:12.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.663+0000 7fefdb6b6640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fefb8065a90 0x7fefb8067f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fefc40ad000 tx=0x7fefc4008040 comp rx=0 tx=0).stop 2023-12-19T16:43:12.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.663+0000 7fefdb6b6640 1 -- 172.21.15.26:0/3522719321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefd406c830 msgr2=0x7fefd415d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:12.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.663+0000 7fefdb6b6640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefd406c830 0x7fefd415d880 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c490 tx=0x7fefd000c950 comp rx=0 tx=0).stop 2023-12-19T16:43:12.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.667+0000 7fefdb6b6640 1 -- 172.21.15.26:0/3522719321 shutdown_connections 2023-12-19T16:43:12.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.667+0000 7fefdb6b6640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fefd40c1a50 0x7fefd4152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:12.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.667+0000 7fefdb6b6640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fefb8065a90 0x7fefb8067f50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:12.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.667+0000 7fefdb6b6640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fefd40ada30 0x7fefd41580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:12.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.667+0000 7fefdb6b6640 1 --2- 172.21.15.26:0/3522719321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fefd406c830 0x7fefd415d880 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:12.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.667+0000 7fefdb6b6640 1 -- 172.21.15.26:0/3522719321 >> 172.21.15.26:0/3522719321 conn(0x7fefd40b0270 msgr2=0x7fefd40ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:12.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.667+0000 7fefdb6b6640 1 -- 172.21.15.26:0/3522719321 shutdown_connections 2023-12-19T16:43:12.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:12.667+0000 7fefdb6b6640 1 -- 172.21.15.26:0/3522719321 wait complete. 2023-12-19T16:43:12.698 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T16:43:12.698 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T16:43:12.698 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T16:43:12.698 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:43:12.699 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T16:43:12.699 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T16:43:12.699 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T16:43:12.699 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:43:12.699 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T16:43:12.699 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T16:43:12.699 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T16:43:12.699 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:43:12.700 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T16:43:12.700 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T16:43:12.700 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T16:43:12.700 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:43:12.713 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T16:43:12.713 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T16:43:12.720 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T16:43:12.721 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T16:43:12.722 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T16:43:12.722 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T16:43:12.757 INFO:tasks.ceph.mds.c:Started 2023-12-19T16:43:12.758 DEBUG:teuthology.parallel:result is None 2023-12-19T16:43:12.765 INFO:tasks.ceph.mds.d:Started 2023-12-19T16:43:12.765 DEBUG:teuthology.parallel:result is None 2023-12-19T16:43:12.768 INFO:tasks.ceph.mds.b:Started 2023-12-19T16:43:12.768 DEBUG:teuthology.parallel:result is None 2023-12-19T16:43:12.846 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T16:43:12.854 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T16:43:12.854 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T16:43:14.193 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-12-19T16:43:18.713 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T16:43:18.713 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T16:43:18.718 INFO:tasks.ceph.mds.a:Started 2023-12-19T16:43:18.718 DEBUG:teuthology.parallel:result is None 2023-12-19T16:43:18.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T16:43:18.814 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T16:43:18.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.871+0000 7fbfc516d640 1 Processor -- start 2023-12-19T16:43:18.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.871+0000 7fbfc516d640 1 -- start start 2023-12-19T16:43:18.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfc516d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbfc0154220 0x7fbfc015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:18.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfc516d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbfc006b3d0 0x7fbfc00ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:18.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfc516d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc006bc30 con 0x7fbfc00cec40 2023-12-19T16:43:18.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfc516d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc00cf0d0 con 0x7fbfc006b3d0 2023-12-19T16:43:18.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfc516d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00cc6e0 con 0x7fbfc0154220 2023-12-19T16:43:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfb7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbfc006b3d0 0x7fbfc00ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfb7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbfc006b3d0 0x7fbfc00ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57420/0 (socket says 172.21.15.26:57420) 2023-12-19T16:43:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfb7fff640 1 -- 172.21.15.26:0/3080259762 learned_addr learned my addr 172.21.15.26:0/3080259762 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfbed9d640 1 --2- 172.21.15.26:0/3080259762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbfc0154220 0x7fbfc015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfb7fff640 1 -- 172.21.15.26:0/3080259762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbfc0154220 msgr2=0x7fbfc015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfb7fff640 1 --2- 172.21.15.26:0/3080259762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbfc0154220 0x7fbfc015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfb7fff640 1 -- 172.21.15.26:0/3080259762 >> v1:172.21.15.160:6789/0 conn(0x7fbfc00cec40 legacy=0x7fbfc015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfb7fff640 1 -- 172.21.15.26:0/3080259762 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc015afc0 con 0x7fbfc006b3d0 2023-12-19T16:43:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfbed9d640 1 --2- 172.21.15.26:0/3080259762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbfc0154220 0x7fbfc015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:18.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfb7fff640 1 --2- 172.21.15.26:0/3080259762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc006b3d0 0x7fbfc00ce700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009870 tx=0x7fbfa801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2eeed9b9caa99e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:18.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.875+0000 7fbfbe59c640 1 -- 172.21.15.26:0/3080259762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80ac410 con 0x7fbfc006b3d0 2023-12-19T16:43:18.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfbe59c640 1 -- 172.21.15.26:0/3080259762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa80ac570 con 0x7fbfc006b3d0 2023-12-19T16:43:18.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfbe59c640 1 -- 172.21.15.26:0/3080259762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80ac900 con 0x7fbfc006b3d0 2023-12-19T16:43:18.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 -- 172.21.15.26:0/3080259762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc006b3d0 msgr2=0x7fbfc00ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:18.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 --2- 172.21.15.26:0/3080259762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc006b3d0 0x7fbfc00ce700 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009870 tx=0x7fbfa801c4d0 comp rx=0 tx=0).stop 2023-12-19T16:43:18.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 -- 172.21.15.26:0/3080259762 shutdown_connections 2023-12-19T16:43:18.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 --2- 172.21.15.26:0/3080259762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc006b3d0 0x7fbfc00ce700 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:18.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 --2- 172.21.15.26:0/3080259762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbfc0154220 0x7fbfc015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:18.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 -- 172.21.15.26:0/3080259762 >> 172.21.15.26:0/3080259762 conn(0x7fbfc00658e0 msgr2=0x7fbfc0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:18.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 -- 172.21.15.26:0/3080259762 shutdown_connections 2023-12-19T16:43:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 -- 172.21.15.26:0/3080259762 wait complete. 2023-12-19T16:43:18.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 Processor -- start 2023-12-19T16:43:18.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 -- start start 2023-12-19T16:43:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbfc006b3d0 0x7fbfc0158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc00cec40 0x7fbfc0149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfc0154220 0x7fbfc00cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc0151140 con 0x7fbfc00cec40 2023-12-19T16:43:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc0151390 con 0x7fbfc0154220 2023-12-19T16:43:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfc516d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc0151670 con 0x7fbfc006b3d0 2023-12-19T16:43:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc00cec40 0x7fbfc0149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfbed9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbfc006b3d0 0x7fbfc0158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfbed9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbfc006b3d0 0x7fbfc0158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:37020/0 (socket says 172.21.15.26:37020) 2023-12-19T16:43:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfbed9d640 1 -- 172.21.15.26:0/1333950513 learned_addr learned my addr 172.21.15.26:0/1333950513 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.879+0000 7fbfbf59e640 1 --2- 172.21.15.26:0/1333950513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfc0154220 0x7fbfc00cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.883+0000 7fbfb7fff640 1 -- 172.21.15.26:0/1333950513 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbfc006b3d0 msgr2=0x7fbfc0158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.883+0000 7fbfb7fff640 1 --2- 172.21.15.26:0/1333950513 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbfc006b3d0 0x7fbfc0158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.883+0000 7fbfb7fff640 1 -- 172.21.15.26:0/1333950513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfc0154220 msgr2=0x7fbfc00cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.883+0000 7fbfb7fff640 1 --2- 172.21.15.26:0/1333950513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfc0154220 0x7fbfc00cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.883+0000 7fbfb7fff640 1 -- 172.21.15.26:0/1333950513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa80094d0 con 0x7fbfc00cec40 2023-12-19T16:43:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.883+0000 7fbfb7fff640 1 --2- 172.21.15.26:0/1333950513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc00cec40 0x7fbfc0149ba0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80ac000 tx=0x7fbfa8019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.883+0000 7fbfb77fe640 1 -- 172.21.15.26:0/1333950513 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80bb070 con 0x7fbfc00cec40 2023-12-19T16:43:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.883+0000 7fbfb77fe640 1 -- 172.21.15.26:0/1333950513 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa80b65f0 con 0x7fbfc00cec40 2023-12-19T16:43:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.883+0000 7fbfb77fe640 1 -- 172.21.15.26:0/1333950513 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80ac410 con 0x7fbfc00cec40 2023-12-19T16:43:18.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.883+0000 7fbfc516d640 1 -- 172.21.15.26:0/1333950513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc0151950 con 0x7fbfc00cec40 2023-12-19T16:43:18.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.883+0000 7fbfc516d640 1 -- 172.21.15.26:0/1333950513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc0153a10 con 0x7fbfc00cec40 2023-12-19T16:43:18.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.883+0000 7fbfc516d640 1 -- 172.21.15.26:0/1333950513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc0067be0 con 0x7fbfc00cec40 2023-12-19T16:43:18.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.883+0000 7fbfb77fe640 1 -- 172.21.15.26:0/1333950513 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbfa80b67c0 con 0x7fbfc00cec40 2023-12-19T16:43:18.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.887+0000 7fbfb77fe640 1 --2- 172.21.15.26:0/1333950513 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbfa0065a90 0x7fbfa0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:18.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.887+0000 7fbfb77fe640 1 -- 172.21.15.26:0/1333950513 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbfa811cc00 con 0x7fbfc00cec40 2023-12-19T16:43:18.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.887+0000 7fbfbed9d640 1 --2- 172.21.15.26:0/1333950513 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbfa0065a90 0x7fbfa0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.887+0000 7fbfbed9d640 1 --2- 172.21.15.26:0/1333950513 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbfa0065a90 0x7fbfa0067f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0006fd0 tx=0x7fbfb0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:18.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:18.891+0000 7fbfb77fe640 1 -- 172.21.15.26:0/1333950513 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbfa80ea960 con 0x7fbfc00cec40 2023-12-19T16:43:19.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.059+0000 7fbfc516d640 1 -- 172.21.15.26:0/1333950513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbfc014a0e0 con 0x7fbfc00cec40 2023-12-19T16:43:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.835+0000 7fbfb77fe640 1 -- 172.21.15.26:0/1333950513 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbfa80edf80 con 0x7fbfc00cec40 2023-12-19T16:43:19.839 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2023-12-19T16:43:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.835+0000 7fbfc516d640 1 -- 172.21.15.26:0/1333950513 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbfa0065a90 msgr2=0x7fbfa0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.835+0000 7fbfc516d640 1 --2- 172.21.15.26:0/1333950513 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbfa0065a90 0x7fbfa0067f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0006fd0 tx=0x7fbfb0008040 comp rx=0 tx=0).stop 2023-12-19T16:43:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.835+0000 7fbfc516d640 1 -- 172.21.15.26:0/1333950513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc00cec40 msgr2=0x7fbfc0149ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.835+0000 7fbfc516d640 1 --2- 172.21.15.26:0/1333950513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc00cec40 0x7fbfc0149ba0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80ac000 tx=0x7fbfa8019270 comp rx=0 tx=0).stop 2023-12-19T16:43:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.835+0000 7fbfc516d640 1 -- 172.21.15.26:0/1333950513 shutdown_connections 2023-12-19T16:43:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.835+0000 7fbfc516d640 1 --2- 172.21.15.26:0/1333950513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbfc0154220 0x7fbfc00cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.835+0000 7fbfc516d640 1 --2- 172.21.15.26:0/1333950513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbfc00cec40 0x7fbfc0149ba0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.835+0000 7fbfc516d640 1 --2- 172.21.15.26:0/1333950513 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbfa0065a90 0x7fbfa0067f50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:19.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.835+0000 7fbfc516d640 1 --2- 172.21.15.26:0/1333950513 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbfc006b3d0 0x7fbfc0158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:19.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.835+0000 7fbfc516d640 1 -- 172.21.15.26:0/1333950513 >> 172.21.15.26:0/1333950513 conn(0x7fbfc00658e0 msgr2=0x7fbfc015cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:19.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.835+0000 7fbfc516d640 1 -- 172.21.15.26:0/1333950513 shutdown_connections 2023-12-19T16:43:19.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:19.835+0000 7fbfc516d640 1 -- 172.21.15.26:0/1333950513 wait complete. 2023-12-19T16:43:19.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T16:43:20.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.007+0000 7f7e07ee5640 1 Processor -- start 2023-12-19T16:43:20.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.007+0000 7f7e07ee5640 1 -- start start 2023-12-19T16:43:20.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.007+0000 7f7e07ee5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7e0006b3d0 0x7f7e000ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.007+0000 7f7e07ee5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e000cec40 0x7f7e0015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.007+0000 7f7e07ee5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0006bc30 con 0x7f7e00154220 2023-12-19T16:43:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.007+0000 7f7e07ee5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e000cf0d0 con 0x7f7e0006b3d0 2023-12-19T16:43:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.007+0000 7f7e07ee5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e000cc6e0 con 0x7f7e000cec40 2023-12-19T16:43:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e06482640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e000cec40 0x7f7e0015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e05480640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7e0006b3d0 0x7f7e000ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e06482640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e000cec40 0x7f7e0015b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:37032/0 (socket says 172.21.15.26:37032) 2023-12-19T16:43:20.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e05480640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7e0006b3d0 0x7f7e000ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:45354/0 (socket says 172.21.15.26:45354) 2023-12-19T16:43:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e06482640 1 -- 172.21.15.26:0/2949935053 learned_addr learned my addr 172.21.15.26:0/2949935053 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e05c81640 1 --1- 172.21.15.26:0/2949935053 >> v1:172.21.15.160:6789/0 conn(0x7f7e00154220 0x7f7e0015aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:35820/0 (socket says 172.21.15.26:35820) 2023-12-19T16:43:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e06482640 1 -- 172.21.15.26:0/2949935053 >> v2:172.21.15.160:3300/0 conn(0x7f7e0006b3d0 msgr2=0x7f7e000ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e06482640 1 --2- 172.21.15.26:0/2949935053 >> v2:172.21.15.160:3300/0 conn(0x7f7e0006b3d0 0x7f7e000ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e06482640 1 -- 172.21.15.26:0/2949935053 >> v1:172.21.15.160:6789/0 conn(0x7f7e00154220 legacy=0x7f7e0015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e06482640 1 -- 172.21.15.26:0/2949935053 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0006b950 con 0x7f7e000cec40 2023-12-19T16:43:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e06482640 1 --2- 172.21.15.26:0/2949935053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e000cec40 0x7f7e0015b1c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7df800d150 tx=0x7f7df801ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6a983b6eb6a76a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e04c7f640 1 -- 172.21.15.26:0/2949935053 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df800b2d0 con 0x7f7e000cec40 2023-12-19T16:43:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e04c7f640 1 -- 172.21.15.26:0/2949935053 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df800b430 con 0x7f7e000cec40 2023-12-19T16:43:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.011+0000 7f7e04c7f640 1 -- 172.21.15.26:0/2949935053 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df80a7cd0 con 0x7f7e000cec40 2023-12-19T16:43:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2949935053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e000cec40 msgr2=0x7f7e0015b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 --2- 172.21.15.26:0/2949935053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e000cec40 0x7f7e0015b1c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7df800d150 tx=0x7f7df801ebe0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2949935053 shutdown_connections 2023-12-19T16:43:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 --2- 172.21.15.26:0/2949935053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e000cec40 0x7f7e0015b1c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 --2- 172.21.15.26:0/2949935053 >> v2:172.21.15.160:3300/0 conn(0x7f7e0006b3d0 0x7f7e000ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2949935053 >> 172.21.15.26:0/2949935053 conn(0x7f7e000658e0 msgr2=0x7f7e00065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2949935053 shutdown_connections 2023-12-19T16:43:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2949935053 wait complete. 2023-12-19T16:43:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 Processor -- start 2023-12-19T16:43:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 -- start start 2023-12-19T16:43:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e0006b3d0 0x7f7e00156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e000cec40 0x7f7e00070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e00154220 0x7f7e000b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e00070aa0 con 0x7f7e00154220 2023-12-19T16:43:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e00156da0 con 0x7f7e0006b3d0 2023-12-19T16:43:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e07ee5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0006fe20 con 0x7f7e000cec40 2023-12-19T16:43:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e05c81640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e0006b3d0 0x7f7e00156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e05c81640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e0006b3d0 0x7f7e00156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:45356/0 (socket says 172.21.15.26:45356) 2023-12-19T16:43:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.015+0000 7f7e05c81640 1 -- 172.21.15.26:0/2382039447 learned_addr learned my addr 172.21.15.26:0/2382039447 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7e05480640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e000cec40 0x7f7e00070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7e06482640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e00154220 0x7f7e000b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7e05c81640 1 -- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e000cec40 msgr2=0x7f7e00070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7e05c81640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e000cec40 0x7f7e00070ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7e05c81640 1 -- 172.21.15.26:0/2382039447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e00154220 msgr2=0x7f7e000b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7e05c81640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e00154220 0x7f7e000b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7e05c81640 1 -- 172.21.15.26:0/2382039447 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df800bd40 con 0x7f7e0006b3d0 2023-12-19T16:43:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7e05480640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e000cec40 0x7f7e00070ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7e05c81640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e0006b3d0 0x7f7e00156860 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00c4c0 tx=0x7f7dfc00c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7df67fc640 1 -- 172.21.15.26:0/2382039447 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc007480 con 0x7f7e0006b3d0 2023-12-19T16:43:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7df67fc640 1 -- 172.21.15.26:0/2382039447 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc0075e0 con 0x7f7e0006b3d0 2023-12-19T16:43:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7df67fc640 1 -- 172.21.15.26:0/2382039447 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc004a10 con 0x7f7e0006b3d0 2023-12-19T16:43:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2382039447 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e00070100 con 0x7f7e0006b3d0 2023-12-19T16:43:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2382039447 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e000706c0 con 0x7f7e0006b3d0 2023-12-19T16:43:20.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2382039447 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e00070880 con 0x7f7e0006b3d0 2023-12-19T16:43:20.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.019+0000 7f7df67fc640 1 -- 172.21.15.26:0/2382039447 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7dfc004b70 con 0x7f7e0006b3d0 2023-12-19T16:43:20.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.023+0000 7f7df67fc640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7de4065a60 0x7f7de4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.023+0000 7f7df67fc640 1 -- 172.21.15.26:0/2382039447 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7dfc0822f0 con 0x7f7e0006b3d0 2023-12-19T16:43:20.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.023+0000 7f7e05480640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7de4065a60 0x7f7de4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.023+0000 7f7e05480640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7de4065a60 0x7f7de4067f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7e000ce720 tx=0x7f7df0009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.027+0000 7f7df67fc640 1 -- 172.21.15.26:0/2382039447 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7dfc050000 con 0x7f7e0006b3d0 2023-12-19T16:43:20.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2382039447 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7e00070d10 con 0x7f7e0006b3d0 2023-12-19T16:43:20.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7df67fc640 1 -- 172.21.15.26:0/2382039447 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f7dfc018020 con 0x7f7e0006b3d0 2023-12-19T16:43:20.240 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:20.241 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T16:43:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2023-12-19T16:43:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2023-12-19T16:43:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg==", 2023-12-19T16:43:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:43:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:43:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:43:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.241 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2023-12-19T16:43:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg==", 2023-12-19T16:43:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:43:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:43:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:43:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2023-12-19T16:43:20.242 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg==", 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw==", 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:43:20.243 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig==", 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA==", 2023-12-19T16:43:20.244 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ==", 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.245 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ==", 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A==", 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.246 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ==", 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.247 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg==", 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg==", 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.248 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ==", 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2023-12-19T16:43:20.249 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ==", 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ==", 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:43:20.250 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ==", 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.251 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2023-12-19T16:43:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlT5ynDxAA7Sjbdwrc1RyAXJboysB54Q==", 2023-12-19T16:43:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:43:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:43:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:43:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-mds", 2023-12-19T16:43:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDax4Flzh0LHRAA7IT1NYh8dvsegnOVZWvqpQ==", 2023-12-19T16:43:20.252 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-mds" 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-mgr", 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDax4FllycLHRAAQAgithk+KC5G11HQsBeV+g==", 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-mgr" 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-osd", 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDax4Fl3jsLHRAAr18e4g5k6YBC/L1dkFy8Nw==", 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.253 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-osd" 2023-12-19T16:43:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rbd", 2023-12-19T16:43:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDax4FlakQLHRAAoA3CL2A66P9YxAWojWwGpg==", 2023-12-19T16:43:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rbd" 2023-12-19T16:43:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.254 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDax4Fl1EwLHRAA7XuU+xZY71F/VPWt012fVQ==", 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rgw", 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDax4Flk1ULHRAAoE3gHK+/lnPOyI8kNbAfgg==", 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rgw" 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2023-12-19T16:43:20.255 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA==", 2023-12-19T16:43:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:43:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T16:43:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:43:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2023-12-19T16:43:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg==", 2023-12-19T16:43:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:43:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:43:20.256 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T16:43:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:43:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:20.257 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-12-19T16:43:20.257 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T16:43:20.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7de4065a60 msgr2=0x7f7de4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7e07ee5640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7de4065a60 0x7f7de4067f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7e000ce720 tx=0x7f7df0009190 comp rx=0 tx=0).stop 2023-12-19T16:43:20.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e0006b3d0 msgr2=0x7f7e00156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7e07ee5640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e0006b3d0 0x7f7e00156860 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00c4c0 tx=0x7f7dfc00c980 comp rx=0 tx=0).stop 2023-12-19T16:43:20.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2382039447 shutdown_connections 2023-12-19T16:43:20.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7e07ee5640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e00154220 0x7f7e000b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7e07ee5640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7de4065a60 0x7f7de4067f20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7e07ee5640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e000cec40 0x7f7e00070ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7e07ee5640 1 --2- 172.21.15.26:0/2382039447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e0006b3d0 0x7f7e00156860 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2382039447 >> 172.21.15.26:0/2382039447 conn(0x7f7e000658e0 msgr2=0x7f7e001534f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:20.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2382039447 shutdown_connections 2023-12-19T16:43:20.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.235+0000 7f7e07ee5640 1 -- 172.21.15.26:0/2382039447 wait complete. 2023-12-19T16:43:20.261 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-12-19T16:43:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe61b0e5640 1 Processor -- start 2023-12-19T16:43:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe61b0e5640 1 -- start start 2023-12-19T16:43:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe61b0e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe61406b3d0 0x7fe6140ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe61b0e5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61406bc30 con 0x7fe6140cec40 2023-12-19T16:43:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe61b0e5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6140cc6e0 con 0x7fe614154220 2023-12-19T16:43:20.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe61b0e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140cf0d0 con 0x7fe61406b3d0 2023-12-19T16:43:20.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe619682640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe6140cec40 0x7fe61415b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53780/0 (socket says 172.21.15.26:53780) 2023-12-19T16:43:20.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe619682640 1 -- 172.21.15.26:0/3837985079 learned_addr learned my addr 172.21.15.26:0/3837985079 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:20.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe613fff640 1 --2- 172.21.15.26:0/3837985079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe61406b3d0 0x7fe6140ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe6137fe640 1 -- 172.21.15.26:0/3837985079 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969162544 0 0) 0x7fe61406bc30 con 0x7fe6140cec40 2023-12-19T16:43:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe6137fe640 1 -- 172.21.15.26:0/3837985079 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe604005740 con 0x7fe6140cec40 2023-12-19T16:43:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe6137fe640 1 -- 172.21.15.26:0/3837985079 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857734482 0 0) 0x7fe6140cc6e0 con 0x7fe614154220 2023-12-19T16:43:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe6137fe640 1 -- 172.21.15.26:0/3837985079 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe61406bc30 con 0x7fe614154220 2023-12-19T16:43:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe613fff640 1 -- 172.21.15.26:0/3837985079 >> v1:172.21.15.160:6789/0 conn(0x7fe614154220 legacy=0x7fe61415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe613fff640 1 -- 172.21.15.26:0/3837985079 >> v1:172.21.15.26:6789/0 conn(0x7fe6140cec40 legacy=0x7fe61415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.403+0000 7fe613fff640 1 -- 172.21.15.26:0/3837985079 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61406b950 con 0x7fe61406b3d0 2023-12-19T16:43:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.407+0000 7fe613fff640 1 --2- 172.21.15.26:0/3837985079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe61406b3d0 0x7fe6140ce700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe6000097f0 tx=0x7fe60001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2ad6ee2a5e35d90 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.407+0000 7fe6137fe640 1 -- 172.21.15.26:0/3837985079 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6000aa5f0 con 0x7fe61406b3d0 2023-12-19T16:43:20.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.407+0000 7fe6137fe640 1 -- 172.21.15.26:0/3837985079 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6000aa750 con 0x7fe61406b3d0 2023-12-19T16:43:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.407+0000 7fe6137fe640 1 -- 172.21.15.26:0/3837985079 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6000aaac0 con 0x7fe61406b3d0 2023-12-19T16:43:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.407+0000 7fe61b0e5640 1 -- 172.21.15.26:0/3837985079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe61406b3d0 msgr2=0x7fe6140ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.407+0000 7fe61b0e5640 1 --2- 172.21.15.26:0/3837985079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe61406b3d0 0x7fe6140ce700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe6000097f0 tx=0x7fe60001c490 comp rx=0 tx=0).stop 2023-12-19T16:43:20.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.407+0000 7fe61b0e5640 1 -- 172.21.15.26:0/3837985079 shutdown_connections 2023-12-19T16:43:20.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.407+0000 7fe61b0e5640 1 --2- 172.21.15.26:0/3837985079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe61406b3d0 0x7fe6140ce700 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.407+0000 7fe61b0e5640 1 -- 172.21.15.26:0/3837985079 >> 172.21.15.26:0/3837985079 conn(0x7fe6140658e0 msgr2=0x7fe614065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:20.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.407+0000 7fe61b0e5640 1 -- 172.21.15.26:0/3837985079 shutdown_connections 2023-12-19T16:43:20.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.407+0000 7fe61b0e5640 1 -- 172.21.15.26:0/3837985079 wait complete. 2023-12-19T16:43:20.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe61b0e5640 1 Processor -- start 2023-12-19T16:43:20.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe61b0e5640 1 -- start start 2023-12-19T16:43:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe61b0e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61406b3d0 0x7fe61414de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe61b0e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6140cec40 0x7fe614151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe61b0e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe614154220 0x7fe614144410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe61b0e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe614070a30 con 0x7fe614154220 2023-12-19T16:43:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe61b0e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61414ea70 con 0x7fe61406b3d0 2023-12-19T16:43:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe61b0e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61414e790 con 0x7fe6140cec40 2023-12-19T16:43:20.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe619682640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe614154220 0x7fe614144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe619682640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe614154220 0x7fe614144410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57452/0 (socket says 172.21.15.26:57452) 2023-12-19T16:43:20.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe619682640 1 -- 172.21.15.26:0/327883127 learned_addr learned my addr 172.21.15.26:0/327883127 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:20.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe618e81640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61406b3d0 0x7fe61414de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe613fff640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6140cec40 0x7fe614151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe619682640 1 -- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6140cec40 msgr2=0x7fe614151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe619682640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6140cec40 0x7fe614151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe619682640 1 -- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61406b3d0 msgr2=0x7fe61414de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe619682640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61406b3d0 0x7fe61414de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe619682640 1 -- 172.21.15.26:0/327883127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe600009420 con 0x7fe614154220 2023-12-19T16:43:20.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe619682640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe614154220 0x7fe614144410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00cba0 tx=0x7fe60c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:20.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe6117fa640 1 -- 172.21.15.26:0/327883127 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00ace0 con 0x7fe614154220 2023-12-19T16:43:20.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe6117fa640 1 -- 172.21.15.26:0/327883127 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c00ae40 con 0x7fe614154220 2023-12-19T16:43:20.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe6117fa640 1 -- 172.21.15.26:0/327883127 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c012620 con 0x7fe614154220 2023-12-19T16:43:20.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe61b0e5640 1 -- 172.21.15.26:0/327883127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6140703e0 con 0x7fe614154220 2023-12-19T16:43:20.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.411+0000 7fe61b0e5640 1 -- 172.21.15.26:0/327883127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe614070ec0 con 0x7fe614154220 2023-12-19T16:43:20.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.415+0000 7fe61b0e5640 1 -- 172.21.15.26:0/327883127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe614070ba0 con 0x7fe614154220 2023-12-19T16:43:20.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.415+0000 7fe6117fa640 1 -- 172.21.15.26:0/327883127 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe60c007070 con 0x7fe614154220 2023-12-19T16:43:20.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.415+0000 7fe6117fa640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe5f8065a60 0x7fe5f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.415+0000 7fe6117fa640 1 -- 172.21.15.26:0/327883127 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe60c082e90 con 0x7fe614154220 2023-12-19T16:43:20.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.415+0000 7fe618e81640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe5f8065a60 0x7fe5f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.415+0000 7fe618e81640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe5f8065a60 0x7fe5f8067f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe608001720 tx=0x7fe608009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:20.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.419+0000 7fe6117fa640 1 -- 172.21.15.26:0/327883127 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe60c050bb0 con 0x7fe614154220 2023-12-19T16:43:20.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.635+0000 7fe61b0e5640 1 -- 172.21.15.26:0/327883127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fe614074270 con 0x7fe614154220 2023-12-19T16:43:20.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.639+0000 7fe6117fa640 1 -- 172.21.15.26:0/327883127 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe60c0840a0 con 0x7fe614154220 2023-12-19T16:43:20.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.639+0000 7fe61b0e5640 1 -- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe5f8065a60 msgr2=0x7fe5f8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.639+0000 7fe61b0e5640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe5f8065a60 0x7fe5f8067f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe608001720 tx=0x7fe608009040 comp rx=0 tx=0).stop 2023-12-19T16:43:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.639+0000 7fe61b0e5640 1 -- 172.21.15.26:0/327883127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe614154220 msgr2=0x7fe614144410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.639+0000 7fe61b0e5640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe614154220 0x7fe614144410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00cba0 tx=0x7fe60c00a430 comp rx=0 tx=0).stop 2023-12-19T16:43:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.639+0000 7fe61b0e5640 1 -- 172.21.15.26:0/327883127 shutdown_connections 2023-12-19T16:43:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.639+0000 7fe61b0e5640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe614154220 0x7fe614144410 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.639+0000 7fe61b0e5640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe5f8065a60 0x7fe5f8067f20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.639+0000 7fe61b0e5640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe6140cec40 0x7fe614151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.639+0000 7fe61b0e5640 1 --2- 172.21.15.26:0/327883127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61406b3d0 0x7fe61414de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.639+0000 7fe61b0e5640 1 -- 172.21.15.26:0/327883127 >> 172.21.15.26:0/327883127 conn(0x7fe6140658e0 msgr2=0x7fe6140cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:20.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.639+0000 7fe61b0e5640 1 -- 172.21.15.26:0/327883127 shutdown_connections 2023-12-19T16:43:20.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.639+0000 7fe61b0e5640 1 -- 172.21.15.26:0/327883127 wait complete. 2023-12-19T16:43:20.666 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-12-19T16:43:20.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.803+0000 7f410c9cc640 1 Processor -- start 2023-12-19T16:43:20.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.803+0000 7f410c9cc640 1 -- start start 2023-12-19T16:43:20.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f410c9cc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4108154220 0x7f410815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f410c9cc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41080cee10 0x7f410815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f410c9cc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f410806bc30 con 0x7f410806b3d0 2023-12-19T16:43:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f410c9cc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41080cc6e0 con 0x7f4108154220 2023-12-19T16:43:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f410c9cc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410815afc0 con 0x7f41080cee10 2023-12-19T16:43:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f4106d9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41080cee10 0x7f410815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f410659c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4108154220 0x7f410815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f410659c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4108154220 0x7f410815aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:45384/0 (socket says 172.21.15.26:45384) 2023-12-19T16:43:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f410659c640 1 -- 172.21.15.26:0/1457302307 learned_addr learned my addr 172.21.15.26:0/1457302307 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f4105d9b640 1 --1- 172.21.15.26:0/1457302307 >> v1:172.21.15.160:6789/0 conn(0x7f410806b3d0 0x7f41080ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:35838/0 (socket says 172.21.15.26:35838) 2023-12-19T16:43:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f410659c640 1 -- 172.21.15.26:0/1457302307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41080cee10 msgr2=0x7f410815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f410659c640 1 --2- 172.21.15.26:0/1457302307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41080cee10 0x7f410815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f410659c640 1 -- 172.21.15.26:0/1457302307 >> v1:172.21.15.160:6789/0 conn(0x7f410806b3d0 legacy=0x7f41080ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f410659c640 1 -- 172.21.15.26:0/1457302307 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410806b950 con 0x7f4108154220 2023-12-19T16:43:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f4106d9d640 1 --2- 172.21.15.26:0/1457302307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41080cee10 0x7f410815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:20.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.807+0000 7f410559a640 1 -- 172.21.15.26:0/1457302307 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156769018 0 0) 0x7f410806bc30 con 0x7f410806b3d0 2023-12-19T16:43:20.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.811+0000 7f410659c640 1 --2- 172.21.15.26:0/1457302307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4108154220 0x7f410815aa80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009870 tx=0x7f40f801c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6690aa3b2cef1781 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:20.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.811+0000 7f410559a640 1 -- 172.21.15.26:0/1457302307 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f80aa670 con 0x7f4108154220 2023-12-19T16:43:20.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.811+0000 7f410559a640 1 -- 172.21.15.26:0/1457302307 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f80aa7d0 con 0x7f4108154220 2023-12-19T16:43:20.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.811+0000 7f410559a640 1 -- 172.21.15.26:0/1457302307 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f80aab60 con 0x7f4108154220 2023-12-19T16:43:20.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.811+0000 7f410c9cc640 1 -- 172.21.15.26:0/1457302307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4108154220 msgr2=0x7f410815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.811+0000 7f410c9cc640 1 --2- 172.21.15.26:0/1457302307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4108154220 0x7f410815aa80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009870 tx=0x7f40f801c510 comp rx=0 tx=0).stop 2023-12-19T16:43:20.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.811+0000 7f410c9cc640 1 -- 172.21.15.26:0/1457302307 shutdown_connections 2023-12-19T16:43:20.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.811+0000 7f410c9cc640 1 --2- 172.21.15.26:0/1457302307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41080cee10 0x7f410815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.811+0000 7f410c9cc640 1 --2- 172.21.15.26:0/1457302307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4108154220 0x7f410815aa80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.811+0000 7f410c9cc640 1 -- 172.21.15.26:0/1457302307 >> 172.21.15.26:0/1457302307 conn(0x7f41080658e0 msgr2=0x7f4108065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:20.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.811+0000 7f410c9cc640 1 -- 172.21.15.26:0/1457302307 shutdown_connections 2023-12-19T16:43:20.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.811+0000 7f410c9cc640 1 -- 172.21.15.26:0/1457302307 wait complete. 2023-12-19T16:43:20.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410c9cc640 1 Processor -- start 2023-12-19T16:43:20.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410c9cc640 1 -- start start 2023-12-19T16:43:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410c9cc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410806b3d0 0x7f410813e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410c9cc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41080cee10 0x7f4108158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410c9cc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4108154220 0x7f4108158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410c9cc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41080b33e0 con 0x7f410806b3d0 2023-12-19T16:43:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410c9cc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410814fcc0 con 0x7f4108154220 2023-12-19T16:43:20.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410c9cc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41080b27a0 con 0x7f41080cee10 2023-12-19T16:43:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410806b3d0 0x7f410813e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410806b3d0 0x7f410813e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57464/0 (socket says 172.21.15.26:57464) 2023-12-19T16:43:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410659c640 1 -- 172.21.15.26:0/3526176049 learned_addr learned my addr 172.21.15.26:0/3526176049 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f4106d9d640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4108154220 0x7f4108158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f4105d9b640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41080cee10 0x7f4108158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410659c640 1 -- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41080cee10 msgr2=0x7f4108158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410659c640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41080cee10 0x7f4108158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410659c640 1 -- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4108154220 msgr2=0x7f4108158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410659c640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4108154220 0x7f4108158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.815+0000 7f410659c640 1 -- 172.21.15.26:0/3526176049 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4100009a70 con 0x7f410806b3d0 2023-12-19T16:43:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.819+0000 7f410659c640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410806b3d0 0x7f410813e380 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f40f801a140 tx=0x7f40f801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.819+0000 7f40f6ffd640 1 -- 172.21.15.26:0/3526176049 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f80aad50 con 0x7f410806b3d0 2023-12-19T16:43:20.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.819+0000 7f40f6ffd640 1 -- 172.21.15.26:0/3526176049 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f80b7070 con 0x7f410806b3d0 2023-12-19T16:43:20.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.819+0000 7f410c9cc640 1 -- 172.21.15.26:0/3526176049 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f80094d0 con 0x7f410806b3d0 2023-12-19T16:43:20.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.819+0000 7f40f6ffd640 1 -- 172.21.15.26:0/3526176049 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8004220 con 0x7f410806b3d0 2023-12-19T16:43:20.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.819+0000 7f410c9cc640 1 -- 172.21.15.26:0/3526176049 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41080b2d20 con 0x7f410806b3d0 2023-12-19T16:43:20.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.819+0000 7f410c9cc640 1 -- 172.21.15.26:0/3526176049 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4108074270 con 0x7f410806b3d0 2023-12-19T16:43:20.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.819+0000 7f40f6ffd640 1 -- 172.21.15.26:0/3526176049 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f40f8004380 con 0x7f410806b3d0 2023-12-19T16:43:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.819+0000 7f40f6ffd640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f40e8065a90 0x7f40e8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.819+0000 7f40f6ffd640 1 -- 172.21.15.26:0/3526176049 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f40f811d390 con 0x7f410806b3d0 2023-12-19T16:43:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.823+0000 7f4105d9b640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f40e8065a90 0x7f40e8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.823+0000 7f4105d9b640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f40e8065a90 0x7f40e8067f50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f41081f85a0 tx=0x7f40fc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:20.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:20.823+0000 7f40f6ffd640 1 -- 172.21.15.26:0/3526176049 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f40f80eb0a0 con 0x7f410806b3d0 2023-12-19T16:43:20.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-12-19T16:43:21.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.035+0000 7f410c9cc640 1 -- 172.21.15.26:0/3526176049 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f41080ae490 con 0x7f410806b3d0 2023-12-19T16:43:21.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.039+0000 7f40f6ffd640 1 -- 172.21.15.26:0/3526176049 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f40f80b90a0 con 0x7f410806b3d0 2023-12-19T16:43:21.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.043+0000 7f410c9cc640 1 -- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f40e8065a90 msgr2=0x7f40e8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.043+0000 7f410c9cc640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f40e8065a90 0x7f40e8067f50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f41081f85a0 tx=0x7f40fc009040 comp rx=0 tx=0).stop 2023-12-19T16:43:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.043+0000 7f410c9cc640 1 -- 172.21.15.26:0/3526176049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410806b3d0 msgr2=0x7f410813e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.043+0000 7f410c9cc640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410806b3d0 0x7f410813e380 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f40f801a140 tx=0x7f40f801ad50 comp rx=0 tx=0).stop 2023-12-19T16:43:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.043+0000 7f410c9cc640 1 -- 172.21.15.26:0/3526176049 shutdown_connections 2023-12-19T16:43:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.043+0000 7f410c9cc640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4108154220 0x7f4108158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.043+0000 7f410c9cc640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f40e8065a90 0x7f40e8067f50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.043+0000 7f410c9cc640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41080cee10 0x7f4108158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.043+0000 7f410c9cc640 1 --2- 172.21.15.26:0/3526176049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f410806b3d0 0x7f410813e380 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.043+0000 7f410c9cc640 1 -- 172.21.15.26:0/3526176049 >> 172.21.15.26:0/3526176049 conn(0x7f41080658e0 msgr2=0x7f41080aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.043+0000 7f410c9cc640 1 -- 172.21.15.26:0/3526176049 shutdown_connections 2023-12-19T16:43:21.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.043+0000 7f410c9cc640 1 -- 172.21.15.26:0/3526176049 wait complete. 2023-12-19T16:43:21.067 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-12-19T16:43:21.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33cc7e9640 1 Processor -- start 2023-12-19T16:43:21.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33cc7e9640 1 -- start start 2023-12-19T16:43:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33cc7e9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f33c4155030 0x7f33c415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33cc7e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33c415f370 0x7f33c406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33cc7e9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c406c970 con 0x7f33c40ce740 2023-12-19T16:43:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33cc7e9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c415b1f0 con 0x7f33c4155030 2023-12-19T16:43:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33cc7e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c415b4d0 con 0x7f33c415f370 2023-12-19T16:43:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33c9d84640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33c415f370 0x7f33c406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33ca585640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f33c4155030 0x7f33c415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33ca585640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f33c4155030 0x7f33c415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:45406/0 (socket says 172.21.15.26:45406) 2023-12-19T16:43:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33ca585640 1 -- 172.21.15.26:0/4191504032 learned_addr learned my addr 172.21.15.26:0/4191504032 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33c9583640 1 -- 172.21.15.26:0/4191504032 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171391393 0 0) 0x7f33c406c970 con 0x7f33c40ce740 2023-12-19T16:43:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33c9583640 1 -- 172.21.15.26:0/4191504032 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33b0003530 con 0x7f33c40ce740 2023-12-19T16:43:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33c9d84640 1 -- 172.21.15.26:0/4191504032 >> v2:172.21.15.160:3300/0 conn(0x7f33c4155030 msgr2=0x7f33c415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33c9d84640 1 --2- 172.21.15.26:0/4191504032 >> v2:172.21.15.160:3300/0 conn(0x7f33c4155030 0x7f33c415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33c9d84640 1 -- 172.21.15.26:0/4191504032 >> v1:172.21.15.160:6789/0 conn(0x7f33c40ce740 legacy=0x7f33c40ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33c9d84640 1 -- 172.21.15.26:0/4191504032 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c415b7b0 con 0x7f33c415f370 2023-12-19T16:43:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.207+0000 7f33ca585640 1 --2- 172.21.15.26:0/4191504032 >> v2:172.21.15.160:3300/0 conn(0x7f33c4155030 0x7f33c415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:21.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.211+0000 7f33c9d84640 1 --2- 172.21.15.26:0/4191504032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33c415f370 0x7f33c406c080 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f33b40097c0 tx=0x7f33b401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a36219306d5b318 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:21.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.211+0000 7f33c9583640 1 -- 172.21.15.26:0/4191504032 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b40a6040 con 0x7f33c415f370 2023-12-19T16:43:21.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.211+0000 7f33c9583640 1 -- 172.21.15.26:0/4191504032 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b40aa600 con 0x7f33c415f370 2023-12-19T16:43:21.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.211+0000 7f33c9583640 1 -- 172.21.15.26:0/4191504032 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b40aa970 con 0x7f33c415f370 2023-12-19T16:43:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.211+0000 7f33cc7e9640 1 -- 172.21.15.26:0/4191504032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33c415f370 msgr2=0x7f33c406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.211+0000 7f33cc7e9640 1 --2- 172.21.15.26:0/4191504032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33c415f370 0x7f33c406c080 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f33b40097c0 tx=0x7f33b401c480 comp rx=0 tx=0).stop 2023-12-19T16:43:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.211+0000 7f33cc7e9640 1 -- 172.21.15.26:0/4191504032 shutdown_connections 2023-12-19T16:43:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.211+0000 7f33cc7e9640 1 --2- 172.21.15.26:0/4191504032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33c415f370 0x7f33c406c080 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.211+0000 7f33cc7e9640 1 --2- 172.21.15.26:0/4191504032 >> v2:172.21.15.160:3300/0 conn(0x7f33c4155030 0x7f33c415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.211+0000 7f33cc7e9640 1 -- 172.21.15.26:0/4191504032 >> 172.21.15.26:0/4191504032 conn(0x7f33c40658e0 msgr2=0x7f33c4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.211+0000 7f33cc7e9640 1 -- 172.21.15.26:0/4191504032 shutdown_connections 2023-12-19T16:43:21.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.211+0000 7f33cc7e9640 1 -- 172.21.15.26:0/4191504032 wait complete. 2023-12-19T16:43:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33cc7e9640 1 Processor -- start 2023-12-19T16:43:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33cc7e9640 1 -- start start 2023-12-19T16:43:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33cc7e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33c40ce740 0x7f33c40ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33cc7e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c4155030 0x7f33c4145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33cc7e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33c415f370 0x7f33c414d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33cc7e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c4145b50 con 0x7f33c4155030 2023-12-19T16:43:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33cc7e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c4153780 con 0x7f33c415f370 2023-12-19T16:43:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33cc7e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c4153a60 con 0x7f33c40ce740 2023-12-19T16:43:21.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33c9d84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c4155030 0x7f33c4145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33c9d84640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c4155030 0x7f33c4145610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57480/0 (socket says 172.21.15.26:57480) 2023-12-19T16:43:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33c9d84640 1 -- 172.21.15.26:0/2822491235 learned_addr learned my addr 172.21.15.26:0/2822491235 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33ca585640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33c40ce740 0x7f33c40ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33cad86640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33c415f370 0x7f33c414d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33c9d84640 1 -- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33c40ce740 msgr2=0x7f33c40ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33c9d84640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33c40ce740 0x7f33c40ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33c9d84640 1 -- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33c415f370 msgr2=0x7f33c414d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33c9d84640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33c415f370 0x7f33c414d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33c9d84640 1 -- 172.21.15.26:0/2822491235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c00094d0 con 0x7f33c4155030 2023-12-19T16:43:21.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33c9d84640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c4155030 0x7f33c4145610 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f33b401a0b0 tx=0x7f33b4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:21.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33baffd640 1 -- 172.21.15.26:0/2822491235 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b40aa770 con 0x7f33c4155030 2023-12-19T16:43:21.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33baffd640 1 -- 172.21.15.26:0/2822491235 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b40b7070 con 0x7f33c4155030 2023-12-19T16:43:21.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33baffd640 1 -- 172.21.15.26:0/2822491235 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b40aaad0 con 0x7f33c4155030 2023-12-19T16:43:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33cc7e9640 1 -- 172.21.15.26:0/2822491235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b4009420 con 0x7f33c4155030 2023-12-19T16:43:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.215+0000 7f33cc7e9640 1 -- 172.21.15.26:0/2822491235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c406c740 con 0x7f33c4155030 2023-12-19T16:43:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.219+0000 7f33cc7e9640 1 -- 172.21.15.26:0/2822491235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c4074270 con 0x7f33c4155030 2023-12-19T16:43:21.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.219+0000 7f33baffd640 1 -- 172.21.15.26:0/2822491235 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f33b40aac30 con 0x7f33c4155030 2023-12-19T16:43:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.219+0000 7f33baffd640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f33a8065a60 0x7f33a8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:21.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.219+0000 7f33baffd640 1 -- 172.21.15.26:0/2822491235 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f33b411d100 con 0x7f33c4155030 2023-12-19T16:43:21.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.219+0000 7f33ca585640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f33a8065a60 0x7f33a8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:21.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.223+0000 7f33ca585640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f33a8065a60 0x7f33a8067f20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f33c40ae710 tx=0x7f33c0009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:21.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.223+0000 7f33baffd640 1 -- 172.21.15.26:0/2822491235 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f33b40eae20 con 0x7f33c4155030 2023-12-19T16:43:21.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.435+0000 7f33cc7e9640 1 -- 172.21.15.26:0/2822491235 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f33c40ccc30 con 0x7f33c4155030 2023-12-19T16:43:21.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.443+0000 7f33baffd640 1 -- 172.21.15.26:0/2822491235 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f33b40b90a0 con 0x7f33c4155030 2023-12-19T16:43:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.443+0000 7f33cc7e9640 1 -- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f33a8065a60 msgr2=0x7f33a8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.443+0000 7f33cc7e9640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f33a8065a60 0x7f33a8067f20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f33c40ae710 tx=0x7f33c0009240 comp rx=0 tx=0).stop 2023-12-19T16:43:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.443+0000 7f33cc7e9640 1 -- 172.21.15.26:0/2822491235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c4155030 msgr2=0x7f33c4145610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.443+0000 7f33cc7e9640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c4155030 0x7f33c4145610 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f33b401a0b0 tx=0x7f33b4019220 comp rx=0 tx=0).stop 2023-12-19T16:43:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.443+0000 7f33cc7e9640 1 -- 172.21.15.26:0/2822491235 shutdown_connections 2023-12-19T16:43:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.443+0000 7f33cc7e9640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33c415f370 0x7f33c414d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.443+0000 7f33cc7e9640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33c4155030 0x7f33c4145610 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.443+0000 7f33cc7e9640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f33a8065a60 0x7f33a8067f20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.443+0000 7f33cc7e9640 1 --2- 172.21.15.26:0/2822491235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33c40ce740 0x7f33c40ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.443+0000 7f33cc7e9640 1 -- 172.21.15.26:0/2822491235 >> 172.21.15.26:0/2822491235 conn(0x7f33c40658e0 msgr2=0x7f33c406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.443+0000 7f33cc7e9640 1 -- 172.21.15.26:0/2822491235 shutdown_connections 2023-12-19T16:43:21.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.443+0000 7f33cc7e9640 1 -- 172.21.15.26:0/2822491235 wait complete. 2023-12-19T16:43:21.468 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-12-19T16:43:21.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f2b266640 1 Processor -- start 2023-12-19T16:43:21.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f2b266640 1 -- start start 2023-12-19T16:43:21.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f2b266640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4f24155030 0x7f4f2415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:21.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f2b266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f240ce740 0x7f4f240cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:21.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f2b266640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2406c970 con 0x7f4f2415f370 2023-12-19T16:43:21.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f2b266640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f2406c790 con 0x7f4f24155030 2023-12-19T16:43:21.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f2b266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f240cf0f0 con 0x7f4f240ce740 2023-12-19T16:43:21.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f28801640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4f2415f370 0x7f4f2406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53786/0 (socket says 172.21.15.26:53786) 2023-12-19T16:43:21.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f28801640 1 -- 172.21.15.26:0/3672791028 learned_addr learned my addr 172.21.15.26:0/3672791028 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:21.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f29002640 1 --2- 172.21.15.26:0/3672791028 >> v2:172.21.15.26:3300/0 conn(0x7f4f24155030 0x7f4f2415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:21.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f29803640 1 --2- 172.21.15.26:0/3672791028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f240ce740 0x7f4f240cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:21.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f29002640 1 -- 172.21.15.26:0/3672791028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f240ce740 msgr2=0x7f4f240cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:21.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f29002640 1 --2- 172.21.15.26:0/3672791028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f240ce740 0x7f4f240cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f29002640 1 -- 172.21.15.26:0/3672791028 >> v1:172.21.15.26:6789/0 conn(0x7f4f2415f370 legacy=0x7f4f2406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f1bfff640 1 -- 172.21.15.26:0/3672791028 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157832927 0 0) 0x7f4f2406c970 con 0x7f4f2415f370 2023-12-19T16:43:21.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f29002640 1 -- 172.21.15.26:0/3672791028 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2415b1f0 con 0x7f4f24155030 2023-12-19T16:43:21.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.607+0000 7f4f29803640 1 --2- 172.21.15.26:0/3672791028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f240ce740 0x7f4f240cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:21.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.611+0000 7f4f29002640 1 --2- 172.21.15.26:0/3672791028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f24155030 0x7f4f2415bcd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4f200097c0 tx=0x7f4f2001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3755e24ef228f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:21.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.611+0000 7f4f1bfff640 1 -- 172.21.15.26:0/3672791028 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f200ab650 con 0x7f4f24155030 2023-12-19T16:43:21.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.611+0000 7f4f1bfff640 1 -- 172.21.15.26:0/3672791028 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f200ab7b0 con 0x7f4f24155030 2023-12-19T16:43:21.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.611+0000 7f4f1bfff640 1 -- 172.21.15.26:0/3672791028 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f200abb40 con 0x7f4f24155030 2023-12-19T16:43:21.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.611+0000 7f4f2b266640 1 -- 172.21.15.26:0/3672791028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f24155030 msgr2=0x7f4f2415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.611+0000 7f4f2b266640 1 --2- 172.21.15.26:0/3672791028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f24155030 0x7f4f2415bcd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4f200097c0 tx=0x7f4f2001c450 comp rx=0 tx=0).stop 2023-12-19T16:43:21.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.611+0000 7f4f2b266640 1 -- 172.21.15.26:0/3672791028 shutdown_connections 2023-12-19T16:43:21.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.611+0000 7f4f2b266640 1 --2- 172.21.15.26:0/3672791028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f240ce740 0x7f4f240cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.611+0000 7f4f2b266640 1 --2- 172.21.15.26:0/3672791028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f24155030 0x7f4f2415bcd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.611+0000 7f4f2b266640 1 -- 172.21.15.26:0/3672791028 >> 172.21.15.26:0/3672791028 conn(0x7f4f240658e0 msgr2=0x7f4f24065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:21.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.611+0000 7f4f2b266640 1 -- 172.21.15.26:0/3672791028 shutdown_connections 2023-12-19T16:43:21.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.611+0000 7f4f2b266640 1 -- 172.21.15.26:0/3672791028 wait complete. 2023-12-19T16:43:21.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f2b266640 1 Processor -- start 2023-12-19T16:43:21.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f2b266640 1 -- start start 2023-12-19T16:43:21.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f2b266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f240ce740 0x7f4f241592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:21.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f2b266640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f24155030 0x7f4f24152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:21.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f2b266640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2415f370 0x7f4f24153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:21.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f2b266640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f24157350 con 0x7f4f2415f370 2023-12-19T16:43:21.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f2b266640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f241599c0 con 0x7f4f24155030 2023-12-19T16:43:21.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f2b266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f24159ca0 con 0x7f4f240ce740 2023-12-19T16:43:21.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f29803640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2415f370 0x7f4f24153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:21.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f29803640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2415f370 0x7f4f24153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57496/0 (socket says 172.21.15.26:57496) 2023-12-19T16:43:21.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f29803640 1 -- 172.21.15.26:0/1049030285 learned_addr learned my addr 172.21.15.26:0/1049030285 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:21.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f29002640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f240ce740 0x7f4f241592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f28801640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f24155030 0x7f4f24152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f29803640 1 -- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f240ce740 msgr2=0x7f4f241592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f29803640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f240ce740 0x7f4f241592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f29803640 1 -- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f24155030 msgr2=0x7f4f24152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f29803640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f24155030 0x7f4f24152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f29803640 1 -- 172.21.15.26:0/1049030285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f20009420 con 0x7f4f2415f370 2023-12-19T16:43:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f28801640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f24155030 0x7f4f24152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f29803640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2415f370 0x7f4f24153380 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c007e70 tx=0x7f4f1c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f19ffb640 1 -- 172.21.15.26:0/1049030285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c003ba0 con 0x7f4f2415f370 2023-12-19T16:43:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f19ffb640 1 -- 172.21.15.26:0/1049030285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1c00f040 con 0x7f4f2415f370 2023-12-19T16:43:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f2b266640 1 -- 172.21.15.26:0/1049030285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f24159f80 con 0x7f4f2415f370 2023-12-19T16:43:21.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f19ffb640 1 -- 172.21.15.26:0/1049030285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c00de80 con 0x7f4f2415f370 2023-12-19T16:43:21.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.615+0000 7f4f2b266640 1 -- 172.21.15.26:0/1049030285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f241505a0 con 0x7f4f2415f370 2023-12-19T16:43:21.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.619+0000 7f4f2b266640 1 -- 172.21.15.26:0/1049030285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f24074270 con 0x7f4f2415f370 2023-12-19T16:43:21.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.619+0000 7f4f19ffb640 1 -- 172.21.15.26:0/1049030285 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4f1c004030 con 0x7f4f2415f370 2023-12-19T16:43:21.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.619+0000 7f4f19ffb640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f08065a90 0x7f4f08067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:21.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.619+0000 7f4f19ffb640 1 -- 172.21.15.26:0/1049030285 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4f1c083110 con 0x7f4f2415f370 2023-12-19T16:43:21.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.619+0000 7f4f29002640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f08065a90 0x7f4f08067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:21.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.619+0000 7f4f29002640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f08065a90 0x7f4f08067f50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f200af000 tx=0x7f4f200a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:21.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.623+0000 7f4f19ffb640 1 -- 172.21.15.26:0/1049030285 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4f1c050e20 con 0x7f4f2415f370 2023-12-19T16:43:21.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.835+0000 7f4f2b266640 1 -- 172.21.15.26:0/1049030285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f4f2406c240 con 0x7f4f2415f370 2023-12-19T16:43:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.839+0000 7f4f19ffb640 1 -- 172.21.15.26:0/1049030285 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4f1c054440 con 0x7f4f2415f370 2023-12-19T16:43:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.843+0000 7f4f2b266640 1 -- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f08065a90 msgr2=0x7f4f08067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.843+0000 7f4f2b266640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f08065a90 0x7f4f08067f50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f200af000 tx=0x7f4f200a7000 comp rx=0 tx=0).stop 2023-12-19T16:43:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.843+0000 7f4f2b266640 1 -- 172.21.15.26:0/1049030285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2415f370 msgr2=0x7f4f24153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.843+0000 7f4f2b266640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2415f370 0x7f4f24153380 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c007e70 tx=0x7f4f1c00d4a0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.843+0000 7f4f2b266640 1 -- 172.21.15.26:0/1049030285 shutdown_connections 2023-12-19T16:43:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.843+0000 7f4f2b266640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2415f370 0x7f4f24153380 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.843+0000 7f4f2b266640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f24155030 0x7f4f24152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.843+0000 7f4f2b266640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f08065a90 0x7f4f08067f50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.843+0000 7f4f2b266640 1 --2- 172.21.15.26:0/1049030285 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f240ce740 0x7f4f241592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.843+0000 7f4f2b266640 1 -- 172.21.15.26:0/1049030285 >> 172.21.15.26:0/1049030285 conn(0x7f4f240658e0 msgr2=0x7f4f240b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.843+0000 7f4f2b266640 1 -- 172.21.15.26:0/1049030285 shutdown_connections 2023-12-19T16:43:21.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:21.843+0000 7f4f2b266640 1 -- 172.21.15.26:0/1049030285 wait complete. 2023-12-19T16:43:21.867 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-12-19T16:43:22.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f165b9b3640 1 Processor -- start 2023-12-19T16:43:22.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f165b9b3640 1 -- start start 2023-12-19T16:43:22.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f165b9b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f165415a7c0 0x7f165415ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f165b9b3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f165415e2f0 con 0x7f16540cf700 2023-12-19T16:43:22.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f165b9b3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16540c5b40 con 0x7f165406f9d0 2023-12-19T16:43:22.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f165b9b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16540c5e20 con 0x7f165415a7c0 2023-12-19T16:43:22.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f1659f50640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f16540cf700 0x7f16540cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53800/0 (socket says 172.21.15.26:53800) 2023-12-19T16:43:22.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f1659f50640 1 -- 172.21.15.26:0/4131791467 learned_addr learned my addr 172.21.15.26:0/4131791467 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:22.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f165974f640 1 --2- 172.21.15.26:0/4131791467 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f165415a7c0 0x7f165415ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f164bfff640 1 -- 172.21.15.26:0/4131791467 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2665953671 0 0) 0x7f165415e2f0 con 0x7f16540cf700 2023-12-19T16:43:22.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f164bfff640 1 -- 172.21.15.26:0/4131791467 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1640003530 con 0x7f16540cf700 2023-12-19T16:43:22.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f164bfff640 1 -- 172.21.15.26:0/4131791467 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513332194 0 0) 0x7f16540c5b40 con 0x7f165406f9d0 2023-12-19T16:43:22.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f164bfff640 1 -- 172.21.15.26:0/4131791467 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f165415e2f0 con 0x7f165406f9d0 2023-12-19T16:43:22.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f165974f640 1 -- 172.21.15.26:0/4131791467 >> v1:172.21.15.160:6789/0 conn(0x7f165406f9d0 legacy=0x7f16540ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f165974f640 1 -- 172.21.15.26:0/4131791467 >> v1:172.21.15.26:6789/0 conn(0x7f16540cf700 legacy=0x7f16540cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.007+0000 7f165974f640 1 -- 172.21.15.26:0/4131791467 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16540ce770 con 0x7f165415a7c0 2023-12-19T16:43:22.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.011+0000 7f165974f640 1 --2- 172.21.15.26:0/4131791467 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f165415a7c0 0x7f165415ddb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f16500097d0 tx=0x7f165001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63920669817f8560 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:22.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.011+0000 7f164bfff640 1 -- 172.21.15.26:0/4131791467 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16500ab670 con 0x7f165415a7c0 2023-12-19T16:43:22.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.011+0000 7f164bfff640 1 -- 172.21.15.26:0/4131791467 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16500ab7d0 con 0x7f165415a7c0 2023-12-19T16:43:22.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.011+0000 7f164bfff640 1 -- 172.21.15.26:0/4131791467 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16500abb60 con 0x7f165415a7c0 2023-12-19T16:43:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.011+0000 7f165b9b3640 1 -- 172.21.15.26:0/4131791467 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f165415a7c0 msgr2=0x7f165415ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.011+0000 7f165b9b3640 1 --2- 172.21.15.26:0/4131791467 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f165415a7c0 0x7f165415ddb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f16500097d0 tx=0x7f165001c470 comp rx=0 tx=0).stop 2023-12-19T16:43:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.011+0000 7f165b9b3640 1 -- 172.21.15.26:0/4131791467 shutdown_connections 2023-12-19T16:43:22.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.011+0000 7f165b9b3640 1 --2- 172.21.15.26:0/4131791467 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f165415a7c0 0x7f165415ddb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.011+0000 7f165b9b3640 1 -- 172.21.15.26:0/4131791467 >> 172.21.15.26:0/4131791467 conn(0x7f165406d5d0 msgr2=0x7f1654069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:22.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.011+0000 7f165b9b3640 1 -- 172.21.15.26:0/4131791467 shutdown_connections 2023-12-19T16:43:22.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.011+0000 7f165b9b3640 1 -- 172.21.15.26:0/4131791467 wait complete. 2023-12-19T16:43:22.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f165b9b3640 1 Processor -- start 2023-12-19T16:43:22.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f165b9b3640 1 -- start start 2023-12-19T16:43:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f165b9b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f165406f9d0 0x7f16541538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f165b9b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16540cf700 0x7f16540cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f165b9b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165415a7c0 0x7f1654154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f165b9b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16540cb2b0 con 0x7f165415a7c0 2023-12-19T16:43:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f165b9b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1654154b40 con 0x7f16540cf700 2023-12-19T16:43:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f165b9b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1654153f30 con 0x7f165406f9d0 2023-12-19T16:43:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f1659f50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165415a7c0 0x7f1654154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f1659f50640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165415a7c0 0x7f1654154fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57500/0 (socket says 172.21.15.26:57500) 2023-12-19T16:43:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f1659f50640 1 -- 172.21.15.26:0/2858610337 learned_addr learned my addr 172.21.15.26:0/2858610337 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f1659f50640 1 -- 172.21.15.26:0/2858610337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f165406f9d0 msgr2=0x7f16541538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f1659f50640 1 --2- 172.21.15.26:0/2858610337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f165406f9d0 0x7f16541538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f1659f50640 1 -- 172.21.15.26:0/2858610337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16540cf700 msgr2=0x7f16540cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f1659f50640 1 --2- 172.21.15.26:0/2858610337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16540cf700 0x7f16540cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f1659f50640 1 -- 172.21.15.26:0/2858610337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16500093b0 con 0x7f165415a7c0 2023-12-19T16:43:22.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f1659f50640 1 --2- 172.21.15.26:0/2858610337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165415a7c0 0x7f1654154fd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f164c00cbc0 tx=0x7f164c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:22.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f1649ffb640 1 -- 172.21.15.26:0/2858610337 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c00aca0 con 0x7f165415a7c0 2023-12-19T16:43:22.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f1649ffb640 1 -- 172.21.15.26:0/2858610337 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164c00ae00 con 0x7f165415a7c0 2023-12-19T16:43:22.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f165b9b3640 1 -- 172.21.15.26:0/2858610337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1654154210 con 0x7f165415a7c0 2023-12-19T16:43:22.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f1649ffb640 1 -- 172.21.15.26:0/2858610337 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164c012600 con 0x7f165415a7c0 2023-12-19T16:43:22.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.015+0000 7f165b9b3640 1 -- 172.21.15.26:0/2858610337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16541547d0 con 0x7f165415a7c0 2023-12-19T16:43:22.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.019+0000 7f165b9b3640 1 -- 172.21.15.26:0/2858610337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1654074270 con 0x7f165415a7c0 2023-12-19T16:43:22.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.019+0000 7f1649ffb640 1 -- 172.21.15.26:0/2858610337 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f164c007070 con 0x7f165415a7c0 2023-12-19T16:43:22.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.019+0000 7f1649ffb640 1 --2- 172.21.15.26:0/2858610337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1638065a90 0x7f1638067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.019+0000 7f1649ffb640 1 -- 172.21.15.26:0/2858610337 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f164c082d20 con 0x7f165415a7c0 2023-12-19T16:43:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.019+0000 7f165974f640 1 --2- 172.21.15.26:0/2858610337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1638065a90 0x7f1638067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.019+0000 7f165974f640 1 --2- 172.21.15.26:0/2858610337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1638065a90 0x7f1638067f50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16500af000 tx=0x7f1650004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:22.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.023+0000 7f1649ffb640 1 -- 172.21.15.26:0/2858610337 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f164c050ad0 con 0x7f165415a7c0 2023-12-19T16:43:22.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.231+0000 7f165b9b3640 1 -- 172.21.15.26:0/2858610337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f1654154cb0 con 0x7f165415a7c0 2023-12-19T16:43:22.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.239+0000 7f1649ffb640 1 -- 172.21.15.26:0/2858610337 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f164c017020 con 0x7f165415a7c0 2023-12-19T16:43:22.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.239+0000 7f165b9b3640 1 -- 172.21.15.26:0/2858610337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1638065a90 msgr2=0x7f1638067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.239+0000 7f165b9b3640 1 --2- 172.21.15.26:0/2858610337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1638065a90 0x7f1638067f50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f16500af000 tx=0x7f1650004740 comp rx=0 tx=0).stop 2023-12-19T16:43:22.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.239+0000 7f165b9b3640 1 -- 172.21.15.26:0/2858610337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165415a7c0 msgr2=0x7f1654154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.239+0000 7f165b9b3640 1 --2- 172.21.15.26:0/2858610337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165415a7c0 0x7f1654154fd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f164c00cbc0 tx=0x7f164c00a060 comp rx=0 tx=0).stop 2023-12-19T16:43:22.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.239+0000 7f165b9b3640 1 -- 172.21.15.26:0/2858610337 shutdown_connections 2023-12-19T16:43:22.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.239+0000 7f165b9b3640 1 --2- 172.21.15.26:0/2858610337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f165415a7c0 0x7f1654154fd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.239+0000 7f165b9b3640 1 --2- 172.21.15.26:0/2858610337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f16540cf700 0x7f16540cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.239+0000 7f165b9b3640 1 --2- 172.21.15.26:0/2858610337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1638065a90 0x7f1638067f50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.239+0000 7f165b9b3640 1 --2- 172.21.15.26:0/2858610337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f165406f9d0 0x7f16541538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.239+0000 7f165b9b3640 1 -- 172.21.15.26:0/2858610337 >> 172.21.15.26:0/2858610337 conn(0x7f165406d5d0 msgr2=0x7f165415f030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:22.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.239+0000 7f165b9b3640 1 -- 172.21.15.26:0/2858610337 shutdown_connections 2023-12-19T16:43:22.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.239+0000 7f165b9b3640 1 -- 172.21.15.26:0/2858610337 wait complete. 2023-12-19T16:43:22.264 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-12-19T16:43:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.403+0000 7f4c37d60640 1 Processor -- start 2023-12-19T16:43:22.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.403+0000 7f4c37d60640 1 -- start start 2023-12-19T16:43:22.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c37d60640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c30154220 0x7f4c3015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c37d60640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c300cee10 0x7f4c3015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c37d60640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3006bc30 con 0x7f4c3006b3d0 2023-12-19T16:43:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c37d60640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c300cc6e0 con 0x7f4c30154220 2023-12-19T16:43:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c37d60640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3015afc0 con 0x7f4c300cee10 2023-12-19T16:43:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c35afc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c30154220 0x7f4c3015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c35afc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c30154220 0x7f4c3015aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57514/0 (socket says 172.21.15.26:57514) 2023-12-19T16:43:22.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c35afc640 1 -- 172.21.15.26:0/617892701 learned_addr learned my addr 172.21.15.26:0/617892701 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c35afc640 1 -- 172.21.15.26:0/617892701 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c300cee10 msgr2=0x7f4c3015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c362fd640 1 --2- 172.21.15.26:0/617892701 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c300cee10 0x7f4c3015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c35afc640 1 --2- 172.21.15.26:0/617892701 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c300cee10 0x7f4c3015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c35afc640 1 -- 172.21.15.26:0/617892701 >> v1:172.21.15.26:6789/0 conn(0x7f4c3006b3d0 legacy=0x7f4c300ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c35afc640 1 -- 172.21.15.26:0/617892701 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3006b950 con 0x7f4c30154220 2023-12-19T16:43:22.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.407+0000 7f4c362fd640 1 --2- 172.21.15.26:0/617892701 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c300cee10 0x7f4c3015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:22.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c35afc640 1 --2- 172.21.15.26:0/617892701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c30154220 0x7f4c3015aa80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009870 tx=0x7f4c2c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631dacba55f8c9ad server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:22.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c34afa640 1 -- 172.21.15.26:0/617892701 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0aa670 con 0x7f4c30154220 2023-12-19T16:43:22.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c34afa640 1 -- 172.21.15.26:0/617892701 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c0aa7d0 con 0x7f4c30154220 2023-12-19T16:43:22.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c34afa640 1 -- 172.21.15.26:0/617892701 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0aab60 con 0x7f4c30154220 2023-12-19T16:43:22.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c37d60640 1 -- 172.21.15.26:0/617892701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c30154220 msgr2=0x7f4c3015aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c37d60640 1 --2- 172.21.15.26:0/617892701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c30154220 0x7f4c3015aa80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009870 tx=0x7f4c2c01c510 comp rx=0 tx=0).stop 2023-12-19T16:43:22.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c37d60640 1 -- 172.21.15.26:0/617892701 shutdown_connections 2023-12-19T16:43:22.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c37d60640 1 --2- 172.21.15.26:0/617892701 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c300cee10 0x7f4c3015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c37d60640 1 --2- 172.21.15.26:0/617892701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c30154220 0x7f4c3015aa80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c37d60640 1 -- 172.21.15.26:0/617892701 >> 172.21.15.26:0/617892701 conn(0x7f4c300658e0 msgr2=0x7f4c30065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c37d60640 1 -- 172.21.15.26:0/617892701 shutdown_connections 2023-12-19T16:43:22.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c37d60640 1 -- 172.21.15.26:0/617892701 wait complete. 2023-12-19T16:43:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c37d60640 1 Processor -- start 2023-12-19T16:43:22.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.411+0000 7f4c37d60640 1 -- start start 2023-12-19T16:43:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c37d60640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c3006b3d0 0x7f4c30155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c37d60640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c300cee10 0x7f4c30147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c37d60640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c30154220 0x7f4c301480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c37d60640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c300b2650 con 0x7f4c3006b3d0 2023-12-19T16:43:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c37d60640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c300b19d0 con 0x7f4c300cee10 2023-12-19T16:43:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c37d60640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c300b1cb0 con 0x7f4c30154220 2023-12-19T16:43:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c35afc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c3006b3d0 0x7f4c30155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c35afc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c3006b3d0 0x7f4c30155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57522/0 (socket says 172.21.15.26:57522) 2023-12-19T16:43:22.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c35afc640 1 -- 172.21.15.26:0/3480247296 learned_addr learned my addr 172.21.15.26:0/3480247296 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c362fd640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c30154220 0x7f4c301480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c35afc640 1 -- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c30154220 msgr2=0x7f4c301480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c35afc640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c30154220 0x7f4c301480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c352fb640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c300cee10 0x7f4c30147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c35afc640 1 -- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c300cee10 msgr2=0x7f4c30147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c35afc640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c300cee10 0x7f4c30147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c35afc640 1 -- 172.21.15.26:0/3480247296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c0094d0 con 0x7f4c3006b3d0 2023-12-19T16:43:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c362fd640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c30154220 0x7f4c301480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c352fb640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c300cee10 0x7f4c30147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c35afc640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c3006b3d0 0x7f4c30155fa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0ac040 tx=0x7f4c2c01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c267fc640 1 -- 172.21.15.26:0/3480247296 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0b3800 con 0x7f4c3006b3d0 2023-12-19T16:43:22.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c267fc640 1 -- 172.21.15.26:0/3480247296 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c0b3980 con 0x7f4c3006b3d0 2023-12-19T16:43:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c267fc640 1 -- 172.21.15.26:0/3480247296 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0b3d30 con 0x7f4c3006b3d0 2023-12-19T16:43:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c37d60640 1 -- 172.21.15.26:0/3480247296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c300b1f90 con 0x7f4c3006b3d0 2023-12-19T16:43:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c37d60640 1 -- 172.21.15.26:0/3480247296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c30148ee0 con 0x7f4c3006b3d0 2023-12-19T16:43:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c37d60640 1 -- 172.21.15.26:0/3480247296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c300b2790 con 0x7f4c3006b3d0 2023-12-19T16:43:22.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.415+0000 7f4c267fc640 1 -- 172.21.15.26:0/3480247296 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4c2c0c73a0 con 0x7f4c3006b3d0 2023-12-19T16:43:22.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.419+0000 7f4c267fc640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4c14065a60 0x7f4c14067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.419+0000 7f4c267fc640 1 -- 172.21.15.26:0/3480247296 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4c2c11db10 con 0x7f4c3006b3d0 2023-12-19T16:43:22.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.419+0000 7f4c352fb640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4c14065a60 0x7f4c14067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.419+0000 7f4c352fb640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4c14065a60 0x7f4c14067f20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c300b3340 tx=0x7f4c20009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:22.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.423+0000 7f4c267fc640 1 -- 172.21.15.26:0/3480247296 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4c2c0eb960 con 0x7f4c3006b3d0 2023-12-19T16:43:22.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.635+0000 7f4c37d60640 1 -- 172.21.15.26:0/3480247296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f4c30074270 con 0x7f4c3006b3d0 2023-12-19T16:43:22.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.639+0000 7f4c267fc640 1 -- 172.21.15.26:0/3480247296 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4c2c0c30a0 con 0x7f4c3006b3d0 2023-12-19T16:43:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.643+0000 7f4c37d60640 1 -- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4c14065a60 msgr2=0x7f4c14067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.643+0000 7f4c37d60640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4c14065a60 0x7f4c14067f20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c300b3340 tx=0x7f4c20009040 comp rx=0 tx=0).stop 2023-12-19T16:43:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.643+0000 7f4c37d60640 1 -- 172.21.15.26:0/3480247296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c3006b3d0 msgr2=0x7f4c30155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.643+0000 7f4c37d60640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c3006b3d0 0x7f4c30155fa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c0ac040 tx=0x7f4c2c01ad50 comp rx=0 tx=0).stop 2023-12-19T16:43:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.643+0000 7f4c37d60640 1 -- 172.21.15.26:0/3480247296 shutdown_connections 2023-12-19T16:43:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.643+0000 7f4c37d60640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4c14065a60 0x7f4c14067f20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.643+0000 7f4c37d60640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4c30154220 0x7f4c301480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.643+0000 7f4c37d60640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c300cee10 0x7f4c30147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.643+0000 7f4c37d60640 1 --2- 172.21.15.26:0/3480247296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c3006b3d0 0x7f4c30155fa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.643+0000 7f4c37d60640 1 -- 172.21.15.26:0/3480247296 >> 172.21.15.26:0/3480247296 conn(0x7f4c300658e0 msgr2=0x7f4c300ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.643+0000 7f4c37d60640 1 -- 172.21.15.26:0/3480247296 shutdown_connections 2023-12-19T16:43:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.643+0000 7f4c37d60640 1 -- 172.21.15.26:0/3480247296 wait complete. 2023-12-19T16:43:22.666 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:43:22.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.803+0000 7f21418f2640 1 Processor -- start 2023-12-19T16:43:22.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.803+0000 7f21418f2640 1 -- start start 2023-12-19T16:43:22.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.807+0000 7f21418f2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f213c0b34e0 0x7f213c0ad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.807+0000 7f21418f2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213c0cc0d0 con 0x7f213c0cbc50 2023-12-19T16:43:22.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.807+0000 7f21418f2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213c15ddc0 con 0x7f213c06cb30 2023-12-19T16:43:22.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.807+0000 7f21418f2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f213c15e0a0 con 0x7f213c0b34e0 2023-12-19T16:43:22.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.807+0000 7f2133fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f213c0cbc50 0x7f213c0b2dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53822/0 (socket says 172.21.15.26:53822) 2023-12-19T16:43:22.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.807+0000 7f2133fff640 1 -- 172.21.15.26:0/1944114681 learned_addr learned my addr 172.21.15.26:0/1944114681 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:22.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.807+0000 7f213b7fe640 1 --2- 172.21.15.26:0/1944114681 >> v2:172.21.15.160:3300/0 conn(0x7f213c0b34e0 0x7f213c0ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.807+0000 7f213a7fc640 1 -- 172.21.15.26:0/1944114681 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665383900 0 0) 0x7f2124001120 con 0x7f213c0cbc50 2023-12-19T16:43:22.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.807+0000 7f213a7fc640 1 -- 172.21.15.26:0/1944114681 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2128003530 con 0x7f213c0cbc50 2023-12-19T16:43:22.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.807+0000 7f213b7fe640 1 -- 172.21.15.26:0/1944114681 >> v1:172.21.15.160:6789/0 conn(0x7f213c06cb30 legacy=0x7f213c15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.807+0000 7f213a7fc640 1 -- 172.21.15.26:0/1944114681 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153616297 0 0) 0x7f213c15ddc0 con 0x7f213c06cb30 2023-12-19T16:43:22.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.807+0000 7f213b7fe640 1 -- 172.21.15.26:0/1944114681 >> v1:172.21.15.26:6789/0 conn(0x7f213c0cbc50 legacy=0x7f213c0b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.807+0000 7f213b7fe640 1 -- 172.21.15.26:0/1944114681 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c0cb450 con 0x7f213c0b34e0 2023-12-19T16:43:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.811+0000 7f213b7fe640 1 --2- 172.21.15.26:0/1944114681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f213c0b34e0 0x7f213c0ad180 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2134009d90 tx=0x7f213401c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6892241b02f19a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.811+0000 7f213a7fc640 1 -- 172.21.15.26:0/1944114681 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c0aded0 con 0x7f213c0b34e0 2023-12-19T16:43:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.811+0000 7f213a7fc640 1 -- 172.21.15.26:0/1944114681 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21340a58a0 con 0x7f213c0b34e0 2023-12-19T16:43:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.811+0000 7f213a7fc640 1 -- 172.21.15.26:0/1944114681 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21340a5c30 con 0x7f213c0b34e0 2023-12-19T16:43:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.811+0000 7f21418f2640 1 -- 172.21.15.26:0/1944114681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f213c0b34e0 msgr2=0x7f213c0ad180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.811+0000 7f21418f2640 1 --2- 172.21.15.26:0/1944114681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f213c0b34e0 0x7f213c0ad180 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2134009d90 tx=0x7f213401c840 comp rx=0 tx=0).stop 2023-12-19T16:43:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.811+0000 7f21418f2640 1 -- 172.21.15.26:0/1944114681 shutdown_connections 2023-12-19T16:43:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.811+0000 7f21418f2640 1 --2- 172.21.15.26:0/1944114681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f213c0b34e0 0x7f213c0ad180 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.811+0000 7f21418f2640 1 -- 172.21.15.26:0/1944114681 >> 172.21.15.26:0/1944114681 conn(0x7f213c0658e0 msgr2=0x7f213c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.811+0000 7f21418f2640 1 -- 172.21.15.26:0/1944114681 shutdown_connections 2023-12-19T16:43:22.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.811+0000 7f21418f2640 1 -- 172.21.15.26:0/1944114681 wait complete. 2023-12-19T16:43:22.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.811+0000 7f21418f2640 1 Processor -- start 2023-12-19T16:43:22.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.811+0000 7f21418f2640 1 -- start start 2023-12-19T16:43:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f21418f2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f213c06cb30 0x7f213c15c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f21418f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c0b34e0 0x7f213c12b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f21418f2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f213c0cbc50 0x7f213c12bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f21418f2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c129a40 con 0x7f213c0b34e0 2023-12-19T16:43:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f21418f2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c15ccf0 con 0x7f213c06cb30 2023-12-19T16:43:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f21418f2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213c1402e0 con 0x7f213c0cbc50 2023-12-19T16:43:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f2133fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c0b34e0 0x7f213c12b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f2133fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c0b34e0 0x7f213c12b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57524/0 (socket says 172.21.15.26:57524) 2023-12-19T16:43:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f2133fff640 1 -- 172.21.15.26:0/2318079685 learned_addr learned my addr 172.21.15.26:0/2318079685 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f213b7fe640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f213c0cbc50 0x7f213c12bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f213affd640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f213c06cb30 0x7f213c15c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f2133fff640 1 -- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f213c0cbc50 msgr2=0x7f213c12bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f2133fff640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f213c0cbc50 0x7f213c12bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f2133fff640 1 -- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f213c06cb30 msgr2=0x7f213c15c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f2133fff640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f213c06cb30 0x7f213c15c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f2133fff640 1 -- 172.21.15.26:0/2318079685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2134009a70 con 0x7f213c0b34e0 2023-12-19T16:43:22.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f213b7fe640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f213c0cbc50 0x7f213c12bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f213affd640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f213c06cb30 0x7f213c15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f2133fff640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c0b34e0 0x7f213c12b310 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f212400c780 tx=0x7f212400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f2138ff9640 1 -- 172.21.15.26:0/2318079685 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124005ac0 con 0x7f213c0b34e0 2023-12-19T16:43:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f2138ff9640 1 -- 172.21.15.26:0/2318079685 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2124005c20 con 0x7f213c0b34e0 2023-12-19T16:43:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f2138ff9640 1 -- 172.21.15.26:0/2318079685 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124007070 con 0x7f213c0b34e0 2023-12-19T16:43:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f21418f2640 1 -- 172.21.15.26:0/2318079685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213c128dc0 con 0x7f213c0b34e0 2023-12-19T16:43:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f21418f2640 1 -- 172.21.15.26:0/2318079685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c129380 con 0x7f213c0b34e0 2023-12-19T16:43:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.815+0000 7f21418f2640 1 -- 172.21.15.26:0/2318079685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213c0cef90 con 0x7f213c0b34e0 2023-12-19T16:43:22.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.819+0000 7f2138ff9640 1 -- 172.21.15.26:0/2318079685 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2124007230 con 0x7f213c0b34e0 2023-12-19T16:43:22.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.819+0000 7f2138ff9640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f211c065a60 0x7f211c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:22.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.819+0000 7f2138ff9640 1 -- 172.21.15.26:0/2318079685 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2124082510 con 0x7f213c0b34e0 2023-12-19T16:43:22.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.819+0000 7f213affd640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f211c065a60 0x7f211c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:22.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.819+0000 7f213affd640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f211c065a60 0x7f211c067f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f212c000f70 tx=0x7f212c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:22.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.823+0000 7f2138ff9640 1 -- 172.21.15.26:0/2318079685 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2124050210 con 0x7f213c0b34e0 2023-12-19T16:43:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.979+0000 7f21418f2640 1 -- 172.21.15.26:0/2318079685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f213c129b80 con 0x7f213c0b34e0 2023-12-19T16:43:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.979+0000 7f2138ff9640 1 -- 172.21.15.26:0/2318079685 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+12389 (secure 0 0 0) 0x7f213c129b80 con 0x7f213c0b34e0 2023-12-19T16:43:22.985 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:22.985 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:43:19.831954+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:43:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.983+0000 7f21418f2640 1 -- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f211c065a60 msgr2=0x7f211c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.983+0000 7f21418f2640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f211c065a60 0x7f211c067f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f212c000f70 tx=0x7f212c009040 comp rx=0 tx=0).stop 2023-12-19T16:43:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.983+0000 7f21418f2640 1 -- 172.21.15.26:0/2318079685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c0b34e0 msgr2=0x7f213c12b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.983+0000 7f21418f2640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c0b34e0 0x7f213c12b310 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f212400c780 tx=0x7f212400cc40 comp rx=0 tx=0).stop 2023-12-19T16:43:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.983+0000 7f21418f2640 1 -- 172.21.15.26:0/2318079685 shutdown_connections 2023-12-19T16:43:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.983+0000 7f21418f2640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f211c065a60 0x7f211c067f20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.983+0000 7f21418f2640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f213c0cbc50 0x7f213c12bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.983+0000 7f21418f2640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c0b34e0 0x7f213c12b310 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.983+0000 7f21418f2640 1 --2- 172.21.15.26:0/2318079685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f213c06cb30 0x7f213c15c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.983+0000 7f21418f2640 1 -- 172.21.15.26:0/2318079685 >> 172.21.15.26:0/2318079685 conn(0x7f213c0658e0 msgr2=0x7f213c160cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:22.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.983+0000 7f21418f2640 1 -- 172.21.15.26:0/2318079685 shutdown_connections 2023-12-19T16:43:22.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:22.983+0000 7f21418f2640 1 -- 172.21.15.26:0/2318079685 wait complete. 2023-12-19T16:43:23.006 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:43:23.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:43:23.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.147+0000 7fb02336d640 1 Processor -- start 2023-12-19T16:43:23.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.147+0000 7fb02336d640 1 -- start start 2023-12-19T16:43:23.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.147+0000 7fb02336d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb01c06f6c0 0x7fb01c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.147+0000 7fb02336d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c06fb60 con 0x7fb01c159c00 2023-12-19T16:43:23.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.147+0000 7fb02336d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c06fe40 con 0x7fb01c159820 2023-12-19T16:43:23.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.147+0000 7fb02336d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01c15e470 con 0x7fb01c06f6c0 2023-12-19T16:43:23.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.147+0000 7fb02190a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb01c159c00 0x7fb01c070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53826/0 (socket says 172.21.15.26:53826) 2023-12-19T16:43:23.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.147+0000 7fb02190a640 1 -- 172.21.15.26:0/12551423 learned_addr learned my addr 172.21.15.26:0/12551423 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:23.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.147+0000 7fb020908640 1 --2- 172.21.15.26:0/12551423 >> v2:172.21.15.160:3300/0 conn(0x7fb01c06f6c0 0x7fb01c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb013fff640 1 -- 172.21.15.26:0/12551423 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635157936 0 0) 0x7fb01c06fb60 con 0x7fb01c159c00 2023-12-19T16:43:23.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb013fff640 1 -- 172.21.15.26:0/12551423 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb008005740 con 0x7fb01c159c00 2023-12-19T16:43:23.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb020908640 1 -- 172.21.15.26:0/12551423 >> v1:172.21.15.160:6789/0 conn(0x7fb01c159820 legacy=0x7fb01c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb013fff640 1 -- 172.21.15.26:0/12551423 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5712665 0 0) 0x7fb01c06fe40 con 0x7fb01c159820 2023-12-19T16:43:23.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb020908640 1 -- 172.21.15.26:0/12551423 >> v1:172.21.15.26:6789/0 conn(0x7fb01c159c00 legacy=0x7fb01c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb020908640 1 -- 172.21.15.26:0/12551423 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c070120 con 0x7fb01c06f6c0 2023-12-19T16:43:23.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb020908640 1 --2- 172.21.15.26:0/12551423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb01c06f6c0 0x7fb01c15df30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0097c0 tx=0x7fb00c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3b3a3f3ded6060d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:23.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb013fff640 1 -- 172.21.15.26:0/12551423 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0ab650 con 0x7fb01c06f6c0 2023-12-19T16:43:23.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb013fff640 1 -- 172.21.15.26:0/12551423 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c0ab7b0 con 0x7fb01c06f6c0 2023-12-19T16:43:23.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb013fff640 1 -- 172.21.15.26:0/12551423 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0abb40 con 0x7fb01c06f6c0 2023-12-19T16:43:23.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb02336d640 1 -- 172.21.15.26:0/12551423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb01c06f6c0 msgr2=0x7fb01c15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb02336d640 1 --2- 172.21.15.26:0/12551423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb01c06f6c0 0x7fb01c15df30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0097c0 tx=0x7fb00c01c450 comp rx=0 tx=0).stop 2023-12-19T16:43:23.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb02336d640 1 -- 172.21.15.26:0/12551423 shutdown_connections 2023-12-19T16:43:23.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.151+0000 7fb02336d640 1 --2- 172.21.15.26:0/12551423 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb01c06f6c0 0x7fb01c15df30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb02336d640 1 -- 172.21.15.26:0/12551423 >> 172.21.15.26:0/12551423 conn(0x7fb01c0ce730 msgr2=0x7fb01c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:23.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb02336d640 1 -- 172.21.15.26:0/12551423 shutdown_connections 2023-12-19T16:43:23.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb02336d640 1 -- 172.21.15.26:0/12551423 wait complete. 2023-12-19T16:43:23.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb02336d640 1 Processor -- start 2023-12-19T16:43:23.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb02336d640 1 -- start start 2023-12-19T16:43:23.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb02336d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb01c06f6c0 0x7fb01c15c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb02336d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01c159820 0x7fb01c0c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb02336d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb01c159c00 0x7fb01c0ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb02336d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0c3bd0 con 0x7fb01c159820 2023-12-19T16:43:23.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb02336d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0c2fc0 con 0x7fb01c159c00 2023-12-19T16:43:23.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb02336d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0c32a0 con 0x7fb01c06f6c0 2023-12-19T16:43:23.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb020908640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01c159820 0x7fb01c0c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb020908640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01c159820 0x7fb01c0c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57526/0 (socket says 172.21.15.26:57526) 2023-12-19T16:43:23.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb020908640 1 -- 172.21.15.26:0/572091073 learned_addr learned my addr 172.21.15.26:0/572091073 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:23.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb02190a640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb01c159c00 0x7fb01c0ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb021109640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb01c06f6c0 0x7fb01c15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb020908640 1 -- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb01c06f6c0 msgr2=0x7fb01c15c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb020908640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb01c06f6c0 0x7fb01c15c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb020908640 1 -- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb01c159c00 msgr2=0x7fb01c0ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb020908640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb01c159c00 0x7fb01c0ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb020908640 1 -- 172.21.15.26:0/572091073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c009420 con 0x7fb01c159820 2023-12-19T16:43:23.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb02190a640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb01c159c00 0x7fb01c0ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:23.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.155+0000 7fb020908640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01c159820 0x7fb01c0c5770 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0ad040 tx=0x7fb00c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:23.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.159+0000 7fb011ffb640 1 -- 172.21.15.26:0/572091073 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0bf610 con 0x7fb01c159820 2023-12-19T16:43:23.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.159+0000 7fb011ffb640 1 -- 172.21.15.26:0/572091073 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c0bf770 con 0x7fb01c159820 2023-12-19T16:43:23.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.159+0000 7fb011ffb640 1 -- 172.21.15.26:0/572091073 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0bfb20 con 0x7fb01c159820 2023-12-19T16:43:23.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.159+0000 7fb02336d640 1 -- 172.21.15.26:0/572091073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c06b880 con 0x7fb01c159820 2023-12-19T16:43:23.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.159+0000 7fb02336d640 1 -- 172.21.15.26:0/572091073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c0c3860 con 0x7fb01c159820 2023-12-19T16:43:23.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.159+0000 7fb02336d640 1 -- 172.21.15.26:0/572091073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01c0c3d80 con 0x7fb01c159820 2023-12-19T16:43:23.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.159+0000 7fb011ffb640 1 -- 172.21.15.26:0/572091073 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb00c0bfc80 con 0x7fb01c159820 2023-12-19T16:43:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.163+0000 7fb011ffb640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb000065a60 0x7fb000067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.163+0000 7fb011ffb640 1 -- 172.21.15.26:0/572091073 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb00c11d860 con 0x7fb01c159820 2023-12-19T16:43:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.163+0000 7fb021109640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb000065a60 0x7fb000067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.163+0000 7fb021109640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb000065a60 0x7fb000067f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb01c14f740 tx=0x7fb018009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:23.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.163+0000 7fb011ffb640 1 -- 172.21.15.26:0/572091073 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb00c0eb4d0 con 0x7fb01c159820 2023-12-19T16:43:23.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.319+0000 7fb02336d640 1 -- 172.21.15.26:0/572091073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb01c074270 con 0x7fb01c159820 2023-12-19T16:43:23.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.319+0000 7fb011ffb640 1 -- 172.21.15.26:0/572091073 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb00c0eeaf0 con 0x7fb01c159820 2023-12-19T16:43:23.327 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:43:23.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.323+0000 7fb02336d640 1 -- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb000065a60 msgr2=0x7fb000067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.323+0000 7fb02336d640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb000065a60 0x7fb000067f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb01c14f740 tx=0x7fb018009040 comp rx=0 tx=0).stop 2023-12-19T16:43:23.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.323+0000 7fb02336d640 1 -- 172.21.15.26:0/572091073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01c159820 msgr2=0x7fb01c0c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.323+0000 7fb02336d640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01c159820 0x7fb01c0c5770 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0ad040 tx=0x7fb00c01ac90 comp rx=0 tx=0).stop 2023-12-19T16:43:23.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.323+0000 7fb02336d640 1 -- 172.21.15.26:0/572091073 shutdown_connections 2023-12-19T16:43:23.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.323+0000 7fb02336d640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb01c159c00 0x7fb01c0ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.323+0000 7fb02336d640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb01c159820 0x7fb01c0c5770 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.323+0000 7fb02336d640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb000065a60 0x7fb000067f20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.323+0000 7fb02336d640 1 --2- 172.21.15.26:0/572091073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb01c06f6c0 0x7fb01c15c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.323+0000 7fb02336d640 1 -- 172.21.15.26:0/572091073 >> 172.21.15.26:0/572091073 conn(0x7fb01c0ce730 msgr2=0x7fb01c06ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:23.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.323+0000 7fb02336d640 1 -- 172.21.15.26:0/572091073 shutdown_connections 2023-12-19T16:43:23.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.323+0000 7fb02336d640 1 -- 172.21.15.26:0/572091073 wait complete. 2023-12-19T16:43:23.348 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T16:43:23.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.487+0000 7f5a1aa73640 1 Processor -- start 2023-12-19T16:43:23.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.487+0000 7f5a1aa73640 1 -- start start 2023-12-19T16:43:23.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.487+0000 7f5a1aa73640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5a140cb1d0 0x7f5a140cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.487+0000 7f5a1aa73640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5a1406bee0 0x7f5a1406c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.487+0000 7f5a1aa73640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1406d4b0 con 0x7f5a14164c90 2023-12-19T16:43:23.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.487+0000 7f5a1aa73640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a140cc030 con 0x7f5a1406bee0 2023-12-19T16:43:23.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.487+0000 7f5a1aa73640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140cbaf0 con 0x7f5a140cb1d0 2023-12-19T16:43:23.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.487+0000 7f5a13fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5a1406bee0 0x7f5a1406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.487+0000 7f5a19010640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5a14164c90 0x7f5a14165120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53828/0 (socket says 172.21.15.26:53828) 2023-12-19T16:43:23.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.487+0000 7f5a19010640 1 -- 172.21.15.26:0/430035533 learned_addr learned my addr 172.21.15.26:0/430035533 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:23.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.487+0000 7f5a1880f640 1 --2- 172.21.15.26:0/430035533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5a140cb1d0 0x7f5a140cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.491+0000 7f5a13fff640 1 -- 172.21.15.26:0/430035533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5a140cb1d0 msgr2=0x7f5a140cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:23.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.491+0000 7f5a13fff640 1 --2- 172.21.15.26:0/430035533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5a140cb1d0 0x7f5a140cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.491+0000 7f5a13fff640 1 -- 172.21.15.26:0/430035533 >> v1:172.21.15.26:6789/0 conn(0x7f5a14164c90 legacy=0x7f5a14165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.491+0000 7f5a13fff640 1 -- 172.21.15.26:0/430035533 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1406cc10 con 0x7f5a1406bee0 2023-12-19T16:43:23.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.491+0000 7f5a13fff640 1 --2- 172.21.15.26:0/430035533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a1406bee0 0x7f5a1406c340 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5a000097c0 tx=0x7f5a0001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106484cf2d04a34d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.491+0000 7f5a137fe640 1 -- 172.21.15.26:0/430035533 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a000ac620 con 0x7f5a1406bee0 2023-12-19T16:43:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.491+0000 7f5a137fe640 1 -- 172.21.15.26:0/430035533 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a000ac780 con 0x7f5a1406bee0 2023-12-19T16:43:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.491+0000 7f5a137fe640 1 -- 172.21.15.26:0/430035533 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a000acb10 con 0x7f5a1406bee0 2023-12-19T16:43:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.491+0000 7f5a1aa73640 1 -- 172.21.15.26:0/430035533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a1406bee0 msgr2=0x7f5a1406c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.491+0000 7f5a1aa73640 1 --2- 172.21.15.26:0/430035533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a1406bee0 0x7f5a1406c340 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5a000097c0 tx=0x7f5a0001c3c0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 -- 172.21.15.26:0/430035533 shutdown_connections 2023-12-19T16:43:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 --2- 172.21.15.26:0/430035533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a1406bee0 0x7f5a1406c340 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 --2- 172.21.15.26:0/430035533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5a140cb1d0 0x7f5a140cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 -- 172.21.15.26:0/430035533 >> 172.21.15.26:0/430035533 conn(0x7f5a140658e0 msgr2=0x7f5a14065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 -- 172.21.15.26:0/430035533 shutdown_connections 2023-12-19T16:43:23.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 -- 172.21.15.26:0/430035533 wait complete. 2023-12-19T16:43:23.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 Processor -- start 2023-12-19T16:43:23.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 -- start start 2023-12-19T16:43:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a1406bee0 0x7f5a1415a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a140cb1d0 0x7f5a1415cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5a14164c90 0x7f5a1415d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140b01b0 con 0x7f5a140cb1d0 2023-12-19T16:43:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1415ae00 con 0x7f5a1406bee0 2023-12-19T16:43:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1aa73640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1415c730 con 0x7f5a14164c90 2023-12-19T16:43:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a13fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a140cb1d0 0x7f5a1415cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a13fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a140cb1d0 0x7f5a1415cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57544/0 (socket says 172.21.15.26:57544) 2023-12-19T16:43:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a13fff640 1 -- 172.21.15.26:0/4105397374 learned_addr learned my addr 172.21.15.26:0/4105397374 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:23.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a19010640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5a14164c90 0x7f5a1415d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a1880f640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a1406bee0 0x7f5a1415a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a13fff640 1 -- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5a14164c90 msgr2=0x7f5a1415d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a13fff640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5a14164c90 0x7f5a1415d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a13fff640 1 -- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a1406bee0 msgr2=0x7f5a1415a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a13fff640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a1406bee0 0x7f5a1415a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a13fff640 1 -- 172.21.15.26:0/4105397374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a00009420 con 0x7f5a140cb1d0 2023-12-19T16:43:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.495+0000 7f5a19010640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5a14164c90 0x7f5a1415d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.499+0000 7f5a13fff640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a140cb1d0 0x7f5a1415cbc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5a000af040 tx=0x7f5a0001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.499+0000 7f5a117fa640 1 -- 172.21.15.26:0/4105397374 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a000b8450 con 0x7f5a140cb1d0 2023-12-19T16:43:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.499+0000 7f5a117fa640 1 -- 172.21.15.26:0/4105397374 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a000b85d0 con 0x7f5a140cb1d0 2023-12-19T16:43:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.499+0000 7f5a117fa640 1 -- 172.21.15.26:0/4105397374 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a000b8980 con 0x7f5a140cb1d0 2023-12-19T16:43:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.499+0000 7f5a1aa73640 1 -- 172.21.15.26:0/4105397374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1415b010 con 0x7f5a140cb1d0 2023-12-19T16:43:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.499+0000 7f5a1aa73640 1 -- 172.21.15.26:0/4105397374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1415c0e0 con 0x7f5a140cb1d0 2023-12-19T16:43:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.499+0000 7f5a1aa73640 1 -- 172.21.15.26:0/4105397374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1415c8a0 con 0x7f5a140cb1d0 2023-12-19T16:43:23.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.499+0000 7f5a117fa640 1 -- 172.21.15.26:0/4105397374 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5a000aa070 con 0x7f5a140cb1d0 2023-12-19T16:43:23.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.499+0000 7f5a117fa640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f59f8065a60 0x7f59f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.499+0000 7f5a117fa640 1 -- 172.21.15.26:0/4105397374 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5a0011d3f0 con 0x7f5a140cb1d0 2023-12-19T16:43:23.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.499+0000 7f5a1880f640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f59f8065a60 0x7f59f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.503+0000 7f5a1880f640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f59f8065a60 0x7f59f8067f20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009b60 tx=0x7f5a08009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:23.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.503+0000 7f5a117fa640 1 -- 172.21.15.26:0/4105397374 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5a000eafd0 con 0x7f5a140cb1d0 2023-12-19T16:43:23.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a1aa73640 1 -- 172.21.15.26:0/4105397374 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5a14074270 con 0x7f5a140cb1d0 2023-12-19T16:43:23.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a117fa640 1 -- 172.21.15.26:0/4105397374 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v33) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5a000b1090 con 0x7f5a140cb1d0 2023-12-19T16:43:23.668 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:23.668 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T16:43:23.668 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:23.668 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T16:43:23.668 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T16:43:23.668 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:23.668 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T16:43:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a1aa73640 1 -- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f59f8065a60 msgr2=0x7f59f8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a1aa73640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f59f8065a60 0x7f59f8067f20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009b60 tx=0x7f5a08009190 comp rx=0 tx=0).stop 2023-12-19T16:43:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a1aa73640 1 -- 172.21.15.26:0/4105397374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a140cb1d0 msgr2=0x7f5a1415cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a1aa73640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a140cb1d0 0x7f5a1415cbc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5a000af040 tx=0x7f5a0001ac00 comp rx=0 tx=0).stop 2023-12-19T16:43:23.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a1aa73640 1 -- 172.21.15.26:0/4105397374 shutdown_connections 2023-12-19T16:43:23.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a1aa73640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f59f8065a60 0x7f59f8067f20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a1aa73640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5a14164c90 0x7f5a1415d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a1aa73640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a140cb1d0 0x7f5a1415cbc0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a1aa73640 1 --2- 172.21.15.26:0/4105397374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5a1406bee0 0x7f5a1415a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a1aa73640 1 -- 172.21.15.26:0/4105397374 >> 172.21.15.26:0/4105397374 conn(0x7f5a140658e0 msgr2=0x7f5a14065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:23.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a1aa73640 1 -- 172.21.15.26:0/4105397374 shutdown_connections 2023-12-19T16:43:23.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.663+0000 7f5a1aa73640 1 -- 172.21.15.26:0/4105397374 wait complete. 2023-12-19T16:43:23.689 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T16:43:23.689 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T16:43:23.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd5546c4640 1 Processor -- start 2023-12-19T16:43:23.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd5546c4640 1 -- start start 2023-12-19T16:43:23.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd5546c4640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd54c15aad0 0x7fd54c06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd5546c4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd54c0b3f90 0x7fd54c0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd5546c4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54c06cbf0 con 0x7fd54c0b1590 2023-12-19T16:43:23.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd5546c4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54c0ca8b0 con 0x7fd54c0b3f90 2023-12-19T16:43:23.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd5546c4640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54c06b6f0 con 0x7fd54c15aad0 2023-12-19T16:43:23.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd552c61640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd54c0b3f90 0x7fd54c0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd551c5f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd54c0b1590 0x7fd54c070870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53840/0 (socket says 172.21.15.26:53840) 2023-12-19T16:43:23.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd551c5f640 1 -- 172.21.15.26:0/3640726164 learned_addr learned my addr 172.21.15.26:0/3640726164 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:23.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd552460640 1 --2- 172.21.15.26:0/3640726164 >> v2:172.21.15.160:3300/0 conn(0x7fd54c15aad0 0x7fd54c06c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd552c61640 1 -- 172.21.15.26:0/3640726164 >> v2:172.21.15.160:3300/0 conn(0x7fd54c15aad0 msgr2=0x7fd54c06c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:23.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd552c61640 1 --2- 172.21.15.26:0/3640726164 >> v2:172.21.15.160:3300/0 conn(0x7fd54c15aad0 0x7fd54c06c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd552c61640 1 -- 172.21.15.26:0/3640726164 >> v1:172.21.15.26:6789/0 conn(0x7fd54c0b1590 legacy=0x7fd54c070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.831+0000 7fd552c61640 1 -- 172.21.15.26:0/3640726164 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c06b9d0 con 0x7fd54c0b3f90 2023-12-19T16:43:23.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.835+0000 7fd552c61640 1 --2- 172.21.15.26:0/3640726164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0b3f90 0x7fd54c0cad40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0cc010 tx=0x7fd54401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92ec22154dec1dc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:23.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.835+0000 7fd55145e640 1 -- 172.21.15.26:0/3640726164 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5440a58b0 con 0x7fd54c0b3f90 2023-12-19T16:43:23.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.835+0000 7fd55145e640 1 -- 172.21.15.26:0/3640726164 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5440a5a10 con 0x7fd54c0b3f90 2023-12-19T16:43:23.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.835+0000 7fd55145e640 1 -- 172.21.15.26:0/3640726164 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5440a5da0 con 0x7fd54c0b3f90 2023-12-19T16:43:23.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.835+0000 7fd5546c4640 1 -- 172.21.15.26:0/3640726164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0b3f90 msgr2=0x7fd54c0cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.835+0000 7fd5546c4640 1 --2- 172.21.15.26:0/3640726164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0b3f90 0x7fd54c0cad40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0cc010 tx=0x7fd54401c8e0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.835+0000 7fd5546c4640 1 -- 172.21.15.26:0/3640726164 shutdown_connections 2023-12-19T16:43:23.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.835+0000 7fd5546c4640 1 --2- 172.21.15.26:0/3640726164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0b3f90 0x7fd54c0cad40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.835+0000 7fd5546c4640 1 --2- 172.21.15.26:0/3640726164 >> v2:172.21.15.160:3300/0 conn(0x7fd54c15aad0 0x7fd54c06c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.835+0000 7fd5546c4640 1 -- 172.21.15.26:0/3640726164 >> 172.21.15.26:0/3640726164 conn(0x7fd54c0b0270 msgr2=0x7fd54c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:23.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.835+0000 7fd5546c4640 1 -- 172.21.15.26:0/3640726164 shutdown_connections 2023-12-19T16:43:23.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.835+0000 7fd5546c4640 1 -- 172.21.15.26:0/3640726164 wait complete. 2023-12-19T16:43:23.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd5546c4640 1 Processor -- start 2023-12-19T16:43:23.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd5546c4640 1 -- start start 2023-12-19T16:43:23.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd5546c4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd54c0b1590 0x7fd54c151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd5546c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0b3f90 0x7fd54c154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd5546c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd54c15aad0 0x7fd54c1554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd5546c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c1524f0 con 0x7fd54c0b3f90 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd5546c4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c158320 con 0x7fd54c0b1590 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd5546c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c1541a0 con 0x7fd54c15aad0 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd551c5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0b3f90 0x7fd54c154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd551c5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0b3f90 0x7fd54c154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57562/0 (socket says 172.21.15.26:57562) 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd551c5f640 1 -- 172.21.15.26:0/2374780871 learned_addr learned my addr 172.21.15.26:0/2374780871 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd552460640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd54c0b1590 0x7fd54c151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd552c61640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd54c15aad0 0x7fd54c1554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd551c5f640 1 -- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd54c15aad0 msgr2=0x7fd54c1554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd551c5f640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd54c15aad0 0x7fd54c1554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd551c5f640 1 -- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd54c0b1590 msgr2=0x7fd54c151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd551c5f640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd54c0b1590 0x7fd54c151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd551c5f640 1 -- 172.21.15.26:0/2374780871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd544009a70 con 0x7fd54c0b3f90 2023-12-19T16:43:23.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd551c5f640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0b3f90 0x7fd54c154f60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00c4d0 tx=0x7fd53c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:23.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd542ffd640 1 -- 172.21.15.26:0/2374780871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c005cf0 con 0x7fd54c0b3f90 2023-12-19T16:43:23.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd542ffd640 1 -- 172.21.15.26:0/2374780871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c005e50 con 0x7fd54c0b3f90 2023-12-19T16:43:23.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd542ffd640 1 -- 172.21.15.26:0/2374780871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c007270 con 0x7fd54c0b3f90 2023-12-19T16:43:23.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd5546c4640 1 -- 172.21.15.26:0/2374780871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54c154480 con 0x7fd54c0b3f90 2023-12-19T16:43:23.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd5546c4640 1 -- 172.21.15.26:0/2374780871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54c154a40 con 0x7fd54c0b3f90 2023-12-19T16:43:23.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.839+0000 7fd5546c4640 1 -- 172.21.15.26:0/2374780871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54c154c60 con 0x7fd54c0b3f90 2023-12-19T16:43:23.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.843+0000 7fd542ffd640 1 -- 172.21.15.26:0/2374780871 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd53c0073d0 con 0x7fd54c0b3f90 2023-12-19T16:43:23.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.843+0000 7fd542ffd640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd530065a60 0x7fd530067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:23.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.843+0000 7fd542ffd640 1 -- 172.21.15.26:0/2374780871 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd53c082130 con 0x7fd54c0b3f90 2023-12-19T16:43:23.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.843+0000 7fd552460640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd530065a60 0x7fd530067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:23.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.843+0000 7fd552460640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd530065a60 0x7fd530067f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd548009a50 tx=0x7fd548008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:23.847+0000 7fd542ffd640 1 -- 172.21.15.26:0/2374780871 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd53c04fe50 con 0x7fd54c0b3f90 2023-12-19T16:43:24.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.003+0000 7fd5546c4640 1 -- 172.21.15.26:0/2374780871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd54c1559e0 con 0x7fd54c0b3f90 2023-12-19T16:43:24.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.451+0000 7fd542ffd640 1 -- 172.21.15.26:0/2374780871 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v34) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd53c053470 con 0x7fd54c0b3f90 2023-12-19T16:43:24.459 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T16:43:24.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.455+0000 7fd5546c4640 1 -- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd530065a60 msgr2=0x7fd530067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:24.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.455+0000 7fd5546c4640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd530065a60 0x7fd530067f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd548009a50 tx=0x7fd548008040 comp rx=0 tx=0).stop 2023-12-19T16:43:24.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.455+0000 7fd5546c4640 1 -- 172.21.15.26:0/2374780871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0b3f90 msgr2=0x7fd54c154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:24.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.455+0000 7fd5546c4640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0b3f90 0x7fd54c154f60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00c4d0 tx=0x7fd53c00c990 comp rx=0 tx=0).stop 2023-12-19T16:43:24.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.455+0000 7fd5546c4640 1 -- 172.21.15.26:0/2374780871 shutdown_connections 2023-12-19T16:43:24.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.455+0000 7fd5546c4640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd530065a60 0x7fd530067f20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:24.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.455+0000 7fd5546c4640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd54c15aad0 0x7fd54c1554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:24.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.455+0000 7fd5546c4640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0b3f90 0x7fd54c154f60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:24.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.455+0000 7fd5546c4640 1 --2- 172.21.15.26:0/2374780871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd54c0b1590 0x7fd54c151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:24.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.455+0000 7fd5546c4640 1 -- 172.21.15.26:0/2374780871 >> 172.21.15.26:0/2374780871 conn(0x7fd54c0b0270 msgr2=0x7fd54c1599f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:24.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.459+0000 7fd5546c4640 1 -- 172.21.15.26:0/2374780871 shutdown_connections 2023-12-19T16:43:24.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.459+0000 7fd5546c4640 1 -- 172.21.15.26:0/2374780871 wait complete. 2023-12-19T16:43:24.488 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T16:43:24.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6aec28c640 1 Processor -- start 2023-12-19T16:43:24.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6aec28c640 1 -- start start 2023-12-19T16:43:24.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6aec28c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ae40adac0 0x7f6ae40c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:24.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6aec28c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ae40703c0 0x7f6ae406bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:24.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6aec28c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae406b3d0 con 0x7f6ae415a480 2023-12-19T16:43:24.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6aec28c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae40ca5e0 con 0x7f6ae40adac0 2023-12-19T16:43:24.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6aec28c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40ca8c0 con 0x7f6ae40703c0 2023-12-19T16:43:24.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6ae9827640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6ae415a480 0x7f6ae406fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53844/0 (socket says 172.21.15.26:53844) 2023-12-19T16:43:24.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6aea028640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ae40adac0 0x7f6ae40c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:24.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6ae9827640 1 -- 172.21.15.26:0/1862325079 learned_addr learned my addr 172.21.15.26:0/1862325079 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:24.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6aea829640 1 --2- 172.21.15.26:0/1862325079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ae40703c0 0x7f6ae406bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:24.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6aea028640 1 -- 172.21.15.26:0/1862325079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ae40703c0 msgr2=0x7f6ae406bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:24.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6aea028640 1 --2- 172.21.15.26:0/1862325079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ae40703c0 0x7f6ae406bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:24.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6aea028640 1 -- 172.21.15.26:0/1862325079 >> v1:172.21.15.26:6789/0 conn(0x7f6ae415a480 legacy=0x7f6ae406fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:24.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.635+0000 7f6aea028640 1 -- 172.21.15.26:0/1862325079 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae40caba0 con 0x7f6ae40adac0 2023-12-19T16:43:24.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.639+0000 7f6aea028640 1 --2- 172.21.15.26:0/1862325079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ae40adac0 0x7f6ae40c9d90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000a960 tx=0x7f6ae001d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26a761c0d3a42e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.639+0000 7f6ae9026640 1 -- 172.21.15.26:0/1862325079 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00a6830 con 0x7f6ae40adac0 2023-12-19T16:43:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.639+0000 7f6ae9026640 1 -- 172.21.15.26:0/1862325079 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae00a6990 con 0x7f6ae40adac0 2023-12-19T16:43:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.639+0000 7f6ae9026640 1 -- 172.21.15.26:0/1862325079 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00a6d20 con 0x7f6ae40adac0 2023-12-19T16:43:24.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.639+0000 7f6aec28c640 1 -- 172.21.15.26:0/1862325079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ae40adac0 msgr2=0x7f6ae40c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:24.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.639+0000 7f6aec28c640 1 --2- 172.21.15.26:0/1862325079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ae40adac0 0x7f6ae40c9d90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000a960 tx=0x7f6ae001d820 comp rx=0 tx=0).stop 2023-12-19T16:43:24.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.639+0000 7f6aec28c640 1 -- 172.21.15.26:0/1862325079 shutdown_connections 2023-12-19T16:43:24.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.639+0000 7f6aec28c640 1 --2- 172.21.15.26:0/1862325079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ae40703c0 0x7f6ae406bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:24.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.639+0000 7f6aec28c640 1 --2- 172.21.15.26:0/1862325079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ae40adac0 0x7f6ae40c9d90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:24.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.639+0000 7f6aec28c640 1 -- 172.21.15.26:0/1862325079 >> 172.21.15.26:0/1862325079 conn(0x7f6ae40660d0 msgr2=0x7f6ae40ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:24.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.639+0000 7f6aec28c640 1 -- 172.21.15.26:0/1862325079 shutdown_connections 2023-12-19T16:43:24.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.639+0000 7f6aec28c640 1 -- 172.21.15.26:0/1862325079 wait complete. 2023-12-19T16:43:24.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.639+0000 7f6aec28c640 1 Processor -- start 2023-12-19T16:43:24.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aec28c640 1 -- start start 2023-12-19T16:43:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aec28c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ae40703c0 0x7f6ae40b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aec28c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ae40adac0 0x7f6ae41f8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aec28c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ae415a480 0x7f6ae41f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aec28c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40c5cc0 con 0x7f6ae415a480 2023-12-19T16:43:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aec28c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae4158480 con 0x7f6ae40adac0 2023-12-19T16:43:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aec28c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae41581e0 con 0x7f6ae40703c0 2023-12-19T16:43:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aea829640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ae415a480 0x7f6ae41f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aea829640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ae415a480 0x7f6ae41f8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57588/0 (socket says 172.21.15.26:57588) 2023-12-19T16:43:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aea829640 1 -- 172.21.15.26:0/3948095139 learned_addr learned my addr 172.21.15.26:0/3948095139 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aea028640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ae40703c0 0x7f6ae40b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6ae9827640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ae40adac0 0x7f6ae41f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aea829640 1 -- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ae40703c0 msgr2=0x7f6ae40b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aea829640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ae40703c0 0x7f6ae40b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aea829640 1 -- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ae40adac0 msgr2=0x7f6ae41f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aea829640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ae40adac0 0x7f6ae41f8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aea829640 1 -- 172.21.15.26:0/3948095139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae000a5c0 con 0x7f6ae415a480 2023-12-19T16:43:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6ae9827640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ae40adac0 0x7f6ae41f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:24.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aea829640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ae415a480 0x7f6ae41f8ec0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6adc007de0 tx=0x7f6adc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:24.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6adaffd640 1 -- 172.21.15.26:0/3948095139 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc014620 con 0x7f6ae415a480 2023-12-19T16:43:24.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6adaffd640 1 -- 172.21.15.26:0/3948095139 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc00ee90 con 0x7f6ae415a480 2023-12-19T16:43:24.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6adaffd640 1 -- 172.21.15.26:0/3948095139 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0149e0 con 0x7f6ae415a480 2023-12-19T16:43:24.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aec28c640 1 -- 172.21.15.26:0/3948095139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae40c5950 con 0x7f6ae415a480 2023-12-19T16:43:24.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aec28c640 1 -- 172.21.15.26:0/3948095139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae40a8e10 con 0x7f6ae415a480 2023-12-19T16:43:24.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.643+0000 7f6aec28c640 1 -- 172.21.15.26:0/3948095139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae40c5e70 con 0x7f6ae415a480 2023-12-19T16:43:24.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.647+0000 7f6adaffd640 1 -- 172.21.15.26:0/3948095139 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6adc014b40 con 0x7f6ae415a480 2023-12-19T16:43:24.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.647+0000 7f6adaffd640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6ac8065a60 0x7f6ac8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.647+0000 7f6adaffd640 1 -- 172.21.15.26:0/3948095139 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6adc0838f0 con 0x7f6ae415a480 2023-12-19T16:43:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.647+0000 7f6aea028640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6ac8065a60 0x7f6ac8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.647+0000 7f6aea028640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6ac8065a60 0x7f6ac8067f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00b0000 tx=0x7f6ae0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:24.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.651+0000 7f6adaffd640 1 -- 172.21.15.26:0/3948095139 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6ae40c5e70 con 0x7f6ae415a480 2023-12-19T16:43:24.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:24.807+0000 7f6aec28c640 1 -- 172.21.15.26:0/3948095139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6ae40c5e70 con 0x7f6ae415a480 2023-12-19T16:43:25.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.459+0000 7f6adaffd640 1 -- 172.21.15.26:0/3948095139 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v35) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6ae40c5e70 con 0x7f6ae415a480 2023-12-19T16:43:25.465 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T16:43:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.467+0000 7f6aec28c640 1 -- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6ac8065a60 msgr2=0x7f6ac8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.467+0000 7f6aec28c640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6ac8065a60 0x7f6ac8067f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00b0000 tx=0x7f6ae0008040 comp rx=0 tx=0).stop 2023-12-19T16:43:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.467+0000 7f6aec28c640 1 -- 172.21.15.26:0/3948095139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ae415a480 msgr2=0x7f6ae41f8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.467+0000 7f6aec28c640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ae415a480 0x7f6ae41f8ec0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6adc007de0 tx=0x7f6adc00e4b0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.467+0000 7f6aec28c640 1 -- 172.21.15.26:0/3948095139 shutdown_connections 2023-12-19T16:43:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.467+0000 7f6aec28c640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ae415a480 0x7f6ae41f8ec0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.467+0000 7f6aec28c640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ae40adac0 0x7f6ae41f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.467+0000 7f6aec28c640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6ac8065a60 0x7f6ac8067f20 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00b0000 tx=0x7f6ae0008040 comp rx=0 tx=0).stop 2023-12-19T16:43:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.467+0000 7f6aec28c640 1 --2- 172.21.15.26:0/3948095139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ae40703c0 0x7f6ae40b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.467+0000 7f6aec28c640 1 -- 172.21.15.26:0/3948095139 >> 172.21.15.26:0/3948095139 conn(0x7f6ae40660d0 msgr2=0x7f6ae4159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.471+0000 7f6aec28c640 1 -- 172.21.15.26:0/3948095139 shutdown_connections 2023-12-19T16:43:25.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.471+0000 7f6aec28c640 1 -- 172.21.15.26:0/3948095139 wait complete. 2023-12-19T16:43:25.511 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T16:43:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f5406551640 1 Processor -- start 2023-12-19T16:43:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f5406551640 1 -- start start 2023-12-19T16:43:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f5406551640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f540015cc90 0x7f54000cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f5406551640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f540015d070 0x7f54000cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f5406551640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54000cdb20 con 0x7f5400160a20 2023-12-19T16:43:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f5406551640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54000cde00 con 0x7f540015cc90 2023-12-19T16:43:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f5406551640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54000ce0e0 con 0x7f540015d070 2023-12-19T16:43:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f5404d4e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f540015d070 0x7f54000cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f540554f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f540015cc90 0x7f54000cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f5405d50640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f5400160a20 0x7f54000b45a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:35864/0 (socket says 172.21.15.26:35864) 2023-12-19T16:43:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f5404d4e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f540015d070 0x7f54000cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:37198/0 (socket says 172.21.15.26:37198) 2023-12-19T16:43:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f5405d50640 1 -- 172.21.15.26:0/3305610837 learned_addr learned my addr 172.21.15.26:0/3305610837 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f53f7fff640 1 -- 172.21.15.26:0/3305610837 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792054844 0 0) 0x7f54000cdb20 con 0x7f5400160a20 2023-12-19T16:43:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f53f7fff640 1 -- 172.21.15.26:0/3305610837 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53ec003530 con 0x7f5400160a20 2023-12-19T16:43:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f540554f640 1 -- 172.21.15.26:0/3305610837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f540015d070 msgr2=0x7f54000cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f540554f640 1 --2- 172.21.15.26:0/3305610837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f540015d070 0x7f54000cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f540554f640 1 -- 172.21.15.26:0/3305610837 >> v1:172.21.15.160:6789/0 conn(0x7f5400160a20 legacy=0x7f54000b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f540554f640 1 -- 172.21.15.26:0/3305610837 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54000ce3c0 con 0x7f540015cc90 2023-12-19T16:43:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.655+0000 7f5404d4e640 1 --2- 172.21.15.26:0/3305610837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f540015d070 0x7f54000cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.659+0000 7f540554f640 1 --2- 172.21.15.26:0/3305610837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f540015cc90 0x7f54000cd2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0097c0 tx=0x7f53fc01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bda172fa6e9a8b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.659+0000 7f53f7fff640 1 -- 172.21.15.26:0/3305610837 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0a7070 con 0x7f540015cc90 2023-12-19T16:43:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.659+0000 7f53f7fff640 1 -- 172.21.15.26:0/3305610837 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc0ab430 con 0x7f540015cc90 2023-12-19T16:43:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.659+0000 7f53f7fff640 1 -- 172.21.15.26:0/3305610837 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0ab7c0 con 0x7f540015cc90 2023-12-19T16:43:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.659+0000 7f5406551640 1 -- 172.21.15.26:0/3305610837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f540015cc90 msgr2=0x7f54000cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.659+0000 7f5406551640 1 --2- 172.21.15.26:0/3305610837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f540015cc90 0x7f54000cd2f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0097c0 tx=0x7f53fc01c450 comp rx=0 tx=0).stop 2023-12-19T16:43:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.659+0000 7f5406551640 1 -- 172.21.15.26:0/3305610837 shutdown_connections 2023-12-19T16:43:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.659+0000 7f5406551640 1 --2- 172.21.15.26:0/3305610837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f540015d070 0x7f54000cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.659+0000 7f5406551640 1 --2- 172.21.15.26:0/3305610837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f540015cc90 0x7f54000cd2f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.659+0000 7f5406551640 1 -- 172.21.15.26:0/3305610837 >> 172.21.15.26:0/3305610837 conn(0x7f540006a320 msgr2=0x7f54000658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.659+0000 7f5406551640 1 -- 172.21.15.26:0/3305610837 shutdown_connections 2023-12-19T16:43:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.659+0000 7f5406551640 1 -- 172.21.15.26:0/3305610837 wait complete. 2023-12-19T16:43:25.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f5406551640 1 Processor -- start 2023-12-19T16:43:25.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f5406551640 1 -- start start 2023-12-19T16:43:25.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f5406551640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f540015cc90 0x7f540015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:25.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f5406551640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f540015d070 0x7f5400153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:25.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f5406551640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5400160a20 0x7f540015ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:25.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f5406551640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54000c7130 con 0x7f540015cc90 2023-12-19T16:43:25.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f5406551640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54001552a0 con 0x7f540015d070 2023-12-19T16:43:25.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f5406551640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54001546b0 con 0x7f5400160a20 2023-12-19T16:43:25.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f540554f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f540015cc90 0x7f540015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:25.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f540554f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f540015cc90 0x7f540015bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57592/0 (socket says 172.21.15.26:57592) 2023-12-19T16:43:25.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f540554f640 1 -- 172.21.15.26:0/3710376734 learned_addr learned my addr 172.21.15.26:0/3710376734 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:25.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f5405d50640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5400160a20 0x7f540015ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:25.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f5404d4e640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f540015d070 0x7f5400153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:25.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f540554f640 1 -- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5400160a20 msgr2=0x7f540015ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:25.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f540554f640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5400160a20 0x7f540015ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f540554f640 1 -- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f540015d070 msgr2=0x7f5400153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:25.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f540554f640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f540015d070 0x7f5400153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f540554f640 1 -- 172.21.15.26:0/3710376734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f0009420 con 0x7f540015cc90 2023-12-19T16:43:25.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f540554f640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f540015cc90 0x7f540015bce0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0ad040 tx=0x7f53fc0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:25.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f53f5ffb640 1 -- 172.21.15.26:0/3710376734 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0b45f0 con 0x7f540015cc90 2023-12-19T16:43:25.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f5406551640 1 -- 172.21.15.26:0/3710376734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53fc009420 con 0x7f540015cc90 2023-12-19T16:43:25.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f53f5ffb640 1 -- 172.21.15.26:0/3710376734 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc0b4770 con 0x7f540015cc90 2023-12-19T16:43:25.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f53f5ffb640 1 -- 172.21.15.26:0/3710376734 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0b4b20 con 0x7f540015cc90 2023-12-19T16:43:25.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.663+0000 7f5406551640 1 -- 172.21.15.26:0/3710376734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5400154c50 con 0x7f540015cc90 2023-12-19T16:43:25.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.667+0000 7f5406551640 1 -- 172.21.15.26:0/3710376734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5400061930 con 0x7f540015cc90 2023-12-19T16:43:25.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.667+0000 7f53f5ffb640 1 -- 172.21.15.26:0/3710376734 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f53fc0b4c80 con 0x7f540015cc90 2023-12-19T16:43:25.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.667+0000 7f53f5ffb640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53e4065a60 0x7f53e4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:25.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.667+0000 7f53f5ffb640 1 -- 172.21.15.26:0/3710376734 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f53fc11d4c0 con 0x7f540015cc90 2023-12-19T16:43:25.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.667+0000 7f5404d4e640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53e4065a60 0x7f53e4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:25.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.667+0000 7f5404d4e640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53e4065a60 0x7f53e4067f20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54000ccad0 tx=0x7f53f0009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:25.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.671+0000 7f53f5ffb640 1 -- 172.21.15.26:0/3710376734 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f53fc0eaf90 con 0x7f540015cc90 2023-12-19T16:43:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.863+0000 7f5406551640 1 -- 172.21.15.26:0/3710376734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f54000d0970 con 0x7f540015cc90 2023-12-19T16:43:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.871+0000 7f53f5ffb640 1 -- 172.21.15.26:0/3710376734 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f53fc0ee5b0 con 0x7f540015cc90 2023-12-19T16:43:25.879 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 4 and data pool 5 2023-12-19T16:43:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.879+0000 7f5406551640 1 -- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53e4065a60 msgr2=0x7f53e4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.879+0000 7f5406551640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53e4065a60 0x7f53e4067f20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54000ccad0 tx=0x7f53f0009190 comp rx=0 tx=0).stop 2023-12-19T16:43:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.879+0000 7f5406551640 1 -- 172.21.15.26:0/3710376734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f540015cc90 msgr2=0x7f540015bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.879+0000 7f5406551640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f540015cc90 0x7f540015bce0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0ad040 tx=0x7f53fc0191f0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.879+0000 7f5406551640 1 -- 172.21.15.26:0/3710376734 shutdown_connections 2023-12-19T16:43:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.879+0000 7f5406551640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53e4065a60 0x7f53e4067f20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.879+0000 7f5406551640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5400160a20 0x7f540015ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.879+0000 7f5406551640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f540015d070 0x7f5400153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.879+0000 7f5406551640 1 --2- 172.21.15.26:0/3710376734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f540015cc90 0x7f540015bce0 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0ad040 tx=0x7f53fc0191f0 comp rx=0 tx=0).stop 2023-12-19T16:43:25.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.879+0000 7f5406551640 1 -- 172.21.15.26:0/3710376734 >> 172.21.15.26:0/3710376734 conn(0x7f540006a320 msgr2=0x7f54000b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:25.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.883+0000 7f5406551640 1 -- 172.21.15.26:0/3710376734 shutdown_connections 2023-12-19T16:43:25.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:25.887+0000 7f5406551640 1 -- 172.21.15.26:0/3710376734 wait complete. 2023-12-19T16:43:25.926 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:43:26.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.063+0000 7f4296266640 1 Processor -- start 2023-12-19T16:43:26.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.063+0000 7f4296266640 1 -- start start 2023-12-19T16:43:26.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f4296266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42900afd70 0x7f42900d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f4296266640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429006b730 con 0x7f42900cc9b0 2023-12-19T16:43:26.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f4296266640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42901617d0 con 0x7f42900cd8e0 2023-12-19T16:43:26.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f4296266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42900b3020 con 0x7f42900afd70 2023-12-19T16:43:26.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f42900cc9b0 0x7f42900cd1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47330/0 (socket says 172.21.15.26:47330) 2023-12-19T16:43:26.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428ffff640 1 -- 172.21.15.26:0/2871328254 learned_addr learned my addr 172.21.15.26:0/2871328254 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f4294803640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42900afd70 0x7f42900d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250706884 0 0) 0x7f429006b730 con 0x7f42900cc9b0 2023-12-19T16:43:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4278003530 con 0x7f42900cc9b0 2023-12-19T16:43:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972331469 0 0) 0x7f42901617d0 con 0x7f42900cd8e0 2023-12-19T16:43:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f429006b730 con 0x7f42900cd8e0 2023-12-19T16:43:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2307016449 0 0) 0x7f4278003530 con 0x7f42900cc9b0 2023-12-19T16:43:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42900afd70 msgr2=0x7f42900d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42900afd70 0x7f42900d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 >> v1:172.21.15.160:6789/0 conn(0x7f42900cd8e0 legacy=0x7f42900cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4290161a40 con 0x7f42900cc9b0 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f4280001680 con 0x7f42900cc9b0 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f4294803640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42900afd70 0x7f42900d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 >> v1:172.21.15.26:6789/0 conn(0x7f42900cc9b0 legacy=0x7f42900cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42900afd70 0x7f4278003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42780040d0 0x7f42780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4278004d10 0x7f4278005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42900b3020 con 0x7f42780040d0 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427800bb50 con 0x7f4278004d10 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427800bd10 con 0x7f42900afd70 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428f7fe640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42780040d0 0x7f42780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428ffff640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42900afd70 0x7f4278003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f4294803640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4278004d10 0x7f4278005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428f7fe640 1 -- 172.21.15.26:0/2871328254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42900afd70 msgr2=0x7f4278003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428f7fe640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42900afd70 0x7f4278003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428f7fe640 1 -- 172.21.15.26:0/2871328254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4278004d10 msgr2=0x7f4278005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428f7fe640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4278004d10 0x7f4278005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f428f7fe640 1 -- 172.21.15.26:0/2871328254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4288009a70 con 0x7f42780040d0 2023-12-19T16:43:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.067+0000 7f4294803640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4278004d10 0x7f4278005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:26.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.071+0000 7f428f7fe640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42780040d0 0x7f42780047d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f42840018d0 tx=0x7f428401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70959fce9c0898eb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.071+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42840052d0 con 0x7f42780040d0 2023-12-19T16:43:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.071+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4284005450 con 0x7f42780040d0 2023-12-19T16:43:26.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.071+0000 7f428effd640 1 -- 172.21.15.26:0/2871328254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4284003840 con 0x7f42780040d0 2023-12-19T16:43:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.071+0000 7f4296266640 1 -- 172.21.15.26:0/2871328254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42780040d0 msgr2=0x7f42780047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.071+0000 7f4296266640 1 --2- 172.21.15.26:0/2871328254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42780040d0 0x7f42780047d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f42840018d0 tx=0x7f428401c560 comp rx=0 tx=0).stop 2023-12-19T16:43:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.071+0000 7f4294803640 1 -- 172.21.15.26:0/2871328254 reap_dead start 2023-12-19T16:43:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.071+0000 7f4296266640 1 -- 172.21.15.26:0/2871328254 shutdown_connections 2023-12-19T16:43:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.071+0000 7f4296266640 1 -- 172.21.15.26:0/2871328254 >> 172.21.15.26:0/2871328254 conn(0x7f429006d570 msgr2=0x7f429006d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.071+0000 7f4296266640 1 -- 172.21.15.26:0/2871328254 shutdown_connections 2023-12-19T16:43:26.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.071+0000 7f4296266640 1 -- 172.21.15.26:0/2871328254 wait complete. 2023-12-19T16:43:26.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.071+0000 7f4296266640 1 Processor -- start 2023-12-19T16:43:26.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f4296266640 1 -- start start 2023-12-19T16:43:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f4296266640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42780040d0 0x7f4290071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f4296266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4290156d80 0x7f42900b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f4296266640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42900b62f0 0x7f42900b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f4296266640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42900b5310 con 0x7f42780040d0 2023-12-19T16:43:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f4296266640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4290157240 con 0x7f42900b62f0 2023-12-19T16:43:26.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f4296266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42900b55b0 con 0x7f4290156d80 2023-12-19T16:43:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f428ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42780040d0 0x7f4290071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f428ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42780040d0 0x7f4290071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55152/0 (socket says 172.21.15.26:55152) 2023-12-19T16:43:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f428ffff640 1 -- 172.21.15.26:0/2581565146 learned_addr learned my addr 172.21.15.26:0/2581565146 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f428f7fe640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4290156d80 0x7f42900b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f4294803640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42900b62f0 0x7f42900b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f428ffff640 1 -- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4290156d80 msgr2=0x7f42900b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f428ffff640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4290156d80 0x7f42900b5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f428ffff640 1 -- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42900b62f0 msgr2=0x7f42900b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f428ffff640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42900b62f0 0x7f42900b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f428ffff640 1 -- 172.21.15.26:0/2581565146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42900b5890 con 0x7f42780040d0 2023-12-19T16:43:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f4294803640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42900b62f0 0x7f42900b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:26.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f428ffff640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42780040d0 0x7f4290071660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f428000e7b0 tx=0x7f428000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f428cff9640 1 -- 172.21.15.26:0/2581565146 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428000c6c0 con 0x7f42780040d0 2023-12-19T16:43:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f428cff9640 1 -- 172.21.15.26:0/2581565146 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428000c820 con 0x7f42780040d0 2023-12-19T16:43:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f4296266640 1 -- 172.21.15.26:0/2581565146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4290157520 con 0x7f42780040d0 2023-12-19T16:43:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f428cff9640 1 -- 172.21.15.26:0/2581565146 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428000cbd0 con 0x7f42780040d0 2023-12-19T16:43:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f4296266640 1 -- 172.21.15.26:0/2581565146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4290071d70 con 0x7f42780040d0 2023-12-19T16:43:26.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.075+0000 7f4296266640 1 -- 172.21.15.26:0/2581565146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42900b5170 con 0x7f42780040d0 2023-12-19T16:43:26.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.079+0000 7f428cff9640 1 -- 172.21.15.26:0/2581565146 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f428000cd30 con 0x7f42780040d0 2023-12-19T16:43:26.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.079+0000 7f428cff9640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f427c065a90 0x7f427c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.079+0000 7f428cff9640 1 -- 172.21.15.26:0/2581565146 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4280082fa0 con 0x7f42780040d0 2023-12-19T16:43:26.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.079+0000 7f428f7fe640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f427c065a90 0x7f427c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.079+0000 7f428f7fe640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f427c065a90 0x7f427c067f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f42840ad000 tx=0x7f42840a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:26.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.083+0000 7f428cff9640 1 -- 172.21.15.26:0/2581565146 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f42800b40a0 con 0x7f42780040d0 2023-12-19T16:43:26.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.243+0000 7f4296266640 1 -- 172.21.15.26:0/2581565146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42900b5ad0 con 0x7f42780040d0 2023-12-19T16:43:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.243+0000 7f428cff9640 1 -- 172.21.15.26:0/2581565146 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16114 (secure 0 0 0) 0x7f42900b5ad0 con 0x7f42780040d0 2023-12-19T16:43:26.249 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:26.249 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":36,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:43:25.865727+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:43:24.052043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T16:43:24.856934+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:43:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.243+0000 7f4296266640 1 -- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f427c065a90 msgr2=0x7f427c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.243+0000 7f4296266640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f427c065a90 0x7f427c067f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f42840ad000 tx=0x7f42840a7070 comp rx=0 tx=0).stop 2023-12-19T16:43:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.247+0000 7f4296266640 1 -- 172.21.15.26:0/2581565146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42780040d0 msgr2=0x7f4290071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.247+0000 7f4296266640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42780040d0 0x7f4290071660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f428000e7b0 tx=0x7f428000ec70 comp rx=0 tx=0).stop 2023-12-19T16:43:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.247+0000 7f4296266640 1 -- 172.21.15.26:0/2581565146 shutdown_connections 2023-12-19T16:43:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.247+0000 7f4296266640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42900b62f0 0x7f42900b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.247+0000 7f4296266640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f427c065a90 0x7f427c067f50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.247+0000 7f4296266640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4290156d80 0x7f42900b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.247+0000 7f4296266640 1 --2- 172.21.15.26:0/2581565146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42780040d0 0x7f4290071660 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.247+0000 7f4296266640 1 -- 172.21.15.26:0/2581565146 >> 172.21.15.26:0/2581565146 conn(0x7f429006d570 msgr2=0x7f42900cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.247+0000 7f4296266640 1 -- 172.21.15.26:0/2581565146 shutdown_connections 2023-12-19T16:43:26.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.247+0000 7f4296266640 1 -- 172.21.15.26:0/2581565146 wait complete. 2023-12-19T16:43:26.271 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:43:26.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.407+0000 7f50a1352640 1 Processor -- start 2023-12-19T16:43:26.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.407+0000 7f50a1352640 1 -- start start 2023-12-19T16:43:26.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.411+0000 7f50a1352640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f509c0afd70 0x7f509c0d0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.411+0000 7f50a1352640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509c06b7c0 con 0x7f509c0cc770 2023-12-19T16:43:26.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.411+0000 7f50a1352640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509c0b3240 con 0x7f509c0cd840 2023-12-19T16:43:26.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.411+0000 7f50a1352640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509c161730 con 0x7f509c0afd70 2023-12-19T16:43:26.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.411+0000 7f509affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f509c0cc770 0x7f509c0cd130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47340/0 (socket says 172.21.15.26:47340) 2023-12-19T16:43:26.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.411+0000 7f509affd640 1 -- 172.21.15.26:0/2461587172 learned_addr learned my addr 172.21.15.26:0/2461587172 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:26.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.411+0000 7f509b7fe640 1 --2- 172.21.15.26:0/2461587172 >> v2:172.21.15.26:3300/0 conn(0x7f509c0afd70 0x7f509c0d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.411+0000 7f509a7fc640 1 -- 172.21.15.26:0/2461587172 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534376588 0 0) 0x7f509c06b7c0 con 0x7f509c0cc770 2023-12-19T16:43:26.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.411+0000 7f509a7fc640 1 -- 172.21.15.26:0/2461587172 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5088003530 con 0x7f509c0cc770 2023-12-19T16:43:26.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.411+0000 7f509b7fe640 1 -- 172.21.15.26:0/2461587172 >> v1:172.21.15.160:6789/0 conn(0x7f509c0cd840 legacy=0x7f509c0cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.411+0000 7f509a7fc640 1 -- 172.21.15.26:0/2461587172 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448758102 0 0) 0x7f509c0b3240 con 0x7f509c0cd840 2023-12-19T16:43:26.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.411+0000 7f509b7fe640 1 -- 172.21.15.26:0/2461587172 >> v1:172.21.15.26:6789/0 conn(0x7f509c0cc770 legacy=0x7f509c0cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.411+0000 7f509b7fe640 1 -- 172.21.15.26:0/2461587172 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c161a10 con 0x7f509c0afd70 2023-12-19T16:43:26.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f509b7fe640 1 --2- 172.21.15.26:0/2461587172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f509c0afd70 0x7f509c0d0520 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5094009d90 tx=0x7f509401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9144d5703722b1d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:26.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f509a7fc640 1 -- 172.21.15.26:0/2461587172 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50940a5a30 con 0x7f509c0afd70 2023-12-19T16:43:26.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f509a7fc640 1 -- 172.21.15.26:0/2461587172 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50940a5b90 con 0x7f509c0afd70 2023-12-19T16:43:26.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f509a7fc640 1 -- 172.21.15.26:0/2461587172 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50940ad440 con 0x7f509c0afd70 2023-12-19T16:43:26.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f50a1352640 1 -- 172.21.15.26:0/2461587172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f509c0afd70 msgr2=0x7f509c0d0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f50a1352640 1 --2- 172.21.15.26:0/2461587172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f509c0afd70 0x7f509c0d0520 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5094009d90 tx=0x7f509401c950 comp rx=0 tx=0).stop 2023-12-19T16:43:26.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f50a1352640 1 -- 172.21.15.26:0/2461587172 shutdown_connections 2023-12-19T16:43:26.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f50a1352640 1 --2- 172.21.15.26:0/2461587172 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f509c0afd70 0x7f509c0d0520 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f50a1352640 1 -- 172.21.15.26:0/2461587172 >> 172.21.15.26:0/2461587172 conn(0x7f509c06d570 msgr2=0x7f509c06d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:26.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f50a1352640 1 -- 172.21.15.26:0/2461587172 shutdown_connections 2023-12-19T16:43:26.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f50a1352640 1 -- 172.21.15.26:0/2461587172 wait complete. 2023-12-19T16:43:26.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f50a1352640 1 Processor -- start 2023-12-19T16:43:26.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f50a1352640 1 -- start start 2023-12-19T16:43:26.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f50a1352640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f509c0afd70 0x7f509c0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.415+0000 7f50a1352640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f509c0cc770 0x7f509c0b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f50a1352640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f509c0cd840 0x7f509c0b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f50a1352640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c15c410 con 0x7f509c0afd70 2023-12-19T16:43:26.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f50a1352640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0b47c0 con 0x7f509c0cc770 2023-12-19T16:43:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f50a1352640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c15b800 con 0x7f509c0cd840 2023-12-19T16:43:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f509affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f509c0afd70 0x7f509c0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f509affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f509c0afd70 0x7f509c0c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55168/0 (socket says 172.21.15.26:55168) 2023-12-19T16:43:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f509affd640 1 -- 172.21.15.26:0/2025946193 learned_addr learned my addr 172.21.15.26:0/2025946193 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f509affd640 1 -- 172.21.15.26:0/2025946193 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f509c0cd840 msgr2=0x7f509c0b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f509affd640 1 --2- 172.21.15.26:0/2025946193 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f509c0cd840 0x7f509c0b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f509affd640 1 -- 172.21.15.26:0/2025946193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f509c0cc770 msgr2=0x7f509c0b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f509affd640 1 --2- 172.21.15.26:0/2025946193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f509c0cc770 0x7f509c0b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f509affd640 1 -- 172.21.15.26:0/2025946193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5094009a70 con 0x7f509c0afd70 2023-12-19T16:43:26.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f509affd640 1 --2- 172.21.15.26:0/2025946193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f509c0afd70 0x7f509c0c6ca0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f508c00c450 tx=0x7f508c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f50937fe640 1 -- 172.21.15.26:0/2025946193 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c005db0 con 0x7f509c0afd70 2023-12-19T16:43:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f50937fe640 1 -- 172.21.15.26:0/2025946193 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c007070 con 0x7f509c0afd70 2023-12-19T16:43:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f50a1352640 1 -- 172.21.15.26:0/2025946193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509c15bae0 con 0x7f509c0afd70 2023-12-19T16:43:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f50937fe640 1 -- 172.21.15.26:0/2025946193 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c007400 con 0x7f509c0afd70 2023-12-19T16:43:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f50a1352640 1 -- 172.21.15.26:0/2025946193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c15c0a0 con 0x7f509c0afd70 2023-12-19T16:43:26.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f50a1352640 1 -- 172.21.15.26:0/2025946193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509c074270 con 0x7f509c0afd70 2023-12-19T16:43:26.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.419+0000 7f50937fe640 1 -- 172.21.15.26:0/2025946193 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f508c007560 con 0x7f509c0afd70 2023-12-19T16:43:26.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.423+0000 7f50937fe640 1 --2- 172.21.15.26:0/2025946193 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f507c065a60 0x7f507c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.423+0000 7f50937fe640 1 -- 172.21.15.26:0/2025946193 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f508c082850 con 0x7f509c0afd70 2023-12-19T16:43:26.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.423+0000 7f5093fff640 1 --2- 172.21.15.26:0/2025946193 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f507c065a60 0x7f507c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.423+0000 7f5093fff640 1 --2- 172.21.15.26:0/2025946193 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f507c065a60 0x7f507c067f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5084007fd0 tx=0x7f5084007900 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:26.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.423+0000 7f50937fe640 1 -- 172.21.15.26:0/2025946193 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f508c050170 con 0x7f509c0afd70 2023-12-19T16:43:26.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.583+0000 7f50a1352640 1 -- 172.21.15.26:0/2025946193 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f509c0b27c0 con 0x7f509c0afd70 2023-12-19T16:43:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.587+0000 7f50937fe640 1 -- 172.21.15.26:0/2025946193 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16114 (secure 0 0 0) 0x7f508c017020 con 0x7f509c0afd70 2023-12-19T16:43:26.591 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:26.591 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":36,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:43:25.865727+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:43:24.052043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T16:43:24.856934+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:43:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.587+0000 7f50a1352640 1 -- 172.21.15.26:0/2025946193 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f507c065a60 msgr2=0x7f507c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.587+0000 7f50a1352640 1 --2- 172.21.15.26:0/2025946193 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f507c065a60 0x7f507c067f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5084007fd0 tx=0x7f5084007900 comp rx=0 tx=0).stop 2023-12-19T16:43:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.587+0000 7f50a1352640 1 -- 172.21.15.26:0/2025946193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f509c0afd70 msgr2=0x7f509c0c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.587+0000 7f50a1352640 1 --2- 172.21.15.26:0/2025946193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f509c0afd70 0x7f509c0c6ca0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f508c00c450 tx=0x7f508c00c910 comp rx=0 tx=0).stop 2023-12-19T16:43:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.587+0000 7f50a1352640 1 -- 172.21.15.26:0/2025946193 shutdown_connections 2023-12-19T16:43:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.587+0000 7f50a1352640 1 --2- 172.21.15.26:0/2025946193 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f507c065a60 0x7f507c067f20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.587+0000 7f50a1352640 1 --2- 172.21.15.26:0/2025946193 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f509c0cd840 0x7f509c0b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.587+0000 7f50a1352640 1 --2- 172.21.15.26:0/2025946193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f509c0cc770 0x7f509c0b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.587+0000 7f50a1352640 1 --2- 172.21.15.26:0/2025946193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f509c0afd70 0x7f509c0c6ca0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.587+0000 7f50a1352640 1 -- 172.21.15.26:0/2025946193 >> 172.21.15.26:0/2025946193 conn(0x7f509c06d570 msgr2=0x7f509c0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.587+0000 7f50a1352640 1 -- 172.21.15.26:0/2025946193 shutdown_connections 2023-12-19T16:43:26.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.587+0000 7f50a1352640 1 -- 172.21.15.26:0/2025946193 wait complete. 2023-12-19T16:43:26.612 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T16:43:26.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.751+0000 7f54b07c0640 1 Processor -- start 2023-12-19T16:43:26.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54b07c0640 1 -- start start 2023-12-19T16:43:26.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54b07c0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54a815a9c0 0x7f54a80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54b07c0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f54a815ada0 0x7f54a80b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54b07c0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54a80b46c0 con 0x7f54a806ac80 2023-12-19T16:43:26.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54b07c0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a80cc270 con 0x7f54a815ada0 2023-12-19T16:43:26.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54b07c0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a8070d90 con 0x7f54a815a9c0 2023-12-19T16:43:26.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54ae55c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54a815a9c0 0x7f54a80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54add5b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f54a815ada0 0x7f54a80b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54ae55c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54a815a9c0 0x7f54a80706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:60156/0 (socket says 172.21.15.26:60156) 2023-12-19T16:43:26.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54aed5d640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f54a806ac80 0x7f54a80b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:45180/0 (socket says 172.21.15.26:45180) 2023-12-19T16:43:26.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54ae55c640 1 -- 172.21.15.26:0/1261412450 learned_addr learned my addr 172.21.15.26:0/1261412450 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:26.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54ae55c640 1 -- 172.21.15.26:0/1261412450 >> v2:172.21.15.160:3300/0 conn(0x7f54a815ada0 msgr2=0x7f54a80b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:26.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54ae55c640 1 --2- 172.21.15.26:0/1261412450 >> v2:172.21.15.160:3300/0 conn(0x7f54a815ada0 0x7f54a80b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54ad55a640 1 -- 172.21.15.26:0/1261412450 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397204754 0 0) 0x7f54a80b46c0 con 0x7f54a806ac80 2023-12-19T16:43:26.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54ae55c640 1 -- 172.21.15.26:0/1261412450 >> v1:172.21.15.160:6789/0 conn(0x7f54a806ac80 legacy=0x7f54a80b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.755+0000 7f54ae55c640 1 -- 172.21.15.26:0/1261412450 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a8071070 con 0x7f54a815a9c0 2023-12-19T16:43:26.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.759+0000 7f54ae55c640 1 --2- 172.21.15.26:0/1261412450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54a815a9c0 0x7f54a80706d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009870 tx=0x7f54a401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=602b294062975763 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:26.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.759+0000 7f54ad55a640 1 -- 172.21.15.26:0/1261412450 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a40ab680 con 0x7f54a815a9c0 2023-12-19T16:43:26.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.759+0000 7f54ad55a640 1 -- 172.21.15.26:0/1261412450 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a40ab7e0 con 0x7f54a815a9c0 2023-12-19T16:43:26.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.759+0000 7f54ad55a640 1 -- 172.21.15.26:0/1261412450 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a40abb70 con 0x7f54a815a9c0 2023-12-19T16:43:26.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.759+0000 7f54b07c0640 1 -- 172.21.15.26:0/1261412450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54a815a9c0 msgr2=0x7f54a80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.759+0000 7f54b07c0640 1 --2- 172.21.15.26:0/1261412450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54a815a9c0 0x7f54a80706d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f54a4009870 tx=0x7f54a401c480 comp rx=0 tx=0).stop 2023-12-19T16:43:26.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.759+0000 7f54b07c0640 1 -- 172.21.15.26:0/1261412450 shutdown_connections 2023-12-19T16:43:26.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.759+0000 7f54b07c0640 1 --2- 172.21.15.26:0/1261412450 >> v2:172.21.15.160:3300/0 conn(0x7f54a815ada0 0x7f54a80b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.759+0000 7f54b07c0640 1 --2- 172.21.15.26:0/1261412450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54a815a9c0 0x7f54a80706d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.759+0000 7f54b07c0640 1 -- 172.21.15.26:0/1261412450 >> 172.21.15.26:0/1261412450 conn(0x7f54a80ce730 msgr2=0x7f54a80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:26.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.759+0000 7f54b07c0640 1 -- 172.21.15.26:0/1261412450 shutdown_connections 2023-12-19T16:43:26.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.759+0000 7f54b07c0640 1 -- 172.21.15.26:0/1261412450 wait complete. 2023-12-19T16:43:26.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54b07c0640 1 Processor -- start 2023-12-19T16:43:26.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54b07c0640 1 -- start start 2023-12-19T16:43:26.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54b07c0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54a806ac80 0x7f54a81551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54b07c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54a815a9c0 0x7f54a8155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54b07c0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54a815ada0 0x7f54a8150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54b07c0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a8152aa0 con 0x7f54a815a9c0 2023-12-19T16:43:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54b07c0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a80c7ec0 con 0x7f54a806ac80 2023-12-19T16:43:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54b07c0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a80c81a0 con 0x7f54a815ada0 2023-12-19T16:43:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54add5b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54a815a9c0 0x7f54a8155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54add5b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54a815a9c0 0x7f54a8155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55174/0 (socket says 172.21.15.26:55174) 2023-12-19T16:43:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54add5b640 1 -- 172.21.15.26:0/267580188 learned_addr learned my addr 172.21.15.26:0/267580188 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54ae55c640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54a806ac80 0x7f54a81551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54aed5d640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54a815ada0 0x7f54a8150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54add5b640 1 -- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54a815ada0 msgr2=0x7f54a8150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54add5b640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54a815ada0 0x7f54a8150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54add5b640 1 -- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54a806ac80 msgr2=0x7f54a81551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54add5b640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54a806ac80 0x7f54a81551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54add5b640 1 -- 172.21.15.26:0/267580188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a40094d0 con 0x7f54a815a9c0 2023-12-19T16:43:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54add5b640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54a815a9c0 0x7f54a8155720 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f549800e930 tx=0x7f549800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f549effd640 1 -- 172.21.15.26:0/267580188 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5498009690 con 0x7f54a815a9c0 2023-12-19T16:43:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f549effd640 1 -- 172.21.15.26:0/267580188 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549800ca00 con 0x7f54a815a9c0 2023-12-19T16:43:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f549effd640 1 -- 172.21.15.26:0/267580188 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549800cd70 con 0x7f54a815a9c0 2023-12-19T16:43:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54b07c0640 1 -- 172.21.15.26:0/267580188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a80b1680 con 0x7f54a815a9c0 2023-12-19T16:43:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54b07c0640 1 -- 172.21.15.26:0/267580188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a8151150 con 0x7f54a815a9c0 2023-12-19T16:43:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.763+0000 7f54b07c0640 1 -- 172.21.15.26:0/267580188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a8074270 con 0x7f54a815a9c0 2023-12-19T16:43:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.767+0000 7f549effd640 1 -- 172.21.15.26:0/267580188 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5498013470 con 0x7f54a815a9c0 2023-12-19T16:43:26.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.767+0000 7f549effd640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f548c065a60 0x7f548c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:26.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.767+0000 7f549effd640 1 -- 172.21.15.26:0/267580188 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5498083370 con 0x7f54a815a9c0 2023-12-19T16:43:26.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.767+0000 7f54ae55c640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f548c065a60 0x7f548c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:26.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.767+0000 7f54ae55c640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f548c065a60 0x7f548c067f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f54a40af000 tx=0x7f54a40b6040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:26.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.771+0000 7f549effd640 1 -- 172.21.15.26:0/267580188 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5498050e90 con 0x7f54a815a9c0 2023-12-19T16:43:26.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:26.963+0000 7f54b07c0640 1 -- 172.21.15.26:0/267580188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f54a80cd7b0 con 0x7f54a815a9c0 2023-12-19T16:43:27.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:43:27.524 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:43:27.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:27.879+0000 7f549effd640 1 -- 172.21.15.26:0/267580188 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f54980544b0 con 0x7f54a815a9c0 2023-12-19T16:43:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:27.883+0000 7f54b07c0640 1 -- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f548c065a60 msgr2=0x7f548c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:27.883+0000 7f54b07c0640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f548c065a60 0x7f548c067f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f54a40af000 tx=0x7f54a40b6040 comp rx=0 tx=0).stop 2023-12-19T16:43:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:27.883+0000 7f54b07c0640 1 -- 172.21.15.26:0/267580188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54a815a9c0 msgr2=0x7f54a8155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:27.883+0000 7f54b07c0640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54a815a9c0 0x7f54a8155720 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f549800e930 tx=0x7f549800edf0 comp rx=0 tx=0).stop 2023-12-19T16:43:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:27.883+0000 7f54b07c0640 1 -- 172.21.15.26:0/267580188 shutdown_connections 2023-12-19T16:43:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:27.883+0000 7f54b07c0640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f548c065a60 0x7f548c067f20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:27.883+0000 7f54b07c0640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54a815ada0 0x7f54a8150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:27.883+0000 7f54b07c0640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54a815a9c0 0x7f54a8155720 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:27.883+0000 7f54b07c0640 1 --2- 172.21.15.26:0/267580188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54a806ac80 0x7f54a81551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:27.883+0000 7f54b07c0640 1 -- 172.21.15.26:0/267580188 >> 172.21.15.26:0/267580188 conn(0x7f54a80ce730 msgr2=0x7f54a80b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:27.883+0000 7f54b07c0640 1 -- 172.21.15.26:0/267580188 shutdown_connections 2023-12-19T16:43:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:27.883+0000 7f54b07c0640 1 -- 172.21.15.26:0/267580188 wait complete. 2023-12-19T16:43:27.910 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T16:43:27.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T16:43:28.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.055+0000 7f3601f2c640 1 Processor -- start 2023-12-19T16:43:28.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.055+0000 7f3601f2c640 1 -- start start 2023-12-19T16:43:28.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.055+0000 7f3601f2c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f35fc15a9c0 0x7f35fc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:28.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.055+0000 7f3601f2c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35fc06ac80 0x7f35fc0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:28.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.055+0000 7f3601f2c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35fc070d90 con 0x7f35fc15ada0 2023-12-19T16:43:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.055+0000 7f3601f2c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc0cc270 con 0x7f35fc15a9c0 2023-12-19T16:43:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.055+0000 7f3601f2c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc071070 con 0x7f35fc06ac80 2023-12-19T16:43:28.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.059+0000 7f35fb7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f35fc15a9c0 0x7f35fc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:28.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.059+0000 7f35fbfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35fc06ac80 0x7f35fc0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:28.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.059+0000 7f35fb7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f35fc15a9c0 0x7f35fc0706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:54984/0 (socket says 172.21.15.26:54984) 2023-12-19T16:43:28.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.059+0000 7f35fb7fe640 1 -- 172.21.15.26:0/3082934850 learned_addr learned my addr 172.21.15.26:0/3082934850 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:28.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.059+0000 7f35fbfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35fc06ac80 0x7f35fc0b01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:60182/0 (socket says 172.21.15.26:60182) 2023-12-19T16:43:28.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.059+0000 7f35fbfff640 1 -- 172.21.15.26:0/3082934850 >> v2:172.21.15.160:3300/0 conn(0x7f35fc15a9c0 msgr2=0x7f35fc0706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:28.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.059+0000 7f35fbfff640 1 --2- 172.21.15.26:0/3082934850 >> v2:172.21.15.160:3300/0 conn(0x7f35fc15a9c0 0x7f35fc0706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:28.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.059+0000 7f35fbfff640 1 -- 172.21.15.26:0/3082934850 >> v1:172.21.15.160:6789/0 conn(0x7f35fc15ada0 legacy=0x7f35fc0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:28.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.059+0000 7f35fbfff640 1 -- 172.21.15.26:0/3082934850 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc071350 con 0x7f35fc06ac80 2023-12-19T16:43:28.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.059+0000 7f35fb7fe640 1 --2- 172.21.15.26:0/3082934850 >> v2:172.21.15.160:3300/0 conn(0x7f35fc15a9c0 0x7f35fc0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:28.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.059+0000 7f35fbfff640 1 --2- 172.21.15.26:0/3082934850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35fc06ac80 0x7f35fc0b01b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009d90 tx=0x7f35f401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=820f961b9253c767 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f35fa7fc640 1 -- 172.21.15.26:0/3082934850 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40a5860 con 0x7f35fc06ac80 2023-12-19T16:43:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f35fa7fc640 1 -- 172.21.15.26:0/3082934850 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f40a59c0 con 0x7f35fc06ac80 2023-12-19T16:43:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f35fa7fc640 1 -- 172.21.15.26:0/3082934850 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40a5d50 con 0x7f35fc06ac80 2023-12-19T16:43:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 -- 172.21.15.26:0/3082934850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35fc06ac80 msgr2=0x7f35fc0b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:28.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 --2- 172.21.15.26:0/3082934850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35fc06ac80 0x7f35fc0b01b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009d90 tx=0x7f35f401c890 comp rx=0 tx=0).stop 2023-12-19T16:43:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 -- 172.21.15.26:0/3082934850 shutdown_connections 2023-12-19T16:43:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 --2- 172.21.15.26:0/3082934850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35fc06ac80 0x7f35fc0b01b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 --2- 172.21.15.26:0/3082934850 >> v2:172.21.15.160:3300/0 conn(0x7f35fc15a9c0 0x7f35fc0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 -- 172.21.15.26:0/3082934850 >> 172.21.15.26:0/3082934850 conn(0x7f35fc0ce730 msgr2=0x7f35fc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 -- 172.21.15.26:0/3082934850 shutdown_connections 2023-12-19T16:43:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 -- 172.21.15.26:0/3082934850 wait complete. 2023-12-19T16:43:28.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 Processor -- start 2023-12-19T16:43:28.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 -- start start 2023-12-19T16:43:28.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35fc06ac80 0x7f35fc157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:28.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35fc15a9c0 0x7f35fc153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35fc15ada0 0x7f35fc1fb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0c9700 con 0x7f35fc15ada0 2023-12-19T16:43:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0c9ea0 con 0x7f35fc15a9c0 2023-12-19T16:43:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f3601f2c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc158190 con 0x7f35fc06ac80 2023-12-19T16:43:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f35fbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35fc15ada0 0x7f35fc1fb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.063+0000 7f35fbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35fc15ada0 0x7f35fc1fb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55184/0 (socket says 172.21.15.26:55184) 2023-12-19T16:43:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35fbfff640 1 -- 172.21.15.26:0/2853765021 learned_addr learned my addr 172.21.15.26:0/2853765021 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35fb7fe640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35fc06ac80 0x7f35fc157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35faffd640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35fc15a9c0 0x7f35fc153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35fbfff640 1 -- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35fc06ac80 msgr2=0x7f35fc157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35fbfff640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35fc06ac80 0x7f35fc157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35fbfff640 1 -- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35fc15a9c0 msgr2=0x7f35fc153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35fbfff640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35fc15a9c0 0x7f35fc153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:28.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35fbfff640 1 -- 172.21.15.26:0/2853765021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ec0094d0 con 0x7f35fc15ada0 2023-12-19T16:43:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35fbfff640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35fc15ada0 0x7f35fc1fb280 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f35f40b0040 tx=0x7f35f4019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35e3fff640 1 -- 172.21.15.26:0/2853765021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40b9860 con 0x7f35fc15ada0 2023-12-19T16:43:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35e3fff640 1 -- 172.21.15.26:0/2853765021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f40b99c0 con 0x7f35fc15ada0 2023-12-19T16:43:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f3601f2c640 1 -- 172.21.15.26:0/2853765021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f4009a70 con 0x7f35fc15ada0 2023-12-19T16:43:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35e3fff640 1 -- 172.21.15.26:0/2853765021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40b9d50 con 0x7f35fc15ada0 2023-12-19T16:43:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f3601f2c640 1 -- 172.21.15.26:0/2853765021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc156250 con 0x7f35fc15ada0 2023-12-19T16:43:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f3601f2c640 1 -- 172.21.15.26:0/2853765021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35fc0b3c90 con 0x7f35fc15ada0 2023-12-19T16:43:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35e3fff640 1 -- 172.21.15.26:0/2853765021 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f35f40ad440 con 0x7f35fc15ada0 2023-12-19T16:43:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35e3fff640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35e8065a60 0x7f35e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:28.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.067+0000 7f35e3fff640 1 -- 172.21.15.26:0/2853765021 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35f411e5e0 con 0x7f35fc15ada0 2023-12-19T16:43:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.071+0000 7f35fb7fe640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35e8065a60 0x7f35e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.071+0000 7f35fb7fe640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35e8065a60 0x7f35e8067f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0b2190 tx=0x7f35ec009240 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.075+0000 7f35e3fff640 1 -- 172.21.15.26:0/2853765021 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f35f40ebf90 con 0x7f35fc15ada0 2023-12-19T16:43:28.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.267+0000 7f3601f2c640 1 -- 172.21.15.26:0/2853765021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f35fc074270 con 0x7f35fc15ada0 2023-12-19T16:43:28.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.883+0000 7f35e3fff640 1 -- 172.21.15.26:0/2853765021 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f35f40ef5b0 con 0x7f35fc15ada0 2023-12-19T16:43:28.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.887+0000 7f3601f2c640 1 -- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35e8065a60 msgr2=0x7f35e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:28.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.887+0000 7f3601f2c640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35e8065a60 0x7f35e8067f20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0b2190 tx=0x7f35ec009240 comp rx=0 tx=0).stop 2023-12-19T16:43:28.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.887+0000 7f3601f2c640 1 -- 172.21.15.26:0/2853765021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35fc15ada0 msgr2=0x7f35fc1fb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:28.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.887+0000 7f3601f2c640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35fc15ada0 0x7f35fc1fb280 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f35f40b0040 tx=0x7f35f4019630 comp rx=0 tx=0).stop 2023-12-19T16:43:28.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.887+0000 7f3601f2c640 1 -- 172.21.15.26:0/2853765021 shutdown_connections 2023-12-19T16:43:28.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.887+0000 7f3601f2c640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35fc15ada0 0x7f35fc1fb280 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:28.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.887+0000 7f3601f2c640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35fc15a9c0 0x7f35fc153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.887+0000 7f3601f2c640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35e8065a60 0x7f35e8067f20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.887+0000 7f3601f2c640 1 --2- 172.21.15.26:0/2853765021 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35fc06ac80 0x7f35fc157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.887+0000 7f3601f2c640 1 -- 172.21.15.26:0/2853765021 >> 172.21.15.26:0/2853765021 conn(0x7f35fc0ce730 msgr2=0x7f35fc0ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.887+0000 7f3601f2c640 1 -- 172.21.15.26:0/2853765021 shutdown_connections 2023-12-19T16:43:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:28.887+0000 7f3601f2c640 1 -- 172.21.15.26:0/2853765021 wait complete. 2023-12-19T16:43:28.914 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T16:43:28.914 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:43:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa83722a640 1 Processor -- start 2023-12-19T16:43:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa83722a640 1 -- start start 2023-12-19T16:43:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa83722a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa83015cde0 0x7fa83006b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa83722a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa8300d0300 0x7fa83006b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa83722a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8300ce180 con 0x7fa8300cc810 2023-12-19T16:43:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa83722a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8300ccf20 con 0x7fa8300d0300 2023-12-19T16:43:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa83722a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8300cdea0 con 0x7fa83015cde0 2023-12-19T16:43:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa835a27640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa8300d0300 0x7fa83006b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa835a27640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa8300d0300 0x7fa83006b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55196/0 (socket says 172.21.15.26:55196) 2023-12-19T16:43:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa835a27640 1 -- 172.21.15.26:0/720712364 learned_addr learned my addr 172.21.15.26:0/720712364 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa836228640 1 --2- 172.21.15.26:0/720712364 >> v2:172.21.15.160:3300/0 conn(0x7fa83015cde0 0x7fa83006b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa835a27640 1 -- 172.21.15.26:0/720712364 >> v2:172.21.15.160:3300/0 conn(0x7fa83015cde0 msgr2=0x7fa83006b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa835a27640 1 --2- 172.21.15.26:0/720712364 >> v2:172.21.15.160:3300/0 conn(0x7fa83015cde0 0x7fa83006b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa835a27640 1 -- 172.21.15.26:0/720712364 >> v1:172.21.15.160:6789/0 conn(0x7fa8300cc810 legacy=0x7fa830069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.055+0000 7fa835a27640 1 -- 172.21.15.26:0/720712364 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa830160910 con 0x7fa8300d0300 2023-12-19T16:43:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.059+0000 7fa835a27640 1 --2- 172.21.15.26:0/720712364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8300d0300 0x7fa83006b8d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa830079c10 tx=0x7fa82001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef0b0c8dc0cc626 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.059+0000 7fa835226640 1 -- 172.21.15.26:0/720712364 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8200ab600 con 0x7fa8300d0300 2023-12-19T16:43:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.059+0000 7fa835226640 1 -- 172.21.15.26:0/720712364 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8200a7050 con 0x7fa8300d0300 2023-12-19T16:43:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.059+0000 7fa835226640 1 -- 172.21.15.26:0/720712364 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8200ab980 con 0x7fa8300d0300 2023-12-19T16:43:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.059+0000 7fa83722a640 1 -- 172.21.15.26:0/720712364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8300d0300 msgr2=0x7fa83006b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.059+0000 7fa83722a640 1 --2- 172.21.15.26:0/720712364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8300d0300 0x7fa83006b8d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa830079c10 tx=0x7fa82001c430 comp rx=0 tx=0).stop 2023-12-19T16:43:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.059+0000 7fa83722a640 1 -- 172.21.15.26:0/720712364 shutdown_connections 2023-12-19T16:43:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.059+0000 7fa83722a640 1 --2- 172.21.15.26:0/720712364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8300d0300 0x7fa83006b8d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.059+0000 7fa83722a640 1 --2- 172.21.15.26:0/720712364 >> v2:172.21.15.160:3300/0 conn(0x7fa83015cde0 0x7fa83006b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.059+0000 7fa83722a640 1 -- 172.21.15.26:0/720712364 >> 172.21.15.26:0/720712364 conn(0x7fa8300cd210 msgr2=0x7fa8300cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.059+0000 7fa83722a640 1 -- 172.21.15.26:0/720712364 shutdown_connections 2023-12-19T16:43:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.059+0000 7fa83722a640 1 -- 172.21.15.26:0/720712364 wait complete. 2023-12-19T16:43:29.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.059+0000 7fa83722a640 1 Processor -- start 2023-12-19T16:43:29.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa83722a640 1 -- start start 2023-12-19T16:43:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa83722a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8300cc810 0x7fa830155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa83722a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa8300d0300 0x7fa830146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa83722a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa83015cde0 0x7fa8300b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa83722a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8301470c0 con 0x7fa8300cc810 2023-12-19T16:43:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa83722a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa830154f30 con 0x7fa83015cde0 2023-12-19T16:43:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa83722a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa830155210 con 0x7fa8300d0300 2023-12-19T16:43:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa836228640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8300cc810 0x7fa830155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa836228640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8300cc810 0x7fa830155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55212/0 (socket says 172.21.15.26:55212) 2023-12-19T16:43:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa836228640 1 -- 172.21.15.26:0/1071494323 learned_addr learned my addr 172.21.15.26:0/1071494323 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa835a27640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa8300d0300 0x7fa830146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa836a29640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa83015cde0 0x7fa8300b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa836228640 1 -- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa8300d0300 msgr2=0x7fa830146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa836228640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa8300d0300 0x7fa830146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa836228640 1 -- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa83015cde0 msgr2=0x7fa8300b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa836228640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa83015cde0 0x7fa8300b5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa836228640 1 -- 172.21.15.26:0/1071494323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8200094d0 con 0x7fa8300cc810 2023-12-19T16:43:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa836228640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8300cc810 0x7fa830155fd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00d5f0 tx=0x7fa82c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa826ffd640 1 -- 172.21.15.26:0/1071494323 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0125f0 con 0x7fa8300cc810 2023-12-19T16:43:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa826ffd640 1 -- 172.21.15.26:0/1071494323 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c012750 con 0x7fa8300cc810 2023-12-19T16:43:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa826ffd640 1 -- 172.21.15.26:0/1071494323 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c012ae0 con 0x7fa8300cc810 2023-12-19T16:43:29.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa83722a640 1 -- 172.21.15.26:0/1071494323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8301554f0 con 0x7fa8300cc810 2023-12-19T16:43:29.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa83722a640 1 -- 172.21.15.26:0/1071494323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa830155ab0 con 0x7fa8300cc810 2023-12-19T16:43:29.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.063+0000 7fa83722a640 1 -- 172.21.15.26:0/1071494323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa830067400 con 0x7fa8300cc810 2023-12-19T16:43:29.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.067+0000 7fa826ffd640 1 -- 172.21.15.26:0/1071494323 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa82c012c40 con 0x7fa8300cc810 2023-12-19T16:43:29.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.067+0000 7fa826ffd640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa814065a60 0x7fa814067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.067+0000 7fa826ffd640 1 -- 172.21.15.26:0/1071494323 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa82c083520 con 0x7fa8300cc810 2023-12-19T16:43:29.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.067+0000 7fa835a27640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa814065a60 0x7fa814067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.067+0000 7fa835a27640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa814065a60 0x7fa814067f20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa8200ad000 tx=0x7fa8200a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:29.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.071+0000 7fa826ffd640 1 -- 172.21.15.26:0/1071494323 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa82c050f70 con 0x7fa8300cc810 2023-12-19T16:43:29.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.259+0000 7fa83722a640 1 -- 172.21.15.26:0/1071494323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8300c3c90 con 0x7fa8300cc810 2023-12-19T16:43:29.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.259+0000 7fa826ffd640 1 -- 172.21.15.26:0/1071494323 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4715 (secure 0 0 0) 0x7fa82c054590 con 0x7fa8300cc810 2023-12-19T16:43:29.266 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:29.266 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1991431867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1991431867},{"type":"v1","addr":"172.21.15.160:6837","nonce":1991431867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3495925867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3495925867},{"type":"v1","addr":"172.21.15.160:6835","nonce":3495925867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3757604412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3757604412},{"type":"v1","addr":"172.21.15.26:6833","nonce":3757604412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:43:25.866059+0000","modified":"2023-12-19T16:43:28.880638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2660001678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2660001678},{"type":"v1","addr":"172.21.15.26:6835","nonce":2660001678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T16:43:29.266 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2023-12-19T16:43:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.263+0000 7fa83722a640 1 -- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa814065a60 msgr2=0x7fa814067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.263+0000 7fa83722a640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa814065a60 0x7fa814067f20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa8200ad000 tx=0x7fa8200a6000 comp rx=0 tx=0).stop 2023-12-19T16:43:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.263+0000 7fa83722a640 1 -- 172.21.15.26:0/1071494323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8300cc810 msgr2=0x7fa830155fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.263+0000 7fa83722a640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8300cc810 0x7fa830155fd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00d5f0 tx=0x7fa82c00dab0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.263+0000 7fa83722a640 1 -- 172.21.15.26:0/1071494323 shutdown_connections 2023-12-19T16:43:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.263+0000 7fa83722a640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa83015cde0 0x7fa8300b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.263+0000 7fa83722a640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa814065a60 0x7fa814067f20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.263+0000 7fa83722a640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa8300d0300 0x7fa830146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.263+0000 7fa83722a640 1 --2- 172.21.15.26:0/1071494323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8300cc810 0x7fa830155fd0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.263+0000 7fa83722a640 1 -- 172.21.15.26:0/1071494323 >> 172.21.15.26:0/1071494323 conn(0x7fa8300cd210 msgr2=0x7fa8300d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.263+0000 7fa83722a640 1 -- 172.21.15.26:0/1071494323 shutdown_connections 2023-12-19T16:43:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.263+0000 7fa83722a640 1 -- 172.21.15.26:0/1071494323 wait complete. 2023-12-19T16:43:29.287 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:43:29.298 INFO:tasks.ceph.mds.a.smithi026.stderr:2023-12-19T16:43:29.291+0000 7f1d03e0c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T16:43:29.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.423+0000 7fa6e1739640 1 Processor -- start 2023-12-19T16:43:29.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.423+0000 7fa6e1739640 1 -- start start 2023-12-19T16:43:29.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.427+0000 7fa6e1739640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa6dc15f370 0x7fa6dc15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.427+0000 7fa6e1739640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6dc06c970 con 0x7fa6dc155030 2023-12-19T16:43:29.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.427+0000 7fa6e1739640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6dc15b1f0 con 0x7fa6dc0ce740 2023-12-19T16:43:29.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.427+0000 7fa6e1739640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6dc06c790 con 0x7fa6dc15f370 2023-12-19T16:43:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.427+0000 7fa6d3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa6dc15f370 0x7fa6dc15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.427+0000 7fa6daffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa6dc155030 0x7fa6dc06c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47348/0 (socket says 172.21.15.26:47348) 2023-12-19T16:43:29.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.427+0000 7fa6daffd640 1 -- 172.21.15.26:0/1879865275 learned_addr learned my addr 172.21.15.26:0/1879865275 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:29.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.427+0000 7fa6d3fff640 1 -- 172.21.15.26:0/1879865275 >> v1:172.21.15.160:6789/0 conn(0x7fa6dc0ce740 legacy=0x7fa6dc0ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.427+0000 7fa6d3fff640 1 -- 172.21.15.26:0/1879865275 >> v1:172.21.15.26:6789/0 conn(0x7fa6dc155030 legacy=0x7fa6dc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.427+0000 7fa6d3fff640 1 -- 172.21.15.26:0/1879865275 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc15b4d0 con 0x7fa6dc15f370 2023-12-19T16:43:29.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.427+0000 7fa6da7fc640 1 -- 172.21.15.26:0/1879865275 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282383422 0 0) 0x7fa6dc06c970 con 0x7fa6dc155030 2023-12-19T16:43:29.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6d3fff640 1 --2- 172.21.15.26:0/1879865275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc15f370 0x7fa6dc15bcd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40097c0 tx=0x7fa6c401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4c42d46a4079c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:29.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6da7fc640 1 -- 172.21.15.26:0/1879865275 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40ab670 con 0x7fa6dc15f370 2023-12-19T16:43:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6da7fc640 1 -- 172.21.15.26:0/1879865275 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c40ab7d0 con 0x7fa6dc15f370 2023-12-19T16:43:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6da7fc640 1 -- 172.21.15.26:0/1879865275 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40abb60 con 0x7fa6dc15f370 2023-12-19T16:43:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 -- 172.21.15.26:0/1879865275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc15f370 msgr2=0x7fa6dc15bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 --2- 172.21.15.26:0/1879865275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc15f370 0x7fa6dc15bcd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40097c0 tx=0x7fa6c401c450 comp rx=0 tx=0).stop 2023-12-19T16:43:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 -- 172.21.15.26:0/1879865275 shutdown_connections 2023-12-19T16:43:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 --2- 172.21.15.26:0/1879865275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc15f370 0x7fa6dc15bcd0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 -- 172.21.15.26:0/1879865275 >> 172.21.15.26:0/1879865275 conn(0x7fa6dc0658e0 msgr2=0x7fa6dc065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 -- 172.21.15.26:0/1879865275 shutdown_connections 2023-12-19T16:43:29.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 -- 172.21.15.26:0/1879865275 wait complete. 2023-12-19T16:43:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 Processor -- start 2023-12-19T16:43:29.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 -- start start 2023-12-19T16:43:29.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0ce740 0x7fa6dc14aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6dc155030 0x7fa6dc15bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa6dc15f370 0x7fa6dc151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc15c490 con 0x7fa6dc0ce740 2023-12-19T16:43:29.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc15c770 con 0x7fa6dc155030 2023-12-19T16:43:29.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.431+0000 7fa6e1739640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc15ca50 con 0x7fa6dc15f370 2023-12-19T16:43:29.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6daffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0ce740 0x7fa6dc14aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6daffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0ce740 0x7fa6dc14aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55226/0 (socket says 172.21.15.26:55226) 2023-12-19T16:43:29.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6daffd640 1 -- 172.21.15.26:0/571225948 learned_addr learned my addr 172.21.15.26:0/571225948 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:29.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6d3fff640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6dc155030 0x7fa6dc15bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6db7fe640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa6dc15f370 0x7fa6dc151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6daffd640 1 -- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa6dc15f370 msgr2=0x7fa6dc151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6daffd640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa6dc15f370 0x7fa6dc151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6daffd640 1 -- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6dc155030 msgr2=0x7fa6dc15bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6daffd640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6dc155030 0x7fa6dc15bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6daffd640 1 -- 172.21.15.26:0/571225948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c4009420 con 0x7fa6dc0ce740 2023-12-19T16:43:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6db7fe640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa6dc15f370 0x7fa6dc151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:29.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6daffd640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0ce740 0x7fa6dc14aa90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00c4a0 tx=0x7fa6cc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6d37fe640 1 -- 172.21.15.26:0/571225948 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0074e0 con 0x7fa6dc0ce740 2023-12-19T16:43:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6d37fe640 1 -- 172.21.15.26:0/571225948 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc007640 con 0x7fa6dc0ce740 2023-12-19T16:43:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6d37fe640 1 -- 172.21.15.26:0/571225948 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc004a30 con 0x7fa6dc0ce740 2023-12-19T16:43:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6e1739640 1 -- 172.21.15.26:0/571225948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6dc0cf0e0 con 0x7fa6dc0ce740 2023-12-19T16:43:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6e1739640 1 -- 172.21.15.26:0/571225948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc15d010 con 0x7fa6dc0ce740 2023-12-19T16:43:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6e1739640 1 -- 172.21.15.26:0/571225948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6dc15d200 con 0x7fa6dc0ce740 2023-12-19T16:43:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.435+0000 7fa6d37fe640 1 -- 172.21.15.26:0/571225948 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa6cc004b90 con 0x7fa6dc0ce740 2023-12-19T16:43:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.439+0000 7fa6d37fe640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa6bc065a60 0x7fa6bc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.439+0000 7fa6d37fe640 1 -- 172.21.15.26:0/571225948 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6cc082410 con 0x7fa6dc0ce740 2023-12-19T16:43:29.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.439+0000 7fa6d3fff640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa6bc065a60 0x7fa6bc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.439+0000 7fa6d3fff640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa6bc065a60 0x7fa6bc067f20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40af000 tx=0x7fa6c40b6040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:29.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.443+0000 7fa6d37fe640 1 -- 172.21.15.26:0/571225948 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa6cc04fde0 con 0x7fa6dc0ce740 2023-12-19T16:43:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6e1739640 1 -- 172.21.15.26:0/571225948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6dc074270 con 0x7fa6dc0ce740 2023-12-19T16:43:29.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6d37fe640 1 -- 172.21.15.26:0/571225948 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16101 (secure 0 0 0) 0x7fa6cc018090 con 0x7fa6dc0ce740 2023-12-19T16:43:29.608 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:29.608 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":37,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:43:26.869121+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:43:24.052043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T16:43:24.856934+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:43:29.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6e1739640 1 -- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa6bc065a60 msgr2=0x7fa6bc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6e1739640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa6bc065a60 0x7fa6bc067f20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40af000 tx=0x7fa6c40b6040 comp rx=0 tx=0).stop 2023-12-19T16:43:29.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6e1739640 1 -- 172.21.15.26:0/571225948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0ce740 msgr2=0x7fa6dc14aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6e1739640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0ce740 0x7fa6dc14aa90 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00c4a0 tx=0x7fa6cc00c960 comp rx=0 tx=0).stop 2023-12-19T16:43:29.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6e1739640 1 -- 172.21.15.26:0/571225948 shutdown_connections 2023-12-19T16:43:29.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6e1739640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa6bc065a60 0x7fa6bc067f20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6e1739640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa6dc15f370 0x7fa6dc151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6e1739640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6dc155030 0x7fa6dc15bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6e1739640 1 --2- 172.21.15.26:0/571225948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6dc0ce740 0x7fa6dc14aa90 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6e1739640 1 -- 172.21.15.26:0/571225948 >> 172.21.15.26:0/571225948 conn(0x7fa6dc0658e0 msgr2=0x7fa6dc0ad240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:29.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6e1739640 1 -- 172.21.15.26:0/571225948 shutdown_connections 2023-12-19T16:43:29.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.603+0000 7fa6e1739640 1 -- 172.21.15.26:0/571225948 wait complete. 2023-12-19T16:43:29.631 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T16:43:29.631 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:43:29.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.807+0000 7f8717396640 1 Processor -- start 2023-12-19T16:43:29.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.807+0000 7f8717396640 1 -- start start 2023-12-19T16:43:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.807+0000 7f8717396640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8710155030 0x7f871015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.807+0000 7f8717396640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f871015f370 0x7f871006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.807+0000 7f8717396640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871006c970 con 0x7f87100ce740 2023-12-19T16:43:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.807+0000 7f8717396640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871015b1f0 con 0x7f8710155030 2023-12-19T16:43:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.807+0000 7f8717396640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871015b4d0 con 0x7f871015f370 2023-12-19T16:43:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.807+0000 7f8715933640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f87100ce740 0x7f87100ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47350/0 (socket says 172.21.15.26:47350) 2023-12-19T16:43:29.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.807+0000 7f8715933640 1 -- 172.21.15.26:0/4130551758 learned_addr learned my addr 172.21.15.26:0/4130551758 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.807+0000 7f8715132640 1 --2- 172.21.15.26:0/4130551758 >> v2:172.21.15.160:3300/0 conn(0x7f8710155030 0x7f871015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.807+0000 7f8714931640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f871015f370 0x7f871006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.807+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721796260 0 0) 0x7f871006c970 con 0x7f87100ce740 2023-12-19T16:43:29.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86fc003530 con 0x7f87100ce740 2023-12-19T16:43:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2030806140 0 0) 0x7f86fc003530 con 0x7f87100ce740 2023-12-19T16:43:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f871015f370 msgr2=0x7f871006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f871015f370 0x7f871006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 >> v2:172.21.15.160:3300/0 conn(0x7f8710155030 msgr2=0x7f871015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 --2- 172.21.15.26:0/4130551758 >> v2:172.21.15.160:3300/0 conn(0x7f8710155030 0x7f871015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871015b7b0 con 0x7f87100ce740 2023-12-19T16:43:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f8708001a00 con 0x7f87100ce740 2023-12-19T16:43:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 >> v1:172.21.15.26:6789/0 conn(0x7f87100ce740 legacy=0x7f87100ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86fc003980 0x7f86fc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86fc004370 0x7f86fc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86fc004ec0 0x7f86fc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871015b1f0 con 0x7f86fc004ec0 2023-12-19T16:43:29.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871015b4d0 con 0x7f86fc004370 2023-12-19T16:43:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86fc00bed0 con 0x7f86fc003980 2023-12-19T16:43:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8715132640 1 --2- 172.21.15.26:0/4130551758 >> v2:172.21.15.160:3300/0 conn(0x7f8710155030 0x7f871015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8714931640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f871015f370 0x7f871006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8715933640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86fc004ec0 0x7f86fc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8715132640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86fc003980 0x7f86fc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8714931640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86fc004370 0x7f86fc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8715933640 1 -- 172.21.15.26:0/4130551758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86fc003980 msgr2=0x7f86fc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8715933640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86fc003980 0x7f86fc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8715933640 1 -- 172.21.15.26:0/4130551758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86fc004370 msgr2=0x7f86fc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8715933640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86fc004370 0x7f86fc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8715933640 1 -- 172.21.15.26:0/4130551758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86fc003530 con 0x7f86fc004ec0 2023-12-19T16:43:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8715132640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86fc003980 0x7f86fc003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.811+0000 7f8715933640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86fc004ec0 0x7f86fc00b990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f870800a830 tx=0x7f870801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b7f7121e9e5a6e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87080ad630 con 0x7f86fc004ec0 2023-12-19T16:43:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87080a8070 con 0x7f86fc004ec0 2023-12-19T16:43:29.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8707fff640 1 -- 172.21.15.26:0/4130551758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87080ada30 con 0x7f86fc004ec0 2023-12-19T16:43:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 -- 172.21.15.26:0/4130551758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86fc004ec0 msgr2=0x7f86fc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 --2- 172.21.15.26:0/4130551758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86fc004ec0 0x7f86fc00b990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f870800a830 tx=0x7f870801d360 comp rx=0 tx=0).stop 2023-12-19T16:43:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8715933640 1 -- 172.21.15.26:0/4130551758 reap_dead start 2023-12-19T16:43:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 -- 172.21.15.26:0/4130551758 shutdown_connections 2023-12-19T16:43:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 -- 172.21.15.26:0/4130551758 >> 172.21.15.26:0/4130551758 conn(0x7f87100658e0 msgr2=0x7f8710065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 -- 172.21.15.26:0/4130551758 shutdown_connections 2023-12-19T16:43:29.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 -- 172.21.15.26:0/4130551758 wait complete. 2023-12-19T16:43:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 Processor -- start 2023-12-19T16:43:29.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 -- start start 2023-12-19T16:43:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86fc004ec0 0x7f87100b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87101493c0 0x7f87100b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87100c4450 0x7f87100c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87100c3fc0 con 0x7f86fc004ec0 2023-12-19T16:43:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87100c3700 con 0x7f87100c4450 2023-12-19T16:43:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8717396640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8710070bd0 con 0x7f87101493c0 2023-12-19T16:43:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.815+0000 7f8715132640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86fc004ec0 0x7f87100b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8715132640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86fc004ec0 0x7f87100b2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55248/0 (socket says 172.21.15.26:55248) 2023-12-19T16:43:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8715132640 1 -- 172.21.15.26:0/3463399533 learned_addr learned my addr 172.21.15.26:0/3463399533 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8715132640 1 -- 172.21.15.26:0/3463399533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87101493c0 msgr2=0x7f87100b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8715132640 1 --2- 172.21.15.26:0/3463399533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87101493c0 0x7f87100b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8715933640 1 --2- 172.21.15.26:0/3463399533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87100c4450 0x7f87100c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8715132640 1 -- 172.21.15.26:0/3463399533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87100c4450 msgr2=0x7f87100c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8715132640 1 --2- 172.21.15.26:0/3463399533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87100c4450 0x7f87100c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8715132640 1 -- 172.21.15.26:0/3463399533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87100c3970 con 0x7f86fc004ec0 2023-12-19T16:43:29.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8715132640 1 --2- 172.21.15.26:0/3463399533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86fc004ec0 0x7f87100b2a80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f870c0118d0 tx=0x7f870c011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8705ffb640 1 -- 172.21.15.26:0/3463399533 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c00fbd0 con 0x7f86fc004ec0 2023-12-19T16:43:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8705ffb640 1 -- 172.21.15.26:0/3463399533 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870c00fd30 con 0x7f86fc004ec0 2023-12-19T16:43:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8705ffb640 1 -- 172.21.15.26:0/3463399533 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c015680 con 0x7f86fc004ec0 2023-12-19T16:43:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8717396640 1 -- 172.21.15.26:0/3463399533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87100c3c50 con 0x7f86fc004ec0 2023-12-19T16:43:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8717396640 1 -- 172.21.15.26:0/3463399533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87100c4e10 con 0x7f86fc004ec0 2023-12-19T16:43:29.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8717396640 1 -- 172.21.15.26:0/3463399533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8710074270 con 0x7f86fc004ec0 2023-12-19T16:43:29.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.819+0000 7f8705ffb640 1 -- 172.21.15.26:0/3463399533 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f870c0157e0 con 0x7f86fc004ec0 2023-12-19T16:43:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.823+0000 7f8705ffb640 1 --2- 172.21.15.26:0/3463399533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f86f4065a60 0x7f86f4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.823+0000 7f8705ffb640 1 -- 172.21.15.26:0/3463399533 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f870c083cd0 con 0x7f86fc004ec0 2023-12-19T16:43:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.823+0000 7f8714931640 1 --2- 172.21.15.26:0/3463399533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f86f4065a60 0x7f86f4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:29.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.823+0000 7f8714931640 1 --2- 172.21.15.26:0/3463399533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f86f4065a60 0x7f86f4067f20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f87100cb140 tx=0x7f8700008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:29.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.827+0000 7f8705ffb640 1 -- 172.21.15.26:0/3463399533 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f870c0516b0 con 0x7f86fc004ec0 2023-12-19T16:43:29.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.983+0000 7f8717396640 1 -- 172.21.15.26:0/3463399533 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f871006cc50 con 0x7f86f4065a60 2023-12-19T16:43:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.995+0000 7f8705ffb640 1 -- 172.21.15.26:0/3463399533 <== mgr.4106 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278639 (secure 0 0 0) 0x7f87000117b0 con 0x7f86f4065a60 2023-12-19T16:43:30.003 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:43:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.999+0000 7f8717396640 1 -- 172.21.15.26:0/3463399533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f86f4065a60 msgr2=0x7f86f4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.999+0000 7f8717396640 1 --2- 172.21.15.26:0/3463399533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f86f4065a60 0x7f86f4067f20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f87100cb140 tx=0x7f8700008040 comp rx=0 tx=0).stop 2023-12-19T16:43:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.999+0000 7f8717396640 1 -- 172.21.15.26:0/3463399533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86fc004ec0 msgr2=0x7f87100b2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.999+0000 7f8717396640 1 --2- 172.21.15.26:0/3463399533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86fc004ec0 0x7f87100b2a80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f870c0118d0 tx=0x7f870c011d90 comp rx=0 tx=0).stop 2023-12-19T16:43:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.999+0000 7f8717396640 1 -- 172.21.15.26:0/3463399533 shutdown_connections 2023-12-19T16:43:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.999+0000 7f8717396640 1 --2- 172.21.15.26:0/3463399533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87100c4450 0x7f87100c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.999+0000 7f8717396640 1 --2- 172.21.15.26:0/3463399533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f86f4065a60 0x7f86f4067f20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.999+0000 7f8717396640 1 --2- 172.21.15.26:0/3463399533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87101493c0 0x7f87100b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.999+0000 7f8717396640 1 --2- 172.21.15.26:0/3463399533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86fc004ec0 0x7f87100b2a80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.999+0000 7f8717396640 1 -- 172.21.15.26:0/3463399533 >> 172.21.15.26:0/3463399533 conn(0x7f87100658e0 msgr2=0x7f87101545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:30.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.999+0000 7f8717396640 1 -- 172.21.15.26:0/3463399533 shutdown_connections 2023-12-19T16:43:30.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:29.999+0000 7f8717396640 1 -- 172.21.15.26:0/3463399533 wait complete. 2023-12-19T16:43:30.023 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2023-12-19T16:43:29.650113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":74,"acting":74,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236056,"kb_used_data":4056,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738664,"statfs":{"total":773094113280,"available":772852391936,"internally_reserved":0,"allocated":4153344,"data_stored":2299951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.194586"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856487+0000","last_change":"2023-12-19T16:43:25.892565+0000","last_active":"2023-12-19T16:43:27.856487+0000","last_peered":"2023-12-19T16:43:27.856487+0000","last_clean":"2023-12-19T16:43:27.856487+0000","last_became_active":"2023-12-19T16:43:25.892189+0000","last_became_peered":"2023-12-19T16:43:25.892189+0000","last_unstale":"2023-12-19T16:43:27.856487+0000","last_undegraded":"2023-12-19T16:43:27.856487+0000","last_fullsized":"2023-12-19T16:43:27.856487+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:20.383856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857570+0000","last_change":"2023-12-19T16:43:25.893897+0000","last_active":"2023-12-19T16:43:27.857570+0000","last_peered":"2023-12-19T16:43:27.857570+0000","last_clean":"2023-12-19T16:43:27.857570+0000","last_became_active":"2023-12-19T16:43:25.893654+0000","last_became_peered":"2023-12-19T16:43:25.893654+0000","last_unstale":"2023-12-19T16:43:27.857570+0000","last_undegraded":"2023-12-19T16:43:27.857570+0000","last_fullsized":"2023-12-19T16:43:27.857570+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:51:10.988939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463472+0000","last_change":"2023-12-19T16:43:24.463472+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463472+0000","last_undegraded":"2023-12-19T16:43:24.463472+0000","last_fullsized":"2023-12-19T16:43:24.463472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:39.252728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857152+0000","last_change":"2023-12-19T16:43:25.482086+0000","last_active":"2023-12-19T16:43:27.857152+0000","last_peered":"2023-12-19T16:43:27.857152+0000","last_clean":"2023-12-19T16:43:27.857152+0000","last_became_active":"2023-12-19T16:43:25.481816+0000","last_became_peered":"2023-12-19T16:43:25.481816+0000","last_unstale":"2023-12-19T16:43:27.857152+0000","last_undegraded":"2023-12-19T16:43:27.857152+0000","last_fullsized":"2023-12-19T16:43:27.857152+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:10.136641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463773+0000","last_change":"2023-12-19T16:43:24.463773+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463773+0000","last_undegraded":"2023-12-19T16:43:24.463773+0000","last_fullsized":"2023-12-19T16:43:24.463773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:38.921262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463938+0000","last_change":"2023-12-19T16:43:24.463938+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463938+0000","last_undegraded":"2023-12-19T16:43:24.463938+0000","last_fullsized":"2023-12-19T16:43:24.463938+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:52:24.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460161+0000","last_change":"2023-12-19T16:43:24.460161+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460161+0000","last_undegraded":"2023-12-19T16:43:24.460161+0000","last_fullsized":"2023-12-19T16:43:24.460161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:55:53.394261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458485+0000","last_change":"2023-12-19T16:43:24.458485+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458485+0000","last_undegraded":"2023-12-19T16:43:24.458485+0000","last_fullsized":"2023-12-19T16:43:24.458485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:47:45.880613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462394+0000","last_change":"2023-12-19T16:43:24.462394+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462394+0000","last_undegraded":"2023-12-19T16:43:24.462394+0000","last_fullsized":"2023-12-19T16:43:24.462394+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:49:45.049214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458111+0000","last_change":"2023-12-19T16:43:24.458111+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458111+0000","last_undegraded":"2023-12-19T16:43:24.458111+0000","last_fullsized":"2023-12-19T16:43:24.458111+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:08:40.828978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462419+0000","last_change":"2023-12-19T16:43:24.462419+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462419+0000","last_undegraded":"2023-12-19T16:43:24.462419+0000","last_fullsized":"2023-12-19T16:43:24.462419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:34:42.254411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857236+0000","last_change":"2023-12-19T16:43:25.485279+0000","last_active":"2023-12-19T16:43:27.857236+0000","last_peered":"2023-12-19T16:43:27.857236+0000","last_clean":"2023-12-19T16:43:27.857236+0000","last_became_active":"2023-12-19T16:43:25.484950+0000","last_became_peered":"2023-12-19T16:43:25.484950+0000","last_unstale":"2023-12-19T16:43:27.857236+0000","last_undegraded":"2023-12-19T16:43:27.857236+0000","last_fullsized":"2023-12-19T16:43:27.857236+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:30.752537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458128+0000","last_change":"2023-12-19T16:43:24.458128+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458128+0000","last_undegraded":"2023-12-19T16:43:24.458128+0000","last_fullsized":"2023-12-19T16:43:24.458128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:38.921262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462234+0000","last_change":"2023-12-19T16:43:24.462234+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462234+0000","last_undegraded":"2023-12-19T16:43:24.462234+0000","last_fullsized":"2023-12-19T16:43:24.462234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:22:22.341913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458845+0000","last_change":"2023-12-19T16:43:24.458845+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458845+0000","last_undegraded":"2023-12-19T16:43:24.458845+0000","last_fullsized":"2023-12-19T16:43:24.458845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:03.546398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461365+0000","last_change":"2023-12-19T16:43:24.461365+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461365+0000","last_undegraded":"2023-12-19T16:43:24.461365+0000","last_fullsized":"2023-12-19T16:43:24.461365+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:54.432676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461815+0000","last_change":"2023-12-19T16:43:24.461815+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461815+0000","last_undegraded":"2023-12-19T16:43:24.461815+0000","last_fullsized":"2023-12-19T16:43:24.461815+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:08.723483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463008+0000","last_change":"2023-12-19T16:43:24.463008+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463008+0000","last_undegraded":"2023-12-19T16:43:24.463008+0000","last_fullsized":"2023-12-19T16:43:24.463008+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:39:45.423376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462583+0000","last_change":"2023-12-19T16:43:24.462583+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462583+0000","last_undegraded":"2023-12-19T16:43:24.462583+0000","last_fullsized":"2023-12-19T16:43:24.462583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:08.723483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460039+0000","last_change":"2023-12-19T16:43:24.460039+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460039+0000","last_undegraded":"2023-12-19T16:43:24.460039+0000","last_fullsized":"2023-12-19T16:43:24.460039+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:50:19.100332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459813+0000","last_change":"2023-12-19T16:43:24.459813+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459813+0000","last_undegraded":"2023-12-19T16:43:24.459813+0000","last_fullsized":"2023-12-19T16:43:24.459813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:43:43.874824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459181+0000","last_change":"2023-12-19T16:43:24.459181+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459181+0000","last_undegraded":"2023-12-19T16:43:24.459181+0000","last_fullsized":"2023-12-19T16:43:24.459181+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:30:19.111754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856897+0000","last_change":"2023-12-19T16:43:25.486440+0000","last_active":"2023-12-19T16:43:27.856897+0000","last_peered":"2023-12-19T16:43:27.856897+0000","last_clean":"2023-12-19T16:43:27.856897+0000","last_became_active":"2023-12-19T16:43:25.486112+0000","last_became_peered":"2023-12-19T16:43:25.486112+0000","last_unstale":"2023-12-19T16:43:27.856897+0000","last_undegraded":"2023-12-19T16:43:27.856897+0000","last_fullsized":"2023-12-19T16:43:27.856897+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:47:36.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463128+0000","last_change":"2023-12-19T16:43:24.463128+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463128+0000","last_undegraded":"2023-12-19T16:43:24.463128+0000","last_fullsized":"2023-12-19T16:43:24.463128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:05.966306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459816+0000","last_change":"2023-12-19T16:43:24.459816+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459816+0000","last_undegraded":"2023-12-19T16:43:24.459816+0000","last_fullsized":"2023-12-19T16:43:24.459816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:30:57.984641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458115+0000","last_change":"2023-12-19T16:43:24.458115+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458115+0000","last_undegraded":"2023-12-19T16:43:24.458115+0000","last_fullsized":"2023-12-19T16:43:24.458115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:22:22.341913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461943+0000","last_change":"2023-12-19T16:43:24.461943+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461943+0000","last_undegraded":"2023-12-19T16:43:24.461943+0000","last_fullsized":"2023-12-19T16:43:24.461943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:48:55.560180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857319+0000","last_change":"2023-12-19T16:43:25.486390+0000","last_active":"2023-12-19T16:43:27.857319+0000","last_peered":"2023-12-19T16:43:27.857319+0000","last_clean":"2023-12-19T16:43:27.857319+0000","last_became_active":"2023-12-19T16:43:25.486082+0000","last_became_peered":"2023-12-19T16:43:25.486082+0000","last_unstale":"2023-12-19T16:43:27.857319+0000","last_undegraded":"2023-12-19T16:43:27.857319+0000","last_fullsized":"2023-12-19T16:43:27.857319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:03:37.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856659+0000","last_change":"2023-12-19T16:43:25.908732+0000","last_active":"2023-12-19T16:43:27.856659+0000","last_peered":"2023-12-19T16:43:27.856659+0000","last_clean":"2023-12-19T16:43:27.856659+0000","last_became_active":"2023-12-19T16:43:25.901839+0000","last_became_peered":"2023-12-19T16:43:25.901839+0000","last_unstale":"2023-12-19T16:43:27.856659+0000","last_undegraded":"2023-12-19T16:43:27.856659+0000","last_fullsized":"2023-12-19T16:43:27.856659+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:20.393474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460656+0000","last_change":"2023-12-19T16:43:24.460656+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460656+0000","last_undegraded":"2023-12-19T16:43:24.460656+0000","last_fullsized":"2023-12-19T16:43:24.460656+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:01.953049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856999+0000","last_change":"2023-12-19T16:43:25.485176+0000","last_active":"2023-12-19T16:43:27.856999+0000","last_peered":"2023-12-19T16:43:27.856999+0000","last_clean":"2023-12-19T16:43:27.856999+0000","last_became_active":"2023-12-19T16:43:25.484842+0000","last_became_peered":"2023-12-19T16:43:25.484842+0000","last_unstale":"2023-12-19T16:43:27.856999+0000","last_undegraded":"2023-12-19T16:43:27.856999+0000","last_fullsized":"2023-12-19T16:43:27.856999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:49.582698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857140+0000","last_change":"2023-12-19T16:43:25.905903+0000","last_active":"2023-12-19T16:43:27.857140+0000","last_peered":"2023-12-19T16:43:27.857140+0000","last_clean":"2023-12-19T16:43:27.857140+0000","last_became_active":"2023-12-19T16:43:25.897356+0000","last_became_peered":"2023-12-19T16:43:25.897356+0000","last_unstale":"2023-12-19T16:43:27.857140+0000","last_undegraded":"2023-12-19T16:43:27.857140+0000","last_fullsized":"2023-12-19T16:43:27.857140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:36:39.824632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460764+0000","last_change":"2023-12-19T16:43:24.460764+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460764+0000","last_undegraded":"2023-12-19T16:43:24.460764+0000","last_fullsized":"2023-12-19T16:43:24.460764+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:45.708612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459149+0000","last_change":"2023-12-19T16:43:24.459149+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459149+0000","last_undegraded":"2023-12-19T16:43:24.459149+0000","last_fullsized":"2023-12-19T16:43:24.459149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:50:38.846004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461667+0000","last_change":"2023-12-19T16:43:24.461667+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461667+0000","last_undegraded":"2023-12-19T16:43:24.461667+0000","last_fullsized":"2023-12-19T16:43:24.461667+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:01.953049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463302+0000","last_change":"2023-12-19T16:43:24.463302+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463302+0000","last_undegraded":"2023-12-19T16:43:24.463302+0000","last_fullsized":"2023-12-19T16:43:24.463302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:24:17.651413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":173,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-19T16:43:21.144188+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:43:21.144188+0000","last_peered":"2023-12-19T16:43:21.144188+0000","last_clean":"2023-12-19T16:43:21.144188+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:43:21.144188+0000","last_undegraded":"2023-12-19T16:43:21.144188+0000","last_fullsized":"2023-12-19T16:43:21.144188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":958464,"data_stored":727125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.44600000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.53300000000000003}]},{"osd":3,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68200000000000005}]},{"osd":6,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.626},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.65300000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.623}]},{"osd":1,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.3799999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.59699999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.68000000000000005}]},{"osd":3,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.673,"5min":4.673,"15min":4.673},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":0.70799999999999996}]},{"osd":4,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90900000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.92700000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.94699999999999995}]}]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86199999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.83699999999999997}]},{"osd":2,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.628}]},{"osd":3,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.60499999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80100000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.73899999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.71799999999999997}]},{"osd":2,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.621}]},{"osd":3,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.748}]},{"osd":5,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.83199999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85099999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.98399999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.496},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.439}]},{"osd":1,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.45200000000000001}]},{"osd":4,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.78800000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":0.753}]},{"osd":7,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":5.0759999999999996,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.94999999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.5150000000000001,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.45900000000000002}]},{"osd":1,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.7590000000000003,"5min":5.7590000000000003,"15min":5.7590000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":5.8070000000000004,"5min":5.8070000000000004,"15min":5.8070000000000004},"last":0.439}]},{"osd":3,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.7599999999999998,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":5.7190000000000003,"5min":5.7190000000000003,"15min":5.7190000000000003},"last":0.48699999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.73299999999999998}]},{"osd":5,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.878},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.82899999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":0.70299999999999996}]},{"osd":7,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.85999999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.133,"5min":4.133,"15min":4.133},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.437}]},{"osd":4,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.72899999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":1007616,"data_stored":752284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.255},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.35099999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.59899999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47399999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.64300000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.627}]},{"osd":7,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.61299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:43:30.029 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:43:30.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.203+0000 7f733ec41640 1 Processor -- start 2023-12-19T16:43:30.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.203+0000 7f733ec41640 1 -- start start 2023-12-19T16:43:30.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.203+0000 7f733ec41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f733806c7f0 0x7f73380cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.203+0000 7f733ec41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73380ccc20 0x7f73380b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.203+0000 7f733ec41640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733806cc80 con 0x7f733815c7f0 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.203+0000 7f733ec41640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733806d770 con 0x7f733806c7f0 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.203+0000 7f733ec41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733806ced0 con 0x7f73380ccc20 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.203+0000 7f733c9dd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f733815c7f0 0x7f73380adb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47362/0 (socket says 172.21.15.26:47362) 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.203+0000 7f733c9dd640 1 -- 172.21.15.26:0/2325440297 learned_addr learned my addr 172.21.15.26:0/2325440297 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.203+0000 7f7337fff640 1 --2- 172.21.15.26:0/2325440297 >> v2:172.21.15.26:3300/0 conn(0x7f733806c7f0 0x7f73380cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.203+0000 7f733d1de640 1 --2- 172.21.15.26:0/2325440297 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73380ccc20 0x7f73380b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.203+0000 7f73377fe640 1 -- 172.21.15.26:0/2325440297 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632650199 0 0) 0x7f733806cc80 con 0x7f733815c7f0 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f73377fe640 1 -- 172.21.15.26:0/2325440297 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7328003530 con 0x7f733815c7f0 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f7337fff640 1 -- 172.21.15.26:0/2325440297 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73380ccc20 msgr2=0x7f73380b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f7337fff640 1 --2- 172.21.15.26:0/2325440297 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73380ccc20 0x7f73380b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f7337fff640 1 -- 172.21.15.26:0/2325440297 >> v1:172.21.15.26:6789/0 conn(0x7f733815c7f0 legacy=0x7f73380adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f7337fff640 1 -- 172.21.15.26:0/2325440297 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733806d1b0 con 0x7f733806c7f0 2023-12-19T16:43:30.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f7337fff640 1 --2- 172.21.15.26:0/2325440297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733806c7f0 0x7f73380cc6e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f73240097c0 tx=0x7f732401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957265b53d1fabce server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:30.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f73377fe640 1 -- 172.21.15.26:0/2325440297 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73240ab600 con 0x7f733806c7f0 2023-12-19T16:43:30.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f73377fe640 1 -- 172.21.15.26:0/2325440297 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73240a7050 con 0x7f733806c7f0 2023-12-19T16:43:30.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f73377fe640 1 -- 172.21.15.26:0/2325440297 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73240ab980 con 0x7f733806c7f0 2023-12-19T16:43:30.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f733ec41640 1 -- 172.21.15.26:0/2325440297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733806c7f0 msgr2=0x7f73380cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f733ec41640 1 --2- 172.21.15.26:0/2325440297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733806c7f0 0x7f73380cc6e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f73240097c0 tx=0x7f732401c480 comp rx=0 tx=0).stop 2023-12-19T16:43:30.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f733ec41640 1 -- 172.21.15.26:0/2325440297 shutdown_connections 2023-12-19T16:43:30.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f733ec41640 1 --2- 172.21.15.26:0/2325440297 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73380ccc20 0x7f73380b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f733ec41640 1 --2- 172.21.15.26:0/2325440297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733806c7f0 0x7f73380cc6e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f733ec41640 1 -- 172.21.15.26:0/2325440297 >> 172.21.15.26:0/2325440297 conn(0x7f73380658e0 msgr2=0x7f7338065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:30.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.207+0000 7f733ec41640 1 -- 172.21.15.26:0/2325440297 shutdown_connections 2023-12-19T16:43:30.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733ec41640 1 -- 172.21.15.26:0/2325440297 wait complete. 2023-12-19T16:43:30.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733ec41640 1 Processor -- start 2023-12-19T16:43:30.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733ec41640 1 -- start start 2023-12-19T16:43:30.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733ec41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733806c7f0 0x7f733813e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733ec41640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73380ccc20 0x7f7338165e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733ec41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f733815c7f0 0x7f7338154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733ec41640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73380c8de0 con 0x7f733806c7f0 2023-12-19T16:43:30.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733ec41640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73381663a0 con 0x7f73380ccc20 2023-12-19T16:43:30.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733ec41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73380c8160 con 0x7f733815c7f0 2023-12-19T16:43:30.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733c9dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733806c7f0 0x7f733813e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733c9dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733806c7f0 0x7f733813e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55270/0 (socket says 172.21.15.26:55270) 2023-12-19T16:43:30.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733c9dd640 1 -- 172.21.15.26:0/604568730 learned_addr learned my addr 172.21.15.26:0/604568730 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:30.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733d1de640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f733815c7f0 0x7f7338154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f7337fff640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73380ccc20 0x7f7338165e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733c9dd640 1 -- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f733815c7f0 msgr2=0x7f7338154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733c9dd640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f733815c7f0 0x7f7338154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733c9dd640 1 -- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73380ccc20 msgr2=0x7f7338165e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733c9dd640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73380ccc20 0x7f7338165e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733c9dd640 1 -- 172.21.15.26:0/604568730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7324009420 con 0x7f733806c7f0 2023-12-19T16:43:30.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.211+0000 7f733c9dd640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733806c7f0 0x7f733813e380 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f732c00c450 tx=0x7f732c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.215+0000 7f73357fa640 1 -- 172.21.15.26:0/604568730 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c005cf0 con 0x7f733806c7f0 2023-12-19T16:43:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.215+0000 7f73357fa640 1 -- 172.21.15.26:0/604568730 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c005e50 con 0x7f733806c7f0 2023-12-19T16:43:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.215+0000 7f73357fa640 1 -- 172.21.15.26:0/604568730 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c007270 con 0x7f733806c7f0 2023-12-19T16:43:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.215+0000 7f733ec41640 1 -- 172.21.15.26:0/604568730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73380c8440 con 0x7f733806c7f0 2023-12-19T16:43:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.215+0000 7f733ec41640 1 -- 172.21.15.26:0/604568730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73380c8a00 con 0x7f733806c7f0 2023-12-19T16:43:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.215+0000 7f733ec41640 1 -- 172.21.15.26:0/604568730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7338074270 con 0x7f733806c7f0 2023-12-19T16:43:30.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.215+0000 7f73357fa640 1 -- 172.21.15.26:0/604568730 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f732c0073d0 con 0x7f733806c7f0 2023-12-19T16:43:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.215+0000 7f73357fa640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f731c065a60 0x7f731c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.215+0000 7f73357fa640 1 -- 172.21.15.26:0/604568730 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f732c0824f0 con 0x7f733806c7f0 2023-12-19T16:43:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.215+0000 7f7337fff640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f731c065a60 0x7f731c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.219+0000 7f7337fff640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f731c065a60 0x7f731c067f20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f73240ad000 tx=0x7f7324008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.219+0000 7f73357fa640 1 -- 172.21.15.26:0/604568730 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f732c04fff0 con 0x7f733806c7f0 2023-12-19T16:43:30.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.371+0000 7f733ec41640 1 -- 172.21.15.26:0/604568730 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7338070fd0 con 0x7f731c065a60 2023-12-19T16:43:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.379+0000 7f73357fa640 1 -- 172.21.15.26:0/604568730 <== mgr.4106 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278639 (secure 0 0 0) 0x7f7338070fd0 con 0x7f731c065a60 2023-12-19T16:43:30.388 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:43:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.383+0000 7f733ec41640 1 -- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f731c065a60 msgr2=0x7f731c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.383+0000 7f733ec41640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f731c065a60 0x7f731c067f20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f73240ad000 tx=0x7f7324008040 comp rx=0 tx=0).stop 2023-12-19T16:43:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.383+0000 7f733ec41640 1 -- 172.21.15.26:0/604568730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733806c7f0 msgr2=0x7f733813e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.383+0000 7f733ec41640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733806c7f0 0x7f733813e380 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f732c00c450 tx=0x7f732c00c910 comp rx=0 tx=0).stop 2023-12-19T16:43:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.383+0000 7f733ec41640 1 -- 172.21.15.26:0/604568730 shutdown_connections 2023-12-19T16:43:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.383+0000 7f733ec41640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f731c065a60 0x7f731c067f20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.383+0000 7f733ec41640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f733815c7f0 0x7f7338154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.383+0000 7f733ec41640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73380ccc20 0x7f7338165e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.383+0000 7f733ec41640 1 --2- 172.21.15.26:0/604568730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f733806c7f0 0x7f733813e380 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.383+0000 7f733ec41640 1 -- 172.21.15.26:0/604568730 >> 172.21.15.26:0/604568730 conn(0x7f73380658e0 msgr2=0x7f733806f580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.383+0000 7f733ec41640 1 -- 172.21.15.26:0/604568730 shutdown_connections 2023-12-19T16:43:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.383+0000 7f733ec41640 1 -- 172.21.15.26:0/604568730 wait complete. 2023-12-19T16:43:30.409 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2023-12-19T16:43:29.650113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":74,"acting":74,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236056,"kb_used_data":4056,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738664,"statfs":{"total":773094113280,"available":772852391936,"internally_reserved":0,"allocated":4153344,"data_stored":2299951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.194586"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856487+0000","last_change":"2023-12-19T16:43:25.892565+0000","last_active":"2023-12-19T16:43:27.856487+0000","last_peered":"2023-12-19T16:43:27.856487+0000","last_clean":"2023-12-19T16:43:27.856487+0000","last_became_active":"2023-12-19T16:43:25.892189+0000","last_became_peered":"2023-12-19T16:43:25.892189+0000","last_unstale":"2023-12-19T16:43:27.856487+0000","last_undegraded":"2023-12-19T16:43:27.856487+0000","last_fullsized":"2023-12-19T16:43:27.856487+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:20.383856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857570+0000","last_change":"2023-12-19T16:43:25.893897+0000","last_active":"2023-12-19T16:43:27.857570+0000","last_peered":"2023-12-19T16:43:27.857570+0000","last_clean":"2023-12-19T16:43:27.857570+0000","last_became_active":"2023-12-19T16:43:25.893654+0000","last_became_peered":"2023-12-19T16:43:25.893654+0000","last_unstale":"2023-12-19T16:43:27.857570+0000","last_undegraded":"2023-12-19T16:43:27.857570+0000","last_fullsized":"2023-12-19T16:43:27.857570+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:51:10.988939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463472+0000","last_change":"2023-12-19T16:43:24.463472+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463472+0000","last_undegraded":"2023-12-19T16:43:24.463472+0000","last_fullsized":"2023-12-19T16:43:24.463472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:39.252728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857152+0000","last_change":"2023-12-19T16:43:25.482086+0000","last_active":"2023-12-19T16:43:27.857152+0000","last_peered":"2023-12-19T16:43:27.857152+0000","last_clean":"2023-12-19T16:43:27.857152+0000","last_became_active":"2023-12-19T16:43:25.481816+0000","last_became_peered":"2023-12-19T16:43:25.481816+0000","last_unstale":"2023-12-19T16:43:27.857152+0000","last_undegraded":"2023-12-19T16:43:27.857152+0000","last_fullsized":"2023-12-19T16:43:27.857152+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:10.136641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463773+0000","last_change":"2023-12-19T16:43:24.463773+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463773+0000","last_undegraded":"2023-12-19T16:43:24.463773+0000","last_fullsized":"2023-12-19T16:43:24.463773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:38.921262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463938+0000","last_change":"2023-12-19T16:43:24.463938+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463938+0000","last_undegraded":"2023-12-19T16:43:24.463938+0000","last_fullsized":"2023-12-19T16:43:24.463938+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:52:24.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460161+0000","last_change":"2023-12-19T16:43:24.460161+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460161+0000","last_undegraded":"2023-12-19T16:43:24.460161+0000","last_fullsized":"2023-12-19T16:43:24.460161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:55:53.394261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458485+0000","last_change":"2023-12-19T16:43:24.458485+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458485+0000","last_undegraded":"2023-12-19T16:43:24.458485+0000","last_fullsized":"2023-12-19T16:43:24.458485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:47:45.880613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462394+0000","last_change":"2023-12-19T16:43:24.462394+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462394+0000","last_undegraded":"2023-12-19T16:43:24.462394+0000","last_fullsized":"2023-12-19T16:43:24.462394+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:49:45.049214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458111+0000","last_change":"2023-12-19T16:43:24.458111+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458111+0000","last_undegraded":"2023-12-19T16:43:24.458111+0000","last_fullsized":"2023-12-19T16:43:24.458111+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:08:40.828978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462419+0000","last_change":"2023-12-19T16:43:24.462419+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462419+0000","last_undegraded":"2023-12-19T16:43:24.462419+0000","last_fullsized":"2023-12-19T16:43:24.462419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:34:42.254411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857236+0000","last_change":"2023-12-19T16:43:25.485279+0000","last_active":"2023-12-19T16:43:27.857236+0000","last_peered":"2023-12-19T16:43:27.857236+0000","last_clean":"2023-12-19T16:43:27.857236+0000","last_became_active":"2023-12-19T16:43:25.484950+0000","last_became_peered":"2023-12-19T16:43:25.484950+0000","last_unstale":"2023-12-19T16:43:27.857236+0000","last_undegraded":"2023-12-19T16:43:27.857236+0000","last_fullsized":"2023-12-19T16:43:27.857236+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:30.752537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458128+0000","last_change":"2023-12-19T16:43:24.458128+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458128+0000","last_undegraded":"2023-12-19T16:43:24.458128+0000","last_fullsized":"2023-12-19T16:43:24.458128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:38.921262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462234+0000","last_change":"2023-12-19T16:43:24.462234+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462234+0000","last_undegraded":"2023-12-19T16:43:24.462234+0000","last_fullsized":"2023-12-19T16:43:24.462234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:22:22.341913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458845+0000","last_change":"2023-12-19T16:43:24.458845+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458845+0000","last_undegraded":"2023-12-19T16:43:24.458845+0000","last_fullsized":"2023-12-19T16:43:24.458845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:03.546398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461365+0000","last_change":"2023-12-19T16:43:24.461365+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461365+0000","last_undegraded":"2023-12-19T16:43:24.461365+0000","last_fullsized":"2023-12-19T16:43:24.461365+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:54.432676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461815+0000","last_change":"2023-12-19T16:43:24.461815+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461815+0000","last_undegraded":"2023-12-19T16:43:24.461815+0000","last_fullsized":"2023-12-19T16:43:24.461815+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:08.723483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463008+0000","last_change":"2023-12-19T16:43:24.463008+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463008+0000","last_undegraded":"2023-12-19T16:43:24.463008+0000","last_fullsized":"2023-12-19T16:43:24.463008+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:39:45.423376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462583+0000","last_change":"2023-12-19T16:43:24.462583+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462583+0000","last_undegraded":"2023-12-19T16:43:24.462583+0000","last_fullsized":"2023-12-19T16:43:24.462583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:08.723483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460039+0000","last_change":"2023-12-19T16:43:24.460039+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460039+0000","last_undegraded":"2023-12-19T16:43:24.460039+0000","last_fullsized":"2023-12-19T16:43:24.460039+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:50:19.100332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459813+0000","last_change":"2023-12-19T16:43:24.459813+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459813+0000","last_undegraded":"2023-12-19T16:43:24.459813+0000","last_fullsized":"2023-12-19T16:43:24.459813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:43:43.874824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459181+0000","last_change":"2023-12-19T16:43:24.459181+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459181+0000","last_undegraded":"2023-12-19T16:43:24.459181+0000","last_fullsized":"2023-12-19T16:43:24.459181+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:30:19.111754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856897+0000","last_change":"2023-12-19T16:43:25.486440+0000","last_active":"2023-12-19T16:43:27.856897+0000","last_peered":"2023-12-19T16:43:27.856897+0000","last_clean":"2023-12-19T16:43:27.856897+0000","last_became_active":"2023-12-19T16:43:25.486112+0000","last_became_peered":"2023-12-19T16:43:25.486112+0000","last_unstale":"2023-12-19T16:43:27.856897+0000","last_undegraded":"2023-12-19T16:43:27.856897+0000","last_fullsized":"2023-12-19T16:43:27.856897+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:47:36.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463128+0000","last_change":"2023-12-19T16:43:24.463128+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463128+0000","last_undegraded":"2023-12-19T16:43:24.463128+0000","last_fullsized":"2023-12-19T16:43:24.463128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:05.966306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459816+0000","last_change":"2023-12-19T16:43:24.459816+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459816+0000","last_undegraded":"2023-12-19T16:43:24.459816+0000","last_fullsized":"2023-12-19T16:43:24.459816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:30:57.984641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458115+0000","last_change":"2023-12-19T16:43:24.458115+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458115+0000","last_undegraded":"2023-12-19T16:43:24.458115+0000","last_fullsized":"2023-12-19T16:43:24.458115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:22:22.341913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461943+0000","last_change":"2023-12-19T16:43:24.461943+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461943+0000","last_undegraded":"2023-12-19T16:43:24.461943+0000","last_fullsized":"2023-12-19T16:43:24.461943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:48:55.560180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857319+0000","last_change":"2023-12-19T16:43:25.486390+0000","last_active":"2023-12-19T16:43:27.857319+0000","last_peered":"2023-12-19T16:43:27.857319+0000","last_clean":"2023-12-19T16:43:27.857319+0000","last_became_active":"2023-12-19T16:43:25.486082+0000","last_became_peered":"2023-12-19T16:43:25.486082+0000","last_unstale":"2023-12-19T16:43:27.857319+0000","last_undegraded":"2023-12-19T16:43:27.857319+0000","last_fullsized":"2023-12-19T16:43:27.857319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:03:37.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856659+0000","last_change":"2023-12-19T16:43:25.908732+0000","last_active":"2023-12-19T16:43:27.856659+0000","last_peered":"2023-12-19T16:43:27.856659+0000","last_clean":"2023-12-19T16:43:27.856659+0000","last_became_active":"2023-12-19T16:43:25.901839+0000","last_became_peered":"2023-12-19T16:43:25.901839+0000","last_unstale":"2023-12-19T16:43:27.856659+0000","last_undegraded":"2023-12-19T16:43:27.856659+0000","last_fullsized":"2023-12-19T16:43:27.856659+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:20.393474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460656+0000","last_change":"2023-12-19T16:43:24.460656+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460656+0000","last_undegraded":"2023-12-19T16:43:24.460656+0000","last_fullsized":"2023-12-19T16:43:24.460656+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:01.953049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856999+0000","last_change":"2023-12-19T16:43:25.485176+0000","last_active":"2023-12-19T16:43:27.856999+0000","last_peered":"2023-12-19T16:43:27.856999+0000","last_clean":"2023-12-19T16:43:27.856999+0000","last_became_active":"2023-12-19T16:43:25.484842+0000","last_became_peered":"2023-12-19T16:43:25.484842+0000","last_unstale":"2023-12-19T16:43:27.856999+0000","last_undegraded":"2023-12-19T16:43:27.856999+0000","last_fullsized":"2023-12-19T16:43:27.856999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:49.582698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857140+0000","last_change":"2023-12-19T16:43:25.905903+0000","last_active":"2023-12-19T16:43:27.857140+0000","last_peered":"2023-12-19T16:43:27.857140+0000","last_clean":"2023-12-19T16:43:27.857140+0000","last_became_active":"2023-12-19T16:43:25.897356+0000","last_became_peered":"2023-12-19T16:43:25.897356+0000","last_unstale":"2023-12-19T16:43:27.857140+0000","last_undegraded":"2023-12-19T16:43:27.857140+0000","last_fullsized":"2023-12-19T16:43:27.857140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:36:39.824632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460764+0000","last_change":"2023-12-19T16:43:24.460764+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460764+0000","last_undegraded":"2023-12-19T16:43:24.460764+0000","last_fullsized":"2023-12-19T16:43:24.460764+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:45.708612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459149+0000","last_change":"2023-12-19T16:43:24.459149+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459149+0000","last_undegraded":"2023-12-19T16:43:24.459149+0000","last_fullsized":"2023-12-19T16:43:24.459149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:50:38.846004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461667+0000","last_change":"2023-12-19T16:43:24.461667+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461667+0000","last_undegraded":"2023-12-19T16:43:24.461667+0000","last_fullsized":"2023-12-19T16:43:24.461667+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:01.953049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463302+0000","last_change":"2023-12-19T16:43:24.463302+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463302+0000","last_undegraded":"2023-12-19T16:43:24.463302+0000","last_fullsized":"2023-12-19T16:43:24.463302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:24:17.651413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":173,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-19T16:43:21.144188+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:43:21.144188+0000","last_peered":"2023-12-19T16:43:21.144188+0000","last_clean":"2023-12-19T16:43:21.144188+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:43:21.144188+0000","last_undegraded":"2023-12-19T16:43:21.144188+0000","last_fullsized":"2023-12-19T16:43:21.144188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":958464,"data_stored":727125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.44600000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.53300000000000003}]},{"osd":3,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68200000000000005}]},{"osd":6,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.626},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.65300000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.623}]},{"osd":1,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.3799999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.59699999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.68000000000000005}]},{"osd":3,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.673,"5min":4.673,"15min":4.673},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":0.70799999999999996}]},{"osd":4,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90900000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.92700000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.94699999999999995}]}]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86199999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.83699999999999997}]},{"osd":2,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.628}]},{"osd":3,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.60499999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80100000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.73899999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.71799999999999997}]},{"osd":2,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.621}]},{"osd":3,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.748}]},{"osd":5,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.83199999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85099999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.98399999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.496},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.439}]},{"osd":1,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.45200000000000001}]},{"osd":4,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.78800000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":0.753}]},{"osd":7,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":5.0759999999999996,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.94999999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.5150000000000001,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.45900000000000002}]},{"osd":1,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.7590000000000003,"5min":5.7590000000000003,"15min":5.7590000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":5.8070000000000004,"5min":5.8070000000000004,"15min":5.8070000000000004},"last":0.439}]},{"osd":3,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.7599999999999998,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":5.7190000000000003,"5min":5.7190000000000003,"15min":5.7190000000000003},"last":0.48699999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.73299999999999998}]},{"osd":5,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.878},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.82899999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":0.70299999999999996}]},{"osd":7,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.85999999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.133,"5min":4.133,"15min":4.133},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.437}]},{"osd":4,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.72899999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":1007616,"data_stored":752284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.255},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.35099999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.59899999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47399999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.64300000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.627}]},{"osd":7,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.61299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:43:30.414 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:43:30.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b62241640 1 Processor -- start 2023-12-19T16:43:30.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b62241640 1 -- start start 2023-12-19T16:43:30.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b62241640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b5c155030 0x7f4b5c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b62241640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4b5c15f370 0x7f4b5c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b62241640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b5c0ce740 0x7f4b5c0cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b62241640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5c0cf130 con 0x7f4b5c0ce740 2023-12-19T16:43:30.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b62241640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5c06c970 con 0x7f4b5c15f370 2023-12-19T16:43:30.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b62241640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c15b1f0 con 0x7f4b5c155030 2023-12-19T16:43:30.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b5bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b5c0ce740 0x7f4b5c0cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b5bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4b5c0ce740 0x7f4b5c0cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55282/0 (socket says 172.21.15.26:55282) 2023-12-19T16:43:30.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b5bfff640 1 -- 172.21.15.26:0/3318492124 learned_addr learned my addr 172.21.15.26:0/3318492124 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:30.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b5b7fe640 1 --2- 172.21.15.26:0/3318492124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b5c155030 0x7f4b5c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b5affd640 1 --2- 172.21.15.26:0/3318492124 >> v2:172.21.15.160:3300/0 conn(0x7f4b5c15f370 0x7f4b5c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b5bfff640 1 -- 172.21.15.26:0/3318492124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b5c155030 msgr2=0x7f4b5c15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:30.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b5bfff640 1 --2- 172.21.15.26:0/3318492124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b5c155030 0x7f4b5c15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b5bfff640 1 -- 172.21.15.26:0/3318492124 >> v2:172.21.15.160:3300/0 conn(0x7f4b5c15f370 msgr2=0x7f4b5c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:30.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b5bfff640 1 --2- 172.21.15.26:0/3318492124 >> v2:172.21.15.160:3300/0 conn(0x7f4b5c15f370 0x7f4b5c06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b5bfff640 1 -- 172.21.15.26:0/3318492124 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c15b4d0 con 0x7f4b5c0ce740 2023-12-19T16:43:30.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b5b7fe640 1 --2- 172.21.15.26:0/3318492124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b5c155030 0x7f4b5c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:30.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.591+0000 7f4b5affd640 1 --2- 172.21.15.26:0/3318492124 >> v2:172.21.15.160:3300/0 conn(0x7f4b5c15f370 0x7f4b5c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:30.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.595+0000 7f4b5bfff640 1 --2- 172.21.15.26:0/3318492124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c0ce740 0x7f4b5c0cebf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0b3ac0 tx=0x7f4b5401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63761f663237f3c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:30.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.595+0000 7f4b5a7fc640 1 -- 172.21.15.26:0/3318492124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b540a5840 con 0x7f4b5c0ce740 2023-12-19T16:43:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.595+0000 7f4b5a7fc640 1 -- 172.21.15.26:0/3318492124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b540a59a0 con 0x7f4b5c0ce740 2023-12-19T16:43:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.595+0000 7f4b5a7fc640 1 -- 172.21.15.26:0/3318492124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b540a5d30 con 0x7f4b5c0ce740 2023-12-19T16:43:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.595+0000 7f4b62241640 1 -- 172.21.15.26:0/3318492124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c0ce740 msgr2=0x7f4b5c0cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.595+0000 7f4b62241640 1 --2- 172.21.15.26:0/3318492124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c0ce740 0x7f4b5c0cebf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0b3ac0 tx=0x7f4b5401c9a0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.595+0000 7f4b62241640 1 -- 172.21.15.26:0/3318492124 shutdown_connections 2023-12-19T16:43:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.595+0000 7f4b62241640 1 --2- 172.21.15.26:0/3318492124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c0ce740 0x7f4b5c0cebf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.595+0000 7f4b62241640 1 --2- 172.21.15.26:0/3318492124 >> v2:172.21.15.160:3300/0 conn(0x7f4b5c15f370 0x7f4b5c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.595+0000 7f4b62241640 1 --2- 172.21.15.26:0/3318492124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b5c155030 0x7f4b5c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.595+0000 7f4b62241640 1 -- 172.21.15.26:0/3318492124 >> 172.21.15.26:0/3318492124 conn(0x7f4b5c0658e0 msgr2=0x7f4b5c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.595+0000 7f4b62241640 1 -- 172.21.15.26:0/3318492124 shutdown_connections 2023-12-19T16:43:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.595+0000 7f4b62241640 1 -- 172.21.15.26:0/3318492124 wait complete. 2023-12-19T16:43:30.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b62241640 1 Processor -- start 2023-12-19T16:43:30.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b62241640 1 -- start start 2023-12-19T16:43:30.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b62241640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b5c0ce740 0x7f4b5c156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b62241640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c155030 0x7f4b5c1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b62241640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b5c15f370 0x7f4b5c1f9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b62241640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c13d0b0 con 0x7f4b5c155030 2023-12-19T16:43:30.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b62241640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c156ef0 con 0x7f4b5c0ce740 2023-12-19T16:43:30.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b62241640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c1571d0 con 0x7f4b5c15f370 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b5affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c155030 0x7f4b5c1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b5affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c155030 0x7f4b5c1566c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55284/0 (socket says 172.21.15.26:55284) 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b5affd640 1 -- 172.21.15.26:0/2100723579 learned_addr learned my addr 172.21.15.26:0/2100723579 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b5b7fe640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b5c0ce740 0x7f4b5c156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b5bfff640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b5c15f370 0x7f4b5c1f9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b5affd640 1 -- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b5c15f370 msgr2=0x7f4b5c1f9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b5affd640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b5c15f370 0x7f4b5c1f9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b5affd640 1 -- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b5c0ce740 msgr2=0x7f4b5c156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b5affd640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b5c0ce740 0x7f4b5c156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b5affd640 1 -- 172.21.15.26:0/2100723579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b54009b30 con 0x7f4b5c155030 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b5affd640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c155030 0x7f4b5c1566c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000c7f0 tx=0x7f4b5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b43fff640 1 -- 172.21.15.26:0/2100723579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5000cea0 con 0x7f4b5c155030 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b43fff640 1 -- 172.21.15.26:0/2100723579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b50007a30 con 0x7f4b5c155030 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b43fff640 1 -- 172.21.15.26:0/2100723579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50007de0 con 0x7f4b5c155030 2023-12-19T16:43:30.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b62241640 1 -- 172.21.15.26:0/2100723579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5c06c740 con 0x7f4b5c155030 2023-12-19T16:43:30.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.599+0000 7f4b62241640 1 -- 172.21.15.26:0/2100723579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5c1574b0 con 0x7f4b5c155030 2023-12-19T16:43:30.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.603+0000 7f4b62241640 1 -- 172.21.15.26:0/2100723579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5c13d260 con 0x7f4b5c155030 2023-12-19T16:43:30.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.603+0000 7f4b43fff640 1 -- 172.21.15.26:0/2100723579 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4b500113f0 con 0x7f4b5c155030 2023-12-19T16:43:30.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.603+0000 7f4b43fff640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4b48065a60 0x7f4b48067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.603+0000 7f4b43fff640 1 -- 172.21.15.26:0/2100723579 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b500832e0 con 0x7f4b5c155030 2023-12-19T16:43:30.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.603+0000 7f4b5b7fe640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4b48065a60 0x7f4b48067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.603+0000 7f4b5b7fe640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4b48065a60 0x7f4b48067f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c006fd0 tx=0x7f4b4c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:30.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.607+0000 7f4b43fff640 1 -- 172.21.15.26:0/2100723579 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4b5c13d260 con 0x7f4b5c155030 2023-12-19T16:43:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.763+0000 7f4b62241640 1 -- 172.21.15.26:0/2100723579 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4b5c06c5c0 con 0x7f4b48065a60 2023-12-19T16:43:30.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.771+0000 7f4b43fff640 1 -- 172.21.15.26:0/2100723579 <== mgr.4106 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278639 (secure 0 0 0) 0x7f4b4c00c050 con 0x7f4b48065a60 2023-12-19T16:43:30.777 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:30.779 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:43:30.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.771+0000 7f4b62241640 1 -- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4b48065a60 msgr2=0x7f4b48067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.771+0000 7f4b62241640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4b48065a60 0x7f4b48067f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c006fd0 tx=0x7f4b4c008040 comp rx=0 tx=0).stop 2023-12-19T16:43:30.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.771+0000 7f4b62241640 1 -- 172.21.15.26:0/2100723579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c155030 msgr2=0x7f4b5c1566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.771+0000 7f4b62241640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c155030 0x7f4b5c1566c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000c7f0 tx=0x7f4b5000ccb0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.771+0000 7f4b62241640 1 -- 172.21.15.26:0/2100723579 shutdown_connections 2023-12-19T16:43:30.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.771+0000 7f4b62241640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4b48065a60 0x7f4b48067f20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.771+0000 7f4b62241640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4b5c15f370 0x7f4b5c1f9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.771+0000 7f4b62241640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4b5c155030 0x7f4b5c1566c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.771+0000 7f4b62241640 1 --2- 172.21.15.26:0/2100723579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4b5c0ce740 0x7f4b5c156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.771+0000 7f4b62241640 1 -- 172.21.15.26:0/2100723579 >> 172.21.15.26:0/2100723579 conn(0x7f4b5c0658e0 msgr2=0x7f4b5c0b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:30.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.771+0000 7f4b62241640 1 -- 172.21.15.26:0/2100723579 shutdown_connections 2023-12-19T16:43:30.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.771+0000 7f4b62241640 1 -- 172.21.15.26:0/2100723579 wait complete. 2023-12-19T16:43:30.797 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2023-12-19T16:43:29.650113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":74,"acting":74,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236056,"kb_used_data":4056,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738664,"statfs":{"total":773094113280,"available":772852391936,"internally_reserved":0,"allocated":4153344,"data_stored":2299951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.194586"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856487+0000","last_change":"2023-12-19T16:43:25.892565+0000","last_active":"2023-12-19T16:43:27.856487+0000","last_peered":"2023-12-19T16:43:27.856487+0000","last_clean":"2023-12-19T16:43:27.856487+0000","last_became_active":"2023-12-19T16:43:25.892189+0000","last_became_peered":"2023-12-19T16:43:25.892189+0000","last_unstale":"2023-12-19T16:43:27.856487+0000","last_undegraded":"2023-12-19T16:43:27.856487+0000","last_fullsized":"2023-12-19T16:43:27.856487+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:20.383856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857570+0000","last_change":"2023-12-19T16:43:25.893897+0000","last_active":"2023-12-19T16:43:27.857570+0000","last_peered":"2023-12-19T16:43:27.857570+0000","last_clean":"2023-12-19T16:43:27.857570+0000","last_became_active":"2023-12-19T16:43:25.893654+0000","last_became_peered":"2023-12-19T16:43:25.893654+0000","last_unstale":"2023-12-19T16:43:27.857570+0000","last_undegraded":"2023-12-19T16:43:27.857570+0000","last_fullsized":"2023-12-19T16:43:27.857570+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:51:10.988939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463472+0000","last_change":"2023-12-19T16:43:24.463472+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463472+0000","last_undegraded":"2023-12-19T16:43:24.463472+0000","last_fullsized":"2023-12-19T16:43:24.463472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:39.252728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857152+0000","last_change":"2023-12-19T16:43:25.482086+0000","last_active":"2023-12-19T16:43:27.857152+0000","last_peered":"2023-12-19T16:43:27.857152+0000","last_clean":"2023-12-19T16:43:27.857152+0000","last_became_active":"2023-12-19T16:43:25.481816+0000","last_became_peered":"2023-12-19T16:43:25.481816+0000","last_unstale":"2023-12-19T16:43:27.857152+0000","last_undegraded":"2023-12-19T16:43:27.857152+0000","last_fullsized":"2023-12-19T16:43:27.857152+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:10.136641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463773+0000","last_change":"2023-12-19T16:43:24.463773+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463773+0000","last_undegraded":"2023-12-19T16:43:24.463773+0000","last_fullsized":"2023-12-19T16:43:24.463773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:38.921262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463938+0000","last_change":"2023-12-19T16:43:24.463938+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463938+0000","last_undegraded":"2023-12-19T16:43:24.463938+0000","last_fullsized":"2023-12-19T16:43:24.463938+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:52:24.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460161+0000","last_change":"2023-12-19T16:43:24.460161+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460161+0000","last_undegraded":"2023-12-19T16:43:24.460161+0000","last_fullsized":"2023-12-19T16:43:24.460161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:55:53.394261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458485+0000","last_change":"2023-12-19T16:43:24.458485+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458485+0000","last_undegraded":"2023-12-19T16:43:24.458485+0000","last_fullsized":"2023-12-19T16:43:24.458485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:47:45.880613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462394+0000","last_change":"2023-12-19T16:43:24.462394+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462394+0000","last_undegraded":"2023-12-19T16:43:24.462394+0000","last_fullsized":"2023-12-19T16:43:24.462394+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:49:45.049214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458111+0000","last_change":"2023-12-19T16:43:24.458111+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458111+0000","last_undegraded":"2023-12-19T16:43:24.458111+0000","last_fullsized":"2023-12-19T16:43:24.458111+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:08:40.828978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462419+0000","last_change":"2023-12-19T16:43:24.462419+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462419+0000","last_undegraded":"2023-12-19T16:43:24.462419+0000","last_fullsized":"2023-12-19T16:43:24.462419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:34:42.254411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857236+0000","last_change":"2023-12-19T16:43:25.485279+0000","last_active":"2023-12-19T16:43:27.857236+0000","last_peered":"2023-12-19T16:43:27.857236+0000","last_clean":"2023-12-19T16:43:27.857236+0000","last_became_active":"2023-12-19T16:43:25.484950+0000","last_became_peered":"2023-12-19T16:43:25.484950+0000","last_unstale":"2023-12-19T16:43:27.857236+0000","last_undegraded":"2023-12-19T16:43:27.857236+0000","last_fullsized":"2023-12-19T16:43:27.857236+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:30.752537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458128+0000","last_change":"2023-12-19T16:43:24.458128+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458128+0000","last_undegraded":"2023-12-19T16:43:24.458128+0000","last_fullsized":"2023-12-19T16:43:24.458128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:38.921262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462234+0000","last_change":"2023-12-19T16:43:24.462234+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462234+0000","last_undegraded":"2023-12-19T16:43:24.462234+0000","last_fullsized":"2023-12-19T16:43:24.462234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:22:22.341913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458845+0000","last_change":"2023-12-19T16:43:24.458845+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458845+0000","last_undegraded":"2023-12-19T16:43:24.458845+0000","last_fullsized":"2023-12-19T16:43:24.458845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:03.546398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461365+0000","last_change":"2023-12-19T16:43:24.461365+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461365+0000","last_undegraded":"2023-12-19T16:43:24.461365+0000","last_fullsized":"2023-12-19T16:43:24.461365+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:54.432676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461815+0000","last_change":"2023-12-19T16:43:24.461815+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461815+0000","last_undegraded":"2023-12-19T16:43:24.461815+0000","last_fullsized":"2023-12-19T16:43:24.461815+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:08.723483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463008+0000","last_change":"2023-12-19T16:43:24.463008+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463008+0000","last_undegraded":"2023-12-19T16:43:24.463008+0000","last_fullsized":"2023-12-19T16:43:24.463008+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:39:45.423376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462583+0000","last_change":"2023-12-19T16:43:24.462583+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462583+0000","last_undegraded":"2023-12-19T16:43:24.462583+0000","last_fullsized":"2023-12-19T16:43:24.462583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:08.723483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460039+0000","last_change":"2023-12-19T16:43:24.460039+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460039+0000","last_undegraded":"2023-12-19T16:43:24.460039+0000","last_fullsized":"2023-12-19T16:43:24.460039+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:50:19.100332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459813+0000","last_change":"2023-12-19T16:43:24.459813+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459813+0000","last_undegraded":"2023-12-19T16:43:24.459813+0000","last_fullsized":"2023-12-19T16:43:24.459813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:43:43.874824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459181+0000","last_change":"2023-12-19T16:43:24.459181+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459181+0000","last_undegraded":"2023-12-19T16:43:24.459181+0000","last_fullsized":"2023-12-19T16:43:24.459181+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:30:19.111754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856897+0000","last_change":"2023-12-19T16:43:25.486440+0000","last_active":"2023-12-19T16:43:27.856897+0000","last_peered":"2023-12-19T16:43:27.856897+0000","last_clean":"2023-12-19T16:43:27.856897+0000","last_became_active":"2023-12-19T16:43:25.486112+0000","last_became_peered":"2023-12-19T16:43:25.486112+0000","last_unstale":"2023-12-19T16:43:27.856897+0000","last_undegraded":"2023-12-19T16:43:27.856897+0000","last_fullsized":"2023-12-19T16:43:27.856897+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:47:36.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463128+0000","last_change":"2023-12-19T16:43:24.463128+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463128+0000","last_undegraded":"2023-12-19T16:43:24.463128+0000","last_fullsized":"2023-12-19T16:43:24.463128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:05.966306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459816+0000","last_change":"2023-12-19T16:43:24.459816+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459816+0000","last_undegraded":"2023-12-19T16:43:24.459816+0000","last_fullsized":"2023-12-19T16:43:24.459816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:30:57.984641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458115+0000","last_change":"2023-12-19T16:43:24.458115+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458115+0000","last_undegraded":"2023-12-19T16:43:24.458115+0000","last_fullsized":"2023-12-19T16:43:24.458115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:22:22.341913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461943+0000","last_change":"2023-12-19T16:43:24.461943+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461943+0000","last_undegraded":"2023-12-19T16:43:24.461943+0000","last_fullsized":"2023-12-19T16:43:24.461943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:48:55.560180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857319+0000","last_change":"2023-12-19T16:43:25.486390+0000","last_active":"2023-12-19T16:43:27.857319+0000","last_peered":"2023-12-19T16:43:27.857319+0000","last_clean":"2023-12-19T16:43:27.857319+0000","last_became_active":"2023-12-19T16:43:25.486082+0000","last_became_peered":"2023-12-19T16:43:25.486082+0000","last_unstale":"2023-12-19T16:43:27.857319+0000","last_undegraded":"2023-12-19T16:43:27.857319+0000","last_fullsized":"2023-12-19T16:43:27.857319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:03:37.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856659+0000","last_change":"2023-12-19T16:43:25.908732+0000","last_active":"2023-12-19T16:43:27.856659+0000","last_peered":"2023-12-19T16:43:27.856659+0000","last_clean":"2023-12-19T16:43:27.856659+0000","last_became_active":"2023-12-19T16:43:25.901839+0000","last_became_peered":"2023-12-19T16:43:25.901839+0000","last_unstale":"2023-12-19T16:43:27.856659+0000","last_undegraded":"2023-12-19T16:43:27.856659+0000","last_fullsized":"2023-12-19T16:43:27.856659+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:20.393474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460656+0000","last_change":"2023-12-19T16:43:24.460656+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460656+0000","last_undegraded":"2023-12-19T16:43:24.460656+0000","last_fullsized":"2023-12-19T16:43:24.460656+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:01.953049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856999+0000","last_change":"2023-12-19T16:43:25.485176+0000","last_active":"2023-12-19T16:43:27.856999+0000","last_peered":"2023-12-19T16:43:27.856999+0000","last_clean":"2023-12-19T16:43:27.856999+0000","last_became_active":"2023-12-19T16:43:25.484842+0000","last_became_peered":"2023-12-19T16:43:25.484842+0000","last_unstale":"2023-12-19T16:43:27.856999+0000","last_undegraded":"2023-12-19T16:43:27.856999+0000","last_fullsized":"2023-12-19T16:43:27.856999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:49.582698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857140+0000","last_change":"2023-12-19T16:43:25.905903+0000","last_active":"2023-12-19T16:43:27.857140+0000","last_peered":"2023-12-19T16:43:27.857140+0000","last_clean":"2023-12-19T16:43:27.857140+0000","last_became_active":"2023-12-19T16:43:25.897356+0000","last_became_peered":"2023-12-19T16:43:25.897356+0000","last_unstale":"2023-12-19T16:43:27.857140+0000","last_undegraded":"2023-12-19T16:43:27.857140+0000","last_fullsized":"2023-12-19T16:43:27.857140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:36:39.824632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460764+0000","last_change":"2023-12-19T16:43:24.460764+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460764+0000","last_undegraded":"2023-12-19T16:43:24.460764+0000","last_fullsized":"2023-12-19T16:43:24.460764+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:45.708612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459149+0000","last_change":"2023-12-19T16:43:24.459149+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459149+0000","last_undegraded":"2023-12-19T16:43:24.459149+0000","last_fullsized":"2023-12-19T16:43:24.459149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:50:38.846004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461667+0000","last_change":"2023-12-19T16:43:24.461667+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461667+0000","last_undegraded":"2023-12-19T16:43:24.461667+0000","last_fullsized":"2023-12-19T16:43:24.461667+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:01.953049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463302+0000","last_change":"2023-12-19T16:43:24.463302+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463302+0000","last_undegraded":"2023-12-19T16:43:24.463302+0000","last_fullsized":"2023-12-19T16:43:24.463302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:24:17.651413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":173,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-19T16:43:21.144188+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:43:21.144188+0000","last_peered":"2023-12-19T16:43:21.144188+0000","last_clean":"2023-12-19T16:43:21.144188+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:43:21.144188+0000","last_undegraded":"2023-12-19T16:43:21.144188+0000","last_fullsized":"2023-12-19T16:43:21.144188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":958464,"data_stored":727125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.44600000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.53300000000000003}]},{"osd":3,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68200000000000005}]},{"osd":6,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.626},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.65300000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.623}]},{"osd":1,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.3799999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.59699999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.68000000000000005}]},{"osd":3,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.673,"5min":4.673,"15min":4.673},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":0.70799999999999996}]},{"osd":4,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90900000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.92700000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.94699999999999995}]}]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86199999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.83699999999999997}]},{"osd":2,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.628}]},{"osd":3,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.60499999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80100000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.73899999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.71799999999999997}]},{"osd":2,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.621}]},{"osd":3,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.748}]},{"osd":5,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.83199999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85099999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.98399999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.496},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.439}]},{"osd":1,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.45200000000000001}]},{"osd":4,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.78800000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":0.753}]},{"osd":7,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":5.0759999999999996,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.94999999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.5150000000000001,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.45900000000000002}]},{"osd":1,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.7590000000000003,"5min":5.7590000000000003,"15min":5.7590000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":5.8070000000000004,"5min":5.8070000000000004,"15min":5.8070000000000004},"last":0.439}]},{"osd":3,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.7599999999999998,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":5.7190000000000003,"5min":5.7190000000000003,"15min":5.7190000000000003},"last":0.48699999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.73299999999999998}]},{"osd":5,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.878},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.82899999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":0.70299999999999996}]},{"osd":7,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.85999999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.133,"5min":4.133,"15min":4.133},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.437}]},{"osd":4,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.72899999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":1007616,"data_stored":752284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.255},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.35099999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.59899999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47399999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.64300000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.627}]},{"osd":7,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.61299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:43:30.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-12-19T16:43:30.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2023-12-19T16:43:30.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2023-12-19T16:43:30.802 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.802 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-12-19T16:43:30.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-12-19T16:43:30.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2023-12-19T16:43:30.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-12-19T16:43:30.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-12-19T16:43:30.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2023-12-19T16:43:30.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-12-19T16:43:30.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-12-19T16:43:30.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2023-12-19T16:43:30.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-12-19T16:43:30.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2023-12-19T16:43:30.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-12-19T16:43:30.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-12-19T16:43:30.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-12-19T16:43:30.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2023-12-19T16:43:30.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2023-12-19T16:43:30.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-12-19T16:43:30.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-12-19T16:43:30.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2023-12-19T16:43:30.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2023-12-19T16:43:30.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-12-19T16:43:30.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2023-12-19T16:43:30.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-12-19T16:43:30.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2023-12-19T16:43:30.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-12-19T16:43:30.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-12-19T16:43:30.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-12-19T16:43:30.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-12-19T16:43:30.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2023-12-19T16:43:30.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.463472+0000', 'last_change': '2023-12-19T16:43:24.463472+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.463472+0000', 'last_undegraded': '2023-12-19T16:43:24.463472+0000', 'last_fullsized': '2023-12-19T16:43:24.463472+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T23:37:39.252728+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-19T16:43:30.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-12-19T16:43:30.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-12-19T16:43:30.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2023-12-19T16:43:30.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.463773+0000', 'last_change': '2023-12-19T16:43:24.463773+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.463773+0000', 'last_undegraded': '2023-12-19T16:43:24.463773+0000', 'last_fullsized': '2023-12-19T16:43:24.463773+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T22:25:38.921262+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-19T16:43:30.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-12-19T16:43:30.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-12-19T16:43:30.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.463938+0000', 'last_change': '2023-12-19T16:43:24.463938+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.463938+0000', 'last_undegraded': '2023-12-19T16:43:24.463938+0000', 'last_fullsized': '2023-12-19T16:43:24.463938+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T17:52:24.576402+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-19T16:43:30.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-12-19T16:43:30.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2023-12-19T16:43:30.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.460161+0000', 'last_change': '2023-12-19T16:43:24.460161+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.460161+0000', 'last_undegraded': '2023-12-19T16:43:24.460161+0000', 'last_fullsized': '2023-12-19T16:43:24.460161+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T22:55:53.394261+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-19T16:43:30.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-12-19T16:43:30.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2023-12-19T16:43:30.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.458485+0000', 'last_change': '2023-12-19T16:43:24.458485+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.458485+0000', 'last_undegraded': '2023-12-19T16:43:24.458485+0000', 'last_fullsized': '2023-12-19T16:43:24.458485+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T03:47:45.880613+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-19T16:43:30.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-12-19T16:43:30.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2023-12-19T16:43:30.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.462394+0000', 'last_change': '2023-12-19T16:43:24.462394+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.462394+0000', 'last_undegraded': '2023-12-19T16:43:24.462394+0000', 'last_fullsized': '2023-12-19T16:43:24.462394+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T23:49:45.049214+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-19T16:43:30.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-12-19T16:43:30.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2023-12-19T16:43:30.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.458111+0000', 'last_change': '2023-12-19T16:43:24.458111+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.458111+0000', 'last_undegraded': '2023-12-19T16:43:24.458111+0000', 'last_fullsized': '2023-12-19T16:43:24.458111+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T03:08:40.828978+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-19T16:43:30.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-12-19T16:43:30.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2023-12-19T16:43:30.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.462419+0000', 'last_change': '2023-12-19T16:43:24.462419+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.462419+0000', 'last_undegraded': '2023-12-19T16:43:24.462419+0000', 'last_fullsized': '2023-12-19T16:43:24.462419+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T18:34:42.254411+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-19T16:43:30.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-12-19T16:43:30.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-12-19T16:43:30.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2023-12-19T16:43:30.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.458128+0000', 'last_change': '2023-12-19T16:43:24.458128+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.458128+0000', 'last_undegraded': '2023-12-19T16:43:24.458128+0000', 'last_fullsized': '2023-12-19T16:43:24.458128+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T22:25:38.921262+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-19T16:43:30.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-12-19T16:43:30.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2023-12-19T16:43:30.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.462234+0000', 'last_change': '2023-12-19T16:43:24.462234+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.462234+0000', 'last_undegraded': '2023-12-19T16:43:24.462234+0000', 'last_fullsized': '2023-12-19T16:43:24.462234+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T21:22:22.341913+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-19T16:43:30.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-12-19T16:43:30.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2023-12-19T16:43:30.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.458845+0000', 'last_change': '2023-12-19T16:43:24.458845+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.458845+0000', 'last_undegraded': '2023-12-19T16:43:24.458845+0000', 'last_fullsized': '2023-12-19T16:43:24.458845+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T03:03:03.546398+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-19T16:43:30.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-12-19T16:43:30.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2023-12-19T16:43:30.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.461365+0000', 'last_change': '2023-12-19T16:43:24.461365+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.461365+0000', 'last_undegraded': '2023-12-19T16:43:24.461365+0000', 'last_fullsized': '2023-12-19T16:43:24.461365+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T18:18:54.432676+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-19T16:43:30.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-12-19T16:43:30.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2023-12-19T16:43:30.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.461815+0000', 'last_change': '2023-12-19T16:43:24.461815+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.461815+0000', 'last_undegraded': '2023-12-19T16:43:24.461815+0000', 'last_fullsized': '2023-12-19T16:43:24.461815+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T20:30:08.723483+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-19T16:43:30.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-12-19T16:43:30.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2023-12-19T16:43:30.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.463008+0000', 'last_change': '2023-12-19T16:43:24.463008+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.463008+0000', 'last_undegraded': '2023-12-19T16:43:24.463008+0000', 'last_fullsized': '2023-12-19T16:43:24.463008+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T18:39:45.423376+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-19T16:43:30.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-12-19T16:43:30.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2023-12-19T16:43:30.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.462583+0000', 'last_change': '2023-12-19T16:43:24.462583+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.462583+0000', 'last_undegraded': '2023-12-19T16:43:24.462583+0000', 'last_fullsized': '2023-12-19T16:43:24.462583+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T20:30:08.723483+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-19T16:43:30.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-12-19T16:43:30.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2023-12-19T16:43:30.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.460039+0000', 'last_change': '2023-12-19T16:43:24.460039+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.460039+0000', 'last_undegraded': '2023-12-19T16:43:24.460039+0000', 'last_fullsized': '2023-12-19T16:43:24.460039+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T02:50:19.100332+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-19T16:43:30.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-12-19T16:43:30.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2023-12-19T16:43:30.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.459813+0000', 'last_change': '2023-12-19T16:43:24.459813+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.459813+0000', 'last_undegraded': '2023-12-19T16:43:24.459813+0000', 'last_fullsized': '2023-12-19T16:43:24.459813+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T03:43:43.874824+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-19T16:43:30.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-12-19T16:43:30.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2023-12-19T16:43:30.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.459181+0000', 'last_change': '2023-12-19T16:43:24.459181+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.459181+0000', 'last_undegraded': '2023-12-19T16:43:24.459181+0000', 'last_fullsized': '2023-12-19T16:43:24.459181+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T17:30:19.111754+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-19T16:43:30.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-12-19T16:43:30.814 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.814 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-12-19T16:43:30.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-12-19T16:43:30.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.463128+0000', 'last_change': '2023-12-19T16:43:24.463128+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.463128+0000', 'last_undegraded': '2023-12-19T16:43:24.463128+0000', 'last_fullsized': '2023-12-19T16:43:24.463128+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T00:00:05.966306+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-19T16:43:30.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-12-19T16:43:30.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2023-12-19T16:43:30.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.459816+0000', 'last_change': '2023-12-19T16:43:24.459816+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.459816+0000', 'last_undegraded': '2023-12-19T16:43:24.459816+0000', 'last_fullsized': '2023-12-19T16:43:24.459816+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T21:30:57.984641+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-19T16:43:30.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-12-19T16:43:30.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.815 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2023-12-19T16:43:30.815 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.458115+0000', 'last_change': '2023-12-19T16:43:24.458115+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.458115+0000', 'last_undegraded': '2023-12-19T16:43:24.458115+0000', 'last_fullsized': '2023-12-19T16:43:24.458115+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T21:22:22.341913+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-19T16:43:30.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-12-19T16:43:30.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2023-12-19T16:43:30.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.461943+0000', 'last_change': '2023-12-19T16:43:24.461943+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.461943+0000', 'last_undegraded': '2023-12-19T16:43:24.461943+0000', 'last_fullsized': '2023-12-19T16:43:24.461943+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T20:48:55.560180+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-19T16:43:30.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-12-19T16:43:30.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2023-12-19T16:43:30.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.460656+0000', 'last_change': '2023-12-19T16:43:24.460656+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.460656+0000', 'last_undegraded': '2023-12-19T16:43:24.460656+0000', 'last_fullsized': '2023-12-19T16:43:24.460656+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T02:05:01.953049+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-19T16:43:30.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-12-19T16:43:30.816 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.816 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-12-19T16:43:30.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.460764+0000', 'last_change': '2023-12-19T16:43:24.460764+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.460764+0000', 'last_undegraded': '2023-12-19T16:43:24.460764+0000', 'last_fullsized': '2023-12-19T16:43:24.460764+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T01:15:45.708612+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-19T16:43:30.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-12-19T16:43:30.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2023-12-19T16:43:30.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.459149+0000', 'last_change': '2023-12-19T16:43:24.459149+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.459149+0000', 'last_undegraded': '2023-12-19T16:43:24.459149+0000', 'last_fullsized': '2023-12-19T16:43:24.459149+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T18:50:38.846004+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-19T16:43:30.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-12-19T16:43:30.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2023-12-19T16:43:30.817 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.461667+0000', 'last_change': '2023-12-19T16:43:24.461667+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.461667+0000', 'last_undegraded': '2023-12-19T16:43:24.461667+0000', 'last_fullsized': '2023-12-19T16:43:24.461667+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T02:05:01.953049+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-19T16:43:30.817 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-12-19T16:43:30.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-19T16:43:25.454250+0000', 'last_change': '2023-12-19T16:43:25.454250+0000', 'last_active': '2023-12-19T16:43:25.454250+0000', 'last_peered': '2023-12-19T16:43:25.454250+0000', 'last_clean': '2023-12-19T16:43:25.454250+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:25.454250+0000', 'last_undegraded': '2023-12-19T16:43:25.454250+0000', 'last_fullsized': '2023-12-19T16:43:25.454250+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:25.454250+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-19T16:43:30.818 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-12-19T16:43:30.818 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 34, 'state': 'creating+peering', 'last_fresh': '2023-12-19T16:43:24.463302+0000', 'last_change': '2023-12-19T16:43:24.463302+0000', 'last_active': '2023-12-19T16:43:24.447378+0000', 'last_peered': '2023-12-19T16:43:24.447378+0000', 'last_clean': '2023-12-19T16:43:24.447378+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-19T16:43:24.463302+0000', 'last_undegraded': '2023-12-19T16:43:24.463302+0000', 'last_fullsized': '2023-12-19T16:43:24.463302+0000', 'mapping_epoch': 34, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 34, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'last_clean_scrub_stamp': '2023-12-19T16:43:24.447378+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-20T17:24:17.651413+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-19T16:43:30.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-19T16:43:30.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.979+0000 7f156e8c4640 1 Processor -- start 2023-12-19T16:43:30.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.979+0000 7f156e8c4640 1 -- start start 2023-12-19T16:43:30.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.979+0000 7f156e8c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680b85e0 0x7f15680cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.979+0000 7f156e8c4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15680d0d40 con 0x7f156815cda0 2023-12-19T16:43:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.979+0000 7f156e8c4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1568161150 con 0x7f15680cde60 2023-12-19T16:43:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.979+0000 7f156e8c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15680ce2f0 con 0x7f15680b85e0 2023-12-19T16:43:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.979+0000 7f1567fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f156815cda0 0x7f1568160920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47370/0 (socket says 172.21.15.26:47370) 2023-12-19T16:43:30.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.979+0000 7f1567fff640 1 -- 172.21.15.26:0/1277587185 learned_addr learned my addr 172.21.15.26:0/1277587185 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.979+0000 7f15677fe640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680b85e0 0x7f15680cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.979+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376670654 0 0) 0x7f15680d0d40 con 0x7f156815cda0 2023-12-19T16:43:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.979+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1550003530 con 0x7f156815cda0 2023-12-19T16:43:30.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.979+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373815115 0 0) 0x7f1568161150 con 0x7f15680cde60 2023-12-19T16:43:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.979+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15680d0d40 con 0x7f15680cde60 2023-12-19T16:43:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3975498287 0 0) 0x7f1550003530 con 0x7f156815cda0 2023-12-19T16:43:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680b85e0 msgr2=0x7f15680cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680b85e0 0x7f15680cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 >> v1:172.21.15.160:6789/0 conn(0x7f15680cde60 legacy=0x7f15680d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1568161430 con 0x7f156815cda0 2023-12-19T16:43:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f1558001680 con 0x7f156815cda0 2023-12-19T16:43:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f15677fe640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680b85e0 0x7f15680cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 >> v1:172.21.15.26:6789/0 conn(0x7f156815cda0 legacy=0x7f1568160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1550003b40 0x7f1550003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680b85e0 0x7f15500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1550004d10 0x7f1550005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15680ce2f0 con 0x7f1550003b40 2023-12-19T16:43:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155000ba40 con 0x7f1550004d10 2023-12-19T16:43:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155000bb80 con 0x7f15680b85e0 2023-12-19T16:43:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1567fff640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1550003b40 0x7f1550003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f15677fe640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680b85e0 0x7f15500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f156ce61640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1550004d10 0x7f1550005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1567fff640 1 -- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680b85e0 msgr2=0x7f15500047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1567fff640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680b85e0 0x7f15500047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1567fff640 1 -- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1550004d10 msgr2=0x7f1550005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1567fff640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1550004d10 0x7f1550005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1567fff640 1 -- 172.21.15.26:0/1277587185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c0093b0 con 0x7f1550003b40 2023-12-19T16:43:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f15677fe640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680b85e0 0x7f15500047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:30.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f156ce61640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1550004d10 0x7f1550005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:30.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1567fff640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1550003b40 0x7f1550003f40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f155800c190 tx=0x7f155801cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c8a8a57250e86d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:30.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.983+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580058e0 con 0x7f1550003b40 2023-12-19T16:43:30.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1558005a60 con 0x7f1550003b40 2023-12-19T16:43:30.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f1566ffd640 1 -- 172.21.15.26:0/1277587185 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558005e10 con 0x7f1550003b40 2023-12-19T16:43:30.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 -- 172.21.15.26:0/1277587185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1550003b40 msgr2=0x7f1550003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 --2- 172.21.15.26:0/1277587185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1550003b40 0x7f1550003f40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f155800c190 tx=0x7f155801cb90 comp rx=0 tx=0).stop 2023-12-19T16:43:30.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156ce61640 1 -- 172.21.15.26:0/1277587185 reap_dead start 2023-12-19T16:43:30.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 -- 172.21.15.26:0/1277587185 shutdown_connections 2023-12-19T16:43:30.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 -- 172.21.15.26:0/1277587185 >> 172.21.15.26:0/1277587185 conn(0x7f15680658e0 msgr2=0x7f1568065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:30.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 -- 172.21.15.26:0/1277587185 shutdown_connections 2023-12-19T16:43:30.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 -- 172.21.15.26:0/1277587185 wait complete. 2023-12-19T16:43:30.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 Processor -- start 2023-12-19T16:43:30.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 -- start start 2023-12-19T16:43:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1550003b40 0x7f15681f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680c9d20 0x7f15681f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15680ca1a0 0x7f15681f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15680b4e30 con 0x7f15680ca1a0 2023-12-19T16:43:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15680b41b0 con 0x7f1550003b40 2023-12-19T16:43:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156e8c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15680b4490 con 0x7f15680c9d20 2023-12-19T16:43:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156ce61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15680ca1a0 0x7f15681f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156ce61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15680ca1a0 0x7f15681f93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55306/0 (socket says 172.21.15.26:55306) 2023-12-19T16:43:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.987+0000 7f156ce61640 1 -- 172.21.15.26:0/2095291231 learned_addr learned my addr 172.21.15.26:0/2095291231 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f1567fff640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1550003b40 0x7f15681f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f15677fe640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680c9d20 0x7f15681f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f156ce61640 1 -- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680c9d20 msgr2=0x7f15681f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f156ce61640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680c9d20 0x7f15681f8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f156ce61640 1 -- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1550003b40 msgr2=0x7f15681f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f156ce61640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1550003b40 0x7f15681f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f156ce61640 1 -- 172.21.15.26:0/2095291231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15680b4770 con 0x7f15680ca1a0 2023-12-19T16:43:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f15677fe640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680c9d20 0x7f15681f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f156ce61640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15680ca1a0 0x7f15681f93a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1560001340 tx=0x7f156000ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f1564ff9640 1 -- 172.21.15.26:0/2095291231 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560006b60 con 0x7f15680ca1a0 2023-12-19T16:43:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f1564ff9640 1 -- 172.21.15.26:0/2095291231 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1560006cc0 con 0x7f15680ca1a0 2023-12-19T16:43:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f1564ff9640 1 -- 172.21.15.26:0/2095291231 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560003bc0 con 0x7f15680ca1a0 2023-12-19T16:43:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f156e8c4640 1 -- 172.21.15.26:0/2095291231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15680b4a50 con 0x7f15680ca1a0 2023-12-19T16:43:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f156e8c4640 1 -- 172.21.15.26:0/2095291231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15681f9d10 con 0x7f15680ca1a0 2023-12-19T16:43:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f156e8c4640 1 -- 172.21.15.26:0/2095291231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15680b4c10 con 0x7f15680ca1a0 2023-12-19T16:43:30.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f1564ff9640 1 -- 172.21.15.26:0/2095291231 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1560003d20 con 0x7f15680ca1a0 2023-12-19T16:43:30.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f1564ff9640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1554065a60 0x7f1554067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:30.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.991+0000 7f1564ff9640 1 -- 172.21.15.26:0/2095291231 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1560082d60 con 0x7f15680ca1a0 2023-12-19T16:43:30.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.995+0000 7f1567fff640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1554065a60 0x7f1554067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.995+0000 7f1567fff640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1554065a60 0x7f1554067f20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f15580aa000 tx=0x7f1558001a00 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:31.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:30.995+0000 7f1564ff9640 1 -- 172.21.15.26:0/2095291231 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1560050710 con 0x7f15680ca1a0 2023-12-19T16:43:31.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f156e8c4640 1 -- 172.21.15.26:0/2095291231 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1568074270 con 0x7f15680ca1a0 2023-12-19T16:43:31.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f1564ff9640 1 -- 172.21.15.26:0/2095291231 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1716 (secure 0 0 0) 0x7f1560053d30 con 0x7f15680ca1a0 2023-12-19T16:43:31.215 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:31.216 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":88,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":37,"num_osds":8,"num_up_osds":8,"osd_up_since":1703004134,"num_in_osds":8,"osd_in_since":1703004125,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":60},{"state_name":"creating+peering","count":27},{"state_name":"active+clean","count":10}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590368,"bytes_used":241721344,"bytes_avail":772852391936,"bytes_total":773094113280,"unknown_pgs_ratio":0.6185566782951355,"inactive_pgs_ratio":0.27835050225257874,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5106}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2023-12-19T16:43:15.647638+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-19T16:43:31.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f156e8c4640 1 -- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1554065a60 msgr2=0x7f1554067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f156e8c4640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1554065a60 0x7f1554067f20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f15580aa000 tx=0x7f1558001a00 comp rx=0 tx=0).stop 2023-12-19T16:43:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f156e8c4640 1 -- 172.21.15.26:0/2095291231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15680ca1a0 msgr2=0x7f15681f93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f156e8c4640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15680ca1a0 0x7f15681f93a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1560001340 tx=0x7f156000ebe0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f156e8c4640 1 -- 172.21.15.26:0/2095291231 shutdown_connections 2023-12-19T16:43:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f156e8c4640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15680ca1a0 0x7f15681f93a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f156e8c4640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1554065a60 0x7f1554067f20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f156e8c4640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15680c9d20 0x7f15681f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f156e8c4640 1 --2- 172.21.15.26:0/2095291231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1550003b40 0x7f15681f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f156e8c4640 1 -- 172.21.15.26:0/2095291231 >> 172.21.15.26:0/2095291231 conn(0x7f15680658e0 msgr2=0x7f15680b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f156e8c4640 1 -- 172.21.15.26:0/2095291231 shutdown_connections 2023-12-19T16:43:31.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.211+0000 7f156e8c4640 1 -- 172.21.15.26:0/2095291231 wait complete. 2023-12-19T16:43:31.237 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-19T16:43:31.237 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:43:31.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f202d2c4640 1 Processor -- start 2023-12-19T16:43:31.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f202d2c4640 1 -- start start 2023-12-19T16:43:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f202d2c4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20280d3300 0x7f20280d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f202d2c4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20280d29d0 con 0x7f202815cd40 2023-12-19T16:43:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f202d2c4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202815d680 con 0x7f20280ce040 2023-12-19T16:43:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f202d2c4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20280ce4b0 con 0x7f20280d3300 2023-12-19T16:43:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f20277fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20280d3300 0x7f20280d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f2026ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f202815cd40 0x7f2028160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47376/0 (socket says 172.21.15.26:47376) 2023-12-19T16:43:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f20277fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20280d3300 0x7f20280d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55314/0 (socket says 172.21.15.26:55314) 2023-12-19T16:43:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f2026ffd640 1 -- 172.21.15.26:0/1937348165 learned_addr learned my addr 172.21.15.26:0/1937348165 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:31.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f20277fe640 1 -- 172.21.15.26:0/1937348165 >> v1:172.21.15.160:6789/0 conn(0x7f20280ce040 legacy=0x7f20280d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:31.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f20277fe640 1 -- 172.21.15.26:0/1937348165 >> v1:172.21.15.26:6789/0 conn(0x7f202815cd40 legacy=0x7f2028160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:31.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f20277fe640 1 -- 172.21.15.26:0/1937348165 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2028161750 con 0x7f20280d3300 2023-12-19T16:43:31.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.375+0000 7f20267fc640 1 -- 172.21.15.26:0/1937348165 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037792917 0 0) 0x7f20180012a0 con 0x7f202815cd40 2023-12-19T16:43:31.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.379+0000 7f20277fe640 1 --2- 172.21.15.26:0/1937348165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20280d3300 0x7f20280d0e30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2020009d90 tx=0x7f202001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829696914e960a68 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:31.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.379+0000 7f20267fc640 1 -- 172.21.15.26:0/1937348165 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200a59a0 con 0x7f20280d3300 2023-12-19T16:43:31.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.379+0000 7f20267fc640 1 -- 172.21.15.26:0/1937348165 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20200a5b00 con 0x7f20280d3300 2023-12-19T16:43:31.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.379+0000 7f20267fc640 1 -- 172.21.15.26:0/1937348165 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200a5e90 con 0x7f20280d3300 2023-12-19T16:43:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.379+0000 7f202d2c4640 1 -- 172.21.15.26:0/1937348165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20280d3300 msgr2=0x7f20280d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.379+0000 7f202d2c4640 1 --2- 172.21.15.26:0/1937348165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20280d3300 0x7f20280d0e30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2020009d90 tx=0x7f202001c8e0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.379+0000 7f202d2c4640 1 -- 172.21.15.26:0/1937348165 shutdown_connections 2023-12-19T16:43:31.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.379+0000 7f202d2c4640 1 --2- 172.21.15.26:0/1937348165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20280d3300 0x7f20280d0e30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.379+0000 7f202d2c4640 1 -- 172.21.15.26:0/1937348165 >> 172.21.15.26:0/1937348165 conn(0x7f202806a120 msgr2=0x7f202806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:31.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.379+0000 7f202d2c4640 1 -- 172.21.15.26:0/1937348165 shutdown_connections 2023-12-19T16:43:31.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.379+0000 7f202d2c4640 1 -- 172.21.15.26:0/1937348165 wait complete. 2023-12-19T16:43:31.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f202d2c4640 1 Processor -- start 2023-12-19T16:43:31.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f202d2c4640 1 -- start start 2023-12-19T16:43:31.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f202d2c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20280ce040 0x7f2028156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:31.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f202d2c4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20280d3300 0x7f2028156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:31.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f202d2c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f202815cd40 0x7f20280c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:31.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f202d2c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202813c6a0 con 0x7f20280ce040 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f202d2c4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280c5bd0 con 0x7f20280d3300 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f202d2c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20280c4fc0 con 0x7f202815cd40 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f2026ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20280ce040 0x7f2028156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f2026ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20280ce040 0x7f2028156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55328/0 (socket says 172.21.15.26:55328) 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f2026ffd640 1 -- 172.21.15.26:0/1900844009 learned_addr learned my addr 172.21.15.26:0/1900844009 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f20277fe640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f202815cd40 0x7f20280c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f201ffff640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20280d3300 0x7f2028156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f2026ffd640 1 -- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f202815cd40 msgr2=0x7f20280c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f2026ffd640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f202815cd40 0x7f20280c6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f2026ffd640 1 -- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20280d3300 msgr2=0x7f2028156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f2026ffd640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20280d3300 0x7f2028156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f2026ffd640 1 -- 172.21.15.26:0/1900844009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2020009a70 con 0x7f20280ce040 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f201ffff640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20280d3300 0x7f2028156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:31.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f2026ffd640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20280ce040 0x7f2028156310 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2018001ec0 tx=0x7f201800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f201f7fe640 1 -- 172.21.15.26:0/1900844009 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2018007480 con 0x7f20280ce040 2023-12-19T16:43:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f201f7fe640 1 -- 172.21.15.26:0/1900844009 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20180075e0 con 0x7f20280ce040 2023-12-19T16:43:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f201f7fe640 1 -- 172.21.15.26:0/1900844009 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20180059f0 con 0x7f20280ce040 2023-12-19T16:43:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f202d2c4640 1 -- 172.21.15.26:0/1900844009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20280c52a0 con 0x7f20280ce040 2023-12-19T16:43:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f202d2c4640 1 -- 172.21.15.26:0/1900844009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20280c5860 con 0x7f20280ce040 2023-12-19T16:43:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.383+0000 7f202d2c4640 1 -- 172.21.15.26:0/1900844009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2028074270 con 0x7f20280ce040 2023-12-19T16:43:31.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.387+0000 7f201f7fe640 1 -- 172.21.15.26:0/1900844009 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2018005b50 con 0x7f20280ce040 2023-12-19T16:43:31.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.387+0000 7f201f7fe640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2008065a60 0x7f2008067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:31.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.387+0000 7f201f7fe640 1 -- 172.21.15.26:0/1900844009 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20180825f0 con 0x7f20280ce040 2023-12-19T16:43:31.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.387+0000 7f201ffff640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2008065a60 0x7f2008067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:31.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.387+0000 7f201ffff640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2008065a60 0x7f2008067f20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f20280b6e10 tx=0x7f2010008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:31.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.391+0000 7f201f7fe640 1 -- 172.21.15.26:0/1900844009 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f201804ffa0 con 0x7f20280ce040 2023-12-19T16:43:31.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.543+0000 7f202d2c4640 1 -- 172.21.15.26:0/1900844009 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f20280cce20 con 0x7f2008065a60 2023-12-19T16:43:31.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.551+0000 7f201f7fe640 1 -- 172.21.15.26:0/1900844009 <== mgr.4106 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278639 (secure 0 0 0) 0x7f20280cce20 con 0x7f2008065a60 2023-12-19T16:43:31.559 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:31.561 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:43:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.555+0000 7f202d2c4640 1 -- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2008065a60 msgr2=0x7f2008067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.555+0000 7f202d2c4640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2008065a60 0x7f2008067f20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f20280b6e10 tx=0x7f2010008040 comp rx=0 tx=0).stop 2023-12-19T16:43:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.555+0000 7f202d2c4640 1 -- 172.21.15.26:0/1900844009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20280ce040 msgr2=0x7f2028156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.555+0000 7f202d2c4640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20280ce040 0x7f2028156310 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2018001ec0 tx=0x7f201800c9b0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.555+0000 7f202d2c4640 1 -- 172.21.15.26:0/1900844009 shutdown_connections 2023-12-19T16:43:31.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.555+0000 7f202d2c4640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2008065a60 0x7f2008067f20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.555+0000 7f202d2c4640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f202815cd40 0x7f20280c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.555+0000 7f202d2c4640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20280d3300 0x7f2028156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.555+0000 7f202d2c4640 1 --2- 172.21.15.26:0/1900844009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20280ce040 0x7f2028156310 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:31.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.555+0000 7f202d2c4640 1 -- 172.21.15.26:0/1900844009 >> 172.21.15.26:0/1900844009 conn(0x7f202806a120 msgr2=0x7f20280b64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:31.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.555+0000 7f202d2c4640 1 -- 172.21.15.26:0/1900844009 shutdown_connections 2023-12-19T16:43:31.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:31.555+0000 7f202d2c4640 1 -- 172.21.15.26:0/1900844009 wait complete. 2023-12-19T16:43:31.580 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2023-12-19T16:43:29.650113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":74,"acting":74,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236056,"kb_used_data":4056,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738664,"statfs":{"total":773094113280,"available":772852391936,"internally_reserved":0,"allocated":4153344,"data_stored":2299951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.194586"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856487+0000","last_change":"2023-12-19T16:43:25.892565+0000","last_active":"2023-12-19T16:43:27.856487+0000","last_peered":"2023-12-19T16:43:27.856487+0000","last_clean":"2023-12-19T16:43:27.856487+0000","last_became_active":"2023-12-19T16:43:25.892189+0000","last_became_peered":"2023-12-19T16:43:25.892189+0000","last_unstale":"2023-12-19T16:43:27.856487+0000","last_undegraded":"2023-12-19T16:43:27.856487+0000","last_fullsized":"2023-12-19T16:43:27.856487+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:20.383856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857570+0000","last_change":"2023-12-19T16:43:25.893897+0000","last_active":"2023-12-19T16:43:27.857570+0000","last_peered":"2023-12-19T16:43:27.857570+0000","last_clean":"2023-12-19T16:43:27.857570+0000","last_became_active":"2023-12-19T16:43:25.893654+0000","last_became_peered":"2023-12-19T16:43:25.893654+0000","last_unstale":"2023-12-19T16:43:27.857570+0000","last_undegraded":"2023-12-19T16:43:27.857570+0000","last_fullsized":"2023-12-19T16:43:27.857570+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:51:10.988939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463472+0000","last_change":"2023-12-19T16:43:24.463472+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463472+0000","last_undegraded":"2023-12-19T16:43:24.463472+0000","last_fullsized":"2023-12-19T16:43:24.463472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:39.252728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857152+0000","last_change":"2023-12-19T16:43:25.482086+0000","last_active":"2023-12-19T16:43:27.857152+0000","last_peered":"2023-12-19T16:43:27.857152+0000","last_clean":"2023-12-19T16:43:27.857152+0000","last_became_active":"2023-12-19T16:43:25.481816+0000","last_became_peered":"2023-12-19T16:43:25.481816+0000","last_unstale":"2023-12-19T16:43:27.857152+0000","last_undegraded":"2023-12-19T16:43:27.857152+0000","last_fullsized":"2023-12-19T16:43:27.857152+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:10.136641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463773+0000","last_change":"2023-12-19T16:43:24.463773+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463773+0000","last_undegraded":"2023-12-19T16:43:24.463773+0000","last_fullsized":"2023-12-19T16:43:24.463773+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:38.921262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463938+0000","last_change":"2023-12-19T16:43:24.463938+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463938+0000","last_undegraded":"2023-12-19T16:43:24.463938+0000","last_fullsized":"2023-12-19T16:43:24.463938+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:52:24.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460161+0000","last_change":"2023-12-19T16:43:24.460161+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460161+0000","last_undegraded":"2023-12-19T16:43:24.460161+0000","last_fullsized":"2023-12-19T16:43:24.460161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:55:53.394261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458485+0000","last_change":"2023-12-19T16:43:24.458485+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458485+0000","last_undegraded":"2023-12-19T16:43:24.458485+0000","last_fullsized":"2023-12-19T16:43:24.458485+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:47:45.880613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462394+0000","last_change":"2023-12-19T16:43:24.462394+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462394+0000","last_undegraded":"2023-12-19T16:43:24.462394+0000","last_fullsized":"2023-12-19T16:43:24.462394+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:49:45.049214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458111+0000","last_change":"2023-12-19T16:43:24.458111+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458111+0000","last_undegraded":"2023-12-19T16:43:24.458111+0000","last_fullsized":"2023-12-19T16:43:24.458111+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:08:40.828978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462419+0000","last_change":"2023-12-19T16:43:24.462419+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462419+0000","last_undegraded":"2023-12-19T16:43:24.462419+0000","last_fullsized":"2023-12-19T16:43:24.462419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:34:42.254411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857236+0000","last_change":"2023-12-19T16:43:25.485279+0000","last_active":"2023-12-19T16:43:27.857236+0000","last_peered":"2023-12-19T16:43:27.857236+0000","last_clean":"2023-12-19T16:43:27.857236+0000","last_became_active":"2023-12-19T16:43:25.484950+0000","last_became_peered":"2023-12-19T16:43:25.484950+0000","last_unstale":"2023-12-19T16:43:27.857236+0000","last_undegraded":"2023-12-19T16:43:27.857236+0000","last_fullsized":"2023-12-19T16:43:27.857236+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:30.752537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458128+0000","last_change":"2023-12-19T16:43:24.458128+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458128+0000","last_undegraded":"2023-12-19T16:43:24.458128+0000","last_fullsized":"2023-12-19T16:43:24.458128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:38.921262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462234+0000","last_change":"2023-12-19T16:43:24.462234+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462234+0000","last_undegraded":"2023-12-19T16:43:24.462234+0000","last_fullsized":"2023-12-19T16:43:24.462234+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:22:22.341913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458845+0000","last_change":"2023-12-19T16:43:24.458845+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458845+0000","last_undegraded":"2023-12-19T16:43:24.458845+0000","last_fullsized":"2023-12-19T16:43:24.458845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:03.546398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461365+0000","last_change":"2023-12-19T16:43:24.461365+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461365+0000","last_undegraded":"2023-12-19T16:43:24.461365+0000","last_fullsized":"2023-12-19T16:43:24.461365+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:54.432676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461815+0000","last_change":"2023-12-19T16:43:24.461815+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461815+0000","last_undegraded":"2023-12-19T16:43:24.461815+0000","last_fullsized":"2023-12-19T16:43:24.461815+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:08.723483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463008+0000","last_change":"2023-12-19T16:43:24.463008+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463008+0000","last_undegraded":"2023-12-19T16:43:24.463008+0000","last_fullsized":"2023-12-19T16:43:24.463008+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:39:45.423376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.462583+0000","last_change":"2023-12-19T16:43:24.462583+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.462583+0000","last_undegraded":"2023-12-19T16:43:24.462583+0000","last_fullsized":"2023-12-19T16:43:24.462583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:08.723483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460039+0000","last_change":"2023-12-19T16:43:24.460039+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460039+0000","last_undegraded":"2023-12-19T16:43:24.460039+0000","last_fullsized":"2023-12-19T16:43:24.460039+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:50:19.100332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459813+0000","last_change":"2023-12-19T16:43:24.459813+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459813+0000","last_undegraded":"2023-12-19T16:43:24.459813+0000","last_fullsized":"2023-12-19T16:43:24.459813+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:43:43.874824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459181+0000","last_change":"2023-12-19T16:43:24.459181+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459181+0000","last_undegraded":"2023-12-19T16:43:24.459181+0000","last_fullsized":"2023-12-19T16:43:24.459181+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:30:19.111754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856897+0000","last_change":"2023-12-19T16:43:25.486440+0000","last_active":"2023-12-19T16:43:27.856897+0000","last_peered":"2023-12-19T16:43:27.856897+0000","last_clean":"2023-12-19T16:43:27.856897+0000","last_became_active":"2023-12-19T16:43:25.486112+0000","last_became_peered":"2023-12-19T16:43:25.486112+0000","last_unstale":"2023-12-19T16:43:27.856897+0000","last_undegraded":"2023-12-19T16:43:27.856897+0000","last_fullsized":"2023-12-19T16:43:27.856897+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:47:36.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463128+0000","last_change":"2023-12-19T16:43:24.463128+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463128+0000","last_undegraded":"2023-12-19T16:43:24.463128+0000","last_fullsized":"2023-12-19T16:43:24.463128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:05.966306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459816+0000","last_change":"2023-12-19T16:43:24.459816+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459816+0000","last_undegraded":"2023-12-19T16:43:24.459816+0000","last_fullsized":"2023-12-19T16:43:24.459816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:30:57.984641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.458115+0000","last_change":"2023-12-19T16:43:24.458115+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.458115+0000","last_undegraded":"2023-12-19T16:43:24.458115+0000","last_fullsized":"2023-12-19T16:43:24.458115+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:22:22.341913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461943+0000","last_change":"2023-12-19T16:43:24.461943+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461943+0000","last_undegraded":"2023-12-19T16:43:24.461943+0000","last_fullsized":"2023-12-19T16:43:24.461943+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:48:55.560180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857319+0000","last_change":"2023-12-19T16:43:25.486390+0000","last_active":"2023-12-19T16:43:27.857319+0000","last_peered":"2023-12-19T16:43:27.857319+0000","last_clean":"2023-12-19T16:43:27.857319+0000","last_became_active":"2023-12-19T16:43:25.486082+0000","last_became_peered":"2023-12-19T16:43:25.486082+0000","last_unstale":"2023-12-19T16:43:27.857319+0000","last_undegraded":"2023-12-19T16:43:27.857319+0000","last_fullsized":"2023-12-19T16:43:27.857319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:03:37.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856659+0000","last_change":"2023-12-19T16:43:25.908732+0000","last_active":"2023-12-19T16:43:27.856659+0000","last_peered":"2023-12-19T16:43:27.856659+0000","last_clean":"2023-12-19T16:43:27.856659+0000","last_became_active":"2023-12-19T16:43:25.901839+0000","last_became_peered":"2023-12-19T16:43:25.901839+0000","last_unstale":"2023-12-19T16:43:27.856659+0000","last_undegraded":"2023-12-19T16:43:27.856659+0000","last_fullsized":"2023-12-19T16:43:27.856659+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:20.393474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460656+0000","last_change":"2023-12-19T16:43:24.460656+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460656+0000","last_undegraded":"2023-12-19T16:43:24.460656+0000","last_fullsized":"2023-12-19T16:43:24.460656+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:01.953049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856999+0000","last_change":"2023-12-19T16:43:25.485176+0000","last_active":"2023-12-19T16:43:27.856999+0000","last_peered":"2023-12-19T16:43:27.856999+0000","last_clean":"2023-12-19T16:43:27.856999+0000","last_became_active":"2023-12-19T16:43:25.484842+0000","last_became_peered":"2023-12-19T16:43:25.484842+0000","last_unstale":"2023-12-19T16:43:27.856999+0000","last_undegraded":"2023-12-19T16:43:27.856999+0000","last_fullsized":"2023-12-19T16:43:27.856999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:49.582698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857140+0000","last_change":"2023-12-19T16:43:25.905903+0000","last_active":"2023-12-19T16:43:27.857140+0000","last_peered":"2023-12-19T16:43:27.857140+0000","last_clean":"2023-12-19T16:43:27.857140+0000","last_became_active":"2023-12-19T16:43:25.897356+0000","last_became_peered":"2023-12-19T16:43:25.897356+0000","last_unstale":"2023-12-19T16:43:27.857140+0000","last_undegraded":"2023-12-19T16:43:27.857140+0000","last_fullsized":"2023-12-19T16:43:27.857140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:36:39.824632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.460764+0000","last_change":"2023-12-19T16:43:24.460764+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.460764+0000","last_undegraded":"2023-12-19T16:43:24.460764+0000","last_fullsized":"2023-12-19T16:43:24.460764+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:45.708612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.459149+0000","last_change":"2023-12-19T16:43:24.459149+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.459149+0000","last_undegraded":"2023-12-19T16:43:24.459149+0000","last_fullsized":"2023-12-19T16:43:24.459149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:50:38.846004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.461667+0000","last_change":"2023-12-19T16:43:24.461667+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.461667+0000","last_undegraded":"2023-12-19T16:43:24.461667+0000","last_fullsized":"2023-12-19T16:43:24.461667+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:01.953049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T16:43:25.454250+0000","last_change":"2023-12-19T16:43:25.454250+0000","last_active":"2023-12-19T16:43:25.454250+0000","last_peered":"2023-12-19T16:43:25.454250+0000","last_clean":"2023-12-19T16:43:25.454250+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:25.454250+0000","last_undegraded":"2023-12-19T16:43:25.454250+0000","last_fullsized":"2023-12-19T16:43:25.454250+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":34,"state":"creating+peering","last_fresh":"2023-12-19T16:43:24.463302+0000","last_change":"2023-12-19T16:43:24.463302+0000","last_active":"2023-12-19T16:43:24.447378+0000","last_peered":"2023-12-19T16:43:24.447378+0000","last_clean":"2023-12-19T16:43:24.447378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T16:43:24.463302+0000","last_undegraded":"2023-12-19T16:43:24.463302+0000","last_fullsized":"2023-12-19T16:43:24.463302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:24:17.651413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":173,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-19T16:43:21.144188+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:43:21.144188+0000","last_peered":"2023-12-19T16:43:21.144188+0000","last_clean":"2023-12-19T16:43:21.144188+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:43:21.144188+0000","last_undegraded":"2023-12-19T16:43:21.144188+0000","last_fullsized":"2023-12-19T16:43:21.144188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":1},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":958464,"data_stored":727125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":0.44600000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.53300000000000003}]},{"osd":3,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68200000000000005}]},{"osd":6,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.626},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.65300000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.623}]},{"osd":1,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.3799999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.59699999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.68000000000000005}]},{"osd":3,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.673,"5min":4.673,"15min":4.673},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":0.70799999999999996}]},{"osd":4,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.90900000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.92700000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.94699999999999995}]}]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.86199999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.83699999999999997}]},{"osd":2,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.628}]},{"osd":3,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.60499999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80100000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.73899999999999999}]}]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.71799999999999997}]},{"osd":2,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.621}]},{"osd":3,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.748}]},{"osd":5,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.83199999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85099999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.98399999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.496},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.439}]},{"osd":1,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.45200000000000001}]},{"osd":4,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.78800000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":0.753}]},{"osd":7,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":5.0759999999999996,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.94999999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.5150000000000001,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.45900000000000002}]},{"osd":1,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.7590000000000003,"5min":5.7590000000000003,"15min":5.7590000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":5.8070000000000004,"5min":5.8070000000000004,"15min":5.8070000000000004},"last":0.439}]},{"osd":3,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.7599999999999998,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":5.7190000000000003,"5min":5.7190000000000003,"15min":5.7190000000000003},"last":0.48699999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.73299999999999998}]},{"osd":5,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.878},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.82899999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":0.70299999999999996}]},{"osd":7,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.85999999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":364544,"data_stored":136757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46600000000000003}]},{"osd":3,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.133,"5min":4.133,"15min":4.133},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.437}]},{"osd":4,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.72899999999999998}]}]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":1007616,"data_stored":752284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.255},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.35099999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.59899999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47399999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.64300000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.627}]},{"osd":7,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.61299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:43:34.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:43:34.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb4a1263640 1 Processor -- start 2023-12-19T16:43:34.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb4a1263640 1 -- start start 2023-12-19T16:43:34.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb4a1263640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb49c15f370 0x7fb49c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:34.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb4a1263640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49c06c970 con 0x7fb49c155030 2023-12-19T16:43:34.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb4a1263640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb49c15b1f0 con 0x7fb49c0ce740 2023-12-19T16:43:34.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb4a1263640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49c06c790 con 0x7fb49c15f370 2023-12-19T16:43:34.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb49affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb49c155030 0x7fb49c06c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47382/0 (socket says 172.21.15.26:47382) 2023-12-19T16:43:34.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb49affd640 1 -- 172.21.15.26:0/1653879860 learned_addr learned my addr 172.21.15.26:0/1653879860 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:34.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb493fff640 1 --2- 172.21.15.26:0/1653879860 >> v2:172.21.15.26:3300/0 conn(0x7fb49c15f370 0x7fb49c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:34.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb49a7fc640 1 -- 172.21.15.26:0/1653879860 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516056313 0 0) 0x7fb49c06c970 con 0x7fb49c155030 2023-12-19T16:43:34.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb49a7fc640 1 -- 172.21.15.26:0/1653879860 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb488003530 con 0x7fb49c155030 2023-12-19T16:43:34.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb493fff640 1 -- 172.21.15.26:0/1653879860 >> v1:172.21.15.160:6789/0 conn(0x7fb49c0ce740 legacy=0x7fb49c0ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:34.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb493fff640 1 -- 172.21.15.26:0/1653879860 >> v1:172.21.15.26:6789/0 conn(0x7fb49c155030 legacy=0x7fb49c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:34.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.735+0000 7fb493fff640 1 -- 172.21.15.26:0/1653879860 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c15b4d0 con 0x7fb49c15f370 2023-12-19T16:43:34.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.739+0000 7fb493fff640 1 --2- 172.21.15.26:0/1653879860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb49c15f370 0x7fb49c15bcd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb4840097c0 tx=0x7fb48401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d28ce8c0f7e5c26 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:34.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.739+0000 7fb49a7fc640 1 -- 172.21.15.26:0/1653879860 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840ab670 con 0x7fb49c15f370 2023-12-19T16:43:34.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.739+0000 7fb49a7fc640 1 -- 172.21.15.26:0/1653879860 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4840ab7d0 con 0x7fb49c15f370 2023-12-19T16:43:34.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.739+0000 7fb49a7fc640 1 -- 172.21.15.26:0/1653879860 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840abb60 con 0x7fb49c15f370 2023-12-19T16:43:34.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.739+0000 7fb4a1263640 1 -- 172.21.15.26:0/1653879860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb49c15f370 msgr2=0x7fb49c15bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:34.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.739+0000 7fb4a1263640 1 --2- 172.21.15.26:0/1653879860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb49c15f370 0x7fb49c15bcd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb4840097c0 tx=0x7fb48401c450 comp rx=0 tx=0).stop 2023-12-19T16:43:34.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.739+0000 7fb4a1263640 1 -- 172.21.15.26:0/1653879860 shutdown_connections 2023-12-19T16:43:34.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.739+0000 7fb4a1263640 1 --2- 172.21.15.26:0/1653879860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb49c15f370 0x7fb49c15bcd0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:34.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.739+0000 7fb4a1263640 1 -- 172.21.15.26:0/1653879860 >> 172.21.15.26:0/1653879860 conn(0x7fb49c0658e0 msgr2=0x7fb49c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:34.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.739+0000 7fb4a1263640 1 -- 172.21.15.26:0/1653879860 shutdown_connections 2023-12-19T16:43:34.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.739+0000 7fb4a1263640 1 -- 172.21.15.26:0/1653879860 wait complete. 2023-12-19T16:43:34.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb4a1263640 1 Processor -- start 2023-12-19T16:43:34.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb4a1263640 1 -- start start 2023-12-19T16:43:34.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb4a1263640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb49c0ce740 0x7fb49c1478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:34.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb4a1263640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb49c155030 0x7fb49c14b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:34.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb4a1263640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb49c15f370 0x7fb49c070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:34.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb4a1263640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0af0f0 con 0x7fb49c155030 2023-12-19T16:43:34.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb4a1263640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0aec60 con 0x7fb49c0ce740 2023-12-19T16:43:34.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb4a1263640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c0ae050 con 0x7fb49c15f370 2023-12-19T16:43:34.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb493fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb49c155030 0x7fb49c14b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:34.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb493fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb49c155030 0x7fb49c14b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55354/0 (socket says 172.21.15.26:55354) 2023-12-19T16:43:34.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb493fff640 1 -- 172.21.15.26:0/1522887686 learned_addr learned my addr 172.21.15.26:0/1522887686 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:34.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb49affd640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb49c0ce740 0x7fb49c1478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:34.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb49b7fe640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb49c15f370 0x7fb49c070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:34.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb493fff640 1 -- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb49c15f370 msgr2=0x7fb49c070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:34.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb493fff640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb49c15f370 0x7fb49c070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:34.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb493fff640 1 -- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb49c0ce740 msgr2=0x7fb49c1478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:34.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb493fff640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb49c0ce740 0x7fb49c1478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:34.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb493fff640 1 -- 172.21.15.26:0/1522887686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb484009420 con 0x7fb49c155030 2023-12-19T16:43:34.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb49b7fe640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb49c15f370 0x7fb49c070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:34.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb493fff640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb49c155030 0x7fb49c14b4d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb48401a080 tx=0x7fb4840191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:34.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb4937fe640 1 -- 172.21.15.26:0/1522887686 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840abe90 con 0x7fb49c155030 2023-12-19T16:43:34.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb4937fe640 1 -- 172.21.15.26:0/1522887686 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4840bf450 con 0x7fb49c155030 2023-12-19T16:43:34.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb4937fe640 1 -- 172.21.15.26:0/1522887686 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840bf800 con 0x7fb49c155030 2023-12-19T16:43:34.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb4a1263640 1 -- 172.21.15.26:0/1522887686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49c0cf0e0 con 0x7fb49c155030 2023-12-19T16:43:34.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.743+0000 7fb4a1263640 1 -- 172.21.15.26:0/1522887686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49c0ae610 con 0x7fb49c155030 2023-12-19T16:43:34.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.747+0000 7fb4a1263640 1 -- 172.21.15.26:0/1522887686 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49c149200 con 0x7fb49c155030 2023-12-19T16:43:34.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.747+0000 7fb4937fe640 1 -- 172.21.15.26:0/1522887686 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb4840bf960 con 0x7fb49c155030 2023-12-19T16:43:34.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.747+0000 7fb4937fe640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb47c065a60 0x7fb47c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:34.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.747+0000 7fb4937fe640 1 -- 172.21.15.26:0/1522887686 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb48411da60 con 0x7fb49c155030 2023-12-19T16:43:34.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.747+0000 7fb49affd640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb47c065a60 0x7fb47c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:34.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.747+0000 7fb49affd640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb47c065a60 0x7fb47c067f20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0011f0 tx=0x7fb48c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:34.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.751+0000 7fb4937fe640 1 -- 172.21.15.26:0/1522887686 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb4840eb410 con 0x7fb49c155030 2023-12-19T16:43:34.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.903+0000 7fb4a1263640 1 -- 172.21.15.26:0/1522887686 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb49c0cef60 con 0x7fb47c065a60 2023-12-19T16:43:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.915+0000 7fb4937fe640 1 -- 172.21.15.26:0/1522887686 <== mgr.4106 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288614 (secure 0 0 0) 0x7fb48c001cf0 con 0x7fb47c065a60 2023-12-19T16:43:34.921 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:34.923 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:43:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.915+0000 7fb4a1263640 1 -- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb47c065a60 msgr2=0x7fb47c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.915+0000 7fb4a1263640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb47c065a60 0x7fb47c067f20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0011f0 tx=0x7fb48c009040 comp rx=0 tx=0).stop 2023-12-19T16:43:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.915+0000 7fb4a1263640 1 -- 172.21.15.26:0/1522887686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb49c155030 msgr2=0x7fb49c14b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.915+0000 7fb4a1263640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb49c155030 0x7fb49c14b4d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb48401a080 tx=0x7fb4840191f0 comp rx=0 tx=0).stop 2023-12-19T16:43:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.915+0000 7fb4a1263640 1 -- 172.21.15.26:0/1522887686 shutdown_connections 2023-12-19T16:43:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.915+0000 7fb4a1263640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb47c065a60 0x7fb47c067f20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.915+0000 7fb4a1263640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb49c15f370 0x7fb49c070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.915+0000 7fb4a1263640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb49c155030 0x7fb49c14b4d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:34.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.915+0000 7fb4a1263640 1 --2- 172.21.15.26:0/1522887686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb49c0ce740 0x7fb49c1478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:34.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.915+0000 7fb4a1263640 1 -- 172.21.15.26:0/1522887686 >> 172.21.15.26:0/1522887686 conn(0x7fb49c0658e0 msgr2=0x7fb49c065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:34.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.915+0000 7fb4a1263640 1 -- 172.21.15.26:0/1522887686 shutdown_connections 2023-12-19T16:43:34.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:34.915+0000 7fb4a1263640 1 -- 172.21.15.26:0/1522887686 wait complete. 2023-12-19T16:43:34.941 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2023-12-19T16:43:33.651023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236456,"kb_used_data":4456,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754738264,"statfs":{"total":773094113280,"available":772851982336,"internally_reserved":0,"allocated":4562944,"data_stored":2480788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001846"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856703+0000","last_change":"2023-12-19T16:43:25.896447+0000","last_active":"2023-12-19T16:43:27.856703+0000","last_peered":"2023-12-19T16:43:27.856703+0000","last_clean":"2023-12-19T16:43:27.856703+0000","last_became_active":"2023-12-19T16:43:25.896075+0000","last_became_peered":"2023-12-19T16:43:25.896075+0000","last_unstale":"2023-12-19T16:43:27.856703+0000","last_undegraded":"2023-12-19T16:43:27.856703+0000","last_fullsized":"2023-12-19T16:43:27.856703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:49:46.056086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857857+0000","last_change":"2023-12-19T16:43:25.906364+0000","last_active":"2023-12-19T16:43:27.857857+0000","last_peered":"2023-12-19T16:43:27.857857+0000","last_clean":"2023-12-19T16:43:27.857857+0000","last_became_active":"2023-12-19T16:43:25.904111+0000","last_became_peered":"2023-12-19T16:43:25.904111+0000","last_unstale":"2023-12-19T16:43:27.857857+0000","last_undegraded":"2023-12-19T16:43:27.857857+0000","last_fullsized":"2023-12-19T16:43:27.857857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:49:05.605334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856535+0000","last_change":"2023-12-19T16:43:25.901076+0000","last_active":"2023-12-19T16:43:27.856535+0000","last_peered":"2023-12-19T16:43:27.856535+0000","last_clean":"2023-12-19T16:43:27.856535+0000","last_became_active":"2023-12-19T16:43:25.900466+0000","last_became_peered":"2023-12-19T16:43:25.900466+0000","last_unstale":"2023-12-19T16:43:27.856535+0000","last_undegraded":"2023-12-19T16:43:27.856535+0000","last_fullsized":"2023-12-19T16:43:27.856535+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:05:47.124261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.858021+0000","last_change":"2023-12-19T16:43:25.891582+0000","last_active":"2023-12-19T16:43:27.858021+0000","last_peered":"2023-12-19T16:43:27.858021+0000","last_clean":"2023-12-19T16:43:27.858021+0000","last_became_active":"2023-12-19T16:43:25.891335+0000","last_became_peered":"2023-12-19T16:43:25.891335+0000","last_unstale":"2023-12-19T16:43:27.858021+0000","last_undegraded":"2023-12-19T16:43:27.858021+0000","last_fullsized":"2023-12-19T16:43:27.858021+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:33:32.718755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.880741+0000","last_change":"2023-12-19T16:43:25.892341+0000","last_active":"2023-12-19T16:43:26.880741+0000","last_peered":"2023-12-19T16:43:26.880741+0000","last_clean":"2023-12-19T16:43:26.880741+0000","last_became_active":"2023-12-19T16:43:25.891307+0000","last_became_peered":"2023-12-19T16:43:25.891307+0000","last_unstale":"2023-12-19T16:43:26.880741+0000","last_undegraded":"2023-12-19T16:43:26.880741+0000","last_fullsized":"2023-12-19T16:43:26.880741+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:09.730355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857120+0000","last_change":"2023-12-19T16:43:25.897019+0000","last_active":"2023-12-19T16:43:27.857120+0000","last_peered":"2023-12-19T16:43:27.857120+0000","last_clean":"2023-12-19T16:43:27.857120+0000","last_became_active":"2023-12-19T16:43:25.895219+0000","last_became_peered":"2023-12-19T16:43:25.895219+0000","last_unstale":"2023-12-19T16:43:27.857120+0000","last_undegraded":"2023-12-19T16:43:27.857120+0000","last_fullsized":"2023-12-19T16:43:27.857120+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:22:28.451150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.885681+0000","last_change":"2023-12-19T16:43:25.903985+0000","last_active":"2023-12-19T16:43:26.885681+0000","last_peered":"2023-12-19T16:43:26.885681+0000","last_clean":"2023-12-19T16:43:26.885681+0000","last_became_active":"2023-12-19T16:43:25.903701+0000","last_became_peered":"2023-12-19T16:43:25.903701+0000","last_unstale":"2023-12-19T16:43:26.885681+0000","last_undegraded":"2023-12-19T16:43:26.885681+0000","last_fullsized":"2023-12-19T16:43:26.885681+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:42:10.952399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.885795+0000","last_change":"2023-12-19T16:43:25.899013+0000","last_active":"2023-12-19T16:43:26.885795+0000","last_peered":"2023-12-19T16:43:26.885795+0000","last_clean":"2023-12-19T16:43:26.885795+0000","last_became_active":"2023-12-19T16:43:25.898735+0000","last_became_peered":"2023-12-19T16:43:25.898735+0000","last_unstale":"2023-12-19T16:43:26.885795+0000","last_undegraded":"2023-12-19T16:43:26.885795+0000","last_fullsized":"2023-12-19T16:43:26.885795+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:10:08.483249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.886506+0000","last_change":"2023-12-19T16:43:25.902133+0000","last_active":"2023-12-19T16:43:26.886506+0000","last_peered":"2023-12-19T16:43:26.886506+0000","last_clean":"2023-12-19T16:43:26.886506+0000","last_became_active":"2023-12-19T16:43:25.901875+0000","last_became_peered":"2023-12-19T16:43:25.901875+0000","last_unstale":"2023-12-19T16:43:26.886506+0000","last_undegraded":"2023-12-19T16:43:26.886506+0000","last_fullsized":"2023-12-19T16:43:26.886506+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:39:46.430248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857817+0000","last_change":"2023-12-19T16:43:25.896072+0000","last_active":"2023-12-19T16:43:27.857817+0000","last_peered":"2023-12-19T16:43:27.857817+0000","last_clean":"2023-12-19T16:43:27.857817+0000","last_became_active":"2023-12-19T16:43:25.895798+0000","last_became_peered":"2023-12-19T16:43:25.895798+0000","last_unstale":"2023-12-19T16:43:27.857817+0000","last_undegraded":"2023-12-19T16:43:27.857817+0000","last_fullsized":"2023-12-19T16:43:27.857817+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:42:10.952399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.885853+0000","last_change":"2023-12-19T16:43:25.910608+0000","last_active":"2023-12-19T16:43:26.885853+0000","last_peered":"2023-12-19T16:43:26.885853+0000","last_clean":"2023-12-19T16:43:26.885853+0000","last_became_active":"2023-12-19T16:43:25.910292+0000","last_became_peered":"2023-12-19T16:43:25.910292+0000","last_unstale":"2023-12-19T16:43:26.885853+0000","last_undegraded":"2023-12-19T16:43:26.885853+0000","last_fullsized":"2023-12-19T16:43:26.885853+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:29:42.005516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856160+0000","last_change":"2023-12-19T16:43:25.891534+0000","last_active":"2023-12-19T16:43:27.856160+0000","last_peered":"2023-12-19T16:43:27.856160+0000","last_clean":"2023-12-19T16:43:27.856160+0000","last_became_active":"2023-12-19T16:43:25.891227+0000","last_became_peered":"2023-12-19T16:43:25.891227+0000","last_unstale":"2023-12-19T16:43:27.856160+0000","last_undegraded":"2023-12-19T16:43:27.856160+0000","last_fullsized":"2023-12-19T16:43:27.856160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:22:28.451150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.880590+0000","last_change":"2023-12-19T16:43:25.900282+0000","last_active":"2023-12-19T16:43:26.880590+0000","last_peered":"2023-12-19T16:43:26.880590+0000","last_clean":"2023-12-19T16:43:26.880590+0000","last_became_active":"2023-12-19T16:43:25.900012+0000","last_became_peered":"2023-12-19T16:43:25.900012+0000","last_unstale":"2023-12-19T16:43:26.880590+0000","last_undegraded":"2023-12-19T16:43:26.880590+0000","last_fullsized":"2023-12-19T16:43:26.880590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:55.439548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857385+0000","last_change":"2023-12-19T16:43:25.891036+0000","last_active":"2023-12-19T16:43:27.857385+0000","last_peered":"2023-12-19T16:43:27.857385+0000","last_clean":"2023-12-19T16:43:27.857385+0000","last_became_active":"2023-12-19T16:43:25.890734+0000","last_became_peered":"2023-12-19T16:43:25.890734+0000","last_unstale":"2023-12-19T16:43:27.857385+0000","last_undegraded":"2023-12-19T16:43:27.857385+0000","last_fullsized":"2023-12-19T16:43:27.857385+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:05.197926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856220+0000","last_change":"2023-12-19T16:43:25.898057+0000","last_active":"2023-12-19T16:43:27.856220+0000","last_peered":"2023-12-19T16:43:27.856220+0000","last_clean":"2023-12-19T16:43:27.856220+0000","last_became_active":"2023-12-19T16:43:25.897769+0000","last_became_peered":"2023-12-19T16:43:25.897769+0000","last_unstale":"2023-12-19T16:43:27.856220+0000","last_undegraded":"2023-12-19T16:43:27.856220+0000","last_fullsized":"2023-12-19T16:43:27.856220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:24:10.489956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.885613+0000","last_change":"2023-12-19T16:43:25.903376+0000","last_active":"2023-12-19T16:43:26.885613+0000","last_peered":"2023-12-19T16:43:26.885613+0000","last_clean":"2023-12-19T16:43:26.885613+0000","last_became_active":"2023-12-19T16:43:25.902968+0000","last_became_peered":"2023-12-19T16:43:25.902968+0000","last_unstale":"2023-12-19T16:43:26.885613+0000","last_undegraded":"2023-12-19T16:43:26.885613+0000","last_fullsized":"2023-12-19T16:43:26.885613+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:40.028569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.886034+0000","last_change":"2023-12-19T16:43:25.905824+0000","last_active":"2023-12-19T16:43:26.886034+0000","last_peered":"2023-12-19T16:43:26.886034+0000","last_clean":"2023-12-19T16:43:26.886034+0000","last_became_active":"2023-12-19T16:43:25.905487+0000","last_became_peered":"2023-12-19T16:43:25.905487+0000","last_unstale":"2023-12-19T16:43:26.886034+0000","last_undegraded":"2023-12-19T16:43:26.886034+0000","last_fullsized":"2023-12-19T16:43:26.886034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:37:54.151970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857961+0000","last_change":"2023-12-19T16:43:25.902712+0000","last_active":"2023-12-19T16:43:27.857961+0000","last_peered":"2023-12-19T16:43:27.857961+0000","last_clean":"2023-12-19T16:43:27.857961+0000","last_became_active":"2023-12-19T16:43:25.902378+0000","last_became_peered":"2023-12-19T16:43:25.902378+0000","last_unstale":"2023-12-19T16:43:27.857961+0000","last_undegraded":"2023-12-19T16:43:27.857961+0000","last_fullsized":"2023-12-19T16:43:27.857961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:05:47.124261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857356+0000","last_change":"2023-12-19T16:43:25.905970+0000","last_active":"2023-12-19T16:43:27.857356+0000","last_peered":"2023-12-19T16:43:27.857356+0000","last_clean":"2023-12-19T16:43:27.857356+0000","last_became_active":"2023-12-19T16:43:25.905767+0000","last_became_peered":"2023-12-19T16:43:25.905767+0000","last_unstale":"2023-12-19T16:43:27.857356+0000","last_undegraded":"2023-12-19T16:43:27.857356+0000","last_fullsized":"2023-12-19T16:43:27.857356+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:38:08.385496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.880780+0000","last_change":"2023-12-19T16:43:25.895929+0000","last_active":"2023-12-19T16:43:26.880780+0000","last_peered":"2023-12-19T16:43:26.880780+0000","last_clean":"2023-12-19T16:43:26.880780+0000","last_became_active":"2023-12-19T16:43:25.895670+0000","last_became_peered":"2023-12-19T16:43:25.895670+0000","last_unstale":"2023-12-19T16:43:26.880780+0000","last_undegraded":"2023-12-19T16:43:26.880780+0000","last_fullsized":"2023-12-19T16:43:26.880780+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:48:56.567052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.886077+0000","last_change":"2023-12-19T16:43:25.894603+0000","last_active":"2023-12-19T16:43:26.886077+0000","last_peered":"2023-12-19T16:43:26.886077+0000","last_clean":"2023-12-19T16:43:26.886077+0000","last_became_active":"2023-12-19T16:43:25.891512+0000","last_became_peered":"2023-12-19T16:43:25.891512+0000","last_unstale":"2023-12-19T16:43:26.886077+0000","last_undegraded":"2023-12-19T16:43:26.886077+0000","last_fullsized":"2023-12-19T16:43:26.886077+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:22:23.348785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.886030+0000","last_change":"2023-12-19T16:43:25.892683+0000","last_active":"2023-12-19T16:43:26.886030+0000","last_peered":"2023-12-19T16:43:26.886030+0000","last_clean":"2023-12-19T16:43:26.886030+0000","last_became_active":"2023-12-19T16:43:25.891929+0000","last_became_peered":"2023-12-19T16:43:25.891929+0000","last_unstale":"2023-12-19T16:43:26.886030+0000","last_undegraded":"2023-12-19T16:43:26.886030+0000","last_fullsized":"2023-12-19T16:43:26.886030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:14.135290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856487+0000","last_change":"2023-12-19T16:43:25.892565+0000","last_active":"2023-12-19T16:43:27.856487+0000","last_peered":"2023-12-19T16:43:27.856487+0000","last_clean":"2023-12-19T16:43:27.856487+0000","last_became_active":"2023-12-19T16:43:25.892189+0000","last_became_peered":"2023-12-19T16:43:25.892189+0000","last_unstale":"2023-12-19T16:43:27.856487+0000","last_undegraded":"2023-12-19T16:43:27.856487+0000","last_fullsized":"2023-12-19T16:43:27.856487+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:20.383856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856452+0000","last_change":"2023-12-19T16:43:25.889051+0000","last_active":"2023-12-19T16:43:27.856452+0000","last_peered":"2023-12-19T16:43:27.856452+0000","last_clean":"2023-12-19T16:43:27.856452+0000","last_became_active":"2023-12-19T16:43:25.888706+0000","last_became_peered":"2023-12-19T16:43:25.888706+0000","last_unstale":"2023-12-19T16:43:27.856452+0000","last_undegraded":"2023-12-19T16:43:27.856452+0000","last_fullsized":"2023-12-19T16:43:27.856452+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:37:54.151970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.858033+0000","last_change":"2023-12-19T16:43:25.899867+0000","last_active":"2023-12-19T16:43:27.858033+0000","last_peered":"2023-12-19T16:43:27.858033+0000","last_clean":"2023-12-19T16:43:27.858033+0000","last_became_active":"2023-12-19T16:43:25.899532+0000","last_became_peered":"2023-12-19T16:43:25.899532+0000","last_unstale":"2023-12-19T16:43:27.858033+0000","last_undegraded":"2023-12-19T16:43:27.858033+0000","last_fullsized":"2023-12-19T16:43:27.858033+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:24:10.489956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.858058+0000","last_change":"2023-12-19T16:43:25.899574+0000","last_active":"2023-12-19T16:43:27.858058+0000","last_peered":"2023-12-19T16:43:27.858058+0000","last_clean":"2023-12-19T16:43:27.858058+0000","last_became_active":"2023-12-19T16:43:25.899294+0000","last_became_peered":"2023-12-19T16:43:25.899294+0000","last_unstale":"2023-12-19T16:43:27.858058+0000","last_undegraded":"2023-12-19T16:43:27.858058+0000","last_fullsized":"2023-12-19T16:43:27.858058+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:03:10.533550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.858012+0000","last_change":"2023-12-19T16:43:25.883989+0000","last_active":"2023-12-19T16:43:27.858012+0000","last_peered":"2023-12-19T16:43:27.858012+0000","last_clean":"2023-12-19T16:43:27.858012+0000","last_became_active":"2023-12-19T16:43:25.883729+0000","last_became_peered":"2023-12-19T16:43:25.883729+0000","last_unstale":"2023-12-19T16:43:27.858012+0000","last_undegraded":"2023-12-19T16:43:27.858012+0000","last_fullsized":"2023-12-19T16:43:27.858012+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:29.924684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856581+0000","last_change":"2023-12-19T16:43:25.889190+0000","last_active":"2023-12-19T16:43:27.856581+0000","last_peered":"2023-12-19T16:43:27.856581+0000","last_clean":"2023-12-19T16:43:27.856581+0000","last_became_active":"2023-12-19T16:43:25.888861+0000","last_became_peered":"2023-12-19T16:43:25.888861+0000","last_unstale":"2023-12-19T16:43:27.856581+0000","last_undegraded":"2023-12-19T16:43:27.856581+0000","last_fullsized":"2023-12-19T16:43:27.856581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:42:10.952399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.886110+0000","last_change":"2023-12-19T16:43:25.894821+0000","last_active":"2023-12-19T16:43:26.886110+0000","last_peered":"2023-12-19T16:43:26.886110+0000","last_clean":"2023-12-19T16:43:26.886110+0000","last_became_active":"2023-12-19T16:43:25.894534+0000","last_became_peered":"2023-12-19T16:43:25.894534+0000","last_unstale":"2023-12-19T16:43:26.886110+0000","last_undegraded":"2023-12-19T16:43:26.886110+0000","last_fullsized":"2023-12-19T16:43:26.886110+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:40.028569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.885350+0000","last_change":"2023-12-19T16:43:25.898292+0000","last_active":"2023-12-19T16:43:26.885350+0000","last_peered":"2023-12-19T16:43:26.885350+0000","last_clean":"2023-12-19T16:43:26.885350+0000","last_became_active":"2023-12-19T16:43:25.898014+0000","last_became_peered":"2023-12-19T16:43:25.898014+0000","last_unstale":"2023-12-19T16:43:26.885350+0000","last_undegraded":"2023-12-19T16:43:26.885350+0000","last_fullsized":"2023-12-19T16:43:26.885350+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:29.924684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856271+0000","last_change":"2023-12-19T16:43:25.895612+0000","last_active":"2023-12-19T16:43:27.856271+0000","last_peered":"2023-12-19T16:43:27.856271+0000","last_clean":"2023-12-19T16:43:27.856271+0000","last_became_active":"2023-12-19T16:43:25.894992+0000","last_became_peered":"2023-12-19T16:43:25.894992+0000","last_unstale":"2023-12-19T16:43:27.856271+0000","last_undegraded":"2023-12-19T16:43:27.856271+0000","last_fullsized":"2023-12-19T16:43:27.856271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:39.928134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.885323+0000","last_change":"2023-12-19T16:43:25.895799+0000","last_active":"2023-12-19T16:43:26.885323+0000","last_peered":"2023-12-19T16:43:26.885323+0000","last_clean":"2023-12-19T16:43:26.885323+0000","last_became_active":"2023-12-19T16:43:25.895551+0000","last_became_peered":"2023-12-19T16:43:25.895551+0000","last_unstale":"2023-12-19T16:43:26.885323+0000","last_undegraded":"2023-12-19T16:43:26.885323+0000","last_fullsized":"2023-12-19T16:43:26.885323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:33:32.718755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857570+0000","last_change":"2023-12-19T16:43:25.893897+0000","last_active":"2023-12-19T16:43:27.857570+0000","last_peered":"2023-12-19T16:43:27.857570+0000","last_clean":"2023-12-19T16:43:27.857570+0000","last_became_active":"2023-12-19T16:43:25.893654+0000","last_became_peered":"2023-12-19T16:43:25.893654+0000","last_unstale":"2023-12-19T16:43:27.857570+0000","last_undegraded":"2023-12-19T16:43:27.857570+0000","last_fullsized":"2023-12-19T16:43:27.857570+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:51:10.988939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"36'4","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.903575+0000","last_change":"2023-12-19T16:43:25.485001+0000","last_active":"2023-12-19T16:43:25.903575+0000","last_peered":"2023-12-19T16:43:25.903575+0000","last_clean":"2023-12-19T16:43:25.903575+0000","last_became_active":"2023-12-19T16:43:25.484725+0000","last_became_peered":"2023-12-19T16:43:25.484725+0000","last_unstale":"2023-12-19T16:43:25.903575+0000","last_undegraded":"2023-12-19T16:43:25.903575+0000","last_fullsized":"2023-12-19T16:43:25.903575+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:39.252728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856744+0000","last_change":"2023-12-19T16:43:25.899009+0000","last_active":"2023-12-19T16:43:27.856744+0000","last_peered":"2023-12-19T16:43:27.856744+0000","last_clean":"2023-12-19T16:43:27.856744+0000","last_became_active":"2023-12-19T16:43:25.896852+0000","last_became_peered":"2023-12-19T16:43:25.896852+0000","last_unstale":"2023-12-19T16:43:27.856744+0000","last_undegraded":"2023-12-19T16:43:27.856744+0000","last_fullsized":"2023-12-19T16:43:27.856744+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:39:46.430248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857152+0000","last_change":"2023-12-19T16:43:25.482086+0000","last_active":"2023-12-19T16:43:27.857152+0000","last_peered":"2023-12-19T16:43:27.857152+0000","last_clean":"2023-12-19T16:43:27.857152+0000","last_became_active":"2023-12-19T16:43:25.481816+0000","last_became_peered":"2023-12-19T16:43:25.481816+0000","last_unstale":"2023-12-19T16:43:27.857152+0000","last_undegraded":"2023-12-19T16:43:27.857152+0000","last_fullsized":"2023-12-19T16:43:27.857152+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:10.136641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.886289+0000","last_change":"2023-12-19T16:43:25.887884+0000","last_active":"2023-12-19T16:43:26.886289+0000","last_peered":"2023-12-19T16:43:26.886289+0000","last_clean":"2023-12-19T16:43:26.886289+0000","last_became_active":"2023-12-19T16:43:25.887569+0000","last_became_peered":"2023-12-19T16:43:25.887569+0000","last_unstale":"2023-12-19T16:43:26.886289+0000","last_undegraded":"2023-12-19T16:43:26.886289+0000","last_fullsized":"2023-12-19T16:43:26.886289+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:24:10.489956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.910271+0000","last_change":"2023-12-19T16:43:25.488952+0000","last_active":"2023-12-19T16:43:25.910271+0000","last_peered":"2023-12-19T16:43:25.910271+0000","last_clean":"2023-12-19T16:43:25.910271+0000","last_became_active":"2023-12-19T16:43:25.488636+0000","last_became_peered":"2023-12-19T16:43:25.488636+0000","last_unstale":"2023-12-19T16:43:25.910271+0000","last_undegraded":"2023-12-19T16:43:25.910271+0000","last_fullsized":"2023-12-19T16:43:25.910271+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:38.921262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.858133+0000","last_change":"2023-12-19T16:43:25.889673+0000","last_active":"2023-12-19T16:43:27.858133+0000","last_peered":"2023-12-19T16:43:27.858133+0000","last_clean":"2023-12-19T16:43:27.858133+0000","last_became_active":"2023-12-19T16:43:25.889363+0000","last_became_peered":"2023-12-19T16:43:25.889363+0000","last_unstale":"2023-12-19T16:43:27.858133+0000","last_undegraded":"2023-12-19T16:43:27.858133+0000","last_fullsized":"2023-12-19T16:43:27.858133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:29:42.005516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.910368+0000","last_change":"2023-12-19T16:43:25.489865+0000","last_active":"2023-12-19T16:43:25.910368+0000","last_peered":"2023-12-19T16:43:25.910368+0000","last_clean":"2023-12-19T16:43:25.910368+0000","last_became_active":"2023-12-19T16:43:25.489580+0000","last_became_peered":"2023-12-19T16:43:25.489580+0000","last_unstale":"2023-12-19T16:43:25.910368+0000","last_undegraded":"2023-12-19T16:43:25.910368+0000","last_fullsized":"2023-12-19T16:43:25.910368+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:52:24.576402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.885713+0000","last_change":"2023-12-19T16:43:25.896467+0000","last_active":"2023-12-19T16:43:26.885713+0000","last_peered":"2023-12-19T16:43:26.885713+0000","last_clean":"2023-12-19T16:43:26.885713+0000","last_became_active":"2023-12-19T16:43:25.896210+0000","last_became_peered":"2023-12-19T16:43:25.896210+0000","last_unstale":"2023-12-19T16:43:26.885713+0000","last_undegraded":"2023-12-19T16:43:26.885713+0000","last_fullsized":"2023-12-19T16:43:26.885713+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:27.606909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.906556+0000","last_change":"2023-12-19T16:43:25.492913+0000","last_active":"2023-12-19T16:43:25.906556+0000","last_peered":"2023-12-19T16:43:25.906556+0000","last_clean":"2023-12-19T16:43:25.906556+0000","last_became_active":"2023-12-19T16:43:25.492692+0000","last_became_peered":"2023-12-19T16:43:25.492692+0000","last_unstale":"2023-12-19T16:43:25.906556+0000","last_undegraded":"2023-12-19T16:43:25.906556+0000","last_fullsized":"2023-12-19T16:43:25.906556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:55:53.394261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856136+0000","last_change":"2023-12-19T16:43:25.893444+0000","last_active":"2023-12-19T16:43:27.856136+0000","last_peered":"2023-12-19T16:43:27.856136+0000","last_clean":"2023-12-19T16:43:27.856136+0000","last_became_active":"2023-12-19T16:43:25.892933+0000","last_became_peered":"2023-12-19T16:43:25.892933+0000","last_unstale":"2023-12-19T16:43:27.856136+0000","last_undegraded":"2023-12-19T16:43:27.856136+0000","last_fullsized":"2023-12-19T16:43:27.856136+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:49:05.605334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.906505+0000","last_change":"2023-12-19T16:43:25.489544+0000","last_active":"2023-12-19T16:43:25.906505+0000","last_peered":"2023-12-19T16:43:25.906505+0000","last_clean":"2023-12-19T16:43:25.906505+0000","last_became_active":"2023-12-19T16:43:25.489220+0000","last_became_peered":"2023-12-19T16:43:25.489220+0000","last_unstale":"2023-12-19T16:43:25.906505+0000","last_undegraded":"2023-12-19T16:43:25.906505+0000","last_fullsized":"2023-12-19T16:43:25.906505+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:47:45.880613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.885471+0000","last_change":"2023-12-19T16:43:25.895219+0000","last_active":"2023-12-19T16:43:26.885471+0000","last_peered":"2023-12-19T16:43:26.885471+0000","last_clean":"2023-12-19T16:43:26.885471+0000","last_became_active":"2023-12-19T16:43:25.894941+0000","last_became_peered":"2023-12-19T16:43:25.894941+0000","last_unstale":"2023-12-19T16:43:26.885471+0000","last_undegraded":"2023-12-19T16:43:26.885471+0000","last_fullsized":"2023-12-19T16:43:26.885471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:39.928134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.885069+0000","last_change":"2023-12-19T16:43:25.485071+0000","last_active":"2023-12-19T16:43:25.885069+0000","last_peered":"2023-12-19T16:43:25.885069+0000","last_clean":"2023-12-19T16:43:25.885069+0000","last_became_active":"2023-12-19T16:43:25.484799+0000","last_became_peered":"2023-12-19T16:43:25.484799+0000","last_unstale":"2023-12-19T16:43:25.885069+0000","last_undegraded":"2023-12-19T16:43:25.885069+0000","last_fullsized":"2023-12-19T16:43:25.885069+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:49:45.049214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857257+0000","last_change":"2023-12-19T16:43:25.886936+0000","last_active":"2023-12-19T16:43:27.857257+0000","last_peered":"2023-12-19T16:43:27.857257+0000","last_clean":"2023-12-19T16:43:27.857257+0000","last_became_active":"2023-12-19T16:43:25.886644+0000","last_became_peered":"2023-12-19T16:43:25.886644+0000","last_unstale":"2023-12-19T16:43:27.857257+0000","last_undegraded":"2023-12-19T16:43:27.857257+0000","last_fullsized":"2023-12-19T16:43:27.857257+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:27.606909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.874567+0000","last_change":"2023-12-19T16:43:25.483626+0000","last_active":"2023-12-19T16:43:25.874567+0000","last_peered":"2023-12-19T16:43:25.874567+0000","last_clean":"2023-12-19T16:43:25.874567+0000","last_became_active":"2023-12-19T16:43:25.483316+0000","last_became_peered":"2023-12-19T16:43:25.483316+0000","last_unstale":"2023-12-19T16:43:25.874567+0000","last_undegraded":"2023-12-19T16:43:25.874567+0000","last_fullsized":"2023-12-19T16:43:25.874567+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:08:40.828978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856415+0000","last_change":"2023-12-19T16:43:25.890834+0000","last_active":"2023-12-19T16:43:27.856415+0000","last_peered":"2023-12-19T16:43:27.856415+0000","last_clean":"2023-12-19T16:43:27.856415+0000","last_became_active":"2023-12-19T16:43:25.890324+0000","last_became_peered":"2023-12-19T16:43:25.890324+0000","last_unstale":"2023-12-19T16:43:27.856415+0000","last_undegraded":"2023-12-19T16:43:27.856415+0000","last_fullsized":"2023-12-19T16:43:27.856415+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:05.197926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.885757+0000","last_change":"2023-12-19T16:43:25.479699+0000","last_active":"2023-12-19T16:43:25.885757+0000","last_peered":"2023-12-19T16:43:25.885757+0000","last_clean":"2023-12-19T16:43:25.885757+0000","last_became_active":"2023-12-19T16:43:25.479443+0000","last_became_peered":"2023-12-19T16:43:25.479443+0000","last_unstale":"2023-12-19T16:43:25.885757+0000","last_undegraded":"2023-12-19T16:43:25.885757+0000","last_fullsized":"2023-12-19T16:43:25.885757+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:34:42.254411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857489+0000","last_change":"2023-12-19T16:43:25.902749+0000","last_active":"2023-12-19T16:43:27.857489+0000","last_peered":"2023-12-19T16:43:27.857489+0000","last_clean":"2023-12-19T16:43:27.857489+0000","last_became_active":"2023-12-19T16:43:25.902427+0000","last_became_peered":"2023-12-19T16:43:25.902427+0000","last_unstale":"2023-12-19T16:43:27.857489+0000","last_undegraded":"2023-12-19T16:43:27.857489+0000","last_fullsized":"2023-12-19T16:43:27.857489+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:30:58.401961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857236+0000","last_change":"2023-12-19T16:43:25.485279+0000","last_active":"2023-12-19T16:43:27.857236+0000","last_peered":"2023-12-19T16:43:27.857236+0000","last_clean":"2023-12-19T16:43:27.857236+0000","last_became_active":"2023-12-19T16:43:25.484950+0000","last_became_peered":"2023-12-19T16:43:25.484950+0000","last_unstale":"2023-12-19T16:43:27.857236+0000","last_undegraded":"2023-12-19T16:43:27.857236+0000","last_fullsized":"2023-12-19T16:43:27.857236+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:30.752537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857635+0000","last_change":"2023-12-19T16:43:25.900706+0000","last_active":"2023-12-19T16:43:27.857635+0000","last_peered":"2023-12-19T16:43:27.857635+0000","last_clean":"2023-12-19T16:43:27.857635+0000","last_became_active":"2023-12-19T16:43:25.900478+0000","last_became_peered":"2023-12-19T16:43:25.900478+0000","last_unstale":"2023-12-19T16:43:27.857635+0000","last_undegraded":"2023-12-19T16:43:27.857635+0000","last_fullsized":"2023-12-19T16:43:27.857635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:22:09.023704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"36'3","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.894088+0000","last_change":"2023-12-19T16:43:25.480969+0000","last_active":"2023-12-19T16:43:25.894088+0000","last_peered":"2023-12-19T16:43:25.894088+0000","last_clean":"2023-12-19T16:43:25.894088+0000","last_became_active":"2023-12-19T16:43:25.480608+0000","last_became_peered":"2023-12-19T16:43:25.480608+0000","last_unstale":"2023-12-19T16:43:25.894088+0000","last_undegraded":"2023-12-19T16:43:25.894088+0000","last_fullsized":"2023-12-19T16:43:25.894088+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:38.921262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856684+0000","last_change":"2023-12-19T16:43:25.901274+0000","last_active":"2023-12-19T16:43:27.856684+0000","last_peered":"2023-12-19T16:43:27.856684+0000","last_clean":"2023-12-19T16:43:27.856684+0000","last_became_active":"2023-12-19T16:43:25.900818+0000","last_became_peered":"2023-12-19T16:43:25.900818+0000","last_unstale":"2023-12-19T16:43:27.856684+0000","last_undegraded":"2023-12-19T16:43:27.856684+0000","last_fullsized":"2023-12-19T16:43:27.856684+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:16:43.256480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"36'6","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.913458+0000","last_change":"2023-12-19T16:43:25.491969+0000","last_active":"2023-12-19T16:43:25.913458+0000","last_peered":"2023-12-19T16:43:25.913458+0000","last_clean":"2023-12-19T16:43:25.913458+0000","last_became_active":"2023-12-19T16:43:25.489932+0000","last_became_peered":"2023-12-19T16:43:25.489932+0000","last_unstale":"2023-12-19T16:43:25.913458+0000","last_undegraded":"2023-12-19T16:43:25.913458+0000","last_fullsized":"2023-12-19T16:43:25.913458+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:22:22.341913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857252+0000","last_change":"2023-12-19T16:43:25.882040+0000","last_active":"2023-12-19T16:43:27.857252+0000","last_peered":"2023-12-19T16:43:27.857252+0000","last_clean":"2023-12-19T16:43:27.857252+0000","last_became_active":"2023-12-19T16:43:25.881781+0000","last_became_peered":"2023-12-19T16:43:25.881781+0000","last_unstale":"2023-12-19T16:43:27.857252+0000","last_undegraded":"2023-12-19T16:43:27.857252+0000","last_fullsized":"2023-12-19T16:43:27.857252+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:37:54.151970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.902821+0000","last_change":"2023-12-19T16:43:25.488465+0000","last_active":"2023-12-19T16:43:25.902821+0000","last_peered":"2023-12-19T16:43:25.902821+0000","last_clean":"2023-12-19T16:43:25.902821+0000","last_became_active":"2023-12-19T16:43:25.487509+0000","last_became_peered":"2023-12-19T16:43:25.487509+0000","last_unstale":"2023-12-19T16:43:25.902821+0000","last_undegraded":"2023-12-19T16:43:25.902821+0000","last_fullsized":"2023-12-19T16:43:25.902821+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:03.546398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857982+0000","last_change":"2023-12-19T16:43:25.899810+0000","last_active":"2023-12-19T16:43:27.857982+0000","last_peered":"2023-12-19T16:43:27.857982+0000","last_clean":"2023-12-19T16:43:27.857982+0000","last_became_active":"2023-12-19T16:43:25.899506+0000","last_became_peered":"2023-12-19T16:43:25.899506+0000","last_unstale":"2023-12-19T16:43:27.857982+0000","last_undegraded":"2023-12-19T16:43:27.857982+0000","last_fullsized":"2023-12-19T16:43:27.857982+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:42:10.952399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.872724+0000","last_change":"2023-12-19T16:43:25.479701+0000","last_active":"2023-12-19T16:43:25.872724+0000","last_peered":"2023-12-19T16:43:25.872724+0000","last_clean":"2023-12-19T16:43:25.872724+0000","last_became_active":"2023-12-19T16:43:25.479439+0000","last_became_peered":"2023-12-19T16:43:25.479439+0000","last_unstale":"2023-12-19T16:43:25.872724+0000","last_undegraded":"2023-12-19T16:43:25.872724+0000","last_fullsized":"2023-12-19T16:43:25.872724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:54.432676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.885432+0000","last_change":"2023-12-19T16:43:25.899855+0000","last_active":"2023-12-19T16:43:26.885432+0000","last_peered":"2023-12-19T16:43:26.885432+0000","last_clean":"2023-12-19T16:43:26.885432+0000","last_became_active":"2023-12-19T16:43:25.899545+0000","last_became_peered":"2023-12-19T16:43:25.899545+0000","last_unstale":"2023-12-19T16:43:26.885432+0000","last_undegraded":"2023-12-19T16:43:26.885432+0000","last_fullsized":"2023-12-19T16:43:26.885432+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:56.387441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.885159+0000","last_change":"2023-12-19T16:43:25.479546+0000","last_active":"2023-12-19T16:43:25.885159+0000","last_peered":"2023-12-19T16:43:25.885159+0000","last_clean":"2023-12-19T16:43:25.885159+0000","last_became_active":"2023-12-19T16:43:25.479208+0000","last_became_peered":"2023-12-19T16:43:25.479208+0000","last_unstale":"2023-12-19T16:43:25.885159+0000","last_undegraded":"2023-12-19T16:43:25.885159+0000","last_fullsized":"2023-12-19T16:43:25.885159+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:08.723483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856372+0000","last_change":"2023-12-19T16:43:25.899712+0000","last_active":"2023-12-19T16:43:27.856372+0000","last_peered":"2023-12-19T16:43:27.856372+0000","last_clean":"2023-12-19T16:43:27.856372+0000","last_became_active":"2023-12-19T16:43:25.896810+0000","last_became_peered":"2023-12-19T16:43:25.896810+0000","last_unstale":"2023-12-19T16:43:27.856372+0000","last_undegraded":"2023-12-19T16:43:27.856372+0000","last_fullsized":"2023-12-19T16:43:27.856372+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:22:23.348785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"36'8","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.911244+0000","last_change":"2023-12-19T16:43:25.475624+0000","last_active":"2023-12-19T16:43:25.911244+0000","last_peered":"2023-12-19T16:43:25.911244+0000","last_clean":"2023-12-19T16:43:25.911244+0000","last_became_active":"2023-12-19T16:43:25.475315+0000","last_became_peered":"2023-12-19T16:43:25.475315+0000","last_unstale":"2023-12-19T16:43:25.911244+0000","last_undegraded":"2023-12-19T16:43:25.911244+0000","last_fullsized":"2023-12-19T16:43:25.911244+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:39:45.423376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.886304+0000","last_change":"2023-12-19T16:43:25.893879+0000","last_active":"2023-12-19T16:43:26.886304+0000","last_peered":"2023-12-19T16:43:26.886304+0000","last_clean":"2023-12-19T16:43:26.886304+0000","last_became_active":"2023-12-19T16:43:25.893422+0000","last_became_peered":"2023-12-19T16:43:25.893422+0000","last_unstale":"2023-12-19T16:43:26.886304+0000","last_undegraded":"2023-12-19T16:43:26.886304+0000","last_fullsized":"2023-12-19T16:43:26.886304+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:34:43.261283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.907826+0000","last_change":"2023-12-19T16:43:25.487683+0000","last_active":"2023-12-19T16:43:25.907826+0000","last_peered":"2023-12-19T16:43:25.907826+0000","last_clean":"2023-12-19T16:43:25.907826+0000","last_became_active":"2023-12-19T16:43:25.487405+0000","last_became_peered":"2023-12-19T16:43:25.487405+0000","last_unstale":"2023-12-19T16:43:25.907826+0000","last_undegraded":"2023-12-19T16:43:25.907826+0000","last_fullsized":"2023-12-19T16:43:25.907826+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:08.723483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856713+0000","last_change":"2023-12-19T16:43:25.889732+0000","last_active":"2023-12-19T16:43:27.856713+0000","last_peered":"2023-12-19T16:43:27.856713+0000","last_clean":"2023-12-19T16:43:27.856713+0000","last_became_active":"2023-12-19T16:43:25.889433+0000","last_became_peered":"2023-12-19T16:43:25.889433+0000","last_unstale":"2023-12-19T16:43:27.856713+0000","last_undegraded":"2023-12-19T16:43:27.856713+0000","last_fullsized":"2023-12-19T16:43:27.856713+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:03:10.533550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.902576+0000","last_change":"2023-12-19T16:43:25.487722+0000","last_active":"2023-12-19T16:43:25.902576+0000","last_peered":"2023-12-19T16:43:25.902576+0000","last_clean":"2023-12-19T16:43:25.902576+0000","last_became_active":"2023-12-19T16:43:25.487120+0000","last_became_peered":"2023-12-19T16:43:25.487120+0000","last_unstale":"2023-12-19T16:43:25.902576+0000","last_undegraded":"2023-12-19T16:43:25.902576+0000","last_fullsized":"2023-12-19T16:43:25.902576+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:50:19.100332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.858135+0000","last_change":"2023-12-19T16:43:25.894992+0000","last_active":"2023-12-19T16:43:27.858135+0000","last_peered":"2023-12-19T16:43:27.858135+0000","last_clean":"2023-12-19T16:43:27.858135+0000","last_became_active":"2023-12-19T16:43:25.894436+0000","last_became_peered":"2023-12-19T16:43:25.894436+0000","last_unstale":"2023-12-19T16:43:27.858135+0000","last_undegraded":"2023-12-19T16:43:27.858135+0000","last_fullsized":"2023-12-19T16:43:27.858135+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:56.387441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.904126+0000","last_change":"2023-12-19T16:43:25.485086+0000","last_active":"2023-12-19T16:43:25.904126+0000","last_peered":"2023-12-19T16:43:25.904126+0000","last_clean":"2023-12-19T16:43:25.904126+0000","last_became_active":"2023-12-19T16:43:25.484800+0000","last_became_peered":"2023-12-19T16:43:25.484800+0000","last_unstale":"2023-12-19T16:43:25.904126+0000","last_undegraded":"2023-12-19T16:43:25.904126+0000","last_fullsized":"2023-12-19T16:43:25.904126+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:43:43.874824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.880761+0000","last_change":"2023-12-19T16:43:25.893940+0000","last_active":"2023-12-19T16:43:26.880761+0000","last_peered":"2023-12-19T16:43:26.880761+0000","last_clean":"2023-12-19T16:43:26.880761+0000","last_became_active":"2023-12-19T16:43:25.893690+0000","last_became_peered":"2023-12-19T16:43:25.893690+0000","last_unstale":"2023-12-19T16:43:26.880761+0000","last_undegraded":"2023-12-19T16:43:26.880761+0000","last_fullsized":"2023-12-19T16:43:26.880761+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:40.259600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.874922+0000","last_change":"2023-12-19T16:43:25.487955+0000","last_active":"2023-12-19T16:43:25.874922+0000","last_peered":"2023-12-19T16:43:25.874922+0000","last_clean":"2023-12-19T16:43:25.874922+0000","last_became_active":"2023-12-19T16:43:25.487666+0000","last_became_peered":"2023-12-19T16:43:25.487666+0000","last_unstale":"2023-12-19T16:43:25.874922+0000","last_undegraded":"2023-12-19T16:43:25.874922+0000","last_fullsized":"2023-12-19T16:43:25.874922+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:30:19.111754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.880573+0000","last_change":"2023-12-19T16:43:25.894810+0000","last_active":"2023-12-19T16:43:26.880573+0000","last_peered":"2023-12-19T16:43:26.880573+0000","last_clean":"2023-12-19T16:43:26.880573+0000","last_became_active":"2023-12-19T16:43:25.894561+0000","last_became_peered":"2023-12-19T16:43:25.894561+0000","last_unstale":"2023-12-19T16:43:26.880573+0000","last_undegraded":"2023-12-19T16:43:26.880573+0000","last_fullsized":"2023-12-19T16:43:26.880573+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:46.715484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856897+0000","last_change":"2023-12-19T16:43:25.486440+0000","last_active":"2023-12-19T16:43:27.856897+0000","last_peered":"2023-12-19T16:43:27.856897+0000","last_clean":"2023-12-19T16:43:27.856897+0000","last_became_active":"2023-12-19T16:43:25.486112+0000","last_became_peered":"2023-12-19T16:43:25.486112+0000","last_unstale":"2023-12-19T16:43:27.856897+0000","last_undegraded":"2023-12-19T16:43:27.856897+0000","last_fullsized":"2023-12-19T16:43:27.856897+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:47:36.130533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.880683+0000","last_change":"2023-12-19T16:43:25.892883+0000","last_active":"2023-12-19T16:43:26.880683+0000","last_peered":"2023-12-19T16:43:26.880683+0000","last_clean":"2023-12-19T16:43:26.880683+0000","last_became_active":"2023-12-19T16:43:25.890975+0000","last_became_peered":"2023-12-19T16:43:25.890975+0000","last_unstale":"2023-12-19T16:43:26.880683+0000","last_undegraded":"2023-12-19T16:43:26.880683+0000","last_fullsized":"2023-12-19T16:43:26.880683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:24:18.658285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.882709+0000","last_change":"2023-12-19T16:43:25.486081+0000","last_active":"2023-12-19T16:43:25.882709+0000","last_peered":"2023-12-19T16:43:25.882709+0000","last_clean":"2023-12-19T16:43:25.882709+0000","last_became_active":"2023-12-19T16:43:25.484540+0000","last_became_peered":"2023-12-19T16:43:25.484540+0000","last_unstale":"2023-12-19T16:43:25.882709+0000","last_undegraded":"2023-12-19T16:43:25.882709+0000","last_fullsized":"2023-12-19T16:43:25.882709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:05.966306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.880485+0000","last_change":"2023-12-19T16:43:25.885735+0000","last_active":"2023-12-19T16:43:26.880485+0000","last_peered":"2023-12-19T16:43:26.880485+0000","last_clean":"2023-12-19T16:43:26.880485+0000","last_became_active":"2023-12-19T16:43:25.885482+0000","last_became_peered":"2023-12-19T16:43:25.885482+0000","last_unstale":"2023-12-19T16:43:26.880485+0000","last_undegraded":"2023-12-19T16:43:26.880485+0000","last_fullsized":"2023-12-19T16:43:26.880485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:06.973178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.874832+0000","last_change":"2023-12-19T16:43:25.485439+0000","last_active":"2023-12-19T16:43:25.874832+0000","last_peered":"2023-12-19T16:43:25.874832+0000","last_clean":"2023-12-19T16:43:25.874832+0000","last_became_active":"2023-12-19T16:43:25.485059+0000","last_became_peered":"2023-12-19T16:43:25.485059+0000","last_unstale":"2023-12-19T16:43:25.874832+0000","last_undegraded":"2023-12-19T16:43:25.874832+0000","last_fullsized":"2023-12-19T16:43:25.874832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:30:57.984641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.885574+0000","last_change":"2023-12-19T16:43:25.903699+0000","last_active":"2023-12-19T16:43:26.885574+0000","last_peered":"2023-12-19T16:43:26.885574+0000","last_clean":"2023-12-19T16:43:26.885574+0000","last_became_active":"2023-12-19T16:43:25.903451+0000","last_became_peered":"2023-12-19T16:43:25.903451+0000","last_unstale":"2023-12-19T16:43:26.885574+0000","last_undegraded":"2023-12-19T16:43:26.885574+0000","last_fullsized":"2023-12-19T16:43:26.885574+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:22:09.023704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.878134+0000","last_change":"2023-12-19T16:43:25.481421+0000","last_active":"2023-12-19T16:43:25.878134+0000","last_peered":"2023-12-19T16:43:25.878134+0000","last_clean":"2023-12-19T16:43:25.878134+0000","last_became_active":"2023-12-19T16:43:25.478132+0000","last_became_peered":"2023-12-19T16:43:25.478132+0000","last_unstale":"2023-12-19T16:43:25.878134+0000","last_undegraded":"2023-12-19T16:43:25.878134+0000","last_fullsized":"2023-12-19T16:43:25.878134+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:22:22.341913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.880714+0000","last_change":"2023-12-19T16:43:25.884899+0000","last_active":"2023-12-19T16:43:26.880714+0000","last_peered":"2023-12-19T16:43:26.880714+0000","last_clean":"2023-12-19T16:43:26.880714+0000","last_became_active":"2023-12-19T16:43:25.884657+0000","last_became_peered":"2023-12-19T16:43:25.884657+0000","last_unstale":"2023-12-19T16:43:26.880714+0000","last_undegraded":"2023-12-19T16:43:26.880714+0000","last_fullsized":"2023-12-19T16:43:26.880714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:07:56.454266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.904842+0000","last_change":"2023-12-19T16:43:25.484492+0000","last_active":"2023-12-19T16:43:25.904842+0000","last_peered":"2023-12-19T16:43:25.904842+0000","last_clean":"2023-12-19T16:43:25.904842+0000","last_became_active":"2023-12-19T16:43:25.484176+0000","last_became_peered":"2023-12-19T16:43:25.484176+0000","last_unstale":"2023-12-19T16:43:25.904842+0000","last_undegraded":"2023-12-19T16:43:25.904842+0000","last_fullsized":"2023-12-19T16:43:25.904842+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:48:55.560180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856548+0000","last_change":"2023-12-19T16:43:25.899472+0000","last_active":"2023-12-19T16:43:27.856548+0000","last_peered":"2023-12-19T16:43:27.856548+0000","last_clean":"2023-12-19T16:43:27.856548+0000","last_became_active":"2023-12-19T16:43:25.898733+0000","last_became_peered":"2023-12-19T16:43:25.898733+0000","last_unstale":"2023-12-19T16:43:27.856548+0000","last_undegraded":"2023-12-19T16:43:27.856548+0000","last_fullsized":"2023-12-19T16:43:27.856548+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:22:09.023704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857319+0000","last_change":"2023-12-19T16:43:25.486390+0000","last_active":"2023-12-19T16:43:27.857319+0000","last_peered":"2023-12-19T16:43:27.857319+0000","last_clean":"2023-12-19T16:43:27.857319+0000","last_became_active":"2023-12-19T16:43:25.486082+0000","last_became_peered":"2023-12-19T16:43:25.486082+0000","last_unstale":"2023-12-19T16:43:27.857319+0000","last_undegraded":"2023-12-19T16:43:27.857319+0000","last_fullsized":"2023-12-19T16:43:27.857319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:03:37.009029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856659+0000","last_change":"2023-12-19T16:43:25.908732+0000","last_active":"2023-12-19T16:43:27.856659+0000","last_peered":"2023-12-19T16:43:27.856659+0000","last_clean":"2023-12-19T16:43:27.856659+0000","last_became_active":"2023-12-19T16:43:25.901839+0000","last_became_peered":"2023-12-19T16:43:25.901839+0000","last_unstale":"2023-12-19T16:43:27.856659+0000","last_undegraded":"2023-12-19T16:43:27.856659+0000","last_fullsized":"2023-12-19T16:43:27.856659+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:20.393474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.875212+0000","last_change":"2023-12-19T16:43:25.481088+0000","last_active":"2023-12-19T16:43:25.875212+0000","last_peered":"2023-12-19T16:43:25.875212+0000","last_clean":"2023-12-19T16:43:25.875212+0000","last_became_active":"2023-12-19T16:43:25.480631+0000","last_became_peered":"2023-12-19T16:43:25.480631+0000","last_unstale":"2023-12-19T16:43:25.875212+0000","last_undegraded":"2023-12-19T16:43:25.875212+0000","last_fullsized":"2023-12-19T16:43:25.875212+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:01.953049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856539+0000","last_change":"2023-12-19T16:43:25.904679+0000","last_active":"2023-12-19T16:43:27.856539+0000","last_peered":"2023-12-19T16:43:27.856539+0000","last_clean":"2023-12-19T16:43:27.856539+0000","last_became_active":"2023-12-19T16:43:25.904336+0000","last_became_peered":"2023-12-19T16:43:25.904336+0000","last_unstale":"2023-12-19T16:43:27.856539+0000","last_undegraded":"2023-12-19T16:43:27.856539+0000","last_fullsized":"2023-12-19T16:43:27.856539+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:34:43.261283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856999+0000","last_change":"2023-12-19T16:43:25.485176+0000","last_active":"2023-12-19T16:43:27.856999+0000","last_peered":"2023-12-19T16:43:27.856999+0000","last_clean":"2023-12-19T16:43:27.856999+0000","last_became_active":"2023-12-19T16:43:25.484842+0000","last_became_peered":"2023-12-19T16:43:25.484842+0000","last_unstale":"2023-12-19T16:43:27.856999+0000","last_undegraded":"2023-12-19T16:43:27.856999+0000","last_fullsized":"2023-12-19T16:43:27.856999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:49.582698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.857140+0000","last_change":"2023-12-19T16:43:25.905903+0000","last_active":"2023-12-19T16:43:27.857140+0000","last_peered":"2023-12-19T16:43:27.857140+0000","last_clean":"2023-12-19T16:43:27.857140+0000","last_became_active":"2023-12-19T16:43:25.897356+0000","last_became_peered":"2023-12-19T16:43:25.897356+0000","last_unstale":"2023-12-19T16:43:27.857140+0000","last_undegraded":"2023-12-19T16:43:27.857140+0000","last_fullsized":"2023-12-19T16:43:27.857140+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:36:39.824632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.872811+0000","last_change":"2023-12-19T16:43:25.479140+0000","last_active":"2023-12-19T16:43:25.872811+0000","last_peered":"2023-12-19T16:43:25.872811+0000","last_clean":"2023-12-19T16:43:25.872811+0000","last_became_active":"2023-12-19T16:43:25.478696+0000","last_became_peered":"2023-12-19T16:43:25.478696+0000","last_unstale":"2023-12-19T16:43:25.872811+0000","last_undegraded":"2023-12-19T16:43:25.872811+0000","last_fullsized":"2023-12-19T16:43:25.872811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:45.708612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:27.856758+0000","last_change":"2023-12-19T16:43:25.898714+0000","last_active":"2023-12-19T16:43:27.856758+0000","last_peered":"2023-12-19T16:43:27.856758+0000","last_clean":"2023-12-19T16:43:27.856758+0000","last_became_active":"2023-12-19T16:43:25.898444+0000","last_became_peered":"2023-12-19T16:43:25.898444+0000","last_unstale":"2023-12-19T16:43:27.856758+0000","last_undegraded":"2023-12-19T16:43:27.856758+0000","last_fullsized":"2023-12-19T16:43:27.856758+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:16:43.256480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.874929+0000","last_change":"2023-12-19T16:43:25.484784+0000","last_active":"2023-12-19T16:43:25.874929+0000","last_peered":"2023-12-19T16:43:25.874929+0000","last_clean":"2023-12-19T16:43:25.874929+0000","last_became_active":"2023-12-19T16:43:25.484501+0000","last_became_peered":"2023-12-19T16:43:25.484501+0000","last_unstale":"2023-12-19T16:43:25.874929+0000","last_undegraded":"2023-12-19T16:43:25.874929+0000","last_fullsized":"2023-12-19T16:43:25.874929+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:50:38.846004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.885875+0000","last_change":"2023-12-19T16:43:25.882194+0000","last_active":"2023-12-19T16:43:26.885875+0000","last_peered":"2023-12-19T16:43:26.885875+0000","last_clean":"2023-12-19T16:43:26.885875+0000","last_became_active":"2023-12-19T16:43:25.879862+0000","last_became_peered":"2023-12-19T16:43:25.879862+0000","last_unstale":"2023-12-19T16:43:26.885875+0000","last_undegraded":"2023-12-19T16:43:26.885875+0000","last_fullsized":"2023-12-19T16:43:26.885875+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:49:46.056086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.886114+0000","last_change":"2023-12-19T16:43:25.479846+0000","last_active":"2023-12-19T16:43:25.886114+0000","last_peered":"2023-12-19T16:43:25.886114+0000","last_clean":"2023-12-19T16:43:25.886114+0000","last_became_active":"2023-12-19T16:43:25.479548+0000","last_became_peered":"2023-12-19T16:43:25.479548+0000","last_unstale":"2023-12-19T16:43:25.886114+0000","last_undegraded":"2023-12-19T16:43:25.886114+0000","last_fullsized":"2023-12-19T16:43:25.886114+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:01.953049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-12-19T16:43:26.885578+0000","last_change":"2023-12-19T16:43:25.905799+0000","last_active":"2023-12-19T16:43:26.885578+0000","last_peered":"2023-12-19T16:43:26.885578+0000","last_clean":"2023-12-19T16:43:26.885578+0000","last_became_active":"2023-12-19T16:43:25.905068+0000","last_became_peered":"2023-12-19T16:43:25.905068+0000","last_unstale":"2023-12-19T16:43:26.885578+0000","last_undegraded":"2023-12-19T16:43:26.885578+0000","last_fullsized":"2023-12-19T16:43:26.885578+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:25.454250+0000","last_clean_scrub_stamp":"2023-12-19T16:43:25.454250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:24:32.589731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.885224+0000","last_change":"2023-12-19T16:43:25.487462+0000","last_active":"2023-12-19T16:43:25.885224+0000","last_peered":"2023-12-19T16:43:25.885224+0000","last_clean":"2023-12-19T16:43:25.885224+0000","last_became_active":"2023-12-19T16:43:25.487174+0000","last_became_peered":"2023-12-19T16:43:25.487174+0000","last_unstale":"2023-12-19T16:43:25.885224+0000","last_undegraded":"2023-12-19T16:43:25.885224+0000","last_fullsized":"2023-12-19T16:43:25.885224+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:43:24.447378+0000","last_clean_scrub_stamp":"2023-12-19T16:43:24.447378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:24:17.651413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":176,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-12-19T16:43:25.887905+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:43:25.887905+0000","last_peered":"2023-12-19T16:43:25.887905+0000","last_clean":"2023-12-19T16:43:25.887905+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:43:25.887905+0000","last_undegraded":"2023-12-19T16:43:25.887905+0000","last_fullsized":"2023-12-19T16:43:25.887905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":1011712,"data_stored":752374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.0049999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":1.093}]},{"osd":2,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":1.157}]},{"osd":3,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":1.117}]},{"osd":4,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.72599999999999998}]},{"osd":6,"last update":"Tue Dec 19 16:43:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79500000000000004}]}]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":412,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":421888,"data_stored":162532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.621}]},{"osd":1,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":4.3799999999999999,"5min":4.3799999999999999,"15min":4.3799999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.56200000000000006}]},{"osd":2,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.70999999999999996}]},{"osd":3,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":4.673,"5min":4.673,"15min":4.673},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":0.502}]},{"osd":4,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.58599999999999997}]},{"osd":5,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.69499999999999995}]},{"osd":7,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.68200000000000005}]}]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":412,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":421888,"data_stored":162588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57699999999999996}]},{"osd":1,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.66000000000000003}]},{"osd":2,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.62},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.72999999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.72199999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80600000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:43:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.75700000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":412,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":421888,"data_stored":162020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.65400000000000003}]},{"osd":1,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.57299999999999995}]},{"osd":2,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.66800000000000004}]},{"osd":3,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.71999999999999997}]},{"osd":5,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.50600000000000001}]},{"osd":6,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.53300000000000003}]},{"osd":7,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.58799999999999997}]}]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":430080,"data_stored":163502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.504}]},{"osd":1,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":2.4910000000000001,"5min":2.4910000000000001,"15min":2.4910000000000001},"last":0.40699999999999997}]},{"osd":2,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":5.1219999999999999,"5min":5.1219999999999999,"15min":5.1219999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.0179999999999998,"5min":2.0179999999999998,"15min":2.0179999999999998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2690000000000001,"5min":2.2690000000000001,"15min":2.2690000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.73199999999999998}]},{"osd":6,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.956,"5min":1.956,"15min":1.956},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":0.69099999999999995}]},{"osd":7,"last update":"Tue Dec 19 16:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.749},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":5.0759999999999996,"5min":5.0759999999999996,"15min":5.0759999999999996},"last":0.72699999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":417792,"data_stored":161986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.5150000000000001,"5min":2.5150000000000001,"15min":2.5150000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.30499999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.7590000000000003,"5min":5.7590000000000003,"15min":5.7590000000000003},"last":0.433},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":5.8070000000000004,"5min":5.8070000000000004,"15min":5.8070000000000004},"last":0.38800000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.7599999999999998,"5min":2.7599999999999998,"15min":2.7599999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":5.7190000000000003,"5min":5.7190000000000003,"15min":5.7190000000000003},"last":0.47799999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.8940000000000001,"5min":3.8940000000000001,"15min":3.8940000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.9260000000000002,"5min":2.9260000000000002,"15min":2.9260000000000002},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":0.64200000000000002}]},{"osd":7,"last update":"Tue Dec 19 16:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.0529999999999999,"5min":2.0529999999999999,"15min":2.0529999999999999},"last":0.65000000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509458,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":430080,"data_stored":163502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.502}]},{"osd":2,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.434}]},{"osd":3,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.133,"5min":4.133,"15min":4.133},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":0.95099999999999996}]},{"osd":5,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.1269999999999998,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":4.0460000000000003,"5min":4.0460000000000003,"15min":4.0460000000000003},"last":0.98099999999999998}]},{"osd":6,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.0880000000000001,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":1.014}]},{"osd":7,"last update":"Tue Dec 19 16:43:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.093,"5min":4.093,"15min":4.093},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.996}]}]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":1007616,"data_stored":752284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.255},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"last":0.35099999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.59899999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.47399999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.649,"5min":1.649,"15min":1.649},"last":0.64300000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.627}]},{"osd":7,"last update":"Tue Dec 19 16:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.61299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:43:34.949 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T16:43:34.949 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T16:43:35.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f247bc8c640 1 Processor -- start 2023-12-19T16:43:35.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f247bc8c640 1 -- start start 2023-12-19T16:43:35.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f247bc8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f24740adac0 0x7f24740c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:35.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f247bc8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f247415a480 0x7f247406fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:35.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f247bc8c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f24740701f0 0x7f247415cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:35.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f247bc8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2474070660 con 0x7f24740adac0 2023-12-19T16:43:35.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f247bc8c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f247406b3d0 con 0x7f24740701f0 2023-12-19T16:43:35.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f247bc8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24740ca5e0 con 0x7f247415a480 2023-12-19T16:43:35.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f2479a28640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f24740adac0 0x7f24740c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:35.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f2479a28640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f24740adac0 0x7f24740c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55360/0 (socket says 172.21.15.26:55360) 2023-12-19T16:43:35.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f2479a28640 1 -- 172.21.15.26:0/603112621 learned_addr learned my addr 172.21.15.26:0/603112621 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:35.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f2479227640 1 --2- 172.21.15.26:0/603112621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f247415a480 0x7f247406fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:35.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f247a229640 1 --2- 172.21.15.26:0/603112621 >> v2:172.21.15.160:3300/0 conn(0x7f24740701f0 0x7f247415cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:35.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f2479a28640 1 -- 172.21.15.26:0/603112621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f247415a480 msgr2=0x7f247406fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:35.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f2479a28640 1 --2- 172.21.15.26:0/603112621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f247415a480 0x7f247406fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f2479a28640 1 -- 172.21.15.26:0/603112621 >> v2:172.21.15.160:3300/0 conn(0x7f24740701f0 msgr2=0x7f247415cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:35.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f2479a28640 1 --2- 172.21.15.26:0/603112621 >> v2:172.21.15.160:3300/0 conn(0x7f24740701f0 0x7f247415cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.131+0000 7f2479a28640 1 -- 172.21.15.26:0/603112621 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24740ca8c0 con 0x7f24740adac0 2023-12-19T16:43:35.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.135+0000 7f2479a28640 1 --2- 172.21.15.26:0/603112621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24740adac0 0x7f24740c9d90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f24740cb180 tx=0x7f247001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d209add9aadc44c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:35.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.135+0000 7f2478a26640 1 -- 172.21.15.26:0/603112621 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24700ab610 con 0x7f24740adac0 2023-12-19T16:43:35.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.135+0000 7f2478a26640 1 -- 172.21.15.26:0/603112621 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24700ab770 con 0x7f24740adac0 2023-12-19T16:43:35.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.135+0000 7f2478a26640 1 -- 172.21.15.26:0/603112621 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24700abb00 con 0x7f24740adac0 2023-12-19T16:43:35.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.135+0000 7f247bc8c640 1 -- 172.21.15.26:0/603112621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24740adac0 msgr2=0x7f24740c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:35.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.135+0000 7f247bc8c640 1 --2- 172.21.15.26:0/603112621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24740adac0 0x7f24740c9d90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f24740cb180 tx=0x7f247001c4d0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.135+0000 7f247bc8c640 1 -- 172.21.15.26:0/603112621 shutdown_connections 2023-12-19T16:43:35.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.135+0000 7f247bc8c640 1 --2- 172.21.15.26:0/603112621 >> v2:172.21.15.160:3300/0 conn(0x7f24740701f0 0x7f247415cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.135+0000 7f247bc8c640 1 --2- 172.21.15.26:0/603112621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f247415a480 0x7f247406fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.135+0000 7f247bc8c640 1 --2- 172.21.15.26:0/603112621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24740adac0 0x7f24740c9d90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.135+0000 7f247bc8c640 1 -- 172.21.15.26:0/603112621 >> 172.21.15.26:0/603112621 conn(0x7f24740660d0 msgr2=0x7f24740ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:35.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.135+0000 7f247bc8c640 1 -- 172.21.15.26:0/603112621 shutdown_connections 2023-12-19T16:43:35.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.135+0000 7f247bc8c640 1 -- 172.21.15.26:0/603112621 wait complete. 2023-12-19T16:43:35.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247bc8c640 1 Processor -- start 2023-12-19T16:43:35.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247bc8c640 1 -- start start 2023-12-19T16:43:35.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247bc8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24740701f0 0x7f247414fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:35.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247bc8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24740adac0 0x7f24740c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:35.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247bc8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f247415a480 0x7f247415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:35.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247bc8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24741583c0 con 0x7f247415a480 2023-12-19T16:43:35.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247bc8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24740c8560 con 0x7f24740adac0 2023-12-19T16:43:35.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247bc8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24741574d0 con 0x7f24740701f0 2023-12-19T16:43:35.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247a229640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f247415a480 0x7f247415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:35.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247a229640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f247415a480 0x7f247415b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55362/0 (socket says 172.21.15.26:55362) 2023-12-19T16:43:35.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247a229640 1 -- 172.21.15.26:0/94016836 learned_addr learned my addr 172.21.15.26:0/94016836 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:35.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f2479227640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24740adac0 0x7f24740c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:35.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f2479a28640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24740701f0 0x7f247414fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:35.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247a229640 1 -- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24740701f0 msgr2=0x7f247414fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:35.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247a229640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24740701f0 0x7f247414fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247a229640 1 -- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24740adac0 msgr2=0x7f24740c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:35.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247a229640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24740adac0 0x7f24740c64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247a229640 1 -- 172.21.15.26:0/94016836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24700094d0 con 0x7f247415a480 2023-12-19T16:43:35.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247a229640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f247415a480 0x7f247415b730 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f246c007e60 tx=0x7f246c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:35.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f246a7fc640 1 -- 172.21.15.26:0/94016836 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c014620 con 0x7f247415a480 2023-12-19T16:43:35.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f246a7fc640 1 -- 172.21.15.26:0/94016836 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246c00ee90 con 0x7f247415a480 2023-12-19T16:43:35.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f246a7fc640 1 -- 172.21.15.26:0/94016836 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c0149e0 con 0x7f247415a480 2023-12-19T16:43:35.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247bc8c640 1 -- 172.21.15.26:0/94016836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24741577b0 con 0x7f247415a480 2023-12-19T16:43:35.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.139+0000 7f247bc8c640 1 -- 172.21.15.26:0/94016836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2474157d70 con 0x7f247415a480 2023-12-19T16:43:35.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.143+0000 7f247bc8c640 1 -- 172.21.15.26:0/94016836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2474158530 con 0x7f247415a480 2023-12-19T16:43:35.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.143+0000 7f246a7fc640 1 -- 172.21.15.26:0/94016836 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f246c014b40 con 0x7f247415a480 2023-12-19T16:43:35.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.143+0000 7f246a7fc640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2458065a60 0x7f2458067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:35.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.143+0000 7f246a7fc640 1 -- 172.21.15.26:0/94016836 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f246c083a10 con 0x7f247415a480 2023-12-19T16:43:35.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.143+0000 7f2479a28640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2458065a60 0x7f2458067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:35.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.143+0000 7f2479a28640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2458065a60 0x7f2458067f20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f24700af000 tx=0x7f24700b6040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:35.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.147+0000 7f246a7fc640 1 -- 172.21.15.26:0/94016836 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f246c0513e0 con 0x7f247415a480 2023-12-19T16:43:35.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.359+0000 7f247bc8c640 1 -- 172.21.15.26:0/94016836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2474074270 con 0x7f247415a480 2023-12-19T16:43:35.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.363+0000 7f246a7fc640 1 -- 172.21.15.26:0/94016836 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f246c054a00 con 0x7f247415a480 2023-12-19T16:43:35.370 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2023-12-19T16:43:35.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.367+0000 7f247bc8c640 1 -- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2458065a60 msgr2=0x7f2458067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:35.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.367+0000 7f247bc8c640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2458065a60 0x7f2458067f20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f24700af000 tx=0x7f24700b6040 comp rx=0 tx=0).stop 2023-12-19T16:43:35.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.367+0000 7f247bc8c640 1 -- 172.21.15.26:0/94016836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f247415a480 msgr2=0x7f247415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:35.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.367+0000 7f247bc8c640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f247415a480 0x7f247415b730 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f246c007e60 tx=0x7f246c00e4b0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.367+0000 7f247bc8c640 1 -- 172.21.15.26:0/94016836 shutdown_connections 2023-12-19T16:43:35.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.367+0000 7f247bc8c640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f247415a480 0x7f247415b730 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.367+0000 7f247bc8c640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24740adac0 0x7f24740c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.367+0000 7f247bc8c640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2458065a60 0x7f2458067f20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.367+0000 7f247bc8c640 1 --2- 172.21.15.26:0/94016836 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24740701f0 0x7f247414fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.367+0000 7f247bc8c640 1 -- 172.21.15.26:0/94016836 >> 172.21.15.26:0/94016836 conn(0x7f24740660d0 msgr2=0x7f247415c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:35.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.367+0000 7f247bc8c640 1 -- 172.21.15.26:0/94016836 shutdown_connections 2023-12-19T16:43:35.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.367+0000 7f247bc8c640 1 -- 172.21.15.26:0/94016836 wait complete. 2023-12-19T16:43:35.391 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:43:35.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.527+0000 7f7f78a66640 1 Processor -- start 2023-12-19T16:43:35.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.527+0000 7f7f78a66640 1 -- start start 2023-12-19T16:43:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.531+0000 7f7f78a66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7f741577d0 0x7f7f7415ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.531+0000 7f7f78a66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7415e110 con 0x7f7f740cea80 2023-12-19T16:43:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.531+0000 7f7f78a66640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7415e3f0 con 0x7f7f741573f0 2023-12-19T16:43:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.531+0000 7f7f78a66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7415a300 con 0x7f7f741577d0 2023-12-19T16:43:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.531+0000 7f7f72d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7f740cea80 0x7f7f7406f6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47392/0 (socket says 172.21.15.26:47392) 2023-12-19T16:43:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.531+0000 7f7f72d9d640 1 -- 172.21.15.26:0/3660744782 learned_addr learned my addr 172.21.15.26:0/3660744782 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.531+0000 7f7f6bfff640 1 --2- 172.21.15.26:0/3660744782 >> v2:172.21.15.160:3300/0 conn(0x7f7f741577d0 0x7f7f7415ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.531+0000 7f7f71d9b640 1 -- 172.21.15.26:0/3660744782 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333236598 0 0) 0x7f7f7415e110 con 0x7f7f740cea80 2023-12-19T16:43:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.531+0000 7f7f71d9b640 1 -- 172.21.15.26:0/3660744782 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f60003530 con 0x7f7f740cea80 2023-12-19T16:43:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.531+0000 7f7f6bfff640 1 -- 172.21.15.26:0/3660744782 >> v1:172.21.15.160:6789/0 conn(0x7f7f741573f0 legacy=0x7f7f7415d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.531+0000 7f7f71d9b640 1 -- 172.21.15.26:0/3660744782 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745895733 0 0) 0x7f7f7415e3f0 con 0x7f7f741573f0 2023-12-19T16:43:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.531+0000 7f7f6bfff640 1 -- 172.21.15.26:0/3660744782 >> v1:172.21.15.26:6789/0 conn(0x7f7f740cea80 legacy=0x7f7f7406f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.531+0000 7f7f6bfff640 1 -- 172.21.15.26:0/3660744782 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7415a5e0 con 0x7f7f741577d0 2023-12-19T16:43:35.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.535+0000 7f7f6bfff640 1 --2- 172.21.15.26:0/3660744782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f741577d0 0x7f7f7415ab00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0097c0 tx=0x7f7f5c01c280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2d510b8f928e93a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:35.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.535+0000 7f7f71d9b640 1 -- 172.21.15.26:0/3660744782 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c001a00 con 0x7f7f741577d0 2023-12-19T16:43:35.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.535+0000 7f7f71d9b640 1 -- 172.21.15.26:0/3660744782 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c0a7050 con 0x7f7f741577d0 2023-12-19T16:43:35.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.535+0000 7f7f71d9b640 1 -- 172.21.15.26:0/3660744782 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c0ab620 con 0x7f7f741577d0 2023-12-19T16:43:35.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.535+0000 7f7f78a66640 1 -- 172.21.15.26:0/3660744782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f741577d0 msgr2=0x7f7f7415ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:35.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.535+0000 7f7f78a66640 1 --2- 172.21.15.26:0/3660744782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f741577d0 0x7f7f7415ab00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0097c0 tx=0x7f7f5c01c280 comp rx=0 tx=0).stop 2023-12-19T16:43:35.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.535+0000 7f7f78a66640 1 -- 172.21.15.26:0/3660744782 shutdown_connections 2023-12-19T16:43:35.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.535+0000 7f7f78a66640 1 --2- 172.21.15.26:0/3660744782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f741577d0 0x7f7f7415ab00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.535+0000 7f7f78a66640 1 -- 172.21.15.26:0/3660744782 >> 172.21.15.26:0/3660744782 conn(0x7f7f740b1aa0 msgr2=0x7f7f740b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:35.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.535+0000 7f7f78a66640 1 -- 172.21.15.26:0/3660744782 shutdown_connections 2023-12-19T16:43:35.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.535+0000 7f7f78a66640 1 -- 172.21.15.26:0/3660744782 wait complete. 2023-12-19T16:43:35.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.535+0000 7f7f78a66640 1 Processor -- start 2023-12-19T16:43:35.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.535+0000 7f7f78a66640 1 -- start start 2023-12-19T16:43:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f78a66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f740cea80 0x7f7f74133b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f78a66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f741573f0 0x7f7f74143930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f78a66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f741577d0 0x7f7f7414fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f78a66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f741511a0 con 0x7f7f741577d0 2023-12-19T16:43:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f78a66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f74143e70 con 0x7f7f741573f0 2023-12-19T16:43:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f78a66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f74150620 con 0x7f7f740cea80 2023-12-19T16:43:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f72d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f741577d0 0x7f7f7414fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f72d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f741577d0 0x7f7f7414fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55368/0 (socket says 172.21.15.26:55368) 2023-12-19T16:43:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f72d9d640 1 -- 172.21.15.26:0/2502329833 learned_addr learned my addr 172.21.15.26:0/2502329833 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f6bfff640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f741573f0 0x7f7f74143930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f7259c640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f740cea80 0x7f7f74133b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f72d9d640 1 -- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f740cea80 msgr2=0x7f7f74133b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f72d9d640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f740cea80 0x7f7f74133b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f72d9d640 1 -- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f741573f0 msgr2=0x7f7f74143930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f72d9d640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f741573f0 0x7f7f74143930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f72d9d640 1 -- 172.21.15.26:0/2502329833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c009420 con 0x7f7f741577d0 2023-12-19T16:43:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f72d9d640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f741577d0 0x7f7f7414fd40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c002240 tx=0x7f7f6c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f6affd640 1 -- 172.21.15.26:0/2502329833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00aa20 con 0x7f7f741577d0 2023-12-19T16:43:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f6affd640 1 -- 172.21.15.26:0/2502329833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c00ab80 con 0x7f7f741577d0 2023-12-19T16:43:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f6affd640 1 -- 172.21.15.26:0/2502329833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c004e40 con 0x7f7f741577d0 2023-12-19T16:43:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f78a66640 1 -- 172.21.15.26:0/2502329833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f74150900 con 0x7f7f741577d0 2023-12-19T16:43:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f78a66640 1 -- 172.21.15.26:0/2502329833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f74150ec0 con 0x7f7f741577d0 2023-12-19T16:43:35.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f78a66640 1 -- 172.21.15.26:0/2502329833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f74074270 con 0x7f7f741577d0 2023-12-19T16:43:35.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.539+0000 7f7f6affd640 1 -- 172.21.15.26:0/2502329833 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7f6c0070d0 con 0x7f7f741577d0 2023-12-19T16:43:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.543+0000 7f7f6affd640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7f54065a60 0x7f7f54067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.543+0000 7f7f6affd640 1 -- 172.21.15.26:0/2502329833 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f6c082af0 con 0x7f7f741577d0 2023-12-19T16:43:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.543+0000 7f7f7259c640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7f54065a60 0x7f7f54067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.543+0000 7f7f7259c640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7f54065a60 0x7f7f54067f20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f64000f70 tx=0x7f7f64009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.547+0000 7f7f6affd640 1 -- 172.21.15.26:0/2502329833 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7f6c050380 con 0x7f7f741577d0 2023-12-19T16:43:35.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.735+0000 7f7f78a66640 1 -- 172.21.15.26:0/2502329833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f740708a0 con 0x7f7f741577d0 2023-12-19T16:43:35.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.735+0000 7f7f6affd640 1 -- 172.21.15.26:0/2502329833 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f7f6c018070 con 0x7f7f741577d0 2023-12-19T16:43:35.741 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:43:35.741 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1991431867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1991431867},{"type":"v1","addr":"172.21.15.160:6837","nonce":1991431867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3495925867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3495925867},{"type":"v1","addr":"172.21.15.160:6835","nonce":3495925867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3757604412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3757604412},{"type":"v1","addr":"172.21.15.26:6833","nonce":3757604412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:43:25.866059+0000","modified":"2023-12-19T16:43:28.880638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2660001678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2660001678},{"type":"v1","addr":"172.21.15.26:6835","nonce":2660001678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T16:43:35.741 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2023-12-19T16:43:35.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.735+0000 7f7f78a66640 1 -- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7f54065a60 msgr2=0x7f7f54067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:35.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.735+0000 7f7f78a66640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7f54065a60 0x7f7f54067f20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f64000f70 tx=0x7f7f64009040 comp rx=0 tx=0).stop 2023-12-19T16:43:35.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.735+0000 7f7f78a66640 1 -- 172.21.15.26:0/2502329833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f741577d0 msgr2=0x7f7f7414fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:35.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.735+0000 7f7f78a66640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f741577d0 0x7f7f7414fd40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c002240 tx=0x7f7f6c00cf50 comp rx=0 tx=0).stop 2023-12-19T16:43:35.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.735+0000 7f7f78a66640 1 -- 172.21.15.26:0/2502329833 shutdown_connections 2023-12-19T16:43:35.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.735+0000 7f7f78a66640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f741577d0 0x7f7f7414fd40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.735+0000 7f7f78a66640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f741573f0 0x7f7f74143930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.735+0000 7f7f78a66640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7f54065a60 0x7f7f54067f20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.735+0000 7f7f78a66640 1 --2- 172.21.15.26:0/2502329833 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f740cea80 0x7f7f74133b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:35.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.735+0000 7f7f78a66640 1 -- 172.21.15.26:0/2502329833 >> 172.21.15.26:0/2502329833 conn(0x7f7f740b1aa0 msgr2=0x7f7f740b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:35.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.739+0000 7f7f78a66640 1 -- 172.21.15.26:0/2502329833 shutdown_connections 2023-12-19T16:43:35.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:35.739+0000 7f7f78a66640 1 -- 172.21.15.26:0/2502329833 wait complete. 2023-12-19T16:43:36.165 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T16:43:25.866059+0000', 'modified': '2023-12-19T16:43:28.880638+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5106}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5106': {'gid': 5106, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/2660001678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 2660001678}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 2660001678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T16:43:36.165 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T16:43:36.165 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-19T16:43:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf33cb6640 1 Processor -- start 2023-12-19T16:43:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf33cb6640 1 -- start start 2023-12-19T16:43:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf33cb6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf2c0cb1d0 0x7fdf2c0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf33cb6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdf2c0b1d40 0x7fdf2c0b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf33cb6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c06d4b0 con 0x7fdf2c06bee0 2023-12-19T16:43:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf33cb6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c0cc030 con 0x7fdf2c0cb1d0 2023-12-19T16:43:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf33cb6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c0cbaf0 con 0x7fdf2c0b1d40 2023-12-19T16:43:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf31a52640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf2c0cb1d0 0x7fdf2c0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf31251640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdf2c06bee0 0x7fdf2c06c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35142/0 (socket says 172.21.15.26:35142) 2023-12-19T16:43:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf31251640 1 -- 172.21.15.26:0/213946636 learned_addr learned my addr 172.21.15.26:0/213946636 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf32253640 1 --2- 172.21.15.26:0/213946636 >> v2:172.21.15.160:3300/0 conn(0x7fdf2c0b1d40 0x7fdf2c0b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf31a52640 1 -- 172.21.15.26:0/213946636 >> v2:172.21.15.160:3300/0 conn(0x7fdf2c0b1d40 msgr2=0x7fdf2c0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf31a52640 1 --2- 172.21.15.26:0/213946636 >> v2:172.21.15.160:3300/0 conn(0x7fdf2c0b1d40 0x7fdf2c0b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf31a52640 1 -- 172.21.15.26:0/213946636 >> v1:172.21.15.26:6789/0 conn(0x7fdf2c06bee0 legacy=0x7fdf2c06c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf31a52640 1 -- 172.21.15.26:0/213946636 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c06cc10 con 0x7fdf2c0cb1d0 2023-12-19T16:43:36.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.311+0000 7fdf30a50640 1 -- 172.21.15.26:0/213946636 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942694002 0 0) 0x7fdf2c06d4b0 con 0x7fdf2c06bee0 2023-12-19T16:43:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.315+0000 7fdf31a52640 1 --2- 172.21.15.26:0/213946636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cb1d0 0x7fdf2c0cb5b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0cca50 tx=0x7fdf2801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a8a0a5daf41732 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.315+0000 7fdf30a50640 1 -- 172.21.15.26:0/213946636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf280ac630 con 0x7fdf2c0cb1d0 2023-12-19T16:43:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.315+0000 7fdf30a50640 1 -- 172.21.15.26:0/213946636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf280ac790 con 0x7fdf2c0cb1d0 2023-12-19T16:43:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.315+0000 7fdf30a50640 1 -- 172.21.15.26:0/213946636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf280acb20 con 0x7fdf2c0cb1d0 2023-12-19T16:43:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.315+0000 7fdf33cb6640 1 -- 172.21.15.26:0/213946636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cb1d0 msgr2=0x7fdf2c0cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.315+0000 7fdf33cb6640 1 --2- 172.21.15.26:0/213946636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cb1d0 0x7fdf2c0cb5b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0cca50 tx=0x7fdf2801c3d0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.315+0000 7fdf33cb6640 1 -- 172.21.15.26:0/213946636 shutdown_connections 2023-12-19T16:43:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.315+0000 7fdf33cb6640 1 --2- 172.21.15.26:0/213946636 >> v2:172.21.15.160:3300/0 conn(0x7fdf2c0b1d40 0x7fdf2c0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.315+0000 7fdf33cb6640 1 --2- 172.21.15.26:0/213946636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cb1d0 0x7fdf2c0cb5b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.315+0000 7fdf33cb6640 1 -- 172.21.15.26:0/213946636 >> 172.21.15.26:0/213946636 conn(0x7fdf2c0658e0 msgr2=0x7fdf2c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.315+0000 7fdf33cb6640 1 -- 172.21.15.26:0/213946636 shutdown_connections 2023-12-19T16:43:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.315+0000 7fdf33cb6640 1 -- 172.21.15.26:0/213946636 wait complete. 2023-12-19T16:43:36.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf33cb6640 1 Processor -- start 2023-12-19T16:43:36.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf33cb6640 1 -- start start 2023-12-19T16:43:36.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf33cb6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c06bee0 0x7fdf2c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf33cb6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf2c0b1d40 0x7fdf2c0afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf33cb6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf2c0cb1d0 0x7fdf2c1619e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf33cb6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c149a40 con 0x7fdf2c06bee0 2023-12-19T16:43:36.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf33cb6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0cd930 con 0x7fdf2c0cb1d0 2023-12-19T16:43:36.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf33cb6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0b0150 con 0x7fdf2c0b1d40 2023-12-19T16:43:36.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf31a52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c06bee0 0x7fdf2c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf31a52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c06bee0 0x7fdf2c0ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54800/0 (socket says 172.21.15.26:54800) 2023-12-19T16:43:36.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf31a52640 1 -- 172.21.15.26:0/2205198555 learned_addr learned my addr 172.21.15.26:0/2205198555 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:36.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf32253640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf2c0cb1d0 0x7fdf2c1619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf31251640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf2c0b1d40 0x7fdf2c0afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf31a52640 1 -- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf2c0b1d40 msgr2=0x7fdf2c0afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf31a52640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf2c0b1d40 0x7fdf2c0afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf31a52640 1 -- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf2c0cb1d0 msgr2=0x7fdf2c1619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf31a52640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf2c0cb1d0 0x7fdf2c1619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf31a52640 1 -- 172.21.15.26:0/2205198555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28009420 con 0x7fdf2c06bee0 2023-12-19T16:43:36.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf31a52640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c06bee0 0x7fdf2c0ce6b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdf28000c10 tx=0x7fdf2801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:36.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf227fc640 1 -- 172.21.15.26:0/2205198555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28004230 con 0x7fdf2c06bee0 2023-12-19T16:43:36.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf227fc640 1 -- 172.21.15.26:0/2205198555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf28004390 con 0x7fdf2c06bee0 2023-12-19T16:43:36.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf227fc640 1 -- 172.21.15.26:0/2205198555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28003860 con 0x7fdf2c06bee0 2023-12-19T16:43:36.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf33cb6640 1 -- 172.21.15.26:0/2205198555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c0cdbd0 con 0x7fdf2c06bee0 2023-12-19T16:43:36.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.319+0000 7fdf33cb6640 1 -- 172.21.15.26:0/2205198555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c0ce190 con 0x7fdf2c06bee0 2023-12-19T16:43:36.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf227fc640 1 -- 172.21.15.26:0/2205198555 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdf280039c0 con 0x7fdf2c06bee0 2023-12-19T16:43:36.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf227fc640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf10065a60 0x7fdf10067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf227fc640 1 -- 172.21.15.26:0/2205198555 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf2811da50 con 0x7fdf2c06bee0 2023-12-19T16:43:36.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf31251640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf10065a60 0x7fdf10067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf37011000 1 Processor -- start 2023-12-19T16:43:36.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf37011000 1 -- start start 2023-12-19T16:43:36.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf31251640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf10065a60 0x7fdf10067f20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0b2550 tx=0x7fdf1c007df0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:36.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf37011000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9070 0x55ca3dcf9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf37011000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9990 0x55ca3dcaec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf37011000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dcaf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf37011000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca3dad2c90 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf37011000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca3db04220 con 0x55ca3dcf9070 2023-12-19T16:43:36.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf37011000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca3db017f0 con 0x55ca3dcf9990 2023-12-19T16:43:36.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf32253640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dcaf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf32253640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dcaf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54806/0 (socket says 172.21.15.26:54806) 2023-12-19T16:43:36.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.323+0000 7fdf32253640 1 -- 172.21.15.26:0/3261669546 learned_addr learned my addr 172.21.15.26:0/3261669546 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:36.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf31a52640 1 --2- 172.21.15.26:0/3261669546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9070 0x55ca3dcf9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf31251640 1 --2- 172.21.15.26:0/3261669546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9990 0x55ca3dcaec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf32253640 1 -- 172.21.15.26:0/3261669546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9990 msgr2=0x55ca3dcaec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:36.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf32253640 1 --2- 172.21.15.26:0/3261669546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9990 0x55ca3dcaec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf32253640 1 -- 172.21.15.26:0/3261669546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9070 msgr2=0x55ca3dcf9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:36.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf32253640 1 --2- 172.21.15.26:0/3261669546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9070 0x55ca3dcf9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf32253640 1 -- 172.21.15.26:0/3261669546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca3d7f9890 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf32253640 1 --2- 172.21.15.26:0/3261669546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dcaf5a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdf240114a0 tx=0x7fdf24009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af85cf0235aaecf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:36.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf0f7fe640 1 -- 172.21.15.26:0/3261669546 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400e040 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf0f7fe640 1 -- 172.21.15.26:0/3261669546 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2400f040 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf0f7fe640 1 -- 172.21.15.26:0/3261669546 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24015a40 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 -- 172.21.15.26:0/3261669546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 msgr2=0x55ca3dcaf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 --2- 172.21.15.26:0/3261669546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dcaf5a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdf240114a0 tx=0x7fdf24009750 comp rx=0 tx=0).stop 2023-12-19T16:43:36.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 -- 172.21.15.26:0/3261669546 shutdown_connections 2023-12-19T16:43:36.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 --2- 172.21.15.26:0/3261669546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dcaf5a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 --2- 172.21.15.26:0/3261669546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9990 0x55ca3dcaec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 --2- 172.21.15.26:0/3261669546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9070 0x55ca3dcf9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 -- 172.21.15.26:0/3261669546 >> 172.21.15.26:0/3261669546 conn(0x55ca3dcb5950 msgr2=0x55ca3dcad560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:36.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 -- 172.21.15.26:0/3261669546 shutdown_connections 2023-12-19T16:43:36.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 -- 172.21.15.26:0/3261669546 wait complete. 2023-12-19T16:43:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 -- start start 2023-12-19T16:43:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 Processor -- start 2023-12-19T16:43:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dcb4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9070 0x55ca3dcbb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9990 0x55ca3dcbb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca3dafd0e0 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca3da9c310 con 0x55ca3dcf9990 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca3daba770 con 0x55ca3dcf9070 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf37011000 10 client.0 _ll_register_callbacks cb 0x55ca3dc9ab60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf31a52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dcb4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf31a52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dcb4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54814/0 (socket says 172.21.15.26:54814) 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.327+0000 7fdf31a52640 1 -- 172.21.15.26:0/662047138 learned_addr learned my addr 172.21.15.26:0/662047138 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf31251640 1 --2- 172.21.15.26:0/662047138 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9070 0x55ca3dcbb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf32253640 1 --2- 172.21.15.26:0/662047138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9990 0x55ca3dcbb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf31a52640 1 -- 172.21.15.26:0/662047138 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9070 msgr2=0x55ca3dcbb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf31a52640 1 --2- 172.21.15.26:0/662047138 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9070 0x55ca3dcbb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf31a52640 1 -- 172.21.15.26:0/662047138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9990 msgr2=0x55ca3dcbb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf31a52640 1 --2- 172.21.15.26:0/662047138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9990 0x55ca3dcbb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf31a52640 1 -- 172.21.15.26:0/662047138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf24011230 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf31a52640 1 --2- 172.21.15.26:0/662047138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dcb4780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdf2801c940 tx=0x7fdf280eeb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc9b67bf4b0e61c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf0cff9640 1 -- 172.21.15.26:0/662047138 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf280f7010 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 -- 172.21.15.26:0/662047138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca3daccc10 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf0cff9640 1 -- 172.21.15.26:0/662047138 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf28003c70 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf0cff9640 1 -- 172.21.15.26:0/662047138 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf28003e30 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 10 client.5301 fetch_fsmap learned FSMap version 17 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 -- 172.21.15.26:0/662047138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55ca3dac2260 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf0cff9640 1 -- 172.21.15.26:0/662047138 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2810a020 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf0cff9640 1 -- 172.21.15.26:0/662047138 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdf280f6b80 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 10 client.5301 fetch_fsmap finished waiting for FSMap version 17 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 10 client.5301 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 20 client.5301 populate_metadata read hostname 'smithi026' 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 --2- 172.21.15.26:0/662047138 >> [v2:172.21.15.26:6834/2660001678,v1:172.21.15.26:6835/2660001678] conn(0x55ca3dd008d0 0x55ca3dd00cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 4 client.5301 mds_command: new command op to 5106 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 -- 172.21.15.26:0/662047138 --> [v2:172.21.15.26:6834/2660001678,v1:172.21.15.26:6835/2660001678] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca3dafba30 con 0x55ca3dd008d0 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf31251640 1 --2- 172.21.15.26:0/662047138 >> [v2:172.21.15.26:6834/2660001678,v1:172.21.15.26:6835/2660001678] conn(0x55ca3dd008d0 0x55ca3dd00cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf31251640 1 --2- 172.21.15.26:0/662047138 >> [v2:172.21.15.26:6834/2660001678,v1:172.21.15.26:6835/2660001678] conn(0x55ca3dd008d0 0x55ca3dd00cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca9658b47edd61f8 server_cookie=7836b7873fbd2739 in_seq=0 out_seq=0 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf0cff9640 10 client.5301 ms_handle_connect on v2:172.21.15.26:6834/2660001678 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf0cff9640 1 -- 172.21.15.26:0/662047138 <== mds.0 v2:172.21.15.26:6834/2660001678 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ca3dafba30 con 0x55ca3dd008d0 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf0cff9640 10 client.5301 handle_command_reply: tid=0 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 client.5301 shutdown 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 -- 172.21.15.26:0/662047138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 msgr2=0x55ca3dcb4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 --2- 172.21.15.26:0/662047138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dcb4780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdf2801c940 tx=0x7fdf280eeb00 comp rx=0 tx=0).stop 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 -- 172.21.15.26:0/662047138 shutdown_connections 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 --2- 172.21.15.26:0/662047138 >> [v2:172.21.15.26:6834/2660001678,v1:172.21.15.26:6835/2660001678] conn(0x55ca3dd008d0 0x55ca3dd00cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 --2- 172.21.15.26:0/662047138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9990 0x55ca3dcbb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 --2- 172.21.15.26:0/662047138 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9070 0x55ca3dcbb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 --2- 172.21.15.26:0/662047138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dcb4780 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf0cff9640 0 client.5301 ms_handle_reset on v2:172.21.15.26:6834/2660001678 2023-12-19T16:43:36.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 -- 172.21.15.26:0/662047138 >> 172.21.15.26:0/662047138 conn(0x55ca3dcb5950 msgr2=0x55ca3dcadc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:36.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 -- 172.21.15.26:0/662047138 shutdown_connections 2023-12-19T16:43:36.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 1 -- 172.21.15.26:0/662047138 wait complete. 2023-12-19T16:43:36.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.331+0000 7fdf37011000 20 client.5301 trim_cache size 0 max 16384 2023-12-19T16:43:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.347+0000 7fdf37011000 1 Processor -- start 2023-12-19T16:43:36.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.347+0000 7fdf37011000 1 -- start start 2023-12-19T16:43:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.347+0000 7fdf37011000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dd512f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.347+0000 7fdf37011000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9070 0x55ca3dd51830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.347+0000 7fdf37011000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9990 0x55ca3dd51d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.347+0000 7fdf37011000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca3d7ba940 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.347+0000 7fdf37011000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca3db054b0 con 0x55ca3dcf9990 2023-12-19T16:43:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.347+0000 7fdf37011000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca3dcb28c0 con 0x55ca3dcf9070 2023-12-19T16:43:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.347+0000 7fdf31a52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dd512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.347+0000 7fdf31a52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dd512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54822/0 (socket says 172.21.15.26:54822) 2023-12-19T16:43:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.347+0000 7fdf31a52640 1 -- 172.21.15.26:0/1085722601 learned_addr learned my addr 172.21.15.26:0/1085722601 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:36.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf31251640 1 --2- 172.21.15.26:0/1085722601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9070 0x55ca3dd51830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf32253640 1 --2- 172.21.15.26:0/1085722601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9990 0x55ca3dd51d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf31a52640 1 -- 172.21.15.26:0/1085722601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9070 msgr2=0x55ca3dd51830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf31a52640 1 --2- 172.21.15.26:0/1085722601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9070 0x55ca3dd51830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf31a52640 1 -- 172.21.15.26:0/1085722601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9990 msgr2=0x55ca3dd51d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf31a52640 1 --2- 172.21.15.26:0/1085722601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9990 0x55ca3dd51d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf31a52640 1 -- 172.21.15.26:0/1085722601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca3daa83b0 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf31251640 1 --2- 172.21.15.26:0/1085722601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9070 0x55ca3dd51830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf31a52640 1 --2- 172.21.15.26:0/1085722601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dd512f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdf280eb5e0 tx=0x7fdf280c2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f197446c4124e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf0ffff640 1 -- 172.21.15.26:0/1085722601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf280eec20 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf0ffff640 1 -- 172.21.15.26:0/1085722601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2810a040 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf0ffff640 1 -- 172.21.15.26:0/1085722601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf280f7010 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 -- 172.21.15.26:0/1085722601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 msgr2=0x55ca3dd512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 --2- 172.21.15.26:0/1085722601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dd512f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdf280eb5e0 tx=0x7fdf280c2000 comp rx=0 tx=0).stop 2023-12-19T16:43:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 -- 172.21.15.26:0/1085722601 shutdown_connections 2023-12-19T16:43:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 --2- 172.21.15.26:0/1085722601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcf9990 0x55ca3dd51d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 --2- 172.21.15.26:0/1085722601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9070 0x55ca3dd51830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 --2- 172.21.15.26:0/1085722601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dd512f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 -- 172.21.15.26:0/1085722601 >> 172.21.15.26:0/1085722601 conn(0x55ca3dcb5950 msgr2=0x55ca3dcbaf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 -- 172.21.15.26:0/1085722601 shutdown_connections 2023-12-19T16:43:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 -- 172.21.15.26:0/1085722601 wait complete. 2023-12-19T16:43:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 -- start start 2023-12-19T16:43:36.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 Processor -- start 2023-12-19T16:43:36.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dd39f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcf9070 0x55ca3dd3a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9990 0x55ca3dd3a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca3dafbf70 con 0x55ca3dcf9070 2023-12-19T16:43:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca3dbcdb30 con 0x55ca3dcaf1c0 2023-12-19T16:43:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca3db40ce0 con 0x55ca3dcf9990 2023-12-19T16:43:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf37011000 10 client.0 _ll_register_callbacks cb 0x55ca3daab640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T16:43:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf31251640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcf9070 0x55ca3dd3a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf31251640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcf9070 0x55ca3dd3a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54838/0 (socket says 172.21.15.26:54838) 2023-12-19T16:43:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf31251640 1 -- 172.21.15.26:0/2597172251 learned_addr learned my addr 172.21.15.26:0/2597172251 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf31a52640 1 --2- 172.21.15.26:0/2597172251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dd39f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.351+0000 7fdf32253640 1 --2- 172.21.15.26:0/2597172251 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9990 0x55ca3dd3a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf31251640 1 -- 172.21.15.26:0/2597172251 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9990 msgr2=0x55ca3dd3a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf31251640 1 --2- 172.21.15.26:0/2597172251 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9990 0x55ca3dd3a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf31251640 1 -- 172.21.15.26:0/2597172251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcaf1c0 msgr2=0x55ca3dd39f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf31251640 1 --2- 172.21.15.26:0/2597172251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dd39f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf31251640 1 -- 172.21.15.26:0/2597172251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca3dac2260 con 0x55ca3dcf9070 2023-12-19T16:43:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf31251640 1 --2- 172.21.15.26:0/2597172251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcf9070 0x55ca3dd3a480 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c14fe00 tx=0x7fdf1c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b804a5059220ad server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T16:43:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf0cff9640 1 -- 172.21.15.26:0/2597172251 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c001d80 con 0x55ca3dcf9070 2023-12-19T16:43:36.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf0cff9640 1 -- 172.21.15.26:0/2597172251 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf1c01b070 con 0x55ca3dcf9070 2023-12-19T16:43:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 -- 172.21.15.26:0/2597172251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca3dac2e00 con 0x55ca3dcf9070 2023-12-19T16:43:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf0cff9640 1 -- 172.21.15.26:0/2597172251 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf1c0133f0 con 0x55ca3dcf9070 2023-12-19T16:43:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 10 client.5307 fetch_fsmap learned FSMap version 17 2023-12-19T16:43:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 -- 172.21.15.26:0/2597172251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55ca3d7f9cc0 con 0x55ca3dcf9070 2023-12-19T16:43:36.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf0cff9640 1 -- 172.21.15.26:0/2597172251 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca3dac2e00 con 0x55ca3dcf9070 2023-12-19T16:43:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf0cff9640 1 -- 172.21.15.26:0/2597172251 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdf1c012070 con 0x55ca3dcf9070 2023-12-19T16:43:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 10 client.5307 fetch_fsmap finished waiting for FSMap version 17 2023-12-19T16:43:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 10 client.5307 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T16:43:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 20 client.5307 populate_metadata read hostname 'smithi026' 2023-12-19T16:43:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 --2- 172.21.15.26:0/2597172251 >> [v2:172.21.15.26:6834/2660001678,v1:172.21.15.26:6835/2660001678] conn(0x55ca3dd3af00 0x55ca3dd3b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 4 client.5307 mds_command: new command op to 5106 tid=0[{"prefix": "status"}] 2023-12-19T16:43:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 -- 172.21.15.26:0/2597172251 --> [v2:172.21.15.26:6834/2660001678,v1:172.21.15.26:6835/2660001678] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ca3daf6160 con 0x55ca3dd3af00 2023-12-19T16:43:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf31a52640 1 --2- 172.21.15.26:0/2597172251 >> [v2:172.21.15.26:6834/2660001678,v1:172.21.15.26:6835/2660001678] conn(0x55ca3dd3af00 0x55ca3dd3b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf0cff9640 10 client.5307 ms_handle_connect on v2:172.21.15.26:6834/2660001678 2023-12-19T16:43:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf31a52640 1 --2- 172.21.15.26:0/2597172251 >> [v2:172.21.15.26:6834/2660001678,v1:172.21.15.26:6835/2660001678] conn(0x55ca3dd3af00 0x55ca3dd3b2e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b113569214e67f07 server_cookie=f82dd62a31db787e in_seq=0 out_seq=0 2023-12-19T16:43:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf0cff9640 1 -- 172.21.15.26:0/2597172251 <== mds.0 v2:172.21.15.26:6834/2660001678 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55ca3daf6160 con 0x55ca3dd3af00 2023-12-19T16:43:36.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf0cff9640 10 client.5307 handle_command_reply: tid=0 2023-12-19T16:43:36.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 client.5307 shutdown 2023-12-19T16:43:36.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 -- 172.21.15.26:0/2597172251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcf9070 msgr2=0x55ca3dd3a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 --2- 172.21.15.26:0/2597172251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcf9070 0x55ca3dd3a480 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c14fe00 tx=0x7fdf1c012000 comp rx=0 tx=0).stop 2023-12-19T16:43:36.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 -- 172.21.15.26:0/2597172251 shutdown_connections 2023-12-19T16:43:36.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 --2- 172.21.15.26:0/2597172251 >> [v2:172.21.15.26:6834/2660001678,v1:172.21.15.26:6835/2660001678] conn(0x55ca3dd3af00 0x55ca3dd3b2e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 --2- 172.21.15.26:0/2597172251 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ca3dcf9990 0x55ca3dd3a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 --2- 172.21.15.26:0/2597172251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ca3dcf9070 0x55ca3dd3a480 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 --2- 172.21.15.26:0/2597172251 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca3dcaf1c0 0x55ca3dd39f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf0cff9640 0 client.5307 ms_handle_reset on v2:172.21.15.26:6834/2660001678 2023-12-19T16:43:36.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 -- 172.21.15.26:0/2597172251 >> 172.21.15.26:0/2597172251 conn(0x55ca3dcb5950 msgr2=0x55ca3dcf6060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:36.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 -- 172.21.15.26:0/2597172251 shutdown_connections 2023-12-19T16:43:36.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 1 -- 172.21.15.26:0/2597172251 wait complete. 2023-12-19T16:43:36.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf37011000 20 client.5307 trim_cache size 0 max 16384 2023-12-19T16:43:36.373 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T16:43:36.373 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T16:43:36.373 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T16:43:36.373 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5106, 2023-12-19T16:43:36.373 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T16:43:36.374 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T16:43:36.374 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T16:43:36.374 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 10.490193758, 2023-12-19T16:43:36.374 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 17, 2023-12-19T16:43:36.374 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 37, 2023-12-19T16:43:36.374 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 37, 2023-12-19T16:43:36.375 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 17.541513394999999 2023-12-19T16:43:36.375 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T16:43:36.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf33cb6640 1 -- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf10065a60 msgr2=0x7fdf10067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf33cb6640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf10065a60 0x7fdf10067f20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0b2550 tx=0x7fdf1c007df0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf33cb6640 1 -- 172.21.15.26:0/2205198555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c06bee0 msgr2=0x7fdf2c0ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf33cb6640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c06bee0 0x7fdf2c0ce6b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdf28000c10 tx=0x7fdf2801ac10 comp rx=0 tx=0).stop 2023-12-19T16:43:36.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf33cb6640 1 -- 172.21.15.26:0/2205198555 shutdown_connections 2023-12-19T16:43:36.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf33cb6640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf2c0cb1d0 0x7fdf2c1619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf33cb6640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf10065a60 0x7fdf10067f20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf33cb6640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf2c0b1d40 0x7fdf2c0afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf33cb6640 1 --2- 172.21.15.26:0/2205198555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c06bee0 0x7fdf2c0ce6b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.355+0000 7fdf33cb6640 1 -- 172.21.15.26:0/2205198555 >> 172.21.15.26:0/2205198555 conn(0x7fdf2c0658e0 msgr2=0x7fdf2c06f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:36.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.359+0000 7fdf33cb6640 1 -- 172.21.15.26:0/2205198555 shutdown_connections 2023-12-19T16:43:36.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.359+0000 7fdf33cb6640 1 -- 172.21.15.26:0/2205198555 wait complete. 2023-12-19T16:43:36.382 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T16:43:36.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.519+0000 7fa8820fc640 1 Processor -- start 2023-12-19T16:43:36.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.519+0000 7fa8820fc640 1 -- start start 2023-12-19T16:43:36.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.523+0000 7fa8820fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa87c154220 0x7fa87c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.523+0000 7fa8820fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa87c06b3d0 0x7fa87c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.523+0000 7fa8820fc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87c06bc30 con 0x7fa87c0cec40 2023-12-19T16:43:36.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.523+0000 7fa8820fc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87c0cf0d0 con 0x7fa87c154220 2023-12-19T16:43:36.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.523+0000 7fa8820fc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87c0cc6e0 con 0x7fa87c06b3d0 2023-12-19T16:43:36.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.523+0000 7fa87b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa87c154220 0x7fa87c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.523+0000 7fa87b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa87c154220 0x7fa87c15aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54842/0 (socket says 172.21.15.26:54842) 2023-12-19T16:43:36.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.523+0000 7fa87b7fe640 1 -- 172.21.15.26:0/2203182215 learned_addr learned my addr 172.21.15.26:0/2203182215 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:36.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.523+0000 7fa87affd640 1 --2- 172.21.15.26:0/2203182215 >> v2:172.21.15.160:3300/0 conn(0x7fa87c06b3d0 0x7fa87c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.523+0000 7fa87b7fe640 1 -- 172.21.15.26:0/2203182215 >> v2:172.21.15.160:3300/0 conn(0x7fa87c06b3d0 msgr2=0x7fa87c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:36.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.523+0000 7fa87b7fe640 1 --2- 172.21.15.26:0/2203182215 >> v2:172.21.15.160:3300/0 conn(0x7fa87c06b3d0 0x7fa87c0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.523+0000 7fa87b7fe640 1 -- 172.21.15.26:0/2203182215 >> v1:172.21.15.160:6789/0 conn(0x7fa87c0cec40 legacy=0x7fa87c15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.523+0000 7fa87b7fe640 1 -- 172.21.15.26:0/2203182215 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c15afc0 con 0x7fa87c154220 2023-12-19T16:43:36.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa87b7fe640 1 --2- 172.21.15.26:0/2203182215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c154220 0x7fa87c15aa80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0cf880 tx=0x7fa86c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e65a51fed07808 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:36.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa87a7fc640 1 -- 172.21.15.26:0/2203182215 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0ab5f0 con 0x7fa87c154220 2023-12-19T16:43:36.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa87a7fc640 1 -- 172.21.15.26:0/2203182215 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c0ab750 con 0x7fa87c154220 2023-12-19T16:43:36.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa87a7fc640 1 -- 172.21.15.26:0/2203182215 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0abae0 con 0x7fa87c154220 2023-12-19T16:43:36.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa8820fc640 1 -- 172.21.15.26:0/2203182215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c154220 msgr2=0x7fa87c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa8820fc640 1 --2- 172.21.15.26:0/2203182215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c154220 0x7fa87c15aa80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0cf880 tx=0x7fa86c01c4d0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa8820fc640 1 -- 172.21.15.26:0/2203182215 shutdown_connections 2023-12-19T16:43:36.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa8820fc640 1 --2- 172.21.15.26:0/2203182215 >> v2:172.21.15.160:3300/0 conn(0x7fa87c06b3d0 0x7fa87c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa8820fc640 1 --2- 172.21.15.26:0/2203182215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c154220 0x7fa87c15aa80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa8820fc640 1 -- 172.21.15.26:0/2203182215 >> 172.21.15.26:0/2203182215 conn(0x7fa87c0658e0 msgr2=0x7fa87c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:36.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa8820fc640 1 -- 172.21.15.26:0/2203182215 shutdown_connections 2023-12-19T16:43:36.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa8820fc640 1 -- 172.21.15.26:0/2203182215 wait complete. 2023-12-19T16:43:36.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa8820fc640 1 Processor -- start 2023-12-19T16:43:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.527+0000 7fa8820fc640 1 -- start start 2023-12-19T16:43:36.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa8820fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa87c06b3d0 0x7fa87c14ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa8820fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa87c0cec40 0x7fa87c0703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa8820fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c154220 0x7fa87c070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa8820fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c0b3a00 con 0x7fa87c154220 2023-12-19T16:43:36.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa8820fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c14da00 con 0x7fa87c0cec40 2023-12-19T16:43:36.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa8820fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87c0b3460 con 0x7fa87c06b3d0 2023-12-19T16:43:36.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa87bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c154220 0x7fa87c070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa87bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c154220 0x7fa87c070900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54848/0 (socket says 172.21.15.26:54848) 2023-12-19T16:43:36.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa87affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa87c0cec40 0x7fa87c0703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa87bfff640 1 -- 172.21.15.26:0/1902262569 learned_addr learned my addr 172.21.15.26:0/1902262569 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:36.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa87b7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa87c06b3d0 0x7fa87c14ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa87bfff640 1 -- 172.21.15.26:0/1902262569 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa87c06b3d0 msgr2=0x7fa87c14ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa87bfff640 1 --2- 172.21.15.26:0/1902262569 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa87c06b3d0 0x7fa87c14ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa87bfff640 1 -- 172.21.15.26:0/1902262569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa87c0cec40 msgr2=0x7fa87c0703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa87bfff640 1 --2- 172.21.15.26:0/1902262569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa87c0cec40 0x7fa87c0703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa87bfff640 1 -- 172.21.15.26:0/1902262569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0094d0 con 0x7fa87c154220 2023-12-19T16:43:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa87bfff640 1 --2- 172.21.15.26:0/1902262569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c154220 0x7fa87c070900 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa87400c730 tx=0x7fa87400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa863fff640 1 -- 172.21.15.26:0/1902262569 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874003b80 con 0x7fa87c154220 2023-12-19T16:43:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa863fff640 1 -- 172.21.15.26:0/1902262569 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa874003ce0 con 0x7fa87c154220 2023-12-19T16:43:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa863fff640 1 -- 172.21.15.26:0/1902262569 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87400aa30 con 0x7fa87c154220 2023-12-19T16:43:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa8820fc640 1 -- 172.21.15.26:0/1902262569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87c0b3720 con 0x7fa87c154220 2023-12-19T16:43:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa8820fc640 1 -- 172.21.15.26:0/1902262569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87c14f480 con 0x7fa87c154220 2023-12-19T16:43:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa8820fc640 1 -- 172.21.15.26:0/1902262569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87c074270 con 0x7fa87c154220 2023-12-19T16:43:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.531+0000 7fa863fff640 1 -- 172.21.15.26:0/1902262569 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa87400ab90 con 0x7fa87c154220 2023-12-19T16:43:36.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.535+0000 7fa863fff640 1 --2- 172.21.15.26:0/1902262569 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa868065a60 0x7fa868067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:36.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.535+0000 7fa863fff640 1 -- 172.21.15.26:0/1902262569 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa874082d50 con 0x7fa87c154220 2023-12-19T16:43:36.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.535+0000 7fa87b7fe640 1 --2- 172.21.15.26:0/1902262569 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa868065a60 0x7fa868067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:36.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.535+0000 7fa87b7fe640 1 --2- 172.21.15.26:0/1902262569 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa868065a60 0x7fa868067f20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0af000 tx=0x7fa86c0b6040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:36.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.539+0000 7fa863fff640 1 -- 172.21.15.26:0/1902262569 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa874086030 con 0x7fa87c154220 2023-12-19T16:43:36.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.731+0000 7fa8820fc640 1 -- 172.21.15.26:0/1902262569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa87c155310 con 0x7fa87c154220 2023-12-19T16:43:36.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.731+0000 7fa863fff640 1 -- 172.21.15.26:0/1902262569 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa874050850 con 0x7fa87c154220 2023-12-19T16:43:36.737 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T16:43:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.735+0000 7fa8820fc640 1 -- 172.21.15.26:0/1902262569 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa868065a60 msgr2=0x7fa868067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.735+0000 7fa8820fc640 1 --2- 172.21.15.26:0/1902262569 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa868065a60 0x7fa868067f20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa86c0af000 tx=0x7fa86c0b6040 comp rx=0 tx=0).stop 2023-12-19T16:43:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.735+0000 7fa8820fc640 1 -- 172.21.15.26:0/1902262569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c154220 msgr2=0x7fa87c070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:36.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.735+0000 7fa8820fc640 1 --2- 172.21.15.26:0/1902262569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c154220 0x7fa87c070900 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa87400c730 tx=0x7fa87400cbf0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.735+0000 7fa8820fc640 1 -- 172.21.15.26:0/1902262569 shutdown_connections 2023-12-19T16:43:36.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.735+0000 7fa8820fc640 1 --2- 172.21.15.26:0/1902262569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa87c154220 0x7fa87c070900 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.735+0000 7fa8820fc640 1 --2- 172.21.15.26:0/1902262569 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa87c0cec40 0x7fa87c0703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.735+0000 7fa8820fc640 1 --2- 172.21.15.26:0/1902262569 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa868065a60 0x7fa868067f20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.735+0000 7fa8820fc640 1 --2- 172.21.15.26:0/1902262569 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa87c06b3d0 0x7fa87c14ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:36.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.735+0000 7fa8820fc640 1 -- 172.21.15.26:0/1902262569 >> 172.21.15.26:0/1902262569 conn(0x7fa87c0658e0 msgr2=0x7fa87c067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:36.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.735+0000 7fa8820fc640 1 -- 172.21.15.26:0/1902262569 shutdown_connections 2023-12-19T16:43:36.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:36.735+0000 7fa8820fc640 1 -- 172.21.15.26:0/1902262569 wait complete. 2023-12-19T16:43:36.759 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-19T16:43:36.759 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T16:43:36.759 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T16:43:36.759 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T16:43:36.759 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T16:43:36.759 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-19T16:43:36.759 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-19T16:43:36.759 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T16:43:36.759 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:43:36.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:36.760 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T16:43:36.770 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T16:43:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T16:43:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T16:43:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:43:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T16:43:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:43:36.771 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:43:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T16:43:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T16:43:36.772 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:43:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T16:43:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T16:43:36.772 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T16:43:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:43:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T16:43:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T16:43:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42324sec preferred_lft 42324sec 2023-12-19T16:43:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T16:43:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:43:36.773 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:43:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:43:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T16:43:36.774 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T16:43:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T16:43:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T16:43:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:43:36.775 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T16:43:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T16:43:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T16:43:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:43:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-19T16:43:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:43:36.776 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T16:43:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T16:43:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-19T16:43:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:43:36.776 INFO:teuthology.orchestra.run.smithi026.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T16:43:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T16:43:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-19T16:43:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:43:36.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:36.777 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T16:43:36.838 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T16:43:36.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:36.840 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T16:43:36.893 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T16:43:36.893 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T16:43:36.893 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T16:43:36.893 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T16:43:36.893 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T16:43:36.893 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T16:43:36.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:36.894 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:43:36.894 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:43:36.894 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T16:43:36.894 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T16:43:36.894 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T16:43:36.894 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:43:36.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:36.987 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T16:43:36.997 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T16:43:36.997 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T16:43:36.997 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:43:36.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:36.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T16:43:37.060 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:43:37.060 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T16:43:37.111 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:43:37.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:43:37.111 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T16:43:37.162 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T16:43:37.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:37.162 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:43:37.254 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:43:37.254 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:43:37.267 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:43:37.267 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:43:37.267 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T16:43:37.267 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T16:43:37.268 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T16:43:37.271 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T16:43:37.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:37.271 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T16:43:37.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:37.294 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:43:37.356 INFO:teuthology.orchestra.run.smithi026.stdout:client4925 2023-12-19T16:43:37.356 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:43:37.357 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:43:37.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-19T16:43:37.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.595+0000 7f7abf0d1640 1 Processor -- start 2023-12-19T16:43:37.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7abf0d1640 1 -- start start 2023-12-19T16:43:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7abf0d1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7ab806b3d0 0x7f7ab80ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7abf0d1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7ab80cec40 0x7f7ab815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7abf0d1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab806bc30 con 0x7f7ab8154220 2023-12-19T16:43:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7abf0d1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab80cf0d0 con 0x7f7ab80cec40 2023-12-19T16:43:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7abf0d1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab80cc6e0 con 0x7f7ab806b3d0 2023-12-19T16:43:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7abce6d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7ab8154220 0x7f7ab815aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35150/0 (socket says 172.21.15.26:35150) 2023-12-19T16:43:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7abce6d640 1 -- 172.21.15.26:0/488970834 learned_addr learned my addr 172.21.15.26:0/488970834 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7abd66e640 1 --2- 172.21.15.26:0/488970834 >> v2:172.21.15.26:3300/0 conn(0x7f7ab80cec40 0x7f7ab815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7ab7fff640 1 --2- 172.21.15.26:0/488970834 >> v2:172.21.15.160:3300/0 conn(0x7f7ab806b3d0 0x7f7ab80ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7ab77fe640 1 -- 172.21.15.26:0/488970834 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716007652 0 0) 0x7f7ab806bc30 con 0x7f7ab8154220 2023-12-19T16:43:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7ab77fe640 1 -- 172.21.15.26:0/488970834 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aa8003530 con 0x7f7ab8154220 2023-12-19T16:43:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7abd66e640 1 -- 172.21.15.26:0/488970834 >> v2:172.21.15.160:3300/0 conn(0x7f7ab806b3d0 msgr2=0x7f7ab80ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7abd66e640 1 --2- 172.21.15.26:0/488970834 >> v2:172.21.15.160:3300/0 conn(0x7f7ab806b3d0 0x7f7ab80ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7abd66e640 1 -- 172.21.15.26:0/488970834 >> v1:172.21.15.26:6789/0 conn(0x7f7ab8154220 legacy=0x7f7ab815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:37.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.599+0000 7f7abd66e640 1 -- 172.21.15.26:0/488970834 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab806b950 con 0x7f7ab80cec40 2023-12-19T16:43:37.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7abd66e640 1 --2- 172.21.15.26:0/488970834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab80cec40 0x7f7ab815b1c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80cf880 tx=0x7f7ab001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ad8940a56856eb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:37.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7ab77fe640 1 -- 172.21.15.26:0/488970834 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00a59a0 con 0x7f7ab80cec40 2023-12-19T16:43:37.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7ab77fe640 1 -- 172.21.15.26:0/488970834 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab00a5b00 con 0x7f7ab80cec40 2023-12-19T16:43:37.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7ab77fe640 1 -- 172.21.15.26:0/488970834 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00a5e90 con 0x7f7ab80cec40 2023-12-19T16:43:37.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7abf0d1640 1 -- 172.21.15.26:0/488970834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab80cec40 msgr2=0x7f7ab815b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:37.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7abf0d1640 1 --2- 172.21.15.26:0/488970834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab80cec40 0x7f7ab815b1c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80cf880 tx=0x7f7ab001c8e0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7abf0d1640 1 -- 172.21.15.26:0/488970834 shutdown_connections 2023-12-19T16:43:37.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7abf0d1640 1 --2- 172.21.15.26:0/488970834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab80cec40 0x7f7ab815b1c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7abf0d1640 1 --2- 172.21.15.26:0/488970834 >> v2:172.21.15.160:3300/0 conn(0x7f7ab806b3d0 0x7f7ab80ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7abf0d1640 1 -- 172.21.15.26:0/488970834 >> 172.21.15.26:0/488970834 conn(0x7f7ab80658e0 msgr2=0x7f7ab8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:37.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7abf0d1640 1 -- 172.21.15.26:0/488970834 shutdown_connections 2023-12-19T16:43:37.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7abf0d1640 1 -- 172.21.15.26:0/488970834 wait complete. 2023-12-19T16:43:37.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7abf0d1640 1 Processor -- start 2023-12-19T16:43:37.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.603+0000 7f7abf0d1640 1 -- start start 2023-12-19T16:43:37.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abf0d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab806b3d0 0x7f7ab814e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:37.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abf0d1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ab80cec40 0x7f7ab814dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:37.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abf0d1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab8154220 0x7f7ab806fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:37.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abf0d1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab8070360 con 0x7f7ab806b3d0 2023-12-19T16:43:37.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abf0d1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab8070640 con 0x7f7ab8154220 2023-12-19T16:43:37.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abf0d1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab8070920 con 0x7f7ab80cec40 2023-12-19T16:43:37.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abce6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab806b3d0 0x7f7ab814e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:37.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abce6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab806b3d0 0x7f7ab814e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54856/0 (socket says 172.21.15.26:54856) 2023-12-19T16:43:37.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abce6d640 1 -- 172.21.15.26:0/503508197 learned_addr learned my addr 172.21.15.26:0/503508197 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:37.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7ab7fff640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ab80cec40 0x7f7ab814dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:37.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abd66e640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab8154220 0x7f7ab806fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:37.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abce6d640 1 -- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ab80cec40 msgr2=0x7f7ab814dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:37.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abce6d640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ab80cec40 0x7f7ab814dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abce6d640 1 -- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab8154220 msgr2=0x7f7ab806fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:37.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abce6d640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab8154220 0x7f7ab806fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abce6d640 1 -- 172.21.15.26:0/503508197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab0009a70 con 0x7f7ab806b3d0 2023-12-19T16:43:37.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abd66e640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab8154220 0x7f7ab806fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:37.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abce6d640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab806b3d0 0x7f7ab814e810 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00c4d0 tx=0x7f7aac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:37.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7ab57fa640 1 -- 172.21.15.26:0/503508197 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac005cf0 con 0x7f7ab806b3d0 2023-12-19T16:43:37.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7ab57fa640 1 -- 172.21.15.26:0/503508197 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aac005e50 con 0x7f7ab806b3d0 2023-12-19T16:43:37.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7ab57fa640 1 -- 172.21.15.26:0/503508197 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac007270 con 0x7f7ab806b3d0 2023-12-19T16:43:37.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abf0d1640 1 -- 172.21.15.26:0/503508197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab8070c00 con 0x7f7ab806b3d0 2023-12-19T16:43:37.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abf0d1640 1 -- 172.21.15.26:0/503508197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab814e100 con 0x7f7ab806b3d0 2023-12-19T16:43:37.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7abf0d1640 1 -- 172.21.15.26:0/503508197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab8074270 con 0x7f7ab806b3d0 2023-12-19T16:43:37.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.607+0000 7f7ab57fa640 1 -- 172.21.15.26:0/503508197 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f7aac0073d0 con 0x7f7ab806b3d0 2023-12-19T16:43:37.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.611+0000 7f7ab57fa640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a9c065a60 0x7f7a9c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:37.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.611+0000 7f7ab57fa640 1 -- 172.21.15.26:0/503508197 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7aac0827a0 con 0x7f7ab806b3d0 2023-12-19T16:43:37.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.611+0000 7f7ab7fff640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a9c065a60 0x7f7a9c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.611+0000 7f7ab7fff640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a9c065a60 0x7f7a9c067f20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80b3b30 tx=0x7f7aa4009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:37.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.615+0000 7f7ab57fa640 1 -- 172.21.15.26:0/503508197 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7aac0500e0 con 0x7f7ab806b3d0 2023-12-19T16:43:37.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.795+0000 7f7abf0d1640 1 -- 172.21.15.26:0/503508197 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ab80ccef0 con 0x7f7a9c065a60 2023-12-19T16:43:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.795+0000 7f7ab57fa640 1 -- 172.21.15.26:0/503508197 <== mgr.4106 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f7aa4004040 con 0x7f7a9c065a60 2023-12-19T16:43:37.802 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T16:43:37.802 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:43:37.802 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2023-12-19T16:43:37.802 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:43:37.802 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T16:43:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.799+0000 7f7abf0d1640 1 -- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a9c065a60 msgr2=0x7f7a9c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.799+0000 7f7abf0d1640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a9c065a60 0x7f7a9c067f20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80b3b30 tx=0x7f7aa4009190 comp rx=0 tx=0).stop 2023-12-19T16:43:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.799+0000 7f7abf0d1640 1 -- 172.21.15.26:0/503508197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab806b3d0 msgr2=0x7f7ab814e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.799+0000 7f7abf0d1640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab806b3d0 0x7f7ab814e810 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00c4d0 tx=0x7f7aac00c990 comp rx=0 tx=0).stop 2023-12-19T16:43:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.799+0000 7f7abf0d1640 1 -- 172.21.15.26:0/503508197 shutdown_connections 2023-12-19T16:43:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.799+0000 7f7abf0d1640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab8154220 0x7f7ab806fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.799+0000 7f7abf0d1640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a9c065a60 0x7f7a9c067f20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.799+0000 7f7abf0d1640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ab80cec40 0x7f7ab814dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.799+0000 7f7abf0d1640 1 --2- 172.21.15.26:0/503508197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab806b3d0 0x7f7ab814e810 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.799+0000 7f7abf0d1640 1 -- 172.21.15.26:0/503508197 >> 172.21.15.26:0/503508197 conn(0x7f7ab80658e0 msgr2=0x7f7ab80b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.799+0000 7f7abf0d1640 1 -- 172.21.15.26:0/503508197 shutdown_connections 2023-12-19T16:43:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.799+0000 7f7abf0d1640 1 -- 172.21.15.26:0/503508197 wait complete. 2023-12-19T16:43:37.823 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-19T16:43:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f5570f0a640 1 Processor -- start 2023-12-19T16:43:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f5570f0a640 1 -- start start 2023-12-19T16:43:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f5570f0a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f556c1599e0 0x7f556c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f5570f0a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f556c0cb150 0x7f556c0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f5570f0a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f556c0cbaf0 con 0x7f556c15d530 2023-12-19T16:43:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f5570f0a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f556c15e050 con 0x7f556c0cb150 2023-12-19T16:43:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f5570f0a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c15e240 con 0x7f556c1599e0 2023-12-19T16:43:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f5563fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f556c0cb150 0x7f556c0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f556ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f556c15d530 0x7f556c15d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35158/0 (socket says 172.21.15.26:35158) 2023-12-19T16:43:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f556ad9d640 1 -- 172.21.15.26:0/1087864226 learned_addr learned my addr 172.21.15.26:0/1087864226 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f556a59c640 1 --2- 172.21.15.26:0/1087864226 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f556c1599e0 0x7f556c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f5563fff640 1 -- 172.21.15.26:0/1087864226 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f556c1599e0 msgr2=0x7f556c06f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f5563fff640 1 --2- 172.21.15.26:0/1087864226 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f556c1599e0 0x7f556c06f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f5563fff640 1 -- 172.21.15.26:0/1087864226 >> v1:172.21.15.26:6789/0 conn(0x7f556c15d530 legacy=0x7f556c15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f5563fff640 1 -- 172.21.15.26:0/1087864226 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c0c9520 con 0x7f556c0cb150 2023-12-19T16:43:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.963+0000 7f5569d9b640 1 -- 172.21.15.26:0/1087864226 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013913537 0 0) 0x7f556c0cbaf0 con 0x7f556c15d530 2023-12-19T16:43:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.967+0000 7f5563fff640 1 --2- 172.21.15.26:0/1087864226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f556c0cb150 0x7f556c0cb5b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f55540097c0 tx=0x7f555401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad4e96bf50190a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.967+0000 7f5569d9b640 1 -- 172.21.15.26:0/1087864226 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55540ab680 con 0x7f556c0cb150 2023-12-19T16:43:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.967+0000 7f5569d9b640 1 -- 172.21.15.26:0/1087864226 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55540ab7e0 con 0x7f556c0cb150 2023-12-19T16:43:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.967+0000 7f5569d9b640 1 -- 172.21.15.26:0/1087864226 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55540abb70 con 0x7f556c0cb150 2023-12-19T16:43:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.967+0000 7f5570f0a640 1 -- 172.21.15.26:0/1087864226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f556c0cb150 msgr2=0x7f556c0cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.967+0000 7f5570f0a640 1 --2- 172.21.15.26:0/1087864226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f556c0cb150 0x7f556c0cb5b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f55540097c0 tx=0x7f555401c480 comp rx=0 tx=0).stop 2023-12-19T16:43:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.967+0000 7f5570f0a640 1 -- 172.21.15.26:0/1087864226 shutdown_connections 2023-12-19T16:43:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.967+0000 7f5570f0a640 1 --2- 172.21.15.26:0/1087864226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f556c0cb150 0x7f556c0cb5b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.967+0000 7f5570f0a640 1 --2- 172.21.15.26:0/1087864226 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f556c1599e0 0x7f556c06f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.967+0000 7f5570f0a640 1 -- 172.21.15.26:0/1087864226 >> 172.21.15.26:0/1087864226 conn(0x7f556c066160 msgr2=0x7f556c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:37.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.967+0000 7f5570f0a640 1 -- 172.21.15.26:0/1087864226 shutdown_connections 2023-12-19T16:43:37.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.967+0000 7f5570f0a640 1 -- 172.21.15.26:0/1087864226 wait complete. 2023-12-19T16:43:37.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5570f0a640 1 Processor -- start 2023-12-19T16:43:37.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5570f0a640 1 -- start start 2023-12-19T16:43:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5570f0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f556c0cb150 0x7f556c151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5570f0a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f556c1599e0 0x7f556c0c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5570f0a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f556c15d530 0x7f556c155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5570f0a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c157ef0 con 0x7f556c0cb150 2023-12-19T16:43:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5570f0a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c158670 con 0x7f556c1599e0 2023-12-19T16:43:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5570f0a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c0c7f30 con 0x7f556c15d530 2023-12-19T16:43:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f556a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f556c0cb150 0x7f556c151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f556a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f556c0cb150 0x7f556c151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54876/0 (socket says 172.21.15.26:54876) 2023-12-19T16:43:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f556a59c640 1 -- 172.21.15.26:0/2950726619 learned_addr learned my addr 172.21.15.26:0/2950726619 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5563fff640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f556c1599e0 0x7f556c0c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f556ad9d640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f556c15d530 0x7f556c155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:37.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f556a59c640 1 -- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f556c15d530 msgr2=0x7f556c155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f556a59c640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f556c15d530 0x7f556c155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f556a59c640 1 -- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f556c1599e0 msgr2=0x7f556c0c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f556a59c640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f556c1599e0 0x7f556c0c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f556a59c640 1 -- 172.21.15.26:0/2950726619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5554009420 con 0x7f556c0cb150 2023-12-19T16:43:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f556a59c640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f556c0cb150 0x7f556c151b30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f555c00e930 tx=0x7f555c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5562ffd640 1 -- 172.21.15.26:0/2950726619 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c009690 con 0x7f556c0cb150 2023-12-19T16:43:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5562ffd640 1 -- 172.21.15.26:0/2950726619 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555c00ca00 con 0x7f556c0cb150 2023-12-19T16:43:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5570f0a640 1 -- 172.21.15.26:0/2950726619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c152070 con 0x7f556c0cb150 2023-12-19T16:43:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5562ffd640 1 -- 172.21.15.26:0/2950726619 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c00cd70 con 0x7f556c0cb150 2023-12-19T16:43:37.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.971+0000 7f5570f0a640 1 -- 172.21.15.26:0/2950726619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c158390 con 0x7f556c0cb150 2023-12-19T16:43:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.975+0000 7f5570f0a640 1 -- 172.21.15.26:0/2950726619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556c074270 con 0x7f556c0cb150 2023-12-19T16:43:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.975+0000 7f5562ffd640 1 -- 172.21.15.26:0/2950726619 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f555c013470 con 0x7f556c0cb150 2023-12-19T16:43:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.975+0000 7f5562ffd640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f554c065a90 0x7f554c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:37.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.975+0000 7f5562ffd640 1 -- 172.21.15.26:0/2950726619 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f555c0833e0 con 0x7f556c0cb150 2023-12-19T16:43:37.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.975+0000 7f5563fff640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f554c065a90 0x7f554c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:37.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.975+0000 7f5563fff640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f554c065a90 0x7f554c067f50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55540af000 tx=0x7f5554004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:37.979+0000 7f5562ffd640 1 -- 172.21.15.26:0/2950726619 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f555c050e50 con 0x7f556c0cb150 2023-12-19T16:43:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.179+0000 7f5570f0a640 1 -- 172.21.15.26:0/2950726619 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f556c0b0820 con 0x7f556c0cb150 2023-12-19T16:43:38.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.379+0000 7f5562ffd640 1 -- 172.21.15.26:0/2950726619 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 6) v1 ==== 85245+0+0 (secure 0 0 0) 0x7f555c0830c0 con 0x7f556c0cb150 2023-12-19T16:43:38.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.379+0000 7f5562ffd640 1 -- 172.21.15.26:0/2950726619 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7f555c06e4d0 con 0x7f556c0cb150 2023-12-19T16:43:38.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.383+0000 7f5570f0a640 1 -- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f554c065a90 msgr2=0x7f554c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:38.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.383+0000 7f5570f0a640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f554c065a90 0x7f554c067f50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f55540af000 tx=0x7f5554004740 comp rx=0 tx=0).stop 2023-12-19T16:43:38.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.383+0000 7f5570f0a640 1 -- 172.21.15.26:0/2950726619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f556c0cb150 msgr2=0x7f556c151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:38.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.383+0000 7f5570f0a640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f556c0cb150 0x7f556c151b30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f555c00e930 tx=0x7f555c00edf0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.383+0000 7f5570f0a640 1 -- 172.21.15.26:0/2950726619 shutdown_connections 2023-12-19T16:43:38.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.383+0000 7f5570f0a640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f554c065a90 0x7f554c067f50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.383+0000 7f5570f0a640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f556c15d530 0x7f556c155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.383+0000 7f5570f0a640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f556c1599e0 0x7f556c0c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.383+0000 7f5570f0a640 1 --2- 172.21.15.26:0/2950726619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f556c0cb150 0x7f556c151b30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.383+0000 7f5570f0a640 1 -- 172.21.15.26:0/2950726619 >> 172.21.15.26:0/2950726619 conn(0x7f556c066160 msgr2=0x7f556c0af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:38.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.383+0000 7f5570f0a640 1 -- 172.21.15.26:0/2950726619 shutdown_connections 2023-12-19T16:43:38.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.383+0000 7f5570f0a640 1 -- 172.21.15.26:0/2950726619 wait complete. 2023-12-19T16:43:38.408 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-19T16:43:38.559 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:38.557+0000 7f70d189b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:43:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.563+0000 7f2d82fbc640 1 Processor -- start 2023-12-19T16:43:38.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.563+0000 7f2d82fbc640 1 -- start start 2023-12-19T16:43:38.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d82fbc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2d7c15cdb0 0x7f2d7c06d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d82fbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d7c0658e0 0x7f2d7c065d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d82fbc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7c0cd940 con 0x7f2d7c0ca060 2023-12-19T16:43:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d82fbc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7c0662d0 con 0x7f2d7c15cdb0 2023-12-19T16:43:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d82fbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c15d190 con 0x7f2d7c0658e0 2023-12-19T16:43:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d80d58640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2d7c15cdb0 0x7f2d7c06d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d80d58640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2d7c15cdb0 0x7f2d7c06d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54900/0 (socket says 172.21.15.26:54900) 2023-12-19T16:43:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d80d58640 1 -- 172.21.15.26:0/1242239894 learned_addr learned my addr 172.21.15.26:0/1242239894 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:38.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d81559640 1 --2- 172.21.15.26:0/1242239894 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d7c0658e0 0x7f2d7c065d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:38.572 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f25744f0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:43:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d80d58640 1 -- 172.21.15.26:0/1242239894 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d7c0658e0 msgr2=0x7f2d7c065d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d80d58640 1 --2- 172.21.15.26:0/1242239894 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d7c0658e0 0x7f2d7c065d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d80d58640 1 -- 172.21.15.26:0/1242239894 >> v1:172.21.15.160:6789/0 conn(0x7f2d7c0ca060 legacy=0x7f2d7c161280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d7b7fe640 1 -- 172.21.15.26:0/1242239894 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624233501 0 0) 0x7f2d7c0cd940 con 0x7f2d7c0ca060 2023-12-19T16:43:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.567+0000 7f2d80d58640 1 -- 172.21.15.26:0/1242239894 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c0cdf50 con 0x7f2d7c15cdb0 2023-12-19T16:43:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d80d58640 1 --2- 172.21.15.26:0/1242239894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7c15cdb0 0x7f2d7c06d610 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2d700098a0 tx=0x7f2d7001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4f59ecf5f81866 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d7b7fe640 1 -- 172.21.15.26:0/1242239894 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d700ac630 con 0x7f2d7c15cdb0 2023-12-19T16:43:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d7b7fe640 1 -- 172.21.15.26:0/1242239894 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d700ac790 con 0x7f2d7c15cdb0 2023-12-19T16:43:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d7b7fe640 1 -- 172.21.15.26:0/1242239894 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d700acb20 con 0x7f2d7c15cdb0 2023-12-19T16:43:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d82fbc640 1 -- 172.21.15.26:0/1242239894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7c15cdb0 msgr2=0x7f2d7c06d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d82fbc640 1 --2- 172.21.15.26:0/1242239894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7c15cdb0 0x7f2d7c06d610 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2d700098a0 tx=0x7f2d7001c3d0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d82fbc640 1 -- 172.21.15.26:0/1242239894 shutdown_connections 2023-12-19T16:43:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d82fbc640 1 --2- 172.21.15.26:0/1242239894 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d7c0658e0 0x7f2d7c065d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d82fbc640 1 --2- 172.21.15.26:0/1242239894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7c15cdb0 0x7f2d7c06d610 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d82fbc640 1 -- 172.21.15.26:0/1242239894 >> 172.21.15.26:0/1242239894 conn(0x7f2d7c069d20 msgr2=0x7f2d7c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:38.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d82fbc640 1 -- 172.21.15.26:0/1242239894 shutdown_connections 2023-12-19T16:43:38.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d82fbc640 1 -- 172.21.15.26:0/1242239894 wait complete. 2023-12-19T16:43:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d82fbc640 1 Processor -- start 2023-12-19T16:43:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.571+0000 7f2d82fbc640 1 -- start start 2023-12-19T16:43:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d82fbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7c0658e0 0x7f2d7c14d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d82fbc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d7c0ca060 0x7f2d7c145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d82fbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d7c15cdb0 0x7f2d7c150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d82fbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c1509f0 con 0x7f2d7c0658e0 2023-12-19T16:43:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d82fbc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c150100 con 0x7f2d7c0ca060 2023-12-19T16:43:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d82fbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c1503a0 con 0x7f2d7c15cdb0 2023-12-19T16:43:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d80d58640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7c0658e0 0x7f2d7c14d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d80d58640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7c0658e0 0x7f2d7c14d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54912/0 (socket says 172.21.15.26:54912) 2023-12-19T16:43:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d80d58640 1 -- 172.21.15.26:0/3803020494 learned_addr learned my addr 172.21.15.26:0/3803020494 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d7bfff640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d7c0ca060 0x7f2d7c145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d81559640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d7c15cdb0 0x7f2d7c150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d80d58640 1 -- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d7c15cdb0 msgr2=0x7f2d7c150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d80d58640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d7c15cdb0 0x7f2d7c150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d80d58640 1 -- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d7c0ca060 msgr2=0x7f2d7c145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d80d58640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d7c0ca060 0x7f2d7c145440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d80d58640 1 -- 172.21.15.26:0/3803020494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d700094d0 con 0x7f2d7c0658e0 2023-12-19T16:43:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d81559640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d7c15cdb0 0x7f2d7c150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:38.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d80d58640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7c0658e0 0x7f2d7c14d300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2d700097f0 tx=0x7f2d7001ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d797fa640 1 -- 172.21.15.26:0/3803020494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70004220 con 0x7f2d7c0658e0 2023-12-19T16:43:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d797fa640 1 -- 172.21.15.26:0/3803020494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d70004380 con 0x7f2d7c0658e0 2023-12-19T16:43:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d797fa640 1 -- 172.21.15.26:0/3803020494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70003820 con 0x7f2d7c0658e0 2023-12-19T16:43:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d82fbc640 1 -- 172.21.15.26:0/3803020494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7c150680 con 0x7f2d7c0658e0 2023-12-19T16:43:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d82fbc640 1 -- 172.21.15.26:0/3803020494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c14d840 con 0x7f2d7c0658e0 2023-12-19T16:43:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d82fbc640 1 -- 172.21.15.26:0/3803020494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7c074270 con 0x7f2d7c0658e0 2023-12-19T16:43:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.575+0000 7f2d797fa640 1 -- 172.21.15.26:0/3803020494 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 6) v1 ==== 85245+0+0 (secure 0 0 0) 0x7f2d700b3050 con 0x7f2d7c0658e0 2023-12-19T16:43:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.579+0000 7f2d797fa640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2d60065ab0 0x7f2d60067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.579+0000 7f2d797fa640 1 -- 172.21.15.26:0/3803020494 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d7011d4b0 con 0x7f2d7c0658e0 2023-12-19T16:43:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.579+0000 7f2d7bfff640 1 -- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2d60065ab0 msgr2=0x7f2d60067f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:43:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.579+0000 7f2d7bfff640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2d60065ab0 0x7f2d60067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:43:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.583+0000 7f2d797fa640 1 -- 172.21.15.26:0/3803020494 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2d700eaef0 con 0x7f2d7c0658e0 2023-12-19T16:43:38.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.743+0000 7f2d82fbc640 1 -- 172.21.15.26:0/3803020494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f2d7c070c80 con 0x7f2d7c0658e0 2023-12-19T16:43:38.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.747+0000 7f2d797fa640 1 -- 172.21.15.26:0/3803020494 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2d700b10d0 con 0x7f2d7c0658e0 2023-12-19T16:43:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.751+0000 7f2d82fbc640 1 -- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2d60065ab0 msgr2=0x7f2d60067f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:43:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.751+0000 7f2d82fbc640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2d60065ab0 0x7f2d60067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.751+0000 7f2d82fbc640 1 -- 172.21.15.26:0/3803020494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7c0658e0 msgr2=0x7f2d7c14d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.751+0000 7f2d82fbc640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7c0658e0 0x7f2d7c14d300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2d700097f0 tx=0x7f2d7001ac10 comp rx=0 tx=0).stop 2023-12-19T16:43:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.751+0000 7f2d82fbc640 1 -- 172.21.15.26:0/3803020494 shutdown_connections 2023-12-19T16:43:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.751+0000 7f2d82fbc640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2d60065ab0 0x7f2d60067f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.751+0000 7f2d82fbc640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d7c15cdb0 0x7f2d7c150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.751+0000 7f2d82fbc640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d7c0ca060 0x7f2d7c145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.751+0000 7f2d82fbc640 1 --2- 172.21.15.26:0/3803020494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7c0658e0 0x7f2d7c14d300 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.751+0000 7f2d82fbc640 1 -- 172.21.15.26:0/3803020494 >> 172.21.15.26:0/3803020494 conn(0x7f2d7c069d20 msgr2=0x7f2d7c0c5360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.751+0000 7f2d82fbc640 1 -- 172.21.15.26:0/3803020494 shutdown_connections 2023-12-19T16:43:38.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.751+0000 7f2d82fbc640 1 -- 172.21.15.26:0/3803020494 wait complete. 2023-12-19T16:43:38.776 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-19T16:43:38.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.919+0000 7f51439a6640 1 Processor -- start 2023-12-19T16:43:38.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.919+0000 7f51439a6640 1 -- start start 2023-12-19T16:43:38.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.919+0000 7f51439a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f513c15d0d0 0x7f513c0b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:38.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.919+0000 7f51439a6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c15e200 con 0x7f513c15ccf0 2023-12-19T16:43:38.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.919+0000 7f51439a6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513c0d0aa0 con 0x7f513c0b4600 2023-12-19T16:43:38.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.919+0000 7f51439a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c15d680 con 0x7f513c15d0d0 2023-12-19T16:43:38.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.919+0000 7f5141742640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f513c15ccf0 0x7f513c0d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35188/0 (socket says 172.21.15.26:35188) 2023-12-19T16:43:38.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.919+0000 7f5141742640 1 -- 172.21.15.26:0/1725293238 learned_addr learned my addr 172.21.15.26:0/1725293238 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.919+0000 7f5140f41640 1 --2- 172.21.15.26:0/1725293238 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f513c15d0d0 0x7f513c0b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.923+0000 7f5133fff640 1 -- 172.21.15.26:0/1725293238 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12323718 0 0) 0x7f513c0d0aa0 con 0x7f513c0b4600 2023-12-19T16:43:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.923+0000 7f5133fff640 1 -- 172.21.15.26:0/1725293238 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5128003530 con 0x7f513c0b4600 2023-12-19T16:43:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.923+0000 7f5133fff640 1 -- 172.21.15.26:0/1725293238 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848905999 0 0) 0x7f513c15e200 con 0x7f513c15ccf0 2023-12-19T16:43:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.923+0000 7f5133fff640 1 -- 172.21.15.26:0/1725293238 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f513c0d0aa0 con 0x7f513c15ccf0 2023-12-19T16:43:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.923+0000 7f5140f41640 1 -- 172.21.15.26:0/1725293238 >> v1:172.21.15.160:6789/0 conn(0x7f513c0b4600 legacy=0x7f513c160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.923+0000 7f5140f41640 1 -- 172.21.15.26:0/1725293238 >> v1:172.21.15.26:6789/0 conn(0x7f513c15ccf0 legacy=0x7f513c0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.923+0000 7f5140f41640 1 -- 172.21.15.26:0/1725293238 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c15d960 con 0x7f513c15d0d0 2023-12-19T16:43:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.923+0000 7f5140f41640 1 --2- 172.21.15.26:0/1725293238 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f513c15d0d0 0x7f513c0b8400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f512c0097f0 tx=0x7f512c01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=629fa03e3125b954 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.923+0000 7f5133fff640 1 -- 172.21.15.26:0/1725293238 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0aa5f0 con 0x7f513c15d0d0 2023-12-19T16:43:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.923+0000 7f5133fff640 1 -- 172.21.15.26:0/1725293238 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512c0aa750 con 0x7f513c15d0d0 2023-12-19T16:43:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.923+0000 7f5133fff640 1 -- 172.21.15.26:0/1725293238 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c0aaae0 con 0x7f513c15d0d0 2023-12-19T16:43:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 -- 172.21.15.26:0/1725293238 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f513c15d0d0 msgr2=0x7f513c0b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 --2- 172.21.15.26:0/1725293238 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f513c15d0d0 0x7f513c0b8400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f512c0097f0 tx=0x7f512c01c490 comp rx=0 tx=0).stop 2023-12-19T16:43:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 -- 172.21.15.26:0/1725293238 shutdown_connections 2023-12-19T16:43:38.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 --2- 172.21.15.26:0/1725293238 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f513c15d0d0 0x7f513c0b8400 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 -- 172.21.15.26:0/1725293238 >> 172.21.15.26:0/1725293238 conn(0x7f513c065ee0 msgr2=0x7f513c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:38.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 -- 172.21.15.26:0/1725293238 shutdown_connections 2023-12-19T16:43:38.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 -- 172.21.15.26:0/1725293238 wait complete. 2023-12-19T16:43:38.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 Processor -- start 2023-12-19T16:43:38.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 -- start start 2023-12-19T16:43:38.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f513c0b4600 0x7f513c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:38.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f513c15ccf0 0x7f513c1589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c15d0d0 0x7f513c0b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0c7790 con 0x7f513c15d0d0 2023-12-19T16:43:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0ca560 con 0x7f513c15ccf0 2023-12-19T16:43:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f51439a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513c0ca840 con 0x7f513c0b4600 2023-12-19T16:43:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f5141f43640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c15d0d0 0x7f513c0b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f5141f43640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c15d0d0 0x7f513c0b6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54926/0 (socket says 172.21.15.26:54926) 2023-12-19T16:43:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f5141f43640 1 -- 172.21.15.26:0/3057692286 learned_addr learned my addr 172.21.15.26:0/3057692286 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f5140f41640 1 --2- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f513c15ccf0 0x7f513c1589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.927+0000 7f5141742640 1 --2- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f513c0b4600 0x7f513c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5141f43640 1 -- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f513c0b4600 msgr2=0x7f513c0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5141f43640 1 --2- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f513c0b4600 0x7f513c0c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5141f43640 1 -- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f513c15ccf0 msgr2=0x7f513c1589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5141f43640 1 --2- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f513c15ccf0 0x7f513c1589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5141f43640 1 -- 172.21.15.26:0/3057692286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c009420 con 0x7f513c15d0d0 2023-12-19T16:43:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5141f43640 1 --2- 172.21.15.26:0/3057692286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c15d0d0 0x7f513c0b6d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f513400c700 tx=0x7f513400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5131ffb640 1 -- 172.21.15.26:0/3057692286 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134003b80 con 0x7f513c15d0d0 2023-12-19T16:43:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5131ffb640 1 -- 172.21.15.26:0/3057692286 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5134003ce0 con 0x7f513c15d0d0 2023-12-19T16:43:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f51439a6640 1 -- 172.21.15.26:0/3057692286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513c0cab20 con 0x7f513c15d0d0 2023-12-19T16:43:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5131ffb640 1 -- 172.21.15.26:0/3057692286 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513400aa90 con 0x7f513c15d0d0 2023-12-19T16:43:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f51439a6640 1 -- 172.21.15.26:0/3057692286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c158f10 con 0x7f513c15d0d0 2023-12-19T16:43:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f51439a6640 1 -- 172.21.15.26:0/3057692286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513c0c74f0 con 0x7f513c15d0d0 2023-12-19T16:43:38.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5131ffb640 1 -- 172.21.15.26:0/3057692286 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 6) v1 ==== 85245+0+0 (secure 0 0 0) 0x7f513400abf0 con 0x7f513c15d0d0 2023-12-19T16:43:38.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5131ffb640 1 --2- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5120065ab0 0x7f5120067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:38.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5131ffb640 1 -- 172.21.15.26:0/3057692286 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5134082f00 con 0x7f513c15d0d0 2023-12-19T16:43:38.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5141742640 1 -- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5120065ab0 msgr2=0x7f5120067f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:43:38.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.931+0000 7f5141742640 1 --2- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5120065ab0 0x7f5120067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:43:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:38.935+0000 7f5131ffb640 1 -- 172.21.15.26:0/3057692286 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f513c0c74f0 con 0x7f513c15d0d0 2023-12-19T16:43:38.979 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:38.973+0000 7f70d189b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:43:39.022 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:39.015+0000 7f25744f0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:43:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.099+0000 7f51439a6640 1 -- 172.21.15.26:0/3057692286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f513c0c78d0 con 0x7f513c15d0d0 2023-12-19T16:43:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.107+0000 7f5131ffb640 1 -- 172.21.15.26:0/3057692286 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f513c0c78d0 con 0x7f513c15d0d0 2023-12-19T16:43:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.107+0000 7f51439a6640 1 -- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5120065ab0 msgr2=0x7f5120067f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:43:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.107+0000 7f51439a6640 1 --2- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5120065ab0 0x7f5120067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.107+0000 7f51439a6640 1 -- 172.21.15.26:0/3057692286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c15d0d0 msgr2=0x7f513c0b6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.107+0000 7f51439a6640 1 --2- 172.21.15.26:0/3057692286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c15d0d0 0x7f513c0b6d90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f513400c700 tx=0x7f513400cbc0 comp rx=0 tx=0).stop 2023-12-19T16:43:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.107+0000 7f51439a6640 1 -- 172.21.15.26:0/3057692286 shutdown_connections 2023-12-19T16:43:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.111+0000 7f51439a6640 1 --2- 172.21.15.26:0/3057692286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f513c15d0d0 0x7f513c0b6d90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.111+0000 7f51439a6640 1 --2- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f513c15ccf0 0x7f513c1589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.111+0000 7f51439a6640 1 --2- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5120065ab0 0x7f5120067f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.111+0000 7f51439a6640 1 --2- 172.21.15.26:0/3057692286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f513c0b4600 0x7f513c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.111+0000 7f51439a6640 1 -- 172.21.15.26:0/3057692286 >> 172.21.15.26:0/3057692286 conn(0x7f513c065ee0 msgr2=0x7f513c15e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.111+0000 7f51439a6640 1 -- 172.21.15.26:0/3057692286 shutdown_connections 2023-12-19T16:43:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.111+0000 7f51439a6640 1 -- 172.21.15.26:0/3057692286 wait complete. 2023-12-19T16:43:39.129 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:39.125+0000 7f70d189b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:43:39.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:39.135 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-19T16:43:39.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:39.150 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-12-19T16:43:39.184 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:39.179+0000 7f25744f0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:43:39.196 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir0 1M 2023-12-19T16:43:39.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.379+0000 7fb3dd620640 1 Processor -- start 2023-12-19T16:43:39.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3dd620640 1 -- start start 2023-12-19T16:43:39.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3dd620640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3d815a9c0 0x7fb3d80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:39.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3dd620640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3d806ac80 0x7fb3d80b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:39.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3dd620640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d8070d90 con 0x7fb3d815ada0 2023-12-19T16:43:39.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3dd620640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d80cc270 con 0x7fb3d815a9c0 2023-12-19T16:43:39.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3dd620640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d8071070 con 0x7fb3d806ac80 2023-12-19T16:43:39.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3cffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb3d815ada0 0x7fb3d80b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35192/0 (socket says 172.21.15.26:35192) 2023-12-19T16:43:39.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3cffff640 1 -- 172.21.15.26:0/4082238767 learned_addr learned my addr 172.21.15.26:0/4082238767 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:39.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3d6ffd640 1 --2- 172.21.15.26:0/4082238767 >> v2:172.21.15.26:3300/0 conn(0x7fb3d815a9c0 0x7fb3d80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:39.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3d77fe640 1 --2- 172.21.15.26:0/4082238767 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3d806ac80 0x7fb3d80b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:39.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3d6ffd640 1 -- 172.21.15.26:0/4082238767 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3d806ac80 msgr2=0x7fb3d80b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:39.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3d6ffd640 1 --2- 172.21.15.26:0/4082238767 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3d806ac80 0x7fb3d80b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:39.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3d6ffd640 1 -- 172.21.15.26:0/4082238767 >> v1:172.21.15.26:6789/0 conn(0x7fb3d815ada0 legacy=0x7fb3d80b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:39.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.383+0000 7fb3d6ffd640 1 -- 172.21.15.26:0/4082238767 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d8071350 con 0x7fb3d815a9c0 2023-12-19T16:43:39.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3d6ffd640 1 --2- 172.21.15.26:0/4082238767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d815a9c0 0x7fb3d80706d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80097c0 tx=0x7fb3c801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53866a18ed26d280 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:39.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3d67fc640 1 -- 172.21.15.26:0/4082238767 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c80ab600 con 0x7fb3d815a9c0 2023-12-19T16:43:39.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3d67fc640 1 -- 172.21.15.26:0/4082238767 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c80a7050 con 0x7fb3d815a9c0 2023-12-19T16:43:39.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3d67fc640 1 -- 172.21.15.26:0/4082238767 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c80ab980 con 0x7fb3d815a9c0 2023-12-19T16:43:39.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3dd620640 1 -- 172.21.15.26:0/4082238767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d815a9c0 msgr2=0x7fb3d80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:39.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3dd620640 1 --2- 172.21.15.26:0/4082238767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d815a9c0 0x7fb3d80706d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80097c0 tx=0x7fb3c801c3c0 comp rx=0 tx=0).stop 2023-12-19T16:43:39.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3dd620640 1 -- 172.21.15.26:0/4082238767 shutdown_connections 2023-12-19T16:43:39.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3dd620640 1 --2- 172.21.15.26:0/4082238767 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3d806ac80 0x7fb3d80b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:39.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3dd620640 1 --2- 172.21.15.26:0/4082238767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d815a9c0 0x7fb3d80706d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:39.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3dd620640 1 -- 172.21.15.26:0/4082238767 >> 172.21.15.26:0/4082238767 conn(0x7fb3d80ce730 msgr2=0x7fb3d80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:39.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3dd620640 1 -- 172.21.15.26:0/4082238767 shutdown_connections 2023-12-19T16:43:39.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3dd620640 1 -- 172.21.15.26:0/4082238767 wait complete. 2023-12-19T16:43:39.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3dd620640 1 Processor -- start 2023-12-19T16:43:39.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.387+0000 7fb3dd620640 1 -- start start 2023-12-19T16:43:39.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3dd620640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3d806ac80 0x7fb3d8151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3dd620640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3d815a9c0 0x7fb3d80c2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3dd620640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d815ada0 0x7fb3d80c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3dd620640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d8151ff0 con 0x7fb3d815ada0 2023-12-19T16:43:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3dd620640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80c84a0 con 0x7fb3d815a9c0 2023-12-19T16:43:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3dd620640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d80c7ee0 con 0x7fb3d806ac80 2023-12-19T16:43:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d815ada0 0x7fb3d80c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3d77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d815ada0 0x7fb3d80c2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54944/0 (socket says 172.21.15.26:54944) 2023-12-19T16:43:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3d77fe640 1 -- 172.21.15.26:0/1179557390 learned_addr learned my addr 172.21.15.26:0/1179557390 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3d6ffd640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3d806ac80 0x7fb3d8151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3cffff640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3d815a9c0 0x7fb3d80c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3d77fe640 1 -- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3d806ac80 msgr2=0x7fb3d8151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:39.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3d77fe640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3d806ac80 0x7fb3d8151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:39.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3d77fe640 1 -- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3d815a9c0 msgr2=0x7fb3d80c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:39.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3d77fe640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3d815a9c0 0x7fb3d80c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:39.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3d77fe640 1 -- 172.21.15.26:0/1179557390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c8009420 con 0x7fb3d815ada0 2023-12-19T16:43:39.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3d77fe640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d815ada0 0x7fb3d80c2ca0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000efc0 tx=0x7fb3d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:39.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3cf7fe640 1 -- 172.21.15.26:0/1179557390 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0009ba0 con 0x7fb3d815ada0 2023-12-19T16:43:39.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3cf7fe640 1 -- 172.21.15.26:0/1179557390 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d000f040 con 0x7fb3d815ada0 2023-12-19T16:43:39.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3dd620640 1 -- 172.21.15.26:0/1179557390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d80c81c0 con 0x7fb3d815ada0 2023-12-19T16:43:39.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3cf7fe640 1 -- 172.21.15.26:0/1179557390 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d00146c0 con 0x7fb3d815ada0 2023-12-19T16:43:39.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3dd620640 1 -- 172.21.15.26:0/1179557390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d815b600 con 0x7fb3d815ada0 2023-12-19T16:43:39.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.391+0000 7fb3dd620640 1 -- 172.21.15.26:0/1179557390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d8074270 con 0x7fb3d815ada0 2023-12-19T16:43:39.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.395+0000 7fb3cf7fe640 1 -- 172.21.15.26:0/1179557390 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 6) v1 ==== 85245+0+0 (secure 0 0 0) 0x7fb3d0014820 con 0x7fb3d815ada0 2023-12-19T16:43:39.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.395+0000 7fb3cf7fe640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 0x7fb3b8067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:39.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.395+0000 7fb3cf7fe640 1 -- 172.21.15.26:0/1179557390 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3d0083f10 con 0x7fb3d815ada0 2023-12-19T16:43:39.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.395+0000 7fb3d6ffd640 1 -- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 msgr2=0x7fb3b8067f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:43:39.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.395+0000 7fb3d6ffd640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 0x7fb3b8067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:43:39.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.399+0000 7fb3cf7fe640 1 -- 172.21.15.26:0/1179557390 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb3d00518c0 con 0x7fb3d815ada0 2023-12-19T16:43:39.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.575+0000 7fb3dd620640 1 -- 172.21.15.26:0/1179557390 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3d8167ac0 con 0x7fb3b8065ab0 2023-12-19T16:43:39.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.599+0000 7fb3d6ffd640 1 -- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 msgr2=0x7fb3b8067f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:43:39.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:39.599+0000 7fb3d6ffd640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 0x7fb3b8067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T16:43:39.613 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:39.609+0000 7f70d189b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:43:39.671 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:39.665+0000 7f70d189b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:43:39.701 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:39.695+0000 7f25744f0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:43:39.726 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:39.721+0000 7f70d189b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:43:39.763 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:39.759+0000 7f25744f0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:43:39.812 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:39.809+0000 7f70d189b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:43:39.822 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:39.815+0000 7f25744f0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:43:39.870 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:39.865+0000 7f70d189b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:43:39.913 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:39.907+0000 7f25744f0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:43:39.954 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:39.949+0000 7f70d189b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:43:39.972 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:39.967+0000 7f25744f0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:43:40.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:40.003+0000 7fb3d6ffd640 1 -- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 msgr2=0x7fb3b8067f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:43:40.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:40.003+0000 7fb3d6ffd640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 0x7fb3b8067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-19T16:43:40.012 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:40.009+0000 7f70d189b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:43:40.062 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:40.055+0000 7f25744f0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:43:40.121 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:40.115+0000 7f25744f0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:43:40.155 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:40.153+0000 7f70d189b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:43:40.213 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:40.209+0000 7f70d189b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:43:40.272 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:40.267+0000 7f25744f0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:43:40.334 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:40.327+0000 7f25744f0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:43:40.454 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:43:40.454 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:43:40.454 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:43:40.462 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:40.457+0000 7f70d189b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:43:40.584 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:43:40.585 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:43:40.585 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:43:40.593 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:40.587+0000 7f25744f0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:43:40.597 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:40.593+0000 7f70d189b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:43:40.721 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:40.717+0000 7f70d189b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:43:40.736 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:40.731+0000 7f25744f0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:43:40.791 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:40.785+0000 7f70d189b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:43:40.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:40.807+0000 7fb3d6ffd640 1 -- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 msgr2=0x7fb3b8067f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:43:40.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:40.807+0000 7fb3d6ffd640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 0x7fb3b8067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-19T16:43:40.867 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:40.863+0000 7f25744f0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:43:40.927 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:40.921+0000 7f70d189b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:43:40.942 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:40.935+0000 7f25744f0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:43:41.085 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:41.079+0000 7f25744f0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:43:41.462 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:41.457+0000 7f70d189b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:43:41.549 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:41.545+0000 7f70d189b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:43:41.657 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:41.651+0000 7f25744f0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:43:41.697 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:41.693+0000 7f70d189b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:43:41.745 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:41.739+0000 7f25744f0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:43:41.781 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:41.777+0000 7f70d189b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:43:41.905 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:41.899+0000 7f25744f0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:43:41.994 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:41.987+0000 7f25744f0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:43:42.100 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:42.097+0000 7f70d189b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:43:42.161 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:42.157+0000 7f70d189b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:43:42.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:42.167+0000 7fb3cf7fe640 1 -- 172.21.15.26:0/1179557390 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 7) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fb3d0051c90 con 0x7fb3d815ada0 2023-12-19T16:43:42.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:42.167+0000 7fb3cf7fe640 1 -- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 msgr2=0x7fb3b8067f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:43:42.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:42.167+0000 7fb3cf7fe640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 0x7fb3b8067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:42.296 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:42.293+0000 7f7054716640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:43:42.296 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:42.293+0000 7f7054716640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:43:42.296 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:42.293+0000 7f7054716640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:43:42.296 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:42.293+0000 7f7054716640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:43:42.296 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:42.293+0000 7f7054716640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:43:42.296 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:42.293+0000 7f7056f1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:43:42.296 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:42.293+0000 7f7056f1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:43:42.296 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:42.293+0000 7f7056f1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:43:42.296 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:42.293+0000 7f7056f1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:43:42.296 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:43:42.293+0000 7f7056f1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:43:42.347 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:42.343+0000 7f25744f0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:43:42.411 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:43:42.407+0000 7f25744f0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:43:43.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.175+0000 7fb3cf7fe640 1 -- 172.21.15.26:0/1179557390 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 8) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fb3d003cb90 con 0x7fb3d815ada0 2023-12-19T16:43:43.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.175+0000 7fb3cf7fe640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 0x7fb3b8067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.175+0000 7fb3cf7fe640 1 -- 172.21.15.26:0/1179557390 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3b801bfa0 con 0x7fb3b8065ab0 2023-12-19T16:43:43.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.175+0000 7fb3d6ffd640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 0x7fb3b8067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.179+0000 7fb3d6ffd640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 0x7fb3b8067f70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80ad000 tx=0x7fb3c80a6000 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:43.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.267+0000 7fb3cf7fe640 1 -- 172.21.15.26:0/1179557390 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb3b801bfa0 con 0x7fb3b8065ab0 2023-12-19T16:43:43.273 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path /snap_test_dir1/dir0 2023-12-19T16:43:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.271+0000 7fb3dd620640 1 -- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 msgr2=0x7fb3b8067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.271+0000 7fb3dd620640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 0x7fb3b8067f70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80ad000 tx=0x7fb3c80a6000 comp rx=0 tx=0).stop 2023-12-19T16:43:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.271+0000 7fb3dd620640 1 -- 172.21.15.26:0/1179557390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d815ada0 msgr2=0x7fb3d80c2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.271+0000 7fb3dd620640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d815ada0 0x7fb3d80c2ca0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000efc0 tx=0x7fb3d000c490 comp rx=0 tx=0).stop 2023-12-19T16:43:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.271+0000 7fb3dd620640 1 -- 172.21.15.26:0/1179557390 shutdown_connections 2023-12-19T16:43:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.271+0000 7fb3dd620640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3d815ada0 0x7fb3d80c2ca0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.271+0000 7fb3dd620640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3d815a9c0 0x7fb3d80c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.271+0000 7fb3dd620640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb3b8065ab0 0x7fb3b8067f70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.271+0000 7fb3dd620640 1 --2- 172.21.15.26:0/1179557390 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb3d806ac80 0x7fb3d8151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.271+0000 7fb3dd620640 1 -- 172.21.15.26:0/1179557390 >> 172.21.15.26:0/1179557390 conn(0x7fb3d80ce730 msgr2=0x7fb3d806b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.271+0000 7fb3dd620640 1 -- 172.21.15.26:0/1179557390 shutdown_connections 2023-12-19T16:43:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.271+0000 7fb3dd620640 1 -- 172.21.15.26:0/1179557390 wait complete. 2023-12-19T16:43:43.295 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-12-19T16:43:43.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:43.295 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-12-19T16:43:43.306 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir1 1M 2023-12-19T16:43:43.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.490+0000 7f30a60c3640 1 Processor -- start 2023-12-19T16:43:43.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.490+0000 7f30a60c3640 1 -- start start 2023-12-19T16:43:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f30a60c3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30a006bac0 0x7f30a0070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f30a60c3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f30a006f0a0 0x7f30a0070dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f30a60c3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a015af20 con 0x7f30a015ab40 2023-12-19T16:43:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f30a60c3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a00aea90 con 0x7f30a006bac0 2023-12-19T16:43:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f30a60c3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a00aed70 con 0x7f30a006f0a0 2023-12-19T16:43:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f30a50c1640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f30a015ab40 0x7f30a006b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35198/0 (socket says 172.21.15.26:35198) 2023-12-19T16:43:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f30a50c1640 1 -- 172.21.15.26:0/2359983269 learned_addr learned my addr 172.21.15.26:0/2359983269 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f30a48c0640 1 --2- 172.21.15.26:0/2359983269 >> v2:172.21.15.26:3300/0 conn(0x7f30a006bac0 0x7f30a0070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f3097fff640 1 -- 172.21.15.26:0/2359983269 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914783290 0 0) 0x7f30a015af20 con 0x7f30a015ab40 2023-12-19T16:43:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f30a58c2640 1 --2- 172.21.15.26:0/2359983269 >> v2:172.21.15.160:3300/0 conn(0x7f30a006f0a0 0x7f30a0070dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f3097fff640 1 -- 172.21.15.26:0/2359983269 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f308c003530 con 0x7f30a015ab40 2023-12-19T16:43:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f30a48c0640 1 -- 172.21.15.26:0/2359983269 >> v2:172.21.15.160:3300/0 conn(0x7f30a006f0a0 msgr2=0x7f30a0070dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f30a48c0640 1 --2- 172.21.15.26:0/2359983269 >> v2:172.21.15.160:3300/0 conn(0x7f30a006f0a0 0x7f30a0070dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f30a48c0640 1 -- 172.21.15.26:0/2359983269 >> v1:172.21.15.26:6789/0 conn(0x7f30a015ab40 legacy=0x7f30a006b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.494+0000 7f30a48c0640 1 -- 172.21.15.26:0/2359983269 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a00af050 con 0x7f30a006bac0 2023-12-19T16:43:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f30a48c0640 1 --2- 172.21.15.26:0/2359983269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a006bac0 0x7f30a0070880 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f30a00b1650 tx=0x7f309001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd1e860be9a4209 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f3097fff640 1 -- 172.21.15.26:0/2359983269 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090004230 con 0x7f30a006bac0 2023-12-19T16:43:43.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f3097fff640 1 -- 172.21.15.26:0/2359983269 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30900a7050 con 0x7f30a006bac0 2023-12-19T16:43:43.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f3097fff640 1 -- 172.21.15.26:0/2359983269 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090003620 con 0x7f30a006bac0 2023-12-19T16:43:43.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f30a60c3640 1 -- 172.21.15.26:0/2359983269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a006bac0 msgr2=0x7f30a0070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:43.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f30a60c3640 1 --2- 172.21.15.26:0/2359983269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a006bac0 0x7f30a0070880 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f30a00b1650 tx=0x7f309001c460 comp rx=0 tx=0).stop 2023-12-19T16:43:43.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f30a60c3640 1 -- 172.21.15.26:0/2359983269 shutdown_connections 2023-12-19T16:43:43.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f30a60c3640 1 --2- 172.21.15.26:0/2359983269 >> v2:172.21.15.160:3300/0 conn(0x7f30a006f0a0 0x7f30a0070dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f30a60c3640 1 --2- 172.21.15.26:0/2359983269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a006bac0 0x7f30a0070880 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f30a60c3640 1 -- 172.21.15.26:0/2359983269 >> 172.21.15.26:0/2359983269 conn(0x7f30a00658e0 msgr2=0x7f30a0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:43.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f30a60c3640 1 -- 172.21.15.26:0/2359983269 shutdown_connections 2023-12-19T16:43:43.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f30a60c3640 1 -- 172.21.15.26:0/2359983269 wait complete. 2023-12-19T16:43:43.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f30a60c3640 1 Processor -- start 2023-12-19T16:43:43.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.498+0000 7f30a60c3640 1 -- start start 2023-12-19T16:43:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a60c3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30a006bac0 0x7f30a014c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a60c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a006f0a0 0x7f30a015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a60c3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30a015ab40 0x7f30a01587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a60c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00b22d0 con 0x7f30a006f0a0 2023-12-19T16:43:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a60c3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00b16c0 con 0x7f30a015ab40 2023-12-19T16:43:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a60c3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a00b19a0 con 0x7f30a006bac0 2023-12-19T16:43:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a48c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a006f0a0 0x7f30a015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a48c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a006f0a0 0x7f30a015d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54964/0 (socket says 172.21.15.26:54964) 2023-12-19T16:43:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a48c0640 1 -- 172.21.15.26:0/125403424 learned_addr learned my addr 172.21.15.26:0/125403424 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a50c1640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30a006bac0 0x7f30a014c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a58c2640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30a015ab40 0x7f30a01587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a48c0640 1 -- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30a006bac0 msgr2=0x7f30a014c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a48c0640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30a006bac0 0x7f30a014c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a48c0640 1 -- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30a015ab40 msgr2=0x7f30a01587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a48c0640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30a015ab40 0x7f30a01587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a48c0640 1 -- 172.21.15.26:0/125403424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3090009420 con 0x7f30a006f0a0 2023-12-19T16:43:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a48c0640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a006f0a0 0x7f30a015d7c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3090019070 tx=0x7f3090019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f3095ffb640 1 -- 172.21.15.26:0/125403424 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090003820 con 0x7f30a006f0a0 2023-12-19T16:43:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f3095ffb640 1 -- 172.21.15.26:0/125403424 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3090003980 con 0x7f30a006f0a0 2023-12-19T16:43:43.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a60c3640 1 -- 172.21.15.26:0/125403424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a00b1c80 con 0x7f30a006f0a0 2023-12-19T16:43:43.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f3095ffb640 1 -- 172.21.15.26:0/125403424 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090003cf0 con 0x7f30a006f0a0 2023-12-19T16:43:43.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a60c3640 1 -- 172.21.15.26:0/125403424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a014cbf0 con 0x7f30a006f0a0 2023-12-19T16:43:43.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f30a60c3640 1 -- 172.21.15.26:0/125403424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a0061930 con 0x7f30a006f0a0 2023-12-19T16:43:43.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.502+0000 7f3095ffb640 1 -- 172.21.15.26:0/125403424 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f30900b2050 con 0x7f30a006f0a0 2023-12-19T16:43:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.506+0000 7f3095ffb640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3084065900 0x7f3084067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.506+0000 7f3095ffb640 1 -- 172.21.15.26:0/125403424 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f309011cfd0 con 0x7f30a006f0a0 2023-12-19T16:43:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.506+0000 7f30a50c1640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3084065900 0x7f3084067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.506+0000 7f30a50c1640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3084065900 0x7f3084067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f309c001210 tx=0x7f309c009040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:43.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.510+0000 7f3095ffb640 1 -- 172.21.15.26:0/125403424 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f30900eaa70 con 0x7f30a006f0a0 2023-12-19T16:43:43.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.686+0000 7f30a60c3640 1 -- 172.21.15.26:0/125403424 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f30a0167750 con 0x7f3084065900 2023-12-19T16:43:43.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.750+0000 7f3095ffb640 1 -- 172.21.15.26:0/125403424 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f309c001420 con 0x7f3084065900 2023-12-19T16:43:43.758 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path /snap_test_dir1/dir1 2023-12-19T16:43:43.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.754+0000 7f30a60c3640 1 -- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3084065900 msgr2=0x7f3084067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:43.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.754+0000 7f30a60c3640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3084065900 0x7f3084067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f309c001210 tx=0x7f309c009040 comp rx=0 tx=0).stop 2023-12-19T16:43:43.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.754+0000 7f30a60c3640 1 -- 172.21.15.26:0/125403424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a006f0a0 msgr2=0x7f30a015d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:43.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.754+0000 7f30a60c3640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a006f0a0 0x7f30a015d7c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3090019070 tx=0x7f3090019200 comp rx=0 tx=0).stop 2023-12-19T16:43:43.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.754+0000 7f30a60c3640 1 -- 172.21.15.26:0/125403424 shutdown_connections 2023-12-19T16:43:43.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.754+0000 7f30a60c3640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30a015ab40 0x7f30a01587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.754+0000 7f30a60c3640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30a006f0a0 0x7f30a015d7c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.754+0000 7f30a60c3640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3084065900 0x7f3084067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.754+0000 7f30a60c3640 1 --2- 172.21.15.26:0/125403424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30a006bac0 0x7f30a014c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.754+0000 7f30a60c3640 1 -- 172.21.15.26:0/125403424 >> 172.21.15.26:0/125403424 conn(0x7f30a00658e0 msgr2=0x7f30a0065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:43.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.754+0000 7f30a60c3640 1 -- 172.21.15.26:0/125403424 shutdown_connections 2023-12-19T16:43:43.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.754+0000 7f30a60c3640 1 -- 172.21.15.26:0/125403424 wait complete. 2023-12-19T16:43:43.779 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-12-19T16:43:43.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:43.779 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-12-19T16:43:43.791 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir2 1M 2023-12-19T16:43:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f311a6640 1 Processor -- start 2023-12-19T16:43:43.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f311a6640 1 -- start start 2023-12-19T16:43:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f311a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f2c159c00 0x7f4f2c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f311a6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2c06fb60 con 0x7f4f2c159820 2023-12-19T16:43:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f311a6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2c06fe40 con 0x7f4f2c06f6c0 2023-12-19T16:43:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f311a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c15e470 con 0x7f4f2c159c00 2023-12-19T16:43:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4f2c159820 0x7f4f2c0c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35212/0 (socket says 172.21.15.26:35212) 2023-12-19T16:43:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2ad9d640 1 -- 172.21.15.26:0/1285925812 learned_addr learned my addr 172.21.15.26:0/1285925812 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2b59e640 1 --2- 172.21.15.26:0/1285925812 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f2c159c00 0x7f4f2c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375407959 0 0) 0x7f4f2c06fb60 con 0x7f4f2c159820 2023-12-19T16:43:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f18005740 con 0x7f4f2c159820 2023-12-19T16:43:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 530139285 0 0) 0x7f4f18005740 con 0x7f4f2c159820 2023-12-19T16:43:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f2c159c00 msgr2=0x7f4f2c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 --2- 172.21.15.26:0/1285925812 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f2c159c00 0x7f4f2c15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 >> v1:172.21.15.160:6789/0 conn(0x7f4f2c06f6c0 legacy=0x7f4f2c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2c070120 con 0x7f4f2c159820 2023-12-19T16:43:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f4f1c001680 con 0x7f4f2c159820 2023-12-19T16:43:43.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 >> v1:172.21.15.26:6789/0 conn(0x7f4f2c159820 legacy=0x7f4f2c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 --2- 172.21.15.26:0/1285925812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f18005bc0 0x7f4f18005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 --2- 172.21.15.26:0/1285925812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f180069e0 0x7f4f18006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 --2- 172.21.15.26:0/1285925812 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f2c159c00 0x7f4f180073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c15e470 con 0x7f4f18005bc0 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1800dc50 con 0x7f4f180069e0 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1800de10 con 0x7f4f2c159c00 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2ad9d640 1 --2- 172.21.15.26:0/1285925812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f18005bc0 0x7f4f18005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f23fff640 1 --2- 172.21.15.26:0/1285925812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f180069e0 0x7f4f18006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2b59e640 1 --2- 172.21.15.26:0/1285925812 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f2c159c00 0x7f4f180073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2ad9d640 1 -- 172.21.15.26:0/1285925812 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f2c159c00 msgr2=0x7f4f180073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2ad9d640 1 --2- 172.21.15.26:0/1285925812 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f2c159c00 0x7f4f180073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2ad9d640 1 -- 172.21.15.26:0/1285925812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f180069e0 msgr2=0x7f4f18006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2ad9d640 1 --2- 172.21.15.26:0/1285925812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f180069e0 0x7f4f18006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f2ad9d640 1 -- 172.21.15.26:0/1285925812 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f18005740 con 0x7f4f18005bc0 2023-12-19T16:43:43.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.978+0000 7f4f23fff640 1 --2- 172.21.15.26:0/1285925812 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f180069e0 0x7f4f18006e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:43.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.982+0000 7f4f2ad9d640 1 --2- 172.21.15.26:0/1285925812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f18005bc0 0x7f4f18005fe0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00c210 tx=0x7f4f1c01cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11d3ba1a7faaaa4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:43.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.982+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c005ae0 con 0x7f4f18005bc0 2023-12-19T16:43:43.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.982+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1c005c40 con 0x7f4f18005bc0 2023-12-19T16:43:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.982+0000 7f4f2a59c640 1 -- 172.21.15.26:0/1285925812 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0a57c0 con 0x7f4f18005bc0 2023-12-19T16:43:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.982+0000 7f4f311a6640 1 -- 172.21.15.26:0/1285925812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f18005bc0 msgr2=0x7f4f18005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.982+0000 7f4f311a6640 1 --2- 172.21.15.26:0/1285925812 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f18005bc0 0x7f4f18005fe0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00c210 tx=0x7f4f1c01cbd0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.982+0000 7f4f2b59e640 1 -- 172.21.15.26:0/1285925812 reap_dead start 2023-12-19T16:43:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.982+0000 7f4f311a6640 1 -- 172.21.15.26:0/1285925812 shutdown_connections 2023-12-19T16:43:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.982+0000 7f4f311a6640 1 -- 172.21.15.26:0/1285925812 >> 172.21.15.26:0/1285925812 conn(0x7f4f2c0ce730 msgr2=0x7f4f2c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.982+0000 7f4f311a6640 1 -- 172.21.15.26:0/1285925812 shutdown_connections 2023-12-19T16:43:43.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.982+0000 7f4f311a6640 1 -- 172.21.15.26:0/1285925812 wait complete. 2023-12-19T16:43:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f311a6640 1 Processor -- start 2023-12-19T16:43:43.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f311a6640 1 -- start start 2023-12-19T16:43:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f311a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f18005bc0 0x7f4f2c151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f311a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2c070c90 0x7f4f2c0acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f311a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f2c153910 0x7f4f2c153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f311a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c154e70 con 0x7f4f2c070c90 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f311a6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c0b2970 con 0x7f4f2c153910 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f311a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2c155080 con 0x7f4f18005bc0 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f23fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2c070c90 0x7f4f2c0acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f23fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2c070c90 0x7f4f2c0acac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54978/0 (socket says 172.21.15.26:54978) 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f23fff640 1 -- 172.21.15.26:0/2637869615 learned_addr learned my addr 172.21.15.26:0/2637869615 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f2ad9d640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f18005bc0 0x7f4f2c151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f2b59e640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f2c153910 0x7f4f2c153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f23fff640 1 -- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f18005bc0 msgr2=0x7f4f2c151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f23fff640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f18005bc0 0x7f4f2c151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f23fff640 1 -- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f2c153910 msgr2=0x7f4f2c153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f23fff640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f2c153910 0x7f4f2c153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f23fff640 1 -- 172.21.15.26:0/2637869615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2c06b880 con 0x7f4f2c070c90 2023-12-19T16:43:43.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f2b59e640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f2c153910 0x7f4f2c153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f23fff640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2c070c90 0x7f4f2c0acac0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400e4e0 tx=0x7f4f1400e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f237fe640 1 -- 172.21.15.26:0/2637869615 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f140052e0 con 0x7f4f2c070c90 2023-12-19T16:43:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f237fe640 1 -- 172.21.15.26:0/2637869615 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f14005440 con 0x7f4f2c070c90 2023-12-19T16:43:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f237fe640 1 -- 172.21.15.26:0/2637869615 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14003840 con 0x7f4f2c070c90 2023-12-19T16:43:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.986+0000 7f4f311a6640 1 -- 172.21.15.26:0/2637869615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2c155360 con 0x7f4f2c070c90 2023-12-19T16:43:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.990+0000 7f4f311a6640 1 -- 172.21.15.26:0/2637869615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2c155920 con 0x7f4f2c070c90 2023-12-19T16:43:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.990+0000 7f4f311a6640 1 -- 172.21.15.26:0/2637869615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2c074270 con 0x7f4f2c070c90 2023-12-19T16:43:43.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.990+0000 7f4f237fe640 1 -- 172.21.15.26:0/2637869615 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f4f140039a0 con 0x7f4f2c070c90 2023-12-19T16:43:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.990+0000 7f4f237fe640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f0c065900 0x7f4f0c067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.990+0000 7f4f237fe640 1 -- 172.21.15.26:0/2637869615 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f4f14082c50 con 0x7f4f2c070c90 2023-12-19T16:43:43.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.990+0000 7f4f2ad9d640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f0c065900 0x7f4f0c067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:43.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.990+0000 7f4f2ad9d640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f0c065900 0x7f4f0c067dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0ab000 tx=0x7f4f1c0a7070 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:44.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:43.994+0000 7f4f237fe640 1 -- 172.21.15.26:0/2637869615 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4f14050780 con 0x7f4f2c070c90 2023-12-19T16:43:44.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.170+0000 7f4f311a6640 1 -- 172.21.15.26:0/2637869615 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f2c06b700 con 0x7f4f0c065900 2023-12-19T16:43:44.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.226+0000 7f4f237fe640 1 -- 172.21.15.26:0/2637869615 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 9) v1 ==== 85348+0+0 (secure 0 0 0) 0x7f4f14050b50 con 0x7f4f2c070c90 2023-12-19T16:43:44.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.234+0000 7f4f237fe640 1 -- 172.21.15.26:0/2637869615 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f4f1c0b5040 con 0x7f4f0c065900 2023-12-19T16:43:44.242 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path /snap_test_dir1/dir2 2023-12-19T16:43:44.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.238+0000 7f4f311a6640 1 -- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f0c065900 msgr2=0x7f4f0c067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:44.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.238+0000 7f4f311a6640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f0c065900 0x7f4f0c067dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0ab000 tx=0x7f4f1c0a7070 comp rx=0 tx=0).stop 2023-12-19T16:43:44.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.238+0000 7f4f311a6640 1 -- 172.21.15.26:0/2637869615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2c070c90 msgr2=0x7f4f2c0acac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:44.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.238+0000 7f4f311a6640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2c070c90 0x7f4f2c0acac0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400e4e0 tx=0x7f4f1400e9a0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.238+0000 7f4f311a6640 1 -- 172.21.15.26:0/2637869615 shutdown_connections 2023-12-19T16:43:44.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.238+0000 7f4f311a6640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f2c153910 0x7f4f2c153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.238+0000 7f4f311a6640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f2c070c90 0x7f4f2c0acac0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.238+0000 7f4f311a6640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f0c065900 0x7f4f0c067dc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.238+0000 7f4f311a6640 1 --2- 172.21.15.26:0/2637869615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f18005bc0 0x7f4f2c151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.238+0000 7f4f311a6640 1 -- 172.21.15.26:0/2637869615 >> 172.21.15.26:0/2637869615 conn(0x7f4f2c0ce730 msgr2=0x7f4f2c155d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:44.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.238+0000 7f4f311a6640 1 -- 172.21.15.26:0/2637869615 shutdown_connections 2023-12-19T16:43:44.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.238+0000 7f4f311a6640 1 -- 172.21.15.26:0/2637869615 wait complete. 2023-12-19T16:43:44.264 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-12-19T16:43:44.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:44.264 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-12-19T16:43:44.275 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir3 1M 2023-12-19T16:43:44.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f3739640 1 Processor -- start 2023-12-19T16:43:44.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f3739640 1 -- start start 2023-12-19T16:43:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f3739640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f64ec06d8e0 0x7f64ec06dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f3739640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec06bcf0 0x7f64ec15c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f3739640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec06e200 con 0x7f64ec15c740 2023-12-19T16:43:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f3739640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec0cccf0 con 0x7f64ec06d8e0 2023-12-19T16:43:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f3739640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0ccfd0 con 0x7f64ec06bcf0 2023-12-19T16:43:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f14d5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f64ec06d8e0 0x7f64ec06dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f14d5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f64ec06d8e0 0x7f64ec06dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54992/0 (socket says 172.21.15.26:54992) 2023-12-19T16:43:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f14d5640 1 -- 172.21.15.26:0/1600432206 learned_addr learned my addr 172.21.15.26:0/1600432206 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:44.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f0cd4640 1 --2- 172.21.15.26:0/1600432206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec06bcf0 0x7f64ec15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:44.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f14d5640 1 -- 172.21.15.26:0/1600432206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec06bcf0 msgr2=0x7f64ec15c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:44.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f14d5640 1 --2- 172.21.15.26:0/1600432206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec06bcf0 0x7f64ec15c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f14d5640 1 -- 172.21.15.26:0/1600432206 >> v1:172.21.15.160:6789/0 conn(0x7f64ec15c740 legacy=0x7f64ec15b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:44.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f14d5640 1 -- 172.21.15.26:0/1600432206 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec0cd2b0 con 0x7f64ec06d8e0 2023-12-19T16:43:44.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.462+0000 7f64f0cd4640 1 --2- 172.21.15.26:0/1600432206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec06bcf0 0x7f64ec15c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:44.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.466+0000 7f64f14d5640 1 --2- 172.21.15.26:0/1600432206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64ec06d8e0 0x7f64ec06dcc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009870 tx=0x7f64e801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc4d175a8193e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:44.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.466+0000 7f64e3fff640 1 -- 172.21.15.26:0/1600432206 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e80ac600 con 0x7f64ec06d8e0 2023-12-19T16:43:44.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.466+0000 7f64e3fff640 1 -- 172.21.15.26:0/1600432206 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e80a8050 con 0x7f64ec06d8e0 2023-12-19T16:43:44.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.466+0000 7f64e3fff640 1 -- 172.21.15.26:0/1600432206 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e80ac980 con 0x7f64ec06d8e0 2023-12-19T16:43:44.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.466+0000 7f64f3739640 1 -- 172.21.15.26:0/1600432206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64ec06d8e0 msgr2=0x7f64ec06dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:44.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.466+0000 7f64f3739640 1 --2- 172.21.15.26:0/1600432206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64ec06d8e0 0x7f64ec06dcc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009870 tx=0x7f64e801c4b0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.466+0000 7f64f3739640 1 -- 172.21.15.26:0/1600432206 shutdown_connections 2023-12-19T16:43:44.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.466+0000 7f64f3739640 1 --2- 172.21.15.26:0/1600432206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec06bcf0 0x7f64ec15c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.466+0000 7f64f3739640 1 --2- 172.21.15.26:0/1600432206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64ec06d8e0 0x7f64ec06dcc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.466+0000 7f64f3739640 1 -- 172.21.15.26:0/1600432206 >> 172.21.15.26:0/1600432206 conn(0x7f64ec0b0b60 msgr2=0x7f64ec0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:44.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.466+0000 7f64f3739640 1 -- 172.21.15.26:0/1600432206 shutdown_connections 2023-12-19T16:43:44.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.466+0000 7f64f3739640 1 -- 172.21.15.26:0/1600432206 wait complete. 2023-12-19T16:43:44.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f3739640 1 Processor -- start 2023-12-19T16:43:44.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f3739640 1 -- start start 2023-12-19T16:43:44.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f3739640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64ec06bcf0 0x7f64ec157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:44.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f3739640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64ec06d8e0 0x7f64ec155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:44.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f3739640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec15c740 0x7f64ec070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:44.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f3739640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec1524c0 con 0x7f64ec06d8e0 2023-12-19T16:43:44.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f3739640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec1527a0 con 0x7f64ec06bcf0 2023-12-19T16:43:44.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f3739640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec152a80 con 0x7f64ec15c740 2023-12-19T16:43:44.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f0cd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64ec06d8e0 0x7f64ec155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:44.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f0cd4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64ec06d8e0 0x7f64ec155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54996/0 (socket says 172.21.15.26:54996) 2023-12-19T16:43:44.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f0cd4640 1 -- 172.21.15.26:0/138674101 learned_addr learned my addr 172.21.15.26:0/138674101 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:44.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f14d5640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64ec06bcf0 0x7f64ec157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:44.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f1cd6640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec15c740 0x7f64ec070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:44.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f0cd4640 1 -- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec15c740 msgr2=0x7f64ec070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:44.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f0cd4640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec15c740 0x7f64ec070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f0cd4640 1 -- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64ec06bcf0 msgr2=0x7f64ec157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:44.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f0cd4640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64ec06bcf0 0x7f64ec157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f0cd4640 1 -- 172.21.15.26:0/138674101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e80094d0 con 0x7f64ec06d8e0 2023-12-19T16:43:44.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f0cd4640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64ec06d8e0 0x7f64ec155c50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00d800 tx=0x7f64dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:44.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64e1ffb640 1 -- 172.21.15.26:0/138674101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc0096c0 con 0x7f64ec06d8e0 2023-12-19T16:43:44.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64e1ffb640 1 -- 172.21.15.26:0/138674101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc009820 con 0x7f64ec06d8e0 2023-12-19T16:43:44.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64e1ffb640 1 -- 172.21.15.26:0/138674101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc009bd0 con 0x7f64ec06d8e0 2023-12-19T16:43:44.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f3739640 1 -- 172.21.15.26:0/138674101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec152d60 con 0x7f64ec06d8e0 2023-12-19T16:43:44.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f3739640 1 -- 172.21.15.26:0/138674101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec156190 con 0x7f64ec06d8e0 2023-12-19T16:43:44.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.470+0000 7f64f3739640 1 -- 172.21.15.26:0/138674101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec074270 con 0x7f64ec06d8e0 2023-12-19T16:43:44.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.474+0000 7f64e1ffb640 1 -- 172.21.15.26:0/138674101 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 85348+0+0 (secure 0 0 0) 0x7f64dc009d30 con 0x7f64ec06d8e0 2023-12-19T16:43:44.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.474+0000 7f64e1ffb640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f64d0065bd0 0x7f64d0068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:44.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.474+0000 7f64e1ffb640 1 -- 172.21.15.26:0/138674101 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f64dc083a70 con 0x7f64ec06d8e0 2023-12-19T16:43:44.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.474+0000 7f64f14d5640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f64d0065bd0 0x7f64d0068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:44.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.474+0000 7f64f14d5640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f64d0065bd0 0x7f64d0068090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f64e80ae000 tx=0x7f64e80a7000 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.478+0000 7f64e1ffb640 1 -- 172.21.15.26:0/138674101 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f64dc0512c0 con 0x7f64ec06d8e0 2023-12-19T16:43:44.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.658+0000 7f64f3739640 1 -- 172.21.15.26:0/138674101 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f64ec152910 con 0x7f64d0065bd0 2023-12-19T16:43:44.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.718+0000 7f64e1ffb640 1 -- 172.21.15.26:0/138674101 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f64ec152910 con 0x7f64d0065bd0 2023-12-19T16:43:44.727 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path /snap_test_dir1/dir3 2023-12-19T16:43:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.722+0000 7f64f3739640 1 -- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f64d0065bd0 msgr2=0x7f64d0068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.722+0000 7f64f3739640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f64d0065bd0 0x7f64d0068090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f64e80ae000 tx=0x7f64e80a7000 comp rx=0 tx=0).stop 2023-12-19T16:43:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.722+0000 7f64f3739640 1 -- 172.21.15.26:0/138674101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64ec06d8e0 msgr2=0x7f64ec155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.722+0000 7f64f3739640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64ec06d8e0 0x7f64ec155c50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00d800 tx=0x7f64dc00dcc0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.722+0000 7f64f3739640 1 -- 172.21.15.26:0/138674101 shutdown_connections 2023-12-19T16:43:44.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.722+0000 7f64f3739640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f64d0065bd0 0x7f64d0068090 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.722+0000 7f64f3739640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64ec15c740 0x7f64ec070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.722+0000 7f64f3739640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64ec06d8e0 0x7f64ec155c50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.722+0000 7f64f3739640 1 --2- 172.21.15.26:0/138674101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64ec06bcf0 0x7f64ec157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.722+0000 7f64f3739640 1 -- 172.21.15.26:0/138674101 >> 172.21.15.26:0/138674101 conn(0x7f64ec0b0b60 msgr2=0x7f64ec164b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:44.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.722+0000 7f64f3739640 1 -- 172.21.15.26:0/138674101 shutdown_connections 2023-12-19T16:43:44.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.722+0000 7f64f3739640 1 -- 172.21.15.26:0/138674101 wait complete. 2023-12-19T16:43:44.748 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-12-19T16:43:44.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:44.748 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-12-19T16:43:44.760 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir4 1M 2023-12-19T16:43:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.942+0000 7f599ca95640 1 Processor -- start 2023-12-19T16:43:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.942+0000 7f599ca95640 1 -- start start 2023-12-19T16:43:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.942+0000 7f599ca95640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5998159820 0x7f599815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.942+0000 7f599ca95640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599806fb60 con 0x7f599806f6c0 2023-12-19T16:43:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.942+0000 7f599ca95640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599806fe40 con 0x7f5998159c00 2023-12-19T16:43:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.942+0000 7f599ca95640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599815e470 con 0x7f5998159820 2023-12-19T16:43:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.942+0000 7f598ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f599806f6c0 0x7f59980c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35228/0 (socket says 172.21.15.26:35228) 2023-12-19T16:43:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.942+0000 7f598ffff640 1 -- 172.21.15.26:0/2550515251 learned_addr learned my addr 172.21.15.26:0/2550515251 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.942+0000 7f599659c640 1 --2- 172.21.15.26:0/2550515251 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5998159820 0x7f599815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.942+0000 7f5995d9b640 1 -- 172.21.15.26:0/2550515251 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623414991 0 0) 0x7f599806fb60 con 0x7f599806f6c0 2023-12-19T16:43:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.942+0000 7f5995d9b640 1 -- 172.21.15.26:0/2550515251 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5984003530 con 0x7f599806f6c0 2023-12-19T16:43:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.942+0000 7f5995d9b640 1 -- 172.21.15.26:0/2550515251 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807233274 0 0) 0x7f599806fe40 con 0x7f5998159c00 2023-12-19T16:43:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.946+0000 7f5995d9b640 1 -- 172.21.15.26:0/2550515251 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f599806fb60 con 0x7f5998159c00 2023-12-19T16:43:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.946+0000 7f599659c640 1 -- 172.21.15.26:0/2550515251 >> v1:172.21.15.160:6789/0 conn(0x7f5998159c00 legacy=0x7f5998070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.946+0000 7f599659c640 1 -- 172.21.15.26:0/2550515251 >> v1:172.21.15.26:6789/0 conn(0x7f599806f6c0 legacy=0x7f59980c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.946+0000 7f599659c640 1 -- 172.21.15.26:0/2550515251 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5998070120 con 0x7f5998159820 2023-12-19T16:43:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.946+0000 7f599659c640 1 --2- 172.21.15.26:0/2550515251 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5998159820 0x7f599815df30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f59880097f0 tx=0x7f598801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68fcd0ca37ed6c53 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.946+0000 7f5995d9b640 1 -- 172.21.15.26:0/2550515251 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59880aa5f0 con 0x7f5998159820 2023-12-19T16:43:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.946+0000 7f5995d9b640 1 -- 172.21.15.26:0/2550515251 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59880aa750 con 0x7f5998159820 2023-12-19T16:43:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.946+0000 7f5995d9b640 1 -- 172.21.15.26:0/2550515251 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59880aaac0 con 0x7f5998159820 2023-12-19T16:43:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 -- 172.21.15.26:0/2550515251 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5998159820 msgr2=0x7f599815df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 --2- 172.21.15.26:0/2550515251 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5998159820 0x7f599815df30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f59880097f0 tx=0x7f598801c490 comp rx=0 tx=0).stop 2023-12-19T16:43:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 -- 172.21.15.26:0/2550515251 shutdown_connections 2023-12-19T16:43:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 --2- 172.21.15.26:0/2550515251 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5998159820 0x7f599815df30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 -- 172.21.15.26:0/2550515251 >> 172.21.15.26:0/2550515251 conn(0x7f59980ce730 msgr2=0x7f59980ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 -- 172.21.15.26:0/2550515251 shutdown_connections 2023-12-19T16:43:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 -- 172.21.15.26:0/2550515251 wait complete. 2023-12-19T16:43:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 Processor -- start 2023-12-19T16:43:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 -- start start 2023-12-19T16:43:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f599806f6c0 0x7f599815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5998159820 0x7f5998151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5998159c00 0x7f5998155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599815c020 con 0x7f5998159c00 2023-12-19T16:43:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599815a0f0 con 0x7f5998159820 2023-12-19T16:43:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599ca95640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599815a3d0 con 0x7f599806f6c0 2023-12-19T16:43:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f5996d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5998159c00 0x7f5998155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f5996d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5998159c00 0x7f5998155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55008/0 (socket says 172.21.15.26:55008) 2023-12-19T16:43:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f5996d9d640 1 -- 172.21.15.26:0/561280961 learned_addr learned my addr 172.21.15.26:0/561280961 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f598ffff640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5998159820 0x7f5998151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599659c640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f599806f6c0 0x7f599815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f5996d9d640 1 -- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f599806f6c0 msgr2=0x7f599815aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f5996d9d640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f599806f6c0 0x7f599815aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f5996d9d640 1 -- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5998159820 msgr2=0x7f5998151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f5996d9d640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5998159820 0x7f5998151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f5996d9d640 1 -- 172.21.15.26:0/561280961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5988009420 con 0x7f5998159c00 2023-12-19T16:43:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f599659c640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f599806f6c0 0x7f599815aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:44.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.950+0000 7f598ffff640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5998159820 0x7f5998151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.954+0000 7f5996d9d640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5998159c00 0x7f5998155e30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f599000c730 tx=0x7f599000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.954+0000 7f598effd640 1 -- 172.21.15.26:0/561280961 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5990003b80 con 0x7f5998159c00 2023-12-19T16:43:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.954+0000 7f598effd640 1 -- 172.21.15.26:0/561280961 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5990003ce0 con 0x7f5998159c00 2023-12-19T16:43:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.954+0000 7f598effd640 1 -- 172.21.15.26:0/561280961 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599000aa30 con 0x7f5998159c00 2023-12-19T16:43:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.954+0000 7f599ca95640 1 -- 172.21.15.26:0/561280961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599806b880 con 0x7f5998159c00 2023-12-19T16:43:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.954+0000 7f599ca95640 1 -- 172.21.15.26:0/561280961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599815a990 con 0x7f5998159c00 2023-12-19T16:43:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.954+0000 7f599ca95640 1 -- 172.21.15.26:0/561280961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5998074270 con 0x7f5998159c00 2023-12-19T16:43:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.954+0000 7f598effd640 1 -- 172.21.15.26:0/561280961 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 85348+0+0 (secure 0 0 0) 0x7f599000ab90 con 0x7f5998159c00 2023-12-19T16:43:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.954+0000 7f598effd640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5978065bd0 0x7f5978068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.954+0000 7f598effd640 1 -- 172.21.15.26:0/561280961 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f59900832a0 con 0x7f5998159c00 2023-12-19T16:43:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.954+0000 7f599659c640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5978065bd0 0x7f5978068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.958+0000 7f599659c640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5978065bd0 0x7f5978068090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f59880ae000 tx=0x7f5988008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:44.958+0000 7f598effd640 1 -- 172.21.15.26:0/561280961 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5990050a40 con 0x7f5998159c00 2023-12-19T16:43:45.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.134+0000 7f599ca95640 1 -- 172.21.15.26:0/561280961 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f599806b700 con 0x7f5978065bd0 2023-12-19T16:43:45.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.198+0000 7f598effd640 1 -- 172.21.15.26:0/561280961 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f59880bb0c0 con 0x7f5978065bd0 2023-12-19T16:43:45.207 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path /snap_test_dir1/dir4 2023-12-19T16:43:45.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.202+0000 7f599ca95640 1 -- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5978065bd0 msgr2=0x7f5978068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:45.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.202+0000 7f599ca95640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5978065bd0 0x7f5978068090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f59880ae000 tx=0x7f5988008040 comp rx=0 tx=0).stop 2023-12-19T16:43:45.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.202+0000 7f599ca95640 1 -- 172.21.15.26:0/561280961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5998159c00 msgr2=0x7f5998155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:45.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.202+0000 7f599ca95640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5998159c00 0x7f5998155e30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f599000c730 tx=0x7f599000cbf0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.202+0000 7f599ca95640 1 -- 172.21.15.26:0/561280961 shutdown_connections 2023-12-19T16:43:45.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.202+0000 7f599ca95640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5998159c00 0x7f5998155e30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.202+0000 7f599ca95640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5998159820 0x7f5998151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.202+0000 7f599ca95640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5978065bd0 0x7f5978068090 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.202+0000 7f599ca95640 1 --2- 172.21.15.26:0/561280961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f599806f6c0 0x7f599815aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.202+0000 7f599ca95640 1 -- 172.21.15.26:0/561280961 >> 172.21.15.26:0/561280961 conn(0x7f59980ce730 msgr2=0x7f599806ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:45.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.202+0000 7f599ca95640 1 -- 172.21.15.26:0/561280961 shutdown_connections 2023-12-19T16:43:45.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.202+0000 7f599ca95640 1 -- 172.21.15.26:0/561280961 wait complete. 2023-12-19T16:43:45.228 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-12-19T16:43:45.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:45.228 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-12-19T16:43:45.239 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir5 1M 2023-12-19T16:43:45.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.422+0000 7f7c0d0e5640 1 Processor -- start 2023-12-19T16:43:45.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.422+0000 7f7c0d0e5640 1 -- start start 2023-12-19T16:43:45.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.422+0000 7f7c0d0e5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7c0815a9c0 0x7f7c080706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.422+0000 7f7c0d0e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c0806ac80 0x7f7c080b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.422+0000 7f7c0d0e5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c08070d90 con 0x7f7c0815ada0 2023-12-19T16:43:45.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.422+0000 7f7c0d0e5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c080cc270 con 0x7f7c0815a9c0 2023-12-19T16:43:45.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.422+0000 7f7c0d0e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c08071070 con 0x7f7c0806ac80 2023-12-19T16:43:45.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.422+0000 7f7c0659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7c0815ada0 0x7f7c080b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35238/0 (socket says 172.21.15.26:35238) 2023-12-19T16:43:45.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.422+0000 7f7c0659c640 1 -- 172.21.15.26:0/4194564115 learned_addr learned my addr 172.21.15.26:0/4194564115 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c06d9d640 1 --2- 172.21.15.26:0/4194564115 >> v2:172.21.15.160:3300/0 conn(0x7f7c0815a9c0 0x7f7c080706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c0759e640 1 --2- 172.21.15.26:0/4194564115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c0806ac80 0x7f7c080b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176066799 0 0) 0x7f7c08070d90 con 0x7f7c0815ada0 2023-12-19T16:43:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bf0003530 con 0x7f7c0815ada0 2023-12-19T16:43:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 673212715 0 0) 0x7f7bf0003530 con 0x7f7c0815ada0 2023-12-19T16:43:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c0806ac80 msgr2=0x7f7c080b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 --2- 172.21.15.26:0/4194564115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c0806ac80 0x7f7c080b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 >> v2:172.21.15.160:3300/0 conn(0x7f7c0815a9c0 msgr2=0x7f7c080706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 --2- 172.21.15.26:0/4194564115 >> v2:172.21.15.160:3300/0 conn(0x7f7c0815a9c0 0x7f7c080706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c08071350 con 0x7f7c0815ada0 2023-12-19T16:43:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f7bfc001680 con 0x7f7c0815ada0 2023-12-19T16:43:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c06d9d640 1 --2- 172.21.15.26:0/4194564115 >> v2:172.21.15.160:3300/0 conn(0x7f7c0815a9c0 0x7f7c080706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:45.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 >> v1:172.21.15.26:6789/0 conn(0x7f7c0815ada0 legacy=0x7f7c080b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 --2- 172.21.15.26:0/4194564115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bf0003980 0x7f7bf0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 --2- 172.21.15.26:0/4194564115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c0806ac80 0x7f7bf00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 --2- 172.21.15.26:0/4194564115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0004d10 0x7f7bf0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c080cc270 con 0x7f7bf0003980 2023-12-19T16:43:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c08071070 con 0x7f7bf0004d10 2023-12-19T16:43:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf000b990 con 0x7f7c0806ac80 2023-12-19T16:43:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c06d9d640 1 --2- 172.21.15.26:0/4194564115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bf0003980 0x7f7bf0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c0759e640 1 --2- 172.21.15.26:0/4194564115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0004d10 0x7f7bf0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c0659c640 1 --2- 172.21.15.26:0/4194564115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c0806ac80 0x7f7bf00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c06d9d640 1 -- 172.21.15.26:0/4194564115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c0806ac80 msgr2=0x7f7bf00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c06d9d640 1 --2- 172.21.15.26:0/4194564115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c0806ac80 0x7f7bf00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c06d9d640 1 -- 172.21.15.26:0/4194564115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0004d10 msgr2=0x7f7bf0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:45.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c06d9d640 1 --2- 172.21.15.26:0/4194564115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0004d10 0x7f7bf0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c06d9d640 1 -- 172.21.15.26:0/4194564115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf0003530 con 0x7f7bf0003980 2023-12-19T16:43:45.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.426+0000 7f7c0759e640 1 --2- 172.21.15.26:0/4194564115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bf0004d10 0x7f7bf0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.430+0000 7f7c06d9d640 1 --2- 172.21.15.26:0/4194564115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bf0003980 0x7f7bf0003da0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009cb0 tx=0x7f7bf801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c0eaf600028766 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.430+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80a57f0 con 0x7f7bf0003980 2023-12-19T16:43:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.430+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf80a5950 con 0x7f7bf0003980 2023-12-19T16:43:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.430+0000 7f7c05d9b640 1 -- 172.21.15.26:0/4194564115 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80a5d00 con 0x7f7bf0003980 2023-12-19T16:43:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.430+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/4194564115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bf0003980 msgr2=0x7f7bf0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.430+0000 7f7c0d0e5640 1 --2- 172.21.15.26:0/4194564115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bf0003980 0x7f7bf0003da0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009cb0 tx=0x7f7bf801cc40 comp rx=0 tx=0).stop 2023-12-19T16:43:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.430+0000 7f7c0759e640 1 -- 172.21.15.26:0/4194564115 reap_dead start 2023-12-19T16:43:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.430+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/4194564115 shutdown_connections 2023-12-19T16:43:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.430+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/4194564115 >> 172.21.15.26:0/4194564115 conn(0x7f7c080ce730 msgr2=0x7f7c080ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.430+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/4194564115 shutdown_connections 2023-12-19T16:43:45.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.430+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/4194564115 wait complete. 2023-12-19T16:43:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.430+0000 7f7c0d0e5640 1 Processor -- start 2023-12-19T16:43:45.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.430+0000 7f7c0d0e5640 1 -- start start 2023-12-19T16:43:45.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c0d0e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bf0003980 0x7f7c0815bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c0d0e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c080c51d0 0x7f7c08153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c0d0e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c0815d7f0 0x7f7c0815dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c0d0e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0815bb90 con 0x7f7bf0003980 2023-12-19T16:43:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c0d0e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c08155db0 con 0x7f7c080c51d0 2023-12-19T16:43:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c0d0e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c08155ad0 con 0x7f7c0815d7f0 2023-12-19T16:43:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c06d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bf0003980 0x7f7c0815bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c06d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bf0003980 0x7f7c0815bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55032/0 (socket says 172.21.15.26:55032) 2023-12-19T16:43:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c06d9d640 1 -- 172.21.15.26:0/1154179753 learned_addr learned my addr 172.21.15.26:0/1154179753 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c0759e640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c0815d7f0 0x7f7c0815dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c0659c640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c080c51d0 0x7f7c08153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c06d9d640 1 -- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c0815d7f0 msgr2=0x7f7c0815dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c06d9d640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c0815d7f0 0x7f7c0815dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c06d9d640 1 -- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c080c51d0 msgr2=0x7f7c08153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c06d9d640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c080c51d0 0x7f7c08153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c06d9d640 1 -- 172.21.15.26:0/1154179753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0815b3b0 con 0x7f7bf0003980 2023-12-19T16:43:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c06d9d640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bf0003980 0x7f7c0815bfe0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80b0040 tx=0x7f7bf0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7bef7fe640 1 -- 172.21.15.26:0/1154179753 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8004220 con 0x7f7bf0003980 2023-12-19T16:43:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7bef7fe640 1 -- 172.21.15.26:0/1154179753 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf80043a0 con 0x7f7bf0003980 2023-12-19T16:43:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/1154179753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0815b690 con 0x7f7bf0003980 2023-12-19T16:43:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7bef7fe640 1 -- 172.21.15.26:0/1154179753 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8004510 con 0x7f7bf0003980 2023-12-19T16:43:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/1154179753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0815a140 con 0x7f7bf0003980 2023-12-19T16:43:45.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/1154179753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0815bd00 con 0x7f7bf0003980 2023-12-19T16:43:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.434+0000 7f7bef7fe640 1 -- 172.21.15.26:0/1154179753 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 85348+0+0 (secure 0 0 0) 0x7f7bf80b4040 con 0x7f7bf0003980 2023-12-19T16:43:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.438+0000 7f7bef7fe640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7bf4065bd0 0x7f7bf4068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.438+0000 7f7bef7fe640 1 -- 172.21.15.26:0/1154179753 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f7bf811db00 con 0x7f7bf0003980 2023-12-19T16:43:45.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.438+0000 7f7c0659c640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7bf4065bd0 0x7f7bf4068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.438+0000 7f7c0659c640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7bf4065bd0 0x7f7bf4068090 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7c080c4a60 tx=0x7f7bfc007900 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:45.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.442+0000 7f7bef7fe640 1 -- 172.21.15.26:0/1154179753 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7c0815bd00 con 0x7f7bf0003980 2023-12-19T16:43:45.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.618+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/1154179753 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c0815b880 con 0x7f7bf4065bd0 2023-12-19T16:43:45.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.686+0000 7f7bef7fe640 1 -- 172.21.15.26:0/1154179753 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f7c0815b880 con 0x7f7bf4065bd0 2023-12-19T16:43:45.692 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path /snap_test_dir1/dir5 2023-12-19T16:43:45.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.686+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7bf4065bd0 msgr2=0x7f7bf4068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:45.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.686+0000 7f7c0d0e5640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7bf4065bd0 0x7f7bf4068090 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7c080c4a60 tx=0x7f7bfc007900 comp rx=0 tx=0).stop 2023-12-19T16:43:45.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.686+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/1154179753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bf0003980 msgr2=0x7f7c0815bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:45.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.686+0000 7f7c0d0e5640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bf0003980 0x7f7c0815bfe0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7bf80b0040 tx=0x7f7bf0005800 comp rx=0 tx=0).stop 2023-12-19T16:43:45.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.686+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/1154179753 shutdown_connections 2023-12-19T16:43:45.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.686+0000 7f7c0d0e5640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7bf4065bd0 0x7f7bf4068090 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.686+0000 7f7c0d0e5640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c0815d7f0 0x7f7c0815dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.686+0000 7f7c0d0e5640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c080c51d0 0x7f7c08153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.686+0000 7f7c0d0e5640 1 --2- 172.21.15.26:0/1154179753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bf0003980 0x7f7c0815bfe0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.686+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/1154179753 >> 172.21.15.26:0/1154179753 conn(0x7f7c080ce730 msgr2=0x7f7c080c90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:45.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.686+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/1154179753 shutdown_connections 2023-12-19T16:43:45.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.686+0000 7f7c0d0e5640 1 -- 172.21.15.26:0/1154179753 wait complete. 2023-12-19T16:43:45.713 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-12-19T16:43:45.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:45.714 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-12-19T16:43:45.724 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir6 1M 2023-12-19T16:43:45.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.910+0000 7f2deee8c640 1 Processor -- start 2023-12-19T16:43:45.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.910+0000 7f2deee8c640 1 -- start start 2023-12-19T16:43:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.910+0000 7f2deee8c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2de806db00 0x7f2de806bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.910+0000 7f2deee8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2de80adf00 0x7f2de80cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.910+0000 7f2deee8c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de80ca360 con 0x7f2de80ccc90 2023-12-19T16:43:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.910+0000 7f2deee8c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de80cee80 con 0x7f2de806db00 2023-12-19T16:43:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.910+0000 7f2deee8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de80cd650 con 0x7f2de80adf00 2023-12-19T16:43:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.914+0000 7f2ded429640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2de80adf00 0x7f2de80cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.914+0000 7f2decc28640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2de806db00 0x7f2de806bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.914+0000 7f2ded429640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2de80adf00 0x7f2de80cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:44174/0 (socket says 172.21.15.26:44174) 2023-12-19T16:43:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.914+0000 7f2ded429640 1 -- 172.21.15.26:0/1753391112 learned_addr learned my addr 172.21.15.26:0/1753391112 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.914+0000 7f2ded429640 1 -- 172.21.15.26:0/1753391112 >> v2:172.21.15.160:3300/0 conn(0x7f2de806db00 msgr2=0x7f2de806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.914+0000 7f2ded429640 1 --2- 172.21.15.26:0/1753391112 >> v2:172.21.15.160:3300/0 conn(0x7f2de806db00 0x7f2de806bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.914+0000 7f2ded429640 1 -- 172.21.15.26:0/1753391112 >> v1:172.21.15.160:6789/0 conn(0x7f2de80ccc90 legacy=0x7f2de80ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.914+0000 7f2ded429640 1 -- 172.21.15.26:0/1753391112 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de80c9750 con 0x7f2de80adf00 2023-12-19T16:43:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.914+0000 7f2decc28640 1 --2- 172.21.15.26:0/1753391112 >> v2:172.21.15.160:3300/0 conn(0x7f2de806db00 0x7f2de806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:45.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.914+0000 7f2ded429640 1 --2- 172.21.15.26:0/1753391112 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2de80adf00 0x7f2de80cd110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009e50 tx=0x7f2de001c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3341101b9c0d7252 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:45.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.914+0000 7f2de77fe640 1 -- 172.21.15.26:0/1753391112 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de00a5840 con 0x7f2de80adf00 2023-12-19T16:43:45.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.914+0000 7f2de77fe640 1 -- 172.21.15.26:0/1753391112 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de00a59a0 con 0x7f2de80adf00 2023-12-19T16:43:45.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.914+0000 7f2de77fe640 1 -- 172.21.15.26:0/1753391112 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de00a5d30 con 0x7f2de80adf00 2023-12-19T16:43:45.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 -- 172.21.15.26:0/1753391112 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2de80adf00 msgr2=0x7f2de80cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:45.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 --2- 172.21.15.26:0/1753391112 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2de80adf00 0x7f2de80cd110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009e50 tx=0x7f2de001c980 comp rx=0 tx=0).stop 2023-12-19T16:43:45.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 -- 172.21.15.26:0/1753391112 shutdown_connections 2023-12-19T16:43:45.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 --2- 172.21.15.26:0/1753391112 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2de80adf00 0x7f2de80cd110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 --2- 172.21.15.26:0/1753391112 >> v2:172.21.15.160:3300/0 conn(0x7f2de806db00 0x7f2de806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 -- 172.21.15.26:0/1753391112 >> 172.21.15.26:0/1753391112 conn(0x7f2de80b0b60 msgr2=0x7f2de80b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:45.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 -- 172.21.15.26:0/1753391112 shutdown_connections 2023-12-19T16:43:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 -- 172.21.15.26:0/1753391112 wait complete. 2023-12-19T16:43:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 Processor -- start 2023-12-19T16:43:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 -- start start 2023-12-19T16:43:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de806db00 0x7f2de8154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2de80adf00 0x7f2de8155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de80ccc90 0x7f2de8152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8155680 con 0x7f2de806db00 2023-12-19T16:43:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de815f9d0 con 0x7f2de80ccc90 2023-12-19T16:43:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2deee8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8071590 con 0x7f2de80adf00 2023-12-19T16:43:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2decc28640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de806db00 0x7f2de8154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2de7fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2de80adf00 0x7f2de8155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2decc28640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de806db00 0x7f2de8154ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59510/0 (socket says 172.21.15.26:59510) 2023-12-19T16:43:45.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2decc28640 1 -- 172.21.15.26:0/3924376769 learned_addr learned my addr 172.21.15.26:0/3924376769 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:45.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.918+0000 7f2ded429640 1 --2- 172.21.15.26:0/3924376769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de80ccc90 0x7f2de8152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.922+0000 7f2decc28640 1 -- 172.21.15.26:0/3924376769 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2de80adf00 msgr2=0x7f2de8155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:45.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.922+0000 7f2decc28640 1 --2- 172.21.15.26:0/3924376769 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2de80adf00 0x7f2de8155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.922+0000 7f2decc28640 1 -- 172.21.15.26:0/3924376769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de80ccc90 msgr2=0x7f2de8152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:45.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.922+0000 7f2decc28640 1 --2- 172.21.15.26:0/3924376769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de80ccc90 0x7f2de8152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:45.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.922+0000 7f2decc28640 1 -- 172.21.15.26:0/3924376769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc009420 con 0x7f2de806db00 2023-12-19T16:43:45.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.922+0000 7f2decc28640 1 --2- 172.21.15.26:0/3924376769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de806db00 0x7f2de8154ee0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00e980 tx=0x7f2ddc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:45.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.922+0000 7f2de57fa640 1 -- 172.21.15.26:0/3924376769 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc00ca00 con 0x7f2de806db00 2023-12-19T16:43:45.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.922+0000 7f2de57fa640 1 -- 172.21.15.26:0/3924376769 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ddc00cb60 con 0x7f2de806db00 2023-12-19T16:43:45.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.922+0000 7f2de57fa640 1 -- 172.21.15.26:0/3924376769 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc013470 con 0x7f2de806db00 2023-12-19T16:43:45.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.922+0000 7f2deee8c640 1 -- 172.21.15.26:0/3924376769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de0009b30 con 0x7f2de806db00 2023-12-19T16:43:45.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.922+0000 7f2deee8c640 1 -- 172.21.15.26:0/3924376769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de8071b50 con 0x7f2de806db00 2023-12-19T16:43:45.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.922+0000 7f2deee8c640 1 -- 172.21.15.26:0/3924376769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de8071d70 con 0x7f2de806db00 2023-12-19T16:43:45.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.922+0000 7f2de57fa640 1 -- 172.21.15.26:0/3924376769 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 85348+0+0 (secure 0 0 0) 0x7f2ddc0135d0 con 0x7f2de806db00 2023-12-19T16:43:45.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.926+0000 7f2de57fa640 1 --2- 172.21.15.26:0/3924376769 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2dcc065bd0 0x7f2dcc068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:45.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.926+0000 7f2de57fa640 1 -- 172.21.15.26:0/3924376769 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f2ddc083700 con 0x7f2de806db00 2023-12-19T16:43:45.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.926+0000 7f2de7fff640 1 --2- 172.21.15.26:0/3924376769 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2dcc065bd0 0x7f2dcc068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:45.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.926+0000 7f2de7fff640 1 --2- 172.21.15.26:0/3924376769 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2dcc065bd0 0x7f2dcc068090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2de8070f00 tx=0x7f2dd4009040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:45.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:45.930+0000 7f2de57fa640 1 -- 172.21.15.26:0/3924376769 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2ddc050ec0 con 0x7f2de806db00 2023-12-19T16:43:46.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.102+0000 7f2deee8c640 1 -- 172.21.15.26:0/3924376769 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2de80c9ff0 con 0x7f2dcc065bd0 2023-12-19T16:43:46.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.166+0000 7f2de57fa640 1 -- 172.21.15.26:0/3924376769 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f2dd40014f0 con 0x7f2dcc065bd0 2023-12-19T16:43:46.174 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path /snap_test_dir1/dir6 2023-12-19T16:43:46.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.170+0000 7f2deee8c640 1 -- 172.21.15.26:0/3924376769 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2dcc065bd0 msgr2=0x7f2dcc068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:46.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.170+0000 7f2deee8c640 1 --2- 172.21.15.26:0/3924376769 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2dcc065bd0 0x7f2dcc068090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2de8070f00 tx=0x7f2dd4009040 comp rx=0 tx=0).stop 2023-12-19T16:43:46.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.170+0000 7f2deee8c640 1 -- 172.21.15.26:0/3924376769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de806db00 msgr2=0x7f2de8154ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:46.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.170+0000 7f2deee8c640 1 --2- 172.21.15.26:0/3924376769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de806db00 0x7f2de8154ee0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00e980 tx=0x7f2ddc00ee40 comp rx=0 tx=0).stop 2023-12-19T16:43:46.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.170+0000 7f2deee8c640 1 -- 172.21.15.26:0/3924376769 shutdown_connections 2023-12-19T16:43:46.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.170+0000 7f2deee8c640 1 --2- 172.21.15.26:0/3924376769 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2de80ccc90 0x7f2de8152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.170+0000 7f2deee8c640 1 --2- 172.21.15.26:0/3924376769 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2dcc065bd0 0x7f2dcc068090 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.170+0000 7f2deee8c640 1 --2- 172.21.15.26:0/3924376769 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2de80adf00 0x7f2de8155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.170+0000 7f2deee8c640 1 --2- 172.21.15.26:0/3924376769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2de806db00 0x7f2de8154ee0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.170+0000 7f2deee8c640 1 -- 172.21.15.26:0/3924376769 >> 172.21.15.26:0/3924376769 conn(0x7f2de80b0b60 msgr2=0x7f2de806d360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:46.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.170+0000 7f2deee8c640 1 -- 172.21.15.26:0/3924376769 shutdown_connections 2023-12-19T16:43:46.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.170+0000 7f2deee8c640 1 -- 172.21.15.26:0/3924376769 wait complete. 2023-12-19T16:43:46.195 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-12-19T16:43:46.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:46.195 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-12-19T16:43:46.206 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir7 1M 2023-12-19T16:43:46.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb47f0640 1 Processor -- start 2023-12-19T16:43:46.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb47f0640 1 -- start start 2023-12-19T16:43:46.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb47f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbac159820 0x7ffbac15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:46.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb47f0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffbac159c00 0x7ffbac070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:46.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb47f0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbac06fb60 con 0x7ffbac06f6c0 2023-12-19T16:43:46.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb47f0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbac15e470 con 0x7ffbac159c00 2023-12-19T16:43:46.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb47f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac06fe40 con 0x7ffbac159820 2023-12-19T16:43:46.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb2d8d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffbac159c00 0x7ffbac070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:46.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb258c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbac159820 0x7ffbac15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:46.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb1d8b640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7ffbac06f6c0 0x7ffbac0c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:56916/0 (socket says 172.21.15.26:56916) 2023-12-19T16:43:46.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb1d8b640 1 -- 172.21.15.26:0/2643221908 learned_addr learned my addr 172.21.15.26:0/2643221908 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:46.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb2d8d640 1 -- 172.21.15.26:0/2643221908 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbac159820 msgr2=0x7ffbac15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:46.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb2d8d640 1 --2- 172.21.15.26:0/2643221908 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbac159820 0x7ffbac15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb2d8d640 1 -- 172.21.15.26:0/2643221908 >> v1:172.21.15.160:6789/0 conn(0x7ffbac06f6c0 legacy=0x7ffbac0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:46.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb2d8d640 1 -- 172.21.15.26:0/2643221908 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac070120 con 0x7ffbac159c00 2023-12-19T16:43:46.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb258c640 1 --2- 172.21.15.26:0/2643221908 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbac159820 0x7ffbac15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:46.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.390+0000 7ffbb158a640 1 -- 172.21.15.26:0/2643221908 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464822786 0 0) 0x7ffbac06fb60 con 0x7ffbac06f6c0 2023-12-19T16:43:46.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.394+0000 7ffbb2d8d640 1 --2- 172.21.15.26:0/2643221908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbac159c00 0x7ffbac070c00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009d90 tx=0x7ffba401c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4fc06de28615330 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:46.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.394+0000 7ffbb158a640 1 -- 172.21.15.26:0/2643221908 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba40a58f0 con 0x7ffbac159c00 2023-12-19T16:43:46.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.394+0000 7ffbb158a640 1 -- 172.21.15.26:0/2643221908 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba40a5a50 con 0x7ffbac159c00 2023-12-19T16:43:46.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.394+0000 7ffbb158a640 1 -- 172.21.15.26:0/2643221908 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba40a5de0 con 0x7ffbac159c00 2023-12-19T16:43:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.394+0000 7ffbb47f0640 1 -- 172.21.15.26:0/2643221908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbac159c00 msgr2=0x7ffbac070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.394+0000 7ffbb47f0640 1 --2- 172.21.15.26:0/2643221908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbac159c00 0x7ffbac070c00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009d90 tx=0x7ffba401c920 comp rx=0 tx=0).stop 2023-12-19T16:43:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.394+0000 7ffbb47f0640 1 -- 172.21.15.26:0/2643221908 shutdown_connections 2023-12-19T16:43:46.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.394+0000 7ffbb47f0640 1 --2- 172.21.15.26:0/2643221908 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbac159c00 0x7ffbac070c00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.394+0000 7ffbb47f0640 1 --2- 172.21.15.26:0/2643221908 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbac159820 0x7ffbac15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.394+0000 7ffbb47f0640 1 -- 172.21.15.26:0/2643221908 >> 172.21.15.26:0/2643221908 conn(0x7ffbac0ce730 msgr2=0x7ffbac0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:46.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.394+0000 7ffbb47f0640 1 -- 172.21.15.26:0/2643221908 shutdown_connections 2023-12-19T16:43:46.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.394+0000 7ffbb47f0640 1 -- 172.21.15.26:0/2643221908 wait complete. 2023-12-19T16:43:46.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb47f0640 1 Processor -- start 2023-12-19T16:43:46.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb47f0640 1 -- start start 2023-12-19T16:43:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb47f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbac06f6c0 0x7ffbac14f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb47f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbac159820 0x7ffbac14fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb47f0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbac159c00 0x7ffbac0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb47f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac1516d0 con 0x7ffbac06f6c0 2023-12-19T16:43:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb47f0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac0ca630 con 0x7ffbac159c00 2023-12-19T16:43:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb47f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac0ca910 con 0x7ffbac159820 2023-12-19T16:43:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb258c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbac06f6c0 0x7ffbac14f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:46.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb258c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbac06f6c0 0x7ffbac14f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59526/0 (socket says 172.21.15.26:59526) 2023-12-19T16:43:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb258c640 1 -- 172.21.15.26:0/344659341 learned_addr learned my addr 172.21.15.26:0/344659341 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb2d8d640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbac159c00 0x7ffbac0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:46.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb1d8b640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbac159820 0x7ffbac14fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb258c640 1 -- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbac159820 msgr2=0x7ffbac14fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb258c640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbac159820 0x7ffbac14fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb258c640 1 -- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbac159c00 msgr2=0x7ffbac0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb258c640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbac159c00 0x7ffbac0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb258c640 1 -- 172.21.15.26:0/344659341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba8009420 con 0x7ffbac06f6c0 2023-12-19T16:43:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb258c640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbac06f6c0 0x7ffbac14f3c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ffba800ebe0 tx=0x7ffba800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffba2ffd640 1 -- 172.21.15.26:0/344659341 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba800eee0 con 0x7ffbac06f6c0 2023-12-19T16:43:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffba2ffd640 1 -- 172.21.15.26:0/344659341 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba800cc30 con 0x7ffbac06f6c0 2023-12-19T16:43:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffba2ffd640 1 -- 172.21.15.26:0/344659341 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8013470 con 0x7ffbac06f6c0 2023-12-19T16:43:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb47f0640 1 -- 172.21.15.26:0/344659341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba4009a70 con 0x7ffbac06f6c0 2023-12-19T16:43:46.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb47f0640 1 -- 172.21.15.26:0/344659341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac06b880 con 0x7ffbac06f6c0 2023-12-19T16:43:46.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.398+0000 7ffbb47f0640 1 -- 172.21.15.26:0/344659341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbac0cadf0 con 0x7ffbac06f6c0 2023-12-19T16:43:46.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.402+0000 7ffba2ffd640 1 -- 172.21.15.26:0/344659341 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7ffba80135d0 con 0x7ffbac06f6c0 2023-12-19T16:43:46.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.402+0000 7ffba2ffd640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb90065c60 0x7ffb90068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:46.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.402+0000 7ffba2ffd640 1 -- 172.21.15.26:0/344659341 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ffba8083c20 con 0x7ffbac06f6c0 2023-12-19T16:43:46.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.402+0000 7ffbb1d8b640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb90065c60 0x7ffb90068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:46.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.402+0000 7ffbb1d8b640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb90065c60 0x7ffb90068120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0c51d0 tx=0x7ffb9c007d60 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:46.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.406+0000 7ffba2ffd640 1 -- 172.21.15.26:0/344659341 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffba8051320 con 0x7ffbac06f6c0 2023-12-19T16:43:46.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.582+0000 7ffbb47f0640 1 -- 172.21.15.26:0/344659341 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbac06b700 con 0x7ffb90065c60 2023-12-19T16:43:46.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.646+0000 7ffba2ffd640 1 -- 172.21.15.26:0/344659341 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ffb9c007300 con 0x7ffb90065c60 2023-12-19T16:43:46.651 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path /snap_test_dir1/dir7 2023-12-19T16:43:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.646+0000 7ffbb47f0640 1 -- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb90065c60 msgr2=0x7ffb90068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.646+0000 7ffbb47f0640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb90065c60 0x7ffb90068120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffbac0c51d0 tx=0x7ffb9c007d60 comp rx=0 tx=0).stop 2023-12-19T16:43:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.646+0000 7ffbb47f0640 1 -- 172.21.15.26:0/344659341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbac06f6c0 msgr2=0x7ffbac14f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.646+0000 7ffbb47f0640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbac06f6c0 0x7ffbac14f3c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ffba800ebe0 tx=0x7ffba800c520 comp rx=0 tx=0).stop 2023-12-19T16:43:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.646+0000 7ffbb47f0640 1 -- 172.21.15.26:0/344659341 shutdown_connections 2023-12-19T16:43:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.646+0000 7ffbb47f0640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbac159c00 0x7ffbac0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.646+0000 7ffbb47f0640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb90065c60 0x7ffb90068120 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.646+0000 7ffbb47f0640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbac159820 0x7ffbac14fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.646+0000 7ffbb47f0640 1 --2- 172.21.15.26:0/344659341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbac06f6c0 0x7ffbac14f3c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.646+0000 7ffbb47f0640 1 -- 172.21.15.26:0/344659341 >> 172.21.15.26:0/344659341 conn(0x7ffbac0ce730 msgr2=0x7ffbac0af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.646+0000 7ffbb47f0640 1 -- 172.21.15.26:0/344659341 shutdown_connections 2023-12-19T16:43:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.646+0000 7ffbb47f0640 1 -- 172.21.15.26:0/344659341 wait complete. 2023-12-19T16:43:46.672 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-12-19T16:43:46.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:46.672 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-12-19T16:43:46.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir8 1M 2023-12-19T16:43:46.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.866+0000 7fa6f416d640 1 Processor -- start 2023-12-19T16:43:46.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.866+0000 7fa6f416d640 1 -- start start 2023-12-19T16:43:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.866+0000 7fa6f416d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa6ec15a9c0 0x7fa6ec0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.866+0000 7fa6f416d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa6ec15ada0 0x7fa6ec0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.866+0000 7fa6f416d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ec0b46c0 con 0x7fa6ec06ac80 2023-12-19T16:43:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.866+0000 7fa6f416d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ec0cc270 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.866+0000 7fa6f416d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ec070d90 con 0x7fa6ec15ada0 2023-12-19T16:43:46.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.866+0000 7fa6f1f09640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa6ec15a9c0 0x7fa6ec0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:46.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.866+0000 7fa6f270a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa6ec06ac80 0x7fa6ec0b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32918/0 (socket says 172.21.15.26:32918) 2023-12-19T16:43:46.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.866+0000 7fa6f270a640 1 -- 172.21.15.26:0/1060695531 learned_addr learned my addr 172.21.15.26:0/1060695531 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:46.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.866+0000 7fa6f1708640 1 --2- 172.21.15.26:0/1060695531 >> v2:172.21.15.160:3300/0 conn(0x7fa6ec15ada0 0x7fa6ec0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:46.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f1f09640 1 -- 172.21.15.26:0/1060695531 >> v2:172.21.15.160:3300/0 conn(0x7fa6ec15ada0 msgr2=0x7fa6ec0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:46.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f1f09640 1 --2- 172.21.15.26:0/1060695531 >> v2:172.21.15.160:3300/0 conn(0x7fa6ec15ada0 0x7fa6ec0b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f1f09640 1 -- 172.21.15.26:0/1060695531 >> v1:172.21.15.26:6789/0 conn(0x7fa6ec06ac80 legacy=0x7fa6ec0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:46.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f0f07640 1 -- 172.21.15.26:0/1060695531 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446458685 0 0) 0x7fa6ec0b46c0 con 0x7fa6ec06ac80 2023-12-19T16:43:46.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f1f09640 1 -- 172.21.15.26:0/1060695531 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec071070 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f1f09640 1 --2- 172.21.15.26:0/1060695531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ec15a9c0 0x7fa6ec0706d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec070160 tx=0x7fa6e801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f9a626dc326efd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:46.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f0f07640 1 -- 172.21.15.26:0/1060695531 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80ac600 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f0f07640 1 -- 172.21.15.26:0/1060695531 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e80a8050 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f0f07640 1 -- 172.21.15.26:0/1060695531 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80ac980 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f416d640 1 -- 172.21.15.26:0/1060695531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ec15a9c0 msgr2=0x7fa6ec0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:46.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f416d640 1 --2- 172.21.15.26:0/1060695531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ec15a9c0 0x7fa6ec0706d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec070160 tx=0x7fa6e801c3a0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f416d640 1 -- 172.21.15.26:0/1060695531 shutdown_connections 2023-12-19T16:43:46.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f416d640 1 --2- 172.21.15.26:0/1060695531 >> v2:172.21.15.160:3300/0 conn(0x7fa6ec15ada0 0x7fa6ec0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f416d640 1 --2- 172.21.15.26:0/1060695531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ec15a9c0 0x7fa6ec0706d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.870+0000 7fa6f416d640 1 -- 172.21.15.26:0/1060695531 >> 172.21.15.26:0/1060695531 conn(0x7fa6ec0ce730 msgr2=0x7fa6ec0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:46.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f416d640 1 -- 172.21.15.26:0/1060695531 shutdown_connections 2023-12-19T16:43:46.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f416d640 1 -- 172.21.15.26:0/1060695531 wait complete. 2023-12-19T16:43:46.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f416d640 1 Processor -- start 2023-12-19T16:43:46.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f416d640 1 -- start start 2023-12-19T16:43:46.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f416d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6ec06ac80 0x7fa6ec0c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:46.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f416d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ec15a9c0 0x7fa6ec0c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:46.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f416d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa6ec15ada0 0x7fa6ec156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:46.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f416d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0c6b40 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f416d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0b2cd0 con 0x7fa6ec06ac80 2023-12-19T16:43:46.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f416d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec0b2f40 con 0x7fa6ec15ada0 2023-12-19T16:43:46.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f1708640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ec15a9c0 0x7fa6ec0c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:46.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f1708640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ec15a9c0 0x7fa6ec0c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59548/0 (socket says 172.21.15.26:59548) 2023-12-19T16:43:46.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f1708640 1 -- 172.21.15.26:0/1876068229 learned_addr learned my addr 172.21.15.26:0/1876068229 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:46.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f1f09640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6ec06ac80 0x7fa6ec0c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:46.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f270a640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa6ec15ada0 0x7fa6ec156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:46.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f1708640 1 -- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa6ec15ada0 msgr2=0x7fa6ec156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:46.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f1708640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa6ec15ada0 0x7fa6ec156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f1708640 1 -- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6ec06ac80 msgr2=0x7fa6ec0c2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:46.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f1708640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6ec06ac80 0x7fa6ec0c2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:46.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f1708640 1 -- 172.21.15.26:0/1876068229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e8009420 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f270a640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa6ec15ada0 0x7fa6ec156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:46.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f1708640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ec15a9c0 0x7fa6ec0c6180 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00e8a0 tx=0x7fa6dc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:46.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6e27fc640 1 -- 172.21.15.26:0/1876068229 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc009510 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6e27fc640 1 -- 172.21.15.26:0/1876068229 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6dc00c940 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6f416d640 1 -- 172.21.15.26:0/1876068229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec0b3220 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.874+0000 7fa6e27fc640 1 -- 172.21.15.26:0/1876068229 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc00ccb0 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.878+0000 7fa6f416d640 1 -- 172.21.15.26:0/1876068229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec0c66c0 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.878+0000 7fa6f416d640 1 -- 172.21.15.26:0/1876068229 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec074270 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.878+0000 7fa6e27fc640 1 -- 172.21.15.26:0/1876068229 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7fa6dc013470 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.878+0000 7fa6e27fc640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa6d0065c60 0x7fa6d0068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:46.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.878+0000 7fa6e27fc640 1 -- 172.21.15.26:0/1876068229 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa6dc083730 con 0x7fa6ec15a9c0 2023-12-19T16:43:46.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.878+0000 7fa6f1f09640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa6d0065c60 0x7fa6d0068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:46.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.878+0000 7fa6f1f09640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa6d0065c60 0x7fa6d0068120 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80ae000 tx=0x7fa6e80a7000 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:46.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:46.882+0000 7fa6e27fc640 1 -- 172.21.15.26:0/1876068229 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa6dc050d60 con 0x7fa6ec15a9c0 2023-12-19T16:43:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.058+0000 7fa6f416d640 1 -- 172.21.15.26:0/1876068229 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6ec0b08c0 con 0x7fa6d0065c60 2023-12-19T16:43:47.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.122+0000 7fa6e27fc640 1 -- 172.21.15.26:0/1876068229 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa6e80043e0 con 0x7fa6d0065c60 2023-12-19T16:43:47.128 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path /snap_test_dir1/dir8 2023-12-19T16:43:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.122+0000 7fa6f416d640 1 -- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa6d0065c60 msgr2=0x7fa6d0068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.122+0000 7fa6f416d640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa6d0065c60 0x7fa6d0068120 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80ae000 tx=0x7fa6e80a7000 comp rx=0 tx=0).stop 2023-12-19T16:43:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.122+0000 7fa6f416d640 1 -- 172.21.15.26:0/1876068229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ec15a9c0 msgr2=0x7fa6ec0c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.122+0000 7fa6f416d640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ec15a9c0 0x7fa6ec0c6180 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00e8a0 tx=0x7fa6dc00eca0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.122+0000 7fa6f416d640 1 -- 172.21.15.26:0/1876068229 shutdown_connections 2023-12-19T16:43:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.122+0000 7fa6f416d640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa6d0065c60 0x7fa6d0068120 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.122+0000 7fa6f416d640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa6ec15ada0 0x7fa6ec156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.122+0000 7fa6f416d640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6ec15a9c0 0x7fa6ec0c6180 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.122+0000 7fa6f416d640 1 --2- 172.21.15.26:0/1876068229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6ec06ac80 0x7fa6ec0c2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.122+0000 7fa6f416d640 1 -- 172.21.15.26:0/1876068229 >> 172.21.15.26:0/1876068229 conn(0x7fa6ec0ce730 msgr2=0x7fa6ec06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.122+0000 7fa6f416d640 1 -- 172.21.15.26:0/1876068229 shutdown_connections 2023-12-19T16:43:47.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.122+0000 7fa6f416d640 1 -- 172.21.15.26:0/1876068229 wait complete. 2023-12-19T16:43:47.149 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-12-19T16:43:47.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:43:47.149 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-12-19T16:43:47.160 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir9 1M 2023-12-19T16:43:47.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4688152640 1 Processor -- start 2023-12-19T16:43:47.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4688152640 1 -- start start 2023-12-19T16:43:47.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4688152640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a9c0 0x7f46800706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4688152640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4680070d90 con 0x7f468006ac80 2023-12-19T16:43:47.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4688152640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4680071070 con 0x7f468015ada0 2023-12-19T16:43:47.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4688152640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800cc270 con 0x7f468015a9c0 2023-12-19T16:43:47.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f46866ef640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f468006ac80 0x7f46800b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32922/0 (socket says 172.21.15.26:32922) 2023-12-19T16:43:47.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f46866ef640 1 -- 172.21.15.26:0/3375943853 learned_addr learned my addr 172.21.15.26:0/3375943853 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4685eee640 1 --2- 172.21.15.26:0/3375943853 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a9c0 0x7f46800706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643830844 0 0) 0x7f4680070d90 con 0x7f468006ac80 2023-12-19T16:43:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f466c003530 con 0x7f468006ac80 2023-12-19T16:43:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2403491062 0 0) 0x7f466c003530 con 0x7f468006ac80 2023-12-19T16:43:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a9c0 msgr2=0x7f46800706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 --2- 172.21.15.26:0/3375943853 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a9c0 0x7f46800706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 >> v1:172.21.15.160:6789/0 conn(0x7f468015ada0 legacy=0x7f46800b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4680071350 con 0x7f468006ac80 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f4678001a00 con 0x7f468006ac80 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 >> v1:172.21.15.26:6789/0 conn(0x7f468006ac80 legacy=0x7f46800b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 --2- 172.21.15.26:0/3375943853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466c0039b0 0x7f466c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 --2- 172.21.15.26:0/3375943853 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a9c0 0x7f466c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 --2- 172.21.15.26:0/3375943853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f466c004d10 0x7f466c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46800cc270 con 0x7f466c0039b0 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c00bac0 con 0x7f466c004d10 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c00bc80 con 0x7f468015a9c0 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f4685eee640 1 --2- 172.21.15.26:0/3375943853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466c0039b0 0x7f466c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f46856ed640 1 --2- 172.21.15.26:0/3375943853 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a9c0 0x7f466c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.342+0000 7f46866ef640 1 --2- 172.21.15.26:0/3375943853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f466c004d10 0x7f466c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f4685eee640 1 -- 172.21.15.26:0/3375943853 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a9c0 msgr2=0x7f466c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:47.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f4685eee640 1 --2- 172.21.15.26:0/3375943853 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a9c0 0x7f466c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f4685eee640 1 -- 172.21.15.26:0/3375943853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f466c004d10 msgr2=0x7f466c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:47.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f4685eee640 1 --2- 172.21.15.26:0/3375943853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f466c004d10 0x7f466c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f4685eee640 1 -- 172.21.15.26:0/3375943853 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c003530 con 0x7f466c0039b0 2023-12-19T16:43:47.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f46856ed640 1 --2- 172.21.15.26:0/3375943853 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f468015a9c0 0x7f466c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:47.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f4685eee640 1 --2- 172.21.15.26:0/3375943853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466c0039b0 0x7f466c003dd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f467c009ae0 tx=0x7f467c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f8b14dd4a039c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:47.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0a5550 con 0x7f466c0039b0 2023-12-19T16:43:47.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c0a56b0 con 0x7f466c0039b0 2023-12-19T16:43:47.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f4684eec640 1 -- 172.21.15.26:0/3375943853 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0a5a60 con 0x7f466c0039b0 2023-12-19T16:43:47.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f4688152640 1 -- 172.21.15.26:0/3375943853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466c0039b0 msgr2=0x7f466c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f4688152640 1 --2- 172.21.15.26:0/3375943853 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f466c0039b0 0x7f466c003dd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f467c009ae0 tx=0x7f467c01c960 comp rx=0 tx=0).stop 2023-12-19T16:43:47.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f46866ef640 1 -- 172.21.15.26:0/3375943853 reap_dead start 2023-12-19T16:43:47.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f4688152640 1 -- 172.21.15.26:0/3375943853 shutdown_connections 2023-12-19T16:43:47.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.346+0000 7f4688152640 1 -- 172.21.15.26:0/3375943853 >> 172.21.15.26:0/3375943853 conn(0x7f46800ce730 msgr2=0x7f46800ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:47.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f4688152640 1 -- 172.21.15.26:0/3375943853 shutdown_connections 2023-12-19T16:43:47.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f4688152640 1 -- 172.21.15.26:0/3375943853 wait complete. 2023-12-19T16:43:47.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f4688152640 1 Processor -- start 2023-12-19T16:43:47.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f4688152640 1 -- start start 2023-12-19T16:43:47.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f4688152640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f466c0039b0 0x7f468014f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f4688152640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468014f950 0x7f468015d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f4688152640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468015df30 0x7f468015b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f4688152640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468015e3b0 con 0x7f468015df30 2023-12-19T16:43:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f4688152640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46800c6df0 con 0x7f468014f950 2023-12-19T16:43:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f4688152640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800c6220 con 0x7f466c0039b0 2023-12-19T16:43:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f46866ef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468015df30 0x7f468015b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f46866ef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468015df30 0x7f468015b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59560/0 (socket says 172.21.15.26:59560) 2023-12-19T16:43:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f46866ef640 1 -- 172.21.15.26:0/1056634210 learned_addr learned my addr 172.21.15.26:0/1056634210 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f46856ed640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468014f950 0x7f468015d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f4685eee640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f466c0039b0 0x7f468014f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f46866ef640 1 -- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f466c0039b0 msgr2=0x7f468014f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f46866ef640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f466c0039b0 0x7f468014f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f46866ef640 1 -- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468014f950 msgr2=0x7f468015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f46866ef640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468014f950 0x7f468015d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f46866ef640 1 -- 172.21.15.26:0/1056634210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800c64c0 con 0x7f468015df30 2023-12-19T16:43:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f4685eee640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f466c0039b0 0x7f468014f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f46856ed640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468014f950 0x7f468015d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.350+0000 7f46866ef640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468015df30 0x7f468015b450 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4680070160 tx=0x7f467800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.354+0000 7f46767fc640 1 -- 172.21.15.26:0/1056634210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467800eb30 con 0x7f468015df30 2023-12-19T16:43:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.354+0000 7f46767fc640 1 -- 172.21.15.26:0/1056634210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467800ec90 con 0x7f468015df30 2023-12-19T16:43:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.354+0000 7f46767fc640 1 -- 172.21.15.26:0/1056634210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678013420 con 0x7f468015df30 2023-12-19T16:43:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.354+0000 7f4688152640 1 -- 172.21.15.26:0/1056634210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46800c67a0 con 0x7f468015df30 2023-12-19T16:43:47.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.354+0000 7f4688152640 1 -- 172.21.15.26:0/1056634210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468014f6e0 con 0x7f468015df30 2023-12-19T16:43:47.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.354+0000 7f4688152640 1 -- 172.21.15.26:0/1056634210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4680074270 con 0x7f468015df30 2023-12-19T16:43:47.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.354+0000 7f46767fc640 1 -- 172.21.15.26:0/1056634210 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f4678013580 con 0x7f468015df30 2023-12-19T16:43:47.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.354+0000 7f46767fc640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4664065c60 0x7f4664068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.354+0000 7f46767fc640 1 -- 172.21.15.26:0/1056634210 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f4678083aa0 con 0x7f468015df30 2023-12-19T16:43:47.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.354+0000 7f4685eee640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4664065c60 0x7f4664068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.358+0000 7f4685eee640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4664065c60 0x7f4664068120 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f467c0af000 tx=0x7f467c0046d0 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:47.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.358+0000 7f46767fc640 1 -- 172.21.15.26:0/1056634210 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4678051040 con 0x7f468015df30 2023-12-19T16:43:47.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.538+0000 7f4688152640 1 -- 172.21.15.26:0/1056634210 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f46800706d0 con 0x7f4664065c60 2023-12-19T16:43:47.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.602+0000 7f46767fc640 1 -- 172.21.15.26:0/1056634210 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f467c004040 con 0x7f4664065c60 2023-12-19T16:43:47.609 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path /snap_test_dir1/dir9 2023-12-19T16:43:47.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.606+0000 7f4688152640 1 -- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4664065c60 msgr2=0x7f4664068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.606+0000 7f4688152640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4664065c60 0x7f4664068120 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f467c0af000 tx=0x7f467c0046d0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.606+0000 7f4688152640 1 -- 172.21.15.26:0/1056634210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468015df30 msgr2=0x7f468015b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.606+0000 7f4688152640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468015df30 0x7f468015b450 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4680070160 tx=0x7f467800e530 comp rx=0 tx=0).stop 2023-12-19T16:43:47.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.606+0000 7f4688152640 1 -- 172.21.15.26:0/1056634210 shutdown_connections 2023-12-19T16:43:47.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.606+0000 7f4688152640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468015df30 0x7f468015b450 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.606+0000 7f4688152640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f468014f950 0x7f468015d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.606+0000 7f4688152640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4664065c60 0x7f4664068120 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.606+0000 7f4688152640 1 --2- 172.21.15.26:0/1056634210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f466c0039b0 0x7f468014f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.606+0000 7f4688152640 1 -- 172.21.15.26:0/1056634210 >> 172.21.15.26:0/1056634210 conn(0x7f46800ce730 msgr2=0x7f46800ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:47.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.606+0000 7f4688152640 1 -- 172.21.15.26:0/1056634210 shutdown_connections 2023-12-19T16:43:47.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.606+0000 7f4688152640 1 -- 172.21.15.26:0/1056634210 wait complete. 2023-12-19T16:43:47.630 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-12-19T16:43:47.630 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir0 1M 2023-12-19T16:43:47.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10dad98640 1 Processor -- start 2023-12-19T16:43:47.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10dad98640 1 -- start start 2023-12-19T16:43:47.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10dad98640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f10d4159820 0x7f10d415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10dad98640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d406fb60 con 0x7f10d406f6c0 2023-12-19T16:43:47.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10dad98640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d406fe40 con 0x7f10d4159c00 2023-12-19T16:43:47.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10dad98640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d415e470 con 0x7f10d4159820 2023-12-19T16:43:47.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f10d406f6c0 0x7f10d40c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32934/0 (socket says 172.21.15.26:32934) 2023-12-19T16:43:47.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d3fff640 1 -- 172.21.15.26:0/1411498758 learned_addr learned my addr 172.21.15.26:0/1411498758 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d8b34640 1 --2- 172.21.15.26:0/1411498758 >> v2:172.21.15.160:3300/0 conn(0x7f10d4159820 0x7f10d415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177366785 0 0) 0x7f10d406fb60 con 0x7f10d406f6c0 2023-12-19T16:43:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10c4003530 con 0x7f10d406f6c0 2023-12-19T16:43:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 16094934 0 0) 0x7f10c4003530 con 0x7f10d406f6c0 2023-12-19T16:43:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 >> v2:172.21.15.160:3300/0 conn(0x7f10d4159820 msgr2=0x7f10d415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d37fe640 1 --2- 172.21.15.26:0/1411498758 >> v2:172.21.15.160:3300/0 conn(0x7f10d4159820 0x7f10d415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 >> v1:172.21.15.160:6789/0 conn(0x7f10d4159c00 legacy=0x7f10d4070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d4070120 con 0x7f10d406f6c0 2023-12-19T16:43:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f10c0001680 con 0x7f10d406f6c0 2023-12-19T16:43:47.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 >> v1:172.21.15.26:6789/0 conn(0x7f10d406f6c0 legacy=0x7f10d40c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d37fe640 1 --2- 172.21.15.26:0/1411498758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10c4003980 0x7f10c4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.770+0000 7f10d37fe640 1 --2- 172.21.15.26:0/1411498758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c4004380 0x7f10c4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d37fe640 1 --2- 172.21.15.26:0/1411498758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10c4005050 0x7f10c400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d415e470 con 0x7f10c4003980 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c400bfb0 con 0x7f10c4004380 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c400c170 con 0x7f10c4005050 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d8b34640 1 --2- 172.21.15.26:0/1411498758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10c4003980 0x7f10c4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d3fff640 1 --2- 172.21.15.26:0/1411498758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c4004380 0x7f10c4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d9335640 1 --2- 172.21.15.26:0/1411498758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10c4005050 0x7f10c400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d8b34640 1 -- 172.21.15.26:0/1411498758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10c4005050 msgr2=0x7f10c400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d8b34640 1 --2- 172.21.15.26:0/1411498758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10c4005050 0x7f10c400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d8b34640 1 -- 172.21.15.26:0/1411498758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c4004380 msgr2=0x7f10c4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d8b34640 1 --2- 172.21.15.26:0/1411498758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c4004380 0x7f10c4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d8b34640 1 -- 172.21.15.26:0/1411498758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c4003530 con 0x7f10c4003980 2023-12-19T16:43:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d3fff640 1 --2- 172.21.15.26:0/1411498758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c4004380 0x7f10c4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d9335640 1 -- 172.21.15.26:0/1411498758 reap_dead start 2023-12-19T16:43:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d8b34640 1 --2- 172.21.15.26:0/1411498758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10c4003980 0x7f10c4003d60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009ae0 tx=0x7f10c801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c471dac78be0ae29 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c80a5640 con 0x7f10c4003980 2023-12-19T16:43:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c80a57a0 con 0x7f10c4003980 2023-12-19T16:43:47.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.774+0000 7f10d37fe640 1 -- 172.21.15.26:0/1411498758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c80a5b50 con 0x7f10c4003980 2023-12-19T16:43:47.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 -- 172.21.15.26:0/1411498758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10c4003980 msgr2=0x7f10c4003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 --2- 172.21.15.26:0/1411498758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10c4003980 0x7f10c4003d60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009ae0 tx=0x7f10c801ca50 comp rx=0 tx=0).stop 2023-12-19T16:43:47.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 -- 172.21.15.26:0/1411498758 shutdown_connections 2023-12-19T16:43:47.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 --2- 172.21.15.26:0/1411498758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10c4003980 0x7f10c4003d60 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 -- 172.21.15.26:0/1411498758 >> 172.21.15.26:0/1411498758 conn(0x7f10d40ce730 msgr2=0x7f10d40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:47.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 -- 172.21.15.26:0/1411498758 shutdown_connections 2023-12-19T16:43:47.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 -- 172.21.15.26:0/1411498758 wait complete. 2023-12-19T16:43:47.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 Processor -- start 2023-12-19T16:43:47.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 -- start start 2023-12-19T16:43:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c4003980 0x7f10d40af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d40b0a60 0x7f10d40c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10d41ed590 0x7f10d41eda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d40afb30 con 0x7f10d40b0a60 2023-12-19T16:43:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d40aeb70 con 0x7f10c4003980 2023-12-19T16:43:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10dad98640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d41edf50 con 0x7f10d41ed590 2023-12-19T16:43:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d40b0a60 0x7f10d40c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d40b0a60 0x7f10d40c4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59570/0 (socket says 172.21.15.26:59570) 2023-12-19T16:43:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10d3fff640 1 -- 172.21.15.26:0/2754893602 learned_addr learned my addr 172.21.15.26:0/2754893602 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10d8b34640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c4003980 0x7f10d40af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.778+0000 7f10d9335640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10d41ed590 0x7f10d41eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10d3fff640 1 -- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10d41ed590 msgr2=0x7f10d41eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10d3fff640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10d41ed590 0x7f10d41eda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10d3fff640 1 -- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c4003980 msgr2=0x7f10d40af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10d3fff640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c4003980 0x7f10d40af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10d3fff640 1 -- 172.21.15.26:0/2754893602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d40c3530 con 0x7f10d40b0a60 2023-12-19T16:43:47.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10d3fff640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d40b0a60 0x7f10d40c4360 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f10c000e810 tx=0x7f10c000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10d17fa640 1 -- 172.21.15.26:0/2754893602 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c000c850 con 0x7f10d40b0a60 2023-12-19T16:43:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10d17fa640 1 -- 172.21.15.26:0/2754893602 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c000c9b0 con 0x7f10d40b0a60 2023-12-19T16:43:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10d17fa640 1 -- 172.21.15.26:0/2754893602 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c000cd20 con 0x7f10d40b0a60 2023-12-19T16:43:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10dad98640 1 -- 172.21.15.26:0/2754893602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d40c3810 con 0x7f10d40b0a60 2023-12-19T16:43:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10dad98640 1 -- 172.21.15.26:0/2754893602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d40af0d0 con 0x7f10d40b0a60 2023-12-19T16:43:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10dad98640 1 -- 172.21.15.26:0/2754893602 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d40af2f0 con 0x7f10d40b0a60 2023-12-19T16:43:47.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10d17fa640 1 -- 172.21.15.26:0/2754893602 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f10c001d460 con 0x7f10d40b0a60 2023-12-19T16:43:47.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10d17fa640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f10b8065c90 0x7f10b8068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:47.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10d17fa640 1 -- 172.21.15.26:0/2754893602 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f10c00834b0 con 0x7f10d40b0a60 2023-12-19T16:43:47.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.782+0000 7f10d8b34640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f10b8065c90 0x7f10b8068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:47.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.786+0000 7f10d8b34640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f10b8065c90 0x7f10b8068150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f10c80af000 tx=0x7f10c80046d0 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:47.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.786+0000 7f10d17fa640 1 -- 172.21.15.26:0/2754893602 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f10c0050b10 con 0x7f10d40b0a60 2023-12-19T16:43:47.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:47.966+0000 7f10dad98640 1 -- 172.21.15.26:0/2754893602 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f10d40706e0 con 0x7f10b8065c90 2023-12-19T16:43:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.018+0000 7f10d17fa640 1 -- 172.21.15.26:0/2754893602 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f10c80a5910 con 0x7f10b8065c90 2023-12-19T16:43:48.024 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-12-19T16:43:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.018+0000 7f10dad98640 1 -- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f10b8065c90 msgr2=0x7f10b8068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.018+0000 7f10dad98640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f10b8065c90 0x7f10b8068150 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f10c80af000 tx=0x7f10c80046d0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.018+0000 7f10dad98640 1 -- 172.21.15.26:0/2754893602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d40b0a60 msgr2=0x7f10d40c4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.018+0000 7f10dad98640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d40b0a60 0x7f10d40c4360 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f10c000e810 tx=0x7f10c000ecd0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.018+0000 7f10dad98640 1 -- 172.21.15.26:0/2754893602 shutdown_connections 2023-12-19T16:43:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.018+0000 7f10dad98640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f10b8065c90 0x7f10b8068150 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.018+0000 7f10dad98640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10d41ed590 0x7f10d41eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.018+0000 7f10dad98640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10d40b0a60 0x7f10d40c4360 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.018+0000 7f10dad98640 1 --2- 172.21.15.26:0/2754893602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10c4003980 0x7f10d40af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.018+0000 7f10dad98640 1 -- 172.21.15.26:0/2754893602 >> 172.21.15.26:0/2754893602 conn(0x7f10d40ce730 msgr2=0x7f10d4157f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.018+0000 7f10dad98640 1 -- 172.21.15.26:0/2754893602 shutdown_connections 2023-12-19T16:43:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.018+0000 7f10dad98640 1 -- 172.21.15.26:0/2754893602 wait complete. 2023-12-19T16:43:48.045 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-12-19T16:43:48.045 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir1 1M 2023-12-19T16:43:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8c7b3640 1 Processor -- start 2023-12-19T16:43:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8c7b3640 1 -- start start 2023-12-19T16:43:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8c7b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fae8415a9c0 0x7fae840706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8c7b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae8406ac80 0x7fae840b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8c7b3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae84070d90 con 0x7fae8415ada0 2023-12-19T16:43:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8c7b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae840cc270 con 0x7fae8415a9c0 2023-12-19T16:43:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8c7b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae84071070 con 0x7fae8406ac80 2023-12-19T16:43:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8a54f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fae8415a9c0 0x7fae840706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae89d4e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae8415ada0 0x7fae840b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32936/0 (socket says 172.21.15.26:32936) 2023-12-19T16:43:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae89d4e640 1 -- 172.21.15.26:0/2175205097 learned_addr learned my addr 172.21.15.26:0/2175205097 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8ad50640 1 --2- 172.21.15.26:0/2175205097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae8406ac80 0x7fae840b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8a54f640 1 -- 172.21.15.26:0/2175205097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae8406ac80 msgr2=0x7fae840b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8a54f640 1 --2- 172.21.15.26:0/2175205097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae8406ac80 0x7fae840b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8a54f640 1 -- 172.21.15.26:0/2175205097 >> v1:172.21.15.26:6789/0 conn(0x7fae8415ada0 legacy=0x7fae840b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8a54f640 1 -- 172.21.15.26:0/2175205097 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae84071350 con 0x7fae8415a9c0 2023-12-19T16:43:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.186+0000 7fae8954d640 1 -- 172.21.15.26:0/2175205097 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3239376556 0 0) 0x7fae84070d90 con 0x7fae8415ada0 2023-12-19T16:43:48.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.190+0000 7fae8a54f640 1 --2- 172.21.15.26:0/2175205097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8415a9c0 0x7fae840706d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fae800097c0 tx=0x7fae8001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3428b5b168bf129c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:48.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.190+0000 7fae8954d640 1 -- 172.21.15.26:0/2175205097 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae800ab600 con 0x7fae8415a9c0 2023-12-19T16:43:48.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.190+0000 7fae8954d640 1 -- 172.21.15.26:0/2175205097 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae800a7050 con 0x7fae8415a9c0 2023-12-19T16:43:48.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.190+0000 7fae8954d640 1 -- 172.21.15.26:0/2175205097 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae800ab980 con 0x7fae8415a9c0 2023-12-19T16:43:48.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.190+0000 7fae8c7b3640 1 -- 172.21.15.26:0/2175205097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8415a9c0 msgr2=0x7fae840706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.190+0000 7fae8c7b3640 1 --2- 172.21.15.26:0/2175205097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8415a9c0 0x7fae840706d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fae800097c0 tx=0x7fae8001c3c0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.190+0000 7fae8c7b3640 1 -- 172.21.15.26:0/2175205097 shutdown_connections 2023-12-19T16:43:48.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.190+0000 7fae8c7b3640 1 --2- 172.21.15.26:0/2175205097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae8406ac80 0x7fae840b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.190+0000 7fae8c7b3640 1 --2- 172.21.15.26:0/2175205097 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8415a9c0 0x7fae840706d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.190+0000 7fae8c7b3640 1 -- 172.21.15.26:0/2175205097 >> 172.21.15.26:0/2175205097 conn(0x7fae840ce730 msgr2=0x7fae840ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:48.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.190+0000 7fae8c7b3640 1 -- 172.21.15.26:0/2175205097 shutdown_connections 2023-12-19T16:43:48.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.190+0000 7fae8c7b3640 1 -- 172.21.15.26:0/2175205097 wait complete. 2023-12-19T16:43:48.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae8c7b3640 1 Processor -- start 2023-12-19T16:43:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae8c7b3640 1 -- start start 2023-12-19T16:43:48.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae8c7b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae8406ac80 0x7fae840c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:48.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae8c7b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8415a9c0 0x7fae84152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae8c7b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae8415ada0 0x7fae84149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae8c7b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae840c5770 con 0x7fae8415a9c0 2023-12-19T16:43:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae8c7b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae84152dc0 con 0x7fae8415ada0 2023-12-19T16:43:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae8c7b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8414a360 con 0x7fae8406ac80 2023-12-19T16:43:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae89d4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8415a9c0 0x7fae84152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae89d4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8415a9c0 0x7fae84152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59592/0 (socket says 172.21.15.26:59592) 2023-12-19T16:43:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae89d4e640 1 -- 172.21.15.26:0/1391038679 learned_addr learned my addr 172.21.15.26:0/1391038679 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae8a54f640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae8406ac80 0x7fae840c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae8ad50640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae8415ada0 0x7fae84149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae89d4e640 1 -- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae8406ac80 msgr2=0x7fae840c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae89d4e640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae8406ac80 0x7fae840c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae89d4e640 1 -- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae8415ada0 msgr2=0x7fae84149e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae89d4e640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae8415ada0 0x7fae84149e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae89d4e640 1 -- 172.21.15.26:0/1391038679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80009420 con 0x7fae8415a9c0 2023-12-19T16:43:48.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae89d4e640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8415a9c0 0x7fae84152880 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fae740011f0 tx=0x7fae7400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:48.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae7affd640 1 -- 172.21.15.26:0/1391038679 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae740074e0 con 0x7fae8415a9c0 2023-12-19T16:43:48.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae7affd640 1 -- 172.21.15.26:0/1391038679 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae74007640 con 0x7fae8415a9c0 2023-12-19T16:43:48.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae7affd640 1 -- 172.21.15.26:0/1391038679 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74004a10 con 0x7fae8415a9c0 2023-12-19T16:43:48.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae8c7b3640 1 -- 172.21.15.26:0/1391038679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae840c8e70 con 0x7fae8415a9c0 2023-12-19T16:43:48.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.194+0000 7fae8c7b3640 1 -- 172.21.15.26:0/1391038679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae840c9430 con 0x7fae8415a9c0 2023-12-19T16:43:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.198+0000 7fae8c7b3640 1 -- 172.21.15.26:0/1391038679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8406fd60 con 0x7fae8415a9c0 2023-12-19T16:43:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.198+0000 7fae7affd640 1 -- 172.21.15.26:0/1391038679 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7fae74004b70 con 0x7fae8415a9c0 2023-12-19T16:43:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.198+0000 7fae7affd640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae68065c60 0x7fae68068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.198+0000 7fae7affd640 1 -- 172.21.15.26:0/1391038679 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fae74082890 con 0x7fae8415a9c0 2023-12-19T16:43:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.198+0000 7fae8a54f640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae68065c60 0x7fae68068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:48.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.198+0000 7fae8a54f640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae68065c60 0x7fae68068120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae800ad000 tx=0x7fae800a6000 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.202+0000 7fae7affd640 1 -- 172.21.15.26:0/1391038679 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fae7404ff60 con 0x7fae8415a9c0 2023-12-19T16:43:48.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.378+0000 7fae8c7b3640 1 -- 172.21.15.26:0/1391038679 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fae84167ac0 con 0x7fae68065c60 2023-12-19T16:43:48.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.430+0000 7fae7affd640 1 -- 172.21.15.26:0/1391038679 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fae800bc0c0 con 0x7fae68065c60 2023-12-19T16:43:48.436 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-12-19T16:43:48.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.430+0000 7fae8c7b3640 1 -- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae68065c60 msgr2=0x7fae68068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.430+0000 7fae8c7b3640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae68065c60 0x7fae68068120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae800ad000 tx=0x7fae800a6000 comp rx=0 tx=0).stop 2023-12-19T16:43:48.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.430+0000 7fae8c7b3640 1 -- 172.21.15.26:0/1391038679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8415a9c0 msgr2=0x7fae84152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.430+0000 7fae8c7b3640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8415a9c0 0x7fae84152880 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fae740011f0 tx=0x7fae7400c9b0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.430+0000 7fae8c7b3640 1 -- 172.21.15.26:0/1391038679 shutdown_connections 2023-12-19T16:43:48.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.430+0000 7fae8c7b3640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae8415ada0 0x7fae84149e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.430+0000 7fae8c7b3640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae8415a9c0 0x7fae84152880 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.430+0000 7fae8c7b3640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae68065c60 0x7fae68068120 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.430+0000 7fae8c7b3640 1 --2- 172.21.15.26:0/1391038679 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae8406ac80 0x7fae840c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.430+0000 7fae8c7b3640 1 -- 172.21.15.26:0/1391038679 >> 172.21.15.26:0/1391038679 conn(0x7fae840ce730 msgr2=0x7fae840b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:48.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.430+0000 7fae8c7b3640 1 -- 172.21.15.26:0/1391038679 shutdown_connections 2023-12-19T16:43:48.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.430+0000 7fae8c7b3640 1 -- 172.21.15.26:0/1391038679 wait complete. 2023-12-19T16:43:48.458 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-12-19T16:43:48.458 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir2 1M 2023-12-19T16:43:48.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993ebbc640 1 Processor -- start 2023-12-19T16:43:48.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993ebbc640 1 -- start start 2023-12-19T16:43:48.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993ebbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9938159140 0x7f99380ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993ebbc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f993806b390 0x7f993806f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993ebbc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993806f490 con 0x7f99380cd190 2023-12-19T16:43:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993ebbc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9938159520 con 0x7f993806b390 2023-12-19T16:43:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993ebbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99380cd690 con 0x7f9938159140 2023-12-19T16:43:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993d159640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f993806b390 0x7f993806f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993d159640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f993806b390 0x7f993806f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59606/0 (socket says 172.21.15.26:59606) 2023-12-19T16:43:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993d159640 1 -- 172.21.15.26:0/2967894427 learned_addr learned my addr 172.21.15.26:0/2967894427 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993c958640 1 --2- 172.21.15.26:0/2967894427 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9938159140 0x7f99380ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993d159640 1 -- 172.21.15.26:0/2967894427 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9938159140 msgr2=0x7f99380ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993d159640 1 --2- 172.21.15.26:0/2967894427 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9938159140 0x7f99380ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.598+0000 7f993d159640 1 -- 172.21.15.26:0/2967894427 >> v1:172.21.15.160:6789/0 conn(0x7f99380cd190 legacy=0x7f993806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f993d159640 1 -- 172.21.15.26:0/2967894427 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99380cd970 con 0x7f993806b390 2023-12-19T16:43:48.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f993c958640 1 --2- 172.21.15.26:0/2967894427 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9938159140 0x7f99380ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:43:48.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f993d159640 1 --2- 172.21.15.26:0/2967894427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993806b390 0x7f993806f920 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9930009e50 tx=0x7f993001daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f2edd19dedb9db server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:48.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f99377fe640 1 -- 172.21.15.26:0/2967894427 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99300a6930 con 0x7f993806b390 2023-12-19T16:43:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f99377fe640 1 -- 172.21.15.26:0/2967894427 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99300a6a90 con 0x7f993806b390 2023-12-19T16:43:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f99377fe640 1 -- 172.21.15.26:0/2967894427 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99300a6e20 con 0x7f993806b390 2023-12-19T16:43:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f993ebbc640 1 -- 172.21.15.26:0/2967894427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993806b390 msgr2=0x7f993806f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f993ebbc640 1 --2- 172.21.15.26:0/2967894427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993806b390 0x7f993806f920 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9930009e50 tx=0x7f993001daf0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f993ebbc640 1 -- 172.21.15.26:0/2967894427 shutdown_connections 2023-12-19T16:43:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f993ebbc640 1 --2- 172.21.15.26:0/2967894427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f993806b390 0x7f993806f920 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f993ebbc640 1 --2- 172.21.15.26:0/2967894427 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9938159140 0x7f99380ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f993ebbc640 1 -- 172.21.15.26:0/2967894427 >> 172.21.15.26:0/2967894427 conn(0x7f99380ce730 msgr2=0x7f99380ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f993ebbc640 1 -- 172.21.15.26:0/2967894427 shutdown_connections 2023-12-19T16:43:48.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.602+0000 7f993ebbc640 1 -- 172.21.15.26:0/2967894427 wait complete. 2023-12-19T16:43:48.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993ebbc640 1 Processor -- start 2023-12-19T16:43:48.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993ebbc640 1 -- start start 2023-12-19T16:43:48.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993ebbc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f993806b390 0x7f993814b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:48.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993ebbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99380cd190 0x7f993814d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:48.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993ebbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9938159140 0x7f99380c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:48.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993ebbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993814d8b0 con 0x7f9938159140 2023-12-19T16:43:48.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993ebbc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99380c8990 con 0x7f993806b390 2023-12-19T16:43:48.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993ebbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99380b2eb0 con 0x7f99380cd190 2023-12-19T16:43:48.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993d159640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9938159140 0x7f99380c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:48.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993d159640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9938159140 0x7f99380c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59608/0 (socket says 172.21.15.26:59608) 2023-12-19T16:43:48.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993d159640 1 -- 172.21.15.26:0/2572954501 learned_addr learned my addr 172.21.15.26:0/2572954501 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:48.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993d159640 1 -- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99380cd190 msgr2=0x7f993814d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993c958640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f993806b390 0x7f993814b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:48.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f9937fff640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99380cd190 0x7f993814d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:48.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993d159640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99380cd190 0x7f993814d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993d159640 1 -- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f993806b390 msgr2=0x7f993814b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993d159640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f993806b390 0x7f993814b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993d159640 1 -- 172.21.15.26:0/2572954501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930009b30 con 0x7f9938159140 2023-12-19T16:43:48.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993c958640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f993806b390 0x7f993814b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:48.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f9937fff640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99380cd190 0x7f993814d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:43:48.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993d159640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9938159140 0x7f99380c8450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f993001b720 tx=0x7f993001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:48.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f99357fa640 1 -- 172.21.15.26:0/2572954501 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930009250 con 0x7f9938159140 2023-12-19T16:43:48.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f99357fa640 1 -- 172.21.15.26:0/2572954501 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99300b8610 con 0x7f9938159140 2023-12-19T16:43:48.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f99357fa640 1 -- 172.21.15.26:0/2572954501 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99300b8a00 con 0x7f9938159140 2023-12-19T16:43:48.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.606+0000 7f993ebbc640 1 -- 172.21.15.26:0/2572954501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993814bd00 con 0x7f9938159140 2023-12-19T16:43:48.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.610+0000 7f993ebbc640 1 -- 172.21.15.26:0/2572954501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993814c2c0 con 0x7f9938159140 2023-12-19T16:43:48.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.610+0000 7f993ebbc640 1 -- 172.21.15.26:0/2572954501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99380c8b20 con 0x7f9938159140 2023-12-19T16:43:48.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.610+0000 7f99357fa640 1 -- 172.21.15.26:0/2572954501 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f99300b8b60 con 0x7f9938159140 2023-12-19T16:43:48.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.610+0000 7f99357fa640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f991c065c60 0x7f991c068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:48.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.610+0000 7f99357fa640 1 -- 172.21.15.26:0/2572954501 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f993011dfd0 con 0x7f9938159140 2023-12-19T16:43:48.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.610+0000 7f993c958640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f991c065c60 0x7f991c068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:48.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.614+0000 7f993c958640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f991c065c60 0x7f991c068120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f992c0097a0 tx=0x7f992c008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:48.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.614+0000 7f99357fa640 1 -- 172.21.15.26:0/2572954501 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f99300eb600 con 0x7f9938159140 2023-12-19T16:43:48.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.794+0000 7f993ebbc640 1 -- 172.21.15.26:0/2572954501 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f993806fe60 con 0x7f991c065c60 2023-12-19T16:43:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.846+0000 7f99357fa640 1 -- 172.21.15.26:0/2572954501 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f992c009510 con 0x7f991c065c60 2023-12-19T16:43:48.851 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-12-19T16:43:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.846+0000 7f993ebbc640 1 -- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f991c065c60 msgr2=0x7f991c068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.846+0000 7f993ebbc640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f991c065c60 0x7f991c068120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f992c0097a0 tx=0x7f992c008040 comp rx=0 tx=0).stop 2023-12-19T16:43:48.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.846+0000 7f993ebbc640 1 -- 172.21.15.26:0/2572954501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9938159140 msgr2=0x7f99380c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:48.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.846+0000 7f993ebbc640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9938159140 0x7f99380c8450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f993001b720 tx=0x7f993001c330 comp rx=0 tx=0).stop 2023-12-19T16:43:48.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.846+0000 7f993ebbc640 1 -- 172.21.15.26:0/2572954501 shutdown_connections 2023-12-19T16:43:48.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.846+0000 7f993ebbc640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9938159140 0x7f99380c8450 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.846+0000 7f993ebbc640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f991c065c60 0x7f991c068120 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.846+0000 7f993ebbc640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99380cd190 0x7f993814d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.846+0000 7f993ebbc640 1 --2- 172.21.15.26:0/2572954501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f993806b390 0x7f993814b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:48.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.846+0000 7f993ebbc640 1 -- 172.21.15.26:0/2572954501 >> 172.21.15.26:0/2572954501 conn(0x7f99380ce730 msgr2=0x7f99380c9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:48.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.846+0000 7f993ebbc640 1 -- 172.21.15.26:0/2572954501 shutdown_connections 2023-12-19T16:43:48.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:48.846+0000 7f993ebbc640 1 -- 172.21.15.26:0/2572954501 wait complete. 2023-12-19T16:43:48.874 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-12-19T16:43:48.874 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir3 1M 2023-12-19T16:43:49.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1210a66640 1 Processor -- start 2023-12-19T16:43:49.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1210a66640 1 -- start start 2023-12-19T16:43:49.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1210a66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f120c159820 0x7f120c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1210a66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120c06fb60 con 0x7f120c159c00 2023-12-19T16:43:49.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1210a66640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120c06fe40 con 0x7f120c06f6c0 2023-12-19T16:43:49.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1210a66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120c15e470 con 0x7f120c159820 2023-12-19T16:43:49.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f120ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f120c159c00 0x7f120c070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32944/0 (socket says 172.21.15.26:32944) 2023-12-19T16:43:49.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f120ad9d640 1 -- 172.21.15.26:0/1954900800 learned_addr learned my addr 172.21.15.26:0/1954900800 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:49.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f120a59c640 1 --2- 172.21.15.26:0/1954900800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f120c159820 0x7f120c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217061501 0 0) 0x7f120c06fb60 con 0x7f120c159c00 2023-12-19T16:43:49.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11f8003530 con 0x7f120c159c00 2023-12-19T16:43:49.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3031842301 0 0) 0x7f120c06fe40 con 0x7f120c06f6c0 2023-12-19T16:43:49.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f120c06fb60 con 0x7f120c06f6c0 2023-12-19T16:43:49.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2809223679 0 0) 0x7f11f8003530 con 0x7f120c159c00 2023-12-19T16:43:49.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f120c159820 msgr2=0x7f120c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:49.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1209d9b640 1 --2- 172.21.15.26:0/1954900800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f120c159820 0x7f120c15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 >> v1:172.21.15.160:6789/0 conn(0x7f120c06f6c0 legacy=0x7f120c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c070120 con 0x7f120c159c00 2023-12-19T16:43:49.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f1204001a00 con 0x7f120c159c00 2023-12-19T16:43:49.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 >> v1:172.21.15.26:6789/0 conn(0x7f120c159c00 legacy=0x7f120c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.014+0000 7f1209d9b640 1 --2- 172.21.15.26:0/1954900800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11f8003b40 0x7f11f8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f1209d9b640 1 --2- 172.21.15.26:0/1954900800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f120c159820 0x7f11f80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f1209d9b640 1 --2- 172.21.15.26:0/1954900800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11f8004d10 0x7f11f8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c15e470 con 0x7f11f8004d10 2023-12-19T16:43:49.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f800dc50 con 0x7f11f8003b40 2023-12-19T16:43:49.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f800ddc0 con 0x7f120c159820 2023-12-19T16:43:49.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f120ad9d640 1 --2- 172.21.15.26:0/1954900800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11f8004d10 0x7f11f8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f120a59c640 1 --2- 172.21.15.26:0/1954900800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11f8003b40 0x7f11f8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f1203fff640 1 --2- 172.21.15.26:0/1954900800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f120c159820 0x7f11f80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f120ad9d640 1 -- 172.21.15.26:0/1954900800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f120c159820 msgr2=0x7f11f80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:49.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f120ad9d640 1 --2- 172.21.15.26:0/1954900800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f120c159820 0x7f11f80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f120ad9d640 1 -- 172.21.15.26:0/1954900800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11f8003b40 msgr2=0x7f11f8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:49.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f120ad9d640 1 --2- 172.21.15.26:0/1954900800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11f8003b40 0x7f11f8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f120ad9d640 1 -- 172.21.15.26:0/1954900800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f8003530 con 0x7f11f8004d10 2023-12-19T16:43:49.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f120ad9d640 1 --2- 172.21.15.26:0/1954900800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11f8004d10 0x7f11f8005190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f120400a7a0 tx=0x7f1204023e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2604b7b0675cd32b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:49.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12040ad830 con 0x7f11f8004d10 2023-12-19T16:43:49.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12040ad9b0 con 0x7f11f8004d10 2023-12-19T16:43:49.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.018+0000 7f1209d9b640 1 -- 172.21.15.26:0/1954900800 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12040add60 con 0x7f11f8004d10 2023-12-19T16:43:49.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 -- 172.21.15.26:0/1954900800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11f8004d10 msgr2=0x7f11f8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 --2- 172.21.15.26:0/1954900800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11f8004d10 0x7f11f8005190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f120400a7a0 tx=0x7f1204023e10 comp rx=0 tx=0).stop 2023-12-19T16:43:49.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f120ad9d640 1 -- 172.21.15.26:0/1954900800 reap_dead start 2023-12-19T16:43:49.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 -- 172.21.15.26:0/1954900800 shutdown_connections 2023-12-19T16:43:49.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 -- 172.21.15.26:0/1954900800 >> 172.21.15.26:0/1954900800 conn(0x7f120c0ce730 msgr2=0x7f120c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:49.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 -- 172.21.15.26:0/1954900800 shutdown_connections 2023-12-19T16:43:49.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 -- 172.21.15.26:0/1954900800 wait complete. 2023-12-19T16:43:49.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 Processor -- start 2023-12-19T16:43:49.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 -- start start 2023-12-19T16:43:49.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11f8004d10 0x7f120c14c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f120c14cb10 0x7f120c15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c15a5f0 0x7f120c15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c15bdc0 con 0x7f120c15a5f0 2023-12-19T16:43:49.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0c2a40 con 0x7f11f8004d10 2023-12-19T16:43:49.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1210a66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0c5770 con 0x7f120c14cb10 2023-12-19T16:43:49.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f120ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c15a5f0 0x7f120c15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f120ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c15a5f0 0x7f120c15aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59618/0 (socket says 172.21.15.26:59618) 2023-12-19T16:43:49.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f120ad9d640 1 -- 172.21.15.26:0/1093602568 learned_addr learned my addr 172.21.15.26:0/1093602568 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:49.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f120a59c640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11f8004d10 0x7f120c14c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f1203fff640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f120c14cb10 0x7f120c15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f120ad9d640 1 -- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f120c14cb10 msgr2=0x7f120c15a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f120ad9d640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f120c14cb10 0x7f120c15a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f120ad9d640 1 -- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11f8004d10 msgr2=0x7f120c14c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f120ad9d640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11f8004d10 0x7f120c14c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.022+0000 7f120ad9d640 1 -- 172.21.15.26:0/1093602568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c0c5a10 con 0x7f120c15a5f0 2023-12-19T16:43:49.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.026+0000 7f120ad9d640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c15a5f0 0x7f120c15aa70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f12040b3040 tx=0x7f11f8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:49.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.026+0000 7f1202ffd640 1 -- 172.21.15.26:0/1093602568 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204001210 con 0x7f120c15a5f0 2023-12-19T16:43:49.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.026+0000 7f1202ffd640 1 -- 172.21.15.26:0/1093602568 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12040ad830 con 0x7f120c15a5f0 2023-12-19T16:43:49.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.026+0000 7f1202ffd640 1 -- 172.21.15.26:0/1093602568 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204001a00 con 0x7f120c15a5f0 2023-12-19T16:43:49.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.026+0000 7f1210a66640 1 -- 172.21.15.26:0/1093602568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120c0c21a0 con 0x7f120c15a5f0 2023-12-19T16:43:49.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.026+0000 7f1210a66640 1 -- 172.21.15.26:0/1093602568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c0c2760 con 0x7f120c15a5f0 2023-12-19T16:43:49.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.026+0000 7f1210a66640 1 -- 172.21.15.26:0/1093602568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120c074270 con 0x7f120c15a5f0 2023-12-19T16:43:49.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.026+0000 7f1202ffd640 1 -- 172.21.15.26:0/1093602568 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f12040add20 con 0x7f120c15a5f0 2023-12-19T16:43:49.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.026+0000 7f1202ffd640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f11ec065c60 0x7f11ec068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.026+0000 7f1202ffd640 1 -- 172.21.15.26:0/1093602568 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f1204124be0 con 0x7f120c15a5f0 2023-12-19T16:43:49.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.026+0000 7f120a59c640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f11ec065c60 0x7f11ec068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.030+0000 7f120a59c640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f11ec065c60 0x7f11ec068120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009be0 tx=0x7f11fc009190 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:49.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.030+0000 7f1202ffd640 1 -- 172.21.15.26:0/1093602568 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f12040f21b0 con 0x7f120c15a5f0 2023-12-19T16:43:49.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.210+0000 7f1210a66640 1 -- 172.21.15.26:0/1093602568 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f120c0706e0 con 0x7f11ec065c60 2023-12-19T16:43:49.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.258+0000 7f1202ffd640 1 -- 172.21.15.26:0/1093602568 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f11fc019070 con 0x7f11ec065c60 2023-12-19T16:43:49.266 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-12-19T16:43:49.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.262+0000 7f1210a66640 1 -- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f11ec065c60 msgr2=0x7f11ec068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.262+0000 7f1210a66640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f11ec065c60 0x7f11ec068120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009be0 tx=0x7f11fc009190 comp rx=0 tx=0).stop 2023-12-19T16:43:49.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.262+0000 7f1210a66640 1 -- 172.21.15.26:0/1093602568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c15a5f0 msgr2=0x7f120c15aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.262+0000 7f1210a66640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c15a5f0 0x7f120c15aa70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f12040b3040 tx=0x7f11f8005800 comp rx=0 tx=0).stop 2023-12-19T16:43:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.262+0000 7f1210a66640 1 -- 172.21.15.26:0/1093602568 shutdown_connections 2023-12-19T16:43:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.262+0000 7f1210a66640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c15a5f0 0x7f120c15aa70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.262+0000 7f1210a66640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f11ec065c60 0x7f11ec068120 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.262+0000 7f1210a66640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f120c14cb10 0x7f120c15a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.262+0000 7f1210a66640 1 --2- 172.21.15.26:0/1093602568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11f8004d10 0x7f120c14c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.262+0000 7f1210a66640 1 -- 172.21.15.26:0/1093602568 >> 172.21.15.26:0/1093602568 conn(0x7f120c0ce730 msgr2=0x7f120c070b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.262+0000 7f1210a66640 1 -- 172.21.15.26:0/1093602568 shutdown_connections 2023-12-19T16:43:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.262+0000 7f1210a66640 1 -- 172.21.15.26:0/1093602568 wait complete. 2023-12-19T16:43:49.287 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-12-19T16:43:49.287 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir4 1M 2023-12-19T16:43:49.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.426+0000 7f7cf818f640 1 Processor -- start 2023-12-19T16:43:49.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf818f640 1 -- start start 2023-12-19T16:43:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf818f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cf015ada0 0x7f7cf00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf818f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7cf006ac80 0x7f7cf00b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf818f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf0070d90 con 0x7f7cf015a9c0 2023-12-19T16:43:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf818f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf00cc270 con 0x7f7cf006ac80 2023-12-19T16:43:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf818f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf0071070 con 0x7f7cf015ada0 2023-12-19T16:43:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf672c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7cf006ac80 0x7f7cf00b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf672c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7cf006ac80 0x7f7cf00b01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59622/0 (socket says 172.21.15.26:59622) 2023-12-19T16:43:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf672c640 1 -- 172.21.15.26:0/3560066956 learned_addr learned my addr 172.21.15.26:0/3560066956 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf572a640 1 --2- 172.21.15.26:0/3560066956 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cf015ada0 0x7f7cf00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf672c640 1 -- 172.21.15.26:0/3560066956 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cf015ada0 msgr2=0x7f7cf00706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf672c640 1 --2- 172.21.15.26:0/3560066956 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cf015ada0 0x7f7cf00706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf672c640 1 -- 172.21.15.26:0/3560066956 >> v1:172.21.15.160:6789/0 conn(0x7f7cf015a9c0 legacy=0x7f7cf00b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.430+0000 7f7cf672c640 1 -- 172.21.15.26:0/3560066956 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf0071350 con 0x7f7cf006ac80 2023-12-19T16:43:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf672c640 1 --2- 172.21.15.26:0/3560066956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf006ac80 0x7f7cf00b01b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009870 tx=0x7f7cec01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1b37852d4adfc4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf4f29640 1 -- 172.21.15.26:0/3560066956 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0ad640 con 0x7f7cf006ac80 2023-12-19T16:43:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf4f29640 1 -- 172.21.15.26:0/3560066956 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec0ad7a0 con 0x7f7cf006ac80 2023-12-19T16:43:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf4f29640 1 -- 172.21.15.26:0/3560066956 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec0adb30 con 0x7f7cf006ac80 2023-12-19T16:43:49.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf818f640 1 -- 172.21.15.26:0/3560066956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf006ac80 msgr2=0x7f7cf00b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf818f640 1 --2- 172.21.15.26:0/3560066956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf006ac80 0x7f7cf00b01b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009870 tx=0x7f7cec01c480 comp rx=0 tx=0).stop 2023-12-19T16:43:49.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf818f640 1 -- 172.21.15.26:0/3560066956 shutdown_connections 2023-12-19T16:43:49.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf818f640 1 --2- 172.21.15.26:0/3560066956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf006ac80 0x7f7cf00b01b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf818f640 1 --2- 172.21.15.26:0/3560066956 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cf015ada0 0x7f7cf00706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf818f640 1 -- 172.21.15.26:0/3560066956 >> 172.21.15.26:0/3560066956 conn(0x7f7cf00ce730 msgr2=0x7f7cf00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:49.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf818f640 1 -- 172.21.15.26:0/3560066956 shutdown_connections 2023-12-19T16:43:49.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf818f640 1 -- 172.21.15.26:0/3560066956 wait complete. 2023-12-19T16:43:49.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf818f640 1 Processor -- start 2023-12-19T16:43:49.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.434+0000 7f7cf818f640 1 -- start start 2023-12-19T16:43:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf818f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cf006ac80 0x7f7cf00b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf818f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf015a9c0 0x7f7cf0153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf818f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf015ada0 0x7f7cf0151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf818f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf00b3b70 con 0x7f7cf015a9c0 2023-12-19T16:43:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf818f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf00b3360 con 0x7f7cf015ada0 2023-12-19T16:43:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf818f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf00b35b0 con 0x7f7cf006ac80 2023-12-19T16:43:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf572a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf015a9c0 0x7f7cf0153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf572a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf015a9c0 0x7f7cf0153010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59638/0 (socket says 172.21.15.26:59638) 2023-12-19T16:43:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf572a640 1 -- 172.21.15.26:0/1766945725 learned_addr learned my addr 172.21.15.26:0/1766945725 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf5f2b640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cf006ac80 0x7f7cf00b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf672c640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf015ada0 0x7f7cf0151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf572a640 1 -- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cf006ac80 msgr2=0x7f7cf00b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf572a640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cf006ac80 0x7f7cf00b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf572a640 1 -- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf015ada0 msgr2=0x7f7cf0151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf572a640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf015ada0 0x7f7cf0151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf572a640 1 -- 172.21.15.26:0/1766945725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec0094d0 con 0x7f7cf015a9c0 2023-12-19T16:43:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf572a640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf015a9c0 0x7f7cf0153010 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000d800 tx=0x7f7ce000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7ce67fc640 1 -- 172.21.15.26:0/1766945725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce000b6c0 con 0x7f7cf015a9c0 2023-12-19T16:43:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7ce67fc640 1 -- 172.21.15.26:0/1766945725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce000b820 con 0x7f7cf015a9c0 2023-12-19T16:43:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7ce67fc640 1 -- 172.21.15.26:0/1766945725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce000bb90 con 0x7f7cf015a9c0 2023-12-19T16:43:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf818f640 1 -- 172.21.15.26:0/1766945725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf00b3890 con 0x7f7cf015a9c0 2023-12-19T16:43:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf818f640 1 -- 172.21.15.26:0/1766945725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf0152af0 con 0x7f7cf015a9c0 2023-12-19T16:43:49.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7cf818f640 1 -- 172.21.15.26:0/1766945725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf0152cb0 con 0x7f7cf015a9c0 2023-12-19T16:43:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.438+0000 7f7ce67fc640 1 -- 172.21.15.26:0/1766945725 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f7ce000bcf0 con 0x7f7cf015a9c0 2023-12-19T16:43:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.442+0000 7f7ce67fc640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cd4065c60 0x7f7cd4068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.442+0000 7f7ce67fc640 1 -- 172.21.15.26:0/1766945725 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f7ce0083ab0 con 0x7f7cf015a9c0 2023-12-19T16:43:49.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.442+0000 7f7cf5f2b640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cd4065c60 0x7f7cd4068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.442+0000 7f7cf5f2b640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cd4065c60 0x7f7cd4068120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80012f0 tx=0x7f7ce8009040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:49.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.446+0000 7f7ce67fc640 1 -- 172.21.15.26:0/1766945725 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7ce0051100 con 0x7f7cf015a9c0 2023-12-19T16:43:49.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.626+0000 7f7cf818f640 1 -- 172.21.15.26:0/1766945725 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cf0167ac0 con 0x7f7cd4065c60 2023-12-19T16:43:49.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.674+0000 7f7ce67fc640 1 -- 172.21.15.26:0/1766945725 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f7ce8001bc0 con 0x7f7cd4065c60 2023-12-19T16:43:49.683 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-12-19T16:43:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.678+0000 7f7cf818f640 1 -- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cd4065c60 msgr2=0x7f7cd4068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.678+0000 7f7cf818f640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cd4065c60 0x7f7cd4068120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80012f0 tx=0x7f7ce8009040 comp rx=0 tx=0).stop 2023-12-19T16:43:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.678+0000 7f7cf818f640 1 -- 172.21.15.26:0/1766945725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf015a9c0 msgr2=0x7f7cf0153010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.678+0000 7f7cf818f640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf015a9c0 0x7f7cf0153010 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7ce000d800 tx=0x7f7ce000dcc0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.678+0000 7f7cf818f640 1 -- 172.21.15.26:0/1766945725 shutdown_connections 2023-12-19T16:43:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.678+0000 7f7cf818f640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cf015ada0 0x7f7cf0151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.678+0000 7f7cf818f640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cf015a9c0 0x7f7cf0153010 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.678+0000 7f7cf818f640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cd4065c60 0x7f7cd4068120 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.678+0000 7f7cf818f640 1 --2- 172.21.15.26:0/1766945725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cf006ac80 0x7f7cf00b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.678+0000 7f7cf818f640 1 -- 172.21.15.26:0/1766945725 >> 172.21.15.26:0/1766945725 conn(0x7f7cf00ce730 msgr2=0x7f7cf00b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.678+0000 7f7cf818f640 1 -- 172.21.15.26:0/1766945725 shutdown_connections 2023-12-19T16:43:49.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.678+0000 7f7cf818f640 1 -- 172.21.15.26:0/1766945725 wait complete. 2023-12-19T16:43:49.704 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-12-19T16:43:49.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir5 1M 2023-12-19T16:43:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.842+0000 7f9c80518640 1 Processor -- start 2023-12-19T16:43:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.842+0000 7f9c80518640 1 -- start start 2023-12-19T16:43:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c80518640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c78159820 0x7f9c7815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c80518640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9c78159c00 0x7f9c78070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c80518640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c7806fb60 con 0x7f9c7806f6c0 2023-12-19T16:43:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c80518640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c7815e470 con 0x7f9c78159c00 2023-12-19T16:43:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c80518640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7806fe40 con 0x7f9c78159820 2023-12-19T16:43:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c7dab3640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9c7806f6c0 0x7f9c780c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32948/0 (socket says 172.21.15.26:32948) 2023-12-19T16:43:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c7dab3640 1 -- 172.21.15.26:0/3703659932 learned_addr learned my addr 172.21.15.26:0/3703659932 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c7e2b4640 1 --2- 172.21.15.26:0/3703659932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c78159820 0x7f9c7815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c7eab5640 1 --2- 172.21.15.26:0/3703659932 >> v2:172.21.15.160:3300/0 conn(0x7f9c78159c00 0x7f9c78070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c7d2b2640 1 -- 172.21.15.26:0/3703659932 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784064181 0 0) 0x7f9c7806fb60 con 0x7f9c7806f6c0 2023-12-19T16:43:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c7d2b2640 1 -- 172.21.15.26:0/3703659932 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c64003530 con 0x7f9c7806f6c0 2023-12-19T16:43:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c7e2b4640 1 -- 172.21.15.26:0/3703659932 >> v2:172.21.15.160:3300/0 conn(0x7f9c78159c00 msgr2=0x7f9c78070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c7e2b4640 1 --2- 172.21.15.26:0/3703659932 >> v2:172.21.15.160:3300/0 conn(0x7f9c78159c00 0x7f9c78070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c7e2b4640 1 -- 172.21.15.26:0/3703659932 >> v1:172.21.15.26:6789/0 conn(0x7f9c7806f6c0 legacy=0x7f9c780c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.846+0000 7f9c7e2b4640 1 -- 172.21.15.26:0/3703659932 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c78070120 con 0x7f9c78159820 2023-12-19T16:43:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c7e2b4640 1 --2- 172.21.15.26:0/3703659932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c78159820 0x7f9c7815df30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9c740097c0 tx=0x7f9c7401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71c3a0e6fde474b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c7d2b2640 1 -- 172.21.15.26:0/3703659932 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c740ad660 con 0x7f9c78159820 2023-12-19T16:43:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c7d2b2640 1 -- 172.21.15.26:0/3703659932 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c740ad7c0 con 0x7f9c78159820 2023-12-19T16:43:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c7d2b2640 1 -- 172.21.15.26:0/3703659932 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c740adb30 con 0x7f9c78159820 2023-12-19T16:43:49.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c80518640 1 -- 172.21.15.26:0/3703659932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c78159820 msgr2=0x7f9c7815df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c80518640 1 --2- 172.21.15.26:0/3703659932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c78159820 0x7f9c7815df30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9c740097c0 tx=0x7f9c7401c450 comp rx=0 tx=0).stop 2023-12-19T16:43:49.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c80518640 1 -- 172.21.15.26:0/3703659932 shutdown_connections 2023-12-19T16:43:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c80518640 1 --2- 172.21.15.26:0/3703659932 >> v2:172.21.15.160:3300/0 conn(0x7f9c78159c00 0x7f9c78070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c80518640 1 --2- 172.21.15.26:0/3703659932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c78159820 0x7f9c7815df30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c80518640 1 -- 172.21.15.26:0/3703659932 >> 172.21.15.26:0/3703659932 conn(0x7f9c780ce730 msgr2=0x7f9c780ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c80518640 1 -- 172.21.15.26:0/3703659932 shutdown_connections 2023-12-19T16:43:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c80518640 1 -- 172.21.15.26:0/3703659932 wait complete. 2023-12-19T16:43:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c80518640 1 Processor -- start 2023-12-19T16:43:49.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.850+0000 7f9c80518640 1 -- start start 2023-12-19T16:43:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c80518640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c7806f6c0 0x7f9c7815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c80518640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c78159820 0x7f9c780c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c80518640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c78159c00 0x7f9c78159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c80518640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7815a9c0 con 0x7f9c7806f6c0 2023-12-19T16:43:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c80518640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7815aca0 con 0x7f9c78159c00 2023-12-19T16:43:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c80518640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7815af80 con 0x7f9c78159820 2023-12-19T16:43:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c7e2b4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c7806f6c0 0x7f9c7815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c7e2b4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c7806f6c0 0x7f9c7815ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59648/0 (socket says 172.21.15.26:59648) 2023-12-19T16:43:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c7e2b4640 1 -- 172.21.15.26:0/4049327434 learned_addr learned my addr 172.21.15.26:0/4049327434 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c7e2b4640 1 -- 172.21.15.26:0/4049327434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c78159820 msgr2=0x7f9c780c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c7e2b4640 1 --2- 172.21.15.26:0/4049327434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c78159820 0x7f9c780c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c7e2b4640 1 -- 172.21.15.26:0/4049327434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c78159c00 msgr2=0x7f9c78159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c7e2b4640 1 --2- 172.21.15.26:0/4049327434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c78159c00 0x7f9c78159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c7e2b4640 1 -- 172.21.15.26:0/4049327434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c74009420 con 0x7f9c7806f6c0 2023-12-19T16:43:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c7e2b4640 1 --2- 172.21.15.26:0/4049327434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c7806f6c0 0x7f9c7815ba60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9c7401ab00 tx=0x7f9c740191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c6effd640 1 -- 172.21.15.26:0/4049327434 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c740bc070 con 0x7f9c7806f6c0 2023-12-19T16:43:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c6effd640 1 -- 172.21.15.26:0/4049327434 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c74006e20 con 0x7f9c7806f6c0 2023-12-19T16:43:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c6effd640 1 -- 172.21.15.26:0/4049327434 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c740b6620 con 0x7f9c7806f6c0 2023-12-19T16:43:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c80518640 1 -- 172.21.15.26:0/4049327434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7806b880 con 0x7f9c7806f6c0 2023-12-19T16:43:49.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c80518640 1 -- 172.21.15.26:0/4049327434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7815b540 con 0x7f9c7806f6c0 2023-12-19T16:43:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c80518640 1 -- 172.21.15.26:0/4049327434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c78074270 con 0x7f9c7806f6c0 2023-12-19T16:43:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.854+0000 7f9c6effd640 1 -- 172.21.15.26:0/4049327434 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f9c740b4040 con 0x7f9c7806f6c0 2023-12-19T16:43:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.858+0000 7f9c6effd640 1 --2- 172.21.15.26:0/4049327434 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c5c065c60 0x7f9c5c068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.858+0000 7f9c6effd640 1 -- 172.21.15.26:0/4049327434 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9c7411cff0 con 0x7f9c7806f6c0 2023-12-19T16:43:49.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.858+0000 7f9c7dab3640 1 --2- 172.21.15.26:0/4049327434 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c5c065c60 0x7f9c5c068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:49.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.858+0000 7f9c7dab3640 1 --2- 172.21.15.26:0/4049327434 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c5c065c60 0x7f9c5c068120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9c68007fd0 tx=0x7f9c68007900 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:49.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:49.858+0000 7f9c6effd640 1 -- 172.21.15.26:0/4049327434 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9c740ea4a0 con 0x7f9c7806f6c0 2023-12-19T16:43:50.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.042+0000 7f9c80518640 1 -- 172.21.15.26:0/4049327434 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c7815ae10 con 0x7f9c5c065c60 2023-12-19T16:43:50.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.090+0000 7f9c6effd640 1 -- 172.21.15.26:0/4049327434 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9c7815ae10 con 0x7f9c5c065c60 2023-12-19T16:43:50.098 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-12-19T16:43:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.094+0000 7f9c80518640 1 -- 172.21.15.26:0/4049327434 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c5c065c60 msgr2=0x7f9c5c068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.094+0000 7f9c80518640 1 --2- 172.21.15.26:0/4049327434 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c5c065c60 0x7f9c5c068120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9c68007fd0 tx=0x7f9c68007900 comp rx=0 tx=0).stop 2023-12-19T16:43:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.094+0000 7f9c80518640 1 -- 172.21.15.26:0/4049327434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c7806f6c0 msgr2=0x7f9c7815ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.094+0000 7f9c80518640 1 --2- 172.21.15.26:0/4049327434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c7806f6c0 0x7f9c7815ba60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9c7401ab00 tx=0x7f9c740191f0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.094+0000 7f9c80518640 1 -- 172.21.15.26:0/4049327434 shutdown_connections 2023-12-19T16:43:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.094+0000 7f9c80518640 1 --2- 172.21.15.26:0/4049327434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c78159c00 0x7f9c78159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.094+0000 7f9c80518640 1 --2- 172.21.15.26:0/4049327434 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c5c065c60 0x7f9c5c068120 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.094+0000 7f9c80518640 1 --2- 172.21.15.26:0/4049327434 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c78159820 0x7f9c780c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.094+0000 7f9c80518640 1 --2- 172.21.15.26:0/4049327434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c7806f6c0 0x7f9c7815ba60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.094+0000 7f9c80518640 1 -- 172.21.15.26:0/4049327434 >> 172.21.15.26:0/4049327434 conn(0x7f9c780ce730 msgr2=0x7f9c780af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:50.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.094+0000 7f9c80518640 1 -- 172.21.15.26:0/4049327434 shutdown_connections 2023-12-19T16:43:50.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.094+0000 7f9c80518640 1 -- 172.21.15.26:0/4049327434 wait complete. 2023-12-19T16:43:50.119 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-12-19T16:43:50.119 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir6 1M 2023-12-19T16:43:50.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.258+0000 7fb60cdbc640 1 Processor -- start 2023-12-19T16:43:50.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb60cdbc640 1 -- start start 2023-12-19T16:43:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb60cdbc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb608145ed0 0x7fb6080ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb60cdbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6080cd100 0x7fb6080ad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb60cdbc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6080cd560 con 0x7fb6080ad7b0 2023-12-19T16:43:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb60cdbc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6080ae260 con 0x7fb608145ed0 2023-12-19T16:43:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb60cdbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6080adf80 con 0x7fb6080cd100 2023-12-19T16:43:50.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb60659c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb608145ed0 0x7fb6080ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:50.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb606d9d640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fb6080ad7b0 0x7fb6081608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:56966/0 (socket says 172.21.15.26:56966) 2023-12-19T16:43:50.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb60659c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb608145ed0 0x7fb6080ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:55558/0 (socket says 172.21.15.26:55558) 2023-12-19T16:43:50.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb606d9d640 1 -- 172.21.15.26:0/3704247705 learned_addr learned my addr 172.21.15.26:0/3704247705 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:50.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb5fffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6080cd100 0x7fb6080ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb605d9b640 1 -- 172.21.15.26:0/3704247705 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160489750 0 0) 0x7fb6080cd560 con 0x7fb6080ad7b0 2023-12-19T16:43:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb605d9b640 1 -- 172.21.15.26:0/3704247705 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5f4003530 con 0x7fb6080ad7b0 2023-12-19T16:43:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb5fffff640 1 -- 172.21.15.26:0/3704247705 >> v2:172.21.15.160:3300/0 conn(0x7fb608145ed0 msgr2=0x7fb6080ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb5fffff640 1 --2- 172.21.15.26:0/3704247705 >> v2:172.21.15.160:3300/0 conn(0x7fb608145ed0 0x7fb6080ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb5fffff640 1 -- 172.21.15.26:0/3704247705 >> v1:172.21.15.160:6789/0 conn(0x7fb6080ad7b0 legacy=0x7fb6081608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb5fffff640 1 -- 172.21.15.26:0/3704247705 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6080c9770 con 0x7fb6080cd100 2023-12-19T16:43:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.262+0000 7fb60659c640 1 --2- 172.21.15.26:0/3704247705 >> v2:172.21.15.160:3300/0 conn(0x7fb608145ed0 0x7fb6080ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:50.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb5fffff640 1 --2- 172.21.15.26:0/3704247705 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6080cd100 0x7fb6080ad270 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00097c0 tx=0x7fb5f001c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12f72df377f737fe server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:50.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb605d9b640 1 -- 172.21.15.26:0/3704247705 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00ab600 con 0x7fb6080cd100 2023-12-19T16:43:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb605d9b640 1 -- 172.21.15.26:0/3704247705 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f00a7050 con 0x7fb6080cd100 2023-12-19T16:43:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb605d9b640 1 -- 172.21.15.26:0/3704247705 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00ab980 con 0x7fb6080cd100 2023-12-19T16:43:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb60cdbc640 1 -- 172.21.15.26:0/3704247705 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6080cd100 msgr2=0x7fb6080ad270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb60cdbc640 1 --2- 172.21.15.26:0/3704247705 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6080cd100 0x7fb6080ad270 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00097c0 tx=0x7fb5f001c330 comp rx=0 tx=0).stop 2023-12-19T16:43:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb60cdbc640 1 -- 172.21.15.26:0/3704247705 shutdown_connections 2023-12-19T16:43:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb60cdbc640 1 --2- 172.21.15.26:0/3704247705 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6080cd100 0x7fb6080ad270 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb60cdbc640 1 --2- 172.21.15.26:0/3704247705 >> v2:172.21.15.160:3300/0 conn(0x7fb608145ed0 0x7fb6080ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb60cdbc640 1 -- 172.21.15.26:0/3704247705 >> 172.21.15.26:0/3704247705 conn(0x7fb6080b0b60 msgr2=0x7fb6080b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb60cdbc640 1 -- 172.21.15.26:0/3704247705 shutdown_connections 2023-12-19T16:43:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb60cdbc640 1 -- 172.21.15.26:0/3704247705 wait complete. 2023-12-19T16:43:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb60cdbc640 1 Processor -- start 2023-12-19T16:43:50.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.266+0000 7fb60cdbc640 1 -- start start 2023-12-19T16:43:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb60cdbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6080ad7b0 0x7fb60815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb60cdbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6080cd100 0x7fb6080ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb60cdbc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb608145ed0 0x7fb6080ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb60cdbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60815e410 con 0x7fb6080cd100 2023-12-19T16:43:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb60cdbc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6080c2ae0 con 0x7fb608145ed0 2023-12-19T16:43:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb60cdbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60815e130 con 0x7fb6080ad7b0 2023-12-19T16:43:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb5fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6080cd100 0x7fb6080ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb5fffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6080cd100 0x7fb6080ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59658/0 (socket says 172.21.15.26:59658) 2023-12-19T16:43:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb5fffff640 1 -- 172.21.15.26:0/1297758956 learned_addr learned my addr 172.21.15.26:0/1297758956 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:50.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb60659c640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6080ad7b0 0x7fb60815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb606d9d640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb608145ed0 0x7fb6080ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb5fffff640 1 -- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6080ad7b0 msgr2=0x7fb60815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb5fffff640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6080ad7b0 0x7fb60815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb5fffff640 1 -- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb608145ed0 msgr2=0x7fb6080ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb5fffff640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb608145ed0 0x7fb6080ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb5fffff640 1 -- 172.21.15.26:0/1297758956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f8009420 con 0x7fb6080cd100 2023-12-19T16:43:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb5fffff640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6080cd100 0x7fb6080ca150 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00ae040 tx=0x7fb5f00190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb5feffd640 1 -- 172.21.15.26:0/1297758956 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00b4430 con 0x7fb6080cd100 2023-12-19T16:43:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb5feffd640 1 -- 172.21.15.26:0/1297758956 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f00b45b0 con 0x7fb6080cd100 2023-12-19T16:43:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb5feffd640 1 -- 172.21.15.26:0/1297758956 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00b4960 con 0x7fb6080cd100 2023-12-19T16:43:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb60cdbc640 1 -- 172.21.15.26:0/1297758956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f0009420 con 0x7fb6080cd100 2023-12-19T16:43:50.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb60cdbc640 1 -- 172.21.15.26:0/1297758956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6080c3010 con 0x7fb6080cd100 2023-12-19T16:43:50.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.270+0000 7fb60cdbc640 1 -- 172.21.15.26:0/1297758956 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6080afb00 con 0x7fb6080cd100 2023-12-19T16:43:50.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.274+0000 7fb5feffd640 1 -- 172.21.15.26:0/1297758956 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7fb5f00b2040 con 0x7fb6080cd100 2023-12-19T16:43:50.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.274+0000 7fb5feffd640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb5e8065c60 0x7fb5e8068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:50.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.274+0000 7fb5feffd640 1 -- 172.21.15.26:0/1297758956 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb5f011d350 con 0x7fb6080cd100 2023-12-19T16:43:50.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.274+0000 7fb60659c640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb5e8065c60 0x7fb5e8068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:50.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.274+0000 7fb60659c640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb5e8065c60 0x7fb5e8068120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009cf0 tx=0x7fb5f8009190 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:50.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.278+0000 7fb5feffd640 1 -- 172.21.15.26:0/1297758956 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb5f00ea930 con 0x7fb6080cd100 2023-12-19T16:43:50.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.454+0000 7fb60cdbc640 1 -- 172.21.15.26:0/1297758956 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6081462b0 con 0x7fb5e8065c60 2023-12-19T16:43:50.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.498+0000 7fb5feffd640 1 -- 172.21.15.26:0/1297758956 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb5f8009940 con 0x7fb5e8065c60 2023-12-19T16:43:50.505 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-12-19T16:43:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.498+0000 7fb60cdbc640 1 -- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb5e8065c60 msgr2=0x7fb5e8068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.498+0000 7fb60cdbc640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb5e8065c60 0x7fb5e8068120 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009cf0 tx=0x7fb5f8009190 comp rx=0 tx=0).stop 2023-12-19T16:43:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.498+0000 7fb60cdbc640 1 -- 172.21.15.26:0/1297758956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6080cd100 msgr2=0x7fb6080ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.498+0000 7fb60cdbc640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6080cd100 0x7fb6080ca150 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00ae040 tx=0x7fb5f00190d0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.502+0000 7fb60cdbc640 1 -- 172.21.15.26:0/1297758956 shutdown_connections 2023-12-19T16:43:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.502+0000 7fb60cdbc640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb608145ed0 0x7fb6080ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.502+0000 7fb60cdbc640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6080cd100 0x7fb6080ca150 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.502+0000 7fb60cdbc640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb5e8065c60 0x7fb5e8068120 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.502+0000 7fb60cdbc640 1 --2- 172.21.15.26:0/1297758956 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6080ad7b0 0x7fb60815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.502+0000 7fb60cdbc640 1 -- 172.21.15.26:0/1297758956 >> 172.21.15.26:0/1297758956 conn(0x7fb6080b0b60 msgr2=0x7fb60806df00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:50.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.502+0000 7fb60cdbc640 1 -- 172.21.15.26:0/1297758956 shutdown_connections 2023-12-19T16:43:50.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.502+0000 7fb60cdbc640 1 -- 172.21.15.26:0/1297758956 wait complete. 2023-12-19T16:43:50.526 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-12-19T16:43:50.526 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir7 1M 2023-12-19T16:43:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd97841640 1 Processor -- start 2023-12-19T16:43:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd97841640 1 -- start start 2023-12-19T16:43:50.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd97841640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd9015ada0 0x7fbd900706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:50.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd97841640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd90070d90 con 0x7fbd9006ac80 2023-12-19T16:43:50.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd97841640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd90071070 con 0x7fbd9015a9c0 2023-12-19T16:43:50.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd97841640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900cc270 con 0x7fbd9015ada0 2023-12-19T16:43:50.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd95dde640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbd9006ac80 0x7fbd900b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32962/0 (socket says 172.21.15.26:32962) 2023-12-19T16:43:50.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd95dde640 1 -- 172.21.15.26:0/4132922687 learned_addr learned my addr 172.21.15.26:0/4132922687 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:50.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd94ddc640 1 --2- 172.21.15.26:0/4132922687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd9015ada0 0x7fbd900706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd87fff640 1 -- 172.21.15.26:0/4132922687 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303365478 0 0) 0x7fbd90070d90 con 0x7fbd9006ac80 2023-12-19T16:43:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd87fff640 1 -- 172.21.15.26:0/4132922687 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd7c003530 con 0x7fbd9006ac80 2023-12-19T16:43:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd87fff640 1 -- 172.21.15.26:0/4132922687 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713487621 0 0) 0x7fbd90071070 con 0x7fbd9015a9c0 2023-12-19T16:43:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd87fff640 1 -- 172.21.15.26:0/4132922687 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd90070d90 con 0x7fbd9015a9c0 2023-12-19T16:43:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd94ddc640 1 -- 172.21.15.26:0/4132922687 >> v1:172.21.15.160:6789/0 conn(0x7fbd9015a9c0 legacy=0x7fbd900b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd94ddc640 1 -- 172.21.15.26:0/4132922687 >> v1:172.21.15.26:6789/0 conn(0x7fbd9006ac80 legacy=0x7fbd900b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.666+0000 7fbd94ddc640 1 -- 172.21.15.26:0/4132922687 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd90071350 con 0x7fbd9015ada0 2023-12-19T16:43:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.670+0000 7fbd94ddc640 1 --2- 172.21.15.26:0/4132922687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd9015ada0 0x7fbd900706d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbd800097f0 tx=0x7fbd8001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=114e2de682168b88 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:50.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.670+0000 7fbd87fff640 1 -- 172.21.15.26:0/4132922687 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd800ab620 con 0x7fbd9015ada0 2023-12-19T16:43:50.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.670+0000 7fbd87fff640 1 -- 172.21.15.26:0/4132922687 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd800ab780 con 0x7fbd9015ada0 2023-12-19T16:43:50.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.670+0000 7fbd87fff640 1 -- 172.21.15.26:0/4132922687 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd800abb10 con 0x7fbd9015ada0 2023-12-19T16:43:50.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.670+0000 7fbd97841640 1 -- 172.21.15.26:0/4132922687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd9015ada0 msgr2=0x7fbd900706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.670+0000 7fbd97841640 1 --2- 172.21.15.26:0/4132922687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd9015ada0 0x7fbd900706d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbd800097f0 tx=0x7fbd8001c400 comp rx=0 tx=0).stop 2023-12-19T16:43:50.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.670+0000 7fbd97841640 1 -- 172.21.15.26:0/4132922687 shutdown_connections 2023-12-19T16:43:50.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.670+0000 7fbd97841640 1 --2- 172.21.15.26:0/4132922687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd9015ada0 0x7fbd900706d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.670+0000 7fbd97841640 1 -- 172.21.15.26:0/4132922687 >> 172.21.15.26:0/4132922687 conn(0x7fbd900ce730 msgr2=0x7fbd900ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:50.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.670+0000 7fbd97841640 1 -- 172.21.15.26:0/4132922687 shutdown_connections 2023-12-19T16:43:50.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.670+0000 7fbd97841640 1 -- 172.21.15.26:0/4132922687 wait complete. 2023-12-19T16:43:50.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd97841640 1 Processor -- start 2023-12-19T16:43:50.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd97841640 1 -- start start 2023-12-19T16:43:50.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd97841640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9006ac80 0x7fbd900c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:50.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd97841640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd9015a9c0 0x7fbd90151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:50.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd97841640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015ada0 0x7fbd900c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:50.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd97841640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900c7150 con 0x7fbd9015ada0 2023-12-19T16:43:50.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd97841640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900c8540 con 0x7fbd9006ac80 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd97841640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900c6820 con 0x7fbd9015a9c0 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd95dde640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015ada0 0x7fbd900c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd95dde640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015ada0 0x7fbd900c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59670/0 (socket says 172.21.15.26:59670) 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd95dde640 1 -- 172.21.15.26:0/490829312 learned_addr learned my addr 172.21.15.26:0/490829312 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd94ddc640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd9015a9c0 0x7fbd90151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd955dd640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9006ac80 0x7fbd900c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd95dde640 1 -- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd9015a9c0 msgr2=0x7fbd90151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd95dde640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd9015a9c0 0x7fbd90151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd95dde640 1 -- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9006ac80 msgr2=0x7fbd900c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd95dde640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9006ac80 0x7fbd900c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd95dde640 1 -- 172.21.15.26:0/490829312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd80009420 con 0x7fbd9015ada0 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd95dde640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015ada0 0x7fbd900c9700 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800cb10 tx=0x7fbd8800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd85ffb640 1 -- 172.21.15.26:0/490829312 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8800ac70 con 0x7fbd9015ada0 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd85ffb640 1 -- 172.21.15.26:0/490829312 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8800add0 con 0x7fbd9015ada0 2023-12-19T16:43:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd85ffb640 1 -- 172.21.15.26:0/490829312 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88012620 con 0x7fbd9015ada0 2023-12-19T16:43:50.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.674+0000 7fbd97841640 1 -- 172.21.15.26:0/490829312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd900c6b00 con 0x7fbd9015ada0 2023-12-19T16:43:50.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.678+0000 7fbd97841640 1 -- 172.21.15.26:0/490829312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd900c7b20 con 0x7fbd9015ada0 2023-12-19T16:43:50.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.678+0000 7fbd97841640 1 -- 172.21.15.26:0/490829312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd900c72c0 con 0x7fbd9015ada0 2023-12-19T16:43:50.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.678+0000 7fbd85ffb640 1 -- 172.21.15.26:0/490829312 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7fbd88007070 con 0x7fbd9015ada0 2023-12-19T16:43:50.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.678+0000 7fbd85ffb640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd74065c60 0x7fbd74068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:50.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.678+0000 7fbd85ffb640 1 -- 172.21.15.26:0/490829312 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fbd88083630 con 0x7fbd9015ada0 2023-12-19T16:43:50.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.678+0000 7fbd955dd640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd74065c60 0x7fbd74068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:50.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.678+0000 7fbd955dd640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd74065c60 0x7fbd74068120 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0011f0 tx=0x7fbd8c009040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:50.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.682+0000 7fbd85ffb640 1 -- 172.21.15.26:0/490829312 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbd88050bd0 con 0x7fbd9015ada0 2023-12-19T16:43:50.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.858+0000 7fbd97841640 1 -- 172.21.15.26:0/490829312 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd90168ad0 con 0x7fbd74065c60 2023-12-19T16:43:50.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.906+0000 7fbd85ffb640 1 -- 172.21.15.26:0/490829312 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fbd8c001cb0 con 0x7fbd74065c60 2023-12-19T16:43:50.913 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-12-19T16:43:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.906+0000 7fbd97841640 1 -- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd74065c60 msgr2=0x7fbd74068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.906+0000 7fbd97841640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd74065c60 0x7fbd74068120 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0011f0 tx=0x7fbd8c009040 comp rx=0 tx=0).stop 2023-12-19T16:43:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.906+0000 7fbd97841640 1 -- 172.21.15.26:0/490829312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015ada0 msgr2=0x7fbd900c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.906+0000 7fbd97841640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015ada0 0x7fbd900c9700 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800cb10 tx=0x7fbd8800a060 comp rx=0 tx=0).stop 2023-12-19T16:43:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.910+0000 7fbd97841640 1 -- 172.21.15.26:0/490829312 shutdown_connections 2023-12-19T16:43:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.910+0000 7fbd97841640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd9015ada0 0x7fbd900c9700 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.910+0000 7fbd97841640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd74065c60 0x7fbd74068120 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.910+0000 7fbd97841640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd9015a9c0 0x7fbd90151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.910+0000 7fbd97841640 1 --2- 172.21.15.26:0/490829312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9006ac80 0x7fbd900c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.910+0000 7fbd97841640 1 -- 172.21.15.26:0/490829312 >> 172.21.15.26:0/490829312 conn(0x7fbd900ce730 msgr2=0x7fbd900ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.910+0000 7fbd97841640 1 -- 172.21.15.26:0/490829312 shutdown_connections 2023-12-19T16:43:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:50.910+0000 7fbd97841640 1 -- 172.21.15.26:0/490829312 wait complete. 2023-12-19T16:43:50.934 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-12-19T16:43:50.934 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir8 1M 2023-12-19T16:43:51.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a1337b640 1 Processor -- start 2023-12-19T16:43:51.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a1337b640 1 -- start start 2023-12-19T16:43:51.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a1337b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7a0c1645b0 0x7f7a0c0cb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a1337b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a0c0addb0 0x7f7a0c0c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a1337b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0c06f8d0 con 0x7f7a0c0c9b10 2023-12-19T16:43:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a1337b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0c0c96f0 con 0x7f7a0c1645b0 2023-12-19T16:43:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a1337b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c06f310 con 0x7f7a0c0addb0 2023-12-19T16:43:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a11117640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7a0c1645b0 0x7f7a0c0cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a11117640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7a0c1645b0 0x7f7a0c0cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59678/0 (socket says 172.21.15.26:59678) 2023-12-19T16:43:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a11117640 1 -- 172.21.15.26:0/735036664 learned_addr learned my addr 172.21.15.26:0/735036664 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a11918640 1 --2- 172.21.15.26:0/735036664 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a0c0addb0 0x7f7a0c0c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a11117640 1 -- 172.21.15.26:0/735036664 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a0c0addb0 msgr2=0x7f7a0c0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a11117640 1 --2- 172.21.15.26:0/735036664 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a0c0addb0 0x7f7a0c0c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a11117640 1 -- 172.21.15.26:0/735036664 >> v1:172.21.15.160:6789/0 conn(0x7f7a0c0c9b10 legacy=0x7f7a0c0ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:51.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.074+0000 7f7a11117640 1 -- 172.21.15.26:0/735036664 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c06f5f0 con 0x7f7a0c1645b0 2023-12-19T16:43:51.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.078+0000 7f7a11117640 1 --2- 172.21.15.26:0/735036664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a0c1645b0 0x7f7a0c0cb570 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009870 tx=0x7f7a0801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32cdb329f25ef42 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:51.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.078+0000 7f7a03fff640 1 -- 172.21.15.26:0/735036664 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a080a95f0 con 0x7f7a0c1645b0 2023-12-19T16:43:51.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.078+0000 7f7a03fff640 1 -- 172.21.15.26:0/735036664 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a080a9750 con 0x7f7a0c1645b0 2023-12-19T16:43:51.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.078+0000 7f7a03fff640 1 -- 172.21.15.26:0/735036664 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a080a9ae0 con 0x7f7a0c1645b0 2023-12-19T16:43:51.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.078+0000 7f7a1337b640 1 -- 172.21.15.26:0/735036664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a0c1645b0 msgr2=0x7f7a0c0cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:51.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.078+0000 7f7a1337b640 1 --2- 172.21.15.26:0/735036664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a0c1645b0 0x7f7a0c0cb570 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7a08009870 tx=0x7f7a0801c510 comp rx=0 tx=0).stop 2023-12-19T16:43:51.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.078+0000 7f7a1337b640 1 -- 172.21.15.26:0/735036664 shutdown_connections 2023-12-19T16:43:51.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.078+0000 7f7a1337b640 1 --2- 172.21.15.26:0/735036664 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a0c0addb0 0x7f7a0c0c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.078+0000 7f7a1337b640 1 --2- 172.21.15.26:0/735036664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a0c1645b0 0x7f7a0c0cb570 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.078+0000 7f7a1337b640 1 -- 172.21.15.26:0/735036664 >> 172.21.15.26:0/735036664 conn(0x7f7a0c065ab0 msgr2=0x7f7a0c065e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:51.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.078+0000 7f7a1337b640 1 -- 172.21.15.26:0/735036664 shutdown_connections 2023-12-19T16:43:51.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.078+0000 7f7a1337b640 1 -- 172.21.15.26:0/735036664 wait complete. 2023-12-19T16:43:51.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a1337b640 1 Processor -- start 2023-12-19T16:43:51.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a1337b640 1 -- start start 2023-12-19T16:43:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a1337b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a0c0addb0 0x7f7a0c0b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a1337b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a0c0c9b10 0x7f7a0c15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a1337b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a0c1645b0 0x7f7a0c156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a1337b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0b1d60 con 0x7f7a0c0addb0 2023-12-19T16:43:51.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a1337b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0b2040 con 0x7f7a0c1645b0 2023-12-19T16:43:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a1337b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0b2320 con 0x7f7a0c0c9b10 2023-12-19T16:43:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a11117640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a0c0addb0 0x7f7a0c0b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a11117640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a0c0addb0 0x7f7a0c0b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59690/0 (socket says 172.21.15.26:59690) 2023-12-19T16:43:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a11117640 1 -- 172.21.15.26:0/1655343016 learned_addr learned my addr 172.21.15.26:0/1655343016 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a10916640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a0c0c9b10 0x7f7a0c15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a11918640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a0c1645b0 0x7f7a0c156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a11117640 1 -- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a0c0c9b10 msgr2=0x7f7a0c15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a11117640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a0c0c9b10 0x7f7a0c15bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a11117640 1 -- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a0c1645b0 msgr2=0x7f7a0c156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a11117640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a0c1645b0 0x7f7a0c156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a11117640 1 -- 172.21.15.26:0/1655343016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a080094d0 con 0x7f7a0c0addb0 2023-12-19T16:43:51.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a11117640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a0c0addb0 0x7f7a0c0b1530 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7a0801cdb0 tx=0x7f7a0801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:51.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a01ffb640 1 -- 172.21.15.26:0/1655343016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a080a95f0 con 0x7f7a0c0addb0 2023-12-19T16:43:51.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a01ffb640 1 -- 172.21.15.26:0/1655343016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a080b1800 con 0x7f7a0c0addb0 2023-12-19T16:43:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a01ffb640 1 -- 172.21.15.26:0/1655343016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a080b1bb0 con 0x7f7a0c0addb0 2023-12-19T16:43:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a1337b640 1 -- 172.21.15.26:0/1655343016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c0b2600 con 0x7f7a0c0addb0 2023-12-19T16:43:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.082+0000 7f7a1337b640 1 -- 172.21.15.26:0/1655343016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c15dde0 con 0x7f7a0c0addb0 2023-12-19T16:43:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.086+0000 7f7a1337b640 1 -- 172.21.15.26:0/1655343016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0c04e850 con 0x7f7a0c0addb0 2023-12-19T16:43:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.086+0000 7f7a01ffb640 1 -- 172.21.15.26:0/1655343016 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f7a080ba3a0 con 0x7f7a0c0addb0 2023-12-19T16:43:51.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.086+0000 7f7a01ffb640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f79f0065c60 0x7f79f0068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:51.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.086+0000 7f7a01ffb640 1 -- 172.21.15.26:0/1655343016 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f7a0811dc40 con 0x7f7a0c0addb0 2023-12-19T16:43:51.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.086+0000 7f7a10916640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f79f0065c60 0x7f79f0068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:51.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.086+0000 7f7a10916640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f79f0065c60 0x7f79f0068120 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c15d0f0 tx=0x7f79fc009040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:51.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.090+0000 7f7a01ffb640 1 -- 172.21.15.26:0/1655343016 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7a080eb230 con 0x7f7a0c0addb0 2023-12-19T16:43:51.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.266+0000 7f7a1337b640 1 -- 172.21.15.26:0/1655343016 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a0c0b21b0 con 0x7f79f0065c60 2023-12-19T16:43:51.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.310+0000 7f7a01ffb640 1 -- 172.21.15.26:0/1655343016 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f7a0c0b21b0 con 0x7f79f0065c60 2023-12-19T16:43:51.318 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-12-19T16:43:51.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.314+0000 7f7a1337b640 1 -- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f79f0065c60 msgr2=0x7f79f0068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:51.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.314+0000 7f7a1337b640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f79f0065c60 0x7f79f0068120 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c15d0f0 tx=0x7f79fc009040 comp rx=0 tx=0).stop 2023-12-19T16:43:51.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.314+0000 7f7a1337b640 1 -- 172.21.15.26:0/1655343016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a0c0addb0 msgr2=0x7f7a0c0b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.314+0000 7f7a1337b640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a0c0addb0 0x7f7a0c0b1530 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7a0801cdb0 tx=0x7f7a0801ad50 comp rx=0 tx=0).stop 2023-12-19T16:43:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.314+0000 7f7a1337b640 1 -- 172.21.15.26:0/1655343016 shutdown_connections 2023-12-19T16:43:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.314+0000 7f7a1337b640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a0c1645b0 0x7f7a0c156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.314+0000 7f7a1337b640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f79f0065c60 0x7f79f0068120 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.314+0000 7f7a1337b640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a0c0c9b10 0x7f7a0c15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.314+0000 7f7a1337b640 1 --2- 172.21.15.26:0/1655343016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a0c0addb0 0x7f7a0c0b1530 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.314+0000 7f7a1337b640 1 -- 172.21.15.26:0/1655343016 >> 172.21.15.26:0/1655343016 conn(0x7f7a0c065ab0 msgr2=0x7f7a0c06c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.314+0000 7f7a1337b640 1 -- 172.21.15.26:0/1655343016 shutdown_connections 2023-12-19T16:43:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.314+0000 7f7a1337b640 1 -- 172.21.15.26:0/1655343016 wait complete. 2023-12-19T16:43:51.339 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-12-19T16:43:51.339 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir9 1M 2023-12-19T16:43:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.478+0000 7ff96317b640 1 Processor -- start 2023-12-19T16:43:51.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.478+0000 7ff96317b640 1 -- start start 2023-12-19T16:43:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.478+0000 7ff96317b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff95c159820 0x7ff95c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:51.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.478+0000 7ff96317b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff95c06f6c0 0x7ff95c0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.478+0000 7ff96317b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff95c06fb60 con 0x7ff95c159c00 2023-12-19T16:43:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.478+0000 7ff96317b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95c15e470 con 0x7ff95c06f6c0 2023-12-19T16:43:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.478+0000 7ff96317b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95c06fe40 con 0x7ff95c159820 2023-12-19T16:43:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.478+0000 7ff95bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff95c06f6c0 0x7ff95c0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.478+0000 7ff961718640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff95c159c00 0x7ff95c070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32970/0 (socket says 172.21.15.26:32970) 2023-12-19T16:43:51.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.478+0000 7ff961718640 1 -- 172.21.15.26:0/712108444 learned_addr learned my addr 172.21.15.26:0/712108444 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:43:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.482+0000 7ff960f17640 1 --2- 172.21.15.26:0/712108444 >> v2:172.21.15.160:3300/0 conn(0x7ff95c159820 0x7ff95c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.482+0000 7ff95bfff640 1 -- 172.21.15.26:0/712108444 >> v2:172.21.15.160:3300/0 conn(0x7ff95c159820 msgr2=0x7ff95c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:43:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.482+0000 7ff95bfff640 1 --2- 172.21.15.26:0/712108444 >> v2:172.21.15.160:3300/0 conn(0x7ff95c159820 0x7ff95c15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.482+0000 7ff95bfff640 1 -- 172.21.15.26:0/712108444 >> v1:172.21.15.26:6789/0 conn(0x7ff95c159c00 legacy=0x7ff95c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.482+0000 7ff95bfff640 1 -- 172.21.15.26:0/712108444 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c070120 con 0x7ff95c06f6c0 2023-12-19T16:43:51.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.482+0000 7ff95b7fe640 1 -- 172.21.15.26:0/712108444 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263071966 0 0) 0x7ff95c06fb60 con 0x7ff95c159c00 2023-12-19T16:43:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.482+0000 7ff95bfff640 1 --2- 172.21.15.26:0/712108444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c06f6c0 0x7ff95c0c8e80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0c8e10 tx=0x7ff94801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da466b832188814 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.482+0000 7ff95b7fe640 1 -- 172.21.15.26:0/712108444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9480a7070 con 0x7ff95c06f6c0 2023-12-19T16:43:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.482+0000 7ff95b7fe640 1 -- 172.21.15.26:0/712108444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9480ab430 con 0x7ff95c06f6c0 2023-12-19T16:43:51.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.482+0000 7ff95b7fe640 1 -- 172.21.15.26:0/712108444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9480ab7c0 con 0x7ff95c06f6c0 2023-12-19T16:43:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.482+0000 7ff96317b640 1 -- 172.21.15.26:0/712108444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c06f6c0 msgr2=0x7ff95c0c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.482+0000 7ff96317b640 1 --2- 172.21.15.26:0/712108444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c06f6c0 0x7ff95c0c8e80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff95c0c8e10 tx=0x7ff94801c430 comp rx=0 tx=0).stop 2023-12-19T16:43:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 -- 172.21.15.26:0/712108444 shutdown_connections 2023-12-19T16:43:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 --2- 172.21.15.26:0/712108444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c06f6c0 0x7ff95c0c8e80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 --2- 172.21.15.26:0/712108444 >> v2:172.21.15.160:3300/0 conn(0x7ff95c159820 0x7ff95c15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 -- 172.21.15.26:0/712108444 >> 172.21.15.26:0/712108444 conn(0x7ff95c0ce730 msgr2=0x7ff95c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 -- 172.21.15.26:0/712108444 shutdown_connections 2023-12-19T16:43:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 -- 172.21.15.26:0/712108444 wait complete. 2023-12-19T16:43:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 Processor -- start 2023-12-19T16:43:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 -- start start 2023-12-19T16:43:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff95c06f6c0 0x7ff95c0c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff95c159820 0x7ff95c0c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c159c00 0x7ff95c0acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c0c3ae0 con 0x7ff95c159c00 2023-12-19T16:43:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c0c3dc0 con 0x7ff95c159820 2023-12-19T16:43:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff96317b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c0c40a0 con 0x7ff95c06f6c0 2023-12-19T16:43:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff961718640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c159c00 0x7ff95c0acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff961718640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c159c00 0x7ff95c0acac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59700/0 (socket says 172.21.15.26:59700) 2023-12-19T16:43:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff961718640 1 -- 172.21.15.26:0/2002193382 learned_addr learned my addr 172.21.15.26:0/2002193382 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:43:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff960f17640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff95c06f6c0 0x7ff95c0c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.486+0000 7ff95bfff640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff95c159820 0x7ff95c0c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:51.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff961718640 1 -- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff95c06f6c0 msgr2=0x7ff95c0c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:51.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff961718640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff95c06f6c0 0x7ff95c0c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff961718640 1 -- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff95c159820 msgr2=0x7ff95c0c2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff961718640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff95c159820 0x7ff95c0c2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff961718640 1 -- 172.21.15.26:0/2002193382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff948009420 con 0x7ff95c159c00 2023-12-19T16:43:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff960f17640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff95c06f6c0 0x7ff95c0c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff95bfff640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff95c159820 0x7ff95c0c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:43:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff961718640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c159c00 0x7ff95c0acac0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff95400c780 tx=0x7ff95400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff9597fa640 1 -- 172.21.15.26:0/2002193382 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95400ce90 con 0x7ff95c159c00 2023-12-19T16:43:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff9597fa640 1 -- 172.21.15.26:0/2002193382 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95400a640 con 0x7ff95c159c00 2023-12-19T16:43:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff9597fa640 1 -- 172.21.15.26:0/2002193382 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95400aa70 con 0x7ff95c159c00 2023-12-19T16:43:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff96317b640 1 -- 172.21.15.26:0/2002193382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95c06b880 con 0x7ff95c159c00 2023-12-19T16:43:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff96317b640 1 -- 172.21.15.26:0/2002193382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95c0c4660 con 0x7ff95c159c00 2023-12-19T16:43:51.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff96317b640 1 -- 172.21.15.26:0/2002193382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95c074270 con 0x7ff95c159c00 2023-12-19T16:43:51.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff9597fa640 1 -- 172.21.15.26:0/2002193382 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7ff95400abd0 con 0x7ff95c159c00 2023-12-19T16:43:51.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff9597fa640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff940065c90 0x7ff940068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:43:51.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.490+0000 7ff9597fa640 1 -- 172.21.15.26:0/2002193382 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff954082960 con 0x7ff95c159c00 2023-12-19T16:43:51.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.494+0000 7ff960f17640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff940065c90 0x7ff940068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:43:51.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.494+0000 7ff960f17640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff940065c90 0x7ff940068150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff950007e50 tx=0x7ff950008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:43:51.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.494+0000 7ff9597fa640 1 -- 172.21.15.26:0/2002193382 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff954050060 con 0x7ff95c159c00 2023-12-19T16:43:51.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.674+0000 7ff96317b640 1 -- 172.21.15.26:0/2002193382 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff95c0c3f30 con 0x7ff940065c90 2023-12-19T16:43:51.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.722+0000 7ff9597fa640 1 -- 172.21.15.26:0/2002193382 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff95c0c3f30 con 0x7ff940065c90 2023-12-19T16:43:51.728 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-12-19T16:43:51.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.722+0000 7ff96317b640 1 -- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff940065c90 msgr2=0x7ff940068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.722+0000 7ff96317b640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff940065c90 0x7ff940068150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff950007e50 tx=0x7ff950008040 comp rx=0 tx=0).stop 2023-12-19T16:43:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.722+0000 7ff96317b640 1 -- 172.21.15.26:0/2002193382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c159c00 msgr2=0x7ff95c0acac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:43:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.722+0000 7ff96317b640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c159c00 0x7ff95c0acac0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff95400c780 tx=0x7ff95400cc40 comp rx=0 tx=0).stop 2023-12-19T16:43:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.722+0000 7ff96317b640 1 -- 172.21.15.26:0/2002193382 shutdown_connections 2023-12-19T16:43:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.722+0000 7ff96317b640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95c159c00 0x7ff95c0acac0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.722+0000 7ff96317b640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff95c159820 0x7ff95c0c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.722+0000 7ff96317b640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff940065c90 0x7ff940068150 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.722+0000 7ff96317b640 1 --2- 172.21.15.26:0/2002193382 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff95c06f6c0 0x7ff95c0c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:43:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.722+0000 7ff96317b640 1 -- 172.21.15.26:0/2002193382 >> 172.21.15.26:0/2002193382 conn(0x7ff95c0ce730 msgr2=0x7ff95c0c97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:43:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.722+0000 7ff96317b640 1 -- 172.21.15.26:0/2002193382 shutdown_connections 2023-12-19T16:43:51.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:43:51.722+0000 7ff96317b640 1 -- 172.21.15.26:0/2002193382 wait complete. 2023-12-19T16:43:51.750 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-12-19T16:43:57.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:43:57.581 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:44:00.037 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:44:00.032+0000 7f703dee9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:44:00.037 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:44:00.032+0000 7f703dee9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:44:00.037 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:44:00.032+0000 7f703dee9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:44:00.037 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:44:00.032+0000 7f703dee9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:44:00.037 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:44:00.032+0000 7f703dee9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:44:27.665 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:44:27.670 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:44:57.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:44:57.734 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:45:27.802 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:45:27.807 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:45:57.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:45:57.871 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:46:27.930 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:46:27.935 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:46:57.993 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:46:57.998 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:47:28.058 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:47:28.063 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:47:58.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:47:58.235 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:48:00.207 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:48:00.202+0000 7f70406ee640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:48:00.207 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:48:00.202+0000 7f70406ee640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:48:00.207 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:48:00.202+0000 7f70406ee640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:48:00.207 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:48:00.202+0000 7f70406ee640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:48:00.207 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:48:00.202+0000 7f70406ee640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:48:28.285 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:48:28.290 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:48:58.348 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:48:58.356 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:49:28.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:49:28.417 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:49:58.475 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:49:58.480 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:50:28.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:50:28.552 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:50:58.603 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:50:58.608 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:51:28.666 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:51:28.671 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:51:58.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:51:58.734 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:52:28.792 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:52:28.796 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:52:58.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:52:58.880 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:53:28.936 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:53:28.941 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:53:59.000 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:53:59.004 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:54:19.751 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir0 1M 2023-12-19T16:54:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.907+0000 7ff663925640 1 Processor -- start 2023-12-19T16:54:19.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.907+0000 7ff663925640 1 -- start start 2023-12-19T16:54:19.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.907+0000 7ff663925640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff65c159820 0x7ff65c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:19.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.907+0000 7ff663925640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65c06fb60 con 0x7ff65c159c00 2023-12-19T16:54:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.907+0000 7ff663925640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65c06fe40 con 0x7ff65c06f6c0 2023-12-19T16:54:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.907+0000 7ff663925640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65c15e470 con 0x7ff65c159820 2023-12-19T16:54:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.907+0000 7ff6616c1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff65c159820 0x7ff65c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.907+0000 7ff661ec2640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff65c159c00 0x7ff65c070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59626/0 (socket says 172.21.15.26:59626) 2023-12-19T16:54:19.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.907+0000 7ff661ec2640 1 -- 172.21.15.26:0/2432713958 learned_addr learned my addr 172.21.15.26:0/2432713958 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.907+0000 7ff6616c1640 1 -- 172.21.15.26:0/2432713958 >> v1:172.21.15.160:6789/0 conn(0x7ff65c06f6c0 legacy=0x7ff65c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.907+0000 7ff6616c1640 1 -- 172.21.15.26:0/2432713958 >> v1:172.21.15.26:6789/0 conn(0x7ff65c159c00 legacy=0x7ff65c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.907+0000 7ff6616c1640 1 -- 172.21.15.26:0/2432713958 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c070120 con 0x7ff65c159820 2023-12-19T16:54:19.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.907+0000 7ff653fff640 1 -- 172.21.15.26:0/2432713958 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337142887 0 0) 0x7ff65c06fb60 con 0x7ff65c159c00 2023-12-19T16:54:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.911+0000 7ff6616c1640 1 --2- 172.21.15.26:0/2432713958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c159820 0x7ff65c15df30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff6580097c0 tx=0x7ff65801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c1a882453841c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.911+0000 7ff653fff640 1 -- 172.21.15.26:0/2432713958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6580ab650 con 0x7ff65c159820 2023-12-19T16:54:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.911+0000 7ff653fff640 1 -- 172.21.15.26:0/2432713958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6580ab7b0 con 0x7ff65c159820 2023-12-19T16:54:19.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.911+0000 7ff653fff640 1 -- 172.21.15.26:0/2432713958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6580abb40 con 0x7ff65c159820 2023-12-19T16:54:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.911+0000 7ff663925640 1 -- 172.21.15.26:0/2432713958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c159820 msgr2=0x7ff65c15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.911+0000 7ff663925640 1 --2- 172.21.15.26:0/2432713958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c159820 0x7ff65c15df30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff6580097c0 tx=0x7ff65801c450 comp rx=0 tx=0).stop 2023-12-19T16:54:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.911+0000 7ff663925640 1 -- 172.21.15.26:0/2432713958 shutdown_connections 2023-12-19T16:54:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.911+0000 7ff663925640 1 --2- 172.21.15.26:0/2432713958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c159820 0x7ff65c15df30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.911+0000 7ff663925640 1 -- 172.21.15.26:0/2432713958 >> 172.21.15.26:0/2432713958 conn(0x7ff65c0ce730 msgr2=0x7ff65c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.911+0000 7ff663925640 1 -- 172.21.15.26:0/2432713958 shutdown_connections 2023-12-19T16:54:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.911+0000 7ff663925640 1 -- 172.21.15.26:0/2432713958 wait complete. 2023-12-19T16:54:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff663925640 1 Processor -- start 2023-12-19T16:54:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff663925640 1 -- start start 2023-12-19T16:54:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff663925640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff65c06f6c0 0x7ff65c0c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff663925640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff65c159820 0x7ff65c1513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff663925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c159c00 0x7ff65c0ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff663925640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c151920 con 0x7ff65c159c00 2023-12-19T16:54:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff663925640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0c3c30 con 0x7ff65c159820 2023-12-19T16:54:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff663925640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65c0c3020 con 0x7ff65c06f6c0 2023-12-19T16:54:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff661ec2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c159c00 0x7ff65c0ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff661ec2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c159c00 0x7ff65c0ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50160/0 (socket says 172.21.15.26:50160) 2023-12-19T16:54:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff661ec2640 1 -- 172.21.15.26:0/3717908350 learned_addr learned my addr 172.21.15.26:0/3717908350 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff6616c1640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff65c06f6c0 0x7ff65c0c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff660ec0640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff65c159820 0x7ff65c1513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff661ec2640 1 -- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff65c06f6c0 msgr2=0x7ff65c0c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff661ec2640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff65c06f6c0 0x7ff65c0c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff661ec2640 1 -- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff65c159820 msgr2=0x7ff65c1513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff661ec2640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff65c159820 0x7ff65c1513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff661ec2640 1 -- 172.21.15.26:0/3717908350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff658009420 con 0x7ff65c159c00 2023-12-19T16:54:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff661ec2640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c159c00 0x7ff65c0ca4f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c7b0 tx=0x7ff65400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff651ffb640 1 -- 172.21.15.26:0/3717908350 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65400a640 con 0x7ff65c159c00 2023-12-19T16:54:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff651ffb640 1 -- 172.21.15.26:0/3717908350 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65400a7a0 con 0x7ff65c159c00 2023-12-19T16:54:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff651ffb640 1 -- 172.21.15.26:0/3717908350 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65400ab90 con 0x7ff65c159c00 2023-12-19T16:54:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff663925640 1 -- 172.21.15.26:0/3717908350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff65c06b880 con 0x7ff65c159c00 2023-12-19T16:54:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.915+0000 7ff663925640 1 -- 172.21.15.26:0/3717908350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65c0c35e0 con 0x7ff65c159c00 2023-12-19T16:54:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.919+0000 7ff663925640 1 -- 172.21.15.26:0/3717908350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65c074270 con 0x7ff65c159c00 2023-12-19T16:54:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.919+0000 7ff651ffb640 1 -- 172.21.15.26:0/3717908350 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7ff65400acf0 con 0x7ff65c159c00 2023-12-19T16:54:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.919+0000 7ff651ffb640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff640065c90 0x7ff640068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.919+0000 7ff651ffb640 1 -- 172.21.15.26:0/3717908350 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff6540828a0 con 0x7ff65c159c00 2023-12-19T16:54:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.919+0000 7ff6616c1640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff640065c90 0x7ff640068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.919+0000 7ff6616c1640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff640065c90 0x7ff640068150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff6580af000 tx=0x7ff6580a7000 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:19.923+0000 7ff651ffb640 1 -- 172.21.15.26:0/3717908350 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff654050040 con 0x7ff65c159c00 2023-12-19T16:54:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.099+0000 7ff663925640 1 -- 172.21.15.26:0/3717908350 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff65c06b700 con 0x7ff640065c90 2023-12-19T16:54:20.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.171+0000 7ff651ffb640 1 -- 172.21.15.26:0/3717908350 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff658004090 con 0x7ff640065c90 2023-12-19T16:54:20.173 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-12-19T16:54:20.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.171+0000 7ff663925640 1 -- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff640065c90 msgr2=0x7ff640068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:20.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.171+0000 7ff663925640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff640065c90 0x7ff640068150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff6580af000 tx=0x7ff6580a7000 comp rx=0 tx=0).stop 2023-12-19T16:54:20.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.171+0000 7ff663925640 1 -- 172.21.15.26:0/3717908350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c159c00 msgr2=0x7ff65c0ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:20.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.171+0000 7ff663925640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c159c00 0x7ff65c0ca4f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c7b0 tx=0x7ff65400cc70 comp rx=0 tx=0).stop 2023-12-19T16:54:20.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.171+0000 7ff663925640 1 -- 172.21.15.26:0/3717908350 shutdown_connections 2023-12-19T16:54:20.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.171+0000 7ff663925640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff65c159c00 0x7ff65c0ca4f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.171+0000 7ff663925640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff65c159820 0x7ff65c1513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.171+0000 7ff663925640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff640065c90 0x7ff640068150 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.171+0000 7ff663925640 1 --2- 172.21.15.26:0/3717908350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff65c06f6c0 0x7ff65c0c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.171+0000 7ff663925640 1 -- 172.21.15.26:0/3717908350 >> 172.21.15.26:0/3717908350 conn(0x7ff65c0ce730 msgr2=0x7ff65c1551e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:20.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.171+0000 7ff663925640 1 -- 172.21.15.26:0/3717908350 shutdown_connections 2023-12-19T16:54:20.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.171+0000 7ff663925640 1 -- 172.21.15.26:0/3717908350 wait complete. 2023-12-19T16:54:20.194 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-12-19T16:54:20.194 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir1 1M 2023-12-19T16:54:20.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c270b6640 1 Processor -- start 2023-12-19T16:54:20.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c270b6640 1 -- start start 2023-12-19T16:54:20.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c270b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c20159820 0x7f9c2015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:20.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c270b6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9c2006f6c0 0x7f9c200c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:20.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c270b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c2006fb60 con 0x7f9c20159c00 2023-12-19T16:54:20.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c270b6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c2015e470 con 0x7f9c2006f6c0 2023-12-19T16:54:20.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c270b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2006fe40 con 0x7f9c20159820 2023-12-19T16:54:20.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c1ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9c2006f6c0 0x7f9c200c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:20.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c25653640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9c20159c00 0x7f9c20070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59636/0 (socket says 172.21.15.26:59636) 2023-12-19T16:54:20.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c25653640 1 -- 172.21.15.26:0/2261145068 learned_addr learned my addr 172.21.15.26:0/2261145068 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:20.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c24e52640 1 --2- 172.21.15.26:0/2261145068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c20159820 0x7f9c2015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:20.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c1ffff640 1 -- 172.21.15.26:0/2261145068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c20159820 msgr2=0x7f9c2015df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:20.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c1ffff640 1 --2- 172.21.15.26:0/2261145068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c20159820 0x7f9c2015df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c1ffff640 1 -- 172.21.15.26:0/2261145068 >> v1:172.21.15.26:6789/0 conn(0x7f9c20159c00 legacy=0x7f9c20070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:20.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c1ffff640 1 -- 172.21.15.26:0/2261145068 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c20070120 con 0x7f9c2006f6c0 2023-12-19T16:54:20.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.339+0000 7f9c1f7fe640 1 -- 172.21.15.26:0/2261145068 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309444838 0 0) 0x7f9c2006fb60 con 0x7f9c20159c00 2023-12-19T16:54:20.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.343+0000 7f9c1ffff640 1 --2- 172.21.15.26:0/2261145068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c2006f6c0 0x7f9c200c8e80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0097c0 tx=0x7f9c0c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3f0c52198a9395 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:20.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.343+0000 7f9c1f7fe640 1 -- 172.21.15.26:0/2261145068 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0ab650 con 0x7f9c2006f6c0 2023-12-19T16:54:20.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.343+0000 7f9c1f7fe640 1 -- 172.21.15.26:0/2261145068 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c0ab7b0 con 0x7f9c2006f6c0 2023-12-19T16:54:20.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.343+0000 7f9c1f7fe640 1 -- 172.21.15.26:0/2261145068 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0abb40 con 0x7f9c2006f6c0 2023-12-19T16:54:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.343+0000 7f9c270b6640 1 -- 172.21.15.26:0/2261145068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c2006f6c0 msgr2=0x7f9c200c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.343+0000 7f9c270b6640 1 --2- 172.21.15.26:0/2261145068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c2006f6c0 0x7f9c200c8e80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0097c0 tx=0x7f9c0c01c450 comp rx=0 tx=0).stop 2023-12-19T16:54:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.343+0000 7f9c270b6640 1 -- 172.21.15.26:0/2261145068 shutdown_connections 2023-12-19T16:54:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.343+0000 7f9c270b6640 1 --2- 172.21.15.26:0/2261145068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c2006f6c0 0x7f9c200c8e80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.343+0000 7f9c270b6640 1 --2- 172.21.15.26:0/2261145068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c20159820 0x7f9c2015df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.343+0000 7f9c270b6640 1 -- 172.21.15.26:0/2261145068 >> 172.21.15.26:0/2261145068 conn(0x7f9c200ce730 msgr2=0x7f9c200ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.343+0000 7f9c270b6640 1 -- 172.21.15.26:0/2261145068 shutdown_connections 2023-12-19T16:54:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.343+0000 7f9c270b6640 1 -- 172.21.15.26:0/2261145068 wait complete. 2023-12-19T16:54:20.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c270b6640 1 Processor -- start 2023-12-19T16:54:20.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c270b6640 1 -- start start 2023-12-19T16:54:20.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c270b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c2006f6c0 0x7f9c20150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:20.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c270b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c20159820 0x7f9c20159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:20.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c270b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c20159c00 0x7f9c2015a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:20.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c270b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c200c58a0 con 0x7f9c20159820 2023-12-19T16:54:20.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c270b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c200c5b80 con 0x7f9c20159c00 2023-12-19T16:54:20.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c270b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c200c5e60 con 0x7f9c2006f6c0 2023-12-19T16:54:20.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c20159820 0x7f9c20159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:20.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c1ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c20159820 0x7f9c20159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50176/0 (socket says 172.21.15.26:50176) 2023-12-19T16:54:20.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c1ffff640 1 -- 172.21.15.26:0/3464812466 learned_addr learned my addr 172.21.15.26:0/3464812466 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:20.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c24e52640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c2006f6c0 0x7f9c20150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:20.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c25653640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c20159c00 0x7f9c2015a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:20.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c1ffff640 1 -- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c2006f6c0 msgr2=0x7f9c20150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:20.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c1ffff640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c2006f6c0 0x7f9c20150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c1ffff640 1 -- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c20159c00 msgr2=0x7f9c2015a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:20.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c1ffff640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c20159c00 0x7f9c2015a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c1ffff640 1 -- 172.21.15.26:0/3464812466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c009420 con 0x7f9c20159820 2023-12-19T16:54:20.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c1ffff640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c20159820 0x7f9c20159fe0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0ad040 tx=0x7f9c0c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:20.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c1d7fa640 1 -- 172.21.15.26:0/3464812466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0b5810 con 0x7f9c20159820 2023-12-19T16:54:20.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c1d7fa640 1 -- 172.21.15.26:0/3464812466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c0b5990 con 0x7f9c20159820 2023-12-19T16:54:20.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c270b6640 1 -- 172.21.15.26:0/3464812466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2006b880 con 0x7f9c20159820 2023-12-19T16:54:20.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c1d7fa640 1 -- 172.21.15.26:0/3464812466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0b5d40 con 0x7f9c20159820 2023-12-19T16:54:20.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.347+0000 7f9c270b6640 1 -- 172.21.15.26:0/3464812466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c200c6420 con 0x7f9c20159820 2023-12-19T16:54:20.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.351+0000 7f9c270b6640 1 -- 172.21.15.26:0/3464812466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c20074270 con 0x7f9c20159820 2023-12-19T16:54:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.351+0000 7f9c1d7fa640 1 -- 172.21.15.26:0/3464812466 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f9c0c0bf410 con 0x7f9c20159820 2023-12-19T16:54:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.351+0000 7f9c1d7fa640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c04065c60 0x7f9c04068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:20.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.351+0000 7f9c1d7fa640 1 -- 172.21.15.26:0/3464812466 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9c0c11e610 con 0x7f9c20159820 2023-12-19T16:54:20.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.351+0000 7f9c24e52640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c04065c60 0x7f9c04068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:20.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.351+0000 7f9c24e52640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c04065c60 0x7f9c04068120 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400b9d0 tx=0x7f9c14008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.355+0000 7f9c1d7fa640 1 -- 172.21.15.26:0/3464812466 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9c0c0ebc40 con 0x7f9c20159820 2023-12-19T16:54:20.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.535+0000 7f9c270b6640 1 -- 172.21.15.26:0/3464812466 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c2006b700 con 0x7f9c04065c60 2023-12-19T16:54:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.599+0000 7f9c1d7fa640 1 -- 172.21.15.26:0/3464812466 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f9c1400b5c0 con 0x7f9c04065c60 2023-12-19T16:54:20.604 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-12-19T16:54:20.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.603+0000 7f9c270b6640 1 -- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c04065c60 msgr2=0x7f9c04068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.603+0000 7f9c270b6640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c04065c60 0x7f9c04068120 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400b9d0 tx=0x7f9c14008040 comp rx=0 tx=0).stop 2023-12-19T16:54:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.603+0000 7f9c270b6640 1 -- 172.21.15.26:0/3464812466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c20159820 msgr2=0x7f9c20159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.603+0000 7f9c270b6640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c20159820 0x7f9c20159fe0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0ad040 tx=0x7f9c0c01ac90 comp rx=0 tx=0).stop 2023-12-19T16:54:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.603+0000 7f9c270b6640 1 -- 172.21.15.26:0/3464812466 shutdown_connections 2023-12-19T16:54:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.603+0000 7f9c270b6640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c20159c00 0x7f9c2015a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.603+0000 7f9c270b6640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c20159820 0x7f9c20159fe0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.603+0000 7f9c270b6640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c04065c60 0x7f9c04068120 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.603+0000 7f9c270b6640 1 --2- 172.21.15.26:0/3464812466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c2006f6c0 0x7f9c20150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.603+0000 7f9c270b6640 1 -- 172.21.15.26:0/3464812466 >> 172.21.15.26:0/3464812466 conn(0x7f9c200ce730 msgr2=0x7f9c201515b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.603+0000 7f9c270b6640 1 -- 172.21.15.26:0/3464812466 shutdown_connections 2023-12-19T16:54:20.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.603+0000 7f9c270b6640 1 -- 172.21.15.26:0/3464812466 wait complete. 2023-12-19T16:54:20.625 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-12-19T16:54:20.625 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir2 1M 2023-12-19T16:54:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.767+0000 7fb6828e5640 1 Processor -- start 2023-12-19T16:54:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.767+0000 7fb6828e5640 1 -- start start 2023-12-19T16:54:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.767+0000 7fb6828e5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb67c0cd2d0 0x7fb67c0ad270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.767+0000 7fb6828e5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb67c0ad7b0 0x7fb67c0ca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.767+0000 7fb6828e5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c0ae260 con 0x7fb67c145ed0 2023-12-19T16:54:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.767+0000 7fb6828e5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67c0adf80 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.767+0000 7fb6828e5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67c0c9770 con 0x7fb67c0cd2d0 2023-12-19T16:54:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.771+0000 7fb680e82640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb67c0ad7b0 0x7fb67c0ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.771+0000 7fb680e82640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb67c0ad7b0 0x7fb67c0ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50182/0 (socket says 172.21.15.26:50182) 2023-12-19T16:54:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.771+0000 7fb680e82640 1 -- 172.21.15.26:0/1410010424 learned_addr learned my addr 172.21.15.26:0/1410010424 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:20.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.771+0000 7fb67b7fe640 1 --2- 172.21.15.26:0/1410010424 >> v2:172.21.15.160:3300/0 conn(0x7fb67c0cd2d0 0x7fb67c0ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.771+0000 7fb680e82640 1 -- 172.21.15.26:0/1410010424 >> v2:172.21.15.160:3300/0 conn(0x7fb67c0cd2d0 msgr2=0x7fb67c0ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.771+0000 7fb680e82640 1 --2- 172.21.15.26:0/1410010424 >> v2:172.21.15.160:3300/0 conn(0x7fb67c0cd2d0 0x7fb67c0ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.771+0000 7fb680e82640 1 -- 172.21.15.26:0/1410010424 >> v1:172.21.15.160:6789/0 conn(0x7fb67c145ed0 legacy=0x7fb67c0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.771+0000 7fb680e82640 1 -- 172.21.15.26:0/1410010424 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c0c9a50 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.771+0000 7fb67b7fe640 1 --2- 172.21.15.26:0/1410010424 >> v2:172.21.15.160:3300/0 conn(0x7fb67c0cd2d0 0x7fb67c0ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:54:20.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.771+0000 7fb680e82640 1 --2- 172.21.15.26:0/1410010424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb67c0ad7b0 0x7fb67c0ca250 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb67c066ac0 tx=0x7fb67401c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afd87e1e9d44771 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:20.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.771+0000 7fb67affd640 1 -- 172.21.15.26:0/1410010424 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6740a5660 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.771+0000 7fb67affd640 1 -- 172.21.15.26:0/1410010424 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6740a57c0 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.771+0000 7fb67affd640 1 -- 172.21.15.26:0/1410010424 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6740a5b50 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 -- 172.21.15.26:0/1410010424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb67c0ad7b0 msgr2=0x7fb67c0ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:20.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 --2- 172.21.15.26:0/1410010424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb67c0ad7b0 0x7fb67c0ca250 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb67c066ac0 tx=0x7fb67401c840 comp rx=0 tx=0).stop 2023-12-19T16:54:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 -- 172.21.15.26:0/1410010424 shutdown_connections 2023-12-19T16:54:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 --2- 172.21.15.26:0/1410010424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb67c0ad7b0 0x7fb67c0ca250 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 --2- 172.21.15.26:0/1410010424 >> v2:172.21.15.160:3300/0 conn(0x7fb67c0cd2d0 0x7fb67c0ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 -- 172.21.15.26:0/1410010424 >> 172.21.15.26:0/1410010424 conn(0x7fb67c0b0b60 msgr2=0x7fb67c0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 -- 172.21.15.26:0/1410010424 shutdown_connections 2023-12-19T16:54:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 -- 172.21.15.26:0/1410010424 wait complete. 2023-12-19T16:54:20.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 Processor -- start 2023-12-19T16:54:20.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 -- start start 2023-12-19T16:54:20.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb67c0ad7b0 0x7fb67c0c1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb67c0cd2d0 0x7fb67c15d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb67c145ed0 0x7fb67c15d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c14c3e0 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c14c6c0 con 0x7fb67c0cd2d0 2023-12-19T16:54:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb6828e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c14c9a0 con 0x7fb67c145ed0 2023-12-19T16:54:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb67bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb67c0ad7b0 0x7fb67c0c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb67b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb67c0cd2d0 0x7fb67c15d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb67bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb67c0ad7b0 0x7fb67c0c1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50198/0 (socket says 172.21.15.26:50198) 2023-12-19T16:54:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb67bfff640 1 -- 172.21.15.26:0/2510026545 learned_addr learned my addr 172.21.15.26:0/2510026545 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.775+0000 7fb680e82640 1 --2- 172.21.15.26:0/2510026545 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb67c145ed0 0x7fb67c15d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb67bfff640 1 -- 172.21.15.26:0/2510026545 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb67c145ed0 msgr2=0x7fb67c15d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb67bfff640 1 --2- 172.21.15.26:0/2510026545 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb67c145ed0 0x7fb67c15d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb67bfff640 1 -- 172.21.15.26:0/2510026545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb67c0cd2d0 msgr2=0x7fb67c15d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:20.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb67bfff640 1 --2- 172.21.15.26:0/2510026545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb67c0cd2d0 0x7fb67c15d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:20.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb67bfff640 1 -- 172.21.15.26:0/2510026545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb674009b30 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb67bfff640 1 --2- 172.21.15.26:0/2510026545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb67c0ad7b0 0x7fb67c0c1640 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00c450 tx=0x7fb66c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb678ff9640 1 -- 172.21.15.26:0/2510026545 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c012830 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb678ff9640 1 -- 172.21.15.26:0/2510026545 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66c0129b0 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb678ff9640 1 -- 172.21.15.26:0/2510026545 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66c012d60 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb6828e5640 1 -- 172.21.15.26:0/2510026545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c14cc80 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb6828e5640 1 -- 172.21.15.26:0/2510026545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c0c1b80 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb6828e5640 1 -- 172.21.15.26:0/2510026545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67c074270 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb678ff9640 1 -- 172.21.15.26:0/2510026545 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fb66c01c460 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb678ff9640 1 --2- 172.21.15.26:0/2510026545 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb668065c60 0x7fb668068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.779+0000 7fb678ff9640 1 -- 172.21.15.26:0/2510026545 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb66c0832d0 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.783+0000 7fb67b7fe640 1 --2- 172.21.15.26:0/2510026545 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb668065c60 0x7fb668068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:20.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.783+0000 7fb67b7fe640 1 --2- 172.21.15.26:0/2510026545 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb668065c60 0x7fb668068120 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0cde20 tx=0x7fb670008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:20.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.783+0000 7fb678ff9640 1 -- 172.21.15.26:0/2510026545 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb66c050a50 con 0x7fb67c0ad7b0 2023-12-19T16:54:20.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:20.959+0000 7fb6828e5640 1 -- 172.21.15.26:0/2510026545 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb67c14c830 con 0x7fb668065c60 2023-12-19T16:54:21.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.031+0000 7fb678ff9640 1 -- 172.21.15.26:0/2510026545 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb67c14c830 con 0x7fb668065c60 2023-12-19T16:54:21.033 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-12-19T16:54:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.031+0000 7fb6828e5640 1 -- 172.21.15.26:0/2510026545 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb668065c60 msgr2=0x7fb668068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.031+0000 7fb6828e5640 1 --2- 172.21.15.26:0/2510026545 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb668065c60 0x7fb668068120 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0cde20 tx=0x7fb670008040 comp rx=0 tx=0).stop 2023-12-19T16:54:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.031+0000 7fb6828e5640 1 -- 172.21.15.26:0/2510026545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb67c0ad7b0 msgr2=0x7fb67c0c1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.031+0000 7fb6828e5640 1 --2- 172.21.15.26:0/2510026545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb67c0ad7b0 0x7fb67c0c1640 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb66c00c450 tx=0x7fb66c00c910 comp rx=0 tx=0).stop 2023-12-19T16:54:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.031+0000 7fb6828e5640 1 -- 172.21.15.26:0/2510026545 shutdown_connections 2023-12-19T16:54:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.031+0000 7fb6828e5640 1 --2- 172.21.15.26:0/2510026545 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb668065c60 0x7fb668068120 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.031+0000 7fb6828e5640 1 --2- 172.21.15.26:0/2510026545 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb67c145ed0 0x7fb67c15d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.031+0000 7fb6828e5640 1 --2- 172.21.15.26:0/2510026545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb67c0cd2d0 0x7fb67c15d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.031+0000 7fb6828e5640 1 --2- 172.21.15.26:0/2510026545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb67c0ad7b0 0x7fb67c0c1640 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.031+0000 7fb6828e5640 1 -- 172.21.15.26:0/2510026545 >> 172.21.15.26:0/2510026545 conn(0x7fb67c0b0b60 msgr2=0x7fb67c06de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.031+0000 7fb6828e5640 1 -- 172.21.15.26:0/2510026545 shutdown_connections 2023-12-19T16:54:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.031+0000 7fb6828e5640 1 -- 172.21.15.26:0/2510026545 wait complete. 2023-12-19T16:54:21.055 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-12-19T16:54:21.055 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir3 1M 2023-12-19T16:54:21.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bd5c41640 1 Processor -- start 2023-12-19T16:54:21.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bd5c41640 1 -- start start 2023-12-19T16:54:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bd5c41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7bd015ada0 0x7f7bd00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:21.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bd5c41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bd006ac80 0x7f7bd00b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:21.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bd5c41640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd0070d90 con 0x7f7bd015a9c0 2023-12-19T16:54:21.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bd5c41640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd00cc270 con 0x7f7bd015ada0 2023-12-19T16:54:21.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bd5c41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd0071070 con 0x7f7bd006ac80 2023-12-19T16:54:21.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bceffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7bd015ada0 0x7f7bd00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:21.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bcf7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7bd015a9c0 0x7f7bd00b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59640/0 (socket says 172.21.15.26:59640) 2023-12-19T16:54:21.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bcf7fe640 1 -- 172.21.15.26:0/1867249406 learned_addr learned my addr 172.21.15.26:0/1867249406 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:21.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bcffff640 1 --2- 172.21.15.26:0/1867249406 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bd006ac80 0x7f7bd00b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:21.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bceffd640 1 -- 172.21.15.26:0/1867249406 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bd006ac80 msgr2=0x7f7bd00b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:21.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bceffd640 1 --2- 172.21.15.26:0/1867249406 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bd006ac80 0x7f7bd00b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bceffd640 1 -- 172.21.15.26:0/1867249406 >> v1:172.21.15.26:6789/0 conn(0x7f7bd015a9c0 legacy=0x7f7bd00b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bce7fc640 1 -- 172.21.15.26:0/1867249406 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883348198 0 0) 0x7f7bd0070d90 con 0x7f7bd015a9c0 2023-12-19T16:54:21.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.199+0000 7f7bceffd640 1 -- 172.21.15.26:0/1867249406 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd0071350 con 0x7f7bd015ada0 2023-12-19T16:54:21.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.203+0000 7f7bceffd640 1 --2- 172.21.15.26:0/1867249406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd015ada0 0x7f7bd00706d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40097c0 tx=0x7f7bc401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbce1912406b8b46 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:21.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.203+0000 7f7bce7fc640 1 -- 172.21.15.26:0/1867249406 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40ab600 con 0x7f7bd015ada0 2023-12-19T16:54:21.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.203+0000 7f7bce7fc640 1 -- 172.21.15.26:0/1867249406 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc40a7050 con 0x7f7bd015ada0 2023-12-19T16:54:21.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.203+0000 7f7bce7fc640 1 -- 172.21.15.26:0/1867249406 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40ab980 con 0x7f7bd015ada0 2023-12-19T16:54:21.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.203+0000 7f7bd5c41640 1 -- 172.21.15.26:0/1867249406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd015ada0 msgr2=0x7f7bd00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.203+0000 7f7bd5c41640 1 --2- 172.21.15.26:0/1867249406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd015ada0 0x7f7bd00706d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40097c0 tx=0x7f7bc401c3c0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.203+0000 7f7bd5c41640 1 -- 172.21.15.26:0/1867249406 shutdown_connections 2023-12-19T16:54:21.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.203+0000 7f7bd5c41640 1 --2- 172.21.15.26:0/1867249406 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bd006ac80 0x7f7bd00b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.203+0000 7f7bd5c41640 1 --2- 172.21.15.26:0/1867249406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd015ada0 0x7f7bd00706d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.203+0000 7f7bd5c41640 1 -- 172.21.15.26:0/1867249406 >> 172.21.15.26:0/1867249406 conn(0x7f7bd00ce730 msgr2=0x7f7bd00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:21.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.203+0000 7f7bd5c41640 1 -- 172.21.15.26:0/1867249406 shutdown_connections 2023-12-19T16:54:21.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.203+0000 7f7bd5c41640 1 -- 172.21.15.26:0/1867249406 wait complete. 2023-12-19T16:54:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bd5c41640 1 Processor -- start 2023-12-19T16:54:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bd5c41640 1 -- start start 2023-12-19T16:54:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bd5c41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd006ac80 0x7f7bd00af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bd5c41640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bd015a9c0 0x7f7bd0151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bd5c41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bd015ada0 0x7f7bd0151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bd5c41640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd00c5cc0 con 0x7f7bd006ac80 2023-12-19T16:54:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bd5c41640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd015d380 con 0x7f7bd015a9c0 2023-12-19T16:54:21.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bd5c41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd015c800 con 0x7f7bd015ada0 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bcf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd006ac80 0x7f7bd00af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bcf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd006ac80 0x7f7bd00af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50224/0 (socket says 172.21.15.26:50224) 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bcf7fe640 1 -- 172.21.15.26:0/608831139 learned_addr learned my addr 172.21.15.26:0/608831139 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bcffff640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bd015ada0 0x7f7bd0151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bceffd640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bd015a9c0 0x7f7bd0151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bcf7fe640 1 -- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bd015ada0 msgr2=0x7f7bd0151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bcf7fe640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bd015ada0 0x7f7bd0151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bcf7fe640 1 -- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bd015a9c0 msgr2=0x7f7bd0151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bcf7fe640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bd015a9c0 0x7f7bd0151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bcf7fe640 1 -- 172.21.15.26:0/608831139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc4009420 con 0x7f7bd006ac80 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bcffff640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bd015ada0 0x7f7bd0151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bcf7fe640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd006ac80 0x7f7bd00af0b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00011f0 tx=0x7f7bc000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bb7fff640 1 -- 172.21.15.26:0/608831139 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00074e0 con 0x7f7bd006ac80 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bb7fff640 1 -- 172.21.15.26:0/608831139 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc0007640 con 0x7f7bd006ac80 2023-12-19T16:54:21.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bb7fff640 1 -- 172.21.15.26:0/608831139 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0004a10 con 0x7f7bd006ac80 2023-12-19T16:54:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.207+0000 7f7bd5c41640 1 -- 172.21.15.26:0/608831139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd015cae0 con 0x7f7bd006ac80 2023-12-19T16:54:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.211+0000 7f7bd5c41640 1 -- 172.21.15.26:0/608831139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd015d0a0 con 0x7f7bd006ac80 2023-12-19T16:54:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.211+0000 7f7bd5c41640 1 -- 172.21.15.26:0/608831139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd00c5ec0 con 0x7f7bd006ac80 2023-12-19T16:54:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.211+0000 7f7bb7fff640 1 -- 172.21.15.26:0/608831139 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f7bc0004b70 con 0x7f7bd006ac80 2023-12-19T16:54:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.211+0000 7f7bb7fff640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7bbc065c60 0x7f7bbc068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:21.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.211+0000 7f7bb7fff640 1 -- 172.21.15.26:0/608831139 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f7bc00828a0 con 0x7f7bd006ac80 2023-12-19T16:54:21.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.211+0000 7f7bceffd640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7bbc065c60 0x7f7bbc068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:21.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.215+0000 7f7bceffd640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7bbc065c60 0x7f7bbc068120 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40ad000 tx=0x7f7bc4008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:21.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.215+0000 7f7bb7fff640 1 -- 172.21.15.26:0/608831139 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7bc004ff70 con 0x7f7bd006ac80 2023-12-19T16:54:21.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.399+0000 7f7bd5c41640 1 -- 172.21.15.26:0/608831139 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bd0167ac0 con 0x7f7bbc065c60 2023-12-19T16:54:21.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.463+0000 7f7bb7fff640 1 -- 172.21.15.26:0/608831139 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f7bc40ab6c0 con 0x7f7bbc065c60 2023-12-19T16:54:21.469 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-12-19T16:54:21.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.467+0000 7f7bd5c41640 1 -- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7bbc065c60 msgr2=0x7f7bbc068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.467+0000 7f7bd5c41640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7bbc065c60 0x7f7bbc068120 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40ad000 tx=0x7f7bc4008040 comp rx=0 tx=0).stop 2023-12-19T16:54:21.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.467+0000 7f7bd5c41640 1 -- 172.21.15.26:0/608831139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd006ac80 msgr2=0x7f7bd00af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.467+0000 7f7bd5c41640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd006ac80 0x7f7bd00af0b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00011f0 tx=0x7f7bc000c9b0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.467+0000 7f7bd5c41640 1 -- 172.21.15.26:0/608831139 shutdown_connections 2023-12-19T16:54:21.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.467+0000 7f7bd5c41640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7bbc065c60 0x7f7bbc068120 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.467+0000 7f7bd5c41640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bd015ada0 0x7f7bd0151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.467+0000 7f7bd5c41640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bd015a9c0 0x7f7bd0151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.467+0000 7f7bd5c41640 1 --2- 172.21.15.26:0/608831139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bd006ac80 0x7f7bd00af0b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.467+0000 7f7bd5c41640 1 -- 172.21.15.26:0/608831139 >> 172.21.15.26:0/608831139 conn(0x7f7bd00ce730 msgr2=0x7f7bd00ae160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:21.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.467+0000 7f7bd5c41640 1 -- 172.21.15.26:0/608831139 shutdown_connections 2023-12-19T16:54:21.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.467+0000 7f7bd5c41640 1 -- 172.21.15.26:0/608831139 wait complete. 2023-12-19T16:54:21.490 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-12-19T16:54:21.490 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir4 1M 2023-12-19T16:54:21.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.635+0000 7f128c198640 1 Processor -- start 2023-12-19T16:54:21.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.635+0000 7f128c198640 1 -- start start 2023-12-19T16:54:21.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f128c198640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f128415ada0 0x7f12840706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:21.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f128c198640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128406ac80 0x7f12840b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:21.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f128c198640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1284070d90 con 0x7f128415a9c0 2023-12-19T16:54:21.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f128c198640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12840cc270 con 0x7f128415ada0 2023-12-19T16:54:21.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f128c198640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1284071070 con 0x7f128406ac80 2023-12-19T16:54:21.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f1289f34640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f128415a9c0 0x7f12840b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59650/0 (socket says 172.21.15.26:59650) 2023-12-19T16:54:21.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f1289f34640 1 -- 172.21.15.26:0/1375231037 learned_addr learned my addr 172.21.15.26:0/1375231037 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:21.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f1289733640 1 --2- 172.21.15.26:0/1375231037 >> v2:172.21.15.160:3300/0 conn(0x7f128415ada0 0x7f12840706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:21.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f128a735640 1 --2- 172.21.15.26:0/1375231037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128406ac80 0x7f12840b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:21.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f1288f32640 1 -- 172.21.15.26:0/1375231037 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574988067 0 0) 0x7f1284070d90 con 0x7f128415a9c0 2023-12-19T16:54:21.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f1288f32640 1 -- 172.21.15.26:0/1375231037 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1270003530 con 0x7f128415a9c0 2023-12-19T16:54:21.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f1289733640 1 -- 172.21.15.26:0/1375231037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128406ac80 msgr2=0x7f12840b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:21.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f1289733640 1 --2- 172.21.15.26:0/1375231037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128406ac80 0x7f12840b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f1289733640 1 -- 172.21.15.26:0/1375231037 >> v1:172.21.15.26:6789/0 conn(0x7f128415a9c0 legacy=0x7f12840b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f1288f32640 1 -- 172.21.15.26:0/1375231037 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1389249998 0 0) 0x7f1270003530 con 0x7f128415a9c0 2023-12-19T16:54:21.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.639+0000 7f1289733640 1 -- 172.21.15.26:0/1375231037 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1284071350 con 0x7f128415ada0 2023-12-19T16:54:21.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f1289733640 1 --2- 172.21.15.26:0/1375231037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f128415ada0 0x7f12840706d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f12740097c0 tx=0x7f127401c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=931ef41e5de08ec5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:21.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f1288f32640 1 -- 172.21.15.26:0/1375231037 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280001680 con 0x7f128415ada0 2023-12-19T16:54:21.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f1288f32640 1 -- 172.21.15.26:0/1375231037 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12740a7050 con 0x7f128415ada0 2023-12-19T16:54:21.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f1288f32640 1 -- 172.21.15.26:0/1375231037 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12740ab620 con 0x7f128415ada0 2023-12-19T16:54:21.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 -- 172.21.15.26:0/1375231037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f128415ada0 msgr2=0x7f12840706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 --2- 172.21.15.26:0/1375231037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f128415ada0 0x7f12840706d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f12740097c0 tx=0x7f127401c2e0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 -- 172.21.15.26:0/1375231037 shutdown_connections 2023-12-19T16:54:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 --2- 172.21.15.26:0/1375231037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128406ac80 0x7f12840b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 --2- 172.21.15.26:0/1375231037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f128415ada0 0x7f12840706d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 -- 172.21.15.26:0/1375231037 >> 172.21.15.26:0/1375231037 conn(0x7f12840ce730 msgr2=0x7f12840ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 -- 172.21.15.26:0/1375231037 shutdown_connections 2023-12-19T16:54:21.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 -- 172.21.15.26:0/1375231037 wait complete. 2023-12-19T16:54:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 Processor -- start 2023-12-19T16:54:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 -- start start 2023-12-19T16:54:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f128406ac80 0x7f12840c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128415a9c0 0x7f12841584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:21.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f128415ada0 0x7f12841589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12840c70d0 con 0x7f128415ada0 2023-12-19T16:54:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1284159c30 con 0x7f128406ac80 2023-12-19T16:54:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.643+0000 7f128c198640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1284157d20 con 0x7f128415a9c0 2023-12-19T16:54:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f128a735640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f128415ada0 0x7f12841589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f128a735640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f128415ada0 0x7f12841589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50230/0 (socket says 172.21.15.26:50230) 2023-12-19T16:54:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f128a735640 1 -- 172.21.15.26:0/1410020622 learned_addr learned my addr 172.21.15.26:0/1410020622 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:21.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f1289733640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128415a9c0 0x7f12841584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:21.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f1289f34640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f128406ac80 0x7f12840c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:21.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f128a735640 1 -- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128415a9c0 msgr2=0x7f12841584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f128a735640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128415a9c0 0x7f12841584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f128a735640 1 -- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f128406ac80 msgr2=0x7f12840c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f128a735640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f128406ac80 0x7f12840c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f128a735640 1 -- 172.21.15.26:0/1410020622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1274009420 con 0x7f128415ada0 2023-12-19T16:54:21.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f1289f34640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f128406ac80 0x7f12840c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:21.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f128a735640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f128415ada0 0x7f12841589f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f127c00efc0 tx=0x7f127c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:21.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f127a7fc640 1 -- 172.21.15.26:0/1410020622 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c014480 con 0x7f128415ada0 2023-12-19T16:54:21.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f127a7fc640 1 -- 172.21.15.26:0/1410020622 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c00f040 con 0x7f128415ada0 2023-12-19T16:54:21.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f128c198640 1 -- 172.21.15.26:0/1410020622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1284159e80 con 0x7f128415ada0 2023-12-19T16:54:21.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f127a7fc640 1 -- 172.21.15.26:0/1410020622 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c014820 con 0x7f128415ada0 2023-12-19T16:54:21.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f128c198640 1 -- 172.21.15.26:0/1410020622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128415a160 con 0x7f128415ada0 2023-12-19T16:54:21.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f128c198640 1 -- 172.21.15.26:0/1410020622 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1284074270 con 0x7f128415ada0 2023-12-19T16:54:21.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.647+0000 7f127a7fc640 1 -- 172.21.15.26:0/1410020622 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f127c014980 con 0x7f128415ada0 2023-12-19T16:54:21.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.651+0000 7f127a7fc640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1268065c60 0x7f1268068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:21.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.651+0000 7f127a7fc640 1 -- 172.21.15.26:0/1410020622 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f127c0841f0 con 0x7f128415ada0 2023-12-19T16:54:21.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.651+0000 7f1289f34640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1268065c60 0x7f1268068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:21.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.651+0000 7f1289f34640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1268065c60 0x7f1268068120 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1280005890 tx=0x7f1280009040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:21.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.651+0000 7f127a7fc640 1 -- 172.21.15.26:0/1410020622 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f127c051710 con 0x7f128415ada0 2023-12-19T16:54:21.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.831+0000 7f128c198640 1 -- 172.21.15.26:0/1410020622 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1284167ac0 con 0x7f1268065c60 2023-12-19T16:54:21.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.899+0000 7f127a7fc640 1 -- 172.21.15.26:0/1410020622 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f12800015e0 con 0x7f1268065c60 2023-12-19T16:54:21.902 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-12-19T16:54:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.899+0000 7f128c198640 1 -- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1268065c60 msgr2=0x7f1268068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.899+0000 7f128c198640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1268065c60 0x7f1268068120 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1280005890 tx=0x7f1280009040 comp rx=0 tx=0).stop 2023-12-19T16:54:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.899+0000 7f128c198640 1 -- 172.21.15.26:0/1410020622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f128415ada0 msgr2=0x7f12841589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.899+0000 7f128c198640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f128415ada0 0x7f12841589f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f127c00efc0 tx=0x7f127c00c490 comp rx=0 tx=0).stop 2023-12-19T16:54:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.899+0000 7f128c198640 1 -- 172.21.15.26:0/1410020622 shutdown_connections 2023-12-19T16:54:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.899+0000 7f128c198640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f128415ada0 0x7f12841589f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.899+0000 7f128c198640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1268065c60 0x7f1268068120 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.899+0000 7f128c198640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128415a9c0 0x7f12841584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.899+0000 7f128c198640 1 --2- 172.21.15.26:0/1410020622 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f128406ac80 0x7f12840c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.899+0000 7f128c198640 1 -- 172.21.15.26:0/1410020622 >> 172.21.15.26:0/1410020622 conn(0x7f12840ce730 msgr2=0x7f1284156b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.899+0000 7f128c198640 1 -- 172.21.15.26:0/1410020622 shutdown_connections 2023-12-19T16:54:21.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:21.899+0000 7f128c198640 1 -- 172.21.15.26:0/1410020622 wait complete. 2023-12-19T16:54:21.924 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-12-19T16:54:21.924 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir5 1M 2023-12-19T16:54:22.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.067+0000 7fbe99441640 1 Processor -- start 2023-12-19T16:54:22.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.067+0000 7fbe99441640 1 -- start start 2023-12-19T16:54:22.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe99441640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe94159c00 0x7fbe9415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe99441640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9406fb60 con 0x7fbe9406f6c0 2023-12-19T16:54:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe99441640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9406fe40 con 0x7fbe94159820 2023-12-19T16:54:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe99441640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9415e470 con 0x7fbe94159c00 2023-12-19T16:54:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe8bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbe9406f6c0 0x7fbe94070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59652/0 (socket says 172.21.15.26:59652) 2023-12-19T16:54:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe8bfff640 1 -- 172.21.15.26:0/2549747022 learned_addr learned my addr 172.21.15.26:0/2549747022 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe937fe640 1 --2- 172.21.15.26:0/2549747022 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe94159c00 0x7fbe9415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554857160 0 0) 0x7fbe9406fb60 con 0x7fbe9406f6c0 2023-12-19T16:54:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe80003530 con 0x7fbe9406f6c0 2023-12-19T16:54:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1904826500 0 0) 0x7fbe80003530 con 0x7fbe9406f6c0 2023-12-19T16:54:22.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe94159c00 msgr2=0x7fbe9415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:22.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 --2- 172.21.15.26:0/2549747022 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe94159c00 0x7fbe9415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 >> v1:172.21.15.160:6789/0 conn(0x7fbe94159820 legacy=0x7fbe940c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe94070120 con 0x7fbe9406f6c0 2023-12-19T16:54:22.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fbe7c001680 con 0x7fbe9406f6c0 2023-12-19T16:54:22.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 >> v1:172.21.15.26:6789/0 conn(0x7fbe9406f6c0 legacy=0x7fbe94070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 --2- 172.21.15.26:0/2549747022 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe94159c00 0x7fbe800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 --2- 172.21.15.26:0/2549747022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe80003f30 0x7fbe800047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 --2- 172.21.15.26:0/2549747022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe80004d10 0x7fbe80005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9415e470 con 0x7fbe80003f30 2023-12-19T16:54:22.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8000ba70 con 0x7fbe80004d10 2023-12-19T16:54:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8000bc30 con 0x7fbe94159c00 2023-12-19T16:54:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe8bfff640 1 --2- 172.21.15.26:0/2549747022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe80003f30 0x7fbe800047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe92ffd640 1 --2- 172.21.15.26:0/2549747022 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe94159c00 0x7fbe800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe937fe640 1 --2- 172.21.15.26:0/2549747022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe80004d10 0x7fbe80005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe8bfff640 1 -- 172.21.15.26:0/2549747022 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe94159c00 msgr2=0x7fbe800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe8bfff640 1 --2- 172.21.15.26:0/2549747022 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe94159c00 0x7fbe800039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe8bfff640 1 -- 172.21.15.26:0/2549747022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe80004d10 msgr2=0x7fbe80005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe8bfff640 1 --2- 172.21.15.26:0/2549747022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe80004d10 0x7fbe80005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe8bfff640 1 -- 172.21.15.26:0/2549747022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe80003530 con 0x7fbe80003f30 2023-12-19T16:54:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.071+0000 7fbe937fe640 1 --2- 172.21.15.26:0/2549747022 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe80004d10 0x7fbe80005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.075+0000 7fbe8bfff640 1 --2- 172.21.15.26:0/2549747022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe80003f30 0x7fbe800047d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00c210 tx=0x7fbe7c01f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d116f32b32abc7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.075+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c005ae0 con 0x7fbe80003f30 2023-12-19T16:54:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.075+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c0ab070 con 0x7fbe80003f30 2023-12-19T16:54:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.075+0000 7fbe927fc640 1 -- 172.21.15.26:0/2549747022 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c007070 con 0x7fbe80003f30 2023-12-19T16:54:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.075+0000 7fbe99441640 1 -- 172.21.15.26:0/2549747022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe80003f30 msgr2=0x7fbe800047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.075+0000 7fbe99441640 1 --2- 172.21.15.26:0/2549747022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe80003f30 0x7fbe800047d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00c210 tx=0x7fbe7c01f370 comp rx=0 tx=0).stop 2023-12-19T16:54:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.075+0000 7fbe937fe640 1 -- 172.21.15.26:0/2549747022 reap_dead start 2023-12-19T16:54:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.075+0000 7fbe99441640 1 -- 172.21.15.26:0/2549747022 shutdown_connections 2023-12-19T16:54:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.075+0000 7fbe99441640 1 -- 172.21.15.26:0/2549747022 >> 172.21.15.26:0/2549747022 conn(0x7fbe940ce730 msgr2=0x7fbe940ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.075+0000 7fbe99441640 1 -- 172.21.15.26:0/2549747022 shutdown_connections 2023-12-19T16:54:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.075+0000 7fbe99441640 1 -- 172.21.15.26:0/2549747022 wait complete. 2023-12-19T16:54:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe99441640 1 Processor -- start 2023-12-19T16:54:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe99441640 1 -- start start 2023-12-19T16:54:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe99441640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe80003f30 0x7fbe9415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe99441640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe94155ba0 0x7fbe940cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe99441640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe940ac860 0x7fbe940cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe99441640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940ad9e0 con 0x7fbe80003f30 2023-12-19T16:54:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe99441640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940acdd0 con 0x7fbe94155ba0 2023-12-19T16:54:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe99441640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940ad0b0 con 0x7fbe940ac860 2023-12-19T16:54:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe92ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe80003f30 0x7fbe9415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe92ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe80003f30 0x7fbe9415b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50244/0 (socket says 172.21.15.26:50244) 2023-12-19T16:54:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe92ffd640 1 -- 172.21.15.26:0/1748571459 learned_addr learned my addr 172.21.15.26:0/1748571459 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe937fe640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe940ac860 0x7fbe940cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe8bfff640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe94155ba0 0x7fbe940cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe92ffd640 1 -- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe940ac860 msgr2=0x7fbe940cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe92ffd640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe940ac860 0x7fbe940cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe92ffd640 1 -- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe94155ba0 msgr2=0x7fbe940cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe92ffd640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe94155ba0 0x7fbe940cd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe92ffd640 1 -- 172.21.15.26:0/1748571459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe940ad390 con 0x7fbe80003f30 2023-12-19T16:54:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe92ffd640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe80003f30 0x7fbe9415b4c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400f7f0 tx=0x7fbe8400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe8b7fe640 1 -- 172.21.15.26:0/1748571459 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe840050d0 con 0x7fbe80003f30 2023-12-19T16:54:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe8b7fe640 1 -- 172.21.15.26:0/1748571459 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe84005230 con 0x7fbe80003f30 2023-12-19T16:54:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe99441640 1 -- 172.21.15.26:0/1748571459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe940ad670 con 0x7fbe80003f30 2023-12-19T16:54:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe8b7fe640 1 -- 172.21.15.26:0/1748571459 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400d730 con 0x7fbe80003f30 2023-12-19T16:54:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe99441640 1 -- 172.21.15.26:0/1748571459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe940b0150 con 0x7fbe80003f30 2023-12-19T16:54:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.079+0000 7fbe99441640 1 -- 172.21.15.26:0/1748571459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe94074270 con 0x7fbe80003f30 2023-12-19T16:54:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.083+0000 7fbe8b7fe640 1 -- 172.21.15.26:0/1748571459 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fbe8400d890 con 0x7fbe80003f30 2023-12-19T16:54:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.083+0000 7fbe8b7fe640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbe74065c60 0x7fbe74068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.083+0000 7fbe8b7fe640 1 -- 172.21.15.26:0/1748571459 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fbe84083690 con 0x7fbe80003f30 2023-12-19T16:54:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.083+0000 7fbe8bfff640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbe74065c60 0x7fbe74068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.083+0000 7fbe8bfff640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbe74065c60 0x7fbe74068120 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c0af000 tx=0x7fbe7c0aa000 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.087+0000 7fbe8b7fe640 1 -- 172.21.15.26:0/1748571459 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbe84050c30 con 0x7fbe80003f30 2023-12-19T16:54:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.263+0000 7fbe99441640 1 -- 172.21.15.26:0/1748571459 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe9415bbd0 con 0x7fbe74065c60 2023-12-19T16:54:22.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.335+0000 7fbe8b7fe640 1 -- 172.21.15.26:0/1748571459 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fbe7c000ee0 con 0x7fbe74065c60 2023-12-19T16:54:22.340 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-12-19T16:54:22.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.339+0000 7fbe99441640 1 -- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbe74065c60 msgr2=0x7fbe74068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.339+0000 7fbe99441640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbe74065c60 0x7fbe74068120 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c0af000 tx=0x7fbe7c0aa000 comp rx=0 tx=0).stop 2023-12-19T16:54:22.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.339+0000 7fbe99441640 1 -- 172.21.15.26:0/1748571459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe80003f30 msgr2=0x7fbe9415b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.339+0000 7fbe99441640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe80003f30 0x7fbe9415b4c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400f7f0 tx=0x7fbe8400fcb0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.339+0000 7fbe99441640 1 -- 172.21.15.26:0/1748571459 shutdown_connections 2023-12-19T16:54:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.339+0000 7fbe99441640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbe74065c60 0x7fbe74068120 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.339+0000 7fbe99441640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe940ac860 0x7fbe940cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.339+0000 7fbe99441640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe94155ba0 0x7fbe940cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.339+0000 7fbe99441640 1 --2- 172.21.15.26:0/1748571459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe80003f30 0x7fbe9415b4c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.339+0000 7fbe99441640 1 -- 172.21.15.26:0/1748571459 >> 172.21.15.26:0/1748571459 conn(0x7fbe940ce730 msgr2=0x7fbe9406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.339+0000 7fbe99441640 1 -- 172.21.15.26:0/1748571459 shutdown_connections 2023-12-19T16:54:22.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.339+0000 7fbe99441640 1 -- 172.21.15.26:0/1748571459 wait complete. 2023-12-19T16:54:22.361 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-12-19T16:54:22.361 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir6 1M 2023-12-19T16:54:22.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01dc596640 1 Processor -- start 2023-12-19T16:54:22.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01dc596640 1 -- start start 2023-12-19T16:54:22.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01dc596640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01d415a9c0 0x7f01d40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01dc596640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f01d406ac80 0x7f01d406b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01dc596640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d4070d90 con 0x7f01d415ada0 2023-12-19T16:54:22.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01dc596640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d40cc270 con 0x7f01d415a9c0 2023-12-19T16:54:22.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01dc596640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d4071070 con 0x7f01d406ac80 2023-12-19T16:54:22.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01da332640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01d415a9c0 0x7f01d40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01d9b31640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f01d415ada0 0x7f01d40b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59660/0 (socket says 172.21.15.26:59660) 2023-12-19T16:54:22.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01d9b31640 1 -- 172.21.15.26:0/3403338994 learned_addr learned my addr 172.21.15.26:0/3403338994 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:22.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01dab33640 1 --2- 172.21.15.26:0/3403338994 >> v2:172.21.15.160:3300/0 conn(0x7f01d406ac80 0x7f01d406b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01da332640 1 -- 172.21.15.26:0/3403338994 >> v2:172.21.15.160:3300/0 conn(0x7f01d406ac80 msgr2=0x7f01d406b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:22.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01da332640 1 --2- 172.21.15.26:0/3403338994 >> v2:172.21.15.160:3300/0 conn(0x7f01d406ac80 0x7f01d406b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01da332640 1 -- 172.21.15.26:0/3403338994 >> v1:172.21.15.26:6789/0 conn(0x7f01d415ada0 legacy=0x7f01d40b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01da332640 1 -- 172.21.15.26:0/3403338994 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d4071350 con 0x7f01d415a9c0 2023-12-19T16:54:22.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01d9330640 1 -- 172.21.15.26:0/3403338994 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316976184 0 0) 0x7f01d4070d90 con 0x7f01d415ada0 2023-12-19T16:54:22.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.507+0000 7f01dab33640 1 --2- 172.21.15.26:0/3403338994 >> v2:172.21.15.160:3300/0 conn(0x7f01d406ac80 0x7f01d406b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:54:22.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.511+0000 7f01da332640 1 --2- 172.21.15.26:0/3403338994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d415a9c0 0x7f01d40706d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f01d40cc3b0 tx=0x7f01d001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88fe24d0264f6c30 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:22.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.511+0000 7f01d9330640 1 -- 172.21.15.26:0/3403338994 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00ac600 con 0x7f01d415a9c0 2023-12-19T16:54:22.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.511+0000 7f01d9330640 1 -- 172.21.15.26:0/3403338994 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d00a8050 con 0x7f01d415a9c0 2023-12-19T16:54:22.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.511+0000 7f01d9330640 1 -- 172.21.15.26:0/3403338994 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00ac980 con 0x7f01d415a9c0 2023-12-19T16:54:22.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.511+0000 7f01dc596640 1 -- 172.21.15.26:0/3403338994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d415a9c0 msgr2=0x7f01d40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.511+0000 7f01dc596640 1 --2- 172.21.15.26:0/3403338994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d415a9c0 0x7f01d40706d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f01d40cc3b0 tx=0x7f01d001c380 comp rx=0 tx=0).stop 2023-12-19T16:54:22.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.511+0000 7f01dc596640 1 -- 172.21.15.26:0/3403338994 shutdown_connections 2023-12-19T16:54:22.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.511+0000 7f01dc596640 1 --2- 172.21.15.26:0/3403338994 >> v2:172.21.15.160:3300/0 conn(0x7f01d406ac80 0x7f01d406b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.511+0000 7f01dc596640 1 --2- 172.21.15.26:0/3403338994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d415a9c0 0x7f01d40706d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.511+0000 7f01dc596640 1 -- 172.21.15.26:0/3403338994 >> 172.21.15.26:0/3403338994 conn(0x7f01d40ce730 msgr2=0x7f01d40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:22.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.511+0000 7f01dc596640 1 -- 172.21.15.26:0/3403338994 shutdown_connections 2023-12-19T16:54:22.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.511+0000 7f01dc596640 1 -- 172.21.15.26:0/3403338994 wait complete. 2023-12-19T16:54:22.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01dc596640 1 Processor -- start 2023-12-19T16:54:22.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01dc596640 1 -- start start 2023-12-19T16:54:22.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01dc596640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01d406ac80 0x7f01d40c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01dc596640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d415a9c0 0x7f01d4153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01dc596640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01d415ada0 0x7f01d40c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01dc596640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40c6a00 con 0x7f01d415a9c0 2023-12-19T16:54:22.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01dc596640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40b2200 con 0x7f01d415ada0 2023-12-19T16:54:22.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01dc596640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d40b2a60 con 0x7f01d406ac80 2023-12-19T16:54:22.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01d9b31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d415a9c0 0x7f01d4153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01d9b31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d415a9c0 0x7f01d4153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50252/0 (socket says 172.21.15.26:50252) 2023-12-19T16:54:22.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01d9b31640 1 -- 172.21.15.26:0/3841057366 learned_addr learned my addr 172.21.15.26:0/3841057366 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01da332640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01d406ac80 0x7f01d40c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01dab33640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01d415ada0 0x7f01d40c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01d9b31640 1 -- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01d406ac80 msgr2=0x7f01d40c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01d9b31640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01d406ac80 0x7f01d40c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01d9b31640 1 -- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01d415ada0 msgr2=0x7f01d40c1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01d9b31640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01d415ada0 0x7f01d40c1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01d9b31640 1 -- 172.21.15.26:0/3841057366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d0009420 con 0x7f01d415a9c0 2023-12-19T16:54:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01d9b31640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d415a9c0 0x7f01d4153730 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f01c400c480 tx=0x7f01c400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01caffd640 1 -- 172.21.15.26:0/3841057366 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c40074e0 con 0x7f01d415a9c0 2023-12-19T16:54:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01caffd640 1 -- 172.21.15.26:0/3841057366 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c4007660 con 0x7f01d415a9c0 2023-12-19T16:54:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01caffd640 1 -- 172.21.15.26:0/3841057366 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4004a30 con 0x7f01d415a9c0 2023-12-19T16:54:22.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01dc596640 1 -- 172.21.15.26:0/3841057366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d40c6720 con 0x7f01d415a9c0 2023-12-19T16:54:22.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.515+0000 7f01dc596640 1 -- 172.21.15.26:0/3841057366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d40b26f0 con 0x7f01d415a9c0 2023-12-19T16:54:22.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.519+0000 7f01dc596640 1 -- 172.21.15.26:0/3841057366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d40c85c0 con 0x7f01d415a9c0 2023-12-19T16:54:22.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.519+0000 7f01caffd640 1 -- 172.21.15.26:0/3841057366 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f01c4004b90 con 0x7f01d415a9c0 2023-12-19T16:54:22.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.519+0000 7f01caffd640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b8065c60 0x7f01b8068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.519+0000 7f01caffd640 1 -- 172.21.15.26:0/3841057366 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f01c4082720 con 0x7f01d415a9c0 2023-12-19T16:54:22.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.519+0000 7f01da332640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b8065c60 0x7f01b8068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.519+0000 7f01da332640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b8065c60 0x7f01b8068120 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f01d00ae000 tx=0x7f01d00a7000 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:22.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.523+0000 7f01caffd640 1 -- 172.21.15.26:0/3841057366 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f01c404feb0 con 0x7f01d415a9c0 2023-12-19T16:54:22.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.703+0000 7f01dc596640 1 -- 172.21.15.26:0/3841057366 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f01d4167ac0 con 0x7f01b8065c60 2023-12-19T16:54:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.771+0000 7f01caffd640 1 -- 172.21.15.26:0/3841057366 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f01d00bc0c0 con 0x7f01b8065c60 2023-12-19T16:54:22.775 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-12-19T16:54:22.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.775+0000 7f01dc596640 1 -- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b8065c60 msgr2=0x7f01b8068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.775+0000 7f01dc596640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b8065c60 0x7f01b8068120 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f01d00ae000 tx=0x7f01d00a7000 comp rx=0 tx=0).stop 2023-12-19T16:54:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.775+0000 7f01dc596640 1 -- 172.21.15.26:0/3841057366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d415a9c0 msgr2=0x7f01d4153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.775+0000 7f01dc596640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d415a9c0 0x7f01d4153730 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f01c400c480 tx=0x7f01c400c940 comp rx=0 tx=0).stop 2023-12-19T16:54:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.775+0000 7f01dc596640 1 -- 172.21.15.26:0/3841057366 shutdown_connections 2023-12-19T16:54:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.775+0000 7f01dc596640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01d415ada0 0x7f01d40c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.775+0000 7f01dc596640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01d415a9c0 0x7f01d4153730 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.775+0000 7f01dc596640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b8065c60 0x7f01b8068120 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.775+0000 7f01dc596640 1 --2- 172.21.15.26:0/3841057366 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01d406ac80 0x7f01d40c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.775+0000 7f01dc596640 1 -- 172.21.15.26:0/3841057366 >> 172.21.15.26:0/3841057366 conn(0x7f01d40ce730 msgr2=0x7f01d40af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.775+0000 7f01dc596640 1 -- 172.21.15.26:0/3841057366 shutdown_connections 2023-12-19T16:54:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.775+0000 7f01dc596640 1 -- 172.21.15.26:0/3841057366 wait complete. 2023-12-19T16:54:22.796 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-12-19T16:54:22.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir7 1M 2023-12-19T16:54:22.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.939+0000 7f5485420640 1 Processor -- start 2023-12-19T16:54:22.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.939+0000 7f5485420640 1 -- start start 2023-12-19T16:54:22.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f5485420640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5480145ed0 0x7f54800ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f5485420640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54800cd560 con 0x7f54800cd100 2023-12-19T16:54:22.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f5485420640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54800ae260 con 0x7f54800ad980 2023-12-19T16:54:22.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f5485420640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54800adf80 con 0x7f5480145ed0 2023-12-19T16:54:22.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f5477fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f54800cd100 0x7f54800ad270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59672/0 (socket says 172.21.15.26:59672) 2023-12-19T16:54:22.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f5477fff640 1 -- 172.21.15.26:0/3090990430 learned_addr learned my addr 172.21.15.26:0/3090990430 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:22.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f547effd640 1 --2- 172.21.15.26:0/3090990430 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5480145ed0 0x7f54800ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f547e7fc640 1 -- 172.21.15.26:0/3090990430 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2063939956 0 0) 0x7f54800cd560 con 0x7f54800cd100 2023-12-19T16:54:22.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f547e7fc640 1 -- 172.21.15.26:0/3090990430 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f546c003530 con 0x7f54800cd100 2023-12-19T16:54:22.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f547e7fc640 1 -- 172.21.15.26:0/3090990430 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1120095615 0 0) 0x7f54800ae260 con 0x7f54800ad980 2023-12-19T16:54:22.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f547e7fc640 1 -- 172.21.15.26:0/3090990430 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54800cd560 con 0x7f54800ad980 2023-12-19T16:54:22.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f547e7fc640 1 -- 172.21.15.26:0/3090990430 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1395277113 0 0) 0x7f546c003530 con 0x7f54800cd100 2023-12-19T16:54:22.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f547effd640 1 -- 172.21.15.26:0/3090990430 >> v1:172.21.15.160:6789/0 conn(0x7f54800ad980 legacy=0x7f54800ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f547effd640 1 -- 172.21.15.26:0/3090990430 >> v1:172.21.15.26:6789/0 conn(0x7f54800cd100 legacy=0x7f54800ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.943+0000 7f547effd640 1 -- 172.21.15.26:0/3090990430 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54800c9770 con 0x7f5480145ed0 2023-12-19T16:54:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f547effd640 1 --2- 172.21.15.26:0/3090990430 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5480145ed0 0x7f54800ccbc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f54700097f0 tx=0x7f547001c290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39eac64c61573652 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f547e7fc640 1 -- 172.21.15.26:0/3090990430 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468001680 con 0x7f5480145ed0 2023-12-19T16:54:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f547e7fc640 1 -- 172.21.15.26:0/3090990430 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54700a7050 con 0x7f5480145ed0 2023-12-19T16:54:22.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f547e7fc640 1 -- 172.21.15.26:0/3090990430 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54700ab620 con 0x7f5480145ed0 2023-12-19T16:54:22.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f5485420640 1 -- 172.21.15.26:0/3090990430 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5480145ed0 msgr2=0x7f54800ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f5485420640 1 --2- 172.21.15.26:0/3090990430 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5480145ed0 0x7f54800ccbc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f54700097f0 tx=0x7f547001c290 comp rx=0 tx=0).stop 2023-12-19T16:54:22.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f5485420640 1 -- 172.21.15.26:0/3090990430 shutdown_connections 2023-12-19T16:54:22.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f5485420640 1 --2- 172.21.15.26:0/3090990430 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5480145ed0 0x7f54800ccbc0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f5485420640 1 -- 172.21.15.26:0/3090990430 >> 172.21.15.26:0/3090990430 conn(0x7f54800b0b60 msgr2=0x7f54800b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:22.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f5485420640 1 -- 172.21.15.26:0/3090990430 shutdown_connections 2023-12-19T16:54:22.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f5485420640 1 -- 172.21.15.26:0/3090990430 wait complete. 2023-12-19T16:54:22.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f5485420640 1 Processor -- start 2023-12-19T16:54:22.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f5485420640 1 -- start start 2023-12-19T16:54:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f5485420640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54800ad980 0x7f54800c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f5485420640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54800cd100 0x7f548006cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.947+0000 7f5485420640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5480145ed0 0x7f548015f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5485420640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54800c7940 con 0x7f54800cd100 2023-12-19T16:54:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5485420640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548006d280 con 0x7f5480145ed0 2023-12-19T16:54:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5485420640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548006d4e0 con 0x7f54800ad980 2023-12-19T16:54:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5477fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54800cd100 0x7f548006cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5477fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54800cd100 0x7f548006cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50266/0 (socket says 172.21.15.26:50266) 2023-12-19T16:54:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5477fff640 1 -- 172.21.15.26:0/1443601108 learned_addr learned my addr 172.21.15.26:0/1443601108 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f547effd640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54800ad980 0x7f54800c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f547f7fe640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5480145ed0 0x7f548015f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5477fff640 1 -- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54800ad980 msgr2=0x7f54800c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5477fff640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54800ad980 0x7f54800c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5477fff640 1 -- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5480145ed0 msgr2=0x7f548015f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5477fff640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5480145ed0 0x7f548015f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5477fff640 1 -- 172.21.15.26:0/1443601108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470009420 con 0x7f54800cd100 2023-12-19T16:54:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5477fff640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54800cd100 0x7f548006cd40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f546800c750 tx=0x7f546800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:22.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f54777fe640 1 -- 172.21.15.26:0/1443601108 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468005af0 con 0x7f54800cd100 2023-12-19T16:54:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f54777fe640 1 -- 172.21.15.26:0/1443601108 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5468005c50 con 0x7f54800cd100 2023-12-19T16:54:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f54777fe640 1 -- 172.21.15.26:0/1443601108 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5468007070 con 0x7f54800cd100 2023-12-19T16:54:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5485420640 1 -- 172.21.15.26:0/1443601108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548006d7c0 con 0x7f54800cd100 2023-12-19T16:54:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5485420640 1 -- 172.21.15.26:0/1443601108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548006dd80 con 0x7f54800cd100 2023-12-19T16:54:22.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f5485420640 1 -- 172.21.15.26:0/1443601108 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5480074270 con 0x7f54800cd100 2023-12-19T16:54:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.951+0000 7f54777fe640 1 -- 172.21.15.26:0/1443601108 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f54680071d0 con 0x7f54800cd100 2023-12-19T16:54:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.955+0000 7f54777fe640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5460065c60 0x7f5460068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:22.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.955+0000 7f54777fe640 1 -- 172.21.15.26:0/1443601108 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f54680831b0 con 0x7f54800cd100 2023-12-19T16:54:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.955+0000 7f547effd640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5460065c60 0x7f5460068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:22.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.955+0000 7f547effd640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5460065c60 0x7f5460068120 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54700ad000 tx=0x7f54700a6000 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:22.959+0000 7f54777fe640 1 -- 172.21.15.26:0/1443601108 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5468050930 con 0x7f54800cd100 2023-12-19T16:54:23.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.135+0000 7f5485420640 1 -- 172.21.15.26:0/1443601108 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f54801462b0 con 0x7f5460065c60 2023-12-19T16:54:23.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.203+0000 7f54777fe640 1 -- 172.21.15.26:0/1443601108 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f54700b5640 con 0x7f5460065c60 2023-12-19T16:54:23.208 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-12-19T16:54:23.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.207+0000 7f5485420640 1 -- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5460065c60 msgr2=0x7f5460068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:23.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.207+0000 7f5485420640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5460065c60 0x7f5460068120 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f54700ad000 tx=0x7f54700a6000 comp rx=0 tx=0).stop 2023-12-19T16:54:23.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.207+0000 7f5485420640 1 -- 172.21.15.26:0/1443601108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54800cd100 msgr2=0x7f548006cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:23.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.207+0000 7f5485420640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54800cd100 0x7f548006cd40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f546800c750 tx=0x7f546800cc10 comp rx=0 tx=0).stop 2023-12-19T16:54:23.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.207+0000 7f5485420640 1 -- 172.21.15.26:0/1443601108 shutdown_connections 2023-12-19T16:54:23.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.207+0000 7f5485420640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5480145ed0 0x7f548015f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.207+0000 7f5485420640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54800cd100 0x7f548006cd40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.207+0000 7f5485420640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5460065c60 0x7f5460068120 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.207+0000 7f5485420640 1 --2- 172.21.15.26:0/1443601108 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54800ad980 0x7f54800c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.207+0000 7f5485420640 1 -- 172.21.15.26:0/1443601108 >> 172.21.15.26:0/1443601108 conn(0x7f54800b0b60 msgr2=0x7f54800b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:23.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.207+0000 7f5485420640 1 -- 172.21.15.26:0/1443601108 shutdown_connections 2023-12-19T16:54:23.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.207+0000 7f5485420640 1 -- 172.21.15.26:0/1443601108 wait complete. 2023-12-19T16:54:23.229 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-12-19T16:54:23.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir8 1M 2023-12-19T16:54:23.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc222339640 1 Processor -- start 2023-12-19T16:54:23.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc222339640 1 -- start start 2023-12-19T16:54:23.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc222339640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc21c15a9c0 0x7fc21c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc222339640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21c15ada0 0x7fc21c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc222339640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc21c06ac80 0x7fc21c06b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc222339640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21c0cc270 con 0x7fc21c06ac80 2023-12-19T16:54:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc222339640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21c0b46c0 con 0x7fc21c15a9c0 2023-12-19T16:54:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc222339640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c070d90 con 0x7fc21c15ada0 2023-12-19T16:54:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc2208d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc21c06ac80 0x7fc21c06b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc2208d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc21c06ac80 0x7fc21c06b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50272/0 (socket says 172.21.15.26:50272) 2023-12-19T16:54:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc2208d6640 1 -- 172.21.15.26:0/2541909554 learned_addr learned my addr 172.21.15.26:0/2541909554 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc21b7fe640 1 --2- 172.21.15.26:0/2541909554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21c15ada0 0x7fc21c0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc21bfff640 1 --2- 172.21.15.26:0/2541909554 >> v2:172.21.15.160:3300/0 conn(0x7fc21c15a9c0 0x7fc21c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc2208d6640 1 -- 172.21.15.26:0/2541909554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21c15ada0 msgr2=0x7fc21c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc2208d6640 1 --2- 172.21.15.26:0/2541909554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21c15ada0 0x7fc21c0b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc2208d6640 1 -- 172.21.15.26:0/2541909554 >> v2:172.21.15.160:3300/0 conn(0x7fc21c15a9c0 msgr2=0x7fc21c0706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc2208d6640 1 --2- 172.21.15.26:0/2541909554 >> v2:172.21.15.160:3300/0 conn(0x7fc21c15a9c0 0x7fc21c0706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.375+0000 7fc2208d6640 1 -- 172.21.15.26:0/2541909554 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c071070 con 0x7fc21c06ac80 2023-12-19T16:54:23.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.379+0000 7fc2208d6640 1 --2- 172.21.15.26:0/2541909554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c06ac80 0x7fc21c06b0b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0cf590 tx=0x7fc21401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa2f69177490db7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:23.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.379+0000 7fc21affd640 1 -- 172.21.15.26:0/2541909554 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140a5790 con 0x7fc21c06ac80 2023-12-19T16:54:23.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.379+0000 7fc21affd640 1 -- 172.21.15.26:0/2541909554 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2140a58f0 con 0x7fc21c06ac80 2023-12-19T16:54:23.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.379+0000 7fc21affd640 1 -- 172.21.15.26:0/2541909554 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140a5c80 con 0x7fc21c06ac80 2023-12-19T16:54:23.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.379+0000 7fc222339640 1 -- 172.21.15.26:0/2541909554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c06ac80 msgr2=0x7fc21c06b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.379+0000 7fc222339640 1 --2- 172.21.15.26:0/2541909554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c06ac80 0x7fc21c06b0b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0cf590 tx=0x7fc21401c910 comp rx=0 tx=0).stop 2023-12-19T16:54:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.379+0000 7fc222339640 1 -- 172.21.15.26:0/2541909554 shutdown_connections 2023-12-19T16:54:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.379+0000 7fc222339640 1 --2- 172.21.15.26:0/2541909554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c06ac80 0x7fc21c06b0b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.379+0000 7fc222339640 1 --2- 172.21.15.26:0/2541909554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21c15ada0 0x7fc21c0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.379+0000 7fc222339640 1 --2- 172.21.15.26:0/2541909554 >> v2:172.21.15.160:3300/0 conn(0x7fc21c15a9c0 0x7fc21c0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.379+0000 7fc222339640 1 -- 172.21.15.26:0/2541909554 >> 172.21.15.26:0/2541909554 conn(0x7fc21c0ce730 msgr2=0x7fc21c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.379+0000 7fc222339640 1 -- 172.21.15.26:0/2541909554 shutdown_connections 2023-12-19T16:54:23.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.379+0000 7fc222339640 1 -- 172.21.15.26:0/2541909554 wait complete. 2023-12-19T16:54:23.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc222339640 1 Processor -- start 2023-12-19T16:54:23.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc222339640 1 -- start start 2023-12-19T16:54:23.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc222339640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc21c06ac80 0x7fc21c14ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc222339640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c15a9c0 0x7fc21c0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc222339640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21c15ada0 0x7fc21c158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc222339640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c1549f0 con 0x7fc21c15a9c0 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc222339640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c154100 con 0x7fc21c06ac80 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc222339640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c1543a0 con 0x7fc21c15ada0 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc21b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c15a9c0 0x7fc21c0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc21b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c15a9c0 0x7fc21c0b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50276/0 (socket says 172.21.15.26:50276) 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc21b7fe640 1 -- 172.21.15.26:0/492084439 learned_addr learned my addr 172.21.15.26:0/492084439 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc21bfff640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc21c06ac80 0x7fc21c14ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc2208d6640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21c15ada0 0x7fc21c158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc21b7fe640 1 -- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21c15ada0 msgr2=0x7fc21c158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc21b7fe640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21c15ada0 0x7fc21c158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc21b7fe640 1 -- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc21c06ac80 msgr2=0x7fc21c14ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc21b7fe640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc21c06ac80 0x7fc21c14ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc21b7fe640 1 -- 172.21.15.26:0/492084439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214009b30 con 0x7fc21c15a9c0 2023-12-19T16:54:23.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc21b7fe640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c15a9c0 0x7fc21c0b2d50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d5f0 tx=0x7fc21000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:23.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc218ff9640 1 -- 172.21.15.26:0/492084439 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2100125f0 con 0x7fc21c15a9c0 2023-12-19T16:54:23.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc218ff9640 1 -- 172.21.15.26:0/492084439 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc210012750 con 0x7fc21c15a9c0 2023-12-19T16:54:23.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc218ff9640 1 -- 172.21.15.26:0/492084439 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210012ae0 con 0x7fc21c15a9c0 2023-12-19T16:54:23.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc222339640 1 -- 172.21.15.26:0/492084439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21c154680 con 0x7fc21c15a9c0 2023-12-19T16:54:23.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.383+0000 7fc222339640 1 -- 172.21.15.26:0/492084439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c154e80 con 0x7fc21c15a9c0 2023-12-19T16:54:23.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.387+0000 7fc222339640 1 -- 172.21.15.26:0/492084439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21c074270 con 0x7fc21c15a9c0 2023-12-19T16:54:23.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.387+0000 7fc218ff9640 1 -- 172.21.15.26:0/492084439 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fc210012c40 con 0x7fc21c15a9c0 2023-12-19T16:54:23.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.387+0000 7fc218ff9640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc208065c60 0x7fc208068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.387+0000 7fc218ff9640 1 -- 172.21.15.26:0/492084439 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc210083910 con 0x7fc21c15a9c0 2023-12-19T16:54:23.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.387+0000 7fc21bfff640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc208065c60 0x7fc208068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.387+0000 7fc21bfff640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc208065c60 0x7fc208068120 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc20c007a60 tx=0x7fc20c008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:23.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.391+0000 7fc218ff9640 1 -- 172.21.15.26:0/492084439 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc210050ff0 con 0x7fc21c15a9c0 2023-12-19T16:54:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.571+0000 7fc222339640 1 -- 172.21.15.26:0/492084439 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc21c0cd7b0 con 0x7fc208065c60 2023-12-19T16:54:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.639+0000 7fc218ff9640 1 -- 172.21.15.26:0/492084439 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc21c0cd7b0 con 0x7fc208065c60 2023-12-19T16:54:23.642 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-12-19T16:54:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.639+0000 7fc222339640 1 -- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc208065c60 msgr2=0x7fc208068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.639+0000 7fc222339640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc208065c60 0x7fc208068120 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc20c007a60 tx=0x7fc20c008040 comp rx=0 tx=0).stop 2023-12-19T16:54:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.639+0000 7fc222339640 1 -- 172.21.15.26:0/492084439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c15a9c0 msgr2=0x7fc21c0b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.639+0000 7fc222339640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c15a9c0 0x7fc21c0b2d50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc21000d5f0 tx=0x7fc21000dab0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.639+0000 7fc222339640 1 -- 172.21.15.26:0/492084439 shutdown_connections 2023-12-19T16:54:23.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.639+0000 7fc222339640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc208065c60 0x7fc208068120 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.639+0000 7fc222339640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc21c15ada0 0x7fc21c158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.639+0000 7fc222339640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc21c15a9c0 0x7fc21c0b2d50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.639+0000 7fc222339640 1 --2- 172.21.15.26:0/492084439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc21c06ac80 0x7fc21c14ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.639+0000 7fc222339640 1 -- 172.21.15.26:0/492084439 >> 172.21.15.26:0/492084439 conn(0x7fc21c0ce730 msgr2=0x7fc21c1564b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:23.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.639+0000 7fc222339640 1 -- 172.21.15.26:0/492084439 shutdown_connections 2023-12-19T16:54:23.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.639+0000 7fc222339640 1 -- 172.21.15.26:0/492084439 wait complete. 2023-12-19T16:54:23.663 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-12-19T16:54:23.663 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir9 1M 2023-12-19T16:54:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e2728c640 1 Processor -- start 2023-12-19T16:54:23.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e2728c640 1 -- start start 2023-12-19T16:54:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e2728c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e20161230 0x7f8e200ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e2728c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8e2006c3f0 0x7f8e2006c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e2728c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e20164e50 0x7f8e201652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e2728c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e200b01b0 con 0x7f8e20164e50 2023-12-19T16:54:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e2728c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e200adfb0 con 0x7f8e2006c3f0 2023-12-19T16:54:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e2728c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e200ca4e0 con 0x7f8e20161230 2023-12-19T16:54:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e25829640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e20164e50 0x7f8e201652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e25829640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e20164e50 0x7f8e201652e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50290/0 (socket says 172.21.15.26:50290) 2023-12-19T16:54:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e25829640 1 -- 172.21.15.26:0/3488430598 learned_addr learned my addr 172.21.15.26:0/3488430598 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:23.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e24827640 1 --2- 172.21.15.26:0/3488430598 >> v2:172.21.15.160:3300/0 conn(0x7f8e2006c3f0 0x7f8e2006c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e25028640 1 --2- 172.21.15.26:0/3488430598 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e20161230 0x7f8e200ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e25829640 1 -- 172.21.15.26:0/3488430598 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e20161230 msgr2=0x7f8e200ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:23.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e25829640 1 --2- 172.21.15.26:0/3488430598 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e20161230 0x7f8e200ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e25829640 1 -- 172.21.15.26:0/3488430598 >> v2:172.21.15.160:3300/0 conn(0x7f8e2006c3f0 msgr2=0x7f8e2006c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:23.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e25829640 1 --2- 172.21.15.26:0/3488430598 >> v2:172.21.15.160:3300/0 conn(0x7f8e2006c3f0 0x7f8e2006c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.807+0000 7f8e25829640 1 -- 172.21.15.26:0/3488430598 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e200b0400 con 0x7f8e20164e50 2023-12-19T16:54:23.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.811+0000 7f8e25829640 1 --2- 172.21.15.26:0/3488430598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e20164e50 0x7f8e201652e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8e200cf6a0 tx=0x7f8e1801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fe1eb2052ba651 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:23.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.811+0000 7f8e17fff640 1 -- 172.21.15.26:0/3488430598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180a5790 con 0x7f8e20164e50 2023-12-19T16:54:23.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.811+0000 7f8e17fff640 1 -- 172.21.15.26:0/3488430598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e200c9e00 con 0x7f8e20164e50 2023-12-19T16:54:23.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.811+0000 7f8e17fff640 1 -- 172.21.15.26:0/3488430598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180a5b10 con 0x7f8e20164e50 2023-12-19T16:54:23.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.811+0000 7f8e2728c640 1 -- 172.21.15.26:0/3488430598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e20164e50 msgr2=0x7f8e201652e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:23.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.811+0000 7f8e2728c640 1 --2- 172.21.15.26:0/3488430598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e20164e50 0x7f8e201652e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8e200cf6a0 tx=0x7f8e1801c910 comp rx=0 tx=0).stop 2023-12-19T16:54:23.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.811+0000 7f8e2728c640 1 -- 172.21.15.26:0/3488430598 shutdown_connections 2023-12-19T16:54:23.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.811+0000 7f8e2728c640 1 --2- 172.21.15.26:0/3488430598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e20164e50 0x7f8e201652e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.811+0000 7f8e2728c640 1 --2- 172.21.15.26:0/3488430598 >> v2:172.21.15.160:3300/0 conn(0x7f8e2006c3f0 0x7f8e2006c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.811+0000 7f8e2728c640 1 --2- 172.21.15.26:0/3488430598 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e20161230 0x7f8e200ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.811+0000 7f8e2728c640 1 -- 172.21.15.26:0/3488430598 >> 172.21.15.26:0/3488430598 conn(0x7f8e200ce730 msgr2=0x7f8e200ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.811+0000 7f8e2728c640 1 -- 172.21.15.26:0/3488430598 shutdown_connections 2023-12-19T16:54:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.811+0000 7f8e2728c640 1 -- 172.21.15.26:0/3488430598 wait complete. 2023-12-19T16:54:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e2728c640 1 Processor -- start 2023-12-19T16:54:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e2728c640 1 -- start start 2023-12-19T16:54:23.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e2728c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e2006c3f0 0x7f8e20152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e2728c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e20161230 0x7f8e2006f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e2728c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e20164e50 0x7f8e2006f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e2728c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2015b860 con 0x7f8e20164e50 2023-12-19T16:54:23.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e2728c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e200708b0 con 0x7f8e2006c3f0 2023-12-19T16:54:23.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e2728c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2006fca0 con 0x7f8e20161230 2023-12-19T16:54:23.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e25829640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e20164e50 0x7f8e2006f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e25829640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e20164e50 0x7f8e2006f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50302/0 (socket says 172.21.15.26:50302) 2023-12-19T16:54:23.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e25829640 1 -- 172.21.15.26:0/1862120182 learned_addr learned my addr 172.21.15.26:0/1862120182 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:23.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e24827640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e20161230 0x7f8e2006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e25028640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e2006c3f0 0x7f8e20152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e25829640 1 -- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e20161230 msgr2=0x7f8e2006f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:23.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e25829640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e20161230 0x7f8e2006f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e25829640 1 -- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e2006c3f0 msgr2=0x7f8e20152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e25829640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e2006c3f0 0x7f8e20152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e25829640 1 -- 172.21.15.26:0/1862120182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e18009b30 con 0x7f8e20164e50 2023-12-19T16:54:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e25829640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e20164e50 0x7f8e2006f660 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8e180af040 tx=0x7f8e180196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e15ffb640 1 -- 172.21.15.26:0/1862120182 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180b8640 con 0x7f8e20164e50 2023-12-19T16:54:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e15ffb640 1 -- 172.21.15.26:0/1862120182 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e180b87a0 con 0x7f8e20164e50 2023-12-19T16:54:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e2728c640 1 -- 172.21.15.26:0/1862120182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2006ff80 con 0x7f8e20164e50 2023-12-19T16:54:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e15ffb640 1 -- 172.21.15.26:0/1862120182 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e180b8b50 con 0x7f8e20164e50 2023-12-19T16:54:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.815+0000 7f8e2728c640 1 -- 172.21.15.26:0/1862120182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e20070540 con 0x7f8e20164e50 2023-12-19T16:54:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.819+0000 7f8e2728c640 1 -- 172.21.15.26:0/1862120182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e200b0970 con 0x7f8e20164e50 2023-12-19T16:54:23.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.819+0000 7f8e15ffb640 1 -- 172.21.15.26:0/1862120182 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f8e180b8cb0 con 0x7f8e20164e50 2023-12-19T16:54:23.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.819+0000 7f8e15ffb640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8e04065c60 0x7f8e04068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.819+0000 7f8e15ffb640 1 -- 172.21.15.26:0/1862120182 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8e1811e0f0 con 0x7f8e20164e50 2023-12-19T16:54:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.819+0000 7f8e25028640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8e04065c60 0x7f8e04068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:23.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.819+0000 7f8e25028640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8e04065c60 0x7f8e04068120 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c007940 tx=0x7f8e1c008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:23.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.823+0000 7f8e15ffb640 1 -- 172.21.15.26:0/1862120182 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8e180eb9a0 con 0x7f8e20164e50 2023-12-19T16:54:24.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:23.999+0000 7f8e2728c640 1 -- 172.21.15.26:0/1862120182 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e200c1ab0 con 0x7f8e04065c60 2023-12-19T16:54:24.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.067+0000 7f8e15ffb640 1 -- 172.21.15.26:0/1862120182 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f8e1c007590 con 0x7f8e04065c60 2023-12-19T16:54:24.072 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-12-19T16:54:24.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.071+0000 7f8e2728c640 1 -- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8e04065c60 msgr2=0x7f8e04068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.071+0000 7f8e2728c640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8e04065c60 0x7f8e04068120 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c007940 tx=0x7f8e1c008040 comp rx=0 tx=0).stop 2023-12-19T16:54:24.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.071+0000 7f8e2728c640 1 -- 172.21.15.26:0/1862120182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e20164e50 msgr2=0x7f8e2006f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.071+0000 7f8e2728c640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e20164e50 0x7f8e2006f660 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8e180af040 tx=0x7f8e180196b0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.071+0000 7f8e2728c640 1 -- 172.21.15.26:0/1862120182 shutdown_connections 2023-12-19T16:54:24.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.071+0000 7f8e2728c640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e20164e50 0x7f8e2006f660 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.071+0000 7f8e2728c640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8e04065c60 0x7f8e04068120 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.071+0000 7f8e2728c640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8e20161230 0x7f8e2006f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.071+0000 7f8e2728c640 1 --2- 172.21.15.26:0/1862120182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e2006c3f0 0x7f8e20152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.071+0000 7f8e2728c640 1 -- 172.21.15.26:0/1862120182 >> 172.21.15.26:0/1862120182 conn(0x7f8e200ce730 msgr2=0x7f8e200ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:24.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.071+0000 7f8e2728c640 1 -- 172.21.15.26:0/1862120182 shutdown_connections 2023-12-19T16:54:24.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.071+0000 7f8e2728c640 1 -- 172.21.15.26:0/1862120182 wait complete. 2023-12-19T16:54:24.093 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-12-19T16:54:24.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:24.093 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-12-19T16:54:24.105 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:24.105 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:24.105 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:24.105 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:24.105 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:24.105 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:24.105 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:24.110 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:24.111 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:24.111 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:24.111 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:24.112 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-19-16_44_00_UTC', 'scheduled-2023-12-19-16_45_00_UTC', 'scheduled-2023-12-19-16_46_00_UTC', 'scheduled-2023-12-19-16_47_00_UTC', 'scheduled-2023-12-19-16_48_00_UTC', 'scheduled-2023-12-19-16_49_00_UTC', 'scheduled-2023-12-19-16_50_00_UTC', 'scheduled-2023-12-19-16_51_00_UTC', 'scheduled-2023-12-19-16_52_00_UTC', 'scheduled-2023-12-19-16_53_00_UTC', 'scheduled-2023-12-19-16_54_00_UTC'] 2023-12-19T16:54:24.112 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir0 2023-12-19T16:54:24.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.295+0000 7f8f4b739640 1 Processor -- start 2023-12-19T16:54:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f4b739640 1 -- start start 2023-12-19T16:54:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f4b739640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8f441585f0 0x7f8f440cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f4b739640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f440cb150 0x7f8f440cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f4b739640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4415b1b0 con 0x7f8f440cc310 2023-12-19T16:54:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f4b739640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f440cbb10 con 0x7f8f441585f0 2023-12-19T16:54:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f4b739640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f440b47a0 con 0x7f8f440cb150 2023-12-19T16:54:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f494d5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8f441585f0 0x7f8f440cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f48cd4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f440cb150 0x7f8f440cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f49cd6640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f8f440cc310 0x7f8f440cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:48970/0 (socket says 172.21.15.26:48970) 2023-12-19T16:54:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f49cd6640 1 -- 172.21.15.26:0/2053854976 learned_addr learned my addr 172.21.15.26:0/2053854976 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f494d5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8f441585f0 0x7f8f440cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:49424/0 (socket says 172.21.15.26:49424) 2023-12-19T16:54:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f494d5640 1 -- 172.21.15.26:0/2053854976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f440cb150 msgr2=0x7f8f440cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f494d5640 1 --2- 172.21.15.26:0/2053854976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f440cb150 0x7f8f440cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f494d5640 1 -- 172.21.15.26:0/2053854976 >> v1:172.21.15.160:6789/0 conn(0x7f8f440cc310 legacy=0x7f8f440cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f494d5640 1 -- 172.21.15.26:0/2053854976 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4415b490 con 0x7f8f441585f0 2023-12-19T16:54:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f48cd4640 1 --2- 172.21.15.26:0/2053854976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f440cb150 0x7f8f440cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:54:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.299+0000 7f8f3bfff640 1 -- 172.21.15.26:0/2053854976 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39888387 0 0) 0x7f8f4415b1b0 con 0x7f8f440cc310 2023-12-19T16:54:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.303+0000 7f8f494d5640 1 --2- 172.21.15.26:0/2053854976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f441585f0 0x7f8f440cdce0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8f400097c0 tx=0x7f8f4001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16746ea792480b9a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.303+0000 7f8f3bfff640 1 -- 172.21.15.26:0/2053854976 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400ac620 con 0x7f8f441585f0 2023-12-19T16:54:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.303+0000 7f8f3bfff640 1 -- 172.21.15.26:0/2053854976 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f400ac780 con 0x7f8f441585f0 2023-12-19T16:54:24.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.303+0000 7f8f3bfff640 1 -- 172.21.15.26:0/2053854976 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400acb10 con 0x7f8f441585f0 2023-12-19T16:54:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.303+0000 7f8f4b739640 1 -- 172.21.15.26:0/2053854976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f441585f0 msgr2=0x7f8f440cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.303+0000 7f8f4b739640 1 --2- 172.21.15.26:0/2053854976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f441585f0 0x7f8f440cdce0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8f400097c0 tx=0x7f8f4001c3c0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.303+0000 7f8f4b739640 1 -- 172.21.15.26:0/2053854976 shutdown_connections 2023-12-19T16:54:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.303+0000 7f8f4b739640 1 --2- 172.21.15.26:0/2053854976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f440cb150 0x7f8f440cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.303+0000 7f8f4b739640 1 --2- 172.21.15.26:0/2053854976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f441585f0 0x7f8f440cdce0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.303+0000 7f8f4b739640 1 -- 172.21.15.26:0/2053854976 >> 172.21.15.26:0/2053854976 conn(0x7f8f44066160 msgr2=0x7f8f440b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.303+0000 7f8f4b739640 1 -- 172.21.15.26:0/2053854976 shutdown_connections 2023-12-19T16:54:24.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.303+0000 7f8f4b739640 1 -- 172.21.15.26:0/2053854976 wait complete. 2023-12-19T16:54:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.303+0000 7f8f4b739640 1 Processor -- start 2023-12-19T16:54:24.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f4b739640 1 -- start start 2023-12-19T16:54:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f4b739640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440cb150 0x7f8f4414aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f4b739640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f440cc310 0x7f8f44153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f4b739640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f441585f0 0x7f8f441504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f4b739640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f441502b0 con 0x7f8f440cb150 2023-12-19T16:54:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f4b739640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4414fa10 con 0x7f8f441585f0 2023-12-19T16:54:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f4b739640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4414fcf0 con 0x7f8f440cc310 2023-12-19T16:54:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f494d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440cb150 0x7f8f4414aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f494d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440cb150 0x7f8f4414aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50304/0 (socket says 172.21.15.26:50304) 2023-12-19T16:54:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f494d5640 1 -- 172.21.15.26:0/961016092 learned_addr learned my addr 172.21.15.26:0/961016092 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f48cd4640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f440cc310 0x7f8f44153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:24.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f49cd6640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f441585f0 0x7f8f441504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:24.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f494d5640 1 -- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f440cc310 msgr2=0x7f8f44153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f494d5640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f440cc310 0x7f8f44153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f494d5640 1 -- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f441585f0 msgr2=0x7f8f441504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f494d5640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f441585f0 0x7f8f441504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f494d5640 1 -- 172.21.15.26:0/961016092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f34009420 con 0x7f8f440cb150 2023-12-19T16:54:24.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f494d5640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440cb150 0x7f8f4414aa60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8f400af040 tx=0x7f8f4001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:24.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f39ffb640 1 -- 172.21.15.26:0/961016092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400b8450 con 0x7f8f440cb150 2023-12-19T16:54:24.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f39ffb640 1 -- 172.21.15.26:0/961016092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f400b85d0 con 0x7f8f440cb150 2023-12-19T16:54:24.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f39ffb640 1 -- 172.21.15.26:0/961016092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400b8980 con 0x7f8f440cb150 2023-12-19T16:54:24.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f4b739640 1 -- 172.21.15.26:0/961016092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f40009420 con 0x7f8f440cb150 2023-12-19T16:54:24.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f4b739640 1 -- 172.21.15.26:0/961016092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4415c3e0 con 0x7f8f440cb150 2023-12-19T16:54:24.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.307+0000 7f8f4b739640 1 -- 172.21.15.26:0/961016092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f44074270 con 0x7f8f440cb150 2023-12-19T16:54:24.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.311+0000 7f8f39ffb640 1 -- 172.21.15.26:0/961016092 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f8f400aa070 con 0x7f8f440cb150 2023-12-19T16:54:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.311+0000 7f8f39ffb640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8f28065c60 0x7f8f28068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.311+0000 7f8f39ffb640 1 -- 172.21.15.26:0/961016092 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8f4011dbc0 con 0x7f8f440cb150 2023-12-19T16:54:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.311+0000 7f8f48cd4640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8f28065c60 0x7f8f28068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:24.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.311+0000 7f8f48cd4640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8f28065c60 0x7f8f28068120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f441525b0 tx=0x7f8f34009190 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:24.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.315+0000 7f8f39ffb640 1 -- 172.21.15.26:0/961016092 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8f400eb2b0 con 0x7f8f440cb150 2023-12-19T16:54:24.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.487+0000 7f8f4b739640 1 -- 172.21.15.26:0/961016092 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f4415c260 con 0x7f8f28065c60 2023-12-19T16:54:24.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.499+0000 7f8f39ffb640 1 -- 172.21.15.26:0/961016092 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f8f34011ec0 con 0x7f8f28065c60 2023-12-19T16:54:24.502 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:24.502 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:43", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.499+0000 7f8f4b739640 1 -- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8f28065c60 msgr2=0x7f8f28068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.499+0000 7f8f4b739640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8f28065c60 0x7f8f28068120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f441525b0 tx=0x7f8f34009190 comp rx=0 tx=0).stop 2023-12-19T16:54:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.499+0000 7f8f4b739640 1 -- 172.21.15.26:0/961016092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440cb150 msgr2=0x7f8f4414aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.499+0000 7f8f4b739640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440cb150 0x7f8f4414aa60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8f400af040 tx=0x7f8f4001ac00 comp rx=0 tx=0).stop 2023-12-19T16:54:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.499+0000 7f8f4b739640 1 -- 172.21.15.26:0/961016092 shutdown_connections 2023-12-19T16:54:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.499+0000 7f8f4b739640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f441585f0 0x7f8f441504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.499+0000 7f8f4b739640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8f28065c60 0x7f8f28068120 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.499+0000 7f8f4b739640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f440cc310 0x7f8f44153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.499+0000 7f8f4b739640 1 --2- 172.21.15.26:0/961016092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f440cb150 0x7f8f4414aa60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.499+0000 7f8f4b739640 1 -- 172.21.15.26:0/961016092 >> 172.21.15.26:0/961016092 conn(0x7f8f44066160 msgr2=0x7f8f440ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.499+0000 7f8f4b739640 1 -- 172.21.15.26:0/961016092 shutdown_connections 2023-12-19T16:54:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.499+0000 7f8f4b739640 1 -- 172.21.15.26:0/961016092 wait complete. 2023-12-19T16:54:24.523 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:43", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:24.523 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-12-19T00:00:00', 'created': '2023-12-19T16:43:43', 'first': '2023-12-19T16:44:00', 'last': '2023-12-19T16:54:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-19T16:54:24.524 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-19T16:54:24.524 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-19T16:54:24.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:24.524 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-12-19T16:54:24.535 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:24.535 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:24.535 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:24.535 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:24.535 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:24.535 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:24.535 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:24.536 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:24.536 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:24.536 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:24.536 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:24.537 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-19-16_44_00_UTC', 'scheduled-2023-12-19-16_45_00_UTC', 'scheduled-2023-12-19-16_46_00_UTC', 'scheduled-2023-12-19-16_47_00_UTC', 'scheduled-2023-12-19-16_48_00_UTC', 'scheduled-2023-12-19-16_49_00_UTC', 'scheduled-2023-12-19-16_50_00_UTC', 'scheduled-2023-12-19-16_51_00_UTC', 'scheduled-2023-12-19-16_52_00_UTC', 'scheduled-2023-12-19-16_53_00_UTC', 'scheduled-2023-12-19-16_54_00_UTC'] 2023-12-19T16:54:24.537 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir1 2023-12-19T16:54:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc788e4640 1 Processor -- start 2023-12-19T16:54:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc788e4640 1 -- start start 2023-12-19T16:54:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc788e4640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcc7415ab20 0x7fcc740cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc788e4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc740d0140 0x7fcc740cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc788e4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc74071d00 con 0x7fcc740b35e0 2023-12-19T16:54:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc788e4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc740afcd0 con 0x7fcc740d0140 2023-12-19T16:54:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc788e4640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc740713d0 con 0x7fcc7415ab20 2023-12-19T16:54:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc7359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc740d0140 0x7fcc740cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc7359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc740d0140 0x7fcc740cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50310/0 (socket says 172.21.15.26:50310) 2023-12-19T16:54:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc7359e640 1 -- 172.21.15.26:0/2008840814 learned_addr learned my addr 172.21.15.26:0/2008840814 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc72d9d640 1 --2- 172.21.15.26:0/2008840814 >> v2:172.21.15.160:3300/0 conn(0x7fcc7415ab20 0x7fcc740cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc7359e640 1 -- 172.21.15.26:0/2008840814 >> v2:172.21.15.160:3300/0 conn(0x7fcc7415ab20 msgr2=0x7fcc740cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc7359e640 1 --2- 172.21.15.26:0/2008840814 >> v2:172.21.15.160:3300/0 conn(0x7fcc7415ab20 0x7fcc740cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc7359e640 1 -- 172.21.15.26:0/2008840814 >> v1:172.21.15.160:6789/0 conn(0x7fcc740b35e0 legacy=0x7fcc740cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc7359e640 1 -- 172.21.15.26:0/2008840814 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc740716b0 con 0x7fcc740d0140 2023-12-19T16:54:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.723+0000 7fcc72d9d640 1 --2- 172.21.15.26:0/2008840814 >> v2:172.21.15.160:3300/0 conn(0x7fcc7415ab20 0x7fcc740cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:24.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.727+0000 7fcc7359e640 1 --2- 172.21.15.26:0/2008840814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc740d0140 0x7fcc740cc0e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcc740ce310 tx=0x7fcc6c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8d4907a33746c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:24.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.727+0000 7fcc7259c640 1 -- 172.21.15.26:0/2008840814 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0a5800 con 0x7fcc740d0140 2023-12-19T16:54:24.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.727+0000 7fcc7259c640 1 -- 172.21.15.26:0/2008840814 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6c0a5960 con 0x7fcc740d0140 2023-12-19T16:54:24.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.727+0000 7fcc7259c640 1 -- 172.21.15.26:0/2008840814 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c0a5cf0 con 0x7fcc740d0140 2023-12-19T16:54:24.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.727+0000 7fcc788e4640 1 -- 172.21.15.26:0/2008840814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc740d0140 msgr2=0x7fcc740cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.727+0000 7fcc788e4640 1 --2- 172.21.15.26:0/2008840814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc740d0140 0x7fcc740cc0e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcc740ce310 tx=0x7fcc6c01c9a0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.727+0000 7fcc788e4640 1 -- 172.21.15.26:0/2008840814 shutdown_connections 2023-12-19T16:54:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.727+0000 7fcc788e4640 1 --2- 172.21.15.26:0/2008840814 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc740d0140 0x7fcc740cc0e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.727+0000 7fcc788e4640 1 --2- 172.21.15.26:0/2008840814 >> v2:172.21.15.160:3300/0 conn(0x7fcc7415ab20 0x7fcc740cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.727+0000 7fcc788e4640 1 -- 172.21.15.26:0/2008840814 >> 172.21.15.26:0/2008840814 conn(0x7fcc740cd210 msgr2=0x7fcc740cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.727+0000 7fcc788e4640 1 -- 172.21.15.26:0/2008840814 shutdown_connections 2023-12-19T16:54:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.727+0000 7fcc788e4640 1 -- 172.21.15.26:0/2008840814 wait complete. 2023-12-19T16:54:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc788e4640 1 Processor -- start 2023-12-19T16:54:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc788e4640 1 -- start start 2023-12-19T16:54:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc788e4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc740b35e0 0x7fcc7406b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc788e4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc740d0140 0x7fcc74149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc788e4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc7415ab20 0x7fcc740cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc788e4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc74162a80 con 0x7fcc740d0140 2023-12-19T16:54:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc788e4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc74163600 con 0x7fcc740b35e0 2023-12-19T16:54:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc788e4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc74157e30 con 0x7fcc7415ab20 2023-12-19T16:54:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc6bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc740d0140 0x7fcc74149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc6bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc740d0140 0x7fcc74149740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50316/0 (socket says 172.21.15.26:50316) 2023-12-19T16:54:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc6bfff640 1 -- 172.21.15.26:0/406613726 learned_addr learned my addr 172.21.15.26:0/406613726 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc6bfff640 1 -- 172.21.15.26:0/406613726 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc7415ab20 msgr2=0x7fcc740cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc6bfff640 1 --2- 172.21.15.26:0/406613726 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc7415ab20 0x7fcc740cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc6bfff640 1 -- 172.21.15.26:0/406613726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc740b35e0 msgr2=0x7fcc7406b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc6bfff640 1 --2- 172.21.15.26:0/406613726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc740b35e0 0x7fcc7406b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc6bfff640 1 -- 172.21.15.26:0/406613726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c009b30 con 0x7fcc740d0140 2023-12-19T16:54:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc6bfff640 1 --2- 172.21.15.26:0/406613726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc740d0140 0x7fcc74149740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00c500 tx=0x7fcc5c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc6b7fe640 1 -- 172.21.15.26:0/406613726 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c0052d0 con 0x7fcc740d0140 2023-12-19T16:54:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc6b7fe640 1 -- 172.21.15.26:0/406613726 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c005430 con 0x7fcc740d0140 2023-12-19T16:54:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc6b7fe640 1 -- 172.21.15.26:0/406613726 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c012830 con 0x7fcc740d0140 2023-12-19T16:54:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc788e4640 1 -- 172.21.15.26:0/406613726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc74162cd0 con 0x7fcc740d0140 2023-12-19T16:54:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc788e4640 1 -- 172.21.15.26:0/406613726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc74163290 con 0x7fcc740d0140 2023-12-19T16:54:24.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.731+0000 7fcc788e4640 1 -- 172.21.15.26:0/406613726 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7404e850 con 0x7fcc740d0140 2023-12-19T16:54:24.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.735+0000 7fcc6b7fe640 1 -- 172.21.15.26:0/406613726 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fcc5c012990 con 0x7fcc740d0140 2023-12-19T16:54:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.735+0000 7fcc6b7fe640 1 --2- 172.21.15.26:0/406613726 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcc54065c60 0x7fcc54068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.735+0000 7fcc6b7fe640 1 -- 172.21.15.26:0/406613726 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fcc5c0831c0 con 0x7fcc740d0140 2023-12-19T16:54:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.735+0000 7fcc72d9d640 1 --2- 172.21.15.26:0/406613726 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcc54065c60 0x7fcc54068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:24.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.735+0000 7fcc72d9d640 1 --2- 172.21.15.26:0/406613726 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcc54065c60 0x7fcc54068120 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc64006f10 tx=0x7fcc64008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:24.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.739+0000 7fcc6b7fe640 1 -- 172.21.15.26:0/406613726 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcc5c050770 con 0x7fcc740d0140 2023-12-19T16:54:24.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.915+0000 7fcc788e4640 1 -- 172.21.15.26:0/406613726 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc7415af00 con 0x7fcc54065c60 2023-12-19T16:54:24.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.923+0000 7fcc6b7fe640 1 -- 172.21.15.26:0/406613726 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fcc6400be40 con 0x7fcc54065c60 2023-12-19T16:54:24.927 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:24.928 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:43", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.923+0000 7fcc788e4640 1 -- 172.21.15.26:0/406613726 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcc54065c60 msgr2=0x7fcc54068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.923+0000 7fcc788e4640 1 --2- 172.21.15.26:0/406613726 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcc54065c60 0x7fcc54068120 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcc64006f10 tx=0x7fcc64008040 comp rx=0 tx=0).stop 2023-12-19T16:54:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.923+0000 7fcc788e4640 1 -- 172.21.15.26:0/406613726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc740d0140 msgr2=0x7fcc74149740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.923+0000 7fcc788e4640 1 --2- 172.21.15.26:0/406613726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc740d0140 0x7fcc74149740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00c500 tx=0x7fcc5c00c9c0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.927+0000 7fcc788e4640 1 -- 172.21.15.26:0/406613726 shutdown_connections 2023-12-19T16:54:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.927+0000 7fcc788e4640 1 --2- 172.21.15.26:0/406613726 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcc54065c60 0x7fcc54068120 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.927+0000 7fcc788e4640 1 --2- 172.21.15.26:0/406613726 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc7415ab20 0x7fcc740cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.927+0000 7fcc788e4640 1 --2- 172.21.15.26:0/406613726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc740d0140 0x7fcc74149740 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.927+0000 7fcc788e4640 1 --2- 172.21.15.26:0/406613726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc740b35e0 0x7fcc7406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.927+0000 7fcc788e4640 1 -- 172.21.15.26:0/406613726 >> 172.21.15.26:0/406613726 conn(0x7fcc740cd210 msgr2=0x7fcc740b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.927+0000 7fcc788e4640 1 -- 172.21.15.26:0/406613726 shutdown_connections 2023-12-19T16:54:24.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:24.927+0000 7fcc788e4640 1 -- 172.21.15.26:0/406613726 wait complete. 2023-12-19T16:54:24.948 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:43", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:24.948 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-12-19T00:00:00', 'created': '2023-12-19T16:43:43', 'first': '2023-12-19T16:44:00', 'last': '2023-12-19T16:54:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-19T16:54:24.948 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-19T16:54:24.948 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-19T16:54:24.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:24.949 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-12-19T16:54:24.959 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:24.959 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:24.959 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:24.959 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:24.959 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:24.959 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:24.959 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:24.959 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:24.959 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:24.959 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:24.959 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:24.960 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-19-16_44_00_UTC', 'scheduled-2023-12-19-16_45_00_UTC', 'scheduled-2023-12-19-16_46_00_UTC', 'scheduled-2023-12-19-16_47_00_UTC', 'scheduled-2023-12-19-16_48_00_UTC', 'scheduled-2023-12-19-16_49_00_UTC', 'scheduled-2023-12-19-16_50_00_UTC', 'scheduled-2023-12-19-16_51_00_UTC', 'scheduled-2023-12-19-16_52_00_UTC', 'scheduled-2023-12-19-16_53_00_UTC', 'scheduled-2023-12-19-16_54_00_UTC'] 2023-12-19T16:54:24.960 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir2 2023-12-19T16:54:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff99b28c640 1 Processor -- start 2023-12-19T16:54:25.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff99b28c640 1 -- start start 2023-12-19T16:54:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff99b28c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff9941599e0 0x7ff99406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff99b28c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff99415dc40 0x7ff9940cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff99b28c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99415e240 con 0x7ff9940cb150 2023-12-19T16:54:25.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff99b28c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99415e050 con 0x7ff99415dc40 2023-12-19T16:54:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff99b28c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9940cbb20 con 0x7ff9941599e0 2023-12-19T16:54:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff999829640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff99415dc40 0x7ff9940cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff999829640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff99415dc40 0x7ff9940cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50318/0 (socket says 172.21.15.26:50318) 2023-12-19T16:54:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff999829640 1 -- 172.21.15.26:0/2893890397 learned_addr learned my addr 172.21.15.26:0/2893890397 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff999028640 1 --2- 172.21.15.26:0/2893890397 >> v2:172.21.15.160:3300/0 conn(0x7ff9941599e0 0x7ff99406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:25.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff999829640 1 -- 172.21.15.26:0/2893890397 >> v2:172.21.15.160:3300/0 conn(0x7ff9941599e0 msgr2=0x7ff99406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff999829640 1 --2- 172.21.15.26:0/2893890397 >> v2:172.21.15.160:3300/0 conn(0x7ff9941599e0 0x7ff99406f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff999829640 1 -- 172.21.15.26:0/2893890397 >> v1:172.21.15.160:6789/0 conn(0x7ff9940cb150 legacy=0x7ff99415d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:25.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.147+0000 7ff999829640 1 -- 172.21.15.26:0/2893890397 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9940c9520 con 0x7ff99415dc40 2023-12-19T16:54:25.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff999829640 1 --2- 172.21.15.26:0/2893890397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415dc40 0x7ff9940cb5e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff9940b48f0 tx=0x7ff98c01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408609db3dcb3316 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff98bfff640 1 -- 172.21.15.26:0/2893890397 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0a7a70 con 0x7ff99415dc40 2023-12-19T16:54:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff98bfff640 1 -- 172.21.15.26:0/2893890397 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c0a7bd0 con 0x7ff99415dc40 2023-12-19T16:54:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff98bfff640 1 -- 172.21.15.26:0/2893890397 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0af3f0 con 0x7ff99415dc40 2023-12-19T16:54:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff99b28c640 1 -- 172.21.15.26:0/2893890397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415dc40 msgr2=0x7ff9940cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff99b28c640 1 --2- 172.21.15.26:0/2893890397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415dc40 0x7ff9940cb5e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff9940b48f0 tx=0x7ff98c01eba0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff99b28c640 1 -- 172.21.15.26:0/2893890397 shutdown_connections 2023-12-19T16:54:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff99b28c640 1 --2- 172.21.15.26:0/2893890397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415dc40 0x7ff9940cb5e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff99b28c640 1 --2- 172.21.15.26:0/2893890397 >> v2:172.21.15.160:3300/0 conn(0x7ff9941599e0 0x7ff99406f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff99b28c640 1 -- 172.21.15.26:0/2893890397 >> 172.21.15.26:0/2893890397 conn(0x7ff994066160 msgr2=0x7ff9940b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff99b28c640 1 -- 172.21.15.26:0/2893890397 shutdown_connections 2023-12-19T16:54:25.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff99b28c640 1 -- 172.21.15.26:0/2893890397 wait complete. 2023-12-19T16:54:25.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff99b28c640 1 Processor -- start 2023-12-19T16:54:25.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.151+0000 7ff99b28c640 1 -- start start 2023-12-19T16:54:25.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff99b28c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9940cb150 0x7ff99414b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff99b28c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9941599e0 0x7ff99415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff99b28c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415dc40 0x7ff99415bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff99b28c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940c2510 con 0x7ff99415dc40 2023-12-19T16:54:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff99b28c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99415c270 con 0x7ff9940cb150 2023-12-19T16:54:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff99b28c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99415c550 con 0x7ff9941599e0 2023-12-19T16:54:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff999829640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415dc40 0x7ff99415bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff999829640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415dc40 0x7ff99415bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50330/0 (socket says 172.21.15.26:50330) 2023-12-19T16:54:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff999829640 1 -- 172.21.15.26:0/1192887771 learned_addr learned my addr 172.21.15.26:0/1192887771 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff999028640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9940cb150 0x7ff99414b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff998827640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9941599e0 0x7ff99415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff999829640 1 -- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9941599e0 msgr2=0x7ff99415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff999829640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9941599e0 0x7ff99415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff999829640 1 -- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9940cb150 msgr2=0x7ff99414b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:25.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff999829640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9940cb150 0x7ff99414b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff999829640 1 -- 172.21.15.26:0/1192887771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c009b30 con 0x7ff99415dc40 2023-12-19T16:54:25.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff999028640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9940cb150 0x7ff99414b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:25.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff999829640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415dc40 0x7ff99415bc00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0b1040 tx=0x7ff9940cb010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:25.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff989ffb640 1 -- 172.21.15.26:0/1192887771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0a7a70 con 0x7ff99415dc40 2023-12-19T16:54:25.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff989ffb640 1 -- 172.21.15.26:0/1192887771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c00e150 con 0x7ff99415dc40 2023-12-19T16:54:25.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff99b28c640 1 -- 172.21.15.26:0/1192887771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99414be30 con 0x7ff99415dc40 2023-12-19T16:54:25.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff989ffb640 1 -- 172.21.15.26:0/1192887771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c009510 con 0x7ff99415dc40 2023-12-19T16:54:25.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff99b28c640 1 -- 172.21.15.26:0/1192887771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9940b0680 con 0x7ff99415dc40 2023-12-19T16:54:25.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.155+0000 7ff99b28c640 1 -- 172.21.15.26:0/1192887771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9940c2650 con 0x7ff99415dc40 2023-12-19T16:54:25.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.159+0000 7ff989ffb640 1 -- 172.21.15.26:0/1192887771 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7ff98c0c13f0 con 0x7ff99415dc40 2023-12-19T16:54:25.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.159+0000 7ff989ffb640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff978065c60 0x7ff978068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:25.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.159+0000 7ff989ffb640 1 -- 172.21.15.26:0/1192887771 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff98c11d680 con 0x7ff99415dc40 2023-12-19T16:54:25.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.159+0000 7ff999028640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff978065c60 0x7ff978068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:25.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.159+0000 7ff999028640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff978065c60 0x7ff978068120 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff99415cae0 tx=0x7ff990008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:25.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.163+0000 7ff989ffb640 1 -- 172.21.15.26:0/1192887771 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff9940c2650 con 0x7ff99415dc40 2023-12-19T16:54:25.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.343+0000 7ff99b28c640 1 -- 172.21.15.26:0/1192887771 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff994159dc0 con 0x7ff978065c60 2023-12-19T16:54:25.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.351+0000 7ff989ffb640 1 -- 172.21.15.26:0/1192887771 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7ff990007680 con 0x7ff978065c60 2023-12-19T16:54:25.354 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:25.355 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:44", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.351+0000 7ff99b28c640 1 -- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff978065c60 msgr2=0x7ff978068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.351+0000 7ff99b28c640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff978065c60 0x7ff978068120 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff99415cae0 tx=0x7ff990008040 comp rx=0 tx=0).stop 2023-12-19T16:54:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.351+0000 7ff99b28c640 1 -- 172.21.15.26:0/1192887771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415dc40 msgr2=0x7ff99415bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.351+0000 7ff99b28c640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415dc40 0x7ff99415bc00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0b1040 tx=0x7ff9940cb010 comp rx=0 tx=0).stop 2023-12-19T16:54:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.351+0000 7ff99b28c640 1 -- 172.21.15.26:0/1192887771 shutdown_connections 2023-12-19T16:54:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.351+0000 7ff99b28c640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415dc40 0x7ff99415bc00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.351+0000 7ff99b28c640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff978065c60 0x7ff978068120 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.351+0000 7ff99b28c640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9941599e0 0x7ff99415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.351+0000 7ff99b28c640 1 --2- 172.21.15.26:0/1192887771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9940cb150 0x7ff99414b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.351+0000 7ff99b28c640 1 -- 172.21.15.26:0/1192887771 >> 172.21.15.26:0/1192887771 conn(0x7ff994066160 msgr2=0x7ff99406c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.351+0000 7ff99b28c640 1 -- 172.21.15.26:0/1192887771 shutdown_connections 2023-12-19T16:54:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.351+0000 7ff99b28c640 1 -- 172.21.15.26:0/1192887771 wait complete. 2023-12-19T16:54:25.375 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:44", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:25.376 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-12-19T00:00:00', 'created': '2023-12-19T16:43:44', 'first': '2023-12-19T16:44:00', 'last': '2023-12-19T16:54:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-19T16:54:25.376 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-19T16:54:25.376 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-19T16:54:25.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:25.376 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-12-19T16:54:25.387 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:25.387 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:25.387 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:25.387 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:25.387 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:25.387 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:25.387 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:25.387 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:25.387 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:25.387 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:25.387 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:25.388 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-19-16_44_00_UTC', 'scheduled-2023-12-19-16_45_00_UTC', 'scheduled-2023-12-19-16_46_00_UTC', 'scheduled-2023-12-19-16_47_00_UTC', 'scheduled-2023-12-19-16_48_00_UTC', 'scheduled-2023-12-19-16_49_00_UTC', 'scheduled-2023-12-19-16_50_00_UTC', 'scheduled-2023-12-19-16_51_00_UTC', 'scheduled-2023-12-19-16_52_00_UTC', 'scheduled-2023-12-19-16_53_00_UTC', 'scheduled-2023-12-19-16_54_00_UTC'] 2023-12-19T16:54:25.388 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir3 2023-12-19T16:54:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9cd262d640 1 Processor -- start 2023-12-19T16:54:25.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9cd262d640 1 -- start start 2023-12-19T16:54:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9cd262d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ccc0b88b0 0x7f9ccc0b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9cd262d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ccc0b71f0 con 0x7f9ccc15cc70 2023-12-19T16:54:25.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9cd262d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ccc0d1370 con 0x7f9ccc0b6140 2023-12-19T16:54:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9cd262d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ccc0b65e0 con 0x7f9ccc0b88b0 2023-12-19T16:54:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9ccbfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9ccc15cc70 0x7f9ccc0d0c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59674/0 (socket says 172.21.15.26:59674) 2023-12-19T16:54:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9ccbfff640 1 -- 172.21.15.26:0/1147049142 learned_addr learned my addr 172.21.15.26:0/1147049142 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9ccb7fe640 1 --2- 172.21.15.26:0/1147049142 >> v2:172.21.15.26:3300/0 conn(0x7f9ccc0b88b0 0x7f9ccc0b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9ccaffd640 1 -- 172.21.15.26:0/1147049142 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3795790969 0 0) 0x7f9ccc0b71f0 con 0x7f9ccc15cc70 2023-12-19T16:54:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9ccaffd640 1 -- 172.21.15.26:0/1147049142 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cb4003530 con 0x7f9ccc15cc70 2023-12-19T16:54:25.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9ccb7fe640 1 -- 172.21.15.26:0/1147049142 >> v1:172.21.15.160:6789/0 conn(0x7f9ccc0b6140 legacy=0x7f9ccc0cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9ccb7fe640 1 -- 172.21.15.26:0/1147049142 >> v1:172.21.15.26:6789/0 conn(0x7f9ccc15cc70 legacy=0x7f9ccc0d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:25.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.575+0000 7f9ccb7fe640 1 -- 172.21.15.26:0/1147049142 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc0b68c0 con 0x7f9ccc0b88b0 2023-12-19T16:54:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.579+0000 7f9ccb7fe640 1 --2- 172.21.15.26:0/1147049142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ccc0b88b0 0x7f9ccc0b5c00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00097c0 tx=0x7f9cc001ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14eab51fa687d0b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.579+0000 7f9ccaffd640 1 -- 172.21.15.26:0/1147049142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00a8930 con 0x7f9ccc0b88b0 2023-12-19T16:54:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.579+0000 7f9ccaffd640 1 -- 172.21.15.26:0/1147049142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc00a8a90 con 0x7f9ccc0b88b0 2023-12-19T16:54:25.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.579+0000 7f9ccaffd640 1 -- 172.21.15.26:0/1147049142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00a8e20 con 0x7f9ccc0b88b0 2023-12-19T16:54:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.579+0000 7f9cd262d640 1 -- 172.21.15.26:0/1147049142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ccc0b88b0 msgr2=0x7f9ccc0b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.579+0000 7f9cd262d640 1 --2- 172.21.15.26:0/1147049142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ccc0b88b0 0x7f9ccc0b5c00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00097c0 tx=0x7f9cc001ec30 comp rx=0 tx=0).stop 2023-12-19T16:54:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.579+0000 7f9cd262d640 1 -- 172.21.15.26:0/1147049142 shutdown_connections 2023-12-19T16:54:25.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.579+0000 7f9cd262d640 1 --2- 172.21.15.26:0/1147049142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ccc0b88b0 0x7f9ccc0b5c00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.579+0000 7f9cd262d640 1 -- 172.21.15.26:0/1147049142 >> 172.21.15.26:0/1147049142 conn(0x7f9ccc06a120 msgr2=0x7f9ccc06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.579+0000 7f9cd262d640 1 -- 172.21.15.26:0/1147049142 shutdown_connections 2023-12-19T16:54:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.579+0000 7f9cd262d640 1 -- 172.21.15.26:0/1147049142 wait complete. 2023-12-19T16:54:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.579+0000 7f9cd262d640 1 Processor -- start 2023-12-19T16:54:25.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cd262d640 1 -- start start 2023-12-19T16:54:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cd262d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ccc0b6140 0x7f9ccc14e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cd262d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ccc0b88b0 0x7f9ccc1f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cd262d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ccc15cc70 0x7f9ccc0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cd262d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc14e9e0 con 0x7f9ccc0b6140 2023-12-19T16:54:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cd262d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc1f6b20 con 0x7f9ccc0b88b0 2023-12-19T16:54:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cd262d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0c8860 con 0x7f9ccc15cc70 2023-12-19T16:54:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9ccbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ccc0b6140 0x7f9ccc14e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9ccbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ccc0b6140 0x7f9ccc14e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50358/0 (socket says 172.21.15.26:50358) 2023-12-19T16:54:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9ccbfff640 1 -- 172.21.15.26:0/1060437160 learned_addr learned my addr 172.21.15.26:0/1060437160 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9ccb7fe640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ccc0b88b0 0x7f9ccc1f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:25.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cd0bca640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ccc15cc70 0x7f9ccc0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9ccbfff640 1 -- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ccc15cc70 msgr2=0x7f9ccc0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9ccbfff640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ccc15cc70 0x7f9ccc0c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9ccbfff640 1 -- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ccc0b88b0 msgr2=0x7f9ccc1f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9ccbfff640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ccc0b88b0 0x7f9ccc1f65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9ccbfff640 1 -- 172.21.15.26:0/1060437160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc0009420 con 0x7f9ccc0b6140 2023-12-19T16:54:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9ccbfff640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ccc0b6140 0x7f9ccc14e4a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00c420 tx=0x7f9cbc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cc8ff9640 1 -- 172.21.15.26:0/1060437160 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc005cf0 con 0x7f9ccc0b6140 2023-12-19T16:54:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cc8ff9640 1 -- 172.21.15.26:0/1060437160 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc005e50 con 0x7f9ccc0b6140 2023-12-19T16:54:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cd262d640 1 -- 172.21.15.26:0/1060437160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc0c7c50 con 0x7f9ccc0b6140 2023-12-19T16:54:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cc8ff9640 1 -- 172.21.15.26:0/1060437160 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0072d0 con 0x7f9ccc0b6140 2023-12-19T16:54:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cd262d640 1 -- 172.21.15.26:0/1060437160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc0c8210 con 0x7f9ccc0b6140 2023-12-19T16:54:25.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.583+0000 7f9cd262d640 1 -- 172.21.15.26:0/1060437160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ccc074270 con 0x7f9ccc0b6140 2023-12-19T16:54:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.587+0000 7f9cc8ff9640 1 -- 172.21.15.26:0/1060437160 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f9cbc007430 con 0x7f9ccc0b6140 2023-12-19T16:54:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.587+0000 7f9cc8ff9640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9cb8065c90 0x7f9cb8068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.587+0000 7f9cc8ff9640 1 -- 172.21.15.26:0/1060437160 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9cbc082930 con 0x7f9ccc0b6140 2023-12-19T16:54:25.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.587+0000 7f9ccb7fe640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9cb8065c90 0x7f9cb8068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:25.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.587+0000 7f9ccb7fe640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9cb8065c90 0x7f9cb8068150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00b2000 tx=0x7f9cc0008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:25.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.591+0000 7f9cc8ff9640 1 -- 172.21.15.26:0/1060437160 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9cbc050160 con 0x7f9ccc0b6140 2023-12-19T16:54:25.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.767+0000 7f9cd262d640 1 -- 172.21.15.26:0/1060437160 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ccc0b6e80 con 0x7f9cb8065c90 2023-12-19T16:54:25.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.775+0000 7f9cc8ff9640 1 -- 172.21.15.26:0/1060437160 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f9cc00bb480 con 0x7f9cb8065c90 2023-12-19T16:54:25.778 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:25.778 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:44", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:25.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.775+0000 7f9cd262d640 1 -- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9cb8065c90 msgr2=0x7f9cb8068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.775+0000 7f9cd262d640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9cb8065c90 0x7f9cb8068150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00b2000 tx=0x7f9cc0008040 comp rx=0 tx=0).stop 2023-12-19T16:54:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.775+0000 7f9cd262d640 1 -- 172.21.15.26:0/1060437160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ccc0b6140 msgr2=0x7f9ccc14e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.775+0000 7f9cd262d640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ccc0b6140 0x7f9ccc14e4a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00c420 tx=0x7f9cbc00c8e0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.775+0000 7f9cd262d640 1 -- 172.21.15.26:0/1060437160 shutdown_connections 2023-12-19T16:54:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.775+0000 7f9cd262d640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9cb8065c90 0x7f9cb8068150 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.775+0000 7f9cd262d640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ccc15cc70 0x7f9ccc0c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.775+0000 7f9cd262d640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ccc0b88b0 0x7f9ccc1f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.775+0000 7f9cd262d640 1 --2- 172.21.15.26:0/1060437160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ccc0b6140 0x7f9ccc14e4a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.775+0000 7f9cd262d640 1 -- 172.21.15.26:0/1060437160 >> 172.21.15.26:0/1060437160 conn(0x7f9ccc06a120 msgr2=0x7f9ccc0b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.775+0000 7f9cd262d640 1 -- 172.21.15.26:0/1060437160 shutdown_connections 2023-12-19T16:54:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:25.775+0000 7f9cd262d640 1 -- 172.21.15.26:0/1060437160 wait complete. 2023-12-19T16:54:25.800 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:44", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:25.800 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-12-19T00:00:00', 'created': '2023-12-19T16:43:44', 'first': '2023-12-19T16:44:00', 'last': '2023-12-19T16:54:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-19T16:54:25.800 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-19T16:54:25.800 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-19T16:54:25.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:25.800 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-12-19T16:54:25.811 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:25.811 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:25.811 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:25.811 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:25.811 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:25.811 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:25.811 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:25.811 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:25.811 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:25.811 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:25.811 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:25.812 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-19-16_44_00_UTC', 'scheduled-2023-12-19-16_45_00_UTC', 'scheduled-2023-12-19-16_46_00_UTC', 'scheduled-2023-12-19-16_47_00_UTC', 'scheduled-2023-12-19-16_48_00_UTC', 'scheduled-2023-12-19-16_49_00_UTC', 'scheduled-2023-12-19-16_50_00_UTC', 'scheduled-2023-12-19-16_51_00_UTC', 'scheduled-2023-12-19-16_52_00_UTC', 'scheduled-2023-12-19-16_53_00_UTC', 'scheduled-2023-12-19-16_54_00_UTC'] 2023-12-19T16:54:25.812 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir4 2023-12-19T16:54:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee17160640 1 Processor -- start 2023-12-19T16:54:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee17160640 1 -- start start 2023-12-19T16:54:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee17160640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee101599e0 0x7fee1006f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee17160640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fee100cb150 0x7fee100cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee17160640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1015e240 con 0x7fee1015d530 2023-12-19T16:54:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee17160640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1015e050 con 0x7fee101599e0 2023-12-19T16:54:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee17160640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee100cbb20 con 0x7fee100cb150 2023-12-19T16:54:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee14efc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee101599e0 0x7fee1006f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee14efc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee101599e0 0x7fee1006f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40460/0 (socket says 172.21.15.26:40460) 2023-12-19T16:54:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee14efc640 1 -- 172.21.15.26:0/376833669 learned_addr learned my addr 172.21.15.26:0/376833669 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee0ffff640 1 --2- 172.21.15.26:0/376833669 >> v2:172.21.15.160:3300/0 conn(0x7fee100cb150 0x7fee100cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee14efc640 1 -- 172.21.15.26:0/376833669 >> v2:172.21.15.160:3300/0 conn(0x7fee100cb150 msgr2=0x7fee100cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee14efc640 1 --2- 172.21.15.26:0/376833669 >> v2:172.21.15.160:3300/0 conn(0x7fee100cb150 0x7fee100cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee14efc640 1 -- 172.21.15.26:0/376833669 >> v1:172.21.15.160:6789/0 conn(0x7fee1015d530 legacy=0x7fee1015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:26.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.003+0000 7fee14efc640 1 -- 172.21.15.26:0/376833669 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee100c9520 con 0x7fee101599e0 2023-12-19T16:54:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.007+0000 7fee14efc640 1 --2- 172.21.15.26:0/376833669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee101599e0 0x7fee1006f6d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fee100cc220 tx=0x7fee0401eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a856886ef43c4c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.007+0000 7fee0f7fe640 1 -- 172.21.15.26:0/376833669 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee040a7c60 con 0x7fee101599e0 2023-12-19T16:54:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.007+0000 7fee0f7fe640 1 -- 172.21.15.26:0/376833669 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee040a7dc0 con 0x7fee101599e0 2023-12-19T16:54:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.007+0000 7fee0f7fe640 1 -- 172.21.15.26:0/376833669 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee040af660 con 0x7fee101599e0 2023-12-19T16:54:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.007+0000 7fee17160640 1 -- 172.21.15.26:0/376833669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee101599e0 msgr2=0x7fee1006f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.007+0000 7fee17160640 1 --2- 172.21.15.26:0/376833669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee101599e0 0x7fee1006f6d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fee100cc220 tx=0x7fee0401eb90 comp rx=0 tx=0).stop 2023-12-19T16:54:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.007+0000 7fee17160640 1 -- 172.21.15.26:0/376833669 shutdown_connections 2023-12-19T16:54:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.007+0000 7fee17160640 1 --2- 172.21.15.26:0/376833669 >> v2:172.21.15.160:3300/0 conn(0x7fee100cb150 0x7fee100cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.007+0000 7fee17160640 1 --2- 172.21.15.26:0/376833669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee101599e0 0x7fee1006f6d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.007+0000 7fee17160640 1 -- 172.21.15.26:0/376833669 >> 172.21.15.26:0/376833669 conn(0x7fee10066160 msgr2=0x7fee100b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.007+0000 7fee17160640 1 -- 172.21.15.26:0/376833669 shutdown_connections 2023-12-19T16:54:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.007+0000 7fee17160640 1 -- 172.21.15.26:0/376833669 wait complete. 2023-12-19T16:54:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee17160640 1 Processor -- start 2023-12-19T16:54:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee17160640 1 -- start start 2023-12-19T16:54:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee17160640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee100cb150 0x7fee1014f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee17160640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee101599e0 0x7fee100b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee17160640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee1015d530 0x7fee100b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee17160640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee100b32b0 con 0x7fee100cb150 2023-12-19T16:54:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee17160640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee10150980 con 0x7fee101599e0 2023-12-19T16:54:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee17160640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1014fd70 con 0x7fee1015d530 2023-12-19T16:54:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee14efc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee100cb150 0x7fee1014f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee14efc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee100cb150 0x7fee1014f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40472/0 (socket says 172.21.15.26:40472) 2023-12-19T16:54:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee14efc640 1 -- 172.21.15.26:0/789137672 learned_addr learned my addr 172.21.15.26:0/789137672 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee0ffff640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee101599e0 0x7fee100b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee156fd640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee1015d530 0x7fee100b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee14efc640 1 -- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee1015d530 msgr2=0x7fee100b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee14efc640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee1015d530 0x7fee100b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee14efc640 1 -- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee101599e0 msgr2=0x7fee100b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee14efc640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee101599e0 0x7fee100b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee14efc640 1 -- 172.21.15.26:0/789137672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0400b770 con 0x7fee100cb150 2023-12-19T16:54:26.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee156fd640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee1015d530 0x7fee100b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:26.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee0ffff640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee101599e0 0x7fee100b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:26.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee14efc640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee100cb150 0x7fee1014f120 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fee0401c7c0 tx=0x7fee100cb010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:26.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee0d7fa640 1 -- 172.21.15.26:0/789137672 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee04006ee0 con 0x7fee100cb150 2023-12-19T16:54:26.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee0d7fa640 1 -- 172.21.15.26:0/789137672 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee040afe80 con 0x7fee100cb150 2023-12-19T16:54:26.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee17160640 1 -- 172.21.15.26:0/789137672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee10150050 con 0x7fee100cb150 2023-12-19T16:54:26.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee0d7fa640 1 -- 172.21.15.26:0/789137672 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee040ba620 con 0x7fee100cb150 2023-12-19T16:54:26.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee17160640 1 -- 172.21.15.26:0/789137672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee10150610 con 0x7fee100cb150 2023-12-19T16:54:26.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.011+0000 7fee17160640 1 -- 172.21.15.26:0/789137672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee10074270 con 0x7fee100cb150 2023-12-19T16:54:26.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.015+0000 7fee0d7fa640 1 -- 172.21.15.26:0/789137672 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fee040b6040 con 0x7fee100cb150 2023-12-19T16:54:26.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.015+0000 7fee0d7fa640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fedf4065c60 0x7fedf4068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.015+0000 7fee0d7fa640 1 -- 172.21.15.26:0/789137672 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fee04120720 con 0x7fee100cb150 2023-12-19T16:54:26.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.015+0000 7fee0ffff640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fedf4065c60 0x7fedf4068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.015+0000 7fee0ffff640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fedf4065c60 0x7fedf4068120 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fedfc007a70 tx=0x7fedfc008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:26.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.019+0000 7fee0d7fa640 1 -- 172.21.15.26:0/789137672 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fee040edcc0 con 0x7fee100cb150 2023-12-19T16:54:26.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.195+0000 7fee17160640 1 -- 172.21.15.26:0/789137672 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee10159dc0 con 0x7fedf4065c60 2023-12-19T16:54:26.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.203+0000 7fee0d7fa640 1 -- 172.21.15.26:0/789137672 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fedfc007660 con 0x7fedf4065c60 2023-12-19T16:54:26.208 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:26.208 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:45", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.207+0000 7fee17160640 1 -- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fedf4065c60 msgr2=0x7fedf4068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.207+0000 7fee17160640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fedf4065c60 0x7fedf4068120 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fedfc007a70 tx=0x7fedfc008040 comp rx=0 tx=0).stop 2023-12-19T16:54:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.207+0000 7fee17160640 1 -- 172.21.15.26:0/789137672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee100cb150 msgr2=0x7fee1014f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.207+0000 7fee17160640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee100cb150 0x7fee1014f120 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fee0401c7c0 tx=0x7fee100cb010 comp rx=0 tx=0).stop 2023-12-19T16:54:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.207+0000 7fee17160640 1 -- 172.21.15.26:0/789137672 shutdown_connections 2023-12-19T16:54:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.207+0000 7fee17160640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fedf4065c60 0x7fedf4068120 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.207+0000 7fee17160640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee1015d530 0x7fee100b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.207+0000 7fee17160640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee101599e0 0x7fee100b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.207+0000 7fee17160640 1 --2- 172.21.15.26:0/789137672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee100cb150 0x7fee1014f120 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.207+0000 7fee17160640 1 -- 172.21.15.26:0/789137672 >> 172.21.15.26:0/789137672 conn(0x7fee10066160 msgr2=0x7fee100ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.207+0000 7fee17160640 1 -- 172.21.15.26:0/789137672 shutdown_connections 2023-12-19T16:54:26.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.207+0000 7fee17160640 1 -- 172.21.15.26:0/789137672 wait complete. 2023-12-19T16:54:26.229 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:45", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:26.230 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-12-19T00:00:00', 'created': '2023-12-19T16:43:45', 'first': '2023-12-19T16:44:00', 'last': '2023-12-19T16:54:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-19T16:54:26.230 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-19T16:54:26.230 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-19T16:54:26.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:26.230 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-12-19T16:54:26.240 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:26.240 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:26.241 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:26.241 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:26.241 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:26.241 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:26.241 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:26.241 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:26.241 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:26.241 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:26.241 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:26.242 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-19-16_44_00_UTC', 'scheduled-2023-12-19-16_45_00_UTC', 'scheduled-2023-12-19-16_46_00_UTC', 'scheduled-2023-12-19-16_47_00_UTC', 'scheduled-2023-12-19-16_48_00_UTC', 'scheduled-2023-12-19-16_49_00_UTC', 'scheduled-2023-12-19-16_50_00_UTC', 'scheduled-2023-12-19-16_51_00_UTC', 'scheduled-2023-12-19-16_52_00_UTC', 'scheduled-2023-12-19-16_53_00_UTC', 'scheduled-2023-12-19-16_54_00_UTC'] 2023-12-19T16:54:26.242 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir5 2023-12-19T16:54:26.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.427+0000 7f6f45f0a640 1 Processor -- start 2023-12-19T16:54:26.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f45f0a640 1 -- start start 2023-12-19T16:54:26.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f45f0a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f400cb150 0x7f6f400cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f45f0a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6f4015e690 0x7f6f4015eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f45f0a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f400ce3f0 con 0x7f6f401585f0 2023-12-19T16:54:26.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f45f0a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f400cbb10 con 0x7f6f4015e690 2023-12-19T16:54:26.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f45f0a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400b47a0 con 0x7f6f400cb150 2023-12-19T16:54:26.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6f401585f0 0x7f6f400cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55916/0 (socket says 172.21.15.26:55916) 2023-12-19T16:54:26.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3f7fe640 1 -- 172.21.15.26:0/3157978225 learned_addr learned my addr 172.21.15.26:0/3157978225 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:26.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3ffff640 1 --2- 172.21.15.26:0/3157978225 >> v2:172.21.15.160:3300/0 conn(0x7f6f4015e690 0x7f6f4015eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3effd640 1 --2- 172.21.15.26:0/3157978225 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f400cb150 0x7f6f400cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643574402 0 0) 0x7f6f400ce3f0 con 0x7f6f401585f0 2023-12-19T16:54:26.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f28003530 con 0x7f6f401585f0 2023-12-19T16:54:26.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4103567822 0 0) 0x7f6f28003530 con 0x7f6f401585f0 2023-12-19T16:54:26.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f400cb150 msgr2=0x7f6f400cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:26.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 --2- 172.21.15.26:0/3157978225 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f400cb150 0x7f6f400cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 >> v2:172.21.15.160:3300/0 conn(0x7f6f4015e690 msgr2=0x7f6f4015eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 --2- 172.21.15.26:0/3157978225 >> v2:172.21.15.160:3300/0 conn(0x7f6f4015e690 0x7f6f4015eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4015b1b0 con 0x7f6f401585f0 2023-12-19T16:54:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f6f30001680 con 0x7f6f401585f0 2023-12-19T16:54:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 >> v1:172.21.15.26:6789/0 conn(0x7f6f401585f0 legacy=0x7f6f400cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 --2- 172.21.15.26:0/3157978225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f28003980 0x7f6f28003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 --2- 172.21.15.26:0/3157978225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f280047d0 0x7f6f28004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 --2- 172.21.15.26:0/3157978225 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f400cb150 0x7f6f28005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400cbb10 con 0x7f6f28003980 2023-12-19T16:54:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400b47a0 con 0x7f6f280047d0 2023-12-19T16:54:26.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2800b9c0 con 0x7f6f400cb150 2023-12-19T16:54:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3f7fe640 1 --2- 172.21.15.26:0/3157978225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f28003980 0x7f6f28003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3ffff640 1 --2- 172.21.15.26:0/3157978225 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f400cb150 0x7f6f28005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3effd640 1 --2- 172.21.15.26:0/3157978225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f280047d0 0x7f6f28004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3f7fe640 1 -- 172.21.15.26:0/3157978225 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f400cb150 msgr2=0x7f6f28005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3f7fe640 1 --2- 172.21.15.26:0/3157978225 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f400cb150 0x7f6f28005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3f7fe640 1 -- 172.21.15.26:0/3157978225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f280047d0 msgr2=0x7f6f28004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3f7fe640 1 --2- 172.21.15.26:0/3157978225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f280047d0 0x7f6f28004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.431+0000 7f6f3f7fe640 1 -- 172.21.15.26:0/3157978225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f28003530 con 0x7f6f28003980 2023-12-19T16:54:26.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.435+0000 7f6f3f7fe640 1 --2- 172.21.15.26:0/3157978225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f28003980 0x7f6f28003da0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000c240 tx=0x7f6f3001cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471f39fd09b489af server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:26.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.435+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f300058e0 con 0x7f6f28003980 2023-12-19T16:54:26.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.435+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f30005a40 con 0x7f6f28003980 2023-12-19T16:54:26.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.435+0000 7f6f3e7fc640 1 -- 172.21.15.26:0/3157978225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f30005dd0 con 0x7f6f28003980 2023-12-19T16:54:26.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.435+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3157978225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f28003980 msgr2=0x7f6f28003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.435+0000 7f6f45f0a640 1 --2- 172.21.15.26:0/3157978225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f28003980 0x7f6f28003da0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000c240 tx=0x7f6f3001cbd0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.435+0000 7f6f3ffff640 1 -- 172.21.15.26:0/3157978225 reap_dead start 2023-12-19T16:54:26.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.435+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3157978225 shutdown_connections 2023-12-19T16:54:26.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.435+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3157978225 >> 172.21.15.26:0/3157978225 conn(0x7f6f40066160 msgr2=0x7f6f400b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:26.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.435+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3157978225 shutdown_connections 2023-12-19T16:54:26.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.435+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3157978225 wait complete. 2023-12-19T16:54:26.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f45f0a640 1 Processor -- start 2023-12-19T16:54:26.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f45f0a640 1 -- start start 2023-12-19T16:54:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f45f0a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f28003980 0x7f6f40155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f45f0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f400b04e0 0x7f6f400c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f45f0a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4015d750 0x7f6f40163fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f45f0a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400aedd0 con 0x7f6f400b04e0 2023-12-19T16:54:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f45f0a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400b0050 con 0x7f6f4015d750 2023-12-19T16:54:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f45f0a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400c8660 con 0x7f6f28003980 2023-12-19T16:54:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f3effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f400b04e0 0x7f6f400c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f3effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f400b04e0 0x7f6f400c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40486/0 (socket says 172.21.15.26:40486) 2023-12-19T16:54:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f3effd640 1 -- 172.21.15.26:0/3598111597 learned_addr learned my addr 172.21.15.26:0/3598111597 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f3effd640 1 -- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f28003980 msgr2=0x7f6f40155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f3f7fe640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f28003980 0x7f6f40155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f3ffff640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4015d750 0x7f6f40163fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f3effd640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f28003980 0x7f6f40155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f3effd640 1 -- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4015d750 msgr2=0x7f6f40163fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f3effd640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4015d750 0x7f6f40163fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f3effd640 1 -- 172.21.15.26:0/3598111597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f400af720 con 0x7f6f400b04e0 2023-12-19T16:54:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f3f7fe640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f28003980 0x7f6f40155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:54:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f3effd640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f400b04e0 0x7f6f400c8120 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6f34011960 tx=0x7f6f34011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f27fff640 1 -- 172.21.15.26:0/3598111597 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3400faf0 con 0x7f6f400b04e0 2023-12-19T16:54:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f27fff640 1 -- 172.21.15.26:0/3598111597 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3400fc50 con 0x7f6f400b04e0 2023-12-19T16:54:26.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3598111597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f400afa00 con 0x7f6f400b04e0 2023-12-19T16:54:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f27fff640 1 -- 172.21.15.26:0/3598111597 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34014440 con 0x7f6f400b04e0 2023-12-19T16:54:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.439+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3598111597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4015dd10 con 0x7f6f400b04e0 2023-12-19T16:54:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.443+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3598111597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f400b01c0 con 0x7f6f400b04e0 2023-12-19T16:54:26.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.443+0000 7f6f27fff640 1 -- 172.21.15.26:0/3598111597 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f6f340145a0 con 0x7f6f400b04e0 2023-12-19T16:54:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.443+0000 7f6f27fff640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f2c065c90 0x7f6f2c068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.443+0000 7f6f27fff640 1 -- 172.21.15.26:0/3598111597 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f6f34083df0 con 0x7f6f400b04e0 2023-12-19T16:54:26.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.443+0000 7f6f3f7fe640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f2c065c90 0x7f6f2c068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.443+0000 7f6f3f7fe640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f2c065c90 0x7f6f2c068150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6f300aa000 tx=0x7f6f30001a80 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:26.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.447+0000 7f6f27fff640 1 -- 172.21.15.26:0/3598111597 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6f340514a0 con 0x7f6f400b04e0 2023-12-19T16:54:26.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.623+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3598111597 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f401589d0 con 0x7f6f2c065c90 2023-12-19T16:54:26.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.635+0000 7f6f27fff640 1 -- 172.21.15.26:0/3598111597 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f6f300015c0 con 0x7f6f2c065c90 2023-12-19T16:54:26.637 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:26.637 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:45", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.635+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f2c065c90 msgr2=0x7f6f2c068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.635+0000 7f6f45f0a640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f2c065c90 0x7f6f2c068150 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6f300aa000 tx=0x7f6f30001a80 comp rx=0 tx=0).stop 2023-12-19T16:54:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.635+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3598111597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f400b04e0 msgr2=0x7f6f400c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.635+0000 7f6f45f0a640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f400b04e0 0x7f6f400c8120 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6f34011960 tx=0x7f6f34011e20 comp rx=0 tx=0).stop 2023-12-19T16:54:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.635+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3598111597 shutdown_connections 2023-12-19T16:54:26.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.635+0000 7f6f45f0a640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4015d750 0x7f6f40163fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.635+0000 7f6f45f0a640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f400b04e0 0x7f6f400c8120 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.635+0000 7f6f45f0a640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f2c065c90 0x7f6f2c068150 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.635+0000 7f6f45f0a640 1 --2- 172.21.15.26:0/3598111597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f28003980 0x7f6f40155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.635+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3598111597 >> 172.21.15.26:0/3598111597 conn(0x7f6f40066160 msgr2=0x7f6f400cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.635+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3598111597 shutdown_connections 2023-12-19T16:54:26.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.635+0000 7f6f45f0a640 1 -- 172.21.15.26:0/3598111597 wait complete. 2023-12-19T16:54:26.659 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:45", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:26.659 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-12-19T00:00:00', 'created': '2023-12-19T16:43:45', 'first': '2023-12-19T16:44:00', 'last': '2023-12-19T16:54:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-19T16:54:26.659 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-19T16:54:26.659 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-19T16:54:26.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:26.659 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-12-19T16:54:26.670 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:26.670 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:26.670 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:26.670 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:26.670 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:26.670 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:26.670 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:26.670 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:26.670 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_01_UTC 2023-12-19T16:54:26.670 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:26.670 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:26.671 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-19-16_44_00_UTC', 'scheduled-2023-12-19-16_45_00_UTC', 'scheduled-2023-12-19-16_46_00_UTC', 'scheduled-2023-12-19-16_47_00_UTC', 'scheduled-2023-12-19-16_48_00_UTC', 'scheduled-2023-12-19-16_49_00_UTC', 'scheduled-2023-12-19-16_50_00_UTC', 'scheduled-2023-12-19-16_51_00_UTC', 'scheduled-2023-12-19-16_52_01_UTC', 'scheduled-2023-12-19-16_53_00_UTC', 'scheduled-2023-12-19-16_54_00_UTC'] 2023-12-19T16:54:26.671 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir6 2023-12-19T16:54:26.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.855+0000 7fcced6e5640 1 Processor -- start 2023-12-19T16:54:26.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.855+0000 7fcced6e5640 1 -- start start 2023-12-19T16:54:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.855+0000 7fcced6e5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcce80cb600 0x7fcce815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.855+0000 7fcced6e5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce80ce3f0 con 0x7fcce80cb150 2023-12-19T16:54:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.855+0000 7fcced6e5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce80cba50 con 0x7fcce81585f0 2023-12-19T16:54:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.855+0000 7fcced6e5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce80b47a0 con 0x7fcce80cb600 2023-12-19T16:54:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.855+0000 7fcce77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcce80cb600 0x7fcce815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.855+0000 7fccdffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcce80cb150 0x7fcce815e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55926/0 (socket says 172.21.15.26:55926) 2023-12-19T16:54:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.855+0000 7fccdffff640 1 -- 172.21.15.26:0/2400059310 learned_addr learned my addr 172.21.15.26:0/2400059310 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:26.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.855+0000 7fcce77fe640 1 -- 172.21.15.26:0/2400059310 >> v1:172.21.15.160:6789/0 conn(0x7fcce81585f0 legacy=0x7fcce80cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.855+0000 7fcce77fe640 1 -- 172.21.15.26:0/2400059310 >> v1:172.21.15.26:6789/0 conn(0x7fcce80cb150 legacy=0x7fcce815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.855+0000 7fcce77fe640 1 -- 172.21.15.26:0/2400059310 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce815b1b0 con 0x7fcce80cb600 2023-12-19T16:54:26.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.859+0000 7fcce77fe640 1 --2- 172.21.15.26:0/2400059310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce80cb600 0x7fcce815eda0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009d90 tx=0x7fcce001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a030e14b516d322f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:26.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.859+0000 7fcce67fc640 1 -- 172.21.15.26:0/2400059310 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce00a5950 con 0x7fcce80cb600 2023-12-19T16:54:26.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.859+0000 7fcce67fc640 1 -- 172.21.15.26:0/2400059310 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce00a5ab0 con 0x7fcce80cb600 2023-12-19T16:54:26.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.859+0000 7fcce67fc640 1 -- 172.21.15.26:0/2400059310 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce00a5e40 con 0x7fcce80cb600 2023-12-19T16:54:26.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.859+0000 7fcced6e5640 1 -- 172.21.15.26:0/2400059310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce80cb600 msgr2=0x7fcce815eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.859+0000 7fcced6e5640 1 --2- 172.21.15.26:0/2400059310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce80cb600 0x7fcce815eda0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009d90 tx=0x7fcce001c890 comp rx=0 tx=0).stop 2023-12-19T16:54:26.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.859+0000 7fcced6e5640 1 -- 172.21.15.26:0/2400059310 shutdown_connections 2023-12-19T16:54:26.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.859+0000 7fcced6e5640 1 --2- 172.21.15.26:0/2400059310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce80cb600 0x7fcce815eda0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.859+0000 7fcced6e5640 1 -- 172.21.15.26:0/2400059310 >> 172.21.15.26:0/2400059310 conn(0x7fcce8066160 msgr2=0x7fcce80b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:26.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.859+0000 7fcced6e5640 1 -- 172.21.15.26:0/2400059310 shutdown_connections 2023-12-19T16:54:26.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.859+0000 7fcced6e5640 1 -- 172.21.15.26:0/2400059310 wait complete. 2023-12-19T16:54:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.859+0000 7fcced6e5640 1 Processor -- start 2023-12-19T16:54:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.859+0000 7fcced6e5640 1 -- start start 2023-12-19T16:54:26.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcced6e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcce80cb150 0x7fcce8152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcced6e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcce80cb600 0x7fcce815a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcced6e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce81585f0 0x7fcce806c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcced6e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce80c76a0 con 0x7fcce81585f0 2023-12-19T16:54:26.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcced6e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce80c6600 con 0x7fcce80cb150 2023-12-19T16:54:26.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcced6e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce80c68e0 con 0x7fcce80cb600 2023-12-19T16:54:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcce77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce81585f0 0x7fcce806c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcce77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce81585f0 0x7fcce806c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40508/0 (socket says 172.21.15.26:40508) 2023-12-19T16:54:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcce77fe640 1 -- 172.21.15.26:0/1425254646 learned_addr learned my addr 172.21.15.26:0/1425254646 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcce6ffd640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcce80cb150 0x7fcce8152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcce77fe640 1 -- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcce80cb600 msgr2=0x7fcce815a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcce77fe640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcce80cb600 0x7fcce815a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcce77fe640 1 -- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcce80cb150 msgr2=0x7fcce8152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcce77fe640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcce80cb150 0x7fcce8152b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcce77fe640 1 -- 172.21.15.26:0/1425254646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce0009a70 con 0x7fcce81585f0 2023-12-19T16:54:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcce6ffd640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcce80cb150 0x7fcce8152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:54:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcce77fe640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce81585f0 0x7fcce806c490 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcce001a4c0 tx=0x7fcce0019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fccdf7fe640 1 -- 172.21.15.26:0/1425254646 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce00addc0 con 0x7fcce81585f0 2023-12-19T16:54:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fccdf7fe640 1 -- 172.21.15.26:0/1425254646 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce00b9450 con 0x7fcce81585f0 2023-12-19T16:54:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fccdf7fe640 1 -- 172.21.15.26:0/1425254646 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce00b97e0 con 0x7fcce81585f0 2023-12-19T16:54:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcced6e5640 1 -- 172.21.15.26:0/1425254646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce80c6bc0 con 0x7fcce81585f0 2023-12-19T16:54:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcced6e5640 1 -- 172.21.15.26:0/1425254646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce80c7180 con 0x7fcce81585f0 2023-12-19T16:54:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.863+0000 7fcced6e5640 1 -- 172.21.15.26:0/1425254646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce80c7370 con 0x7fcce81585f0 2023-12-19T16:54:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.867+0000 7fccdf7fe640 1 -- 172.21.15.26:0/1425254646 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fcce00b9940 con 0x7fcce81585f0 2023-12-19T16:54:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.867+0000 7fccdf7fe640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fccc8065c60 0x7fccc8068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.867+0000 7fccdf7fe640 1 -- 172.21.15.26:0/1425254646 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fcce011efb0 con 0x7fcce81585f0 2023-12-19T16:54:26.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.867+0000 7fcce6ffd640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fccc8065c60 0x7fccc8068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:26.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.867+0000 7fcce6ffd640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fccc8065c60 0x7fccc8068120 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fccd8000f70 tx=0x7fccd8009040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:26.871+0000 7fccdf7fe640 1 -- 172.21.15.26:0/1425254646 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcce00ec670 con 0x7fcce81585f0 2023-12-19T16:54:27.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.047+0000 7fcced6e5640 1 -- 172.21.15.26:0/1425254646 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcce81589d0 con 0x7fccc8065c60 2023-12-19T16:54:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.055+0000 7fccdf7fe640 1 -- 172.21.15.26:0/1425254646 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fccd8001370 con 0x7fccc8065c60 2023-12-19T16:54:27.060 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:27.060 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:46", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.059+0000 7fcced6e5640 1 -- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fccc8065c60 msgr2=0x7fccc8068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.059+0000 7fcced6e5640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fccc8065c60 0x7fccc8068120 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fccd8000f70 tx=0x7fccd8009040 comp rx=0 tx=0).stop 2023-12-19T16:54:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.059+0000 7fcced6e5640 1 -- 172.21.15.26:0/1425254646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce81585f0 msgr2=0x7fcce806c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.059+0000 7fcced6e5640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce81585f0 0x7fcce806c490 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcce001a4c0 tx=0x7fcce0019630 comp rx=0 tx=0).stop 2023-12-19T16:54:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.059+0000 7fcced6e5640 1 -- 172.21.15.26:0/1425254646 shutdown_connections 2023-12-19T16:54:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.059+0000 7fcced6e5640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcce81585f0 0x7fcce806c490 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.059+0000 7fcced6e5640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fccc8065c60 0x7fccc8068120 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.059+0000 7fcced6e5640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcce80cb600 0x7fcce815a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.059+0000 7fcced6e5640 1 --2- 172.21.15.26:0/1425254646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcce80cb150 0x7fcce8152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.059+0000 7fcced6e5640 1 -- 172.21.15.26:0/1425254646 >> 172.21.15.26:0/1425254646 conn(0x7fcce8066160 msgr2=0x7fcce8158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.059+0000 7fcced6e5640 1 -- 172.21.15.26:0/1425254646 shutdown_connections 2023-12-19T16:54:27.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.059+0000 7fcced6e5640 1 -- 172.21.15.26:0/1425254646 wait complete. 2023-12-19T16:54:27.081 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:46", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:27.081 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-12-19T00:00:00', 'created': '2023-12-19T16:43:46', 'first': '2023-12-19T16:44:00', 'last': '2023-12-19T16:54:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-19T16:54:27.082 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-19T16:54:27.082 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-19T16:54:27.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:27.082 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-12-19T16:54:27.092 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:27.092 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:27.092 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:27.092 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:27.092 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:27.092 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:27.092 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:27.092 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:27.093 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:27.093 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:27.093 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:27.094 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-19-16_44_00_UTC', 'scheduled-2023-12-19-16_45_00_UTC', 'scheduled-2023-12-19-16_46_00_UTC', 'scheduled-2023-12-19-16_47_00_UTC', 'scheduled-2023-12-19-16_48_00_UTC', 'scheduled-2023-12-19-16_49_00_UTC', 'scheduled-2023-12-19-16_50_00_UTC', 'scheduled-2023-12-19-16_51_00_UTC', 'scheduled-2023-12-19-16_52_00_UTC', 'scheduled-2023-12-19-16_53_00_UTC', 'scheduled-2023-12-19-16_54_00_UTC'] 2023-12-19T16:54:27.094 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir7 2023-12-19T16:54:27.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f29a6895640 1 Processor -- start 2023-12-19T16:54:27.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f29a6895640 1 -- start start 2023-12-19T16:54:27.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f29a6895640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f29a00cb600 0x7f29a015eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:27.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f29a6895640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a00ce3f0 con 0x7f29a01585f0 2023-12-19T16:54:27.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f29a6895640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a00cba50 con 0x7f29a00cb150 2023-12-19T16:54:27.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f29a6895640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a00b47a0 con 0x7f29a00cb600 2023-12-19T16:54:27.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f299ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f29a01585f0 0x7f29a00cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55940/0 (socket says 172.21.15.26:55940) 2023-12-19T16:54:27.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f299ffff640 1 -- 172.21.15.26:0/593182569 learned_addr learned my addr 172.21.15.26:0/593182569 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:27.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f29a4e32640 1 --2- 172.21.15.26:0/593182569 >> v2:172.21.15.26:3300/0 conn(0x7f29a00cb600 0x7f29a015eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:27.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f299effd640 1 -- 172.21.15.26:0/593182569 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326571390 0 0) 0x7f29a00ce3f0 con 0x7f29a01585f0 2023-12-19T16:54:27.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f299effd640 1 -- 172.21.15.26:0/593182569 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2988003530 con 0x7f29a01585f0 2023-12-19T16:54:27.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f29a4e32640 1 -- 172.21.15.26:0/593182569 >> v1:172.21.15.160:6789/0 conn(0x7f29a00cb150 legacy=0x7f29a015e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:27.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f29a4e32640 1 -- 172.21.15.26:0/593182569 >> v1:172.21.15.26:6789/0 conn(0x7f29a01585f0 legacy=0x7f29a00cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.279+0000 7f29a4e32640 1 -- 172.21.15.26:0/593182569 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a015b1b0 con 0x7f29a00cb600 2023-12-19T16:54:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.283+0000 7f29a4e32640 1 --2- 172.21.15.26:0/593182569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a00cb600 0x7f29a015eda0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2998009d90 tx=0x7f299801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc2f38ee900584e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.283+0000 7f299effd640 1 -- 172.21.15.26:0/593182569 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980a5910 con 0x7f29a00cb600 2023-12-19T16:54:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.283+0000 7f299effd640 1 -- 172.21.15.26:0/593182569 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29980a5a70 con 0x7f29a00cb600 2023-12-19T16:54:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.283+0000 7f299effd640 1 -- 172.21.15.26:0/593182569 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980a5e00 con 0x7f29a00cb600 2023-12-19T16:54:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.283+0000 7f29a6895640 1 -- 172.21.15.26:0/593182569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a00cb600 msgr2=0x7f29a015eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.283+0000 7f29a6895640 1 --2- 172.21.15.26:0/593182569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a00cb600 0x7f29a015eda0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2998009d90 tx=0x7f299801c850 comp rx=0 tx=0).stop 2023-12-19T16:54:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.283+0000 7f29a6895640 1 -- 172.21.15.26:0/593182569 shutdown_connections 2023-12-19T16:54:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.283+0000 7f29a6895640 1 --2- 172.21.15.26:0/593182569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a00cb600 0x7f29a015eda0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.283+0000 7f29a6895640 1 -- 172.21.15.26:0/593182569 >> 172.21.15.26:0/593182569 conn(0x7f29a0066160 msgr2=0x7f29a00b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.283+0000 7f29a6895640 1 -- 172.21.15.26:0/593182569 shutdown_connections 2023-12-19T16:54:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.283+0000 7f29a6895640 1 -- 172.21.15.26:0/593182569 wait complete. 2023-12-19T16:54:27.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.283+0000 7f29a6895640 1 Processor -- start 2023-12-19T16:54:27.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a6895640 1 -- start start 2023-12-19T16:54:27.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a6895640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29a00cb150 0x7f29a0157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:27.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a6895640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29a00cb600 0x7f29a0151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:27.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a6895640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a01585f0 0x7f29a00ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:27.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a6895640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a00c5920 con 0x7f29a01585f0 2023-12-19T16:54:27.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a6895640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a006cbf0 con 0x7f29a00cb600 2023-12-19T16:54:27.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a6895640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a01fa390 con 0x7f29a00cb150 2023-12-19T16:54:27.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a4e32640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a01585f0 0x7f29a00ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:27.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a4e32640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a01585f0 0x7f29a00ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40534/0 (socket says 172.21.15.26:40534) 2023-12-19T16:54:27.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a4e32640 1 -- 172.21.15.26:0/1414143509 learned_addr learned my addr 172.21.15.26:0/1414143509 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:27.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f299f7fe640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29a00cb600 0x7f29a0151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:27.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f299ffff640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29a00cb150 0x7f29a0157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:27.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a4e32640 1 -- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29a00cb150 msgr2=0x7f29a0157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a4e32640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29a00cb150 0x7f29a0157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a4e32640 1 -- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29a00cb600 msgr2=0x7f29a0151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a4e32640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29a00cb600 0x7f29a0151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a4e32640 1 -- 172.21.15.26:0/1414143509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2998009a70 con 0x7f29a01585f0 2023-12-19T16:54:27.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a4e32640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a01585f0 0x7f29a00ac860 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f299801a480 tx=0x7f29980195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:27.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f299cff9640 1 -- 172.21.15.26:0/1414143509 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980b9860 con 0x7f29a01585f0 2023-12-19T16:54:27.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f299cff9640 1 -- 172.21.15.26:0/1414143509 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29980a5900 con 0x7f29a01585f0 2023-12-19T16:54:27.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a6895640 1 -- 172.21.15.26:0/1414143509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a01fa670 con 0x7f29a01585f0 2023-12-19T16:54:27.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f299cff9640 1 -- 172.21.15.26:0/1414143509 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29980b9bc0 con 0x7f29a01585f0 2023-12-19T16:54:27.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a6895640 1 -- 172.21.15.26:0/1414143509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a006d0e0 con 0x7f29a01585f0 2023-12-19T16:54:27.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.287+0000 7f29a6895640 1 -- 172.21.15.26:0/1414143509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a0074270 con 0x7f29a01585f0 2023-12-19T16:54:27.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.291+0000 7f299cff9640 1 -- 172.21.15.26:0/1414143509 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f29980b9d20 con 0x7f29a01585f0 2023-12-19T16:54:27.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.291+0000 7f299cff9640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f298c065c60 0x7f298c068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:27.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.291+0000 7f299cff9640 1 -- 172.21.15.26:0/1414143509 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f299811ebd0 con 0x7f29a01585f0 2023-12-19T16:54:27.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.291+0000 7f299ffff640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f298c065c60 0x7f298c068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:27.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.291+0000 7f299ffff640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f298c065c60 0x7f298c068120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f29900011f0 tx=0x7f2990009040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:27.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.295+0000 7f299cff9640 1 -- 172.21.15.26:0/1414143509 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f29980ec200 con 0x7f29a01585f0 2023-12-19T16:54:27.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.471+0000 7f29a6895640 1 -- 172.21.15.26:0/1414143509 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29a01589d0 con 0x7f298c065c60 2023-12-19T16:54:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.483+0000 7f299cff9640 1 -- 172.21.15.26:0/1414143509 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f2990001340 con 0x7f298c065c60 2023-12-19T16:54:27.485 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:27.485 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:46", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:27.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.483+0000 7f29a6895640 1 -- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f298c065c60 msgr2=0x7f298c068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.483+0000 7f29a6895640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f298c065c60 0x7f298c068120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f29900011f0 tx=0x7f2990009040 comp rx=0 tx=0).stop 2023-12-19T16:54:27.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.483+0000 7f29a6895640 1 -- 172.21.15.26:0/1414143509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a01585f0 msgr2=0x7f29a00ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.483+0000 7f29a6895640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a01585f0 0x7f29a00ac860 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f299801a480 tx=0x7f29980195f0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.483+0000 7f29a6895640 1 -- 172.21.15.26:0/1414143509 shutdown_connections 2023-12-19T16:54:27.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.483+0000 7f29a6895640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a01585f0 0x7f29a00ac860 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.483+0000 7f29a6895640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29a00cb600 0x7f29a0151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.483+0000 7f29a6895640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f298c065c60 0x7f298c068120 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.483+0000 7f29a6895640 1 --2- 172.21.15.26:0/1414143509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29a00cb150 0x7f29a0157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.483+0000 7f29a6895640 1 -- 172.21.15.26:0/1414143509 >> 172.21.15.26:0/1414143509 conn(0x7f29a0066160 msgr2=0x7f29a0158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.483+0000 7f29a6895640 1 -- 172.21.15.26:0/1414143509 shutdown_connections 2023-12-19T16:54:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.483+0000 7f29a6895640 1 -- 172.21.15.26:0/1414143509 wait complete. 2023-12-19T16:54:27.506 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:46", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:27.506 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-12-19T00:00:00', 'created': '2023-12-19T16:43:46', 'first': '2023-12-19T16:44:00', 'last': '2023-12-19T16:54:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-19T16:54:27.506 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-19T16:54:27.506 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-19T16:54:27.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:27.507 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-12-19T16:54:27.517 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:27.517 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:27.517 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:27.517 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:27.517 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:27.517 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:27.517 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:27.517 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:27.517 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:27.517 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:27.517 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:27.518 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-19-16_44_00_UTC', 'scheduled-2023-12-19-16_45_00_UTC', 'scheduled-2023-12-19-16_46_00_UTC', 'scheduled-2023-12-19-16_47_00_UTC', 'scheduled-2023-12-19-16_48_00_UTC', 'scheduled-2023-12-19-16_49_00_UTC', 'scheduled-2023-12-19-16_50_00_UTC', 'scheduled-2023-12-19-16_51_00_UTC', 'scheduled-2023-12-19-16_52_00_UTC', 'scheduled-2023-12-19-16_53_00_UTC', 'scheduled-2023-12-19-16_54_00_UTC'] 2023-12-19T16:54:27.519 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir8 2023-12-19T16:54:27.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa301f640 1 Processor -- start 2023-12-19T16:54:27.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa301f640 1 -- start start 2023-12-19T16:54:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa301f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf9c15cd10 0x7faf9c0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa301f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf9c06d3e0 0x7faf9c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa301f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c15d0f0 con 0x7faf9c06c130 2023-12-19T16:54:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa301f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c06c5a0 con 0x7faf9c15cd10 2023-12-19T16:54:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa301f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c0c98b0 con 0x7faf9c06d3e0 2023-12-19T16:54:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa0dbb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf9c15cd10 0x7faf9c0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa0dbb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf9c15cd10 0x7faf9c0cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40536/0 (socket says 172.21.15.26:40536) 2023-12-19T16:54:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa0dbb640 1 -- 172.21.15.26:0/3890209037 learned_addr learned my addr 172.21.15.26:0/3890209037 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7faf9bfff640 1 --2- 172.21.15.26:0/3890209037 >> v2:172.21.15.160:3300/0 conn(0x7faf9c06d3e0 0x7faf9c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa0dbb640 1 -- 172.21.15.26:0/3890209037 >> v2:172.21.15.160:3300/0 conn(0x7faf9c06d3e0 msgr2=0x7faf9c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa0dbb640 1 --2- 172.21.15.26:0/3890209037 >> v2:172.21.15.160:3300/0 conn(0x7faf9c06d3e0 0x7faf9c06bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa0dbb640 1 -- 172.21.15.26:0/3890209037 >> v1:172.21.15.160:6789/0 conn(0x7faf9c06c130 legacy=0x7faf9c160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.703+0000 7fafa0dbb640 1 -- 172.21.15.26:0/3890209037 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c0cd0e0 con 0x7faf9c15cd10 2023-12-19T16:54:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.707+0000 7fafa0dbb640 1 --2- 172.21.15.26:0/3890209037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9c15cd10 0x7faf9c0cb680 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7faf9c06e780 tx=0x7faf9001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1416a49ee4c94938 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.707+0000 7faf9b7fe640 1 -- 172.21.15.26:0/3890209037 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf900a89f0 con 0x7faf9c15cd10 2023-12-19T16:54:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.707+0000 7faf9b7fe640 1 -- 172.21.15.26:0/3890209037 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf900a8b50 con 0x7faf9c15cd10 2023-12-19T16:54:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.707+0000 7faf9b7fe640 1 -- 172.21.15.26:0/3890209037 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf900b0440 con 0x7faf9c15cd10 2023-12-19T16:54:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.707+0000 7fafa301f640 1 -- 172.21.15.26:0/3890209037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9c15cd10 msgr2=0x7faf9c0cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.707+0000 7fafa301f640 1 --2- 172.21.15.26:0/3890209037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9c15cd10 0x7faf9c0cb680 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7faf9c06e780 tx=0x7faf9001ecf0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.707+0000 7fafa301f640 1 -- 172.21.15.26:0/3890209037 shutdown_connections 2023-12-19T16:54:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.707+0000 7fafa301f640 1 --2- 172.21.15.26:0/3890209037 >> v2:172.21.15.160:3300/0 conn(0x7faf9c06d3e0 0x7faf9c06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.707+0000 7fafa301f640 1 --2- 172.21.15.26:0/3890209037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9c15cd10 0x7faf9c0cb680 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.707+0000 7fafa301f640 1 -- 172.21.15.26:0/3890209037 >> 172.21.15.26:0/3890209037 conn(0x7faf9c0cf260 msgr2=0x7faf9c0cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.707+0000 7fafa301f640 1 -- 172.21.15.26:0/3890209037 shutdown_connections 2023-12-19T16:54:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.707+0000 7fafa301f640 1 -- 172.21.15.26:0/3890209037 wait complete. 2023-12-19T16:54:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa301f640 1 Processor -- start 2023-12-19T16:54:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa301f640 1 -- start start 2023-12-19T16:54:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa301f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf9c06c130 0x7faf9c148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa301f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c06d3e0 0x7faf9c1f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa301f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9c15cd10 0x7faf9c15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa301f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0c7a20 con 0x7faf9c15cd10 2023-12-19T16:54:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa301f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0c7d00 con 0x7faf9c06d3e0 2023-12-19T16:54:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa301f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0c7fe0 con 0x7faf9c06c130 2023-12-19T16:54:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa15bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9c15cd10 0x7faf9c15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa15bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9c15cd10 0x7faf9c15bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40544/0 (socket says 172.21.15.26:40544) 2023-12-19T16:54:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa15bc640 1 -- 172.21.15.26:0/1066776241 learned_addr learned my addr 172.21.15.26:0/1066776241 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7faf9bfff640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c06d3e0 0x7faf9c1f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa0dbb640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf9c06c130 0x7faf9c148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa15bc640 1 -- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf9c06c130 msgr2=0x7faf9c148680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa15bc640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf9c06c130 0x7faf9c148680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa15bc640 1 -- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c06d3e0 msgr2=0x7faf9c1f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa15bc640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c06d3e0 0x7faf9c1f26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa15bc640 1 -- 172.21.15.26:0/1066776241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf900094d0 con 0x7faf9c15cd10 2023-12-19T16:54:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7faf9bfff640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c06d3e0 0x7faf9c1f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa15bc640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9c15cd10 0x7faf9c15bd80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faf9400c730 tx=0x7faf9400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7faf997fa640 1 -- 172.21.15.26:0/1066776241 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf94003b80 con 0x7faf9c15cd10 2023-12-19T16:54:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7faf997fa640 1 -- 172.21.15.26:0/1066776241 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf94003ce0 con 0x7faf9c15cd10 2023-12-19T16:54:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7faf997fa640 1 -- 172.21.15.26:0/1066776241 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9400aa30 con 0x7faf9c15cd10 2023-12-19T16:54:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa301f640 1 -- 172.21.15.26:0/1066776241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c0c82c0 con 0x7faf9c15cd10 2023-12-19T16:54:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.711+0000 7fafa301f640 1 -- 172.21.15.26:0/1066776241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c148bc0 con 0x7faf9c15cd10 2023-12-19T16:54:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.715+0000 7fafa301f640 1 -- 172.21.15.26:0/1066776241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9c074270 con 0x7faf9c15cd10 2023-12-19T16:54:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.715+0000 7faf997fa640 1 -- 172.21.15.26:0/1066776241 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7faf9400ab90 con 0x7faf9c15cd10 2023-12-19T16:54:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.715+0000 7faf997fa640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faf80065c60 0x7faf80068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.715+0000 7faf997fa640 1 -- 172.21.15.26:0/1066776241 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7faf94083200 con 0x7faf9c15cd10 2023-12-19T16:54:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.715+0000 7fafa0dbb640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faf80065c60 0x7faf80068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.715+0000 7fafa0dbb640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faf80065c60 0x7faf80068120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf900b2000 tx=0x7faf90009040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:27.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.719+0000 7faf997fa640 1 -- 172.21.15.26:0/1066776241 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faf940507a0 con 0x7faf9c15cd10 2023-12-19T16:54:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.895+0000 7fafa301f640 1 -- 172.21.15.26:0/1066776241 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf9c0c7e70 con 0x7faf80065c60 2023-12-19T16:54:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.903+0000 7faf997fa640 1 -- 172.21.15.26:0/1066776241 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7faf9c0c7e70 con 0x7faf80065c60 2023-12-19T16:54:27.908 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:27.908 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:47", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.907+0000 7fafa301f640 1 -- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faf80065c60 msgr2=0x7faf80068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.907+0000 7fafa301f640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faf80065c60 0x7faf80068120 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf900b2000 tx=0x7faf90009040 comp rx=0 tx=0).stop 2023-12-19T16:54:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.907+0000 7fafa301f640 1 -- 172.21.15.26:0/1066776241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9c15cd10 msgr2=0x7faf9c15bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.907+0000 7fafa301f640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9c15cd10 0x7faf9c15bd80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faf9400c730 tx=0x7faf9400cbf0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.907+0000 7fafa301f640 1 -- 172.21.15.26:0/1066776241 shutdown_connections 2023-12-19T16:54:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.907+0000 7fafa301f640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9c15cd10 0x7faf9c15bd80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.907+0000 7fafa301f640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9c06d3e0 0x7faf9c1f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.907+0000 7fafa301f640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faf80065c60 0x7faf80068120 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.907+0000 7fafa301f640 1 --2- 172.21.15.26:0/1066776241 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf9c06c130 0x7faf9c148680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.907+0000 7fafa301f640 1 -- 172.21.15.26:0/1066776241 >> 172.21.15.26:0/1066776241 conn(0x7faf9c0cf260 msgr2=0x7faf9c0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.907+0000 7fafa301f640 1 -- 172.21.15.26:0/1066776241 shutdown_connections 2023-12-19T16:54:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:27.907+0000 7fafa301f640 1 -- 172.21.15.26:0/1066776241 wait complete. 2023-12-19T16:54:27.929 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:47", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:27.929 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-12-19T00:00:00', 'created': '2023-12-19T16:43:47', 'first': '2023-12-19T16:44:00', 'last': '2023-12-19T16:54:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-19T16:54:27.929 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-19T16:54:27.929 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-19T16:54:27.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:27.930 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-12-19T16:54:27.941 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:27.941 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:27.941 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:27.941 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:27.941 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_01_UTC 2023-12-19T16:54:27.942 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:27.942 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:27.942 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:27.942 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:27.942 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:27.942 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:27.943 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-19-16_44_00_UTC', 'scheduled-2023-12-19-16_45_00_UTC', 'scheduled-2023-12-19-16_46_00_UTC', 'scheduled-2023-12-19-16_47_00_UTC', 'scheduled-2023-12-19-16_48_01_UTC', 'scheduled-2023-12-19-16_49_00_UTC', 'scheduled-2023-12-19-16_50_00_UTC', 'scheduled-2023-12-19-16_51_00_UTC', 'scheduled-2023-12-19-16_52_00_UTC', 'scheduled-2023-12-19-16_53_00_UTC', 'scheduled-2023-12-19-16_54_00_UTC'] 2023-12-19T16:54:27.943 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir9 2023-12-19T16:54:28.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd981f39640 1 Processor -- start 2023-12-19T16:54:28.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd981f39640 1 -- start start 2023-12-19T16:54:28.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd981f39640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd97c1599e0 0x7fd97c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:28.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd981f39640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97c15e240 con 0x7fd97c15dc40 2023-12-19T16:54:28.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd981f39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97c0c9520 con 0x7fd97c0cb150 2023-12-19T16:54:28.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd981f39640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97c15e050 con 0x7fd97c1599e0 2023-12-19T16:54:28.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd97bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd97c15dc40 0x7fd97c0ca000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55950/0 (socket says 172.21.15.26:55950) 2023-12-19T16:54:28.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd97bfff640 1 -- 172.21.15.26:0/3686239670 learned_addr learned my addr 172.21.15.26:0/3686239670 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:28.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd97b7fe640 1 --2- 172.21.15.26:0/3686239670 >> v2:172.21.15.160:3300/0 conn(0x7fd97c1599e0 0x7fd97c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:28.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd97a7fc640 1 -- 172.21.15.26:0/3686239670 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550217025 0 0) 0x7fd97c15e240 con 0x7fd97c15dc40 2023-12-19T16:54:28.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd97a7fc640 1 -- 172.21.15.26:0/3686239670 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd964003530 con 0x7fd97c15dc40 2023-12-19T16:54:28.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd97a7fc640 1 -- 172.21.15.26:0/3686239670 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1521959639 0 0) 0x7fd97c0c9520 con 0x7fd97c0cb150 2023-12-19T16:54:28.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd97a7fc640 1 -- 172.21.15.26:0/3686239670 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd97c15e240 con 0x7fd97c0cb150 2023-12-19T16:54:28.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd97b7fe640 1 -- 172.21.15.26:0/3686239670 >> v1:172.21.15.160:6789/0 conn(0x7fd97c0cb150 legacy=0x7fd97c15d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd97b7fe640 1 -- 172.21.15.26:0/3686239670 >> v1:172.21.15.26:6789/0 conn(0x7fd97c15dc40 legacy=0x7fd97c0ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.127+0000 7fd97b7fe640 1 -- 172.21.15.26:0/3686239670 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c0c9800 con 0x7fd97c1599e0 2023-12-19T16:54:28.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.131+0000 7fd97b7fe640 1 --2- 172.21.15.26:0/3686239670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd97c1599e0 0x7fd97c06f6d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0097c0 tx=0x7fd96c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80fbee0933a3b973 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.131+0000 7fd97a7fc640 1 -- 172.21.15.26:0/3686239670 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c0caf20 con 0x7fd97c1599e0 2023-12-19T16:54:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.131+0000 7fd97a7fc640 1 -- 172.21.15.26:0/3686239670 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c0a7070 con 0x7fd97c1599e0 2023-12-19T16:54:28.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.131+0000 7fd97a7fc640 1 -- 172.21.15.26:0/3686239670 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0ab860 con 0x7fd97c1599e0 2023-12-19T16:54:28.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.131+0000 7fd981f39640 1 -- 172.21.15.26:0/3686239670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd97c1599e0 msgr2=0x7fd97c06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.131+0000 7fd981f39640 1 --2- 172.21.15.26:0/3686239670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd97c1599e0 0x7fd97c06f6d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0097c0 tx=0x7fd96c01c480 comp rx=0 tx=0).stop 2023-12-19T16:54:28.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.131+0000 7fd981f39640 1 -- 172.21.15.26:0/3686239670 shutdown_connections 2023-12-19T16:54:28.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.131+0000 7fd981f39640 1 --2- 172.21.15.26:0/3686239670 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd97c1599e0 0x7fd97c06f6d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.131+0000 7fd981f39640 1 -- 172.21.15.26:0/3686239670 >> 172.21.15.26:0/3686239670 conn(0x7fd97c066160 msgr2=0x7fd97c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:28.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.131+0000 7fd981f39640 1 -- 172.21.15.26:0/3686239670 shutdown_connections 2023-12-19T16:54:28.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.131+0000 7fd981f39640 1 -- 172.21.15.26:0/3686239670 wait complete. 2023-12-19T16:54:28.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd981f39640 1 Processor -- start 2023-12-19T16:54:28.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd981f39640 1 -- start start 2023-12-19T16:54:28.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd981f39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97c0cb150 0x7fd97c152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd981f39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd97c1599e0 0x7fd97c14a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd981f39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd97c15dc40 0x7fd97c0c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd981f39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c14ac80 con 0x7fd97c0cb150 2023-12-19T16:54:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd981f39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c153420 con 0x7fd97c1599e0 2023-12-19T16:54:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd981f39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c152810 con 0x7fd97c15dc40 2023-12-19T16:54:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd97b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97c0cb150 0x7fd97c152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd97b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97c0cb150 0x7fd97c152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40550/0 (socket says 172.21.15.26:40550) 2023-12-19T16:54:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd97b7fe640 1 -- 172.21.15.26:0/438909424 learned_addr learned my addr 172.21.15.26:0/438909424 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd97bfff640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd97c15dc40 0x7fd97c0c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd97affd640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd97c1599e0 0x7fd97c14a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd97b7fe640 1 -- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd97c15dc40 msgr2=0x7fd97c0c13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd97b7fe640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd97c15dc40 0x7fd97c0c13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd97b7fe640 1 -- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd97c1599e0 msgr2=0x7fd97c14a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd97b7fe640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd97c1599e0 0x7fd97c14a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd97b7fe640 1 -- 172.21.15.26:0/438909424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c009420 con 0x7fd97c0cb150 2023-12-19T16:54:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd97b7fe640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97c0cb150 0x7fd97c152180 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0ad040 tx=0x7fd96c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd963fff640 1 -- 172.21.15.26:0/438909424 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0abe10 con 0x7fd97c0cb150 2023-12-19T16:54:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd963fff640 1 -- 172.21.15.26:0/438909424 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c0bf410 con 0x7fd97c0cb150 2023-12-19T16:54:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd963fff640 1 -- 172.21.15.26:0/438909424 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0bf7c0 con 0x7fd97c0cb150 2023-12-19T16:54:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd981f39640 1 -- 172.21.15.26:0/438909424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c0cbaf0 con 0x7fd97c0cb150 2023-12-19T16:54:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd981f39640 1 -- 172.21.15.26:0/438909424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c152dd0 con 0x7fd97c0cb150 2023-12-19T16:54:28.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.135+0000 7fd981f39640 1 -- 172.21.15.26:0/438909424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97c074270 con 0x7fd97c0cb150 2023-12-19T16:54:28.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.139+0000 7fd963fff640 1 -- 172.21.15.26:0/438909424 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fd96c0b5400 con 0x7fd97c0cb150 2023-12-19T16:54:28.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.139+0000 7fd963fff640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd968065c90 0x7fd968068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:28.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.139+0000 7fd963fff640 1 -- 172.21.15.26:0/438909424 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fd96c11dcd0 con 0x7fd97c0cb150 2023-12-19T16:54:28.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.139+0000 7fd97affd640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd968065c90 0x7fd968068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:28.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.139+0000 7fd97affd640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd968065c90 0x7fd968068150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd97c151e00 tx=0x7fd970009040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:28.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.143+0000 7fd963fff640 1 -- 172.21.15.26:0/438909424 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd96c0eb210 con 0x7fd97c0cb150 2023-12-19T16:54:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.319+0000 7fd981f39640 1 -- 172.21.15.26:0/438909424 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd97c0cb970 con 0x7fd968065c90 2023-12-19T16:54:28.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.327+0000 7fd963fff640 1 -- 172.21.15.26:0/438909424 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fd970001cb0 con 0x7fd968065c90 2023-12-19T16:54:28.331 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:28.332 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:47", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:28.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.331+0000 7fd981f39640 1 -- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd968065c90 msgr2=0x7fd968068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.331+0000 7fd981f39640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd968065c90 0x7fd968068150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd97c151e00 tx=0x7fd970009040 comp rx=0 tx=0).stop 2023-12-19T16:54:28.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.331+0000 7fd981f39640 1 -- 172.21.15.26:0/438909424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97c0cb150 msgr2=0x7fd97c152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.331+0000 7fd981f39640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97c0cb150 0x7fd97c152180 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0ad040 tx=0x7fd96c01acc0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.331+0000 7fd981f39640 1 -- 172.21.15.26:0/438909424 shutdown_connections 2023-12-19T16:54:28.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.331+0000 7fd981f39640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd968065c90 0x7fd968068150 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.331+0000 7fd981f39640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd97c15dc40 0x7fd97c0c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.331+0000 7fd981f39640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd97c1599e0 0x7fd97c14a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.331+0000 7fd981f39640 1 --2- 172.21.15.26:0/438909424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97c0cb150 0x7fd97c152180 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.331+0000 7fd981f39640 1 -- 172.21.15.26:0/438909424 >> 172.21.15.26:0/438909424 conn(0x7fd97c066160 msgr2=0x7fd97c0ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:28.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.331+0000 7fd981f39640 1 -- 172.21.15.26:0/438909424 shutdown_connections 2023-12-19T16:54:28.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.331+0000 7fd981f39640 1 -- 172.21.15.26:0/438909424 wait complete. 2023-12-19T16:54:28.352 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:43:47", "first": "2023-12-19T16:44:00", "last": "2023-12-19T16:54:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-19T16:54:28.353 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-12-19T00:00:00', 'created': '2023-12-19T16:43:47', 'first': '2023-12-19T16:44:00', 'last': '2023-12-19T16:54:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-19T16:54:28.353 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-19T16:54:28.353 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-19T16:54:28.353 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir0 1M 2023-12-19T16:54:28.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb59a339640 1 Processor -- start 2023-12-19T16:54:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb59a339640 1 -- start start 2023-12-19T16:54:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb59a339640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb594159820 0x7fb59415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb59a339640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59406fb60 con 0x7fb59406f6c0 2023-12-19T16:54:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb59a339640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59406fe40 con 0x7fb594159c00 2023-12-19T16:54:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb59a339640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59415e470 con 0x7fb594159820 2023-12-19T16:54:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb5937fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb59406f6c0 0x7fb5940c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55960/0 (socket says 172.21.15.26:55960) 2023-12-19T16:54:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb5937fe640 1 -- 172.21.15.26:0/2572468463 learned_addr learned my addr 172.21.15.26:0/2572468463 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb593fff640 1 --2- 172.21.15.26:0/2572468463 >> v2:172.21.15.160:3300/0 conn(0x7fb594159820 0x7fb59415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:28.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594203371 0 0) 0x7fb59406fb60 con 0x7fb59406f6c0 2023-12-19T16:54:28.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb57c005740 con 0x7fb59406f6c0 2023-12-19T16:54:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053724500 0 0) 0x7fb59406fe40 con 0x7fb594159c00 2023-12-19T16:54:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb59406fb60 con 0x7fb594159c00 2023-12-19T16:54:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1096013552 0 0) 0x7fb57c005740 con 0x7fb59406f6c0 2023-12-19T16:54:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 >> v2:172.21.15.160:3300/0 conn(0x7fb594159820 msgr2=0x7fb59415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.495+0000 7fb592ffd640 1 --2- 172.21.15.26:0/2572468463 >> v2:172.21.15.160:3300/0 conn(0x7fb594159820 0x7fb59415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 >> v1:172.21.15.160:6789/0 conn(0x7fb594159c00 legacy=0x7fb594070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb594070120 con 0x7fb59406f6c0 2023-12-19T16:54:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fb588001680 con 0x7fb59406f6c0 2023-12-19T16:54:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 >> v1:172.21.15.26:6789/0 conn(0x7fb59406f6c0 legacy=0x7fb5940c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb592ffd640 1 --2- 172.21.15.26:0/2572468463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57c005d20 0x7fb57c006120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:28.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb592ffd640 1 --2- 172.21.15.26:0/2572468463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb57c0069e0 0x7fb57c006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb592ffd640 1 --2- 172.21.15.26:0/2572468463 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb57c0073a0 0x7fb57c00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59415e470 con 0x7fb57c0069e0 2023-12-19T16:54:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c00e220 con 0x7fb57c005d20 2023-12-19T16:54:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57c00e3e0 con 0x7fb57c0073a0 2023-12-19T16:54:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb5937fe640 1 --2- 172.21.15.26:0/2572468463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb57c0069e0 0x7fb57c006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb593fff640 1 --2- 172.21.15.26:0/2572468463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57c005d20 0x7fb57c006120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb5988d6640 1 --2- 172.21.15.26:0/2572468463 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb57c0073a0 0x7fb57c00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb5937fe640 1 -- 172.21.15.26:0/2572468463 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb57c0073a0 msgr2=0x7fb57c00dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb5937fe640 1 --2- 172.21.15.26:0/2572468463 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb57c0073a0 0x7fb57c00dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb5937fe640 1 -- 172.21.15.26:0/2572468463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57c005d20 msgr2=0x7fb57c006120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb5937fe640 1 --2- 172.21.15.26:0/2572468463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57c005d20 0x7fb57c006120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb5937fe640 1 -- 172.21.15.26:0/2572468463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c005740 con 0x7fb57c0069e0 2023-12-19T16:54:28.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb5988d6640 1 -- 172.21.15.26:0/2572468463 reap_dead start 2023-12-19T16:54:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb5937fe640 1 --2- 172.21.15.26:0/2572468463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb57c0069e0 0x7fb57c006e60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb59415d480 tx=0x7fb58801cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44417fc6bf52cf7d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5880058e0 con 0x7fb57c0069e0 2023-12-19T16:54:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb588005a60 con 0x7fb57c0069e0 2023-12-19T16:54:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.499+0000 7fb592ffd640 1 -- 172.21.15.26:0/2572468463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588005e50 con 0x7fb57c0069e0 2023-12-19T16:54:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 -- 172.21.15.26:0/2572468463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb57c0069e0 msgr2=0x7fb57c006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 --2- 172.21.15.26:0/2572468463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb57c0069e0 0x7fb57c006e60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb59415d480 tx=0x7fb58801cb80 comp rx=0 tx=0).stop 2023-12-19T16:54:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 -- 172.21.15.26:0/2572468463 shutdown_connections 2023-12-19T16:54:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 --2- 172.21.15.26:0/2572468463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb57c0069e0 0x7fb57c006e60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 -- 172.21.15.26:0/2572468463 >> 172.21.15.26:0/2572468463 conn(0x7fb5940ce730 msgr2=0x7fb5940ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 -- 172.21.15.26:0/2572468463 shutdown_connections 2023-12-19T16:54:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 -- 172.21.15.26:0/2572468463 wait complete. 2023-12-19T16:54:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 Processor -- start 2023-12-19T16:54:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 -- start start 2023-12-19T16:54:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57c0069e0 0x7fb5940b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59415c760 0x7fb5940c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb59415cbe0 0x7fb5940ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb594154c80 con 0x7fb59415c760 2023-12-19T16:54:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5941540b0 con 0x7fb57c0069e0 2023-12-19T16:54:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb59a339640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb594154350 con 0x7fb59415cbe0 2023-12-19T16:54:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb5937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59415c760 0x7fb5940c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb5937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59415c760 0x7fb5940c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40564/0 (socket says 172.21.15.26:40564) 2023-12-19T16:54:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb5937fe640 1 -- 172.21.15.26:0/1047106757 learned_addr learned my addr 172.21.15.26:0/1047106757 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb593fff640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57c0069e0 0x7fb5940b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.503+0000 7fb5988d6640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb59415cbe0 0x7fb5940ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb5937fe640 1 -- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb59415cbe0 msgr2=0x7fb5940ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb5937fe640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb59415cbe0 0x7fb5940ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb5937fe640 1 -- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57c0069e0 msgr2=0x7fb5940b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb5937fe640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57c0069e0 0x7fb5940b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb5937fe640 1 -- 172.21.15.26:0/1047106757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb594154630 con 0x7fb59415c760 2023-12-19T16:54:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb5988d6640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb59415cbe0 0x7fb5940ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:28.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb5937fe640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59415c760 0x7fb5940c9e90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb58801c9b0 tx=0x7fb57c007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:28.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb590ff9640 1 -- 172.21.15.26:0/1047106757 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5880a5c90 con 0x7fb59415c760 2023-12-19T16:54:28.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb590ff9640 1 -- 172.21.15.26:0/1047106757 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5880a5df0 con 0x7fb59415c760 2023-12-19T16:54:28.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb59a339640 1 -- 172.21.15.26:0/1047106757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb594154910 con 0x7fb59415c760 2023-12-19T16:54:28.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb590ff9640 1 -- 172.21.15.26:0/1047106757 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588004a30 con 0x7fb59415c760 2023-12-19T16:54:28.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb59a339640 1 -- 172.21.15.26:0/1047106757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5940afb80 con 0x7fb59415c760 2023-12-19T16:54:28.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb59a339640 1 -- 172.21.15.26:0/1047106757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb594074270 con 0x7fb59415c760 2023-12-19T16:54:28.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb590ff9640 1 -- 172.21.15.26:0/1047106757 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fb588004b90 con 0x7fb59415c760 2023-12-19T16:54:28.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb590ff9640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb580065c90 0x7fb580068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:28.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.507+0000 7fb590ff9640 1 -- 172.21.15.26:0/1047106757 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb58811d590 con 0x7fb59415c760 2023-12-19T16:54:28.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.511+0000 7fb593fff640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb580065c90 0x7fb580068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:28.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.511+0000 7fb593fff640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb580065c90 0x7fb580068150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb584009b80 tx=0x7fb584009190 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:28.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.511+0000 7fb590ff9640 1 -- 172.21.15.26:0/1047106757 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb5880eabc0 con 0x7fb59415c760 2023-12-19T16:54:28.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.691+0000 7fb59a339640 1 -- 172.21.15.26:0/1047106757 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5940706e0 con 0x7fb580065c90 2023-12-19T16:54:28.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.771+0000 7fb590ff9640 1 -- 172.21.15.26:0/1047106757 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fb584019070 con 0x7fb580065c90 2023-12-19T16:54:28.777 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-12-19T16:54:28.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.775+0000 7fb59a339640 1 -- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb580065c90 msgr2=0x7fb580068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.775+0000 7fb59a339640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb580065c90 0x7fb580068150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb584009b80 tx=0x7fb584009190 comp rx=0 tx=0).stop 2023-12-19T16:54:28.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.775+0000 7fb59a339640 1 -- 172.21.15.26:0/1047106757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59415c760 msgr2=0x7fb5940c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:28.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.775+0000 7fb59a339640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59415c760 0x7fb5940c9e90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb58801c9b0 tx=0x7fb57c007a10 comp rx=0 tx=0).stop 2023-12-19T16:54:28.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.775+0000 7fb59a339640 1 -- 172.21.15.26:0/1047106757 shutdown_connections 2023-12-19T16:54:28.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.775+0000 7fb59a339640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb580065c90 0x7fb580068150 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.775+0000 7fb59a339640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb59415cbe0 0x7fb5940ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.775+0000 7fb59a339640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59415c760 0x7fb5940c9e90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.775+0000 7fb59a339640 1 --2- 172.21.15.26:0/1047106757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb57c0069e0 0x7fb5940b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:28.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.775+0000 7fb59a339640 1 -- 172.21.15.26:0/1047106757 >> 172.21.15.26:0/1047106757 conn(0x7fb5940ce730 msgr2=0x7fb59415d260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:28.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.775+0000 7fb59a339640 1 -- 172.21.15.26:0/1047106757 shutdown_connections 2023-12-19T16:54:28.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:28.775+0000 7fb59a339640 1 -- 172.21.15.26:0/1047106757 wait complete. 2023-12-19T16:54:28.798 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-12-19T16:54:28.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:28.798 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-12-19T16:54:28.808 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:28.808 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:28.808 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:28.809 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:28.809 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:28.809 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:28.809 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:28.809 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:28.809 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:28.809 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:28.809 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:28.810 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:28.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:28.811 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_44_00_UTC) 2023-12-19T16:54:28.874 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:28.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:28.874 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_45_00_UTC) 2023-12-19T16:54:28.932 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:28.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:28.932 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_46_00_UTC) 2023-12-19T16:54:28.988 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:28.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:28.988 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_47_00_UTC) 2023-12-19T16:54:29.043 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:29.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:29.044 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_48_00_UTC) 2023-12-19T16:54:29.073 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:54:29.093 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:54:29.104 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:29.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:29.104 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_49_00_UTC) 2023-12-19T16:54:29.201 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:29.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:29.201 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_50_00_UTC) 2023-12-19T16:54:29.264 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:29.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:29.264 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_51_00_UTC) 2023-12-19T16:54:29.319 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:29.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:29.320 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_52_00_UTC) 2023-12-19T16:54:29.375 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:29.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:29.375 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_53_00_UTC) 2023-12-19T16:54:29.434 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:29.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:29.434 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-19-16_54_00_UTC) 2023-12-19T16:54:29.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:29.491 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-12-19T16:54:29.546 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir1 1M 2023-12-19T16:54:29.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c70996640 1 Processor -- start 2023-12-19T16:54:29.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c70996640 1 -- start start 2023-12-19T16:54:29.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c70996640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c6c159820 0x7f6c6c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:29.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c70996640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6c6c159c00 0x7f6c6c070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:29.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c70996640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c06fb60 con 0x7f6c6c06f6c0 2023-12-19T16:54:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c70996640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c6c15e470 con 0x7f6c6c159c00 2023-12-19T16:54:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c70996640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c06fe40 con 0x7f6c6c159820 2023-12-19T16:54:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c63fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6c6c06f6c0 0x7f6c6c0c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55970/0 (socket says 172.21.15.26:55970) 2023-12-19T16:54:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c63fff640 1 -- 172.21.15.26:0/1542851363 learned_addr learned my addr 172.21.15.26:0/1542851363 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c6ad9d640 1 --2- 172.21.15.26:0/1542851363 >> v2:172.21.15.26:3300/0 conn(0x7f6c6c159c00 0x7f6c6c070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c6a59c640 1 --2- 172.21.15.26:0/1542851363 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c6c159820 0x7f6c6c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c69d9b640 1 -- 172.21.15.26:0/1542851363 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579293079 0 0) 0x7f6c6c06fb60 con 0x7f6c6c06f6c0 2023-12-19T16:54:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c69d9b640 1 -- 172.21.15.26:0/1542851363 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c58005740 con 0x7f6c6c06f6c0 2023-12-19T16:54:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c6ad9d640 1 -- 172.21.15.26:0/1542851363 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c6c159820 msgr2=0x7f6c6c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c6ad9d640 1 --2- 172.21.15.26:0/1542851363 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c6c159820 0x7f6c6c15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c6ad9d640 1 -- 172.21.15.26:0/1542851363 >> v1:172.21.15.26:6789/0 conn(0x7f6c6c06f6c0 legacy=0x7f6c6c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.731+0000 7f6c6ad9d640 1 -- 172.21.15.26:0/1542851363 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c070120 con 0x7f6c6c159c00 2023-12-19T16:54:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.735+0000 7f6c6ad9d640 1 --2- 172.21.15.26:0/1542851363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c6c159c00 0x7f6c6c070c00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009d90 tx=0x7f6c6401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcebba933cd1b65 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.735+0000 7f6c69d9b640 1 -- 172.21.15.26:0/1542851363 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c640a58f0 con 0x7f6c6c159c00 2023-12-19T16:54:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.735+0000 7f6c69d9b640 1 -- 172.21.15.26:0/1542851363 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c640a5a50 con 0x7f6c6c159c00 2023-12-19T16:54:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.735+0000 7f6c69d9b640 1 -- 172.21.15.26:0/1542851363 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c640a5dc0 con 0x7f6c6c159c00 2023-12-19T16:54:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.735+0000 7f6c70996640 1 -- 172.21.15.26:0/1542851363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c6c159c00 msgr2=0x7f6c6c070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.735+0000 7f6c70996640 1 --2- 172.21.15.26:0/1542851363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c6c159c00 0x7f6c6c070c00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009d90 tx=0x7f6c6401c920 comp rx=0 tx=0).stop 2023-12-19T16:54:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.735+0000 7f6c70996640 1 -- 172.21.15.26:0/1542851363 shutdown_connections 2023-12-19T16:54:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.735+0000 7f6c70996640 1 --2- 172.21.15.26:0/1542851363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c6c159c00 0x7f6c6c070c00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.735+0000 7f6c70996640 1 --2- 172.21.15.26:0/1542851363 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c6c159820 0x7f6c6c15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.735+0000 7f6c70996640 1 -- 172.21.15.26:0/1542851363 >> 172.21.15.26:0/1542851363 conn(0x7f6c6c0ce730 msgr2=0x7f6c6c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.735+0000 7f6c70996640 1 -- 172.21.15.26:0/1542851363 shutdown_connections 2023-12-19T16:54:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.735+0000 7f6c70996640 1 -- 172.21.15.26:0/1542851363 wait complete. 2023-12-19T16:54:29.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c70996640 1 Processor -- start 2023-12-19T16:54:29.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c70996640 1 -- start start 2023-12-19T16:54:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c70996640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c6c06f6c0 0x7f6c6c14e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c70996640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c6c159820 0x7f6c6c150520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c70996640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c6c159c00 0x7f6c6c0c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c70996640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c150fc0 con 0x7f6c6c159c00 2023-12-19T16:54:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c70996640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c14ed40 con 0x7f6c6c06f6c0 2023-12-19T16:54:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c70996640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0b1e20 con 0x7f6c6c159820 2023-12-19T16:54:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c6ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c6c159c00 0x7f6c6c0c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c6ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c6c159c00 0x7f6c6c0c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40580/0 (socket says 172.21.15.26:40580) 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c6ad9d640 1 -- 172.21.15.26:0/3088539485 learned_addr learned my addr 172.21.15.26:0/3088539485 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c63fff640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c6c159820 0x7f6c6c150520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c6a59c640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c6c06f6c0 0x7f6c6c14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c6ad9d640 1 -- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c6c159820 msgr2=0x7f6c6c150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c6ad9d640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c6c159820 0x7f6c6c150520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c6ad9d640 1 -- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c6c06f6c0 msgr2=0x7f6c6c14e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c6ad9d640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c6c06f6c0 0x7f6c6c14e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c6ad9d640 1 -- 172.21.15.26:0/3088539485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c64009a70 con 0x7f6c6c159c00 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c63fff640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c6c159820 0x7f6c6c150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c6ad9d640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c6c159c00 0x7f6c6c0c5770 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6c6401a550 tx=0x7f6c640196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c62ffd640 1 -- 172.21.15.26:0/3088539485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c640ad850 con 0x7f6c6c159c00 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c62ffd640 1 -- 172.21.15.26:0/3088539485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c640ad9b0 con 0x7f6c6c159c00 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c62ffd640 1 -- 172.21.15.26:0/3088539485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c640add40 con 0x7f6c6c159c00 2023-12-19T16:54:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c70996640 1 -- 172.21.15.26:0/3088539485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6c06b880 con 0x7f6c6c159c00 2023-12-19T16:54:29.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.739+0000 7f6c70996640 1 -- 172.21.15.26:0/3088539485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6c0b17d0 con 0x7f6c6c159c00 2023-12-19T16:54:29.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.743+0000 7f6c70996640 1 -- 172.21.15.26:0/3088539485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6c0addd0 con 0x7f6c6c159c00 2023-12-19T16:54:29.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.743+0000 7f6c62ffd640 1 -- 172.21.15.26:0/3088539485 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f6c640b4070 con 0x7f6c6c159c00 2023-12-19T16:54:29.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.743+0000 7f6c62ffd640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c4c065c60 0x7f6c4c068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:29.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.743+0000 7f6c62ffd640 1 -- 172.21.15.26:0/3088539485 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f6c6411e2e0 con 0x7f6c6c159c00 2023-12-19T16:54:29.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.743+0000 7f6c6a59c640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c4c065c60 0x7f6c4c068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:29.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.743+0000 7f6c6a59c640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c4c065c60 0x7f6c4c068120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00b9d0 tx=0x7f6c5c008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:29.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.747+0000 7f6c62ffd640 1 -- 172.21.15.26:0/3088539485 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6c640eb880 con 0x7f6c6c159c00 2023-12-19T16:54:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:29.927+0000 7f6c70996640 1 -- 172.21.15.26:0/3088539485 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c6c06b700 con 0x7f6c4c065c60 2023-12-19T16:54:30.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.007+0000 7f6c62ffd640 1 -- 172.21.15.26:0/3088539485 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f6c5c00b5c0 con 0x7f6c4c065c60 2023-12-19T16:54:30.009 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-12-19T16:54:30.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.007+0000 7f6c70996640 1 -- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c4c065c60 msgr2=0x7f6c4c068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:30.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.007+0000 7f6c70996640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c4c065c60 0x7f6c4c068120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c00b9d0 tx=0x7f6c5c008040 comp rx=0 tx=0).stop 2023-12-19T16:54:30.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.007+0000 7f6c70996640 1 -- 172.21.15.26:0/3088539485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c6c159c00 msgr2=0x7f6c6c0c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:30.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.007+0000 7f6c70996640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c6c159c00 0x7f6c6c0c5770 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6c6401a550 tx=0x7f6c640196c0 comp rx=0 tx=0).stop 2023-12-19T16:54:30.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.007+0000 7f6c70996640 1 -- 172.21.15.26:0/3088539485 shutdown_connections 2023-12-19T16:54:30.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.007+0000 7f6c70996640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c6c159c00 0x7f6c6c0c5770 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:30.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.007+0000 7f6c70996640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c4c065c60 0x7f6c4c068120 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:30.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.007+0000 7f6c70996640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c6c159820 0x7f6c6c150520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:30.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.007+0000 7f6c70996640 1 --2- 172.21.15.26:0/3088539485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c6c06f6c0 0x7f6c6c14e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:30.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.007+0000 7f6c70996640 1 -- 172.21.15.26:0/3088539485 >> 172.21.15.26:0/3088539485 conn(0x7f6c6c0ce730 msgr2=0x7f6c6c06ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:30.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.007+0000 7f6c70996640 1 -- 172.21.15.26:0/3088539485 shutdown_connections 2023-12-19T16:54:30.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.007+0000 7f6c70996640 1 -- 172.21.15.26:0/3088539485 wait complete. 2023-12-19T16:54:30.030 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-12-19T16:54:30.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:30.030 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-12-19T16:54:30.040 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:30.040 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:30.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:30.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:30.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:30.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:30.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:30.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:30.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:30.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:30.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:30.042 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:30.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:30.042 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_44_00_UTC) 2023-12-19T16:54:30.101 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:30.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:30.101 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_45_00_UTC) 2023-12-19T16:54:30.160 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:30.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:30.160 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_46_00_UTC) 2023-12-19T16:54:30.216 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:30.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:30.216 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_47_00_UTC) 2023-12-19T16:54:30.272 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:30.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:30.272 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_48_00_UTC) 2023-12-19T16:54:30.336 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:30.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:30.337 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_49_00_UTC) 2023-12-19T16:54:30.396 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:30.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:30.396 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_50_00_UTC) 2023-12-19T16:54:30.452 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:30.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:30.452 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_51_00_UTC) 2023-12-19T16:54:30.507 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:30.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:30.508 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_52_00_UTC) 2023-12-19T16:54:30.564 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:30.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:30.564 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_53_00_UTC) 2023-12-19T16:54:30.620 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:30.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:30.620 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-19-16_54_00_UTC) 2023-12-19T16:54:30.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:30.680 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-12-19T16:54:30.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir2 1M 2023-12-19T16:54:30.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.915+0000 7fa304095640 1 Processor -- start 2023-12-19T16:54:30.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.915+0000 7fa304095640 1 -- start start 2023-12-19T16:54:30.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.915+0000 7fa304095640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2fc06c3f0 0x7fa2fc06c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:30.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.915+0000 7fa304095640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2fc0c8df0 0x7fa2fc0c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:30.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.915+0000 7fa304095640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2fc0b01b0 con 0x7fa2fc161230 2023-12-19T16:54:30.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.915+0000 7fa304095640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2fc0ca4e0 con 0x7fa2fc06c3f0 2023-12-19T16:54:30.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.915+0000 7fa304095640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0b0400 con 0x7fa2fc0c8df0 2023-12-19T16:54:30.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.915+0000 7fa301630640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2fc06c3f0 0x7fa2fc06c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:30.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.915+0000 7fa301630640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2fc06c3f0 0x7fa2fc06c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40582/0 (socket says 172.21.15.26:40582) 2023-12-19T16:54:30.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.915+0000 7fa301630640 1 -- 172.21.15.26:0/1481998667 learned_addr learned my addr 172.21.15.26:0/1481998667 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:30.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.919+0000 7fa302632640 1 --2- 172.21.15.26:0/1481998667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2fc0c8df0 0x7fa2fc0c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:30.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.919+0000 7fa301630640 1 -- 172.21.15.26:0/1481998667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2fc0c8df0 msgr2=0x7fa2fc0c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:30.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.919+0000 7fa301630640 1 --2- 172.21.15.26:0/1481998667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2fc0c8df0 0x7fa2fc0c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:30.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.919+0000 7fa301630640 1 -- 172.21.15.26:0/1481998667 >> v1:172.21.15.160:6789/0 conn(0x7fa2fc161230 legacy=0x7fa2fc0ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:30.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.919+0000 7fa301630640 1 -- 172.21.15.26:0/1481998667 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc0b06e0 con 0x7fa2fc06c3f0 2023-12-19T16:54:30.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.919+0000 7fa301630640 1 --2- 172.21.15.26:0/1481998667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2fc06c3f0 0x7fa2fc06c850 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009870 tx=0x7fa2ec01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3972559e1d0f17c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:30.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.919+0000 7fa300e2f640 1 -- 172.21.15.26:0/1481998667 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0a6040 con 0x7fa2fc06c3f0 2023-12-19T16:54:30.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.919+0000 7fa300e2f640 1 -- 172.21.15.26:0/1481998667 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec0aa600 con 0x7fa2fc06c3f0 2023-12-19T16:54:30.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.919+0000 7fa300e2f640 1 -- 172.21.15.26:0/1481998667 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0aa990 con 0x7fa2fc06c3f0 2023-12-19T16:54:30.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.919+0000 7fa304095640 1 -- 172.21.15.26:0/1481998667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2fc06c3f0 msgr2=0x7fa2fc06c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:30.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.919+0000 7fa304095640 1 --2- 172.21.15.26:0/1481998667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2fc06c3f0 0x7fa2fc06c850 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009870 tx=0x7fa2ec01c480 comp rx=0 tx=0).stop 2023-12-19T16:54:30.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 -- 172.21.15.26:0/1481998667 shutdown_connections 2023-12-19T16:54:30.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 --2- 172.21.15.26:0/1481998667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2fc0c8df0 0x7fa2fc0c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:30.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 --2- 172.21.15.26:0/1481998667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2fc06c3f0 0x7fa2fc06c850 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:30.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 -- 172.21.15.26:0/1481998667 >> 172.21.15.26:0/1481998667 conn(0x7fa2fc0ce730 msgr2=0x7fa2fc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:30.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 -- 172.21.15.26:0/1481998667 shutdown_connections 2023-12-19T16:54:30.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 -- 172.21.15.26:0/1481998667 wait complete. 2023-12-19T16:54:30.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 Processor -- start 2023-12-19T16:54:30.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 -- start start 2023-12-19T16:54:30.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2fc06c3f0 0x7fa2fc152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:30.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2fc0c8df0 0x7fa2fc0ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:30.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2fc161230 0x7fa2fc157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:30.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0704b0 con 0x7fa2fc161230 2023-12-19T16:54:30.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc06f830 con 0x7fa2fc0c8df0 2023-12-19T16:54:30.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa304095640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc06fb10 con 0x7fa2fc06c3f0 2023-12-19T16:54:30.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa302632640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2fc161230 0x7fa2fc157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:30.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa302632640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2fc161230 0x7fa2fc157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40586/0 (socket says 172.21.15.26:40586) 2023-12-19T16:54:30.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa302632640 1 -- 172.21.15.26:0/3634981685 learned_addr learned my addr 172.21.15.26:0/3634981685 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:30.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa301e31640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2fc06c3f0 0x7fa2fc152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:30.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa301630640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2fc0c8df0 0x7fa2fc0ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:30.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa302632640 1 -- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2fc06c3f0 msgr2=0x7fa2fc152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:30.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa302632640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2fc06c3f0 0x7fa2fc152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:30.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa302632640 1 -- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2fc0c8df0 msgr2=0x7fa2fc0ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:30.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa302632640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2fc0c8df0 0x7fa2fc0ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:30.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa302632640 1 -- 172.21.15.26:0/3634981685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec0094d0 con 0x7fa2fc161230 2023-12-19T16:54:30.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.923+0000 7fa302632640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2fc161230 0x7fa2fc157b30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400efc0 tx=0x7fa2f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:30.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.927+0000 7fa2f27fc640 1 -- 172.21.15.26:0/3634981685 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4014480 con 0x7fa2fc161230 2023-12-19T16:54:30.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.927+0000 7fa2f27fc640 1 -- 172.21.15.26:0/3634981685 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f400f040 con 0x7fa2fc161230 2023-12-19T16:54:30.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.927+0000 7fa2f27fc640 1 -- 172.21.15.26:0/3634981685 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4014820 con 0x7fa2fc161230 2023-12-19T16:54:30.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.927+0000 7fa304095640 1 -- 172.21.15.26:0/3634981685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2fc06fdf0 con 0x7fa2fc161230 2023-12-19T16:54:30.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.927+0000 7fa304095640 1 -- 172.21.15.26:0/3634981685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2fc153120 con 0x7fa2fc161230 2023-12-19T16:54:30.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.927+0000 7fa304095640 1 -- 172.21.15.26:0/3634981685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2fc074270 con 0x7fa2fc161230 2023-12-19T16:54:30.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.927+0000 7fa2f27fc640 1 -- 172.21.15.26:0/3634981685 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fa2f4014980 con 0x7fa2fc161230 2023-12-19T16:54:30.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.927+0000 7fa2f27fc640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2e0065c60 0x7fa2e0068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:30.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.927+0000 7fa2f27fc640 1 -- 172.21.15.26:0/3634981685 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa2f4084140 con 0x7fa2fc161230 2023-12-19T16:54:30.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.927+0000 7fa301e31640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2e0065c60 0x7fa2e0068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:30.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.931+0000 7fa301e31640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2e0065c60 0x7fa2e0068120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8007fd0 tx=0x7fa2f8009040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:30.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:30.931+0000 7fa2f27fc640 1 -- 172.21.15.26:0/3634981685 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa2f4051710 con 0x7fa2fc161230 2023-12-19T16:54:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.107+0000 7fa304095640 1 -- 172.21.15.26:0/3634981685 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2fc0ca160 con 0x7fa2e0065c60 2023-12-19T16:54:31.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.191+0000 7fa2f27fc640 1 -- 172.21.15.26:0/3634981685 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fa2f80014f0 con 0x7fa2e0065c60 2023-12-19T16:54:31.195 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-12-19T16:54:31.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.191+0000 7fa304095640 1 -- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2e0065c60 msgr2=0x7fa2e0068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:31.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.191+0000 7fa304095640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2e0065c60 0x7fa2e0068120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8007fd0 tx=0x7fa2f8009040 comp rx=0 tx=0).stop 2023-12-19T16:54:31.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.191+0000 7fa304095640 1 -- 172.21.15.26:0/3634981685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2fc161230 msgr2=0x7fa2fc157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:31.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.191+0000 7fa304095640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2fc161230 0x7fa2fc157b30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400efc0 tx=0x7fa2f400c490 comp rx=0 tx=0).stop 2023-12-19T16:54:31.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.191+0000 7fa304095640 1 -- 172.21.15.26:0/3634981685 shutdown_connections 2023-12-19T16:54:31.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.191+0000 7fa304095640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2fc161230 0x7fa2fc157b30 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:31.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.191+0000 7fa304095640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2fc0c8df0 0x7fa2fc0ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:31.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.191+0000 7fa304095640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2e0065c60 0x7fa2e0068120 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:31.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.191+0000 7fa304095640 1 --2- 172.21.15.26:0/3634981685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2fc06c3f0 0x7fa2fc152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:31.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.191+0000 7fa304095640 1 -- 172.21.15.26:0/3634981685 >> 172.21.15.26:0/3634981685 conn(0x7fa2fc0ce730 msgr2=0x7fa2fc0ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:31.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.191+0000 7fa304095640 1 -- 172.21.15.26:0/3634981685 shutdown_connections 2023-12-19T16:54:31.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:31.191+0000 7fa304095640 1 -- 172.21.15.26:0/3634981685 wait complete. 2023-12-19T16:54:31.215 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-12-19T16:54:31.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:31.215 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-12-19T16:54:31.226 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:31.226 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:31.226 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:31.226 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:31.226 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:31.226 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:31.226 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:31.226 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:31.226 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:31.226 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:31.226 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:31.227 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:31.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:31.228 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_44_00_UTC) 2023-12-19T16:54:31.285 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:31.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:31.285 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_45_00_UTC) 2023-12-19T16:54:31.348 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:31.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:31.349 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_46_00_UTC) 2023-12-19T16:54:31.412 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:31.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:31.413 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_47_00_UTC) 2023-12-19T16:54:31.476 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:31.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:31.476 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_48_00_UTC) 2023-12-19T16:54:31.541 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:31.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:31.541 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_49_00_UTC) 2023-12-19T16:54:31.604 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:31.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:31.604 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_50_00_UTC) 2023-12-19T16:54:31.663 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:31.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:31.664 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_51_00_UTC) 2023-12-19T16:54:31.720 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:31.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:31.721 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_52_00_UTC) 2023-12-19T16:54:31.780 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:31.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:31.780 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_53_00_UTC) 2023-12-19T16:54:31.836 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:31.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:31.836 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-19-16_54_00_UTC) 2023-12-19T16:54:31.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:31.896 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-12-19T16:54:31.953 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir3 1M 2023-12-19T16:54:32.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.139+0000 7f787f066640 1 Processor -- start 2023-12-19T16:54:32.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.139+0000 7f787f066640 1 -- start start 2023-12-19T16:54:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.139+0000 7f787f066640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f787815ada0 0x7f78780706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.139+0000 7f787f066640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7878070d90 con 0x7f787806ac80 2023-12-19T16:54:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.139+0000 7f787f066640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7878071070 con 0x7f787815a9c0 2023-12-19T16:54:32.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.139+0000 7f787f066640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78780cc270 con 0x7f787815ada0 2023-12-19T16:54:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.143+0000 7f787d603640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f787806ac80 0x7f78780b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55976/0 (socket says 172.21.15.26:55976) 2023-12-19T16:54:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.143+0000 7f787d603640 1 -- 172.21.15.26:0/914109392 learned_addr learned my addr 172.21.15.26:0/914109392 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.143+0000 7f7877fff640 1 --2- 172.21.15.26:0/914109392 >> v2:172.21.15.160:3300/0 conn(0x7f787815ada0 0x7f78780706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.143+0000 7f78777fe640 1 -- 172.21.15.26:0/914109392 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317686853 0 0) 0x7f7878070d90 con 0x7f787806ac80 2023-12-19T16:54:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.143+0000 7f78777fe640 1 -- 172.21.15.26:0/914109392 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7868003530 con 0x7f787806ac80 2023-12-19T16:54:32.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.143+0000 7f7877fff640 1 -- 172.21.15.26:0/914109392 >> v1:172.21.15.160:6789/0 conn(0x7f787815a9c0 legacy=0x7f78780b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:32.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.143+0000 7f7877fff640 1 -- 172.21.15.26:0/914109392 >> v1:172.21.15.26:6789/0 conn(0x7f787806ac80 legacy=0x7f78780b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:32.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.143+0000 7f7877fff640 1 -- 172.21.15.26:0/914109392 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7878071350 con 0x7f787815ada0 2023-12-19T16:54:32.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.143+0000 7f7877fff640 1 --2- 172.21.15.26:0/914109392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787815ada0 0x7f78780706d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f78640097c0 tx=0x7f786401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1496b81513c44bd7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:32.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.143+0000 7f78777fe640 1 -- 172.21.15.26:0/914109392 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640ac640 con 0x7f787815ada0 2023-12-19T16:54:32.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.143+0000 7f78777fe640 1 -- 172.21.15.26:0/914109392 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78640ac7a0 con 0x7f787815ada0 2023-12-19T16:54:32.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.143+0000 7f78777fe640 1 -- 172.21.15.26:0/914109392 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640acb10 con 0x7f787815ada0 2023-12-19T16:54:32.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 -- 172.21.15.26:0/914109392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787815ada0 msgr2=0x7f78780706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:32.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 --2- 172.21.15.26:0/914109392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787815ada0 0x7f78780706d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f78640097c0 tx=0x7f786401c3c0 comp rx=0 tx=0).stop 2023-12-19T16:54:32.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 -- 172.21.15.26:0/914109392 shutdown_connections 2023-12-19T16:54:32.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 --2- 172.21.15.26:0/914109392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787815ada0 0x7f78780706d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:32.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 -- 172.21.15.26:0/914109392 >> 172.21.15.26:0/914109392 conn(0x7f78780ce730 msgr2=0x7f78780ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:32.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 -- 172.21.15.26:0/914109392 shutdown_connections 2023-12-19T16:54:32.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 -- 172.21.15.26:0/914109392 wait complete. 2023-12-19T16:54:32.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 Processor -- start 2023-12-19T16:54:32.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 -- start start 2023-12-19T16:54:32.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787806ac80 0x7f7878152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:32.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f787815a9c0 0x7f78780b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:32.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f787815ada0 0x7f7878151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:32.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780c5da0 con 0x7f787815a9c0 2023-12-19T16:54:32.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780c5770 con 0x7f787806ac80 2023-12-19T16:54:32.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f787f066640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78781520a0 con 0x7f787815ada0 2023-12-19T16:54:32.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f7877fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f787815a9c0 0x7f78780b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:32.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f7877fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f787815a9c0 0x7f78780b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40600/0 (socket says 172.21.15.26:40600) 2023-12-19T16:54:32.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.147+0000 7f7877fff640 1 -- 172.21.15.26:0/3734446142 learned_addr learned my addr 172.21.15.26:0/3734446142 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:32.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f787ce02640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787806ac80 0x7f7878152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:32.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f787d603640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f787815ada0 0x7f7878151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:32.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f7877fff640 1 -- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f787815ada0 msgr2=0x7f7878151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:32.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f7877fff640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f787815ada0 0x7f7878151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:32.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f7877fff640 1 -- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787806ac80 msgr2=0x7f7878152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:32.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f7877fff640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787806ac80 0x7f7878152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:32.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f7877fff640 1 -- 172.21.15.26:0/3734446142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864009420 con 0x7f787815a9c0 2023-12-19T16:54:32.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f7877fff640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f787815a9c0 0x7f78780b02e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7864097eb0 tx=0x7f786401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:32.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f78757fa640 1 -- 172.21.15.26:0/3734446142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640ac910 con 0x7f787815a9c0 2023-12-19T16:54:32.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f78757fa640 1 -- 172.21.15.26:0/3734446142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78640aca70 con 0x7f787815a9c0 2023-12-19T16:54:32.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f787f066640 1 -- 172.21.15.26:0/3734446142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78780c7e20 con 0x7f787815a9c0 2023-12-19T16:54:32.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f78757fa640 1 -- 172.21.15.26:0/3734446142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640acde0 con 0x7f787815a9c0 2023-12-19T16:54:32.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f787f066640 1 -- 172.21.15.26:0/3734446142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78780c83d0 con 0x7f787815a9c0 2023-12-19T16:54:32.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f787f066640 1 -- 172.21.15.26:0/3734446142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78780c5ee0 con 0x7f787815a9c0 2023-12-19T16:54:32.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f78757fa640 1 -- 172.21.15.26:0/3734446142 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f7864003620 con 0x7f787815a9c0 2023-12-19T16:54:32.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f78757fa640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f785c065c60 0x7f785c068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:32.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.151+0000 7f78757fa640 1 -- 172.21.15.26:0/3734446142 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f786411db20 con 0x7f787815a9c0 2023-12-19T16:54:32.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.155+0000 7f787ce02640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f785c065c60 0x7f785c068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:32.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.155+0000 7f787ce02640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f785c065c60 0x7f785c068120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7878155770 tx=0x7f786c009040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:32.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.155+0000 7f78757fa640 1 -- 172.21.15.26:0/3734446142 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f78780c5ee0 con 0x7f787815a9c0 2023-12-19T16:54:32.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.335+0000 7f787f066640 1 -- 172.21.15.26:0/3734446142 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7878168ad0 con 0x7f785c065c60 2023-12-19T16:54:32.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.415+0000 7f78757fa640 1 -- 172.21.15.26:0/3734446142 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f786c001470 con 0x7f785c065c60 2023-12-19T16:54:32.420 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-12-19T16:54:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.419+0000 7f787f066640 1 -- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f785c065c60 msgr2=0x7f785c068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.419+0000 7f787f066640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f785c065c60 0x7f785c068120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7878155770 tx=0x7f786c009040 comp rx=0 tx=0).stop 2023-12-19T16:54:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.419+0000 7f787f066640 1 -- 172.21.15.26:0/3734446142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f787815a9c0 msgr2=0x7f78780b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.419+0000 7f787f066640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f787815a9c0 0x7f78780b02e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7864097eb0 tx=0x7f786401ac00 comp rx=0 tx=0).stop 2023-12-19T16:54:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.419+0000 7f787f066640 1 -- 172.21.15.26:0/3734446142 shutdown_connections 2023-12-19T16:54:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.419+0000 7f787f066640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f785c065c60 0x7f785c068120 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.419+0000 7f787f066640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f787815ada0 0x7f7878151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.419+0000 7f787f066640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f787815a9c0 0x7f78780b02e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.419+0000 7f787f066640 1 --2- 172.21.15.26:0/3734446142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f787806ac80 0x7f7878152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.419+0000 7f787f066640 1 -- 172.21.15.26:0/3734446142 >> 172.21.15.26:0/3734446142 conn(0x7f78780ce730 msgr2=0x7f78781560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.419+0000 7f787f066640 1 -- 172.21.15.26:0/3734446142 shutdown_connections 2023-12-19T16:54:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:32.419+0000 7f787f066640 1 -- 172.21.15.26:0/3734446142 wait complete. 2023-12-19T16:54:32.442 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-12-19T16:54:32.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:32.442 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-12-19T16:54:32.452 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:32.452 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:32.452 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:32.453 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:32.453 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:32.453 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:32.453 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:32.453 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:32.453 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:32.453 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:32.453 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:32.454 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:32.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:32.454 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_44_00_UTC) 2023-12-19T16:54:32.523 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:32.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:32.524 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_45_00_UTC) 2023-12-19T16:54:32.589 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:32.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:32.590 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_46_00_UTC) 2023-12-19T16:54:32.652 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:32.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:32.652 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_47_00_UTC) 2023-12-19T16:54:32.718 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:32.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:32.719 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_48_00_UTC) 2023-12-19T16:54:32.778 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:32.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:32.778 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_49_00_UTC) 2023-12-19T16:54:32.843 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:32.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:32.843 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_50_00_UTC) 2023-12-19T16:54:32.907 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:32.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:32.908 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_51_00_UTC) 2023-12-19T16:54:32.971 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:32.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:32.971 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_52_00_UTC) 2023-12-19T16:54:33.029 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:33.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:33.029 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_53_00_UTC) 2023-12-19T16:54:33.089 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:33.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:33.090 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-19-16_54_00_UTC) 2023-12-19T16:54:33.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:33.154 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-12-19T16:54:33.209 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir4 1M 2023-12-19T16:54:33.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e68d98640 1 Processor -- start 2023-12-19T16:54:33.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e68d98640 1 -- start start 2023-12-19T16:54:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e68d98640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e6406f6c0 0x7f2e6415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e68d98640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e64159c00 0x7f2e64070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e68d98640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e6406fb60 con 0x7f2e64159820 2023-12-19T16:54:33.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e68d98640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6415e470 con 0x7f2e6406f6c0 2023-12-19T16:54:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e68d98640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6406fe40 con 0x7f2e64159c00 2023-12-19T16:54:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e5bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e6406f6c0 0x7f2e6415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e5bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e6406f6c0 0x7f2e6415df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40614/0 (socket says 172.21.15.26:40614) 2023-12-19T16:54:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e5bfff640 1 -- 172.21.15.26:0/1672797616 learned_addr learned my addr 172.21.15.26:0/1672797616 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e62d9d640 1 --2- 172.21.15.26:0/1672797616 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e64159c00 0x7f2e64070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e5bfff640 1 -- 172.21.15.26:0/1672797616 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e64159c00 msgr2=0x7f2e64070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e5bfff640 1 --2- 172.21.15.26:0/1672797616 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e64159c00 0x7f2e64070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e5bfff640 1 -- 172.21.15.26:0/1672797616 >> v1:172.21.15.160:6789/0 conn(0x7f2e64159820 legacy=0x7f2e640c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:33.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.395+0000 7f2e5bfff640 1 -- 172.21.15.26:0/1672797616 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e64070120 con 0x7f2e6406f6c0 2023-12-19T16:54:33.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.399+0000 7f2e5bfff640 1 --2- 172.21.15.26:0/1672797616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6406f6c0 0x7f2e6415df30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009870 tx=0x7f2e4c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e2dd3a56cc8df server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.399+0000 7f2e61d9b640 1 -- 172.21.15.26:0/1672797616 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0ac600 con 0x7f2e6406f6c0 2023-12-19T16:54:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.399+0000 7f2e61d9b640 1 -- 172.21.15.26:0/1672797616 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4c0a8050 con 0x7f2e6406f6c0 2023-12-19T16:54:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.399+0000 7f2e61d9b640 1 -- 172.21.15.26:0/1672797616 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c0ac980 con 0x7f2e6406f6c0 2023-12-19T16:54:33.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.399+0000 7f2e68d98640 1 -- 172.21.15.26:0/1672797616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6406f6c0 msgr2=0x7f2e6415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:33.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.399+0000 7f2e68d98640 1 --2- 172.21.15.26:0/1672797616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6406f6c0 0x7f2e6415df30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009870 tx=0x7f2e4c01c4d0 comp rx=0 tx=0).stop 2023-12-19T16:54:33.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.399+0000 7f2e68d98640 1 -- 172.21.15.26:0/1672797616 shutdown_connections 2023-12-19T16:54:33.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.399+0000 7f2e68d98640 1 --2- 172.21.15.26:0/1672797616 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e64159c00 0x7f2e64070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:33.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.399+0000 7f2e68d98640 1 --2- 172.21.15.26:0/1672797616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6406f6c0 0x7f2e6415df30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:33.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.399+0000 7f2e68d98640 1 -- 172.21.15.26:0/1672797616 >> 172.21.15.26:0/1672797616 conn(0x7f2e640ce730 msgr2=0x7f2e640ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:33.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.399+0000 7f2e68d98640 1 -- 172.21.15.26:0/1672797616 shutdown_connections 2023-12-19T16:54:33.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.399+0000 7f2e68d98640 1 -- 172.21.15.26:0/1672797616 wait complete. 2023-12-19T16:54:33.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e68d98640 1 Processor -- start 2023-12-19T16:54:33.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e68d98640 1 -- start start 2023-12-19T16:54:33.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e68d98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6406f6c0 0x7f2e641497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:33.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e68d98640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e64159820 0x7f2e6414ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:33.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e68d98640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e64159c00 0x7f2e64145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:33.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e68d98640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64153b70 con 0x7f2e6406f6c0 2023-12-19T16:54:33.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e68d98640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6414c0f0 con 0x7f2e64159c00 2023-12-19T16:54:33.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e68d98640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6414c3d0 con 0x7f2e64159820 2023-12-19T16:54:33.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e6259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6406f6c0 0x7f2e641497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:33.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e6259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6406f6c0 0x7f2e641497d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40616/0 (socket says 172.21.15.26:40616) 2023-12-19T16:54:33.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e6259c640 1 -- 172.21.15.26:0/2252088117 learned_addr learned my addr 172.21.15.26:0/2252088117 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:33.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e62d9d640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e64159c00 0x7f2e64145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:33.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e5bfff640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e64159820 0x7f2e6414ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:33.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e6259c640 1 -- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e64159820 msgr2=0x7f2e6414ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:33.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e6259c640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e64159820 0x7f2e6414ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:33.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e6259c640 1 -- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e64159c00 msgr2=0x7f2e64145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:33.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e6259c640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e64159c00 0x7f2e64145cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:33.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e6259c640 1 -- 172.21.15.26:0/2252088117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c0094d0 con 0x7f2e6406f6c0 2023-12-19T16:54:33.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e6259c640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6406f6c0 0x7f2e641497d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400c450 tx=0x7f2e5400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:33.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e5affd640 1 -- 172.21.15.26:0/2252088117 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54012830 con 0x7f2e6406f6c0 2023-12-19T16:54:33.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e5affd640 1 -- 172.21.15.26:0/2252088117 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e540129b0 con 0x7f2e6406f6c0 2023-12-19T16:54:33.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e5affd640 1 -- 172.21.15.26:0/2252088117 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54012d60 con 0x7f2e6406f6c0 2023-12-19T16:54:33.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e68d98640 1 -- 172.21.15.26:0/2252088117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6406b880 con 0x7f2e6406f6c0 2023-12-19T16:54:33.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e68d98640 1 -- 172.21.15.26:0/2252088117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6414c990 con 0x7f2e6406f6c0 2023-12-19T16:54:33.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.403+0000 7f2e68d98640 1 -- 172.21.15.26:0/2252088117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6414cb80 con 0x7f2e6406f6c0 2023-12-19T16:54:33.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.407+0000 7f2e5affd640 1 -- 172.21.15.26:0/2252088117 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f2e5401c460 con 0x7f2e6406f6c0 2023-12-19T16:54:33.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.407+0000 7f2e5affd640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2e44065c60 0x7f2e44068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:33.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.407+0000 7f2e5affd640 1 -- 172.21.15.26:0/2252088117 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f2e54083300 con 0x7f2e6406f6c0 2023-12-19T16:54:33.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.407+0000 7f2e5bfff640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2e44065c60 0x7f2e44068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:33.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.407+0000 7f2e5bfff640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2e44065c60 0x7f2e44068120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0ae000 tx=0x7f2e4c0a7000 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.411+0000 7f2e5affd640 1 -- 172.21.15.26:0/2252088117 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2e540509e0 con 0x7f2e6406f6c0 2023-12-19T16:54:33.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.587+0000 7f2e68d98640 1 -- 172.21.15.26:0/2252088117 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e6414c260 con 0x7f2e44065c60 2023-12-19T16:54:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.663+0000 7f2e5affd640 1 -- 172.21.15.26:0/2252088117 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f2e6414c260 con 0x7f2e44065c60 2023-12-19T16:54:33.668 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-12-19T16:54:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.667+0000 7f2e68d98640 1 -- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2e44065c60 msgr2=0x7f2e44068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.667+0000 7f2e68d98640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2e44065c60 0x7f2e44068120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0ae000 tx=0x7f2e4c0a7000 comp rx=0 tx=0).stop 2023-12-19T16:54:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.667+0000 7f2e68d98640 1 -- 172.21.15.26:0/2252088117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6406f6c0 msgr2=0x7f2e641497d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.667+0000 7f2e68d98640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6406f6c0 0x7f2e641497d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400c450 tx=0x7f2e5400c910 comp rx=0 tx=0).stop 2023-12-19T16:54:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.667+0000 7f2e68d98640 1 -- 172.21.15.26:0/2252088117 shutdown_connections 2023-12-19T16:54:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.667+0000 7f2e68d98640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e64159c00 0x7f2e64145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.667+0000 7f2e68d98640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2e44065c60 0x7f2e44068120 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.667+0000 7f2e68d98640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e64159820 0x7f2e6414ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.667+0000 7f2e68d98640 1 --2- 172.21.15.26:0/2252088117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6406f6c0 0x7f2e641497d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.667+0000 7f2e68d98640 1 -- 172.21.15.26:0/2252088117 >> 172.21.15.26:0/2252088117 conn(0x7f2e640ce730 msgr2=0x7f2e640ae6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.667+0000 7f2e68d98640 1 -- 172.21.15.26:0/2252088117 shutdown_connections 2023-12-19T16:54:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:33.667+0000 7f2e68d98640 1 -- 172.21.15.26:0/2252088117 wait complete. 2023-12-19T16:54:33.690 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-12-19T16:54:33.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:33.690 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-12-19T16:54:33.700 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:33.700 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:33.700 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:33.701 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:33.701 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:33.701 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:33.701 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:33.701 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:33.701 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:33.701 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:33.701 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:33.702 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:33.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:33.702 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_44_00_UTC) 2023-12-19T16:54:33.761 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:33.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:33.762 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_45_00_UTC) 2023-12-19T16:54:33.820 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:33.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:33.820 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_46_00_UTC) 2023-12-19T16:54:33.882 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:33.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:33.882 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_47_00_UTC) 2023-12-19T16:54:33.941 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:33.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:33.941 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_48_00_UTC) 2023-12-19T16:54:34.000 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:34.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:34.000 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_49_00_UTC) 2023-12-19T16:54:34.060 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:34.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:34.061 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_50_00_UTC) 2023-12-19T16:54:34.124 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:34.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:34.124 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_51_00_UTC) 2023-12-19T16:54:34.184 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:34.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:34.184 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_52_00_UTC) 2023-12-19T16:54:34.244 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:34.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:34.244 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_53_00_UTC) 2023-12-19T16:54:34.304 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:34.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:34.304 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-19-16_54_00_UTC) 2023-12-19T16:54:34.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:34.364 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-12-19T16:54:34.420 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir5 1M 2023-12-19T16:54:34.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.607+0000 7f15b0d17640 1 Processor -- start 2023-12-19T16:54:34.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.607+0000 7f15b0d17640 1 -- start start 2023-12-19T16:54:34.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.611+0000 7f15b0d17640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f15ac161230 0x7f15ac0ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:34.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.611+0000 7f15b0d17640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15ac0c8df0 0x7f15ac0c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:34.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.611+0000 7f15b0d17640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ac0b01b0 con 0x7f15ac06c3f0 2023-12-19T16:54:34.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.611+0000 7f15b0d17640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ac0adfb0 con 0x7f15ac0c8df0 2023-12-19T16:54:34.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.611+0000 7f15b0d17640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ac0ca4e0 con 0x7f15ac161230 2023-12-19T16:54:34.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.611+0000 7f15aad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15ac0c8df0 0x7f15ac0c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:34.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.611+0000 7f15aad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15ac0c8df0 0x7f15ac0c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40622/0 (socket says 172.21.15.26:40622) 2023-12-19T16:54:34.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.611+0000 7f15aad9d640 1 -- 172.21.15.26:0/136627318 learned_addr learned my addr 172.21.15.26:0/136627318 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:34.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.611+0000 7f15aa59c640 1 --2- 172.21.15.26:0/136627318 >> v2:172.21.15.160:3300/0 conn(0x7f15ac161230 0x7f15ac0ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:34.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.611+0000 7f15aad9d640 1 -- 172.21.15.26:0/136627318 >> v2:172.21.15.160:3300/0 conn(0x7f15ac161230 msgr2=0x7f15ac0ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:34.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.611+0000 7f15aad9d640 1 --2- 172.21.15.26:0/136627318 >> v2:172.21.15.160:3300/0 conn(0x7f15ac161230 0x7f15ac0ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:34.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.611+0000 7f15aad9d640 1 -- 172.21.15.26:0/136627318 >> v1:172.21.15.160:6789/0 conn(0x7f15ac06c3f0 legacy=0x7f15ac06c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:34.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.611+0000 7f15aad9d640 1 -- 172.21.15.26:0/136627318 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac0b0400 con 0x7f15ac0c8df0 2023-12-19T16:54:34.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.615+0000 7f15aad9d640 1 --2- 172.21.15.26:0/136627318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15ac0c8df0 0x7f15ac0c9280 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0ca0c0 tx=0x7f15a401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4253c79a0b5ccb26 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:34.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.619+0000 7f15a959a640 1 -- 172.21.15.26:0/136627318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a40a5790 con 0x7f15ac0c8df0 2023-12-19T16:54:34.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.619+0000 7f15a959a640 1 -- 172.21.15.26:0/136627318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a40a58f0 con 0x7f15ac0c8df0 2023-12-19T16:54:34.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.619+0000 7f15a959a640 1 -- 172.21.15.26:0/136627318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a40a5c80 con 0x7f15ac0c8df0 2023-12-19T16:54:34.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.619+0000 7f15b0d17640 1 -- 172.21.15.26:0/136627318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15ac0c8df0 msgr2=0x7f15ac0c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:34.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.619+0000 7f15b0d17640 1 --2- 172.21.15.26:0/136627318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15ac0c8df0 0x7f15ac0c9280 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0ca0c0 tx=0x7f15a401c8d0 comp rx=0 tx=0).stop 2023-12-19T16:54:34.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.619+0000 7f15b0d17640 1 -- 172.21.15.26:0/136627318 shutdown_connections 2023-12-19T16:54:34.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.619+0000 7f15b0d17640 1 --2- 172.21.15.26:0/136627318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15ac0c8df0 0x7f15ac0c9280 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:34.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.619+0000 7f15b0d17640 1 --2- 172.21.15.26:0/136627318 >> v2:172.21.15.160:3300/0 conn(0x7f15ac161230 0x7f15ac0ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:34.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.619+0000 7f15b0d17640 1 -- 172.21.15.26:0/136627318 >> 172.21.15.26:0/136627318 conn(0x7f15ac0ce730 msgr2=0x7f15ac0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.619+0000 7f15b0d17640 1 -- 172.21.15.26:0/136627318 shutdown_connections 2023-12-19T16:54:34.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.619+0000 7f15b0d17640 1 -- 172.21.15.26:0/136627318 wait complete. 2023-12-19T16:54:34.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.623+0000 7f15b0d17640 1 Processor -- start 2023-12-19T16:54:34.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.623+0000 7f15b0d17640 1 -- start start 2023-12-19T16:54:34.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.623+0000 7f15b0d17640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15ac06c3f0 0x7f15ac1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:34.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.623+0000 7f15b0d17640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15ac0c8df0 0x7f15ac163400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:34.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.623+0000 7f15b0d17640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15ac161230 0x7f15ac0c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:34.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.623+0000 7f15b0d17640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac1646d0 con 0x7f15ac0c8df0 2023-12-19T16:54:34.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.623+0000 7f15b0d17640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac163ac0 con 0x7f15ac06c3f0 2023-12-19T16:54:34.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.623+0000 7f15b0d17640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac163da0 con 0x7f15ac161230 2023-12-19T16:54:34.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.623+0000 7f15a9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15ac0c8df0 0x7f15ac163400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:34.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.623+0000 7f15aad9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15ac161230 0x7f15ac0c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:34.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.623+0000 7f15aad9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15ac161230 0x7f15ac0c4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:48560/0 (socket says 172.21.15.26:48560) 2023-12-19T16:54:34.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.623+0000 7f15aad9d640 1 -- 172.21.15.26:0/127901460 learned_addr learned my addr 172.21.15.26:0/127901460 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:34.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.627+0000 7f15a9d9b640 1 -- 172.21.15.26:0/127901460 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15ac161230 msgr2=0x7f15ac0c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:34.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.627+0000 7f15a9d9b640 1 --2- 172.21.15.26:0/127901460 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15ac161230 0x7f15ac0c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:34.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.627+0000 7f15a9d9b640 1 -- 172.21.15.26:0/127901460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15ac06c3f0 msgr2=0x7f15ac1517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:34.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.627+0000 7f15a9d9b640 1 --2- 172.21.15.26:0/127901460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15ac06c3f0 0x7f15ac1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:34.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.627+0000 7f15a9d9b640 1 -- 172.21.15.26:0/127901460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a4009b30 con 0x7f15ac0c8df0 2023-12-19T16:54:34.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.627+0000 7f15a9d9b640 1 --2- 172.21.15.26:0/127901460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15ac0c8df0 0x7f15ac163400 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f15a000b410 tx=0x7f15a000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:34.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.627+0000 7f159affd640 1 -- 172.21.15.26:0/127901460 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0004020 con 0x7f15ac0c8df0 2023-12-19T16:54:34.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.627+0000 7f159affd640 1 -- 172.21.15.26:0/127901460 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a0004180 con 0x7f15ac0c8df0 2023-12-19T16:54:34.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.627+0000 7f159affd640 1 -- 172.21.15.26:0/127901460 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0011630 con 0x7f15ac0c8df0 2023-12-19T16:54:34.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.627+0000 7f15b0d17640 1 -- 172.21.15.26:0/127901460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ac164080 con 0x7f15ac0c8df0 2023-12-19T16:54:34.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.627+0000 7f15b0d17640 1 -- 172.21.15.26:0/127901460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ac0c4ed0 con 0x7f15ac0c8df0 2023-12-19T16:54:34.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.627+0000 7f15b0d17640 1 -- 172.21.15.26:0/127901460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ac0c16d0 con 0x7f15ac0c8df0 2023-12-19T16:54:34.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.631+0000 7f159affd640 1 -- 172.21.15.26:0/127901460 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f15a0011790 con 0x7f15ac0c8df0 2023-12-19T16:54:34.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.631+0000 7f159affd640 1 --2- 172.21.15.26:0/127901460 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f158c065cb0 0x7f158c068170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:34.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.631+0000 7f159affd640 1 -- 172.21.15.26:0/127901460 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 7431+0+0 (secure 0 0 0) 0x7f15a00836f0 con 0x7f15ac0c8df0 2023-12-19T16:54:34.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.631+0000 7f15aa59c640 1 --2- 172.21.15.26:0/127901460 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f158c065cb0 0x7f158c068170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:34.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.635+0000 7f15aa59c640 1 --2- 172.21.15.26:0/127901460 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f158c065cb0 0x7f158c068170 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f159c0098f0 tx=0x7f159c008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:34.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:34.639+0000 7f159affd640 1 -- 172.21.15.26:0/127901460 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f15a00508e0 con 0x7f15ac0c8df0 2023-12-19T16:54:35.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.043+0000 7f15b0d17640 1 -- 172.21.15.26:0/127901460 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f15ac0ca160 con 0x7f158c065cb0 2023-12-19T16:54:35.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.287+0000 7f159affd640 1 -- 172.21.15.26:0/127901460 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f159c009e70 con 0x7f158c065cb0 2023-12-19T16:54:35.293 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-12-19T16:54:35.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.291+0000 7f15b0d17640 1 -- 172.21.15.26:0/127901460 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f158c065cb0 msgr2=0x7f158c068170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:35.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.291+0000 7f15b0d17640 1 --2- 172.21.15.26:0/127901460 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f158c065cb0 0x7f158c068170 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f159c0098f0 tx=0x7f159c008040 comp rx=0 tx=0).stop 2023-12-19T16:54:35.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.291+0000 7f15b0d17640 1 -- 172.21.15.26:0/127901460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15ac0c8df0 msgr2=0x7f15ac163400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:35.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.291+0000 7f15b0d17640 1 --2- 172.21.15.26:0/127901460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15ac0c8df0 0x7f15ac163400 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f15a000b410 tx=0x7f15a000b8d0 comp rx=0 tx=0).stop 2023-12-19T16:54:35.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.291+0000 7f15b0d17640 1 -- 172.21.15.26:0/127901460 shutdown_connections 2023-12-19T16:54:35.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.291+0000 7f15b0d17640 1 --2- 172.21.15.26:0/127901460 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f158c065cb0 0x7f158c068170 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:35.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.291+0000 7f15b0d17640 1 --2- 172.21.15.26:0/127901460 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15ac161230 0x7f15ac0c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:35.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.291+0000 7f15b0d17640 1 --2- 172.21.15.26:0/127901460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15ac0c8df0 0x7f15ac163400 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:35.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.291+0000 7f15b0d17640 1 --2- 172.21.15.26:0/127901460 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15ac06c3f0 0x7f15ac1517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:35.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.291+0000 7f15b0d17640 1 -- 172.21.15.26:0/127901460 >> 172.21.15.26:0/127901460 conn(0x7f15ac0ce730 msgr2=0x7f15ac06c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:35.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.291+0000 7f15b0d17640 1 -- 172.21.15.26:0/127901460 shutdown_connections 2023-12-19T16:54:35.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:35.291+0000 7f15b0d17640 1 -- 172.21.15.26:0/127901460 wait complete. 2023-12-19T16:54:35.314 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-12-19T16:54:35.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:35.314 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-12-19T16:54:35.325 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:35.325 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:35.325 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:35.325 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:35.325 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:35.325 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:35.325 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:35.325 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:35.325 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:35.325 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:35.325 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:35.326 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:35.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:35.326 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_44_00_UTC) 2023-12-19T16:54:35.390 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:35.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:35.390 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_45_00_UTC) 2023-12-19T16:54:35.448 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:35.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:35.449 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_46_00_UTC) 2023-12-19T16:54:35.508 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:35.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:35.509 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_47_00_UTC) 2023-12-19T16:54:35.572 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:35.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:35.572 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_48_00_UTC) 2023-12-19T16:54:35.653 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:35.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:35.653 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_49_00_UTC) 2023-12-19T16:54:35.712 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:35.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:35.712 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_50_00_UTC) 2023-12-19T16:54:35.796 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:35.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:35.796 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_51_00_UTC) 2023-12-19T16:54:35.856 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:35.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:35.856 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_52_00_UTC) 2023-12-19T16:54:35.915 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:35.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:35.915 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_53_00_UTC) 2023-12-19T16:54:35.977 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:35.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:35.977 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-19-16_54_00_UTC) 2023-12-19T16:54:36.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:36.035 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-12-19T16:54:36.088 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir6 1M 2023-12-19T16:54:36.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047f59e640 1 Processor -- start 2023-12-19T16:54:36.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047f59e640 1 -- start start 2023-12-19T16:54:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047f59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f048015cc90 0x7f04800cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047f59e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0480160a20 0x7f04800d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047f59e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04800cdb20 con 0x7f048015d070 2023-12-19T16:54:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047f59e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04800cde00 con 0x7f048015cc90 2023-12-19T16:54:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047f59e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04800ce0e0 con 0x7f0480160a20 2023-12-19T16:54:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f048015cc90 0x7f04800cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f048015cc90 0x7f04800cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51418/0 (socket says 172.21.15.26:51418) 2023-12-19T16:54:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047e59c640 1 -- 172.21.15.26:0/903242011 learned_addr learned my addr 172.21.15.26:0/903242011 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:36.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047ed9d640 1 --2- 172.21.15.26:0/903242011 >> v2:172.21.15.160:3300/0 conn(0x7f0480160a20 0x7f04800d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:36.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047e59c640 1 -- 172.21.15.26:0/903242011 >> v2:172.21.15.160:3300/0 conn(0x7f0480160a20 msgr2=0x7f04800d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:36.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047e59c640 1 --2- 172.21.15.26:0/903242011 >> v2:172.21.15.160:3300/0 conn(0x7f0480160a20 0x7f04800d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:36.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047e59c640 1 -- 172.21.15.26:0/903242011 >> v1:172.21.15.160:6789/0 conn(0x7f048015d070 legacy=0x7f04800cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:36.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.275+0000 7f047e59c640 1 -- 172.21.15.26:0/903242011 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04800ce3c0 con 0x7f048015cc90 2023-12-19T16:54:36.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.279+0000 7f047e59c640 1 --2- 172.21.15.26:0/903242011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048015cc90 0x7f04800cd2f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f04800b2320 tx=0x7f047001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c510e5dc9c8b048 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:36.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.279+0000 7f047d59a640 1 -- 172.21.15.26:0/903242011 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04700ab670 con 0x7f048015cc90 2023-12-19T16:54:36.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.279+0000 7f047d59a640 1 -- 172.21.15.26:0/903242011 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04700ab7d0 con 0x7f048015cc90 2023-12-19T16:54:36.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.279+0000 7f047d59a640 1 -- 172.21.15.26:0/903242011 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04700abb60 con 0x7f048015cc90 2023-12-19T16:54:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.279+0000 7f047f59e640 1 -- 172.21.15.26:0/903242011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048015cc90 msgr2=0x7f04800cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.279+0000 7f047f59e640 1 --2- 172.21.15.26:0/903242011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048015cc90 0x7f04800cd2f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f04800b2320 tx=0x7f047001c490 comp rx=0 tx=0).stop 2023-12-19T16:54:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.279+0000 7f047f59e640 1 -- 172.21.15.26:0/903242011 shutdown_connections 2023-12-19T16:54:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.279+0000 7f047f59e640 1 --2- 172.21.15.26:0/903242011 >> v2:172.21.15.160:3300/0 conn(0x7f0480160a20 0x7f04800d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.279+0000 7f047f59e640 1 --2- 172.21.15.26:0/903242011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048015cc90 0x7f04800cd2f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.279+0000 7f047f59e640 1 -- 172.21.15.26:0/903242011 >> 172.21.15.26:0/903242011 conn(0x7f048006a320 msgr2=0x7f04800658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.279+0000 7f047f59e640 1 -- 172.21.15.26:0/903242011 shutdown_connections 2023-12-19T16:54:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.279+0000 7f047f59e640 1 -- 172.21.15.26:0/903242011 wait complete. 2023-12-19T16:54:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047f59e640 1 Processor -- start 2023-12-19T16:54:36.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047f59e640 1 -- start start 2023-12-19T16:54:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047f59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f048015cc90 0x7f04800c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047f59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f048015d070 0x7f0480158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0480160a20 0x7f048015d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047f59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0480159df0 con 0x7f0480160a20 2023-12-19T16:54:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047f59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048015a210 con 0x7f048015d070 2023-12-19T16:54:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047f59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0480159170 con 0x7f048015cc90 2023-12-19T16:54:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0480160a20 0x7f048015d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0480160a20 0x7f048015d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51434/0 (socket says 172.21.15.26:51434) 2023-12-19T16:54:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047ed9d640 1 -- 172.21.15.26:0/2498103811 learned_addr learned my addr 172.21.15.26:0/2498103811 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047dd9b640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f048015d070 0x7f0480158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047e59c640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f048015cc90 0x7f04800c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047ed9d640 1 -- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f048015cc90 msgr2=0x7f04800c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047ed9d640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f048015cc90 0x7f04800c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047ed9d640 1 -- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f048015d070 msgr2=0x7f0480158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047ed9d640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f048015d070 0x7f0480158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047ed9d640 1 -- 172.21.15.26:0/2498103811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04700094d0 con 0x7f0480160a20 2023-12-19T16:54:36.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047ed9d640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0480160a20 0x7f048015d490 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0478009fd0 tx=0x7f047800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:36.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f046effd640 1 -- 172.21.15.26:0/2498103811 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478014420 con 0x7f0480160a20 2023-12-19T16:54:36.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f046effd640 1 -- 172.21.15.26:0/2498103811 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0478014580 con 0x7f0480160a20 2023-12-19T16:54:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f046effd640 1 -- 172.21.15.26:0/2498103811 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0478014930 con 0x7f0480160a20 2023-12-19T16:54:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047f59e640 1 -- 172.21.15.26:0/2498103811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0480159450 con 0x7f0480160a20 2023-12-19T16:54:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.283+0000 7f047f59e640 1 -- 172.21.15.26:0/2498103811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0480159a10 con 0x7f0480160a20 2023-12-19T16:54:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.287+0000 7f047f59e640 1 -- 172.21.15.26:0/2498103811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0480159f30 con 0x7f0480160a20 2023-12-19T16:54:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.287+0000 7f046effd640 1 -- 172.21.15.26:0/2498103811 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f0478014a90 con 0x7f0480160a20 2023-12-19T16:54:36.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.287+0000 7f046effd640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0460065c90 0x7f0460068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:36.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.287+0000 7f046effd640 1 -- 172.21.15.26:0/2498103811 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 7015+0+0 (secure 0 0 0) 0x7f0478084210 con 0x7f0480160a20 2023-12-19T16:54:36.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.287+0000 7f047e59c640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0460065c90 0x7f0460068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:36.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.287+0000 7f047e59c640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0460065c90 0x7f0460068150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f04700af000 tx=0x7f04700b6040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.291+0000 7f046effd640 1 -- 172.21.15.26:0/2498103811 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0480159f30 con 0x7f0480160a20 2023-12-19T16:54:36.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.467+0000 7f047f59e640 1 -- 172.21.15.26:0/2498103811 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0480159f30 con 0x7f0460065c90 2023-12-19T16:54:36.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.547+0000 7f046effd640 1 -- 172.21.15.26:0/2498103811 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f0480159f30 con 0x7f0460065c90 2023-12-19T16:54:36.550 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-12-19T16:54:36.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.547+0000 7f047f59e640 1 -- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0460065c90 msgr2=0x7f0460068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:36.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.547+0000 7f047f59e640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0460065c90 0x7f0460068150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f04700af000 tx=0x7f04700b6040 comp rx=0 tx=0).stop 2023-12-19T16:54:36.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.547+0000 7f047f59e640 1 -- 172.21.15.26:0/2498103811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0480160a20 msgr2=0x7f048015d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:36.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.547+0000 7f047f59e640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0480160a20 0x7f048015d490 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0478009fd0 tx=0x7f047800e4b0 comp rx=0 tx=0).stop 2023-12-19T16:54:36.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.547+0000 7f047f59e640 1 -- 172.21.15.26:0/2498103811 shutdown_connections 2023-12-19T16:54:36.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.547+0000 7f047f59e640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0480160a20 0x7f048015d490 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:36.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.547+0000 7f047f59e640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f048015d070 0x7f0480158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:36.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.547+0000 7f047f59e640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0460065c90 0x7f0460068150 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:36.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.547+0000 7f047f59e640 1 --2- 172.21.15.26:0/2498103811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f048015cc90 0x7f04800c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:36.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.547+0000 7f047f59e640 1 -- 172.21.15.26:0/2498103811 >> 172.21.15.26:0/2498103811 conn(0x7f048006a320 msgr2=0x7f048015fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:36.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.547+0000 7f047f59e640 1 -- 172.21.15.26:0/2498103811 shutdown_connections 2023-12-19T16:54:36.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:36.547+0000 7f047f59e640 1 -- 172.21.15.26:0/2498103811 wait complete. 2023-12-19T16:54:36.571 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-12-19T16:54:36.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:36.571 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-12-19T16:54:36.581 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:36.582 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:36.582 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:36.582 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:36.582 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:36.582 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:36.582 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:36.582 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:36.582 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_01_UTC 2023-12-19T16:54:36.582 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:36.582 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:36.583 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:36.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:36.583 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_44_00_UTC) 2023-12-19T16:54:36.672 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:36.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:36.672 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_45_00_UTC) 2023-12-19T16:54:36.736 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:36.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:36.736 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_46_00_UTC) 2023-12-19T16:54:36.793 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:36.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:36.793 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_47_00_UTC) 2023-12-19T16:54:36.852 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:36.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:36.852 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_48_00_UTC) 2023-12-19T16:54:36.911 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:36.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:36.912 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_49_00_UTC) 2023-12-19T16:54:36.967 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:36.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:36.967 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_50_00_UTC) 2023-12-19T16:54:37.025 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:37.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:37.025 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_51_00_UTC) 2023-12-19T16:54:37.085 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_52_01_UTC 2023-12-19T16:54:37.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:37.085 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_52_01_UTC) 2023-12-19T16:54:37.148 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:37.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:37.148 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_53_00_UTC) 2023-12-19T16:54:37.207 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:37.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:37.208 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-19-16_54_00_UTC) 2023-12-19T16:54:37.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:37.265 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-12-19T16:54:37.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir7 1M 2023-12-19T16:54:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.515+0000 7fd20e7df640 1 Processor -- start 2023-12-19T16:54:37.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.515+0000 7fd20e7df640 1 -- start start 2023-12-19T16:54:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.515+0000 7fd20e7df640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd2080cfcb0 0x7fd208160a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.515+0000 7fd20e7df640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd208160f80 0x7fd2080d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.515+0000 7fd20e7df640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2080cd490 con 0x7fd20815a770 2023-12-19T16:54:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.515+0000 7fd20e7df640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2080cd770 con 0x7fd208160f80 2023-12-19T16:54:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.515+0000 7fd20e7df640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2080cda50 con 0x7fd2080cfcb0 2023-12-19T16:54:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.519+0000 7fd20dfde640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd208160f80 0x7fd2080d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.519+0000 7fd20dfde640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd208160f80 0x7fd2080d0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51438/0 (socket says 172.21.15.26:51438) 2023-12-19T16:54:37.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.519+0000 7fd20dfde640 1 -- 172.21.15.26:0/1093805156 learned_addr learned my addr 172.21.15.26:0/1093805156 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.519+0000 7fd20cfdc640 1 --2- 172.21.15.26:0/1093805156 >> v2:172.21.15.160:3300/0 conn(0x7fd2080cfcb0 0x7fd208160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.519+0000 7fd20dfde640 1 -- 172.21.15.26:0/1093805156 >> v2:172.21.15.160:3300/0 conn(0x7fd2080cfcb0 msgr2=0x7fd208160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.519+0000 7fd20dfde640 1 --2- 172.21.15.26:0/1093805156 >> v2:172.21.15.160:3300/0 conn(0x7fd2080cfcb0 0x7fd208160a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.519+0000 7fd20dfde640 1 -- 172.21.15.26:0/1093805156 >> v1:172.21.15.160:6789/0 conn(0x7fd20815a770 legacy=0x7fd2080cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.519+0000 7fd20dfde640 1 -- 172.21.15.26:0/1093805156 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2080cdd30 con 0x7fd208160f80 2023-12-19T16:54:37.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.519+0000 7fd20cfdc640 1 --2- 172.21.15.26:0/1093805156 >> v2:172.21.15.160:3300/0 conn(0x7fd2080cfcb0 0x7fd208160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:37.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.519+0000 7fd20dfde640 1 --2- 172.21.15.26:0/1093805156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd208160f80 0x7fd2080d0da0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd2080cd0d0 tx=0x7fd20001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5c92852fbd6f62 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:37.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.519+0000 7fd1fffff640 1 -- 172.21.15.26:0/1093805156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2000a5830 con 0x7fd208160f80 2023-12-19T16:54:37.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.519+0000 7fd1fffff640 1 -- 172.21.15.26:0/1093805156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2000a5990 con 0x7fd208160f80 2023-12-19T16:54:37.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.519+0000 7fd1fffff640 1 -- 172.21.15.26:0/1093805156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2000a5d20 con 0x7fd208160f80 2023-12-19T16:54:37.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 -- 172.21.15.26:0/1093805156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd208160f80 msgr2=0x7fd2080d0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:37.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 --2- 172.21.15.26:0/1093805156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd208160f80 0x7fd2080d0da0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd2080cd0d0 tx=0x7fd20001c9d0 comp rx=0 tx=0).stop 2023-12-19T16:54:37.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 -- 172.21.15.26:0/1093805156 shutdown_connections 2023-12-19T16:54:37.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 --2- 172.21.15.26:0/1093805156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd208160f80 0x7fd2080d0da0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:37.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 --2- 172.21.15.26:0/1093805156 >> v2:172.21.15.160:3300/0 conn(0x7fd2080cfcb0 0x7fd208160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:37.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 -- 172.21.15.26:0/1093805156 >> 172.21.15.26:0/1093805156 conn(0x7fd20806a150 msgr2=0x7fd20806a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:37.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 -- 172.21.15.26:0/1093805156 shutdown_connections 2023-12-19T16:54:37.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 -- 172.21.15.26:0/1093805156 wait complete. 2023-12-19T16:54:37.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 Processor -- start 2023-12-19T16:54:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 -- start start 2023-12-19T16:54:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2080cfcb0 0x7fd20814cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd20815a770 0x7fd2080c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd208160f80 0x7fd2081561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20813af70 con 0x7fd2080cfcb0 2023-12-19T16:54:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20813a660 con 0x7fd208160f80 2023-12-19T16:54:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20e7df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2080c90a0 con 0x7fd20815a770 2023-12-19T16:54:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20d7dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2080cfcb0 0x7fd20814cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20d7dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2080cfcb0 0x7fd20814cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51452/0 (socket says 172.21.15.26:51452) 2023-12-19T16:54:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20d7dd640 1 -- 172.21.15.26:0/1588051425 learned_addr learned my addr 172.21.15.26:0/1588051425 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:37.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20dfde640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd208160f80 0x7fd2081561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:37.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20cfdc640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd20815a770 0x7fd2080c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20d7dd640 1 -- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd20815a770 msgr2=0x7fd2080c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20d7dd640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd20815a770 0x7fd2080c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20d7dd640 1 -- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd208160f80 msgr2=0x7fd2081561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20d7dd640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd208160f80 0x7fd2081561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20d7dd640 1 -- 172.21.15.26:0/1588051425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd200009b30 con 0x7fd2080cfcb0 2023-12-19T16:54:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.523+0000 7fd20dfde640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd208160f80 0x7fd2081561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.527+0000 7fd20d7dd640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2080cfcb0 0x7fd20814cff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd20400c500 tx=0x7fd20400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.527+0000 7fd1fdffb640 1 -- 172.21.15.26:0/1588051425 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204005220 con 0x7fd2080cfcb0 2023-12-19T16:54:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.527+0000 7fd1fdffb640 1 -- 172.21.15.26:0/1588051425 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd204005380 con 0x7fd2080cfcb0 2023-12-19T16:54:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.527+0000 7fd1fdffb640 1 -- 172.21.15.26:0/1588051425 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204007270 con 0x7fd2080cfcb0 2023-12-19T16:54:37.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.527+0000 7fd20e7df640 1 -- 172.21.15.26:0/1588051425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20813a920 con 0x7fd2080cfcb0 2023-12-19T16:54:37.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.527+0000 7fd20e7df640 1 -- 172.21.15.26:0/1588051425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20814d5d0 con 0x7fd2080cfcb0 2023-12-19T16:54:37.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.527+0000 7fd20e7df640 1 -- 172.21.15.26:0/1588051425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd208061930 con 0x7fd2080cfcb0 2023-12-19T16:54:37.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.527+0000 7fd1fdffb640 1 -- 172.21.15.26:0/1588051425 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fd2040073d0 con 0x7fd2080cfcb0 2023-12-19T16:54:37.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.527+0000 7fd1fdffb640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd1ec065c60 0x7fd1ec068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:37.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.527+0000 7fd1fdffb640 1 -- 172.21.15.26:0/1588051425 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6903+0+0 (secure 0 0 0) 0x7fd2040833d0 con 0x7fd2080cfcb0 2023-12-19T16:54:37.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.527+0000 7fd20cfdc640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd1ec065c60 0x7fd1ec068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:37.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.531+0000 7fd20cfdc640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd1ec065c60 0x7fd1ec068120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd20815a0e0 tx=0x7fd1f8008040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.531+0000 7fd1fdffb640 1 -- 172.21.15.26:0/1588051425 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd2040505d0 con 0x7fd2080cfcb0 2023-12-19T16:54:37.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.739+0000 7fd20e7df640 1 -- 172.21.15.26:0/1588051425 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd20815ab50 con 0x7fd1ec065c60 2023-12-19T16:54:37.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.855+0000 7fd1fdffb640 1 -- 172.21.15.26:0/1588051425 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fd1f800c590 con 0x7fd1ec065c60 2023-12-19T16:54:37.860 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-12-19T16:54:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.859+0000 7fd20e7df640 1 -- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd1ec065c60 msgr2=0x7fd1ec068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.859+0000 7fd20e7df640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd1ec065c60 0x7fd1ec068120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd20815a0e0 tx=0x7fd1f8008040 comp rx=0 tx=0).stop 2023-12-19T16:54:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.859+0000 7fd20e7df640 1 -- 172.21.15.26:0/1588051425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2080cfcb0 msgr2=0x7fd20814cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.859+0000 7fd20e7df640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2080cfcb0 0x7fd20814cff0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd20400c500 tx=0x7fd20400c9c0 comp rx=0 tx=0).stop 2023-12-19T16:54:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.859+0000 7fd20e7df640 1 -- 172.21.15.26:0/1588051425 shutdown_connections 2023-12-19T16:54:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.859+0000 7fd20e7df640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd208160f80 0x7fd2081561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.859+0000 7fd20e7df640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd1ec065c60 0x7fd1ec068120 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.859+0000 7fd20e7df640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd20815a770 0x7fd2080c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.859+0000 7fd20e7df640 1 --2- 172.21.15.26:0/1588051425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2080cfcb0 0x7fd20814cff0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.859+0000 7fd20e7df640 1 -- 172.21.15.26:0/1588051425 >> 172.21.15.26:0/1588051425 conn(0x7fd20806a150 msgr2=0x7fd20806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.859+0000 7fd20e7df640 1 -- 172.21.15.26:0/1588051425 shutdown_connections 2023-12-19T16:54:37.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:37.859+0000 7fd20e7df640 1 -- 172.21.15.26:0/1588051425 wait complete. 2023-12-19T16:54:37.881 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-12-19T16:54:37.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:37.881 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-12-19T16:54:37.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:37.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:37.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:37.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:37.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:37.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:37.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:37.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:37.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:37.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:37.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:37.893 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:37.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:37.893 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_44_00_UTC) 2023-12-19T16:54:37.957 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:37.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:37.957 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_45_00_UTC) 2023-12-19T16:54:38.015 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:38.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:38.015 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_46_00_UTC) 2023-12-19T16:54:38.071 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:38.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:38.072 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_47_00_UTC) 2023-12-19T16:54:38.127 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:38.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:38.128 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_48_00_UTC) 2023-12-19T16:54:38.183 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:38.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:38.184 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_49_00_UTC) 2023-12-19T16:54:38.239 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:38.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:38.239 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_50_00_UTC) 2023-12-19T16:54:38.299 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:38.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:38.299 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_51_00_UTC) 2023-12-19T16:54:38.360 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:38.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:38.360 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_52_00_UTC) 2023-12-19T16:54:38.415 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:38.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:38.415 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_53_00_UTC) 2023-12-19T16:54:38.474 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:38.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:38.475 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-19-16_54_00_UTC) 2023-12-19T16:54:38.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:38.531 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-12-19T16:54:38.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir8 1M 2023-12-19T16:54:38.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac352d7640 1 Processor -- start 2023-12-19T16:54:38.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac352d7640 1 -- start start 2023-12-19T16:54:38.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac352d7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac3015ab40 0x7fac3006b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:38.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac352d7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac3015af20 con 0x7fac3006f0a0 2023-12-19T16:54:38.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac352d7640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac3015f420 con 0x7fac30071890 2023-12-19T16:54:38.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac352d7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac300aea90 con 0x7fac3015ab40 2023-12-19T16:54:38.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fac3006f0a0 0x7fac3015e600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60180/0 (socket says 172.21.15.26:60180) 2023-12-19T16:54:38.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2f7fe640 1 -- 172.21.15.26:0/2531030834 learned_addr learned my addr 172.21.15.26:0/2531030834 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:38.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2ffff640 1 --2- 172.21.15.26:0/2531030834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac3015ab40 0x7fac3006b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:38.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714007100 0 0) 0x7fac3015af20 con 0x7fac3006f0a0 2023-12-19T16:54:38.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac18003530 con 0x7fac3006f0a0 2023-12-19T16:54:38.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150380465 0 0) 0x7fac3015f420 con 0x7fac30071890 2023-12-19T16:54:38.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac3015af20 con 0x7fac30071890 2023-12-19T16:54:38.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3878592160 0 0) 0x7fac18003530 con 0x7fac3006f0a0 2023-12-19T16:54:38.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac3015ab40 msgr2=0x7fac3006b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:38.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 --2- 172.21.15.26:0/2531030834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac3015ab40 0x7fac3006b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:38.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 >> v1:172.21.15.160:6789/0 conn(0x7fac30071890 legacy=0x7fac3015ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:38.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac300aed70 con 0x7fac3006f0a0 2023-12-19T16:54:38.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fac24001680 con 0x7fac3006f0a0 2023-12-19T16:54:38.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 >> v1:172.21.15.26:6789/0 conn(0x7fac3006f0a0 legacy=0x7fac3015e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:38.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 --2- 172.21.15.26:0/2531030834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac18003b40 0x7fac18003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:38.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 --2- 172.21.15.26:0/2531030834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac3015ab40 0x7fac180047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:38.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 --2- 172.21.15.26:0/2531030834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac18004d10 0x7fac18005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:38.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac300aea90 con 0x7fac18004d10 2023-12-19T16:54:38.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1800ba40 con 0x7fac18003b40 2023-12-19T16:54:38.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1800bbb0 con 0x7fac3015ab40 2023-12-19T16:54:38.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac34ad6640 1 --2- 172.21.15.26:0/2531030834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac18004d10 0x7fac18005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:38.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2ffff640 1 --2- 172.21.15.26:0/2531030834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac18003b40 0x7fac18003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:38.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.791+0000 7fac2f7fe640 1 --2- 172.21.15.26:0/2531030834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac3015ab40 0x7fac180047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:38.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac34ad6640 1 -- 172.21.15.26:0/2531030834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac3015ab40 msgr2=0x7fac180047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:38.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac34ad6640 1 --2- 172.21.15.26:0/2531030834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac3015ab40 0x7fac180047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:38.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac34ad6640 1 -- 172.21.15.26:0/2531030834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac18003b40 msgr2=0x7fac18003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:38.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac34ad6640 1 --2- 172.21.15.26:0/2531030834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac18003b40 0x7fac18003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:38.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac34ad6640 1 -- 172.21.15.26:0/2531030834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac18003530 con 0x7fac18004d10 2023-12-19T16:54:38.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac34ad6640 1 --2- 172.21.15.26:0/2531030834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac18004d10 0x7fac18005190 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fac28002040 tx=0x7fac2801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8feb2fa6f3609a71 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:38.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac24007ea0 con 0x7fac18004d10 2023-12-19T16:54:38.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac28003bf0 con 0x7fac18004d10 2023-12-19T16:54:38.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac2effd640 1 -- 172.21.15.26:0/2531030834 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac280050d0 con 0x7fac18004d10 2023-12-19T16:54:38.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac352d7640 1 -- 172.21.15.26:0/2531030834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac18004d10 msgr2=0x7fac18005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:38.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac352d7640 1 --2- 172.21.15.26:0/2531030834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac18004d10 0x7fac18005190 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fac28002040 tx=0x7fac2801c8e0 comp rx=0 tx=0).stop 2023-12-19T16:54:38.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac34ad6640 1 -- 172.21.15.26:0/2531030834 reap_dead start 2023-12-19T16:54:38.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac352d7640 1 -- 172.21.15.26:0/2531030834 shutdown_connections 2023-12-19T16:54:38.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac352d7640 1 -- 172.21.15.26:0/2531030834 >> 172.21.15.26:0/2531030834 conn(0x7fac300658e0 msgr2=0x7fac30065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:38.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac352d7640 1 -- 172.21.15.26:0/2531030834 shutdown_connections 2023-12-19T16:54:38.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.795+0000 7fac352d7640 1 -- 172.21.15.26:0/2531030834 wait complete. 2023-12-19T16:54:38.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac352d7640 1 Processor -- start 2023-12-19T16:54:38.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac352d7640 1 -- start start 2023-12-19T16:54:38.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac352d7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac18004d10 0x7fac300a8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:38.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac352d7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac3014f520 0x7fac300abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:38.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac352d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac3015cdc0 0x7fac3015d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:38.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac352d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3015d780 con 0x7fac3015cdc0 2023-12-19T16:54:38.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac352d7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac300a8980 con 0x7fac18004d10 2023-12-19T16:54:38.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac352d7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac300acc00 con 0x7fac3014f520 2023-12-19T16:54:38.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac34ad6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac3015cdc0 0x7fac3015d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:38.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac34ad6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac3015cdc0 0x7fac3015d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51470/0 (socket says 172.21.15.26:51470) 2023-12-19T16:54:38.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac34ad6640 1 -- 172.21.15.26:0/2693117247 learned_addr learned my addr 172.21.15.26:0/2693117247 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:38.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac2f7fe640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac3014f520 0x7fac300abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:38.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac2ffff640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac18004d10 0x7fac300a8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:38.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac34ad6640 1 -- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac3014f520 msgr2=0x7fac300abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:38.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac34ad6640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac3014f520 0x7fac300abc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:38.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac34ad6640 1 -- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac18004d10 msgr2=0x7fac300a8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:38.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac34ad6640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac18004d10 0x7fac300a8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:38.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac34ad6640 1 -- 172.21.15.26:0/2693117247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac300acee0 con 0x7fac3015cdc0 2023-12-19T16:54:38.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac34ad6640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac3015cdc0 0x7fac3015d240 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fac28005040 tx=0x7fac18005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:38.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac2cff9640 1 -- 172.21.15.26:0/2693117247 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac28006bb0 con 0x7fac3015cdc0 2023-12-19T16:54:38.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac2cff9640 1 -- 172.21.15.26:0/2693117247 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac280b4040 con 0x7fac3015cdc0 2023-12-19T16:54:38.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.799+0000 7fac2cff9640 1 -- 172.21.15.26:0/2693117247 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac280050d0 con 0x7fac3015cdc0 2023-12-19T16:54:38.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.803+0000 7fac352d7640 1 -- 172.21.15.26:0/2693117247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac300ab170 con 0x7fac3015cdc0 2023-12-19T16:54:38.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.803+0000 7fac352d7640 1 -- 172.21.15.26:0/2693117247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac300ab730 con 0x7fac3015cdc0 2023-12-19T16:54:38.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.803+0000 7fac352d7640 1 -- 172.21.15.26:0/2693117247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac300aafd0 con 0x7fac3015cdc0 2023-12-19T16:54:38.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.803+0000 7fac2cff9640 1 -- 172.21.15.26:0/2693117247 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fac28005440 con 0x7fac3015cdc0 2023-12-19T16:54:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.803+0000 7fac2cff9640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fac1c065c90 0x7fac1c068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.803+0000 7fac2cff9640 1 -- 172.21.15.26:0/2693117247 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6727+0+0 (secure 0 0 0) 0x7fac280aa070 con 0x7fac3015cdc0 2023-12-19T16:54:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.803+0000 7fac2ffff640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fac1c065c90 0x7fac1c068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.807+0000 7fac2ffff640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fac1c065c90 0x7fac1c068150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac20009be0 tx=0x7fac20009190 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:38.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:38.807+0000 7fac2cff9640 1 -- 172.21.15.26:0/2693117247 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fac300aafd0 con 0x7fac3015cdc0 2023-12-19T16:54:39.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.015+0000 7fac352d7640 1 -- 172.21.15.26:0/2693117247 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fac300af330 con 0x7fac1c065c90 2023-12-19T16:54:39.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.131+0000 7fac2cff9640 1 -- 172.21.15.26:0/2693117247 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fac20019070 con 0x7fac1c065c90 2023-12-19T16:54:39.136 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-12-19T16:54:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.135+0000 7fac352d7640 1 -- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fac1c065c90 msgr2=0x7fac1c068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.135+0000 7fac352d7640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fac1c065c90 0x7fac1c068150 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac20009be0 tx=0x7fac20009190 comp rx=0 tx=0).stop 2023-12-19T16:54:39.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.135+0000 7fac352d7640 1 -- 172.21.15.26:0/2693117247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac3015cdc0 msgr2=0x7fac3015d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.135+0000 7fac352d7640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac3015cdc0 0x7fac3015d240 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fac28005040 tx=0x7fac18005800 comp rx=0 tx=0).stop 2023-12-19T16:54:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.135+0000 7fac352d7640 1 -- 172.21.15.26:0/2693117247 shutdown_connections 2023-12-19T16:54:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.135+0000 7fac352d7640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac3015cdc0 0x7fac3015d240 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.135+0000 7fac352d7640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fac1c065c90 0x7fac1c068150 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.135+0000 7fac352d7640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac3014f520 0x7fac300abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.135+0000 7fac352d7640 1 --2- 172.21.15.26:0/2693117247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac18004d10 0x7fac300a8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:39.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.135+0000 7fac352d7640 1 -- 172.21.15.26:0/2693117247 >> 172.21.15.26:0/2693117247 conn(0x7fac300658e0 msgr2=0x7fac300cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.135+0000 7fac352d7640 1 -- 172.21.15.26:0/2693117247 shutdown_connections 2023-12-19T16:54:39.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:39.139+0000 7fac352d7640 1 -- 172.21.15.26:0/2693117247 wait complete. 2023-12-19T16:54:39.171 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-12-19T16:54:39.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:39.171 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-12-19T16:54:39.181 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:39.181 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:39.181 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:39.181 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:39.182 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:39.182 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:39.182 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:39.182 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:39.182 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:39.182 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:39.182 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:39.183 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:39.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:39.183 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_44_00_UTC) 2023-12-19T16:54:39.243 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:39.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:39.244 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_45_00_UTC) 2023-12-19T16:54:39.299 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:39.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:39.300 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_46_00_UTC) 2023-12-19T16:54:39.355 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:39.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:39.355 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_47_00_UTC) 2023-12-19T16:54:39.411 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_48_00_UTC 2023-12-19T16:54:39.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:39.411 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_48_00_UTC) 2023-12-19T16:54:39.471 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:39.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:39.471 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_49_00_UTC) 2023-12-19T16:54:39.527 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:39.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:39.528 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_50_00_UTC) 2023-12-19T16:54:39.584 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:39.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:39.584 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_51_00_UTC) 2023-12-19T16:54:39.682 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:39.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:39.682 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_52_00_UTC) 2023-12-19T16:54:39.715 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:39.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:39.716 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_53_00_UTC) 2023-12-19T16:54:39.776 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:39.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:39.776 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-19-16_54_00_UTC) 2023-12-19T16:54:39.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:39.832 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-12-19T16:54:39.885 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir9 1M 2023-12-19T16:54:40.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f668af6d640 1 Processor -- start 2023-12-19T16:54:40.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f668af6d640 1 -- start start 2023-12-19T16:54:40.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f668af6d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f66840706d0 0x7f66840b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:40.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f668af6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6684070ae0 con 0x7f668415ada0 2023-12-19T16:54:40.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f668af6d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6684070d90 con 0x7f668415a9c0 2023-12-19T16:54:40.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f668af6d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66840cc270 con 0x7f66840706d0 2023-12-19T16:54:40.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f6683fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f668415ada0 0x7f668406ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60182/0 (socket says 172.21.15.26:60182) 2023-12-19T16:54:40.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f6683fff640 1 -- 172.21.15.26:0/879131330 learned_addr learned my addr 172.21.15.26:0/879131330 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:40.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f668950a640 1 --2- 172.21.15.26:0/879131330 >> v2:172.21.15.160:3300/0 conn(0x7f66840706d0 0x7f66840b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:40.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f66837fe640 1 -- 172.21.15.26:0/879131330 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2087446348 0 0) 0x7f6684070ae0 con 0x7f668415ada0 2023-12-19T16:54:40.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f66837fe640 1 -- 172.21.15.26:0/879131330 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6674003530 con 0x7f668415ada0 2023-12-19T16:54:40.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f66837fe640 1 -- 172.21.15.26:0/879131330 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783357378 0 0) 0x7f6684070d90 con 0x7f668415a9c0 2023-12-19T16:54:40.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f66837fe640 1 -- 172.21.15.26:0/879131330 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6684070ae0 con 0x7f668415a9c0 2023-12-19T16:54:40.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f668950a640 1 -- 172.21.15.26:0/879131330 >> v1:172.21.15.160:6789/0 conn(0x7f668415a9c0 legacy=0x7f66840b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:40.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f668950a640 1 -- 172.21.15.26:0/879131330 >> v1:172.21.15.26:6789/0 conn(0x7f668415ada0 legacy=0x7f668406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:40.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.075+0000 7f668950a640 1 -- 172.21.15.26:0/879131330 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6684071070 con 0x7f66840706d0 2023-12-19T16:54:40.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.079+0000 7f668950a640 1 --2- 172.21.15.26:0/879131330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66840706d0 0x7f66840b01b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f667c009d90 tx=0x7f667c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c683d098d5fed39 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:40.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.079+0000 7f66837fe640 1 -- 172.21.15.26:0/879131330 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0a5860 con 0x7f66840706d0 2023-12-19T16:54:40.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.079+0000 7f66837fe640 1 -- 172.21.15.26:0/879131330 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c0a59c0 con 0x7f66840706d0 2023-12-19T16:54:40.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.079+0000 7f66837fe640 1 -- 172.21.15.26:0/879131330 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0a5d50 con 0x7f66840706d0 2023-12-19T16:54:40.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.079+0000 7f668af6d640 1 -- 172.21.15.26:0/879131330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66840706d0 msgr2=0x7f66840b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:40.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.079+0000 7f668af6d640 1 --2- 172.21.15.26:0/879131330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66840706d0 0x7f66840b01b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f667c009d90 tx=0x7f667c01c890 comp rx=0 tx=0).stop 2023-12-19T16:54:40.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.079+0000 7f668af6d640 1 -- 172.21.15.26:0/879131330 shutdown_connections 2023-12-19T16:54:40.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.079+0000 7f668af6d640 1 --2- 172.21.15.26:0/879131330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66840706d0 0x7f66840b01b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:40.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.079+0000 7f668af6d640 1 -- 172.21.15.26:0/879131330 >> 172.21.15.26:0/879131330 conn(0x7f66840ce730 msgr2=0x7f66840ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:40.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.079+0000 7f668af6d640 1 -- 172.21.15.26:0/879131330 shutdown_connections 2023-12-19T16:54:40.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.079+0000 7f668af6d640 1 -- 172.21.15.26:0/879131330 wait complete. 2023-12-19T16:54:40.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668af6d640 1 Processor -- start 2023-12-19T16:54:40.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668af6d640 1 -- start start 2023-12-19T16:54:40.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668af6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66840706d0 0x7f66840c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:40.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668af6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f668415a9c0 0x7f66841517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:40.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668af6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f668415ada0 0x7f66840c2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668af6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66840c7800 con 0x7f668415ada0 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668af6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66840c8160 con 0x7f66840706d0 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668af6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66840c71a0 con 0x7f668415a9c0 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668950a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f668415ada0 0x7f66840c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668950a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f668415ada0 0x7f66840c2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51478/0 (socket says 172.21.15.26:51478) 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668950a640 1 -- 172.21.15.26:0/596836326 learned_addr learned my addr 172.21.15.26:0/596836326 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f6688d09640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66840706d0 0x7f66840c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f6683fff640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f668415a9c0 0x7f66841517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668950a640 1 -- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f668415a9c0 msgr2=0x7f66841517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668950a640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f668415a9c0 0x7f66841517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668950a640 1 -- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66840706d0 msgr2=0x7f66840c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668950a640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66840706d0 0x7f66840c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668950a640 1 -- 172.21.15.26:0/596836326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c009a70 con 0x7f668415ada0 2023-12-19T16:54:40.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668950a640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f668415ada0 0x7f66840c2430 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f667c0b0040 tx=0x7f667c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:40.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f66817fa640 1 -- 172.21.15.26:0/596836326 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0b9650 con 0x7f668415ada0 2023-12-19T16:54:40.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f66817fa640 1 -- 172.21.15.26:0/596836326 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c0b97b0 con 0x7f668415ada0 2023-12-19T16:54:40.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f66817fa640 1 -- 172.21.15.26:0/596836326 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0b9b20 con 0x7f668415ada0 2023-12-19T16:54:40.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668af6d640 1 -- 172.21.15.26:0/596836326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66840b3580 con 0x7f668415ada0 2023-12-19T16:54:40.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.083+0000 7f668af6d640 1 -- 172.21.15.26:0/596836326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66840c7420 con 0x7f668415ada0 2023-12-19T16:54:40.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.087+0000 7f668af6d640 1 -- 172.21.15.26:0/596836326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66840c7940 con 0x7f668415ada0 2023-12-19T16:54:40.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.087+0000 7f66817fa640 1 -- 172.21.15.26:0/596836326 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f667c0b9c80 con 0x7f668415ada0 2023-12-19T16:54:40.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.087+0000 7f66817fa640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6668065c60 0x7f6668068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:40.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.091+0000 7f66817fa640 1 -- 172.21.15.26:0/596836326 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6551+0+0 (secure 0 0 0) 0x7f667c11e980 con 0x7f668415ada0 2023-12-19T16:54:40.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.091+0000 7f6688d09640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6668065c60 0x7f6668068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:40.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.095+0000 7f6688d09640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6668065c60 0x7f6668068120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f66780011f0 tx=0x7f6678009040 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:40.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.095+0000 7f66817fa640 1 -- 172.21.15.26:0/596836326 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f66840c7940 con 0x7f668415ada0 2023-12-19T16:54:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.299+0000 7f668af6d640 1 -- 172.21.15.26:0/596836326 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f668406b390 con 0x7f6668065c60 2023-12-19T16:54:40.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.375+0000 7f66817fa640 1 -- 172.21.15.26:0/596836326 <== mgr.5224 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f66780014e0 con 0x7f6668065c60 2023-12-19T16:54:40.377 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-12-19T16:54:40.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.375+0000 7f668af6d640 1 -- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6668065c60 msgr2=0x7f6668068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:40.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.375+0000 7f668af6d640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6668065c60 0x7f6668068120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f66780011f0 tx=0x7f6678009040 comp rx=0 tx=0).stop 2023-12-19T16:54:40.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.375+0000 7f668af6d640 1 -- 172.21.15.26:0/596836326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f668415ada0 msgr2=0x7f66840c2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:40.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.375+0000 7f668af6d640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f668415ada0 0x7f66840c2430 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f667c0b0040 tx=0x7f667c019630 comp rx=0 tx=0).stop 2023-12-19T16:54:40.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.375+0000 7f668af6d640 1 -- 172.21.15.26:0/596836326 shutdown_connections 2023-12-19T16:54:40.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.375+0000 7f668af6d640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f668415ada0 0x7f66840c2430 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:40.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.375+0000 7f668af6d640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6668065c60 0x7f6668068120 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:40.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.375+0000 7f668af6d640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f668415a9c0 0x7f66841517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:40.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.375+0000 7f668af6d640 1 --2- 172.21.15.26:0/596836326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66840706d0 0x7f66840c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:40.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.375+0000 7f668af6d640 1 -- 172.21.15.26:0/596836326 >> 172.21.15.26:0/596836326 conn(0x7f66840ce730 msgr2=0x7f66840b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:40.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.375+0000 7f668af6d640 1 -- 172.21.15.26:0/596836326 shutdown_connections 2023-12-19T16:54:40.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:40.375+0000 7f668af6d640 1 -- 172.21.15.26:0/596836326 wait complete. 2023-12-19T16:54:40.398 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-12-19T16:54:40.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:40.399 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-12-19T16:54:40.410 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:40.410 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:40.411 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:40.411 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:40.411 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_48_01_UTC 2023-12-19T16:54:40.411 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:40.411 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:40.411 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:40.411 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:40.411 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:40.411 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:40.411 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_44_00_UTC 2023-12-19T16:54:40.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:40.411 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_44_00_UTC) 2023-12-19T16:54:40.469 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_45_00_UTC 2023-12-19T16:54:40.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:40.469 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_45_00_UTC) 2023-12-19T16:54:40.532 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_46_00_UTC 2023-12-19T16:54:40.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:40.532 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_46_00_UTC) 2023-12-19T16:54:40.588 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_47_00_UTC 2023-12-19T16:54:40.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:40.588 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_47_00_UTC) 2023-12-19T16:54:40.682 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_48_01_UTC 2023-12-19T16:54:40.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:40.682 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_48_01_UTC) 2023-12-19T16:54:40.710 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_49_00_UTC 2023-12-19T16:54:40.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:40.710 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_49_00_UTC) 2023-12-19T16:54:40.767 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_50_00_UTC 2023-12-19T16:54:40.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:40.768 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_50_00_UTC) 2023-12-19T16:54:40.823 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_51_00_UTC 2023-12-19T16:54:40.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:40.823 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_51_00_UTC) 2023-12-19T16:54:40.882 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_52_00_UTC 2023-12-19T16:54:40.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:40.882 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_52_00_UTC) 2023-12-19T16:54:40.939 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_53_00_UTC 2023-12-19T16:54:40.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:40.940 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_53_00_UTC) 2023-12-19T16:54:40.995 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_54_00_UTC 2023-12-19T16:54:40.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:40.996 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-19-16_54_00_UTC) 2023-12-19T16:54:41.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:41.052 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-12-19T16:54:41.112 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-19T16:54:41.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.307+0000 7f268dde9640 1 Processor -- start 2023-12-19T16:54:41.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.307+0000 7f268dde9640 1 -- start start 2023-12-19T16:54:41.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.307+0000 7f268dde9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f26880cb600 0x7f268815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:41.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.307+0000 7f268dde9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26880ce3f0 con 0x7f26881585f0 2023-12-19T16:54:41.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.307+0000 7f268dde9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26880cba50 con 0x7f26880cb150 2023-12-19T16:54:41.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.307+0000 7f268dde9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26880b47a0 con 0x7f26880cb600 2023-12-19T16:54:41.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.307+0000 7f26877fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f26881585f0 0x7f26880cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60184/0 (socket says 172.21.15.26:60184) 2023-12-19T16:54:41.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.307+0000 7f26877fe640 1 -- 172.21.15.26:0/2063829901 learned_addr learned my addr 172.21.15.26:0/2063829901 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:41.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.307+0000 7f2687fff640 1 --2- 172.21.15.26:0/2063829901 >> v2:172.21.15.160:3300/0 conn(0x7f26880cb600 0x7f268815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:41.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.307+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649414536 0 0) 0x7f26880ce3f0 con 0x7f26881585f0 2023-12-19T16:54:41.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2670003530 con 0x7f26881585f0 2023-12-19T16:54:41.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1357180602 0 0) 0x7f2670003530 con 0x7f26881585f0 2023-12-19T16:54:41.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 >> v2:172.21.15.160:3300/0 conn(0x7f26880cb600 msgr2=0x7f268815eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:41.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 --2- 172.21.15.26:0/2063829901 >> v2:172.21.15.160:3300/0 conn(0x7f26880cb600 0x7f268815eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:41.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 >> v1:172.21.15.160:6789/0 conn(0x7f26880cb150 legacy=0x7f268815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268815b1b0 con 0x7f26881585f0 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f2678001680 con 0x7f26881585f0 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 >> v1:172.21.15.26:6789/0 conn(0x7f26881585f0 legacy=0x7f26880cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 --2- 172.21.15.26:0/2063829901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2670003980 0x7f2670003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 --2- 172.21.15.26:0/2063829901 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26700047d0 0x7f2670004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 --2- 172.21.15.26:0/2063829901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2670005190 0x7f267000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26880b47a0 con 0x7f2670003980 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267000c130 con 0x7f2670005190 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267000c2f0 con 0x7f26700047d0 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26877fe640 1 --2- 172.21.15.26:0/2063829901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2670003980 0x7f2670003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f2687fff640 1 --2- 172.21.15.26:0/2063829901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2670005190 0x7f267000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f2686ffd640 1 --2- 172.21.15.26:0/2063829901 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26700047d0 0x7f2670004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26877fe640 1 -- 172.21.15.26:0/2063829901 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26700047d0 msgr2=0x7f2670004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:41.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26877fe640 1 --2- 172.21.15.26:0/2063829901 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26700047d0 0x7f2670004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:41.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26877fe640 1 -- 172.21.15.26:0/2063829901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2670005190 msgr2=0x7f267000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:41.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26877fe640 1 --2- 172.21.15.26:0/2063829901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2670005190 0x7f267000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:41.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26877fe640 1 -- 172.21.15.26:0/2063829901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2670003530 con 0x7f2670003980 2023-12-19T16:54:41.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f2687fff640 1 -- 172.21.15.26:0/2063829901 reap_dead start 2023-12-19T16:54:41.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f2686ffd640 1 --2- 172.21.15.26:0/2063829901 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26700047d0 0x7f2670004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:41.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.311+0000 7f26877fe640 1 --2- 172.21.15.26:0/2063829901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2670003980 0x7f2670003da0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f267800c240 tx=0x7f267801cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebfe2758c59d6d49 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:41.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2678005ae0 con 0x7f2670003980 2023-12-19T16:54:41.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2678005c40 con 0x7f2670003980 2023-12-19T16:54:41.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f26867fc640 1 -- 172.21.15.26:0/2063829901 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26780a5ab0 con 0x7f2670003980 2023-12-19T16:54:41.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 -- 172.21.15.26:0/2063829901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2670003980 msgr2=0x7f2670003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:41.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 --2- 172.21.15.26:0/2063829901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2670003980 0x7f2670003da0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f267800c240 tx=0x7f267801cb90 comp rx=0 tx=0).stop 2023-12-19T16:54:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 -- 172.21.15.26:0/2063829901 shutdown_connections 2023-12-19T16:54:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 --2- 172.21.15.26:0/2063829901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2670003980 0x7f2670003da0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 -- 172.21.15.26:0/2063829901 >> 172.21.15.26:0/2063829901 conn(0x7f2688066160 msgr2=0x7f26880b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 -- 172.21.15.26:0/2063829901 shutdown_connections 2023-12-19T16:54:41.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 -- 172.21.15.26:0/2063829901 wait complete. 2023-12-19T16:54:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 Processor -- start 2023-12-19T16:54:41.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 -- start start 2023-12-19T16:54:41.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2670003980 0x7f2688155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26880b12c0 0x7f268806b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26880b1cb0 0x7f268806b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2688155590 con 0x7f26880b12c0 2023-12-19T16:54:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26881f4e30 con 0x7f2670003980 2023-12-19T16:54:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f268dde9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26880b1830 con 0x7f26880b1cb0 2023-12-19T16:54:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.315+0000 7f2686ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26880b12c0 0x7f268806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f2686ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26880b12c0 0x7f268806b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51494/0 (socket says 172.21.15.26:51494) 2023-12-19T16:54:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f2686ffd640 1 -- 172.21.15.26:0/1599435746 learned_addr learned my addr 172.21.15.26:0/1599435746 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f2687fff640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26880b1cb0 0x7f268806b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f26877fe640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2670003980 0x7f2688155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f2686ffd640 1 -- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26880b1cb0 msgr2=0x7f268806b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f2686ffd640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26880b1cb0 0x7f268806b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:41.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f2686ffd640 1 -- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2670003980 msgr2=0x7f2688155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f2686ffd640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2670003980 0x7f2688155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f2686ffd640 1 -- 172.21.15.26:0/1599435746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26881f4500 con 0x7f26880b12c0 2023-12-19T16:54:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f2686ffd640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26880b12c0 0x7f268806b310 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f267c00e480 tx=0x7f267c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f266ffff640 1 -- 172.21.15.26:0/1599435746 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c017070 con 0x7f26880b12c0 2023-12-19T16:54:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f266ffff640 1 -- 172.21.15.26:0/1599435746 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c0052e0 con 0x7f26880b12c0 2023-12-19T16:54:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f268dde9640 1 -- 172.21.15.26:0/1599435746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26881f47e0 con 0x7f26880b12c0 2023-12-19T16:54:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f266ffff640 1 -- 172.21.15.26:0/1599435746 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c005490 con 0x7f26880b12c0 2023-12-19T16:54:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f268dde9640 1 -- 172.21.15.26:0/1599435746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26880aefc0 con 0x7f26880b12c0 2023-12-19T16:54:41.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f268dde9640 1 -- 172.21.15.26:0/1599435746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2688074270 con 0x7f26880b12c0 2023-12-19T16:54:41.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.319+0000 7f266ffff640 1 -- 172.21.15.26:0/1599435746 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f267c007020 con 0x7f26880b12c0 2023-12-19T16:54:41.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.323+0000 7f266ffff640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2674065c90 0x7f2674068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:41.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.323+0000 7f266ffff640 1 -- 172.21.15.26:0/1599435746 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f267c082ef0 con 0x7f26880b12c0 2023-12-19T16:54:41.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.323+0000 7f26877fe640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2674065c90 0x7f2674068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:41.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.323+0000 7f26877fe640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2674065c90 0x7f2674068150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f26780ab000 tx=0x7f26780a7070 comp rx=0 tx=0).ready entity=mgr.5224 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:41.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.327+0000 7f266ffff640 1 -- 172.21.15.26:0/1599435746 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f267c050370 con 0x7f26880b12c0 2023-12-19T16:54:41.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.519+0000 7f268dde9640 1 -- 172.21.15.26:0/1599435746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f26881f4f70 con 0x7f26880b12c0 2023-12-19T16:54:41.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.639+0000 7f266ffff640 1 -- 172.21.15.26:0/1599435746 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f267c050740 con 0x7f26880b12c0 2023-12-19T16:54:41.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.643+0000 7f266ffff640 1 -- 172.21.15.26:0/1599435746 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v14) v1 ==== 93+0+0 (secure 0 0 0) 0x7f267c07e100 con 0x7f26880b12c0 2023-12-19T16:54:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.643+0000 7f268dde9640 1 -- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2674065c90 msgr2=0x7f2674068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.643+0000 7f268dde9640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2674065c90 0x7f2674068150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f26780ab000 tx=0x7f26780a7070 comp rx=0 tx=0).stop 2023-12-19T16:54:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.643+0000 7f268dde9640 1 -- 172.21.15.26:0/1599435746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26880b12c0 msgr2=0x7f268806b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.643+0000 7f268dde9640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26880b12c0 0x7f268806b310 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f267c00e480 tx=0x7f267c00e940 comp rx=0 tx=0).stop 2023-12-19T16:54:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.643+0000 7f268dde9640 1 -- 172.21.15.26:0/1599435746 shutdown_connections 2023-12-19T16:54:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.643+0000 7f268dde9640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2674065c90 0x7f2674068150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.643+0000 7f268dde9640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26880b1cb0 0x7f268806b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.643+0000 7f268dde9640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26880b12c0 0x7f268806b310 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.643+0000 7f268dde9640 1 --2- 172.21.15.26:0/1599435746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2670003980 0x7f2688155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:41.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.643+0000 7f268dde9640 1 -- 172.21.15.26:0/1599435746 >> 172.21.15.26:0/1599435746 conn(0x7f2688066160 msgr2=0x7f2688157b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:41.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.643+0000 7f268dde9640 1 -- 172.21.15.26:0/1599435746 shutdown_connections 2023-12-19T16:54:41.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:41.643+0000 7f268dde9640 1 -- 172.21.15.26:0/1599435746 wait complete. 2023-12-19T16:54:41.679 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T16:54:41.680 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:54:41.753 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:54:41.833 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T16:54:41.833 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:54:41.834 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:54:41.845 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:54:41.845 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:54:41.855 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:41.851+0000 7f433f82c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:54:41.862 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:41.861+0000 7ff878f58180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:54:41.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:54:42.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6522134640 1 Processor -- start 2023-12-19T16:54:42.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6522134640 1 -- start start 2023-12-19T16:54:42.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6522134640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f651c0ada30 0x7f651c0c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6522134640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f651c0cb570 0x7f651c0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6522134640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c0ade10 con 0x7f651c164f00 2023-12-19T16:54:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6522134640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651c0b16e0 con 0x7f651c0cb570 2023-12-19T16:54:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6522134640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0cc9d0 con 0x7f651c0ada30 2023-12-19T16:54:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6521933640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f651c0cb570 0x7f651c0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6521933640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f651c0cb570 0x7f651c0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51524/0 (socket says 172.21.15.26:51524) 2023-12-19T16:54:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6521933640 1 -- 172.21.15.26:0/2786911587 learned_addr learned my addr 172.21.15.26:0/2786911587 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6521132640 1 --2- 172.21.15.26:0/2786911587 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f651c0ada30 0x7f651c0c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6521933640 1 -- 172.21.15.26:0/2786911587 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f651c0ada30 msgr2=0x7f651c0c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6521933640 1 --2- 172.21.15.26:0/2786911587 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f651c0ada30 0x7f651c0c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6521933640 1 -- 172.21.15.26:0/2786911587 >> v1:172.21.15.160:6789/0 conn(0x7f651c164f00 legacy=0x7f651c0cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.055+0000 7f6521933640 1 -- 172.21.15.26:0/2786911587 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c161830 con 0x7f651c0cb570 2023-12-19T16:54:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.059+0000 7f6521933640 1 --2- 172.21.15.26:0/2786911587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651c0cb570 0x7f651c0cce60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6514009e50 tx=0x7f651401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bfe3ed7b1dd6e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.059+0000 7f6513fff640 1 -- 172.21.15.26:0/2786911587 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140a57d0 con 0x7f651c0cb570 2023-12-19T16:54:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.059+0000 7f6513fff640 1 -- 172.21.15.26:0/2786911587 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65140a5930 con 0x7f651c0cb570 2023-12-19T16:54:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.059+0000 7f6513fff640 1 -- 172.21.15.26:0/2786911587 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140a5cc0 con 0x7f651c0cb570 2023-12-19T16:54:42.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.059+0000 7f6522134640 1 -- 172.21.15.26:0/2786911587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651c0cb570 msgr2=0x7f651c0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:42.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.059+0000 7f6522134640 1 --2- 172.21.15.26:0/2786911587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651c0cb570 0x7f651c0cce60 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6514009e50 tx=0x7f651401ca10 comp rx=0 tx=0).stop 2023-12-19T16:54:42.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.059+0000 7f6522134640 1 -- 172.21.15.26:0/2786911587 shutdown_connections 2023-12-19T16:54:42.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.059+0000 7f6522134640 1 --2- 172.21.15.26:0/2786911587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651c0cb570 0x7f651c0cce60 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.059+0000 7f6522134640 1 --2- 172.21.15.26:0/2786911587 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f651c0ada30 0x7f651c0c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.059+0000 7f6522134640 1 -- 172.21.15.26:0/2786911587 >> 172.21.15.26:0/2786911587 conn(0x7f651c06b0f0 msgr2=0x7f651c06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:42.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.059+0000 7f6522134640 1 -- 172.21.15.26:0/2786911587 shutdown_connections 2023-12-19T16:54:42.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.059+0000 7f6522134640 1 -- 172.21.15.26:0/2786911587 wait complete. 2023-12-19T16:54:42.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6522134640 1 Processor -- start 2023-12-19T16:54:42.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6522134640 1 -- start start 2023-12-19T16:54:42.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6522134640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f651c0ada30 0x7f651c15cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6522134640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f651c0cb570 0x7f651c070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6522134640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651c164f00 0x7f651c1641e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6522134640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c161b40 con 0x7f651c164f00 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6522134640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c071440 con 0x7f651c0ada30 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6522134640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c071720 con 0x7f651c0cb570 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6521933640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651c164f00 0x7f651c1641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6521132640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f651c0ada30 0x7f651c15cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6521933640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651c164f00 0x7f651c1641e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51528/0 (socket says 172.21.15.26:51528) 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6521933640 1 -- 172.21.15.26:0/623042007 learned_addr learned my addr 172.21.15.26:0/623042007 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6520931640 1 --2- 172.21.15.26:0/623042007 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f651c0cb570 0x7f651c070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6521933640 1 -- 172.21.15.26:0/623042007 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f651c0cb570 msgr2=0x7f651c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6521933640 1 --2- 172.21.15.26:0/623042007 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f651c0cb570 0x7f651c070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6521933640 1 -- 172.21.15.26:0/623042007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f651c0ada30 msgr2=0x7f651c15cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6521933640 1 --2- 172.21.15.26:0/623042007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f651c0ada30 0x7f651c15cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6521933640 1 -- 172.21.15.26:0/623042007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6514009b30 con 0x7f651c164f00 2023-12-19T16:54:42.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6521933640 1 --2- 172.21.15.26:0/623042007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651c164f00 0x7f651c1641e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f651401a640 tx=0x7f65140197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:42.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6511ffb640 1 -- 172.21.15.26:0/623042007 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140b8670 con 0x7f651c164f00 2023-12-19T16:54:42.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6511ffb640 1 -- 172.21.15.26:0/623042007 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65140b8810 con 0x7f651c164f00 2023-12-19T16:54:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6511ffb640 1 -- 172.21.15.26:0/623042007 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140b8bc0 con 0x7f651c164f00 2023-12-19T16:54:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6522134640 1 -- 172.21.15.26:0/623042007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c161e00 con 0x7f651c164f00 2023-12-19T16:54:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6522134640 1 -- 172.21.15.26:0/623042007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c070ec0 con 0x7f651c164f00 2023-12-19T16:54:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.063+0000 7f6522134640 1 -- 172.21.15.26:0/623042007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c162040 con 0x7f651c164f00 2023-12-19T16:54:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.067+0000 7f6511ffb640 1 -- 172.21.15.26:0/623042007 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f65140b8d20 con 0x7f651c164f00 2023-12-19T16:54:42.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.067+0000 7f6511ffb640 1 --2- 172.21.15.26:0/623042007 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6500065c10 0x7f65000680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.067+0000 7f6511ffb640 1 -- 172.21.15.26:0/623042007 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6279+0+0 (secure 0 0 0) 0x7f651411eab0 con 0x7f651c164f00 2023-12-19T16:54:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.067+0000 7f6521132640 1 -- 172.21.15.26:0/623042007 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6500065c10 msgr2=0x7f65000680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:54:42.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.067+0000 7f6521132640 1 --2- 172.21.15.26:0/623042007 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6500065c10 0x7f65000680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:54:42.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.071+0000 7f6511ffb640 1 -- 172.21.15.26:0/623042007 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f65140ec180 con 0x7f651c164f00 2023-12-19T16:54:42.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.239+0000 7f6522134640 1 -- 172.21.15.26:0/623042007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f651c071940 con 0x7f651c164f00 2023-12-19T16:54:42.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.239+0000 7f6511ffb640 1 -- 172.21.15.26:0/623042007 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16975 (secure 0 0 0) 0x7f65140ef7a0 con 0x7f651c164f00 2023-12-19T16:54:42.243 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:42.243 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":46,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:54:41.636887+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:43:24.052043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T16:43:24.856934+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":212,"snap_epoch":46,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6833/38098":"2023-12-20T16:43:42.161288+0000","172.21.15.160:6832/38098":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/1318901201":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/713602764":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/810298028":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/1246358820":"2023-12-20T16:43:42.161288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":28},{"begin":31,"length":10},{"begin":42,"length":9},{"begin":52,"length":1},{"begin":54,"length":9},{"begin":64,"length":9},{"begin":74,"length":9},{"begin":84,"length":11},{"begin":96,"length":9},{"begin":106,"length":107}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":19,"length":1},{"begin":69,"length":1},{"begin":79,"length":1},{"begin":89,"length":1},{"begin":101,"length":1},{"begin":111,"length":1},{"begin":206,"length":7}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:54:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.239+0000 7f6522134640 1 -- 172.21.15.26:0/623042007 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6500065c10 msgr2=0x7f65000680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:54:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.239+0000 7f6522134640 1 --2- 172.21.15.26:0/623042007 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6500065c10 0x7f65000680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.239+0000 7f6522134640 1 -- 172.21.15.26:0/623042007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651c164f00 msgr2=0x7f651c1641e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.239+0000 7f6522134640 1 --2- 172.21.15.26:0/623042007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651c164f00 0x7f651c1641e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f651401a640 tx=0x7f65140197b0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.239+0000 7f6522134640 1 -- 172.21.15.26:0/623042007 shutdown_connections 2023-12-19T16:54:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.239+0000 7f6522134640 1 --2- 172.21.15.26:0/623042007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f651c164f00 0x7f651c1641e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.243+0000 7f6522134640 1 --2- 172.21.15.26:0/623042007 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6500065c10 0x7f65000680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.243+0000 7f6522134640 1 --2- 172.21.15.26:0/623042007 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f651c0cb570 0x7f651c070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.243+0000 7f6522134640 1 --2- 172.21.15.26:0/623042007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f651c0ada30 0x7f651c15cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.243+0000 7f6522134640 1 -- 172.21.15.26:0/623042007 >> 172.21.15.26:0/623042007 conn(0x7f651c06b0f0 msgr2=0x7f651c0b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.243+0000 7f6522134640 1 -- 172.21.15.26:0/623042007 shutdown_connections 2023-12-19T16:54:42.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.243+0000 7f6522134640 1 -- 172.21.15.26:0/623042007 wait complete. 2023-12-19T16:54:42.265 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:43:24.052043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:43:24.856934+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 212, 'snap_epoch': 46, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:54:42.265 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:54:42.293 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:42.289+0000 7ff878f58180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:54:42.310 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:42.307+0000 7f433f82c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:54:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f1897118640 1 Processor -- start 2023-12-19T16:54:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f1897118640 1 -- start start 2023-12-19T16:54:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f1897118640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f18900b8600 0x7f18900b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f1897118640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f18900b4b30 0x7f18900d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f1897118640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18900b5bd0 con 0x7f189015cde0 2023-12-19T16:54:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f1897118640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189015e380 con 0x7f18900b8600 2023-12-19T16:54:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f1897118640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18900b4fc0 con 0x7f18900b4b30 2023-12-19T16:54:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f1894eb4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f189015cde0 0x7f189015d1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60188/0 (socket says 172.21.15.26:60188) 2023-12-19T16:54:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f1894eb4640 1 -- 172.21.15.26:0/1531610053 learned_addr learned my addr 172.21.15.26:0/1531610053 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f188ffff640 1 --2- 172.21.15.26:0/1531610053 >> v2:172.21.15.26:3300/0 conn(0x7f18900b8600 0x7f18900b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f18956b5640 1 --2- 172.21.15.26:0/1531610053 >> v2:172.21.15.160:3300/0 conn(0x7f18900b4b30 0x7f18900d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f188ffff640 1 -- 172.21.15.26:0/1531610053 >> v2:172.21.15.160:3300/0 conn(0x7f18900b4b30 msgr2=0x7f18900d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f188ffff640 1 --2- 172.21.15.26:0/1531610053 >> v2:172.21.15.160:3300/0 conn(0x7f18900b4b30 0x7f18900d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f188ffff640 1 -- 172.21.15.26:0/1531610053 >> v1:172.21.15.26:6789/0 conn(0x7f189015cde0 legacy=0x7f189015d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f188ffff640 1 -- 172.21.15.26:0/1531610053 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18900d0ee0 con 0x7f18900b8600 2023-12-19T16:54:42.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.411+0000 7f188f7fe640 1 -- 172.21.15.26:0/1531610053 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3944299797 0 0) 0x7f18900b5bd0 con 0x7f189015cde0 2023-12-19T16:54:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.415+0000 7f188ffff640 1 --2- 172.21.15.26:0/1531610053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18900b8600 0x7f18900b45f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f18900cd3e0 tx=0x7f187c01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a02df4dfd30a5a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.415+0000 7f188f7fe640 1 -- 172.21.15.26:0/1531610053 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c0a8950 con 0x7f18900b8600 2023-12-19T16:54:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.415+0000 7f188f7fe640 1 -- 172.21.15.26:0/1531610053 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c0a8ab0 con 0x7f18900b8600 2023-12-19T16:54:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.415+0000 7f188f7fe640 1 -- 172.21.15.26:0/1531610053 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c0a8e40 con 0x7f18900b8600 2023-12-19T16:54:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.415+0000 7f1897118640 1 -- 172.21.15.26:0/1531610053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18900b8600 msgr2=0x7f18900b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.415+0000 7f1897118640 1 --2- 172.21.15.26:0/1531610053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18900b8600 0x7f18900b45f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f18900cd3e0 tx=0x7f187c01ec50 comp rx=0 tx=0).stop 2023-12-19T16:54:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.415+0000 7f1897118640 1 -- 172.21.15.26:0/1531610053 shutdown_connections 2023-12-19T16:54:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.415+0000 7f1897118640 1 --2- 172.21.15.26:0/1531610053 >> v2:172.21.15.160:3300/0 conn(0x7f18900b4b30 0x7f18900d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.415+0000 7f1897118640 1 --2- 172.21.15.26:0/1531610053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18900b8600 0x7f18900b45f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.415+0000 7f1897118640 1 -- 172.21.15.26:0/1531610053 >> 172.21.15.26:0/1531610053 conn(0x7f1890065ee0 msgr2=0x7f18900d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.415+0000 7f1897118640 1 -- 172.21.15.26:0/1531610053 shutdown_connections 2023-12-19T16:54:42.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.415+0000 7f1897118640 1 -- 172.21.15.26:0/1531610053 wait complete. 2023-12-19T16:54:42.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1897118640 1 Processor -- start 2023-12-19T16:54:42.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1897118640 1 -- start start 2023-12-19T16:54:42.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1897118640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18900b4b30 0x7f1890152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1897118640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18900b8600 0x7f1890155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1897118640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f189015cde0 0x7f18900b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1897118640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18901540f0 con 0x7f18900b4b30 2023-12-19T16:54:42.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1897118640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18901534e0 con 0x7f18900b8600 2023-12-19T16:54:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1897118640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18901537c0 con 0x7f189015cde0 2023-12-19T16:54:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1894eb4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18900b4b30 0x7f1890152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1894eb4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18900b4b30 0x7f1890152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51542/0 (socket says 172.21.15.26:51542) 2023-12-19T16:54:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1894eb4640 1 -- 172.21.15.26:0/2186489901 learned_addr learned my addr 172.21.15.26:0/2186489901 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f188ffff640 1 --2- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18900b8600 0x7f1890155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f18956b5640 1 --2- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f189015cde0 0x7f18900b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1894eb4640 1 -- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f189015cde0 msgr2=0x7f18900b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1894eb4640 1 --2- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f189015cde0 0x7f18900b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1894eb4640 1 -- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18900b8600 msgr2=0x7f1890155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1894eb4640 1 --2- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18900b8600 0x7f1890155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1894eb4640 1 -- 172.21.15.26:0/2186489901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c009420 con 0x7f18900b4b30 2023-12-19T16:54:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.419+0000 7f1894eb4640 1 --2- 172.21.15.26:0/2186489901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18900b4b30 0x7f1890152e70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f188400c480 tx=0x7f188400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:42.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.423+0000 7f188d7fa640 1 -- 172.21.15.26:0/2186489901 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18840074e0 con 0x7f18900b4b30 2023-12-19T16:54:42.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.423+0000 7f188d7fa640 1 -- 172.21.15.26:0/2186489901 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1884007660 con 0x7f18900b4b30 2023-12-19T16:54:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.423+0000 7f1897118640 1 -- 172.21.15.26:0/2186489901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1890153aa0 con 0x7f18900b4b30 2023-12-19T16:54:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.423+0000 7f188d7fa640 1 -- 172.21.15.26:0/2186489901 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884004a30 con 0x7f18900b4b30 2023-12-19T16:54:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.423+0000 7f1897118640 1 -- 172.21.15.26:0/2186489901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1890152540 con 0x7f18900b4b30 2023-12-19T16:54:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.423+0000 7f1897118640 1 -- 172.21.15.26:0/2186489901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1890074270 con 0x7f18900b4b30 2023-12-19T16:54:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.423+0000 7f188d7fa640 1 -- 172.21.15.26:0/2186489901 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f1884004b90 con 0x7f18900b4b30 2023-12-19T16:54:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.423+0000 7f188d7fa640 1 --2- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1874065c10 0x7f18740680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.423+0000 7f188d7fa640 1 -- 172.21.15.26:0/2186489901 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6279+0+0 (secure 0 0 0) 0x7f1884082b60 con 0x7f18900b4b30 2023-12-19T16:54:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.423+0000 7f188ffff640 1 -- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1874065c10 msgr2=0x7f18740680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:54:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.423+0000 7f188ffff640 1 --2- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1874065c10 0x7f18740680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:54:42.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.427+0000 7f188d7fa640 1 -- 172.21.15.26:0/2186489901 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f18840501b0 con 0x7f18900b4b30 2023-12-19T16:54:42.443 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:42.441+0000 7ff878f58180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:54:42.477 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:42.475+0000 7f433f82c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:54:42.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f1897118640 1 -- 172.21.15.26:0/2186489901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18900d27c0 con 0x7f18900b4b30 2023-12-19T16:54:42.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f188d7fa640 1 -- 172.21.15.26:0/2186489901 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1884018020 con 0x7f18900b4b30 2023-12-19T16:54:42.626 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:54:42.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f1897118640 1 -- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1874065c10 msgr2=0x7f18740680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:54:42.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f1897118640 1 --2- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1874065c10 0x7f18740680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f1897118640 1 -- 172.21.15.26:0/2186489901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18900b4b30 msgr2=0x7f1890152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:42.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f1897118640 1 --2- 172.21.15.26:0/2186489901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18900b4b30 0x7f1890152e70 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f188400c480 tx=0x7f188400c940 comp rx=0 tx=0).stop 2023-12-19T16:54:42.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f1897118640 1 -- 172.21.15.26:0/2186489901 shutdown_connections 2023-12-19T16:54:42.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f1897118640 1 --2- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1874065c10 0x7f18740680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f1897118640 1 --2- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f189015cde0 0x7f18900b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f1897118640 1 --2- 172.21.15.26:0/2186489901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18900b8600 0x7f1890155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f1897118640 1 --2- 172.21.15.26:0/2186489901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18900b4b30 0x7f1890152e70 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f1897118640 1 -- 172.21.15.26:0/2186489901 >> 172.21.15.26:0/2186489901 conn(0x7f1890065ee0 msgr2=0x7f18900b7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:42.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f1897118640 1 -- 172.21.15.26:0/2186489901 shutdown_connections 2023-12-19T16:54:42.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.623+0000 7f1897118640 1 -- 172.21.15.26:0/2186489901 wait complete. 2023-12-19T16:54:42.648 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:54:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6cb0ef2640 1 Processor -- start 2023-12-19T16:54:42.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6cb0ef2640 1 -- start start 2023-12-19T16:54:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6cb0ef2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac15ccf0 0x7f6cac0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6cb0ef2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6cac15d0d0 0x7f6cac0b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6cb0ef2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6cac0b4600 0x7f6cac0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6cb0ef2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cac0b4f70 con 0x7f6cac0b4600 2023-12-19T16:54:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6cb0ef2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cac15e200 con 0x7f6cac15d0d0 2023-12-19T16:54:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6cb0ef2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac0d0aa0 con 0x7f6cac15ccf0 2023-12-19T16:54:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6caad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6cac0b4600 0x7f6cac0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6caad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6cac0b4600 0x7f6cac0b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51556/0 (socket says 172.21.15.26:51556) 2023-12-19T16:54:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6caad9d640 1 -- 172.21.15.26:0/2169466857 learned_addr learned my addr 172.21.15.26:0/2169466857 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6ca3fff640 1 --2- 172.21.15.26:0/2169466857 >> v2:172.21.15.160:3300/0 conn(0x7f6cac15d0d0 0x7f6cac0b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6caad9d640 1 -- 172.21.15.26:0/2169466857 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac15ccf0 msgr2=0x7f6cac0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:42.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6caad9d640 1 --2- 172.21.15.26:0/2169466857 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac15ccf0 0x7f6cac0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6caad9d640 1 -- 172.21.15.26:0/2169466857 >> v2:172.21.15.160:3300/0 conn(0x7f6cac15d0d0 msgr2=0x7f6cac0b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6caad9d640 1 --2- 172.21.15.26:0/2169466857 >> v2:172.21.15.160:3300/0 conn(0x7f6cac15d0d0 0x7f6cac0b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6caad9d640 1 -- 172.21.15.26:0/2169466857 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac0d1380 con 0x7f6cac0b4600 2023-12-19T16:54:42.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.843+0000 7f6ca3fff640 1 --2- 172.21.15.26:0/2169466857 >> v2:172.21.15.160:3300/0 conn(0x7f6cac15d0d0 0x7f6cac0b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:42.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.847+0000 7f6caad9d640 1 --2- 172.21.15.26:0/2169466857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac0b4600 0x7f6cac0b4a30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0d1570 tx=0x7f6ca401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb8d567d6a54c99 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.847+0000 7f6ca9d9b640 1 -- 172.21.15.26:0/2169466857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40a5730 con 0x7f6cac0b4600 2023-12-19T16:54:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.847+0000 7f6ca9d9b640 1 -- 172.21.15.26:0/2169466857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca40a5890 con 0x7f6cac0b4600 2023-12-19T16:54:42.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.847+0000 7f6ca9d9b640 1 -- 172.21.15.26:0/2169466857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40a5c20 con 0x7f6cac0b4600 2023-12-19T16:54:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.847+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2169466857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac0b4600 msgr2=0x7f6cac0b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.847+0000 7f6cb0ef2640 1 --2- 172.21.15.26:0/2169466857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac0b4600 0x7f6cac0b4a30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6cac0d1570 tx=0x7f6ca401c9a0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.847+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2169466857 shutdown_connections 2023-12-19T16:54:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.847+0000 7f6cb0ef2640 1 --2- 172.21.15.26:0/2169466857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac0b4600 0x7f6cac0b4a30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.847+0000 7f6cb0ef2640 1 --2- 172.21.15.26:0/2169466857 >> v2:172.21.15.160:3300/0 conn(0x7f6cac15d0d0 0x7f6cac0b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.847+0000 7f6cb0ef2640 1 --2- 172.21.15.26:0/2169466857 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac15ccf0 0x7f6cac0d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.847+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2169466857 >> 172.21.15.26:0/2169466857 conn(0x7f6cac065ee0 msgr2=0x7f6cac0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.847+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2169466857 shutdown_connections 2023-12-19T16:54:42.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.847+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2169466857 wait complete. 2023-12-19T16:54:42.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6cb0ef2640 1 Processor -- start 2023-12-19T16:54:42.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6cb0ef2640 1 -- start start 2023-12-19T16:54:42.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6cb0ef2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac0b4600 0x7f6cac0c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6cb0ef2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac15ccf0 0x7f6cac1584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6cb0ef2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac15d0d0 0x7f6cac0c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6cb0ef2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac166540 con 0x7f6cac0b4600 2023-12-19T16:54:42.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6cb0ef2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac158e20 con 0x7f6cac15ccf0 2023-12-19T16:54:42.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6cb0ef2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac159100 con 0x7f6cac15d0d0 2023-12-19T16:54:42.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6caa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac0b4600 0x7f6cac0c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6caa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac0b4600 0x7f6cac0c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51566/0 (socket says 172.21.15.26:51566) 2023-12-19T16:54:42.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6caa59c640 1 -- 172.21.15.26:0/2021424519 learned_addr learned my addr 172.21.15.26:0/2021424519 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:42.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6caad9d640 1 --2- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac15d0d0 0x7f6cac0c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6ca3fff640 1 --2- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac15ccf0 0x7f6cac1584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6caad9d640 1 -- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac15ccf0 msgr2=0x7f6cac1584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6caad9d640 1 --2- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac15ccf0 0x7f6cac1584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6caad9d640 1 -- 172.21.15.26:0/2021424519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac0b4600 msgr2=0x7f6cac0c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6caad9d640 1 --2- 172.21.15.26:0/2021424519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac0b4600 0x7f6cac0c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6caad9d640 1 -- 172.21.15.26:0/2021424519 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4009b30 con 0x7f6cac15d0d0 2023-12-19T16:54:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6caad9d640 1 --2- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac15d0d0 0x7f6cac0c66b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40ab000 tx=0x7f6ca4019740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6ca2ffd640 1 -- 172.21.15.26:0/2021424519 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40b8670 con 0x7f6cac15d0d0 2023-12-19T16:54:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6ca2ffd640 1 -- 172.21.15.26:0/2021424519 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca40b87d0 con 0x7f6cac15d0d0 2023-12-19T16:54:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6ca2ffd640 1 -- 172.21.15.26:0/2021424519 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca40b8b60 con 0x7f6cac15d0d0 2023-12-19T16:54:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.851+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2021424519 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac1593e0 con 0x7f6cac15d0d0 2023-12-19T16:54:42.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.855+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2021424519 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac0b5c10 con 0x7f6cac15d0d0 2023-12-19T16:54:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.855+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2021424519 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cac074270 con 0x7f6cac15d0d0 2023-12-19T16:54:42.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.855+0000 7f6ca2ffd640 1 -- 172.21.15.26:0/2021424519 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f6ca40b8cc0 con 0x7f6cac15d0d0 2023-12-19T16:54:42.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.855+0000 7f6ca2ffd640 1 --2- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c8c065c10 0x7f6c8c0680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:42.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.855+0000 7f6ca2ffd640 1 -- 172.21.15.26:0/2021424519 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7571+0+0 (secure 0 0 0) 0x7f6ca411f630 con 0x7f6cac15d0d0 2023-12-19T16:54:42.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.855+0000 7f6caa59c640 1 -- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c8c065c10 msgr2=0x7f6c8c0680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:54:42.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.855+0000 7f6caa59c640 1 --2- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c8c065c10 0x7f6c8c0680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:54:42.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:42.859+0000 7f6ca2ffd640 1 -- 172.21.15.26:0/2021424519 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6ca40ec740 con 0x7f6cac15d0d0 2023-12-19T16:54:42.979 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:42.977+0000 7ff878f58180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:54:43.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.027+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2021424519 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6cac159240 con 0x7f6cac15d0d0 2023-12-19T16:54:43.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.027+0000 7f6ca2ffd640 1 -- 172.21.15.26:0/2021424519 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6cac159240 con 0x7f6cac15d0d0 2023-12-19T16:54:43.033 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:54:43.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.031+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c8c065c10 msgr2=0x7f6c8c0680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:54:43.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.031+0000 7f6cb0ef2640 1 --2- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c8c065c10 0x7f6c8c0680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.031+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac15d0d0 msgr2=0x7f6cac0c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:43.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.031+0000 7f6cb0ef2640 1 --2- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac15d0d0 0x7f6cac0c66b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40ab000 tx=0x7f6ca4019740 comp rx=0 tx=0).stop 2023-12-19T16:54:43.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.031+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2021424519 shutdown_connections 2023-12-19T16:54:43.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.031+0000 7f6cb0ef2640 1 --2- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c8c065c10 0x7f6c8c0680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.031+0000 7f6cb0ef2640 1 --2- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cac15d0d0 0x7f6cac0c66b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.031+0000 7f6cb0ef2640 1 --2- 172.21.15.26:0/2021424519 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cac15ccf0 0x7f6cac1584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.031+0000 7f6cb0ef2640 1 --2- 172.21.15.26:0/2021424519 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cac0b4600 0x7f6cac0c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.031+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2021424519 >> 172.21.15.26:0/2021424519 conn(0x7f6cac065ee0 msgr2=0x7f6cac15e970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:43.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.031+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2021424519 shutdown_connections 2023-12-19T16:54:43.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.031+0000 7f6cb0ef2640 1 -- 172.21.15.26:0/2021424519 wait complete. 2023-12-19T16:54:43.044 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:43.041+0000 7ff878f58180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:54:43.046 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:43.043+0000 7f433f82c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:54:43.072 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:54:43.117 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:43.113+0000 7ff878f58180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:54:43.143 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:43.135+0000 7f433f82c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:54:43.200 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:43.195+0000 7f433f82c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:54:43.204 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:43.201+0000 7ff878f58180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:54:43.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.243+0000 7f34f56fc640 1 Processor -- start 2023-12-19T16:54:43.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.243+0000 7f34f56fc640 1 -- start start 2023-12-19T16:54:43.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34f56fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34f015ccf0 0x7f34f00d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:43.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34f56fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34f00b4600 0x7f34f00b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:43.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34f56fc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f00b4f70 con 0x7f34f015d0d0 2023-12-19T16:54:43.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34f56fc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f015e200 con 0x7f34f00b4600 2023-12-19T16:54:43.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34f56fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00d0aa0 con 0x7f34f015ccf0 2023-12-19T16:54:43.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34ef7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34f00b4600 0x7f34f00b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:43.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34ee7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f34f015d0d0 0x7f34f00b8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60194/0 (socket says 172.21.15.26:60194) 2023-12-19T16:54:43.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34ee7fc640 1 -- 172.21.15.26:0/838351564 learned_addr learned my addr 172.21.15.26:0/838351564 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:43.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34eeffd640 1 --2- 172.21.15.26:0/838351564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34f015ccf0 0x7f34f00d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:43.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34ef7fe640 1 -- 172.21.15.26:0/838351564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34f015ccf0 msgr2=0x7f34f00d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:43.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34ef7fe640 1 --2- 172.21.15.26:0/838351564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34f015ccf0 0x7f34f00d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34ef7fe640 1 -- 172.21.15.26:0/838351564 >> v1:172.21.15.26:6789/0 conn(0x7f34f015d0d0 legacy=0x7f34f00b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:43.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34ef7fe640 1 -- 172.21.15.26:0/838351564 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00d1380 con 0x7f34f00b4600 2023-12-19T16:54:43.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.247+0000 7f34edffb640 1 -- 172.21.15.26:0/838351564 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205062510 0 0) 0x7f34f00b4f70 con 0x7f34f015d0d0 2023-12-19T16:54:43.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.251+0000 7f34ef7fe640 1 --2- 172.21.15.26:0/838351564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f00b4600 0x7f34f00b4a30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f34f00b5100 tx=0x7f34e801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b80d3f95fb6507c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.251+0000 7f34edffb640 1 -- 172.21.15.26:0/838351564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e80a59c0 con 0x7f34f00b4600 2023-12-19T16:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.251+0000 7f34edffb640 1 -- 172.21.15.26:0/838351564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e80a5b20 con 0x7f34f00b4600 2023-12-19T16:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.251+0000 7f34edffb640 1 -- 172.21.15.26:0/838351564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e80ad440 con 0x7f34f00b4600 2023-12-19T16:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.251+0000 7f34f56fc640 1 -- 172.21.15.26:0/838351564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f00b4600 msgr2=0x7f34f00b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:43.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.251+0000 7f34f56fc640 1 --2- 172.21.15.26:0/838351564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f00b4600 0x7f34f00b4a30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f34f00b5100 tx=0x7f34e801c900 comp rx=0 tx=0).stop 2023-12-19T16:54:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.251+0000 7f34f56fc640 1 -- 172.21.15.26:0/838351564 shutdown_connections 2023-12-19T16:54:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.251+0000 7f34f56fc640 1 --2- 172.21.15.26:0/838351564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f00b4600 0x7f34f00b4a30 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.251+0000 7f34f56fc640 1 --2- 172.21.15.26:0/838351564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34f015ccf0 0x7f34f00d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.251+0000 7f34f56fc640 1 -- 172.21.15.26:0/838351564 >> 172.21.15.26:0/838351564 conn(0x7f34f0065ee0 msgr2=0x7f34f00cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:43.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.251+0000 7f34f56fc640 1 -- 172.21.15.26:0/838351564 shutdown_connections 2023-12-19T16:54:43.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.251+0000 7f34f56fc640 1 -- 172.21.15.26:0/838351564 wait complete. 2023-12-19T16:54:43.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34f56fc640 1 Processor -- start 2023-12-19T16:54:43.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34f56fc640 1 -- start start 2023-12-19T16:54:43.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34f56fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34f00b4600 0x7f34f0159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:43.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34f56fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34f015ccf0 0x7f34f015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:43.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34f56fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f015d0d0 0x7f34f015b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:43.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34f56fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f0158990 con 0x7f34f015d0d0 2023-12-19T16:54:43.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34f56fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f01586b0 con 0x7f34f00b4600 2023-12-19T16:54:43.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34f56fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00cac50 con 0x7f34f015ccf0 2023-12-19T16:54:43.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34ef7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f015d0d0 0x7f34f015b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:43.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34ef7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f015d0d0 0x7f34f015b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51582/0 (socket says 172.21.15.26:51582) 2023-12-19T16:54:43.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34ef7fe640 1 -- 172.21.15.26:0/252800441 learned_addr learned my addr 172.21.15.26:0/252800441 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:43.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34eeffd640 1 --2- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34f00b4600 0x7f34f0159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:43.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34ee7fc640 1 --2- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34f015ccf0 0x7f34f015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:43.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34ef7fe640 1 -- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34f015ccf0 msgr2=0x7f34f015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:43.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34ef7fe640 1 --2- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34f015ccf0 0x7f34f015b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34ef7fe640 1 -- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34f00b4600 msgr2=0x7f34f0159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:43.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34ef7fe640 1 --2- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34f00b4600 0x7f34f0159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34ef7fe640 1 -- 172.21.15.26:0/252800441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e8009a70 con 0x7f34f015d0d0 2023-12-19T16:54:43.262 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:43.257+0000 7ff878f58180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:54:43.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.255+0000 7f34ef7fe640 1 --2- 172.21.15.26:0/252800441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f015d0d0 0x7f34f015b5f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f34e801cde0 tx=0x7f34e80196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.259+0000 7f34d77fe640 1 -- 172.21.15.26:0/252800441 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e80bf070 con 0x7f34f015d0d0 2023-12-19T16:54:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.259+0000 7f34d77fe640 1 -- 172.21.15.26:0/252800441 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e80aeb60 con 0x7f34f015d0d0 2023-12-19T16:54:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.259+0000 7f34d77fe640 1 -- 172.21.15.26:0/252800441 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e80b7410 con 0x7f34f015d0d0 2023-12-19T16:54:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.259+0000 7f34f56fc640 1 -- 172.21.15.26:0/252800441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f00caf30 con 0x7f34f015d0d0 2023-12-19T16:54:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.259+0000 7f34f56fc640 1 -- 172.21.15.26:0/252800441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f01595a0 con 0x7f34f015d0d0 2023-12-19T16:54:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.259+0000 7f34f56fc640 1 -- 172.21.15.26:0/252800441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f00caab0 con 0x7f34f015d0d0 2023-12-19T16:54:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.259+0000 7f34d77fe640 1 -- 172.21.15.26:0/252800441 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f34e80b4050 con 0x7f34f015d0d0 2023-12-19T16:54:43.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.259+0000 7f34d77fe640 1 --2- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f34dc065c10 0x7f34dc0680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:43.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.259+0000 7f34d77fe640 1 -- 172.21.15.26:0/252800441 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7571+0+0 (secure 0 0 0) 0x7f34e811e440 con 0x7f34f015d0d0 2023-12-19T16:54:43.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.259+0000 7f34eeffd640 1 -- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f34dc065c10 msgr2=0x7f34dc0680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:54:43.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.259+0000 7f34eeffd640 1 --2- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f34dc065c10 0x7f34dc0680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:54:43.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.267+0000 7f34d77fe640 1 -- 172.21.15.26:0/252800441 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f34f00caab0 con 0x7f34f015d0d0 2023-12-19T16:54:43.295 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:43.291+0000 7f433f82c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:54:43.345 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:43.341+0000 7ff878f58180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:54:43.355 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:43.351+0000 7f433f82c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:54:43.400 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:43.397+0000 7ff878f58180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:54:43.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.423+0000 7f34f56fc640 1 -- 172.21.15.26:0/252800441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34f00caab0 con 0x7f34f015d0d0 2023-12-19T16:54:43.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.423+0000 7f34d77fe640 1 -- 172.21.15.26:0/252800441 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f34f00caab0 con 0x7f34f015d0d0 2023-12-19T16:54:43.427 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:54:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.427+0000 7f34f56fc640 1 -- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f34dc065c10 msgr2=0x7f34dc0680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:54:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.427+0000 7f34f56fc640 1 --2- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f34dc065c10 0x7f34dc0680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.427+0000 7f34f56fc640 1 -- 172.21.15.26:0/252800441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f015d0d0 msgr2=0x7f34f015b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.427+0000 7f34f56fc640 1 --2- 172.21.15.26:0/252800441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f015d0d0 0x7f34f015b5f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f34e801cde0 tx=0x7f34e80196a0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.427+0000 7f34f56fc640 1 -- 172.21.15.26:0/252800441 shutdown_connections 2023-12-19T16:54:43.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.427+0000 7f34f56fc640 1 --2- 172.21.15.26:0/252800441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34f015d0d0 0x7f34f015b5f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.427+0000 7f34f56fc640 1 --2- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f34dc065c10 0x7f34dc0680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.427+0000 7f34f56fc640 1 --2- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34f015ccf0 0x7f34f015b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.427+0000 7f34f56fc640 1 --2- 172.21.15.26:0/252800441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34f00b4600 0x7f34f0159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.427+0000 7f34f56fc640 1 -- 172.21.15.26:0/252800441 >> 172.21.15.26:0/252800441 conn(0x7f34f0065ee0 msgr2=0x7f34f00b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:43.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.427+0000 7f34f56fc640 1 -- 172.21.15.26:0/252800441 shutdown_connections 2023-12-19T16:54:43.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.427+0000 7f34f56fc640 1 -- 172.21.15.26:0/252800441 wait complete. 2023-12-19T16:54:43.445 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:43.443+0000 7f433f82c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:54:43.449 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T16:54:43.504 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:43.499+0000 7f433f82c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:54:43.543 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:43.541+0000 7ff878f58180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:54:43.601 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:43.597+0000 7ff878f58180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:54:43.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.635+0000 7f96e076d640 1 Processor -- start 2023-12-19T16:54:43.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96e076d640 1 -- start start 2023-12-19T16:54:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96e076d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f96d80d3300 0x7f96d80d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96e076d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d80d29d0 con 0x7f96d815cd40 2023-12-19T16:54:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96e076d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d815d680 con 0x7f96d80ce040 2023-12-19T16:54:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96e076d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d80ce4b0 con 0x7f96d80d3300 2023-12-19T16:54:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96de509640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f96d815cd40 0x7f96d8160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60196/0 (socket says 172.21.15.26:60196) 2023-12-19T16:54:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96de509640 1 -- 172.21.15.26:0/2606712863 learned_addr learned my addr 172.21.15.26:0/2606712863 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:43.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96ded0a640 1 --2- 172.21.15.26:0/2606712863 >> v2:172.21.15.160:3300/0 conn(0x7f96d80d3300 0x7f96d80d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96dd507640 1 -- 172.21.15.26:0/2606712863 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431586118 0 0) 0x7f96d40012a0 con 0x7f96d815cd40 2023-12-19T16:54:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96dd507640 1 -- 172.21.15.26:0/2606712863 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96c4003530 con 0x7f96d815cd40 2023-12-19T16:54:43.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96dd507640 1 -- 172.21.15.26:0/2606712863 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126208728 0 0) 0x7f96d815d680 con 0x7f96d80ce040 2023-12-19T16:54:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96dd507640 1 -- 172.21.15.26:0/2606712863 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d40012a0 con 0x7f96d80ce040 2023-12-19T16:54:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96ded0a640 1 -- 172.21.15.26:0/2606712863 >> v1:172.21.15.160:6789/0 conn(0x7f96d80ce040 legacy=0x7f96d80d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96ded0a640 1 -- 172.21.15.26:0/2606712863 >> v1:172.21.15.26:6789/0 conn(0x7f96d815cd40 legacy=0x7f96d8160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:43.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.639+0000 7f96ded0a640 1 -- 172.21.15.26:0/2606712863 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d8161750 con 0x7f96d80d3300 2023-12-19T16:54:43.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.643+0000 7f96ded0a640 1 --2- 172.21.15.26:0/2606712863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96d80d3300 0x7f96d80d0e30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009d90 tx=0x7f96d001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c455cafaa7ae35ca server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.643+0000 7f96dd507640 1 -- 172.21.15.26:0/2606712863 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00a59e0 con 0x7f96d80d3300 2023-12-19T16:54:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.643+0000 7f96dd507640 1 -- 172.21.15.26:0/2606712863 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d00a5b40 con 0x7f96d80d3300 2023-12-19T16:54:43.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.643+0000 7f96dd507640 1 -- 172.21.15.26:0/2606712863 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00ad440 con 0x7f96d80d3300 2023-12-19T16:54:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.643+0000 7f96e076d640 1 -- 172.21.15.26:0/2606712863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96d80d3300 msgr2=0x7f96d80d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.643+0000 7f96e076d640 1 --2- 172.21.15.26:0/2606712863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96d80d3300 0x7f96d80d0e30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009d90 tx=0x7f96d001c920 comp rx=0 tx=0).stop 2023-12-19T16:54:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.643+0000 7f96e076d640 1 -- 172.21.15.26:0/2606712863 shutdown_connections 2023-12-19T16:54:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.643+0000 7f96e076d640 1 --2- 172.21.15.26:0/2606712863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96d80d3300 0x7f96d80d0e30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.643+0000 7f96e076d640 1 -- 172.21.15.26:0/2606712863 >> 172.21.15.26:0/2606712863 conn(0x7f96d806a120 msgr2=0x7f96d806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.643+0000 7f96e076d640 1 -- 172.21.15.26:0/2606712863 shutdown_connections 2023-12-19T16:54:43.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.643+0000 7f96e076d640 1 -- 172.21.15.26:0/2606712863 wait complete. 2023-12-19T16:54:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96e076d640 1 Processor -- start 2023-12-19T16:54:43.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96e076d640 1 -- start start 2023-12-19T16:54:43.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96e076d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d80ce040 0x7f96d8159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96e076d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96d80d3300 0x7f96d8150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96e076d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d815cd40 0x7f96d8158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96e076d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d815c1e0 con 0x7f96d80ce040 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96e076d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d815a4e0 con 0x7f96d80d3300 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96e076d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d815a7c0 con 0x7f96d815cd40 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96de509640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d80ce040 0x7f96d8159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96de509640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d80ce040 0x7f96d8159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51584/0 (socket says 172.21.15.26:51584) 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96de509640 1 -- 172.21.15.26:0/2541376343 learned_addr learned my addr 172.21.15.26:0/2541376343 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96ded0a640 1 --2- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d815cd40 0x7f96d8158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96ddd08640 1 --2- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96d80d3300 0x7f96d8150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96de509640 1 -- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d815cd40 msgr2=0x7f96d8158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96de509640 1 --2- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d815cd40 0x7f96d8158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96de509640 1 -- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96d80d3300 msgr2=0x7f96d8150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:43.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96de509640 1 --2- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96d80d3300 0x7f96d8150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96de509640 1 -- 172.21.15.26:0/2541376343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d0009a70 con 0x7f96d80ce040 2023-12-19T16:54:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96de509640 1 --2- 172.21.15.26:0/2541376343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d80ce040 0x7f96d8159900 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c700 tx=0x7f96d400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:43.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96ceffd640 1 -- 172.21.15.26:0/2541376343 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4007270 con 0x7f96d80ce040 2023-12-19T16:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96ceffd640 1 -- 172.21.15.26:0/2541376343 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d40073d0 con 0x7f96d80ce040 2023-12-19T16:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96e076d640 1 -- 172.21.15.26:0/2541376343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d815c480 con 0x7f96d80ce040 2023-12-19T16:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96ceffd640 1 -- 172.21.15.26:0/2541376343 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d40057b0 con 0x7f96d80ce040 2023-12-19T16:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96e076d640 1 -- 172.21.15.26:0/2541376343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d815aaa0 con 0x7f96d80ce040 2023-12-19T16:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.647+0000 7f96e076d640 1 -- 172.21.15.26:0/2541376343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d815c6c0 con 0x7f96d80ce040 2023-12-19T16:54:43.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.651+0000 7f96ceffd640 1 -- 172.21.15.26:0/2541376343 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f96d4005910 con 0x7f96d80ce040 2023-12-19T16:54:43.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.651+0000 7f96ceffd640 1 --2- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f96bc065c40 0x7f96bc068100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:43.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.651+0000 7f96ceffd640 1 -- 172.21.15.26:0/2541376343 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7571+0+0 (secure 0 0 0) 0x7f96d4083300 con 0x7f96d80ce040 2023-12-19T16:54:43.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.651+0000 7f96ddd08640 1 -- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f96bc065c40 msgr2=0x7f96bc068100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:54:43.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.651+0000 7f96ddd08640 1 --2- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f96bc065c40 0x7f96bc068100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:54:43.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.655+0000 7f96ceffd640 1 -- 172.21.15.26:0/2541376343 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f96d40504c0 con 0x7f96d80ce040 2023-12-19T16:54:43.686 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:43.683+0000 7f433f82c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:54:43.754 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:43.751+0000 7f433f82c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:54:43.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96e076d640 1 -- 172.21.15.26:0/2541376343 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f96d815c6c0 con 0x7f96d80ce040 2023-12-19T16:54:43.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96ceffd640 1 -- 172.21.15.26:0/2541376343 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v47) v1 ==== 80+0+348 (secure 0 0 0) 0x7f96d815c6c0 con 0x7f96d80ce040 2023-12-19T16:54:43.837 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6833/38098 2023-12-20T16:43:42.161288+0000 2023-12-19T16:54:43.838 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6832/38098 2023-12-20T16:43:42.161288+0000 2023-12-19T16:54:43.838 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1318901201 2023-12-20T16:43:42.161288+0000 2023-12-19T16:54:43.838 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/713602764 2023-12-20T16:43:42.161288+0000 2023-12-19T16:54:43.838 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/810298028 2023-12-20T16:43:42.161288+0000 2023-12-19T16:54:43.838 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1246358820 2023-12-20T16:43:42.161288+0000 2023-12-19T16:54:43.838 INFO:teuthology.orchestra.run.smithi026.stderr:listed 6 entries 2023-12-19T16:54:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96e076d640 1 -- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f96bc065c40 msgr2=0x7f96bc068100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:54:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96e076d640 1 --2- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f96bc065c40 0x7f96bc068100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96e076d640 1 -- 172.21.15.26:0/2541376343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d80ce040 msgr2=0x7f96d8159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96e076d640 1 --2- 172.21.15.26:0/2541376343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d80ce040 0x7f96d8159900 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c700 tx=0x7f96d400cbc0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96e076d640 1 -- 172.21.15.26:0/2541376343 shutdown_connections 2023-12-19T16:54:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96e076d640 1 --2- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f96bc065c40 0x7f96bc068100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96e076d640 1 --2- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d815cd40 0x7f96d8158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96e076d640 1 --2- 172.21.15.26:0/2541376343 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96d80d3300 0x7f96d8150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96e076d640 1 --2- 172.21.15.26:0/2541376343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d80ce040 0x7f96d8159900 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96e076d640 1 -- 172.21.15.26:0/2541376343 >> 172.21.15.26:0/2541376343 conn(0x7f96d806a120 msgr2=0x7f96d815fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96e076d640 1 -- 172.21.15.26:0/2541376343 shutdown_connections 2023-12-19T16:54:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:43.835+0000 7f96e076d640 1 -- 172.21.15.26:0/2541376343 wait complete. 2023-12-19T16:54:43.845 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:54:43.845 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:54:43.846 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:54:43.854 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:43.853+0000 7ff878f58180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:54:43.862 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T16:54:43.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:43.862 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-19T16:54:43.970 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T16:54:43.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:54:43.970 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T16:54:43.987 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:54:43.987 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:54:43.992 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:43.989+0000 7ff878f58180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:54:43.998 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:54:43.998 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:54:43.998 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:54:44.031 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:54:44.031 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:54:44.031 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:54:44.040 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:54:44.041 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:54:44.041 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:54:44.041 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:44.035+0000 7f433f82c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:54:44.051 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:54:44.051 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:54:44.119 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:44.117+0000 7ff878f58180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:54:44.190 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:44.189+0000 7ff878f58180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:54:44.190 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:44.187+0000 7f433f82c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:54:44.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3f3c0640 1 Processor -- start 2023-12-19T16:54:44.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3f3c0640 1 -- start start 2023-12-19T16:54:44.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3f3c0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6e38154220 0x7f6e3815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3f3c0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e3806b3d0 0x7f6e380ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3f3c0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e380cec40 0x7f6e3815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3f3c0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e380cf0d0 con 0x7f6e380cec40 2023-12-19T16:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3f3c0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e3806bc30 con 0x7f6e38154220 2023-12-19T16:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3f3c0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e380cc6e0 con 0x7f6e3806b3d0 2023-12-19T16:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3d95d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e380cec40 0x7f6e3815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3d95d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e380cec40 0x7f6e3815d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51594/0 (socket says 172.21.15.26:51594) 2023-12-19T16:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3d95d640 1 -- 172.21.15.26:0/3502728560 learned_addr learned my addr 172.21.15.26:0/3502728560 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3c95b640 1 --2- 172.21.15.26:0/3502728560 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e3806b3d0 0x7f6e380ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3d15c640 1 --2- 172.21.15.26:0/3502728560 >> v2:172.21.15.160:3300/0 conn(0x7f6e38154220 0x7f6e3815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3d95d640 1 -- 172.21.15.26:0/3502728560 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e3806b3d0 msgr2=0x7f6e380ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3d95d640 1 --2- 172.21.15.26:0/3502728560 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e3806b3d0 0x7f6e380ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3d95d640 1 -- 172.21.15.26:0/3502728560 >> v2:172.21.15.160:3300/0 conn(0x7f6e38154220 msgr2=0x7f6e3815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:44.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3d95d640 1 --2- 172.21.15.26:0/3502728560 >> v2:172.21.15.160:3300/0 conn(0x7f6e38154220 0x7f6e3815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3d95d640 1 -- 172.21.15.26:0/3502728560 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3815afc0 con 0x7f6e380cec40 2023-12-19T16:54:44.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.227+0000 7f6e3c95b640 1 --2- 172.21.15.26:0/3502728560 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e3806b3d0 0x7f6e380ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:54:44.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.231+0000 7f6e3d95d640 1 --2- 172.21.15.26:0/3502728560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380cec40 0x7f6e3815d5f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6e380667b0 tx=0x7f6e3001c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2864356f74502d76 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:44.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e2ffff640 1 -- 172.21.15.26:0/3502728560 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e300a5560 con 0x7f6e380cec40 2023-12-19T16:54:44.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e2ffff640 1 -- 172.21.15.26:0/3502728560 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e300a56c0 con 0x7f6e380cec40 2023-12-19T16:54:44.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e2ffff640 1 -- 172.21.15.26:0/3502728560 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e300a5a50 con 0x7f6e380cec40 2023-12-19T16:54:44.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3502728560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380cec40 msgr2=0x7f6e3815d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:44.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 --2- 172.21.15.26:0/3502728560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380cec40 0x7f6e3815d5f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6e380667b0 tx=0x7f6e3001c6e0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3502728560 shutdown_connections 2023-12-19T16:54:44.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 --2- 172.21.15.26:0/3502728560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e380cec40 0x7f6e3815d5f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 --2- 172.21.15.26:0/3502728560 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e3806b3d0 0x7f6e380ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 --2- 172.21.15.26:0/3502728560 >> v2:172.21.15.160:3300/0 conn(0x7f6e38154220 0x7f6e3815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3502728560 >> 172.21.15.26:0/3502728560 conn(0x7f6e380658e0 msgr2=0x7f6e38065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:44.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3502728560 shutdown_connections 2023-12-19T16:54:44.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3502728560 wait complete. 2023-12-19T16:54:44.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 Processor -- start 2023-12-19T16:54:44.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 -- start start 2023-12-19T16:54:44.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e3806b3d0 0x7f6e380cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e380cec40 0x7f6e380cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e38154220 0x7f6e380b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e380b3060 con 0x7f6e38154220 2023-12-19T16:54:44.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e380b3340 con 0x7f6e3806b3d0 2023-12-19T16:54:44.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.235+0000 7f6e3f3c0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e380b3620 con 0x7f6e380cec40 2023-12-19T16:54:44.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3d95d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e38154220 0x7f6e380b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3d15c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e3806b3d0 0x7f6e380cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3c95b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e380cec40 0x7f6e380cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3d95d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e38154220 0x7f6e380b23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51602/0 (socket says 172.21.15.26:51602) 2023-12-19T16:54:44.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3d95d640 1 -- 172.21.15.26:0/3402669711 learned_addr learned my addr 172.21.15.26:0/3402669711 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:44.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3d95d640 1 -- 172.21.15.26:0/3402669711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e380cec40 msgr2=0x7f6e380cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:44.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3d95d640 1 --2- 172.21.15.26:0/3402669711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e380cec40 0x7f6e380cbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3d95d640 1 -- 172.21.15.26:0/3402669711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e3806b3d0 msgr2=0x7f6e380cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:44.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3d95d640 1 --2- 172.21.15.26:0/3402669711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e3806b3d0 0x7f6e380cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3d95d640 1 -- 172.21.15.26:0/3402669711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e28009420 con 0x7f6e38154220 2023-12-19T16:54:44.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3c95b640 1 --2- 172.21.15.26:0/3402669711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e380cec40 0x7f6e380cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:54:44.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3d95d640 1 --2- 172.21.15.26:0/3402669711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e38154220 0x7f6e380b23c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6e3001ad90 tx=0x7f6e30019480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:44.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e2dffb640 1 -- 172.21.15.26:0/3402669711 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e300a5560 con 0x7f6e38154220 2023-12-19T16:54:44.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e2dffb640 1 -- 172.21.15.26:0/3402669711 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e300b8670 con 0x7f6e38154220 2023-12-19T16:54:44.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e2dffb640 1 -- 172.21.15.26:0/3402669711 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e300b8a20 con 0x7f6e38154220 2023-12-19T16:54:44.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3402669711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e30009b30 con 0x7f6e38154220 2023-12-19T16:54:44.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3402669711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e38154c90 con 0x7f6e38154220 2023-12-19T16:54:44.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3402669711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e380b3790 con 0x7f6e38154220 2023-12-19T16:54:44.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.239+0000 7f6e2dffb640 1 -- 172.21.15.26:0/3402669711 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f6e300b8b80 con 0x7f6e38154220 2023-12-19T16:54:44.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.243+0000 7f6e2dffb640 1 --2- 172.21.15.26:0/3402669711 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6e1c065c10 0x7f6e1c0680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.243+0000 7f6e2dffb640 1 -- 172.21.15.26:0/3402669711 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7571+0+0 (secure 0 0 0) 0x7f6e3011eeb0 con 0x7f6e38154220 2023-12-19T16:54:44.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.243+0000 7f6e3d15c640 1 -- 172.21.15.26:0/3402669711 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6e1c065c10 msgr2=0x7f6e1c0680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:54:44.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.243+0000 7f6e3d15c640 1 --2- 172.21.15.26:0/3402669711 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6e1c065c10 0x7f6e1c0680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:54:44.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.247+0000 7f6e2dffb640 1 -- 172.21.15.26:0/3402669711 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6e300ebfe0 con 0x7f6e38154220 2023-12-19T16:54:44.326 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:44.325+0000 7ff878f58180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:54:44.332 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:44.327+0000 7f433f82c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:54:44.408 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:44.403+0000 7f433f82c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:54:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3402669711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e38154e80 con 0x7f6e38154220 2023-12-19T16:54:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e2dffb640 1 -- 172.21.15.26:0/3402669711 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f6e30009510 con 0x7f6e38154220 2023-12-19T16:54:44.442 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:44.442 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1991431867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1991431867},{"type":"v1","addr":"172.21.15.160:6837","nonce":1991431867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3495925867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3495925867},{"type":"v1","addr":"172.21.15.160:6835","nonce":3495925867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3757604412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3757604412},{"type":"v1","addr":"172.21.15.26:6833","nonce":3757604412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:43:25.866059+0000","modified":"2023-12-19T16:43:28.880638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2660001678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2660001678},{"type":"v1","addr":"172.21.15.26:6835","nonce":2660001678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T16:54:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2023-12-19T16:54:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3402669711 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6e1c065c10 msgr2=0x7f6e1c0680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:54:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e3f3c0640 1 --2- 172.21.15.26:0/3402669711 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6e1c065c10 0x7f6e1c0680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3402669711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e38154220 msgr2=0x7f6e380b23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e3f3c0640 1 --2- 172.21.15.26:0/3402669711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e38154220 0x7f6e380b23c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6e3001ad90 tx=0x7f6e30019480 comp rx=0 tx=0).stop 2023-12-19T16:54:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3402669711 shutdown_connections 2023-12-19T16:54:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e3f3c0640 1 --2- 172.21.15.26:0/3402669711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e38154220 0x7f6e380b23c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e3f3c0640 1 --2- 172.21.15.26:0/3402669711 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6e1c065c10 0x7f6e1c0680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e3f3c0640 1 --2- 172.21.15.26:0/3402669711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e380cec40 0x7f6e380cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e3f3c0640 1 --2- 172.21.15.26:0/3402669711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e3806b3d0 0x7f6e380cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3402669711 >> 172.21.15.26:0/3402669711 conn(0x7f6e380658e0 msgr2=0x7f6e380ae360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3402669711 shutdown_connections 2023-12-19T16:54:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.439+0000 7f6e3f3c0640 1 -- 172.21.15.26:0/3402669711 wait complete. 2023-12-19T16:54:44.464 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:54:44.554 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:44.551+0000 7f433f82c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:54:44.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.603+0000 7f62f0e66640 1 Processor -- start 2023-12-19T16:54:44.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62f0e66640 1 -- start start 2023-12-19T16:54:44.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62f0e66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f62ec155030 0x7f62ec15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62f0e66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62ec15f370 0x7f62ec06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62f0e66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62ec0ce740 0x7f62ec0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62f0e66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec0cf0f0 con 0x7f62ec15f370 2023-12-19T16:54:44.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62f0e66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec06c970 con 0x7f62ec155030 2023-12-19T16:54:44.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62f0e66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec15b1f0 con 0x7f62ec0ce740 2023-12-19T16:54:44.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62e9d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62ec15f370 0x7f62ec06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62e9d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62ec15f370 0x7f62ec06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51618/0 (socket says 172.21.15.26:51618) 2023-12-19T16:54:44.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62e9d9b640 1 -- 172.21.15.26:0/306855140 learned_addr learned my addr 172.21.15.26:0/306855140 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:44.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62ead9d640 1 --2- 172.21.15.26:0/306855140 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62ec0ce740 0x7f62ec0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62ea59c640 1 --2- 172.21.15.26:0/306855140 >> v2:172.21.15.160:3300/0 conn(0x7f62ec155030 0x7f62ec15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62e9d9b640 1 -- 172.21.15.26:0/306855140 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62ec0ce740 msgr2=0x7f62ec0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:44.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62e9d9b640 1 --2- 172.21.15.26:0/306855140 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62ec0ce740 0x7f62ec0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62e9d9b640 1 -- 172.21.15.26:0/306855140 >> v2:172.21.15.160:3300/0 conn(0x7f62ec155030 msgr2=0x7f62ec15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:44.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62e9d9b640 1 --2- 172.21.15.26:0/306855140 >> v2:172.21.15.160:3300/0 conn(0x7f62ec155030 0x7f62ec15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.607+0000 7f62e9d9b640 1 -- 172.21.15.26:0/306855140 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec15b4d0 con 0x7f62ec15f370 2023-12-19T16:54:44.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.611+0000 7f62e9d9b640 1 --2- 172.21.15.26:0/306855140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62ec15f370 0x7f62ec06c080 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0cf4a0 tx=0x7f62e001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c913e82f4f96969 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:44.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.611+0000 7f62e959a640 1 -- 172.21.15.26:0/306855140 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e00aa610 con 0x7f62ec15f370 2023-12-19T16:54:44.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.611+0000 7f62e959a640 1 -- 172.21.15.26:0/306855140 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e00a6040 con 0x7f62ec15f370 2023-12-19T16:54:44.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.611+0000 7f62e959a640 1 -- 172.21.15.26:0/306855140 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e00aa990 con 0x7f62ec15f370 2023-12-19T16:54:44.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 -- 172.21.15.26:0/306855140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62ec15f370 msgr2=0x7f62ec06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:44.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 --2- 172.21.15.26:0/306855140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62ec15f370 0x7f62ec06c080 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0cf4a0 tx=0x7f62e001c4d0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 -- 172.21.15.26:0/306855140 shutdown_connections 2023-12-19T16:54:44.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 --2- 172.21.15.26:0/306855140 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62ec0ce740 0x7f62ec0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 --2- 172.21.15.26:0/306855140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62ec15f370 0x7f62ec06c080 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 --2- 172.21.15.26:0/306855140 >> v2:172.21.15.160:3300/0 conn(0x7f62ec155030 0x7f62ec15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 -- 172.21.15.26:0/306855140 >> 172.21.15.26:0/306855140 conn(0x7f62ec0658e0 msgr2=0x7f62ec065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:44.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 -- 172.21.15.26:0/306855140 shutdown_connections 2023-12-19T16:54:44.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 -- 172.21.15.26:0/306855140 wait complete. 2023-12-19T16:54:44.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 Processor -- start 2023-12-19T16:54:44.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 -- start start 2023-12-19T16:54:44.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62ec0ce740 0x7f62ec1400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62ec155030 0x7f62ec14e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec15f370 0x7f62ec1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0add10 con 0x7f62ec155030 2023-12-19T16:54:44.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec1f4f10 con 0x7f62ec15f370 2023-12-19T16:54:44.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62f0e66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec14ed10 con 0x7f62ec0ce740 2023-12-19T16:54:44.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62e9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62ec155030 0x7f62ec14e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62e9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62ec155030 0x7f62ec14e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51620/0 (socket says 172.21.15.26:51620) 2023-12-19T16:54:44.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62e9d9b640 1 -- 172.21.15.26:0/4071311286 learned_addr learned my addr 172.21.15.26:0/4071311286 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:44.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62ea59c640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62ec0ce740 0x7f62ec1400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.615+0000 7f62ead9d640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec15f370 0x7f62ec1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62e9d9b640 1 -- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62ec0ce740 msgr2=0x7f62ec1400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:44.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62e9d9b640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62ec0ce740 0x7f62ec1400e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62e9d9b640 1 -- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec15f370 msgr2=0x7f62ec1530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:44.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62e9d9b640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec15f370 0x7f62ec1530d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62e9d9b640 1 -- 172.21.15.26:0/4071311286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e00094d0 con 0x7f62ec155030 2023-12-19T16:54:44.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62ead9d640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec15f370 0x7f62ec1530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:54:44.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62ea59c640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62ec0ce740 0x7f62ec1400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:54:44.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62e9d9b640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62ec155030 0x7f62ec14e410 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f62e00ac040 tx=0x7f62e0019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:44.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62daffd640 1 -- 172.21.15.26:0/4071311286 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e00be410 con 0x7f62ec155030 2023-12-19T16:54:44.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62daffd640 1 -- 172.21.15.26:0/4071311286 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e00be570 con 0x7f62ec155030 2023-12-19T16:54:44.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62f0e66640 1 -- 172.21.15.26:0/4071311286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec06c740 con 0x7f62ec155030 2023-12-19T16:54:44.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62daffd640 1 -- 172.21.15.26:0/4071311286 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e00be920 con 0x7f62ec155030 2023-12-19T16:54:44.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62f0e66640 1 -- 172.21.15.26:0/4071311286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec14f2d0 con 0x7f62ec155030 2023-12-19T16:54:44.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62f0e66640 1 -- 172.21.15.26:0/4071311286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec0af2b0 con 0x7f62ec155030 2023-12-19T16:54:44.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.619+0000 7f62daffd640 1 -- 172.21.15.26:0/4071311286 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f62e00bea80 con 0x7f62ec155030 2023-12-19T16:54:44.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.623+0000 7f62daffd640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f62cc065c10 0x7f62cc0680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.623+0000 7f62daffd640 1 -- 172.21.15.26:0/4071311286 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7571+0+0 (secure 0 0 0) 0x7f62e011e450 con 0x7f62ec155030 2023-12-19T16:54:44.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.623+0000 7f62ea59c640 1 -- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f62cc065c10 msgr2=0x7f62cc0680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:54:44.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.623+0000 7f62ea59c640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f62cc065c10 0x7f62cc0680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:54:44.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.627+0000 7f62daffd640 1 -- 172.21.15.26:0/4071311286 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f62ec0af2b0 con 0x7f62ec155030 2023-12-19T16:54:44.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.787+0000 7f62f0e66640 1 -- 172.21.15.26:0/4071311286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62ec0af2b0 con 0x7f62ec155030 2023-12-19T16:54:44.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.791+0000 7f62daffd640 1 -- 172.21.15.26:0/4071311286 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+18917 (secure 0 0 0) 0x7f62ec0af2b0 con 0x7f62ec155030 2023-12-19T16:54:44.793 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:44.793 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":47,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:54:42.470125+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:43:24.052043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T16:43:24.856934+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6833/38098":"2023-12-20T16:43:42.161288+0000","172.21.15.160:6832/38098":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/1318901201":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/713602764":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/810298028":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/1246358820":"2023-12-20T16:43:42.161288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":44}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":30,"length":1},{"begin":41,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":63,"length":1},{"begin":73,"length":1},{"begin":83,"length":1},{"begin":95,"length":1},{"begin":213,"length":8}]}],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":3},{"begin":6,"length":1},{"begin":8,"length":1},{"begin":10,"length":1},{"begin":12,"length":3},{"begin":16,"length":1},{"begin":18,"length":1},{"begin":20,"length":1},{"begin":23,"length":3},{"begin":27,"length":1},{"begin":29,"length":1},{"begin":31,"length":2},{"begin":34,"length":3},{"begin":38,"length":1},{"begin":40,"length":1},{"begin":42,"length":1},{"begin":44,"length":3},{"begin":48,"length":1},{"begin":50,"length":1},{"begin":52,"length":1},{"begin":54,"length":1},{"begin":56,"length":3},{"begin":60,"length":1},{"begin":62,"length":1},{"begin":64,"length":1},{"begin":66,"length":3},{"begin":70,"length":1},{"begin":72,"length":1},{"begin":74,"length":1},{"begin":76,"length":3},{"begin":80,"length":1},{"begin":82,"length":1},{"begin":84,"length":1},{"begin":86,"length":3},{"begin":90,"length":1},{"begin":92,"length":1},{"begin":94,"length":1},{"begin":96,"length":1},{"begin":98,"length":3},{"begin":102,"length":1},{"begin":104,"length":1},{"begin":106,"length":1},{"begin":108,"length":1},{"begin":110,"length":1},{"begin":112,"length":65}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:54:44.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.791+0000 7f62f0e66640 1 -- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f62cc065c10 msgr2=0x7f62cc0680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:54:44.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.791+0000 7f62f0e66640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f62cc065c10 0x7f62cc0680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.791+0000 7f62f0e66640 1 -- 172.21.15.26:0/4071311286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62ec155030 msgr2=0x7f62ec14e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:44.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.791+0000 7f62f0e66640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62ec155030 0x7f62ec14e410 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f62e00ac040 tx=0x7f62e0019270 comp rx=0 tx=0).stop 2023-12-19T16:54:44.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.791+0000 7f62f0e66640 1 -- 172.21.15.26:0/4071311286 shutdown_connections 2023-12-19T16:54:44.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.791+0000 7f62f0e66640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62ec15f370 0x7f62ec1530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.791+0000 7f62f0e66640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62ec155030 0x7f62ec14e410 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.791+0000 7f62f0e66640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f62cc065c10 0x7f62cc0680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.791+0000 7f62f0e66640 1 --2- 172.21.15.26:0/4071311286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62ec0ce740 0x7f62ec1400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.791+0000 7f62f0e66640 1 -- 172.21.15.26:0/4071311286 >> 172.21.15.26:0/4071311286 conn(0x7f62ec0658e0 msgr2=0x7f62ec065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:44.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.791+0000 7f62f0e66640 1 -- 172.21.15.26:0/4071311286 shutdown_connections 2023-12-19T16:54:44.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.791+0000 7f62f0e66640 1 -- 172.21.15.26:0/4071311286 wait complete. 2023-12-19T16:54:44.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:43:24.052043+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:43:24.856934+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 47, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:54:44.816 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:54:44.867 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:44.865+0000 7ff878f58180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:54:44.951 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:44.949+0000 7ff878f58180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:54:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.959+0000 7ff11b7bc640 1 Processor -- start 2023-12-19T16:54:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.959+0000 7ff11b7bc640 1 -- start start 2023-12-19T16:54:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff11b7bc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff1140b88c0 0x7ff1140d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff11b7bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1140b4600 0x7ff1140b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff11b7bc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1140d1310 con 0x7ff11415cf60 2023-12-19T16:54:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff11b7bc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1140b4ff0 con 0x7ff1140b88c0 2023-12-19T16:54:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff11b7bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1140d1570 con 0x7ff1140b4600 2023-12-19T16:54:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff119558640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff11415cf60 0x7ff1140d0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60206/0 (socket says 172.21.15.26:60206) 2023-12-19T16:54:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff119558640 1 -- 172.21.15.26:0/2200589931 learned_addr learned my addr 172.21.15.26:0/2200589931 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff118d57640 1 --2- 172.21.15.26:0/2200589931 >> v2:172.21.15.160:3300/0 conn(0x7ff1140b88c0 0x7ff1140d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff119d59640 1 --2- 172.21.15.26:0/2200589931 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1140b4600 0x7ff1140b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff10bfff640 1 -- 172.21.15.26:0/2200589931 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017695145 0 0) 0x7ff1140d1310 con 0x7ff11415cf60 2023-12-19T16:54:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff10bfff640 1 -- 172.21.15.26:0/2200589931 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff100003530 con 0x7ff11415cf60 2023-12-19T16:54:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff118d57640 1 -- 172.21.15.26:0/2200589931 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1140b4600 msgr2=0x7ff1140b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff118d57640 1 --2- 172.21.15.26:0/2200589931 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1140b4600 0x7ff1140b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff118d57640 1 -- 172.21.15.26:0/2200589931 >> v1:172.21.15.26:6789/0 conn(0x7ff11415cf60 legacy=0x7ff1140d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff118d57640 1 -- 172.21.15.26:0/2200589931 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1140d1850 con 0x7ff1140b88c0 2023-12-19T16:54:44.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.963+0000 7ff119d59640 1 --2- 172.21.15.26:0/2200589931 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1140b4600 0x7ff1140b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:54:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.967+0000 7ff118d57640 1 --2- 172.21.15.26:0/2200589931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1140b88c0 0x7ff1140d2610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff1040097c0 tx=0x7ff10401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6349978f5bfffa7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.967+0000 7ff10bfff640 1 -- 172.21.15.26:0/2200589931 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1040ab600 con 0x7ff1140b88c0 2023-12-19T16:54:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.967+0000 7ff10bfff640 1 -- 172.21.15.26:0/2200589931 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1040a7050 con 0x7ff1140b88c0 2023-12-19T16:54:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.967+0000 7ff10bfff640 1 -- 172.21.15.26:0/2200589931 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1040ab980 con 0x7ff1140b88c0 2023-12-19T16:54:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.967+0000 7ff11b7bc640 1 -- 172.21.15.26:0/2200589931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1140b88c0 msgr2=0x7ff1140d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.967+0000 7ff11b7bc640 1 --2- 172.21.15.26:0/2200589931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1140b88c0 0x7ff1140d2610 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff1040097c0 tx=0x7ff10401c3c0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.967+0000 7ff11b7bc640 1 -- 172.21.15.26:0/2200589931 shutdown_connections 2023-12-19T16:54:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.967+0000 7ff11b7bc640 1 --2- 172.21.15.26:0/2200589931 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1140b4600 0x7ff1140b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.967+0000 7ff11b7bc640 1 --2- 172.21.15.26:0/2200589931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1140b88c0 0x7ff1140d2610 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.967+0000 7ff11b7bc640 1 -- 172.21.15.26:0/2200589931 >> 172.21.15.26:0/2200589931 conn(0x7ff114065ee0 msgr2=0x7ff1140cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.967+0000 7ff11b7bc640 1 -- 172.21.15.26:0/2200589931 shutdown_connections 2023-12-19T16:54:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.967+0000 7ff11b7bc640 1 -- 172.21.15.26:0/2200589931 wait complete. 2023-12-19T16:54:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff11b7bc640 1 Processor -- start 2023-12-19T16:54:44.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff11b7bc640 1 -- start start 2023-12-19T16:54:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff11b7bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1140b4600 0x7ff114159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff11b7bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1140b88c0 0x7ff114147270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff11b7bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff11415cf60 0x7ff114152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff11b7bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1140c8f50 con 0x7ff1140b4600 2023-12-19T16:54:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff11b7bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1141477b0 con 0x7ff11415cf60 2023-12-19T16:54:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff11b7bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1140c85f0 con 0x7ff1140b88c0 2023-12-19T16:54:44.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff119558640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1140b4600 0x7ff114159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff119558640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1140b4600 0x7ff114159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51636/0 (socket says 172.21.15.26:51636) 2023-12-19T16:54:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff119558640 1 -- 172.21.15.26:0/322920585 learned_addr learned my addr 172.21.15.26:0/322920585 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff118d57640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1140b88c0 0x7ff114147270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff119d59640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff11415cf60 0x7ff114152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff119558640 1 -- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1140b88c0 msgr2=0x7ff114147270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff119558640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1140b88c0 0x7ff114147270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff119558640 1 -- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff11415cf60 msgr2=0x7ff114152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff119558640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff11415cf60 0x7ff114152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff119558640 1 -- 172.21.15.26:0/322920585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c009a70 con 0x7ff1140b4600 2023-12-19T16:54:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff118d57640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1140b88c0 0x7ff114147270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:54:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff119558640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1140b4600 0x7ff114159530 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff11000c420 tx=0x7ff11000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff109ffb640 1 -- 172.21.15.26:0/322920585 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110005cf0 con 0x7ff1140b4600 2023-12-19T16:54:44.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff109ffb640 1 -- 172.21.15.26:0/322920585 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff110005e50 con 0x7ff1140b4600 2023-12-19T16:54:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff109ffb640 1 -- 172.21.15.26:0/322920585 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1100072d0 con 0x7ff1140b4600 2023-12-19T16:54:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff11b7bc640 1 -- 172.21.15.26:0/322920585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff104009420 con 0x7ff1140b4600 2023-12-19T16:54:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.971+0000 7ff11b7bc640 1 -- 172.21.15.26:0/322920585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1140c8b70 con 0x7ff1140b4600 2023-12-19T16:54:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.975+0000 7ff11b7bc640 1 -- 172.21.15.26:0/322920585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff114074270 con 0x7ff1140b4600 2023-12-19T16:54:44.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.975+0000 7ff109ffb640 1 -- 172.21.15.26:0/322920585 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7ff110007430 con 0x7ff1140b4600 2023-12-19T16:54:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.975+0000 7ff109ffb640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0f8065c10 0x7ff0f80680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.975+0000 7ff109ffb640 1 -- 172.21.15.26:0/322920585 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7571+0+0 (secure 0 0 0) 0x7ff110082ff0 con 0x7ff1140b4600 2023-12-19T16:54:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.975+0000 7ff118d57640 1 -- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0f8065c10 msgr2=0x7ff0f80680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:54:44.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.975+0000 7ff118d57640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0f8065c10 0x7ff0f80680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:54:44.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:44.979+0000 7ff109ffb640 1 -- 172.21.15.26:0/322920585 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff1100b40a0 con 0x7ff1140b4600 2023-12-19T16:54:45.101 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.097+0000 7ff878f58180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:54:45.137 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:45.135+0000 7f433f82c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:54:45.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.139+0000 7ff11b7bc640 1 -- 172.21.15.26:0/322920585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff1140b7330 con 0x7ff1140b4600 2023-12-19T16:54:45.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.139+0000 7ff109ffb640 1 -- 172.21.15.26:0/322920585 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff1100170a0 con 0x7ff1140b4600 2023-12-19T16:54:45.144 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:54:45.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.143+0000 7ff11b7bc640 1 -- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0f8065c10 msgr2=0x7ff0f80680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:54:45.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.143+0000 7ff11b7bc640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0f8065c10 0x7ff0f80680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.143+0000 7ff11b7bc640 1 -- 172.21.15.26:0/322920585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1140b4600 msgr2=0x7ff114159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.143+0000 7ff11b7bc640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1140b4600 0x7ff114159530 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff11000c420 tx=0x7ff11000c820 comp rx=0 tx=0).stop 2023-12-19T16:54:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.143+0000 7ff11b7bc640 1 -- 172.21.15.26:0/322920585 shutdown_connections 2023-12-19T16:54:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.143+0000 7ff11b7bc640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff11415cf60 0x7ff114152240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.143+0000 7ff11b7bc640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0f8065c10 0x7ff0f80680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.143+0000 7ff11b7bc640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1140b88c0 0x7ff114147270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.143+0000 7ff11b7bc640 1 --2- 172.21.15.26:0/322920585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1140b4600 0x7ff114159530 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.143+0000 7ff11b7bc640 1 -- 172.21.15.26:0/322920585 >> 172.21.15.26:0/322920585 conn(0x7ff114065ee0 msgr2=0x7ff1140d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.143+0000 7ff11b7bc640 1 -- 172.21.15.26:0/322920585 shutdown_connections 2023-12-19T16:54:45.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.143+0000 7ff11b7bc640 1 -- 172.21.15.26:0/322920585 wait complete. 2023-12-19T16:54:45.166 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:54:45.184 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.181+0000 7ff878f58180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:54:45.228 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:45.223+0000 7f433f82c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:54:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030fa95640 1 Processor -- start 2023-12-19T16:54:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030fa95640 1 -- start start 2023-12-19T16:54:45.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030fa95640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0308159140 0x7f03080ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:45.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030fa95640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03080cd190 0x7f030806ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:45.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030fa95640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030806b610 con 0x7f030806b1c0 2023-12-19T16:54:45.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030fa95640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f030806f490 con 0x7f03080cd190 2023-12-19T16:54:45.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030fa95640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0308159520 con 0x7f0308159140 2023-12-19T16:54:45.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030e032640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f030806b1c0 0x7f030815ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60222/0 (socket says 172.21.15.26:60222) 2023-12-19T16:54:45.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030e032640 1 -- 172.21.15.26:0/2036687717 learned_addr learned my addr 172.21.15.26:0/2036687717 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:45.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030d030640 1 --2- 172.21.15.26:0/2036687717 >> v2:172.21.15.26:3300/0 conn(0x7f03080cd190 0x7f030806ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030d831640 1 --2- 172.21.15.26:0/2036687717 >> v2:172.21.15.160:3300/0 conn(0x7f0308159140 0x7f03080ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030c82f640 1 -- 172.21.15.26:0/2036687717 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895395319 0 0) 0x7f03000014a0 con 0x7f030806b1c0 2023-12-19T16:54:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030d030640 1 -- 172.21.15.26:0/2036687717 >> v2:172.21.15.160:3300/0 conn(0x7f0308159140 msgr2=0x7f03080ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030d030640 1 --2- 172.21.15.26:0/2036687717 >> v2:172.21.15.160:3300/0 conn(0x7f0308159140 0x7f03080ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030d030640 1 -- 172.21.15.26:0/2036687717 >> v1:172.21.15.26:6789/0 conn(0x7f030806b1c0 legacy=0x7f030815ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:45.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.311+0000 7f030d030640 1 -- 172.21.15.26:0/2036687717 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03080cd690 con 0x7f03080cd190 2023-12-19T16:54:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.315+0000 7f030d030640 1 --2- 172.21.15.26:0/2036687717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03080cd190 0x7f030806ac80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f03080cf670 tx=0x7f02f801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3604437fcf8c7e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.315+0000 7f030c82f640 1 -- 172.21.15.26:0/2036687717 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f80ab600 con 0x7f03080cd190 2023-12-19T16:54:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.315+0000 7f030c82f640 1 -- 172.21.15.26:0/2036687717 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f80a7050 con 0x7f03080cd190 2023-12-19T16:54:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.315+0000 7f030c82f640 1 -- 172.21.15.26:0/2036687717 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f80ab980 con 0x7f03080cd190 2023-12-19T16:54:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.315+0000 7f030fa95640 1 -- 172.21.15.26:0/2036687717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03080cd190 msgr2=0x7f030806ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:45.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.315+0000 7f030fa95640 1 --2- 172.21.15.26:0/2036687717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03080cd190 0x7f030806ac80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f03080cf670 tx=0x7f02f801c430 comp rx=0 tx=0).stop 2023-12-19T16:54:45.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.315+0000 7f030fa95640 1 -- 172.21.15.26:0/2036687717 shutdown_connections 2023-12-19T16:54:45.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.315+0000 7f030fa95640 1 --2- 172.21.15.26:0/2036687717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03080cd190 0x7f030806ac80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.315+0000 7f030fa95640 1 --2- 172.21.15.26:0/2036687717 >> v2:172.21.15.160:3300/0 conn(0x7f0308159140 0x7f03080ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.315+0000 7f030fa95640 1 -- 172.21.15.26:0/2036687717 >> 172.21.15.26:0/2036687717 conn(0x7f03080ce730 msgr2=0x7f03080ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:45.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.315+0000 7f030fa95640 1 -- 172.21.15.26:0/2036687717 shutdown_connections 2023-12-19T16:54:45.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.315+0000 7f030fa95640 1 -- 172.21.15.26:0/2036687717 wait complete. 2023-12-19T16:54:45.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030fa95640 1 Processor -- start 2023-12-19T16:54:45.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030fa95640 1 -- start start 2023-12-19T16:54:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030fa95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030806b1c0 0x7f0308149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030fa95640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03080cd190 0x7f0308152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030fa95640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0308159140 0x7f03080c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030fa95640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03080c4500 con 0x7f030806b1c0 2023-12-19T16:54:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030fa95640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0308152a80 con 0x7f0308159140 2023-12-19T16:54:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030fa95640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03080c2300 con 0x7f03080cd190 2023-12-19T16:54:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030d831640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030806b1c0 0x7f0308149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030d831640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030806b1c0 0x7f0308149720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51658/0 (socket says 172.21.15.26:51658) 2023-12-19T16:54:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030d831640 1 -- 172.21.15.26:0/2272893331 learned_addr learned my addr 172.21.15.26:0/2272893331 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030e032640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0308159140 0x7f03080c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030d030640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03080cd190 0x7f0308152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030d831640 1 -- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03080cd190 msgr2=0x7f0308152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030d831640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03080cd190 0x7f0308152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030d831640 1 -- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0308159140 msgr2=0x7f03080c2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030d831640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0308159140 0x7f03080c2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030d831640 1 -- 172.21.15.26:0/2272893331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f8009420 con 0x7f030806b1c0 2023-12-19T16:54:45.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030d030640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03080cd190 0x7f0308152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030d831640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030806b1c0 0x7f0308149720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f030400d5f0 tx=0x7f030400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f02fe7fc640 1 -- 172.21.15.26:0/2272893331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03040125f0 con 0x7f030806b1c0 2023-12-19T16:54:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f02fe7fc640 1 -- 172.21.15.26:0/2272893331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0304012750 con 0x7f030806b1c0 2023-12-19T16:54:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030fa95640 1 -- 172.21.15.26:0/2272893331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03081524c0 con 0x7f030806b1c0 2023-12-19T16:54:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f02fe7fc640 1 -- 172.21.15.26:0/2272893331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304012ae0 con 0x7f030806b1c0 2023-12-19T16:54:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.319+0000 7f030fa95640 1 -- 172.21.15.26:0/2272893331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03080c25a0 con 0x7f030806b1c0 2023-12-19T16:54:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.323+0000 7f030fa95640 1 -- 172.21.15.26:0/2272893331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0308074270 con 0x7f030806b1c0 2023-12-19T16:54:45.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.323+0000 7f02fe7fc640 1 -- 172.21.15.26:0/2272893331 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f0304012c40 con 0x7f030806b1c0 2023-12-19T16:54:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.323+0000 7f02fe7fc640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f02ec065c10 0x7f02ec0680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.323+0000 7f02fe7fc640 1 -- 172.21.15.26:0/2272893331 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 7571+0+0 (secure 0 0 0) 0x7f0304083fe0 con 0x7f030806b1c0 2023-12-19T16:54:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.323+0000 7f030d030640 1 -- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f02ec065c10 msgr2=0x7f02ec0680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:54:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.323+0000 7f030d030640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f02ec065c10 0x7f02ec0680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:54:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.327+0000 7f02fe7fc640 1 -- 172.21.15.26:0/2272893331 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0304050fe0 con 0x7f030806b1c0 2023-12-19T16:54:45.389 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:45.387+0000 7f433f82c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:54:45.477 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:45.475+0000 7f433f82c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:54:45.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.487+0000 7f030fa95640 1 -- 172.21.15.26:0/2272893331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f030806db10 con 0x7f030806b1c0 2023-12-19T16:54:45.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.491+0000 7f02fe7fc640 1 -- 172.21.15.26:0/2272893331 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0304054600 con 0x7f030806b1c0 2023-12-19T16:54:45.493 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:54:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.491+0000 7f030fa95640 1 -- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f02ec065c10 msgr2=0x7f02ec0680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:54:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.491+0000 7f030fa95640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f02ec065c10 0x7f02ec0680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.491+0000 7f030fa95640 1 -- 172.21.15.26:0/2272893331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030806b1c0 msgr2=0x7f0308149720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.491+0000 7f030fa95640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030806b1c0 0x7f0308149720 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f030400d5f0 tx=0x7f030400dab0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.491+0000 7f030fa95640 1 -- 172.21.15.26:0/2272893331 shutdown_connections 2023-12-19T16:54:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.491+0000 7f030fa95640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0308159140 0x7f03080c2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.491+0000 7f030fa95640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f02ec065c10 0x7f02ec0680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.491+0000 7f030fa95640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03080cd190 0x7f0308152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.491+0000 7f030fa95640 1 --2- 172.21.15.26:0/2272893331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f030806b1c0 0x7f0308149720 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.491+0000 7f030fa95640 1 -- 172.21.15.26:0/2272893331 >> 172.21.15.26:0/2272893331 conn(0x7f03080ce730 msgr2=0x7f03080af430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:45.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.491+0000 7f030fa95640 1 -- 172.21.15.26:0/2272893331 shutdown_connections 2023-12-19T16:54:45.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.491+0000 7f030fa95640 1 -- 172.21.15.26:0/2272893331 wait complete. 2023-12-19T16:54:45.498 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.497+0000 7ff878f58180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:54:45.516 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:54:45.560 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.557+0000 7ff878f58180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:54:45.639 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.637+0000 7ff8017cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:54:45.639 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.637+0000 7ff8017cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:54:45.640 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.637+0000 7ff8017cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:54:45.640 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.637+0000 7ff8017cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:54:45.640 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.637+0000 7ff8017cc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:54:45.640 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.637+0000 7ff8047d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:54:45.640 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.637+0000 7ff8047d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:54:45.640 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.637+0000 7ff8047d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:54:45.640 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.637+0000 7ff8047d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:54:45.640 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:54:45.637+0000 7ff8047d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:54:45.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04b5ee5640 1 Processor -- start 2023-12-19T16:54:45.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04b5ee5640 1 -- start start 2023-12-19T16:54:45.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04b5ee5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f04b015ccf0 0x7f04b00d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04b5ee5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b015e200 con 0x7f04b015d0d0 2023-12-19T16:54:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04b5ee5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b00d0aa0 con 0x7f04b00b4600 2023-12-19T16:54:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04b5ee5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b00d1380 con 0x7f04b015ccf0 2023-12-19T16:54:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04aeffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f04b015d0d0 0x7f04b00b8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60234/0 (socket says 172.21.15.26:60234) 2023-12-19T16:54:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04aeffd640 1 -- 172.21.15.26:0/438090923 learned_addr learned my addr 172.21.15.26:0/438090923 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:45.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04af7fe640 1 --2- 172.21.15.26:0/438090923 >> v2:172.21.15.160:3300/0 conn(0x7f04b015ccf0 0x7f04b00d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:45.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04ae7fc640 1 -- 172.21.15.26:0/438090923 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190365467 0 0) 0x7f04b00d0aa0 con 0x7f04b00b4600 2023-12-19T16:54:45.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04ae7fc640 1 -- 172.21.15.26:0/438090923 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0498003530 con 0x7f04b00b4600 2023-12-19T16:54:45.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04ae7fc640 1 -- 172.21.15.26:0/438090923 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204276759 0 0) 0x7f04b015e200 con 0x7f04b015d0d0 2023-12-19T16:54:45.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04af7fe640 1 -- 172.21.15.26:0/438090923 >> v1:172.21.15.160:6789/0 conn(0x7f04b00b4600 legacy=0x7f04b00b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:45.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04af7fe640 1 -- 172.21.15.26:0/438090923 >> v1:172.21.15.26:6789/0 conn(0x7f04b015d0d0 legacy=0x7f04b00b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:45.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.667+0000 7f04af7fe640 1 -- 172.21.15.26:0/438090923 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b015d680 con 0x7f04b015ccf0 2023-12-19T16:54:45.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.671+0000 7f04af7fe640 1 --2- 172.21.15.26:0/438090923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04b015ccf0 0x7f04b00d0e40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f04a00097c0 tx=0x7f04a001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=780d4b81817ee89 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:45.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.671+0000 7f04ae7fc640 1 -- 172.21.15.26:0/438090923 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a00ab650 con 0x7f04b015ccf0 2023-12-19T16:54:45.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.671+0000 7f04ae7fc640 1 -- 172.21.15.26:0/438090923 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a00ab7b0 con 0x7f04b015ccf0 2023-12-19T16:54:45.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.671+0000 7f04ae7fc640 1 -- 172.21.15.26:0/438090923 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a00abb40 con 0x7f04b015ccf0 2023-12-19T16:54:45.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.671+0000 7f04b5ee5640 1 -- 172.21.15.26:0/438090923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04b015ccf0 msgr2=0x7f04b00d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:45.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.671+0000 7f04b5ee5640 1 --2- 172.21.15.26:0/438090923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04b015ccf0 0x7f04b00d0e40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f04a00097c0 tx=0x7f04a001c450 comp rx=0 tx=0).stop 2023-12-19T16:54:45.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.671+0000 7f04b5ee5640 1 -- 172.21.15.26:0/438090923 shutdown_connections 2023-12-19T16:54:45.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.671+0000 7f04b5ee5640 1 --2- 172.21.15.26:0/438090923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04b015ccf0 0x7f04b00d0e40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.671+0000 7f04b5ee5640 1 -- 172.21.15.26:0/438090923 >> 172.21.15.26:0/438090923 conn(0x7f04b0065ee0 msgr2=0x7f04b00cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:45.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.671+0000 7f04b5ee5640 1 -- 172.21.15.26:0/438090923 shutdown_connections 2023-12-19T16:54:45.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.671+0000 7f04b5ee5640 1 -- 172.21.15.26:0/438090923 wait complete. 2023-12-19T16:54:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04b5ee5640 1 Processor -- start 2023-12-19T16:54:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04b5ee5640 1 -- start start 2023-12-19T16:54:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04b5ee5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04b00b4600 0x7f04b00cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04b5ee5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04b015ccf0 0x7f04b015a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04b5ee5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b015d0d0 0x7f04b015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04b5ee5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b01576c0 con 0x7f04b015d0d0 2023-12-19T16:54:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04b5ee5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b0156100 con 0x7f04b015ccf0 2023-12-19T16:54:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04b5ee5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b0156d20 con 0x7f04b00b4600 2023-12-19T16:54:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b015d0d0 0x7f04b015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:45.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04aeffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04b015ccf0 0x7f04b015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04affff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b015d0d0 0x7f04b015ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51666/0 (socket says 172.21.15.26:51666) 2023-12-19T16:54:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04affff640 1 -- 172.21.15.26:0/3152680535 learned_addr learned my addr 172.21.15.26:0/3152680535 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04af7fe640 1 --2- 172.21.15.26:0/3152680535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04b00b4600 0x7f04b00cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04af7fe640 1 -- 172.21.15.26:0/3152680535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04b015ccf0 msgr2=0x7f04b015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04af7fe640 1 --2- 172.21.15.26:0/3152680535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04b015ccf0 0x7f04b015a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04af7fe640 1 -- 172.21.15.26:0/3152680535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b015d0d0 msgr2=0x7f04b015ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04af7fe640 1 --2- 172.21.15.26:0/3152680535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b015d0d0 0x7f04b015ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.675+0000 7f04af7fe640 1 -- 172.21.15.26:0/3152680535 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04a0009420 con 0x7f04b00b4600 2023-12-19T16:54:45.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.679+0000 7f04af7fe640 1 --2- 172.21.15.26:0/3152680535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04b00b4600 0x7f04b00cc4b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f04a00ac000 tx=0x7f04a001ac90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.679+0000 7f0497fff640 1 -- 172.21.15.26:0/3152680535 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0004220 con 0x7f04b00b4600 2023-12-19T16:54:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.679+0000 7f0497fff640 1 -- 172.21.15.26:0/3152680535 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a0004380 con 0x7f04b00b4600 2023-12-19T16:54:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.679+0000 7f0497fff640 1 -- 172.21.15.26:0/3152680535 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a0003820 con 0x7f04b00b4600 2023-12-19T16:54:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.679+0000 7f04b5ee5640 1 -- 172.21.15.26:0/3152680535 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b0157000 con 0x7f04b00b4600 2023-12-19T16:54:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.679+0000 7f04b5ee5640 1 -- 172.21.15.26:0/3152680535 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b00ccb90 con 0x7f04b00b4600 2023-12-19T16:54:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.679+0000 7f04b5ee5640 1 -- 172.21.15.26:0/3152680535 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b0074270 con 0x7f04b00b4600 2023-12-19T16:54:45.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.679+0000 7f0497fff640 1 -- 172.21.15.26:0/3152680535 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 15) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f04a0003980 con 0x7f04b00b4600 2023-12-19T16:54:45.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.679+0000 7f0497fff640 1 -- 172.21.15.26:0/3152680535 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6984+0+0 (secure 0 0 0) 0x7f04a011dd70 con 0x7f04b00b4600 2023-12-19T16:54:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.683+0000 7f0497fff640 1 -- 172.21.15.26:0/3152680535 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f04a00eb7e0 con 0x7f04b00b4600 2023-12-19T16:54:45.828 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:45.823+0000 7f433f82c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:54:45.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.843+0000 7f04b5ee5640 1 -- 172.21.15.26:0/3152680535 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f04b0160620 con 0x7f04b00b4600 2023-12-19T16:54:45.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.843+0000 7f0497fff640 1 -- 172.21.15.26:0/3152680535 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7f04a00c40a0 con 0x7f04b00b4600 2023-12-19T16:54:45.847 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:54:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.847+0000 7f04b5ee5640 1 -- 172.21.15.26:0/3152680535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04b00b4600 msgr2=0x7f04b00cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.847+0000 7f04b5ee5640 1 --2- 172.21.15.26:0/3152680535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04b00b4600 0x7f04b00cc4b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f04a00ac000 tx=0x7f04a001ac90 comp rx=0 tx=0).stop 2023-12-19T16:54:45.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.847+0000 7f04b5ee5640 1 -- 172.21.15.26:0/3152680535 shutdown_connections 2023-12-19T16:54:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.847+0000 7f04b5ee5640 1 --2- 172.21.15.26:0/3152680535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04b015d0d0 0x7f04b015ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.847+0000 7f04b5ee5640 1 --2- 172.21.15.26:0/3152680535 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f04b015ccf0 0x7f04b015a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.847+0000 7f04b5ee5640 1 --2- 172.21.15.26:0/3152680535 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f04b00b4600 0x7f04b00cc4b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.847+0000 7f04b5ee5640 1 -- 172.21.15.26:0/3152680535 >> 172.21.15.26:0/3152680535 conn(0x7f04b0065ee0 msgr2=0x7f04b0167bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.847+0000 7f04b5ee5640 1 -- 172.21.15.26:0/3152680535 shutdown_connections 2023-12-19T16:54:45.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:45.847+0000 7f04b5ee5640 1 -- 172.21.15.26:0/3152680535 wait complete. 2023-12-19T16:54:45.869 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:54:45.895 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:54:45.891+0000 7f433f82c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:54:46.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.015+0000 7f6b780d1640 1 Processor -- start 2023-12-19T16:54:46.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.015+0000 7f6b780d1640 1 -- start start 2023-12-19T16:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b780d1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6b700cb1d0 0x7f6b700cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b780d1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b7006bee0 0x7f6b7006c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b780d1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7006d4b0 con 0x7f6b70164c90 2023-12-19T16:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b780d1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b700cc030 con 0x7f6b700cb1d0 2023-12-19T16:54:46.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b780d1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700cbaf0 con 0x7f6b7006bee0 2023-12-19T16:54:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b7666e640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f6b70164c90 0x7f6b70165120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:37318/0 (socket says 172.21.15.26:37318) 2023-12-19T16:54:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b7566c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b7006bee0 0x7f6b7006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b7666e640 1 -- 172.21.15.26:0/3030705583 learned_addr learned my addr 172.21.15.26:0/3030705583 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b75e6d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6b700cb1d0 0x7f6b700cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b75e6d640 1 -- 172.21.15.26:0/3030705583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b7006bee0 msgr2=0x7f6b7006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b75e6d640 1 --2- 172.21.15.26:0/3030705583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b7006bee0 0x7f6b7006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b75e6d640 1 -- 172.21.15.26:0/3030705583 >> v1:172.21.15.160:6789/0 conn(0x7f6b70164c90 legacy=0x7f6b70165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b75e6d640 1 -- 172.21.15.26:0/3030705583 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7006cc10 con 0x7f6b700cb1d0 2023-12-19T16:54:46.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.019+0000 7f6b74e6b640 1 -- 172.21.15.26:0/3030705583 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769611729 0 0) 0x7f6b7006d4b0 con 0x7f6b70164c90 2023-12-19T16:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b75e6d640 1 --2- 172.21.15.26:0/3030705583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b700cb1d0 0x7f6b700cb5b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097c0 tx=0x7f6b6c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d606e2a7e3fce897 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b74e6b640 1 -- 172.21.15.26:0/3030705583 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c0ac620 con 0x7f6b700cb1d0 2023-12-19T16:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b74e6b640 1 -- 172.21.15.26:0/3030705583 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c0ac780 con 0x7f6b700cb1d0 2023-12-19T16:54:46.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b74e6b640 1 -- 172.21.15.26:0/3030705583 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c0acb10 con 0x7f6b700cb1d0 2023-12-19T16:54:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b780d1640 1 -- 172.21.15.26:0/3030705583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b700cb1d0 msgr2=0x7f6b700cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b780d1640 1 --2- 172.21.15.26:0/3030705583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b700cb1d0 0x7f6b700cb5b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097c0 tx=0x7f6b6c01c3c0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b780d1640 1 -- 172.21.15.26:0/3030705583 shutdown_connections 2023-12-19T16:54:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b780d1640 1 --2- 172.21.15.26:0/3030705583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b7006bee0 0x7f6b7006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b780d1640 1 --2- 172.21.15.26:0/3030705583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b700cb1d0 0x7f6b700cb5b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b780d1640 1 -- 172.21.15.26:0/3030705583 >> 172.21.15.26:0/3030705583 conn(0x7f6b700658e0 msgr2=0x7f6b70065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b780d1640 1 -- 172.21.15.26:0/3030705583 shutdown_connections 2023-12-19T16:54:46.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b780d1640 1 -- 172.21.15.26:0/3030705583 wait complete. 2023-12-19T16:54:46.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b780d1640 1 Processor -- start 2023-12-19T16:54:46.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.023+0000 7f6b780d1640 1 -- start start 2023-12-19T16:54:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b780d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7006bee0 0x7f6b70149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b780d1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b700cb1d0 0x7f6b700afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b780d1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b70164c90 0x7f6b700ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b780d1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700af8f0 con 0x7f6b7006bee0 2023-12-19T16:54:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b780d1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7014f980 con 0x7f6b70164c90 2023-12-19T16:54:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b780d1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700af230 con 0x7f6b700cb1d0 2023-12-19T16:54:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b75e6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7006bee0 0x7f6b70149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b75e6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7006bee0 0x7f6b70149ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54054/0 (socket says 172.21.15.26:54054) 2023-12-19T16:54:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b75e6d640 1 -- 172.21.15.26:0/1070691822 learned_addr learned my addr 172.21.15.26:0/1070691822 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b7666e640 1 --2- 172.21.15.26:0/1070691822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b70164c90 0x7f6b700ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b7566c640 1 --2- 172.21.15.26:0/1070691822 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b700cb1d0 0x7f6b700afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b75e6d640 1 -- 172.21.15.26:0/1070691822 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b700cb1d0 msgr2=0x7f6b700afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b75e6d640 1 --2- 172.21.15.26:0/1070691822 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b700cb1d0 0x7f6b700afd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b75e6d640 1 -- 172.21.15.26:0/1070691822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b70164c90 msgr2=0x7f6b700ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b75e6d640 1 --2- 172.21.15.26:0/1070691822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b70164c90 0x7f6b700ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b75e6d640 1 -- 172.21.15.26:0/1070691822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c009420 con 0x7f6b7006bee0 2023-12-19T16:54:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b75e6d640 1 --2- 172.21.15.26:0/1070691822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7006bee0 0x7f6b70149ad0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0af040 tx=0x7f6b6c01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b667fc640 1 -- 172.21.15.26:0/1070691822 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c0b8450 con 0x7f6b7006bee0 2023-12-19T16:54:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b667fc640 1 -- 172.21.15.26:0/1070691822 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c0b85d0 con 0x7f6b7006bee0 2023-12-19T16:54:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b667fc640 1 -- 172.21.15.26:0/1070691822 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c0b8980 con 0x7f6b7006bee0 2023-12-19T16:54:46.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b780d1640 1 -- 172.21.15.26:0/1070691822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b700af510 con 0x7f6b7006bee0 2023-12-19T16:54:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b780d1640 1 -- 172.21.15.26:0/1070691822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7014f050 con 0x7f6b7006bee0 2023-12-19T16:54:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b780d1640 1 -- 172.21.15.26:0/1070691822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b700af090 con 0x7f6b7006bee0 2023-12-19T16:54:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.027+0000 7f6b667fc640 1 -- 172.21.15.26:0/1070691822 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f6b6c0aa070 con 0x7f6b7006bee0 2023-12-19T16:54:46.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.031+0000 7f6b667fc640 1 -- 172.21.15.26:0/1070691822 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6984+0+0 (secure 0 0 0) 0x7f6b6c11d5f0 con 0x7f6b7006bee0 2023-12-19T16:54:46.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.035+0000 7f6b667fc640 1 -- 172.21.15.26:0/1070691822 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6b700af090 con 0x7f6b7006bee0 2023-12-19T16:54:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.223+0000 7f6b780d1640 1 -- 172.21.15.26:0/1070691822 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b700af090 con 0x7f6b7006bee0 2023-12-19T16:54:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.223+0000 7f6b667fc640 1 -- 172.21.15.26:0/1070691822 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f6b700af090 con 0x7f6b7006bee0 2023-12-19T16:54:46.228 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:46.228 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1991431867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1991431867},{"type":"v1","addr":"172.21.15.160:6837","nonce":1991431867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3495925867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3495925867},{"type":"v1","addr":"172.21.15.160:6835","nonce":3495925867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3757604412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3757604412},{"type":"v1","addr":"172.21.15.26:6833","nonce":3757604412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:43:25.866059+0000","modified":"2023-12-19T16:43:28.880638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2660001678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2660001678},{"type":"v1","addr":"172.21.15.26:6835","nonce":2660001678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T16:54:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2023-12-19T16:54:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.227+0000 7f6b780d1640 1 -- 172.21.15.26:0/1070691822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7006bee0 msgr2=0x7f6b70149ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.227+0000 7f6b780d1640 1 --2- 172.21.15.26:0/1070691822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7006bee0 0x7f6b70149ad0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0af040 tx=0x7f6b6c01ac00 comp rx=0 tx=0).stop 2023-12-19T16:54:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.227+0000 7f6b780d1640 1 -- 172.21.15.26:0/1070691822 shutdown_connections 2023-12-19T16:54:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.227+0000 7f6b780d1640 1 --2- 172.21.15.26:0/1070691822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b70164c90 0x7f6b700ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.227+0000 7f6b780d1640 1 --2- 172.21.15.26:0/1070691822 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b700cb1d0 0x7f6b700afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.227+0000 7f6b780d1640 1 --2- 172.21.15.26:0/1070691822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7006bee0 0x7f6b70149ad0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.227+0000 7f6b780d1640 1 -- 172.21.15.26:0/1070691822 >> 172.21.15.26:0/1070691822 conn(0x7f6b700658e0 msgr2=0x7f6b701650e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.227+0000 7f6b780d1640 1 -- 172.21.15.26:0/1070691822 shutdown_connections 2023-12-19T16:54:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.227+0000 7f6b780d1640 1 -- 172.21.15.26:0/1070691822 wait complete. 2023-12-19T16:54:46.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:54:46.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.431+0000 7feef8073640 1 Processor -- start 2023-12-19T16:54:46.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.431+0000 7feef8073640 1 -- start start 2023-12-19T16:54:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef8073640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7feef00cb1d0 0x7feef00cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef8073640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feef006bee0 0x7feef006c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef8073640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef006d4b0 con 0x7feef0164c90 2023-12-19T16:54:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef8073640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef00cc030 con 0x7feef006bee0 2023-12-19T16:54:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef8073640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef00cbaf0 con 0x7feef00cb1d0 2023-12-19T16:54:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef560e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feef006bee0 0x7feef006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef560e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feef006bee0 0x7feef006c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54062/0 (socket says 172.21.15.26:54062) 2023-12-19T16:54:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef560e640 1 -- 172.21.15.26:0/393954939 learned_addr learned my addr 172.21.15.26:0/393954939 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef6610640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feef0164c90 0x7feef0165100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46854/0 (socket says 172.21.15.26:46854) 2023-12-19T16:54:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef560e640 1 -- 172.21.15.26:0/393954939 >> v2:172.21.15.160:3300/0 conn(0x7feef00cb1d0 msgr2=0x7feef00cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef560e640 1 --2- 172.21.15.26:0/393954939 >> v2:172.21.15.160:3300/0 conn(0x7feef00cb1d0 0x7feef00cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef560e640 1 -- 172.21.15.26:0/393954939 >> v1:172.21.15.26:6789/0 conn(0x7feef0164c90 legacy=0x7feef0165100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef560e640 1 -- 172.21.15.26:0/393954939 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef006cc10 con 0x7feef006bee0 2023-12-19T16:54:46.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.435+0000 7feef4e0d640 1 -- 172.21.15.26:0/393954939 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658306047 0 0) 0x7feef006d4b0 con 0x7feef0164c90 2023-12-19T16:54:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef560e640 1 --2- 172.21.15.26:0/393954939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feef006bee0 0x7feef006c340 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7feef00ccb20 tx=0x7feee001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748f55b90a4e3248 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef4e0d640 1 -- 172.21.15.26:0/393954939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee00ab640 con 0x7feef006bee0 2023-12-19T16:54:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef4e0d640 1 -- 172.21.15.26:0/393954939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee00a7070 con 0x7feef006bee0 2023-12-19T16:54:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef4e0d640 1 -- 172.21.15.26:0/393954939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee00ab9c0 con 0x7feef006bee0 2023-12-19T16:54:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef8073640 1 -- 172.21.15.26:0/393954939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feef006bee0 msgr2=0x7feef006c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef8073640 1 --2- 172.21.15.26:0/393954939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feef006bee0 0x7feef006c340 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7feef00ccb20 tx=0x7feee001c440 comp rx=0 tx=0).stop 2023-12-19T16:54:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef8073640 1 -- 172.21.15.26:0/393954939 shutdown_connections 2023-12-19T16:54:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef8073640 1 --2- 172.21.15.26:0/393954939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feef006bee0 0x7feef006c340 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef8073640 1 --2- 172.21.15.26:0/393954939 >> v2:172.21.15.160:3300/0 conn(0x7feef00cb1d0 0x7feef00cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef8073640 1 -- 172.21.15.26:0/393954939 >> 172.21.15.26:0/393954939 conn(0x7feef00658e0 msgr2=0x7feef0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef8073640 1 -- 172.21.15.26:0/393954939 shutdown_connections 2023-12-19T16:54:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef8073640 1 -- 172.21.15.26:0/393954939 wait complete. 2023-12-19T16:54:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef8073640 1 Processor -- start 2023-12-19T16:54:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.439+0000 7feef8073640 1 -- start start 2023-12-19T16:54:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef8073640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feef006bee0 0x7feef0151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef8073640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feef00cb1d0 0x7feef00ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef8073640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feef0164c90 0x7feef01f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef8073640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef00ae7a0 con 0x7feef006bee0 2023-12-19T16:54:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef8073640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef00ceef0 con 0x7feef0164c90 2023-12-19T16:54:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef8073640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef01fae60 con 0x7feef00cb1d0 2023-12-19T16:54:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef5e0f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feef006bee0 0x7feef0151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef5e0f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feef006bee0 0x7feef0151e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54066/0 (socket says 172.21.15.26:54066) 2023-12-19T16:54:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef5e0f640 1 -- 172.21.15.26:0/685704892 learned_addr learned my addr 172.21.15.26:0/685704892 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:46.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef6610640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feef0164c90 0x7feef01f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef560e640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feef00cb1d0 0x7feef00ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef5e0f640 1 -- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feef00cb1d0 msgr2=0x7feef00ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef5e0f640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feef00cb1d0 0x7feef00ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef5e0f640 1 -- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feef0164c90 msgr2=0x7feef01f9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef5e0f640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feef0164c90 0x7feef01f9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef5e0f640 1 -- 172.21.15.26:0/685704892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee00094d0 con 0x7feef006bee0 2023-12-19T16:54:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef5e0f640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feef006bee0 0x7feef0151e00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7feeec00b550 tx=0x7feeec00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feee67fc640 1 -- 172.21.15.26:0/685704892 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec010870 con 0x7feef006bee0 2023-12-19T16:54:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feee67fc640 1 -- 172.21.15.26:0/685704892 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec0109f0 con 0x7feef006bee0 2023-12-19T16:54:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feee67fc640 1 -- 172.21.15.26:0/685704892 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec010da0 con 0x7feef006bee0 2023-12-19T16:54:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef8073640 1 -- 172.21.15.26:0/685704892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef01fab80 con 0x7feef006bee0 2023-12-19T16:54:46.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef8073640 1 -- 172.21.15.26:0/685704892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef00ce7d0 con 0x7feef006bee0 2023-12-19T16:54:46.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feef8073640 1 -- 172.21.15.26:0/685704892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef00b0fa0 con 0x7feef006bee0 2023-12-19T16:54:46.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.443+0000 7feee67fc640 1 -- 172.21.15.26:0/685704892 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 84939+0+0 (secure 0 0 0) 0x7feeec00f3c0 con 0x7feef006bee0 2023-12-19T16:54:46.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.447+0000 7feee67fc640 1 -- 172.21.15.26:0/685704892 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6984+0+0 (secure 0 0 0) 0x7feeec083260 con 0x7feef006bee0 2023-12-19T16:54:46.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.451+0000 7feee67fc640 1 -- 172.21.15.26:0/685704892 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feeec050a80 con 0x7feef006bee0 2023-12-19T16:54:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.587+0000 7feee67fc640 1 -- 172.21.15.26:0/685704892 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 16) v1 ==== 85066+0+0 (secure 0 0 0) 0x7feeec050530 con 0x7feef006bee0 2023-12-19T16:54:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.587+0000 7feee67fc640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feed401c9c0 0x7feed406d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.591+0000 7feef560e640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feed401c9c0 0x7feed406d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.591+0000 7feef560e640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feed401c9c0 0x7feed406d950 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feee00af000 tx=0x7feee00a7000 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:46.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.619+0000 7feef8073640 1 -- 172.21.15.26:0/685704892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feef0074270 con 0x7feef006bee0 2023-12-19T16:54:46.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.623+0000 7feee67fc640 1 -- 172.21.15.26:0/685704892 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18111 (secure 0 0 0) 0x7feeec01e090 con 0x7feef006bee0 2023-12-19T16:54:46.629 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:46.629 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:54:46.567144+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:43:24.052043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T16:43:24.856934+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/2949585094":"2023-12-20T16:54:45.562800+0000","172.21.15.160:0/3215987599":"2023-12-20T16:54:45.562800+0000","172.21.15.160:0/3268674744":"2023-12-20T16:54:45.562800+0000","172.21.15.160:0/3711839001":"2023-12-20T16:54:45.562800+0000","172.21.15.160:0/632897543":"2023-12-20T16:54:45.562800+0000","172.21.15.160:0/2891388829":"2023-12-20T16:54:45.562800+0000","172.21.15.160:6833/38098":"2023-12-20T16:43:42.161288+0000","172.21.15.160:6832/38098":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/3358653677":"2023-12-20T16:54:45.562800+0000","172.21.15.160:0/1318901201":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/713602764":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/810298028":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/1246358820":"2023-12-20T16:43:42.161288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":105,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:54:46.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.623+0000 7feef8073640 1 -- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feed401c9c0 msgr2=0x7feed406d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.623+0000 7feef8073640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feed401c9c0 0x7feed406d950 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feee00af000 tx=0x7feee00a7000 comp rx=0 tx=0).stop 2023-12-19T16:54:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.623+0000 7feef8073640 1 -- 172.21.15.26:0/685704892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feef006bee0 msgr2=0x7feef0151e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.623+0000 7feef8073640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feef006bee0 0x7feef0151e00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7feeec00b550 tx=0x7feeec00ba10 comp rx=0 tx=0).stop 2023-12-19T16:54:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.623+0000 7feef8073640 1 -- 172.21.15.26:0/685704892 shutdown_connections 2023-12-19T16:54:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.623+0000 7feef8073640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feef0164c90 0x7feef01f9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.623+0000 7feef8073640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feed401c9c0 0x7feed406d950 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.623+0000 7feef8073640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feef00cb1d0 0x7feef00ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.623+0000 7feef8073640 1 --2- 172.21.15.26:0/685704892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feef006bee0 0x7feef0151e00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.623+0000 7feef8073640 1 -- 172.21.15.26:0/685704892 >> 172.21.15.26:0/685704892 conn(0x7feef00658e0 msgr2=0x7feef00b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.623+0000 7feef8073640 1 -- 172.21.15.26:0/685704892 shutdown_connections 2023-12-19T16:54:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.623+0000 7feef8073640 1 -- 172.21.15.26:0/685704892 wait complete. 2023-12-19T16:54:46.646 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T16:54:46.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:54:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.787+0000 7f91bd5a6640 1 Processor -- start 2023-12-19T16:54:46.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.787+0000 7f91bd5a6640 1 -- start start 2023-12-19T16:54:46.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.787+0000 7f91bd5a6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91b815c7f0 0x7f91b806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.787+0000 7f91bd5a6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f91b80b30d0 0x7f91b80b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.787+0000 7f91bd5a6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b80b3a20 con 0x7f91b80adb80 2023-12-19T16:54:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.787+0000 7f91bd5a6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b806d770 con 0x7f91b815c7f0 2023-12-19T16:54:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.787+0000 7f91bd5a6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b806ced0 con 0x7f91b80b30d0 2023-12-19T16:54:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.787+0000 7f91b6ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91b815c7f0 0x7f91b806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.787+0000 7f91b6ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91b815c7f0 0x7f91b806c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54076/0 (socket says 172.21.15.26:54076) 2023-12-19T16:54:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.787+0000 7f91b6ffd640 1 -- 172.21.15.26:0/1306504235 learned_addr learned my addr 172.21.15.26:0/1306504235 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.787+0000 7f91b77fe640 1 --2- 172.21.15.26:0/1306504235 >> v2:172.21.15.160:3300/0 conn(0x7f91b80b30d0 0x7f91b80b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.787+0000 7f91b6ffd640 1 -- 172.21.15.26:0/1306504235 >> v2:172.21.15.160:3300/0 conn(0x7f91b80b30d0 msgr2=0x7f91b80b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.787+0000 7f91b6ffd640 1 --2- 172.21.15.26:0/1306504235 >> v2:172.21.15.160:3300/0 conn(0x7f91b80b30d0 0x7f91b80b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91b6ffd640 1 -- 172.21.15.26:0/1306504235 >> v1:172.21.15.160:6789/0 conn(0x7f91b80adb80 legacy=0x7f91b80cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91b6ffd640 1 -- 172.21.15.26:0/1306504235 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b806d1b0 con 0x7f91b815c7f0 2023-12-19T16:54:46.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91b77fe640 1 --2- 172.21.15.26:0/1306504235 >> v2:172.21.15.160:3300/0 conn(0x7f91b80b30d0 0x7f91b80b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:46.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91b6ffd640 1 --2- 172.21.15.26:0/1306504235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b815c7f0 0x7f91b806c7f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f91b80cc420 tx=0x7f91a801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65cb3faa6e83fce server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:46.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91b67fc640 1 -- 172.21.15.26:0/1306504235 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a80ab600 con 0x7f91b815c7f0 2023-12-19T16:54:46.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91b67fc640 1 -- 172.21.15.26:0/1306504235 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a80a7050 con 0x7f91b815c7f0 2023-12-19T16:54:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91b67fc640 1 -- 172.21.15.26:0/1306504235 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a80ab980 con 0x7f91b815c7f0 2023-12-19T16:54:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91bd5a6640 1 -- 172.21.15.26:0/1306504235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b815c7f0 msgr2=0x7f91b806c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91bd5a6640 1 --2- 172.21.15.26:0/1306504235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b815c7f0 0x7f91b806c7f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f91b80cc420 tx=0x7f91a801c500 comp rx=0 tx=0).stop 2023-12-19T16:54:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91bd5a6640 1 -- 172.21.15.26:0/1306504235 shutdown_connections 2023-12-19T16:54:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91bd5a6640 1 --2- 172.21.15.26:0/1306504235 >> v2:172.21.15.160:3300/0 conn(0x7f91b80b30d0 0x7f91b80b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91bd5a6640 1 --2- 172.21.15.26:0/1306504235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b815c7f0 0x7f91b806c7f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91bd5a6640 1 -- 172.21.15.26:0/1306504235 >> 172.21.15.26:0/1306504235 conn(0x7f91b80658e0 msgr2=0x7f91b8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91bd5a6640 1 -- 172.21.15.26:0/1306504235 shutdown_connections 2023-12-19T16:54:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.791+0000 7f91bd5a6640 1 -- 172.21.15.26:0/1306504235 wait complete. 2023-12-19T16:54:46.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91bd5a6640 1 Processor -- start 2023-12-19T16:54:46.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91bd5a6640 1 -- start start 2023-12-19T16:54:46.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91bd5a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91b80adb80 0x7f91b814a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91bd5a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91b80b30d0 0x7f91b813c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91bd5a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b815c7f0 0x7f91b806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91bd5a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b81592d0 con 0x7f91b815c7f0 2023-12-19T16:54:46.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91bd5a6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b814ad80 con 0x7f91b80adb80 2023-12-19T16:54:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91bd5a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b813c6a0 con 0x7f91b80b30d0 2023-12-19T16:54:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b815c7f0 0x7f91b806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b815c7f0 0x7f91b806fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54084/0 (socket says 172.21.15.26:54084) 2023-12-19T16:54:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91b77fe640 1 -- 172.21.15.26:0/2367108040 learned_addr learned my addr 172.21.15.26:0/2367108040 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91b6ffd640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91b80adb80 0x7f91b814a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91affff640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91b80b30d0 0x7f91b813c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91b77fe640 1 -- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91b80b30d0 msgr2=0x7f91b813c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91b77fe640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91b80b30d0 0x7f91b813c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91b77fe640 1 -- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91b80adb80 msgr2=0x7f91b814a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91b77fe640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91b80adb80 0x7f91b814a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91b77fe640 1 -- 172.21.15.26:0/2367108040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a80094d0 con 0x7f91b815c7f0 2023-12-19T16:54:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91b77fe640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b815c7f0 0x7f91b806fca0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f91b0007f20 tx=0x7f91b000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:46.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91af7fe640 1 -- 172.21.15.26:0/2367108040 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b000dee0 con 0x7f91b815c7f0 2023-12-19T16:54:46.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91af7fe640 1 -- 172.21.15.26:0/2367108040 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b000f040 con 0x7f91b815c7f0 2023-12-19T16:54:46.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91af7fe640 1 -- 172.21.15.26:0/2367108040 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0003de0 con 0x7f91b815c7f0 2023-12-19T16:54:46.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91bd5a6640 1 -- 172.21.15.26:0/2367108040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b814ce40 con 0x7f91b815c7f0 2023-12-19T16:54:46.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.795+0000 7f91bd5a6640 1 -- 172.21.15.26:0/2367108040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b814d400 con 0x7f91b815c7f0 2023-12-19T16:54:46.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.799+0000 7f91bd5a6640 1 -- 172.21.15.26:0/2367108040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b8159410 con 0x7f91b815c7f0 2023-12-19T16:54:46.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.799+0000 7f91af7fe640 1 -- 172.21.15.26:0/2367108040 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f91b0004030 con 0x7f91b815c7f0 2023-12-19T16:54:46.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.799+0000 7f91af7fe640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91980658b0 0x7f9198067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:46.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.799+0000 7f91af7fe640 1 -- 172.21.15.26:0/2367108040 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7044+0+0 (secure 0 0 0) 0x7f91b0083620 con 0x7f91b815c7f0 2023-12-19T16:54:46.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.799+0000 7f91b6ffd640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91980658b0 0x7f9198067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:46.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.803+0000 7f91b6ffd640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91980658b0 0x7f9198067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f91a80ad000 tx=0x7f91a8008040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:46.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:46.803+0000 7f91af7fe640 1 -- 172.21.15.26:0/2367108040 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f91b8159410 con 0x7f91b815c7f0 2023-12-19T16:54:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91bd5a6640 1 -- 172.21.15.26:0/2367108040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91b8159410 con 0x7f91b815c7f0 2023-12-19T16:54:47.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91af7fe640 1 -- 172.21.15.26:0/2367108040 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f91b8159410 con 0x7f91b815c7f0 2023-12-19T16:54:47.006 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:47.006 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1991431867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1991431867},{"type":"v1","addr":"172.21.15.160:6837","nonce":1991431867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3495925867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3495925867},{"type":"v1","addr":"172.21.15.160:6835","nonce":3495925867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3757604412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3757604412},{"type":"v1","addr":"172.21.15.26:6833","nonce":3757604412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:43:25.866059+0000","modified":"2023-12-19T16:43:28.880638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2660001678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2660001678},{"type":"v1","addr":"172.21.15.26:6835","nonce":2660001678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T16:54:47.007 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2023-12-19T16:54:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91bd5a6640 1 -- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91980658b0 msgr2=0x7f9198067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91bd5a6640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91980658b0 0x7f9198067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f91a80ad000 tx=0x7f91a8008040 comp rx=0 tx=0).stop 2023-12-19T16:54:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91bd5a6640 1 -- 172.21.15.26:0/2367108040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b815c7f0 msgr2=0x7f91b806fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91bd5a6640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b815c7f0 0x7f91b806fca0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f91b0007f20 tx=0x7f91b000d560 comp rx=0 tx=0).stop 2023-12-19T16:54:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91bd5a6640 1 -- 172.21.15.26:0/2367108040 shutdown_connections 2023-12-19T16:54:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91bd5a6640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91b815c7f0 0x7f91b806fca0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91bd5a6640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91980658b0 0x7f9198067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91bd5a6640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91b80b30d0 0x7f91b813c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91bd5a6640 1 --2- 172.21.15.26:0/2367108040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91b80adb80 0x7f91b814a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91bd5a6640 1 -- 172.21.15.26:0/2367108040 >> 172.21.15.26:0/2367108040 conn(0x7f91b80658e0 msgr2=0x7f91b80b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:47.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91bd5a6640 1 -- 172.21.15.26:0/2367108040 shutdown_connections 2023-12-19T16:54:47.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.003+0000 7f91bd5a6640 1 -- 172.21.15.26:0/2367108040 wait complete. 2023-12-19T16:54:47.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:54:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.171+0000 7fb55ce8c640 1 Processor -- start 2023-12-19T16:54:47.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.171+0000 7fb55ce8c640 1 -- start start 2023-12-19T16:54:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb55ce8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb55815f370 0x7fb55815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb55ce8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55806c970 con 0x7fb558155030 2023-12-19T16:54:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb55ce8c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55815b1f0 con 0x7fb5580ce740 2023-12-19T16:54:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb55ce8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55806c790 con 0x7fb55815f370 2023-12-19T16:54:47.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb55659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb558155030 0x7fb55806c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46868/0 (socket says 172.21.15.26:46868) 2023-12-19T16:54:47.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb55659c640 1 -- 172.21.15.26:0/938019411 learned_addr learned my addr 172.21.15.26:0/938019411 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:47.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452597106 0 0) 0x7fb55806c970 con 0x7fb558155030 2023-12-19T16:54:47.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb544003530 con 0x7fb558155030 2023-12-19T16:54:47.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3037552027 0 0) 0x7fb544003530 con 0x7fb558155030 2023-12-19T16:54:47.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb55815f370 msgr2=0x7fb55815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:47.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb54ffff640 1 --2- 172.21.15.26:0/938019411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb55815f370 0x7fb55815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 --2- 172.21.15.26:0/938019411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb55815f370 0x7fb55815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 >> v1:172.21.15.160:6789/0 conn(0x7fb5580ce740 legacy=0x7fb5580ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55815b4d0 con 0x7fb558155030 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fb548001680 con 0x7fb558155030 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 >> v1:172.21.15.26:6789/0 conn(0x7fb558155030 legacy=0x7fb55806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 --2- 172.21.15.26:0/938019411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5440039b0 0x7fb544003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 --2- 172.21.15.26:0/938019411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb5440047d0 0x7fb544004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 --2- 172.21.15.26:0/938019411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5440051e0 0x7fb54400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55806c790 con 0x7fb5440051e0 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54400c110 con 0x7fb5440039b0 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54400c2d0 con 0x7fb5440047d0 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb54ffff640 1 --2- 172.21.15.26:0/938019411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb55815f370 0x7fb55815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb556d9d640 1 --2- 172.21.15.26:0/938019411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5440051e0 0x7fb54400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb55659c640 1 --2- 172.21.15.26:0/938019411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5440039b0 0x7fb544003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb556d9d640 1 -- 172.21.15.26:0/938019411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb5440047d0 msgr2=0x7fb544004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb556d9d640 1 --2- 172.21.15.26:0/938019411 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb5440047d0 0x7fb544004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb556d9d640 1 -- 172.21.15.26:0/938019411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5440039b0 msgr2=0x7fb544003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb556d9d640 1 --2- 172.21.15.26:0/938019411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5440039b0 0x7fb544003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb556d9d640 1 -- 172.21.15.26:0/938019411 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb544003530 con 0x7fb5440051e0 2023-12-19T16:54:47.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.175+0000 7fb55659c640 1 --2- 172.21.15.26:0/938019411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5440039b0 0x7fb544003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:54:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.179+0000 7fb556d9d640 1 --2- 172.21.15.26:0/938019411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5440051e0 0x7fb54400b990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb5500044b0 tx=0x7fb55001c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3fae8f76bc4195 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.179+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5500a9680 con 0x7fb5440051e0 2023-12-19T16:54:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.179+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5500a9800 con 0x7fb5440051e0 2023-12-19T16:54:47.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.179+0000 7fb555d9b640 1 -- 172.21.15.26:0/938019411 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5500a9bb0 con 0x7fb5440051e0 2023-12-19T16:54:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.179+0000 7fb55ce8c640 1 -- 172.21.15.26:0/938019411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5440051e0 msgr2=0x7fb54400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.179+0000 7fb55ce8c640 1 --2- 172.21.15.26:0/938019411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5440051e0 0x7fb54400b990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb5500044b0 tx=0x7fb55001c680 comp rx=0 tx=0).stop 2023-12-19T16:54:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.179+0000 7fb556d9d640 1 -- 172.21.15.26:0/938019411 reap_dead start 2023-12-19T16:54:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.179+0000 7fb55ce8c640 1 -- 172.21.15.26:0/938019411 shutdown_connections 2023-12-19T16:54:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.179+0000 7fb55ce8c640 1 -- 172.21.15.26:0/938019411 >> 172.21.15.26:0/938019411 conn(0x7fb5580658e0 msgr2=0x7fb558065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.179+0000 7fb55ce8c640 1 -- 172.21.15.26:0/938019411 shutdown_connections 2023-12-19T16:54:47.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.179+0000 7fb55ce8c640 1 -- 172.21.15.26:0/938019411 wait complete. 2023-12-19T16:54:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb55ce8c640 1 Processor -- start 2023-12-19T16:54:47.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb55ce8c640 1 -- start start 2023-12-19T16:54:47.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb55ce8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5440051e0 0x7fb5581433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb55ce8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb558158790 0x7fb5580c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb55ce8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb55806b920 0x7fb55806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb55ce8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55806c2e0 con 0x7fb558158790 2023-12-19T16:54:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb55ce8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb558140a40 con 0x7fb5440051e0 2023-12-19T16:54:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb55ce8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5581438e0 con 0x7fb55806b920 2023-12-19T16:54:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb54ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb558158790 0x7fb5580c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb54ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb558158790 0x7fb5580c37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54098/0 (socket says 172.21.15.26:54098) 2023-12-19T16:54:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb54ffff640 1 -- 172.21.15.26:0/3785239841 learned_addr learned my addr 172.21.15.26:0/3785239841 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb55659c640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5440051e0 0x7fb5581433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb556d9d640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb55806b920 0x7fb55806bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb54ffff640 1 -- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb55806b920 msgr2=0x7fb55806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb54ffff640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb55806b920 0x7fb55806bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb54ffff640 1 -- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5440051e0 msgr2=0x7fb5581433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb54ffff640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5440051e0 0x7fb5581433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb54ffff640 1 -- 172.21.15.26:0/3785239841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb558158ce0 con 0x7fb558158790 2023-12-19T16:54:47.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb54ffff640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb558158790 0x7fb5580c37b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb54000c470 tx=0x7fb54000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:47.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb54effd640 1 -- 172.21.15.26:0/3785239841 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540004230 con 0x7fb558158790 2023-12-19T16:54:47.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb54effd640 1 -- 172.21.15.26:0/3785239841 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb540004390 con 0x7fb558158790 2023-12-19T16:54:47.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb54effd640 1 -- 172.21.15.26:0/3785239841 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540003860 con 0x7fb558158790 2023-12-19T16:54:47.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb55ce8c640 1 -- 172.21.15.26:0/3785239841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb558158fc0 con 0x7fb558158790 2023-12-19T16:54:47.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb55ce8c640 1 -- 172.21.15.26:0/3785239841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb558159580 con 0x7fb558158790 2023-12-19T16:54:47.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.183+0000 7fb55ce8c640 1 -- 172.21.15.26:0/3785239841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb558074270 con 0x7fb558158790 2023-12-19T16:54:47.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.187+0000 7fb54effd640 1 -- 172.21.15.26:0/3785239841 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fb5400039c0 con 0x7fb558158790 2023-12-19T16:54:47.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.187+0000 7fb54effd640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb5380658b0 0x7fb538067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.187+0000 7fb54effd640 1 -- 172.21.15.26:0/3785239841 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7044+0+0 (secure 0 0 0) 0x7fb540082eb0 con 0x7fb558158790 2023-12-19T16:54:47.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.187+0000 7fb55659c640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb5380658b0 0x7fb538067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.187+0000 7fb55659c640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb5380658b0 0x7fb538067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb548004c50 tx=0x7fb548009040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:47.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.191+0000 7fb54effd640 1 -- 172.21.15.26:0/3785239841 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb540050720 con 0x7fb558158790 2023-12-19T16:54:47.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.387+0000 7fb55ce8c640 1 -- 172.21.15.26:0/3785239841 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5580ccc30 con 0x7fb558158790 2023-12-19T16:54:47.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.387+0000 7fb54effd640 1 -- 172.21.15.26:0/3785239841 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4715 (secure 0 0 0) 0x7fb540053d40 con 0x7fb558158790 2023-12-19T16:54:47.391 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:47.391 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1991431867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1991431867},{"type":"v1","addr":"172.21.15.160:6837","nonce":1991431867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3495925867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3495925867},{"type":"v1","addr":"172.21.15.160:6835","nonce":3495925867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3757604412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3757604412},{"type":"v1","addr":"172.21.15.26:6833","nonce":3757604412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:43:25.866059+0000","modified":"2023-12-19T16:43:28.880638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5106},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5106":{"gid":5106,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2660001678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2660001678},{"type":"v1","addr":"172.21.15.26:6835","nonce":2660001678}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T16:54:47.391 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 17 2023-12-19T16:54:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.387+0000 7fb55ce8c640 1 -- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb5380658b0 msgr2=0x7fb538067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.387+0000 7fb55ce8c640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb5380658b0 0x7fb538067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb548004c50 tx=0x7fb548009040 comp rx=0 tx=0).stop 2023-12-19T16:54:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.387+0000 7fb55ce8c640 1 -- 172.21.15.26:0/3785239841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb558158790 msgr2=0x7fb5580c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.387+0000 7fb55ce8c640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb558158790 0x7fb5580c37b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb54000c470 tx=0x7fb54000c930 comp rx=0 tx=0).stop 2023-12-19T16:54:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.387+0000 7fb55ce8c640 1 -- 172.21.15.26:0/3785239841 shutdown_connections 2023-12-19T16:54:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.391+0000 7fb55ce8c640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb5380658b0 0x7fb538067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.391+0000 7fb55ce8c640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb55806b920 0x7fb55806bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.391+0000 7fb55ce8c640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb558158790 0x7fb5580c37b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.391+0000 7fb55ce8c640 1 --2- 172.21.15.26:0/3785239841 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5440051e0 0x7fb5581433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.391+0000 7fb55ce8c640 1 -- 172.21.15.26:0/3785239841 >> 172.21.15.26:0/3785239841 conn(0x7fb5580658e0 msgr2=0x7fb55806cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.391+0000 7fb55ce8c640 1 -- 172.21.15.26:0/3785239841 shutdown_connections 2023-12-19T16:54:47.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.391+0000 7fb55ce8c640 1 -- 172.21.15.26:0/3785239841 wait complete. 2023-12-19T16:54:47.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:54:47.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.551+0000 7fe52adf0640 1 Processor -- start 2023-12-19T16:54:47.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.551+0000 7fe52adf0640 1 -- start start 2023-12-19T16:54:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.551+0000 7fe52adf0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe524155030 0x7fe52415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.551+0000 7fe52adf0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe52415f370 0x7fe52406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.551+0000 7fe52adf0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe52406c970 con 0x7fe5240ce740 2023-12-19T16:54:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.551+0000 7fe52adf0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe52415b1f0 con 0x7fe524155030 2023-12-19T16:54:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.551+0000 7fe52adf0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe52415b4d0 con 0x7fe52415f370 2023-12-19T16:54:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.555+0000 7fe52938d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe5240ce740 0x7fe5240ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46874/0 (socket says 172.21.15.26:46874) 2023-12-19T16:54:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.555+0000 7fe52938d640 1 -- 172.21.15.26:0/2135017820 learned_addr learned my addr 172.21.15.26:0/2135017820 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.555+0000 7fe528b8c640 1 --2- 172.21.15.26:0/2135017820 >> v2:172.21.15.160:3300/0 conn(0x7fe524155030 0x7fe52415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.555+0000 7fe523fff640 1 --2- 172.21.15.26:0/2135017820 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe52415f370 0x7fe52406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.555+0000 7fe5237fe640 1 -- 172.21.15.26:0/2135017820 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221806119 0 0) 0x7fe52406c970 con 0x7fe5240ce740 2023-12-19T16:54:47.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.555+0000 7fe5237fe640 1 -- 172.21.15.26:0/2135017820 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe514003530 con 0x7fe5240ce740 2023-12-19T16:54:47.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.555+0000 7fe528b8c640 1 -- 172.21.15.26:0/2135017820 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe52415f370 msgr2=0x7fe52406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:47.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.555+0000 7fe528b8c640 1 --2- 172.21.15.26:0/2135017820 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe52415f370 0x7fe52406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.555+0000 7fe528b8c640 1 -- 172.21.15.26:0/2135017820 >> v1:172.21.15.26:6789/0 conn(0x7fe5240ce740 legacy=0x7fe5240ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.555+0000 7fe528b8c640 1 -- 172.21.15.26:0/2135017820 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52415b7b0 con 0x7fe524155030 2023-12-19T16:54:47.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.555+0000 7fe523fff640 1 --2- 172.21.15.26:0/2135017820 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe52415f370 0x7fe52406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:54:47.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe528b8c640 1 --2- 172.21.15.26:0/2135017820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe524155030 0x7fe52415bcd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5180097c0 tx=0x7fe51801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c8fd14e39aebfad server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:47.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe5237fe640 1 -- 172.21.15.26:0/2135017820 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5180a6040 con 0x7fe524155030 2023-12-19T16:54:47.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe5237fe640 1 -- 172.21.15.26:0/2135017820 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5180aa600 con 0x7fe524155030 2023-12-19T16:54:47.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe5237fe640 1 -- 172.21.15.26:0/2135017820 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5180aa990 con 0x7fe524155030 2023-12-19T16:54:47.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe52adf0640 1 -- 172.21.15.26:0/2135017820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe524155030 msgr2=0x7fe52415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe52adf0640 1 --2- 172.21.15.26:0/2135017820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe524155030 0x7fe52415bcd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe5180097c0 tx=0x7fe51801c480 comp rx=0 tx=0).stop 2023-12-19T16:54:47.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe52adf0640 1 -- 172.21.15.26:0/2135017820 shutdown_connections 2023-12-19T16:54:47.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe52adf0640 1 --2- 172.21.15.26:0/2135017820 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe52415f370 0x7fe52406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe52adf0640 1 --2- 172.21.15.26:0/2135017820 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe524155030 0x7fe52415bcd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe52adf0640 1 -- 172.21.15.26:0/2135017820 >> 172.21.15.26:0/2135017820 conn(0x7fe5240658e0 msgr2=0x7fe524065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:47.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe52adf0640 1 -- 172.21.15.26:0/2135017820 shutdown_connections 2023-12-19T16:54:47.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe52adf0640 1 -- 172.21.15.26:0/2135017820 wait complete. 2023-12-19T16:54:47.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe52adf0640 1 Processor -- start 2023-12-19T16:54:47.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.559+0000 7fe52adf0640 1 -- start start 2023-12-19T16:54:47.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52adf0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5240ce740 0x7fe52413ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52adf0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe524155030 0x7fe5240acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52adf0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52415f370 0x7fe5240b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52adf0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52414e780 con 0x7fe52415f370 2023-12-19T16:54:47.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52adf0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52414dec0 con 0x7fe524155030 2023-12-19T16:54:47.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52adf0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5240b2f20 con 0x7fe5240ce740 2023-12-19T16:54:47.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52938d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52415f370 0x7fe5240b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52938d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52415f370 0x7fe5240b2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54112/0 (socket says 172.21.15.26:54112) 2023-12-19T16:54:47.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52938d640 1 -- 172.21.15.26:0/2662032595 learned_addr learned my addr 172.21.15.26:0/2662032595 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:47.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe528b8c640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5240ce740 0x7fe52413ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe523fff640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe524155030 0x7fe5240acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52938d640 1 -- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5240ce740 msgr2=0x7fe52413ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52938d640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5240ce740 0x7fe52413ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52938d640 1 -- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe524155030 msgr2=0x7fe5240acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52938d640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe524155030 0x7fe5240acab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52938d640 1 -- 172.21.15.26:0/2662032595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe510009420 con 0x7fe52415f370 2023-12-19T16:54:47.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52938d640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52415f370 0x7fe5240b2310 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00caa0 tx=0x7fe51c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:47.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe5217fa640 1 -- 172.21.15.26:0/2662032595 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c00ac00 con 0x7fe52415f370 2023-12-19T16:54:47.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe5217fa640 1 -- 172.21.15.26:0/2662032595 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c00ad60 con 0x7fe52415f370 2023-12-19T16:54:47.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe5217fa640 1 -- 172.21.15.26:0/2662032595 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c012620 con 0x7fe52415f370 2023-12-19T16:54:47.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52adf0640 1 -- 172.21.15.26:0/2662032595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe518009420 con 0x7fe52415f370 2023-12-19T16:54:47.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52adf0640 1 -- 172.21.15.26:0/2662032595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe52406c740 con 0x7fe52415f370 2023-12-19T16:54:47.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.563+0000 7fe52adf0640 1 -- 172.21.15.26:0/2662032595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe52414e8c0 con 0x7fe52415f370 2023-12-19T16:54:47.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.567+0000 7fe5217fa640 1 -- 172.21.15.26:0/2662032595 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fe51c007070 con 0x7fe52415f370 2023-12-19T16:54:47.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.567+0000 7fe5217fa640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe5080658b0 0x7fe508067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.567+0000 7fe5217fa640 1 -- 172.21.15.26:0/2662032595 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7044+0+0 (secure 0 0 0) 0x7fe51c083270 con 0x7fe52415f370 2023-12-19T16:54:47.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.567+0000 7fe528b8c640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe5080658b0 0x7fe508067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.567+0000 7fe528b8c640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe5080658b0 0x7fe508067d70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe5180ae000 tx=0x7fe5180b5040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:47.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.571+0000 7fe5217fa640 1 -- 172.21.15.26:0/2662032595 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe52414e8c0 con 0x7fe52415f370 2023-12-19T16:54:47.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.731+0000 7fe52adf0640 1 -- 172.21.15.26:0/2662032595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe52414e8c0 con 0x7fe52415f370 2023-12-19T16:54:47.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.735+0000 7fe5217fa640 1 -- 172.21.15.26:0/2662032595 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+18111 (secure 0 0 0) 0x7fe52414e8c0 con 0x7fe52415f370 2023-12-19T16:54:47.738 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:47.738 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:54:46.567144+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:43:24.052043+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T16:43:24.856934+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/2949585094":"2023-12-20T16:54:45.562800+0000","172.21.15.160:0/3215987599":"2023-12-20T16:54:45.562800+0000","172.21.15.160:0/3268674744":"2023-12-20T16:54:45.562800+0000","172.21.15.160:0/3711839001":"2023-12-20T16:54:45.562800+0000","172.21.15.160:0/632897543":"2023-12-20T16:54:45.562800+0000","172.21.15.160:0/2891388829":"2023-12-20T16:54:45.562800+0000","172.21.15.160:6833/38098":"2023-12-20T16:43:42.161288+0000","172.21.15.160:6832/38098":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/3358653677":"2023-12-20T16:54:45.562800+0000","172.21.15.160:0/1318901201":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/713602764":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/810298028":"2023-12-20T16:43:42.161288+0000","172.21.15.160:0/1246358820":"2023-12-20T16:43:42.161288+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":105,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:54:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.735+0000 7fe52adf0640 1 -- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe5080658b0 msgr2=0x7fe508067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.735+0000 7fe52adf0640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe5080658b0 0x7fe508067d70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe5180ae000 tx=0x7fe5180b5040 comp rx=0 tx=0).stop 2023-12-19T16:54:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.735+0000 7fe52adf0640 1 -- 172.21.15.26:0/2662032595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52415f370 msgr2=0x7fe5240b2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.735+0000 7fe52adf0640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52415f370 0x7fe5240b2310 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00caa0 tx=0x7fe51c00cf60 comp rx=0 tx=0).stop 2023-12-19T16:54:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.735+0000 7fe52adf0640 1 -- 172.21.15.26:0/2662032595 shutdown_connections 2023-12-19T16:54:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.735+0000 7fe52adf0640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe52415f370 0x7fe5240b2310 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.735+0000 7fe52adf0640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe524155030 0x7fe5240acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.735+0000 7fe52adf0640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe5080658b0 0x7fe508067d70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.735+0000 7fe52adf0640 1 --2- 172.21.15.26:0/2662032595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5240ce740 0x7fe52413ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.735+0000 7fe52adf0640 1 -- 172.21.15.26:0/2662032595 >> 172.21.15.26:0/2662032595 conn(0x7fe5240658e0 msgr2=0x7fe5240cd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.735+0000 7fe52adf0640 1 -- 172.21.15.26:0/2662032595 shutdown_connections 2023-12-19T16:54:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.735+0000 7fe52adf0640 1 -- 172.21.15.26:0/2662032595 wait complete. 2023-12-19T16:54:47.759 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T16:54:47.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.899+0000 7fed9356d640 1 Processor -- start 2023-12-19T16:54:47.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.899+0000 7fed9356d640 1 -- start start 2023-12-19T16:54:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.903+0000 7fed9356d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed8c159a10 0x7fed8c0b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.903+0000 7fed9356d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fed8c070ab0 0x7fed8c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.903+0000 7fed9356d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8c159df0 con 0x7fed8c0b3fe0 2023-12-19T16:54:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.903+0000 7fed9356d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8c0b2850 con 0x7fed8c159a10 2023-12-19T16:54:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.903+0000 7fed9356d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8c0cf4f0 con 0x7fed8c070ab0 2023-12-19T16:54:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.903+0000 7fed91309640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed8c159a10 0x7fed8c0b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.903+0000 7fed91309640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed8c159a10 0x7fed8c0b3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54120/0 (socket says 172.21.15.26:54120) 2023-12-19T16:54:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.903+0000 7fed91309640 1 -- 172.21.15.26:0/2652681176 learned_addr learned my addr 172.21.15.26:0/2652681176 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.903+0000 7fed91b0a640 1 --2- 172.21.15.26:0/2652681176 >> v2:172.21.15.160:3300/0 conn(0x7fed8c070ab0 0x7fed8c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.903+0000 7fed91309640 1 -- 172.21.15.26:0/2652681176 >> v2:172.21.15.160:3300/0 conn(0x7fed8c070ab0 msgr2=0x7fed8c0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.903+0000 7fed91309640 1 --2- 172.21.15.26:0/2652681176 >> v2:172.21.15.160:3300/0 conn(0x7fed8c070ab0 0x7fed8c0cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.903+0000 7fed91309640 1 -- 172.21.15.26:0/2652681176 >> v1:172.21.15.26:6789/0 conn(0x7fed8c0b3fe0 legacy=0x7fed8c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.903+0000 7fed91309640 1 -- 172.21.15.26:0/2652681176 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c0b2a60 con 0x7fed8c159a10 2023-12-19T16:54:47.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed91309640 1 --2- 172.21.15.26:0/2652681176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed8c159a10 0x7fed8c0b3540 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fed8c071100 tx=0x7fed8801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b1d96c69abb3b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed83fff640 1 -- 172.21.15.26:0/2652681176 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed880ab640 con 0x7fed8c159a10 2023-12-19T16:54:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed83fff640 1 -- 172.21.15.26:0/2652681176 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed880a7070 con 0x7fed8c159a10 2023-12-19T16:54:47.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed83fff640 1 -- 172.21.15.26:0/2652681176 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed880ab9c0 con 0x7fed8c159a10 2023-12-19T16:54:47.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 -- 172.21.15.26:0/2652681176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed8c159a10 msgr2=0x7fed8c0b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 --2- 172.21.15.26:0/2652681176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed8c159a10 0x7fed8c0b3540 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fed8c071100 tx=0x7fed8801c440 comp rx=0 tx=0).stop 2023-12-19T16:54:47.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 -- 172.21.15.26:0/2652681176 shutdown_connections 2023-12-19T16:54:47.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 --2- 172.21.15.26:0/2652681176 >> v2:172.21.15.160:3300/0 conn(0x7fed8c070ab0 0x7fed8c0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 --2- 172.21.15.26:0/2652681176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed8c159a10 0x7fed8c0b3540 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 -- 172.21.15.26:0/2652681176 >> 172.21.15.26:0/2652681176 conn(0x7fed8c06a320 msgr2=0x7fed8c0b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:47.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 -- 172.21.15.26:0/2652681176 shutdown_connections 2023-12-19T16:54:47.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 -- 172.21.15.26:0/2652681176 wait complete. 2023-12-19T16:54:47.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 Processor -- start 2023-12-19T16:54:47.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 -- start start 2023-12-19T16:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed8c070ab0 0x7fed8c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed8c0b3fe0 0x7fed8c14d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed8c159a10 0x7fed8c149a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c15a520 con 0x7fed8c070ab0 2023-12-19T16:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0adc00 con 0x7fed8c0b3fe0 2023-12-19T16:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.907+0000 7fed9356d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0adee0 con 0x7fed8c159a10 2023-12-19T16:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed91309640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed8c070ab0 0x7fed8c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed91309640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed8c070ab0 0x7fed8c159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54136/0 (socket says 172.21.15.26:54136) 2023-12-19T16:54:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed91309640 1 -- 172.21.15.26:0/2348713889 learned_addr learned my addr 172.21.15.26:0/2348713889 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed90b08640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed8c0b3fe0 0x7fed8c14d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed91b0a640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed8c159a10 0x7fed8c149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed91309640 1 -- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed8c159a10 msgr2=0x7fed8c149a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed91309640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed8c159a10 0x7fed8c149a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed91309640 1 -- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed8c0b3fe0 msgr2=0x7fed8c14d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed91309640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed8c0b3fe0 0x7fed8c14d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed91309640 1 -- 172.21.15.26:0/2348713889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed880094d0 con 0x7fed8c070ab0 2023-12-19T16:54:47.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed91b0a640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed8c159a10 0x7fed8c149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:47.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed91309640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed8c070ab0 0x7fed8c159fe0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fed88019050 tx=0x7fed8801ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:47.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed81ffb640 1 -- 172.21.15.26:0/2348713889 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed880be3f0 con 0x7fed8c070ab0 2023-12-19T16:54:47.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed81ffb640 1 -- 172.21.15.26:0/2348713889 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed880be550 con 0x7fed8c070ab0 2023-12-19T16:54:47.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed81ffb640 1 -- 172.21.15.26:0/2348713889 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed880be900 con 0x7fed8c070ab0 2023-12-19T16:54:47.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed9356d640 1 -- 172.21.15.26:0/2348713889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8c0ae1c0 con 0x7fed8c070ab0 2023-12-19T16:54:47.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed9356d640 1 -- 172.21.15.26:0/2348713889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8c14a540 con 0x7fed8c070ab0 2023-12-19T16:54:47.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed9356d640 1 -- 172.21.15.26:0/2348713889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8c14a730 con 0x7fed8c070ab0 2023-12-19T16:54:47.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.911+0000 7fed81ffb640 1 -- 172.21.15.26:0/2348713889 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fed880c7440 con 0x7fed8c070ab0 2023-12-19T16:54:47.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.915+0000 7fed81ffb640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed700658e0 0x7fed70067da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:47.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.915+0000 7fed81ffb640 1 -- 172.21.15.26:0/2348713889 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 7044+0+0 (secure 0 0 0) 0x7fed8811dc60 con 0x7fed8c070ab0 2023-12-19T16:54:47.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.915+0000 7fed90b08640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed700658e0 0x7fed70067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:47.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.915+0000 7fed90b08640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed700658e0 0x7fed70067da0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed8c14bcc0 tx=0x7fed7c009040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:47.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:47.919+0000 7fed81ffb640 1 -- 172.21.15.26:0/2348713889 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fed880eb400 con 0x7fed8c070ab0 2023-12-19T16:54:48.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.111+0000 7fed9356d640 1 -- 172.21.15.26:0/2348713889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fed8c074270 con 0x7fed8c070ab0 2023-12-19T16:54:48.115 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T16:54:48.111+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T16:54:48.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.123+0000 7fed81ffb640 1 -- 172.21.15.26:0/2348713889 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fed880eea20 con 0x7fed8c070ab0 2023-12-19T16:54:48.128 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T16:54:48.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.127+0000 7fed9356d640 1 -- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed700658e0 msgr2=0x7fed70067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:48.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.127+0000 7fed9356d640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed700658e0 0x7fed70067da0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fed8c14bcc0 tx=0x7fed7c009040 comp rx=0 tx=0).stop 2023-12-19T16:54:48.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.127+0000 7fed9356d640 1 -- 172.21.15.26:0/2348713889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed8c070ab0 msgr2=0x7fed8c159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:48.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.127+0000 7fed9356d640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed8c070ab0 0x7fed8c159fe0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fed88019050 tx=0x7fed8801ac80 comp rx=0 tx=0).stop 2023-12-19T16:54:48.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.127+0000 7fed9356d640 1 -- 172.21.15.26:0/2348713889 shutdown_connections 2023-12-19T16:54:48.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.127+0000 7fed9356d640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed700658e0 0x7fed70067da0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:48.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.127+0000 7fed9356d640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed8c159a10 0x7fed8c149a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:48.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.127+0000 7fed9356d640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed8c0b3fe0 0x7fed8c14d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:48.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.127+0000 7fed9356d640 1 --2- 172.21.15.26:0/2348713889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed8c070ab0 0x7fed8c159fe0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:48.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.127+0000 7fed9356d640 1 -- 172.21.15.26:0/2348713889 >> 172.21.15.26:0/2348713889 conn(0x7fed8c06a320 msgr2=0x7fed8c14f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:48.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.127+0000 7fed9356d640 1 -- 172.21.15.26:0/2348713889 shutdown_connections 2023-12-19T16:54:48.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.127+0000 7fed9356d640 1 -- 172.21.15.26:0/2348713889 wait complete. 2023-12-19T16:54:48.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T16:54:48.194 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T16:54:48.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.295+0000 7f9185a2a640 1 Processor -- start 2023-12-19T16:54:48.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.295+0000 7f9185a2a640 1 -- start start 2023-12-19T16:54:48.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.295+0000 7f9185a2a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91801591f0 0x7f91800b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:48.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.295+0000 7f9185a2a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91800affe0 0x7f91800b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:48.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.295+0000 7f9185a2a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91800b0970 con 0x7f91801595d0 2023-12-19T16:54:48.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.295+0000 7f9185a2a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91800ce060 con 0x7f91801591f0 2023-12-19T16:54:48.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.295+0000 7f9185a2a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91800cdd80 con 0x7f91800affe0 2023-12-19T16:54:48.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.295+0000 7f9184a28640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91801591f0 0x7f91800b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:48.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.299+0000 7f9184a28640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91801591f0 0x7f91800b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54176/0 (socket says 172.21.15.26:54176) 2023-12-19T16:54:48.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.299+0000 7f9184a28640 1 -- 172.21.15.26:0/3827189249 learned_addr learned my addr 172.21.15.26:0/3827189249 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:48.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.299+0000 7f9185229640 1 --2- 172.21.15.26:0/3827189249 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91800affe0 0x7f91800b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:48.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.299+0000 7f9184a28640 1 -- 172.21.15.26:0/3827189249 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91800affe0 msgr2=0x7f91800b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:48.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.299+0000 7f9184a28640 1 --2- 172.21.15.26:0/3827189249 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91800affe0 0x7f91800b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:48.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.299+0000 7f9184a28640 1 -- 172.21.15.26:0/3827189249 >> v1:172.21.15.26:6789/0 conn(0x7f91801595d0 legacy=0x7f91800cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:48.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.299+0000 7f917f7fe640 1 -- 172.21.15.26:0/3827189249 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347648114 0 0) 0x7f916c001120 con 0x7f91801595d0 2023-12-19T16:54:48.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.299+0000 7f9184a28640 1 -- 172.21.15.26:0/3827189249 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91800c8cc0 con 0x7f91801591f0 2023-12-19T16:54:48.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.299+0000 7f9185229640 1 --2- 172.21.15.26:0/3827189249 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91800affe0 0x7f91800b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:54:48.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.299+0000 7f9184a28640 1 --2- 172.21.15.26:0/3827189249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91801591f0 0x7f91800b1fc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9174009870 tx=0x7f917401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b200461017f13466 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:48.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.299+0000 7f917f7fe640 1 -- 172.21.15.26:0/3827189249 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91740ab600 con 0x7f91801591f0 2023-12-19T16:54:48.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.299+0000 7f917f7fe640 1 -- 172.21.15.26:0/3827189249 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91740ab760 con 0x7f91801591f0 2023-12-19T16:54:48.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.299+0000 7f917f7fe640 1 -- 172.21.15.26:0/3827189249 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91740abaf0 con 0x7f91801591f0 2023-12-19T16:54:48.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 -- 172.21.15.26:0/3827189249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91801591f0 msgr2=0x7f91800b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:48.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 --2- 172.21.15.26:0/3827189249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91801591f0 0x7f91800b1fc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9174009870 tx=0x7f917401c480 comp rx=0 tx=0).stop 2023-12-19T16:54:48.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 -- 172.21.15.26:0/3827189249 shutdown_connections 2023-12-19T16:54:48.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 --2- 172.21.15.26:0/3827189249 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91800affe0 0x7f91800b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:48.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 --2- 172.21.15.26:0/3827189249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91801591f0 0x7f91800b1fc0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:48.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 -- 172.21.15.26:0/3827189249 >> 172.21.15.26:0/3827189249 conn(0x7f918006b0f0 msgr2=0x7f918006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:48.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 -- 172.21.15.26:0/3827189249 shutdown_connections 2023-12-19T16:54:48.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 -- 172.21.15.26:0/3827189249 wait complete. 2023-12-19T16:54:48.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 Processor -- start 2023-12-19T16:54:48.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 -- start start 2023-12-19T16:54:48.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91800affe0 0x7f918015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:48.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91801591f0 0x7f918014f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:48.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91801595d0 0x7f9180150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:48.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918014f880 con 0x7f91801595d0 2023-12-19T16:54:48.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9180150670 con 0x7f91801591f0 2023-12-19T16:54:48.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185a2a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918015b1b0 con 0x7f91800affe0 2023-12-19T16:54:48.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185229640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91801595d0 0x7f9180150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:48.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185229640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91801595d0 0x7f9180150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54178/0 (socket says 172.21.15.26:54178) 2023-12-19T16:54:48.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9185229640 1 -- 172.21.15.26:0/4254480199 learned_addr learned my addr 172.21.15.26:0/4254480199 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:48.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f9184a28640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91800affe0 0x7f918015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:48.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.303+0000 7f917ffff640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91801591f0 0x7f918014f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:48.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f9185229640 1 -- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91800affe0 msgr2=0x7f918015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:48.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f9185229640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91800affe0 0x7f918015ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:48.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f9185229640 1 -- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91801591f0 msgr2=0x7f918014f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:48.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f9185229640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91801591f0 0x7f918014f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:48.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f9185229640 1 -- 172.21.15.26:0/4254480199 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91740094d0 con 0x7f91801595d0 2023-12-19T16:54:48.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f9185229640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91801595d0 0x7f9180150b00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f917800cf20 tx=0x7f917800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:48.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f917d7fa640 1 -- 172.21.15.26:0/4254480199 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917800d840 con 0x7f91801595d0 2023-12-19T16:54:48.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f917d7fa640 1 -- 172.21.15.26:0/4254480199 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917800d9a0 con 0x7f91801595d0 2023-12-19T16:54:48.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f917d7fa640 1 -- 172.21.15.26:0/4254480199 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917800dd50 con 0x7f91801595d0 2023-12-19T16:54:48.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f9185a2a640 1 -- 172.21.15.26:0/4254480199 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91800cdb50 con 0x7f91801595d0 2023-12-19T16:54:48.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f9185a2a640 1 -- 172.21.15.26:0/4254480199 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918014fd40 con 0x7f91801595d0 2023-12-19T16:54:48.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f9185a2a640 1 -- 172.21.15.26:0/4254480199 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918004e850 con 0x7f91801595d0 2023-12-19T16:54:48.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f917d7fa640 1 -- 172.21.15.26:0/4254480199 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f917801e3b0 con 0x7f91801595d0 2023-12-19T16:54:48.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f917d7fa640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91640658b0 0x7f9164067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:48.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f917d7fa640 1 -- 172.21.15.26:0/4254480199 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7086+0+0 (secure 0 0 0) 0x7f9178083cf0 con 0x7f91801595d0 2023-12-19T16:54:48.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.307+0000 7f9184a28640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91640658b0 0x7f9164067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:48.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.311+0000 7f9184a28640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91640658b0 0x7f9164067d70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f91740af000 tx=0x7f91740a7000 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:48.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.311+0000 7f917d7fa640 1 -- 172.21.15.26:0/4254480199 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9178051540 con 0x7f91801595d0 2023-12-19T16:54:48.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.487+0000 7f917d7fa640 1 -- 172.21.15.26:0/4254480199 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 17) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f9178051910 con 0x7f91801595d0 2023-12-19T16:54:48.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:48.503+0000 7f9185a2a640 1 -- 172.21.15.26:0/4254480199 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f91801507b0 con 0x7f91801595d0 2023-12-19T16:54:49.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.127+0000 7f917d7fa640 1 -- 172.21.15.26:0/4254480199 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9178080c40 con 0x7f91801595d0 2023-12-19T16:54:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.127+0000 7f9185a2a640 1 -- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91640658b0 msgr2=0x7f9164067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.127+0000 7f9185a2a640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91640658b0 0x7f9164067d70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f91740af000 tx=0x7f91740a7000 comp rx=0 tx=0).stop 2023-12-19T16:54:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.127+0000 7f9185a2a640 1 -- 172.21.15.26:0/4254480199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91801595d0 msgr2=0x7f9180150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.127+0000 7f9185a2a640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91801595d0 0x7f9180150b00 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f917800cf20 tx=0x7f917800a7d0 comp rx=0 tx=0).stop 2023-12-19T16:54:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.127+0000 7f9185a2a640 1 -- 172.21.15.26:0/4254480199 shutdown_connections 2023-12-19T16:54:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.127+0000 7f9185a2a640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91801595d0 0x7f9180150b00 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.127+0000 7f9185a2a640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91801591f0 0x7f918014f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:49.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.127+0000 7f9185a2a640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f91640658b0 0x7f9164067d70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:49.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.127+0000 7f9185a2a640 1 --2- 172.21.15.26:0/4254480199 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91800affe0 0x7f918015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:49.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.127+0000 7f9185a2a640 1 -- 172.21.15.26:0/4254480199 >> 172.21.15.26:0/4254480199 conn(0x7f918006b0f0 msgr2=0x7f91800c95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:49.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.127+0000 7f9185a2a640 1 -- 172.21.15.26:0/4254480199 shutdown_connections 2023-12-19T16:54:49.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.127+0000 7f9185a2a640 1 -- 172.21.15.26:0/4254480199 wait complete. 2023-12-19T16:54:49.154 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T16:54:49.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f3242f96640 1 Processor -- start 2023-12-19T16:54:49.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f3242f96640 1 -- start start 2023-12-19T16:54:49.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f3242f96640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f323c159830 0x7f323c15d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:49.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f3242f96640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f323c0c9310 0x7f323c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:49.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f3242f96640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323c15e3d0 con 0x7f323c0cb110 2023-12-19T16:54:49.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f3242f96640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323c06f7c0 con 0x7f323c0c9310 2023-12-19T16:54:49.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f3242f96640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323c0ca2b0 con 0x7f323c159830 2023-12-19T16:54:49.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f323bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f323c0c9310 0x7f323c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:49.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f323bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f323c0c9310 0x7f323c0cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54194/0 (socket says 172.21.15.26:54194) 2023-12-19T16:54:49.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f323bfff640 1 -- 172.21.15.26:0/145974738 learned_addr learned my addr 172.21.15.26:0/145974738 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:49.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f3240d32640 1 --2- 172.21.15.26:0/145974738 >> v2:172.21.15.160:3300/0 conn(0x7f323c159830 0x7f323c15d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:49.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f323bfff640 1 -- 172.21.15.26:0/145974738 >> v2:172.21.15.160:3300/0 conn(0x7f323c159830 msgr2=0x7f323c15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:49.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f323bfff640 1 --2- 172.21.15.26:0/145974738 >> v2:172.21.15.160:3300/0 conn(0x7f323c159830 0x7f323c15d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:49.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f323bfff640 1 -- 172.21.15.26:0/145974738 >> v1:172.21.15.160:6789/0 conn(0x7f323c0cb110 legacy=0x7f323c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:49.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.343+0000 7f323bfff640 1 -- 172.21.15.26:0/145974738 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c15de10 con 0x7f323c0c9310 2023-12-19T16:54:49.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.347+0000 7f323bfff640 1 --2- 172.21.15.26:0/145974738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f323c0c9310 0x7f323c0cc430 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f323c06ce60 tx=0x7f322801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525796079f188fba server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:49.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.347+0000 7f323b7fe640 1 -- 172.21.15.26:0/145974738 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32280ab600 con 0x7f323c0c9310 2023-12-19T16:54:49.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.347+0000 7f323b7fe640 1 -- 172.21.15.26:0/145974738 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32280ab760 con 0x7f323c0c9310 2023-12-19T16:54:49.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.347+0000 7f323b7fe640 1 -- 172.21.15.26:0/145974738 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32280abaf0 con 0x7f323c0c9310 2023-12-19T16:54:49.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.347+0000 7f3242f96640 1 -- 172.21.15.26:0/145974738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f323c0c9310 msgr2=0x7f323c0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:49.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.347+0000 7f3242f96640 1 --2- 172.21.15.26:0/145974738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f323c0c9310 0x7f323c0cc430 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f323c06ce60 tx=0x7f322801c400 comp rx=0 tx=0).stop 2023-12-19T16:54:49.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.347+0000 7f3242f96640 1 -- 172.21.15.26:0/145974738 shutdown_connections 2023-12-19T16:54:49.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.347+0000 7f3242f96640 1 --2- 172.21.15.26:0/145974738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f323c0c9310 0x7f323c0cc430 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:49.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.347+0000 7f3242f96640 1 --2- 172.21.15.26:0/145974738 >> v2:172.21.15.160:3300/0 conn(0x7f323c159830 0x7f323c15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:49.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.347+0000 7f3242f96640 1 -- 172.21.15.26:0/145974738 >> 172.21.15.26:0/145974738 conn(0x7f323c0b0270 msgr2=0x7f323c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:49.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.347+0000 7f3242f96640 1 -- 172.21.15.26:0/145974738 shutdown_connections 2023-12-19T16:54:49.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.347+0000 7f3242f96640 1 -- 172.21.15.26:0/145974738 wait complete. 2023-12-19T16:54:49.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f3242f96640 1 Processor -- start 2023-12-19T16:54:49.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f3242f96640 1 -- start start 2023-12-19T16:54:49.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f3242f96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f323c0c9310 0x7f323c0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:49.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f3242f96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f323c0cb110 0x7f323c0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:49.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f3242f96640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f323c159830 0x7f323c14dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:49.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f3242f96640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0c8350 con 0x7f323c0cb110 2023-12-19T16:54:49.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f3242f96640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0c2830 con 0x7f323c159830 2023-12-19T16:54:49.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f3242f96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0c30d0 con 0x7f323c0c9310 2023-12-19T16:54:49.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f323bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f323c0cb110 0x7f323c0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:49.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f323bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f323c0cb110 0x7f323c0c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54198/0 (socket says 172.21.15.26:54198) 2023-12-19T16:54:49.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f323bfff640 1 -- 172.21.15.26:0/773847988 learned_addr learned my addr 172.21.15.26:0/773847988 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:49.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f3241533640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f323c159830 0x7f323c14dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:49.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f3240d32640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f323c0c9310 0x7f323c0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:49.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f323bfff640 1 -- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f323c0c9310 msgr2=0x7f323c0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:49.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f323bfff640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f323c0c9310 0x7f323c0c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:49.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f323bfff640 1 -- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f323c159830 msgr2=0x7f323c14dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:49.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f323bfff640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f323c159830 0x7f323c14dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f323bfff640 1 -- 172.21.15.26:0/773847988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32280094d0 con 0x7f323c0cb110 2023-12-19T16:54:49.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f323bfff640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f323c0cb110 0x7f323c0c7e10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f322801aab0 tx=0x7f322801ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:49.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f32397fa640 1 -- 172.21.15.26:0/773847988 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32280b4800 con 0x7f323c0cb110 2023-12-19T16:54:49.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f32397fa640 1 -- 172.21.15.26:0/773847988 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32280b4960 con 0x7f323c0cb110 2023-12-19T16:54:49.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f3242f96640 1 -- 172.21.15.26:0/773847988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c0c8580 con 0x7f323c0cb110 2023-12-19T16:54:49.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f32397fa640 1 -- 172.21.15.26:0/773847988 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32280b4cf0 con 0x7f323c0cb110 2023-12-19T16:54:49.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.351+0000 7f3242f96640 1 -- 172.21.15.26:0/773847988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c0c2d60 con 0x7f323c0cb110 2023-12-19T16:54:49.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.355+0000 7f3242f96640 1 -- 172.21.15.26:0/773847988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323c074270 con 0x7f323c0cb110 2023-12-19T16:54:49.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.355+0000 7f32397fa640 1 -- 172.21.15.26:0/773847988 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f32280c7440 con 0x7f323c0cb110 2023-12-19T16:54:49.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.355+0000 7f32397fa640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3220065af0 0x7f3220067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:49.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.355+0000 7f32397fa640 1 -- 172.21.15.26:0/773847988 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 7086+0+0 (secure 0 0 0) 0x7f322811e2a0 con 0x7f323c0cb110 2023-12-19T16:54:49.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.355+0000 7f3240d32640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3220065af0 0x7f3220067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:49.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.355+0000 7f3240d32640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3220065af0 0x7f3220067fb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f32300079c0 tx=0x7f3230008040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:49.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.359+0000 7f32397fa640 1 -- 172.21.15.26:0/773847988 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f32280eb760 con 0x7f323c0cb110 2023-12-19T16:54:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:49.515+0000 7f3242f96640 1 -- 172.21.15.26:0/773847988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f323c0ce240 con 0x7f323c0cb110 2023-12-19T16:54:50.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.135+0000 7f32397fa640 1 -- 172.21.15.26:0/773847988 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f32280ebb30 con 0x7f323c0cb110 2023-12-19T16:54:50.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.139+0000 7f32397fa640 1 -- 172.21.15.26:0/773847988 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) v1 ==== 181+0+0 (secure 0 0 0) 0x7f32280eb760 con 0x7f323c0cb110 2023-12-19T16:54:50.142 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T16:54:50.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.143+0000 7f3242f96640 1 -- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3220065af0 msgr2=0x7f3220067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:50.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.143+0000 7f3242f96640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3220065af0 0x7f3220067fb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f32300079c0 tx=0x7f3230008040 comp rx=0 tx=0).stop 2023-12-19T16:54:50.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.143+0000 7f3242f96640 1 -- 172.21.15.26:0/773847988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f323c0cb110 msgr2=0x7f323c0c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:50.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.143+0000 7f3242f96640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f323c0cb110 0x7f323c0c7e10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f322801aab0 tx=0x7f322801ac40 comp rx=0 tx=0).stop 2023-12-19T16:54:50.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.143+0000 7f3242f96640 1 -- 172.21.15.26:0/773847988 shutdown_connections 2023-12-19T16:54:50.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.143+0000 7f3242f96640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f323c159830 0x7f323c14dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:50.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.143+0000 7f3242f96640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f323c0cb110 0x7f323c0c7e10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:50.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.143+0000 7f3242f96640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3220065af0 0x7f3220067fb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:50.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.143+0000 7f3242f96640 1 --2- 172.21.15.26:0/773847988 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f323c0c9310 0x7f323c0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:50.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.143+0000 7f3242f96640 1 -- 172.21.15.26:0/773847988 >> 172.21.15.26:0/773847988 conn(0x7f323c0b0270 msgr2=0x7f323c06c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:50.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.143+0000 7f3242f96640 1 -- 172.21.15.26:0/773847988 shutdown_connections 2023-12-19T16:54:50.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.143+0000 7f3242f96640 1 -- 172.21.15.26:0/773847988 wait complete. 2023-12-19T16:54:50.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T16:54:50.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475fc8c640 1 Processor -- start 2023-12-19T16:54:50.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475fc8c640 1 -- start start 2023-12-19T16:54:50.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475fc8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4758158500 0x7f475806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:50.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475fc8c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f47580cf190 0x7f475815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:50.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475fc8c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f475815bbf0 con 0x7f475806c5c0 2023-12-19T16:54:50.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475fc8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475815b2c0 con 0x7f4758158500 2023-12-19T16:54:50.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475fc8c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475815b5a0 con 0x7f47580cf190 2023-12-19T16:54:50.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475da28640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4758158500 0x7f475806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475da28640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4758158500 0x7f475806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54202/0 (socket says 172.21.15.26:54202) 2023-12-19T16:54:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475da28640 1 -- 172.21.15.26:0/1760954353 learned_addr learned my addr 172.21.15.26:0/1760954353 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475e229640 1 --2- 172.21.15.26:0/1760954353 >> v2:172.21.15.160:3300/0 conn(0x7f47580cf190 0x7f475815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475da28640 1 -- 172.21.15.26:0/1760954353 >> v2:172.21.15.160:3300/0 conn(0x7f47580cf190 msgr2=0x7f475815c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475da28640 1 --2- 172.21.15.26:0/1760954353 >> v2:172.21.15.160:3300/0 conn(0x7f47580cf190 0x7f475815c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475da28640 1 -- 172.21.15.26:0/1760954353 >> v1:172.21.15.160:6789/0 conn(0x7f475806c5c0 legacy=0x7f47580cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475da28640 1 -- 172.21.15.26:0/1760954353 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475815b880 con 0x7f4758158500 2023-12-19T16:54:50.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.311+0000 7f475e229640 1 --2- 172.21.15.26:0/1760954353 >> v2:172.21.15.160:3300/0 conn(0x7f47580cf190 0x7f475815c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:54:50.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.315+0000 7f475da28640 1 --2- 172.21.15.26:0/1760954353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4758158500 0x7f475806c080 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f475806d450 tx=0x7f475401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eae2604dbcdea58 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.315+0000 7f475ca26640 1 -- 172.21.15.26:0/1760954353 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47540aa640 con 0x7f4758158500 2023-12-19T16:54:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.315+0000 7f475ca26640 1 -- 172.21.15.26:0/1760954353 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47540aa7a0 con 0x7f4758158500 2023-12-19T16:54:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.315+0000 7f475ca26640 1 -- 172.21.15.26:0/1760954353 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47540aab30 con 0x7f4758158500 2023-12-19T16:54:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.315+0000 7f475fc8c640 1 -- 172.21.15.26:0/1760954353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4758158500 msgr2=0x7f475806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.315+0000 7f475fc8c640 1 --2- 172.21.15.26:0/1760954353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4758158500 0x7f475806c080 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f475806d450 tx=0x7f475401c4c0 comp rx=0 tx=0).stop 2023-12-19T16:54:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.315+0000 7f475fc8c640 1 -- 172.21.15.26:0/1760954353 shutdown_connections 2023-12-19T16:54:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.315+0000 7f475fc8c640 1 --2- 172.21.15.26:0/1760954353 >> v2:172.21.15.160:3300/0 conn(0x7f47580cf190 0x7f475815c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.315+0000 7f475fc8c640 1 --2- 172.21.15.26:0/1760954353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4758158500 0x7f475806c080 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:50.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.315+0000 7f475fc8c640 1 -- 172.21.15.26:0/1760954353 >> 172.21.15.26:0/1760954353 conn(0x7f47580b1aa0 msgr2=0x7f47580b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:50.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.315+0000 7f475fc8c640 1 -- 172.21.15.26:0/1760954353 shutdown_connections 2023-12-19T16:54:50.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.315+0000 7f475fc8c640 1 -- 172.21.15.26:0/1760954353 wait complete. 2023-12-19T16:54:50.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475fc8c640 1 Processor -- start 2023-12-19T16:54:50.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475fc8c640 1 -- start start 2023-12-19T16:54:50.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475fc8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475806c5c0 0x7f4758140780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:50.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475fc8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47580cf190 0x7f475813d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:50.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475fc8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4758158500 0x7f475813dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475fc8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47580ca0d0 con 0x7f47580cf190 2023-12-19T16:54:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475fc8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47580ca3b0 con 0x7f4758158500 2023-12-19T16:54:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475fc8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47580ca690 con 0x7f475806c5c0 2023-12-19T16:54:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475d227640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47580cf190 0x7f475813d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475d227640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47580cf190 0x7f475813d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54204/0 (socket says 172.21.15.26:54204) 2023-12-19T16:54:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475d227640 1 -- 172.21.15.26:0/3250658794 learned_addr learned my addr 172.21.15.26:0/3250658794 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475e229640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4758158500 0x7f475813dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475da28640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475806c5c0 0x7f4758140780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475d227640 1 -- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475806c5c0 msgr2=0x7f4758140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:50.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475d227640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475806c5c0 0x7f4758140780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475d227640 1 -- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4758158500 msgr2=0x7f475813dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475d227640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4758158500 0x7f475813dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475d227640 1 -- 172.21.15.26:0/3250658794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47540094d0 con 0x7f47580cf190 2023-12-19T16:54:50.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475d227640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47580cf190 0x7f475813d360 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f474800c410 tx=0x7f474800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:50.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f474e7fc640 1 -- 172.21.15.26:0/3250658794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748011790 con 0x7f47580cf190 2023-12-19T16:54:50.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f474e7fc640 1 -- 172.21.15.26:0/3250658794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47480118f0 con 0x7f47580cf190 2023-12-19T16:54:50.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f474e7fc640 1 -- 172.21.15.26:0/3250658794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748011ce0 con 0x7f47580cf190 2023-12-19T16:54:50.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475fc8c640 1 -- 172.21.15.26:0/3250658794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47580ca970 con 0x7f47580cf190 2023-12-19T16:54:50.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.319+0000 7f475fc8c640 1 -- 172.21.15.26:0/3250658794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4758140cc0 con 0x7f47580cf190 2023-12-19T16:54:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.323+0000 7f475fc8c640 1 -- 172.21.15.26:0/3250658794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47580cae90 con 0x7f47580cf190 2023-12-19T16:54:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.323+0000 7f474e7fc640 1 -- 172.21.15.26:0/3250658794 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f474801c460 con 0x7f47580cf190 2023-12-19T16:54:50.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.323+0000 7f474e7fc640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f473c065b80 0x7f473c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.323+0000 7f474e7fc640 1 -- 172.21.15.26:0/3250658794 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6648+0+0 (secure 0 0 0) 0x7f47480832a0 con 0x7f47580cf190 2023-12-19T16:54:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.323+0000 7f475da28640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f473c065b80 0x7f473c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:50.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.323+0000 7f475da28640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f473c065b80 0x7f473c068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f47540ae000 tx=0x7f47540b5040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:50.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.327+0000 7f474e7fc640 1 -- 172.21.15.26:0/3250658794 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f47580cae90 con 0x7f47580cf190 2023-12-19T16:54:50.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:50.483+0000 7f475fc8c640 1 -- 172.21.15.26:0/3250658794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f47580cae90 con 0x7f47580cf190 2023-12-19T16:54:51.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.139+0000 7f474e7fc640 1 -- 172.21.15.26:0/3250658794 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) v1 ==== 169+0+0 (secure 0 0 0) 0x7f47580cae90 con 0x7f47580cf190 2023-12-19T16:54:51.142 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T16:54:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.139+0000 7f475fc8c640 1 -- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f473c065b80 msgr2=0x7f473c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:51.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.139+0000 7f475fc8c640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f473c065b80 0x7f473c068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f47540ae000 tx=0x7f47540b5040 comp rx=0 tx=0).stop 2023-12-19T16:54:51.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.139+0000 7f475fc8c640 1 -- 172.21.15.26:0/3250658794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47580cf190 msgr2=0x7f475813d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:51.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.139+0000 7f475fc8c640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47580cf190 0x7f475813d360 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f474800c410 tx=0x7f474800c8d0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.143+0000 7f475fc8c640 1 -- 172.21.15.26:0/3250658794 shutdown_connections 2023-12-19T16:54:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.143+0000 7f475fc8c640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4758158500 0x7f475813dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.143+0000 7f475fc8c640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47580cf190 0x7f475813d360 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.143+0000 7f475fc8c640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f473c065b80 0x7f473c068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.143+0000 7f475fc8c640 1 --2- 172.21.15.26:0/3250658794 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475806c5c0 0x7f4758140780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.143+0000 7f475fc8c640 1 -- 172.21.15.26:0/3250658794 >> 172.21.15.26:0/3250658794 conn(0x7f47580b1aa0 msgr2=0x7f47580cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.143+0000 7f475fc8c640 1 -- 172.21.15.26:0/3250658794 shutdown_connections 2023-12-19T16:54:51.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.143+0000 7f475fc8c640 1 -- 172.21.15.26:0/3250658794 wait complete. 2023-12-19T16:54:51.168 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-19T16:54:51.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb970820640 1 Processor -- start 2023-12-19T16:54:51.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb970820640 1 -- start start 2023-12-19T16:54:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb970820640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb96c1599e0 0x7fb96c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb970820640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb96c0cb150 0x7fb96c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb970820640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb96c15e240 con 0x7fb96c15d530 2023-12-19T16:54:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb970820640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96c15e050 con 0x7fb96c1599e0 2023-12-19T16:54:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb970820640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96c0cbb20 con 0x7fb96c0cb150 2023-12-19T16:54:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb96a59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb96c1599e0 0x7fb96c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb96a59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb96c1599e0 0x7fb96c06f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54214/0 (socket says 172.21.15.26:54214) 2023-12-19T16:54:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb96a59c640 1 -- 172.21.15.26:0/713632381 learned_addr learned my addr 172.21.15.26:0/713632381 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb963fff640 1 --2- 172.21.15.26:0/713632381 >> v2:172.21.15.160:3300/0 conn(0x7fb96c0cb150 0x7fb96c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb96a59c640 1 -- 172.21.15.26:0/713632381 >> v2:172.21.15.160:3300/0 conn(0x7fb96c0cb150 msgr2=0x7fb96c0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb96a59c640 1 --2- 172.21.15.26:0/713632381 >> v2:172.21.15.160:3300/0 conn(0x7fb96c0cb150 0x7fb96c0cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb96a59c640 1 -- 172.21.15.26:0/713632381 >> v1:172.21.15.160:6789/0 conn(0x7fb96c15d530 legacy=0x7fb96c15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:51.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.323+0000 7fb96a59c640 1 -- 172.21.15.26:0/713632381 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96c0c9520 con 0x7fb96c1599e0 2023-12-19T16:54:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.327+0000 7fb96a59c640 1 --2- 172.21.15.26:0/713632381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb96c1599e0 0x7fb96c06f6d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0cc220 tx=0x7fb95c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a99e76931e8a068 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.327+0000 7fb969d9b640 1 -- 172.21.15.26:0/713632381 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0aa660 con 0x7fb96c1599e0 2023-12-19T16:54:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.327+0000 7fb969d9b640 1 -- 172.21.15.26:0/713632381 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95c0aa7c0 con 0x7fb96c1599e0 2023-12-19T16:54:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.327+0000 7fb969d9b640 1 -- 172.21.15.26:0/713632381 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0aab50 con 0x7fb96c1599e0 2023-12-19T16:54:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.327+0000 7fb970820640 1 -- 172.21.15.26:0/713632381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb96c1599e0 msgr2=0x7fb96c06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.327+0000 7fb970820640 1 --2- 172.21.15.26:0/713632381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb96c1599e0 0x7fb96c06f6d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0cc220 tx=0x7fb95c01c500 comp rx=0 tx=0).stop 2023-12-19T16:54:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.327+0000 7fb970820640 1 -- 172.21.15.26:0/713632381 shutdown_connections 2023-12-19T16:54:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.327+0000 7fb970820640 1 --2- 172.21.15.26:0/713632381 >> v2:172.21.15.160:3300/0 conn(0x7fb96c0cb150 0x7fb96c0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.327+0000 7fb970820640 1 --2- 172.21.15.26:0/713632381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb96c1599e0 0x7fb96c06f6d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.327+0000 7fb970820640 1 -- 172.21.15.26:0/713632381 >> 172.21.15.26:0/713632381 conn(0x7fb96c066160 msgr2=0x7fb96c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.327+0000 7fb970820640 1 -- 172.21.15.26:0/713632381 shutdown_connections 2023-12-19T16:54:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.327+0000 7fb970820640 1 -- 172.21.15.26:0/713632381 wait complete. 2023-12-19T16:54:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb970820640 1 Processor -- start 2023-12-19T16:54:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb970820640 1 -- start start 2023-12-19T16:54:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb970820640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb96c0cb150 0x7fb96c1531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb970820640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb96c1599e0 0x7fb96c0c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb970820640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb96c15d530 0x7fb96c1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb970820640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0c6db0 con 0x7fb96c0cb150 2023-12-19T16:54:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb970820640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0c7090 con 0x7fb96c1599e0 2023-12-19T16:54:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb970820640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96c0c7370 con 0x7fb96c15d530 2023-12-19T16:54:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb96a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb96c0cb150 0x7fb96c1531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb96a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb96c0cb150 0x7fb96c1531f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54224/0 (socket says 172.21.15.26:54224) 2023-12-19T16:54:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb96a59c640 1 -- 172.21.15.26:0/1772889975 learned_addr learned my addr 172.21.15.26:0/1772889975 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb963fff640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb96c1599e0 0x7fb96c0c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb96ad9d640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb96c15d530 0x7fb96c1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb96a59c640 1 -- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb96c15d530 msgr2=0x7fb96c1539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb96a59c640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb96c15d530 0x7fb96c1539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb96a59c640 1 -- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb96c1599e0 msgr2=0x7fb96c0c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb96a59c640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb96c1599e0 0x7fb96c0c7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb96a59c640 1 -- 172.21.15.26:0/1772889975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c0094d0 con 0x7fb96c0cb150 2023-12-19T16:54:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb96a59c640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb96c0cb150 0x7fb96c1531f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0ac040 tx=0x7fb95c0192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb962ffd640 1 -- 172.21.15.26:0/1772889975 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0b3800 con 0x7fb96c0cb150 2023-12-19T16:54:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb962ffd640 1 -- 172.21.15.26:0/1772889975 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95c0b3960 con 0x7fb96c0cb150 2023-12-19T16:54:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb970820640 1 -- 172.21.15.26:0/1772889975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96c0c7650 con 0x7fb96c0cb150 2023-12-19T16:54:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb962ffd640 1 -- 172.21.15.26:0/1772889975 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c0b3d10 con 0x7fb96c0cb150 2023-12-19T16:54:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.331+0000 7fb970820640 1 -- 172.21.15.26:0/1772889975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96c153730 con 0x7fb96c0cb150 2023-12-19T16:54:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.335+0000 7fb970820640 1 -- 172.21.15.26:0/1772889975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb96c074270 con 0x7fb96c0cb150 2023-12-19T16:54:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.335+0000 7fb962ffd640 1 -- 172.21.15.26:0/1772889975 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb95c0c6390 con 0x7fb96c0cb150 2023-12-19T16:54:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.335+0000 7fb962ffd640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb94c065b80 0x7fb94c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.335+0000 7fb962ffd640 1 -- 172.21.15.26:0/1772889975 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fb95c0a6040 con 0x7fb96c0cb150 2023-12-19T16:54:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.335+0000 7fb963fff640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb94c065b80 0x7fb94c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:51.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.335+0000 7fb963fff640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb94c065b80 0x7fb94c068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0c2490 tx=0x7fb954008040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.339+0000 7fb962ffd640 1 -- 172.21.15.26:0/1772889975 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb95c0eb6b0 con 0x7fb96c0cb150 2023-12-19T16:54:51.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.495+0000 7fb970820640 1 -- 172.21.15.26:0/1772889975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fb96c0c2050 con 0x7fb96c0cb150 2023-12-19T16:54:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.503+0000 7fb962ffd640 1 -- 172.21.15.26:0/1772889975 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v13) v1 ==== 120+0+0 (secure 0 0 0) 0x7fb95c0eecd0 con 0x7fb96c0cb150 2023-12-19T16:54:51.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.503+0000 7fb970820640 1 -- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb94c065b80 msgr2=0x7fb94c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.503+0000 7fb970820640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb94c065b80 0x7fb94c068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb96c0c2490 tx=0x7fb954008040 comp rx=0 tx=0).stop 2023-12-19T16:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.503+0000 7fb970820640 1 -- 172.21.15.26:0/1772889975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb96c0cb150 msgr2=0x7fb96c1531f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.503+0000 7fb970820640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb96c0cb150 0x7fb96c1531f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb95c0ac040 tx=0x7fb95c0192a0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.503+0000 7fb970820640 1 -- 172.21.15.26:0/1772889975 shutdown_connections 2023-12-19T16:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.503+0000 7fb970820640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb94c065b80 0x7fb94c068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.503+0000 7fb970820640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb96c15d530 0x7fb96c1539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.503+0000 7fb970820640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb96c1599e0 0x7fb96c0c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.503+0000 7fb970820640 1 --2- 172.21.15.26:0/1772889975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb96c0cb150 0x7fb96c1531f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.503+0000 7fb970820640 1 -- 172.21.15.26:0/1772889975 >> 172.21.15.26:0/1772889975 conn(0x7fb96c066160 msgr2=0x7fb96c0ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.503+0000 7fb970820640 1 -- 172.21.15.26:0/1772889975 shutdown_connections 2023-12-19T16:54:51.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.503+0000 7fb970820640 1 -- 172.21.15.26:0/1772889975 wait complete. 2023-12-19T16:54:51.527 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T16:54:51.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036ed17640 1 Processor -- start 2023-12-19T16:54:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036ed17640 1 -- start start 2023-12-19T16:54:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036ed17640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f036815cc70 0x7f03680d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036ed17640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f03680b88b0 0x7f03680b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036ed17640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03680b71f0 con 0x7f03680b6140 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036ed17640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03680b65e0 con 0x7f036815cc70 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036ed17640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03680b68c0 con 0x7f03680b88b0 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036cab3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f036815cc70 0x7f03680d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036d2b4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f03680b6140 0x7f03680cfbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46904/0 (socket says 172.21.15.26:46904) 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036cab3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f036815cc70 0x7f03680d0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54236/0 (socket says 172.21.15.26:54236) 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036d2b4640 1 -- 172.21.15.26:0/3887344415 learned_addr learned my addr 172.21.15.26:0/3887344415 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f0367fff640 1 --2- 172.21.15.26:0/3887344415 >> v2:172.21.15.160:3300/0 conn(0x7f03680b88b0 0x7f03680b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036cab3640 1 -- 172.21.15.26:0/3887344415 >> v2:172.21.15.160:3300/0 conn(0x7f03680b88b0 msgr2=0x7f03680b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036cab3640 1 --2- 172.21.15.26:0/3887344415 >> v2:172.21.15.160:3300/0 conn(0x7f03680b88b0 0x7f03680b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036cab3640 1 -- 172.21.15.26:0/3887344415 >> v1:172.21.15.26:6789/0 conn(0x7f03680b6140 legacy=0x7f03680cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f036cab3640 1 -- 172.21.15.26:0/3887344415 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03680b6ba0 con 0x7f036815cc70 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f0367fff640 1 --2- 172.21.15.26:0/3887344415 >> v2:172.21.15.160:3300/0 conn(0x7f03680b88b0 0x7f03680b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.671+0000 7f03677fe640 1 -- 172.21.15.26:0/3887344415 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589783246 0 0) 0x7f03680b71f0 con 0x7f03680b6140 2023-12-19T16:54:51.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.675+0000 7f036cab3640 1 --2- 172.21.15.26:0/3887344415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f036815cc70 0x7f03680d0c60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f03680d3ce0 tx=0x7f035c01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769e9de374a6d02e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:51.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.675+0000 7f03677fe640 1 -- 172.21.15.26:0/3887344415 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c0a8950 con 0x7f036815cc70 2023-12-19T16:54:51.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.675+0000 7f03677fe640 1 -- 172.21.15.26:0/3887344415 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c0a8ab0 con 0x7f036815cc70 2023-12-19T16:54:51.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.675+0000 7f03677fe640 1 -- 172.21.15.26:0/3887344415 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c0a8e40 con 0x7f036815cc70 2023-12-19T16:54:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.675+0000 7f036ed17640 1 -- 172.21.15.26:0/3887344415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f036815cc70 msgr2=0x7f03680d0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.675+0000 7f036ed17640 1 --2- 172.21.15.26:0/3887344415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f036815cc70 0x7f03680d0c60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f03680d3ce0 tx=0x7f035c01ec50 comp rx=0 tx=0).stop 2023-12-19T16:54:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.675+0000 7f036ed17640 1 -- 172.21.15.26:0/3887344415 shutdown_connections 2023-12-19T16:54:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.675+0000 7f036ed17640 1 --2- 172.21.15.26:0/3887344415 >> v2:172.21.15.160:3300/0 conn(0x7f03680b88b0 0x7f03680b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.675+0000 7f036ed17640 1 --2- 172.21.15.26:0/3887344415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f036815cc70 0x7f03680d0c60 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.675+0000 7f036ed17640 1 -- 172.21.15.26:0/3887344415 >> 172.21.15.26:0/3887344415 conn(0x7f036806a120 msgr2=0x7f036806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.675+0000 7f036ed17640 1 -- 172.21.15.26:0/3887344415 shutdown_connections 2023-12-19T16:54:51.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.675+0000 7f036ed17640 1 -- 172.21.15.26:0/3887344415 wait complete. 2023-12-19T16:54:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036ed17640 1 Processor -- start 2023-12-19T16:54:51.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036ed17640 1 -- start start 2023-12-19T16:54:51.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036ed17640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03680b6140 0x7f036814ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:51.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036ed17640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03680b88b0 0x7f03680c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:51.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036ed17640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f036815cc70 0x7f0368159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:51.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036ed17640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03680c8780 con 0x7f036815cc70 2023-12-19T16:54:51.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036ed17640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03680c7b70 con 0x7f03680b88b0 2023-12-19T16:54:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036ed17640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03680c7e50 con 0x7f03680b6140 2023-12-19T16:54:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036d2b4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f036815cc70 0x7f0368159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036d2b4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f036815cc70 0x7f0368159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54240/0 (socket says 172.21.15.26:54240) 2023-12-19T16:54:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036d2b4640 1 -- 172.21.15.26:0/3989308470 learned_addr learned my addr 172.21.15.26:0/3989308470 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036cab3640 1 --2- 172.21.15.26:0/3989308470 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03680b6140 0x7f036814ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036d2b4640 1 -- 172.21.15.26:0/3989308470 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03680b6140 msgr2=0x7f036814ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036d2b4640 1 --2- 172.21.15.26:0/3989308470 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03680b6140 0x7f036814ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036d2b4640 1 -- 172.21.15.26:0/3989308470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03680b88b0 msgr2=0x7f03680c7200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-19T16:54:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036d2b4640 1 --2- 172.21.15.26:0/3989308470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03680b88b0 0x7f03680c7200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036d2b4640 1 -- 172.21.15.26:0/3989308470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c009420 con 0x7f036815cc70 2023-12-19T16:54:51.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036d2b4640 1 --2- 172.21.15.26:0/3989308470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f036815cc70 0x7f0368159080 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f036000c780 tx=0x7f036000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:51.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f03657fa640 1 -- 172.21.15.26:0/3989308470 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036000ce90 con 0x7f036815cc70 2023-12-19T16:54:51.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f03657fa640 1 -- 172.21.15.26:0/3989308470 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036000a640 con 0x7f036815cc70 2023-12-19T16:54:51.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f03657fa640 1 -- 172.21.15.26:0/3989308470 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036000aa70 con 0x7f036815cc70 2023-12-19T16:54:51.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036ed17640 1 -- 172.21.15.26:0/3989308470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03680c8130 con 0x7f036815cc70 2023-12-19T16:54:51.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.679+0000 7f036ed17640 1 -- 172.21.15.26:0/3989308470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03680c7740 con 0x7f036815cc70 2023-12-19T16:54:51.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.683+0000 7f036ed17640 1 -- 172.21.15.26:0/3989308470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03680c88f0 con 0x7f036815cc70 2023-12-19T16:54:51.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.683+0000 7f03657fa640 1 -- 172.21.15.26:0/3989308470 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f036000abd0 con 0x7f036815cc70 2023-12-19T16:54:51.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.683+0000 7f03657fa640 1 --2- 172.21.15.26:0/3989308470 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f034c065b80 0x7f034c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:51.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.683+0000 7f03657fa640 1 -- 172.21.15.26:0/3989308470 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f0360082b80 con 0x7f036815cc70 2023-12-19T16:54:51.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.683+0000 7f036cab3640 1 --2- 172.21.15.26:0/3989308470 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f034c065b80 0x7f034c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:51.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.683+0000 7f036cab3640 1 --2- 172.21.15.26:0/3989308470 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f034c065b80 0x7f034c068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f035c0b2000 tx=0x7f035c0aa070 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.687+0000 7f03657fa640 1 -- 172.21.15.26:0/3989308470 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0360050180 con 0x7f036815cc70 2023-12-19T16:54:51.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.843+0000 7f036ed17640 1 -- 172.21.15.26:0/3989308470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0368074270 con 0x7f036815cc70 2023-12-19T16:54:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.851+0000 7f03657fa640 1 -- 172.21.15.26:0/3989308470 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7f03600180a0 con 0x7f036815cc70 2023-12-19T16:54:51.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.851+0000 7f036ed17640 1 -- 172.21.15.26:0/3989308470 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f034c065b80 msgr2=0x7f034c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.851+0000 7f036ed17640 1 --2- 172.21.15.26:0/3989308470 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f034c065b80 0x7f034c068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f035c0b2000 tx=0x7f035c0aa070 comp rx=0 tx=0).stop 2023-12-19T16:54:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.851+0000 7f036ed17640 1 -- 172.21.15.26:0/3989308470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f036815cc70 msgr2=0x7f0368159080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.851+0000 7f036ed17640 1 --2- 172.21.15.26:0/3989308470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f036815cc70 0x7f0368159080 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f036000c780 tx=0x7f036000cc40 comp rx=0 tx=0).stop 2023-12-19T16:54:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.851+0000 7f036ed17640 1 -- 172.21.15.26:0/3989308470 shutdown_connections 2023-12-19T16:54:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.851+0000 7f036ed17640 1 --2- 172.21.15.26:0/3989308470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f036815cc70 0x7f0368159080 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.851+0000 7f036ed17640 1 --2- 172.21.15.26:0/3989308470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03680b88b0 0x7f03680c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.851+0000 7f036ed17640 1 --2- 172.21.15.26:0/3989308470 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f034c065b80 0x7f034c068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.851+0000 7f036ed17640 1 --2- 172.21.15.26:0/3989308470 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03680b6140 0x7f036814ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.851+0000 7f036ed17640 1 -- 172.21.15.26:0/3989308470 >> 172.21.15.26:0/3989308470 conn(0x7f036806a120 msgr2=0x7f036815d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.851+0000 7f036ed17640 1 -- 172.21.15.26:0/3989308470 shutdown_connections 2023-12-19T16:54:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:51.851+0000 7f036ed17640 1 -- 172.21.15.26:0/3989308470 wait complete. 2023-12-19T16:54:51.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-19T16:54:52.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.015+0000 7f74ead7b640 1 Processor -- start 2023-12-19T16:54:52.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.015+0000 7f74ead7b640 1 -- start start 2023-12-19T16:54:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74ead7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74e41599e0 0x7f74e406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74ead7b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f74e40cb150 0x7f74e40cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74ead7b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f74e415d530 0x7f74e415d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74ead7b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e40cbaf0 con 0x7f74e40cb150 2023-12-19T16:54:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74ead7b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74e415e240 con 0x7f74e415d530 2023-12-19T16:54:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74ead7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e40c9520 con 0x7f74e41599e0 2023-12-19T16:54:52.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74e3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f74e40cb150 0x7f74e40cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74e3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f74e40cb150 0x7f74e40cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54248/0 (socket says 172.21.15.26:54248) 2023-12-19T16:54:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74e3fff640 1 -- 172.21.15.26:0/3139977698 learned_addr learned my addr 172.21.15.26:0/3139977698 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74e8b17640 1 --2- 172.21.15.26:0/3139977698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74e41599e0 0x7f74e406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74e9318640 1 --2- 172.21.15.26:0/3139977698 >> v2:172.21.15.160:3300/0 conn(0x7f74e415d530 0x7f74e415d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74e3fff640 1 -- 172.21.15.26:0/3139977698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74e41599e0 msgr2=0x7f74e406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74e3fff640 1 --2- 172.21.15.26:0/3139977698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74e41599e0 0x7f74e406f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74e3fff640 1 -- 172.21.15.26:0/3139977698 >> v2:172.21.15.160:3300/0 conn(0x7f74e415d530 msgr2=0x7f74e415d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74e3fff640 1 --2- 172.21.15.26:0/3139977698 >> v2:172.21.15.160:3300/0 conn(0x7f74e415d530 0x7f74e415d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.019+0000 7f74e3fff640 1 -- 172.21.15.26:0/3139977698 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e40c9800 con 0x7f74e40cb150 2023-12-19T16:54:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74e3fff640 1 --2- 172.21.15.26:0/3139977698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74e40cb150 0x7f74e40cb5b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f74e40cc220 tx=0x7f74d001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d23c43ab9c356c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74e37fe640 1 -- 172.21.15.26:0/3139977698 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d00ab680 con 0x7f74e40cb150 2023-12-19T16:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74e37fe640 1 -- 172.21.15.26:0/3139977698 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d00ab7e0 con 0x7f74e40cb150 2023-12-19T16:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74e37fe640 1 -- 172.21.15.26:0/3139977698 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d00abb70 con 0x7f74e40cb150 2023-12-19T16:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74ead7b640 1 -- 172.21.15.26:0/3139977698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74e40cb150 msgr2=0x7f74e40cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74ead7b640 1 --2- 172.21.15.26:0/3139977698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74e40cb150 0x7f74e40cb5b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f74e40cc220 tx=0x7f74d001c500 comp rx=0 tx=0).stop 2023-12-19T16:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74ead7b640 1 -- 172.21.15.26:0/3139977698 shutdown_connections 2023-12-19T16:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74ead7b640 1 --2- 172.21.15.26:0/3139977698 >> v2:172.21.15.160:3300/0 conn(0x7f74e415d530 0x7f74e415d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74ead7b640 1 --2- 172.21.15.26:0/3139977698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74e40cb150 0x7f74e40cb5b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74ead7b640 1 --2- 172.21.15.26:0/3139977698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74e41599e0 0x7f74e406f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74ead7b640 1 -- 172.21.15.26:0/3139977698 >> 172.21.15.26:0/3139977698 conn(0x7f74e4066160 msgr2=0x7f74e40b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74ead7b640 1 -- 172.21.15.26:0/3139977698 shutdown_connections 2023-12-19T16:54:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74ead7b640 1 -- 172.21.15.26:0/3139977698 wait complete. 2023-12-19T16:54:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74ead7b640 1 Processor -- start 2023-12-19T16:54:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.023+0000 7f74ead7b640 1 -- start start 2023-12-19T16:54:52.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74ead7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74e40cb150 0x7f74e4153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74ead7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74e41599e0 0x7f74e40c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74ead7b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74e415d530 0x7f74e41f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74ead7b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e40c27a0 con 0x7f74e40cb150 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74ead7b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e40c2d20 con 0x7f74e415d530 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74ead7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74e41f94a0 con 0x7f74e41599e0 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e8b17640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74e40cb150 0x7f74e4153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e8b17640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74e40cb150 0x7f74e4153c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54260/0 (socket says 172.21.15.26:54260) 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e8b17640 1 -- 172.21.15.26:0/1870136565 learned_addr learned my addr 172.21.15.26:0/1870136565 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e9318640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74e415d530 0x7f74e41f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e3fff640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74e41599e0 0x7f74e40c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e8b17640 1 -- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74e41599e0 msgr2=0x7f74e40c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e8b17640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74e41599e0 0x7f74e40c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e8b17640 1 -- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74e415d530 msgr2=0x7f74e41f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e8b17640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74e415d530 0x7f74e41f8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e8b17640 1 -- 172.21.15.26:0/1870136565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d00094d0 con 0x7f74e40cb150 2023-12-19T16:54:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e3fff640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74e41599e0 0x7f74e40c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e8b17640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74e40cb150 0x7f74e4153c70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e960 tx=0x7f74d800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e17fa640 1 -- 172.21.15.26:0/1870136565 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d8009690 con 0x7f74e40cb150 2023-12-19T16:54:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e17fa640 1 -- 172.21.15.26:0/1870136565 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d800ca00 con 0x7f74e40cb150 2023-12-19T16:54:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74e17fa640 1 -- 172.21.15.26:0/1870136565 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d800cd70 con 0x7f74e40cb150 2023-12-19T16:54:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74ead7b640 1 -- 172.21.15.26:0/1870136565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e41f9780 con 0x7f74e40cb150 2023-12-19T16:54:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74ead7b640 1 -- 172.21.15.26:0/1870136565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e41541b0 con 0x7f74e40cb150 2023-12-19T16:54:52.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.027+0000 7f74ead7b640 1 -- 172.21.15.26:0/1870136565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74e41f9910 con 0x7f74e40cb150 2023-12-19T16:54:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.031+0000 7f74e17fa640 1 -- 172.21.15.26:0/1870136565 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f74d8013470 con 0x7f74e40cb150 2023-12-19T16:54:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.031+0000 7f74e17fa640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f74c8065b80 0x7f74c8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:52.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.031+0000 7f74e17fa640 1 -- 172.21.15.26:0/1870136565 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f74d80837e0 con 0x7f74e40cb150 2023-12-19T16:54:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.031+0000 7f74e3fff640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f74c8065b80 0x7f74c8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.031+0000 7f74e3fff640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f74c8065b80 0x7f74c8068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f74d00af000 tx=0x7f74d00a7000 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:52.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.035+0000 7f74e17fa640 1 -- 172.21.15.26:0/1870136565 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f74d8050e90 con 0x7f74e40cb150 2023-12-19T16:54:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.191+0000 7f74ead7b640 1 -- 172.21.15.26:0/1870136565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f74e4074270 con 0x7f74e40cb150 2023-12-19T16:54:52.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.195+0000 7f74e17fa640 1 -- 172.21.15.26:0/1870136565 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v15) v1 ==== 115+0+0 (secure 0 0 0) 0x7f74d80544b0 con 0x7f74e40cb150 2023-12-19T16:54:52.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.199+0000 7f74ead7b640 1 -- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f74c8065b80 msgr2=0x7f74c8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:52.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.199+0000 7f74ead7b640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f74c8065b80 0x7f74c8068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f74d00af000 tx=0x7f74d00a7000 comp rx=0 tx=0).stop 2023-12-19T16:54:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.199+0000 7f74ead7b640 1 -- 172.21.15.26:0/1870136565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74e40cb150 msgr2=0x7f74e4153c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.199+0000 7f74ead7b640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74e40cb150 0x7f74e4153c70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f74d800e960 tx=0x7f74d800ee20 comp rx=0 tx=0).stop 2023-12-19T16:54:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.199+0000 7f74ead7b640 1 -- 172.21.15.26:0/1870136565 shutdown_connections 2023-12-19T16:54:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.199+0000 7f74ead7b640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74e415d530 0x7f74e41f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.199+0000 7f74ead7b640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f74c8065b80 0x7f74c8068040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.199+0000 7f74ead7b640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74e41599e0 0x7f74e40c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.199+0000 7f74ead7b640 1 --2- 172.21.15.26:0/1870136565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74e40cb150 0x7f74e4153c70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.199+0000 7f74ead7b640 1 -- 172.21.15.26:0/1870136565 >> 172.21.15.26:0/1870136565 conn(0x7f74e4066160 msgr2=0x7f74e40ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.199+0000 7f74ead7b640 1 -- 172.21.15.26:0/1870136565 shutdown_connections 2023-12-19T16:54:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.199+0000 7f74ead7b640 1 -- 172.21.15.26:0/1870136565 wait complete. 2023-12-19T16:54:52.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-12-19T16:54:52.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.359+0000 7f2b61a8c640 1 Processor -- start 2023-12-19T16:54:52.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.359+0000 7f2b61a8c640 1 -- start start 2023-12-19T16:54:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b61a8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5c15f370 0x7f2b5c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b61a8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b5c0ce740 0x7f2b5c0cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b61a8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5c06c970 con 0x7f2b5c155030 2023-12-19T16:54:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b61a8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5c06c790 con 0x7f2b5c0ce740 2023-12-19T16:54:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b61a8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c0cf130 con 0x7f2b5c15f370 2023-12-19T16:54:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b5b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2b5c155030 0x7f2b5c06c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46914/0 (socket says 172.21.15.26:46914) 2023-12-19T16:54:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b5b7fe640 1 -- 172.21.15.26:0/2667517493 learned_addr learned my addr 172.21.15.26:0/2667517493 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b5bfff640 1 --2- 172.21.15.26:0/2667517493 >> v2:172.21.15.26:3300/0 conn(0x7f2b5c0ce740 0x7f2b5c0cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b5affd640 1 --2- 172.21.15.26:0/2667517493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5c15f370 0x7f2b5c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b5bfff640 1 -- 172.21.15.26:0/2667517493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5c15f370 msgr2=0x7f2b5c15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b5bfff640 1 --2- 172.21.15.26:0/2667517493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5c15f370 0x7f2b5c15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b5bfff640 1 -- 172.21.15.26:0/2667517493 >> v1:172.21.15.26:6789/0 conn(0x7f2b5c155030 legacy=0x7f2b5c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b5bfff640 1 -- 172.21.15.26:0/2667517493 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c15b1f0 con 0x7f2b5c0ce740 2023-12-19T16:54:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.363+0000 7f2b5affd640 1 --2- 172.21.15.26:0/2667517493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5c15f370 0x7f2b5c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:54:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b5bfff640 1 --2- 172.21.15.26:0/2667517493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c0ce740 0x7f2b5c0cebf0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009d90 tx=0x7f2b5401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ef7122401d199d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b5a7fc640 1 -- 172.21.15.26:0/2667517493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b540a59e0 con 0x7f2b5c0ce740 2023-12-19T16:54:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b5a7fc640 1 -- 172.21.15.26:0/2667517493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b540a5b40 con 0x7f2b5c0ce740 2023-12-19T16:54:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b5a7fc640 1 -- 172.21.15.26:0/2667517493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b540ad440 con 0x7f2b5c0ce740 2023-12-19T16:54:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b61a8c640 1 -- 172.21.15.26:0/2667517493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c0ce740 msgr2=0x7f2b5c0cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b61a8c640 1 --2- 172.21.15.26:0/2667517493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c0ce740 0x7f2b5c0cebf0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009d90 tx=0x7f2b5401c920 comp rx=0 tx=0).stop 2023-12-19T16:54:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b61a8c640 1 -- 172.21.15.26:0/2667517493 shutdown_connections 2023-12-19T16:54:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b61a8c640 1 --2- 172.21.15.26:0/2667517493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c0ce740 0x7f2b5c0cebf0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b61a8c640 1 --2- 172.21.15.26:0/2667517493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5c15f370 0x7f2b5c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b61a8c640 1 -- 172.21.15.26:0/2667517493 >> 172.21.15.26:0/2667517493 conn(0x7f2b5c0658e0 msgr2=0x7f2b5c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b61a8c640 1 -- 172.21.15.26:0/2667517493 shutdown_connections 2023-12-19T16:54:52.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b61a8c640 1 -- 172.21.15.26:0/2667517493 wait complete. 2023-12-19T16:54:52.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b61a8c640 1 Processor -- start 2023-12-19T16:54:52.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.367+0000 7f2b61a8c640 1 -- start start 2023-12-19T16:54:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b61a8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b5c0ce740 0x7f2b5c070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b61a8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c155030 0x7f2b5c070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b61a8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5c15f370 0x7f2b5c1f9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b61a8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c1fd210 con 0x7f2b5c155030 2023-12-19T16:54:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b61a8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c1fc900 con 0x7f2b5c0ce740 2023-12-19T16:54:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b61a8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c1fa0f0 con 0x7f2b5c15f370 2023-12-19T16:54:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b5affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c155030 0x7f2b5c070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:52.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b5affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c155030 0x7f2b5c070ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54280/0 (socket says 172.21.15.26:54280) 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b5affd640 1 -- 172.21.15.26:0/436151932 learned_addr learned my addr 172.21.15.26:0/436151932 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b5bfff640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5c15f370 0x7f2b5c1f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b5affd640 1 -- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5c15f370 msgr2=0x7f2b5c1f9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b5b7fe640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b5c0ce740 0x7f2b5c070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b5affd640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5c15f370 0x7f2b5c1f9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b5affd640 1 -- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b5c0ce740 msgr2=0x7f2b5c070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b5affd640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b5c0ce740 0x7f2b5c070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b5affd640 1 -- 172.21.15.26:0/436151932 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b54009a70 con 0x7f2b5c155030 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b5bfff640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5c15f370 0x7f2b5c1f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b5b7fe640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b5c0ce740 0x7f2b5c070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b5affd640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c155030 0x7f2b5c070ba0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000c7f0 tx=0x7f2b5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b43fff640 1 -- 172.21.15.26:0/436151932 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50007820 con 0x7f2b5c155030 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b43fff640 1 -- 172.21.15.26:0/436151932 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b50007980 con 0x7f2b5c155030 2023-12-19T16:54:52.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b43fff640 1 -- 172.21.15.26:0/436151932 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b50007cf0 con 0x7f2b5c155030 2023-12-19T16:54:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b61a8c640 1 -- 172.21.15.26:0/436151932 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5c1fcb50 con 0x7f2b5c155030 2023-12-19T16:54:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b61a8c640 1 -- 172.21.15.26:0/436151932 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5c1fd630 con 0x7f2b5c155030 2023-12-19T16:54:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b61a8c640 1 -- 172.21.15.26:0/436151932 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5c074270 con 0x7f2b5c155030 2023-12-19T16:54:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.371+0000 7f2b43fff640 1 -- 172.21.15.26:0/436151932 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2b500113f0 con 0x7f2b5c155030 2023-12-19T16:54:52.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.375+0000 7f2b43fff640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b48065b80 0x7f2b48068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:52.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.375+0000 7f2b43fff640 1 -- 172.21.15.26:0/436151932 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f2b500838c0 con 0x7f2b5c155030 2023-12-19T16:54:52.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.375+0000 7f2b5b7fe640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b48065b80 0x7f2b48068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:52.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.375+0000 7f2b5b7fe640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b48065b80 0x7f2b48068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c005890 tx=0x7f2b4c009040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:52.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.379+0000 7f2b43fff640 1 -- 172.21.15.26:0/436151932 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2b50050ee0 con 0x7f2b5c155030 2023-12-19T16:54:52.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:52.599+0000 7f2b61a8c640 1 -- 172.21.15.26:0/436151932 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f2b5c151a60 con 0x7f2b5c155030 2023-12-19T16:54:53.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.139+0000 7f2b43fff640 1 -- 172.21.15.26:0/436151932 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v447) v1 ==== 158+0+0 (secure 0 0 0) 0x7f2b50054500 con 0x7f2b5c155030 2023-12-19T16:54:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.143+0000 7f2b61a8c640 1 -- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b48065b80 msgr2=0x7f2b48068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.143+0000 7f2b61a8c640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b48065b80 0x7f2b48068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c005890 tx=0x7f2b4c009040 comp rx=0 tx=0).stop 2023-12-19T16:54:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.143+0000 7f2b61a8c640 1 -- 172.21.15.26:0/436151932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c155030 msgr2=0x7f2b5c070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.143+0000 7f2b61a8c640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c155030 0x7f2b5c070ba0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2b5000c7f0 tx=0x7f2b5000ccb0 comp rx=0 tx=0).stop 2023-12-19T16:54:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.143+0000 7f2b61a8c640 1 -- 172.21.15.26:0/436151932 shutdown_connections 2023-12-19T16:54:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.143+0000 7f2b61a8c640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b48065b80 0x7f2b48068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.143+0000 7f2b61a8c640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b5c15f370 0x7f2b5c1f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.143+0000 7f2b61a8c640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c155030 0x7f2b5c070ba0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.143+0000 7f2b61a8c640 1 --2- 172.21.15.26:0/436151932 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b5c0ce740 0x7f2b5c070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.143+0000 7f2b61a8c640 1 -- 172.21.15.26:0/436151932 >> 172.21.15.26:0/436151932 conn(0x7f2b5c0658e0 msgr2=0x7f2b5c0b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.143+0000 7f2b61a8c640 1 -- 172.21.15.26:0/436151932 shutdown_connections 2023-12-19T16:54:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.143+0000 7f2b61a8c640 1 -- 172.21.15.26:0/436151932 wait complete. 2023-12-19T16:54:53.170 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-12-19T16:54:53.170 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-12-19T16:54:53.171 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-19T16:54:53.171 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-12-19T16:54:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b68611640 1 Processor -- start 2023-12-19T16:54:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b68611640 1 -- start start 2023-12-19T16:54:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b68611640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1b600b30d0 0x7f1b600b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b68611640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b6006cc80 con 0x7f1b6015c7f0 2023-12-19T16:54:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b68611640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b600b3a20 con 0x7f1b6006c7f0 2023-12-19T16:54:53.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b68611640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6006d770 con 0x7f1b600b30d0 2023-12-19T16:54:53.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b663ad640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1b6015c7f0 0x7f1b600adb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46924/0 (socket says 172.21.15.26:46924) 2023-12-19T16:54:53.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b663ad640 1 -- 172.21.15.26:0/3083034429 learned_addr learned my addr 172.21.15.26:0/3083034429 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:53.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b66bae640 1 --2- 172.21.15.26:0/3083034429 >> v2:172.21.15.160:3300/0 conn(0x7f1b600b30d0 0x7f1b600b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:53.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186750288 0 0) 0x7f1b6006cc80 con 0x7f1b6015c7f0 2023-12-19T16:54:53.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b4c005740 con 0x7f1b6015c7f0 2023-12-19T16:54:53.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512589765 0 0) 0x7f1b600b3a20 con 0x7f1b6006c7f0 2023-12-19T16:54:53.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b6006cc80 con 0x7f1b6006c7f0 2023-12-19T16:54:53.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1935676439 0 0) 0x7f1b4c005740 con 0x7f1b6015c7f0 2023-12-19T16:54:53.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 >> v2:172.21.15.160:3300/0 conn(0x7f1b600b30d0 msgr2=0x7f1b600b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:53.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b653ab640 1 --2- 172.21.15.26:0/3083034429 >> v2:172.21.15.160:3300/0 conn(0x7f1b600b30d0 0x7f1b600b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:53.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 >> v1:172.21.15.160:6789/0 conn(0x7f1b6006c7f0 legacy=0x7f1b600cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:53.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6006ced0 con 0x7f1b6015c7f0 2023-12-19T16:54:53.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f1b5c001680 con 0x7f1b6015c7f0 2023-12-19T16:54:53.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 >> v1:172.21.15.26:6789/0 conn(0x7f1b6015c7f0 legacy=0x7f1b600adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:53.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.315+0000 7f1b653ab640 1 --2- 172.21.15.26:0/3083034429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c005d20 0x7f1b4c006120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:53.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b653ab640 1 --2- 172.21.15.26:0/3083034429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b4c0069e0 0x7f1b4c006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:53.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b653ab640 1 --2- 172.21.15.26:0/3083034429 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b4c0073a0 0x7f1b4c00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6006d770 con 0x7f1b4c005d20 2023-12-19T16:54:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c00e220 con 0x7f1b4c0069e0 2023-12-19T16:54:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c00e3e0 con 0x7f1b4c0073a0 2023-12-19T16:54:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b663ad640 1 --2- 172.21.15.26:0/3083034429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c005d20 0x7f1b4c006120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b65bac640 1 --2- 172.21.15.26:0/3083034429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b4c0069e0 0x7f1b4c006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b66bae640 1 --2- 172.21.15.26:0/3083034429 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b4c0073a0 0x7f1b4c00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b663ad640 1 -- 172.21.15.26:0/3083034429 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b4c0073a0 msgr2=0x7f1b4c00dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b663ad640 1 --2- 172.21.15.26:0/3083034429 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b4c0073a0 0x7f1b4c00dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b663ad640 1 -- 172.21.15.26:0/3083034429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b4c0069e0 msgr2=0x7f1b4c006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b663ad640 1 --2- 172.21.15.26:0/3083034429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b4c0069e0 0x7f1b4c006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b663ad640 1 -- 172.21.15.26:0/3083034429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c005740 con 0x7f1b4c005d20 2023-12-19T16:54:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b65bac640 1 --2- 172.21.15.26:0/3083034429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b4c0069e0 0x7f1b4c006e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b66bae640 1 -- 172.21.15.26:0/3083034429 reap_dead start 2023-12-19T16:54:53.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b663ad640 1 --2- 172.21.15.26:0/3083034429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c005d20 0x7f1b4c006120 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00c160 tx=0x7f1b5c01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7ed05af74881d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:53.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0058e0 con 0x7f1b4c005d20 2023-12-19T16:54:53.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5c005a60 con 0x7f1b4c005d20 2023-12-19T16:54:53.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.319+0000 7f1b653ab640 1 -- 172.21.15.26:0/3083034429 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c005e50 con 0x7f1b4c005d20 2023-12-19T16:54:53.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 -- 172.21.15.26:0/3083034429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c005d20 msgr2=0x7f1b4c006120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:53.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 --2- 172.21.15.26:0/3083034429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c005d20 0x7f1b4c006120 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c00c160 tx=0x7f1b5c01cbc0 comp rx=0 tx=0).stop 2023-12-19T16:54:53.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 -- 172.21.15.26:0/3083034429 shutdown_connections 2023-12-19T16:54:53.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 --2- 172.21.15.26:0/3083034429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c005d20 0x7f1b4c006120 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:53.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 -- 172.21.15.26:0/3083034429 >> 172.21.15.26:0/3083034429 conn(0x7f1b600658e0 msgr2=0x7f1b60065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:53.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 -- 172.21.15.26:0/3083034429 shutdown_connections 2023-12-19T16:54:53.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 -- 172.21.15.26:0/3083034429 wait complete. 2023-12-19T16:54:53.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 Processor -- start 2023-12-19T16:54:53.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 -- start start 2023-12-19T16:54:53.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b4c005d20 0x7f1b6015ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:53.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b600ac850 0x7f1b600accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:53.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b601518c0 0x7f1b60151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:53.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60159f00 con 0x7f1b601518c0 2023-12-19T16:54:53.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b600ad210 con 0x7f1b600ac850 2023-12-19T16:54:53.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b68611640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60159280 con 0x7f1b4c005d20 2023-12-19T16:54:53.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b66bae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b601518c0 0x7f1b60151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:53.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b66bae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b601518c0 0x7f1b60151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54284/0 (socket says 172.21.15.26:54284) 2023-12-19T16:54:53.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b66bae640 1 -- 172.21.15.26:0/3235983139 learned_addr learned my addr 172.21.15.26:0/3235983139 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:53.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b65bac640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b600ac850 0x7f1b600accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:53.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.323+0000 7f1b663ad640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b4c005d20 0x7f1b6015ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:53.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b66bae640 1 -- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b4c005d20 msgr2=0x7f1b6015ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:53.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b66bae640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b4c005d20 0x7f1b6015ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:53.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b66bae640 1 -- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b600ac850 msgr2=0x7f1b600accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:53.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b66bae640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b600ac850 0x7f1b600accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:53.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b66bae640 1 -- 172.21.15.26:0/3235983139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b60159560 con 0x7f1b601518c0 2023-12-19T16:54:53.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b663ad640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b4c005d20 0x7f1b6015ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:53.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b66bae640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b601518c0 0x7f1b60151d40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1b600cf1b0 tx=0x7f1b580115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:53.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b56ffd640 1 -- 172.21.15.26:0/3235983139 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800e070 con 0x7f1b601518c0 2023-12-19T16:54:53.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b56ffd640 1 -- 172.21.15.26:0/3235983139 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b58011e10 con 0x7f1b601518c0 2023-12-19T16:54:53.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b68611640 1 -- 172.21.15.26:0/3235983139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b60159840 con 0x7f1b601518c0 2023-12-19T16:54:53.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b56ffd640 1 -- 172.21.15.26:0/3235983139 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b580039a0 con 0x7f1b601518c0 2023-12-19T16:54:53.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b68611640 1 -- 172.21.15.26:0/3235983139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6015b2c0 con 0x7f1b601518c0 2023-12-19T16:54:53.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b68611640 1 -- 172.21.15.26:0/3235983139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6015a040 con 0x7f1b601518c0 2023-12-19T16:54:53.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b56ffd640 1 -- 172.21.15.26:0/3235983139 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1b58003b00 con 0x7f1b601518c0 2023-12-19T16:54:53.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b56ffd640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b44065bb0 0x7f1b44068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:53.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b56ffd640 1 -- 172.21.15.26:0/3235983139 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f1b58083cd0 con 0x7f1b601518c0 2023-12-19T16:54:53.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.327+0000 7f1b663ad640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b44065bb0 0x7f1b44068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:53.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.331+0000 7f1b663ad640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b44065bb0 0x7f1b44068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0aa000 tx=0x7f1b5c001a00 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:53.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.331+0000 7f1b56ffd640 1 -- 172.21.15.26:0/3235983139 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1b6015a040 con 0x7f1b601518c0 2023-12-19T16:54:53.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:53.551+0000 7f1b68611640 1 -- 172.21.15.26:0/3235983139 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f1b6015a040 con 0x7f1b601518c0 2023-12-19T16:54:54.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.147+0000 7f1b56ffd640 1 -- 172.21.15.26:0/3235983139 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v448) v1 ==== 157+0+0 (secure 0 0 0) 0x7f1b6015a040 con 0x7f1b601518c0 2023-12-19T16:54:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.151+0000 7f1b68611640 1 -- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b44065bb0 msgr2=0x7f1b44068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.151+0000 7f1b68611640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b44065bb0 0x7f1b44068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0aa000 tx=0x7f1b5c001a00 comp rx=0 tx=0).stop 2023-12-19T16:54:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.151+0000 7f1b68611640 1 -- 172.21.15.26:0/3235983139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b601518c0 msgr2=0x7f1b60151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.151+0000 7f1b68611640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b601518c0 0x7f1b60151d40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1b600cf1b0 tx=0x7f1b580115a0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.151+0000 7f1b68611640 1 -- 172.21.15.26:0/3235983139 shutdown_connections 2023-12-19T16:54:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.151+0000 7f1b68611640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b601518c0 0x7f1b60151d40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.151+0000 7f1b68611640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b600ac850 0x7f1b600accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.151+0000 7f1b68611640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b44065bb0 0x7f1b44068070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.151+0000 7f1b68611640 1 --2- 172.21.15.26:0/3235983139 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b4c005d20 0x7f1b6015ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.151+0000 7f1b68611640 1 -- 172.21.15.26:0/3235983139 >> 172.21.15.26:0/3235983139 conn(0x7f1b600658e0 msgr2=0x7f1b6015cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:54.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.151+0000 7f1b68611640 1 -- 172.21.15.26:0/3235983139 shutdown_connections 2023-12-19T16:54:54.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.151+0000 7f1b68611640 1 -- 172.21.15.26:0/3235983139 wait complete. 2023-12-19T16:54:54.177 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T16:54:54.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.327+0000 7f498f76d640 1 Processor -- start 2023-12-19T16:54:54.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.327+0000 7f498f76d640 1 -- start start 2023-12-19T16:54:54.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.327+0000 7f498f76d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4988161230 0x7f49880ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:54.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.327+0000 7f498f76d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f498806c3f0 0x7f498806c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:54.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.327+0000 7f498f76d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49880b01b0 con 0x7f4988164e50 2023-12-19T16:54:54.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.327+0000 7f498f76d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49880c9e00 con 0x7f498806c3f0 2023-12-19T16:54:54.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.327+0000 7f498f76d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49880adfb0 con 0x7f4988161230 2023-12-19T16:54:54.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.327+0000 7f498cd08640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f498806c3f0 0x7f498806c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:54.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.327+0000 7f498d509640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4988161230 0x7f49880ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:54.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.327+0000 7f498d509640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4988161230 0x7f49880ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:47134/0 (socket says 172.21.15.26:47134) 2023-12-19T16:54:54.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.327+0000 7f498d509640 1 -- 172.21.15.26:0/518435206 learned_addr learned my addr 172.21.15.26:0/518435206 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:54.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.331+0000 7f498cd08640 1 -- 172.21.15.26:0/518435206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4988161230 msgr2=0x7f49880ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:54.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.331+0000 7f498cd08640 1 --2- 172.21.15.26:0/518435206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4988161230 0x7f49880ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.331+0000 7f498cd08640 1 -- 172.21.15.26:0/518435206 >> v1:172.21.15.160:6789/0 conn(0x7f4988164e50 legacy=0x7f49881652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:54.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.331+0000 7f498cd08640 1 -- 172.21.15.26:0/518435206 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49880ca4e0 con 0x7f498806c3f0 2023-12-19T16:54:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.331+0000 7f497ffff640 1 -- 172.21.15.26:0/518435206 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914096086 0 0) 0x7f49880b01b0 con 0x7f4988164e50 2023-12-19T16:54:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.331+0000 7f498d509640 1 --2- 172.21.15.26:0/518435206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4988161230 0x7f49880ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:54:54.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.331+0000 7f498cd08640 1 --2- 172.21.15.26:0/518435206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f498806c3f0 0x7f498806c850 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f49780097c0 tx=0x7f497801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22374b91afa7f2f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:54.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.331+0000 7f497ffff640 1 -- 172.21.15.26:0/518435206 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49780ac620 con 0x7f498806c3f0 2023-12-19T16:54:54.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.331+0000 7f497ffff640 1 -- 172.21.15.26:0/518435206 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49780ac780 con 0x7f498806c3f0 2023-12-19T16:54:54.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.331+0000 7f497ffff640 1 -- 172.21.15.26:0/518435206 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49780acb10 con 0x7f498806c3f0 2023-12-19T16:54:54.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 -- 172.21.15.26:0/518435206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f498806c3f0 msgr2=0x7f498806c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:54.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 --2- 172.21.15.26:0/518435206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f498806c3f0 0x7f498806c850 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f49780097c0 tx=0x7f497801c3c0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 -- 172.21.15.26:0/518435206 shutdown_connections 2023-12-19T16:54:54.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 --2- 172.21.15.26:0/518435206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f498806c3f0 0x7f498806c850 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 --2- 172.21.15.26:0/518435206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4988161230 0x7f49880ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 -- 172.21.15.26:0/518435206 >> 172.21.15.26:0/518435206 conn(0x7f49880ce730 msgr2=0x7f49880ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:54.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 -- 172.21.15.26:0/518435206 shutdown_connections 2023-12-19T16:54:54.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 -- 172.21.15.26:0/518435206 wait complete. 2023-12-19T16:54:54.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 Processor -- start 2023-12-19T16:54:54.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 -- start start 2023-12-19T16:54:54.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498806c3f0 0x7f498814a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:54.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4988161230 0x7f4988151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:54.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4988164e50 0x7f4988152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:54.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4988156580 con 0x7f498806c3f0 2023-12-19T16:54:54.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498815f390 con 0x7f4988161230 2023-12-19T16:54:54.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498f76d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498815f670 con 0x7f4988164e50 2023-12-19T16:54:54.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498d509640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498806c3f0 0x7f498814a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:54.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498d509640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498806c3f0 0x7f498814a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54290/0 (socket says 172.21.15.26:54290) 2023-12-19T16:54:54.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498d509640 1 -- 172.21.15.26:0/2766826348 learned_addr learned my addr 172.21.15.26:0/2766826348 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:54.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498dd0a640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4988164e50 0x7f4988152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:54.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498cd08640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4988161230 0x7f4988151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:54.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498d509640 1 -- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4988164e50 msgr2=0x7f4988152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:54.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498d509640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4988164e50 0x7f4988152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498d509640 1 -- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4988161230 msgr2=0x7f4988151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498d509640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4988161230 0x7f4988151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.335+0000 7f498d509640 1 -- 172.21.15.26:0/2766826348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49840094d0 con 0x7f498806c3f0 2023-12-19T16:54:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.339+0000 7f498d509640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498806c3f0 0x7f498814a2a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f498400e930 tx=0x7f498400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.339+0000 7f497dffb640 1 -- 172.21.15.26:0/2766826348 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498400ca00 con 0x7f498806c3f0 2023-12-19T16:54:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.339+0000 7f497dffb640 1 -- 172.21.15.26:0/2766826348 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498400cb60 con 0x7f498806c3f0 2023-12-19T16:54:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.339+0000 7f497dffb640 1 -- 172.21.15.26:0/2766826348 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984013470 con 0x7f498806c3f0 2023-12-19T16:54:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.339+0000 7f498f76d640 1 -- 172.21.15.26:0/2766826348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4978009420 con 0x7f498806c3f0 2023-12-19T16:54:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.339+0000 7f498f76d640 1 -- 172.21.15.26:0/2766826348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498815fc30 con 0x7f498806c3f0 2023-12-19T16:54:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.339+0000 7f498f76d640 1 -- 172.21.15.26:0/2766826348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4988074270 con 0x7f498806c3f0 2023-12-19T16:54:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.339+0000 7f497dffb640 1 -- 172.21.15.26:0/2766826348 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f49840135d0 con 0x7f498806c3f0 2023-12-19T16:54:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.339+0000 7f497dffb640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f496c065b80 0x7f496c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.339+0000 7f497dffb640 1 -- 172.21.15.26:0/2766826348 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f49840836c0 con 0x7f498806c3f0 2023-12-19T16:54:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.339+0000 7f498cd08640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f496c065b80 0x7f496c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.343+0000 7f498cd08640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f496c065b80 0x7f496c068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f49780ae000 tx=0x7f4978008040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:54.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.343+0000 7f497dffb640 1 -- 172.21.15.26:0/2766826348 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4984050ea0 con 0x7f498806c3f0 2023-12-19T16:54:54.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.499+0000 7f498f76d640 1 -- 172.21.15.26:0/2766826348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f49880b46b0 con 0x7f498806c3f0 2023-12-19T16:54:54.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.507+0000 7f497dffb640 1 -- 172.21.15.26:0/2766826348 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f49840544c0 con 0x7f498806c3f0 2023-12-19T16:54:54.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.507+0000 7f498f76d640 1 -- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f496c065b80 msgr2=0x7f496c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:54.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.507+0000 7f498f76d640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f496c065b80 0x7f496c068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f49780ae000 tx=0x7f4978008040 comp rx=0 tx=0).stop 2023-12-19T16:54:54.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.507+0000 7f498f76d640 1 -- 172.21.15.26:0/2766826348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498806c3f0 msgr2=0x7f498814a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:54.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.507+0000 7f498f76d640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498806c3f0 0x7f498814a2a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f498400e930 tx=0x7f498400edf0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.507+0000 7f498f76d640 1 -- 172.21.15.26:0/2766826348 shutdown_connections 2023-12-19T16:54:54.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.507+0000 7f498f76d640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f496c065b80 0x7f496c068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.507+0000 7f498f76d640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4988164e50 0x7f4988152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.507+0000 7f498f76d640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4988161230 0x7f4988151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.507+0000 7f498f76d640 1 --2- 172.21.15.26:0/2766826348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498806c3f0 0x7f498814a2a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.507+0000 7f498f76d640 1 -- 172.21.15.26:0/2766826348 >> 172.21.15.26:0/2766826348 conn(0x7f49880ce730 msgr2=0x7f49880ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:54.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.507+0000 7f498f76d640 1 -- 172.21.15.26:0/2766826348 shutdown_connections 2023-12-19T16:54:54.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.507+0000 7f498f76d640 1 -- 172.21.15.26:0/2766826348 wait complete. 2023-12-19T16:54:54.530 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T16:54:54.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:54:54.647 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:54:54.740 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T16:54:54.740 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:54:54.740 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:54:54.750 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:54:54.750 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:54:54.750 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:54:54.760 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:54:54.760 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:54:54.760 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:54:54.805 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:54:54.805 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:54:54.805 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:54:54.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:54:54.819 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T16:54:54.819 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:54:54.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.987+0000 7f4dd7466640 1 Processor -- start 2023-12-19T16:54:54.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.987+0000 7f4dd7466640 1 -- start start 2023-12-19T16:54:54.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.987+0000 7f4dd7466640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4dd0154220 0x7f4dd015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:54.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.987+0000 7f4dd7466640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4dd006b3d0 0x7f4dd00ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:54.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.987+0000 7f4dd7466640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4dd00cec40 0x7f4dd015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:54.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.987+0000 7f4dd7466640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd00cf0d0 con 0x7f4dd006b3d0 2023-12-19T16:54:54.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.991+0000 7f4dd7466640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd006bc30 con 0x7f4dd0154220 2023-12-19T16:54:54.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.991+0000 7f4dd7466640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00cc6e0 con 0x7f4dd00cec40 2023-12-19T16:54:54.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.991+0000 7f4dd4a01640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4dd006b3d0 0x7f4dd00ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:54.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.991+0000 7f4dd4a01640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4dd006b3d0 0x7f4dd00ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54304/0 (socket says 172.21.15.26:54304) 2023-12-19T16:54:54.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.991+0000 7f4dd4a01640 1 -- 172.21.15.26:0/750240708 learned_addr learned my addr 172.21.15.26:0/750240708 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:54.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.991+0000 7f4dd5a03640 1 --2- 172.21.15.26:0/750240708 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4dd00cec40 0x7f4dd015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:54.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.991+0000 7f4dd5202640 1 --2- 172.21.15.26:0/750240708 >> v2:172.21.15.160:3300/0 conn(0x7f4dd0154220 0x7f4dd015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:54.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.991+0000 7f4dd4a01640 1 -- 172.21.15.26:0/750240708 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4dd00cec40 msgr2=0x7f4dd015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:54.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.991+0000 7f4dd4a01640 1 --2- 172.21.15.26:0/750240708 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4dd00cec40 0x7f4dd015d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.991+0000 7f4dd4a01640 1 -- 172.21.15.26:0/750240708 >> v2:172.21.15.160:3300/0 conn(0x7f4dd0154220 msgr2=0x7f4dd015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:54.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.991+0000 7f4dd4a01640 1 --2- 172.21.15.26:0/750240708 >> v2:172.21.15.160:3300/0 conn(0x7f4dd0154220 0x7f4dd015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.991+0000 7f4dd4a01640 1 -- 172.21.15.26:0/750240708 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd015afc0 con 0x7f4dd006b3d0 2023-12-19T16:54:54.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.991+0000 7f4dd4a01640 1 --2- 172.21.15.26:0/750240708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd006b3d0 0x7f4dd00ce700 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00667b0 tx=0x7f4dc001c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d03fef94eb6753 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:54.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dc7fff640 1 -- 172.21.15.26:0/750240708 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00ab600 con 0x7f4dd006b3d0 2023-12-19T16:54:54.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dc7fff640 1 -- 172.21.15.26:0/750240708 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc00a7050 con 0x7f4dd006b3d0 2023-12-19T16:54:54.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dc7fff640 1 -- 172.21.15.26:0/750240708 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00ab980 con 0x7f4dd006b3d0 2023-12-19T16:54:54.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 -- 172.21.15.26:0/750240708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd006b3d0 msgr2=0x7f4dd00ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:54.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 --2- 172.21.15.26:0/750240708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd006b3d0 0x7f4dd00ce700 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4dd00667b0 tx=0x7f4dc001c210 comp rx=0 tx=0).stop 2023-12-19T16:54:54.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 -- 172.21.15.26:0/750240708 shutdown_connections 2023-12-19T16:54:54.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 --2- 172.21.15.26:0/750240708 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4dd00cec40 0x7f4dd015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 --2- 172.21.15.26:0/750240708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd006b3d0 0x7f4dd00ce700 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 --2- 172.21.15.26:0/750240708 >> v2:172.21.15.160:3300/0 conn(0x7f4dd0154220 0x7f4dd015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:54.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 -- 172.21.15.26:0/750240708 >> 172.21.15.26:0/750240708 conn(0x7f4dd00658e0 msgr2=0x7f4dd0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:54.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 -- 172.21.15.26:0/750240708 shutdown_connections 2023-12-19T16:54:54.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 -- 172.21.15.26:0/750240708 wait complete. 2023-12-19T16:54:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 Processor -- start 2023-12-19T16:54:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 -- start start 2023-12-19T16:54:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4dd006b3d0 0x7f4dd0150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dd00cec40 0x7f4dd00aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.995+0000 7f4dd7466640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0154220 0x7f4dd00af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd7466640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00ae010 con 0x7f4dd0154220 2023-12-19T16:54:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd7466640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00b2980 con 0x7f4dd00cec40 2023-12-19T16:54:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd7466640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd00ae2b0 con 0x7f4dd006b3d0 2023-12-19T16:54:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd5a03640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0154220 0x7f4dd00af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd5a03640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0154220 0x7f4dd00af2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54318/0 (socket says 172.21.15.26:54318) 2023-12-19T16:54:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd5a03640 1 -- 172.21.15.26:0/2440753736 learned_addr learned my addr 172.21.15.26:0/2440753736 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd5202640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4dd006b3d0 0x7f4dd0150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd4a01640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dd00cec40 0x7f4dd00aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:55.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd5a03640 1 -- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4dd006b3d0 msgr2=0x7f4dd0150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:55.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd5a03640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4dd006b3d0 0x7f4dd0150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd5a03640 1 -- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dd00cec40 msgr2=0x7f4dd00aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd5a03640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dd00cec40 0x7f4dd00aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd5a03640 1 -- 172.21.15.26:0/2440753736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc00094d0 con 0x7f4dd0154220 2023-12-19T16:54:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd5a03640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0154220 0x7f4dd00af2d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800eed0 tx=0x7f4dc800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dc5ffb640 1 -- 172.21.15.26:0/2440753736 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8010040 con 0x7f4dd0154220 2023-12-19T16:54:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dc5ffb640 1 -- 172.21.15.26:0/2440753736 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc8014400 con 0x7f4dd0154220 2023-12-19T16:54:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dc5ffb640 1 -- 172.21.15.26:0/2440753736 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8009ea0 con 0x7f4dd0154220 2023-12-19T16:54:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd7466640 1 -- 172.21.15.26:0/2440753736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd00ae590 con 0x7f4dd0154220 2023-12-19T16:54:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd7466640 1 -- 172.21.15.26:0/2440753736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd00af810 con 0x7f4dd0154220 2023-12-19T16:54:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dd7466640 1 -- 172.21.15.26:0/2440753736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd0074270 con 0x7f4dd0154220 2023-12-19T16:54:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:54.999+0000 7f4dc5ffb640 1 -- 172.21.15.26:0/2440753736 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4dc80147b0 con 0x7f4dd0154220 2023-12-19T16:54:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.003+0000 7f4dc5ffb640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4db4065b80 0x7f4db4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.003+0000 7f4dc5ffb640 1 -- 172.21.15.26:0/2440753736 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f4dc8083c80 con 0x7f4dd0154220 2023-12-19T16:54:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.003+0000 7f4dd5202640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4db4065b80 0x7f4db4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.003+0000 7f4dd5202640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4db4065b80 0x7f4db4068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4dd015d2c0 tx=0x7f4dcc009190 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:55.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.007+0000 7f4dc5ffb640 1 -- 172.21.15.26:0/2440753736 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4dc8051220 con 0x7f4dd0154220 2023-12-19T16:54:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.203+0000 7f4dd7466640 1 -- 172.21.15.26:0/2440753736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dd00aea40 con 0x7f4dd0154220 2023-12-19T16:54:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.203+0000 7f4dc5ffb640 1 -- 172.21.15.26:0/2440753736 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f4dc8018090 con 0x7f4dd0154220 2023-12-19T16:54:55.207 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:55.207 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4793,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1991431867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":1991431867},{"type":"v1","addr":"172.21.15.160:6837","nonce":1991431867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5029,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3495925867","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3495925867},{"type":"v1","addr":"172.21.15.160:6835","nonce":3495925867}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3757604412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3757604412},{"type":"v1","addr":"172.21.15.26:6833","nonce":3757604412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5907,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1077656165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1077656165},{"type":"v1","addr":"172.21.15.26:6835","nonce":1077656165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-12-19T16:54:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 19 2023-12-19T16:54:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.207+0000 7f4dd7466640 1 -- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4db4065b80 msgr2=0x7f4db4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.207+0000 7f4dd7466640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4db4065b80 0x7f4db4068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4dd015d2c0 tx=0x7f4dcc009190 comp rx=0 tx=0).stop 2023-12-19T16:54:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.207+0000 7f4dd7466640 1 -- 172.21.15.26:0/2440753736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0154220 msgr2=0x7f4dd00af2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.207+0000 7f4dd7466640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0154220 0x7f4dd00af2d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800eed0 tx=0x7f4dc800b480 comp rx=0 tx=0).stop 2023-12-19T16:54:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.207+0000 7f4dd7466640 1 -- 172.21.15.26:0/2440753736 shutdown_connections 2023-12-19T16:54:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.207+0000 7f4dd7466640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4dd0154220 0x7f4dd00af2d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.207+0000 7f4dd7466640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4dd00cec40 0x7f4dd00aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.207+0000 7f4dd7466640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4db4065b80 0x7f4db4068040 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.207+0000 7f4dd7466640 1 --2- 172.21.15.26:0/2440753736 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4dd006b3d0 0x7f4dd0150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.207+0000 7f4dd7466640 1 -- 172.21.15.26:0/2440753736 >> 172.21.15.26:0/2440753736 conn(0x7f4dd00658e0 msgr2=0x7f4dd014bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.207+0000 7f4dd7466640 1 -- 172.21.15.26:0/2440753736 shutdown_connections 2023-12-19T16:54:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.207+0000 7f4dd7466640 1 -- 172.21.15.26:0/2440753736 wait complete. 2023-12-19T16:54:55.229 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T16:54:55.230 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T16:54:55.230 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T16:54:55.230 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:54:55.230 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T16:54:55.230 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T16:54:55.230 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T16:54:55.230 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:54:55.231 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T16:54:55.231 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T16:54:55.231 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T16:54:55.231 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:54:55.231 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T16:54:55.231 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T16:54:55.231 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T16:54:55.232 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:54:55.243 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T16:54:55.243 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T16:54:55.244 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T16:54:55.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T16:54:55.246 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T16:54:55.246 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T16:54:55.248 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T16:54:55.248 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T16:54:55.292 INFO:tasks.ceph.mds.b:Started 2023-12-19T16:54:55.292 DEBUG:teuthology.parallel:result is None 2023-12-19T16:54:55.293 INFO:tasks.ceph.mds.a:Started 2023-12-19T16:54:55.293 DEBUG:teuthology.parallel:result is None 2023-12-19T16:54:55.293 INFO:tasks.ceph.mds.c:Started 2023-12-19T16:54:55.294 DEBUG:teuthology.parallel:result is None 2023-12-19T16:54:55.294 INFO:tasks.ceph.mds.d:Started 2023-12-19T16:54:55.295 DEBUG:teuthology.parallel:result is None 2023-12-19T16:54:55.295 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T16:54:55.380 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T16:54:55.383 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T16:54:55.387 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T16:54:55.387 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T16:54:55.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.443+0000 7f1265925640 1 Processor -- start 2023-12-19T16:54:55.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.443+0000 7f1265925640 1 -- start start 2023-12-19T16:54:55.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f1265925640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12600cee10 0x7f1260164980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:55.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f1265925640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126006bc30 con 0x7f1260154220 2023-12-19T16:54:55.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f1265925640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12600cc6e0 con 0x7f126006b3d0 2023-12-19T16:54:55.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f1265925640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126006b950 con 0x7f12600cee10 2023-12-19T16:54:55.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f125effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1260154220 0x7f126015aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46928/0 (socket says 172.21.15.26:46928) 2023-12-19T16:54:55.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f125effd640 1 -- 172.21.15.26:0/1395212697 learned_addr learned my addr 172.21.15.26:0/1395212697 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f125f7fe640 1 --2- 172.21.15.26:0/1395212697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12600cee10 0x7f1260164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f125dffb640 1 -- 172.21.15.26:0/1395212697 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483555040 0 0) 0x7f126006bc30 con 0x7f1260154220 2023-12-19T16:54:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f125dffb640 1 -- 172.21.15.26:0/1395212697 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1248003530 con 0x7f1260154220 2023-12-19T16:54:55.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f125dffb640 1 -- 172.21.15.26:0/1395212697 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810667299 0 0) 0x7f12600cc6e0 con 0x7f126006b3d0 2023-12-19T16:54:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f125dffb640 1 -- 172.21.15.26:0/1395212697 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f126006bc30 con 0x7f126006b3d0 2023-12-19T16:54:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f125f7fe640 1 -- 172.21.15.26:0/1395212697 >> v1:172.21.15.160:6789/0 conn(0x7f126006b3d0 legacy=0x7f12600ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f125f7fe640 1 -- 172.21.15.26:0/1395212697 >> v1:172.21.15.26:6789/0 conn(0x7f1260154220 legacy=0x7f126015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:55.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.447+0000 7f125f7fe640 1 -- 172.21.15.26:0/1395212697 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12600cc930 con 0x7f12600cee10 2023-12-19T16:54:55.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f125f7fe640 1 --2- 172.21.15.26:0/1395212697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12600cee10 0x7f1260164980 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1258009dc0 tx=0x7f125801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86e1f36c9ce338c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:55.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f125dffb640 1 -- 172.21.15.26:0/1395212697 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12580a5900 con 0x7f12600cee10 2023-12-19T16:54:55.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f125dffb640 1 -- 172.21.15.26:0/1395212697 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12580a5a60 con 0x7f12600cee10 2023-12-19T16:54:55.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f125dffb640 1 -- 172.21.15.26:0/1395212697 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12580a5df0 con 0x7f12600cee10 2023-12-19T16:54:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 -- 172.21.15.26:0/1395212697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12600cee10 msgr2=0x7f1260164980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 --2- 172.21.15.26:0/1395212697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12600cee10 0x7f1260164980 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1258009dc0 tx=0x7f125801c940 comp rx=0 tx=0).stop 2023-12-19T16:54:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 -- 172.21.15.26:0/1395212697 shutdown_connections 2023-12-19T16:54:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 --2- 172.21.15.26:0/1395212697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12600cee10 0x7f1260164980 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 -- 172.21.15.26:0/1395212697 >> 172.21.15.26:0/1395212697 conn(0x7f12600658e0 msgr2=0x7f1260065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 -- 172.21.15.26:0/1395212697 shutdown_connections 2023-12-19T16:54:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 -- 172.21.15.26:0/1395212697 wait complete. 2023-12-19T16:54:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 Processor -- start 2023-12-19T16:54:55.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 -- start start 2023-12-19T16:54:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f126006b3d0 0x7f1260154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12600cee10 0x7f12600b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1260154220 0x7f12600b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12600b1790 con 0x7f12600cee10 2023-12-19T16:54:55.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12600b1a70 con 0x7f1260154220 2023-12-19T16:54:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.451+0000 7f1265925640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12600b1d50 con 0x7f126006b3d0 2023-12-19T16:54:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f125e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12600cee10 0x7f12600b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f125effd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f126006b3d0 0x7f1260154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f125effd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f126006b3d0 0x7f1260154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:47210/0 (socket says 172.21.15.26:47210) 2023-12-19T16:54:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f125effd640 1 -- 172.21.15.26:0/1962057747 learned_addr learned my addr 172.21.15.26:0/1962057747 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f125f7fe640 1 --2- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1260154220 0x7f12600b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f125effd640 1 -- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1260154220 msgr2=0x7f12600b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f125effd640 1 --2- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1260154220 0x7f12600b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f125effd640 1 -- 172.21.15.26:0/1962057747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12600cee10 msgr2=0x7f12600b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f125effd640 1 --2- 172.21.15.26:0/1962057747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12600cee10 0x7f12600b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f125effd640 1 -- 172.21.15.26:0/1962057747 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1258009a70 con 0x7f126006b3d0 2023-12-19T16:54:55.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f125effd640 1 --2- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f126006b3d0 0x7f1260154870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f125000c420 tx=0x7f125000c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f1247fff640 1 -- 172.21.15.26:0/1962057747 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250005cf0 con 0x7f126006b3d0 2023-12-19T16:54:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f1247fff640 1 -- 172.21.15.26:0/1962057747 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1250005e50 con 0x7f126006b3d0 2023-12-19T16:54:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f1265925640 1 -- 172.21.15.26:0/1962057747 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12600b2030 con 0x7f126006b3d0 2023-12-19T16:54:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f1247fff640 1 -- 172.21.15.26:0/1962057747 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12500072d0 con 0x7f126006b3d0 2023-12-19T16:54:55.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f1265925640 1 -- 172.21.15.26:0/1962057747 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12600b32b0 con 0x7f126006b3d0 2023-12-19T16:54:55.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f1265925640 1 -- 172.21.15.26:0/1962057747 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1260074270 con 0x7f126006b3d0 2023-12-19T16:54:55.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f1247fff640 1 -- 172.21.15.26:0/1962057747 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1250007430 con 0x7f126006b3d0 2023-12-19T16:54:55.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.455+0000 7f1247fff640 1 --2- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f124c065b80 0x7f124c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:55.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.459+0000 7f1247fff640 1 -- 172.21.15.26:0/1962057747 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f1250082a10 con 0x7f126006b3d0 2023-12-19T16:54:55.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.459+0000 7f125e7fc640 1 --2- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f124c065b80 0x7f124c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:55.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.459+0000 7f125e7fc640 1 --2- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f124c065b80 0x7f124c068040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1260153510 tx=0x7f1254009040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:55.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.463+0000 7f1247fff640 1 -- 172.21.15.26:0/1962057747 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1250050150 con 0x7f126006b3d0 2023-12-19T16:54:55.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:55.631+0000 7f1265925640 1 -- 172.21.15.26:0/1962057747 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f12600b1e90 con 0x7f126006b3d0 2023-12-19T16:54:56.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.431+0000 7f1247fff640 1 -- 172.21.15.26:0/1962057747 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v57) v1 ==== 97+0+0 (secure 0 0 0) 0x7f12600b1e90 con 0x7f126006b3d0 2023-12-19T16:54:56.433 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2023-12-19T16:54:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.431+0000 7f1265925640 1 -- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f124c065b80 msgr2=0x7f124c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.431+0000 7f1265925640 1 --2- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f124c065b80 0x7f124c068040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1260153510 tx=0x7f1254009040 comp rx=0 tx=0).stop 2023-12-19T16:54:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.431+0000 7f1265925640 1 -- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f126006b3d0 msgr2=0x7f1260154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.431+0000 7f1265925640 1 --2- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f126006b3d0 0x7f1260154870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f125000c420 tx=0x7f125000c8e0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.431+0000 7f1265925640 1 -- 172.21.15.26:0/1962057747 shutdown_connections 2023-12-19T16:54:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.431+0000 7f1265925640 1 --2- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1260154220 0x7f12600b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.431+0000 7f1265925640 1 --2- 172.21.15.26:0/1962057747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12600cee10 0x7f12600b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.431+0000 7f1265925640 1 --2- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f124c065b80 0x7f124c068040 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.431+0000 7f1265925640 1 --2- 172.21.15.26:0/1962057747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f126006b3d0 0x7f1260154870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.431+0000 7f1265925640 1 -- 172.21.15.26:0/1962057747 >> 172.21.15.26:0/1962057747 conn(0x7f12600658e0 msgr2=0x7f1260152360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.431+0000 7f1265925640 1 -- 172.21.15.26:0/1962057747 shutdown_connections 2023-12-19T16:54:56.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.431+0000 7f1265925640 1 -- 172.21.15.26:0/1962057747 wait complete. 2023-12-19T16:54:56.460 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T16:54:56.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.603+0000 7f4fa6b96640 1 Processor -- start 2023-12-19T16:54:56.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4fa6b96640 1 -- start start 2023-12-19T16:54:56.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4fa6b96640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4fa0154220 0x7f4fa015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:56.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4fa6b96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fa006b3d0 0x7f4fa00ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:56.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4fa6b96640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa006bc30 con 0x7f4fa00cec40 2023-12-19T16:54:56.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4fa6b96640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa00cf0d0 con 0x7f4fa0154220 2023-12-19T16:54:56.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4fa6b96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa00cc6e0 con 0x7f4fa006b3d0 2023-12-19T16:54:56.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4fa5133640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4fa00cec40 0x7f4fa015d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47602/0 (socket says 172.21.15.26:47602) 2023-12-19T16:54:56.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4fa5133640 1 -- 172.21.15.26:0/419485026 learned_addr learned my addr 172.21.15.26:0/419485026 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:56.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4fa4932640 1 --2- 172.21.15.26:0/419485026 >> v2:172.21.15.160:3300/0 conn(0x7f4fa0154220 0x7f4fa015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:56.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4f9ffff640 1 --2- 172.21.15.26:0/419485026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fa006b3d0 0x7f4fa00ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:56.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4f9f7fe640 1 -- 172.21.15.26:0/419485026 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528510001 0 0) 0x7f4fa006bc30 con 0x7f4fa00cec40 2023-12-19T16:54:56.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4f9f7fe640 1 -- 172.21.15.26:0/419485026 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f90003530 con 0x7f4fa00cec40 2023-12-19T16:54:56.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4fa4932640 1 -- 172.21.15.26:0/419485026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fa006b3d0 msgr2=0x7f4fa00ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:56.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4fa4932640 1 --2- 172.21.15.26:0/419485026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fa006b3d0 0x7f4fa00ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4fa4932640 1 -- 172.21.15.26:0/419485026 >> v1:172.21.15.26:6789/0 conn(0x7f4fa00cec40 legacy=0x7f4fa015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:56.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4fa4932640 1 -- 172.21.15.26:0/419485026 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa015afc0 con 0x7f4fa0154220 2023-12-19T16:54:56.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.607+0000 7f4f9ffff640 1 --2- 172.21.15.26:0/419485026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fa006b3d0 0x7f4fa00ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:54:56.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4fa4932640 1 --2- 172.21.15.26:0/419485026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fa0154220 0x7f4fa015aa80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4f940097c0 tx=0x7f4f9401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc06de629dcc63cd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:56.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4f9f7fe640 1 -- 172.21.15.26:0/419485026 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f940ab600 con 0x7f4fa0154220 2023-12-19T16:54:56.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4f9f7fe640 1 -- 172.21.15.26:0/419485026 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f940a7050 con 0x7f4fa0154220 2023-12-19T16:54:56.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4f9f7fe640 1 -- 172.21.15.26:0/419485026 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f940ab960 con 0x7f4fa0154220 2023-12-19T16:54:56.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4fa6b96640 1 -- 172.21.15.26:0/419485026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fa0154220 msgr2=0x7f4fa015aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:56.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4fa6b96640 1 --2- 172.21.15.26:0/419485026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fa0154220 0x7f4fa015aa80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4f940097c0 tx=0x7f4f9401c450 comp rx=0 tx=0).stop 2023-12-19T16:54:56.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4fa6b96640 1 -- 172.21.15.26:0/419485026 shutdown_connections 2023-12-19T16:54:56.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4fa6b96640 1 --2- 172.21.15.26:0/419485026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fa006b3d0 0x7f4fa00ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4fa6b96640 1 --2- 172.21.15.26:0/419485026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fa0154220 0x7f4fa015aa80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4fa6b96640 1 -- 172.21.15.26:0/419485026 >> 172.21.15.26:0/419485026 conn(0x7f4fa00658e0 msgr2=0x7f4fa0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:56.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4fa6b96640 1 -- 172.21.15.26:0/419485026 shutdown_connections 2023-12-19T16:54:56.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4fa6b96640 1 -- 172.21.15.26:0/419485026 wait complete. 2023-12-19T16:54:56.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4fa6b96640 1 Processor -- start 2023-12-19T16:54:56.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.611+0000 7f4fa6b96640 1 -- start start 2023-12-19T16:54:56.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa6b96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa006b3d0 0x7f4fa006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:56.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa6b96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fa00cec40 0x7f4fa0149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:56.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa6b96640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fa0154220 0x7f4fa014d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:56.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa6b96640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa00702c0 con 0x7f4fa006b3d0 2023-12-19T16:54:56.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa6b96640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa0150210 con 0x7f4fa0154220 2023-12-19T16:54:56.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa6b96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa0151ea0 con 0x7f4fa00cec40 2023-12-19T16:54:56.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa4932640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa006b3d0 0x7f4fa006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:56.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa4932640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa006b3d0 0x7f4fa006fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49680/0 (socket says 172.21.15.26:49680) 2023-12-19T16:54:56.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa4932640 1 -- 172.21.15.26:0/4208865928 learned_addr learned my addr 172.21.15.26:0/4208865928 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:56.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4f9ffff640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fa00cec40 0x7f4fa0149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:56.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa5133640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fa0154220 0x7f4fa014d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:56.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa4932640 1 -- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fa00cec40 msgr2=0x7f4fa0149c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:56.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa4932640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fa00cec40 0x7f4fa0149c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa4932640 1 -- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fa0154220 msgr2=0x7f4fa014d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:56.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa4932640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fa0154220 0x7f4fa014d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa4932640 1 -- 172.21.15.26:0/4208865928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c009420 con 0x7f4fa006b3d0 2023-12-19T16:54:56.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa4932640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa006b3d0 0x7f4fa006fd80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4f94000c10 tx=0x7f4f940191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:56.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4f9d7fa640 1 -- 172.21.15.26:0/4208865928 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f94003ab0 con 0x7f4fa006b3d0 2023-12-19T16:54:56.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4f9d7fa640 1 -- 172.21.15.26:0/4208865928 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f94003c10 con 0x7f4fa006b3d0 2023-12-19T16:54:56.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4f9d7fa640 1 -- 172.21.15.26:0/4208865928 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f940ab400 con 0x7f4fa006b3d0 2023-12-19T16:54:56.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa6b96640 1 -- 172.21.15.26:0/4208865928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f94009420 con 0x7f4fa006b3d0 2023-12-19T16:54:56.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa6b96640 1 -- 172.21.15.26:0/4208865928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa0151570 con 0x7f4fa006b3d0 2023-12-19T16:54:56.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.615+0000 7f4fa6b96640 1 -- 172.21.15.26:0/4208865928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa0074270 con 0x7f4fa006b3d0 2023-12-19T16:54:56.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.619+0000 7f4f9d7fa640 1 -- 172.21.15.26:0/4208865928 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4f940b2040 con 0x7f4fa006b3d0 2023-12-19T16:54:56.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.619+0000 7f4f9d7fa640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f84065bb0 0x7f4f84068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:56.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.619+0000 7f4f9d7fa640 1 -- 172.21.15.26:0/4208865928 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f4f9411d080 con 0x7f4fa006b3d0 2023-12-19T16:54:56.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.619+0000 7f4f9ffff640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f84065bb0 0x7f4f84068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:56.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.619+0000 7f4f9ffff640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f84065bb0 0x7f4f84068070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0150410 tx=0x7f4f8c009190 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:56.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.623+0000 7f4f9d7fa640 1 -- 172.21.15.26:0/4208865928 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4f940ea8b0 con 0x7f4fa006b3d0 2023-12-19T16:54:56.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.831+0000 7f4fa6b96640 1 -- 172.21.15.26:0/4208865928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4fa014a170 con 0x7f4fa006b3d0 2023-12-19T16:54:56.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.831+0000 7f4f9d7fa640 1 -- 172.21.15.26:0/4208865928 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4f940b0090 con 0x7f4fa006b3d0 2023-12-19T16:54:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:56.835 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T16:54:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2023-12-19T16:54:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.835 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2023-12-19T16:54:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg==", 2023-12-19T16:54:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:54:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:54:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:54:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2023-12-19T16:54:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg==", 2023-12-19T16:54:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.836 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg==", 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.837 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw==", 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2023-12-19T16:54:56.838 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig==", 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA==", 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.839 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ==", 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ==", 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.840 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A==", 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.841 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ==", 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg==", 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:54:56.842 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg==", 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2023-12-19T16:54:56.843 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ==", 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ==", 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:54:56.844 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ==", 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2023-12-19T16:54:56.845 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ==", 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlT5ynDxAA7Sjbdwrc1RyAXJboysB54Q==", 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:54:56.846 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA==", 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:56.847 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2023-12-19T16:54:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg==", 2023-12-19T16:54:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:54:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:54:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T16:54:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:54:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:56.848 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-12-19T16:54:56.848 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T16:54:56.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.835+0000 7f4fa6b96640 1 -- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f84065bb0 msgr2=0x7f4f84068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:56.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.835+0000 7f4fa6b96640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f84065bb0 0x7f4f84068070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0150410 tx=0x7f4f8c009190 comp rx=0 tx=0).stop 2023-12-19T16:54:56.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.835+0000 7f4fa6b96640 1 -- 172.21.15.26:0/4208865928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa006b3d0 msgr2=0x7f4fa006fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:56.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.835+0000 7f4fa6b96640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa006b3d0 0x7f4fa006fd80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4f94000c10 tx=0x7f4f940191f0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.835+0000 7f4fa6b96640 1 -- 172.21.15.26:0/4208865928 shutdown_connections 2023-12-19T16:54:56.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.835+0000 7f4fa6b96640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4fa0154220 0x7f4fa014d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.835+0000 7f4fa6b96640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f84065bb0 0x7f4f84068070 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.835+0000 7f4fa6b96640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4fa00cec40 0x7f4fa0149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.835+0000 7f4fa6b96640 1 --2- 172.21.15.26:0/4208865928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa006b3d0 0x7f4fa006fd80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:56.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.835+0000 7f4fa6b96640 1 -- 172.21.15.26:0/4208865928 >> 172.21.15.26:0/4208865928 conn(0x7f4fa00658e0 msgr2=0x7f4fa00adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:56.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.835+0000 7f4fa6b96640 1 -- 172.21.15.26:0/4208865928 shutdown_connections 2023-12-19T16:54:56.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.835+0000 7f4fa6b96640 1 -- 172.21.15.26:0/4208865928 wait complete. 2023-12-19T16:54:56.857 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:54:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ade33640 1 Processor -- start 2023-12-19T16:54:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ade33640 1 -- start start 2023-12-19T16:54:57.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ade33640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f43a80b2030 0x7f43a8071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ade33640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43a80b3960 0x7f43a80b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ade33640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a815e7c0 con 0x7f43a80affe0 2023-12-19T16:54:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ade33640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a80c97a0 con 0x7f43a80b2030 2023-12-19T16:54:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ade33640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a8072360 con 0x7f43a80b3960 2023-12-19T16:54:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ace31640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f43a80b2030 0x7f43a8071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ace31640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f43a80b2030 0x7f43a8071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49694/0 (socket says 172.21.15.26:49694) 2023-12-19T16:54:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ace31640 1 -- 172.21.15.26:0/3619234696 learned_addr learned my addr 172.21.15.26:0/3619234696 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43a7fff640 1 --1- 172.21.15.26:0/3619234696 >> v1:172.21.15.26:6789/0 conn(0x7f43a80affe0 0x7f43a80c9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47618/0 (socket says 172.21.15.26:47618) 2023-12-19T16:54:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ad632640 1 --2- 172.21.15.26:0/3619234696 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43a80b3960 0x7f43a80b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ace31640 1 -- 172.21.15.26:0/3619234696 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43a80b3960 msgr2=0x7f43a80b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ace31640 1 --2- 172.21.15.26:0/3619234696 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43a80b3960 0x7f43a80b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ace31640 1 -- 172.21.15.26:0/3619234696 >> v1:172.21.15.26:6789/0 conn(0x7f43a80affe0 legacy=0x7f43a80c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43ace31640 1 -- 172.21.15.26:0/3619234696 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a80b04c0 con 0x7f43a80b2030 2023-12-19T16:54:57.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:56.999+0000 7f43a77fe640 1 -- 172.21.15.26:0/3619234696 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213883050 0 0) 0x7f43a815e7c0 con 0x7f43a80affe0 2023-12-19T16:54:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43ace31640 1 --2- 172.21.15.26:0/3619234696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a80b2030 0x7f43a8071e20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f439c0098a0 tx=0x7f439c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50dd4ba69debae9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43a77fe640 1 -- 172.21.15.26:0/3619234696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c0aa650 con 0x7f43a80b2030 2023-12-19T16:54:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43a77fe640 1 -- 172.21.15.26:0/3619234696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c0aa7b0 con 0x7f43a80b2030 2023-12-19T16:54:57.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43a77fe640 1 -- 172.21.15.26:0/3619234696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c0aab40 con 0x7f43a80b2030 2023-12-19T16:54:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43ade33640 1 -- 172.21.15.26:0/3619234696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a80b2030 msgr2=0x7f43a8071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43ade33640 1 --2- 172.21.15.26:0/3619234696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a80b2030 0x7f43a8071e20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f439c0098a0 tx=0x7f439c01c4f0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43ade33640 1 -- 172.21.15.26:0/3619234696 shutdown_connections 2023-12-19T16:54:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43ade33640 1 --2- 172.21.15.26:0/3619234696 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43a80b3960 0x7f43a80b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43ade33640 1 --2- 172.21.15.26:0/3619234696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a80b2030 0x7f43a8071e20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43ade33640 1 -- 172.21.15.26:0/3619234696 >> 172.21.15.26:0/3619234696 conn(0x7f43a806b0f0 msgr2=0x7f43a806b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43ade33640 1 -- 172.21.15.26:0/3619234696 shutdown_connections 2023-12-19T16:54:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43ade33640 1 -- 172.21.15.26:0/3619234696 wait complete. 2023-12-19T16:54:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43ade33640 1 Processor -- start 2023-12-19T16:54:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.003+0000 7f43ade33640 1 -- start start 2023-12-19T16:54:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ade33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a80affe0 0x7f43a814c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ade33640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43a80b2030 0x7f43a8159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ade33640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43a80b3960 0x7f43a8159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ade33640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80c73b0 con 0x7f43a80affe0 2023-12-19T16:54:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ade33640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a814caa0 con 0x7f43a80b3960 2023-12-19T16:54:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ade33640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80c70d0 con 0x7f43a80b2030 2023-12-19T16:54:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ace31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a80affe0 0x7f43a814c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ace31640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a80affe0 0x7f43a814c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49700/0 (socket says 172.21.15.26:49700) 2023-12-19T16:54:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ace31640 1 -- 172.21.15.26:0/1165404025 learned_addr learned my addr 172.21.15.26:0/1165404025 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43a7fff640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43a80b2030 0x7f43a8159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ad632640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43a80b3960 0x7f43a8159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ace31640 1 -- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43a80b2030 msgr2=0x7f43a8159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ace31640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43a80b2030 0x7f43a8159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ace31640 1 -- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43a80b3960 msgr2=0x7f43a8159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ace31640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43a80b3960 0x7f43a8159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ace31640 1 -- 172.21.15.26:0/1165404025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c0094d0 con 0x7f43a80affe0 2023-12-19T16:54:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ad632640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43a80b3960 0x7f43a8159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:54:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ace31640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a80affe0 0x7f43a814c560 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f439c0ac040 tx=0x7f439c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43a57fa640 1 -- 172.21.15.26:0/1165404025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c0b3800 con 0x7f43a80affe0 2023-12-19T16:54:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43a57fa640 1 -- 172.21.15.26:0/1165404025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c0b3980 con 0x7f43a80affe0 2023-12-19T16:54:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43a57fa640 1 -- 172.21.15.26:0/1165404025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c0b3d30 con 0x7f43a80affe0 2023-12-19T16:54:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ade33640 1 -- 172.21.15.26:0/1165404025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a80c5740 con 0x7f43a80affe0 2023-12-19T16:54:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ade33640 1 -- 172.21.15.26:0/1165404025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a80c5d00 con 0x7f43a80affe0 2023-12-19T16:54:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.007+0000 7f43ade33640 1 -- 172.21.15.26:0/1165404025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a80c55a0 con 0x7f43a80affe0 2023-12-19T16:54:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.011+0000 7f43a57fa640 1 -- 172.21.15.26:0/1165404025 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f439c0b2380 con 0x7f43a80affe0 2023-12-19T16:54:57.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.011+0000 7f43a57fa640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f438c065b80 0x7f438c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.011+0000 7f43a57fa640 1 -- 172.21.15.26:0/1165404025 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f439c11db60 con 0x7f43a80affe0 2023-12-19T16:54:57.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.011+0000 7f43a7fff640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f438c065b80 0x7f438c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.011+0000 7f43a7fff640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f438c065b80 0x7f438c068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43a806f400 tx=0x7f4394007c10 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:57.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.015+0000 7f43a57fa640 1 -- 172.21.15.26:0/1165404025 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f43a80c55a0 con 0x7f43a80affe0 2023-12-19T16:54:57.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.175+0000 7f43ade33640 1 -- 172.21.15.26:0/1165404025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43a80c55a0 con 0x7f43a80affe0 2023-12-19T16:54:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.175+0000 7f43a57fa640 1 -- 172.21.15.26:0/1165404025 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+13494 (secure 0 0 0) 0x7f43a80c55a0 con 0x7f43a80affe0 2023-12-19T16:54:57.179 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:57.179 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":57,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:54:56.422916+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:54:57.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.179+0000 7f43ade33640 1 -- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f438c065b80 msgr2=0x7f438c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.179+0000 7f43ade33640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f438c065b80 0x7f438c068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43a806f400 tx=0x7f4394007c10 comp rx=0 tx=0).stop 2023-12-19T16:54:57.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.179+0000 7f43ade33640 1 -- 172.21.15.26:0/1165404025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a80affe0 msgr2=0x7f43a814c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.179+0000 7f43ade33640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a80affe0 0x7f43a814c560 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f439c0ac040 tx=0x7f439c01ad30 comp rx=0 tx=0).stop 2023-12-19T16:54:57.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.179+0000 7f43ade33640 1 -- 172.21.15.26:0/1165404025 shutdown_connections 2023-12-19T16:54:57.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.179+0000 7f43ade33640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43a80b3960 0x7f43a8159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.179+0000 7f43ade33640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f438c065b80 0x7f438c068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.179+0000 7f43ade33640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43a80b2030 0x7f43a8159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.179+0000 7f43ade33640 1 --2- 172.21.15.26:0/1165404025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a80affe0 0x7f43a814c560 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.179+0000 7f43ade33640 1 -- 172.21.15.26:0/1165404025 >> 172.21.15.26:0/1165404025 conn(0x7f43a806b0f0 msgr2=0x7f43a80cedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:57.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.179+0000 7f43ade33640 1 -- 172.21.15.26:0/1165404025 shutdown_connections 2023-12-19T16:54:57.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.179+0000 7f43ade33640 1 -- 172.21.15.26:0/1165404025 wait complete. 2023-12-19T16:54:57.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:54:57.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:54:57.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.343+0000 7f205e4d7640 1 Processor -- start 2023-12-19T16:54:57.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.343+0000 7f205e4d7640 1 -- start start 2023-12-19T16:54:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f205e4d7640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2058160f80 0x7f20580b0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f205e4d7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f205806b3b0 0x7f205806b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f205e4d7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205806bd40 con 0x7f2058162780 2023-12-19T16:54:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f205e4d7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20580aec00 con 0x7f205806b3b0 2023-12-19T16:54:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f205e4d7640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20580af4a0 con 0x7f2058160f80 2023-12-19T16:54:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f205dcd6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f205806b3b0 0x7f205806b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f205ccd4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2058162780 0x7f20580cb910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47634/0 (socket says 172.21.15.26:47634) 2023-12-19T16:54:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f205ccd4640 1 -- 172.21.15.26:0/2958368732 learned_addr learned my addr 172.21.15.26:0/2958368732 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f205d4d5640 1 --2- 172.21.15.26:0/2958368732 >> v2:172.21.15.160:3300/0 conn(0x7f2058160f80 0x7f20580b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f205dcd6640 1 -- 172.21.15.26:0/2958368732 >> v2:172.21.15.160:3300/0 conn(0x7f2058160f80 msgr2=0x7f20580b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f205dcd6640 1 --2- 172.21.15.26:0/2958368732 >> v2:172.21.15.160:3300/0 conn(0x7f2058160f80 0x7f20580b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f205dcd6640 1 -- 172.21.15.26:0/2958368732 >> v1:172.21.15.26:6789/0 conn(0x7f2058162780 legacy=0x7f20580cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f205dcd6640 1 -- 172.21.15.26:0/2958368732 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20580b0fe0 con 0x7f205806b3b0 2023-12-19T16:54:57.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.347+0000 7f204ffff640 1 -- 172.21.15.26:0/2958368732 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571624443 0 0) 0x7f2048001120 con 0x7f2058162780 2023-12-19T16:54:57.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205dcd6640 1 --2- 172.21.15.26:0/2958368732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205806b3b0 0x7f205806b800 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f205806c0e0 tx=0x7f205001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d39dcb90ead19c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:57.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f204ffff640 1 -- 172.21.15.26:0/2958368732 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20500a5910 con 0x7f205806b3b0 2023-12-19T16:54:57.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f204ffff640 1 -- 172.21.15.26:0/2958368732 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20500a5a70 con 0x7f205806b3b0 2023-12-19T16:54:57.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f204ffff640 1 -- 172.21.15.26:0/2958368732 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20500a5e00 con 0x7f205806b3b0 2023-12-19T16:54:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 -- 172.21.15.26:0/2958368732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205806b3b0 msgr2=0x7f205806b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 --2- 172.21.15.26:0/2958368732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205806b3b0 0x7f205806b800 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f205806c0e0 tx=0x7f205001c850 comp rx=0 tx=0).stop 2023-12-19T16:54:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 -- 172.21.15.26:0/2958368732 shutdown_connections 2023-12-19T16:54:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 --2- 172.21.15.26:0/2958368732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f205806b3b0 0x7f205806b800 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 --2- 172.21.15.26:0/2958368732 >> v2:172.21.15.160:3300/0 conn(0x7f2058160f80 0x7f20580b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 -- 172.21.15.26:0/2958368732 >> 172.21.15.26:0/2958368732 conn(0x7f20580658e0 msgr2=0x7f2058065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 -- 172.21.15.26:0/2958368732 shutdown_connections 2023-12-19T16:54:57.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 -- 172.21.15.26:0/2958368732 wait complete. 2023-12-19T16:54:57.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 Processor -- start 2023-12-19T16:54:57.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 -- start start 2023-12-19T16:54:57.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f205806b3b0 0x7f20581508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2058160f80 0x7f2058154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2058162780 0x7f205806f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2058154a90 con 0x7f2058162780 2023-12-19T16:54:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2058156800 con 0x7f2058160f80 2023-12-19T16:54:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.351+0000 7f205e4d7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2058156aa0 con 0x7f205806b3b0 2023-12-19T16:54:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205dcd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2058162780 0x7f205806f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205dcd6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2058162780 0x7f205806f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49718/0 (socket says 172.21.15.26:49718) 2023-12-19T16:54:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205dcd6640 1 -- 172.21.15.26:0/4188688575 learned_addr learned my addr 172.21.15.26:0/4188688575 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205ccd4640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2058160f80 0x7f2058154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205d4d5640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f205806b3b0 0x7f20581508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205dcd6640 1 -- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f205806b3b0 msgr2=0x7f20581508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205dcd6640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f205806b3b0 0x7f20581508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205dcd6640 1 -- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2058160f80 msgr2=0x7f2058154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205dcd6640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2058160f80 0x7f2058154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205dcd6640 1 -- 172.21.15.26:0/4188688575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2050009a70 con 0x7f2058162780 2023-12-19T16:54:57.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205dcd6640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2058162780 0x7f205806f1f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f205001c890 tx=0x7f20500195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:57.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f204dffb640 1 -- 172.21.15.26:0/4188688575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20500ad850 con 0x7f2058162780 2023-12-19T16:54:57.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f204dffb640 1 -- 172.21.15.26:0/4188688575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20500ad9b0 con 0x7f2058162780 2023-12-19T16:54:57.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205e4d7640 1 -- 172.21.15.26:0/4188688575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20581624f0 con 0x7f2058162780 2023-12-19T16:54:57.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f204dffb640 1 -- 172.21.15.26:0/4188688575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20500add60 con 0x7f2058162780 2023-12-19T16:54:57.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205e4d7640 1 -- 172.21.15.26:0/4188688575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205806f730 con 0x7f2058162780 2023-12-19T16:54:57.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f205e4d7640 1 -- 172.21.15.26:0/4188688575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2058154c00 con 0x7f2058162780 2023-12-19T16:54:57.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.355+0000 7f204dffb640 1 -- 172.21.15.26:0/4188688575 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f20500b4070 con 0x7f2058162780 2023-12-19T16:54:57.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.359+0000 7f204dffb640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f203c065b80 0x7f203c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.359+0000 7f204dffb640 1 -- 172.21.15.26:0/4188688575 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f205011db60 con 0x7f2058162780 2023-12-19T16:54:57.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.359+0000 7f205d4d5640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f203c065b80 0x7f203c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.359+0000 7f205d4d5640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f203c065b80 0x7f203c068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2054007a60 tx=0x7f2054008040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:57.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.359+0000 7f204dffb640 1 -- 172.21.15.26:0/4188688575 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f20500eb5b0 con 0x7f2058162780 2023-12-19T16:54:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.519+0000 7f205e4d7640 1 -- 172.21.15.26:0/4188688575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2058156f50 con 0x7f2058162780 2023-12-19T16:54:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.519+0000 7f204dffb640 1 -- 172.21.15.26:0/4188688575 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20500eebd0 con 0x7f2058162780 2023-12-19T16:54:57.523 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:54:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.523+0000 7f205e4d7640 1 -- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f203c065b80 msgr2=0x7f203c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.523+0000 7f205e4d7640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f203c065b80 0x7f203c068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2054007a60 tx=0x7f2054008040 comp rx=0 tx=0).stop 2023-12-19T16:54:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.523+0000 7f205e4d7640 1 -- 172.21.15.26:0/4188688575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2058162780 msgr2=0x7f205806f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.523+0000 7f205e4d7640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2058162780 0x7f205806f1f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f205001c890 tx=0x7f20500195f0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.523+0000 7f205e4d7640 1 -- 172.21.15.26:0/4188688575 shutdown_connections 2023-12-19T16:54:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.523+0000 7f205e4d7640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2058162780 0x7f205806f1f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.523+0000 7f205e4d7640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2058160f80 0x7f2058154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.523+0000 7f205e4d7640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f203c065b80 0x7f203c068040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.523+0000 7f205e4d7640 1 --2- 172.21.15.26:0/4188688575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f205806b3b0 0x7f20581508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.523+0000 7f205e4d7640 1 -- 172.21.15.26:0/4188688575 >> 172.21.15.26:0/4188688575 conn(0x7f20580658e0 msgr2=0x7f2058157a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.523+0000 7f205e4d7640 1 -- 172.21.15.26:0/4188688575 shutdown_connections 2023-12-19T16:54:57.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.523+0000 7f205e4d7640 1 -- 172.21.15.26:0/4188688575 wait complete. 2023-12-19T16:54:57.545 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T16:54:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fbb59e640 1 Processor -- start 2023-12-19T16:54:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fbb59e640 1 -- start start 2023-12-19T16:54:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fbb59e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6fbc0cdd80 0x7f6fbc1676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fbb59e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc0b6cf0 con 0x7f6fbc0b8760 2023-12-19T16:54:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fbb59e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc0b6fb0 con 0x7f6fbc15cca0 2023-12-19T16:54:57.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fbb59e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc0b7c30 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fb9d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6fbc0b8760 0x7f6fbc0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47638/0 (socket says 172.21.15.26:47638) 2023-12-19T16:54:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fb9d9b640 1 -- 172.21.15.26:0/3187374180 learned_addr learned my addr 172.21.15.26:0/3187374180 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fbad9d640 1 --2- 172.21.15.26:0/3187374180 >> v2:172.21.15.160:3300/0 conn(0x7f6fbc0cdd80 0x7f6fbc1676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fb959a640 1 -- 172.21.15.26:0/3187374180 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757524188 0 0) 0x7f6fbc0b6cf0 con 0x7f6fbc0b8760 2023-12-19T16:54:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fb959a640 1 -- 172.21.15.26:0/3187374180 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa4003530 con 0x7f6fbc0b8760 2023-12-19T16:54:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fb959a640 1 -- 172.21.15.26:0/3187374180 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99405355 0 0) 0x7f6fbc0b6fb0 con 0x7f6fbc15cca0 2023-12-19T16:54:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fb959a640 1 -- 172.21.15.26:0/3187374180 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fbc0b6cf0 con 0x7f6fbc15cca0 2023-12-19T16:54:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fbad9d640 1 -- 172.21.15.26:0/3187374180 >> v1:172.21.15.160:6789/0 conn(0x7f6fbc15cca0 legacy=0x7f6fbc0b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fbad9d640 1 -- 172.21.15.26:0/3187374180 >> v1:172.21.15.26:6789/0 conn(0x7f6fbc0b8760 legacy=0x7f6fbc0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.687+0000 7f6fbad9d640 1 -- 172.21.15.26:0/3187374180 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc0b7290 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.691+0000 7f6fbad9d640 1 --2- 172.21.15.26:0/3187374180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fbc0cdd80 0x7f6fbc1676b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4009d90 tx=0x7f6fb401c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa8a6e9fa2a254ba server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.691+0000 7f6fb959a640 1 -- 172.21.15.26:0/3187374180 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40a59e0 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.691+0000 7f6fb959a640 1 -- 172.21.15.26:0/3187374180 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb40a5b40 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.691+0000 7f6fb959a640 1 -- 172.21.15.26:0/3187374180 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40ad440 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.691+0000 7f6fbb59e640 1 -- 172.21.15.26:0/3187374180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fbc0cdd80 msgr2=0x7f6fbc1676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.691+0000 7f6fbb59e640 1 --2- 172.21.15.26:0/3187374180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fbc0cdd80 0x7f6fbc1676b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4009d90 tx=0x7f6fb401c920 comp rx=0 tx=0).stop 2023-12-19T16:54:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.691+0000 7f6fbb59e640 1 -- 172.21.15.26:0/3187374180 shutdown_connections 2023-12-19T16:54:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.691+0000 7f6fbb59e640 1 --2- 172.21.15.26:0/3187374180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fbc0cdd80 0x7f6fbc1676b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.691+0000 7f6fbb59e640 1 -- 172.21.15.26:0/3187374180 >> 172.21.15.26:0/3187374180 conn(0x7f6fbc06a120 msgr2=0x7f6fbc06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.691+0000 7f6fbb59e640 1 -- 172.21.15.26:0/3187374180 shutdown_connections 2023-12-19T16:54:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.691+0000 7f6fbb59e640 1 -- 172.21.15.26:0/3187374180 wait complete. 2023-12-19T16:54:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fbb59e640 1 Processor -- start 2023-12-19T16:54:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fbb59e640 1 -- start start 2023-12-19T16:54:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fbb59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fbc0b8760 0x7f6fbc0c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fbb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fbc0cdd80 0x7f6fbc1521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fbb59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fbc15cca0 0x7f6fbc0c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fbb59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0c8d90 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fbb59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0c8fc0 con 0x7f6fbc0b8760 2023-12-19T16:54:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fbb59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0cc010 con 0x7f6fbc15cca0 2023-12-19T16:54:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fb9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fbc0cdd80 0x7f6fbc1521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fb9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fbc0cdd80 0x7f6fbc1521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49732/0 (socket says 172.21.15.26:49732) 2023-12-19T16:54:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fb9d9b640 1 -- 172.21.15.26:0/4130408405 learned_addr learned my addr 172.21.15.26:0/4130408405 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fbad9d640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fbc15cca0 0x7f6fbc0c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fba59c640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fbc0b8760 0x7f6fbc0c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fb9d9b640 1 -- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fbc15cca0 msgr2=0x7f6fbc0c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fb9d9b640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fbc15cca0 0x7f6fbc0c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fb9d9b640 1 -- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fbc0b8760 msgr2=0x7f6fbc0c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fb9d9b640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fbc0b8760 0x7f6fbc0c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fb9d9b640 1 -- 172.21.15.26:0/4130408405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb4009a70 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fb9d9b640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fbc0cdd80 0x7f6fbc1521d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000c4d0 tx=0x7f6fb000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6faaffd640 1 -- 172.21.15.26:0/4130408405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0005cf0 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6faaffd640 1 -- 172.21.15.26:0/4130408405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb0005e50 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6faaffd640 1 -- 172.21.15.26:0/4130408405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb00072b0 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fbb59e640 1 -- 172.21.15.26:0/4130408405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc0cc2f0 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.695+0000 7f6fbb59e640 1 -- 172.21.15.26:0/4130408405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc0cc8b0 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.699+0000 7f6fbb59e640 1 -- 172.21.15.26:0/4130408405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc067400 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.699+0000 7f6faaffd640 1 -- 172.21.15.26:0/4130408405 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6fb0007410 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.699+0000 7f6faaffd640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f9c065b80 0x7f6f9c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:57.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.699+0000 7f6faaffd640 1 -- 172.21.15.26:0/4130408405 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f6fb00828d0 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.699+0000 7f6fba59c640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f9c065b80 0x7f6f9c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:57.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.699+0000 7f6fba59c640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f9c065b80 0x7f6f9c068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6fac001720 tx=0x7f6fac009040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:57.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.703+0000 7f6faaffd640 1 -- 172.21.15.26:0/4130408405 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6fb0050200 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.863+0000 7f6fbb59e640 1 -- 172.21.15.26:0/4130408405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6fbc0d20e0 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.863+0000 7f6faaffd640 1 -- 172.21.15.26:0/4130408405 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v57) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6fb0017020 con 0x7f6fbc0cdd80 2023-12-19T16:54:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:54:57.867 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T16:54:57.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:54:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T16:54:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T16:54:57.868 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:54:57.868 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T16:54:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.867+0000 7f6fbb59e640 1 -- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f9c065b80 msgr2=0x7f6f9c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.867+0000 7f6fbb59e640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f9c065b80 0x7f6f9c068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6fac001720 tx=0x7f6fac009040 comp rx=0 tx=0).stop 2023-12-19T16:54:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.867+0000 7f6fbb59e640 1 -- 172.21.15.26:0/4130408405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fbc0cdd80 msgr2=0x7f6fbc1521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.867+0000 7f6fbb59e640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fbc0cdd80 0x7f6fbc1521d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000c4d0 tx=0x7f6fb000c990 comp rx=0 tx=0).stop 2023-12-19T16:54:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.867+0000 7f6fbb59e640 1 -- 172.21.15.26:0/4130408405 shutdown_connections 2023-12-19T16:54:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.867+0000 7f6fbb59e640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f9c065b80 0x7f6f9c068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.867+0000 7f6fbb59e640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fbc15cca0 0x7f6fbc0c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.867+0000 7f6fbb59e640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fbc0cdd80 0x7f6fbc1521d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.867+0000 7f6fbb59e640 1 --2- 172.21.15.26:0/4130408405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fbc0b8760 0x7f6fbc0c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.867+0000 7f6fbb59e640 1 -- 172.21.15.26:0/4130408405 >> 172.21.15.26:0/4130408405 conn(0x7f6fbc06a120 msgr2=0x7f6fbc0b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.867+0000 7f6fbb59e640 1 -- 172.21.15.26:0/4130408405 shutdown_connections 2023-12-19T16:54:57.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:57.867+0000 7f6fbb59e640 1 -- 172.21.15.26:0/4130408405 wait complete. 2023-12-19T16:54:57.889 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T16:54:57.889 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T16:54:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.035+0000 7fa2bd02d640 1 Processor -- start 2023-12-19T16:54:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.035+0000 7fa2bd02d640 1 -- start start 2023-12-19T16:54:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.035+0000 7fa2bd02d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2b815ccf0 0x7fa2b80ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.035+0000 7fa2bd02d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa2b806c0f0 0x7fa2b80ac960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.035+0000 7fa2bd02d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b80cd100 con 0x7fa2b80ce7f0 2023-12-19T16:54:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.035+0000 7fa2bd02d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b806e000 con 0x7fa2b815ccf0 2023-12-19T16:54:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.035+0000 7fa2bd02d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b806da40 con 0x7fa2b806c0f0 2023-12-19T16:54:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.035+0000 7fa2b6d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2b815ccf0 0x7fa2b80ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.035+0000 7fa2b659c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa2b80ce7f0 0x7fa2b806b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47652/0 (socket says 172.21.15.26:47652) 2023-12-19T16:54:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.035+0000 7fa2b659c640 1 -- 172.21.15.26:0/3522066385 learned_addr learned my addr 172.21.15.26:0/3522066385 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:54:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.039+0000 7fa2b759e640 1 --2- 172.21.15.26:0/3522066385 >> v2:172.21.15.160:3300/0 conn(0x7fa2b806c0f0 0x7fa2b80ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.039+0000 7fa2b6d9d640 1 -- 172.21.15.26:0/3522066385 >> v2:172.21.15.160:3300/0 conn(0x7fa2b806c0f0 msgr2=0x7fa2b80ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:58.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.039+0000 7fa2b6d9d640 1 --2- 172.21.15.26:0/3522066385 >> v2:172.21.15.160:3300/0 conn(0x7fa2b806c0f0 0x7fa2b80ac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.039+0000 7fa2b6d9d640 1 -- 172.21.15.26:0/3522066385 >> v1:172.21.15.26:6789/0 conn(0x7fa2b80ce7f0 legacy=0x7fa2b806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.039+0000 7fa2b6d9d640 1 -- 172.21.15.26:0/3522066385 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b806dd20 con 0x7fa2b815ccf0 2023-12-19T16:54:58.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.039+0000 7fa2b5d9b640 1 -- 172.21.15.26:0/3522066385 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380985575 0 0) 0x7fa2b80cd100 con 0x7fa2b80ce7f0 2023-12-19T16:54:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.039+0000 7fa2b6d9d640 1 --2- 172.21.15.26:0/3522066385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b815ccf0 0x7fa2b80ae3f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80cd070 tx=0x7fa2a801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bbdc671217cd6a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.039+0000 7fa2b5d9b640 1 -- 172.21.15.26:0/3522066385 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8004230 con 0x7fa2b815ccf0 2023-12-19T16:54:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.039+0000 7fa2b5d9b640 1 -- 172.21.15.26:0/3522066385 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a80a7050 con 0x7fa2b815ccf0 2023-12-19T16:54:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.039+0000 7fa2b5d9b640 1 -- 172.21.15.26:0/3522066385 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8003620 con 0x7fa2b815ccf0 2023-12-19T16:54:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 -- 172.21.15.26:0/3522066385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b815ccf0 msgr2=0x7fa2b80ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 --2- 172.21.15.26:0/3522066385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b815ccf0 0x7fa2b80ae3f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa2b80cd070 tx=0x7fa2a801c3b0 comp rx=0 tx=0).stop 2023-12-19T16:54:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 -- 172.21.15.26:0/3522066385 shutdown_connections 2023-12-19T16:54:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 --2- 172.21.15.26:0/3522066385 >> v2:172.21.15.160:3300/0 conn(0x7fa2b806c0f0 0x7fa2b80ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 --2- 172.21.15.26:0/3522066385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b815ccf0 0x7fa2b80ae3f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 -- 172.21.15.26:0/3522066385 >> 172.21.15.26:0/3522066385 conn(0x7fa2b80cf020 msgr2=0x7fa2b80cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 -- 172.21.15.26:0/3522066385 shutdown_connections 2023-12-19T16:54:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 -- 172.21.15.26:0/3522066385 wait complete. 2023-12-19T16:54:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 Processor -- start 2023-12-19T16:54:58.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 -- start start 2023-12-19T16:54:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2b806c0f0 0x7fa2b80c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:58.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2b80ce7f0 0x7fa2b8159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b815ccf0 0x7fa2b8157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b815d8e0 con 0x7fa2b815ccf0 2023-12-19T16:54:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80c48d0 con 0x7fa2b80ce7f0 2023-12-19T16:54:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2bd02d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b80c4bb0 con 0x7fa2b806c0f0 2023-12-19T16:54:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2b759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b815ccf0 0x7fa2b8157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2b759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b815ccf0 0x7fa2b8157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49752/0 (socket says 172.21.15.26:49752) 2023-12-19T16:54:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2b759e640 1 -- 172.21.15.26:0/2831591198 learned_addr learned my addr 172.21.15.26:0/2831591198 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2b659c640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2b80ce7f0 0x7fa2b8159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2b6d9d640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2b806c0f0 0x7fa2b80c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2b759e640 1 -- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2b806c0f0 msgr2=0x7fa2b80c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2b759e640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2b806c0f0 0x7fa2b80c37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2b759e640 1 -- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2b80ce7f0 msgr2=0x7fa2b8159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:58.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2b759e640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2b80ce7f0 0x7fa2b8159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.043+0000 7fa2b759e640 1 -- 172.21.15.26:0/2831591198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a80093b0 con 0x7fa2b815ccf0 2023-12-19T16:54:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.047+0000 7fa2b759e640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b815ccf0 0x7fa2b8157b40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0007e40 tx=0x7fa2b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.047+0000 7fa29f7fe640 1 -- 172.21.15.26:0/2831591198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0014620 con 0x7fa2b815ccf0 2023-12-19T16:54:58.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.047+0000 7fa29f7fe640 1 -- 172.21.15.26:0/2831591198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b000ee90 con 0x7fa2b815ccf0 2023-12-19T16:54:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.047+0000 7fa29f7fe640 1 -- 172.21.15.26:0/2831591198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b00149e0 con 0x7fa2b815ccf0 2023-12-19T16:54:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.047+0000 7fa2bd02d640 1 -- 172.21.15.26:0/2831591198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b80cedc0 con 0x7fa2b815ccf0 2023-12-19T16:54:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.047+0000 7fa2bd02d640 1 -- 172.21.15.26:0/2831591198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b80c3d10 con 0x7fa2b815ccf0 2023-12-19T16:54:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.047+0000 7fa2bd02d640 1 -- 172.21.15.26:0/2831591198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b80cec00 con 0x7fa2b815ccf0 2023-12-19T16:54:58.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.047+0000 7fa29f7fe640 1 -- 172.21.15.26:0/2831591198 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa2b0014b40 con 0x7fa2b815ccf0 2023-12-19T16:54:58.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.047+0000 7fa29f7fe640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2a4065b80 0x7fa2a4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:58.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.047+0000 7fa29f7fe640 1 -- 172.21.15.26:0/2831591198 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fa2b0083a80 con 0x7fa2b815ccf0 2023-12-19T16:54:58.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.051+0000 7fa2b6d9d640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2a4065b80 0x7fa2a4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:58.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.051+0000 7fa2b6d9d640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2a4065b80 0x7fa2a4068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80ad000 tx=0x7fa2a80a6000 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:58.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.051+0000 7fa29f7fe640 1 -- 172.21.15.26:0/2831591198 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa2b00514e0 con 0x7fa2b815ccf0 2023-12-19T16:54:58.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:58.211+0000 7fa2bd02d640 1 -- 172.21.15.26:0/2831591198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa2b8074270 con 0x7fa2b815ccf0 2023-12-19T16:54:59.152 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:54:59.156 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:54:59.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.171+0000 7fa29f7fe640 1 -- 172.21.15.26:0/2831591198 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v58) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa2b0054b00 con 0x7fa2b815ccf0 2023-12-19T16:54:59.175 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T16:54:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.175+0000 7fa2bd02d640 1 -- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2a4065b80 msgr2=0x7fa2a4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.175+0000 7fa2bd02d640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2a4065b80 0x7fa2a4068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80ad000 tx=0x7fa2a80a6000 comp rx=0 tx=0).stop 2023-12-19T16:54:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.175+0000 7fa2bd02d640 1 -- 172.21.15.26:0/2831591198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b815ccf0 msgr2=0x7fa2b8157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.175+0000 7fa2bd02d640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b815ccf0 0x7fa2b8157b40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0007e40 tx=0x7fa2b000e4b0 comp rx=0 tx=0).stop 2023-12-19T16:54:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.175+0000 7fa2bd02d640 1 -- 172.21.15.26:0/2831591198 shutdown_connections 2023-12-19T16:54:59.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.175+0000 7fa2bd02d640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2b815ccf0 0x7fa2b8157b40 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.175+0000 7fa2bd02d640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2b80ce7f0 0x7fa2b8159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.175+0000 7fa2bd02d640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2a4065b80 0x7fa2a4068040 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80ad000 tx=0x7fa2a80a6000 comp rx=0 tx=0).stop 2023-12-19T16:54:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.175+0000 7fa2bd02d640 1 --2- 172.21.15.26:0/2831591198 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2b806c0f0 0x7fa2b80c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.175+0000 7fa2bd02d640 1 -- 172.21.15.26:0/2831591198 >> 172.21.15.26:0/2831591198 conn(0x7fa2b80cf020 msgr2=0x7fa2b80b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.179+0000 7fa2bd02d640 1 -- 172.21.15.26:0/2831591198 shutdown_connections 2023-12-19T16:54:59.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.179+0000 7fa2bd02d640 1 -- 172.21.15.26:0/2831591198 wait complete. 2023-12-19T16:54:59.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T16:54:59.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.415+0000 7fc571ce5640 1 Processor -- start 2023-12-19T16:54:59.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.415+0000 7fc571ce5640 1 -- start start 2023-12-19T16:54:59.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.415+0000 7fc571ce5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc56c15ada0 0x7fc56c0ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:59.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.415+0000 7fc571ce5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc56c070db0 0x7fc56c0b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:59.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.415+0000 7fc571ce5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56c06b500 con 0x7fc56c0b0a60 2023-12-19T16:54:59.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.415+0000 7fc571ce5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc56c0715b0 con 0x7fc56c070db0 2023-12-19T16:54:59.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.415+0000 7fc571ce5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c0adfd0 con 0x7fc56c15ada0 2023-12-19T16:54:59.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56affd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc56c070db0 0x7fc56c0b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:59.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56affd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc56c070db0 0x7fc56c0b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:37918/0 (socket says 172.21.15.26:37918) 2023-12-19T16:54:59.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56bfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fc56c0b0a60 0x7fc56c0ae170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:45140/0 (socket says 172.21.15.26:45140) 2023-12-19T16:54:59.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56affd640 1 -- 172.21.15.26:0/2194920449 learned_addr learned my addr 172.21.15.26:0/2194920449 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:59.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56b7fe640 1 --2- 172.21.15.26:0/2194920449 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc56c15ada0 0x7fc56c0ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:59.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56affd640 1 -- 172.21.15.26:0/2194920449 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc56c15ada0 msgr2=0x7fc56c0ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:54:59.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56affd640 1 --2- 172.21.15.26:0/2194920449 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc56c15ada0 0x7fc56c0ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:59.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56affd640 1 -- 172.21.15.26:0/2194920449 >> v1:172.21.15.160:6789/0 conn(0x7fc56c0b0a60 legacy=0x7fc56c0ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:59.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56affd640 1 -- 172.21.15.26:0/2194920449 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56c070b60 con 0x7fc56c070db0 2023-12-19T16:54:59.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56a7fc640 1 -- 172.21.15.26:0/2194920449 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318524345 0 0) 0x7fc56c06b500 con 0x7fc56c0b0a60 2023-12-19T16:54:59.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56affd640 1 --2- 172.21.15.26:0/2194920449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc56c070db0 0x7fc56c0b0520 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc5600098d0 tx=0x7fc56001c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32fca54c859ea536 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56a7fc640 1 -- 172.21.15.26:0/2194920449 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5600aa650 con 0x7fc56c070db0 2023-12-19T16:54:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56a7fc640 1 -- 172.21.15.26:0/2194920449 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5600aa7b0 con 0x7fc56c070db0 2023-12-19T16:54:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.419+0000 7fc56a7fc640 1 -- 172.21.15.26:0/2194920449 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5600aab40 con 0x7fc56c070db0 2023-12-19T16:54:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 -- 172.21.15.26:0/2194920449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc56c070db0 msgr2=0x7fc56c0b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:59.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 --2- 172.21.15.26:0/2194920449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc56c070db0 0x7fc56c0b0520 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc5600098d0 tx=0x7fc56001c4f0 comp rx=0 tx=0).stop 2023-12-19T16:54:59.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 -- 172.21.15.26:0/2194920449 shutdown_connections 2023-12-19T16:54:59.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 --2- 172.21.15.26:0/2194920449 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc56c070db0 0x7fc56c0b0520 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:59.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 --2- 172.21.15.26:0/2194920449 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc56c15ada0 0x7fc56c0ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:59.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 -- 172.21.15.26:0/2194920449 >> 172.21.15.26:0/2194920449 conn(0x7fc56c0660d0 msgr2=0x7fc56c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:54:59.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 -- 172.21.15.26:0/2194920449 shutdown_connections 2023-12-19T16:54:59.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 -- 172.21.15.26:0/2194920449 wait complete. 2023-12-19T16:54:59.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 Processor -- start 2023-12-19T16:54:59.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 -- start start 2023-12-19T16:54:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56c070db0 0x7fc56c152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc56c0b0a60 0x7fc56c150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc56c15ada0 0x7fc56c14cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c151060 con 0x7fc56c070db0 2023-12-19T16:54:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c151300 con 0x7fc56c15ada0 2023-12-19T16:54:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc571ce5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56c1515e0 con 0x7fc56c0b0a60 2023-12-19T16:54:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc56b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56c070db0 0x7fc56c152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc56b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56c070db0 0x7fc56c152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49762/0 (socket says 172.21.15.26:49762) 2023-12-19T16:54:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc56b7fe640 1 -- 172.21.15.26:0/1032161150 learned_addr learned my addr 172.21.15.26:0/1032161150 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:54:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc56affd640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc56c0b0a60 0x7fc56c150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.423+0000 7fc56bfff640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc56c15ada0 0x7fc56c14cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc56b7fe640 1 -- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc56c0b0a60 msgr2=0x7fc56c150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:59.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc56b7fe640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc56c0b0a60 0x7fc56c150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc56b7fe640 1 -- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc56c15ada0 msgr2=0x7fc56c14cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:54:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc56b7fe640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc56c15ada0 0x7fc56c14cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:54:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc56b7fe640 1 -- 172.21.15.26:0/1032161150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5600094d0 con 0x7fc56c070db0 2023-12-19T16:54:59.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc56b7fe640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56c070db0 0x7fc56c152a30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00e870 tx=0x7fc55c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:59.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc553fff640 1 -- 172.21.15.26:0/1032161150 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c009510 con 0x7fc56c070db0 2023-12-19T16:54:59.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc553fff640 1 -- 172.21.15.26:0/1032161150 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55c00eec0 con 0x7fc56c070db0 2023-12-19T16:54:59.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc553fff640 1 -- 172.21.15.26:0/1032161150 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c00cb40 con 0x7fc56c070db0 2023-12-19T16:54:59.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc571ce5640 1 -- 172.21.15.26:0/1032161150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56c1518c0 con 0x7fc56c070db0 2023-12-19T16:54:59.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc571ce5640 1 -- 172.21.15.26:0/1032161150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56c14d290 con 0x7fc56c070db0 2023-12-19T16:54:59.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc571ce5640 1 -- 172.21.15.26:0/1032161150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56c074270 con 0x7fc56c070db0 2023-12-19T16:54:59.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc553fff640 1 -- 172.21.15.26:0/1032161150 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc55c00cca0 con 0x7fc56c070db0 2023-12-19T16:54:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.427+0000 7fc553fff640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc558065b80 0x7fc558068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:54:59.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.431+0000 7fc553fff640 1 -- 172.21.15.26:0/1032161150 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5959+0+0 (secure 0 0 0) 0x7fc55c083c50 con 0x7fc56c070db0 2023-12-19T16:54:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.431+0000 7fc56affd640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc558065b80 0x7fc558068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:54:59.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.431+0000 7fc56affd640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc558065b80 0x7fc558068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc5600ae000 tx=0x7fc5600a7000 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:54:59.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.431+0000 7fc553fff640 1 -- 172.21.15.26:0/1032161150 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc55c051520 con 0x7fc56c070db0 2023-12-19T16:54:59.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:54:59.595+0000 7fc571ce5640 1 -- 172.21.15.26:0/1032161150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc56c0adbb0 con 0x7fc56c070db0 2023-12-19T16:55:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.179+0000 7fc553fff640 1 -- 172.21.15.26:0/1032161150 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v59) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc55c054b40 con 0x7fc56c070db0 2023-12-19T16:55:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T16:55:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.183+0000 7fc571ce5640 1 -- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc558065b80 msgr2=0x7fc558068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.183+0000 7fc571ce5640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc558065b80 0x7fc558068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc5600ae000 tx=0x7fc5600a7000 comp rx=0 tx=0).stop 2023-12-19T16:55:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.183+0000 7fc571ce5640 1 -- 172.21.15.26:0/1032161150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56c070db0 msgr2=0x7fc56c152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.183+0000 7fc571ce5640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56c070db0 0x7fc56c152a30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00e870 tx=0x7fc55c00ec70 comp rx=0 tx=0).stop 2023-12-19T16:55:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.187+0000 7fc571ce5640 1 -- 172.21.15.26:0/1032161150 shutdown_connections 2023-12-19T16:55:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.187+0000 7fc571ce5640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc56c15ada0 0x7fc56c14cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.187+0000 7fc571ce5640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc558065b80 0x7fc558068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.187+0000 7fc571ce5640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc56c0b0a60 0x7fc56c150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.187+0000 7fc571ce5640 1 --2- 172.21.15.26:0/1032161150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56c070db0 0x7fc56c152a30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.187+0000 7fc571ce5640 1 -- 172.21.15.26:0/1032161150 >> 172.21.15.26:0/1032161150 conn(0x7fc56c0660d0 msgr2=0x7fc56c159f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.187+0000 7fc571ce5640 1 -- 172.21.15.26:0/1032161150 shutdown_connections 2023-12-19T16:55:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.187+0000 7fc571ce5640 1 -- 172.21.15.26:0/1032161150 wait complete. 2023-12-19T16:55:00.228 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T16:55:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1da139640 1 Processor -- start 2023-12-19T16:55:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1da139640 1 -- start start 2023-12-19T16:55:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1da139640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff1d4159c00 0x7ff1d415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1da139640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d406fb60 con 0x7ff1d406f6c0 2023-12-19T16:55:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1da139640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d406fe40 con 0x7ff1d4159820 2023-12-19T16:55:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1da139640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d415e470 con 0x7ff1d4159c00 2023-12-19T16:55:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d2ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff1d406f6c0 0x7ff1d4070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47658/0 (socket says 172.21.15.26:47658) 2023-12-19T16:55:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d2ffd640 1 -- 172.21.15.26:0/1608587010 learned_addr learned my addr 172.21.15.26:0/1608587010 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:55:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984842011 0 0) 0x7ff1d406fb60 con 0x7ff1d406f6c0 2023-12-19T16:55:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d3fff640 1 --2- 172.21.15.26:0/1608587010 >> v2:172.21.15.160:3300/0 conn(0x7ff1d4159c00 0x7ff1d415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1bc005740 con 0x7ff1d406f6c0 2023-12-19T16:55:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 748460768 0 0) 0x7ff1bc005740 con 0x7ff1d406f6c0 2023-12-19T16:55:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 >> v2:172.21.15.160:3300/0 conn(0x7ff1d4159c00 msgr2=0x7ff1d415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 --2- 172.21.15.26:0/1608587010 >> v2:172.21.15.160:3300/0 conn(0x7ff1d4159c00 0x7ff1d415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 >> v1:172.21.15.160:6789/0 conn(0x7ff1d4159820 legacy=0x7ff1d40c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d4070120 con 0x7ff1d406f6c0 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7ff1c8001680 con 0x7ff1d406f6c0 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 >> v1:172.21.15.26:6789/0 conn(0x7ff1d406f6c0 legacy=0x7ff1d4070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 --2- 172.21.15.26:0/1608587010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1bc005b90 0x7ff1bc005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 --2- 172.21.15.26:0/1608587010 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1bc0069e0 0x7ff1bc006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 --2- 172.21.15.26:0/1608587010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc0073a0 0x7ff1bc00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d415e470 con 0x7ff1bc0073a0 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc00e240 con 0x7ff1bc005b90 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc00e400 con 0x7ff1bc0069e0 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d3fff640 1 --2- 172.21.15.26:0/1608587010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc0073a0 0x7ff1bc00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d37fe640 1 --2- 172.21.15.26:0/1608587010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1bc005b90 0x7ff1bc005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.375+0000 7ff1d2ffd640 1 --2- 172.21.15.26:0/1608587010 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1bc0069e0 0x7ff1bc006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1d3fff640 1 -- 172.21.15.26:0/1608587010 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1bc0069e0 msgr2=0x7ff1bc006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1d3fff640 1 --2- 172.21.15.26:0/1608587010 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1bc0069e0 0x7ff1bc006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1d3fff640 1 -- 172.21.15.26:0/1608587010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1bc005b90 msgr2=0x7ff1bc005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1d3fff640 1 --2- 172.21.15.26:0/1608587010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1bc005b90 0x7ff1bc005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1d3fff640 1 -- 172.21.15.26:0/1608587010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc005740 con 0x7ff1bc0073a0 2023-12-19T16:55:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1d3fff640 1 -- 172.21.15.26:0/1608587010 reap_dead start 2023-12-19T16:55:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1d3fff640 1 --2- 172.21.15.26:0/1608587010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc0073a0 0x7ff1bc00dba0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009f90 tx=0x7ff1cc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b2273e5d976811 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0a5a70 con 0x7ff1bc0073a0 2023-12-19T16:55:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc00f040 con 0x7ff1bc0073a0 2023-12-19T16:55:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1d27fc640 1 -- 172.21.15.26:0/1608587010 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0a5e10 con 0x7ff1bc0073a0 2023-12-19T16:55:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1da139640 1 -- 172.21.15.26:0/1608587010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc0073a0 msgr2=0x7ff1bc00dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1da139640 1 --2- 172.21.15.26:0/1608587010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc0073a0 0x7ff1bc00dba0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009f90 tx=0x7ff1cc01cc60 comp rx=0 tx=0).stop 2023-12-19T16:55:00.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1da139640 1 -- 172.21.15.26:0/1608587010 shutdown_connections 2023-12-19T16:55:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1da139640 1 --2- 172.21.15.26:0/1608587010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1bc0073a0 0x7ff1bc00dba0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1da139640 1 -- 172.21.15.26:0/1608587010 >> 172.21.15.26:0/1608587010 conn(0x7ff1d40ce730 msgr2=0x7ff1d40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.379+0000 7ff1da139640 1 -- 172.21.15.26:0/1608587010 shutdown_connections 2023-12-19T16:55:00.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1da139640 1 -- 172.21.15.26:0/1608587010 wait complete. 2023-12-19T16:55:00.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1da139640 1 Processor -- start 2023-12-19T16:55:00.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1da139640 1 -- start start 2023-12-19T16:55:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1da139640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1bc0073a0 0x7ff1d415c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1da139640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1d40af370 0x7ff1d40aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1da139640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d41574a0 0x7ff1d4157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1da139640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4157e90 con 0x7ff1d41574a0 2023-12-19T16:55:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1da139640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d414b670 con 0x7ff1d40af370 2023-12-19T16:55:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1da139640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d40c2960 con 0x7ff1bc0073a0 2023-12-19T16:55:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d41574a0 0x7ff1d4157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d41574a0 0x7ff1d4157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49780/0 (socket says 172.21.15.26:49780) 2023-12-19T16:55:00.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1d3fff640 1 -- 172.21.15.26:0/936653360 learned_addr learned my addr 172.21.15.26:0/936653360 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:00.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1d3fff640 1 -- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1bc0073a0 msgr2=0x7ff1d415c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1d2ffd640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1d40af370 0x7ff1d40aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1d3fff640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1bc0073a0 0x7ff1d415c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1d3fff640 1 -- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1d40af370 msgr2=0x7ff1d40aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1d3fff640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1d40af370 0x7ff1d40aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1d3fff640 1 -- 172.21.15.26:0/936653360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d40c2c40 con 0x7ff1d41574a0 2023-12-19T16:55:00.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1d2ffd640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1d40af370 0x7ff1d40aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:55:00.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1d3fff640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d41574a0 0x7ff1d4157950 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc01c6d0 tx=0x7ff1cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:00.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1bbfff640 1 -- 172.21.15.26:0/936653360 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0bb070 con 0x7ff1d41574a0 2023-12-19T16:55:00.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1bbfff640 1 -- 172.21.15.26:0/936653360 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc003c70 con 0x7ff1d41574a0 2023-12-19T16:55:00.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1bbfff640 1 -- 172.21.15.26:0/936653360 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0ae780 con 0x7ff1d41574a0 2023-12-19T16:55:00.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.383+0000 7ff1da139640 1 -- 172.21.15.26:0/936653360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d40c2f20 con 0x7ff1d41574a0 2023-12-19T16:55:00.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.387+0000 7ff1da139640 1 -- 172.21.15.26:0/936653360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d40c34e0 con 0x7ff1d41574a0 2023-12-19T16:55:00.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.387+0000 7ff1da139640 1 -- 172.21.15.26:0/936653360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d40c36d0 con 0x7ff1d41574a0 2023-12-19T16:55:00.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.387+0000 7ff1bbfff640 1 -- 172.21.15.26:0/936653360 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff1cc0ae8e0 con 0x7ff1d41574a0 2023-12-19T16:55:00.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.387+0000 7ff1bbfff640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff1c0065b80 0x7ff1c0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.387+0000 7ff1bbfff640 1 -- 172.21.15.26:0/936653360 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6341+0+0 (secure 0 0 0) 0x7ff1cc11e210 con 0x7ff1d41574a0 2023-12-19T16:55:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.387+0000 7ff1d37fe640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff1c0065b80 0x7ff1c0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.391+0000 7ff1d37fe640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff1c0065b80 0x7ff1c0068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4007970 tx=0x7ff1c4012040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:00.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.391+0000 7ff1bbfff640 1 -- 172.21.15.26:0/936653360 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff1cc0eb820 con 0x7ff1d41574a0 2023-12-19T16:55:00.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.579+0000 7ff1da139640 1 -- 172.21.15.26:0/936653360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff1d4074270 con 0x7ff1d41574a0 2023-12-19T16:55:00.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.587+0000 7ff1bbfff640 1 -- 172.21.15.26:0/936653360 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff1cc0eee40 con 0x7ff1d41574a0 2023-12-19T16:55:00.595 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 6 and data pool 7 2023-12-19T16:55:00.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.599+0000 7ff1da139640 1 -- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff1c0065b80 msgr2=0x7ff1c0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.599+0000 7ff1da139640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff1c0065b80 0x7ff1c0068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4007970 tx=0x7ff1c4012040 comp rx=0 tx=0).stop 2023-12-19T16:55:00.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.599+0000 7ff1da139640 1 -- 172.21.15.26:0/936653360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d41574a0 msgr2=0x7ff1d4157950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.599+0000 7ff1da139640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d41574a0 0x7ff1d4157950 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc01c6d0 tx=0x7ff1cc009750 comp rx=0 tx=0).stop 2023-12-19T16:55:00.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.599+0000 7ff1da139640 1 -- 172.21.15.26:0/936653360 shutdown_connections 2023-12-19T16:55:00.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.599+0000 7ff1da139640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d41574a0 0x7ff1d4157950 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.599+0000 7ff1da139640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1d40af370 0x7ff1d40aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.599+0000 7ff1da139640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff1c0065b80 0x7ff1c0068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.599+0000 7ff1da139640 1 --2- 172.21.15.26:0/936653360 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1bc0073a0 0x7ff1d415c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.599+0000 7ff1da139640 1 -- 172.21.15.26:0/936653360 >> 172.21.15.26:0/936653360 conn(0x7ff1d40ce730 msgr2=0x7ff1d40ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:00.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.603+0000 7ff1da139640 1 -- 172.21.15.26:0/936653360 shutdown_connections 2023-12-19T16:55:00.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.603+0000 7ff1da139640 1 -- 172.21.15.26:0/936653360 wait complete. 2023-12-19T16:55:00.641 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:55:00.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab8e8c640 1 Processor -- start 2023-12-19T16:55:00.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab8e8c640 1 -- start start 2023-12-19T16:55:00.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab8e8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ab4155030 0x7f0ab415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab8e8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ab415f370 0x7f0ab406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab8e8c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0ab40ce740 0x7f0ab40cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab8e8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab40cf130 con 0x7f0ab415f370 2023-12-19T16:55:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab8e8c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab406c970 con 0x7f0ab40ce740 2023-12-19T16:55:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab8e8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab415b1f0 con 0x7f0ab4155030 2023-12-19T16:55:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab1d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ab415f370 0x7f0ab406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab1d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ab415f370 0x7f0ab406c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49792/0 (socket says 172.21.15.26:49792) 2023-12-19T16:55:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab1d9b640 1 -- 172.21.15.26:0/1562867888 learned_addr learned my addr 172.21.15.26:0/1562867888 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab259c640 1 --2- 172.21.15.26:0/1562867888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ab4155030 0x7f0ab415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab2d9d640 1 --2- 172.21.15.26:0/1562867888 >> v2:172.21.15.160:3300/0 conn(0x7f0ab40ce740 0x7f0ab40cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab1d9b640 1 -- 172.21.15.26:0/1562867888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ab4155030 msgr2=0x7f0ab415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab1d9b640 1 --2- 172.21.15.26:0/1562867888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ab4155030 0x7f0ab415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab1d9b640 1 -- 172.21.15.26:0/1562867888 >> v2:172.21.15.160:3300/0 conn(0x7f0ab40ce740 msgr2=0x7f0ab40cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab1d9b640 1 --2- 172.21.15.26:0/1562867888 >> v2:172.21.15.160:3300/0 conn(0x7f0ab40ce740 0x7f0ab40cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.783+0000 7f0ab1d9b640 1 -- 172.21.15.26:0/1562867888 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab415b4d0 con 0x7f0ab415f370 2023-12-19T16:55:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.787+0000 7f0ab1d9b640 1 --2- 172.21.15.26:0/1562867888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ab415f370 0x7f0ab406c080 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0ab406f300 tx=0x7f0aa801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959da8a86ce35f38 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.787+0000 7f0ab159a640 1 -- 172.21.15.26:0/1562867888 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80aa610 con 0x7f0ab415f370 2023-12-19T16:55:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.787+0000 7f0ab159a640 1 -- 172.21.15.26:0/1562867888 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa80a6040 con 0x7f0ab415f370 2023-12-19T16:55:00.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.787+0000 7f0ab159a640 1 -- 172.21.15.26:0/1562867888 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80aa990 con 0x7f0ab415f370 2023-12-19T16:55:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.787+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/1562867888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ab415f370 msgr2=0x7f0ab406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.787+0000 7f0ab8e8c640 1 --2- 172.21.15.26:0/1562867888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ab415f370 0x7f0ab406c080 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0ab406f300 tx=0x7f0aa801c4d0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.787+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/1562867888 shutdown_connections 2023-12-19T16:55:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.787+0000 7f0ab8e8c640 1 --2- 172.21.15.26:0/1562867888 >> v2:172.21.15.160:3300/0 conn(0x7f0ab40ce740 0x7f0ab40cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.787+0000 7f0ab8e8c640 1 --2- 172.21.15.26:0/1562867888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ab415f370 0x7f0ab406c080 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.787+0000 7f0ab8e8c640 1 --2- 172.21.15.26:0/1562867888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ab4155030 0x7f0ab415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.787+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/1562867888 >> 172.21.15.26:0/1562867888 conn(0x7f0ab40658e0 msgr2=0x7f0ab4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.787+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/1562867888 shutdown_connections 2023-12-19T16:55:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.787+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/1562867888 wait complete. 2023-12-19T16:55:00.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab8e8c640 1 Processor -- start 2023-12-19T16:55:00.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab8e8c640 1 -- start start 2023-12-19T16:55:00.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab8e8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ab40ce740 0x7f0ab414f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab8e8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ab4155030 0x7f0ab41494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab8e8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ab415f370 0x7f0ab4147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab8e8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab4148aa0 con 0x7f0ab4155030 2023-12-19T16:55:00.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab8e8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab41499f0 con 0x7f0ab40ce740 2023-12-19T16:55:00.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab8e8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab414e3f0 con 0x7f0ab415f370 2023-12-19T16:55:00.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ab4155030 0x7f0ab41494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ab4155030 0x7f0ab41494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49798/0 (socket says 172.21.15.26:49798) 2023-12-19T16:55:00.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab1d9b640 1 -- 172.21.15.26:0/558481479 learned_addr learned my addr 172.21.15.26:0/558481479 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:00.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab259c640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ab40ce740 0x7f0ab414f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab2d9d640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ab415f370 0x7f0ab4147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab1d9b640 1 -- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ab415f370 msgr2=0x7f0ab4147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab1d9b640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ab415f370 0x7f0ab4147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab1d9b640 1 -- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ab40ce740 msgr2=0x7f0ab414f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab1d9b640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ab40ce740 0x7f0ab414f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab1d9b640 1 -- 172.21.15.26:0/558481479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa80094d0 con 0x7f0ab4155030 2023-12-19T16:55:00.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab1d9b640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ab4155030 0x7f0ab41494b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80ac040 tx=0x7f0aa8019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:00.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0aa2ffd640 1 -- 172.21.15.26:0/558481479 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80be610 con 0x7f0ab4155030 2023-12-19T16:55:00.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0aa2ffd640 1 -- 172.21.15.26:0/558481479 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa80be770 con 0x7f0ab4155030 2023-12-19T16:55:00.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0aa2ffd640 1 -- 172.21.15.26:0/558481479 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80beb20 con 0x7f0ab4155030 2023-12-19T16:55:00.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/558481479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab406c740 con 0x7f0ab4155030 2023-12-19T16:55:00.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/558481479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab414e9b0 con 0x7f0ab4155030 2023-12-19T16:55:00.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.791+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/558481479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab414eba0 con 0x7f0ab4155030 2023-12-19T16:55:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.795+0000 7f0aa2ffd640 1 -- 172.21.15.26:0/558481479 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0aa80b4400 con 0x7f0ab4155030 2023-12-19T16:55:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.795+0000 7f0aa2ffd640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0a94065b80 0x7f0a94068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.795+0000 7f0aa2ffd640 1 -- 172.21.15.26:0/558481479 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f0aa811df50 con 0x7f0ab4155030 2023-12-19T16:55:00.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.795+0000 7f0ab259c640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0a94065b80 0x7f0a94068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:00.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.795+0000 7f0ab259c640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0a94065b80 0x7f0a94068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4007940 tx=0x7f0aa4008040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:00.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.799+0000 7f0aa2ffd640 1 -- 172.21.15.26:0/558481479 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0aa80eb420 con 0x7f0ab4155030 2023-12-19T16:55:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.959+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/558481479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ab4074270 con 0x7f0ab4155030 2023-12-19T16:55:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.959+0000 7f0aa2ffd640 1 -- 172.21.15.26:0/558481479 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+17163 (secure 0 0 0) 0x7f0aa80eea40 con 0x7f0ab4155030 2023-12-19T16:55:00.963 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:55:00.963 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":60,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:55:00.584404+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:54:58.260985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T16:54:59.644467+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:55:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.963+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0a94065b80 msgr2=0x7f0a94068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.963+0000 7f0ab8e8c640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0a94065b80 0x7f0a94068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4007940 tx=0x7f0aa4008040 comp rx=0 tx=0).stop 2023-12-19T16:55:00.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.963+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/558481479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ab4155030 msgr2=0x7f0ab41494b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:00.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.963+0000 7f0ab8e8c640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ab4155030 0x7f0ab41494b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80ac040 tx=0x7f0aa8019270 comp rx=0 tx=0).stop 2023-12-19T16:55:00.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.963+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/558481479 shutdown_connections 2023-12-19T16:55:00.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.963+0000 7f0ab8e8c640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0a94065b80 0x7f0a94068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.963+0000 7f0ab8e8c640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ab415f370 0x7f0ab4147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.963+0000 7f0ab8e8c640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ab4155030 0x7f0ab41494b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.963+0000 7f0ab8e8c640 1 --2- 172.21.15.26:0/558481479 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ab40ce740 0x7f0ab414f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:00.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.963+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/558481479 >> 172.21.15.26:0/558481479 conn(0x7f0ab40658e0 msgr2=0x7f0ab415bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:00.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.963+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/558481479 shutdown_connections 2023-12-19T16:55:00.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:00.963+0000 7f0ab8e8c640 1 -- 172.21.15.26:0/558481479 wait complete. 2023-12-19T16:55:00.985 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:55:01.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.123+0000 7efe7fe8c640 1 Processor -- start 2023-12-19T16:55:01.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.127+0000 7efe7fe8c640 1 -- start start 2023-12-19T16:55:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.127+0000 7efe7fe8c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efe780d3300 0x7efe780d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.127+0000 7efe7fe8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe780d29d0 con 0x7efe780ce040 2023-12-19T16:55:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.127+0000 7efe7fe8c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7815d680 con 0x7efe7815cd40 2023-12-19T16:55:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.127+0000 7efe7fe8c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe780ce4b0 con 0x7efe780d3300 2023-12-19T16:55:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.127+0000 7efe7d427640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efe780ce040 0x7efe780d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47674/0 (socket says 172.21.15.26:47674) 2023-12-19T16:55:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.127+0000 7efe7d427640 1 -- 172.21.15.26:0/1271511956 learned_addr learned my addr 172.21.15.26:0/1271511956 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:55:01.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.127+0000 7efe7e429640 1 --2- 172.21.15.26:0/1271511956 >> v2:172.21.15.160:3300/0 conn(0x7efe780d3300 0x7efe780d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:01.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.127+0000 7efe7cc26640 1 -- 172.21.15.26:0/1271511956 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900757658 0 0) 0x7efe68001120 con 0x7efe780ce040 2023-12-19T16:55:01.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.127+0000 7efe7cc26640 1 -- 172.21.15.26:0/1271511956 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe64003530 con 0x7efe780ce040 2023-12-19T16:55:01.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.127+0000 7efe7e429640 1 -- 172.21.15.26:0/1271511956 >> v1:172.21.15.160:6789/0 conn(0x7efe7815cd40 legacy=0x7efe78160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:01.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.127+0000 7efe7e429640 1 -- 172.21.15.26:0/1271511956 >> v1:172.21.15.26:6789/0 conn(0x7efe780ce040 legacy=0x7efe780d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:01.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.127+0000 7efe7e429640 1 -- 172.21.15.26:0/1271511956 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe78161750 con 0x7efe780d3300 2023-12-19T16:55:01.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.131+0000 7efe7e429640 1 --2- 172.21.15.26:0/1271511956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe780d3300 0x7efe780d0e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efe70009d90 tx=0x7efe7001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22b1b852f8b267e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:01.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.131+0000 7efe7cc26640 1 -- 172.21.15.26:0/1271511956 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700a59e0 con 0x7efe780d3300 2023-12-19T16:55:01.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.131+0000 7efe7cc26640 1 -- 172.21.15.26:0/1271511956 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe700a5b40 con 0x7efe780d3300 2023-12-19T16:55:01.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.131+0000 7efe7cc26640 1 -- 172.21.15.26:0/1271511956 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700ad440 con 0x7efe780d3300 2023-12-19T16:55:01.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.131+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1271511956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe780d3300 msgr2=0x7efe780d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:01.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.131+0000 7efe7fe8c640 1 --2- 172.21.15.26:0/1271511956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe780d3300 0x7efe780d0e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efe70009d90 tx=0x7efe7001c920 comp rx=0 tx=0).stop 2023-12-19T16:55:01.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.131+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1271511956 shutdown_connections 2023-12-19T16:55:01.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.131+0000 7efe7fe8c640 1 --2- 172.21.15.26:0/1271511956 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe780d3300 0x7efe780d0e30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:01.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.131+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1271511956 >> 172.21.15.26:0/1271511956 conn(0x7efe7806a120 msgr2=0x7efe7806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:01.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.131+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1271511956 shutdown_connections 2023-12-19T16:55:01.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.131+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1271511956 wait complete. 2023-12-19T16:55:01.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.131+0000 7efe7fe8c640 1 Processor -- start 2023-12-19T16:55:01.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.131+0000 7efe7fe8c640 1 -- start start 2023-12-19T16:55:01.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7fe8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe780ce040 0x7efe781531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:01.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7fe8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe780d3300 0x7efe780b6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:01.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7fe8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe7815cd40 0x7efe780b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:01.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7fe8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe780ca380 con 0x7efe7815cd40 2023-12-19T16:55:01.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7fe8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe78153710 con 0x7efe780d3300 2023-12-19T16:55:01.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7fe8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7815c520 con 0x7efe780ce040 2023-12-19T16:55:01.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7e429640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe7815cd40 0x7efe780b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:01.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7e429640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe7815cd40 0x7efe780b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49802/0 (socket says 172.21.15.26:49802) 2023-12-19T16:55:01.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7e429640 1 -- 172.21.15.26:0/1453859898 learned_addr learned my addr 172.21.15.26:0/1453859898 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7dc28640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe780ce040 0x7efe781531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7d427640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe780d3300 0x7efe780b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7e429640 1 -- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe780ce040 msgr2=0x7efe781531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7e429640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe780ce040 0x7efe781531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7e429640 1 -- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe780d3300 msgr2=0x7efe780b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7e429640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe780d3300 0x7efe780b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:01.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7e429640 1 -- 172.21.15.26:0/1453859898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe70009a70 con 0x7efe7815cd40 2023-12-19T16:55:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7e429640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe7815cd40 0x7efe780b6b70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efe700b0040 tx=0x7efe700196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe6e7fc640 1 -- 172.21.15.26:0/1453859898 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700add90 con 0x7efe7815cd40 2023-12-19T16:55:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe6e7fc640 1 -- 172.21.15.26:0/1453859898 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe70009450 con 0x7efe7815cd40 2023-12-19T16:55:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1453859898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7815c800 con 0x7efe7815cd40 2023-12-19T16:55:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe6e7fc640 1 -- 172.21.15.26:0/1453859898 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700b9690 con 0x7efe7815cd40 2023-12-19T16:55:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1453859898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe780ca010 con 0x7efe7815cd40 2023-12-19T16:55:01.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.135+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1453859898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe78074270 con 0x7efe7815cd40 2023-12-19T16:55:01.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.139+0000 7efe6e7fc640 1 -- 172.21.15.26:0/1453859898 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7efe700b97f0 con 0x7efe7815cd40 2023-12-19T16:55:01.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.139+0000 7efe6e7fc640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe5c065bb0 0x7efe5c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:01.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.139+0000 7efe6e7fc640 1 -- 172.21.15.26:0/1453859898 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6441+0+0 (secure 0 0 0) 0x7efe7011ed10 con 0x7efe7815cd40 2023-12-19T16:55:01.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.139+0000 7efe7dc28640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe5c065bb0 0x7efe5c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:01.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.139+0000 7efe7dc28640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe5c065bb0 0x7efe5c068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe74000fc0 tx=0x7efe74009040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:01.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.143+0000 7efe6e7fc640 1 -- 172.21.15.26:0/1453859898 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efe700ec260 con 0x7efe7815cd40 2023-12-19T16:55:01.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1453859898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe7815f9d0 con 0x7efe7815cd40 2023-12-19T16:55:01.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe6e7fc640 1 -- 172.21.15.26:0/1453859898 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+17163 (secure 0 0 0) 0x7efe700b2090 con 0x7efe7815cd40 2023-12-19T16:55:01.306 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:55:01.306 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":60,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:55:00.584404+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:54:58.260985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T16:54:59.644467+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:55:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe5c065bb0 msgr2=0x7efe5c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe7fe8c640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe5c065bb0 0x7efe5c068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe74000fc0 tx=0x7efe74009040 comp rx=0 tx=0).stop 2023-12-19T16:55:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1453859898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe7815cd40 msgr2=0x7efe780b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe7fe8c640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe7815cd40 0x7efe780b6b70 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7efe700b0040 tx=0x7efe700196c0 comp rx=0 tx=0).stop 2023-12-19T16:55:01.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1453859898 shutdown_connections 2023-12-19T16:55:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe7fe8c640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe7815cd40 0x7efe780b6b70 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe7fe8c640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe780d3300 0x7efe780b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe7fe8c640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe5c065bb0 0x7efe5c068070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe7fe8c640 1 --2- 172.21.15.26:0/1453859898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe780ce040 0x7efe781531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1453859898 >> 172.21.15.26:0/1453859898 conn(0x7efe7806a120 msgr2=0x7efe7815fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1453859898 shutdown_connections 2023-12-19T16:55:01.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.303+0000 7efe7fe8c640 1 -- 172.21.15.26:0/1453859898 wait complete. 2023-12-19T16:55:01.328 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T16:55:01.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.471+0000 7fa352820640 1 Processor -- start 2023-12-19T16:55:01.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.471+0000 7fa352820640 1 -- start start 2023-12-19T16:55:01.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.471+0000 7fa352820640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa34c06f6c0 0x7fa34c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:01.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.471+0000 7fa352820640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa34c06fb60 con 0x7fa34c159820 2023-12-19T16:55:01.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.471+0000 7fa352820640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa34c06fe40 con 0x7fa34c159c00 2023-12-19T16:55:01.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.471+0000 7fa352820640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c15e470 con 0x7fa34c06f6c0 2023-12-19T16:55:01.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.471+0000 7fa34bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa34c159820 0x7fa34c0c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47690/0 (socket says 172.21.15.26:47690) 2023-12-19T16:55:01.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.471+0000 7fa34bfff640 1 -- 172.21.15.26:0/2970005333 learned_addr learned my addr 172.21.15.26:0/2970005333 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:55:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.471+0000 7fa34b7fe640 1 --2- 172.21.15.26:0/2970005333 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa34c06f6c0 0x7fa34c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.471+0000 7fa34affd640 1 -- 172.21.15.26:0/2970005333 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808970480 0 0) 0x7fa34c06fb60 con 0x7fa34c159820 2023-12-19T16:55:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.471+0000 7fa34affd640 1 -- 172.21.15.26:0/2970005333 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa334003530 con 0x7fa34c159820 2023-12-19T16:55:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.475+0000 7fa34affd640 1 -- 172.21.15.26:0/2970005333 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288787403 0 0) 0x7fa34c06fe40 con 0x7fa34c159c00 2023-12-19T16:55:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.475+0000 7fa34affd640 1 -- 172.21.15.26:0/2970005333 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa34c06fb60 con 0x7fa34c159c00 2023-12-19T16:55:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.475+0000 7fa34b7fe640 1 -- 172.21.15.26:0/2970005333 >> v1:172.21.15.160:6789/0 conn(0x7fa34c159c00 legacy=0x7fa34c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.475+0000 7fa34affd640 1 -- 172.21.15.26:0/2970005333 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2130307609 0 0) 0x7fa334003530 con 0x7fa34c159820 2023-12-19T16:55:01.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.475+0000 7fa34b7fe640 1 -- 172.21.15.26:0/2970005333 >> v1:172.21.15.26:6789/0 conn(0x7fa34c159820 legacy=0x7fa34c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:01.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.475+0000 7fa34b7fe640 1 -- 172.21.15.26:0/2970005333 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa34c070120 con 0x7fa34c06f6c0 2023-12-19T16:55:01.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.475+0000 7fa34b7fe640 1 --2- 172.21.15.26:0/2970005333 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa34c06f6c0 0x7fa34c15df30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa3400097f0 tx=0x7fa34001c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c79a8a8fc7aa14e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:01.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.475+0000 7fa34affd640 1 -- 172.21.15.26:0/2970005333 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c001680 con 0x7fa34c06f6c0 2023-12-19T16:55:01.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.475+0000 7fa34affd640 1 -- 172.21.15.26:0/2970005333 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3400a7050 con 0x7fa34c06f6c0 2023-12-19T16:55:01.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.475+0000 7fa34affd640 1 -- 172.21.15.26:0/2970005333 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3400ab620 con 0x7fa34c06f6c0 2023-12-19T16:55:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.475+0000 7fa352820640 1 -- 172.21.15.26:0/2970005333 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa34c06f6c0 msgr2=0x7fa34c15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.475+0000 7fa352820640 1 --2- 172.21.15.26:0/2970005333 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa34c06f6c0 0x7fa34c15df30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa3400097f0 tx=0x7fa34001c3b0 comp rx=0 tx=0).stop 2023-12-19T16:55:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa352820640 1 -- 172.21.15.26:0/2970005333 shutdown_connections 2023-12-19T16:55:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa352820640 1 --2- 172.21.15.26:0/2970005333 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa34c06f6c0 0x7fa34c15df30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa352820640 1 -- 172.21.15.26:0/2970005333 >> 172.21.15.26:0/2970005333 conn(0x7fa34c0ce730 msgr2=0x7fa34c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa352820640 1 -- 172.21.15.26:0/2970005333 shutdown_connections 2023-12-19T16:55:01.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa352820640 1 -- 172.21.15.26:0/2970005333 wait complete. 2023-12-19T16:55:01.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa352820640 1 Processor -- start 2023-12-19T16:55:01.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa352820640 1 -- start start 2023-12-19T16:55:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa352820640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa34c06f6c0 0x7fa34c15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa352820640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa34c159820 0x7fa34c15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa352820640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa34c159c00 0x7fa34c0c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa352820640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c14c4f0 con 0x7fa34c159c00 2023-12-19T16:55:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa352820640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c14bce0 con 0x7fa34c06f6c0 2023-12-19T16:55:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa352820640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34c15c300 con 0x7fa34c159820 2023-12-19T16:55:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa350dbd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa34c159c00 0x7fa34c0c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa350dbd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa34c159c00 0x7fa34c0c3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49804/0 (socket says 172.21.15.26:49804) 2023-12-19T16:55:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa350dbd640 1 -- 172.21.15.26:0/2113032515 learned_addr learned my addr 172.21.15.26:0/2113032515 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa34bfff640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa34c06f6c0 0x7fa34c15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa34b7fe640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa34c159820 0x7fa34c15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa350dbd640 1 -- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa34c159820 msgr2=0x7fa34c15bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa350dbd640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa34c159820 0x7fa34c15bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:01.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa350dbd640 1 -- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa34c06f6c0 msgr2=0x7fa34c15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:01.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa350dbd640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa34c06f6c0 0x7fa34c15a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:01.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.479+0000 7fa350dbd640 1 -- 172.21.15.26:0/2113032515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa340009420 con 0x7fa34c159c00 2023-12-19T16:55:01.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.483+0000 7fa350dbd640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa34c159c00 0x7fa34c0c3e20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa34400c730 tx=0x7fa34400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:01.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.483+0000 7fa348ff9640 1 -- 172.21.15.26:0/2113032515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344003b80 con 0x7fa34c159c00 2023-12-19T16:55:01.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.483+0000 7fa348ff9640 1 -- 172.21.15.26:0/2113032515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa344003ce0 con 0x7fa34c159c00 2023-12-19T16:55:01.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.483+0000 7fa348ff9640 1 -- 172.21.15.26:0/2113032515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34400aa90 con 0x7fa34c159c00 2023-12-19T16:55:01.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.483+0000 7fa352820640 1 -- 172.21.15.26:0/2113032515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa34c06b880 con 0x7fa34c159c00 2023-12-19T16:55:01.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.483+0000 7fa352820640 1 -- 172.21.15.26:0/2113032515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa34c14c210 con 0x7fa34c159c00 2023-12-19T16:55:01.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.483+0000 7fa352820640 1 -- 172.21.15.26:0/2113032515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34c074270 con 0x7fa34c159c00 2023-12-19T16:55:01.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.483+0000 7fa348ff9640 1 -- 172.21.15.26:0/2113032515 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa34400abf0 con 0x7fa34c159c00 2023-12-19T16:55:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.483+0000 7fa348ff9640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa338065b80 0x7fa338068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.483+0000 7fa348ff9640 1 -- 172.21.15.26:0/2113032515 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fa344083310 con 0x7fa34c159c00 2023-12-19T16:55:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.483+0000 7fa34bfff640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa338065b80 0x7fa338068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.487+0000 7fa34bfff640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa338065b80 0x7fa338068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa33c000f70 tx=0x7fa33c001700 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.487+0000 7fa348ff9640 1 -- 172.21.15.26:0/2113032515 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa344050980 con 0x7fa34c159c00 2023-12-19T16:55:01.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:01.683+0000 7fa352820640 1 -- 172.21.15.26:0/2113032515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa34c0cbf20 con 0x7fa34c159c00 2023-12-19T16:55:02.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.603+0000 7fa348ff9640 1 -- 172.21.15.26:0/2113032515 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa344053fa0 con 0x7fa34c159c00 2023-12-19T16:55:02.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.607+0000 7fa352820640 1 -- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa338065b80 msgr2=0x7fa338068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.607+0000 7fa352820640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa338065b80 0x7fa338068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa33c000f70 tx=0x7fa33c001700 comp rx=0 tx=0).stop 2023-12-19T16:55:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.607+0000 7fa352820640 1 -- 172.21.15.26:0/2113032515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa34c159c00 msgr2=0x7fa34c0c3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.607+0000 7fa352820640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa34c159c00 0x7fa34c0c3e20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa34400c730 tx=0x7fa34400cbf0 comp rx=0 tx=0).stop 2023-12-19T16:55:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.607+0000 7fa352820640 1 -- 172.21.15.26:0/2113032515 shutdown_connections 2023-12-19T16:55:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.607+0000 7fa352820640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa34c159c00 0x7fa34c0c3e20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.607+0000 7fa352820640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa338065b80 0x7fa338068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.607+0000 7fa352820640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa34c159820 0x7fa34c15bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.607+0000 7fa352820640 1 --2- 172.21.15.26:0/2113032515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa34c06f6c0 0x7fa34c15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.607+0000 7fa352820640 1 -- 172.21.15.26:0/2113032515 >> 172.21.15.26:0/2113032515 conn(0x7fa34c0ce730 msgr2=0x7fa34c070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.607+0000 7fa352820640 1 -- 172.21.15.26:0/2113032515 shutdown_connections 2023-12-19T16:55:02.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.607+0000 7fa352820640 1 -- 172.21.15.26:0/2113032515 wait complete. 2023-12-19T16:55:02.633 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T16:55:02.634 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T16:55:02.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde58220640 1 Processor -- start 2023-12-19T16:55:02.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde58220640 1 -- start start 2023-12-19T16:55:02.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde58220640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde500706d0 0x7fde500b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:02.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde58220640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde50070b20 con 0x7fde5015a9c0 2023-12-19T16:55:02.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde58220640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde50070d90 con 0x7fde5015ada0 2023-12-19T16:55:02.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde58220640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde500cc270 con 0x7fde500706d0 2023-12-19T16:55:02.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde55fbc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fde5015a9c0 0x7fde500b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47692/0 (socket says 172.21.15.26:47692) 2023-12-19T16:55:02.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde55fbc640 1 -- 172.21.15.26:0/3913763810 learned_addr learned my addr 172.21.15.26:0/3913763810 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:55:02.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde567bd640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde500706d0 0x7fde500b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:02.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 543687417 0 0) 0x7fde50070b20 con 0x7fde5015a9c0 2023-12-19T16:55:02.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde3c003530 con 0x7fde5015a9c0 2023-12-19T16:55:02.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3221623284 0 0) 0x7fde3c003530 con 0x7fde5015a9c0 2023-12-19T16:55:02.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde500706d0 msgr2=0x7fde500b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:02.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde500706d0 0x7fde500b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:02.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 >> v1:172.21.15.160:6789/0 conn(0x7fde5015ada0 legacy=0x7fde5006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde50071070 con 0x7fde5015a9c0 2023-12-19T16:55:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fde4c001680 con 0x7fde5015a9c0 2023-12-19T16:55:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 >> v1:172.21.15.26:6789/0 conn(0x7fde5015a9c0 legacy=0x7fde500b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde3c0039b0 0x7fde3c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde500706d0 0x7fde3c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde3c004d10 0x7fde3c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500cc270 con 0x7fde3c0039b0 2023-12-19T16:55:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c00ba90 con 0x7fde3c004d10 2023-12-19T16:55:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c00bc50 con 0x7fde500706d0 2023-12-19T16:55:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde55fbc640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde3c0039b0 0x7fde3c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde567bd640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde3c004d10 0x7fde3c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde557bb640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde500706d0 0x7fde3c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:02.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde55fbc640 1 -- 172.21.15.26:0/3913763810 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde500706d0 msgr2=0x7fde3c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde55fbc640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde500706d0 0x7fde3c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde55fbc640 1 -- 172.21.15.26:0/3913763810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde3c004d10 msgr2=0x7fde3c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:02.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.783+0000 7fde55fbc640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde3c004d10 0x7fde3c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde55fbc640 1 -- 172.21.15.26:0/3913763810 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c003530 con 0x7fde3c0039b0 2023-12-19T16:55:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde567bd640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde3c004d10 0x7fde3c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:55:02.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde557bb640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde500706d0 0x7fde3c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:55:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde55fbc640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde3c0039b0 0x7fde3c003db0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00c210 tx=0x7fde4c01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32242e58022d242b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c0a5b10 con 0x7fde3c0039b0 2023-12-19T16:55:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4c0a5c90 con 0x7fde3c0039b0 2023-12-19T16:55:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde54fba640 1 -- 172.21.15.26:0/3913763810 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c0058e0 con 0x7fde3c0039b0 2023-12-19T16:55:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde58220640 1 -- 172.21.15.26:0/3913763810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde3c0039b0 msgr2=0x7fde3c003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde58220640 1 --2- 172.21.15.26:0/3913763810 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde3c0039b0 0x7fde3c003db0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fde4c00c210 tx=0x7fde4c01cc30 comp rx=0 tx=0).stop 2023-12-19T16:55:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde567bd640 1 -- 172.21.15.26:0/3913763810 reap_dead start 2023-12-19T16:55:02.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde58220640 1 -- 172.21.15.26:0/3913763810 shutdown_connections 2023-12-19T16:55:02.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde58220640 1 -- 172.21.15.26:0/3913763810 >> 172.21.15.26:0/3913763810 conn(0x7fde500ce730 msgr2=0x7fde500ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:02.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde58220640 1 -- 172.21.15.26:0/3913763810 shutdown_connections 2023-12-19T16:55:02.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.787+0000 7fde58220640 1 -- 172.21.15.26:0/3913763810 wait complete. 2023-12-19T16:55:02.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde58220640 1 Processor -- start 2023-12-19T16:55:02.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde58220640 1 -- start start 2023-12-19T16:55:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde58220640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde3c0039b0 0x7fde50157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde58220640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5006c730 0x7fde500c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde58220640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde500c1ee0 0x7fde500c2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde58220640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500c2b40 con 0x7fde5006c730 2023-12-19T16:55:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde58220640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde501f5670 con 0x7fde500c1ee0 2023-12-19T16:55:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde58220640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde500c28d0 con 0x7fde3c0039b0 2023-12-19T16:55:02.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde557bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5006c730 0x7fde500c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde557bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5006c730 0x7fde500c3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49810/0 (socket says 172.21.15.26:49810) 2023-12-19T16:55:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde557bb640 1 -- 172.21.15.26:0/775399557 learned_addr learned my addr 172.21.15.26:0/775399557 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde567bd640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde500c1ee0 0x7fde500c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde55fbc640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde3c0039b0 0x7fde50157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde557bb640 1 -- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde3c0039b0 msgr2=0x7fde50157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde557bb640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde3c0039b0 0x7fde50157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde557bb640 1 -- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde500c1ee0 msgr2=0x7fde500c2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde557bb640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde500c1ee0 0x7fde500c2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde557bb640 1 -- 172.21.15.26:0/775399557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde500c47c0 con 0x7fde5006c730 2023-12-19T16:55:02.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde557bb640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5006c730 0x7fde500c3f00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fde4000d5b0 tx=0x7fde4000da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde467fc640 1 -- 172.21.15.26:0/775399557 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40007220 con 0x7fde5006c730 2023-12-19T16:55:02.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.791+0000 7fde467fc640 1 -- 172.21.15.26:0/775399557 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde40007380 con 0x7fde5006c730 2023-12-19T16:55:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.795+0000 7fde58220640 1 -- 172.21.15.26:0/775399557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde500c4aa0 con 0x7fde5006c730 2023-12-19T16:55:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.795+0000 7fde467fc640 1 -- 172.21.15.26:0/775399557 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde400067e0 con 0x7fde5006c730 2023-12-19T16:55:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.795+0000 7fde58220640 1 -- 172.21.15.26:0/775399557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde501f5b50 con 0x7fde5006c730 2023-12-19T16:55:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.795+0000 7fde58220640 1 -- 172.21.15.26:0/775399557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde501f5d70 con 0x7fde5006c730 2023-12-19T16:55:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.795+0000 7fde467fc640 1 -- 172.21.15.26:0/775399557 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fde40006940 con 0x7fde5006c730 2023-12-19T16:55:02.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.795+0000 7fde467fc640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fde34065b80 0x7fde34068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:02.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.795+0000 7fde467fc640 1 -- 172.21.15.26:0/775399557 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fde400834c0 con 0x7fde5006c730 2023-12-19T16:55:02.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.795+0000 7fde55fbc640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fde34065b80 0x7fde34068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:02.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.795+0000 7fde55fbc640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fde34065b80 0x7fde34068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0ab000 tx=0x7fde4c0a7070 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:02.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.799+0000 7fde467fc640 1 -- 172.21.15.26:0/775399557 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fde40050970 con 0x7fde5006c730 2023-12-19T16:55:02.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:02.991+0000 7fde58220640 1 -- 172.21.15.26:0/775399557 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fde500c4c90 con 0x7fde5006c730 2023-12-19T16:55:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.607+0000 7fde467fc640 1 -- 172.21.15.26:0/775399557 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7fde40053f90 con 0x7fde5006c730 2023-12-19T16:55:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.607+0000 7fde58220640 1 -- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fde34065b80 msgr2=0x7fde34068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.607+0000 7fde58220640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fde34065b80 0x7fde34068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0ab000 tx=0x7fde4c0a7070 comp rx=0 tx=0).stop 2023-12-19T16:55:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.607+0000 7fde58220640 1 -- 172.21.15.26:0/775399557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5006c730 msgr2=0x7fde500c3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.607+0000 7fde58220640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5006c730 0x7fde500c3f00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fde4000d5b0 tx=0x7fde4000da70 comp rx=0 tx=0).stop 2023-12-19T16:55:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.607+0000 7fde58220640 1 -- 172.21.15.26:0/775399557 shutdown_connections 2023-12-19T16:55:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.607+0000 7fde58220640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fde500c1ee0 0x7fde500c2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.607+0000 7fde58220640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5006c730 0x7fde500c3f00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.607+0000 7fde58220640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fde34065b80 0x7fde34068040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.607+0000 7fde58220640 1 --2- 172.21.15.26:0/775399557 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fde3c0039b0 0x7fde50157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.607+0000 7fde58220640 1 -- 172.21.15.26:0/775399557 >> 172.21.15.26:0/775399557 conn(0x7fde500ce730 msgr2=0x7fde500b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.607+0000 7fde58220640 1 -- 172.21.15.26:0/775399557 shutdown_connections 2023-12-19T16:55:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.607+0000 7fde58220640 1 -- 172.21.15.26:0/775399557 wait complete. 2023-12-19T16:55:03.635 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T16:55:03.636 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:55:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.779+0000 7f8e9f59e640 1 Processor -- start 2023-12-19T16:55:03.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.779+0000 7f8e9f59e640 1 -- start start 2023-12-19T16:55:03.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.779+0000 7f8e9f59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ea015ce40 0x7f8ea00d0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:03.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.779+0000 7f8e9f59e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea00d1ef0 con 0x7f8ea00cde60 2023-12-19T16:55:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.779+0000 7f8e9f59e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea00d12e0 con 0x7f8ea00b8310 2023-12-19T16:55:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.779+0000 7f8e9f59e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea00d15c0 con 0x7f8ea015ce40 2023-12-19T16:55:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.783+0000 7f8e9e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ea015ce40 0x7f8ea00d0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.783+0000 7f8e9e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ea015ce40 0x7f8ea00d0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49812/0 (socket says 172.21.15.26:49812) 2023-12-19T16:55:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.783+0000 7f8e9e59c640 1 -- 172.21.15.26:0/2276262267 learned_addr learned my addr 172.21.15.26:0/2276262267 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:03.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.783+0000 7f8e9e59c640 1 -- 172.21.15.26:0/2276262267 >> v1:172.21.15.160:6789/0 conn(0x7f8ea00b8310 legacy=0x7f8ea00cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:03.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.783+0000 7f8e9e59c640 1 -- 172.21.15.26:0/2276262267 >> v1:172.21.15.26:6789/0 conn(0x7f8ea00cde60 legacy=0x7f8ea00b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:03.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.783+0000 7f8e9e59c640 1 -- 172.21.15.26:0/2276262267 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea00d18a0 con 0x7f8ea015ce40 2023-12-19T16:55:03.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.783+0000 7f8e9dd9b640 1 -- 172.21.15.26:0/2276262267 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025602772 0 0) 0x7f8ea00d1ef0 con 0x7f8ea00cde60 2023-12-19T16:55:03.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.783+0000 7f8e9e59c640 1 --2- 172.21.15.26:0/2276262267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea015ce40 0x7f8ea00d0a90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009870 tx=0x7f8e9001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f5193010c0315e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:03.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.783+0000 7f8e9dd9b640 1 -- 172.21.15.26:0/2276262267 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e900a8a30 con 0x7f8ea015ce40 2023-12-19T16:55:03.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.783+0000 7f8e9dd9b640 1 -- 172.21.15.26:0/2276262267 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e900a8b90 con 0x7f8ea015ce40 2023-12-19T16:55:03.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.783+0000 7f8e9dd9b640 1 -- 172.21.15.26:0/2276262267 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e900b0440 con 0x7f8ea015ce40 2023-12-19T16:55:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 -- 172.21.15.26:0/2276262267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea015ce40 msgr2=0x7f8ea00d0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 --2- 172.21.15.26:0/2276262267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea015ce40 0x7f8ea00d0a90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009870 tx=0x7f8e9001ed30 comp rx=0 tx=0).stop 2023-12-19T16:55:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 -- 172.21.15.26:0/2276262267 shutdown_connections 2023-12-19T16:55:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 --2- 172.21.15.26:0/2276262267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea015ce40 0x7f8ea00d0a90 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 -- 172.21.15.26:0/2276262267 >> 172.21.15.26:0/2276262267 conn(0x7f8ea0065ab0 msgr2=0x7f8ea0065ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 -- 172.21.15.26:0/2276262267 shutdown_connections 2023-12-19T16:55:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 -- 172.21.15.26:0/2276262267 wait complete. 2023-12-19T16:55:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 Processor -- start 2023-12-19T16:55:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 -- start start 2023-12-19T16:55:03.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ea00b8310 0x7f8ea015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:03.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ea00cde60 0x7f8ea0151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:03.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea015ce40 0x7f8ea0151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:03.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea015ffa0 con 0x7f8ea015ce40 2023-12-19T16:55:03.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea01f5720 con 0x7f8ea00b8310 2023-12-19T16:55:03.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9f59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea01f5a00 con 0x7f8ea00cde60 2023-12-19T16:55:03.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea015ce40 0x7f8ea0151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:03.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea015ce40 0x7f8ea0151e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49814/0 (socket says 172.21.15.26:49814) 2023-12-19T16:55:03.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9ed9d640 1 -- 172.21.15.26:0/36184704 learned_addr learned my addr 172.21.15.26:0/36184704 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:03.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9ed9d640 1 -- 172.21.15.26:0/36184704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ea00cde60 msgr2=0x7f8ea0151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:03.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9ed9d640 1 --2- 172.21.15.26:0/36184704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ea00cde60 0x7f8ea0151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:03.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9ed9d640 1 -- 172.21.15.26:0/36184704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ea00b8310 msgr2=0x7f8ea015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:03.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9ed9d640 1 --2- 172.21.15.26:0/36184704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ea00b8310 0x7f8ea015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:03.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.787+0000 7f8e9ed9d640 1 -- 172.21.15.26:0/36184704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e900094d0 con 0x7f8ea015ce40 2023-12-19T16:55:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.791+0000 7f8e9ed9d640 1 --2- 172.21.15.26:0/36184704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea015ce40 0x7f8ea0151e40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800c770 tx=0x7f8e9800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.791+0000 7f8e96ffd640 1 -- 172.21.15.26:0/36184704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98003db0 con 0x7f8ea015ce40 2023-12-19T16:55:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.791+0000 7f8e96ffd640 1 -- 172.21.15.26:0/36184704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9800ce80 con 0x7f8ea015ce40 2023-12-19T16:55:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.791+0000 7f8e9f59e640 1 -- 172.21.15.26:0/36184704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea01f5ce0 con 0x7f8ea015ce40 2023-12-19T16:55:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.791+0000 7f8e96ffd640 1 -- 172.21.15.26:0/36184704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e980052b0 con 0x7f8ea015ce40 2023-12-19T16:55:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.791+0000 7f8e9f59e640 1 -- 172.21.15.26:0/36184704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea015b620 con 0x7f8ea015ce40 2023-12-19T16:55:03.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.791+0000 7f8e9f59e640 1 -- 172.21.15.26:0/36184704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea01f5580 con 0x7f8ea015ce40 2023-12-19T16:55:03.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.791+0000 7f8e96ffd640 1 -- 172.21.15.26:0/36184704 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8e98005410 con 0x7f8ea015ce40 2023-12-19T16:55:03.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.791+0000 7f8e96ffd640 1 --2- 172.21.15.26:0/36184704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8e80065b80 0x7f8e80068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:03.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.791+0000 7f8e96ffd640 1 -- 172.21.15.26:0/36184704 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f8e98082e90 con 0x7f8ea015ce40 2023-12-19T16:55:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.791+0000 7f8e9e59c640 1 --2- 172.21.15.26:0/36184704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8e80065b80 0x7f8e80068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:03.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.795+0000 7f8e9e59c640 1 --2- 172.21.15.26:0/36184704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8e80065b80 0x7f8e80068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e900b2000 tx=0x7f8e90004740 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:03.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.795+0000 7f8e96ffd640 1 -- 172.21.15.26:0/36184704 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8ea01f5580 con 0x7f8ea015ce40 2023-12-19T16:55:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e9f59e640 1 -- 172.21.15.26:0/36184704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ea00d2360 con 0x7f8ea015ce40 2023-12-19T16:55:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e96ffd640 1 -- 172.21.15.26:0/36184704 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f8ea00d2360 con 0x7f8ea015ce40 2023-12-19T16:55:03.990 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:55:03.990 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1947010592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1947010592},{"type":"v1","addr":"172.21.15.26:6833","nonce":1947010592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2764469548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2764469548},{"type":"v1","addr":"172.21.15.160:6837","nonce":2764469548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5803,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/255772664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":255772664},{"type":"v1","addr":"172.21.15.160:6836","nonce":255772664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:55:00.584757+0000","modified":"2023-12-19T16:55:03.604713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6000":{"gid":6000,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1847960813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1847960813},{"type":"v1","addr":"172.21.15.26:6835","nonce":1847960813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T16:55:03.990 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2023-12-19T16:55:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e9f59e640 1 -- 172.21.15.26:0/36184704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8e80065b80 msgr2=0x7f8e80068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e9f59e640 1 --2- 172.21.15.26:0/36184704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8e80065b80 0x7f8e80068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e900b2000 tx=0x7f8e90004740 comp rx=0 tx=0).stop 2023-12-19T16:55:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e9f59e640 1 -- 172.21.15.26:0/36184704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea015ce40 msgr2=0x7f8ea0151e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e9f59e640 1 --2- 172.21.15.26:0/36184704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea015ce40 0x7f8ea0151e40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800c770 tx=0x7f8e9800cc30 comp rx=0 tx=0).stop 2023-12-19T16:55:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e9f59e640 1 -- 172.21.15.26:0/36184704 shutdown_connections 2023-12-19T16:55:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e9f59e640 1 --2- 172.21.15.26:0/36184704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea015ce40 0x7f8ea0151e40 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e9f59e640 1 --2- 172.21.15.26:0/36184704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8e80065b80 0x7f8e80068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e9f59e640 1 --2- 172.21.15.26:0/36184704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ea00cde60 0x7f8ea0151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e9f59e640 1 --2- 172.21.15.26:0/36184704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ea00b8310 0x7f8ea015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e9f59e640 1 -- 172.21.15.26:0/36184704 >> 172.21.15.26:0/36184704 conn(0x7f8ea0065ab0 msgr2=0x7f8ea00b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e9f59e640 1 -- 172.21.15.26:0/36184704 shutdown_connections 2023-12-19T16:55:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:03.987+0000 7f8e9f59e640 1 -- 172.21.15.26:0/36184704 wait complete. 2023-12-19T16:55:04.012 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:55:04.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.151+0000 7f67d4866640 1 Processor -- start 2023-12-19T16:55:04.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.151+0000 7f67d4866640 1 -- start start 2023-12-19T16:55:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67d4866640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f67d015d260 0x7f67d006e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67d4866640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67d00b34f0 0x7f67d00b3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67d4866640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d00b3060 con 0x7f67d00658e0 2023-12-19T16:55:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67d4866640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d015d820 con 0x7f67d015d260 2023-12-19T16:55:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67d4866640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d015db00 con 0x7f67d00b34f0 2023-12-19T16:55:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67c7fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f67d015d260 0x7f67d006e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67ced9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67d00b34f0 0x7f67d00b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67c7fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f67d015d260 0x7f67d006e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:38024/0 (socket says 172.21.15.26:38024) 2023-12-19T16:55:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67c7fff640 1 -- 172.21.15.26:0/1646472493 learned_addr learned my addr 172.21.15.26:0/1646472493 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67ce59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f67d00658e0 0x7f67d0065cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:45208/0 (socket says 172.21.15.26:45208) 2023-12-19T16:55:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67cdd9b640 1 -- 172.21.15.26:0/1646472493 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1049392147 0 0) 0x7f67d00b3060 con 0x7f67d00658e0 2023-12-19T16:55:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67cdd9b640 1 -- 172.21.15.26:0/1646472493 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67bc003530 con 0x7f67d00658e0 2023-12-19T16:55:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67ced9d640 1 -- 172.21.15.26:0/1646472493 >> v2:172.21.15.160:3300/0 conn(0x7f67d015d260 msgr2=0x7f67d006e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67ced9d640 1 --2- 172.21.15.26:0/1646472493 >> v2:172.21.15.160:3300/0 conn(0x7f67d015d260 0x7f67d006e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67ced9d640 1 -- 172.21.15.26:0/1646472493 >> v1:172.21.15.160:6789/0 conn(0x7f67d00658e0 legacy=0x7f67d0065cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67ced9d640 1 -- 172.21.15.26:0/1646472493 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d015dde0 con 0x7f67d00b34f0 2023-12-19T16:55:04.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.155+0000 7f67c7fff640 1 --2- 172.21.15.26:0/1646472493 >> v2:172.21.15.160:3300/0 conn(0x7f67d015d260 0x7f67d006e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:55:04.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67ced9d640 1 --2- 172.21.15.26:0/1646472493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67d00b34f0 0x7f67d00b3960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009d90 tx=0x7f67c801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b58920ada578e89 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:04.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67cdd9b640 1 -- 172.21.15.26:0/1646472493 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80a5860 con 0x7f67d00b34f0 2023-12-19T16:55:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67cdd9b640 1 -- 172.21.15.26:0/1646472493 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c80a59c0 con 0x7f67d00b34f0 2023-12-19T16:55:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67cdd9b640 1 -- 172.21.15.26:0/1646472493 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80a5d50 con 0x7f67d00b34f0 2023-12-19T16:55:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67d4866640 1 -- 172.21.15.26:0/1646472493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67d00b34f0 msgr2=0x7f67d00b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67d4866640 1 --2- 172.21.15.26:0/1646472493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67d00b34f0 0x7f67d00b3960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009d90 tx=0x7f67c801c890 comp rx=0 tx=0).stop 2023-12-19T16:55:04.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67d4866640 1 -- 172.21.15.26:0/1646472493 shutdown_connections 2023-12-19T16:55:04.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67d4866640 1 --2- 172.21.15.26:0/1646472493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67d00b34f0 0x7f67d00b3960 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67d4866640 1 --2- 172.21.15.26:0/1646472493 >> v2:172.21.15.160:3300/0 conn(0x7f67d015d260 0x7f67d006e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67d4866640 1 -- 172.21.15.26:0/1646472493 >> 172.21.15.26:0/1646472493 conn(0x7f67d006a320 msgr2=0x7f67d00b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:04.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67d4866640 1 -- 172.21.15.26:0/1646472493 shutdown_connections 2023-12-19T16:55:04.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67d4866640 1 -- 172.21.15.26:0/1646472493 wait complete. 2023-12-19T16:55:04.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67d4866640 1 Processor -- start 2023-12-19T16:55:04.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.159+0000 7f67d4866640 1 -- start start 2023-12-19T16:55:04.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67d4866640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d00658e0 0x7f67d01f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67d4866640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67d00b34f0 0x7f67d015c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67d4866640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67d015d260 0x7f67d0161de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67d4866640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d0142100 con 0x7f67d015d260 2023-12-19T16:55:04.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67d4866640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d01f98e0 con 0x7f67d00658e0 2023-12-19T16:55:04.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67d4866640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d0144b90 con 0x7f67d00b34f0 2023-12-19T16:55:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67ced9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67d015d260 0x7f67d0161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67ced9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67d015d260 0x7f67d0161de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49816/0 (socket says 172.21.15.26:49816) 2023-12-19T16:55:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67ced9d640 1 -- 172.21.15.26:0/53456579 learned_addr learned my addr 172.21.15.26:0/53456579 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67ce59c640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d00658e0 0x7f67d01f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67c7fff640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67d00b34f0 0x7f67d015c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67ced9d640 1 -- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67d00b34f0 msgr2=0x7f67d015c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67ced9d640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67d00b34f0 0x7f67d015c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67ced9d640 1 -- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d00658e0 msgr2=0x7f67d01f93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67ced9d640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d00658e0 0x7f67d01f93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67ced9d640 1 -- 172.21.15.26:0/53456579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b80094d0 con 0x7f67d015d260 2023-12-19T16:55:04.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67ced9d640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67d015d260 0x7f67d0161de0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f67c80b0040 tx=0x7f67c8019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:04.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67c6ffd640 1 -- 172.21.15.26:0/53456579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80b9650 con 0x7f67d015d260 2023-12-19T16:55:04.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67c6ffd640 1 -- 172.21.15.26:0/53456579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c80b97b0 con 0x7f67d015d260 2023-12-19T16:55:04.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67c6ffd640 1 -- 172.21.15.26:0/53456579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80b9b60 con 0x7f67d015d260 2023-12-19T16:55:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67d4866640 1 -- 172.21.15.26:0/53456579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c8009a70 con 0x7f67d015d260 2023-12-19T16:55:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67d4866640 1 -- 172.21.15.26:0/53456579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d01452b0 con 0x7f67d015d260 2023-12-19T16:55:04.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.163+0000 7f67d4866640 1 -- 172.21.15.26:0/53456579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d0074270 con 0x7f67d015d260 2023-12-19T16:55:04.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.167+0000 7f67c6ffd640 1 -- 172.21.15.26:0/53456579 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f67c80b9cc0 con 0x7f67d015d260 2023-12-19T16:55:04.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.167+0000 7f67c6ffd640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f67b0065b80 0x7f67b0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.167+0000 7f67c6ffd640 1 -- 172.21.15.26:0/53456579 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f67c811e6e0 con 0x7f67d015d260 2023-12-19T16:55:04.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.167+0000 7f67ce59c640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f67b0065b80 0x7f67b0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.167+0000 7f67ce59c640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f67b0065b80 0x7f67b0068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f67c00011f0 tx=0x7f67c0007d40 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:04.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.171+0000 7f67c6ffd640 1 -- 172.21.15.26:0/53456579 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f67c80ebdd0 con 0x7f67d015d260 2023-12-19T16:55:04.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.327+0000 7f67d4866640 1 -- 172.21.15.26:0/53456579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67d00aca60 con 0x7f67d015d260 2023-12-19T16:55:04.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.331+0000 7f67c6ffd640 1 -- 172.21.15.26:0/53456579 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+17150 (secure 0 0 0) 0x7f67c80ef3f0 con 0x7f67d015d260 2023-12-19T16:55:04.334 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:55:04.334 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":61,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:55:01.587892+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:54:58.260985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T16:54:59.644467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:55:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.331+0000 7f67d4866640 1 -- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f67b0065b80 msgr2=0x7f67b0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.331+0000 7f67d4866640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f67b0065b80 0x7f67b0068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f67c00011f0 tx=0x7f67c0007d40 comp rx=0 tx=0).stop 2023-12-19T16:55:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.331+0000 7f67d4866640 1 -- 172.21.15.26:0/53456579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67d015d260 msgr2=0x7f67d0161de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.331+0000 7f67d4866640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67d015d260 0x7f67d0161de0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f67c80b0040 tx=0x7f67c8019630 comp rx=0 tx=0).stop 2023-12-19T16:55:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.331+0000 7f67d4866640 1 -- 172.21.15.26:0/53456579 shutdown_connections 2023-12-19T16:55:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.331+0000 7f67d4866640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67d015d260 0x7f67d0161de0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.331+0000 7f67d4866640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f67b0065b80 0x7f67b0068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.331+0000 7f67d4866640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67d00b34f0 0x7f67d015c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.331+0000 7f67d4866640 1 --2- 172.21.15.26:0/53456579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67d00658e0 0x7f67d01f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.331+0000 7f67d4866640 1 -- 172.21.15.26:0/53456579 >> 172.21.15.26:0/53456579 conn(0x7f67d006a320 msgr2=0x7f67d013ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:04.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.331+0000 7f67d4866640 1 -- 172.21.15.26:0/53456579 shutdown_connections 2023-12-19T16:55:04.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.331+0000 7f67d4866640 1 -- 172.21.15.26:0/53456579 wait complete. 2023-12-19T16:55:04.356 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T16:55:04.356 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:55:04.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.495+0000 7fad56cf9640 1 Processor -- start 2023-12-19T16:55:04.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.495+0000 7fad56cf9640 1 -- start start 2023-12-19T16:55:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.495+0000 7fad56cf9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fad5006b3d0 0x7fad500ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.495+0000 7fad56cf9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad500cec40 0x7fad5015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad56cf9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5006bc30 con 0x7fad50154220 2023-12-19T16:55:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad56cf9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad500cf0d0 con 0x7fad5006b3d0 2023-12-19T16:55:04.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad56cf9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad500cc6e0 con 0x7fad500cec40 2023-12-19T16:55:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad54a95640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fad50154220 0x7fad5015aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47712/0 (socket says 172.21.15.26:47712) 2023-12-19T16:55:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad54a95640 1 -- 172.21.15.26:0/3922884118 learned_addr learned my addr 172.21.15.26:0/3922884118 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:55:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad55296640 1 --2- 172.21.15.26:0/3922884118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad500cec40 0x7fad5015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4ffff640 1 --2- 172.21.15.26:0/3922884118 >> v2:172.21.15.160:3300/0 conn(0x7fad5006b3d0 0x7fad500ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753655107 0 0) 0x7fad5006bc30 con 0x7fad50154220 2023-12-19T16:55:04.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad40003530 con 0x7fad50154220 2023-12-19T16:55:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1158183141 0 0) 0x7fad40003530 con 0x7fad50154220 2023-12-19T16:55:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad500cec40 msgr2=0x7fad5015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 --2- 172.21.15.26:0/3922884118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad500cec40 0x7fad5015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 >> v2:172.21.15.160:3300/0 conn(0x7fad5006b3d0 msgr2=0x7fad500ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 --2- 172.21.15.26:0/3922884118 >> v2:172.21.15.160:3300/0 conn(0x7fad5006b3d0 0x7fad500ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5006b950 con 0x7fad50154220 2023-12-19T16:55:04.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fad44001680 con 0x7fad50154220 2023-12-19T16:55:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad55296640 1 --2- 172.21.15.26:0/3922884118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad500cec40 0x7fad5015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:55:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 >> v1:172.21.15.26:6789/0 conn(0x7fad50154220 legacy=0x7fad5015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 --2- 172.21.15.26:0/3922884118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad40003980 0x7fad40003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 --2- 172.21.15.26:0/3922884118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad500cec40 0x7fad400047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 --2- 172.21.15.26:0/3922884118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad40004d10 0x7fad40005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad500cf0d0 con 0x7fad40004d10 2023-12-19T16:55:04.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad500cc6e0 con 0x7fad40003980 2023-12-19T16:55:04.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4000b990 con 0x7fad500cec40 2023-12-19T16:55:04.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad55296640 1 --2- 172.21.15.26:0/3922884118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad40004d10 0x7fad40005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad4ffff640 1 --2- 172.21.15.26:0/3922884118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad500cec40 0x7fad400047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad54a95640 1 --2- 172.21.15.26:0/3922884118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad40003980 0x7fad40003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad55296640 1 -- 172.21.15.26:0/3922884118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad500cec40 msgr2=0x7fad400047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:04.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad55296640 1 --2- 172.21.15.26:0/3922884118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad500cec40 0x7fad400047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad55296640 1 -- 172.21.15.26:0/3922884118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad40003980 msgr2=0x7fad40003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:04.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad55296640 1 --2- 172.21.15.26:0/3922884118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad40003980 0x7fad40003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.499+0000 7fad55296640 1 -- 172.21.15.26:0/3922884118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad48009a70 con 0x7fad40004d10 2023-12-19T16:55:04.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.503+0000 7fad55296640 1 --2- 172.21.15.26:0/3922884118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad40004d10 0x7fad40005190 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fad4800e350 tx=0x7fad4801cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a34c80e55d8df8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:04.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.503+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480a5df0 con 0x7fad40004d10 2023-12-19T16:55:04.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.503+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4800f040 con 0x7fad40004d10 2023-12-19T16:55:04.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.503+0000 7fad4f7fe640 1 -- 172.21.15.26:0/3922884118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480ad640 con 0x7fad40004d10 2023-12-19T16:55:04.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.503+0000 7fad56cf9640 1 -- 172.21.15.26:0/3922884118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad40004d10 msgr2=0x7fad40005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.503+0000 7fad56cf9640 1 --2- 172.21.15.26:0/3922884118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad40004d10 0x7fad40005190 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fad4800e350 tx=0x7fad4801cde0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.503+0000 7fad55296640 1 -- 172.21.15.26:0/3922884118 reap_dead start 2023-12-19T16:55:04.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.503+0000 7fad56cf9640 1 -- 172.21.15.26:0/3922884118 shutdown_connections 2023-12-19T16:55:04.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.503+0000 7fad56cf9640 1 -- 172.21.15.26:0/3922884118 >> 172.21.15.26:0/3922884118 conn(0x7fad500658e0 msgr2=0x7fad50065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:04.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.503+0000 7fad56cf9640 1 -- 172.21.15.26:0/3922884118 shutdown_connections 2023-12-19T16:55:04.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.503+0000 7fad56cf9640 1 -- 172.21.15.26:0/3922884118 wait complete. 2023-12-19T16:55:04.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.503+0000 7fad56cf9640 1 Processor -- start 2023-12-19T16:55:04.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.503+0000 7fad56cf9640 1 -- start start 2023-12-19T16:55:04.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad56cf9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad40004d10 0x7fad5013ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad56cf9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad50146bb0 0x7fad50140b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad56cf9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad50141050 0x7fad50147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad56cf9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad50141500 con 0x7fad40004d10 2023-12-19T16:55:04.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad56cf9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad50143830 con 0x7fad50146bb0 2023-12-19T16:55:04.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad56cf9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad501405d0 con 0x7fad50141050 2023-12-19T16:55:04.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad54a95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad40004d10 0x7fad5013ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad54a95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad40004d10 0x7fad5013ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49828/0 (socket says 172.21.15.26:49828) 2023-12-19T16:55:04.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad54a95640 1 -- 172.21.15.26:0/1247878951 learned_addr learned my addr 172.21.15.26:0/1247878951 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad55296640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad50141050 0x7fad50147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad4ffff640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad50146bb0 0x7fad50140b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad54a95640 1 -- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad50141050 msgr2=0x7fad50147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad54a95640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad50141050 0x7fad50147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad54a95640 1 -- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad50146bb0 msgr2=0x7fad50140b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad54a95640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad50146bb0 0x7fad50140b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad54a95640 1 -- 172.21.15.26:0/1247878951 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5014c560 con 0x7fad40004d10 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad4ffff640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad50146bb0 0x7fad50140b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad54a95640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad40004d10 0x7fad5013ad70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fad44007310 tx=0x7fad440077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad4d7fa640 1 -- 172.21.15.26:0/1247878951 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4400f9c0 con 0x7fad40004d10 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad4d7fa640 1 -- 172.21.15.26:0/1247878951 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4400fb40 con 0x7fad40004d10 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad4d7fa640 1 -- 172.21.15.26:0/1247878951 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad440133f0 con 0x7fad40004d10 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad56cf9640 1 -- 172.21.15.26:0/1247878951 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad50142f90 con 0x7fad40004d10 2023-12-19T16:55:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad56cf9640 1 -- 172.21.15.26:0/1247878951 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad50143550 con 0x7fad40004d10 2023-12-19T16:55:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad56cf9640 1 -- 172.21.15.26:0/1247878951 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad50074270 con 0x7fad40004d10 2023-12-19T16:55:04.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.507+0000 7fad4d7fa640 1 -- 172.21.15.26:0/1247878951 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fad44013550 con 0x7fad40004d10 2023-12-19T16:55:04.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.511+0000 7fad4d7fa640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fad34065b80 0x7fad34068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.511+0000 7fad4d7fa640 1 -- 172.21.15.26:0/1247878951 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fad440837a0 con 0x7fad40004d10 2023-12-19T16:55:04.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.511+0000 7fad4ffff640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fad34065b80 0x7fad34068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.511+0000 7fad4ffff640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fad34065b80 0x7fad34068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fad5006f430 tx=0x7fad3c008040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.515+0000 7fad4d7fa640 1 -- 172.21.15.26:0/1247878951 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fad44050d60 con 0x7fad40004d10 2023-12-19T16:55:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.671+0000 7fad56cf9640 1 -- 172.21.15.26:0/1247878951 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad500ccc10 con 0x7fad34065b80 2023-12-19T16:55:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.683+0000 7fad4d7fa640 1 -- 172.21.15.26:0/1247878951 <== mgr.5587 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288826 (secure 0 0 0) 0x7fad3c0098b0 con 0x7fad34065b80 2023-12-19T16:55:04.689 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:55:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:55:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.683+0000 7fad56cf9640 1 -- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fad34065b80 msgr2=0x7fad34068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.683+0000 7fad56cf9640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fad34065b80 0x7fad34068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fad5006f430 tx=0x7fad3c008040 comp rx=0 tx=0).stop 2023-12-19T16:55:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.683+0000 7fad56cf9640 1 -- 172.21.15.26:0/1247878951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad40004d10 msgr2=0x7fad5013ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.683+0000 7fad56cf9640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad40004d10 0x7fad5013ad70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fad44007310 tx=0x7fad440077d0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.683+0000 7fad56cf9640 1 -- 172.21.15.26:0/1247878951 shutdown_connections 2023-12-19T16:55:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.683+0000 7fad56cf9640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fad34065b80 0x7fad34068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.683+0000 7fad56cf9640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad50141050 0x7fad50147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.683+0000 7fad56cf9640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad50146bb0 0x7fad50140b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.683+0000 7fad56cf9640 1 --2- 172.21.15.26:0/1247878951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad40004d10 0x7fad5013ad70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.683+0000 7fad56cf9640 1 -- 172.21.15.26:0/1247878951 >> 172.21.15.26:0/1247878951 conn(0x7fad500658e0 msgr2=0x7fad50153500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.687+0000 7fad56cf9640 1 -- 172.21.15.26:0/1247878951 shutdown_connections 2023-12-19T16:55:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.687+0000 7fad56cf9640 1 -- 172.21.15.26:0/1247878951 wait complete. 2023-12-19T16:55:04.708 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-19T16:55:03.592982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307592,"kb_used_data":6728,"kb_used_omap":0,"kb_used_meta":300800,"kb_avail":754667128,"statfs":{"total":773094113280,"available":772779139072,"internally_reserved":0,"allocated":6889472,"data_stored":3808980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308019200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.420217"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626736+0000","last_change":"2023-12-19T16:55:00.620221+0000","last_active":"2023-12-19T16:55:00.626736+0000","last_peered":"2023-12-19T16:55:00.626736+0000","last_clean":"2023-12-19T16:55:00.626736+0000","last_became_active":"2023-12-19T16:55:00.618228+0000","last_became_peered":"2023-12-19T16:55:00.618228+0000","last_unstale":"2023-12-19T16:55:00.626736+0000","last_undegraded":"2023-12-19T16:55:00.626736+0000","last_fullsized":"2023-12-19T16:55:00.626736+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:37:20.239966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.618139+0000","last_change":"2023-12-19T16:55:00.609820+0000","last_active":"2023-12-19T16:55:00.618139+0000","last_peered":"2023-12-19T16:55:00.618139+0000","last_clean":"2023-12-19T16:55:00.618139+0000","last_became_active":"2023-12-19T16:55:00.609537+0000","last_became_peered":"2023-12-19T16:55:00.609537+0000","last_unstale":"2023-12-19T16:55:00.618139+0000","last_undegraded":"2023-12-19T16:55:00.618139+0000","last_fullsized":"2023-12-19T16:55:00.618139+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:35.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.617828+0000","last_change":"2023-12-19T16:55:00.612390+0000","last_active":"2023-12-19T16:55:00.617828+0000","last_peered":"2023-12-19T16:55:00.617828+0000","last_clean":"2023-12-19T16:55:00.617828+0000","last_became_active":"2023-12-19T16:55:00.612111+0000","last_became_peered":"2023-12-19T16:55:00.612111+0000","last_unstale":"2023-12-19T16:55:00.617828+0000","last_undegraded":"2023-12-19T16:55:00.617828+0000","last_fullsized":"2023-12-19T16:55:00.617828+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:16.382474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630698+0000","last_change":"2023-12-19T16:55:00.625117+0000","last_active":"2023-12-19T16:55:00.630698+0000","last_peered":"2023-12-19T16:55:00.630698+0000","last_clean":"2023-12-19T16:55:00.630698+0000","last_became_active":"2023-12-19T16:55:00.624770+0000","last_became_peered":"2023-12-19T16:55:00.624770+0000","last_unstale":"2023-12-19T16:55:00.630698+0000","last_undegraded":"2023-12-19T16:55:00.630698+0000","last_fullsized":"2023-12-19T16:55:00.630698+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:39.380311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.610069+0000","last_change":"2023-12-19T16:55:00.605142+0000","last_active":"2023-12-19T16:55:00.610069+0000","last_peered":"2023-12-19T16:55:00.610069+0000","last_clean":"2023-12-19T16:55:00.610069+0000","last_became_active":"2023-12-19T16:55:00.604724+0000","last_became_peered":"2023-12-19T16:55:00.604724+0000","last_unstale":"2023-12-19T16:55:00.610069+0000","last_undegraded":"2023-12-19T16:55:00.610069+0000","last_fullsized":"2023-12-19T16:55:00.610069+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:09:51.151857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.622450+0000","last_change":"2023-12-19T16:55:00.609294+0000","last_active":"2023-12-19T16:55:00.622450+0000","last_peered":"2023-12-19T16:55:00.622450+0000","last_clean":"2023-12-19T16:55:00.622450+0000","last_became_active":"2023-12-19T16:55:00.609015+0000","last_became_peered":"2023-12-19T16:55:00.609015+0000","last_unstale":"2023-12-19T16:55:00.622450+0000","last_undegraded":"2023-12-19T16:55:00.622450+0000","last_fullsized":"2023-12-19T16:55:00.622450+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:03:13.666792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.627669+0000","last_change":"2023-12-19T16:55:00.626722+0000","last_active":"2023-12-19T16:55:00.627669+0000","last_peered":"2023-12-19T16:55:00.627669+0000","last_clean":"2023-12-19T16:55:00.627669+0000","last_became_active":"2023-12-19T16:55:00.626453+0000","last_became_peered":"2023-12-19T16:55:00.626453+0000","last_unstale":"2023-12-19T16:55:00.627669+0000","last_undegraded":"2023-12-19T16:55:00.627669+0000","last_fullsized":"2023-12-19T16:55:00.627669+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:08.692322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630446+0000","last_change":"2023-12-19T16:55:00.621881+0000","last_active":"2023-12-19T16:55:00.630446+0000","last_peered":"2023-12-19T16:55:00.630446+0000","last_clean":"2023-12-19T16:55:00.630446+0000","last_became_active":"2023-12-19T16:55:00.615081+0000","last_became_peered":"2023-12-19T16:55:00.615081+0000","last_unstale":"2023-12-19T16:55:00.630446+0000","last_undegraded":"2023-12-19T16:55:00.630446+0000","last_fullsized":"2023-12-19T16:55:00.630446+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:58:36.985090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630401+0000","last_change":"2023-12-19T16:55:00.622678+0000","last_active":"2023-12-19T16:55:00.630401+0000","last_peered":"2023-12-19T16:55:00.630401+0000","last_clean":"2023-12-19T16:55:00.630401+0000","last_became_active":"2023-12-19T16:55:00.622338+0000","last_became_peered":"2023-12-19T16:55:00.622338+0000","last_unstale":"2023-12-19T16:55:00.630401+0000","last_undegraded":"2023-12-19T16:55:00.630401+0000","last_fullsized":"2023-12-19T16:55:00.630401+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:50:51.444657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630327+0000","last_change":"2023-12-19T16:55:00.622394+0000","last_active":"2023-12-19T16:55:00.630327+0000","last_peered":"2023-12-19T16:55:00.630327+0000","last_clean":"2023-12-19T16:55:00.630327+0000","last_became_active":"2023-12-19T16:55:00.621864+0000","last_became_peered":"2023-12-19T16:55:00.621864+0000","last_unstale":"2023-12-19T16:55:00.630327+0000","last_undegraded":"2023-12-19T16:55:00.630327+0000","last_fullsized":"2023-12-19T16:55:00.630327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:32:07.178020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630679+0000","last_change":"2023-12-19T16:55:00.623233+0000","last_active":"2023-12-19T16:55:00.630679+0000","last_peered":"2023-12-19T16:55:00.630679+0000","last_clean":"2023-12-19T16:55:00.630679+0000","last_became_active":"2023-12-19T16:55:00.622954+0000","last_became_peered":"2023-12-19T16:55:00.622954+0000","last_unstale":"2023-12-19T16:55:00.630679+0000","last_undegraded":"2023-12-19T16:55:00.630679+0000","last_fullsized":"2023-12-19T16:55:00.630679+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:00:35.005593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.625316+0000","last_change":"2023-12-19T16:55:00.616111+0000","last_active":"2023-12-19T16:55:00.625316+0000","last_peered":"2023-12-19T16:55:00.625316+0000","last_clean":"2023-12-19T16:55:00.625316+0000","last_became_active":"2023-12-19T16:55:00.614258+0000","last_became_peered":"2023-12-19T16:55:00.614258+0000","last_unstale":"2023-12-19T16:55:00.625316+0000","last_undegraded":"2023-12-19T16:55:00.625316+0000","last_fullsized":"2023-12-19T16:55:00.625316+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:20.438452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.620644+0000","last_change":"2023-12-19T16:55:00.613795+0000","last_active":"2023-12-19T16:55:00.620644+0000","last_peered":"2023-12-19T16:55:00.620644+0000","last_clean":"2023-12-19T16:55:00.620644+0000","last_became_active":"2023-12-19T16:55:00.613524+0000","last_became_peered":"2023-12-19T16:55:00.613524+0000","last_unstale":"2023-12-19T16:55:00.620644+0000","last_undegraded":"2023-12-19T16:55:00.620644+0000","last_fullsized":"2023-12-19T16:55:00.620644+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:51.888168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626507+0000","last_change":"2023-12-19T16:55:00.614515+0000","last_active":"2023-12-19T16:55:00.626507+0000","last_peered":"2023-12-19T16:55:00.626507+0000","last_clean":"2023-12-19T16:55:00.626507+0000","last_became_active":"2023-12-19T16:55:00.614267+0000","last_became_peered":"2023-12-19T16:55:00.614267+0000","last_unstale":"2023-12-19T16:55:00.626507+0000","last_undegraded":"2023-12-19T16:55:00.626507+0000","last_fullsized":"2023-12-19T16:55:00.626507+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:56:22.346881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.625729+0000","last_change":"2023-12-19T16:55:00.614372+0000","last_active":"2023-12-19T16:55:00.625729+0000","last_peered":"2023-12-19T16:55:00.625729+0000","last_clean":"2023-12-19T16:55:00.625729+0000","last_became_active":"2023-12-19T16:55:00.614124+0000","last_became_peered":"2023-12-19T16:55:00.614124+0000","last_unstale":"2023-12-19T16:55:00.625729+0000","last_undegraded":"2023-12-19T16:55:00.625729+0000","last_fullsized":"2023-12-19T16:55:00.625729+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:54:08.915596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619546+0000","last_change":"2023-12-19T16:55:00.612879+0000","last_active":"2023-12-19T16:55:00.619546+0000","last_peered":"2023-12-19T16:55:00.619546+0000","last_clean":"2023-12-19T16:55:00.619546+0000","last_became_active":"2023-12-19T16:55:00.612488+0000","last_became_peered":"2023-12-19T16:55:00.612488+0000","last_unstale":"2023-12-19T16:55:00.619546+0000","last_undegraded":"2023-12-19T16:55:00.619546+0000","last_fullsized":"2023-12-19T16:55:00.619546+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:29:50.782409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.610468+0000","last_change":"2023-12-19T16:55:00.600944+0000","last_active":"2023-12-19T16:55:00.610468+0000","last_peered":"2023-12-19T16:55:00.610468+0000","last_clean":"2023-12-19T16:55:00.610468+0000","last_became_active":"2023-12-19T16:55:00.600645+0000","last_became_peered":"2023-12-19T16:55:00.600645+0000","last_unstale":"2023-12-19T16:55:00.610468+0000","last_undegraded":"2023-12-19T16:55:00.610468+0000","last_fullsized":"2023-12-19T16:55:00.610468+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:38.197379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.618066+0000","last_change":"2023-12-19T16:55:00.610429+0000","last_active":"2023-12-19T16:55:00.618066+0000","last_peered":"2023-12-19T16:55:00.618066+0000","last_clean":"2023-12-19T16:55:00.618066+0000","last_became_active":"2023-12-19T16:55:00.607100+0000","last_became_peered":"2023-12-19T16:55:00.607100+0000","last_unstale":"2023-12-19T16:55:00.618066+0000","last_undegraded":"2023-12-19T16:55:00.618066+0000","last_fullsized":"2023-12-19T16:55:00.618066+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:47.181094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628947+0000","last_change":"2023-12-19T16:55:00.625929+0000","last_active":"2023-12-19T16:55:00.628947+0000","last_peered":"2023-12-19T16:55:00.628947+0000","last_clean":"2023-12-19T16:55:00.628947+0000","last_became_active":"2023-12-19T16:55:00.625570+0000","last_became_peered":"2023-12-19T16:55:00.625570+0000","last_unstale":"2023-12-19T16:55:00.628947+0000","last_undegraded":"2023-12-19T16:55:00.628947+0000","last_fullsized":"2023-12-19T16:55:00.628947+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:31:45.628790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.611055+0000","last_change":"2023-12-19T16:55:00.608863+0000","last_active":"2023-12-19T16:55:00.611055+0000","last_peered":"2023-12-19T16:55:00.611055+0000","last_clean":"2023-12-19T16:55:00.611055+0000","last_became_active":"2023-12-19T16:55:00.608027+0000","last_became_peered":"2023-12-19T16:55:00.608027+0000","last_unstale":"2023-12-19T16:55:00.611055+0000","last_undegraded":"2023-12-19T16:55:00.611055+0000","last_fullsized":"2023-12-19T16:55:00.611055+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:54.572124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626029+0000","last_change":"2023-12-19T16:55:00.617614+0000","last_active":"2023-12-19T16:55:00.626029+0000","last_peered":"2023-12-19T16:55:00.626029+0000","last_clean":"2023-12-19T16:55:00.626029+0000","last_became_active":"2023-12-19T16:55:00.616766+0000","last_became_peered":"2023-12-19T16:55:00.616766+0000","last_unstale":"2023-12-19T16:55:00.626029+0000","last_undegraded":"2023-12-19T16:55:00.626029+0000","last_fullsized":"2023-12-19T16:55:00.626029+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:15.272341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.627110+0000","last_change":"2023-12-19T16:55:00.617539+0000","last_active":"2023-12-19T16:55:00.627110+0000","last_peered":"2023-12-19T16:55:00.627110+0000","last_clean":"2023-12-19T16:55:00.627110+0000","last_became_active":"2023-12-19T16:55:00.617090+0000","last_became_peered":"2023-12-19T16:55:00.617090+0000","last_unstale":"2023-12-19T16:55:00.627110+0000","last_undegraded":"2023-12-19T16:55:00.627110+0000","last_fullsized":"2023-12-19T16:55:00.627110+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:59:07.234068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628229+0000","last_change":"2023-12-19T16:55:00.624739+0000","last_active":"2023-12-19T16:55:00.628229+0000","last_peered":"2023-12-19T16:55:00.628229+0000","last_clean":"2023-12-19T16:55:00.628229+0000","last_became_active":"2023-12-19T16:55:00.624319+0000","last_became_peered":"2023-12-19T16:55:00.624319+0000","last_unstale":"2023-12-19T16:55:00.628229+0000","last_undegraded":"2023-12-19T16:55:00.628229+0000","last_fullsized":"2023-12-19T16:55:00.628229+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:49:32.328163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.620249+0000","last_change":"2023-12-19T16:55:00.615210+0000","last_active":"2023-12-19T16:55:00.620249+0000","last_peered":"2023-12-19T16:55:00.620249+0000","last_clean":"2023-12-19T16:55:00.620249+0000","last_became_active":"2023-12-19T16:55:00.614909+0000","last_became_peered":"2023-12-19T16:55:00.614909+0000","last_unstale":"2023-12-19T16:55:00.620249+0000","last_undegraded":"2023-12-19T16:55:00.620249+0000","last_fullsized":"2023-12-19T16:55:00.620249+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:17:29.306993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628146+0000","last_change":"2023-12-19T16:55:00.624275+0000","last_active":"2023-12-19T16:55:00.628146+0000","last_peered":"2023-12-19T16:55:00.628146+0000","last_clean":"2023-12-19T16:55:00.628146+0000","last_became_active":"2023-12-19T16:55:00.623919+0000","last_became_peered":"2023-12-19T16:55:00.623919+0000","last_unstale":"2023-12-19T16:55:00.628146+0000","last_undegraded":"2023-12-19T16:55:00.628146+0000","last_fullsized":"2023-12-19T16:55:00.628146+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:16.382474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.623300+0000","last_change":"2023-12-19T16:55:00.620298+0000","last_active":"2023-12-19T16:55:00.623300+0000","last_peered":"2023-12-19T16:55:00.623300+0000","last_clean":"2023-12-19T16:55:00.623300+0000","last_became_active":"2023-12-19T16:55:00.619959+0000","last_became_peered":"2023-12-19T16:55:00.619959+0000","last_unstale":"2023-12-19T16:55:00.623300+0000","last_undegraded":"2023-12-19T16:55:00.623300+0000","last_fullsized":"2023-12-19T16:55:00.623300+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:49:39.135191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.606623+0000","last_change":"2023-12-19T16:55:00.599837+0000","last_active":"2023-12-19T16:55:00.606623+0000","last_peered":"2023-12-19T16:55:00.606623+0000","last_clean":"2023-12-19T16:55:00.606623+0000","last_became_active":"2023-12-19T16:55:00.599458+0000","last_became_peered":"2023-12-19T16:55:00.599458+0000","last_unstale":"2023-12-19T16:55:00.606623+0000","last_undegraded":"2023-12-19T16:55:00.606623+0000","last_fullsized":"2023-12-19T16:55:00.606623+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:17:29.306993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619153+0000","last_change":"2023-12-19T16:55:00.610412+0000","last_active":"2023-12-19T16:55:00.619153+0000","last_peered":"2023-12-19T16:55:00.619153+0000","last_clean":"2023-12-19T16:55:00.619153+0000","last_became_active":"2023-12-19T16:55:00.610144+0000","last_became_peered":"2023-12-19T16:55:00.610144+0000","last_unstale":"2023-12-19T16:55:00.619153+0000","last_undegraded":"2023-12-19T16:55:00.619153+0000","last_fullsized":"2023-12-19T16:55:00.619153+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:56.751969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619783+0000","last_change":"2023-12-19T16:55:00.613888+0000","last_active":"2023-12-19T16:55:00.619783+0000","last_peered":"2023-12-19T16:55:00.619783+0000","last_clean":"2023-12-19T16:55:00.619783+0000","last_became_active":"2023-12-19T16:55:00.613516+0000","last_became_peered":"2023-12-19T16:55:00.613516+0000","last_unstale":"2023-12-19T16:55:00.619783+0000","last_undegraded":"2023-12-19T16:55:00.619783+0000","last_fullsized":"2023-12-19T16:55:00.619783+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:29:27.000630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619094+0000","last_change":"2023-12-19T16:55:00.616545+0000","last_active":"2023-12-19T16:55:00.619094+0000","last_peered":"2023-12-19T16:55:00.619094+0000","last_clean":"2023-12-19T16:55:00.619094+0000","last_became_active":"2023-12-19T16:55:00.616229+0000","last_became_peered":"2023-12-19T16:55:00.616229+0000","last_unstale":"2023-12-19T16:55:00.619094+0000","last_undegraded":"2023-12-19T16:55:00.619094+0000","last_fullsized":"2023-12-19T16:55:00.619094+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:10.646267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.621651+0000","last_change":"2023-12-19T16:55:00.605354+0000","last_active":"2023-12-19T16:55:00.621651+0000","last_peered":"2023-12-19T16:55:00.621651+0000","last_clean":"2023-12-19T16:55:00.621651+0000","last_became_active":"2023-12-19T16:55:00.605004+0000","last_became_peered":"2023-12-19T16:55:00.605004+0000","last_unstale":"2023-12-19T16:55:00.621651+0000","last_undegraded":"2023-12-19T16:55:00.621651+0000","last_fullsized":"2023-12-19T16:55:00.621651+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:20:45.257107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.611498+0000","last_change":"2023-12-19T16:55:00.607628+0000","last_active":"2023-12-19T16:55:00.611498+0000","last_peered":"2023-12-19T16:55:00.611498+0000","last_clean":"2023-12-19T16:55:00.611498+0000","last_became_active":"2023-12-19T16:55:00.607355+0000","last_became_peered":"2023-12-19T16:55:00.607355+0000","last_unstale":"2023-12-19T16:55:00.611498+0000","last_undegraded":"2023-12-19T16:55:00.611498+0000","last_fullsized":"2023-12-19T16:55:00.611498+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:20:06.746240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.627736+0000","last_change":"2023-12-19T16:55:00.623978+0000","last_active":"2023-12-19T16:55:00.627736+0000","last_peered":"2023-12-19T16:55:00.627736+0000","last_clean":"2023-12-19T16:55:00.627736+0000","last_became_active":"2023-12-19T16:55:00.623661+0000","last_became_peered":"2023-12-19T16:55:00.623661+0000","last_unstale":"2023-12-19T16:55:00.627736+0000","last_undegraded":"2023-12-19T16:55:00.627736+0000","last_fullsized":"2023-12-19T16:55:00.627736+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:56.751969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"60'4","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.627626+0000","last_change":"2023-12-19T16:55:00.203278+0000","last_active":"2023-12-19T16:55:00.627626+0000","last_peered":"2023-12-19T16:55:00.627626+0000","last_clean":"2023-12-19T16:55:00.627626+0000","last_became_active":"2023-12-19T16:55:00.202906+0000","last_became_peered":"2023-12-19T16:55:00.202906+0000","last_unstale":"2023-12-19T16:55:00.627626+0000","last_undegraded":"2023-12-19T16:55:00.627626+0000","last_fullsized":"2023-12-19T16:55:00.627626+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:45:00.387064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619873+0000","last_change":"2023-12-19T16:55:00.613979+0000","last_active":"2023-12-19T16:55:00.619873+0000","last_peered":"2023-12-19T16:55:00.619873+0000","last_clean":"2023-12-19T16:55:00.619873+0000","last_became_active":"2023-12-19T16:55:00.612765+0000","last_became_peered":"2023-12-19T16:55:00.612765+0000","last_unstale":"2023-12-19T16:55:00.619873+0000","last_undegraded":"2023-12-19T16:55:00.619873+0000","last_fullsized":"2023-12-19T16:55:00.619873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:37:19.208105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.620329+0000","last_change":"2023-12-19T16:55:00.202642+0000","last_active":"2023-12-19T16:55:00.620329+0000","last_peered":"2023-12-19T16:55:00.620329+0000","last_clean":"2023-12-19T16:55:00.620329+0000","last_became_active":"2023-12-19T16:55:00.202330+0000","last_became_peered":"2023-12-19T16:55:00.202330+0000","last_unstale":"2023-12-19T16:55:00.620329+0000","last_undegraded":"2023-12-19T16:55:00.620329+0000","last_fullsized":"2023-12-19T16:55:00.620329+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:34:07.118135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.611708+0000","last_change":"2023-12-19T16:55:00.608631+0000","last_active":"2023-12-19T16:55:00.611708+0000","last_peered":"2023-12-19T16:55:00.611708+0000","last_clean":"2023-12-19T16:55:00.611708+0000","last_became_active":"2023-12-19T16:55:00.608349+0000","last_became_peered":"2023-12-19T16:55:00.608349+0000","last_unstale":"2023-12-19T16:55:00.611708+0000","last_undegraded":"2023-12-19T16:55:00.611708+0000","last_fullsized":"2023-12-19T16:55:00.611708+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:40:05.720430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626950+0000","last_change":"2023-12-19T16:55:00.201865+0000","last_active":"2023-12-19T16:55:00.626950+0000","last_peered":"2023-12-19T16:55:00.626950+0000","last_clean":"2023-12-19T16:55:00.626950+0000","last_became_active":"2023-12-19T16:55:00.195845+0000","last_became_peered":"2023-12-19T16:55:00.195845+0000","last_unstale":"2023-12-19T16:55:00.626950+0000","last_undegraded":"2023-12-19T16:55:00.626950+0000","last_fullsized":"2023-12-19T16:55:00.626950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:18.347062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.624843+0000","last_change":"2023-12-19T16:55:00.619596+0000","last_active":"2023-12-19T16:55:00.624843+0000","last_peered":"2023-12-19T16:55:00.624843+0000","last_clean":"2023-12-19T16:55:00.624843+0000","last_became_active":"2023-12-19T16:55:00.619308+0000","last_became_peered":"2023-12-19T16:55:00.619308+0000","last_unstale":"2023-12-19T16:55:00.624843+0000","last_undegraded":"2023-12-19T16:55:00.624843+0000","last_fullsized":"2023-12-19T16:55:00.624843+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:28.722500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619957+0000","last_change":"2023-12-19T16:55:00.197689+0000","last_active":"2023-12-19T16:55:00.619957+0000","last_peered":"2023-12-19T16:55:00.619957+0000","last_clean":"2023-12-19T16:55:00.619957+0000","last_became_active":"2023-12-19T16:55:00.197410+0000","last_became_peered":"2023-12-19T16:55:00.197410+0000","last_unstale":"2023-12-19T16:55:00.619957+0000","last_undegraded":"2023-12-19T16:55:00.619957+0000","last_fullsized":"2023-12-19T16:55:00.619957+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:18.347062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619073+0000","last_change":"2023-12-19T16:55:00.614966+0000","last_active":"2023-12-19T16:55:00.619073+0000","last_peered":"2023-12-19T16:55:00.619073+0000","last_clean":"2023-12-19T16:55:00.619073+0000","last_became_active":"2023-12-19T16:55:00.614699+0000","last_became_peered":"2023-12-19T16:55:00.614699+0000","last_unstale":"2023-12-19T16:55:00.619073+0000","last_undegraded":"2023-12-19T16:55:00.619073+0000","last_fullsized":"2023-12-19T16:55:00.619073+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:37:19.208105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.622818+0000","last_change":"2023-12-19T16:55:00.205410+0000","last_active":"2023-12-19T16:55:00.622818+0000","last_peered":"2023-12-19T16:55:00.622818+0000","last_clean":"2023-12-19T16:55:00.622818+0000","last_became_active":"2023-12-19T16:55:00.205108+0000","last_became_peered":"2023-12-19T16:55:00.205108+0000","last_unstale":"2023-12-19T16:55:00.622818+0000","last_undegraded":"2023-12-19T16:55:00.622818+0000","last_fullsized":"2023-12-19T16:55:00.622818+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:55.744148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.621735+0000","last_change":"2023-12-19T16:55:00.610650+0000","last_active":"2023-12-19T16:55:00.621735+0000","last_peered":"2023-12-19T16:55:00.621735+0000","last_clean":"2023-12-19T16:55:00.621735+0000","last_became_active":"2023-12-19T16:55:00.610355+0000","last_became_peered":"2023-12-19T16:55:00.610355+0000","last_unstale":"2023-12-19T16:55:00.621735+0000","last_undegraded":"2023-12-19T16:55:00.621735+0000","last_fullsized":"2023-12-19T16:55:00.621735+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:36:43.798547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.615413+0000","last_change":"2023-12-19T16:55:00.203032+0000","last_active":"2023-12-19T16:55:00.615413+0000","last_peered":"2023-12-19T16:55:00.615413+0000","last_clean":"2023-12-19T16:55:00.615413+0000","last_became_active":"2023-12-19T16:55:00.202321+0000","last_became_peered":"2023-12-19T16:55:00.202321+0000","last_unstale":"2023-12-19T16:55:00.615413+0000","last_undegraded":"2023-12-19T16:55:00.615413+0000","last_fullsized":"2023-12-19T16:55:00.615413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:31.179320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630432+0000","last_change":"2023-12-19T16:55:00.615247+0000","last_active":"2023-12-19T16:55:00.630432+0000","last_peered":"2023-12-19T16:55:00.630432+0000","last_clean":"2023-12-19T16:55:00.630432+0000","last_became_active":"2023-12-19T16:55:00.614972+0000","last_became_peered":"2023-12-19T16:55:00.614972+0000","last_unstale":"2023-12-19T16:55:00.630432+0000","last_undegraded":"2023-12-19T16:55:00.630432+0000","last_fullsized":"2023-12-19T16:55:00.630432+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:09:07.090780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.595852+0000","last_change":"2023-12-19T16:55:00.208977+0000","last_active":"2023-12-19T16:55:00.595852+0000","last_peered":"2023-12-19T16:55:00.595852+0000","last_clean":"2023-12-19T16:55:00.595852+0000","last_became_active":"2023-12-19T16:55:00.208559+0000","last_became_peered":"2023-12-19T16:55:00.208559+0000","last_unstale":"2023-12-19T16:55:00.595852+0000","last_undegraded":"2023-12-19T16:55:00.595852+0000","last_fullsized":"2023-12-19T16:55:00.595852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:31:44.620969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626085+0000","last_change":"2023-12-19T16:55:00.621564+0000","last_active":"2023-12-19T16:55:00.626085+0000","last_peered":"2023-12-19T16:55:00.626085+0000","last_clean":"2023-12-19T16:55:00.626085+0000","last_became_active":"2023-12-19T16:55:00.620575+0000","last_became_peered":"2023-12-19T16:55:00.620575+0000","last_unstale":"2023-12-19T16:55:00.626085+0000","last_undegraded":"2023-12-19T16:55:00.626085+0000","last_fullsized":"2023-12-19T16:55:00.626085+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:40.188691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.590625+0000","last_change":"2023-12-19T16:55:00.209278+0000","last_active":"2023-12-19T16:55:00.590625+0000","last_peered":"2023-12-19T16:55:00.590625+0000","last_clean":"2023-12-19T16:55:00.590625+0000","last_became_active":"2023-12-19T16:55:00.208723+0000","last_became_peered":"2023-12-19T16:55:00.208723+0000","last_unstale":"2023-12-19T16:55:00.590625+0000","last_undegraded":"2023-12-19T16:55:00.590625+0000","last_fullsized":"2023-12-19T16:55:00.590625+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:08:07.783530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.618039+0000","last_change":"2023-12-19T16:55:00.607387+0000","last_active":"2023-12-19T16:55:00.618039+0000","last_peered":"2023-12-19T16:55:00.618039+0000","last_clean":"2023-12-19T16:55:00.618039+0000","last_became_active":"2023-12-19T16:55:00.606750+0000","last_became_peered":"2023-12-19T16:55:00.606750+0000","last_unstale":"2023-12-19T16:55:00.618039+0000","last_undegraded":"2023-12-19T16:55:00.618039+0000","last_fullsized":"2023-12-19T16:55:00.618039+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:40.188691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.593440+0000","last_change":"2023-12-19T16:55:00.197477+0000","last_active":"2023-12-19T16:55:00.593440+0000","last_peered":"2023-12-19T16:55:00.593440+0000","last_clean":"2023-12-19T16:55:00.593440+0000","last_became_active":"2023-12-19T16:55:00.197165+0000","last_became_peered":"2023-12-19T16:55:00.197165+0000","last_unstale":"2023-12-19T16:55:00.593440+0000","last_undegraded":"2023-12-19T16:55:00.593440+0000","last_fullsized":"2023-12-19T16:55:00.593440+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:48:05.029740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.623814+0000","last_change":"2023-12-19T16:55:00.616611+0000","last_active":"2023-12-19T16:55:00.623814+0000","last_peered":"2023-12-19T16:55:00.623814+0000","last_clean":"2023-12-19T16:55:00.623814+0000","last_became_active":"2023-12-19T16:55:00.616337+0000","last_became_peered":"2023-12-19T16:55:00.616337+0000","last_unstale":"2023-12-19T16:55:00.623814+0000","last_undegraded":"2023-12-19T16:55:00.623814+0000","last_fullsized":"2023-12-19T16:55:00.623814+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:17:37.827059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.591089+0000","last_change":"2023-12-19T16:55:00.208869+0000","last_active":"2023-12-19T16:55:00.591089+0000","last_peered":"2023-12-19T16:55:00.591089+0000","last_clean":"2023-12-19T16:55:00.591089+0000","last_became_active":"2023-12-19T16:55:00.208526+0000","last_became_peered":"2023-12-19T16:55:00.208526+0000","last_unstale":"2023-12-19T16:55:00.591089+0000","last_undegraded":"2023-12-19T16:55:00.591089+0000","last_fullsized":"2023-12-19T16:55:00.591089+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:44:59.307702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.625861+0000","last_change":"2023-12-19T16:55:00.615664+0000","last_active":"2023-12-19T16:55:00.625861+0000","last_peered":"2023-12-19T16:55:00.625861+0000","last_clean":"2023-12-19T16:55:00.625861+0000","last_became_active":"2023-12-19T16:55:00.615384+0000","last_became_peered":"2023-12-19T16:55:00.615384+0000","last_unstale":"2023-12-19T16:55:00.625861+0000","last_undegraded":"2023-12-19T16:55:00.625861+0000","last_fullsized":"2023-12-19T16:55:00.625861+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:17:23.459042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"60'3","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.623815+0000","last_change":"2023-12-19T16:55:00.200264+0000","last_active":"2023-12-19T16:55:00.623815+0000","last_peered":"2023-12-19T16:55:00.623815+0000","last_clean":"2023-12-19T16:55:00.623815+0000","last_became_active":"2023-12-19T16:55:00.199925+0000","last_became_peered":"2023-12-19T16:55:00.199925+0000","last_unstale":"2023-12-19T16:55:00.623815+0000","last_undegraded":"2023-12-19T16:55:00.623815+0000","last_fullsized":"2023-12-19T16:55:00.623815+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:43:24.212407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626532+0000","last_change":"2023-12-19T16:55:00.619067+0000","last_active":"2023-12-19T16:55:00.626532+0000","last_peered":"2023-12-19T16:55:00.626532+0000","last_clean":"2023-12-19T16:55:00.626532+0000","last_became_active":"2023-12-19T16:55:00.618681+0000","last_became_peered":"2023-12-19T16:55:00.618681+0000","last_unstale":"2023-12-19T16:55:00.626532+0000","last_undegraded":"2023-12-19T16:55:00.626532+0000","last_fullsized":"2023-12-19T16:55:00.626532+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:04:02.627170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"60'6","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.629964+0000","last_change":"2023-12-19T16:55:00.201264+0000","last_active":"2023-12-19T16:55:00.629964+0000","last_peered":"2023-12-19T16:55:00.629964+0000","last_clean":"2023-12-19T16:55:00.629964+0000","last_became_active":"2023-12-19T16:55:00.197880+0000","last_became_peered":"2023-12-19T16:55:00.197880+0000","last_unstale":"2023-12-19T16:55:00.629964+0000","last_undegraded":"2023-12-19T16:55:00.629964+0000","last_fullsized":"2023-12-19T16:55:00.629964+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:53:38.563630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.620124+0000","last_change":"2023-12-19T16:55:00.612990+0000","last_active":"2023-12-19T16:55:00.620124+0000","last_peered":"2023-12-19T16:55:00.620124+0000","last_clean":"2023-12-19T16:55:00.620124+0000","last_became_active":"2023-12-19T16:55:00.612420+0000","last_became_peered":"2023-12-19T16:55:00.612420+0000","last_unstale":"2023-12-19T16:55:00.620124+0000","last_undegraded":"2023-12-19T16:55:00.620124+0000","last_fullsized":"2023-12-19T16:55:00.620124+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:28.866354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.622914+0000","last_change":"2023-12-19T16:55:00.211256+0000","last_active":"2023-12-19T16:55:00.622914+0000","last_peered":"2023-12-19T16:55:00.622914+0000","last_clean":"2023-12-19T16:55:00.622914+0000","last_became_active":"2023-12-19T16:55:00.210933+0000","last_became_peered":"2023-12-19T16:55:00.210933+0000","last_unstale":"2023-12-19T16:55:00.622914+0000","last_undegraded":"2023-12-19T16:55:00.622914+0000","last_fullsized":"2023-12-19T16:55:00.622914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:45:25.749655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630661+0000","last_change":"2023-12-19T16:55:00.623214+0000","last_active":"2023-12-19T16:55:00.630661+0000","last_peered":"2023-12-19T16:55:00.630661+0000","last_clean":"2023-12-19T16:55:00.630661+0000","last_became_active":"2023-12-19T16:55:00.622966+0000","last_became_peered":"2023-12-19T16:55:00.622966+0000","last_unstale":"2023-12-19T16:55:00.630661+0000","last_undegraded":"2023-12-19T16:55:00.630661+0000","last_fullsized":"2023-12-19T16:55:00.630661+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:25.203498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.601259+0000","last_change":"2023-12-19T16:55:00.199254+0000","last_active":"2023-12-19T16:55:00.601259+0000","last_peered":"2023-12-19T16:55:00.601259+0000","last_clean":"2023-12-19T16:55:00.601259+0000","last_became_active":"2023-12-19T16:55:00.198742+0000","last_became_peered":"2023-12-19T16:55:00.198742+0000","last_unstale":"2023-12-19T16:55:00.601259+0000","last_undegraded":"2023-12-19T16:55:00.601259+0000","last_fullsized":"2023-12-19T16:55:00.601259+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:19.175671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.625886+0000","last_change":"2023-12-19T16:55:00.615792+0000","last_active":"2023-12-19T16:55:00.625886+0000","last_peered":"2023-12-19T16:55:00.625886+0000","last_clean":"2023-12-19T16:55:00.625886+0000","last_became_active":"2023-12-19T16:55:00.615414+0000","last_became_peered":"2023-12-19T16:55:00.615414+0000","last_unstale":"2023-12-19T16:55:00.625886+0000","last_undegraded":"2023-12-19T16:55:00.625886+0000","last_fullsized":"2023-12-19T16:55:00.625886+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:01:36.879382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.598010+0000","last_change":"2023-12-19T16:55:00.210853+0000","last_active":"2023-12-19T16:55:00.598010+0000","last_peered":"2023-12-19T16:55:00.598010+0000","last_clean":"2023-12-19T16:55:00.598010+0000","last_became_active":"2023-12-19T16:55:00.210537+0000","last_became_peered":"2023-12-19T16:55:00.210537+0000","last_unstale":"2023-12-19T16:55:00.598010+0000","last_undegraded":"2023-12-19T16:55:00.598010+0000","last_fullsized":"2023-12-19T16:55:00.598010+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:40:04.712609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626845+0000","last_change":"2023-12-19T16:55:00.620556+0000","last_active":"2023-12-19T16:55:00.626845+0000","last_peered":"2023-12-19T16:55:00.626845+0000","last_clean":"2023-12-19T16:55:00.626845+0000","last_became_active":"2023-12-19T16:55:00.620205+0000","last_became_peered":"2023-12-19T16:55:00.620205+0000","last_unstale":"2023-12-19T16:55:00.626845+0000","last_undegraded":"2023-12-19T16:55:00.626845+0000","last_fullsized":"2023-12-19T16:55:00.626845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:54:42.214844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"60'8","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630779+0000","last_change":"2023-12-19T16:55:00.209380+0000","last_active":"2023-12-19T16:55:00.630779+0000","last_peered":"2023-12-19T16:55:00.630779+0000","last_clean":"2023-12-19T16:55:00.630779+0000","last_became_active":"2023-12-19T16:55:00.209113+0000","last_became_peered":"2023-12-19T16:55:00.209113+0000","last_unstale":"2023-12-19T16:55:00.630779+0000","last_undegraded":"2023-12-19T16:55:00.630779+0000","last_fullsized":"2023-12-19T16:55:00.630779+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:20:12.924446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.618033+0000","last_change":"2023-12-19T16:55:00.611455+0000","last_active":"2023-12-19T16:55:00.618033+0000","last_peered":"2023-12-19T16:55:00.618033+0000","last_clean":"2023-12-19T16:55:00.618033+0000","last_became_active":"2023-12-19T16:55:00.611203+0000","last_became_peered":"2023-12-19T16:55:00.611203+0000","last_unstale":"2023-12-19T16:55:00.618033+0000","last_undegraded":"2023-12-19T16:55:00.618033+0000","last_fullsized":"2023-12-19T16:55:00.618033+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:54:55.529301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628669+0000","last_change":"2023-12-19T16:55:00.201631+0000","last_active":"2023-12-19T16:55:00.628669+0000","last_peered":"2023-12-19T16:55:00.628669+0000","last_clean":"2023-12-19T16:55:00.628669+0000","last_became_active":"2023-12-19T16:55:00.197371+0000","last_became_peered":"2023-12-19T16:55:00.197371+0000","last_unstale":"2023-12-19T16:55:00.628669+0000","last_undegraded":"2023-12-19T16:55:00.628669+0000","last_fullsized":"2023-12-19T16:55:00.628669+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:36:16.360153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626955+0000","last_change":"2023-12-19T16:55:00.617789+0000","last_active":"2023-12-19T16:55:00.626955+0000","last_peered":"2023-12-19T16:55:00.626955+0000","last_clean":"2023-12-19T16:55:00.626955+0000","last_became_active":"2023-12-19T16:55:00.617525+0000","last_became_peered":"2023-12-19T16:55:00.617525+0000","last_unstale":"2023-12-19T16:55:00.626955+0000","last_undegraded":"2023-12-19T16:55:00.626955+0000","last_fullsized":"2023-12-19T16:55:00.626955+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:25:01.530638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.624694+0000","last_change":"2023-12-19T16:55:00.197851+0000","last_active":"2023-12-19T16:55:00.624694+0000","last_peered":"2023-12-19T16:55:00.624694+0000","last_clean":"2023-12-19T16:55:00.624694+0000","last_became_active":"2023-12-19T16:55:00.197534+0000","last_became_peered":"2023-12-19T16:55:00.197534+0000","last_unstale":"2023-12-19T16:55:00.624694+0000","last_undegraded":"2023-12-19T16:55:00.624694+0000","last_fullsized":"2023-12-19T16:55:00.624694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:41:03.128639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628968+0000","last_change":"2023-12-19T16:55:00.626098+0000","last_active":"2023-12-19T16:55:00.628968+0000","last_peered":"2023-12-19T16:55:00.628968+0000","last_clean":"2023-12-19T16:55:00.628968+0000","last_became_active":"2023-12-19T16:55:00.625829+0000","last_became_peered":"2023-12-19T16:55:00.625829+0000","last_unstale":"2023-12-19T16:55:00.628968+0000","last_undegraded":"2023-12-19T16:55:00.628968+0000","last_fullsized":"2023-12-19T16:55:00.628968+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:10.646267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.621798+0000","last_change":"2023-12-19T16:55:00.194182+0000","last_active":"2023-12-19T16:55:00.621798+0000","last_peered":"2023-12-19T16:55:00.621798+0000","last_clean":"2023-12-19T16:55:00.621798+0000","last_became_active":"2023-12-19T16:55:00.193893+0000","last_became_peered":"2023-12-19T16:55:00.193893+0000","last_unstale":"2023-12-19T16:55:00.621798+0000","last_undegraded":"2023-12-19T16:55:00.621798+0000","last_fullsized":"2023-12-19T16:55:00.621798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:07.257748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628132+0000","last_change":"2023-12-19T16:55:00.623640+0000","last_active":"2023-12-19T16:55:00.628132+0000","last_peered":"2023-12-19T16:55:00.628132+0000","last_clean":"2023-12-19T16:55:00.628132+0000","last_became_active":"2023-12-19T16:55:00.623299+0000","last_became_peered":"2023-12-19T16:55:00.623299+0000","last_unstale":"2023-12-19T16:55:00.628132+0000","last_undegraded":"2023-12-19T16:55:00.628132+0000","last_fullsized":"2023-12-19T16:55:00.628132+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:38.197379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.605103+0000","last_change":"2023-12-19T16:55:00.201895+0000","last_active":"2023-12-19T16:55:00.605103+0000","last_peered":"2023-12-19T16:55:00.605103+0000","last_clean":"2023-12-19T16:55:00.605103+0000","last_became_active":"2023-12-19T16:55:00.201614+0000","last_became_peered":"2023-12-19T16:55:00.201614+0000","last_unstale":"2023-12-19T16:55:00.605103+0000","last_undegraded":"2023-12-19T16:55:00.605103+0000","last_fullsized":"2023-12-19T16:55:00.605103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:43:24.212407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.625836+0000","last_change":"2023-12-19T16:55:00.615759+0000","last_active":"2023-12-19T16:55:00.625836+0000","last_peered":"2023-12-19T16:55:00.625836+0000","last_clean":"2023-12-19T16:55:00.625836+0000","last_became_active":"2023-12-19T16:55:00.615495+0000","last_became_peered":"2023-12-19T16:55:00.615495+0000","last_unstale":"2023-12-19T16:55:00.625836+0000","last_undegraded":"2023-12-19T16:55:00.625836+0000","last_fullsized":"2023-12-19T16:55:00.625836+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:55:35.867260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.622669+0000","last_change":"2023-12-19T16:55:00.203924+0000","last_active":"2023-12-19T16:55:00.622669+0000","last_peered":"2023-12-19T16:55:00.622669+0000","last_clean":"2023-12-19T16:55:00.622669+0000","last_became_active":"2023-12-19T16:55:00.203565+0000","last_became_peered":"2023-12-19T16:55:00.203565+0000","last_unstale":"2023-12-19T16:55:00.622669+0000","last_undegraded":"2023-12-19T16:55:00.622669+0000","last_fullsized":"2023-12-19T16:55:00.622669+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:27.714679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.621086+0000","last_change":"2023-12-19T16:55:00.607146+0000","last_active":"2023-12-19T16:55:00.621086+0000","last_peered":"2023-12-19T16:55:00.621086+0000","last_clean":"2023-12-19T16:55:00.621086+0000","last_became_active":"2023-12-19T16:55:00.606795+0000","last_became_peered":"2023-12-19T16:55:00.606795+0000","last_unstale":"2023-12-19T16:55:00.621086+0000","last_undegraded":"2023-12-19T16:55:00.621086+0000","last_fullsized":"2023-12-19T16:55:00.621086+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:40:05.720430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.596512+0000","last_change":"2023-12-19T16:55:00.201065+0000","last_active":"2023-12-19T16:55:00.596512+0000","last_peered":"2023-12-19T16:55:00.596512+0000","last_clean":"2023-12-19T16:55:00.596512+0000","last_became_active":"2023-12-19T16:55:00.199073+0000","last_became_peered":"2023-12-19T16:55:00.199073+0000","last_unstale":"2023-12-19T16:55:00.596512+0000","last_undegraded":"2023-12-19T16:55:00.596512+0000","last_fullsized":"2023-12-19T16:55:00.596512+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:38.569496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.620780+0000","last_change":"2023-12-19T16:55:00.617233+0000","last_active":"2023-12-19T16:55:00.620780+0000","last_peered":"2023-12-19T16:55:00.620780+0000","last_clean":"2023-12-19T16:55:00.620780+0000","last_became_active":"2023-12-19T16:55:00.616534+0000","last_became_peered":"2023-12-19T16:55:00.616534+0000","last_unstale":"2023-12-19T16:55:00.620780+0000","last_undegraded":"2023-12-19T16:55:00.620780+0000","last_fullsized":"2023-12-19T16:55:00.620780+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:10:03.674528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.592255+0000","last_change":"2023-12-19T16:55:00.206479+0000","last_active":"2023-12-19T16:55:00.592255+0000","last_peered":"2023-12-19T16:55:00.592255+0000","last_clean":"2023-12-19T16:55:00.592255+0000","last_became_active":"2023-12-19T16:55:00.206161+0000","last_became_peered":"2023-12-19T16:55:00.206161+0000","last_unstale":"2023-12-19T16:55:00.592255+0000","last_undegraded":"2023-12-19T16:55:00.592255+0000","last_fullsized":"2023-12-19T16:55:00.592255+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:01.143913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.631035+0000","last_change":"2023-12-19T16:55:00.627537+0000","last_active":"2023-12-19T16:55:00.631035+0000","last_peered":"2023-12-19T16:55:00.631035+0000","last_clean":"2023-12-19T16:55:00.631035+0000","last_became_active":"2023-12-19T16:55:00.627273+0000","last_became_peered":"2023-12-19T16:55:00.627273+0000","last_unstale":"2023-12-19T16:55:00.631035+0000","last_undegraded":"2023-12-19T16:55:00.631035+0000","last_fullsized":"2023-12-19T16:55:00.631035+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:26.235392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.601359+0000","last_change":"2023-12-19T16:55:00.198286+0000","last_active":"2023-12-19T16:55:00.601359+0000","last_peered":"2023-12-19T16:55:00.601359+0000","last_clean":"2023-12-19T16:55:00.601359+0000","last_became_active":"2023-12-19T16:55:00.198035+0000","last_became_peered":"2023-12-19T16:55:00.198035+0000","last_unstale":"2023-12-19T16:55:00.601359+0000","last_undegraded":"2023-12-19T16:55:00.601359+0000","last_fullsized":"2023-12-19T16:55:00.601359+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:11:20.624994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619855+0000","last_change":"2023-12-19T16:55:00.612694+0000","last_active":"2023-12-19T16:55:00.619855+0000","last_peered":"2023-12-19T16:55:00.619855+0000","last_clean":"2023-12-19T16:55:00.619855+0000","last_became_active":"2023-12-19T16:55:00.612386+0000","last_became_peered":"2023-12-19T16:55:00.612386+0000","last_unstale":"2023-12-19T16:55:00.619855+0000","last_undegraded":"2023-12-19T16:55:00.619855+0000","last_fullsized":"2023-12-19T16:55:00.619855+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:47.181094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.606072+0000","last_change":"2023-12-19T16:55:00.200798+0000","last_active":"2023-12-19T16:55:00.606072+0000","last_peered":"2023-12-19T16:55:00.606072+0000","last_clean":"2023-12-19T16:55:00.606072+0000","last_became_active":"2023-12-19T16:55:00.200496+0000","last_became_peered":"2023-12-19T16:55:00.200496+0000","last_unstale":"2023-12-19T16:55:00.606072+0000","last_undegraded":"2023-12-19T16:55:00.606072+0000","last_fullsized":"2023-12-19T16:55:00.606072+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:19.175671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.605763+0000","last_change":"2023-12-19T16:55:00.602695+0000","last_active":"2023-12-19T16:55:00.605763+0000","last_peered":"2023-12-19T16:55:00.605763+0000","last_clean":"2023-12-19T16:55:00.605763+0000","last_became_active":"2023-12-19T16:55:00.602401+0000","last_became_peered":"2023-12-19T16:55:00.602401+0000","last_unstale":"2023-12-19T16:55:00.605763+0000","last_undegraded":"2023-12-19T16:55:00.605763+0000","last_fullsized":"2023-12-19T16:55:00.605763+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:57:52.945189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.602668+0000","last_change":"2023-12-19T16:55:00.204037+0000","last_active":"2023-12-19T16:55:00.602668+0000","last_peered":"2023-12-19T16:55:00.602668+0000","last_clean":"2023-12-19T16:55:00.602668+0000","last_became_active":"2023-12-19T16:55:00.203733+0000","last_became_peered":"2023-12-19T16:55:00.203733+0000","last_unstale":"2023-12-19T16:55:00.602668+0000","last_undegraded":"2023-12-19T16:55:00.602668+0000","last_fullsized":"2023-12-19T16:55:00.602668+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:47:48.610110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626509+0000","last_change":"2023-12-19T16:55:00.617942+0000","last_active":"2023-12-19T16:55:00.626509+0000","last_peered":"2023-12-19T16:55:00.626509+0000","last_clean":"2023-12-19T16:55:00.626509+0000","last_became_active":"2023-12-19T16:55:00.617630+0000","last_became_peered":"2023-12-19T16:55:00.617630+0000","last_unstale":"2023-12-19T16:55:00.626509+0000","last_undegraded":"2023-12-19T16:55:00.626509+0000","last_fullsized":"2023-12-19T16:55:00.626509+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:35.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.603366+0000","last_change":"2023-12-19T16:55:00.206586+0000","last_active":"2023-12-19T16:55:00.603366+0000","last_peered":"2023-12-19T16:55:00.603366+0000","last_clean":"2023-12-19T16:55:00.603366+0000","last_became_active":"2023-12-19T16:55:00.205636+0000","last_became_peered":"2023-12-19T16:55:00.205636+0000","last_unstale":"2023-12-19T16:55:00.603366+0000","last_undegraded":"2023-12-19T16:55:00.603366+0000","last_fullsized":"2023-12-19T16:55:00.603366+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:41:11.221952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.621586+0000","last_change":"2023-12-19T16:55:00.605339+0000","last_active":"2023-12-19T16:55:00.621586+0000","last_peered":"2023-12-19T16:55:00.621586+0000","last_clean":"2023-12-19T16:55:00.621586+0000","last_became_active":"2023-12-19T16:55:00.605071+0000","last_became_peered":"2023-12-19T16:55:00.605071+0000","last_unstale":"2023-12-19T16:55:00.621586+0000","last_undegraded":"2023-12-19T16:55:00.621586+0000","last_fullsized":"2023-12-19T16:55:00.621586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:36.450183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.602260+0000","last_change":"2023-12-19T16:55:00.203382+0000","last_active":"2023-12-19T16:55:00.602260+0000","last_peered":"2023-12-19T16:55:00.602260+0000","last_clean":"2023-12-19T16:55:00.602260+0000","last_became_active":"2023-12-19T16:55:00.201558+0000","last_became_peered":"2023-12-19T16:55:00.201558+0000","last_unstale":"2023-12-19T16:55:00.602260+0000","last_undegraded":"2023-12-19T16:55:00.602260+0000","last_fullsized":"2023-12-19T16:55:00.602260+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:41:11.221952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.625988+0000","last_change":"2023-12-19T16:55:00.616185+0000","last_active":"2023-12-19T16:55:00.625988+0000","last_peered":"2023-12-19T16:55:00.625988+0000","last_clean":"2023-12-19T16:55:00.625988+0000","last_became_active":"2023-12-19T16:55:00.615819+0000","last_became_peered":"2023-12-19T16:55:00.615819+0000","last_unstale":"2023-12-19T16:55:00.625988+0000","last_undegraded":"2023-12-19T16:55:00.625988+0000","last_fullsized":"2023-12-19T16:55:00.625988+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:05.573479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.593875+0000","last_change":"2023-12-19T16:55:00.198555+0000","last_active":"2023-12-19T16:55:00.593875+0000","last_peered":"2023-12-19T16:55:00.593875+0000","last_clean":"2023-12-19T16:55:00.593875+0000","last_became_active":"2023-12-19T16:55:00.198243+0000","last_became_peered":"2023-12-19T16:55:00.198243+0000","last_unstale":"2023-12-19T16:55:00.593875+0000","last_undegraded":"2023-12-19T16:55:00.593875+0000","last_fullsized":"2023-12-19T16:55:00.593875+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:32:15.694790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.621467+0000","last_change":"2023-12-19T16:55:00.615219+0000","last_active":"2023-12-19T16:55:00.621467+0000","last_peered":"2023-12-19T16:55:00.621467+0000","last_clean":"2023-12-19T16:55:00.621467+0000","last_became_active":"2023-12-19T16:55:00.614524+0000","last_became_peered":"2023-12-19T16:55:00.614524+0000","last_unstale":"2023-12-19T16:55:00.621467+0000","last_undegraded":"2023-12-19T16:55:00.621467+0000","last_fullsized":"2023-12-19T16:55:00.621467+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:28.722500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.603374+0000","last_change":"2023-12-19T16:55:00.204127+0000","last_active":"2023-12-19T16:55:00.603374+0000","last_peered":"2023-12-19T16:55:00.603374+0000","last_clean":"2023-12-19T16:55:00.603374+0000","last_became_active":"2023-12-19T16:55:00.203834+0000","last_became_peered":"2023-12-19T16:55:00.203834+0000","last_unstale":"2023-12-19T16:55:00.603374+0000","last_undegraded":"2023-12-19T16:55:00.603374+0000","last_fullsized":"2023-12-19T16:55:00.603374+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:11:20.624994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.617688+0000","last_change":"2023-12-19T16:55:00.606922+0000","last_active":"2023-12-19T16:55:00.617688+0000","last_peered":"2023-12-19T16:55:00.617688+0000","last_clean":"2023-12-19T16:55:00.617688+0000","last_became_active":"2023-12-19T16:55:00.606648+0000","last_became_peered":"2023-12-19T16:55:00.606648+0000","last_unstale":"2023-12-19T16:55:00.617688+0000","last_undegraded":"2023-12-19T16:55:00.617688+0000","last_fullsized":"2023-12-19T16:55:00.617688+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:49:32.328163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.590828+0000","last_change":"2023-12-19T16:55:00.208978+0000","last_active":"2023-12-19T16:55:00.590828+0000","last_peered":"2023-12-19T16:55:00.590828+0000","last_clean":"2023-12-19T16:55:00.590828+0000","last_became_active":"2023-12-19T16:55:00.208361+0000","last_became_peered":"2023-12-19T16:55:00.208361+0000","last_unstale":"2023-12-19T16:55:00.590828+0000","last_undegraded":"2023-12-19T16:55:00.590828+0000","last_fullsized":"2023-12-19T16:55:00.590828+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:01:32.454618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.610674+0000","last_change":"2023-12-19T16:55:00.608542+0000","last_active":"2023-12-19T16:55:00.610674+0000","last_peered":"2023-12-19T16:55:00.610674+0000","last_clean":"2023-12-19T16:55:00.610674+0000","last_became_active":"2023-12-19T16:55:00.608183+0000","last_became_peered":"2023-12-19T16:55:00.608183+0000","last_unstale":"2023-12-19T16:55:00.610674+0000","last_undegraded":"2023-12-19T16:55:00.610674+0000","last_fullsized":"2023-12-19T16:55:00.610674+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:38.688313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628018+0000","last_change":"2023-12-19T16:55:00.201570+0000","last_active":"2023-12-19T16:55:00.628018+0000","last_peered":"2023-12-19T16:55:00.628018+0000","last_clean":"2023-12-19T16:55:00.628018+0000","last_became_active":"2023-12-19T16:55:00.200298+0000","last_became_peered":"2023-12-19T16:55:00.200298+0000","last_unstale":"2023-12-19T16:55:00.628018+0000","last_undegraded":"2023-12-19T16:55:00.628018+0000","last_fullsized":"2023-12-19T16:55:00.628018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:01.816707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"49'139","reported_seq":280,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.186028+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:55:00.186028+0000","last_peered":"2023-12-19T16:55:00.186028+0000","last_clean":"2023-12-19T16:55:00.186028+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:55:00.186028+0000","last_undegraded":"2023-12-19T16:55:00.186028+0000","last_fullsized":"2023-12-19T16:55:00.186028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509595,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1302528,"data_stored":918398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35699999999999998,"5min":0.33900000000000002,"15min":0.30199999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":2.6389999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.72299999999999998,"15min":0.72399999999999998},"min":{"1min":0.52400000000000002,"5min":0.45000000000000001,"15min":0.32700000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":2.532},"last":0.84399999999999997}]},{"osd":1,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.435,"15min":0.36399999999999999},"max":{"1min":1.03,"5min":1.05,"15min":1.05},"last":0.75},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.746,"15min":0.71799999999999997},"min":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.36799999999999999},"max":{"1min":1.0069999999999999,"5min":1.046,"15min":1.093},"last":0.91000000000000003}]},{"osd":2,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.438},"max":{"1min":0.94699999999999995,"5min":1.006,"15min":1.006},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.746,"15min":0.72899999999999998},"min":{"1min":0.57899999999999996,"5min":0.47299999999999998,"15min":0.36599999999999999},"max":{"1min":1.004,"5min":1.0429999999999999,"15min":1.157},"last":0.80500000000000005}]},{"osd":3,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.70899999999999996},"min":{"1min":0.49299999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.93400000000000005,"5min":0.996,"15min":0.996},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76300000000000001,"15min":0.75},"min":{"1min":0.63300000000000001,"5min":0.54500000000000004,"15min":0.436},"max":{"1min":0.98799999999999999,"5min":1.0740000000000001,"15min":1.117},"last":0.65400000000000003}]},{"osd":4,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.28499999999999998},"max":{"1min":0.78900000000000003,"5min":0.89800000000000002,"15min":0.93600000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.36599999999999999},"max":{"1min":0.85399999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.72099999999999997}]},{"osd":5,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35699999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.96599999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.441,"15min":0.438},"max":{"1min":0.97099999999999997,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.76800000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.39800000000000002},"max":{"1min":0.98599999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.497,"5min":0.433,"15min":0.40899999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.872}]}]},{"osd":6,"up_from":15,"seq":64424509595,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":712704,"data_stored":328612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.39000000000000001,"5min":0.38800000000000001,"15min":0.309},"max":{"1min":0.91400000000000003,"5min":0.999,"15min":1.8600000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71899999999999997,"15min":0.68999999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.34999999999999998},"max":{"1min":0.93999999999999995,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.55000000000000004}]},{"osd":1,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71399999999999997,"15min":0.76200000000000001},"min":{"1min":0.46899999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79300000000000004,"5min":0.99399999999999999,"15min":4.3799999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.75,"5min":0.748,"15min":0.75},"min":{"1min":0.51500000000000001,"5min":0.46100000000000002,"15min":0.39500000000000002},"max":{"1min":0.89400000000000002,"5min":1.0189999999999999,"15min":1.583},"last":0.625}]},{"osd":2,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76600000000000001,"15min":0.747},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.35199999999999998},"max":{"1min":0.95599999999999996,"5min":1.087,"15min":1.3180000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.54800000000000004,"5min":0.498,"15min":0.44400000000000001},"max":{"1min":0.95299999999999996,"5min":1.0549999999999999,"15min":1.1879999999999999},"last":0.5}]},{"osd":3,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76700000000000002,"15min":0.85899999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.51900000000000002},"max":{"1min":0.92700000000000005,"5min":1.1020000000000001,"15min":4.673},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72399999999999998,"15min":0.74099999999999999},"min":{"1min":0.42699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":1.0740000000000001,"15min":1.962},"last":0.59599999999999997}]},{"osd":4,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.96999999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.40999999999999998,"5min":0.39800000000000002,"15min":0.35699999999999998},"max":{"1min":0.90800000000000003,"5min":0.99299999999999999,"15min":1.006},"last":0.443}]},{"osd":5,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":0.61499999999999999},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.33300000000000002},"max":{"1min":0.79000000000000004,"5min":0.91300000000000003,"15min":1.2050000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.46200000000000002,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":0.89500000000000002,"5min":1.046,"15min":1.046},"last":0.71799999999999997}]},{"osd":7,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.51400000000000001,"5min":0.48599999999999999,"15min":0.438},"max":{"1min":0.97999999999999998,"5min":1.0249999999999999,"15min":1.0620000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.371},"max":{"1min":0.92500000000000004,"5min":1.032,"15min":1.1220000000000001},"last":0.66600000000000004}]}]},{"osd":5,"up_from":15,"seq":64424509595,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":708608,"data_stored":328010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85099999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.72499999999999998,"15min":0.71599999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84699999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59399999999999997}]},{"osd":1,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.872,"5min":0.94899999999999995,"15min":0.97399999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.90800000000000003,"5min":0.91700000000000004,"15min":0.98799999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.1120000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.46400000000000002},"max":{"1min":0.96499999999999997,"5min":0.98999999999999999,"15min":1.087},"last":0.68600000000000005}]},{"osd":3,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":0.89800000000000002,"5min":0.96299999999999997,"15min":1.0129999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.65200000000000002}]},{"osd":4,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55500000000000005,"15min":0.57399999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82199999999999995,"5min":0.85599999999999998,"15min":1.165},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.60599999999999998},"min":{"1min":0.44600000000000001,"5min":0.36099999999999999,"15min":0.32400000000000001},"max":{"1min":0.77200000000000002,"5min":0.97199999999999998,"15min":1.0649999999999999},"last":0.51900000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.34699999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.153},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.433,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78700000000000003,"5min":0.90300000000000002,"15min":0.98999999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.50600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.78700000000000003,"5min":0.88300000000000001,"15min":1.133},"last":0.752},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66900000000000004,"15min":0.67800000000000005},"min":{"1min":0.48799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.76500000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509595,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53892,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317948,"statfs":{"total":96636764160,"available":96581578752,"internally_reserved":0,"allocated":716800,"data_stored":329504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.35699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.879,"5min":0.997,"15min":1.093},"last":0.755},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93500000000000005,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.61099999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71299999999999997,"15min":0.72199999999999998},"min":{"1min":0.53100000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88800000000000001,"5min":1.0209999999999999,"15min":1.8700000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.38500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86299999999999999,"5min":0.98999999999999999,"15min":1.05},"last":0.78700000000000003}]},{"osd":2,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.53900000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.755},"min":{"1min":0.53800000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91600000000000004,"5min":1.0549999999999999,"15min":1.075},"last":0.71899999999999997}]},{"osd":3,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.52700000000000002,"5min":0.497,"15min":0.46200000000000002},"max":{"1min":0.88400000000000001,"5min":0.90500000000000003,"15min":1.0249999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.75800000000000001,"15min":0.77100000000000002},"min":{"1min":0.56799999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96599999999999997,"5min":0.99199999999999999,"15min":1.3129999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.997,"15min":0.997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.42699999999999999,"5min":0.39400000000000002,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.89300000000000002,"15min":0.92800000000000005},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.63300000000000001},"min":{"1min":0.44600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79600000000000004,"5min":0.97199999999999998,"15min":1.0820000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66100000000000003,"15min":0.67600000000000005},"min":{"1min":0.432,"5min":0.41699999999999998,"15min":0.376},"max":{"1min":0.78100000000000003,"5min":0.95199999999999996,"15min":1.0189999999999999},"last":0.82799999999999996}]},{"osd":7,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.42199999999999999},"max":{"1min":0.78400000000000003,"5min":0.93200000000000005,"15min":1.0429999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.68799999999999994,"15min":0.71899999999999997},"min":{"1min":0.503,"5min":0.46700000000000003,"15min":0.44900000000000001},"max":{"1min":0.79300000000000004,"5min":1.0680000000000001,"15min":1.107},"last":0.77200000000000002}]}]},{"osd":3,"up_from":15,"seq":64424509595,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35844,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335996,"statfs":{"total":96636764160,"available":96600059904,"internally_reserved":0,"allocated":716800,"data_stored":328578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.443,"15min":0.45800000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49299999999999999,"15min":0.51500000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.738},"last":0.52900000000000003}]},{"osd":1,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45300000000000001,"15min":0.47699999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69499999999999995,"5min":0.77800000000000002,"15min":1.141},"last":0.432},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.56299999999999994},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.952,"5min":1.952,"15min":2.4910000000000001},"last":0.54500000000000004}]},{"osd":2,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54800000000000004,"15min":0.56699999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64300000000000002,"5min":0.84099999999999997,"15min":1.821},"last":0.372},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.57299999999999995,"15min":0.64000000000000001},"min":{"1min":0.372,"5min":0.36799999999999999,"15min":0.35399999999999998},"max":{"1min":2.855,"5min":2.855,"15min":5.1219999999999999},"last":0.46600000000000003}]},{"osd":4,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.79500000000000004,"15min":0.80600000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.234,"5min":4.234,"15min":4.234},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.76600000000000001,"15min":0.80700000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96799999999999997,"5min":1.093,"15min":2.2690000000000001},"last":0.79900000000000004}]},{"osd":5,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.77800000000000002,"15min":0.80400000000000005},"min":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.88300000000000001,"5min":1.0640000000000001,"15min":2.5350000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.9079999999999999},"last":0.76600000000000001}]},{"osd":6,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.599,"5min":1.599,"15min":1.956},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.012,"5min":1.129,"15min":1.974},"last":0.89100000000000001}]},{"osd":7,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82399999999999995,"15min":0.85599999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.51900000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.9279999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.85399999999999998,"15min":0.91700000000000004},"min":{"1min":0.624,"5min":0.60599999999999998,"15min":0.48799999999999999},"max":{"1min":1.623,"5min":1.623,"15min":5.0759999999999996},"last":0.77300000000000002}]}]},{"osd":2,"up_from":15,"seq":64424509594,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39944,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331896,"statfs":{"total":96636764160,"available":96595861504,"internally_reserved":0,"allocated":720896,"data_stored":329526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.49199999999999999,"15min":0.48899999999999999},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.28199999999999997},"max":{"1min":5.4870000000000001,"5min":5.4870000000000001,"15min":5.4870000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.012,"5min":0.60599999999999998,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.32500000000000001,"15min":0.29599999999999999},"max":{"1min":10.161,"5min":10.161,"15min":10.161},"last":0.32400000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.624,"15min":0.66700000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.39900000000000002},"max":{"1min":2.4740000000000002,"5min":2.4740000000000002,"15min":5.7590000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.51900000000000002,"15min":0.58499999999999996},"min":{"1min":0.35199999999999998,"5min":0.33000000000000002,"15min":0.29299999999999998},"max":{"1min":2.911,"5min":2.911,"15min":5.8070000000000004},"last":0.51400000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.60799999999999998},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":2.7599999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58099999999999996,"15min":0.63600000000000001},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.35099999999999998},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":5.7190000000000003},"last":0.42499999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.77100000000000002,"15min":0.83099999999999996},"min":{"1min":0.59399999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.76600000000000001,"5min":1.3149999999999999,"15min":3.8940000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.54500000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":5.5430000000000001,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.83199999999999996,"15min":0.81799999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.601,"5min":5.601,"15min":5.601},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.81399999999999995,"15min":0.86299999999999999},"min":{"1min":0.55700000000000005,"5min":0.55100000000000005,"15min":0.54900000000000004},"max":{"1min":1.27,"5min":1.3899999999999999,"15min":2.9260000000000002},"last":0.80400000000000005}]},{"osd":6,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.81899999999999995,"15min":0.84399999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.52400000000000002},"max":{"1min":1.236,"5min":1.2569999999999999,"15min":2.165},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77900000000000003,"15min":0.82799999999999996},"min":{"1min":0.55800000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.90800000000000003,"5min":1.1739999999999999,"15min":2.121},"last":0.68600000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.84399999999999997,"15min":0.84199999999999997},"min":{"1min":0.63,"5min":0.55500000000000005,"15min":0.52300000000000002},"max":{"1min":5.673,"5min":5.673,"15min":5.673},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.83699999999999997,"15min":0.86599999999999999},"min":{"1min":0.60599999999999998,"5min":0.58299999999999996,"15min":0.57799999999999996},"max":{"1min":1.3009999999999999,"5min":1.429,"15min":2.0529999999999999},"last":0.73499999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509595,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49788,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94322052,"statfs":{"total":96636764160,"available":96585781248,"internally_reserved":0,"allocated":708608,"data_stored":328010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.44700000000000001,"15min":0.46899999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55000000000000004,"5min":0.70399999999999996,"15min":2.3050000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.84199999999999997,"5min":0.85999999999999999,"15min":0.89800000000000002},"last":0.70499999999999996}]},{"osd":2,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.50600000000000001,"15min":0.53300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67200000000000004,"5min":0.80100000000000005,"15min":1.992},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56399999999999995,"15min":0.55500000000000005},"min":{"1min":0.35999999999999999,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":0.72899999999999998,"5min":0.82899999999999996,"15min":0.97299999999999998},"last":0.38400000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61199999999999999,"15min":0.65300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.34799999999999998},"max":{"1min":0.79700000000000004,"5min":0.91500000000000004,"15min":4.133},"last":0.433},{"interface":"front","average":{"1min":0.496,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.34599999999999997,"15min":0.33700000000000002},"max":{"1min":0.70499999999999996,"5min":0.79000000000000004,"15min":0.91900000000000004},"last":0.44900000000000001}]},{"osd":4,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.76000000000000001,"15min":0.78900000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.202,"5min":1.202,"15min":2.1949999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.80600000000000005,"15min":0.80500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.42399999999999999},"max":{"1min":1.181,"5min":1.23,"15min":2.048},"last":0.56499999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78600000000000003,"15min":0.80100000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.288,"5min":1.288,"15min":2.1269999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81399999999999995,"15min":0.85199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":4.0460000000000003},"last":0.57799999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.80200000000000005,"15min":0.82399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.41999999999999998},"max":{"1min":1.0720000000000001,"5min":1.129,"15min":2.1499999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.81399999999999995,"15min":0.83099999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.391,"5min":1.391,"15min":2.0880000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.84199999999999997,"15min":0.89000000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":4.093},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.84299999999999997,"15min":0.86399999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.53800000000000003},"max":{"1min":1.147,"5min":1.246,"15min":2.2349999999999999},"last":0.59299999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542300,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1302528,"data_stored":918342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.60599999999999998,"5min":0.70999999999999996,"15min":0.72499999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.5,"15min":0.47699999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26300000000000001},"max":{"1min":0.70299999999999996,"5min":0.751,"15min":0.85499999999999998},"last":0.41699999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.29299999999999998},"max":{"1min":0.67000000000000004,"5min":0.77800000000000002,"15min":0.89700000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54500000000000004,"15min":0.52400000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.45800000000000002}]},{"osd":3,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52800000000000002,"15min":0.50800000000000001},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.60299999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40899999999999997},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.80600000000000005,"15min":0.79200000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.22,"5min":1.391,"15min":1.512},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81000000000000005,"15min":0.78600000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0529999999999999,"5min":1.177,"15min":1.52},"last":0.70099999999999996}]},{"osd":5,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80800000000000005,"15min":0.80600000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.41499999999999998},"max":{"1min":1.145,"5min":1.2390000000000001,"15min":1.748},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81200000000000006,"15min":0.78800000000000003},"min":{"1min":0.56999999999999995,"5min":0.52300000000000002,"15min":0.51600000000000001},"max":{"1min":1.119,"5min":1.163,"15min":1.649},"last":0.61199999999999999}]},{"osd":6,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.81799999999999995,"15min":0.78300000000000003},"min":{"1min":0.55500000000000005,"5min":0.54400000000000004,"15min":0.46999999999999997},"max":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82499999999999996,"15min":0.80800000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.44700000000000001},"max":{"1min":1.0289999999999999,"5min":1.1200000000000001,"15min":1.2070000000000001},"last":0.63200000000000001}]},{"osd":7,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.84599999999999997,"15min":0.81200000000000006},"min":{"1min":0.505,"5min":0.505,"15min":0.48999999999999999},"max":{"1min":1.1739999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82599999999999996,"15min":0.81200000000000006},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1910000000000001,"5min":1.212,"15min":1.661},"last":0.66000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:55:04.715 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:55:04.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.895+0000 7f0d8db39640 1 Processor -- start 2023-12-19T16:55:04.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.895+0000 7f0d8db39640 1 -- start start 2023-12-19T16:55:04.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.895+0000 7f0d8db39640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0d88155030 0x7f0d8815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.895+0000 7f0d8db39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d880ce740 0x7f0d880cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.895+0000 7f0d8db39640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8806c970 con 0x7f0d8815f370 2023-12-19T16:55:04.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.895+0000 7f0d8db39640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8806c790 con 0x7f0d88155030 2023-12-19T16:55:04.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.895+0000 7f0d8db39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880cf0f0 con 0x7f0d880ce740 2023-12-19T16:55:04.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.895+0000 7f0d86ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0d8815f370 0x7f0d8806c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47718/0 (socket says 172.21.15.26:47718) 2023-12-19T16:55:04.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.899+0000 7f0d86ffd640 1 -- 172.21.15.26:0/1634672121 learned_addr learned my addr 172.21.15.26:0/1634672121 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:55:04.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.899+0000 7f0d877fe640 1 --2- 172.21.15.26:0/1634672121 >> v2:172.21.15.160:3300/0 conn(0x7f0d88155030 0x7f0d8815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.899+0000 7f0d87fff640 1 --2- 172.21.15.26:0/1634672121 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d880ce740 0x7f0d880cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.899+0000 7f0d867fc640 1 -- 172.21.15.26:0/1634672121 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2289764214 0 0) 0x7f0d8806c970 con 0x7f0d8815f370 2023-12-19T16:55:04.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.899+0000 7f0d867fc640 1 -- 172.21.15.26:0/1634672121 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d70003530 con 0x7f0d8815f370 2023-12-19T16:55:04.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.899+0000 7f0d87fff640 1 -- 172.21.15.26:0/1634672121 >> v2:172.21.15.160:3300/0 conn(0x7f0d88155030 msgr2=0x7f0d8815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:04.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.899+0000 7f0d87fff640 1 --2- 172.21.15.26:0/1634672121 >> v2:172.21.15.160:3300/0 conn(0x7f0d88155030 0x7f0d8815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.899+0000 7f0d87fff640 1 -- 172.21.15.26:0/1634672121 >> v1:172.21.15.26:6789/0 conn(0x7f0d8815f370 legacy=0x7f0d8806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.899+0000 7f0d87fff640 1 -- 172.21.15.26:0/1634672121 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8815b1f0 con 0x7f0d880ce740 2023-12-19T16:55:04.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.899+0000 7f0d877fe640 1 --2- 172.21.15.26:0/1634672121 >> v2:172.21.15.160:3300/0 conn(0x7f0d88155030 0x7f0d8815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:55:04.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.899+0000 7f0d87fff640 1 --2- 172.21.15.26:0/1634672121 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d880ce740 0x7f0d880cebb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009d90 tx=0x7f0d8001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=488f12622fc4c75a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:04.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.899+0000 7f0d867fc640 1 -- 172.21.15.26:0/1634672121 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d800a59e0 con 0x7f0d880ce740 2023-12-19T16:55:04.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.899+0000 7f0d867fc640 1 -- 172.21.15.26:0/1634672121 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d800a5b40 con 0x7f0d880ce740 2023-12-19T16:55:04.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d867fc640 1 -- 172.21.15.26:0/1634672121 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d800ad440 con 0x7f0d880ce740 2023-12-19T16:55:04.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 -- 172.21.15.26:0/1634672121 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d880ce740 msgr2=0x7f0d880cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 --2- 172.21.15.26:0/1634672121 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d880ce740 0x7f0d880cebb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009d90 tx=0x7f0d8001c920 comp rx=0 tx=0).stop 2023-12-19T16:55:04.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 -- 172.21.15.26:0/1634672121 shutdown_connections 2023-12-19T16:55:04.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 --2- 172.21.15.26:0/1634672121 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d880ce740 0x7f0d880cebb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 --2- 172.21.15.26:0/1634672121 >> v2:172.21.15.160:3300/0 conn(0x7f0d88155030 0x7f0d8815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 -- 172.21.15.26:0/1634672121 >> 172.21.15.26:0/1634672121 conn(0x7f0d880658e0 msgr2=0x7f0d88065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:04.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 -- 172.21.15.26:0/1634672121 shutdown_connections 2023-12-19T16:55:04.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 -- 172.21.15.26:0/1634672121 wait complete. 2023-12-19T16:55:04.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 Processor -- start 2023-12-19T16:55:04.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 -- start start 2023-12-19T16:55:04.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d880ce740 0x7f0d88150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d88155030 0x7f0d8815c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d8815f370 0x7f0d8814e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8814f7f0 con 0x7f0d88155030 2023-12-19T16:55:04.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88152e60 con 0x7f0d8815f370 2023-12-19T16:55:04.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d8db39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8814f230 con 0x7f0d880ce740 2023-12-19T16:55:04.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d86ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d88155030 0x7f0d8815c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d86ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d88155030 0x7f0d8815c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49830/0 (socket says 172.21.15.26:49830) 2023-12-19T16:55:04.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.903+0000 7f0d86ffd640 1 -- 172.21.15.26:0/2968264974 learned_addr learned my addr 172.21.15.26:0/2968264974 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d86ffd640 1 -- 172.21.15.26:0/2968264974 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d880ce740 msgr2=0x7f0d88150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d86ffd640 1 --2- 172.21.15.26:0/2968264974 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d880ce740 0x7f0d88150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d86ffd640 1 -- 172.21.15.26:0/2968264974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d8815f370 msgr2=0x7f0d8814e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d86ffd640 1 --2- 172.21.15.26:0/2968264974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d8815f370 0x7f0d8814e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d86ffd640 1 -- 172.21.15.26:0/2968264974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d78009420 con 0x7f0d88155030 2023-12-19T16:55:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d86ffd640 1 --2- 172.21.15.26:0/2968264974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d88155030 0x7f0d8815c6c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00c420 tx=0x7f0d7c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d6ffff640 1 -- 172.21.15.26:0/2968264974 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c005cf0 con 0x7f0d88155030 2023-12-19T16:55:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d6ffff640 1 -- 172.21.15.26:0/2968264974 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c005e50 con 0x7f0d88155030 2023-12-19T16:55:04.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d8db39640 1 -- 172.21.15.26:0/2968264974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d80009a70 con 0x7f0d88155030 2023-12-19T16:55:04.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d6ffff640 1 -- 172.21.15.26:0/2968264974 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0072d0 con 0x7f0d88155030 2023-12-19T16:55:04.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d8db39640 1 -- 172.21.15.26:0/2968264974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d88152af0 con 0x7f0d88155030 2023-12-19T16:55:04.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d8db39640 1 -- 172.21.15.26:0/2968264974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d88152950 con 0x7f0d88155030 2023-12-19T16:55:04.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d6ffff640 1 -- 172.21.15.26:0/2968264974 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0d7c007430 con 0x7f0d88155030 2023-12-19T16:55:04.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d6ffff640 1 --2- 172.21.15.26:0/2968264974 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0d74065b80 0x7f0d74068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:04.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d6ffff640 1 -- 172.21.15.26:0/2968264974 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f0d7c082bd0 con 0x7f0d88155030 2023-12-19T16:55:04.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.907+0000 7f0d877fe640 1 --2- 172.21.15.26:0/2968264974 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0d74065b80 0x7f0d74068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:04.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.911+0000 7f0d877fe640 1 --2- 172.21.15.26:0/2968264974 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0d74065b80 0x7f0d74068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d78006e90 tx=0x7f0d78006de0 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:04.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:04.911+0000 7f0d6ffff640 1 -- 172.21.15.26:0/2968264974 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0d88152950 con 0x7f0d88155030 2023-12-19T16:55:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.067+0000 7f0d8db39640 1 -- 172.21.15.26:0/2968264974 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d88152fd0 con 0x7f0d74065b80 2023-12-19T16:55:05.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.075+0000 7f0d6ffff640 1 -- 172.21.15.26:0/2968264974 <== mgr.5587 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288826 (secure 0 0 0) 0x7f0d88152fd0 con 0x7f0d74065b80 2023-12-19T16:55:05.083 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:55:05.084 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:55:05.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.079+0000 7f0d8db39640 1 -- 172.21.15.26:0/2968264974 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0d74065b80 msgr2=0x7f0d74068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.079+0000 7f0d8db39640 1 --2- 172.21.15.26:0/2968264974 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0d74065b80 0x7f0d74068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0d78006e90 tx=0x7f0d78006de0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.079+0000 7f0d8db39640 1 -- 172.21.15.26:0/2968264974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d88155030 msgr2=0x7f0d8815c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.079+0000 7f0d8db39640 1 --2- 172.21.15.26:0/2968264974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d88155030 0x7f0d8815c6c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00c420 tx=0x7f0d7c00c8e0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.079+0000 7f0d8db39640 1 -- 172.21.15.26:0/2968264974 shutdown_connections 2023-12-19T16:55:05.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.079+0000 7f0d8db39640 1 --2- 172.21.15.26:0/2968264974 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d8815f370 0x7f0d8814e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.079+0000 7f0d8db39640 1 --2- 172.21.15.26:0/2968264974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d88155030 0x7f0d8815c6c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.079+0000 7f0d8db39640 1 --2- 172.21.15.26:0/2968264974 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0d74065b80 0x7f0d74068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.079+0000 7f0d8db39640 1 --2- 172.21.15.26:0/2968264974 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d880ce740 0x7f0d88150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.079+0000 7f0d8db39640 1 -- 172.21.15.26:0/2968264974 >> 172.21.15.26:0/2968264974 conn(0x7f0d880658e0 msgr2=0x7f0d8815dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:05.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.079+0000 7f0d8db39640 1 -- 172.21.15.26:0/2968264974 shutdown_connections 2023-12-19T16:55:05.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.079+0000 7f0d8db39640 1 -- 172.21.15.26:0/2968264974 wait complete. 2023-12-19T16:55:05.104 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-19T16:55:03.592982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307592,"kb_used_data":6728,"kb_used_omap":0,"kb_used_meta":300800,"kb_avail":754667128,"statfs":{"total":773094113280,"available":772779139072,"internally_reserved":0,"allocated":6889472,"data_stored":3808980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308019200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.420217"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626736+0000","last_change":"2023-12-19T16:55:00.620221+0000","last_active":"2023-12-19T16:55:00.626736+0000","last_peered":"2023-12-19T16:55:00.626736+0000","last_clean":"2023-12-19T16:55:00.626736+0000","last_became_active":"2023-12-19T16:55:00.618228+0000","last_became_peered":"2023-12-19T16:55:00.618228+0000","last_unstale":"2023-12-19T16:55:00.626736+0000","last_undegraded":"2023-12-19T16:55:00.626736+0000","last_fullsized":"2023-12-19T16:55:00.626736+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:37:20.239966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.618139+0000","last_change":"2023-12-19T16:55:00.609820+0000","last_active":"2023-12-19T16:55:00.618139+0000","last_peered":"2023-12-19T16:55:00.618139+0000","last_clean":"2023-12-19T16:55:00.618139+0000","last_became_active":"2023-12-19T16:55:00.609537+0000","last_became_peered":"2023-12-19T16:55:00.609537+0000","last_unstale":"2023-12-19T16:55:00.618139+0000","last_undegraded":"2023-12-19T16:55:00.618139+0000","last_fullsized":"2023-12-19T16:55:00.618139+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:35.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.617828+0000","last_change":"2023-12-19T16:55:00.612390+0000","last_active":"2023-12-19T16:55:00.617828+0000","last_peered":"2023-12-19T16:55:00.617828+0000","last_clean":"2023-12-19T16:55:00.617828+0000","last_became_active":"2023-12-19T16:55:00.612111+0000","last_became_peered":"2023-12-19T16:55:00.612111+0000","last_unstale":"2023-12-19T16:55:00.617828+0000","last_undegraded":"2023-12-19T16:55:00.617828+0000","last_fullsized":"2023-12-19T16:55:00.617828+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:16.382474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630698+0000","last_change":"2023-12-19T16:55:00.625117+0000","last_active":"2023-12-19T16:55:00.630698+0000","last_peered":"2023-12-19T16:55:00.630698+0000","last_clean":"2023-12-19T16:55:00.630698+0000","last_became_active":"2023-12-19T16:55:00.624770+0000","last_became_peered":"2023-12-19T16:55:00.624770+0000","last_unstale":"2023-12-19T16:55:00.630698+0000","last_undegraded":"2023-12-19T16:55:00.630698+0000","last_fullsized":"2023-12-19T16:55:00.630698+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:39.380311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.610069+0000","last_change":"2023-12-19T16:55:00.605142+0000","last_active":"2023-12-19T16:55:00.610069+0000","last_peered":"2023-12-19T16:55:00.610069+0000","last_clean":"2023-12-19T16:55:00.610069+0000","last_became_active":"2023-12-19T16:55:00.604724+0000","last_became_peered":"2023-12-19T16:55:00.604724+0000","last_unstale":"2023-12-19T16:55:00.610069+0000","last_undegraded":"2023-12-19T16:55:00.610069+0000","last_fullsized":"2023-12-19T16:55:00.610069+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:09:51.151857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.622450+0000","last_change":"2023-12-19T16:55:00.609294+0000","last_active":"2023-12-19T16:55:00.622450+0000","last_peered":"2023-12-19T16:55:00.622450+0000","last_clean":"2023-12-19T16:55:00.622450+0000","last_became_active":"2023-12-19T16:55:00.609015+0000","last_became_peered":"2023-12-19T16:55:00.609015+0000","last_unstale":"2023-12-19T16:55:00.622450+0000","last_undegraded":"2023-12-19T16:55:00.622450+0000","last_fullsized":"2023-12-19T16:55:00.622450+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:03:13.666792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.627669+0000","last_change":"2023-12-19T16:55:00.626722+0000","last_active":"2023-12-19T16:55:00.627669+0000","last_peered":"2023-12-19T16:55:00.627669+0000","last_clean":"2023-12-19T16:55:00.627669+0000","last_became_active":"2023-12-19T16:55:00.626453+0000","last_became_peered":"2023-12-19T16:55:00.626453+0000","last_unstale":"2023-12-19T16:55:00.627669+0000","last_undegraded":"2023-12-19T16:55:00.627669+0000","last_fullsized":"2023-12-19T16:55:00.627669+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:08.692322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630446+0000","last_change":"2023-12-19T16:55:00.621881+0000","last_active":"2023-12-19T16:55:00.630446+0000","last_peered":"2023-12-19T16:55:00.630446+0000","last_clean":"2023-12-19T16:55:00.630446+0000","last_became_active":"2023-12-19T16:55:00.615081+0000","last_became_peered":"2023-12-19T16:55:00.615081+0000","last_unstale":"2023-12-19T16:55:00.630446+0000","last_undegraded":"2023-12-19T16:55:00.630446+0000","last_fullsized":"2023-12-19T16:55:00.630446+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:58:36.985090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630401+0000","last_change":"2023-12-19T16:55:00.622678+0000","last_active":"2023-12-19T16:55:00.630401+0000","last_peered":"2023-12-19T16:55:00.630401+0000","last_clean":"2023-12-19T16:55:00.630401+0000","last_became_active":"2023-12-19T16:55:00.622338+0000","last_became_peered":"2023-12-19T16:55:00.622338+0000","last_unstale":"2023-12-19T16:55:00.630401+0000","last_undegraded":"2023-12-19T16:55:00.630401+0000","last_fullsized":"2023-12-19T16:55:00.630401+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:50:51.444657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630327+0000","last_change":"2023-12-19T16:55:00.622394+0000","last_active":"2023-12-19T16:55:00.630327+0000","last_peered":"2023-12-19T16:55:00.630327+0000","last_clean":"2023-12-19T16:55:00.630327+0000","last_became_active":"2023-12-19T16:55:00.621864+0000","last_became_peered":"2023-12-19T16:55:00.621864+0000","last_unstale":"2023-12-19T16:55:00.630327+0000","last_undegraded":"2023-12-19T16:55:00.630327+0000","last_fullsized":"2023-12-19T16:55:00.630327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:32:07.178020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630679+0000","last_change":"2023-12-19T16:55:00.623233+0000","last_active":"2023-12-19T16:55:00.630679+0000","last_peered":"2023-12-19T16:55:00.630679+0000","last_clean":"2023-12-19T16:55:00.630679+0000","last_became_active":"2023-12-19T16:55:00.622954+0000","last_became_peered":"2023-12-19T16:55:00.622954+0000","last_unstale":"2023-12-19T16:55:00.630679+0000","last_undegraded":"2023-12-19T16:55:00.630679+0000","last_fullsized":"2023-12-19T16:55:00.630679+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:00:35.005593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.625316+0000","last_change":"2023-12-19T16:55:00.616111+0000","last_active":"2023-12-19T16:55:00.625316+0000","last_peered":"2023-12-19T16:55:00.625316+0000","last_clean":"2023-12-19T16:55:00.625316+0000","last_became_active":"2023-12-19T16:55:00.614258+0000","last_became_peered":"2023-12-19T16:55:00.614258+0000","last_unstale":"2023-12-19T16:55:00.625316+0000","last_undegraded":"2023-12-19T16:55:00.625316+0000","last_fullsized":"2023-12-19T16:55:00.625316+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:20.438452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.620644+0000","last_change":"2023-12-19T16:55:00.613795+0000","last_active":"2023-12-19T16:55:00.620644+0000","last_peered":"2023-12-19T16:55:00.620644+0000","last_clean":"2023-12-19T16:55:00.620644+0000","last_became_active":"2023-12-19T16:55:00.613524+0000","last_became_peered":"2023-12-19T16:55:00.613524+0000","last_unstale":"2023-12-19T16:55:00.620644+0000","last_undegraded":"2023-12-19T16:55:00.620644+0000","last_fullsized":"2023-12-19T16:55:00.620644+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:51.888168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626507+0000","last_change":"2023-12-19T16:55:00.614515+0000","last_active":"2023-12-19T16:55:00.626507+0000","last_peered":"2023-12-19T16:55:00.626507+0000","last_clean":"2023-12-19T16:55:00.626507+0000","last_became_active":"2023-12-19T16:55:00.614267+0000","last_became_peered":"2023-12-19T16:55:00.614267+0000","last_unstale":"2023-12-19T16:55:00.626507+0000","last_undegraded":"2023-12-19T16:55:00.626507+0000","last_fullsized":"2023-12-19T16:55:00.626507+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:56:22.346881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.625729+0000","last_change":"2023-12-19T16:55:00.614372+0000","last_active":"2023-12-19T16:55:00.625729+0000","last_peered":"2023-12-19T16:55:00.625729+0000","last_clean":"2023-12-19T16:55:00.625729+0000","last_became_active":"2023-12-19T16:55:00.614124+0000","last_became_peered":"2023-12-19T16:55:00.614124+0000","last_unstale":"2023-12-19T16:55:00.625729+0000","last_undegraded":"2023-12-19T16:55:00.625729+0000","last_fullsized":"2023-12-19T16:55:00.625729+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:54:08.915596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619546+0000","last_change":"2023-12-19T16:55:00.612879+0000","last_active":"2023-12-19T16:55:00.619546+0000","last_peered":"2023-12-19T16:55:00.619546+0000","last_clean":"2023-12-19T16:55:00.619546+0000","last_became_active":"2023-12-19T16:55:00.612488+0000","last_became_peered":"2023-12-19T16:55:00.612488+0000","last_unstale":"2023-12-19T16:55:00.619546+0000","last_undegraded":"2023-12-19T16:55:00.619546+0000","last_fullsized":"2023-12-19T16:55:00.619546+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:29:50.782409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.610468+0000","last_change":"2023-12-19T16:55:00.600944+0000","last_active":"2023-12-19T16:55:00.610468+0000","last_peered":"2023-12-19T16:55:00.610468+0000","last_clean":"2023-12-19T16:55:00.610468+0000","last_became_active":"2023-12-19T16:55:00.600645+0000","last_became_peered":"2023-12-19T16:55:00.600645+0000","last_unstale":"2023-12-19T16:55:00.610468+0000","last_undegraded":"2023-12-19T16:55:00.610468+0000","last_fullsized":"2023-12-19T16:55:00.610468+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:38.197379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.618066+0000","last_change":"2023-12-19T16:55:00.610429+0000","last_active":"2023-12-19T16:55:00.618066+0000","last_peered":"2023-12-19T16:55:00.618066+0000","last_clean":"2023-12-19T16:55:00.618066+0000","last_became_active":"2023-12-19T16:55:00.607100+0000","last_became_peered":"2023-12-19T16:55:00.607100+0000","last_unstale":"2023-12-19T16:55:00.618066+0000","last_undegraded":"2023-12-19T16:55:00.618066+0000","last_fullsized":"2023-12-19T16:55:00.618066+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:47.181094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628947+0000","last_change":"2023-12-19T16:55:00.625929+0000","last_active":"2023-12-19T16:55:00.628947+0000","last_peered":"2023-12-19T16:55:00.628947+0000","last_clean":"2023-12-19T16:55:00.628947+0000","last_became_active":"2023-12-19T16:55:00.625570+0000","last_became_peered":"2023-12-19T16:55:00.625570+0000","last_unstale":"2023-12-19T16:55:00.628947+0000","last_undegraded":"2023-12-19T16:55:00.628947+0000","last_fullsized":"2023-12-19T16:55:00.628947+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:31:45.628790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.611055+0000","last_change":"2023-12-19T16:55:00.608863+0000","last_active":"2023-12-19T16:55:00.611055+0000","last_peered":"2023-12-19T16:55:00.611055+0000","last_clean":"2023-12-19T16:55:00.611055+0000","last_became_active":"2023-12-19T16:55:00.608027+0000","last_became_peered":"2023-12-19T16:55:00.608027+0000","last_unstale":"2023-12-19T16:55:00.611055+0000","last_undegraded":"2023-12-19T16:55:00.611055+0000","last_fullsized":"2023-12-19T16:55:00.611055+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:54.572124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626029+0000","last_change":"2023-12-19T16:55:00.617614+0000","last_active":"2023-12-19T16:55:00.626029+0000","last_peered":"2023-12-19T16:55:00.626029+0000","last_clean":"2023-12-19T16:55:00.626029+0000","last_became_active":"2023-12-19T16:55:00.616766+0000","last_became_peered":"2023-12-19T16:55:00.616766+0000","last_unstale":"2023-12-19T16:55:00.626029+0000","last_undegraded":"2023-12-19T16:55:00.626029+0000","last_fullsized":"2023-12-19T16:55:00.626029+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:15.272341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.627110+0000","last_change":"2023-12-19T16:55:00.617539+0000","last_active":"2023-12-19T16:55:00.627110+0000","last_peered":"2023-12-19T16:55:00.627110+0000","last_clean":"2023-12-19T16:55:00.627110+0000","last_became_active":"2023-12-19T16:55:00.617090+0000","last_became_peered":"2023-12-19T16:55:00.617090+0000","last_unstale":"2023-12-19T16:55:00.627110+0000","last_undegraded":"2023-12-19T16:55:00.627110+0000","last_fullsized":"2023-12-19T16:55:00.627110+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:59:07.234068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628229+0000","last_change":"2023-12-19T16:55:00.624739+0000","last_active":"2023-12-19T16:55:00.628229+0000","last_peered":"2023-12-19T16:55:00.628229+0000","last_clean":"2023-12-19T16:55:00.628229+0000","last_became_active":"2023-12-19T16:55:00.624319+0000","last_became_peered":"2023-12-19T16:55:00.624319+0000","last_unstale":"2023-12-19T16:55:00.628229+0000","last_undegraded":"2023-12-19T16:55:00.628229+0000","last_fullsized":"2023-12-19T16:55:00.628229+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:49:32.328163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.620249+0000","last_change":"2023-12-19T16:55:00.615210+0000","last_active":"2023-12-19T16:55:00.620249+0000","last_peered":"2023-12-19T16:55:00.620249+0000","last_clean":"2023-12-19T16:55:00.620249+0000","last_became_active":"2023-12-19T16:55:00.614909+0000","last_became_peered":"2023-12-19T16:55:00.614909+0000","last_unstale":"2023-12-19T16:55:00.620249+0000","last_undegraded":"2023-12-19T16:55:00.620249+0000","last_fullsized":"2023-12-19T16:55:00.620249+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:17:29.306993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628146+0000","last_change":"2023-12-19T16:55:00.624275+0000","last_active":"2023-12-19T16:55:00.628146+0000","last_peered":"2023-12-19T16:55:00.628146+0000","last_clean":"2023-12-19T16:55:00.628146+0000","last_became_active":"2023-12-19T16:55:00.623919+0000","last_became_peered":"2023-12-19T16:55:00.623919+0000","last_unstale":"2023-12-19T16:55:00.628146+0000","last_undegraded":"2023-12-19T16:55:00.628146+0000","last_fullsized":"2023-12-19T16:55:00.628146+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:16.382474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.623300+0000","last_change":"2023-12-19T16:55:00.620298+0000","last_active":"2023-12-19T16:55:00.623300+0000","last_peered":"2023-12-19T16:55:00.623300+0000","last_clean":"2023-12-19T16:55:00.623300+0000","last_became_active":"2023-12-19T16:55:00.619959+0000","last_became_peered":"2023-12-19T16:55:00.619959+0000","last_unstale":"2023-12-19T16:55:00.623300+0000","last_undegraded":"2023-12-19T16:55:00.623300+0000","last_fullsized":"2023-12-19T16:55:00.623300+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:49:39.135191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.606623+0000","last_change":"2023-12-19T16:55:00.599837+0000","last_active":"2023-12-19T16:55:00.606623+0000","last_peered":"2023-12-19T16:55:00.606623+0000","last_clean":"2023-12-19T16:55:00.606623+0000","last_became_active":"2023-12-19T16:55:00.599458+0000","last_became_peered":"2023-12-19T16:55:00.599458+0000","last_unstale":"2023-12-19T16:55:00.606623+0000","last_undegraded":"2023-12-19T16:55:00.606623+0000","last_fullsized":"2023-12-19T16:55:00.606623+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:17:29.306993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619153+0000","last_change":"2023-12-19T16:55:00.610412+0000","last_active":"2023-12-19T16:55:00.619153+0000","last_peered":"2023-12-19T16:55:00.619153+0000","last_clean":"2023-12-19T16:55:00.619153+0000","last_became_active":"2023-12-19T16:55:00.610144+0000","last_became_peered":"2023-12-19T16:55:00.610144+0000","last_unstale":"2023-12-19T16:55:00.619153+0000","last_undegraded":"2023-12-19T16:55:00.619153+0000","last_fullsized":"2023-12-19T16:55:00.619153+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:56.751969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619783+0000","last_change":"2023-12-19T16:55:00.613888+0000","last_active":"2023-12-19T16:55:00.619783+0000","last_peered":"2023-12-19T16:55:00.619783+0000","last_clean":"2023-12-19T16:55:00.619783+0000","last_became_active":"2023-12-19T16:55:00.613516+0000","last_became_peered":"2023-12-19T16:55:00.613516+0000","last_unstale":"2023-12-19T16:55:00.619783+0000","last_undegraded":"2023-12-19T16:55:00.619783+0000","last_fullsized":"2023-12-19T16:55:00.619783+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:29:27.000630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619094+0000","last_change":"2023-12-19T16:55:00.616545+0000","last_active":"2023-12-19T16:55:00.619094+0000","last_peered":"2023-12-19T16:55:00.619094+0000","last_clean":"2023-12-19T16:55:00.619094+0000","last_became_active":"2023-12-19T16:55:00.616229+0000","last_became_peered":"2023-12-19T16:55:00.616229+0000","last_unstale":"2023-12-19T16:55:00.619094+0000","last_undegraded":"2023-12-19T16:55:00.619094+0000","last_fullsized":"2023-12-19T16:55:00.619094+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:10.646267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.621651+0000","last_change":"2023-12-19T16:55:00.605354+0000","last_active":"2023-12-19T16:55:00.621651+0000","last_peered":"2023-12-19T16:55:00.621651+0000","last_clean":"2023-12-19T16:55:00.621651+0000","last_became_active":"2023-12-19T16:55:00.605004+0000","last_became_peered":"2023-12-19T16:55:00.605004+0000","last_unstale":"2023-12-19T16:55:00.621651+0000","last_undegraded":"2023-12-19T16:55:00.621651+0000","last_fullsized":"2023-12-19T16:55:00.621651+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:20:45.257107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.611498+0000","last_change":"2023-12-19T16:55:00.607628+0000","last_active":"2023-12-19T16:55:00.611498+0000","last_peered":"2023-12-19T16:55:00.611498+0000","last_clean":"2023-12-19T16:55:00.611498+0000","last_became_active":"2023-12-19T16:55:00.607355+0000","last_became_peered":"2023-12-19T16:55:00.607355+0000","last_unstale":"2023-12-19T16:55:00.611498+0000","last_undegraded":"2023-12-19T16:55:00.611498+0000","last_fullsized":"2023-12-19T16:55:00.611498+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:20:06.746240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.627736+0000","last_change":"2023-12-19T16:55:00.623978+0000","last_active":"2023-12-19T16:55:00.627736+0000","last_peered":"2023-12-19T16:55:00.627736+0000","last_clean":"2023-12-19T16:55:00.627736+0000","last_became_active":"2023-12-19T16:55:00.623661+0000","last_became_peered":"2023-12-19T16:55:00.623661+0000","last_unstale":"2023-12-19T16:55:00.627736+0000","last_undegraded":"2023-12-19T16:55:00.627736+0000","last_fullsized":"2023-12-19T16:55:00.627736+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:56.751969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"60'4","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.627626+0000","last_change":"2023-12-19T16:55:00.203278+0000","last_active":"2023-12-19T16:55:00.627626+0000","last_peered":"2023-12-19T16:55:00.627626+0000","last_clean":"2023-12-19T16:55:00.627626+0000","last_became_active":"2023-12-19T16:55:00.202906+0000","last_became_peered":"2023-12-19T16:55:00.202906+0000","last_unstale":"2023-12-19T16:55:00.627626+0000","last_undegraded":"2023-12-19T16:55:00.627626+0000","last_fullsized":"2023-12-19T16:55:00.627626+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:45:00.387064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619873+0000","last_change":"2023-12-19T16:55:00.613979+0000","last_active":"2023-12-19T16:55:00.619873+0000","last_peered":"2023-12-19T16:55:00.619873+0000","last_clean":"2023-12-19T16:55:00.619873+0000","last_became_active":"2023-12-19T16:55:00.612765+0000","last_became_peered":"2023-12-19T16:55:00.612765+0000","last_unstale":"2023-12-19T16:55:00.619873+0000","last_undegraded":"2023-12-19T16:55:00.619873+0000","last_fullsized":"2023-12-19T16:55:00.619873+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:37:19.208105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.620329+0000","last_change":"2023-12-19T16:55:00.202642+0000","last_active":"2023-12-19T16:55:00.620329+0000","last_peered":"2023-12-19T16:55:00.620329+0000","last_clean":"2023-12-19T16:55:00.620329+0000","last_became_active":"2023-12-19T16:55:00.202330+0000","last_became_peered":"2023-12-19T16:55:00.202330+0000","last_unstale":"2023-12-19T16:55:00.620329+0000","last_undegraded":"2023-12-19T16:55:00.620329+0000","last_fullsized":"2023-12-19T16:55:00.620329+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:34:07.118135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.611708+0000","last_change":"2023-12-19T16:55:00.608631+0000","last_active":"2023-12-19T16:55:00.611708+0000","last_peered":"2023-12-19T16:55:00.611708+0000","last_clean":"2023-12-19T16:55:00.611708+0000","last_became_active":"2023-12-19T16:55:00.608349+0000","last_became_peered":"2023-12-19T16:55:00.608349+0000","last_unstale":"2023-12-19T16:55:00.611708+0000","last_undegraded":"2023-12-19T16:55:00.611708+0000","last_fullsized":"2023-12-19T16:55:00.611708+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:40:05.720430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626950+0000","last_change":"2023-12-19T16:55:00.201865+0000","last_active":"2023-12-19T16:55:00.626950+0000","last_peered":"2023-12-19T16:55:00.626950+0000","last_clean":"2023-12-19T16:55:00.626950+0000","last_became_active":"2023-12-19T16:55:00.195845+0000","last_became_peered":"2023-12-19T16:55:00.195845+0000","last_unstale":"2023-12-19T16:55:00.626950+0000","last_undegraded":"2023-12-19T16:55:00.626950+0000","last_fullsized":"2023-12-19T16:55:00.626950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:18.347062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.624843+0000","last_change":"2023-12-19T16:55:00.619596+0000","last_active":"2023-12-19T16:55:00.624843+0000","last_peered":"2023-12-19T16:55:00.624843+0000","last_clean":"2023-12-19T16:55:00.624843+0000","last_became_active":"2023-12-19T16:55:00.619308+0000","last_became_peered":"2023-12-19T16:55:00.619308+0000","last_unstale":"2023-12-19T16:55:00.624843+0000","last_undegraded":"2023-12-19T16:55:00.624843+0000","last_fullsized":"2023-12-19T16:55:00.624843+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:28.722500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619957+0000","last_change":"2023-12-19T16:55:00.197689+0000","last_active":"2023-12-19T16:55:00.619957+0000","last_peered":"2023-12-19T16:55:00.619957+0000","last_clean":"2023-12-19T16:55:00.619957+0000","last_became_active":"2023-12-19T16:55:00.197410+0000","last_became_peered":"2023-12-19T16:55:00.197410+0000","last_unstale":"2023-12-19T16:55:00.619957+0000","last_undegraded":"2023-12-19T16:55:00.619957+0000","last_fullsized":"2023-12-19T16:55:00.619957+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:18.347062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619073+0000","last_change":"2023-12-19T16:55:00.614966+0000","last_active":"2023-12-19T16:55:00.619073+0000","last_peered":"2023-12-19T16:55:00.619073+0000","last_clean":"2023-12-19T16:55:00.619073+0000","last_became_active":"2023-12-19T16:55:00.614699+0000","last_became_peered":"2023-12-19T16:55:00.614699+0000","last_unstale":"2023-12-19T16:55:00.619073+0000","last_undegraded":"2023-12-19T16:55:00.619073+0000","last_fullsized":"2023-12-19T16:55:00.619073+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:37:19.208105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.622818+0000","last_change":"2023-12-19T16:55:00.205410+0000","last_active":"2023-12-19T16:55:00.622818+0000","last_peered":"2023-12-19T16:55:00.622818+0000","last_clean":"2023-12-19T16:55:00.622818+0000","last_became_active":"2023-12-19T16:55:00.205108+0000","last_became_peered":"2023-12-19T16:55:00.205108+0000","last_unstale":"2023-12-19T16:55:00.622818+0000","last_undegraded":"2023-12-19T16:55:00.622818+0000","last_fullsized":"2023-12-19T16:55:00.622818+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:55.744148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.621735+0000","last_change":"2023-12-19T16:55:00.610650+0000","last_active":"2023-12-19T16:55:00.621735+0000","last_peered":"2023-12-19T16:55:00.621735+0000","last_clean":"2023-12-19T16:55:00.621735+0000","last_became_active":"2023-12-19T16:55:00.610355+0000","last_became_peered":"2023-12-19T16:55:00.610355+0000","last_unstale":"2023-12-19T16:55:00.621735+0000","last_undegraded":"2023-12-19T16:55:00.621735+0000","last_fullsized":"2023-12-19T16:55:00.621735+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:36:43.798547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.615413+0000","last_change":"2023-12-19T16:55:00.203032+0000","last_active":"2023-12-19T16:55:00.615413+0000","last_peered":"2023-12-19T16:55:00.615413+0000","last_clean":"2023-12-19T16:55:00.615413+0000","last_became_active":"2023-12-19T16:55:00.202321+0000","last_became_peered":"2023-12-19T16:55:00.202321+0000","last_unstale":"2023-12-19T16:55:00.615413+0000","last_undegraded":"2023-12-19T16:55:00.615413+0000","last_fullsized":"2023-12-19T16:55:00.615413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:31.179320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630432+0000","last_change":"2023-12-19T16:55:00.615247+0000","last_active":"2023-12-19T16:55:00.630432+0000","last_peered":"2023-12-19T16:55:00.630432+0000","last_clean":"2023-12-19T16:55:00.630432+0000","last_became_active":"2023-12-19T16:55:00.614972+0000","last_became_peered":"2023-12-19T16:55:00.614972+0000","last_unstale":"2023-12-19T16:55:00.630432+0000","last_undegraded":"2023-12-19T16:55:00.630432+0000","last_fullsized":"2023-12-19T16:55:00.630432+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:09:07.090780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.595852+0000","last_change":"2023-12-19T16:55:00.208977+0000","last_active":"2023-12-19T16:55:00.595852+0000","last_peered":"2023-12-19T16:55:00.595852+0000","last_clean":"2023-12-19T16:55:00.595852+0000","last_became_active":"2023-12-19T16:55:00.208559+0000","last_became_peered":"2023-12-19T16:55:00.208559+0000","last_unstale":"2023-12-19T16:55:00.595852+0000","last_undegraded":"2023-12-19T16:55:00.595852+0000","last_fullsized":"2023-12-19T16:55:00.595852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:31:44.620969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626085+0000","last_change":"2023-12-19T16:55:00.621564+0000","last_active":"2023-12-19T16:55:00.626085+0000","last_peered":"2023-12-19T16:55:00.626085+0000","last_clean":"2023-12-19T16:55:00.626085+0000","last_became_active":"2023-12-19T16:55:00.620575+0000","last_became_peered":"2023-12-19T16:55:00.620575+0000","last_unstale":"2023-12-19T16:55:00.626085+0000","last_undegraded":"2023-12-19T16:55:00.626085+0000","last_fullsized":"2023-12-19T16:55:00.626085+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:40.188691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.590625+0000","last_change":"2023-12-19T16:55:00.209278+0000","last_active":"2023-12-19T16:55:00.590625+0000","last_peered":"2023-12-19T16:55:00.590625+0000","last_clean":"2023-12-19T16:55:00.590625+0000","last_became_active":"2023-12-19T16:55:00.208723+0000","last_became_peered":"2023-12-19T16:55:00.208723+0000","last_unstale":"2023-12-19T16:55:00.590625+0000","last_undegraded":"2023-12-19T16:55:00.590625+0000","last_fullsized":"2023-12-19T16:55:00.590625+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:08:07.783530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.618039+0000","last_change":"2023-12-19T16:55:00.607387+0000","last_active":"2023-12-19T16:55:00.618039+0000","last_peered":"2023-12-19T16:55:00.618039+0000","last_clean":"2023-12-19T16:55:00.618039+0000","last_became_active":"2023-12-19T16:55:00.606750+0000","last_became_peered":"2023-12-19T16:55:00.606750+0000","last_unstale":"2023-12-19T16:55:00.618039+0000","last_undegraded":"2023-12-19T16:55:00.618039+0000","last_fullsized":"2023-12-19T16:55:00.618039+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:40.188691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.593440+0000","last_change":"2023-12-19T16:55:00.197477+0000","last_active":"2023-12-19T16:55:00.593440+0000","last_peered":"2023-12-19T16:55:00.593440+0000","last_clean":"2023-12-19T16:55:00.593440+0000","last_became_active":"2023-12-19T16:55:00.197165+0000","last_became_peered":"2023-12-19T16:55:00.197165+0000","last_unstale":"2023-12-19T16:55:00.593440+0000","last_undegraded":"2023-12-19T16:55:00.593440+0000","last_fullsized":"2023-12-19T16:55:00.593440+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:48:05.029740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.623814+0000","last_change":"2023-12-19T16:55:00.616611+0000","last_active":"2023-12-19T16:55:00.623814+0000","last_peered":"2023-12-19T16:55:00.623814+0000","last_clean":"2023-12-19T16:55:00.623814+0000","last_became_active":"2023-12-19T16:55:00.616337+0000","last_became_peered":"2023-12-19T16:55:00.616337+0000","last_unstale":"2023-12-19T16:55:00.623814+0000","last_undegraded":"2023-12-19T16:55:00.623814+0000","last_fullsized":"2023-12-19T16:55:00.623814+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:17:37.827059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.591089+0000","last_change":"2023-12-19T16:55:00.208869+0000","last_active":"2023-12-19T16:55:00.591089+0000","last_peered":"2023-12-19T16:55:00.591089+0000","last_clean":"2023-12-19T16:55:00.591089+0000","last_became_active":"2023-12-19T16:55:00.208526+0000","last_became_peered":"2023-12-19T16:55:00.208526+0000","last_unstale":"2023-12-19T16:55:00.591089+0000","last_undegraded":"2023-12-19T16:55:00.591089+0000","last_fullsized":"2023-12-19T16:55:00.591089+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:44:59.307702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.625861+0000","last_change":"2023-12-19T16:55:00.615664+0000","last_active":"2023-12-19T16:55:00.625861+0000","last_peered":"2023-12-19T16:55:00.625861+0000","last_clean":"2023-12-19T16:55:00.625861+0000","last_became_active":"2023-12-19T16:55:00.615384+0000","last_became_peered":"2023-12-19T16:55:00.615384+0000","last_unstale":"2023-12-19T16:55:00.625861+0000","last_undegraded":"2023-12-19T16:55:00.625861+0000","last_fullsized":"2023-12-19T16:55:00.625861+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:17:23.459042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"60'3","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.623815+0000","last_change":"2023-12-19T16:55:00.200264+0000","last_active":"2023-12-19T16:55:00.623815+0000","last_peered":"2023-12-19T16:55:00.623815+0000","last_clean":"2023-12-19T16:55:00.623815+0000","last_became_active":"2023-12-19T16:55:00.199925+0000","last_became_peered":"2023-12-19T16:55:00.199925+0000","last_unstale":"2023-12-19T16:55:00.623815+0000","last_undegraded":"2023-12-19T16:55:00.623815+0000","last_fullsized":"2023-12-19T16:55:00.623815+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:43:24.212407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626532+0000","last_change":"2023-12-19T16:55:00.619067+0000","last_active":"2023-12-19T16:55:00.626532+0000","last_peered":"2023-12-19T16:55:00.626532+0000","last_clean":"2023-12-19T16:55:00.626532+0000","last_became_active":"2023-12-19T16:55:00.618681+0000","last_became_peered":"2023-12-19T16:55:00.618681+0000","last_unstale":"2023-12-19T16:55:00.626532+0000","last_undegraded":"2023-12-19T16:55:00.626532+0000","last_fullsized":"2023-12-19T16:55:00.626532+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:04:02.627170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"60'6","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.629964+0000","last_change":"2023-12-19T16:55:00.201264+0000","last_active":"2023-12-19T16:55:00.629964+0000","last_peered":"2023-12-19T16:55:00.629964+0000","last_clean":"2023-12-19T16:55:00.629964+0000","last_became_active":"2023-12-19T16:55:00.197880+0000","last_became_peered":"2023-12-19T16:55:00.197880+0000","last_unstale":"2023-12-19T16:55:00.629964+0000","last_undegraded":"2023-12-19T16:55:00.629964+0000","last_fullsized":"2023-12-19T16:55:00.629964+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:53:38.563630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.620124+0000","last_change":"2023-12-19T16:55:00.612990+0000","last_active":"2023-12-19T16:55:00.620124+0000","last_peered":"2023-12-19T16:55:00.620124+0000","last_clean":"2023-12-19T16:55:00.620124+0000","last_became_active":"2023-12-19T16:55:00.612420+0000","last_became_peered":"2023-12-19T16:55:00.612420+0000","last_unstale":"2023-12-19T16:55:00.620124+0000","last_undegraded":"2023-12-19T16:55:00.620124+0000","last_fullsized":"2023-12-19T16:55:00.620124+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:28.866354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.622914+0000","last_change":"2023-12-19T16:55:00.211256+0000","last_active":"2023-12-19T16:55:00.622914+0000","last_peered":"2023-12-19T16:55:00.622914+0000","last_clean":"2023-12-19T16:55:00.622914+0000","last_became_active":"2023-12-19T16:55:00.210933+0000","last_became_peered":"2023-12-19T16:55:00.210933+0000","last_unstale":"2023-12-19T16:55:00.622914+0000","last_undegraded":"2023-12-19T16:55:00.622914+0000","last_fullsized":"2023-12-19T16:55:00.622914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:45:25.749655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630661+0000","last_change":"2023-12-19T16:55:00.623214+0000","last_active":"2023-12-19T16:55:00.630661+0000","last_peered":"2023-12-19T16:55:00.630661+0000","last_clean":"2023-12-19T16:55:00.630661+0000","last_became_active":"2023-12-19T16:55:00.622966+0000","last_became_peered":"2023-12-19T16:55:00.622966+0000","last_unstale":"2023-12-19T16:55:00.630661+0000","last_undegraded":"2023-12-19T16:55:00.630661+0000","last_fullsized":"2023-12-19T16:55:00.630661+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:25.203498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.601259+0000","last_change":"2023-12-19T16:55:00.199254+0000","last_active":"2023-12-19T16:55:00.601259+0000","last_peered":"2023-12-19T16:55:00.601259+0000","last_clean":"2023-12-19T16:55:00.601259+0000","last_became_active":"2023-12-19T16:55:00.198742+0000","last_became_peered":"2023-12-19T16:55:00.198742+0000","last_unstale":"2023-12-19T16:55:00.601259+0000","last_undegraded":"2023-12-19T16:55:00.601259+0000","last_fullsized":"2023-12-19T16:55:00.601259+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:19.175671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.625886+0000","last_change":"2023-12-19T16:55:00.615792+0000","last_active":"2023-12-19T16:55:00.625886+0000","last_peered":"2023-12-19T16:55:00.625886+0000","last_clean":"2023-12-19T16:55:00.625886+0000","last_became_active":"2023-12-19T16:55:00.615414+0000","last_became_peered":"2023-12-19T16:55:00.615414+0000","last_unstale":"2023-12-19T16:55:00.625886+0000","last_undegraded":"2023-12-19T16:55:00.625886+0000","last_fullsized":"2023-12-19T16:55:00.625886+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:01:36.879382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.598010+0000","last_change":"2023-12-19T16:55:00.210853+0000","last_active":"2023-12-19T16:55:00.598010+0000","last_peered":"2023-12-19T16:55:00.598010+0000","last_clean":"2023-12-19T16:55:00.598010+0000","last_became_active":"2023-12-19T16:55:00.210537+0000","last_became_peered":"2023-12-19T16:55:00.210537+0000","last_unstale":"2023-12-19T16:55:00.598010+0000","last_undegraded":"2023-12-19T16:55:00.598010+0000","last_fullsized":"2023-12-19T16:55:00.598010+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:40:04.712609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626845+0000","last_change":"2023-12-19T16:55:00.620556+0000","last_active":"2023-12-19T16:55:00.626845+0000","last_peered":"2023-12-19T16:55:00.626845+0000","last_clean":"2023-12-19T16:55:00.626845+0000","last_became_active":"2023-12-19T16:55:00.620205+0000","last_became_peered":"2023-12-19T16:55:00.620205+0000","last_unstale":"2023-12-19T16:55:00.626845+0000","last_undegraded":"2023-12-19T16:55:00.626845+0000","last_fullsized":"2023-12-19T16:55:00.626845+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:54:42.214844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"60'8","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.630779+0000","last_change":"2023-12-19T16:55:00.209380+0000","last_active":"2023-12-19T16:55:00.630779+0000","last_peered":"2023-12-19T16:55:00.630779+0000","last_clean":"2023-12-19T16:55:00.630779+0000","last_became_active":"2023-12-19T16:55:00.209113+0000","last_became_peered":"2023-12-19T16:55:00.209113+0000","last_unstale":"2023-12-19T16:55:00.630779+0000","last_undegraded":"2023-12-19T16:55:00.630779+0000","last_fullsized":"2023-12-19T16:55:00.630779+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:20:12.924446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.618033+0000","last_change":"2023-12-19T16:55:00.611455+0000","last_active":"2023-12-19T16:55:00.618033+0000","last_peered":"2023-12-19T16:55:00.618033+0000","last_clean":"2023-12-19T16:55:00.618033+0000","last_became_active":"2023-12-19T16:55:00.611203+0000","last_became_peered":"2023-12-19T16:55:00.611203+0000","last_unstale":"2023-12-19T16:55:00.618033+0000","last_undegraded":"2023-12-19T16:55:00.618033+0000","last_fullsized":"2023-12-19T16:55:00.618033+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:54:55.529301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628669+0000","last_change":"2023-12-19T16:55:00.201631+0000","last_active":"2023-12-19T16:55:00.628669+0000","last_peered":"2023-12-19T16:55:00.628669+0000","last_clean":"2023-12-19T16:55:00.628669+0000","last_became_active":"2023-12-19T16:55:00.197371+0000","last_became_peered":"2023-12-19T16:55:00.197371+0000","last_unstale":"2023-12-19T16:55:00.628669+0000","last_undegraded":"2023-12-19T16:55:00.628669+0000","last_fullsized":"2023-12-19T16:55:00.628669+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:36:16.360153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626955+0000","last_change":"2023-12-19T16:55:00.617789+0000","last_active":"2023-12-19T16:55:00.626955+0000","last_peered":"2023-12-19T16:55:00.626955+0000","last_clean":"2023-12-19T16:55:00.626955+0000","last_became_active":"2023-12-19T16:55:00.617525+0000","last_became_peered":"2023-12-19T16:55:00.617525+0000","last_unstale":"2023-12-19T16:55:00.626955+0000","last_undegraded":"2023-12-19T16:55:00.626955+0000","last_fullsized":"2023-12-19T16:55:00.626955+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:25:01.530638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.624694+0000","last_change":"2023-12-19T16:55:00.197851+0000","last_active":"2023-12-19T16:55:00.624694+0000","last_peered":"2023-12-19T16:55:00.624694+0000","last_clean":"2023-12-19T16:55:00.624694+0000","last_became_active":"2023-12-19T16:55:00.197534+0000","last_became_peered":"2023-12-19T16:55:00.197534+0000","last_unstale":"2023-12-19T16:55:00.624694+0000","last_undegraded":"2023-12-19T16:55:00.624694+0000","last_fullsized":"2023-12-19T16:55:00.624694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:41:03.128639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628968+0000","last_change":"2023-12-19T16:55:00.626098+0000","last_active":"2023-12-19T16:55:00.628968+0000","last_peered":"2023-12-19T16:55:00.628968+0000","last_clean":"2023-12-19T16:55:00.628968+0000","last_became_active":"2023-12-19T16:55:00.625829+0000","last_became_peered":"2023-12-19T16:55:00.625829+0000","last_unstale":"2023-12-19T16:55:00.628968+0000","last_undegraded":"2023-12-19T16:55:00.628968+0000","last_fullsized":"2023-12-19T16:55:00.628968+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:10.646267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"60'1","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.621798+0000","last_change":"2023-12-19T16:55:00.194182+0000","last_active":"2023-12-19T16:55:00.621798+0000","last_peered":"2023-12-19T16:55:00.621798+0000","last_clean":"2023-12-19T16:55:00.621798+0000","last_became_active":"2023-12-19T16:55:00.193893+0000","last_became_peered":"2023-12-19T16:55:00.193893+0000","last_unstale":"2023-12-19T16:55:00.621798+0000","last_undegraded":"2023-12-19T16:55:00.621798+0000","last_fullsized":"2023-12-19T16:55:00.621798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:07.257748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628132+0000","last_change":"2023-12-19T16:55:00.623640+0000","last_active":"2023-12-19T16:55:00.628132+0000","last_peered":"2023-12-19T16:55:00.628132+0000","last_clean":"2023-12-19T16:55:00.628132+0000","last_became_active":"2023-12-19T16:55:00.623299+0000","last_became_peered":"2023-12-19T16:55:00.623299+0000","last_unstale":"2023-12-19T16:55:00.628132+0000","last_undegraded":"2023-12-19T16:55:00.628132+0000","last_fullsized":"2023-12-19T16:55:00.628132+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:38.197379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.605103+0000","last_change":"2023-12-19T16:55:00.201895+0000","last_active":"2023-12-19T16:55:00.605103+0000","last_peered":"2023-12-19T16:55:00.605103+0000","last_clean":"2023-12-19T16:55:00.605103+0000","last_became_active":"2023-12-19T16:55:00.201614+0000","last_became_peered":"2023-12-19T16:55:00.201614+0000","last_unstale":"2023-12-19T16:55:00.605103+0000","last_undegraded":"2023-12-19T16:55:00.605103+0000","last_fullsized":"2023-12-19T16:55:00.605103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:43:24.212407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.625836+0000","last_change":"2023-12-19T16:55:00.615759+0000","last_active":"2023-12-19T16:55:00.625836+0000","last_peered":"2023-12-19T16:55:00.625836+0000","last_clean":"2023-12-19T16:55:00.625836+0000","last_became_active":"2023-12-19T16:55:00.615495+0000","last_became_peered":"2023-12-19T16:55:00.615495+0000","last_unstale":"2023-12-19T16:55:00.625836+0000","last_undegraded":"2023-12-19T16:55:00.625836+0000","last_fullsized":"2023-12-19T16:55:00.625836+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:55:35.867260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.622669+0000","last_change":"2023-12-19T16:55:00.203924+0000","last_active":"2023-12-19T16:55:00.622669+0000","last_peered":"2023-12-19T16:55:00.622669+0000","last_clean":"2023-12-19T16:55:00.622669+0000","last_became_active":"2023-12-19T16:55:00.203565+0000","last_became_peered":"2023-12-19T16:55:00.203565+0000","last_unstale":"2023-12-19T16:55:00.622669+0000","last_undegraded":"2023-12-19T16:55:00.622669+0000","last_fullsized":"2023-12-19T16:55:00.622669+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:27.714679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.621086+0000","last_change":"2023-12-19T16:55:00.607146+0000","last_active":"2023-12-19T16:55:00.621086+0000","last_peered":"2023-12-19T16:55:00.621086+0000","last_clean":"2023-12-19T16:55:00.621086+0000","last_became_active":"2023-12-19T16:55:00.606795+0000","last_became_peered":"2023-12-19T16:55:00.606795+0000","last_unstale":"2023-12-19T16:55:00.621086+0000","last_undegraded":"2023-12-19T16:55:00.621086+0000","last_fullsized":"2023-12-19T16:55:00.621086+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:40:05.720430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.596512+0000","last_change":"2023-12-19T16:55:00.201065+0000","last_active":"2023-12-19T16:55:00.596512+0000","last_peered":"2023-12-19T16:55:00.596512+0000","last_clean":"2023-12-19T16:55:00.596512+0000","last_became_active":"2023-12-19T16:55:00.199073+0000","last_became_peered":"2023-12-19T16:55:00.199073+0000","last_unstale":"2023-12-19T16:55:00.596512+0000","last_undegraded":"2023-12-19T16:55:00.596512+0000","last_fullsized":"2023-12-19T16:55:00.596512+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:38.569496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.620780+0000","last_change":"2023-12-19T16:55:00.617233+0000","last_active":"2023-12-19T16:55:00.620780+0000","last_peered":"2023-12-19T16:55:00.620780+0000","last_clean":"2023-12-19T16:55:00.620780+0000","last_became_active":"2023-12-19T16:55:00.616534+0000","last_became_peered":"2023-12-19T16:55:00.616534+0000","last_unstale":"2023-12-19T16:55:00.620780+0000","last_undegraded":"2023-12-19T16:55:00.620780+0000","last_fullsized":"2023-12-19T16:55:00.620780+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:10:03.674528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.592255+0000","last_change":"2023-12-19T16:55:00.206479+0000","last_active":"2023-12-19T16:55:00.592255+0000","last_peered":"2023-12-19T16:55:00.592255+0000","last_clean":"2023-12-19T16:55:00.592255+0000","last_became_active":"2023-12-19T16:55:00.206161+0000","last_became_peered":"2023-12-19T16:55:00.206161+0000","last_unstale":"2023-12-19T16:55:00.592255+0000","last_undegraded":"2023-12-19T16:55:00.592255+0000","last_fullsized":"2023-12-19T16:55:00.592255+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:01.143913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.631035+0000","last_change":"2023-12-19T16:55:00.627537+0000","last_active":"2023-12-19T16:55:00.631035+0000","last_peered":"2023-12-19T16:55:00.631035+0000","last_clean":"2023-12-19T16:55:00.631035+0000","last_became_active":"2023-12-19T16:55:00.627273+0000","last_became_peered":"2023-12-19T16:55:00.627273+0000","last_unstale":"2023-12-19T16:55:00.631035+0000","last_undegraded":"2023-12-19T16:55:00.631035+0000","last_fullsized":"2023-12-19T16:55:00.631035+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:26.235392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.601359+0000","last_change":"2023-12-19T16:55:00.198286+0000","last_active":"2023-12-19T16:55:00.601359+0000","last_peered":"2023-12-19T16:55:00.601359+0000","last_clean":"2023-12-19T16:55:00.601359+0000","last_became_active":"2023-12-19T16:55:00.198035+0000","last_became_peered":"2023-12-19T16:55:00.198035+0000","last_unstale":"2023-12-19T16:55:00.601359+0000","last_undegraded":"2023-12-19T16:55:00.601359+0000","last_fullsized":"2023-12-19T16:55:00.601359+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:11:20.624994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.619855+0000","last_change":"2023-12-19T16:55:00.612694+0000","last_active":"2023-12-19T16:55:00.619855+0000","last_peered":"2023-12-19T16:55:00.619855+0000","last_clean":"2023-12-19T16:55:00.619855+0000","last_became_active":"2023-12-19T16:55:00.612386+0000","last_became_peered":"2023-12-19T16:55:00.612386+0000","last_unstale":"2023-12-19T16:55:00.619855+0000","last_undegraded":"2023-12-19T16:55:00.619855+0000","last_fullsized":"2023-12-19T16:55:00.619855+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:47.181094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.606072+0000","last_change":"2023-12-19T16:55:00.200798+0000","last_active":"2023-12-19T16:55:00.606072+0000","last_peered":"2023-12-19T16:55:00.606072+0000","last_clean":"2023-12-19T16:55:00.606072+0000","last_became_active":"2023-12-19T16:55:00.200496+0000","last_became_peered":"2023-12-19T16:55:00.200496+0000","last_unstale":"2023-12-19T16:55:00.606072+0000","last_undegraded":"2023-12-19T16:55:00.606072+0000","last_fullsized":"2023-12-19T16:55:00.606072+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:19.175671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.605763+0000","last_change":"2023-12-19T16:55:00.602695+0000","last_active":"2023-12-19T16:55:00.605763+0000","last_peered":"2023-12-19T16:55:00.605763+0000","last_clean":"2023-12-19T16:55:00.605763+0000","last_became_active":"2023-12-19T16:55:00.602401+0000","last_became_peered":"2023-12-19T16:55:00.602401+0000","last_unstale":"2023-12-19T16:55:00.605763+0000","last_undegraded":"2023-12-19T16:55:00.605763+0000","last_fullsized":"2023-12-19T16:55:00.605763+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:57:52.945189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.602668+0000","last_change":"2023-12-19T16:55:00.204037+0000","last_active":"2023-12-19T16:55:00.602668+0000","last_peered":"2023-12-19T16:55:00.602668+0000","last_clean":"2023-12-19T16:55:00.602668+0000","last_became_active":"2023-12-19T16:55:00.203733+0000","last_became_peered":"2023-12-19T16:55:00.203733+0000","last_unstale":"2023-12-19T16:55:00.602668+0000","last_undegraded":"2023-12-19T16:55:00.602668+0000","last_fullsized":"2023-12-19T16:55:00.602668+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:47:48.610110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.626509+0000","last_change":"2023-12-19T16:55:00.617942+0000","last_active":"2023-12-19T16:55:00.626509+0000","last_peered":"2023-12-19T16:55:00.626509+0000","last_clean":"2023-12-19T16:55:00.626509+0000","last_became_active":"2023-12-19T16:55:00.617630+0000","last_became_peered":"2023-12-19T16:55:00.617630+0000","last_unstale":"2023-12-19T16:55:00.626509+0000","last_undegraded":"2023-12-19T16:55:00.626509+0000","last_fullsized":"2023-12-19T16:55:00.626509+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:35.798776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.603366+0000","last_change":"2023-12-19T16:55:00.206586+0000","last_active":"2023-12-19T16:55:00.603366+0000","last_peered":"2023-12-19T16:55:00.603366+0000","last_clean":"2023-12-19T16:55:00.603366+0000","last_became_active":"2023-12-19T16:55:00.205636+0000","last_became_peered":"2023-12-19T16:55:00.205636+0000","last_unstale":"2023-12-19T16:55:00.603366+0000","last_undegraded":"2023-12-19T16:55:00.603366+0000","last_fullsized":"2023-12-19T16:55:00.603366+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:41:11.221952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.621586+0000","last_change":"2023-12-19T16:55:00.605339+0000","last_active":"2023-12-19T16:55:00.621586+0000","last_peered":"2023-12-19T16:55:00.621586+0000","last_clean":"2023-12-19T16:55:00.621586+0000","last_became_active":"2023-12-19T16:55:00.605071+0000","last_became_peered":"2023-12-19T16:55:00.605071+0000","last_unstale":"2023-12-19T16:55:00.621586+0000","last_undegraded":"2023-12-19T16:55:00.621586+0000","last_fullsized":"2023-12-19T16:55:00.621586+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:36.450183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.602260+0000","last_change":"2023-12-19T16:55:00.203382+0000","last_active":"2023-12-19T16:55:00.602260+0000","last_peered":"2023-12-19T16:55:00.602260+0000","last_clean":"2023-12-19T16:55:00.602260+0000","last_became_active":"2023-12-19T16:55:00.201558+0000","last_became_peered":"2023-12-19T16:55:00.201558+0000","last_unstale":"2023-12-19T16:55:00.602260+0000","last_undegraded":"2023-12-19T16:55:00.602260+0000","last_fullsized":"2023-12-19T16:55:00.602260+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:41:11.221952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.625988+0000","last_change":"2023-12-19T16:55:00.616185+0000","last_active":"2023-12-19T16:55:00.625988+0000","last_peered":"2023-12-19T16:55:00.625988+0000","last_clean":"2023-12-19T16:55:00.625988+0000","last_became_active":"2023-12-19T16:55:00.615819+0000","last_became_peered":"2023-12-19T16:55:00.615819+0000","last_unstale":"2023-12-19T16:55:00.625988+0000","last_undegraded":"2023-12-19T16:55:00.625988+0000","last_fullsized":"2023-12-19T16:55:00.625988+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:05.573479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.593875+0000","last_change":"2023-12-19T16:55:00.198555+0000","last_active":"2023-12-19T16:55:00.593875+0000","last_peered":"2023-12-19T16:55:00.593875+0000","last_clean":"2023-12-19T16:55:00.593875+0000","last_became_active":"2023-12-19T16:55:00.198243+0000","last_became_peered":"2023-12-19T16:55:00.198243+0000","last_unstale":"2023-12-19T16:55:00.593875+0000","last_undegraded":"2023-12-19T16:55:00.593875+0000","last_fullsized":"2023-12-19T16:55:00.593875+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:32:15.694790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.621467+0000","last_change":"2023-12-19T16:55:00.615219+0000","last_active":"2023-12-19T16:55:00.621467+0000","last_peered":"2023-12-19T16:55:00.621467+0000","last_clean":"2023-12-19T16:55:00.621467+0000","last_became_active":"2023-12-19T16:55:00.614524+0000","last_became_peered":"2023-12-19T16:55:00.614524+0000","last_unstale":"2023-12-19T16:55:00.621467+0000","last_undegraded":"2023-12-19T16:55:00.621467+0000","last_fullsized":"2023-12-19T16:55:00.621467+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:28.722500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.603374+0000","last_change":"2023-12-19T16:55:00.204127+0000","last_active":"2023-12-19T16:55:00.603374+0000","last_peered":"2023-12-19T16:55:00.603374+0000","last_clean":"2023-12-19T16:55:00.603374+0000","last_became_active":"2023-12-19T16:55:00.203834+0000","last_became_peered":"2023-12-19T16:55:00.203834+0000","last_unstale":"2023-12-19T16:55:00.603374+0000","last_undegraded":"2023-12-19T16:55:00.603374+0000","last_fullsized":"2023-12-19T16:55:00.603374+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:11:20.624994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.617688+0000","last_change":"2023-12-19T16:55:00.606922+0000","last_active":"2023-12-19T16:55:00.617688+0000","last_peered":"2023-12-19T16:55:00.617688+0000","last_clean":"2023-12-19T16:55:00.617688+0000","last_became_active":"2023-12-19T16:55:00.606648+0000","last_became_peered":"2023-12-19T16:55:00.606648+0000","last_unstale":"2023-12-19T16:55:00.617688+0000","last_undegraded":"2023-12-19T16:55:00.617688+0000","last_fullsized":"2023-12-19T16:55:00.617688+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:49:32.328163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.590828+0000","last_change":"2023-12-19T16:55:00.208978+0000","last_active":"2023-12-19T16:55:00.590828+0000","last_peered":"2023-12-19T16:55:00.590828+0000","last_clean":"2023-12-19T16:55:00.590828+0000","last_became_active":"2023-12-19T16:55:00.208361+0000","last_became_peered":"2023-12-19T16:55:00.208361+0000","last_unstale":"2023-12-19T16:55:00.590828+0000","last_undegraded":"2023-12-19T16:55:00.590828+0000","last_fullsized":"2023-12-19T16:55:00.590828+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:01:32.454618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.610674+0000","last_change":"2023-12-19T16:55:00.608542+0000","last_active":"2023-12-19T16:55:00.610674+0000","last_peered":"2023-12-19T16:55:00.610674+0000","last_clean":"2023-12-19T16:55:00.610674+0000","last_became_active":"2023-12-19T16:55:00.608183+0000","last_became_peered":"2023-12-19T16:55:00.608183+0000","last_unstale":"2023-12-19T16:55:00.610674+0000","last_undegraded":"2023-12-19T16:55:00.610674+0000","last_fullsized":"2023-12-19T16:55:00.610674+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:55:00.172236+0000","last_clean_scrub_stamp":"2023-12-19T16:55:00.172236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:38.688313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.628018+0000","last_change":"2023-12-19T16:55:00.201570+0000","last_active":"2023-12-19T16:55:00.628018+0000","last_peered":"2023-12-19T16:55:00.628018+0000","last_clean":"2023-12-19T16:55:00.628018+0000","last_became_active":"2023-12-19T16:55:00.200298+0000","last_became_peered":"2023-12-19T16:55:00.200298+0000","last_unstale":"2023-12-19T16:55:00.628018+0000","last_undegraded":"2023-12-19T16:55:00.628018+0000","last_fullsized":"2023-12-19T16:55:00.628018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:54:59.164415+0000","last_clean_scrub_stamp":"2023-12-19T16:54:59.164415+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:01.816707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"49'139","reported_seq":280,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-12-19T16:55:00.186028+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:55:00.186028+0000","last_peered":"2023-12-19T16:55:00.186028+0000","last_clean":"2023-12-19T16:55:00.186028+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:55:00.186028+0000","last_undegraded":"2023-12-19T16:55:00.186028+0000","last_fullsized":"2023-12-19T16:55:00.186028+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509595,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1302528,"data_stored":918398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35699999999999998,"5min":0.33900000000000002,"15min":0.30199999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":2.6389999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.72299999999999998,"15min":0.72399999999999998},"min":{"1min":0.52400000000000002,"5min":0.45000000000000001,"15min":0.32700000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":2.532},"last":0.84399999999999997}]},{"osd":1,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72899999999999998,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.435,"15min":0.36399999999999999},"max":{"1min":1.03,"5min":1.05,"15min":1.05},"last":0.75},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.746,"15min":0.71799999999999997},"min":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.36799999999999999},"max":{"1min":1.0069999999999999,"5min":1.046,"15min":1.093},"last":0.91000000000000003}]},{"osd":2,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.438},"max":{"1min":0.94699999999999995,"5min":1.006,"15min":1.006},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.746,"15min":0.72899999999999998},"min":{"1min":0.57899999999999996,"5min":0.47299999999999998,"15min":0.36599999999999999},"max":{"1min":1.004,"5min":1.0429999999999999,"15min":1.157},"last":0.80500000000000005}]},{"osd":3,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.70899999999999996},"min":{"1min":0.49299999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.93400000000000005,"5min":0.996,"15min":0.996},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.76300000000000001,"15min":0.75},"min":{"1min":0.63300000000000001,"5min":0.54500000000000004,"15min":0.436},"max":{"1min":0.98799999999999999,"5min":1.0740000000000001,"15min":1.117},"last":0.65400000000000003}]},{"osd":4,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.59299999999999997},"min":{"1min":0.39400000000000002,"5min":0.373,"15min":0.28499999999999998},"max":{"1min":0.78900000000000003,"5min":0.89800000000000002,"15min":0.93600000000000005},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.40799999999999997,"5min":0.40200000000000002,"15min":0.36599999999999999},"max":{"1min":0.85399999999999998,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.72099999999999997}]},{"osd":5,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61099999999999999,"15min":0.622},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35699999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.96599999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65900000000000003,"15min":0.64900000000000002},"min":{"1min":0.46500000000000002,"5min":0.441,"15min":0.438},"max":{"1min":0.97099999999999997,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.76800000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:54:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.39800000000000002},"max":{"1min":0.98599999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.497,"5min":0.433,"15min":0.40899999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.872}]}]},{"osd":6,"up_from":15,"seq":64424509595,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31744,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340096,"statfs":{"total":96636764160,"available":96604258304,"internally_reserved":0,"allocated":712704,"data_stored":328612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.39000000000000001,"5min":0.38800000000000001,"15min":0.309},"max":{"1min":0.91400000000000003,"5min":0.999,"15min":1.8600000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71899999999999997,"15min":0.68999999999999995},"min":{"1min":0.44,"5min":0.44,"15min":0.34999999999999998},"max":{"1min":0.93999999999999995,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.55000000000000004}]},{"osd":1,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.71399999999999997,"15min":0.76200000000000001},"min":{"1min":0.46899999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79300000000000004,"5min":0.99399999999999999,"15min":4.3799999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.75,"5min":0.748,"15min":0.75},"min":{"1min":0.51500000000000001,"5min":0.46100000000000002,"15min":0.39500000000000002},"max":{"1min":0.89400000000000002,"5min":1.0189999999999999,"15min":1.583},"last":0.625}]},{"osd":2,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76600000000000001,"15min":0.747},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.35199999999999998},"max":{"1min":0.95599999999999996,"5min":1.087,"15min":1.3180000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74299999999999999,"15min":0.72899999999999998},"min":{"1min":0.54800000000000004,"5min":0.498,"15min":0.44400000000000001},"max":{"1min":0.95299999999999996,"5min":1.0549999999999999,"15min":1.1879999999999999},"last":0.5}]},{"osd":3,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76700000000000002,"15min":0.85899999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.51900000000000002},"max":{"1min":0.92700000000000005,"5min":1.1020000000000001,"15min":4.673},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72399999999999998,"15min":0.74099999999999999},"min":{"1min":0.42699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":1.0740000000000001,"15min":1.962},"last":0.59599999999999997}]},{"osd":4,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57699999999999996,"15min":0.58699999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.85599999999999998,"5min":0.89500000000000002,"15min":0.96999999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.40999999999999998,"5min":0.39800000000000002,"15min":0.35699999999999998},"max":{"1min":0.90800000000000003,"5min":0.99299999999999999,"15min":1.006},"last":0.443}]},{"osd":5,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59299999999999997,"15min":0.61499999999999999},"min":{"1min":0.43099999999999999,"5min":0.41199999999999998,"15min":0.33300000000000002},"max":{"1min":0.79000000000000004,"5min":0.91300000000000003,"15min":1.2050000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66400000000000003,"15min":0.66600000000000004},"min":{"1min":0.46200000000000002,"5min":0.45600000000000002,"15min":0.41499999999999998},"max":{"1min":0.89500000000000002,"5min":1.046,"15min":1.046},"last":0.71799999999999997}]},{"osd":7,"last update":"Tue Dec 19 16:54:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.51400000000000001,"5min":0.48599999999999999,"15min":0.438},"max":{"1min":0.97999999999999998,"5min":1.0249999999999999,"15min":1.0620000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.48499999999999999,"5min":0.45800000000000002,"15min":0.371},"max":{"1min":0.92500000000000004,"5min":1.032,"15min":1.1220000000000001},"last":0.66600000000000004}]}]},{"osd":5,"up_from":15,"seq":64424509595,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":708608,"data_stored":328010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.69299999999999995,"15min":0.68799999999999994},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85099999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.72499999999999998,"15min":0.71599999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84699999999999998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59399999999999997}]},{"osd":1,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.872,"5min":0.94899999999999995,"15min":0.97399999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.90800000000000003,"5min":0.91700000000000004,"15min":0.98799999999999999},"last":0.73199999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.1120000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.46400000000000002},"max":{"1min":0.96499999999999997,"5min":0.98999999999999999,"15min":1.087},"last":0.68600000000000005}]},{"osd":3,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":0.89800000000000002,"5min":0.96299999999999997,"15min":1.0129999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.65200000000000002}]},{"osd":4,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.55500000000000005,"15min":0.57399999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82199999999999995,"5min":0.85599999999999998,"15min":1.165},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59199999999999997,"15min":0.60599999999999998},"min":{"1min":0.44600000000000001,"5min":0.36099999999999999,"15min":0.32400000000000001},"max":{"1min":0.77200000000000002,"5min":0.97199999999999998,"15min":1.0649999999999999},"last":0.51900000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.34699999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.153},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60999999999999999,"15min":0.625},"min":{"1min":0.433,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78700000000000003,"5min":0.90300000000000002,"15min":0.98999999999999999},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Dec 19 16:54:45 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.64300000000000002,"15min":0.65800000000000003},"min":{"1min":0.50600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.78700000000000003,"5min":0.88300000000000001,"15min":1.133},"last":0.752},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66900000000000004,"15min":0.67800000000000005},"min":{"1min":0.48799999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.76500000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509595,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53892,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317948,"statfs":{"total":96636764160,"available":96581578752,"internally_reserved":0,"allocated":716800,"data_stored":329504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.35699999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.879,"5min":0.997,"15min":1.093},"last":0.755},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93500000000000005,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.61099999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71299999999999997,"15min":0.72199999999999998},"min":{"1min":0.53100000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88800000000000001,"5min":1.0209999999999999,"15min":1.8700000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72599999999999998,"15min":0.73399999999999999},"min":{"1min":0.38500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.86299999999999999,"5min":0.98999999999999999,"15min":1.05},"last":0.78700000000000003}]},{"osd":2,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.72899999999999998,"15min":0.72699999999999998},"min":{"1min":0.53900000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73999999999999999,"15min":0.755},"min":{"1min":0.53800000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91600000000000004,"5min":1.0549999999999999,"15min":1.075},"last":0.71899999999999997}]},{"osd":3,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.52700000000000002,"5min":0.497,"15min":0.46200000000000002},"max":{"1min":0.88400000000000001,"5min":0.90500000000000003,"15min":1.0249999999999999},"last":1.2030000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.75800000000000001,"15min":0.77100000000000002},"min":{"1min":0.56799999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.96599999999999997,"5min":0.99199999999999999,"15min":1.3129999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.60599999999999998},"min":{"1min":0.39700000000000002,"5min":0.38900000000000001,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.997,"15min":0.997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.42699999999999999,"5min":0.39400000000000002,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.89300000000000002,"15min":0.92800000000000005},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.63300000000000001},"min":{"1min":0.44600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79600000000000004,"5min":0.97199999999999998,"15min":1.0820000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66100000000000003,"15min":0.67600000000000005},"min":{"1min":0.432,"5min":0.41699999999999998,"15min":0.376},"max":{"1min":0.78100000000000003,"5min":0.95199999999999996,"15min":1.0189999999999999},"last":0.82799999999999996}]},{"osd":7,"last update":"Tue Dec 19 16:54:43 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.42199999999999999},"max":{"1min":0.78400000000000003,"5min":0.93200000000000005,"15min":1.0429999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.68799999999999994,"15min":0.71899999999999997},"min":{"1min":0.503,"5min":0.46700000000000003,"15min":0.44900000000000001},"max":{"1min":0.79300000000000004,"5min":1.0680000000000001,"15min":1.107},"last":0.77200000000000002}]}]},{"osd":3,"up_from":15,"seq":64424509595,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35844,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335996,"statfs":{"total":96636764160,"available":96600059904,"internally_reserved":0,"allocated":716800,"data_stored":328578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.443,"15min":0.45800000000000002},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49299999999999999,"15min":0.51500000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.163,"5min":1.163,"15min":1.738},"last":0.52900000000000003}]},{"osd":1,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.45300000000000001,"15min":0.47699999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69499999999999995,"5min":0.77800000000000002,"15min":1.141},"last":0.432},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.56299999999999994},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.952,"5min":1.952,"15min":2.4910000000000001},"last":0.54500000000000004}]},{"osd":2,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.54800000000000004,"15min":0.56699999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.64300000000000002,"5min":0.84099999999999997,"15min":1.821},"last":0.372},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.57299999999999995,"15min":0.64000000000000001},"min":{"1min":0.372,"5min":0.36799999999999999,"15min":0.35399999999999998},"max":{"1min":2.855,"5min":2.855,"15min":5.1219999999999999},"last":0.46600000000000003}]},{"osd":4,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.79500000000000004,"15min":0.80600000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.234,"5min":4.234,"15min":4.234},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.76600000000000001,"15min":0.80700000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96799999999999997,"5min":1.093,"15min":2.2690000000000001},"last":0.79900000000000004}]},{"osd":5,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.77800000000000002,"15min":0.80400000000000005},"min":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.88300000000000001,"5min":1.0640000000000001,"15min":2.5350000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.9079999999999999},"last":0.76600000000000001}]},{"osd":6,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.599,"5min":1.599,"15min":1.956},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.012,"5min":1.129,"15min":1.974},"last":0.89100000000000001}]},{"osd":7,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.82399999999999995,"15min":0.85599999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.51900000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.9279999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.85399999999999998,"15min":0.91700000000000004},"min":{"1min":0.624,"5min":0.60599999999999998,"15min":0.48799999999999999},"max":{"1min":1.623,"5min":1.623,"15min":5.0759999999999996},"last":0.77300000000000002}]}]},{"osd":2,"up_from":15,"seq":64424509594,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39944,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331896,"statfs":{"total":96636764160,"available":96595861504,"internally_reserved":0,"allocated":720896,"data_stored":329526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.49199999999999999,"15min":0.48899999999999999},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.28199999999999997},"max":{"1min":5.4870000000000001,"5min":5.4870000000000001,"15min":5.4870000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":1.012,"5min":0.60599999999999998,"15min":0.54100000000000004},"min":{"1min":0.35299999999999998,"5min":0.32500000000000001,"15min":0.29599999999999999},"max":{"1min":10.161,"5min":10.161,"15min":10.161},"last":0.32400000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.624,"15min":0.66700000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.39900000000000002},"max":{"1min":2.4740000000000002,"5min":2.4740000000000002,"15min":5.7590000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.51900000000000002,"15min":0.58499999999999996},"min":{"1min":0.35199999999999998,"5min":0.33000000000000002,"15min":0.29299999999999998},"max":{"1min":2.911,"5min":2.911,"15min":5.8070000000000004},"last":0.51400000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58399999999999996,"15min":0.60799999999999998},"min":{"1min":0.39600000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":2.7599999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.58099999999999996,"15min":0.63600000000000001},"min":{"1min":0.41599999999999998,"5min":0.39500000000000002,"15min":0.35099999999999998},"max":{"1min":2.7999999999999998,"5min":2.7999999999999998,"15min":5.7190000000000003},"last":0.42499999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.77100000000000002,"15min":0.83099999999999996},"min":{"1min":0.59399999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.76600000000000001,"5min":1.3149999999999999,"15min":3.8940000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.54500000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":5.5430000000000001,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":0.83199999999999996,"15min":0.81799999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":5.601,"5min":5.601,"15min":5.601},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.81399999999999995,"15min":0.86299999999999999},"min":{"1min":0.55700000000000005,"5min":0.55100000000000005,"15min":0.54900000000000004},"max":{"1min":1.27,"5min":1.3899999999999999,"15min":2.9260000000000002},"last":0.80400000000000005}]},{"osd":6,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.81899999999999995,"15min":0.84399999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.52400000000000002},"max":{"1min":1.236,"5min":1.2569999999999999,"15min":2.165},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77900000000000003,"15min":0.82799999999999996},"min":{"1min":0.55800000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.90800000000000003,"5min":1.1739999999999999,"15min":2.121},"last":0.68600000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:54:36 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":0.84399999999999997,"15min":0.84199999999999997},"min":{"1min":0.63,"5min":0.55500000000000005,"15min":0.52300000000000002},"max":{"1min":5.673,"5min":5.673,"15min":5.673},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.83699999999999997,"15min":0.86599999999999999},"min":{"1min":0.60599999999999998,"5min":0.58299999999999996,"15min":0.57799999999999996},"max":{"1min":1.3009999999999999,"5min":1.429,"15min":2.0529999999999999},"last":0.73499999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509595,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49788,"kb_used_data":692,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94322052,"statfs":{"total":96636764160,"available":96585781248,"internally_reserved":0,"allocated":708608,"data_stored":328010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.44700000000000001,"15min":0.46899999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55000000000000004,"5min":0.70399999999999996,"15min":2.3050000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.84199999999999997,"5min":0.85999999999999999,"15min":0.89800000000000002},"last":0.70499999999999996}]},{"osd":2,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.50600000000000001,"15min":0.53300000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67200000000000004,"5min":0.80100000000000005,"15min":1.992},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56399999999999995,"15min":0.55500000000000005},"min":{"1min":0.35999999999999999,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":0.72899999999999998,"5min":0.82899999999999996,"15min":0.97299999999999998},"last":0.38400000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.61199999999999999,"15min":0.65300000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.34799999999999998},"max":{"1min":0.79700000000000004,"5min":0.91500000000000004,"15min":4.133},"last":0.433},{"interface":"front","average":{"1min":0.496,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.34599999999999997,"15min":0.33700000000000002},"max":{"1min":0.70499999999999996,"5min":0.79000000000000004,"15min":0.91900000000000004},"last":0.44900000000000001}]},{"osd":4,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.76000000000000001,"15min":0.78900000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.202,"5min":1.202,"15min":2.1949999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.80600000000000005,"15min":0.80500000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.42399999999999999},"max":{"1min":1.181,"5min":1.23,"15min":2.048},"last":0.56499999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78600000000000003,"15min":0.80100000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.288,"5min":1.288,"15min":2.1269999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81399999999999995,"15min":0.85199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":4.0460000000000003},"last":0.57799999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.80200000000000005,"15min":0.82399999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.41999999999999998},"max":{"1min":1.0720000000000001,"5min":1.129,"15min":2.1499999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.81399999999999995,"15min":0.83099999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.391,"5min":1.391,"15min":2.0880000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Tue Dec 19 16:54:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.84199999999999997,"15min":0.89000000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":4.093},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.84299999999999997,"15min":0.86399999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.53800000000000003},"max":{"1min":1.147,"5min":1.246,"15min":2.2349999999999999},"last":0.59299999999999997}]}]},{"osd":0,"up_from":14,"seq":60129542300,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1302528,"data_stored":918342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.60599999999999998,"5min":0.70999999999999996,"15min":0.72499999999999998},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.5,"15min":0.47699999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.26300000000000001},"max":{"1min":0.70299999999999996,"5min":0.751,"15min":0.85499999999999998},"last":0.41699999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.29299999999999998},"max":{"1min":0.67000000000000004,"5min":0.77800000000000002,"15min":0.89700000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54500000000000004,"15min":0.52400000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.45800000000000002}]},{"osd":3,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.52800000000000002,"15min":0.50800000000000001},"min":{"1min":0.32500000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.624,"15min":0.60299999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40899999999999997},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.80600000000000005,"15min":0.79200000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.22,"5min":1.391,"15min":1.512},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81000000000000005,"15min":0.78600000000000003},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0529999999999999,"5min":1.177,"15min":1.52},"last":0.70099999999999996}]},{"osd":5,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.80800000000000005,"15min":0.80600000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.41499999999999998},"max":{"1min":1.145,"5min":1.2390000000000001,"15min":1.748},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81200000000000006,"15min":0.78800000000000003},"min":{"1min":0.56999999999999995,"5min":0.52300000000000002,"15min":0.51600000000000001},"max":{"1min":1.119,"5min":1.163,"15min":1.649},"last":0.61199999999999999}]},{"osd":6,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.81799999999999995,"15min":0.78300000000000003},"min":{"1min":0.55500000000000005,"5min":0.54400000000000004,"15min":0.46999999999999997},"max":{"1min":1.2470000000000001,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82499999999999996,"15min":0.80800000000000005},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.44700000000000001},"max":{"1min":1.0289999999999999,"5min":1.1200000000000001,"15min":1.2070000000000001},"last":0.63200000000000001}]},{"osd":7,"last update":"Tue Dec 19 16:54:40 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.84599999999999997,"15min":0.81200000000000006},"min":{"1min":0.505,"5min":0.505,"15min":0.48999999999999999},"max":{"1min":1.1739999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82599999999999996,"15min":0.81200000000000006},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1910000000000001,"5min":1.212,"15min":1.661},"last":0.66000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:55:05.108 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T16:55:05.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T16:55:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf2bd98640 1 Processor -- start 2023-12-19T16:55:05.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf2bd98640 1 -- start start 2023-12-19T16:55:05.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf2bd98640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcf2406cb20 0x7fcf2406f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf2bd98640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf2415a3c0 0x7fcf2415a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf2bd98640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2406f790 con 0x7fcf2406c740 2023-12-19T16:55:05.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf2bd98640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf2415d5a0 con 0x7fcf2406cb20 2023-12-19T16:55:05.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf2bd98640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2415d880 con 0x7fcf2415a3c0 2023-12-19T16:55:05.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf29333640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcf2406cb20 0x7fcf2406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf2a335640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf2415a3c0 0x7fcf2415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf29333640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcf2406cb20 0x7fcf2406f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:38078/0 (socket says 172.21.15.26:38078) 2023-12-19T16:55:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf29333640 1 -- 172.21.15.26:0/3832865224 learned_addr learned my addr 172.21.15.26:0/3832865224 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf29333640 1 -- 172.21.15.26:0/3832865224 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf2415a3c0 msgr2=0x7fcf2415a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf29333640 1 --2- 172.21.15.26:0/3832865224 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf2415a3c0 0x7fcf2415a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf29333640 1 -- 172.21.15.26:0/3832865224 >> v1:172.21.15.160:6789/0 conn(0x7fcf2406c740 legacy=0x7fcf240ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.291+0000 7fcf29333640 1 -- 172.21.15.26:0/3832865224 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2415db60 con 0x7fcf2406cb20 2023-12-19T16:55:05.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.295+0000 7fcf29333640 1 --2- 172.21.15.26:0/3832865224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf2406cb20 0x7fcf2406f250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009870 tx=0x7fcf1401c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=effa2fa2e4b5f669 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:05.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.295+0000 7fcf28b32640 1 -- 172.21.15.26:0/3832865224 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf140aa670 con 0x7fcf2406cb20 2023-12-19T16:55:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.295+0000 7fcf28b32640 1 -- 172.21.15.26:0/3832865224 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf140aa7d0 con 0x7fcf2406cb20 2023-12-19T16:55:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.295+0000 7fcf28b32640 1 -- 172.21.15.26:0/3832865224 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf140aab60 con 0x7fcf2406cb20 2023-12-19T16:55:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.295+0000 7fcf2bd98640 1 -- 172.21.15.26:0/3832865224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf2406cb20 msgr2=0x7fcf2406f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.295+0000 7fcf2bd98640 1 --2- 172.21.15.26:0/3832865224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf2406cb20 0x7fcf2406f250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcf14009870 tx=0x7fcf1401c510 comp rx=0 tx=0).stop 2023-12-19T16:55:05.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.295+0000 7fcf2bd98640 1 -- 172.21.15.26:0/3832865224 shutdown_connections 2023-12-19T16:55:05.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.295+0000 7fcf2bd98640 1 --2- 172.21.15.26:0/3832865224 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf2415a3c0 0x7fcf2415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.295+0000 7fcf2bd98640 1 --2- 172.21.15.26:0/3832865224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf2406cb20 0x7fcf2406f250 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.295+0000 7fcf2bd98640 1 -- 172.21.15.26:0/3832865224 >> 172.21.15.26:0/3832865224 conn(0x7fcf240660d0 msgr2=0x7fcf240ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:05.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.295+0000 7fcf2bd98640 1 -- 172.21.15.26:0/3832865224 shutdown_connections 2023-12-19T16:55:05.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.295+0000 7fcf2bd98640 1 -- 172.21.15.26:0/3832865224 wait complete. 2023-12-19T16:55:05.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf2bd98640 1 Processor -- start 2023-12-19T16:55:05.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf2bd98640 1 -- start start 2023-12-19T16:55:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf2bd98640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf2406c740 0x7fcf240c1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf2bd98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf2406cb20 0x7fcf240c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf2bd98640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf2415a3c0 0x7fcf2415e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf2bd98640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf240c1850 con 0x7fcf2406cb20 2023-12-19T16:55:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf2bd98640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf240c7d00 con 0x7fcf2415a3c0 2023-12-19T16:55:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf2bd98640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf240c7fe0 con 0x7fcf2406c740 2023-12-19T16:55:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf29333640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf2406cb20 0x7fcf240c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf29333640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf2406cb20 0x7fcf240c7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49838/0 (socket says 172.21.15.26:49838) 2023-12-19T16:55:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf29333640 1 -- 172.21.15.26:0/523647215 learned_addr learned my addr 172.21.15.26:0/523647215 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf29b34640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf2406c740 0x7fcf240c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf2a335640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf2415a3c0 0x7fcf2415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf29333640 1 -- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf2406c740 msgr2=0x7fcf240c1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf29333640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf2406c740 0x7fcf240c1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf29333640 1 -- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf2415a3c0 msgr2=0x7fcf2415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf29333640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf2415a3c0 0x7fcf2415e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf29333640 1 -- 172.21.15.26:0/523647215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf140094d0 con 0x7fcf2406cb20 2023-12-19T16:55:05.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf29333640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf2406cb20 0x7fcf240c7230 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcf140ac040 tx=0x7fcf1401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf1a7fc640 1 -- 172.21.15.26:0/523647215 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf140b3800 con 0x7fcf2406cb20 2023-12-19T16:55:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf1a7fc640 1 -- 172.21.15.26:0/523647215 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf140b3960 con 0x7fcf2406cb20 2023-12-19T16:55:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf1a7fc640 1 -- 172.21.15.26:0/523647215 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf140b3d10 con 0x7fcf2406cb20 2023-12-19T16:55:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.299+0000 7fcf2bd98640 1 -- 172.21.15.26:0/523647215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf240c3250 con 0x7fcf2406cb20 2023-12-19T16:55:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.303+0000 7fcf2bd98640 1 -- 172.21.15.26:0/523647215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2414c5d0 con 0x7fcf2406cb20 2023-12-19T16:55:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.303+0000 7fcf2bd98640 1 -- 172.21.15.26:0/523647215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf240c3090 con 0x7fcf2406cb20 2023-12-19T16:55:05.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.303+0000 7fcf1a7fc640 1 -- 172.21.15.26:0/523647215 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fcf140b2380 con 0x7fcf2406cb20 2023-12-19T16:55:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.303+0000 7fcf1a7fc640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcf08065b80 0x7fcf08068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.303+0000 7fcf1a7fc640 1 -- 172.21.15.26:0/523647215 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fcf1411df70 con 0x7fcf2406cb20 2023-12-19T16:55:05.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.303+0000 7fcf29b34640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcf08065b80 0x7fcf08068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.303+0000 7fcf29b34640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcf08065b80 0x7fcf08068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcf20000f70 tx=0x7fcf20009040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:05.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.307+0000 7fcf1a7fc640 1 -- 172.21.15.26:0/523647215 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcf140eb4d0 con 0x7fcf2406cb20 2023-12-19T16:55:05.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.523+0000 7fcf2bd98640 1 -- 172.21.15.26:0/523647215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcf24074270 con 0x7fcf2406cb20 2023-12-19T16:55:05.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.531+0000 7fcf1a7fc640 1 -- 172.21.15.26:0/523647215 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7fcf140eeaf0 con 0x7fcf2406cb20 2023-12-19T16:55:05.534 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2023-12-19T16:55:05.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.531+0000 7fcf2bd98640 1 -- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcf08065b80 msgr2=0x7fcf08068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.531+0000 7fcf2bd98640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcf08065b80 0x7fcf08068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcf20000f70 tx=0x7fcf20009040 comp rx=0 tx=0).stop 2023-12-19T16:55:05.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.531+0000 7fcf2bd98640 1 -- 172.21.15.26:0/523647215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf2406cb20 msgr2=0x7fcf240c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.531+0000 7fcf2bd98640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf2406cb20 0x7fcf240c7230 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcf140ac040 tx=0x7fcf1401ad50 comp rx=0 tx=0).stop 2023-12-19T16:55:05.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.531+0000 7fcf2bd98640 1 -- 172.21.15.26:0/523647215 shutdown_connections 2023-12-19T16:55:05.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.531+0000 7fcf2bd98640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf2415a3c0 0x7fcf2415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.531+0000 7fcf2bd98640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf2406cb20 0x7fcf240c7230 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.531+0000 7fcf2bd98640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcf08065b80 0x7fcf08068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.531+0000 7fcf2bd98640 1 --2- 172.21.15.26:0/523647215 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf2406c740 0x7fcf240c1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.531+0000 7fcf2bd98640 1 -- 172.21.15.26:0/523647215 >> 172.21.15.26:0/523647215 conn(0x7fcf240660d0 msgr2=0x7fcf240cb090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:05.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.531+0000 7fcf2bd98640 1 -- 172.21.15.26:0/523647215 shutdown_connections 2023-12-19T16:55:05.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.531+0000 7fcf2bd98640 1 -- 172.21.15.26:0/523647215 wait complete. 2023-12-19T16:55:05.555 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:55:05.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.695+0000 7fcea8525640 1 Processor -- start 2023-12-19T16:55:05.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.695+0000 7fcea8525640 1 -- start start 2023-12-19T16:55:05.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea8525640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea015f370 0x7fcea015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea8525640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea006c970 con 0x7fcea00ce740 2023-12-19T16:55:05.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea8525640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea015b1f0 con 0x7fcea0155030 2023-12-19T16:55:05.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea8525640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea006c790 con 0x7fcea015f370 2023-12-19T16:55:05.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea6ac2640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcea00ce740 0x7fcea00ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47728/0 (socket says 172.21.15.26:47728) 2023-12-19T16:55:05.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea6ac2640 1 -- 172.21.15.26:0/1363511693 learned_addr learned my addr 172.21.15.26:0/1363511693 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea5ac0640 1 --2- 172.21.15.26:0/1363511693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea015f370 0x7fcea015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142223049 0 0) 0x7fcea006c970 con 0x7fcea00ce740 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce8c003530 con 0x7fcea00ce740 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3807568212 0 0) 0x7fce8c003530 con 0x7fcea00ce740 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea015f370 msgr2=0x7fcea015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 --2- 172.21.15.26:0/1363511693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea015f370 0x7fcea015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 >> v1:172.21.15.160:6789/0 conn(0x7fcea0155030 legacy=0x7fcea006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea015b4d0 con 0x7fcea00ce740 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fce98001a00 con 0x7fcea00ce740 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 >> v1:172.21.15.26:6789/0 conn(0x7fcea00ce740 legacy=0x7fcea00ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 --2- 172.21.15.26:0/1363511693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c0039b0 0x7fce8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 --2- 172.21.15.26:0/1363511693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce8c0047d0 0x7fce8c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 --2- 172.21.15.26:0/1363511693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea015f370 0x7fce8c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea006c790 con 0x7fce8c0039b0 2023-12-19T16:55:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c00ba40 con 0x7fce8c0047d0 2023-12-19T16:55:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c00bbd0 con 0x7fcea015f370 2023-12-19T16:55:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea62c1640 1 --2- 172.21.15.26:0/1363511693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c0039b0 0x7fce8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea5ac0640 1 --2- 172.21.15.26:0/1363511693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce8c0047d0 0x7fce8c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea6ac2640 1 --2- 172.21.15.26:0/1363511693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea015f370 0x7fce8c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea62c1640 1 -- 172.21.15.26:0/1363511693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea015f370 msgr2=0x7fce8c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea62c1640 1 --2- 172.21.15.26:0/1363511693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea015f370 0x7fce8c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea62c1640 1 -- 172.21.15.26:0/1363511693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce8c0047d0 msgr2=0x7fce8c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea62c1640 1 --2- 172.21.15.26:0/1363511693 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce8c0047d0 0x7fce8c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea62c1640 1 -- 172.21.15.26:0/1363511693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c003530 con 0x7fce8c0039b0 2023-12-19T16:55:05.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.699+0000 7fcea6ac2640 1 --2- 172.21.15.26:0/1363511693 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea015f370 0x7fce8c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:55:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.703+0000 7fcea62c1640 1 --2- 172.21.15.26:0/1363511693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c0039b0 0x7fce8c003d90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fce9c007fd0 tx=0x7fce9c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d1bdeff0d36cc0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.703+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0ab600 con 0x7fce8c0039b0 2023-12-19T16:55:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.703+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c0a7050 con 0x7fce8c0039b0 2023-12-19T16:55:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.703+0000 7fcea52bf640 1 -- 172.21.15.26:0/1363511693 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0ab9a0 con 0x7fce8c0039b0 2023-12-19T16:55:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.703+0000 7fcea8525640 1 -- 172.21.15.26:0/1363511693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c0039b0 msgr2=0x7fce8c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.703+0000 7fcea8525640 1 --2- 172.21.15.26:0/1363511693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c0039b0 0x7fce8c003d90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fce9c007fd0 tx=0x7fce9c01c470 comp rx=0 tx=0).stop 2023-12-19T16:55:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.703+0000 7fcea6ac2640 1 -- 172.21.15.26:0/1363511693 reap_dead start 2023-12-19T16:55:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.703+0000 7fcea8525640 1 -- 172.21.15.26:0/1363511693 shutdown_connections 2023-12-19T16:55:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.703+0000 7fcea8525640 1 -- 172.21.15.26:0/1363511693 >> 172.21.15.26:0/1363511693 conn(0x7fcea00658e0 msgr2=0x7fcea0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.703+0000 7fcea8525640 1 -- 172.21.15.26:0/1363511693 shutdown_connections 2023-12-19T16:55:05.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.703+0000 7fcea8525640 1 -- 172.21.15.26:0/1363511693 wait complete. 2023-12-19T16:55:05.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.703+0000 7fcea8525640 1 Processor -- start 2023-12-19T16:55:05.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea8525640 1 -- start start 2023-12-19T16:55:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea8525640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c0039b0 0x7fcea01f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea8525640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea0150040 0x7fcea01ea140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea8525640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea0161200 0x7fcea01616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea8525640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea01f5bb0 con 0x7fce8c0039b0 2023-12-19T16:55:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea8525640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea0161bf0 con 0x7fcea0150040 2023-12-19T16:55:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea8525640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea01eb300 con 0x7fcea0161200 2023-12-19T16:55:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea62c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c0039b0 0x7fcea01f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea62c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c0039b0 0x7fcea01f51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49854/0 (socket says 172.21.15.26:49854) 2023-12-19T16:55:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea62c1640 1 -- 172.21.15.26:0/1229065664 learned_addr learned my addr 172.21.15.26:0/1229065664 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea5ac0640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea0150040 0x7fcea01ea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea6ac2640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea0161200 0x7fcea01616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea62c1640 1 -- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea0161200 msgr2=0x7fcea01616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea62c1640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea0161200 0x7fcea01616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea62c1640 1 -- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea0150040 msgr2=0x7fcea01ea140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea62c1640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea0150040 0x7fcea01ea140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea62c1640 1 -- 172.21.15.26:0/1229065664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea01eaa60 con 0x7fce8c0039b0 2023-12-19T16:55:05.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea5ac0640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea0150040 0x7fcea01ea140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:55:05.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea62c1640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c0039b0 0x7fcea01f51a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0ae040 tx=0x7fce8c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:05.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fce96ffd640 1 -- 172.21.15.26:0/1229065664 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0053d0 con 0x7fce8c0039b0 2023-12-19T16:55:05.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fce96ffd640 1 -- 172.21.15.26:0/1229065664 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c0aba80 con 0x7fce8c0039b0 2023-12-19T16:55:05.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fce96ffd640 1 -- 172.21.15.26:0/1229065664 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0abdf0 con 0x7fce8c0039b0 2023-12-19T16:55:05.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea8525640 1 -- 172.21.15.26:0/1229065664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea01ead40 con 0x7fce8c0039b0 2023-12-19T16:55:05.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea8525640 1 -- 172.21.15.26:0/1229065664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea01f56e0 con 0x7fce8c0039b0 2023-12-19T16:55:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.707+0000 7fcea8525640 1 -- 172.21.15.26:0/1229065664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea01f5900 con 0x7fce8c0039b0 2023-12-19T16:55:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.711+0000 7fce96ffd640 1 -- 172.21.15.26:0/1229065664 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fce9c0b2040 con 0x7fce8c0039b0 2023-12-19T16:55:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.711+0000 7fce96ffd640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fce84065b80 0x7fce84068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.711+0000 7fce96ffd640 1 -- 172.21.15.26:0/1229065664 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fce9c11c9a0 con 0x7fce8c0039b0 2023-12-19T16:55:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.711+0000 7fcea5ac0640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fce84065b80 0x7fce84068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:05.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.711+0000 7fcea5ac0640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fce84065b80 0x7fce84068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcea00cb140 tx=0x7fce90008040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:05.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.715+0000 7fce96ffd640 1 -- 172.21.15.26:0/1229065664 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fce9c0ea020 con 0x7fce8c0039b0 2023-12-19T16:55:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.903+0000 7fcea8525640 1 -- 172.21.15.26:0/1229065664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcea00cef80 con 0x7fce8c0039b0 2023-12-19T16:55:05.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.903+0000 7fce96ffd640 1 -- 172.21.15.26:0/1229065664 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fce9c0ed640 con 0x7fce8c0039b0 2023-12-19T16:55:05.907 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:55:05.907 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1947010592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1947010592},{"type":"v1","addr":"172.21.15.26:6833","nonce":1947010592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2764469548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2764469548},{"type":"v1","addr":"172.21.15.160:6837","nonce":2764469548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5803,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/255772664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":255772664},{"type":"v1","addr":"172.21.15.160:6836","nonce":255772664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:55:00.584757+0000","modified":"2023-12-19T16:55:03.604713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6000":{"gid":6000,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1847960813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1847960813},{"type":"v1","addr":"172.21.15.26:6835","nonce":1847960813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T16:55:05.908 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2023-12-19T16:55:05.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.903+0000 7fcea8525640 1 -- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fce84065b80 msgr2=0x7fce84068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.903+0000 7fcea8525640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fce84065b80 0x7fce84068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcea00cb140 tx=0x7fce90008040 comp rx=0 tx=0).stop 2023-12-19T16:55:05.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.903+0000 7fcea8525640 1 -- 172.21.15.26:0/1229065664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c0039b0 msgr2=0x7fcea01f51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:05.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.903+0000 7fcea8525640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c0039b0 0x7fcea01f51a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0ae040 tx=0x7fce8c005800 comp rx=0 tx=0).stop 2023-12-19T16:55:05.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.907+0000 7fcea8525640 1 -- 172.21.15.26:0/1229065664 shutdown_connections 2023-12-19T16:55:05.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.907+0000 7fcea8525640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fce84065b80 0x7fce84068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.907+0000 7fcea8525640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcea0161200 0x7fcea01616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.907+0000 7fcea8525640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcea0150040 0x7fcea01ea140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.907+0000 7fcea8525640 1 --2- 172.21.15.26:0/1229065664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce8c0039b0 0x7fcea01f51a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.907+0000 7fcea8525640 1 -- 172.21.15.26:0/1229065664 >> 172.21.15.26:0/1229065664 conn(0x7fcea00658e0 msgr2=0x7fcea015dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.907+0000 7fcea8525640 1 -- 172.21.15.26:0/1229065664 shutdown_connections 2023-12-19T16:55:05.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:05.907+0000 7fcea8525640 1 -- 172.21.15.26:0/1229065664 wait complete. 2023-12-19T16:55:06.167 INFO:tasks.ceph.mds.c.smithi026.stderr:2023-12-19T16:55:06.167+0000 7ff59fcd4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T16:55:06.331 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T16:55:00.584757+0000', 'modified': '2023-12-19T16:55:03.604713+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6000}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6000': {'gid': 6000, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/1847960813', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1847960813}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1847960813}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T16:55:06.332 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T16:55:06.332 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-19T16:55:06.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.475+0000 7f07f3d6d640 1 Processor -- start 2023-12-19T16:55:06.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.475+0000 7f07f3d6d640 1 -- start start 2023-12-19T16:55:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f3d6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07ec06cb30 0x7f07ec0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f3d6d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f07ec15d2b0 0x7f07ec15d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f3d6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ec15ddc0 con 0x7f07ec0b2dd0 2023-12-19T16:55:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f3d6d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ec15e0a0 con 0x7f07ec06cb30 2023-12-19T16:55:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f3d6d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ec0cb450 con 0x7f07ec15d2b0 2023-12-19T16:55:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f1b09640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07ec06cb30 0x7f07ec0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f230a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f07ec0b2dd0 0x7f07ec0b3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47516/0 (socket says 172.21.15.26:47516) 2023-12-19T16:55:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f230a640 1 -- 172.21.15.26:0/2472654239 learned_addr learned my addr 172.21.15.26:0/2472654239 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:55:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f1308640 1 --2- 172.21.15.26:0/2472654239 >> v2:172.21.15.160:3300/0 conn(0x7f07ec15d2b0 0x7f07ec15d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f0b07640 1 -- 172.21.15.26:0/2472654239 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005224472 0 0) 0x7f07ec15ddc0 con 0x7f07ec0b2dd0 2023-12-19T16:55:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f1b09640 1 -- 172.21.15.26:0/2472654239 >> v2:172.21.15.160:3300/0 conn(0x7f07ec15d2b0 msgr2=0x7f07ec15d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f1b09640 1 --2- 172.21.15.26:0/2472654239 >> v2:172.21.15.160:3300/0 conn(0x7f07ec15d2b0 0x7f07ec15d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f1b09640 1 -- 172.21.15.26:0/2472654239 >> v1:172.21.15.26:6789/0 conn(0x7f07ec0b2dd0 legacy=0x7f07ec0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f1b09640 1 -- 172.21.15.26:0/2472654239 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec0cb730 con 0x7f07ec06cb30 2023-12-19T16:55:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f1308640 1 --2- 172.21.15.26:0/2472654239 >> v2:172.21.15.160:3300/0 conn(0x7f07ec15d2b0 0x7f07ec15d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:55:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.479+0000 7f07f1b09640 1 --2- 172.21.15.26:0/2472654239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07ec06cb30 0x7f07ec0cbc50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0667b0 tx=0x7f07e801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be21bfbb3ae12eeb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f0b07640 1 -- 172.21.15.26:0/2472654239 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e80ab5f0 con 0x7f07ec06cb30 2023-12-19T16:55:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f0b07640 1 -- 172.21.15.26:0/2472654239 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e80ab750 con 0x7f07ec06cb30 2023-12-19T16:55:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f0b07640 1 -- 172.21.15.26:0/2472654239 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e80abae0 con 0x7f07ec06cb30 2023-12-19T16:55:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 -- 172.21.15.26:0/2472654239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07ec06cb30 msgr2=0x7f07ec0cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 --2- 172.21.15.26:0/2472654239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07ec06cb30 0x7f07ec0cbc50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0667b0 tx=0x7f07e801c460 comp rx=0 tx=0).stop 2023-12-19T16:55:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 -- 172.21.15.26:0/2472654239 shutdown_connections 2023-12-19T16:55:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 --2- 172.21.15.26:0/2472654239 >> v2:172.21.15.160:3300/0 conn(0x7f07ec15d2b0 0x7f07ec15d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 --2- 172.21.15.26:0/2472654239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07ec06cb30 0x7f07ec0cbc50 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 -- 172.21.15.26:0/2472654239 >> 172.21.15.26:0/2472654239 conn(0x7f07ec0658e0 msgr2=0x7f07ec065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 -- 172.21.15.26:0/2472654239 shutdown_connections 2023-12-19T16:55:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 -- 172.21.15.26:0/2472654239 wait complete. 2023-12-19T16:55:06.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 Processor -- start 2023-12-19T16:55:06.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 -- start start 2023-12-19T16:55:06.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07ec06cb30 0x7f07ec1f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07ec0b2dd0 0x7f07ec145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07ec15d2b0 0x7f07ec06f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec1446a0 con 0x7f07ec0b2dd0 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec1f81d0 con 0x7f07ec06cb30 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f3d6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ec143a20 con 0x7f07ec15d2b0 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.483+0000 7f07f1308640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07ec0b2dd0 0x7f07ec145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07f1308640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07ec0b2dd0 0x7f07ec145410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57410/0 (socket says 172.21.15.26:57410) 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07f1308640 1 -- 172.21.15.26:0/2983322728 learned_addr learned my addr 172.21.15.26:0/2983322728 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07f1b09640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07ec06cb30 0x7f07ec1f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07f230a640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07ec15d2b0 0x7f07ec06f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07f1308640 1 -- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07ec15d2b0 msgr2=0x7f07ec06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07f1308640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07ec15d2b0 0x7f07ec06f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07f1308640 1 -- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07ec06cb30 msgr2=0x7f07ec1f7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07f1308640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07ec06cb30 0x7f07ec1f7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07f1308640 1 -- 172.21.15.26:0/2983322728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e8009420 con 0x7f07ec0b2dd0 2023-12-19T16:55:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07f1308640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07ec0b2dd0 0x7f07ec145410 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00d800 tx=0x7f07dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07e27fc640 1 -- 172.21.15.26:0/2983322728 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc00b6c0 con 0x7f07ec0b2dd0 2023-12-19T16:55:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07e27fc640 1 -- 172.21.15.26:0/2983322728 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc00b820 con 0x7f07ec0b2dd0 2023-12-19T16:55:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07e27fc640 1 -- 172.21.15.26:0/2983322728 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc00bbd0 con 0x7f07ec0b2dd0 2023-12-19T16:55:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07f3d6d640 1 -- 172.21.15.26:0/2983322728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ec143d00 con 0x7f07ec0b2dd0 2023-12-19T16:55:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07f3d6d640 1 -- 172.21.15.26:0/2983322728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ec1442c0 con 0x7f07ec0b2dd0 2023-12-19T16:55:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.487+0000 7f07e27fc640 1 -- 172.21.15.26:0/2983322728 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f07dc00bd30 con 0x7f07ec0b2dd0 2023-12-19T16:55:06.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07e27fc640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f07d0065b80 0x7f07d0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07e27fc640 1 -- 172.21.15.26:0/2983322728 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f07dc083a50 con 0x7f07ec0b2dd0 2023-12-19T16:55:06.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 Processor -- start 2023-12-19T16:55:06.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f1b09640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f07d0065b80 0x7f07d0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 -- start start 2023-12-19T16:55:06.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f1b09640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f07d0065b80 0x7f07d0068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f07e80af000 tx=0x7f07e80a7000 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:06.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628ad070 0x561e628ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 0x561e62862c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 0x561e628635a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e62686c90 con 0x561e628631c0 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e626b8220 con 0x561e628ad990 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e626b57f0 con 0x561e628ad070 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f230a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 0x561e628635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f230a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 0x561e628635a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57416/0 (socket says 172.21.15.26:57416) 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f230a640 1 -- 172.21.15.26:0/2070648292 learned_addr learned my addr 172.21.15.26:0/2070648292 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f1308640 1 --2- 172.21.15.26:0/2070648292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 0x561e62862c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f1b09640 1 --2- 172.21.15.26:0/2070648292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628ad070 0x561e628ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f230a640 1 -- 172.21.15.26:0/2070648292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628ad070 msgr2=0x561e628ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f230a640 1 --2- 172.21.15.26:0/2070648292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628ad070 0x561e628ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f230a640 1 -- 172.21.15.26:0/2070648292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 msgr2=0x561e62862c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f230a640 1 --2- 172.21.15.26:0/2070648292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 0x561e62862c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f230a640 1 -- 172.21.15.26:0/2070648292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e623ad890 con 0x561e628631c0 2023-12-19T16:55:06.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f230a640 1 --2- 172.21.15.26:0/2070648292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 0x561e628635a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f07e40072d0 tx=0x7f07e4007790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf49d7e6b4e979a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:06.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07cf7fe640 1 -- 172.21.15.26:0/2070648292 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e40079e0 con 0x561e628631c0 2023-12-19T16:55:06.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07cf7fe640 1 -- 172.21.15.26:0/2070648292 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e4001e30 con 0x561e628631c0 2023-12-19T16:55:06.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07cf7fe640 1 -- 172.21.15.26:0/2070648292 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4014480 con 0x561e628631c0 2023-12-19T16:55:06.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 -- 172.21.15.26:0/2070648292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 msgr2=0x561e628635a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 --2- 172.21.15.26:0/2070648292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 0x561e628635a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f07e40072d0 tx=0x7f07e4007790 comp rx=0 tx=0).stop 2023-12-19T16:55:06.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 -- 172.21.15.26:0/2070648292 shutdown_connections 2023-12-19T16:55:06.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 --2- 172.21.15.26:0/2070648292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 0x561e628635a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 --2- 172.21.15.26:0/2070648292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 0x561e62862c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 --2- 172.21.15.26:0/2070648292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628ad070 0x561e628ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 -- 172.21.15.26:0/2070648292 >> 172.21.15.26:0/2070648292 conn(0x561e62869950 msgr2=0x561e62861560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:06.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 -- 172.21.15.26:0/2070648292 shutdown_connections 2023-12-19T16:55:06.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.491+0000 7f07f70be000 1 -- 172.21.15.26:0/2070648292 wait complete. 2023-12-19T16:55:06.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 1 -- start start 2023-12-19T16:55:06.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 1 Processor -- start 2023-12-19T16:55:06.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628631c0 0x561e62868720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad070 0x561e6286f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 0x561e6286f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e626b10e0 con 0x561e628ad070 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e62650310 con 0x561e628ad990 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e6266e770 con 0x561e628631c0 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 10 client.0 _ll_register_callbacks cb 0x561e6284eb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f1308640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad070 0x561e6286f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f1308640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad070 0x561e6286f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57418/0 (socket says 172.21.15.26:57418) 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f1308640 1 -- 172.21.15.26:0/2554556646 learned_addr learned my addr 172.21.15.26:0/2554556646 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f1b09640 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628631c0 0x561e62868720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f230a640 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 0x561e6286f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f1308640 1 -- 172.21.15.26:0/2554556646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628631c0 msgr2=0x561e62868720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f1308640 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628631c0 0x561e62868720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f1308640 1 -- 172.21.15.26:0/2554556646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 msgr2=0x561e6286f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f1308640 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 0x561e6286f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f1308640 1 -- 172.21.15.26:0/2554556646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e400f040 con 0x561e628ad070 2023-12-19T16:55:06.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f1b09640 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628631c0 0x561e62868720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:55:06.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f230a640 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 0x561e6286f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:55:06.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f1308640 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad070 0x561e6286f080 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f07dc081b70 tx=0x7f07dc055510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f650260fb6a8ff6b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:06.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T16:55:06.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07ccff9640 1 -- 172.21.15.26:0/2554556646 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc05ccb0 con 0x561e628ad070 2023-12-19T16:55:06.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 1 -- 172.21.15.26:0/2554556646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e62680c10 con 0x561e628ad070 2023-12-19T16:55:06.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07ccff9640 1 -- 172.21.15.26:0/2554556646 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f07dc06c070 con 0x561e628ad070 2023-12-19T16:55:06.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07ccff9640 1 -- 172.21.15.26:0/2554556646 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07dc06c230 con 0x561e628ad070 2023-12-19T16:55:06.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 10 client.6132 fetch_fsmap learned FSMap version 25 2023-12-19T16:55:06.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 1 -- 172.21.15.26:0/2554556646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x561e62676260 con 0x561e628ad070 2023-12-19T16:55:06.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07ccff9640 1 -- 172.21.15.26:0/2554556646 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e62680c10 con 0x561e628ad070 2023-12-19T16:55:06.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07ccff9640 1 -- 172.21.15.26:0/2554556646 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f07dc05c820 con 0x561e628ad070 2023-12-19T16:55:06.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 10 client.6132 fetch_fsmap finished waiting for FSMap version 25 2023-12-19T16:55:06.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 10 client.6132 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T16:55:06.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 20 client.6132 populate_metadata read hostname 'smithi026' 2023-12-19T16:55:06.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.26:6834/1847960813,v1:172.21.15.26:6835/1847960813] conn(0x561e628b48d0 0x561e628b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 4 client.6132 mds_command: new command op to 6000 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T16:55:06.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f70be000 1 -- 172.21.15.26:0/2554556646 --> [v2:172.21.15.26:6834/1847960813,v1:172.21.15.26:6835/1847960813] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e626afa30 con 0x561e628b48d0 2023-12-19T16:55:06.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.495+0000 7f07f1b09640 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.26:6834/1847960813,v1:172.21.15.26:6835/1847960813] conn(0x561e628b48d0 0x561e628b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07ccff9640 10 client.6132 ms_handle_connect on v2:172.21.15.26:6834/1847960813 2023-12-19T16:55:06.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07f1b09640 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.26:6834/1847960813,v1:172.21.15.26:6835/1847960813] conn(0x561e628b48d0 0x561e628b4cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18c30991f646d541 server_cookie=308fd0b1a15ddceb in_seq=0 out_seq=0 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07ccff9640 1 -- 172.21.15.26:0/2554556646 <== mds.0 v2:172.21.15.26:6834/1847960813 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561e626afa30 con 0x561e628b48d0 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07ccff9640 10 client.6132 handle_command_reply: tid=0 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07f70be000 1 client.6132 shutdown 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07f70be000 1 -- 172.21.15.26:0/2554556646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad070 msgr2=0x561e6286f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07f70be000 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad070 0x561e6286f080 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f07dc081b70 tx=0x7f07dc055510 comp rx=0 tx=0).stop 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07f70be000 1 -- 172.21.15.26:0/2554556646 shutdown_connections 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07f70be000 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.26:6834/1847960813,v1:172.21.15.26:6835/1847960813] conn(0x561e628b48d0 0x561e628b4cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07f70be000 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 0x561e6286f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07f70be000 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad070 0x561e6286f080 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07f70be000 1 --2- 172.21.15.26:0/2554556646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628631c0 0x561e62868720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07ccff9640 0 client.6132 ms_handle_reset on v2:172.21.15.26:6834/1847960813 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07f70be000 1 -- 172.21.15.26:0/2554556646 >> 172.21.15.26:0/2554556646 conn(0x561e62869950 msgr2=0x561e62861c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07f70be000 1 -- 172.21.15.26:0/2554556646 shutdown_connections 2023-12-19T16:55:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07f70be000 1 -- 172.21.15.26:0/2554556646 wait complete. 2023-12-19T16:55:06.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.499+0000 7f07f70be000 20 client.6132 trim_cache size 0 max 16384 2023-12-19T16:55:06.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 Processor -- start 2023-12-19T16:55:06.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 -- start start 2023-12-19T16:55:06.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628631c0 0x561e62904ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad070 0x561e62905410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad990 0x561e62905950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6236e940 con 0x561e628ad990 2023-12-19T16:55:06.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e62659860 con 0x561e628ad070 2023-12-19T16:55:06.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e62781b30 con 0x561e628631c0 2023-12-19T16:55:06.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f230a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad990 0x561e62905950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f230a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad990 0x561e62905950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57420/0 (socket says 172.21.15.26:57420) 2023-12-19T16:55:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f230a640 1 -- 172.21.15.26:0/3229847658 learned_addr learned my addr 172.21.15.26:0/3229847658 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f1308640 1 --2- 172.21.15.26:0/3229847658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad070 0x561e62905410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f1b09640 1 --2- 172.21.15.26:0/3229847658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628631c0 0x561e62904ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f230a640 1 -- 172.21.15.26:0/3229847658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628631c0 msgr2=0x561e62904ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f230a640 1 --2- 172.21.15.26:0/3229847658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628631c0 0x561e62904ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f230a640 1 -- 172.21.15.26:0/3229847658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad070 msgr2=0x561e62905410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f230a640 1 --2- 172.21.15.26:0/3229847658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad070 0x561e62905410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f230a640 1 -- 172.21.15.26:0/3229847658 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e6265c3b0 con 0x561e628ad990 2023-12-19T16:55:06.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f1b09640 1 --2- 172.21.15.26:0/3229847658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628631c0 0x561e62904ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:55:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f230a640 1 --2- 172.21.15.26:0/3229847658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad990 0x561e62905950 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f07e40077d0 tx=0x7f07e400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd171ad2fb6a8924 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07cffff640 1 -- 172.21.15.26:0/3229847658 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4001e30 con 0x561e628ad990 2023-12-19T16:55:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07cffff640 1 -- 172.21.15.26:0/3229847658 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e400b040 con 0x561e628ad990 2023-12-19T16:55:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07cffff640 1 -- 172.21.15.26:0/3229847658 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e4004b70 con 0x561e628ad990 2023-12-19T16:55:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 -- 172.21.15.26:0/3229847658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad990 msgr2=0x561e62905950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 --2- 172.21.15.26:0/3229847658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad990 0x561e62905950 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f07e40077d0 tx=0x7f07e400c000 comp rx=0 tx=0).stop 2023-12-19T16:55:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 -- 172.21.15.26:0/3229847658 shutdown_connections 2023-12-19T16:55:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 --2- 172.21.15.26:0/3229847658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628ad990 0x561e62905950 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 --2- 172.21.15.26:0/3229847658 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad070 0x561e62905410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 --2- 172.21.15.26:0/3229847658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628631c0 0x561e62904ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 -- 172.21.15.26:0/3229847658 >> 172.21.15.26:0/3229847658 conn(0x561e62869950 msgr2=0x561e6286ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 -- 172.21.15.26:0/3229847658 shutdown_connections 2023-12-19T16:55:06.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 -- 172.21.15.26:0/3229847658 wait complete. 2023-12-19T16:55:06.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.515+0000 7f07f70be000 1 -- start start 2023-12-19T16:55:06.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 1 Processor -- start 2023-12-19T16:55:06.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 0x561e6290a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628ad070 0x561e6290aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 0x561e628edd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e626f4ce0 con 0x561e628631c0 2023-12-19T16:55:06.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e62657e10 con 0x561e628ad990 2023-12-19T16:55:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e6267ea80 con 0x561e628ad070 2023-12-19T16:55:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 10 client.0 _ll_register_callbacks cb 0x561e6265f640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T16:55:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f1b09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 0x561e6290a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f1b09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 0x561e6290a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57422/0 (socket says 172.21.15.26:57422) 2023-12-19T16:55:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f1b09640 1 -- 172.21.15.26:0/374259875 learned_addr learned my addr 172.21.15.26:0/374259875 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f230a640 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 0x561e628edd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f1308640 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628ad070 0x561e6290aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f1b09640 1 -- 172.21.15.26:0/374259875 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628ad070 msgr2=0x561e6290aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f1b09640 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628ad070 0x561e6290aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f1b09640 1 -- 172.21.15.26:0/374259875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 msgr2=0x561e628edd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f1b09640 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 0x561e628edd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f1b09640 1 -- 172.21.15.26:0/374259875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e623ad890 con 0x561e628631c0 2023-12-19T16:55:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f1308640 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628ad070 0x561e6290aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:55:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f1b09640 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 0x561e6290a520 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f07e80ab9a0 tx=0x7f07e80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336139021a473894 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T16:55:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07ccff9640 1 -- 172.21.15.26:0/374259875 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e80c7040 con 0x561e628631c0 2023-12-19T16:55:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07ccff9640 1 -- 172.21.15.26:0/374259875 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f07e80a8050 con 0x561e628631c0 2023-12-19T16:55:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 1 -- 172.21.15.26:0/374259875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e62676e00 con 0x561e628631c0 2023-12-19T16:55:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07ccff9640 1 -- 172.21.15.26:0/374259875 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07e80abe80 con 0x561e628631c0 2023-12-19T16:55:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 10 client.6138 fetch_fsmap learned FSMap version 25 2023-12-19T16:55:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 1 -- 172.21.15.26:0/374259875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x561e623adcc0 con 0x561e628631c0 2023-12-19T16:55:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07ccff9640 1 -- 172.21.15.26:0/374259875 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e62676e00 con 0x561e628631c0 2023-12-19T16:55:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07ccff9640 1 -- 172.21.15.26:0/374259875 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f07e80aa070 con 0x561e628631c0 2023-12-19T16:55:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 10 client.6138 fetch_fsmap finished waiting for FSMap version 25 2023-12-19T16:55:06.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 10 client.6138 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T16:55:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 20 client.6138 populate_metadata read hostname 'smithi026' 2023-12-19T16:55:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.26:6834/1847960813,v1:172.21.15.26:6835/1847960813] conn(0x561e628ee2a0 0x561e628ee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 4 client.6138 mds_command: new command op to 6000 tid=0[{"prefix": "status"}] 2023-12-19T16:55:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f70be000 1 -- 172.21.15.26:0/374259875 --> [v2:172.21.15.26:6834/1847960813,v1:172.21.15.26:6835/1847960813] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561e626aa160 con 0x561e628ee2a0 2023-12-19T16:55:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f1308640 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.26:6834/1847960813,v1:172.21.15.26:6835/1847960813] conn(0x561e628ee2a0 0x561e628ee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07f1308640 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.26:6834/1847960813,v1:172.21.15.26:6835/1847960813] conn(0x561e628ee2a0 0x561e628ee680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6702c9fbb42bc71b server_cookie=ce6ae0fddc0a67a5 in_seq=0 out_seq=0 2023-12-19T16:55:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.519+0000 7f07ccff9640 10 client.6138 ms_handle_connect on v2:172.21.15.26:6834/1847960813 2023-12-19T16:55:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07ccff9640 1 -- 172.21.15.26:0/374259875 <== mds.0 v2:172.21.15.26:6834/1847960813 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x561e626aa160 con 0x561e628ee2a0 2023-12-19T16:55:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07ccff9640 10 client.6138 handle_command_reply: tid=0 2023-12-19T16:55:06.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f70be000 1 client.6138 shutdown 2023-12-19T16:55:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f70be000 1 -- 172.21.15.26:0/374259875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 msgr2=0x561e6290a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f70be000 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 0x561e6290a520 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f07e80ab9a0 tx=0x7f07e80046d0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f70be000 1 -- 172.21.15.26:0/374259875 shutdown_connections 2023-12-19T16:55:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f70be000 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.26:6834/1847960813,v1:172.21.15.26:6835/1847960813] conn(0x561e628ee2a0 0x561e628ee680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f70be000 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561e628ad990 0x561e628edd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f70be000 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x561e628ad070 0x561e6290aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f70be000 1 --2- 172.21.15.26:0/374259875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561e628631c0 0x561e6290a520 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07ccff9640 0 client.6138 ms_handle_reset on v2:172.21.15.26:6834/1847960813 2023-12-19T16:55:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f70be000 1 -- 172.21.15.26:0/374259875 >> 172.21.15.26:0/374259875 conn(0x561e62869950 msgr2=0x561e628aa060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f70be000 1 -- 172.21.15.26:0/374259875 shutdown_connections 2023-12-19T16:55:06.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f70be000 1 -- 172.21.15.26:0/374259875 wait complete. 2023-12-19T16:55:06.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f70be000 20 client.6138 trim_cache size 0 max 16384 2023-12-19T16:55:06.531 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T16:55:06.531 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T16:55:06.531 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T16:55:06.531 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6000, 2023-12-19T16:55:06.532 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T16:55:06.532 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T16:55:06.532 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T16:55:06.532 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.9368401210000004, 2023-12-19T16:55:06.532 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 25, 2023-12-19T16:55:06.532 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 61, 2023-12-19T16:55:06.532 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 61, 2023-12-19T16:55:06.532 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 11.137556629000001 2023-12-19T16:55:06.532 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T16:55:06.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f3d6d640 1 -- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f07d0065b80 msgr2=0x7f07d0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f3d6d640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f07d0065b80 0x7f07d0068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f07e80af000 tx=0x7f07e80a7000 comp rx=0 tx=0).stop 2023-12-19T16:55:06.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f3d6d640 1 -- 172.21.15.26:0/2983322728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07ec0b2dd0 msgr2=0x7f07ec145410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f3d6d640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07ec0b2dd0 0x7f07ec145410 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00d800 tx=0x7f07dc00dcc0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f3d6d640 1 -- 172.21.15.26:0/2983322728 shutdown_connections 2023-12-19T16:55:06.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f3d6d640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f07d0065b80 0x7f07d0068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f3d6d640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07ec15d2b0 0x7f07ec06f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f3d6d640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07ec0b2dd0 0x7f07ec145410 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f3d6d640 1 --2- 172.21.15.26:0/2983322728 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07ec06cb30 0x7f07ec1f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f3d6d640 1 -- 172.21.15.26:0/2983322728 >> 172.21.15.26:0/2983322728 conn(0x7f07ec0658e0 msgr2=0x7f07ec0cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:06.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f3d6d640 1 -- 172.21.15.26:0/2983322728 shutdown_connections 2023-12-19T16:55:06.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.523+0000 7f07f3d6d640 1 -- 172.21.15.26:0/2983322728 wait complete. 2023-12-19T16:55:06.545 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T16:55:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f6443ef2640 1 Processor -- start 2023-12-19T16:55:06.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f6443ef2640 1 -- start start 2023-12-19T16:55:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f6443ef2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f643c15f370 0x7f643c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f6443ef2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0ce740 0x7f643c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f6443ef2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c06c970 con 0x7f643c155030 2023-12-19T16:55:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f6443ef2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643c06c790 con 0x7f643c15f370 2023-12-19T16:55:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f6443ef2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0cf0f0 con 0x7f643c0ce740 2023-12-19T16:55:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f644148d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f643c15f370 0x7f643c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f644148d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f643c15f370 0x7f643c15bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57430/0 (socket says 172.21.15.26:57430) 2023-12-19T16:55:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f644148d640 1 -- 172.21.15.26:0/3972433753 learned_addr learned my addr 172.21.15.26:0/3972433753 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f644248f640 1 --2- 172.21.15.26:0/3972433753 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0ce740 0x7f643c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f644148d640 1 -- 172.21.15.26:0/3972433753 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0ce740 msgr2=0x7f643c0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f644148d640 1 --2- 172.21.15.26:0/3972433753 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0ce740 0x7f643c0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f644148d640 1 -- 172.21.15.26:0/3972433753 >> v1:172.21.15.160:6789/0 conn(0x7f643c155030 legacy=0x7f643c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.687+0000 7f644148d640 1 -- 172.21.15.26:0/3972433753 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c15b1f0 con 0x7f643c15f370 2023-12-19T16:55:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.691+0000 7f644148d640 1 --2- 172.21.15.26:0/3972433753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f643c15f370 0x7f643c15bcd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f642c009870 tx=0x7f642c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396acdb1c02249a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.691+0000 7f6440c8c640 1 -- 172.21.15.26:0/3972433753 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0aa670 con 0x7f643c15f370 2023-12-19T16:55:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.691+0000 7f6440c8c640 1 -- 172.21.15.26:0/3972433753 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c0aa7d0 con 0x7f643c15f370 2023-12-19T16:55:06.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.691+0000 7f6440c8c640 1 -- 172.21.15.26:0/3972433753 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0aab60 con 0x7f643c15f370 2023-12-19T16:55:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.691+0000 7f6443ef2640 1 -- 172.21.15.26:0/3972433753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f643c15f370 msgr2=0x7f643c15bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.691+0000 7f6443ef2640 1 --2- 172.21.15.26:0/3972433753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f643c15f370 0x7f643c15bcd0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f642c009870 tx=0x7f642c01c510 comp rx=0 tx=0).stop 2023-12-19T16:55:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.691+0000 7f6443ef2640 1 -- 172.21.15.26:0/3972433753 shutdown_connections 2023-12-19T16:55:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.691+0000 7f6443ef2640 1 --2- 172.21.15.26:0/3972433753 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0ce740 0x7f643c0cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.691+0000 7f6443ef2640 1 --2- 172.21.15.26:0/3972433753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f643c15f370 0x7f643c15bcd0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.691+0000 7f6443ef2640 1 -- 172.21.15.26:0/3972433753 >> 172.21.15.26:0/3972433753 conn(0x7f643c0658e0 msgr2=0x7f643c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.691+0000 7f6443ef2640 1 -- 172.21.15.26:0/3972433753 shutdown_connections 2023-12-19T16:55:06.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.691+0000 7f6443ef2640 1 -- 172.21.15.26:0/3972433753 wait complete. 2023-12-19T16:55:06.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f6443ef2640 1 Processor -- start 2023-12-19T16:55:06.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f6443ef2640 1 -- start start 2023-12-19T16:55:06.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f6443ef2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0ce740 0x7f643c0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f6443ef2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f643c155030 0x7f643c0acbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f6443ef2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f643c15f370 0x7f643c14f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f6443ef2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c14ef10 con 0x7f643c15f370 2023-12-19T16:55:06.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f6443ef2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c146100 con 0x7f643c155030 2023-12-19T16:55:06.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f6443ef2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c1463e0 con 0x7f643c0ce740 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f644248f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f643c15f370 0x7f643c14f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f644248f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f643c15f370 0x7f643c14f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57442/0 (socket says 172.21.15.26:57442) 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f644248f640 1 -- 172.21.15.26:0/4232687791 learned_addr learned my addr 172.21.15.26:0/4232687791 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f6441c8e640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0ce740 0x7f643c0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f644148d640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f643c155030 0x7f643c0acbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f644248f640 1 -- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0ce740 msgr2=0x7f643c0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f644248f640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0ce740 0x7f643c0b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f644248f640 1 -- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f643c155030 msgr2=0x7f643c0acbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f644248f640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f643c155030 0x7f643c0acbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f644248f640 1 -- 172.21.15.26:0/4232687791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c0094d0 con 0x7f643c15f370 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f644248f640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f643c15f370 0x7f643c14f680 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f643400efc0 tx=0x7f643400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f64327fc640 1 -- 172.21.15.26:0/4232687791 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434009ba0 con 0x7f643c15f370 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f64327fc640 1 -- 172.21.15.26:0/4232687791 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643400f040 con 0x7f643c15f370 2023-12-19T16:55:06.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f64327fc640 1 -- 172.21.15.26:0/4232687791 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64340146c0 con 0x7f643c15f370 2023-12-19T16:55:06.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f6443ef2640 1 -- 172.21.15.26:0/4232687791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643c1466c0 con 0x7f643c15f370 2023-12-19T16:55:06.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f6443ef2640 1 -- 172.21.15.26:0/4232687791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c0ad110 con 0x7f643c15f370 2023-12-19T16:55:06.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.695+0000 7f6443ef2640 1 -- 172.21.15.26:0/4232687791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643c1468b0 con 0x7f643c15f370 2023-12-19T16:55:06.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.699+0000 7f64327fc640 1 -- 172.21.15.26:0/4232687791 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6434014820 con 0x7f643c15f370 2023-12-19T16:55:06.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.699+0000 7f64327fc640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6420065b80 0x7f6420068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:06.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.699+0000 7f64327fc640 1 -- 172.21.15.26:0/4232687791 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f6434084170 con 0x7f643c15f370 2023-12-19T16:55:06.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.699+0000 7f6441c8e640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6420065b80 0x7f6420068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:06.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.699+0000 7f6441c8e640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6420065b80 0x7f6420068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6438007fd0 tx=0x7f6438009040 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:06.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.703+0000 7f64327fc640 1 -- 172.21.15.26:0/4232687791 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6434051640 con 0x7f643c15f370 2023-12-19T16:55:06.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.891+0000 7f6443ef2640 1 -- 172.21.15.26:0/4232687791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f643c074270 con 0x7f643c15f370 2023-12-19T16:55:06.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.891+0000 7f64327fc640 1 -- 172.21.15.26:0/4232687791 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6434054c60 con 0x7f643c15f370 2023-12-19T16:55:06.900 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T16:55:06.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.895+0000 7f6443ef2640 1 -- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6420065b80 msgr2=0x7f6420068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.895+0000 7f6443ef2640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6420065b80 0x7f6420068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6438007fd0 tx=0x7f6438009040 comp rx=0 tx=0).stop 2023-12-19T16:55:06.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.895+0000 7f6443ef2640 1 -- 172.21.15.26:0/4232687791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f643c15f370 msgr2=0x7f643c14f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:06.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.895+0000 7f6443ef2640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f643c15f370 0x7f643c14f680 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f643400efc0 tx=0x7f643400c490 comp rx=0 tx=0).stop 2023-12-19T16:55:06.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.895+0000 7f6443ef2640 1 -- 172.21.15.26:0/4232687791 shutdown_connections 2023-12-19T16:55:06.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.895+0000 7f6443ef2640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f643c15f370 0x7f643c14f680 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.895+0000 7f6443ef2640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f643c155030 0x7f643c0acbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.895+0000 7f6443ef2640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6420065b80 0x7f6420068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.895+0000 7f6443ef2640 1 --2- 172.21.15.26:0/4232687791 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f643c0ce740 0x7f643c0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:06.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.895+0000 7f6443ef2640 1 -- 172.21.15.26:0/4232687791 >> 172.21.15.26:0/4232687791 conn(0x7f643c0658e0 msgr2=0x7f643c15d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:06.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.895+0000 7f6443ef2640 1 -- 172.21.15.26:0/4232687791 shutdown_connections 2023-12-19T16:55:06.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:06.895+0000 7f6443ef2640 1 -- 172.21.15.26:0/4232687791 wait complete. 2023-12-19T16:55:06.918 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-19T16:55:06.918 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T16:55:06.918 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T16:55:06.918 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T16:55:06.918 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T16:55:06.918 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-19T16:55:06.918 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-19T16:55:06.918 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T16:55:06.918 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:55:06.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:06.919 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T16:55:06.930 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T16:55:06.930 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T16:55:06.930 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T16:55:06.930 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:55:06.930 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T16:55:06.930 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:55:06.930 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:55:06.930 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T16:55:06.930 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T16:55:06.930 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 41634sec preferred_lft 41634sec 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T16:55:06.931 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-19T16:55:06.932 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:55:06.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:06.933 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T16:55:06.995 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T16:55:06.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:06.998 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T16:55:07.055 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T16:55:07.055 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T16:55:07.055 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T16:55:07.055 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T16:55:07.055 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T16:55:07.055 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T16:55:07.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:07.056 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:55:07.056 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:55:07.056 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T16:55:07.056 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T16:55:07.056 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T16:55:07.056 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:55:07.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:07.149 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T16:55:07.158 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T16:55:07.159 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T16:55:07.159 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:55:07.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:07.160 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T16:55:07.222 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:55:07.223 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T16:55:07.273 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:55:07.273 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:55:07.273 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T16:55:07.324 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T16:55:07.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:07.324 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:55:07.420 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:55:07.420 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:55:07.434 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:55:07.434 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:55:07.434 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T16:55:07.434 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T16:55:07.434 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T16:55:07.438 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T16:55:07.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:07.438 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T16:55:07.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:07.460 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:55:07.517 INFO:teuthology.orchestra.run.smithi026.stdout:client5944 2023-12-19T16:55:07.518 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:55:07.518 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:55:07.569 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-19T16:55:07.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.755+0000 7f455359e640 1 Processor -- start 2023-12-19T16:55:07.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.755+0000 7f455359e640 1 -- start start 2023-12-19T16:55:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f455359e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45540b4070 0x7f4554071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f455359e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455415e7c0 con 0x7f45540b2030 2023-12-19T16:55:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f455359e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45540b04c0 con 0x7f45540affe0 2023-12-19T16:55:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f455359e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45540c97a0 con 0x7f45540b4070 2023-12-19T16:55:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f455259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f45540b2030 0x7f45540c9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47528/0 (socket says 172.21.15.26:47528) 2023-12-19T16:55:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f455259c640 1 -- 172.21.15.26:0/3240560834 learned_addr learned my addr 172.21.15.26:0/3240560834 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:55:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4552d9d640 1 --2- 172.21.15.26:0/3240560834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45540b4070 0x7f4554071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850098987 0 0) 0x7f455415e7c0 con 0x7f45540b2030 2023-12-19T16:55:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4540003530 con 0x7f45540b2030 2023-12-19T16:55:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3361187295 0 0) 0x7f4540003530 con 0x7f45540b2030 2023-12-19T16:55:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45540b4070 msgr2=0x7f4554071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 --2- 172.21.15.26:0/3240560834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45540b4070 0x7f4554071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:07.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 >> v1:172.21.15.160:6789/0 conn(0x7f45540affe0 legacy=0x7f45540b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45540b07a0 con 0x7f45540b2030 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f4544001680 con 0x7f45540b2030 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 >> v1:172.21.15.26:6789/0 conn(0x7f45540b2030 legacy=0x7f45540c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 --2- 172.21.15.26:0/3240560834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45540b4070 0x7f4540003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 --2- 172.21.15.26:0/3240560834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4540003f60 0x7f45400047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 --2- 172.21.15.26:0/3240560834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4540004d10 0x7f4540005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45540c97a0 con 0x7f4540003f60 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454000fee0 con 0x7f4540004d10 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45400100a0 con 0x7f45540b4070 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f454bfff640 1 --2- 172.21.15.26:0/3240560834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4540003f60 0x7f45400047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f455259c640 1 --2- 172.21.15.26:0/3240560834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45540b4070 0x7f4540003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f4552d9d640 1 --2- 172.21.15.26:0/3240560834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4540004d10 0x7f4540005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f454bfff640 1 -- 172.21.15.26:0/3240560834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45540b4070 msgr2=0x7f4540003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:07.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f454bfff640 1 --2- 172.21.15.26:0/3240560834 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45540b4070 0x7f4540003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f454bfff640 1 -- 172.21.15.26:0/3240560834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4540004d10 msgr2=0x7f4540005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f454bfff640 1 --2- 172.21.15.26:0/3240560834 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4540004d10 0x7f4540005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.759+0000 7f454bfff640 1 -- 172.21.15.26:0/3240560834 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4540003530 con 0x7f4540003f60 2023-12-19T16:55:07.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.763+0000 7f454bfff640 1 --2- 172.21.15.26:0/3240560834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4540003f60 0x7f45400047d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f453c001860 tx=0x7f453c01eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a4a51d1617b6e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:07.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.763+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c0a87b0 con 0x7f4540003f60 2023-12-19T16:55:07.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.763+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c0a8910 con 0x7f4540003f60 2023-12-19T16:55:07.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.763+0000 7f4551d9b640 1 -- 172.21.15.26:0/3240560834 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c0a8d00 con 0x7f4540003f60 2023-12-19T16:55:07.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.763+0000 7f455359e640 1 -- 172.21.15.26:0/3240560834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4540003f60 msgr2=0x7f45400047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.763+0000 7f455359e640 1 --2- 172.21.15.26:0/3240560834 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4540003f60 0x7f45400047d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f453c001860 tx=0x7f453c01eb90 comp rx=0 tx=0).stop 2023-12-19T16:55:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.763+0000 7f4552d9d640 1 -- 172.21.15.26:0/3240560834 reap_dead start 2023-12-19T16:55:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.763+0000 7f455359e640 1 -- 172.21.15.26:0/3240560834 shutdown_connections 2023-12-19T16:55:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.763+0000 7f455359e640 1 -- 172.21.15.26:0/3240560834 >> 172.21.15.26:0/3240560834 conn(0x7f455406b0f0 msgr2=0x7f455406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.763+0000 7f455359e640 1 -- 172.21.15.26:0/3240560834 shutdown_connections 2023-12-19T16:55:07.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.763+0000 7f455359e640 1 -- 172.21.15.26:0/3240560834 wait complete. 2023-12-19T16:55:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.763+0000 7f455359e640 1 Processor -- start 2023-12-19T16:55:07.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f455359e640 1 -- start start 2023-12-19T16:55:07.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f455359e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4540003f60 0x7f45540c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:07.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f455359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45540c48a0 0x7f45540cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:07.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f455359e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45540c7970 0x7f45540c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:07.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f455359e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455414b500 con 0x7f45540c48a0 2023-12-19T16:55:07.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f455359e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45540c2670 con 0x7f45540c7970 2023-12-19T16:55:07.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f455359e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455414b750 con 0x7f4540003f60 2023-12-19T16:55:07.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f454bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45540c48a0 0x7f45540cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:07.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f454bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45540c48a0 0x7f45540cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57462/0 (socket says 172.21.15.26:57462) 2023-12-19T16:55:07.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f454bfff640 1 -- 172.21.15.26:0/1574311150 learned_addr learned my addr 172.21.15.26:0/1574311150 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f455259c640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4540003f60 0x7f45540c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f4552d9d640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45540c7970 0x7f45540c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f454bfff640 1 -- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4540003f60 msgr2=0x7f45540c1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f454bfff640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4540003f60 0x7f45540c1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f454bfff640 1 -- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45540c7970 msgr2=0x7f45540c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f454bfff640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45540c7970 0x7f45540c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f454bfff640 1 -- 172.21.15.26:0/1574311150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455414ba30 con 0x7f45540c48a0 2023-12-19T16:55:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f454bfff640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45540c48a0 0x7f45540cd6c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f453c0b3040 tx=0x7f4540005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f454affd640 1 -- 172.21.15.26:0/1574311150 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c005310 con 0x7f45540c48a0 2023-12-19T16:55:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f454affd640 1 -- 172.21.15.26:0/1574311150 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c005470 con 0x7f45540c48a0 2023-12-19T16:55:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f454affd640 1 -- 172.21.15.26:0/1574311150 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c006a00 con 0x7f45540c48a0 2023-12-19T16:55:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f455359e640 1 -- 172.21.15.26:0/1574311150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455414bd10 con 0x7f45540c48a0 2023-12-19T16:55:07.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f455359e640 1 -- 172.21.15.26:0/1574311150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45540cdc00 con 0x7f45540c48a0 2023-12-19T16:55:07.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.767+0000 7f455359e640 1 -- 172.21.15.26:0/1574311150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4554067400 con 0x7f45540c48a0 2023-12-19T16:55:07.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.771+0000 7f454affd640 1 -- 172.21.15.26:0/1574311150 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f453c0b7040 con 0x7f45540c48a0 2023-12-19T16:55:07.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.771+0000 7f454affd640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4534065b80 0x7f4534068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:07.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.771+0000 7f454affd640 1 -- 172.21.15.26:0/1574311150 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f453c11fc30 con 0x7f45540c48a0 2023-12-19T16:55:07.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.771+0000 7f455259c640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4534065b80 0x7f4534068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:07.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.771+0000 7f455259c640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4534065b80 0x7f4534068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4544007d30 tx=0x7f4544001520 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:07.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.775+0000 7f454affd640 1 -- 172.21.15.26:0/1574311150 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f453c0ed330 con 0x7f45540c48a0 2023-12-19T16:55:07.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.951+0000 7f455359e640 1 -- 172.21.15.26:0/1574311150 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f45540cde20 con 0x7f4534065b80 2023-12-19T16:55:07.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.955+0000 7f454affd640 1 -- 172.21.15.26:0/1574311150 <== mgr.5587 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f45540cde20 con 0x7f4534065b80 2023-12-19T16:55:07.957 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T16:55:07.957 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:55:07.957 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2023-12-19T16:55:07.957 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:55:07.957 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T16:55:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.955+0000 7f455359e640 1 -- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4534065b80 msgr2=0x7f4534068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.955+0000 7f455359e640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4534065b80 0x7f4534068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4544007d30 tx=0x7f4544001520 comp rx=0 tx=0).stop 2023-12-19T16:55:07.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.955+0000 7f455359e640 1 -- 172.21.15.26:0/1574311150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45540c48a0 msgr2=0x7f45540cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.955+0000 7f455359e640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45540c48a0 0x7f45540cd6c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f453c0b3040 tx=0x7f4540005800 comp rx=0 tx=0).stop 2023-12-19T16:55:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.955+0000 7f455359e640 1 -- 172.21.15.26:0/1574311150 shutdown_connections 2023-12-19T16:55:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.955+0000 7f455359e640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f45540c7970 0x7f45540c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.955+0000 7f455359e640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45540c48a0 0x7f45540cd6c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.955+0000 7f455359e640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4534065b80 0x7f4534068040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.955+0000 7f455359e640 1 --2- 172.21.15.26:0/1574311150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4540003f60 0x7f45540c1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.955+0000 7f455359e640 1 -- 172.21.15.26:0/1574311150 >> 172.21.15.26:0/1574311150 conn(0x7f455406b0f0 msgr2=0x7f45540af020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.955+0000 7f455359e640 1 -- 172.21.15.26:0/1574311150 shutdown_connections 2023-12-19T16:55:07.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:07.955+0000 7f455359e640 1 -- 172.21.15.26:0/1574311150 wait complete. 2023-12-19T16:55:07.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-19T16:55:08.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.119+0000 7fbb0257b640 1 Processor -- start 2023-12-19T16:55:08.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.119+0000 7fbb0257b640 1 -- start start 2023-12-19T16:55:08.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbb0257b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbafc0cb860 0x7fbafc06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:08.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbb0257b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbafc15d530 0x7fbafc15d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:08.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbb0257b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbafc15e240 con 0x7fbafc1599e0 2023-12-19T16:55:08.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbb0257b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbafc0c9520 con 0x7fbafc0cb860 2023-12-19T16:55:08.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbb0257b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc0c9800 con 0x7fbafc15d530 2023-12-19T16:55:08.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbafb7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbafc0cb860 0x7fbafc06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:08.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbafb7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbafc0cb860 0x7fbafc06f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57464/0 (socket says 172.21.15.26:57464) 2023-12-19T16:55:08.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbafb7fe640 1 -- 172.21.15.26:0/171971843 learned_addr learned my addr 172.21.15.26:0/171971843 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:08.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbafbfff640 1 --1- 172.21.15.26:0/171971843 >> v1:172.21.15.26:6789/0 conn(0x7fbafc1599e0 0x7fbafc0cb150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47542/0 (socket says 172.21.15.26:47542) 2023-12-19T16:55:08.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbb00b18640 1 --2- 172.21.15.26:0/171971843 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbafc15d530 0x7fbafc15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:08.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbafb7fe640 1 -- 172.21.15.26:0/171971843 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbafc15d530 msgr2=0x7fbafc15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:08.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbafb7fe640 1 --2- 172.21.15.26:0/171971843 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbafc15d530 0x7fbafc15d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbafb7fe640 1 -- 172.21.15.26:0/171971843 >> v1:172.21.15.26:6789/0 conn(0x7fbafc1599e0 legacy=0x7fbafc0cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:08.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbafb7fe640 1 -- 172.21.15.26:0/171971843 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc0c9ae0 con 0x7fbafc0cb860 2023-12-19T16:55:08.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbafaffd640 1 -- 172.21.15.26:0/171971843 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982783593 0 0) 0x7fbafc15e240 con 0x7fbafc1599e0 2023-12-19T16:55:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.123+0000 7fbafb7fe640 1 --2- 172.21.15.26:0/171971843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0cb860 0x7fbafc06f6d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009870 tx=0x7fbaf001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bee111f45bff7c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbafaffd640 1 -- 172.21.15.26:0/171971843 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf00a9620 con 0x7fbafc0cb860 2023-12-19T16:55:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbafaffd640 1 -- 172.21.15.26:0/171971843 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf00a9780 con 0x7fbafc0cb860 2023-12-19T16:55:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbafaffd640 1 -- 172.21.15.26:0/171971843 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf00a9b10 con 0x7fbafc0cb860 2023-12-19T16:55:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 -- 172.21.15.26:0/171971843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0cb860 msgr2=0x7fbafc06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 --2- 172.21.15.26:0/171971843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0cb860 0x7fbafc06f6d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0009870 tx=0x7fbaf001c540 comp rx=0 tx=0).stop 2023-12-19T16:55:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 -- 172.21.15.26:0/171971843 shutdown_connections 2023-12-19T16:55:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 --2- 172.21.15.26:0/171971843 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbafc15d530 0x7fbafc15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 --2- 172.21.15.26:0/171971843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0cb860 0x7fbafc06f6d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 -- 172.21.15.26:0/171971843 >> 172.21.15.26:0/171971843 conn(0x7fbafc066160 msgr2=0x7fbafc0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 -- 172.21.15.26:0/171971843 shutdown_connections 2023-12-19T16:55:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 -- 172.21.15.26:0/171971843 wait complete. 2023-12-19T16:55:08.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 Processor -- start 2023-12-19T16:55:08.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 -- start start 2023-12-19T16:55:08.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbafc0cb860 0x7fbafc0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:08.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc1599e0 0x7fbafc157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:08.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbafc15d530 0x7fbafc0c2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:08.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc0c4fd0 con 0x7fbafc1599e0 2023-12-19T16:55:08.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc0c2f40 con 0x7fbafc0cb860 2023-12-19T16:55:08.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbb0257b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc0c3220 con 0x7fbafc15d530 2023-12-19T16:55:08.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbafb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc1599e0 0x7fbafc157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbafb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc1599e0 0x7fbafc157df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57480/0 (socket says 172.21.15.26:57480) 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.127+0000 7fbafb7fe640 1 -- 172.21.15.26:0/1678935383 learned_addr learned my addr 172.21.15.26:0/1678935383 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbb00b18640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbafc15d530 0x7fbafc0c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbafbfff640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbafc0cb860 0x7fbafc0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbafb7fe640 1 -- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbafc15d530 msgr2=0x7fbafc0c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbafb7fe640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbafc15d530 0x7fbafc0c2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbafb7fe640 1 -- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbafc0cb860 msgr2=0x7fbafc0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbafb7fe640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbafc0cb860 0x7fbafc0b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbafb7fe640 1 -- 172.21.15.26:0/1678935383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf00094d0 con 0x7fbafc1599e0 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbafb7fe640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc1599e0 0x7fbafc157df0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00ab040 tx=0x7fbaf00192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbaf8ff9640 1 -- 172.21.15.26:0/1678935383 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf00bc450 con 0x7fbafc1599e0 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbaf8ff9640 1 -- 172.21.15.26:0/1678935383 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf00bc5b0 con 0x7fbafc1599e0 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbb0257b640 1 -- 172.21.15.26:0/1678935383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbafc0c4c60 con 0x7fbafc1599e0 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbaf8ff9640 1 -- 172.21.15.26:0/1678935383 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf00bc960 con 0x7fbafc1599e0 2023-12-19T16:55:08.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbb0257b640 1 -- 172.21.15.26:0/1678935383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc0c4880 con 0x7fbafc1599e0 2023-12-19T16:55:08.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbb0257b640 1 -- 172.21.15.26:0/1678935383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbafc0c4a70 con 0x7fbafc1599e0 2023-12-19T16:55:08.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbaf8ff9640 1 -- 172.21.15.26:0/1678935383 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbaf00bcac0 con 0x7fbafc1599e0 2023-12-19T16:55:08.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbaf8ff9640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbae8065b80 0x7fbae8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:08.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.131+0000 7fbaf8ff9640 1 -- 172.21.15.26:0/1678935383 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fbaf011df90 con 0x7fbafc1599e0 2023-12-19T16:55:08.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.135+0000 7fbafbfff640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbae8065b80 0x7fbae8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:08.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.135+0000 7fbafbfff640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbae8065b80 0x7fbae8068040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbaec007fd0 tx=0x7fbaec007ba0 comp rx=0 tx=0).ready entity=mgr.5587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:08.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.135+0000 7fbaf8ff9640 1 -- 172.21.15.26:0/1678935383 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbaf00eb560 con 0x7fbafc1599e0 2023-12-19T16:55:08.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.335+0000 7fbb0257b640 1 -- 172.21.15.26:0/1678935383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fbafc074270 con 0x7fbafc1599e0 2023-12-19T16:55:08.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.623+0000 7fbaf8ff9640 1 -- 172.21.15.26:0/1678935383 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 19) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fbaf00eb930 con 0x7fbafc1599e0 2023-12-19T16:55:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.623+0000 7fbaf8ff9640 1 -- 172.21.15.26:0/1678935383 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7fbaf00f2df0 con 0x7fbafc1599e0 2023-12-19T16:55:08.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.627+0000 7fbb0257b640 1 -- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbae8065b80 msgr2=0x7fbae8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:08.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.627+0000 7fbb0257b640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbae8065b80 0x7fbae8068040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbaec007fd0 tx=0x7fbaec007ba0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.627+0000 7fbb0257b640 1 -- 172.21.15.26:0/1678935383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc1599e0 msgr2=0x7fbafc157df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:08.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.627+0000 7fbb0257b640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc1599e0 0x7fbafc157df0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00ab040 tx=0x7fbaf00192e0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.627+0000 7fbb0257b640 1 -- 172.21.15.26:0/1678935383 shutdown_connections 2023-12-19T16:55:08.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.627+0000 7fbb0257b640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbae8065b80 0x7fbae8068040 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.627+0000 7fbb0257b640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbafc15d530 0x7fbafc0c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.627+0000 7fbb0257b640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc1599e0 0x7fbafc157df0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.627+0000 7fbb0257b640 1 --2- 172.21.15.26:0/1678935383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbafc0cb860 0x7fbafc0b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.627+0000 7fbb0257b640 1 -- 172.21.15.26:0/1678935383 >> 172.21.15.26:0/1678935383 conn(0x7fbafc066160 msgr2=0x7fbafc0ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:08.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.627+0000 7fbb0257b640 1 -- 172.21.15.26:0/1678935383 shutdown_connections 2023-12-19T16:55:08.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.627+0000 7fbb0257b640 1 -- 172.21.15.26:0/1678935383 wait complete. 2023-12-19T16:55:08.652 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-19T16:55:08.805 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:08.800+0000 7f434fe2d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:55:08.805 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:08.803+0000 7f5ad80f0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:55:08.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19fc66640 1 Processor -- start 2023-12-19T16:55:08.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19fc66640 1 -- start start 2023-12-19T16:55:08.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19fc66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff19815ccf0 0x7ff1980d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:08.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19fc66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19815e200 con 0x7ff1980b4600 2023-12-19T16:55:08.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19fc66640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1980d0aa0 con 0x7ff19815d0d0 2023-12-19T16:55:08.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19fc66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1980d1380 con 0x7ff19815ccf0 2023-12-19T16:55:08.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19da02640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff19815ccf0 0x7ff1980d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:08.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19da02640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff19815ccf0 0x7ff1980d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57516/0 (socket says 172.21.15.26:57516) 2023-12-19T16:55:08.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19da02640 1 -- 172.21.15.26:0/269910477 learned_addr learned my addr 172.21.15.26:0/269910477 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:08.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19e203640 1 --1- 172.21.15.26:0/269910477 >> v1:172.21.15.26:6789/0 conn(0x7ff1980b4600 0x7ff1980b4a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47550/0 (socket says 172.21.15.26:47550) 2023-12-19T16:55:08.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19d201640 1 --1- 172.21.15.26:0/269910477 >> v1:172.21.15.160:6789/0 conn(0x7ff19815d0d0 0x7ff1980b8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:39854/0 (socket says 172.21.15.26:39854) 2023-12-19T16:55:08.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19da02640 1 -- 172.21.15.26:0/269910477 >> v1:172.21.15.160:6789/0 conn(0x7ff19815d0d0 legacy=0x7ff1980b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:08.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19da02640 1 -- 172.21.15.26:0/269910477 >> v1:172.21.15.26:6789/0 conn(0x7ff1980b4600 legacy=0x7ff1980b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:08.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19ca00640 1 -- 172.21.15.26:0/269910477 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522805377 0 0) 0x7ff1980d0aa0 con 0x7ff19815d0d0 2023-12-19T16:55:08.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.803+0000 7ff19da02640 1 -- 172.21.15.26:0/269910477 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19815d680 con 0x7ff19815ccf0 2023-12-19T16:55:08.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.807+0000 7ff19da02640 1 --2- 172.21.15.26:0/269910477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19815ccf0 0x7ff1980d0e40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff194009870 tx=0x7ff19401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbd2b6efbebbd96 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.807+0000 7ff19ca00640 1 -- 172.21.15.26:0/269910477 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1940ab600 con 0x7ff19815ccf0 2023-12-19T16:55:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.807+0000 7ff19ca00640 1 -- 172.21.15.26:0/269910477 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1940a7050 con 0x7ff19815ccf0 2023-12-19T16:55:08.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.807+0000 7ff19ca00640 1 -- 172.21.15.26:0/269910477 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1940ab980 con 0x7ff19815ccf0 2023-12-19T16:55:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.807+0000 7ff19fc66640 1 -- 172.21.15.26:0/269910477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19815ccf0 msgr2=0x7ff1980d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.807+0000 7ff19fc66640 1 --2- 172.21.15.26:0/269910477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19815ccf0 0x7ff1980d0e40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff194009870 tx=0x7ff19401c3d0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.807+0000 7ff19fc66640 1 -- 172.21.15.26:0/269910477 shutdown_connections 2023-12-19T16:55:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.807+0000 7ff19fc66640 1 --2- 172.21.15.26:0/269910477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff19815ccf0 0x7ff1980d0e40 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.807+0000 7ff19fc66640 1 -- 172.21.15.26:0/269910477 >> 172.21.15.26:0/269910477 conn(0x7ff198065ee0 msgr2=0x7ff1980cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.807+0000 7ff19fc66640 1 -- 172.21.15.26:0/269910477 shutdown_connections 2023-12-19T16:55:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.807+0000 7ff19fc66640 1 -- 172.21.15.26:0/269910477 wait complete. 2023-12-19T16:55:08.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19fc66640 1 Processor -- start 2023-12-19T16:55:08.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19fc66640 1 -- start start 2023-12-19T16:55:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19fc66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1980b4600 0x7ff1981f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19fc66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff19815ccf0 0x7ff1981eff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19fc66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff19815d0d0 0x7ff1981584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19fc66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1980cb730 con 0x7ff1980b4600 2023-12-19T16:55:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19fc66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1981f0480 con 0x7ff19815ccf0 2023-12-19T16:55:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19fc66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff198159930 con 0x7ff19815d0d0 2023-12-19T16:55:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19da02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1980b4600 0x7ff1981f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19da02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1980b4600 0x7ff1981f29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57524/0 (socket says 172.21.15.26:57524) 2023-12-19T16:55:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19da02640 1 -- 172.21.15.26:0/770786851 learned_addr learned my addr 172.21.15.26:0/770786851 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19d201640 1 --2- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff19815ccf0 0x7ff1981eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19e203640 1 --2- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff19815d0d0 0x7ff1981584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19da02640 1 -- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff19815d0d0 msgr2=0x7ff1981584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19da02640 1 --2- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff19815d0d0 0x7ff1981584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19da02640 1 -- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff19815ccf0 msgr2=0x7ff1981eff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19da02640 1 --2- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff19815ccf0 0x7ff1981eff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19da02640 1 -- 172.21.15.26:0/770786851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1940094d0 con 0x7ff1980b4600 2023-12-19T16:55:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19da02640 1 --2- 172.21.15.26:0/770786851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1980b4600 0x7ff1981f29d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff1940ae040 tx=0x7ff19401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff18e7fc640 1 -- 172.21.15.26:0/770786851 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1940aa970 con 0x7ff1980b4600 2023-12-19T16:55:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff18e7fc640 1 -- 172.21.15.26:0/770786851 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1940aaad0 con 0x7ff1980b4600 2023-12-19T16:55:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff18e7fc640 1 -- 172.21.15.26:0/770786851 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1940aae80 con 0x7ff1980b4600 2023-12-19T16:55:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19fc66640 1 -- 172.21.15.26:0/770786851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff198158d20 con 0x7ff1980b4600 2023-12-19T16:55:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.811+0000 7ff19fc66640 1 -- 172.21.15.26:0/770786851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1981592e0 con 0x7ff1980b4600 2023-12-19T16:55:08.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.815+0000 7ff19fc66640 1 -- 172.21.15.26:0/770786851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff198074270 con 0x7ff1980b4600 2023-12-19T16:55:08.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.815+0000 7ff18e7fc640 1 -- 172.21.15.26:0/770786851 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 85347+0+0 (secure 0 0 0) 0x7ff1940b2040 con 0x7ff1980b4600 2023-12-19T16:55:08.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.815+0000 7ff18e7fc640 1 --2- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff17c065bd0 0x7ff17c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:08.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.815+0000 7ff18e7fc640 1 -- 172.21.15.26:0/770786851 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7ff19411e240 con 0x7ff1980b4600 2023-12-19T16:55:08.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.815+0000 7ff19d201640 1 -- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff17c065bd0 msgr2=0x7ff17c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:55:08.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.815+0000 7ff19d201640 1 --2- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff17c065bd0 0x7ff17c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:55:08.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.819+0000 7ff18e7fc640 1 -- 172.21.15.26:0/770786851 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff1940eb920 con 0x7ff1980b4600 2023-12-19T16:55:08.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.979+0000 7ff19fc66640 1 -- 172.21.15.26:0/770786851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ff1980b7f90 con 0x7ff1980b4600 2023-12-19T16:55:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.983+0000 7ff18e7fc640 1 -- 172.21.15.26:0/770786851 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff1940eef40 con 0x7ff1980b4600 2023-12-19T16:55:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.987+0000 7ff19fc66640 1 -- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff17c065bd0 msgr2=0x7ff17c068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:55:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.987+0000 7ff19fc66640 1 --2- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff17c065bd0 0x7ff17c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.987+0000 7ff19fc66640 1 -- 172.21.15.26:0/770786851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1980b4600 msgr2=0x7ff1981f29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.987+0000 7ff19fc66640 1 --2- 172.21.15.26:0/770786851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1980b4600 0x7ff1981f29d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff1940ae040 tx=0x7ff19401ac10 comp rx=0 tx=0).stop 2023-12-19T16:55:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.987+0000 7ff19fc66640 1 -- 172.21.15.26:0/770786851 shutdown_connections 2023-12-19T16:55:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.987+0000 7ff19fc66640 1 --2- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff17c065bd0 0x7ff17c068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.987+0000 7ff19fc66640 1 --2- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff19815d0d0 0x7ff1981584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.987+0000 7ff19fc66640 1 --2- 172.21.15.26:0/770786851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff19815ccf0 0x7ff1981eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.987+0000 7ff19fc66640 1 --2- 172.21.15.26:0/770786851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1980b4600 0x7ff1981f29d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.987+0000 7ff19fc66640 1 -- 172.21.15.26:0/770786851 >> 172.21.15.26:0/770786851 conn(0x7ff198065ee0 msgr2=0x7ff1980d1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.987+0000 7ff19fc66640 1 -- 172.21.15.26:0/770786851 shutdown_connections 2023-12-19T16:55:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:08.987+0000 7ff19fc66640 1 -- 172.21.15.26:0/770786851 wait complete. 2023-12-19T16:55:09.010 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-19T16:55:09.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.155+0000 7f35fd7db640 1 Processor -- start 2023-12-19T16:55:09.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.155+0000 7f35fd7db640 1 -- start start 2023-12-19T16:55:09.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35fd7db640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f35f80b8600 0x7f35f80b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:09.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35fd7db640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f80b4b30 0x7f35f80d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:09.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35fd7db640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f80b5bd0 con 0x7f35f815cde0 2023-12-19T16:55:09.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35fd7db640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f815e380 con 0x7f35f80b8600 2023-12-19T16:55:09.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35fd7db640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f80b4fc0 con 0x7f35f80b4b30 2023-12-19T16:55:09.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35f6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f35f815cde0 0x7f35f815d1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47564/0 (socket says 172.21.15.26:47564) 2023-12-19T16:55:09.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35f6ffd640 1 -- 172.21.15.26:0/1953283547 learned_addr learned my addr 172.21.15.26:0/1953283547 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:55:09.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35f77fe640 1 --2- 172.21.15.26:0/1953283547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f80b4b30 0x7f35f80d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35effff640 1 --2- 172.21.15.26:0/1953283547 >> v2:172.21.15.160:3300/0 conn(0x7f35f80b8600 0x7f35f80b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35f67fc640 1 -- 172.21.15.26:0/1953283547 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339107036 0 0) 0x7f35f80b5bd0 con 0x7f35f815cde0 2023-12-19T16:55:09.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35f67fc640 1 -- 172.21.15.26:0/1953283547 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35e4003530 con 0x7f35f815cde0 2023-12-19T16:55:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35f77fe640 1 -- 172.21.15.26:0/1953283547 >> v2:172.21.15.160:3300/0 conn(0x7f35f80b8600 msgr2=0x7f35f80b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35f77fe640 1 --2- 172.21.15.26:0/1953283547 >> v2:172.21.15.160:3300/0 conn(0x7f35f80b8600 0x7f35f80b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35f77fe640 1 -- 172.21.15.26:0/1953283547 >> v1:172.21.15.26:6789/0 conn(0x7f35f815cde0 legacy=0x7f35f815d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35f77fe640 1 -- 172.21.15.26:0/1953283547 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f80d0ee0 con 0x7f35f80b4b30 2023-12-19T16:55:09.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.159+0000 7f35effff640 1 --2- 172.21.15.26:0/1953283547 >> v2:172.21.15.160:3300/0 conn(0x7f35f80b8600 0x7f35f80b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:55:09.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35f77fe640 1 --2- 172.21.15.26:0/1953283547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f80b4b30 0x7f35f80d1370 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009d90 tx=0x7f35f001f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14331903fc929d87 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:09.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35f67fc640 1 -- 172.21.15.26:0/1953283547 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f00af600 con 0x7f35f80b4b30 2023-12-19T16:55:09.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35f67fc640 1 -- 172.21.15.26:0/1953283547 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f00ab050 con 0x7f35f80b4b30 2023-12-19T16:55:09.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35f67fc640 1 -- 172.21.15.26:0/1953283547 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f00af980 con 0x7f35f80b4b30 2023-12-19T16:55:09.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35fd7db640 1 -- 172.21.15.26:0/1953283547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f80b4b30 msgr2=0x7f35f80d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:09.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35fd7db640 1 --2- 172.21.15.26:0/1953283547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f80b4b30 0x7f35f80d1370 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009d90 tx=0x7f35f001f130 comp rx=0 tx=0).stop 2023-12-19T16:55:09.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35fd7db640 1 -- 172.21.15.26:0/1953283547 shutdown_connections 2023-12-19T16:55:09.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35fd7db640 1 --2- 172.21.15.26:0/1953283547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f80b4b30 0x7f35f80d1370 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:09.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35fd7db640 1 --2- 172.21.15.26:0/1953283547 >> v2:172.21.15.160:3300/0 conn(0x7f35f80b8600 0x7f35f80b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:09.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35fd7db640 1 -- 172.21.15.26:0/1953283547 >> 172.21.15.26:0/1953283547 conn(0x7f35f8065ee0 msgr2=0x7f35f80d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:09.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35fd7db640 1 -- 172.21.15.26:0/1953283547 shutdown_connections 2023-12-19T16:55:09.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35fd7db640 1 -- 172.21.15.26:0/1953283547 wait complete. 2023-12-19T16:55:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35fd7db640 1 Processor -- start 2023-12-19T16:55:09.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35fd7db640 1 -- start start 2023-12-19T16:55:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.163+0000 7f35fd7db640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f80b4b30 0x7f35f81f2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35fd7db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f80b8600 0x7f35f8159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35fd7db640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f815cde0 0x7f35f80b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:09.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35fd7db640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f81f2550 con 0x7f35f80b8600 2023-12-19T16:55:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35fd7db640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f815aea0 con 0x7f35f815cde0 2023-12-19T16:55:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35fd7db640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f815a520 con 0x7f35f80b4b30 2023-12-19T16:55:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f80b8600 0x7f35f8159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35f6ffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f80b4b30 0x7f35f81f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f80b8600 0x7f35f8159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57538/0 (socket says 172.21.15.26:57538) 2023-12-19T16:55:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35effff640 1 -- 172.21.15.26:0/3441077042 learned_addr learned my addr 172.21.15.26:0/3441077042 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35f77fe640 1 --2- 172.21.15.26:0/3441077042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f815cde0 0x7f35f80b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35effff640 1 -- 172.21.15.26:0/3441077042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f80b4b30 msgr2=0x7f35f81f2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35effff640 1 --2- 172.21.15.26:0/3441077042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f80b4b30 0x7f35f81f2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35effff640 1 -- 172.21.15.26:0/3441077042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f815cde0 msgr2=0x7f35f80b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35effff640 1 --2- 172.21.15.26:0/3441077042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f815cde0 0x7f35f80b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35effff640 1 -- 172.21.15.26:0/3441077042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e0009420 con 0x7f35f80b8600 2023-12-19T16:55:09.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35effff640 1 --2- 172.21.15.26:0/3441077042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f80b8600 0x7f35f8159fe0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f35e000ebe0 tx=0x7f35e000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35ef7fe640 1 -- 172.21.15.26:0/3441077042 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e000eee0 con 0x7f35f80b8600 2023-12-19T16:55:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35ef7fe640 1 -- 172.21.15.26:0/3441077042 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e000cc30 con 0x7f35f80b8600 2023-12-19T16:55:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35ef7fe640 1 -- 172.21.15.26:0/3441077042 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0013470 con 0x7f35f80b8600 2023-12-19T16:55:09.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35fd7db640 1 -- 172.21.15.26:0/3441077042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f0009a70 con 0x7f35f80b8600 2023-12-19T16:55:09.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35fd7db640 1 -- 172.21.15.26:0/3441077042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f80cb060 con 0x7f35f80b8600 2023-12-19T16:55:09.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35fd7db640 1 -- 172.21.15.26:0/3441077042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f8074270 con 0x7f35f80b8600 2023-12-19T16:55:09.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.167+0000 7f35ef7fe640 1 -- 172.21.15.26:0/3441077042 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f35e00135d0 con 0x7f35f80b8600 2023-12-19T16:55:09.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.171+0000 7f35ef7fe640 1 --2- 172.21.15.26:0/3441077042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35d8065bd0 0x7f35d8068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:09.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.171+0000 7f35ef7fe640 1 -- 172.21.15.26:0/3441077042 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f35e00839b0 con 0x7f35f80b8600 2023-12-19T16:55:09.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.171+0000 7f35f6ffd640 1 -- 172.21.15.26:0/3441077042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35d8065bd0 msgr2=0x7f35d8068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:55:09.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.171+0000 7f35f6ffd640 1 --2- 172.21.15.26:0/3441077042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35d8065bd0 0x7f35d8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:55:09.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.175+0000 7f35ef7fe640 1 -- 172.21.15.26:0/3441077042 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f35e0050f90 con 0x7f35f80b8600 2023-12-19T16:55:09.221 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:09.216+0000 7f434fe2d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:55:09.256 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:09.251+0000 7f5ad80f0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:55:09.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.335+0000 7f35fd7db640 1 -- 172.21.15.26:0/3441077042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f35f815d690 con 0x7f35f80b8600 2023-12-19T16:55:09.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.339+0000 7f35ef7fe640 1 -- 172.21.15.26:0/3441077042 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f35e00545b0 con 0x7f35f80b8600 2023-12-19T16:55:09.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.343+0000 7f35fd7db640 1 -- 172.21.15.26:0/3441077042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35d8065bd0 msgr2=0x7f35d8068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:55:09.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.343+0000 7f35fd7db640 1 --2- 172.21.15.26:0/3441077042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35d8065bd0 0x7f35d8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:09.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.343+0000 7f35fd7db640 1 -- 172.21.15.26:0/3441077042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f80b8600 msgr2=0x7f35f8159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:09.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.343+0000 7f35fd7db640 1 --2- 172.21.15.26:0/3441077042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f80b8600 0x7f35f8159fe0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f35e000ebe0 tx=0x7f35e000c520 comp rx=0 tx=0).stop 2023-12-19T16:55:09.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.343+0000 7f35fd7db640 1 -- 172.21.15.26:0/3441077042 shutdown_connections 2023-12-19T16:55:09.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.343+0000 7f35fd7db640 1 --2- 172.21.15.26:0/3441077042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f35f815cde0 0x7f35f80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:09.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.343+0000 7f35fd7db640 1 --2- 172.21.15.26:0/3441077042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35f80b8600 0x7f35f8159fe0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:09.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.343+0000 7f35fd7db640 1 --2- 172.21.15.26:0/3441077042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f35d8065bd0 0x7f35d8068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:09.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.343+0000 7f35fd7db640 1 --2- 172.21.15.26:0/3441077042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f35f80b4b30 0x7f35f81f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:09.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.343+0000 7f35fd7db640 1 -- 172.21.15.26:0/3441077042 >> 172.21.15.26:0/3441077042 conn(0x7f35f8065ee0 msgr2=0x7f35f815f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:09.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.343+0000 7f35fd7db640 1 -- 172.21.15.26:0/3441077042 shutdown_connections 2023-12-19T16:55:09.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.343+0000 7f35fd7db640 1 -- 172.21.15.26:0/3441077042 wait complete. 2023-12-19T16:55:09.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:09.365 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-19T16:55:09.370 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:09.368+0000 7f434fe2d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:55:09.382 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1M 2023-12-19T16:55:09.418 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:09.415+0000 7f5ad80f0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:55:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.563+0000 7f085ff52640 1 Processor -- start 2023-12-19T16:55:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.563+0000 7f085ff52640 1 -- start start 2023-12-19T16:55:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.567+0000 7f085ff52640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f085815ada0 0x7f08580706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.567+0000 7f085ff52640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0858070d90 con 0x7f085815a9c0 2023-12-19T16:55:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.567+0000 7f085ff52640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0858071070 con 0x7f085806ac80 2023-12-19T16:55:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.567+0000 7f085ff52640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08580cc270 con 0x7f085815ada0 2023-12-19T16:55:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.567+0000 7f085d4ed640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f085815ada0 0x7f08580706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.567+0000 7f085d4ed640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f085815ada0 0x7f08580706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57544/0 (socket says 172.21.15.26:57544) 2023-12-19T16:55:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.567+0000 7f085d4ed640 1 -- 172.21.15.26:0/2011630397 learned_addr learned my addr 172.21.15.26:0/2011630397 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.567+0000 7f085dcee640 1 --1- 172.21.15.26:0/2011630397 >> v1:172.21.15.26:6789/0 conn(0x7f085815a9c0 0x7f08580b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47576/0 (socket says 172.21.15.26:47576) 2023-12-19T16:55:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.567+0000 7f085d4ed640 1 -- 172.21.15.26:0/2011630397 >> v1:172.21.15.160:6789/0 conn(0x7f085806ac80 legacy=0x7f08580b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.567+0000 7f085d4ed640 1 -- 172.21.15.26:0/2011630397 >> v1:172.21.15.26:6789/0 conn(0x7f085815a9c0 legacy=0x7f08580b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:09.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.567+0000 7f085d4ed640 1 -- 172.21.15.26:0/2011630397 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0858071350 con 0x7f085815ada0 2023-12-19T16:55:09.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.567+0000 7f085ccec640 1 -- 172.21.15.26:0/2011630397 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053669405 0 0) 0x7f0858070d90 con 0x7f085815a9c0 2023-12-19T16:55:09.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.567+0000 7f085d4ed640 1 --2- 172.21.15.26:0/2011630397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f085815ada0 0x7f08580706d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0848009870 tx=0x7f084801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176c9a30584bbd33 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ccec640 1 -- 172.21.15.26:0/2011630397 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08480a6040 con 0x7f085815ada0 2023-12-19T16:55:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ccec640 1 -- 172.21.15.26:0/2011630397 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08480aa600 con 0x7f085815ada0 2023-12-19T16:55:09.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ccec640 1 -- 172.21.15.26:0/2011630397 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08480aa990 con 0x7f085815ada0 2023-12-19T16:55:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 -- 172.21.15.26:0/2011630397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f085815ada0 msgr2=0x7f08580706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 --2- 172.21.15.26:0/2011630397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f085815ada0 0x7f08580706d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0848009870 tx=0x7f084801c480 comp rx=0 tx=0).stop 2023-12-19T16:55:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 -- 172.21.15.26:0/2011630397 shutdown_connections 2023-12-19T16:55:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 --2- 172.21.15.26:0/2011630397 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f085815ada0 0x7f08580706d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 -- 172.21.15.26:0/2011630397 >> 172.21.15.26:0/2011630397 conn(0x7f08580ce730 msgr2=0x7f08580ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 -- 172.21.15.26:0/2011630397 shutdown_connections 2023-12-19T16:55:09.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 -- 172.21.15.26:0/2011630397 wait complete. 2023-12-19T16:55:09.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 Processor -- start 2023-12-19T16:55:09.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 -- start start 2023-12-19T16:55:09.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f085806ac80 0x7f0858155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:09.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f085815a9c0 0x7f08581543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:09.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f085815ada0 0x7f0858143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:09.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0858153290 con 0x7f085806ac80 2023-12-19T16:55:09.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0858146ae0 con 0x7f085815ada0 2023-12-19T16:55:09.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085ff52640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08580afe50 con 0x7f085815a9c0 2023-12-19T16:55:09.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.571+0000 7f085dcee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f085806ac80 0x7f0858155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:09.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085dcee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f085806ac80 0x7f0858155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57554/0 (socket says 172.21.15.26:57554) 2023-12-19T16:55:09.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085dcee640 1 -- 172.21.15.26:0/1613397944 learned_addr learned my addr 172.21.15.26:0/1613397944 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:09.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085d4ed640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f085815a9c0 0x7f08581543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:09.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085e4ef640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f085815ada0 0x7f0858143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:09.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085d4ed640 1 -- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f085815ada0 msgr2=0x7f0858143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:09.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085d4ed640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f085815ada0 0x7f0858143680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:09.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085d4ed640 1 -- 172.21.15.26:0/1613397944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f085806ac80 msgr2=0x7f0858155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:09.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085d4ed640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f085806ac80 0x7f0858155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:09.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085d4ed640 1 -- 172.21.15.26:0/1613397944 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08480094d0 con 0x7f085815a9c0 2023-12-19T16:55:09.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085dcee640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f085806ac80 0x7f0858155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:55:09.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085d4ed640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f085815a9c0 0x7f08581543b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f08580cf7a0 tx=0x7f084801acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:09.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f084e7fc640 1 -- 172.21.15.26:0/1613397944 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08480aaba0 con 0x7f085815a9c0 2023-12-19T16:55:09.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f084e7fc640 1 -- 172.21.15.26:0/1613397944 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08480b7070 con 0x7f085815a9c0 2023-12-19T16:55:09.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085ff52640 1 -- 172.21.15.26:0/1613397944 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08580b0610 con 0x7f085815a9c0 2023-12-19T16:55:09.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f084e7fc640 1 -- 172.21.15.26:0/1613397944 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0848004030 con 0x7f085815a9c0 2023-12-19T16:55:09.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085ff52640 1 -- 172.21.15.26:0/1613397944 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0858152fb0 con 0x7f085815a9c0 2023-12-19T16:55:09.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f085ff52640 1 -- 172.21.15.26:0/1613397944 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08580afcb0 con 0x7f085815a9c0 2023-12-19T16:55:09.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.575+0000 7f084e7fc640 1 -- 172.21.15.26:0/1613397944 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 19) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f0848004190 con 0x7f085815a9c0 2023-12-19T16:55:09.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.579+0000 7f084e7fc640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 0x7f083c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:09.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.579+0000 7f084e7fc640 1 -- 172.21.15.26:0/1613397944 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f084811cff0 con 0x7f085815a9c0 2023-12-19T16:55:09.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.579+0000 7f085dcee640 1 -- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 msgr2=0x7f083c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:55:09.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.579+0000 7f085dcee640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 0x7f083c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:55:09.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.583+0000 7f084e7fc640 1 -- 172.21.15.26:0/1613397944 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f084814e0a0 con 0x7f085815a9c0 2023-12-19T16:55:09.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.762+0000 7f085ff52640 1 -- 172.21.15.26:0/1613397944 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0858168ad0 con 0x7f083c065bd0 2023-12-19T16:55:09.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.782+0000 7f085dcee640 1 -- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 msgr2=0x7f083c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:55:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:09.782+0000 7f085dcee640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 0x7f083c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T16:55:09.851 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:09.848+0000 7f434fe2d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:55:09.908 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:09.904+0000 7f434fe2d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:55:09.933 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:09.930+0000 7f5ad80f0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:55:09.963 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:09.960+0000 7f434fe2d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:55:09.994 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:09.990+0000 7f5ad80f0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:55:10.048 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:10.044+0000 7f434fe2d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:55:10.052 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:10.050+0000 7f5ad80f0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:55:10.104 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:10.100+0000 7f434fe2d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:55:10.144 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:10.138+0000 7f5ad80f0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:55:10.188 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:10.184+0000 7f434fe2d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:55:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:10.186+0000 7f085dcee640 1 -- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 msgr2=0x7f083c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:55:10.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:10.186+0000 7f085dcee640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 0x7f083c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-19T16:55:10.203 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:10.198+0000 7f5ad80f0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:55:10.243 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:10.240+0000 7f434fe2d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:55:10.292 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:10.286+0000 7f5ad80f0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:55:10.350 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:10.346+0000 7f5ad80f0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:55:10.386 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:10.384+0000 7f434fe2d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:55:10.444 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:10.440+0000 7f434fe2d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:55:10.502 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:10.498+0000 7f5ad80f0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:55:10.563 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:10.558+0000 7f5ad80f0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:55:10.684 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:55:10.684 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:55:10.684 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:55:10.692 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:10.688+0000 7f434fe2d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:55:10.814 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:55:10.814 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:55:10.814 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:55:10.822 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:10.818+0000 7f5ad80f0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:55:10.827 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:10.824+0000 7f434fe2d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:55:10.950 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:10.948+0000 7f434fe2d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:55:10.964 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:10.958+0000 7f5ad80f0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:55:10.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:10.990+0000 7f085dcee640 1 -- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 msgr2=0x7f083c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:55:10.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:10.990+0000 7f085dcee640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 0x7f083c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-19T16:55:11.020 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:11.016+0000 7f434fe2d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:55:11.095 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:11.090+0000 7f5ad80f0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:55:11.155 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:11.152+0000 7f434fe2d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:55:11.169 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:11.166+0000 7f5ad80f0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:55:11.311 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:11.306+0000 7f5ad80f0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:55:11.692 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:11.688+0000 7f434fe2d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:55:11.778 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:11.776+0000 7f434fe2d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:55:11.878 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:11.874+0000 7f5ad80f0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:55:11.925 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:11.920+0000 7f434fe2d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:55:11.966 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:11.962+0000 7f5ad80f0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:55:12.008 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:12.004+0000 7f434fe2d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:55:12.122 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:12.118+0000 7f5ad80f0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:55:12.210 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:12.206+0000 7f5ad80f0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:55:12.322 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:12.320+0000 7f434fe2d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:55:12.382 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:12.380+0000 7f434fe2d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:55:12.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:12.394+0000 7f084e7fc640 1 -- 172.21.15.26:0/1613397944 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mgrmap(e 20) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f08480eaaa0 con 0x7f085815a9c0 2023-12-19T16:55:12.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:12.394+0000 7f084e7fc640 1 -- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 msgr2=0x7f083c068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:55:12.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:12.394+0000 7f084e7fc640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 0x7f083c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:12.508 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:12.504+0000 7f42d1d1a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:55:12.508 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:12.504+0000 7f42d1d1a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:55:12.508 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:12.504+0000 7f42d1d1a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:55:12.508 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:12.504+0000 7f42d1d1a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:55:12.508 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:12.504+0000 7f42d1d1a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:55:12.509 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:12.504+0000 7f42d4d20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:55:12.509 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:12.504+0000 7f42d4d20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:55:12.509 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:12.504+0000 7f42d4d20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:55:12.510 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:12.504+0000 7f42d4d20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:55:12.510 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:55:12.504+0000 7f42d4d20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:55:12.559 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:12.554+0000 7f5ad80f0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:55:12.623 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:55:12.618+0000 7f5ad80f0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:55:13.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.398+0000 7f084e7fc640 1 -- 172.21.15.26:0/1613397944 <== mon.2 v2:172.21.15.160:3301/0 8 ==== mgrmap(e 21) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f08480f5e30 con 0x7f085815a9c0 2023-12-19T16:55:13.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.398+0000 7f084e7fc640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 0x7f083c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:13.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.398+0000 7f084e7fc640 1 -- 172.21.15.26:0/1613397944 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f083c06bbd0 con 0x7f083c065bd0 2023-12-19T16:55:13.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.398+0000 7f085dcee640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 0x7f083c068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.402+0000 7f085dcee640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 0x7f083c068090 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0854000f70 tx=0x7f08540017e0 comp rx=0 tx=0).ready entity=mgr.5965 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.466+0000 7f084e7fc640 1 -- 172.21.15.26:0/1613397944 <== mgr.5965 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f083c06bbd0 con 0x7f083c065bd0 2023-12-19T16:55:13.472 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path snap_test_dir1 2023-12-19T16:55:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.470+0000 7f085ff52640 1 -- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 msgr2=0x7f083c068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.470+0000 7f085ff52640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 0x7f083c068090 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0854000f70 tx=0x7f08540017e0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.470+0000 7f085ff52640 1 -- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f085815a9c0 msgr2=0x7f08581543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.470+0000 7f085ff52640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f085815a9c0 0x7f08581543b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f08580cf7a0 tx=0x7f084801acc0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.470+0000 7f085ff52640 1 -- 172.21.15.26:0/1613397944 shutdown_connections 2023-12-19T16:55:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.470+0000 7f085ff52640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f085815ada0 0x7f0858143680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.470+0000 7f085ff52640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f083c065bd0 0x7f083c068090 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.470+0000 7f085ff52640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f085815a9c0 0x7f08581543b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.470+0000 7f085ff52640 1 --2- 172.21.15.26:0/1613397944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f085806ac80 0x7f0858155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.470+0000 7f085ff52640 1 -- 172.21.15.26:0/1613397944 >> 172.21.15.26:0/1613397944 conn(0x7f08580ce730 msgr2=0x7f0858155880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:13.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.470+0000 7f085ff52640 1 -- 172.21.15.26:0/1613397944 shutdown_connections 2023-12-19T16:55:13.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.470+0000 7f085ff52640 1 -- 172.21.15.26:0/1613397944 wait complete. 2023-12-19T16:55:13.495 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-19T16:55:13.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 2M 2023-12-19T16:55:13.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e3455640 1 Processor -- start 2023-12-19T16:55:13.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e3455640 1 -- start start 2023-12-19T16:55:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e3455640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f44dc15a9c0 0x7f44dc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e3455640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44dc15ada0 0x7f44dc0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e3455640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f44dc06ac80 0x7f44dc06b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e3455640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44dc0cc270 con 0x7f44dc06ac80 2023-12-19T16:55:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e3455640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44dc0b46c0 con 0x7f44dc15a9c0 2023-12-19T16:55:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e3455640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc070d90 con 0x7f44dc15ada0 2023-12-19T16:55:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e19f2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f44dc06ac80 0x7f44dc06b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e19f2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f44dc06ac80 0x7f44dc06b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57568/0 (socket says 172.21.15.26:57568) 2023-12-19T16:55:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e11f1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f44dc15a9c0 0x7f44dc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e19f2640 1 -- 172.21.15.26:0/2646872028 learned_addr learned my addr 172.21.15.26:0/2646872028 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e09f0640 1 --2- 172.21.15.26:0/2646872028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44dc15ada0 0x7f44dc0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:13.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e19f2640 1 -- 172.21.15.26:0/2646872028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44dc15ada0 msgr2=0x7f44dc0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:13.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e19f2640 1 --2- 172.21.15.26:0/2646872028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44dc15ada0 0x7f44dc0b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e19f2640 1 -- 172.21.15.26:0/2646872028 >> v2:172.21.15.160:3300/0 conn(0x7f44dc15a9c0 msgr2=0x7f44dc0706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:13.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e19f2640 1 --2- 172.21.15.26:0/2646872028 >> v2:172.21.15.160:3300/0 conn(0x7f44dc15a9c0 0x7f44dc0706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.638+0000 7f44e19f2640 1 -- 172.21.15.26:0/2646872028 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc071070 con 0x7f44dc06ac80 2023-12-19T16:55:13.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.642+0000 7f44e19f2640 1 --2- 172.21.15.26:0/2646872028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44dc06ac80 0x7f44dc06b0b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0cf590 tx=0x7f44d401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215eddff31064e3d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:13.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.642+0000 7f44d3fff640 1 -- 172.21.15.26:0/2646872028 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40a5790 con 0x7f44dc06ac80 2023-12-19T16:55:13.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.642+0000 7f44d3fff640 1 -- 172.21.15.26:0/2646872028 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d40a58f0 con 0x7f44dc06ac80 2023-12-19T16:55:13.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.642+0000 7f44d3fff640 1 -- 172.21.15.26:0/2646872028 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40a5c80 con 0x7f44dc06ac80 2023-12-19T16:55:13.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.642+0000 7f44e3455640 1 -- 172.21.15.26:0/2646872028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44dc06ac80 msgr2=0x7f44dc06b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:13.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.642+0000 7f44e3455640 1 --2- 172.21.15.26:0/2646872028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44dc06ac80 0x7f44dc06b0b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0cf590 tx=0x7f44d401c910 comp rx=0 tx=0).stop 2023-12-19T16:55:13.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.642+0000 7f44e3455640 1 -- 172.21.15.26:0/2646872028 shutdown_connections 2023-12-19T16:55:13.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.642+0000 7f44e3455640 1 --2- 172.21.15.26:0/2646872028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44dc06ac80 0x7f44dc06b0b0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.642+0000 7f44e3455640 1 --2- 172.21.15.26:0/2646872028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44dc15ada0 0x7f44dc0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.642+0000 7f44e3455640 1 --2- 172.21.15.26:0/2646872028 >> v2:172.21.15.160:3300/0 conn(0x7f44dc15a9c0 0x7f44dc0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.642+0000 7f44e3455640 1 -- 172.21.15.26:0/2646872028 >> 172.21.15.26:0/2646872028 conn(0x7f44dc0ce730 msgr2=0x7f44dc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:13.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.642+0000 7f44e3455640 1 -- 172.21.15.26:0/2646872028 shutdown_connections 2023-12-19T16:55:13.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.642+0000 7f44e3455640 1 -- 172.21.15.26:0/2646872028 wait complete. 2023-12-19T16:55:13.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e3455640 1 Processor -- start 2023-12-19T16:55:13.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e3455640 1 -- start start 2023-12-19T16:55:13.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e3455640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44dc06ac80 0x7f44dc0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:13.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e3455640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44dc15a9c0 0x7f44dc0b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:13.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e3455640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44dc15ada0 0x7f44dc0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:13.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e3455640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc0c9320 con 0x7f44dc15ada0 2023-12-19T16:55:13.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e3455640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc0c8cf0 con 0x7f44dc06ac80 2023-12-19T16:55:13.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e3455640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc0b1a40 con 0x7f44dc15a9c0 2023-12-19T16:55:13.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e19f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44dc15ada0 0x7f44dc0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:13.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e19f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44dc15ada0 0x7f44dc0b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57578/0 (socket says 172.21.15.26:57578) 2023-12-19T16:55:13.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e19f2640 1 -- 172.21.15.26:0/1081343326 learned_addr learned my addr 172.21.15.26:0/1081343326 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:13.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e09f0640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44dc15a9c0 0x7f44dc0b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:13.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e11f1640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44dc06ac80 0x7f44dc0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:13.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e19f2640 1 -- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44dc15a9c0 msgr2=0x7f44dc0b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:13.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e19f2640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44dc15a9c0 0x7f44dc0b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e19f2640 1 -- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44dc06ac80 msgr2=0x7f44dc0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:13.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e19f2640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44dc06ac80 0x7f44dc0c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e19f2640 1 -- 172.21.15.26:0/1081343326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d4009b30 con 0x7f44dc15ada0 2023-12-19T16:55:13.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e19f2640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44dc15ada0 0x7f44dc0b2eb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f44d40af040 tx=0x7f44d40196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:13.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44d1ffb640 1 -- 172.21.15.26:0/1081343326 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40b8670 con 0x7f44dc15ada0 2023-12-19T16:55:13.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44d1ffb640 1 -- 172.21.15.26:0/1081343326 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d40b87d0 con 0x7f44dc15ada0 2023-12-19T16:55:13.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44d1ffb640 1 -- 172.21.15.26:0/1081343326 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40b8b60 con 0x7f44dc15ada0 2023-12-19T16:55:13.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e3455640 1 -- 172.21.15.26:0/1081343326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44dc0b33f0 con 0x7f44dc15ada0 2023-12-19T16:55:13.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.646+0000 7f44e3455640 1 -- 172.21.15.26:0/1081343326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44dc0b3970 con 0x7f44dc15ada0 2023-12-19T16:55:13.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.650+0000 7f44e3455640 1 -- 172.21.15.26:0/1081343326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44dc0c9460 con 0x7f44dc15ada0 2023-12-19T16:55:13.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.650+0000 7f44d1ffb640 1 -- 172.21.15.26:0/1081343326 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f44d40b8cc0 con 0x7f44dc15ada0 2023-12-19T16:55:13.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.650+0000 7f44d1ffb640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f44c0065900 0x7f44c0067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:13.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.650+0000 7f44d1ffb640 1 -- 172.21.15.26:0/1081343326 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f44d411e6c0 con 0x7f44dc15ada0 2023-12-19T16:55:13.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.650+0000 7f44e11f1640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f44c0065900 0x7f44c0067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:13.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.654+0000 7f44e11f1640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f44c0065900 0x7f44c0067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009b60 tx=0x7f44d8009190 comp rx=0 tx=0).ready entity=mgr.5965 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:13.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.654+0000 7f44d1ffb640 1 -- 172.21.15.26:0/1081343326 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f44dc0c9460 con 0x7f44dc15ada0 2023-12-19T16:55:13.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.834+0000 7f44e3455640 1 -- 172.21.15.26:0/1081343326 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7f44dc0c9460 con 0x7f44c0065900 2023-12-19T16:55:13.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.886+0000 7f44d1ffb640 1 -- 172.21.15.26:0/1081343326 <== mgr.5965 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f44dc0c9460 con 0x7f44c0065900 2023-12-19T16:55:13.892 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path snap_test_dir1 2023-12-19T16:55:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.890+0000 7f44e3455640 1 -- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f44c0065900 msgr2=0x7f44c0067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.890+0000 7f44e3455640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f44c0065900 0x7f44c0067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009b60 tx=0x7f44d8009190 comp rx=0 tx=0).stop 2023-12-19T16:55:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.890+0000 7f44e3455640 1 -- 172.21.15.26:0/1081343326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44dc15ada0 msgr2=0x7f44dc0b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.890+0000 7f44e3455640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44dc15ada0 0x7f44dc0b2eb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f44d40af040 tx=0x7f44d40196b0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.890+0000 7f44e3455640 1 -- 172.21.15.26:0/1081343326 shutdown_connections 2023-12-19T16:55:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.890+0000 7f44e3455640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44dc15ada0 0x7f44dc0b2eb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.890+0000 7f44e3455640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f44c0065900 0x7f44c0067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.890+0000 7f44e3455640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44dc15a9c0 0x7f44dc0b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.890+0000 7f44e3455640 1 --2- 172.21.15.26:0/1081343326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44dc06ac80 0x7f44dc0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.890+0000 7f44e3455640 1 -- 172.21.15.26:0/1081343326 >> 172.21.15.26:0/1081343326 conn(0x7f44dc0ce730 msgr2=0x7f44dc0af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.890+0000 7f44e3455640 1 -- 172.21.15.26:0/1081343326 shutdown_connections 2023-12-19T16:55:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:13.890+0000 7f44e3455640 1 -- 172.21.15.26:0/1081343326 wait complete. 2023-12-19T16:55:13.913 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-19T16:55:13.913 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-19-16_56_00 in ~47s... 2023-12-19T16:55:13.913 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-19-16_57_00 in ~107s... 2023-12-19T16:55:13.913 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2023-12-19T16:55:13.913 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-12-19T16:55:14.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.054+0000 7fb56908c640 1 Processor -- start 2023-12-19T16:55:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.054+0000 7fb56908c640 1 -- start start 2023-12-19T16:55:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.054+0000 7fb56908c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb564159090 0x7fb5640c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.054+0000 7fb56908c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb5640ca820 0x7fb56406b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:14.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.054+0000 7fb56908c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5640cc310 con 0x7fb56406b850 2023-12-19T16:55:14.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.054+0000 7fb56908c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56406bd00 con 0x7fb564159090 2023-12-19T16:55:14.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.054+0000 7fb56908c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5640ca110 con 0x7fb5640ca820 2023-12-19T16:55:14.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.054+0000 7fb562d9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb564159090 0x7fb5640c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:14.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.054+0000 7fb55bfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb5640ca820 0x7fb56406b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:14.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.054+0000 7fb56359e640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fb56406b850 0x7fb56415cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:39874/0 (socket says 172.21.15.26:39874) 2023-12-19T16:55:14.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.054+0000 7fb56359e640 1 -- 172.21.15.26:0/1814393274 learned_addr learned my addr 172.21.15.26:0/1814393274 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:55:14.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.058+0000 7fb562d9d640 1 -- 172.21.15.26:0/1814393274 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb5640ca820 msgr2=0x7fb56406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:55:14.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.058+0000 7fb562d9d640 1 --2- 172.21.15.26:0/1814393274 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb5640ca820 0x7fb56406b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:14.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.058+0000 7fb562d9d640 1 -- 172.21.15.26:0/1814393274 >> v1:172.21.15.160:6789/0 conn(0x7fb56406b850 legacy=0x7fb56415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:14.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.058+0000 7fb562d9d640 1 -- 172.21.15.26:0/1814393274 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5640ca3f0 con 0x7fb564159090 2023-12-19T16:55:14.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.058+0000 7fb56259c640 1 -- 172.21.15.26:0/1814393274 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164591728 0 0) 0x7fb5640cc310 con 0x7fb56406b850 2023-12-19T16:55:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.058+0000 7fb562d9d640 1 --2- 172.21.15.26:0/1814393274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb564159090 0x7fb5640c97a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb55400ba50 tx=0x7fb55401e900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9faabd0d861b3f5f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:14.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.058+0000 7fb56259c640 1 -- 172.21.15.26:0/1814393274 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5540a7950 con 0x7fb564159090 2023-12-19T16:55:14.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.058+0000 7fb56259c640 1 -- 172.21.15.26:0/1814393274 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5540ab070 con 0x7fb564159090 2023-12-19T16:55:14.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.058+0000 7fb56259c640 1 -- 172.21.15.26:0/1814393274 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5540a7cd0 con 0x7fb564159090 2023-12-19T16:55:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.058+0000 7fb56908c640 1 -- 172.21.15.26:0/1814393274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb564159090 msgr2=0x7fb5640c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.058+0000 7fb56908c640 1 --2- 172.21.15.26:0/1814393274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb564159090 0x7fb5640c97a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb55400ba50 tx=0x7fb55401e900 comp rx=0 tx=0).stop 2023-12-19T16:55:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 -- 172.21.15.26:0/1814393274 shutdown_connections 2023-12-19T16:55:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 --2- 172.21.15.26:0/1814393274 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb5640ca820 0x7fb56406b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 --2- 172.21.15.26:0/1814393274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb564159090 0x7fb5640c97a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 -- 172.21.15.26:0/1814393274 >> 172.21.15.26:0/1814393274 conn(0x7fb564066160 msgr2=0x7fb5640b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 -- 172.21.15.26:0/1814393274 shutdown_connections 2023-12-19T16:55:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 -- 172.21.15.26:0/1814393274 wait complete. 2023-12-19T16:55:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 Processor -- start 2023-12-19T16:55:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 -- start start 2023-12-19T16:55:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb56406b850 0x7fb564154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ca820 0x7fb564152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb564159090 0x7fb564152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb564156c20 con 0x7fb5640ca820 2023-12-19T16:55:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56415ae10 con 0x7fb56406b850 2023-12-19T16:55:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56908c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56415a570 con 0x7fb564159090 2023-12-19T16:55:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb55bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ca820 0x7fb564152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb55bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ca820 0x7fb564152a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57590/0 (socket says 172.21.15.26:57590) 2023-12-19T16:55:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb55bfff640 1 -- 172.21.15.26:0/118522580 learned_addr learned my addr 172.21.15.26:0/118522580 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:55:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb562d9d640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb56406b850 0x7fb564154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb56359e640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb564159090 0x7fb564152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb55bfff640 1 -- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb564159090 msgr2=0x7fb564152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb55bfff640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb564159090 0x7fb564152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb55bfff640 1 -- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb56406b850 msgr2=0x7fb564154840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb55bfff640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb56406b850 0x7fb564154840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.062+0000 7fb55bfff640 1 -- 172.21.15.26:0/118522580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55400b6b0 con 0x7fb5640ca820 2023-12-19T16:55:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.066+0000 7fb55bfff640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ca820 0x7fb564152a90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00e900 tx=0x7fb54c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.066+0000 7fb55b7fe640 1 -- 172.21.15.26:0/118522580 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c009690 con 0x7fb5640ca820 2023-12-19T16:55:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.066+0000 7fb55b7fe640 1 -- 172.21.15.26:0/118522580 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c00ca00 con 0x7fb5640ca820 2023-12-19T16:55:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.066+0000 7fb56908c640 1 -- 172.21.15.26:0/118522580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56415a850 con 0x7fb5640ca820 2023-12-19T16:55:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.066+0000 7fb55b7fe640 1 -- 172.21.15.26:0/118522580 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00cdb0 con 0x7fb5640ca820 2023-12-19T16:55:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.066+0000 7fb56908c640 1 -- 172.21.15.26:0/118522580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb564153d20 con 0x7fb5640ca820 2023-12-19T16:55:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.066+0000 7fb56908c640 1 -- 172.21.15.26:0/118522580 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb564156980 con 0x7fb5640ca820 2023-12-19T16:55:14.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.066+0000 7fb55b7fe640 1 -- 172.21.15.26:0/118522580 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fb54c013470 con 0x7fb5640ca820 2023-12-19T16:55:14.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.066+0000 7fb55b7fe640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb544065900 0x7fb544067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:55:14.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.066+0000 7fb55b7fe640 1 -- 172.21.15.26:0/118522580 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fb54c083450 con 0x7fb5640ca820 2023-12-19T16:55:14.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.066+0000 7fb562d9d640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb544065900 0x7fb544067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:55:14.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.070+0000 7fb562d9d640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb544065900 0x7fb544067dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb5540b2000 tx=0x7fb5540a9070 comp rx=0 tx=0).ready entity=mgr.5965 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:55:14.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.070+0000 7fb55b7fe640 1 -- 172.21.15.26:0/118522580 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb564156980 con 0x7fb5640ca820 2023-12-19T16:55:14.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.246+0000 7fb56908c640 1 -- 172.21.15.26:0/118522580 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb564159470 con 0x7fb544065900 2023-12-19T16:55:14.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.258+0000 7fb55b7fe640 1 -- 172.21.15.26:0/118522580 <== mgr.5965 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7fb5540040c0 con 0x7fb544065900 2023-12-19T16:55:14.261 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:55:14.261 INFO:teuthology.orchestra.run.smithi026.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-12-19T16:55:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.258+0000 7fb56908c640 1 -- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb544065900 msgr2=0x7fb544067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.258+0000 7fb56908c640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb544065900 0x7fb544067dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb5540b2000 tx=0x7fb5540a9070 comp rx=0 tx=0).stop 2023-12-19T16:55:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.258+0000 7fb56908c640 1 -- 172.21.15.26:0/118522580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ca820 msgr2=0x7fb564152a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:55:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.258+0000 7fb56908c640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ca820 0x7fb564152a90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00e900 tx=0x7fb54c00edc0 comp rx=0 tx=0).stop 2023-12-19T16:55:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.258+0000 7fb56908c640 1 -- 172.21.15.26:0/118522580 shutdown_connections 2023-12-19T16:55:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.258+0000 7fb56908c640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb544065900 0x7fb544067dc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.258+0000 7fb56908c640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb564159090 0x7fb564152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.258+0000 7fb56908c640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ca820 0x7fb564152a90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.258+0000 7fb56908c640 1 --2- 172.21.15.26:0/118522580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb56406b850 0x7fb564154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:55:14.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.258+0000 7fb56908c640 1 -- 172.21.15.26:0/118522580 >> 172.21.15.26:0/118522580 conn(0x7fb564066160 msgr2=0x7fb5641638a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:55:14.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.258+0000 7fb56908c640 1 -- 172.21.15.26:0/118522580 shutdown_connections 2023-12-19T16:55:14.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:55:14.258+0000 7fb56908c640 1 -- 172.21.15.26:0/118522580 wait complete. 2023-12-19T16:55:14.282 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-12-19T16:55:14.283 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2023-12-19T16:55:14.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:14.283 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:14.294 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:14.294 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:14.294 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:15.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:15.295 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:15.307 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:15.308 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:15.308 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:16.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:16.309 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:16.321 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:16.321 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:16.321 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:17.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:17.322 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:17.334 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:17.334 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:17.334 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:18.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:18.335 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:18.347 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:18.347 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:18.347 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:19.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:19.349 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:19.361 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:19.361 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:19.361 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:20.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:20.362 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:20.375 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:20.375 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:20.375 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:21.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:21.376 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:21.388 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:21.388 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:21.388 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:22.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:22.389 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:22.401 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:22.401 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:22.402 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:23.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:23.402 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:23.415 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:23.415 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:23.415 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:24.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:24.458 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:24.470 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:24.471 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:24.471 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:25.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:25.472 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:25.484 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:25.484 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:25.484 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:26.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:26.485 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:26.497 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:26.498 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:26.498 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:27.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:27.498 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:27.511 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:27.511 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:27.511 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:28.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:28.512 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:28.524 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:28.524 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:28.524 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:29.241 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:55:29.246 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:55:29.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:29.525 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:29.537 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:29.537 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:29.537 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:30.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:30.538 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:30.550 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:30.550 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:30.550 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:31.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:31.551 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:31.563 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:31.563 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:31.564 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:32.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:32.565 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:32.577 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:32.577 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:32.577 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:33.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:33.578 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:33.590 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:33.591 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:33.591 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:34.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:34.591 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:34.603 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:34.603 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:34.603 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:35.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:35.604 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:35.617 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:35.617 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:35.617 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:36.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:36.618 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:36.630 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:36.630 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:36.630 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:37.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:37.631 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:37.644 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:37.644 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:37.644 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:38.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:38.645 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:38.657 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:38.658 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:38.658 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:39.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:39.659 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:39.671 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:39.672 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:39.672 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:40.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:40.672 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:40.685 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:40.685 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:40.685 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:41.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:41.686 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:41.698 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:41.698 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:41.698 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:42.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:42.700 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:42.712 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:42.712 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:42.712 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:43.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:43.713 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:43.725 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:43.726 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:43.726 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:44.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:44.726 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:44.739 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:44.739 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:44.739 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:45.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:45.740 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:45.752 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:45.753 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:45.753 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:46.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:46.754 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:46.766 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:46.766 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:46.766 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:47.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:47.767 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:47.780 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:47.780 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:47.780 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:48.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:48.781 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:48.793 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:48.793 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:48.794 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:49.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:49.794 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:49.806 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:49.806 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:49.807 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:50.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:50.807 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:50.820 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:50.820 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:50.820 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:51.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:51.821 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:51.833 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:51.833 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:51.834 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:52.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:52.835 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:52.847 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:52.848 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:52.848 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:53.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:53.849 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:53.861 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:53.861 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:53.861 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:54.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:54.863 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:54.876 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:54.876 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:54.876 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:55.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:55.877 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:55.889 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:55.889 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:55.889 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:56.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:56.890 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:56.902 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:56.902 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:56.902 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:57.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:57.903 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:57.915 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:57.916 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:57.916 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:58.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:58.917 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:58.929 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:58.929 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:58.929 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:55:59.322 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:55:59.326 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:55:59.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:55:59.931 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:55:59.943 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T16:55:59.943 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:55:59.943 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:00.904 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:56:00.900+0000 7f42c3cfe640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:56:00.905 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:56:00.900+0000 7f42c3cfe640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:56:00.905 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:56:00.900+0000 7f42c3cfe640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:56:00.905 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:56:00.900+0000 7f42c3cfe640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:56:00.905 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:56:00.900+0000 7f42c3cfe640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:56:00.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:00.944 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:00.956 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:00.957 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:00.957 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:00.957 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:00.958 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-19-16_56_00_UTC'] 2023-12-19T16:56:00.958 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703004913.9134097, now = 1703004960.9581497, timo = 47 2023-12-19T16:56:01.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:01.959 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:01.970 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:01.971 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:01.971 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:01.971 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:02.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:02.973 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:02.984 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:02.985 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:02.985 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:02.985 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:03.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:03.987 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:03.998 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:03.999 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:03.999 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:03.999 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:05.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:05.001 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:05.012 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:05.013 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:05.013 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:05.013 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:06.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:06.014 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:06.025 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:06.026 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:06.026 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:06.026 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:07.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:07.028 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:07.039 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:07.040 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:07.041 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:07.041 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:08.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:08.041 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:08.052 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:08.054 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:08.054 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:08.054 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:09.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:09.055 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:09.066 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:09.067 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:09.067 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:09.067 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:10.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:10.068 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:10.079 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:10.080 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:10.081 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:10.081 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:11.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:11.082 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:11.092 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:11.094 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:11.094 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:11.094 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:12.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:12.095 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:12.106 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:12.107 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:12.107 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:12.107 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:13.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:13.108 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:13.119 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:13.120 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:13.121 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:13.121 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:14.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:14.122 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:14.134 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:14.134 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:14.135 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:14.135 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:15.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:15.136 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:15.147 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:15.148 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:15.149 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:15.149 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:16.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:16.150 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:16.164 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:16.166 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:16.166 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:16.166 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:17.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:17.168 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:17.179 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:17.180 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:17.180 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:17.180 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:18.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:18.182 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:18.193 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:18.194 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:18.194 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:18.194 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:19.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:19.196 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:19.206 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:19.207 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:19.208 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:19.208 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:20.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:20.209 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:20.220 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:20.221 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:20.221 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:20.221 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:21.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:21.222 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:21.233 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:21.234 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:21.234 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:21.235 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:22.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:22.235 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:22.247 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:22.248 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:22.248 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:22.248 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:23.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:23.249 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:23.261 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:23.262 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:23.262 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:23.262 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:24.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:24.263 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:24.274 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:24.275 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:24.275 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:24.276 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:25.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:25.277 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:25.288 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:25.289 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:25.289 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:25.289 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:26.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:26.290 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:26.301 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:26.303 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:26.303 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:26.303 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:27.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:27.303 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:27.315 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:27.316 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:27.316 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:27.316 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:28.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:28.317 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:28.329 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:28.330 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:28.330 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:28.330 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:29.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:29.331 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:29.342 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:29.343 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:29.343 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:29.343 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:29.379 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:56:29.387 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:56:30.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:30.344 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:30.356 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:30.357 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:30.357 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:30.357 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:31.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:31.358 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:31.369 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:31.370 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:31.370 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:31.371 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:32.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:32.372 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:32.383 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:32.384 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:32.384 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:32.384 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:33.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:33.386 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:33.397 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:33.399 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:33.399 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:33.399 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:34.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:34.400 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:34.410 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:34.412 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:34.412 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:34.412 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:35.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:35.413 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:35.424 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:35.425 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:35.426 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:35.426 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:36.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:36.426 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:36.437 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:36.439 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:36.439 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:36.439 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:37.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:37.440 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:37.451 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:37.452 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:37.452 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:37.452 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:38.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:38.454 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:38.465 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:38.466 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:38.466 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:38.466 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:39.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:39.468 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:39.479 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:39.480 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:39.480 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:39.480 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:40.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:40.481 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:40.493 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:40.494 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:40.494 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:40.494 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:41.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:41.495 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:41.506 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:41.507 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:41.507 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:41.507 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:42.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:42.508 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:42.520 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:42.520 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:42.521 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:42.521 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:43.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:43.522 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:43.534 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:43.535 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:43.535 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:43.535 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:44.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:44.536 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:44.548 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:44.549 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:44.549 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:44.549 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:45.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:45.550 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:45.561 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:45.562 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:45.562 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:45.562 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:46.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:46.563 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:46.574 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:46.575 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:46.575 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:46.576 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:47.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:47.576 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:47.588 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:47.589 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:47.589 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:47.589 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:48.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:48.590 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:48.601 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:48.602 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:48.602 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:48.602 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:49.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:49.604 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:49.618 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:49.618 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:49.619 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:49.619 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:50.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:50.620 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:50.631 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:50.632 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:50.633 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:50.633 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:51.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:51.634 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:51.645 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:51.646 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:51.646 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:51.646 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:52.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:52.648 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:52.658 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:52.659 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:52.659 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:52.659 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:53.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:53.661 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:53.675 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:53.676 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:53.676 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:53.676 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:54.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:54.677 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:54.688 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:54.689 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:54.689 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:54.689 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:55.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:55.691 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:55.703 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:55.704 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:55.704 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:55.704 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:56.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:56.705 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:56.716 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:56.717 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:56.717 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:56.717 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:57.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:57.719 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:57.730 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:57.732 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:57.732 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:57.732 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:58.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:58.734 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:58.745 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:58.746 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:58.746 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:58.746 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:56:59.441 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:56:59.445 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:56:59.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:56:59.747 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:56:59.758 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:56:59.759 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:56:59.759 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:56:59.760 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:57:00.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:00.761 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:57:00.773 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:57:00.774 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC'} 2023-12-19T16:57:00.774 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T16:57:00.774 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:57:01.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:01.775 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:57:01.786 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:57:01.787 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_57_00_UTC 2023-12-19T16:57:01.788 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-16_56_00_UTC', 'scheduled-2023-12-19-16_57_00_UTC'} 2023-12-19T16:57:01.788 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-19-16_57_00_UTC'} 2023-12-19T16:57:01.788 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T16:57:01.788 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-19-16_57_00_UTC'] 2023-12-19T16:57:01.788 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703004913.9134097, now = 1703005021.7883856, timo = 107 2023-12-19T16:57:02.789 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-19T16:57:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.945+0000 7f51ebb0a640 1 Processor -- start 2023-12-19T16:57:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.945+0000 7f51ebb0a640 1 -- start start 2023-12-19T16:57:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51ebb0a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f51e40af4c0 0x7f51e40af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51ebb0a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e40afeb0 0x7f51e415af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:02.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51ebb0a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e40cefe0 con 0x7f51e415ab30 2023-12-19T16:57:02.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51ebb0a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e40cf280 con 0x7f51e40af4c0 2023-12-19T16:57:02.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51ebb0a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40cf560 con 0x7f51e40afeb0 2023-12-19T16:57:02.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51e98a6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f51e415ab30 0x7f51e40b6370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57184/0 (socket says 172.21.15.26:57184) 2023-12-19T16:57:02.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51e98a6640 1 -- 172.21.15.26:0/3832424444 learned_addr learned my addr 172.21.15.26:0/3832424444 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51ea0a7640 1 --2- 172.21.15.26:0/3832424444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e40afeb0 0x7f51e415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51e90a5640 1 --2- 172.21.15.26:0/3832424444 >> v2:172.21.15.160:3300/0 conn(0x7f51e40af4c0 0x7f51e40af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51e88a4640 1 -- 172.21.15.26:0/3832424444 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351180047 0 0) 0x7f51e40cefe0 con 0x7f51e415ab30 2023-12-19T16:57:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51e88a4640 1 -- 172.21.15.26:0/3832424444 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51d0003530 con 0x7f51e415ab30 2023-12-19T16:57:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51ea0a7640 1 -- 172.21.15.26:0/3832424444 >> v2:172.21.15.160:3300/0 conn(0x7f51e40af4c0 msgr2=0x7f51e40af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51ea0a7640 1 --2- 172.21.15.26:0/3832424444 >> v2:172.21.15.160:3300/0 conn(0x7f51e40af4c0 0x7f51e40af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51ea0a7640 1 -- 172.21.15.26:0/3832424444 >> v1:172.21.15.26:6789/0 conn(0x7f51e415ab30 legacy=0x7f51e40b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51e88a4640 1 -- 172.21.15.26:0/3832424444 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 911899468 0 0) 0x7f51d0003530 con 0x7f51e415ab30 2023-12-19T16:57:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.949+0000 7f51ea0a7640 1 -- 172.21.15.26:0/3832424444 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e40cf840 con 0x7f51e40afeb0 2023-12-19T16:57:02.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51ea0a7640 1 --2- 172.21.15.26:0/3832424444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e40afeb0 0x7f51e415af10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009d90 tx=0x7f51dc01c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd68f72016d1bc6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:02.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51e88a4640 1 -- 172.21.15.26:0/3832424444 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e0001680 con 0x7f51e40afeb0 2023-12-19T16:57:02.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51e88a4640 1 -- 172.21.15.26:0/3832424444 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc0a5600 con 0x7f51e40afeb0 2023-12-19T16:57:02.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51e88a4640 1 -- 172.21.15.26:0/3832424444 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc0a5970 con 0x7f51e40afeb0 2023-12-19T16:57:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3832424444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e40afeb0 msgr2=0x7f51e415af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51ebb0a640 1 --2- 172.21.15.26:0/3832424444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e40afeb0 0x7f51e415af10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009d90 tx=0x7f51dc01c7e0 comp rx=0 tx=0).stop 2023-12-19T16:57:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3832424444 shutdown_connections 2023-12-19T16:57:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51ebb0a640 1 --2- 172.21.15.26:0/3832424444 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e40afeb0 0x7f51e415af10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51ebb0a640 1 --2- 172.21.15.26:0/3832424444 >> v2:172.21.15.160:3300/0 conn(0x7f51e40af4c0 0x7f51e40af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3832424444 >> 172.21.15.26:0/3832424444 conn(0x7f51e406ef60 msgr2=0x7f51e406f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3832424444 shutdown_connections 2023-12-19T16:57:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3832424444 wait complete. 2023-12-19T16:57:02.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51ebb0a640 1 Processor -- start 2023-12-19T16:57:02.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.953+0000 7f51ebb0a640 1 -- start start 2023-12-19T16:57:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51ebb0a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e40af4c0 0x7f51e4153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51ebb0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e40afeb0 0x7f51e40c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51ebb0a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51e415ab30 0x7f51e406a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:02.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51ebb0a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e41f8e00 con 0x7f51e40afeb0 2023-12-19T16:57:02.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51ebb0a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40c97e0 con 0x7f51e415ab30 2023-12-19T16:57:02.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51ebb0a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e41f84d0 con 0x7f51e40af4c0 2023-12-19T16:57:02.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51e90a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e40afeb0 0x7f51e40c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:02.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51e90a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e40afeb0 0x7f51e40c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54118/0 (socket says 172.21.15.26:54118) 2023-12-19T16:57:02.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51e90a5640 1 -- 172.21.15.26:0/3285489168 learned_addr learned my addr 172.21.15.26:0/3285489168 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:02.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51e90a5640 1 -- 172.21.15.26:0/3285489168 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e40af4c0 msgr2=0x7f51e4153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51e90a5640 1 --2- 172.21.15.26:0/3285489168 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e40af4c0 0x7f51e4153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51e90a5640 1 -- 172.21.15.26:0/3285489168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51e415ab30 msgr2=0x7f51e406a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51e90a5640 1 --2- 172.21.15.26:0/3285489168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51e415ab30 0x7f51e406a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51e90a5640 1 -- 172.21.15.26:0/3285489168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc009a70 con 0x7f51e40afeb0 2023-12-19T16:57:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51e90a5640 1 --2- 172.21.15.26:0/3285489168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e40afeb0 0x7f51e40c8de0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f51d400e870 tx=0x7f51d400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51da7fc640 1 -- 172.21.15.26:0/3285489168 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d4009510 con 0x7f51e40afeb0 2023-12-19T16:57:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51da7fc640 1 -- 172.21.15.26:0/3285489168 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d400eec0 con 0x7f51e40afeb0 2023-12-19T16:57:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51da7fc640 1 -- 172.21.15.26:0/3285489168 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d400cb40 con 0x7f51e40afeb0 2023-12-19T16:57:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3285489168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e41f87b0 con 0x7f51e40afeb0 2023-12-19T16:57:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3285489168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e41f9290 con 0x7f51e40afeb0 2023-12-19T16:57:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3285489168 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e40b2c60 con 0x7f51e40afeb0 2023-12-19T16:57:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.957+0000 7f51da7fc640 1 -- 172.21.15.26:0/3285489168 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f51d400cca0 con 0x7f51e40afeb0 2023-12-19T16:57:02.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.961+0000 7f51da7fc640 1 --2- 172.21.15.26:0/3285489168 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51c8065bd0 0x7f51c8068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:02.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.961+0000 7f51da7fc640 1 -- 172.21.15.26:0/3285489168 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6843+0+0 (secure 0 0 0) 0x7f51d40839d0 con 0x7f51e40afeb0 2023-12-19T16:57:02.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.961+0000 7f51e98a6640 1 --2- 172.21.15.26:0/3285489168 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51c8065bd0 0x7f51c8068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:02.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.961+0000 7f51e98a6640 1 --2- 172.21.15.26:0/3285489168 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51c8065bd0 0x7f51c8068090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f51e0001ec0 tx=0x7f51e0001a30 comp rx=0 tx=0).ready entity=mgr.5965 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:02.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:02.965+0000 7f51da7fc640 1 -- 172.21.15.26:0/3285489168 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f51d4050d70 con 0x7f51e40afeb0 2023-12-19T16:57:03.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.145+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3285489168 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f51e41f8f40 con 0x7f51c8065bd0 2023-12-19T16:57:03.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.213+0000 7f51da7fc640 1 -- 172.21.15.26:0/3285489168 <== mgr.5965 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f51e41f8f40 con 0x7f51c8065bd0 2023-12-19T16:57:03.220 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path snap_test_dir1 2023-12-19T16:57:03.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.217+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3285489168 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51c8065bd0 msgr2=0x7f51c8068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:03.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.217+0000 7f51ebb0a640 1 --2- 172.21.15.26:0/3285489168 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51c8065bd0 0x7f51c8068090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f51e0001ec0 tx=0x7f51e0001a30 comp rx=0 tx=0).stop 2023-12-19T16:57:03.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.217+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3285489168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e40afeb0 msgr2=0x7f51e40c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:03.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.217+0000 7f51ebb0a640 1 --2- 172.21.15.26:0/3285489168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e40afeb0 0x7f51e40c8de0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f51d400e870 tx=0x7f51d400ec70 comp rx=0 tx=0).stop 2023-12-19T16:57:03.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.217+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3285489168 shutdown_connections 2023-12-19T16:57:03.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.217+0000 7f51ebb0a640 1 --2- 172.21.15.26:0/3285489168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51e415ab30 0x7f51e406a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:03.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.217+0000 7f51ebb0a640 1 --2- 172.21.15.26:0/3285489168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e40afeb0 0x7f51e40c8de0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:03.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.217+0000 7f51ebb0a640 1 --2- 172.21.15.26:0/3285489168 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51c8065bd0 0x7f51c8068090 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:03.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.217+0000 7f51ebb0a640 1 --2- 172.21.15.26:0/3285489168 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e40af4c0 0x7f51e4153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:03.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.217+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3285489168 >> 172.21.15.26:0/3285489168 conn(0x7f51e406ef60 msgr2=0x7f51e406b480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:03.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.217+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3285489168 shutdown_connections 2023-12-19T16:57:03.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.217+0000 7f51ebb0a640 1 -- 172.21.15.26:0/3285489168 wait complete. 2023-12-19T16:57:03.241 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-19T16:57:03.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:03.241 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T16:57:03.251 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:57:03.251 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-16_57_00_UTC 2023-12-19T16:57:03.252 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-16_56_00_UTC 2023-12-19T16:57:03.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:03.252 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-16_56_00_UTC) 2023-12-19T16:57:03.310 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-16_57_00_UTC 2023-12-19T16:57:03.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:03.310 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-16_57_00_UTC) 2023-12-19T16:57:03.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:03.370 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-19T16:57:03.424 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-19T16:57:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e359e640 1 Processor -- start 2023-12-19T16:57:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e359e640 1 -- start start 2023-12-19T16:57:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e359e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2e4071d40 0x7fd2e4071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e359e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e40d0490 con 0x7fd2e406baa0 2023-12-19T16:57:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e359e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e40cfb60 con 0x7fd2e4072120 2023-12-19T16:57:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e359e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e40cfe40 con 0x7fd2e4071d40 2023-12-19T16:57:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e2d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd2e406baa0 0x7fd2e415c400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57194/0 (socket says 172.21.15.26:57194) 2023-12-19T16:57:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e2d9d640 1 -- 172.21.15.26:0/4246788179 learned_addr learned my addr 172.21.15.26:0/4246788179 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e259c640 1 --2- 172.21.15.26:0/4246788179 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2e4071d40 0x7fd2e4071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e159a640 1 -- 172.21.15.26:0/4246788179 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529735747 0 0) 0x7fd2e40d0490 con 0x7fd2e406baa0 2023-12-19T16:57:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e159a640 1 -- 172.21.15.26:0/4246788179 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2cc003530 con 0x7fd2e406baa0 2023-12-19T16:57:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e159a640 1 -- 172.21.15.26:0/4246788179 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2213567716 0 0) 0x7fd2e40cfb60 con 0x7fd2e4072120 2023-12-19T16:57:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e259c640 1 -- 172.21.15.26:0/4246788179 >> v1:172.21.15.160:6789/0 conn(0x7fd2e4072120 legacy=0x7fd2e406b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e259c640 1 -- 172.21.15.26:0/4246788179 >> v1:172.21.15.26:6789/0 conn(0x7fd2e406baa0 legacy=0x7fd2e415c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.609+0000 7fd2e259c640 1 -- 172.21.15.26:0/4246788179 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e40d0120 con 0x7fd2e4071d40 2023-12-19T16:57:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.613+0000 7fd2e259c640 1 --2- 172.21.15.26:0/4246788179 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2e4071d40 0x7fd2e4071230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40097f0 tx=0x7fd2d401c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8ba0028d3dc75a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.613+0000 7fd2e159a640 1 -- 172.21.15.26:0/4246788179 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e40cd080 con 0x7fd2e4071d40 2023-12-19T16:57:03.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.613+0000 7fd2e159a640 1 -- 172.21.15.26:0/4246788179 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d40ab650 con 0x7fd2e4071d40 2023-12-19T16:57:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.613+0000 7fd2e159a640 1 -- 172.21.15.26:0/4246788179 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d40ab9e0 con 0x7fd2e4071d40 2023-12-19T16:57:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.613+0000 7fd2e359e640 1 -- 172.21.15.26:0/4246788179 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2e4071d40 msgr2=0x7fd2e4071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:03.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.613+0000 7fd2e359e640 1 --2- 172.21.15.26:0/4246788179 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2e4071d40 0x7fd2e4071230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40097f0 tx=0x7fd2d401c430 comp rx=0 tx=0).stop 2023-12-19T16:57:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.613+0000 7fd2e359e640 1 -- 172.21.15.26:0/4246788179 shutdown_connections 2023-12-19T16:57:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.613+0000 7fd2e359e640 1 --2- 172.21.15.26:0/4246788179 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2e4071d40 0x7fd2e4071230 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.613+0000 7fd2e359e640 1 -- 172.21.15.26:0/4246788179 >> 172.21.15.26:0/4246788179 conn(0x7fd2e40cd210 msgr2=0x7fd2e40cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.613+0000 7fd2e359e640 1 -- 172.21.15.26:0/4246788179 shutdown_connections 2023-12-19T16:57:03.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.613+0000 7fd2e359e640 1 -- 172.21.15.26:0/4246788179 wait complete. 2023-12-19T16:57:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e359e640 1 Processor -- start 2023-12-19T16:57:03.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e359e640 1 -- start start 2023-12-19T16:57:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e359e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e406baa0 0x7fd2e40af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e359e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2e4071d40 0x7fd2e41f9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e4072120 0x7fd2e4151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e359e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e40ade60 con 0x7fd2e4072120 2023-12-19T16:57:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e359e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e4155d70 con 0x7fd2e406baa0 2023-12-19T16:57:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e359e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e40ad9d0 con 0x7fd2e4071d40 2023-12-19T16:57:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e4072120 0x7fd2e4151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:03.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e4072120 0x7fd2e4151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54124/0 (socket says 172.21.15.26:54124) 2023-12-19T16:57:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e2d9d640 1 -- 172.21.15.26:0/4014544263 learned_addr learned my addr 172.21.15.26:0/4014544263 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e259c640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e406baa0 0x7fd2e40af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e1d9b640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2e4071d40 0x7fd2e41f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e2d9d640 1 -- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2e4071d40 msgr2=0x7fd2e41f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e2d9d640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2e4071d40 0x7fd2e41f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e2d9d640 1 -- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e406baa0 msgr2=0x7fd2e40af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e2d9d640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e406baa0 0x7fd2e40af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e2d9d640 1 -- 172.21.15.26:0/4014544263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d4009420 con 0x7fd2e4072120 2023-12-19T16:57:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e2d9d640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e4072120 0x7fd2e4151c30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00caa0 tx=0x7fd2dc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2d2ffd640 1 -- 172.21.15.26:0/4014544263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc00ac00 con 0x7fd2e4072120 2023-12-19T16:57:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2d2ffd640 1 -- 172.21.15.26:0/4014544263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2dc00ad60 con 0x7fd2e4072120 2023-12-19T16:57:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e359e640 1 -- 172.21.15.26:0/4014544263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e40acdc0 con 0x7fd2e4072120 2023-12-19T16:57:03.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2d2ffd640 1 -- 172.21.15.26:0/4014544263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc012620 con 0x7fd2e4072120 2023-12-19T16:57:03.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.617+0000 7fd2e359e640 1 -- 172.21.15.26:0/4014544263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e40ad380 con 0x7fd2e4072120 2023-12-19T16:57:03.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.621+0000 7fd2e359e640 1 -- 172.21.15.26:0/4014544263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e40670f0 con 0x7fd2e4072120 2023-12-19T16:57:03.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.621+0000 7fd2d2ffd640 1 -- 172.21.15.26:0/4014544263 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 85359+0+0 (secure 0 0 0) 0x7fd2dc007070 con 0x7fd2e4072120 2023-12-19T16:57:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.621+0000 7fd2d2ffd640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd2c4065bd0 0x7fd2c4068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.621+0000 7fd2d2ffd640 1 -- 172.21.15.26:0/4014544263 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6843+0+0 (secure 0 0 0) 0x7fd2dc083740 con 0x7fd2e4072120 2023-12-19T16:57:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.621+0000 7fd2e259c640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd2c4065bd0 0x7fd2c4068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.621+0000 7fd2e259c640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd2c4065bd0 0x7fd2c4068090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40af000 tx=0x7fd2d40a7000 comp rx=0 tx=0).ready entity=mgr.5965 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.625+0000 7fd2d2ffd640 1 -- 172.21.15.26:0/4014544263 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd2e40670f0 con 0x7fd2e4072120 2023-12-19T16:57:03.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:03.821+0000 7fd2e359e640 1 -- 172.21.15.26:0/4014544263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fd2e40670f0 con 0x7fd2e4072120 2023-12-19T16:57:04.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.593+0000 7fd2d2ffd640 1 -- 172.21.15.26:0/4014544263 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fd2dc050f70 con 0x7fd2e4072120 2023-12-19T16:57:04.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.593+0000 7fd2d2ffd640 1 -- 172.21.15.26:0/4014544263 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v26) v1 ==== 93+0+0 (secure 0 0 0) 0x7fd2dc07c810 con 0x7fd2e4072120 2023-12-19T16:57:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.597+0000 7fd2e359e640 1 -- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd2c4065bd0 msgr2=0x7fd2c4068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.597+0000 7fd2e359e640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd2c4065bd0 0x7fd2c4068090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd2d40af000 tx=0x7fd2d40a7000 comp rx=0 tx=0).stop 2023-12-19T16:57:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.597+0000 7fd2e359e640 1 -- 172.21.15.26:0/4014544263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e4072120 msgr2=0x7fd2e4151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.597+0000 7fd2e359e640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e4072120 0x7fd2e4151c30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00caa0 tx=0x7fd2dc00cf60 comp rx=0 tx=0).stop 2023-12-19T16:57:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.597+0000 7fd2e359e640 1 -- 172.21.15.26:0/4014544263 shutdown_connections 2023-12-19T16:57:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.597+0000 7fd2e359e640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e4072120 0x7fd2e4151c30 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.597+0000 7fd2e359e640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd2c4065bd0 0x7fd2c4068090 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:04.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.597+0000 7fd2e359e640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2e4071d40 0x7fd2e41f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:04.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.597+0000 7fd2e359e640 1 --2- 172.21.15.26:0/4014544263 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2e406baa0 0x7fd2e40af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:04.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.597+0000 7fd2e359e640 1 -- 172.21.15.26:0/4014544263 >> 172.21.15.26:0/4014544263 conn(0x7fd2e40cd210 msgr2=0x7fd2e406a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:04.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.597+0000 7fd2e359e640 1 -- 172.21.15.26:0/4014544263 shutdown_connections 2023-12-19T16:57:04.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:04.597+0000 7fd2e359e640 1 -- 172.21.15.26:0/4014544263 wait complete. 2023-12-19T16:57:04.624 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T16:57:04.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:57:04.727 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:57:04.774 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:04.769+0000 7fa5c9a30180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:57:04.777 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:04.775+0000 7f83d2e2c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:57:04.837 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T16:57:04.837 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:57:04.837 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:04.849 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:57:04.849 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:04.902 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:05.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.085+0000 7ff309666640 1 Processor -- start 2023-12-19T16:57:05.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.085+0000 7ff309666640 1 -- start start 2023-12-19T16:57:05.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.085+0000 7ff309666640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff3040cbc50 0x7ff3040b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.085+0000 7ff309666640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff3040b3310 0x7ff3040ad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.085+0000 7ff309666640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3040cc0d0 con 0x7ff30406cb30 2023-12-19T16:57:05.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.085+0000 7ff309666640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff30415ddc0 con 0x7ff3040b3310 2023-12-19T16:57:05.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.085+0000 7ff309666640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff30415e0a0 con 0x7ff3040cbc50 2023-12-19T16:57:05.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.085+0000 7ff3037fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff3040b3310 0x7ff3040ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.085+0000 7ff3037fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff3040b3310 0x7ff3040ad180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54158/0 (socket says 172.21.15.26:54158) 2023-12-19T16:57:05.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.085+0000 7ff3037fe640 1 -- 172.21.15.26:0/2947575123 learned_addr learned my addr 172.21.15.26:0/2947575123 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:05.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.085+0000 7ff2fbfff640 1 --2- 172.21.15.26:0/2947575123 >> v2:172.21.15.160:3300/0 conn(0x7ff3040cbc50 0x7ff3040b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.089+0000 7ff3037fe640 1 -- 172.21.15.26:0/2947575123 >> v2:172.21.15.160:3300/0 conn(0x7ff3040cbc50 msgr2=0x7ff3040b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:05.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.089+0000 7ff3037fe640 1 --2- 172.21.15.26:0/2947575123 >> v2:172.21.15.160:3300/0 conn(0x7ff3040cbc50 0x7ff3040b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.089+0000 7ff3037fe640 1 -- 172.21.15.26:0/2947575123 >> v1:172.21.15.160:6789/0 conn(0x7ff30406cb30 legacy=0x7ff30415d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.089+0000 7ff3037fe640 1 -- 172.21.15.26:0/2947575123 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3040cb450 con 0x7ff3040b3310 2023-12-19T16:57:05.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.089+0000 7ff3037fe640 1 --2- 172.21.15.26:0/2947575123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3040b3310 0x7ff3040ad180 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff3040667b0 tx=0x7ff2fc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6204b5bcbf91e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:05.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.089+0000 7ff3027fc640 1 -- 172.21.15.26:0/2947575123 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3040aded0 con 0x7ff3040b3310 2023-12-19T16:57:05.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.089+0000 7ff3027fc640 1 -- 172.21.15.26:0/2947575123 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc0a5880 con 0x7ff3040b3310 2023-12-19T16:57:05.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.089+0000 7ff3027fc640 1 -- 172.21.15.26:0/2947575123 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0a5c10 con 0x7ff3040b3310 2023-12-19T16:57:05.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.089+0000 7ff309666640 1 -- 172.21.15.26:0/2947575123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3040b3310 msgr2=0x7ff3040ad180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.089+0000 7ff309666640 1 --2- 172.21.15.26:0/2947575123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3040b3310 0x7ff3040ad180 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff3040667b0 tx=0x7ff2fc01c9a0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 -- 172.21.15.26:0/2947575123 shutdown_connections 2023-12-19T16:57:05.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 --2- 172.21.15.26:0/2947575123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3040b3310 0x7ff3040ad180 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 --2- 172.21.15.26:0/2947575123 >> v2:172.21.15.160:3300/0 conn(0x7ff3040cbc50 0x7ff3040b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 -- 172.21.15.26:0/2947575123 >> 172.21.15.26:0/2947575123 conn(0x7ff3040658e0 msgr2=0x7ff304065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:05.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 -- 172.21.15.26:0/2947575123 shutdown_connections 2023-12-19T16:57:05.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 -- 172.21.15.26:0/2947575123 wait complete. 2023-12-19T16:57:05.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 Processor -- start 2023-12-19T16:57:05.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 -- start start 2023-12-19T16:57:05.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff30406cb30 0x7ff3041619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3040b3310 0x7ff30412ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3040cbc50 0x7ff30406f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3041417a0 con 0x7ff3040cbc50 2023-12-19T16:57:05.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff304161ef0 con 0x7ff30406cb30 2023-12-19T16:57:05.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff309666640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040af0e0 con 0x7ff3040b3310 2023-12-19T16:57:05.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff3037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3040cbc50 0x7ff30406f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff302ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff30406cb30 0x7ff3041619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff3037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3040cbc50 0x7ff30406f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54162/0 (socket says 172.21.15.26:54162) 2023-12-19T16:57:05.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff2fbfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3040b3310 0x7ff30412ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff3037fe640 1 -- 172.21.15.26:0/422670204 learned_addr learned my addr 172.21.15.26:0/422670204 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:05.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff3037fe640 1 -- 172.21.15.26:0/422670204 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3040b3310 msgr2=0x7ff30412ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff3037fe640 1 --2- 172.21.15.26:0/422670204 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3040b3310 0x7ff30412ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff3037fe640 1 -- 172.21.15.26:0/422670204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff30406cb30 msgr2=0x7ff3041619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff3037fe640 1 --2- 172.21.15.26:0/422670204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff30406cb30 0x7ff3041619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff3037fe640 1 -- 172.21.15.26:0/422670204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc009b30 con 0x7ff3040cbc50 2023-12-19T16:57:05.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.093+0000 7ff3037fe640 1 --2- 172.21.15.26:0/422670204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3040cbc50 0x7ff30406f7c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0195b0 tx=0x7ff2fc019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.097+0000 7ff2fb7fe640 1 -- 172.21.15.26:0/422670204 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0b8670 con 0x7ff3040cbc50 2023-12-19T16:57:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.097+0000 7ff2fb7fe640 1 -- 172.21.15.26:0/422670204 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc0b87d0 con 0x7ff3040cbc50 2023-12-19T16:57:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.097+0000 7ff2fb7fe640 1 -- 172.21.15.26:0/422670204 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0b8b80 con 0x7ff3040cbc50 2023-12-19T16:57:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.097+0000 7ff309666640 1 -- 172.21.15.26:0/422670204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3040af3c0 con 0x7ff3040cbc50 2023-12-19T16:57:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.097+0000 7ff309666640 1 -- 172.21.15.26:0/422670204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3040af980 con 0x7ff3040cbc50 2023-12-19T16:57:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.097+0000 7ff309666640 1 -- 172.21.15.26:0/422670204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff304074270 con 0x7ff3040cbc50 2023-12-19T16:57:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.097+0000 7ff2fb7fe640 1 -- 172.21.15.26:0/422670204 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7ff2fc0b8ce0 con 0x7ff3040cbc50 2023-12-19T16:57:05.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.097+0000 7ff2fb7fe640 1 --2- 172.21.15.26:0/422670204 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff2e4065b80 0x7ff2e4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.101+0000 7ff2fb7fe640 1 -- 172.21.15.26:0/422670204 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6843+0+0 (secure 0 0 0) 0x7ff2fc11ec50 con 0x7ff3040cbc50 2023-12-19T16:57:05.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.101+0000 7ff302ffd640 1 -- 172.21.15.26:0/422670204 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff2e4065b80 msgr2=0x7ff2e4068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:05.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.101+0000 7ff302ffd640 1 --2- 172.21.15.26:0/422670204 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff2e4065b80 0x7ff2e4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:05.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.105+0000 7ff2fb7fe640 1 -- 172.21.15.26:0/422670204 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff2fc0ec250 con 0x7ff3040cbc50 2023-12-19T16:57:05.193 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:05.191+0000 7f83d2e2c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:57:05.220 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:05.217+0000 7fa5c9a30180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:57:05.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.265+0000 7ff309666640 1 -- 172.21.15.26:0/422670204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3040acf60 con 0x7ff3040cbc50 2023-12-19T16:57:05.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.265+0000 7ff2fb7fe640 1 -- 172.21.15.26:0/422670204 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17796 (secure 0 0 0) 0x7ff2fc0bc090 con 0x7ff3040cbc50 2023-12-19T16:57:05.271 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:05.271 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":64,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:56:13.625893+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:54:58.260985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T16:54:59.644467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.f","mappings":[{"from":0,"to":4}]},{"pgid":"7.2e","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6832/38098":"2023-12-20T16:55:12.385158+0000","172.21.15.160:6833/38098":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/251239631":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/1854082733":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/260287399":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/71378321":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/1751160424":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/3640035110":"2023-12-20T16:55:12.385158+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.269+0000 7ff309666640 1 -- 172.21.15.26:0/422670204 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff2e4065b80 msgr2=0x7ff2e4068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.269+0000 7ff309666640 1 --2- 172.21.15.26:0/422670204 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff2e4065b80 0x7ff2e4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.269+0000 7ff309666640 1 -- 172.21.15.26:0/422670204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3040cbc50 msgr2=0x7ff30406f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.269+0000 7ff309666640 1 --2- 172.21.15.26:0/422670204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3040cbc50 0x7ff30406f7c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0195b0 tx=0x7ff2fc019740 comp rx=0 tx=0).stop 2023-12-19T16:57:05.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.269+0000 7ff309666640 1 -- 172.21.15.26:0/422670204 shutdown_connections 2023-12-19T16:57:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.269+0000 7ff309666640 1 --2- 172.21.15.26:0/422670204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3040cbc50 0x7ff30406f7c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.269+0000 7ff309666640 1 --2- 172.21.15.26:0/422670204 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff2e4065b80 0x7ff2e4068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.269+0000 7ff309666640 1 --2- 172.21.15.26:0/422670204 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3040b3310 0x7ff30412ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.269+0000 7ff309666640 1 --2- 172.21.15.26:0/422670204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff30406cb30 0x7ff3041619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.269+0000 7ff309666640 1 -- 172.21.15.26:0/422670204 >> 172.21.15.26:0/422670204 conn(0x7ff3040658e0 msgr2=0x7ff304146190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:05.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.269+0000 7ff309666640 1 -- 172.21.15.26:0/422670204 shutdown_connections 2023-12-19T16:57:05.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.269+0000 7ff309666640 1 -- 172.21.15.26:0/422670204 wait complete. 2023-12-19T16:57:05.295 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:54:58.260985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:54:59.644467+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:57:05.295 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:57:05.342 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:05.339+0000 7f83d2e2c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:57:05.379 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:05.373+0000 7fa5c9a30180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:57:05.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f81500b6640 1 Processor -- start 2023-12-19T16:57:05.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f81500b6640 1 -- start start 2023-12-19T16:57:05.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f81500b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8148159140 0x7f81480ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f81500b6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f81480cd190 0x7f814806ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f81500b6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814806f490 con 0x7f814806b1c0 2023-12-19T16:57:05.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f81500b6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814806b650 con 0x7f81480cd190 2023-12-19T16:57:05.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f81500b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8148159520 con 0x7f8148159140 2023-12-19T16:57:05.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f814de52640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8148159140 0x7f81480ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f814d651640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f81480cd190 0x7f814806ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f814de52640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8148159140 0x7f81480ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49928/0 (socket says 172.21.15.26:49928) 2023-12-19T16:57:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f814de52640 1 -- 172.21.15.26:0/2964219388 learned_addr learned my addr 172.21.15.26:0/2964219388 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f814de52640 1 -- 172.21.15.26:0/2964219388 >> v2:172.21.15.160:3300/0 conn(0x7f81480cd190 msgr2=0x7f814806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f814de52640 1 --2- 172.21.15.26:0/2964219388 >> v2:172.21.15.160:3300/0 conn(0x7f81480cd190 0x7f814806ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f814de52640 1 -- 172.21.15.26:0/2964219388 >> v1:172.21.15.160:6789/0 conn(0x7f814806b1c0 legacy=0x7f814815ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f814de52640 1 -- 172.21.15.26:0/2964219388 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81480cd690 con 0x7f8148159140 2023-12-19T16:57:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.441+0000 7f814d651640 1 --2- 172.21.15.26:0/2964219388 >> v2:172.21.15.160:3300/0 conn(0x7f81480cd190 0x7f814806ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:05.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.445+0000 7f814de52640 1 --2- 172.21.15.26:0/2964219388 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8148159140 0x7f81480ccc50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8144009870 tx=0x7f814401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76f6cf182f5328db server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:05.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.445+0000 7f814ce50640 1 -- 172.21.15.26:0/2964219388 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81440ad680 con 0x7f8148159140 2023-12-19T16:57:05.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.445+0000 7f814ce50640 1 -- 172.21.15.26:0/2964219388 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81440ad7e0 con 0x7f8148159140 2023-12-19T16:57:05.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.445+0000 7f814ce50640 1 -- 172.21.15.26:0/2964219388 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81440adb70 con 0x7f8148159140 2023-12-19T16:57:05.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.445+0000 7f81500b6640 1 -- 172.21.15.26:0/2964219388 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8148159140 msgr2=0x7f81480ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.445+0000 7f81500b6640 1 --2- 172.21.15.26:0/2964219388 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8148159140 0x7f81480ccc50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8144009870 tx=0x7f814401c510 comp rx=0 tx=0).stop 2023-12-19T16:57:05.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.445+0000 7f81500b6640 1 -- 172.21.15.26:0/2964219388 shutdown_connections 2023-12-19T16:57:05.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.445+0000 7f81500b6640 1 --2- 172.21.15.26:0/2964219388 >> v2:172.21.15.160:3300/0 conn(0x7f81480cd190 0x7f814806ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.445+0000 7f81500b6640 1 --2- 172.21.15.26:0/2964219388 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8148159140 0x7f81480ccc50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.445+0000 7f81500b6640 1 -- 172.21.15.26:0/2964219388 >> 172.21.15.26:0/2964219388 conn(0x7f81480ce730 msgr2=0x7f81480ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.445+0000 7f81500b6640 1 -- 172.21.15.26:0/2964219388 shutdown_connections 2023-12-19T16:57:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.445+0000 7f81500b6640 1 -- 172.21.15.26:0/2964219388 wait complete. 2023-12-19T16:57:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f81500b6640 1 Processor -- start 2023-12-19T16:57:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f81500b6640 1 -- start start 2023-12-19T16:57:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f81500b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f814806b1c0 0x7f81480c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f81500b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81480cd190 0x7f81480b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f81500b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8148159140 0x7f8148167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f81500b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81480b1820 con 0x7f8148159140 2023-12-19T16:57:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f81500b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81480b1b00 con 0x7f81480cd190 2023-12-19T16:57:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f81500b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81480b1de0 con 0x7f814806b1c0 2023-12-19T16:57:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f814de52640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f814806b1c0 0x7f81480c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f814e653640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8148159140 0x7f8148167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f814d651640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81480cd190 0x7f81480b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f814de52640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f814806b1c0 0x7f81480c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:49942/0 (socket says 172.21.15.26:49942) 2023-12-19T16:57:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f814de52640 1 -- 172.21.15.26:0/2609823063 learned_addr learned my addr 172.21.15.26:0/2609823063 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f814de52640 1 -- 172.21.15.26:0/2609823063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81480cd190 msgr2=0x7f81480b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f814de52640 1 --2- 172.21.15.26:0/2609823063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81480cd190 0x7f81480b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f814de52640 1 -- 172.21.15.26:0/2609823063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8148159140 msgr2=0x7f8148167ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f814de52640 1 --2- 172.21.15.26:0/2609823063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8148159140 0x7f8148167ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f814de52640 1 -- 172.21.15.26:0/2609823063 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8138009420 con 0x7f814806b1c0 2023-12-19T16:57:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f814de52640 1 --2- 172.21.15.26:0/2609823063 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f814806b1c0 0x7f81480c4c30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f814401abc0 tx=0x7f81440192b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f813e7fc640 1 -- 172.21.15.26:0/2609823063 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81440ade20 con 0x7f814806b1c0 2023-12-19T16:57:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f813e7fc640 1 -- 172.21.15.26:0/2609823063 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81440b8420 con 0x7f814806b1c0 2023-12-19T16:57:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f81500b6640 1 -- 172.21.15.26:0/2609823063 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81440094d0 con 0x7f814806b1c0 2023-12-19T16:57:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f813e7fc640 1 -- 172.21.15.26:0/2609823063 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81440b87d0 con 0x7f814806b1c0 2023-12-19T16:57:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.449+0000 7f81500b6640 1 -- 172.21.15.26:0/2609823063 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81480b23a0 con 0x7f814806b1c0 2023-12-19T16:57:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.453+0000 7f81500b6640 1 -- 172.21.15.26:0/2609823063 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8148074270 con 0x7f814806b1c0 2023-12-19T16:57:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.453+0000 7f813e7fc640 1 -- 172.21.15.26:0/2609823063 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f81440b4040 con 0x7f814806b1c0 2023-12-19T16:57:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.453+0000 7f813e7fc640 1 --2- 172.21.15.26:0/2609823063 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f812c065bb0 0x7f812c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.453+0000 7f813e7fc640 1 -- 172.21.15.26:0/2609823063 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6843+0+0 (secure 0 0 0) 0x7f814411db00 con 0x7f814806b1c0 2023-12-19T16:57:05.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.453+0000 7f814d651640 1 -- 172.21.15.26:0/2609823063 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f812c065bb0 msgr2=0x7f812c068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:05.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.453+0000 7f814d651640 1 --2- 172.21.15.26:0/2609823063 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f812c065bb0 0x7f812c068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:05.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.457+0000 7f813e7fc640 1 -- 172.21.15.26:0/2609823063 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f814411fe60 con 0x7f814806b1c0 2023-12-19T16:57:05.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.621+0000 7f81500b6640 1 -- 172.21.15.26:0/2609823063 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f81480b2590 con 0x7f814806b1c0 2023-12-19T16:57:05.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.625+0000 7f813e7fc640 1 -- 172.21.15.26:0/2609823063 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f81440eae40 con 0x7f814806b1c0 2023-12-19T16:57:05.628 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:57:05.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.625+0000 7f81500b6640 1 -- 172.21.15.26:0/2609823063 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f812c065bb0 msgr2=0x7f812c068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:05.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.625+0000 7f81500b6640 1 --2- 172.21.15.26:0/2609823063 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f812c065bb0 0x7f812c068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.625+0000 7f81500b6640 1 -- 172.21.15.26:0/2609823063 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f814806b1c0 msgr2=0x7f81480c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.625+0000 7f81500b6640 1 --2- 172.21.15.26:0/2609823063 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f814806b1c0 0x7f81480c4c30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f814401abc0 tx=0x7f81440192b0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.625+0000 7f81500b6640 1 -- 172.21.15.26:0/2609823063 shutdown_connections 2023-12-19T16:57:05.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.625+0000 7f81500b6640 1 --2- 172.21.15.26:0/2609823063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8148159140 0x7f8148167ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.625+0000 7f81500b6640 1 --2- 172.21.15.26:0/2609823063 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81480cd190 0x7f81480b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.625+0000 7f81500b6640 1 --2- 172.21.15.26:0/2609823063 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f812c065bb0 0x7f812c068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.625+0000 7f81500b6640 1 --2- 172.21.15.26:0/2609823063 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f814806b1c0 0x7f81480c4c30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.625+0000 7f81500b6640 1 -- 172.21.15.26:0/2609823063 >> 172.21.15.26:0/2609823063 conn(0x7f81480ce730 msgr2=0x7f8148151dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:05.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.625+0000 7f81500b6640 1 -- 172.21.15.26:0/2609823063 shutdown_connections 2023-12-19T16:57:05.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.625+0000 7f81500b6640 1 -- 172.21.15.26:0/2609823063 wait complete. 2023-12-19T16:57:05.651 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:57:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e736d640 1 Processor -- start 2023-12-19T16:57:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e736d640 1 -- start start 2023-12-19T16:57:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e736d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95e015cde0 0x7f95e00b8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e736d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f95e015d650 0x7f95e00d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e736d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e00b5bd0 con 0x7f95e015d1c0 2023-12-19T16:57:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e736d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e015e380 con 0x7f95e015cde0 2023-12-19T16:57:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e736d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e00b8b40 con 0x7f95e015d650 2023-12-19T16:57:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e5109640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95e015cde0 0x7f95e00b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e5109640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95e015cde0 0x7f95e00b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54174/0 (socket says 172.21.15.26:54174) 2023-12-19T16:57:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e5109640 1 -- 172.21.15.26:0/2294813797 learned_addr learned my addr 172.21.15.26:0/2294813797 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e4908640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f95e015d1c0 0x7f95e00b45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57208/0 (socket says 172.21.15.26:57208) 2023-12-19T16:57:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e590a640 1 --2- 172.21.15.26:0/2294813797 >> v2:172.21.15.160:3300/0 conn(0x7f95e015d650 0x7f95e00d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e5109640 1 -- 172.21.15.26:0/2294813797 >> v2:172.21.15.160:3300/0 conn(0x7f95e015d650 msgr2=0x7f95e00d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e5109640 1 --2- 172.21.15.26:0/2294813797 >> v2:172.21.15.160:3300/0 conn(0x7f95e015d650 0x7f95e00d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.797+0000 7f95e5109640 1 -- 172.21.15.26:0/2294813797 >> v1:172.21.15.26:6789/0 conn(0x7f95e015d1c0 legacy=0x7f95e00b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.801+0000 7f95e5109640 1 -- 172.21.15.26:0/2294813797 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e00d0ee0 con 0x7f95e015cde0 2023-12-19T16:57:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.801+0000 7f95e590a640 1 --2- 172.21.15.26:0/2294813797 >> v2:172.21.15.160:3300/0 conn(0x7f95e015d650 0x7f95e00d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.801+0000 7f95e5109640 1 --2- 172.21.15.26:0/2294813797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e015cde0 0x7f95e00b8600 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f95e00d0e50 tx=0x7f95dc01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02f42a4bcc6c6ac server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.801+0000 7f95d7fff640 1 -- 172.21.15.26:0/2294813797 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0a89f0 con 0x7f95e015cde0 2023-12-19T16:57:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.801+0000 7f95d7fff640 1 -- 172.21.15.26:0/2294813797 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc0a8b50 con 0x7f95e015cde0 2023-12-19T16:57:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.801+0000 7f95d7fff640 1 -- 172.21.15.26:0/2294813797 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0b0440 con 0x7f95e015cde0 2023-12-19T16:57:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.801+0000 7f95e736d640 1 -- 172.21.15.26:0/2294813797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e015cde0 msgr2=0x7f95e00b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.801+0000 7f95e736d640 1 --2- 172.21.15.26:0/2294813797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e015cde0 0x7f95e00b8600 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f95e00d0e50 tx=0x7f95dc01ecf0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.801+0000 7f95e736d640 1 -- 172.21.15.26:0/2294813797 shutdown_connections 2023-12-19T16:57:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.801+0000 7f95e736d640 1 --2- 172.21.15.26:0/2294813797 >> v2:172.21.15.160:3300/0 conn(0x7f95e015d650 0x7f95e00d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.801+0000 7f95e736d640 1 --2- 172.21.15.26:0/2294813797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e015cde0 0x7f95e00b8600 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.801+0000 7f95e736d640 1 -- 172.21.15.26:0/2294813797 >> 172.21.15.26:0/2294813797 conn(0x7f95e0065ee0 msgr2=0x7f95e00d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e736d640 1 -- 172.21.15.26:0/2294813797 shutdown_connections 2023-12-19T16:57:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e736d640 1 -- 172.21.15.26:0/2294813797 wait complete. 2023-12-19T16:57:05.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e736d640 1 Processor -- start 2023-12-19T16:57:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e736d640 1 -- start start 2023-12-19T16:57:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e736d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e015cde0 0x7f95e01f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e736d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e015d1c0 0x7f95e014df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e736d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95e00d1e10 0x7f95e00d22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e736d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e00d2800 con 0x7f95e015d1c0 2023-12-19T16:57:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e736d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e01f34d0 con 0x7f95e015cde0 2023-12-19T16:57:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e736d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e00a9880 con 0x7f95e00d1e10 2023-12-19T16:57:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e4908640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e015d1c0 0x7f95e014df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e4908640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e015d1c0 0x7f95e014df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54178/0 (socket says 172.21.15.26:54178) 2023-12-19T16:57:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e4908640 1 -- 172.21.15.26:0/3560320512 learned_addr learned my addr 172.21.15.26:0/3560320512 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e5109640 1 --2- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e015cde0 0x7f95e01f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e590a640 1 --2- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95e00d1e10 0x7f95e00d22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e4908640 1 -- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95e00d1e10 msgr2=0x7f95e00d22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e4908640 1 --2- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95e00d1e10 0x7f95e00d22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e4908640 1 -- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e015cde0 msgr2=0x7f95e01f2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e4908640 1 --2- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e015cde0 0x7f95e01f2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e4908640 1 -- 172.21.15.26:0/3560320512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc0094d0 con 0x7f95e015d1c0 2023-12-19T16:57:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.805+0000 7f95e4908640 1 --2- 172.21.15.26:0/3560320512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e015d1c0 0x7f95e014df20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f95d000c490 tx=0x7f95d000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.809+0000 7f95d5ffb640 1 -- 172.21.15.26:0/3560320512 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d00074e0 con 0x7f95e015d1c0 2023-12-19T16:57:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.809+0000 7f95d5ffb640 1 -- 172.21.15.26:0/3560320512 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d0007640 con 0x7f95e015d1c0 2023-12-19T16:57:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.809+0000 7f95d5ffb640 1 -- 172.21.15.26:0/3560320512 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0004a10 con 0x7f95e015d1c0 2023-12-19T16:57:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.809+0000 7f95e736d640 1 -- 172.21.15.26:0/3560320512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e00a9b60 con 0x7f95e015d1c0 2023-12-19T16:57:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.809+0000 7f95e736d640 1 -- 172.21.15.26:0/3560320512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e00aa120 con 0x7f95e015d1c0 2023-12-19T16:57:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.809+0000 7f95e736d640 1 -- 172.21.15.26:0/3560320512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e0074270 con 0x7f95e015d1c0 2023-12-19T16:57:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.809+0000 7f95d5ffb640 1 -- 172.21.15.26:0/3560320512 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f95d0004b70 con 0x7f95e015d1c0 2023-12-19T16:57:05.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.809+0000 7f95d5ffb640 1 --2- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f95c4065bd0 0x7f95c4068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:05.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.813+0000 7f95d5ffb640 1 -- 172.21.15.26:0/3560320512 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f95d00830e0 con 0x7f95e015d1c0 2023-12-19T16:57:05.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.813+0000 7f95e5109640 1 -- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f95c4065bd0 msgr2=0x7f95c4068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:05.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.813+0000 7f95e5109640 1 --2- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f95c4065bd0 0x7f95c4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:05.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.817+0000 7f95d5ffb640 1 -- 172.21.15.26:0/3560320512 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f95d0050550 con 0x7f95e015d1c0 2023-12-19T16:57:05.832 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:05.827+0000 7f83d2e2c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:57:05.892 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:05.887+0000 7f83d2e2c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:57:05.908 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:05.905+0000 7fa5c9a30180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:57:05.948 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:05.943+0000 7f83d2e2c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:57:05.969 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:05.965+0000 7fa5c9a30180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:57:05.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.973+0000 7f95e736d640 1 -- 172.21.15.26:0/3560320512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95e00b2040 con 0x7f95e015d1c0 2023-12-19T16:57:05.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.973+0000 7f95d5ffb640 1 -- 172.21.15.26:0/3560320512 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7f95d0053b70 con 0x7f95e015d1c0 2023-12-19T16:57:05.980 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:57:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.977+0000 7f95e736d640 1 -- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f95c4065bd0 msgr2=0x7f95c4068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.977+0000 7f95e736d640 1 --2- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f95c4065bd0 0x7f95c4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.977+0000 7f95e736d640 1 -- 172.21.15.26:0/3560320512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e015d1c0 msgr2=0x7f95e014df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.977+0000 7f95e736d640 1 --2- 172.21.15.26:0/3560320512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e015d1c0 0x7f95e014df20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f95d000c490 tx=0x7f95d000c950 comp rx=0 tx=0).stop 2023-12-19T16:57:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.977+0000 7f95e736d640 1 -- 172.21.15.26:0/3560320512 shutdown_connections 2023-12-19T16:57:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.977+0000 7f95e736d640 1 --2- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f95c4065bd0 0x7f95c4068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.977+0000 7f95e736d640 1 --2- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95e00d1e10 0x7f95e00d22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.977+0000 7f95e736d640 1 --2- 172.21.15.26:0/3560320512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e015d1c0 0x7f95e014df20 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.977+0000 7f95e736d640 1 --2- 172.21.15.26:0/3560320512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e015cde0 0x7f95e01f2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.977+0000 7f95e736d640 1 -- 172.21.15.26:0/3560320512 >> 172.21.15.26:0/3560320512 conn(0x7f95e0065ee0 msgr2=0x7f95e00d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.977+0000 7f95e736d640 1 -- 172.21.15.26:0/3560320512 shutdown_connections 2023-12-19T16:57:05.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:05.977+0000 7f95e736d640 1 -- 172.21.15.26:0/3560320512 wait complete. 2023-12-19T16:57:06.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:57:06.031 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:06.025+0000 7fa5c9a30180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:57:06.037 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:06.035+0000 7f83d2e2c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:57:06.095 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:06.091+0000 7f83d2e2c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:57:06.123 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:06.117+0000 7fa5c9a30180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:57:06.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.145+0000 7f64fbd6d640 1 Processor -- start 2023-12-19T16:57:06.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.145+0000 7f64fbd6d640 1 -- start start 2023-12-19T16:57:06.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.145+0000 7f64fbd6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f64f406ad60 0x7f64f406b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:06.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.145+0000 7f64fbd6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64f4161520 0x7f64f4069d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:06.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.145+0000 7f64fbd6d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f64f406a260 0x7f64f40cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:06.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.145+0000 7f64fbd6d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f406b680 con 0x7f64f406ad60 2023-12-19T16:57:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.145+0000 7f64fbd6d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f40ccbc0 con 0x7f64f406a260 2023-12-19T16:57:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.145+0000 7f64fbd6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f40cd180 con 0x7f64f4161520 2023-12-19T16:57:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.145+0000 7f64f9b09640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f64f406ad60 0x7f64f406b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.145+0000 7f64f9b09640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f64f406ad60 0x7f64f406b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55294/0 (socket says 172.21.15.26:55294) 2023-12-19T16:57:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.145+0000 7f64f9b09640 1 -- 172.21.15.26:0/13259932 learned_addr learned my addr 172.21.15.26:0/13259932 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.149+0000 7f64fa30a640 1 --2- 172.21.15.26:0/13259932 >> v2:172.21.15.160:3300/0 conn(0x7f64f406a260 0x7f64f40cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.149+0000 7f64f9308640 1 --2- 172.21.15.26:0/13259932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64f4161520 0x7f64f4069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.149+0000 7f64f9b09640 1 -- 172.21.15.26:0/13259932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64f4161520 msgr2=0x7f64f4069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.149+0000 7f64f9b09640 1 --2- 172.21.15.26:0/13259932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64f4161520 0x7f64f4069d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.149+0000 7f64f9b09640 1 -- 172.21.15.26:0/13259932 >> v2:172.21.15.160:3300/0 conn(0x7f64f406a260 msgr2=0x7f64f40cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.149+0000 7f64f9b09640 1 --2- 172.21.15.26:0/13259932 >> v2:172.21.15.160:3300/0 conn(0x7f64f406a260 0x7f64f40cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.149+0000 7f64f9b09640 1 -- 172.21.15.26:0/13259932 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f406b8b0 con 0x7f64f406ad60 2023-12-19T16:57:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.149+0000 7f64f9b09640 1 --2- 172.21.15.26:0/13259932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f406ad60 0x7f64f406b140 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f64f40b0440 tx=0x7f64f001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ea353586ef1c96 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:06.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.149+0000 7f64f8b07640 1 -- 172.21.15.26:0/13259932 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f00ac640 con 0x7f64f406ad60 2023-12-19T16:57:06.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.149+0000 7f64f8b07640 1 -- 172.21.15.26:0/13259932 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f00ac7a0 con 0x7f64f406ad60 2023-12-19T16:57:06.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.149+0000 7f64f8b07640 1 -- 172.21.15.26:0/13259932 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f00acb30 con 0x7f64f406ad60 2023-12-19T16:57:06.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 -- 172.21.15.26:0/13259932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f406ad60 msgr2=0x7f64f406b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:06.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 --2- 172.21.15.26:0/13259932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f406ad60 0x7f64f406b140 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f64f40b0440 tx=0x7f64f001c440 comp rx=0 tx=0).stop 2023-12-19T16:57:06.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 -- 172.21.15.26:0/13259932 shutdown_connections 2023-12-19T16:57:06.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 --2- 172.21.15.26:0/13259932 >> v2:172.21.15.160:3300/0 conn(0x7f64f406a260 0x7f64f40cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 --2- 172.21.15.26:0/13259932 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64f4161520 0x7f64f4069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 --2- 172.21.15.26:0/13259932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f406ad60 0x7f64f406b140 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 -- 172.21.15.26:0/13259932 >> 172.21.15.26:0/13259932 conn(0x7f64f406ef60 msgr2=0x7f64f406f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:06.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 -- 172.21.15.26:0/13259932 shutdown_connections 2023-12-19T16:57:06.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 -- 172.21.15.26:0/13259932 wait complete. 2023-12-19T16:57:06.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 Processor -- start 2023-12-19T16:57:06.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 -- start start 2023-12-19T16:57:06.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f406a260 0x7f64f41562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:06.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64f406ad60 0x7f64f415f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:06.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f4161520 0x7f64f415fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:06.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f4159420 con 0x7f64f4161520 2023-12-19T16:57:06.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f415e210 con 0x7f64f406a260 2023-12-19T16:57:06.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fbd6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f415e480 con 0x7f64f406ad60 2023-12-19T16:57:06.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fa30a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f4161520 0x7f64f415fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:06.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fa30a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f4161520 0x7f64f415fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55308/0 (socket says 172.21.15.26:55308) 2023-12-19T16:57:06.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64fa30a640 1 -- 172.21.15.26:0/790864642 learned_addr learned my addr 172.21.15.26:0/790864642 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:06.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64f9308640 1 --2- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64f406ad60 0x7f64f415f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:06.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.153+0000 7f64f9b09640 1 --2- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f406a260 0x7f64f41562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:06.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64fa30a640 1 -- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64f406ad60 msgr2=0x7f64f415f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:06.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64fa30a640 1 --2- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64f406ad60 0x7f64f415f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64fa30a640 1 -- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f406a260 msgr2=0x7f64f41562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:06.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64fa30a640 1 --2- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f406a260 0x7f64f41562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64fa30a640 1 -- 172.21.15.26:0/790864642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f00094d0 con 0x7f64f4161520 2023-12-19T16:57:06.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64fa30a640 1 --2- 172.21.15.26:0/790864642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f4161520 0x7f64f415fd50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009cb0 tx=0x7f64ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:06.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64ea7fc640 1 -- 172.21.15.26:0/790864642 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec009b20 con 0x7f64f4161520 2023-12-19T16:57:06.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64ea7fc640 1 -- 172.21.15.26:0/790864642 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec00f040 con 0x7f64f4161520 2023-12-19T16:57:06.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64ea7fc640 1 -- 172.21.15.26:0/790864642 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0146c0 con 0x7f64f4161520 2023-12-19T16:57:06.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64fbd6d640 1 -- 172.21.15.26:0/790864642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f415e760 con 0x7f64f4161520 2023-12-19T16:57:06.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64fbd6d640 1 -- 172.21.15.26:0/790864642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f4156800 con 0x7f64f4161520 2023-12-19T16:57:06.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64fbd6d640 1 -- 172.21.15.26:0/790864642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f4074270 con 0x7f64f4161520 2023-12-19T16:57:06.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64ea7fc640 1 -- 172.21.15.26:0/790864642 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f64ec014820 con 0x7f64f4161520 2023-12-19T16:57:06.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64ea7fc640 1 --2- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f64d8065b80 0x7f64d8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:06.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64ea7fc640 1 -- 172.21.15.26:0/790864642 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f64ec084530 con 0x7f64f4161520 2023-12-19T16:57:06.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64f9b09640 1 -- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f64d8065b80 msgr2=0x7f64d8068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:06.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.157+0000 7f64f9b09640 1 --2- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f64d8065b80 0x7f64d8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:06.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.165+0000 7f64ea7fc640 1 -- 172.21.15.26:0/790864642 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f64ec0518b0 con 0x7f64f4161520 2023-12-19T16:57:06.181 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:06.179+0000 7f83d2e2c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:57:06.189 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:06.181+0000 7fa5c9a30180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:57:06.238 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:06.235+0000 7f83d2e2c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:57:06.277 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:06.273+0000 7fa5c9a30180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:57:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.329+0000 7f64fbd6d640 1 -- 172.21.15.26:0/790864642 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64f415ec10 con 0x7f64f4161520 2023-12-19T16:57:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.329+0000 7f64ea7fc640 1 -- 172.21.15.26:0/790864642 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64ec054ed0 con 0x7f64f4161520 2023-12-19T16:57:06.336 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:57:06.336 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:06.329+0000 7fa5c9a30180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:57:06.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.333+0000 7f64fbd6d640 1 -- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f64d8065b80 msgr2=0x7f64d8068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.333+0000 7f64fbd6d640 1 --2- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f64d8065b80 0x7f64d8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.333+0000 7f64fbd6d640 1 -- 172.21.15.26:0/790864642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f4161520 msgr2=0x7f64f415fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.333+0000 7f64fbd6d640 1 --2- 172.21.15.26:0/790864642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f4161520 0x7f64f415fd50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f64ec009cb0 tx=0x7f64ec00c490 comp rx=0 tx=0).stop 2023-12-19T16:57:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.333+0000 7f64fbd6d640 1 -- 172.21.15.26:0/790864642 shutdown_connections 2023-12-19T16:57:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.333+0000 7f64fbd6d640 1 --2- 172.21.15.26:0/790864642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64f4161520 0x7f64f415fd50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.333+0000 7f64fbd6d640 1 --2- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f64d8065b80 0x7f64d8068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.333+0000 7f64fbd6d640 1 --2- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f64f406ad60 0x7f64f415f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.333+0000 7f64fbd6d640 1 --2- 172.21.15.26:0/790864642 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64f406a260 0x7f64f41562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.333+0000 7f64fbd6d640 1 -- 172.21.15.26:0/790864642 >> 172.21.15.26:0/790864642 conn(0x7f64f406ef60 msgr2=0x7f64f40b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.333+0000 7f64fbd6d640 1 -- 172.21.15.26:0/790864642 shutdown_connections 2023-12-19T16:57:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.333+0000 7f64fbd6d640 1 -- 172.21.15.26:0/790864642 wait complete. 2023-12-19T16:57:06.359 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T16:57:06.379 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:06.375+0000 7f83d2e2c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:57:06.437 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:06.435+0000 7f83d2e2c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:57:06.490 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:06.485+0000 7fa5c9a30180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:57:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.545+0000 7f5976ab6640 1 Processor -- start 2023-12-19T16:57:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.545+0000 7f5976ab6640 1 -- start start 2023-12-19T16:57:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.549+0000 7f5976ab6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5970155030 0x7f597015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.549+0000 7f5976ab6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597006c970 con 0x7f597015f370 2023-12-19T16:57:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.549+0000 7f5976ab6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597015b1f0 con 0x7f59700ce740 2023-12-19T16:57:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.549+0000 7f5976ab6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597006c790 con 0x7f5970155030 2023-12-19T16:57:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.549+0000 7f596ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f597015f370 0x7f597006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54064/0 (socket says 172.21.15.26:54064) 2023-12-19T16:57:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.549+0000 7f596ffff640 1 -- 172.21.15.26:0/1195400787 learned_addr learned my addr 172.21.15.26:0/1195400787 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.549+0000 7f5974852640 1 --2- 172.21.15.26:0/1195400787 >> v2:172.21.15.160:3300/0 conn(0x7f5970155030 0x7f597015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:06.554 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:06.549+0000 7fa5c9a30180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:57:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.549+0000 7f596f7fe640 1 -- 172.21.15.26:0/1195400787 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317478515 0 0) 0x7f597006c970 con 0x7f597015f370 2023-12-19T16:57:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.549+0000 7f596f7fe640 1 -- 172.21.15.26:0/1195400787 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5960003530 con 0x7f597015f370 2023-12-19T16:57:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.549+0000 7f5974852640 1 -- 172.21.15.26:0/1195400787 >> v1:172.21.15.160:6789/0 conn(0x7f59700ce740 legacy=0x7f59700ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.549+0000 7f5974852640 1 -- 172.21.15.26:0/1195400787 >> v1:172.21.15.26:6789/0 conn(0x7f597015f370 legacy=0x7f597006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.549+0000 7f5974852640 1 -- 172.21.15.26:0/1195400787 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597015b4d0 con 0x7f5970155030 2023-12-19T16:57:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.553+0000 7f5974852640 1 --2- 172.21.15.26:0/1195400787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970155030 0x7f597015bcd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f59640097c0 tx=0x7f596401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b64e5682c8f37df server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:06.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.553+0000 7f596f7fe640 1 -- 172.21.15.26:0/1195400787 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59640ab670 con 0x7f5970155030 2023-12-19T16:57:06.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.553+0000 7f596f7fe640 1 -- 172.21.15.26:0/1195400787 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59640ab7d0 con 0x7f5970155030 2023-12-19T16:57:06.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.553+0000 7f596f7fe640 1 -- 172.21.15.26:0/1195400787 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59640abb60 con 0x7f5970155030 2023-12-19T16:57:06.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.553+0000 7f5976ab6640 1 -- 172.21.15.26:0/1195400787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970155030 msgr2=0x7f597015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:06.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.553+0000 7f5976ab6640 1 --2- 172.21.15.26:0/1195400787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970155030 0x7f597015bcd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f59640097c0 tx=0x7f596401c450 comp rx=0 tx=0).stop 2023-12-19T16:57:06.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.553+0000 7f5976ab6640 1 -- 172.21.15.26:0/1195400787 shutdown_connections 2023-12-19T16:57:06.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.553+0000 7f5976ab6640 1 --2- 172.21.15.26:0/1195400787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970155030 0x7f597015bcd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.553+0000 7f5976ab6640 1 -- 172.21.15.26:0/1195400787 >> 172.21.15.26:0/1195400787 conn(0x7f59700658e0 msgr2=0x7f5970065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:06.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.553+0000 7f5976ab6640 1 -- 172.21.15.26:0/1195400787 shutdown_connections 2023-12-19T16:57:06.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.553+0000 7f5976ab6640 1 -- 172.21.15.26:0/1195400787 wait complete. 2023-12-19T16:57:06.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5976ab6640 1 Processor -- start 2023-12-19T16:57:06.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5976ab6640 1 -- start start 2023-12-19T16:57:06.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5976ab6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f59700ce740 0x7f5970159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:06.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5976ab6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970155030 0x7f597014c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:06.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5976ab6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597015f370 0x7f597014cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:06.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5976ab6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597014d340 con 0x7f597015f370 2023-12-19T16:57:06.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5976ab6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700b2bb0 con 0x7f5970155030 2023-12-19T16:57:06.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5976ab6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59700b2e50 con 0x7f59700ce740 2023-12-19T16:57:06.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5975053640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597015f370 0x7f597014cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:06.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5975053640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597015f370 0x7f597014cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55316/0 (socket says 172.21.15.26:55316) 2023-12-19T16:57:06.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5975053640 1 -- 172.21.15.26:0/1125483378 learned_addr learned my addr 172.21.15.26:0/1125483378 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:06.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5974852640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f59700ce740 0x7f5970159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:06.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f596ffff640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970155030 0x7f597014c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:06.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5975053640 1 -- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f59700ce740 msgr2=0x7f5970159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:06.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5975053640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f59700ce740 0x7f5970159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5975053640 1 -- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970155030 msgr2=0x7f597014c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:06.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5975053640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970155030 0x7f597014c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f5975053640 1 -- 172.21.15.26:0/1125483378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5964009420 con 0x7f597015f370 2023-12-19T16:57:06.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.557+0000 7f596ffff640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970155030 0x7f597014c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:06.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.561+0000 7f5975053640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597015f370 0x7f597014cce0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f596800c730 tx=0x7f596800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:06.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.561+0000 7f596d7fa640 1 -- 172.21.15.26:0/1125483378 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968003b80 con 0x7f597015f370 2023-12-19T16:57:06.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.561+0000 7f596d7fa640 1 -- 172.21.15.26:0/1125483378 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5968003ce0 con 0x7f597015f370 2023-12-19T16:57:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.561+0000 7f5976ab6640 1 -- 172.21.15.26:0/1125483378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59700cf0e0 con 0x7f597015f370 2023-12-19T16:57:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.561+0000 7f596d7fa640 1 -- 172.21.15.26:0/1125483378 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800aa30 con 0x7f597015f370 2023-12-19T16:57:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.561+0000 7f5976ab6640 1 -- 172.21.15.26:0/1125483378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59700b3410 con 0x7f597015f370 2023-12-19T16:57:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.561+0000 7f5976ab6640 1 -- 172.21.15.26:0/1125483378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59700b3600 con 0x7f597015f370 2023-12-19T16:57:06.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.561+0000 7f596d7fa640 1 -- 172.21.15.26:0/1125483378 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f596800ab90 con 0x7f597015f370 2023-12-19T16:57:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.561+0000 7f596d7fa640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5954065b80 0x7f5954068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.561+0000 7f596d7fa640 1 -- 172.21.15.26:0/1125483378 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f5968083440 con 0x7f597015f370 2023-12-19T16:57:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.561+0000 7f5974852640 1 -- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5954065b80 msgr2=0x7f5954068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:06.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.561+0000 7f5974852640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5954065b80 0x7f5954068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:06.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.565+0000 7f596d7fa640 1 -- 172.21.15.26:0/1125483378 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f59680508c0 con 0x7f597015f370 2023-12-19T16:57:06.687 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:57:06.687 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:57:06.687 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:57:06.695 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:06.691+0000 7f83d2e2c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:57:06.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.765+0000 7f5974852640 1 -- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5954065b80 msgr2=0x7f5954068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:06.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.765+0000 7f5974852640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5954065b80 0x7f5954068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T16:57:06.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.793+0000 7f5976ab6640 1 -- 172.21.15.26:0/1125483378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5970074270 con 0x7f597015f370 2023-12-19T16:57:06.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.797+0000 7f596d7fa640 1 -- 172.21.15.26:0/1125483378 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v66) v1 ==== 80+0+464 (secure 0 0 0) 0x7f5968053ee0 con 0x7f597015f370 2023-12-19T16:57:06.801 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6832/38098 2023-12-20T16:55:12.385158+0000 2023-12-19T16:57:06.801 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6833/38098 2023-12-20T16:55:12.385158+0000 2023-12-19T16:57:06.801 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/251239631 2023-12-20T16:55:12.385158+0000 2023-12-19T16:57:06.801 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1854082733 2023-12-20T16:55:12.385158+0000 2023-12-19T16:57:06.801 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/260287399 2023-12-20T16:55:12.385158+0000 2023-12-19T16:57:06.801 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/71378321 2023-12-20T16:55:12.385158+0000 2023-12-19T16:57:06.802 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1751160424 2023-12-20T16:55:12.385158+0000 2023-12-19T16:57:06.802 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3640035110 2023-12-20T16:55:12.385158+0000 2023-12-19T16:57:06.802 INFO:teuthology.orchestra.run.smithi026.stderr:listed 8 entries 2023-12-19T16:57:06.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.797+0000 7f5976ab6640 1 -- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5954065b80 msgr2=0x7f5954068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:06.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.797+0000 7f5976ab6640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5954065b80 0x7f5954068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.797+0000 7f5976ab6640 1 -- 172.21.15.26:0/1125483378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597015f370 msgr2=0x7f597014cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:06.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.797+0000 7f5976ab6640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597015f370 0x7f597014cce0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f596800c730 tx=0x7f596800cbf0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.797+0000 7f5976ab6640 1 -- 172.21.15.26:0/1125483378 shutdown_connections 2023-12-19T16:57:06.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.797+0000 7f5976ab6640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f597015f370 0x7f597014cce0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.797+0000 7f5976ab6640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5970155030 0x7f597014c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.797+0000 7f5976ab6640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5954065b80 0x7f5954068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.797+0000 7f5976ab6640 1 --2- 172.21.15.26:0/1125483378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f59700ce740 0x7f5970159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:06.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.797+0000 7f5976ab6640 1 -- 172.21.15.26:0/1125483378 >> 172.21.15.26:0/1125483378 conn(0x7f59700658e0 msgr2=0x7f597015a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:06.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.797+0000 7f5976ab6640 1 -- 172.21.15.26:0/1125483378 shutdown_connections 2023-12-19T16:57:06.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:06.797+0000 7f5976ab6640 1 -- 172.21.15.26:0/1125483378 wait complete. 2023-12-19T16:57:06.822 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T16:57:06.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:06.822 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-19T16:57:06.830 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:06.827+0000 7f83d2e2c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:57:06.850 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T16:57:06.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:06.850 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T16:57:06.857 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:57:06.857 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:57:06.857 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:57:06.866 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:57:06.866 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:06.867 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:06.861+0000 7fa5c9a30180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:57:06.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:57:06.876 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:57:06.876 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:06.917 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:57:06.917 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:57:06.917 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:06.926 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:57:06.926 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:57:06.956 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:06.951+0000 7f83d2e2c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:57:07.012 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:07.005+0000 7fa5c9a30180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:57:07.027 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:07.023+0000 7f83d2e2c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:57:07.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.097+0000 7f3ef07f0640 1 Processor -- start 2023-12-19T16:57:07.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.097+0000 7f3ef07f0640 1 -- start start 2023-12-19T16:57:07.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.101+0000 7f3ef07f0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3ee815a7c0 0x7f3ee815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.101+0000 7f3ef07f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ee806f9d0 0x7f3ee80ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.101+0000 7f3ef07f0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee815e2f0 con 0x7f3ee80c4070 2023-12-19T16:57:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.101+0000 7f3ef07f0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee80c5b40 con 0x7f3ee815a7c0 2023-12-19T16:57:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.101+0000 7f3ef07f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80c5e20 con 0x7f3ee806f9d0 2023-12-19T16:57:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.101+0000 7f3eee58c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3ee815a7c0 0x7f3ee815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.101+0000 7f3eee58c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3ee815a7c0 0x7f3ee815ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55328/0 (socket says 172.21.15.26:55328) 2023-12-19T16:57:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.101+0000 7f3eee58c640 1 -- 172.21.15.26:0/3513117416 learned_addr learned my addr 172.21.15.26:0/3513117416 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.101+0000 7f3eedd8b640 1 --2- 172.21.15.26:0/3513117416 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ee806f9d0 0x7f3ee80ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.101+0000 7f3eee58c640 1 -- 172.21.15.26:0/3513117416 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ee806f9d0 msgr2=0x7f3ee80ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.101+0000 7f3eee58c640 1 --2- 172.21.15.26:0/3513117416 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ee806f9d0 0x7f3ee80ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.101+0000 7f3eee58c640 1 -- 172.21.15.26:0/3513117416 >> v1:172.21.15.160:6789/0 conn(0x7f3ee80c4070 legacy=0x7f3ee80cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.101+0000 7f3eee58c640 1 -- 172.21.15.26:0/3513117416 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee80ce770 con 0x7f3ee815a7c0 2023-12-19T16:57:07.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3eee58c640 1 --2- 172.21.15.26:0/3513117416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee815a7c0 0x7f3ee815ddb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4006fd0 tx=0x7f3ee401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83295fa395e8a1d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3eed58a640 1 -- 172.21.15.26:0/3513117416 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40ab600 con 0x7f3ee815a7c0 2023-12-19T16:57:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3eed58a640 1 -- 172.21.15.26:0/3513117416 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee40a7050 con 0x7f3ee815a7c0 2023-12-19T16:57:07.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3eed58a640 1 -- 172.21.15.26:0/3513117416 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40ab980 con 0x7f3ee815a7c0 2023-12-19T16:57:07.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3ef07f0640 1 -- 172.21.15.26:0/3513117416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee815a7c0 msgr2=0x7f3ee815ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3ef07f0640 1 --2- 172.21.15.26:0/3513117416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee815a7c0 0x7f3ee815ddb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4006fd0 tx=0x7f3ee401c460 comp rx=0 tx=0).stop 2023-12-19T16:57:07.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3ef07f0640 1 -- 172.21.15.26:0/3513117416 shutdown_connections 2023-12-19T16:57:07.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3ef07f0640 1 --2- 172.21.15.26:0/3513117416 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ee806f9d0 0x7f3ee80ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3ef07f0640 1 --2- 172.21.15.26:0/3513117416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee815a7c0 0x7f3ee815ddb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3ef07f0640 1 -- 172.21.15.26:0/3513117416 >> 172.21.15.26:0/3513117416 conn(0x7f3ee806d5d0 msgr2=0x7f3ee8069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:07.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3ef07f0640 1 -- 172.21.15.26:0/3513117416 shutdown_connections 2023-12-19T16:57:07.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3ef07f0640 1 -- 172.21.15.26:0/3513117416 wait complete. 2023-12-19T16:57:07.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3ef07f0640 1 Processor -- start 2023-12-19T16:57:07.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.105+0000 7f3ef07f0640 1 -- start start 2023-12-19T16:57:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3ef07f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ee806f9d0 0x7f3ee8152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3ef07f0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ee80c4070 0x7f3ee814d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3ef07f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee815a7c0 0x7f3ee81f9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3ef07f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee814d5e0 con 0x7f3ee815a7c0 2023-12-19T16:57:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3ef07f0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee814d7b0 con 0x7f3ee80c4070 2023-12-19T16:57:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3ef07f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee81f9790 con 0x7f3ee806f9d0 2023-12-19T16:57:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3eee58c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ee806f9d0 0x7f3ee8152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3eeed8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee815a7c0 0x7f3ee81f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3eee58c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ee806f9d0 0x7f3ee8152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:57704/0 (socket says 172.21.15.26:57704) 2023-12-19T16:57:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3eee58c640 1 -- 172.21.15.26:0/4157479863 learned_addr learned my addr 172.21.15.26:0/4157479863 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3eedd8b640 1 --2- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ee80c4070 0x7f3ee814d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3eee58c640 1 -- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ee80c4070 msgr2=0x7f3ee814d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3eee58c640 1 --2- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ee80c4070 0x7f3ee814d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3eee58c640 1 -- 172.21.15.26:0/4157479863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee815a7c0 msgr2=0x7f3ee81f9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3eee58c640 1 --2- 172.21.15.26:0/4157479863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee815a7c0 0x7f3ee81f9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3eee58c640 1 -- 172.21.15.26:0/4157479863 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee4009480 con 0x7f3ee806f9d0 2023-12-19T16:57:07.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3eeed8d640 1 --2- 172.21.15.26:0/4157479863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee815a7c0 0x7f3ee81f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:07.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3eee58c640 1 --2- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ee806f9d0 0x7f3ee8152240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3ee401a090 tx=0x7f3ee4019200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:07.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3edeffd640 1 -- 172.21.15.26:0/4157479863 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4005ba0 con 0x7f3ee806f9d0 2023-12-19T16:57:07.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3edeffd640 1 -- 172.21.15.26:0/4157479863 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee4005d00 con 0x7f3ee806f9d0 2023-12-19T16:57:07.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3ef07f0640 1 -- 172.21.15.26:0/4157479863 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee81f9a30 con 0x7f3ee806f9d0 2023-12-19T16:57:07.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3edeffd640 1 -- 172.21.15.26:0/4157479863 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4003b90 con 0x7f3ee806f9d0 2023-12-19T16:57:07.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3ef07f0640 1 -- 172.21.15.26:0/4157479863 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee81f9ff0 con 0x7f3ee806f9d0 2023-12-19T16:57:07.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.109+0000 7f3ef07f0640 1 -- 172.21.15.26:0/4157479863 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee8074270 con 0x7f3ee806f9d0 2023-12-19T16:57:07.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.113+0000 7f3edeffd640 1 -- 172.21.15.26:0/4157479863 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f3ee40b2040 con 0x7f3ee806f9d0 2023-12-19T16:57:07.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.113+0000 7f3edeffd640 1 --2- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3ecc065b80 0x7f3ecc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.113+0000 7f3eedd8b640 1 -- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3ecc065b80 msgr2=0x7f3ecc068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:07.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.113+0000 7f3edeffd640 1 -- 172.21.15.26:0/4157479863 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6915+0+0 (secure 0 0 0) 0x7f3ee411d450 con 0x7f3ee806f9d0 2023-12-19T16:57:07.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.113+0000 7f3eedd8b640 1 --2- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3ecc065b80 0x7f3ecc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:07.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.117+0000 7f3edeffd640 1 -- 172.21.15.26:0/4157479863 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3ee40ea6c0 con 0x7f3ee806f9d0 2023-12-19T16:57:07.146 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:07.141+0000 7fa5c9a30180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:57:07.164 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:07.159+0000 7f83d2e2c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:57:07.221 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:07.217+0000 7fa5c9a30180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:57:07.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.309+0000 7f3ef07f0640 1 -- 172.21.15.26:0/4157479863 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ee80717b0 con 0x7f3ee806f9d0 2023-12-19T16:57:07.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.313+0000 7f3edeffd640 1 -- 172.21.15.26:0/4157479863 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f3ee40edce0 con 0x7f3ee806f9d0 2023-12-19T16:57:07.317 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:07.317 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1947010592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1947010592},{"type":"v1","addr":"172.21.15.26:6833","nonce":1947010592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2764469548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2764469548},{"type":"v1","addr":"172.21.15.160:6837","nonce":2764469548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5803,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/255772664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":255772664},{"type":"v1","addr":"172.21.15.160:6836","nonce":255772664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:55:00.584757+0000","modified":"2023-12-19T16:55:03.604713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6000":{"gid":6000,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1847960813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1847960813},{"type":"v1","addr":"172.21.15.26:6835","nonce":1847960813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T16:57:07.318 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2023-12-19T16:57:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.313+0000 7f3ef07f0640 1 -- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3ecc065b80 msgr2=0x7f3ecc068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.313+0000 7f3ef07f0640 1 --2- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3ecc065b80 0x7f3ecc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.313+0000 7f3ef07f0640 1 -- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ee806f9d0 msgr2=0x7f3ee8152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.313+0000 7f3ef07f0640 1 --2- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ee806f9d0 0x7f3ee8152240 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3ee401a090 tx=0x7f3ee4019200 comp rx=0 tx=0).stop 2023-12-19T16:57:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.313+0000 7f3ef07f0640 1 -- 172.21.15.26:0/4157479863 shutdown_connections 2023-12-19T16:57:07.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.313+0000 7f3ef07f0640 1 --2- 172.21.15.26:0/4157479863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ee815a7c0 0x7f3ee81f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.313+0000 7f3ef07f0640 1 --2- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ee80c4070 0x7f3ee814d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.313+0000 7f3ef07f0640 1 --2- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3ecc065b80 0x7f3ecc068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.313+0000 7f3ef07f0640 1 --2- 172.21.15.26:0/4157479863 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ee806f9d0 0x7f3ee8152240 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.313+0000 7f3ef07f0640 1 -- 172.21.15.26:0/4157479863 >> 172.21.15.26:0/4157479863 conn(0x7f3ee806d5d0 msgr2=0x7f3ee8160f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:07.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.313+0000 7f3ef07f0640 1 -- 172.21.15.26:0/4157479863 shutdown_connections 2023-12-19T16:57:07.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.313+0000 7f3ef07f0640 1 -- 172.21.15.26:0/4157479863 wait complete. 2023-12-19T16:57:07.339 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:07.365 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:07.361+0000 7fa5c9a30180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:57:07.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56ec7a6640 1 Processor -- start 2023-12-19T16:57:07.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56ec7a6640 1 -- start start 2023-12-19T16:57:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56ec7a6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f56e406b3d0 0x7f56e40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56ec7a6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56e40cec40 0x7f56e415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56ec7a6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e406bc30 con 0x7f56e4154220 2023-12-19T16:57:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56ec7a6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e40cf0d0 con 0x7f56e40cec40 2023-12-19T16:57:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56ec7a6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e40cc6e0 con 0x7f56e406b3d0 2023-12-19T16:57:07.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56e9d41640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f56e406b3d0 0x7f56e40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56ead43640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56e40cec40 0x7f56e415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56e9d41640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f56e406b3d0 0x7f56e40ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:42712/0 (socket says 172.21.15.26:42712) 2023-12-19T16:57:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56e9d41640 1 -- 172.21.15.26:0/4106686840 learned_addr learned my addr 172.21.15.26:0/4106686840 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56ea542640 1 --1- 172.21.15.26:0/4106686840 >> v1:172.21.15.160:6789/0 conn(0x7f56e4154220 0x7f56e415aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:40950/0 (socket says 172.21.15.26:40950) 2023-12-19T16:57:07.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56ead43640 1 -- 172.21.15.26:0/4106686840 >> v2:172.21.15.160:3300/0 conn(0x7f56e406b3d0 msgr2=0x7f56e40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:07.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56ead43640 1 --2- 172.21.15.26:0/4106686840 >> v2:172.21.15.160:3300/0 conn(0x7f56e406b3d0 0x7f56e40ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56ead43640 1 -- 172.21.15.26:0/4106686840 >> v1:172.21.15.160:6789/0 conn(0x7f56e4154220 legacy=0x7f56e415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.481+0000 7f56ead43640 1 -- 172.21.15.26:0/4106686840 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e406b950 con 0x7f56e40cec40 2023-12-19T16:57:07.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56ead43640 1 --2- 172.21.15.26:0/4106686840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e40cec40 0x7f56e415b1c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f56e40cf880 tx=0x7f56dc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9637da2200c821c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:07.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56e9540640 1 -- 172.21.15.26:0/4106686840 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc0a59a0 con 0x7f56e40cec40 2023-12-19T16:57:07.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56e9540640 1 -- 172.21.15.26:0/4106686840 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc0a5b00 con 0x7f56e40cec40 2023-12-19T16:57:07.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56e9540640 1 -- 172.21.15.26:0/4106686840 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc0a5e90 con 0x7f56e40cec40 2023-12-19T16:57:07.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56ec7a6640 1 -- 172.21.15.26:0/4106686840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e40cec40 msgr2=0x7f56e415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56ec7a6640 1 --2- 172.21.15.26:0/4106686840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e40cec40 0x7f56e415b1c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f56e40cf880 tx=0x7f56dc01c8e0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56ec7a6640 1 -- 172.21.15.26:0/4106686840 shutdown_connections 2023-12-19T16:57:07.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56ec7a6640 1 --2- 172.21.15.26:0/4106686840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e40cec40 0x7f56e415b1c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56ec7a6640 1 --2- 172.21.15.26:0/4106686840 >> v2:172.21.15.160:3300/0 conn(0x7f56e406b3d0 0x7f56e40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56ec7a6640 1 -- 172.21.15.26:0/4106686840 >> 172.21.15.26:0/4106686840 conn(0x7f56e40658e0 msgr2=0x7f56e4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:07.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56ec7a6640 1 -- 172.21.15.26:0/4106686840 shutdown_connections 2023-12-19T16:57:07.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56ec7a6640 1 -- 172.21.15.26:0/4106686840 wait complete. 2023-12-19T16:57:07.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56ec7a6640 1 Processor -- start 2023-12-19T16:57:07.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.489+0000 7f56ec7a6640 1 -- start start 2023-12-19T16:57:07.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ec7a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e406b3d0 0x7f56e40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ec7a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56e40cec40 0x7f56e40b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ec7a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56e4154220 0x7f56e40ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ec7a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40b2bf0 con 0x7f56e406b3d0 2023-12-19T16:57:07.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ec7a6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e4155960 con 0x7f56e40cec40 2023-12-19T16:57:07.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ec7a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e40b2e20 con 0x7f56e4154220 2023-12-19T16:57:07.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ea542640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e406b3d0 0x7f56e40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ea542640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e406b3d0 0x7f56e40b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55368/0 (socket says 172.21.15.26:55368) 2023-12-19T16:57:07.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ea542640 1 -- 172.21.15.26:0/2869574270 learned_addr learned my addr 172.21.15.26:0/2869574270 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:07.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56e9d41640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56e40cec40 0x7f56e40b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ead43640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56e4154220 0x7f56e40ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ea542640 1 -- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56e4154220 msgr2=0x7f56e40ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ea542640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56e4154220 0x7f56e40ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ea542640 1 -- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56e40cec40 msgr2=0x7f56e40b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ea542640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56e40cec40 0x7f56e40b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ea542640 1 -- 172.21.15.26:0/2869574270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc009a70 con 0x7f56e406b3d0 2023-12-19T16:57:07.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ead43640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56e4154220 0x7f56e40ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:07.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ea542640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e406b3d0 0x7f56e40b2090 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f56e000c490 tx=0x7f56e000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:07.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56daffd640 1 -- 172.21.15.26:0/2869574270 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0007480 con 0x7f56e406b3d0 2023-12-19T16:57:07.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56daffd640 1 -- 172.21.15.26:0/2869574270 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e00075e0 con 0x7f56e406b3d0 2023-12-19T16:57:07.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56daffd640 1 -- 172.21.15.26:0/2869574270 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0004a10 con 0x7f56e406b3d0 2023-12-19T16:57:07.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ec7a6640 1 -- 172.21.15.26:0/2869574270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e40b3100 con 0x7f56e406b3d0 2023-12-19T16:57:07.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ec7a6640 1 -- 172.21.15.26:0/2869574270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e4157c20 con 0x7f56e406b3d0 2023-12-19T16:57:07.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.493+0000 7f56ec7a6640 1 -- 172.21.15.26:0/2869574270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e4074270 con 0x7f56e406b3d0 2023-12-19T16:57:07.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.497+0000 7f56daffd640 1 -- 172.21.15.26:0/2869574270 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f56e0004b70 con 0x7f56e406b3d0 2023-12-19T16:57:07.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.497+0000 7f56daffd640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f56c8065b80 0x7f56c8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.497+0000 7f56daffd640 1 -- 172.21.15.26:0/2869574270 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6915+0+0 (secure 0 0 0) 0x7f56e0082950 con 0x7f56e406b3d0 2023-12-19T16:57:07.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.497+0000 7f56e9d41640 1 -- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f56c8065b80 msgr2=0x7f56c8068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:07.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.497+0000 7f56e9d41640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f56c8065b80 0x7f56c8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:07.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.501+0000 7f56daffd640 1 -- 172.21.15.26:0/2869574270 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f56e004fde0 con 0x7f56e406b3d0 2023-12-19T16:57:07.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.661+0000 7f56ec7a6640 1 -- 172.21.15.26:0/2869574270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56e40cdf60 con 0x7f56e406b3d0 2023-12-19T16:57:07.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.661+0000 7f56daffd640 1 -- 172.21.15.26:0/2869574270 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+17928 (secure 0 0 0) 0x7f56e0018090 con 0x7f56e406b3d0 2023-12-19T16:57:07.668 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:07.668 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":66,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:06.594451+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:54:58.260985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T16:54:59.644467+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.f","mappings":[{"from":0,"to":4}]},{"pgid":"7.2e","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6832/38098":"2023-12-20T16:55:12.385158+0000","172.21.15.160:6833/38098":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/251239631":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/1854082733":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/260287399":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/71378321":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/1751160424":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/3640035110":"2023-12-20T16:55:12.385158+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.665+0000 7f56ec7a6640 1 -- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f56c8065b80 msgr2=0x7f56c8068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:07.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.665+0000 7f56ec7a6640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f56c8065b80 0x7f56c8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.665+0000 7f56ec7a6640 1 -- 172.21.15.26:0/2869574270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e406b3d0 msgr2=0x7f56e40b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.665+0000 7f56ec7a6640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e406b3d0 0x7f56e40b2090 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f56e000c490 tx=0x7f56e000c950 comp rx=0 tx=0).stop 2023-12-19T16:57:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.665+0000 7f56ec7a6640 1 -- 172.21.15.26:0/2869574270 shutdown_connections 2023-12-19T16:57:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.665+0000 7f56ec7a6640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f56c8065b80 0x7f56c8068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.665+0000 7f56ec7a6640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56e4154220 0x7f56e40ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.665+0000 7f56ec7a6640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56e40cec40 0x7f56e40b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.665+0000 7f56ec7a6640 1 --2- 172.21.15.26:0/2869574270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56e406b3d0 0x7f56e40b2090 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.665+0000 7f56ec7a6640 1 -- 172.21.15.26:0/2869574270 >> 172.21.15.26:0/2869574270 conn(0x7f56e40658e0 msgr2=0x7f56e40b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.665+0000 7f56ec7a6640 1 -- 172.21.15.26:0/2869574270 shutdown_connections 2023-12-19T16:57:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.665+0000 7f56ec7a6640 1 -- 172.21.15.26:0/2869574270 wait complete. 2023-12-19T16:57:07.689 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:54:58.260985+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:54:59.644467+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:57:07.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:57:07.704 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:07.699+0000 7f83d2e2c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:57:07.788 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:07.783+0000 7f83d2e2c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:57:07.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e5b855640 1 Processor -- start 2023-12-19T16:57:07.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e5b855640 1 -- start start 2023-12-19T16:57:07.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e5b855640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e5415a9c0 0x7f0e540706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e5b855640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e5406ac80 0x7f0e540b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e5b855640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e54070d90 con 0x7f0e5415ada0 2023-12-19T16:57:07.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e5b855640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e540cc270 con 0x7f0e5415a9c0 2023-12-19T16:57:07.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e5b855640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e54071070 con 0x7f0e5406ac80 2023-12-19T16:57:07.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e595f1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e5415a9c0 0x7f0e540706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e595f1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e5415a9c0 0x7f0e540706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55372/0 (socket says 172.21.15.26:55372) 2023-12-19T16:57:07.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e595f1640 1 -- 172.21.15.26:0/1110532146 learned_addr learned my addr 172.21.15.26:0/1110532146 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:07.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e59df2640 1 --2- 172.21.15.26:0/1110532146 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e5406ac80 0x7f0e540b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e595f1640 1 -- 172.21.15.26:0/1110532146 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e5406ac80 msgr2=0x7f0e540b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:07.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e595f1640 1 --2- 172.21.15.26:0/1110532146 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e5406ac80 0x7f0e540b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e595f1640 1 -- 172.21.15.26:0/1110532146 >> v1:172.21.15.160:6789/0 conn(0x7f0e5415ada0 legacy=0x7f0e540b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.877+0000 7f0e595f1640 1 -- 172.21.15.26:0/1110532146 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e54071350 con 0x7f0e5415a9c0 2023-12-19T16:57:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.881+0000 7f0e595f1640 1 --2- 172.21.15.26:0/1110532146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5415a9c0 0x7f0e540706d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009870 tx=0x7f0e5001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d806664fb95bc6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.881+0000 7f0e4bfff640 1 -- 172.21.15.26:0/1110532146 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500ad640 con 0x7f0e5415a9c0 2023-12-19T16:57:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.881+0000 7f0e4bfff640 1 -- 172.21.15.26:0/1110532146 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e500ad7a0 con 0x7f0e5415a9c0 2023-12-19T16:57:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.881+0000 7f0e4bfff640 1 -- 172.21.15.26:0/1110532146 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500adb30 con 0x7f0e5415a9c0 2023-12-19T16:57:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.881+0000 7f0e5b855640 1 -- 172.21.15.26:0/1110532146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5415a9c0 msgr2=0x7f0e540706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.881+0000 7f0e5b855640 1 --2- 172.21.15.26:0/1110532146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5415a9c0 0x7f0e540706d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009870 tx=0x7f0e5001c480 comp rx=0 tx=0).stop 2023-12-19T16:57:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.881+0000 7f0e5b855640 1 -- 172.21.15.26:0/1110532146 shutdown_connections 2023-12-19T16:57:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.881+0000 7f0e5b855640 1 --2- 172.21.15.26:0/1110532146 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e5406ac80 0x7f0e540b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.881+0000 7f0e5b855640 1 --2- 172.21.15.26:0/1110532146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5415a9c0 0x7f0e540706d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.881+0000 7f0e5b855640 1 -- 172.21.15.26:0/1110532146 >> 172.21.15.26:0/1110532146 conn(0x7f0e540ce730 msgr2=0x7f0e540ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e5b855640 1 -- 172.21.15.26:0/1110532146 shutdown_connections 2023-12-19T16:57:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e5b855640 1 -- 172.21.15.26:0/1110532146 wait complete. 2023-12-19T16:57:07.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e5b855640 1 Processor -- start 2023-12-19T16:57:07.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e5b855640 1 -- start start 2023-12-19T16:57:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e5b855640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e5406ac80 0x7f0e5414f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e5b855640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e5415a9c0 0x7f0e540c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e5b855640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5415ada0 0x7f0e540c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e5b855640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5414ff10 con 0x7f0e5415ada0 2023-12-19T16:57:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e5b855640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e540c2300 con 0x7f0e5406ac80 2023-12-19T16:57:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e5b855640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e54150490 con 0x7f0e5415a9c0 2023-12-19T16:57:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e59df2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5415ada0 0x7f0e540c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e59df2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5415ada0 0x7f0e540c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55374/0 (socket says 172.21.15.26:55374) 2023-12-19T16:57:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e59df2640 1 -- 172.21.15.26:0/1545019391 learned_addr learned my addr 172.21.15.26:0/1545019391 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e58df0640 1 --2- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e5415a9c0 0x7f0e540c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e595f1640 1 --2- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e5406ac80 0x7f0e5414f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e59df2640 1 -- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e5415a9c0 msgr2=0x7f0e540c1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e59df2640 1 --2- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e5415a9c0 0x7f0e540c1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e59df2640 1 -- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e5406ac80 msgr2=0x7f0e5414f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:07.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e59df2640 1 --2- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e5406ac80 0x7f0e5414f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:07.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e59df2640 1 -- 172.21.15.26:0/1545019391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e500094d0 con 0x7f0e5415ada0 2023-12-19T16:57:07.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.885+0000 7f0e59df2640 1 --2- 172.21.15.26:0/1545019391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5415ada0 0x7f0e540c5ff0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00efc0 tx=0x7f0e4c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:07.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.889+0000 7f0e49ffb640 1 -- 172.21.15.26:0/1545019391 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c009ba0 con 0x7f0e5415ada0 2023-12-19T16:57:07.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.889+0000 7f0e49ffb640 1 -- 172.21.15.26:0/1545019391 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4c00f040 con 0x7f0e5415ada0 2023-12-19T16:57:07.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.889+0000 7f0e49ffb640 1 -- 172.21.15.26:0/1545019391 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0146c0 con 0x7f0e5415ada0 2023-12-19T16:57:07.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.889+0000 7f0e5b855640 1 -- 172.21.15.26:0/1545019391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e54150770 con 0x7f0e5415ada0 2023-12-19T16:57:07.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.889+0000 7f0e5b855640 1 -- 172.21.15.26:0/1545019391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e54151170 con 0x7f0e5415ada0 2023-12-19T16:57:07.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.889+0000 7f0e5b855640 1 -- 172.21.15.26:0/1545019391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e54151360 con 0x7f0e5415ada0 2023-12-19T16:57:07.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.889+0000 7f0e49ffb640 1 -- 172.21.15.26:0/1545019391 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f0e4c014820 con 0x7f0e5415ada0 2023-12-19T16:57:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.889+0000 7f0e49ffb640 1 --2- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e38065bb0 0x7f0e38068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.889+0000 7f0e49ffb640 1 -- 172.21.15.26:0/1545019391 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6915+0+0 (secure 0 0 0) 0x7f0e4c0842b0 con 0x7f0e5415ada0 2023-12-19T16:57:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.889+0000 7f0e595f1640 1 -- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e38065bb0 msgr2=0x7f0e38068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:07.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.889+0000 7f0e595f1640 1 --2- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e38065bb0 0x7f0e38068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:07.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:07.893+0000 7f0e49ffb640 1 -- 172.21.15.26:0/1545019391 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0e4c051640 con 0x7f0e5415ada0 2023-12-19T16:57:07.936 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:07.931+0000 7f83d2e2c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:57:07.956 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:07.949+0000 7fa5c9a30180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:57:08.019 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:08.015+0000 7f83d2e2c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:57:08.045 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:08.041+0000 7fa5c9a30180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:57:08.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.053+0000 7f0e5b855640 1 -- 172.21.15.26:0/1545019391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e54074270 con 0x7f0e5415ada0 2023-12-19T16:57:08.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.057+0000 7f0e49ffb640 1 -- 172.21.15.26:0/1545019391 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e4c054c60 con 0x7f0e5415ada0 2023-12-19T16:57:08.060 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:57:08.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.057+0000 7f0e5b855640 1 -- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e38065bb0 msgr2=0x7f0e38068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:08.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.057+0000 7f0e5b855640 1 --2- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e38065bb0 0x7f0e38068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.057+0000 7f0e5b855640 1 -- 172.21.15.26:0/1545019391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5415ada0 msgr2=0x7f0e540c5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.057+0000 7f0e5b855640 1 --2- 172.21.15.26:0/1545019391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5415ada0 0x7f0e540c5ff0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00efc0 tx=0x7f0e4c00c490 comp rx=0 tx=0).stop 2023-12-19T16:57:08.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.057+0000 7f0e5b855640 1 -- 172.21.15.26:0/1545019391 shutdown_connections 2023-12-19T16:57:08.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.057+0000 7f0e5b855640 1 --2- 172.21.15.26:0/1545019391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5415ada0 0x7f0e540c5ff0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.057+0000 7f0e5b855640 1 --2- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e38065bb0 0x7f0e38068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.057+0000 7f0e5b855640 1 --2- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e5415a9c0 0x7f0e540c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.057+0000 7f0e5b855640 1 --2- 172.21.15.26:0/1545019391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e5406ac80 0x7f0e5414f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.057+0000 7f0e5b855640 1 -- 172.21.15.26:0/1545019391 >> 172.21.15.26:0/1545019391 conn(0x7f0e540ce730 msgr2=0x7f0e540b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:08.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.057+0000 7f0e5b855640 1 -- 172.21.15.26:0/1545019391 shutdown_connections 2023-12-19T16:57:08.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.057+0000 7f0e5b855640 1 -- 172.21.15.26:0/1545019391 wait complete. 2023-12-19T16:57:08.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:57:08.207 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:08.201+0000 7fa5c9a30180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:57:08.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.225+0000 7fb9bd939640 1 Processor -- start 2023-12-19T16:57:08.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.225+0000 7fb9bd939640 1 -- start start 2023-12-19T16:57:08.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.229+0000 7fb9bd939640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb9b80706d0 0x7fb9b80b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.229+0000 7fb9bd939640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b8070ae0 con 0x7fb9b815a9c0 2023-12-19T16:57:08.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.229+0000 7fb9bd939640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b8070d90 con 0x7fb9b815ada0 2023-12-19T16:57:08.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.229+0000 7fb9bd939640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b80cc270 con 0x7fb9b80706d0 2023-12-19T16:57:08.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.229+0000 7fb9b6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb9b815a9c0 0x7fb9b80b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54066/0 (socket says 172.21.15.26:54066) 2023-12-19T16:57:08.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.229+0000 7fb9b6ffd640 1 -- 172.21.15.26:0/2200006283 learned_addr learned my addr 172.21.15.26:0/2200006283 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:08.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.229+0000 7fb9b77fe640 1 --2- 172.21.15.26:0/2200006283 >> v2:172.21.15.160:3300/0 conn(0x7fb9b80706d0 0x7fb9b80b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:08.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.229+0000 7fb9b77fe640 1 -- 172.21.15.26:0/2200006283 >> v1:172.21.15.160:6789/0 conn(0x7fb9b815ada0 legacy=0x7fb9b806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.229+0000 7fb9b5ffb640 1 -- 172.21.15.26:0/2200006283 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436116012 0 0) 0x7fb9b8070ae0 con 0x7fb9b815a9c0 2023-12-19T16:57:08.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.229+0000 7fb9b77fe640 1 -- 172.21.15.26:0/2200006283 >> v1:172.21.15.26:6789/0 conn(0x7fb9b815a9c0 legacy=0x7fb9b80b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.229+0000 7fb9b77fe640 1 -- 172.21.15.26:0/2200006283 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b8071070 con 0x7fb9b80706d0 2023-12-19T16:57:08.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.233+0000 7fb9b77fe640 1 --2- 172.21.15.26:0/2200006283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b80706d0 0x7fb9b80b01b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009d90 tx=0x7fb9b001c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4cff051ac18f94f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:08.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.233+0000 7fb9b5ffb640 1 -- 172.21.15.26:0/2200006283 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00a5860 con 0x7fb9b80706d0 2023-12-19T16:57:08.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.233+0000 7fb9b5ffb640 1 -- 172.21.15.26:0/2200006283 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b00a59c0 con 0x7fb9b80706d0 2023-12-19T16:57:08.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.233+0000 7fb9b5ffb640 1 -- 172.21.15.26:0/2200006283 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00a5d50 con 0x7fb9b80706d0 2023-12-19T16:57:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.233+0000 7fb9bd939640 1 -- 172.21.15.26:0/2200006283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b80706d0 msgr2=0x7fb9b80b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.233+0000 7fb9bd939640 1 --2- 172.21.15.26:0/2200006283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b80706d0 0x7fb9b80b01b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009d90 tx=0x7fb9b001c890 comp rx=0 tx=0).stop 2023-12-19T16:57:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.233+0000 7fb9bd939640 1 -- 172.21.15.26:0/2200006283 shutdown_connections 2023-12-19T16:57:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.233+0000 7fb9bd939640 1 --2- 172.21.15.26:0/2200006283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b80706d0 0x7fb9b80b01b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.233+0000 7fb9bd939640 1 -- 172.21.15.26:0/2200006283 >> 172.21.15.26:0/2200006283 conn(0x7fb9b80ce730 msgr2=0x7fb9b80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.233+0000 7fb9bd939640 1 -- 172.21.15.26:0/2200006283 shutdown_connections 2023-12-19T16:57:08.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.233+0000 7fb9bd939640 1 -- 172.21.15.26:0/2200006283 wait complete. 2023-12-19T16:57:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.233+0000 7fb9bd939640 1 Processor -- start 2023-12-19T16:57:08.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.233+0000 7fb9bd939640 1 -- start start 2023-12-19T16:57:08.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9bd939640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b80706d0 0x7fb9b814b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9bd939640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9b815a9c0 0x7fb9b814afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9bd939640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b815ada0 0x7fb9b80c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9bd939640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b80c91f0 con 0x7fb9b80706d0 2023-12-19T16:57:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9bd939640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b80c94d0 con 0x7fb9b815ada0 2023-12-19T16:57:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9bd939640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b80c97b0 con 0x7fb9b815a9c0 2023-12-19T16:57:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9b6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b80706d0 0x7fb9b814b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9b6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b80706d0 0x7fb9b814b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55384/0 (socket says 172.21.15.26:55384) 2023-12-19T16:57:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9b6ffd640 1 -- 172.21.15.26:0/4264159297 learned_addr learned my addr 172.21.15.26:0/4264159297 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9b77fe640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b815ada0 0x7fb9b80c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9b67fc640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9b815a9c0 0x7fb9b814afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9b6ffd640 1 -- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9b815a9c0 msgr2=0x7fb9b814afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9b6ffd640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9b815a9c0 0x7fb9b814afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9b6ffd640 1 -- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b815ada0 msgr2=0x7fb9b80c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9b6ffd640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b815ada0 0x7fb9b80c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9b6ffd640 1 -- 172.21.15.26:0/4264159297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b0009a70 con 0x7fb9b80706d0 2023-12-19T16:57:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9b77fe640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b815ada0 0x7fb9b80c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9b6ffd640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b80706d0 0x7fb9b814b770 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800c4a0 tx=0x7fb9a800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb99ffff640 1 -- 172.21.15.26:0/4264159297 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a80074e0 con 0x7fb9b80706d0 2023-12-19T16:57:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb99ffff640 1 -- 172.21.15.26:0/4264159297 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a8007640 con 0x7fb9b80706d0 2023-12-19T16:57:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9bd939640 1 -- 172.21.15.26:0/4264159297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b8154680 con 0x7fb9b80706d0 2023-12-19T16:57:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb99ffff640 1 -- 172.21.15.26:0/4264159297 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8004a30 con 0x7fb9b80706d0 2023-12-19T16:57:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9bd939640 1 -- 172.21.15.26:0/4264159297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b814b4e0 con 0x7fb9b80706d0 2023-12-19T16:57:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.237+0000 7fb9bd939640 1 -- 172.21.15.26:0/4264159297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b8154870 con 0x7fb9b80706d0 2023-12-19T16:57:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.241+0000 7fb99ffff640 1 -- 172.21.15.26:0/4264159297 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fb9a8004b90 con 0x7fb9b80706d0 2023-12-19T16:57:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.241+0000 7fb99ffff640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb9a4065b80 0x7fb9a4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.241+0000 7fb99ffff640 1 -- 172.21.15.26:0/4264159297 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6915+0+0 (secure 0 0 0) 0x7fb9a8082d00 con 0x7fb9b80706d0 2023-12-19T16:57:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.241+0000 7fb9b67fc640 1 -- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb9a4065b80 msgr2=0x7fb9a4068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:08.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.241+0000 7fb9b67fc640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb9a4065b80 0x7fb9a4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.245+0000 7fb99ffff640 1 -- 172.21.15.26:0/4264159297 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb9a8050100 con 0x7fb9b80706d0 2023-12-19T16:57:08.299 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:08.293+0000 7fa5c9a30180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:57:08.334 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:08.331+0000 7f83d2e2c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:57:08.393 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:08.391+0000 7f83d2e2c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:57:08.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.401+0000 7fb99ffff640 1 -- 172.21.15.26:0/4264159297 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 27) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fb9a80504d0 con 0x7fb9b80706d0 2023-12-19T16:57:08.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.401+0000 7fb99ffff640 1 -- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb9a4065b80 msgr2=0x7fb9a4068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:08.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.401+0000 7fb99ffff640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb9a4065b80 0x7fb9a4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.409+0000 7fb9bd939640 1 -- 172.21.15.26:0/4264159297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9b8074270 con 0x7fb9b80706d0 2023-12-19T16:57:08.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.409+0000 7fb99ffff640 1 -- 172.21.15.26:0/4264159297 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9a806de60 con 0x7fb9b80706d0 2023-12-19T16:57:08.413 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:57:08.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.409+0000 7fb9bd939640 1 -- 172.21.15.26:0/4264159297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b80706d0 msgr2=0x7fb9b814b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.409+0000 7fb9bd939640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b80706d0 0x7fb9b814b770 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800c4a0 tx=0x7fb9a800c960 comp rx=0 tx=0).stop 2023-12-19T16:57:08.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.409+0000 7fb9bd939640 1 -- 172.21.15.26:0/4264159297 shutdown_connections 2023-12-19T16:57:08.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.409+0000 7fb9bd939640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b815ada0 0x7fb9b80c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.409+0000 7fb9bd939640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb9a4065b80 0x7fb9a4068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.409+0000 7fb9bd939640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9b815a9c0 0x7fb9b814afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.409+0000 7fb9bd939640 1 --2- 172.21.15.26:0/4264159297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b80706d0 0x7fb9b814b770 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.409+0000 7fb9bd939640 1 -- 172.21.15.26:0/4264159297 >> 172.21.15.26:0/4264159297 conn(0x7fb9b80ce730 msgr2=0x7fb9b80ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:08.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.409+0000 7fb9bd939640 1 -- 172.21.15.26:0/4264159297 shutdown_connections 2023-12-19T16:57:08.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.409+0000 7fb9bd939640 1 -- 172.21.15.26:0/4264159297 wait complete. 2023-12-19T16:57:08.435 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:57:08.468 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:08.463+0000 7f835bcb1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:08.468 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:08.463+0000 7f835bcb1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:08.468 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:08.463+0000 7f835bcb1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:08.468 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:08.463+0000 7f835bcb1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:08.468 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:08.463+0000 7f835bcb1640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:08.468 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:08.463+0000 7f835a4ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:08.468 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:08.463+0000 7f835a4ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:08.469 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:08.463+0000 7f835a4ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:08.469 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:08.463+0000 7f835a4ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:08.469 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:08.463+0000 7f835a4ae640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:08.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8aa128c640 1 Processor -- start 2023-12-19T16:57:08.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8aa128c640 1 -- start start 2023-12-19T16:57:08.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8aa128c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8a9c15d0d0 0x7f8a9c0b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8aa128c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8a9c0b4600 0x7f8a9c0b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8aa128c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a9c0b4f90 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8aa128c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9c15e200 con 0x7f8a9c0b4600 2023-12-19T16:57:08.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8aa128c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0d0aa0 con 0x7f8a9c15d0d0 2023-12-19T16:57:08.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8a93fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8a9c15d0d0 0x7f8a9c0b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:08.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8a9b7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8a9c0b4600 0x7f8a9c0b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:08.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8a93fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8a9c15d0d0 0x7f8a9c0b8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:57746/0 (socket says 172.21.15.26:57746) 2023-12-19T16:57:08.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8a93fff640 1 -- 172.21.15.26:0/2279427707 learned_addr learned my addr 172.21.15.26:0/2279427707 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:08.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8a9affd640 1 --1- 172.21.15.26:0/2279427707 >> v1:172.21.15.160:6789/0 conn(0x7f8a9c15ccf0 0x7f8a9c0d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:40974/0 (socket says 172.21.15.26:40974) 2023-12-19T16:57:08.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8a93fff640 1 -- 172.21.15.26:0/2279427707 >> v2:172.21.15.160:3300/0 conn(0x7f8a9c0b4600 msgr2=0x7f8a9c0b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:08.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8a93fff640 1 --2- 172.21.15.26:0/2279427707 >> v2:172.21.15.160:3300/0 conn(0x7f8a9c0b4600 0x7f8a9c0b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8a93fff640 1 -- 172.21.15.26:0/2279427707 >> v1:172.21.15.160:6789/0 conn(0x7f8a9c15ccf0 legacy=0x7f8a9c0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.581+0000 7f8a93fff640 1 -- 172.21.15.26:0/2279427707 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c15d680 con 0x7f8a9c15d0d0 2023-12-19T16:57:08.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.585+0000 7f8a93fff640 1 --2- 172.21.15.26:0/2279427707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8a9c15d0d0 0x7f8a9c0b8400 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009870 tx=0x7f8a8401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=265308e9119078f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:08.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.585+0000 7f8a9a7fc640 1 -- 172.21.15.26:0/2279427707 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a840aa670 con 0x7f8a9c15d0d0 2023-12-19T16:57:08.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.585+0000 7f8a9a7fc640 1 -- 172.21.15.26:0/2279427707 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a840aa7d0 con 0x7f8a9c15d0d0 2023-12-19T16:57:08.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.585+0000 7f8a9a7fc640 1 -- 172.21.15.26:0/2279427707 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a840aab60 con 0x7f8a9c15d0d0 2023-12-19T16:57:08.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 -- 172.21.15.26:0/2279427707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8a9c15d0d0 msgr2=0x7f8a9c0b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 --2- 172.21.15.26:0/2279427707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8a9c15d0d0 0x7f8a9c0b8400 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009870 tx=0x7f8a8401c510 comp rx=0 tx=0).stop 2023-12-19T16:57:08.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 -- 172.21.15.26:0/2279427707 shutdown_connections 2023-12-19T16:57:08.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 --2- 172.21.15.26:0/2279427707 >> v2:172.21.15.160:3300/0 conn(0x7f8a9c0b4600 0x7f8a9c0b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 --2- 172.21.15.26:0/2279427707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8a9c15d0d0 0x7f8a9c0b8400 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 -- 172.21.15.26:0/2279427707 >> 172.21.15.26:0/2279427707 conn(0x7f8a9c065ee0 msgr2=0x7f8a9c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:08.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 -- 172.21.15.26:0/2279427707 shutdown_connections 2023-12-19T16:57:08.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 -- 172.21.15.26:0/2279427707 wait complete. 2023-12-19T16:57:08.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 Processor -- start 2023-12-19T16:57:08.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 -- start start 2023-12-19T16:57:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a9c0b4600 0x7f8a9c1541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a9c15ccf0 0x7f8a9c0c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8a9c15d0d0 0x7f8a9c0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0c74d0 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0c6850 con 0x7f8a9c0b4600 2023-12-19T16:57:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8aa128c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0c6b30 con 0x7f8a9c15d0d0 2023-12-19T16:57:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8a93fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a9c15ccf0 0x7f8a9c0c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8a93fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a9c15ccf0 0x7f8a9c0c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55388/0 (socket says 172.21.15.26:55388) 2023-12-19T16:57:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8a93fff640 1 -- 172.21.15.26:0/1505825906 learned_addr learned my addr 172.21.15.26:0/1505825906 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:08.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8a9affd640 1 --2- 172.21.15.26:0/1505825906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a9c0b4600 0x7f8a9c1541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8a9b7fe640 1 --2- 172.21.15.26:0/1505825906 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8a9c15d0d0 0x7f8a9c0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8a93fff640 1 -- 172.21.15.26:0/1505825906 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8a9c15d0d0 msgr2=0x7f8a9c0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8a93fff640 1 --2- 172.21.15.26:0/1505825906 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8a9c15d0d0 0x7f8a9c0b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8a93fff640 1 -- 172.21.15.26:0/1505825906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a9c0b4600 msgr2=0x7f8a9c1541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8a93fff640 1 --2- 172.21.15.26:0/1505825906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a9c0b4600 0x7f8a9c1541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8a93fff640 1 -- 172.21.15.26:0/1505825906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a840094d0 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.589+0000 7f8a9b7fe640 1 --2- 172.21.15.26:0/1505825906 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8a9c15d0d0 0x7f8a9c0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:08.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.593+0000 7f8a93fff640 1 --2- 172.21.15.26:0/1505825906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a9c15ccf0 0x7f8a9c0c8ed0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8a840ac040 tx=0x7f8a840192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.593+0000 7f8a937fe640 1 -- 172.21.15.26:0/1505825906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a840bf680 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.593+0000 7f8a937fe640 1 -- 172.21.15.26:0/1505825906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a840b8070 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.593+0000 7f8a937fe640 1 -- 172.21.15.26:0/1505825906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a840bfa40 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.593+0000 7f8aa128c640 1 -- 172.21.15.26:0/1505825906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9c0c6e10 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.593+0000 7f8aa128c640 1 -- 172.21.15.26:0/1505825906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9c0c78f0 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.593+0000 7f8aa128c640 1 -- 172.21.15.26:0/1505825906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9c074270 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.593+0000 7f8a937fe640 1 -- 172.21.15.26:0/1505825906 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 27) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f8a840b4400 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.593+0000 7f8a937fe640 1 -- 172.21.15.26:0/1505825906 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f8a840ba070 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.597+0000 7f8a937fe640 1 -- 172.21.15.26:0/1505825906 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8a840eb980 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.655 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:08.649+0000 7fa5c9a30180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:57:08.720 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:08.713+0000 7fa5c9a30180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:57:08.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.765+0000 7f8aa128c640 1 -- 172.21.15.26:0/1505825906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8a9c0cff00 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.765+0000 7f8a937fe640 1 -- 172.21.15.26:0/1505825906 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8a840eefa0 con 0x7f8a9c15ccf0 2023-12-19T16:57:08.771 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:57:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.769+0000 7f8aa128c640 1 -- 172.21.15.26:0/1505825906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a9c15ccf0 msgr2=0x7f8a9c0c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.769+0000 7f8aa128c640 1 --2- 172.21.15.26:0/1505825906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a9c15ccf0 0x7f8a9c0c8ed0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8a840ac040 tx=0x7f8a840192b0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.769+0000 7f8aa128c640 1 -- 172.21.15.26:0/1505825906 shutdown_connections 2023-12-19T16:57:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.769+0000 7f8aa128c640 1 --2- 172.21.15.26:0/1505825906 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8a9c15d0d0 0x7f8a9c0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.769+0000 7f8aa128c640 1 --2- 172.21.15.26:0/1505825906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a9c15ccf0 0x7f8a9c0c8ed0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.769+0000 7f8aa128c640 1 --2- 172.21.15.26:0/1505825906 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a9c0b4600 0x7f8a9c1541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.769+0000 7f8aa128c640 1 -- 172.21.15.26:0/1505825906 >> 172.21.15.26:0/1505825906 conn(0x7f8a9c065ee0 msgr2=0x7f8a9c0ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.769+0000 7f8aa128c640 1 -- 172.21.15.26:0/1505825906 shutdown_connections 2023-12-19T16:57:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.769+0000 7f8aa128c640 1 -- 172.21.15.26:0/1505825906 wait complete. 2023-12-19T16:57:08.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:57:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.929+0000 7fbb5c56d640 1 Processor -- start 2023-12-19T16:57:08.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.929+0000 7fbb5c56d640 1 -- start start 2023-12-19T16:57:08.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.933+0000 7fbb5c56d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbb5406b3d0 0x7fbb540ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.933+0000 7fbb5c56d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5406bc30 con 0x7fbb54154220 2023-12-19T16:57:08.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.933+0000 7fbb5c56d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb540cc6e0 con 0x7fbb540cec40 2023-12-19T16:57:08.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.933+0000 7fbb5c56d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb540cf0d0 con 0x7fbb5406b3d0 2023-12-19T16:57:08.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.933+0000 7fbb5a309640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbb54154220 0x7fbb5415aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54082/0 (socket says 172.21.15.26:54082) 2023-12-19T16:57:08.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.933+0000 7fbb5a309640 1 -- 172.21.15.26:0/2447075133 learned_addr learned my addr 172.21.15.26:0/2447075133 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:08.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.933+0000 7fbb59b08640 1 --2- 172.21.15.26:0/2447075133 >> v2:172.21.15.26:3300/0 conn(0x7fbb5406b3d0 0x7fbb540ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:08.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.933+0000 7fbb59307640 1 -- 172.21.15.26:0/2447075133 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800800340 0 0) 0x7fbb5406bc30 con 0x7fbb54154220 2023-12-19T16:57:08.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.933+0000 7fbb59307640 1 -- 172.21.15.26:0/2447075133 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb40005740 con 0x7fbb54154220 2023-12-19T16:57:08.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.933+0000 7fbb59b08640 1 -- 172.21.15.26:0/2447075133 >> v1:172.21.15.160:6789/0 conn(0x7fbb540cec40 legacy=0x7fbb5415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.933+0000 7fbb59b08640 1 -- 172.21.15.26:0/2447075133 >> v1:172.21.15.26:6789/0 conn(0x7fbb54154220 legacy=0x7fbb5415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.933+0000 7fbb59b08640 1 -- 172.21.15.26:0/2447075133 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5406b950 con 0x7fbb5406b3d0 2023-12-19T16:57:08.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.937+0000 7fbb59b08640 1 --2- 172.21.15.26:0/2447075133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb5406b3d0 0x7fbb540ce700 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbb440097c0 tx=0x7fbb4401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8cbf2c26841bd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:08.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.937+0000 7fbb59307640 1 -- 172.21.15.26:0/2447075133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440ab650 con 0x7fbb5406b3d0 2023-12-19T16:57:08.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.937+0000 7fbb59307640 1 -- 172.21.15.26:0/2447075133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb440ab7b0 con 0x7fbb5406b3d0 2023-12-19T16:57:08.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.937+0000 7fbb59307640 1 -- 172.21.15.26:0/2447075133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440abb40 con 0x7fbb5406b3d0 2023-12-19T16:57:08.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.937+0000 7fbb5c56d640 1 -- 172.21.15.26:0/2447075133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb5406b3d0 msgr2=0x7fbb540ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.937+0000 7fbb5c56d640 1 --2- 172.21.15.26:0/2447075133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb5406b3d0 0x7fbb540ce700 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbb440097c0 tx=0x7fbb4401c450 comp rx=0 tx=0).stop 2023-12-19T16:57:08.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.937+0000 7fbb5c56d640 1 -- 172.21.15.26:0/2447075133 shutdown_connections 2023-12-19T16:57:08.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.937+0000 7fbb5c56d640 1 --2- 172.21.15.26:0/2447075133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb5406b3d0 0x7fbb540ce700 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.937+0000 7fbb5c56d640 1 -- 172.21.15.26:0/2447075133 >> 172.21.15.26:0/2447075133 conn(0x7fbb540658e0 msgr2=0x7fbb54065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:08.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.937+0000 7fbb5c56d640 1 -- 172.21.15.26:0/2447075133 shutdown_connections 2023-12-19T16:57:08.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.937+0000 7fbb5c56d640 1 -- 172.21.15.26:0/2447075133 wait complete. 2023-12-19T16:57:08.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.937+0000 7fbb5c56d640 1 Processor -- start 2023-12-19T16:57:08.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.937+0000 7fbb5c56d640 1 -- start start 2023-12-19T16:57:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5c56d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb5406b3d0 0x7fbb540b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5c56d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb540cec40 0x7fbb54154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5c56d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54154220 0x7fbb540af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5c56d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54154b40 con 0x7fbb54154220 2023-12-19T16:57:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5c56d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54154dc0 con 0x7fbb540cec40 2023-12-19T16:57:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5c56d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541550a0 con 0x7fbb5406b3d0 2023-12-19T16:57:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5ab0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54154220 0x7fbb540af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:08.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5ab0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54154220 0x7fbb540af210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55404/0 (socket says 172.21.15.26:55404) 2023-12-19T16:57:08.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5ab0a640 1 -- 172.21.15.26:0/3757807695 learned_addr learned my addr 172.21.15.26:0/3757807695 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:08.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5a309640 1 --2- 172.21.15.26:0/3757807695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb5406b3d0 0x7fbb540b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:08.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb59b08640 1 --2- 172.21.15.26:0/3757807695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb540cec40 0x7fbb54154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:08.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5ab0a640 1 -- 172.21.15.26:0/3757807695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb5406b3d0 msgr2=0x7fbb540b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5ab0a640 1 --2- 172.21.15.26:0/3757807695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb5406b3d0 0x7fbb540b26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5ab0a640 1 -- 172.21.15.26:0/3757807695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb540cec40 msgr2=0x7fbb54154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:08.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5ab0a640 1 --2- 172.21.15.26:0/3757807695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb540cec40 0x7fbb54154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:08.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5ab0a640 1 -- 172.21.15.26:0/3757807695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb44009420 con 0x7fbb54154220 2023-12-19T16:57:08.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5ab0a640 1 --2- 172.21.15.26:0/3757807695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54154220 0x7fbb540af210 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00c770 tx=0x7fbb4c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb4affd640 1 -- 172.21.15.26:0/3757807695 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c003b80 con 0x7fbb54154220 2023-12-19T16:57:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb4affd640 1 -- 172.21.15.26:0/3757807695 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c003ce0 con 0x7fbb54154220 2023-12-19T16:57:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb4affd640 1 -- 172.21.15.26:0/3757807695 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c00aa90 con 0x7fbb54154220 2023-12-19T16:57:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5c56d640 1 -- 172.21.15.26:0/3757807695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb54155380 con 0x7fbb54154220 2023-12-19T16:57:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5c56d640 1 -- 172.21.15.26:0/3757807695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb54155940 con 0x7fbb54154220 2023-12-19T16:57:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb5c56d640 1 -- 172.21.15.26:0/3757807695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb54074270 con 0x7fbb54154220 2023-12-19T16:57:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.941+0000 7fbb4affd640 1 -- 172.21.15.26:0/3757807695 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 27) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fbb4c00abf0 con 0x7fbb54154220 2023-12-19T16:57:08.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.945+0000 7fbb4affd640 1 -- 172.21.15.26:0/3757807695 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7129+0+0 (secure 0 0 0) 0x7fbb4c082fa0 con 0x7fbb54154220 2023-12-19T16:57:08.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:08.949+0000 7fbb4affd640 1 -- 172.21.15.26:0/3757807695 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbb4c0507e0 con 0x7fbb54154220 2023-12-19T16:57:09.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.137+0000 7fbb5c56d640 1 -- 172.21.15.26:0/3757807695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb540cdf60 con 0x7fbb54154220 2023-12-19T16:57:09.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.137+0000 7fbb4affd640 1 -- 172.21.15.26:0/3757807695 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fbb4c053e00 con 0x7fbb54154220 2023-12-19T16:57:09.143 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:09.143 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1947010592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1947010592},{"type":"v1","addr":"172.21.15.26:6833","nonce":1947010592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2764469548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2764469548},{"type":"v1","addr":"172.21.15.160:6837","nonce":2764469548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5803,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/255772664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":255772664},{"type":"v1","addr":"172.21.15.160:6836","nonce":255772664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:55:00.584757+0000","modified":"2023-12-19T16:55:03.604713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6000":{"gid":6000,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1847960813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1847960813},{"type":"v1","addr":"172.21.15.26:6835","nonce":1847960813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T16:57:09.144 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2023-12-19T16:57:09.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.141+0000 7fbb5c56d640 1 -- 172.21.15.26:0/3757807695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54154220 msgr2=0x7fbb540af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.141+0000 7fbb5c56d640 1 --2- 172.21.15.26:0/3757807695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54154220 0x7fbb540af210 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00c770 tx=0x7fbb4c00cc30 comp rx=0 tx=0).stop 2023-12-19T16:57:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.141+0000 7fbb5c56d640 1 -- 172.21.15.26:0/3757807695 shutdown_connections 2023-12-19T16:57:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.141+0000 7fbb5c56d640 1 --2- 172.21.15.26:0/3757807695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb54154220 0x7fbb540af210 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.141+0000 7fbb5c56d640 1 --2- 172.21.15.26:0/3757807695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb540cec40 0x7fbb54154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.141+0000 7fbb5c56d640 1 --2- 172.21.15.26:0/3757807695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb5406b3d0 0x7fbb540b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.141+0000 7fbb5c56d640 1 -- 172.21.15.26:0/3757807695 >> 172.21.15.26:0/3757807695 conn(0x7fbb540658e0 msgr2=0x7fbb540ae740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.141+0000 7fbb5c56d640 1 -- 172.21.15.26:0/3757807695 shutdown_connections 2023-12-19T16:57:09.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.141+0000 7fbb5c56d640 1 -- 172.21.15.26:0/3757807695 wait complete. 2023-12-19T16:57:09.164 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:09.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.301+0000 7f93a7c2d640 1 Processor -- start 2023-12-19T16:57:09.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.301+0000 7f93a7c2d640 1 -- start start 2023-12-19T16:57:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.301+0000 7f93a7c2d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f93a015f370 0x7f93a015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.301+0000 7f93a7c2d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a006c970 con 0x7f93a00ce740 2023-12-19T16:57:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.301+0000 7f93a7c2d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a015b1f0 con 0x7f93a0155030 2023-12-19T16:57:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.301+0000 7f93a7c2d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a006c790 con 0x7f93a015f370 2023-12-19T16:57:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.305+0000 7f93a51c8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f93a015f370 0x7f93a015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.305+0000 7f93a61ca640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f93a00ce740 0x7f93a00ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54094/0 (socket says 172.21.15.26:54094) 2023-12-19T16:57:09.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.305+0000 7f93a61ca640 1 -- 172.21.15.26:0/2149529161 learned_addr learned my addr 172.21.15.26:0/2149529161 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:09.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.305+0000 7f93a51c8640 1 -- 172.21.15.26:0/2149529161 >> v1:172.21.15.160:6789/0 conn(0x7f93a0155030 legacy=0x7f93a006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:09.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.305+0000 7f93a51c8640 1 -- 172.21.15.26:0/2149529161 >> v1:172.21.15.26:6789/0 conn(0x7f93a00ce740 legacy=0x7f93a00ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:09.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.305+0000 7f93a51c8640 1 -- 172.21.15.26:0/2149529161 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a015b4d0 con 0x7f93a015f370 2023-12-19T16:57:09.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.305+0000 7f93a51c8640 1 --2- 172.21.15.26:0/2149529161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a015f370 0x7f93a015bcd0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f93900097c0 tx=0x7f939001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bec02171765cbe server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:09.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.305+0000 7f93a49c7640 1 -- 172.21.15.26:0/2149529161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93900a7070 con 0x7f93a015f370 2023-12-19T16:57:09.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.305+0000 7f93a49c7640 1 -- 172.21.15.26:0/2149529161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93900ab430 con 0x7f93a015f370 2023-12-19T16:57:09.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.305+0000 7f93a49c7640 1 -- 172.21.15.26:0/2149529161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93900ab7c0 con 0x7f93a015f370 2023-12-19T16:57:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 -- 172.21.15.26:0/2149529161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a015f370 msgr2=0x7f93a015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 --2- 172.21.15.26:0/2149529161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a015f370 0x7f93a015bcd0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f93900097c0 tx=0x7f939001c410 comp rx=0 tx=0).stop 2023-12-19T16:57:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 -- 172.21.15.26:0/2149529161 shutdown_connections 2023-12-19T16:57:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 --2- 172.21.15.26:0/2149529161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a015f370 0x7f93a015bcd0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 -- 172.21.15.26:0/2149529161 >> 172.21.15.26:0/2149529161 conn(0x7f93a00658e0 msgr2=0x7f93a0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 -- 172.21.15.26:0/2149529161 shutdown_connections 2023-12-19T16:57:09.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 -- 172.21.15.26:0/2149529161 wait complete. 2023-12-19T16:57:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 Processor -- start 2023-12-19T16:57:09.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 -- start start 2023-12-19T16:57:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93a00ce740 0x7f93a0147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0155030 0x7f93a014a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a015f370 0x7f93a0135de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:09.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a014b510 con 0x7f93a0155030 2023-12-19T16:57:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a014af50 con 0x7f93a00ce740 2023-12-19T16:57:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a7c2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a014b230 con 0x7f93a015f370 2023-12-19T16:57:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a51c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0155030 0x7f93a014a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a51c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0155030 0x7f93a014a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55414/0 (socket says 172.21.15.26:55414) 2023-12-19T16:57:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a51c8640 1 -- 172.21.15.26:0/1570653478 learned_addr learned my addr 172.21.15.26:0/1570653478 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a59c9640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93a00ce740 0x7f93a0147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.309+0000 7f93a61ca640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a015f370 0x7f93a0135de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93a51c8640 1 -- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a015f370 msgr2=0x7f93a0135de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93a51c8640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a015f370 0x7f93a0135de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93a51c8640 1 -- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93a00ce740 msgr2=0x7f93a0147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93a51c8640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93a00ce740 0x7f93a0147990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93a51c8640 1 -- 172.21.15.26:0/1570653478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9390009420 con 0x7f93a0155030 2023-12-19T16:57:09.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93a51c8640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0155030 0x7f93a014a590 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f939001a040 tx=0x7f93900191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93967fc640 1 -- 172.21.15.26:0/1570653478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93900bf450 con 0x7f93a0155030 2023-12-19T16:57:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93967fc640 1 -- 172.21.15.26:0/1570653478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93900bf5b0 con 0x7f93a0155030 2023-12-19T16:57:09.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93a7c2d640 1 -- 172.21.15.26:0/1570653478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a00cf0e0 con 0x7f93a0155030 2023-12-19T16:57:09.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93967fc640 1 -- 172.21.15.26:0/1570653478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93900bf960 con 0x7f93a0155030 2023-12-19T16:57:09.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93a7c2d640 1 -- 172.21.15.26:0/1570653478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a00b1c00 con 0x7f93a0155030 2023-12-19T16:57:09.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93a7c2d640 1 -- 172.21.15.26:0/1570653478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a00b1df0 con 0x7f93a0155030 2023-12-19T16:57:09.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93967fc640 1 -- 172.21.15.26:0/1570653478 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 27) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f93900bfac0 con 0x7f93a0155030 2023-12-19T16:57:09.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.313+0000 7f93967fc640 1 -- 172.21.15.26:0/1570653478 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f939011e0b0 con 0x7f93a0155030 2023-12-19T16:57:09.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.317+0000 7f93967fc640 1 -- 172.21.15.26:0/1570653478 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f93900eb8f0 con 0x7f93a0155030 2023-12-19T16:57:09.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.405+0000 7f93967fc640 1 -- 172.21.15.26:0/1570653478 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 28) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f93900eb3a0 con 0x7f93a0155030 2023-12-19T16:57:09.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.405+0000 7f93967fc640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f938401c9b0 0x7f938406db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:09.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.405+0000 7f93a59c9640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f938401c9b0 0x7f938406db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:09.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.409+0000 7f93a59c9640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f938401c9b0 0x7f938406db40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f939c007fd0 tx=0x7f939c007e40 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:09.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.481+0000 7f93a7c2d640 1 -- 172.21.15.26:0/1570653478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93a014b370 con 0x7f93a0155030 2023-12-19T16:57:09.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.485+0000 7f93967fc640 1 -- 172.21.15.26:0/1570653478 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+18239 (secure 0 0 0) 0x7f93a014b370 con 0x7f93a0155030 2023-12-19T16:57:09.489 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:09.489 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":67,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:08.396778+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:54:58.260985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T16:54:59.644467+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.f","mappings":[{"from":0,"to":4}]},{"pgid":"7.2e","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1932793085":"2023-12-20T16:57:08.396745+0000","172.21.15.160:0/1963799314":"2023-12-20T16:57:08.396745+0000","172.21.15.160:0/3211010716":"2023-12-20T16:57:08.396745+0000","172.21.15.160:6832/38098":"2023-12-20T16:55:12.385158+0000","172.21.15.160:6833/38098":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/251239631":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/1854082733":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/1407525802":"2023-12-20T16:57:08.396745+0000","172.21.15.160:0/260287399":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/71378321":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/936066837":"2023-12-20T16:57:08.396745+0000","172.21.15.160:0/1751160424":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/1332973623":"2023-12-20T16:57:08.396745+0000","172.21.15.160:0/3640035110":"2023-12-20T16:55:12.385158+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.485+0000 7f93a7c2d640 1 -- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f938401c9b0 msgr2=0x7f938406db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.485+0000 7f93a7c2d640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f938401c9b0 0x7f938406db40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f939c007fd0 tx=0x7f939c007e40 comp rx=0 tx=0).stop 2023-12-19T16:57:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.485+0000 7f93a7c2d640 1 -- 172.21.15.26:0/1570653478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0155030 msgr2=0x7f93a014a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.485+0000 7f93a7c2d640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0155030 0x7f93a014a590 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f939001a040 tx=0x7f93900191b0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.485+0000 7f93a7c2d640 1 -- 172.21.15.26:0/1570653478 shutdown_connections 2023-12-19T16:57:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.485+0000 7f93a7c2d640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f938401c9b0 0x7f938406db40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.485+0000 7f93a7c2d640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a015f370 0x7f93a0135de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.485+0000 7f93a7c2d640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0155030 0x7f93a014a590 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.485+0000 7f93a7c2d640 1 --2- 172.21.15.26:0/1570653478 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93a00ce740 0x7f93a0147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.485+0000 7f93a7c2d640 1 -- 172.21.15.26:0/1570653478 >> 172.21.15.26:0/1570653478 conn(0x7f93a00658e0 msgr2=0x7f93a0065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.485+0000 7f93a7c2d640 1 -- 172.21.15.26:0/1570653478 shutdown_connections 2023-12-19T16:57:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.485+0000 7f93a7c2d640 1 -- 172.21.15.26:0/1570653478 wait complete. 2023-12-19T16:57:09.510 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T16:57:09.511 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:57:09.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.649+0000 7f62db109640 1 Processor -- start 2023-12-19T16:57:09.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.649+0000 7f62db109640 1 -- start start 2023-12-19T16:57:09.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62db109640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62d406b3d0 0x7f62d40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:09.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62db109640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f62d40cec40 0x7f62d415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:09.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62db109640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d406bc30 con 0x7f62d4154220 2023-12-19T16:57:09.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62db109640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d40cf0d0 con 0x7f62d406b3d0 2023-12-19T16:57:09.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62db109640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d40cc6e0 con 0x7f62d40cec40 2023-12-19T16:57:09.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62d3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62d406b3d0 0x7f62d40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:09.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62d3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62d406b3d0 0x7f62d40ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55424/0 (socket says 172.21.15.26:55424) 2023-12-19T16:57:09.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62d3fff640 1 -- 172.21.15.26:0/397121667 learned_addr learned my addr 172.21.15.26:0/397121667 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:09.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62d96a6640 1 --2- 172.21.15.26:0/397121667 >> v2:172.21.15.160:3300/0 conn(0x7f62d40cec40 0x7f62d415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62d3fff640 1 -- 172.21.15.26:0/397121667 >> v2:172.21.15.160:3300/0 conn(0x7f62d40cec40 msgr2=0x7f62d415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62d3fff640 1 --2- 172.21.15.26:0/397121667 >> v2:172.21.15.160:3300/0 conn(0x7f62d40cec40 0x7f62d415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62d3fff640 1 -- 172.21.15.26:0/397121667 >> v1:172.21.15.26:6789/0 conn(0x7f62d4154220 legacy=0x7f62d415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62d3fff640 1 -- 172.21.15.26:0/397121667 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d406b950 con 0x7f62d406b3d0 2023-12-19T16:57:09.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62d37fe640 1 -- 172.21.15.26:0/397121667 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317197717 0 0) 0x7f62d406bc30 con 0x7f62d4154220 2023-12-19T16:57:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.653+0000 7f62d3fff640 1 --2- 172.21.15.26:0/397121667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62d406b3d0 0x7f62d40ce700 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f62d4157bd0 tx=0x7f62c001eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac384b3c96ddbbb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62d37fe640 1 -- 172.21.15.26:0/397121667 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c00a7c70 con 0x7f62d406b3d0 2023-12-19T16:57:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62d37fe640 1 -- 172.21.15.26:0/397121667 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c00a7dd0 con 0x7f62d406b3d0 2023-12-19T16:57:09.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62d37fe640 1 -- 172.21.15.26:0/397121667 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c00af660 con 0x7f62d406b3d0 2023-12-19T16:57:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 -- 172.21.15.26:0/397121667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62d406b3d0 msgr2=0x7f62d40ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 --2- 172.21.15.26:0/397121667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62d406b3d0 0x7f62d40ce700 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f62d4157bd0 tx=0x7f62c001eba0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 -- 172.21.15.26:0/397121667 shutdown_connections 2023-12-19T16:57:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 --2- 172.21.15.26:0/397121667 >> v2:172.21.15.160:3300/0 conn(0x7f62d40cec40 0x7f62d415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 --2- 172.21.15.26:0/397121667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62d406b3d0 0x7f62d40ce700 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 -- 172.21.15.26:0/397121667 >> 172.21.15.26:0/397121667 conn(0x7f62d40658e0 msgr2=0x7f62d4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 -- 172.21.15.26:0/397121667 shutdown_connections 2023-12-19T16:57:09.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 -- 172.21.15.26:0/397121667 wait complete. 2023-12-19T16:57:09.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 Processor -- start 2023-12-19T16:57:09.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 -- start start 2023-12-19T16:57:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62d406b3d0 0x7f62d4154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d40cec40 0x7f62d415cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62d4154220 0x7f62d406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d414dc00 con 0x7f62d406b3d0 2023-12-19T16:57:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d414b510 con 0x7f62d40cec40 2023-12-19T16:57:09.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62db109640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d415d2f0 con 0x7f62d4154220 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.657+0000 7f62d8ea5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62d406b3d0 0x7f62d4154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d8ea5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62d406b3d0 0x7f62d4154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55430/0 (socket says 172.21.15.26:55430) 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d8ea5640 1 -- 172.21.15.26:0/1739641848 learned_addr learned my addr 172.21.15.26:0/1739641848 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d3fff640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d40cec40 0x7f62d415cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d8ea5640 1 -- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62d4154220 msgr2=0x7f62d406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d8ea5640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62d4154220 0x7f62d406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d8ea5640 1 -- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d40cec40 msgr2=0x7f62d415cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d8ea5640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d40cec40 0x7f62d415cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d8ea5640 1 -- 172.21.15.26:0/1739641848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c000b770 con 0x7f62d406b3d0 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d3fff640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d40cec40 0x7f62d415cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d8ea5640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62d406b3d0 0x7f62d4154690 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c4d0 tx=0x7f62c800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d17fa640 1 -- 172.21.15.26:0/1739641848 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8005480 con 0x7f62d406b3d0 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d17fa640 1 -- 172.21.15.26:0/1739641848 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c8003600 con 0x7f62d406b3d0 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d17fa640 1 -- 172.21.15.26:0/1739641848 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80039b0 con 0x7f62d406b3d0 2023-12-19T16:57:09.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62db109640 1 -- 172.21.15.26:0/1739641848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d414cf80 con 0x7f62d406b3d0 2023-12-19T16:57:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62db109640 1 -- 172.21.15.26:0/1739641848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d414d540 con 0x7f62d406b3d0 2023-12-19T16:57:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62db109640 1 -- 172.21.15.26:0/1739641848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d4074270 con 0x7f62d406b3d0 2023-12-19T16:57:09.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d17fa640 1 -- 172.21.15.26:0/1739641848 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f62c8003b10 con 0x7f62d406b3d0 2023-12-19T16:57:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d17fa640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f62b80658b0 0x7f62b8067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.661+0000 7f62d17fa640 1 -- 172.21.15.26:0/1739641848 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f62c80829b0 con 0x7f62d406b3d0 2023-12-19T16:57:09.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.665+0000 7f62d3fff640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f62b80658b0 0x7f62b8067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:09.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.665+0000 7f62d3fff640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f62b80658b0 0x7f62b8067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f62c00b1000 tx=0x7f62c00a9070 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:09.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.669+0000 7f62d17fa640 1 -- 172.21.15.26:0/1739641848 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f62c8050110 con 0x7f62d406b3d0 2023-12-19T16:57:09.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.857+0000 7f62db109640 1 -- 172.21.15.26:0/1739641848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62d40b0160 con 0x7f62d406b3d0 2023-12-19T16:57:09.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.857+0000 7f62d17fa640 1 -- 172.21.15.26:0/1739641848 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f62c8017020 con 0x7f62d406b3d0 2023-12-19T16:57:09.864 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:09.864 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1947010592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1947010592},{"type":"v1","addr":"172.21.15.26:6833","nonce":1947010592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2764469548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2764469548},{"type":"v1","addr":"172.21.15.160:6837","nonce":2764469548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5803,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/255772664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":255772664},{"type":"v1","addr":"172.21.15.160:6836","nonce":255772664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:55:00.584757+0000","modified":"2023-12-19T16:55:03.604713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6000":{"gid":6000,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1847960813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1847960813},{"type":"v1","addr":"172.21.15.26:6835","nonce":1847960813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T16:57:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2023-12-19T16:57:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.861+0000 7f62db109640 1 -- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f62b80658b0 msgr2=0x7f62b8067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:09.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.861+0000 7f62db109640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f62b80658b0 0x7f62b8067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f62c00b1000 tx=0x7f62c00a9070 comp rx=0 tx=0).stop 2023-12-19T16:57:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.861+0000 7f62db109640 1 -- 172.21.15.26:0/1739641848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62d406b3d0 msgr2=0x7f62d4154690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.861+0000 7f62db109640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62d406b3d0 0x7f62d4154690 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c4d0 tx=0x7f62c800c990 comp rx=0 tx=0).stop 2023-12-19T16:57:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.861+0000 7f62db109640 1 -- 172.21.15.26:0/1739641848 shutdown_connections 2023-12-19T16:57:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.861+0000 7f62db109640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f62b80658b0 0x7f62b8067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.861+0000 7f62db109640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62d4154220 0x7f62d406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.861+0000 7f62db109640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62d40cec40 0x7f62d415cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.861+0000 7f62db109640 1 --2- 172.21.15.26:0/1739641848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62d406b3d0 0x7f62d4154690 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:09.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.861+0000 7f62db109640 1 -- 172.21.15.26:0/1739641848 >> 172.21.15.26:0/1739641848 conn(0x7f62d40658e0 msgr2=0x7f62d4065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:09.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.861+0000 7f62db109640 1 -- 172.21.15.26:0/1739641848 shutdown_connections 2023-12-19T16:57:09.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:09.861+0000 7f62db109640 1 -- 172.21.15.26:0/1739641848 wait complete. 2023-12-19T16:57:09.886 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:57:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.025+0000 7f6b15339640 1 Processor -- start 2023-12-19T16:57:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.025+0000 7f6b15339640 1 -- start start 2023-12-19T16:57:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b15339640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6b1015c7f0 0x7f6b1006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b15339640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b100b30d0 0x7f6b100b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b15339640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b100b3a20 con 0x7f6b100adb80 2023-12-19T16:57:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b15339640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1006d770 con 0x7f6b100b30d0 2023-12-19T16:57:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b15339640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1006ced0 con 0x7f6b1015c7f0 2023-12-19T16:57:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b0f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b100b30d0 0x7f6b100b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b0f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b100b30d0 0x7f6b100b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55442/0 (socket says 172.21.15.26:55442) 2023-12-19T16:57:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b07fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6b100adb80 0x7f6b100cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54114/0 (socket says 172.21.15.26:54114) 2023-12-19T16:57:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b0f7fe640 1 -- 172.21.15.26:0/3659432784 learned_addr learned my addr 172.21.15.26:0/3659432784 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b0effd640 1 --2- 172.21.15.26:0/3659432784 >> v2:172.21.15.160:3300/0 conn(0x7f6b1015c7f0 0x7f6b1006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b0f7fe640 1 -- 172.21.15.26:0/3659432784 >> v2:172.21.15.160:3300/0 conn(0x7f6b1015c7f0 msgr2=0x7f6b1006c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b0f7fe640 1 --2- 172.21.15.26:0/3659432784 >> v2:172.21.15.160:3300/0 conn(0x7f6b1015c7f0 0x7f6b1006c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b0f7fe640 1 -- 172.21.15.26:0/3659432784 >> v1:172.21.15.26:6789/0 conn(0x7f6b100adb80 legacy=0x7f6b100cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.029+0000 7f6b0f7fe640 1 -- 172.21.15.26:0/3659432784 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1006d1b0 con 0x7f6b100b30d0 2023-12-19T16:57:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b0f7fe640 1 --2- 172.21.15.26:0/3659432784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100b30d0 0x7f6b100b34e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6b100670d0 tx=0x7f6b0801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a622f5cdea0703 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b0e7fc640 1 -- 172.21.15.26:0/3659432784 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080a55f0 con 0x7f6b100b30d0 2023-12-19T16:57:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b0e7fc640 1 -- 172.21.15.26:0/3659432784 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b080a5750 con 0x7f6b100b30d0 2023-12-19T16:57:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b0e7fc640 1 -- 172.21.15.26:0/3659432784 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080a5ae0 con 0x7f6b100b30d0 2023-12-19T16:57:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b15339640 1 -- 172.21.15.26:0/3659432784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100b30d0 msgr2=0x7f6b100b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b15339640 1 --2- 172.21.15.26:0/3659432784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100b30d0 0x7f6b100b34e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6b100670d0 tx=0x7f6b0801c9d0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b15339640 1 -- 172.21.15.26:0/3659432784 shutdown_connections 2023-12-19T16:57:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b15339640 1 --2- 172.21.15.26:0/3659432784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100b30d0 0x7f6b100b34e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b15339640 1 --2- 172.21.15.26:0/3659432784 >> v2:172.21.15.160:3300/0 conn(0x7f6b1015c7f0 0x7f6b1006c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b15339640 1 -- 172.21.15.26:0/3659432784 >> 172.21.15.26:0/3659432784 conn(0x7f6b100658e0 msgr2=0x7f6b10065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b15339640 1 -- 172.21.15.26:0/3659432784 shutdown_connections 2023-12-19T16:57:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b15339640 1 -- 172.21.15.26:0/3659432784 wait complete. 2023-12-19T16:57:10.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b15339640 1 Processor -- start 2023-12-19T16:57:10.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.033+0000 7f6b15339640 1 -- start start 2023-12-19T16:57:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b15339640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b100adb80 0x7f6b10136a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b15339640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b100b30d0 0x7f6b10167030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b15339640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b1015c7f0 0x7f6b10134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b15339640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b10166ba0 con 0x7f6b1015c7f0 2023-12-19T16:57:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b15339640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b10165f90 con 0x7f6b100b30d0 2023-12-19T16:57:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b15339640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b10166270 con 0x7f6b100adb80 2023-12-19T16:57:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b0f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b1015c7f0 0x7f6b10134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b0f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b1015c7f0 0x7f6b10134180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55454/0 (socket says 172.21.15.26:55454) 2023-12-19T16:57:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b0f7fe640 1 -- 172.21.15.26:0/1536958941 learned_addr learned my addr 172.21.15.26:0/1536958941 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b07fff640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b100b30d0 0x7f6b10167030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b0effd640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b100adb80 0x7f6b10136a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b0f7fe640 1 -- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b100adb80 msgr2=0x7f6b10136a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b0f7fe640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b100adb80 0x7f6b10136a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b0f7fe640 1 -- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b100b30d0 msgr2=0x7f6b10167030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b0f7fe640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b100b30d0 0x7f6b10167030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b0f7fe640 1 -- 172.21.15.26:0/1536958941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b08009b30 con 0x7f6b1015c7f0 2023-12-19T16:57:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b0f7fe640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b1015c7f0 0x7f6b10134180 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6b080195e0 tx=0x7f6b08019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b077fe640 1 -- 172.21.15.26:0/1536958941 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080a55f0 con 0x7f6b1015c7f0 2023-12-19T16:57:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b077fe640 1 -- 172.21.15.26:0/1536958941 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b080b8690 con 0x7f6b1015c7f0 2023-12-19T16:57:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b15339640 1 -- 172.21.15.26:0/1536958941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b10166550 con 0x7f6b1015c7f0 2023-12-19T16:57:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b077fe640 1 -- 172.21.15.26:0/1536958941 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080b8a40 con 0x7f6b1015c7f0 2023-12-19T16:57:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b15339640 1 -- 172.21.15.26:0/1536958941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b10136fc0 con 0x7f6b1015c7f0 2023-12-19T16:57:10.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b15339640 1 -- 172.21.15.26:0/1536958941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b10166d50 con 0x7f6b1015c7f0 2023-12-19T16:57:10.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.037+0000 7f6b077fe640 1 -- 172.21.15.26:0/1536958941 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f6b080b8ba0 con 0x7f6b1015c7f0 2023-12-19T16:57:10.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.041+0000 7f6b077fe640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6af00658b0 0x7f6af0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.041+0000 7f6b077fe640 1 -- 172.21.15.26:0/1536958941 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f6b0811e540 con 0x7f6b1015c7f0 2023-12-19T16:57:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.041+0000 7f6b0effd640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6af00658b0 0x7f6af0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.041+0000 7f6b0effd640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6af00658b0 0x7f6af0067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009ac0 tx=0x7f6b00009190 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:10.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.045+0000 7f6b077fe640 1 -- 172.21.15.26:0/1536958941 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6b10166d50 con 0x7f6b1015c7f0 2023-12-19T16:57:10.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b15339640 1 -- 172.21.15.26:0/1536958941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b10074270 con 0x7f6b1015c7f0 2023-12-19T16:57:10.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b077fe640 1 -- 172.21.15.26:0/1536958941 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f6b080bc090 con 0x7f6b1015c7f0 2023-12-19T16:57:10.237 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:10.237 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1947010592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1947010592},{"type":"v1","addr":"172.21.15.26:6833","nonce":1947010592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2764469548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2764469548},{"type":"v1","addr":"172.21.15.160:6837","nonce":2764469548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5803,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/255772664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":255772664},{"type":"v1","addr":"172.21.15.160:6836","nonce":255772664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:55:00.584757+0000","modified":"2023-12-19T16:55:03.604713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6000":{"gid":6000,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1847960813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1847960813},{"type":"v1","addr":"172.21.15.26:6835","nonce":1847960813}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T16:57:10.237 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2023-12-19T16:57:10.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b15339640 1 -- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6af00658b0 msgr2=0x7f6af0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b15339640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6af00658b0 0x7f6af0067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009ac0 tx=0x7f6b00009190 comp rx=0 tx=0).stop 2023-12-19T16:57:10.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b15339640 1 -- 172.21.15.26:0/1536958941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b1015c7f0 msgr2=0x7f6b10134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b15339640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b1015c7f0 0x7f6b10134180 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6b080195e0 tx=0x7f6b08019770 comp rx=0 tx=0).stop 2023-12-19T16:57:10.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b15339640 1 -- 172.21.15.26:0/1536958941 shutdown_connections 2023-12-19T16:57:10.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b15339640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b1015c7f0 0x7f6b10134180 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b15339640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b100b30d0 0x7f6b10167030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b15339640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6af00658b0 0x7f6af0067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b15339640 1 --2- 172.21.15.26:0/1536958941 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b100adb80 0x7f6b10136a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b15339640 1 -- 172.21.15.26:0/1536958941 >> 172.21.15.26:0/1536958941 conn(0x7f6b100658e0 msgr2=0x7f6b10148a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b15339640 1 -- 172.21.15.26:0/1536958941 shutdown_connections 2023-12-19T16:57:10.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.233+0000 7f6b15339640 1 -- 172.21.15.26:0/1536958941 wait complete. 2023-12-19T16:57:10.258 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.397+0000 7ffb8a895640 1 Processor -- start 2023-12-19T16:57:10.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.397+0000 7ffb8a895640 1 -- start start 2023-12-19T16:57:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.397+0000 7ffb8a895640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb840d0480 0x7ffb84069d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.397+0000 7ffb8a895640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb84160890 con 0x7ffb840ccc20 2023-12-19T16:57:10.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.397+0000 7ffb8a895640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb840cdaf0 con 0x7ffb84161620 2023-12-19T16:57:10.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.397+0000 7ffb8a895640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb84160b40 con 0x7ffb840d0480 2023-12-19T16:57:10.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.401+0000 7ffb83fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ffb840ccc20 0x7ffb840cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54118/0 (socket says 172.21.15.26:54118) 2023-12-19T16:57:10.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.401+0000 7ffb83fff640 1 -- 172.21.15.26:0/2744024467 learned_addr learned my addr 172.21.15.26:0/2744024467 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:10.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.401+0000 7ffb88e32640 1 --2- 172.21.15.26:0/2744024467 >> v2:172.21.15.26:3300/0 conn(0x7ffb840d0480 0x7ffb84069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.401+0000 7ffb88e32640 1 -- 172.21.15.26:0/2744024467 >> v1:172.21.15.160:6789/0 conn(0x7ffb84161620 legacy=0x7ffb840cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.401+0000 7ffb88e32640 1 -- 172.21.15.26:0/2744024467 >> v1:172.21.15.26:6789/0 conn(0x7ffb840ccc20 legacy=0x7ffb840cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.401+0000 7ffb88e32640 1 -- 172.21.15.26:0/2744024467 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb84160e20 con 0x7ffb840d0480 2023-12-19T16:57:10.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.401+0000 7ffb88e32640 1 --2- 172.21.15.26:0/2744024467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb840d0480 0x7ffb84069d20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009d90 tx=0x7ffb7c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8eaca8219a916b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:10.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.405+0000 7ffb82ffd640 1 -- 172.21.15.26:0/2744024467 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0a5890 con 0x7ffb840d0480 2023-12-19T16:57:10.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.405+0000 7ffb82ffd640 1 -- 172.21.15.26:0/2744024467 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c0a59f0 con 0x7ffb840d0480 2023-12-19T16:57:10.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.405+0000 7ffb82ffd640 1 -- 172.21.15.26:0/2744024467 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0a5d80 con 0x7ffb840d0480 2023-12-19T16:57:10.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.405+0000 7ffb8a895640 1 -- 172.21.15.26:0/2744024467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb840d0480 msgr2=0x7ffb84069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.405+0000 7ffb8a895640 1 --2- 172.21.15.26:0/2744024467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb840d0480 0x7ffb84069d20 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009d90 tx=0x7ffb7c01c8c0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.405+0000 7ffb8a895640 1 -- 172.21.15.26:0/2744024467 shutdown_connections 2023-12-19T16:57:10.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.405+0000 7ffb8a895640 1 --2- 172.21.15.26:0/2744024467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb840d0480 0x7ffb84069d20 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.405+0000 7ffb8a895640 1 -- 172.21.15.26:0/2744024467 >> 172.21.15.26:0/2744024467 conn(0x7ffb8406d510 msgr2=0x7ffb8406d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:10.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.405+0000 7ffb8a895640 1 -- 172.21.15.26:0/2744024467 shutdown_connections 2023-12-19T16:57:10.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.405+0000 7ffb8a895640 1 -- 172.21.15.26:0/2744024467 wait complete. 2023-12-19T16:57:10.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb8a895640 1 Processor -- start 2023-12-19T16:57:10.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb8a895640 1 -- start start 2023-12-19T16:57:10.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb8a895640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb840ccc20 0x7ffb840af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb8a895640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb840d0480 0x7ffb840c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb8a895640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb84161620 0x7ffb841465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb8a895640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb840c79b0 con 0x7ffb840d0480 2023-12-19T16:57:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb8a895640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb840c7520 con 0x7ffb840ccc20 2023-12-19T16:57:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb8a895640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb840c6910 con 0x7ffb84161620 2023-12-19T16:57:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb840d0480 0x7ffb840c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb840d0480 0x7ffb840c47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55472/0 (socket says 172.21.15.26:55472) 2023-12-19T16:57:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb837fe640 1 -- 172.21.15.26:0/353281437 learned_addr learned my addr 172.21.15.26:0/353281437 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb83fff640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb840ccc20 0x7ffb840af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb88e32640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb84161620 0x7ffb841465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb837fe640 1 -- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb84161620 msgr2=0x7ffb841465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb837fe640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb84161620 0x7ffb841465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb837fe640 1 -- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb840ccc20 msgr2=0x7ffb840af440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb837fe640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb840ccc20 0x7ffb840af440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb837fe640 1 -- 172.21.15.26:0/353281437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c009a70 con 0x7ffb840d0480 2023-12-19T16:57:10.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb83fff640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb840ccc20 0x7ffb840af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:10.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb837fe640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb840d0480 0x7ffb840c47e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800c3c0 tx=0x7ffb7800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:10.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb80ff9640 1 -- 172.21.15.26:0/353281437 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb780052d0 con 0x7ffb840d0480 2023-12-19T16:57:10.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb80ff9640 1 -- 172.21.15.26:0/353281437 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb78005430 con 0x7ffb840d0480 2023-12-19T16:57:10.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb8a895640 1 -- 172.21.15.26:0/353281437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb840c6bf0 con 0x7ffb840d0480 2023-12-19T16:57:10.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb80ff9640 1 -- 172.21.15.26:0/353281437 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78003840 con 0x7ffb840d0480 2023-12-19T16:57:10.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb8a895640 1 -- 172.21.15.26:0/353281437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb840c71b0 con 0x7ffb840d0480 2023-12-19T16:57:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.409+0000 7ffb8a895640 1 -- 172.21.15.26:0/353281437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb840c7690 con 0x7ffb840d0480 2023-12-19T16:57:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.413+0000 7ffb80ff9640 1 -- 172.21.15.26:0/353281437 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 85066+0+0 (secure 0 0 0) 0x7ffb780039a0 con 0x7ffb840d0480 2023-12-19T16:57:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.413+0000 7ffb80ff9640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb700658b0 0x7ffb70067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.413+0000 7ffb80ff9640 1 -- 172.21.15.26:0/353281437 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7129+0+0 (secure 0 0 0) 0x7ffb78082950 con 0x7ffb840d0480 2023-12-19T16:57:10.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.413+0000 7ffb83fff640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb700658b0 0x7ffb70067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.413+0000 7ffb83fff640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb700658b0 0x7ffb70067d70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb74005890 tx=0x7ffb74009040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:10.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.417+0000 7ffb80ff9640 1 -- 172.21.15.26:0/353281437 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffb7804fff0 con 0x7ffb840d0480 2023-12-19T16:57:10.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb8a895640 1 -- 172.21.15.26:0/353281437 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb84074270 con 0x7ffb840d0480 2023-12-19T16:57:10.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb80ff9640 1 -- 172.21.15.26:0/353281437 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+18239 (secure 0 0 0) 0x7ffb78053610 con 0x7ffb840d0480 2023-12-19T16:57:10.582 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:10.582 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":67,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:08.396778+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:54:58.260985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T16:54:59.644467+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.f","mappings":[{"from":0,"to":4}]},{"pgid":"7.2e","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1932793085":"2023-12-20T16:57:08.396745+0000","172.21.15.160:0/1963799314":"2023-12-20T16:57:08.396745+0000","172.21.15.160:0/3211010716":"2023-12-20T16:57:08.396745+0000","172.21.15.160:6832/38098":"2023-12-20T16:55:12.385158+0000","172.21.15.160:6833/38098":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/251239631":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/1854082733":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/1407525802":"2023-12-20T16:57:08.396745+0000","172.21.15.160:0/260287399":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/71378321":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/936066837":"2023-12-20T16:57:08.396745+0000","172.21.15.160:0/1751160424":"2023-12-20T16:55:12.385158+0000","172.21.15.160:0/1332973623":"2023-12-20T16:57:08.396745+0000","172.21.15.160:0/3640035110":"2023-12-20T16:55:12.385158+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb8a895640 1 -- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb700658b0 msgr2=0x7ffb70067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb8a895640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb700658b0 0x7ffb70067d70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb74005890 tx=0x7ffb74009040 comp rx=0 tx=0).stop 2023-12-19T16:57:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb8a895640 1 -- 172.21.15.26:0/353281437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb840d0480 msgr2=0x7ffb840c47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb8a895640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb840d0480 0x7ffb840c47e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800c3c0 tx=0x7ffb7800c880 comp rx=0 tx=0).stop 2023-12-19T16:57:10.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb8a895640 1 -- 172.21.15.26:0/353281437 shutdown_connections 2023-12-19T16:57:10.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb8a895640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb700658b0 0x7ffb70067d70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb8a895640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb84161620 0x7ffb841465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb8a895640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb840d0480 0x7ffb840c47e0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb8a895640 1 --2- 172.21.15.26:0/353281437 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb840ccc20 0x7ffb840af440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb8a895640 1 -- 172.21.15.26:0/353281437 >> 172.21.15.26:0/353281437 conn(0x7ffb8406d510 msgr2=0x7ffb8415bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:10.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb8a895640 1 -- 172.21.15.26:0/353281437 shutdown_connections 2023-12-19T16:57:10.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.577+0000 7ffb8a895640 1 -- 172.21.15.26:0/353281437 wait complete. 2023-12-19T16:57:10.604 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T16:57:10.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.745+0000 7fcd91f7b640 1 Processor -- start 2023-12-19T16:57:10.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.745+0000 7fcd91f7b640 1 -- start start 2023-12-19T16:57:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd91f7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd8c06b3d0 0x7fcd8c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd91f7b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd8c0cec40 0x7fcd8c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd91f7b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c06bc30 con 0x7fcd8c154220 2023-12-19T16:57:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd91f7b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8c0cf0d0 con 0x7fcd8c0cec40 2023-12-19T16:57:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd91f7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0cc6e0 con 0x7fcd8c06b3d0 2023-12-19T16:57:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd8bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd8c0cec40 0x7fcd8c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd8bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd8c0cec40 0x7fcd8c15b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55488/0 (socket says 172.21.15.26:55488) 2023-12-19T16:57:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd8bfff640 1 -- 172.21.15.26:0/2367862815 learned_addr learned my addr 172.21.15.26:0/2367862815 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd8affd640 1 --2- 172.21.15.26:0/2367862815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd8c06b3d0 0x7fcd8c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd8bfff640 1 -- 172.21.15.26:0/2367862815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd8c06b3d0 msgr2=0x7fcd8c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd8bfff640 1 --2- 172.21.15.26:0/2367862815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd8c06b3d0 0x7fcd8c0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd8bfff640 1 -- 172.21.15.26:0/2367862815 >> v1:172.21.15.160:6789/0 conn(0x7fcd8c154220 legacy=0x7fcd8c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd8bfff640 1 -- 172.21.15.26:0/2367862815 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c06b950 con 0x7fcd8c0cec40 2023-12-19T16:57:10.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd8affd640 1 --2- 172.21.15.26:0/2367862815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd8c06b3d0 0x7fcd8c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:10.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.749+0000 7fcd8bfff640 1 --2- 172.21.15.26:0/2367862815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd8c0cec40 0x7fcd8c15b1c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009e50 tx=0x7fcd8401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b08e62a5e5aa52b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:10.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd8a7fc640 1 -- 172.21.15.26:0/2367862815 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840a58a0 con 0x7fcd8c0cec40 2023-12-19T16:57:10.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd8a7fc640 1 -- 172.21.15.26:0/2367862815 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd840a5a00 con 0x7fcd8c0cec40 2023-12-19T16:57:10.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd8a7fc640 1 -- 172.21.15.26:0/2367862815 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840a5d90 con 0x7fcd8c0cec40 2023-12-19T16:57:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2367862815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd8c0cec40 msgr2=0x7fcd8c15b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 --2- 172.21.15.26:0/2367862815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd8c0cec40 0x7fcd8c15b1c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009e50 tx=0x7fcd8401c9e0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2367862815 shutdown_connections 2023-12-19T16:57:10.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 --2- 172.21.15.26:0/2367862815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd8c0cec40 0x7fcd8c15b1c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 --2- 172.21.15.26:0/2367862815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd8c06b3d0 0x7fcd8c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2367862815 >> 172.21.15.26:0/2367862815 conn(0x7fcd8c0658e0 msgr2=0x7fcd8c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2367862815 shutdown_connections 2023-12-19T16:57:10.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2367862815 wait complete. 2023-12-19T16:57:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 Processor -- start 2023-12-19T16:57:10.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 -- start start 2023-12-19T16:57:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd8c06b3d0 0x7fcd8c146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd8c0cec40 0x7fcd8c0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd8c154220 0x7fcd8c145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c0b2530 con 0x7fcd8c154220 2023-12-19T16:57:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c145e80 con 0x7fcd8c06b3d0 2023-12-19T16:57:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.753+0000 7fcd91f7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c152740 con 0x7fcd8c0cec40 2023-12-19T16:57:10.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd8bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd8c154220 0x7fcd8c145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd8bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd8c154220 0x7fcd8c145940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55494/0 (socket says 172.21.15.26:55494) 2023-12-19T16:57:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd8b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd8c06b3d0 0x7fcd8c146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd8affd640 1 --2- 172.21.15.26:0/2093576212 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd8c0cec40 0x7fcd8c0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd8bfff640 1 -- 172.21.15.26:0/2093576212 learned_addr learned my addr 172.21.15.26:0/2093576212 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd8bfff640 1 -- 172.21.15.26:0/2093576212 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd8c0cec40 msgr2=0x7fcd8c0b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd8bfff640 1 --2- 172.21.15.26:0/2093576212 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd8c0cec40 0x7fcd8c0b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd8bfff640 1 -- 172.21.15.26:0/2093576212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd8c06b3d0 msgr2=0x7fcd8c146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd8bfff640 1 --2- 172.21.15.26:0/2093576212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd8c06b3d0 0x7fcd8c146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd8bfff640 1 -- 172.21.15.26:0/2093576212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd84009b30 con 0x7fcd8c154220 2023-12-19T16:57:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd8bfff640 1 --2- 172.21.15.26:0/2093576212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd8c154220 0x7fcd8c145940 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcd840af040 tx=0x7fcd84019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd73fff640 1 -- 172.21.15.26:0/2093576212 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840b8670 con 0x7fcd8c154220 2023-12-19T16:57:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd73fff640 1 -- 172.21.15.26:0/2093576212 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd840b87d0 con 0x7fcd8c154220 2023-12-19T16:57:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd73fff640 1 -- 172.21.15.26:0/2093576212 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840b8b60 con 0x7fcd8c154220 2023-12-19T16:57:10.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2093576212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c152a20 con 0x7fcd8c154220 2023-12-19T16:57:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2093576212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c0b2150 con 0x7fcd8c154220 2023-12-19T16:57:10.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2093576212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8c074270 con 0x7fcd8c154220 2023-12-19T16:57:10.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.757+0000 7fcd73fff640 1 -- 172.21.15.26:0/2093576212 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fcd840b8cc0 con 0x7fcd8c154220 2023-12-19T16:57:10.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.761+0000 7fcd73fff640 1 --2- 172.21.15.26:0/2093576212 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd78065af0 0x7fcd78067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:10.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.761+0000 7fcd73fff640 1 -- 172.21.15.26:0/2093576212 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7129+0+0 (secure 0 0 0) 0x7fcd8411ec40 con 0x7fcd8c154220 2023-12-19T16:57:10.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.761+0000 7fcd8b7fe640 1 --2- 172.21.15.26:0/2093576212 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd78065af0 0x7fcd78067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:10.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.761+0000 7fcd8b7fe640 1 --2- 172.21.15.26:0/2093576212 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd78065af0 0x7fcd78067fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0cf730 tx=0x7fcd7c009040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:10.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.765+0000 7fcd73fff640 1 -- 172.21.15.26:0/2093576212 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcd840ec1e0 con 0x7fcd8c154220 2023-12-19T16:57:10.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.953+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2093576212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcd8c1579b0 con 0x7fcd8c154220 2023-12-19T16:57:10.960 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T16:57:10.957+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T16:57:10.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.965+0000 7fcd73fff640 1 -- 172.21.15.26:0/2093576212 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcd840bc090 con 0x7fcd8c154220 2023-12-19T16:57:10.971 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T16:57:10.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.969+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2093576212 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd78065af0 msgr2=0x7fcd78067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.969+0000 7fcd91f7b640 1 --2- 172.21.15.26:0/2093576212 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd78065af0 0x7fcd78067fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0cf730 tx=0x7fcd7c009040 comp rx=0 tx=0).stop 2023-12-19T16:57:10.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.969+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2093576212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd8c154220 msgr2=0x7fcd8c145940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:10.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.969+0000 7fcd91f7b640 1 --2- 172.21.15.26:0/2093576212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd8c154220 0x7fcd8c145940 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcd840af040 tx=0x7fcd84019780 comp rx=0 tx=0).stop 2023-12-19T16:57:10.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.969+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2093576212 shutdown_connections 2023-12-19T16:57:10.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.969+0000 7fcd91f7b640 1 --2- 172.21.15.26:0/2093576212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd8c154220 0x7fcd8c145940 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.969+0000 7fcd91f7b640 1 --2- 172.21.15.26:0/2093576212 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd78065af0 0x7fcd78067fb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.969+0000 7fcd91f7b640 1 --2- 172.21.15.26:0/2093576212 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd8c0cec40 0x7fcd8c0b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.969+0000 7fcd91f7b640 1 --2- 172.21.15.26:0/2093576212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd8c06b3d0 0x7fcd8c146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:10.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.969+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2093576212 >> 172.21.15.26:0/2093576212 conn(0x7fcd8c0658e0 msgr2=0x7fcd8c15c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:10.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.969+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2093576212 shutdown_connections 2023-12-19T16:57:10.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:10.969+0000 7fcd91f7b640 1 -- 172.21.15.26:0/2093576212 wait complete. 2023-12-19T16:57:10.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T16:57:11.036 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T16:57:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.137+0000 7f86697a2640 1 Processor -- start 2023-12-19T16:57:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.137+0000 7f86697a2640 1 -- start start 2023-12-19T16:57:11.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.137+0000 7f86697a2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8664159d20 0x7f86640ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:11.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.137+0000 7f86697a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86640cdee0 0x7f86640b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:11.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.137+0000 7f86697a2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86640adde0 con 0x7f86640cc0a0 2023-12-19T16:57:11.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.137+0000 7f86697a2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86640c9c20 con 0x7f8664159d20 2023-12-19T16:57:11.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.137+0000 7f86697a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86640b28d0 con 0x7f86640cdee0 2023-12-19T16:57:11.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.137+0000 7f86637fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8664159d20 0x7f86640ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:11.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.137+0000 7f8663fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f86640cc0a0 0x7f86640cc480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54126/0 (socket says 172.21.15.26:54126) 2023-12-19T16:57:11.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.137+0000 7f8663fff640 1 -- 172.21.15.26:0/1236132318 learned_addr learned my addr 172.21.15.26:0/1236132318 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:11.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.141+0000 7f8668fa1640 1 --2- 172.21.15.26:0/1236132318 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86640cdee0 0x7f86640b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:11.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.141+0000 7f86637fe640 1 -- 172.21.15.26:0/1236132318 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86640cdee0 msgr2=0x7f86640b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:11.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.141+0000 7f86637fe640 1 --2- 172.21.15.26:0/1236132318 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86640cdee0 0x7f86640b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:11.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.141+0000 7f86637fe640 1 -- 172.21.15.26:0/1236132318 >> v1:172.21.15.26:6789/0 conn(0x7f86640cc0a0 legacy=0x7f86640cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:11.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.141+0000 7f86637fe640 1 -- 172.21.15.26:0/1236132318 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640adb00 con 0x7f8664159d20 2023-12-19T16:57:11.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.141+0000 7f8662ffd640 1 -- 172.21.15.26:0/1236132318 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590456026 0 0) 0x7f86640adde0 con 0x7f86640cc0a0 2023-12-19T16:57:11.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.141+0000 7f86637fe640 1 --2- 172.21.15.26:0/1236132318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8664159d20 0x7f86640ccb90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f86580097c0 tx=0x7f865801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf0cdb70ce0dcae server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:11.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.141+0000 7f8662ffd640 1 -- 172.21.15.26:0/1236132318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86580ac640 con 0x7f8664159d20 2023-12-19T16:57:11.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.141+0000 7f8662ffd640 1 -- 172.21.15.26:0/1236132318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86580ac7a0 con 0x7f8664159d20 2023-12-19T16:57:11.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.141+0000 7f8662ffd640 1 -- 172.21.15.26:0/1236132318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86580acb30 con 0x7f8664159d20 2023-12-19T16:57:11.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.141+0000 7f86697a2640 1 -- 172.21.15.26:0/1236132318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8664159d20 msgr2=0x7f86640ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:11.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.141+0000 7f86697a2640 1 --2- 172.21.15.26:0/1236132318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8664159d20 0x7f86640ccb90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f86580097c0 tx=0x7f865801c3c0 comp rx=0 tx=0).stop 2023-12-19T16:57:11.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.141+0000 7f86697a2640 1 -- 172.21.15.26:0/1236132318 shutdown_connections 2023-12-19T16:57:11.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86697a2640 1 --2- 172.21.15.26:0/1236132318 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86640cdee0 0x7f86640b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:11.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86697a2640 1 --2- 172.21.15.26:0/1236132318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8664159d20 0x7f86640ccb90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:11.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86697a2640 1 -- 172.21.15.26:0/1236132318 >> 172.21.15.26:0/1236132318 conn(0x7f866406b0f0 msgr2=0x7f866406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:11.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86697a2640 1 -- 172.21.15.26:0/1236132318 shutdown_connections 2023-12-19T16:57:11.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86697a2640 1 -- 172.21.15.26:0/1236132318 wait complete. 2023-12-19T16:57:11.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86697a2640 1 Processor -- start 2023-12-19T16:57:11.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86697a2640 1 -- start start 2023-12-19T16:57:11.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86697a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86640cc0a0 0x7f866414e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:11.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86697a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86640cdee0 0x7f86641541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:11.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86697a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8664159d20 0x7f866415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:11.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86697a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86641524b0 con 0x7f86640cc0a0 2023-12-19T16:57:11.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86697a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86641518a0 con 0x7f8664159d20 2023-12-19T16:57:11.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86697a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8664151b80 con 0x7f86640cdee0 2023-12-19T16:57:11.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f8663fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86640cc0a0 0x7f866414e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:11.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f8663fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86640cc0a0 0x7f866414e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55524/0 (socket says 172.21.15.26:55524) 2023-12-19T16:57:11.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f8663fff640 1 -- 172.21.15.26:0/180396019 learned_addr learned my addr 172.21.15.26:0/180396019 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:11.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f8668fa1640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8664159d20 0x7f866415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:11.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f86637fe640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86640cdee0 0x7f86641541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:11.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f8663fff640 1 -- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86640cdee0 msgr2=0x7f86641541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:11.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f8663fff640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86640cdee0 0x7f86641541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:11.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f8663fff640 1 -- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8664159d20 msgr2=0x7f866415a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:11.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f8663fff640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8664159d20 0x7f866415a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:11.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.145+0000 7f8663fff640 1 -- 172.21.15.26:0/180396019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8658009420 con 0x7f86640cc0a0 2023-12-19T16:57:11.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.149+0000 7f8663fff640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86640cc0a0 0x7f866414e900 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f86540011f0 tx=0x7f865400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.149+0000 7f8660ff9640 1 -- 172.21.15.26:0/180396019 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540074e0 con 0x7f86640cc0a0 2023-12-19T16:57:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.149+0000 7f8660ff9640 1 -- 172.21.15.26:0/180396019 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8654007640 con 0x7f86640cc0a0 2023-12-19T16:57:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.149+0000 7f8660ff9640 1 -- 172.21.15.26:0/180396019 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654004a10 con 0x7f86640cc0a0 2023-12-19T16:57:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.149+0000 7f86697a2640 1 -- 172.21.15.26:0/180396019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8664151e60 con 0x7f86640cc0a0 2023-12-19T16:57:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.149+0000 7f86697a2640 1 -- 172.21.15.26:0/180396019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866414ee40 con 0x7f86640cc0a0 2023-12-19T16:57:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.149+0000 7f86697a2640 1 -- 172.21.15.26:0/180396019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866404e850 con 0x7f86640cc0a0 2023-12-19T16:57:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.149+0000 7f8660ff9640 1 -- 172.21.15.26:0/180396019 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f8654004b70 con 0x7f86640cc0a0 2023-12-19T16:57:11.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.149+0000 7f8660ff9640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8650065af0 0x7f8650067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:11.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.149+0000 7f8660ff9640 1 -- 172.21.15.26:0/180396019 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7215+0+0 (secure 0 0 0) 0x7f8654082a60 con 0x7f86640cc0a0 2023-12-19T16:57:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.149+0000 7f86637fe640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8650065af0 0x7f8650067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.153+0000 7f86637fe640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8650065af0 0x7f8650067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86580ae000 tx=0x7f86580a7000 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:11.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.153+0000 7f8660ff9640 1 -- 172.21.15.26:0/180396019 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f865404fe90 con 0x7f86640cc0a0 2023-12-19T16:57:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.345+0000 7f86697a2640 1 -- 172.21.15.26:0/180396019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f86640cd3c0 con 0x7f86640cc0a0 2023-12-19T16:57:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.969+0000 7f8660ff9640 1 -- 172.21.15.26:0/180396019 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8654018070 con 0x7f86640cc0a0 2023-12-19T16:57:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.969+0000 7f86697a2640 1 -- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8650065af0 msgr2=0x7f8650067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.969+0000 7f86697a2640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8650065af0 0x7f8650067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f86580ae000 tx=0x7f86580a7000 comp rx=0 tx=0).stop 2023-12-19T16:57:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.969+0000 7f86697a2640 1 -- 172.21.15.26:0/180396019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86640cc0a0 msgr2=0x7f866414e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.969+0000 7f86697a2640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86640cc0a0 0x7f866414e900 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f86540011f0 tx=0x7f865400c9b0 comp rx=0 tx=0).stop 2023-12-19T16:57:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.969+0000 7f86697a2640 1 -- 172.21.15.26:0/180396019 shutdown_connections 2023-12-19T16:57:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.969+0000 7f86697a2640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8664159d20 0x7f866415a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.969+0000 7f86697a2640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8650065af0 0x7f8650067fb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.969+0000 7f86697a2640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86640cdee0 0x7f86641541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.969+0000 7f86697a2640 1 --2- 172.21.15.26:0/180396019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86640cc0a0 0x7f866414e900 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.969+0000 7f86697a2640 1 -- 172.21.15.26:0/180396019 >> 172.21.15.26:0/180396019 conn(0x7f866406b0f0 msgr2=0x7f866415df60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.969+0000 7f86697a2640 1 -- 172.21.15.26:0/180396019 shutdown_connections 2023-12-19T16:57:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:11.969+0000 7f86697a2640 1 -- 172.21.15.26:0/180396019 wait complete. 2023-12-19T16:57:11.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T16:57:12.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.181+0000 7f4333bbe640 1 Processor -- start 2023-12-19T16:57:12.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.181+0000 7f4333bbe640 1 -- start start 2023-12-19T16:57:12.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4333bbe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f432c159830 0x7f432c15d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4333bbe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f432c0c9310 0x7f432c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4333bbe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f432c0cb110 0x7f432c0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4333bbe640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432c06f7c0 con 0x7f432c0c9310 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4333bbe640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432c0ca2b0 con 0x7f432c0cb110 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4333bbe640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432c15e3d0 con 0x7f432c159830 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4331159640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f432c0c9310 0x7f432c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4331159640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f432c0c9310 0x7f432c0cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55540/0 (socket says 172.21.15.26:55540) 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4331159640 1 -- 172.21.15.26:0/3990512111 learned_addr learned my addr 172.21.15.26:0/3990512111 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f433195a640 1 --2- 172.21.15.26:0/3990512111 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f432c159830 0x7f432c15d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f433215b640 1 --2- 172.21.15.26:0/3990512111 >> v2:172.21.15.160:3300/0 conn(0x7f432c0cb110 0x7f432c0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4331159640 1 -- 172.21.15.26:0/3990512111 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f432c159830 msgr2=0x7f432c15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4331159640 1 --2- 172.21.15.26:0/3990512111 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f432c159830 0x7f432c15d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4331159640 1 -- 172.21.15.26:0/3990512111 >> v2:172.21.15.160:3300/0 conn(0x7f432c0cb110 msgr2=0x7f432c0cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:12.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4331159640 1 --2- 172.21.15.26:0/3990512111 >> v2:172.21.15.160:3300/0 conn(0x7f432c0cb110 0x7f432c0cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f4331159640 1 -- 172.21.15.26:0/3990512111 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c0cba80 con 0x7f432c0c9310 2023-12-19T16:57:12.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.185+0000 7f433215b640 1 --2- 172.21.15.26:0/3990512111 >> v2:172.21.15.160:3300/0 conn(0x7f432c0cb110 0x7f432c0cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:12.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4331159640 1 --2- 172.21.15.26:0/3990512111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f432c0c9310 0x7f432c0cc430 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f432c0c21d0 tx=0x7f431c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a347944e53977d38 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4330958640 1 -- 172.21.15.26:0/3990512111 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0ab600 con 0x7f432c0c9310 2023-12-19T16:57:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4330958640 1 -- 172.21.15.26:0/3990512111 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c0ab760 con 0x7f432c0c9310 2023-12-19T16:57:12.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4330958640 1 -- 172.21.15.26:0/3990512111 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0abaf0 con 0x7f432c0c9310 2023-12-19T16:57:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4333bbe640 1 -- 172.21.15.26:0/3990512111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f432c0c9310 msgr2=0x7f432c0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4333bbe640 1 --2- 172.21.15.26:0/3990512111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f432c0c9310 0x7f432c0cc430 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f432c0c21d0 tx=0x7f431c01c440 comp rx=0 tx=0).stop 2023-12-19T16:57:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4333bbe640 1 -- 172.21.15.26:0/3990512111 shutdown_connections 2023-12-19T16:57:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4333bbe640 1 --2- 172.21.15.26:0/3990512111 >> v2:172.21.15.160:3300/0 conn(0x7f432c0cb110 0x7f432c0cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4333bbe640 1 --2- 172.21.15.26:0/3990512111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f432c0c9310 0x7f432c0cc430 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4333bbe640 1 --2- 172.21.15.26:0/3990512111 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f432c159830 0x7f432c15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4333bbe640 1 -- 172.21.15.26:0/3990512111 >> 172.21.15.26:0/3990512111 conn(0x7f432c0b0270 msgr2=0x7f432c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4333bbe640 1 -- 172.21.15.26:0/3990512111 shutdown_connections 2023-12-19T16:57:12.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4333bbe640 1 -- 172.21.15.26:0/3990512111 wait complete. 2023-12-19T16:57:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4333bbe640 1 Processor -- start 2023-12-19T16:57:12.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.189+0000 7f4333bbe640 1 -- start start 2023-12-19T16:57:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f4333bbe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f432c0c9310 0x7f432c0b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:12.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f4333bbe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f432c0cb110 0x7f432c0c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f4333bbe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f432c159830 0x7f432c1f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f4333bbe640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0ccdc0 con 0x7f432c0c9310 2023-12-19T16:57:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f4333bbe640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0c50d0 con 0x7f432c0cb110 2023-12-19T16:57:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f4333bbe640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0c4830 con 0x7f432c159830 2023-12-19T16:57:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f433195a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f432c0c9310 0x7f432c0b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f433195a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f432c0c9310 0x7f432c0b3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55552/0 (socket says 172.21.15.26:55552) 2023-12-19T16:57:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f433195a640 1 -- 172.21.15.26:0/141948554 learned_addr learned my addr 172.21.15.26:0/141948554 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f433215b640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f432c159830 0x7f432c1f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:12.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f4331159640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f432c0cb110 0x7f432c0c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:12.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f433195a640 1 -- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f432c159830 msgr2=0x7f432c1f4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:12.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f433195a640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f432c159830 0x7f432c1f4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f433195a640 1 -- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f432c0cb110 msgr2=0x7f432c0c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:12.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f433195a640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f432c0cb110 0x7f432c0c4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f433195a640 1 -- 172.21.15.26:0/141948554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c0094d0 con 0x7f432c0c9310 2023-12-19T16:57:12.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f433215b640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f432c159830 0x7f432c1f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:12.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f4331159640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f432c0cb110 0x7f432c0c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:12.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f433195a640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f432c0c9310 0x7f432c0b3e10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f432800d5f0 tx=0x7f432800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:12.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f43227fc640 1 -- 172.21.15.26:0/141948554 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43280125f0 con 0x7f432c0c9310 2023-12-19T16:57:12.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f43227fc640 1 -- 172.21.15.26:0/141948554 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4328012750 con 0x7f432c0c9310 2023-12-19T16:57:12.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f4333bbe640 1 -- 172.21.15.26:0/141948554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432c06fa80 con 0x7f432c0c9310 2023-12-19T16:57:12.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f43227fc640 1 -- 172.21.15.26:0/141948554 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328012ae0 con 0x7f432c0c9310 2023-12-19T16:57:12.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f4333bbe640 1 -- 172.21.15.26:0/141948554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432c0c4df0 con 0x7f432c0c9310 2023-12-19T16:57:12.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f4333bbe640 1 -- 172.21.15.26:0/141948554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432c0cf640 con 0x7f432c0c9310 2023-12-19T16:57:12.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.193+0000 7f43227fc640 1 -- 172.21.15.26:0/141948554 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f4328012c40 con 0x7f432c0c9310 2023-12-19T16:57:12.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.197+0000 7f43227fc640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4310065b20 0x7f4310067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:12.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.197+0000 7f43227fc640 1 -- 172.21.15.26:0/141948554 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7215+0+0 (secure 0 0 0) 0x7f4328083c30 con 0x7f432c0c9310 2023-12-19T16:57:12.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.197+0000 7f4331159640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4310065b20 0x7f4310067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:12.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.197+0000 7f4331159640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4310065b20 0x7f4310067fe0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f431c0af000 tx=0x7f431c0a7000 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:12.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.197+0000 7f43227fc640 1 -- 172.21.15.26:0/141948554 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4328050fb0 con 0x7f432c0c9310 2023-12-19T16:57:12.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.357+0000 7f4333bbe640 1 -- 172.21.15.26:0/141948554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f432c074270 con 0x7f432c0c9310 2023-12-19T16:57:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.521+0000 7f43227fc640 1 -- 172.21.15.26:0/141948554 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v69) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4328007380 con 0x7f432c0c9310 2023-12-19T16:57:12.525 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T16:57:12.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.521+0000 7f4333bbe640 1 -- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4310065b20 msgr2=0x7f4310067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:12.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.521+0000 7f4333bbe640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4310065b20 0x7f4310067fe0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f431c0af000 tx=0x7f431c0a7000 comp rx=0 tx=0).stop 2023-12-19T16:57:12.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.521+0000 7f4333bbe640 1 -- 172.21.15.26:0/141948554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f432c0c9310 msgr2=0x7f432c0b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:12.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.521+0000 7f4333bbe640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f432c0c9310 0x7f432c0b3e10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f432800d5f0 tx=0x7f432800dab0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.521+0000 7f4333bbe640 1 -- 172.21.15.26:0/141948554 shutdown_connections 2023-12-19T16:57:12.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.521+0000 7f4333bbe640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4310065b20 0x7f4310067fe0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.521+0000 7f4333bbe640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f432c159830 0x7f432c1f4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.521+0000 7f4333bbe640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f432c0cb110 0x7f432c0c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.521+0000 7f4333bbe640 1 --2- 172.21.15.26:0/141948554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f432c0c9310 0x7f432c0b3e10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.521+0000 7f4333bbe640 1 -- 172.21.15.26:0/141948554 >> 172.21.15.26:0/141948554 conn(0x7f432c0b0270 msgr2=0x7f432c0b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:12.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.521+0000 7f4333bbe640 1 -- 172.21.15.26:0/141948554 shutdown_connections 2023-12-19T16:57:12.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.521+0000 7f4333bbe640 1 -- 172.21.15.26:0/141948554 wait complete. 2023-12-19T16:57:12.550 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T16:57:12.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.701+0000 7f58423e9640 1 Processor -- start 2023-12-19T16:57:12.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.701+0000 7f58423e9640 1 -- start start 2023-12-19T16:57:12.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.701+0000 7f58423e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f583c0c9310 0x7f583c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:12.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.701+0000 7f58423e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f583c0cb110 0x7f583c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:12.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f58423e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583c0cbaa0 con 0x7f583c159830 2023-12-19T16:57:12.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f58423e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583c06f7c0 con 0x7f583c0c9310 2023-12-19T16:57:12.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f58423e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583c0ca2b0 con 0x7f583c0cb110 2023-12-19T16:57:12.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f583b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f583c0c9310 0x7f583c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:12.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f583b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f583c0c9310 0x7f583c0cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55566/0 (socket says 172.21.15.26:55566) 2023-12-19T16:57:12.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f583b7fe640 1 -- 172.21.15.26:0/901660765 learned_addr learned my addr 172.21.15.26:0/901660765 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:12.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f5840986640 1 --2- 172.21.15.26:0/901660765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f583c0cb110 0x7f583c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:12.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f583b7fe640 1 -- 172.21.15.26:0/901660765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f583c0cb110 msgr2=0x7f583c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:12.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f583b7fe640 1 --2- 172.21.15.26:0/901660765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f583c0cb110 0x7f583c0cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f583b7fe640 1 -- 172.21.15.26:0/901660765 >> v1:172.21.15.26:6789/0 conn(0x7f583c159830 legacy=0x7f583c15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:12.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f583b7fe640 1 -- 172.21.15.26:0/901660765 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c15e3d0 con 0x7f583c0c9310 2023-12-19T16:57:12.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f583b7fe640 1 --2- 172.21.15.26:0/901660765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c0c9310 0x7f583c0cc430 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5830009870 tx=0x7f583001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8612df36e9affff8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:12.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f583affd640 1 -- 172.21.15.26:0/901660765 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58300ac600 con 0x7f583c0c9310 2023-12-19T16:57:12.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f583affd640 1 -- 172.21.15.26:0/901660765 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58300a8050 con 0x7f583c0c9310 2023-12-19T16:57:12.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.705+0000 7f583affd640 1 -- 172.21.15.26:0/901660765 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58300ac980 con 0x7f583c0c9310 2023-12-19T16:57:12.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 -- 172.21.15.26:0/901660765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c0c9310 msgr2=0x7f583c0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:12.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 --2- 172.21.15.26:0/901660765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c0c9310 0x7f583c0cc430 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f5830009870 tx=0x7f583001c4b0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 -- 172.21.15.26:0/901660765 shutdown_connections 2023-12-19T16:57:12.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 --2- 172.21.15.26:0/901660765 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f583c0cb110 0x7f583c0cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 --2- 172.21.15.26:0/901660765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c0c9310 0x7f583c0cc430 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 -- 172.21.15.26:0/901660765 >> 172.21.15.26:0/901660765 conn(0x7f583c0b0270 msgr2=0x7f583c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:12.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 -- 172.21.15.26:0/901660765 shutdown_connections 2023-12-19T16:57:12.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 -- 172.21.15.26:0/901660765 wait complete. 2023-12-19T16:57:12.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 Processor -- start 2023-12-19T16:57:12.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 -- start start 2023-12-19T16:57:12.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f583c0c9310 0x7f583c14be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:12.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c0cb110 0x7f583c1662d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:12.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f583c159830 0x7f583c166f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:12.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c165880 con 0x7f583c0cb110 2023-12-19T16:57:12.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c165e40 con 0x7f583c159830 2023-12-19T16:57:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f58423e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583c0c3900 con 0x7f583c0c9310 2023-12-19T16:57:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f583b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c0cb110 0x7f583c1662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f583b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c0cb110 0x7f583c1662d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55578/0 (socket says 172.21.15.26:55578) 2023-12-19T16:57:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.709+0000 7f583b7fe640 1 -- 172.21.15.26:0/3252074734 learned_addr learned my addr 172.21.15.26:0/3252074734 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f5840986640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f583c159830 0x7f583c166f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f583bfff640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f583c0c9310 0x7f583c14be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f583b7fe640 1 -- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f583c0c9310 msgr2=0x7f583c14be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f583b7fe640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f583c0c9310 0x7f583c14be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f583b7fe640 1 -- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f583c159830 msgr2=0x7f583c166f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f583b7fe640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f583c159830 0x7f583c166f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f583b7fe640 1 -- 172.21.15.26:0/3252074734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58300094d0 con 0x7f583c0cb110 2023-12-19T16:57:12.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f583bfff640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f583c0c9310 0x7f583c14be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:12.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f583b7fe640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c0cb110 0x7f583c1662d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f583001a0e0 tx=0x7f583001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:12.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f5838ff9640 1 -- 172.21.15.26:0/3252074734 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58300b7650 con 0x7f583c0cb110 2023-12-19T16:57:12.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f5838ff9640 1 -- 172.21.15.26:0/3252074734 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58300b77b0 con 0x7f583c0cb110 2023-12-19T16:57:12.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f58423e9640 1 -- 172.21.15.26:0/3252074734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583c06fa80 con 0x7f583c0cb110 2023-12-19T16:57:12.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f5838ff9640 1 -- 172.21.15.26:0/3252074734 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58300b7b20 con 0x7f583c0cb110 2023-12-19T16:57:12.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f58423e9640 1 -- 172.21.15.26:0/3252074734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583c0c49f0 con 0x7f583c0cb110 2023-12-19T16:57:12.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f58423e9640 1 -- 172.21.15.26:0/3252074734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583c165ff0 con 0x7f583c0cb110 2023-12-19T16:57:12.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f5838ff9640 1 -- 172.21.15.26:0/3252074734 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f58300b3070 con 0x7f583c0cb110 2023-12-19T16:57:12.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.713+0000 7f5838ff9640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5828065af0 0x7f5828067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:12.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.717+0000 7f5838ff9640 1 -- 172.21.15.26:0/3252074734 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6777+0+0 (secure 0 0 0) 0x7f583011dd80 con 0x7f583c0cb110 2023-12-19T16:57:12.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.717+0000 7f583bfff640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5828065af0 0x7f5828067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:12.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.717+0000 7f583bfff640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5828065af0 0x7f5828067fb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f582c000f70 tx=0x7f582c009040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:12.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.721+0000 7f5838ff9640 1 -- 172.21.15.26:0/3252074734 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f583c165ff0 con 0x7f583c0cb110 2023-12-19T16:57:12.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.877+0000 7f58423e9640 1 -- 172.21.15.26:0/3252074734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f583c165fb0 con 0x7f583c0cb110 2023-12-19T16:57:12.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:12.977+0000 7f5838ff9640 1 -- 172.21.15.26:0/3252074734 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f58300eb4e0 con 0x7f583c0cb110 2023-12-19T16:57:13.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.633+0000 7f5838ff9640 1 -- 172.21.15.26:0/3252074734 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v70) v1 ==== 169+0+0 (secure 0 0 0) 0x7f58300c2ba0 con 0x7f583c0cb110 2023-12-19T16:57:13.640 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T16:57:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.637+0000 7f58423e9640 1 -- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5828065af0 msgr2=0x7f5828067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.637+0000 7f58423e9640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5828065af0 0x7f5828067fb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f582c000f70 tx=0x7f582c009040 comp rx=0 tx=0).stop 2023-12-19T16:57:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.637+0000 7f58423e9640 1 -- 172.21.15.26:0/3252074734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c0cb110 msgr2=0x7f583c1662d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.637+0000 7f58423e9640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c0cb110 0x7f583c1662d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f583001a0e0 tx=0x7f583001acf0 comp rx=0 tx=0).stop 2023-12-19T16:57:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.637+0000 7f58423e9640 1 -- 172.21.15.26:0/3252074734 shutdown_connections 2023-12-19T16:57:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.637+0000 7f58423e9640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f583c159830 0x7f583c166f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.637+0000 7f58423e9640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f583c0cb110 0x7f583c1662d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.637+0000 7f58423e9640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5828065af0 0x7f5828067fb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.637+0000 7f58423e9640 1 --2- 172.21.15.26:0/3252074734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f583c0c9310 0x7f583c14be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.637+0000 7f58423e9640 1 -- 172.21.15.26:0/3252074734 >> 172.21.15.26:0/3252074734 conn(0x7f583c0b0270 msgr2=0x7f583c0ccca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:13.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.637+0000 7f58423e9640 1 -- 172.21.15.26:0/3252074734 shutdown_connections 2023-12-19T16:57:13.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.637+0000 7f58423e9640 1 -- 172.21.15.26:0/3252074734 wait complete. 2023-12-19T16:57:13.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-19T16:57:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.817+0000 7fef4e69a640 1 Processor -- start 2023-12-19T16:57:13.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.817+0000 7fef4e69a640 1 -- start start 2023-12-19T16:57:13.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.817+0000 7fef4e69a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef480cd670 0x7fef480cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:13.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.817+0000 7fef4e69a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fef480b8600 0x7fef480d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:13.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.817+0000 7fef4e69a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef480b7e20 con 0x7fef4815ccf0 2023-12-19T16:57:13.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.817+0000 7fef4e69a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef480cdf90 con 0x7fef480cd670 2023-12-19T16:57:13.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.817+0000 7fef4e69a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef480b6d80 con 0x7fef480b8600 2023-12-19T16:57:13.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.817+0000 7fef4d698640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef480cd670 0x7fef480cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:13.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.817+0000 7fef4ce97640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fef4815ccf0 0x7fef480b6520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54148/0 (socket says 172.21.15.26:54148) 2023-12-19T16:57:13.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.817+0000 7fef4ce97640 1 -- 172.21.15.26:0/739630902 learned_addr learned my addr 172.21.15.26:0/739630902 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:13.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.817+0000 7fef4d698640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef480cd670 0x7fef480cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55580/0 (socket says 172.21.15.26:55580) 2023-12-19T16:57:13.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.817+0000 7fef4de99640 1 --2- 172.21.15.26:0/739630902 >> v2:172.21.15.160:3300/0 conn(0x7fef480b8600 0x7fef480d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:13.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.821+0000 7fef4d698640 1 -- 172.21.15.26:0/739630902 >> v2:172.21.15.160:3300/0 conn(0x7fef480b8600 msgr2=0x7fef480d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:13.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.821+0000 7fef4d698640 1 --2- 172.21.15.26:0/739630902 >> v2:172.21.15.160:3300/0 conn(0x7fef480b8600 0x7fef480d09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:13.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.821+0000 7fef4d698640 1 -- 172.21.15.26:0/739630902 >> v1:172.21.15.26:6789/0 conn(0x7fef4815ccf0 legacy=0x7fef480b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:13.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.821+0000 7fef4d698640 1 -- 172.21.15.26:0/739630902 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef480b7060 con 0x7fef480cd670 2023-12-19T16:57:13.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.821+0000 7fef4d698640 1 --2- 172.21.15.26:0/739630902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef480cd670 0x7fef480cda50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fef4808bc10 tx=0x7fef4401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6693a573348be28 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.821+0000 7fef3ffff640 1 -- 172.21.15.26:0/739630902 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef440a7070 con 0x7fef480cd670 2023-12-19T16:57:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.821+0000 7fef3ffff640 1 -- 172.21.15.26:0/739630902 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef440ab430 con 0x7fef480cd670 2023-12-19T16:57:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.821+0000 7fef3ffff640 1 -- 172.21.15.26:0/739630902 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef440ab7c0 con 0x7fef480cd670 2023-12-19T16:57:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.821+0000 7fef4e69a640 1 -- 172.21.15.26:0/739630902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef480cd670 msgr2=0x7fef480cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.821+0000 7fef4e69a640 1 --2- 172.21.15.26:0/739630902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef480cd670 0x7fef480cda50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fef4808bc10 tx=0x7fef4401c430 comp rx=0 tx=0).stop 2023-12-19T16:57:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 -- 172.21.15.26:0/739630902 shutdown_connections 2023-12-19T16:57:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 --2- 172.21.15.26:0/739630902 >> v2:172.21.15.160:3300/0 conn(0x7fef480b8600 0x7fef480d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 --2- 172.21.15.26:0/739630902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef480cd670 0x7fef480cda50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 -- 172.21.15.26:0/739630902 >> 172.21.15.26:0/739630902 conn(0x7fef4806a120 msgr2=0x7fef4806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 -- 172.21.15.26:0/739630902 shutdown_connections 2023-12-19T16:57:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 -- 172.21.15.26:0/739630902 wait complete. 2023-12-19T16:57:13.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 Processor -- start 2023-12-19T16:57:13.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 -- start start 2023-12-19T16:57:13.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fef480b8600 0x7fef4814d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:13.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef480cd670 0x7fef4814bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4815ccf0 0x7fef480cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4814d3f0 con 0x7fef4815ccf0 2023-12-19T16:57:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef480cc7e0 con 0x7fef480cd670 2023-12-19T16:57:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4e69a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4814dd50 con 0x7fef480b8600 2023-12-19T16:57:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4de99640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4815ccf0 0x7fef480cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4de99640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4815ccf0 0x7fef480cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55588/0 (socket says 172.21.15.26:55588) 2023-12-19T16:57:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4de99640 1 -- 172.21.15.26:0/977233540 learned_addr learned my addr 172.21.15.26:0/977233540 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4de99640 1 -- 172.21.15.26:0/977233540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fef480b8600 msgr2=0x7fef4814d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4de99640 1 --2- 172.21.15.26:0/977233540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fef480b8600 0x7fef4814d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4de99640 1 -- 172.21.15.26:0/977233540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef480cd670 msgr2=0x7fef4814bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4de99640 1 --2- 172.21.15.26:0/977233540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef480cd670 0x7fef4814bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4de99640 1 -- 172.21.15.26:0/977233540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef44009420 con 0x7fef4815ccf0 2023-12-19T16:57:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef4de99640 1 --2- 172.21.15.26:0/977233540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4815ccf0 0x7fef480cb4d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fef4000efc0 tx=0x7fef4000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:13.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.825+0000 7fef3dffb640 1 -- 172.21.15.26:0/977233540 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40014480 con 0x7fef4815ccf0 2023-12-19T16:57:13.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.829+0000 7fef3dffb640 1 -- 172.21.15.26:0/977233540 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4000f040 con 0x7fef4815ccf0 2023-12-19T16:57:13.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.829+0000 7fef4e69a640 1 -- 172.21.15.26:0/977233540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4815c540 con 0x7fef4815ccf0 2023-12-19T16:57:13.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.829+0000 7fef3dffb640 1 -- 172.21.15.26:0/977233540 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40014820 con 0x7fef4815ccf0 2023-12-19T16:57:13.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.829+0000 7fef4e69a640 1 -- 172.21.15.26:0/977233540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4814d010 con 0x7fef4815ccf0 2023-12-19T16:57:13.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.829+0000 7fef4e69a640 1 -- 172.21.15.26:0/977233540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4804e850 con 0x7fef4815ccf0 2023-12-19T16:57:13.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.829+0000 7fef3dffb640 1 -- 172.21.15.26:0/977233540 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fef40014980 con 0x7fef4815ccf0 2023-12-19T16:57:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.829+0000 7fef3dffb640 1 --2- 172.21.15.26:0/977233540 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fef2c065b80 0x7fef2c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.829+0000 7fef3dffb640 1 -- 172.21.15.26:0/977233540 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fef40084670 con 0x7fef4815ccf0 2023-12-19T16:57:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.829+0000 7fef4d698640 1 --2- 172.21.15.26:0/977233540 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fef2c065b80 0x7fef2c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:13.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.833+0000 7fef4d698640 1 --2- 172.21.15.26:0/977233540 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fef2c065b80 0x7fef2c068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fef440af000 tx=0x7fef44008040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.833+0000 7fef3dffb640 1 -- 172.21.15.26:0/977233540 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fef40051b30 con 0x7fef4815ccf0 2023-12-19T16:57:13.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.993+0000 7fef4e69a640 1 -- 172.21.15.26:0/977233540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fef4814d530 con 0x7fef4815ccf0 2023-12-19T16:57:14.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.997+0000 7fef3dffb640 1 -- 172.21.15.26:0/977233540 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v19) v1 ==== 120+0+0 (secure 0 0 0) 0x7fef4814d530 con 0x7fef4815ccf0 2023-12-19T16:57:14.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.997+0000 7fef4e69a640 1 -- 172.21.15.26:0/977233540 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fef2c065b80 msgr2=0x7fef2c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.997+0000 7fef4e69a640 1 --2- 172.21.15.26:0/977233540 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fef2c065b80 0x7fef2c068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fef440af000 tx=0x7fef44008040 comp rx=0 tx=0).stop 2023-12-19T16:57:14.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.997+0000 7fef4e69a640 1 -- 172.21.15.26:0/977233540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4815ccf0 msgr2=0x7fef480cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.997+0000 7fef4e69a640 1 --2- 172.21.15.26:0/977233540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4815ccf0 0x7fef480cb4d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fef4000efc0 tx=0x7fef4000c490 comp rx=0 tx=0).stop 2023-12-19T16:57:14.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.997+0000 7fef4e69a640 1 -- 172.21.15.26:0/977233540 shutdown_connections 2023-12-19T16:57:14.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.997+0000 7fef4e69a640 1 --2- 172.21.15.26:0/977233540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef4815ccf0 0x7fef480cb4d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.997+0000 7fef4e69a640 1 --2- 172.21.15.26:0/977233540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef480cd670 0x7fef4814bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.997+0000 7fef4e69a640 1 --2- 172.21.15.26:0/977233540 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fef2c065b80 0x7fef2c068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.997+0000 7fef4e69a640 1 --2- 172.21.15.26:0/977233540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fef480b8600 0x7fef4814d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.997+0000 7fef4e69a640 1 -- 172.21.15.26:0/977233540 >> 172.21.15.26:0/977233540 conn(0x7fef4806a120 msgr2=0x7fef481509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.997+0000 7fef4e69a640 1 -- 172.21.15.26:0/977233540 shutdown_connections 2023-12-19T16:57:14.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:13.997+0000 7fef4e69a640 1 -- 172.21.15.26:0/977233540 wait complete. 2023-12-19T16:57:14.024 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T16:57:14.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.173+0000 7f4a188b6640 1 Processor -- start 2023-12-19T16:57:14.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.173+0000 7f4a188b6640 1 -- start start 2023-12-19T16:57:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.177+0000 7f4a188b6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4a141599e0 0x7f4a1406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.177+0000 7f4a188b6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4a140cb150 0x7f4a140cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.177+0000 7f4a188b6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1415e240 con 0x7f4a1415d530 2023-12-19T16:57:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.177+0000 7f4a188b6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1415e050 con 0x7f4a141599e0 2023-12-19T16:57:14.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.177+0000 7f4a188b6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a140cbb20 con 0x7f4a140cb150 2023-12-19T16:57:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.177+0000 7f4a1259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4a141599e0 0x7f4a1406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.177+0000 7f4a1259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4a141599e0 0x7f4a1406f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55600/0 (socket says 172.21.15.26:55600) 2023-12-19T16:57:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.177+0000 7f4a1259c640 1 -- 172.21.15.26:0/2800776639 learned_addr learned my addr 172.21.15.26:0/2800776639 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.177+0000 7f4a0bfff640 1 --2- 172.21.15.26:0/2800776639 >> v2:172.21.15.160:3300/0 conn(0x7f4a140cb150 0x7f4a140cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.177+0000 7f4a1259c640 1 -- 172.21.15.26:0/2800776639 >> v2:172.21.15.160:3300/0 conn(0x7f4a140cb150 msgr2=0x7f4a140cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.177+0000 7f4a1259c640 1 --2- 172.21.15.26:0/2800776639 >> v2:172.21.15.160:3300/0 conn(0x7f4a140cb150 0x7f4a140cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.177+0000 7f4a1259c640 1 -- 172.21.15.26:0/2800776639 >> v1:172.21.15.160:6789/0 conn(0x7f4a1415d530 legacy=0x7f4a1415d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.177+0000 7f4a1259c640 1 -- 172.21.15.26:0/2800776639 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a140c9520 con 0x7f4a141599e0 2023-12-19T16:57:14.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a1259c640 1 --2- 172.21.15.26:0/2800776639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a141599e0 0x7f4a1406f6d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4a140cc220 tx=0x7f4a0401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f27cbd23eceb87e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a11d9b640 1 -- 172.21.15.26:0/2800776639 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a040aa660 con 0x7f4a141599e0 2023-12-19T16:57:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a11d9b640 1 -- 172.21.15.26:0/2800776639 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a040aa7c0 con 0x7f4a141599e0 2023-12-19T16:57:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a11d9b640 1 -- 172.21.15.26:0/2800776639 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a040aab50 con 0x7f4a141599e0 2023-12-19T16:57:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a188b6640 1 -- 172.21.15.26:0/2800776639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a141599e0 msgr2=0x7f4a1406f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a188b6640 1 --2- 172.21.15.26:0/2800776639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a141599e0 0x7f4a1406f6d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4a140cc220 tx=0x7f4a0401c500 comp rx=0 tx=0).stop 2023-12-19T16:57:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a188b6640 1 -- 172.21.15.26:0/2800776639 shutdown_connections 2023-12-19T16:57:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a188b6640 1 --2- 172.21.15.26:0/2800776639 >> v2:172.21.15.160:3300/0 conn(0x7f4a140cb150 0x7f4a140cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a188b6640 1 --2- 172.21.15.26:0/2800776639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a141599e0 0x7f4a1406f6d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a188b6640 1 -- 172.21.15.26:0/2800776639 >> 172.21.15.26:0/2800776639 conn(0x7f4a14066160 msgr2=0x7f4a140b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a188b6640 1 -- 172.21.15.26:0/2800776639 shutdown_connections 2023-12-19T16:57:14.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a188b6640 1 -- 172.21.15.26:0/2800776639 wait complete. 2023-12-19T16:57:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a188b6640 1 Processor -- start 2023-12-19T16:57:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.181+0000 7f4a188b6640 1 -- start start 2023-12-19T16:57:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a188b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a140cb150 0x7f4a1414e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a188b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a141599e0 0x7f4a1414dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a188b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1415d530 0x7f4a1415a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a188b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1415c5c0 con 0x7f4a1415d530 2023-12-19T16:57:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a188b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a14159f40 con 0x7f4a141599e0 2023-12-19T16:57:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a188b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1415bd20 con 0x7f4a140cb150 2023-12-19T16:57:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1415d530 0x7f4a1415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1415d530 0x7f4a1415a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55602/0 (socket says 172.21.15.26:55602) 2023-12-19T16:57:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a12d9d640 1 -- 172.21.15.26:0/422424264 learned_addr learned my addr 172.21.15.26:0/422424264 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a0bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a141599e0 0x7f4a1414dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a1259c640 1 --2- 172.21.15.26:0/422424264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a140cb150 0x7f4a1414e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a12d9d640 1 -- 172.21.15.26:0/422424264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a140cb150 msgr2=0x7f4a1414e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a12d9d640 1 --2- 172.21.15.26:0/422424264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a140cb150 0x7f4a1414e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a12d9d640 1 -- 172.21.15.26:0/422424264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a141599e0 msgr2=0x7f4a1414dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a12d9d640 1 --2- 172.21.15.26:0/422424264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a141599e0 0x7f4a1414dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a12d9d640 1 -- 172.21.15.26:0/422424264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a040094d0 con 0x7f4a1415d530 2023-12-19T16:57:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a12d9d640 1 --2- 172.21.15.26:0/422424264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1415d530 0x7f4a1415a670 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00c730 tx=0x7f4a0c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a0affd640 1 -- 172.21.15.26:0/422424264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c003b80 con 0x7f4a1415d530 2023-12-19T16:57:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a0affd640 1 -- 172.21.15.26:0/422424264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c003ce0 con 0x7f4a1415d530 2023-12-19T16:57:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a188b6640 1 -- 172.21.15.26:0/422424264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1415c000 con 0x7f4a1415d530 2023-12-19T16:57:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a0affd640 1 -- 172.21.15.26:0/422424264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c00aa30 con 0x7f4a1415d530 2023-12-19T16:57:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a188b6640 1 -- 172.21.15.26:0/422424264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1415c390 con 0x7f4a1415d530 2023-12-19T16:57:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a188b6640 1 -- 172.21.15.26:0/422424264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a14074270 con 0x7f4a1415d530 2023-12-19T16:57:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.185+0000 7f4a0affd640 1 -- 172.21.15.26:0/422424264 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4a0c00ab90 con 0x7f4a1415d530 2023-12-19T16:57:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.189+0000 7f4a0affd640 1 --2- 172.21.15.26:0/422424264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49f4065b80 0x7f49f4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.189+0000 7f4a0affd640 1 -- 172.21.15.26:0/422424264 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f4a0c0831b0 con 0x7f4a1415d530 2023-12-19T16:57:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.189+0000 7f4a1259c640 1 --2- 172.21.15.26:0/422424264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49f4065b80 0x7f49f4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.189+0000 7f4a1259c640 1 --2- 172.21.15.26:0/422424264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49f4065b80 0x7f49f4068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4a040ae000 tx=0x7f4a040b5040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:14.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.193+0000 7f4a0affd640 1 -- 172.21.15.26:0/422424264 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4a0c0508c0 con 0x7f4a1415d530 2023-12-19T16:57:14.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.349+0000 7f4a188b6640 1 -- 172.21.15.26:0/422424264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4a140c2050 con 0x7f4a1415d530 2023-12-19T16:57:14.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.353+0000 7f4a0affd640 1 -- 172.21.15.26:0/422424264 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4a0c053ee0 con 0x7f4a1415d530 2023-12-19T16:57:14.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.357+0000 7f4a188b6640 1 -- 172.21.15.26:0/422424264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49f4065b80 msgr2=0x7f49f4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.357+0000 7f4a188b6640 1 --2- 172.21.15.26:0/422424264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49f4065b80 0x7f49f4068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4a040ae000 tx=0x7f4a040b5040 comp rx=0 tx=0).stop 2023-12-19T16:57:14.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.357+0000 7f4a188b6640 1 -- 172.21.15.26:0/422424264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1415d530 msgr2=0x7f4a1415a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.357+0000 7f4a188b6640 1 --2- 172.21.15.26:0/422424264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1415d530 0x7f4a1415a670 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00c730 tx=0x7f4a0c00cbf0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.357+0000 7f4a188b6640 1 -- 172.21.15.26:0/422424264 shutdown_connections 2023-12-19T16:57:14.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.357+0000 7f4a188b6640 1 --2- 172.21.15.26:0/422424264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a1415d530 0x7f4a1415a670 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.357+0000 7f4a188b6640 1 --2- 172.21.15.26:0/422424264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a141599e0 0x7f4a1414dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.357+0000 7f4a188b6640 1 --2- 172.21.15.26:0/422424264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49f4065b80 0x7f49f4068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.357+0000 7f4a188b6640 1 --2- 172.21.15.26:0/422424264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a140cb150 0x7f4a1414e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.357+0000 7f4a188b6640 1 -- 172.21.15.26:0/422424264 >> 172.21.15.26:0/422424264 conn(0x7f4a14066160 msgr2=0x7f4a1415d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:14.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.357+0000 7f4a188b6640 1 -- 172.21.15.26:0/422424264 shutdown_connections 2023-12-19T16:57:14.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.357+0000 7f4a188b6640 1 -- 172.21.15.26:0/422424264 wait complete. 2023-12-19T16:57:14.381 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-19T16:57:14.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.521+0000 7ff0c308c640 1 Processor -- start 2023-12-19T16:57:14.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.521+0000 7ff0c308c640 1 -- start start 2023-12-19T16:57:14.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.521+0000 7ff0c308c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0bc1599e0 0x7ff0bc06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0c308c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff0bc15dc40 0x7ff0bc0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0c308c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0bc0cbaf0 con 0x7ff0bc0cb150 2023-12-19T16:57:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0c308c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0bc15e050 con 0x7ff0bc15dc40 2023-12-19T16:57:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0c308c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc15e240 con 0x7ff0bc1599e0 2023-12-19T16:57:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0c1629640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff0bc15dc40 0x7ff0bc0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0c1629640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff0bc15dc40 0x7ff0bc0cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55614/0 (socket says 172.21.15.26:55614) 2023-12-19T16:57:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0c1629640 1 -- 172.21.15.26:0/1545067074 learned_addr learned my addr 172.21.15.26:0/1545067074 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0c0e28640 1 --2- 172.21.15.26:0/1545067074 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0bc1599e0 0x7ff0bc06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0c1629640 1 -- 172.21.15.26:0/1545067074 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0bc1599e0 msgr2=0x7ff0bc06f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:14.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0c1629640 1 --2- 172.21.15.26:0/1545067074 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0bc1599e0 0x7ff0bc06f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0c1629640 1 -- 172.21.15.26:0/1545067074 >> v1:172.21.15.160:6789/0 conn(0x7ff0bc0cb150 legacy=0x7ff0bc15d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0c1629640 1 -- 172.21.15.26:0/1545067074 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc0c9520 con 0x7ff0bc15dc40 2023-12-19T16:57:14.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0c1629640 1 --2- 172.21.15.26:0/1545067074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0bc15dc40 0x7ff0bc0cb5b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400d150 tx=0x7ff0b401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bb8e95442ec11e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:14.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0bb7fe640 1 -- 172.21.15.26:0/1545067074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40a7ce0 con 0x7ff0bc15dc40 2023-12-19T16:57:14.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0bb7fe640 1 -- 172.21.15.26:0/1545067074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b40a7e40 con 0x7ff0bc15dc40 2023-12-19T16:57:14.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.525+0000 7ff0bb7fe640 1 -- 172.21.15.26:0/1545067074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b400b2f0 con 0x7ff0bc15dc40 2023-12-19T16:57:14.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 -- 172.21.15.26:0/1545067074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0bc15dc40 msgr2=0x7ff0bc0cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 --2- 172.21.15.26:0/1545067074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0bc15dc40 0x7ff0bc0cb5b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400d150 tx=0x7ff0b401ec10 comp rx=0 tx=0).stop 2023-12-19T16:57:14.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 -- 172.21.15.26:0/1545067074 shutdown_connections 2023-12-19T16:57:14.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 --2- 172.21.15.26:0/1545067074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0bc15dc40 0x7ff0bc0cb5b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 --2- 172.21.15.26:0/1545067074 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0bc1599e0 0x7ff0bc06f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 -- 172.21.15.26:0/1545067074 >> 172.21.15.26:0/1545067074 conn(0x7ff0bc066160 msgr2=0x7ff0bc0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:14.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 -- 172.21.15.26:0/1545067074 shutdown_connections 2023-12-19T16:57:14.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 -- 172.21.15.26:0/1545067074 wait complete. 2023-12-19T16:57:14.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 Processor -- start 2023-12-19T16:57:14.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 -- start start 2023-12-19T16:57:14.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0bc0cb150 0x7ff0bc0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0bc1599e0 0x7ff0bc1543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0bc15dc40 0x7ff0bc1f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc149f70 con 0x7ff0bc1599e0 2023-12-19T16:57:14.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc0afc00 con 0x7ff0bc15dc40 2023-12-19T16:57:14.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c308c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc0ae880 con 0x7ff0bc0cb150 2023-12-19T16:57:14.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0bbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0bc1599e0 0x7ff0bc1543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c0e28640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0bc0cb150 0x7ff0bc0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0bbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0bc1599e0 0x7ff0bc1543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55626/0 (socket says 172.21.15.26:55626) 2023-12-19T16:57:14.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0bbfff640 1 -- 172.21.15.26:0/1469378615 learned_addr learned my addr 172.21.15.26:0/1469378615 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:14.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.529+0000 7ff0c1629640 1 --2- 172.21.15.26:0/1469378615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0bc15dc40 0x7ff0bc1f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0bbfff640 1 -- 172.21.15.26:0/1469378615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0bc0cb150 msgr2=0x7ff0bc0ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0bbfff640 1 --2- 172.21.15.26:0/1469378615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0bc0cb150 0x7ff0bc0ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0bbfff640 1 -- 172.21.15.26:0/1469378615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0bc15dc40 msgr2=0x7ff0bc1f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0bbfff640 1 --2- 172.21.15.26:0/1469378615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0bc15dc40 0x7ff0bc1f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0bbfff640 1 -- 172.21.15.26:0/1469378615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b400bd40 con 0x7ff0bc1599e0 2023-12-19T16:57:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0bbfff640 1 --2- 172.21.15.26:0/1469378615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0bc1599e0 0x7ff0bc1543d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c500 tx=0x7ff0a800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0b97fa640 1 -- 172.21.15.26:0/1469378615 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8005220 con 0x7ff0bc1599e0 2023-12-19T16:57:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0b97fa640 1 -- 172.21.15.26:0/1469378615 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a8005380 con 0x7ff0bc1599e0 2023-12-19T16:57:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0b97fa640 1 -- 172.21.15.26:0/1469378615 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8007270 con 0x7ff0bc1599e0 2023-12-19T16:57:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0c308c640 1 -- 172.21.15.26:0/1469378615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0bc0aeb60 con 0x7ff0bc1599e0 2023-12-19T16:57:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0c308c640 1 -- 172.21.15.26:0/1469378615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0bc0af120 con 0x7ff0bc1599e0 2023-12-19T16:57:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0c308c640 1 -- 172.21.15.26:0/1469378615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0bc074270 con 0x7ff0bc1599e0 2023-12-19T16:57:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0b97fa640 1 -- 172.21.15.26:0/1469378615 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff0a80073d0 con 0x7ff0bc1599e0 2023-12-19T16:57:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.533+0000 7ff0b97fa640 1 --2- 172.21.15.26:0/1469378615 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0a0065b50 0x7ff0a0068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.537+0000 7ff0b97fa640 1 -- 172.21.15.26:0/1469378615 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6277+0+0 (secure 0 0 0) 0x7ff0a8082d80 con 0x7ff0bc1599e0 2023-12-19T16:57:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.537+0000 7ff0c0e28640 1 --2- 172.21.15.26:0/1469378615 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0a0065b50 0x7ff0a0068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.537+0000 7ff0c0e28640 1 --2- 172.21.15.26:0/1469378615 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0a0065b50 0x7ff0a0068010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0007a60 tx=0x7ff0b0008040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.541+0000 7ff0b97fa640 1 -- 172.21.15.26:0/1469378615 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff0a8050410 con 0x7ff0bc1599e0 2023-12-19T16:57:14.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.697+0000 7ff0c308c640 1 -- 172.21.15.26:0/1469378615 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7ff0bc0c9180 con 0x7ff0bc1599e0 2023-12-19T16:57:14.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.701+0000 7ff0b97fa640 1 -- 172.21.15.26:0/1469378615 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff0a8053a30 con 0x7ff0bc1599e0 2023-12-19T16:57:14.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.705+0000 7ff0c308c640 1 -- 172.21.15.26:0/1469378615 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0a0065b50 msgr2=0x7ff0a0068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.705+0000 7ff0c308c640 1 --2- 172.21.15.26:0/1469378615 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0a0065b50 0x7ff0a0068010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0007a60 tx=0x7ff0b0008040 comp rx=0 tx=0).stop 2023-12-19T16:57:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.705+0000 7ff0c308c640 1 -- 172.21.15.26:0/1469378615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0bc1599e0 msgr2=0x7ff0bc1543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.705+0000 7ff0c308c640 1 --2- 172.21.15.26:0/1469378615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0bc1599e0 0x7ff0bc1543d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff0a800c500 tx=0x7ff0a800c9c0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.705+0000 7ff0c308c640 1 -- 172.21.15.26:0/1469378615 shutdown_connections 2023-12-19T16:57:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.705+0000 7ff0c308c640 1 --2- 172.21.15.26:0/1469378615 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0bc15dc40 0x7ff0bc1f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.705+0000 7ff0c308c640 1 --2- 172.21.15.26:0/1469378615 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0bc1599e0 0x7ff0bc1543d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.705+0000 7ff0c308c640 1 --2- 172.21.15.26:0/1469378615 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0a0065b50 0x7ff0a0068010 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.705+0000 7ff0c308c640 1 --2- 172.21.15.26:0/1469378615 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0bc0cb150 0x7ff0bc0ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.705+0000 7ff0c308c640 1 -- 172.21.15.26:0/1469378615 >> 172.21.15.26:0/1469378615 conn(0x7ff0bc066160 msgr2=0x7ff0bc0ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.705+0000 7ff0c308c640 1 -- 172.21.15.26:0/1469378615 shutdown_connections 2023-12-19T16:57:14.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.705+0000 7ff0c308c640 1 -- 172.21.15.26:0/1469378615 wait complete. 2023-12-19T16:57:14.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-12-19T16:57:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.869+0000 7f15ef0c4640 1 Processor -- start 2023-12-19T16:57:14.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.869+0000 7f15ef0c4640 1 -- start start 2023-12-19T16:57:14.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.869+0000 7f15ef0c4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15e8155030 0x7f15e815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.869+0000 7f15ef0c4640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f15e80ce740 0x7f15e80ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.869+0000 7f15ef0c4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e80cf0b0 con 0x7f15e815f370 2023-12-19T16:57:14.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.869+0000 7f15ef0c4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e806c790 con 0x7f15e8155030 2023-12-19T16:57:14.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.869+0000 7f15ef0c4640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e806c970 con 0x7f15e80ce740 2023-12-19T16:57:14.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.869+0000 7f15ece60640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15e8155030 0x7f15e815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.869+0000 7f15ece60640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15e8155030 0x7f15e815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55642/0 (socket says 172.21.15.26:55642) 2023-12-19T16:57:14.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.869+0000 7f15ece60640 1 -- 172.21.15.26:0/2760111089 learned_addr learned my addr 172.21.15.26:0/2760111089 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:14.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.869+0000 7f15ed661640 1 --2- 172.21.15.26:0/2760111089 >> v2:172.21.15.160:3300/0 conn(0x7f15e80ce740 0x7f15e80ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.873+0000 7f15ece60640 1 -- 172.21.15.26:0/2760111089 >> v2:172.21.15.160:3300/0 conn(0x7f15e80ce740 msgr2=0x7f15e80ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:14.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.873+0000 7f15ece60640 1 --2- 172.21.15.26:0/2760111089 >> v2:172.21.15.160:3300/0 conn(0x7f15e80ce740 0x7f15e80ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.873+0000 7f15ece60640 1 -- 172.21.15.26:0/2760111089 >> v1:172.21.15.160:6789/0 conn(0x7f15e815f370 legacy=0x7f15e806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:14.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.873+0000 7f15ece60640 1 -- 172.21.15.26:0/2760111089 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e815b1f0 con 0x7f15e8155030 2023-12-19T16:57:14.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.873+0000 7f15ece60640 1 --2- 172.21.15.26:0/2760111089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e8155030 0x7f15e815bcd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f15e80cc390 tx=0x7f15dc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b970a49f3ed00003 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:14.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.873+0000 7f15e77fe640 1 -- 172.21.15.26:0/2760111089 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0aa5f0 con 0x7f15e8155030 2023-12-19T16:57:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.873+0000 7f15e77fe640 1 -- 172.21.15.26:0/2760111089 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15dc0aa750 con 0x7f15e8155030 2023-12-19T16:57:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.873+0000 7f15e77fe640 1 -- 172.21.15.26:0/2760111089 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0aaae0 con 0x7f15e8155030 2023-12-19T16:57:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.873+0000 7f15ef0c4640 1 -- 172.21.15.26:0/2760111089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e8155030 msgr2=0x7f15e815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.873+0000 7f15ef0c4640 1 --2- 172.21.15.26:0/2760111089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e8155030 0x7f15e815bcd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f15e80cc390 tx=0x7f15dc01c490 comp rx=0 tx=0).stop 2023-12-19T16:57:14.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 -- 172.21.15.26:0/2760111089 shutdown_connections 2023-12-19T16:57:14.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 --2- 172.21.15.26:0/2760111089 >> v2:172.21.15.160:3300/0 conn(0x7f15e80ce740 0x7f15e80ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 --2- 172.21.15.26:0/2760111089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e8155030 0x7f15e815bcd0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 -- 172.21.15.26:0/2760111089 >> 172.21.15.26:0/2760111089 conn(0x7f15e80658e0 msgr2=0x7f15e8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:14.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 -- 172.21.15.26:0/2760111089 shutdown_connections 2023-12-19T16:57:14.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 -- 172.21.15.26:0/2760111089 wait complete. 2023-12-19T16:57:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 Processor -- start 2023-12-19T16:57:14.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 -- start start 2023-12-19T16:57:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15e80ce740 0x7f15e813f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e8155030 0x7f15e8155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15e815f370 0x7f15e8151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e8150be0 con 0x7f15e8155030 2023-12-19T16:57:14.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80b09b0 con 0x7f15e815f370 2023-12-19T16:57:14.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ef0c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e8142980 con 0x7f15e80ce740 2023-12-19T16:57:14.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15e7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e8155030 0x7f15e8155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15e7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e8155030 0x7f15e8155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55644/0 (socket says 172.21.15.26:55644) 2023-12-19T16:57:14.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15e7fff640 1 -- 172.21.15.26:0/352825337 learned_addr learned my addr 172.21.15.26:0/352825337 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:14.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ed661640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15e815f370 0x7f15e8151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15ece60640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15e80ce740 0x7f15e813f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15e7fff640 1 -- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15e80ce740 msgr2=0x7f15e813f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15e7fff640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15e80ce740 0x7f15e813f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15e7fff640 1 -- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15e815f370 msgr2=0x7f15e8151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:14.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15e7fff640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15e815f370 0x7f15e8151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.877+0000 7f15e7fff640 1 -- 172.21.15.26:0/352825337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15dc0094d0 con 0x7f15e8155030 2023-12-19T16:57:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.881+0000 7f15e7fff640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e8155030 0x7f15e8155770 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f15d400b410 tx=0x7f15d400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.881+0000 7f15e57fa640 1 -- 172.21.15.26:0/352825337 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d4004020 con 0x7f15e8155030 2023-12-19T16:57:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.881+0000 7f15e57fa640 1 -- 172.21.15.26:0/352825337 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d4004180 con 0x7f15e8155030 2023-12-19T16:57:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.881+0000 7f15ef0c4640 1 -- 172.21.15.26:0/352825337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e80b03f0 con 0x7f15e8155030 2023-12-19T16:57:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.881+0000 7f15e57fa640 1 -- 172.21.15.26:0/352825337 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d4011630 con 0x7f15e8155030 2023-12-19T16:57:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.881+0000 7f15ef0c4640 1 -- 172.21.15.26:0/352825337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e8150800 con 0x7f15e8155030 2023-12-19T16:57:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.881+0000 7f15ef0c4640 1 -- 172.21.15.26:0/352825337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e8150d20 con 0x7f15e8155030 2023-12-19T16:57:14.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.881+0000 7f15e57fa640 1 -- 172.21.15.26:0/352825337 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f15d4011790 con 0x7f15e8155030 2023-12-19T16:57:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.881+0000 7f15e57fa640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f15cc065b80 0x7f15cc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.881+0000 7f15e57fa640 1 -- 172.21.15.26:0/352825337 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f15d40830a0 con 0x7f15e8155030 2023-12-19T16:57:14.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.885+0000 7f15ece60640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f15cc065b80 0x7f15cc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.885+0000 7f15ece60640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f15cc065b80 0x7f15cc068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0ae000 tx=0x7f15dc0b5040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:14.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:14.885+0000 7f15e57fa640 1 -- 172.21.15.26:0/352825337 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f15e8150d20 con 0x7f15e8155030 2023-12-19T16:57:15.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.105+0000 7f15ef0c4640 1 -- 172.21.15.26:0/352825337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f15e8150d20 con 0x7f15e8155030 2023-12-19T16:57:15.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.645+0000 7f15e57fa640 1 -- 172.21.15.26:0/352825337 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v536) v1 ==== 154+0+0 (secure 0 0 0) 0x7f15e8150d20 con 0x7f15e8155030 2023-12-19T16:57:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.649+0000 7f15ef0c4640 1 -- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f15cc065b80 msgr2=0x7f15cc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.649+0000 7f15ef0c4640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f15cc065b80 0x7f15cc068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f15dc0ae000 tx=0x7f15dc0b5040 comp rx=0 tx=0).stop 2023-12-19T16:57:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.649+0000 7f15ef0c4640 1 -- 172.21.15.26:0/352825337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e8155030 msgr2=0x7f15e8155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.649+0000 7f15ef0c4640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e8155030 0x7f15e8155770 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f15d400b410 tx=0x7f15d400b8d0 comp rx=0 tx=0).stop 2023-12-19T16:57:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.649+0000 7f15ef0c4640 1 -- 172.21.15.26:0/352825337 shutdown_connections 2023-12-19T16:57:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.649+0000 7f15ef0c4640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15e815f370 0x7f15e8151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.649+0000 7f15ef0c4640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15e8155030 0x7f15e8155770 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.649+0000 7f15ef0c4640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f15cc065b80 0x7f15cc068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.649+0000 7f15ef0c4640 1 --2- 172.21.15.26:0/352825337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15e80ce740 0x7f15e813f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.649+0000 7f15ef0c4640 1 -- 172.21.15.26:0/352825337 >> 172.21.15.26:0/352825337 conn(0x7f15e80658e0 msgr2=0x7f15e806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.649+0000 7f15ef0c4640 1 -- 172.21.15.26:0/352825337 shutdown_connections 2023-12-19T16:57:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.649+0000 7f15ef0c4640 1 -- 172.21.15.26:0/352825337 wait complete. 2023-12-19T16:57:15.677 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-12-19T16:57:15.678 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-12-19T16:57:15.678 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-19T16:57:15.678 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-12-19T16:57:15.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f092d4d1640 1 Processor -- start 2023-12-19T16:57:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f092d4d1640 1 -- start start 2023-12-19T16:57:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f092d4d1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09280b8610 0x7f09280d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f092d4d1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09280cd820 0x7f09280cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f092d4d1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092815d110 con 0x7f092815cd30 2023-12-19T16:57:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f092d4d1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09280ce1f0 con 0x7f09280b8610 2023-12-19T16:57:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f092d4d1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09280d12e0 con 0x7f09280cd820 2023-12-19T16:57:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f091e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09280b8610 0x7f09280d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f091e7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09280b8610 0x7f09280d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55652/0 (socket says 172.21.15.26:55652) 2023-12-19T16:57:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f091e7fc640 1 -- 172.21.15.26:0/1835588851 learned_addr learned my addr 172.21.15.26:0/1835588851 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f0926ffd640 1 --1- 172.21.15.26:0/1835588851 >> v1:172.21.15.26:6789/0 conn(0x7f092815cd30 0x7f09280d0b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54150/0 (socket says 172.21.15.26:54150) 2023-12-19T16:57:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f091e7fc640 1 -- 172.21.15.26:0/1835588851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09280cd820 msgr2=0x7f09280cdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f09277fe640 1 --2- 172.21.15.26:0/1835588851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09280cd820 0x7f09280cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f091e7fc640 1 --2- 172.21.15.26:0/1835588851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09280cd820 0x7f09280cdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f091e7fc640 1 -- 172.21.15.26:0/1835588851 >> v1:172.21.15.26:6789/0 conn(0x7f092815cd30 legacy=0x7f09280d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f091e7fc640 1 -- 172.21.15.26:0/1835588851 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09280d15c0 con 0x7f09280b8610 2023-12-19T16:57:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f09267fc640 1 -- 172.21.15.26:0/1835588851 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993519804 0 0) 0x7f092815d110 con 0x7f092815cd30 2023-12-19T16:57:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.821+0000 7f09277fe640 1 --2- 172.21.15.26:0/1835588851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09280cd820 0x7f09280cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.825+0000 7f091e7fc640 1 --2- 172.21.15.26:0/1835588851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09280b8610 0x7f09280d2380 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0918009870 tx=0x7f091801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19da0256085eca9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.825+0000 7f09267fc640 1 -- 172.21.15.26:0/1835588851 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180a9400 con 0x7f09280b8610 2023-12-19T16:57:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.825+0000 7f09267fc640 1 -- 172.21.15.26:0/1835588851 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09180a9560 con 0x7f09280b8610 2023-12-19T16:57:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.825+0000 7f09267fc640 1 -- 172.21.15.26:0/1835588851 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180a98f0 con 0x7f09280b8610 2023-12-19T16:57:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.825+0000 7f092d4d1640 1 -- 172.21.15.26:0/1835588851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09280b8610 msgr2=0x7f09280d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.825+0000 7f092d4d1640 1 --2- 172.21.15.26:0/1835588851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09280b8610 0x7f09280d2380 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0918009870 tx=0x7f091801c540 comp rx=0 tx=0).stop 2023-12-19T16:57:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.825+0000 7f092d4d1640 1 -- 172.21.15.26:0/1835588851 shutdown_connections 2023-12-19T16:57:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.825+0000 7f092d4d1640 1 --2- 172.21.15.26:0/1835588851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09280cd820 0x7f09280cdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.825+0000 7f092d4d1640 1 --2- 172.21.15.26:0/1835588851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09280b8610 0x7f09280d2380 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.825+0000 7f092d4d1640 1 -- 172.21.15.26:0/1835588851 >> 172.21.15.26:0/1835588851 conn(0x7f0928065ce0 msgr2=0x7f09280660f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.825+0000 7f092d4d1640 1 -- 172.21.15.26:0/1835588851 shutdown_connections 2023-12-19T16:57:15.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.825+0000 7f092d4d1640 1 -- 172.21.15.26:0/1835588851 wait complete. 2023-12-19T16:57:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f092d4d1640 1 Processor -- start 2023-12-19T16:57:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f092d4d1640 1 -- start start 2023-12-19T16:57:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f092d4d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09280b8610 0x7f0928154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f092d4d1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09280cd820 0x7f09280c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f092d4d1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092815cd30 0x7f0928156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f092d4d1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092814cc20 con 0x7f09280b8610 2023-12-19T16:57:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f092d4d1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09280c7800 con 0x7f09280cd820 2023-12-19T16:57:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f092d4d1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0928154e20 con 0x7f092815cd30 2023-12-19T16:57:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f0926ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09280b8610 0x7f0928154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f0926ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09280b8610 0x7f0928154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55658/0 (socket says 172.21.15.26:55658) 2023-12-19T16:57:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f0926ffd640 1 -- 172.21.15.26:0/3433807741 learned_addr learned my addr 172.21.15.26:0/3433807741 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f0926ffd640 1 -- 172.21.15.26:0/3433807741 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092815cd30 msgr2=0x7f0928156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f0926ffd640 1 --2- 172.21.15.26:0/3433807741 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092815cd30 0x7f0928156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f0926ffd640 1 -- 172.21.15.26:0/3433807741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09280cd820 msgr2=0x7f09280c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f0926ffd640 1 --2- 172.21.15.26:0/3433807741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09280cd820 0x7f09280c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:15.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f0926ffd640 1 -- 172.21.15.26:0/3433807741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09180094d0 con 0x7f09280b8610 2023-12-19T16:57:15.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f0926ffd640 1 --2- 172.21.15.26:0/3433807741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09280b8610 0x7f0928154840 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f091000c480 tx=0x7f091000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:15.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f091ffff640 1 -- 172.21.15.26:0/3433807741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09100074e0 con 0x7f09280b8610 2023-12-19T16:57:15.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f091ffff640 1 -- 172.21.15.26:0/3433807741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0910007660 con 0x7f09280b8610 2023-12-19T16:57:15.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f091ffff640 1 -- 172.21.15.26:0/3433807741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0910004a30 con 0x7f09280b8610 2023-12-19T16:57:15.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f092d4d1640 1 -- 172.21.15.26:0/3433807741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09280c79e0 con 0x7f09280b8610 2023-12-19T16:57:15.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.829+0000 7f092d4d1640 1 -- 172.21.15.26:0/3433807741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09280c7fa0 con 0x7f09280b8610 2023-12-19T16:57:15.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.833+0000 7f092d4d1640 1 -- 172.21.15.26:0/3433807741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09280c8190 con 0x7f09280b8610 2023-12-19T16:57:15.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.833+0000 7f091ffff640 1 -- 172.21.15.26:0/3433807741 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0910004b90 con 0x7f09280b8610 2023-12-19T16:57:15.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.833+0000 7f091ffff640 1 --2- 172.21.15.26:0/3433807741 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0908065b80 0x7f0908068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:15.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.833+0000 7f091ffff640 1 -- 172.21.15.26:0/3433807741 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f09100827b0 con 0x7f09280b8610 2023-12-19T16:57:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.833+0000 7f091e7fc640 1 --2- 172.21.15.26:0/3433807741 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0908065b80 0x7f0908068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.833+0000 7f091e7fc640 1 --2- 172.21.15.26:0/3433807741 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0908065b80 0x7f0908068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09180ae000 tx=0x7f0918008040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:15.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:15.837+0000 7f091ffff640 1 -- 172.21.15.26:0/3433807741 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f09100538a0 con 0x7f09280b8610 2023-12-19T16:57:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.049+0000 7f092d4d1640 1 -- 172.21.15.26:0/3433807741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f092804e850 con 0x7f09280b8610 2023-12-19T16:57:16.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.653+0000 7f091ffff640 1 -- 172.21.15.26:0/3433807741 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v537) v1 ==== 159+0+0 (secure 0 0 0) 0x7f0910053430 con 0x7f09280b8610 2023-12-19T16:57:16.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.653+0000 7f092d4d1640 1 -- 172.21.15.26:0/3433807741 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0908065b80 msgr2=0x7f0908068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:16.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.653+0000 7f092d4d1640 1 --2- 172.21.15.26:0/3433807741 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0908065b80 0x7f0908068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09180ae000 tx=0x7f0918008040 comp rx=0 tx=0).stop 2023-12-19T16:57:16.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.653+0000 7f092d4d1640 1 -- 172.21.15.26:0/3433807741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09280b8610 msgr2=0x7f0928154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:16.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.653+0000 7f092d4d1640 1 --2- 172.21.15.26:0/3433807741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09280b8610 0x7f0928154840 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f091000c480 tx=0x7f091000c940 comp rx=0 tx=0).stop 2023-12-19T16:57:16.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.653+0000 7f092d4d1640 1 -- 172.21.15.26:0/3433807741 shutdown_connections 2023-12-19T16:57:16.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.653+0000 7f092d4d1640 1 --2- 172.21.15.26:0/3433807741 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0908065b80 0x7f0908068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:16.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.653+0000 7f092d4d1640 1 --2- 172.21.15.26:0/3433807741 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092815cd30 0x7f0928156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:16.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.653+0000 7f092d4d1640 1 --2- 172.21.15.26:0/3433807741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09280cd820 0x7f09280c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:16.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.653+0000 7f092d4d1640 1 --2- 172.21.15.26:0/3433807741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09280b8610 0x7f0928154840 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:16.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.657+0000 7f092d4d1640 1 -- 172.21.15.26:0/3433807741 >> 172.21.15.26:0/3433807741 conn(0x7f0928065ce0 msgr2=0x7f092815bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:16.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.657+0000 7f092d4d1640 1 -- 172.21.15.26:0/3433807741 shutdown_connections 2023-12-19T16:57:16.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.657+0000 7f092d4d1640 1 -- 172.21.15.26:0/3433807741 wait complete. 2023-12-19T16:57:16.683 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T16:57:16.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.825+0000 7fce3fc20640 1 Processor -- start 2023-12-19T16:57:16.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3fc20640 1 -- start start 2023-12-19T16:57:16.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3fc20640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce38159820 0x7fce3815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:16.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3fc20640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fce38159c00 0x7fce38070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:16.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3fc20640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3806fb60 con 0x7fce3806f6c0 2023-12-19T16:57:16.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3fc20640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3815e470 con 0x7fce38159820 2023-12-19T16:57:16.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3fc20640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3806fe40 con 0x7fce38159c00 2023-12-19T16:57:16.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3d9bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce38159820 0x7fce3815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:16.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3d9bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce38159820 0x7fce3815df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58766/0 (socket says 172.21.15.26:58766) 2023-12-19T16:57:16.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3d9bc640 1 -- 172.21.15.26:0/3265286851 learned_addr learned my addr 172.21.15.26:0/3265286851 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:16.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3e1bd640 1 --2- 172.21.15.26:0/3265286851 >> v2:172.21.15.160:3300/0 conn(0x7fce38159c00 0x7fce38070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:16.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3d9bc640 1 -- 172.21.15.26:0/3265286851 >> v2:172.21.15.160:3300/0 conn(0x7fce38159c00 msgr2=0x7fce38070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:16.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3d9bc640 1 --2- 172.21.15.26:0/3265286851 >> v2:172.21.15.160:3300/0 conn(0x7fce38159c00 0x7fce38070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:16.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3d9bc640 1 -- 172.21.15.26:0/3265286851 >> v1:172.21.15.160:6789/0 conn(0x7fce3806f6c0 legacy=0x7fce380c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:16.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3d9bc640 1 -- 172.21.15.26:0/3265286851 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce38070120 con 0x7fce38159820 2023-12-19T16:57:16.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.829+0000 7fce3e1bd640 1 --2- 172.21.15.26:0/3265286851 >> v2:172.21.15.160:3300/0 conn(0x7fce38159c00 0x7fce38070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:16.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3d9bc640 1 --2- 172.21.15.26:0/3265286851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce38159820 0x7fce3815df30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fce3806f200 tx=0x7fce3401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c6b4782fcd3f14 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:16.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3c9ba640 1 -- 172.21.15.26:0/3265286851 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce340ac600 con 0x7fce38159820 2023-12-19T16:57:16.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3c9ba640 1 -- 172.21.15.26:0/3265286851 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce340a8050 con 0x7fce38159820 2023-12-19T16:57:16.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3c9ba640 1 -- 172.21.15.26:0/3265286851 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce340ac980 con 0x7fce38159820 2023-12-19T16:57:16.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3fc20640 1 -- 172.21.15.26:0/3265286851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce38159820 msgr2=0x7fce3815df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:16.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3fc20640 1 --2- 172.21.15.26:0/3265286851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce38159820 0x7fce3815df30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fce3806f200 tx=0x7fce3401c490 comp rx=0 tx=0).stop 2023-12-19T16:57:16.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3fc20640 1 -- 172.21.15.26:0/3265286851 shutdown_connections 2023-12-19T16:57:16.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3fc20640 1 --2- 172.21.15.26:0/3265286851 >> v2:172.21.15.160:3300/0 conn(0x7fce38159c00 0x7fce38070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:16.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3fc20640 1 --2- 172.21.15.26:0/3265286851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce38159820 0x7fce3815df30 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:16.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3fc20640 1 -- 172.21.15.26:0/3265286851 >> 172.21.15.26:0/3265286851 conn(0x7fce380ce730 msgr2=0x7fce380ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:16.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3fc20640 1 -- 172.21.15.26:0/3265286851 shutdown_connections 2023-12-19T16:57:16.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3fc20640 1 -- 172.21.15.26:0/3265286851 wait complete. 2023-12-19T16:57:16.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3fc20640 1 Processor -- start 2023-12-19T16:57:16.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.833+0000 7fce3fc20640 1 -- start start 2023-12-19T16:57:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3fc20640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3806f6c0 0x7fce3814dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3fc20640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce38159820 0x7fce3815a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3fc20640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce38159c00 0x7fce3815a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3fc20640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce380c6660 con 0x7fce3806f6c0 2023-12-19T16:57:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3fc20640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce380c5770 con 0x7fce38159820 2023-12-19T16:57:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3fc20640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce380c5a50 con 0x7fce38159c00 2023-12-19T16:57:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3d9bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3806f6c0 0x7fce3814dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:16.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3d9bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3806f6c0 0x7fce3814dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58774/0 (socket says 172.21.15.26:58774) 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3d9bc640 1 -- 172.21.15.26:0/3908040570 learned_addr learned my addr 172.21.15.26:0/3908040570 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3d1bb640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce38159820 0x7fce3815a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3e1bd640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce38159c00 0x7fce3815a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3d9bc640 1 -- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce38159c00 msgr2=0x7fce3815a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3d9bc640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce38159c00 0x7fce3815a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3d9bc640 1 -- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce38159820 msgr2=0x7fce3815a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3d9bc640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce38159820 0x7fce3815a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3d9bc640 1 -- 172.21.15.26:0/3908040570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce340094d0 con 0x7fce3806f6c0 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3d1bb640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce38159820 0x7fce3815a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3d9bc640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3806f6c0 0x7fce3814dbb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fce3401a0c0 tx=0x7fce3401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce2e7fc640 1 -- 172.21.15.26:0/3908040570 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce340bd070 con 0x7fce3806f6c0 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce2e7fc640 1 -- 172.21.15.26:0/3908040570 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce340b7420 con 0x7fce3806f6c0 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce2e7fc640 1 -- 172.21.15.26:0/3908040570 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce340b7790 con 0x7fce3806f6c0 2023-12-19T16:57:16.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3fc20640 1 -- 172.21.15.26:0/3908040570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3806b880 con 0x7fce3806f6c0 2023-12-19T16:57:16.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3fc20640 1 -- 172.21.15.26:0/3908040570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce380c6010 con 0x7fce3806f6c0 2023-12-19T16:57:16.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce3fc20640 1 -- 172.21.15.26:0/3908040570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce38074270 con 0x7fce3806f6c0 2023-12-19T16:57:16.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.837+0000 7fce2e7fc640 1 -- 172.21.15.26:0/3908040570 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fce340b3040 con 0x7fce3806f6c0 2023-12-19T16:57:16.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.841+0000 7fce2e7fc640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fce1c065b80 0x7fce1c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:16.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.841+0000 7fce2e7fc640 1 -- 172.21.15.26:0/3908040570 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fce3411cf20 con 0x7fce3806f6c0 2023-12-19T16:57:16.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.841+0000 7fce3d1bb640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fce1c065b80 0x7fce1c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:16.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.841+0000 7fce3d1bb640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fce1c065b80 0x7fce1c068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fce38167a60 tx=0x7fce28009040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:16.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:16.845+0000 7fce2e7fc640 1 -- 172.21.15.26:0/3908040570 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fce340ea3e0 con 0x7fce3806f6c0 2023-12-19T16:57:17.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.001+0000 7fce3fc20640 1 -- 172.21.15.26:0/3908040570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fce3806b300 con 0x7fce3806f6c0 2023-12-19T16:57:17.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.005+0000 7fce2e7fc640 1 -- 172.21.15.26:0/3908040570 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fce340eda00 con 0x7fce3806f6c0 2023-12-19T16:57:17.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.005+0000 7fce3fc20640 1 -- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fce1c065b80 msgr2=0x7fce1c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:17.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.005+0000 7fce3fc20640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fce1c065b80 0x7fce1c068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fce38167a60 tx=0x7fce28009040 comp rx=0 tx=0).stop 2023-12-19T16:57:17.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.005+0000 7fce3fc20640 1 -- 172.21.15.26:0/3908040570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3806f6c0 msgr2=0x7fce3814dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:17.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.005+0000 7fce3fc20640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3806f6c0 0x7fce3814dbb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fce3401a0c0 tx=0x7fce3401acd0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.005+0000 7fce3fc20640 1 -- 172.21.15.26:0/3908040570 shutdown_connections 2023-12-19T16:57:17.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.005+0000 7fce3fc20640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fce1c065b80 0x7fce1c068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.005+0000 7fce3fc20640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce38159c00 0x7fce3815a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.005+0000 7fce3fc20640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce38159820 0x7fce3815a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.005+0000 7fce3fc20640 1 --2- 172.21.15.26:0/3908040570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce3806f6c0 0x7fce3814dbb0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.005+0000 7fce3fc20640 1 -- 172.21.15.26:0/3908040570 >> 172.21.15.26:0/3908040570 conn(0x7fce380ce730 msgr2=0x7fce380afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:17.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.005+0000 7fce3fc20640 1 -- 172.21.15.26:0/3908040570 shutdown_connections 2023-12-19T16:57:17.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.005+0000 7fce3fc20640 1 -- 172.21.15.26:0/3908040570 wait complete. 2023-12-19T16:57:17.031 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T16:57:17.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:57:17.134 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:57:17.232 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T16:57:17.232 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:57:17.232 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:17.241 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:57:17.242 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:57:17.242 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:17.251 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:57:17.251 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:57:17.251 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:17.296 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:57:17.296 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:57:17.297 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:17.305 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:57:17.306 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T16:57:17.306 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:57:17.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9af59e640 1 Processor -- start 2023-12-19T16:57:17.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9af59e640 1 -- start start 2023-12-19T16:57:17.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9af59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9b015ca10 0x7fb9b00b8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9af59e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb9b00b4b50 0x7fb9b00b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9af59e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b00b5200 con 0x7fb9b00b8740 2023-12-19T16:57:17.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9af59e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b00b5cb0 con 0x7fb9b00b4b50 2023-12-19T16:57:17.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9af59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00d0cc0 con 0x7fb9b015ca10 2023-12-19T16:57:17.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9a7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb9b00b8740 0x7fb9b00b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55218/0 (socket says 172.21.15.26:55218) 2023-12-19T16:57:17.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9a7fff640 1 -- 172.21.15.26:0/3099190699 learned_addr learned my addr 172.21.15.26:0/3099190699 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:17.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9aed9d640 1 --2- 172.21.15.26:0/3099190699 >> v2:172.21.15.160:3300/0 conn(0x7fb9b00b4b50 0x7fb9b00b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291862717 0 0) 0x7fb9b00b5200 con 0x7fb9b00b8740 2023-12-19T16:57:17.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9ae59c640 1 --2- 172.21.15.26:0/3099190699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9b015ca10 0x7fb9b00b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb99c003530 con 0x7fb9b00b8740 2023-12-19T16:57:17.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1956834490 0 0) 0x7fb99c003530 con 0x7fb9b00b8740 2023-12-19T16:57:17.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9b015ca10 msgr2=0x7fb9b00b8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:17.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 --2- 172.21.15.26:0/3099190699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9b015ca10 0x7fb9b00b8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 >> v2:172.21.15.160:3300/0 conn(0x7fb9b00b4b50 msgr2=0x7fb9b00b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:17.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 --2- 172.21.15.26:0/3099190699 >> v2:172.21.15.160:3300/0 conn(0x7fb9b00b4b50 0x7fb9b00b6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b00b5f50 con 0x7fb9b00b8740 2023-12-19T16:57:17.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fb9980016a0 con 0x7fb9b00b8740 2023-12-19T16:57:17.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 >> v1:172.21.15.26:6789/0 conn(0x7fb9b00b8740 legacy=0x7fb9b00b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 --2- 172.21.15.26:0/3099190699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9b015ca10 0x7fb99c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 --2- 172.21.15.26:0/3099190699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb99c003f70 0x7fb99c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 --2- 172.21.15.26:0/3099190699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb99c004d10 0x7fb99c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00b5cb0 con 0x7fb99c003f70 2023-12-19T16:57:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00d0cc0 con 0x7fb99c004d10 2023-12-19T16:57:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c00b9c0 con 0x7fb9b015ca10 2023-12-19T16:57:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9a7fff640 1 --2- 172.21.15.26:0/3099190699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb99c003f70 0x7fb99c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9ae59c640 1 --2- 172.21.15.26:0/3099190699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9b015ca10 0x7fb99c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9aed9d640 1 --2- 172.21.15.26:0/3099190699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb99c004d10 0x7fb99c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9a7fff640 1 -- 172.21.15.26:0/3099190699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9b015ca10 msgr2=0x7fb99c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:17.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9a7fff640 1 --2- 172.21.15.26:0/3099190699 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9b015ca10 0x7fb99c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9a7fff640 1 -- 172.21.15.26:0/3099190699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb99c004d10 msgr2=0x7fb99c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:17.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9a7fff640 1 --2- 172.21.15.26:0/3099190699 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb99c004d10 0x7fb99c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.481+0000 7fb9a7fff640 1 -- 172.21.15.26:0/3099190699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c003530 con 0x7fb99c003f70 2023-12-19T16:57:17.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.485+0000 7fb9a7fff640 1 --2- 172.21.15.26:0/3099190699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb99c003f70 0x7fb99c0047d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c210 tx=0x7fb99801f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b913a7588e5c6dd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:17.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.485+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998005ae0 con 0x7fb99c003f70 2023-12-19T16:57:17.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.485+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9980ab070 con 0x7fb99c003f70 2023-12-19T16:57:17.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.485+0000 7fb9add9b640 1 -- 172.21.15.26:0/3099190699 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998005e80 con 0x7fb99c003f70 2023-12-19T16:57:17.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.485+0000 7fb9af59e640 1 -- 172.21.15.26:0/3099190699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb99c003f70 msgr2=0x7fb99c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:17.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.485+0000 7fb9af59e640 1 --2- 172.21.15.26:0/3099190699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb99c003f70 0x7fb99c0047d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c210 tx=0x7fb99801f390 comp rx=0 tx=0).stop 2023-12-19T16:57:17.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.485+0000 7fb9aed9d640 1 -- 172.21.15.26:0/3099190699 reap_dead start 2023-12-19T16:57:17.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.485+0000 7fb9af59e640 1 -- 172.21.15.26:0/3099190699 shutdown_connections 2023-12-19T16:57:17.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.485+0000 7fb9af59e640 1 -- 172.21.15.26:0/3099190699 >> 172.21.15.26:0/3099190699 conn(0x7fb9b006a120 msgr2=0x7fb9b006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:17.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.485+0000 7fb9af59e640 1 -- 172.21.15.26:0/3099190699 shutdown_connections 2023-12-19T16:57:17.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.485+0000 7fb9af59e640 1 -- 172.21.15.26:0/3099190699 wait complete. 2023-12-19T16:57:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9af59e640 1 Processor -- start 2023-12-19T16:57:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9af59e640 1 -- start start 2023-12-19T16:57:17.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9af59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c003f70 0x7fb9b015e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9af59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b014dce0 0x7fb9b01f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9af59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b014e120 0x7fb9b01f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9af59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b015ee60 con 0x7fb9b014e120 2023-12-19T16:57:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9af59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b015e490 con 0x7fb9b014dce0 2023-12-19T16:57:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9af59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b01f3120 con 0x7fb99c003f70 2023-12-19T16:57:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9aed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b014e120 0x7fb9b01f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9aed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b014e120 0x7fb9b01f2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58792/0 (socket says 172.21.15.26:58792) 2023-12-19T16:57:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9aed9d640 1 -- 172.21.15.26:0/1276639046 learned_addr learned my addr 172.21.15.26:0/1276639046 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9a7fff640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b014dce0 0x7fb9b01f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9ae59c640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c003f70 0x7fb9b015e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9aed9d640 1 -- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c003f70 msgr2=0x7fb9b015e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9aed9d640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c003f70 0x7fb9b015e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9aed9d640 1 -- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b014dce0 msgr2=0x7fb9b01f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:17.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9aed9d640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b014dce0 0x7fb9b01f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9aed9d640 1 -- 172.21.15.26:0/1276639046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b01f3400 con 0x7fb9b014e120 2023-12-19T16:57:17.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9aed9d640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b014e120 0x7fb9b01f2b10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00b5520 tx=0x7fb9a800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:17.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9a6ffd640 1 -- 172.21.15.26:0/1276639046 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a800e040 con 0x7fb9b014e120 2023-12-19T16:57:17.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9a6ffd640 1 -- 172.21.15.26:0/1276639046 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a800f040 con 0x7fb9b014e120 2023-12-19T16:57:17.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9a6ffd640 1 -- 172.21.15.26:0/1276639046 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8015430 con 0x7fb9b014e120 2023-12-19T16:57:17.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9af59e640 1 -- 172.21.15.26:0/1276639046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b015de40 con 0x7fb9b014e120 2023-12-19T16:57:17.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.489+0000 7fb9af59e640 1 -- 172.21.15.26:0/1276639046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b015fb80 con 0x7fb9b014e120 2023-12-19T16:57:17.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.493+0000 7fb9af59e640 1 -- 172.21.15.26:0/1276639046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b0067400 con 0x7fb9b014e120 2023-12-19T16:57:17.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.493+0000 7fb9a6ffd640 1 -- 172.21.15.26:0/1276639046 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb9a8015590 con 0x7fb9b014e120 2023-12-19T16:57:17.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.493+0000 7fb9a6ffd640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb990065b80 0x7fb990068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.493+0000 7fb9a6ffd640 1 -- 172.21.15.26:0/1276639046 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fb9a8083c30 con 0x7fb9b014e120 2023-12-19T16:57:17.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.493+0000 7fb9ae59c640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb990065b80 0x7fb990068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.493+0000 7fb9ae59c640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb990065b80 0x7fb990068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009b00 tx=0x7fb9a0009190 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.497+0000 7fb9a6ffd640 1 -- 172.21.15.26:0/1276639046 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb9a8051180 con 0x7fb9b014e120 2023-12-19T16:57:17.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9af59e640 1 -- 172.21.15.26:0/1276639046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9b00b6200 con 0x7fb9b014e120 2023-12-19T16:57:17.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9a6ffd640 1 -- 172.21.15.26:0/1276639046 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fb9a80547a0 con 0x7fb9b014e120 2023-12-19T16:57:17.698 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:17.698 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5390,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/1947010592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1947010592},{"type":"v1","addr":"172.21.15.26:6833","nonce":1947010592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5399,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2764469548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2764469548},{"type":"v1","addr":"172.21.15.160:6837","nonce":2764469548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5803,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/255772664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":255772664},{"type":"v1","addr":"172.21.15.160:6836","nonce":255772664}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6429,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2125711758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":2125711758},{"type":"v1","addr":"172.21.15.26:6835","nonce":2125711758}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[]} 2023-12-19T16:57:17.698 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 27 2023-12-19T16:57:17.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9af59e640 1 -- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb990065b80 msgr2=0x7fb990068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9af59e640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb990065b80 0x7fb990068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009b00 tx=0x7fb9a0009190 comp rx=0 tx=0).stop 2023-12-19T16:57:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9af59e640 1 -- 172.21.15.26:0/1276639046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b014e120 msgr2=0x7fb9b01f2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9af59e640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b014e120 0x7fb9b01f2b10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00b5520 tx=0x7fb9a800f1b0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9af59e640 1 -- 172.21.15.26:0/1276639046 shutdown_connections 2023-12-19T16:57:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9af59e640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9b014e120 0x7fb9b01f2b10 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9af59e640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9b014dce0 0x7fb9b01f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9af59e640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb990065b80 0x7fb990068040 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9af59e640 1 --2- 172.21.15.26:0/1276639046 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb99c003f70 0x7fb9b015e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9af59e640 1 -- 172.21.15.26:0/1276639046 >> 172.21.15.26:0/1276639046 conn(0x7fb9b006a120 msgr2=0x7fb9b00b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9af59e640 1 -- 172.21.15.26:0/1276639046 shutdown_connections 2023-12-19T16:57:17.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.693+0000 7fb9af59e640 1 -- 172.21.15.26:0/1276639046 wait complete. 2023-12-19T16:57:17.719 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T16:57:17.720 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T16:57:17.720 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T16:57:17.720 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:57:17.720 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T16:57:17.720 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T16:57:17.720 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T16:57:17.720 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:57:17.721 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T16:57:17.721 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T16:57:17.721 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T16:57:17.721 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:57:17.721 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T16:57:17.721 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T16:57:17.721 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T16:57:17.721 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:57:17.733 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T16:57:17.733 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T16:57:17.734 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T16:57:17.734 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T16:57:17.735 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T16:57:17.735 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T16:57:17.736 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T16:57:17.736 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T16:57:17.780 INFO:tasks.ceph.mds.d:Started 2023-12-19T16:57:17.780 DEBUG:teuthology.parallel:result is None 2023-12-19T16:57:17.782 INFO:tasks.ceph.mds.b:Started 2023-12-19T16:57:17.782 DEBUG:teuthology.parallel:result is None 2023-12-19T16:57:17.783 INFO:tasks.ceph.mds.a:Started 2023-12-19T16:57:17.784 DEBUG:teuthology.parallel:result is None 2023-12-19T16:57:17.786 INFO:tasks.ceph.mds.c:Started 2023-12-19T16:57:17.786 DEBUG:teuthology.parallel:result is None 2023-12-19T16:57:17.786 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T16:57:17.870 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T16:57:17.870 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T16:57:17.872 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T16:57:17.873 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T16:57:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.933+0000 7fbda7ba6640 1 Processor -- start 2023-12-19T16:57:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda7ba6640 1 -- start start 2023-12-19T16:57:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda7ba6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbda00cb1d0 0x7fbda00cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda7ba6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda006bee0 0x7fbda006c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda7ba6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbda0164c90 0x7fbda0165120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda7ba6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda00cc030 con 0x7fbda00cb1d0 2023-12-19T16:57:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda7ba6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda006d4b0 con 0x7fbda0164c90 2023-12-19T16:57:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda7ba6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda00cbaf0 con 0x7fbda006bee0 2023-12-19T16:57:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda5942640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbda00cb1d0 0x7fbda00cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda5942640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbda00cb1d0 0x7fbda00cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58836/0 (socket says 172.21.15.26:58836) 2023-12-19T16:57:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda5942640 1 -- 172.21.15.26:0/4076612464 learned_addr learned my addr 172.21.15.26:0/4076612464 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda6143640 1 --2- 172.21.15.26:0/4076612464 >> v2:172.21.15.160:3300/0 conn(0x7fbda0164c90 0x7fbda0165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda5141640 1 --2- 172.21.15.26:0/4076612464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda006bee0 0x7fbda006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda5942640 1 -- 172.21.15.26:0/4076612464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda006bee0 msgr2=0x7fbda006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda5942640 1 --2- 172.21.15.26:0/4076612464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda006bee0 0x7fbda006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda5942640 1 -- 172.21.15.26:0/4076612464 >> v2:172.21.15.160:3300/0 conn(0x7fbda0164c90 msgr2=0x7fbda0165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda5942640 1 --2- 172.21.15.26:0/4076612464 >> v2:172.21.15.160:3300/0 conn(0x7fbda0164c90 0x7fbda0165120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda5942640 1 -- 172.21.15.26:0/4076612464 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda006cc10 con 0x7fbda00cb1d0 2023-12-19T16:57:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.937+0000 7fbda5141640 1 --2- 172.21.15.26:0/4076612464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda006bee0 0x7fbda006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:17.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda5942640 1 --2- 172.21.15.26:0/4076612464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda00cb1d0 0x7fbda00cb5b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbda00adcc0 tx=0x7fbd9c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19407d0a4ca0e017 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:17.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda4940640 1 -- 172.21.15.26:0/4076612464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0ab600 con 0x7fbda00cb1d0 2023-12-19T16:57:17.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda4940640 1 -- 172.21.15.26:0/4076612464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c0ab760 con 0x7fbda00cb1d0 2023-12-19T16:57:17.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda4940640 1 -- 172.21.15.26:0/4076612464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0abaf0 con 0x7fbda00cb1d0 2023-12-19T16:57:17.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda7ba6640 1 -- 172.21.15.26:0/4076612464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda00cb1d0 msgr2=0x7fbda00cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:17.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda7ba6640 1 --2- 172.21.15.26:0/4076612464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda00cb1d0 0x7fbda00cb5b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbda00adcc0 tx=0x7fbd9c01c440 comp rx=0 tx=0).stop 2023-12-19T16:57:17.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda7ba6640 1 -- 172.21.15.26:0/4076612464 shutdown_connections 2023-12-19T16:57:17.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda7ba6640 1 --2- 172.21.15.26:0/4076612464 >> v2:172.21.15.160:3300/0 conn(0x7fbda0164c90 0x7fbda0165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda7ba6640 1 --2- 172.21.15.26:0/4076612464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda006bee0 0x7fbda006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda7ba6640 1 --2- 172.21.15.26:0/4076612464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda00cb1d0 0x7fbda00cb5b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda7ba6640 1 -- 172.21.15.26:0/4076612464 >> 172.21.15.26:0/4076612464 conn(0x7fbda00658e0 msgr2=0x7fbda0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:17.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda7ba6640 1 -- 172.21.15.26:0/4076612464 shutdown_connections 2023-12-19T16:57:17.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda7ba6640 1 -- 172.21.15.26:0/4076612464 wait complete. 2023-12-19T16:57:17.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.941+0000 7fbda7ba6640 1 Processor -- start 2023-12-19T16:57:17.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda7ba6640 1 -- start start 2023-12-19T16:57:17.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda7ba6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda006bee0 0x7fbda0146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda7ba6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda00cb1d0 0x7fbda014be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda7ba6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda0164c90 0x7fbda014c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda7ba6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda014fcb0 con 0x7fbda006bee0 2023-12-19T16:57:17.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda7ba6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda014c910 con 0x7fbda00cb1d0 2023-12-19T16:57:17.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda7ba6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda01468c0 con 0x7fbda0164c90 2023-12-19T16:57:17.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda5942640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda006bee0 0x7fbda0146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda5942640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda006bee0 0x7fbda0146380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58842/0 (socket says 172.21.15.26:58842) 2023-12-19T16:57:17.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda5942640 1 -- 172.21.15.26:0/1255969862 learned_addr learned my addr 172.21.15.26:0/1255969862 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:17.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda5141640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda00cb1d0 0x7fbda014be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda6143640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda0164c90 0x7fbda014c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda5942640 1 -- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda0164c90 msgr2=0x7fbda014c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:17.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda5942640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda0164c90 0x7fbda014c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda5942640 1 -- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda00cb1d0 msgr2=0x7fbda014be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:17.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda5942640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda00cb1d0 0x7fbda014be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:17.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda5942640 1 -- 172.21.15.26:0/1255969862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c0094d0 con 0x7fbda006bee0 2023-12-19T16:57:17.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda5942640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda006bee0 0x7fbda0146380 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0ad040 tx=0x7fbd9c01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:17.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbd967fc640 1 -- 172.21.15.26:0/1255969862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0b4800 con 0x7fbda006bee0 2023-12-19T16:57:17.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbd967fc640 1 -- 172.21.15.26:0/1255969862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c0b4980 con 0x7fbda006bee0 2023-12-19T16:57:17.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbd967fc640 1 -- 172.21.15.26:0/1255969862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0b4d30 con 0x7fbda006bee0 2023-12-19T16:57:17.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda7ba6640 1 -- 172.21.15.26:0/1255969862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda014f0a0 con 0x7fbda006bee0 2023-12-19T16:57:17.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda7ba6640 1 -- 172.21.15.26:0/1255969862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda014f660 con 0x7fbda006bee0 2023-12-19T16:57:17.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.945+0000 7fbda7ba6640 1 -- 172.21.15.26:0/1255969862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda0074270 con 0x7fbda006bee0 2023-12-19T16:57:17.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.949+0000 7fbd967fc640 1 -- 172.21.15.26:0/1255969862 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbd9c0b3380 con 0x7fbda006bee0 2023-12-19T16:57:17.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.949+0000 7fbd967fc640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd84065b80 0x7fbd84068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:17.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.949+0000 7fbd967fc640 1 -- 172.21.15.26:0/1255969862 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fbd9c11d9a0 con 0x7fbda006bee0 2023-12-19T16:57:17.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.949+0000 7fbda5141640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd84065b80 0x7fbd84068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:17.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.949+0000 7fbda5141640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd84065b80 0x7fbd84068040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbda00b2f50 tx=0x7fbd90008040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:17.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:17.953+0000 7fbd967fc640 1 -- 172.21.15.26:0/1255969862 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbd9c0eaf10 con 0x7fbda006bee0 2023-12-19T16:57:18.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.121+0000 7fbda7ba6640 1 -- 172.21.15.26:0/1255969862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbda0147010 con 0x7fbda006bee0 2023-12-19T16:57:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.917+0000 7fbd967fc640 1 -- 172.21.15.26:0/1255969862 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbd9c0c4080 con 0x7fbda006bee0 2023-12-19T16:57:18.926 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2023-12-19T16:57:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.921+0000 7fbda7ba6640 1 -- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd84065b80 msgr2=0x7fbd84068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.921+0000 7fbda7ba6640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd84065b80 0x7fbd84068040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbda00b2f50 tx=0x7fbd90008040 comp rx=0 tx=0).stop 2023-12-19T16:57:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.921+0000 7fbda7ba6640 1 -- 172.21.15.26:0/1255969862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda006bee0 msgr2=0x7fbda0146380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.921+0000 7fbda7ba6640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda006bee0 0x7fbda0146380 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0ad040 tx=0x7fbd9c01ac80 comp rx=0 tx=0).stop 2023-12-19T16:57:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.921+0000 7fbda7ba6640 1 -- 172.21.15.26:0/1255969862 shutdown_connections 2023-12-19T16:57:18.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.921+0000 7fbda7ba6640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd84065b80 0x7fbd84068040 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:18.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.921+0000 7fbda7ba6640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda0164c90 0x7fbda014c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:18.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.921+0000 7fbda7ba6640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda00cb1d0 0x7fbda014be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:18.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.921+0000 7fbda7ba6640 1 --2- 172.21.15.26:0/1255969862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda006bee0 0x7fbda0146380 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:18.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.921+0000 7fbda7ba6640 1 -- 172.21.15.26:0/1255969862 >> 172.21.15.26:0/1255969862 conn(0x7fbda00658e0 msgr2=0x7fbda0162840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:18.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.921+0000 7fbda7ba6640 1 -- 172.21.15.26:0/1255969862 shutdown_connections 2023-12-19T16:57:18.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:18.921+0000 7fbda7ba6640 1 -- 172.21.15.26:0/1255969862 wait complete. 2023-12-19T16:57:18.948 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T16:57:19.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.089+0000 7f377ced1640 1 Processor -- start 2023-12-19T16:57:19.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.089+0000 7f377ced1640 1 -- start start 2023-12-19T16:57:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.089+0000 7f377ced1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3778154220 0x7f377815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.089+0000 7f377ced1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f37780cee10 0x7f377815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f377ced1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377806bc30 con 0x7f377806b3d0 2023-12-19T16:57:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f377ced1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37780cc6e0 con 0x7f37780cee10 2023-12-19T16:57:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f377ced1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377815afc0 con 0x7f3778154220 2023-12-19T16:57:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f3776d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f37780cee10 0x7f377815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f376ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f377806b3d0 0x7f37780ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55238/0 (socket says 172.21.15.26:55238) 2023-12-19T16:57:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f376ffff640 1 -- 172.21.15.26:0/349966152 learned_addr learned my addr 172.21.15.26:0/349966152 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f377659c640 1 --2- 172.21.15.26:0/349966152 >> v2:172.21.15.160:3300/0 conn(0x7f3778154220 0x7f377815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f3776d9d640 1 -- 172.21.15.26:0/349966152 >> v2:172.21.15.160:3300/0 conn(0x7f3778154220 msgr2=0x7f377815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f3776d9d640 1 --2- 172.21.15.26:0/349966152 >> v2:172.21.15.160:3300/0 conn(0x7f3778154220 0x7f377815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f3776d9d640 1 -- 172.21.15.26:0/349966152 >> v1:172.21.15.26:6789/0 conn(0x7f377806b3d0 legacy=0x7f37780ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f3776d9d640 1 -- 172.21.15.26:0/349966152 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377806b950 con 0x7f37780cee10 2023-12-19T16:57:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f3775d9b640 1 -- 172.21.15.26:0/349966152 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1390636525 0 0) 0x7f377806bc30 con 0x7f377806b3d0 2023-12-19T16:57:19.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f377659c640 1 --2- 172.21.15.26:0/349966152 >> v2:172.21.15.160:3300/0 conn(0x7f3778154220 0x7f377815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:19.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f3776d9d640 1 --2- 172.21.15.26:0/349966152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37780cee10 0x7f377815b1c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f37780667b0 tx=0x7f377001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ce4c26d29b06e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:19.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f3775d9b640 1 -- 172.21.15.26:0/349966152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37700a59a0 con 0x7f37780cee10 2023-12-19T16:57:19.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f3775d9b640 1 -- 172.21.15.26:0/349966152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37700a5b00 con 0x7f37780cee10 2023-12-19T16:57:19.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.093+0000 7f3775d9b640 1 -- 172.21.15.26:0/349966152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37700a5e90 con 0x7f37780cee10 2023-12-19T16:57:19.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 -- 172.21.15.26:0/349966152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37780cee10 msgr2=0x7f377815b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 --2- 172.21.15.26:0/349966152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37780cee10 0x7f377815b1c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f37780667b0 tx=0x7f377001c8e0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 -- 172.21.15.26:0/349966152 shutdown_connections 2023-12-19T16:57:19.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 --2- 172.21.15.26:0/349966152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37780cee10 0x7f377815b1c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 --2- 172.21.15.26:0/349966152 >> v2:172.21.15.160:3300/0 conn(0x7f3778154220 0x7f377815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 -- 172.21.15.26:0/349966152 >> 172.21.15.26:0/349966152 conn(0x7f37780658e0 msgr2=0x7f3778065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:19.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 -- 172.21.15.26:0/349966152 shutdown_connections 2023-12-19T16:57:19.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 -- 172.21.15.26:0/349966152 wait complete. 2023-12-19T16:57:19.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 Processor -- start 2023-12-19T16:57:19.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 -- start start 2023-12-19T16:57:19.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f377806b3d0 0x7f377814ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37780cee10 0x7f3778152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3778154220 0x7f37781f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377814b9a0 con 0x7f37780cee10 2023-12-19T16:57:19.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3778152dc0 con 0x7f3778154220 2023-12-19T16:57:19.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f377ced1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3778150bf0 con 0x7f377806b3d0 2023-12-19T16:57:19.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f376ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37780cee10 0x7f3778152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f376ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37780cee10 0x7f3778152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58852/0 (socket says 172.21.15.26:58852) 2023-12-19T16:57:19.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.097+0000 7f376ffff640 1 -- 172.21.15.26:0/1037798855 learned_addr learned my addr 172.21.15.26:0/1037798855 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:19.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f3776d9d640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3778154220 0x7f37781f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f377659c640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f377806b3d0 0x7f377814ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f376ffff640 1 -- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f377806b3d0 msgr2=0x7f377814ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f376ffff640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f377806b3d0 0x7f377814ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f376ffff640 1 -- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3778154220 msgr2=0x7f37781f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f376ffff640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3778154220 0x7f37781f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f376ffff640 1 -- 172.21.15.26:0/1037798855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3770009a70 con 0x7f37780cee10 2023-12-19T16:57:19.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f3776d9d640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3778154220 0x7f37781f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:19.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f376ffff640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37780cee10 0x7f3778152880 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f376000c480 tx=0x7f376000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:19.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f376effd640 1 -- 172.21.15.26:0/1037798855 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600074e0 con 0x7f37780cee10 2023-12-19T16:57:19.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f376effd640 1 -- 172.21.15.26:0/1037798855 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3760007660 con 0x7f37780cee10 2023-12-19T16:57:19.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f376effd640 1 -- 172.21.15.26:0/1037798855 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760004a30 con 0x7f37780cee10 2023-12-19T16:57:19.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f377ced1640 1 -- 172.21.15.26:0/1037798855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3778152ff0 con 0x7f37780cee10 2023-12-19T16:57:19.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f377ced1640 1 -- 172.21.15.26:0/1037798855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37780b2060 con 0x7f37780cee10 2023-12-19T16:57:19.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f377ced1640 1 -- 172.21.15.26:0/1037798855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3778074270 con 0x7f37780cee10 2023-12-19T16:57:19.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f376effd640 1 -- 172.21.15.26:0/1037798855 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3760004b90 con 0x7f37780cee10 2023-12-19T16:57:19.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f376effd640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3758065b80 0x7f3758068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.101+0000 7f376effd640 1 -- 172.21.15.26:0/1037798855 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f3760082410 con 0x7f37780cee10 2023-12-19T16:57:19.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.105+0000 7f377659c640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3758065b80 0x7f3758068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.105+0000 7f377659c640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3758065b80 0x7f3758068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f376800c900 tx=0x7f3768008040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:19.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.105+0000 7f376effd640 1 -- 172.21.15.26:0/1037798855 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3760053850 con 0x7f37780cee10 2023-12-19T16:57:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.317+0000 7f377ced1640 1 -- 172.21.15.26:0/1037798855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f37780cdf60 con 0x7f37780cee10 2023-12-19T16:57:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.321+0000 7f376effd640 1 -- 172.21.15.26:0/1037798855 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f37600533e0 con 0x7f37780cee10 2023-12-19T16:57:19.324 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:19.324 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T16:57:19.324 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2023-12-19T16:57:19.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.325 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2023-12-19T16:57:19.325 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg==", 2023-12-19T16:57:19.325 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.325 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:19.325 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:57:19.325 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:57:19.325 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.325 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.325 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.325 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.325 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg==", 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg==", 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:19.326 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw==", 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.327 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig==", 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA==", 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.328 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ==", 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.329 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.330 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2023-12-19T16:57:19.330 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ==", 2023-12-19T16:57:19.330 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.330 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:19.330 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:19.330 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.331 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.331 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.331 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2023-12-19T16:57:19.331 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A==", 2023-12-19T16:57:19.331 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.331 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:19.331 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:19.331 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.331 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.331 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ==", 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg==", 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:19.332 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg==", 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.333 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ==", 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ==", 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:57:19.334 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:57:19.335 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:57:19.335 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.335 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.335 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2023-12-19T16:57:19.335 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ==", 2023-12-19T16:57:19.335 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.335 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:19.335 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:57:19.335 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:57:19.335 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:57:19.335 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.335 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ==", 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlT5ynDxAA7Sjbdwrc1RyAXJboysB54Q==", 2023-12-19T16:57:19.336 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.337 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:57:19.337 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:57:19.337 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:57:19.337 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.337 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.337 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.337 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.337 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2023-12-19T16:57:19.337 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA==", 2023-12-19T16:57:19.337 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.337 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:57:19.337 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T16:57:19.338 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.338 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.338 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:19.338 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:19.338 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2023-12-19T16:57:19.338 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg==", 2023-12-19T16:57:19.338 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:19.338 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:57:19.338 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T16:57:19.342 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:19.342 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.342 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:19.342 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-12-19T16:57:19.342 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T16:57:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.321+0000 7f377ced1640 1 -- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3758065b80 msgr2=0x7f3758068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.321+0000 7f377ced1640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3758065b80 0x7f3758068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f376800c900 tx=0x7f3768008040 comp rx=0 tx=0).stop 2023-12-19T16:57:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.321+0000 7f377ced1640 1 -- 172.21.15.26:0/1037798855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37780cee10 msgr2=0x7f3778152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.321+0000 7f377ced1640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37780cee10 0x7f3778152880 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f376000c480 tx=0x7f376000c940 comp rx=0 tx=0).stop 2023-12-19T16:57:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.321+0000 7f377ced1640 1 -- 172.21.15.26:0/1037798855 shutdown_connections 2023-12-19T16:57:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.321+0000 7f377ced1640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3778154220 0x7f37781f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.321+0000 7f377ced1640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37780cee10 0x7f3778152880 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.321+0000 7f377ced1640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3758065b80 0x7f3758068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.321+0000 7f377ced1640 1 --2- 172.21.15.26:0/1037798855 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f377806b3d0 0x7f377814ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.321+0000 7f377ced1640 1 -- 172.21.15.26:0/1037798855 >> 172.21.15.26:0/1037798855 conn(0x7f37780658e0 msgr2=0x7f37781594a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.321+0000 7f377ced1640 1 -- 172.21.15.26:0/1037798855 shutdown_connections 2023-12-19T16:57:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.321+0000 7f377ced1640 1 -- 172.21.15.26:0/1037798855 wait complete. 2023-12-19T16:57:19.346 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:19.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fecb7b640 1 Processor -- start 2023-12-19T16:57:19.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fecb7b640 1 -- start start 2023-12-19T16:57:19.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fecb7b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7fe8154220 0x7f7fe815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fecb7b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7fe80cee10 0x7f7fe815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fecb7b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe806bc30 con 0x7f7fe806b3d0 2023-12-19T16:57:19.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fecb7b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe80cc6e0 con 0x7f7fe8154220 2023-12-19T16:57:19.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fecb7b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe815afc0 con 0x7f7fe80cee10 2023-12-19T16:57:19.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fe659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7fe8154220 0x7f7fe815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fe659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7fe8154220 0x7f7fe815aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58864/0 (socket says 172.21.15.26:58864) 2023-12-19T16:57:19.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fe659c640 1 -- 172.21.15.26:0/3276971690 learned_addr learned my addr 172.21.15.26:0/3276971690 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:19.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fe6d9d640 1 --2- 172.21.15.26:0/3276971690 >> v2:172.21.15.160:3300/0 conn(0x7f7fe80cee10 0x7f7fe815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fe659c640 1 -- 172.21.15.26:0/3276971690 >> v2:172.21.15.160:3300/0 conn(0x7f7fe80cee10 msgr2=0x7f7fe815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:19.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fe659c640 1 --2- 172.21.15.26:0/3276971690 >> v2:172.21.15.160:3300/0 conn(0x7f7fe80cee10 0x7f7fe815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fe659c640 1 -- 172.21.15.26:0/3276971690 >> v1:172.21.15.160:6789/0 conn(0x7f7fe806b3d0 legacy=0x7f7fe80ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:19.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fe659c640 1 -- 172.21.15.26:0/3276971690 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe806b950 con 0x7f7fe8154220 2023-12-19T16:57:19.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.485+0000 7f7fe6d9d640 1 --2- 172.21.15.26:0/3276971690 >> v2:172.21.15.160:3300/0 conn(0x7f7fe80cee10 0x7f7fe815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:19.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.489+0000 7f7fe659c640 1 --2- 172.21.15.26:0/3276971690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe8154220 0x7f7fe815aa80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80cdf30 tx=0x7f7fd801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfdafd0884d3431 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:19.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.489+0000 7f7fe5d9b640 1 -- 172.21.15.26:0/3276971690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd80aa5f0 con 0x7f7fe8154220 2023-12-19T16:57:19.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.489+0000 7f7fe5d9b640 1 -- 172.21.15.26:0/3276971690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd80aa750 con 0x7f7fe8154220 2023-12-19T16:57:19.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.489+0000 7f7fe5d9b640 1 -- 172.21.15.26:0/3276971690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd80aaae0 con 0x7f7fe8154220 2023-12-19T16:57:19.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.489+0000 7f7fecb7b640 1 -- 172.21.15.26:0/3276971690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe8154220 msgr2=0x7f7fe815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.489+0000 7f7fecb7b640 1 --2- 172.21.15.26:0/3276971690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe8154220 0x7f7fe815aa80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7fe80cdf30 tx=0x7f7fd801c490 comp rx=0 tx=0).stop 2023-12-19T16:57:19.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.489+0000 7f7fecb7b640 1 -- 172.21.15.26:0/3276971690 shutdown_connections 2023-12-19T16:57:19.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.489+0000 7f7fecb7b640 1 --2- 172.21.15.26:0/3276971690 >> v2:172.21.15.160:3300/0 conn(0x7f7fe80cee10 0x7f7fe815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.489+0000 7f7fecb7b640 1 --2- 172.21.15.26:0/3276971690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe8154220 0x7f7fe815aa80 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.489+0000 7f7fecb7b640 1 -- 172.21.15.26:0/3276971690 >> 172.21.15.26:0/3276971690 conn(0x7f7fe80658e0 msgr2=0x7f7fe8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:19.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.489+0000 7f7fecb7b640 1 -- 172.21.15.26:0/3276971690 shutdown_connections 2023-12-19T16:57:19.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.489+0000 7f7fecb7b640 1 -- 172.21.15.26:0/3276971690 wait complete. 2023-12-19T16:57:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fecb7b640 1 Processor -- start 2023-12-19T16:57:19.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fecb7b640 1 -- start start 2023-12-19T16:57:19.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fecb7b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fe806b3d0 0x7f7fe8152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fecb7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fe80cee10 0x7f7fe8153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fecb7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe8154220 0x7f7fe815ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fecb7b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe815e7b0 con 0x7f7fe8154220 2023-12-19T16:57:19.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fecb7b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8155410 con 0x7f7fe806b3d0 2023-12-19T16:57:19.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fecb7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe81556f0 con 0x7f7fe80cee10 2023-12-19T16:57:19.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fe6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe8154220 0x7f7fe815ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fe6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe8154220 0x7f7fe815ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58878/0 (socket says 172.21.15.26:58878) 2023-12-19T16:57:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fe6d9d640 1 -- 172.21.15.26:0/2437913417 learned_addr learned my addr 172.21.15.26:0/2437913417 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fe659c640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fe806b3d0 0x7f7fe8152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fdffff640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fe80cee10 0x7f7fe8153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fe6d9d640 1 -- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fe80cee10 msgr2=0x7f7fe8153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fe6d9d640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fe80cee10 0x7f7fe8153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fe6d9d640 1 -- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fe806b3d0 msgr2=0x7f7fe8152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fe6d9d640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fe806b3d0 0x7f7fe8152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fe6d9d640 1 -- 172.21.15.26:0/2437913417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd80094d0 con 0x7f7fe8154220 2023-12-19T16:57:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fdffff640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fe80cee10 0x7f7fe8153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:19.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fe6d9d640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe8154220 0x7f7fe815ec40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000cef0 tx=0x7f7fe000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:19.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fdeffd640 1 -- 172.21.15.26:0/2437913417 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000d840 con 0x7f7fe8154220 2023-12-19T16:57:19.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fdeffd640 1 -- 172.21.15.26:0/2437913417 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe000d9c0 con 0x7f7fe8154220 2023-12-19T16:57:19.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fdeffd640 1 -- 172.21.15.26:0/2437913417 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe000dd70 con 0x7f7fe8154220 2023-12-19T16:57:19.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fecb7b640 1 -- 172.21.15.26:0/2437913417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe81559d0 con 0x7f7fe8154220 2023-12-19T16:57:19.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.493+0000 7f7fecb7b640 1 -- 172.21.15.26:0/2437913417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe8155050 con 0x7f7fe8154220 2023-12-19T16:57:19.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.497+0000 7f7fecb7b640 1 -- 172.21.15.26:0/2437913417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe8155e80 con 0x7f7fe8154220 2023-12-19T16:57:19.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.497+0000 7f7fdeffd640 1 -- 172.21.15.26:0/2437913417 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7fe001e3b0 con 0x7f7fe8154220 2023-12-19T16:57:19.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.497+0000 7f7fdeffd640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7fc8065b80 0x7f7fc8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.497+0000 7f7fdeffd640 1 -- 172.21.15.26:0/2437913417 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f7fe0083c00 con 0x7f7fe8154220 2023-12-19T16:57:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.497+0000 7f7fe659c640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7fc8065b80 0x7f7fc8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.497+0000 7f7fe659c640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7fc8065b80 0x7f7fc8068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80ae000 tx=0x7f7fd80b5040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.501+0000 7f7fdeffd640 1 -- 172.21.15.26:0/2437913417 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7fe00515c0 con 0x7f7fe8154220 2023-12-19T16:57:19.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fecb7b640 1 -- 172.21.15.26:0/2437913417 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fe8074270 con 0x7f7fe8154220 2023-12-19T16:57:19.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fdeffd640 1 -- 172.21.15.26:0/2437913417 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+13494 (secure 0 0 0) 0x7f7fe001a090 con 0x7f7fe8154220 2023-12-19T16:57:19.666 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:19.666 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":75,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:18.911951+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:19.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fecb7b640 1 -- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7fc8065b80 msgr2=0x7f7fc8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fecb7b640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7fc8065b80 0x7f7fc8068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80ae000 tx=0x7f7fd80b5040 comp rx=0 tx=0).stop 2023-12-19T16:57:19.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fecb7b640 1 -- 172.21.15.26:0/2437913417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe8154220 msgr2=0x7f7fe815ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fecb7b640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe8154220 0x7f7fe815ec40 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7fe000cef0 tx=0x7f7fe000a710 comp rx=0 tx=0).stop 2023-12-19T16:57:19.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fecb7b640 1 -- 172.21.15.26:0/2437913417 shutdown_connections 2023-12-19T16:57:19.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fecb7b640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fe8154220 0x7f7fe815ec40 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fecb7b640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7fc8065b80 0x7f7fc8068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fecb7b640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fe80cee10 0x7f7fe8153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fecb7b640 1 --2- 172.21.15.26:0/2437913417 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fe806b3d0 0x7f7fe8152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fecb7b640 1 -- 172.21.15.26:0/2437913417 >> 172.21.15.26:0/2437913417 conn(0x7f7fe80658e0 msgr2=0x7f7fe80cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:19.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fecb7b640 1 -- 172.21.15.26:0/2437913417 shutdown_connections 2023-12-19T16:57:19.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.661+0000 7f7fecb7b640 1 -- 172.21.15.26:0/2437913417 wait complete. 2023-12-19T16:57:19.688 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:57:19.689 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:57:19.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.829+0000 7fc7ad673640 1 Processor -- start 2023-12-19T16:57:19.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.829+0000 7fc7ad673640 1 -- start start 2023-12-19T16:57:19.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7ad673640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a80ce760 0x7fc7a806c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7ad673640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a806c870 con 0x7fc7a81570f0 2023-12-19T16:57:19.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7ad673640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a806cb50 con 0x7fc7a8156d10 2023-12-19T16:57:19.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7ad673640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80cadf0 con 0x7fc7a80ce760 2023-12-19T16:57:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc7a81570f0 0x7fc7a80cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55242/0 (socket says 172.21.15.26:55242) 2023-12-19T16:57:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a77fe640 1 -- 172.21.15.26:0/1837616721 learned_addr learned my addr 172.21.15.26:0/1837616721 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc79ffff640 1 --2- 172.21.15.26:0/1837616721 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a80ce760 0x7fc7a806c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038814852 0 0) 0x7fc7a806c870 con 0x7fc7a81570f0 2023-12-19T16:57:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc794003530 con 0x7fc7a81570f0 2023-12-19T16:57:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208189286 0 0) 0x7fc7a806cb50 con 0x7fc7a8156d10 2023-12-19T16:57:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a806c870 con 0x7fc7a8156d10 2023-12-19T16:57:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2192027594 0 0) 0x7fc794003530 con 0x7fc7a81570f0 2023-12-19T16:57:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a80ce760 msgr2=0x7fc7a806c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:19.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 --2- 172.21.15.26:0/1837616721 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a80ce760 0x7fc7a806c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 >> v1:172.21.15.160:6789/0 conn(0x7fc7a8156d10 legacy=0x7fc7a815d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a806ce30 con 0x7fc7a81570f0 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fc7a0001a00 con 0x7fc7a81570f0 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 >> v1:172.21.15.26:6789/0 conn(0x7fc7a81570f0 legacy=0x7fc7a80cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 --2- 172.21.15.26:0/1837616721 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a80ce760 0x7fc794003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 --2- 172.21.15.26:0/1837616721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7940040a0 0x7fc7940047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 --2- 172.21.15.26:0/1837616721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc794004d10 0x7fc794005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80cadf0 con 0x7fc794004d10 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79400bb20 con 0x7fc7940040a0 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79400bcb0 con 0x7fc7a80ce760 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a77fe640 1 --2- 172.21.15.26:0/1837616721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc794004d10 0x7fc794005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc79ffff640 1 --2- 172.21.15.26:0/1837616721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7940040a0 0x7fc7940047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a6ffd640 1 --2- 172.21.15.26:0/1837616721 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a80ce760 0x7fc794003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a77fe640 1 -- 172.21.15.26:0/1837616721 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a80ce760 msgr2=0x7fc794003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a77fe640 1 --2- 172.21.15.26:0/1837616721 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a80ce760 0x7fc794003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a77fe640 1 -- 172.21.15.26:0/1837616721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7940040a0 msgr2=0x7fc7940047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a77fe640 1 --2- 172.21.15.26:0/1837616721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7940040a0 0x7fc7940047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.833+0000 7fc7a77fe640 1 -- 172.21.15.26:0/1837616721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc794003530 con 0x7fc794004d10 2023-12-19T16:57:19.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.837+0000 7fc7a77fe640 1 --2- 172.21.15.26:0/1837616721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc794004d10 0x7fc794005190 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000a7a0 tx=0x7fc7a001d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3841fd5807f4023 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.837+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00ad630 con 0x7fc794004d10 2023-12-19T16:57:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.837+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a00a8070 con 0x7fc794004d10 2023-12-19T16:57:19.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.837+0000 7fc7a67fc640 1 -- 172.21.15.26:0/1837616721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00ada30 con 0x7fc794004d10 2023-12-19T16:57:19.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.837+0000 7fc7ad673640 1 -- 172.21.15.26:0/1837616721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc794004d10 msgr2=0x7fc794005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.837+0000 7fc7ad673640 1 --2- 172.21.15.26:0/1837616721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc794004d10 0x7fc794005190 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000a7a0 tx=0x7fc7a001d2a0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.837+0000 7fc7a77fe640 1 -- 172.21.15.26:0/1837616721 reap_dead start 2023-12-19T16:57:19.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.837+0000 7fc7ad673640 1 -- 172.21.15.26:0/1837616721 shutdown_connections 2023-12-19T16:57:19.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.837+0000 7fc7ad673640 1 -- 172.21.15.26:0/1837616721 >> 172.21.15.26:0/1837616721 conn(0x7fc7a8069ef0 msgr2=0x7fc7a806a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:19.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.837+0000 7fc7ad673640 1 -- 172.21.15.26:0/1837616721 shutdown_connections 2023-12-19T16:57:19.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.837+0000 7fc7ad673640 1 -- 172.21.15.26:0/1837616721 wait complete. 2023-12-19T16:57:19.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7ad673640 1 Processor -- start 2023-12-19T16:57:19.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7ad673640 1 -- start start 2023-12-19T16:57:19.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7ad673640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794004d10 0x7fc7a80ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7ad673640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a80c5fa0 0x7fc7a80c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7ad673640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7a80c6400 0x7fc7a80c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7ad673640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80af7b0 con 0x7fc7a80c6400 2023-12-19T16:57:19.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7ad673640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80c5930 con 0x7fc794004d10 2023-12-19T16:57:19.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7ad673640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80aeb30 con 0x7fc7a80c5fa0 2023-12-19T16:57:19.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7a80c6400 0x7fc7a80c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7a80c6400 0x7fc7a80c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58894/0 (socket says 172.21.15.26:58894) 2023-12-19T16:57:19.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7a77fe640 1 -- 172.21.15.26:0/2117702912 learned_addr learned my addr 172.21.15.26:0/2117702912 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:19.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc79ffff640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a80c5fa0 0x7fc7a80c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7a6ffd640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794004d10 0x7fc7a80ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7a77fe640 1 -- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a80c5fa0 msgr2=0x7fc7a80c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7a77fe640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a80c5fa0 0x7fc7a80c7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7a77fe640 1 -- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794004d10 msgr2=0x7fc7a80ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:19.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7a77fe640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794004d10 0x7fc7a80ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:19.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7a77fe640 1 -- 172.21.15.26:0/2117702912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a80aee10 con 0x7fc7a80c6400 2023-12-19T16:57:19.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7a77fe640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7a80c6400 0x7fc7a80c8040 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc7a001d0d0 tx=0x7fc794005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:19.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc79f7fe640 1 -- 172.21.15.26:0/2117702912 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00013b0 con 0x7fc7a80c6400 2023-12-19T16:57:19.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc79f7fe640 1 -- 172.21.15.26:0/2117702912 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a0001800 con 0x7fc7a80c6400 2023-12-19T16:57:19.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7ad673640 1 -- 172.21.15.26:0/2117702912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a80af0f0 con 0x7fc7a80c6400 2023-12-19T16:57:19.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc79f7fe640 1 -- 172.21.15.26:0/2117702912 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0004d70 con 0x7fc7a80c6400 2023-12-19T16:57:19.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7ad673640 1 -- 172.21.15.26:0/2117702912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a80c8580 con 0x7fc7a80c6400 2023-12-19T16:57:19.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.841+0000 7fc7ad673640 1 -- 172.21.15.26:0/2117702912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a80c2dd0 con 0x7fc7a80c6400 2023-12-19T16:57:19.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.845+0000 7fc79f7fe640 1 -- 172.21.15.26:0/2117702912 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc7a00aa030 con 0x7fc7a80c6400 2023-12-19T16:57:19.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.845+0000 7fc79f7fe640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc788065b80 0x7fc788068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:19.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.845+0000 7fc79f7fe640 1 -- 172.21.15.26:0/2117702912 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fc798001d30 con 0x7fc7a80c6400 2023-12-19T16:57:19.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.845+0000 7fc7a6ffd640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc788065b80 0x7fc788068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:19.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.845+0000 7fc7a6ffd640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc788065b80 0x7fc788068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc798000c10 tx=0x7fc7980014b0 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:19.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:19.849+0000 7fc79f7fe640 1 -- 172.21.15.26:0/2117702912 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc7a80c2dd0 con 0x7fc7a80c6400 2023-12-19T16:57:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.009+0000 7fc7ad673640 1 -- 172.21.15.26:0/2117702912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7a80c2dd0 con 0x7fc7a80c6400 2023-12-19T16:57:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.009+0000 7fc79f7fe640 1 -- 172.21.15.26:0/2117702912 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7a80c2dd0 con 0x7fc7a80c6400 2023-12-19T16:57:20.017 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:57:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.013+0000 7fc7ad673640 1 -- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc788065b80 msgr2=0x7fc788068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.013+0000 7fc7ad673640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc788065b80 0x7fc788068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc798000c10 tx=0x7fc7980014b0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.013+0000 7fc7ad673640 1 -- 172.21.15.26:0/2117702912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7a80c6400 msgr2=0x7fc7a80c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.013+0000 7fc7ad673640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7a80c6400 0x7fc7a80c8040 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc7a001d0d0 tx=0x7fc794005800 comp rx=0 tx=0).stop 2023-12-19T16:57:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.013+0000 7fc7ad673640 1 -- 172.21.15.26:0/2117702912 shutdown_connections 2023-12-19T16:57:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.013+0000 7fc7ad673640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7a80c6400 0x7fc7a80c8040 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.013+0000 7fc7ad673640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc788065b80 0x7fc788068040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.013+0000 7fc7ad673640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc7a80c5fa0 0x7fc7a80c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.013+0000 7fc7ad673640 1 --2- 172.21.15.26:0/2117702912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc794004d10 0x7fc7a80ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.013+0000 7fc7ad673640 1 -- 172.21.15.26:0/2117702912 >> 172.21.15.26:0/2117702912 conn(0x7fc7a8069ef0 msgr2=0x7fc7a80ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:20.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.013+0000 7fc7ad673640 1 -- 172.21.15.26:0/2117702912 shutdown_connections 2023-12-19T16:57:20.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.013+0000 7fc7ad673640 1 -- 172.21.15.26:0/2117702912 wait complete. 2023-12-19T16:57:20.038 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T16:57:20.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f216617b640 1 Processor -- start 2023-12-19T16:57:20.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f216617b640 1 -- start start 2023-12-19T16:57:20.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f216617b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f21600b88e0 0x7f21600b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:20.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f216617b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21600b4b40 0x7f21600b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:20.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f216617b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21600d2960 con 0x7f216015cf60 2023-12-19T16:57:20.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f216617b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21600b4f40 con 0x7f21600b4b40 2023-12-19T16:57:20.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f216617b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21600d15f0 con 0x7f21600b88e0 2023-12-19T16:57:20.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f215f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f216015cf60 0x7f21600d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55248/0 (socket says 172.21.15.26:55248) 2023-12-19T16:57:20.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f215f7fe640 1 -- 172.21.15.26:0/4267362476 learned_addr learned my addr 172.21.15.26:0/4267362476 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:20.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f215ffff640 1 --2- 172.21.15.26:0/4267362476 >> v2:172.21.15.26:3300/0 conn(0x7f21600b4b40 0x7f21600b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:20.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f215effd640 1 --2- 172.21.15.26:0/4267362476 >> v2:172.21.15.160:3300/0 conn(0x7f21600b88e0 0x7f21600b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:20.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f215e7fc640 1 -- 172.21.15.26:0/4267362476 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 680185351 0 0) 0x7f21600d2960 con 0x7f216015cf60 2023-12-19T16:57:20.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f215e7fc640 1 -- 172.21.15.26:0/4267362476 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2148003530 con 0x7f216015cf60 2023-12-19T16:57:20.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f215ffff640 1 -- 172.21.15.26:0/4267362476 >> v2:172.21.15.160:3300/0 conn(0x7f21600b88e0 msgr2=0x7f21600b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:20.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f215ffff640 1 --2- 172.21.15.26:0/4267362476 >> v2:172.21.15.160:3300/0 conn(0x7f21600b88e0 0x7f21600b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f215ffff640 1 -- 172.21.15.26:0/4267362476 >> v1:172.21.15.26:6789/0 conn(0x7f216015cf60 legacy=0x7f21600d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f215ffff640 1 -- 172.21.15.26:0/4267362476 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21600d18c0 con 0x7f21600b4b40 2023-12-19T16:57:20.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.177+0000 7f215effd640 1 --2- 172.21.15.26:0/4267362476 >> v2:172.21.15.160:3300/0 conn(0x7f21600b88e0 0x7f21600b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:20.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.181+0000 7f215ffff640 1 --2- 172.21.15.26:0/4267362476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21600b4b40 0x7f21600b5cb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f21600cf4b0 tx=0x7f215801f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e089450d26075536 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:20.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.181+0000 7f215e7fc640 1 -- 172.21.15.26:0/4267362476 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21580b0440 con 0x7f21600b4b40 2023-12-19T16:57:20.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.181+0000 7f215e7fc640 1 -- 172.21.15.26:0/4267362476 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21580b05a0 con 0x7f21600b4b40 2023-12-19T16:57:20.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.181+0000 7f215e7fc640 1 -- 172.21.15.26:0/4267362476 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21580b0930 con 0x7f21600b4b40 2023-12-19T16:57:20.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.181+0000 7f216617b640 1 -- 172.21.15.26:0/4267362476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21600b4b40 msgr2=0x7f21600b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:20.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.181+0000 7f216617b640 1 --2- 172.21.15.26:0/4267362476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21600b4b40 0x7f21600b5cb0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f21600cf4b0 tx=0x7f215801f0f0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.181+0000 7f216617b640 1 -- 172.21.15.26:0/4267362476 shutdown_connections 2023-12-19T16:57:20.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.181+0000 7f216617b640 1 --2- 172.21.15.26:0/4267362476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21600b4b40 0x7f21600b5cb0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.181+0000 7f216617b640 1 --2- 172.21.15.26:0/4267362476 >> v2:172.21.15.160:3300/0 conn(0x7f21600b88e0 0x7f21600b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.181+0000 7f216617b640 1 -- 172.21.15.26:0/4267362476 >> 172.21.15.26:0/4267362476 conn(0x7f2160065ee0 msgr2=0x7f21600cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:20.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.181+0000 7f216617b640 1 -- 172.21.15.26:0/4267362476 shutdown_connections 2023-12-19T16:57:20.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.181+0000 7f216617b640 1 -- 172.21.15.26:0/4267362476 wait complete. 2023-12-19T16:57:20.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f216617b640 1 Processor -- start 2023-12-19T16:57:20.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f216617b640 1 -- start start 2023-12-19T16:57:20.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f216617b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21600b4b40 0x7f21600c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:20.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f216617b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21600b88e0 0x7f21600cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:20.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f216617b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f216015cf60 0x7f21600cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:20.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f216617b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21600ccbc0 con 0x7f21600b88e0 2023-12-19T16:57:20.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f216617b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21600ccea0 con 0x7f21600b4b40 2023-12-19T16:57:20.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f216617b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216015e430 con 0x7f216015cf60 2023-12-19T16:57:20.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f215effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21600b88e0 0x7f21600cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:20.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f215effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21600b88e0 0x7f21600cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58906/0 (socket says 172.21.15.26:58906) 2023-12-19T16:57:20.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f215effd640 1 -- 172.21.15.26:0/1667676895 learned_addr learned my addr 172.21.15.26:0/1667676895 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:20.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f215f7fe640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21600b4b40 0x7f21600c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:20.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f215ffff640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f216015cf60 0x7f21600cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:20.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f215effd640 1 -- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f216015cf60 msgr2=0x7f21600cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:20.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f215effd640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f216015cf60 0x7f21600cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f215effd640 1 -- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21600b4b40 msgr2=0x7f21600c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:20.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f215effd640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21600b4b40 0x7f21600c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f215effd640 1 -- 172.21.15.26:0/1667676895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2158009a70 con 0x7f21600b88e0 2023-12-19T16:57:20.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f215f7fe640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21600b4b40 0x7f21600c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:20.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f215ffff640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f216015cf60 0x7f21600cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:20.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f215effd640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21600b88e0 0x7f21600cbe00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f215400e810 tx=0x7f215400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:20.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f2147fff640 1 -- 172.21.15.26:0/1667676895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215400c6d0 con 0x7f21600b88e0 2023-12-19T16:57:20.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f2147fff640 1 -- 172.21.15.26:0/1667676895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215400c830 con 0x7f21600b88e0 2023-12-19T16:57:20.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f216617b640 1 -- 172.21.15.26:0/1667676895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216015e710 con 0x7f21600b88e0 2023-12-19T16:57:20.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f2147fff640 1 -- 172.21.15.26:0/1667676895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215400cbe0 con 0x7f21600b88e0 2023-12-19T16:57:20.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.185+0000 7f216617b640 1 -- 172.21.15.26:0/1667676895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21600cc880 con 0x7f21600b88e0 2023-12-19T16:57:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.189+0000 7f216617b640 1 -- 172.21.15.26:0/1667676895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2160074270 con 0x7f21600b88e0 2023-12-19T16:57:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.189+0000 7f2147fff640 1 -- 172.21.15.26:0/1667676895 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f215400cd40 con 0x7f21600b88e0 2023-12-19T16:57:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.189+0000 7f2147fff640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f214c065bd0 0x7f214c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.189+0000 7f2147fff640 1 -- 172.21.15.26:0/1667676895 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f21540831a0 con 0x7f21600b88e0 2023-12-19T16:57:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.189+0000 7f215f7fe640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f214c065bd0 0x7f214c068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:20.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.189+0000 7f215f7fe640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f214c065bd0 0x7f214c068090 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21500011f0 tx=0x7f2150009040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.193+0000 7f2147fff640 1 -- 172.21.15.26:0/1667676895 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2154050a20 con 0x7f21600b88e0 2023-12-19T16:57:20.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.349+0000 7f216617b640 1 -- 172.21.15.26:0/1667676895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f21600d1e50 con 0x7f21600b88e0 2023-12-19T16:57:20.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.349+0000 7f2147fff640 1 -- 172.21.15.26:0/1667676895 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2154054040 con 0x7f21600b88e0 2023-12-19T16:57:20.356 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:20.357 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T16:57:20.357 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:20.357 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T16:57:20.357 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T16:57:20.357 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:20.357 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T16:57:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.353+0000 7f216617b640 1 -- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f214c065bd0 msgr2=0x7f214c068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.353+0000 7f216617b640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f214c065bd0 0x7f214c068090 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21500011f0 tx=0x7f2150009040 comp rx=0 tx=0).stop 2023-12-19T16:57:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.353+0000 7f216617b640 1 -- 172.21.15.26:0/1667676895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21600b88e0 msgr2=0x7f21600cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.353+0000 7f216617b640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21600b88e0 0x7f21600cbe00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f215400e810 tx=0x7f215400ecd0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.353+0000 7f216617b640 1 -- 172.21.15.26:0/1667676895 shutdown_connections 2023-12-19T16:57:20.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.353+0000 7f216617b640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f214c065bd0 0x7f214c068090 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.353+0000 7f216617b640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f216015cf60 0x7f21600cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.353+0000 7f216617b640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21600b88e0 0x7f21600cbe00 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.353+0000 7f216617b640 1 --2- 172.21.15.26:0/1667676895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21600b4b40 0x7f21600c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.353+0000 7f216617b640 1 -- 172.21.15.26:0/1667676895 >> 172.21.15.26:0/1667676895 conn(0x7f2160065ee0 msgr2=0x7f21600b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:20.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.353+0000 7f216617b640 1 -- 172.21.15.26:0/1667676895 shutdown_connections 2023-12-19T16:57:20.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.353+0000 7f216617b640 1 -- 172.21.15.26:0/1667676895 wait complete. 2023-12-19T16:57:20.378 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T16:57:20.378 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T16:57:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.521+0000 7f3f3afce640 1 Processor -- start 2023-12-19T16:57:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.521+0000 7f3f3afce640 1 -- start start 2023-12-19T16:57:20.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.521+0000 7f3f3afce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3f340b3f90 0x7f3f3406c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:20.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.521+0000 7f3f3afce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3406cbf0 con 0x7f3f3415aad0 2023-12-19T16:57:20.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.521+0000 7f3f3afce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f340ca8b0 con 0x7f3f340b1590 2023-12-19T16:57:20.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.521+0000 7f3f3afce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f3406b6f0 con 0x7f3f340b3f90 2023-12-19T16:57:20.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.521+0000 7f3f38d6a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3f3415aad0 0x7f3f34070870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55250/0 (socket says 172.21.15.26:55250) 2023-12-19T16:57:20.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.521+0000 7f3f38d6a640 1 -- 172.21.15.26:0/1422435824 learned_addr learned my addr 172.21.15.26:0/1422435824 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:20.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.525+0000 7f3f3956b640 1 --2- 172.21.15.26:0/1422435824 >> v2:172.21.15.160:3300/0 conn(0x7f3f340b3f90 0x7f3f3406c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:20.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.525+0000 7f3f337fe640 1 -- 172.21.15.26:0/1422435824 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333561263 0 0) 0x7f3f28001120 con 0x7f3f3415aad0 2023-12-19T16:57:20.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.525+0000 7f3f337fe640 1 -- 172.21.15.26:0/1422435824 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f24003530 con 0x7f3f3415aad0 2023-12-19T16:57:20.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.525+0000 7f3f3956b640 1 -- 172.21.15.26:0/1422435824 >> v1:172.21.15.160:6789/0 conn(0x7f3f340b1590 legacy=0x7f3f340cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:20.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.525+0000 7f3f3956b640 1 -- 172.21.15.26:0/1422435824 >> v1:172.21.15.26:6789/0 conn(0x7f3f3415aad0 legacy=0x7f3f34070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:20.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.525+0000 7f3f3956b640 1 -- 172.21.15.26:0/1422435824 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3406b9d0 con 0x7f3f340b3f90 2023-12-19T16:57:20.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.525+0000 7f3f3956b640 1 --2- 172.21.15.26:0/1422435824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f340b3f90 0x7f3f3406c6b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009d90 tx=0x7f3f2c01c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c384a762e6032303 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:20.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.525+0000 7f3f337fe640 1 -- 172.21.15.26:0/1422435824 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0a58f0 con 0x7f3f340b3f90 2023-12-19T16:57:20.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.525+0000 7f3f337fe640 1 -- 172.21.15.26:0/1422435824 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2c0a5a50 con 0x7f3f340b3f90 2023-12-19T16:57:20.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.525+0000 7f3f337fe640 1 -- 172.21.15.26:0/1422435824 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0a5de0 con 0x7f3f340b3f90 2023-12-19T16:57:20.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 -- 172.21.15.26:0/1422435824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f340b3f90 msgr2=0x7f3f3406c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:20.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 --2- 172.21.15.26:0/1422435824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f340b3f90 0x7f3f3406c6b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009d90 tx=0x7f3f2c01c920 comp rx=0 tx=0).stop 2023-12-19T16:57:20.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 -- 172.21.15.26:0/1422435824 shutdown_connections 2023-12-19T16:57:20.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 --2- 172.21.15.26:0/1422435824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f340b3f90 0x7f3f3406c6b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 -- 172.21.15.26:0/1422435824 >> 172.21.15.26:0/1422435824 conn(0x7f3f340b0270 msgr2=0x7f3f340b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:20.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 -- 172.21.15.26:0/1422435824 shutdown_connections 2023-12-19T16:57:20.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 -- 172.21.15.26:0/1422435824 wait complete. 2023-12-19T16:57:20.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 Processor -- start 2023-12-19T16:57:20.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 -- start start 2023-12-19T16:57:20.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f340b1590 0x7f3f34159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:20.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f340b3f90 0x7f3f341597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:20.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3415aad0 0x7f3f34159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:20.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f341595b0 con 0x7f3f340b1590 2023-12-19T16:57:20.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f340ab310 con 0x7f3f3415aad0 2023-12-19T16:57:20.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3afce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f340cc3b0 con 0x7f3f340b3f90 2023-12-19T16:57:20.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f38d6a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f340b1590 0x7f3f34159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:20.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f38d6a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f340b1590 0x7f3f34159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58922/0 (socket says 172.21.15.26:58922) 2023-12-19T16:57:20.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f38d6a640 1 -- 172.21.15.26:0/2473856356 learned_addr learned my addr 172.21.15.26:0/2473856356 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:20.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f3956b640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3415aad0 0x7f3f34159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:20.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f38d6a640 1 -- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f340b3f90 msgr2=0x7f3f341597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:20.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.529+0000 7f3f33fff640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f340b3f90 0x7f3f341597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:20.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f38d6a640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f340b3f90 0x7f3f341597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f38d6a640 1 -- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3415aad0 msgr2=0x7f3f34159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:20.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f38d6a640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3415aad0 0x7f3f34159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:20.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f38d6a640 1 -- 172.21.15.26:0/2473856356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2c009a70 con 0x7f3f340b1590 2023-12-19T16:57:20.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f3956b640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3415aad0 0x7f3f34159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:20.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f33fff640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f340b3f90 0x7f3f341597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:20.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f38d6a640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f340b1590 0x7f3f34159070 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800c420 tx=0x7f3f2800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:20.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f317fa640 1 -- 172.21.15.26:0/2473856356 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f28005cf0 con 0x7f3f340b1590 2023-12-19T16:57:20.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f317fa640 1 -- 172.21.15.26:0/2473856356 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f28005e50 con 0x7f3f340b1590 2023-12-19T16:57:20.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f317fa640 1 -- 172.21.15.26:0/2473856356 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f280072f0 con 0x7f3f340b1590 2023-12-19T16:57:20.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f3afce640 1 -- 172.21.15.26:0/2473856356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f340cc690 con 0x7f3f340b1590 2023-12-19T16:57:20.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f3afce640 1 -- 172.21.15.26:0/2473856356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f340ccc50 con 0x7f3f340b1590 2023-12-19T16:57:20.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f3afce640 1 -- 172.21.15.26:0/2473856356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f340cce40 con 0x7f3f340b1590 2023-12-19T16:57:20.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f317fa640 1 -- 172.21.15.26:0/2473856356 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3f28007450 con 0x7f3f340b1590 2023-12-19T16:57:20.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f317fa640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f18065b80 0x7f3f18068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:20.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f317fa640 1 -- 172.21.15.26:0/2473856356 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f3f28082760 con 0x7f3f340b1590 2023-12-19T16:57:20.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.533+0000 7f3f33fff640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f18065b80 0x7f3f18068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:20.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.537+0000 7f3f33fff640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f18065b80 0x7f3f18068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f340add80 tx=0x7f3f20009040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:20.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.537+0000 7f3f317fa640 1 -- 172.21.15.26:0/2473856356 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3f28050070 con 0x7f3f340b1590 2023-12-19T16:57:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:20.697+0000 7f3f3afce640 1 -- 172.21.15.26:0/2473856356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3f3406cbf0 con 0x7f3f340b1590 2023-12-19T16:57:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.681+0000 7f3f317fa640 1 -- 172.21.15.26:0/2473856356 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3f3406cbf0 con 0x7f3f340b1590 2023-12-19T16:57:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T16:57:21.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.685+0000 7f3f3afce640 1 -- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f18065b80 msgr2=0x7f3f18068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:21.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.685+0000 7f3f3afce640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f18065b80 0x7f3f18068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3f340add80 tx=0x7f3f20009040 comp rx=0 tx=0).stop 2023-12-19T16:57:21.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.685+0000 7f3f3afce640 1 -- 172.21.15.26:0/2473856356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f340b1590 msgr2=0x7f3f34159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:21.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.685+0000 7f3f3afce640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f340b1590 0x7f3f34159070 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800c420 tx=0x7f3f2800c8e0 comp rx=0 tx=0).stop 2023-12-19T16:57:21.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.685+0000 7f3f3afce640 1 -- 172.21.15.26:0/2473856356 shutdown_connections 2023-12-19T16:57:21.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.685+0000 7f3f3afce640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3415aad0 0x7f3f34159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:21.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.685+0000 7f3f3afce640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f18065b80 0x7f3f18068040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:21.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.685+0000 7f3f3afce640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f340b3f90 0x7f3f341597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:21.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.685+0000 7f3f3afce640 1 --2- 172.21.15.26:0/2473856356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f340b1590 0x7f3f34159070 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:21.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.685+0000 7f3f3afce640 1 -- 172.21.15.26:0/2473856356 >> 172.21.15.26:0/2473856356 conn(0x7f3f340b0270 msgr2=0x7f3f340c1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:21.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.685+0000 7f3f3afce640 1 -- 172.21.15.26:0/2473856356 shutdown_connections 2023-12-19T16:57:21.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.685+0000 7f3f3afce640 1 -- 172.21.15.26:0/2473856356 wait complete. 2023-12-19T16:57:21.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T16:57:21.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.865+0000 7fc6ff196640 1 Processor -- start 2023-12-19T16:57:21.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.865+0000 7fc6ff196640 1 -- start start 2023-12-19T16:57:21.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.869+0000 7fc6ff196640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6f806c740 0x7fc6f806f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:21.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.869+0000 7fc6ff196640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc6f806cb20 0x7fc6f80ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:21.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.869+0000 7fc6ff196640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f806f790 con 0x7fc6f80cad40 2023-12-19T16:57:21.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.869+0000 7fc6ff196640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f815d5a0 con 0x7fc6f806cb20 2023-12-19T16:57:21.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.869+0000 7fc6ff196640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f815d880 con 0x7fc6f806c740 2023-12-19T16:57:21.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.869+0000 7fc6fcf32640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6f806c740 0x7fc6f806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:21.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.869+0000 7fc6f7fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc6f806cb20 0x7fc6f80ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:21.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.869+0000 7fc6fd733640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fc6f80cad40 0x7fc6f80c9080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:57318/0 (socket says 172.21.15.26:57318) 2023-12-19T16:57:21.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.869+0000 7fc6fd733640 1 -- 172.21.15.26:0/1668365556 learned_addr learned my addr 172.21.15.26:0/1668365556 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:21.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.869+0000 7fc6fcf32640 1 -- 172.21.15.26:0/1668365556 >> v2:172.21.15.160:3300/0 conn(0x7fc6f806cb20 msgr2=0x7fc6f80ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:21.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.869+0000 7fc6fcf32640 1 --2- 172.21.15.26:0/1668365556 >> v2:172.21.15.160:3300/0 conn(0x7fc6f806cb20 0x7fc6f80ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:21.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.869+0000 7fc6fcf32640 1 -- 172.21.15.26:0/1668365556 >> v1:172.21.15.160:6789/0 conn(0x7fc6f80cad40 legacy=0x7fc6f80c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:21.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.869+0000 7fc6fcf32640 1 -- 172.21.15.26:0/1668365556 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f815db60 con 0x7fc6f806c740 2023-12-19T16:57:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6fcf32640 1 --2- 172.21.15.26:0/1668365556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6f806c740 0x7fc6f806f250 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0097c0 tx=0x7fc6ec01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33403f7cb1380a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6f77fe640 1 -- 172.21.15.26:0/1668365556 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0ad660 con 0x7fc6f806c740 2023-12-19T16:57:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6f77fe640 1 -- 172.21.15.26:0/1668365556 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ec0ad7c0 con 0x7fc6f806c740 2023-12-19T16:57:21.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6f77fe640 1 -- 172.21.15.26:0/1668365556 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0adb50 con 0x7fc6f806c740 2023-12-19T16:57:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6ff196640 1 -- 172.21.15.26:0/1668365556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6f806c740 msgr2=0x7fc6f806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6ff196640 1 --2- 172.21.15.26:0/1668365556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6f806c740 0x7fc6f806f250 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0097c0 tx=0x7fc6ec01c450 comp rx=0 tx=0).stop 2023-12-19T16:57:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6ff196640 1 -- 172.21.15.26:0/1668365556 shutdown_connections 2023-12-19T16:57:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6ff196640 1 --2- 172.21.15.26:0/1668365556 >> v2:172.21.15.160:3300/0 conn(0x7fc6f806cb20 0x7fc6f80ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:21.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6ff196640 1 --2- 172.21.15.26:0/1668365556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6f806c740 0x7fc6f806f250 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6ff196640 1 -- 172.21.15.26:0/1668365556 >> 172.21.15.26:0/1668365556 conn(0x7fc6f80660d0 msgr2=0x7fc6f80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6ff196640 1 -- 172.21.15.26:0/1668365556 shutdown_connections 2023-12-19T16:57:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6ff196640 1 -- 172.21.15.26:0/1668365556 wait complete. 2023-12-19T16:57:21.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6ff196640 1 Processor -- start 2023-12-19T16:57:21.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.873+0000 7fc6ff196640 1 -- start start 2023-12-19T16:57:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6ff196640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6f806c740 0x7fc6f80c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6ff196640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6f806cb20 0x7fc6f814a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6ff196640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6f80cad40 0x7fc6f815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6ff196640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f814c500 con 0x7fc6f80cad40 2023-12-19T16:57:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6ff196640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f814f890 con 0x7fc6f806c740 2023-12-19T16:57:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6ff196640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f814e220 con 0x7fc6f806cb20 2023-12-19T16:57:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6fd733640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6f80cad40 0x7fc6f815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6fd733640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6f80cad40 0x7fc6f815c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58924/0 (socket says 172.21.15.26:58924) 2023-12-19T16:57:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6fd733640 1 -- 172.21.15.26:0/4132665922 learned_addr learned my addr 172.21.15.26:0/4132665922 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6f7fff640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6f806cb20 0x7fc6f814a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6fcf32640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6f806c740 0x7fc6f80c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:21.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6fd733640 1 -- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6f806cb20 msgr2=0x7fc6f814a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:21.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6fd733640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6f806cb20 0x7fc6f814a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:21.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6fd733640 1 -- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6f806c740 msgr2=0x7fc6f80c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:21.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6fd733640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6f806c740 0x7fc6f80c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:21.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6fd733640 1 -- 172.21.15.26:0/4132665922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec009420 con 0x7fc6f80cad40 2023-12-19T16:57:21.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6f7fff640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6f806cb20 0x7fc6f814a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:21.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6fd733640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6f80cad40 0x7fc6f815c2a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000ca80 tx=0x7fc6f000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:21.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6f57fa640 1 -- 172.21.15.26:0/4132665922 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f000ac30 con 0x7fc6f80cad40 2023-12-19T16:57:21.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6f57fa640 1 -- 172.21.15.26:0/4132665922 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f000ad90 con 0x7fc6f80cad40 2023-12-19T16:57:21.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6ff196640 1 -- 172.21.15.26:0/4132665922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f814dc60 con 0x7fc6f80cad40 2023-12-19T16:57:21.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6f57fa640 1 -- 172.21.15.26:0/4132665922 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f0012600 con 0x7fc6f80cad40 2023-12-19T16:57:21.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6ff196640 1 -- 172.21.15.26:0/4132665922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f814f520 con 0x7fc6f80cad40 2023-12-19T16:57:21.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.877+0000 7fc6ff196640 1 -- 172.21.15.26:0/4132665922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f814fa20 con 0x7fc6f80cad40 2023-12-19T16:57:21.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.881+0000 7fc6f57fa640 1 -- 172.21.15.26:0/4132665922 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc6f0007070 con 0x7fc6f80cad40 2023-12-19T16:57:21.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.881+0000 7fc6f57fa640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc6dc065b80 0x7fc6dc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.881+0000 7fc6f57fa640 1 -- 172.21.15.26:0/4132665922 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5959+0+0 (secure 0 0 0) 0x7fc6f0083730 con 0x7fc6f80cad40 2023-12-19T16:57:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.881+0000 7fc6fcf32640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc6dc065b80 0x7fc6dc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:21.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.881+0000 7fc6fcf32640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc6dc065b80 0x7fc6dc068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0af000 tx=0x7fc6ec008040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:21.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:21.885+0000 7fc6f57fa640 1 -- 172.21.15.26:0/4132665922 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc6f0050f90 con 0x7fc6f80cad40 2023-12-19T16:57:22.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.045+0000 7fc6ff196640 1 -- 172.21.15.26:0/4132665922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc6f8074270 con 0x7fc6f80cad40 2023-12-19T16:57:22.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.689+0000 7fc6f57fa640 1 -- 172.21.15.26:0/4132665922 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc6f00545b0 con 0x7fc6f80cad40 2023-12-19T16:57:22.694 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T16:57:22.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.697+0000 7fc6ff196640 1 -- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc6dc065b80 msgr2=0x7fc6dc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:22.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.697+0000 7fc6ff196640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc6dc065b80 0x7fc6dc068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0af000 tx=0x7fc6ec008040 comp rx=0 tx=0).stop 2023-12-19T16:57:22.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.697+0000 7fc6ff196640 1 -- 172.21.15.26:0/4132665922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6f80cad40 msgr2=0x7fc6f815c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:22.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.697+0000 7fc6ff196640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6f80cad40 0x7fc6f815c2a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc6f000ca80 tx=0x7fc6f000cf40 comp rx=0 tx=0).stop 2023-12-19T16:57:22.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.697+0000 7fc6ff196640 1 -- 172.21.15.26:0/4132665922 shutdown_connections 2023-12-19T16:57:22.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.697+0000 7fc6ff196640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6f80cad40 0x7fc6f815c2a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:22.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.697+0000 7fc6ff196640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc6dc065b80 0x7fc6dc068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:22.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.697+0000 7fc6ff196640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6f806cb20 0x7fc6f814a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:22.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.697+0000 7fc6ff196640 1 --2- 172.21.15.26:0/4132665922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6f806c740 0x7fc6f80c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:22.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.697+0000 7fc6ff196640 1 -- 172.21.15.26:0/4132665922 >> 172.21.15.26:0/4132665922 conn(0x7fc6f80660d0 msgr2=0x7fc6f806f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:22.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.701+0000 7fc6ff196640 1 -- 172.21.15.26:0/4132665922 shutdown_connections 2023-12-19T16:57:22.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.701+0000 7fc6ff196640 1 -- 172.21.15.26:0/4132665922 wait complete. 2023-12-19T16:57:22.741 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T16:57:22.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.885+0000 7f468b1df640 1 Processor -- start 2023-12-19T16:57:22.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.885+0000 7f468b1df640 1 -- start start 2023-12-19T16:57:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f468b1df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46840ce110 0x7f468415dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f468b1df640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f46840ac860 0x7f46840cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f468b1df640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468406d2f0 con 0x7f468406b4d0 2023-12-19T16:57:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f468b1df640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468415e1a0 con 0x7f46840ac860 2023-12-19T16:57:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f468b1df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4684072e00 con 0x7f46840ce110 2023-12-19T16:57:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f46899dc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f468406b4d0 0x7f46840ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55262/0 (socket says 172.21.15.26:55262) 2023-12-19T16:57:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f46899dc640 1 -- 172.21.15.26:0/500379060 learned_addr learned my addr 172.21.15.26:0/500379060 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f468a9de640 1 --2- 172.21.15.26:0/500379060 >> v2:172.21.15.160:3300/0 conn(0x7f46840ac860 0x7f46840cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f468a1dd640 1 --2- 172.21.15.26:0/500379060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46840ce110 0x7f468415dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f46891db640 1 -- 172.21.15.26:0/500379060 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539711548 0 0) 0x7f468406d2f0 con 0x7f468406b4d0 2023-12-19T16:57:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f46891db640 1 -- 172.21.15.26:0/500379060 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4670003530 con 0x7f468406b4d0 2023-12-19T16:57:22.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f468a9de640 1 -- 172.21.15.26:0/500379060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46840ce110 msgr2=0x7f468415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:22.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f468a9de640 1 --2- 172.21.15.26:0/500379060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46840ce110 0x7f468415dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:22.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f468a9de640 1 -- 172.21.15.26:0/500379060 >> v1:172.21.15.26:6789/0 conn(0x7f468406b4d0 legacy=0x7f46840ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:22.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f468a9de640 1 -- 172.21.15.26:0/500379060 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46840730e0 con 0x7f46840ac860 2023-12-19T16:57:22.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.889+0000 7f468a1dd640 1 --2- 172.21.15.26:0/500379060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46840ce110 0x7f468415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f468a9de640 1 --2- 172.21.15.26:0/500379060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46840ac860 0x7f46840cd0a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f467c009d90 tx=0x7f467c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2720a49383a7b892 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f46891db640 1 -- 172.21.15.26:0/500379060 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0a5970 con 0x7f46840ac860 2023-12-19T16:57:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f46891db640 1 -- 172.21.15.26:0/500379060 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c0a5ad0 con 0x7f46840ac860 2023-12-19T16:57:22.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f46891db640 1 -- 172.21.15.26:0/500379060 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0a5e60 con 0x7f46840ac860 2023-12-19T16:57:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f468b1df640 1 -- 172.21.15.26:0/500379060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46840ac860 msgr2=0x7f46840cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f468b1df640 1 --2- 172.21.15.26:0/500379060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46840ac860 0x7f46840cd0a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f467c009d90 tx=0x7f467c01c890 comp rx=0 tx=0).stop 2023-12-19T16:57:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f468b1df640 1 -- 172.21.15.26:0/500379060 shutdown_connections 2023-12-19T16:57:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f468b1df640 1 --2- 172.21.15.26:0/500379060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46840ac860 0x7f46840cd0a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f468b1df640 1 --2- 172.21.15.26:0/500379060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46840ce110 0x7f468415dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f468b1df640 1 -- 172.21.15.26:0/500379060 >> 172.21.15.26:0/500379060 conn(0x7f46840cede0 msgr2=0x7f46840cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:22.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f468b1df640 1 -- 172.21.15.26:0/500379060 shutdown_connections 2023-12-19T16:57:22.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f468b1df640 1 -- 172.21.15.26:0/500379060 wait complete. 2023-12-19T16:57:22.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f468b1df640 1 Processor -- start 2023-12-19T16:57:22.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.893+0000 7f468b1df640 1 -- start start 2023-12-19T16:57:22.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468b1df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46840ac860 0x7f46840c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:22.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468b1df640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46840ce110 0x7f46840c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:22.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468b1df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468415bae0 0x7f468415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:22.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468b1df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46840c28c0 con 0x7f468415bae0 2023-12-19T16:57:22.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468b1df640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46840c3860 con 0x7f46840ce110 2023-12-19T16:57:22.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468b1df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46840c2aa0 con 0x7f46840ac860 2023-12-19T16:57:22.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468a9de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468415bae0 0x7f468415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:22.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468a1dd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46840ac860 0x7f46840c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:22.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f46899dc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46840ce110 0x7f46840c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:22.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468a1dd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46840ac860 0x7f46840c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:37466/0 (socket says 172.21.15.26:37466) 2023-12-19T16:57:22.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468a9de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468415bae0 0x7f468415bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58936/0 (socket says 172.21.15.26:58936) 2023-12-19T16:57:22.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468a1dd640 1 -- 172.21.15.26:0/4287143476 learned_addr learned my addr 172.21.15.26:0/4287143476 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:22.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f46899dc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46840ce110 0x7f46840c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:56218/0 (socket says 172.21.15.26:56218) 2023-12-19T16:57:22.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468a1dd640 1 -- 172.21.15.26:0/4287143476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46840ce110 msgr2=0x7f46840c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:22.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468a1dd640 1 --2- 172.21.15.26:0/4287143476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46840ce110 0x7f46840c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:22.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468a1dd640 1 -- 172.21.15.26:0/4287143476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468415bae0 msgr2=0x7f468415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:22.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468a1dd640 1 --2- 172.21.15.26:0/4287143476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468415bae0 0x7f468415bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:22.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468a1dd640 1 -- 172.21.15.26:0/4287143476 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4680009420 con 0x7f46840ac860 2023-12-19T16:57:22.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468a9de640 1 --2- 172.21.15.26:0/4287143476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468415bae0 0x7f468415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:22.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468a1dd640 1 --2- 172.21.15.26:0/4287143476 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46840ac860 0x7f46840c9410 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f468000e980 tx=0x7f468000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:22.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f467affd640 1 -- 172.21.15.26:0/4287143476 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000ca00 con 0x7f46840ac860 2023-12-19T16:57:22.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f467affd640 1 -- 172.21.15.26:0/4287143476 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468000cb60 con 0x7f46840ac860 2023-12-19T16:57:22.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f467affd640 1 -- 172.21.15.26:0/4287143476 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680013470 con 0x7f46840ac860 2023-12-19T16:57:22.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468b1df640 1 -- 172.21.15.26:0/4287143476 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c009a70 con 0x7f46840ac860 2023-12-19T16:57:22.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468b1df640 1 -- 172.21.15.26:0/4287143476 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46840c3060 con 0x7f46840ac860 2023-12-19T16:57:22.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.897+0000 7f468b1df640 1 -- 172.21.15.26:0/4287143476 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4684061930 con 0x7f46840ac860 2023-12-19T16:57:22.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.901+0000 7f467affd640 1 -- 172.21.15.26:0/4287143476 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f46800059a0 con 0x7f46840ac860 2023-12-19T16:57:22.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.901+0000 7f467affd640 1 --2- 172.21.15.26:0/4287143476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4668065b80 0x7f4668068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:22.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.901+0000 7f467affd640 1 -- 172.21.15.26:0/4287143476 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f4680083960 con 0x7f46840ac860 2023-12-19T16:57:22.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.901+0000 7f46899dc640 1 --2- 172.21.15.26:0/4287143476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4668065b80 0x7f4668068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:22.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.901+0000 7f46899dc640 1 --2- 172.21.15.26:0/4287143476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4668065b80 0x7f4668068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f46840b06c0 tx=0x7f4674007d40 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:22.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:22.905+0000 7f467affd640 1 -- 172.21.15.26:0/4287143476 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4680051020 con 0x7f46840ac860 2023-12-19T16:57:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.093+0000 7f468b1df640 1 -- 172.21.15.26:0/4287143476 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f46840ae840 con 0x7f46840ac860 2023-12-19T16:57:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.105+0000 7f467affd640 1 -- 172.21.15.26:0/4287143476 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v29) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4680054640 con 0x7f46840ac860 2023-12-19T16:57:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 8 and data pool 9 2023-12-19T16:57:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.113+0000 7f468b1df640 1 -- 172.21.15.26:0/4287143476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4668065b80 msgr2=0x7f4668068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.113+0000 7f468b1df640 1 --2- 172.21.15.26:0/4287143476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4668065b80 0x7f4668068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f46840b06c0 tx=0x7f4674007d40 comp rx=0 tx=0).stop 2023-12-19T16:57:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.113+0000 7f468b1df640 1 -- 172.21.15.26:0/4287143476 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46840ac860 msgr2=0x7f46840c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.113+0000 7f468b1df640 1 --2- 172.21.15.26:0/4287143476 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46840ac860 0x7f46840c9410 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f468000e980 tx=0x7f468000ee40 comp rx=0 tx=0).stop 2023-12-19T16:57:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.113+0000 7f468b1df640 1 -- 172.21.15.26:0/4287143476 shutdown_connections 2023-12-19T16:57:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.113+0000 7f468b1df640 1 --2- 172.21.15.26:0/4287143476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f468415bae0 0x7f468415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.113+0000 7f468b1df640 1 --2- 172.21.15.26:0/4287143476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46840ce110 0x7f46840c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.113+0000 7f468b1df640 1 --2- 172.21.15.26:0/4287143476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4668065b80 0x7f4668068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.113+0000 7f468b1df640 1 --2- 172.21.15.26:0/4287143476 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46840ac860 0x7f46840c9410 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.113+0000 7f468b1df640 1 -- 172.21.15.26:0/4287143476 >> 172.21.15.26:0/4287143476 conn(0x7f46840cede0 msgr2=0x7f46840af020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:23.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.117+0000 7f468b1df640 1 -- 172.21.15.26:0/4287143476 shutdown_connections 2023-12-19T16:57:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.117+0000 7f468b1df640 1 -- 172.21.15.26:0/4287143476 wait complete. 2023-12-19T16:57:23.156 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f966ad0b640 1 Processor -- start 2023-12-19T16:57:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f966ad0b640 1 -- start start 2023-12-19T16:57:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f966ad0b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96640b8760 0x7f96640cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f966ad0b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96640cdfe0 con 0x7f96640cdbb0 2023-12-19T16:57:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f966ad0b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96640b6cf0 con 0x7f966415cca0 2023-12-19T16:57:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f966ad0b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96640b7c30 con 0x7f96640b8760 2023-12-19T16:57:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f966a50a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f96640cdbb0 0x7f96640d09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55264/0 (socket says 172.21.15.26:55264) 2023-12-19T16:57:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f966a50a640 1 -- 172.21.15.26:0/2538261594 learned_addr learned my addr 172.21.15.26:0/2538261594 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:23.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f9669508640 1 --2- 172.21.15.26:0/2538261594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96640b8760 0x7f96640cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f9668d07640 1 -- 172.21.15.26:0/2538261594 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2499173189 0 0) 0x7f96640cdfe0 con 0x7f96640cdbb0 2023-12-19T16:57:23.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f9668d07640 1 -- 172.21.15.26:0/2538261594 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9650003530 con 0x7f96640cdbb0 2023-12-19T16:57:23.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f9668d07640 1 -- 172.21.15.26:0/2538261594 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458818934 0 0) 0x7f96640b6cf0 con 0x7f966415cca0 2023-12-19T16:57:23.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f9668d07640 1 -- 172.21.15.26:0/2538261594 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96640cdfe0 con 0x7f966415cca0 2023-12-19T16:57:23.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f9668d07640 1 -- 172.21.15.26:0/2538261594 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 610701213 0 0) 0x7f9650003530 con 0x7f96640cdbb0 2023-12-19T16:57:23.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f9669508640 1 -- 172.21.15.26:0/2538261594 >> v1:172.21.15.160:6789/0 conn(0x7f966415cca0 legacy=0x7f96640b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f9669508640 1 -- 172.21.15.26:0/2538261594 >> v1:172.21.15.26:6789/0 conn(0x7f96640cdbb0 legacy=0x7f96640d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.297+0000 7f9669508640 1 -- 172.21.15.26:0/2538261594 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96640b6fb0 con 0x7f96640b8760 2023-12-19T16:57:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.301+0000 7f9669508640 1 --2- 172.21.15.26:0/2538261594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96640b8760 0x7f96640cd670 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f96540097f0 tx=0x7f965401c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3ff2c28fb5e7443 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.301+0000 7f9668d07640 1 -- 172.21.15.26:0/2538261594 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f965c001a20 con 0x7f96640b8760 2023-12-19T16:57:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.301+0000 7f9668d07640 1 -- 172.21.15.26:0/2538261594 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96540a7050 con 0x7f96640b8760 2023-12-19T16:57:23.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.301+0000 7f9668d07640 1 -- 172.21.15.26:0/2538261594 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540ab600 con 0x7f96640b8760 2023-12-19T16:57:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.301+0000 7f966ad0b640 1 -- 172.21.15.26:0/2538261594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96640b8760 msgr2=0x7f96640cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.301+0000 7f966ad0b640 1 --2- 172.21.15.26:0/2538261594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96640b8760 0x7f96640cd670 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f96540097f0 tx=0x7f965401c3b0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.301+0000 7f966ad0b640 1 -- 172.21.15.26:0/2538261594 shutdown_connections 2023-12-19T16:57:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.301+0000 7f966ad0b640 1 --2- 172.21.15.26:0/2538261594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96640b8760 0x7f96640cd670 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.301+0000 7f966ad0b640 1 -- 172.21.15.26:0/2538261594 >> 172.21.15.26:0/2538261594 conn(0x7f966406a120 msgr2=0x7f966406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.301+0000 7f966ad0b640 1 -- 172.21.15.26:0/2538261594 shutdown_connections 2023-12-19T16:57:23.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.301+0000 7f966ad0b640 1 -- 172.21.15.26:0/2538261594 wait complete. 2023-12-19T16:57:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f966ad0b640 1 Processor -- start 2023-12-19T16:57:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f966ad0b640 1 -- start start 2023-12-19T16:57:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f966ad0b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96640b8760 0x7f96641f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f966ad0b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96640cdbb0 0x7f96641539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f966ad0b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f966415cca0 0x7f96640b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f966ad0b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9664159af0 con 0x7f96640cdbb0 2023-12-19T16:57:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f966ad0b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96641f5380 con 0x7f966415cca0 2023-12-19T16:57:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f966ad0b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96641f5660 con 0x7f96640b8760 2023-12-19T16:57:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f9669508640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96640cdbb0 0x7f96641539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f9669508640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96640cdbb0 0x7f96641539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58944/0 (socket says 172.21.15.26:58944) 2023-12-19T16:57:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f9669508640 1 -- 172.21.15.26:0/4281213523 learned_addr learned my addr 172.21.15.26:0/4281213523 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f9669d09640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96640b8760 0x7f96641f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f966a50a640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f966415cca0 0x7f96640b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f9669508640 1 -- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96640b8760 msgr2=0x7f96641f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f9669508640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96640b8760 0x7f96641f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f9669508640 1 -- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f966415cca0 msgr2=0x7f96640b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f9669508640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f966415cca0 0x7f96640b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f9669508640 1 -- 172.21.15.26:0/4281213523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9654009420 con 0x7f96640cdbb0 2023-12-19T16:57:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f966a50a640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f966415cca0 0x7f96640b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f9669d09640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96640b8760 0x7f96641f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f9669508640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96640cdbb0 0x7f96641539f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9654009740 tx=0x7f9654019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f965a7fc640 1 -- 172.21.15.26:0/4281213523 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96540bb070 con 0x7f96640cdbb0 2023-12-19T16:57:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f965a7fc640 1 -- 172.21.15.26:0/4281213523 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9654003830 con 0x7f96640cdbb0 2023-12-19T16:57:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f965a7fc640 1 -- 172.21.15.26:0/4281213523 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9654003ba0 con 0x7f96640cdbb0 2023-12-19T16:57:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f966ad0b640 1 -- 172.21.15.26:0/4281213523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96641f5940 con 0x7f96640cdbb0 2023-12-19T16:57:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.305+0000 7f966ad0b640 1 -- 172.21.15.26:0/4281213523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9664153f30 con 0x7f96640cdbb0 2023-12-19T16:57:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.309+0000 7f966ad0b640 1 -- 172.21.15.26:0/4281213523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9664159c60 con 0x7f96640cdbb0 2023-12-19T16:57:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.309+0000 7f965a7fc640 1 -- 172.21.15.26:0/4281213523 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f96540b2050 con 0x7f96640cdbb0 2023-12-19T16:57:23.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.309+0000 7f965a7fc640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9648065b80 0x7f9648068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.309+0000 7f965a7fc640 1 -- 172.21.15.26:0/4281213523 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f965411db90 con 0x7f96640cdbb0 2023-12-19T16:57:23.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.309+0000 7f9669d09640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9648065b80 0x7f9648068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.309+0000 7f9669d09640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9648065b80 0x7f9648068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f96600011f0 tx=0x7f9660009040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:23.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.313+0000 7f965a7fc640 1 -- 172.21.15.26:0/4281213523 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f96540eb030 con 0x7f96640cdbb0 2023-12-19T16:57:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.469+0000 7f966ad0b640 1 -- 172.21.15.26:0/4281213523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9664159c30 con 0x7f96640cdbb0 2023-12-19T16:57:23.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.473+0000 7f965a7fc640 1 -- 172.21.15.26:0/4281213523 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17215 (secure 0 0 0) 0x7f96540b0090 con 0x7f96640cdbb0 2023-12-19T16:57:23.477 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:23.477 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":78,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:23.100388+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:20.751846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:22.096742+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.473+0000 7f966ad0b640 1 -- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9648065b80 msgr2=0x7f9648068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.473+0000 7f966ad0b640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9648065b80 0x7f9648068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f96600011f0 tx=0x7f9660009040 comp rx=0 tx=0).stop 2023-12-19T16:57:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.473+0000 7f966ad0b640 1 -- 172.21.15.26:0/4281213523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96640cdbb0 msgr2=0x7f96641539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.473+0000 7f966ad0b640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96640cdbb0 0x7f96641539f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9654009740 tx=0x7f9654019150 comp rx=0 tx=0).stop 2023-12-19T16:57:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.473+0000 7f966ad0b640 1 -- 172.21.15.26:0/4281213523 shutdown_connections 2023-12-19T16:57:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.473+0000 7f966ad0b640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f966415cca0 0x7f96640b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.473+0000 7f966ad0b640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96640cdbb0 0x7f96641539f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.473+0000 7f966ad0b640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9648065b80 0x7f9648068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.473+0000 7f966ad0b640 1 --2- 172.21.15.26:0/4281213523 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96640b8760 0x7f96641f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.473+0000 7f966ad0b640 1 -- 172.21.15.26:0/4281213523 >> 172.21.15.26:0/4281213523 conn(0x7f966406a120 msgr2=0x7f96640b4050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.473+0000 7f966ad0b640 1 -- 172.21.15.26:0/4281213523 shutdown_connections 2023-12-19T16:57:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.473+0000 7f966ad0b640 1 -- 172.21.15.26:0/4281213523 wait complete. 2023-12-19T16:57:23.499 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ada08c640 1 Processor -- start 2023-12-19T16:57:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ada08c640 1 -- start start 2023-12-19T16:57:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ada08c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ad415c7f0 0x7f4ad406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ada08c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ad40b30d0 0x7f4ad40b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ada08c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad40b3a60 con 0x7f4ad40adb80 2023-12-19T16:57:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ada08c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad406d770 con 0x7f4ad415c7f0 2023-12-19T16:57:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ada08c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad406ced0 con 0x7f4ad40b30d0 2023-12-19T16:57:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ad37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ad415c7f0 0x7f4ad406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ad2ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4ad40adb80 0x7f4ad40cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55276/0 (socket says 172.21.15.26:55276) 2023-12-19T16:57:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ad2ffd640 1 -- 172.21.15.26:0/2816381439 learned_addr learned my addr 172.21.15.26:0/2816381439 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ad3fff640 1 --2- 172.21.15.26:0/2816381439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ad40b30d0 0x7f4ad40b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ad37fe640 1 -- 172.21.15.26:0/2816381439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ad40b30d0 msgr2=0x7f4ad40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ad37fe640 1 --2- 172.21.15.26:0/2816381439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ad40b30d0 0x7f4ad40b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ad37fe640 1 -- 172.21.15.26:0/2816381439 >> v1:172.21.15.26:6789/0 conn(0x7f4ad40adb80 legacy=0x7f4ad40cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ad37fe640 1 -- 172.21.15.26:0/2816381439 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad406d1b0 con 0x7f4ad415c7f0 2023-12-19T16:57:23.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.637+0000 7f4ad27fc640 1 -- 172.21.15.26:0/2816381439 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200100420 0 0) 0x7f4ad40b3a60 con 0x7f4ad40adb80 2023-12-19T16:57:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.641+0000 7f4ad37fe640 1 --2- 172.21.15.26:0/2816381439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ad415c7f0 0x7f4ad406c7f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40097c0 tx=0x7f4ac401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47c714399f1b275 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.641+0000 7f4ad27fc640 1 -- 172.21.15.26:0/2816381439 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac40ab600 con 0x7f4ad415c7f0 2023-12-19T16:57:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.641+0000 7f4ad27fc640 1 -- 172.21.15.26:0/2816381439 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac40a7050 con 0x7f4ad415c7f0 2023-12-19T16:57:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.641+0000 7f4ad27fc640 1 -- 172.21.15.26:0/2816381439 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac40ab980 con 0x7f4ad415c7f0 2023-12-19T16:57:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.641+0000 7f4ada08c640 1 -- 172.21.15.26:0/2816381439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ad415c7f0 msgr2=0x7f4ad406c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.641+0000 7f4ada08c640 1 --2- 172.21.15.26:0/2816381439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ad415c7f0 0x7f4ad406c7f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40097c0 tx=0x7f4ac401c480 comp rx=0 tx=0).stop 2023-12-19T16:57:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.641+0000 7f4ada08c640 1 -- 172.21.15.26:0/2816381439 shutdown_connections 2023-12-19T16:57:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.641+0000 7f4ada08c640 1 --2- 172.21.15.26:0/2816381439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ad40b30d0 0x7f4ad40b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.641+0000 7f4ada08c640 1 --2- 172.21.15.26:0/2816381439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ad415c7f0 0x7f4ad406c7f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.641+0000 7f4ada08c640 1 -- 172.21.15.26:0/2816381439 >> 172.21.15.26:0/2816381439 conn(0x7f4ad40658e0 msgr2=0x7f4ad4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.641+0000 7f4ada08c640 1 -- 172.21.15.26:0/2816381439 shutdown_connections 2023-12-19T16:57:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.641+0000 7f4ada08c640 1 -- 172.21.15.26:0/2816381439 wait complete. 2023-12-19T16:57:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ada08c640 1 Processor -- start 2023-12-19T16:57:23.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ada08c640 1 -- start start 2023-12-19T16:57:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ada08c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ad40adb80 0x7f4ad4155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ada08c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ad40b30d0 0x7f4ad41fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ada08c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ad415c7f0 0x7f4ad41f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ada08c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad41f9b30 con 0x7f4ad415c7f0 2023-12-19T16:57:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ada08c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad41f97b0 con 0x7f4ad40adb80 2023-12-19T16:57:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ada08c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad41fd2e0 con 0x7f4ad40b30d0 2023-12-19T16:57:23.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ad3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ad415c7f0 0x7f4ad41f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ad3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ad415c7f0 0x7f4ad41f9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58960/0 (socket says 172.21.15.26:58960) 2023-12-19T16:57:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ad3fff640 1 -- 172.21.15.26:0/656327895 learned_addr learned my addr 172.21.15.26:0/656327895 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ad2ffd640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ad40b30d0 0x7f4ad41fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ad37fe640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ad40adb80 0x7f4ad4155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ad3fff640 1 -- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ad40b30d0 msgr2=0x7f4ad41fcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ad3fff640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ad40b30d0 0x7f4ad41fcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ad3fff640 1 -- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ad40adb80 msgr2=0x7f4ad4155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ad3fff640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ad40adb80 0x7f4ad4155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ad3fff640 1 -- 172.21.15.26:0/656327895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac4009420 con 0x7f4ad415c7f0 2023-12-19T16:57:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ad2ffd640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ad40b30d0 0x7f4ad41fcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ad37fe640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ad40adb80 0x7f4ad4155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ad3fff640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ad415c7f0 0x7f4ad41f9f50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4acc007e40 tx=0x7f4acc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4abbfff640 1 -- 172.21.15.26:0/656327895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc014620 con 0x7f4ad415c7f0 2023-12-19T16:57:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4abbfff640 1 -- 172.21.15.26:0/656327895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4acc00ee90 con 0x7f4ad415c7f0 2023-12-19T16:57:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ada08c640 1 -- 172.21.15.26:0/656327895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad41fd560 con 0x7f4ad415c7f0 2023-12-19T16:57:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4abbfff640 1 -- 172.21.15.26:0/656327895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4acc0149e0 con 0x7f4ad415c7f0 2023-12-19T16:57:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.645+0000 7f4ada08c640 1 -- 172.21.15.26:0/656327895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad41fdb20 con 0x7f4ad415c7f0 2023-12-19T16:57:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.649+0000 7f4ada08c640 1 -- 172.21.15.26:0/656327895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad41fdd40 con 0x7f4ad415c7f0 2023-12-19T16:57:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.649+0000 7f4abbfff640 1 -- 172.21.15.26:0/656327895 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4acc014b40 con 0x7f4ad415c7f0 2023-12-19T16:57:23.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.649+0000 7f4abbfff640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4ac0065c00 0x7f4ac00680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.649+0000 7f4abbfff640 1 -- 172.21.15.26:0/656327895 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f4acc083eb0 con 0x7f4ad415c7f0 2023-12-19T16:57:23.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.649+0000 7f4ad37fe640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4ac0065c00 0x7f4ac00680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.649+0000 7f4ad37fe640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4ac0065c00 0x7f4ac00680c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40ad000 tx=0x7f4ac40a6000 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:23.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.653+0000 7f4abbfff640 1 -- 172.21.15.26:0/656327895 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4acc0514f0 con 0x7f4ad415c7f0 2023-12-19T16:57:23.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.813+0000 7f4ada08c640 1 -- 172.21.15.26:0/656327895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ad41f9c70 con 0x7f4ad415c7f0 2023-12-19T16:57:23.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.813+0000 7f4abbfff640 1 -- 172.21.15.26:0/656327895 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+17215 (secure 0 0 0) 0x7f4ad41f9c70 con 0x7f4ad415c7f0 2023-12-19T16:57:23.818 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:23.819 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":78,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:23.100388+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:20.751846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:22.096742+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:23.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.817+0000 7f4ada08c640 1 -- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4ac0065c00 msgr2=0x7f4ac00680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.817+0000 7f4ada08c640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4ac0065c00 0x7f4ac00680c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40ad000 tx=0x7f4ac40a6000 comp rx=0 tx=0).stop 2023-12-19T16:57:23.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.817+0000 7f4ada08c640 1 -- 172.21.15.26:0/656327895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ad415c7f0 msgr2=0x7f4ad41f9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.817+0000 7f4ada08c640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ad415c7f0 0x7f4ad41f9f50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4acc007e40 tx=0x7f4acc00e4b0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.817+0000 7f4ada08c640 1 -- 172.21.15.26:0/656327895 shutdown_connections 2023-12-19T16:57:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.817+0000 7f4ada08c640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ad415c7f0 0x7f4ad41f9f50 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.817+0000 7f4ada08c640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4ac0065c00 0x7f4ac00680c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.817+0000 7f4ada08c640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4ad40b30d0 0x7f4ad41fcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.817+0000 7f4ada08c640 1 --2- 172.21.15.26:0/656327895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ad40adb80 0x7f4ad4155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.817+0000 7f4ada08c640 1 -- 172.21.15.26:0/656327895 >> 172.21.15.26:0/656327895 conn(0x7f4ad40658e0 msgr2=0x7f4ad41507a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.817+0000 7f4ada08c640 1 -- 172.21.15.26:0/656327895 shutdown_connections 2023-12-19T16:57:23.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.817+0000 7f4ada08c640 1 -- 172.21.15.26:0/656327895 wait complete. 2023-12-19T16:57:23.840 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T16:57:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.981+0000 7f61b3866640 1 Processor -- start 2023-12-19T16:57:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.981+0000 7f61b3866640 1 -- start start 2023-12-19T16:57:23.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.981+0000 7f61b3866640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61ac161230 0x7f61ac0ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.981+0000 7f61b3866640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61ac06c3f0 0x7f61ac06c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.981+0000 7f61b3866640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f61ac164e50 0x7f61ac1652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.981+0000 7f61b3866640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ac0b01b0 con 0x7f61ac161230 2023-12-19T16:57:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.981+0000 7f61b3866640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ac0adfb0 con 0x7f61ac164e50 2023-12-19T16:57:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.981+0000 7f61b3866640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0ca4e0 con 0x7f61ac06c3f0 2023-12-19T16:57:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.981+0000 7f61b1602640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61ac161230 0x7f61ac0ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.981+0000 7f61b1602640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61ac161230 0x7f61ac0ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58966/0 (socket says 172.21.15.26:58966) 2023-12-19T16:57:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.981+0000 7f61b1602640 1 -- 172.21.15.26:0/3662224231 learned_addr learned my addr 172.21.15.26:0/3662224231 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.985+0000 7f61b1e03640 1 --2- 172.21.15.26:0/3662224231 >> v2:172.21.15.160:3300/0 conn(0x7f61ac164e50 0x7f61ac1652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.985+0000 7f61b0e01640 1 --2- 172.21.15.26:0/3662224231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61ac06c3f0 0x7f61ac06c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.985+0000 7f61b1602640 1 -- 172.21.15.26:0/3662224231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61ac06c3f0 msgr2=0x7f61ac06c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.985+0000 7f61b1602640 1 --2- 172.21.15.26:0/3662224231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61ac06c3f0 0x7f61ac06c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.985+0000 7f61b1602640 1 -- 172.21.15.26:0/3662224231 >> v2:172.21.15.160:3300/0 conn(0x7f61ac164e50 msgr2=0x7f61ac1652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.985+0000 7f61b1602640 1 --2- 172.21.15.26:0/3662224231 >> v2:172.21.15.160:3300/0 conn(0x7f61ac164e50 0x7f61ac1652e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.985+0000 7f61b1602640 1 -- 172.21.15.26:0/3662224231 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac0b0400 con 0x7f61ac161230 2023-12-19T16:57:23.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.985+0000 7f61b1602640 1 --2- 172.21.15.26:0/3662224231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61ac161230 0x7f61ac0ada70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0ca090 tx=0x7f61a801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f98a54f3ec3a56 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:23.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.985+0000 7f61a3fff640 1 -- 172.21.15.26:0/3662224231 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a80ab600 con 0x7f61ac161230 2023-12-19T16:57:23.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.985+0000 7f61a3fff640 1 -- 172.21.15.26:0/3662224231 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac0c9e00 con 0x7f61ac161230 2023-12-19T16:57:23.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.985+0000 7f61a3fff640 1 -- 172.21.15.26:0/3662224231 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a80ab980 con 0x7f61ac161230 2023-12-19T16:57:23.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 -- 172.21.15.26:0/3662224231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61ac161230 msgr2=0x7f61ac0ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 --2- 172.21.15.26:0/3662224231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61ac161230 0x7f61ac0ada70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0ca090 tx=0x7f61a801c440 comp rx=0 tx=0).stop 2023-12-19T16:57:23.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 -- 172.21.15.26:0/3662224231 shutdown_connections 2023-12-19T16:57:23.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 --2- 172.21.15.26:0/3662224231 >> v2:172.21.15.160:3300/0 conn(0x7f61ac164e50 0x7f61ac1652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 --2- 172.21.15.26:0/3662224231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61ac06c3f0 0x7f61ac06c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 --2- 172.21.15.26:0/3662224231 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61ac161230 0x7f61ac0ada70 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 -- 172.21.15.26:0/3662224231 >> 172.21.15.26:0/3662224231 conn(0x7f61ac0ce730 msgr2=0x7f61ac0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:23.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 -- 172.21.15.26:0/3662224231 shutdown_connections 2023-12-19T16:57:23.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 -- 172.21.15.26:0/3662224231 wait complete. 2023-12-19T16:57:23.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 Processor -- start 2023-12-19T16:57:23.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 -- start start 2023-12-19T16:57:23.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61ac06c3f0 0x7f61ac15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61ac161230 0x7f61ac152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61ac164e50 0x7f61ac1541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0c4980 con 0x7f61ac161230 2023-12-19T16:57:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0c2f80 con 0x7f61ac06c3f0 2023-12-19T16:57:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b3866640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ac0c4c20 con 0x7f61ac164e50 2023-12-19T16:57:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b0e01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61ac161230 0x7f61ac152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b0e01640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61ac161230 0x7f61ac152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58972/0 (socket says 172.21.15.26:58972) 2023-12-19T16:57:23.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b0e01640 1 -- 172.21.15.26:0/1651125690 learned_addr learned my addr 172.21.15.26:0/1651125690 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:23.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b1e03640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61ac164e50 0x7f61ac1541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b1602640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61ac06c3f0 0x7f61ac15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b0e01640 1 -- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61ac164e50 msgr2=0x7f61ac1541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.989+0000 7f61b0e01640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61ac164e50 0x7f61ac1541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61b0e01640 1 -- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61ac06c3f0 msgr2=0x7f61ac15aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61b0e01640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61ac06c3f0 0x7f61ac15aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61b0e01640 1 -- 172.21.15.26:0/1651125690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a80094d0 con 0x7f61ac161230 2023-12-19T16:57:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61b1e03640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61ac164e50 0x7f61ac1541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61b1602640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61ac06c3f0 0x7f61ac15aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61b0e01640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61ac161230 0x7f61ac152d70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e870 tx=0x7f619c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61a1ffb640 1 -- 172.21.15.26:0/1651125690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c009510 con 0x7f61ac161230 2023-12-19T16:57:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61a1ffb640 1 -- 172.21.15.26:0/1651125690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c00eec0 con 0x7f61ac161230 2023-12-19T16:57:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61a1ffb640 1 -- 172.21.15.26:0/1651125690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c00cb40 con 0x7f61ac161230 2023-12-19T16:57:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61b3866640 1 -- 172.21.15.26:0/1651125690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ac0c4f00 con 0x7f61ac161230 2023-12-19T16:57:23.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61b3866640 1 -- 172.21.15.26:0/1651125690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ac0c2be0 con 0x7f61ac161230 2023-12-19T16:57:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61b3866640 1 -- 172.21.15.26:0/1651125690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ac0c47e0 con 0x7f61ac161230 2023-12-19T16:57:23.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61a1ffb640 1 -- 172.21.15.26:0/1651125690 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f619c00cca0 con 0x7f61ac161230 2023-12-19T16:57:24.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.993+0000 7f61a1ffb640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6190065bb0 0x7f6190068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:24.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.997+0000 7f61a1ffb640 1 -- 172.21.15.26:0/1651125690 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f619c0836c0 con 0x7f61ac161230 2023-12-19T16:57:24.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.997+0000 7f61b1602640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6190065bb0 0x7f6190068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:24.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.997+0000 7f61b1602640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6190065bb0 0x7f6190068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f61a80af000 tx=0x7f61a8008040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:24.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:23.997+0000 7f61a1ffb640 1 -- 172.21.15.26:0/1651125690 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f61ac0c47e0 con 0x7f61ac161230 2023-12-19T16:57:24.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:24.189+0000 7f61b3866640 1 -- 172.21.15.26:0/1651125690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f61ac0c47e0 con 0x7f61ac161230 2023-12-19T16:57:25.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.117+0000 7f61a1ffb640 1 -- 172.21.15.26:0/1651125690 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7f61ac0c47e0 con 0x7f61ac161230 2023-12-19T16:57:25.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.117+0000 7f61b3866640 1 -- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6190065bb0 msgr2=0x7f6190068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:25.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.117+0000 7f61b3866640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6190065bb0 0x7f6190068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f61a80af000 tx=0x7f61a8008040 comp rx=0 tx=0).stop 2023-12-19T16:57:25.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.117+0000 7f61b3866640 1 -- 172.21.15.26:0/1651125690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61ac161230 msgr2=0x7f61ac152d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:25.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.117+0000 7f61b3866640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61ac161230 0x7f61ac152d70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e870 tx=0x7f619c00ec70 comp rx=0 tx=0).stop 2023-12-19T16:57:25.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.117+0000 7f61b3866640 1 -- 172.21.15.26:0/1651125690 shutdown_connections 2023-12-19T16:57:25.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.117+0000 7f61b3866640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6190065bb0 0x7f6190068070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:25.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.117+0000 7f61b3866640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61ac164e50 0x7f61ac1541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:25.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.117+0000 7f61b3866640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61ac161230 0x7f61ac152d70 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:25.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.117+0000 7f61b3866640 1 --2- 172.21.15.26:0/1651125690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61ac06c3f0 0x7f61ac15aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:25.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.117+0000 7f61b3866640 1 -- 172.21.15.26:0/1651125690 >> 172.21.15.26:0/1651125690 conn(0x7f61ac0ce730 msgr2=0x7f61ac06f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:25.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.117+0000 7f61b3866640 1 -- 172.21.15.26:0/1651125690 shutdown_connections 2023-12-19T16:57:25.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.117+0000 7f61b3866640 1 -- 172.21.15.26:0/1651125690 wait complete. 2023-12-19T16:57:25.147 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T16:57:25.147 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T16:57:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.293+0000 7f70e678f640 1 Processor -- start 2023-12-19T16:57:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.293+0000 7f70e678f640 1 -- start start 2023-12-19T16:57:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.293+0000 7f70e678f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f70e006b110 0x7f70e0070030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.293+0000 7f70e678f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e006f490 con 0x7f70e006ac80 2023-12-19T16:57:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.293+0000 7f70e678f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e0159520 con 0x7f70e0159140 2023-12-19T16:57:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.293+0000 7f70e678f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e00cd690 con 0x7f70e006b110 2023-12-19T16:57:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.293+0000 7f70e4d2c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f70e006b110 0x7f70e0070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.293+0000 7f70df7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f70e006ac80 0x7f70e006f920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55278/0 (socket says 172.21.15.26:55278) 2023-12-19T16:57:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.293+0000 7f70df7fe640 1 -- 172.21.15.26:0/2561593957 learned_addr learned my addr 172.21.15.26:0/2561593957 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.293+0000 7f70e4d2c640 1 -- 172.21.15.26:0/2561593957 >> v1:172.21.15.160:6789/0 conn(0x7f70e0159140 legacy=0x7f70e00ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.293+0000 7f70e4d2c640 1 -- 172.21.15.26:0/2561593957 >> v1:172.21.15.26:6789/0 conn(0x7f70e006ac80 legacy=0x7f70e006f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.293+0000 7f70e4d2c640 1 -- 172.21.15.26:0/2561593957 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e00cd970 con 0x7f70e006b110 2023-12-19T16:57:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.293+0000 7f70deffd640 1 -- 172.21.15.26:0/2561593957 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818972497 0 0) 0x7f70e006f490 con 0x7f70e006ac80 2023-12-19T16:57:25.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.297+0000 7f70e4d2c640 1 --2- 172.21.15.26:0/2561593957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e006b110 0x7f70e0070030 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009d90 tx=0x7f70d801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5786dfb95179f880 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.297+0000 7f70deffd640 1 -- 172.21.15.26:0/2561593957 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d80a59e0 con 0x7f70e006b110 2023-12-19T16:57:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.297+0000 7f70deffd640 1 -- 172.21.15.26:0/2561593957 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d80a5b40 con 0x7f70e006b110 2023-12-19T16:57:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.297+0000 7f70deffd640 1 -- 172.21.15.26:0/2561593957 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d80ad440 con 0x7f70e006b110 2023-12-19T16:57:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.297+0000 7f70e678f640 1 -- 172.21.15.26:0/2561593957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e006b110 msgr2=0x7f70e0070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.297+0000 7f70e678f640 1 --2- 172.21.15.26:0/2561593957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e006b110 0x7f70e0070030 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009d90 tx=0x7f70d801c920 comp rx=0 tx=0).stop 2023-12-19T16:57:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.297+0000 7f70e678f640 1 -- 172.21.15.26:0/2561593957 shutdown_connections 2023-12-19T16:57:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.297+0000 7f70e678f640 1 --2- 172.21.15.26:0/2561593957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e006b110 0x7f70e0070030 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.297+0000 7f70e678f640 1 -- 172.21.15.26:0/2561593957 >> 172.21.15.26:0/2561593957 conn(0x7f70e00ce730 msgr2=0x7f70e00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.297+0000 7f70e678f640 1 -- 172.21.15.26:0/2561593957 shutdown_connections 2023-12-19T16:57:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.297+0000 7f70e678f640 1 -- 172.21.15.26:0/2561593957 wait complete. 2023-12-19T16:57:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70e678f640 1 Processor -- start 2023-12-19T16:57:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70e678f640 1 -- start start 2023-12-19T16:57:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70e678f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e006ac80 0x7f70e00c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70e678f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e006b110 0x7f70e0151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70e678f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70e0159140 0x7f70e00c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70e678f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00c86d0 con 0x7f70e006ac80 2023-12-19T16:57:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70e678f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00c83f0 con 0x7f70e006b110 2023-12-19T16:57:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70e678f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00b21b0 con 0x7f70e0159140 2023-12-19T16:57:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70dffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e006ac80 0x7f70e00c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70dffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e006ac80 0x7f70e00c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58978/0 (socket says 172.21.15.26:58978) 2023-12-19T16:57:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70dffff640 1 -- 172.21.15.26:0/3603913852 learned_addr learned my addr 172.21.15.26:0/3603913852 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70df7fe640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e006b110 0x7f70e0151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70e4d2c640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70e0159140 0x7f70e00c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70dffff640 1 -- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70e0159140 msgr2=0x7f70e00c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70dffff640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70e0159140 0x7f70e00c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70dffff640 1 -- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e006b110 msgr2=0x7f70e0151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70dffff640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e006b110 0x7f70e0151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70dffff640 1 -- 172.21.15.26:0/3603913852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d8009a70 con 0x7f70e006ac80 2023-12-19T16:57:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70df7fe640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e006b110 0x7f70e0151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70e4d2c640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70e0159140 0x7f70e00c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70dffff640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e006ac80 0x7f70e00c7df0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f70d000c3c0 tx=0x7f70d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:25.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70dcff9640 1 -- 172.21.15.26:0/3603913852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d00052d0 con 0x7f70e006ac80 2023-12-19T16:57:25.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70dcff9640 1 -- 172.21.15.26:0/3603913852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d0005430 con 0x7f70e006ac80 2023-12-19T16:57:25.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70dcff9640 1 -- 172.21.15.26:0/3603913852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0003840 con 0x7f70e006ac80 2023-12-19T16:57:25.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70e678f640 1 -- 172.21.15.26:0/3603913852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e00b2490 con 0x7f70e006ac80 2023-12-19T16:57:25.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.301+0000 7f70e678f640 1 -- 172.21.15.26:0/3603913852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e00c21d0 con 0x7f70e006ac80 2023-12-19T16:57:25.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.305+0000 7f70e678f640 1 -- 172.21.15.26:0/3603913852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e00ce240 con 0x7f70e006ac80 2023-12-19T16:57:25.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.305+0000 7f70dcff9640 1 -- 172.21.15.26:0/3603913852 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f70d00039a0 con 0x7f70e006ac80 2023-12-19T16:57:25.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.305+0000 7f70dcff9640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f70cc065b80 0x7f70cc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:25.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.305+0000 7f70dcff9640 1 -- 172.21.15.26:0/3603913852 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f70d0082bd0 con 0x7f70e006ac80 2023-12-19T16:57:25.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.305+0000 7f70df7fe640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f70cc065b80 0x7f70cc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:25.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.309+0000 7f70df7fe640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f70cc065b80 0x7f70cc068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f70e01f67d0 tx=0x7f70d4007c60 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:25.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.309+0000 7f70dcff9640 1 -- 172.21.15.26:0/3603913852 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f70d0050100 con 0x7f70e006ac80 2023-12-19T16:57:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:25.497+0000 7f70e678f640 1 -- 172.21.15.26:0/3603913852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f70e0074270 con 0x7f70e006ac80 2023-12-19T16:57:26.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.117+0000 7f70dcff9640 1 -- 172.21.15.26:0/3603913852 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7f70d00170c0 con 0x7f70e006ac80 2023-12-19T16:57:26.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.121+0000 7f70e678f640 1 -- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f70cc065b80 msgr2=0x7f70cc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:26.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.121+0000 7f70e678f640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f70cc065b80 0x7f70cc068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f70e01f67d0 tx=0x7f70d4007c60 comp rx=0 tx=0).stop 2023-12-19T16:57:26.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.121+0000 7f70e678f640 1 -- 172.21.15.26:0/3603913852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e006ac80 msgr2=0x7f70e00c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:26.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.121+0000 7f70e678f640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e006ac80 0x7f70e00c7df0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f70d000c3c0 tx=0x7f70d000c880 comp rx=0 tx=0).stop 2023-12-19T16:57:26.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.121+0000 7f70e678f640 1 -- 172.21.15.26:0/3603913852 shutdown_connections 2023-12-19T16:57:26.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.121+0000 7f70e678f640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f70cc065b80 0x7f70cc068040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.121+0000 7f70e678f640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70e0159140 0x7f70e00c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.121+0000 7f70e678f640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70e006b110 0x7f70e0151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.121+0000 7f70e678f640 1 --2- 172.21.15.26:0/3603913852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70e006ac80 0x7f70e00c7df0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.121+0000 7f70e678f640 1 -- 172.21.15.26:0/3603913852 >> 172.21.15.26:0/3603913852 conn(0x7f70e00ce730 msgr2=0x7f70e00c8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:26.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.121+0000 7f70e678f640 1 -- 172.21.15.26:0/3603913852 shutdown_connections 2023-12-19T16:57:26.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.121+0000 7f70e678f640 1 -- 172.21.15.26:0/3603913852 wait complete. 2023-12-19T16:57:26.148 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T16:57:26.149 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:57:26.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.289+0000 7fb43968c640 1 Processor -- start 2023-12-19T16:57:26.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.293+0000 7fb43968c640 1 -- start start 2023-12-19T16:57:26.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.293+0000 7fb43968c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb43415c7f0 0x7fb43406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:26.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.293+0000 7fb43968c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4340adb80 0x7fb4340cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:26.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.293+0000 7fb43968c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb43406ced0 con 0x7fb4340ccc20 2023-12-19T16:57:26.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.293+0000 7fb43968c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43406d770 con 0x7fb43415c7f0 2023-12-19T16:57:26.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.293+0000 7fb43968c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43406d1b0 con 0x7fb4340adb80 2023-12-19T16:57:26.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.293+0000 7fb432ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb43415c7f0 0x7fb43406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:26.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.293+0000 7fb432ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb43415c7f0 0x7fb43406c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52784/0 (socket says 172.21.15.26:52784) 2023-12-19T16:57:26.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.293+0000 7fb432ffd640 1 -- 172.21.15.26:0/56950186 learned_addr learned my addr 172.21.15.26:0/56950186 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:26.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.293+0000 7fb432ffd640 1 -- 172.21.15.26:0/56950186 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4340adb80 msgr2=0x7fb4340cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:26.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.293+0000 7fb432ffd640 1 --2- 172.21.15.26:0/56950186 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4340adb80 0x7fb4340cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.293+0000 7fb432ffd640 1 -- 172.21.15.26:0/56950186 >> v1:172.21.15.160:6789/0 conn(0x7fb4340ccc20 legacy=0x7fb4340b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:26.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.293+0000 7fb432ffd640 1 -- 172.21.15.26:0/56950186 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43406d490 con 0x7fb43415c7f0 2023-12-19T16:57:26.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb432ffd640 1 --2- 172.21.15.26:0/56950186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43415c7f0 0x7fb43406c7f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb4240098a0 tx=0x7fb42401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893ce8e6d6cb9e71 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:26.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb4327fc640 1 -- 172.21.15.26:0/56950186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4240ab600 con 0x7fb43415c7f0 2023-12-19T16:57:26.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb4327fc640 1 -- 172.21.15.26:0/56950186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4240a7050 con 0x7fb43415c7f0 2023-12-19T16:57:26.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb4327fc640 1 -- 172.21.15.26:0/56950186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4240ab980 con 0x7fb43415c7f0 2023-12-19T16:57:26.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb43968c640 1 -- 172.21.15.26:0/56950186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43415c7f0 msgr2=0x7fb43406c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:26.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb43968c640 1 --2- 172.21.15.26:0/56950186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43415c7f0 0x7fb43406c7f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb4240098a0 tx=0x7fb42401c4e0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb43968c640 1 -- 172.21.15.26:0/56950186 shutdown_connections 2023-12-19T16:57:26.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb43968c640 1 --2- 172.21.15.26:0/56950186 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4340adb80 0x7fb4340cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb43968c640 1 --2- 172.21.15.26:0/56950186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb43415c7f0 0x7fb43406c7f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb43968c640 1 -- 172.21.15.26:0/56950186 >> 172.21.15.26:0/56950186 conn(0x7fb4340658e0 msgr2=0x7fb434065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:26.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb43968c640 1 -- 172.21.15.26:0/56950186 shutdown_connections 2023-12-19T16:57:26.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb43968c640 1 -- 172.21.15.26:0/56950186 wait complete. 2023-12-19T16:57:26.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb43968c640 1 Processor -- start 2023-12-19T16:57:26.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.297+0000 7fb43968c640 1 -- start start 2023-12-19T16:57:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb43968c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4340adb80 0x7fb43414dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb43968c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4340ccc20 0x7fb434149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb43968c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb43415c7f0 0x7fb4341559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb43968c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb434154630 con 0x7fb4340ccc20 2023-12-19T16:57:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb43968c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb434154910 con 0x7fb4340adb80 2023-12-19T16:57:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb43968c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb434154bf0 con 0x7fb43415c7f0 2023-12-19T16:57:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb42bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4340ccc20 0x7fb434149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:26.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb42bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4340ccc20 0x7fb434149bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52796/0 (socket says 172.21.15.26:52796) 2023-12-19T16:57:26.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb42bfff640 1 -- 172.21.15.26:0/2379337025 learned_addr learned my addr 172.21.15.26:0/2379337025 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:26.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb4337fe640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb43415c7f0 0x7fb4341559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:26.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb432ffd640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4340adb80 0x7fb43414dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:26.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb42bfff640 1 -- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb43415c7f0 msgr2=0x7fb4341559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:26.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb42bfff640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb43415c7f0 0x7fb4341559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb42bfff640 1 -- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4340adb80 msgr2=0x7fb43414dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:26.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb42bfff640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4340adb80 0x7fb43414dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb42bfff640 1 -- 172.21.15.26:0/2379337025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4240094d0 con 0x7fb4340ccc20 2023-12-19T16:57:26.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb432ffd640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4340adb80 0x7fb43414dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:26.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb42bfff640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4340ccc20 0x7fb434149bf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00b550 tx=0x7fb41c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:26.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb42b7fe640 1 -- 172.21.15.26:0/2379337025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c010870 con 0x7fb4340ccc20 2023-12-19T16:57:26.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb42b7fe640 1 -- 172.21.15.26:0/2379337025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c0109f0 con 0x7fb4340ccc20 2023-12-19T16:57:26.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb42b7fe640 1 -- 172.21.15.26:0/2379337025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c010da0 con 0x7fb4340ccc20 2023-12-19T16:57:26.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb43968c640 1 -- 172.21.15.26:0/2379337025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb434154ed0 con 0x7fb4340ccc20 2023-12-19T16:57:26.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb43968c640 1 -- 172.21.15.26:0/2379337025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb434155490 con 0x7fb4340ccc20 2023-12-19T16:57:26.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.301+0000 7fb43968c640 1 -- 172.21.15.26:0/2379337025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4341556b0 con 0x7fb4340ccc20 2023-12-19T16:57:26.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.305+0000 7fb42b7fe640 1 -- 172.21.15.26:0/2379337025 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb41c00f3c0 con 0x7fb4340ccc20 2023-12-19T16:57:26.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.305+0000 7fb42b7fe640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb414065b80 0x7fb414068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:26.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.305+0000 7fb42b7fe640 1 -- 172.21.15.26:0/2379337025 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fb41c083660 con 0x7fb4340ccc20 2023-12-19T16:57:26.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.305+0000 7fb432ffd640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb414065b80 0x7fb414068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:26.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.305+0000 7fb432ffd640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb414065b80 0x7fb414068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb4240ad000 tx=0x7fb4240a6000 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:26.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.309+0000 7fb42b7fe640 1 -- 172.21.15.26:0/2379337025 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb41c050c40 con 0x7fb4340ccc20 2023-12-19T16:57:26.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.501+0000 7fb43968c640 1 -- 172.21.15.26:0/2379337025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb434074270 con 0x7fb4340ccc20 2023-12-19T16:57:26.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.501+0000 7fb42b7fe640 1 -- 172.21.15.26:0/2379337025 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4709 (secure 0 0 0) 0x7fb41c054260 con 0x7fb4340ccc20 2023-12-19T16:57:26.509 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:26.509 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/436365775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":436365775},{"type":"v1","addr":"172.21.15.160:6836","nonce":436365775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3014042376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3014042376},{"type":"v1","addr":"172.21.15.160:6837","nonce":3014042376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1357342534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1357342534},{"type":"v1","addr":"172.21.15.26:6835","nonce":1357342534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:57:23.100695+0000","modified":"2023-12-19T16:57:26.118941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6522":{"gid":6522,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/325854666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":325854666},{"type":"v1","addr":"172.21.15.26:6834","nonce":325854666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-19T16:57:26.510 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 33 2023-12-19T16:57:26.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.505+0000 7fb43968c640 1 -- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb414065b80 msgr2=0x7fb414068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:26.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.505+0000 7fb43968c640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb414065b80 0x7fb414068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb4240ad000 tx=0x7fb4240a6000 comp rx=0 tx=0).stop 2023-12-19T16:57:26.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.505+0000 7fb43968c640 1 -- 172.21.15.26:0/2379337025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4340ccc20 msgr2=0x7fb434149bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:26.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.505+0000 7fb43968c640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4340ccc20 0x7fb434149bf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00b550 tx=0x7fb41c00ba10 comp rx=0 tx=0).stop 2023-12-19T16:57:26.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.505+0000 7fb43968c640 1 -- 172.21.15.26:0/2379337025 shutdown_connections 2023-12-19T16:57:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.505+0000 7fb43968c640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb414065b80 0x7fb414068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.505+0000 7fb43968c640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb43415c7f0 0x7fb4341559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.505+0000 7fb43968c640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4340ccc20 0x7fb434149bf0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.505+0000 7fb43968c640 1 --2- 172.21.15.26:0/2379337025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4340adb80 0x7fb43414dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.505+0000 7fb43968c640 1 -- 172.21.15.26:0/2379337025 >> 172.21.15.26:0/2379337025 conn(0x7fb4340658e0 msgr2=0x7fb4340ae360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.505+0000 7fb43968c640 1 -- 172.21.15.26:0/2379337025 shutdown_connections 2023-12-19T16:57:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.505+0000 7fb43968c640 1 -- 172.21.15.26:0/2379337025 wait complete. 2023-12-19T16:57:26.530 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49cd52c640 1 Processor -- start 2023-12-19T16:57:26.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49cd52c640 1 -- start start 2023-12-19T16:57:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49cd52c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f49c815cd40 0x7f49c80ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49cd52c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49c8160900 0x7f49c8160d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49cd52c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c80d29d0 con 0x7f49c80d2bf0 2023-12-19T16:57:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49cd52c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c8161280 con 0x7f49c815cd40 2023-12-19T16:57:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49cd52c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c815d680 con 0x7f49c8160900 2023-12-19T16:57:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49c77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f49c80d2bf0 0x7f49c80d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53620/0 (socket says 172.21.15.26:53620) 2023-12-19T16:57:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49c77fe640 1 -- 172.21.15.26:0/655423148 learned_addr learned my addr 172.21.15.26:0/655423148 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:26.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49c6ffd640 1 --2- 172.21.15.26:0/655423148 >> v2:172.21.15.160:3300/0 conn(0x7f49c815cd40 0x7f49c80ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49bffff640 1 --2- 172.21.15.26:0/655423148 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49c8160900 0x7f49c8160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49c67fc640 1 -- 172.21.15.26:0/655423148 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006270837 0 0) 0x7f49c00014a0 con 0x7f49c80d2bf0 2023-12-19T16:57:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49c67fc640 1 -- 172.21.15.26:0/655423148 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b4003530 con 0x7f49c80d2bf0 2023-12-19T16:57:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49c6ffd640 1 -- 172.21.15.26:0/655423148 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49c8160900 msgr2=0x7f49c8160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49c6ffd640 1 --2- 172.21.15.26:0/655423148 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49c8160900 0x7f49c8160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49c6ffd640 1 -- 172.21.15.26:0/655423148 >> v1:172.21.15.26:6789/0 conn(0x7f49c80d2bf0 legacy=0x7f49c80d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:26.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.669+0000 7f49c6ffd640 1 -- 172.21.15.26:0/655423148 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c8161750 con 0x7f49c815cd40 2023-12-19T16:57:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.673+0000 7f49c6ffd640 1 --2- 172.21.15.26:0/655423148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c815cd40 0x7f49c80ce040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f49b80097a0 tx=0x7f49b801c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725263ccb4e43c1f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.673+0000 7f49c67fc640 1 -- 172.21.15.26:0/655423148 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8004230 con 0x7f49c815cd40 2023-12-19T16:57:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.673+0000 7f49c67fc640 1 -- 172.21.15.26:0/655423148 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b80a7050 con 0x7f49c815cd40 2023-12-19T16:57:26.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.673+0000 7f49c67fc640 1 -- 172.21.15.26:0/655423148 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8003620 con 0x7f49c815cd40 2023-12-19T16:57:26.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 -- 172.21.15.26:0/655423148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c815cd40 msgr2=0x7f49c80ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:26.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 --2- 172.21.15.26:0/655423148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c815cd40 0x7f49c80ce040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f49b80097a0 tx=0x7f49b801c460 comp rx=0 tx=0).stop 2023-12-19T16:57:26.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 -- 172.21.15.26:0/655423148 shutdown_connections 2023-12-19T16:57:26.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 --2- 172.21.15.26:0/655423148 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49c8160900 0x7f49c8160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 --2- 172.21.15.26:0/655423148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c815cd40 0x7f49c80ce040 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 -- 172.21.15.26:0/655423148 >> 172.21.15.26:0/655423148 conn(0x7f49c806a120 msgr2=0x7f49c806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 -- 172.21.15.26:0/655423148 shutdown_connections 2023-12-19T16:57:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 -- 172.21.15.26:0/655423148 wait complete. 2023-12-19T16:57:26.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 Processor -- start 2023-12-19T16:57:26.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 -- start start 2023-12-19T16:57:26.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80d2bf0 0x7f49c80cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:26.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c815cd40 0x7f49c80c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:26.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49c8160900 0x7f49c81548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:26.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c815c2c0 con 0x7f49c80d2bf0 2023-12-19T16:57:26.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c814dbf0 con 0x7f49c815cd40 2023-12-19T16:57:26.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49cd52c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c80c8bf0 con 0x7f49c8160900 2023-12-19T16:57:26.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80d2bf0 0x7f49c80cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80d2bf0 0x7f49c80cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52806/0 (socket says 172.21.15.26:52806) 2023-12-19T16:57:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49c6ffd640 1 -- 172.21.15.26:0/826761923 learned_addr learned my addr 172.21.15.26:0/826761923 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49c77fe640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49c8160900 0x7f49c81548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49bffff640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c815cd40 0x7f49c80c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49c6ffd640 1 -- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49c8160900 msgr2=0x7f49c81548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49c6ffd640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49c8160900 0x7f49c81548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49c6ffd640 1 -- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c815cd40 msgr2=0x7f49c80c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49c6ffd640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c815cd40 0x7f49c80c6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49c6ffd640 1 -- 172.21.15.26:0/826761923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b80093b0 con 0x7f49c80d2bf0 2023-12-19T16:57:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49bffff640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c815cd40 0x7f49c80c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49c6ffd640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80d2bf0 0x7f49c80cc680 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f49b8019070 tx=0x7f49b8019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.677+0000 7f49bf7fe640 1 -- 172.21.15.26:0/826761923 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8005ba0 con 0x7f49c80d2bf0 2023-12-19T16:57:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.681+0000 7f49bf7fe640 1 -- 172.21.15.26:0/826761923 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b8005d00 con 0x7f49c80d2bf0 2023-12-19T16:57:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.681+0000 7f49bf7fe640 1 -- 172.21.15.26:0/826761923 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8003b10 con 0x7f49c80d2bf0 2023-12-19T16:57:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.681+0000 7f49cd52c640 1 -- 172.21.15.26:0/826761923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c80c8390 con 0x7f49c80d2bf0 2023-12-19T16:57:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.681+0000 7f49cd52c640 1 -- 172.21.15.26:0/826761923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c80c8910 con 0x7f49c80d2bf0 2023-12-19T16:57:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.681+0000 7f49cd52c640 1 -- 172.21.15.26:0/826761923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c815c400 con 0x7f49c80d2bf0 2023-12-19T16:57:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.681+0000 7f49bf7fe640 1 -- 172.21.15.26:0/826761923 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f49b80b2050 con 0x7f49c80d2bf0 2023-12-19T16:57:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.681+0000 7f49bf7fe640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49a8065b80 0x7f49a8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.681+0000 7f49bf7fe640 1 -- 172.21.15.26:0/826761923 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f49b811d890 con 0x7f49c80d2bf0 2023-12-19T16:57:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.681+0000 7f49bffff640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49a8065b80 0x7f49a8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.681+0000 7f49bffff640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49a8065b80 0x7f49a8068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f49c815cab0 tx=0x7f49b0008040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.685+0000 7f49bf7fe640 1 -- 172.21.15.26:0/826761923 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f49b80ead40 con 0x7f49c80d2bf0 2023-12-19T16:57:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.845+0000 7f49cd52c640 1 -- 172.21.15.26:0/826761923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49c8074270 con 0x7f49c80d2bf0 2023-12-19T16:57:26.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.845+0000 7f49bf7fe640 1 -- 172.21.15.26:0/826761923 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+17202 (secure 0 0 0) 0x7f49b811be60 con 0x7f49c80d2bf0 2023-12-19T16:57:26.851 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:26.851 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":79,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:24.103915+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:20.751846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:22.096742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.849+0000 7f49cd52c640 1 -- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49a8065b80 msgr2=0x7f49a8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.849+0000 7f49cd52c640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49a8065b80 0x7f49a8068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f49c815cab0 tx=0x7f49b0008040 comp rx=0 tx=0).stop 2023-12-19T16:57:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.849+0000 7f49cd52c640 1 -- 172.21.15.26:0/826761923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80d2bf0 msgr2=0x7f49c80cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.849+0000 7f49cd52c640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80d2bf0 0x7f49c80cc680 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f49b8019070 tx=0x7f49b8019200 comp rx=0 tx=0).stop 2023-12-19T16:57:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.849+0000 7f49cd52c640 1 -- 172.21.15.26:0/826761923 shutdown_connections 2023-12-19T16:57:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.849+0000 7f49cd52c640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49a8065b80 0x7f49a8068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.849+0000 7f49cd52c640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49c8160900 0x7f49c81548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.849+0000 7f49cd52c640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c815cd40 0x7f49c80c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.849+0000 7f49cd52c640 1 --2- 172.21.15.26:0/826761923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49c80d2bf0 0x7f49c80cc680 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.849+0000 7f49cd52c640 1 -- 172.21.15.26:0/826761923 >> 172.21.15.26:0/826761923 conn(0x7f49c806a120 msgr2=0x7f49c80b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.849+0000 7f49cd52c640 1 -- 172.21.15.26:0/826761923 shutdown_connections 2023-12-19T16:57:26.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:26.849+0000 7f49cd52c640 1 -- 172.21.15.26:0/826761923 wait complete. 2023-12-19T16:57:26.873 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T16:57:26.873 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:57:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.009+0000 7fa341bce640 1 Processor -- start 2023-12-19T16:57:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa341bce640 1 -- start start 2023-12-19T16:57:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa341bce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa33c154220 0x7fa33c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa341bce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33c06b3d0 0x7fa33c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa341bce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33c06bc30 con 0x7fa33c0cec40 2023-12-19T16:57:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa341bce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa33c0cf0d0 con 0x7fa33c154220 2023-12-19T16:57:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa341bce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0cc6e0 con 0x7fa33c06b3d0 2023-12-19T16:57:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa33bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa33c0cec40 0x7fa33c15d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53628/0 (socket says 172.21.15.26:53628) 2023-12-19T16:57:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa33bfff640 1 -- 172.21.15.26:0/3644371627 learned_addr learned my addr 172.21.15.26:0/3644371627 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa33affd640 1 --2- 172.21.15.26:0/3644371627 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33c06b3d0 0x7fa33c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa33b7fe640 1 --2- 172.21.15.26:0/3644371627 >> v2:172.21.15.160:3300/0 conn(0x7fa33c154220 0x7fa33c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa33a7fc640 1 -- 172.21.15.26:0/3644371627 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870058899 0 0) 0x7fa33c06bc30 con 0x7fa33c0cec40 2023-12-19T16:57:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa33a7fc640 1 -- 172.21.15.26:0/3644371627 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa324003530 con 0x7fa33c0cec40 2023-12-19T16:57:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa33affd640 1 -- 172.21.15.26:0/3644371627 >> v2:172.21.15.160:3300/0 conn(0x7fa33c154220 msgr2=0x7fa33c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa33affd640 1 --2- 172.21.15.26:0/3644371627 >> v2:172.21.15.160:3300/0 conn(0x7fa33c154220 0x7fa33c15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa33affd640 1 -- 172.21.15.26:0/3644371627 >> v1:172.21.15.26:6789/0 conn(0x7fa33c0cec40 legacy=0x7fa33c15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.013+0000 7fa33affd640 1 -- 172.21.15.26:0/3644371627 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c15afc0 con 0x7fa33c06b3d0 2023-12-19T16:57:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.017+0000 7fa33affd640 1 --2- 172.21.15.26:0/3644371627 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33c06b3d0 0x7fa33c0ce700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa3300097c0 tx=0x7fa33001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28a07844dbee7e45 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.017+0000 7fa33a7fc640 1 -- 172.21.15.26:0/3644371627 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3300ab600 con 0x7fa33c06b3d0 2023-12-19T16:57:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.017+0000 7fa33a7fc640 1 -- 172.21.15.26:0/3644371627 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3300a7050 con 0x7fa33c06b3d0 2023-12-19T16:57:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.017+0000 7fa33a7fc640 1 -- 172.21.15.26:0/3644371627 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3300ab980 con 0x7fa33c06b3d0 2023-12-19T16:57:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.017+0000 7fa341bce640 1 -- 172.21.15.26:0/3644371627 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33c06b3d0 msgr2=0x7fa33c0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.017+0000 7fa341bce640 1 --2- 172.21.15.26:0/3644371627 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33c06b3d0 0x7fa33c0ce700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa3300097c0 tx=0x7fa33001c450 comp rx=0 tx=0).stop 2023-12-19T16:57:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.017+0000 7fa341bce640 1 -- 172.21.15.26:0/3644371627 shutdown_connections 2023-12-19T16:57:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.017+0000 7fa341bce640 1 --2- 172.21.15.26:0/3644371627 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33c06b3d0 0x7fa33c0ce700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.017+0000 7fa341bce640 1 --2- 172.21.15.26:0/3644371627 >> v2:172.21.15.160:3300/0 conn(0x7fa33c154220 0x7fa33c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.017+0000 7fa341bce640 1 -- 172.21.15.26:0/3644371627 >> 172.21.15.26:0/3644371627 conn(0x7fa33c0658e0 msgr2=0x7fa33c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.017+0000 7fa341bce640 1 -- 172.21.15.26:0/3644371627 shutdown_connections 2023-12-19T16:57:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.017+0000 7fa341bce640 1 -- 172.21.15.26:0/3644371627 wait complete. 2023-12-19T16:57:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.017+0000 7fa341bce640 1 Processor -- start 2023-12-19T16:57:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa341bce640 1 -- start start 2023-12-19T16:57:27.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa341bce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33c06b3d0 0x7fa33c071230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa341bce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa33c0cec40 0x7fa33c06f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa341bce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa33c154220 0x7fa33c1f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa341bce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c13c980 con 0x7fa33c154220 2023-12-19T16:57:27.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa341bce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c06f8b0 con 0x7fa33c0cec40 2023-12-19T16:57:27.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa341bce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c1f5400 con 0x7fa33c06b3d0 2023-12-19T16:57:27.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa33c154220 0x7fa33c1f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa33bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa33c154220 0x7fa33c1f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52810/0 (socket says 172.21.15.26:52810) 2023-12-19T16:57:27.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa33bfff640 1 -- 172.21.15.26:0/476270432 learned_addr learned my addr 172.21.15.26:0/476270432 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:27.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa33affd640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa33c0cec40 0x7fa33c06f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa33b7fe640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33c06b3d0 0x7fa33c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa33bfff640 1 -- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33c06b3d0 msgr2=0x7fa33c071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa33bfff640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33c06b3d0 0x7fa33c071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa33bfff640 1 -- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa33c0cec40 msgr2=0x7fa33c06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa33bfff640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa33c0cec40 0x7fa33c06f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa33bfff640 1 -- 172.21.15.26:0/476270432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa330009420 con 0x7fa33c154220 2023-12-19T16:57:27.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa33bfff640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa33c154220 0x7fa33c1f4ec0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa33400caa0 tx=0x7fa33400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa323fff640 1 -- 172.21.15.26:0/476270432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33400ac00 con 0x7fa33c154220 2023-12-19T16:57:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa323fff640 1 -- 172.21.15.26:0/476270432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33400ad60 con 0x7fa33c154220 2023-12-19T16:57:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa323fff640 1 -- 172.21.15.26:0/476270432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa334012620 con 0x7fa33c154220 2023-12-19T16:57:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa341bce640 1 -- 172.21.15.26:0/476270432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33c13bfe0 con 0x7fa33c154220 2023-12-19T16:57:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa341bce640 1 -- 172.21.15.26:0/476270432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33c13c5a0 con 0x7fa33c154220 2023-12-19T16:57:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.021+0000 7fa341bce640 1 -- 172.21.15.26:0/476270432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33c13cac0 con 0x7fa33c154220 2023-12-19T16:57:27.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.025+0000 7fa323fff640 1 -- 172.21.15.26:0/476270432 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa334007070 con 0x7fa33c154220 2023-12-19T16:57:27.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.025+0000 7fa323fff640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa328065bb0 0x7fa328068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.025+0000 7fa323fff640 1 -- 172.21.15.26:0/476270432 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fa334083530 con 0x7fa33c154220 2023-12-19T16:57:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.025+0000 7fa33b7fe640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa328065bb0 0x7fa328068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.025+0000 7fa33b7fe640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa328065bb0 0x7fa328068070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009b60 tx=0x7fa32c009190 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:27.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.029+0000 7fa323fff640 1 -- 172.21.15.26:0/476270432 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa33c13cac0 con 0x7fa33c154220 2023-12-19T16:57:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.185+0000 7fa341bce640 1 -- 172.21.15.26:0/476270432 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa33c0ccef0 con 0x7fa328065bb0 2023-12-19T16:57:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.197+0000 7fa323fff640 1 -- 172.21.15.26:0/476270432 <== mgr.5606 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288902 (secure 0 0 0) 0x7fa32c011ac0 con 0x7fa328065bb0 2023-12-19T16:57:27.201 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:27.203 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:57:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.197+0000 7fa341bce640 1 -- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa328065bb0 msgr2=0x7fa328068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.197+0000 7fa341bce640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa328065bb0 0x7fa328068070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009b60 tx=0x7fa32c009190 comp rx=0 tx=0).stop 2023-12-19T16:57:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.197+0000 7fa341bce640 1 -- 172.21.15.26:0/476270432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa33c154220 msgr2=0x7fa33c1f4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.197+0000 7fa341bce640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa33c154220 0x7fa33c1f4ec0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fa33400caa0 tx=0x7fa33400cf60 comp rx=0 tx=0).stop 2023-12-19T16:57:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.197+0000 7fa341bce640 1 -- 172.21.15.26:0/476270432 shutdown_connections 2023-12-19T16:57:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.197+0000 7fa341bce640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa33c154220 0x7fa33c1f4ec0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.197+0000 7fa341bce640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa33c0cec40 0x7fa33c06f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.197+0000 7fa341bce640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa328065bb0 0x7fa328068070 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.197+0000 7fa341bce640 1 --2- 172.21.15.26:0/476270432 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33c06b3d0 0x7fa33c071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.197+0000 7fa341bce640 1 -- 172.21.15.26:0/476270432 >> 172.21.15.26:0/476270432 conn(0x7fa33c0658e0 msgr2=0x7fa33c065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.197+0000 7fa341bce640 1 -- 172.21.15.26:0/476270432 shutdown_connections 2023-12-19T16:57:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.197+0000 7fa341bce640 1 -- 172.21.15.26:0/476270432 wait complete. 2023-12-19T16:57:27.222 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-19T16:57:26.422389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":309384,"kb_used_data":8520,"kb_used_omap":0,"kb_used_meta":300800,"kb_avail":754665336,"statfs":{"total":773094113280,"available":772777304064,"internally_reserved":0,"allocated":8724480,"data_stored":4820972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308019200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.738191"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113601+0000","last_change":"2023-12-19T16:57:23.130736+0000","last_active":"2023-12-19T16:57:24.113601+0000","last_peered":"2023-12-19T16:57:24.113601+0000","last_clean":"2023-12-19T16:57:24.113601+0000","last_became_active":"2023-12-19T16:57:23.130370+0000","last_became_peered":"2023-12-19T16:57:23.130370+0000","last_unstale":"2023-12-19T16:57:24.113601+0000","last_undegraded":"2023-12-19T16:57:24.113601+0000","last_fullsized":"2023-12-19T16:57:24.113601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:14:33.246576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138549+0000","last_change":"2023-12-19T16:57:23.134802+0000","last_active":"2023-12-19T16:57:23.138549+0000","last_peered":"2023-12-19T16:57:23.138549+0000","last_clean":"2023-12-19T16:57:23.138549+0000","last_became_active":"2023-12-19T16:57:23.132831+0000","last_became_peered":"2023-12-19T16:57:23.132831+0000","last_unstale":"2023-12-19T16:57:23.138549+0000","last_undegraded":"2023-12-19T16:57:23.138549+0000","last_fullsized":"2023-12-19T16:57:23.138549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:30:52.952843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114747+0000","last_change":"2023-12-19T16:57:23.130998+0000","last_active":"2023-12-19T16:57:24.114747+0000","last_peered":"2023-12-19T16:57:24.114747+0000","last_clean":"2023-12-19T16:57:24.114747+0000","last_became_active":"2023-12-19T16:57:23.130698+0000","last_became_peered":"2023-12-19T16:57:23.130698+0000","last_unstale":"2023-12-19T16:57:24.114747+0000","last_undegraded":"2023-12-19T16:57:24.114747+0000","last_fullsized":"2023-12-19T16:57:24.114747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:46.510340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135506+0000","last_change":"2023-12-19T16:57:23.128782+0000","last_active":"2023-12-19T16:57:23.135506+0000","last_peered":"2023-12-19T16:57:23.135506+0000","last_clean":"2023-12-19T16:57:23.135506+0000","last_became_active":"2023-12-19T16:57:23.128460+0000","last_became_peered":"2023-12-19T16:57:23.128460+0000","last_unstale":"2023-12-19T16:57:23.135506+0000","last_undegraded":"2023-12-19T16:57:23.135506+0000","last_fullsized":"2023-12-19T16:57:23.135506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:41.666771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113211+0000","last_change":"2023-12-19T16:57:23.136824+0000","last_active":"2023-12-19T16:57:24.113211+0000","last_peered":"2023-12-19T16:57:24.113211+0000","last_clean":"2023-12-19T16:57:24.113211+0000","last_became_active":"2023-12-19T16:57:23.136559+0000","last_became_peered":"2023-12-19T16:57:23.136559+0000","last_unstale":"2023-12-19T16:57:24.113211+0000","last_undegraded":"2023-12-19T16:57:24.113211+0000","last_fullsized":"2023-12-19T16:57:24.113211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:17.304496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135720+0000","last_change":"2023-12-19T16:57:23.123111+0000","last_active":"2023-12-19T16:57:23.135720+0000","last_peered":"2023-12-19T16:57:23.135720+0000","last_clean":"2023-12-19T16:57:23.135720+0000","last_became_active":"2023-12-19T16:57:23.122673+0000","last_became_peered":"2023-12-19T16:57:23.122673+0000","last_unstale":"2023-12-19T16:57:23.135720+0000","last_undegraded":"2023-12-19T16:57:23.135720+0000","last_fullsized":"2023-12-19T16:57:23.135720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:18.315096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.129268+0000","last_change":"2023-12-19T16:57:23.126244+0000","last_active":"2023-12-19T16:57:23.129268+0000","last_peered":"2023-12-19T16:57:23.129268+0000","last_clean":"2023-12-19T16:57:23.129268+0000","last_became_active":"2023-12-19T16:57:23.125955+0000","last_became_peered":"2023-12-19T16:57:23.125955+0000","last_unstale":"2023-12-19T16:57:23.129268+0000","last_undegraded":"2023-12-19T16:57:23.129268+0000","last_fullsized":"2023-12-19T16:57:23.129268+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:29:23.721263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.132545+0000","last_change":"2023-12-19T16:57:23.126053+0000","last_active":"2023-12-19T16:57:23.132545+0000","last_peered":"2023-12-19T16:57:23.132545+0000","last_clean":"2023-12-19T16:57:23.132545+0000","last_became_active":"2023-12-19T16:57:23.125779+0000","last_became_peered":"2023-12-19T16:57:23.125779+0000","last_unstale":"2023-12-19T16:57:23.132545+0000","last_undegraded":"2023-12-19T16:57:23.132545+0000","last_fullsized":"2023-12-19T16:57:23.132545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:48.226902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135992+0000","last_change":"2023-12-19T16:57:23.132976+0000","last_active":"2023-12-19T16:57:23.135992+0000","last_peered":"2023-12-19T16:57:23.135992+0000","last_clean":"2023-12-19T16:57:23.135992+0000","last_became_active":"2023-12-19T16:57:23.132486+0000","last_became_peered":"2023-12-19T16:57:23.132486+0000","last_unstale":"2023-12-19T16:57:23.135992+0000","last_undegraded":"2023-12-19T16:57:23.135992+0000","last_fullsized":"2023-12-19T16:57:23.135992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:55.558144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.120451+0000","last_change":"2023-12-19T16:57:23.116398+0000","last_active":"2023-12-19T16:57:23.120451+0000","last_peered":"2023-12-19T16:57:23.120451+0000","last_clean":"2023-12-19T16:57:23.120451+0000","last_became_active":"2023-12-19T16:57:23.115111+0000","last_became_peered":"2023-12-19T16:57:23.115111+0000","last_unstale":"2023-12-19T16:57:23.120451+0000","last_undegraded":"2023-12-19T16:57:23.120451+0000","last_fullsized":"2023-12-19T16:57:23.120451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:41.666771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.141084+0000","last_change":"2023-12-19T16:57:23.135226+0000","last_active":"2023-12-19T16:57:23.141084+0000","last_peered":"2023-12-19T16:57:23.141084+0000","last_clean":"2023-12-19T16:57:23.141084+0000","last_became_active":"2023-12-19T16:57:23.134983+0000","last_became_peered":"2023-12-19T16:57:23.134983+0000","last_unstale":"2023-12-19T16:57:23.141084+0000","last_undegraded":"2023-12-19T16:57:23.141084+0000","last_fullsized":"2023-12-19T16:57:23.141084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:50:43.426372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136632+0000","last_change":"2023-12-19T16:57:23.132583+0000","last_active":"2023-12-19T16:57:23.136632+0000","last_peered":"2023-12-19T16:57:23.136632+0000","last_clean":"2023-12-19T16:57:23.136632+0000","last_became_active":"2023-12-19T16:57:23.131805+0000","last_became_peered":"2023-12-19T16:57:23.131805+0000","last_unstale":"2023-12-19T16:57:23.136632+0000","last_undegraded":"2023-12-19T16:57:23.136632+0000","last_fullsized":"2023-12-19T16:57:23.136632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:40:55.499800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114226+0000","last_change":"2023-12-19T16:57:23.124679+0000","last_active":"2023-12-19T16:57:24.114226+0000","last_peered":"2023-12-19T16:57:24.114226+0000","last_clean":"2023-12-19T16:57:24.114226+0000","last_became_active":"2023-12-19T16:57:23.123923+0000","last_became_peered":"2023-12-19T16:57:23.123923+0000","last_unstale":"2023-12-19T16:57:24.114226+0000","last_undegraded":"2023-12-19T16:57:24.114226+0000","last_fullsized":"2023-12-19T16:57:24.114226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:24:44.642159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.126977+0000","last_change":"2023-12-19T16:57:23.125493+0000","last_active":"2023-12-19T16:57:23.126977+0000","last_peered":"2023-12-19T16:57:23.126977+0000","last_clean":"2023-12-19T16:57:23.126977+0000","last_became_active":"2023-12-19T16:57:23.125176+0000","last_became_peered":"2023-12-19T16:57:23.125176+0000","last_unstale":"2023-12-19T16:57:23.126977+0000","last_undegraded":"2023-12-19T16:57:23.126977+0000","last_fullsized":"2023-12-19T16:57:23.126977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:25.937792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.126948+0000","last_change":"2023-12-19T16:57:23.123164+0000","last_active":"2023-12-19T16:57:23.126948+0000","last_peered":"2023-12-19T16:57:23.126948+0000","last_clean":"2023-12-19T16:57:23.126948+0000","last_became_active":"2023-12-19T16:57:23.122887+0000","last_became_peered":"2023-12-19T16:57:23.122887+0000","last_unstale":"2023-12-19T16:57:23.126948+0000","last_undegraded":"2023-12-19T16:57:23.126948+0000","last_fullsized":"2023-12-19T16:57:23.126948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:17.774629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136274+0000","last_change":"2023-12-19T16:57:23.124537+0000","last_active":"2023-12-19T16:57:23.136274+0000","last_peered":"2023-12-19T16:57:23.136274+0000","last_clean":"2023-12-19T16:57:23.136274+0000","last_became_active":"2023-12-19T16:57:23.124289+0000","last_became_peered":"2023-12-19T16:57:23.124289+0000","last_unstale":"2023-12-19T16:57:23.136274+0000","last_undegraded":"2023-12-19T16:57:23.136274+0000","last_fullsized":"2023-12-19T16:57:23.136274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:33:59.056471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136290+0000","last_change":"2023-12-19T16:57:23.129182+0000","last_active":"2023-12-19T16:57:23.136290+0000","last_peered":"2023-12-19T16:57:23.136290+0000","last_clean":"2023-12-19T16:57:23.136290+0000","last_became_active":"2023-12-19T16:57:23.128918+0000","last_became_peered":"2023-12-19T16:57:23.128918+0000","last_unstale":"2023-12-19T16:57:23.136290+0000","last_undegraded":"2023-12-19T16:57:23.136290+0000","last_fullsized":"2023-12-19T16:57:23.136290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:18.698967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114900+0000","last_change":"2023-12-19T16:57:23.131112+0000","last_active":"2023-12-19T16:57:24.114900+0000","last_peered":"2023-12-19T16:57:24.114900+0000","last_clean":"2023-12-19T16:57:24.114900+0000","last_became_active":"2023-12-19T16:57:23.130636+0000","last_became_peered":"2023-12-19T16:57:23.130636+0000","last_unstale":"2023-12-19T16:57:24.114900+0000","last_undegraded":"2023-12-19T16:57:24.114900+0000","last_fullsized":"2023-12-19T16:57:24.114900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:19.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113458+0000","last_change":"2023-12-19T16:57:23.133197+0000","last_active":"2023-12-19T16:57:24.113458+0000","last_peered":"2023-12-19T16:57:24.113458+0000","last_clean":"2023-12-19T16:57:24.113458+0000","last_became_active":"2023-12-19T16:57:23.132931+0000","last_became_peered":"2023-12-19T16:57:23.132931+0000","last_unstale":"2023-12-19T16:57:24.113458+0000","last_undegraded":"2023-12-19T16:57:24.113458+0000","last_fullsized":"2023-12-19T16:57:24.113458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:45:27.147311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.139636+0000","last_change":"2023-12-19T16:57:23.135826+0000","last_active":"2023-12-19T16:57:23.139636+0000","last_peered":"2023-12-19T16:57:23.139636+0000","last_clean":"2023-12-19T16:57:23.139636+0000","last_became_active":"2023-12-19T16:57:23.135549+0000","last_became_peered":"2023-12-19T16:57:23.135549+0000","last_unstale":"2023-12-19T16:57:23.139636+0000","last_undegraded":"2023-12-19T16:57:23.139636+0000","last_fullsized":"2023-12-19T16:57:23.139636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:44:42.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114690+0000","last_change":"2023-12-19T16:57:23.136730+0000","last_active":"2023-12-19T16:57:24.114690+0000","last_peered":"2023-12-19T16:57:24.114690+0000","last_clean":"2023-12-19T16:57:24.114690+0000","last_became_active":"2023-12-19T16:57:23.136383+0000","last_became_peered":"2023-12-19T16:57:23.136383+0000","last_unstale":"2023-12-19T16:57:24.114690+0000","last_undegraded":"2023-12-19T16:57:24.114690+0000","last_fullsized":"2023-12-19T16:57:24.114690+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:01:37.696798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138398+0000","last_change":"2023-12-19T16:57:23.132534+0000","last_active":"2023-12-19T16:57:23.138398+0000","last_peered":"2023-12-19T16:57:23.138398+0000","last_clean":"2023-12-19T16:57:23.138398+0000","last_became_active":"2023-12-19T16:57:23.127032+0000","last_became_peered":"2023-12-19T16:57:23.127032+0000","last_unstale":"2023-12-19T16:57:23.138398+0000","last_undegraded":"2023-12-19T16:57:23.138398+0000","last_fullsized":"2023-12-19T16:57:23.138398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:57:01.765156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114938+0000","last_change":"2023-12-19T16:57:23.133732+0000","last_active":"2023-12-19T16:57:24.114938+0000","last_peered":"2023-12-19T16:57:24.114938+0000","last_clean":"2023-12-19T16:57:24.114938+0000","last_became_active":"2023-12-19T16:57:23.131968+0000","last_became_peered":"2023-12-19T16:57:23.131968+0000","last_unstale":"2023-12-19T16:57:24.114938+0000","last_undegraded":"2023-12-19T16:57:24.114938+0000","last_fullsized":"2023-12-19T16:57:24.114938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:40:55.499800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113983+0000","last_change":"2023-12-19T16:57:23.124802+0000","last_active":"2023-12-19T16:57:24.113983+0000","last_peered":"2023-12-19T16:57:24.113983+0000","last_clean":"2023-12-19T16:57:24.113983+0000","last_became_active":"2023-12-19T16:57:23.123833+0000","last_became_peered":"2023-12-19T16:57:23.123833+0000","last_unstale":"2023-12-19T16:57:24.113983+0000","last_undegraded":"2023-12-19T16:57:24.113983+0000","last_fullsized":"2023-12-19T16:57:24.113983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:01:31.130549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113667+0000","last_change":"2023-12-19T16:57:23.128944+0000","last_active":"2023-12-19T16:57:24.113667+0000","last_peered":"2023-12-19T16:57:24.113667+0000","last_clean":"2023-12-19T16:57:24.113667+0000","last_became_active":"2023-12-19T16:57:23.126743+0000","last_became_peered":"2023-12-19T16:57:23.126743+0000","last_unstale":"2023-12-19T16:57:24.113667+0000","last_undegraded":"2023-12-19T16:57:24.113667+0000","last_fullsized":"2023-12-19T16:57:24.113667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:38.439897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.127887+0000","last_change":"2023-12-19T16:57:23.121036+0000","last_active":"2023-12-19T16:57:23.127887+0000","last_peered":"2023-12-19T16:57:23.127887+0000","last_clean":"2023-12-19T16:57:23.127887+0000","last_became_active":"2023-12-19T16:57:23.120729+0000","last_became_peered":"2023-12-19T16:57:23.120729+0000","last_unstale":"2023-12-19T16:57:23.127887+0000","last_undegraded":"2023-12-19T16:57:23.127887+0000","last_fullsized":"2023-12-19T16:57:23.127887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:50:43.426372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136205+0000","last_change":"2023-12-19T16:57:23.124745+0000","last_active":"2023-12-19T16:57:23.136205+0000","last_peered":"2023-12-19T16:57:23.136205+0000","last_clean":"2023-12-19T16:57:23.136205+0000","last_became_active":"2023-12-19T16:57:23.123945+0000","last_became_peered":"2023-12-19T16:57:23.123945+0000","last_unstale":"2023-12-19T16:57:23.136205+0000","last_undegraded":"2023-12-19T16:57:23.136205+0000","last_fullsized":"2023-12-19T16:57:23.136205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:31:40.754830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113418+0000","last_change":"2023-12-19T16:57:23.138702+0000","last_active":"2023-12-19T16:57:24.113418+0000","last_peered":"2023-12-19T16:57:24.113418+0000","last_clean":"2023-12-19T16:57:24.113418+0000","last_became_active":"2023-12-19T16:57:23.138434+0000","last_became_peered":"2023-12-19T16:57:23.138434+0000","last_unstale":"2023-12-19T16:57:24.113418+0000","last_undegraded":"2023-12-19T16:57:24.113418+0000","last_fullsized":"2023-12-19T16:57:24.113418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:49:37.385210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.134542+0000","last_change":"2023-12-19T16:57:23.128468+0000","last_active":"2023-12-19T16:57:23.134542+0000","last_peered":"2023-12-19T16:57:23.134542+0000","last_clean":"2023-12-19T16:57:23.134542+0000","last_became_active":"2023-12-19T16:57:23.128215+0000","last_became_peered":"2023-12-19T16:57:23.128215+0000","last_unstale":"2023-12-19T16:57:23.134542+0000","last_undegraded":"2023-12-19T16:57:23.134542+0000","last_fullsized":"2023-12-19T16:57:23.134542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:57:00.650289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114662+0000","last_change":"2023-12-19T16:57:23.133111+0000","last_active":"2023-12-19T16:57:24.114662+0000","last_peered":"2023-12-19T16:57:24.114662+0000","last_clean":"2023-12-19T16:57:24.114662+0000","last_became_active":"2023-12-19T16:57:23.132832+0000","last_became_peered":"2023-12-19T16:57:23.132832+0000","last_unstale":"2023-12-19T16:57:24.114662+0000","last_undegraded":"2023-12-19T16:57:24.114662+0000","last_fullsized":"2023-12-19T16:57:24.114662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:03:16.069895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113155+0000","last_change":"2023-12-19T16:57:23.126064+0000","last_active":"2023-12-19T16:57:24.113155+0000","last_peered":"2023-12-19T16:57:24.113155+0000","last_clean":"2023-12-19T16:57:24.113155+0000","last_became_active":"2023-12-19T16:57:23.125813+0000","last_became_peered":"2023-12-19T16:57:23.125813+0000","last_unstale":"2023-12-19T16:57:24.113155+0000","last_undegraded":"2023-12-19T16:57:24.113155+0000","last_fullsized":"2023-12-19T16:57:24.113155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:08:04.712091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136386+0000","last_change":"2023-12-19T16:57:23.128888+0000","last_active":"2023-12-19T16:57:23.136386+0000","last_peered":"2023-12-19T16:57:23.136386+0000","last_clean":"2023-12-19T16:57:23.136386+0000","last_became_active":"2023-12-19T16:57:23.128641+0000","last_became_peered":"2023-12-19T16:57:23.128641+0000","last_unstale":"2023-12-19T16:57:23.136386+0000","last_undegraded":"2023-12-19T16:57:23.136386+0000","last_fullsized":"2023-12-19T16:57:23.136386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:12:53.710291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.137582+0000","last_change":"2023-12-19T16:57:23.133306+0000","last_active":"2023-12-19T16:57:23.137582+0000","last_peered":"2023-12-19T16:57:23.137582+0000","last_clean":"2023-12-19T16:57:23.137582+0000","last_became_active":"2023-12-19T16:57:23.133024+0000","last_became_peered":"2023-12-19T16:57:23.133024+0000","last_unstale":"2023-12-19T16:57:23.137582+0000","last_undegraded":"2023-12-19T16:57:23.137582+0000","last_fullsized":"2023-12-19T16:57:23.137582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:16:28.306438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"78'4","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135848+0000","last_change":"2023-12-19T16:57:22.709897+0000","last_active":"2023-12-19T16:57:23.135848+0000","last_peered":"2023-12-19T16:57:23.135848+0000","last_clean":"2023-12-19T16:57:23.135848+0000","last_became_active":"2023-12-19T16:57:22.709584+0000","last_became_peered":"2023-12-19T16:57:22.709584+0000","last_unstale":"2023-12-19T16:57:23.135848+0000","last_undegraded":"2023-12-19T16:57:23.135848+0000","last_fullsized":"2023-12-19T16:57:23.135848+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:38:28.758941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.141042+0000","last_change":"2023-12-19T16:57:23.139483+0000","last_active":"2023-12-19T16:57:23.141042+0000","last_peered":"2023-12-19T16:57:23.141042+0000","last_clean":"2023-12-19T16:57:23.141042+0000","last_became_active":"2023-12-19T16:57:23.139235+0000","last_became_peered":"2023-12-19T16:57:23.139235+0000","last_unstale":"2023-12-19T16:57:23.141042+0000","last_undegraded":"2023-12-19T16:57:23.141042+0000","last_fullsized":"2023-12-19T16:57:23.141042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:02:07.215083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136691+0000","last_change":"2023-12-19T16:57:22.714305+0000","last_active":"2023-12-19T16:57:23.136691+0000","last_peered":"2023-12-19T16:57:23.136691+0000","last_clean":"2023-12-19T16:57:23.136691+0000","last_became_active":"2023-12-19T16:57:22.714024+0000","last_became_peered":"2023-12-19T16:57:22.714024+0000","last_unstale":"2023-12-19T16:57:23.136691+0000","last_undegraded":"2023-12-19T16:57:23.136691+0000","last_fullsized":"2023-12-19T16:57:23.136691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:40.592521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136143+0000","last_change":"2023-12-19T16:57:23.124415+0000","last_active":"2023-12-19T16:57:23.136143+0000","last_peered":"2023-12-19T16:57:23.136143+0000","last_clean":"2023-12-19T16:57:23.136143+0000","last_became_active":"2023-12-19T16:57:23.123427+0000","last_became_peered":"2023-12-19T16:57:23.123427+0000","last_unstale":"2023-12-19T16:57:23.136143+0000","last_undegraded":"2023-12-19T16:57:23.136143+0000","last_fullsized":"2023-12-19T16:57:23.136143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:10:18.501944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.141326+0000","last_change":"2023-12-19T16:57:22.711651+0000","last_active":"2023-12-19T16:57:23.141326+0000","last_peered":"2023-12-19T16:57:23.141326+0000","last_clean":"2023-12-19T16:57:23.141326+0000","last_became_active":"2023-12-19T16:57:22.711220+0000","last_became_peered":"2023-12-19T16:57:22.711220+0000","last_unstale":"2023-12-19T16:57:23.141326+0000","last_undegraded":"2023-12-19T16:57:23.141326+0000","last_fullsized":"2023-12-19T16:57:23.141326+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:32:22.752099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136226+0000","last_change":"2023-12-19T16:57:23.126509+0000","last_active":"2023-12-19T16:57:23.136226+0000","last_peered":"2023-12-19T16:57:23.136226+0000","last_clean":"2023-12-19T16:57:23.136226+0000","last_became_active":"2023-12-19T16:57:23.126058+0000","last_became_peered":"2023-12-19T16:57:23.126058+0000","last_unstale":"2023-12-19T16:57:23.136226+0000","last_undegraded":"2023-12-19T16:57:23.136226+0000","last_fullsized":"2023-12-19T16:57:23.136226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:16:27.678499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135680+0000","last_change":"2023-12-19T16:57:22.711931+0000","last_active":"2023-12-19T16:57:23.135680+0000","last_peered":"2023-12-19T16:57:23.135680+0000","last_clean":"2023-12-19T16:57:23.135680+0000","last_became_active":"2023-12-19T16:57:22.711554+0000","last_became_peered":"2023-12-19T16:57:22.711554+0000","last_unstale":"2023-12-19T16:57:23.135680+0000","last_undegraded":"2023-12-19T16:57:23.135680+0000","last_fullsized":"2023-12-19T16:57:23.135680+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:24.929320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.133287+0000","last_change":"2023-12-19T16:57:23.127410+0000","last_active":"2023-12-19T16:57:23.133287+0000","last_peered":"2023-12-19T16:57:23.133287+0000","last_clean":"2023-12-19T16:57:23.133287+0000","last_became_active":"2023-12-19T16:57:23.126421+0000","last_became_peered":"2023-12-19T16:57:23.126421+0000","last_unstale":"2023-12-19T16:57:23.133287+0000","last_undegraded":"2023-12-19T16:57:23.133287+0000","last_fullsized":"2023-12-19T16:57:23.133287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:06.429698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.125438+0000","last_change":"2023-12-19T16:57:22.709387+0000","last_active":"2023-12-19T16:57:23.125438+0000","last_peered":"2023-12-19T16:57:23.125438+0000","last_clean":"2023-12-19T16:57:23.125438+0000","last_became_active":"2023-12-19T16:57:22.708768+0000","last_became_peered":"2023-12-19T16:57:22.708768+0000","last_unstale":"2023-12-19T16:57:23.125438+0000","last_undegraded":"2023-12-19T16:57:23.125438+0000","last_fullsized":"2023-12-19T16:57:23.125438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:44:00.239607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138190+0000","last_change":"2023-12-19T16:57:23.131957+0000","last_active":"2023-12-19T16:57:23.138190+0000","last_peered":"2023-12-19T16:57:23.138190+0000","last_clean":"2023-12-19T16:57:23.138190+0000","last_became_active":"2023-12-19T16:57:23.131125+0000","last_became_peered":"2023-12-19T16:57:23.131125+0000","last_unstale":"2023-12-19T16:57:23.138190+0000","last_undegraded":"2023-12-19T16:57:23.138190+0000","last_fullsized":"2023-12-19T16:57:23.138190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:06.098693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.129151+0000","last_change":"2023-12-19T16:57:22.716999+0000","last_active":"2023-12-19T16:57:23.129151+0000","last_peered":"2023-12-19T16:57:23.129151+0000","last_clean":"2023-12-19T16:57:23.129151+0000","last_became_active":"2023-12-19T16:57:22.716609+0000","last_became_peered":"2023-12-19T16:57:22.716609+0000","last_unstale":"2023-12-19T16:57:23.129151+0000","last_undegraded":"2023-12-19T16:57:23.129151+0000","last_fullsized":"2023-12-19T16:57:23.129151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:53:30.738332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114316+0000","last_change":"2023-12-19T16:57:23.134032+0000","last_active":"2023-12-19T16:57:24.114316+0000","last_peered":"2023-12-19T16:57:24.114316+0000","last_clean":"2023-12-19T16:57:24.114316+0000","last_became_active":"2023-12-19T16:57:23.133495+0000","last_became_peered":"2023-12-19T16:57:23.133495+0000","last_unstale":"2023-12-19T16:57:24.114316+0000","last_undegraded":"2023-12-19T16:57:24.114316+0000","last_fullsized":"2023-12-19T16:57:24.114316+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:10:13.311295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.109484+0000","last_change":"2023-12-19T16:57:22.703510+0000","last_active":"2023-12-19T16:57:23.109484+0000","last_peered":"2023-12-19T16:57:23.109484+0000","last_clean":"2023-12-19T16:57:23.109484+0000","last_became_active":"2023-12-19T16:57:22.703224+0000","last_became_peered":"2023-12-19T16:57:22.703224+0000","last_unstale":"2023-12-19T16:57:23.109484+0000","last_undegraded":"2023-12-19T16:57:23.109484+0000","last_fullsized":"2023-12-19T16:57:23.109484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:17.631958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136353+0000","last_change":"2023-12-19T16:57:23.129729+0000","last_active":"2023-12-19T16:57:23.136353+0000","last_peered":"2023-12-19T16:57:23.136353+0000","last_clean":"2023-12-19T16:57:23.136353+0000","last_became_active":"2023-12-19T16:57:23.129380+0000","last_became_peered":"2023-12-19T16:57:23.129380+0000","last_unstale":"2023-12-19T16:57:23.136353+0000","last_undegraded":"2023-12-19T16:57:23.136353+0000","last_fullsized":"2023-12-19T16:57:23.136353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:17:22.521240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.114129+0000","last_change":"2023-12-19T16:57:22.716068+0000","last_active":"2023-12-19T16:57:23.114129+0000","last_peered":"2023-12-19T16:57:23.114129+0000","last_clean":"2023-12-19T16:57:23.114129+0000","last_became_active":"2023-12-19T16:57:22.715572+0000","last_became_peered":"2023-12-19T16:57:22.715572+0000","last_unstale":"2023-12-19T16:57:23.114129+0000","last_undegraded":"2023-12-19T16:57:23.114129+0000","last_fullsized":"2023-12-19T16:57:23.114129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:23:04.958220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.126352+0000","last_change":"2023-12-19T16:57:23.123122+0000","last_active":"2023-12-19T16:57:23.126352+0000","last_peered":"2023-12-19T16:57:23.126352+0000","last_clean":"2023-12-19T16:57:23.126352+0000","last_became_active":"2023-12-19T16:57:23.122848+0000","last_became_peered":"2023-12-19T16:57:23.122848+0000","last_unstale":"2023-12-19T16:57:23.126352+0000","last_undegraded":"2023-12-19T16:57:23.126352+0000","last_fullsized":"2023-12-19T16:57:23.126352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:18.640429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.115052+0000","last_change":"2023-12-19T16:57:22.722055+0000","last_active":"2023-12-19T16:57:24.115052+0000","last_peered":"2023-12-19T16:57:24.115052+0000","last_clean":"2023-12-19T16:57:24.115052+0000","last_became_active":"2023-12-19T16:57:22.721767+0000","last_became_peered":"2023-12-19T16:57:22.721767+0000","last_unstale":"2023-12-19T16:57:24.115052+0000","last_undegraded":"2023-12-19T16:57:24.115052+0000","last_fullsized":"2023-12-19T16:57:24.115052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:17.631958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.115106+0000","last_change":"2023-12-19T16:57:23.141377+0000","last_active":"2023-12-19T16:57:24.115106+0000","last_peered":"2023-12-19T16:57:24.115106+0000","last_clean":"2023-12-19T16:57:24.115106+0000","last_became_active":"2023-12-19T16:57:23.141083+0000","last_became_peered":"2023-12-19T16:57:23.141083+0000","last_unstale":"2023-12-19T16:57:24.115106+0000","last_undegraded":"2023-12-19T16:57:24.115106+0000","last_fullsized":"2023-12-19T16:57:24.115106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:30:47.901104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.118661+0000","last_change":"2023-12-19T16:57:22.718327+0000","last_active":"2023-12-19T16:57:23.118661+0000","last_peered":"2023-12-19T16:57:23.118661+0000","last_clean":"2023-12-19T16:57:23.118661+0000","last_became_active":"2023-12-19T16:57:22.718000+0000","last_became_peered":"2023-12-19T16:57:22.718000+0000","last_unstale":"2023-12-19T16:57:23.118661+0000","last_undegraded":"2023-12-19T16:57:23.118661+0000","last_fullsized":"2023-12-19T16:57:23.118661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:46.213640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.140162+0000","last_change":"2023-12-19T16:57:23.124970+0000","last_active":"2023-12-19T16:57:23.140162+0000","last_peered":"2023-12-19T16:57:23.140162+0000","last_clean":"2023-12-19T16:57:23.140162+0000","last_became_active":"2023-12-19T16:57:23.124725+0000","last_became_peered":"2023-12-19T16:57:23.124725+0000","last_unstale":"2023-12-19T16:57:23.140162+0000","last_undegraded":"2023-12-19T16:57:23.140162+0000","last_fullsized":"2023-12-19T16:57:23.140162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:45.596570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"78'3","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.115188+0000","last_change":"2023-12-19T16:57:22.719555+0000","last_active":"2023-12-19T16:57:24.115188+0000","last_peered":"2023-12-19T16:57:24.115188+0000","last_clean":"2023-12-19T16:57:24.115188+0000","last_became_active":"2023-12-19T16:57:22.719293+0000","last_became_peered":"2023-12-19T16:57:22.719293+0000","last_unstale":"2023-12-19T16:57:24.115188+0000","last_undegraded":"2023-12-19T16:57:24.115188+0000","last_fullsized":"2023-12-19T16:57:24.115188+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:43:53.045116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.115062+0000","last_change":"2023-12-19T16:57:23.141880+0000","last_active":"2023-12-19T16:57:24.115062+0000","last_peered":"2023-12-19T16:57:24.115062+0000","last_clean":"2023-12-19T16:57:24.115062+0000","last_became_active":"2023-12-19T16:57:23.141603+0000","last_became_peered":"2023-12-19T16:57:23.141603+0000","last_unstale":"2023-12-19T16:57:24.115062+0000","last_undegraded":"2023-12-19T16:57:24.115062+0000","last_fullsized":"2023-12-19T16:57:24.115062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:41.600992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"78'6","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.140817+0000","last_change":"2023-12-19T16:57:22.713027+0000","last_active":"2023-12-19T16:57:23.140817+0000","last_peered":"2023-12-19T16:57:23.140817+0000","last_clean":"2023-12-19T16:57:23.140817+0000","last_became_active":"2023-12-19T16:57:22.712761+0000","last_became_peered":"2023-12-19T16:57:22.712761+0000","last_unstale":"2023-12-19T16:57:23.140817+0000","last_undegraded":"2023-12-19T16:57:23.140817+0000","last_fullsized":"2023-12-19T16:57:23.140817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:14:35.501559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.133922+0000","last_change":"2023-12-19T16:57:23.126785+0000","last_active":"2023-12-19T16:57:23.133922+0000","last_peered":"2023-12-19T16:57:23.133922+0000","last_clean":"2023-12-19T16:57:23.133922+0000","last_became_active":"2023-12-19T16:57:23.126493+0000","last_became_peered":"2023-12-19T16:57:23.126493+0000","last_unstale":"2023-12-19T16:57:23.133922+0000","last_undegraded":"2023-12-19T16:57:23.133922+0000","last_fullsized":"2023-12-19T16:57:23.133922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:38.775519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135785+0000","last_change":"2023-12-19T16:57:22.718269+0000","last_active":"2023-12-19T16:57:23.135785+0000","last_peered":"2023-12-19T16:57:23.135785+0000","last_clean":"2023-12-19T16:57:23.135785+0000","last_became_active":"2023-12-19T16:57:22.717961+0000","last_became_peered":"2023-12-19T16:57:22.717961+0000","last_unstale":"2023-12-19T16:57:23.135785+0000","last_undegraded":"2023-12-19T16:57:23.135785+0000","last_fullsized":"2023-12-19T16:57:23.135785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:45.501869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114816+0000","last_change":"2023-12-19T16:57:23.134650+0000","last_active":"2023-12-19T16:57:24.114816+0000","last_peered":"2023-12-19T16:57:24.114816+0000","last_clean":"2023-12-19T16:57:24.114816+0000","last_became_active":"2023-12-19T16:57:23.131775+0000","last_became_peered":"2023-12-19T16:57:23.131775+0000","last_unstale":"2023-12-19T16:57:24.114816+0000","last_undegraded":"2023-12-19T16:57:24.114816+0000","last_fullsized":"2023-12-19T16:57:24.114816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:35.829847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114064+0000","last_change":"2023-12-19T16:57:22.718127+0000","last_active":"2023-12-19T16:57:24.114064+0000","last_peered":"2023-12-19T16:57:24.114064+0000","last_clean":"2023-12-19T16:57:24.114064+0000","last_became_active":"2023-12-19T16:57:22.717819+0000","last_became_peered":"2023-12-19T16:57:22.717819+0000","last_unstale":"2023-12-19T16:57:24.114064+0000","last_undegraded":"2023-12-19T16:57:24.114064+0000","last_fullsized":"2023-12-19T16:57:24.114064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:37:26.150257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135805+0000","last_change":"2023-12-19T16:57:23.132252+0000","last_active":"2023-12-19T16:57:23.135805+0000","last_peered":"2023-12-19T16:57:23.135805+0000","last_clean":"2023-12-19T16:57:23.135805+0000","last_became_active":"2023-12-19T16:57:23.131995+0000","last_became_peered":"2023-12-19T16:57:23.131995+0000","last_unstale":"2023-12-19T16:57:23.135805+0000","last_undegraded":"2023-12-19T16:57:23.135805+0000","last_fullsized":"2023-12-19T16:57:23.135805+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:09.688251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.109491+0000","last_change":"2023-12-19T16:57:22.711732+0000","last_active":"2023-12-19T16:57:23.109491+0000","last_peered":"2023-12-19T16:57:23.109491+0000","last_clean":"2023-12-19T16:57:23.109491+0000","last_became_active":"2023-12-19T16:57:22.711446+0000","last_became_peered":"2023-12-19T16:57:22.711446+0000","last_unstale":"2023-12-19T16:57:23.109491+0000","last_undegraded":"2023-12-19T16:57:23.109491+0000","last_fullsized":"2023-12-19T16:57:23.109491+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:08.679779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114525+0000","last_change":"2023-12-19T16:57:23.133197+0000","last_active":"2023-12-19T16:57:24.114525+0000","last_peered":"2023-12-19T16:57:24.114525+0000","last_clean":"2023-12-19T16:57:24.114525+0000","last_became_active":"2023-12-19T16:57:23.132333+0000","last_became_peered":"2023-12-19T16:57:23.132333+0000","last_unstale":"2023-12-19T16:57:24.114525+0000","last_undegraded":"2023-12-19T16:57:24.114525+0000","last_fullsized":"2023-12-19T16:57:24.114525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:25:24.949466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"78'8","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114240+0000","last_change":"2023-12-19T16:57:22.719852+0000","last_active":"2023-12-19T16:57:24.114240+0000","last_peered":"2023-12-19T16:57:24.114240+0000","last_clean":"2023-12-19T16:57:24.114240+0000","last_became_active":"2023-12-19T16:57:22.719525+0000","last_became_peered":"2023-12-19T16:57:22.719525+0000","last_unstale":"2023-12-19T16:57:24.114240+0000","last_undegraded":"2023-12-19T16:57:24.114240+0000","last_fullsized":"2023-12-19T16:57:24.114240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:54.549672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114960+0000","last_change":"2023-12-19T16:57:23.131325+0000","last_active":"2023-12-19T16:57:24.114960+0000","last_peered":"2023-12-19T16:57:24.114960+0000","last_clean":"2023-12-19T16:57:24.114960+0000","last_became_active":"2023-12-19T16:57:23.131003+0000","last_became_peered":"2023-12-19T16:57:23.131003+0000","last_unstale":"2023-12-19T16:57:24.114960+0000","last_undegraded":"2023-12-19T16:57:24.114960+0000","last_fullsized":"2023-12-19T16:57:24.114960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:54:16.935719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138107+0000","last_change":"2023-12-19T16:57:22.718632+0000","last_active":"2023-12-19T16:57:23.138107+0000","last_peered":"2023-12-19T16:57:23.138107+0000","last_clean":"2023-12-19T16:57:23.138107+0000","last_became_active":"2023-12-19T16:57:22.717992+0000","last_became_peered":"2023-12-19T16:57:22.717992+0000","last_unstale":"2023-12-19T16:57:23.138107+0000","last_undegraded":"2023-12-19T16:57:23.138107+0000","last_fullsized":"2023-12-19T16:57:23.138107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:34.821375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.126930+0000","last_change":"2023-12-19T16:57:23.121177+0000","last_active":"2023-12-19T16:57:23.126930+0000","last_peered":"2023-12-19T16:57:23.126930+0000","last_clean":"2023-12-19T16:57:23.126930+0000","last_became_active":"2023-12-19T16:57:23.120893+0000","last_became_peered":"2023-12-19T16:57:23.120893+0000","last_unstale":"2023-12-19T16:57:23.126930+0000","last_undegraded":"2023-12-19T16:57:23.126930+0000","last_fullsized":"2023-12-19T16:57:23.126930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:19.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.137536+0000","last_change":"2023-12-19T16:57:22.713505+0000","last_active":"2023-12-19T16:57:23.137536+0000","last_peered":"2023-12-19T16:57:23.137536+0000","last_clean":"2023-12-19T16:57:23.137536+0000","last_became_active":"2023-12-19T16:57:22.713180+0000","last_became_peered":"2023-12-19T16:57:22.713180+0000","last_unstale":"2023-12-19T16:57:23.137536+0000","last_undegraded":"2023-12-19T16:57:23.137536+0000","last_fullsized":"2023-12-19T16:57:23.137536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:08.953488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.131470+0000","last_change":"2023-12-19T16:57:23.125306+0000","last_active":"2023-12-19T16:57:23.131470+0000","last_peered":"2023-12-19T16:57:23.131470+0000","last_clean":"2023-12-19T16:57:23.131470+0000","last_became_active":"2023-12-19T16:57:23.125044+0000","last_became_peered":"2023-12-19T16:57:23.125044+0000","last_unstale":"2023-12-19T16:57:23.131470+0000","last_undegraded":"2023-12-19T16:57:23.131470+0000","last_fullsized":"2023-12-19T16:57:23.131470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:04:04.164209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.134604+0000","last_change":"2023-12-19T16:57:22.715552+0000","last_active":"2023-12-19T16:57:23.134604+0000","last_peered":"2023-12-19T16:57:23.134604+0000","last_clean":"2023-12-19T16:57:23.134604+0000","last_became_active":"2023-12-19T16:57:22.715216+0000","last_became_peered":"2023-12-19T16:57:22.715216+0000","last_unstale":"2023-12-19T16:57:23.134604+0000","last_undegraded":"2023-12-19T16:57:23.134604+0000","last_fullsized":"2023-12-19T16:57:23.134604+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:29:22.712791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.115004+0000","last_change":"2023-12-19T16:57:23.141558+0000","last_active":"2023-12-19T16:57:24.115004+0000","last_peered":"2023-12-19T16:57:24.115004+0000","last_clean":"2023-12-19T16:57:24.115004+0000","last_became_active":"2023-12-19T16:57:23.141296+0000","last_became_peered":"2023-12-19T16:57:23.141296+0000","last_unstale":"2023-12-19T16:57:24.115004+0000","last_undegraded":"2023-12-19T16:57:24.115004+0000","last_fullsized":"2023-12-19T16:57:24.115004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:13.543929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.121866+0000","last_change":"2023-12-19T16:57:22.716127+0000","last_active":"2023-12-19T16:57:23.121866+0000","last_peered":"2023-12-19T16:57:23.121866+0000","last_clean":"2023-12-19T16:57:23.121866+0000","last_became_active":"2023-12-19T16:57:22.714244+0000","last_became_peered":"2023-12-19T16:57:22.714244+0000","last_unstale":"2023-12-19T16:57:23.121866+0000","last_undegraded":"2023-12-19T16:57:23.121866+0000","last_fullsized":"2023-12-19T16:57:23.121866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:07:34.317614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.133903+0000","last_change":"2023-12-19T16:57:23.126079+0000","last_active":"2023-12-19T16:57:23.133903+0000","last_peered":"2023-12-19T16:57:23.133903+0000","last_clean":"2023-12-19T16:57:23.133903+0000","last_became_active":"2023-12-19T16:57:23.122323+0000","last_became_peered":"2023-12-19T16:57:23.122323+0000","last_unstale":"2023-12-19T16:57:23.133903+0000","last_undegraded":"2023-12-19T16:57:23.133903+0000","last_fullsized":"2023-12-19T16:57:23.133903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:30:47.901104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138417+0000","last_change":"2023-12-19T16:57:22.709791+0000","last_active":"2023-12-19T16:57:23.138417+0000","last_peered":"2023-12-19T16:57:23.138417+0000","last_clean":"2023-12-19T16:57:23.138417+0000","last_became_active":"2023-12-19T16:57:22.709391+0000","last_became_peered":"2023-12-19T16:57:22.709391+0000","last_unstale":"2023-12-19T16:57:23.138417+0000","last_undegraded":"2023-12-19T16:57:23.138417+0000","last_fullsized":"2023-12-19T16:57:23.138417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:01:36.688326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136118+0000","last_change":"2023-12-19T16:57:23.124281+0000","last_active":"2023-12-19T16:57:23.136118+0000","last_peered":"2023-12-19T16:57:23.136118+0000","last_clean":"2023-12-19T16:57:23.136118+0000","last_became_active":"2023-12-19T16:57:23.123986+0000","last_became_peered":"2023-12-19T16:57:23.123986+0000","last_unstale":"2023-12-19T16:57:23.136118+0000","last_undegraded":"2023-12-19T16:57:23.136118+0000","last_fullsized":"2023-12-19T16:57:23.136118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:01:00.969856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.105792+0000","last_change":"2023-12-19T16:57:22.716245+0000","last_active":"2023-12-19T16:57:23.105792+0000","last_peered":"2023-12-19T16:57:23.105792+0000","last_clean":"2023-12-19T16:57:23.105792+0000","last_became_active":"2023-12-19T16:57:22.714702+0000","last_became_peered":"2023-12-19T16:57:22.714702+0000","last_unstale":"2023-12-19T16:57:23.105792+0000","last_undegraded":"2023-12-19T16:57:23.105792+0000","last_fullsized":"2023-12-19T16:57:23.105792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:08.679779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138447+0000","last_change":"2023-12-19T16:57:23.136320+0000","last_active":"2023-12-19T16:57:23.138447+0000","last_peered":"2023-12-19T16:57:23.138447+0000","last_clean":"2023-12-19T16:57:23.138447+0000","last_became_active":"2023-12-19T16:57:23.135951+0000","last_became_peered":"2023-12-19T16:57:23.135951+0000","last_unstale":"2023-12-19T16:57:23.138447+0000","last_undegraded":"2023-12-19T16:57:23.138447+0000","last_fullsized":"2023-12-19T16:57:23.138447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:57.991487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.109119+0000","last_change":"2023-12-19T16:57:22.709715+0000","last_active":"2023-12-19T16:57:23.109119+0000","last_peered":"2023-12-19T16:57:23.109119+0000","last_clean":"2023-12-19T16:57:23.109119+0000","last_became_active":"2023-12-19T16:57:22.707699+0000","last_became_peered":"2023-12-19T16:57:22.707699+0000","last_unstale":"2023-12-19T16:57:23.109119+0000","last_undegraded":"2023-12-19T16:57:23.109119+0000","last_fullsized":"2023-12-19T16:57:23.109119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:43:53.045116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135844+0000","last_change":"2023-12-19T16:57:23.127990+0000","last_active":"2023-12-19T16:57:23.135844+0000","last_peered":"2023-12-19T16:57:23.135844+0000","last_clean":"2023-12-19T16:57:23.135844+0000","last_became_active":"2023-12-19T16:57:23.127614+0000","last_became_peered":"2023-12-19T16:57:23.127614+0000","last_unstale":"2023-12-19T16:57:23.135844+0000","last_undegraded":"2023-12-19T16:57:23.135844+0000","last_fullsized":"2023-12-19T16:57:23.135844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:54:16.935719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138991+0000","last_change":"2023-12-19T16:57:22.713812+0000","last_active":"2023-12-19T16:57:23.138991+0000","last_peered":"2023-12-19T16:57:23.138991+0000","last_clean":"2023-12-19T16:57:23.138991+0000","last_became_active":"2023-12-19T16:57:22.713491+0000","last_became_peered":"2023-12-19T16:57:22.713491+0000","last_unstale":"2023-12-19T16:57:23.138991+0000","last_undegraded":"2023-12-19T16:57:23.138991+0000","last_fullsized":"2023-12-19T16:57:23.138991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:54.549672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136767+0000","last_change":"2023-12-19T16:57:23.130707+0000","last_active":"2023-12-19T16:57:23.136767+0000","last_peered":"2023-12-19T16:57:23.136767+0000","last_clean":"2023-12-19T16:57:23.136767+0000","last_became_active":"2023-12-19T16:57:23.130421+0000","last_became_peered":"2023-12-19T16:57:23.130421+0000","last_unstale":"2023-12-19T16:57:23.136767+0000","last_undegraded":"2023-12-19T16:57:23.136767+0000","last_fullsized":"2023-12-19T16:57:23.136767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:25:24.949466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.114777+0000","last_change":"2023-12-19T16:57:22.715702+0000","last_active":"2023-12-19T16:57:23.114777+0000","last_peered":"2023-12-19T16:57:23.114777+0000","last_clean":"2023-12-19T16:57:23.114777+0000","last_became_active":"2023-12-19T16:57:22.715425+0000","last_became_peered":"2023-12-19T16:57:22.715425+0000","last_unstale":"2023-12-19T16:57:23.114777+0000","last_undegraded":"2023-12-19T16:57:23.114777+0000","last_fullsized":"2023-12-19T16:57:23.114777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:14:22.112729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.125131+0000","last_change":"2023-12-19T16:57:23.122478+0000","last_active":"2023-12-19T16:57:23.125131+0000","last_peered":"2023-12-19T16:57:23.125131+0000","last_clean":"2023-12-19T16:57:23.125131+0000","last_became_active":"2023-12-19T16:57:23.122217+0000","last_became_peered":"2023-12-19T16:57:23.122217+0000","last_unstale":"2023-12-19T16:57:23.125131+0000","last_undegraded":"2023-12-19T16:57:23.125131+0000","last_fullsized":"2023-12-19T16:57:23.125131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:40:53.359468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114088+0000","last_change":"2023-12-19T16:57:22.716637+0000","last_active":"2023-12-19T16:57:24.114088+0000","last_peered":"2023-12-19T16:57:24.114088+0000","last_clean":"2023-12-19T16:57:24.114088+0000","last_became_active":"2023-12-19T16:57:22.716330+0000","last_became_peered":"2023-12-19T16:57:22.716330+0000","last_unstale":"2023-12-19T16:57:24.114088+0000","last_undegraded":"2023-12-19T16:57:24.114088+0000","last_fullsized":"2023-12-19T16:57:24.114088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:44:00.239607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136244+0000","last_change":"2023-12-19T16:57:23.126630+0000","last_active":"2023-12-19T16:57:23.136244+0000","last_peered":"2023-12-19T16:57:23.136244+0000","last_clean":"2023-12-19T16:57:23.136244+0000","last_became_active":"2023-12-19T16:57:23.126384+0000","last_became_peered":"2023-12-19T16:57:23.126384+0000","last_unstale":"2023-12-19T16:57:23.136244+0000","last_undegraded":"2023-12-19T16:57:23.136244+0000","last_fullsized":"2023-12-19T16:57:23.136244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:31:56.777048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.110773+0000","last_change":"2023-12-19T16:57:22.704220+0000","last_active":"2023-12-19T16:57:23.110773+0000","last_peered":"2023-12-19T16:57:23.110773+0000","last_clean":"2023-12-19T16:57:23.110773+0000","last_became_active":"2023-12-19T16:57:22.703934+0000","last_became_peered":"2023-12-19T16:57:22.703934+0000","last_unstale":"2023-12-19T16:57:23.110773+0000","last_undegraded":"2023-12-19T16:57:23.110773+0000","last_fullsized":"2023-12-19T16:57:23.110773+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:54.549672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.132671+0000","last_change":"2023-12-19T16:57:23.124841+0000","last_active":"2023-12-19T16:57:23.132671+0000","last_peered":"2023-12-19T16:57:23.132671+0000","last_clean":"2023-12-19T16:57:23.132671+0000","last_became_active":"2023-12-19T16:57:23.124563+0000","last_became_peered":"2023-12-19T16:57:23.124563+0000","last_unstale":"2023-12-19T16:57:23.132671+0000","last_undegraded":"2023-12-19T16:57:23.132671+0000","last_fullsized":"2023-12-19T16:57:23.132671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:28.949332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.105795+0000","last_change":"2023-12-19T16:57:22.716766+0000","last_active":"2023-12-19T16:57:23.105795+0000","last_peered":"2023-12-19T16:57:23.105795+0000","last_clean":"2023-12-19T16:57:23.105795+0000","last_became_active":"2023-12-19T16:57:22.716413+0000","last_became_peered":"2023-12-19T16:57:22.716413+0000","last_unstale":"2023-12-19T16:57:23.105795+0000","last_undegraded":"2023-12-19T16:57:23.105795+0000","last_fullsized":"2023-12-19T16:57:23.105795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:17.631958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114802+0000","last_change":"2023-12-19T16:57:23.135384+0000","last_active":"2023-12-19T16:57:24.114802+0000","last_peered":"2023-12-19T16:57:24.114802+0000","last_clean":"2023-12-19T16:57:24.114802+0000","last_became_active":"2023-12-19T16:57:23.135061+0000","last_became_peered":"2023-12-19T16:57:23.135061+0000","last_unstale":"2023-12-19T16:57:24.114802+0000","last_undegraded":"2023-12-19T16:57:24.114802+0000","last_fullsized":"2023-12-19T16:57:24.114802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:47.222111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.114572+0000","last_change":"2023-12-19T16:57:22.713968+0000","last_active":"2023-12-19T16:57:23.114572+0000","last_peered":"2023-12-19T16:57:23.114572+0000","last_clean":"2023-12-19T16:57:23.114572+0000","last_became_active":"2023-12-19T16:57:22.713679+0000","last_became_peered":"2023-12-19T16:57:22.713679+0000","last_unstale":"2023-12-19T16:57:23.114572+0000","last_undegraded":"2023-12-19T16:57:23.114572+0000","last_fullsized":"2023-12-19T16:57:23.114572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:21:53.731379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.129117+0000","last_change":"2023-12-19T16:57:23.116246+0000","last_active":"2023-12-19T16:57:23.129117+0000","last_peered":"2023-12-19T16:57:23.129117+0000","last_clean":"2023-12-19T16:57:23.129117+0000","last_became_active":"2023-12-19T16:57:23.115985+0000","last_became_peered":"2023-12-19T16:57:23.115985+0000","last_unstale":"2023-12-19T16:57:23.129117+0000","last_undegraded":"2023-12-19T16:57:23.129117+0000","last_fullsized":"2023-12-19T16:57:23.129117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:54:16.935719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.105796+0000","last_change":"2023-12-19T16:57:22.712441+0000","last_active":"2023-12-19T16:57:23.105796+0000","last_peered":"2023-12-19T16:57:23.105796+0000","last_clean":"2023-12-19T16:57:23.105796+0000","last_became_active":"2023-12-19T16:57:22.711466+0000","last_became_peered":"2023-12-19T16:57:22.711466+0000","last_unstale":"2023-12-19T16:57:23.105796+0000","last_undegraded":"2023-12-19T16:57:23.105796+0000","last_fullsized":"2023-12-19T16:57:23.105796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:24.929320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136330+0000","last_change":"2023-12-19T16:57:23.129224+0000","last_active":"2023-12-19T16:57:23.136330+0000","last_peered":"2023-12-19T16:57:23.136330+0000","last_clean":"2023-12-19T16:57:23.136330+0000","last_became_active":"2023-12-19T16:57:23.128803+0000","last_became_peered":"2023-12-19T16:57:23.128803+0000","last_unstale":"2023-12-19T16:57:23.136330+0000","last_undegraded":"2023-12-19T16:57:23.136330+0000","last_fullsized":"2023-12-19T16:57:23.136330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:54:45.499902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.112078+0000","last_change":"2023-12-19T16:57:22.705820+0000","last_active":"2023-12-19T16:57:23.112078+0000","last_peered":"2023-12-19T16:57:23.112078+0000","last_clean":"2023-12-19T16:57:23.112078+0000","last_became_active":"2023-12-19T16:57:22.704940+0000","last_became_peered":"2023-12-19T16:57:22.704940+0000","last_unstale":"2023-12-19T16:57:23.112078+0000","last_undegraded":"2023-12-19T16:57:23.112078+0000","last_fullsized":"2023-12-19T16:57:23.112078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:12.535457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.139807+0000","last_change":"2023-12-19T16:57:23.136635+0000","last_active":"2023-12-19T16:57:23.139807+0000","last_peered":"2023-12-19T16:57:23.139807+0000","last_clean":"2023-12-19T16:57:23.139807+0000","last_became_active":"2023-12-19T16:57:23.135238+0000","last_became_peered":"2023-12-19T16:57:23.135238+0000","last_unstale":"2023-12-19T16:57:23.139807+0000","last_undegraded":"2023-12-19T16:57:23.139807+0000","last_fullsized":"2023-12-19T16:57:23.139807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:47.059374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114150+0000","last_change":"2023-12-19T16:57:22.717887+0000","last_active":"2023-12-19T16:57:24.114150+0000","last_peered":"2023-12-19T16:57:24.114150+0000","last_clean":"2023-12-19T16:57:24.114150+0000","last_became_active":"2023-12-19T16:57:22.717600+0000","last_became_peered":"2023-12-19T16:57:22.717600+0000","last_unstale":"2023-12-19T16:57:24.114150+0000","last_undegraded":"2023-12-19T16:57:24.114150+0000","last_fullsized":"2023-12-19T16:57:24.114150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:23:04.958220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"67'161","reported_seq":375,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.117050+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:57:23.117050+0000","last_peered":"2023-12-19T16:57:23.117050+0000","last_clean":"2023-12-19T16:57:23.117050+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:57:23.117050+0000","last_undegraded":"2023-12-19T16:57:23.117050+0000","last_fullsized":"2023-12-19T16:57:23.117050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509623,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1536000,"data_stored":1044967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.35699999999999998,"15min":0.30199999999999999},"max":{"1min":0.95699999999999996,"5min":1.3460000000000001,"15min":2.6389999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73599999999999999,"15min":0.72199999999999998},"min":{"1min":0.58699999999999997,"5min":0.45000000000000001,"15min":0.32700000000000001},"max":{"1min":0.91000000000000003,"5min":1.6930000000000001,"15min":2.532},"last":0.42199999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.75900000000000001,"15min":0.72299999999999998},"min":{"1min":0.67400000000000004,"5min":0.436,"15min":0.36399999999999999},"max":{"1min":0.93000000000000005,"5min":1.212,"15min":1.212},"last":0.436},{"interface":"front","average":{"1min":0.751,"5min":0.745,"15min":0.72799999999999998},"min":{"1min":0.64800000000000002,"5min":0.438,"15min":0.36799999999999999},"max":{"1min":0.86599999999999999,"5min":1.74,"15min":1.74},"last":0.47199999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72999999999999998,"15min":0.70599999999999996},"min":{"1min":0.61799999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84299999999999997,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.76600000000000001,"15min":0.73699999999999999},"min":{"1min":0.60999999999999999,"5min":0.47299999999999998,"15min":0.36599999999999999},"max":{"1min":0.95799999999999996,"5min":1.395,"15min":1.395},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.74299999999999999,"15min":0.72099999999999997},"min":{"1min":0.61899999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88600000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.76000000000000001,"15min":0.75},"min":{"1min":0.65300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90800000000000003,"5min":1.266,"15min":1.266},"last":0.73499999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.45000000000000001,"5min":0.32700000000000001,"15min":0.28499999999999998},"max":{"1min":0.78800000000000003,"5min":1.841,"15min":1.841},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.47499999999999998,"5min":0.40200000000000002,"15min":0.36599999999999999},"max":{"1min":0.69899999999999995,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.53500000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.48999999999999999,"5min":0.36899999999999999,"15min":0.35699999999999998},"max":{"1min":0.76800000000000002,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.66000000000000003,"15min":0.64400000000000002},"min":{"1min":0.503,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73099999999999998,"5min":1.784,"15min":1.784},"last":0.68600000000000005}]},{"osd":6,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70499999999999996,"15min":0.69199999999999995},"min":{"1min":0.57199999999999995,"5min":0.45700000000000002,"15min":0.39800000000000002},"max":{"1min":0.879,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.52700000000000002,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.70299999999999996,"5min":2.105,"15min":2.105},"last":0.72199999999999998}]}]},{"osd":6,"up_from":15,"seq":64424509623,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":933888,"data_stored":454439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70099999999999996,"15min":0.69799999999999995},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.309},"max":{"1min":0.88200000000000001,"5min":0.999,"15min":1.8600000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":1.0089999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.63600000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71199999999999997,"15min":0.73399999999999999},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.33400000000000002},"max":{"1min":0.88400000000000001,"5min":0.99399999999999999,"15min":4.3799999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73499999999999999,"15min":0.73699999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.39500000000000002},"max":{"1min":0.91900000000000004,"5min":1.0189999999999999,"15min":1.583},"last":0.75}]},{"osd":2,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.755,"15min":0.74299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.35199999999999998},"max":{"1min":0.95899999999999996,"5min":1.087,"15min":1.3180000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.58299999999999996,"5min":0.48199999999999998,"15min":0.44400000000000001},"max":{"1min":0.98199999999999998,"5min":1.0549999999999999,"15min":1.1879999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76100000000000001,"15min":0.80500000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97499999999999998,"5min":1.1020000000000001,"15min":4.673},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.54400000000000004,"5min":0.42699999999999999,"15min":0.41399999999999998},"max":{"1min":0.94199999999999995,"5min":1.0740000000000001,"15min":1.962},"last":0.627}]},{"osd":4,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.35899999999999999},"max":{"1min":0.69999999999999996,"5min":0.89500000000000002,"15min":0.96999999999999997},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61399999999999999,"15min":0.60199999999999998},"min":{"1min":0.45000000000000001,"5min":0.39800000000000002,"15min":0.35699999999999998},"max":{"1min":0.878,"5min":0.93500000000000005,"15min":1.006},"last":0.77300000000000002}]},{"osd":5,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.60399999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.33300000000000002},"max":{"1min":0.80300000000000005,"5min":0.91300000000000003,"15min":1.2050000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68100000000000005,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.41499999999999998},"max":{"1min":0.96299999999999997,"5min":1.046,"15min":1.046},"last":0.80300000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.49299999999999999,"15min":0.438},"max":{"1min":1.006,"5min":1.0249999999999999,"15min":1.0620000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71299999999999997,"15min":0.70599999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":1.032,"15min":1.1220000000000001},"last":0.73799999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509623,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":942080,"data_stored":455091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.59099999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.876,"5min":0.88400000000000001,"15min":0.95799999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73999999999999999,"15min":0.72399999999999998},"min":{"1min":0.57499999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93899999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57199999999999995}]},{"osd":1,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73599999999999999,"15min":0.73899999999999999},"min":{"1min":0.60699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.58599999999999997,"5min":0.374,"15min":0.374},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.89200000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.75,"15min":0.747},"min":{"1min":0.60299999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.026,"5min":1.0409999999999999,"15min":1.1120000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.59199999999999997,"5min":0.5,"15min":0.46400000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.087},"last":0.81999999999999995}]},{"osd":3,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.626,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.77300000000000002,"15min":0.76200000000000001},"min":{"1min":0.56699999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.085,"5min":1.089,"15min":1.089},"last":0.998}]},{"osd":4,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.41199999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":0.83999999999999997,"15min":1.165},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.434,"5min":0.40799999999999997,"15min":0.32400000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.0649999999999999},"last":0.91100000000000003}]},{"osd":6,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.89500000000000002,"5min":0.98799999999999999,"15min":1.153},"last":0.62},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.46800000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.98999999999999999},"last":0.86599999999999999}]},{"osd":7,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64900000000000002,"15min":0.65300000000000002},"min":{"1min":0.47999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.133},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67900000000000005,"15min":0.67800000000000005},"min":{"1min":0.49199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.014,"5min":1.131,"15min":1.131},"last":0.84299999999999997}]}]},{"osd":4,"up_from":15,"seq":64424509623,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54116,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317724,"statfs":{"total":96636764160,"available":96581349376,"internally_reserved":0,"allocated":946176,"data_stored":456003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.72599999999999998,"15min":0.71199999999999997},"min":{"1min":0.66500000000000004,"5min":0.316,"15min":0.316},"max":{"1min":0.93300000000000005,"5min":1.877,"15min":1.877},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70899999999999996,"15min":0.69999999999999996},"min":{"1min":0.502,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.94899999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75600000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.34799999999999998},"max":{"1min":0.93799999999999994,"5min":1.0209999999999999,"15min":1.8700000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.745,"15min":0.73599999999999999},"min":{"1min":0.52300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92100000000000004,"5min":2.02,"15min":2.02},"last":0.77800000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.67600000000000005,"5min":0.47099999999999997,"15min":0.38700000000000001},"max":{"1min":0.90600000000000003,"5min":1.375,"15min":1.375},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.77000000000000002,"15min":0.75700000000000001},"min":{"1min":0.65500000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89200000000000002,"5min":1.927,"15min":1.927},"last":0.73899999999999999}]},{"osd":3,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71699999999999997,"15min":0.71299999999999997},"min":{"1min":0.53200000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90500000000000003,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.59999999999999998,"5min":0.52300000000000002,"15min":0.49399999999999999},"max":{"1min":0.94799999999999995,"5min":0.97599999999999998,"15min":1.3129999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.35899999999999999},"max":{"1min":0.77600000000000002,"5min":1.99,"15min":1.99},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.45300000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81100000000000005,"5min":0.89300000000000002,"15min":0.92800000000000005},"last":0.64400000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59799999999999998,"15min":0.621},"min":{"1min":0.45000000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72099999999999997,"5min":0.872,"15min":1.0820000000000001},"last":0.623},{"interface":"front","average":{"1min":0.629,"5min":0.65300000000000002,"15min":0.66600000000000004},"min":{"1min":0.46100000000000002,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":0.81299999999999994,"5min":1.96,"15min":1.96},"last":0.80800000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.434,"5min":0.376,"15min":0.376},"max":{"1min":0.88500000000000001,"5min":0.93200000000000005,"15min":1.0429999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68500000000000005,"15min":0.70599999999999996},"min":{"1min":0.50900000000000001,"5min":0.46400000000000002,"15min":0.44900000000000001},"max":{"1min":0.92300000000000004,"5min":1.103,"15min":1.107},"last":0.73099999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509623,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36064,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335776,"statfs":{"total":96636764160,"available":96599834624,"internally_reserved":0,"allocated":942080,"data_stored":454495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45000000000000001,"15min":0.45300000000000001},"min":{"1min":0.315,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69199999999999995,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.49099999999999999,"15min":0.51000000000000001},"min":{"1min":0.36699999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.75900000000000001,"5min":1.163,"15min":1.738},"last":0.441}]},{"osd":1,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.46800000000000003},"min":{"1min":0.34899999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67500000000000004,"5min":0.77800000000000002,"15min":1.141},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67200000000000004,"5min":1.952,"15min":2.4910000000000001},"last":0.45800000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.55700000000000005},"min":{"1min":0.38700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76800000000000002,"5min":0.85399999999999998,"15min":1.821},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56899999999999995,"15min":0.60199999999999998},"min":{"1min":0.45700000000000002,"5min":0.372,"15min":0.35399999999999998},"max":{"1min":0.73799999999999999,"5min":2.855,"15min":5.1219999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78500000000000003,"15min":0.79500000000000004},"min":{"1min":0.56399999999999995,"5min":0.432,"15min":0.432},"max":{"1min":0.95899999999999996,"5min":4.234,"15min":4.234},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.754,"15min":0.79300000000000004},"min":{"1min":0.61899999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":1.111,"15min":2.2690000000000001},"last":0.57099999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76200000000000001,"15min":0.79300000000000004},"min":{"1min":0.55800000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95399999999999996,"5min":1.0640000000000001,"15min":2.5350000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97799999999999998,"5min":1.8300000000000001,"15min":1.9079999999999999},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.78700000000000003,"15min":0.81299999999999994},"min":{"1min":0.65200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0589999999999999,"5min":1.599,"15min":1.956},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78000000000000003,"15min":0.80600000000000005},"min":{"1min":0.56799999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.99199999999999999,"5min":1.129,"15min":1.974},"last":0.60099999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81299999999999994,"15min":0.84099999999999997},"min":{"1min":0.64500000000000002,"5min":0.54700000000000004,"15min":0.51900000000000002},"max":{"1min":1.077,"5min":1.1279999999999999,"15min":1.9279999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.84299999999999997,"15min":0.88600000000000001},"min":{"1min":0.70899999999999996,"5min":0.60599999999999998,"15min":0.48799999999999999},"max":{"1min":1.0680000000000001,"5min":1.623,"15min":5.0759999999999996},"last":0.61899999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509622,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40160,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331680,"statfs":{"total":96636764160,"available":96595640320,"internally_reserved":0,"allocated":942080,"data_stored":454495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.49299999999999999,"15min":0.47799999999999998},"min":{"1min":0.34399999999999997,"5min":0.307,"15min":0.28199999999999997},"max":{"1min":0.64100000000000001,"5min":5.4870000000000001,"15min":5.4870000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.59899999999999998,"15min":0.53800000000000003},"min":{"1min":0.39100000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72299999999999998,"5min":10.161,"15min":10.161},"last":0.38200000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.64700000000000002},"min":{"1min":0.52800000000000002,"5min":0.41099999999999998,"15min":0.39900000000000002},"max":{"1min":0.82799999999999996,"5min":2.4740000000000002,"15min":5.7590000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":2.911,"15min":5.8070000000000004},"last":0.40899999999999997}]},{"osd":3,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.45000000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.753,"5min":1.1990000000000001,"15min":2.7599999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.59499999999999997,"15min":0.61099999999999999},"min":{"1min":0.47499999999999998,"5min":0.41099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":2.7999999999999998,"15min":5.7190000000000003},"last":0.44400000000000001}]},{"osd":4,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.81100000000000005},"min":{"1min":0.63600000000000001,"5min":0.57999999999999996,"15min":0.53000000000000003},"max":{"1min":1.143,"5min":2.4449999999999998,"15min":3.8940000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81200000000000006,"15min":0.80100000000000005},"min":{"1min":0.61699999999999999,"5min":0.53700000000000003,"15min":0.42999999999999999},"max":{"1min":1.0600000000000001,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.84299999999999997,"15min":0.81200000000000006},"min":{"1min":0.65500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.004,"5min":5.601,"15min":5.601},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.81499999999999995,"15min":0.84099999999999997},"min":{"1min":0.57699999999999996,"5min":0.55100000000000005,"15min":0.54900000000000004},"max":{"1min":1.02,"5min":2.569,"15min":2.9260000000000002},"last":0.70499999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.84099999999999997,"15min":0.83999999999999997},"min":{"1min":0.65200000000000002,"5min":0.55900000000000005,"15min":0.52400000000000002},"max":{"1min":1.0780000000000001,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77600000000000002,"15min":0.80900000000000005},"min":{"1min":0.59799999999999998,"5min":0.53800000000000003,"15min":0.505},"max":{"1min":1.0940000000000001,"5min":1.1739999999999999,"15min":2.121},"last":0.83699999999999997}]},{"osd":7,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.872,"15min":0.83599999999999997},"min":{"1min":0.61399999999999999,"5min":0.56699999999999995,"15min":0.52300000000000002},"max":{"1min":1.0369999999999999,"5min":5.673,"15min":5.673},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.82999999999999996,"15min":0.85299999999999998},"min":{"1min":0.60899999999999999,"5min":0.60599999999999998,"15min":0.57799999999999996},"max":{"1min":1.123,"5min":1.429,"15min":2.0529999999999999},"last":0.80100000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509623,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50012,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321828,"statfs":{"total":96636764160,"available":96585551872,"internally_reserved":0,"allocated":937984,"data_stored":454529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.432,"15min":0.45500000000000002},"min":{"1min":0.30499999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70399999999999996,"15min":2.3050000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.52100000000000002},"min":{"1min":0.34599999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.74299999999999999,"5min":0.84199999999999997,"15min":0.89800000000000002},"last":0.371}]},{"osd":2,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.496,"15min":0.51900000000000002},"min":{"1min":0.37,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66500000000000004,"5min":0.80100000000000005,"15min":1.992},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.40300000000000002,"5min":0.35999999999999999,"15min":0.29799999999999999},"max":{"1min":0.81899999999999995,"5min":0.82899999999999996,"15min":0.97299999999999998},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59399999999999997,"15min":0.63200000000000001},"min":{"1min":0.48199999999999998,"5min":0.36899999999999999,"15min":0.34799999999999998},"max":{"1min":0.871,"5min":0.871,"15min":4.133},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36499999999999999,"5min":0.35799999999999998,"15min":0.33700000000000002},"max":{"1min":0.755,"5min":0.76400000000000001,"15min":0.91900000000000004},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.77300000000000002},"min":{"1min":0.53400000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94099999999999995,"5min":1.202,"15min":2.1949999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.56899999999999995,"5min":0.44400000000000001,"15min":0.42399999999999999},"max":{"1min":1.0580000000000001,"5min":1.181,"15min":2.048},"last":0.70999999999999996}]},{"osd":5,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77600000000000002,"15min":0.79200000000000004},"min":{"1min":0.60699999999999998,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.014,"5min":1.288,"15min":2.1269999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79700000000000004,"15min":0.82499999999999996},"min":{"1min":0.54700000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.016,"5min":1.3380000000000001,"15min":4.0460000000000003},"last":0.97499999999999998}]},{"osd":6,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.80600000000000005},"min":{"1min":0.496,"5min":0.48299999999999998,"15min":0.41999999999999998},"max":{"1min":1.048,"5min":1.0740000000000001,"15min":2.1499999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.59499999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92400000000000004,"5min":1.391,"15min":2.0880000000000001},"last":0.95399999999999996}]},{"osd":7,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83299999999999996,"15min":0.85899999999999999},"min":{"1min":0.66900000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.002,"5min":1.2230000000000001,"15min":4.093},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.81399999999999995,"15min":0.84499999999999997},"min":{"1min":0.65900000000000003,"5min":0.58999999999999997,"15min":0.53800000000000003},"max":{"1min":1.097,"5min":1.147,"15min":2.2349999999999999},"last":0.90700000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542328,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1544192,"data_stored":1046953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.43099999999999999,"15min":0.42699999999999999},"min":{"1min":0.34999999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.504,"15min":0.48799999999999999},"min":{"1min":0.42999999999999999,"5min":0.28299999999999997,"15min":0.26300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.85499999999999998},"last":0.50800000000000001}]},{"osd":2,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.43099999999999999,"5min":0.35899999999999999,"15min":0.29299999999999998},"max":{"1min":0.67900000000000005,"5min":0.77800000000000002,"15min":0.89700000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.44500000000000001,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46000000000000002}]},{"osd":3,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.38400000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.77700000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.434},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.49399999999999999,"5min":0.41999999999999998,"15min":0.40899999999999997},"max":{"1min":0.83799999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55000000000000004}]},{"osd":4,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.79400000000000004,"15min":0.78900000000000003},"min":{"1min":0.58599999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2090000000000001,"5min":1.444,"15min":1.512},"last":0.626},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.80400000000000005,"15min":0.78900000000000003},"min":{"1min":0.61499999999999999,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.1020000000000001,"5min":1.177,"15min":1.52},"last":0.56599999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.80900000000000005,"15min":0.80400000000000005},"min":{"1min":0.59099999999999997,"5min":0.502,"15min":0.41499999999999998},"max":{"1min":1.1599999999999999,"5min":1.294,"15min":1.748},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.81699999999999995,"15min":0.79500000000000004},"min":{"1min":0.65300000000000002,"5min":0.55000000000000004,"15min":0.51600000000000001},"max":{"1min":1.097,"5min":1.256,"15min":1.649},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.82299999999999995,"15min":0.79200000000000004},"min":{"1min":0.56200000000000006,"5min":0.52300000000000002,"15min":0.46999999999999997},"max":{"1min":1.2509999999999999,"5min":1.429,"15min":1.429},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.82199999999999995,"15min":0.81299999999999994},"min":{"1min":0.628,"5min":0.54200000000000004,"15min":0.44700000000000001},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.66000000000000003}]},{"osd":7,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.84699999999999998,"15min":0.82099999999999995},"min":{"1min":0.68500000000000005,"5min":0.505,"15min":0.48999999999999999},"max":{"1min":1.3129999999999999,"5min":1.377,"15min":1.377},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.83499999999999996,"15min":0.81799999999999995},"min":{"1min":0.62,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2649999999999999,"5min":1.3300000000000001,"15min":1.661},"last":0.70699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:57:27.228 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:57:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.405+0000 7fa2e1ff3640 1 Processor -- start 2023-12-19T16:57:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.405+0000 7fa2e1ff3640 1 -- start start 2023-12-19T16:57:27.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.405+0000 7fa2e1ff3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2dc06cb30 0x7fa2dc0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.405+0000 7fa2e1ff3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2dc15ddc0 con 0x7fa2dc15d2b0 2023-12-19T16:57:27.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.405+0000 7fa2e1ff3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2dc15e0a0 con 0x7fa2dc0b34e0 2023-12-19T16:57:27.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.405+0000 7fa2e1ff3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2dc0cb450 con 0x7fa2dc06cb30 2023-12-19T16:57:27.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.405+0000 7fa2db7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2dc06cb30 0x7fa2dc0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.405+0000 7fa2db7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa2dc06cb30 0x7fa2dc0cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52820/0 (socket says 172.21.15.26:52820) 2023-12-19T16:57:27.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.405+0000 7fa2db7fe640 1 -- 172.21.15.26:0/1296002280 learned_addr learned my addr 172.21.15.26:0/1296002280 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:27.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.405+0000 7fa2daffd640 1 --1- 172.21.15.26:0/1296002280 >> v1:172.21.15.26:6789/0 conn(0x7fa2dc15d2b0 0x7fa2dc0b2dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53640/0 (socket says 172.21.15.26:53640) 2023-12-19T16:57:27.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.409+0000 7fa2db7fe640 1 -- 172.21.15.26:0/1296002280 >> v1:172.21.15.160:6789/0 conn(0x7fa2dc0b34e0 legacy=0x7fa2dc0ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:27.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.409+0000 7fa2db7fe640 1 -- 172.21.15.26:0/1296002280 >> v1:172.21.15.26:6789/0 conn(0x7fa2dc15d2b0 legacy=0x7fa2dc0b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.409+0000 7fa2db7fe640 1 -- 172.21.15.26:0/1296002280 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc0cb730 con 0x7fa2dc06cb30 2023-12-19T16:57:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.409+0000 7fa2db7fe640 1 --2- 172.21.15.26:0/1296002280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2dc06cb30 0x7fa2dc0cbc50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009870 tx=0x7fa2cc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e290175339af3a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.409+0000 7fa2da7fc640 1 -- 172.21.15.26:0/1296002280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0aa680 con 0x7fa2dc06cb30 2023-12-19T16:57:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.409+0000 7fa2da7fc640 1 -- 172.21.15.26:0/1296002280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc0aa7e0 con 0x7fa2dc06cb30 2023-12-19T16:57:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.409+0000 7fa2da7fc640 1 -- 172.21.15.26:0/1296002280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0aab70 con 0x7fa2dc06cb30 2023-12-19T16:57:27.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1296002280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2dc06cb30 msgr2=0x7fa2dc0cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 --2- 172.21.15.26:0/1296002280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2dc06cb30 0x7fa2dc0cbc50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009870 tx=0x7fa2cc01c500 comp rx=0 tx=0).stop 2023-12-19T16:57:27.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1296002280 shutdown_connections 2023-12-19T16:57:27.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 --2- 172.21.15.26:0/1296002280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2dc06cb30 0x7fa2dc0cbc50 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1296002280 >> 172.21.15.26:0/1296002280 conn(0x7fa2dc0658e0 msgr2=0x7fa2dc065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:27.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1296002280 shutdown_connections 2023-12-19T16:57:27.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1296002280 wait complete. 2023-12-19T16:57:27.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 Processor -- start 2023-12-19T16:57:27.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 -- start start 2023-12-19T16:57:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2dc06cb30 0x7fa2dc158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2dc0b34e0 0x7fa2dc159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2dc15d2b0 0x7fa2dc152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc15a740 con 0x7fa2dc0b34e0 2023-12-19T16:57:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc070640 con 0x7fa2dc06cb30 2023-12-19T16:57:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2e1ff3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc157b20 con 0x7fa2dc15d2b0 2023-12-19T16:57:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2daffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2dc0b34e0 0x7fa2dc159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2daffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2dc0b34e0 0x7fa2dc159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52826/0 (socket says 172.21.15.26:52826) 2023-12-19T16:57:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2daffd640 1 -- 172.21.15.26:0/1790329258 learned_addr learned my addr 172.21.15.26:0/1790329258 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2db7fe640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2dc06cb30 0x7fa2dc158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.413+0000 7fa2dbfff640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2dc15d2b0 0x7fa2dc152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2daffd640 1 -- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2dc15d2b0 msgr2=0x7fa2dc152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2daffd640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2dc15d2b0 0x7fa2dc152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2daffd640 1 -- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2dc06cb30 msgr2=0x7fa2dc158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2daffd640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2dc06cb30 0x7fa2dc158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2daffd640 1 -- 172.21.15.26:0/1790329258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0094d0 con 0x7fa2dc0b34e0 2023-12-19T16:57:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2daffd640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2dc0b34e0 0x7fa2dc159000 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000c460 tx=0x7fa2d000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:27.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2c3fff640 1 -- 172.21.15.26:0/1790329258 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0007480 con 0x7fa2dc0b34e0 2023-12-19T16:57:27.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2c3fff640 1 -- 172.21.15.26:0/1790329258 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d00075e0 con 0x7fa2dc0b34e0 2023-12-19T16:57:27.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2c3fff640 1 -- 172.21.15.26:0/1790329258 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0004a10 con 0x7fa2dc0b34e0 2023-12-19T16:57:27.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1790329258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2dc157e00 con 0x7fa2dc0b34e0 2023-12-19T16:57:27.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1790329258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2dc1583c0 con 0x7fa2dc0b34e0 2023-12-19T16:57:27.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1790329258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2dc074270 con 0x7fa2dc0b34e0 2023-12-19T16:57:27.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2c3fff640 1 -- 172.21.15.26:0/1790329258 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa2d0004b70 con 0x7fa2dc0b34e0 2023-12-19T16:57:27.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.417+0000 7fa2c3fff640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2c8065b80 0x7fa2c8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.421+0000 7fa2c3fff640 1 -- 172.21.15.26:0/1790329258 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fa2d0082730 con 0x7fa2dc0b34e0 2023-12-19T16:57:27.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.421+0000 7fa2db7fe640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2c8065b80 0x7fa2c8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.421+0000 7fa2db7fe640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2c8065b80 0x7fa2c8068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0ae000 tx=0x7fa2cc0b5040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:27.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.421+0000 7fa2c3fff640 1 -- 172.21.15.26:0/1790329258 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa2d004fd10 con 0x7fa2dc0b34e0 2023-12-19T16:57:27.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.581+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1790329258 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa2dc160ae0 con 0x7fa2c8065b80 2023-12-19T16:57:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.589+0000 7fa2c3fff640 1 -- 172.21.15.26:0/1790329258 <== mgr.5606 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288902 (secure 0 0 0) 0x7fa2cc0aa9d0 con 0x7fa2c8065b80 2023-12-19T16:57:27.596 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:57:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.593+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2c8065b80 msgr2=0x7fa2c8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.593+0000 7fa2e1ff3640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2c8065b80 0x7fa2c8068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0ae000 tx=0x7fa2cc0b5040 comp rx=0 tx=0).stop 2023-12-19T16:57:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.593+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1790329258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2dc0b34e0 msgr2=0x7fa2dc159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.593+0000 7fa2e1ff3640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2dc0b34e0 0x7fa2dc159000 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000c460 tx=0x7fa2d000c920 comp rx=0 tx=0).stop 2023-12-19T16:57:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.593+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1790329258 shutdown_connections 2023-12-19T16:57:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.593+0000 7fa2e1ff3640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa2c8065b80 0x7fa2c8068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.593+0000 7fa2e1ff3640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa2dc15d2b0 0x7fa2dc152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.593+0000 7fa2e1ff3640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2dc0b34e0 0x7fa2dc159000 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.593+0000 7fa2e1ff3640 1 --2- 172.21.15.26:0/1790329258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2dc06cb30 0x7fa2dc158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.593+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1790329258 >> 172.21.15.26:0/1790329258 conn(0x7fa2dc0658e0 msgr2=0x7fa2dc15e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.593+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1790329258 shutdown_connections 2023-12-19T16:57:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.593+0000 7fa2e1ff3640 1 -- 172.21.15.26:0/1790329258 wait complete. 2023-12-19T16:57:27.616 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-19T16:57:26.422389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":309384,"kb_used_data":8520,"kb_used_omap":0,"kb_used_meta":300800,"kb_avail":754665336,"statfs":{"total":773094113280,"available":772777304064,"internally_reserved":0,"allocated":8724480,"data_stored":4820972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308019200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.738191"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113601+0000","last_change":"2023-12-19T16:57:23.130736+0000","last_active":"2023-12-19T16:57:24.113601+0000","last_peered":"2023-12-19T16:57:24.113601+0000","last_clean":"2023-12-19T16:57:24.113601+0000","last_became_active":"2023-12-19T16:57:23.130370+0000","last_became_peered":"2023-12-19T16:57:23.130370+0000","last_unstale":"2023-12-19T16:57:24.113601+0000","last_undegraded":"2023-12-19T16:57:24.113601+0000","last_fullsized":"2023-12-19T16:57:24.113601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:14:33.246576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138549+0000","last_change":"2023-12-19T16:57:23.134802+0000","last_active":"2023-12-19T16:57:23.138549+0000","last_peered":"2023-12-19T16:57:23.138549+0000","last_clean":"2023-12-19T16:57:23.138549+0000","last_became_active":"2023-12-19T16:57:23.132831+0000","last_became_peered":"2023-12-19T16:57:23.132831+0000","last_unstale":"2023-12-19T16:57:23.138549+0000","last_undegraded":"2023-12-19T16:57:23.138549+0000","last_fullsized":"2023-12-19T16:57:23.138549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:30:52.952843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114747+0000","last_change":"2023-12-19T16:57:23.130998+0000","last_active":"2023-12-19T16:57:24.114747+0000","last_peered":"2023-12-19T16:57:24.114747+0000","last_clean":"2023-12-19T16:57:24.114747+0000","last_became_active":"2023-12-19T16:57:23.130698+0000","last_became_peered":"2023-12-19T16:57:23.130698+0000","last_unstale":"2023-12-19T16:57:24.114747+0000","last_undegraded":"2023-12-19T16:57:24.114747+0000","last_fullsized":"2023-12-19T16:57:24.114747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:46.510340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135506+0000","last_change":"2023-12-19T16:57:23.128782+0000","last_active":"2023-12-19T16:57:23.135506+0000","last_peered":"2023-12-19T16:57:23.135506+0000","last_clean":"2023-12-19T16:57:23.135506+0000","last_became_active":"2023-12-19T16:57:23.128460+0000","last_became_peered":"2023-12-19T16:57:23.128460+0000","last_unstale":"2023-12-19T16:57:23.135506+0000","last_undegraded":"2023-12-19T16:57:23.135506+0000","last_fullsized":"2023-12-19T16:57:23.135506+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:41.666771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113211+0000","last_change":"2023-12-19T16:57:23.136824+0000","last_active":"2023-12-19T16:57:24.113211+0000","last_peered":"2023-12-19T16:57:24.113211+0000","last_clean":"2023-12-19T16:57:24.113211+0000","last_became_active":"2023-12-19T16:57:23.136559+0000","last_became_peered":"2023-12-19T16:57:23.136559+0000","last_unstale":"2023-12-19T16:57:24.113211+0000","last_undegraded":"2023-12-19T16:57:24.113211+0000","last_fullsized":"2023-12-19T16:57:24.113211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:17.304496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135720+0000","last_change":"2023-12-19T16:57:23.123111+0000","last_active":"2023-12-19T16:57:23.135720+0000","last_peered":"2023-12-19T16:57:23.135720+0000","last_clean":"2023-12-19T16:57:23.135720+0000","last_became_active":"2023-12-19T16:57:23.122673+0000","last_became_peered":"2023-12-19T16:57:23.122673+0000","last_unstale":"2023-12-19T16:57:23.135720+0000","last_undegraded":"2023-12-19T16:57:23.135720+0000","last_fullsized":"2023-12-19T16:57:23.135720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:18.315096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.129268+0000","last_change":"2023-12-19T16:57:23.126244+0000","last_active":"2023-12-19T16:57:23.129268+0000","last_peered":"2023-12-19T16:57:23.129268+0000","last_clean":"2023-12-19T16:57:23.129268+0000","last_became_active":"2023-12-19T16:57:23.125955+0000","last_became_peered":"2023-12-19T16:57:23.125955+0000","last_unstale":"2023-12-19T16:57:23.129268+0000","last_undegraded":"2023-12-19T16:57:23.129268+0000","last_fullsized":"2023-12-19T16:57:23.129268+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:29:23.721263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.132545+0000","last_change":"2023-12-19T16:57:23.126053+0000","last_active":"2023-12-19T16:57:23.132545+0000","last_peered":"2023-12-19T16:57:23.132545+0000","last_clean":"2023-12-19T16:57:23.132545+0000","last_became_active":"2023-12-19T16:57:23.125779+0000","last_became_peered":"2023-12-19T16:57:23.125779+0000","last_unstale":"2023-12-19T16:57:23.132545+0000","last_undegraded":"2023-12-19T16:57:23.132545+0000","last_fullsized":"2023-12-19T16:57:23.132545+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:48.226902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135992+0000","last_change":"2023-12-19T16:57:23.132976+0000","last_active":"2023-12-19T16:57:23.135992+0000","last_peered":"2023-12-19T16:57:23.135992+0000","last_clean":"2023-12-19T16:57:23.135992+0000","last_became_active":"2023-12-19T16:57:23.132486+0000","last_became_peered":"2023-12-19T16:57:23.132486+0000","last_unstale":"2023-12-19T16:57:23.135992+0000","last_undegraded":"2023-12-19T16:57:23.135992+0000","last_fullsized":"2023-12-19T16:57:23.135992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:55.558144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.120451+0000","last_change":"2023-12-19T16:57:23.116398+0000","last_active":"2023-12-19T16:57:23.120451+0000","last_peered":"2023-12-19T16:57:23.120451+0000","last_clean":"2023-12-19T16:57:23.120451+0000","last_became_active":"2023-12-19T16:57:23.115111+0000","last_became_peered":"2023-12-19T16:57:23.115111+0000","last_unstale":"2023-12-19T16:57:23.120451+0000","last_undegraded":"2023-12-19T16:57:23.120451+0000","last_fullsized":"2023-12-19T16:57:23.120451+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:41.666771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.141084+0000","last_change":"2023-12-19T16:57:23.135226+0000","last_active":"2023-12-19T16:57:23.141084+0000","last_peered":"2023-12-19T16:57:23.141084+0000","last_clean":"2023-12-19T16:57:23.141084+0000","last_became_active":"2023-12-19T16:57:23.134983+0000","last_became_peered":"2023-12-19T16:57:23.134983+0000","last_unstale":"2023-12-19T16:57:23.141084+0000","last_undegraded":"2023-12-19T16:57:23.141084+0000","last_fullsized":"2023-12-19T16:57:23.141084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:50:43.426372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136632+0000","last_change":"2023-12-19T16:57:23.132583+0000","last_active":"2023-12-19T16:57:23.136632+0000","last_peered":"2023-12-19T16:57:23.136632+0000","last_clean":"2023-12-19T16:57:23.136632+0000","last_became_active":"2023-12-19T16:57:23.131805+0000","last_became_peered":"2023-12-19T16:57:23.131805+0000","last_unstale":"2023-12-19T16:57:23.136632+0000","last_undegraded":"2023-12-19T16:57:23.136632+0000","last_fullsized":"2023-12-19T16:57:23.136632+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:40:55.499800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114226+0000","last_change":"2023-12-19T16:57:23.124679+0000","last_active":"2023-12-19T16:57:24.114226+0000","last_peered":"2023-12-19T16:57:24.114226+0000","last_clean":"2023-12-19T16:57:24.114226+0000","last_became_active":"2023-12-19T16:57:23.123923+0000","last_became_peered":"2023-12-19T16:57:23.123923+0000","last_unstale":"2023-12-19T16:57:24.114226+0000","last_undegraded":"2023-12-19T16:57:24.114226+0000","last_fullsized":"2023-12-19T16:57:24.114226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:24:44.642159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.126977+0000","last_change":"2023-12-19T16:57:23.125493+0000","last_active":"2023-12-19T16:57:23.126977+0000","last_peered":"2023-12-19T16:57:23.126977+0000","last_clean":"2023-12-19T16:57:23.126977+0000","last_became_active":"2023-12-19T16:57:23.125176+0000","last_became_peered":"2023-12-19T16:57:23.125176+0000","last_unstale":"2023-12-19T16:57:23.126977+0000","last_undegraded":"2023-12-19T16:57:23.126977+0000","last_fullsized":"2023-12-19T16:57:23.126977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:25.937792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.126948+0000","last_change":"2023-12-19T16:57:23.123164+0000","last_active":"2023-12-19T16:57:23.126948+0000","last_peered":"2023-12-19T16:57:23.126948+0000","last_clean":"2023-12-19T16:57:23.126948+0000","last_became_active":"2023-12-19T16:57:23.122887+0000","last_became_peered":"2023-12-19T16:57:23.122887+0000","last_unstale":"2023-12-19T16:57:23.126948+0000","last_undegraded":"2023-12-19T16:57:23.126948+0000","last_fullsized":"2023-12-19T16:57:23.126948+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:17.774629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136274+0000","last_change":"2023-12-19T16:57:23.124537+0000","last_active":"2023-12-19T16:57:23.136274+0000","last_peered":"2023-12-19T16:57:23.136274+0000","last_clean":"2023-12-19T16:57:23.136274+0000","last_became_active":"2023-12-19T16:57:23.124289+0000","last_became_peered":"2023-12-19T16:57:23.124289+0000","last_unstale":"2023-12-19T16:57:23.136274+0000","last_undegraded":"2023-12-19T16:57:23.136274+0000","last_fullsized":"2023-12-19T16:57:23.136274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:33:59.056471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136290+0000","last_change":"2023-12-19T16:57:23.129182+0000","last_active":"2023-12-19T16:57:23.136290+0000","last_peered":"2023-12-19T16:57:23.136290+0000","last_clean":"2023-12-19T16:57:23.136290+0000","last_became_active":"2023-12-19T16:57:23.128918+0000","last_became_peered":"2023-12-19T16:57:23.128918+0000","last_unstale":"2023-12-19T16:57:23.136290+0000","last_undegraded":"2023-12-19T16:57:23.136290+0000","last_fullsized":"2023-12-19T16:57:23.136290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:18.698967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114900+0000","last_change":"2023-12-19T16:57:23.131112+0000","last_active":"2023-12-19T16:57:24.114900+0000","last_peered":"2023-12-19T16:57:24.114900+0000","last_clean":"2023-12-19T16:57:24.114900+0000","last_became_active":"2023-12-19T16:57:23.130636+0000","last_became_peered":"2023-12-19T16:57:23.130636+0000","last_unstale":"2023-12-19T16:57:24.114900+0000","last_undegraded":"2023-12-19T16:57:24.114900+0000","last_fullsized":"2023-12-19T16:57:24.114900+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:19.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113458+0000","last_change":"2023-12-19T16:57:23.133197+0000","last_active":"2023-12-19T16:57:24.113458+0000","last_peered":"2023-12-19T16:57:24.113458+0000","last_clean":"2023-12-19T16:57:24.113458+0000","last_became_active":"2023-12-19T16:57:23.132931+0000","last_became_peered":"2023-12-19T16:57:23.132931+0000","last_unstale":"2023-12-19T16:57:24.113458+0000","last_undegraded":"2023-12-19T16:57:24.113458+0000","last_fullsized":"2023-12-19T16:57:24.113458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:45:27.147311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.139636+0000","last_change":"2023-12-19T16:57:23.135826+0000","last_active":"2023-12-19T16:57:23.139636+0000","last_peered":"2023-12-19T16:57:23.139636+0000","last_clean":"2023-12-19T16:57:23.139636+0000","last_became_active":"2023-12-19T16:57:23.135549+0000","last_became_peered":"2023-12-19T16:57:23.135549+0000","last_unstale":"2023-12-19T16:57:23.139636+0000","last_undegraded":"2023-12-19T16:57:23.139636+0000","last_fullsized":"2023-12-19T16:57:23.139636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:44:42.525331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114690+0000","last_change":"2023-12-19T16:57:23.136730+0000","last_active":"2023-12-19T16:57:24.114690+0000","last_peered":"2023-12-19T16:57:24.114690+0000","last_clean":"2023-12-19T16:57:24.114690+0000","last_became_active":"2023-12-19T16:57:23.136383+0000","last_became_peered":"2023-12-19T16:57:23.136383+0000","last_unstale":"2023-12-19T16:57:24.114690+0000","last_undegraded":"2023-12-19T16:57:24.114690+0000","last_fullsized":"2023-12-19T16:57:24.114690+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:01:37.696798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138398+0000","last_change":"2023-12-19T16:57:23.132534+0000","last_active":"2023-12-19T16:57:23.138398+0000","last_peered":"2023-12-19T16:57:23.138398+0000","last_clean":"2023-12-19T16:57:23.138398+0000","last_became_active":"2023-12-19T16:57:23.127032+0000","last_became_peered":"2023-12-19T16:57:23.127032+0000","last_unstale":"2023-12-19T16:57:23.138398+0000","last_undegraded":"2023-12-19T16:57:23.138398+0000","last_fullsized":"2023-12-19T16:57:23.138398+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:57:01.765156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114938+0000","last_change":"2023-12-19T16:57:23.133732+0000","last_active":"2023-12-19T16:57:24.114938+0000","last_peered":"2023-12-19T16:57:24.114938+0000","last_clean":"2023-12-19T16:57:24.114938+0000","last_became_active":"2023-12-19T16:57:23.131968+0000","last_became_peered":"2023-12-19T16:57:23.131968+0000","last_unstale":"2023-12-19T16:57:24.114938+0000","last_undegraded":"2023-12-19T16:57:24.114938+0000","last_fullsized":"2023-12-19T16:57:24.114938+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:40:55.499800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113983+0000","last_change":"2023-12-19T16:57:23.124802+0000","last_active":"2023-12-19T16:57:24.113983+0000","last_peered":"2023-12-19T16:57:24.113983+0000","last_clean":"2023-12-19T16:57:24.113983+0000","last_became_active":"2023-12-19T16:57:23.123833+0000","last_became_peered":"2023-12-19T16:57:23.123833+0000","last_unstale":"2023-12-19T16:57:24.113983+0000","last_undegraded":"2023-12-19T16:57:24.113983+0000","last_fullsized":"2023-12-19T16:57:24.113983+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:01:31.130549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113667+0000","last_change":"2023-12-19T16:57:23.128944+0000","last_active":"2023-12-19T16:57:24.113667+0000","last_peered":"2023-12-19T16:57:24.113667+0000","last_clean":"2023-12-19T16:57:24.113667+0000","last_became_active":"2023-12-19T16:57:23.126743+0000","last_became_peered":"2023-12-19T16:57:23.126743+0000","last_unstale":"2023-12-19T16:57:24.113667+0000","last_undegraded":"2023-12-19T16:57:24.113667+0000","last_fullsized":"2023-12-19T16:57:24.113667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:38.439897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.127887+0000","last_change":"2023-12-19T16:57:23.121036+0000","last_active":"2023-12-19T16:57:23.127887+0000","last_peered":"2023-12-19T16:57:23.127887+0000","last_clean":"2023-12-19T16:57:23.127887+0000","last_became_active":"2023-12-19T16:57:23.120729+0000","last_became_peered":"2023-12-19T16:57:23.120729+0000","last_unstale":"2023-12-19T16:57:23.127887+0000","last_undegraded":"2023-12-19T16:57:23.127887+0000","last_fullsized":"2023-12-19T16:57:23.127887+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:50:43.426372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136205+0000","last_change":"2023-12-19T16:57:23.124745+0000","last_active":"2023-12-19T16:57:23.136205+0000","last_peered":"2023-12-19T16:57:23.136205+0000","last_clean":"2023-12-19T16:57:23.136205+0000","last_became_active":"2023-12-19T16:57:23.123945+0000","last_became_peered":"2023-12-19T16:57:23.123945+0000","last_unstale":"2023-12-19T16:57:23.136205+0000","last_undegraded":"2023-12-19T16:57:23.136205+0000","last_fullsized":"2023-12-19T16:57:23.136205+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:31:40.754830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113418+0000","last_change":"2023-12-19T16:57:23.138702+0000","last_active":"2023-12-19T16:57:24.113418+0000","last_peered":"2023-12-19T16:57:24.113418+0000","last_clean":"2023-12-19T16:57:24.113418+0000","last_became_active":"2023-12-19T16:57:23.138434+0000","last_became_peered":"2023-12-19T16:57:23.138434+0000","last_unstale":"2023-12-19T16:57:24.113418+0000","last_undegraded":"2023-12-19T16:57:24.113418+0000","last_fullsized":"2023-12-19T16:57:24.113418+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:49:37.385210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.134542+0000","last_change":"2023-12-19T16:57:23.128468+0000","last_active":"2023-12-19T16:57:23.134542+0000","last_peered":"2023-12-19T16:57:23.134542+0000","last_clean":"2023-12-19T16:57:23.134542+0000","last_became_active":"2023-12-19T16:57:23.128215+0000","last_became_peered":"2023-12-19T16:57:23.128215+0000","last_unstale":"2023-12-19T16:57:23.134542+0000","last_undegraded":"2023-12-19T16:57:23.134542+0000","last_fullsized":"2023-12-19T16:57:23.134542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:57:00.650289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114662+0000","last_change":"2023-12-19T16:57:23.133111+0000","last_active":"2023-12-19T16:57:24.114662+0000","last_peered":"2023-12-19T16:57:24.114662+0000","last_clean":"2023-12-19T16:57:24.114662+0000","last_became_active":"2023-12-19T16:57:23.132832+0000","last_became_peered":"2023-12-19T16:57:23.132832+0000","last_unstale":"2023-12-19T16:57:24.114662+0000","last_undegraded":"2023-12-19T16:57:24.114662+0000","last_fullsized":"2023-12-19T16:57:24.114662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:03:16.069895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.113155+0000","last_change":"2023-12-19T16:57:23.126064+0000","last_active":"2023-12-19T16:57:24.113155+0000","last_peered":"2023-12-19T16:57:24.113155+0000","last_clean":"2023-12-19T16:57:24.113155+0000","last_became_active":"2023-12-19T16:57:23.125813+0000","last_became_peered":"2023-12-19T16:57:23.125813+0000","last_unstale":"2023-12-19T16:57:24.113155+0000","last_undegraded":"2023-12-19T16:57:24.113155+0000","last_fullsized":"2023-12-19T16:57:24.113155+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:08:04.712091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136386+0000","last_change":"2023-12-19T16:57:23.128888+0000","last_active":"2023-12-19T16:57:23.136386+0000","last_peered":"2023-12-19T16:57:23.136386+0000","last_clean":"2023-12-19T16:57:23.136386+0000","last_became_active":"2023-12-19T16:57:23.128641+0000","last_became_peered":"2023-12-19T16:57:23.128641+0000","last_unstale":"2023-12-19T16:57:23.136386+0000","last_undegraded":"2023-12-19T16:57:23.136386+0000","last_fullsized":"2023-12-19T16:57:23.136386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:12:53.710291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.137582+0000","last_change":"2023-12-19T16:57:23.133306+0000","last_active":"2023-12-19T16:57:23.137582+0000","last_peered":"2023-12-19T16:57:23.137582+0000","last_clean":"2023-12-19T16:57:23.137582+0000","last_became_active":"2023-12-19T16:57:23.133024+0000","last_became_peered":"2023-12-19T16:57:23.133024+0000","last_unstale":"2023-12-19T16:57:23.137582+0000","last_undegraded":"2023-12-19T16:57:23.137582+0000","last_fullsized":"2023-12-19T16:57:23.137582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:16:28.306438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"78'4","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135848+0000","last_change":"2023-12-19T16:57:22.709897+0000","last_active":"2023-12-19T16:57:23.135848+0000","last_peered":"2023-12-19T16:57:23.135848+0000","last_clean":"2023-12-19T16:57:23.135848+0000","last_became_active":"2023-12-19T16:57:22.709584+0000","last_became_peered":"2023-12-19T16:57:22.709584+0000","last_unstale":"2023-12-19T16:57:23.135848+0000","last_undegraded":"2023-12-19T16:57:23.135848+0000","last_fullsized":"2023-12-19T16:57:23.135848+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:38:28.758941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.141042+0000","last_change":"2023-12-19T16:57:23.139483+0000","last_active":"2023-12-19T16:57:23.141042+0000","last_peered":"2023-12-19T16:57:23.141042+0000","last_clean":"2023-12-19T16:57:23.141042+0000","last_became_active":"2023-12-19T16:57:23.139235+0000","last_became_peered":"2023-12-19T16:57:23.139235+0000","last_unstale":"2023-12-19T16:57:23.141042+0000","last_undegraded":"2023-12-19T16:57:23.141042+0000","last_fullsized":"2023-12-19T16:57:23.141042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:02:07.215083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136691+0000","last_change":"2023-12-19T16:57:22.714305+0000","last_active":"2023-12-19T16:57:23.136691+0000","last_peered":"2023-12-19T16:57:23.136691+0000","last_clean":"2023-12-19T16:57:23.136691+0000","last_became_active":"2023-12-19T16:57:22.714024+0000","last_became_peered":"2023-12-19T16:57:22.714024+0000","last_unstale":"2023-12-19T16:57:23.136691+0000","last_undegraded":"2023-12-19T16:57:23.136691+0000","last_fullsized":"2023-12-19T16:57:23.136691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:40.592521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136143+0000","last_change":"2023-12-19T16:57:23.124415+0000","last_active":"2023-12-19T16:57:23.136143+0000","last_peered":"2023-12-19T16:57:23.136143+0000","last_clean":"2023-12-19T16:57:23.136143+0000","last_became_active":"2023-12-19T16:57:23.123427+0000","last_became_peered":"2023-12-19T16:57:23.123427+0000","last_unstale":"2023-12-19T16:57:23.136143+0000","last_undegraded":"2023-12-19T16:57:23.136143+0000","last_fullsized":"2023-12-19T16:57:23.136143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:10:18.501944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.141326+0000","last_change":"2023-12-19T16:57:22.711651+0000","last_active":"2023-12-19T16:57:23.141326+0000","last_peered":"2023-12-19T16:57:23.141326+0000","last_clean":"2023-12-19T16:57:23.141326+0000","last_became_active":"2023-12-19T16:57:22.711220+0000","last_became_peered":"2023-12-19T16:57:22.711220+0000","last_unstale":"2023-12-19T16:57:23.141326+0000","last_undegraded":"2023-12-19T16:57:23.141326+0000","last_fullsized":"2023-12-19T16:57:23.141326+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:32:22.752099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136226+0000","last_change":"2023-12-19T16:57:23.126509+0000","last_active":"2023-12-19T16:57:23.136226+0000","last_peered":"2023-12-19T16:57:23.136226+0000","last_clean":"2023-12-19T16:57:23.136226+0000","last_became_active":"2023-12-19T16:57:23.126058+0000","last_became_peered":"2023-12-19T16:57:23.126058+0000","last_unstale":"2023-12-19T16:57:23.136226+0000","last_undegraded":"2023-12-19T16:57:23.136226+0000","last_fullsized":"2023-12-19T16:57:23.136226+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:16:27.678499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135680+0000","last_change":"2023-12-19T16:57:22.711931+0000","last_active":"2023-12-19T16:57:23.135680+0000","last_peered":"2023-12-19T16:57:23.135680+0000","last_clean":"2023-12-19T16:57:23.135680+0000","last_became_active":"2023-12-19T16:57:22.711554+0000","last_became_peered":"2023-12-19T16:57:22.711554+0000","last_unstale":"2023-12-19T16:57:23.135680+0000","last_undegraded":"2023-12-19T16:57:23.135680+0000","last_fullsized":"2023-12-19T16:57:23.135680+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:24.929320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.133287+0000","last_change":"2023-12-19T16:57:23.127410+0000","last_active":"2023-12-19T16:57:23.133287+0000","last_peered":"2023-12-19T16:57:23.133287+0000","last_clean":"2023-12-19T16:57:23.133287+0000","last_became_active":"2023-12-19T16:57:23.126421+0000","last_became_peered":"2023-12-19T16:57:23.126421+0000","last_unstale":"2023-12-19T16:57:23.133287+0000","last_undegraded":"2023-12-19T16:57:23.133287+0000","last_fullsized":"2023-12-19T16:57:23.133287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:06.429698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.125438+0000","last_change":"2023-12-19T16:57:22.709387+0000","last_active":"2023-12-19T16:57:23.125438+0000","last_peered":"2023-12-19T16:57:23.125438+0000","last_clean":"2023-12-19T16:57:23.125438+0000","last_became_active":"2023-12-19T16:57:22.708768+0000","last_became_peered":"2023-12-19T16:57:22.708768+0000","last_unstale":"2023-12-19T16:57:23.125438+0000","last_undegraded":"2023-12-19T16:57:23.125438+0000","last_fullsized":"2023-12-19T16:57:23.125438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:44:00.239607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138190+0000","last_change":"2023-12-19T16:57:23.131957+0000","last_active":"2023-12-19T16:57:23.138190+0000","last_peered":"2023-12-19T16:57:23.138190+0000","last_clean":"2023-12-19T16:57:23.138190+0000","last_became_active":"2023-12-19T16:57:23.131125+0000","last_became_peered":"2023-12-19T16:57:23.131125+0000","last_unstale":"2023-12-19T16:57:23.138190+0000","last_undegraded":"2023-12-19T16:57:23.138190+0000","last_fullsized":"2023-12-19T16:57:23.138190+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:06.098693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.129151+0000","last_change":"2023-12-19T16:57:22.716999+0000","last_active":"2023-12-19T16:57:23.129151+0000","last_peered":"2023-12-19T16:57:23.129151+0000","last_clean":"2023-12-19T16:57:23.129151+0000","last_became_active":"2023-12-19T16:57:22.716609+0000","last_became_peered":"2023-12-19T16:57:22.716609+0000","last_unstale":"2023-12-19T16:57:23.129151+0000","last_undegraded":"2023-12-19T16:57:23.129151+0000","last_fullsized":"2023-12-19T16:57:23.129151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:53:30.738332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114316+0000","last_change":"2023-12-19T16:57:23.134032+0000","last_active":"2023-12-19T16:57:24.114316+0000","last_peered":"2023-12-19T16:57:24.114316+0000","last_clean":"2023-12-19T16:57:24.114316+0000","last_became_active":"2023-12-19T16:57:23.133495+0000","last_became_peered":"2023-12-19T16:57:23.133495+0000","last_unstale":"2023-12-19T16:57:24.114316+0000","last_undegraded":"2023-12-19T16:57:24.114316+0000","last_fullsized":"2023-12-19T16:57:24.114316+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:10:13.311295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.109484+0000","last_change":"2023-12-19T16:57:22.703510+0000","last_active":"2023-12-19T16:57:23.109484+0000","last_peered":"2023-12-19T16:57:23.109484+0000","last_clean":"2023-12-19T16:57:23.109484+0000","last_became_active":"2023-12-19T16:57:22.703224+0000","last_became_peered":"2023-12-19T16:57:22.703224+0000","last_unstale":"2023-12-19T16:57:23.109484+0000","last_undegraded":"2023-12-19T16:57:23.109484+0000","last_fullsized":"2023-12-19T16:57:23.109484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:17.631958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136353+0000","last_change":"2023-12-19T16:57:23.129729+0000","last_active":"2023-12-19T16:57:23.136353+0000","last_peered":"2023-12-19T16:57:23.136353+0000","last_clean":"2023-12-19T16:57:23.136353+0000","last_became_active":"2023-12-19T16:57:23.129380+0000","last_became_peered":"2023-12-19T16:57:23.129380+0000","last_unstale":"2023-12-19T16:57:23.136353+0000","last_undegraded":"2023-12-19T16:57:23.136353+0000","last_fullsized":"2023-12-19T16:57:23.136353+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:17:22.521240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.114129+0000","last_change":"2023-12-19T16:57:22.716068+0000","last_active":"2023-12-19T16:57:23.114129+0000","last_peered":"2023-12-19T16:57:23.114129+0000","last_clean":"2023-12-19T16:57:23.114129+0000","last_became_active":"2023-12-19T16:57:22.715572+0000","last_became_peered":"2023-12-19T16:57:22.715572+0000","last_unstale":"2023-12-19T16:57:23.114129+0000","last_undegraded":"2023-12-19T16:57:23.114129+0000","last_fullsized":"2023-12-19T16:57:23.114129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:23:04.958220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.126352+0000","last_change":"2023-12-19T16:57:23.123122+0000","last_active":"2023-12-19T16:57:23.126352+0000","last_peered":"2023-12-19T16:57:23.126352+0000","last_clean":"2023-12-19T16:57:23.126352+0000","last_became_active":"2023-12-19T16:57:23.122848+0000","last_became_peered":"2023-12-19T16:57:23.122848+0000","last_unstale":"2023-12-19T16:57:23.126352+0000","last_undegraded":"2023-12-19T16:57:23.126352+0000","last_fullsized":"2023-12-19T16:57:23.126352+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:18.640429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.115052+0000","last_change":"2023-12-19T16:57:22.722055+0000","last_active":"2023-12-19T16:57:24.115052+0000","last_peered":"2023-12-19T16:57:24.115052+0000","last_clean":"2023-12-19T16:57:24.115052+0000","last_became_active":"2023-12-19T16:57:22.721767+0000","last_became_peered":"2023-12-19T16:57:22.721767+0000","last_unstale":"2023-12-19T16:57:24.115052+0000","last_undegraded":"2023-12-19T16:57:24.115052+0000","last_fullsized":"2023-12-19T16:57:24.115052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:17.631958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.115106+0000","last_change":"2023-12-19T16:57:23.141377+0000","last_active":"2023-12-19T16:57:24.115106+0000","last_peered":"2023-12-19T16:57:24.115106+0000","last_clean":"2023-12-19T16:57:24.115106+0000","last_became_active":"2023-12-19T16:57:23.141083+0000","last_became_peered":"2023-12-19T16:57:23.141083+0000","last_unstale":"2023-12-19T16:57:24.115106+0000","last_undegraded":"2023-12-19T16:57:24.115106+0000","last_fullsized":"2023-12-19T16:57:24.115106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:30:47.901104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.118661+0000","last_change":"2023-12-19T16:57:22.718327+0000","last_active":"2023-12-19T16:57:23.118661+0000","last_peered":"2023-12-19T16:57:23.118661+0000","last_clean":"2023-12-19T16:57:23.118661+0000","last_became_active":"2023-12-19T16:57:22.718000+0000","last_became_peered":"2023-12-19T16:57:22.718000+0000","last_unstale":"2023-12-19T16:57:23.118661+0000","last_undegraded":"2023-12-19T16:57:23.118661+0000","last_fullsized":"2023-12-19T16:57:23.118661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:46.213640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.140162+0000","last_change":"2023-12-19T16:57:23.124970+0000","last_active":"2023-12-19T16:57:23.140162+0000","last_peered":"2023-12-19T16:57:23.140162+0000","last_clean":"2023-12-19T16:57:23.140162+0000","last_became_active":"2023-12-19T16:57:23.124725+0000","last_became_peered":"2023-12-19T16:57:23.124725+0000","last_unstale":"2023-12-19T16:57:23.140162+0000","last_undegraded":"2023-12-19T16:57:23.140162+0000","last_fullsized":"2023-12-19T16:57:23.140162+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:45.596570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"78'3","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.115188+0000","last_change":"2023-12-19T16:57:22.719555+0000","last_active":"2023-12-19T16:57:24.115188+0000","last_peered":"2023-12-19T16:57:24.115188+0000","last_clean":"2023-12-19T16:57:24.115188+0000","last_became_active":"2023-12-19T16:57:22.719293+0000","last_became_peered":"2023-12-19T16:57:22.719293+0000","last_unstale":"2023-12-19T16:57:24.115188+0000","last_undegraded":"2023-12-19T16:57:24.115188+0000","last_fullsized":"2023-12-19T16:57:24.115188+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:43:53.045116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.115062+0000","last_change":"2023-12-19T16:57:23.141880+0000","last_active":"2023-12-19T16:57:24.115062+0000","last_peered":"2023-12-19T16:57:24.115062+0000","last_clean":"2023-12-19T16:57:24.115062+0000","last_became_active":"2023-12-19T16:57:23.141603+0000","last_became_peered":"2023-12-19T16:57:23.141603+0000","last_unstale":"2023-12-19T16:57:24.115062+0000","last_undegraded":"2023-12-19T16:57:24.115062+0000","last_fullsized":"2023-12-19T16:57:24.115062+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:41.600992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"78'6","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.140817+0000","last_change":"2023-12-19T16:57:22.713027+0000","last_active":"2023-12-19T16:57:23.140817+0000","last_peered":"2023-12-19T16:57:23.140817+0000","last_clean":"2023-12-19T16:57:23.140817+0000","last_became_active":"2023-12-19T16:57:22.712761+0000","last_became_peered":"2023-12-19T16:57:22.712761+0000","last_unstale":"2023-12-19T16:57:23.140817+0000","last_undegraded":"2023-12-19T16:57:23.140817+0000","last_fullsized":"2023-12-19T16:57:23.140817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:14:35.501559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.133922+0000","last_change":"2023-12-19T16:57:23.126785+0000","last_active":"2023-12-19T16:57:23.133922+0000","last_peered":"2023-12-19T16:57:23.133922+0000","last_clean":"2023-12-19T16:57:23.133922+0000","last_became_active":"2023-12-19T16:57:23.126493+0000","last_became_peered":"2023-12-19T16:57:23.126493+0000","last_unstale":"2023-12-19T16:57:23.133922+0000","last_undegraded":"2023-12-19T16:57:23.133922+0000","last_fullsized":"2023-12-19T16:57:23.133922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:38.775519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135785+0000","last_change":"2023-12-19T16:57:22.718269+0000","last_active":"2023-12-19T16:57:23.135785+0000","last_peered":"2023-12-19T16:57:23.135785+0000","last_clean":"2023-12-19T16:57:23.135785+0000","last_became_active":"2023-12-19T16:57:22.717961+0000","last_became_peered":"2023-12-19T16:57:22.717961+0000","last_unstale":"2023-12-19T16:57:23.135785+0000","last_undegraded":"2023-12-19T16:57:23.135785+0000","last_fullsized":"2023-12-19T16:57:23.135785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:45.501869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114816+0000","last_change":"2023-12-19T16:57:23.134650+0000","last_active":"2023-12-19T16:57:24.114816+0000","last_peered":"2023-12-19T16:57:24.114816+0000","last_clean":"2023-12-19T16:57:24.114816+0000","last_became_active":"2023-12-19T16:57:23.131775+0000","last_became_peered":"2023-12-19T16:57:23.131775+0000","last_unstale":"2023-12-19T16:57:24.114816+0000","last_undegraded":"2023-12-19T16:57:24.114816+0000","last_fullsized":"2023-12-19T16:57:24.114816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:35.829847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114064+0000","last_change":"2023-12-19T16:57:22.718127+0000","last_active":"2023-12-19T16:57:24.114064+0000","last_peered":"2023-12-19T16:57:24.114064+0000","last_clean":"2023-12-19T16:57:24.114064+0000","last_became_active":"2023-12-19T16:57:22.717819+0000","last_became_peered":"2023-12-19T16:57:22.717819+0000","last_unstale":"2023-12-19T16:57:24.114064+0000","last_undegraded":"2023-12-19T16:57:24.114064+0000","last_fullsized":"2023-12-19T16:57:24.114064+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:37:26.150257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135805+0000","last_change":"2023-12-19T16:57:23.132252+0000","last_active":"2023-12-19T16:57:23.135805+0000","last_peered":"2023-12-19T16:57:23.135805+0000","last_clean":"2023-12-19T16:57:23.135805+0000","last_became_active":"2023-12-19T16:57:23.131995+0000","last_became_peered":"2023-12-19T16:57:23.131995+0000","last_unstale":"2023-12-19T16:57:23.135805+0000","last_undegraded":"2023-12-19T16:57:23.135805+0000","last_fullsized":"2023-12-19T16:57:23.135805+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:09.688251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.109491+0000","last_change":"2023-12-19T16:57:22.711732+0000","last_active":"2023-12-19T16:57:23.109491+0000","last_peered":"2023-12-19T16:57:23.109491+0000","last_clean":"2023-12-19T16:57:23.109491+0000","last_became_active":"2023-12-19T16:57:22.711446+0000","last_became_peered":"2023-12-19T16:57:22.711446+0000","last_unstale":"2023-12-19T16:57:23.109491+0000","last_undegraded":"2023-12-19T16:57:23.109491+0000","last_fullsized":"2023-12-19T16:57:23.109491+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:08.679779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114525+0000","last_change":"2023-12-19T16:57:23.133197+0000","last_active":"2023-12-19T16:57:24.114525+0000","last_peered":"2023-12-19T16:57:24.114525+0000","last_clean":"2023-12-19T16:57:24.114525+0000","last_became_active":"2023-12-19T16:57:23.132333+0000","last_became_peered":"2023-12-19T16:57:23.132333+0000","last_unstale":"2023-12-19T16:57:24.114525+0000","last_undegraded":"2023-12-19T16:57:24.114525+0000","last_fullsized":"2023-12-19T16:57:24.114525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:25:24.949466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"78'8","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114240+0000","last_change":"2023-12-19T16:57:22.719852+0000","last_active":"2023-12-19T16:57:24.114240+0000","last_peered":"2023-12-19T16:57:24.114240+0000","last_clean":"2023-12-19T16:57:24.114240+0000","last_became_active":"2023-12-19T16:57:22.719525+0000","last_became_peered":"2023-12-19T16:57:22.719525+0000","last_unstale":"2023-12-19T16:57:24.114240+0000","last_undegraded":"2023-12-19T16:57:24.114240+0000","last_fullsized":"2023-12-19T16:57:24.114240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:54.549672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114960+0000","last_change":"2023-12-19T16:57:23.131325+0000","last_active":"2023-12-19T16:57:24.114960+0000","last_peered":"2023-12-19T16:57:24.114960+0000","last_clean":"2023-12-19T16:57:24.114960+0000","last_became_active":"2023-12-19T16:57:23.131003+0000","last_became_peered":"2023-12-19T16:57:23.131003+0000","last_unstale":"2023-12-19T16:57:24.114960+0000","last_undegraded":"2023-12-19T16:57:24.114960+0000","last_fullsized":"2023-12-19T16:57:24.114960+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:54:16.935719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138107+0000","last_change":"2023-12-19T16:57:22.718632+0000","last_active":"2023-12-19T16:57:23.138107+0000","last_peered":"2023-12-19T16:57:23.138107+0000","last_clean":"2023-12-19T16:57:23.138107+0000","last_became_active":"2023-12-19T16:57:22.717992+0000","last_became_peered":"2023-12-19T16:57:22.717992+0000","last_unstale":"2023-12-19T16:57:23.138107+0000","last_undegraded":"2023-12-19T16:57:23.138107+0000","last_fullsized":"2023-12-19T16:57:23.138107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:34.821375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.126930+0000","last_change":"2023-12-19T16:57:23.121177+0000","last_active":"2023-12-19T16:57:23.126930+0000","last_peered":"2023-12-19T16:57:23.126930+0000","last_clean":"2023-12-19T16:57:23.126930+0000","last_became_active":"2023-12-19T16:57:23.120893+0000","last_became_peered":"2023-12-19T16:57:23.120893+0000","last_unstale":"2023-12-19T16:57:23.126930+0000","last_undegraded":"2023-12-19T16:57:23.126930+0000","last_fullsized":"2023-12-19T16:57:23.126930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:19.956977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.137536+0000","last_change":"2023-12-19T16:57:22.713505+0000","last_active":"2023-12-19T16:57:23.137536+0000","last_peered":"2023-12-19T16:57:23.137536+0000","last_clean":"2023-12-19T16:57:23.137536+0000","last_became_active":"2023-12-19T16:57:22.713180+0000","last_became_peered":"2023-12-19T16:57:22.713180+0000","last_unstale":"2023-12-19T16:57:23.137536+0000","last_undegraded":"2023-12-19T16:57:23.137536+0000","last_fullsized":"2023-12-19T16:57:23.137536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:08.953488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.131470+0000","last_change":"2023-12-19T16:57:23.125306+0000","last_active":"2023-12-19T16:57:23.131470+0000","last_peered":"2023-12-19T16:57:23.131470+0000","last_clean":"2023-12-19T16:57:23.131470+0000","last_became_active":"2023-12-19T16:57:23.125044+0000","last_became_peered":"2023-12-19T16:57:23.125044+0000","last_unstale":"2023-12-19T16:57:23.131470+0000","last_undegraded":"2023-12-19T16:57:23.131470+0000","last_fullsized":"2023-12-19T16:57:23.131470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:04:04.164209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"78'1","reported_seq":11,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.134604+0000","last_change":"2023-12-19T16:57:22.715552+0000","last_active":"2023-12-19T16:57:23.134604+0000","last_peered":"2023-12-19T16:57:23.134604+0000","last_clean":"2023-12-19T16:57:23.134604+0000","last_became_active":"2023-12-19T16:57:22.715216+0000","last_became_peered":"2023-12-19T16:57:22.715216+0000","last_unstale":"2023-12-19T16:57:23.134604+0000","last_undegraded":"2023-12-19T16:57:23.134604+0000","last_fullsized":"2023-12-19T16:57:23.134604+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:29:22.712791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.115004+0000","last_change":"2023-12-19T16:57:23.141558+0000","last_active":"2023-12-19T16:57:24.115004+0000","last_peered":"2023-12-19T16:57:24.115004+0000","last_clean":"2023-12-19T16:57:24.115004+0000","last_became_active":"2023-12-19T16:57:23.141296+0000","last_became_peered":"2023-12-19T16:57:23.141296+0000","last_unstale":"2023-12-19T16:57:24.115004+0000","last_undegraded":"2023-12-19T16:57:24.115004+0000","last_fullsized":"2023-12-19T16:57:24.115004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:13.543929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.121866+0000","last_change":"2023-12-19T16:57:22.716127+0000","last_active":"2023-12-19T16:57:23.121866+0000","last_peered":"2023-12-19T16:57:23.121866+0000","last_clean":"2023-12-19T16:57:23.121866+0000","last_became_active":"2023-12-19T16:57:22.714244+0000","last_became_peered":"2023-12-19T16:57:22.714244+0000","last_unstale":"2023-12-19T16:57:23.121866+0000","last_undegraded":"2023-12-19T16:57:23.121866+0000","last_fullsized":"2023-12-19T16:57:23.121866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:07:34.317614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.133903+0000","last_change":"2023-12-19T16:57:23.126079+0000","last_active":"2023-12-19T16:57:23.133903+0000","last_peered":"2023-12-19T16:57:23.133903+0000","last_clean":"2023-12-19T16:57:23.133903+0000","last_became_active":"2023-12-19T16:57:23.122323+0000","last_became_peered":"2023-12-19T16:57:23.122323+0000","last_unstale":"2023-12-19T16:57:23.133903+0000","last_undegraded":"2023-12-19T16:57:23.133903+0000","last_fullsized":"2023-12-19T16:57:23.133903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:30:47.901104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138417+0000","last_change":"2023-12-19T16:57:22.709791+0000","last_active":"2023-12-19T16:57:23.138417+0000","last_peered":"2023-12-19T16:57:23.138417+0000","last_clean":"2023-12-19T16:57:23.138417+0000","last_became_active":"2023-12-19T16:57:22.709391+0000","last_became_peered":"2023-12-19T16:57:22.709391+0000","last_unstale":"2023-12-19T16:57:23.138417+0000","last_undegraded":"2023-12-19T16:57:23.138417+0000","last_fullsized":"2023-12-19T16:57:23.138417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:01:36.688326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136118+0000","last_change":"2023-12-19T16:57:23.124281+0000","last_active":"2023-12-19T16:57:23.136118+0000","last_peered":"2023-12-19T16:57:23.136118+0000","last_clean":"2023-12-19T16:57:23.136118+0000","last_became_active":"2023-12-19T16:57:23.123986+0000","last_became_peered":"2023-12-19T16:57:23.123986+0000","last_unstale":"2023-12-19T16:57:23.136118+0000","last_undegraded":"2023-12-19T16:57:23.136118+0000","last_fullsized":"2023-12-19T16:57:23.136118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:01:00.969856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.105792+0000","last_change":"2023-12-19T16:57:22.716245+0000","last_active":"2023-12-19T16:57:23.105792+0000","last_peered":"2023-12-19T16:57:23.105792+0000","last_clean":"2023-12-19T16:57:23.105792+0000","last_became_active":"2023-12-19T16:57:22.714702+0000","last_became_peered":"2023-12-19T16:57:22.714702+0000","last_unstale":"2023-12-19T16:57:23.105792+0000","last_undegraded":"2023-12-19T16:57:23.105792+0000","last_fullsized":"2023-12-19T16:57:23.105792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:08.679779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138447+0000","last_change":"2023-12-19T16:57:23.136320+0000","last_active":"2023-12-19T16:57:23.138447+0000","last_peered":"2023-12-19T16:57:23.138447+0000","last_clean":"2023-12-19T16:57:23.138447+0000","last_became_active":"2023-12-19T16:57:23.135951+0000","last_became_peered":"2023-12-19T16:57:23.135951+0000","last_unstale":"2023-12-19T16:57:23.138447+0000","last_undegraded":"2023-12-19T16:57:23.138447+0000","last_fullsized":"2023-12-19T16:57:23.138447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:57.991487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.109119+0000","last_change":"2023-12-19T16:57:22.709715+0000","last_active":"2023-12-19T16:57:23.109119+0000","last_peered":"2023-12-19T16:57:23.109119+0000","last_clean":"2023-12-19T16:57:23.109119+0000","last_became_active":"2023-12-19T16:57:22.707699+0000","last_became_peered":"2023-12-19T16:57:22.707699+0000","last_unstale":"2023-12-19T16:57:23.109119+0000","last_undegraded":"2023-12-19T16:57:23.109119+0000","last_fullsized":"2023-12-19T16:57:23.109119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:43:53.045116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.135844+0000","last_change":"2023-12-19T16:57:23.127990+0000","last_active":"2023-12-19T16:57:23.135844+0000","last_peered":"2023-12-19T16:57:23.135844+0000","last_clean":"2023-12-19T16:57:23.135844+0000","last_became_active":"2023-12-19T16:57:23.127614+0000","last_became_peered":"2023-12-19T16:57:23.127614+0000","last_unstale":"2023-12-19T16:57:23.135844+0000","last_undegraded":"2023-12-19T16:57:23.135844+0000","last_fullsized":"2023-12-19T16:57:23.135844+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:54:16.935719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"78'2","reported_seq":12,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.138991+0000","last_change":"2023-12-19T16:57:22.713812+0000","last_active":"2023-12-19T16:57:23.138991+0000","last_peered":"2023-12-19T16:57:23.138991+0000","last_clean":"2023-12-19T16:57:23.138991+0000","last_became_active":"2023-12-19T16:57:22.713491+0000","last_became_peered":"2023-12-19T16:57:22.713491+0000","last_unstale":"2023-12-19T16:57:23.138991+0000","last_undegraded":"2023-12-19T16:57:23.138991+0000","last_fullsized":"2023-12-19T16:57:23.138991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:54.549672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136767+0000","last_change":"2023-12-19T16:57:23.130707+0000","last_active":"2023-12-19T16:57:23.136767+0000","last_peered":"2023-12-19T16:57:23.136767+0000","last_clean":"2023-12-19T16:57:23.136767+0000","last_became_active":"2023-12-19T16:57:23.130421+0000","last_became_peered":"2023-12-19T16:57:23.130421+0000","last_unstale":"2023-12-19T16:57:23.136767+0000","last_undegraded":"2023-12-19T16:57:23.136767+0000","last_fullsized":"2023-12-19T16:57:23.136767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:25:24.949466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.114777+0000","last_change":"2023-12-19T16:57:22.715702+0000","last_active":"2023-12-19T16:57:23.114777+0000","last_peered":"2023-12-19T16:57:23.114777+0000","last_clean":"2023-12-19T16:57:23.114777+0000","last_became_active":"2023-12-19T16:57:22.715425+0000","last_became_peered":"2023-12-19T16:57:22.715425+0000","last_unstale":"2023-12-19T16:57:23.114777+0000","last_undegraded":"2023-12-19T16:57:23.114777+0000","last_fullsized":"2023-12-19T16:57:23.114777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:14:22.112729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.125131+0000","last_change":"2023-12-19T16:57:23.122478+0000","last_active":"2023-12-19T16:57:23.125131+0000","last_peered":"2023-12-19T16:57:23.125131+0000","last_clean":"2023-12-19T16:57:23.125131+0000","last_became_active":"2023-12-19T16:57:23.122217+0000","last_became_peered":"2023-12-19T16:57:23.122217+0000","last_unstale":"2023-12-19T16:57:23.125131+0000","last_undegraded":"2023-12-19T16:57:23.125131+0000","last_fullsized":"2023-12-19T16:57:23.125131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:40:53.359468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114088+0000","last_change":"2023-12-19T16:57:22.716637+0000","last_active":"2023-12-19T16:57:24.114088+0000","last_peered":"2023-12-19T16:57:24.114088+0000","last_clean":"2023-12-19T16:57:24.114088+0000","last_became_active":"2023-12-19T16:57:22.716330+0000","last_became_peered":"2023-12-19T16:57:22.716330+0000","last_unstale":"2023-12-19T16:57:24.114088+0000","last_undegraded":"2023-12-19T16:57:24.114088+0000","last_fullsized":"2023-12-19T16:57:24.114088+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:44:00.239607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136244+0000","last_change":"2023-12-19T16:57:23.126630+0000","last_active":"2023-12-19T16:57:23.136244+0000","last_peered":"2023-12-19T16:57:23.136244+0000","last_clean":"2023-12-19T16:57:23.136244+0000","last_became_active":"2023-12-19T16:57:23.126384+0000","last_became_peered":"2023-12-19T16:57:23.126384+0000","last_unstale":"2023-12-19T16:57:23.136244+0000","last_undegraded":"2023-12-19T16:57:23.136244+0000","last_fullsized":"2023-12-19T16:57:23.136244+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:31:56.777048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.110773+0000","last_change":"2023-12-19T16:57:22.704220+0000","last_active":"2023-12-19T16:57:23.110773+0000","last_peered":"2023-12-19T16:57:23.110773+0000","last_clean":"2023-12-19T16:57:23.110773+0000","last_became_active":"2023-12-19T16:57:22.703934+0000","last_became_peered":"2023-12-19T16:57:22.703934+0000","last_unstale":"2023-12-19T16:57:23.110773+0000","last_undegraded":"2023-12-19T16:57:23.110773+0000","last_fullsized":"2023-12-19T16:57:23.110773+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:54.549672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.132671+0000","last_change":"2023-12-19T16:57:23.124841+0000","last_active":"2023-12-19T16:57:23.132671+0000","last_peered":"2023-12-19T16:57:23.132671+0000","last_clean":"2023-12-19T16:57:23.132671+0000","last_became_active":"2023-12-19T16:57:23.124563+0000","last_became_peered":"2023-12-19T16:57:23.124563+0000","last_unstale":"2023-12-19T16:57:23.132671+0000","last_undegraded":"2023-12-19T16:57:23.132671+0000","last_fullsized":"2023-12-19T16:57:23.132671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:28.949332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.105795+0000","last_change":"2023-12-19T16:57:22.716766+0000","last_active":"2023-12-19T16:57:23.105795+0000","last_peered":"2023-12-19T16:57:23.105795+0000","last_clean":"2023-12-19T16:57:23.105795+0000","last_became_active":"2023-12-19T16:57:22.716413+0000","last_became_peered":"2023-12-19T16:57:22.716413+0000","last_unstale":"2023-12-19T16:57:23.105795+0000","last_undegraded":"2023-12-19T16:57:23.105795+0000","last_fullsized":"2023-12-19T16:57:23.105795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:17.631958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114802+0000","last_change":"2023-12-19T16:57:23.135384+0000","last_active":"2023-12-19T16:57:24.114802+0000","last_peered":"2023-12-19T16:57:24.114802+0000","last_clean":"2023-12-19T16:57:24.114802+0000","last_became_active":"2023-12-19T16:57:23.135061+0000","last_became_peered":"2023-12-19T16:57:23.135061+0000","last_unstale":"2023-12-19T16:57:24.114802+0000","last_undegraded":"2023-12-19T16:57:24.114802+0000","last_fullsized":"2023-12-19T16:57:24.114802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:47.222111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.114572+0000","last_change":"2023-12-19T16:57:22.713968+0000","last_active":"2023-12-19T16:57:23.114572+0000","last_peered":"2023-12-19T16:57:23.114572+0000","last_clean":"2023-12-19T16:57:23.114572+0000","last_became_active":"2023-12-19T16:57:22.713679+0000","last_became_peered":"2023-12-19T16:57:22.713679+0000","last_unstale":"2023-12-19T16:57:23.114572+0000","last_undegraded":"2023-12-19T16:57:23.114572+0000","last_fullsized":"2023-12-19T16:57:23.114572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:21:53.731379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.129117+0000","last_change":"2023-12-19T16:57:23.116246+0000","last_active":"2023-12-19T16:57:23.129117+0000","last_peered":"2023-12-19T16:57:23.129117+0000","last_clean":"2023-12-19T16:57:23.129117+0000","last_became_active":"2023-12-19T16:57:23.115985+0000","last_became_peered":"2023-12-19T16:57:23.115985+0000","last_unstale":"2023-12-19T16:57:23.129117+0000","last_undegraded":"2023-12-19T16:57:23.129117+0000","last_fullsized":"2023-12-19T16:57:23.129117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:54:16.935719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.105796+0000","last_change":"2023-12-19T16:57:22.712441+0000","last_active":"2023-12-19T16:57:23.105796+0000","last_peered":"2023-12-19T16:57:23.105796+0000","last_clean":"2023-12-19T16:57:23.105796+0000","last_became_active":"2023-12-19T16:57:22.711466+0000","last_became_peered":"2023-12-19T16:57:22.711466+0000","last_unstale":"2023-12-19T16:57:23.105796+0000","last_undegraded":"2023-12-19T16:57:23.105796+0000","last_fullsized":"2023-12-19T16:57:23.105796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:24.929320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.136330+0000","last_change":"2023-12-19T16:57:23.129224+0000","last_active":"2023-12-19T16:57:23.136330+0000","last_peered":"2023-12-19T16:57:23.136330+0000","last_clean":"2023-12-19T16:57:23.136330+0000","last_became_active":"2023-12-19T16:57:23.128803+0000","last_became_peered":"2023-12-19T16:57:23.128803+0000","last_unstale":"2023-12-19T16:57:23.136330+0000","last_undegraded":"2023-12-19T16:57:23.136330+0000","last_fullsized":"2023-12-19T16:57:23.136330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:54:45.499902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.112078+0000","last_change":"2023-12-19T16:57:22.705820+0000","last_active":"2023-12-19T16:57:23.112078+0000","last_peered":"2023-12-19T16:57:23.112078+0000","last_clean":"2023-12-19T16:57:23.112078+0000","last_became_active":"2023-12-19T16:57:22.704940+0000","last_became_peered":"2023-12-19T16:57:22.704940+0000","last_unstale":"2023-12-19T16:57:23.112078+0000","last_undegraded":"2023-12-19T16:57:23.112078+0000","last_fullsized":"2023-12-19T16:57:23.112078+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:12.535457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.139807+0000","last_change":"2023-12-19T16:57:23.136635+0000","last_active":"2023-12-19T16:57:23.139807+0000","last_peered":"2023-12-19T16:57:23.139807+0000","last_clean":"2023-12-19T16:57:23.139807+0000","last_became_active":"2023-12-19T16:57:23.135238+0000","last_became_peered":"2023-12-19T16:57:23.135238+0000","last_unstale":"2023-12-19T16:57:23.139807+0000","last_undegraded":"2023-12-19T16:57:23.139807+0000","last_fullsized":"2023-12-19T16:57:23.139807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:22.683977+0000","last_clean_scrub_stamp":"2023-12-19T16:57:22.683977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:47.059374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-19T16:57:24.114150+0000","last_change":"2023-12-19T16:57:22.717887+0000","last_active":"2023-12-19T16:57:24.114150+0000","last_peered":"2023-12-19T16:57:24.114150+0000","last_clean":"2023-12-19T16:57:24.114150+0000","last_became_active":"2023-12-19T16:57:22.717600+0000","last_became_peered":"2023-12-19T16:57:22.717600+0000","last_unstale":"2023-12-19T16:57:24.114150+0000","last_undegraded":"2023-12-19T16:57:24.114150+0000","last_fullsized":"2023-12-19T16:57:24.114150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:21.675505+0000","last_clean_scrub_stamp":"2023-12-19T16:57:21.675505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:23:04.958220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"67'161","reported_seq":375,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-19T16:57:23.117050+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:57:23.117050+0000","last_peered":"2023-12-19T16:57:23.117050+0000","last_clean":"2023-12-19T16:57:23.117050+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:57:23.117050+0000","last_undegraded":"2023-12-19T16:57:23.117050+0000","last_fullsized":"2023-12-19T16:57:23.117050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509623,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32548,"kb_used_data":1500,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339292,"statfs":{"total":96636764160,"available":96603435008,"internally_reserved":0,"allocated":1536000,"data_stored":1044967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.70099999999999996},"min":{"1min":0.55000000000000004,"5min":0.35699999999999998,"15min":0.30199999999999999},"max":{"1min":0.95699999999999996,"5min":1.3460000000000001,"15min":2.6389999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73599999999999999,"15min":0.72199999999999998},"min":{"1min":0.58699999999999997,"5min":0.45000000000000001,"15min":0.32700000000000001},"max":{"1min":0.91000000000000003,"5min":1.6930000000000001,"15min":2.532},"last":0.42199999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.75900000000000001,"15min":0.72299999999999998},"min":{"1min":0.67400000000000004,"5min":0.436,"15min":0.36399999999999999},"max":{"1min":0.93000000000000005,"5min":1.212,"15min":1.212},"last":0.436},{"interface":"front","average":{"1min":0.751,"5min":0.745,"15min":0.72799999999999998},"min":{"1min":0.64800000000000002,"5min":0.438,"15min":0.36799999999999999},"max":{"1min":0.86599999999999999,"5min":1.74,"15min":1.74},"last":0.47199999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72999999999999998,"15min":0.70599999999999996},"min":{"1min":0.61799999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84299999999999997,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.76600000000000001,"15min":0.73699999999999999},"min":{"1min":0.60999999999999999,"5min":0.47299999999999998,"15min":0.36599999999999999},"max":{"1min":0.95799999999999996,"5min":1.395,"15min":1.395},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.74299999999999999,"15min":0.72099999999999997},"min":{"1min":0.61899999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.88600000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.76000000000000001,"15min":0.75},"min":{"1min":0.65300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90800000000000003,"5min":1.266,"15min":1.266},"last":0.73499999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.45000000000000001,"5min":0.32700000000000001,"15min":0.28499999999999998},"max":{"1min":0.78800000000000003,"5min":1.841,"15min":1.841},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.47499999999999998,"5min":0.40200000000000002,"15min":0.36599999999999999},"max":{"1min":0.69899999999999995,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.53500000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.623,"15min":0.61199999999999999},"min":{"1min":0.48999999999999999,"5min":0.36899999999999999,"15min":0.35699999999999998},"max":{"1min":0.76800000000000002,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.66000000000000003,"15min":0.64400000000000002},"min":{"1min":0.503,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.73099999999999998,"5min":1.784,"15min":1.784},"last":0.68600000000000005}]},{"osd":6,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70499999999999996,"15min":0.69199999999999995},"min":{"1min":0.57199999999999995,"5min":0.45700000000000002,"15min":0.39800000000000002},"max":{"1min":0.879,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.52700000000000002,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.70299999999999996,"5min":2.105,"15min":2.105},"last":0.72199999999999998}]}]},{"osd":6,"up_from":15,"seq":64424509623,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":933888,"data_stored":454439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70099999999999996,"15min":0.69799999999999995},"min":{"1min":0.42599999999999999,"5min":0.38700000000000001,"15min":0.309},"max":{"1min":0.88200000000000001,"5min":0.999,"15min":1.8600000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71699999999999997,"15min":0.69699999999999995},"min":{"1min":0.51800000000000002,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":1.0089999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.63600000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71199999999999997,"15min":0.73399999999999999},"min":{"1min":0.496,"5min":0.41399999999999998,"15min":0.33400000000000002},"max":{"1min":0.88400000000000001,"5min":0.99399999999999999,"15min":4.3799999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73499999999999999,"15min":0.73699999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.39500000000000002},"max":{"1min":0.91900000000000004,"5min":1.0189999999999999,"15min":1.583},"last":0.75}]},{"osd":2,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.755,"15min":0.74299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.35199999999999998},"max":{"1min":0.95899999999999996,"5min":1.087,"15min":1.3180000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.58299999999999996,"5min":0.48199999999999998,"15min":0.44400000000000001},"max":{"1min":0.98199999999999998,"5min":1.0549999999999999,"15min":1.1879999999999999},"last":0.46400000000000002}]},{"osd":3,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.76100000000000001,"15min":0.80500000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97499999999999998,"5min":1.1020000000000001,"15min":4.673},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.54400000000000004,"5min":0.42699999999999999,"15min":0.41399999999999998},"max":{"1min":0.94199999999999995,"5min":1.0740000000000001,"15min":1.962},"last":0.627}]},{"osd":4,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.35899999999999999},"max":{"1min":0.69999999999999996,"5min":0.89500000000000002,"15min":0.96999999999999997},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61399999999999999,"15min":0.60199999999999998},"min":{"1min":0.45000000000000001,"5min":0.39800000000000002,"15min":0.35699999999999998},"max":{"1min":0.878,"5min":0.93500000000000005,"15min":1.006},"last":0.77300000000000002}]},{"osd":5,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60699999999999998,"15min":0.60399999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.33300000000000002},"max":{"1min":0.80300000000000005,"5min":0.91300000000000003,"15min":1.2050000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68100000000000005,"15min":0.66000000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.41499999999999998},"max":{"1min":0.96299999999999997,"5min":1.046,"15min":1.046},"last":0.80300000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.49299999999999999,"15min":0.438},"max":{"1min":1.006,"5min":1.0249999999999999,"15min":1.0620000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.71299999999999997,"15min":0.70599999999999996},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.371},"max":{"1min":0.91100000000000003,"5min":1.032,"15min":1.1220000000000001},"last":0.73799999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509623,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":942080,"data_stored":455091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.68899999999999995,"15min":0.69899999999999995},"min":{"1min":0.59099999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.876,"5min":0.88400000000000001,"15min":0.95799999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.73999999999999999,"15min":0.72399999999999998},"min":{"1min":0.57499999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93899999999999995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57199999999999995}]},{"osd":1,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.73599999999999999,"15min":0.73899999999999999},"min":{"1min":0.60699999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.58599999999999997,"5min":0.374,"15min":0.374},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.89200000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.75,"15min":0.747},"min":{"1min":0.60299999999999998,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.026,"5min":1.0409999999999999,"15min":1.1120000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.72799999999999998,"15min":0.72499999999999998},"min":{"1min":0.59199999999999997,"5min":0.5,"15min":0.46400000000000002},"max":{"1min":1.036,"5min":1.036,"15min":1.087},"last":0.81999999999999995}]},{"osd":3,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.72699999999999998,"15min":0.72799999999999998},"min":{"1min":0.626,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.77300000000000002,"15min":0.76200000000000001},"min":{"1min":0.56699999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.085,"5min":1.089,"15min":1.089},"last":0.998}]},{"osd":4,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54200000000000004,"15min":0.56399999999999995},"min":{"1min":0.41199999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76900000000000002,"5min":0.83999999999999997,"15min":1.165},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.434,"5min":0.40799999999999997,"15min":0.32400000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.0649999999999999},"last":0.91100000000000003}]},{"osd":6,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61299999999999999,"15min":0.61099999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34699999999999998},"max":{"1min":0.89500000000000002,"5min":0.98799999999999999,"15min":1.153},"last":0.62},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.46800000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.98999999999999999},"last":0.86599999999999999}]},{"osd":7,"last update":"Tue Dec 19 16:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64900000000000002,"15min":0.65300000000000002},"min":{"1min":0.47999999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.133},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67900000000000005,"15min":0.67800000000000005},"min":{"1min":0.49199999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.014,"5min":1.131,"15min":1.131},"last":0.84299999999999997}]}]},{"osd":4,"up_from":15,"seq":64424509623,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54116,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317724,"statfs":{"total":96636764160,"available":96581349376,"internally_reserved":0,"allocated":946176,"data_stored":456003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.72599999999999998,"15min":0.71199999999999997},"min":{"1min":0.66500000000000004,"5min":0.316,"15min":0.316},"max":{"1min":0.93300000000000005,"5min":1.877,"15min":1.877},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70899999999999996,"15min":0.69999999999999996},"min":{"1min":0.502,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.94899999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.75600000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.51500000000000001,"5min":0.44400000000000001,"15min":0.34799999999999998},"max":{"1min":0.93799999999999994,"5min":1.0209999999999999,"15min":1.8700000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.745,"15min":0.73599999999999999},"min":{"1min":0.52300000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92100000000000004,"5min":2.02,"15min":2.02},"last":0.77800000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.67600000000000005,"5min":0.47099999999999997,"15min":0.38700000000000001},"max":{"1min":0.90600000000000003,"5min":1.375,"15min":1.375},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.77000000000000002,"15min":0.75700000000000001},"min":{"1min":0.65500000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89200000000000002,"5min":1.927,"15min":1.927},"last":0.73899999999999999}]},{"osd":3,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.71699999999999997,"15min":0.71299999999999997},"min":{"1min":0.53200000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90500000000000003,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.754,"15min":0.76300000000000001},"min":{"1min":0.59999999999999998,"5min":0.52300000000000002,"15min":0.49399999999999999},"max":{"1min":0.94799999999999995,"5min":0.97599999999999998,"15min":1.3129999999999999},"last":0.69599999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.35899999999999999},"max":{"1min":0.77600000000000002,"5min":1.99,"15min":1.99},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.45300000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81100000000000005,"5min":0.89300000000000002,"15min":0.92800000000000005},"last":0.64400000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59799999999999998,"15min":0.621},"min":{"1min":0.45000000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72099999999999997,"5min":0.872,"15min":1.0820000000000001},"last":0.623},{"interface":"front","average":{"1min":0.629,"5min":0.65300000000000002,"15min":0.66600000000000004},"min":{"1min":0.46100000000000002,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":0.81299999999999994,"5min":1.96,"15min":1.96},"last":0.80800000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:56:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.434,"5min":0.376,"15min":0.376},"max":{"1min":0.88500000000000001,"5min":0.93200000000000005,"15min":1.0429999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68500000000000005,"15min":0.70599999999999996},"min":{"1min":0.50900000000000001,"5min":0.46400000000000002,"15min":0.44900000000000001},"max":{"1min":0.92300000000000004,"5min":1.103,"15min":1.107},"last":0.73099999999999998}]}]},{"osd":3,"up_from":15,"seq":64424509623,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36064,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335776,"statfs":{"total":96636764160,"available":96599834624,"internally_reserved":0,"allocated":942080,"data_stored":454495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45000000000000001,"15min":0.45300000000000001},"min":{"1min":0.315,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69199999999999995,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.49099999999999999,"15min":0.51000000000000001},"min":{"1min":0.36699999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.75900000000000001,"5min":1.163,"15min":1.738},"last":0.441}]},{"osd":1,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.46800000000000003},"min":{"1min":0.34899999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67500000000000004,"5min":0.77800000000000002,"15min":1.141},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.54100000000000004,"15min":0.55000000000000004},"min":{"1min":0.37,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67200000000000004,"5min":1.952,"15min":2.4910000000000001},"last":0.45800000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.55700000000000005},"min":{"1min":0.38700000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.76800000000000002,"5min":0.85399999999999998,"15min":1.821},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56899999999999995,"15min":0.60199999999999998},"min":{"1min":0.45700000000000002,"5min":0.372,"15min":0.35399999999999998},"max":{"1min":0.73799999999999999,"5min":2.855,"15min":5.1219999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78500000000000003,"15min":0.79500000000000004},"min":{"1min":0.56399999999999995,"5min":0.432,"15min":0.432},"max":{"1min":0.95899999999999996,"5min":4.234,"15min":4.234},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.754,"15min":0.79300000000000004},"min":{"1min":0.61899999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":1.111,"15min":2.2690000000000001},"last":0.57099999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76200000000000001,"15min":0.79300000000000004},"min":{"1min":0.55800000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.95399999999999996,"5min":1.0640000000000001,"15min":2.5350000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.54900000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97799999999999998,"5min":1.8300000000000001,"15min":1.9079999999999999},"last":0.58299999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.78700000000000003,"15min":0.81299999999999994},"min":{"1min":0.65200000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0589999999999999,"5min":1.599,"15min":1.956},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78000000000000003,"15min":0.80600000000000005},"min":{"1min":0.56799999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.99199999999999999,"5min":1.129,"15min":1.974},"last":0.60099999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81299999999999994,"15min":0.84099999999999997},"min":{"1min":0.64500000000000002,"5min":0.54700000000000004,"15min":0.51900000000000002},"max":{"1min":1.077,"5min":1.1279999999999999,"15min":1.9279999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.84299999999999997,"15min":0.88600000000000001},"min":{"1min":0.70899999999999996,"5min":0.60599999999999998,"15min":0.48799999999999999},"max":{"1min":1.0680000000000001,"5min":1.623,"15min":5.0759999999999996},"last":0.61899999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509622,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40160,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331680,"statfs":{"total":96636764160,"available":96595640320,"internally_reserved":0,"allocated":942080,"data_stored":454495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.49299999999999999,"15min":0.47799999999999998},"min":{"1min":0.34399999999999997,"5min":0.307,"15min":0.28199999999999997},"max":{"1min":0.64100000000000001,"5min":5.4870000000000001,"15min":5.4870000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.59899999999999998,"15min":0.53800000000000003},"min":{"1min":0.39100000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72299999999999998,"5min":10.161,"15min":10.161},"last":0.38200000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.64700000000000002},"min":{"1min":0.52800000000000002,"5min":0.41099999999999998,"15min":0.39900000000000002},"max":{"1min":0.82799999999999996,"5min":2.4740000000000002,"15min":5.7590000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55900000000000005},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.29299999999999998},"max":{"1min":0.71499999999999997,"5min":2.911,"15min":5.8070000000000004},"last":0.40899999999999997}]},{"osd":3,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.58699999999999997,"15min":0.60199999999999998},"min":{"1min":0.45000000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.753,"5min":1.1990000000000001,"15min":2.7599999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.59499999999999997,"15min":0.61099999999999999},"min":{"1min":0.47499999999999998,"5min":0.41099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":2.7999999999999998,"15min":5.7190000000000003},"last":0.44400000000000001}]},{"osd":4,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.81100000000000005},"min":{"1min":0.63600000000000001,"5min":0.57999999999999996,"15min":0.53000000000000003},"max":{"1min":1.143,"5min":2.4449999999999998,"15min":3.8940000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81200000000000006,"15min":0.80100000000000005},"min":{"1min":0.61699999999999999,"5min":0.53700000000000003,"15min":0.42999999999999999},"max":{"1min":1.0600000000000001,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":0.81999999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.84299999999999997,"15min":0.81200000000000006},"min":{"1min":0.65500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.004,"5min":5.601,"15min":5.601},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.81499999999999995,"15min":0.84099999999999997},"min":{"1min":0.57699999999999996,"5min":0.55100000000000005,"15min":0.54900000000000004},"max":{"1min":1.02,"5min":2.569,"15min":2.9260000000000002},"last":0.70499999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.84099999999999997,"15min":0.83999999999999997},"min":{"1min":0.65200000000000002,"5min":0.55900000000000005,"15min":0.52400000000000002},"max":{"1min":1.0780000000000001,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77600000000000002,"15min":0.80900000000000005},"min":{"1min":0.59799999999999998,"5min":0.53800000000000003,"15min":0.505},"max":{"1min":1.0940000000000001,"5min":1.1739999999999999,"15min":2.121},"last":0.83699999999999997}]},{"osd":7,"last update":"Tue Dec 19 16:56:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.872,"15min":0.83599999999999997},"min":{"1min":0.61399999999999999,"5min":0.56699999999999995,"15min":0.52300000000000002},"max":{"1min":1.0369999999999999,"5min":5.673,"15min":5.673},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.82999999999999996,"15min":0.85299999999999998},"min":{"1min":0.60899999999999999,"5min":0.60599999999999998,"15min":0.57799999999999996},"max":{"1min":1.123,"5min":1.429,"15min":2.0529999999999999},"last":0.80100000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509623,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50012,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321828,"statfs":{"total":96636764160,"available":96585551872,"internally_reserved":0,"allocated":937984,"data_stored":454529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.432,"15min":0.45500000000000002},"min":{"1min":0.30499999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.70299999999999996,"5min":0.70399999999999996,"15min":2.3050000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51400000000000001,"15min":0.52100000000000002},"min":{"1min":0.34599999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.74299999999999999,"5min":0.84199999999999997,"15min":0.89800000000000002},"last":0.371}]},{"osd":2,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.496,"15min":0.51900000000000002},"min":{"1min":0.37,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66500000000000004,"5min":0.80100000000000005,"15min":1.992},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.40300000000000002,"5min":0.35999999999999999,"15min":0.29799999999999999},"max":{"1min":0.81899999999999995,"5min":0.82899999999999996,"15min":0.97299999999999998},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59399999999999997,"15min":0.63200000000000001},"min":{"1min":0.48199999999999998,"5min":0.36899999999999999,"15min":0.34799999999999998},"max":{"1min":0.871,"5min":0.871,"15min":4.133},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36499999999999999,"5min":0.35799999999999998,"15min":0.33700000000000002},"max":{"1min":0.755,"5min":0.76400000000000001,"15min":0.91900000000000004},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.77300000000000002},"min":{"1min":0.53400000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94099999999999995,"5min":1.202,"15min":2.1949999999999998},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.56899999999999995,"5min":0.44400000000000001,"15min":0.42399999999999999},"max":{"1min":1.0580000000000001,"5min":1.181,"15min":2.048},"last":0.70999999999999996}]},{"osd":5,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77600000000000002,"15min":0.79200000000000004},"min":{"1min":0.60699999999999998,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.014,"5min":1.288,"15min":2.1269999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79700000000000004,"15min":0.82499999999999996},"min":{"1min":0.54700000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.016,"5min":1.3380000000000001,"15min":4.0460000000000003},"last":0.97499999999999998}]},{"osd":6,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.80600000000000005},"min":{"1min":0.496,"5min":0.48299999999999998,"15min":0.41999999999999998},"max":{"1min":1.048,"5min":1.0740000000000001,"15min":2.1499999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79800000000000004,"15min":0.81499999999999995},"min":{"1min":0.59499999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92400000000000004,"5min":1.391,"15min":2.0880000000000001},"last":0.95399999999999996}]},{"osd":7,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83299999999999996,"15min":0.85899999999999999},"min":{"1min":0.66900000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.002,"5min":1.2230000000000001,"15min":4.093},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.81399999999999995,"15min":0.84499999999999997},"min":{"1min":0.65900000000000003,"5min":0.58999999999999997,"15min":0.53800000000000003},"max":{"1min":1.097,"5min":1.147,"15min":2.2349999999999999},"last":0.90700000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542328,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1508,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1544192,"data_stored":1046953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.43099999999999999,"15min":0.42699999999999999},"min":{"1min":0.34999999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.504,"15min":0.48799999999999999},"min":{"1min":0.42999999999999999,"5min":0.28299999999999997,"15min":0.26300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.85499999999999998},"last":0.50800000000000001}]},{"osd":2,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.497,"15min":0.48999999999999999},"min":{"1min":0.43099999999999999,"5min":0.35899999999999999,"15min":0.29299999999999998},"max":{"1min":0.67900000000000005,"5min":0.77800000000000002,"15min":0.89700000000000002},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55500000000000005,"15min":0.53400000000000003},"min":{"1min":0.44500000000000001,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46000000000000002}]},{"osd":3,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.54000000000000004,"15min":0.52200000000000002},"min":{"1min":0.38400000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.77700000000000002,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.434},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.63100000000000001,"15min":0.61499999999999999},"min":{"1min":0.49399999999999999,"5min":0.41999999999999998,"15min":0.40899999999999997},"max":{"1min":0.83799999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55000000000000004}]},{"osd":4,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.79400000000000004,"15min":0.78900000000000003},"min":{"1min":0.58599999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2090000000000001,"5min":1.444,"15min":1.512},"last":0.626},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.80400000000000005,"15min":0.78900000000000003},"min":{"1min":0.61499999999999999,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.1020000000000001,"5min":1.177,"15min":1.52},"last":0.56599999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.80900000000000005,"15min":0.80400000000000005},"min":{"1min":0.59099999999999997,"5min":0.502,"15min":0.41499999999999998},"max":{"1min":1.1599999999999999,"5min":1.294,"15min":1.748},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.81699999999999995,"15min":0.79500000000000004},"min":{"1min":0.65300000000000002,"5min":0.55000000000000004,"15min":0.51600000000000001},"max":{"1min":1.097,"5min":1.256,"15min":1.649},"last":0.65400000000000003}]},{"osd":6,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.82299999999999995,"15min":0.79200000000000004},"min":{"1min":0.56200000000000006,"5min":0.52300000000000002,"15min":0.46999999999999997},"max":{"1min":1.2509999999999999,"5min":1.429,"15min":1.429},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.82199999999999995,"15min":0.81299999999999994},"min":{"1min":0.628,"5min":0.54200000000000004,"15min":0.44700000000000001},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.66000000000000003}]},{"osd":7,"last update":"Tue Dec 19 16:56:47 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.84699999999999998,"15min":0.82099999999999995},"min":{"1min":0.68500000000000005,"5min":0.505,"15min":0.48999999999999999},"max":{"1min":1.3129999999999999,"5min":1.377,"15min":1.377},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.83499999999999996,"15min":0.81799999999999995},"min":{"1min":0.62,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.2649999999999999,"5min":1.3300000000000001,"15min":1.661},"last":0.70699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:57:27.621 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T16:57:27.621 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T16:57:27.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f6695139640 1 Processor -- start 2023-12-19T16:57:27.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f6695139640 1 -- start start 2023-12-19T16:57:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f6695139640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f669015cc70 0x7f66900b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f6695139640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f669015d050 0x7f66900b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f6695139640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66900cae70 0x7f66900cb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f6695139640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6690161730 con 0x7f66900cae70 2023-12-19T16:57:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f6695139640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66900cb860 con 0x7f669015d050 2023-12-19T16:57:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f6695139640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6690160ab0 con 0x7f669015cc70 2023-12-19T16:57:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f668f59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66900cae70 0x7f66900cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f668f59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66900cae70 0x7f66900cb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52836/0 (socket says 172.21.15.26:52836) 2023-12-19T16:57:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f668f59e640 1 -- 172.21.15.26:0/2264922799 learned_addr learned my addr 172.21.15.26:0/2264922799 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f6687fff640 1 --2- 172.21.15.26:0/2264922799 >> v2:172.21.15.160:3300/0 conn(0x7f669015d050 0x7f66900b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f668ed9d640 1 --2- 172.21.15.26:0/2264922799 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f669015cc70 0x7f66900b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f668f59e640 1 -- 172.21.15.26:0/2264922799 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f669015cc70 msgr2=0x7f66900b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f668f59e640 1 --2- 172.21.15.26:0/2264922799 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f669015cc70 0x7f66900b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f668f59e640 1 -- 172.21.15.26:0/2264922799 >> v2:172.21.15.160:3300/0 conn(0x7f669015d050 msgr2=0x7f66900b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f668f59e640 1 --2- 172.21.15.26:0/2264922799 >> v2:172.21.15.160:3300/0 conn(0x7f669015d050 0x7f66900b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.805+0000 7f668f59e640 1 -- 172.21.15.26:0/2264922799 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6690160d90 con 0x7f66900cae70 2023-12-19T16:57:27.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.809+0000 7f668f59e640 1 --2- 172.21.15.26:0/2264922799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66900cae70 0x7f66900cb320 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f66900b1660 tx=0x7f668801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa47f0bb293014d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:27.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.809+0000 7f668e59c640 1 -- 172.21.15.26:0/2264922799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880a5820 con 0x7f66900cae70 2023-12-19T16:57:27.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.809+0000 7f668e59c640 1 -- 172.21.15.26:0/2264922799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66880a5980 con 0x7f66900cae70 2023-12-19T16:57:27.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.809+0000 7f668e59c640 1 -- 172.21.15.26:0/2264922799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880a5d10 con 0x7f66900cae70 2023-12-19T16:57:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.809+0000 7f6695139640 1 -- 172.21.15.26:0/2264922799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66900cae70 msgr2=0x7f66900cb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.809+0000 7f6695139640 1 --2- 172.21.15.26:0/2264922799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66900cae70 0x7f66900cb320 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f66900b1660 tx=0x7f668801c9a0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.809+0000 7f6695139640 1 -- 172.21.15.26:0/2264922799 shutdown_connections 2023-12-19T16:57:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.809+0000 7f6695139640 1 --2- 172.21.15.26:0/2264922799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66900cae70 0x7f66900cb320 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.809+0000 7f6695139640 1 --2- 172.21.15.26:0/2264922799 >> v2:172.21.15.160:3300/0 conn(0x7f669015d050 0x7f66900b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.809+0000 7f6695139640 1 --2- 172.21.15.26:0/2264922799 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f669015cc70 0x7f66900b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.809+0000 7f6695139640 1 -- 172.21.15.26:0/2264922799 >> 172.21.15.26:0/2264922799 conn(0x7f66900cefe0 msgr2=0x7f66900cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:27.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.809+0000 7f6695139640 1 -- 172.21.15.26:0/2264922799 shutdown_connections 2023-12-19T16:57:27.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.809+0000 7f6695139640 1 -- 172.21.15.26:0/2264922799 wait complete. 2023-12-19T16:57:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f6695139640 1 Processor -- start 2023-12-19T16:57:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f6695139640 1 -- start start 2023-12-19T16:57:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f6695139640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f66900cae70 0x7f66901583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f6695139640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f669015cc70 0x7f66900b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f6695139640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669015d050 0x7f66900c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f6695139640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6690158910 con 0x7f669015d050 2023-12-19T16:57:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f6695139640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66900b3540 con 0x7f669015cc70 2023-12-19T16:57:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f6695139640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66900b3820 con 0x7f66900cae70 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f668f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669015d050 0x7f66900c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f668f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669015d050 0x7f66900c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52838/0 (socket says 172.21.15.26:52838) 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f668f59e640 1 -- 172.21.15.26:0/1253183985 learned_addr learned my addr 172.21.15.26:0/1253183985 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f6687fff640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f669015cc70 0x7f66900b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f668ed9d640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f66900cae70 0x7f66901583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f668f59e640 1 -- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f66900cae70 msgr2=0x7f66901583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f668f59e640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f66900cae70 0x7f66901583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f668f59e640 1 -- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f669015cc70 msgr2=0x7f66900b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f668f59e640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f669015cc70 0x7f66900b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f668f59e640 1 -- 172.21.15.26:0/1253183985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6688009b30 con 0x7f669015d050 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f6687fff640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f669015cc70 0x7f66900b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f668f59e640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669015d050 0x7f66900c60b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f668801b050 tx=0x7f6688019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f66877fe640 1 -- 172.21.15.26:0/1253183985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880b8670 con 0x7f669015d050 2023-12-19T16:57:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f66877fe640 1 -- 172.21.15.26:0/1253183985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66880b87d0 con 0x7f669015d050 2023-12-19T16:57:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f66877fe640 1 -- 172.21.15.26:0/1253183985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880b8bc0 con 0x7f669015d050 2023-12-19T16:57:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f6695139640 1 -- 172.21.15.26:0/1253183985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66900b3b00 con 0x7f669015d050 2023-12-19T16:57:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.813+0000 7f6695139640 1 -- 172.21.15.26:0/1253183985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66900b40c0 con 0x7f669015d050 2023-12-19T16:57:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.817+0000 7f6695139640 1 -- 172.21.15.26:0/1253183985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66900b42b0 con 0x7f669015d050 2023-12-19T16:57:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.817+0000 7f66877fe640 1 -- 172.21.15.26:0/1253183985 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f66880b8d20 con 0x7f669015d050 2023-12-19T16:57:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.817+0000 7f66877fe640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6670065b80 0x7f6670068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.817+0000 7f66877fe640 1 -- 172.21.15.26:0/1253183985 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f668811eb10 con 0x7f669015d050 2023-12-19T16:57:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.817+0000 7f668ed9d640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6670065b80 0x7f6670068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.817+0000 7f668ed9d640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6670065b80 0x7f6670068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66900b4e10 tx=0x7f6680008040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:27.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:27.821+0000 7f66877fe640 1 -- 172.21.15.26:0/1253183985 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f66880ec160 con 0x7f669015d050 2023-12-19T16:57:28.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.033+0000 7f6695139640 1 -- 172.21.15.26:0/1253183985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f66900b42b0 con 0x7f669015d050 2023-12-19T16:57:28.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.037+0000 7f66877fe640 1 -- 172.21.15.26:0/1253183985 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6688009510 con 0x7f669015d050 2023-12-19T16:57:28.042 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2023-12-19T16:57:28.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.037+0000 7f6695139640 1 -- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6670065b80 msgr2=0x7f6670068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:28.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.037+0000 7f6695139640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6670065b80 0x7f6670068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66900b4e10 tx=0x7f6680008040 comp rx=0 tx=0).stop 2023-12-19T16:57:28.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.037+0000 7f6695139640 1 -- 172.21.15.26:0/1253183985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669015d050 msgr2=0x7f66900c60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:28.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.037+0000 7f6695139640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669015d050 0x7f66900c60b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f668801b050 tx=0x7f6688019740 comp rx=0 tx=0).stop 2023-12-19T16:57:28.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.037+0000 7f6695139640 1 -- 172.21.15.26:0/1253183985 shutdown_connections 2023-12-19T16:57:28.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.037+0000 7f6695139640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f669015d050 0x7f66900c60b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.037+0000 7f6695139640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f669015cc70 0x7f66900b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.037+0000 7f6695139640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6670065b80 0x7f6670068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.037+0000 7f6695139640 1 --2- 172.21.15.26:0/1253183985 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f66900cae70 0x7f66901583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.037+0000 7f6695139640 1 -- 172.21.15.26:0/1253183985 >> 172.21.15.26:0/1253183985 conn(0x7f66900cefe0 msgr2=0x7f66900c3740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.037+0000 7f6695139640 1 -- 172.21.15.26:0/1253183985 shutdown_connections 2023-12-19T16:57:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.037+0000 7f6695139640 1 -- 172.21.15.26:0/1253183985 wait complete. 2023-12-19T16:57:28.064 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:57:28.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.201+0000 7f6b95d8f640 1 Processor -- start 2023-12-19T16:57:28.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.201+0000 7f6b95d8f640 1 -- start start 2023-12-19T16:57:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b95d8f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b9015c7f0 0x7f6b9006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:28.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b95d8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b900adb80 0x7f6b900cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:28.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b95d8f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b9006ced0 con 0x7f6b900ccc20 2023-12-19T16:57:28.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b95d8f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b9006d770 con 0x7f6b9015c7f0 2023-12-19T16:57:28.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b95d8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9006d1b0 con 0x7f6b900adb80 2023-12-19T16:57:28.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b8f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b9015c7f0 0x7f6b9006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:28.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b8f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b9015c7f0 0x7f6b9006c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52846/0 (socket says 172.21.15.26:52846) 2023-12-19T16:57:28.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b8ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6b900ccc20 0x7f6b900b30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53646/0 (socket says 172.21.15.26:53646) 2023-12-19T16:57:28.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b8f7fe640 1 -- 172.21.15.26:0/1593471987 learned_addr learned my addr 172.21.15.26:0/1593471987 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:28.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b8effd640 1 --2- 172.21.15.26:0/1593471987 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b900adb80 0x7f6b900cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:28.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b8f7fe640 1 -- 172.21.15.26:0/1593471987 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b900adb80 msgr2=0x7f6b900cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:28.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b8f7fe640 1 --2- 172.21.15.26:0/1593471987 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b900adb80 0x7f6b900cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b8f7fe640 1 -- 172.21.15.26:0/1593471987 >> v1:172.21.15.26:6789/0 conn(0x7f6b900ccc20 legacy=0x7f6b900b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:28.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b8f7fe640 1 -- 172.21.15.26:0/1593471987 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9006d490 con 0x7f6b9015c7f0 2023-12-19T16:57:28.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.205+0000 7f6b8e7fc640 1 -- 172.21.15.26:0/1593471987 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367807902 0 0) 0x7f6b9006ced0 con 0x7f6b900ccc20 2023-12-19T16:57:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b8f7fe640 1 --2- 172.21.15.26:0/1593471987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b9015c7f0 0x7f6b9006c7f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009870 tx=0x7f6b8001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14683af4bc4cc068 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b8e7fc640 1 -- 172.21.15.26:0/1593471987 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b800ab650 con 0x7f6b9015c7f0 2023-12-19T16:57:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b8e7fc640 1 -- 172.21.15.26:0/1593471987 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b800ab7b0 con 0x7f6b9015c7f0 2023-12-19T16:57:28.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b8e7fc640 1 -- 172.21.15.26:0/1593471987 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b800abb40 con 0x7f6b9015c7f0 2023-12-19T16:57:28.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 -- 172.21.15.26:0/1593471987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b9015c7f0 msgr2=0x7f6b9006c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:28.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 --2- 172.21.15.26:0/1593471987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b9015c7f0 0x7f6b9006c7f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009870 tx=0x7f6b8001c540 comp rx=0 tx=0).stop 2023-12-19T16:57:28.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 -- 172.21.15.26:0/1593471987 shutdown_connections 2023-12-19T16:57:28.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 --2- 172.21.15.26:0/1593471987 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b900adb80 0x7f6b900cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 --2- 172.21.15.26:0/1593471987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b9015c7f0 0x7f6b9006c7f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 -- 172.21.15.26:0/1593471987 >> 172.21.15.26:0/1593471987 conn(0x7f6b900658e0 msgr2=0x7f6b90065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:28.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 -- 172.21.15.26:0/1593471987 shutdown_connections 2023-12-19T16:57:28.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 -- 172.21.15.26:0/1593471987 wait complete. 2023-12-19T16:57:28.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 Processor -- start 2023-12-19T16:57:28.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 -- start start 2023-12-19T16:57:28.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b900adb80 0x7f6b900b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:28.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b900ccc20 0x7f6b9014c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b9015c7f0 0x7f6b90148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9014d620 con 0x7f6b9015c7f0 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9014a4f0 con 0x7f6b900ccc20 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.209+0000 7f6b95d8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9014f980 con 0x7f6b900adb80 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b8ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b9015c7f0 0x7f6b90148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b8ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b9015c7f0 0x7f6b90148ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52850/0 (socket says 172.21.15.26:52850) 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b8ffff640 1 -- 172.21.15.26:0/3462308144 learned_addr learned my addr 172.21.15.26:0/3462308144 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b8f7fe640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b900adb80 0x7f6b900b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b8effd640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b900ccc20 0x7f6b9014c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b8ffff640 1 -- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b900adb80 msgr2=0x7f6b900b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b8ffff640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b900adb80 0x7f6b900b16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b8ffff640 1 -- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b900ccc20 msgr2=0x7f6b9014c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b8ffff640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b900ccc20 0x7f6b9014c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b8ffff640 1 -- 172.21.15.26:0/3462308144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b800094d0 con 0x7f6b9015c7f0 2023-12-19T16:57:28.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b8ffff640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b9015c7f0 0x7f6b90148ef0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800c7e0 tx=0x7f6b8800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:28.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b77fff640 1 -- 172.21.15.26:0/3462308144 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8800a640 con 0x7f6b9015c7f0 2023-12-19T16:57:28.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b77fff640 1 -- 172.21.15.26:0/3462308144 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8800a7a0 con 0x7f6b9015c7f0 2023-12-19T16:57:28.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b95d8f640 1 -- 172.21.15.26:0/3462308144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b9014fc60 con 0x7f6b9015c7f0 2023-12-19T16:57:28.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b77fff640 1 -- 172.21.15.26:0/3462308144 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8800ab50 con 0x7f6b9015c7f0 2023-12-19T16:57:28.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b95d8f640 1 -- 172.21.15.26:0/3462308144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b90150220 con 0x7f6b9015c7f0 2023-12-19T16:57:28.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b95d8f640 1 -- 172.21.15.26:0/3462308144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9014a6c0 con 0x7f6b9015c7f0 2023-12-19T16:57:28.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.213+0000 7f6b77fff640 1 -- 172.21.15.26:0/3462308144 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6b8800acb0 con 0x7f6b9015c7f0 2023-12-19T16:57:28.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.217+0000 7f6b77fff640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b7c065bb0 0x7f6b7c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:28.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.217+0000 7f6b77fff640 1 -- 172.21.15.26:0/3462308144 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f6b88082e60 con 0x7f6b9015c7f0 2023-12-19T16:57:28.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.217+0000 7f6b8f7fe640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b7c065bb0 0x7f6b7c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:28.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.217+0000 7f6b8f7fe640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b7c065bb0 0x7f6b7c068070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b800af000 tx=0x7f6b800b6040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:28.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.221+0000 7f6b77fff640 1 -- 172.21.15.26:0/3462308144 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6b9014a6c0 con 0x7f6b9015c7f0 2023-12-19T16:57:28.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.409+0000 7f6b95d8f640 1 -- 172.21.15.26:0/3462308144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b9014a6c0 con 0x7f6b9015c7f0 2023-12-19T16:57:28.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.409+0000 7f6b77fff640 1 -- 172.21.15.26:0/3462308144 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f6b9014a6c0 con 0x7f6b9015c7f0 2023-12-19T16:57:28.416 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:28.416 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/436365775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":436365775},{"type":"v1","addr":"172.21.15.160:6836","nonce":436365775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3014042376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3014042376},{"type":"v1","addr":"172.21.15.160:6837","nonce":3014042376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1357342534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1357342534},{"type":"v1","addr":"172.21.15.26:6835","nonce":1357342534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:57:23.100695+0000","modified":"2023-12-19T16:57:26.118941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6522":{"gid":6522,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/325854666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":325854666},{"type":"v1","addr":"172.21.15.26:6834","nonce":325854666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-19T16:57:28.416 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 33 2023-12-19T16:57:28.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.413+0000 7f6b95d8f640 1 -- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b7c065bb0 msgr2=0x7f6b7c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:28.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.413+0000 7f6b95d8f640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b7c065bb0 0x7f6b7c068070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6b800af000 tx=0x7f6b800b6040 comp rx=0 tx=0).stop 2023-12-19T16:57:28.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.413+0000 7f6b95d8f640 1 -- 172.21.15.26:0/3462308144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b9015c7f0 msgr2=0x7f6b90148ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:28.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.413+0000 7f6b95d8f640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b9015c7f0 0x7f6b90148ef0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800c7e0 tx=0x7f6b8800cca0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.413+0000 7f6b95d8f640 1 -- 172.21.15.26:0/3462308144 shutdown_connections 2023-12-19T16:57:28.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.413+0000 7f6b95d8f640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b9015c7f0 0x7f6b90148ef0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.413+0000 7f6b95d8f640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b900ccc20 0x7f6b9014c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.413+0000 7f6b95d8f640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b7c065bb0 0x7f6b7c068070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.413+0000 7f6b95d8f640 1 --2- 172.21.15.26:0/3462308144 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b900adb80 0x7f6b900b16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.413+0000 7f6b95d8f640 1 -- 172.21.15.26:0/3462308144 >> 172.21.15.26:0/3462308144 conn(0x7f6b900658e0 msgr2=0x7f6b900ae4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:28.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.413+0000 7f6b95d8f640 1 -- 172.21.15.26:0/3462308144 shutdown_connections 2023-12-19T16:57:28.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.413+0000 7f6b95d8f640 1 -- 172.21.15.26:0/3462308144 wait complete. 2023-12-19T16:57:28.673 INFO:tasks.ceph.mds.a.smithi026.stderr:2023-12-19T16:57:28.669+0000 7eff4c7fe640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T16:57:28.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T16:57:23.100695+0000', 'modified': '2023-12-19T16:57:26.118941+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6522}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6522': {'gid': 6522, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6834/325854666', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 325854666}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 325854666}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T16:57:28.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T16:57:28.842 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-19T16:57:28.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.985+0000 7f254741f640 1 Processor -- start 2023-12-19T16:57:28.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.985+0000 7f254741f640 1 -- start start 2023-12-19T16:57:28.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.985+0000 7f254741f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f25400b88e0 0x7f25400b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:28.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.985+0000 7f254741f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25400d2960 con 0x7f254015cf60 2023-12-19T16:57:28.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.985+0000 7f254741f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25400b4f40 con 0x7f25400b4b40 2023-12-19T16:57:28.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.985+0000 7f254741f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25400d15f0 con 0x7f25400b88e0 2023-12-19T16:57:28.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.985+0000 7f25451bb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f254015cf60 0x7f25400d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53648/0 (socket says 172.21.15.26:53648) 2023-12-19T16:57:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.985+0000 7f25451bb640 1 -- 172.21.15.26:0/3237672305 learned_addr learned my addr 172.21.15.26:0/3237672305 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.985+0000 7f25449ba640 1 --2- 172.21.15.26:0/3237672305 >> v2:172.21.15.26:3300/0 conn(0x7f25400b88e0 0x7f25400b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.985+0000 7f25449ba640 1 -- 172.21.15.26:0/3237672305 >> v1:172.21.15.160:6789/0 conn(0x7f25400b4b40 legacy=0x7f25400b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.985+0000 7f25449ba640 1 -- 172.21.15.26:0/3237672305 >> v1:172.21.15.26:6789/0 conn(0x7f254015cf60 legacy=0x7f25400d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.985+0000 7f2537fff640 1 -- 172.21.15.26:0/3237672305 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166084535 0 0) 0x7f25400d2960 con 0x7f254015cf60 2023-12-19T16:57:28.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.985+0000 7f25449ba640 1 -- 172.21.15.26:0/3237672305 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25400d18c0 con 0x7f25400b88e0 2023-12-19T16:57:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.989+0000 7f25449ba640 1 --2- 172.21.15.26:0/3237672305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25400b88e0 0x7f25400b4600 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f25300097c0 tx=0x7f253001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6ec1d3a5354b2a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:28.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.989+0000 7f2537fff640 1 -- 172.21.15.26:0/3237672305 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300a8970 con 0x7f25400b88e0 2023-12-19T16:57:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.989+0000 7f2537fff640 1 -- 172.21.15.26:0/3237672305 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25300a8ad0 con 0x7f25400b88e0 2023-12-19T16:57:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.989+0000 7f2537fff640 1 -- 172.21.15.26:0/3237672305 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300a8e60 con 0x7f25400b88e0 2023-12-19T16:57:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.989+0000 7f254741f640 1 -- 172.21.15.26:0/3237672305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25400b88e0 msgr2=0x7f25400b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.989+0000 7f254741f640 1 --2- 172.21.15.26:0/3237672305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25400b88e0 0x7f25400b4600 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f25300097c0 tx=0x7f253001ec70 comp rx=0 tx=0).stop 2023-12-19T16:57:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.989+0000 7f254741f640 1 -- 172.21.15.26:0/3237672305 shutdown_connections 2023-12-19T16:57:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.989+0000 7f254741f640 1 --2- 172.21.15.26:0/3237672305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25400b88e0 0x7f25400b4600 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.989+0000 7f254741f640 1 -- 172.21.15.26:0/3237672305 >> 172.21.15.26:0/3237672305 conn(0x7f2540065ee0 msgr2=0x7f25400cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.989+0000 7f254741f640 1 -- 172.21.15.26:0/3237672305 shutdown_connections 2023-12-19T16:57:28.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.989+0000 7f254741f640 1 -- 172.21.15.26:0/3237672305 wait complete. 2023-12-19T16:57:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f254741f640 1 Processor -- start 2023-12-19T16:57:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f254741f640 1 -- start start 2023-12-19T16:57:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f254741f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25400b4b40 0x7f2540153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f254741f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25400b88e0 0x7f2540151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f254741f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f254015cf60 0x7f25401521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f254741f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2540150210 con 0x7f25400b88e0 2023-12-19T16:57:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f254741f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254015f560 con 0x7f254015cf60 2023-12-19T16:57:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f254741f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254015f840 con 0x7f25400b4b40 2023-12-19T16:57:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f25449ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25400b88e0 0x7f2540151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f25449ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25400b88e0 0x7f2540151c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52874/0 (socket says 172.21.15.26:52874) 2023-12-19T16:57:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f25449ba640 1 -- 172.21.15.26:0/4111668675 learned_addr learned my addr 172.21.15.26:0/4111668675 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f25451bb640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25400b4b40 0x7f2540153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f25459bc640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f254015cf60 0x7f25401521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f25449ba640 1 -- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25400b4b40 msgr2=0x7f2540153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f25449ba640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25400b4b40 0x7f2540153600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f25449ba640 1 -- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f254015cf60 msgr2=0x7f25401521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f25449ba640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f254015cf60 0x7f25401521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f25449ba640 1 -- 172.21.15.26:0/4111668675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2530009420 con 0x7f25400b88e0 2023-12-19T16:57:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f25449ba640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25400b88e0 0x7f2540151c70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f253001c8a0 tx=0x7f253001d4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f2535ffb640 1 -- 172.21.15.26:0/4111668675 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300b1e10 con 0x7f25400b88e0 2023-12-19T16:57:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f2535ffb640 1 -- 172.21.15.26:0/4111668675 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2530004030 con 0x7f25400b88e0 2023-12-19T16:57:29.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f254741f640 1 -- 172.21.15.26:0/4111668675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254015fb20 con 0x7f25400b88e0 2023-12-19T16:57:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f2535ffb640 1 -- 172.21.15.26:0/4111668675 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300043a0 con 0x7f25400b88e0 2023-12-19T16:57:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.993+0000 7f254741f640 1 -- 172.21.15.26:0/4111668675 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2540153b40 con 0x7f25400b88e0 2023-12-19T16:57:29.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.997+0000 7f2535ffb640 1 -- 172.21.15.26:0/4111668675 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2530003620 con 0x7f25400b88e0 2023-12-19T16:57:29.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.997+0000 7f2535ffb640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2524065b80 0x7f2524068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.997+0000 7f2535ffb640 1 -- 172.21.15.26:0/4111668675 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f2530121420 con 0x7f25400b88e0 2023-12-19T16:57:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.997+0000 7f25451bb640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2524065b80 0x7f2524068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.997+0000 7f254a78c000 1 Processor -- start 2023-12-19T16:57:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.997+0000 7f254a78c000 1 -- start start 2023-12-19T16:57:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:28.997+0000 7f25451bb640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2524065b80 0x7f2524068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f253c007fd0 tx=0x7f253c007ba0 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c070 0x55ce8152c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce8152c990 0x55ce814e1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce814e21c0 0x55ce814e25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce81305c90 con 0x55ce814e21c0 2023-12-19T16:57:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce81337220 con 0x55ce8152c070 2023-12-19T16:57:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce813347f0 con 0x55ce8152c990 2023-12-19T16:57:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f25459bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce814e21c0 0x55ce814e25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f25459bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce814e21c0 0x55ce814e25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52880/0 (socket says 172.21.15.26:52880) 2023-12-19T16:57:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f25459bc640 1 -- 172.21.15.26:0/2622048942 learned_addr learned my addr 172.21.15.26:0/2622048942 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f25451bb640 1 --2- 172.21.15.26:0/2622048942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c070 0x55ce8152c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f25449ba640 1 --2- 172.21.15.26:0/2622048942 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce8152c990 0x55ce814e1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f25459bc640 1 -- 172.21.15.26:0/2622048942 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce8152c990 msgr2=0x55ce814e1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f25459bc640 1 --2- 172.21.15.26:0/2622048942 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce8152c990 0x55ce814e1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f25459bc640 1 -- 172.21.15.26:0/2622048942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c070 msgr2=0x55ce8152c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f25459bc640 1 --2- 172.21.15.26:0/2622048942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c070 0x55ce8152c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f25459bc640 1 -- 172.21.15.26:0/2622048942 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce8102c890 con 0x55ce814e21c0 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f25449ba640 1 --2- 172.21.15.26:0/2622048942 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce8152c990 0x55ce814e1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f25451bb640 1 --2- 172.21.15.26:0/2622048942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c070 0x55ce8152c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f25459bc640 1 --2- 172.21.15.26:0/2622048942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce814e21c0 0x55ce814e25a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2538001b90 tx=0x7f2538002050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ece4e7379c82923 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f252affd640 1 -- 172.21.15.26:0/2622048942 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538003b90 con 0x55ce814e21c0 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f252affd640 1 -- 172.21.15.26:0/2622048942 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2538003cf0 con 0x55ce814e21c0 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f252affd640 1 -- 172.21.15.26:0/2622048942 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253800ea00 con 0x55ce814e21c0 2023-12-19T16:57:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 -- 172.21.15.26:0/2622048942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce814e21c0 msgr2=0x55ce814e25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 --2- 172.21.15.26:0/2622048942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce814e21c0 0x55ce814e25a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2538001b90 tx=0x7f2538002050 comp rx=0 tx=0).stop 2023-12-19T16:57:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 -- 172.21.15.26:0/2622048942 shutdown_connections 2023-12-19T16:57:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 --2- 172.21.15.26:0/2622048942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce814e21c0 0x55ce814e25a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 --2- 172.21.15.26:0/2622048942 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce8152c990 0x55ce814e1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 --2- 172.21.15.26:0/2622048942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c070 0x55ce8152c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 -- 172.21.15.26:0/2622048942 >> 172.21.15.26:0/2622048942 conn(0x55ce814e8950 msgr2=0x55ce814e0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 -- 172.21.15.26:0/2622048942 shutdown_connections 2023-12-19T16:57:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 -- 172.21.15.26:0/2622048942 wait complete. 2023-12-19T16:57:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 -- start start 2023-12-19T16:57:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.001+0000 7f254a78c000 1 Processor -- start 2023-12-19T16:57:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce814e21c0 0x55ce814e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 0x55ce814ee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c990 0x55ce814ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce813300e0 con 0x55ce8152c070 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce812cf310 con 0x55ce8152c990 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce812ed770 con 0x55ce814e21c0 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 10 client.0 _ll_register_callbacks cb 0x55ce814cdb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f25449ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 0x55ce814ee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f25449ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 0x55ce814ee080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52894/0 (socket says 172.21.15.26:52894) 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f25449ba640 1 -- 172.21.15.26:0/2356700228 learned_addr learned my addr 172.21.15.26:0/2356700228 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f25451bb640 1 --2- 172.21.15.26:0/2356700228 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce814e21c0 0x55ce814e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f25459bc640 1 --2- 172.21.15.26:0/2356700228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c990 0x55ce814ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f25449ba640 1 -- 172.21.15.26:0/2356700228 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce814e21c0 msgr2=0x55ce814e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f25449ba640 1 --2- 172.21.15.26:0/2356700228 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce814e21c0 0x55ce814e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f25449ba640 1 -- 172.21.15.26:0/2356700228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c990 msgr2=0x55ce814ee5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f25449ba640 1 --2- 172.21.15.26:0/2356700228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c990 0x55ce814ee5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f25449ba640 1 -- 172.21.15.26:0/2356700228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253800f040 con 0x55ce8152c070 2023-12-19T16:57:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f25449ba640 1 --2- 172.21.15.26:0/2356700228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 0x55ce814ee080 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f253001b880 tx=0x7f25300f0ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8379b72f4d3731 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f251ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T16:57:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f251ffff640 1 -- 172.21.15.26:0/2356700228 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300f2730 con 0x55ce8152c070 2023-12-19T16:57:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f251ffff640 1 -- 172.21.15.26:0/2356700228 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f25300f6560 con 0x55ce8152c070 2023-12-19T16:57:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 1 -- 172.21.15.26:0/2356700228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce812ffc10 con 0x55ce8152c070 2023-12-19T16:57:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f251ffff640 1 -- 172.21.15.26:0/2356700228 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25300fa710 con 0x55ce8152c070 2023-12-19T16:57:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 10 client.6660 fetch_fsmap learned FSMap version 33 2023-12-19T16:57:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 1 -- 172.21.15.26:0/2356700228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55ce812f5260 con 0x55ce8152c070 2023-12-19T16:57:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f251ffff640 1 -- 172.21.15.26:0/2356700228 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ce812ffc10 con 0x55ce8152c070 2023-12-19T16:57:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f251ffff640 1 -- 172.21.15.26:0/2356700228 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f25300f2990 con 0x55ce8152c070 2023-12-19T16:57:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 10 client.6660 fetch_fsmap finished waiting for FSMap version 33 2023-12-19T16:57:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 10 client.6660 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T16:57:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 20 client.6660 populate_metadata read hostname 'smithi026' 2023-12-19T16:57:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 1 --2- 172.21.15.26:0/2356700228 >> [v2:172.21.15.26:6833/325854666,v1:172.21.15.26:6834/325854666] conn(0x55ce815338d0 0x55ce81533cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 4 client.6660 mds_command: new command op to 6522 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T16:57:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 1 -- 172.21.15.26:0/2356700228 --> [v2:172.21.15.26:6833/325854666,v1:172.21.15.26:6834/325854666] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce8132ea30 con 0x55ce815338d0 2023-12-19T16:57:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f25451bb640 1 --2- 172.21.15.26:0/2356700228 >> [v2:172.21.15.26:6833/325854666,v1:172.21.15.26:6834/325854666] conn(0x55ce815338d0 0x55ce81533cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f25451bb640 1 --2- 172.21.15.26:0/2356700228 >> [v2:172.21.15.26:6833/325854666,v1:172.21.15.26:6834/325854666] conn(0x55ce815338d0 0x55ce81533cb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=824de8bd08a04ed6 server_cookie=191525479230b88c in_seq=0 out_seq=0 2023-12-19T16:57:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f251ffff640 10 client.6660 ms_handle_connect on v2:172.21.15.26:6833/325854666 2023-12-19T16:57:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f251ffff640 1 -- 172.21.15.26:0/2356700228 <== mds.0 v2:172.21.15.26:6833/325854666 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ce8132ea30 con 0x55ce815338d0 2023-12-19T16:57:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f251ffff640 10 client.6660 handle_command_reply: tid=0 2023-12-19T16:57:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.005+0000 7f254a78c000 1 client.6660 shutdown 2023-12-19T16:57:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.009+0000 7f254a78c000 1 -- 172.21.15.26:0/2356700228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 msgr2=0x55ce814ee080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.009+0000 7f254a78c000 1 --2- 172.21.15.26:0/2356700228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 0x55ce814ee080 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f253001b880 tx=0x7f25300f0ce0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.009+0000 7f254a78c000 1 -- 172.21.15.26:0/2356700228 shutdown_connections 2023-12-19T16:57:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.009+0000 7f254a78c000 1 --2- 172.21.15.26:0/2356700228 >> [v2:172.21.15.26:6833/325854666,v1:172.21.15.26:6834/325854666] conn(0x55ce815338d0 0x55ce81533cb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.009+0000 7f254a78c000 1 --2- 172.21.15.26:0/2356700228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c990 0x55ce814ee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.009+0000 7f254a78c000 1 --2- 172.21.15.26:0/2356700228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 0x55ce814ee080 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.009+0000 7f251ffff640 0 client.6660 ms_handle_reset on v2:172.21.15.26:6833/325854666 2023-12-19T16:57:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.009+0000 7f254a78c000 1 --2- 172.21.15.26:0/2356700228 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce814e21c0 0x55ce814e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.009+0000 7f254a78c000 1 -- 172.21.15.26:0/2356700228 >> 172.21.15.26:0/2356700228 conn(0x55ce814e8950 msgr2=0x55ce814e0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.009+0000 7f254a78c000 1 -- 172.21.15.26:0/2356700228 shutdown_connections 2023-12-19T16:57:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.009+0000 7f254a78c000 1 -- 172.21.15.26:0/2356700228 wait complete. 2023-12-19T16:57:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.009+0000 7f254a78c000 20 client.6660 trim_cache size 0 max 16384 2023-12-19T16:57:29.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.021+0000 7f254a78c000 1 Processor -- start 2023-12-19T16:57:29.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.021+0000 7f254a78c000 1 -- start start 2023-12-19T16:57:29.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce814e21c0 0x55ce81583ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce8152c070 0x55ce81584410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c990 0x55ce81584950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce80fed940 con 0x55ce8152c990 2023-12-19T16:57:29.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce812d8860 con 0x55ce814e21c0 2023-12-19T16:57:29.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce814e4d70 con 0x55ce8152c070 2023-12-19T16:57:29.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f25459bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c990 0x55ce81584950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f25459bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c990 0x55ce81584950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52908/0 (socket says 172.21.15.26:52908) 2023-12-19T16:57:29.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f25459bc640 1 -- 172.21.15.26:0/3304966707 learned_addr learned my addr 172.21.15.26:0/3304966707 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:29.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f25451bb640 1 --2- 172.21.15.26:0/3304966707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce814e21c0 0x55ce81583ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f25449ba640 1 --2- 172.21.15.26:0/3304966707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce8152c070 0x55ce81584410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f25459bc640 1 -- 172.21.15.26:0/3304966707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce8152c070 msgr2=0x55ce81584410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f25459bc640 1 --2- 172.21.15.26:0/3304966707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce8152c070 0x55ce81584410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f25459bc640 1 -- 172.21.15.26:0/3304966707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce814e21c0 msgr2=0x55ce81583ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f25459bc640 1 --2- 172.21.15.26:0/3304966707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce814e21c0 0x55ce81583ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f25459bc640 1 -- 172.21.15.26:0/3304966707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce812db3b0 con 0x55ce8152c990 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f25449ba640 1 --2- 172.21.15.26:0/3304966707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce8152c070 0x55ce81584410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f25459bc640 1 --2- 172.21.15.26:0/3304966707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c990 0x55ce81584950 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2538017040 tx=0x7f253800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356ef8193b097dfb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f252b7fe640 1 -- 172.21.15.26:0/3304966707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538007ea0 con 0x55ce8152c990 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f252b7fe640 1 -- 172.21.15.26:0/3304966707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2538003c10 con 0x55ce8152c990 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f252b7fe640 1 -- 172.21.15.26:0/3304966707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253800e800 con 0x55ce8152c990 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 -- 172.21.15.26:0/3304966707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c990 msgr2=0x55ce81584950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 --2- 172.21.15.26:0/3304966707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c990 0x55ce81584950 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2538017040 tx=0x7f253800a000 comp rx=0 tx=0).stop 2023-12-19T16:57:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 -- 172.21.15.26:0/3304966707 shutdown_connections 2023-12-19T16:57:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 --2- 172.21.15.26:0/3304966707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c990 0x55ce81584950 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 --2- 172.21.15.26:0/3304966707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce8152c070 0x55ce81584410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 --2- 172.21.15.26:0/3304966707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce814e21c0 0x55ce81583ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 -- 172.21.15.26:0/3304966707 >> 172.21.15.26:0/3304966707 conn(0x55ce814e8950 msgr2=0x55ce814edf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 -- 172.21.15.26:0/3304966707 shutdown_connections 2023-12-19T16:57:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 -- 172.21.15.26:0/3304966707 wait complete. 2023-12-19T16:57:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 -- start start 2023-12-19T16:57:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.025+0000 7f254a78c000 1 Processor -- start 2023-12-19T16:57:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce814e21c0 0x55ce81588ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 0x55ce81589000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c990 0x55ce815840e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce8132d8f0 con 0x55ce8152c070 2023-12-19T16:57:29.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce81373ce0 con 0x55ce8152c990 2023-12-19T16:57:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce81331670 con 0x55ce814e21c0 2023-12-19T16:57:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 10 client.0 _ll_register_callbacks cb 0x55ce812de640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T16:57:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25449ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 0x55ce81589000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25451bb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce814e21c0 0x55ce81588ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25451bb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce814e21c0 0x55ce81588ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:38044/0 (socket says 172.21.15.26:38044) 2023-12-19T16:57:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25449ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 0x55ce81589000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52922/0 (socket says 172.21.15.26:52922) 2023-12-19T16:57:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25451bb640 1 -- 172.21.15.26:0/506968396 learned_addr learned my addr 172.21.15.26:0/506968396 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25459bc640 1 --2- 172.21.15.26:0/506968396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c990 0x55ce815840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25449ba640 1 -- 172.21.15.26:0/506968396 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce814e21c0 msgr2=0x55ce81588ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25449ba640 1 --2- 172.21.15.26:0/506968396 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce814e21c0 0x55ce81588ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25449ba640 1 -- 172.21.15.26:0/506968396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c990 msgr2=0x55ce815840e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:29.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25449ba640 1 --2- 172.21.15.26:0/506968396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c990 0x55ce815840e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25449ba640 1 -- 172.21.15.26:0/506968396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce8102c890 con 0x55ce8152c070 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25459bc640 1 --2- 172.21.15.26:0/506968396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c990 0x55ce815840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25449ba640 1 --2- 172.21.15.26:0/506968396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 0x55ce81589000 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f253009b490 tx=0x7f253001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc8f6ef5e744de2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f251ffff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f251ffff640 1 -- 172.21.15.26:0/506968396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300f62d0 con 0x55ce8152c070 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f251ffff640 1 -- 172.21.15.26:0/506968396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f25300f6560 con 0x55ce8152c070 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 -- 172.21.15.26:0/506968396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce812f5e00 con 0x55ce8152c070 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f251ffff640 1 -- 172.21.15.26:0/506968396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2530004500 con 0x55ce8152c070 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 10 client.6666 fetch_fsmap learned FSMap version 33 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 -- 172.21.15.26:0/506968396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55ce8102ccc0 con 0x55ce8152c070 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f251ffff640 1 -- 172.21.15.26:0/506968396 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ce812f5e00 con 0x55ce8152c070 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f251ffff640 1 -- 172.21.15.26:0/506968396 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f25300fa280 con 0x55ce8152c070 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 10 client.6666 fetch_fsmap finished waiting for FSMap version 33 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 10 client.6666 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T16:57:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 20 client.6666 populate_metadata read hostname 'smithi026' 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 --2- 172.21.15.26:0/506968396 >> [v2:172.21.15.26:6833/325854666,v1:172.21.15.26:6834/325854666] conn(0x55ce81584620 0x55ce81584a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 4 client.6666 mds_command: new command op to 6522 tid=0[{"prefix": "status"}] 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 -- 172.21.15.26:0/506968396 --> [v2:172.21.15.26:6833/325854666,v1:172.21.15.26:6834/325854666] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ce81329160 con 0x55ce81584620 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25451bb640 1 --2- 172.21.15.26:0/506968396 >> [v2:172.21.15.26:6833/325854666,v1:172.21.15.26:6834/325854666] conn(0x55ce81584620 0x55ce81584a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f251ffff640 10 client.6666 ms_handle_connect on v2:172.21.15.26:6833/325854666 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f25451bb640 1 --2- 172.21.15.26:0/506968396 >> [v2:172.21.15.26:6833/325854666,v1:172.21.15.26:6834/325854666] conn(0x55ce81584620 0x55ce81584a00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9af6a77f032fa58 server_cookie=7164d414dfadc9a1 in_seq=0 out_seq=0 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f251ffff640 1 -- 172.21.15.26:0/506968396 <== mds.0 v2:172.21.15.26:6833/325854666 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55ce81329160 con 0x55ce81584620 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f251ffff640 10 client.6666 handle_command_reply: tid=0 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 client.6666 shutdown 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 -- 172.21.15.26:0/506968396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 msgr2=0x55ce81589000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 --2- 172.21.15.26:0/506968396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 0x55ce81589000 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f253009b490 tx=0x7f253001ba10 comp rx=0 tx=0).stop 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 -- 172.21.15.26:0/506968396 shutdown_connections 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 --2- 172.21.15.26:0/506968396 >> [v2:172.21.15.26:6833/325854666,v1:172.21.15.26:6834/325854666] conn(0x55ce81584620 0x55ce81584a00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 --2- 172.21.15.26:0/506968396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ce8152c990 0x55ce815840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 --2- 172.21.15.26:0/506968396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55ce8152c070 0x55ce81589000 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 --2- 172.21.15.26:0/506968396 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55ce814e21c0 0x55ce81588ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f251ffff640 0 client.6666 ms_handle_reset on v2:172.21.15.26:6833/325854666 2023-12-19T16:57:29.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.029+0000 7f254a78c000 1 -- 172.21.15.26:0/506968396 >> 172.21.15.26:0/506968396 conn(0x55ce814e8950 msgr2=0x55ce81529060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:29.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254a78c000 1 -- 172.21.15.26:0/506968396 shutdown_connections 2023-12-19T16:57:29.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254a78c000 1 -- 172.21.15.26:0/506968396 wait complete. 2023-12-19T16:57:29.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254a78c000 20 client.6666 trim_cache size 0 max 16384 2023-12-19T16:57:29.045 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T16:57:29.045 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T16:57:29.045 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T16:57:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6522, 2023-12-19T16:57:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T16:57:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T16:57:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T16:57:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.9325342350000003, 2023-12-19T16:57:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 33, 2023-12-19T16:57:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 79, 2023-12-19T16:57:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 79, 2023-12-19T16:57:29.046 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 11.158126234999999 2023-12-19T16:57:29.046 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T16:57:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254741f640 1 -- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2524065b80 msgr2=0x7f2524068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254741f640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2524065b80 0x7f2524068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f253c007fd0 tx=0x7f253c007ba0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254741f640 1 -- 172.21.15.26:0/4111668675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25400b88e0 msgr2=0x7f2540151c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254741f640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25400b88e0 0x7f2540151c70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f253001c8a0 tx=0x7f253001d4b0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254741f640 1 -- 172.21.15.26:0/4111668675 shutdown_connections 2023-12-19T16:57:29.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254741f640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f254015cf60 0x7f25401521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254741f640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25400b88e0 0x7f2540151c70 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254741f640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2524065b80 0x7f2524068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254741f640 1 --2- 172.21.15.26:0/4111668675 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25400b4b40 0x7f2540153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254741f640 1 -- 172.21.15.26:0/4111668675 >> 172.21.15.26:0/4111668675 conn(0x7f2540065ee0 msgr2=0x7f25400d2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:29.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254741f640 1 -- 172.21.15.26:0/4111668675 shutdown_connections 2023-12-19T16:57:29.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.033+0000 7f254741f640 1 -- 172.21.15.26:0/4111668675 wait complete. 2023-12-19T16:57:29.056 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T16:57:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.193+0000 7f0cbd441640 1 Processor -- start 2023-12-19T16:57:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cbd441640 1 -- start start 2023-12-19T16:57:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cbd441640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb806b3d0 0x7f0cb80ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cbd441640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb806bc30 con 0x7f0cb8154220 2023-12-19T16:57:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cbd441640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb80cc6e0 con 0x7f0cb80cec40 2023-12-19T16:57:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cbd441640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb80cf0d0 con 0x7f0cb806b3d0 2023-12-19T16:57:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0cb8154220 0x7f0cb815aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53664/0 (socket says 172.21.15.26:53664) 2023-12-19T16:57:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb6ffd640 1 -- 172.21.15.26:0/3777773532 learned_addr learned my addr 172.21.15.26:0/3777773532 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0caffff640 1 --2- 172.21.15.26:0/3777773532 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb806b3d0 0x7f0cb80ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313212745 0 0) 0x7f0cb806bc30 con 0x7f0cb8154220 2023-12-19T16:57:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ca4003530 con 0x7f0cb8154220 2023-12-19T16:57:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1038418764 0 0) 0x7f0ca4003530 con 0x7f0cb8154220 2023-12-19T16:57:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb806b3d0 msgr2=0x7f0cb80ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 --2- 172.21.15.26:0/3777773532 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb806b3d0 0x7f0cb80ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 >> v1:172.21.15.160:6789/0 conn(0x7f0cb80cec40 legacy=0x7f0cb815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb806b950 con 0x7f0cb8154220 2023-12-19T16:57:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f0ca8001680 con 0x7f0cb8154220 2023-12-19T16:57:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 >> v1:172.21.15.26:6789/0 conn(0x7f0cb8154220 legacy=0x7f0cb815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 --2- 172.21.15.26:0/3777773532 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ca40039b0 0x7f0ca4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 --2- 172.21.15.26:0/3777773532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca40043a0 0x7f0ca4004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 --2- 172.21.15.26:0/3777773532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ca4005070 0x7f0ca400ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb80cf0d0 con 0x7f0ca40043a0 2023-12-19T16:57:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40104e0 con 0x7f0ca4005070 2023-12-19T16:57:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca400bb90 con 0x7f0ca40039b0 2023-12-19T16:57:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0caffff640 1 --2- 172.21.15.26:0/3777773532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca40043a0 0x7f0ca4004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb6ffd640 1 --2- 172.21.15.26:0/3777773532 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ca40039b0 0x7f0ca4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb77fe640 1 --2- 172.21.15.26:0/3777773532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ca4005070 0x7f0ca400ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0caffff640 1 -- 172.21.15.26:0/3777773532 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ca40039b0 msgr2=0x7f0ca4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0caffff640 1 --2- 172.21.15.26:0/3777773532 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ca40039b0 0x7f0ca4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0caffff640 1 -- 172.21.15.26:0/3777773532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ca4005070 msgr2=0x7f0ca400ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0caffff640 1 --2- 172.21.15.26:0/3777773532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ca4005070 0x7f0ca400ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0caffff640 1 -- 172.21.15.26:0/3777773532 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca4003530 con 0x7f0ca40043a0 2023-12-19T16:57:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.197+0000 7f0cb77fe640 1 --2- 172.21.15.26:0/3777773532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ca4005070 0x7f0ca400ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.201+0000 7f0caffff640 1 --2- 172.21.15.26:0/3777773532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca40043a0 0x7f0ca4004870 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8066670 tx=0x7f0ca001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a6bd5f131b0775 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.201+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00a6430 con 0x7f0ca40043a0 2023-12-19T16:57:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.201+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca00a6590 con 0x7f0ca40043a0 2023-12-19T16:57:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.201+0000 7f0cb67fc640 1 -- 172.21.15.26:0/3777773532 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00a6940 con 0x7f0ca40043a0 2023-12-19T16:57:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.201+0000 7f0cbd441640 1 -- 172.21.15.26:0/3777773532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca40043a0 msgr2=0x7f0ca4004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.201+0000 7f0cbd441640 1 --2- 172.21.15.26:0/3777773532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ca40043a0 0x7f0ca4004870 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8066670 tx=0x7f0ca001c930 comp rx=0 tx=0).stop 2023-12-19T16:57:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.201+0000 7f0cb77fe640 1 -- 172.21.15.26:0/3777773532 reap_dead start 2023-12-19T16:57:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.201+0000 7f0cbd441640 1 -- 172.21.15.26:0/3777773532 shutdown_connections 2023-12-19T16:57:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.201+0000 7f0cbd441640 1 -- 172.21.15.26:0/3777773532 >> 172.21.15.26:0/3777773532 conn(0x7f0cb80658e0 msgr2=0x7f0cb8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.201+0000 7f0cbd441640 1 -- 172.21.15.26:0/3777773532 shutdown_connections 2023-12-19T16:57:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.201+0000 7f0cbd441640 1 -- 172.21.15.26:0/3777773532 wait complete. 2023-12-19T16:57:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cbd441640 1 Processor -- start 2023-12-19T16:57:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cbd441640 1 -- start start 2023-12-19T16:57:29.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cbd441640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ca40043a0 0x7f0cb80b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cbd441640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb806f370 0x7f0cb81ed9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cbd441640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb81f1660 0x7f0cb81f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cbd441640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb80b3670 con 0x7f0cb81f1660 2023-12-19T16:57:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cbd441640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb81f2020 con 0x7f0ca40043a0 2023-12-19T16:57:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cbd441640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb80706f0 con 0x7f0cb806f370 2023-12-19T16:57:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cb77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb81f1660 0x7f0cb81f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cb77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb81f1660 0x7f0cb81f1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52948/0 (socket says 172.21.15.26:52948) 2023-12-19T16:57:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cb77fe640 1 -- 172.21.15.26:0/3349887026 learned_addr learned my addr 172.21.15.26:0/3349887026 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0caffff640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb806f370 0x7f0cb81ed9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cb6ffd640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ca40043a0 0x7f0cb80b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cb77fe640 1 -- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb806f370 msgr2=0x7f0cb81ed9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cb77fe640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb806f370 0x7f0cb81ed9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cb77fe640 1 -- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ca40043a0 msgr2=0x7f0cb80b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cb77fe640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ca40043a0 0x7f0cb80b3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cb77fe640 1 -- 172.21.15.26:0/3349887026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb81edf30 con 0x7f0cb81f1660 2023-12-19T16:57:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cb6ffd640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ca40043a0 0x7f0cb80b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cb77fe640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb81f1660 0x7f0cb81f1ae0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80cc400 tx=0x7f0cb000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0caf7fe640 1 -- 172.21.15.26:0/3349887026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0003b90 con 0x7f0cb81f1660 2023-12-19T16:57:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0caf7fe640 1 -- 172.21.15.26:0/3349887026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb0003cf0 con 0x7f0cb81f1660 2023-12-19T16:57:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0caf7fe640 1 -- 172.21.15.26:0/3349887026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb0005310 con 0x7f0cb81f1660 2023-12-19T16:57:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cbd441640 1 -- 172.21.15.26:0/3349887026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb806fae0 con 0x7f0cb81f1660 2023-12-19T16:57:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.205+0000 7f0cbd441640 1 -- 172.21.15.26:0/3349887026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb80700a0 con 0x7f0cb81f1660 2023-12-19T16:57:29.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.209+0000 7f0cbd441640 1 -- 172.21.15.26:0/3349887026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb8070860 con 0x7f0cb81f1660 2023-12-19T16:57:29.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.209+0000 7f0caf7fe640 1 -- 172.21.15.26:0/3349887026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0cb0005470 con 0x7f0cb81f1660 2023-12-19T16:57:29.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.209+0000 7f0caf7fe640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0c98065b80 0x7f0c98068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:29.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.209+0000 7f0caf7fe640 1 -- 172.21.15.26:0/3349887026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f0cb0083250 con 0x7f0cb81f1660 2023-12-19T16:57:29.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.209+0000 7f0cb6ffd640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0c98065b80 0x7f0c98068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:29.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.209+0000 7f0cb6ffd640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0c98065b80 0x7f0c98068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8000c10 tx=0x7f0ca8001160 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:29.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.213+0000 7f0caf7fe640 1 -- 172.21.15.26:0/3349887026 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0cb0050810 con 0x7f0cb81f1660 2023-12-19T16:57:29.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0cbd441640 1 -- 172.21.15.26:0/3349887026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0cb8074270 con 0x7f0cb81f1660 2023-12-19T16:57:29.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0caf7fe640 1 -- 172.21.15.26:0/3349887026 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0cb0003e60 con 0x7f0cb81f1660 2023-12-19T16:57:29.409 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T16:57:29.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0cbd441640 1 -- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0c98065b80 msgr2=0x7f0c98068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0cbd441640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0c98065b80 0x7f0c98068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8000c10 tx=0x7f0ca8001160 comp rx=0 tx=0).stop 2023-12-19T16:57:29.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0cbd441640 1 -- 172.21.15.26:0/3349887026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb81f1660 msgr2=0x7f0cb81f1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:29.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0cbd441640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb81f1660 0x7f0cb81f1ae0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80cc400 tx=0x7f0cb000ead0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0cbd441640 1 -- 172.21.15.26:0/3349887026 shutdown_connections 2023-12-19T16:57:29.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0cbd441640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb81f1660 0x7f0cb81f1ae0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0cbd441640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0c98065b80 0x7f0c98068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0cbd441640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb806f370 0x7f0cb81ed9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0cbd441640 1 --2- 172.21.15.26:0/3349887026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ca40043a0 0x7f0cb80b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:29.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0cbd441640 1 -- 172.21.15.26:0/3349887026 >> 172.21.15.26:0/3349887026 conn(0x7f0cb80658e0 msgr2=0x7f0cb80cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:29.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0cbd441640 1 -- 172.21.15.26:0/3349887026 shutdown_connections 2023-12-19T16:57:29.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:29.405+0000 7f0cbd441640 1 -- 172.21.15.26:0/3349887026 wait complete. 2023-12-19T16:57:29.431 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-19T16:57:29.431 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T16:57:29.431 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T16:57:29.431 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T16:57:29.431 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T16:57:29.432 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-19T16:57:29.432 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-19T16:57:29.432 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T16:57:29.432 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:57:29.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:29.432 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T16:57:29.442 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T16:57:29.442 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T16:57:29.442 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T16:57:29.442 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 41491sec preferred_lft 41491sec 2023-12-19T16:57:29.443 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:57:29.444 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T16:57:29.445 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T16:57:29.445 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-19T16:57:29.445 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:57:29.445 INFO:teuthology.orchestra.run.smithi026.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T16:57:29.445 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T16:57:29.445 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-19T16:57:29.445 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:57:29.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:29.445 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T16:57:29.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:57:29.503 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T16:57:29.506 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:57:29.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:29.507 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T16:57:29.562 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T16:57:29.562 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T16:57:29.562 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T16:57:29.562 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T16:57:29.562 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T16:57:29.562 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T16:57:29.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:29.563 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:57:29.563 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:57:29.563 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T16:57:29.563 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T16:57:29.563 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T16:57:29.563 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:57:29.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:29.661 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T16:57:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T16:57:29.670 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T16:57:29.671 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:57:29.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:29.671 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T16:57:29.733 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:57:29.733 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T16:57:29.788 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:57:29.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:57:29.789 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T16:57:29.840 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T16:57:29.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:29.840 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:57:29.935 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:57:29.935 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:57:29.948 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:57:29.948 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:57:29.948 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T16:57:29.948 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T16:57:29.948 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T16:57:29.952 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T16:57:29.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:29.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T16:57:29.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:29.974 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:57:30.036 INFO:teuthology.orchestra.run.smithi026.stdout:client6394 2023-12-19T16:57:30.036 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:57:30.036 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:30.089 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-19T16:57:30.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.269+0000 7f2829ef9640 1 Processor -- start 2023-12-19T16:57:30.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.269+0000 7f2829ef9640 1 -- start start 2023-12-19T16:57:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.273+0000 7f2829ef9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28241573f0 0x7f282415ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:30.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.273+0000 7f2829ef9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28240cea80 0x7f28240cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.273+0000 7f2829ef9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282415e110 con 0x7f28241577d0 2023-12-19T16:57:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.273+0000 7f2829ef9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28240cf450 con 0x7f28241573f0 2023-12-19T16:57:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.273+0000 7f2829ef9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282415e3f0 con 0x7f28240cea80 2023-12-19T16:57:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.273+0000 7f28237fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28241573f0 0x7f282415ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.273+0000 7f28237fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28241573f0 0x7f282415ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52952/0 (socket says 172.21.15.26:52952) 2023-12-19T16:57:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.273+0000 7f28237fe640 1 -- 172.21.15.26:0/2485688944 learned_addr learned my addr 172.21.15.26:0/2485688944 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.273+0000 7f2823fff640 1 --2- 172.21.15.26:0/2485688944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28240cea80 0x7f28240cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.273+0000 7f28237fe640 1 -- 172.21.15.26:0/2485688944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28240cea80 msgr2=0x7f28240cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.273+0000 7f28237fe640 1 --2- 172.21.15.26:0/2485688944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28240cea80 0x7f28240cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.273+0000 7f28237fe640 1 -- 172.21.15.26:0/2485688944 >> v1:172.21.15.160:6789/0 conn(0x7f28241577d0 legacy=0x7f282415d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:30.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.273+0000 7f28237fe640 1 -- 172.21.15.26:0/2485688944 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282415a300 con 0x7f28241573f0 2023-12-19T16:57:30.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f28237fe640 1 --2- 172.21.15.26:0/2485688944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28241573f0 0x7f282415ab00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2814009870 tx=0x7f281401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7154e28f327a66d4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:30.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f28227fc640 1 -- 172.21.15.26:0/2485688944 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28140aa650 con 0x7f28241573f0 2023-12-19T16:57:30.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f28227fc640 1 -- 172.21.15.26:0/2485688944 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28140aa7b0 con 0x7f28241573f0 2023-12-19T16:57:30.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f28227fc640 1 -- 172.21.15.26:0/2485688944 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28140aab40 con 0x7f28241573f0 2023-12-19T16:57:30.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f2829ef9640 1 -- 172.21.15.26:0/2485688944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28241573f0 msgr2=0x7f282415ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f2829ef9640 1 --2- 172.21.15.26:0/2485688944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28241573f0 0x7f282415ab00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2814009870 tx=0x7f281401c510 comp rx=0 tx=0).stop 2023-12-19T16:57:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f2829ef9640 1 -- 172.21.15.26:0/2485688944 shutdown_connections 2023-12-19T16:57:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f2829ef9640 1 --2- 172.21.15.26:0/2485688944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28240cea80 0x7f28240cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f2829ef9640 1 --2- 172.21.15.26:0/2485688944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28241573f0 0x7f282415ab00 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f2829ef9640 1 -- 172.21.15.26:0/2485688944 >> 172.21.15.26:0/2485688944 conn(0x7f28240b1aa0 msgr2=0x7f28240b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f2829ef9640 1 -- 172.21.15.26:0/2485688944 shutdown_connections 2023-12-19T16:57:30.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f2829ef9640 1 -- 172.21.15.26:0/2485688944 wait complete. 2023-12-19T16:57:30.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f2829ef9640 1 Processor -- start 2023-12-19T16:57:30.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.277+0000 7f2829ef9640 1 -- start start 2023-12-19T16:57:30.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f2829ef9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28240cea80 0x7f28240c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:30.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f2829ef9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28241573f0 0x7f282413bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:30.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f2829ef9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28241577d0 0x7f282414c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:30.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f2829ef9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282414e920 con 0x7f28240cea80 2023-12-19T16:57:30.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f2829ef9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282413c430 con 0x7f28241577d0 2023-12-19T16:57:30.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f2829ef9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28241523d0 con 0x7f28241573f0 2023-12-19T16:57:30.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f28237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28240cea80 0x7f28240c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:30.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f28237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28240cea80 0x7f28240c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52964/0 (socket says 172.21.15.26:52964) 2023-12-19T16:57:30.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f28237fe640 1 -- 172.21.15.26:0/1829008995 learned_addr learned my addr 172.21.15.26:0/1829008995 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:30.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f2823fff640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28241577d0 0x7f282414c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:30.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f2822ffd640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28241573f0 0x7f282413bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f28237fe640 1 -- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28241573f0 msgr2=0x7f282413bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f28237fe640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28241573f0 0x7f282413bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f28237fe640 1 -- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28241577d0 msgr2=0x7f282414c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f28237fe640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28241577d0 0x7f282414c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f28237fe640 1 -- 172.21.15.26:0/1829008995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28140094d0 con 0x7f28240cea80 2023-12-19T16:57:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f2822ffd640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28241573f0 0x7f282413bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f28237fe640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28240cea80 0x7f28240c9e20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f28140ac040 tx=0x7f281401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f280bfff640 1 -- 172.21.15.26:0/1829008995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28140b3800 con 0x7f28240cea80 2023-12-19T16:57:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f280bfff640 1 -- 172.21.15.26:0/1829008995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28140b3980 con 0x7f28240cea80 2023-12-19T16:57:30.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f280bfff640 1 -- 172.21.15.26:0/1829008995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28140b3d30 con 0x7f28240cea80 2023-12-19T16:57:30.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f2829ef9640 1 -- 172.21.15.26:0/1829008995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282414e260 con 0x7f28240cea80 2023-12-19T16:57:30.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f2829ef9640 1 -- 172.21.15.26:0/1829008995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282414df00 con 0x7f28240cea80 2023-12-19T16:57:30.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f2829ef9640 1 -- 172.21.15.26:0/1829008995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2824074270 con 0x7f28240cea80 2023-12-19T16:57:30.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.281+0000 7f280bfff640 1 -- 172.21.15.26:0/1829008995 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f28140c73a0 con 0x7f28240cea80 2023-12-19T16:57:30.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.285+0000 7f280bfff640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2810065bb0 0x7f2810068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:30.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.285+0000 7f280bfff640 1 -- 172.21.15.26:0/1829008995 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f281411e300 con 0x7f28240cea80 2023-12-19T16:57:30.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.285+0000 7f2822ffd640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2810065bb0 0x7f2810068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:30.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.285+0000 7f2822ffd640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2810065bb0 0x7f2810068070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f282413cc60 tx=0x7f2818009040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:30.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.289+0000 7f280bfff640 1 -- 172.21.15.26:0/1829008995 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f28140eb860 con 0x7f28240cea80 2023-12-19T16:57:30.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.465+0000 7f2829ef9640 1 -- 172.21.15.26:0/1829008995 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f282414e0f0 con 0x7f2810065bb0 2023-12-19T16:57:30.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.469+0000 7f280bfff640 1 -- 172.21.15.26:0/1829008995 <== mgr.5606 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f282414e0f0 con 0x7f2810065bb0 2023-12-19T16:57:30.473 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T16:57:30.473 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:30.473 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2023-12-19T16:57:30.474 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:30.474 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T16:57:30.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.469+0000 7f2829ef9640 1 -- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2810065bb0 msgr2=0x7f2810068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:30.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.469+0000 7f2829ef9640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2810065bb0 0x7f2810068070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f282413cc60 tx=0x7f2818009040 comp rx=0 tx=0).stop 2023-12-19T16:57:30.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.469+0000 7f2829ef9640 1 -- 172.21.15.26:0/1829008995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28240cea80 msgr2=0x7f28240c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.469+0000 7f2829ef9640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28240cea80 0x7f28240c9e20 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f28140ac040 tx=0x7f281401ad50 comp rx=0 tx=0).stop 2023-12-19T16:57:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.469+0000 7f2829ef9640 1 -- 172.21.15.26:0/1829008995 shutdown_connections 2023-12-19T16:57:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.469+0000 7f2829ef9640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28241577d0 0x7f282414c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.469+0000 7f2829ef9640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2810065bb0 0x7f2810068070 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.469+0000 7f2829ef9640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28241573f0 0x7f282413bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.469+0000 7f2829ef9640 1 --2- 172.21.15.26:0/1829008995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28240cea80 0x7f28240c9e20 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.469+0000 7f2829ef9640 1 -- 172.21.15.26:0/1829008995 >> 172.21.15.26:0/1829008995 conn(0x7f28240b1aa0 msgr2=0x7f282415cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.469+0000 7f2829ef9640 1 -- 172.21.15.26:0/1829008995 shutdown_connections 2023-12-19T16:57:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.469+0000 7f2829ef9640 1 -- 172.21.15.26:0/1829008995 wait complete. 2023-12-19T16:57:30.494 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-19T16:57:30.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11c21f3640 1 Processor -- start 2023-12-19T16:57:30.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11c21f3640 1 -- start start 2023-12-19T16:57:30.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11c21f3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f11bc1585f0 0x7f11bc0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:30.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11c21f3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f11bc0cb150 0x7f11bc0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:30.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11c21f3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11bc15b1b0 con 0x7f11bc0cc310 2023-12-19T16:57:30.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11c21f3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11bc0cbb40 con 0x7f11bc0cb150 2023-12-19T16:57:30.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11c21f3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11bc0b47a0 con 0x7f11bc1585f0 2023-12-19T16:57:30.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11baffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f11bc0cb150 0x7f11bc0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:30.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11baffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f11bc0cb150 0x7f11bc0cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52976/0 (socket says 172.21.15.26:52976) 2023-12-19T16:57:30.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11baffd640 1 -- 172.21.15.26:0/3763447921 learned_addr learned my addr 172.21.15.26:0/3763447921 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:30.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11bbfff640 1 --1- 172.21.15.26:0/3763447921 >> v1:172.21.15.26:6789/0 conn(0x7f11bc0cc310 0x7f11bc0cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53668/0 (socket says 172.21.15.26:53668) 2023-12-19T16:57:30.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11bb7fe640 1 --2- 172.21.15.26:0/3763447921 >> v2:172.21.15.160:3300/0 conn(0x7f11bc1585f0 0x7f11bc0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:30.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11baffd640 1 -- 172.21.15.26:0/3763447921 >> v2:172.21.15.160:3300/0 conn(0x7f11bc1585f0 msgr2=0x7f11bc0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:30.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11baffd640 1 --2- 172.21.15.26:0/3763447921 >> v2:172.21.15.160:3300/0 conn(0x7f11bc1585f0 0x7f11bc0cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11baffd640 1 -- 172.21.15.26:0/3763447921 >> v1:172.21.15.26:6789/0 conn(0x7f11bc0cc310 legacy=0x7f11bc0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:30.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.637+0000 7f11baffd640 1 -- 172.21.15.26:0/3763447921 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc15b490 con 0x7f11bc0cb150 2023-12-19T16:57:30.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.641+0000 7f11baffd640 1 --2- 172.21.15.26:0/3763447921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11bc0cb150 0x7f11bc0cb600 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0cc2c0 tx=0x7f11b001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683d4fc3818bcc2e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:30.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.641+0000 7f11ba7fc640 1 -- 172.21.15.26:0/3763447921 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b00ab640 con 0x7f11bc0cb150 2023-12-19T16:57:30.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.641+0000 7f11ba7fc640 1 -- 172.21.15.26:0/3763447921 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b00a7070 con 0x7f11bc0cb150 2023-12-19T16:57:30.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.641+0000 7f11ba7fc640 1 -- 172.21.15.26:0/3763447921 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b00ab9c0 con 0x7f11bc0cb150 2023-12-19T16:57:30.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.641+0000 7f11c21f3640 1 -- 172.21.15.26:0/3763447921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11bc0cb150 msgr2=0x7f11bc0cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:30.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.641+0000 7f11c21f3640 1 --2- 172.21.15.26:0/3763447921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11bc0cb150 0x7f11bc0cb600 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0cc2c0 tx=0x7f11b001c440 comp rx=0 tx=0).stop 2023-12-19T16:57:30.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.641+0000 7f11c21f3640 1 -- 172.21.15.26:0/3763447921 shutdown_connections 2023-12-19T16:57:30.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.641+0000 7f11c21f3640 1 --2- 172.21.15.26:0/3763447921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11bc0cb150 0x7f11bc0cb600 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.641+0000 7f11c21f3640 1 --2- 172.21.15.26:0/3763447921 >> v2:172.21.15.160:3300/0 conn(0x7f11bc1585f0 0x7f11bc0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.641+0000 7f11c21f3640 1 -- 172.21.15.26:0/3763447921 >> 172.21.15.26:0/3763447921 conn(0x7f11bc066160 msgr2=0x7f11bc0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:30.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.641+0000 7f11c21f3640 1 -- 172.21.15.26:0/3763447921 shutdown_connections 2023-12-19T16:57:30.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.641+0000 7f11c21f3640 1 -- 172.21.15.26:0/3763447921 wait complete. 2023-12-19T16:57:30.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11c21f3640 1 Processor -- start 2023-12-19T16:57:30.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11c21f3640 1 -- start start 2023-12-19T16:57:30.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11c21f3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0cb150 0x7f11bc0af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:30.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11c21f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11bc0cc310 0x7f11bc0afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:30.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11c21f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f11bc1585f0 0x7f11bc1f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11c21f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc1fad40 con 0x7f11bc0cc310 2023-12-19T16:57:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11c21f3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0c8e30 con 0x7f11bc0cb150 2023-12-19T16:57:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11c21f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0c9110 con 0x7f11bc1585f0 2023-12-19T16:57:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11baffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11bc0cc310 0x7f11bc0afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11baffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11bc0cc310 0x7f11bc0afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52986/0 (socket says 172.21.15.26:52986) 2023-12-19T16:57:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11baffd640 1 -- 172.21.15.26:0/3531788096 learned_addr learned my addr 172.21.15.26:0/3531788096 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11bb7fe640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0cb150 0x7f11bc0af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11bbfff640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f11bc1585f0 0x7f11bc1f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11baffd640 1 -- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f11bc1585f0 msgr2=0x7f11bc1f9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:30.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11baffd640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f11bc1585f0 0x7f11bc1f9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11baffd640 1 -- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0cb150 msgr2=0x7f11bc0af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:30.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11baffd640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0cb150 0x7f11bc0af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:30.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11baffd640 1 -- 172.21.15.26:0/3531788096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b00094d0 con 0x7f11bc0cc310 2023-12-19T16:57:30.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11bbfff640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f11bc1585f0 0x7f11bc1f9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:30.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11baffd640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11bc0cc310 0x7f11bc0afa40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f11b001a070 tx=0x7f11b001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:30.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11a3fff640 1 -- 172.21.15.26:0/3531788096 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b00be3f0 con 0x7f11bc0cc310 2023-12-19T16:57:30.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11a3fff640 1 -- 172.21.15.26:0/3531788096 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b00be550 con 0x7f11bc0cc310 2023-12-19T16:57:30.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11c21f3640 1 -- 172.21.15.26:0/3531788096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11bc15c3e0 con 0x7f11bc0cc310 2023-12-19T16:57:30.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11a3fff640 1 -- 172.21.15.26:0/3531788096 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b00be900 con 0x7f11bc0cc310 2023-12-19T16:57:30.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.645+0000 7f11c21f3640 1 -- 172.21.15.26:0/3531788096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11bc0c96d0 con 0x7f11bc0cc310 2023-12-19T16:57:30.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.649+0000 7f11c21f3640 1 -- 172.21.15.26:0/3531788096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11bc0c98c0 con 0x7f11bc0cc310 2023-12-19T16:57:30.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.649+0000 7f11a3fff640 1 -- 172.21.15.26:0/3531788096 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f11b00b43f0 con 0x7f11bc0cc310 2023-12-19T16:57:30.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.649+0000 7f11a3fff640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f11a8065b80 0x7f11a8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:30.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.649+0000 7f11a3fff640 1 -- 172.21.15.26:0/3531788096 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f11b011ded0 con 0x7f11bc0cc310 2023-12-19T16:57:30.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.649+0000 7f11bb7fe640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f11a8065b80 0x7f11a8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:30.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.649+0000 7f11bb7fe640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f11a8065b80 0x7f11a8068040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f11bc155440 tx=0x7f11ac008040 comp rx=0 tx=0).ready entity=mgr.5606 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:30.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.653+0000 7f11a3fff640 1 -- 172.21.15.26:0/3531788096 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f11b00eb390 con 0x7f11bc0cc310 2023-12-19T16:57:30.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:30.853+0000 7f11c21f3640 1 -- 172.21.15.26:0/3531788096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f11bc074270 con 0x7f11bc0cc310 2023-12-19T16:57:31.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.541+0000 7f11a3fff640 1 -- 172.21.15.26:0/3531788096 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 31) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f11b011c110 con 0x7f11bc0cc310 2023-12-19T16:57:31.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.541+0000 7f11a3fff640 1 -- 172.21.15.26:0/3531788096 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v31) v1 ==== 92+0+0 (secure 0 0 0) 0x7f11b011ff80 con 0x7f11bc0cc310 2023-12-19T16:57:31.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.545+0000 7f11c21f3640 1 -- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f11a8065b80 msgr2=0x7f11a8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:31.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.545+0000 7f11c21f3640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f11a8065b80 0x7f11a8068040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f11bc155440 tx=0x7f11ac008040 comp rx=0 tx=0).stop 2023-12-19T16:57:31.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.545+0000 7f11c21f3640 1 -- 172.21.15.26:0/3531788096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11bc0cc310 msgr2=0x7f11bc0afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:31.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.545+0000 7f11c21f3640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11bc0cc310 0x7f11bc0afa40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f11b001a070 tx=0x7f11b001ac80 comp rx=0 tx=0).stop 2023-12-19T16:57:31.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.545+0000 7f11c21f3640 1 -- 172.21.15.26:0/3531788096 shutdown_connections 2023-12-19T16:57:31.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.545+0000 7f11c21f3640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f11a8065b80 0x7f11a8068040 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.545+0000 7f11c21f3640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f11bc1585f0 0x7f11bc1f9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.545+0000 7f11c21f3640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11bc0cc310 0x7f11bc0afa40 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.545+0000 7f11c21f3640 1 --2- 172.21.15.26:0/3531788096 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11bc0cb150 0x7f11bc0af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.545+0000 7f11c21f3640 1 -- 172.21.15.26:0/3531788096 >> 172.21.15.26:0/3531788096 conn(0x7f11bc066160 msgr2=0x7f11bc0b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:31.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.545+0000 7f11c21f3640 1 -- 172.21.15.26:0/3531788096 shutdown_connections 2023-12-19T16:57:31.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.545+0000 7f11c21f3640 1 -- 172.21.15.26:0/3531788096 wait complete. 2023-12-19T16:57:31.583 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-19T16:57:31.723 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:31.719+0000 7f6173da6180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:57:31.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c78cc3640 1 Processor -- start 2023-12-19T16:57:31.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c78cc3640 1 -- start start 2023-12-19T16:57:31.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c78cc3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c7415cde0 0x7f7c740b8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:31.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c78cc3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c7415d650 0x7f7c740d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:31.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c78cc3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c740b5bd0 con 0x7f7c7415d1c0 2023-12-19T16:57:31.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c78cc3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c7415e380 con 0x7f7c7415cde0 2023-12-19T16:57:31.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c78cc3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740b8b40 con 0x7f7c7415d650 2023-12-19T16:57:31.746 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f8a4f17e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:57:31.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c7259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c7415cde0 0x7f7c740b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:31.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c7259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c7415cde0 0x7f7c740b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53012/0 (socket says 172.21.15.26:53012) 2023-12-19T16:57:31.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c7259c640 1 -- 172.21.15.26:0/2361823993 learned_addr learned my addr 172.21.15.26:0/2361823993 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:31.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c72d9d640 1 --2- 172.21.15.26:0/2361823993 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c7415d650 0x7f7c740d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:31.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c7259c640 1 -- 172.21.15.26:0/2361823993 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c7415d650 msgr2=0x7f7c740d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:31.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c7259c640 1 --2- 172.21.15.26:0/2361823993 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c7415d650 0x7f7c740d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c7259c640 1 -- 172.21.15.26:0/2361823993 >> v1:172.21.15.26:6789/0 conn(0x7f7c7415d1c0 legacy=0x7f7c740b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:31.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.741+0000 7f7c7259c640 1 -- 172.21.15.26:0/2361823993 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740d0ee0 con 0x7f7c7415cde0 2023-12-19T16:57:31.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.745+0000 7f7c7259c640 1 --2- 172.21.15.26:0/2361823993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c7415cde0 0x7f7c740b8600 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7c640098a0 tx=0x7f7c6401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd65c47bd2f90f4f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:31.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.745+0000 7f7c7159a640 1 -- 172.21.15.26:0/2361823993 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c640a8a10 con 0x7f7c7415cde0 2023-12-19T16:57:31.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.745+0000 7f7c7159a640 1 -- 172.21.15.26:0/2361823993 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c640a8b70 con 0x7f7c7415cde0 2023-12-19T16:57:31.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.745+0000 7f7c7159a640 1 -- 172.21.15.26:0/2361823993 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c640b0440 con 0x7f7c7415cde0 2023-12-19T16:57:31.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.745+0000 7f7c78cc3640 1 -- 172.21.15.26:0/2361823993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c7415cde0 msgr2=0x7f7c740b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:31.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.745+0000 7f7c78cc3640 1 --2- 172.21.15.26:0/2361823993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c7415cde0 0x7f7c740b8600 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7c640098a0 tx=0x7f7c6401ed10 comp rx=0 tx=0).stop 2023-12-19T16:57:31.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.745+0000 7f7c78cc3640 1 -- 172.21.15.26:0/2361823993 shutdown_connections 2023-12-19T16:57:31.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.745+0000 7f7c78cc3640 1 --2- 172.21.15.26:0/2361823993 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c7415d650 0x7f7c740d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.745+0000 7f7c78cc3640 1 --2- 172.21.15.26:0/2361823993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c7415cde0 0x7f7c740b8600 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.745+0000 7f7c78cc3640 1 -- 172.21.15.26:0/2361823993 >> 172.21.15.26:0/2361823993 conn(0x7f7c74065ee0 msgr2=0x7f7c740d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:31.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.745+0000 7f7c78cc3640 1 -- 172.21.15.26:0/2361823993 shutdown_connections 2023-12-19T16:57:31.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.745+0000 7f7c78cc3640 1 -- 172.21.15.26:0/2361823993 wait complete. 2023-12-19T16:57:31.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c78cc3640 1 Processor -- start 2023-12-19T16:57:31.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c78cc3640 1 -- start start 2023-12-19T16:57:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c78cc3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c7415cde0 0x7f7c74069f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c78cc3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c7415d1c0 0x7f7c740cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c78cc3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c740cc590 0x7f7c7415acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c78cc3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741f3750 con 0x7f7c740cc590 2023-12-19T16:57:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c78cc3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741f3960 con 0x7f7c7415d1c0 2023-12-19T16:57:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c78cc3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741f3c40 con 0x7f7c7415cde0 2023-12-19T16:57:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c72d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c740cc590 0x7f7c7415acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:31.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c72d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c740cc590 0x7f7c7415acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53024/0 (socket says 172.21.15.26:53024) 2023-12-19T16:57:31.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c72d9d640 1 -- 172.21.15.26:0/1953109854 learned_addr learned my addr 172.21.15.26:0/1953109854 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:31.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c71d9b640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c7415d1c0 0x7f7c740cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:31.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c7259c640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c7415cde0 0x7f7c74069f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:31.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c7259c640 1 -- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c7415d1c0 msgr2=0x7f7c740cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:31.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c7259c640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c7415d1c0 0x7f7c740cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c7259c640 1 -- 172.21.15.26:0/1953109854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c740cc590 msgr2=0x7f7c7415acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:31.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c7259c640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c740cc590 0x7f7c7415acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c7259c640 1 -- 172.21.15.26:0/1953109854 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c640094d0 con 0x7f7c7415cde0 2023-12-19T16:57:31.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c72d9d640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c740cc590 0x7f7c7415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:31.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c7259c640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c7415cde0 0x7f7c74069f80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c640af000 tx=0x7f7c6401bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:31.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c62ffd640 1 -- 172.21.15.26:0/1953109854 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c640be070 con 0x7f7c7415cde0 2023-12-19T16:57:31.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c62ffd640 1 -- 172.21.15.26:0/1953109854 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c640ba400 con 0x7f7c7415cde0 2023-12-19T16:57:31.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c62ffd640 1 -- 172.21.15.26:0/1953109854 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c640ba7d0 con 0x7f7c7415cde0 2023-12-19T16:57:31.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c78cc3640 1 -- 172.21.15.26:0/1953109854 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c741f3f20 con 0x7f7c7415cde0 2023-12-19T16:57:31.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.749+0000 7f7c78cc3640 1 -- 172.21.15.26:0/1953109854 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c740cbbe0 con 0x7f7c7415cde0 2023-12-19T16:57:31.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.753+0000 7f7c78cc3640 1 -- 172.21.15.26:0/1953109854 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c740cc970 con 0x7f7c7415cde0 2023-12-19T16:57:31.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.753+0000 7f7c62ffd640 1 -- 172.21.15.26:0/1953109854 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 31) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f7c640c3410 con 0x7f7c7415cde0 2023-12-19T16:57:31.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.753+0000 7f7c62ffd640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7c54065bd0 0x7f7c54068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:31.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.753+0000 7f7c62ffd640 1 -- 172.21.15.26:0/1953109854 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f7c64120ff0 con 0x7f7c7415cde0 2023-12-19T16:57:31.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.753+0000 7f7c71d9b640 1 -- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7c54065bd0 msgr2=0x7f7c54068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:31.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.753+0000 7f7c71d9b640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7c54065bd0 0x7f7c54068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:31.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.757+0000 7f7c62ffd640 1 -- 172.21.15.26:0/1953109854 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7c640ee6c0 con 0x7f7c7415cde0 2023-12-19T16:57:31.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.917+0000 7f7c78cc3640 1 -- 172.21.15.26:0/1953109854 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7c74074270 con 0x7f7c7415cde0 2023-12-19T16:57:31.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.925+0000 7f7c62ffd640 1 -- 172.21.15.26:0/1953109854 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7c640f1ce0 con 0x7f7c7415cde0 2023-12-19T16:57:31.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.925+0000 7f7c78cc3640 1 -- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7c54065bd0 msgr2=0x7f7c54068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:31.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.925+0000 7f7c78cc3640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7c54065bd0 0x7f7c54068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.925+0000 7f7c78cc3640 1 -- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c7415cde0 msgr2=0x7f7c74069f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:31.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.925+0000 7f7c78cc3640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c7415cde0 0x7f7c74069f80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c640af000 tx=0x7f7c6401bab0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.925+0000 7f7c78cc3640 1 -- 172.21.15.26:0/1953109854 shutdown_connections 2023-12-19T16:57:31.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.925+0000 7f7c78cc3640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c740cc590 0x7f7c7415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.925+0000 7f7c78cc3640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c7415d1c0 0x7f7c740cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.925+0000 7f7c78cc3640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7c54065bd0 0x7f7c54068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.925+0000 7f7c78cc3640 1 --2- 172.21.15.26:0/1953109854 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7c7415cde0 0x7f7c74069f80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:31.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.925+0000 7f7c78cc3640 1 -- 172.21.15.26:0/1953109854 >> 172.21.15.26:0/1953109854 conn(0x7f7c74065ee0 msgr2=0x7f7c740d2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:31.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.925+0000 7f7c78cc3640 1 -- 172.21.15.26:0/1953109854 shutdown_connections 2023-12-19T16:57:31.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:31.925+0000 7f7c78cc3640 1 -- 172.21.15.26:0/1953109854 wait complete. 2023-12-19T16:57:31.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-19T16:57:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.093+0000 7f0edc266640 1 Processor -- start 2023-12-19T16:57:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0edc266640 1 -- start start 2023-12-19T16:57:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0edc266640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0ed4159820 0x7f0ed415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0edc266640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ed406f6c0 0x7f0ed40c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0edc266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ed4159c00 0x7f0ed4070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0edc266640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed415e470 con 0x7f0ed406f6c0 2023-12-19T16:57:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0edc266640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed406fb60 con 0x7f0ed4159820 2023-12-19T16:57:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0edc266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed406fe40 con 0x7f0ed4159c00 2023-12-19T16:57:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0ed9801640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ed406f6c0 0x7f0ed40c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0eda803640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ed4159c00 0x7f0ed4070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0ed9801640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ed406f6c0 0x7f0ed40c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53032/0 (socket says 172.21.15.26:53032) 2023-12-19T16:57:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0ed9801640 1 -- 172.21.15.26:0/2006188658 learned_addr learned my addr 172.21.15.26:0/2006188658 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0eda002640 1 --2- 172.21.15.26:0/2006188658 >> v2:172.21.15.160:3300/0 conn(0x7f0ed4159820 0x7f0ed415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0ed9801640 1 -- 172.21.15.26:0/2006188658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ed4159c00 msgr2=0x7f0ed4070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0ed9801640 1 --2- 172.21.15.26:0/2006188658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ed4159c00 0x7f0ed4070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0ed9801640 1 -- 172.21.15.26:0/2006188658 >> v2:172.21.15.160:3300/0 conn(0x7f0ed4159820 msgr2=0x7f0ed415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0ed9801640 1 --2- 172.21.15.26:0/2006188658 >> v2:172.21.15.160:3300/0 conn(0x7f0ed4159820 0x7f0ed415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.097+0000 7f0ed9801640 1 -- 172.21.15.26:0/2006188658 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed4070120 con 0x7f0ed406f6c0 2023-12-19T16:57:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.101+0000 7f0ed9801640 1 --2- 172.21.15.26:0/2006188658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed406f6c0 0x7f0ed40c8e80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40c8e10 tx=0x7f0ec401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720afc8f166beb8f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.101+0000 7f0ed9000640 1 -- 172.21.15.26:0/2006188658 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40aa5f0 con 0x7f0ed406f6c0 2023-12-19T16:57:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.101+0000 7f0ed9000640 1 -- 172.21.15.26:0/2006188658 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec40aa750 con 0x7f0ed406f6c0 2023-12-19T16:57:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.101+0000 7f0ed9000640 1 -- 172.21.15.26:0/2006188658 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40aaae0 con 0x7f0ed406f6c0 2023-12-19T16:57:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.101+0000 7f0edc266640 1 -- 172.21.15.26:0/2006188658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed406f6c0 msgr2=0x7f0ed40c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.101+0000 7f0edc266640 1 --2- 172.21.15.26:0/2006188658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed406f6c0 0x7f0ed40c8e80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40c8e10 tx=0x7f0ec401c4d0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.101+0000 7f0edc266640 1 -- 172.21.15.26:0/2006188658 shutdown_connections 2023-12-19T16:57:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.101+0000 7f0edc266640 1 --2- 172.21.15.26:0/2006188658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ed4159c00 0x7f0ed4070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.101+0000 7f0edc266640 1 --2- 172.21.15.26:0/2006188658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed406f6c0 0x7f0ed40c8e80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.101+0000 7f0edc266640 1 --2- 172.21.15.26:0/2006188658 >> v2:172.21.15.160:3300/0 conn(0x7f0ed4159820 0x7f0ed415df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.101+0000 7f0edc266640 1 -- 172.21.15.26:0/2006188658 >> 172.21.15.26:0/2006188658 conn(0x7f0ed40ce730 msgr2=0x7f0ed40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.101+0000 7f0edc266640 1 -- 172.21.15.26:0/2006188658 shutdown_connections 2023-12-19T16:57:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.101+0000 7f0edc266640 1 -- 172.21.15.26:0/2006188658 wait complete. 2023-12-19T16:57:32.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0edc266640 1 Processor -- start 2023-12-19T16:57:32.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0edc266640 1 -- start start 2023-12-19T16:57:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0edc266640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed406f6c0 0x7f0ed415c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0edc266640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4159820 0x7f0ed40c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0edc266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ed4159c00 0x7f0ed4142e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0edc266640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed4143b90 con 0x7f0ed406f6c0 2023-12-19T16:57:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0edc266640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed415ce80 con 0x7f0ed4159820 2023-12-19T16:57:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0edc266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40c3b30 con 0x7f0ed4159c00 2023-12-19T16:57:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0eda002640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed406f6c0 0x7f0ed415c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0eda002640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed406f6c0 0x7f0ed415c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53036/0 (socket says 172.21.15.26:53036) 2023-12-19T16:57:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0eda002640 1 -- 172.21.15.26:0/1623622421 learned_addr learned my addr 172.21.15.26:0/1623622421 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0ed9801640 1 --2- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4159820 0x7f0ed40c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0eda803640 1 --2- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ed4159c00 0x7f0ed4142e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0eda002640 1 -- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ed4159c00 msgr2=0x7f0ed4142e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0eda002640 1 --2- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ed4159c00 0x7f0ed4142e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0eda002640 1 -- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4159820 msgr2=0x7f0ed40c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0eda002640 1 --2- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4159820 0x7f0ed40c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0eda002640 1 -- 172.21.15.26:0/1623622421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec40094d0 con 0x7f0ed406f6c0 2023-12-19T16:57:32.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0eda002640 1 --2- 172.21.15.26:0/1623622421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed406f6c0 0x7f0ed415c940 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000e960 tx=0x7f0ed000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:32.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0eca7fc640 1 -- 172.21.15.26:0/1623622421 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0009690 con 0x7f0ed406f6c0 2023-12-19T16:57:32.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0eca7fc640 1 -- 172.21.15.26:0/1623622421 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed000ca00 con 0x7f0ed406f6c0 2023-12-19T16:57:32.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0edc266640 1 -- 172.21.15.26:0/1623622421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed406b880 con 0x7f0ed406f6c0 2023-12-19T16:57:32.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0eca7fc640 1 -- 172.21.15.26:0/1623622421 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed000cd70 con 0x7f0ed406f6c0 2023-12-19T16:57:32.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0edc266640 1 -- 172.21.15.26:0/1623622421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed4143960 con 0x7f0ed406f6c0 2023-12-19T16:57:32.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.105+0000 7f0edc266640 1 -- 172.21.15.26:0/1623622421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed4074270 con 0x7f0ed406f6c0 2023-12-19T16:57:32.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.109+0000 7f0eca7fc640 1 -- 172.21.15.26:0/1623622421 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 31) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f0ed0013470 con 0x7f0ed406f6c0 2023-12-19T16:57:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.109+0000 7f0eca7fc640 1 --2- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0eb8065bd0 0x7f0eb8068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.109+0000 7f0eca7fc640 1 -- 172.21.15.26:0/1623622421 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f0ed00837f0 con 0x7f0ed406f6c0 2023-12-19T16:57:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.109+0000 7f0ed9801640 1 -- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0eb8065bd0 msgr2=0x7f0eb8068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.109+0000 7f0ed9801640 1 --2- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0eb8065bd0 0x7f0eb8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:32.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.113+0000 7f0eca7fc640 1 -- 172.21.15.26:0/1623622421 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0ed0050e70 con 0x7f0ed406f6c0 2023-12-19T16:57:32.144 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:32.139+0000 7f6173da6180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:57:32.196 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:32.193+0000 7f8a4f17e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:57:32.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.273+0000 7f0edc266640 1 -- 172.21.15.26:0/1623622421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f0ed406b6d0 con 0x7f0ed406f6c0 2023-12-19T16:57:32.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.281+0000 7f0eca7fc640 1 -- 172.21.15.26:0/1623622421 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0ed0081e60 con 0x7f0ed406f6c0 2023-12-19T16:57:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.281+0000 7f0edc266640 1 -- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0eb8065bd0 msgr2=0x7f0eb8068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.281+0000 7f0edc266640 1 --2- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0eb8065bd0 0x7f0eb8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.281+0000 7f0edc266640 1 -- 172.21.15.26:0/1623622421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed406f6c0 msgr2=0x7f0ed415c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.281+0000 7f0edc266640 1 --2- 172.21.15.26:0/1623622421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed406f6c0 0x7f0ed415c940 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000e960 tx=0x7f0ed000ee20 comp rx=0 tx=0).stop 2023-12-19T16:57:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.281+0000 7f0edc266640 1 -- 172.21.15.26:0/1623622421 shutdown_connections 2023-12-19T16:57:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.281+0000 7f0edc266640 1 --2- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0eb8065bd0 0x7f0eb8068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.281+0000 7f0edc266640 1 --2- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ed4159c00 0x7f0ed4142e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.281+0000 7f0edc266640 1 --2- 172.21.15.26:0/1623622421 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ed4159820 0x7f0ed40c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.281+0000 7f0edc266640 1 --2- 172.21.15.26:0/1623622421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ed406f6c0 0x7f0ed415c940 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.281+0000 7f0edc266640 1 -- 172.21.15.26:0/1623622421 >> 172.21.15.26:0/1623622421 conn(0x7f0ed40ce730 msgr2=0x7f0ed40c9880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:32.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.281+0000 7f0edc266640 1 -- 172.21.15.26:0/1623622421 shutdown_connections 2023-12-19T16:57:32.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.281+0000 7f0edc266640 1 -- 172.21.15.26:0/1623622421 wait complete. 2023-12-19T16:57:32.297 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:32.295+0000 7f6173da6180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:57:32.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:32.307 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-19T16:57:32.323 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-12-19T16:57:32.358 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:32.353+0000 7f8a4f17e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:57:32.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.497+0000 7f76b1bbc640 1 Processor -- start 2023-12-19T16:57:32.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.497+0000 7f76b1bbc640 1 -- start start 2023-12-19T16:57:32.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76b1bbc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f76ac15cc70 0x7f76ac0d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:32.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76b1bbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76ac0b88b0 0x7f76ac0b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:32.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76b1bbc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ac0b71f0 con 0x7f76ac0b6140 2023-12-19T16:57:32.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76b1bbc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76ac0b65e0 con 0x7f76ac15cc70 2023-12-19T16:57:32.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76b1bbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0b68c0 con 0x7f76ac0b88b0 2023-12-19T16:57:32.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76aaffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76ac0b88b0 0x7f76ac0b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:32.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76ab7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f76ac15cc70 0x7f76ac0d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:32.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76aaffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76ac0b88b0 0x7f76ac0b5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:38156/0 (socket says 172.21.15.26:38156) 2023-12-19T16:57:32.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76aaffd640 1 -- 172.21.15.26:0/2084274367 learned_addr learned my addr 172.21.15.26:0/2084274367 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:32.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76ab7fe640 1 -- 172.21.15.26:0/2084274367 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76ac0b88b0 msgr2=0x7f76ac0b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:32.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76ab7fe640 1 --2- 172.21.15.26:0/2084274367 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76ac0b88b0 0x7f76ac0b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76ab7fe640 1 -- 172.21.15.26:0/2084274367 >> v1:172.21.15.160:6789/0 conn(0x7f76ac0b6140 legacy=0x7f76ac0cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:32.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76ab7fe640 1 -- 172.21.15.26:0/2084274367 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac0b6ba0 con 0x7f76ac15cc70 2023-12-19T16:57:32.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76aa7fc640 1 -- 172.21.15.26:0/2084274367 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149070153 0 0) 0x7f76ac0b71f0 con 0x7f76ac0b6140 2023-12-19T16:57:32.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.501+0000 7f76aaffd640 1 --2- 172.21.15.26:0/2084274367 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76ac0b88b0 0x7f76ac0b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:32.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76ab7fe640 1 --2- 172.21.15.26:0/2084274367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76ac15cc70 0x7f76ac0d0c60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f769c0097c0 tx=0x7f769c01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a21da52308155899 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76aa7fc640 1 -- 172.21.15.26:0/2084274367 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c0a8970 con 0x7f76ac15cc70 2023-12-19T16:57:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76aa7fc640 1 -- 172.21.15.26:0/2084274367 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c0a8ad0 con 0x7f76ac15cc70 2023-12-19T16:57:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76aa7fc640 1 -- 172.21.15.26:0/2084274367 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c0a8e60 con 0x7f76ac15cc70 2023-12-19T16:57:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76b1bbc640 1 -- 172.21.15.26:0/2084274367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76ac15cc70 msgr2=0x7f76ac0d0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76b1bbc640 1 --2- 172.21.15.26:0/2084274367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76ac15cc70 0x7f76ac0d0c60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f769c0097c0 tx=0x7f769c01ec70 comp rx=0 tx=0).stop 2023-12-19T16:57:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76b1bbc640 1 -- 172.21.15.26:0/2084274367 shutdown_connections 2023-12-19T16:57:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76b1bbc640 1 --2- 172.21.15.26:0/2084274367 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76ac0b88b0 0x7f76ac0b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76b1bbc640 1 --2- 172.21.15.26:0/2084274367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76ac15cc70 0x7f76ac0d0c60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76b1bbc640 1 -- 172.21.15.26:0/2084274367 >> 172.21.15.26:0/2084274367 conn(0x7f76ac06a120 msgr2=0x7f76ac06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:32.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76b1bbc640 1 -- 172.21.15.26:0/2084274367 shutdown_connections 2023-12-19T16:57:32.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76b1bbc640 1 -- 172.21.15.26:0/2084274367 wait complete. 2023-12-19T16:57:32.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76b1bbc640 1 Processor -- start 2023-12-19T16:57:32.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.505+0000 7f76b1bbc640 1 -- start start 2023-12-19T16:57:32.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76b1bbc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76ac0b6140 0x7f76ac0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:32.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76b1bbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76ac0b88b0 0x7f76ac0b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:32.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76b1bbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76ac15cc70 0x7f76ac0c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:32.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76b1bbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0cba10 con 0x7f76ac15cc70 2023-12-19T16:57:32.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76b1bbc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0cbcf0 con 0x7f76ac0b6140 2023-12-19T16:57:32.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76b1bbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac0cad90 con 0x7f76ac0b88b0 2023-12-19T16:57:32.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76ac15cc70 0x7f76ac0c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:32.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76ac15cc70 0x7f76ac0c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53048/0 (socket says 172.21.15.26:53048) 2023-12-19T16:57:32.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76abfff640 1 -- 172.21.15.26:0/2905691341 learned_addr learned my addr 172.21.15.26:0/2905691341 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:32.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76ab7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76ac0b6140 0x7f76ac0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:32.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76aaffd640 1 --2- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76ac0b88b0 0x7f76ac0b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:32.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76ab7fe640 1 -- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76ac0b88b0 msgr2=0x7f76ac0b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:32.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76ab7fe640 1 --2- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f76ac0b88b0 0x7f76ac0b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76ab7fe640 1 -- 172.21.15.26:0/2905691341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76ac15cc70 msgr2=0x7f76ac0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:32.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76ab7fe640 1 --2- 172.21.15.26:0/2905691341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76ac15cc70 0x7f76ac0c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:32.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76ab7fe640 1 -- 172.21.15.26:0/2905691341 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00094d0 con 0x7f76ac0b6140 2023-12-19T16:57:32.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76ab7fe640 1 --2- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76ac0b6140 0x7f76ac0b26a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f769c0af000 tx=0x7f769c01ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:32.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f7693fff640 1 -- 172.21.15.26:0/2905691341 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c0b0dd0 con 0x7f76ac0b6140 2023-12-19T16:57:32.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f7693fff640 1 -- 172.21.15.26:0/2905691341 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c0b9430 con 0x7f76ac0b6140 2023-12-19T16:57:32.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f7693fff640 1 -- 172.21.15.26:0/2905691341 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c0b97e0 con 0x7f76ac0b6140 2023-12-19T16:57:32.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76b1bbc640 1 -- 172.21.15.26:0/2905691341 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c009420 con 0x7f76ac0b6140 2023-12-19T16:57:32.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76b1bbc640 1 -- 172.21.15.26:0/2905691341 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ac0cb350 con 0x7f76ac0b6140 2023-12-19T16:57:32.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.509+0000 7f76b1bbc640 1 -- 172.21.15.26:0/2905691341 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ac074270 con 0x7f76ac0b6140 2023-12-19T16:57:32.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.513+0000 7f7693fff640 1 -- 172.21.15.26:0/2905691341 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 31) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f769c0b7040 con 0x7f76ac0b6140 2023-12-19T16:57:32.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.513+0000 7f7693fff640 1 --2- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 0x7f7698068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:32.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.513+0000 7f7693fff640 1 -- 172.21.15.26:0/2905691341 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f769c120ed0 con 0x7f76ac0b6140 2023-12-19T16:57:32.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.513+0000 7f76aaffd640 1 -- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 msgr2=0x7f7698068110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:32.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.513+0000 7f76aaffd640 1 --2- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 0x7f7698068110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:32.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.517+0000 7f7693fff640 1 -- 172.21.15.26:0/2905691341 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f769c0ee530 con 0x7f76ac0b6140 2023-12-19T16:57:32.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.693+0000 7f76b1bbc640 1 -- 172.21.15.26:0/2905691341 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f76ac15d050 con 0x7f7698065c50 2023-12-19T16:57:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.717+0000 7f76aaffd640 1 -- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 msgr2=0x7f7698068110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:32.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:32.717+0000 7f76aaffd640 1 --2- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 0x7f7698068110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T16:57:32.786 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:32.783+0000 7f6173da6180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:57:32.846 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:32.843+0000 7f6173da6180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:57:32.876 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:32.869+0000 7f8a4f17e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:57:32.902 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:32.899+0000 7f6173da6180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:57:32.938 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:32.933+0000 7f8a4f17e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:57:32.987 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:32.983+0000 7f6173da6180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:57:32.996 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:32.993+0000 7f8a4f17e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:57:33.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:33.043+0000 7f6173da6180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:57:33.089 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:33.085+0000 7f8a4f17e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:57:33.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:33.121+0000 7f76aaffd640 1 -- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 msgr2=0x7f7698068110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:33.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:33.121+0000 7f76aaffd640 1 --2- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 0x7f7698068110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-19T16:57:33.130 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:33.127+0000 7f6173da6180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:57:33.148 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:33.145+0000 7f8a4f17e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:57:33.186 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:33.183+0000 7f6173da6180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:57:33.237 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:33.233+0000 7f8a4f17e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:57:33.296 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:33.293+0000 7f8a4f17e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:57:33.328 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:33.323+0000 7f6173da6180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:57:33.386 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:33.383+0000 7f6173da6180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:57:33.447 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:33.441+0000 7f8a4f17e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:57:33.509 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:33.505+0000 7f8a4f17e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:57:33.626 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:57:33.627 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:57:33.627 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:57:33.634 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:33.631+0000 7f6173da6180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:57:33.760 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:57:33.760 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:57:33.760 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:57:33.768 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:33.765+0000 7f8a4f17e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:57:33.770 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:33.767+0000 7f6173da6180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:57:33.896 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:33.891+0000 7f6173da6180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:57:33.912 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:33.909+0000 7f8a4f17e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:57:33.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:33.925+0000 7f76aaffd640 1 -- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 msgr2=0x7f7698068110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:33.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:33.925+0000 7f76aaffd640 1 --2- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 0x7f7698068110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-19T16:57:33.967 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:33.963+0000 7f6173da6180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:57:34.043 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:34.037+0000 7f8a4f17e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:57:34.102 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:34.099+0000 7f6173da6180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:57:34.118 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:34.113+0000 7f8a4f17e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:57:34.262 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:34.257+0000 7f8a4f17e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:57:34.641 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:34.639+0000 7f6173da6180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:57:34.726 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:34.723+0000 7f6173da6180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:57:34.832 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:34.825+0000 7f8a4f17e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:57:34.875 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:34.871+0000 7f6173da6180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:57:34.920 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:34.917+0000 7f8a4f17e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:57:34.959 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:34.955+0000 7f6173da6180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:57:35.077 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:35.073+0000 7f8a4f17e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:57:35.165 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:35.161+0000 7f8a4f17e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:57:35.277 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:35.271+0000 7f6173da6180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:57:35.337 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:35.331+0000 7f6173da6180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:57:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:35.345+0000 7f7693fff640 1 -- 172.21.15.26:0/2905691341 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mgrmap(e 32) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f769c0ee900 con 0x7f76ac0b6140 2023-12-19T16:57:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:35.345+0000 7f7693fff640 1 -- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 msgr2=0x7f7698068110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:35.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:35.345+0000 7f7693fff640 1 --2- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 0x7f7698068110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:35.472 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:35.467+0000 7f60f6d1a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:35.472 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:35.467+0000 7f60f6d1a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:35.472 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:35.467+0000 7f60f6d1a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:35.472 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:35.467+0000 7f60f6d1a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:35.472 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:35.467+0000 7f60f6d1a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:35.473 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:35.467+0000 7f60f8d1e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:35.473 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:35.467+0000 7f60f8d1e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:35.473 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:35.467+0000 7f60f8d1e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:35.473 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:35.467+0000 7f60f8d1e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:35.473 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:35.467+0000 7f60f8d1e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:35.511 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:35.505+0000 7f8a4f17e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:57:35.575 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:35.569+0000 7f8a4f17e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:57:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.349+0000 7f7693fff640 1 -- 172.21.15.26:0/2905691341 <== mon.1 v2:172.21.15.160:3300/0 8 ==== mgrmap(e 33) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f769c0b2030 con 0x7f76ac0b6140 2023-12-19T16:57:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.349+0000 7f7693fff640 1 --2- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 0x7f7698068110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:36.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.349+0000 7f7693fff640 1 -- 172.21.15.26:0/2905691341 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f769806bc50 con 0x7f7698065c50 2023-12-19T16:57:36.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.353+0000 7f76aaffd640 1 --2- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 0x7f7698068110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:36.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.357+0000 7f76aaffd640 1 --2- 172.21.15.26:0/2905691341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7698065c50 0x7f7698068110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f76ac1538b0 tx=0x7f76a0009240 comp rx=0 tx=0).ready entity=mgr.5873 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:36.371 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:36.367+0000 7f6100d2e640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-12-19T16:57:36.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.365+0000 7f7693fff640 1 -- 172.21.15.26:0/2905691341 <== mgr.5873 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f769806bc50 con 0x7f7698065c50 2023-12-19T16:57:36.372 INFO:teuthology.orchestra.run.smithi026.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-12-19T16:57:36.381 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-12-19T16:57:36.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:36.382 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-19T16:57:36.395 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-19T16:57:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8087266640 1 Processor -- start 2023-12-19T16:57:36.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8087266640 1 -- start start 2023-12-19T16:57:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8087266640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80800adc60 0x7f80800ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8087266640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8080071100 0x7f808006cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8087266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f808006d010 0x7f8080164ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8087266640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8080071590 con 0x7f80800adc60 2023-12-19T16:57:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8087266640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80801615b0 con 0x7f8080071100 2023-12-19T16:57:36.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8087266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80800c8d90 con 0x7f808006d010 2023-12-19T16:57:36.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8085002640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80800adc60 0x7f80800ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:36.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8085002640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80800adc60 0x7f80800ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44722/0 (socket says 172.21.15.26:44722) 2023-12-19T16:57:36.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8085002640 1 -- 172.21.15.26:0/2302237117 learned_addr learned my addr 172.21.15.26:0/2302237117 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8084801640 1 --2- 172.21.15.26:0/2302237117 >> v2:172.21.15.160:3300/0 conn(0x7f8080071100 0x7f808006cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8085803640 1 --2- 172.21.15.26:0/2302237117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f808006d010 0x7f8080164ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8085002640 1 -- 172.21.15.26:0/2302237117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f808006d010 msgr2=0x7f8080164ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8085002640 1 --2- 172.21.15.26:0/2302237117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f808006d010 0x7f8080164ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8085002640 1 -- 172.21.15.26:0/2302237117 >> v2:172.21.15.160:3300/0 conn(0x7f8080071100 msgr2=0x7f808006cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8085002640 1 --2- 172.21.15.26:0/2302237117 >> v2:172.21.15.160:3300/0 conn(0x7f8080071100 0x7f808006cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:36.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.585+0000 7f8085002640 1 -- 172.21.15.26:0/2302237117 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800c9070 con 0x7f80800adc60 2023-12-19T16:57:36.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.589+0000 7f8085002640 1 --2- 172.21.15.26:0/2302237117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80800adc60 0x7f80800ca640 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f80800ca150 tx=0x7f807c01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553e9957ac94cd5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:36.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.589+0000 7f8077fff640 1 -- 172.21.15.26:0/2302237117 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0ab600 con 0x7f80800adc60 2023-12-19T16:57:36.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.589+0000 7f8077fff640 1 -- 172.21.15.26:0/2302237117 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c0a7050 con 0x7f80800adc60 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.589+0000 7f8077fff640 1 -- 172.21.15.26:0/2302237117 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0ab980 con 0x7f80800adc60 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.589+0000 7f8087266640 1 -- 172.21.15.26:0/2302237117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80800adc60 msgr2=0x7f80800ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.589+0000 7f8087266640 1 --2- 172.21.15.26:0/2302237117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80800adc60 0x7f80800ca640 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f80800ca150 tx=0x7f807c01c210 comp rx=0 tx=0).stop 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.589+0000 7f8087266640 1 -- 172.21.15.26:0/2302237117 shutdown_connections 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.589+0000 7f8087266640 1 --2- 172.21.15.26:0/2302237117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f808006d010 0x7f8080164ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.589+0000 7f8087266640 1 --2- 172.21.15.26:0/2302237117 >> v2:172.21.15.160:3300/0 conn(0x7f8080071100 0x7f808006cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.589+0000 7f8087266640 1 --2- 172.21.15.26:0/2302237117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80800adc60 0x7f80800ca640 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.589+0000 7f8087266640 1 -- 172.21.15.26:0/2302237117 >> 172.21.15.26:0/2302237117 conn(0x7f8080066160 msgr2=0x7f80800b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.589+0000 7f8087266640 1 -- 172.21.15.26:0/2302237117 shutdown_connections 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.589+0000 7f8087266640 1 -- 172.21.15.26:0/2302237117 wait complete. 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8087266640 1 Processor -- start 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8087266640 1 -- start start 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8087266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f808006d010 0x7f8080160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:36.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8087266640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080071100 0x7f808015f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8087266640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80800adc60 0x7f808014ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8087266640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8080152170 con 0x7f8080071100 2023-12-19T16:57:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8087266640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80801606b0 con 0x7f80800adc60 2023-12-19T16:57:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8087266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808015eab0 con 0x7f808006d010 2023-12-19T16:57:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8084801640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080071100 0x7f808015f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8084801640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080071100 0x7f808015f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44728/0 (socket says 172.21.15.26:44728) 2023-12-19T16:57:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8084801640 1 -- 172.21.15.26:0/105095201 learned_addr learned my addr 172.21.15.26:0/105095201 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8085002640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f808006d010 0x7f8080160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8085803640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80800adc60 0x7f808014ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8084801640 1 -- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f808006d010 msgr2=0x7f8080160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8084801640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f808006d010 0x7f8080160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8084801640 1 -- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80800adc60 msgr2=0x7f808014ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:36.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8084801640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80800adc60 0x7f808014ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:36.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8084801640 1 -- 172.21.15.26:0/105095201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c0094d0 con 0x7f8080071100 2023-12-19T16:57:36.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8084801640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080071100 0x7f808015f500 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f807000d830 tx=0x7f807000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:36.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8075ffb640 1 -- 172.21.15.26:0/105095201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807000b6c0 con 0x7f8080071100 2023-12-19T16:57:36.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8075ffb640 1 -- 172.21.15.26:0/105095201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807000b820 con 0x7f8080071100 2023-12-19T16:57:36.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8075ffb640 1 -- 172.21.15.26:0/105095201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807000bb90 con 0x7f8080071100 2023-12-19T16:57:36.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8087266640 1 -- 172.21.15.26:0/105095201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8080151650 con 0x7f8080071100 2023-12-19T16:57:36.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.593+0000 7f8087266640 1 -- 172.21.15.26:0/105095201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808015ed00 con 0x7f8080071100 2023-12-19T16:57:36.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.597+0000 7f8087266640 1 -- 172.21.15.26:0/105095201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8080151b30 con 0x7f8080071100 2023-12-19T16:57:36.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.597+0000 7f8075ffb640 1 -- 172.21.15.26:0/105095201 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 33) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f807001e460 con 0x7f8080071100 2023-12-19T16:57:36.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.597+0000 7f8075ffb640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8064065900 0x7f8064067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:36.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.597+0000 7f8075ffb640 1 -- 172.21.15.26:0/105095201 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f8070083830 con 0x7f8080071100 2023-12-19T16:57:36.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.597+0000 7f8085002640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8064065900 0x7f8064067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:36.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.597+0000 7f8085002640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8064065900 0x7f8064067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f807c0ad000 tx=0x7f807c008040 comp rx=0 tx=0).ready entity=mgr.5873 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:36.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.601+0000 7f8075ffb640 1 -- 172.21.15.26:0/105095201 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f80700511a0 con 0x7f8080071100 2023-12-19T16:57:36.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:36.797+0000 7f8087266640 1 -- 172.21.15.26:0/105095201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f8080074270 con 0x7f8080071100 2023-12-19T16:57:37.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.401+0000 7f8075ffb640 1 -- 172.21.15.26:0/105095201 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f8070051570 con 0x7f8080071100 2023-12-19T16:57:37.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.405+0000 7f8075ffb640 1 -- 172.21.15.26:0/105095201 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v34) v1 ==== 93+0+0 (secure 0 0 0) 0x7f807007f250 con 0x7f8080071100 2023-12-19T16:57:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.405+0000 7f8087266640 1 -- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8064065900 msgr2=0x7f8064067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.405+0000 7f8087266640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8064065900 0x7f8064067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f807c0ad000 tx=0x7f807c008040 comp rx=0 tx=0).stop 2023-12-19T16:57:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.405+0000 7f8087266640 1 -- 172.21.15.26:0/105095201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080071100 msgr2=0x7f808015f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.405+0000 7f8087266640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080071100 0x7f808015f500 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f807000d830 tx=0x7f807000dcf0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.405+0000 7f8087266640 1 -- 172.21.15.26:0/105095201 shutdown_connections 2023-12-19T16:57:37.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.405+0000 7f8087266640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80800adc60 0x7f808014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.405+0000 7f8087266640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8080071100 0x7f808015f500 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.405+0000 7f8087266640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8064065900 0x7f8064067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.405+0000 7f8087266640 1 --2- 172.21.15.26:0/105095201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f808006d010 0x7f8080160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.405+0000 7f8087266640 1 -- 172.21.15.26:0/105095201 >> 172.21.15.26:0/105095201 conn(0x7f8080066160 msgr2=0x7f808015a570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:37.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.405+0000 7f8087266640 1 -- 172.21.15.26:0/105095201 shutdown_connections 2023-12-19T16:57:37.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.409+0000 7f8087266640 1 -- 172.21.15.26:0/105095201 wait complete. 2023-12-19T16:57:37.436 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T16:57:37.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:57:37.492 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:57:37.579 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T16:57:37.580 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:57:37.580 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:37.589 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:57:37.589 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:37.590 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:37.587+0000 7f79d6506180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:57:37.590 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:37.585+0000 7ff967a0a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:57:37.599 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:37.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.781+0000 7f1ed6e66640 1 Processor -- start 2023-12-19T16:57:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.781+0000 7f1ed6e66640 1 -- start start 2023-12-19T16:57:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.781+0000 7f1ed6e66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1ed00cc9b0 0x7f1ed00cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:37.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.781+0000 7f1ed6e66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed00afd70 0x7f1ed00cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.781+0000 7f1ed6e66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed006b730 con 0x7f1ed00ce520 2023-12-19T16:57:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.781+0000 7f1ed6e66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed01617d0 con 0x7f1ed00cc9b0 2023-12-19T16:57:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.781+0000 7f1ed6e66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed00b3020 con 0x7f1ed00afd70 2023-12-19T16:57:37.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.785+0000 7f1ecffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed00afd70 0x7f1ed00cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.785+0000 7f1ed5403640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1ed00ce520 0x7f1ed0069d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:37990/0 (socket says 172.21.15.26:37990) 2023-12-19T16:57:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.785+0000 7f1ed4c02640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1ed00cc9b0 0x7f1ed00cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.785+0000 7f1ed5403640 1 -- 172.21.15.26:0/167035240 learned_addr learned my addr 172.21.15.26:0/167035240 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.785+0000 7f1ed4c02640 1 -- 172.21.15.26:0/167035240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed00afd70 msgr2=0x7f1ed00cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.785+0000 7f1ecf7fe640 1 -- 172.21.15.26:0/167035240 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486352257 0 0) 0x7f1ed006b730 con 0x7f1ed00ce520 2023-12-19T16:57:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.785+0000 7f1ed4c02640 1 --2- 172.21.15.26:0/167035240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed00afd70 0x7f1ed00cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.785+0000 7f1ed4c02640 1 -- 172.21.15.26:0/167035240 >> v1:172.21.15.26:6789/0 conn(0x7f1ed00ce520 legacy=0x7f1ed0069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.785+0000 7f1ed4c02640 1 -- 172.21.15.26:0/167035240 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed0161a40 con 0x7f1ed00cc9b0 2023-12-19T16:57:37.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.785+0000 7f1ecffff640 1 --2- 172.21.15.26:0/167035240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed00afd70 0x7f1ed00cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.785+0000 7f1ed4c02640 1 --2- 172.21.15.26:0/167035240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00cc9b0 0x7f1ed00cd1d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40097c0 tx=0x7f1ec401c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e80e63b27151bc5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:37.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.785+0000 7f1ecf7fe640 1 -- 172.21.15.26:0/167035240 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40ab600 con 0x7f1ed00cc9b0 2023-12-19T16:57:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.789+0000 7f1ecf7fe640 1 -- 172.21.15.26:0/167035240 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec40a7050 con 0x7f1ed00cc9b0 2023-12-19T16:57:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.789+0000 7f1ecf7fe640 1 -- 172.21.15.26:0/167035240 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40ab980 con 0x7f1ed00cc9b0 2023-12-19T16:57:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.789+0000 7f1ed6e66640 1 -- 172.21.15.26:0/167035240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00cc9b0 msgr2=0x7f1ed00cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.789+0000 7f1ed6e66640 1 --2- 172.21.15.26:0/167035240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00cc9b0 0x7f1ed00cd1d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40097c0 tx=0x7f1ec401c3f0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.789+0000 7f1ed6e66640 1 -- 172.21.15.26:0/167035240 shutdown_connections 2023-12-19T16:57:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.789+0000 7f1ed6e66640 1 --2- 172.21.15.26:0/167035240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed00afd70 0x7f1ed00cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.789+0000 7f1ed6e66640 1 --2- 172.21.15.26:0/167035240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00cc9b0 0x7f1ed00cd1d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.789+0000 7f1ed6e66640 1 -- 172.21.15.26:0/167035240 >> 172.21.15.26:0/167035240 conn(0x7f1ed006d570 msgr2=0x7f1ed006d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:37.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.789+0000 7f1ed6e66640 1 -- 172.21.15.26:0/167035240 shutdown_connections 2023-12-19T16:57:37.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.789+0000 7f1ed6e66640 1 -- 172.21.15.26:0/167035240 wait complete. 2023-12-19T16:57:37.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.789+0000 7f1ed6e66640 1 Processor -- start 2023-12-19T16:57:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.789+0000 7f1ed6e66640 1 -- start start 2023-12-19T16:57:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.789+0000 7f1ed6e66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00afd70 0x7f1ed015d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed6e66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00cc9b0 0x7f1ed00caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed6e66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed00ce520 0x7f1ed00cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed6e66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed01575e0 con 0x7f1ed00cc9b0 2023-12-19T16:57:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed6e66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed015df90 con 0x7f1ed00afd70 2023-12-19T16:57:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed6e66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed0160640 con 0x7f1ed00ce520 2023-12-19T16:57:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ecffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00cc9b0 0x7f1ed00caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed5403640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed00ce520 0x7f1ed00cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed5403640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed00ce520 0x7f1ed00cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:60718/0 (socket says 172.21.15.26:60718) 2023-12-19T16:57:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed5403640 1 -- 172.21.15.26:0/2870526450 learned_addr learned my addr 172.21.15.26:0/2870526450 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed4c02640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00afd70 0x7f1ed015d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed5403640 1 -- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00afd70 msgr2=0x7f1ed015d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:37.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed5403640 1 --2- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00afd70 0x7f1ed015d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed5403640 1 -- 172.21.15.26:0/2870526450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00cc9b0 msgr2=0x7f1ed00caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed5403640 1 --2- 172.21.15.26:0/2870526450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00cc9b0 0x7f1ed00caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed5403640 1 -- 172.21.15.26:0/2870526450 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc009420 con 0x7f1ed00ce520 2023-12-19T16:57:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed4c02640 1 --2- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00afd70 0x7f1ed015d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:37.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed5403640 1 --2- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed00ce520 0x7f1ed00cb4c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8001570 tx=0x7f1ec800cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ecd7fa640 1 -- 172.21.15.26:0/2870526450 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800a640 con 0x7f1ed00ce520 2023-12-19T16:57:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ecd7fa640 1 -- 172.21.15.26:0/2870526450 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec800a7c0 con 0x7f1ed00ce520 2023-12-19T16:57:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ecd7fa640 1 -- 172.21.15.26:0/2870526450 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800ab70 con 0x7f1ed00ce520 2023-12-19T16:57:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed6e66640 1 -- 172.21.15.26:0/2870526450 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec4009420 con 0x7f1ed00ce520 2023-12-19T16:57:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed6e66640 1 -- 172.21.15.26:0/2870526450 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed015dca0 con 0x7f1ed00ce520 2023-12-19T16:57:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ed6e66640 1 -- 172.21.15.26:0/2870526450 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed0157720 con 0x7f1ed00ce520 2023-12-19T16:57:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.793+0000 7f1ecd7fa640 1 -- 172.21.15.26:0/2870526450 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f1ec800acd0 con 0x7f1ed00ce520 2023-12-19T16:57:37.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.797+0000 7f1ecd7fa640 1 --2- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1eb4065b80 0x7f1eb4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:37.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.797+0000 7f1ecd7fa640 1 -- 172.21.15.26:0/2870526450 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f1ec8082bf0 con 0x7f1ed00ce520 2023-12-19T16:57:37.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.797+0000 7f1ed4c02640 1 -- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1eb4065b80 msgr2=0x7f1eb4068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:37.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.797+0000 7f1ed4c02640 1 --2- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1eb4065b80 0x7f1eb4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:37.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.801+0000 7f1ecd7fa640 1 -- 172.21.15.26:0/2870526450 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1ed0157720 con 0x7f1ed00ce520 2023-12-19T16:57:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.981+0000 7f1ed6e66640 1 -- 172.21.15.26:0/2870526450 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ed0157720 con 0x7f1ed00ce520 2023-12-19T16:57:37.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.985+0000 7f1ecd7fa640 1 -- 172.21.15.26:0/2870526450 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17699 (secure 0 0 0) 0x7f1ed0157720 con 0x7f1ed00ce520 2023-12-19T16:57:37.989 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:37.989 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":80,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:35.339995+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:20.751846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:22.096742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6833/38098":"2023-12-20T16:57:35.339958+0000","172.21.15.160:6832/38098":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/3906095528":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/2024777587":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/936215346":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/704170657":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/2914461989":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/2579181328":"2023-12-20T16:57:35.339958+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:37.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.985+0000 7f1ed6e66640 1 -- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1eb4065b80 msgr2=0x7f1eb4068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:37.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.985+0000 7f1ed6e66640 1 --2- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1eb4065b80 0x7f1eb4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.985+0000 7f1ed6e66640 1 -- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed00ce520 msgr2=0x7f1ed00cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:37.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.985+0000 7f1ed6e66640 1 --2- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed00ce520 0x7f1ed00cb4c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8001570 tx=0x7f1ec800cca0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.985+0000 7f1ed6e66640 1 -- 172.21.15.26:0/2870526450 shutdown_connections 2023-12-19T16:57:37.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.985+0000 7f1ed6e66640 1 --2- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1eb4065b80 0x7f1eb4068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.985+0000 7f1ed6e66640 1 --2- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed00ce520 0x7f1ed00cb4c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.985+0000 7f1ed6e66640 1 --2- 172.21.15.26:0/2870526450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00cc9b0 0x7f1ed00caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.985+0000 7f1ed6e66640 1 --2- 172.21.15.26:0/2870526450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00afd70 0x7f1ed015d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:37.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.985+0000 7f1ed6e66640 1 -- 172.21.15.26:0/2870526450 >> 172.21.15.26:0/2870526450 conn(0x7f1ed006d570 msgr2=0x7f1ed006d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:37.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.985+0000 7f1ed6e66640 1 -- 172.21.15.26:0/2870526450 shutdown_connections 2023-12-19T16:57:37.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:37.985+0000 7f1ed6e66640 1 -- 172.21.15.26:0/2870526450 wait complete. 2023-12-19T16:57:38.012 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:57:20.751846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:57:22.096742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:57:38.012 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:57:38.013 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:38.011+0000 7f79d6506180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:57:38.057 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:38.053+0000 7ff967a0a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:57:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.157+0000 7fa33e612640 1 Processor -- start 2023-12-19T16:57:38.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.157+0000 7fa33e612640 1 -- start start 2023-12-19T16:57:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa33e612640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa33815d0d0 0x7fa3380b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa33e612640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa3380b4600 0x7fa3380b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa33e612640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33815e200 con 0x7fa33815ccf0 2023-12-19T16:57:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa33e612640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3380b4f20 con 0x7fa33815d0d0 2023-12-19T16:57:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa33e612640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3380d0aa0 con 0x7fa3380b4600 2023-12-19T16:57:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa3377fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa33815d0d0 0x7fa3380b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa3377fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa33815d0d0 0x7fa3380b8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44758/0 (socket says 172.21.15.26:44758) 2023-12-19T16:57:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa3377fe640 1 -- 172.21.15.26:0/2307158400 learned_addr learned my addr 172.21.15.26:0/2307158400 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa33cbaf640 1 --2- 172.21.15.26:0/2307158400 >> v2:172.21.15.160:3300/0 conn(0x7fa3380b4600 0x7fa3380b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa3377fe640 1 -- 172.21.15.26:0/2307158400 >> v2:172.21.15.160:3300/0 conn(0x7fa3380b4600 msgr2=0x7fa3380b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:38.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa3377fe640 1 --2- 172.21.15.26:0/2307158400 >> v2:172.21.15.160:3300/0 conn(0x7fa3380b4600 0x7fa3380b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa3377fe640 1 -- 172.21.15.26:0/2307158400 >> v1:172.21.15.160:6789/0 conn(0x7fa33815ccf0 legacy=0x7fa3380d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa3377fe640 1 -- 172.21.15.26:0/2307158400 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33815d680 con 0x7fa33815d0d0 2023-12-19T16:57:38.166 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:38.159+0000 7f79d6506180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:57:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.161+0000 7fa3377fe640 1 --2- 172.21.15.26:0/2307158400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa33815d0d0 0x7fa3380b8400 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa3380b2500 tx=0x7fa32c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0bf1a9d621b0e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.165+0000 7fa336ffd640 1 -- 172.21.15.26:0/2307158400 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0aa430 con 0x7fa33815d0d0 2023-12-19T16:57:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.165+0000 7fa336ffd640 1 -- 172.21.15.26:0/2307158400 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32c0aa590 con 0x7fa33815d0d0 2023-12-19T16:57:38.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.165+0000 7fa336ffd640 1 -- 172.21.15.26:0/2307158400 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0aa920 con 0x7fa33815d0d0 2023-12-19T16:57:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.165+0000 7fa33e612640 1 -- 172.21.15.26:0/2307158400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa33815d0d0 msgr2=0x7fa3380b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.165+0000 7fa33e612640 1 --2- 172.21.15.26:0/2307158400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa33815d0d0 0x7fa3380b8400 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa3380b2500 tx=0x7fa32c01c4d0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.165+0000 7fa33e612640 1 -- 172.21.15.26:0/2307158400 shutdown_connections 2023-12-19T16:57:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.165+0000 7fa33e612640 1 --2- 172.21.15.26:0/2307158400 >> v2:172.21.15.160:3300/0 conn(0x7fa3380b4600 0x7fa3380b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.165+0000 7fa33e612640 1 --2- 172.21.15.26:0/2307158400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa33815d0d0 0x7fa3380b8400 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.165+0000 7fa33e612640 1 -- 172.21.15.26:0/2307158400 >> 172.21.15.26:0/2307158400 conn(0x7fa338065ee0 msgr2=0x7fa3380cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.165+0000 7fa33e612640 1 -- 172.21.15.26:0/2307158400 shutdown_connections 2023-12-19T16:57:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.165+0000 7fa33e612640 1 -- 172.21.15.26:0/2307158400 wait complete. 2023-12-19T16:57:38.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.165+0000 7fa33e612640 1 Processor -- start 2023-12-19T16:57:38.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.165+0000 7fa33e612640 1 -- start start 2023-12-19T16:57:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa33e612640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3380b4600 0x7fa33815c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa33e612640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33815ccf0 0x7fa338155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa33e612640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa33815d0d0 0x7fa3380cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa33e612640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3380cc310 con 0x7fa3380b4600 2023-12-19T16:57:38.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa33e612640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3380cc5f0 con 0x7fa33815d0d0 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa33e612640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3380cc8d0 con 0x7fa33815ccf0 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa337fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3380b4600 0x7fa33815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa337fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3380b4600 0x7fa33815c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44762/0 (socket says 172.21.15.26:44762) 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa337fff640 1 -- 172.21.15.26:0/3454233547 learned_addr learned my addr 172.21.15.26:0/3454233547 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa3377fe640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33815ccf0 0x7fa338155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa33cbaf640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa33815d0d0 0x7fa3380cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa3377fe640 1 -- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa33815d0d0 msgr2=0x7fa3380cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa3377fe640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa33815d0d0 0x7fa3380cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa3377fe640 1 -- 172.21.15.26:0/3454233547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3380b4600 msgr2=0x7fa33815c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa3377fe640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3380b4600 0x7fa33815c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa3377fe640 1 -- 172.21.15.26:0/3454233547 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c0094d0 con 0x7fa33815ccf0 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa33cbaf640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa33815d0d0 0x7fa3380cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa337fff640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3380b4600 0x7fa33815c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa3377fe640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33815ccf0 0x7fa338155cf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0ab000 tx=0x7fa32c01ad10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa334ff9640 1 -- 172.21.15.26:0/3454233547 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0aa430 con 0x7fa33815ccf0 2023-12-19T16:57:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa334ff9640 1 -- 172.21.15.26:0/3454233547 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32c0b3820 con 0x7fa33815ccf0 2023-12-19T16:57:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa33e612640 1 -- 172.21.15.26:0/3454233547 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa338157c70 con 0x7fa33815ccf0 2023-12-19T16:57:38.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa334ff9640 1 -- 172.21.15.26:0/3454233547 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0b3bd0 con 0x7fa33815ccf0 2023-12-19T16:57:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa33e612640 1 -- 172.21.15.26:0/3454233547 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3380cbf00 con 0x7fa33815ccf0 2023-12-19T16:57:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.169+0000 7fa33e612640 1 -- 172.21.15.26:0/3454233547 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa338074270 con 0x7fa33815ccf0 2023-12-19T16:57:38.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.173+0000 7fa334ff9640 1 -- 172.21.15.26:0/3454233547 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fa32c0b3d30 con 0x7fa33815ccf0 2023-12-19T16:57:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.173+0000 7fa334ff9640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa324065b80 0x7fa324068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.173+0000 7fa334ff9640 1 -- 172.21.15.26:0/3454233547 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fa32c11dfe0 con 0x7fa33815ccf0 2023-12-19T16:57:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.173+0000 7fa337fff640 1 -- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa324065b80 msgr2=0x7fa324068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:38.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.173+0000 7fa337fff640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa324065b80 0x7fa324068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:38.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.177+0000 7fa334ff9640 1 -- 172.21.15.26:0/3454233547 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa32c0eb3e0 con 0x7fa33815ccf0 2023-12-19T16:57:38.219 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:38.213+0000 7ff967a0a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:57:38.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa33e612640 1 -- 172.21.15.26:0/3454233547 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3380cefd0 con 0x7fa33815ccf0 2023-12-19T16:57:38.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa334ff9640 1 -- 172.21.15.26:0/3454233547 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa32c0eea00 con 0x7fa33815ccf0 2023-12-19T16:57:38.341 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:57:38.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa33e612640 1 -- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa324065b80 msgr2=0x7fa324068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:38.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa33e612640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa324065b80 0x7fa324068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa33e612640 1 -- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33815ccf0 msgr2=0x7fa338155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa33e612640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33815ccf0 0x7fa338155cf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa32c0ab000 tx=0x7fa32c01ad10 comp rx=0 tx=0).stop 2023-12-19T16:57:38.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa33e612640 1 -- 172.21.15.26:0/3454233547 shutdown_connections 2023-12-19T16:57:38.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa33e612640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa33815d0d0 0x7fa3380cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa33e612640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa324065b80 0x7fa324068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa33e612640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa33815ccf0 0x7fa338155cf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa33e612640 1 --2- 172.21.15.26:0/3454233547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3380b4600 0x7fa33815c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa33e612640 1 -- 172.21.15.26:0/3454233547 >> 172.21.15.26:0/3454233547 conn(0x7fa338065ee0 msgr2=0x7fa3380cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:38.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa33e612640 1 -- 172.21.15.26:0/3454233547 shutdown_connections 2023-12-19T16:57:38.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.337+0000 7fa33e612640 1 -- 172.21.15.26:0/3454233547 wait complete. 2023-12-19T16:57:38.364 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:57:38.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.549+0000 7fb03746f640 1 Processor -- start 2023-12-19T16:57:38.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.549+0000 7fb03746f640 1 -- start start 2023-12-19T16:57:38.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb03746f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb03015cc90 0x7fb0300cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb03746f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb030160a20 0x7fb0300d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb03746f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0300cdb20 con 0x7fb03015d070 2023-12-19T16:57:38.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb03746f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0300cde00 con 0x7fb03015cc90 2023-12-19T16:57:38.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb03746f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0300ce0e0 con 0x7fb030160a20 2023-12-19T16:57:38.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb03646d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb03015cc90 0x7fb0300cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb03646d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb03015cc90 0x7fb0300cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44768/0 (socket says 172.21.15.26:44768) 2023-12-19T16:57:38.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb03646d640 1 -- 172.21.15.26:0/2841721884 learned_addr learned my addr 172.21.15.26:0/2841721884 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:38.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb036c6e640 1 --2- 172.21.15.26:0/2841721884 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb030160a20 0x7fb0300d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb03646d640 1 -- 172.21.15.26:0/2841721884 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb030160a20 msgr2=0x7fb0300d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb03646d640 1 --2- 172.21.15.26:0/2841721884 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb030160a20 0x7fb0300d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb03646d640 1 -- 172.21.15.26:0/2841721884 >> v1:172.21.15.160:6789/0 conn(0x7fb03015d070 legacy=0x7fb0300cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb03646d640 1 -- 172.21.15.26:0/2841721884 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0300ce3c0 con 0x7fb03015cc90 2023-12-19T16:57:38.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.553+0000 7fb03546b640 1 -- 172.21.15.26:0/2841721884 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304288221 0 0) 0x7fb0300cdb20 con 0x7fb03015d070 2023-12-19T16:57:38.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03646d640 1 --2- 172.21.15.26:0/2841721884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03015cc90 0x7fb0300cd2f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009870 tx=0x7fb02c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef26b0376d5f1dfd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:38.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03546b640 1 -- 172.21.15.26:0/2841721884 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c0aa650 con 0x7fb03015cc90 2023-12-19T16:57:38.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03546b640 1 -- 172.21.15.26:0/2841721884 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c0aa7b0 con 0x7fb03015cc90 2023-12-19T16:57:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03546b640 1 -- 172.21.15.26:0/2841721884 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c0aab40 con 0x7fb03015cc90 2023-12-19T16:57:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03746f640 1 -- 172.21.15.26:0/2841721884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03015cc90 msgr2=0x7fb0300cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03746f640 1 --2- 172.21.15.26:0/2841721884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03015cc90 0x7fb0300cd2f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009870 tx=0x7fb02c01c510 comp rx=0 tx=0).stop 2023-12-19T16:57:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03746f640 1 -- 172.21.15.26:0/2841721884 shutdown_connections 2023-12-19T16:57:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03746f640 1 --2- 172.21.15.26:0/2841721884 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb030160a20 0x7fb0300d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03746f640 1 --2- 172.21.15.26:0/2841721884 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03015cc90 0x7fb0300cd2f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03746f640 1 -- 172.21.15.26:0/2841721884 >> 172.21.15.26:0/2841721884 conn(0x7fb03006a320 msgr2=0x7fb0300658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03746f640 1 -- 172.21.15.26:0/2841721884 shutdown_connections 2023-12-19T16:57:38.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03746f640 1 -- 172.21.15.26:0/2841721884 wait complete. 2023-12-19T16:57:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03746f640 1 Processor -- start 2023-12-19T16:57:38.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.557+0000 7fb03746f640 1 -- start start 2023-12-19T16:57:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb03746f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03015cc90 0x7fb0300c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb03746f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03015d070 0x7fb03015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb03746f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb030160a20 0x7fb0301565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb03746f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb030154b30 con 0x7fb03015d070 2023-12-19T16:57:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb03746f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0301556b0 con 0x7fb03015cc90 2023-12-19T16:57:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb03746f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb030154d80 con 0x7fb030160a20 2023-12-19T16:57:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb035c6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03015d070 0x7fb03015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb035c6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03015d070 0x7fb03015b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44776/0 (socket says 172.21.15.26:44776) 2023-12-19T16:57:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb035c6c640 1 -- 172.21.15.26:0/2608902531 learned_addr learned my addr 172.21.15.26:0/2608902531 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:38.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb036c6e640 1 --2- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb030160a20 0x7fb0301565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb03646d640 1 --2- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03015cc90 0x7fb0300c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb035c6c640 1 -- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb030160a20 msgr2=0x7fb0301565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb035c6c640 1 --2- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb030160a20 0x7fb0301565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb035c6c640 1 -- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03015cc90 msgr2=0x7fb0300c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb035c6c640 1 --2- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03015cc90 0x7fb0300c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb035c6c640 1 -- 172.21.15.26:0/2608902531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c0094d0 con 0x7fb03015d070 2023-12-19T16:57:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb035c6c640 1 --2- 172.21.15.26:0/2608902531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03015d070 0x7fb03015b9d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb02000c500 tx=0x7fb02000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb026ffd640 1 -- 172.21.15.26:0/2608902531 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020003600 con 0x7fb03015d070 2023-12-19T16:57:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb026ffd640 1 -- 172.21.15.26:0/2608902531 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb020003760 con 0x7fb03015d070 2023-12-19T16:57:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb026ffd640 1 -- 172.21.15.26:0/2608902531 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020003b10 con 0x7fb03015d070 2023-12-19T16:57:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb03746f640 1 -- 172.21.15.26:0/2608902531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb030155060 con 0x7fb03015d070 2023-12-19T16:57:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb03746f640 1 -- 172.21.15.26:0/2608902531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0300c5100 con 0x7fb03015d070 2023-12-19T16:57:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb03746f640 1 -- 172.21.15.26:0/2608902531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb030061930 con 0x7fb03015d070 2023-12-19T16:57:38.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.561+0000 7fb026ffd640 1 -- 172.21.15.26:0/2608902531 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fb020003c70 con 0x7fb03015d070 2023-12-19T16:57:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.565+0000 7fb026ffd640 1 --2- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb014065bb0 0x7fb014068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.565+0000 7fb026ffd640 1 -- 172.21.15.26:0/2608902531 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fb020082c60 con 0x7fb03015d070 2023-12-19T16:57:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.565+0000 7fb03646d640 1 -- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb014065bb0 msgr2=0x7fb014068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:38.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.565+0000 7fb03646d640 1 --2- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb014065bb0 0x7fb014068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:38.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.569+0000 7fb026ffd640 1 -- 172.21.15.26:0/2608902531 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb0200502d0 con 0x7fb03015d070 2023-12-19T16:57:38.654 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:38.651+0000 7f79d6506180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:57:38.714 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:38.711+0000 7f79d6506180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:57:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.729+0000 7fb03746f640 1 -- 172.21.15.26:0/2608902531 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb0300c9290 con 0x7fb03015d070 2023-12-19T16:57:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.729+0000 7fb026ffd640 1 -- 172.21.15.26:0/2608902531 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb020017020 con 0x7fb03015d070 2023-12-19T16:57:38.735 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:57:38.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.733+0000 7fb03746f640 1 -- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb014065bb0 msgr2=0x7fb014068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:38.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.733+0000 7fb03746f640 1 --2- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb014065bb0 0x7fb014068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.733+0000 7fb03746f640 1 -- 172.21.15.26:0/2608902531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03015d070 msgr2=0x7fb03015b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.733+0000 7fb03746f640 1 --2- 172.21.15.26:0/2608902531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03015d070 0x7fb03015b9d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb02000c500 tx=0x7fb02000c9c0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.733+0000 7fb03746f640 1 -- 172.21.15.26:0/2608902531 shutdown_connections 2023-12-19T16:57:38.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.733+0000 7fb03746f640 1 --2- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb014065bb0 0x7fb014068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.733+0000 7fb03746f640 1 --2- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb030160a20 0x7fb0301565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.733+0000 7fb03746f640 1 --2- 172.21.15.26:0/2608902531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03015d070 0x7fb03015b9d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.733+0000 7fb03746f640 1 --2- 172.21.15.26:0/2608902531 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb03015cc90 0x7fb0300c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.733+0000 7fb03746f640 1 -- 172.21.15.26:0/2608902531 >> 172.21.15.26:0/2608902531 conn(0x7fb03006a320 msgr2=0x7fb03015e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:38.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.733+0000 7fb03746f640 1 -- 172.21.15.26:0/2608902531 shutdown_connections 2023-12-19T16:57:38.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.733+0000 7fb03746f640 1 -- 172.21.15.26:0/2608902531 wait complete. 2023-12-19T16:57:38.744 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:38.741+0000 7ff967a0a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:57:38.767 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:57:38.772 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:38.767+0000 7f79d6506180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:57:38.814 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:38.809+0000 7ff967a0a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:57:38.857 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:38.855+0000 7f79d6506180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:57:38.876 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:38.869+0000 7ff967a0a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:57:38.917 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:38.911+0000 7f79d6506180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:57:38.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.913+0000 7f53827f3640 1 Processor -- start 2023-12-19T16:57:38.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.913+0000 7f53827f3640 1 -- start start 2023-12-19T16:57:38.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.913+0000 7f53827f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f537c15cde0 0x7f537c0b8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.913+0000 7f53827f3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537c0b5bd0 con 0x7f537c0b4d00 2023-12-19T16:57:38.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.913+0000 7f53827f3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f537c0d0ee0 con 0x7f537c15d1c0 2023-12-19T16:57:38.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.913+0000 7f53827f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c15e380 con 0x7f537c15cde0 2023-12-19T16:57:38.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.913+0000 7f5380d90640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f537c0b4d00 0x7f537c0d1370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38004/0 (socket says 172.21.15.26:38004) 2023-12-19T16:57:38.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.913+0000 7f5380d90640 1 -- 172.21.15.26:0/526044525 learned_addr learned my addr 172.21.15.26:0/526044525 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:38.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.913+0000 7f537bfff640 1 --2- 172.21.15.26:0/526044525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f537c15cde0 0x7f537c0b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.917+0000 7f537affd640 1 -- 172.21.15.26:0/526044525 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538788521 0 0) 0x7f537c0b5bd0 con 0x7f537c0b4d00 2023-12-19T16:57:38.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.917+0000 7f537affd640 1 -- 172.21.15.26:0/526044525 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5364003530 con 0x7f537c0b4d00 2023-12-19T16:57:38.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.917+0000 7f537bfff640 1 -- 172.21.15.26:0/526044525 >> v1:172.21.15.160:6789/0 conn(0x7f537c15d1c0 legacy=0x7f537c0b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.917+0000 7f537bfff640 1 -- 172.21.15.26:0/526044525 >> v1:172.21.15.26:6789/0 conn(0x7f537c0b4d00 legacy=0x7f537c0d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.917+0000 7f537bfff640 1 -- 172.21.15.26:0/526044525 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c15dae0 con 0x7f537c15cde0 2023-12-19T16:57:38.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.917+0000 7f537bfff640 1 --2- 172.21.15.26:0/526044525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f537c15cde0 0x7f537c0b8600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f536c0097f0 tx=0x7f536c01ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0e8299a7069df68 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:38.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.917+0000 7f537affd640 1 -- 172.21.15.26:0/526044525 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c0a89b0 con 0x7f537c15cde0 2023-12-19T16:57:38.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.917+0000 7f537affd640 1 -- 172.21.15.26:0/526044525 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c0a8b10 con 0x7f537c15cde0 2023-12-19T16:57:38.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.917+0000 7f537affd640 1 -- 172.21.15.26:0/526044525 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c0a8ea0 con 0x7f537c15cde0 2023-12-19T16:57:38.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 -- 172.21.15.26:0/526044525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f537c15cde0 msgr2=0x7f537c0b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 --2- 172.21.15.26:0/526044525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f537c15cde0 0x7f537c0b8600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f536c0097f0 tx=0x7f536c01ecb0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 -- 172.21.15.26:0/526044525 shutdown_connections 2023-12-19T16:57:38.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 --2- 172.21.15.26:0/526044525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f537c15cde0 0x7f537c0b8600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 -- 172.21.15.26:0/526044525 >> 172.21.15.26:0/526044525 conn(0x7f537c065ee0 msgr2=0x7f537c0d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:38.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 -- 172.21.15.26:0/526044525 shutdown_connections 2023-12-19T16:57:38.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 -- 172.21.15.26:0/526044525 wait complete. 2023-12-19T16:57:38.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 Processor -- start 2023-12-19T16:57:38.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 -- start start 2023-12-19T16:57:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f537c0b4d00 0x7f537c1f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f537c15cde0 0x7f537c14fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f537c15d1c0 0x7f537c0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c149e10 con 0x7f537c15cde0 2023-12-19T16:57:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c150250 con 0x7f537c15d1c0 2023-12-19T16:57:38.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f53827f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c1f6bc0 con 0x7f537c0b4d00 2023-12-19T16:57:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f537b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f537c15cde0 0x7f537c14fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f537bfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f537c0b4d00 0x7f537c1f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f537b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f537c15cde0 0x7f537c14fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44788/0 (socket says 172.21.15.26:44788) 2023-12-19T16:57:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f537b7fe640 1 -- 172.21.15.26:0/4249202331 learned_addr learned my addr 172.21.15.26:0/4249202331 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f537bfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f537c0b4d00 0x7f537c1f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:60772/0 (socket says 172.21.15.26:60772) 2023-12-19T16:57:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.921+0000 7f5380d90640 1 --2- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f537c15d1c0 0x7f537c0b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f537bfff640 1 -- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f537c15d1c0 msgr2=0x7f537c0b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f537bfff640 1 --2- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f537c15d1c0 0x7f537c0b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f537bfff640 1 -- 172.21.15.26:0/4249202331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f537c15cde0 msgr2=0x7f537c14fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f537bfff640 1 --2- 172.21.15.26:0/4249202331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f537c15cde0 0x7f537c14fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f537bfff640 1 -- 172.21.15.26:0/4249202331 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c009420 con 0x7f537c0b4d00 2023-12-19T16:57:38.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f537b7fe640 1 --2- 172.21.15.26:0/4249202331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f537c15cde0 0x7f537c14fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f537bfff640 1 --2- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f537c0b4d00 0x7f537c1f5cb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f536c0af000 tx=0x7f536c01ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f5378ff9640 1 -- 172.21.15.26:0/4249202331 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c0b9630 con 0x7f537c0b4d00 2023-12-19T16:57:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f5378ff9640 1 -- 172.21.15.26:0/4249202331 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536c0b97b0 con 0x7f537c0b4d00 2023-12-19T16:57:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f53827f3640 1 -- 172.21.15.26:0/4249202331 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537c0b13b0 con 0x7f537c0b4d00 2023-12-19T16:57:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f5378ff9640 1 -- 172.21.15.26:0/4249202331 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536c0b9b60 con 0x7f537c0b4d00 2023-12-19T16:57:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f53827f3640 1 -- 172.21.15.26:0/4249202331 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537c1f6600 con 0x7f537c0b4d00 2023-12-19T16:57:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f53827f3640 1 -- 172.21.15.26:0/4249202331 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537c15e080 con 0x7f537c0b4d00 2023-12-19T16:57:38.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.925+0000 7f5378ff9640 1 -- 172.21.15.26:0/4249202331 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f536c0b7040 con 0x7f537c0b4d00 2023-12-19T16:57:38.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.929+0000 7f5378ff9640 1 --2- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5368065b80 0x7f5368068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:38.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.929+0000 7f5378ff9640 1 -- 172.21.15.26:0/4249202331 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f536c121370 con 0x7f537c0b4d00 2023-12-19T16:57:38.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.929+0000 7f537b7fe640 1 -- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5368065b80 msgr2=0x7f5368068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:38.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.929+0000 7f537b7fe640 1 --2- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5368065b80 0x7f5368068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:38.933+0000 7f5378ff9640 1 -- 172.21.15.26:0/4249202331 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f537c15e080 con 0x7f537c0b4d00 2023-12-19T16:57:38.970 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:38.965+0000 7ff967a0a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:57:39.003 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:38.999+0000 7f79d6506180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:57:39.030 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:39.025+0000 7ff967a0a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:57:39.060 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:39.055+0000 7f79d6506180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:57:39.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.089+0000 7f53827f3640 1 -- 172.21.15.26:0/4249202331 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f537c15e050 con 0x7f537c0b4d00 2023-12-19T16:57:39.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.089+0000 7f5378ff9640 1 -- 172.21.15.26:0/4249202331 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f536c0ee920 con 0x7f537c0b4d00 2023-12-19T16:57:39.095 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:57:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.093+0000 7f53827f3640 1 -- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5368065b80 msgr2=0x7f5368068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.093+0000 7f53827f3640 1 --2- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5368065b80 0x7f5368068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.093+0000 7f53827f3640 1 -- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f537c0b4d00 msgr2=0x7f537c1f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.093+0000 7f53827f3640 1 --2- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f537c0b4d00 0x7f537c1f5cb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f536c0af000 tx=0x7f536c01ba50 comp rx=0 tx=0).stop 2023-12-19T16:57:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.093+0000 7f53827f3640 1 -- 172.21.15.26:0/4249202331 shutdown_connections 2023-12-19T16:57:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.093+0000 7f53827f3640 1 --2- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f537c15d1c0 0x7f537c0b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.093+0000 7f53827f3640 1 --2- 172.21.15.26:0/4249202331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f537c15cde0 0x7f537c14fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.093+0000 7f53827f3640 1 --2- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5368065b80 0x7f5368068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.093+0000 7f53827f3640 1 --2- 172.21.15.26:0/4249202331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f537c0b4d00 0x7f537c1f5cb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.093+0000 7f53827f3640 1 -- 172.21.15.26:0/4249202331 >> 172.21.15.26:0/4249202331 conn(0x7f537c065ee0 msgr2=0x7f537c15ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.093+0000 7f53827f3640 1 -- 172.21.15.26:0/4249202331 shutdown_connections 2023-12-19T16:57:39.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.093+0000 7f53827f3640 1 -- 172.21.15.26:0/4249202331 wait complete. 2023-12-19T16:57:39.118 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T16:57:39.121 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:39.117+0000 7ff967a0a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:57:39.185 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:39.181+0000 7ff967a0a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:57:39.208 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:39.203+0000 7f79d6506180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:57:39.268 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:39.263+0000 7f79d6506180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:57:39.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9eb0dbc640 1 Processor -- start 2023-12-19T16:57:39.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9eb0dbc640 1 -- start start 2023-12-19T16:57:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9eb0dbc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9eac0ccc20 0x7f9eac161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9eb0dbc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eac160890 con 0x7f9eac0d0480 2023-12-19T16:57:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9eb0dbc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eac0cdaf0 con 0x7f9eac0cd050 2023-12-19T16:57:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9eb0dbc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eac160b40 con 0x7f9eac0ccc20 2023-12-19T16:57:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9eaad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9eac0d0480 0x7f9eac069d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38018/0 (socket says 172.21.15.26:38018) 2023-12-19T16:57:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9eaad9d640 1 -- 172.21.15.26:0/1690369354 learned_addr learned my addr 172.21.15.26:0/1690369354 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9eaa59c640 1 --2- 172.21.15.26:0/1690369354 >> v2:172.21.15.160:3300/0 conn(0x7f9eac0ccc20 0x7f9eac161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9ea959a640 1 -- 172.21.15.26:0/1690369354 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245508747 0 0) 0x7f9ea40014a0 con 0x7f9eac0d0480 2023-12-19T16:57:39.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9ea959a640 1 -- 172.21.15.26:0/1690369354 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e94003530 con 0x7f9eac0d0480 2023-12-19T16:57:39.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9ea959a640 1 -- 172.21.15.26:0/1690369354 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1221815782 0 0) 0x7f9e94003530 con 0x7f9eac0d0480 2023-12-19T16:57:39.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9eaa59c640 1 -- 172.21.15.26:0/1690369354 >> v1:172.21.15.160:6789/0 conn(0x7f9eac0cd050 legacy=0x7f9eac0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:39.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9eaa59c640 1 -- 172.21.15.26:0/1690369354 >> v1:172.21.15.26:6789/0 conn(0x7f9eac0d0480 legacy=0x7f9eac069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:39.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.265+0000 7f9eaa59c640 1 -- 172.21.15.26:0/1690369354 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac160e20 con 0x7f9eac0ccc20 2023-12-19T16:57:39.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.269+0000 7f9eaa59c640 1 --2- 172.21.15.26:0/1690369354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9eac0ccc20 0x7f9eac161620 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0097c0 tx=0x7f9eac06ac40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=431f5f0ef4b772c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.269+0000 7f9ea959a640 1 -- 172.21.15.26:0/1690369354 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea4001b90 con 0x7f9eac0ccc20 2023-12-19T16:57:39.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.269+0000 7f9ea959a640 1 -- 172.21.15.26:0/1690369354 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c0a7050 con 0x7f9eac0ccc20 2023-12-19T16:57:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.269+0000 7f9ea959a640 1 -- 172.21.15.26:0/1690369354 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0ab620 con 0x7f9eac0ccc20 2023-12-19T16:57:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.269+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/1690369354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9eac0ccc20 msgr2=0x7f9eac161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:39.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.269+0000 7f9eb0dbc640 1 --2- 172.21.15.26:0/1690369354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9eac0ccc20 0x7f9eac161620 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0097c0 tx=0x7f9eac06ac40 comp rx=0 tx=0).stop 2023-12-19T16:57:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/1690369354 shutdown_connections 2023-12-19T16:57:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eb0dbc640 1 --2- 172.21.15.26:0/1690369354 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9eac0ccc20 0x7f9eac161620 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/1690369354 >> 172.21.15.26:0/1690369354 conn(0x7f9eac06d510 msgr2=0x7f9eac06d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/1690369354 shutdown_connections 2023-12-19T16:57:39.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/1690369354 wait complete. 2023-12-19T16:57:39.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eb0dbc640 1 Processor -- start 2023-12-19T16:57:39.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eb0dbc640 1 -- start start 2023-12-19T16:57:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eb0dbc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9eac0ccc20 0x7f9eac153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eb0dbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac0cd050 0x7f9eac0ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eb0dbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9eac0d0480 0x7f9eac0aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eb0dbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac143740 con 0x7f9eac0cd050 2023-12-19T16:57:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eb0dbc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac1541d0 con 0x7f9eac0ccc20 2023-12-19T16:57:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eb0dbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac143cc0 con 0x7f9eac0d0480 2023-12-19T16:57:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9ea9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac0cd050 0x7f9eac0ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9ea9d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac0cd050 0x7f9eac0ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44802/0 (socket says 172.21.15.26:44802) 2023-12-19T16:57:39.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9ea9d9b640 1 -- 172.21.15.26:0/2198379761 learned_addr learned my addr 172.21.15.26:0/2198379761 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eaa59c640 1 --2- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9eac0ccc20 0x7f9eac153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.273+0000 7f9eaad9d640 1 --2- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9eac0d0480 0x7f9eac0aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.277+0000 7f9ea9d9b640 1 -- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9eac0d0480 msgr2=0x7f9eac0aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.277+0000 7f9ea9d9b640 1 --2- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9eac0d0480 0x7f9eac0aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.277+0000 7f9ea9d9b640 1 -- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9eac0ccc20 msgr2=0x7f9eac153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.277+0000 7f9ea9d9b640 1 --2- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9eac0ccc20 0x7f9eac153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.277+0000 7f9ea9d9b640 1 -- 172.21.15.26:0/2198379761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c009420 con 0x7f9eac0cd050 2023-12-19T16:57:39.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.277+0000 7f9ea9d9b640 1 --2- 172.21.15.26:0/2198379761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac0cd050 0x7f9eac0ae620 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c500 tx=0x7f9ea000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.277+0000 7f9e9affd640 1 -- 172.21.15.26:0/2198379761 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0005220 con 0x7f9eac0cd050 2023-12-19T16:57:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.277+0000 7f9e9affd640 1 -- 172.21.15.26:0/2198379761 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea0005380 con 0x7f9eac0cd050 2023-12-19T16:57:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.277+0000 7f9e9affd640 1 -- 172.21.15.26:0/2198379761 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0007270 con 0x7f9eac0cd050 2023-12-19T16:57:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.277+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/2198379761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac0af0a0 con 0x7f9eac0cd050 2023-12-19T16:57:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.277+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/2198379761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac0af5f0 con 0x7f9eac0cd050 2023-12-19T16:57:39.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.277+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/2198379761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eac143e70 con 0x7f9eac0cd050 2023-12-19T16:57:39.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.277+0000 7f9e9affd640 1 -- 172.21.15.26:0/2198379761 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f9ea00073d0 con 0x7f9eac0cd050 2023-12-19T16:57:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.281+0000 7f9e9affd640 1 --2- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9e8c065b80 0x7f9e8c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.281+0000 7f9e9affd640 1 -- 172.21.15.26:0/2198379761 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f9ea0082f50 con 0x7f9eac0cd050 2023-12-19T16:57:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.281+0000 7f9eaa59c640 1 -- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9e8c065b80 msgr2=0x7f9e8c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:39.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.281+0000 7f9eaa59c640 1 --2- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9e8c065b80 0x7f9e8c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:39.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.285+0000 7f9e9affd640 1 -- 172.21.15.26:0/2198379761 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9eac143e70 con 0x7f9eac0cd050 2023-12-19T16:57:39.340 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:39.337+0000 7ff967a0a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:57:39.402 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:39.397+0000 7ff967a0a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:57:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.453+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/2198379761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9eac143e30 con 0x7f9eac0cd050 2023-12-19T16:57:39.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.453+0000 7f9e9affd640 1 -- 172.21.15.26:0/2198379761 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v80) v1 ==== 80+0+466 (secure 0 0 0) 0x7f9ea00503e0 con 0x7f9eac0cd050 2023-12-19T16:57:39.460 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6833/38098 2023-12-20T16:57:35.339958+0000 2023-12-19T16:57:39.460 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6832/38098 2023-12-20T16:57:35.339958+0000 2023-12-19T16:57:39.460 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3906095528 2023-12-20T16:57:35.339958+0000 2023-12-19T16:57:39.460 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2024777587 2023-12-20T16:57:35.339958+0000 2023-12-19T16:57:39.460 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/936215346 2023-12-20T16:57:35.339958+0000 2023-12-19T16:57:39.460 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/704170657 2023-12-20T16:57:35.339958+0000 2023-12-19T16:57:39.460 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2914461989 2023-12-20T16:57:35.339958+0000 2023-12-19T16:57:39.460 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2579181328 2023-12-20T16:57:35.339958+0000 2023-12-19T16:57:39.461 INFO:teuthology.orchestra.run.smithi026.stderr:listed 8 entries 2023-12-19T16:57:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.457+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9e8c065b80 msgr2=0x7f9e8c068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.457+0000 7f9eb0dbc640 1 --2- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9e8c065b80 0x7f9e8c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.457+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/2198379761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac0cd050 msgr2=0x7f9eac0ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.457+0000 7f9eb0dbc640 1 --2- 172.21.15.26:0/2198379761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac0cd050 0x7f9eac0ae620 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000c500 tx=0x7f9ea000c9c0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.457+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/2198379761 shutdown_connections 2023-12-19T16:57:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.457+0000 7f9eb0dbc640 1 --2- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9e8c065b80 0x7f9e8c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.457+0000 7f9eb0dbc640 1 --2- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9eac0d0480 0x7f9eac0aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.457+0000 7f9eb0dbc640 1 --2- 172.21.15.26:0/2198379761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eac0cd050 0x7f9eac0ae620 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.457+0000 7f9eb0dbc640 1 --2- 172.21.15.26:0/2198379761 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9eac0ccc20 0x7f9eac153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.457+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/2198379761 >> 172.21.15.26:0/2198379761 conn(0x7f9eac06d510 msgr2=0x7f9eac0b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.457+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/2198379761 shutdown_connections 2023-12-19T16:57:39.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.457+0000 7f9eb0dbc640 1 -- 172.21.15.26:0/2198379761 wait complete. 2023-12-19T16:57:39.481 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T16:57:39.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:39.481 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-19T16:57:39.510 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T16:57:39.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:57:39.510 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T16:57:39.512 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:57:39.512 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:57:39.512 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:57:39.521 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:39.515+0000 7f79d6506180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:57:39.579 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:57:39.579 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:39.588 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:57:39.589 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:57:39.589 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:39.634 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:57:39.634 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:57:39.634 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:39.643 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:57:39.644 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:57:39.657 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:39.655+0000 7f79d6506180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:57:39.659 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:57:39.659 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:57:39.659 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:57:39.668 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:39.665+0000 7ff967a0a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:57:39.784 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:39.779+0000 7f79d6506180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:57:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9edb76d640 1 Processor -- start 2023-12-19T16:57:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9edb76d640 1 -- start start 2023-12-19T16:57:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9edb76d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ed4154220 0x7f9ed415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9edb76d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9ed406b3d0 0x7f9ed40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9edb76d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ed40cec40 0x7f9ed415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9edb76d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed40cf0d0 con 0x7f9ed4154220 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9edb76d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed406bc30 con 0x7f9ed406b3d0 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9edb76d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40cc6e0 con 0x7f9ed40cec40 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9ed9509640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ed4154220 0x7f9ed415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9ed9509640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ed4154220 0x7f9ed415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44812/0 (socket says 172.21.15.26:44812) 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9ed9509640 1 -- 172.21.15.26:0/2212210331 learned_addr learned my addr 172.21.15.26:0/2212210331 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9ed9d0a640 1 --2- 172.21.15.26:0/2212210331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ed40cec40 0x7f9ed415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9ed8d08640 1 --2- 172.21.15.26:0/2212210331 >> v2:172.21.15.160:3300/0 conn(0x7f9ed406b3d0 0x7f9ed40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9ed9509640 1 -- 172.21.15.26:0/2212210331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ed40cec40 msgr2=0x7f9ed415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9ed9509640 1 --2- 172.21.15.26:0/2212210331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ed40cec40 0x7f9ed415d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9ed9509640 1 -- 172.21.15.26:0/2212210331 >> v2:172.21.15.160:3300/0 conn(0x7f9ed406b3d0 msgr2=0x7f9ed40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9ed9509640 1 --2- 172.21.15.26:0/2212210331 >> v2:172.21.15.160:3300/0 conn(0x7f9ed406b3d0 0x7f9ed40ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.801+0000 7f9ed9509640 1 -- 172.21.15.26:0/2212210331 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed415afc0 con 0x7f9ed4154220 2023-12-19T16:57:39.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.805+0000 7f9ed9509640 1 --2- 172.21.15.26:0/2212210331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed4154220 0x7f9ed415aa80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40cf880 tx=0x7f9ed001e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee8e00cb9f9e9b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.805+0000 7f9ecbfff640 1 -- 172.21.15.26:0/2212210331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00a7970 con 0x7f9ed4154220 2023-12-19T16:57:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.805+0000 7f9ecbfff640 1 -- 172.21.15.26:0/2212210331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed00a7ad0 con 0x7f9ed4154220 2023-12-19T16:57:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.805+0000 7f9ecbfff640 1 -- 172.21.15.26:0/2212210331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00a7e60 con 0x7f9ed4154220 2023-12-19T16:57:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.805+0000 7f9edb76d640 1 -- 172.21.15.26:0/2212210331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed4154220 msgr2=0x7f9ed415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.805+0000 7f9edb76d640 1 --2- 172.21.15.26:0/2212210331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed4154220 0x7f9ed415aa80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40cf880 tx=0x7f9ed001e8e0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.805+0000 7f9edb76d640 1 -- 172.21.15.26:0/2212210331 shutdown_connections 2023-12-19T16:57:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.805+0000 7f9edb76d640 1 --2- 172.21.15.26:0/2212210331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ed40cec40 0x7f9ed415d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.805+0000 7f9edb76d640 1 --2- 172.21.15.26:0/2212210331 >> v2:172.21.15.160:3300/0 conn(0x7f9ed406b3d0 0x7f9ed40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.805+0000 7f9edb76d640 1 --2- 172.21.15.26:0/2212210331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed4154220 0x7f9ed415aa80 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.805+0000 7f9edb76d640 1 -- 172.21.15.26:0/2212210331 >> 172.21.15.26:0/2212210331 conn(0x7f9ed40658e0 msgr2=0x7f9ed4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.805+0000 7f9edb76d640 1 -- 172.21.15.26:0/2212210331 shutdown_connections 2023-12-19T16:57:39.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.805+0000 7f9edb76d640 1 -- 172.21.15.26:0/2212210331 wait complete. 2023-12-19T16:57:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9edb76d640 1 Processor -- start 2023-12-19T16:57:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9edb76d640 1 -- start start 2023-12-19T16:57:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9edb76d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed406b3d0 0x7f9ed414cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9edb76d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed40cec40 0x7f9ed40b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9edb76d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ed4154220 0x7f9ed40aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9edb76d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed414d370 con 0x7f9ed40cec40 2023-12-19T16:57:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9edb76d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed414d650 con 0x7f9ed406b3d0 2023-12-19T16:57:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9edb76d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed414d930 con 0x7f9ed4154220 2023-12-19T16:57:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ed8d08640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed40cec40 0x7f9ed40b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:39.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ed8d08640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed40cec40 0x7f9ed40b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44828/0 (socket says 172.21.15.26:44828) 2023-12-19T16:57:39.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ed8d08640 1 -- 172.21.15.26:0/2973844309 learned_addr learned my addr 172.21.15.26:0/2973844309 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:39.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ed9509640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed406b3d0 0x7f9ed414cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:39.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ed9d0a640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ed4154220 0x7f9ed40aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:39.815 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:39.809+0000 7ff967a0a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:57:39.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ed8d08640 1 -- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ed4154220 msgr2=0x7f9ed40aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:39.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ed8d08640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ed4154220 0x7f9ed40aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ed8d08640 1 -- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed406b3d0 msgr2=0x7f9ed414cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ed8d08640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed406b3d0 0x7f9ed414cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ed8d08640 1 -- 172.21.15.26:0/2973844309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed000b770 con 0x7f9ed40cec40 2023-12-19T16:57:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ed8d08640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed40cec40 0x7f9ed40b3370 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400d800 tx=0x7f9ec400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ec9ffb640 1 -- 172.21.15.26:0/2973844309 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec40096a0 con 0x7f9ed40cec40 2023-12-19T16:57:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ec9ffb640 1 -- 172.21.15.26:0/2973844309 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec4009800 con 0x7f9ed40cec40 2023-12-19T16:57:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9ec9ffb640 1 -- 172.21.15.26:0/2973844309 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4009990 con 0x7f9ed40cec40 2023-12-19T16:57:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9edb76d640 1 -- 172.21.15.26:0/2973844309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed40b2b70 con 0x7f9ed40cec40 2023-12-19T16:57:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.809+0000 7f9edb76d640 1 -- 172.21.15.26:0/2973844309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed414d060 con 0x7f9ed40cec40 2023-12-19T16:57:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.813+0000 7f9edb76d640 1 -- 172.21.15.26:0/2973844309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed4074270 con 0x7f9ed40cec40 2023-12-19T16:57:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.813+0000 7f9ec9ffb640 1 -- 172.21.15.26:0/2973844309 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f9ec4009af0 con 0x7f9ed40cec40 2023-12-19T16:57:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.813+0000 7f9ec9ffb640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9eb8065bb0 0x7f9eb8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.813+0000 7f9ec9ffb640 1 -- 172.21.15.26:0/2973844309 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f9ec4083aa0 con 0x7f9ed40cec40 2023-12-19T16:57:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.813+0000 7f9ed9509640 1 -- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9eb8065bb0 msgr2=0x7f9eb8068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:39.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.813+0000 7f9ed9509640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9eb8065bb0 0x7f9eb8068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:39.821+0000 7f9ec9ffb640 1 -- 172.21.15.26:0/2973844309 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9ec4050fa0 con 0x7f9ed40cec40 2023-12-19T16:57:39.857 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:39.851+0000 7f79d6506180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:57:39.950 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:39.945+0000 7ff967a0a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:57:39.992 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:39.987+0000 7f79d6506180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:57:40.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.017+0000 7f9ed9509640 1 -- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9eb8065bb0 msgr2=0x7f9eb8068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:40.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.017+0000 7f9ed9509640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9eb8065bb0 0x7f9eb8068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T16:57:40.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.017+0000 7f9edb76d640 1 -- 172.21.15.26:0/2973844309 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ed40cdf60 con 0x7f9ed40cec40 2023-12-19T16:57:40.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.017+0000 7f9ec9ffb640 1 -- 172.21.15.26:0/2973844309 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f9ec400b3a0 con 0x7f9ed40cec40 2023-12-19T16:57:40.023 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:40.023 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/436365775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":436365775},{"type":"v1","addr":"172.21.15.160:6836","nonce":436365775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3014042376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3014042376},{"type":"v1","addr":"172.21.15.160:6837","nonce":3014042376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1357342534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1357342534},{"type":"v1","addr":"172.21.15.26:6835","nonce":1357342534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:57:23.100695+0000","modified":"2023-12-19T16:57:26.118941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6522":{"gid":6522,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/325854666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":325854666},{"type":"v1","addr":"172.21.15.26:6834","nonce":325854666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-19T16:57:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 33 2023-12-19T16:57:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.021+0000 7f9edb76d640 1 -- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9eb8065bb0 msgr2=0x7f9eb8068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.021+0000 7f9edb76d640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9eb8065bb0 0x7f9eb8068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.021+0000 7f9edb76d640 1 -- 172.21.15.26:0/2973844309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed40cec40 msgr2=0x7f9ed40b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.021+0000 7f9edb76d640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed40cec40 0x7f9ed40b3370 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400d800 tx=0x7f9ec400dcc0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.021+0000 7f9edb76d640 1 -- 172.21.15.26:0/2973844309 shutdown_connections 2023-12-19T16:57:40.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.021+0000 7f9edb76d640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9eb8065bb0 0x7f9eb8068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.021+0000 7f9edb76d640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ed4154220 0x7f9ed40aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.021+0000 7f9edb76d640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed40cec40 0x7f9ed40b3370 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.021+0000 7f9edb76d640 1 --2- 172.21.15.26:0/2973844309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ed406b3d0 0x7f9ed414cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.021+0000 7f9edb76d640 1 -- 172.21.15.26:0/2973844309 >> 172.21.15.26:0/2973844309 conn(0x7f9ed40658e0 msgr2=0x7f9ed40cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:40.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.021+0000 7f9edb76d640 1 -- 172.21.15.26:0/2973844309 shutdown_connections 2023-12-19T16:57:40.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.021+0000 7f9edb76d640 1 -- 172.21.15.26:0/2973844309 wait complete. 2023-12-19T16:57:40.027 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:40.021+0000 7ff967a0a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:57:40.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:40.172 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:40.169+0000 7ff967a0a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:57:40.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0e2125640 1 Processor -- start 2023-12-19T16:57:40.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0e2125640 1 -- start start 2023-12-19T16:57:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0e2125640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc0d0480 0x7fe0dc069d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0e2125640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0dc160890 con 0x7fe0dc0ccc20 2023-12-19T16:57:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0e2125640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0dc0cdaf0 con 0x7fe0dc161620 2023-12-19T16:57:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0e2125640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc160b40 con 0x7fe0dc0d0480 2023-12-19T16:57:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0db7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe0dc0ccc20 0x7fe0dc0cd050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38034/0 (socket says 172.21.15.26:38034) 2023-12-19T16:57:40.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0db7fe640 1 -- 172.21.15.26:0/782061121 learned_addr learned my addr 172.21.15.26:0/782061121 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:40.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0dbfff640 1 --2- 172.21.15.26:0/782061121 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc0d0480 0x7fe0dc069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562546682 0 0) 0x7fe0cc001120 con 0x7fe0dc0ccc20 2023-12-19T16:57:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0c4003530 con 0x7fe0dc0ccc20 2023-12-19T16:57:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610196461 0 0) 0x7fe0dc0cdaf0 con 0x7fe0dc161620 2023-12-19T16:57:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0cc001120 con 0x7fe0dc161620 2023-12-19T16:57:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3902262772 0 0) 0x7fe0c4003530 con 0x7fe0dc0ccc20 2023-12-19T16:57:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc0d0480 msgr2=0x7fe0dc069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:40.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0da7fc640 1 --2- 172.21.15.26:0/782061121 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc0d0480 0x7fe0dc069d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 >> v1:172.21.15.160:6789/0 conn(0x7fe0dc161620 legacy=0x7fe0dc0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc160e20 con 0x7fe0dc0ccc20 2023-12-19T16:57:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fe0cc001810 con 0x7fe0dc0ccc20 2023-12-19T16:57:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 >> v1:172.21.15.26:6789/0 conn(0x7fe0dc0ccc20 legacy=0x7fe0dc0cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.229+0000 7fe0da7fc640 1 --2- 172.21.15.26:0/782061121 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc0d0480 0x7fe0c4003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0da7fc640 1 --2- 172.21.15.26:0/782061121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c40040a0 0x7fe0c40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0da7fc640 1 --2- 172.21.15.26:0/782061121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c4004d10 0x7fe0c4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc160b40 con 0x7fe0c40040a0 2023-12-19T16:57:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c400bb20 con 0x7fe0c4004d10 2023-12-19T16:57:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c400bcb0 con 0x7fe0dc0d0480 2023-12-19T16:57:40.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0daffd640 1 --2- 172.21.15.26:0/782061121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c40040a0 0x7fe0c40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0db7fe640 1 --2- 172.21.15.26:0/782061121 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc0d0480 0x7fe0c4003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0dbfff640 1 --2- 172.21.15.26:0/782061121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c4004d10 0x7fe0c4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0daffd640 1 -- 172.21.15.26:0/782061121 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc0d0480 msgr2=0x7fe0c4003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0daffd640 1 --2- 172.21.15.26:0/782061121 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc0d0480 0x7fe0c4003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0daffd640 1 -- 172.21.15.26:0/782061121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c4004d10 msgr2=0x7fe0c4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0daffd640 1 --2- 172.21.15.26:0/782061121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0c4004d10 0x7fe0c4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0daffd640 1 -- 172.21.15.26:0/782061121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c4003530 con 0x7fe0c40040a0 2023-12-19T16:57:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0daffd640 1 --2- 172.21.15.26:0/782061121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c40040a0 0x7fe0c40047d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00018d0 tx=0x7fe0d001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456168ee66db6c8f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d00052d0 con 0x7fe0c40040a0 2023-12-19T16:57:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d0005450 con 0x7fe0c40040a0 2023-12-19T16:57:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.233+0000 7fe0da7fc640 1 -- 172.21.15.26:0/782061121 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d0003840 con 0x7fe0c40040a0 2023-12-19T16:57:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 -- 172.21.15.26:0/782061121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c40040a0 msgr2=0x7fe0c40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 --2- 172.21.15.26:0/782061121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0c40040a0 0x7fe0c40047d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00018d0 tx=0x7fe0d001c560 comp rx=0 tx=0).stop 2023-12-19T16:57:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0dbfff640 1 -- 172.21.15.26:0/782061121 reap_dead start 2023-12-19T16:57:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 -- 172.21.15.26:0/782061121 shutdown_connections 2023-12-19T16:57:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 -- 172.21.15.26:0/782061121 >> 172.21.15.26:0/782061121 conn(0x7fe0dc06d510 msgr2=0x7fe0dc06d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 -- 172.21.15.26:0/782061121 shutdown_connections 2023-12-19T16:57:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 -- 172.21.15.26:0/782061121 wait complete. 2023-12-19T16:57:40.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 Processor -- start 2023-12-19T16:57:40.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 -- start start 2023-12-19T16:57:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0c40040a0 0x7fe0dc0c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0c8280 0x7fe0dc0c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc159470 0x7fe0dc159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc15aea0 con 0x7fe0dc159470 2023-12-19T16:57:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc15a260 con 0x7fe0dc0c8280 2023-12-19T16:57:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.237+0000 7fe0e2125640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc15a500 con 0x7fe0c40040a0 2023-12-19T16:57:40.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc159470 0x7fe0dc159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc159470 0x7fe0dc159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44844/0 (socket says 172.21.15.26:44844) 2023-12-19T16:57:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0dbfff640 1 -- 172.21.15.26:0/1840607646 learned_addr learned my addr 172.21.15.26:0/1840607646 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0db7fe640 1 --2- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0c40040a0 0x7fe0dc0c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0daffd640 1 --2- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0c8280 0x7fe0dc0c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0dbfff640 1 -- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0c40040a0 msgr2=0x7fe0dc0c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0dbfff640 1 --2- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0c40040a0 0x7fe0dc0c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0dbfff640 1 -- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0c8280 msgr2=0x7fe0dc0c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0dbfff640 1 --2- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0c8280 0x7fe0dc0c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0dbfff640 1 -- 172.21.15.26:0/1840607646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc15a7e0 con 0x7fe0dc159470 2023-12-19T16:57:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0dbfff640 1 --2- 172.21.15.26:0/1840607646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc159470 0x7fe0dc159900 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0b3610 tx=0x7fe0d400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0c3fff640 1 -- 172.21.15.26:0/1840607646 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400e040 con 0x7fe0dc159470 2023-12-19T16:57:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0c3fff640 1 -- 172.21.15.26:0/1840607646 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d400f040 con 0x7fe0dc159470 2023-12-19T16:57:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0c3fff640 1 -- 172.21.15.26:0/1840607646 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4015430 con 0x7fe0dc159470 2023-12-19T16:57:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0e2125640 1 -- 172.21.15.26:0/1840607646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc15aac0 con 0x7fe0dc159470 2023-12-19T16:57:40.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0e2125640 1 -- 172.21.15.26:0/1840607646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc15b870 con 0x7fe0dc159470 2023-12-19T16:57:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0e2125640 1 -- 172.21.15.26:0/1840607646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc15afe0 con 0x7fe0dc159470 2023-12-19T16:57:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.241+0000 7fe0c3fff640 1 -- 172.21.15.26:0/1840607646 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fe0d4015590 con 0x7fe0dc159470 2023-12-19T16:57:40.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.245+0000 7fe0c3fff640 1 --2- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe0c8065bb0 0x7fe0c8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.245+0000 7fe0c3fff640 1 -- 172.21.15.26:0/1840607646 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fe0d4084060 con 0x7fe0dc159470 2023-12-19T16:57:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.245+0000 7fe0db7fe640 1 -- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe0c8065bb0 msgr2=0x7fe0c8068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:40.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.245+0000 7fe0db7fe640 1 --2- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe0c8065bb0 0x7fe0c8068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:40.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.249+0000 7fe0c3fff640 1 -- 172.21.15.26:0/1840607646 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe0dc15afe0 con 0x7fe0dc159470 2023-12-19T16:57:40.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.405+0000 7fe0e2125640 1 -- 172.21.15.26:0/1840607646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0dc15afe0 con 0x7fe0dc159470 2023-12-19T16:57:40.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.409+0000 7fe0c3fff640 1 -- 172.21.15.26:0/1840607646 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17699 (secure 0 0 0) 0x7fe0dc15afe0 con 0x7fe0dc159470 2023-12-19T16:57:40.413 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:40.414 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":80,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:35.339995+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:20.751846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:22.096742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6833/38098":"2023-12-20T16:57:35.339958+0000","172.21.15.160:6832/38098":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/3906095528":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/2024777587":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/936215346":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/704170657":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/2914461989":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/2579181328":"2023-12-20T16:57:35.339958+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.409+0000 7fe0e2125640 1 -- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe0c8065bb0 msgr2=0x7fe0c8068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.409+0000 7fe0e2125640 1 --2- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe0c8065bb0 0x7fe0c8068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.409+0000 7fe0e2125640 1 -- 172.21.15.26:0/1840607646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc159470 msgr2=0x7fe0dc159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.409+0000 7fe0e2125640 1 --2- 172.21.15.26:0/1840607646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc159470 0x7fe0dc159900 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0b3610 tx=0x7fe0d400f580 comp rx=0 tx=0).stop 2023-12-19T16:57:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.409+0000 7fe0e2125640 1 -- 172.21.15.26:0/1840607646 shutdown_connections 2023-12-19T16:57:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.409+0000 7fe0e2125640 1 --2- 172.21.15.26:0/1840607646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc159470 0x7fe0dc159900 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.409+0000 7fe0e2125640 1 --2- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0c8280 0x7fe0dc0c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.409+0000 7fe0e2125640 1 --2- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe0c8065bb0 0x7fe0c8068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.409+0000 7fe0e2125640 1 --2- 172.21.15.26:0/1840607646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0c40040a0 0x7fe0dc0c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.409+0000 7fe0e2125640 1 -- 172.21.15.26:0/1840607646 >> 172.21.15.26:0/1840607646 conn(0x7fe0dc06d510 msgr2=0x7fe0dc15cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.409+0000 7fe0e2125640 1 -- 172.21.15.26:0/1840607646 shutdown_connections 2023-12-19T16:57:40.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.409+0000 7fe0e2125640 1 -- 172.21.15.26:0/1840607646 wait complete. 2023-12-19T16:57:40.435 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:57:20.751846+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:57:22.096742+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:57:40.436 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:57:40.542 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:40.539+0000 7f79d6506180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:57:40.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1d0f39640 1 Processor -- start 2023-12-19T16:57:40.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1d0f39640 1 -- start start 2023-12-19T16:57:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1d0f39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1cc15cd20 0x7fb1cc0b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1d0f39640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb1cc0b89c0 0x7fb1cc0b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1d0f39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1cc0b5c00 con 0x7fb1cc0b4c00 2023-12-19T16:57:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1d0f39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1cc0b50b0 con 0x7fb1cc15cd20 2023-12-19T16:57:40.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1d0f39640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1cc15d440 con 0x7fb1cc0b89c0 2023-12-19T16:57:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1ca59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1cc15cd20 0x7fb1cc0b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1ca59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1cc15cd20 0x7fb1cc0b8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44854/0 (socket says 172.21.15.26:44854) 2023-12-19T16:57:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1ca59c640 1 -- 172.21.15.26:0/564827544 learned_addr learned my addr 172.21.15.26:0/564827544 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1c3fff640 1 --2- 172.21.15.26:0/564827544 >> v2:172.21.15.160:3300/0 conn(0x7fb1cc0b89c0 0x7fb1cc0b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1ca59c640 1 -- 172.21.15.26:0/564827544 >> v2:172.21.15.160:3300/0 conn(0x7fb1cc0b89c0 msgr2=0x7fb1cc0b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1ca59c640 1 --2- 172.21.15.26:0/564827544 >> v2:172.21.15.160:3300/0 conn(0x7fb1cc0b89c0 0x7fb1cc0b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1ca59c640 1 -- 172.21.15.26:0/564827544 >> v1:172.21.15.160:6789/0 conn(0x7fb1cc0b4c00 legacy=0x7fb1cc0b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.581+0000 7fb1ca59c640 1 -- 172.21.15.26:0/564827544 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc15d6e0 con 0x7fb1cc15cd20 2023-12-19T16:57:40.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.585+0000 7fb1ca59c640 1 --2- 172.21.15.26:0/564827544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc15cd20 0x7fb1cc0b8480 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc08ba40 tx=0x7fb1bc01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53720e6c7c5b0d0e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:40.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.585+0000 7fb1c9d9b640 1 -- 172.21.15.26:0/564827544 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0a89f0 con 0x7fb1cc15cd20 2023-12-19T16:57:40.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.585+0000 7fb1c9d9b640 1 -- 172.21.15.26:0/564827544 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc0a8b50 con 0x7fb1cc15cd20 2023-12-19T16:57:40.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.585+0000 7fb1c9d9b640 1 -- 172.21.15.26:0/564827544 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc0b0440 con 0x7fb1cc15cd20 2023-12-19T16:57:40.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.585+0000 7fb1d0f39640 1 -- 172.21.15.26:0/564827544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc15cd20 msgr2=0x7fb1cc0b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.585+0000 7fb1d0f39640 1 --2- 172.21.15.26:0/564827544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc15cd20 0x7fb1cc0b8480 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc08ba40 tx=0x7fb1bc01ecf0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.585+0000 7fb1d0f39640 1 -- 172.21.15.26:0/564827544 shutdown_connections 2023-12-19T16:57:40.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.585+0000 7fb1d0f39640 1 --2- 172.21.15.26:0/564827544 >> v2:172.21.15.160:3300/0 conn(0x7fb1cc0b89c0 0x7fb1cc0b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.585+0000 7fb1d0f39640 1 --2- 172.21.15.26:0/564827544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc15cd20 0x7fb1cc0b8480 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.585+0000 7fb1d0f39640 1 -- 172.21.15.26:0/564827544 >> 172.21.15.26:0/564827544 conn(0x7fb1cc06a120 msgr2=0x7fb1cc06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:40.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.585+0000 7fb1d0f39640 1 -- 172.21.15.26:0/564827544 shutdown_connections 2023-12-19T16:57:40.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.585+0000 7fb1d0f39640 1 -- 172.21.15.26:0/564827544 wait complete. 2023-12-19T16:57:40.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1d0f39640 1 Processor -- start 2023-12-19T16:57:40.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1d0f39640 1 -- start start 2023-12-19T16:57:40.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1d0f39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1cc0b4c00 0x7fb1cc157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1d0f39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1cc0b89c0 0x7fb1cc1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1d0f39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc15cd20 0x7fb1cc1496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1d0f39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc149bf0 con 0x7fb1cc15cd20 2023-12-19T16:57:40.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1d0f39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc158040 con 0x7fb1cc0b89c0 2023-12-19T16:57:40.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1d0f39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc158320 con 0x7fb1cc0b4c00 2023-12-19T16:57:40.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1cad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc15cd20 0x7fb1cc1496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1cad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc15cd20 0x7fb1cc1496b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44866/0 (socket says 172.21.15.26:44866) 2023-12-19T16:57:40.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1cad9d640 1 -- 172.21.15.26:0/3464200341 learned_addr learned my addr 172.21.15.26:0/3464200341 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:40.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1c3fff640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1cc0b89c0 0x7fb1cc1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1ca59c640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1cc0b4c00 0x7fb1cc157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1cad9d640 1 -- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1cc0b4c00 msgr2=0x7fb1cc157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1cad9d640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1cc0b4c00 0x7fb1cc157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1cad9d640 1 -- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1cc0b89c0 msgr2=0x7fb1cc1590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1cad9d640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1cc0b89c0 0x7fb1cc1590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1cad9d640 1 -- 172.21.15.26:0/3464200341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc0094d0 con 0x7fb1cc15cd20 2023-12-19T16:57:40.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1ca59c640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1cc0b4c00 0x7fb1cc157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:40.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1c3fff640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1cc0b89c0 0x7fb1cc1590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:40.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1cad9d640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc15cd20 0x7fb1cc1496b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400c770 tx=0x7fb1c400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:40.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1c2ffd640 1 -- 172.21.15.26:0/3464200341 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4003b80 con 0x7fb1cc15cd20 2023-12-19T16:57:40.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1c2ffd640 1 -- 172.21.15.26:0/3464200341 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c4003ce0 con 0x7fb1cc15cd20 2023-12-19T16:57:40.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1c2ffd640 1 -- 172.21.15.26:0/3464200341 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c400aa90 con 0x7fb1cc15cd20 2023-12-19T16:57:40.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1d0f39640 1 -- 172.21.15.26:0/3464200341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1cc158600 con 0x7fb1cc15cd20 2023-12-19T16:57:40.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.589+0000 7fb1d0f39640 1 -- 172.21.15.26:0/3464200341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1cc158bc0 con 0x7fb1cc15cd20 2023-12-19T16:57:40.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.593+0000 7fb1d0f39640 1 -- 172.21.15.26:0/3464200341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1cc158de0 con 0x7fb1cc15cd20 2023-12-19T16:57:40.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.593+0000 7fb1c2ffd640 1 -- 172.21.15.26:0/3464200341 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fb1c400abf0 con 0x7fb1cc15cd20 2023-12-19T16:57:40.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.593+0000 7fb1c2ffd640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb1ac065b80 0x7fb1ac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.593+0000 7fb1c2ffd640 1 -- 172.21.15.26:0/3464200341 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fb1c4083550 con 0x7fb1cc15cd20 2023-12-19T16:57:40.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.593+0000 7fb1ca59c640 1 -- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb1ac065b80 msgr2=0x7fb1ac068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:40.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.597+0000 7fb1ca59c640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb1ac065b80 0x7fb1ac068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:40.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.597+0000 7fb1c2ffd640 1 -- 172.21.15.26:0/3464200341 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb1c4050950 con 0x7fb1cc15cd20 2023-12-19T16:57:40.630 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:40.623+0000 7f79d6506180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:57:40.754 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:40.749+0000 7ff967a0a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:57:40.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1d0f39640 1 -- 172.21.15.26:0/3464200341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1cc074270 con 0x7fb1cc15cd20 2023-12-19T16:57:40.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1c2ffd640 1 -- 172.21.15.26:0/3464200341 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1c4053f70 con 0x7fb1cc15cd20 2023-12-19T16:57:40.765 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:57:40.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1d0f39640 1 -- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb1ac065b80 msgr2=0x7fb1ac068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:40.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1d0f39640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb1ac065b80 0x7fb1ac068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1d0f39640 1 -- 172.21.15.26:0/3464200341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc15cd20 msgr2=0x7fb1cc1496b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1d0f39640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc15cd20 0x7fb1cc1496b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400c770 tx=0x7fb1c400cc30 comp rx=0 tx=0).stop 2023-12-19T16:57:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1d0f39640 1 -- 172.21.15.26:0/3464200341 shutdown_connections 2023-12-19T16:57:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1d0f39640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1cc15cd20 0x7fb1cc1496b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1d0f39640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1cc0b89c0 0x7fb1cc1590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1d0f39640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb1ac065b80 0x7fb1ac068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1d0f39640 1 --2- 172.21.15.26:0/3464200341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1cc0b4c00 0x7fb1cc157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1d0f39640 1 -- 172.21.15.26:0/3464200341 >> 172.21.15.26:0/3464200341 conn(0x7fb1cc06a120 msgr2=0x7fb1cc160820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1d0f39640 1 -- 172.21.15.26:0/3464200341 shutdown_connections 2023-12-19T16:57:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.761+0000 7fb1d0f39640 1 -- 172.21.15.26:0/3464200341 wait complete. 2023-12-19T16:57:40.778 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:40.775+0000 7f79d6506180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:57:40.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:57:40.845 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:40.841+0000 7ff967a0a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:57:40.863 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:40.859+0000 7f79d6506180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:57:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d8d39640 1 Processor -- start 2023-12-19T16:57:40.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d8d39640 1 -- start start 2023-12-19T16:57:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d8d39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24d415d0d0 0x7f24d40b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d8d39640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f24d40b4600 0x7f24d40b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d8d39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d40b4f90 con 0x7f24d415ccf0 2023-12-19T16:57:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d8d39640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d415e200 con 0x7f24d40b4600 2023-12-19T16:57:40.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d8d39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d40d0aa0 con 0x7f24d415d0d0 2023-12-19T16:57:40.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d2d9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f24d40b4600 0x7f24d40b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d2d9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f24d40b4600 0x7f24d40b4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:48110/0 (socket says 172.21.15.26:48110) 2023-12-19T16:57:40.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d259c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f24d415ccf0 0x7f24d40d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:33792/0 (socket says 172.21.15.26:33792) 2023-12-19T16:57:40.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d2d9d640 1 -- 172.21.15.26:0/2421395342 learned_addr learned my addr 172.21.15.26:0/2421395342 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:40.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d1d9b640 1 --2- 172.21.15.26:0/2421395342 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24d415d0d0 0x7f24d40b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d2d9d640 1 -- 172.21.15.26:0/2421395342 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24d415d0d0 msgr2=0x7f24d40b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:40.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d2d9d640 1 --2- 172.21.15.26:0/2421395342 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24d415d0d0 0x7f24d40b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d2d9d640 1 -- 172.21.15.26:0/2421395342 >> v1:172.21.15.160:6789/0 conn(0x7f24d415ccf0 legacy=0x7f24d40d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d2d9d640 1 -- 172.21.15.26:0/2421395342 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d415d680 con 0x7f24d40b4600 2023-12-19T16:57:40.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.933+0000 7f24d159a640 1 -- 172.21.15.26:0/2421395342 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786145032 0 0) 0x7f24c4001120 con 0x7f24d415ccf0 2023-12-19T16:57:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.937+0000 7f24d2d9d640 1 --2- 172.21.15.26:0/2421395342 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24d40b4600 0x7f24d40b4a50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009e50 tx=0x7f24cc01c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95ee085425140b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.937+0000 7f24d159a640 1 -- 172.21.15.26:0/2421395342 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc0a58a0 con 0x7f24d40b4600 2023-12-19T16:57:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.937+0000 7f24d159a640 1 -- 172.21.15.26:0/2421395342 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc0a5a00 con 0x7f24d40b4600 2023-12-19T16:57:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.937+0000 7f24d159a640 1 -- 172.21.15.26:0/2421395342 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc0a5d90 con 0x7f24d40b4600 2023-12-19T16:57:40.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.937+0000 7f24d8d39640 1 -- 172.21.15.26:0/2421395342 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24d40b4600 msgr2=0x7f24d40b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.937+0000 7f24d8d39640 1 --2- 172.21.15.26:0/2421395342 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24d40b4600 0x7f24d40b4a50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009e50 tx=0x7f24cc01c9e0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.937+0000 7f24d8d39640 1 -- 172.21.15.26:0/2421395342 shutdown_connections 2023-12-19T16:57:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.937+0000 7f24d8d39640 1 --2- 172.21.15.26:0/2421395342 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24d40b4600 0x7f24d40b4a50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.937+0000 7f24d8d39640 1 --2- 172.21.15.26:0/2421395342 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24d415d0d0 0x7f24d40b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.937+0000 7f24d8d39640 1 -- 172.21.15.26:0/2421395342 >> 172.21.15.26:0/2421395342 conn(0x7f24d4065ee0 msgr2=0x7f24d40cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:40.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.937+0000 7f24d8d39640 1 -- 172.21.15.26:0/2421395342 shutdown_connections 2023-12-19T16:57:40.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.937+0000 7f24d8d39640 1 -- 172.21.15.26:0/2421395342 wait complete. 2023-12-19T16:57:40.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.941+0000 7f24d8d39640 1 Processor -- start 2023-12-19T16:57:40.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.941+0000 7f24d8d39640 1 -- start start 2023-12-19T16:57:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.941+0000 7f24d8d39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24d40b4600 0x7f24d4153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.941+0000 7f24d8d39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24d415ccf0 0x7f24d40c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.941+0000 7f24d8d39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24d415d0d0 0x7f24d415f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.941+0000 7f24d8d39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d40c8c10 con 0x7f24d415ccf0 2023-12-19T16:57:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.941+0000 7f24d8d39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d40c9030 con 0x7f24d415d0d0 2023-12-19T16:57:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.941+0000 7f24d8d39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d40c7f90 con 0x7f24d40b4600 2023-12-19T16:57:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24d415ccf0 0x7f24d40c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d1d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24d415ccf0 0x7f24d40c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44882/0 (socket says 172.21.15.26:44882) 2023-12-19T16:57:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d1d9b640 1 -- 172.21.15.26:0/4289404141 learned_addr learned my addr 172.21.15.26:0/4289404141 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d259c640 1 --2- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24d40b4600 0x7f24d4153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d2d9d640 1 --2- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24d415d0d0 0x7f24d415f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d1d9b640 1 -- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24d40b4600 msgr2=0x7f24d4153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d1d9b640 1 --2- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24d40b4600 0x7f24d4153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d1d9b640 1 -- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24d415d0d0 msgr2=0x7f24d415f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d1d9b640 1 --2- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24d415d0d0 0x7f24d415f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:40.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d1d9b640 1 -- 172.21.15.26:0/4289404141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc009b30 con 0x7f24d415ccf0 2023-12-19T16:57:40.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d1d9b640 1 --2- 172.21.15.26:0/4289404141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24d415ccf0 0x7f24d40c6fe0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f24c800e870 tx=0x7f24c800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:40.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24c2ffd640 1 -- 172.21.15.26:0/4289404141 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8009510 con 0x7f24d415ccf0 2023-12-19T16:57:40.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24c2ffd640 1 -- 172.21.15.26:0/4289404141 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c800eec0 con 0x7f24d415ccf0 2023-12-19T16:57:40.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d8d39640 1 -- 172.21.15.26:0/4289404141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d40c8270 con 0x7f24d415ccf0 2023-12-19T16:57:40.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24c2ffd640 1 -- 172.21.15.26:0/4289404141 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c800cb40 con 0x7f24d415ccf0 2023-12-19T16:57:40.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d8d39640 1 -- 172.21.15.26:0/4289404141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d40c8830 con 0x7f24d415ccf0 2023-12-19T16:57:40.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24d8d39640 1 -- 172.21.15.26:0/4289404141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d40c8e80 con 0x7f24d415ccf0 2023-12-19T16:57:40.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.945+0000 7f24c2ffd640 1 -- 172.21.15.26:0/4289404141 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f24c800cca0 con 0x7f24d415ccf0 2023-12-19T16:57:40.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.949+0000 7f24c2ffd640 1 --2- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f24b4065b80 0x7f24b4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:40.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.949+0000 7f24c2ffd640 1 -- 172.21.15.26:0/4289404141 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f24c8083f30 con 0x7f24d415ccf0 2023-12-19T16:57:40.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.949+0000 7f24d259c640 1 -- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f24b4065b80 msgr2=0x7f24b4068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:57:40.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.949+0000 7f24d259c640 1 --2- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f24b4065b80 0x7f24b4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:57:40.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:40.949+0000 7f24c2ffd640 1 -- 172.21.15.26:0/4289404141 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f24c80af0a0 con 0x7f24d415ccf0 2023-12-19T16:57:41.009 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:41.005+0000 7ff967a0a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:57:41.097 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:41.093+0000 7ff967a0a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:57:41.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.109+0000 7f24d8d39640 1 -- 172.21.15.26:0/4289404141 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f24d4074270 con 0x7f24d415ccf0 2023-12-19T16:57:41.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.109+0000 7f24c2ffd640 1 -- 172.21.15.26:0/4289404141 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f24c80514c0 con 0x7f24d415ccf0 2023-12-19T16:57:41.115 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:57:41.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.109+0000 7f24d8d39640 1 -- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f24b4065b80 msgr2=0x7f24b4068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:57:41.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.109+0000 7f24d8d39640 1 --2- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f24b4065b80 0x7f24b4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.109+0000 7f24d8d39640 1 -- 172.21.15.26:0/4289404141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24d415ccf0 msgr2=0x7f24d40c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:41.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.109+0000 7f24d8d39640 1 --2- 172.21.15.26:0/4289404141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24d415ccf0 0x7f24d40c6fe0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f24c800e870 tx=0x7f24c800ec70 comp rx=0 tx=0).stop 2023-12-19T16:57:41.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.113+0000 7f24d8d39640 1 -- 172.21.15.26:0/4289404141 shutdown_connections 2023-12-19T16:57:41.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.113+0000 7f24d8d39640 1 --2- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24d415d0d0 0x7f24d415f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.113+0000 7f24d8d39640 1 --2- 172.21.15.26:0/4289404141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24d415ccf0 0x7f24d40c6fe0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.113+0000 7f24d8d39640 1 --2- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f24b4065b80 0x7f24b4068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.113+0000 7f24d8d39640 1 --2- 172.21.15.26:0/4289404141 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24d40b4600 0x7f24d4153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.113+0000 7f24d8d39640 1 -- 172.21.15.26:0/4289404141 >> 172.21.15.26:0/4289404141 conn(0x7f24d4065ee0 msgr2=0x7f24d40b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:41.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.113+0000 7f24d8d39640 1 -- 172.21.15.26:0/4289404141 shutdown_connections 2023-12-19T16:57:41.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.113+0000 7f24d8d39640 1 -- 172.21.15.26:0/4289404141 wait complete. 2023-12-19T16:57:41.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:57:41.184 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:41.179+0000 7f79d6506180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:57:41.245 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:41.243+0000 7f79d6506180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:57:41.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd15ae83640 1 Processor -- start 2023-12-19T16:57:41.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd15ae83640 1 -- start start 2023-12-19T16:57:41.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd15ae83640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd154160a20 0x7fd154167d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd15ae83640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1540cdb20 con 0x7fd15415d070 2023-12-19T16:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd15ae83640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1540cde00 con 0x7fd15415cc90 2023-12-19T16:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd15ae83640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540ce0e0 con 0x7fd154160a20 2023-12-19T16:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd159680640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd15415d070 0x7fd1540cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38036/0 (socket says 172.21.15.26:38036) 2023-12-19T16:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd159680640 1 -- 172.21.15.26:0/3275117832 learned_addr learned my addr 172.21.15.26:0/3275117832 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd15a682640 1 --2- 172.21.15.26:0/3275117832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd154160a20 0x7fd154167d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858741915 0 0) 0x7fd1540cdb20 con 0x7fd15415d070 2023-12-19T16:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd140003530 con 0x7fd15415d070 2023-12-19T16:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1238088422 0 0) 0x7fd140003530 con 0x7fd15415d070 2023-12-19T16:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd154160a20 msgr2=0x7fd154167d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 --2- 172.21.15.26:0/3275117832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd154160a20 0x7fd154167d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 >> v1:172.21.15.160:6789/0 conn(0x7fd15415cc90 legacy=0x7fd1540cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1540ce3c0 con 0x7fd15415d070 2023-12-19T16:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd15a682640 1 --2- 172.21.15.26:0/3275117832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd154160a20 0x7fd154167d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fd144001680 con 0x7fd15415d070 2023-12-19T16:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 >> v1:172.21.15.26:6789/0 conn(0x7fd15415d070 legacy=0x7fd1540cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 --2- 172.21.15.26:0/3275117832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1400039b0 0x7fd140003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 --2- 172.21.15.26:0/3275117832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1400047d0 0x7fd140004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 --2- 172.21.15.26:0/3275117832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd154160a20 0x7fd140005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540ce0e0 con 0x7fd1400039b0 2023-12-19T16:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14000ba40 con 0x7fd1400047d0 2023-12-19T16:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14000bbb0 con 0x7fd154160a20 2023-12-19T16:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd159e81640 1 --2- 172.21.15.26:0/3275117832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1400039b0 0x7fd140003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd159680640 1 --2- 172.21.15.26:0/3275117832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1400047d0 0x7fd140004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:41.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd15a682640 1 --2- 172.21.15.26:0/3275117832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd154160a20 0x7fd140005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:41.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd159e81640 1 -- 172.21.15.26:0/3275117832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd154160a20 msgr2=0x7fd140005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:41.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd159e81640 1 --2- 172.21.15.26:0/3275117832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd154160a20 0x7fd140005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd159e81640 1 -- 172.21.15.26:0/3275117832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1400047d0 msgr2=0x7fd140004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:41.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd159e81640 1 --2- 172.21.15.26:0/3275117832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1400047d0 0x7fd140004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.285+0000 7fd159e81640 1 -- 172.21.15.26:0/3275117832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c009a70 con 0x7fd1400039b0 2023-12-19T16:57:41.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.289+0000 7fd159e81640 1 --2- 172.21.15.26:0/3275117832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1400039b0 0x7fd140003dd0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd150001750 tx=0x7fd15001c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9561e901488b83b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:41.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.289+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1500052d0 con 0x7fd1400039b0 2023-12-19T16:57:41.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.289+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1500a7050 con 0x7fd1400039b0 2023-12-19T16:57:41.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.289+0000 7fd158e7f640 1 -- 172.21.15.26:0/3275117832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd150003600 con 0x7fd1400039b0 2023-12-19T16:57:41.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.289+0000 7fd15ae83640 1 -- 172.21.15.26:0/3275117832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1400039b0 msgr2=0x7fd140003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.289+0000 7fd15ae83640 1 --2- 172.21.15.26:0/3275117832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1400039b0 0x7fd140003dd0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd150001750 tx=0x7fd15001c310 comp rx=0 tx=0).stop 2023-12-19T16:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.289+0000 7fd15a682640 1 -- 172.21.15.26:0/3275117832 reap_dead start 2023-12-19T16:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.289+0000 7fd15ae83640 1 -- 172.21.15.26:0/3275117832 shutdown_connections 2023-12-19T16:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.289+0000 7fd15ae83640 1 -- 172.21.15.26:0/3275117832 >> 172.21.15.26:0/3275117832 conn(0x7fd15406a320 msgr2=0x7fd1540658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.289+0000 7fd15ae83640 1 -- 172.21.15.26:0/3275117832 shutdown_connections 2023-12-19T16:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.289+0000 7fd15ae83640 1 -- 172.21.15.26:0/3275117832 wait complete. 2023-12-19T16:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd15ae83640 1 Processor -- start 2023-12-19T16:57:41.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd15ae83640 1 -- start start 2023-12-19T16:57:41.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd15ae83640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1400039b0 0x7fd15415a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:41.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd15ae83640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd15415b390 0x7fd1541f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:41.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd15ae83640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1540cad00 0x7fd1540b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:41.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd15ae83640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1541f6180 con 0x7fd15415b390 2023-12-19T16:57:41.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd15ae83640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1541f6460 con 0x7fd1400039b0 2023-12-19T16:57:41.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd15ae83640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1541f6740 con 0x7fd1540cad00 2023-12-19T16:57:41.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd159680640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd15415b390 0x7fd1541f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:41.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd159680640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd15415b390 0x7fd1541f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44892/0 (socket says 172.21.15.26:44892) 2023-12-19T16:57:41.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd159680640 1 -- 172.21.15.26:0/1930855067 learned_addr learned my addr 172.21.15.26:0/1930855067 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:41.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd15a682640 1 --2- 172.21.15.26:0/1930855067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1540cad00 0x7fd1540b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:41.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd159e81640 1 --2- 172.21.15.26:0/1930855067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1400039b0 0x7fd15415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd159680640 1 -- 172.21.15.26:0/1930855067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1540cad00 msgr2=0x7fd1540b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd159680640 1 --2- 172.21.15.26:0/1930855067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1540cad00 0x7fd1540b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd159680640 1 -- 172.21.15.26:0/1930855067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1400039b0 msgr2=0x7fd15415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd159680640 1 --2- 172.21.15.26:0/1930855067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1400039b0 0x7fd15415a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd159680640 1 -- 172.21.15.26:0/1930855067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd154164730 con 0x7fd15415b390 2023-12-19T16:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd159680640 1 --2- 172.21.15.26:0/1930855067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd15415b390 0x7fd1541f5770 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd14400e780 tx=0x7fd14400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:41.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd14a7fc640 1 -- 172.21.15.26:0/1930855067 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14400c6c0 con 0x7fd15415b390 2023-12-19T16:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd14a7fc640 1 -- 172.21.15.26:0/1930855067 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14400c860 con 0x7fd15415b390 2023-12-19T16:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd15ae83640 1 -- 172.21.15.26:0/1930855067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15415aa40 con 0x7fd15415b390 2023-12-19T16:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.293+0000 7fd14a7fc640 1 -- 172.21.15.26:0/1930855067 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14400cc10 con 0x7fd15415b390 2023-12-19T16:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.297+0000 7fd15ae83640 1 -- 172.21.15.26:0/1930855067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1541f5cb0 con 0x7fd15415b390 2023-12-19T16:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.297+0000 7fd15ae83640 1 -- 172.21.15.26:0/1930855067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd154164970 con 0x7fd15415b390 2023-12-19T16:57:41.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.297+0000 7fd14a7fc640 1 -- 172.21.15.26:0/1930855067 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 35) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fd14401d460 con 0x7fd15415b390 2023-12-19T16:57:41.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.297+0000 7fd14a7fc640 1 -- 172.21.15.26:0/1930855067 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fd144083090 con 0x7fd15415b390 2023-12-19T16:57:41.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.301+0000 7fd14a7fc640 1 -- 172.21.15.26:0/1930855067 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd154164970 con 0x7fd15415b390 2023-12-19T16:57:41.318 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:41.315+0000 7f795ec79640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:41.318 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:41.315+0000 7f795ec79640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:41.318 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:41.315+0000 7f795ec79640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:41.318 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:41.315+0000 7f795ec79640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:41.318 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:41.315+0000 7f795ec79640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:41.318 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:41.315+0000 7f7961c7f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:41.318 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:41.315+0000 7f7961c7f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:41.318 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:41.315+0000 7f7961c7f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:41.318 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:41.315+0000 7f7961c7f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:41.319 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:57:41.315+0000 7f7961c7f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:57:41.450 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:41.445+0000 7ff967a0a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:57:41.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.465+0000 7fd15ae83640 1 -- 172.21.15.26:0/1930855067 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd154061930 con 0x7fd15415b390 2023-12-19T16:57:41.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.465+0000 7fd14a7fc640 1 -- 172.21.15.26:0/1930855067 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd144050920 con 0x7fd15415b390 2023-12-19T16:57:41.472 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:57:41.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.469+0000 7fd15ae83640 1 -- 172.21.15.26:0/1930855067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd15415b390 msgr2=0x7fd1541f5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:41.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.469+0000 7fd15ae83640 1 --2- 172.21.15.26:0/1930855067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd15415b390 0x7fd1541f5770 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd14400e780 tx=0x7fd14400ec40 comp rx=0 tx=0).stop 2023-12-19T16:57:41.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.469+0000 7fd15ae83640 1 -- 172.21.15.26:0/1930855067 shutdown_connections 2023-12-19T16:57:41.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.469+0000 7fd15ae83640 1 --2- 172.21.15.26:0/1930855067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd1540cad00 0x7fd1540b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.469+0000 7fd15ae83640 1 --2- 172.21.15.26:0/1930855067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd15415b390 0x7fd1541f5770 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.469+0000 7fd15ae83640 1 --2- 172.21.15.26:0/1930855067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1400039b0 0x7fd15415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.469+0000 7fd15ae83640 1 -- 172.21.15.26:0/1930855067 >> 172.21.15.26:0/1930855067 conn(0x7fd15406a320 msgr2=0x7fd15415b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:41.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.469+0000 7fd15ae83640 1 -- 172.21.15.26:0/1930855067 shutdown_connections 2023-12-19T16:57:41.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.469+0000 7fd15ae83640 1 -- 172.21.15.26:0/1930855067 wait complete. 2023-12-19T16:57:41.494 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:57:41.515 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:57:41.509+0000 7ff967a0a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:57:41.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5332f25640 1 Processor -- start 2023-12-19T16:57:41.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5332f25640 1 -- start start 2023-12-19T16:57:41.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5332f25640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f532c154220 0x7f532c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:41.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5332f25640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f532c06b3d0 0x7f532c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:41.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5332f25640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532c0cec40 0x7f532c15d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:41.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5332f25640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c0cf0d0 con 0x7f532c154220 2023-12-19T16:57:41.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5332f25640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c06bc30 con 0x7f532c06b3d0 2023-12-19T16:57:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5332f25640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c0cc6e0 con 0x7f532c0cec40 2023-12-19T16:57:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5330cc1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f532c154220 0x7f532c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5330cc1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f532c154220 0x7f532c15aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44904/0 (socket says 172.21.15.26:44904) 2023-12-19T16:57:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5330cc1640 1 -- 172.21.15.26:0/1075178649 learned_addr learned my addr 172.21.15.26:0/1075178649 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f53314c2640 1 --2- 172.21.15.26:0/1075178649 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532c0cec40 0x7f532c15d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f532bfff640 1 --2- 172.21.15.26:0/1075178649 >> v2:172.21.15.160:3300/0 conn(0x7f532c06b3d0 0x7f532c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5330cc1640 1 -- 172.21.15.26:0/1075178649 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532c0cec40 msgr2=0x7f532c15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5330cc1640 1 --2- 172.21.15.26:0/1075178649 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532c0cec40 0x7f532c15d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5330cc1640 1 -- 172.21.15.26:0/1075178649 >> v2:172.21.15.160:3300/0 conn(0x7f532c06b3d0 msgr2=0x7f532c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5330cc1640 1 --2- 172.21.15.26:0/1075178649 >> v2:172.21.15.160:3300/0 conn(0x7f532c06b3d0 0x7f532c0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.633+0000 7f5330cc1640 1 -- 172.21.15.26:0/1075178649 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c15afc0 con 0x7f532c154220 2023-12-19T16:57:41.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.637+0000 7f5330cc1640 1 --2- 172.21.15.26:0/1075178649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f532c154220 0x7f532c15aa80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f532c0cf880 tx=0x7f532001c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06f649c00b1ab43 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:41.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.637+0000 7f532b7fe640 1 -- 172.21.15.26:0/1075178649 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53200ab600 con 0x7f532c154220 2023-12-19T16:57:41.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.637+0000 7f532b7fe640 1 -- 172.21.15.26:0/1075178649 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53200a7050 con 0x7f532c154220 2023-12-19T16:57:41.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.637+0000 7f532b7fe640 1 -- 172.21.15.26:0/1075178649 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53200ab980 con 0x7f532c154220 2023-12-19T16:57:41.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.637+0000 7f5332f25640 1 -- 172.21.15.26:0/1075178649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f532c154220 msgr2=0x7f532c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:41.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.637+0000 7f5332f25640 1 --2- 172.21.15.26:0/1075178649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f532c154220 0x7f532c15aa80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f532c0cf880 tx=0x7f532001c210 comp rx=0 tx=0).stop 2023-12-19T16:57:41.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.637+0000 7f5332f25640 1 -- 172.21.15.26:0/1075178649 shutdown_connections 2023-12-19T16:57:41.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.637+0000 7f5332f25640 1 --2- 172.21.15.26:0/1075178649 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532c0cec40 0x7f532c15d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.637+0000 7f5332f25640 1 --2- 172.21.15.26:0/1075178649 >> v2:172.21.15.160:3300/0 conn(0x7f532c06b3d0 0x7f532c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.637+0000 7f5332f25640 1 --2- 172.21.15.26:0/1075178649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f532c154220 0x7f532c15aa80 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.637+0000 7f5332f25640 1 -- 172.21.15.26:0/1075178649 >> 172.21.15.26:0/1075178649 conn(0x7f532c0658e0 msgr2=0x7f532c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:41.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.637+0000 7f5332f25640 1 -- 172.21.15.26:0/1075178649 shutdown_connections 2023-12-19T16:57:41.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.637+0000 7f5332f25640 1 -- 172.21.15.26:0/1075178649 wait complete. 2023-12-19T16:57:41.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f5332f25640 1 Processor -- start 2023-12-19T16:57:41.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f5332f25640 1 -- start start 2023-12-19T16:57:41.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f5332f25640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f532c06b3d0 0x7f532c06f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f5332f25640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532c0cec40 0x7f532c0cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f5332f25640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f532c154220 0x7f532c1f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f5332f25640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c1f46e0 con 0x7f532c154220 2023-12-19T16:57:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f5332f25640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c0701f0 con 0x7f532c06b3d0 2023-12-19T16:57:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f5332f25640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c06f950 con 0x7f532c0cec40 2023-12-19T16:57:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f53314c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f532c154220 0x7f532c1f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f53314c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f532c154220 0x7f532c1f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44910/0 (socket says 172.21.15.26:44910) 2023-12-19T16:57:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f53314c2640 1 -- 172.21.15.26:0/1443101178 learned_addr learned my addr 172.21.15.26:0/1443101178 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f5330cc1640 1 --2- 172.21.15.26:0/1443101178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f532c06b3d0 0x7f532c06f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f532bfff640 1 --2- 172.21.15.26:0/1443101178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532c0cec40 0x7f532c0cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f53314c2640 1 -- 172.21.15.26:0/1443101178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532c0cec40 msgr2=0x7f532c0cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f53314c2640 1 --2- 172.21.15.26:0/1443101178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532c0cec40 0x7f532c0cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f53314c2640 1 -- 172.21.15.26:0/1443101178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f532c06b3d0 msgr2=0x7f532c06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:41.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f53314c2640 1 --2- 172.21.15.26:0/1443101178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f532c06b3d0 0x7f532c06f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f53314c2640 1 -- 172.21.15.26:0/1443101178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53200094d0 con 0x7f532c154220 2023-12-19T16:57:41.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f53314c2640 1 --2- 172.21.15.26:0/1443101178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f532c154220 0x7f532c1f4b00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f532400efc0 tx=0x7f532400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:41.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f53297fa640 1 -- 172.21.15.26:0/1443101178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5324014480 con 0x7f532c154220 2023-12-19T16:57:41.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f53297fa640 1 -- 172.21.15.26:0/1443101178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532400f040 con 0x7f532c154220 2023-12-19T16:57:41.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f53297fa640 1 -- 172.21.15.26:0/1443101178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5324014820 con 0x7f532c154220 2023-12-19T16:57:41.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f5332f25640 1 -- 172.21.15.26:0/1443101178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c06fc30 con 0x7f532c154220 2023-12-19T16:57:41.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.641+0000 7f5332f25640 1 -- 172.21.15.26:0/1443101178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c0b2f10 con 0x7f532c154220 2023-12-19T16:57:41.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.645+0000 7f5332f25640 1 -- 172.21.15.26:0/1443101178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532c074270 con 0x7f532c154220 2023-12-19T16:57:41.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.645+0000 7f53297fa640 1 -- 172.21.15.26:0/1443101178 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 35) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f5324014980 con 0x7f532c154220 2023-12-19T16:57:41.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.645+0000 7f53297fa640 1 -- 172.21.15.26:0/1443101178 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f5324084000 con 0x7f532c154220 2023-12-19T16:57:41.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.649+0000 7f53297fa640 1 -- 172.21.15.26:0/1443101178 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f53240518d0 con 0x7f532c154220 2023-12-19T16:57:41.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.837+0000 7f5332f25640 1 -- 172.21.15.26:0/1443101178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f532c0cab90 con 0x7f532c154220 2023-12-19T16:57:41.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.837+0000 7f53297fa640 1 -- 172.21.15.26:0/1443101178 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f5324018090 con 0x7f532c154220 2023-12-19T16:57:41.844 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:41.844 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/436365775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":436365775},{"type":"v1","addr":"172.21.15.160:6836","nonce":436365775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3014042376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3014042376},{"type":"v1","addr":"172.21.15.160:6837","nonce":3014042376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1357342534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1357342534},{"type":"v1","addr":"172.21.15.26:6835","nonce":1357342534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:57:23.100695+0000","modified":"2023-12-19T16:57:26.118941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6522":{"gid":6522,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/325854666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":325854666},{"type":"v1","addr":"172.21.15.26:6834","nonce":325854666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-19T16:57:41.844 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 33 2023-12-19T16:57:41.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.841+0000 7f5332f25640 1 -- 172.21.15.26:0/1443101178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f532c154220 msgr2=0x7f532c1f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:41.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.841+0000 7f5332f25640 1 --2- 172.21.15.26:0/1443101178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f532c154220 0x7f532c1f4b00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f532400efc0 tx=0x7f532400c490 comp rx=0 tx=0).stop 2023-12-19T16:57:41.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.841+0000 7f5332f25640 1 -- 172.21.15.26:0/1443101178 shutdown_connections 2023-12-19T16:57:41.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.841+0000 7f5332f25640 1 --2- 172.21.15.26:0/1443101178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f532c154220 0x7f532c1f4b00 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.841+0000 7f5332f25640 1 --2- 172.21.15.26:0/1443101178 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f532c0cec40 0x7f532c0cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.841+0000 7f5332f25640 1 --2- 172.21.15.26:0/1443101178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f532c06b3d0 0x7f532c06f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:41.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.841+0000 7f5332f25640 1 -- 172.21.15.26:0/1443101178 >> 172.21.15.26:0/1443101178 conn(0x7f532c0658e0 msgr2=0x7f532c065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:41.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.841+0000 7f5332f25640 1 -- 172.21.15.26:0/1443101178 shutdown_connections 2023-12-19T16:57:41.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:41.841+0000 7f5332f25640 1 -- 172.21.15.26:0/1443101178 wait complete. 2023-12-19T16:57:41.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.045+0000 7f4f51939640 1 Processor -- start 2023-12-19T16:57:42.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.045+0000 7f4f51939640 1 -- start start 2023-12-19T16:57:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.045+0000 7f4f51939640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f4c0cee50 0x7f4f4c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.045+0000 7f4f51939640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4c06c970 con 0x7f4f4c155030 2023-12-19T16:57:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.045+0000 7f4f51939640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f4c15b1f0 con 0x7f4f4c15f370 2023-12-19T16:57:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.045+0000 7f4f51939640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c06c790 con 0x7f4f4c0cee50 2023-12-19T16:57:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.045+0000 7f4f4affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4f4c155030 0x7f4f4c06c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38040/0 (socket says 172.21.15.26:38040) 2023-12-19T16:57:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.045+0000 7f4f4affd640 1 -- 172.21.15.26:0/1861509374 learned_addr learned my addr 172.21.15.26:0/1861509374 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.045+0000 7f4f4b7fe640 1 --2- 172.21.15.26:0/1861509374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f4c0cee50 0x7f4f4c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.049+0000 7f4f4a7fc640 1 -- 172.21.15.26:0/1861509374 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189660716 0 0) 0x7f4f4c06c970 con 0x7f4f4c155030 2023-12-19T16:57:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.049+0000 7f4f4a7fc640 1 -- 172.21.15.26:0/1861509374 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f38003530 con 0x7f4f4c155030 2023-12-19T16:57:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.049+0000 7f4f4a7fc640 1 -- 172.21.15.26:0/1861509374 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669355217 0 0) 0x7f4f4c15b1f0 con 0x7f4f4c15f370 2023-12-19T16:57:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.049+0000 7f4f4a7fc640 1 -- 172.21.15.26:0/1861509374 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f4c06c970 con 0x7f4f4c15f370 2023-12-19T16:57:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.049+0000 7f4f4b7fe640 1 -- 172.21.15.26:0/1861509374 >> v1:172.21.15.160:6789/0 conn(0x7f4f4c15f370 legacy=0x7f4f4c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.049+0000 7f4f4b7fe640 1 -- 172.21.15.26:0/1861509374 >> v1:172.21.15.26:6789/0 conn(0x7f4f4c155030 legacy=0x7f4f4c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.049+0000 7f4f4b7fe640 1 -- 172.21.15.26:0/1861509374 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c15b4d0 con 0x7f4f4c0cee50 2023-12-19T16:57:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.049+0000 7f4f4b7fe640 1 --2- 172.21.15.26:0/1861509374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f4c0cee50 0x7f4f4c15bcd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009dc0 tx=0x7f4f4401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b3522446de74fb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.049+0000 7f4f4a7fc640 1 -- 172.21.15.26:0/1861509374 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440a5a20 con 0x7f4f4c0cee50 2023-12-19T16:57:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.049+0000 7f4f4a7fc640 1 -- 172.21.15.26:0/1861509374 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f440a5b80 con 0x7f4f4c0cee50 2023-12-19T16:57:42.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.049+0000 7f4f4a7fc640 1 -- 172.21.15.26:0/1861509374 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440ad440 con 0x7f4f4c0cee50 2023-12-19T16:57:42.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 -- 172.21.15.26:0/1861509374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f4c0cee50 msgr2=0x7f4f4c15bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 --2- 172.21.15.26:0/1861509374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f4c0cee50 0x7f4f4c15bcd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009dc0 tx=0x7f4f4401c940 comp rx=0 tx=0).stop 2023-12-19T16:57:42.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 -- 172.21.15.26:0/1861509374 shutdown_connections 2023-12-19T16:57:42.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 --2- 172.21.15.26:0/1861509374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f4c0cee50 0x7f4f4c15bcd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 -- 172.21.15.26:0/1861509374 >> 172.21.15.26:0/1861509374 conn(0x7f4f4c0658e0 msgr2=0x7f4f4c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:42.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 -- 172.21.15.26:0/1861509374 shutdown_connections 2023-12-19T16:57:42.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 -- 172.21.15.26:0/1861509374 wait complete. 2023-12-19T16:57:42.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 Processor -- start 2023-12-19T16:57:42.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 -- start start 2023-12-19T16:57:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f4c0cee50 0x7f4f4c15de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f4c155030 0x7f4f4c0b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f4c15f370 0x7f4f4c150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c150150 con 0x7f4f4c0cee50 2023-12-19T16:57:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0b1310 con 0x7f4f4c155030 2023-12-19T16:57:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f51939640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4c0b15f0 con 0x7f4f4c15f370 2023-12-19T16:57:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f4affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f4c0cee50 0x7f4f4c15de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f4affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f4c0cee50 0x7f4f4c15de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44912/0 (socket says 172.21.15.26:44912) 2023-12-19T16:57:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f4affd640 1 -- 172.21.15.26:0/3460300418 learned_addr learned my addr 172.21.15.26:0/3460300418 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f4b7fe640 1 --2- 172.21.15.26:0/3460300418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f4c15f370 0x7f4f4c150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f43fff640 1 --2- 172.21.15.26:0/3460300418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f4c155030 0x7f4f4c0b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f4affd640 1 -- 172.21.15.26:0/3460300418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f4c15f370 msgr2=0x7f4f4c150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f4affd640 1 --2- 172.21.15.26:0/3460300418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f4c15f370 0x7f4f4c150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f4affd640 1 -- 172.21.15.26:0/3460300418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f4c155030 msgr2=0x7f4f4c0b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f4affd640 1 --2- 172.21.15.26:0/3460300418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f4c155030 0x7f4f4c0b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f4affd640 1 -- 172.21.15.26:0/3460300418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f44009a70 con 0x7f4f4c0cee50 2023-12-19T16:57:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f4b7fe640 1 --2- 172.21.15.26:0/3460300418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f4c15f370 0x7f4f4c150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f43fff640 1 --2- 172.21.15.26:0/3460300418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f4c155030 0x7f4f4c0b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.053+0000 7f4f4affd640 1 --2- 172.21.15.26:0/3460300418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f4c0cee50 0x7f4f4c15de10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00c4f0 tx=0x7f4f3c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.057+0000 7f4f48ff9640 1 -- 172.21.15.26:0/3460300418 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c005cf0 con 0x7f4f4c0cee50 2023-12-19T16:57:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.057+0000 7f4f48ff9640 1 -- 172.21.15.26:0/3460300418 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c005e50 con 0x7f4f4c0cee50 2023-12-19T16:57:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.057+0000 7f4f51939640 1 -- 172.21.15.26:0/3460300418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4c0b18d0 con 0x7f4f4c0cee50 2023-12-19T16:57:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.057+0000 7f4f48ff9640 1 -- 172.21.15.26:0/3460300418 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c007270 con 0x7f4f4c0cee50 2023-12-19T16:57:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.057+0000 7f4f51939640 1 -- 172.21.15.26:0/3460300418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f4c0b1e90 con 0x7f4f4c0cee50 2023-12-19T16:57:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.057+0000 7f4f51939640 1 -- 172.21.15.26:0/3460300418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4c0b2080 con 0x7f4f4c0cee50 2023-12-19T16:57:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.057+0000 7f4f48ff9640 1 -- 172.21.15.26:0/3460300418 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 35) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f4f3c0073d0 con 0x7f4f4c0cee50 2023-12-19T16:57:42.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.057+0000 7f4f48ff9640 1 -- 172.21.15.26:0/3460300418 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f4f3c0826c0 con 0x7f4f4c0cee50 2023-12-19T16:57:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.061+0000 7f4f48ff9640 1 -- 172.21.15.26:0/3460300418 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4f3c050040 con 0x7f4f4c0cee50 2023-12-19T16:57:42.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.221+0000 7f4f51939640 1 -- 172.21.15.26:0/3460300418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f4c074270 con 0x7f4f4c0cee50 2023-12-19T16:57:42.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.221+0000 7f4f48ff9640 1 -- 172.21.15.26:0/3460300418 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18074 (secure 0 0 0) 0x7f4f3c017090 con 0x7f4f4c0cee50 2023-12-19T16:57:42.226 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:42.227 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":81,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:41.248623+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:20.751846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:22.096742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3016063689":"2023-12-20T16:57:41.248587+0000","172.21.15.160:0/537357214":"2023-12-20T16:57:41.248587+0000","172.21.15.160:0/932487207":"2023-12-20T16:57:41.248587+0000","172.21.15.160:0/3937793296":"2023-12-20T16:57:41.248587+0000","172.21.15.160:0/3913339488":"2023-12-20T16:57:41.248587+0000","172.21.15.160:0/374927106":"2023-12-20T16:57:41.248587+0000","172.21.15.160:6833/38098":"2023-12-20T16:57:35.339958+0000","172.21.15.160:6832/38098":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/3906095528":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/2024777587":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/936215346":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/704170657":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/2914461989":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/2579181328":"2023-12-20T16:57:35.339958+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:42.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.225+0000 7f4f51939640 1 -- 172.21.15.26:0/3460300418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f4c0cee50 msgr2=0x7f4f4c15de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.225+0000 7f4f51939640 1 --2- 172.21.15.26:0/3460300418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f4c0cee50 0x7f4f4c15de10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00c4f0 tx=0x7f4f3c00c9b0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.225+0000 7f4f51939640 1 -- 172.21.15.26:0/3460300418 shutdown_connections 2023-12-19T16:57:42.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.225+0000 7f4f51939640 1 --2- 172.21.15.26:0/3460300418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f4c15f370 0x7f4f4c150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.225+0000 7f4f51939640 1 --2- 172.21.15.26:0/3460300418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f4c155030 0x7f4f4c0b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.225+0000 7f4f51939640 1 --2- 172.21.15.26:0/3460300418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f4c0cee50 0x7f4f4c15de10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.225+0000 7f4f51939640 1 -- 172.21.15.26:0/3460300418 >> 172.21.15.26:0/3460300418 conn(0x7f4f4c0658e0 msgr2=0x7f4f4c065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:42.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.225+0000 7f4f51939640 1 -- 172.21.15.26:0/3460300418 shutdown_connections 2023-12-19T16:57:42.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.225+0000 7f4f51939640 1 -- 172.21.15.26:0/3460300418 wait complete. 2023-12-19T16:57:42.248 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T16:57:42.248 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:57:42.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.389+0000 7fc043b25640 1 Processor -- start 2023-12-19T16:57:42.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc043b25640 1 -- start start 2023-12-19T16:57:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc043b25640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc03c154220 0x7fc03c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc043b25640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc03c06b3d0 0x7fc03c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc043b25640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc03c06bc30 con 0x7fc03c0cec40 2023-12-19T16:57:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc043b25640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03c0cf0d0 con 0x7fc03c154220 2023-12-19T16:57:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc043b25640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0cc6e0 con 0x7fc03c06b3d0 2023-12-19T16:57:42.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc0420c2640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc03c0cec40 0x7fc03c15d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38042/0 (socket says 172.21.15.26:38042) 2023-12-19T16:57:42.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc0420c2640 1 -- 172.21.15.26:0/3536082850 learned_addr learned my addr 172.21.15.26:0/3536082850 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:42.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc0418c1640 1 --2- 172.21.15.26:0/3536082850 >> v2:172.21.15.160:3300/0 conn(0x7fc03c154220 0x7fc03c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc0410c0640 1 --2- 172.21.15.26:0/3536082850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc03c06b3d0 0x7fc03c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc0408bf640 1 -- 172.21.15.26:0/3536082850 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336481306 0 0) 0x7fc03c06bc30 con 0x7fc03c0cec40 2023-12-19T16:57:42.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc0408bf640 1 -- 172.21.15.26:0/3536082850 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc028003530 con 0x7fc03c0cec40 2023-12-19T16:57:42.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc0408bf640 1 -- 172.21.15.26:0/3536082850 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1203029795 0 0) 0x7fc028003530 con 0x7fc03c0cec40 2023-12-19T16:57:42.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc0418c1640 1 -- 172.21.15.26:0/3536082850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc03c06b3d0 msgr2=0x7fc03c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:42.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc0418c1640 1 --2- 172.21.15.26:0/3536082850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc03c06b3d0 0x7fc03c0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc0418c1640 1 -- 172.21.15.26:0/3536082850 >> v1:172.21.15.26:6789/0 conn(0x7fc03c0cec40 legacy=0x7fc03c15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc0418c1640 1 -- 172.21.15.26:0/3536082850 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c15afc0 con 0x7fc03c154220 2023-12-19T16:57:42.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.393+0000 7fc0410c0640 1 --2- 172.21.15.26:0/3536082850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc03c06b3d0 0x7fc03c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.397+0000 7fc0418c1640 1 --2- 172.21.15.26:0/3536082850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c154220 0x7fc03c15aa80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc0380097c0 tx=0x7fc03801c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9052258940f695d4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.397+0000 7fc0408bf640 1 -- 172.21.15.26:0/3536082850 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc034001a00 con 0x7fc03c154220 2023-12-19T16:57:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.397+0000 7fc0408bf640 1 -- 172.21.15.26:0/3536082850 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0380a7050 con 0x7fc03c154220 2023-12-19T16:57:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.397+0000 7fc0408bf640 1 -- 172.21.15.26:0/3536082850 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0380ab620 con 0x7fc03c154220 2023-12-19T16:57:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.397+0000 7fc043b25640 1 -- 172.21.15.26:0/3536082850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c154220 msgr2=0x7fc03c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.397+0000 7fc043b25640 1 --2- 172.21.15.26:0/3536082850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c154220 0x7fc03c15aa80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc0380097c0 tx=0x7fc03801c370 comp rx=0 tx=0).stop 2023-12-19T16:57:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.397+0000 7fc043b25640 1 -- 172.21.15.26:0/3536082850 shutdown_connections 2023-12-19T16:57:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.397+0000 7fc043b25640 1 --2- 172.21.15.26:0/3536082850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc03c06b3d0 0x7fc03c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.397+0000 7fc043b25640 1 --2- 172.21.15.26:0/3536082850 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c154220 0x7fc03c15aa80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.397+0000 7fc043b25640 1 -- 172.21.15.26:0/3536082850 >> 172.21.15.26:0/3536082850 conn(0x7fc03c0658e0 msgr2=0x7fc03c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.397+0000 7fc043b25640 1 -- 172.21.15.26:0/3536082850 shutdown_connections 2023-12-19T16:57:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.397+0000 7fc043b25640 1 -- 172.21.15.26:0/3536082850 wait complete. 2023-12-19T16:57:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc043b25640 1 Processor -- start 2023-12-19T16:57:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc043b25640 1 -- start start 2023-12-19T16:57:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc043b25640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc03c06b3d0 0x7fc03c14e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc043b25640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc03c0cec40 0x7fc03c1f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc043b25640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c154220 0x7fc03c0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc043b25640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c06f370 con 0x7fc03c06b3d0 2023-12-19T16:57:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc043b25640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0b2ce0 con 0x7fc03c154220 2023-12-19T16:57:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc043b25640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c0b2fa0 con 0x7fc03c0cec40 2023-12-19T16:57:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0418c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc03c06b3d0 0x7fc03c14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0418c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc03c06b3d0 0x7fc03c14e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44916/0 (socket says 172.21.15.26:44916) 2023-12-19T16:57:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0418c1640 1 -- 172.21.15.26:0/247642424 learned_addr learned my addr 172.21.15.26:0/247642424 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0410c0640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc03c0cec40 0x7fc03c1f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0420c2640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c154220 0x7fc03c0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0418c1640 1 -- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc03c0cec40 msgr2=0x7fc03c1f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0418c1640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc03c0cec40 0x7fc03c1f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0418c1640 1 -- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c154220 msgr2=0x7fc03c0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0418c1640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c154220 0x7fc03c0706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0418c1640 1 -- 172.21.15.26:0/247642424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc02c009420 con 0x7fc03c06b3d0 2023-12-19T16:57:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0418c1640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc03c06b3d0 0x7fc03c14e610 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc03801cc10 tx=0x7fc038019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0327fc640 1 -- 172.21.15.26:0/247642424 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0380b9070 con 0x7fc03c06b3d0 2023-12-19T16:57:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0327fc640 1 -- 172.21.15.26:0/247642424 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0380abd10 con 0x7fc03c06b3d0 2023-12-19T16:57:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc0327fc640 1 -- 172.21.15.26:0/247642424 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0380b4660 con 0x7fc03c06b3d0 2023-12-19T16:57:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc043b25640 1 -- 172.21.15.26:0/247642424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc038009420 con 0x7fc03c06b3d0 2023-12-19T16:57:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc043b25640 1 -- 172.21.15.26:0/247642424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c0b3560 con 0x7fc03c06b3d0 2023-12-19T16:57:42.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.401+0000 7fc043b25640 1 -- 172.21.15.26:0/247642424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c074270 con 0x7fc03c06b3d0 2023-12-19T16:57:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.405+0000 7fc0327fc640 1 -- 172.21.15.26:0/247642424 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 36) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fc0380b2040 con 0x7fc03c06b3d0 2023-12-19T16:57:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.405+0000 7fc0327fc640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc0200658e0 0x7fc020067da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.405+0000 7fc0327fc640 1 -- 172.21.15.26:0/247642424 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fc03811c470 con 0x7fc03c06b3d0 2023-12-19T16:57:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.405+0000 7fc0410c0640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc0200658e0 0x7fc020067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.405+0000 7fc0410c0640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc0200658e0 0x7fc020067da0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0683b0 tx=0x7fc02c008040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.409+0000 7fc0327fc640 1 -- 172.21.15.26:0/247642424 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc0380e9c30 con 0x7fc03c06b3d0 2023-12-19T16:57:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.601+0000 7fc043b25640 1 -- 172.21.15.26:0/247642424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc03c0cab90 con 0x7fc03c06b3d0 2023-12-19T16:57:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.605+0000 7fc0327fc640 1 -- 172.21.15.26:0/247642424 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4709 (secure 0 0 0) 0x7fc0380ed250 con 0x7fc03c06b3d0 2023-12-19T16:57:42.609 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:42.609 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/436365775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":436365775},{"type":"v1","addr":"172.21.15.160:6836","nonce":436365775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3014042376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3014042376},{"type":"v1","addr":"172.21.15.160:6837","nonce":3014042376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1357342534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1357342534},{"type":"v1","addr":"172.21.15.26:6835","nonce":1357342534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:57:23.100695+0000","modified":"2023-12-19T16:57:26.118941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6522":{"gid":6522,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/325854666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":325854666},{"type":"v1","addr":"172.21.15.26:6834","nonce":325854666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-19T16:57:42.609 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 33 2023-12-19T16:57:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.605+0000 7fc043b25640 1 -- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc0200658e0 msgr2=0x7fc020067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.605+0000 7fc043b25640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc0200658e0 0x7fc020067da0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0683b0 tx=0x7fc02c008040 comp rx=0 tx=0).stop 2023-12-19T16:57:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.605+0000 7fc043b25640 1 -- 172.21.15.26:0/247642424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc03c06b3d0 msgr2=0x7fc03c14e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.605+0000 7fc043b25640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc03c06b3d0 0x7fc03c14e610 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc03801cc10 tx=0x7fc038019110 comp rx=0 tx=0).stop 2023-12-19T16:57:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.605+0000 7fc043b25640 1 -- 172.21.15.26:0/247642424 shutdown_connections 2023-12-19T16:57:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.605+0000 7fc043b25640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc03c154220 0x7fc03c0706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.605+0000 7fc043b25640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc0200658e0 0x7fc020067da0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.605+0000 7fc043b25640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc03c0cec40 0x7fc03c1f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.605+0000 7fc043b25640 1 --2- 172.21.15.26:0/247642424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc03c06b3d0 0x7fc03c14e610 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.605+0000 7fc043b25640 1 -- 172.21.15.26:0/247642424 >> 172.21.15.26:0/247642424 conn(0x7fc03c0658e0 msgr2=0x7fc03c0adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.605+0000 7fc043b25640 1 -- 172.21.15.26:0/247642424 shutdown_connections 2023-12-19T16:57:42.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.605+0000 7fc043b25640 1 -- 172.21.15.26:0/247642424 wait complete. 2023-12-19T16:57:42.630 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:57:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.769+0000 7f729ac20640 1 Processor -- start 2023-12-19T16:57:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.769+0000 7f729ac20640 1 -- start start 2023-12-19T16:57:42.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.769+0000 7f729ac20640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f729415c7f0 0x7f729406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.769+0000 7f729ac20640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729406ced0 con 0x7f72940adb80 2023-12-19T16:57:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.769+0000 7f729ac20640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729406d1b0 con 0x7f72940b30d0 2023-12-19T16:57:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.769+0000 7f729ac20640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f729406d770 con 0x7f729415c7f0 2023-12-19T16:57:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.769+0000 7f7293fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f72940adb80 0x7f72940cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38046/0 (socket says 172.21.15.26:38046) 2023-12-19T16:57:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.769+0000 7f7293fff640 1 -- 172.21.15.26:0/1555302403 learned_addr learned my addr 172.21.15.26:0/1555302403 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.769+0000 7f72989bc640 1 --2- 172.21.15.26:0/1555302403 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f729415c7f0 0x7f729406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.769+0000 7f72937fe640 1 -- 172.21.15.26:0/1555302403 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511478788 0 0) 0x7f729406ced0 con 0x7f72940adb80 2023-12-19T16:57:42.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f72937fe640 1 -- 172.21.15.26:0/1555302403 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7284003530 con 0x7f72940adb80 2023-12-19T16:57:42.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f72937fe640 1 -- 172.21.15.26:0/1555302403 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504997626 0 0) 0x7f729406d1b0 con 0x7f72940b30d0 2023-12-19T16:57:42.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f72989bc640 1 -- 172.21.15.26:0/1555302403 >> v1:172.21.15.160:6789/0 conn(0x7f72940b30d0 legacy=0x7f72940cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f72989bc640 1 -- 172.21.15.26:0/1555302403 >> v1:172.21.15.26:6789/0 conn(0x7f72940adb80 legacy=0x7f72940cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f72989bc640 1 -- 172.21.15.26:0/1555302403 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f729406d490 con 0x7f729415c7f0 2023-12-19T16:57:42.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f72989bc640 1 --2- 172.21.15.26:0/1555302403 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f729415c7f0 0x7f729406c7f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f72880097f0 tx=0x7f728801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b665e72cdec8400 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:42.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f72937fe640 1 -- 172.21.15.26:0/1555302403 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72880ac630 con 0x7f729415c7f0 2023-12-19T16:57:42.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f72937fe640 1 -- 172.21.15.26:0/1555302403 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72880ac790 con 0x7f729415c7f0 2023-12-19T16:57:42.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f72937fe640 1 -- 172.21.15.26:0/1555302403 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72880acb00 con 0x7f729415c7f0 2023-12-19T16:57:42.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f729ac20640 1 -- 172.21.15.26:0/1555302403 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f729415c7f0 msgr2=0x7f729406c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f729ac20640 1 --2- 172.21.15.26:0/1555302403 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f729415c7f0 0x7f729406c7f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f72880097f0 tx=0x7f728801c4c0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f729ac20640 1 -- 172.21.15.26:0/1555302403 shutdown_connections 2023-12-19T16:57:42.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f729ac20640 1 --2- 172.21.15.26:0/1555302403 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f729415c7f0 0x7f729406c7f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.773+0000 7f729ac20640 1 -- 172.21.15.26:0/1555302403 >> 172.21.15.26:0/1555302403 conn(0x7f72940658e0 msgr2=0x7f7294065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:42.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f729ac20640 1 -- 172.21.15.26:0/1555302403 shutdown_connections 2023-12-19T16:57:42.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f729ac20640 1 -- 172.21.15.26:0/1555302403 wait complete. 2023-12-19T16:57:42.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f729ac20640 1 Processor -- start 2023-12-19T16:57:42.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f729ac20640 1 -- start start 2023-12-19T16:57:42.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f729ac20640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72940adb80 0x7f7294147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f729ac20640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f72940b30d0 0x7f729414f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f729ac20640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f729415c7f0 0x7f7294142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f729ac20640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7294147b90 con 0x7f729415c7f0 2023-12-19T16:57:42.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f729ac20640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72941468b0 con 0x7f72940adb80 2023-12-19T16:57:42.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f729ac20640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7294146b90 con 0x7f72940b30d0 2023-12-19T16:57:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f72991bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f729415c7f0 0x7f7294142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f72989bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72940adb80 0x7f7294147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f72991bd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f729415c7f0 0x7f7294142960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44930/0 (socket says 172.21.15.26:44930) 2023-12-19T16:57:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f72991bd640 1 -- 172.21.15.26:0/134665373 learned_addr learned my addr 172.21.15.26:0/134665373 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f7293fff640 1 --2- 172.21.15.26:0/134665373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f72940b30d0 0x7f729414f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f72991bd640 1 -- 172.21.15.26:0/134665373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f72940b30d0 msgr2=0x7f729414f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f72991bd640 1 --2- 172.21.15.26:0/134665373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f72940b30d0 0x7f729414f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f72991bd640 1 -- 172.21.15.26:0/134665373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72940adb80 msgr2=0x7f7294147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f72991bd640 1 --2- 172.21.15.26:0/134665373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72940adb80 0x7f7294147650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f72991bd640 1 -- 172.21.15.26:0/134665373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7288009420 con 0x7f729415c7f0 2023-12-19T16:57:42.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.777+0000 7f72991bd640 1 --2- 172.21.15.26:0/134665373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f729415c7f0 0x7f7294142960 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c780 tx=0x7f728c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:42.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.781+0000 7f72917fa640 1 -- 172.21.15.26:0/134665373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c003db0 con 0x7f729415c7f0 2023-12-19T16:57:42.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.781+0000 7f72917fa640 1 -- 172.21.15.26:0/134665373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728c00ce90 con 0x7f729415c7f0 2023-12-19T16:57:42.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.781+0000 7f729ac20640 1 -- 172.21.15.26:0/134665373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7294146e70 con 0x7f729415c7f0 2023-12-19T16:57:42.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.781+0000 7f72917fa640 1 -- 172.21.15.26:0/134665373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c0052b0 con 0x7f729415c7f0 2023-12-19T16:57:42.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.781+0000 7f729ac20640 1 -- 172.21.15.26:0/134665373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729414ff30 con 0x7f729415c7f0 2023-12-19T16:57:42.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.781+0000 7f729ac20640 1 -- 172.21.15.26:0/134665373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7294074270 con 0x7f729415c7f0 2023-12-19T16:57:42.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.781+0000 7f72917fa640 1 -- 172.21.15.26:0/134665373 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 36) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f728c005410 con 0x7f729415c7f0 2023-12-19T16:57:42.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.781+0000 7f72917fa640 1 --2- 172.21.15.26:0/134665373 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f72780658b0 0x7f7278067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:42.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.781+0000 7f72917fa640 1 -- 172.21.15.26:0/134665373 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f728c083000 con 0x7f729415c7f0 2023-12-19T16:57:42.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.781+0000 7f72989bc640 1 --2- 172.21.15.26:0/134665373 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f72780658b0 0x7f7278067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:42.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.785+0000 7f72989bc640 1 --2- 172.21.15.26:0/134665373 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f72780658b0 0x7f7278067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f72880ae000 tx=0x7f72880a7000 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:42.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.785+0000 7f72917fa640 1 -- 172.21.15.26:0/134665373 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f728c050870 con 0x7f729415c7f0 2023-12-19T16:57:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.977+0000 7f729ac20640 1 -- 172.21.15.26:0/134665373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7294070fd0 con 0x7f729415c7f0 2023-12-19T16:57:42.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.977+0000 7f72917fa640 1 -- 172.21.15.26:0/134665373 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f728c053e90 con 0x7f729415c7f0 2023-12-19T16:57:42.983 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:42.984 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/436365775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":436365775},{"type":"v1","addr":"172.21.15.160:6836","nonce":436365775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3014042376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3014042376},{"type":"v1","addr":"172.21.15.160:6837","nonce":3014042376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1357342534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1357342534},{"type":"v1","addr":"172.21.15.26:6835","nonce":1357342534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:57:23.100695+0000","modified":"2023-12-19T16:57:26.118941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6522":{"gid":6522,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/325854666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":325854666},{"type":"v1","addr":"172.21.15.26:6834","nonce":325854666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-19T16:57:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 33 2023-12-19T16:57:42.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.981+0000 7f729ac20640 1 -- 172.21.15.26:0/134665373 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f72780658b0 msgr2=0x7f7278067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.981+0000 7f729ac20640 1 --2- 172.21.15.26:0/134665373 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f72780658b0 0x7f7278067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f72880ae000 tx=0x7f72880a7000 comp rx=0 tx=0).stop 2023-12-19T16:57:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.981+0000 7f729ac20640 1 -- 172.21.15.26:0/134665373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f729415c7f0 msgr2=0x7f7294142960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.981+0000 7f729ac20640 1 --2- 172.21.15.26:0/134665373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f729415c7f0 0x7f7294142960 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f728c00c780 tx=0x7f728c00cc40 comp rx=0 tx=0).stop 2023-12-19T16:57:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.981+0000 7f729ac20640 1 -- 172.21.15.26:0/134665373 shutdown_connections 2023-12-19T16:57:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.981+0000 7f729ac20640 1 --2- 172.21.15.26:0/134665373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f729415c7f0 0x7f7294142960 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.981+0000 7f729ac20640 1 --2- 172.21.15.26:0/134665373 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f72780658b0 0x7f7278067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.981+0000 7f729ac20640 1 --2- 172.21.15.26:0/134665373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f72940b30d0 0x7f729414f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.981+0000 7f729ac20640 1 --2- 172.21.15.26:0/134665373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72940adb80 0x7f7294147650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.981+0000 7f729ac20640 1 -- 172.21.15.26:0/134665373 >> 172.21.15.26:0/134665373 conn(0x7f72940658e0 msgr2=0x7f72940cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.981+0000 7f729ac20640 1 -- 172.21.15.26:0/134665373 shutdown_connections 2023-12-19T16:57:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:42.981+0000 7f729ac20640 1 -- 172.21.15.26:0/134665373 wait complete. 2023-12-19T16:57:43.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:43.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5ab59e640 1 Processor -- start 2023-12-19T16:57:43.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5ab59e640 1 -- start start 2023-12-19T16:57:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5ab59e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc5ac15cbf0 0x7fc5ac0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5ab59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5ac0b4420 0x7fc5ac0b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5ab59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5ac0b5ad0 0x7fc5ac0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5ab59e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ac0b4dc0 con 0x7fc5ac0b5ad0 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5ab59e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ac15cfd0 con 0x7fc5ac15cbf0 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5ab59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0b6fc0 con 0x7fc5ac0b4420 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5aad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5ac0b5ad0 0x7fc5ac0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5aad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5ac0b5ad0 0x7fc5ac0b5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44940/0 (socket says 172.21.15.26:44940) 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5aad9d640 1 -- 172.21.15.26:0/1178917595 learned_addr learned my addr 172.21.15.26:0/1178917595 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5aa59c640 1 --2- 172.21.15.26:0/1178917595 >> v2:172.21.15.160:3300/0 conn(0x7fc5ac15cbf0 0x7fc5ac0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5a3fff640 1 --2- 172.21.15.26:0/1178917595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5ac0b4420 0x7fc5ac0b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5aad9d640 1 -- 172.21.15.26:0/1178917595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5ac0b4420 msgr2=0x7fc5ac0b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5aad9d640 1 --2- 172.21.15.26:0/1178917595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5ac0b4420 0x7fc5ac0b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5aad9d640 1 -- 172.21.15.26:0/1178917595 >> v2:172.21.15.160:3300/0 conn(0x7fc5ac15cbf0 msgr2=0x7fc5ac0b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5aad9d640 1 --2- 172.21.15.26:0/1178917595 >> v2:172.21.15.160:3300/0 conn(0x7fc5ac15cbf0 0x7fc5ac0b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5aad9d640 1 -- 172.21.15.26:0/1178917595 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac0b72a0 con 0x7fc5ac0b5ad0 2023-12-19T16:57:43.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.145+0000 7fc5a3fff640 1 --2- 172.21.15.26:0/1178917595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5ac0b4420 0x7fc5ac0b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:43.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.149+0000 7fc5aad9d640 1 --2- 172.21.15.26:0/1178917595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5ac0b5ad0 0x7fc5ac0b5f60 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac073840 tx=0x7fc5a401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ee99f665c57221 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.149+0000 7fc5a9d9b640 1 -- 172.21.15.26:0/1178917595 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a40a57c0 con 0x7fc5ac0b5ad0 2023-12-19T16:57:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.149+0000 7fc5a9d9b640 1 -- 172.21.15.26:0/1178917595 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a40a5920 con 0x7fc5ac0b5ad0 2023-12-19T16:57:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.149+0000 7fc5a9d9b640 1 -- 172.21.15.26:0/1178917595 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a40a5cb0 con 0x7fc5ac0b5ad0 2023-12-19T16:57:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.149+0000 7fc5ab59e640 1 -- 172.21.15.26:0/1178917595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5ac0b5ad0 msgr2=0x7fc5ac0b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.149+0000 7fc5ab59e640 1 --2- 172.21.15.26:0/1178917595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5ac0b5ad0 0x7fc5ac0b5f60 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac073840 tx=0x7fc5a401c940 comp rx=0 tx=0).stop 2023-12-19T16:57:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.149+0000 7fc5ab59e640 1 -- 172.21.15.26:0/1178917595 shutdown_connections 2023-12-19T16:57:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.149+0000 7fc5ab59e640 1 --2- 172.21.15.26:0/1178917595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5ac0b5ad0 0x7fc5ac0b5f60 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.149+0000 7fc5ab59e640 1 --2- 172.21.15.26:0/1178917595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5ac0b4420 0x7fc5ac0b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.149+0000 7fc5ab59e640 1 --2- 172.21.15.26:0/1178917595 >> v2:172.21.15.160:3300/0 conn(0x7fc5ac15cbf0 0x7fc5ac0b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.149+0000 7fc5ab59e640 1 -- 172.21.15.26:0/1178917595 >> 172.21.15.26:0/1178917595 conn(0x7fc5ac06a120 msgr2=0x7fc5ac06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.149+0000 7fc5ab59e640 1 -- 172.21.15.26:0/1178917595 shutdown_connections 2023-12-19T16:57:43.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.149+0000 7fc5ab59e640 1 -- 172.21.15.26:0/1178917595 wait complete. 2023-12-19T16:57:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5ab59e640 1 Processor -- start 2023-12-19T16:57:43.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5ab59e640 1 -- start start 2023-12-19T16:57:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5ab59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5ac0b4420 0x7fc5ac14fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5ab59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5ac0b5ad0 0x7fc5ac153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5ab59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5ac15cbf0 0x7fc5ac0c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5ab59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0d0fe0 con 0x7fc5ac0b4420 2023-12-19T16:57:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5ab59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0d2d30 con 0x7fc5ac0b5ad0 2023-12-19T16:57:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5ab59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0d2490 con 0x7fc5ac15cbf0 2023-12-19T16:57:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5aa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5ac0b4420 0x7fc5ac14fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5aa59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5ac0b4420 0x7fc5ac14fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44944/0 (socket says 172.21.15.26:44944) 2023-12-19T16:57:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5aa59c640 1 -- 172.21.15.26:0/834296334 learned_addr learned my addr 172.21.15.26:0/834296334 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5aad9d640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5ac15cbf0 0x7fc5ac0c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5a3fff640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5ac0b5ad0 0x7fc5ac153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5aa59c640 1 -- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5ac15cbf0 msgr2=0x7fc5ac0c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5aa59c640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5ac15cbf0 0x7fc5ac0c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5aa59c640 1 -- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5ac0b5ad0 msgr2=0x7fc5ac153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5aa59c640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5ac0b5ad0 0x7fc5ac153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5aa59c640 1 -- 172.21.15.26:0/834296334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a4009b30 con 0x7fc5ac0b4420 2023-12-19T16:57:43.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5aa59c640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5ac0b4420 0x7fc5ac14fac0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e870 tx=0x7fc59c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:43.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5a2ffd640 1 -- 172.21.15.26:0/834296334 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c009510 con 0x7fc5ac0b4420 2023-12-19T16:57:43.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5a2ffd640 1 -- 172.21.15.26:0/834296334 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c00eec0 con 0x7fc5ac0b4420 2023-12-19T16:57:43.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5ab59e640 1 -- 172.21.15.26:0/834296334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ac0d2770 con 0x7fc5ac0b4420 2023-12-19T16:57:43.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5a2ffd640 1 -- 172.21.15.26:0/834296334 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00cb40 con 0x7fc5ac0b4420 2023-12-19T16:57:43.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5ab59e640 1 -- 172.21.15.26:0/834296334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac14c600 con 0x7fc5ac0b4420 2023-12-19T16:57:43.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.153+0000 7fc5ab59e640 1 -- 172.21.15.26:0/834296334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac067400 con 0x7fc5ac0b4420 2023-12-19T16:57:43.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.157+0000 7fc5a2ffd640 1 -- 172.21.15.26:0/834296334 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 36) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fc59c00cca0 con 0x7fc5ac0b4420 2023-12-19T16:57:43.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.157+0000 7fc5a2ffd640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc58c0658b0 0x7fc58c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.157+0000 7fc5a2ffd640 1 -- 172.21.15.26:0/834296334 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fc59c0834d0 con 0x7fc5ac0b4420 2023-12-19T16:57:43.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.157+0000 7fc5a3fff640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc58c0658b0 0x7fc58c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.157+0000 7fc5a3fff640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc58c0658b0 0x7fc58c067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0d2290 tx=0x7fc594008040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:43.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.161+0000 7fc5a2ffd640 1 -- 172.21.15.26:0/834296334 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc59c050bc0 con 0x7fc5ac0b4420 2023-12-19T16:57:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.325+0000 7fc5ab59e640 1 -- 172.21.15.26:0/834296334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5ac1518b0 con 0x7fc5ac0b4420 2023-12-19T16:57:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.325+0000 7fc5a2ffd640 1 -- 172.21.15.26:0/834296334 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18074 (secure 0 0 0) 0x7fc59c0541e0 con 0x7fc5ac0b4420 2023-12-19T16:57:43.331 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:43.331 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":81,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:41.248623+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:20.751846+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:22.096742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3016063689":"2023-12-20T16:57:41.248587+0000","172.21.15.160:0/537357214":"2023-12-20T16:57:41.248587+0000","172.21.15.160:0/932487207":"2023-12-20T16:57:41.248587+0000","172.21.15.160:0/3937793296":"2023-12-20T16:57:41.248587+0000","172.21.15.160:0/3913339488":"2023-12-20T16:57:41.248587+0000","172.21.15.160:0/374927106":"2023-12-20T16:57:41.248587+0000","172.21.15.160:6833/38098":"2023-12-20T16:57:35.339958+0000","172.21.15.160:6832/38098":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/3906095528":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/2024777587":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/936215346":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/704170657":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/2914461989":"2023-12-20T16:57:35.339958+0000","172.21.15.160:0/2579181328":"2023-12-20T16:57:35.339958+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:43.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.329+0000 7fc5ab59e640 1 -- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc58c0658b0 msgr2=0x7fc58c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.329+0000 7fc5ab59e640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc58c0658b0 0x7fc58c067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0d2290 tx=0x7fc594008040 comp rx=0 tx=0).stop 2023-12-19T16:57:43.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.329+0000 7fc5ab59e640 1 -- 172.21.15.26:0/834296334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5ac0b4420 msgr2=0x7fc5ac14fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.329+0000 7fc5ab59e640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5ac0b4420 0x7fc5ac14fac0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e870 tx=0x7fc59c00ec70 comp rx=0 tx=0).stop 2023-12-19T16:57:43.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.329+0000 7fc5ab59e640 1 -- 172.21.15.26:0/834296334 shutdown_connections 2023-12-19T16:57:43.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.329+0000 7fc5ab59e640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc58c0658b0 0x7fc58c067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.329+0000 7fc5ab59e640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5ac15cbf0 0x7fc5ac0c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.329+0000 7fc5ab59e640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5ac0b5ad0 0x7fc5ac153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.329+0000 7fc5ab59e640 1 --2- 172.21.15.26:0/834296334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5ac0b4420 0x7fc5ac14fac0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.329+0000 7fc5ab59e640 1 -- 172.21.15.26:0/834296334 >> 172.21.15.26:0/834296334 conn(0x7fc5ac06a120 msgr2=0x7fc5ac14dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:43.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.329+0000 7fc5ab59e640 1 -- 172.21.15.26:0/834296334 shutdown_connections 2023-12-19T16:57:43.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.329+0000 7fc5ab59e640 1 -- 172.21.15.26:0/834296334 wait complete. 2023-12-19T16:57:43.353 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T16:57:43.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.493+0000 7f0e4bd8f640 1 Processor -- start 2023-12-19T16:57:43.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.493+0000 7f0e4bd8f640 1 -- start start 2023-12-19T16:57:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.493+0000 7f0e4bd8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e4406c7f0 0x7f0e440cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.493+0000 7f0e4bd8f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4406cc50 con 0x7f0e440ccc20 2023-12-19T16:57:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.493+0000 7f0e4bd8f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4406ced0 con 0x7f0e4415c7f0 2023-12-19T16:57:43.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.493+0000 7f0e4bd8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4406d770 con 0x7f0e4406c7f0 2023-12-19T16:57:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.493+0000 7f0e4a32c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0e440ccc20 0x7f0e440b30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38050/0 (socket says 172.21.15.26:38050) 2023-12-19T16:57:43.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.493+0000 7f0e4a32c640 1 -- 172.21.15.26:0/3094672601 learned_addr learned my addr 172.21.15.26:0/3094672601 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.493+0000 7f0e4932a640 1 --2- 172.21.15.26:0/3094672601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e4406c7f0 0x7f0e440cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.493+0000 7f0e48b29640 1 -- 172.21.15.26:0/3094672601 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443988763 0 0) 0x7f0e4406cc50 con 0x7f0e440ccc20 2023-12-19T16:57:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.497+0000 7f0e48b29640 1 -- 172.21.15.26:0/3094672601 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e30005740 con 0x7f0e440ccc20 2023-12-19T16:57:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.497+0000 7f0e48b29640 1 -- 172.21.15.26:0/3094672601 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260833309 0 0) 0x7f0e4406ced0 con 0x7f0e4415c7f0 2023-12-19T16:57:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.497+0000 7f0e48b29640 1 -- 172.21.15.26:0/3094672601 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e4406cc50 con 0x7f0e4415c7f0 2023-12-19T16:57:43.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.497+0000 7f0e4932a640 1 -- 172.21.15.26:0/3094672601 >> v1:172.21.15.160:6789/0 conn(0x7f0e4415c7f0 legacy=0x7f0e440adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.497+0000 7f0e4932a640 1 -- 172.21.15.26:0/3094672601 >> v1:172.21.15.26:6789/0 conn(0x7f0e440ccc20 legacy=0x7f0e440b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.497+0000 7f0e48b29640 1 -- 172.21.15.26:0/3094672601 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3708925967 0 0) 0x7f0e30005740 con 0x7f0e440ccc20 2023-12-19T16:57:43.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.497+0000 7f0e4932a640 1 -- 172.21.15.26:0/3094672601 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4406d1b0 con 0x7f0e4406c7f0 2023-12-19T16:57:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.497+0000 7f0e4932a640 1 --2- 172.21.15.26:0/3094672601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e4406c7f0 0x7f0e440cc6e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e340097f0 tx=0x7f0e3401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eab8264494d3e033 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.497+0000 7f0e48b29640 1 -- 172.21.15.26:0/3094672601 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c001a00 con 0x7f0e4406c7f0 2023-12-19T16:57:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.497+0000 7f0e48b29640 1 -- 172.21.15.26:0/3094672601 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e340a7050 con 0x7f0e4406c7f0 2023-12-19T16:57:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.497+0000 7f0e48b29640 1 -- 172.21.15.26:0/3094672601 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e340ab620 con 0x7f0e4406c7f0 2023-12-19T16:57:43.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/3094672601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e4406c7f0 msgr2=0x7f0e440cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 --2- 172.21.15.26:0/3094672601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e4406c7f0 0x7f0e440cc6e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0e340097f0 tx=0x7f0e3401c3e0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/3094672601 shutdown_connections 2023-12-19T16:57:43.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 --2- 172.21.15.26:0/3094672601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e4406c7f0 0x7f0e440cc6e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/3094672601 >> 172.21.15.26:0/3094672601 conn(0x7f0e440658e0 msgr2=0x7f0e44065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:43.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/3094672601 shutdown_connections 2023-12-19T16:57:43.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/3094672601 wait complete. 2023-12-19T16:57:43.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 Processor -- start 2023-12-19T16:57:43.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 -- start start 2023-12-19T16:57:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e4406c7f0 0x7f0e4415a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e440ccc20 0x7f0e441530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e4415c7f0 0x7f0e44154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e441f9a40 con 0x7f0e4415c7f0 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4413daa0 con 0x7f0e4406c7f0 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4bd8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e44159590 con 0x7f0e440ccc20 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4a32c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e4415c7f0 0x7f0e44154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4a32c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e4415c7f0 0x7f0e44154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44950/0 (socket says 172.21.15.26:44950) 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4a32c640 1 -- 172.21.15.26:0/548759095 learned_addr learned my addr 172.21.15.26:0/548759095 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e49b2b640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e4406c7f0 0x7f0e4415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4932a640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e440ccc20 0x7f0e441530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4a32c640 1 -- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e440ccc20 msgr2=0x7f0e441530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4a32c640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e440ccc20 0x7f0e441530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4a32c640 1 -- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e4406c7f0 msgr2=0x7f0e4415a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4a32c640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e4406c7f0 0x7f0e4415a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4a32c640 1 -- 172.21.15.26:0/548759095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e34009420 con 0x7f0e4415c7f0 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.501+0000 7f0e4932a640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e440ccc20 0x7f0e441530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.505+0000 7f0e4a32c640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e4415c7f0 0x7f0e44154bc0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00cc90 tx=0x7f0e3c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:43.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.505+0000 7f0e3a7fc640 1 -- 172.21.15.26:0/548759095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c00cea0 con 0x7f0e4415c7f0 2023-12-19T16:57:43.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.505+0000 7f0e3a7fc640 1 -- 172.21.15.26:0/548759095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3c012400 con 0x7f0e4415c7f0 2023-12-19T16:57:43.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.505+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/548759095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e44159870 con 0x7f0e4415c7f0 2023-12-19T16:57:43.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.505+0000 7f0e3a7fc640 1 -- 172.21.15.26:0/548759095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c0127f0 con 0x7f0e4415c7f0 2023-12-19T16:57:43.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.505+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/548759095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e44159e30 con 0x7f0e4415c7f0 2023-12-19T16:57:43.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.505+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/548759095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4415a020 con 0x7f0e4415c7f0 2023-12-19T16:57:43.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.505+0000 7f0e3a7fc640 1 -- 172.21.15.26:0/548759095 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 36) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f0e3c012950 con 0x7f0e4415c7f0 2023-12-19T16:57:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.505+0000 7f0e3a7fc640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e280658b0 0x7f0e28067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.505+0000 7f0e3a7fc640 1 -- 172.21.15.26:0/548759095 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f0e3c083b00 con 0x7f0e4415c7f0 2023-12-19T16:57:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.509+0000 7f0e49b2b640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e280658b0 0x7f0e28067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.509+0000 7f0e49b2b640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e280658b0 0x7f0e28067d70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e400011f0 tx=0x7f0e40009040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.509+0000 7f0e3a7fc640 1 -- 172.21.15.26:0/548759095 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0e3c085e60 con 0x7f0e4415c7f0 2023-12-19T16:57:43.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.533+0000 7f0e3a7fc640 1 -- 172.21.15.26:0/548759095 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 37) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f0e3c0515c0 con 0x7f0e4415c7f0 2023-12-19T16:57:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.705+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/548759095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0e44074270 con 0x7f0e4415c7f0 2023-12-19T16:57:43.710 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T16:57:43.705+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T16:57:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.713+0000 7f0e3a7fc640 1 -- 172.21.15.26:0/548759095 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0e3c06d6a0 con 0x7f0e4415c7f0 2023-12-19T16:57:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T16:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.717+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e280658b0 msgr2=0x7f0e28067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.717+0000 7f0e4bd8f640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e280658b0 0x7f0e28067d70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e400011f0 tx=0x7f0e40009040 comp rx=0 tx=0).stop 2023-12-19T16:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.717+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/548759095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e4415c7f0 msgr2=0x7f0e44154bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.717+0000 7f0e4bd8f640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e4415c7f0 0x7f0e44154bc0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c00cc90 tx=0x7f0e3c00a430 comp rx=0 tx=0).stop 2023-12-19T16:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.717+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/548759095 shutdown_connections 2023-12-19T16:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.717+0000 7f0e4bd8f640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e4415c7f0 0x7f0e44154bc0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.717+0000 7f0e4bd8f640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e280658b0 0x7f0e28067d70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.717+0000 7f0e4bd8f640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e440ccc20 0x7f0e441530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.717+0000 7f0e4bd8f640 1 --2- 172.21.15.26:0/548759095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e4406c7f0 0x7f0e4415a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.717+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/548759095 >> 172.21.15.26:0/548759095 conn(0x7f0e440658e0 msgr2=0x7f0e4406fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.717+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/548759095 shutdown_connections 2023-12-19T16:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.717+0000 7f0e4bd8f640 1 -- 172.21.15.26:0/548759095 wait complete. 2023-12-19T16:57:43.727 INFO:tasks.ceph.mds.a.smithi026.stderr: -3775> 2023-12-19T16:57:28.669+0000 7eff4c7fe640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T16:57:43.742 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T16:57:43.777 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T16:57:43.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.881+0000 7f05b16e5640 1 Processor -- start 2023-12-19T16:57:43.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.881+0000 7f05b16e5640 1 -- start start 2023-12-19T16:57:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.885+0000 7f05b16e5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f05ac0cd710 0x7f05ac0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.885+0000 7f05b16e5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac072330 con 0x7f05ac15bb70 2023-12-19T16:57:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.885+0000 7f05b16e5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac0cc950 con 0x7f05ac0b02c0 2023-12-19T16:57:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.885+0000 7f05b16e5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ac0cc760 con 0x7f05ac0cd710 2023-12-19T16:57:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.885+0000 7f05a3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f05ac15bb70 0x7f05ac071a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38056/0 (socket says 172.21.15.26:38056) 2023-12-19T16:57:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.885+0000 7f05a3fff640 1 -- 172.21.15.26:0/3218467526 learned_addr learned my addr 172.21.15.26:0/3218467526 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.885+0000 7f05ab7fe640 1 --2- 172.21.15.26:0/3218467526 >> v2:172.21.15.160:3300/0 conn(0x7f05ac0cd710 0x7f05ac0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.885+0000 7f05aa7fc640 1 -- 172.21.15.26:0/3218467526 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790823181 0 0) 0x7f05ac072330 con 0x7f05ac15bb70 2023-12-19T16:57:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.885+0000 7f05aa7fc640 1 -- 172.21.15.26:0/3218467526 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0598003530 con 0x7f05ac15bb70 2023-12-19T16:57:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.885+0000 7f05ab7fe640 1 -- 172.21.15.26:0/3218467526 >> v1:172.21.15.160:6789/0 conn(0x7f05ac0b02c0 legacy=0x7f05ac0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.885+0000 7f05ab7fe640 1 -- 172.21.15.26:0/3218467526 >> v1:172.21.15.26:6789/0 conn(0x7f05ac15bb70 legacy=0x7f05ac071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.885+0000 7f05ab7fe640 1 -- 172.21.15.26:0/3218467526 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac0ccc30 con 0x7f05ac0cd710 2023-12-19T16:57:43.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.889+0000 7f05ab7fe640 1 --2- 172.21.15.26:0/3218467526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05ac0cd710 0x7f05ac0cdf40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009dc0 tx=0x7f05a401c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e1024dab38ff1b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:43.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.889+0000 7f05aa7fc640 1 -- 172.21.15.26:0/3218467526 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0cec90 con 0x7f05ac0cd710 2023-12-19T16:57:43.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.889+0000 7f05aa7fc640 1 -- 172.21.15.26:0/3218467526 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a40a5760 con 0x7f05ac0cd710 2023-12-19T16:57:43.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.889+0000 7f05aa7fc640 1 -- 172.21.15.26:0/3218467526 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a40a5af0 con 0x7f05ac0cd710 2023-12-19T16:57:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.889+0000 7f05b16e5640 1 -- 172.21.15.26:0/3218467526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05ac0cd710 msgr2=0x7f05ac0cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.889+0000 7f05b16e5640 1 --2- 172.21.15.26:0/3218467526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05ac0cd710 0x7f05ac0cdf40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009dc0 tx=0x7f05a401c8a0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.889+0000 7f05b16e5640 1 -- 172.21.15.26:0/3218467526 shutdown_connections 2023-12-19T16:57:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.889+0000 7f05b16e5640 1 --2- 172.21.15.26:0/3218467526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05ac0cd710 0x7f05ac0cdf40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.889+0000 7f05b16e5640 1 -- 172.21.15.26:0/3218467526 >> 172.21.15.26:0/3218467526 conn(0x7f05ac0733c0 msgr2=0x7f05ac06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.889+0000 7f05b16e5640 1 -- 172.21.15.26:0/3218467526 shutdown_connections 2023-12-19T16:57:43.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.889+0000 7f05b16e5640 1 -- 172.21.15.26:0/3218467526 wait complete. 2023-12-19T16:57:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05b16e5640 1 Processor -- start 2023-12-19T16:57:43.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05b16e5640 1 -- start start 2023-12-19T16:57:43.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05b16e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05ac0b02c0 0x7f05ac159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05b16e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05ac0cd710 0x7f05ac0afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05b16e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05ac15bb70 0x7f05ac0aa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05b16e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0aea40 con 0x7f05ac0b02c0 2023-12-19T16:57:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05b16e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0addc0 con 0x7f05ac0cd710 2023-12-19T16:57:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05b16e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0ae0a0 con 0x7f05ac15bb70 2023-12-19T16:57:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05aaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05ac0b02c0 0x7f05ac159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05aaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05ac0b02c0 0x7f05ac159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44974/0 (socket says 172.21.15.26:44974) 2023-12-19T16:57:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05aaffd640 1 -- 172.21.15.26:0/910209972 learned_addr learned my addr 172.21.15.26:0/910209972 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05ab7fe640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05ac15bb70 0x7f05ac0aa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05a3fff640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05ac0cd710 0x7f05ac0afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05aaffd640 1 -- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05ac15bb70 msgr2=0x7f05ac0aa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05aaffd640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05ac15bb70 0x7f05ac0aa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05aaffd640 1 -- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05ac0cd710 msgr2=0x7f05ac0afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:43.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05aaffd640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05ac0cd710 0x7f05ac0afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:43.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05aaffd640 1 -- 172.21.15.26:0/910209972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a4009a70 con 0x7f05ac0b02c0 2023-12-19T16:57:43.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05aaffd640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05ac0b02c0 0x7f05ac159bb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f059c00c3c0 tx=0x7f059c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:43.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05a37fe640 1 -- 172.21.15.26:0/910209972 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c0052d0 con 0x7f05ac0b02c0 2023-12-19T16:57:43.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05a37fe640 1 -- 172.21.15.26:0/910209972 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059c005430 con 0x7f05ac0b02c0 2023-12-19T16:57:43.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05a37fe640 1 -- 172.21.15.26:0/910209972 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c003840 con 0x7f05ac0b02c0 2023-12-19T16:57:43.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05b16e5640 1 -- 172.21.15.26:0/910209972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac0ae380 con 0x7f05ac0b02c0 2023-12-19T16:57:43.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.893+0000 7f05b16e5640 1 -- 172.21.15.26:0/910209972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac0aaf20 con 0x7f05ac0b02c0 2023-12-19T16:57:43.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.897+0000 7f05b16e5640 1 -- 172.21.15.26:0/910209972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac0aeb80 con 0x7f05ac0b02c0 2023-12-19T16:57:43.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.897+0000 7f05a37fe640 1 -- 172.21.15.26:0/910209972 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 37) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f059c0039a0 con 0x7f05ac0b02c0 2023-12-19T16:57:43.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.897+0000 7f05a37fe640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f058c065b40 0x7f058c068000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:43.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.897+0000 7f05a37fe640 1 -- 172.21.15.26:0/910209972 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f059c082c80 con 0x7f05ac0b02c0 2023-12-19T16:57:43.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.897+0000 7f05a3fff640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f058c065b40 0x7f058c068000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:43.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.897+0000 7f05a3fff640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f058c065b40 0x7f058c068000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f05ac15a630 tx=0x7f0594007d40 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:43.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:43.901+0000 7f05a37fe640 1 -- 172.21.15.26:0/910209972 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f05ac0aeb80 con 0x7f05ac0b02c0 2023-12-19T16:57:44.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.089+0000 7f05b16e5640 1 -- 172.21.15.26:0/910209972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f05ac0aeb80 con 0x7f05ac0b02c0 2023-12-19T16:57:44.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.717+0000 7f05a37fe640 1 -- 172.21.15.26:0/910209972 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) v1 ==== 104+0+0 (secure 0 0 0) 0x7f05ac0aeb80 con 0x7f05ac0b02c0 2023-12-19T16:57:44.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.721+0000 7f05b16e5640 1 -- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f058c065b40 msgr2=0x7f058c068000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:44.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.721+0000 7f05b16e5640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f058c065b40 0x7f058c068000 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f05ac15a630 tx=0x7f0594007d40 comp rx=0 tx=0).stop 2023-12-19T16:57:44.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.721+0000 7f05b16e5640 1 -- 172.21.15.26:0/910209972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05ac0b02c0 msgr2=0x7f05ac159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:44.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.721+0000 7f05b16e5640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05ac0b02c0 0x7f05ac159bb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f059c00c3c0 tx=0x7f059c00c880 comp rx=0 tx=0).stop 2023-12-19T16:57:44.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.721+0000 7f05b16e5640 1 -- 172.21.15.26:0/910209972 shutdown_connections 2023-12-19T16:57:44.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.721+0000 7f05b16e5640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f058c065b40 0x7f058c068000 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:44.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.721+0000 7f05b16e5640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05ac15bb70 0x7f05ac0aa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:44.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.721+0000 7f05b16e5640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05ac0cd710 0x7f05ac0afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:44.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.721+0000 7f05b16e5640 1 --2- 172.21.15.26:0/910209972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05ac0b02c0 0x7f05ac159bb0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:44.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.721+0000 7f05b16e5640 1 -- 172.21.15.26:0/910209972 >> 172.21.15.26:0/910209972 conn(0x7f05ac0733c0 msgr2=0x7f05ac15c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:44.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.721+0000 7f05b16e5640 1 -- 172.21.15.26:0/910209972 shutdown_connections 2023-12-19T16:57:44.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.721+0000 7f05b16e5640 1 -- 172.21.15.26:0/910209972 wait complete. 2023-12-19T16:57:44.747 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T16:57:44.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.937+0000 7fd554ce5640 1 Processor -- start 2023-12-19T16:57:44.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.937+0000 7fd554ce5640 1 -- start start 2023-12-19T16:57:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.937+0000 7fd554ce5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd550159830 0x7fd55015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.937+0000 7fd554ce5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5500c9310 0x7fd5500cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.937+0000 7fd554ce5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55015e3d0 con 0x7fd5500cb110 2023-12-19T16:57:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.937+0000 7fd554ce5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55006f7c0 con 0x7fd550159830 2023-12-19T16:57:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.937+0000 7fd554ce5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5500ca2b0 con 0x7fd5500c9310 2023-12-19T16:57:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.937+0000 7fd547fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5500c9310 0x7fd5500cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.937+0000 7fd54e59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd550159830 0x7fd55015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.937+0000 7fd547fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5500c9310 0x7fd5500cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:32794/0 (socket says 172.21.15.26:32794) 2023-12-19T16:57:44.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.937+0000 7fd547fff640 1 -- 172.21.15.26:0/1448249317 learned_addr learned my addr 172.21.15.26:0/1448249317 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.941+0000 7fd547fff640 1 -- 172.21.15.26:0/1448249317 >> v2:172.21.15.160:3300/0 conn(0x7fd550159830 msgr2=0x7fd55015d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.941+0000 7fd547fff640 1 --2- 172.21.15.26:0/1448249317 >> v2:172.21.15.160:3300/0 conn(0x7fd550159830 0x7fd55015d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.941+0000 7fd547fff640 1 -- 172.21.15.26:0/1448249317 >> v1:172.21.15.160:6789/0 conn(0x7fd5500cb110 legacy=0x7fd5500701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.941+0000 7fd547fff640 1 -- 172.21.15.26:0/1448249317 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55015de10 con 0x7fd5500c9310 2023-12-19T16:57:44.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.941+0000 7fd54dd9b640 1 -- 172.21.15.26:0/1448249317 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846647319 0 0) 0x7fd55015e3d0 con 0x7fd5500cb110 2023-12-19T16:57:44.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.941+0000 7fd547fff640 1 --2- 172.21.15.26:0/1448249317 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5500c9310 0x7fd5500cc430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd538009870 tx=0x7fd53801e690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7446549c19a2089b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:44.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.941+0000 7fd54dd9b640 1 -- 172.21.15.26:0/1448249317 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5380ab660 con 0x7fd5500c9310 2023-12-19T16:57:44.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.941+0000 7fd54dd9b640 1 -- 172.21.15.26:0/1448249317 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5380ab7c0 con 0x7fd5500c9310 2023-12-19T16:57:44.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.941+0000 7fd54dd9b640 1 -- 172.21.15.26:0/1448249317 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5380abb50 con 0x7fd5500c9310 2023-12-19T16:57:44.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 -- 172.21.15.26:0/1448249317 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5500c9310 msgr2=0x7fd5500cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:44.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 --2- 172.21.15.26:0/1448249317 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5500c9310 0x7fd5500cc430 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd538009870 tx=0x7fd53801e690 comp rx=0 tx=0).stop 2023-12-19T16:57:44.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 -- 172.21.15.26:0/1448249317 shutdown_connections 2023-12-19T16:57:44.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 --2- 172.21.15.26:0/1448249317 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5500c9310 0x7fd5500cc430 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:44.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 --2- 172.21.15.26:0/1448249317 >> v2:172.21.15.160:3300/0 conn(0x7fd550159830 0x7fd55015d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 -- 172.21.15.26:0/1448249317 >> 172.21.15.26:0/1448249317 conn(0x7fd5500b0270 msgr2=0x7fd5500b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 -- 172.21.15.26:0/1448249317 shutdown_connections 2023-12-19T16:57:44.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 -- 172.21.15.26:0/1448249317 wait complete. 2023-12-19T16:57:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 Processor -- start 2023-12-19T16:57:44.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 -- start start 2023-12-19T16:57:44.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5500c9310 0x7fd550155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5500cb110 0x7fd5500ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd550159830 0x7fd5500cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5500cd1f0 con 0x7fd5500c9310 2023-12-19T16:57:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5500cd470 con 0x7fd550159830 2023-12-19T16:57:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd554ce5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5500cd750 con 0x7fd5500cb110 2023-12-19T16:57:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd54e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5500c9310 0x7fd550155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd54e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5500c9310 0x7fd550155bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44990/0 (socket says 172.21.15.26:44990) 2023-12-19T16:57:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd54e59c640 1 -- 172.21.15.26:0/455368113 learned_addr learned my addr 172.21.15.26:0/455368113 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd54ed9d640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd550159830 0x7fd5500cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd547fff640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5500cb110 0x7fd5500ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd54e59c640 1 -- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5500cb110 msgr2=0x7fd5500ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd54e59c640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5500cb110 0x7fd5500ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd54e59c640 1 -- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd550159830 msgr2=0x7fd5500cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd54e59c640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd550159830 0x7fd5500cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd54e59c640 1 -- 172.21.15.26:0/455368113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5380094d0 con 0x7fd5500c9310 2023-12-19T16:57:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.945+0000 7fd54e59c640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5500c9310 0x7fd550155bd0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd54000e870 tx=0x7fd54000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.949+0000 7fd546ffd640 1 -- 172.21.15.26:0/455368113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd540009510 con 0x7fd5500c9310 2023-12-19T16:57:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.949+0000 7fd546ffd640 1 -- 172.21.15.26:0/455368113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54000eec0 con 0x7fd5500c9310 2023-12-19T16:57:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.949+0000 7fd546ffd640 1 -- 172.21.15.26:0/455368113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54000cb40 con 0x7fd5500c9310 2023-12-19T16:57:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.949+0000 7fd554ce5640 1 -- 172.21.15.26:0/455368113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5500cbab0 con 0x7fd5500c9310 2023-12-19T16:57:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.949+0000 7fd554ce5640 1 -- 172.21.15.26:0/455368113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5500cdd10 con 0x7fd5500c9310 2023-12-19T16:57:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.949+0000 7fd554ce5640 1 -- 172.21.15.26:0/455368113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd550074270 con 0x7fd5500c9310 2023-12-19T16:57:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.949+0000 7fd546ffd640 1 -- 172.21.15.26:0/455368113 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 37) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fd54000cca0 con 0x7fd5500c9310 2023-12-19T16:57:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.949+0000 7fd546ffd640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd530065af0 0x7fd530067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.949+0000 7fd546ffd640 1 -- 172.21.15.26:0/455368113 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7129+0+0 (secure 0 0 0) 0x7fd540083990 con 0x7fd5500c9310 2023-12-19T16:57:44.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.949+0000 7fd547fff640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd530065af0 0x7fd530067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.953+0000 7fd547fff640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd530065af0 0x7fd530067fb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd5380b0000 tx=0x7fd5380a9000 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:44.953+0000 7fd546ffd640 1 -- 172.21.15.26:0/455368113 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd540050cd0 con 0x7fd5500c9310 2023-12-19T16:57:45.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.109+0000 7fd554ce5640 1 -- 172.21.15.26:0/455368113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd550067bb0 con 0x7fd5500c9310 2023-12-19T16:57:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.729+0000 7fd546ffd640 1 -- 172.21.15.26:0/455368113 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd5400510a0 con 0x7fd5500c9310 2023-12-19T16:57:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.729+0000 7fd546ffd640 1 -- 172.21.15.26:0/455368113 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd54007e8c0 con 0x7fd5500c9310 2023-12-19T16:57:45.735 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T16:57:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.733+0000 7fd554ce5640 1 -- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd530065af0 msgr2=0x7fd530067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.733+0000 7fd554ce5640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd530065af0 0x7fd530067fb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd5380b0000 tx=0x7fd5380a9000 comp rx=0 tx=0).stop 2023-12-19T16:57:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.733+0000 7fd554ce5640 1 -- 172.21.15.26:0/455368113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5500c9310 msgr2=0x7fd550155bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.733+0000 7fd554ce5640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5500c9310 0x7fd550155bd0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd54000e870 tx=0x7fd54000ec70 comp rx=0 tx=0).stop 2023-12-19T16:57:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.733+0000 7fd554ce5640 1 -- 172.21.15.26:0/455368113 shutdown_connections 2023-12-19T16:57:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.733+0000 7fd554ce5640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd550159830 0x7fd5500cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.733+0000 7fd554ce5640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd530065af0 0x7fd530067fb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.733+0000 7fd554ce5640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5500cb110 0x7fd5500ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.733+0000 7fd554ce5640 1 --2- 172.21.15.26:0/455368113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5500c9310 0x7fd550155bd0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.733+0000 7fd554ce5640 1 -- 172.21.15.26:0/455368113 >> 172.21.15.26:0/455368113 conn(0x7fd5500b0270 msgr2=0x7fd5500c6530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.733+0000 7fd554ce5640 1 -- 172.21.15.26:0/455368113 shutdown_connections 2023-12-19T16:57:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.733+0000 7fd554ce5640 1 -- 172.21.15.26:0/455368113 wait complete. 2023-12-19T16:57:45.763 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T16:57:45.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.909+0000 7f2e30cd1640 1 Processor -- start 2023-12-19T16:57:45.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.909+0000 7f2e30cd1640 1 -- start start 2023-12-19T16:57:45.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.909+0000 7f2e30cd1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2e2c15aad0 0x7f2e2c06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:45.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.909+0000 7f2e30cd1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e2c0b3f90 0x7f2e2c0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:45.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.909+0000 7f2e30cd1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2c06cbf0 con 0x7f2e2c0b1590 2023-12-19T16:57:45.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.909+0000 7f2e30cd1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2c0ca8b0 con 0x7f2e2c0b3f90 2023-12-19T16:57:45.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.909+0000 7f2e30cd1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2c06b6f0 con 0x7f2e2c15aad0 2023-12-19T16:57:45.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.909+0000 7f2e2ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e2c0b3f90 0x7f2e2c0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:45.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.909+0000 7f2e2ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e2c0b3f90 0x7f2e2c0cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43270/0 (socket says 172.21.15.26:43270) 2023-12-19T16:57:45.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.909+0000 7f2e2ad9d640 1 -- 172.21.15.26:0/2922133611 learned_addr learned my addr 172.21.15.26:0/2922133611 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:45.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.909+0000 7f2e2a59c640 1 --2- 172.21.15.26:0/2922133611 >> v2:172.21.15.160:3300/0 conn(0x7f2e2c15aad0 0x7f2e2c06c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:45.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.909+0000 7f2e2ad9d640 1 -- 172.21.15.26:0/2922133611 >> v2:172.21.15.160:3300/0 conn(0x7f2e2c15aad0 msgr2=0x7f2e2c06c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:45.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e2ad9d640 1 --2- 172.21.15.26:0/2922133611 >> v2:172.21.15.160:3300/0 conn(0x7f2e2c15aad0 0x7f2e2c06c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:45.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e2ad9d640 1 -- 172.21.15.26:0/2922133611 >> v1:172.21.15.160:6789/0 conn(0x7f2e2c0b1590 legacy=0x7f2e2c070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:45.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e2ad9d640 1 -- 172.21.15.26:0/2922133611 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c06b9d0 con 0x7f2e2c0b3f90 2023-12-19T16:57:45.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e2ad9d640 1 --2- 172.21.15.26:0/2922133611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c0b3f90 0x7f2e2c0cad40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0cc010 tx=0x7f2e2401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511017a72ac2a505 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e29d9b640 1 -- 172.21.15.26:0/2922133611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e240a5770 con 0x7f2e2c0b3f90 2023-12-19T16:57:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e29d9b640 1 -- 172.21.15.26:0/2922133611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e240a58d0 con 0x7f2e2c0b3f90 2023-12-19T16:57:45.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e29d9b640 1 -- 172.21.15.26:0/2922133611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e240a5c60 con 0x7f2e2c0b3f90 2023-12-19T16:57:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e30cd1640 1 -- 172.21.15.26:0/2922133611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c0b3f90 msgr2=0x7f2e2c0cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e30cd1640 1 --2- 172.21.15.26:0/2922133611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c0b3f90 0x7f2e2c0cad40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0cc010 tx=0x7f2e2401c9a0 comp rx=0 tx=0).stop 2023-12-19T16:57:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e30cd1640 1 -- 172.21.15.26:0/2922133611 shutdown_connections 2023-12-19T16:57:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e30cd1640 1 --2- 172.21.15.26:0/2922133611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c0b3f90 0x7f2e2c0cad40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e30cd1640 1 --2- 172.21.15.26:0/2922133611 >> v2:172.21.15.160:3300/0 conn(0x7f2e2c15aad0 0x7f2e2c06c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e30cd1640 1 -- 172.21.15.26:0/2922133611 >> 172.21.15.26:0/2922133611 conn(0x7f2e2c0b0270 msgr2=0x7f2e2c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e30cd1640 1 -- 172.21.15.26:0/2922133611 shutdown_connections 2023-12-19T16:57:45.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.913+0000 7f2e30cd1640 1 -- 172.21.15.26:0/2922133611 wait complete. 2023-12-19T16:57:45.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e30cd1640 1 Processor -- start 2023-12-19T16:57:45.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e30cd1640 1 -- start start 2023-12-19T16:57:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e30cd1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c0b1590 0x7f2e2c0b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e30cd1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e2c0b3f90 0x7f2e2c0c18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e30cd1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e2c15aad0 0x7f2e2c156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e30cd1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c1577c0 con 0x7f2e2c0b1590 2023-12-19T16:57:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e30cd1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c142330 con 0x7f2e2c15aad0 2023-12-19T16:57:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e30cd1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2c0b29d0 con 0x7f2e2c0b3f90 2023-12-19T16:57:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e2a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c0b1590 0x7f2e2c0b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e2a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c0b1590 0x7f2e2c0b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43280/0 (socket says 172.21.15.26:43280) 2023-12-19T16:57:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e2a59c640 1 -- 172.21.15.26:0/1270769835 learned_addr learned my addr 172.21.15.26:0/1270769835 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e2ad9d640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e2c15aad0 0x7f2e2c156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:45.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e23fff640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e2c0b3f90 0x7f2e2c0c18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e2a59c640 1 -- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e2c0b3f90 msgr2=0x7f2e2c0c18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e2a59c640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e2c0b3f90 0x7f2e2c0c18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e2a59c640 1 -- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e2c15aad0 msgr2=0x7f2e2c156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e2a59c640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e2c15aad0 0x7f2e2c156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e2a59c640 1 -- 172.21.15.26:0/1270769835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e24009b30 con 0x7f2e2c0b1590 2023-12-19T16:57:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e2ad9d640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e2c15aad0 0x7f2e2c156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e2a59c640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c0b1590 0x7f2e2c0b23a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00e870 tx=0x7f2e1c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e22ffd640 1 -- 172.21.15.26:0/1270769835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c009510 con 0x7f2e2c0b1590 2023-12-19T16:57:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e22ffd640 1 -- 172.21.15.26:0/1270769835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c00eec0 con 0x7f2e2c0b1590 2023-12-19T16:57:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e22ffd640 1 -- 172.21.15.26:0/1270769835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c00cb40 con 0x7f2e2c0b1590 2023-12-19T16:57:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.917+0000 7f2e30cd1640 1 -- 172.21.15.26:0/1270769835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2c0b2cb0 con 0x7f2e2c0b1590 2023-12-19T16:57:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.921+0000 7f2e30cd1640 1 -- 172.21.15.26:0/1270769835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2c0b3270 con 0x7f2e2c0b1590 2023-12-19T16:57:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.921+0000 7f2e30cd1640 1 -- 172.21.15.26:0/1270769835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2c074270 con 0x7f2e2c0b1590 2023-12-19T16:57:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.921+0000 7f2e22ffd640 1 -- 172.21.15.26:0/1270769835 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2e1c00ccc0 con 0x7f2e2c0b1590 2023-12-19T16:57:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.921+0000 7f2e22ffd640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2e0c065b80 0x7f2e0c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:45.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.921+0000 7f2e22ffd640 1 -- 172.21.15.26:0/1270769835 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f2e1c083850 con 0x7f2e2c0b1590 2023-12-19T16:57:45.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.921+0000 7f2e23fff640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2e0c065b80 0x7f2e0c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:45.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.925+0000 7f2e23fff640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2e0c065b80 0x7f2e0c068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0ca030 tx=0x7f2e14009040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:45.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:45.925+0000 7f2e22ffd640 1 -- 172.21.15.26:0/1270769835 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2e1c050c80 con 0x7f2e2c0b1590 2023-12-19T16:57:46.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.085+0000 7f2e30cd1640 1 -- 172.21.15.26:0/1270769835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2e2c0c3070 con 0x7f2e2c0b1590 2023-12-19T16:57:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.729+0000 7f2e22ffd640 1 -- 172.21.15.26:0/1270769835 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2e1c0542a0 con 0x7f2e2c0b1590 2023-12-19T16:57:46.733 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T16:57:46.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.733+0000 7f2e30cd1640 1 -- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2e0c065b80 msgr2=0x7f2e0c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:46.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.733+0000 7f2e30cd1640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2e0c065b80 0x7f2e0c068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0ca030 tx=0x7f2e14009040 comp rx=0 tx=0).stop 2023-12-19T16:57:46.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.733+0000 7f2e30cd1640 1 -- 172.21.15.26:0/1270769835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c0b1590 msgr2=0x7f2e2c0b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:46.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.733+0000 7f2e30cd1640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c0b1590 0x7f2e2c0b23a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00e870 tx=0x7f2e1c00ec70 comp rx=0 tx=0).stop 2023-12-19T16:57:46.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.733+0000 7f2e30cd1640 1 -- 172.21.15.26:0/1270769835 shutdown_connections 2023-12-19T16:57:46.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.733+0000 7f2e30cd1640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e2c15aad0 0x7f2e2c156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:46.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.733+0000 7f2e30cd1640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2e0c065b80 0x7f2e0c068040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:46.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.733+0000 7f2e30cd1640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e2c0b3f90 0x7f2e2c0c18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:46.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.733+0000 7f2e30cd1640 1 --2- 172.21.15.26:0/1270769835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e2c0b1590 0x7f2e2c0b23a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:46.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.733+0000 7f2e30cd1640 1 -- 172.21.15.26:0/1270769835 >> 172.21.15.26:0/1270769835 conn(0x7f2e2c0b0270 msgr2=0x7f2e2c0af950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:46.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.733+0000 7f2e30cd1640 1 -- 172.21.15.26:0/1270769835 shutdown_connections 2023-12-19T16:57:46.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.733+0000 7f2e30cd1640 1 -- 172.21.15.26:0/1270769835 wait complete. 2023-12-19T16:57:46.766 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-19T16:57:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.909+0000 7f9139fce640 1 Processor -- start 2023-12-19T16:57:46.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.909+0000 7f9139fce640 1 -- start start 2023-12-19T16:57:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f9139fce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91341585f0 0x7f91340cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:46.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f9139fce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91340cb150 0x7f91340cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f9139fce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913415b1b0 con 0x7f91340cc310 2023-12-19T16:57:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f9139fce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91340cbb10 con 0x7f91340cb150 2023-12-19T16:57:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f9139fce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91340b47a0 con 0x7f91341585f0 2023-12-19T16:57:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f9133fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f91340cc310 0x7f91340cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55962/0 (socket says 172.21.15.26:55962) 2023-12-19T16:57:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f9133fff640 1 -- 172.21.15.26:0/947099146 learned_addr learned my addr 172.21.15.26:0/947099146 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:46.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f9132ffd640 1 --2- 172.21.15.26:0/947099146 >> v2:172.21.15.26:3300/0 conn(0x7f91340cb150 0x7f91340cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f91337fe640 1 --2- 172.21.15.26:0/947099146 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91341585f0 0x7f91340cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f91327fc640 1 -- 172.21.15.26:0/947099146 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029983146 0 0) 0x7f913415b1b0 con 0x7f91340cc310 2023-12-19T16:57:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f91327fc640 1 -- 172.21.15.26:0/947099146 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f911c003530 con 0x7f91340cc310 2023-12-19T16:57:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f9132ffd640 1 -- 172.21.15.26:0/947099146 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91341585f0 msgr2=0x7f91340cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f9132ffd640 1 --2- 172.21.15.26:0/947099146 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91341585f0 0x7f91340cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f9132ffd640 1 -- 172.21.15.26:0/947099146 >> v1:172.21.15.26:6789/0 conn(0x7f91340cc310 legacy=0x7f91340cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f9132ffd640 1 -- 172.21.15.26:0/947099146 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913415b490 con 0x7f91340cb150 2023-12-19T16:57:46.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.913+0000 7f91337fe640 1 --2- 172.21.15.26:0/947099146 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91341585f0 0x7f91340cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f9132ffd640 1 --2- 172.21.15.26:0/947099146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cb150 0x7f91340cb5d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f91280097c0 tx=0x7f912801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd4b31e6324c084 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f91327fc640 1 -- 172.21.15.26:0/947099146 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91280ac620 con 0x7f91340cb150 2023-12-19T16:57:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f91327fc640 1 -- 172.21.15.26:0/947099146 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91280ac780 con 0x7f91340cb150 2023-12-19T16:57:46.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f91327fc640 1 -- 172.21.15.26:0/947099146 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91280acb10 con 0x7f91340cb150 2023-12-19T16:57:46.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f9139fce640 1 -- 172.21.15.26:0/947099146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cb150 msgr2=0x7f91340cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:46.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f9139fce640 1 --2- 172.21.15.26:0/947099146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cb150 0x7f91340cb5d0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f91280097c0 tx=0x7f912801c3c0 comp rx=0 tx=0).stop 2023-12-19T16:57:46.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f9139fce640 1 -- 172.21.15.26:0/947099146 shutdown_connections 2023-12-19T16:57:46.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f9139fce640 1 --2- 172.21.15.26:0/947099146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cb150 0x7f91340cb5d0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:46.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f9139fce640 1 --2- 172.21.15.26:0/947099146 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91341585f0 0x7f91340cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:46.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f9139fce640 1 -- 172.21.15.26:0/947099146 >> 172.21.15.26:0/947099146 conn(0x7f9134066160 msgr2=0x7f91340b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:46.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f9139fce640 1 -- 172.21.15.26:0/947099146 shutdown_connections 2023-12-19T16:57:46.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f9139fce640 1 -- 172.21.15.26:0/947099146 wait complete. 2023-12-19T16:57:46.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f9139fce640 1 Processor -- start 2023-12-19T16:57:46.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.917+0000 7f9139fce640 1 -- start start 2023-12-19T16:57:46.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9139fce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91340cb150 0x7f9134153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:46.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9139fce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cc310 0x7f91340b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:46.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9139fce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91341585f0 0x7f913414c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:46.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9139fce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9134150370 con 0x7f91340cc310 2023-12-19T16:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9139fce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913406c4b0 con 0x7f91340cb150 2023-12-19T16:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9139fce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913406cd50 con 0x7f91341585f0 2023-12-19T16:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9132ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cc310 0x7f91340b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9132ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cc310 0x7f91340b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43302/0 (socket says 172.21.15.26:43302) 2023-12-19T16:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9132ffd640 1 -- 172.21.15.26:0/1686840482 learned_addr learned my addr 172.21.15.26:0/1686840482 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9133fff640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91341585f0 0x7f913414c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f91337fe640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91340cb150 0x7f9134153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9132ffd640 1 -- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91341585f0 msgr2=0x7f913414c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9132ffd640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91341585f0 0x7f913414c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:46.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9132ffd640 1 -- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91340cb150 msgr2=0x7f9134153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9132ffd640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91340cb150 0x7f9134153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9132ffd640 1 -- 172.21.15.26:0/1686840482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9128009420 con 0x7f91340cc310 2023-12-19T16:57:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9132ffd640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cc310 0x7f91340b2cf0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9128019ff0 tx=0x7f9128019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f911bfff640 1 -- 172.21.15.26:0/1686840482 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91280b5410 con 0x7f91340cc310 2023-12-19T16:57:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f911bfff640 1 -- 172.21.15.26:0/1686840482 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91280b5590 con 0x7f91340cc310 2023-12-19T16:57:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f911bfff640 1 -- 172.21.15.26:0/1686840482 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91280b5940 con 0x7f91340cc310 2023-12-19T16:57:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9139fce640 1 -- 172.21.15.26:0/1686840482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913415c3e0 con 0x7f91340cc310 2023-12-19T16:57:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9139fce640 1 -- 172.21.15.26:0/1686840482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913406d1e0 con 0x7f91340cc310 2023-12-19T16:57:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f9139fce640 1 -- 172.21.15.26:0/1686840482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913414c810 con 0x7f91340cc310 2023-12-19T16:57:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.921+0000 7f911bfff640 1 -- 172.21.15.26:0/1686840482 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f91280b3040 con 0x7f91340cc310 2023-12-19T16:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.925+0000 7f911bfff640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9120065b80 0x7f9120068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.925+0000 7f911bfff640 1 -- 172.21.15.26:0/1686840482 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f912811d670 con 0x7f91340cc310 2023-12-19T16:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.925+0000 7f91337fe640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9120065b80 0x7f9120068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:46.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.925+0000 7f91337fe640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9120065b80 0x7f9120068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f912400ba10 tx=0x7f9124008040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:46.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:46.929+0000 7f911bfff640 1 -- 172.21.15.26:0/1686840482 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f91280eac60 con 0x7f91340cc310 2023-12-19T16:57:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.085+0000 7f9139fce640 1 -- 172.21.15.26:0/1686840482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f9134074270 con 0x7f91340cc310 2023-12-19T16:57:47.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.089+0000 7f911bfff640 1 -- 172.21.15.26:0/1686840482 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v25) v1 ==== 120+0+0 (secure 0 0 0) 0x7f91280ee280 con 0x7f91340cc310 2023-12-19T16:57:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.093+0000 7f9139fce640 1 -- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9120065b80 msgr2=0x7f9120068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.093+0000 7f9139fce640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9120065b80 0x7f9120068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f912400ba10 tx=0x7f9124008040 comp rx=0 tx=0).stop 2023-12-19T16:57:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.093+0000 7f9139fce640 1 -- 172.21.15.26:0/1686840482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cc310 msgr2=0x7f91340b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.093+0000 7f9139fce640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cc310 0x7f91340b2cf0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9128019ff0 tx=0x7f9128019160 comp rx=0 tx=0).stop 2023-12-19T16:57:47.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.093+0000 7f9139fce640 1 -- 172.21.15.26:0/1686840482 shutdown_connections 2023-12-19T16:57:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.093+0000 7f9139fce640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9120065b80 0x7f9120068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.093+0000 7f9139fce640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91341585f0 0x7f913414c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.093+0000 7f9139fce640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cc310 0x7f91340b2cf0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.093+0000 7f9139fce640 1 --2- 172.21.15.26:0/1686840482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91340cb150 0x7f9134153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.093+0000 7f9139fce640 1 -- 172.21.15.26:0/1686840482 >> 172.21.15.26:0/1686840482 conn(0x7f9134066160 msgr2=0x7f913415a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.093+0000 7f9139fce640 1 -- 172.21.15.26:0/1686840482 shutdown_connections 2023-12-19T16:57:47.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.093+0000 7f9139fce640 1 -- 172.21.15.26:0/1686840482 wait complete. 2023-12-19T16:57:47.117 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T16:57:47.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd1f2b6640 1 Processor -- start 2023-12-19T16:57:47.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd1f2b6640 1 -- start start 2023-12-19T16:57:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd1f2b6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdd180d2660 0x7fdd18167930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd1f2b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd180d21d0 con 0x7fdd180b4600 2023-12-19T16:57:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd1f2b6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd180d12f0 con 0x7fdd180b8940 2023-12-19T16:57:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd1f2b6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd180d15c0 con 0x7fdd180d2660 2023-12-19T16:57:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd1d052640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdd180b4600 0x7fdd180b49e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55970/0 (socket says 172.21.15.26:55970) 2023-12-19T16:57:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd1d052640 1 -- 172.21.15.26:0/2686664094 learned_addr learned my addr 172.21.15.26:0/2686664094 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:47.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd1d853640 1 --2- 172.21.15.26:0/2686664094 >> v2:172.21.15.160:3300/0 conn(0x7fdd180d2660 0x7fdd18167930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd0ffff640 1 -- 172.21.15.26:0/2686664094 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024240156 0 0) 0x7fdd180d21d0 con 0x7fdd180b4600 2023-12-19T16:57:47.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd0ffff640 1 -- 172.21.15.26:0/2686664094 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd04003530 con 0x7fdd180b4600 2023-12-19T16:57:47.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd0ffff640 1 -- 172.21.15.26:0/2686664094 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1587543796 0 0) 0x7fdd180d12f0 con 0x7fdd180b8940 2023-12-19T16:57:47.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd0ffff640 1 -- 172.21.15.26:0/2686664094 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd180d21d0 con 0x7fdd180b8940 2023-12-19T16:57:47.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd1d853640 1 -- 172.21.15.26:0/2686664094 >> v1:172.21.15.160:6789/0 conn(0x7fdd180b8940 legacy=0x7fdd180d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd1d853640 1 -- 172.21.15.26:0/2686664094 >> v1:172.21.15.26:6789/0 conn(0x7fdd180b4600 legacy=0x7fdd180b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.261+0000 7fdd1d853640 1 -- 172.21.15.26:0/2686664094 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd180d18a0 con 0x7fdd180d2660 2023-12-19T16:57:47.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.265+0000 7fdd1d853640 1 --2- 172.21.15.26:0/2686664094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd180d2660 0x7fdd18167930 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009d90 tx=0x7fdd1001f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a745fc212a44074 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:47.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.265+0000 7fdd0ffff640 1 -- 172.21.15.26:0/2686664094 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd100af600 con 0x7fdd180d2660 2023-12-19T16:57:47.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.265+0000 7fdd0ffff640 1 -- 172.21.15.26:0/2686664094 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd100ab050 con 0x7fdd180d2660 2023-12-19T16:57:47.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.265+0000 7fdd0ffff640 1 -- 172.21.15.26:0/2686664094 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd100af980 con 0x7fdd180d2660 2023-12-19T16:57:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.265+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/2686664094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd180d2660 msgr2=0x7fdd18167930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.265+0000 7fdd1f2b6640 1 --2- 172.21.15.26:0/2686664094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd180d2660 0x7fdd18167930 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009d90 tx=0x7fdd1001f130 comp rx=0 tx=0).stop 2023-12-19T16:57:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.265+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/2686664094 shutdown_connections 2023-12-19T16:57:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.265+0000 7fdd1f2b6640 1 --2- 172.21.15.26:0/2686664094 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd180d2660 0x7fdd18167930 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.265+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/2686664094 >> 172.21.15.26:0/2686664094 conn(0x7fdd18065ee0 msgr2=0x7fdd180cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.265+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/2686664094 shutdown_connections 2023-12-19T16:57:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.265+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/2686664094 wait complete. 2023-12-19T16:57:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.269+0000 7fdd1f2b6640 1 Processor -- start 2023-12-19T16:57:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.269+0000 7fdd1f2b6640 1 -- start start 2023-12-19T16:57:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1f2b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd180b4600 0x7fdd181540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1f2b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180b8940 0x7fdd1814e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1f2b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd180d2660 0x7fdd18150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1f2b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1814e690 con 0x7fdd180b8940 2023-12-19T16:57:47.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1f2b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd181f6b10 con 0x7fdd180d2660 2023-12-19T16:57:47.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1f2b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd181f5f00 con 0x7fdd180b4600 2023-12-19T16:57:47.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1d052640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd180b4600 0x7fdd181540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1d052640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd180b4600 0x7fdd181540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:60998/0 (socket says 172.21.15.26:60998) 2023-12-19T16:57:47.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1d052640 1 -- 172.21.15.26:0/3911945723 learned_addr learned my addr 172.21.15.26:0/3911945723 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:47.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1d853640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd180d2660 0x7fdd18150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1c851640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180b8940 0x7fdd1814e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1d052640 1 -- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd180d2660 msgr2=0x7fdd18150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1d052640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd180d2660 0x7fdd18150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1d052640 1 -- 172.21.15.26:0/3911945723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180b8940 msgr2=0x7fdd1814e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1d052640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180b8940 0x7fdd1814e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1d052640 1 -- 172.21.15.26:0/3911945723 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd10009a70 con 0x7fdd180b4600 2023-12-19T16:57:47.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1c851640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180b8940 0x7fdd1814e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:47.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.273+0000 7fdd1d052640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd180b4600 0x7fdd181540e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400c4d0 tx=0x7fdd1400c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.277+0000 7fdd0dffb640 1 -- 172.21.15.26:0/3911945723 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd14005cf0 con 0x7fdd180b4600 2023-12-19T16:57:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.277+0000 7fdd0dffb640 1 -- 172.21.15.26:0/3911945723 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd14005e50 con 0x7fdd180b4600 2023-12-19T16:57:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.277+0000 7fdd0dffb640 1 -- 172.21.15.26:0/3911945723 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd14005cf0 con 0x7fdd180b4600 2023-12-19T16:57:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.277+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/3911945723 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd181f61e0 con 0x7fdd180b4600 2023-12-19T16:57:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.277+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/3911945723 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd181f67a0 con 0x7fdd180b4600 2023-12-19T16:57:47.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.277+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/3911945723 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd18074270 con 0x7fdd180b4600 2023-12-19T16:57:47.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.281+0000 7fdd0dffb640 1 -- 172.21.15.26:0/3911945723 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fdd14007070 con 0x7fdd180b4600 2023-12-19T16:57:47.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.281+0000 7fdd0dffb640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdcfc065be0 0x7fdcfc0680a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.281+0000 7fdd0dffb640 1 -- 172.21.15.26:0/3911945723 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fdd14082680 con 0x7fdd180b4600 2023-12-19T16:57:47.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.281+0000 7fdd1c851640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdcfc065be0 0x7fdcfc0680a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.285+0000 7fdd1c851640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdcfc065be0 0x7fdcfc0680a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdd1815c8d0 tx=0x7fdd08009040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:47.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.285+0000 7fdd0dffb640 1 -- 172.21.15.26:0/3911945723 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdd1404fd90 con 0x7fdd180b4600 2023-12-19T16:57:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.445+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/3911945723 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fdd180c92e0 con 0x7fdd180b4600 2023-12-19T16:57:47.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.453+0000 7fdd0dffb640 1 -- 172.21.15.26:0/3911945723 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdd140533b0 con 0x7fdd180b4600 2023-12-19T16:57:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.457+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdcfc065be0 msgr2=0x7fdcfc0680a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.457+0000 7fdd1f2b6640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdcfc065be0 0x7fdcfc0680a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdd1815c8d0 tx=0x7fdd08009040 comp rx=0 tx=0).stop 2023-12-19T16:57:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.457+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd180b4600 msgr2=0x7fdd181540e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.457+0000 7fdd1f2b6640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd180b4600 0x7fdd181540e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400c4d0 tx=0x7fdd1400c990 comp rx=0 tx=0).stop 2023-12-19T16:57:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.457+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/3911945723 shutdown_connections 2023-12-19T16:57:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.457+0000 7fdd1f2b6640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd180d2660 0x7fdd18150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.457+0000 7fdd1f2b6640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd180b8940 0x7fdd1814e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.457+0000 7fdd1f2b6640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdcfc065be0 0x7fdcfc0680a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.457+0000 7fdd1f2b6640 1 --2- 172.21.15.26:0/3911945723 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd180b4600 0x7fdd181540e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.457+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/3911945723 >> 172.21.15.26:0/3911945723 conn(0x7fdd18065ee0 msgr2=0x7fdd180b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.457+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/3911945723 shutdown_connections 2023-12-19T16:57:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.457+0000 7fdd1f2b6640 1 -- 172.21.15.26:0/3911945723 wait complete. 2023-12-19T16:57:47.480 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-19T16:57:47.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.621+0000 7fbf261f3640 1 Processor -- start 2023-12-19T16:57:47.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.621+0000 7fbf261f3640 1 -- start start 2023-12-19T16:57:47.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.621+0000 7fbf261f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf2015cd10 0x7fbf200cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.621+0000 7fbf261f3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbf2006d3e0 0x7fbf2006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.621+0000 7fbf261f3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2015d0f0 con 0x7fbf2006c130 2023-12-19T16:57:47.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.621+0000 7fbf261f3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2006c5c0 con 0x7fbf2006d3e0 2023-12-19T16:57:47.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.621+0000 7fbf261f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200c98b0 con 0x7fbf2015cd10 2023-12-19T16:57:47.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.621+0000 7fbf1f7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf2015cd10 0x7fbf200cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.621+0000 7fbf1effd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbf2006d3e0 0x7fbf2006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.621+0000 7fbf1f7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf2015cd10 0x7fbf200cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:32772/0 (socket says 172.21.15.26:32772) 2023-12-19T16:57:47.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.621+0000 7fbf1f7fe640 1 -- 172.21.15.26:0/1489110260 learned_addr learned my addr 172.21.15.26:0/1489110260 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:47.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.621+0000 7fbf1effd640 1 -- 172.21.15.26:0/1489110260 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf2015cd10 msgr2=0x7fbf200cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:47.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.621+0000 7fbf1effd640 1 --2- 172.21.15.26:0/1489110260 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf2015cd10 0x7fbf200cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf1effd640 1 -- 172.21.15.26:0/1489110260 >> v1:172.21.15.160:6789/0 conn(0x7fbf2006c130 legacy=0x7fbf20160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf1effd640 1 -- 172.21.15.26:0/1489110260 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf200cd0e0 con 0x7fbf2006d3e0 2023-12-19T16:57:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf1effd640 1 --2- 172.21.15.26:0/1489110260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2006d3e0 0x7fbf2006bbf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbf140097c0 tx=0x7fbf1401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7933211c32f2d3b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf1e7fc640 1 -- 172.21.15.26:0/1489110260 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf140a8970 con 0x7fbf2006d3e0 2023-12-19T16:57:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf1e7fc640 1 -- 172.21.15.26:0/1489110260 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf140a8ad0 con 0x7fbf2006d3e0 2023-12-19T16:57:47.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf1e7fc640 1 -- 172.21.15.26:0/1489110260 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf140a8e60 con 0x7fbf2006d3e0 2023-12-19T16:57:47.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf261f3640 1 -- 172.21.15.26:0/1489110260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2006d3e0 msgr2=0x7fbf2006bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf261f3640 1 --2- 172.21.15.26:0/1489110260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2006d3e0 0x7fbf2006bbf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbf140097c0 tx=0x7fbf1401ec70 comp rx=0 tx=0).stop 2023-12-19T16:57:47.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf261f3640 1 -- 172.21.15.26:0/1489110260 shutdown_connections 2023-12-19T16:57:47.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf261f3640 1 --2- 172.21.15.26:0/1489110260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2006d3e0 0x7fbf2006bbf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf261f3640 1 --2- 172.21.15.26:0/1489110260 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf2015cd10 0x7fbf200cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf261f3640 1 -- 172.21.15.26:0/1489110260 >> 172.21.15.26:0/1489110260 conn(0x7fbf200cf260 msgr2=0x7fbf200cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:47.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf261f3640 1 -- 172.21.15.26:0/1489110260 shutdown_connections 2023-12-19T16:57:47.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.625+0000 7fbf261f3640 1 -- 172.21.15.26:0/1489110260 wait complete. 2023-12-19T16:57:47.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf261f3640 1 Processor -- start 2023-12-19T16:57:47.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf261f3640 1 -- start start 2023-12-19T16:57:47.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf261f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf2006c130 0x7fbf200c4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf261f3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2006d3e0 0x7fbf201587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf261f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf2015cd10 0x7fbf200c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf261f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200c4840 con 0x7fbf2006c130 2023-12-19T16:57:47.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf261f3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200c35d0 con 0x7fbf2006d3e0 2023-12-19T16:57:47.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf261f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf200b2c60 con 0x7fbf2015cd10 2023-12-19T16:57:47.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf1f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf2006c130 0x7fbf200c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf1f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf2006c130 0x7fbf200c4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43320/0 (socket says 172.21.15.26:43320) 2023-12-19T16:57:47.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf1f7fe640 1 -- 172.21.15.26:0/1029871224 learned_addr learned my addr 172.21.15.26:0/1029871224 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:47.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf1effd640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2006d3e0 0x7fbf201587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf1ffff640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf2015cd10 0x7fbf200c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf1f7fe640 1 -- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf2015cd10 msgr2=0x7fbf200c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf1f7fe640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf2015cd10 0x7fbf200c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf1f7fe640 1 -- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2006d3e0 msgr2=0x7fbf201587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf1f7fe640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2006d3e0 0x7fbf201587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf1f7fe640 1 -- 172.21.15.26:0/1029871224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf14009420 con 0x7fbf2006c130 2023-12-19T16:57:47.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf1ffff640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf2015cd10 0x7fbf200c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:47.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf1effd640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2006d3e0 0x7fbf201587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:47.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.629+0000 7fbf1f7fe640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf2006c130 0x7fbf200c4300 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000e7f0 tx=0x7fbf1000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:47.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.633+0000 7fbf07fff640 1 -- 172.21.15.26:0/1029871224 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10009580 con 0x7fbf2006c130 2023-12-19T16:57:47.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.633+0000 7fbf07fff640 1 -- 172.21.15.26:0/1029871224 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1000c940 con 0x7fbf2006c130 2023-12-19T16:57:47.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.633+0000 7fbf07fff640 1 -- 172.21.15.26:0/1029871224 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1000ccf0 con 0x7fbf2006c130 2023-12-19T16:57:47.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.633+0000 7fbf261f3640 1 -- 172.21.15.26:0/1029871224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf200c3820 con 0x7fbf2006c130 2023-12-19T16:57:47.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.633+0000 7fbf261f3640 1 -- 172.21.15.26:0/1029871224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf200c3de0 con 0x7fbf2006c130 2023-12-19T16:57:47.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.633+0000 7fbf261f3640 1 -- 172.21.15.26:0/1029871224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf20074270 con 0x7fbf2006c130 2023-12-19T16:57:47.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.633+0000 7fbf07fff640 1 -- 172.21.15.26:0/1029871224 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbf10013470 con 0x7fbf2006c130 2023-12-19T16:57:47.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.633+0000 7fbf07fff640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbf0c065bb0 0x7fbf0c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.633+0000 7fbf07fff640 1 -- 172.21.15.26:0/1029871224 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fbf10083a60 con 0x7fbf2006c130 2023-12-19T16:57:47.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.633+0000 7fbf1effd640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbf0c065bb0 0x7fbf0c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.637+0000 7fbf1effd640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbf0c065bb0 0x7fbf0c068070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbf140b2000 tx=0x7fbf14008040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:47.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.637+0000 7fbf07fff640 1 -- 172.21.15.26:0/1029871224 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbf10051030 con 0x7fbf2006c130 2023-12-19T16:57:47.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.797+0000 7fbf261f3640 1 -- 172.21.15.26:0/1029871224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fbf2006ca30 con 0x7fbf2006c130 2023-12-19T16:57:47.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.801+0000 7fbf07fff640 1 -- 172.21.15.26:0/1029871224 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v27) v1 ==== 115+0+0 (secure 0 0 0) 0x7fbf100170b0 con 0x7fbf2006c130 2023-12-19T16:57:47.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.805+0000 7fbf261f3640 1 -- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbf0c065bb0 msgr2=0x7fbf0c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.805+0000 7fbf261f3640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbf0c065bb0 0x7fbf0c068070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbf140b2000 tx=0x7fbf14008040 comp rx=0 tx=0).stop 2023-12-19T16:57:47.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.805+0000 7fbf261f3640 1 -- 172.21.15.26:0/1029871224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf2006c130 msgr2=0x7fbf200c4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.805+0000 7fbf261f3640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf2006c130 0x7fbf200c4300 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000e7f0 tx=0x7fbf1000ecb0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.805+0000 7fbf261f3640 1 -- 172.21.15.26:0/1029871224 shutdown_connections 2023-12-19T16:57:47.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.805+0000 7fbf261f3640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbf0c065bb0 0x7fbf0c068070 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.805+0000 7fbf261f3640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf2015cd10 0x7fbf200c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.805+0000 7fbf261f3640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf2006d3e0 0x7fbf201587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.805+0000 7fbf261f3640 1 --2- 172.21.15.26:0/1029871224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf2006c130 0x7fbf200c4300 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.805+0000 7fbf261f3640 1 -- 172.21.15.26:0/1029871224 >> 172.21.15.26:0/1029871224 conn(0x7fbf200cf260 msgr2=0x7fbf200cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:47.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.805+0000 7fbf261f3640 1 -- 172.21.15.26:0/1029871224 shutdown_connections 2023-12-19T16:57:47.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.805+0000 7fbf261f3640 1 -- 172.21.15.26:0/1029871224 wait complete. 2023-12-19T16:57:47.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-12-19T16:57:47.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.965+0000 7f42d616d640 1 Processor -- start 2023-12-19T16:57:47.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.965+0000 7f42d616d640 1 -- start start 2023-12-19T16:57:47.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42d616d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f42d0154220 0x7f42d015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42d616d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f42d00cee10 0x7f42d015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42d616d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d006bc30 con 0x7f42d006b3d0 2023-12-19T16:57:47.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42d616d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d00cc6e0 con 0x7f42d00cee10 2023-12-19T16:57:47.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42d616d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d015afc0 con 0x7f42d0154220 2023-12-19T16:57:47.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42cffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f42d00cee10 0x7f42d015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42ceffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f42d006b3d0 0x7f42d00ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55984/0 (socket says 172.21.15.26:55984) 2023-12-19T16:57:47.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42ceffd640 1 -- 172.21.15.26:0/1615776705 learned_addr learned my addr 172.21.15.26:0/1615776705 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:47.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42cf7fe640 1 --2- 172.21.15.26:0/1615776705 >> v2:172.21.15.160:3300/0 conn(0x7f42d0154220 0x7f42d015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42cffff640 1 -- 172.21.15.26:0/1615776705 >> v2:172.21.15.160:3300/0 conn(0x7f42d0154220 msgr2=0x7f42d015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:47.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42cffff640 1 --2- 172.21.15.26:0/1615776705 >> v2:172.21.15.160:3300/0 conn(0x7f42d0154220 0x7f42d015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42cffff640 1 -- 172.21.15.26:0/1615776705 >> v1:172.21.15.26:6789/0 conn(0x7f42d006b3d0 legacy=0x7f42d00ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42cffff640 1 -- 172.21.15.26:0/1615776705 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d006b950 con 0x7f42d00cee10 2023-12-19T16:57:47.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.969+0000 7f42ce7fc640 1 -- 172.21.15.26:0/1615776705 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317932392 0 0) 0x7f42d006bc30 con 0x7f42d006b3d0 2023-12-19T16:57:47.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42cffff640 1 --2- 172.21.15.26:0/1615776705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d00cee10 0x7f42d015b1c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f42d00667b0 tx=0x7f42c801eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d495d29a205817 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:47.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42ce7fc640 1 -- 172.21.15.26:0/1615776705 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c80a7bc0 con 0x7f42d00cee10 2023-12-19T16:57:47.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42ce7fc640 1 -- 172.21.15.26:0/1615776705 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c80a7d20 con 0x7f42d00cee10 2023-12-19T16:57:47.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42ce7fc640 1 -- 172.21.15.26:0/1615776705 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c80af660 con 0x7f42d00cee10 2023-12-19T16:57:47.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42d616d640 1 -- 172.21.15.26:0/1615776705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d00cee10 msgr2=0x7f42d015b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42d616d640 1 --2- 172.21.15.26:0/1615776705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d00cee10 0x7f42d015b1c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f42d00667b0 tx=0x7f42c801eaf0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42d616d640 1 -- 172.21.15.26:0/1615776705 shutdown_connections 2023-12-19T16:57:47.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42d616d640 1 --2- 172.21.15.26:0/1615776705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d00cee10 0x7f42d015b1c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42d616d640 1 --2- 172.21.15.26:0/1615776705 >> v2:172.21.15.160:3300/0 conn(0x7f42d0154220 0x7f42d015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42d616d640 1 -- 172.21.15.26:0/1615776705 >> 172.21.15.26:0/1615776705 conn(0x7f42d00658e0 msgr2=0x7f42d0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:47.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42d616d640 1 -- 172.21.15.26:0/1615776705 shutdown_connections 2023-12-19T16:57:47.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42d616d640 1 -- 172.21.15.26:0/1615776705 wait complete. 2023-12-19T16:57:47.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42d616d640 1 Processor -- start 2023-12-19T16:57:47.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.973+0000 7f42d616d640 1 -- start start 2023-12-19T16:57:47.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42d616d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42d006b3d0 0x7f42d013af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42d616d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d00cee10 0x7f42d014f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42d616d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42d0154220 0x7f42d01f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42d616d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d014ec40 con 0x7f42d00cee10 2023-12-19T16:57:47.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42d616d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d014f1c0 con 0x7f42d0154220 2023-12-19T16:57:47.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42d616d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d01f9050 con 0x7f42d006b3d0 2023-12-19T16:57:47.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42ceffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d00cee10 0x7f42d014f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42ceffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d00cee10 0x7f42d014f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43344/0 (socket says 172.21.15.26:43344) 2023-12-19T16:57:47.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42ceffd640 1 -- 172.21.15.26:0/1435414257 learned_addr learned my addr 172.21.15.26:0/1435414257 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:47.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42cffff640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42d0154220 0x7f42d01f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42cf7fe640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42d006b3d0 0x7f42d013af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42ceffd640 1 -- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42d006b3d0 msgr2=0x7f42d013af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42ceffd640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42d006b3d0 0x7f42d013af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42ceffd640 1 -- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42d0154220 msgr2=0x7f42d01f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:47.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42ceffd640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42d0154220 0x7f42d01f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:47.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42ceffd640 1 -- 172.21.15.26:0/1435414257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c8009a70 con 0x7f42d00cee10 2023-12-19T16:57:47.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42cffff640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42d0154220 0x7f42d01f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:47.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42ceffd640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d00cee10 0x7f42d014f650 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f42c400c480 tx=0x7f42c400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:47.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42b7fff640 1 -- 172.21.15.26:0/1435414257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c40074e0 con 0x7f42d00cee10 2023-12-19T16:57:47.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42b7fff640 1 -- 172.21.15.26:0/1435414257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c4007660 con 0x7f42d00cee10 2023-12-19T16:57:47.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42b7fff640 1 -- 172.21.15.26:0/1435414257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4004a30 con 0x7f42d00cee10 2023-12-19T16:57:47.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42d616d640 1 -- 172.21.15.26:0/1435414257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d01f9330 con 0x7f42d00cee10 2023-12-19T16:57:47.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42d616d640 1 -- 172.21.15.26:0/1435414257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d014fb90 con 0x7f42d00cee10 2023-12-19T16:57:47.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42d616d640 1 -- 172.21.15.26:0/1435414257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d0074270 con 0x7f42d00cee10 2023-12-19T16:57:47.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.977+0000 7f42b7fff640 1 -- 172.21.15.26:0/1435414257 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f42c4004b90 con 0x7f42d00cee10 2023-12-19T16:57:47.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.981+0000 7f42b7fff640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f42bc065b80 0x7f42bc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:47.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.981+0000 7f42b7fff640 1 -- 172.21.15.26:0/1435414257 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f42c4082710 con 0x7f42d00cee10 2023-12-19T16:57:47.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.981+0000 7f42cf7fe640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f42bc065b80 0x7f42bc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:47.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.981+0000 7f42cf7fe640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f42bc065b80 0x7f42bc068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009af0 tx=0x7f42c0008040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:47.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:47.985+0000 7f42b7fff640 1 -- 172.21.15.26:0/1435414257 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f42c40538a0 con 0x7f42d00cee10 2023-12-19T16:57:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.197+0000 7f42d616d640 1 -- 172.21.15.26:0/1435414257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f42d00cdf60 con 0x7f42d00cee10 2023-12-19T16:57:48.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.733+0000 7f42b7fff640 1 -- 172.21.15.26:0/1435414257 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v565) v1 ==== 156+0+0 (secure 0 0 0) 0x7f42c4053430 con 0x7f42d00cee10 2023-12-19T16:57:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.733+0000 7f42d616d640 1 -- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f42bc065b80 msgr2=0x7f42bc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.733+0000 7f42d616d640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f42bc065b80 0x7f42bc068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009af0 tx=0x7f42c0008040 comp rx=0 tx=0).stop 2023-12-19T16:57:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.733+0000 7f42d616d640 1 -- 172.21.15.26:0/1435414257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d00cee10 msgr2=0x7f42d014f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.733+0000 7f42d616d640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d00cee10 0x7f42d014f650 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f42c400c480 tx=0x7f42c400c940 comp rx=0 tx=0).stop 2023-12-19T16:57:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.733+0000 7f42d616d640 1 -- 172.21.15.26:0/1435414257 shutdown_connections 2023-12-19T16:57:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.733+0000 7f42d616d640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42d0154220 0x7f42d01f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.733+0000 7f42d616d640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d00cee10 0x7f42d014f650 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.733+0000 7f42d616d640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f42bc065b80 0x7f42bc068040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.733+0000 7f42d616d640 1 --2- 172.21.15.26:0/1435414257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42d006b3d0 0x7f42d013af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.733+0000 7f42d616d640 1 -- 172.21.15.26:0/1435414257 >> 172.21.15.26:0/1435414257 conn(0x7f42d00658e0 msgr2=0x7f42d015dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:48.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.733+0000 7f42d616d640 1 -- 172.21.15.26:0/1435414257 shutdown_connections 2023-12-19T16:57:48.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.733+0000 7f42d616d640 1 -- 172.21.15.26:0/1435414257 wait complete. 2023-12-19T16:57:48.762 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-12-19T16:57:48.762 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-12-19T16:57:48.763 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-19T16:57:48.763 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-12-19T16:57:48.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.905+0000 7f0e699a6640 1 Processor -- start 2023-12-19T16:57:48.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.905+0000 7f0e699a6640 1 -- start start 2023-12-19T16:57:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e699a6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e64154220 0x7f0e6415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e699a6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0e6406b3d0 0x7f0e640ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:48.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e699a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e640cec40 0x7f0e6415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e699a6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e640cf0d0 con 0x7f0e64154220 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e699a6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e6406bc30 con 0x7f0e6406b3d0 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e699a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640cc6e0 con 0x7f0e640cec40 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e62ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e64154220 0x7f0e6415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e62ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e64154220 0x7f0e6415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43356/0 (socket says 172.21.15.26:43356) 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e62ffd640 1 -- 172.21.15.26:0/92907528 learned_addr learned my addr 172.21.15.26:0/92907528 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e5bfff640 1 --2- 172.21.15.26:0/92907528 >> v2:172.21.15.160:3300/0 conn(0x7f0e6406b3d0 0x7f0e640ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e637fe640 1 --2- 172.21.15.26:0/92907528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e640cec40 0x7f0e6415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e62ffd640 1 -- 172.21.15.26:0/92907528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e640cec40 msgr2=0x7f0e6415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e62ffd640 1 --2- 172.21.15.26:0/92907528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e640cec40 0x7f0e6415d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e62ffd640 1 -- 172.21.15.26:0/92907528 >> v2:172.21.15.160:3300/0 conn(0x7f0e6406b3d0 msgr2=0x7f0e640ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e62ffd640 1 --2- 172.21.15.26:0/92907528 >> v2:172.21.15.160:3300/0 conn(0x7f0e6406b3d0 0x7f0e640ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e62ffd640 1 -- 172.21.15.26:0/92907528 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6415afc0 con 0x7f0e64154220 2023-12-19T16:57:48.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e637fe640 1 --2- 172.21.15.26:0/92907528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e640cec40 0x7f0e6415d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:48.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.909+0000 7f0e5bfff640 1 --2- 172.21.15.26:0/92907528 >> v2:172.21.15.160:3300/0 conn(0x7f0e6406b3d0 0x7f0e640ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:48.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e62ffd640 1 --2- 172.21.15.26:0/92907528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e64154220 0x7f0e6415aa80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0e640cf880 tx=0x7f0e5401c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b007d70ee576fca server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e627fc640 1 -- 172.21.15.26:0/92907528 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e540ab600 con 0x7f0e64154220 2023-12-19T16:57:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e627fc640 1 -- 172.21.15.26:0/92907528 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e540a7050 con 0x7f0e64154220 2023-12-19T16:57:48.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e627fc640 1 -- 172.21.15.26:0/92907528 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e540ab980 con 0x7f0e64154220 2023-12-19T16:57:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e699a6640 1 -- 172.21.15.26:0/92907528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e64154220 msgr2=0x7f0e6415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e699a6640 1 --2- 172.21.15.26:0/92907528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e64154220 0x7f0e6415aa80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0e640cf880 tx=0x7f0e5401c210 comp rx=0 tx=0).stop 2023-12-19T16:57:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e699a6640 1 -- 172.21.15.26:0/92907528 shutdown_connections 2023-12-19T16:57:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e699a6640 1 --2- 172.21.15.26:0/92907528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e640cec40 0x7f0e6415d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e699a6640 1 --2- 172.21.15.26:0/92907528 >> v2:172.21.15.160:3300/0 conn(0x7f0e6406b3d0 0x7f0e640ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e699a6640 1 --2- 172.21.15.26:0/92907528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e64154220 0x7f0e6415aa80 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e699a6640 1 -- 172.21.15.26:0/92907528 >> 172.21.15.26:0/92907528 conn(0x7f0e640658e0 msgr2=0x7f0e64065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e699a6640 1 -- 172.21.15.26:0/92907528 shutdown_connections 2023-12-19T16:57:48.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e699a6640 1 -- 172.21.15.26:0/92907528 wait complete. 2023-12-19T16:57:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e699a6640 1 Processor -- start 2023-12-19T16:57:48.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.913+0000 7f0e699a6640 1 -- start start 2023-12-19T16:57:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e699a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e6406b3d0 0x7f0e64143a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e699a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e640cec40 0x7f0e64143340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e699a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e64154220 0x7f0e64150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e699a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e64143880 con 0x7f0e640cec40 2023-12-19T16:57:48.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e699a6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6414f760 con 0x7f0e6406b3d0 2023-12-19T16:57:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e699a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6414fd20 con 0x7f0e64154220 2023-12-19T16:57:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e5bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e640cec40 0x7f0e64143340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e5bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e640cec40 0x7f0e64143340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43366/0 (socket says 172.21.15.26:43366) 2023-12-19T16:57:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e5bfff640 1 -- 172.21.15.26:0/3926322043 learned_addr learned my addr 172.21.15.26:0/3926322043 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e62ffd640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e6406b3d0 0x7f0e64143a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e637fe640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e64154220 0x7f0e64150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e5bfff640 1 -- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e64154220 msgr2=0x7f0e64150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e5bfff640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e64154220 0x7f0e64150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e5bfff640 1 -- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e6406b3d0 msgr2=0x7f0e64143a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e5bfff640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e6406b3d0 0x7f0e64143a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e5bfff640 1 -- 172.21.15.26:0/3926322043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e540094d0 con 0x7f0e640cec40 2023-12-19T16:57:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e62ffd640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e6406b3d0 0x7f0e64143a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:48.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e5bfff640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e640cec40 0x7f0e64143340 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00d830 tx=0x7f0e4c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:48.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e60ff9640 1 -- 172.21.15.26:0/3926322043 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c00be10 con 0x7f0e640cec40 2023-12-19T16:57:48.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e60ff9640 1 -- 172.21.15.26:0/3926322043 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4c004030 con 0x7f0e640cec40 2023-12-19T16:57:48.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e699a6640 1 -- 172.21.15.26:0/3926322043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e640b3200 con 0x7f0e640cec40 2023-12-19T16:57:48.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e60ff9640 1 -- 172.21.15.26:0/3926322043 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4c0043c0 con 0x7f0e640cec40 2023-12-19T16:57:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e699a6640 1 -- 172.21.15.26:0/3926322043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6414f9b0 con 0x7f0e640cec40 2023-12-19T16:57:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.917+0000 7f0e699a6640 1 -- 172.21.15.26:0/3926322043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e640b3720 con 0x7f0e640cec40 2023-12-19T16:57:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.921+0000 7f0e60ff9640 1 -- 172.21.15.26:0/3926322043 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0e4c0098b0 con 0x7f0e640cec40 2023-12-19T16:57:48.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.921+0000 7f0e60ff9640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e44065b80 0x7f0e44068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:48.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.921+0000 7f0e60ff9640 1 -- 172.21.15.26:0/3926322043 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f0e4c083780 con 0x7f0e640cec40 2023-12-19T16:57:48.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.921+0000 7f0e62ffd640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e44065b80 0x7f0e44068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:48.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.921+0000 7f0e62ffd640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e44065b80 0x7f0e44068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0e540ad000 tx=0x7f0e540a6000 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:48.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:48.925+0000 7f0e60ff9640 1 -- 172.21.15.26:0/3926322043 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0e4c050de0 con 0x7f0e640cec40 2023-12-19T16:57:49.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.133+0000 7f0e699a6640 1 -- 172.21.15.26:0/3926322043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f0e64074270 con 0x7f0e640cec40 2023-12-19T16:57:49.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.741+0000 7f0e60ff9640 1 -- 172.21.15.26:0/3926322043 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v566) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0e4c00b3a0 con 0x7f0e640cec40 2023-12-19T16:57:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.741+0000 7f0e699a6640 1 -- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e44065b80 msgr2=0x7f0e44068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.741+0000 7f0e699a6640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e44065b80 0x7f0e44068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0e540ad000 tx=0x7f0e540a6000 comp rx=0 tx=0).stop 2023-12-19T16:57:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.741+0000 7f0e699a6640 1 -- 172.21.15.26:0/3926322043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e640cec40 msgr2=0x7f0e64143340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.741+0000 7f0e699a6640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e640cec40 0x7f0e64143340 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c00d830 tx=0x7f0e4c00dcf0 comp rx=0 tx=0).stop 2023-12-19T16:57:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.741+0000 7f0e699a6640 1 -- 172.21.15.26:0/3926322043 shutdown_connections 2023-12-19T16:57:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.741+0000 7f0e699a6640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0e44065b80 0x7f0e44068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.741+0000 7f0e699a6640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e64154220 0x7f0e64150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.741+0000 7f0e699a6640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e640cec40 0x7f0e64143340 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:49.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.741+0000 7f0e699a6640 1 --2- 172.21.15.26:0/3926322043 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e6406b3d0 0x7f0e64143a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.741+0000 7f0e699a6640 1 -- 172.21.15.26:0/3926322043 >> 172.21.15.26:0/3926322043 conn(0x7f0e640658e0 msgr2=0x7f0e640cb240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.741+0000 7f0e699a6640 1 -- 172.21.15.26:0/3926322043 shutdown_connections 2023-12-19T16:57:49.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.741+0000 7f0e699a6640 1 -- 172.21.15.26:0/3926322043 wait complete. 2023-12-19T16:57:49.769 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T16:57:49.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.913+0000 7fc89adce640 1 Processor -- start 2023-12-19T16:57:49.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.913+0000 7fc89adce640 1 -- start start 2023-12-19T16:57:49.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.913+0000 7fc89adce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc89415a9c0 0x7fc8940706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:49.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.913+0000 7fc89adce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc89415ada0 0x7fc8940b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:49.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.913+0000 7fc89adce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8940b46c0 con 0x7fc89406ac80 2023-12-19T16:57:49.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.913+0000 7fc89adce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8940cc270 con 0x7fc89415ada0 2023-12-19T16:57:49.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.913+0000 7fc89adce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc894070d90 con 0x7fc89415a9c0 2023-12-19T16:57:49.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.913+0000 7fc89936b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc89406ac80 0x7fc8940b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55994/0 (socket says 172.21.15.26:55994) 2023-12-19T16:57:49.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.913+0000 7fc89936b640 1 -- 172.21.15.26:0/520040718 learned_addr learned my addr 172.21.15.26:0/520040718 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:49.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.917+0000 7fc893fff640 1 --2- 172.21.15.26:0/520040718 >> v2:172.21.15.160:3300/0 conn(0x7fc89415ada0 0x7fc8940b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.917+0000 7fc898b6a640 1 --2- 172.21.15.26:0/520040718 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc89415a9c0 0x7fc8940706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.917+0000 7fc8937fe640 1 -- 172.21.15.26:0/520040718 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492614919 0 0) 0x7fc8940b46c0 con 0x7fc89406ac80 2023-12-19T16:57:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.917+0000 7fc8937fe640 1 -- 172.21.15.26:0/520040718 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc884003530 con 0x7fc89406ac80 2023-12-19T16:57:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.917+0000 7fc898b6a640 1 -- 172.21.15.26:0/520040718 >> v2:172.21.15.160:3300/0 conn(0x7fc89415ada0 msgr2=0x7fc8940b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.917+0000 7fc898b6a640 1 --2- 172.21.15.26:0/520040718 >> v2:172.21.15.160:3300/0 conn(0x7fc89415ada0 0x7fc8940b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.917+0000 7fc8937fe640 1 -- 172.21.15.26:0/520040718 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2271896941 0 0) 0x7fc884003530 con 0x7fc89406ac80 2023-12-19T16:57:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.917+0000 7fc898b6a640 1 -- 172.21.15.26:0/520040718 >> v1:172.21.15.26:6789/0 conn(0x7fc89406ac80 legacy=0x7fc8940b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:49.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.917+0000 7fc898b6a640 1 -- 172.21.15.26:0/520040718 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc894071070 con 0x7fc89415a9c0 2023-12-19T16:57:49.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.917+0000 7fc898b6a640 1 --2- 172.21.15.26:0/520040718 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc89415a9c0 0x7fc8940706d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc8880097c0 tx=0x7fc88801c2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b67ab6038a9f2fe1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.917+0000 7fc8937fe640 1 -- 172.21.15.26:0/520040718 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c001a00 con 0x7fc89415a9c0 2023-12-19T16:57:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc8937fe640 1 -- 172.21.15.26:0/520040718 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8880a7050 con 0x7fc89415a9c0 2023-12-19T16:57:49.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc8937fe640 1 -- 172.21.15.26:0/520040718 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8880ab620 con 0x7fc89415a9c0 2023-12-19T16:57:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 -- 172.21.15.26:0/520040718 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc89415a9c0 msgr2=0x7fc8940706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 --2- 172.21.15.26:0/520040718 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc89415a9c0 0x7fc8940706d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc8880097c0 tx=0x7fc88801c2e0 comp rx=0 tx=0).stop 2023-12-19T16:57:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 -- 172.21.15.26:0/520040718 shutdown_connections 2023-12-19T16:57:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 --2- 172.21.15.26:0/520040718 >> v2:172.21.15.160:3300/0 conn(0x7fc89415ada0 0x7fc8940b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 --2- 172.21.15.26:0/520040718 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc89415a9c0 0x7fc8940706d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 -- 172.21.15.26:0/520040718 >> 172.21.15.26:0/520040718 conn(0x7fc8940ce730 msgr2=0x7fc8940ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 -- 172.21.15.26:0/520040718 shutdown_connections 2023-12-19T16:57:49.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 -- 172.21.15.26:0/520040718 wait complete. 2023-12-19T16:57:49.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 Processor -- start 2023-12-19T16:57:49.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 -- start start 2023-12-19T16:57:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc89406ac80 0x7fc894154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc89415a9c0 0x7fc8940ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc89415ada0 0x7fc8940ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:49.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89414d4f0 con 0x7fc89415ada0 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc894155450 con 0x7fc89406ac80 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.921+0000 7fc89adce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8940ca5e0 con 0x7fc89415a9c0 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc89936b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc89415ada0 0x7fc8940ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc89936b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc89415ada0 0x7fc8940ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43374/0 (socket says 172.21.15.26:43374) 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc89936b640 1 -- 172.21.15.26:0/2602663474 learned_addr learned my addr 172.21.15.26:0/2602663474 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc893fff640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc89415a9c0 0x7fc8940ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc898b6a640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc89406ac80 0x7fc894154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc89936b640 1 -- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc89415a9c0 msgr2=0x7fc8940ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc89936b640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc89415a9c0 0x7fc8940ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc89936b640 1 -- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc89406ac80 msgr2=0x7fc894154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc89936b640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc89406ac80 0x7fc894154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc89936b640 1 -- 172.21.15.26:0/2602663474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc888009420 con 0x7fc89415ada0 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc893fff640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc89415a9c0 0x7fc8940ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:49.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc898b6a640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc89406ac80 0x7fc894154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc89936b640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc89415ada0 0x7fc8940ac860 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0018d0 tx=0x7fc88c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc8917fa640 1 -- 172.21.15.26:0/2602663474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c012400 con 0x7fc89415ada0 2023-12-19T16:57:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc8917fa640 1 -- 172.21.15.26:0/2602663474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c012560 con 0x7fc89415ada0 2023-12-19T16:57:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc8917fa640 1 -- 172.21.15.26:0/2602663474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c012910 con 0x7fc89415ada0 2023-12-19T16:57:49.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc89adce640 1 -- 172.21.15.26:0/2602663474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8940c5770 con 0x7fc89415ada0 2023-12-19T16:57:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc89adce640 1 -- 172.21.15.26:0/2602663474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc894155960 con 0x7fc89415ada0 2023-12-19T16:57:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc89adce640 1 -- 172.21.15.26:0/2602663474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc894074270 con 0x7fc89415ada0 2023-12-19T16:57:49.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc8917fa640 1 -- 172.21.15.26:0/2602663474 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc88c012a70 con 0x7fc89415ada0 2023-12-19T16:57:49.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.925+0000 7fc8917fa640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc878065bb0 0x7fc878068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:49.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.929+0000 7fc8917fa640 1 -- 172.21.15.26:0/2602663474 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fc88c083d50 con 0x7fc89415ada0 2023-12-19T16:57:49.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.929+0000 7fc898b6a640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc878065bb0 0x7fc878068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:49.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.929+0000 7fc898b6a640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc878065bb0 0x7fc878068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc8880ad000 tx=0x7fc888008040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:49.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:49.933+0000 7fc8917fa640 1 -- 172.21.15.26:0/2602663474 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc88c051320 con 0x7fc89415ada0 2023-12-19T16:57:50.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.113+0000 7fc89adce640 1 -- 172.21.15.26:0/2602663474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc8940cd7b0 con 0x7fc89415ada0 2023-12-19T16:57:50.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.117+0000 7fc8917fa640 1 -- 172.21.15.26:0/2602663474 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc88c054940 con 0x7fc89415ada0 2023-12-19T16:57:50.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.117+0000 7fc89adce640 1 -- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc878065bb0 msgr2=0x7fc878068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:50.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.117+0000 7fc89adce640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc878065bb0 0x7fc878068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc8880ad000 tx=0x7fc888008040 comp rx=0 tx=0).stop 2023-12-19T16:57:50.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.117+0000 7fc89adce640 1 -- 172.21.15.26:0/2602663474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc89415ada0 msgr2=0x7fc8940ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:50.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.117+0000 7fc89adce640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc89415ada0 0x7fc8940ac860 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0018d0 tx=0x7fc88c00a430 comp rx=0 tx=0).stop 2023-12-19T16:57:50.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.121+0000 7fc89adce640 1 -- 172.21.15.26:0/2602663474 shutdown_connections 2023-12-19T16:57:50.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.121+0000 7fc89adce640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc89415ada0 0x7fc8940ac860 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:50.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.121+0000 7fc89adce640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc878065bb0 0x7fc878068070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:50.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.121+0000 7fc89adce640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc89415a9c0 0x7fc8940ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:50.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.121+0000 7fc89adce640 1 --2- 172.21.15.26:0/2602663474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc89406ac80 0x7fc894154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:50.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.121+0000 7fc89adce640 1 -- 172.21.15.26:0/2602663474 >> 172.21.15.26:0/2602663474 conn(0x7fc8940ce730 msgr2=0x7fc894156f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:50.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.121+0000 7fc89adce640 1 -- 172.21.15.26:0/2602663474 shutdown_connections 2023-12-19T16:57:50.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.121+0000 7fc89adce640 1 -- 172.21.15.26:0/2602663474 wait complete. 2023-12-19T16:57:50.144 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T16:57:50.144 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:57:50.226 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:57:50.323 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T16:57:50.324 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:57:50.324 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:50.333 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:57:50.333 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:57:50.333 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:50.343 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:57:50.343 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:57:50.343 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:50.384 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:57:50.385 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:57:50.385 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:57:50.397 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:57:50.397 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T16:57:50.397 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:57:50.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.569+0000 7fe7c78d1640 1 Processor -- start 2023-12-19T16:57:50.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.569+0000 7fe7c78d1640 1 -- start start 2023-12-19T16:57:50.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7c78d1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe7c0154220 0x7fe7c015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:50.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7c78d1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe7c00cee10 0x7fe7c015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:50.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7c78d1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c006bc30 con 0x7fe7c006b3d0 2023-12-19T16:57:50.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7c78d1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c00cc6e0 con 0x7fe7c0154220 2023-12-19T16:57:50.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7c78d1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c015afc0 con 0x7fe7c00cee10 2023-12-19T16:57:50.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7c4e6c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe7c006b3d0 0x7fe7c00ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56010/0 (socket says 172.21.15.26:56010) 2023-12-19T16:57:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7c4e6c640 1 -- 172.21.15.26:0/1119321093 learned_addr learned my addr 172.21.15.26:0/1119321093 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7c566d640 1 --2- 172.21.15.26:0/1119321093 >> v2:172.21.15.26:3300/0 conn(0x7fe7c0154220 0x7fe7c015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7c5e6e640 1 --2- 172.21.15.26:0/1119321093 >> v2:172.21.15.160:3300/0 conn(0x7fe7c00cee10 0x7fe7c015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7c566d640 1 -- 172.21.15.26:0/1119321093 >> v2:172.21.15.160:3300/0 conn(0x7fe7c00cee10 msgr2=0x7fe7c015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7c566d640 1 --2- 172.21.15.26:0/1119321093 >> v2:172.21.15.160:3300/0 conn(0x7fe7c00cee10 0x7fe7c015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7c566d640 1 -- 172.21.15.26:0/1119321093 >> v1:172.21.15.26:6789/0 conn(0x7fe7c006b3d0 legacy=0x7fe7c00ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:50.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7c566d640 1 -- 172.21.15.26:0/1119321093 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c006b950 con 0x7fe7c0154220 2023-12-19T16:57:50.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.573+0000 7fe7b7fff640 1 -- 172.21.15.26:0/1119321093 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346957958 0 0) 0x7fe7c006bc30 con 0x7fe7c006b3d0 2023-12-19T16:57:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7c566d640 1 --2- 172.21.15.26:0/1119321093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c0154220 0x7fe7c015aa80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00cdf30 tx=0x7fe7bc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2306b0118c0880c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7b7fff640 1 -- 172.21.15.26:0/1119321093 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0a7070 con 0x7fe7c0154220 2023-12-19T16:57:50.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7b7fff640 1 -- 172.21.15.26:0/1119321093 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc0ab430 con 0x7fe7c0154220 2023-12-19T16:57:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7b7fff640 1 -- 172.21.15.26:0/1119321093 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0ab7c0 con 0x7fe7c0154220 2023-12-19T16:57:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7c78d1640 1 -- 172.21.15.26:0/1119321093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c0154220 msgr2=0x7fe7c015aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7c78d1640 1 --2- 172.21.15.26:0/1119321093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c0154220 0x7fe7c015aa80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00cdf30 tx=0x7fe7bc01c430 comp rx=0 tx=0).stop 2023-12-19T16:57:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7c78d1640 1 -- 172.21.15.26:0/1119321093 shutdown_connections 2023-12-19T16:57:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7c78d1640 1 --2- 172.21.15.26:0/1119321093 >> v2:172.21.15.160:3300/0 conn(0x7fe7c00cee10 0x7fe7c015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7c78d1640 1 --2- 172.21.15.26:0/1119321093 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c0154220 0x7fe7c015aa80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7c78d1640 1 -- 172.21.15.26:0/1119321093 >> 172.21.15.26:0/1119321093 conn(0x7fe7c00658e0 msgr2=0x7fe7c0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7c78d1640 1 -- 172.21.15.26:0/1119321093 shutdown_connections 2023-12-19T16:57:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7c78d1640 1 -- 172.21.15.26:0/1119321093 wait complete. 2023-12-19T16:57:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7c78d1640 1 Processor -- start 2023-12-19T16:57:50.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.577+0000 7fe7c78d1640 1 -- start start 2023-12-19T16:57:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c78d1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c006b3d0 0x7fe7c0137d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c78d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00cee10 0x7fe7c00af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c78d1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7c0154220 0x7fe7c00afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c78d1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c0138280 con 0x7fe7c00cee10 2023-12-19T16:57:50.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c78d1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c00b2150 con 0x7fe7c006b3d0 2023-12-19T16:57:50.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c78d1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c00b2430 con 0x7fe7c0154220 2023-12-19T16:57:50.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c4e6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00cee10 0x7fe7c00af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:50.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c4e6c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00cee10 0x7fe7c00af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43390/0 (socket says 172.21.15.26:43390) 2023-12-19T16:57:50.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c4e6c640 1 -- 172.21.15.26:0/77391922 learned_addr learned my addr 172.21.15.26:0/77391922 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:50.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c5e6e640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7c0154220 0x7fe7c00afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c566d640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c006b3d0 0x7fe7c0137d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c4e6c640 1 -- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7c0154220 msgr2=0x7fe7c00afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c4e6c640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7c0154220 0x7fe7c00afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c4e6c640 1 -- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c006b3d0 msgr2=0x7fe7c0137d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c4e6c640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c006b3d0 0x7fe7c0137d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c4e6c640 1 -- 172.21.15.26:0/77391922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc009420 con 0x7fe7c00cee10 2023-12-19T16:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c5e6e640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7c0154220 0x7fe7c00afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c566d640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c006b3d0 0x7fe7c0137d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c4e6c640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00cee10 0x7fe7c00af620 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000c480 tx=0x7fe7b000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7b5ffb640 1 -- 172.21.15.26:0/77391922 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b00074e0 con 0x7fe7c00cee10 2023-12-19T16:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7b5ffb640 1 -- 172.21.15.26:0/77391922 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b0007660 con 0x7fe7c00cee10 2023-12-19T16:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c78d1640 1 -- 172.21.15.26:0/77391922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c00b2710 con 0x7fe7c00cee10 2023-12-19T16:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7b5ffb640 1 -- 172.21.15.26:0/77391922 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0004a30 con 0x7fe7c00cee10 2023-12-19T16:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c78d1640 1 -- 172.21.15.26:0/77391922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c00b2cd0 con 0x7fe7c00cee10 2023-12-19T16:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7c78d1640 1 -- 172.21.15.26:0/77391922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c00b2ef0 con 0x7fe7c00cee10 2023-12-19T16:57:50.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.581+0000 7fe7b5ffb640 1 -- 172.21.15.26:0/77391922 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe7b0004b90 con 0x7fe7c00cee10 2023-12-19T16:57:50.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.585+0000 7fe7b5ffb640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe7a4065b80 0x7fe7a4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:50.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.585+0000 7fe7b5ffb640 1 -- 172.21.15.26:0/77391922 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fe7b0082a90 con 0x7fe7c00cee10 2023-12-19T16:57:50.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.585+0000 7fe7c566d640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe7a4065b80 0x7fe7a4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:50.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.585+0000 7fe7c566d640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe7a4065b80 0x7fe7a4068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0af000 tx=0x7fe7bc008040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:50.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.589+0000 7fe7b5ffb640 1 -- 172.21.15.26:0/77391922 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe7b0053b80 con 0x7fe7c00cee10 2023-12-19T16:57:50.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.777+0000 7fe7c78d1640 1 -- 172.21.15.26:0/77391922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7c00ccec0 con 0x7fe7c00cee10 2023-12-19T16:57:50.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.777+0000 7fe7b5ffb640 1 -- 172.21.15.26:0/77391922 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fe7b0018020 con 0x7fe7c00cee10 2023-12-19T16:57:50.783 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:50.784 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/436365775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":436365775},{"type":"v1","addr":"172.21.15.160:6836","nonce":436365775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3014042376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3014042376},{"type":"v1","addr":"172.21.15.160:6837","nonce":3014042376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6283,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1357342534","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1357342534},{"type":"v1","addr":"172.21.15.26:6835","nonce":1357342534}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/2197672479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2197672479},{"type":"v1","addr":"172.21.15.26:6834","nonce":2197672479}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[]} 2023-12-19T16:57:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 35 2023-12-19T16:57:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.781+0000 7fe7c78d1640 1 -- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe7a4065b80 msgr2=0x7fe7a4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.781+0000 7fe7c78d1640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe7a4065b80 0x7fe7a4068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc0af000 tx=0x7fe7bc008040 comp rx=0 tx=0).stop 2023-12-19T16:57:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.781+0000 7fe7c78d1640 1 -- 172.21.15.26:0/77391922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00cee10 msgr2=0x7fe7c00af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.781+0000 7fe7c78d1640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00cee10 0x7fe7c00af620 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000c480 tx=0x7fe7b000c940 comp rx=0 tx=0).stop 2023-12-19T16:57:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.781+0000 7fe7c78d1640 1 -- 172.21.15.26:0/77391922 shutdown_connections 2023-12-19T16:57:50.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.781+0000 7fe7c78d1640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe7a4065b80 0x7fe7a4068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:50.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.781+0000 7fe7c78d1640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7c0154220 0x7fe7c00afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:50.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.781+0000 7fe7c78d1640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c00cee10 0x7fe7c00af620 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:50.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.781+0000 7fe7c78d1640 1 --2- 172.21.15.26:0/77391922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c006b3d0 0x7fe7c0137d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:50.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.781+0000 7fe7c78d1640 1 -- 172.21.15.26:0/77391922 >> 172.21.15.26:0/77391922 conn(0x7fe7c00658e0 msgr2=0x7fe7c0065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:50.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.781+0000 7fe7c78d1640 1 -- 172.21.15.26:0/77391922 shutdown_connections 2023-12-19T16:57:50.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:50.781+0000 7fe7c78d1640 1 -- 172.21.15.26:0/77391922 wait complete. 2023-12-19T16:57:50.805 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T16:57:50.805 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T16:57:50.805 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T16:57:50.805 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:57:50.806 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T16:57:50.806 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T16:57:50.806 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T16:57:50.806 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:57:50.806 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T16:57:50.806 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T16:57:50.806 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T16:57:50.807 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:57:50.807 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T16:57:50.807 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T16:57:50.807 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T16:57:50.807 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:57:50.819 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T16:57:50.819 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T16:57:50.821 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T16:57:50.821 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T16:57:50.823 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T16:57:50.823 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T16:57:50.824 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T16:57:50.824 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T16:57:50.867 INFO:tasks.ceph.mds.a:Started 2023-12-19T16:57:50.868 DEBUG:teuthology.parallel:result is None 2023-12-19T16:57:50.868 INFO:tasks.ceph.mds.b:Started 2023-12-19T16:57:50.868 DEBUG:teuthology.parallel:result is None 2023-12-19T16:57:50.869 INFO:tasks.ceph.mds.c:Started 2023-12-19T16:57:50.869 DEBUG:teuthology.parallel:result is None 2023-12-19T16:57:50.870 INFO:tasks.ceph.mds.d:Started 2023-12-19T16:57:50.870 DEBUG:teuthology.parallel:result is None 2023-12-19T16:57:50.870 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T16:57:50.955 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T16:57:50.956 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T16:57:50.956 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T16:57:50.957 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T16:57:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.021+0000 7f0f501bc640 1 Processor -- start 2023-12-19T16:57:51.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.021+0000 7f0f501bc640 1 -- start start 2023-12-19T16:57:51.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.021+0000 7f0f501bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f480b2d50 0x7f0f4816c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:51.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.021+0000 7f0f501bc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4806d4b0 con 0x7f0f480cb1d0 2023-12-19T16:57:51.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.021+0000 7f0f501bc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4806cc10 con 0x7f0f4806bee0 2023-12-19T16:57:51.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.021+0000 7f0f501bc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f480cc030 con 0x7f0f480b2d50 2023-12-19T16:57:51.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.025+0000 7f0f4df58640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0f480cb1d0 0x7f0f480cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56040/0 (socket says 172.21.15.26:56040) 2023-12-19T16:57:51.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.025+0000 7f0f4e759640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f480b2d50 0x7f0f4816c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:51.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.025+0000 7f0f4df58640 1 -- 172.21.15.26:0/4085885987 learned_addr learned my addr 172.21.15.26:0/4085885987 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:51.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.025+0000 7f0f4e759640 1 -- 172.21.15.26:0/4085885987 >> v1:172.21.15.160:6789/0 conn(0x7f0f4806bee0 legacy=0x7f0f4806c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:51.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.025+0000 7f0f4e759640 1 -- 172.21.15.26:0/4085885987 >> v1:172.21.15.26:6789/0 conn(0x7f0f480cb1d0 legacy=0x7f0f480cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:51.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.025+0000 7f0f4e759640 1 -- 172.21.15.26:0/4085885987 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4806cef0 con 0x7f0f480b2d50 2023-12-19T16:57:51.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.025+0000 7f0f4cf56640 1 -- 172.21.15.26:0/4085885987 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650828468 0 0) 0x7f0f4806d4b0 con 0x7f0f480cb1d0 2023-12-19T16:57:51.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.025+0000 7f0f4e759640 1 --2- 172.21.15.26:0/4085885987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f480b2d50 0x7f0f4816c020 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009d90 tx=0x7f0f4001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b936b4d7085d1652 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:51.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.025+0000 7f0f4cf56640 1 -- 172.21.15.26:0/4085885987 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f400a5980 con 0x7f0f480b2d50 2023-12-19T16:57:51.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.025+0000 7f0f4cf56640 1 -- 172.21.15.26:0/4085885987 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f400a5ae0 con 0x7f0f480b2d50 2023-12-19T16:57:51.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.025+0000 7f0f4cf56640 1 -- 172.21.15.26:0/4085885987 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f400a5e70 con 0x7f0f480b2d50 2023-12-19T16:57:51.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 -- 172.21.15.26:0/4085885987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f480b2d50 msgr2=0x7f0f4816c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:51.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 --2- 172.21.15.26:0/4085885987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f480b2d50 0x7f0f4816c020 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009d90 tx=0x7f0f4001c8c0 comp rx=0 tx=0).stop 2023-12-19T16:57:51.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 -- 172.21.15.26:0/4085885987 shutdown_connections 2023-12-19T16:57:51.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 --2- 172.21.15.26:0/4085885987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f480b2d50 0x7f0f4816c020 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:51.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 -- 172.21.15.26:0/4085885987 >> 172.21.15.26:0/4085885987 conn(0x7f0f480658e0 msgr2=0x7f0f48065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:51.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 -- 172.21.15.26:0/4085885987 shutdown_connections 2023-12-19T16:57:51.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 -- 172.21.15.26:0/4085885987 wait complete. 2023-12-19T16:57:51.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 Processor -- start 2023-12-19T16:57:51.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 -- start start 2023-12-19T16:57:51.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f4806bee0 0x7f0f48152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:51.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f480b2d50 0x7f0f48151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:51.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f480cb1d0 0x7f0f48163890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:51.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f48155f60 con 0x7f0f480cb1d0 2023-12-19T16:57:51.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f48156240 con 0x7f0f480b2d50 2023-12-19T16:57:51.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f501bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f48156520 con 0x7f0f4806bee0 2023-12-19T16:57:51.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f4e759640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f480cb1d0 0x7f0f48163890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:51.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f4e759640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f480cb1d0 0x7f0f48163890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43452/0 (socket says 172.21.15.26:43452) 2023-12-19T16:57:51.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f4e759640 1 -- 172.21.15.26:0/3370735843 learned_addr learned my addr 172.21.15.26:0/3370735843 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:51.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f4d757640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f480b2d50 0x7f0f48151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:51.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f4df58640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f4806bee0 0x7f0f48152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:51.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f4e759640 1 -- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f4806bee0 msgr2=0x7f0f48152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:51.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f4e759640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f4806bee0 0x7f0f48152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:51.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f4e759640 1 -- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f480b2d50 msgr2=0x7f0f48151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:51.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f4e759640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f480b2d50 0x7f0f48151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:51.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f4e759640 1 -- 172.21.15.26:0/3370735843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f40009a70 con 0x7f0f480cb1d0 2023-12-19T16:57:51.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.029+0000 7f0f4df58640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f4806bee0 0x7f0f48152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:51.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.033+0000 7f0f4e759640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f480cb1d0 0x7f0f48163890 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0f4001a4f0 tx=0x7f0f40019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:51.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.033+0000 7f0f3e7fc640 1 -- 172.21.15.26:0/3370735843 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f400ad950 con 0x7f0f480cb1d0 2023-12-19T16:57:51.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.033+0000 7f0f3e7fc640 1 -- 172.21.15.26:0/3370735843 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f400adab0 con 0x7f0f480cb1d0 2023-12-19T16:57:51.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.033+0000 7f0f3e7fc640 1 -- 172.21.15.26:0/3370735843 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f400ade20 con 0x7f0f480cb1d0 2023-12-19T16:57:51.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.033+0000 7f0f501bc640 1 -- 172.21.15.26:0/3370735843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f48156800 con 0x7f0f480cb1d0 2023-12-19T16:57:51.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.033+0000 7f0f501bc640 1 -- 172.21.15.26:0/3370735843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f480b0000 con 0x7f0f480cb1d0 2023-12-19T16:57:51.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.033+0000 7f0f501bc640 1 -- 172.21.15.26:0/3370735843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f480ae150 con 0x7f0f480cb1d0 2023-12-19T16:57:51.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.033+0000 7f0f3e7fc640 1 -- 172.21.15.26:0/3370735843 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0f400b9450 con 0x7f0f480cb1d0 2023-12-19T16:57:51.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.033+0000 7f0f3e7fc640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f2c065b80 0x7f0f2c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:51.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.033+0000 7f0f3e7fc640 1 -- 172.21.15.26:0/3370735843 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f0f4011ea30 con 0x7f0f480cb1d0 2023-12-19T16:57:51.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.033+0000 7f0f4df58640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f2c065b80 0x7f0f2c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:51.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.037+0000 7f0f4df58640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f2c065b80 0x7f0f2c068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f44000f70 tx=0x7f0f44007dd0 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:51.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.037+0000 7f0f3e7fc640 1 -- 172.21.15.26:0/3370735843 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0f400ec010 con 0x7f0f480cb1d0 2023-12-19T16:57:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.213+0000 7f0f501bc640 1 -- 172.21.15.26:0/3370735843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0f48156660 con 0x7f0f480cb1d0 2023-12-19T16:57:52.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:51.997+0000 7f0f3e7fc640 1 -- 172.21.15.26:0/3370735843 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v89) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0f48156660 con 0x7f0f480cb1d0 2023-12-19T16:57:52.004 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2023-12-19T16:57:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.001+0000 7f0f501bc640 1 -- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f2c065b80 msgr2=0x7f0f2c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.001+0000 7f0f501bc640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f2c065b80 0x7f0f2c068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f44000f70 tx=0x7f0f44007dd0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.001+0000 7f0f501bc640 1 -- 172.21.15.26:0/3370735843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f480cb1d0 msgr2=0x7f0f48163890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.001+0000 7f0f501bc640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f480cb1d0 0x7f0f48163890 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0f4001a4f0 tx=0x7f0f40019660 comp rx=0 tx=0).stop 2023-12-19T16:57:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.001+0000 7f0f501bc640 1 -- 172.21.15.26:0/3370735843 shutdown_connections 2023-12-19T16:57:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.001+0000 7f0f501bc640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f480cb1d0 0x7f0f48163890 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.001+0000 7f0f501bc640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f480b2d50 0x7f0f48151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.001+0000 7f0f501bc640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0f2c065b80 0x7f0f2c068040 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.001+0000 7f0f501bc640 1 --2- 172.21.15.26:0/3370735843 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f4806bee0 0x7f0f48152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.001+0000 7f0f501bc640 1 -- 172.21.15.26:0/3370735843 >> 172.21.15.26:0/3370735843 conn(0x7f0f480658e0 msgr2=0x7f0f48065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.001+0000 7f0f501bc640 1 -- 172.21.15.26:0/3370735843 shutdown_connections 2023-12-19T16:57:52.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.001+0000 7f0f501bc640 1 -- 172.21.15.26:0/3370735843 wait complete. 2023-12-19T16:57:52.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T16:57:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.173+0000 7f5dd47bc640 1 Processor -- start 2023-12-19T16:57:52.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.173+0000 7f5dd47bc640 1 -- start start 2023-12-19T16:57:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd47bc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5dcc15c7f0 0x7f5dcc06c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd47bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5dcc0adb80 0x7f5dcc0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd47bc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc06ced0 con 0x7f5dcc0ccc20 2023-12-19T16:57:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd47bc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dcc06d770 con 0x7f5dcc15c7f0 2023-12-19T16:57:52.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd47bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc06d1b0 con 0x7f5dcc0adb80 2023-12-19T16:57:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd1d57640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5dcc0adb80 0x7f5dcc0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd2d59640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f5dcc0ccc20 0x7f5dcc0b30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46212/0 (socket says 172.21.15.26:46212) 2023-12-19T16:57:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd2d59640 1 -- 172.21.15.26:0/791573361 learned_addr learned my addr 172.21.15.26:0/791573361 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd1d57640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5dcc0adb80 0x7f5dcc0cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:32848/0 (socket says 172.21.15.26:32848) 2023-12-19T16:57:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd2558640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5dcc15c7f0 0x7f5dcc06c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd1d57640 1 -- 172.21.15.26:0/791573361 >> v2:172.21.15.160:3300/0 conn(0x7f5dcc15c7f0 msgr2=0x7f5dcc06c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd1d57640 1 --2- 172.21.15.26:0/791573361 >> v2:172.21.15.160:3300/0 conn(0x7f5dcc15c7f0 0x7f5dcc06c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd1d57640 1 -- 172.21.15.26:0/791573361 >> v1:172.21.15.160:6789/0 conn(0x7f5dcc0ccc20 legacy=0x7f5dcc0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd1d57640 1 -- 172.21.15.26:0/791573361 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc06d490 con 0x7f5dcc0adb80 2023-12-19T16:57:52.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.177+0000 7f5dd1556640 1 -- 172.21.15.26:0/791573361 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231695182 0 0) 0x7f5dcc06ced0 con 0x7f5dcc0ccc20 2023-12-19T16:57:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd1d57640 1 --2- 172.21.15.26:0/791573361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5dcc0adb80 0x7f5dcc0cc6e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc0097c0 tx=0x7f5dbc01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3666f9e3eba4a302 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd1556640 1 -- 172.21.15.26:0/791573361 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc0ab600 con 0x7f5dcc0adb80 2023-12-19T16:57:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd1556640 1 -- 172.21.15.26:0/791573361 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc0a7050 con 0x7f5dcc0adb80 2023-12-19T16:57:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd1556640 1 -- 172.21.15.26:0/791573361 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc0ab980 con 0x7f5dcc0adb80 2023-12-19T16:57:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd47bc640 1 -- 172.21.15.26:0/791573361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5dcc0adb80 msgr2=0x7f5dcc0cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd47bc640 1 --2- 172.21.15.26:0/791573361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5dcc0adb80 0x7f5dcc0cc6e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc0097c0 tx=0x7f5dbc01c480 comp rx=0 tx=0).stop 2023-12-19T16:57:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd47bc640 1 -- 172.21.15.26:0/791573361 shutdown_connections 2023-12-19T16:57:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd47bc640 1 --2- 172.21.15.26:0/791573361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5dcc0adb80 0x7f5dcc0cc6e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd47bc640 1 --2- 172.21.15.26:0/791573361 >> v2:172.21.15.160:3300/0 conn(0x7f5dcc15c7f0 0x7f5dcc06c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd47bc640 1 -- 172.21.15.26:0/791573361 >> 172.21.15.26:0/791573361 conn(0x7f5dcc0658e0 msgr2=0x7f5dcc065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:52.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd47bc640 1 -- 172.21.15.26:0/791573361 shutdown_connections 2023-12-19T16:57:52.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd47bc640 1 -- 172.21.15.26:0/791573361 wait complete. 2023-12-19T16:57:52.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd47bc640 1 Processor -- start 2023-12-19T16:57:52.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.181+0000 7f5dd47bc640 1 -- start start 2023-12-19T16:57:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd47bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dcc0adb80 0x7f5dcc159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd47bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5dcc0ccc20 0x7f5dcc1493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd47bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dcc15c7f0 0x7f5dcc0cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd47bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc1498e0 con 0x7f5dcc0adb80 2023-12-19T16:57:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd47bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc152cf0 con 0x7f5dcc15c7f0 2023-12-19T16:57:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd47bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc0ace50 con 0x7f5dcc0ccc20 2023-12-19T16:57:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd2558640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dcc0adb80 0x7f5dcc159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd2558640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dcc0adb80 0x7f5dcc159e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43468/0 (socket says 172.21.15.26:43468) 2023-12-19T16:57:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd2558640 1 -- 172.21.15.26:0/4130377981 learned_addr learned my addr 172.21.15.26:0/4130377981 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd1d57640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5dcc0ccc20 0x7f5dcc1493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd2d59640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dcc15c7f0 0x7f5dcc0cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd2558640 1 -- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5dcc0ccc20 msgr2=0x7f5dcc1493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd2558640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5dcc0ccc20 0x7f5dcc1493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd2558640 1 -- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dcc15c7f0 msgr2=0x7f5dcc0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd2558640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dcc15c7f0 0x7f5dcc0cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd2558640 1 -- 172.21.15.26:0/4130377981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc009420 con 0x7f5dcc0adb80 2023-12-19T16:57:52.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd2558640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dcc0adb80 0x7f5dcc159e00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800e870 tx=0x7f5dc800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dc2ffd640 1 -- 172.21.15.26:0/4130377981 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8009510 con 0x7f5dcc0adb80 2023-12-19T16:57:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dc2ffd640 1 -- 172.21.15.26:0/4130377981 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc800eec0 con 0x7f5dcc0adb80 2023-12-19T16:57:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dc2ffd640 1 -- 172.21.15.26:0/4130377981 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc800cb40 con 0x7f5dcc0adb80 2023-12-19T16:57:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd47bc640 1 -- 172.21.15.26:0/4130377981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc0ad130 con 0x7f5dcc0adb80 2023-12-19T16:57:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd47bc640 1 -- 172.21.15.26:0/4130377981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc153220 con 0x7f5dcc0adb80 2023-12-19T16:57:52.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.185+0000 7f5dd47bc640 1 -- 172.21.15.26:0/4130377981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dcc1533e0 con 0x7f5dcc0adb80 2023-12-19T16:57:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.189+0000 7f5dc2ffd640 1 -- 172.21.15.26:0/4130377981 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5dc800cca0 con 0x7f5dcc0adb80 2023-12-19T16:57:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.189+0000 7f5dc2ffd640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5db0065b80 0x7f5db0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.189+0000 7f5dc2ffd640 1 -- 172.21.15.26:0/4130377981 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f5dc80832c0 con 0x7f5dcc0adb80 2023-12-19T16:57:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.189+0000 7f5dd1d57640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5db0065b80 0x7f5db0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.189+0000 7f5dd1d57640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5db0065b80 0x7f5db0068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc0097c0 tx=0x7f5dbc0a6000 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.193+0000 7f5dc2ffd640 1 -- 172.21.15.26:0/4130377981 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5dc8050c80 con 0x7f5dcc0adb80 2023-12-19T16:57:52.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.401+0000 7f5dd47bc640 1 -- 172.21.15.26:0/4130377981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5dcc0ad2c0 con 0x7f5dcc0adb80 2023-12-19T16:57:52.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.401+0000 7f5dc2ffd640 1 -- 172.21.15.26:0/4130377981 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5dc80542a0 con 0x7f5dcc0adb80 2023-12-19T16:57:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:52.407 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T16:57:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2023-12-19T16:57:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2023-12-19T16:57:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg==", 2023-12-19T16:57:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.407 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:57:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:57:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2023-12-19T16:57:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg==", 2023-12-19T16:57:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:57:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:57:52.408 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2023-12-19T16:57:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg==", 2023-12-19T16:57:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:57:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:57:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.409 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw==", 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2023-12-19T16:57:52.410 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig==", 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA==", 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.411 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ==", 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ==", 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.412 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A==", 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.413 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ==", 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg==", 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:52.414 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2023-12-19T16:57:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg==", 2023-12-19T16:57:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:57:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:57:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.415 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ==", 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ==", 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:52.416 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ==", 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.417 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ==", 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlT5ynDxAA7Sjbdwrc1RyAXJboysB54Q==", 2023-12-19T16:57:52.418 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA==", 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.419 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:57:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2023-12-19T16:57:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg==", 2023-12-19T16:57:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:57:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:57:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T16:57:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:57:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:52.420 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-12-19T16:57:52.420 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T16:57:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.405+0000 7f5dd47bc640 1 -- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5db0065b80 msgr2=0x7f5db0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.405+0000 7f5dd47bc640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5db0065b80 0x7f5db0068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc0097c0 tx=0x7f5dbc0a6000 comp rx=0 tx=0).stop 2023-12-19T16:57:52.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.405+0000 7f5dd47bc640 1 -- 172.21.15.26:0/4130377981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dcc0adb80 msgr2=0x7f5dcc159e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.405+0000 7f5dd47bc640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dcc0adb80 0x7f5dcc159e00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5dc800e870 tx=0x7f5dc800ec70 comp rx=0 tx=0).stop 2023-12-19T16:57:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.405+0000 7f5dd47bc640 1 -- 172.21.15.26:0/4130377981 shutdown_connections 2023-12-19T16:57:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.405+0000 7f5dd47bc640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5dcc15c7f0 0x7f5dcc0cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.405+0000 7f5dd47bc640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5db0065b80 0x7f5db0068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.405+0000 7f5dd47bc640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5dcc0ccc20 0x7f5dcc1493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.405+0000 7f5dd47bc640 1 --2- 172.21.15.26:0/4130377981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5dcc0adb80 0x7f5dcc159e00 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.405+0000 7f5dd47bc640 1 -- 172.21.15.26:0/4130377981 >> 172.21.15.26:0/4130377981 conn(0x7f5dcc0658e0 msgr2=0x7f5dcc070140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.405+0000 7f5dd47bc640 1 -- 172.21.15.26:0/4130377981 shutdown_connections 2023-12-19T16:57:52.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.405+0000 7f5dd47bc640 1 -- 172.21.15.26:0/4130377981 wait complete. 2023-12-19T16:57:52.428 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:52.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6337012640 1 Processor -- start 2023-12-19T16:57:52.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6337012640 1 -- start start 2023-12-19T16:57:52.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6337012640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6330154220 0x7f633015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6337012640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63300cee10 0x7f633015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6337012640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633006bc30 con 0x7f633006b3d0 2023-12-19T16:57:52.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6337012640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63300cc6e0 con 0x7f6330154220 2023-12-19T16:57:52.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6337012640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633015afc0 con 0x7f63300cee10 2023-12-19T16:57:52.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6334dae640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6330154220 0x7f633015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6334dae640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6330154220 0x7f633015aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43476/0 (socket says 172.21.15.26:43476) 2023-12-19T16:57:52.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6334dae640 1 -- 172.21.15.26:0/108403206 learned_addr learned my addr 172.21.15.26:0/108403206 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:52.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f63355af640 1 --2- 172.21.15.26:0/108403206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63300cee10 0x7f633015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6334dae640 1 -- 172.21.15.26:0/108403206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63300cee10 msgr2=0x7f633015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:52.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6334dae640 1 --2- 172.21.15.26:0/108403206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63300cee10 0x7f633015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6334dae640 1 -- 172.21.15.26:0/108403206 >> v1:172.21.15.160:6789/0 conn(0x7f633006b3d0 legacy=0x7f63300ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:52.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.569+0000 7f6334dae640 1 -- 172.21.15.26:0/108403206 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633006b950 con 0x7f6330154220 2023-12-19T16:57:52.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.573+0000 7f6334dae640 1 --2- 172.21.15.26:0/108403206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6330154220 0x7f633015aa80 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6324009870 tx=0x7f632401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713967f49acbe68f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:52.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.573+0000 7f632f7fe640 1 -- 172.21.15.26:0/108403206 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63240aa670 con 0x7f6330154220 2023-12-19T16:57:52.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.573+0000 7f632f7fe640 1 -- 172.21.15.26:0/108403206 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63240aa7d0 con 0x7f6330154220 2023-12-19T16:57:52.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.573+0000 7f632f7fe640 1 -- 172.21.15.26:0/108403206 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63240aab60 con 0x7f6330154220 2023-12-19T16:57:52.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.573+0000 7f6337012640 1 -- 172.21.15.26:0/108403206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6330154220 msgr2=0x7f633015aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.573+0000 7f6337012640 1 --2- 172.21.15.26:0/108403206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6330154220 0x7f633015aa80 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6324009870 tx=0x7f632401c510 comp rx=0 tx=0).stop 2023-12-19T16:57:52.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.573+0000 7f6337012640 1 -- 172.21.15.26:0/108403206 shutdown_connections 2023-12-19T16:57:52.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.573+0000 7f6337012640 1 --2- 172.21.15.26:0/108403206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f63300cee10 0x7f633015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.573+0000 7f6337012640 1 --2- 172.21.15.26:0/108403206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6330154220 0x7f633015aa80 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.573+0000 7f6337012640 1 -- 172.21.15.26:0/108403206 >> 172.21.15.26:0/108403206 conn(0x7f63300658e0 msgr2=0x7f6330065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:52.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.573+0000 7f6337012640 1 -- 172.21.15.26:0/108403206 shutdown_connections 2023-12-19T16:57:52.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.573+0000 7f6337012640 1 -- 172.21.15.26:0/108403206 wait complete. 2023-12-19T16:57:52.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f6337012640 1 Processor -- start 2023-12-19T16:57:52.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f6337012640 1 -- start start 2023-12-19T16:57:52.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f6337012640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633006b3d0 0x7f6330151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f6337012640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63300cee10 0x7f63300b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f6337012640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6330154220 0x7f633014e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f6337012640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6330151010 con 0x7f6330154220 2023-12-19T16:57:52.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f6337012640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63301f6340 con 0x7f63300cee10 2023-12-19T16:57:52.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f6337012640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63301f56c0 con 0x7f633006b3d0 2023-12-19T16:57:52.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f63355af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6330154220 0x7f633014e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f63355af640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6330154220 0x7f633014e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43488/0 (socket says 172.21.15.26:43488) 2023-12-19T16:57:52.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f63355af640 1 -- 172.21.15.26:0/2264467927 learned_addr learned my addr 172.21.15.26:0/2264467927 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:52.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f6334dae640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633006b3d0 0x7f6330151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f632ffff640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63300cee10 0x7f63300b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f63355af640 1 -- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633006b3d0 msgr2=0x7f6330151990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f63355af640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633006b3d0 0x7f6330151990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f63355af640 1 -- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63300cee10 msgr2=0x7f63300b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f63355af640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63300cee10 0x7f63300b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f63355af640 1 -- 172.21.15.26:0/2264467927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63240094d0 con 0x7f6330154220 2023-12-19T16:57:52.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f632ffff640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63300cee10 0x7f63300b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:52.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f63355af640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6330154220 0x7f633014e190 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6328007e70 tx=0x7f632800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:52.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f632d7fa640 1 -- 172.21.15.26:0/2264467927 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328014620 con 0x7f6330154220 2023-12-19T16:57:52.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f632d7fa640 1 -- 172.21.15.26:0/2264467927 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63280147a0 con 0x7f6330154220 2023-12-19T16:57:52.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f6337012640 1 -- 172.21.15.26:0/2264467927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63301f59a0 con 0x7f6330154220 2023-12-19T16:57:52.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f632d7fa640 1 -- 172.21.15.26:0/2264467927 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328014b50 con 0x7f6330154220 2023-12-19T16:57:52.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.577+0000 7f6337012640 1 -- 172.21.15.26:0/2264467927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63301f5f60 con 0x7f6330154220 2023-12-19T16:57:52.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.581+0000 7f6337012640 1 -- 172.21.15.26:0/2264467927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633006f370 con 0x7f6330154220 2023-12-19T16:57:52.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.581+0000 7f632d7fa640 1 -- 172.21.15.26:0/2264467927 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6328014cb0 con 0x7f6330154220 2023-12-19T16:57:52.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.581+0000 7f632d7fa640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6314065b80 0x7f6314068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.581+0000 7f632d7fa640 1 -- 172.21.15.26:0/2264467927 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f6328083c30 con 0x7f6330154220 2023-12-19T16:57:52.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.581+0000 7f6334dae640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6314065b80 0x7f6314068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.581+0000 7f6334dae640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6314065b80 0x7f6314068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f63240ae000 tx=0x7f63240a7000 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:52.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.585+0000 7f632d7fa640 1 -- 172.21.15.26:0/2264467927 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f633006f370 con 0x7f6330154220 2023-12-19T16:57:52.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f6337012640 1 -- 172.21.15.26:0/2264467927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f633006f370 con 0x7f6330154220 2023-12-19T16:57:52.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f632d7fa640 1 -- 172.21.15.26:0/2264467927 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+13494 (secure 0 0 0) 0x7f633006f370 con 0x7f6330154220 2023-12-19T16:57:52.750 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:52.750 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:51.995266+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:52.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f6337012640 1 -- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6314065b80 msgr2=0x7f6314068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f6337012640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6314065b80 0x7f6314068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f63240ae000 tx=0x7f63240a7000 comp rx=0 tx=0).stop 2023-12-19T16:57:52.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f6337012640 1 -- 172.21.15.26:0/2264467927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6330154220 msgr2=0x7f633014e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f6337012640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6330154220 0x7f633014e190 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6328007e70 tx=0x7f632800e4b0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f6337012640 1 -- 172.21.15.26:0/2264467927 shutdown_connections 2023-12-19T16:57:52.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f6337012640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6330154220 0x7f633014e190 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f6337012640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63300cee10 0x7f63300b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f6337012640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6314065b80 0x7f6314068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f6337012640 1 --2- 172.21.15.26:0/2264467927 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f633006b3d0 0x7f6330151990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f6337012640 1 -- 172.21.15.26:0/2264467927 >> 172.21.15.26:0/2264467927 conn(0x7f63300658e0 msgr2=0x7f6330065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:52.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f6337012640 1 -- 172.21.15.26:0/2264467927 shutdown_connections 2023-12-19T16:57:52.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.745+0000 7f6337012640 1 -- 172.21.15.26:0/2264467927 wait complete. 2023-12-19T16:57:52.772 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:57:52.773 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:57:52.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.913+0000 7fd7ae4f2640 1 Processor -- start 2023-12-19T16:57:52.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.913+0000 7fd7ae4f2640 1 -- start start 2023-12-19T16:57:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7ae4f2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7a8159140 0x7fd7a80ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7ae4f2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd7a80cd190 0x7fd7a806ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7ae4f2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a806f490 con 0x7fd7a806b1c0 2023-12-19T16:57:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7ae4f2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a806b650 con 0x7fd7a80cd190 2023-12-19T16:57:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7ae4f2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a8159520 con 0x7fd7a8159140 2023-12-19T16:57:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7aca8f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd7a806b1c0 0x7fd7a815ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56054/0 (socket says 172.21.15.26:56054) 2023-12-19T16:57:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7aca8f640 1 -- 172.21.15.26:0/1527670948 learned_addr learned my addr 172.21.15.26:0/1527670948 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7a7fff640 1 --2- 172.21.15.26:0/1527670948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7a8159140 0x7fd7a80ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7a77fe640 1 --2- 172.21.15.26:0/1527670948 >> v2:172.21.15.160:3300/0 conn(0x7fd7a80cd190 0x7fd7a806ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7a6ffd640 1 -- 172.21.15.26:0/1527670948 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282315792 0 0) 0x7fd7a806f490 con 0x7fd7a806b1c0 2023-12-19T16:57:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7a6ffd640 1 -- 172.21.15.26:0/1527670948 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd790003530 con 0x7fd7a806b1c0 2023-12-19T16:57:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7a7fff640 1 -- 172.21.15.26:0/1527670948 >> v2:172.21.15.160:3300/0 conn(0x7fd7a80cd190 msgr2=0x7fd7a806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:52.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7a7fff640 1 --2- 172.21.15.26:0/1527670948 >> v2:172.21.15.160:3300/0 conn(0x7fd7a80cd190 0x7fd7a806ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7a7fff640 1 -- 172.21.15.26:0/1527670948 >> v1:172.21.15.26:6789/0 conn(0x7fd7a806b1c0 legacy=0x7fd7a815ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7a6ffd640 1 -- 172.21.15.26:0/1527670948 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2699294689 0 0) 0x7fd790003530 con 0x7fd7a806b1c0 2023-12-19T16:57:52.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.917+0000 7fd7a7fff640 1 -- 172.21.15.26:0/1527670948 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a80cd690 con 0x7fd7a8159140 2023-12-19T16:57:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7a7fff640 1 --2- 172.21.15.26:0/1527670948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7a8159140 0x7fd7a80ccc50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd7980097c0 tx=0x7fd79801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97721522a7cd6c02 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7a6ffd640 1 -- 172.21.15.26:0/1527670948 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7980ac630 con 0x7fd7a8159140 2023-12-19T16:57:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7a6ffd640 1 -- 172.21.15.26:0/1527670948 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7980ac790 con 0x7fd7a8159140 2023-12-19T16:57:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7a6ffd640 1 -- 172.21.15.26:0/1527670948 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7980acb00 con 0x7fd7a8159140 2023-12-19T16:57:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/1527670948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7a8159140 msgr2=0x7fd7a80ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7ae4f2640 1 --2- 172.21.15.26:0/1527670948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7a8159140 0x7fd7a80ccc50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd7980097c0 tx=0x7fd79801c450 comp rx=0 tx=0).stop 2023-12-19T16:57:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/1527670948 shutdown_connections 2023-12-19T16:57:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7ae4f2640 1 --2- 172.21.15.26:0/1527670948 >> v2:172.21.15.160:3300/0 conn(0x7fd7a80cd190 0x7fd7a806ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7ae4f2640 1 --2- 172.21.15.26:0/1527670948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7a8159140 0x7fd7a80ccc50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/1527670948 >> 172.21.15.26:0/1527670948 conn(0x7fd7a80ce730 msgr2=0x7fd7a80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/1527670948 shutdown_connections 2023-12-19T16:57:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/1527670948 wait complete. 2023-12-19T16:57:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7ae4f2640 1 Processor -- start 2023-12-19T16:57:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.921+0000 7fd7ae4f2640 1 -- start start 2023-12-19T16:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7ae4f2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a806b1c0 0x7fd7a80b3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7ae4f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a80cd190 0x7fd7a80c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7ae4f2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7a8159140 0x7fd7a80b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7ae4f2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a80c8450 con 0x7fd7a80cd190 2023-12-19T16:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7ae4f2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a80b2a80 con 0x7fd7a806b1c0 2023-12-19T16:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7ae4f2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a80b19e0 con 0x7fd7a8159140 2023-12-19T16:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a80cd190 0x7fd7a80c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a7fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a806b1c0 0x7fd7a80b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a80cd190 0x7fd7a80c4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43496/0 (socket says 172.21.15.26:43496) 2023-12-19T16:57:52.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a77fe640 1 -- 172.21.15.26:0/2873118206 learned_addr learned my addr 172.21.15.26:0/2873118206 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:52.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7aca8f640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7a8159140 0x7fd7a80b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a77fe640 1 -- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7a8159140 msgr2=0x7fd7a80b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a77fe640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7a8159140 0x7fd7a80b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a77fe640 1 -- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a806b1c0 msgr2=0x7fd7a80b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a77fe640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a806b1c0 0x7fd7a80b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a77fe640 1 -- 172.21.15.26:0/2873118206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd798009420 con 0x7fd7a80cd190 2023-12-19T16:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a7fff640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a806b1c0 0x7fd7a80b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a77fe640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a80cd190 0x7fd7a80c4b10 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00e940 tx=0x7fd79c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a4ff9640 1 -- 172.21.15.26:0/2873118206 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00cc40 con 0x7fd7a80cd190 2023-12-19T16:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a4ff9640 1 -- 172.21.15.26:0/2873118206 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79c00cda0 con 0x7fd7a80cd190 2023-12-19T16:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7a4ff9640 1 -- 172.21.15.26:0/2873118206 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0115f0 con 0x7fd7a80cd190 2023-12-19T16:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/2873118206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a80b1cc0 con 0x7fd7a80cd190 2023-12-19T16:57:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/2873118206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a80b2280 con 0x7fd7a80cd190 2023-12-19T16:57:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.925+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/2873118206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a8074270 con 0x7fd7a80cd190 2023-12-19T16:57:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.929+0000 7fd7a4ff9640 1 -- 172.21.15.26:0/2873118206 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd79c011750 con 0x7fd7a80cd190 2023-12-19T16:57:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.929+0000 7fd7a4ff9640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd794065b80 0x7fd794068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.929+0000 7fd7a4ff9640 1 -- 172.21.15.26:0/2873118206 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fd79c083170 con 0x7fd7a80cd190 2023-12-19T16:57:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.929+0000 7fd7a7fff640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd794065b80 0x7fd794068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.929+0000 7fd7a7fff640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd794065b80 0x7fd794068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd7980ae000 tx=0x7fd7980a7000 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:52.933+0000 7fd7a4ff9640 1 -- 172.21.15.26:0/2873118206 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd79c050a20 con 0x7fd7a80cd190 2023-12-19T16:57:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/2873118206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7a80711b0 con 0x7fd7a80cd190 2023-12-19T16:57:53.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7a4ff9640 1 -- 172.21.15.26:0/2873118206 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd79c054040 con 0x7fd7a80cd190 2023-12-19T16:57:53.097 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:57:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd794065b80 msgr2=0x7fd794068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7ae4f2640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd794065b80 0x7fd794068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd7980ae000 tx=0x7fd7980a7000 comp rx=0 tx=0).stop 2023-12-19T16:57:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/2873118206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a80cd190 msgr2=0x7fd7a80c4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7ae4f2640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a80cd190 0x7fd7a80c4b10 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00e940 tx=0x7fd79c00ee00 comp rx=0 tx=0).stop 2023-12-19T16:57:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/2873118206 shutdown_connections 2023-12-19T16:57:53.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7ae4f2640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd794065b80 0x7fd794068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7ae4f2640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd7a8159140 0x7fd7a80b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7ae4f2640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7a80cd190 0x7fd7a80c4b10 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7ae4f2640 1 --2- 172.21.15.26:0/2873118206 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7a806b1c0 0x7fd7a80b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/2873118206 >> 172.21.15.26:0/2873118206 conn(0x7fd7a80ce730 msgr2=0x7fd7a80cbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:53.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/2873118206 shutdown_connections 2023-12-19T16:57:53.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.093+0000 7fd7ae4f2640 1 -- 172.21.15.26:0/2873118206 wait complete. 2023-12-19T16:57:53.119 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T16:57:53.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.257+0000 7fdfd58d1640 1 Processor -- start 2023-12-19T16:57:53.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.257+0000 7fdfd58d1640 1 -- start start 2023-12-19T16:57:53.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfd58d1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cee10 0x7fdfd0164980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfd58d1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd006bc30 con 0x7fdfd0154220 2023-12-19T16:57:53.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfd58d1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd00cc6e0 con 0x7fdfd006b3d0 2023-12-19T16:57:53.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfd58d1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd006b950 con 0x7fdfd00cee10 2023-12-19T16:57:53.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfceffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdfd0154220 0x7fdfd015aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56070/0 (socket says 172.21.15.26:56070) 2023-12-19T16:57:53.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfceffd640 1 -- 172.21.15.26:0/1548383058 learned_addr learned my addr 172.21.15.26:0/1548383058 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfcf7fe640 1 --2- 172.21.15.26:0/1548383058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cee10 0x7fdfd0164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377314268 0 0) 0x7fdfd006bc30 con 0x7fdfd0154220 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfbc005740 con 0x7fdfd0154220 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5483647 0 0) 0x7fdfd00cc6e0 con 0x7fdfd006b3d0 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfd006bc30 con 0x7fdfd006b3d0 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2377813477 0 0) 0x7fdfbc005740 con 0x7fdfd0154220 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cee10 msgr2=0x7fdfd0164980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 --2- 172.21.15.26:0/1548383058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cee10 0x7fdfd0164980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 >> v1:172.21.15.160:6789/0 conn(0x7fdfd006b3d0 legacy=0x7fdfd00ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd00cc930 con 0x7fdfd0154220 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fdfc0001680 con 0x7fdfd0154220 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfcf7fe640 1 --2- 172.21.15.26:0/1548383058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cee10 0x7fdfd0164980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 >> v1:172.21.15.26:6789/0 conn(0x7fdfd0154220 legacy=0x7fdfd015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 --2- 172.21.15.26:0/1548383058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cee10 0x7fdfbc005d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 --2- 172.21.15.26:0/1548383058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc006290 0x7fdfbc0069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 --2- 172.21.15.26:0/1548383058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfbc006f20 0x7fdfbc0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd006b950 con 0x7fdfbc006290 2023-12-19T16:57:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc00dc50 con 0x7fdfbc006f20 2023-12-19T16:57:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc00de10 con 0x7fdfd00cee10 2023-12-19T16:57:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfc7fff640 1 --2- 172.21.15.26:0/1548383058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc006290 0x7fdfbc0069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfceffd640 1 --2- 172.21.15.26:0/1548383058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cee10 0x7fdfbc005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfcf7fe640 1 --2- 172.21.15.26:0/1548383058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfbc006f20 0x7fdfbc0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfc7fff640 1 -- 172.21.15.26:0/1548383058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cee10 msgr2=0x7fdfbc005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfc7fff640 1 --2- 172.21.15.26:0/1548383058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cee10 0x7fdfbc005d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfc7fff640 1 -- 172.21.15.26:0/1548383058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfbc006f20 msgr2=0x7fdfbc0073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfc7fff640 1 --2- 172.21.15.26:0/1548383058 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfbc006f20 0x7fdfbc0073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.261+0000 7fdfc7fff640 1 -- 172.21.15.26:0/1548383058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc8009a70 con 0x7fdfbc006290 2023-12-19T16:57:53.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.265+0000 7fdfc7fff640 1 --2- 172.21.15.26:0/1548383058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc006290 0x7fdfbc0069e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80018d0 tx=0x7fdfb801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9251d20eaa268552 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:53.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.265+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80052d0 con 0x7fdfbc006290 2023-12-19T16:57:53.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.265+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb8005450 con 0x7fdfbc006290 2023-12-19T16:57:53.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.265+0000 7fdfce7fc640 1 -- 172.21.15.26:0/1548383058 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb80a8930 con 0x7fdfbc006290 2023-12-19T16:57:53.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.265+0000 7fdfd58d1640 1 -- 172.21.15.26:0/1548383058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc006290 msgr2=0x7fdfbc0069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:53.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.265+0000 7fdfd58d1640 1 --2- 172.21.15.26:0/1548383058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc006290 0x7fdfbc0069e0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80018d0 tx=0x7fdfb801ecd0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.265+0000 7fdfcf7fe640 1 -- 172.21.15.26:0/1548383058 reap_dead start 2023-12-19T16:57:53.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.265+0000 7fdfd58d1640 1 -- 172.21.15.26:0/1548383058 shutdown_connections 2023-12-19T16:57:53.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.265+0000 7fdfd58d1640 1 -- 172.21.15.26:0/1548383058 >> 172.21.15.26:0/1548383058 conn(0x7fdfd00658e0 msgr2=0x7fdfd0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:53.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.265+0000 7fdfd58d1640 1 -- 172.21.15.26:0/1548383058 shutdown_connections 2023-12-19T16:57:53.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.265+0000 7fdfd58d1640 1 -- 172.21.15.26:0/1548383058 wait complete. 2023-12-19T16:57:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.265+0000 7fdfd58d1640 1 Processor -- start 2023-12-19T16:57:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.265+0000 7fdfd58d1640 1 -- start start 2023-12-19T16:57:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfd58d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc006290 0x7fdfd0154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfd58d1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd014f470 0x7fdfd00b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfd58d1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfd00b20c0 0x7fdfd00709e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfd58d1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd0154f70 con 0x7fdfbc006290 2023-12-19T16:57:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfd58d1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd00b2550 con 0x7fdfd00b20c0 2023-12-19T16:57:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfd58d1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd014fcc0 con 0x7fdfd014f470 2023-12-19T16:57:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfceffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc006290 0x7fdfd0154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfceffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc006290 0x7fdfd0154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43516/0 (socket says 172.21.15.26:43516) 2023-12-19T16:57:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfceffd640 1 -- 172.21.15.26:0/235712703 learned_addr learned my addr 172.21.15.26:0/235712703 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfc7fff640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd014f470 0x7fdfd00b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfcf7fe640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfd00b20c0 0x7fdfd00709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfceffd640 1 -- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd014f470 msgr2=0x7fdfd00b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfceffd640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd014f470 0x7fdfd00b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfceffd640 1 -- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfd00b20c0 msgr2=0x7fdfd00709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfceffd640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfd00b20c0 0x7fdfd00709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfceffd640 1 -- 172.21.15.26:0/235712703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd0150840 con 0x7fdfbc006290 2023-12-19T16:57:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfcf7fe640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfd00b20c0 0x7fdfd00709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:57:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfceffd640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc006290 0x7fdfd0154a30 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0007340 tx=0x7fdfc0007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfccff9640 1 -- 172.21.15.26:0/235712703 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000f960 con 0x7fdfbc006290 2023-12-19T16:57:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfccff9640 1 -- 172.21.15.26:0/235712703 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc000fac0 con 0x7fdfbc006290 2023-12-19T16:57:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfccff9640 1 -- 172.21.15.26:0/235712703 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc000fe70 con 0x7fdfbc006290 2023-12-19T16:57:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfd58d1640 1 -- 172.21.15.26:0/235712703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd014ff10 con 0x7fdfbc006290 2023-12-19T16:57:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfd58d1640 1 -- 172.21.15.26:0/235712703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd01504d0 con 0x7fdfbc006290 2023-12-19T16:57:53.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfd58d1640 1 -- 172.21.15.26:0/235712703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd0074270 con 0x7fdfbc006290 2023-12-19T16:57:53.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.269+0000 7fdfccff9640 1 -- 172.21.15.26:0/235712703 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fdfc001d460 con 0x7fdfbc006290 2023-12-19T16:57:53.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.273+0000 7fdfccff9640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfb0065b80 0x7fdfb0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.273+0000 7fdfccff9640 1 -- 172.21.15.26:0/235712703 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fdfc00832d0 con 0x7fdfbc006290 2023-12-19T16:57:53.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.273+0000 7fdfc7fff640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfb0065b80 0x7fdfb0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.273+0000 7fdfc7fff640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfb0065b80 0x7fdfb0068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80018d0 tx=0x7fdfb80aa070 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.277+0000 7fdfccff9640 1 -- 172.21.15.26:0/235712703 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdfc0050c60 con 0x7fdfbc006290 2023-12-19T16:57:53.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfd58d1640 1 -- 172.21.15.26:0/235712703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdfd00b3990 con 0x7fdfbc006290 2023-12-19T16:57:53.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfccff9640 1 -- 172.21.15.26:0/235712703 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v89) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdfc0054280 con 0x7fdfbc006290 2023-12-19T16:57:53.441 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:53.442 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T16:57:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:57:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T16:57:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T16:57:53.442 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:57:53.442 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T16:57:53.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfd58d1640 1 -- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfb0065b80 msgr2=0x7fdfb0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:53.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfd58d1640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfb0065b80 0x7fdfb0068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80018d0 tx=0x7fdfb80aa070 comp rx=0 tx=0).stop 2023-12-19T16:57:53.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfd58d1640 1 -- 172.21.15.26:0/235712703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc006290 msgr2=0x7fdfd0154a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:53.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfd58d1640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc006290 0x7fdfd0154a30 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdfc0007340 tx=0x7fdfc0007800 comp rx=0 tx=0).stop 2023-12-19T16:57:53.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfd58d1640 1 -- 172.21.15.26:0/235712703 shutdown_connections 2023-12-19T16:57:53.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfd58d1640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfd00b20c0 0x7fdfd00709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfd58d1640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfb0065b80 0x7fdfb0068040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfd58d1640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd014f470 0x7fdfd00b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfd58d1640 1 --2- 172.21.15.26:0/235712703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc006290 0x7fdfd0154a30 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfd58d1640 1 -- 172.21.15.26:0/235712703 >> 172.21.15.26:0/235712703 conn(0x7fdfd00658e0 msgr2=0x7fdfd00aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:53.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfd58d1640 1 -- 172.21.15.26:0/235712703 shutdown_connections 2023-12-19T16:57:53.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.437+0000 7fdfd58d1640 1 -- 172.21.15.26:0/235712703 wait complete. 2023-12-19T16:57:53.462 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T16:57:53.462 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T16:57:53.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.605+0000 7f9cf7b60640 1 Processor -- start 2023-12-19T16:57:53.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.605+0000 7f9cf7b60640 1 -- start start 2023-12-19T16:57:53.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.605+0000 7f9cf7b60640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9cf0159830 0x7f9cf015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.605+0000 7f9cf7b60640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9cf00c9310 0x7f9cf00cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.605+0000 7f9cf7b60640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf015e3d0 con 0x7f9cf00cb110 2023-12-19T16:57:53.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.605+0000 7f9cf7b60640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf006f7c0 con 0x7f9cf0159830 2023-12-19T16:57:53.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.605+0000 7f9cf7b60640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00ca2b0 con 0x7f9cf00c9310 2023-12-19T16:57:53.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.605+0000 7f9cf58fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9cf0159830 0x7f9cf015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.605+0000 7f9cf58fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9cf0159830 0x7f9cf015d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43530/0 (socket says 172.21.15.26:43530) 2023-12-19T16:57:53.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.605+0000 7f9cf58fc640 1 -- 172.21.15.26:0/1269632855 learned_addr learned my addr 172.21.15.26:0/1269632855 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:53.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.605+0000 7f9cf50fb640 1 --2- 172.21.15.26:0/1269632855 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9cf00c9310 0x7f9cf00cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf58fc640 1 -- 172.21.15.26:0/1269632855 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9cf00c9310 msgr2=0x7f9cf00cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:53.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf58fc640 1 --2- 172.21.15.26:0/1269632855 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9cf00c9310 0x7f9cf00cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf58fc640 1 -- 172.21.15.26:0/1269632855 >> v1:172.21.15.160:6789/0 conn(0x7f9cf00cb110 legacy=0x7f9cf00701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:53.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf58fc640 1 -- 172.21.15.26:0/1269632855 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf015de10 con 0x7f9cf0159830 2023-12-19T16:57:53.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf58fc640 1 --2- 172.21.15.26:0/1269632855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cf0159830 0x7f9cf015d4e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009870 tx=0x7f9cec01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a71e7ef0eb1cc8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:53.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf48fa640 1 -- 172.21.15.26:0/1269632855 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0ab680 con 0x7f9cf0159830 2023-12-19T16:57:53.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf48fa640 1 -- 172.21.15.26:0/1269632855 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec0ab7e0 con 0x7f9cf0159830 2023-12-19T16:57:53.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf48fa640 1 -- 172.21.15.26:0/1269632855 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0abb70 con 0x7f9cf0159830 2023-12-19T16:57:53.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf7b60640 1 -- 172.21.15.26:0/1269632855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cf0159830 msgr2=0x7f9cf015d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:53.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf7b60640 1 --2- 172.21.15.26:0/1269632855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cf0159830 0x7f9cf015d4e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009870 tx=0x7f9cec01c480 comp rx=0 tx=0).stop 2023-12-19T16:57:53.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf7b60640 1 -- 172.21.15.26:0/1269632855 shutdown_connections 2023-12-19T16:57:53.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf7b60640 1 --2- 172.21.15.26:0/1269632855 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9cf00c9310 0x7f9cf00cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf7b60640 1 --2- 172.21.15.26:0/1269632855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cf0159830 0x7f9cf015d4e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.609+0000 7f9cf7b60640 1 -- 172.21.15.26:0/1269632855 >> 172.21.15.26:0/1269632855 conn(0x7f9cf00b0270 msgr2=0x7f9cf00b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:53.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf7b60640 1 -- 172.21.15.26:0/1269632855 shutdown_connections 2023-12-19T16:57:53.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf7b60640 1 -- 172.21.15.26:0/1269632855 wait complete. 2023-12-19T16:57:53.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf7b60640 1 Processor -- start 2023-12-19T16:57:53.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf7b60640 1 -- start start 2023-12-19T16:57:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf7b60640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cf00c9310 0x7f9cf00c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf7b60640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9cf00cb110 0x7f9cf014ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf7b60640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9cf0159830 0x7f9cf01578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf7b60640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf01503a0 con 0x7f9cf00c9310 2023-12-19T16:57:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf7b60640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00cc5b0 con 0x7f9cf0159830 2023-12-19T16:57:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf7b60640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0158290 con 0x7f9cf00cb110 2023-12-19T16:57:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf58fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cf00c9310 0x7f9cf00c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf58fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cf00c9310 0x7f9cf00c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43534/0 (socket says 172.21.15.26:43534) 2023-12-19T16:57:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf58fc640 1 -- 172.21.15.26:0/2062581304 learned_addr learned my addr 172.21.15.26:0/2062581304 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf50fb640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9cf00cb110 0x7f9cf014ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf60fd640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9cf0159830 0x7f9cf01578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf58fc640 1 -- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9cf00cb110 msgr2=0x7f9cf014ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf58fc640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9cf00cb110 0x7f9cf014ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf58fc640 1 -- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9cf0159830 msgr2=0x7f9cf01578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf58fc640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9cf0159830 0x7f9cf01578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf58fc640 1 -- 172.21.15.26:0/2062581304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec0094d0 con 0x7f9cf00c9310 2023-12-19T16:57:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf50fb640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9cf00cb110 0x7f9cf014ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf60fd640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9cf0159830 0x7f9cf01578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf58fc640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cf00c9310 0x7f9cf00c5ca0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0ad040 tx=0x7f9cec01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9ce67fc640 1 -- 172.21.15.26:0/2062581304 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0b4800 con 0x7f9cf00c9310 2023-12-19T16:57:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9ce67fc640 1 -- 172.21.15.26:0/2062581304 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec0b4980 con 0x7f9cf00c9310 2023-12-19T16:57:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9ce67fc640 1 -- 172.21.15.26:0/2062581304 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec0b4d30 con 0x7f9cf00c9310 2023-12-19T16:57:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.613+0000 7f9cf7b60640 1 -- 172.21.15.26:0/2062581304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf00cbab0 con 0x7f9cf00c9310 2023-12-19T16:57:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.617+0000 7f9cf7b60640 1 -- 172.21.15.26:0/2062581304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf0158850 con 0x7f9cf00c9310 2023-12-19T16:57:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.617+0000 7f9cf7b60640 1 -- 172.21.15.26:0/2062581304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf0074270 con 0x7f9cf00c9310 2023-12-19T16:57:53.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.617+0000 7f9ce67fc640 1 -- 172.21.15.26:0/2062581304 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9cec0c7440 con 0x7f9cf00c9310 2023-12-19T16:57:53.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.617+0000 7f9ce67fc640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9cd4065bb0 0x7f9cd4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:53.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.617+0000 7f9ce67fc640 1 -- 172.21.15.26:0/2062581304 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f9cec11e190 con 0x7f9cf00c9310 2023-12-19T16:57:53.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.617+0000 7f9cf50fb640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9cd4065bb0 0x7f9cd4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:53.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.621+0000 7f9cf50fb640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9cd4065bb0 0x7f9cd4068070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9cf01595a0 tx=0x7f9ce0008040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:53.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.621+0000 7f9ce67fc640 1 -- 172.21.15.26:0/2062581304 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9cec0eb990 con 0x7f9cf00c9310 2023-12-19T16:57:53.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:53.781+0000 7f9cf7b60640 1 -- 172.21.15.26:0/2062581304 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9cf00c9760 con 0x7f9cf00c9310 2023-12-19T16:57:54.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.773+0000 7f9ce67fc640 1 -- 172.21.15.26:0/2062581304 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v90) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9cec0ab640 con 0x7f9cf00c9310 2023-12-19T16:57:54.778 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T16:57:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.777+0000 7f9cf7b60640 1 -- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9cd4065bb0 msgr2=0x7f9cd4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.777+0000 7f9cf7b60640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9cd4065bb0 0x7f9cd4068070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9cf01595a0 tx=0x7f9ce0008040 comp rx=0 tx=0).stop 2023-12-19T16:57:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.777+0000 7f9cf7b60640 1 -- 172.21.15.26:0/2062581304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cf00c9310 msgr2=0x7f9cf00c5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.777+0000 7f9cf7b60640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cf00c9310 0x7f9cf00c5ca0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0ad040 tx=0x7f9cec01acc0 comp rx=0 tx=0).stop 2023-12-19T16:57:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.777+0000 7f9cf7b60640 1 -- 172.21.15.26:0/2062581304 shutdown_connections 2023-12-19T16:57:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.777+0000 7f9cf7b60640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9cf0159830 0x7f9cf01578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.777+0000 7f9cf7b60640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9cd4065bb0 0x7f9cd4068070 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.777+0000 7f9cf7b60640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9cf00cb110 0x7f9cf014ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.777+0000 7f9cf7b60640 1 --2- 172.21.15.26:0/2062581304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cf00c9310 0x7f9cf00c5ca0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.777+0000 7f9cf7b60640 1 -- 172.21.15.26:0/2062581304 >> 172.21.15.26:0/2062581304 conn(0x7f9cf00b0270 msgr2=0x7f9cf006b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.777+0000 7f9cf7b60640 1 -- 172.21.15.26:0/2062581304 shutdown_connections 2023-12-19T16:57:54.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.781+0000 7f9cf7b60640 1 -- 172.21.15.26:0/2062581304 wait complete. 2023-12-19T16:57:54.808 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T16:57:54.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.957+0000 7f98e8d0b640 1 Processor -- start 2023-12-19T16:57:54.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.957+0000 7f98e8d0b640 1 -- start start 2023-12-19T16:57:54.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e8d0b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f98e4157ba0 0x7f98e406b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:54.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e8d0b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e406b8f0 0x7f98e40ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:54.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e8d0b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e4158050 con 0x7f98e4153f20 2023-12-19T16:57:54.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e8d0b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e40b0600 con 0x7f98e4157ba0 2023-12-19T16:57:54.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e8d0b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e406bd80 con 0x7f98e406b8f0 2023-12-19T16:57:54.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f98e4157ba0 0x7f98e406b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:54.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f98e4157ba0 0x7f98e406b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43550/0 (socket says 172.21.15.26:43550) 2023-12-19T16:57:54.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e2ffd640 1 -- 172.21.15.26:0/4214404177 learned_addr learned my addr 172.21.15.26:0/4214404177 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:54.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e37fe640 1 --1- 172.21.15.26:0/4214404177 >> v1:172.21.15.160:6789/0 conn(0x7f98e4153f20 0x7f98e40b3270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46242/0 (socket says 172.21.15.26:46242) 2023-12-19T16:57:54.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e3fff640 1 --2- 172.21.15.26:0/4214404177 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e406b8f0 0x7f98e40ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:54.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e2ffd640 1 -- 172.21.15.26:0/4214404177 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e406b8f0 msgr2=0x7f98e40ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:54.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e2ffd640 1 --2- 172.21.15.26:0/4214404177 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e406b8f0 0x7f98e40ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:54.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e2ffd640 1 -- 172.21.15.26:0/4214404177 >> v1:172.21.15.160:6789/0 conn(0x7f98e4153f20 legacy=0x7f98e40b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:54.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e2ffd640 1 -- 172.21.15.26:0/4214404177 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e415a2b0 con 0x7f98e4157ba0 2023-12-19T16:57:54.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.961+0000 7f98e27fc640 1 -- 172.21.15.26:0/4214404177 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176471996 0 0) 0x7f98e4158050 con 0x7f98e4153f20 2023-12-19T16:57:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e2ffd640 1 --2- 172.21.15.26:0/4214404177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e4157ba0 0x7f98e406b3b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009870 tx=0x7f98d801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517fb96fead14492 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e27fc640 1 -- 172.21.15.26:0/4214404177 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80ab680 con 0x7f98e4157ba0 2023-12-19T16:57:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e27fc640 1 -- 172.21.15.26:0/4214404177 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d80ab7e0 con 0x7f98e4157ba0 2023-12-19T16:57:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e27fc640 1 -- 172.21.15.26:0/4214404177 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80abb70 con 0x7f98e4157ba0 2023-12-19T16:57:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 -- 172.21.15.26:0/4214404177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e4157ba0 msgr2=0x7f98e406b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 --2- 172.21.15.26:0/4214404177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e4157ba0 0x7f98e406b3b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009870 tx=0x7f98d801c480 comp rx=0 tx=0).stop 2023-12-19T16:57:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 -- 172.21.15.26:0/4214404177 shutdown_connections 2023-12-19T16:57:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 --2- 172.21.15.26:0/4214404177 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e406b8f0 0x7f98e40ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 --2- 172.21.15.26:0/4214404177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e4157ba0 0x7f98e406b3b0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:54.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 -- 172.21.15.26:0/4214404177 >> 172.21.15.26:0/4214404177 conn(0x7f98e40658e0 msgr2=0x7f98e4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:54.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 -- 172.21.15.26:0/4214404177 shutdown_connections 2023-12-19T16:57:54.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 -- 172.21.15.26:0/4214404177 wait complete. 2023-12-19T16:57:54.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 Processor -- start 2023-12-19T16:57:54.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 -- start start 2023-12-19T16:57:54.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98e406b8f0 0x7f98e41453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e4153f20 0x7f98e415c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e4157ba0 0x7f98e415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e4155380 con 0x7f98e4153f20 2023-12-19T16:57:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e41458e0 con 0x7f98e406b8f0 2023-12-19T16:57:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.965+0000 7f98e8d0b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e4154300 con 0x7f98e4157ba0 2023-12-19T16:57:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e4153f20 0x7f98e415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e4153f20 0x7f98e415c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43566/0 (socket says 172.21.15.26:43566) 2023-12-19T16:57:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e2ffd640 1 -- 172.21.15.26:0/3625165030 learned_addr learned my addr 172.21.15.26:0/3625165030 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e37fe640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98e406b8f0 0x7f98e41453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e3fff640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e4157ba0 0x7f98e415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e2ffd640 1 -- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e4157ba0 msgr2=0x7f98e415c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e2ffd640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e4157ba0 0x7f98e415c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e2ffd640 1 -- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98e406b8f0 msgr2=0x7f98e41453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e2ffd640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98e406b8f0 0x7f98e41453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e2ffd640 1 -- 172.21.15.26:0/3625165030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d80094d0 con 0x7f98e4153f20 2023-12-19T16:57:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e2ffd640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e4153f20 0x7f98e415c080 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009870 tx=0x7f98d8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98cbfff640 1 -- 172.21.15.26:0/3625165030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8004220 con 0x7f98e4153f20 2023-12-19T16:57:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98cbfff640 1 -- 172.21.15.26:0/3625165030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d8004380 con 0x7f98e4153f20 2023-12-19T16:57:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e8d0b640 1 -- 172.21.15.26:0/3625165030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e41545c0 con 0x7f98e4153f20 2023-12-19T16:57:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98cbfff640 1 -- 172.21.15.26:0/3625165030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8003820 con 0x7f98e4153f20 2023-12-19T16:57:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e8d0b640 1 -- 172.21.15.26:0/3625165030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e4154b80 con 0x7f98e4153f20 2023-12-19T16:57:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98e8d0b640 1 -- 172.21.15.26:0/3625165030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e4061930 con 0x7f98e4153f20 2023-12-19T16:57:54.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.969+0000 7f98cbfff640 1 -- 172.21.15.26:0/3625165030 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f98d8003b90 con 0x7f98e4153f20 2023-12-19T16:57:54.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.973+0000 7f98cbfff640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f98d0065bb0 0x7f98d0068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:54.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.973+0000 7f98cbfff640 1 -- 172.21.15.26:0/3625165030 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5959+0+0 (secure 0 0 0) 0x7f98d811e890 con 0x7f98e4153f20 2023-12-19T16:57:54.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.973+0000 7f98e37fe640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f98d0065bb0 0x7f98d0068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:54.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.973+0000 7f98e37fe640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f98d0065bb0 0x7f98d0068070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f98d4000f70 tx=0x7f98d4007240 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:54.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:54.973+0000 7f98cbfff640 1 -- 172.21.15.26:0/3625165030 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f98d80ec050 con 0x7f98e4153f20 2023-12-19T16:57:55.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.133+0000 7f98e8d0b640 1 -- 172.21.15.26:0/3625165030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f98e40b2b40 con 0x7f98e4153f20 2023-12-19T16:57:55.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.781+0000 7f98cbfff640 1 -- 172.21.15.26:0/3625165030 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v91) v1 ==== 170+0+0 (secure 0 0 0) 0x7f98d80c40d0 con 0x7f98e4153f20 2023-12-19T16:57:55.788 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T16:57:55.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.789+0000 7f98e8d0b640 1 -- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f98d0065bb0 msgr2=0x7f98d0068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:55.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.789+0000 7f98e8d0b640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f98d0065bb0 0x7f98d0068070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f98d4000f70 tx=0x7f98d4007240 comp rx=0 tx=0).stop 2023-12-19T16:57:55.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.789+0000 7f98e8d0b640 1 -- 172.21.15.26:0/3625165030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e4153f20 msgr2=0x7f98e415c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:55.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.789+0000 7f98e8d0b640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e4153f20 0x7f98e415c080 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009870 tx=0x7f98d8019220 comp rx=0 tx=0).stop 2023-12-19T16:57:55.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.789+0000 7f98e8d0b640 1 -- 172.21.15.26:0/3625165030 shutdown_connections 2023-12-19T16:57:55.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.789+0000 7f98e8d0b640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f98d0065bb0 0x7f98d0068070 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:55.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.789+0000 7f98e8d0b640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98e4157ba0 0x7f98e415c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:55.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.789+0000 7f98e8d0b640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98e4153f20 0x7f98e415c080 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:55.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.789+0000 7f98e8d0b640 1 --2- 172.21.15.26:0/3625165030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98e406b8f0 0x7f98e41453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:55.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.789+0000 7f98e8d0b640 1 -- 172.21.15.26:0/3625165030 >> 172.21.15.26:0/3625165030 conn(0x7f98e40658e0 msgr2=0x7f98e40b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:55.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.789+0000 7f98e8d0b640 1 -- 172.21.15.26:0/3625165030 shutdown_connections 2023-12-19T16:57:55.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.789+0000 7f98e8d0b640 1 -- 172.21.15.26:0/3625165030 wait complete. 2023-12-19T16:57:55.830 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T16:57:55.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.973+0000 7fd279212640 1 Processor -- start 2023-12-19T16:57:55.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.973+0000 7fd279212640 1 -- start start 2023-12-19T16:57:55.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.973+0000 7fd279212640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd27415ada0 0x7fd2740706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:55.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.973+0000 7fd279212640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd27406ac80 0x7fd2740b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:55.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.973+0000 7fd279212640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd274070d90 con 0x7fd27415a9c0 2023-12-19T16:57:55.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.973+0000 7fd279212640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2740cc270 con 0x7fd27415ada0 2023-12-19T16:57:55.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.973+0000 7fd279212640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd274071070 con 0x7fd27406ac80 2023-12-19T16:57:55.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd272d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd27415a9c0 0x7fd2740b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52508/0 (socket says 172.21.15.26:52508) 2023-12-19T16:57:55.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd272d9d640 1 -- 172.21.15.26:0/1987646669 learned_addr learned my addr 172.21.15.26:0/1987646669 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:55.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd27259c640 1 --2- 172.21.15.26:0/1987646669 >> v2:172.21.15.26:3300/0 conn(0x7fd27415ada0 0x7fd2740706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:55.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd27359e640 1 --2- 172.21.15.26:0/1987646669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd27406ac80 0x7fd2740b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:55.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd271d9b640 1 -- 172.21.15.26:0/1987646669 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 802722535 0 0) 0x7fd274070d90 con 0x7fd27415a9c0 2023-12-19T16:57:55.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd271d9b640 1 -- 172.21.15.26:0/1987646669 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd25c003530 con 0x7fd27415a9c0 2023-12-19T16:57:55.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd27259c640 1 -- 172.21.15.26:0/1987646669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd27406ac80 msgr2=0x7fd2740b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:55.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd27259c640 1 --2- 172.21.15.26:0/1987646669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd27406ac80 0x7fd2740b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:55.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd27259c640 1 -- 172.21.15.26:0/1987646669 >> v1:172.21.15.26:6789/0 conn(0x7fd27415a9c0 legacy=0x7fd2740b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:55.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd27259c640 1 -- 172.21.15.26:0/1987646669 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd274071350 con 0x7fd27415ada0 2023-12-19T16:57:55.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd27259c640 1 --2- 172.21.15.26:0/1987646669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27415ada0 0x7fd2740706d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd2680097c0 tx=0x7fd26801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186f92af28508ff3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:55.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd271d9b640 1 -- 172.21.15.26:0/1987646669 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2680ab600 con 0x7fd27415ada0 2023-12-19T16:57:55.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd271d9b640 1 -- 172.21.15.26:0/1987646669 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2680a7050 con 0x7fd27415ada0 2023-12-19T16:57:55.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.977+0000 7fd271d9b640 1 -- 172.21.15.26:0/1987646669 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2680ab980 con 0x7fd27415ada0 2023-12-19T16:57:55.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 -- 172.21.15.26:0/1987646669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27415ada0 msgr2=0x7fd2740706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:55.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 --2- 172.21.15.26:0/1987646669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27415ada0 0x7fd2740706d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd2680097c0 tx=0x7fd26801c3c0 comp rx=0 tx=0).stop 2023-12-19T16:57:55.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 -- 172.21.15.26:0/1987646669 shutdown_connections 2023-12-19T16:57:55.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 --2- 172.21.15.26:0/1987646669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd27406ac80 0x7fd2740b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:55.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 --2- 172.21.15.26:0/1987646669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27415ada0 0x7fd2740706d0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:55.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 -- 172.21.15.26:0/1987646669 >> 172.21.15.26:0/1987646669 conn(0x7fd2740ce730 msgr2=0x7fd2740ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:55.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 -- 172.21.15.26:0/1987646669 shutdown_connections 2023-12-19T16:57:55.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 -- 172.21.15.26:0/1987646669 wait complete. 2023-12-19T16:57:55.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 Processor -- start 2023-12-19T16:57:55.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 -- start start 2023-12-19T16:57:55.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd27406ac80 0x7fd2740b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:55.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd27415a9c0 0x7fd274159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:55.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27415ada0 0x7fd274152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:55.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2741591f0 con 0x7fd27415ada0 2023-12-19T16:57:55.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2740b3b50 con 0x7fd27406ac80 2023-12-19T16:57:55.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd279212640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2740c8010 con 0x7fd27415a9c0 2023-12-19T16:57:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd27359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27415ada0 0x7fd274152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd27359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27415ada0 0x7fd274152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52294/0 (socket says 172.21.15.26:52294) 2023-12-19T16:57:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd27359e640 1 -- 172.21.15.26:0/824192256 learned_addr learned my addr 172.21.15.26:0/824192256 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd272d9d640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd27406ac80 0x7fd2740b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.981+0000 7fd27259c640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd27415a9c0 0x7fd274159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd27359e640 1 -- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd27415a9c0 msgr2=0x7fd274159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd27359e640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd27415a9c0 0x7fd274159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd27359e640 1 -- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd27406ac80 msgr2=0x7fd2740b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd27359e640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd27406ac80 0x7fd2740b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd27359e640 1 -- 172.21.15.26:0/824192256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd268009420 con 0x7fd27415ada0 2023-12-19T16:57:55.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd27359e640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27415ada0 0x7fd274152810 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00efc0 tx=0x7fd26c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:55.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd25b7fe640 1 -- 172.21.15.26:0/824192256 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c014480 con 0x7fd27415ada0 2023-12-19T16:57:55.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd25b7fe640 1 -- 172.21.15.26:0/824192256 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26c00f040 con 0x7fd27415ada0 2023-12-19T16:57:55.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd25b7fe640 1 -- 172.21.15.26:0/824192256 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c014820 con 0x7fd27415ada0 2023-12-19T16:57:55.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd279212640 1 -- 172.21.15.26:0/824192256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2740c82f0 con 0x7fd27415ada0 2023-12-19T16:57:55.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd279212640 1 -- 172.21.15.26:0/824192256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd274159b50 con 0x7fd27415ada0 2023-12-19T16:57:55.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd279212640 1 -- 172.21.15.26:0/824192256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd274159330 con 0x7fd27415ada0 2023-12-19T16:57:55.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd25b7fe640 1 -- 172.21.15.26:0/824192256 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd26c014980 con 0x7fd27415ada0 2023-12-19T16:57:55.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.985+0000 7fd25b7fe640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd260065b80 0x7fd260068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:55.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.989+0000 7fd25b7fe640 1 -- 172.21.15.26:0/824192256 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6341+0+0 (secure 0 0 0) 0x7fd26c084010 con 0x7fd27415ada0 2023-12-19T16:57:55.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.989+0000 7fd272d9d640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd260065b80 0x7fd260068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:55.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.989+0000 7fd272d9d640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd260065b80 0x7fd260068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd264007ad0 tx=0x7fd264007a40 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:55.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:55.993+0000 7fd25b7fe640 1 -- 172.21.15.26:0/824192256 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd274159330 con 0x7fd27415ada0 2023-12-19T16:57:56.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.181+0000 7fd279212640 1 -- 172.21.15.26:0/824192256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd274159330 con 0x7fd27415ada0 2023-12-19T16:57:56.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.189+0000 7fd25b7fe640 1 -- 172.21.15.26:0/824192256 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v37) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd274159330 con 0x7fd27415ada0 2023-12-19T16:57:56.195 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 10 and data pool 11 2023-12-19T16:57:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.197+0000 7fd279212640 1 -- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd260065b80 msgr2=0x7fd260068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.197+0000 7fd279212640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd260065b80 0x7fd260068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd264007ad0 tx=0x7fd264007a40 comp rx=0 tx=0).stop 2023-12-19T16:57:56.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.197+0000 7fd279212640 1 -- 172.21.15.26:0/824192256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27415ada0 msgr2=0x7fd274152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:56.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.197+0000 7fd279212640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27415ada0 0x7fd274152810 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00efc0 tx=0x7fd26c00c490 comp rx=0 tx=0).stop 2023-12-19T16:57:56.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.197+0000 7fd279212640 1 -- 172.21.15.26:0/824192256 shutdown_connections 2023-12-19T16:57:56.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.197+0000 7fd279212640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd27415ada0 0x7fd274152810 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.197+0000 7fd279212640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd260065b80 0x7fd260068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.197+0000 7fd279212640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd27415a9c0 0x7fd274159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.197+0000 7fd279212640 1 --2- 172.21.15.26:0/824192256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd27406ac80 0x7fd2740b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.197+0000 7fd279212640 1 -- 172.21.15.26:0/824192256 >> 172.21.15.26:0/824192256 conn(0x7fd2740ce730 msgr2=0x7fd2740ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:56.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.201+0000 7fd279212640 1 -- 172.21.15.26:0/824192256 shutdown_connections 2023-12-19T16:57:56.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.201+0000 7fd279212640 1 -- 172.21.15.26:0/824192256 wait complete. 2023-12-19T16:57:56.240 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:56.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.377+0000 7fd398a10640 1 Processor -- start 2023-12-19T16:57:56.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.377+0000 7fd398a10640 1 -- start start 2023-12-19T16:57:56.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd398a10640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3940cc0a0 0x7fd3940cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:56.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd398a10640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd394159d20 0x7fd3940cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:56.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd398a10640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3940adde0 con 0x7fd3940cdee0 2023-12-19T16:57:56.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd398a10640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3940c9c20 con 0x7fd3940cc0a0 2023-12-19T16:57:56.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd398a10640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940b28d0 con 0x7fd394159d20 2023-12-19T16:57:56.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd392d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3940cc0a0 0x7fd3940cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:56.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd39359e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd3940cdee0 0x7fd39415d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52512/0 (socket says 172.21.15.26:52512) 2023-12-19T16:57:56.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd39359e640 1 -- 172.21.15.26:0/4128196819 learned_addr learned my addr 172.21.15.26:0/4128196819 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:56.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd38bfff640 1 --2- 172.21.15.26:0/4128196819 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd394159d20 0x7fd3940cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd392d9d640 1 -- 172.21.15.26:0/4128196819 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd394159d20 msgr2=0x7fd3940cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd392d9d640 1 --2- 172.21.15.26:0/4128196819 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd394159d20 0x7fd3940cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd392d9d640 1 -- 172.21.15.26:0/4128196819 >> v1:172.21.15.26:6789/0 conn(0x7fd3940cdee0 legacy=0x7fd39415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd392d9d640 1 -- 172.21.15.26:0/4128196819 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3940ccf00 con 0x7fd3940cc0a0 2023-12-19T16:57:56.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.381+0000 7fd39259c640 1 -- 172.21.15.26:0/4128196819 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089417550 0 0) 0x7fd3940adde0 con 0x7fd3940cdee0 2023-12-19T16:57:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd392d9d640 1 --2- 172.21.15.26:0/4128196819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3940cc0a0 0x7fd3940cc480 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd3840097c0 tx=0x7fd38401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb7a8282625c495 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd39259c640 1 -- 172.21.15.26:0/4128196819 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840ab600 con 0x7fd3940cc0a0 2023-12-19T16:57:56.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd39259c640 1 -- 172.21.15.26:0/4128196819 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3840a7050 con 0x7fd3940cc0a0 2023-12-19T16:57:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd39259c640 1 -- 172.21.15.26:0/4128196819 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840ab980 con 0x7fd3940cc0a0 2023-12-19T16:57:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd398a10640 1 -- 172.21.15.26:0/4128196819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3940cc0a0 msgr2=0x7fd3940cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd398a10640 1 --2- 172.21.15.26:0/4128196819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3940cc0a0 0x7fd3940cc480 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd3840097c0 tx=0x7fd38401c3c0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd398a10640 1 -- 172.21.15.26:0/4128196819 shutdown_connections 2023-12-19T16:57:56.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd398a10640 1 --2- 172.21.15.26:0/4128196819 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd394159d20 0x7fd3940cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd398a10640 1 --2- 172.21.15.26:0/4128196819 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3940cc0a0 0x7fd3940cc480 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd398a10640 1 -- 172.21.15.26:0/4128196819 >> 172.21.15.26:0/4128196819 conn(0x7fd39406b0f0 msgr2=0x7fd39406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd398a10640 1 -- 172.21.15.26:0/4128196819 shutdown_connections 2023-12-19T16:57:56.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd398a10640 1 -- 172.21.15.26:0/4128196819 wait complete. 2023-12-19T16:57:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd398a10640 1 Processor -- start 2023-12-19T16:57:56.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.385+0000 7fd398a10640 1 -- start start 2023-12-19T16:57:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd398a10640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3940cc0a0 0x7fd3940af240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd398a10640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3940cdee0 0x7fd3940c2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd398a10640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd394159d20 0x7fd3940c2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd398a10640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39406f610 con 0x7fd3940cdee0 2023-12-19T16:57:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd398a10640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940c4990 con 0x7fd3940cc0a0 2023-12-19T16:57:56.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd398a10640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3940c2b70 con 0x7fd394159d20 2023-12-19T16:57:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3940cdee0 0x7fd3940c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3940cdee0 0x7fd3940c2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52302/0 (socket says 172.21.15.26:52302) 2023-12-19T16:57:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd38bfff640 1 -- 172.21.15.26:0/940044654 learned_addr learned my addr 172.21.15.26:0/940044654 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd38bfff640 1 -- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd394159d20 msgr2=0x7fd3940c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd38bfff640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd394159d20 0x7fd3940c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd38bfff640 1 -- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3940cc0a0 msgr2=0x7fd3940af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:56.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd392d9d640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3940cc0a0 0x7fd3940af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd38bfff640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3940cc0a0 0x7fd3940af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd38bfff640 1 -- 172.21.15.26:0/940044654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd384009420 con 0x7fd3940cdee0 2023-12-19T16:57:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd392d9d640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3940cc0a0 0x7fd3940af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd38bfff640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3940cdee0 0x7fd3940c2000 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00e900 tx=0x7fd37c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd38b7fe640 1 -- 172.21.15.26:0/940044654 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c009690 con 0x7fd3940cdee0 2023-12-19T16:57:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd38b7fe640 1 -- 172.21.15.26:0/940044654 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37c00ca00 con 0x7fd3940cdee0 2023-12-19T16:57:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd38b7fe640 1 -- 172.21.15.26:0/940044654 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37c00cdb0 con 0x7fd3940cdee0 2023-12-19T16:57:56.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd398a10640 1 -- 172.21.15.26:0/940044654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3940c2e50 con 0x7fd3940cdee0 2023-12-19T16:57:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd398a10640 1 -- 172.21.15.26:0/940044654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39406f2a0 con 0x7fd3940cdee0 2023-12-19T16:57:56.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd398a10640 1 -- 172.21.15.26:0/940044654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39406f7c0 con 0x7fd3940cdee0 2023-12-19T16:57:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.389+0000 7fd38b7fe640 1 -- 172.21.15.26:0/940044654 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd37c013470 con 0x7fd3940cdee0 2023-12-19T16:57:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.393+0000 7fd38b7fe640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd374065b80 0x7fd374068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.393+0000 7fd38b7fe640 1 -- 172.21.15.26:0/940044654 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fd37c083940 con 0x7fd3940cdee0 2023-12-19T16:57:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.393+0000 7fd392d9d640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd374065b80 0x7fd374068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:56.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.393+0000 7fd392d9d640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd374065b80 0x7fd374068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd3840ad000 tx=0x7fd3840a6000 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:56.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.397+0000 7fd38b7fe640 1 -- 172.21.15.26:0/940044654 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd39406f7c0 con 0x7fd3940cdee0 2023-12-19T16:57:56.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.553+0000 7fd398a10640 1 -- 172.21.15.26:0/940044654 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd394067400 con 0x7fd3940cdee0 2023-12-19T16:57:56.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.557+0000 7fd38b7fe640 1 -- 172.21.15.26:0/940044654 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17218 (secure 0 0 0) 0x7fd37c050e70 con 0x7fd3940cdee0 2023-12-19T16:57:56.561 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:56.561 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":92,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:56.187204+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:53.833233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:55.184912+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:56.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.557+0000 7fd398a10640 1 -- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd374065b80 msgr2=0x7fd374068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:56.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.557+0000 7fd398a10640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd374065b80 0x7fd374068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd3840ad000 tx=0x7fd3840a6000 comp rx=0 tx=0).stop 2023-12-19T16:57:56.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.557+0000 7fd398a10640 1 -- 172.21.15.26:0/940044654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3940cdee0 msgr2=0x7fd3940c2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:56.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.557+0000 7fd398a10640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3940cdee0 0x7fd3940c2000 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00e900 tx=0x7fd37c00edc0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.557+0000 7fd398a10640 1 -- 172.21.15.26:0/940044654 shutdown_connections 2023-12-19T16:57:56.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.557+0000 7fd398a10640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd374065b80 0x7fd374068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.557+0000 7fd398a10640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd394159d20 0x7fd3940c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.557+0000 7fd398a10640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3940cdee0 0x7fd3940c2000 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.557+0000 7fd398a10640 1 --2- 172.21.15.26:0/940044654 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3940cc0a0 0x7fd3940af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.557+0000 7fd398a10640 1 -- 172.21.15.26:0/940044654 >> 172.21.15.26:0/940044654 conn(0x7fd39406b0f0 msgr2=0x7fd394158dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.557+0000 7fd398a10640 1 -- 172.21.15.26:0/940044654 shutdown_connections 2023-12-19T16:57:56.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.557+0000 7fd398a10640 1 -- 172.21.15.26:0/940044654 wait complete. 2023-12-19T16:57:56.582 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:56.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c50ed1640 1 Processor -- start 2023-12-19T16:57:56.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c50ed1640 1 -- start start 2023-12-19T16:57:56.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c50ed1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6c4c154220 0x7f6c4c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c50ed1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c4c06b3d0 0x7f6c4c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c50ed1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6c4c0cec40 0x7f6c4c15d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c50ed1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c4c0cf0d0 con 0x7f6c4c0cec40 2023-12-19T16:57:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c50ed1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c4c06bc30 con 0x7f6c4c154220 2023-12-19T16:57:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c50ed1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c0cc6e0 con 0x7f6c4c06b3d0 2023-12-19T16:57:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c4ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6c4c0cec40 0x7f6c4c15d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c4ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6c4c0cec40 0x7f6c4c15d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52304/0 (socket says 172.21.15.26:52304) 2023-12-19T16:57:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c4ad9d640 1 -- 172.21.15.26:0/743507937 learned_addr learned my addr 172.21.15.26:0/743507937 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c49d9b640 1 --2- 172.21.15.26:0/743507937 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c4c06b3d0 0x7f6c4c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c4a59c640 1 --2- 172.21.15.26:0/743507937 >> v2:172.21.15.160:3300/0 conn(0x7f6c4c154220 0x7f6c4c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c4ad9d640 1 -- 172.21.15.26:0/743507937 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c4c06b3d0 msgr2=0x7f6c4c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:56.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c4ad9d640 1 --2- 172.21.15.26:0/743507937 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c4c06b3d0 0x7f6c4c0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c4ad9d640 1 -- 172.21.15.26:0/743507937 >> v2:172.21.15.160:3300/0 conn(0x7f6c4c154220 msgr2=0x7f6c4c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:56.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c4ad9d640 1 --2- 172.21.15.26:0/743507937 >> v2:172.21.15.160:3300/0 conn(0x7f6c4c154220 0x7f6c4c15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.721+0000 7f6c4ad9d640 1 -- 172.21.15.26:0/743507937 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c15afc0 con 0x7f6c4c0cec40 2023-12-19T16:57:56.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.725+0000 7f6c4ad9d640 1 --2- 172.21.15.26:0/743507937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c0cec40 0x7f6c4c15d5f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0667b0 tx=0x7f6c4401c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1817b3711867fe server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:56.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.725+0000 7f6c4959a640 1 -- 172.21.15.26:0/743507937 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440a5560 con 0x7f6c4c0cec40 2023-12-19T16:57:56.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.725+0000 7f6c4959a640 1 -- 172.21.15.26:0/743507937 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c440a56c0 con 0x7f6c4c0cec40 2023-12-19T16:57:56.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.725+0000 7f6c4959a640 1 -- 172.21.15.26:0/743507937 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440a5a50 con 0x7f6c4c0cec40 2023-12-19T16:57:56.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.725+0000 7f6c50ed1640 1 -- 172.21.15.26:0/743507937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c0cec40 msgr2=0x7f6c4c15d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:56.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.725+0000 7f6c50ed1640 1 --2- 172.21.15.26:0/743507937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c0cec40 0x7f6c4c15d5f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0667b0 tx=0x7f6c4401c6e0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.725+0000 7f6c50ed1640 1 -- 172.21.15.26:0/743507937 shutdown_connections 2023-12-19T16:57:56.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.725+0000 7f6c50ed1640 1 --2- 172.21.15.26:0/743507937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c0cec40 0x7f6c4c15d5f0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.725+0000 7f6c50ed1640 1 --2- 172.21.15.26:0/743507937 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c4c06b3d0 0x7f6c4c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.725+0000 7f6c50ed1640 1 --2- 172.21.15.26:0/743507937 >> v2:172.21.15.160:3300/0 conn(0x7f6c4c154220 0x7f6c4c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.725+0000 7f6c50ed1640 1 -- 172.21.15.26:0/743507937 >> 172.21.15.26:0/743507937 conn(0x7f6c4c0658e0 msgr2=0x7f6c4c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:56.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.725+0000 7f6c50ed1640 1 -- 172.21.15.26:0/743507937 shutdown_connections 2023-12-19T16:57:56.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c50ed1640 1 -- 172.21.15.26:0/743507937 wait complete. 2023-12-19T16:57:56.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c50ed1640 1 Processor -- start 2023-12-19T16:57:56.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c50ed1640 1 -- start start 2023-12-19T16:57:56.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c50ed1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c4c06b3d0 0x7f6c4c14bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:56.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c50ed1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c4c0cec40 0x7f6c4c0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:56.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c50ed1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c154220 0x7f6c4c14cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:56.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c50ed1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c15b1a0 con 0x7f6c4c154220 2023-12-19T16:57:56.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c50ed1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c15b480 con 0x7f6c4c0cec40 2023-12-19T16:57:56.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c50ed1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c15b760 con 0x7f6c4c06b3d0 2023-12-19T16:57:56.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c4ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c154220 0x7f6c4c14cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:56.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c4ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c154220 0x7f6c4c14cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52308/0 (socket says 172.21.15.26:52308) 2023-12-19T16:57:56.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c4ad9d640 1 -- 172.21.15.26:0/1675550113 learned_addr learned my addr 172.21.15.26:0/1675550113 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:56.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c4a59c640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c4c06b3d0 0x7f6c4c14bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:56.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c49d9b640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c4c0cec40 0x7f6c4c0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:56.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c4ad9d640 1 -- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c4c06b3d0 msgr2=0x7f6c4c14bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:56.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c4ad9d640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c4c06b3d0 0x7f6c4c14bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c4ad9d640 1 -- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c4c0cec40 msgr2=0x7f6c4c0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:56.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c4ad9d640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c4c0cec40 0x7f6c4c0b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c4ad9d640 1 -- 172.21.15.26:0/1675550113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c44009b30 con 0x7f6c4c154220 2023-12-19T16:57:56.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c49d9b640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c4c0cec40 0x7f6c4c0b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:56.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c4ad9d640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c154220 0x7f6c4c14cff0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6c4401ad90 tx=0x7f6c44019480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:56.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.729+0000 7f6c3affd640 1 -- 172.21.15.26:0/1675550113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440a5560 con 0x7f6c4c154220 2023-12-19T16:57:56.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.733+0000 7f6c3affd640 1 -- 172.21.15.26:0/1675550113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c440b8690 con 0x7f6c4c154220 2023-12-19T16:57:56.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.733+0000 7f6c3affd640 1 -- 172.21.15.26:0/1675550113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440b8a80 con 0x7f6c4c154220 2023-12-19T16:57:56.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.733+0000 7f6c50ed1640 1 -- 172.21.15.26:0/1675550113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4c15ba40 con 0x7f6c4c154220 2023-12-19T16:57:56.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.733+0000 7f6c50ed1640 1 -- 172.21.15.26:0/1675550113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4c14c450 con 0x7f6c4c154220 2023-12-19T16:57:56.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.733+0000 7f6c50ed1640 1 -- 172.21.15.26:0/1675550113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4c074270 con 0x7f6c4c154220 2023-12-19T16:57:56.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.733+0000 7f6c3affd640 1 -- 172.21.15.26:0/1675550113 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6c440b8be0 con 0x7f6c4c154220 2023-12-19T16:57:56.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.733+0000 7f6c3affd640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c2c065b80 0x7f6c2c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:56.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.733+0000 7f6c3affd640 1 -- 172.21.15.26:0/1675550113 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f6c4411e9c0 con 0x7f6c4c154220 2023-12-19T16:57:56.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.733+0000 7f6c4a59c640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c2c065b80 0x7f6c2c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:56.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.737+0000 7f6c4a59c640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c2c065b80 0x7f6c2c068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009af0 tx=0x7f6c3c008040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.737+0000 7f6c3affd640 1 -- 172.21.15.26:0/1675550113 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6c440ec0b0 con 0x7f6c4c154220 2023-12-19T16:57:56.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.897+0000 7f6c50ed1640 1 -- 172.21.15.26:0/1675550113 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c4c15bef0 con 0x7f6c4c154220 2023-12-19T16:57:56.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.901+0000 7f6c3affd640 1 -- 172.21.15.26:0/1675550113 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17218 (secure 0 0 0) 0x7f6c44009510 con 0x7f6c4c154220 2023-12-19T16:57:56.905 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:56.905 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":92,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:56.187204+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:53.833233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:55.184912+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.901+0000 7f6c50ed1640 1 -- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c2c065b80 msgr2=0x7f6c2c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.901+0000 7f6c50ed1640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c2c065b80 0x7f6c2c068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009af0 tx=0x7f6c3c008040 comp rx=0 tx=0).stop 2023-12-19T16:57:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.901+0000 7f6c50ed1640 1 -- 172.21.15.26:0/1675550113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c154220 msgr2=0x7f6c4c14cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.901+0000 7f6c50ed1640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c154220 0x7f6c4c14cff0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6c4401ad90 tx=0x7f6c44019480 comp rx=0 tx=0).stop 2023-12-19T16:57:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.901+0000 7f6c50ed1640 1 -- 172.21.15.26:0/1675550113 shutdown_connections 2023-12-19T16:57:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.901+0000 7f6c50ed1640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c4c154220 0x7f6c4c14cff0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.901+0000 7f6c50ed1640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c4c0cec40 0x7f6c4c0b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.901+0000 7f6c50ed1640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6c2c065b80 0x7f6c2c068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.901+0000 7f6c50ed1640 1 --2- 172.21.15.26:0/1675550113 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c4c06b3d0 0x7f6c4c14bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.901+0000 7f6c50ed1640 1 -- 172.21.15.26:0/1675550113 >> 172.21.15.26:0/1675550113 conn(0x7f6c4c0658e0 msgr2=0x7f6c4c157c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.901+0000 7f6c50ed1640 1 -- 172.21.15.26:0/1675550113 shutdown_connections 2023-12-19T16:57:56.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:56.901+0000 7f6c50ed1640 1 -- 172.21.15.26:0/1675550113 wait complete. 2023-12-19T16:57:56.926 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T16:57:57.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.065+0000 7fdf1458f640 1 Processor -- start 2023-12-19T16:57:57.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf1458f640 1 -- start start 2023-12-19T16:57:57.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf1458f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdf0c15d250 0x7fdf0c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:57.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf1458f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf0c0b01b0 0x7fdf0c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:57.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf1458f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0b0b40 con 0x7fdf0c0ccf10 2023-12-19T16:57:57.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf1458f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c157730 con 0x7fdf0c0b01b0 2023-12-19T16:57:57.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf1458f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c0ac860 con 0x7fdf0c15d250 2023-12-19T16:57:57.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf12b2c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf0c0b01b0 0x7fdf0c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:57.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf12b2c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf0c0b01b0 0x7fdf0c0b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52320/0 (socket says 172.21.15.26:52320) 2023-12-19T16:57:57.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf12b2c640 1 -- 172.21.15.26:0/2280352786 learned_addr learned my addr 172.21.15.26:0/2280352786 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf11b2a640 1 --2- 172.21.15.26:0/2280352786 >> v2:172.21.15.160:3300/0 conn(0x7fdf0c15d250 0x7fdf0c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf12b2c640 1 -- 172.21.15.26:0/2280352786 >> v2:172.21.15.160:3300/0 conn(0x7fdf0c15d250 msgr2=0x7fdf0c15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf12b2c640 1 --2- 172.21.15.26:0/2280352786 >> v2:172.21.15.160:3300/0 conn(0x7fdf0c15d250 0x7fdf0c15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf12b2c640 1 -- 172.21.15.26:0/2280352786 >> v1:172.21.15.160:6789/0 conn(0x7fdf0c0ccf10 legacy=0x7fdf0c0ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:57.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.069+0000 7fdf12b2c640 1 -- 172.21.15.26:0/2280352786 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c15dd10 con 0x7fdf0c0b01b0 2023-12-19T16:57:57.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf12b2c640 1 --2- 172.21.15.26:0/2280352786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0c0b01b0 0x7fdf0c0b0600 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c06efe0 tx=0x7fdf0401f180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6152b15172a7af6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf11329640 1 -- 172.21.15.26:0/2280352786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040af400 con 0x7fdf0c0b01b0 2023-12-19T16:57:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf11329640 1 -- 172.21.15.26:0/2280352786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf040ab050 con 0x7fdf0c0b01b0 2023-12-19T16:57:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf11329640 1 -- 172.21.15.26:0/2280352786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040af780 con 0x7fdf0c0b01b0 2023-12-19T16:57:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf1458f640 1 -- 172.21.15.26:0/2280352786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0c0b01b0 msgr2=0x7fdf0c0b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf1458f640 1 --2- 172.21.15.26:0/2280352786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0c0b01b0 0x7fdf0c0b0600 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c06efe0 tx=0x7fdf0401f180 comp rx=0 tx=0).stop 2023-12-19T16:57:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf1458f640 1 -- 172.21.15.26:0/2280352786 shutdown_connections 2023-12-19T16:57:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf1458f640 1 --2- 172.21.15.26:0/2280352786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0c0b01b0 0x7fdf0c0b0600 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf1458f640 1 --2- 172.21.15.26:0/2280352786 >> v2:172.21.15.160:3300/0 conn(0x7fdf0c15d250 0x7fdf0c15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf1458f640 1 -- 172.21.15.26:0/2280352786 >> 172.21.15.26:0/2280352786 conn(0x7fdf0c0ce730 msgr2=0x7fdf0c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf1458f640 1 -- 172.21.15.26:0/2280352786 shutdown_connections 2023-12-19T16:57:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf1458f640 1 -- 172.21.15.26:0/2280352786 wait complete. 2023-12-19T16:57:57.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf1458f640 1 Processor -- start 2023-12-19T16:57:57.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.073+0000 7fdf1458f640 1 -- start start 2023-12-19T16:57:57.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf1458f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf0c0b01b0 0x7fdf0c148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:57.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf1458f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0c0ccf10 0x7fdf0c1f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:57.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf1458f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf0c15d250 0x7fdf0c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:57.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf1458f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1f36e0 con 0x7fdf0c0ccf10 2023-12-19T16:57:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf1458f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1f3120 con 0x7fdf0c0b01b0 2023-12-19T16:57:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf1458f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1f3400 con 0x7fdf0c15d250 2023-12-19T16:57:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf11b2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0c0ccf10 0x7fdf0c1f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf11b2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0c0ccf10 0x7fdf0c1f2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52328/0 (socket says 172.21.15.26:52328) 2023-12-19T16:57:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf11b2a640 1 -- 172.21.15.26:0/156038346 learned_addr learned my addr 172.21.15.26:0/156038346 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf1232b640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf0c0b01b0 0x7fdf0c148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf12b2c640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf0c15d250 0x7fdf0c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf11b2a640 1 -- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf0c15d250 msgr2=0x7fdf0c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf11b2a640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf0c15d250 0x7fdf0c15ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf11b2a640 1 -- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf0c0b01b0 msgr2=0x7fdf0c148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf11b2a640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf0c0b01b0 0x7fdf0c148cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf11b2a640 1 -- 172.21.15.26:0/156038346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf04009b30 con 0x7fdf0c0ccf10 2023-12-19T16:57:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf11b2a640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0c0ccf10 0x7fdf0c1f2740 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00d5d0 tx=0x7fdefc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:57.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf02ffd640 1 -- 172.21.15.26:0/156038346 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc004240 con 0x7fdf0c0ccf10 2023-12-19T16:57:57.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf02ffd640 1 -- 172.21.15.26:0/156038346 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc0043a0 con 0x7fdf0c0ccf10 2023-12-19T16:57:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf1458f640 1 -- 172.21.15.26:0/156038346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c15e490 con 0x7fdf0c0ccf10 2023-12-19T16:57:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf02ffd640 1 -- 172.21.15.26:0/156038346 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc003840 con 0x7fdf0c0ccf10 2023-12-19T16:57:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf1458f640 1 -- 172.21.15.26:0/156038346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c1f2c80 con 0x7fdf0c0ccf10 2023-12-19T16:57:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf1458f640 1 -- 172.21.15.26:0/156038346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c074270 con 0x7fdf0c0ccf10 2023-12-19T16:57:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.077+0000 7fdf02ffd640 1 -- 172.21.15.26:0/156038346 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fdefc0039a0 con 0x7fdf0c0ccf10 2023-12-19T16:57:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.081+0000 7fdf02ffd640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdef0065b80 0x7fdef0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.081+0000 7fdf02ffd640 1 -- 172.21.15.26:0/156038346 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fdefc083670 con 0x7fdf0c0ccf10 2023-12-19T16:57:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.081+0000 7fdf1232b640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdef0065b80 0x7fdef0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.081+0000 7fdf1232b640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdef0065b80 0x7fdef0068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0cbe60 tx=0x7fdf08007e40 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.085+0000 7fdf02ffd640 1 -- 172.21.15.26:0/156038346 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdefc050ce0 con 0x7fdf0c0ccf10 2023-12-19T16:57:57.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:57.281+0000 7fdf1458f640 1 -- 172.21.15.26:0/156038346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdf0c1616e0 con 0x7fdf0c0ccf10 2023-12-19T16:57:58.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.201+0000 7fdf02ffd640 1 -- 172.21.15.26:0/156038346 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdefc054300 con 0x7fdf0c0ccf10 2023-12-19T16:57:58.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.205+0000 7fdf1458f640 1 -- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdef0065b80 msgr2=0x7fdef0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:58.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.205+0000 7fdf1458f640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdef0065b80 0x7fdef0068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0cbe60 tx=0x7fdf08007e40 comp rx=0 tx=0).stop 2023-12-19T16:57:58.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.205+0000 7fdf1458f640 1 -- 172.21.15.26:0/156038346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0c0ccf10 msgr2=0x7fdf0c1f2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:58.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.205+0000 7fdf1458f640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0c0ccf10 0x7fdf0c1f2740 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00d5d0 tx=0x7fdefc00da90 comp rx=0 tx=0).stop 2023-12-19T16:57:58.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.205+0000 7fdf1458f640 1 -- 172.21.15.26:0/156038346 shutdown_connections 2023-12-19T16:57:58.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.205+0000 7fdf1458f640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdef0065b80 0x7fdef0068040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:58.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.205+0000 7fdf1458f640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf0c15d250 0x7fdf0c15ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:58.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.205+0000 7fdf1458f640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf0c0ccf10 0x7fdf0c1f2740 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:58.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.205+0000 7fdf1458f640 1 --2- 172.21.15.26:0/156038346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf0c0b01b0 0x7fdf0c148cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:58.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.205+0000 7fdf1458f640 1 -- 172.21.15.26:0/156038346 >> 172.21.15.26:0/156038346 conn(0x7fdf0c0ce730 msgr2=0x7fdf0c06f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:58.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.205+0000 7fdf1458f640 1 -- 172.21.15.26:0/156038346 shutdown_connections 2023-12-19T16:57:58.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.205+0000 7fdf1458f640 1 -- 172.21.15.26:0/156038346 wait complete. 2023-12-19T16:57:58.234 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T16:57:58.234 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T16:57:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f275e139640 1 Processor -- start 2023-12-19T16:57:58.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f275e139640 1 -- start start 2023-12-19T16:57:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f275e139640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f27580b08c0 0x7f275815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f275e139640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27580ac860 con 0x7f27580ccf10 2023-12-19T16:57:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f275e139640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275815dd10 con 0x7f275815d250 2023-12-19T16:57:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f275e139640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2758157730 con 0x7f27580b08c0 2023-12-19T16:57:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f27577fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f27580ccf10 0x7f27580ad2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52518/0 (socket says 172.21.15.26:52518) 2023-12-19T16:57:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f27577fe640 1 -- 172.21.15.26:0/2903419882 learned_addr learned my addr 172.21.15.26:0/2903419882 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:58.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f2757fff640 1 --2- 172.21.15.26:0/2903419882 >> v2:172.21.15.26:3300/0 conn(0x7f27580b08c0 0x7f275815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f27567fc640 1 -- 172.21.15.26:0/2903419882 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1272980484 0 0) 0x7f27580ac860 con 0x7f27580ccf10 2023-12-19T16:57:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f27567fc640 1 -- 172.21.15.26:0/2903419882 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2740003530 con 0x7f27580ccf10 2023-12-19T16:57:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f2757fff640 1 -- 172.21.15.26:0/2903419882 >> v1:172.21.15.160:6789/0 conn(0x7f275815d250 legacy=0x7f27580b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f2757fff640 1 -- 172.21.15.26:0/2903419882 >> v1:172.21.15.26:6789/0 conn(0x7f27580ccf10 legacy=0x7f27580ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:58.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.381+0000 7f2757fff640 1 -- 172.21.15.26:0/2903419882 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275815dff0 con 0x7f27580b08c0 2023-12-19T16:57:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.385+0000 7f2757fff640 1 --2- 172.21.15.26:0/2903419882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27580b08c0 0x7f275815d700 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2750009d90 tx=0x7f275001f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a2a843f120bfbc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.385+0000 7f27567fc640 1 -- 172.21.15.26:0/2903419882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27500af600 con 0x7f27580b08c0 2023-12-19T16:57:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.385+0000 7f27567fc640 1 -- 172.21.15.26:0/2903419882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27500ab050 con 0x7f27580b08c0 2023-12-19T16:57:58.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.385+0000 7f27567fc640 1 -- 172.21.15.26:0/2903419882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27500af980 con 0x7f27580b08c0 2023-12-19T16:57:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.385+0000 7f275e139640 1 -- 172.21.15.26:0/2903419882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27580b08c0 msgr2=0x7f275815d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.385+0000 7f275e139640 1 --2- 172.21.15.26:0/2903419882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27580b08c0 0x7f275815d700 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2750009d90 tx=0x7f275001f130 comp rx=0 tx=0).stop 2023-12-19T16:57:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.385+0000 7f275e139640 1 -- 172.21.15.26:0/2903419882 shutdown_connections 2023-12-19T16:57:58.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.385+0000 7f275e139640 1 --2- 172.21.15.26:0/2903419882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27580b08c0 0x7f275815d700 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:58.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.385+0000 7f275e139640 1 -- 172.21.15.26:0/2903419882 >> 172.21.15.26:0/2903419882 conn(0x7f27580ce730 msgr2=0x7f27580ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:58.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.385+0000 7f275e139640 1 -- 172.21.15.26:0/2903419882 shutdown_connections 2023-12-19T16:57:58.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.385+0000 7f275e139640 1 -- 172.21.15.26:0/2903419882 wait complete. 2023-12-19T16:57:58.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f275e139640 1 Processor -- start 2023-12-19T16:57:58.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f275e139640 1 -- start start 2023-12-19T16:57:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f275e139640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27580b08c0 0x7f27581454b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:58.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f275e139640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f27580ccf10 0x7f2758148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f275e139640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f275815d250 0x7f275806f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f275e139640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2758148b60 con 0x7f275815d250 2023-12-19T16:57:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f275e139640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27580b10e0 con 0x7f27580b08c0 2023-12-19T16:57:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f275e139640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27580b1380 con 0x7f27580ccf10 2023-12-19T16:57:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f2757fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f275815d250 0x7f275806f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f2757fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f275815d250 0x7f275806f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52352/0 (socket says 172.21.15.26:52352) 2023-12-19T16:57:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f2757fff640 1 -- 172.21.15.26:0/965588960 learned_addr learned my addr 172.21.15.26:0/965588960 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f27577fe640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27580b08c0 0x7f27581454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f2756ffd640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f27580ccf10 0x7f2758148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f2757fff640 1 -- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f27580ccf10 msgr2=0x7f2758148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:58.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f2757fff640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f27580ccf10 0x7f2758148620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f2757fff640 1 -- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27580b08c0 msgr2=0x7f27581454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f2757fff640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27580b08c0 0x7f27581454b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f2757fff640 1 -- 172.21.15.26:0/965588960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2750009a70 con 0x7f275815d250 2023-12-19T16:57:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f2757fff640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f275815d250 0x7f275806f730 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f27500b2040 tx=0x7f275001bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f273ffff640 1 -- 172.21.15.26:0/965588960 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750009450 con 0x7f275815d250 2023-12-19T16:57:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f273ffff640 1 -- 172.21.15.26:0/965588960 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27500b9410 con 0x7f275815d250 2023-12-19T16:57:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f273ffff640 1 -- 172.21.15.26:0/965588960 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27500b97c0 con 0x7f275815d250 2023-12-19T16:57:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f275e139640 1 -- 172.21.15.26:0/965588960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27580b1660 con 0x7f275815d250 2023-12-19T16:57:58.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.389+0000 7f275e139640 1 -- 172.21.15.26:0/965588960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27580b1c20 con 0x7f275815d250 2023-12-19T16:57:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.393+0000 7f275e139640 1 -- 172.21.15.26:0/965588960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27580aca90 con 0x7f275815d250 2023-12-19T16:57:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.393+0000 7f273ffff640 1 -- 172.21.15.26:0/965588960 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f27500b6050 con 0x7f275815d250 2023-12-19T16:57:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.393+0000 7f273ffff640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2744065bb0 0x7f2744068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:58.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.393+0000 7f273ffff640 1 -- 172.21.15.26:0/965588960 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f2750121b50 con 0x7f275815d250 2023-12-19T16:57:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.393+0000 7f27577fe640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2744065bb0 0x7f2744068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:58.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.393+0000 7f27577fe640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2744065bb0 0x7f2744068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f27480011f0 tx=0x7f2748009040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:58.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.397+0000 7f273ffff640 1 -- 172.21.15.26:0/965588960 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f27580aca90 con 0x7f275815d250 2023-12-19T16:57:58.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:58.589+0000 7f275e139640 1 -- 172.21.15.26:0/965588960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f27580aca90 con 0x7f275815d250 2023-12-19T16:57:59.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.205+0000 7f273ffff640 1 -- 172.21.15.26:0/965588960 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7f27580aca90 con 0x7f275815d250 2023-12-19T16:57:59.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.209+0000 7f275e139640 1 -- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2744065bb0 msgr2=0x7f2744068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:59.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.209+0000 7f275e139640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2744065bb0 0x7f2744068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f27480011f0 tx=0x7f2748009040 comp rx=0 tx=0).stop 2023-12-19T16:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.209+0000 7f275e139640 1 -- 172.21.15.26:0/965588960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f275815d250 msgr2=0x7f275806f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.209+0000 7f275e139640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f275815d250 0x7f275806f730 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f27500b2040 tx=0x7f275001bed0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.209+0000 7f275e139640 1 -- 172.21.15.26:0/965588960 shutdown_connections 2023-12-19T16:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.209+0000 7f275e139640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f275815d250 0x7f275806f730 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.209+0000 7f275e139640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2744065bb0 0x7f2744068070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.209+0000 7f275e139640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f27580ccf10 0x7f2758148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.209+0000 7f275e139640 1 --2- 172.21.15.26:0/965588960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27580b08c0 0x7f27581454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.209+0000 7f275e139640 1 -- 172.21.15.26:0/965588960 >> 172.21.15.26:0/965588960 conn(0x7f27580ce730 msgr2=0x7f27580ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.209+0000 7f275e139640 1 -- 172.21.15.26:0/965588960 shutdown_connections 2023-12-19T16:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.209+0000 7f275e139640 1 -- 172.21.15.26:0/965588960 wait complete. 2023-12-19T16:57:59.234 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T16:57:59.234 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:57:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887f59e640 1 Processor -- start 2023-12-19T16:57:59.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887f59e640 1 -- start start 2023-12-19T16:57:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887f59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88800b2030 0x7f8880071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887f59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88800affe0 0x7f88800c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887f59e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f888015e7c0 con 0x7f88800b3960 2023-12-19T16:57:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887f59e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8880072360 con 0x7f88800affe0 2023-12-19T16:57:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887f59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88800b04c0 con 0x7f88800b2030 2023-12-19T16:57:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887dd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88800affe0 0x7f88800c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887dd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88800affe0 0x7f88800c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52360/0 (socket says 172.21.15.26:52360) 2023-12-19T16:57:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887dd9b640 1 -- 172.21.15.26:0/2807191461 learned_addr learned my addr 172.21.15.26:0/2807191461 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:59.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887e59c640 1 --2- 172.21.15.26:0/2807191461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88800b2030 0x7f8880071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887dd9b640 1 -- 172.21.15.26:0/2807191461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88800b2030 msgr2=0x7f8880071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887dd9b640 1 --2- 172.21.15.26:0/2807191461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88800b2030 0x7f8880071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887dd9b640 1 -- 172.21.15.26:0/2807191461 >> v1:172.21.15.160:6789/0 conn(0x7f88800b3960 legacy=0x7f88800b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887dd9b640 1 -- 172.21.15.26:0/2807191461 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88800b07a0 con 0x7f88800affe0 2023-12-19T16:57:59.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.377+0000 7f887e59c640 1 --2- 172.21.15.26:0/2807191461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88800b2030 0x7f8880071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:57:59.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.381+0000 7f887dd9b640 1 --2- 172.21.15.26:0/2807191461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88800affe0 0x7f88800c9090 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8874009870 tx=0x7f887401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc552b76a5df95d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:59.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.381+0000 7f887d59a640 1 -- 172.21.15.26:0/2807191461 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740aa670 con 0x7f88800affe0 2023-12-19T16:57:59.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.381+0000 7f887d59a640 1 -- 172.21.15.26:0/2807191461 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88740aa7d0 con 0x7f88800affe0 2023-12-19T16:57:59.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.381+0000 7f887d59a640 1 -- 172.21.15.26:0/2807191461 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740aab60 con 0x7f88800affe0 2023-12-19T16:57:59.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.381+0000 7f887f59e640 1 -- 172.21.15.26:0/2807191461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88800affe0 msgr2=0x7f88800c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:59.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.381+0000 7f887f59e640 1 --2- 172.21.15.26:0/2807191461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88800affe0 0x7f88800c9090 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8874009870 tx=0x7f887401c510 comp rx=0 tx=0).stop 2023-12-19T16:57:59.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.381+0000 7f887f59e640 1 -- 172.21.15.26:0/2807191461 shutdown_connections 2023-12-19T16:57:59.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.381+0000 7f887f59e640 1 --2- 172.21.15.26:0/2807191461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88800affe0 0x7f88800c9090 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.381+0000 7f887f59e640 1 --2- 172.21.15.26:0/2807191461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88800b2030 0x7f8880071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.381+0000 7f887f59e640 1 -- 172.21.15.26:0/2807191461 >> 172.21.15.26:0/2807191461 conn(0x7f888006b0f0 msgr2=0x7f888006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:59.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.381+0000 7f887f59e640 1 -- 172.21.15.26:0/2807191461 shutdown_connections 2023-12-19T16:57:59.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.381+0000 7f887f59e640 1 -- 172.21.15.26:0/2807191461 wait complete. 2023-12-19T16:57:59.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.381+0000 7f887f59e640 1 Processor -- start 2023-12-19T16:57:59.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887f59e640 1 -- start start 2023-12-19T16:57:59.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887f59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88800affe0 0x7f888006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:59.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887f59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88800b2030 0x7f8880155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:59.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88800b3960 0x7f88800cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:59.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887f59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88801f62e0 con 0x7f88800b3960 2023-12-19T16:57:59.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887f59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8880156010 con 0x7f88800affe0 2023-12-19T16:57:59.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887f59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88801f6000 con 0x7f88800b2030 2023-12-19T16:57:59.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88800b3960 0x7f88800cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:59.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887e59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88800affe0 0x7f888006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:59.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887dd9b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88800b2030 0x7f8880155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:59.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88800b3960 0x7f88800cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52362/0 (socket says 172.21.15.26:52362) 2023-12-19T16:57:59.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887ed9d640 1 -- 172.21.15.26:0/2330524609 learned_addr learned my addr 172.21.15.26:0/2330524609 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:59.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887e59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88800affe0 0x7f888006fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:41186/0 (socket says 172.21.15.26:41186) 2023-12-19T16:57:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887ed9d640 1 -- 172.21.15.26:0/2330524609 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88800b2030 msgr2=0x7f8880155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887ed9d640 1 --2- 172.21.15.26:0/2330524609 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88800b2030 0x7f8880155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887ed9d640 1 -- 172.21.15.26:0/2330524609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88800affe0 msgr2=0x7f888006fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887ed9d640 1 --2- 172.21.15.26:0/2330524609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88800affe0 0x7f888006fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887ed9d640 1 -- 172.21.15.26:0/2330524609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88740094d0 con 0x7f88800b3960 2023-12-19T16:57:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887ed9d640 1 --2- 172.21.15.26:0/2330524609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88800b3960 0x7f88800cc0a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f887800c770 tx=0x7f887800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:59.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f886effd640 1 -- 172.21.15.26:0/2330524609 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8878003b80 con 0x7f88800b3960 2023-12-19T16:57:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f886effd640 1 -- 172.21.15.26:0/2330524609 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8878003ce0 con 0x7f88800b3960 2023-12-19T16:57:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f886effd640 1 -- 172.21.15.26:0/2330524609 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887800aa90 con 0x7f88800b3960 2023-12-19T16:57:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887f59e640 1 -- 172.21.15.26:0/2330524609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888006f0c0 con 0x7f88800b3960 2023-12-19T16:57:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887f59e640 1 -- 172.21.15.26:0/2330524609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888006f680 con 0x7f88800b3960 2023-12-19T16:57:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.385+0000 7f887f59e640 1 -- 172.21.15.26:0/2330524609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888006ef20 con 0x7f88800b3960 2023-12-19T16:57:59.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.389+0000 7f886effd640 1 -- 172.21.15.26:0/2330524609 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f887800abf0 con 0x7f88800b3960 2023-12-19T16:57:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.389+0000 7f886effd640 1 --2- 172.21.15.26:0/2330524609 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8860065b80 0x7f8860068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.389+0000 7f886effd640 1 -- 172.21.15.26:0/2330524609 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f8878083370 con 0x7f88800b3960 2023-12-19T16:57:59.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.389+0000 7f887e59c640 1 --2- 172.21.15.26:0/2330524609 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8860065b80 0x7f8860068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:59.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.389+0000 7f887e59c640 1 --2- 172.21.15.26:0/2330524609 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8860065b80 0x7f8860068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8870006fd0 tx=0x7f8870008040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:59.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.393+0000 7f886effd640 1 -- 172.21.15.26:0/2330524609 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f888006ef20 con 0x7f88800b3960 2023-12-19T16:57:59.560 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:57:59.564 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:57:59.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.581+0000 7f887f59e640 1 -- 172.21.15.26:0/2330524609 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f888006ef20 con 0x7f88800b3960 2023-12-19T16:57:59.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.585+0000 7f886effd640 1 -- 172.21.15.26:0/2330524609 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f888006ef20 con 0x7f88800b3960 2023-12-19T16:57:59.588 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:59.588 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/426534014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":426534014},{"type":"v1","addr":"172.21.15.160:6837","nonce":426534014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/1088174367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":1088174367},{"type":"v1","addr":"172.21.15.160:6836","nonce":1088174367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6981,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/343942943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":343942943},{"type":"v1","addr":"172.21.15.26:6834","nonce":343942943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:57:56.187532+0000","modified":"2023-12-19T16:57:59.206380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3421559564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3421559564},{"type":"v1","addr":"172.21.15.26:6835","nonce":3421559564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-19T16:57:59.589 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 41 2023-12-19T16:57:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.585+0000 7f887f59e640 1 -- 172.21.15.26:0/2330524609 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8860065b80 msgr2=0x7f8860068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:59.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.585+0000 7f887f59e640 1 --2- 172.21.15.26:0/2330524609 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8860065b80 0x7f8860068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8870006fd0 tx=0x7f8870008040 comp rx=0 tx=0).stop 2023-12-19T16:57:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.585+0000 7f887f59e640 1 -- 172.21.15.26:0/2330524609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88800b3960 msgr2=0x7f88800cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.585+0000 7f887f59e640 1 --2- 172.21.15.26:0/2330524609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88800b3960 0x7f88800cc0a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f887800c770 tx=0x7f887800cc30 comp rx=0 tx=0).stop 2023-12-19T16:57:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.585+0000 7f887f59e640 1 -- 172.21.15.26:0/2330524609 shutdown_connections 2023-12-19T16:57:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.585+0000 7f887f59e640 1 --2- 172.21.15.26:0/2330524609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88800b3960 0x7f88800cc0a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.585+0000 7f887f59e640 1 --2- 172.21.15.26:0/2330524609 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8860065b80 0x7f8860068040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.585+0000 7f887f59e640 1 --2- 172.21.15.26:0/2330524609 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88800b2030 0x7f8880155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.585+0000 7f887f59e640 1 --2- 172.21.15.26:0/2330524609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88800affe0 0x7f888006fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.585+0000 7f887f59e640 1 -- 172.21.15.26:0/2330524609 >> 172.21.15.26:0/2330524609 conn(0x7f888006b0f0 msgr2=0x7f88800ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.585+0000 7f887f59e640 1 -- 172.21.15.26:0/2330524609 shutdown_connections 2023-12-19T16:57:59.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.585+0000 7f887f59e640 1 -- 172.21.15.26:0/2330524609 wait complete. 2023-12-19T16:57:59.609 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:57:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d6a463640 1 Processor -- start 2023-12-19T16:57:59.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d6a463640 1 -- start start 2023-12-19T16:57:59.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d6a463640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d6415ab30 0x7f7d640cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:59.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d6a463640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d6415af10 0x7f7d640d0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d6a463640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d640cf4d0 con 0x7f7d640afd30 2023-12-19T16:57:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d6a463640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d640b6460 con 0x7f7d6415ab30 2023-12-19T16:57:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d6a463640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d640b69e0 con 0x7f7d6415af10 2023-12-19T16:57:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d68a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7d640afd30 0x7f7d640b24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52530/0 (socket says 172.21.15.26:52530) 2023-12-19T16:57:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d63fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d6415ab30 0x7f7d640cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d68a00640 1 -- 172.21.15.26:0/844754303 learned_addr learned my addr 172.21.15.26:0/844754303 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:57:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d5b7fe640 1 --2- 172.21.15.26:0/844754303 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d6415af10 0x7f7d640d0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d63fff640 1 -- 172.21.15.26:0/844754303 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d6415af10 msgr2=0x7f7d640d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:57:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d63fff640 1 --2- 172.21.15.26:0/844754303 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d6415af10 0x7f7d640d0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d63fff640 1 -- 172.21.15.26:0/844754303 >> v1:172.21.15.26:6789/0 conn(0x7f7d640afd30 legacy=0x7f7d640b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d637fe640 1 -- 172.21.15.26:0/844754303 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417136678 0 0) 0x7f7d640cf4d0 con 0x7f7d640afd30 2023-12-19T16:57:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.793+0000 7f7d63fff640 1 -- 172.21.15.26:0/844754303 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d640b6700 con 0x7f7d6415ab30 2023-12-19T16:57:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.797+0000 7f7d63fff640 1 --2- 172.21.15.26:0/844754303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d6415ab30 0x7f7d640cfdd0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7d540097c0 tx=0x7f7d5401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239d3c3ac146003d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.797+0000 7f7d637fe640 1 -- 172.21.15.26:0/844754303 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d540ac640 con 0x7f7d6415ab30 2023-12-19T16:57:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.797+0000 7f7d637fe640 1 -- 172.21.15.26:0/844754303 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d540ac7a0 con 0x7f7d6415ab30 2023-12-19T16:57:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.797+0000 7f7d637fe640 1 -- 172.21.15.26:0/844754303 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d540acb30 con 0x7f7d6415ab30 2023-12-19T16:57:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.797+0000 7f7d6a463640 1 -- 172.21.15.26:0/844754303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d6415ab30 msgr2=0x7f7d640cfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.797+0000 7f7d6a463640 1 --2- 172.21.15.26:0/844754303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d6415ab30 0x7f7d640cfdd0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7d540097c0 tx=0x7f7d5401c3c0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.797+0000 7f7d6a463640 1 -- 172.21.15.26:0/844754303 shutdown_connections 2023-12-19T16:57:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.797+0000 7f7d6a463640 1 --2- 172.21.15.26:0/844754303 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d6415af10 0x7f7d640d0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.797+0000 7f7d6a463640 1 --2- 172.21.15.26:0/844754303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d6415ab30 0x7f7d640cfdd0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.797+0000 7f7d6a463640 1 -- 172.21.15.26:0/844754303 >> 172.21.15.26:0/844754303 conn(0x7f7d6406d510 msgr2=0x7f7d6406d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.797+0000 7f7d6a463640 1 -- 172.21.15.26:0/844754303 shutdown_connections 2023-12-19T16:57:59.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d6a463640 1 -- 172.21.15.26:0/844754303 wait complete. 2023-12-19T16:57:59.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d6a463640 1 Processor -- start 2023-12-19T16:57:59.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d6a463640 1 -- start start 2023-12-19T16:57:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d6a463640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d640afd30 0x7f7d64152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d6a463640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d6415ab30 0x7f7d6415d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d6a463640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d6415af10 0x7f7d640c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:59.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d6a463640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d640c73d0 con 0x7f7d6415ab30 2023-12-19T16:57:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d6a463640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641560f0 con 0x7f7d640afd30 2023-12-19T16:57:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d6a463640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d64151750 con 0x7f7d6415af10 2023-12-19T16:57:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d5b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d6415ab30 0x7f7d6415d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d5b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d6415ab30 0x7f7d6415d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52370/0 (socket says 172.21.15.26:52370) 2023-12-19T16:57:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d5b7fe640 1 -- 172.21.15.26:0/2653899075 learned_addr learned my addr 172.21.15.26:0/2653899075 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:57:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d63fff640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d640afd30 0x7f7d64152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d68a00640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d6415af10 0x7f7d640c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d5b7fe640 1 -- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d6415af10 msgr2=0x7f7d640c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d5b7fe640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d6415af10 0x7f7d640c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d5b7fe640 1 -- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d640afd30 msgr2=0x7f7d64152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d5b7fe640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d640afd30 0x7f7d64152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d5b7fe640 1 -- 172.21.15.26:0/2653899075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d54009420 con 0x7f7d6415ab30 2023-12-19T16:57:59.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.801+0000 7f7d5b7fe640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d6415ab30 0x7f7d6415d960 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c007e40 tx=0x7f7d4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:59.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.805+0000 7f7d617fa640 1 -- 172.21.15.26:0/2653899075 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c014620 con 0x7f7d6415ab30 2023-12-19T16:57:59.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.805+0000 7f7d617fa640 1 -- 172.21.15.26:0/2653899075 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c00ee90 con 0x7f7d6415ab30 2023-12-19T16:57:59.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.805+0000 7f7d617fa640 1 -- 172.21.15.26:0/2653899075 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c0149c0 con 0x7f7d6415ab30 2023-12-19T16:57:59.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.805+0000 7f7d6a463640 1 -- 172.21.15.26:0/2653899075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d64151a30 con 0x7f7d6415ab30 2023-12-19T16:57:59.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.805+0000 7f7d6a463640 1 -- 172.21.15.26:0/2653899075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d640c6ff0 con 0x7f7d6415ab30 2023-12-19T16:57:59.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.805+0000 7f7d6a463640 1 -- 172.21.15.26:0/2653899075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d64074270 con 0x7f7d6415ab30 2023-12-19T16:57:59.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.805+0000 7f7d617fa640 1 -- 172.21.15.26:0/2653899075 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7d4c014b20 con 0x7f7d6415ab30 2023-12-19T16:57:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.805+0000 7f7d617fa640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7d44065b80 0x7f7d44068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:57:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.805+0000 7f7d617fa640 1 -- 172.21.15.26:0/2653899075 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f7d4c083fa0 con 0x7f7d6415ab30 2023-12-19T16:57:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.805+0000 7f7d63fff640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7d44065b80 0x7f7d44068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:57:59.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.809+0000 7f7d63fff640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7d44065b80 0x7f7d44068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7d540ae000 tx=0x7f7d540a7000 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:57:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.809+0000 7f7d617fa640 1 -- 172.21.15.26:0/2653899075 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7d4c0b50a0 con 0x7f7d6415ab30 2023-12-19T16:57:59.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.969+0000 7f7d6a463640 1 -- 172.21.15.26:0/2653899075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d640b1e70 con 0x7f7d6415ab30 2023-12-19T16:57:59.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.969+0000 7f7d617fa640 1 -- 172.21.15.26:0/2653899075 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17205 (secure 0 0 0) 0x7f7d4c051570 con 0x7f7d6415ab30 2023-12-19T16:57:59.975 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:57:59.975 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":93,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:57:57.190175+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:53.833233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:55.184912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:57:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.973+0000 7f7d6a463640 1 -- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7d44065b80 msgr2=0x7f7d44068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.973+0000 7f7d6a463640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7d44065b80 0x7f7d44068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7d540ae000 tx=0x7f7d540a7000 comp rx=0 tx=0).stop 2023-12-19T16:57:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.973+0000 7f7d6a463640 1 -- 172.21.15.26:0/2653899075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d6415ab30 msgr2=0x7f7d6415d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:57:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.973+0000 7f7d6a463640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d6415ab30 0x7f7d6415d960 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c007e40 tx=0x7f7d4c00e4b0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.973+0000 7f7d6a463640 1 -- 172.21.15.26:0/2653899075 shutdown_connections 2023-12-19T16:57:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.973+0000 7f7d6a463640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7d44065b80 0x7f7d44068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.973+0000 7f7d6a463640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d6415af10 0x7f7d640c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.973+0000 7f7d6a463640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d6415ab30 0x7f7d6415d960 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.973+0000 7f7d6a463640 1 --2- 172.21.15.26:0/2653899075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d640afd30 0x7f7d64152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:57:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.973+0000 7f7d6a463640 1 -- 172.21.15.26:0/2653899075 >> 172.21.15.26:0/2653899075 conn(0x7f7d6406d510 msgr2=0x7f7d6406f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:57:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.973+0000 7f7d6a463640 1 -- 172.21.15.26:0/2653899075 shutdown_connections 2023-12-19T16:57:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:57:59.973+0000 7f7d6a463640 1 -- 172.21.15.26:0/2653899075 wait complete. 2023-12-19T16:57:59.997 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T16:57:59.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:58:00.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897ffdb640 1 Processor -- start 2023-12-19T16:58:00.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897ffdb640 1 -- start start 2023-12-19T16:58:00.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897ffdb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f897815a7c0 0x7f897815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897ffdb640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f897806f9d0 0x7f89780ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897ffdb640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897815e2f0 con 0x7f89780c4070 2023-12-19T16:58:00.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897ffdb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89780c5b40 con 0x7f897815a7c0 2023-12-19T16:58:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897ffdb640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89780c5e20 con 0x7f897806f9d0 2023-12-19T16:58:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897dd77640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f897815a7c0 0x7f897815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897dd77640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f897815a7c0 0x7f897815ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52380/0 (socket says 172.21.15.26:52380) 2023-12-19T16:58:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897dd77640 1 -- 172.21.15.26:0/1113826441 learned_addr learned my addr 172.21.15.26:0/1113826441 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897d576640 1 --2- 172.21.15.26:0/1113826441 >> v2:172.21.15.160:3300/0 conn(0x7f897806f9d0 0x7f89780ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897dd77640 1 -- 172.21.15.26:0/1113826441 >> v2:172.21.15.160:3300/0 conn(0x7f897806f9d0 msgr2=0x7f89780ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897dd77640 1 --2- 172.21.15.26:0/1113826441 >> v2:172.21.15.160:3300/0 conn(0x7f897806f9d0 0x7f89780ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897dd77640 1 -- 172.21.15.26:0/1113826441 >> v1:172.21.15.160:6789/0 conn(0x7f89780c4070 legacy=0x7f89780cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:00.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.141+0000 7f897dd77640 1 -- 172.21.15.26:0/1113826441 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89780ce770 con 0x7f897815a7c0 2023-12-19T16:58:00.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.145+0000 7f897dd77640 1 --2- 172.21.15.26:0/1113826441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f897815a7c0 0x7f897815ddb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f897806ab70 tx=0x7f897401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638115a3d1649a12 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:00.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.145+0000 7f897cd75640 1 -- 172.21.15.26:0/1113826441 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89740ac600 con 0x7f897815a7c0 2023-12-19T16:58:00.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.145+0000 7f897cd75640 1 -- 172.21.15.26:0/1113826441 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89740a8050 con 0x7f897815a7c0 2023-12-19T16:58:00.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.145+0000 7f897cd75640 1 -- 172.21.15.26:0/1113826441 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89740ac980 con 0x7f897815a7c0 2023-12-19T16:58:00.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.145+0000 7f897ffdb640 1 -- 172.21.15.26:0/1113826441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f897815a7c0 msgr2=0x7f897815ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.145+0000 7f897ffdb640 1 --2- 172.21.15.26:0/1113826441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f897815a7c0 0x7f897815ddb0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f897806ab70 tx=0x7f897401c490 comp rx=0 tx=0).stop 2023-12-19T16:58:00.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.145+0000 7f897ffdb640 1 -- 172.21.15.26:0/1113826441 shutdown_connections 2023-12-19T16:58:00.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.145+0000 7f897ffdb640 1 --2- 172.21.15.26:0/1113826441 >> v2:172.21.15.160:3300/0 conn(0x7f897806f9d0 0x7f89780ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.145+0000 7f897ffdb640 1 --2- 172.21.15.26:0/1113826441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f897815a7c0 0x7f897815ddb0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.145+0000 7f897ffdb640 1 -- 172.21.15.26:0/1113826441 >> 172.21.15.26:0/1113826441 conn(0x7f897806d5d0 msgr2=0x7f8978069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:00.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.145+0000 7f897ffdb640 1 -- 172.21.15.26:0/1113826441 shutdown_connections 2023-12-19T16:58:00.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.145+0000 7f897ffdb640 1 -- 172.21.15.26:0/1113826441 wait complete. 2023-12-19T16:58:00.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897ffdb640 1 Processor -- start 2023-12-19T16:58:00.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897ffdb640 1 -- start start 2023-12-19T16:58:00.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897ffdb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897806f9d0 0x7f89780c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897ffdb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89780c4070 0x7f89781504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897ffdb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f897815a7c0 0x7f8978152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897ffdb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8978150f80 con 0x7f89780c4070 2023-12-19T16:58:00.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897ffdb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8978150a30 con 0x7f897806f9d0 2023-12-19T16:58:00.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897ffdb640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89781533d0 con 0x7f897815a7c0 2023-12-19T16:58:00.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897d576640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89780c4070 0x7f89781504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897d576640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89780c4070 0x7f89781504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52384/0 (socket says 172.21.15.26:52384) 2023-12-19T16:58:00.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897d576640 1 -- 172.21.15.26:0/1590689954 learned_addr learned my addr 172.21.15.26:0/1590689954 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:00.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897d576640 1 -- 172.21.15.26:0/1590689954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f897815a7c0 msgr2=0x7f8978152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897d576640 1 --2- 172.21.15.26:0/1590689954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f897815a7c0 0x7f8978152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897d576640 1 -- 172.21.15.26:0/1590689954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897806f9d0 msgr2=0x7f89780c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897d576640 1 --2- 172.21.15.26:0/1590689954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897806f9d0 0x7f89780c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897d576640 1 -- 172.21.15.26:0/1590689954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8974009480 con 0x7f89780c4070 2023-12-19T16:58:00.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897d576640 1 --2- 172.21.15.26:0/1590689954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89780c4070 0x7f89781504f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f896800d800 tx=0x7f896800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:00.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f896e7fc640 1 -- 172.21.15.26:0/1590689954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896800b6c0 con 0x7f89780c4070 2023-12-19T16:58:00.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f896e7fc640 1 -- 172.21.15.26:0/1590689954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896800b820 con 0x7f89780c4070 2023-12-19T16:58:00.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f896e7fc640 1 -- 172.21.15.26:0/1590689954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896800bb90 con 0x7f89780c4070 2023-12-19T16:58:00.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897ffdb640 1 -- 172.21.15.26:0/1590689954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89780c8650 con 0x7f89780c4070 2023-12-19T16:58:00.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897ffdb640 1 -- 172.21.15.26:0/1590689954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89780c8c10 con 0x7f89780c4070 2023-12-19T16:58:00.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.149+0000 7f897ffdb640 1 -- 172.21.15.26:0/1590689954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8978074270 con 0x7f89780c4070 2023-12-19T16:58:00.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.153+0000 7f896e7fc640 1 -- 172.21.15.26:0/1590689954 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f896800bcf0 con 0x7f89780c4070 2023-12-19T16:58:00.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.153+0000 7f896e7fc640 1 --2- 172.21.15.26:0/1590689954 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f895c065b80 0x7f895c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.153+0000 7f896e7fc640 1 -- 172.21.15.26:0/1590689954 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f8968083b90 con 0x7f89780c4070 2023-12-19T16:58:00.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.153+0000 7f897dd77640 1 --2- 172.21.15.26:0/1590689954 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f895c065b80 0x7f895c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.153+0000 7f897dd77640 1 --2- 172.21.15.26:0/1590689954 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f895c065b80 0x7f895c068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89740ae000 tx=0x7f8974004740 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:00.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.153+0000 7f896e7fc640 1 -- 172.21.15.26:0/1590689954 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8968051150 con 0x7f89780c4070 2023-12-19T16:58:00.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.313+0000 7f897ffdb640 1 -- 172.21.15.26:0/1590689954 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89780c57f0 con 0x7f895c065b80 2023-12-19T16:58:00.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.325+0000 7f896e7fc640 1 -- 172.21.15.26:0/1590689954 <== mgr.6490 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289050 (secure 0 0 0) 0x7f8974004160 con 0x7f895c065b80 2023-12-19T16:58:00.330 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:58:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.325+0000 7f897ffdb640 1 -- 172.21.15.26:0/1590689954 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f895c065b80 msgr2=0x7f895c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.325+0000 7f897ffdb640 1 --2- 172.21.15.26:0/1590689954 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f895c065b80 0x7f895c068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89740ae000 tx=0x7f8974004740 comp rx=0 tx=0).stop 2023-12-19T16:58:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.325+0000 7f897ffdb640 1 -- 172.21.15.26:0/1590689954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89780c4070 msgr2=0x7f89781504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.325+0000 7f897ffdb640 1 --2- 172.21.15.26:0/1590689954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89780c4070 0x7f89781504f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f896800d800 tx=0x7f896800dcc0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.325+0000 7f897ffdb640 1 -- 172.21.15.26:0/1590689954 shutdown_connections 2023-12-19T16:58:00.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.325+0000 7f897ffdb640 1 --2- 172.21.15.26:0/1590689954 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f895c065b80 0x7f895c068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.325+0000 7f897ffdb640 1 --2- 172.21.15.26:0/1590689954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f897815a7c0 0x7f8978152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.325+0000 7f897ffdb640 1 --2- 172.21.15.26:0/1590689954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89780c4070 0x7f89781504f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.325+0000 7f897ffdb640 1 --2- 172.21.15.26:0/1590689954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f897806f9d0 0x7f89780c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.325+0000 7f897ffdb640 1 -- 172.21.15.26:0/1590689954 >> 172.21.15.26:0/1590689954 conn(0x7f897806d5d0 msgr2=0x7f89780b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:00.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.325+0000 7f897ffdb640 1 -- 172.21.15.26:0/1590689954 shutdown_connections 2023-12-19T16:58:00.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.325+0000 7f897ffdb640 1 -- 172.21.15.26:0/1590689954 wait complete. 2023-12-19T16:58:00.350 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-19T16:57:59.274465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":674,"num_read_kb":1321,"num_write":319,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":310792,"kb_used_data":9928,"kb_used_omap":0,"kb_used_meta":300800,"kb_avail":754663928,"statfs":{"total":773094113280,"available":772775862272,"internally_reserved":0,"allocated":10166272,"data_stored":5596324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308019200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.497523"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202865+0000","last_change":"2023-12-19T16:57:56.204460+0000","last_active":"2023-12-19T16:57:57.202865+0000","last_peered":"2023-12-19T16:57:57.202865+0000","last_clean":"2023-12-19T16:57:57.202865+0000","last_became_active":"2023-12-19T16:57:56.204195+0000","last_became_peered":"2023-12-19T16:57:56.204195+0000","last_unstale":"2023-12-19T16:57:57.202865+0000","last_undegraded":"2023-12-19T16:57:57.202865+0000","last_fullsized":"2023-12-19T16:57:57.202865+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:06.873619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228808+0000","last_change":"2023-12-19T16:57:56.223513+0000","last_active":"2023-12-19T16:57:56.228808+0000","last_peered":"2023-12-19T16:57:56.228808+0000","last_clean":"2023-12-19T16:57:56.228808+0000","last_became_active":"2023-12-19T16:57:56.220518+0000","last_became_peered":"2023-12-19T16:57:56.220518+0000","last_unstale":"2023-12-19T16:57:56.228808+0000","last_undegraded":"2023-12-19T16:57:56.228808+0000","last_fullsized":"2023-12-19T16:57:56.228808+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:27:43.061756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200601+0000","last_change":"2023-12-19T16:57:56.210867+0000","last_active":"2023-12-19T16:57:57.200601+0000","last_peered":"2023-12-19T16:57:57.200601+0000","last_clean":"2023-12-19T16:57:57.200601+0000","last_became_active":"2023-12-19T16:57:56.210587+0000","last_became_peered":"2023-12-19T16:57:56.210587+0000","last_unstale":"2023-12-19T16:57:57.200601+0000","last_undegraded":"2023-12-19T16:57:57.200601+0000","last_fullsized":"2023-12-19T16:57:57.200601+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:03.424934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.224928+0000","last_change":"2023-12-19T16:57:56.221506+0000","last_active":"2023-12-19T16:57:56.224928+0000","last_peered":"2023-12-19T16:57:56.224928+0000","last_clean":"2023-12-19T16:57:56.224928+0000","last_became_active":"2023-12-19T16:57:56.220858+0000","last_became_peered":"2023-12-19T16:57:56.220858+0000","last_unstale":"2023-12-19T16:57:56.224928+0000","last_undegraded":"2023-12-19T16:57:56.224928+0000","last_fullsized":"2023-12-19T16:57:56.224928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:26.922307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.208692+0000","last_change":"2023-12-19T16:57:56.202219+0000","last_active":"2023-12-19T16:57:56.208692+0000","last_peered":"2023-12-19T16:57:56.208692+0000","last_clean":"2023-12-19T16:57:56.208692+0000","last_became_active":"2023-12-19T16:57:56.201406+0000","last_became_peered":"2023-12-19T16:57:56.201406+0000","last_unstale":"2023-12-19T16:57:56.208692+0000","last_undegraded":"2023-12-19T16:57:56.208692+0000","last_fullsized":"2023-12-19T16:57:56.208692+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:27:07.782916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.227209+0000","last_change":"2023-12-19T16:57:56.221312+0000","last_active":"2023-12-19T16:57:56.227209+0000","last_peered":"2023-12-19T16:57:56.227209+0000","last_clean":"2023-12-19T16:57:56.227209+0000","last_became_active":"2023-12-19T16:57:56.220311+0000","last_became_peered":"2023-12-19T16:57:56.220311+0000","last_unstale":"2023-12-19T16:57:56.227209+0000","last_undegraded":"2023-12-19T16:57:56.227209+0000","last_fullsized":"2023-12-19T16:57:56.227209+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:25.807631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.218556+0000","last_change":"2023-12-19T16:57:56.211243+0000","last_active":"2023-12-19T16:57:56.218556+0000","last_peered":"2023-12-19T16:57:56.218556+0000","last_clean":"2023-12-19T16:57:56.218556+0000","last_became_active":"2023-12-19T16:57:56.210911+0000","last_became_peered":"2023-12-19T16:57:56.210911+0000","last_unstale":"2023-12-19T16:57:56.218556+0000","last_undegraded":"2023-12-19T16:57:56.218556+0000","last_fullsized":"2023-12-19T16:57:56.218556+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:48.513902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.222565+0000","last_change":"2023-12-19T16:57:56.218897+0000","last_active":"2023-12-19T16:57:56.222565+0000","last_peered":"2023-12-19T16:57:56.222565+0000","last_clean":"2023-12-19T16:57:56.222565+0000","last_became_active":"2023-12-19T16:57:56.218389+0000","last_became_peered":"2023-12-19T16:57:56.218389+0000","last_unstale":"2023-12-19T16:57:56.222565+0000","last_undegraded":"2023-12-19T16:57:56.222565+0000","last_fullsized":"2023-12-19T16:57:56.222565+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:38:34.153845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.218356+0000","last_change":"2023-12-19T16:57:56.212841+0000","last_active":"2023-12-19T16:57:56.218356+0000","last_peered":"2023-12-19T16:57:56.218356+0000","last_clean":"2023-12-19T16:57:56.218356+0000","last_became_active":"2023-12-19T16:57:56.212288+0000","last_became_peered":"2023-12-19T16:57:56.212288+0000","last_unstale":"2023-12-19T16:57:56.218356+0000","last_undegraded":"2023-12-19T16:57:56.218356+0000","last_fullsized":"2023-12-19T16:57:56.218356+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:06:31.338071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202302+0000","last_change":"2023-12-19T16:57:56.207664+0000","last_active":"2023-12-19T16:57:57.202302+0000","last_peered":"2023-12-19T16:57:57.202302+0000","last_clean":"2023-12-19T16:57:57.202302+0000","last_became_active":"2023-12-19T16:57:56.207395+0000","last_became_peered":"2023-12-19T16:57:56.207395+0000","last_unstale":"2023-12-19T16:57:57.202302+0000","last_undegraded":"2023-12-19T16:57:57.202302+0000","last_fullsized":"2023-12-19T16:57:57.202302+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:14.908673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202783+0000","last_change":"2023-12-19T16:57:56.209088+0000","last_active":"2023-12-19T16:57:57.202783+0000","last_peered":"2023-12-19T16:57:57.202783+0000","last_clean":"2023-12-19T16:57:57.202783+0000","last_became_active":"2023-12-19T16:57:56.208815+0000","last_became_peered":"2023-12-19T16:57:56.208815+0000","last_unstale":"2023-12-19T16:57:57.202783+0000","last_undegraded":"2023-12-19T16:57:57.202783+0000","last_fullsized":"2023-12-19T16:57:57.202783+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:31.374433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.222481+0000","last_change":"2023-12-19T16:57:56.210809+0000","last_active":"2023-12-19T16:57:56.222481+0000","last_peered":"2023-12-19T16:57:56.222481+0000","last_clean":"2023-12-19T16:57:56.222481+0000","last_became_active":"2023-12-19T16:57:56.210560+0000","last_became_peered":"2023-12-19T16:57:56.210560+0000","last_unstale":"2023-12-19T16:57:56.222481+0000","last_undegraded":"2023-12-19T16:57:56.222481+0000","last_fullsized":"2023-12-19T16:57:56.222481+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:30:03.697769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.219804+0000","last_change":"2023-12-19T16:57:56.219090+0000","last_active":"2023-12-19T16:57:56.219804+0000","last_peered":"2023-12-19T16:57:56.219804+0000","last_clean":"2023-12-19T16:57:56.219804+0000","last_became_active":"2023-12-19T16:57:56.218758+0000","last_became_peered":"2023-12-19T16:57:56.218758+0000","last_unstale":"2023-12-19T16:57:56.219804+0000","last_undegraded":"2023-12-19T16:57:56.219804+0000","last_fullsized":"2023-12-19T16:57:56.219804+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:00:18.660813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.218061+0000","last_change":"2023-12-19T16:57:56.212881+0000","last_active":"2023-12-19T16:57:56.218061+0000","last_peered":"2023-12-19T16:57:56.218061+0000","last_clean":"2023-12-19T16:57:56.218061+0000","last_became_active":"2023-12-19T16:57:56.212638+0000","last_became_peered":"2023-12-19T16:57:56.212638+0000","last_unstale":"2023-12-19T16:57:56.218061+0000","last_undegraded":"2023-12-19T16:57:56.218061+0000","last_fullsized":"2023-12-19T16:57:56.218061+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:47:04.684289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.217194+0000","last_change":"2023-12-19T16:57:56.216155+0000","last_active":"2023-12-19T16:57:56.217194+0000","last_peered":"2023-12-19T16:57:56.217194+0000","last_clean":"2023-12-19T16:57:56.217194+0000","last_became_active":"2023-12-19T16:57:56.215817+0000","last_became_peered":"2023-12-19T16:57:56.215817+0000","last_unstale":"2023-12-19T16:57:56.217194+0000","last_undegraded":"2023-12-19T16:57:56.217194+0000","last_fullsized":"2023-12-19T16:57:56.217194+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:41:41.101912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201253+0000","last_change":"2023-12-19T16:57:56.213826+0000","last_active":"2023-12-19T16:57:57.201253+0000","last_peered":"2023-12-19T16:57:57.201253+0000","last_clean":"2023-12-19T16:57:57.201253+0000","last_became_active":"2023-12-19T16:57:56.213551+0000","last_became_peered":"2023-12-19T16:57:56.213551+0000","last_unstale":"2023-12-19T16:57:57.201253+0000","last_undegraded":"2023-12-19T16:57:57.201253+0000","last_fullsized":"2023-12-19T16:57:57.201253+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:26:03.234019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.213581+0000","last_change":"2023-12-19T16:57:56.206859+0000","last_active":"2023-12-19T16:57:56.213581+0000","last_peered":"2023-12-19T16:57:56.213581+0000","last_clean":"2023-12-19T16:57:56.213581+0000","last_became_active":"2023-12-19T16:57:56.206058+0000","last_became_peered":"2023-12-19T16:57:56.206058+0000","last_unstale":"2023-12-19T16:57:56.213581+0000","last_undegraded":"2023-12-19T16:57:56.213581+0000","last_fullsized":"2023-12-19T16:57:56.213581+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:19:26.708980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.224809+0000","last_change":"2023-12-19T16:57:56.220964+0000","last_active":"2023-12-19T16:57:56.224809+0000","last_peered":"2023-12-19T16:57:56.224809+0000","last_clean":"2023-12-19T16:57:56.224809+0000","last_became_active":"2023-12-19T16:57:56.220667+0000","last_became_peered":"2023-12-19T16:57:56.220667+0000","last_unstale":"2023-12-19T16:57:56.224809+0000","last_undegraded":"2023-12-19T16:57:56.224809+0000","last_fullsized":"2023-12-19T16:57:56.224809+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:26:22.970957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.230267+0000","last_change":"2023-12-19T16:57:56.228330+0000","last_active":"2023-12-19T16:57:56.230267+0000","last_peered":"2023-12-19T16:57:56.230267+0000","last_clean":"2023-12-19T16:57:56.230267+0000","last_became_active":"2023-12-19T16:57:56.228047+0000","last_became_peered":"2023-12-19T16:57:56.228047+0000","last_unstale":"2023-12-19T16:57:56.230267+0000","last_undegraded":"2023-12-19T16:57:56.230267+0000","last_fullsized":"2023-12-19T16:57:56.230267+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:41.200331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.215291+0000","last_change":"2023-12-19T16:57:56.213153+0000","last_active":"2023-12-19T16:57:56.215291+0000","last_peered":"2023-12-19T16:57:56.215291+0000","last_clean":"2023-12-19T16:57:56.215291+0000","last_became_active":"2023-12-19T16:57:56.212862+0000","last_became_peered":"2023-12-19T16:57:56.212862+0000","last_unstale":"2023-12-19T16:57:56.215291+0000","last_undegraded":"2023-12-19T16:57:56.215291+0000","last_fullsized":"2023-12-19T16:57:56.215291+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:48:00.681274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201690+0000","last_change":"2023-12-19T16:57:56.205935+0000","last_active":"2023-12-19T16:57:57.201690+0000","last_peered":"2023-12-19T16:57:57.201690+0000","last_clean":"2023-12-19T16:57:57.201690+0000","last_became_active":"2023-12-19T16:57:56.205656+0000","last_became_peered":"2023-12-19T16:57:56.205656+0000","last_unstale":"2023-12-19T16:57:57.201690+0000","last_undegraded":"2023-12-19T16:57:57.201690+0000","last_fullsized":"2023-12-19T16:57:57.201690+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:38:34.153845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201835+0000","last_change":"2023-12-19T16:57:56.207535+0000","last_active":"2023-12-19T16:57:57.201835+0000","last_peered":"2023-12-19T16:57:57.201835+0000","last_clean":"2023-12-19T16:57:57.201835+0000","last_became_active":"2023-12-19T16:57:56.207163+0000","last_became_peered":"2023-12-19T16:57:56.207163+0000","last_unstale":"2023-12-19T16:57:57.201835+0000","last_undegraded":"2023-12-19T16:57:57.201835+0000","last_fullsized":"2023-12-19T16:57:57.201835+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:30:03.697769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201566+0000","last_change":"2023-12-19T16:57:56.214268+0000","last_active":"2023-12-19T16:57:57.201566+0000","last_peered":"2023-12-19T16:57:57.201566+0000","last_clean":"2023-12-19T16:57:57.201566+0000","last_became_active":"2023-12-19T16:57:56.213923+0000","last_became_peered":"2023-12-19T16:57:56.213923+0000","last_unstale":"2023-12-19T16:57:57.201566+0000","last_undegraded":"2023-12-19T16:57:57.201566+0000","last_fullsized":"2023-12-19T16:57:57.201566+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:19:42.700877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201948+0000","last_change":"2023-12-19T16:57:56.221702+0000","last_active":"2023-12-19T16:57:57.201948+0000","last_peered":"2023-12-19T16:57:57.201948+0000","last_clean":"2023-12-19T16:57:57.201948+0000","last_became_active":"2023-12-19T16:57:56.221405+0000","last_became_peered":"2023-12-19T16:57:56.221405+0000","last_unstale":"2023-12-19T16:57:57.201948+0000","last_undegraded":"2023-12-19T16:57:57.201948+0000","last_fullsized":"2023-12-19T16:57:57.201948+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:51:35.243235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202636+0000","last_change":"2023-12-19T16:57:56.200162+0000","last_active":"2023-12-19T16:57:57.202636+0000","last_peered":"2023-12-19T16:57:57.202636+0000","last_clean":"2023-12-19T16:57:57.202636+0000","last_became_active":"2023-12-19T16:57:56.199883+0000","last_became_peered":"2023-12-19T16:57:56.199883+0000","last_unstale":"2023-12-19T16:57:57.202636+0000","last_undegraded":"2023-12-19T16:57:57.202636+0000","last_fullsized":"2023-12-19T16:57:57.202636+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:22.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.226925+0000","last_change":"2023-12-19T16:57:56.222452+0000","last_active":"2023-12-19T16:57:56.226925+0000","last_peered":"2023-12-19T16:57:56.226925+0000","last_clean":"2023-12-19T16:57:56.226925+0000","last_became_active":"2023-12-19T16:57:56.222198+0000","last_became_peered":"2023-12-19T16:57:56.222198+0000","last_unstale":"2023-12-19T16:57:56.226925+0000","last_undegraded":"2023-12-19T16:57:56.226925+0000","last_fullsized":"2023-12-19T16:57:56.226925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:07.344797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200650+0000","last_change":"2023-12-19T16:57:56.215081+0000","last_active":"2023-12-19T16:57:57.200650+0000","last_peered":"2023-12-19T16:57:57.200650+0000","last_clean":"2023-12-19T16:57:57.200650+0000","last_became_active":"2023-12-19T16:57:56.214702+0000","last_became_peered":"2023-12-19T16:57:56.214702+0000","last_unstale":"2023-12-19T16:57:57.200650+0000","last_undegraded":"2023-12-19T16:57:57.200650+0000","last_fullsized":"2023-12-19T16:57:57.200650+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:28:28.939705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202007+0000","last_change":"2023-12-19T16:57:56.206053+0000","last_active":"2023-12-19T16:57:57.202007+0000","last_peered":"2023-12-19T16:57:57.202007+0000","last_clean":"2023-12-19T16:57:57.202007+0000","last_became_active":"2023-12-19T16:57:56.205032+0000","last_became_peered":"2023-12-19T16:57:56.205032+0000","last_unstale":"2023-12-19T16:57:57.202007+0000","last_undegraded":"2023-12-19T16:57:57.202007+0000","last_fullsized":"2023-12-19T16:57:57.202007+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:27:43.061756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200344+0000","last_change":"2023-12-19T16:57:56.215748+0000","last_active":"2023-12-19T16:57:57.200344+0000","last_peered":"2023-12-19T16:57:57.200344+0000","last_clean":"2023-12-19T16:57:57.200344+0000","last_became_active":"2023-12-19T16:57:56.215470+0000","last_became_peered":"2023-12-19T16:57:56.215470+0000","last_unstale":"2023-12-19T16:57:57.200344+0000","last_undegraded":"2023-12-19T16:57:57.200344+0000","last_fullsized":"2023-12-19T16:57:57.200344+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:47:46.403128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228186+0000","last_change":"2023-12-19T16:57:56.226106+0000","last_active":"2023-12-19T16:57:56.228186+0000","last_peered":"2023-12-19T16:57:56.228186+0000","last_clean":"2023-12-19T16:57:56.228186+0000","last_became_active":"2023-12-19T16:57:56.225838+0000","last_became_peered":"2023-12-19T16:57:56.225838+0000","last_unstale":"2023-12-19T16:57:56.228186+0000","last_undegraded":"2023-12-19T16:57:56.228186+0000","last_fullsized":"2023-12-19T16:57:56.228186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:40:53.136322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.226844+0000","last_change":"2023-12-19T16:57:56.218236+0000","last_active":"2023-12-19T16:57:56.226844+0000","last_peered":"2023-12-19T16:57:56.226844+0000","last_clean":"2023-12-19T16:57:56.226844+0000","last_became_active":"2023-12-19T16:57:56.217739+0000","last_became_peered":"2023-12-19T16:57:56.217739+0000","last_unstale":"2023-12-19T16:57:56.226844+0000","last_undegraded":"2023-12-19T16:57:56.226844+0000","last_fullsized":"2023-12-19T16:57:56.226844+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:41:25.952403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228757+0000","last_change":"2023-12-19T16:57:56.222326+0000","last_active":"2023-12-19T16:57:56.228757+0000","last_peered":"2023-12-19T16:57:56.228757+0000","last_clean":"2023-12-19T16:57:56.228757+0000","last_became_active":"2023-12-19T16:57:56.221814+0000","last_became_peered":"2023-12-19T16:57:56.221814+0000","last_unstale":"2023-12-19T16:57:56.228757+0000","last_undegraded":"2023-12-19T16:57:56.228757+0000","last_fullsized":"2023-12-19T16:57:56.228757+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:22.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202910+0000","last_change":"2023-12-19T16:57:56.222016+0000","last_active":"2023-12-19T16:57:57.202910+0000","last_peered":"2023-12-19T16:57:57.202910+0000","last_clean":"2023-12-19T16:57:57.202910+0000","last_became_active":"2023-12-19T16:57:56.221727+0000","last_became_peered":"2023-12-19T16:57:56.221727+0000","last_unstale":"2023-12-19T16:57:57.202910+0000","last_undegraded":"2023-12-19T16:57:57.202910+0000","last_fullsized":"2023-12-19T16:57:57.202910+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:14:01.070538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"92'4","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.203076+0000","last_change":"2023-12-19T16:57:55.812332+0000","last_active":"2023-12-19T16:57:57.203076+0000","last_peered":"2023-12-19T16:57:57.203076+0000","last_clean":"2023-12-19T16:57:57.203076+0000","last_became_active":"2023-12-19T16:57:55.812074+0000","last_became_peered":"2023-12-19T16:57:55.812074+0000","last_unstale":"2023-12-19T16:57:57.203076+0000","last_undegraded":"2023-12-19T16:57:57.203076+0000","last_fullsized":"2023-12-19T16:57:57.203076+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:47.501121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.212474+0000","last_change":"2023-12-19T16:57:56.206379+0000","last_active":"2023-12-19T16:57:56.212474+0000","last_peered":"2023-12-19T16:57:56.212474+0000","last_clean":"2023-12-19T16:57:56.212474+0000","last_became_active":"2023-12-19T16:57:56.205725+0000","last_became_peered":"2023-12-19T16:57:56.205725+0000","last_unstale":"2023-12-19T16:57:56.212474+0000","last_undegraded":"2023-12-19T16:57:56.212474+0000","last_fullsized":"2023-12-19T16:57:56.212474+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:49.316778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.222337+0000","last_change":"2023-12-19T16:57:55.808273+0000","last_active":"2023-12-19T16:57:56.222337+0000","last_peered":"2023-12-19T16:57:56.222337+0000","last_clean":"2023-12-19T16:57:56.222337+0000","last_became_active":"2023-12-19T16:57:55.807915+0000","last_became_peered":"2023-12-19T16:57:55.807915+0000","last_unstale":"2023-12-19T16:57:56.222337+0000","last_undegraded":"2023-12-19T16:57:56.222337+0000","last_fullsized":"2023-12-19T16:57:56.222337+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:13:41.250548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.229117+0000","last_change":"2023-12-19T16:57:56.225647+0000","last_active":"2023-12-19T16:57:56.229117+0000","last_peered":"2023-12-19T16:57:56.229117+0000","last_clean":"2023-12-19T16:57:56.229117+0000","last_became_active":"2023-12-19T16:57:56.225340+0000","last_became_peered":"2023-12-19T16:57:56.225340+0000","last_unstale":"2023-12-19T16:57:56.229117+0000","last_undegraded":"2023-12-19T16:57:56.229117+0000","last_fullsized":"2023-12-19T16:57:56.229117+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:26.922307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202073+0000","last_change":"2023-12-19T16:57:55.813517+0000","last_active":"2023-12-19T16:57:57.202073+0000","last_peered":"2023-12-19T16:57:57.202073+0000","last_clean":"2023-12-19T16:57:57.202073+0000","last_became_active":"2023-12-19T16:57:55.813174+0000","last_became_peered":"2023-12-19T16:57:55.813174+0000","last_unstale":"2023-12-19T16:57:57.202073+0000","last_undegraded":"2023-12-19T16:57:57.202073+0000","last_fullsized":"2023-12-19T16:57:57.202073+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:38.178292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202824+0000","last_change":"2023-12-19T16:57:56.204979+0000","last_active":"2023-12-19T16:57:57.202824+0000","last_peered":"2023-12-19T16:57:57.202824+0000","last_clean":"2023-12-19T16:57:57.202824+0000","last_became_active":"2023-12-19T16:57:56.204647+0000","last_became_peered":"2023-12-19T16:57:56.204647+0000","last_unstale":"2023-12-19T16:57:57.202824+0000","last_undegraded":"2023-12-19T16:57:57.202824+0000","last_fullsized":"2023-12-19T16:57:57.202824+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:47:28.124851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.229144+0000","last_change":"2023-12-19T16:57:55.810481+0000","last_active":"2023-12-19T16:57:56.229144+0000","last_peered":"2023-12-19T16:57:56.229144+0000","last_clean":"2023-12-19T16:57:56.229144+0000","last_became_active":"2023-12-19T16:57:55.810214+0000","last_became_peered":"2023-12-19T16:57:55.810214+0000","last_unstale":"2023-12-19T16:57:56.229144+0000","last_undegraded":"2023-12-19T16:57:56.229144+0000","last_fullsized":"2023-12-19T16:57:56.229144+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:38.178292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.218582+0000","last_change":"2023-12-19T16:57:56.211368+0000","last_active":"2023-12-19T16:57:56.218582+0000","last_peered":"2023-12-19T16:57:56.218582+0000","last_clean":"2023-12-19T16:57:56.218582+0000","last_became_active":"2023-12-19T16:57:56.211086+0000","last_became_peered":"2023-12-19T16:57:56.211086+0000","last_unstale":"2023-12-19T16:57:56.218582+0000","last_undegraded":"2023-12-19T16:57:56.218582+0000","last_fullsized":"2023-12-19T16:57:56.218582+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:23:11.329826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"92'1","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200546+0000","last_change":"2023-12-19T16:57:55.794454+0000","last_active":"2023-12-19T16:57:57.200546+0000","last_peered":"2023-12-19T16:57:57.200546+0000","last_clean":"2023-12-19T16:57:57.200546+0000","last_became_active":"2023-12-19T16:57:55.794124+0000","last_became_peered":"2023-12-19T16:57:55.794124+0000","last_unstale":"2023-12-19T16:57:57.200546+0000","last_undegraded":"2023-12-19T16:57:57.200546+0000","last_fullsized":"2023-12-19T16:57:57.200546+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:47:27.112070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201811+0000","last_change":"2023-12-19T16:57:56.200703+0000","last_active":"2023-12-19T16:57:57.201811+0000","last_peered":"2023-12-19T16:57:57.201811+0000","last_clean":"2023-12-19T16:57:57.201811+0000","last_became_active":"2023-12-19T16:57:56.199760+0000","last_became_peered":"2023-12-19T16:57:56.199760+0000","last_unstale":"2023-12-19T16:57:57.201811+0000","last_undegraded":"2023-12-19T16:57:57.201811+0000","last_fullsized":"2023-12-19T16:57:57.201811+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:25.807631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.227235+0000","last_change":"2023-12-19T16:57:55.809390+0000","last_active":"2023-12-19T16:57:56.227235+0000","last_peered":"2023-12-19T16:57:56.227235+0000","last_clean":"2023-12-19T16:57:56.227235+0000","last_became_active":"2023-12-19T16:57:55.809034+0000","last_became_peered":"2023-12-19T16:57:55.809034+0000","last_unstale":"2023-12-19T16:57:56.227235+0000","last_undegraded":"2023-12-19T16:57:56.227235+0000","last_fullsized":"2023-12-19T16:57:56.227235+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:57:33.844754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.207504+0000","last_change":"2023-12-19T16:57:56.199984+0000","last_active":"2023-12-19T16:57:56.207504+0000","last_peered":"2023-12-19T16:57:56.207504+0000","last_clean":"2023-12-19T16:57:56.207504+0000","last_became_active":"2023-12-19T16:57:56.199737+0000","last_became_peered":"2023-12-19T16:57:56.199737+0000","last_unstale":"2023-12-19T16:57:56.207504+0000","last_undegraded":"2023-12-19T16:57:56.207504+0000","last_fullsized":"2023-12-19T16:57:56.207504+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:06.514750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.197167+0000","last_change":"2023-12-19T16:57:55.802763+0000","last_active":"2023-12-19T16:57:56.197167+0000","last_peered":"2023-12-19T16:57:56.197167+0000","last_clean":"2023-12-19T16:57:56.197167+0000","last_became_active":"2023-12-19T16:57:55.802495+0000","last_became_peered":"2023-12-19T16:57:55.802495+0000","last_unstale":"2023-12-19T16:57:56.197167+0000","last_undegraded":"2023-12-19T16:57:56.197167+0000","last_fullsized":"2023-12-19T16:57:56.197167+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:17:02.208440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.208617+0000","last_change":"2023-12-19T16:57:56.207461+0000","last_active":"2023-12-19T16:57:56.208617+0000","last_peered":"2023-12-19T16:57:56.208617+0000","last_clean":"2023-12-19T16:57:56.208617+0000","last_became_active":"2023-12-19T16:57:56.207185+0000","last_became_peered":"2023-12-19T16:57:56.207185+0000","last_unstale":"2023-12-19T16:57:56.208617+0000","last_undegraded":"2023-12-19T16:57:56.208617+0000","last_fullsized":"2023-12-19T16:57:56.208617+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:40.891178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.192661+0000","last_change":"2023-12-19T16:57:55.806402+0000","last_active":"2023-12-19T16:57:56.192661+0000","last_peered":"2023-12-19T16:57:56.192661+0000","last_clean":"2023-12-19T16:57:56.192661+0000","last_became_active":"2023-12-19T16:57:55.806100+0000","last_became_peered":"2023-12-19T16:57:55.806100+0000","last_unstale":"2023-12-19T16:57:56.192661+0000","last_undegraded":"2023-12-19T16:57:56.192661+0000","last_fullsized":"2023-12-19T16:57:56.192661+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:31:19.980703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.224086+0000","last_change":"2023-12-19T16:57:56.216797+0000","last_active":"2023-12-19T16:57:56.224086+0000","last_peered":"2023-12-19T16:57:56.224086+0000","last_clean":"2023-12-19T16:57:56.224086+0000","last_became_active":"2023-12-19T16:57:56.216503+0000","last_became_peered":"2023-12-19T16:57:56.216503+0000","last_unstale":"2023-12-19T16:57:56.224086+0000","last_undegraded":"2023-12-19T16:57:56.224086+0000","last_fullsized":"2023-12-19T16:57:56.224086+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:26:27.186130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202486+0000","last_change":"2023-12-19T16:57:55.806183+0000","last_active":"2023-12-19T16:57:57.202486+0000","last_peered":"2023-12-19T16:57:57.202486+0000","last_clean":"2023-12-19T16:57:57.202486+0000","last_became_active":"2023-12-19T16:57:55.805922+0000","last_became_peered":"2023-12-19T16:57:55.805922+0000","last_unstale":"2023-12-19T16:57:57.202486+0000","last_undegraded":"2023-12-19T16:57:57.202486+0000","last_fullsized":"2023-12-19T16:57:57.202486+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:45:14.604929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.203004+0000","last_change":"2023-12-19T16:57:56.216009+0000","last_active":"2023-12-19T16:57:57.203004+0000","last_peered":"2023-12-19T16:57:57.203004+0000","last_clean":"2023-12-19T16:57:57.203004+0000","last_became_active":"2023-12-19T16:57:56.215580+0000","last_became_peered":"2023-12-19T16:57:56.215580+0000","last_unstale":"2023-12-19T16:57:57.203004+0000","last_undegraded":"2023-12-19T16:57:57.203004+0000","last_fullsized":"2023-12-19T16:57:57.203004+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:01.343158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.222978+0000","last_change":"2023-12-19T16:57:55.814230+0000","last_active":"2023-12-19T16:57:56.222978+0000","last_peered":"2023-12-19T16:57:56.222978+0000","last_clean":"2023-12-19T16:57:56.222978+0000","last_became_active":"2023-12-19T16:57:55.813772+0000","last_became_peered":"2023-12-19T16:57:55.813772+0000","last_unstale":"2023-12-19T16:57:56.222978+0000","last_undegraded":"2023-12-19T16:57:56.222978+0000","last_fullsized":"2023-12-19T16:57:56.222978+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:38.509297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.226548+0000","last_change":"2023-12-19T16:57:56.220066+0000","last_active":"2023-12-19T16:57:56.226548+0000","last_peered":"2023-12-19T16:57:56.226548+0000","last_clean":"2023-12-19T16:57:56.226548+0000","last_became_active":"2023-12-19T16:57:56.219787+0000","last_became_peered":"2023-12-19T16:57:56.219787+0000","last_unstale":"2023-12-19T16:57:56.226548+0000","last_undegraded":"2023-12-19T16:57:56.226548+0000","last_fullsized":"2023-12-19T16:57:56.226548+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:26:03.234019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"92'3","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200689+0000","last_change":"2023-12-19T16:57:55.807645+0000","last_active":"2023-12-19T16:57:57.200689+0000","last_peered":"2023-12-19T16:57:57.200689+0000","last_clean":"2023-12-19T16:57:57.200689+0000","last_became_active":"2023-12-19T16:57:55.807396+0000","last_became_peered":"2023-12-19T16:57:55.807396+0000","last_unstale":"2023-12-19T16:57:57.200689+0000","last_undegraded":"2023-12-19T16:57:57.200689+0000","last_fullsized":"2023-12-19T16:57:57.200689+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:49.493948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.213516+0000","last_change":"2023-12-19T16:57:56.207048+0000","last_active":"2023-12-19T16:57:56.213516+0000","last_peered":"2023-12-19T16:57:56.213516+0000","last_clean":"2023-12-19T16:57:56.213516+0000","last_became_active":"2023-12-19T16:57:56.206790+0000","last_became_peered":"2023-12-19T16:57:56.206790+0000","last_unstale":"2023-12-19T16:57:56.213516+0000","last_undegraded":"2023-12-19T16:57:56.213516+0000","last_fullsized":"2023-12-19T16:57:56.213516+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:17.947167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"92'6","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.227630+0000","last_change":"2023-12-19T16:57:55.806475+0000","last_active":"2023-12-19T16:57:56.227630+0000","last_peered":"2023-12-19T16:57:56.227630+0000","last_clean":"2023-12-19T16:57:56.227630+0000","last_became_active":"2023-12-19T16:57:55.806186+0000","last_became_peered":"2023-12-19T16:57:55.806186+0000","last_unstale":"2023-12-19T16:57:56.227630+0000","last_undegraded":"2023-12-19T16:57:56.227630+0000","last_fullsized":"2023-12-19T16:57:56.227630+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:48:49.233136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201549+0000","last_change":"2023-12-19T16:57:56.206502+0000","last_active":"2023-12-19T16:57:57.201549+0000","last_peered":"2023-12-19T16:57:57.201549+0000","last_clean":"2023-12-19T16:57:57.201549+0000","last_became_active":"2023-12-19T16:57:56.203540+0000","last_became_peered":"2023-12-19T16:57:56.203540+0000","last_unstale":"2023-12-19T16:57:57.201549+0000","last_undegraded":"2023-12-19T16:57:57.201549+0000","last_fullsized":"2023-12-19T16:57:57.201549+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:35.407670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.224771+0000","last_change":"2023-12-19T16:57:55.810652+0000","last_active":"2023-12-19T16:57:56.224771+0000","last_peered":"2023-12-19T16:57:56.224771+0000","last_clean":"2023-12-19T16:57:56.224771+0000","last_became_active":"2023-12-19T16:57:55.809840+0000","last_became_peered":"2023-12-19T16:57:55.809840+0000","last_unstale":"2023-12-19T16:57:56.224771+0000","last_undegraded":"2023-12-19T16:57:56.224771+0000","last_fullsized":"2023-12-19T16:57:56.224771+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:39:55.692735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.216389+0000","last_change":"2023-12-19T16:57:56.210108+0000","last_active":"2023-12-19T16:57:56.216389+0000","last_peered":"2023-12-19T16:57:56.216389+0000","last_clean":"2023-12-19T16:57:56.216389+0000","last_became_active":"2023-12-19T16:57:56.209817+0000","last_became_peered":"2023-12-19T16:57:56.209817+0000","last_unstale":"2023-12-19T16:57:56.216389+0000","last_undegraded":"2023-12-19T16:57:56.216389+0000","last_fullsized":"2023-12-19T16:57:56.216389+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:33:12.268821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.220910+0000","last_change":"2023-12-19T16:57:55.807539+0000","last_active":"2023-12-19T16:57:56.220910+0000","last_peered":"2023-12-19T16:57:56.220910+0000","last_clean":"2023-12-19T16:57:56.220910+0000","last_became_active":"2023-12-19T16:57:55.807280+0000","last_became_peered":"2023-12-19T16:57:55.807280+0000","last_unstale":"2023-12-19T16:57:56.220910+0000","last_undegraded":"2023-12-19T16:57:56.220910+0000","last_fullsized":"2023-12-19T16:57:56.220910+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:20.306501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.222723+0000","last_change":"2023-12-19T16:57:56.221131+0000","last_active":"2023-12-19T16:57:56.222723+0000","last_peered":"2023-12-19T16:57:56.222723+0000","last_clean":"2023-12-19T16:57:56.222723+0000","last_became_active":"2023-12-19T16:57:56.219872+0000","last_became_peered":"2023-12-19T16:57:56.219872+0000","last_unstale":"2023-12-19T16:57:56.222723+0000","last_undegraded":"2023-12-19T16:57:56.222723+0000","last_fullsized":"2023-12-19T16:57:56.222723+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:59.495836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.201544+0000","last_change":"2023-12-19T16:57:55.806367+0000","last_active":"2023-12-19T16:57:56.201544+0000","last_peered":"2023-12-19T16:57:56.201544+0000","last_clean":"2023-12-19T16:57:56.201544+0000","last_became_active":"2023-12-19T16:57:55.805777+0000","last_became_peered":"2023-12-19T16:57:55.805777+0000","last_unstale":"2023-12-19T16:57:56.201544+0000","last_undegraded":"2023-12-19T16:57:56.201544+0000","last_fullsized":"2023-12-19T16:57:56.201544+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:24.794850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.218442+0000","last_change":"2023-12-19T16:57:56.214616+0000","last_active":"2023-12-19T16:57:56.218442+0000","last_peered":"2023-12-19T16:57:56.218442+0000","last_clean":"2023-12-19T16:57:56.218442+0000","last_became_active":"2023-12-19T16:57:56.212056+0000","last_became_peered":"2023-12-19T16:57:56.212056+0000","last_unstale":"2023-12-19T16:57:56.218442+0000","last_undegraded":"2023-12-19T16:57:56.218442+0000","last_fullsized":"2023-12-19T16:57:56.218442+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:46.636309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"92'8","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201236+0000","last_change":"2023-12-19T16:57:55.808748+0000","last_active":"2023-12-19T16:57:57.201236+0000","last_peered":"2023-12-19T16:57:57.201236+0000","last_clean":"2023-12-19T16:57:57.201236+0000","last_became_active":"2023-12-19T16:57:55.808411+0000","last_became_peered":"2023-12-19T16:57:55.808411+0000","last_unstale":"2023-12-19T16:57:57.201236+0000","last_undegraded":"2023-12-19T16:57:57.201236+0000","last_fullsized":"2023-12-19T16:57:57.201236+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:31:25.032442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202460+0000","last_change":"2023-12-19T16:57:56.207961+0000","last_active":"2023-12-19T16:57:57.202460+0000","last_peered":"2023-12-19T16:57:57.202460+0000","last_clean":"2023-12-19T16:57:57.202460+0000","last_became_active":"2023-12-19T16:57:56.207693+0000","last_became_peered":"2023-12-19T16:57:56.207693+0000","last_unstale":"2023-12-19T16:57:57.202460+0000","last_undegraded":"2023-12-19T16:57:57.202460+0000","last_fullsized":"2023-12-19T16:57:57.202460+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:41.200331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228991+0000","last_change":"2023-12-19T16:57:55.811535+0000","last_active":"2023-12-19T16:57:56.228991+0000","last_peered":"2023-12-19T16:57:56.228991+0000","last_clean":"2023-12-19T16:57:56.228991+0000","last_became_active":"2023-12-19T16:57:55.811204+0000","last_became_peered":"2023-12-19T16:57:55.811204+0000","last_unstale":"2023-12-19T16:57:56.228991+0000","last_undegraded":"2023-12-19T16:57:56.228991+0000","last_fullsized":"2023-12-19T16:57:56.228991+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:50.778566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202997+0000","last_change":"2023-12-19T16:57:56.215837+0000","last_active":"2023-12-19T16:57:57.202997+0000","last_peered":"2023-12-19T16:57:57.202997+0000","last_clean":"2023-12-19T16:57:57.202997+0000","last_became_active":"2023-12-19T16:57:56.215186+0000","last_became_peered":"2023-12-19T16:57:56.215186+0000","last_unstale":"2023-12-19T16:57:57.202997+0000","last_undegraded":"2023-12-19T16:57:57.202997+0000","last_fullsized":"2023-12-19T16:57:57.202997+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:17:12.365967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202408+0000","last_change":"2023-12-19T16:57:55.813600+0000","last_active":"2023-12-19T16:57:57.202408+0000","last_peered":"2023-12-19T16:57:57.202408+0000","last_clean":"2023-12-19T16:57:57.202408+0000","last_became_active":"2023-12-19T16:57:55.812702+0000","last_became_peered":"2023-12-19T16:57:55.812702+0000","last_unstale":"2023-12-19T16:57:57.202408+0000","last_undegraded":"2023-12-19T16:57:57.202408+0000","last_fullsized":"2023-12-19T16:57:57.202408+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:19.138973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.222463+0000","last_change":"2023-12-19T16:57:56.209466+0000","last_active":"2023-12-19T16:57:56.222463+0000","last_peered":"2023-12-19T16:57:56.222463+0000","last_clean":"2023-12-19T16:57:56.222463+0000","last_became_active":"2023-12-19T16:57:56.209212+0000","last_became_peered":"2023-12-19T16:57:56.209212+0000","last_unstale":"2023-12-19T16:57:56.222463+0000","last_undegraded":"2023-12-19T16:57:56.222463+0000","last_fullsized":"2023-12-19T16:57:56.222463+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:22.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228833+0000","last_change":"2023-12-19T16:57:55.810018+0000","last_active":"2023-12-19T16:57:56.228833+0000","last_peered":"2023-12-19T16:57:56.228833+0000","last_clean":"2023-12-19T16:57:56.228833+0000","last_became_active":"2023-12-19T16:57:55.809694+0000","last_became_peered":"2023-12-19T16:57:55.809694+0000","last_unstale":"2023-12-19T16:57:56.228833+0000","last_undegraded":"2023-12-19T16:57:56.228833+0000","last_fullsized":"2023-12-19T16:57:56.228833+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:55:17.579501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.207366+0000","last_change":"2023-12-19T16:57:56.202195+0000","last_active":"2023-12-19T16:57:56.207366+0000","last_peered":"2023-12-19T16:57:56.207366+0000","last_clean":"2023-12-19T16:57:56.207366+0000","last_became_active":"2023-12-19T16:57:56.201920+0000","last_became_peered":"2023-12-19T16:57:56.201920+0000","last_unstale":"2023-12-19T16:57:56.207366+0000","last_undegraded":"2023-12-19T16:57:56.207366+0000","last_fullsized":"2023-12-19T16:57:56.207366+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:34:33.640583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202723+0000","last_change":"2023-12-19T16:57:55.812576+0000","last_active":"2023-12-19T16:57:57.202723+0000","last_peered":"2023-12-19T16:57:57.202723+0000","last_clean":"2023-12-19T16:57:57.202723+0000","last_became_active":"2023-12-19T16:57:55.812318+0000","last_became_peered":"2023-12-19T16:57:55.812318+0000","last_unstale":"2023-12-19T16:57:57.202723+0000","last_undegraded":"2023-12-19T16:57:57.202723+0000","last_fullsized":"2023-12-19T16:57:57.202723+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:32.174861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200359+0000","last_change":"2023-12-19T16:57:56.209955+0000","last_active":"2023-12-19T16:57:57.200359+0000","last_peered":"2023-12-19T16:57:57.200359+0000","last_clean":"2023-12-19T16:57:57.200359+0000","last_became_active":"2023-12-19T16:57:56.209699+0000","last_became_peered":"2023-12-19T16:57:56.209699+0000","last_unstale":"2023-12-19T16:57:57.200359+0000","last_undegraded":"2023-12-19T16:57:57.200359+0000","last_fullsized":"2023-12-19T16:57:57.200359+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:31.579759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.221167+0000","last_change":"2023-12-19T16:57:55.808747+0000","last_active":"2023-12-19T16:57:56.221167+0000","last_peered":"2023-12-19T16:57:56.221167+0000","last_clean":"2023-12-19T16:57:56.221167+0000","last_became_active":"2023-12-19T16:57:55.808419+0000","last_became_peered":"2023-12-19T16:57:55.808419+0000","last_unstale":"2023-12-19T16:57:56.221167+0000","last_undegraded":"2023-12-19T16:57:56.221167+0000","last_fullsized":"2023-12-19T16:57:56.221167+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:04:36.243808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200413+0000","last_change":"2023-12-19T16:57:56.217460+0000","last_active":"2023-12-19T16:57:57.200413+0000","last_peered":"2023-12-19T16:57:57.200413+0000","last_clean":"2023-12-19T16:57:57.200413+0000","last_became_active":"2023-12-19T16:57:56.217201+0000","last_became_peered":"2023-12-19T16:57:56.217201+0000","last_unstale":"2023-12-19T16:57:57.200413+0000","last_undegraded":"2023-12-19T16:57:57.200413+0000","last_fullsized":"2023-12-19T16:57:57.200413+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:14.993354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.201671+0000","last_change":"2023-12-19T16:57:55.808694+0000","last_active":"2023-12-19T16:57:56.201671+0000","last_peered":"2023-12-19T16:57:56.201671+0000","last_clean":"2023-12-19T16:57:56.201671+0000","last_became_active":"2023-12-19T16:57:55.808432+0000","last_became_peered":"2023-12-19T16:57:55.808432+0000","last_unstale":"2023-12-19T16:57:56.201671+0000","last_undegraded":"2023-12-19T16:57:56.201671+0000","last_fullsized":"2023-12-19T16:57:56.201671+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:40.187550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200412+0000","last_change":"2023-12-19T16:57:56.218162+0000","last_active":"2023-12-19T16:57:57.200412+0000","last_peered":"2023-12-19T16:57:57.200412+0000","last_clean":"2023-12-19T16:57:57.200412+0000","last_became_active":"2023-12-19T16:57:56.217899+0000","last_became_peered":"2023-12-19T16:57:56.217899+0000","last_unstale":"2023-12-19T16:57:57.200412+0000","last_undegraded":"2023-12-19T16:57:57.200412+0000","last_fullsized":"2023-12-19T16:57:57.200412+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:02.317738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.201522+0000","last_change":"2023-12-19T16:57:55.807378+0000","last_active":"2023-12-19T16:57:56.201522+0000","last_peered":"2023-12-19T16:57:56.201522+0000","last_clean":"2023-12-19T16:57:56.201522+0000","last_became_active":"2023-12-19T16:57:55.807089+0000","last_became_peered":"2023-12-19T16:57:55.807089+0000","last_unstale":"2023-12-19T16:57:56.201522+0000","last_undegraded":"2023-12-19T16:57:56.201522+0000","last_fullsized":"2023-12-19T16:57:56.201522+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:12.602095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.213602+0000","last_change":"2023-12-19T16:57:56.211851+0000","last_active":"2023-12-19T16:57:56.213602+0000","last_peered":"2023-12-19T16:57:56.213602+0000","last_clean":"2023-12-19T16:57:56.213602+0000","last_became_active":"2023-12-19T16:57:56.211566+0000","last_became_peered":"2023-12-19T16:57:56.211566+0000","last_unstale":"2023-12-19T16:57:56.213602+0000","last_undegraded":"2023-12-19T16:57:56.213602+0000","last_fullsized":"2023-12-19T16:57:56.213602+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:49.738945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.229150+0000","last_change":"2023-12-19T16:57:55.810722+0000","last_active":"2023-12-19T16:57:56.229150+0000","last_peered":"2023-12-19T16:57:56.229150+0000","last_clean":"2023-12-19T16:57:56.229150+0000","last_became_active":"2023-12-19T16:57:55.810455+0000","last_became_peered":"2023-12-19T16:57:55.810455+0000","last_unstale":"2023-12-19T16:57:56.229150+0000","last_undegraded":"2023-12-19T16:57:56.229150+0000","last_fullsized":"2023-12-19T16:57:56.229150+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:17:54.600839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.216424+0000","last_change":"2023-12-19T16:57:56.210339+0000","last_active":"2023-12-19T16:57:56.216424+0000","last_peered":"2023-12-19T16:57:56.216424+0000","last_clean":"2023-12-19T16:57:56.216424+0000","last_became_active":"2023-12-19T16:57:56.210070+0000","last_became_peered":"2023-12-19T16:57:56.210070+0000","last_unstale":"2023-12-19T16:57:56.216424+0000","last_undegraded":"2023-12-19T16:57:56.216424+0000","last_fullsized":"2023-12-19T16:57:56.216424+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:06:42.537687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.201417+0000","last_change":"2023-12-19T16:57:55.813052+0000","last_active":"2023-12-19T16:57:56.201417+0000","last_peered":"2023-12-19T16:57:56.201417+0000","last_clean":"2023-12-19T16:57:56.201417+0000","last_became_active":"2023-12-19T16:57:55.811846+0000","last_became_peered":"2023-12-19T16:57:55.811846+0000","last_unstale":"2023-12-19T16:57:56.201417+0000","last_undegraded":"2023-12-19T16:57:56.201417+0000","last_fullsized":"2023-12-19T16:57:56.201417+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:27:42.048975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202922+0000","last_change":"2023-12-19T16:57:56.221734+0000","last_active":"2023-12-19T16:57:57.202922+0000","last_peered":"2023-12-19T16:57:57.202922+0000","last_clean":"2023-12-19T16:57:57.202922+0000","last_became_active":"2023-12-19T16:57:56.221422+0000","last_became_peered":"2023-12-19T16:57:56.221422+0000","last_unstale":"2023-12-19T16:57:57.202922+0000","last_undegraded":"2023-12-19T16:57:57.202922+0000","last_fullsized":"2023-12-19T16:57:57.202922+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:13.785138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.194202+0000","last_change":"2023-12-19T16:57:55.807966+0000","last_active":"2023-12-19T16:57:56.194202+0000","last_peered":"2023-12-19T16:57:56.194202+0000","last_clean":"2023-12-19T16:57:56.194202+0000","last_became_active":"2023-12-19T16:57:55.807684+0000","last_became_peered":"2023-12-19T16:57:55.807684+0000","last_unstale":"2023-12-19T16:57:56.194202+0000","last_undegraded":"2023-12-19T16:57:56.194202+0000","last_fullsized":"2023-12-19T16:57:56.194202+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:54:49.015317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.203032+0000","last_change":"2023-12-19T16:57:56.214905+0000","last_active":"2023-12-19T16:57:57.203032+0000","last_peered":"2023-12-19T16:57:57.203032+0000","last_clean":"2023-12-19T16:57:57.203032+0000","last_became_active":"2023-12-19T16:57:56.214523+0000","last_became_peered":"2023-12-19T16:57:56.214523+0000","last_unstale":"2023-12-19T16:57:57.203032+0000","last_undegraded":"2023-12-19T16:57:57.203032+0000","last_fullsized":"2023-12-19T16:57:57.203032+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:50.506729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.201524+0000","last_change":"2023-12-19T16:57:55.802937+0000","last_active":"2023-12-19T16:57:56.201524+0000","last_peered":"2023-12-19T16:57:56.201524+0000","last_clean":"2023-12-19T16:57:56.201524+0000","last_became_active":"2023-12-19T16:57:55.802632+0000","last_became_peered":"2023-12-19T16:57:55.802632+0000","last_unstale":"2023-12-19T16:57:56.201524+0000","last_undegraded":"2023-12-19T16:57:56.201524+0000","last_fullsized":"2023-12-19T16:57:56.201524+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:01.420387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.215342+0000","last_change":"2023-12-19T16:57:56.209790+0000","last_active":"2023-12-19T16:57:56.215342+0000","last_peered":"2023-12-19T16:57:56.215342+0000","last_clean":"2023-12-19T16:57:56.215342+0000","last_became_active":"2023-12-19T16:57:56.209512+0000","last_became_peered":"2023-12-19T16:57:56.209512+0000","last_unstale":"2023-12-19T16:57:56.215342+0000","last_undegraded":"2023-12-19T16:57:56.215342+0000","last_fullsized":"2023-12-19T16:57:56.215342+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:51:40.356751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.193363+0000","last_change":"2023-12-19T16:57:55.813127+0000","last_active":"2023-12-19T16:57:56.193363+0000","last_peered":"2023-12-19T16:57:56.193363+0000","last_clean":"2023-12-19T16:57:56.193363+0000","last_became_active":"2023-12-19T16:57:55.812802+0000","last_became_peered":"2023-12-19T16:57:55.812802+0000","last_unstale":"2023-12-19T16:57:56.193363+0000","last_undegraded":"2023-12-19T16:57:56.193363+0000","last_fullsized":"2023-12-19T16:57:56.193363+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:13.746370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228923+0000","last_change":"2023-12-19T16:57:56.224819+0000","last_active":"2023-12-19T16:57:56.228923+0000","last_peered":"2023-12-19T16:57:56.228923+0000","last_clean":"2023-12-19T16:57:56.228923+0000","last_became_active":"2023-12-19T16:57:56.224565+0000","last_became_peered":"2023-12-19T16:57:56.224565+0000","last_unstale":"2023-12-19T16:57:56.228923+0000","last_undegraded":"2023-12-19T16:57:56.228923+0000","last_fullsized":"2023-12-19T16:57:56.228923+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:30:03.697769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.200132+0000","last_change":"2023-12-19T16:57:55.805111+0000","last_active":"2023-12-19T16:57:56.200132+0000","last_peered":"2023-12-19T16:57:56.200132+0000","last_clean":"2023-12-19T16:57:56.200132+0000","last_became_active":"2023-12-19T16:57:55.804806+0000","last_became_peered":"2023-12-19T16:57:55.804806+0000","last_unstale":"2023-12-19T16:57:56.200132+0000","last_undegraded":"2023-12-19T16:57:56.200132+0000","last_fullsized":"2023-12-19T16:57:56.200132+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:32:17.640864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.223157+0000","last_change":"2023-12-19T16:57:56.211714+0000","last_active":"2023-12-19T16:57:56.223157+0000","last_peered":"2023-12-19T16:57:56.223157+0000","last_clean":"2023-12-19T16:57:56.223157+0000","last_became_active":"2023-12-19T16:57:56.211429+0000","last_became_peered":"2023-12-19T16:57:56.211429+0000","last_unstale":"2023-12-19T16:57:56.223157+0000","last_undegraded":"2023-12-19T16:57:56.223157+0000","last_fullsized":"2023-12-19T16:57:56.223157+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:36:45.177981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201298+0000","last_change":"2023-12-19T16:57:55.808730+0000","last_active":"2023-12-19T16:57:57.201298+0000","last_peered":"2023-12-19T16:57:57.201298+0000","last_clean":"2023-12-19T16:57:57.201298+0000","last_became_active":"2023-12-19T16:57:55.808456+0000","last_became_peered":"2023-12-19T16:57:55.808456+0000","last_unstale":"2023-12-19T16:57:57.201298+0000","last_undegraded":"2023-12-19T16:57:57.201298+0000","last_fullsized":"2023-12-19T16:57:57.201298+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:50.778566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228661+0000","last_change":"2023-12-19T16:57:56.221410+0000","last_active":"2023-12-19T16:57:56.228661+0000","last_peered":"2023-12-19T16:57:56.228661+0000","last_clean":"2023-12-19T16:57:56.228661+0000","last_became_active":"2023-12-19T16:57:56.219828+0000","last_became_peered":"2023-12-19T16:57:56.219828+0000","last_unstale":"2023-12-19T16:57:56.228661+0000","last_undegraded":"2023-12-19T16:57:56.228661+0000","last_fullsized":"2023-12-19T16:57:56.228661+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:14.908673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.218692+0000","last_change":"2023-12-19T16:57:55.805687+0000","last_active":"2023-12-19T16:57:56.218692+0000","last_peered":"2023-12-19T16:57:56.218692+0000","last_clean":"2023-12-19T16:57:56.218692+0000","last_became_active":"2023-12-19T16:57:55.805412+0000","last_became_peered":"2023-12-19T16:57:55.805412+0000","last_unstale":"2023-12-19T16:57:56.218692+0000","last_undegraded":"2023-12-19T16:57:56.218692+0000","last_fullsized":"2023-12-19T16:57:56.218692+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:31:25.032442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200415+0000","last_change":"2023-12-19T16:57:56.221354+0000","last_active":"2023-12-19T16:57:57.200415+0000","last_peered":"2023-12-19T16:57:57.200415+0000","last_clean":"2023-12-19T16:57:57.200415+0000","last_became_active":"2023-12-19T16:57:56.220298+0000","last_became_peered":"2023-12-19T16:57:56.220298+0000","last_unstale":"2023-12-19T16:57:57.200415+0000","last_undegraded":"2023-12-19T16:57:57.200415+0000","last_fullsized":"2023-12-19T16:57:57.200415+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:41:32.493785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228938+0000","last_change":"2023-12-19T16:57:55.811556+0000","last_active":"2023-12-19T16:57:56.228938+0000","last_peered":"2023-12-19T16:57:56.228938+0000","last_clean":"2023-12-19T16:57:56.228938+0000","last_became_active":"2023-12-19T16:57:55.811248+0000","last_became_peered":"2023-12-19T16:57:55.811248+0000","last_unstale":"2023-12-19T16:57:56.228938+0000","last_undegraded":"2023-12-19T16:57:56.228938+0000","last_fullsized":"2023-12-19T16:57:56.228938+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:19.138973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"81'181","reported_seq":459,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.194774+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:57:56.194774+0000","last_peered":"2023-12-19T16:57:56.194774+0000","last_clean":"2023-12-19T16:57:56.194774+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:57:56.194774+0000","last_undegraded":"2023-12-19T16:57:56.194774+0000","last_fullsized":"2023-12-19T16:57:56.194774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":674,"num_read_kb":1321,"num_write":275,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":674,"num_read_kb":1321,"num_write":275,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509629,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1676,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1716224,"data_stored":1142342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.30199999999999999},"max":{"1min":1.0169999999999999,"5min":1.3460000000000001,"15min":2.6389999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.72899999999999998,"15min":0.71499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.32700000000000001},"max":{"1min":0.875,"5min":1.6930000000000001,"15min":2.532},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.73299999999999998,"15min":0.71599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.36399999999999999},"max":{"1min":0.875,"5min":1.212,"15min":1.212},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.72999999999999998,"15min":0.72799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":0.95899999999999996,"5min":1.74,"15min":1.74},"last":0.60299999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.436,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89500000000000002,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75900000000000001,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36599999999999999},"max":{"1min":1.048,"5min":1.395,"15min":1.395},"last":0.65500000000000003}]},{"osd":3,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.73299999999999998,"15min":0.72099999999999997},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.44600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97899999999999998,"5min":1.266,"15min":1.266},"last":0.61899999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.28499999999999998},"max":{"1min":0.91000000000000003,"5min":1.841,"15min":1.841},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.44,"5min":0.40600000000000003,"15min":0.36599999999999999},"max":{"1min":0.80800000000000005,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.35699999999999998},"max":{"1min":0.85799999999999998,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98299999999999998,"5min":1.784,"15min":1.784},"last":0.51500000000000001}]},{"osd":6,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70399999999999996,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.45700000000000002,"15min":0.39800000000000002},"max":{"1min":0.94599999999999995,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.50800000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.84599999999999997,"5min":2.105,"15min":2.105},"last":0.58199999999999996}]}]},{"osd":6,"up_from":15,"seq":64424509629,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":1122304,"data_stored":551450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69499999999999995,"15min":0.69999999999999996},"min":{"1min":0.51600000000000001,"5min":0.38700000000000001,"15min":0.309},"max":{"1min":0.93700000000000006,"5min":0.999,"15min":1.8600000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70999999999999996,"15min":0.70099999999999996},"min":{"1min":0.46200000000000002,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":0.93300000000000005,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.501,"5min":0.41399999999999998,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":0.99399999999999999,"15min":4.3799999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.52100000000000002,"5min":0.47799999999999998,"15min":0.39500000000000002},"max":{"1min":0.95199999999999996,"5min":1.0189999999999999,"15min":1.583},"last":0.47399999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.751,"15min":0.74299999999999999},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.35199999999999998},"max":{"1min":0.89200000000000002,"5min":1.087,"15min":1.3180000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0549999999999999,"15min":1.1879999999999999},"last":0.41499999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.753,"15min":0.77800000000000002},"min":{"1min":0.51100000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97699999999999998,"5min":1.1020000000000001,"15min":4.673},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71499999999999997,"15min":0.72499999999999998},"min":{"1min":0.44900000000000001,"5min":0.42699999999999999,"15min":0.41399999999999998},"max":{"1min":0.84299999999999997,"5min":1.0740000000000001,"15min":1.962},"last":0.438}]},{"osd":4,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.88400000000000001,"15min":0.96999999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.35699999999999998},"max":{"1min":0.83699999999999997,"5min":0.93500000000000005,"15min":1.006},"last":0.497}]},{"osd":5,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.33300000000000002},"max":{"1min":0.83999999999999997,"5min":0.90300000000000002,"15min":1.2050000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.68000000000000005,"15min":0.65600000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41499999999999998},"max":{"1min":0.89900000000000002,"5min":1.046,"15min":1.046},"last":0.57199999999999995}]},{"osd":7,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70499999999999996,"15min":0.69599999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.438},"max":{"1min":0.92300000000000004,"5min":1.006,"15min":1.0620000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71299999999999997,"15min":0.70099999999999996},"min":{"1min":0.47399999999999998,"5min":0.46600000000000003,"15min":0.371},"max":{"1min":0.90300000000000002,"5min":1.032,"15min":1.1220000000000001},"last":0.77700000000000002}]}]},{"osd":5,"up_from":15,"seq":64424509629,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":1118208,"data_stored":551448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68100000000000005,"15min":0.69899999999999995},"min":{"1min":0.46899999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.84099999999999997,"5min":0.88400000000000001,"15min":0.95799999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90400000000000003,"5min":0.93999999999999995,"15min":1.0189999999999999},"last":0.54600000000000004}]},{"osd":1,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.48299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95099999999999996,"5min":0.995,"15min":0.995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.495,"5min":0.374,"15min":0.374},"max":{"1min":0.96699999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.53400000000000003}]},{"osd":2,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.5,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.89500000000000002,"5min":1.0409999999999999,"15min":1.1120000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.46400000000000002},"max":{"1min":0.84899999999999998,"5min":1.036,"15min":1.087},"last":0.47499999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.72899999999999998},"min":{"1min":0.51700000000000002,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":0.92000000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.77000000000000002,"15min":0.76100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.998,"5min":1.089,"15min":1.089},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54200000000000004,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77800000000000002,"5min":0.83999999999999997,"15min":1.165},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.32400000000000001},"max":{"1min":0.91100000000000003,"5min":0.92600000000000005,"15min":1.0649999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61199999999999999,"15min":0.60899999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.34699999999999998},"max":{"1min":0.88800000000000001,"5min":0.98799999999999999,"15min":1.153},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.94199999999999995,"15min":0.98999999999999999},"last":0.44}]},{"osd":7,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.133},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.67700000000000005},"min":{"1min":0.45500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1080000000000001,"5min":1.131,"15min":1.131},"last":0.505}]}]},{"osd":4,"up_from":15,"seq":64424509629,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54284,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317556,"statfs":{"total":96636764160,"available":96581177344,"internally_reserved":0,"allocated":1118208,"data_stored":551428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.39500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0740000000000001,"5min":1.877,"15min":1.877},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92800000000000005,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.86699999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.44900000000000001,"5min":0.44400000000000001,"15min":0.34799999999999998},"max":{"1min":0.85799999999999998,"5min":1.0209999999999999,"15min":1.8700000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":2.02,"15min":2.02},"last":0.71999999999999997}]},{"osd":2,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.54700000000000004,"5min":0.47099999999999997,"15min":0.38700000000000001},"max":{"1min":0.97199999999999998,"5min":1.375,"15min":1.375},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76700000000000002,"15min":0.753},"min":{"1min":0.439,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":1.927,"15min":1.927},"last":0.77000000000000002}]},{"osd":3,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88500000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.74099999999999999,"15min":0.75600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92100000000000004,"5min":0.97599999999999998,"15min":1.3129999999999999},"last":0.69499999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.379,"15min":0.35899999999999999},"max":{"1min":0.97699999999999998,"5min":1.99,"15min":1.99},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.42099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82299999999999995,"5min":0.89200000000000002,"15min":0.92800000000000005},"last":0.74099999999999999}]},{"osd":6,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59799999999999998,"15min":0.61599999999999999},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.0820000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.50700000000000001,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":1.042,"5min":1.96,"15min":1.96},"last":0.82299999999999995}]},{"osd":7,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63300000000000001,"15min":0.65200000000000002},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.80500000000000005,"5min":0.89700000000000002,"15min":1.0429999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.54100000000000004,"5min":0.46400000000000002,"15min":0.44900000000000001},"max":{"1min":1.008,"5min":1.103,"15min":1.107},"last":0.90400000000000003}]}]},{"osd":3,"up_from":15,"seq":64424509629,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36240,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335600,"statfs":{"total":96636764160,"available":96599654400,"internally_reserved":0,"allocated":1122304,"data_stored":551470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.44900000000000001},"min":{"1min":0.28000000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63300000000000001,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.32200000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70199999999999996,"5min":1.163,"15min":1.738},"last":0.60399999999999998}]},{"osd":1,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45500000000000002,"15min":0.46500000000000002},"min":{"1min":0.29899999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":1.141},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75900000000000001,"5min":1.952,"15min":2.4910000000000001},"last":0.624}]},{"osd":2,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.53000000000000003,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.85399999999999998,"15min":1.821},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.39700000000000002,"5min":0.372,"15min":0.35399999999999998},"max":{"1min":0.78100000000000003,"5min":2.855,"15min":5.1219999999999999},"last":0.58299999999999996}]},{"osd":4,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.50600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.081,"5min":4.234,"15min":4.234},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.755,"15min":0.78500000000000003},"min":{"1min":0.56399999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0649999999999999,"5min":1.111,"15min":2.2690000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.78600000000000003},"min":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.52800000000000002},"max":{"1min":1.071,"5min":1.071,"15min":2.5350000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77000000000000002,"15min":0.78800000000000003},"min":{"1min":0.54400000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.042,"5min":1.8300000000000001,"15min":1.9079999999999999},"last":0.85499999999999998}]},{"osd":6,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.77600000000000002,"15min":0.80000000000000004},"min":{"1min":0.51200000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98499999999999999,"5min":1.599,"15min":1.956},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76900000000000002,"15min":0.79800000000000004},"min":{"1min":0.54500000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0569999999999999,"5min":1.0780000000000001,"15min":1.974},"last":0.871}]},{"osd":7,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.80600000000000005,"15min":0.83099999999999996},"min":{"1min":0.63,"5min":0.54700000000000004,"15min":0.51900000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.9279999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.82999999999999996,"15min":0.86599999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.48799999999999999},"max":{"1min":0.97199999999999998,"5min":1.623,"15min":5.0759999999999996},"last":0.93999999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509628,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40340,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331500,"statfs":{"total":96636764160,"available":96595456000,"internally_reserved":0,"allocated":1126400,"data_stored":552922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.49399999999999999,"15min":0.47099999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28199999999999997},"max":{"1min":0.59399999999999997,"5min":5.4870000000000001,"15min":5.4870000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.59299999999999997,"15min":0.53300000000000003},"min":{"1min":0.30599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.58899999999999997,"5min":10.161,"15min":10.161},"last":0.38400000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.625,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.41099999999999998,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":2.4740000000000002,"15min":5.7590000000000003},"last":0.439},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70799999999999996,"5min":2.911,"15min":5.8070000000000004},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":1.1990000000000001,"15min":2.7599999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70099999999999996,"5min":2.7999999999999998,"15min":5.7190000000000003},"last":0.42399999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78900000000000003,"15min":0.79800000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3,"5min":2.4449999999999998,"15min":3.8940000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81599999999999995,"15min":0.79300000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.42999999999999999},"max":{"1min":1.226,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":0.55600000000000005}]},{"osd":5,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.84799999999999998,"15min":0.80900000000000005},"min":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.4890000000000001,"5min":5.601,"15min":5.601},"last":0.62},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81100000000000005,"15min":0.82599999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.268,"5min":2.569,"15min":2.9260000000000002},"last":0.54000000000000004}]},{"osd":6,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83399999999999996,"15min":0.83299999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.52400000000000002},"max":{"1min":1.4319999999999999,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.76500000000000001,"15min":0.79800000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":2.121},"last":0.52700000000000002}]},{"osd":7,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.87,"15min":0.82799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52300000000000002},"max":{"1min":1.3340000000000001,"5min":5.673,"15min":5.673},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82199999999999995,"15min":0.84599999999999997},"min":{"1min":0.624,"5min":0.60599999999999998,"15min":0.57799999999999996},"max":{"1min":1.369,"5min":1.429,"15min":2.0529999999999999},"last":0.66400000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509629,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50204,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321636,"statfs":{"total":96636764160,"available":96585355264,"internally_reserved":0,"allocated":1134592,"data_stored":553538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.433,"15min":0.44700000000000001},"min":{"1min":0.27900000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":2.3050000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.50600000000000001,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65900000000000003,"5min":0.84199999999999997,"15min":0.89800000000000002},"last":0.315}]},{"osd":2,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.51200000000000001},"min":{"1min":0.32900000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66000000000000003,"5min":0.80100000000000005,"15min":1.992},"last":0.498},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.29799999999999999},"max":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.97299999999999998},"last":0.42799999999999999}]},{"osd":3,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.61799999999999999},"min":{"1min":0.43099999999999999,"5min":0.36899999999999999,"15min":0.34799999999999998},"max":{"1min":0.76200000000000001,"5min":0.871,"15min":4.133},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33700000000000002},"max":{"1min":0.746,"5min":0.76400000000000001,"15min":0.91900000000000004},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.745,"15min":0.76400000000000001},"min":{"1min":0.45300000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.107,"5min":1.202,"15min":2.1949999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.78100000000000003,"15min":0.78800000000000003},"min":{"1min":0.56499999999999995,"5min":0.44400000000000001,"15min":0.42399999999999999},"max":{"1min":1.01,"5min":1.181,"15min":2.048},"last":0.60599999999999998}]},{"osd":5,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77200000000000002,"15min":0.78500000000000003},"min":{"1min":0.56200000000000006,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.0589999999999999,"5min":1.288,"15min":2.1269999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.59499999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.04,"5min":1.3380000000000001,"15min":4.0460000000000003},"last":0.62}]},{"osd":6,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76200000000000001,"15min":0.79500000000000004},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.41999999999999998},"max":{"1min":0.98599999999999999,"5min":1.0740000000000001,"15min":2.1499999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79300000000000004,"15min":0.80900000000000005},"min":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1659999999999999,"5min":1.391,"15min":2.0880000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82699999999999996,"15min":0.84199999999999997},"min":{"1min":0.63,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.024,"5min":1.2230000000000001,"15min":4.093},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80000000000000004,"15min":0.83599999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.53800000000000003},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":2.2349999999999999},"last":0.65200000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542334,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1668,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1708032,"data_stored":1141726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.434,"15min":0.42899999999999999},"min":{"1min":0.314,"5min":0.246,"15min":0.246},"max":{"1min":0.65600000000000003,"5min":0.752,"15min":0.752},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.49299999999999999},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.26300000000000001},"max":{"1min":0.73299999999999998,"5min":0.81699999999999995,"15min":0.85499999999999998},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.89700000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39800000000000002,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.441}]},{"osd":3,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.84699999999999998,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.628,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.40899999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.503,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.149,"5min":1.444,"15min":1.512},"last":0.629},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.78700000000000003},"min":{"1min":0.56000000000000005,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0349999999999999,"5min":1.1020000000000001,"15min":1.52},"last":0.58699999999999997}]},{"osd":5,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.80200000000000005,"15min":0.80000000000000004},"min":{"1min":0.55700000000000005,"5min":0.502,"15min":0.41499999999999998},"max":{"1min":1.125,"5min":1.294,"15min":1.748},"last":0.502},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81299999999999994,"15min":0.79600000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.51600000000000001},"max":{"1min":1.1830000000000001,"5min":1.256,"15min":1.649},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81100000000000005,"15min":0.79600000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46999999999999997},"max":{"1min":1.2050000000000001,"5min":1.429,"15min":1.429},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.82099999999999995,"15min":0.81499999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.44700000000000001},"max":{"1min":1.224,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.55900000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82999999999999996,"15min":0.82199999999999995},"min":{"1min":0.58599999999999997,"5min":0.505,"15min":0.48999999999999999},"max":{"1min":1.077,"5min":1.377,"15min":1.377},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.81699999999999995},"min":{"1min":0.53200000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0580000000000001,"5min":1.3300000000000001,"15min":1.661},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:58:00.355 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:58:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f846d92c640 1 Processor -- start 2023-12-19T16:58:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f846d92c640 1 -- start start 2023-12-19T16:58:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f846d92c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8468155030 0x7f846815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f846d92c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846806c970 con 0x7f846815f370 2023-12-19T16:58:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f846d92c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846815b1f0 con 0x7f84680ce740 2023-12-19T16:58:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f846d92c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846806c790 con 0x7f8468155030 2023-12-19T16:58:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f845e7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f846815f370 0x7f846806c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52538/0 (socket says 172.21.15.26:52538) 2023-12-19T16:58:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f845e7fc640 1 -- 172.21.15.26:0/1601595564 learned_addr learned my addr 172.21.15.26:0/1601595564 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:00.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448737703 0 0) 0x7f846806c970 con 0x7f846815f370 2023-12-19T16:58:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f8466ffd640 1 --2- 172.21.15.26:0/1601595564 >> v2:172.21.15.160:3300/0 conn(0x7f8468155030 0x7f846815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8454003530 con 0x7f846815f370 2023-12-19T16:58:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3203365691 0 0) 0x7f8454003530 con 0x7f846815f370 2023-12-19T16:58:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 >> v2:172.21.15.160:3300/0 conn(0x7f8468155030 msgr2=0x7f846815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 --2- 172.21.15.26:0/1601595564 >> v2:172.21.15.160:3300/0 conn(0x7f8468155030 0x7f846815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 >> v1:172.21.15.160:6789/0 conn(0x7f84680ce740 legacy=0x7f84680ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846815b4d0 con 0x7f846815f370 2023-12-19T16:58:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f8450001680 con 0x7f846815f370 2023-12-19T16:58:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 >> v1:172.21.15.26:6789/0 conn(0x7f846815f370 legacy=0x7f846806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 --2- 172.21.15.26:0/1601595564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8454003980 0x7f8454003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 --2- 172.21.15.26:0/1601595564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84540047d0 0x7f8454004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 --2- 172.21.15.26:0/1601595564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8454005190 0x7f845400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846806c790 con 0x7f84540047d0 2023-12-19T16:58:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845400c030 con 0x7f8454005190 2023-12-19T16:58:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845400c1f0 con 0x7f8454003980 2023-12-19T16:58:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f845e7fc640 1 --2- 172.21.15.26:0/1601595564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84540047d0 0x7f8454004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f8466ffd640 1 --2- 172.21.15.26:0/1601595564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8454003980 0x7f8454003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84677fe640 1 --2- 172.21.15.26:0/1601595564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8454005190 0x7f845400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f845e7fc640 1 -- 172.21.15.26:0/1601595564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8454003980 msgr2=0x7f8454003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f845e7fc640 1 --2- 172.21.15.26:0/1601595564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8454003980 0x7f8454003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f845e7fc640 1 -- 172.21.15.26:0/1601595564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8454005190 msgr2=0x7f845400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f845e7fc640 1 --2- 172.21.15.26:0/1601595564 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8454005190 0x7f845400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f845e7fc640 1 -- 172.21.15.26:0/1601595564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8454003530 con 0x7f84540047d0 2023-12-19T16:58:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.533+0000 7f84677fe640 1 -- 172.21.15.26:0/1601595564 reap_dead start 2023-12-19T16:58:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.537+0000 7f845e7fc640 1 --2- 172.21.15.26:0/1601595564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84540047d0 0x7f8454004c50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f845000c210 tx=0x7f845001f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9daa3200aacb648f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.537+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450005ae0 con 0x7f84540047d0 2023-12-19T16:58:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.537+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84500ab070 con 0x7f84540047d0 2023-12-19T16:58:00.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.537+0000 7f84667fc640 1 -- 172.21.15.26:0/1601595564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450005ea0 con 0x7f84540047d0 2023-12-19T16:58:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.537+0000 7f846d92c640 1 -- 172.21.15.26:0/1601595564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84540047d0 msgr2=0x7f8454004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.537+0000 7f846d92c640 1 --2- 172.21.15.26:0/1601595564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84540047d0 0x7f8454004c50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f845000c210 tx=0x7f845001f350 comp rx=0 tx=0).stop 2023-12-19T16:58:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.537+0000 7f846d92c640 1 -- 172.21.15.26:0/1601595564 shutdown_connections 2023-12-19T16:58:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.537+0000 7f846d92c640 1 --2- 172.21.15.26:0/1601595564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84540047d0 0x7f8454004c50 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.537+0000 7f846d92c640 1 -- 172.21.15.26:0/1601595564 >> 172.21.15.26:0/1601595564 conn(0x7f84680658e0 msgr2=0x7f8468065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:00.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.537+0000 7f846d92c640 1 -- 172.21.15.26:0/1601595564 shutdown_connections 2023-12-19T16:58:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.537+0000 7f846d92c640 1 -- 172.21.15.26:0/1601595564 wait complete. 2023-12-19T16:58:00.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.537+0000 7f846d92c640 1 Processor -- start 2023-12-19T16:58:00.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.537+0000 7f846d92c640 1 -- start start 2023-12-19T16:58:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f846d92c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84540047d0 0x7f846814baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f846d92c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846814bfe0 0x7f84681597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f846d92c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8468159d20 0x7f846814d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f846d92c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846814c4f0 con 0x7f846814bfe0 2023-12-19T16:58:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f846d92c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846814c7d0 con 0x7f8468159d20 2023-12-19T16:58:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f846d92c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84680b13a0 con 0x7f84540047d0 2023-12-19T16:58:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f845e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846814bfe0 0x7f84681597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f845e7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846814bfe0 0x7f84681597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52398/0 (socket says 172.21.15.26:52398) 2023-12-19T16:58:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f845e7fc640 1 -- 172.21.15.26:0/2924675293 learned_addr learned my addr 172.21.15.26:0/2924675293 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f8466ffd640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84540047d0 0x7f846814baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f84677fe640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8468159d20 0x7f846814d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f845e7fc640 1 -- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84540047d0 msgr2=0x7f846814baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f845e7fc640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84540047d0 0x7f846814baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f845e7fc640 1 -- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8468159d20 msgr2=0x7f846814d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f845e7fc640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8468159d20 0x7f846814d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f845e7fc640 1 -- 172.21.15.26:0/2924675293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84680b1680 con 0x7f846814bfe0 2023-12-19T16:58:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f84677fe640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8468159d20 0x7f846814d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f845e7fc640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846814bfe0 0x7f84681597e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f845001f180 tx=0x7f8454005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f8464ff9640 1 -- 172.21.15.26:0/2924675293 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84500b6070 con 0x7f846814bfe0 2023-12-19T16:58:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f8464ff9640 1 -- 172.21.15.26:0/2924675293 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8450005880 con 0x7f846814bfe0 2023-12-19T16:58:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f8464ff9640 1 -- 172.21.15.26:0/2924675293 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450005bf0 con 0x7f846814bfe0 2023-12-19T16:58:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f846d92c640 1 -- 172.21.15.26:0/2924675293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8468145730 con 0x7f846814bfe0 2023-12-19T16:58:00.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f846d92c640 1 -- 172.21.15.26:0/2924675293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846814dcd0 con 0x7f846814bfe0 2023-12-19T16:58:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f846d92c640 1 -- 172.21.15.26:0/2924675293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84680b1810 con 0x7f846814bfe0 2023-12-19T16:58:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.541+0000 7f8464ff9640 1 -- 172.21.15.26:0/2924675293 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f84500ad020 con 0x7f846814bfe0 2023-12-19T16:58:00.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.545+0000 7f8464ff9640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8448065b80 0x7f8448068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.545+0000 7f8464ff9640 1 -- 172.21.15.26:0/2924675293 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f8450120490 con 0x7f846814bfe0 2023-12-19T16:58:00.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.545+0000 7f8466ffd640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8448065b80 0x7f8448068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.545+0000 7f8466ffd640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8448065b80 0x7f8448068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f84680693e0 tx=0x7f8458009190 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:00.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.549+0000 7f8464ff9640 1 -- 172.21.15.26:0/2924675293 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f84500f1340 con 0x7f846814bfe0 2023-12-19T16:58:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.701+0000 7f846d92c640 1 -- 172.21.15.26:0/2924675293 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f846814e3c0 con 0x7f8448065b80 2023-12-19T16:58:00.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.713+0000 7f8464ff9640 1 -- 172.21.15.26:0/2924675293 <== mgr.6490 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289050 (secure 0 0 0) 0x7f846814e3c0 con 0x7f8448065b80 2023-12-19T16:58:00.718 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:00.720 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:58:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.713+0000 7f846d92c640 1 -- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8448065b80 msgr2=0x7f8448068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.713+0000 7f846d92c640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8448065b80 0x7f8448068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f84680693e0 tx=0x7f8458009190 comp rx=0 tx=0).stop 2023-12-19T16:58:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.713+0000 7f846d92c640 1 -- 172.21.15.26:0/2924675293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846814bfe0 msgr2=0x7f84681597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.713+0000 7f846d92c640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846814bfe0 0x7f84681597e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f845001f180 tx=0x7f8454005800 comp rx=0 tx=0).stop 2023-12-19T16:58:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.713+0000 7f846d92c640 1 -- 172.21.15.26:0/2924675293 shutdown_connections 2023-12-19T16:58:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.713+0000 7f846d92c640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8468159d20 0x7f846814d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.713+0000 7f846d92c640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846814bfe0 0x7f84681597e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.713+0000 7f846d92c640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8448065b80 0x7f8448068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.713+0000 7f846d92c640 1 --2- 172.21.15.26:0/2924675293 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84540047d0 0x7f846814baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.713+0000 7f846d92c640 1 -- 172.21.15.26:0/2924675293 >> 172.21.15.26:0/2924675293 conn(0x7f84680658e0 msgr2=0x7f8468154610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.713+0000 7f846d92c640 1 -- 172.21.15.26:0/2924675293 shutdown_connections 2023-12-19T16:58:00.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.713+0000 7f846d92c640 1 -- 172.21.15.26:0/2924675293 wait complete. 2023-12-19T16:58:00.739 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-19T16:57:59.274465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":674,"num_read_kb":1321,"num_write":319,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":310792,"kb_used_data":9928,"kb_used_omap":0,"kb_used_meta":300800,"kb_avail":754663928,"statfs":{"total":773094113280,"available":772775862272,"internally_reserved":0,"allocated":10166272,"data_stored":5596324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308019200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.497523"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202865+0000","last_change":"2023-12-19T16:57:56.204460+0000","last_active":"2023-12-19T16:57:57.202865+0000","last_peered":"2023-12-19T16:57:57.202865+0000","last_clean":"2023-12-19T16:57:57.202865+0000","last_became_active":"2023-12-19T16:57:56.204195+0000","last_became_peered":"2023-12-19T16:57:56.204195+0000","last_unstale":"2023-12-19T16:57:57.202865+0000","last_undegraded":"2023-12-19T16:57:57.202865+0000","last_fullsized":"2023-12-19T16:57:57.202865+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:06.873619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228808+0000","last_change":"2023-12-19T16:57:56.223513+0000","last_active":"2023-12-19T16:57:56.228808+0000","last_peered":"2023-12-19T16:57:56.228808+0000","last_clean":"2023-12-19T16:57:56.228808+0000","last_became_active":"2023-12-19T16:57:56.220518+0000","last_became_peered":"2023-12-19T16:57:56.220518+0000","last_unstale":"2023-12-19T16:57:56.228808+0000","last_undegraded":"2023-12-19T16:57:56.228808+0000","last_fullsized":"2023-12-19T16:57:56.228808+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:27:43.061756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200601+0000","last_change":"2023-12-19T16:57:56.210867+0000","last_active":"2023-12-19T16:57:57.200601+0000","last_peered":"2023-12-19T16:57:57.200601+0000","last_clean":"2023-12-19T16:57:57.200601+0000","last_became_active":"2023-12-19T16:57:56.210587+0000","last_became_peered":"2023-12-19T16:57:56.210587+0000","last_unstale":"2023-12-19T16:57:57.200601+0000","last_undegraded":"2023-12-19T16:57:57.200601+0000","last_fullsized":"2023-12-19T16:57:57.200601+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:03.424934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.224928+0000","last_change":"2023-12-19T16:57:56.221506+0000","last_active":"2023-12-19T16:57:56.224928+0000","last_peered":"2023-12-19T16:57:56.224928+0000","last_clean":"2023-12-19T16:57:56.224928+0000","last_became_active":"2023-12-19T16:57:56.220858+0000","last_became_peered":"2023-12-19T16:57:56.220858+0000","last_unstale":"2023-12-19T16:57:56.224928+0000","last_undegraded":"2023-12-19T16:57:56.224928+0000","last_fullsized":"2023-12-19T16:57:56.224928+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:26.922307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.208692+0000","last_change":"2023-12-19T16:57:56.202219+0000","last_active":"2023-12-19T16:57:56.208692+0000","last_peered":"2023-12-19T16:57:56.208692+0000","last_clean":"2023-12-19T16:57:56.208692+0000","last_became_active":"2023-12-19T16:57:56.201406+0000","last_became_peered":"2023-12-19T16:57:56.201406+0000","last_unstale":"2023-12-19T16:57:56.208692+0000","last_undegraded":"2023-12-19T16:57:56.208692+0000","last_fullsized":"2023-12-19T16:57:56.208692+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:27:07.782916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.227209+0000","last_change":"2023-12-19T16:57:56.221312+0000","last_active":"2023-12-19T16:57:56.227209+0000","last_peered":"2023-12-19T16:57:56.227209+0000","last_clean":"2023-12-19T16:57:56.227209+0000","last_became_active":"2023-12-19T16:57:56.220311+0000","last_became_peered":"2023-12-19T16:57:56.220311+0000","last_unstale":"2023-12-19T16:57:56.227209+0000","last_undegraded":"2023-12-19T16:57:56.227209+0000","last_fullsized":"2023-12-19T16:57:56.227209+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:25.807631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.218556+0000","last_change":"2023-12-19T16:57:56.211243+0000","last_active":"2023-12-19T16:57:56.218556+0000","last_peered":"2023-12-19T16:57:56.218556+0000","last_clean":"2023-12-19T16:57:56.218556+0000","last_became_active":"2023-12-19T16:57:56.210911+0000","last_became_peered":"2023-12-19T16:57:56.210911+0000","last_unstale":"2023-12-19T16:57:56.218556+0000","last_undegraded":"2023-12-19T16:57:56.218556+0000","last_fullsized":"2023-12-19T16:57:56.218556+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:48.513902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.222565+0000","last_change":"2023-12-19T16:57:56.218897+0000","last_active":"2023-12-19T16:57:56.222565+0000","last_peered":"2023-12-19T16:57:56.222565+0000","last_clean":"2023-12-19T16:57:56.222565+0000","last_became_active":"2023-12-19T16:57:56.218389+0000","last_became_peered":"2023-12-19T16:57:56.218389+0000","last_unstale":"2023-12-19T16:57:56.222565+0000","last_undegraded":"2023-12-19T16:57:56.222565+0000","last_fullsized":"2023-12-19T16:57:56.222565+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:38:34.153845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.218356+0000","last_change":"2023-12-19T16:57:56.212841+0000","last_active":"2023-12-19T16:57:56.218356+0000","last_peered":"2023-12-19T16:57:56.218356+0000","last_clean":"2023-12-19T16:57:56.218356+0000","last_became_active":"2023-12-19T16:57:56.212288+0000","last_became_peered":"2023-12-19T16:57:56.212288+0000","last_unstale":"2023-12-19T16:57:56.218356+0000","last_undegraded":"2023-12-19T16:57:56.218356+0000","last_fullsized":"2023-12-19T16:57:56.218356+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:06:31.338071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202302+0000","last_change":"2023-12-19T16:57:56.207664+0000","last_active":"2023-12-19T16:57:57.202302+0000","last_peered":"2023-12-19T16:57:57.202302+0000","last_clean":"2023-12-19T16:57:57.202302+0000","last_became_active":"2023-12-19T16:57:56.207395+0000","last_became_peered":"2023-12-19T16:57:56.207395+0000","last_unstale":"2023-12-19T16:57:57.202302+0000","last_undegraded":"2023-12-19T16:57:57.202302+0000","last_fullsized":"2023-12-19T16:57:57.202302+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:14.908673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202783+0000","last_change":"2023-12-19T16:57:56.209088+0000","last_active":"2023-12-19T16:57:57.202783+0000","last_peered":"2023-12-19T16:57:57.202783+0000","last_clean":"2023-12-19T16:57:57.202783+0000","last_became_active":"2023-12-19T16:57:56.208815+0000","last_became_peered":"2023-12-19T16:57:56.208815+0000","last_unstale":"2023-12-19T16:57:57.202783+0000","last_undegraded":"2023-12-19T16:57:57.202783+0000","last_fullsized":"2023-12-19T16:57:57.202783+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:31.374433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.222481+0000","last_change":"2023-12-19T16:57:56.210809+0000","last_active":"2023-12-19T16:57:56.222481+0000","last_peered":"2023-12-19T16:57:56.222481+0000","last_clean":"2023-12-19T16:57:56.222481+0000","last_became_active":"2023-12-19T16:57:56.210560+0000","last_became_peered":"2023-12-19T16:57:56.210560+0000","last_unstale":"2023-12-19T16:57:56.222481+0000","last_undegraded":"2023-12-19T16:57:56.222481+0000","last_fullsized":"2023-12-19T16:57:56.222481+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:30:03.697769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.219804+0000","last_change":"2023-12-19T16:57:56.219090+0000","last_active":"2023-12-19T16:57:56.219804+0000","last_peered":"2023-12-19T16:57:56.219804+0000","last_clean":"2023-12-19T16:57:56.219804+0000","last_became_active":"2023-12-19T16:57:56.218758+0000","last_became_peered":"2023-12-19T16:57:56.218758+0000","last_unstale":"2023-12-19T16:57:56.219804+0000","last_undegraded":"2023-12-19T16:57:56.219804+0000","last_fullsized":"2023-12-19T16:57:56.219804+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:00:18.660813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.218061+0000","last_change":"2023-12-19T16:57:56.212881+0000","last_active":"2023-12-19T16:57:56.218061+0000","last_peered":"2023-12-19T16:57:56.218061+0000","last_clean":"2023-12-19T16:57:56.218061+0000","last_became_active":"2023-12-19T16:57:56.212638+0000","last_became_peered":"2023-12-19T16:57:56.212638+0000","last_unstale":"2023-12-19T16:57:56.218061+0000","last_undegraded":"2023-12-19T16:57:56.218061+0000","last_fullsized":"2023-12-19T16:57:56.218061+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:47:04.684289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.217194+0000","last_change":"2023-12-19T16:57:56.216155+0000","last_active":"2023-12-19T16:57:56.217194+0000","last_peered":"2023-12-19T16:57:56.217194+0000","last_clean":"2023-12-19T16:57:56.217194+0000","last_became_active":"2023-12-19T16:57:56.215817+0000","last_became_peered":"2023-12-19T16:57:56.215817+0000","last_unstale":"2023-12-19T16:57:56.217194+0000","last_undegraded":"2023-12-19T16:57:56.217194+0000","last_fullsized":"2023-12-19T16:57:56.217194+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:41:41.101912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201253+0000","last_change":"2023-12-19T16:57:56.213826+0000","last_active":"2023-12-19T16:57:57.201253+0000","last_peered":"2023-12-19T16:57:57.201253+0000","last_clean":"2023-12-19T16:57:57.201253+0000","last_became_active":"2023-12-19T16:57:56.213551+0000","last_became_peered":"2023-12-19T16:57:56.213551+0000","last_unstale":"2023-12-19T16:57:57.201253+0000","last_undegraded":"2023-12-19T16:57:57.201253+0000","last_fullsized":"2023-12-19T16:57:57.201253+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:26:03.234019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.213581+0000","last_change":"2023-12-19T16:57:56.206859+0000","last_active":"2023-12-19T16:57:56.213581+0000","last_peered":"2023-12-19T16:57:56.213581+0000","last_clean":"2023-12-19T16:57:56.213581+0000","last_became_active":"2023-12-19T16:57:56.206058+0000","last_became_peered":"2023-12-19T16:57:56.206058+0000","last_unstale":"2023-12-19T16:57:56.213581+0000","last_undegraded":"2023-12-19T16:57:56.213581+0000","last_fullsized":"2023-12-19T16:57:56.213581+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:19:26.708980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.224809+0000","last_change":"2023-12-19T16:57:56.220964+0000","last_active":"2023-12-19T16:57:56.224809+0000","last_peered":"2023-12-19T16:57:56.224809+0000","last_clean":"2023-12-19T16:57:56.224809+0000","last_became_active":"2023-12-19T16:57:56.220667+0000","last_became_peered":"2023-12-19T16:57:56.220667+0000","last_unstale":"2023-12-19T16:57:56.224809+0000","last_undegraded":"2023-12-19T16:57:56.224809+0000","last_fullsized":"2023-12-19T16:57:56.224809+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:26:22.970957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.230267+0000","last_change":"2023-12-19T16:57:56.228330+0000","last_active":"2023-12-19T16:57:56.230267+0000","last_peered":"2023-12-19T16:57:56.230267+0000","last_clean":"2023-12-19T16:57:56.230267+0000","last_became_active":"2023-12-19T16:57:56.228047+0000","last_became_peered":"2023-12-19T16:57:56.228047+0000","last_unstale":"2023-12-19T16:57:56.230267+0000","last_undegraded":"2023-12-19T16:57:56.230267+0000","last_fullsized":"2023-12-19T16:57:56.230267+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:41.200331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.215291+0000","last_change":"2023-12-19T16:57:56.213153+0000","last_active":"2023-12-19T16:57:56.215291+0000","last_peered":"2023-12-19T16:57:56.215291+0000","last_clean":"2023-12-19T16:57:56.215291+0000","last_became_active":"2023-12-19T16:57:56.212862+0000","last_became_peered":"2023-12-19T16:57:56.212862+0000","last_unstale":"2023-12-19T16:57:56.215291+0000","last_undegraded":"2023-12-19T16:57:56.215291+0000","last_fullsized":"2023-12-19T16:57:56.215291+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:48:00.681274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201690+0000","last_change":"2023-12-19T16:57:56.205935+0000","last_active":"2023-12-19T16:57:57.201690+0000","last_peered":"2023-12-19T16:57:57.201690+0000","last_clean":"2023-12-19T16:57:57.201690+0000","last_became_active":"2023-12-19T16:57:56.205656+0000","last_became_peered":"2023-12-19T16:57:56.205656+0000","last_unstale":"2023-12-19T16:57:57.201690+0000","last_undegraded":"2023-12-19T16:57:57.201690+0000","last_fullsized":"2023-12-19T16:57:57.201690+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:38:34.153845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201835+0000","last_change":"2023-12-19T16:57:56.207535+0000","last_active":"2023-12-19T16:57:57.201835+0000","last_peered":"2023-12-19T16:57:57.201835+0000","last_clean":"2023-12-19T16:57:57.201835+0000","last_became_active":"2023-12-19T16:57:56.207163+0000","last_became_peered":"2023-12-19T16:57:56.207163+0000","last_unstale":"2023-12-19T16:57:57.201835+0000","last_undegraded":"2023-12-19T16:57:57.201835+0000","last_fullsized":"2023-12-19T16:57:57.201835+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:30:03.697769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201566+0000","last_change":"2023-12-19T16:57:56.214268+0000","last_active":"2023-12-19T16:57:57.201566+0000","last_peered":"2023-12-19T16:57:57.201566+0000","last_clean":"2023-12-19T16:57:57.201566+0000","last_became_active":"2023-12-19T16:57:56.213923+0000","last_became_peered":"2023-12-19T16:57:56.213923+0000","last_unstale":"2023-12-19T16:57:57.201566+0000","last_undegraded":"2023-12-19T16:57:57.201566+0000","last_fullsized":"2023-12-19T16:57:57.201566+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:19:42.700877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201948+0000","last_change":"2023-12-19T16:57:56.221702+0000","last_active":"2023-12-19T16:57:57.201948+0000","last_peered":"2023-12-19T16:57:57.201948+0000","last_clean":"2023-12-19T16:57:57.201948+0000","last_became_active":"2023-12-19T16:57:56.221405+0000","last_became_peered":"2023-12-19T16:57:56.221405+0000","last_unstale":"2023-12-19T16:57:57.201948+0000","last_undegraded":"2023-12-19T16:57:57.201948+0000","last_fullsized":"2023-12-19T16:57:57.201948+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:51:35.243235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202636+0000","last_change":"2023-12-19T16:57:56.200162+0000","last_active":"2023-12-19T16:57:57.202636+0000","last_peered":"2023-12-19T16:57:57.202636+0000","last_clean":"2023-12-19T16:57:57.202636+0000","last_became_active":"2023-12-19T16:57:56.199883+0000","last_became_peered":"2023-12-19T16:57:56.199883+0000","last_unstale":"2023-12-19T16:57:57.202636+0000","last_undegraded":"2023-12-19T16:57:57.202636+0000","last_fullsized":"2023-12-19T16:57:57.202636+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:22.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.226925+0000","last_change":"2023-12-19T16:57:56.222452+0000","last_active":"2023-12-19T16:57:56.226925+0000","last_peered":"2023-12-19T16:57:56.226925+0000","last_clean":"2023-12-19T16:57:56.226925+0000","last_became_active":"2023-12-19T16:57:56.222198+0000","last_became_peered":"2023-12-19T16:57:56.222198+0000","last_unstale":"2023-12-19T16:57:56.226925+0000","last_undegraded":"2023-12-19T16:57:56.226925+0000","last_fullsized":"2023-12-19T16:57:56.226925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:07.344797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200650+0000","last_change":"2023-12-19T16:57:56.215081+0000","last_active":"2023-12-19T16:57:57.200650+0000","last_peered":"2023-12-19T16:57:57.200650+0000","last_clean":"2023-12-19T16:57:57.200650+0000","last_became_active":"2023-12-19T16:57:56.214702+0000","last_became_peered":"2023-12-19T16:57:56.214702+0000","last_unstale":"2023-12-19T16:57:57.200650+0000","last_undegraded":"2023-12-19T16:57:57.200650+0000","last_fullsized":"2023-12-19T16:57:57.200650+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:28:28.939705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202007+0000","last_change":"2023-12-19T16:57:56.206053+0000","last_active":"2023-12-19T16:57:57.202007+0000","last_peered":"2023-12-19T16:57:57.202007+0000","last_clean":"2023-12-19T16:57:57.202007+0000","last_became_active":"2023-12-19T16:57:56.205032+0000","last_became_peered":"2023-12-19T16:57:56.205032+0000","last_unstale":"2023-12-19T16:57:57.202007+0000","last_undegraded":"2023-12-19T16:57:57.202007+0000","last_fullsized":"2023-12-19T16:57:57.202007+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:27:43.061756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200344+0000","last_change":"2023-12-19T16:57:56.215748+0000","last_active":"2023-12-19T16:57:57.200344+0000","last_peered":"2023-12-19T16:57:57.200344+0000","last_clean":"2023-12-19T16:57:57.200344+0000","last_became_active":"2023-12-19T16:57:56.215470+0000","last_became_peered":"2023-12-19T16:57:56.215470+0000","last_unstale":"2023-12-19T16:57:57.200344+0000","last_undegraded":"2023-12-19T16:57:57.200344+0000","last_fullsized":"2023-12-19T16:57:57.200344+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:47:46.403128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228186+0000","last_change":"2023-12-19T16:57:56.226106+0000","last_active":"2023-12-19T16:57:56.228186+0000","last_peered":"2023-12-19T16:57:56.228186+0000","last_clean":"2023-12-19T16:57:56.228186+0000","last_became_active":"2023-12-19T16:57:56.225838+0000","last_became_peered":"2023-12-19T16:57:56.225838+0000","last_unstale":"2023-12-19T16:57:56.228186+0000","last_undegraded":"2023-12-19T16:57:56.228186+0000","last_fullsized":"2023-12-19T16:57:56.228186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:40:53.136322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.226844+0000","last_change":"2023-12-19T16:57:56.218236+0000","last_active":"2023-12-19T16:57:56.226844+0000","last_peered":"2023-12-19T16:57:56.226844+0000","last_clean":"2023-12-19T16:57:56.226844+0000","last_became_active":"2023-12-19T16:57:56.217739+0000","last_became_peered":"2023-12-19T16:57:56.217739+0000","last_unstale":"2023-12-19T16:57:56.226844+0000","last_undegraded":"2023-12-19T16:57:56.226844+0000","last_fullsized":"2023-12-19T16:57:56.226844+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:41:25.952403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228757+0000","last_change":"2023-12-19T16:57:56.222326+0000","last_active":"2023-12-19T16:57:56.228757+0000","last_peered":"2023-12-19T16:57:56.228757+0000","last_clean":"2023-12-19T16:57:56.228757+0000","last_became_active":"2023-12-19T16:57:56.221814+0000","last_became_peered":"2023-12-19T16:57:56.221814+0000","last_unstale":"2023-12-19T16:57:56.228757+0000","last_undegraded":"2023-12-19T16:57:56.228757+0000","last_fullsized":"2023-12-19T16:57:56.228757+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:22.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202910+0000","last_change":"2023-12-19T16:57:56.222016+0000","last_active":"2023-12-19T16:57:57.202910+0000","last_peered":"2023-12-19T16:57:57.202910+0000","last_clean":"2023-12-19T16:57:57.202910+0000","last_became_active":"2023-12-19T16:57:56.221727+0000","last_became_peered":"2023-12-19T16:57:56.221727+0000","last_unstale":"2023-12-19T16:57:57.202910+0000","last_undegraded":"2023-12-19T16:57:57.202910+0000","last_fullsized":"2023-12-19T16:57:57.202910+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:14:01.070538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"92'4","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.203076+0000","last_change":"2023-12-19T16:57:55.812332+0000","last_active":"2023-12-19T16:57:57.203076+0000","last_peered":"2023-12-19T16:57:57.203076+0000","last_clean":"2023-12-19T16:57:57.203076+0000","last_became_active":"2023-12-19T16:57:55.812074+0000","last_became_peered":"2023-12-19T16:57:55.812074+0000","last_unstale":"2023-12-19T16:57:57.203076+0000","last_undegraded":"2023-12-19T16:57:57.203076+0000","last_fullsized":"2023-12-19T16:57:57.203076+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:47.501121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.212474+0000","last_change":"2023-12-19T16:57:56.206379+0000","last_active":"2023-12-19T16:57:56.212474+0000","last_peered":"2023-12-19T16:57:56.212474+0000","last_clean":"2023-12-19T16:57:56.212474+0000","last_became_active":"2023-12-19T16:57:56.205725+0000","last_became_peered":"2023-12-19T16:57:56.205725+0000","last_unstale":"2023-12-19T16:57:56.212474+0000","last_undegraded":"2023-12-19T16:57:56.212474+0000","last_fullsized":"2023-12-19T16:57:56.212474+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:49.316778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.222337+0000","last_change":"2023-12-19T16:57:55.808273+0000","last_active":"2023-12-19T16:57:56.222337+0000","last_peered":"2023-12-19T16:57:56.222337+0000","last_clean":"2023-12-19T16:57:56.222337+0000","last_became_active":"2023-12-19T16:57:55.807915+0000","last_became_peered":"2023-12-19T16:57:55.807915+0000","last_unstale":"2023-12-19T16:57:56.222337+0000","last_undegraded":"2023-12-19T16:57:56.222337+0000","last_fullsized":"2023-12-19T16:57:56.222337+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:13:41.250548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.229117+0000","last_change":"2023-12-19T16:57:56.225647+0000","last_active":"2023-12-19T16:57:56.229117+0000","last_peered":"2023-12-19T16:57:56.229117+0000","last_clean":"2023-12-19T16:57:56.229117+0000","last_became_active":"2023-12-19T16:57:56.225340+0000","last_became_peered":"2023-12-19T16:57:56.225340+0000","last_unstale":"2023-12-19T16:57:56.229117+0000","last_undegraded":"2023-12-19T16:57:56.229117+0000","last_fullsized":"2023-12-19T16:57:56.229117+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:26.922307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202073+0000","last_change":"2023-12-19T16:57:55.813517+0000","last_active":"2023-12-19T16:57:57.202073+0000","last_peered":"2023-12-19T16:57:57.202073+0000","last_clean":"2023-12-19T16:57:57.202073+0000","last_became_active":"2023-12-19T16:57:55.813174+0000","last_became_peered":"2023-12-19T16:57:55.813174+0000","last_unstale":"2023-12-19T16:57:57.202073+0000","last_undegraded":"2023-12-19T16:57:57.202073+0000","last_fullsized":"2023-12-19T16:57:57.202073+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:38.178292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202824+0000","last_change":"2023-12-19T16:57:56.204979+0000","last_active":"2023-12-19T16:57:57.202824+0000","last_peered":"2023-12-19T16:57:57.202824+0000","last_clean":"2023-12-19T16:57:57.202824+0000","last_became_active":"2023-12-19T16:57:56.204647+0000","last_became_peered":"2023-12-19T16:57:56.204647+0000","last_unstale":"2023-12-19T16:57:57.202824+0000","last_undegraded":"2023-12-19T16:57:57.202824+0000","last_fullsized":"2023-12-19T16:57:57.202824+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:47:28.124851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.229144+0000","last_change":"2023-12-19T16:57:55.810481+0000","last_active":"2023-12-19T16:57:56.229144+0000","last_peered":"2023-12-19T16:57:56.229144+0000","last_clean":"2023-12-19T16:57:56.229144+0000","last_became_active":"2023-12-19T16:57:55.810214+0000","last_became_peered":"2023-12-19T16:57:55.810214+0000","last_unstale":"2023-12-19T16:57:56.229144+0000","last_undegraded":"2023-12-19T16:57:56.229144+0000","last_fullsized":"2023-12-19T16:57:56.229144+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:38.178292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.218582+0000","last_change":"2023-12-19T16:57:56.211368+0000","last_active":"2023-12-19T16:57:56.218582+0000","last_peered":"2023-12-19T16:57:56.218582+0000","last_clean":"2023-12-19T16:57:56.218582+0000","last_became_active":"2023-12-19T16:57:56.211086+0000","last_became_peered":"2023-12-19T16:57:56.211086+0000","last_unstale":"2023-12-19T16:57:56.218582+0000","last_undegraded":"2023-12-19T16:57:56.218582+0000","last_fullsized":"2023-12-19T16:57:56.218582+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:23:11.329826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"92'1","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200546+0000","last_change":"2023-12-19T16:57:55.794454+0000","last_active":"2023-12-19T16:57:57.200546+0000","last_peered":"2023-12-19T16:57:57.200546+0000","last_clean":"2023-12-19T16:57:57.200546+0000","last_became_active":"2023-12-19T16:57:55.794124+0000","last_became_peered":"2023-12-19T16:57:55.794124+0000","last_unstale":"2023-12-19T16:57:57.200546+0000","last_undegraded":"2023-12-19T16:57:57.200546+0000","last_fullsized":"2023-12-19T16:57:57.200546+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:47:27.112070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201811+0000","last_change":"2023-12-19T16:57:56.200703+0000","last_active":"2023-12-19T16:57:57.201811+0000","last_peered":"2023-12-19T16:57:57.201811+0000","last_clean":"2023-12-19T16:57:57.201811+0000","last_became_active":"2023-12-19T16:57:56.199760+0000","last_became_peered":"2023-12-19T16:57:56.199760+0000","last_unstale":"2023-12-19T16:57:57.201811+0000","last_undegraded":"2023-12-19T16:57:57.201811+0000","last_fullsized":"2023-12-19T16:57:57.201811+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:25.807631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.227235+0000","last_change":"2023-12-19T16:57:55.809390+0000","last_active":"2023-12-19T16:57:56.227235+0000","last_peered":"2023-12-19T16:57:56.227235+0000","last_clean":"2023-12-19T16:57:56.227235+0000","last_became_active":"2023-12-19T16:57:55.809034+0000","last_became_peered":"2023-12-19T16:57:55.809034+0000","last_unstale":"2023-12-19T16:57:56.227235+0000","last_undegraded":"2023-12-19T16:57:56.227235+0000","last_fullsized":"2023-12-19T16:57:56.227235+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:57:33.844754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.207504+0000","last_change":"2023-12-19T16:57:56.199984+0000","last_active":"2023-12-19T16:57:56.207504+0000","last_peered":"2023-12-19T16:57:56.207504+0000","last_clean":"2023-12-19T16:57:56.207504+0000","last_became_active":"2023-12-19T16:57:56.199737+0000","last_became_peered":"2023-12-19T16:57:56.199737+0000","last_unstale":"2023-12-19T16:57:56.207504+0000","last_undegraded":"2023-12-19T16:57:56.207504+0000","last_fullsized":"2023-12-19T16:57:56.207504+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:06.514750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.197167+0000","last_change":"2023-12-19T16:57:55.802763+0000","last_active":"2023-12-19T16:57:56.197167+0000","last_peered":"2023-12-19T16:57:56.197167+0000","last_clean":"2023-12-19T16:57:56.197167+0000","last_became_active":"2023-12-19T16:57:55.802495+0000","last_became_peered":"2023-12-19T16:57:55.802495+0000","last_unstale":"2023-12-19T16:57:56.197167+0000","last_undegraded":"2023-12-19T16:57:56.197167+0000","last_fullsized":"2023-12-19T16:57:56.197167+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:17:02.208440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.208617+0000","last_change":"2023-12-19T16:57:56.207461+0000","last_active":"2023-12-19T16:57:56.208617+0000","last_peered":"2023-12-19T16:57:56.208617+0000","last_clean":"2023-12-19T16:57:56.208617+0000","last_became_active":"2023-12-19T16:57:56.207185+0000","last_became_peered":"2023-12-19T16:57:56.207185+0000","last_unstale":"2023-12-19T16:57:56.208617+0000","last_undegraded":"2023-12-19T16:57:56.208617+0000","last_fullsized":"2023-12-19T16:57:56.208617+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:55:40.891178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.192661+0000","last_change":"2023-12-19T16:57:55.806402+0000","last_active":"2023-12-19T16:57:56.192661+0000","last_peered":"2023-12-19T16:57:56.192661+0000","last_clean":"2023-12-19T16:57:56.192661+0000","last_became_active":"2023-12-19T16:57:55.806100+0000","last_became_peered":"2023-12-19T16:57:55.806100+0000","last_unstale":"2023-12-19T16:57:56.192661+0000","last_undegraded":"2023-12-19T16:57:56.192661+0000","last_fullsized":"2023-12-19T16:57:56.192661+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:31:19.980703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.224086+0000","last_change":"2023-12-19T16:57:56.216797+0000","last_active":"2023-12-19T16:57:56.224086+0000","last_peered":"2023-12-19T16:57:56.224086+0000","last_clean":"2023-12-19T16:57:56.224086+0000","last_became_active":"2023-12-19T16:57:56.216503+0000","last_became_peered":"2023-12-19T16:57:56.216503+0000","last_unstale":"2023-12-19T16:57:56.224086+0000","last_undegraded":"2023-12-19T16:57:56.224086+0000","last_fullsized":"2023-12-19T16:57:56.224086+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:26:27.186130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202486+0000","last_change":"2023-12-19T16:57:55.806183+0000","last_active":"2023-12-19T16:57:57.202486+0000","last_peered":"2023-12-19T16:57:57.202486+0000","last_clean":"2023-12-19T16:57:57.202486+0000","last_became_active":"2023-12-19T16:57:55.805922+0000","last_became_peered":"2023-12-19T16:57:55.805922+0000","last_unstale":"2023-12-19T16:57:57.202486+0000","last_undegraded":"2023-12-19T16:57:57.202486+0000","last_fullsized":"2023-12-19T16:57:57.202486+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:45:14.604929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.203004+0000","last_change":"2023-12-19T16:57:56.216009+0000","last_active":"2023-12-19T16:57:57.203004+0000","last_peered":"2023-12-19T16:57:57.203004+0000","last_clean":"2023-12-19T16:57:57.203004+0000","last_became_active":"2023-12-19T16:57:56.215580+0000","last_became_peered":"2023-12-19T16:57:56.215580+0000","last_unstale":"2023-12-19T16:57:57.203004+0000","last_undegraded":"2023-12-19T16:57:57.203004+0000","last_fullsized":"2023-12-19T16:57:57.203004+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:01.343158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.222978+0000","last_change":"2023-12-19T16:57:55.814230+0000","last_active":"2023-12-19T16:57:56.222978+0000","last_peered":"2023-12-19T16:57:56.222978+0000","last_clean":"2023-12-19T16:57:56.222978+0000","last_became_active":"2023-12-19T16:57:55.813772+0000","last_became_peered":"2023-12-19T16:57:55.813772+0000","last_unstale":"2023-12-19T16:57:56.222978+0000","last_undegraded":"2023-12-19T16:57:56.222978+0000","last_fullsized":"2023-12-19T16:57:56.222978+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:38.509297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.226548+0000","last_change":"2023-12-19T16:57:56.220066+0000","last_active":"2023-12-19T16:57:56.226548+0000","last_peered":"2023-12-19T16:57:56.226548+0000","last_clean":"2023-12-19T16:57:56.226548+0000","last_became_active":"2023-12-19T16:57:56.219787+0000","last_became_peered":"2023-12-19T16:57:56.219787+0000","last_unstale":"2023-12-19T16:57:56.226548+0000","last_undegraded":"2023-12-19T16:57:56.226548+0000","last_fullsized":"2023-12-19T16:57:56.226548+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:26:03.234019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"92'3","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200689+0000","last_change":"2023-12-19T16:57:55.807645+0000","last_active":"2023-12-19T16:57:57.200689+0000","last_peered":"2023-12-19T16:57:57.200689+0000","last_clean":"2023-12-19T16:57:57.200689+0000","last_became_active":"2023-12-19T16:57:55.807396+0000","last_became_peered":"2023-12-19T16:57:55.807396+0000","last_unstale":"2023-12-19T16:57:57.200689+0000","last_undegraded":"2023-12-19T16:57:57.200689+0000","last_fullsized":"2023-12-19T16:57:57.200689+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:49.493948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.213516+0000","last_change":"2023-12-19T16:57:56.207048+0000","last_active":"2023-12-19T16:57:56.213516+0000","last_peered":"2023-12-19T16:57:56.213516+0000","last_clean":"2023-12-19T16:57:56.213516+0000","last_became_active":"2023-12-19T16:57:56.206790+0000","last_became_peered":"2023-12-19T16:57:56.206790+0000","last_unstale":"2023-12-19T16:57:56.213516+0000","last_undegraded":"2023-12-19T16:57:56.213516+0000","last_fullsized":"2023-12-19T16:57:56.213516+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:17.947167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"92'6","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.227630+0000","last_change":"2023-12-19T16:57:55.806475+0000","last_active":"2023-12-19T16:57:56.227630+0000","last_peered":"2023-12-19T16:57:56.227630+0000","last_clean":"2023-12-19T16:57:56.227630+0000","last_became_active":"2023-12-19T16:57:55.806186+0000","last_became_peered":"2023-12-19T16:57:55.806186+0000","last_unstale":"2023-12-19T16:57:56.227630+0000","last_undegraded":"2023-12-19T16:57:56.227630+0000","last_fullsized":"2023-12-19T16:57:56.227630+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:48:49.233136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201549+0000","last_change":"2023-12-19T16:57:56.206502+0000","last_active":"2023-12-19T16:57:57.201549+0000","last_peered":"2023-12-19T16:57:57.201549+0000","last_clean":"2023-12-19T16:57:57.201549+0000","last_became_active":"2023-12-19T16:57:56.203540+0000","last_became_peered":"2023-12-19T16:57:56.203540+0000","last_unstale":"2023-12-19T16:57:57.201549+0000","last_undegraded":"2023-12-19T16:57:57.201549+0000","last_fullsized":"2023-12-19T16:57:57.201549+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:35.407670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.224771+0000","last_change":"2023-12-19T16:57:55.810652+0000","last_active":"2023-12-19T16:57:56.224771+0000","last_peered":"2023-12-19T16:57:56.224771+0000","last_clean":"2023-12-19T16:57:56.224771+0000","last_became_active":"2023-12-19T16:57:55.809840+0000","last_became_peered":"2023-12-19T16:57:55.809840+0000","last_unstale":"2023-12-19T16:57:56.224771+0000","last_undegraded":"2023-12-19T16:57:56.224771+0000","last_fullsized":"2023-12-19T16:57:56.224771+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:39:55.692735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.216389+0000","last_change":"2023-12-19T16:57:56.210108+0000","last_active":"2023-12-19T16:57:56.216389+0000","last_peered":"2023-12-19T16:57:56.216389+0000","last_clean":"2023-12-19T16:57:56.216389+0000","last_became_active":"2023-12-19T16:57:56.209817+0000","last_became_peered":"2023-12-19T16:57:56.209817+0000","last_unstale":"2023-12-19T16:57:56.216389+0000","last_undegraded":"2023-12-19T16:57:56.216389+0000","last_fullsized":"2023-12-19T16:57:56.216389+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:33:12.268821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.220910+0000","last_change":"2023-12-19T16:57:55.807539+0000","last_active":"2023-12-19T16:57:56.220910+0000","last_peered":"2023-12-19T16:57:56.220910+0000","last_clean":"2023-12-19T16:57:56.220910+0000","last_became_active":"2023-12-19T16:57:55.807280+0000","last_became_peered":"2023-12-19T16:57:55.807280+0000","last_unstale":"2023-12-19T16:57:56.220910+0000","last_undegraded":"2023-12-19T16:57:56.220910+0000","last_fullsized":"2023-12-19T16:57:56.220910+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:20.306501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.222723+0000","last_change":"2023-12-19T16:57:56.221131+0000","last_active":"2023-12-19T16:57:56.222723+0000","last_peered":"2023-12-19T16:57:56.222723+0000","last_clean":"2023-12-19T16:57:56.222723+0000","last_became_active":"2023-12-19T16:57:56.219872+0000","last_became_peered":"2023-12-19T16:57:56.219872+0000","last_unstale":"2023-12-19T16:57:56.222723+0000","last_undegraded":"2023-12-19T16:57:56.222723+0000","last_fullsized":"2023-12-19T16:57:56.222723+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:59.495836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.201544+0000","last_change":"2023-12-19T16:57:55.806367+0000","last_active":"2023-12-19T16:57:56.201544+0000","last_peered":"2023-12-19T16:57:56.201544+0000","last_clean":"2023-12-19T16:57:56.201544+0000","last_became_active":"2023-12-19T16:57:55.805777+0000","last_became_peered":"2023-12-19T16:57:55.805777+0000","last_unstale":"2023-12-19T16:57:56.201544+0000","last_undegraded":"2023-12-19T16:57:56.201544+0000","last_fullsized":"2023-12-19T16:57:56.201544+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:24.794850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.218442+0000","last_change":"2023-12-19T16:57:56.214616+0000","last_active":"2023-12-19T16:57:56.218442+0000","last_peered":"2023-12-19T16:57:56.218442+0000","last_clean":"2023-12-19T16:57:56.218442+0000","last_became_active":"2023-12-19T16:57:56.212056+0000","last_became_peered":"2023-12-19T16:57:56.212056+0000","last_unstale":"2023-12-19T16:57:56.218442+0000","last_undegraded":"2023-12-19T16:57:56.218442+0000","last_fullsized":"2023-12-19T16:57:56.218442+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:46.636309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"92'8","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201236+0000","last_change":"2023-12-19T16:57:55.808748+0000","last_active":"2023-12-19T16:57:57.201236+0000","last_peered":"2023-12-19T16:57:57.201236+0000","last_clean":"2023-12-19T16:57:57.201236+0000","last_became_active":"2023-12-19T16:57:55.808411+0000","last_became_peered":"2023-12-19T16:57:55.808411+0000","last_unstale":"2023-12-19T16:57:57.201236+0000","last_undegraded":"2023-12-19T16:57:57.201236+0000","last_fullsized":"2023-12-19T16:57:57.201236+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:31:25.032442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202460+0000","last_change":"2023-12-19T16:57:56.207961+0000","last_active":"2023-12-19T16:57:57.202460+0000","last_peered":"2023-12-19T16:57:57.202460+0000","last_clean":"2023-12-19T16:57:57.202460+0000","last_became_active":"2023-12-19T16:57:56.207693+0000","last_became_peered":"2023-12-19T16:57:56.207693+0000","last_unstale":"2023-12-19T16:57:57.202460+0000","last_undegraded":"2023-12-19T16:57:57.202460+0000","last_fullsized":"2023-12-19T16:57:57.202460+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:41.200331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228991+0000","last_change":"2023-12-19T16:57:55.811535+0000","last_active":"2023-12-19T16:57:56.228991+0000","last_peered":"2023-12-19T16:57:56.228991+0000","last_clean":"2023-12-19T16:57:56.228991+0000","last_became_active":"2023-12-19T16:57:55.811204+0000","last_became_peered":"2023-12-19T16:57:55.811204+0000","last_unstale":"2023-12-19T16:57:56.228991+0000","last_undegraded":"2023-12-19T16:57:56.228991+0000","last_fullsized":"2023-12-19T16:57:56.228991+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:50.778566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202997+0000","last_change":"2023-12-19T16:57:56.215837+0000","last_active":"2023-12-19T16:57:57.202997+0000","last_peered":"2023-12-19T16:57:57.202997+0000","last_clean":"2023-12-19T16:57:57.202997+0000","last_became_active":"2023-12-19T16:57:56.215186+0000","last_became_peered":"2023-12-19T16:57:56.215186+0000","last_unstale":"2023-12-19T16:57:57.202997+0000","last_undegraded":"2023-12-19T16:57:57.202997+0000","last_fullsized":"2023-12-19T16:57:57.202997+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:17:12.365967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202408+0000","last_change":"2023-12-19T16:57:55.813600+0000","last_active":"2023-12-19T16:57:57.202408+0000","last_peered":"2023-12-19T16:57:57.202408+0000","last_clean":"2023-12-19T16:57:57.202408+0000","last_became_active":"2023-12-19T16:57:55.812702+0000","last_became_peered":"2023-12-19T16:57:55.812702+0000","last_unstale":"2023-12-19T16:57:57.202408+0000","last_undegraded":"2023-12-19T16:57:57.202408+0000","last_fullsized":"2023-12-19T16:57:57.202408+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:19.138973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.222463+0000","last_change":"2023-12-19T16:57:56.209466+0000","last_active":"2023-12-19T16:57:56.222463+0000","last_peered":"2023-12-19T16:57:56.222463+0000","last_clean":"2023-12-19T16:57:56.222463+0000","last_became_active":"2023-12-19T16:57:56.209212+0000","last_became_peered":"2023-12-19T16:57:56.209212+0000","last_unstale":"2023-12-19T16:57:56.222463+0000","last_undegraded":"2023-12-19T16:57:56.222463+0000","last_fullsized":"2023-12-19T16:57:56.222463+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:22.645179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228833+0000","last_change":"2023-12-19T16:57:55.810018+0000","last_active":"2023-12-19T16:57:56.228833+0000","last_peered":"2023-12-19T16:57:56.228833+0000","last_clean":"2023-12-19T16:57:56.228833+0000","last_became_active":"2023-12-19T16:57:55.809694+0000","last_became_peered":"2023-12-19T16:57:55.809694+0000","last_unstale":"2023-12-19T16:57:56.228833+0000","last_undegraded":"2023-12-19T16:57:56.228833+0000","last_fullsized":"2023-12-19T16:57:56.228833+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:55:17.579501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.207366+0000","last_change":"2023-12-19T16:57:56.202195+0000","last_active":"2023-12-19T16:57:56.207366+0000","last_peered":"2023-12-19T16:57:56.207366+0000","last_clean":"2023-12-19T16:57:56.207366+0000","last_became_active":"2023-12-19T16:57:56.201920+0000","last_became_peered":"2023-12-19T16:57:56.201920+0000","last_unstale":"2023-12-19T16:57:56.207366+0000","last_undegraded":"2023-12-19T16:57:56.207366+0000","last_fullsized":"2023-12-19T16:57:56.207366+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:34:33.640583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202723+0000","last_change":"2023-12-19T16:57:55.812576+0000","last_active":"2023-12-19T16:57:57.202723+0000","last_peered":"2023-12-19T16:57:57.202723+0000","last_clean":"2023-12-19T16:57:57.202723+0000","last_became_active":"2023-12-19T16:57:55.812318+0000","last_became_peered":"2023-12-19T16:57:55.812318+0000","last_unstale":"2023-12-19T16:57:57.202723+0000","last_undegraded":"2023-12-19T16:57:57.202723+0000","last_fullsized":"2023-12-19T16:57:57.202723+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:32.174861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200359+0000","last_change":"2023-12-19T16:57:56.209955+0000","last_active":"2023-12-19T16:57:57.200359+0000","last_peered":"2023-12-19T16:57:57.200359+0000","last_clean":"2023-12-19T16:57:57.200359+0000","last_became_active":"2023-12-19T16:57:56.209699+0000","last_became_peered":"2023-12-19T16:57:56.209699+0000","last_unstale":"2023-12-19T16:57:57.200359+0000","last_undegraded":"2023-12-19T16:57:57.200359+0000","last_fullsized":"2023-12-19T16:57:57.200359+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:31.579759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.221167+0000","last_change":"2023-12-19T16:57:55.808747+0000","last_active":"2023-12-19T16:57:56.221167+0000","last_peered":"2023-12-19T16:57:56.221167+0000","last_clean":"2023-12-19T16:57:56.221167+0000","last_became_active":"2023-12-19T16:57:55.808419+0000","last_became_peered":"2023-12-19T16:57:55.808419+0000","last_unstale":"2023-12-19T16:57:56.221167+0000","last_undegraded":"2023-12-19T16:57:56.221167+0000","last_fullsized":"2023-12-19T16:57:56.221167+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:04:36.243808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200413+0000","last_change":"2023-12-19T16:57:56.217460+0000","last_active":"2023-12-19T16:57:57.200413+0000","last_peered":"2023-12-19T16:57:57.200413+0000","last_clean":"2023-12-19T16:57:57.200413+0000","last_became_active":"2023-12-19T16:57:56.217201+0000","last_became_peered":"2023-12-19T16:57:56.217201+0000","last_unstale":"2023-12-19T16:57:57.200413+0000","last_undegraded":"2023-12-19T16:57:57.200413+0000","last_fullsized":"2023-12-19T16:57:57.200413+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:14.993354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.201671+0000","last_change":"2023-12-19T16:57:55.808694+0000","last_active":"2023-12-19T16:57:56.201671+0000","last_peered":"2023-12-19T16:57:56.201671+0000","last_clean":"2023-12-19T16:57:56.201671+0000","last_became_active":"2023-12-19T16:57:55.808432+0000","last_became_peered":"2023-12-19T16:57:55.808432+0000","last_unstale":"2023-12-19T16:57:56.201671+0000","last_undegraded":"2023-12-19T16:57:56.201671+0000","last_fullsized":"2023-12-19T16:57:56.201671+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:40.187550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200412+0000","last_change":"2023-12-19T16:57:56.218162+0000","last_active":"2023-12-19T16:57:57.200412+0000","last_peered":"2023-12-19T16:57:57.200412+0000","last_clean":"2023-12-19T16:57:57.200412+0000","last_became_active":"2023-12-19T16:57:56.217899+0000","last_became_peered":"2023-12-19T16:57:56.217899+0000","last_unstale":"2023-12-19T16:57:57.200412+0000","last_undegraded":"2023-12-19T16:57:57.200412+0000","last_fullsized":"2023-12-19T16:57:57.200412+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:02.317738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.201522+0000","last_change":"2023-12-19T16:57:55.807378+0000","last_active":"2023-12-19T16:57:56.201522+0000","last_peered":"2023-12-19T16:57:56.201522+0000","last_clean":"2023-12-19T16:57:56.201522+0000","last_became_active":"2023-12-19T16:57:55.807089+0000","last_became_peered":"2023-12-19T16:57:55.807089+0000","last_unstale":"2023-12-19T16:57:56.201522+0000","last_undegraded":"2023-12-19T16:57:56.201522+0000","last_fullsized":"2023-12-19T16:57:56.201522+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:12.602095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.213602+0000","last_change":"2023-12-19T16:57:56.211851+0000","last_active":"2023-12-19T16:57:56.213602+0000","last_peered":"2023-12-19T16:57:56.213602+0000","last_clean":"2023-12-19T16:57:56.213602+0000","last_became_active":"2023-12-19T16:57:56.211566+0000","last_became_peered":"2023-12-19T16:57:56.211566+0000","last_unstale":"2023-12-19T16:57:56.213602+0000","last_undegraded":"2023-12-19T16:57:56.213602+0000","last_fullsized":"2023-12-19T16:57:56.213602+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:49.738945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.229150+0000","last_change":"2023-12-19T16:57:55.810722+0000","last_active":"2023-12-19T16:57:56.229150+0000","last_peered":"2023-12-19T16:57:56.229150+0000","last_clean":"2023-12-19T16:57:56.229150+0000","last_became_active":"2023-12-19T16:57:55.810455+0000","last_became_peered":"2023-12-19T16:57:55.810455+0000","last_unstale":"2023-12-19T16:57:56.229150+0000","last_undegraded":"2023-12-19T16:57:56.229150+0000","last_fullsized":"2023-12-19T16:57:56.229150+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:17:54.600839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.216424+0000","last_change":"2023-12-19T16:57:56.210339+0000","last_active":"2023-12-19T16:57:56.216424+0000","last_peered":"2023-12-19T16:57:56.216424+0000","last_clean":"2023-12-19T16:57:56.216424+0000","last_became_active":"2023-12-19T16:57:56.210070+0000","last_became_peered":"2023-12-19T16:57:56.210070+0000","last_unstale":"2023-12-19T16:57:56.216424+0000","last_undegraded":"2023-12-19T16:57:56.216424+0000","last_fullsized":"2023-12-19T16:57:56.216424+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:06:42.537687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.201417+0000","last_change":"2023-12-19T16:57:55.813052+0000","last_active":"2023-12-19T16:57:56.201417+0000","last_peered":"2023-12-19T16:57:56.201417+0000","last_clean":"2023-12-19T16:57:56.201417+0000","last_became_active":"2023-12-19T16:57:55.811846+0000","last_became_peered":"2023-12-19T16:57:55.811846+0000","last_unstale":"2023-12-19T16:57:56.201417+0000","last_undegraded":"2023-12-19T16:57:56.201417+0000","last_fullsized":"2023-12-19T16:57:56.201417+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:27:42.048975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.202922+0000","last_change":"2023-12-19T16:57:56.221734+0000","last_active":"2023-12-19T16:57:57.202922+0000","last_peered":"2023-12-19T16:57:57.202922+0000","last_clean":"2023-12-19T16:57:57.202922+0000","last_became_active":"2023-12-19T16:57:56.221422+0000","last_became_peered":"2023-12-19T16:57:56.221422+0000","last_unstale":"2023-12-19T16:57:57.202922+0000","last_undegraded":"2023-12-19T16:57:57.202922+0000","last_fullsized":"2023-12-19T16:57:57.202922+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:13.785138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.194202+0000","last_change":"2023-12-19T16:57:55.807966+0000","last_active":"2023-12-19T16:57:56.194202+0000","last_peered":"2023-12-19T16:57:56.194202+0000","last_clean":"2023-12-19T16:57:56.194202+0000","last_became_active":"2023-12-19T16:57:55.807684+0000","last_became_peered":"2023-12-19T16:57:55.807684+0000","last_unstale":"2023-12-19T16:57:56.194202+0000","last_undegraded":"2023-12-19T16:57:56.194202+0000","last_fullsized":"2023-12-19T16:57:56.194202+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:54:49.015317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.203032+0000","last_change":"2023-12-19T16:57:56.214905+0000","last_active":"2023-12-19T16:57:57.203032+0000","last_peered":"2023-12-19T16:57:57.203032+0000","last_clean":"2023-12-19T16:57:57.203032+0000","last_became_active":"2023-12-19T16:57:56.214523+0000","last_became_peered":"2023-12-19T16:57:56.214523+0000","last_unstale":"2023-12-19T16:57:57.203032+0000","last_undegraded":"2023-12-19T16:57:57.203032+0000","last_fullsized":"2023-12-19T16:57:57.203032+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:50.506729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.201524+0000","last_change":"2023-12-19T16:57:55.802937+0000","last_active":"2023-12-19T16:57:56.201524+0000","last_peered":"2023-12-19T16:57:56.201524+0000","last_clean":"2023-12-19T16:57:56.201524+0000","last_became_active":"2023-12-19T16:57:55.802632+0000","last_became_peered":"2023-12-19T16:57:55.802632+0000","last_unstale":"2023-12-19T16:57:56.201524+0000","last_undegraded":"2023-12-19T16:57:56.201524+0000","last_fullsized":"2023-12-19T16:57:56.201524+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:01.420387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.215342+0000","last_change":"2023-12-19T16:57:56.209790+0000","last_active":"2023-12-19T16:57:56.215342+0000","last_peered":"2023-12-19T16:57:56.215342+0000","last_clean":"2023-12-19T16:57:56.215342+0000","last_became_active":"2023-12-19T16:57:56.209512+0000","last_became_peered":"2023-12-19T16:57:56.209512+0000","last_unstale":"2023-12-19T16:57:56.215342+0000","last_undegraded":"2023-12-19T16:57:56.215342+0000","last_fullsized":"2023-12-19T16:57:56.215342+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:51:40.356751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.193363+0000","last_change":"2023-12-19T16:57:55.813127+0000","last_active":"2023-12-19T16:57:56.193363+0000","last_peered":"2023-12-19T16:57:56.193363+0000","last_clean":"2023-12-19T16:57:56.193363+0000","last_became_active":"2023-12-19T16:57:55.812802+0000","last_became_peered":"2023-12-19T16:57:55.812802+0000","last_unstale":"2023-12-19T16:57:56.193363+0000","last_undegraded":"2023-12-19T16:57:56.193363+0000","last_fullsized":"2023-12-19T16:57:56.193363+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:13.746370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228923+0000","last_change":"2023-12-19T16:57:56.224819+0000","last_active":"2023-12-19T16:57:56.228923+0000","last_peered":"2023-12-19T16:57:56.228923+0000","last_clean":"2023-12-19T16:57:56.228923+0000","last_became_active":"2023-12-19T16:57:56.224565+0000","last_became_peered":"2023-12-19T16:57:56.224565+0000","last_unstale":"2023-12-19T16:57:56.228923+0000","last_undegraded":"2023-12-19T16:57:56.228923+0000","last_fullsized":"2023-12-19T16:57:56.228923+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:30:03.697769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.200132+0000","last_change":"2023-12-19T16:57:55.805111+0000","last_active":"2023-12-19T16:57:56.200132+0000","last_peered":"2023-12-19T16:57:56.200132+0000","last_clean":"2023-12-19T16:57:56.200132+0000","last_became_active":"2023-12-19T16:57:55.804806+0000","last_became_peered":"2023-12-19T16:57:55.804806+0000","last_unstale":"2023-12-19T16:57:56.200132+0000","last_undegraded":"2023-12-19T16:57:56.200132+0000","last_fullsized":"2023-12-19T16:57:56.200132+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:32:17.640864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.223157+0000","last_change":"2023-12-19T16:57:56.211714+0000","last_active":"2023-12-19T16:57:56.223157+0000","last_peered":"2023-12-19T16:57:56.223157+0000","last_clean":"2023-12-19T16:57:56.223157+0000","last_became_active":"2023-12-19T16:57:56.211429+0000","last_became_peered":"2023-12-19T16:57:56.211429+0000","last_unstale":"2023-12-19T16:57:56.223157+0000","last_undegraded":"2023-12-19T16:57:56.223157+0000","last_fullsized":"2023-12-19T16:57:56.223157+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:36:45.177981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.201298+0000","last_change":"2023-12-19T16:57:55.808730+0000","last_active":"2023-12-19T16:57:57.201298+0000","last_peered":"2023-12-19T16:57:57.201298+0000","last_clean":"2023-12-19T16:57:57.201298+0000","last_became_active":"2023-12-19T16:57:55.808456+0000","last_became_peered":"2023-12-19T16:57:55.808456+0000","last_unstale":"2023-12-19T16:57:57.201298+0000","last_undegraded":"2023-12-19T16:57:57.201298+0000","last_fullsized":"2023-12-19T16:57:57.201298+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:50.778566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228661+0000","last_change":"2023-12-19T16:57:56.221410+0000","last_active":"2023-12-19T16:57:56.228661+0000","last_peered":"2023-12-19T16:57:56.228661+0000","last_clean":"2023-12-19T16:57:56.228661+0000","last_became_active":"2023-12-19T16:57:56.219828+0000","last_became_peered":"2023-12-19T16:57:56.219828+0000","last_unstale":"2023-12-19T16:57:56.228661+0000","last_undegraded":"2023-12-19T16:57:56.228661+0000","last_fullsized":"2023-12-19T16:57:56.228661+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:14.908673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.218692+0000","last_change":"2023-12-19T16:57:55.805687+0000","last_active":"2023-12-19T16:57:56.218692+0000","last_peered":"2023-12-19T16:57:56.218692+0000","last_clean":"2023-12-19T16:57:56.218692+0000","last_became_active":"2023-12-19T16:57:55.805412+0000","last_became_peered":"2023-12-19T16:57:55.805412+0000","last_unstale":"2023-12-19T16:57:56.218692+0000","last_undegraded":"2023-12-19T16:57:56.218692+0000","last_fullsized":"2023-12-19T16:57:56.218692+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:31:25.032442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-19T16:57:57.200415+0000","last_change":"2023-12-19T16:57:56.221354+0000","last_active":"2023-12-19T16:57:57.200415+0000","last_peered":"2023-12-19T16:57:57.200415+0000","last_clean":"2023-12-19T16:57:57.200415+0000","last_became_active":"2023-12-19T16:57:56.220298+0000","last_became_peered":"2023-12-19T16:57:56.220298+0000","last_unstale":"2023-12-19T16:57:57.200415+0000","last_undegraded":"2023-12-19T16:57:57.200415+0000","last_fullsized":"2023-12-19T16:57:57.200415+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:55.776357+0000","last_clean_scrub_stamp":"2023-12-19T16:57:55.776357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:41:32.493785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.228938+0000","last_change":"2023-12-19T16:57:55.811556+0000","last_active":"2023-12-19T16:57:56.228938+0000","last_peered":"2023-12-19T16:57:56.228938+0000","last_clean":"2023-12-19T16:57:56.228938+0000","last_became_active":"2023-12-19T16:57:55.811248+0000","last_became_peered":"2023-12-19T16:57:55.811248+0000","last_unstale":"2023-12-19T16:57:56.228938+0000","last_undegraded":"2023-12-19T16:57:56.228938+0000","last_fullsized":"2023-12-19T16:57:56.228938+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:57:54.763576+0000","last_clean_scrub_stamp":"2023-12-19T16:57:54.763576+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:19.138973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"81'181","reported_seq":459,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-12-19T16:57:56.194774+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:57:56.194774+0000","last_peered":"2023-12-19T16:57:56.194774+0000","last_clean":"2023-12-19T16:57:56.194774+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:57:56.194774+0000","last_undegraded":"2023-12-19T16:57:56.194774+0000","last_fullsized":"2023-12-19T16:57:56.194774+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":674,"num_read_kb":1321,"num_write":275,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":674,"num_read_kb":1321,"num_write":275,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":181,"ondisk_log_size":181,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509629,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1676,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1716224,"data_stored":1142342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.30199999999999999},"max":{"1min":1.0169999999999999,"5min":1.3460000000000001,"15min":2.6389999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.72899999999999998,"15min":0.71499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.32700000000000001},"max":{"1min":0.875,"5min":1.6930000000000001,"15min":2.532},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.73299999999999998,"15min":0.71599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.36399999999999999},"max":{"1min":0.875,"5min":1.212,"15min":1.212},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.72999999999999998,"15min":0.72799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":0.95899999999999996,"5min":1.74,"15min":1.74},"last":0.60299999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.436,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89500000000000002,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75900000000000001,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36599999999999999},"max":{"1min":1.048,"5min":1.395,"15min":1.395},"last":0.65500000000000003}]},{"osd":3,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.73299999999999998,"15min":0.72099999999999997},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.44600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97899999999999998,"5min":1.266,"15min":1.266},"last":0.61899999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.28499999999999998},"max":{"1min":0.91000000000000003,"5min":1.841,"15min":1.841},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.44,"5min":0.40600000000000003,"15min":0.36599999999999999},"max":{"1min":0.80800000000000005,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.35699999999999998},"max":{"1min":0.85799999999999998,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98299999999999998,"5min":1.784,"15min":1.784},"last":0.51500000000000001}]},{"osd":6,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70399999999999996,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.45700000000000002,"15min":0.39800000000000002},"max":{"1min":0.94599999999999995,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.50800000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.84599999999999997,"5min":2.105,"15min":2.105},"last":0.58199999999999996}]}]},{"osd":6,"up_from":15,"seq":64424509629,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":1122304,"data_stored":551450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69499999999999995,"15min":0.69999999999999996},"min":{"1min":0.51600000000000001,"5min":0.38700000000000001,"15min":0.309},"max":{"1min":0.93700000000000006,"5min":0.999,"15min":1.8600000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70999999999999996,"15min":0.70099999999999996},"min":{"1min":0.46200000000000002,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":0.93300000000000005,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.501,"5min":0.41399999999999998,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":0.99399999999999999,"15min":4.3799999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.52100000000000002,"5min":0.47799999999999998,"15min":0.39500000000000002},"max":{"1min":0.95199999999999996,"5min":1.0189999999999999,"15min":1.583},"last":0.47399999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.751,"15min":0.74299999999999999},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.35199999999999998},"max":{"1min":0.89200000000000002,"5min":1.087,"15min":1.3180000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0549999999999999,"15min":1.1879999999999999},"last":0.41499999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.753,"15min":0.77800000000000002},"min":{"1min":0.51100000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97699999999999998,"5min":1.1020000000000001,"15min":4.673},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71499999999999997,"15min":0.72499999999999998},"min":{"1min":0.44900000000000001,"5min":0.42699999999999999,"15min":0.41399999999999998},"max":{"1min":0.84299999999999997,"5min":1.0740000000000001,"15min":1.962},"last":0.438}]},{"osd":4,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.88400000000000001,"15min":0.96999999999999997},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.35699999999999998},"max":{"1min":0.83699999999999997,"5min":0.93500000000000005,"15min":1.006},"last":0.497}]},{"osd":5,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.33300000000000002},"max":{"1min":0.83999999999999997,"5min":0.90300000000000002,"15min":1.2050000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.68000000000000005,"15min":0.65600000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41499999999999998},"max":{"1min":0.89900000000000002,"5min":1.046,"15min":1.046},"last":0.57199999999999995}]},{"osd":7,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70499999999999996,"15min":0.69599999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.438},"max":{"1min":0.92300000000000004,"5min":1.006,"15min":1.0620000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71299999999999997,"15min":0.70099999999999996},"min":{"1min":0.47399999999999998,"5min":0.46600000000000003,"15min":0.371},"max":{"1min":0.90300000000000002,"5min":1.032,"15min":1.1220000000000001},"last":0.77700000000000002}]}]},{"osd":5,"up_from":15,"seq":64424509629,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":1118208,"data_stored":551448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68100000000000005,"15min":0.69899999999999995},"min":{"1min":0.46899999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.84099999999999997,"5min":0.88400000000000001,"15min":0.95799999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90400000000000003,"5min":0.93999999999999995,"15min":1.0189999999999999},"last":0.54600000000000004}]},{"osd":1,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.48299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95099999999999996,"5min":0.995,"15min":0.995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.495,"5min":0.374,"15min":0.374},"max":{"1min":0.96699999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.53400000000000003}]},{"osd":2,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.5,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.89500000000000002,"5min":1.0409999999999999,"15min":1.1120000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.46400000000000002},"max":{"1min":0.84899999999999998,"5min":1.036,"15min":1.087},"last":0.47499999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.72899999999999998},"min":{"1min":0.51700000000000002,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":0.92000000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.77000000000000002,"15min":0.76100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.998,"5min":1.089,"15min":1.089},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54200000000000004,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77800000000000002,"5min":0.83999999999999997,"15min":1.165},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.32400000000000001},"max":{"1min":0.91100000000000003,"5min":0.92600000000000005,"15min":1.0649999999999999},"last":0.48799999999999999}]},{"osd":6,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61199999999999999,"15min":0.60899999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.34699999999999998},"max":{"1min":0.88800000000000001,"5min":0.98799999999999999,"15min":1.153},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.94199999999999995,"15min":0.98999999999999999},"last":0.44}]},{"osd":7,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.133},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.67700000000000005},"min":{"1min":0.45500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1080000000000001,"5min":1.131,"15min":1.131},"last":0.505}]}]},{"osd":4,"up_from":15,"seq":64424509629,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54284,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317556,"statfs":{"total":96636764160,"available":96581177344,"internally_reserved":0,"allocated":1118208,"data_stored":551428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.39500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0740000000000001,"5min":1.877,"15min":1.877},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92800000000000005,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.86699999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.44900000000000001,"5min":0.44400000000000001,"15min":0.34799999999999998},"max":{"1min":0.85799999999999998,"5min":1.0209999999999999,"15min":1.8700000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":2.02,"15min":2.02},"last":0.71999999999999997}]},{"osd":2,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.54700000000000004,"5min":0.47099999999999997,"15min":0.38700000000000001},"max":{"1min":0.97199999999999998,"5min":1.375,"15min":1.375},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76700000000000002,"15min":0.753},"min":{"1min":0.439,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":1.927,"15min":1.927},"last":0.77000000000000002}]},{"osd":3,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88500000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.74099999999999999,"15min":0.75600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92100000000000004,"5min":0.97599999999999998,"15min":1.3129999999999999},"last":0.69499999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.379,"15min":0.35899999999999999},"max":{"1min":0.97699999999999998,"5min":1.99,"15min":1.99},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.42099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82299999999999995,"5min":0.89200000000000002,"15min":0.92800000000000005},"last":0.74099999999999999}]},{"osd":6,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59799999999999998,"15min":0.61599999999999999},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.0820000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.50700000000000001,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":1.042,"5min":1.96,"15min":1.96},"last":0.82299999999999995}]},{"osd":7,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63300000000000001,"15min":0.65200000000000002},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.80500000000000005,"5min":0.89700000000000002,"15min":1.0429999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.54100000000000004,"5min":0.46400000000000002,"15min":0.44900000000000001},"max":{"1min":1.008,"5min":1.103,"15min":1.107},"last":0.90400000000000003}]}]},{"osd":3,"up_from":15,"seq":64424509629,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36240,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335600,"statfs":{"total":96636764160,"available":96599654400,"internally_reserved":0,"allocated":1122304,"data_stored":551470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.44900000000000001},"min":{"1min":0.28000000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63300000000000001,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.32200000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70199999999999996,"5min":1.163,"15min":1.738},"last":0.60399999999999998}]},{"osd":1,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45500000000000002,"15min":0.46500000000000002},"min":{"1min":0.29899999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":1.141},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75900000000000001,"5min":1.952,"15min":2.4910000000000001},"last":0.624}]},{"osd":2,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.53000000000000003,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.85399999999999998,"15min":1.821},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.39700000000000002,"5min":0.372,"15min":0.35399999999999998},"max":{"1min":0.78100000000000003,"5min":2.855,"15min":5.1219999999999999},"last":0.58299999999999996}]},{"osd":4,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.50600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.081,"5min":4.234,"15min":4.234},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.755,"15min":0.78500000000000003},"min":{"1min":0.56399999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0649999999999999,"5min":1.111,"15min":2.2690000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.78600000000000003},"min":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.52800000000000002},"max":{"1min":1.071,"5min":1.071,"15min":2.5350000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77000000000000002,"15min":0.78800000000000003},"min":{"1min":0.54400000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.042,"5min":1.8300000000000001,"15min":1.9079999999999999},"last":0.85499999999999998}]},{"osd":6,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.77600000000000002,"15min":0.80000000000000004},"min":{"1min":0.51200000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98499999999999999,"5min":1.599,"15min":1.956},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76900000000000002,"15min":0.79800000000000004},"min":{"1min":0.54500000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0569999999999999,"5min":1.0780000000000001,"15min":1.974},"last":0.871}]},{"osd":7,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.80600000000000005,"15min":0.83099999999999996},"min":{"1min":0.63,"5min":0.54700000000000004,"15min":0.51900000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.9279999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.82999999999999996,"15min":0.86599999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.48799999999999999},"max":{"1min":0.97199999999999998,"5min":1.623,"15min":5.0759999999999996},"last":0.93999999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509628,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40340,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331500,"statfs":{"total":96636764160,"available":96595456000,"internally_reserved":0,"allocated":1126400,"data_stored":552922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.49399999999999999,"15min":0.47099999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28199999999999997},"max":{"1min":0.59399999999999997,"5min":5.4870000000000001,"15min":5.4870000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.59299999999999997,"15min":0.53300000000000003},"min":{"1min":0.30599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.58899999999999997,"5min":10.161,"15min":10.161},"last":0.38400000000000001}]},{"osd":1,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.625,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.41099999999999998,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":2.4740000000000002,"15min":5.7590000000000003},"last":0.439},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70799999999999996,"5min":2.911,"15min":5.8070000000000004},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":1.1990000000000001,"15min":2.7599999999999998},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70099999999999996,"5min":2.7999999999999998,"15min":5.7190000000000003},"last":0.42399999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78900000000000003,"15min":0.79800000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3,"5min":2.4449999999999998,"15min":3.8940000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81599999999999995,"15min":0.79300000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.42999999999999999},"max":{"1min":1.226,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":0.55600000000000005}]},{"osd":5,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.84799999999999998,"15min":0.80900000000000005},"min":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.4890000000000001,"5min":5.601,"15min":5.601},"last":0.62},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81100000000000005,"15min":0.82599999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.268,"5min":2.569,"15min":2.9260000000000002},"last":0.54000000000000004}]},{"osd":6,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83399999999999996,"15min":0.83299999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.52400000000000002},"max":{"1min":1.4319999999999999,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.76500000000000001,"15min":0.79800000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":2.121},"last":0.52700000000000002}]},{"osd":7,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.87,"15min":0.82799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52300000000000002},"max":{"1min":1.3340000000000001,"5min":5.673,"15min":5.673},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82199999999999995,"15min":0.84599999999999997},"min":{"1min":0.624,"5min":0.60599999999999998,"15min":0.57799999999999996},"max":{"1min":1.369,"5min":1.429,"15min":2.0529999999999999},"last":0.66400000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509629,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50204,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321636,"statfs":{"total":96636764160,"available":96585355264,"internally_reserved":0,"allocated":1134592,"data_stored":553538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.433,"15min":0.44700000000000001},"min":{"1min":0.27900000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":2.3050000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.50600000000000001,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65900000000000003,"5min":0.84199999999999997,"15min":0.89800000000000002},"last":0.315}]},{"osd":2,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.51200000000000001},"min":{"1min":0.32900000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66000000000000003,"5min":0.80100000000000005,"15min":1.992},"last":0.498},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.29799999999999999},"max":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.97299999999999998},"last":0.42799999999999999}]},{"osd":3,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.61799999999999999},"min":{"1min":0.43099999999999999,"5min":0.36899999999999999,"15min":0.34799999999999998},"max":{"1min":0.76200000000000001,"5min":0.871,"15min":4.133},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33700000000000002},"max":{"1min":0.746,"5min":0.76400000000000001,"15min":0.91900000000000004},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.745,"15min":0.76400000000000001},"min":{"1min":0.45300000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.107,"5min":1.202,"15min":2.1949999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.78100000000000003,"15min":0.78800000000000003},"min":{"1min":0.56499999999999995,"5min":0.44400000000000001,"15min":0.42399999999999999},"max":{"1min":1.01,"5min":1.181,"15min":2.048},"last":0.60599999999999998}]},{"osd":5,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77200000000000002,"15min":0.78500000000000003},"min":{"1min":0.56200000000000006,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.0589999999999999,"5min":1.288,"15min":2.1269999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.59499999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.04,"5min":1.3380000000000001,"15min":4.0460000000000003},"last":0.62}]},{"osd":6,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76200000000000001,"15min":0.79500000000000004},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.41999999999999998},"max":{"1min":0.98599999999999999,"5min":1.0740000000000001,"15min":2.1499999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79300000000000004,"15min":0.80900000000000005},"min":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1659999999999999,"5min":1.391,"15min":2.0880000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82699999999999996,"15min":0.84199999999999997},"min":{"1min":0.63,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.024,"5min":1.2230000000000001,"15min":4.093},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80000000000000004,"15min":0.83599999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.53800000000000003},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":2.2349999999999999},"last":0.65200000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542334,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1668,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1708032,"data_stored":1141726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.434,"15min":0.42899999999999999},"min":{"1min":0.314,"5min":0.246,"15min":0.246},"max":{"1min":0.65600000000000003,"5min":0.752,"15min":0.752},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.49299999999999999},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.26300000000000001},"max":{"1min":0.73299999999999998,"5min":0.81699999999999995,"15min":0.85499999999999998},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.89700000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39800000000000002,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.441}]},{"osd":3,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.84699999999999998,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.628,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.40899999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.503,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.149,"5min":1.444,"15min":1.512},"last":0.629},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.78700000000000003},"min":{"1min":0.56000000000000005,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0349999999999999,"5min":1.1020000000000001,"15min":1.52},"last":0.58699999999999997}]},{"osd":5,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.80200000000000005,"15min":0.80000000000000004},"min":{"1min":0.55700000000000005,"5min":0.502,"15min":0.41499999999999998},"max":{"1min":1.125,"5min":1.294,"15min":1.748},"last":0.502},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81299999999999994,"15min":0.79600000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.51600000000000001},"max":{"1min":1.1830000000000001,"5min":1.256,"15min":1.649},"last":0.59499999999999997}]},{"osd":6,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81100000000000005,"15min":0.79600000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46999999999999997},"max":{"1min":1.2050000000000001,"5min":1.429,"15min":1.429},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.82099999999999995,"15min":0.81499999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.44700000000000001},"max":{"1min":1.224,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.55900000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82999999999999996,"15min":0.82199999999999995},"min":{"1min":0.58599999999999997,"5min":0.505,"15min":0.48999999999999999},"max":{"1min":1.077,"5min":1.377,"15min":1.377},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.81699999999999995},"min":{"1min":0.53200000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0580000000000001,"5min":1.3300000000000001,"15min":1.661},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:58:00.744 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T16:58:00.744 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T16:58:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95ff925640 1 Processor -- start 2023-12-19T16:58:00.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95ff925640 1 -- start start 2023-12-19T16:58:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95ff925640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f95f815ada0 0x7f95f80ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95ff925640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f806b500 con 0x7f95f8070db0 2023-12-19T16:58:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95ff925640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f8070b60 con 0x7f95f80b0c30 2023-12-19T16:58:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95ff925640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f80715b0 con 0x7f95f815ada0 2023-12-19T16:58:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95fcec0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f95f8070db0 0x7f95f80b0520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52550/0 (socket says 172.21.15.26:52550) 2023-12-19T16:58:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95fcec0640 1 -- 172.21.15.26:0/2055678672 learned_addr learned my addr 172.21.15.26:0/2055678672 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034764181 0 0) 0x7f95f806b500 con 0x7f95f8070db0 2023-12-19T16:58:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95e4003530 con 0x7f95f8070db0 2023-12-19T16:58:00.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95fd6c1640 1 --2- 172.21.15.26:0/2055678672 >> v2:172.21.15.160:3300/0 conn(0x7f95f815ada0 0x7f95f80ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1885936590 0 0) 0x7f95e4003530 con 0x7f95f8070db0 2023-12-19T16:58:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 >> v2:172.21.15.160:3300/0 conn(0x7f95f815ada0 msgr2=0x7f95f80ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 --2- 172.21.15.26:0/2055678672 >> v2:172.21.15.160:3300/0 conn(0x7f95f815ada0 0x7f95f80ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 >> v1:172.21.15.160:6789/0 conn(0x7f95f80b0c30 legacy=0x7f95f806b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f80702c0 con 0x7f95f8070db0 2023-12-19T16:58:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f95e8001680 con 0x7f95f8070db0 2023-12-19T16:58:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 >> v1:172.21.15.26:6789/0 conn(0x7f95f8070db0 legacy=0x7f95f80b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 --2- 172.21.15.26:0/2055678672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e4003980 0x7f95e4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 --2- 172.21.15.26:0/2055678672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95e40047d0 0x7f95e4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 --2- 172.21.15.26:0/2055678672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e4005190 0x7f95e400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80715b0 con 0x7f95e4003980 2023-12-19T16:58:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e400c130 con 0x7f95e4005190 2023-12-19T16:58:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.929+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e400c2f0 con 0x7f95e40047d0 2023-12-19T16:58:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.933+0000 7f95fd6c1640 1 --2- 172.21.15.26:0/2055678672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e4003980 0x7f95e4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.933+0000 7f95fcec0640 1 --2- 172.21.15.26:0/2055678672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95e40047d0 0x7f95e4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.933+0000 7f95fdec2640 1 --2- 172.21.15.26:0/2055678672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e4005190 0x7f95e400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.933+0000 7f95fd6c1640 1 -- 172.21.15.26:0/2055678672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95e40047d0 msgr2=0x7f95e4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.933+0000 7f95fd6c1640 1 --2- 172.21.15.26:0/2055678672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95e40047d0 0x7f95e4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.933+0000 7f95fd6c1640 1 -- 172.21.15.26:0/2055678672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e4005190 msgr2=0x7f95e400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.933+0000 7f95fd6c1640 1 --2- 172.21.15.26:0/2055678672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e4005190 0x7f95e400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.933+0000 7f95fd6c1640 1 -- 172.21.15.26:0/2055678672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e4003530 con 0x7f95e4003980 2023-12-19T16:58:00.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.933+0000 7f95fdec2640 1 -- 172.21.15.26:0/2055678672 reap_dead start 2023-12-19T16:58:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.933+0000 7f95fd6c1640 1 --2- 172.21.15.26:0/2055678672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e4003980 0x7f95e4003da0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009ae0 tx=0x7f95f401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2140ffa8df9dc59d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.933+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f40a6510 con 0x7f95e4003980 2023-12-19T16:58:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.933+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f40a6670 con 0x7f95e4003980 2023-12-19T16:58:00.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.933+0000 7f95effff640 1 -- 172.21.15.26:0/2055678672 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f40a6a20 con 0x7f95e4003980 2023-12-19T16:58:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 -- 172.21.15.26:0/2055678672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e4003980 msgr2=0x7f95e4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 --2- 172.21.15.26:0/2055678672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e4003980 0x7f95e4003da0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009ae0 tx=0x7f95f401ca20 comp rx=0 tx=0).stop 2023-12-19T16:58:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 -- 172.21.15.26:0/2055678672 shutdown_connections 2023-12-19T16:58:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 --2- 172.21.15.26:0/2055678672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95e4003980 0x7f95e4003da0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 -- 172.21.15.26:0/2055678672 >> 172.21.15.26:0/2055678672 conn(0x7f95f80660d0 msgr2=0x7f95f80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 -- 172.21.15.26:0/2055678672 shutdown_connections 2023-12-19T16:58:00.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 -- 172.21.15.26:0/2055678672 wait complete. 2023-12-19T16:58:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 Processor -- start 2023-12-19T16:58:00.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 -- start start 2023-12-19T16:58:00.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e4003980 0x7f95f814dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95f81f9870 0x7f95f80ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f80acda0 0x7f95f815bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80ad220 con 0x7f95f80acda0 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80cabe0 con 0x7f95e4003980 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95ff925640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80c9d30 con 0x7f95f81f9870 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95fdec2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f80acda0 0x7f95f815bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95fdec2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f80acda0 0x7f95f815bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52420/0 (socket says 172.21.15.26:52420) 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95fdec2640 1 -- 172.21.15.26:0/4191755364 learned_addr learned my addr 172.21.15.26:0/4191755364 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95fd6c1640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e4003980 0x7f95f814dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.937+0000 7f95fcec0640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95f81f9870 0x7f95f80ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.941+0000 7f95fdec2640 1 -- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95f81f9870 msgr2=0x7f95f80ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.941+0000 7f95fdec2640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95f81f9870 0x7f95f80ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.941+0000 7f95fdec2640 1 -- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e4003980 msgr2=0x7f95f814dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:00.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.941+0000 7f95fdec2640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e4003980 0x7f95f814dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.941+0000 7f95fdec2640 1 -- 172.21.15.26:0/4191755364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f80c9fd0 con 0x7f95f80acda0 2023-12-19T16:58:00.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.941+0000 7f95fdec2640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f80acda0 0x7f95f815bc40 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f95f80b48c0 tx=0x7f95f000fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.941+0000 7f95edffb640 1 -- 172.21.15.26:0/4191755364 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0006d70 con 0x7f95f80acda0 2023-12-19T16:58:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.941+0000 7f95edffb640 1 -- 172.21.15.26:0/4191755364 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f0003980 con 0x7f95f80acda0 2023-12-19T16:58:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.941+0000 7f95edffb640 1 -- 172.21.15.26:0/4191755364 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0003d30 con 0x7f95f80acda0 2023-12-19T16:58:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.941+0000 7f95ff925640 1 -- 172.21.15.26:0/4191755364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f80ca2b0 con 0x7f95f80acda0 2023-12-19T16:58:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.941+0000 7f95ff925640 1 -- 172.21.15.26:0/4191755364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f80ca870 con 0x7f95f80acda0 2023-12-19T16:58:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.941+0000 7f95ff925640 1 -- 172.21.15.26:0/4191755364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f8074270 con 0x7f95f80acda0 2023-12-19T16:58:00.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.941+0000 7f95edffb640 1 -- 172.21.15.26:0/4191755364 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f95f0005110 con 0x7f95f80acda0 2023-12-19T16:58:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.945+0000 7f95edffb640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f95dc065b80 0x7f95dc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.945+0000 7f95edffb640 1 -- 172.21.15.26:0/4191755364 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f95f00828a0 con 0x7f95f80acda0 2023-12-19T16:58:00.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.945+0000 7f95fd6c1640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f95dc065b80 0x7f95dc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:00.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.945+0000 7f95fd6c1640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f95dc065b80 0x7f95dc068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f95f40af000 tx=0x7f95f40046d0 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:00.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:00.949+0000 7f95edffb640 1 -- 172.21.15.26:0/4191755364 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f95f004ffb0 con 0x7f95f80acda0 2023-12-19T16:58:01.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.161+0000 7f95ff925640 1 -- 172.21.15.26:0/4191755364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f95f80cad90 con 0x7f95f80acda0 2023-12-19T16:58:01.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.169+0000 7f95edffb640 1 -- 172.21.15.26:0/4191755364 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f95f00535d0 con 0x7f95f80acda0 2023-12-19T16:58:01.169 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2023-12-19T16:58:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.169+0000 7f95ff925640 1 -- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f95dc065b80 msgr2=0x7f95dc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.169+0000 7f95ff925640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f95dc065b80 0x7f95dc068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f95f40af000 tx=0x7f95f40046d0 comp rx=0 tx=0).stop 2023-12-19T16:58:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.169+0000 7f95ff925640 1 -- 172.21.15.26:0/4191755364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f80acda0 msgr2=0x7f95f815bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.169+0000 7f95ff925640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f80acda0 0x7f95f815bc40 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f95f80b48c0 tx=0x7f95f000fc70 comp rx=0 tx=0).stop 2023-12-19T16:58:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.169+0000 7f95ff925640 1 -- 172.21.15.26:0/4191755364 shutdown_connections 2023-12-19T16:58:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.169+0000 7f95ff925640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f80acda0 0x7f95f815bc40 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.169+0000 7f95ff925640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f95dc065b80 0x7f95dc068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.169+0000 7f95ff925640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95f81f9870 0x7f95f80ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.169+0000 7f95ff925640 1 --2- 172.21.15.26:0/4191755364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95e4003980 0x7f95f814dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.169+0000 7f95ff925640 1 -- 172.21.15.26:0/4191755364 >> 172.21.15.26:0/4191755364 conn(0x7f95f80660d0 msgr2=0x7f95f80cb140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.169+0000 7f95ff925640 1 -- 172.21.15.26:0/4191755364 shutdown_connections 2023-12-19T16:58:01.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.169+0000 7f95ff925640 1 -- 172.21.15.26:0/4191755364 wait complete. 2023-12-19T16:58:01.191 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:58:01.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.329+0000 7f762fc2d640 1 Processor -- start 2023-12-19T16:58:01.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.329+0000 7f762fc2d640 1 -- start start 2023-12-19T16:58:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762fc2d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f762806b3d0 0x7f76280ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762fc2d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762806bc30 con 0x7f76280cec40 2023-12-19T16:58:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762fc2d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76280cc6e0 con 0x7f7628154220 2023-12-19T16:58:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762fc2d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76280cf0d0 con 0x7f762806b3d0 2023-12-19T16:58:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762e1ca640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f76280cec40 0x7f762815b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52560/0 (socket says 172.21.15.26:52560) 2023-12-19T16:58:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762e1ca640 1 -- 172.21.15.26:0/2876736377 learned_addr learned my addr 172.21.15.26:0/2876736377 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:01.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762d1c8640 1 --2- 172.21.15.26:0/2876736377 >> v2:172.21.15.160:3300/0 conn(0x7f762806b3d0 0x7f76280ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762c9c7640 1 -- 172.21.15.26:0/2876736377 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2977051225 0 0) 0x7f762806bc30 con 0x7f76280cec40 2023-12-19T16:58:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762c9c7640 1 -- 172.21.15.26:0/2876736377 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7614003530 con 0x7f76280cec40 2023-12-19T16:58:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762d1c8640 1 -- 172.21.15.26:0/2876736377 >> v1:172.21.15.160:6789/0 conn(0x7f7628154220 legacy=0x7f762815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762d1c8640 1 -- 172.21.15.26:0/2876736377 >> v1:172.21.15.26:6789/0 conn(0x7f76280cec40 legacy=0x7f762815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:01.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762d1c8640 1 -- 172.21.15.26:0/2876736377 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762806b950 con 0x7f762806b3d0 2023-12-19T16:58:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762d1c8640 1 --2- 172.21.15.26:0/2876736377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f762806b3d0 0x7f76280ce700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f76180097c0 tx=0x7f761801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19de30452cede0c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762c9c7640 1 -- 172.21.15.26:0/2876736377 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180ab650 con 0x7f762806b3d0 2023-12-19T16:58:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762c9c7640 1 -- 172.21.15.26:0/2876736377 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76180ab7b0 con 0x7f762806b3d0 2023-12-19T16:58:01.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762c9c7640 1 -- 172.21.15.26:0/2876736377 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180abb40 con 0x7f762806b3d0 2023-12-19T16:58:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762fc2d640 1 -- 172.21.15.26:0/2876736377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f762806b3d0 msgr2=0x7f76280ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762fc2d640 1 --2- 172.21.15.26:0/2876736377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f762806b3d0 0x7f76280ce700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f76180097c0 tx=0x7f761801c450 comp rx=0 tx=0).stop 2023-12-19T16:58:01.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762fc2d640 1 -- 172.21.15.26:0/2876736377 shutdown_connections 2023-12-19T16:58:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762fc2d640 1 --2- 172.21.15.26:0/2876736377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f762806b3d0 0x7f76280ce700 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762fc2d640 1 -- 172.21.15.26:0/2876736377 >> 172.21.15.26:0/2876736377 conn(0x7f76280658e0 msgr2=0x7f7628065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762fc2d640 1 -- 172.21.15.26:0/2876736377 shutdown_connections 2023-12-19T16:58:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762fc2d640 1 -- 172.21.15.26:0/2876736377 wait complete. 2023-12-19T16:58:01.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.337+0000 7f762fc2d640 1 Processor -- start 2023-12-19T16:58:01.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762fc2d640 1 -- start start 2023-12-19T16:58:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762fc2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f762806b3d0 0x7f76281449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:01.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762fc2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76280cec40 0x7f7628150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762fc2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7628154220 0x7f762814ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762fc2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76281520a0 con 0x7f76280cec40 2023-12-19T16:58:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762fc2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7628151520 con 0x7f762806b3d0 2023-12-19T16:58:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762fc2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7628151800 con 0x7f7628154220 2023-12-19T16:58:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762d1c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76280cec40 0x7f7628150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762d1c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76280cec40 0x7f7628150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52436/0 (socket says 172.21.15.26:52436) 2023-12-19T16:58:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762d1c8640 1 -- 172.21.15.26:0/1506672457 learned_addr learned my addr 172.21.15.26:0/1506672457 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762e1ca640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7628154220 0x7f762814ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762d9c9640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f762806b3d0 0x7f76281449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762d1c8640 1 -- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7628154220 msgr2=0x7f762814ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762d1c8640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7628154220 0x7f762814ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762d1c8640 1 -- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f762806b3d0 msgr2=0x7f76281449b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:01.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762d1c8640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f762806b3d0 0x7f76281449b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:01.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762d1c8640 1 -- 172.21.15.26:0/1506672457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7618009420 con 0x7f76280cec40 2023-12-19T16:58:01.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762e1ca640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7628154220 0x7f762814ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:01.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762d9c9640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f762806b3d0 0x7f76281449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:01.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762d1c8640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76280cec40 0x7f7628150a80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f76180ad040 tx=0x7f761801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:01.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f761e7fc640 1 -- 172.21.15.26:0/1506672457 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180bf610 con 0x7f76280cec40 2023-12-19T16:58:01.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f761e7fc640 1 -- 172.21.15.26:0/1506672457 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76180bf770 con 0x7f76280cec40 2023-12-19T16:58:01.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f761e7fc640 1 -- 172.21.15.26:0/1506672457 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180bfb20 con 0x7f76280cec40 2023-12-19T16:58:01.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762fc2d640 1 -- 172.21.15.26:0/1506672457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7628151ae0 con 0x7f76280cec40 2023-12-19T16:58:01.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762fc2d640 1 -- 172.21.15.26:0/1506672457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7628144ef0 con 0x7f76280cec40 2023-12-19T16:58:01.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.345+0000 7f762fc2d640 1 -- 172.21.15.26:0/1506672457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7628074270 con 0x7f76280cec40 2023-12-19T16:58:01.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.349+0000 7f761e7fc640 1 -- 172.21.15.26:0/1506672457 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f76180bfc80 con 0x7f76280cec40 2023-12-19T16:58:01.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.349+0000 7f761e7fc640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f760c065b80 0x7f760c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:01.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.349+0000 7f761e7fc640 1 -- 172.21.15.26:0/1506672457 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f761811dfe0 con 0x7f76280cec40 2023-12-19T16:58:01.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.349+0000 7f762d9c9640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f760c065b80 0x7f760c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:01.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.349+0000 7f762d9c9640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f760c065b80 0x7f760c068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f76240016f0 tx=0x7f7624009040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:01.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.353+0000 7f761e7fc640 1 -- 172.21.15.26:0/1506672457 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f76180eb630 con 0x7f76280cec40 2023-12-19T16:58:01.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.541+0000 7f762fc2d640 1 -- 172.21.15.26:0/1506672457 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76280cdf60 con 0x7f76280cec40 2023-12-19T16:58:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.545+0000 7f761e7fc640 1 -- 172.21.15.26:0/1506672457 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f76180f3e60 con 0x7f76280cec40 2023-12-19T16:58:01.544 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:01.544 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/426534014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":426534014},{"type":"v1","addr":"172.21.15.160:6837","nonce":426534014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/1088174367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":1088174367},{"type":"v1","addr":"172.21.15.160:6836","nonce":1088174367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6981,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/343942943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":343942943},{"type":"v1","addr":"172.21.15.26:6834","nonce":343942943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:57:56.187532+0000","modified":"2023-12-19T16:57:59.206380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3421559564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3421559564},{"type":"v1","addr":"172.21.15.26:6835","nonce":3421559564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-19T16:58:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 41 2023-12-19T16:58:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.545+0000 7f762fc2d640 1 -- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f760c065b80 msgr2=0x7f760c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.545+0000 7f762fc2d640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f760c065b80 0x7f760c068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f76240016f0 tx=0x7f7624009040 comp rx=0 tx=0).stop 2023-12-19T16:58:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.545+0000 7f762fc2d640 1 -- 172.21.15.26:0/1506672457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76280cec40 msgr2=0x7f7628150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.545+0000 7f762fc2d640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76280cec40 0x7f7628150a80 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f76180ad040 tx=0x7f761801ac90 comp rx=0 tx=0).stop 2023-12-19T16:58:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.545+0000 7f762fc2d640 1 -- 172.21.15.26:0/1506672457 shutdown_connections 2023-12-19T16:58:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.545+0000 7f762fc2d640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f760c065b80 0x7f760c068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.545+0000 7f762fc2d640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7628154220 0x7f762814ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:01.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.545+0000 7f762fc2d640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76280cec40 0x7f7628150a80 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:01.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.545+0000 7f762fc2d640 1 --2- 172.21.15.26:0/1506672457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f762806b3d0 0x7f76281449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:01.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.545+0000 7f762fc2d640 1 -- 172.21.15.26:0/1506672457 >> 172.21.15.26:0/1506672457 conn(0x7f76280658e0 msgr2=0x7f76280ac850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:01.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.545+0000 7f762fc2d640 1 -- 172.21.15.26:0/1506672457 shutdown_connections 2023-12-19T16:58:01.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:01.545+0000 7f762fc2d640 1 -- 172.21.15.26:0/1506672457 wait complete. 2023-12-19T16:58:01.756 INFO:tasks.ceph.mds.a.smithi026.stderr:2023-12-19T16:58:01.757+0000 7f9901dad640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T16:58:01.968 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T16:57:56.187532+0000', 'modified': '2023-12-19T16:57:59.206380+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6990': {'gid': 6990, 'name': 'a', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/3421559564', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 3421559564}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 3421559564}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T16:58:01.968 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T16:58:01.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-19T16:58:02.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.113+0000 7f239e0f9640 1 Processor -- start 2023-12-19T16:58:02.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.113+0000 7f239e0f9640 1 -- start start 2023-12-19T16:58:02.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f239e0f9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2398155030 0x7f239815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f239e0f9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f23980ce740 0x7f23980cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f239e0f9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239806c970 con 0x7f239815f370 2023-12-19T16:58:02.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f239e0f9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239806c790 con 0x7f23980ce740 2023-12-19T16:58:02.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f239e0f9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23980cf130 con 0x7f2398155030 2023-12-19T16:58:02.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f2396ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f239815f370 0x7f239806c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52562/0 (socket says 172.21.15.26:52562) 2023-12-19T16:58:02.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f2396ffd640 1 -- 172.21.15.26:0/2182562835 learned_addr learned my addr 172.21.15.26:0/2182562835 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:02.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f23977fe640 1 --2- 172.21.15.26:0/2182562835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2398155030 0x7f239815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f2397fff640 1 --2- 172.21.15.26:0/2182562835 >> v2:172.21.15.160:3300/0 conn(0x7f23980ce740 0x7f23980cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f23967fc640 1 -- 172.21.15.26:0/2182562835 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200139893 0 0) 0x7f239806c970 con 0x7f239815f370 2023-12-19T16:58:02.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f23967fc640 1 -- 172.21.15.26:0/2182562835 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2380003530 con 0x7f239815f370 2023-12-19T16:58:02.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f2397fff640 1 -- 172.21.15.26:0/2182562835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2398155030 msgr2=0x7f239815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:02.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f2397fff640 1 --2- 172.21.15.26:0/2182562835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2398155030 0x7f239815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f2397fff640 1 -- 172.21.15.26:0/2182562835 >> v1:172.21.15.26:6789/0 conn(0x7f239815f370 legacy=0x7f239806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.117+0000 7f2397fff640 1 -- 172.21.15.26:0/2182562835 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239815b1f0 con 0x7f23980ce740 2023-12-19T16:58:02.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.121+0000 7f2397fff640 1 --2- 172.21.15.26:0/2182562835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23980ce740 0x7f23980cebf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2390009d90 tx=0x7f239001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf3e37c63e89d64d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:02.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.121+0000 7f23967fc640 1 -- 172.21.15.26:0/2182562835 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23900a59e0 con 0x7f23980ce740 2023-12-19T16:58:02.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.121+0000 7f23967fc640 1 -- 172.21.15.26:0/2182562835 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23900a5b40 con 0x7f23980ce740 2023-12-19T16:58:02.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.121+0000 7f23967fc640 1 -- 172.21.15.26:0/2182562835 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23900ad440 con 0x7f23980ce740 2023-12-19T16:58:02.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.121+0000 7f239e0f9640 1 -- 172.21.15.26:0/2182562835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23980ce740 msgr2=0x7f23980cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.121+0000 7f239e0f9640 1 --2- 172.21.15.26:0/2182562835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23980ce740 0x7f23980cebf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2390009d90 tx=0x7f239001c920 comp rx=0 tx=0).stop 2023-12-19T16:58:02.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.121+0000 7f239e0f9640 1 -- 172.21.15.26:0/2182562835 shutdown_connections 2023-12-19T16:58:02.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.121+0000 7f239e0f9640 1 --2- 172.21.15.26:0/2182562835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23980ce740 0x7f23980cebf0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.121+0000 7f239e0f9640 1 --2- 172.21.15.26:0/2182562835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2398155030 0x7f239815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.121+0000 7f239e0f9640 1 -- 172.21.15.26:0/2182562835 >> 172.21.15.26:0/2182562835 conn(0x7f23980658e0 msgr2=0x7f2398065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:02.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.121+0000 7f239e0f9640 1 -- 172.21.15.26:0/2182562835 shutdown_connections 2023-12-19T16:58:02.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.121+0000 7f239e0f9640 1 -- 172.21.15.26:0/2182562835 wait complete. 2023-12-19T16:58:02.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.121+0000 7f239e0f9640 1 Processor -- start 2023-12-19T16:58:02.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f239e0f9640 1 -- start start 2023-12-19T16:58:02.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f239e0f9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23980ce740 0x7f2398149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f239e0f9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2398155030 0x7f239813fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f239e0f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239815f370 0x7f239813ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f239e0f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2398138d10 con 0x7f239815f370 2023-12-19T16:58:02.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f239e0f9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23981413d0 con 0x7f2398155030 2023-12-19T16:58:02.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f239e0f9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2398140800 con 0x7f23980ce740 2023-12-19T16:58:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f2397fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239815f370 0x7f239813ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f2397fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239815f370 0x7f239813ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52450/0 (socket says 172.21.15.26:52450) 2023-12-19T16:58:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f2397fff640 1 -- 172.21.15.26:0/1898942862 learned_addr learned my addr 172.21.15.26:0/1898942862 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f23977fe640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23980ce740 0x7f2398149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f2396ffd640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2398155030 0x7f239813fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f2397fff640 1 -- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23980ce740 msgr2=0x7f2398149c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f2397fff640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23980ce740 0x7f2398149c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f2397fff640 1 -- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2398155030 msgr2=0x7f239813fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f2397fff640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2398155030 0x7f239813fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f2397fff640 1 -- 172.21.15.26:0/1898942862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2390009a70 con 0x7f239815f370 2023-12-19T16:58:02.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f23977fe640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23980ce740 0x7f2398149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:02.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f2396ffd640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2398155030 0x7f239813fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:02.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f2397fff640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239815f370 0x7f239813ea40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f239001ce00 tx=0x7f23900196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:02.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f237ffff640 1 -- 172.21.15.26:0/1898942862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23900ada20 con 0x7f239815f370 2023-12-19T16:58:02.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f237ffff640 1 -- 172.21.15.26:0/1898942862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23900adb80 con 0x7f239815f370 2023-12-19T16:58:02.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f237ffff640 1 -- 172.21.15.26:0/1898942862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23900b8420 con 0x7f239815f370 2023-12-19T16:58:02.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f239e0f9640 1 -- 172.21.15.26:0/1898942862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2398140aa0 con 0x7f239815f370 2023-12-19T16:58:02.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.125+0000 7f239e0f9640 1 -- 172.21.15.26:0/1898942862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2398141060 con 0x7f239815f370 2023-12-19T16:58:02.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f237ffff640 1 -- 172.21.15.26:0/1898942862 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f23900b8580 con 0x7f239815f370 2023-12-19T16:58:02.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f237ffff640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2384065b80 0x7f2384068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f237ffff640 1 -- 172.21.15.26:0/1898942862 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f239011ec20 con 0x7f239815f370 2023-12-19T16:58:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f23977fe640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2384065b80 0x7f2384068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f23a1454000 1 Processor -- start 2023-12-19T16:58:02.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f23a1454000 1 -- start start 2023-12-19T16:58:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f23977fe640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2384065b80 0x7f2384068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2388009bb0 tx=0x7f2388008040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f23a1454000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e216728070 0x55e216728450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f23a1454000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728990 0x55e2166ddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f23a1454000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2166de1c0 0x55e2166de5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f23a1454000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e216501c90 con 0x55e216728990 2023-12-19T16:58:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f23a1454000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e216533220 con 0x55e2166de1c0 2023-12-19T16:58:02.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f23a1454000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2165307f0 con 0x55e216728070 2023-12-19T16:58:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f2396ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728990 0x55e2166ddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f2396ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728990 0x55e2166ddc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52454/0 (socket says 172.21.15.26:52454) 2023-12-19T16:58:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f2396ffd640 1 -- 172.21.15.26:0/1872017634 learned_addr learned my addr 172.21.15.26:0/1872017634 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f23977fe640 1 --2- 172.21.15.26:0/1872017634 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e216728070 0x55e216728450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f2397fff640 1 --2- 172.21.15.26:0/1872017634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2166de1c0 0x55e2166de5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f2396ffd640 1 -- 172.21.15.26:0/1872017634 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e216728070 msgr2=0x55e216728450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:02.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f2396ffd640 1 --2- 172.21.15.26:0/1872017634 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e216728070 0x55e216728450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f2396ffd640 1 -- 172.21.15.26:0/1872017634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2166de1c0 msgr2=0x55e2166de5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:02.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f2396ffd640 1 --2- 172.21.15.26:0/1872017634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2166de1c0 0x55e2166de5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.129+0000 7f2396ffd640 1 -- 172.21.15.26:0/1872017634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e216228890 con 0x55e216728990 2023-12-19T16:58:02.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f2396ffd640 1 --2- 172.21.15.26:0/1872017634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728990 0x55e2166ddc80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f238c000f70 tx=0x7f238c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7156768401ee1e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:02.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f237dffb640 1 -- 172.21.15.26:0/1872017634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c005ad0 con 0x55e216728990 2023-12-19T16:58:02.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f237dffb640 1 -- 172.21.15.26:0/1872017634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c005c30 con 0x55e216728990 2023-12-19T16:58:02.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f237dffb640 1 -- 172.21.15.26:0/1872017634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c0070d0 con 0x55e216728990 2023-12-19T16:58:02.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 -- 172.21.15.26:0/1872017634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728990 msgr2=0x55e2166ddc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 --2- 172.21.15.26:0/1872017634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728990 0x55e2166ddc80 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f238c000f70 tx=0x7f238c000bd0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 -- 172.21.15.26:0/1872017634 shutdown_connections 2023-12-19T16:58:02.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 --2- 172.21.15.26:0/1872017634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2166de1c0 0x55e2166de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 --2- 172.21.15.26:0/1872017634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728990 0x55e2166ddc80 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 --2- 172.21.15.26:0/1872017634 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e216728070 0x55e216728450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 -- 172.21.15.26:0/1872017634 >> 172.21.15.26:0/1872017634 conn(0x55e2166e4950 msgr2=0x55e2166dc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:02.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 -- 172.21.15.26:0/1872017634 shutdown_connections 2023-12-19T16:58:02.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 -- 172.21.15.26:0/1872017634 wait complete. 2023-12-19T16:58:02.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 -- start start 2023-12-19T16:58:02.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 Processor -- start 2023-12-19T16:58:02.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2166de1c0 0x55e2166e3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e216728070 0x55e2166ea080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728990 0x55e2166ea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e21652c0e0 con 0x55e216728990 2023-12-19T16:58:02.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2164cb310 con 0x55e2166de1c0 2023-12-19T16:58:02.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2164e9770 con 0x55e216728070 2023-12-19T16:58:02.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23a1454000 10 client.0 _ll_register_callbacks cb 0x55e2166c9b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T16:58:02.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f2397fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728990 0x55e2166ea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f2396ffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e216728070 0x55e2166ea080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f2396ffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e216728070 0x55e2166ea080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:39902/0 (socket says 172.21.15.26:39902) 2023-12-19T16:58:02.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f2396ffd640 1 -- 172.21.15.26:0/2585820571 learned_addr learned my addr 172.21.15.26:0/2585820571 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:02.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f23977fe640 1 --2- 172.21.15.26:0/2585820571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2166de1c0 0x55e2166e3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f2397fff640 1 -- 172.21.15.26:0/2585820571 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e216728070 msgr2=0x55e2166ea080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:02.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f2397fff640 1 --2- 172.21.15.26:0/2585820571 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e216728070 0x55e2166ea080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f2397fff640 1 -- 172.21.15.26:0/2585820571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2166de1c0 msgr2=0x55e2166e3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:02.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f2397fff640 1 --2- 172.21.15.26:0/2585820571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2166de1c0 0x55e2166e3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.133+0000 7f2397fff640 1 -- 172.21.15.26:0/2585820571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238c00f040 con 0x55e216728990 2023-12-19T16:58:02.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f2397fff640 1 --2- 172.21.15.26:0/2585820571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728990 0x55e2166ea5c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f239011cc50 tx=0x7f23900f0510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5493eaa4a3d55e67 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:02.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23767fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23767fc640 1 -- 172.21.15.26:0/2585820571 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23900f7df0 con 0x55e216728990 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 -- 172.21.15.26:0/2585820571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2164fbc10 con 0x55e216728990 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23767fc640 1 -- 172.21.15.26:0/2585820571 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f23900ef300 con 0x55e216728990 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23767fc640 1 -- 172.21.15.26:0/2585820571 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23900ef500 con 0x55e216728990 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 10 client.7122 fetch_fsmap learned FSMap version 41 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 -- 172.21.15.26:0/2585820571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55e2164f1260 con 0x55e216728990 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23767fc640 1 -- 172.21.15.26:0/2585820571 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e2164fbc10 con 0x55e216728990 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23767fc640 1 -- 172.21.15.26:0/2585820571 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 41) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2390005d20 con 0x55e216728990 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 10 client.7122 fetch_fsmap finished waiting for FSMap version 41 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 10 client.7122 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 20 client.7122 populate_metadata read hostname 'smithi026' 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 --2- 172.21.15.26:0/2585820571 >> [v2:172.21.15.26:6833/3421559564,v1:172.21.15.26:6835/3421559564] conn(0x55e21672f8d0 0x55e21672fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 4 client.7122 mds_command: new command op to 6990 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T16:58:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 -- 172.21.15.26:0/2585820571 --> [v2:172.21.15.26:6833/3421559564,v1:172.21.15.26:6835/3421559564] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e21652aa30 con 0x55e21672f8d0 2023-12-19T16:58:02.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23977fe640 1 --2- 172.21.15.26:0/2585820571 >> [v2:172.21.15.26:6833/3421559564,v1:172.21.15.26:6835/3421559564] conn(0x55e21672f8d0 0x55e21672fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23977fe640 1 --2- 172.21.15.26:0/2585820571 >> [v2:172.21.15.26:6833/3421559564,v1:172.21.15.26:6835/3421559564] conn(0x55e21672f8d0 0x55e21672fcb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2da4c40a82274ea server_cookie=271002007ba78103 in_seq=0 out_seq=0 2023-12-19T16:58:02.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23767fc640 10 client.7122 ms_handle_connect on v2:172.21.15.26:6833/3421559564 2023-12-19T16:58:02.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23767fc640 1 -- 172.21.15.26:0/2585820571 <== mds.0 v2:172.21.15.26:6833/3421559564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e21652aa30 con 0x55e21672f8d0 2023-12-19T16:58:02.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23767fc640 10 client.7122 handle_command_reply: tid=0 2023-12-19T16:58:02.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 client.7122 shutdown 2023-12-19T16:58:02.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 -- 172.21.15.26:0/2585820571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728990 msgr2=0x55e2166ea5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 --2- 172.21.15.26:0/2585820571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728990 0x55e2166ea5c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f239011cc50 tx=0x7f23900f0510 comp rx=0 tx=0).stop 2023-12-19T16:58:02.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 -- 172.21.15.26:0/2585820571 shutdown_connections 2023-12-19T16:58:02.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 --2- 172.21.15.26:0/2585820571 >> [v2:172.21.15.26:6833/3421559564,v1:172.21.15.26:6835/3421559564] conn(0x55e21672f8d0 0x55e21672fcb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 --2- 172.21.15.26:0/2585820571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728990 0x55e2166ea5c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 --2- 172.21.15.26:0/2585820571 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e216728070 0x55e2166ea080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 --2- 172.21.15.26:0/2585820571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e2166de1c0 0x55e2166e3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23767fc640 0 client.7122 ms_handle_reset on v2:172.21.15.26:6833/3421559564 2023-12-19T16:58:02.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 -- 172.21.15.26:0/2585820571 >> 172.21.15.26:0/2585820571 conn(0x55e2166e4950 msgr2=0x55e2166dcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:02.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 -- 172.21.15.26:0/2585820571 shutdown_connections 2023-12-19T16:58:02.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 1 -- 172.21.15.26:0/2585820571 wait complete. 2023-12-19T16:58:02.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.137+0000 7f23a1454000 20 client.7122 trim_cache size 0 max 16384 2023-12-19T16:58:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f23a1454000 1 Processor -- start 2023-12-19T16:58:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f23a1454000 1 -- start start 2023-12-19T16:58:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f23a1454000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e2166de1c0 0x55e2167802f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f23a1454000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 0x55e216780830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f23a1454000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e216728990 0x55e216780d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f23a1454000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2161e9940 con 0x55e216728070 2023-12-19T16:58:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f23a1454000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e21656fce0 con 0x55e216728990 2023-12-19T16:58:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f23a1454000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2164f9a80 con 0x55e2166de1c0 2023-12-19T16:58:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f2396ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 0x55e216780830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f2396ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 0x55e216780830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52472/0 (socket says 172.21.15.26:52472) 2023-12-19T16:58:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f2396ffd640 1 -- 172.21.15.26:0/858483515 learned_addr learned my addr 172.21.15.26:0/858483515 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f2397fff640 1 --2- 172.21.15.26:0/858483515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e216728990 0x55e216780d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f2396ffd640 1 -- 172.21.15.26:0/858483515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e2166de1c0 msgr2=0x55e2167802f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f2396ffd640 1 --2- 172.21.15.26:0/858483515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e2166de1c0 0x55e2167802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f2396ffd640 1 -- 172.21.15.26:0/858483515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e216728990 msgr2=0x55e216780d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f2396ffd640 1 --2- 172.21.15.26:0/858483515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e216728990 0x55e216780d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f2396ffd640 1 -- 172.21.15.26:0/858483515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2164d73b0 con 0x55e216728070 2023-12-19T16:58:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f2397fff640 1 --2- 172.21.15.26:0/858483515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e216728990 0x55e216780d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.153+0000 7f2396ffd640 1 --2- 172.21.15.26:0/858483515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 0x55e216780830 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f239815f7d0 tx=0x7f238c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7dd39a7da7772f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f237e7fc640 1 -- 172.21.15.26:0/858483515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c0047f0 con 0x55e216728070 2023-12-19T16:58:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f237e7fc640 1 -- 172.21.15.26:0/858483515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c004950 con 0x55e216728070 2023-12-19T16:58:02.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f237e7fc640 1 -- 172.21.15.26:0/858483515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c004cc0 con 0x55e216728070 2023-12-19T16:58:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 -- 172.21.15.26:0/858483515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 msgr2=0x55e216780830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 --2- 172.21.15.26:0/858483515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 0x55e216780830 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f239815f7d0 tx=0x7f238c00a000 comp rx=0 tx=0).stop 2023-12-19T16:58:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 -- 172.21.15.26:0/858483515 shutdown_connections 2023-12-19T16:58:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 --2- 172.21.15.26:0/858483515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e216728990 0x55e216780d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 --2- 172.21.15.26:0/858483515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 0x55e216780830 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 --2- 172.21.15.26:0/858483515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e2166de1c0 0x55e2167802f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 -- 172.21.15.26:0/858483515 >> 172.21.15.26:0/858483515 conn(0x55e2166e4950 msgr2=0x55e2166e9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 -- 172.21.15.26:0/858483515 shutdown_connections 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 -- 172.21.15.26:0/858483515 wait complete. 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 -- start start 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 Processor -- start 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e2166de1c0 0x55e216785940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 0x55e216785e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e216728990 0x55e21673a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2165211f0 con 0x55e216728070 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e21652af70 con 0x55e216728990 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2164d4860 con 0x55e2166de1c0 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 10 client.0 _ll_register_callbacks cb 0x55e2164da640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f2396ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 0x55e216785e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f2396ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 0x55e216785e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52480/0 (socket says 172.21.15.26:52480) 2023-12-19T16:58:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f2396ffd640 1 -- 172.21.15.26:0/111962990 learned_addr learned my addr 172.21.15.26:0/111962990 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23977fe640 1 --2- 172.21.15.26:0/111962990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e2166de1c0 0x55e216785940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f2397fff640 1 --2- 172.21.15.26:0/111962990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e216728990 0x55e21673a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f2396ffd640 1 -- 172.21.15.26:0/111962990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e2166de1c0 msgr2=0x55e216785940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f2396ffd640 1 --2- 172.21.15.26:0/111962990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e2166de1c0 0x55e216785940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f2396ffd640 1 -- 172.21.15.26:0/111962990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e216728990 msgr2=0x55e21673a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f2396ffd640 1 --2- 172.21.15.26:0/111962990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e216728990 0x55e21673a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f2396ffd640 1 -- 172.21.15.26:0/111962990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e216228890 con 0x55e216728070 2023-12-19T16:58:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f2396ffd640 1 --2- 172.21.15.26:0/111962990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 0x55e216785e80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f238c009010 tx=0x7f238c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d249792a62412dfa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23767fc640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T16:58:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23767fc640 1 -- 172.21.15.26:0/111962990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c0076d0 con 0x55e216728070 2023-12-19T16:58:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23a1454000 1 -- 172.21.15.26:0/111962990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2164f1e00 con 0x55e216728070 2023-12-19T16:58:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.157+0000 7f23767fc640 1 -- 172.21.15.26:0/111962990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f238c032d90 con 0x55e216728070 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23767fc640 1 -- 172.21.15.26:0/111962990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f238c014480 con 0x55e216728070 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 10 client.7128 fetch_fsmap learned FSMap version 41 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 -- 172.21.15.26:0/111962990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x55e216228cc0 con 0x55e216728070 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23767fc640 1 -- 172.21.15.26:0/111962990 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e2164f1e00 con 0x55e216728070 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23767fc640 1 -- 172.21.15.26:0/111962990 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 41) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f238c02f020 con 0x55e216728070 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 10 client.7128 fetch_fsmap finished waiting for FSMap version 41 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 10 client.7128 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 20 client.7128 populate_metadata read hostname 'smithi026' 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 --2- 172.21.15.26:0/111962990 >> [v2:172.21.15.26:6833/3421559564,v1:172.21.15.26:6835/3421559564] conn(0x55e21673a850 0x55e21673ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 4 client.7128 mds_command: new command op to 6990 tid=0[{"prefix": "status"}] 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 -- 172.21.15.26:0/111962990 --> [v2:172.21.15.26:6833/3421559564,v1:172.21.15.26:6835/3421559564] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e216525160 con 0x55e21673a850 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23977fe640 1 --2- 172.21.15.26:0/111962990 >> [v2:172.21.15.26:6833/3421559564,v1:172.21.15.26:6835/3421559564] conn(0x55e21673a850 0x55e21673ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23977fe640 1 --2- 172.21.15.26:0/111962990 >> [v2:172.21.15.26:6833/3421559564,v1:172.21.15.26:6835/3421559564] conn(0x55e21673a850 0x55e21673ac30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14ad097086053b2 server_cookie=fa8144620f5fbd6b in_seq=0 out_seq=0 2023-12-19T16:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23767fc640 10 client.7128 ms_handle_connect on v2:172.21.15.26:6833/3421559564 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23767fc640 1 -- 172.21.15.26:0/111962990 <== mds.0 v2:172.21.15.26:6833/3421559564 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55e216525160 con 0x55e21673a850 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23767fc640 10 client.7128 handle_command_reply: tid=0 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 client.7128 shutdown 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 -- 172.21.15.26:0/111962990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 msgr2=0x55e216785e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 --2- 172.21.15.26:0/111962990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 0x55e216785e80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f238c009010 tx=0x7f238c000ee0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 -- 172.21.15.26:0/111962990 shutdown_connections 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 --2- 172.21.15.26:0/111962990 >> [v2:172.21.15.26:6833/3421559564,v1:172.21.15.26:6835/3421559564] conn(0x55e21673a850 0x55e21673ac30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 --2- 172.21.15.26:0/111962990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e216728990 0x55e21673a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23767fc640 0 client.7128 ms_handle_reset on v2:172.21.15.26:6833/3421559564 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 --2- 172.21.15.26:0/111962990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55e216728070 0x55e216785e80 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 --2- 172.21.15.26:0/111962990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55e2166de1c0 0x55e216785940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 -- 172.21.15.26:0/111962990 >> 172.21.15.26:0/111962990 conn(0x55e2166e4950 msgr2=0x55e216725030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 -- 172.21.15.26:0/111962990 shutdown_connections 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 1 -- 172.21.15.26:0/111962990 wait complete. 2023-12-19T16:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f23a1454000 20 client.7128 trim_cache size 0 max 16384 2023-12-19T16:58:02.166 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T16:58:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T16:58:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T16:58:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6990, 2023-12-19T16:58:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T16:58:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T16:58:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T16:58:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.9719136649999998, 2023-12-19T16:58:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 41, 2023-12-19T16:58:02.166 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 93, 2023-12-19T16:58:02.167 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 93, 2023-12-19T16:58:02.167 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 11.200136457999999 2023-12-19T16:58:02.167 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T16:58:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f239e0f9640 1 -- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2384065b80 msgr2=0x7f2384068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f239e0f9640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2384065b80 0x7f2384068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2388009bb0 tx=0x7f2388008040 comp rx=0 tx=0).stop 2023-12-19T16:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f239e0f9640 1 -- 172.21.15.26:0/1898942862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239815f370 msgr2=0x7f239813ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f239e0f9640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239815f370 0x7f239813ea40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f239001ce00 tx=0x7f23900196c0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f239e0f9640 1 -- 172.21.15.26:0/1898942862 shutdown_connections 2023-12-19T16:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f239e0f9640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f239815f370 0x7f239813ea40 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f239e0f9640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2398155030 0x7f239813fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f239e0f9640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2384065b80 0x7f2384068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f239e0f9640 1 --2- 172.21.15.26:0/1898942862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23980ce740 0x7f2398149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f239e0f9640 1 -- 172.21.15.26:0/1898942862 >> 172.21.15.26:0/1898942862 conn(0x7f23980658e0 msgr2=0x7f23980b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f239e0f9640 1 -- 172.21.15.26:0/1898942862 shutdown_connections 2023-12-19T16:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.161+0000 7f239e0f9640 1 -- 172.21.15.26:0/1898942862 wait complete. 2023-12-19T16:58:02.181 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T16:58:02.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.329+0000 7ff0d688c640 1 Processor -- start 2023-12-19T16:58:02.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.329+0000 7ff0d688c640 1 -- start start 2023-12-19T16:58:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0d688c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0d006cb30 0x7ff0d00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0d688c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d015ddc0 con 0x7ff0d015d2b0 2023-12-19T16:58:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0d688c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d015e0a0 con 0x7ff0d00b34e0 2023-12-19T16:58:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0d688c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d00cb450 con 0x7ff0d006cb30 2023-12-19T16:58:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0cf7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff0d015d2b0 0x7ff0d00b2dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52578/0 (socket says 172.21.15.26:52578) 2023-12-19T16:58:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0cf7fe640 1 -- 172.21.15.26:0/1207064594 learned_addr learned my addr 172.21.15.26:0/1207064594 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0cffff640 1 --2- 172.21.15.26:0/1207064594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0d006cb30 0x7ff0d00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0ceffd640 1 -- 172.21.15.26:0/1207064594 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408892643 0 0) 0x7ff0d015ddc0 con 0x7ff0d015d2b0 2023-12-19T16:58:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0ceffd640 1 -- 172.21.15.26:0/1207064594 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0b8003530 con 0x7ff0d015d2b0 2023-12-19T16:58:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0ceffd640 1 -- 172.21.15.26:0/1207064594 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302641440 0 0) 0x7ff0d015e0a0 con 0x7ff0d00b34e0 2023-12-19T16:58:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0ceffd640 1 -- 172.21.15.26:0/1207064594 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0d015ddc0 con 0x7ff0d00b34e0 2023-12-19T16:58:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0cffff640 1 -- 172.21.15.26:0/1207064594 >> v1:172.21.15.160:6789/0 conn(0x7ff0d00b34e0 legacy=0x7ff0d00ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0cffff640 1 -- 172.21.15.26:0/1207064594 >> v1:172.21.15.26:6789/0 conn(0x7ff0d015d2b0 legacy=0x7ff0d00b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.333+0000 7ff0cffff640 1 -- 172.21.15.26:0/1207064594 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d00cb730 con 0x7ff0d006cb30 2023-12-19T16:58:02.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.337+0000 7ff0cffff640 1 --2- 172.21.15.26:0/1207064594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0d006cb30 0x7ff0d00cbc50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00097f0 tx=0x7ff0c001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e84fecc30d45f9cc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:02.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.337+0000 7ff0ceffd640 1 -- 172.21.15.26:0/1207064594 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00aa640 con 0x7ff0d006cb30 2023-12-19T16:58:02.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.337+0000 7ff0ceffd640 1 -- 172.21.15.26:0/1207064594 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c00aa7a0 con 0x7ff0d006cb30 2023-12-19T16:58:02.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.337+0000 7ff0ceffd640 1 -- 172.21.15.26:0/1207064594 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00aab30 con 0x7ff0d006cb30 2023-12-19T16:58:02.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.337+0000 7ff0d688c640 1 -- 172.21.15.26:0/1207064594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0d006cb30 msgr2=0x7ff0d00cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.337+0000 7ff0d688c640 1 --2- 172.21.15.26:0/1207064594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0d006cb30 0x7ff0d00cbc50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00097f0 tx=0x7ff0c001c4c0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.337+0000 7ff0d688c640 1 -- 172.21.15.26:0/1207064594 shutdown_connections 2023-12-19T16:58:02.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.337+0000 7ff0d688c640 1 --2- 172.21.15.26:0/1207064594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0d006cb30 0x7ff0d00cbc50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.337+0000 7ff0d688c640 1 -- 172.21.15.26:0/1207064594 >> 172.21.15.26:0/1207064594 conn(0x7ff0d00658e0 msgr2=0x7ff0d0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:02.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.337+0000 7ff0d688c640 1 -- 172.21.15.26:0/1207064594 shutdown_connections 2023-12-19T16:58:02.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.337+0000 7ff0d688c640 1 -- 172.21.15.26:0/1207064594 wait complete. 2023-12-19T16:58:02.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.337+0000 7ff0d688c640 1 Processor -- start 2023-12-19T16:58:02.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.337+0000 7ff0d688c640 1 -- start start 2023-12-19T16:58:02.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0d688c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0d006cb30 0x7ff0d0158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0d688c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d00b34e0 0x7ff0d00aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0d688c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0d015d2b0 0x7ff0d006f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0d688c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d0159490 con 0x7ff0d015d2b0 2023-12-19T16:58:02.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0d688c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d0141140 con 0x7ff0d00b34e0 2023-12-19T16:58:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0d688c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d0141420 con 0x7ff0d006cb30 2023-12-19T16:58:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0cffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0d006cb30 0x7ff0d0158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0d4e29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0d015d2b0 0x7ff0d006f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0cffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0d006cb30 0x7ff0d0158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:39942/0 (socket says 172.21.15.26:39942) 2023-12-19T16:58:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0cffff640 1 -- 172.21.15.26:0/4184106311 learned_addr learned my addr 172.21.15.26:0/4184106311 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0cf7fe640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d00b34e0 0x7ff0d00aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0cffff640 1 -- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d00b34e0 msgr2=0x7ff0d00aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0cffff640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d00b34e0 0x7ff0d00aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0cffff640 1 -- 172.21.15.26:0/4184106311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0d015d2b0 msgr2=0x7ff0d006f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0cffff640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0d015d2b0 0x7ff0d006f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0cffff640 1 -- 172.21.15.26:0/4184106311 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c0009420 con 0x7ff0d006cb30 2023-12-19T16:58:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0d4e29640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0d015d2b0 0x7ff0d006f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:02.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0cffff640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0d006cb30 0x7ff0d0158970 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00097f0 tx=0x7ff0c001ad00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:02.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0ccff9640 1 -- 172.21.15.26:0/4184106311 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00be410 con 0x7ff0d006cb30 2023-12-19T16:58:02.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0ccff9640 1 -- 172.21.15.26:0/4184106311 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c00be590 con 0x7ff0d006cb30 2023-12-19T16:58:02.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0ccff9640 1 -- 172.21.15.26:0/4184106311 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00be940 con 0x7ff0d006cb30 2023-12-19T16:58:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0d688c640 1 -- 172.21.15.26:0/4184106311 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d0144da0 con 0x7ff0d006cb30 2023-12-19T16:58:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0d688c640 1 -- 172.21.15.26:0/4184106311 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d0145360 con 0x7ff0d006cb30 2023-12-19T16:58:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.341+0000 7ff0d688c640 1 -- 172.21.15.26:0/4184106311 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d0074270 con 0x7ff0d006cb30 2023-12-19T16:58:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.345+0000 7ff0ccff9640 1 -- 172.21.15.26:0/4184106311 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff0c00a8050 con 0x7ff0d006cb30 2023-12-19T16:58:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.345+0000 7ff0ccff9640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0bc065b80 0x7ff0bc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.345+0000 7ff0ccff9640 1 -- 172.21.15.26:0/4184106311 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7ff0c011d9e0 con 0x7ff0d006cb30 2023-12-19T16:58:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.345+0000 7ff0cf7fe640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0bc065b80 0x7ff0bc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.345+0000 7ff0cf7fe640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0bc065b80 0x7ff0bc068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00b1130 tx=0x7ff0c4007d40 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:02.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.349+0000 7ff0ccff9640 1 -- 172.21.15.26:0/4184106311 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff0c00eaeb0 con 0x7ff0d006cb30 2023-12-19T16:58:02.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.541+0000 7ff0d688c640 1 -- 172.21.15.26:0/4184106311 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff0d00acf60 con 0x7ff0d006cb30 2023-12-19T16:58:02.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.541+0000 7ff0ccff9640 1 -- 172.21.15.26:0/4184106311 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff0c00ee4d0 con 0x7ff0d006cb30 2023-12-19T16:58:02.542 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T16:58:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.541+0000 7ff0d688c640 1 -- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0bc065b80 msgr2=0x7ff0bc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.541+0000 7ff0d688c640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0bc065b80 0x7ff0bc068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00b1130 tx=0x7ff0c4007d40 comp rx=0 tx=0).stop 2023-12-19T16:58:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.541+0000 7ff0d688c640 1 -- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0d006cb30 msgr2=0x7ff0d0158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.541+0000 7ff0d688c640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0d006cb30 0x7ff0d0158970 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00097f0 tx=0x7ff0c001ad00 comp rx=0 tx=0).stop 2023-12-19T16:58:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.541+0000 7ff0d688c640 1 -- 172.21.15.26:0/4184106311 shutdown_connections 2023-12-19T16:58:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.541+0000 7ff0d688c640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0d015d2b0 0x7ff0d006f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.541+0000 7ff0d688c640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0d00b34e0 0x7ff0d00aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.541+0000 7ff0d688c640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff0bc065b80 0x7ff0bc068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.541+0000 7ff0d688c640 1 --2- 172.21.15.26:0/4184106311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0d006cb30 0x7ff0d0158970 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.541+0000 7ff0d688c640 1 -- 172.21.15.26:0/4184106311 >> 172.21.15.26:0/4184106311 conn(0x7ff0d00658e0 msgr2=0x7ff0d0160c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.545+0000 7ff0d688c640 1 -- 172.21.15.26:0/4184106311 shutdown_connections 2023-12-19T16:58:02.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:02.545+0000 7ff0d688c640 1 -- 172.21.15.26:0/4184106311 wait complete. 2023-12-19T16:58:02.563 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-19T16:58:02.564 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T16:58:02.564 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T16:58:02.564 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T16:58:02.564 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T16:58:02.564 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-19T16:58:02.564 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-19T16:58:02.564 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T16:58:02.564 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:58:02.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:02.564 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T16:58:02.575 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T16:58:02.575 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T16:58:02.575 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T16:58:02.575 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:02.575 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T16:58:02.575 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:02.575 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:58:02.575 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T16:58:02.575 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T16:58:02.575 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:58:02.575 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T16:58:02.576 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T16:58:02.576 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T16:58:02.576 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:58:02.576 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T16:58:02.576 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T16:58:02.576 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 41458sec preferred_lft 41458sec 2023-12-19T16:58:02.576 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T16:58:02.576 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:02.576 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:58:02.576 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:58:02.576 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T16:58:02.576 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T16:58:02.576 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T16:58:02.577 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T16:58:02.577 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:02.577 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T16:58:02.577 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T16:58:02.577 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T16:58:02.577 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:02.577 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-19T16:58:02.577 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:02.577 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T16:58:02.577 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T16:58:02.577 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-19T16:58:02.577 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:02.578 INFO:teuthology.orchestra.run.smithi026.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T16:58:02.578 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T16:58:02.578 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-19T16:58:02.578 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:02.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:02.578 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T16:58:02.635 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T16:58:02.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:02.638 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T16:58:02.689 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T16:58:02.689 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T16:58:02.689 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T16:58:02.689 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T16:58:02.689 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T16:58:02.690 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T16:58:02.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:02.690 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:58:02.690 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:58:02.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T16:58:02.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T16:58:02.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T16:58:02.690 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:58:02.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:02.784 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T16:58:02.793 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T16:58:02.794 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T16:58:02.794 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:58:02.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:02.795 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T16:58:02.857 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:58:02.857 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T16:58:02.912 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:58:02.912 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:58:02.912 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T16:58:02.963 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T16:58:02.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:02.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:58:03.060 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:58:03.060 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:58:03.073 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:58:03.073 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:58:03.073 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T16:58:03.074 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T16:58:03.074 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T16:58:03.078 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T16:58:03.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:03.078 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T16:58:03.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:03.100 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:58:03.163 INFO:teuthology.orchestra.run.smithi026.stdout:client7137 2023-12-19T16:58:03.164 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:58:03.164 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:58:03.217 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-19T16:58:03.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.405+0000 7fb530998640 1 Processor -- start 2023-12-19T16:58:03.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.405+0000 7fb530998640 1 -- start start 2023-12-19T16:58:03.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb530998640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb52c06f9d0 0x7fb52c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:03.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb530998640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb52c0c4070 0x7fb52c0cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:03.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb530998640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c0c5b40 con 0x7fb52c15a7c0 2023-12-19T16:58:03.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb530998640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c0c5e20 con 0x7fb52c06f9d0 2023-12-19T16:58:03.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb530998640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0ce770 con 0x7fb52c0c4070 2023-12-19T16:58:03.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb529d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb52c06f9d0 0x7fb52c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:03.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb52a59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb52c15a7c0 0x7fb52c15ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52592/0 (socket says 172.21.15.26:52592) 2023-12-19T16:58:03.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb52a59c640 1 -- 172.21.15.26:0/963173033 learned_addr learned my addr 172.21.15.26:0/963173033 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:03.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb52ad9d640 1 --2- 172.21.15.26:0/963173033 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb52c0c4070 0x7fb52c0cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:03.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb529d9b640 1 -- 172.21.15.26:0/963173033 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb52c0c4070 msgr2=0x7fb52c0cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:03.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb529d9b640 1 --2- 172.21.15.26:0/963173033 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb52c0c4070 0x7fb52c0cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb529d9b640 1 -- 172.21.15.26:0/963173033 >> v1:172.21.15.26:6789/0 conn(0x7fb52c15a7c0 legacy=0x7fb52c15ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:03.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb529d9b640 1 -- 172.21.15.26:0/963173033 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c0cea50 con 0x7fb52c06f9d0 2023-12-19T16:58:03.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.409+0000 7fb52959a640 1 -- 172.21.15.26:0/963173033 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3848898014 0 0) 0x7fb52c0c5b40 con 0x7fb52c15a7c0 2023-12-19T16:58:03.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb529d9b640 1 --2- 172.21.15.26:0/963173033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52c06f9d0 0x7fb52c0ceff0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb5200097c0 tx=0x7fb52001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f6b9b4c25ab033 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:03.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb52959a640 1 -- 172.21.15.26:0/963173033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5200ab670 con 0x7fb52c06f9d0 2023-12-19T16:58:03.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb52959a640 1 -- 172.21.15.26:0/963173033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5200ab7d0 con 0x7fb52c06f9d0 2023-12-19T16:58:03.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb52959a640 1 -- 172.21.15.26:0/963173033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5200abb60 con 0x7fb52c06f9d0 2023-12-19T16:58:03.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb530998640 1 -- 172.21.15.26:0/963173033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52c06f9d0 msgr2=0x7fb52c0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:03.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb530998640 1 --2- 172.21.15.26:0/963173033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52c06f9d0 0x7fb52c0ceff0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb5200097c0 tx=0x7fb52001c450 comp rx=0 tx=0).stop 2023-12-19T16:58:03.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb530998640 1 -- 172.21.15.26:0/963173033 shutdown_connections 2023-12-19T16:58:03.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb530998640 1 --2- 172.21.15.26:0/963173033 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb52c0c4070 0x7fb52c0cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb530998640 1 --2- 172.21.15.26:0/963173033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52c06f9d0 0x7fb52c0ceff0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb530998640 1 -- 172.21.15.26:0/963173033 >> 172.21.15.26:0/963173033 conn(0x7fb52c06d5d0 msgr2=0x7fb52c069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:03.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb530998640 1 -- 172.21.15.26:0/963173033 shutdown_connections 2023-12-19T16:58:03.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb530998640 1 -- 172.21.15.26:0/963173033 wait complete. 2023-12-19T16:58:03.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb530998640 1 Processor -- start 2023-12-19T16:58:03.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.413+0000 7fb530998640 1 -- start start 2023-12-19T16:58:03.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb530998640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52c06f9d0 0x7fb52c15ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:03.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb530998640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb52c0c4070 0x7fb52c147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb530998640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb52c15a7c0 0x7fb52c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb530998640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c15f700 con 0x7fb52c06f9d0 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb530998640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c147ee0 con 0x7fb52c0c4070 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb530998640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c15e000 con 0x7fb52c15a7c0 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb52a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52c06f9d0 0x7fb52c15ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb52a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52c06f9d0 0x7fb52c15ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52512/0 (socket says 172.21.15.26:52512) 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb52a59c640 1 -- 172.21.15.26:0/802040367 learned_addr learned my addr 172.21.15.26:0/802040367 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb52ad9d640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb52c15a7c0 0x7fb52c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb529d9b640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb52c0c4070 0x7fb52c147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb52a59c640 1 -- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb52c15a7c0 msgr2=0x7fb52c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb52a59c640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb52c15a7c0 0x7fb52c1536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb52a59c640 1 -- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb52c0c4070 msgr2=0x7fb52c147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb52a59c640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb52c0c4070 0x7fb52c147740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb52a59c640 1 -- 172.21.15.26:0/802040367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb520009420 con 0x7fb52c06f9d0 2023-12-19T16:58:03.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb52a59c640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52c06f9d0 0x7fb52c15ffd0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00c430 tx=0x7fb51c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:03.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb51affd640 1 -- 172.21.15.26:0/802040367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0074e0 con 0x7fb52c06f9d0 2023-12-19T16:58:03.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb51affd640 1 -- 172.21.15.26:0/802040367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c007640 con 0x7fb52c06f9d0 2023-12-19T16:58:03.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb51affd640 1 -- 172.21.15.26:0/802040367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0059d0 con 0x7fb52c06f9d0 2023-12-19T16:58:03.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb530998640 1 -- 172.21.15.26:0/802040367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c15e2e0 con 0x7fb52c06f9d0 2023-12-19T16:58:03.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb530998640 1 -- 172.21.15.26:0/802040367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c15e8a0 con 0x7fb52c06f9d0 2023-12-19T16:58:03.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.417+0000 7fb530998640 1 -- 172.21.15.26:0/802040367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c15ea90 con 0x7fb52c06f9d0 2023-12-19T16:58:03.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.421+0000 7fb51affd640 1 -- 172.21.15.26:0/802040367 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb51c005b30 con 0x7fb52c06f9d0 2023-12-19T16:58:03.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.421+0000 7fb51affd640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb50c065b80 0x7fb50c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:03.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.421+0000 7fb51affd640 1 -- 172.21.15.26:0/802040367 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fb51c082ba0 con 0x7fb52c06f9d0 2023-12-19T16:58:03.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.421+0000 7fb529d9b640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb50c065b80 0x7fb50c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:03.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.421+0000 7fb529d9b640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb50c065b80 0x7fb50c068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5200af000 tx=0x7fb5200a7000 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:03.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.425+0000 7fb51affd640 1 -- 172.21.15.26:0/802040367 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb51c0502c0 con 0x7fb52c06f9d0 2023-12-19T16:58:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.605+0000 7fb530998640 1 -- 172.21.15.26:0/802040367 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb52c0cda50 con 0x7fb50c065b80 2023-12-19T16:58:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.605+0000 7fb51affd640 1 -- 172.21.15.26:0/802040367 <== mgr.6490 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb5200ab9c0 con 0x7fb50c065b80 2023-12-19T16:58:03.607 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T16:58:03.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:03.608 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2023-12-19T16:58:03.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:03.608 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T16:58:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.609+0000 7fb530998640 1 -- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb50c065b80 msgr2=0x7fb50c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.609+0000 7fb530998640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb50c065b80 0x7fb50c068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5200af000 tx=0x7fb5200a7000 comp rx=0 tx=0).stop 2023-12-19T16:58:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.609+0000 7fb530998640 1 -- 172.21.15.26:0/802040367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52c06f9d0 msgr2=0x7fb52c15ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.609+0000 7fb530998640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52c06f9d0 0x7fb52c15ffd0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00c430 tx=0x7fb51c00c8f0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.609+0000 7fb530998640 1 -- 172.21.15.26:0/802040367 shutdown_connections 2023-12-19T16:58:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.609+0000 7fb530998640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb50c065b80 0x7fb50c068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.609+0000 7fb530998640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb52c15a7c0 0x7fb52c1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.609+0000 7fb530998640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb52c0c4070 0x7fb52c147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.609+0000 7fb530998640 1 --2- 172.21.15.26:0/802040367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52c06f9d0 0x7fb52c15ffd0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.609+0000 7fb530998640 1 -- 172.21.15.26:0/802040367 >> 172.21.15.26:0/802040367 conn(0x7fb52c06d5d0 msgr2=0x7fb52c157020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.609+0000 7fb530998640 1 -- 172.21.15.26:0/802040367 shutdown_connections 2023-12-19T16:58:03.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.609+0000 7fb530998640 1 -- 172.21.15.26:0/802040367 wait complete. 2023-12-19T16:58:03.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-19T16:58:03.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.773+0000 7f61d4066640 1 Processor -- start 2023-12-19T16:58:03.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.773+0000 7f61d4066640 1 -- start start 2023-12-19T16:58:03.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.773+0000 7f61d4066640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc0adc60 0x7f61cc0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:03.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d4066640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f61cc071100 0x7f61cc06cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:03.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d4066640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61cc070260 con 0x7f61cc06d010 2023-12-19T16:58:03.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d4066640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61cc071560 con 0x7f61cc071100 2023-12-19T16:58:03.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d4066640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc1615b0 con 0x7f61cc0adc60 2023-12-19T16:58:03.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d2603640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f61cc06d010 0x7f61cc164ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52596/0 (socket says 172.21.15.26:52596) 2023-12-19T16:58:03.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d2603640 1 -- 172.21.15.26:0/2018169573 learned_addr learned my addr 172.21.15.26:0/2018169573 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:03.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d1e02640 1 --2- 172.21.15.26:0/2018169573 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc0adc60 0x7f61cc0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:03.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d1601640 1 --2- 172.21.15.26:0/2018169573 >> v2:172.21.15.160:3300/0 conn(0x7f61cc071100 0x7f61cc06cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:03.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d0e00640 1 -- 172.21.15.26:0/2018169573 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889650669 0 0) 0x7f61cc070260 con 0x7f61cc06d010 2023-12-19T16:58:03.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d0e00640 1 -- 172.21.15.26:0/2018169573 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b8003530 con 0x7f61cc06d010 2023-12-19T16:58:03.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d1e02640 1 -- 172.21.15.26:0/2018169573 >> v2:172.21.15.160:3300/0 conn(0x7f61cc071100 msgr2=0x7f61cc06cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:03.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d1e02640 1 --2- 172.21.15.26:0/2018169573 >> v2:172.21.15.160:3300/0 conn(0x7f61cc071100 0x7f61cc06cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d1e02640 1 -- 172.21.15.26:0/2018169573 >> v1:172.21.15.26:6789/0 conn(0x7f61cc06d010 legacy=0x7f61cc164ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:03.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.777+0000 7f61d1e02640 1 -- 172.21.15.26:0/2018169573 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc0c8d90 con 0x7f61cc0adc60 2023-12-19T16:58:03.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d1e02640 1 --2- 172.21.15.26:0/2018169573 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc0adc60 0x7f61cc0ca640 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f61c80097c0 tx=0x7f61c801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b34ca4598c44137a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:03.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d0e00640 1 -- 172.21.15.26:0/2018169573 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c80ab600 con 0x7f61cc0adc60 2023-12-19T16:58:03.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d0e00640 1 -- 172.21.15.26:0/2018169573 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c80a7050 con 0x7f61cc0adc60 2023-12-19T16:58:03.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d0e00640 1 -- 172.21.15.26:0/2018169573 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c80ab960 con 0x7f61cc0adc60 2023-12-19T16:58:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d4066640 1 -- 172.21.15.26:0/2018169573 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc0adc60 msgr2=0x7f61cc0ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d4066640 1 --2- 172.21.15.26:0/2018169573 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc0adc60 0x7f61cc0ca640 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f61c80097c0 tx=0x7f61c801c450 comp rx=0 tx=0).stop 2023-12-19T16:58:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d4066640 1 -- 172.21.15.26:0/2018169573 shutdown_connections 2023-12-19T16:58:03.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d4066640 1 --2- 172.21.15.26:0/2018169573 >> v2:172.21.15.160:3300/0 conn(0x7f61cc071100 0x7f61cc06cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d4066640 1 --2- 172.21.15.26:0/2018169573 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc0adc60 0x7f61cc0ca640 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d4066640 1 -- 172.21.15.26:0/2018169573 >> 172.21.15.26:0/2018169573 conn(0x7f61cc066160 msgr2=0x7f61cc0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:03.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d4066640 1 -- 172.21.15.26:0/2018169573 shutdown_connections 2023-12-19T16:58:03.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d4066640 1 -- 172.21.15.26:0/2018169573 wait complete. 2023-12-19T16:58:03.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d4066640 1 Processor -- start 2023-12-19T16:58:03.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.781+0000 7f61d4066640 1 -- start start 2023-12-19T16:58:03.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d4066640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc06d010 0x7f61cc15a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:03.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d4066640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc071100 0x7f61cc0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:03.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d4066640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc0adc60 0x7f61cc0cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:03.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d4066640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc0cdf60 con 0x7f61cc06d010 2023-12-19T16:58:03.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d4066640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc15b8d0 con 0x7f61cc0adc60 2023-12-19T16:58:03.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d4066640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc15acc0 con 0x7f61cc071100 2023-12-19T16:58:03.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d1e02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc06d010 0x7f61cc15a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d1e02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc06d010 0x7f61cc15a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52514/0 (socket says 172.21.15.26:52514) 2023-12-19T16:58:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d1e02640 1 -- 172.21.15.26:0/378102236 learned_addr learned my addr 172.21.15.26:0/378102236 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d2603640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc0adc60 0x7f61cc0cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d1601640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc071100 0x7f61cc0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d1e02640 1 -- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc071100 msgr2=0x7f61cc0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d1e02640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc071100 0x7f61cc0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d1e02640 1 -- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc0adc60 msgr2=0x7f61cc0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d1e02640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc0adc60 0x7f61cc0cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d1e02640 1 -- 172.21.15.26:0/378102236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c8009420 con 0x7f61cc06d010 2023-12-19T16:58:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d1601640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc071100 0x7f61cc0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d1e02640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc06d010 0x7f61cc15a3b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f61c801ab00 tx=0x7f61c80191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61c27fc640 1 -- 172.21.15.26:0/378102236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c80bb070 con 0x7f61cc06d010 2023-12-19T16:58:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61c27fc640 1 -- 172.21.15.26:0/378102236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c80abd10 con 0x7f61cc06d010 2023-12-19T16:58:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61c27fc640 1 -- 172.21.15.26:0/378102236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c80b5630 con 0x7f61cc06d010 2023-12-19T16:58:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d4066640 1 -- 172.21.15.26:0/378102236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61cc15afa0 con 0x7f61cc06d010 2023-12-19T16:58:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d4066640 1 -- 172.21.15.26:0/378102236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61cc15b560 con 0x7f61cc06d010 2023-12-19T16:58:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61d4066640 1 -- 172.21.15.26:0/378102236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61cc074270 con 0x7f61cc06d010 2023-12-19T16:58:03.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.785+0000 7f61c27fc640 1 -- 172.21.15.26:0/378102236 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f61c80b2070 con 0x7f61cc06d010 2023-12-19T16:58:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.789+0000 7f61c27fc640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f61b0065bb0 0x7f61b0068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.789+0000 7f61c27fc640 1 -- 172.21.15.26:0/378102236 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f61c811cee0 con 0x7f61cc06d010 2023-12-19T16:58:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.789+0000 7f61d1601640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f61b0065bb0 0x7f61b0068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:03.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.789+0000 7f61d1601640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f61b0065bb0 0x7f61b0068070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0c1650 tx=0x7f61bc008040 comp rx=0 tx=0).ready entity=mgr.6490 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:03.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.793+0000 7f61c27fc640 1 -- 172.21.15.26:0/378102236 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f61c80ea300 con 0x7f61cc06d010 2023-12-19T16:58:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:03.997+0000 7f61d4066640 1 -- 172.21.15.26:0/378102236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f61cc0ca420 con 0x7f61cc06d010 2023-12-19T16:58:04.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.552+0000 7f61c27fc640 1 -- 172.21.15.26:0/378102236 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 39) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f61c80ea6d0 con 0x7f61cc06d010 2023-12-19T16:58:04.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.556+0000 7f61c27fc640 1 -- 172.21.15.26:0/378102236 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v39) v1 ==== 92+0+0 (secure 0 0 0) 0x7f61c80b8500 con 0x7f61cc06d010 2023-12-19T16:58:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.556+0000 7f61d4066640 1 -- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f61b0065bb0 msgr2=0x7f61b0068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.556+0000 7f61d4066640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f61b0065bb0 0x7f61b0068070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0c1650 tx=0x7f61bc008040 comp rx=0 tx=0).stop 2023-12-19T16:58:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.556+0000 7f61d4066640 1 -- 172.21.15.26:0/378102236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc06d010 msgr2=0x7f61cc15a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.556+0000 7f61d4066640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc06d010 0x7f61cc15a3b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f61c801ab00 tx=0x7f61c80191f0 comp rx=0 tx=0).stop 2023-12-19T16:58:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.556+0000 7f61d4066640 1 -- 172.21.15.26:0/378102236 shutdown_connections 2023-12-19T16:58:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.556+0000 7f61d4066640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc0adc60 0x7f61cc0cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.556+0000 7f61d4066640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f61b0065bb0 0x7f61b0068070 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.556+0000 7f61d4066640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc071100 0x7f61cc0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.556+0000 7f61d4066640 1 --2- 172.21.15.26:0/378102236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc06d010 0x7f61cc15a3b0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.556+0000 7f61d4066640 1 -- 172.21.15.26:0/378102236 >> 172.21.15.26:0/378102236 conn(0x7f61cc066160 msgr2=0x7f61cc160a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:04.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.556+0000 7f61d4066640 1 -- 172.21.15.26:0/378102236 shutdown_connections 2023-12-19T16:58:04.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.556+0000 7f61d4066640 1 -- 172.21.15.26:0/378102236 wait complete. 2023-12-19T16:58:04.581 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-19T16:58:04.732 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:04.726+0000 7feb34c4d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:58:04.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.740+0000 7f7cdc198640 1 Processor -- start 2023-12-19T16:58:04.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.740+0000 7f7cdc198640 1 -- start start 2023-12-19T16:58:04.743 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:04.740+0000 7fca0dbdb180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:58:04.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.740+0000 7f7cdc198640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7cd415ada0 0x7f7cd40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.740+0000 7f7cdc198640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cd4070d90 con 0x7f7cd415a9c0 2023-12-19T16:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.740+0000 7f7cdc198640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cd4071070 con 0x7f7cd406ac80 2023-12-19T16:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.740+0000 7f7cdc198640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cd40cc270 con 0x7f7cd415ada0 2023-12-19T16:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.744+0000 7f7cd9733640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7cd415ada0 0x7f7cd40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.744+0000 7f7cd9733640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7cd415ada0 0x7f7cd40706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52522/0 (socket says 172.21.15.26:52522) 2023-12-19T16:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.744+0000 7f7cd9733640 1 -- 172.21.15.26:0/2670346064 learned_addr learned my addr 172.21.15.26:0/2670346064 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.744+0000 7f7cd9733640 1 -- 172.21.15.26:0/2670346064 >> v1:172.21.15.160:6789/0 conn(0x7f7cd406ac80 legacy=0x7f7cd40b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.744+0000 7f7cd9733640 1 -- 172.21.15.26:0/2670346064 >> v1:172.21.15.26:6789/0 conn(0x7f7cd415a9c0 legacy=0x7f7cd40b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.744+0000 7f7cd9733640 1 -- 172.21.15.26:0/2670346064 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd4071350 con 0x7f7cd415ada0 2023-12-19T16:58:04.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.744+0000 7f7cd9733640 1 --2- 172.21.15.26:0/2670346064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd415ada0 0x7f7cd40706d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009870 tx=0x7f7cc401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a661b66818a0deb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:04.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.744+0000 7f7cd8f32640 1 -- 172.21.15.26:0/2670346064 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40a6040 con 0x7f7cd415ada0 2023-12-19T16:58:04.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.744+0000 7f7cd8f32640 1 -- 172.21.15.26:0/2670346064 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc40aa600 con 0x7f7cd415ada0 2023-12-19T16:58:04.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.744+0000 7f7cd8f32640 1 -- 172.21.15.26:0/2670346064 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40aa990 con 0x7f7cd415ada0 2023-12-19T16:58:04.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 -- 172.21.15.26:0/2670346064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd415ada0 msgr2=0x7f7cd40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:04.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 --2- 172.21.15.26:0/2670346064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd415ada0 0x7f7cd40706d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009870 tx=0x7f7cc401c480 comp rx=0 tx=0).stop 2023-12-19T16:58:04.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 -- 172.21.15.26:0/2670346064 shutdown_connections 2023-12-19T16:58:04.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 --2- 172.21.15.26:0/2670346064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd415ada0 0x7f7cd40706d0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:04.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 -- 172.21.15.26:0/2670346064 >> 172.21.15.26:0/2670346064 conn(0x7f7cd40ce730 msgr2=0x7f7cd40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:04.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 -- 172.21.15.26:0/2670346064 shutdown_connections 2023-12-19T16:58:04.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 -- 172.21.15.26:0/2670346064 wait complete. 2023-12-19T16:58:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 Processor -- start 2023-12-19T16:58:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 -- start start 2023-12-19T16:58:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cd406ac80 0x7f7cd40c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cd415a9c0 0x7f7cd40c1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd415ada0 0x7f7cd414dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40c1840 con 0x7f7cd415ada0 2023-12-19T16:58:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40c3170 con 0x7f7cd415a9c0 2023-12-19T16:58:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cdc198640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd41522a0 con 0x7f7cd406ac80 2023-12-19T16:58:04.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cda735640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd415ada0 0x7f7cd414dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cd9f34640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cd406ac80 0x7f7cd40c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cda735640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd415ada0 0x7f7cd414dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52526/0 (socket says 172.21.15.26:52526) 2023-12-19T16:58:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cda735640 1 -- 172.21.15.26:0/1560960738 learned_addr learned my addr 172.21.15.26:0/1560960738 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.748+0000 7f7cd9733640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cd415a9c0 0x7f7cd40c1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cd9f34640 1 -- 172.21.15.26:0/1560960738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cd415a9c0 msgr2=0x7f7cd40c1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cd9f34640 1 --2- 172.21.15.26:0/1560960738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cd415a9c0 0x7f7cd40c1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cd9f34640 1 -- 172.21.15.26:0/1560960738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd415ada0 msgr2=0x7f7cd414dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cd9f34640 1 --2- 172.21.15.26:0/1560960738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd415ada0 0x7f7cd414dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cd9f34640 1 -- 172.21.15.26:0/1560960738 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc40094d0 con 0x7f7cd406ac80 2023-12-19T16:58:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cda735640 1 --2- 172.21.15.26:0/1560960738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd415ada0 0x7f7cd414dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cd9f34640 1 --2- 172.21.15.26:0/1560960738 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cd406ac80 0x7f7cd40c8cf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0000f70 tx=0x7f7cd000c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cca7fc640 1 -- 172.21.15.26:0/1560960738 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00052d0 con 0x7f7cd406ac80 2023-12-19T16:58:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cca7fc640 1 -- 172.21.15.26:0/1560960738 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd0005430 con 0x7f7cd406ac80 2023-12-19T16:58:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cdc198640 1 -- 172.21.15.26:0/1560960738 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd414ecc0 con 0x7f7cd406ac80 2023-12-19T16:58:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cca7fc640 1 -- 172.21.15.26:0/1560960738 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0003840 con 0x7f7cd406ac80 2023-12-19T16:58:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cdc198640 1 -- 172.21.15.26:0/1560960738 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd40c33a0 con 0x7f7cd406ac80 2023-12-19T16:58:04.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cdc198640 1 -- 172.21.15.26:0/1560960738 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd4074270 con 0x7f7cd406ac80 2023-12-19T16:58:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cca7fc640 1 -- 172.21.15.26:0/1560960738 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 39) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f7cd00039a0 con 0x7f7cd406ac80 2023-12-19T16:58:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cca7fc640 1 --2- 172.21.15.26:0/1560960738 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cb8065bd0 0x7f7cb8068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cca7fc640 1 -- 172.21.15.26:0/1560960738 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f7cd0082b40 con 0x7f7cd406ac80 2023-12-19T16:58:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cd9733640 1 -- 172.21.15.26:0/1560960738 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cb8065bd0 msgr2=0x7f7cb8068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:04.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.752+0000 7f7cd9733640 1 --2- 172.21.15.26:0/1560960738 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cb8065bd0 0x7f7cb8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:04.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.760+0000 7f7cca7fc640 1 -- 172.21.15.26:0/1560960738 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7cd00501b0 con 0x7f7cd406ac80 2023-12-19T16:58:04.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.920+0000 7f7cdc198640 1 -- 172.21.15.26:0/1560960738 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7cd40c1990 con 0x7f7cd406ac80 2023-12-19T16:58:04.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.924+0000 7f7cca7fc640 1 -- 172.21.15.26:0/1560960738 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7cd00170d0 con 0x7f7cd406ac80 2023-12-19T16:58:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.928+0000 7f7cdc198640 1 -- 172.21.15.26:0/1560960738 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cb8065bd0 msgr2=0x7f7cb8068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.928+0000 7f7cdc198640 1 --2- 172.21.15.26:0/1560960738 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cb8065bd0 0x7f7cb8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.928+0000 7f7cdc198640 1 -- 172.21.15.26:0/1560960738 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cd406ac80 msgr2=0x7f7cd40c8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.928+0000 7f7cdc198640 1 --2- 172.21.15.26:0/1560960738 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cd406ac80 0x7f7cd40c8cf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0000f70 tx=0x7f7cd000c890 comp rx=0 tx=0).stop 2023-12-19T16:58:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.928+0000 7f7cdc198640 1 -- 172.21.15.26:0/1560960738 shutdown_connections 2023-12-19T16:58:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.928+0000 7f7cdc198640 1 --2- 172.21.15.26:0/1560960738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7cd415ada0 0x7f7cd414dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.928+0000 7f7cdc198640 1 --2- 172.21.15.26:0/1560960738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7cd415a9c0 0x7f7cd40c1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.928+0000 7f7cdc198640 1 --2- 172.21.15.26:0/1560960738 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cb8065bd0 0x7f7cb8068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.928+0000 7f7cdc198640 1 --2- 172.21.15.26:0/1560960738 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7cd406ac80 0x7f7cd40c8cf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.928+0000 7f7cdc198640 1 -- 172.21.15.26:0/1560960738 >> 172.21.15.26:0/1560960738 conn(0x7f7cd40ce730 msgr2=0x7f7cd40cb890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.928+0000 7f7cdc198640 1 -- 172.21.15.26:0/1560960738 shutdown_connections 2023-12-19T16:58:04.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:04.928+0000 7f7cdc198640 1 -- 172.21.15.26:0/1560960738 wait complete. 2023-12-19T16:58:04.949 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-19T16:58:05.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.092+0000 7fac48a97640 1 Processor -- start 2023-12-19T16:58:05.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.092+0000 7fac48a97640 1 -- start start 2023-12-19T16:58:05.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac48a97640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fac4415cc90 0x7fac440cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:05.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac48a97640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fac4415d070 0x7fac440cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:05.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac48a97640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac44160a20 0x7fac440cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:05.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac48a97640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac440cfe40 con 0x7fac4415cc90 2023-12-19T16:58:05.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac48a97640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac440cdb20 con 0x7fac4415d070 2023-12-19T16:58:05.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac48a97640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac440cde00 con 0x7fac44160a20 2023-12-19T16:58:05.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac437fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fac4415cc90 0x7fac440cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:05.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac437fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fac4415cc90 0x7fac440cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52538/0 (socket says 172.21.15.26:52538) 2023-12-19T16:58:05.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac437fe640 1 -- 172.21.15.26:0/1135088176 learned_addr learned my addr 172.21.15.26:0/1135088176 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:05.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac43fff640 1 --2- 172.21.15.26:0/1135088176 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac44160a20 0x7fac440cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:05.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac42ffd640 1 --2- 172.21.15.26:0/1135088176 >> v2:172.21.15.160:3300/0 conn(0x7fac4415d070 0x7fac440cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:05.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac437fe640 1 -- 172.21.15.26:0/1135088176 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac44160a20 msgr2=0x7fac440cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:05.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac437fe640 1 --2- 172.21.15.26:0/1135088176 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac44160a20 0x7fac440cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac437fe640 1 -- 172.21.15.26:0/1135088176 >> v2:172.21.15.160:3300/0 conn(0x7fac4415d070 msgr2=0x7fac440cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:05.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac437fe640 1 --2- 172.21.15.26:0/1135088176 >> v2:172.21.15.160:3300/0 conn(0x7fac4415d070 0x7fac440cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.096+0000 7fac437fe640 1 -- 172.21.15.26:0/1135088176 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac440ce0e0 con 0x7fac4415cc90 2023-12-19T16:58:05.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac437fe640 1 --2- 172.21.15.26:0/1135088176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4415cc90 0x7fac440cd2f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fac44071890 tx=0x7fac3401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7fa3f6abd0a29d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:05.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac427fc640 1 -- 172.21.15.26:0/1135088176 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac340aa5f0 con 0x7fac4415cc90 2023-12-19T16:58:05.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac427fc640 1 -- 172.21.15.26:0/1135088176 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac340aa750 con 0x7fac4415cc90 2023-12-19T16:58:05.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac427fc640 1 -- 172.21.15.26:0/1135088176 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac340aaae0 con 0x7fac4415cc90 2023-12-19T16:58:05.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac48a97640 1 -- 172.21.15.26:0/1135088176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4415cc90 msgr2=0x7fac440cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:05.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac48a97640 1 --2- 172.21.15.26:0/1135088176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4415cc90 0x7fac440cd2f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fac44071890 tx=0x7fac3401c4d0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac48a97640 1 -- 172.21.15.26:0/1135088176 shutdown_connections 2023-12-19T16:58:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac48a97640 1 --2- 172.21.15.26:0/1135088176 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac44160a20 0x7fac440cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac48a97640 1 --2- 172.21.15.26:0/1135088176 >> v2:172.21.15.160:3300/0 conn(0x7fac4415d070 0x7fac440cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac48a97640 1 --2- 172.21.15.26:0/1135088176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4415cc90 0x7fac440cd2f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac48a97640 1 -- 172.21.15.26:0/1135088176 >> 172.21.15.26:0/1135088176 conn(0x7fac4406a320 msgr2=0x7fac440658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac48a97640 1 -- 172.21.15.26:0/1135088176 shutdown_connections 2023-12-19T16:58:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac48a97640 1 -- 172.21.15.26:0/1135088176 wait complete. 2023-12-19T16:58:05.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac48a97640 1 Processor -- start 2023-12-19T16:58:05.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.100+0000 7fac48a97640 1 -- start start 2023-12-19T16:58:05.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac48a97640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4415cc90 0x7fac44156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac48a97640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac4415d070 0x7fac440d2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac48a97640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac44160a20 0x7fac440d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac48a97640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac440c7640 con 0x7fac4415cc90 2023-12-19T16:58:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac48a97640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac440cc340 con 0x7fac4415d070 2023-12-19T16:58:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac48a97640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac440c69c0 con 0x7fac44160a20 2023-12-19T16:58:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4415cc90 0x7fac44156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac437fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4415cc90 0x7fac44156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52546/0 (socket says 172.21.15.26:52546) 2023-12-19T16:58:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac437fe640 1 -- 172.21.15.26:0/1621843130 learned_addr learned my addr 172.21.15.26:0/1621843130 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac42ffd640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac4415d070 0x7fac440d2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac43fff640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac44160a20 0x7fac440d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac437fe640 1 -- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac44160a20 msgr2=0x7fac440d31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac437fe640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac44160a20 0x7fac440d31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac437fe640 1 -- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac4415d070 msgr2=0x7fac440d2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac437fe640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac4415d070 0x7fac440d2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac437fe640 1 -- 172.21.15.26:0/1621843130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac340094d0 con 0x7fac4415cc90 2023-12-19T16:58:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac43fff640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac44160a20 0x7fac440d31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac437fe640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4415cc90 0x7fac44156150 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fac340ac040 tx=0x7fac3401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac2bfff640 1 -- 172.21.15.26:0/1621843130 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac340b3800 con 0x7fac4415cc90 2023-12-19T16:58:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac2bfff640 1 -- 172.21.15.26:0/1621843130 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac340b3980 con 0x7fac4415cc90 2023-12-19T16:58:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac48a97640 1 -- 172.21.15.26:0/1621843130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac440c6ca0 con 0x7fac4415cc90 2023-12-19T16:58:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac2bfff640 1 -- 172.21.15.26:0/1621843130 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac340b3d30 con 0x7fac4415cc90 2023-12-19T16:58:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac48a97640 1 -- 172.21.15.26:0/1621843130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac440c7260 con 0x7fac4415cc90 2023-12-19T16:58:05.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac48a97640 1 -- 172.21.15.26:0/1621843130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac44061930 con 0x7fac4415cc90 2023-12-19T16:58:05.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.104+0000 7fac2bfff640 1 -- 172.21.15.26:0/1621843130 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 39) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fac340c6390 con 0x7fac4415cc90 2023-12-19T16:58:05.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.108+0000 7fac2bfff640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fac30065bd0 0x7fac30068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.108+0000 7fac2bfff640 1 -- 172.21.15.26:0/1621843130 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fac340a7070 con 0x7fac4415cc90 2023-12-19T16:58:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.108+0000 7fac42ffd640 1 -- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fac30065bd0 msgr2=0x7fac30068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.108+0000 7fac42ffd640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fac30065bd0 0x7fac30068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:05.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.112+0000 7fac2bfff640 1 -- 172.21.15.26:0/1621843130 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fac340eb790 con 0x7fac4415cc90 2023-12-19T16:58:05.149 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:05.146+0000 7feb34c4d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:58:05.190 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:05.188+0000 7fca0dbdb180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:58:05.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.272+0000 7fac48a97640 1 -- 172.21.15.26:0/1621843130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fac440d0970 con 0x7fac4415cc90 2023-12-19T16:58:05.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.276+0000 7fac2bfff640 1 -- 172.21.15.26:0/1621843130 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7fac340eedb0 con 0x7fac4415cc90 2023-12-19T16:58:05.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.280+0000 7fac48a97640 1 -- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fac30065bd0 msgr2=0x7fac30068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:05.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.280+0000 7fac48a97640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fac30065bd0 0x7fac30068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.280+0000 7fac48a97640 1 -- 172.21.15.26:0/1621843130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4415cc90 msgr2=0x7fac44156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.280+0000 7fac48a97640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4415cc90 0x7fac44156150 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fac340ac040 tx=0x7fac3401ad10 comp rx=0 tx=0).stop 2023-12-19T16:58:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.280+0000 7fac48a97640 1 -- 172.21.15.26:0/1621843130 shutdown_connections 2023-12-19T16:58:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.280+0000 7fac48a97640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fac30065bd0 0x7fac30068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.280+0000 7fac48a97640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac44160a20 0x7fac440d31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.280+0000 7fac48a97640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac4415d070 0x7fac440d2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.280+0000 7fac48a97640 1 --2- 172.21.15.26:0/1621843130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4415cc90 0x7fac44156150 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.280+0000 7fac48a97640 1 -- 172.21.15.26:0/1621843130 >> 172.21.15.26:0/1621843130 conn(0x7fac4406a320 msgr2=0x7fac4415b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.280+0000 7fac48a97640 1 -- 172.21.15.26:0/1621843130 shutdown_connections 2023-12-19T16:58:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.280+0000 7fac48a97640 1 -- 172.21.15.26:0/1621843130 wait complete. 2023-12-19T16:58:05.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-12-19T16:58:05.302 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:05.298+0000 7feb34c4d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:58:05.352 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:05.352+0000 7fca0dbdb180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:58:05.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4179a41640 1 Processor -- start 2023-12-19T16:58:05.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4179a41640 1 -- start start 2023-12-19T16:58:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4179a41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41741585f0 0x7f41740cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4179a41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb600 0x7f41740cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4179a41640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41740cba50 con 0x7f41740cb150 2023-12-19T16:58:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4179a41640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41740b47a0 con 0x7f41741585f0 2023-12-19T16:58:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4179a41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417415b1b0 con 0x7f41740cb600 2023-12-19T16:58:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4172ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41741585f0 0x7f41740cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f41727fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f41740cb150 0x7f41740cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52620/0 (socket says 172.21.15.26:52620) 2023-12-19T16:58:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4172ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41741585f0 0x7f41740cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52560/0 (socket says 172.21.15.26:52560) 2023-12-19T16:58:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f41727fc640 1 -- 172.21.15.26:0/256378396 learned_addr learned my addr 172.21.15.26:0/256378396 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f41737fe640 1 --2- 172.21.15.26:0/256378396 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb600 0x7f41740cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:05.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4172ffd640 1 -- 172.21.15.26:0/256378396 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb600 msgr2=0x7f41740cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:05.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4172ffd640 1 --2- 172.21.15.26:0/256378396 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb600 0x7f41740cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4172ffd640 1 -- 172.21.15.26:0/256378396 >> v1:172.21.15.26:6789/0 conn(0x7f41740cb150 legacy=0x7f41740cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:05.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4172ffd640 1 -- 172.21.15.26:0/256378396 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417415b490 con 0x7f41741585f0 2023-12-19T16:58:05.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.448+0000 7f4171ffb640 1 -- 172.21.15.26:0/256378396 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278808500 0 0) 0x7f41740cba50 con 0x7f41740cb150 2023-12-19T16:58:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.452+0000 7f4172ffd640 1 --2- 172.21.15.26:0/256378396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41741585f0 0x7f41740cdce0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f41640097c0 tx=0x7f416401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f2ace83ca35909 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.452+0000 7f4171ffb640 1 -- 172.21.15.26:0/256378396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41640ac620 con 0x7f41741585f0 2023-12-19T16:58:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.452+0000 7f4171ffb640 1 -- 172.21.15.26:0/256378396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41640ac780 con 0x7f41741585f0 2023-12-19T16:58:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.452+0000 7f4171ffb640 1 -- 172.21.15.26:0/256378396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41640acb10 con 0x7f41741585f0 2023-12-19T16:58:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.452+0000 7f4179a41640 1 -- 172.21.15.26:0/256378396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41741585f0 msgr2=0x7f41740cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.452+0000 7f4179a41640 1 --2- 172.21.15.26:0/256378396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41741585f0 0x7f41740cdce0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f41640097c0 tx=0x7f416401c3c0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.452+0000 7f4179a41640 1 -- 172.21.15.26:0/256378396 shutdown_connections 2023-12-19T16:58:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.452+0000 7f4179a41640 1 --2- 172.21.15.26:0/256378396 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb600 0x7f41740cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.452+0000 7f4179a41640 1 --2- 172.21.15.26:0/256378396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41741585f0 0x7f41740cdce0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.452+0000 7f4179a41640 1 -- 172.21.15.26:0/256378396 >> 172.21.15.26:0/256378396 conn(0x7f4174066160 msgr2=0x7f41740b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:05.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.452+0000 7f4179a41640 1 -- 172.21.15.26:0/256378396 shutdown_connections 2023-12-19T16:58:05.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.452+0000 7f4179a41640 1 -- 172.21.15.26:0/256378396 wait complete. 2023-12-19T16:58:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4179a41640 1 Processor -- start 2023-12-19T16:58:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4179a41640 1 -- start start 2023-12-19T16:58:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4179a41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41740cb150 0x7f41740c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4179a41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb600 0x7f41741f99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4179a41640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41741585f0 0x7f41740af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4179a41640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41741f9f30 con 0x7f41740cb150 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4179a41640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41740c8230 con 0x7f41741585f0 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4179a41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41740c6b80 con 0x7f41740cb600 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4172ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41740cb150 0x7f41740c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4172ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41740cb150 0x7f41740c6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52564/0 (socket says 172.21.15.26:52564) 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4172ffd640 1 -- 172.21.15.26:0/1531108431 learned_addr learned my addr 172.21.15.26:0/1531108431 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f41727fc640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb600 0x7f41741f99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f41737fe640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41741585f0 0x7f41740af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4172ffd640 1 -- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb600 msgr2=0x7f41741f99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4172ffd640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb600 0x7f41741f99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4172ffd640 1 -- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41741585f0 msgr2=0x7f41740af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4172ffd640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41741585f0 0x7f41740af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4172ffd640 1 -- 172.21.15.26:0/1531108431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4164009420 con 0x7f41740cb150 2023-12-19T16:58:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4172ffd640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41740cb150 0x7f41740c6530 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4164019ff0 tx=0x7f4164019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f415bfff640 1 -- 172.21.15.26:0/1531108431 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41640b5410 con 0x7f41740cb150 2023-12-19T16:58:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f415bfff640 1 -- 172.21.15.26:0/1531108431 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41640b5590 con 0x7f41740cb150 2023-12-19T16:58:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4179a41640 1 -- 172.21.15.26:0/1531108431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417415c3e0 con 0x7f41740cb150 2023-12-19T16:58:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f415bfff640 1 -- 172.21.15.26:0/1531108431 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41640b5940 con 0x7f41740cb150 2023-12-19T16:58:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.456+0000 7f4179a41640 1 -- 172.21.15.26:0/1531108431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41740c7140 con 0x7f41740cb150 2023-12-19T16:58:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.460+0000 7f4179a41640 1 -- 172.21.15.26:0/1531108431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4174074270 con 0x7f41740cb150 2023-12-19T16:58:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.460+0000 7f415bfff640 1 -- 172.21.15.26:0/1531108431 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 39) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f41640b3040 con 0x7f41740cb150 2023-12-19T16:58:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.460+0000 7f415bfff640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 0x7f4160068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.460+0000 7f415bfff640 1 -- 172.21.15.26:0/1531108431 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f416411d840 con 0x7f41740cb150 2023-12-19T16:58:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.460+0000 7f41727fc640 1 -- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 msgr2=0x7f4160068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.460+0000 7f41727fc640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 0x7f4160068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.464+0000 7f415bfff640 1 -- 172.21.15.26:0/1531108431 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f41640eada0 con 0x7f41740cb150 2023-12-19T16:58:05.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.644+0000 7f4179a41640 1 -- 172.21.15.26:0/1531108431 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f417415c260 con 0x7f4160065bd0 2023-12-19T16:58:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.664+0000 7f41727fc640 1 -- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 msgr2=0x7f4160068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:05.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:05.664+0000 7f41727fc640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 0x7f4160068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T16:58:05.781 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:05.778+0000 7feb34c4d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:58:05.839 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:05.834+0000 7feb34c4d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:58:05.866 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:05.864+0000 7fca0dbdb180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:58:05.895 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:05.890+0000 7feb34c4d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:58:05.928 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:05.928+0000 7fca0dbdb180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:58:05.981 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:05.978+0000 7feb34c4d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:58:05.987 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:05.984+0000 7fca0dbdb180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:58:06.038 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:06.034+0000 7feb34c4d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:58:06.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:06.068+0000 7f41727fc640 1 -- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 msgr2=0x7f4160068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:06.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:06.068+0000 7f41727fc640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 0x7f4160068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-19T16:58:06.079 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:06.076+0000 7fca0dbdb180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:58:06.122 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:06.118+0000 7feb34c4d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:58:06.138 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:06.136+0000 7fca0dbdb180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:58:06.177 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:06.174+0000 7feb34c4d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:58:06.227 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:06.224+0000 7fca0dbdb180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:58:06.285 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:06.284+0000 7fca0dbdb180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:58:06.319 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:06.314+0000 7feb34c4d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:58:06.376 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:06.370+0000 7feb34c4d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:58:06.436 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:06.436+0000 7fca0dbdb180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:58:06.497 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:06.496+0000 7fca0dbdb180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:58:06.615 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:58:06.616 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:58:06.616 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:58:06.624 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:06.618+0000 7feb34c4d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:58:06.748 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:58:06.748 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:58:06.748 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:58:06.756 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:06.756+0000 7fca0dbdb180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:58:06.758 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:06.754+0000 7feb34c4d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:58:06.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:06.872+0000 7f41727fc640 1 -- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 msgr2=0x7f4160068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:06.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:06.872+0000 7f41727fc640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 0x7f4160068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-19T16:58:06.881 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:06.878+0000 7feb34c4d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:58:06.898 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:06.896+0000 7fca0dbdb180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:58:06.952 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:06.946+0000 7feb34c4d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:58:07.030 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:07.028+0000 7fca0dbdb180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:58:07.088 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:07.082+0000 7feb34c4d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:58:07.105 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:07.104+0000 7fca0dbdb180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:58:07.247 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:07.244+0000 7fca0dbdb180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:58:07.628 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:07.622+0000 7feb34c4d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:58:07.712 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:07.706+0000 7feb34c4d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:58:07.821 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:07.820+0000 7fca0dbdb180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:58:07.862 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:07.858+0000 7feb34c4d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:58:07.909 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:07.908+0000 7fca0dbdb180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:58:07.945 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:07.942+0000 7feb34c4d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:58:08.066 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:08.064+0000 7fca0dbdb180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:58:08.153 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:08.152+0000 7fca0dbdb180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:58:08.260 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:08.254+0000 7feb34c4d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:58:08.320 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:08.314+0000 7feb34c4d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:58:08.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:08.332+0000 7f415bfff640 1 -- 172.21.15.26:0/1531108431 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 40) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f41640aa070 con 0x7f41740cb150 2023-12-19T16:58:08.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:08.332+0000 7f415bfff640 1 -- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 msgr2=0x7f4160068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:08.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:08.332+0000 7f415bfff640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 0x7f4160068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:08.453 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:08.450+0000 7feab7060640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:08.454 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:08.450+0000 7feab7060640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:08.454 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:08.450+0000 7feab7060640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:08.454 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:08.450+0000 7feab7060640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:08.454 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:08.450+0000 7feab7060640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:08.454 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:08.450+0000 7feaba066640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:08.454 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:08.450+0000 7feaba066640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:08.454 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:08.450+0000 7feaba066640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:08.454 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:08.450+0000 7feaba066640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:08.454 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:08.450+0000 7feaba066640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:08.500 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:08.496+0000 7fca0dbdb180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:58:08.565 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:08.564+0000 7fca0dbdb180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:58:09.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.336+0000 7f415bfff640 1 -- 172.21.15.26:0/1531108431 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 41) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f41640eb170 con 0x7f41740cb150 2023-12-19T16:58:09.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.336+0000 7f415bfff640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 0x7f4160068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:09.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.336+0000 7f415bfff640 1 -- 172.21.15.26:0/1531108431 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f416006bbd0 con 0x7f4160065bd0 2023-12-19T16:58:09.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.340+0000 7f41727fc640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 0x7f4160068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:09.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.344+0000 7f41727fc640 1 --2- 172.21.15.26:0/1531108431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4160065bd0 0x7f4160068090 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f41740c5a40 tx=0x7f4168007df0 comp rx=0 tx=0).ready entity=mgr.6251 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:09.355 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:09.350+0000 7feac1074640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-12-19T16:58:09.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.352+0000 7f415bfff640 1 -- 172.21.15.26:0/1531108431 <== mgr.6251 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f416006bbd0 con 0x7f4160065bd0 2023-12-19T16:58:09.356 INFO:teuthology.orchestra.run.smithi026.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-12-19T16:58:09.365 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-12-19T16:58:09.366 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-19T16:58:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.512+0000 7f183152c640 1 Processor -- start 2023-12-19T16:58:09.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.512+0000 7f183152c640 1 -- start start 2023-12-19T16:58:09.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f183152c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f182c1585f0 0x7f182c0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:09.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f183152c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f182c0cb150 0x7f182c0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f183152c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c15b1b0 con 0x7f182c0cc310 2023-12-19T16:58:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f183152c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c0cbb40 con 0x7f182c0cb150 2023-12-19T16:58:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f183152c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c0b47a0 con 0x7f182c1585f0 2023-12-19T16:58:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f1823fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f182c0cb150 0x7f182c0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f1823fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f182c0cb150 0x7f182c0cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46438/0 (socket says 172.21.15.26:46438) 2023-12-19T16:58:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f1823fff640 1 -- 172.21.15.26:0/243773237 learned_addr learned my addr 172.21.15.26:0/243773237 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f182b7fe640 1 --1- 172.21.15.26:0/243773237 >> v1:172.21.15.26:6789/0 conn(0x7f182c0cc310 0x7f182c0cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39760/0 (socket says 172.21.15.26:39760) 2023-12-19T16:58:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f182affd640 1 --2- 172.21.15.26:0/243773237 >> v2:172.21.15.160:3300/0 conn(0x7f182c1585f0 0x7f182c0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f1823fff640 1 -- 172.21.15.26:0/243773237 >> v2:172.21.15.160:3300/0 conn(0x7f182c1585f0 msgr2=0x7f182c0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f1823fff640 1 --2- 172.21.15.26:0/243773237 >> v2:172.21.15.160:3300/0 conn(0x7f182c1585f0 0x7f182c0cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f1823fff640 1 -- 172.21.15.26:0/243773237 >> v1:172.21.15.26:6789/0 conn(0x7f182c0cc310 legacy=0x7f182c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:09.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.516+0000 7f1823fff640 1 -- 172.21.15.26:0/243773237 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c15b490 con 0x7f182c0cb150 2023-12-19T16:58:09.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f1823fff640 1 --2- 172.21.15.26:0/243773237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c0cb150 0x7f182c0cb600 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f182c0cc2c0 tx=0x7f181401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25d486deb0d223 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:09.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f182a7fc640 1 -- 172.21.15.26:0/243773237 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18140ab640 con 0x7f182c0cb150 2023-12-19T16:58:09.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f182a7fc640 1 -- 172.21.15.26:0/243773237 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18140a7070 con 0x7f182c0cb150 2023-12-19T16:58:09.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f182a7fc640 1 -- 172.21.15.26:0/243773237 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18140ab9c0 con 0x7f182c0cb150 2023-12-19T16:58:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f183152c640 1 -- 172.21.15.26:0/243773237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c0cb150 msgr2=0x7f182c0cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f183152c640 1 --2- 172.21.15.26:0/243773237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c0cb150 0x7f182c0cb600 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f182c0cc2c0 tx=0x7f181401c440 comp rx=0 tx=0).stop 2023-12-19T16:58:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f183152c640 1 -- 172.21.15.26:0/243773237 shutdown_connections 2023-12-19T16:58:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f183152c640 1 --2- 172.21.15.26:0/243773237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c0cb150 0x7f182c0cb600 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f183152c640 1 --2- 172.21.15.26:0/243773237 >> v2:172.21.15.160:3300/0 conn(0x7f182c1585f0 0x7f182c0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f183152c640 1 -- 172.21.15.26:0/243773237 >> 172.21.15.26:0/243773237 conn(0x7f182c066160 msgr2=0x7f182c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f183152c640 1 -- 172.21.15.26:0/243773237 shutdown_connections 2023-12-19T16:58:09.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f183152c640 1 -- 172.21.15.26:0/243773237 wait complete. 2023-12-19T16:58:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f183152c640 1 Processor -- start 2023-12-19T16:58:09.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.520+0000 7f183152c640 1 -- start start 2023-12-19T16:58:09.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f183152c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182c0cb150 0x7f182c0b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f183152c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f182c0cc310 0x7f182c0ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f183152c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c1585f0 0x7f182c150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f183152c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0aec00 con 0x7f182c1585f0 2023-12-19T16:58:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f183152c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c150540 con 0x7f182c0cc310 2023-12-19T16:58:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f183152c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0aee10 con 0x7f182c0cb150 2023-12-19T16:58:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f182b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c1585f0 0x7f182c150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:09.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f182b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c1585f0 0x7f182c150000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46444/0 (socket says 172.21.15.26:46444) 2023-12-19T16:58:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f182b7fe640 1 -- 172.21.15.26:0/3940436161 learned_addr learned my addr 172.21.15.26:0/3940436161 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:09.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f1823fff640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f182c0cc310 0x7f182c0ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f182affd640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182c0cb150 0x7f182c0b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f182b7fe640 1 -- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182c0cb150 msgr2=0x7f182c0b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f182b7fe640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182c0cb150 0x7f182c0b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f182b7fe640 1 -- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f182c0cc310 msgr2=0x7f182c0ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f182b7fe640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f182c0cc310 0x7f182c0ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f182b7fe640 1 -- 172.21.15.26:0/3940436161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18140094d0 con 0x7f182c1585f0 2023-12-19T16:58:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f182affd640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182c0cb150 0x7f182c0b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f182b7fe640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c1585f0 0x7f182c150000 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f182400c700 tx=0x7f182400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f18237fe640 1 -- 172.21.15.26:0/3940436161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182400ce10 con 0x7f182c1585f0 2023-12-19T16:58:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f18237fe640 1 -- 172.21.15.26:0/3940436161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182400a640 con 0x7f182c1585f0 2023-12-19T16:58:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f18237fe640 1 -- 172.21.15.26:0/3940436161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182400a9b0 con 0x7f182c1585f0 2023-12-19T16:58:09.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f183152c640 1 -- 172.21.15.26:0/3940436161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c15c3e0 con 0x7f182c1585f0 2023-12-19T16:58:09.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f183152c640 1 -- 172.21.15.26:0/3940436161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c0af3d0 con 0x7f182c1585f0 2023-12-19T16:58:09.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f183152c640 1 -- 172.21.15.26:0/3940436161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c074270 con 0x7f182c1585f0 2023-12-19T16:58:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.524+0000 7f18237fe640 1 -- 172.21.15.26:0/3940436161 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 41) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f182400ab10 con 0x7f182c1585f0 2023-12-19T16:58:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.528+0000 7f18237fe640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f180c065930 0x7f180c067df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.528+0000 7f18237fe640 1 -- 172.21.15.26:0/3940436161 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f1824082980 con 0x7f182c1585f0 2023-12-19T16:58:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.528+0000 7f182affd640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f180c065930 0x7f180c067df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.528+0000 7f182affd640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f180c065930 0x7f180c067df0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f181c007a70 tx=0x7f181c008040 comp rx=0 tx=0).ready entity=mgr.6251 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:09.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.532+0000 7f18237fe640 1 -- 172.21.15.26:0/3940436161 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f18240502f0 con 0x7f182c1585f0 2023-12-19T16:58:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:09.728+0000 7f183152c640 1 -- 172.21.15.26:0/3940436161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f182c0c2050 con 0x7f182c1585f0 2023-12-19T16:58:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.548+0000 7f18237fe640 1 -- 172.21.15.26:0/3940436161 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 42) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f18240506c0 con 0x7f182c1585f0 2023-12-19T16:58:10.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.552+0000 7f18237fe640 1 -- 172.21.15.26:0/3940436161 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v42) v1 ==== 93+0+0 (secure 0 0 0) 0x7f182406e420 con 0x7f182c1585f0 2023-12-19T16:58:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.552+0000 7f183152c640 1 -- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f180c065930 msgr2=0x7f180c067df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.552+0000 7f183152c640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f180c065930 0x7f180c067df0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f181c007a70 tx=0x7f181c008040 comp rx=0 tx=0).stop 2023-12-19T16:58:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.552+0000 7f183152c640 1 -- 172.21.15.26:0/3940436161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c1585f0 msgr2=0x7f182c150000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.552+0000 7f183152c640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c1585f0 0x7f182c150000 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f182400c700 tx=0x7f182400cbc0 comp rx=0 tx=0).stop 2023-12-19T16:58:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.552+0000 7f183152c640 1 -- 172.21.15.26:0/3940436161 shutdown_connections 2023-12-19T16:58:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.552+0000 7f183152c640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182c1585f0 0x7f182c150000 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.552+0000 7f183152c640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f182c0cc310 0x7f182c0ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.552+0000 7f183152c640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f180c065930 0x7f180c067df0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.552+0000 7f183152c640 1 --2- 172.21.15.26:0/3940436161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182c0cb150 0x7f182c0b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.552+0000 7f183152c640 1 -- 172.21.15.26:0/3940436161 >> 172.21.15.26:0/3940436161 conn(0x7f182c066160 msgr2=0x7f182c070980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.552+0000 7f183152c640 1 -- 172.21.15.26:0/3940436161 shutdown_connections 2023-12-19T16:58:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.552+0000 7f183152c640 1 -- 172.21.15.26:0/3940436161 wait complete. 2023-12-19T16:58:10.578 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T16:58:10.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:58:10.662 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:58:10.712 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T16:58:10.712 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:58:10.712 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:58:10.722 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:58:10.722 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:58:10.730 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:10.728+0000 7fce4dde4180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:58:10.733 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:10.730+0000 7f377dfdb180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:58:10.740 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:58:10.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f412d2c3640 1 Processor -- start 2023-12-19T16:58:10.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f412d2c3640 1 -- start start 2023-12-19T16:58:10.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f412d2c3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f41280b2030 0x7f4128071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:10.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f412d2c3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412815e7c0 con 0x7f41280affe0 2023-12-19T16:58:10.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f412d2c3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41280b04c0 con 0x7f41280b3960 2023-12-19T16:58:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f412d2c3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41280c97a0 con 0x7f41280b2030 2023-12-19T16:58:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f41277fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f41280affe0 0x7f41280c9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39762/0 (socket says 172.21.15.26:39762) 2023-12-19T16:58:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f41277fe640 1 -- 172.21.15.26:0/2424990539 learned_addr learned my addr 172.21.15.26:0/2424990539 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f4127fff640 1 --2- 172.21.15.26:0/2424990539 >> v2:172.21.15.160:3300/0 conn(0x7f41280b2030 0x7f4128071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f4126ffd640 1 -- 172.21.15.26:0/2424990539 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482656466 0 0) 0x7f412815e7c0 con 0x7f41280affe0 2023-12-19T16:58:10.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f4126ffd640 1 -- 172.21.15.26:0/2424990539 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4110003530 con 0x7f41280affe0 2023-12-19T16:58:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f4127fff640 1 -- 172.21.15.26:0/2424990539 >> v1:172.21.15.160:6789/0 conn(0x7f41280b3960 legacy=0x7f41280b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f4127fff640 1 -- 172.21.15.26:0/2424990539 >> v1:172.21.15.26:6789/0 conn(0x7f41280affe0 legacy=0x7f41280c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:10.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.928+0000 7f4127fff640 1 -- 172.21.15.26:0/2424990539 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41280b07a0 con 0x7f41280b2030 2023-12-19T16:58:10.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.932+0000 7f4127fff640 1 --2- 172.21.15.26:0/2424990539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41280b2030 0x7f4128071e20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f41180097c0 tx=0x7f411801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a56ff51a9ccc200c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.932+0000 7f4126ffd640 1 -- 172.21.15.26:0/2424990539 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41180ab600 con 0x7f41280b2030 2023-12-19T16:58:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.932+0000 7f4126ffd640 1 -- 172.21.15.26:0/2424990539 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41180a7050 con 0x7f41280b2030 2023-12-19T16:58:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.932+0000 7f4126ffd640 1 -- 172.21.15.26:0/2424990539 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41180ab980 con 0x7f41280b2030 2023-12-19T16:58:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.932+0000 7f412d2c3640 1 -- 172.21.15.26:0/2424990539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41280b2030 msgr2=0x7f4128071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:10.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.932+0000 7f412d2c3640 1 --2- 172.21.15.26:0/2424990539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41280b2030 0x7f4128071e20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f41180097c0 tx=0x7f411801c480 comp rx=0 tx=0).stop 2023-12-19T16:58:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.932+0000 7f412d2c3640 1 -- 172.21.15.26:0/2424990539 shutdown_connections 2023-12-19T16:58:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.932+0000 7f412d2c3640 1 --2- 172.21.15.26:0/2424990539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41280b2030 0x7f4128071e20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.932+0000 7f412d2c3640 1 -- 172.21.15.26:0/2424990539 >> 172.21.15.26:0/2424990539 conn(0x7f412806b0f0 msgr2=0x7f412806b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.932+0000 7f412d2c3640 1 -- 172.21.15.26:0/2424990539 shutdown_connections 2023-12-19T16:58:10.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.932+0000 7f412d2c3640 1 -- 172.21.15.26:0/2424990539 wait complete. 2023-12-19T16:58:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412d2c3640 1 Processor -- start 2023-12-19T16:58:10.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412d2c3640 1 -- start start 2023-12-19T16:58:10.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412d2c3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41280affe0 0x7f4128153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412d2c3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41280b2030 0x7f41280aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412d2c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41280b3960 0x7f41280c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412d2c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4128151280 con 0x7f41280b3960 2023-12-19T16:58:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412d2c3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4128150c00 con 0x7f41280affe0 2023-12-19T16:58:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412d2c3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41280c8090 con 0x7f41280b2030 2023-12-19T16:58:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412cac2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41280b3960 0x7f41280c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412cac2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41280b3960 0x7f41280c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46458/0 (socket says 172.21.15.26:46458) 2023-12-19T16:58:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412cac2640 1 -- 172.21.15.26:0/4262645664 learned_addr learned my addr 172.21.15.26:0/4262645664 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f4127fff640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41280affe0 0x7f4128153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:10.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f41277fe640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41280b2030 0x7f41280aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f4127fff640 1 -- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41280b2030 msgr2=0x7f41280aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:10.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f4127fff640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41280b2030 0x7f41280aec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f4127fff640 1 -- 172.21.15.26:0/4262645664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41280b3960 msgr2=0x7f41280c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f4127fff640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41280b3960 0x7f41280c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f4127fff640 1 -- 172.21.15.26:0/4262645664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4118009420 con 0x7f41280affe0 2023-12-19T16:58:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412cac2640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41280b3960 0x7f41280c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f4127fff640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41280affe0 0x7f4128153330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f412806cc30 tx=0x7f4118019220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f4124ff9640 1 -- 172.21.15.26:0/4262645664 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41180b4840 con 0x7f41280affe0 2023-12-19T16:58:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f4124ff9640 1 -- 172.21.15.26:0/4262645664 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41180b49a0 con 0x7f41280affe0 2023-12-19T16:58:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f4124ff9640 1 -- 172.21.15.26:0/4262645664 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41180b4d50 con 0x7f41280affe0 2023-12-19T16:58:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412d2c3640 1 -- 172.21.15.26:0/4262645664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4128150ea0 con 0x7f41280affe0 2023-12-19T16:58:10.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.936+0000 7f412d2c3640 1 -- 172.21.15.26:0/4262645664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412814b580 con 0x7f41280affe0 2023-12-19T16:58:10.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.940+0000 7f412d2c3640 1 -- 172.21.15.26:0/4262645664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4128150a60 con 0x7f41280affe0 2023-12-19T16:58:10.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.940+0000 7f4124ff9640 1 -- 172.21.15.26:0/4262645664 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 42) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f41180b2040 con 0x7f41280affe0 2023-12-19T16:58:10.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.940+0000 7f4124ff9640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4114065b80 0x7f4114068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:10.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.940+0000 7f4124ff9640 1 -- 172.21.15.26:0/4262645664 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f411811d900 con 0x7f41280affe0 2023-12-19T16:58:10.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.940+0000 7f41277fe640 1 -- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4114065b80 msgr2=0x7f4114068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:10.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.940+0000 7f41277fe640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4114065b80 0x7f4114068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:10.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:10.944+0000 7f4124ff9640 1 -- 172.21.15.26:0/4262645664 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4128150a60 con 0x7f41280affe0 2023-12-19T16:58:11.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.104+0000 7f412d2c3640 1 -- 172.21.15.26:0/4262645664 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41281513c0 con 0x7f41280affe0 2023-12-19T16:58:11.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.108+0000 7f4124ff9640 1 -- 172.21.15.26:0/4262645664 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+17701 (secure 0 0 0) 0x7f41281513c0 con 0x7f41280affe0 2023-12-19T16:58:11.108 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:11.108 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:58:08.323236+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:53.833233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:55.184912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6833/38098":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/2084078801":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/385148557":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/3582055562":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/99434118":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/3106091363":"2023-12-20T16:58:08.323203+0000","172.21.15.160:6832/38098":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/3155542875":"2023-12-20T16:58:08.323203+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:58:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.108+0000 7f412d2c3640 1 -- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4114065b80 msgr2=0x7f4114068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:11.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.108+0000 7f412d2c3640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4114065b80 0x7f4114068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.108+0000 7f412d2c3640 1 -- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41280affe0 msgr2=0x7f4128153330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:11.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.108+0000 7f412d2c3640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41280affe0 0x7f4128153330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f412806cc30 tx=0x7f4118019220 comp rx=0 tx=0).stop 2023-12-19T16:58:11.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.108+0000 7f412d2c3640 1 -- 172.21.15.26:0/4262645664 shutdown_connections 2023-12-19T16:58:11.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.108+0000 7f412d2c3640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41280b3960 0x7f41280c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.108+0000 7f412d2c3640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4114065b80 0x7f4114068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.108+0000 7f412d2c3640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41280b2030 0x7f41280aec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.108+0000 7f412d2c3640 1 --2- 172.21.15.26:0/4262645664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41280affe0 0x7f4128153330 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.108+0000 7f412d2c3640 1 -- 172.21.15.26:0/4262645664 >> 172.21.15.26:0/4262645664 conn(0x7f412806b0f0 msgr2=0x7f412806f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:11.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.108+0000 7f412d2c3640 1 -- 172.21.15.26:0/4262645664 shutdown_connections 2023-12-19T16:58:11.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.108+0000 7f412d2c3640 1 -- 172.21.15.26:0/4262645664 wait complete. 2023-12-19T16:58:11.131 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:57:53.833233+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:57:55.184912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:58:11.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:58:11.153 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:11.150+0000 7f377dfdb180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:58:11.177 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:11.176+0000 7fce4dde4180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:58:11.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f4926095640 1 Processor -- start 2023-12-19T16:58:11.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f4926095640 1 -- start start 2023-12-19T16:58:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f4926095640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4920159820 0x7f492015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f4926095640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f492006f6c0 0x7f49200c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f4926095640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492006fb60 con 0x7f4920159c00 2023-12-19T16:58:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f4926095640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492015e470 con 0x7f492006f6c0 2023-12-19T16:58:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f4926095640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492006fe40 con 0x7f4920159820 2023-12-19T16:58:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f491ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4920159c00 0x7f4920070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39774/0 (socket says 172.21.15.26:39774) 2023-12-19T16:58:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f491ffff640 1 -- 172.21.15.26:0/1403337165 learned_addr learned my addr 172.21.15.26:0/1403337165 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f491f7fe640 1 --2- 172.21.15.26:0/1403337165 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4920159820 0x7f492015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:11.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f491effd640 1 --2- 172.21.15.26:0/1403337165 >> v2:172.21.15.160:3300/0 conn(0x7f492006f6c0 0x7f49200c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:11.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f491e7fc640 1 -- 172.21.15.26:0/1403337165 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556351433 0 0) 0x7f492006fb60 con 0x7f4920159c00 2023-12-19T16:58:11.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f491e7fc640 1 -- 172.21.15.26:0/1403337165 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4908003530 con 0x7f4920159c00 2023-12-19T16:58:11.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f491effd640 1 -- 172.21.15.26:0/1403337165 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4920159820 msgr2=0x7f492015df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:11.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f491effd640 1 --2- 172.21.15.26:0/1403337165 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4920159820 0x7f492015df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f491effd640 1 -- 172.21.15.26:0/1403337165 >> v1:172.21.15.26:6789/0 conn(0x7f4920159c00 legacy=0x7f4920070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:11.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.284+0000 7f491effd640 1 -- 172.21.15.26:0/1403337165 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4920070120 con 0x7f492006f6c0 2023-12-19T16:58:11.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.288+0000 7f491effd640 1 --2- 172.21.15.26:0/1403337165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f492006f6c0 0x7f49200c8e80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f49140097c0 tx=0x7f491401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce6630fe749fc945 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:11.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.288+0000 7f491e7fc640 1 -- 172.21.15.26:0/1403337165 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49140ab650 con 0x7f492006f6c0 2023-12-19T16:58:11.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.288+0000 7f491e7fc640 1 -- 172.21.15.26:0/1403337165 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49140ab7b0 con 0x7f492006f6c0 2023-12-19T16:58:11.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.288+0000 7f491e7fc640 1 -- 172.21.15.26:0/1403337165 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49140abb40 con 0x7f492006f6c0 2023-12-19T16:58:11.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.288+0000 7f4926095640 1 -- 172.21.15.26:0/1403337165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f492006f6c0 msgr2=0x7f49200c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:11.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.288+0000 7f4926095640 1 --2- 172.21.15.26:0/1403337165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f492006f6c0 0x7f49200c8e80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f49140097c0 tx=0x7f491401c450 comp rx=0 tx=0).stop 2023-12-19T16:58:11.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.288+0000 7f4926095640 1 -- 172.21.15.26:0/1403337165 shutdown_connections 2023-12-19T16:58:11.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.288+0000 7f4926095640 1 --2- 172.21.15.26:0/1403337165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f492006f6c0 0x7f49200c8e80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.288+0000 7f4926095640 1 --2- 172.21.15.26:0/1403337165 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4920159820 0x7f492015df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.288+0000 7f4926095640 1 -- 172.21.15.26:0/1403337165 >> 172.21.15.26:0/1403337165 conn(0x7f49200ce730 msgr2=0x7f49200ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:11.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.288+0000 7f4926095640 1 -- 172.21.15.26:0/1403337165 shutdown_connections 2023-12-19T16:58:11.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.288+0000 7f4926095640 1 -- 172.21.15.26:0/1403337165 wait complete. 2023-12-19T16:58:11.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f4926095640 1 Processor -- start 2023-12-19T16:58:11.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f4926095640 1 -- start start 2023-12-19T16:58:11.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f4926095640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f492006f6c0 0x7f49200ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:11.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f4926095640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4920159820 0x7f492015b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:11.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f4926095640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4920159c00 0x7f492015ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:11.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f4926095640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492015ccf0 con 0x7f492006f6c0 2023-12-19T16:58:11.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f4926095640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492015c070 con 0x7f4920159820 2023-12-19T16:58:11.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f4926095640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492015c350 con 0x7f4920159c00 2023-12-19T16:58:11.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f491f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f492006f6c0 0x7f49200ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:11.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f491f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f492006f6c0 0x7f49200ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46460/0 (socket says 172.21.15.26:46460) 2023-12-19T16:58:11.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f491f7fe640 1 -- 172.21.15.26:0/3198335490 learned_addr learned my addr 172.21.15.26:0/3198335490 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:11.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f491ffff640 1 --2- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4920159c00 0x7f492015ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f491effd640 1 --2- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4920159820 0x7f492015b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f491f7fe640 1 -- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4920159c00 msgr2=0x7f492015ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f491f7fe640 1 --2- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4920159c00 0x7f492015ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f491f7fe640 1 -- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4920159820 msgr2=0x7f492015b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f491f7fe640 1 --2- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4920159820 0x7f492015b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f491f7fe640 1 -- 172.21.15.26:0/3198335490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4914009420 con 0x7f492006f6c0 2023-12-19T16:58:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f491f7fe640 1 --2- 172.21.15.26:0/3198335490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f492006f6c0 0x7f49200ca5b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f491000e930 tx=0x7f491000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f4907fff640 1 -- 172.21.15.26:0/3198335490 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910009690 con 0x7f492006f6c0 2023-12-19T16:58:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f4907fff640 1 -- 172.21.15.26:0/3198335490 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491000ca00 con 0x7f492006f6c0 2023-12-19T16:58:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f4926095640 1 -- 172.21.15.26:0/3198335490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492006b880 con 0x7f492006f6c0 2023-12-19T16:58:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f4907fff640 1 -- 172.21.15.26:0/3198335490 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491000cd70 con 0x7f492006f6c0 2023-12-19T16:58:11.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.292+0000 7f4926095640 1 -- 172.21.15.26:0/3198335490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492015c910 con 0x7f492006f6c0 2023-12-19T16:58:11.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.296+0000 7f4926095640 1 -- 172.21.15.26:0/3198335490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4920074270 con 0x7f492006f6c0 2023-12-19T16:58:11.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.296+0000 7f4907fff640 1 -- 172.21.15.26:0/3198335490 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 42) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f4910013470 con 0x7f492006f6c0 2023-12-19T16:58:11.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.296+0000 7f4907fff640 1 --2- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f490c065b80 0x7f490c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:11.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.296+0000 7f4907fff640 1 -- 172.21.15.26:0/3198335490 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f4910083920 con 0x7f492006f6c0 2023-12-19T16:58:11.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.296+0000 7f491effd640 1 -- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f490c065b80 msgr2=0x7f490c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:11.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.296+0000 7f491effd640 1 --2- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f490c065b80 0x7f490c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:11.305 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:11.302+0000 7f377dfdb180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:58:11.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.304+0000 7f4907fff640 1 -- 172.21.15.26:0/3198335490 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4910050f00 con 0x7f492006f6c0 2023-12-19T16:58:11.343 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:11.340+0000 7fce4dde4180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:58:11.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.464+0000 7f4926095640 1 -- 172.21.15.26:0/3198335490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49200b46b0 con 0x7f492006f6c0 2023-12-19T16:58:11.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.464+0000 7f4907fff640 1 -- 172.21.15.26:0/3198335490 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4910054520 con 0x7f492006f6c0 2023-12-19T16:58:11.466 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.464+0000 7f4926095640 1 -- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f490c065b80 msgr2=0x7f490c068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.464+0000 7f4926095640 1 --2- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f490c065b80 0x7f490c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.464+0000 7f4926095640 1 -- 172.21.15.26:0/3198335490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f492006f6c0 msgr2=0x7f49200ca5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.464+0000 7f4926095640 1 --2- 172.21.15.26:0/3198335490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f492006f6c0 0x7f49200ca5b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f491000e930 tx=0x7f491000edf0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.468+0000 7f4926095640 1 -- 172.21.15.26:0/3198335490 shutdown_connections 2023-12-19T16:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.468+0000 7f4926095640 1 --2- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f490c065b80 0x7f490c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.468+0000 7f4926095640 1 --2- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4920159c00 0x7f492015ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.468+0000 7f4926095640 1 --2- 172.21.15.26:0/3198335490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4920159820 0x7f492015b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.468+0000 7f4926095640 1 --2- 172.21.15.26:0/3198335490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f492006f6c0 0x7f49200ca5b0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.468+0000 7f4926095640 1 -- 172.21.15.26:0/3198335490 >> 172.21.15.26:0/3198335490 conn(0x7f49200ce730 msgr2=0x7f492006ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:11.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.468+0000 7f4926095640 1 -- 172.21.15.26:0/3198335490 shutdown_connections 2023-12-19T16:58:11.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.468+0000 7f4926095640 1 -- 172.21.15.26:0/3198335490 wait complete. 2023-12-19T16:58:11.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:58:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb65478f640 1 Processor -- start 2023-12-19T16:58:11.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb65478f640 1 -- start start 2023-12-19T16:58:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb65478f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb64c0706d0 0x7fb64c0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb65478f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb64c070ae0 con 0x7fb64c15ada0 2023-12-19T16:58:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb65478f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb64c070d90 con 0x7fb64c15a9c0 2023-12-19T16:58:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb65478f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64c0cc270 con 0x7fb64c0706d0 2023-12-19T16:58:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb651d2a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb64c15ada0 0x7fb64c06ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39786/0 (socket says 172.21.15.26:39786) 2023-12-19T16:58:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb651d2a640 1 -- 172.21.15.26:0/3222290106 learned_addr learned my addr 172.21.15.26:0/3222290106 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:11.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb652d2c640 1 --2- 172.21.15.26:0/3222290106 >> v2:172.21.15.160:3300/0 conn(0x7fb64c0706d0 0x7fb64c0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb651529640 1 -- 172.21.15.26:0/3222290106 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 954170874 0 0) 0x7fb64c070ae0 con 0x7fb64c15ada0 2023-12-19T16:58:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb651529640 1 -- 172.21.15.26:0/3222290106 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb638003530 con 0x7fb64c15ada0 2023-12-19T16:58:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb652d2c640 1 -- 172.21.15.26:0/3222290106 >> v1:172.21.15.160:6789/0 conn(0x7fb64c15a9c0 legacy=0x7fb64c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb651529640 1 -- 172.21.15.26:0/3222290106 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309443071 0 0) 0x7fb64c070d90 con 0x7fb64c15a9c0 2023-12-19T16:58:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb652d2c640 1 -- 172.21.15.26:0/3222290106 >> v1:172.21.15.26:6789/0 conn(0x7fb64c15ada0 legacy=0x7fb64c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:11.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.680+0000 7fb652d2c640 1 -- 172.21.15.26:0/3222290106 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c071070 con 0x7fb64c0706d0 2023-12-19T16:58:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.684+0000 7fb652d2c640 1 --2- 172.21.15.26:0/3222290106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb64c0706d0 0x7fb64c0b01b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb644009d90 tx=0x7fb64401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7e635c3d6114e6c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:11.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.684+0000 7fb651529640 1 -- 172.21.15.26:0/3222290106 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6440a5860 con 0x7fb64c0706d0 2023-12-19T16:58:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.684+0000 7fb651529640 1 -- 172.21.15.26:0/3222290106 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6440a59c0 con 0x7fb64c0706d0 2023-12-19T16:58:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.684+0000 7fb651529640 1 -- 172.21.15.26:0/3222290106 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6440a5d30 con 0x7fb64c0706d0 2023-12-19T16:58:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.684+0000 7fb65478f640 1 -- 172.21.15.26:0/3222290106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb64c0706d0 msgr2=0x7fb64c0b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:11.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.684+0000 7fb65478f640 1 --2- 172.21.15.26:0/3222290106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb64c0706d0 0x7fb64c0b01b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb644009d90 tx=0x7fb64401c890 comp rx=0 tx=0).stop 2023-12-19T16:58:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.684+0000 7fb65478f640 1 -- 172.21.15.26:0/3222290106 shutdown_connections 2023-12-19T16:58:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.684+0000 7fb65478f640 1 --2- 172.21.15.26:0/3222290106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb64c0706d0 0x7fb64c0b01b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.684+0000 7fb65478f640 1 -- 172.21.15.26:0/3222290106 >> 172.21.15.26:0/3222290106 conn(0x7fb64c0ce730 msgr2=0x7fb64c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.684+0000 7fb65478f640 1 -- 172.21.15.26:0/3222290106 shutdown_connections 2023-12-19T16:58:11.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.684+0000 7fb65478f640 1 -- 172.21.15.26:0/3222290106 wait complete. 2023-12-19T16:58:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65478f640 1 Processor -- start 2023-12-19T16:58:11.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65478f640 1 -- start start 2023-12-19T16:58:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65478f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb64c0706d0 0x7fb64c0c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65478f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb64c15a9c0 0x7fb64c1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65478f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb64c15ada0 0x7fb64c15cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65478f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c15c200 con 0x7fb64c0706d0 2023-12-19T16:58:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65478f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0c5810 con 0x7fb64c15ada0 2023-12-19T16:58:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65478f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0c5af0 con 0x7fb64c15a9c0 2023-12-19T16:58:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65252b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb64c0706d0 0x7fb64c0c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65252b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb64c0706d0 0x7fb64c0c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46462/0 (socket says 172.21.15.26:46462) 2023-12-19T16:58:11.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65252b640 1 -- 172.21.15.26:0/396887766 learned_addr learned my addr 172.21.15.26:0/396887766 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb651d2a640 1 --2- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb64c15a9c0 0x7fb64c1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb652d2c640 1 --2- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb64c15ada0 0x7fb64c15cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65252b640 1 -- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb64c15a9c0 msgr2=0x7fb64c1570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65252b640 1 --2- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb64c15a9c0 0x7fb64c1570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65252b640 1 -- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb64c15ada0 msgr2=0x7fb64c15cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65252b640 1 --2- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb64c15ada0 0x7fb64c15cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65252b640 1 -- 172.21.15.26:0/396887766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb644009a70 con 0x7fb64c0706d0 2023-12-19T16:58:11.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65252b640 1 --2- 172.21.15.26:0/396887766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb64c0706d0 0x7fb64c0c5120 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb64800c480 tx=0x7fb64800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb642ffd640 1 -- 172.21.15.26:0/396887766 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb648005480 con 0x7fb64c0706d0 2023-12-19T16:58:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb642ffd640 1 -- 172.21.15.26:0/396887766 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb648003600 con 0x7fb64c0706d0 2023-12-19T16:58:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb642ffd640 1 -- 172.21.15.26:0/396887766 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6480039b0 con 0x7fb64c0706d0 2023-12-19T16:58:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65478f640 1 -- 172.21.15.26:0/396887766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64c0c5dd0 con 0x7fb64c0706d0 2023-12-19T16:58:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65478f640 1 -- 172.21.15.26:0/396887766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64c154c10 con 0x7fb64c0706d0 2023-12-19T16:58:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.688+0000 7fb65478f640 1 -- 172.21.15.26:0/396887766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64c074270 con 0x7fb64c0706d0 2023-12-19T16:58:11.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.692+0000 7fb642ffd640 1 -- 172.21.15.26:0/396887766 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 42) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fb648003b10 con 0x7fb64c0706d0 2023-12-19T16:58:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.692+0000 7fb642ffd640 1 --2- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb630065b80 0x7fb630068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.692+0000 7fb642ffd640 1 -- 172.21.15.26:0/396887766 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fb6480831b0 con 0x7fb64c0706d0 2023-12-19T16:58:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.692+0000 7fb651d2a640 1 -- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb630065b80 msgr2=0x7fb630068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:11.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.692+0000 7fb651d2a640 1 --2- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb630065b80 0x7fb630068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:11.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.696+0000 7fb642ffd640 1 -- 172.21.15.26:0/396887766 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb6480506c0 con 0x7fb64c0706d0 2023-12-19T16:58:11.796 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:11.790+0000 7f377dfdb180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:58:11.857 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:11.854+0000 7f377dfdb180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:58:11.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb65478f640 1 -- 172.21.15.26:0/396887766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb64c0cd7b0 con 0x7fb64c0706d0 2023-12-19T16:58:11.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb642ffd640 1 -- 172.21.15.26:0/396887766 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb648053ce0 con 0x7fb64c0706d0 2023-12-19T16:58:11.861 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:58:11.862 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fce4dde4180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:58:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb65478f640 1 -- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb630065b80 msgr2=0x7fb630068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:11.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb65478f640 1 --2- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb630065b80 0x7fb630068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb65478f640 1 -- 172.21.15.26:0/396887766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb64c0706d0 msgr2=0x7fb64c0c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:11.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb65478f640 1 --2- 172.21.15.26:0/396887766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb64c0706d0 0x7fb64c0c5120 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fb64800c480 tx=0x7fb64800c940 comp rx=0 tx=0).stop 2023-12-19T16:58:11.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb65478f640 1 -- 172.21.15.26:0/396887766 shutdown_connections 2023-12-19T16:58:11.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb65478f640 1 --2- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb64c15ada0 0x7fb64c15cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb65478f640 1 --2- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb630065b80 0x7fb630068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb65478f640 1 --2- 172.21.15.26:0/396887766 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb64c15a9c0 0x7fb64c1570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb65478f640 1 --2- 172.21.15.26:0/396887766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb64c0706d0 0x7fb64c0c5120 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:11.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb65478f640 1 -- 172.21.15.26:0/396887766 >> 172.21.15.26:0/396887766 conn(0x7fb64c0ce730 msgr2=0x7fb64c0b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:11.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb65478f640 1 -- 172.21.15.26:0/396887766 shutdown_connections 2023-12-19T16:58:11.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:11.860+0000 7fb65478f640 1 -- 172.21.15.26:0/396887766 wait complete. 2023-12-19T16:58:11.883 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:58:11.913 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:11.910+0000 7f377dfdb180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:58:11.929 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:11.928+0000 7fce4dde4180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:58:11.989 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:11.988+0000 7fce4dde4180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:58:11.999 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:11.994+0000 7f377dfdb180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:58:12.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.032+0000 7f551c5a6640 1 Processor -- start 2023-12-19T16:58:12.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.032+0000 7f551c5a6640 1 -- start start 2023-12-19T16:58:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.036+0000 7f551c5a6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f551415ccf0 0x7f55140d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.036+0000 7f551c5a6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551415e200 con 0x7f551415d0d0 2023-12-19T16:58:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.036+0000 7f551c5a6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55140d0aa0 con 0x7f55140b4600 2023-12-19T16:58:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.036+0000 7f551c5a6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55140d1380 con 0x7f551415ccf0 2023-12-19T16:58:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.036+0000 7f551a342640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f551415ccf0 0x7f55140d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.036+0000 7f551a342640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f551415ccf0 0x7f55140d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46474/0 (socket says 172.21.15.26:46474) 2023-12-19T16:58:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.036+0000 7f5519b41640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f551415d0d0 0x7f55140b8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39798/0 (socket says 172.21.15.26:39798) 2023-12-19T16:58:12.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.036+0000 7f551a342640 1 -- 172.21.15.26:0/2728646589 learned_addr learned my addr 172.21.15.26:0/2728646589 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:12.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.036+0000 7f551a342640 1 -- 172.21.15.26:0/2728646589 >> v1:172.21.15.160:6789/0 conn(0x7f55140b4600 legacy=0x7f55140b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.036+0000 7f551a342640 1 -- 172.21.15.26:0/2728646589 >> v1:172.21.15.26:6789/0 conn(0x7f551415d0d0 legacy=0x7f55140b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.036+0000 7f551a342640 1 -- 172.21.15.26:0/2728646589 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551415d680 con 0x7f551415ccf0 2023-12-19T16:58:12.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.036+0000 7f5519340640 1 -- 172.21.15.26:0/2728646589 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134063370 0 0) 0x7f551415e200 con 0x7f551415d0d0 2023-12-19T16:58:12.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.036+0000 7f551a342640 1 --2- 172.21.15.26:0/2728646589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551415ccf0 0x7f55140d0e40 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5510009870 tx=0x7f551001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49812f905ab00344 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:12.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f5519340640 1 -- 172.21.15.26:0/2728646589 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55100aa670 con 0x7f551415ccf0 2023-12-19T16:58:12.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f5519340640 1 -- 172.21.15.26:0/2728646589 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55100aa7d0 con 0x7f551415ccf0 2023-12-19T16:58:12.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f5519340640 1 -- 172.21.15.26:0/2728646589 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55100aab60 con 0x7f551415ccf0 2023-12-19T16:58:12.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 -- 172.21.15.26:0/2728646589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551415ccf0 msgr2=0x7f55140d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 --2- 172.21.15.26:0/2728646589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551415ccf0 0x7f55140d0e40 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5510009870 tx=0x7f551001c510 comp rx=0 tx=0).stop 2023-12-19T16:58:12.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 -- 172.21.15.26:0/2728646589 shutdown_connections 2023-12-19T16:58:12.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 --2- 172.21.15.26:0/2728646589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551415ccf0 0x7f55140d0e40 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 -- 172.21.15.26:0/2728646589 >> 172.21.15.26:0/2728646589 conn(0x7f5514065ee0 msgr2=0x7f55140cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:12.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 -- 172.21.15.26:0/2728646589 shutdown_connections 2023-12-19T16:58:12.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 -- 172.21.15.26:0/2728646589 wait complete. 2023-12-19T16:58:12.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 Processor -- start 2023-12-19T16:58:12.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 -- start start 2023-12-19T16:58:12.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55140b4600 0x7f551415c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551415ccf0 0x7f5514154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f551415d0d0 0x7f5514154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5514152f40 con 0x7f551415ccf0 2023-12-19T16:58:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551415af50 con 0x7f551415d0d0 2023-12-19T16:58:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.040+0000 7f551c5a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551415b210 con 0x7f55140b4600 2023-12-19T16:58:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f5519b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551415ccf0 0x7f5514154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f5519b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551415ccf0 0x7f5514154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46488/0 (socket says 172.21.15.26:46488) 2023-12-19T16:58:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f5519b41640 1 -- 172.21.15.26:0/505150692 learned_addr learned my addr 172.21.15.26:0/505150692 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f551a342640 1 --2- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55140b4600 0x7f551415c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f551ab43640 1 --2- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f551415d0d0 0x7f5514154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f5519b41640 1 -- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55140b4600 msgr2=0x7f551415c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f5519b41640 1 --2- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55140b4600 0x7f551415c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f5519b41640 1 -- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f551415d0d0 msgr2=0x7f5514154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f5519b41640 1 --2- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f551415d0d0 0x7f5514154ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f5519b41640 1 -- 172.21.15.26:0/505150692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55100094d0 con 0x7f551415ccf0 2023-12-19T16:58:12.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f5519b41640 1 --2- 172.21.15.26:0/505150692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551415ccf0 0x7f5514154ab0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f550400c4a0 tx=0x7f550400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:12.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f550affd640 1 -- 172.21.15.26:0/505150692 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55040074e0 con 0x7f551415ccf0 2023-12-19T16:58:12.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f550affd640 1 -- 172.21.15.26:0/505150692 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5504007640 con 0x7f551415ccf0 2023-12-19T16:58:12.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f551c5a6640 1 -- 172.21.15.26:0/505150692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551415b4f0 con 0x7f551415ccf0 2023-12-19T16:58:12.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f550affd640 1 -- 172.21.15.26:0/505150692 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504004a30 con 0x7f551415ccf0 2023-12-19T16:58:12.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f551c5a6640 1 -- 172.21.15.26:0/505150692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551415bab0 con 0x7f551415ccf0 2023-12-19T16:58:12.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.044+0000 7f551c5a6640 1 -- 172.21.15.26:0/505150692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5514074270 con 0x7f551415ccf0 2023-12-19T16:58:12.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.048+0000 7f550affd640 1 -- 172.21.15.26:0/505150692 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 42) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f5504004b90 con 0x7f551415ccf0 2023-12-19T16:58:12.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.048+0000 7f550affd640 1 --2- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f8065b80 0x7f54f8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.048+0000 7f550affd640 1 -- 172.21.15.26:0/505150692 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f5504082a30 con 0x7f551415ccf0 2023-12-19T16:58:12.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.048+0000 7f551a342640 1 -- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f8065b80 msgr2=0x7f54f8068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:12.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.048+0000 7f551a342640 1 --2- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f8065b80 0x7f54f8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:12.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.056+0000 7f550affd640 1 -- 172.21.15.26:0/505150692 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f550404fe30 con 0x7f551415ccf0 2023-12-19T16:58:12.057 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:12.054+0000 7f377dfdb180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:58:12.089 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:12.088+0000 7fce4dde4180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:58:12.142 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:12.138+0000 7f377dfdb180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:58:12.148 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:12.148+0000 7fce4dde4180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:58:12.199 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:12.194+0000 7f377dfdb180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:58:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f551c5a6640 1 -- 172.21.15.26:0/505150692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5514153690 con 0x7f551415ccf0 2023-12-19T16:58:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f550affd640 1 -- 172.21.15.26:0/505150692 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f55040180a0 con 0x7f551415ccf0 2023-12-19T16:58:12.221 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:58:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f551c5a6640 1 -- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f8065b80 msgr2=0x7f54f8068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f551c5a6640 1 --2- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f8065b80 0x7f54f8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f551c5a6640 1 -- 172.21.15.26:0/505150692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551415ccf0 msgr2=0x7f5514154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f551c5a6640 1 --2- 172.21.15.26:0/505150692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551415ccf0 0x7f5514154ab0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f550400c4a0 tx=0x7f550400c960 comp rx=0 tx=0).stop 2023-12-19T16:58:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f551c5a6640 1 -- 172.21.15.26:0/505150692 shutdown_connections 2023-12-19T16:58:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f551c5a6640 1 --2- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f551415d0d0 0x7f5514154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f551c5a6640 1 --2- 172.21.15.26:0/505150692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551415ccf0 0x7f5514154ab0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f551c5a6640 1 --2- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f8065b80 0x7f54f8068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f551c5a6640 1 --2- 172.21.15.26:0/505150692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55140b4600 0x7f551415c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f551c5a6640 1 -- 172.21.15.26:0/505150692 >> 172.21.15.26:0/505150692 conn(0x7f5514065ee0 msgr2=0x7f55140b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:12.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f551c5a6640 1 -- 172.21.15.26:0/505150692 shutdown_connections 2023-12-19T16:58:12.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.220+0000 7f551c5a6640 1 -- 172.21.15.26:0/505150692 wait complete. 2023-12-19T16:58:12.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T16:58:12.244 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:12.240+0000 7fce4dde4180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:58:12.302 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:12.300+0000 7fce4dde4180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:58:12.348 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:12.342+0000 7f377dfdb180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:58:12.407 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:12.402+0000 7f377dfdb180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:58:12.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b7a139640 1 Processor -- start 2023-12-19T16:58:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b7a139640 1 -- start start 2023-12-19T16:58:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b7a139640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b7415cf60 0x7f6b740d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b7a139640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b740d1420 0x7f6b740b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b7a139640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6b740b88e0 0x7f6b740b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b7a139640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b740d2960 con 0x7f6b740d1420 2023-12-19T16:58:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b7a139640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b740d18c0 con 0x7f6b740b88e0 2023-12-19T16:58:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b7a139640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b740d1ba0 con 0x7f6b7415cf60 2023-12-19T16:58:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b72ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b740d1420 0x7f6b740b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b72ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b740d1420 0x7f6b740b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46492/0 (socket says 172.21.15.26:46492) 2023-12-19T16:58:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b72ffd640 1 -- 172.21.15.26:0/1998913028 learned_addr learned my addr 172.21.15.26:0/1998913028 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b737fe640 1 --2- 172.21.15.26:0/1998913028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b7415cf60 0x7f6b740d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b73fff640 1 --2- 172.21.15.26:0/1998913028 >> v2:172.21.15.160:3300/0 conn(0x7f6b740b88e0 0x7f6b740b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b72ffd640 1 -- 172.21.15.26:0/1998913028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b7415cf60 msgr2=0x7f6b740d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b72ffd640 1 --2- 172.21.15.26:0/1998913028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b7415cf60 0x7f6b740d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b72ffd640 1 -- 172.21.15.26:0/1998913028 >> v2:172.21.15.160:3300/0 conn(0x7f6b740b88e0 msgr2=0x7f6b740b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b72ffd640 1 --2- 172.21.15.26:0/1998913028 >> v2:172.21.15.160:3300/0 conn(0x7f6b740b88e0 0x7f6b740b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.432+0000 7f6b72ffd640 1 -- 172.21.15.26:0/1998913028 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b740d1e80 con 0x7f6b740d1420 2023-12-19T16:58:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.436+0000 7f6b72ffd640 1 --2- 172.21.15.26:0/1998913028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b740d1420 0x7f6b740b4600 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f6b7408be50 tx=0x7f6b6801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6fcaaeed098900 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.436+0000 7f6b727fc640 1 -- 172.21.15.26:0/1998913028 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680a89b0 con 0x7f6b740d1420 2023-12-19T16:58:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.436+0000 7f6b727fc640 1 -- 172.21.15.26:0/1998913028 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b680a8b10 con 0x7f6b740d1420 2023-12-19T16:58:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.436+0000 7f6b727fc640 1 -- 172.21.15.26:0/1998913028 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680a8ea0 con 0x7f6b740d1420 2023-12-19T16:58:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.436+0000 7f6b7a139640 1 -- 172.21.15.26:0/1998913028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b740d1420 msgr2=0x7f6b740b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.436+0000 7f6b7a139640 1 --2- 172.21.15.26:0/1998913028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b740d1420 0x7f6b740b4600 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f6b7408be50 tx=0x7f6b6801ecf0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.436+0000 7f6b7a139640 1 -- 172.21.15.26:0/1998913028 shutdown_connections 2023-12-19T16:58:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.436+0000 7f6b7a139640 1 --2- 172.21.15.26:0/1998913028 >> v2:172.21.15.160:3300/0 conn(0x7f6b740b88e0 0x7f6b740b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.436+0000 7f6b7a139640 1 --2- 172.21.15.26:0/1998913028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b740d1420 0x7f6b740b4600 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.436+0000 7f6b7a139640 1 --2- 172.21.15.26:0/1998913028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b7415cf60 0x7f6b740d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.436+0000 7f6b7a139640 1 -- 172.21.15.26:0/1998913028 >> 172.21.15.26:0/1998913028 conn(0x7f6b74065ee0 msgr2=0x7f6b740cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.436+0000 7f6b7a139640 1 -- 172.21.15.26:0/1998913028 shutdown_connections 2023-12-19T16:58:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.436+0000 7f6b7a139640 1 -- 172.21.15.26:0/1998913028 wait complete. 2023-12-19T16:58:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b7a139640 1 Processor -- start 2023-12-19T16:58:12.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b7a139640 1 -- start start 2023-12-19T16:58:12.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b7a139640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b740b88e0 0x7f6b741f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b7a139640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b740d1420 0x7f6b7415d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b7a139640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7415cf60 0x7f6b7415d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b7a139640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7415f9e0 con 0x7f6b7415cf60 2023-12-19T16:58:12.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b7a139640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7415dec0 con 0x7f6b740d1420 2023-12-19T16:58:12.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b7a139640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7415e1a0 con 0x7f6b740b88e0 2023-12-19T16:58:12.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b73fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7415cf60 0x7f6b7415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b73fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7415cf60 0x7f6b7415d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46504/0 (socket says 172.21.15.26:46504) 2023-12-19T16:58:12.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b72ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b740d1420 0x7f6b7415d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b737fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b740b88e0 0x7f6b741f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b73fff640 1 -- 172.21.15.26:0/2780934439 learned_addr learned my addr 172.21.15.26:0/2780934439 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:12.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b72ffd640 1 -- 172.21.15.26:0/2780934439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b740b88e0 msgr2=0x7f6b741f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b72ffd640 1 --2- 172.21.15.26:0/2780934439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b740b88e0 0x7f6b741f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b72ffd640 1 -- 172.21.15.26:0/2780934439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7415cf60 msgr2=0x7f6b7415d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b72ffd640 1 --2- 172.21.15.26:0/2780934439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7415cf60 0x7f6b7415d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b72ffd640 1 -- 172.21.15.26:0/2780934439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b680094d0 con 0x7f6b740d1420 2023-12-19T16:58:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b73fff640 1 --2- 172.21.15.26:0/2780934439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7415cf60 0x7f6b7415d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b72ffd640 1 --2- 172.21.15.26:0/2780934439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b740d1420 0x7f6b7415d340 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6b680af000 tx=0x7f6b6801ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b5bfff640 1 -- 172.21.15.26:0/2780934439 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680b0da0 con 0x7f6b740d1420 2023-12-19T16:58:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b5bfff640 1 -- 172.21.15.26:0/2780934439 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b680bb460 con 0x7f6b740d1420 2023-12-19T16:58:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b7a139640 1 -- 172.21.15.26:0/2780934439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7415e480 con 0x7f6b740d1420 2023-12-19T16:58:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b5bfff640 1 -- 172.21.15.26:0/2780934439 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680bb810 con 0x7f6b740d1420 2023-12-19T16:58:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.440+0000 7f6b7a139640 1 -- 172.21.15.26:0/2780934439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7415ea40 con 0x7f6b740d1420 2023-12-19T16:58:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.444+0000 7f6b7a139640 1 -- 172.21.15.26:0/2780934439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b74066860 con 0x7f6b740d1420 2023-12-19T16:58:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.444+0000 7f6b5bfff640 1 -- 172.21.15.26:0/2780934439 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 42) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f6b680b83f0 con 0x7f6b740d1420 2023-12-19T16:58:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.444+0000 7f6b5bfff640 1 --2- 172.21.15.26:0/2780934439 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b60065b80 0x7f6b60068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.444+0000 7f6b5bfff640 1 -- 172.21.15.26:0/2780934439 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f6b68121190 con 0x7f6b740d1420 2023-12-19T16:58:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.444+0000 7f6b737fe640 1 -- 172.21.15.26:0/2780934439 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b60065b80 msgr2=0x7f6b60068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.444+0000 7f6b737fe640 1 --2- 172.21.15.26:0/2780934439 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b60065b80 0x7f6b60068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:12.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.448+0000 7f6b5bfff640 1 -- 172.21.15.26:0/2780934439 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6b680ee720 con 0x7f6b740d1420 2023-12-19T16:58:12.458 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:12.456+0000 7fce4dde4180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:58:12.521 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:12.520+0000 7fce4dde4180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:58:12.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b7a139640 1 -- 172.21.15.26:0/2780934439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6b7415ec30 con 0x7f6b740d1420 2023-12-19T16:58:12.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b5bfff640 1 -- 172.21.15.26:0/2780934439 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v94) v1 ==== 80+0+465 (secure 0 0 0) 0x7f6b680c0080 con 0x7f6b740d1420 2023-12-19T16:58:12.625 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6833/38098 2023-12-20T16:58:08.323203+0000 2023-12-19T16:58:12.625 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2084078801 2023-12-20T16:58:08.323203+0000 2023-12-19T16:58:12.625 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/385148557 2023-12-20T16:58:08.323203+0000 2023-12-19T16:58:12.625 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3582055562 2023-12-20T16:58:08.323203+0000 2023-12-19T16:58:12.625 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/99434118 2023-12-20T16:58:08.323203+0000 2023-12-19T16:58:12.626 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3106091363 2023-12-20T16:58:08.323203+0000 2023-12-19T16:58:12.626 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6832/38098 2023-12-20T16:58:08.323203+0000 2023-12-19T16:58:12.626 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3155542875 2023-12-20T16:58:08.323203+0000 2023-12-19T16:58:12.626 INFO:teuthology.orchestra.run.smithi026.stderr:listed 8 entries 2023-12-19T16:58:12.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b7a139640 1 -- 172.21.15.26:0/2780934439 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b60065b80 msgr2=0x7f6b60068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:12.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b7a139640 1 --2- 172.21.15.26:0/2780934439 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b60065b80 0x7f6b60068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b7a139640 1 -- 172.21.15.26:0/2780934439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b740d1420 msgr2=0x7f6b7415d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b7a139640 1 --2- 172.21.15.26:0/2780934439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b740d1420 0x7f6b7415d340 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6b680af000 tx=0x7f6b6801ba90 comp rx=0 tx=0).stop 2023-12-19T16:58:12.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b7a139640 1 -- 172.21.15.26:0/2780934439 shutdown_connections 2023-12-19T16:58:12.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b7a139640 1 --2- 172.21.15.26:0/2780934439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b7415cf60 0x7f6b7415d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b7a139640 1 --2- 172.21.15.26:0/2780934439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b740d1420 0x7f6b7415d340 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b7a139640 1 --2- 172.21.15.26:0/2780934439 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b60065b80 0x7f6b60068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b7a139640 1 --2- 172.21.15.26:0/2780934439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b740b88e0 0x7f6b741f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b7a139640 1 -- 172.21.15.26:0/2780934439 >> 172.21.15.26:0/2780934439 conn(0x7f6b74065ee0 msgr2=0x7f6b740b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:12.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b7a139640 1 -- 172.21.15.26:0/2780934439 shutdown_connections 2023-12-19T16:58:12.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.624+0000 7f6b7a139640 1 -- 172.21.15.26:0/2780934439 wait complete. 2023-12-19T16:58:12.647 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T16:58:12.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:12.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-19T16:58:12.652 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:58:12.652 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:58:12.652 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:58:12.661 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:12.658+0000 7f377dfdb180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:58:12.756 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T16:58:12.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:12.757 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T16:58:12.766 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:58:12.766 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:58:12.776 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:58:12.776 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:58:12.776 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:58:12.777 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:58:12.778 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:58:12.778 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:58:12.786 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:58:12.786 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:58:12.787 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:58:12.787 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:12.784+0000 7fce4dde4180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:58:12.798 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:12.794+0000 7f377dfdb180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:58:12.809 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:58:12.809 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:58:12.925 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:12.922+0000 7f377dfdb180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:58:12.931 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:12.928+0000 7fce4dde4180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:58:12.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.968+0000 7f53d81b3640 1 Processor -- start 2023-12-19T16:58:12.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.968+0000 7f53d81b3640 1 -- start start 2023-12-19T16:58:12.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d81b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f53d0155030 0x7f53d015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d81b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53d00ce740 0x7f53d00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d81b3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d006c970 con 0x7f53d015f370 2023-12-19T16:58:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d81b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d006c790 con 0x7f53d0155030 2023-12-19T16:58:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d81b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d00cf0f0 con 0x7f53d00ce740 2023-12-19T16:58:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d5f4f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f53d0155030 0x7f53d015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d574e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f53d015f370 0x7f53d006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39804/0 (socket says 172.21.15.26:39804) 2023-12-19T16:58:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d5f4f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f53d0155030 0x7f53d015bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46514/0 (socket says 172.21.15.26:46514) 2023-12-19T16:58:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d574e640 1 -- 172.21.15.26:0/1565847201 learned_addr learned my addr 172.21.15.26:0/1565847201 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d6750640 1 --2- 172.21.15.26:0/1565847201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53d00ce740 0x7f53d00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d5f4f640 1 -- 172.21.15.26:0/1565847201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53d00ce740 msgr2=0x7f53d00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d5f4f640 1 --2- 172.21.15.26:0/1565847201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53d00ce740 0x7f53d00cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d5f4f640 1 -- 172.21.15.26:0/1565847201 >> v1:172.21.15.26:6789/0 conn(0x7f53d015f370 legacy=0x7f53d006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d4f4d640 1 -- 172.21.15.26:0/1565847201 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146094407 0 0) 0x7f53d006c970 con 0x7f53d015f370 2023-12-19T16:58:12.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.972+0000 7f53d5f4f640 1 -- 172.21.15.26:0/1565847201 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d015b1f0 con 0x7f53d0155030 2023-12-19T16:58:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.976+0000 7f53d5f4f640 1 --2- 172.21.15.26:0/1565847201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53d0155030 0x7f53d015bcd0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0097c0 tx=0x7f53cc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1690f214b53fb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.976+0000 7f53d4f4d640 1 -- 172.21.15.26:0/1565847201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0ab650 con 0x7f53d0155030 2023-12-19T16:58:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.976+0000 7f53d4f4d640 1 -- 172.21.15.26:0/1565847201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc0ab7b0 con 0x7f53d0155030 2023-12-19T16:58:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.976+0000 7f53d4f4d640 1 -- 172.21.15.26:0/1565847201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0abb40 con 0x7f53d0155030 2023-12-19T16:58:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.976+0000 7f53d81b3640 1 -- 172.21.15.26:0/1565847201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53d0155030 msgr2=0x7f53d015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.976+0000 7f53d81b3640 1 --2- 172.21.15.26:0/1565847201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53d0155030 0x7f53d015bcd0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0097c0 tx=0x7f53cc01c450 comp rx=0 tx=0).stop 2023-12-19T16:58:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.976+0000 7f53d81b3640 1 -- 172.21.15.26:0/1565847201 shutdown_connections 2023-12-19T16:58:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.976+0000 7f53d81b3640 1 --2- 172.21.15.26:0/1565847201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53d00ce740 0x7f53d00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.976+0000 7f53d81b3640 1 --2- 172.21.15.26:0/1565847201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53d0155030 0x7f53d015bcd0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.976+0000 7f53d81b3640 1 -- 172.21.15.26:0/1565847201 >> 172.21.15.26:0/1565847201 conn(0x7f53d00658e0 msgr2=0x7f53d0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.976+0000 7f53d81b3640 1 -- 172.21.15.26:0/1565847201 shutdown_connections 2023-12-19T16:58:12.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.976+0000 7f53d81b3640 1 -- 172.21.15.26:0/1565847201 wait complete. 2023-12-19T16:58:12.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.976+0000 7f53d81b3640 1 Processor -- start 2023-12-19T16:58:12.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d81b3640 1 -- start start 2023-12-19T16:58:12.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d81b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53d00ce740 0x7f53d0148470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d81b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53d0155030 0x7f53d014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d81b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53d015f370 0x7f53d015c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d81b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d01f4010 con 0x7f53d015f370 2023-12-19T16:58:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d81b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d01f3d30 con 0x7f53d0155030 2023-12-19T16:58:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d81b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d014e410 con 0x7f53d00ce740 2023-12-19T16:58:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d6750640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53d015f370 0x7f53d015c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d5f4f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53d00ce740 0x7f53d0148470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d5f4f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53d00ce740 0x7f53d0148470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:48680/0 (socket says 172.21.15.26:48680) 2023-12-19T16:58:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d5f4f640 1 -- 172.21.15.26:0/3334999136 learned_addr learned my addr 172.21.15.26:0/3334999136 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d574e640 1 --2- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53d0155030 0x7f53d014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d5f4f640 1 -- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53d0155030 msgr2=0x7f53d014eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d5f4f640 1 --2- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53d0155030 0x7f53d014eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d5f4f640 1 -- 172.21.15.26:0/3334999136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53d015f370 msgr2=0x7f53d015c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:12.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d5f4f640 1 --2- 172.21.15.26:0/3334999136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53d015f370 0x7f53d015c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:12.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d5f4f640 1 -- 172.21.15.26:0/3334999136 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc009420 con 0x7f53d00ce740 2023-12-19T16:58:12.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d6750640 1 --2- 172.21.15.26:0/3334999136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53d015f370 0x7f53d015c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:12.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d5f4f640 1 --2- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53d00ce740 0x7f53d0148470 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0ad040 tx=0x7f53cc01ac90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:12.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53c67fc640 1 -- 172.21.15.26:0/3334999136 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0b5810 con 0x7f53d00ce740 2023-12-19T16:58:12.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53c67fc640 1 -- 172.21.15.26:0/3334999136 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc0b5970 con 0x7f53d00ce740 2023-12-19T16:58:12.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53c67fc640 1 -- 172.21.15.26:0/3334999136 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc0b5d20 con 0x7f53d00ce740 2023-12-19T16:58:12.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d81b3640 1 -- 172.21.15.26:0/3334999136 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d014e6f0 con 0x7f53d00ce740 2023-12-19T16:58:12.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d81b3640 1 -- 172.21.15.26:0/3334999136 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d014f430 con 0x7f53d00ce740 2023-12-19T16:58:12.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.980+0000 7f53d81b3640 1 -- 172.21.15.26:0/3334999136 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d0074270 con 0x7f53d00ce740 2023-12-19T16:58:12.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.984+0000 7f53c67fc640 1 -- 172.21.15.26:0/3334999136 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 42) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f53cc0bf410 con 0x7f53d00ce740 2023-12-19T16:58:12.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.984+0000 7f53c67fc640 1 --2- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53b4065b80 0x7f53b4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:12.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.984+0000 7f53c67fc640 1 -- 172.21.15.26:0/3334999136 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f53cc11e2e0 con 0x7f53d00ce740 2023-12-19T16:58:12.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.984+0000 7f53d574e640 1 -- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53b4065b80 msgr2=0x7f53b4068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:12.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.984+0000 7f53d574e640 1 --2- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53b4065b80 0x7f53b4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:12.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:12.992+0000 7f53c67fc640 1 -- 172.21.15.26:0/3334999136 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f53cc0eb760 con 0x7f53d00ce740 2023-12-19T16:58:13.002 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:12.998+0000 7f377dfdb180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:58:13.065 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:13.064+0000 7fce4dde4180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:58:13.138 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:13.134+0000 7f377dfdb180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:58:13.139 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:13.136+0000 7fce4dde4180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:58:13.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.180+0000 7f53d81b3640 1 -- 172.21.15.26:0/3334999136 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53d014eba0 con 0x7f53d00ce740 2023-12-19T16:58:13.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.184+0000 7f53c67fc640 1 -- 172.21.15.26:0/3334999136 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f53cc0eed80 con 0x7f53d00ce740 2023-12-19T16:58:13.185 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:13.185 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/426534014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":426534014},{"type":"v1","addr":"172.21.15.160:6837","nonce":426534014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/1088174367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":1088174367},{"type":"v1","addr":"172.21.15.160:6836","nonce":1088174367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6981,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/343942943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":343942943},{"type":"v1","addr":"172.21.15.26:6834","nonce":343942943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:57:56.187532+0000","modified":"2023-12-19T16:57:59.206380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3421559564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3421559564},{"type":"v1","addr":"172.21.15.26:6835","nonce":3421559564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-19T16:58:13.185 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 41 2023-12-19T16:58:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.184+0000 7f53d81b3640 1 -- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53b4065b80 msgr2=0x7f53b4068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.184+0000 7f53d81b3640 1 --2- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53b4065b80 0x7f53b4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.184+0000 7f53d81b3640 1 -- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53d00ce740 msgr2=0x7f53d0148470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.184+0000 7f53d81b3640 1 --2- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53d00ce740 0x7f53d0148470 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f53cc0ad040 tx=0x7f53cc01ac90 comp rx=0 tx=0).stop 2023-12-19T16:58:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.184+0000 7f53d81b3640 1 -- 172.21.15.26:0/3334999136 shutdown_connections 2023-12-19T16:58:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.184+0000 7f53d81b3640 1 --2- 172.21.15.26:0/3334999136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53d015f370 0x7f53d015c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.184+0000 7f53d81b3640 1 --2- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53d0155030 0x7f53d014eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.184+0000 7f53d81b3640 1 --2- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53b4065b80 0x7f53b4068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.184+0000 7f53d81b3640 1 --2- 172.21.15.26:0/3334999136 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53d00ce740 0x7f53d0148470 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.184+0000 7f53d81b3640 1 -- 172.21.15.26:0/3334999136 >> 172.21.15.26:0/3334999136 conn(0x7f53d00658e0 msgr2=0x7f53d014be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.184+0000 7f53d81b3640 1 -- 172.21.15.26:0/3334999136 shutdown_connections 2023-12-19T16:58:13.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.184+0000 7f53d81b3640 1 -- 172.21.15.26:0/3334999136 wait complete. 2023-12-19T16:58:13.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:58:13.287 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:13.284+0000 7fce4dde4180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:58:13.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a94dfd640 1 Processor -- start 2023-12-19T16:58:13.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a94dfd640 1 -- start start 2023-12-19T16:58:13.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a94dfd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0a900cc0a0 0x7f0a900cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:13.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a94dfd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a90159d20 0x7f0a900cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:13.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a94dfd640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a900adde0 con 0x7f0a900cdee0 2023-12-19T16:58:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a94dfd640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a900c9c20 con 0x7f0a90159d20 2023-12-19T16:58:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a94dfd640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a900b28d0 con 0x7f0a900cc0a0 2023-12-19T16:58:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a8effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a90159d20 0x7f0a900cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a8f7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0a900cc0a0 0x7f0a900cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a8effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a90159d20 0x7f0a900cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46544/0 (socket says 172.21.15.26:46544) 2023-12-19T16:58:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a8effd640 1 -- 172.21.15.26:0/2519448457 learned_addr learned my addr 172.21.15.26:0/2519448457 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:13.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a8effd640 1 -- 172.21.15.26:0/2519448457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0a900cc0a0 msgr2=0x7f0a900cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a8effd640 1 --2- 172.21.15.26:0/2519448457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0a900cc0a0 0x7f0a900cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a8effd640 1 -- 172.21.15.26:0/2519448457 >> v1:172.21.15.160:6789/0 conn(0x7f0a900cdee0 legacy=0x7f0a9015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a8effd640 1 -- 172.21.15.26:0/2519448457 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a900ccf00 con 0x7f0a90159d20 2023-12-19T16:58:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.352+0000 7f0a8e7fc640 1 -- 172.21.15.26:0/2519448457 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1793028336 0 0) 0x7f0a900adde0 con 0x7f0a900cdee0 2023-12-19T16:58:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.356+0000 7f0a8effd640 1 --2- 172.21.15.26:0/2519448457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a90159d20 0x7f0a900cc9c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009870 tx=0x7f0a8401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6c093afff6480c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.356+0000 7f0a8e7fc640 1 -- 172.21.15.26:0/2519448457 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a840ab600 con 0x7f0a90159d20 2023-12-19T16:58:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.356+0000 7f0a8e7fc640 1 -- 172.21.15.26:0/2519448457 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a840a7050 con 0x7f0a90159d20 2023-12-19T16:58:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.356+0000 7f0a8e7fc640 1 -- 172.21.15.26:0/2519448457 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a840ab980 con 0x7f0a90159d20 2023-12-19T16:58:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.356+0000 7f0a94dfd640 1 -- 172.21.15.26:0/2519448457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a90159d20 msgr2=0x7f0a900cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.356+0000 7f0a94dfd640 1 --2- 172.21.15.26:0/2519448457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a90159d20 0x7f0a900cc9c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f0a84009870 tx=0x7f0a8401c480 comp rx=0 tx=0).stop 2023-12-19T16:58:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.356+0000 7f0a94dfd640 1 -- 172.21.15.26:0/2519448457 shutdown_connections 2023-12-19T16:58:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.356+0000 7f0a94dfd640 1 --2- 172.21.15.26:0/2519448457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a90159d20 0x7f0a900cc9c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.356+0000 7f0a94dfd640 1 --2- 172.21.15.26:0/2519448457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0a900cc0a0 0x7f0a900cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.356+0000 7f0a94dfd640 1 -- 172.21.15.26:0/2519448457 >> 172.21.15.26:0/2519448457 conn(0x7f0a9006b0f0 msgr2=0x7f0a9006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.356+0000 7f0a94dfd640 1 -- 172.21.15.26:0/2519448457 shutdown_connections 2023-12-19T16:58:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.356+0000 7f0a94dfd640 1 -- 172.21.15.26:0/2519448457 wait complete. 2023-12-19T16:58:13.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a94dfd640 1 Processor -- start 2023-12-19T16:58:13.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a94dfd640 1 -- start start 2023-12-19T16:58:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a94dfd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a900cc0a0 0x7f0a90070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a94dfd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0a900cdee0 0x7f0a9006f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a94dfd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a90159d20 0x7f0a901533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a94dfd640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a900706a0 con 0x7f0a900cc0a0 2023-12-19T16:58:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a94dfd640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9006fa90 con 0x7f0a90159d20 2023-12-19T16:58:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a94dfd640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9006fd70 con 0x7f0a900cdee0 2023-12-19T16:58:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a900cc0a0 0x7f0a90070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a8f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a900cc0a0 0x7f0a90070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46558/0 (socket says 172.21.15.26:46558) 2023-12-19T16:58:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a8f7fe640 1 -- 172.21.15.26:0/177904059 learned_addr learned my addr 172.21.15.26:0/177904059 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a8effd640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0a900cdee0 0x7f0a9006f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a8ffff640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a90159d20 0x7f0a901533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a8f7fe640 1 -- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0a900cdee0 msgr2=0x7f0a9006f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a8f7fe640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0a900cdee0 0x7f0a9006f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a8f7fe640 1 -- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a90159d20 msgr2=0x7f0a901533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a8f7fe640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a90159d20 0x7f0a901533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a8f7fe640 1 -- 172.21.15.26:0/177904059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a840094d0 con 0x7f0a900cc0a0 2023-12-19T16:58:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a8ffff640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a90159d20 0x7f0a901533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a8f7fe640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a900cc0a0 0x7f0a90070b30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000e870 tx=0x7f0a8000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a77fff640 1 -- 172.21.15.26:0/177904059 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80009510 con 0x7f0a900cc0a0 2023-12-19T16:58:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a77fff640 1 -- 172.21.15.26:0/177904059 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8000eec0 con 0x7f0a900cc0a0 2023-12-19T16:58:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a77fff640 1 -- 172.21.15.26:0/177904059 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8000cb40 con 0x7f0a900cc0a0 2023-12-19T16:58:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a94dfd640 1 -- 172.21.15.26:0/177904059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a90070050 con 0x7f0a900cc0a0 2023-12-19T16:58:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a94dfd640 1 -- 172.21.15.26:0/177904059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a901538f0 con 0x7f0a900cc0a0 2023-12-19T16:58:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.360+0000 7f0a94dfd640 1 -- 172.21.15.26:0/177904059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a90067400 con 0x7f0a900cc0a0 2023-12-19T16:58:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.364+0000 7f0a77fff640 1 -- 172.21.15.26:0/177904059 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 42) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f0a8000cca0 con 0x7f0a900cc0a0 2023-12-19T16:58:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.364+0000 7f0a77fff640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0a7c065b80 0x7f0a7c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.364+0000 7f0a77fff640 1 -- 172.21.15.26:0/177904059 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f0a80083760 con 0x7f0a900cc0a0 2023-12-19T16:58:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.364+0000 7f0a8effd640 1 -- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0a7c065b80 msgr2=0x7f0a7c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.364+0000 7f0a8effd640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0a7c065b80 0x7f0a7c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:13.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.372+0000 7f0a77fff640 1 -- 172.21.15.26:0/177904059 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0a80050d10 con 0x7f0a900cc0a0 2023-12-19T16:58:13.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.532+0000 7f0a94dfd640 1 -- 172.21.15.26:0/177904059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a900cd3c0 con 0x7f0a900cc0a0 2023-12-19T16:58:13.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.532+0000 7f0a77fff640 1 -- 172.21.15.26:0/177904059 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+17701 (secure 0 0 0) 0x7f0a80054330 con 0x7f0a900cc0a0 2023-12-19T16:58:13.536 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:13.536 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":94,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:58:08.323236+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:53.833233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:55.184912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6833/38098":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/2084078801":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/385148557":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/3582055562":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/99434118":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/3106091363":"2023-12-20T16:58:08.323203+0000","172.21.15.160:6832/38098":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/3155542875":"2023-12-20T16:58:08.323203+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:58:13.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.536+0000 7f0a94dfd640 1 -- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0a7c065b80 msgr2=0x7f0a7c068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:13.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.536+0000 7f0a94dfd640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0a7c065b80 0x7f0a7c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.536+0000 7f0a94dfd640 1 -- 172.21.15.26:0/177904059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a900cc0a0 msgr2=0x7f0a90070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:13.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.536+0000 7f0a94dfd640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a900cc0a0 0x7f0a90070b30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000e870 tx=0x7f0a8000ec70 comp rx=0 tx=0).stop 2023-12-19T16:58:13.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.536+0000 7f0a94dfd640 1 -- 172.21.15.26:0/177904059 shutdown_connections 2023-12-19T16:58:13.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.536+0000 7f0a94dfd640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0a90159d20 0x7f0a901533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.536+0000 7f0a94dfd640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0a7c065b80 0x7f0a7c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.536+0000 7f0a94dfd640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0a900cdee0 0x7f0a9006f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.536+0000 7f0a94dfd640 1 --2- 172.21.15.26:0/177904059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a900cc0a0 0x7f0a90070b30 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.536+0000 7f0a94dfd640 1 -- 172.21.15.26:0/177904059 >> 172.21.15.26:0/177904059 conn(0x7f0a9006b0f0 msgr2=0x7f0a90159060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:13.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.536+0000 7f0a94dfd640 1 -- 172.21.15.26:0/177904059 shutdown_connections 2023-12-19T16:58:13.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.536+0000 7f0a94dfd640 1 -- 172.21.15.26:0/177904059 wait complete. 2023-12-19T16:58:13.558 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:57:53.833233+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:57:55.184912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:58:13.558 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:58:13.685 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:13.682+0000 7f377dfdb180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:58:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500d1e9640 1 Processor -- start 2023-12-19T16:58:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500d1e9640 1 -- start start 2023-12-19T16:58:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500d1e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f500815a9c0 0x7f50080706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500d1e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f500806ac80 0x7f50080b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500d1e9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5008070d90 con 0x7f500815ada0 2023-12-19T16:58:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500d1e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50080cc270 con 0x7f500806ac80 2023-12-19T16:58:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500d1e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5008071070 con 0x7f500815a9c0 2023-12-19T16:58:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500759e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f500806ac80 0x7f50080b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500759e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f500806ac80 0x7f50080b01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46572/0 (socket says 172.21.15.26:46572) 2023-12-19T16:58:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500759e640 1 -- 172.21.15.26:0/177765783 learned_addr learned my addr 172.21.15.26:0/177765783 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f5006d9d640 1 --2- 172.21.15.26:0/177765783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f500815a9c0 0x7f50080706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500759e640 1 -- 172.21.15.26:0/177765783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f500815a9c0 msgr2=0x7f50080706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500759e640 1 --2- 172.21.15.26:0/177765783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f500815a9c0 0x7f50080706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500759e640 1 -- 172.21.15.26:0/177765783 >> v1:172.21.15.160:6789/0 conn(0x7f500815ada0 legacy=0x7f50080b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f500759e640 1 -- 172.21.15.26:0/177765783 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5008071350 con 0x7f500806ac80 2023-12-19T16:58:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f5005d9b640 1 -- 172.21.15.26:0/177765783 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795848316 0 0) 0x7f5008070d90 con 0x7f500815ada0 2023-12-19T16:58:13.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.708+0000 7f5006d9d640 1 --2- 172.21.15.26:0/177765783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f500815a9c0 0x7f50080706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.712+0000 7f500759e640 1 --2- 172.21.15.26:0/177765783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f500806ac80 0x7f50080b01b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5000009e50 tx=0x7f500001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be3e4004d8c9851 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.712+0000 7f5005d9b640 1 -- 172.21.15.26:0/177765783 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50000a5720 con 0x7f500806ac80 2023-12-19T16:58:13.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.712+0000 7f5005d9b640 1 -- 172.21.15.26:0/177765783 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50000a5880 con 0x7f500806ac80 2023-12-19T16:58:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.712+0000 7f5005d9b640 1 -- 172.21.15.26:0/177765783 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50000a5c10 con 0x7f500806ac80 2023-12-19T16:58:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 -- 172.21.15.26:0/177765783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f500806ac80 msgr2=0x7f50080b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 --2- 172.21.15.26:0/177765783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f500806ac80 0x7f50080b01b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5000009e50 tx=0x7f500001c950 comp rx=0 tx=0).stop 2023-12-19T16:58:13.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 -- 172.21.15.26:0/177765783 shutdown_connections 2023-12-19T16:58:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 --2- 172.21.15.26:0/177765783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f500806ac80 0x7f50080b01b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 --2- 172.21.15.26:0/177765783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f500815a9c0 0x7f50080706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 -- 172.21.15.26:0/177765783 >> 172.21.15.26:0/177765783 conn(0x7f50080ce730 msgr2=0x7f50080ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 -- 172.21.15.26:0/177765783 shutdown_connections 2023-12-19T16:58:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 -- 172.21.15.26:0/177765783 wait complete. 2023-12-19T16:58:13.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 Processor -- start 2023-12-19T16:58:13.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 -- start start 2023-12-19T16:58:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f500806ac80 0x7f50080b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f500815a9c0 0x7f50080b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f500815ada0 0x7f5008150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50080b1890 con 0x7f500815ada0 2023-12-19T16:58:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50080b31c0 con 0x7f500815a9c0 2023-12-19T16:58:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.716+0000 7f500d1e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50080c68b0 con 0x7f500806ac80 2023-12-19T16:58:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f500815ada0 0x7f5008150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f500815a9c0 0x7f50080b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f500815ada0 0x7f5008150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46588/0 (socket says 172.21.15.26:46588) 2023-12-19T16:58:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f500815a9c0 0x7f50080b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:36908/0 (socket says 172.21.15.26:36908) 2023-12-19T16:58:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500759e640 1 -- 172.21.15.26:0/4114789494 learned_addr learned my addr 172.21.15.26:0/4114789494 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f5006d9d640 1 --2- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f500806ac80 0x7f50080b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500759e640 1 -- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f500806ac80 msgr2=0x7f50080b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:13.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500759e640 1 --2- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f500806ac80 0x7f50080b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500759e640 1 -- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f500815a9c0 msgr2=0x7f50080b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:13.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500759e640 1 --2- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f500815a9c0 0x7f50080b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500759e640 1 -- 172.21.15.26:0/4114789494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff8009420 con 0x7f500815ada0 2023-12-19T16:58:13.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500659c640 1 --2- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f500815a9c0 0x7f50080b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:13.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500759e640 1 --2- 172.21.15.26:0/4114789494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f500815ada0 0x7f5008150960 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f50000af040 tx=0x7f50000196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:13.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f4fef7fe640 1 -- 172.21.15.26:0/4114789494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50000a5720 con 0x7f500815ada0 2023-12-19T16:58:13.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f4fef7fe640 1 -- 172.21.15.26:0/4114789494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50000b88a0 con 0x7f500815ada0 2023-12-19T16:58:13.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f4fef7fe640 1 -- 172.21.15.26:0/4114789494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50000b8c50 con 0x7f500815ada0 2023-12-19T16:58:13.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500d1e9640 1 -- 172.21.15.26:0/4114789494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5000009b30 con 0x7f500815ada0 2023-12-19T16:58:13.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500d1e9640 1 -- 172.21.15.26:0/4114789494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50080c6e70 con 0x7f500815ada0 2023-12-19T16:58:13.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f500d1e9640 1 -- 172.21.15.26:0/4114789494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5008074270 con 0x7f500815ada0 2023-12-19T16:58:13.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.720+0000 7f4fef7fe640 1 -- 172.21.15.26:0/4114789494 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 42) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f50000c2410 con 0x7f500815ada0 2023-12-19T16:58:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.724+0000 7f4fef7fe640 1 --2- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4ff4065bb0 0x7f4ff4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.724+0000 7f4fef7fe640 1 -- 172.21.15.26:0/4114789494 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f500011e870 con 0x7f500815ada0 2023-12-19T16:58:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.724+0000 7f5006d9d640 1 -- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4ff4065bb0 msgr2=0x7f4ff4068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:13.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.724+0000 7f5006d9d640 1 --2- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4ff4065bb0 0x7f4ff4068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:13.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.728+0000 7f4fef7fe640 1 -- 172.21.15.26:0/4114789494 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f50000ebd90 con 0x7f500815ada0 2023-12-19T16:58:13.771 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:13.766+0000 7f377dfdb180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:58:13.873 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:13.872+0000 7fce4dde4180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:58:13.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.884+0000 7f500d1e9640 1 -- 172.21.15.26:0/4114789494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50080cd7b0 con 0x7f500815ada0 2023-12-19T16:58:13.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.888+0000 7f4fef7fe640 1 -- 172.21.15.26:0/4114789494 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5000009510 con 0x7f500815ada0 2023-12-19T16:58:13.888 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:58:13.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.888+0000 7f500d1e9640 1 -- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4ff4065bb0 msgr2=0x7f4ff4068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:13.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.888+0000 7f500d1e9640 1 --2- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4ff4065bb0 0x7f4ff4068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.888+0000 7f500d1e9640 1 -- 172.21.15.26:0/4114789494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f500815ada0 msgr2=0x7f5008150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:13.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.888+0000 7f500d1e9640 1 --2- 172.21.15.26:0/4114789494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f500815ada0 0x7f5008150960 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f50000af040 tx=0x7f50000196f0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.888+0000 7f500d1e9640 1 -- 172.21.15.26:0/4114789494 shutdown_connections 2023-12-19T16:58:13.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.888+0000 7f500d1e9640 1 --2- 172.21.15.26:0/4114789494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f500815ada0 0x7f5008150960 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.888+0000 7f500d1e9640 1 --2- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f500815a9c0 0x7f50080b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.888+0000 7f500d1e9640 1 --2- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4ff4065bb0 0x7f4ff4068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.888+0000 7f500d1e9640 1 --2- 172.21.15.26:0/4114789494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f500806ac80 0x7f50080b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:13.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.888+0000 7f500d1e9640 1 -- 172.21.15.26:0/4114789494 >> 172.21.15.26:0/4114789494 conn(0x7f50080ce730 msgr2=0x7f5008157240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:13.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.888+0000 7f500d1e9640 1 -- 172.21.15.26:0/4114789494 shutdown_connections 2023-12-19T16:58:13.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:13.888+0000 7f500d1e9640 1 -- 172.21.15.26:0/4114789494 wait complete. 2023-12-19T16:58:13.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:58:13.921 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:13.918+0000 7f377dfdb180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:58:13.964 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:13.964+0000 7fce4dde4180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:58:14.005 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:14.002+0000 7f377dfdb180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:58:14.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc87881f640 1 Processor -- start 2023-12-19T16:58:14.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc87881f640 1 -- start start 2023-12-19T16:58:14.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc87881f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc8740706d0 0x7fc8740b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc87881f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc874070ae0 con 0x7fc87415a9c0 2023-12-19T16:58:14.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc87881f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc874070d90 con 0x7fc87415ada0 2023-12-19T16:58:14.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc87881f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8740cc270 con 0x7fc8740706d0 2023-12-19T16:58:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc87259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc87415a9c0 0x7fc8740b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39812/0 (socket says 172.21.15.26:39812) 2023-12-19T16:58:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc87259c640 1 -- 172.21.15.26:0/1525783597 learned_addr learned my addr 172.21.15.26:0/1525783597 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc872d9d640 1 --2- 172.21.15.26:0/1525783597 >> v2:172.21.15.160:3300/0 conn(0x7fc8740706d0 0x7fc8740b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc871d9b640 1 -- 172.21.15.26:0/1525783597 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764799074 0 0) 0x7fc874070ae0 con 0x7fc87415a9c0 2023-12-19T16:58:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc871d9b640 1 -- 172.21.15.26:0/1525783597 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc860003530 con 0x7fc87415a9c0 2023-12-19T16:58:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc872d9d640 1 -- 172.21.15.26:0/1525783597 >> v1:172.21.15.160:6789/0 conn(0x7fc87415ada0 legacy=0x7fc87406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc872d9d640 1 -- 172.21.15.26:0/1525783597 >> v1:172.21.15.26:6789/0 conn(0x7fc87415a9c0 legacy=0x7fc8740b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.056+0000 7fc872d9d640 1 -- 172.21.15.26:0/1525783597 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc874071070 con 0x7fc8740706d0 2023-12-19T16:58:14.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.060+0000 7fc872d9d640 1 --2- 172.21.15.26:0/1525783597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8740706d0 0x7fc8740b01b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009d90 tx=0x7fc86c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4ee9164215f9dcf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.060+0000 7fc871d9b640 1 -- 172.21.15.26:0/1525783597 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0a5860 con 0x7fc8740706d0 2023-12-19T16:58:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.060+0000 7fc871d9b640 1 -- 172.21.15.26:0/1525783597 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c0a59c0 con 0x7fc8740706d0 2023-12-19T16:58:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.060+0000 7fc871d9b640 1 -- 172.21.15.26:0/1525783597 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0a5d30 con 0x7fc8740706d0 2023-12-19T16:58:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.060+0000 7fc87881f640 1 -- 172.21.15.26:0/1525783597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8740706d0 msgr2=0x7fc8740b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.060+0000 7fc87881f640 1 --2- 172.21.15.26:0/1525783597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8740706d0 0x7fc8740b01b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009d90 tx=0x7fc86c01c890 comp rx=0 tx=0).stop 2023-12-19T16:58:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.060+0000 7fc87881f640 1 -- 172.21.15.26:0/1525783597 shutdown_connections 2023-12-19T16:58:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.060+0000 7fc87881f640 1 --2- 172.21.15.26:0/1525783597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc8740706d0 0x7fc8740b01b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.060+0000 7fc87881f640 1 -- 172.21.15.26:0/1525783597 >> 172.21.15.26:0/1525783597 conn(0x7fc8740ce730 msgr2=0x7fc8740ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.060+0000 7fc87881f640 1 -- 172.21.15.26:0/1525783597 shutdown_connections 2023-12-19T16:58:14.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.060+0000 7fc87881f640 1 -- 172.21.15.26:0/1525783597 wait complete. 2023-12-19T16:58:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc87881f640 1 Processor -- start 2023-12-19T16:58:14.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc87881f640 1 -- start start 2023-12-19T16:58:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc87881f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8740706d0 0x7fc87415d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc87881f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc87415a9c0 0x7fc87414ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc87881f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc87415ada0 0x7fc8740af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc87881f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc874151ea0 con 0x7fc87415ada0 2023-12-19T16:58:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc87881f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87415c120 con 0x7fc87415a9c0 2023-12-19T16:58:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc87881f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87414f390 con 0x7fc8740706d0 2023-12-19T16:58:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc872d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc87415ada0 0x7fc8740af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc872d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc87415ada0 0x7fc8740af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46604/0 (socket says 172.21.15.26:46604) 2023-12-19T16:58:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc872d9d640 1 -- 172.21.15.26:0/213509685 learned_addr learned my addr 172.21.15.26:0/213509685 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc86bfff640 1 --2- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc87415a9c0 0x7fc87414ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc87259c640 1 --2- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8740706d0 0x7fc87415d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc872d9d640 1 -- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8740706d0 msgr2=0x7fc87415d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc872d9d640 1 --2- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8740706d0 0x7fc87415d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc872d9d640 1 -- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc87415a9c0 msgr2=0x7fc87414ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc872d9d640 1 --2- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc87415a9c0 0x7fc87414ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc872d9d640 1 -- 172.21.15.26:0/213509685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c009a70 con 0x7fc87415ada0 2023-12-19T16:58:14.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc872d9d640 1 --2- 172.21.15.26:0/213509685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc87415ada0 0x7fc8740af250 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0194a0 tx=0x7fc86c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:14.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc86affd640 1 -- 172.21.15.26:0/213509685 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0ad640 con 0x7fc87415ada0 2023-12-19T16:58:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc86affd640 1 -- 172.21.15.26:0/213509685 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c0ad7a0 con 0x7fc87415ada0 2023-12-19T16:58:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc86affd640 1 -- 172.21.15.26:0/213509685 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0adb10 con 0x7fc87415ada0 2023-12-19T16:58:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc87881f640 1 -- 172.21.15.26:0/213509685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87415be40 con 0x7fc87415ada0 2023-12-19T16:58:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.064+0000 7fc87881f640 1 -- 172.21.15.26:0/213509685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87415ce40 con 0x7fc87415ada0 2023-12-19T16:58:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.068+0000 7fc87881f640 1 -- 172.21.15.26:0/213509685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc874074270 con 0x7fc87415ada0 2023-12-19T16:58:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.068+0000 7fc86affd640 1 -- 172.21.15.26:0/213509685 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 42) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fc86c0b4070 con 0x7fc87415ada0 2023-12-19T16:58:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.068+0000 7fc86affd640 1 --2- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc854065b80 0x7fc854068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.068+0000 7fc86affd640 1 -- 172.21.15.26:0/213509685 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fc86c11e330 con 0x7fc87415ada0 2023-12-19T16:58:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.068+0000 7fc87259c640 1 -- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc854065b80 msgr2=0x7fc854068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:14.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.068+0000 7fc87259c640 1 --2- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc854065b80 0x7fc854068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:14.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.072+0000 7fc86affd640 1 -- 172.21.15.26:0/213509685 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc86c0eb840 con 0x7fc87415ada0 2023-12-19T16:58:14.125 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:14.124+0000 7fce4dde4180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:58:14.213 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:14.212+0000 7fce4dde4180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:58:14.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.232+0000 7fc87881f640 1 -- 172.21.15.26:0/213509685 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8740cd7b0 con 0x7fc87415ada0 2023-12-19T16:58:14.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.232+0000 7fc86affd640 1 -- 172.21.15.26:0/213509685 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc86c0eee60 con 0x7fc87415ada0 2023-12-19T16:58:14.234 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:58:14.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.232+0000 7fc87881f640 1 -- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc854065b80 msgr2=0x7fc854068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:14.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.232+0000 7fc87881f640 1 --2- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc854065b80 0x7fc854068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.232+0000 7fc87881f640 1 -- 172.21.15.26:0/213509685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc87415ada0 msgr2=0x7fc8740af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.232+0000 7fc87881f640 1 --2- 172.21.15.26:0/213509685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc87415ada0 0x7fc8740af250 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0194a0 tx=0x7fc86c019630 comp rx=0 tx=0).stop 2023-12-19T16:58:14.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.232+0000 7fc87881f640 1 -- 172.21.15.26:0/213509685 shutdown_connections 2023-12-19T16:58:14.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.232+0000 7fc87881f640 1 --2- 172.21.15.26:0/213509685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc87415ada0 0x7fc8740af250 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.232+0000 7fc87881f640 1 --2- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc87415a9c0 0x7fc87414ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.232+0000 7fc87881f640 1 --2- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc854065b80 0x7fc854068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.232+0000 7fc87881f640 1 --2- 172.21.15.26:0/213509685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8740706d0 0x7fc87415d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.232+0000 7fc87881f640 1 -- 172.21.15.26:0/213509685 >> 172.21.15.26:0/213509685 conn(0x7fc8740ce730 msgr2=0x7fc874152490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:14.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.236+0000 7fc87881f640 1 -- 172.21.15.26:0/213509685 shutdown_connections 2023-12-19T16:58:14.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.236+0000 7fc87881f640 1 -- 172.21.15.26:0/213509685 wait complete. 2023-12-19T16:58:14.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:58:14.326 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:14.322+0000 7f377dfdb180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:58:14.386 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:14.382+0000 7f377dfdb180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:58:14.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac299e9640 1 Processor -- start 2023-12-19T16:58:14.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac299e9640 1 -- start start 2023-12-19T16:58:14.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac299e9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fac24160330 0x7fac240d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac299e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac240b8460 0x7fac240b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac299e9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac240b4ef0 con 0x7fac240b4a90 2023-12-19T16:58:14.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac299e9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac240d2370 con 0x7fac24160330 2023-12-19T16:58:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac299e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac240d2800 con 0x7fac240b8460 2023-12-19T16:58:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac22ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fac24160330 0x7fac240d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac22ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fac24160330 0x7fac240d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:36940/0 (socket says 172.21.15.26:36940) 2023-12-19T16:58:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac227fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac240b8460 0x7fac240b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac22ffd640 1 -- 172.21.15.26:0/1809281818 learned_addr learned my addr 172.21.15.26:0/1809281818 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac237fe640 1 --1- 172.21.15.26:0/1809281818 >> v1:172.21.15.160:6789/0 conn(0x7fac240b4a90 0x7fac24069d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:51770/0 (socket says 172.21.15.26:51770) 2023-12-19T16:58:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac22ffd640 1 -- 172.21.15.26:0/1809281818 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac240b8460 msgr2=0x7fac240b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac22ffd640 1 --2- 172.21.15.26:0/1809281818 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac240b8460 0x7fac240b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac22ffd640 1 -- 172.21.15.26:0/1809281818 >> v1:172.21.15.160:6789/0 conn(0x7fac240b4a90 legacy=0x7fac24069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.408+0000 7fac22ffd640 1 -- 172.21.15.26:0/1809281818 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac240d1760 con 0x7fac24160330 2023-12-19T16:58:14.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.412+0000 7fac22ffd640 1 --2- 172.21.15.26:0/1809281818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac24160330 0x7fac240d0f10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fac14009870 tx=0x7fac1401c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ddbeb102adbc48d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:14.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.412+0000 7fac21ffb640 1 -- 172.21.15.26:0/1809281818 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140aa670 con 0x7fac24160330 2023-12-19T16:58:14.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.412+0000 7fac21ffb640 1 -- 172.21.15.26:0/1809281818 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac140aa7d0 con 0x7fac24160330 2023-12-19T16:58:14.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.412+0000 7fac21ffb640 1 -- 172.21.15.26:0/1809281818 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140aab60 con 0x7fac24160330 2023-12-19T16:58:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.412+0000 7fac299e9640 1 -- 172.21.15.26:0/1809281818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac24160330 msgr2=0x7fac240d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.412+0000 7fac299e9640 1 --2- 172.21.15.26:0/1809281818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac24160330 0x7fac240d0f10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fac14009870 tx=0x7fac1401c510 comp rx=0 tx=0).stop 2023-12-19T16:58:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.412+0000 7fac299e9640 1 -- 172.21.15.26:0/1809281818 shutdown_connections 2023-12-19T16:58:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.412+0000 7fac299e9640 1 --2- 172.21.15.26:0/1809281818 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac240b8460 0x7fac240b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.412+0000 7fac299e9640 1 --2- 172.21.15.26:0/1809281818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac24160330 0x7fac240d0f10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.412+0000 7fac299e9640 1 -- 172.21.15.26:0/1809281818 >> 172.21.15.26:0/1809281818 conn(0x7fac24065ee0 msgr2=0x7fac240cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:14.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.412+0000 7fac299e9640 1 -- 172.21.15.26:0/1809281818 shutdown_connections 2023-12-19T16:58:14.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.412+0000 7fac299e9640 1 -- 172.21.15.26:0/1809281818 wait complete. 2023-12-19T16:58:14.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac299e9640 1 Processor -- start 2023-12-19T16:58:14.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac299e9640 1 -- start start 2023-12-19T16:58:14.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac299e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac240b4a90 0x7fac240c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac299e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac240b8460 0x7fac24155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac299e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac24160330 0x7fac240ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac299e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac240c9040 con 0x7fac24160330 2023-12-19T16:58:14.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac299e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac241555b0 con 0x7fac240b4a90 2023-12-19T16:58:14.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac299e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac240ca160 con 0x7fac240b8460 2023-12-19T16:58:14.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac24160330 0x7fac240ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:14.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac22ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac240b4a90 0x7fac240c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:14.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac227fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac240b8460 0x7fac24155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:14.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac22ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac240b4a90 0x7fac240c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:36944/0 (socket says 172.21.15.26:36944) 2023-12-19T16:58:14.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac22ffd640 1 -- 172.21.15.26:0/2884413947 learned_addr learned my addr 172.21.15.26:0/2884413947 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:14.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac237fe640 1 -- 172.21.15.26:0/2884413947 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac240b8460 msgr2=0x7fac24155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac237fe640 1 --2- 172.21.15.26:0/2884413947 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac240b8460 0x7fac24155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac237fe640 1 -- 172.21.15.26:0/2884413947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac240b4a90 msgr2=0x7fac240c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac237fe640 1 --2- 172.21.15.26:0/2884413947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac240b4a90 0x7fac240c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac237fe640 1 -- 172.21.15.26:0/2884413947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac140094d0 con 0x7fac24160330 2023-12-19T16:58:14.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac22ffd640 1 --2- 172.21.15.26:0/2884413947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac240b4a90 0x7fac240c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:14.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac237fe640 1 --2- 172.21.15.26:0/2884413947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac24160330 0x7fac240ca700 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00caa0 tx=0x7fac1c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:14.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac0bfff640 1 -- 172.21.15.26:0/2884413947 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c00aba0 con 0x7fac24160330 2023-12-19T16:58:14.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac0bfff640 1 -- 172.21.15.26:0/2884413947 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c00ad00 con 0x7fac24160330 2023-12-19T16:58:14.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac299e9640 1 -- 172.21.15.26:0/2884413947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac24154590 con 0x7fac24160330 2023-12-19T16:58:14.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac0bfff640 1 -- 172.21.15.26:0/2884413947 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0072b0 con 0x7fac24160330 2023-12-19T16:58:14.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.416+0000 7fac299e9640 1 -- 172.21.15.26:0/2884413947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac24154b50 con 0x7fac24160330 2023-12-19T16:58:14.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.420+0000 7fac299e9640 1 -- 172.21.15.26:0/2884413947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2404e850 con 0x7fac24160330 2023-12-19T16:58:14.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.420+0000 7fac0bfff640 1 -- 172.21.15.26:0/2884413947 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fac1c007410 con 0x7fac24160330 2023-12-19T16:58:14.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.420+0000 7fac0bfff640 1 -- 172.21.15.26:0/2884413947 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fac1c082fe0 con 0x7fac24160330 2023-12-19T16:58:14.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.424+0000 7fac0bfff640 1 -- 172.21.15.26:0/2884413947 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fac1c050960 con 0x7fac24160330 2023-12-19T16:58:14.456 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:14.450+0000 7f370665d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:14.456 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:14.450+0000 7f370665d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:14.456 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:14.450+0000 7f370665d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:14.456 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:14.450+0000 7f370665d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:14.456 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:14.450+0000 7f370665d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:14.456 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:14.450+0000 7f3709663640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:14.456 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:14.450+0000 7f3709663640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:14.457 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:14.450+0000 7f3709663640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:14.457 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:14.450+0000 7f3709663640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:14.457 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:14.450+0000 7f3709663640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:14.565 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:14.564+0000 7fce4dde4180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:58:14.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.588+0000 7fac299e9640 1 -- 172.21.15.26:0/2884413947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac240b6830 con 0x7fac24160330 2023-12-19T16:58:14.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.592+0000 7fac0bfff640 1 -- 172.21.15.26:0/2884413947 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac1c017070 con 0x7fac24160330 2023-12-19T16:58:14.592 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:58:14.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.592+0000 7fac299e9640 1 -- 172.21.15.26:0/2884413947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac24160330 msgr2=0x7fac240ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.592+0000 7fac299e9640 1 --2- 172.21.15.26:0/2884413947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac24160330 0x7fac240ca700 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00caa0 tx=0x7fac1c00cf60 comp rx=0 tx=0).stop 2023-12-19T16:58:14.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.592+0000 7fac299e9640 1 -- 172.21.15.26:0/2884413947 shutdown_connections 2023-12-19T16:58:14.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.592+0000 7fac299e9640 1 --2- 172.21.15.26:0/2884413947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac24160330 0x7fac240ca700 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.592+0000 7fac299e9640 1 --2- 172.21.15.26:0/2884413947 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac240b8460 0x7fac24155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.592+0000 7fac299e9640 1 --2- 172.21.15.26:0/2884413947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac240b4a90 0x7fac240c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.592+0000 7fac299e9640 1 -- 172.21.15.26:0/2884413947 >> 172.21.15.26:0/2884413947 conn(0x7fac24065ee0 msgr2=0x7fac240b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:14.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.592+0000 7fac299e9640 1 -- 172.21.15.26:0/2884413947 shutdown_connections 2023-12-19T16:58:14.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.592+0000 7fac299e9640 1 -- 172.21.15.26:0/2884413947 wait complete. 2023-12-19T16:58:14.614 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:58:14.633 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:14.632+0000 7fce4dde4180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:58:14.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.756+0000 7f1e3eef9640 1 Processor -- start 2023-12-19T16:58:14.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.756+0000 7f1e3eef9640 1 -- start start 2023-12-19T16:58:14.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.760+0000 7f1e3eef9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1e3806b3d0 0x7f1e380ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.760+0000 7f1e3eef9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3806bc30 con 0x7f1e380cec40 2023-12-19T16:58:14.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.760+0000 7f1e3eef9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e380cc6e0 con 0x7f1e38154220 2023-12-19T16:58:14.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.760+0000 7f1e3eef9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e380cf0d0 con 0x7f1e3806b3d0 2023-12-19T16:58:14.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.760+0000 7f1e37fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1e3806b3d0 0x7f1e380ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:14.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.760+0000 7f1e37fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1e3806b3d0 0x7f1e380ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46628/0 (socket says 172.21.15.26:46628) 2023-12-19T16:58:14.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.760+0000 7f1e37fff640 1 -- 172.21.15.26:0/2159474526 learned_addr learned my addr 172.21.15.26:0/2159474526 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:14.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.760+0000 7f1e37fff640 1 -- 172.21.15.26:0/2159474526 >> v1:172.21.15.160:6789/0 conn(0x7f1e38154220 legacy=0x7f1e3815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.760+0000 7f1e37fff640 1 -- 172.21.15.26:0/2159474526 >> v1:172.21.15.26:6789/0 conn(0x7f1e380cec40 legacy=0x7f1e3815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.760+0000 7f1e37fff640 1 -- 172.21.15.26:0/2159474526 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3806b950 con 0x7f1e3806b3d0 2023-12-19T16:58:14.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.764+0000 7f1e37fff640 1 --2- 172.21.15.26:0/2159474526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e3806b3d0 0x7f1e380ce700 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009870 tx=0x7f1e2401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac80efe6580b0a44 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:14.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.764+0000 7f1e377fe640 1 -- 172.21.15.26:0/2159474526 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e240aa670 con 0x7f1e3806b3d0 2023-12-19T16:58:14.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.764+0000 7f1e377fe640 1 -- 172.21.15.26:0/2159474526 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e240aa7d0 con 0x7f1e3806b3d0 2023-12-19T16:58:14.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.764+0000 7f1e377fe640 1 -- 172.21.15.26:0/2159474526 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e240aab60 con 0x7f1e3806b3d0 2023-12-19T16:58:14.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.764+0000 7f1e3eef9640 1 -- 172.21.15.26:0/2159474526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e3806b3d0 msgr2=0x7f1e380ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.764+0000 7f1e3eef9640 1 --2- 172.21.15.26:0/2159474526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e3806b3d0 0x7f1e380ce700 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1e24009870 tx=0x7f1e2401c510 comp rx=0 tx=0).stop 2023-12-19T16:58:14.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.764+0000 7f1e3eef9640 1 -- 172.21.15.26:0/2159474526 shutdown_connections 2023-12-19T16:58:14.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.764+0000 7f1e3eef9640 1 --2- 172.21.15.26:0/2159474526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e3806b3d0 0x7f1e380ce700 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.764+0000 7f1e3eef9640 1 -- 172.21.15.26:0/2159474526 >> 172.21.15.26:0/2159474526 conn(0x7f1e380658e0 msgr2=0x7f1e38065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:14.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.764+0000 7f1e3eef9640 1 -- 172.21.15.26:0/2159474526 shutdown_connections 2023-12-19T16:58:14.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.764+0000 7f1e3eef9640 1 -- 172.21.15.26:0/2159474526 wait complete. 2023-12-19T16:58:14.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.764+0000 7f1e3eef9640 1 Processor -- start 2023-12-19T16:58:14.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.764+0000 7f1e3eef9640 1 -- start start 2023-12-19T16:58:14.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3eef9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e3806b3d0 0x7f1e3814c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3eef9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e380cec40 0x7f1e381f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3eef9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e38154220 0x7f1e381f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:14.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3eef9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3806fa00 con 0x7f1e3806b3d0 2023-12-19T16:58:14.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3eef9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380b2750 con 0x7f1e380cec40 2023-12-19T16:58:14.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3eef9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380b2a30 con 0x7f1e38154220 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3cc95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e3806b3d0 0x7f1e3814c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3cc95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e3806b3d0 0x7f1e3814c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46636/0 (socket says 172.21.15.26:46636) 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3cc95640 1 -- 172.21.15.26:0/1609760053 learned_addr learned my addr 172.21.15.26:0/1609760053 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e37fff640 1 --2- 172.21.15.26:0/1609760053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e380cec40 0x7f1e381f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3d496640 1 --2- 172.21.15.26:0/1609760053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e38154220 0x7f1e381f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3cc95640 1 -- 172.21.15.26:0/1609760053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e38154220 msgr2=0x7f1e381f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3cc95640 1 --2- 172.21.15.26:0/1609760053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e38154220 0x7f1e381f4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3cc95640 1 -- 172.21.15.26:0/1609760053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e380cec40 msgr2=0x7f1e381f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3cc95640 1 --2- 172.21.15.26:0/1609760053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e380cec40 0x7f1e381f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3cc95640 1 -- 172.21.15.26:0/1609760053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e240094d0 con 0x7f1e3806b3d0 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e37fff640 1 --2- 172.21.15.26:0/1609760053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e380cec40 0x7f1e381f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3cc95640 1 --2- 172.21.15.26:0/1609760053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e3806b3d0 0x7f1e3814c4d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00c500 tx=0x7f1e2c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e357fa640 1 -- 172.21.15.26:0/1609760053 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c005220 con 0x7f1e3806b3d0 2023-12-19T16:58:14.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e357fa640 1 -- 172.21.15.26:0/1609760053 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c005380 con 0x7f1e3806b3d0 2023-12-19T16:58:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e357fa640 1 -- 172.21.15.26:0/1609760053 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c007270 con 0x7f1e3806b3d0 2023-12-19T16:58:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3eef9640 1 -- 172.21.15.26:0/1609760053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e380b2d10 con 0x7f1e3806b3d0 2023-12-19T16:58:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3eef9640 1 -- 172.21.15.26:0/1609760053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3814cbe0 con 0x7f1e3806b3d0 2023-12-19T16:58:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e3eef9640 1 -- 172.21.15.26:0/1609760053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e38074270 con 0x7f1e3806b3d0 2023-12-19T16:58:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.768+0000 7f1e357fa640 1 -- 172.21.15.26:0/1609760053 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f1e2c0073d0 con 0x7f1e3806b3d0 2023-12-19T16:58:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.772+0000 7f1e357fa640 1 -- 172.21.15.26:0/1609760053 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f1e2c082be0 con 0x7f1e3806b3d0 2023-12-19T16:58:14.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.776+0000 7f1e357fa640 1 -- 172.21.15.26:0/1609760053 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1e2c050340 con 0x7f1e3806b3d0 2023-12-19T16:58:14.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.968+0000 7f1e3eef9640 1 -- 172.21.15.26:0/1609760053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e380cdf60 con 0x7f1e3806b3d0 2023-12-19T16:58:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.968+0000 7f1e357fa640 1 -- 172.21.15.26:0/1609760053 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f1e2c053960 con 0x7f1e3806b3d0 2023-12-19T16:58:14.969 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:14.969 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/426534014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":426534014},{"type":"v1","addr":"172.21.15.160:6837","nonce":426534014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/1088174367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":1088174367},{"type":"v1","addr":"172.21.15.160:6836","nonce":1088174367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6981,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/343942943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":343942943},{"type":"v1","addr":"172.21.15.26:6834","nonce":343942943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:57:56.187532+0000","modified":"2023-12-19T16:57:59.206380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3421559564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3421559564},{"type":"v1","addr":"172.21.15.26:6835","nonce":3421559564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-19T16:58:14.969 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 41 2023-12-19T16:58:14.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.968+0000 7f1e3eef9640 1 -- 172.21.15.26:0/1609760053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e3806b3d0 msgr2=0x7f1e3814c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:14.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.968+0000 7f1e3eef9640 1 --2- 172.21.15.26:0/1609760053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e3806b3d0 0x7f1e3814c4d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00c500 tx=0x7f1e2c00c9c0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.968+0000 7f1e3eef9640 1 -- 172.21.15.26:0/1609760053 shutdown_connections 2023-12-19T16:58:14.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.968+0000 7f1e3eef9640 1 --2- 172.21.15.26:0/1609760053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1e38154220 0x7f1e381f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.968+0000 7f1e3eef9640 1 --2- 172.21.15.26:0/1609760053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1e380cec40 0x7f1e381f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.968+0000 7f1e3eef9640 1 --2- 172.21.15.26:0/1609760053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1e3806b3d0 0x7f1e3814c4d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:14.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.968+0000 7f1e3eef9640 1 -- 172.21.15.26:0/1609760053 >> 172.21.15.26:0/1609760053 conn(0x7f1e380658e0 msgr2=0x7f1e3815a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:14.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.968+0000 7f1e3eef9640 1 -- 172.21.15.26:0/1609760053 shutdown_connections 2023-12-19T16:58:14.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:14.968+0000 7f1e3eef9640 1 -- 172.21.15.26:0/1609760053 wait complete. 2023-12-19T16:58:14.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:58:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.172+0000 7f0689020640 1 Processor -- start 2023-12-19T16:58:15.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.172+0000 7f0689020640 1 -- start start 2023-12-19T16:58:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f0689020640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f068406b3d0 0x7f06840ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f0689020640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f068406bc30 con 0x7f06840cec40 2023-12-19T16:58:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f0689020640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06840cc6e0 con 0x7f0684154220 2023-12-19T16:58:15.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f0689020640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06840cf0d0 con 0x7f068406b3d0 2023-12-19T16:58:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f068359e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f06840cec40 0x7f068415b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39824/0 (socket says 172.21.15.26:39824) 2023-12-19T16:58:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f068359e640 1 -- 172.21.15.26:0/1153881432 learned_addr learned my addr 172.21.15.26:0/1153881432 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f067bfff640 1 --2- 172.21.15.26:0/1153881432 >> v2:172.21.15.160:3300/0 conn(0x7f068406b3d0 0x7f06840ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f068259c640 1 -- 172.21.15.26:0/1153881432 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120766454 0 0) 0x7f068406bc30 con 0x7f06840cec40 2023-12-19T16:58:15.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f068259c640 1 -- 172.21.15.26:0/1153881432 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0670005740 con 0x7f06840cec40 2023-12-19T16:58:15.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f068259c640 1 -- 172.21.15.26:0/1153881432 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740685984 0 0) 0x7f06840cc6e0 con 0x7f0684154220 2023-12-19T16:58:15.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f068259c640 1 -- 172.21.15.26:0/1153881432 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f068406bc30 con 0x7f0684154220 2023-12-19T16:58:15.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f067bfff640 1 -- 172.21.15.26:0/1153881432 >> v1:172.21.15.160:6789/0 conn(0x7f0684154220 legacy=0x7f068415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f067bfff640 1 -- 172.21.15.26:0/1153881432 >> v1:172.21.15.26:6789/0 conn(0x7f06840cec40 legacy=0x7f068415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.176+0000 7f067bfff640 1 -- 172.21.15.26:0/1153881432 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068406b950 con 0x7f068406b3d0 2023-12-19T16:58:15.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f067bfff640 1 --2- 172.21.15.26:0/1153881432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f068406b3d0 0x7f06840ce700 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f066c0097c0 tx=0x7f066c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb8c3ddc9a351884 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:15.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f068259c640 1 -- 172.21.15.26:0/1153881432 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0ab650 con 0x7f068406b3d0 2023-12-19T16:58:15.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f068259c640 1 -- 172.21.15.26:0/1153881432 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c0ab7b0 con 0x7f068406b3d0 2023-12-19T16:58:15.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f068259c640 1 -- 172.21.15.26:0/1153881432 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0abb40 con 0x7f068406b3d0 2023-12-19T16:58:15.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f0689020640 1 -- 172.21.15.26:0/1153881432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f068406b3d0 msgr2=0x7f06840ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f0689020640 1 --2- 172.21.15.26:0/1153881432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f068406b3d0 0x7f06840ce700 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f066c0097c0 tx=0x7f066c01c450 comp rx=0 tx=0).stop 2023-12-19T16:58:15.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f0689020640 1 -- 172.21.15.26:0/1153881432 shutdown_connections 2023-12-19T16:58:15.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f0689020640 1 --2- 172.21.15.26:0/1153881432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f068406b3d0 0x7f06840ce700 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f0689020640 1 -- 172.21.15.26:0/1153881432 >> 172.21.15.26:0/1153881432 conn(0x7f06840658e0 msgr2=0x7f0684065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f0689020640 1 -- 172.21.15.26:0/1153881432 shutdown_connections 2023-12-19T16:58:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f0689020640 1 -- 172.21.15.26:0/1153881432 wait complete. 2023-12-19T16:58:15.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f0689020640 1 Processor -- start 2023-12-19T16:58:15.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f0689020640 1 -- start start 2023-12-19T16:58:15.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.180+0000 7f0689020640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f068406b3d0 0x7f0684158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f0689020640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06840cec40 0x7f0684159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f0689020640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0684154220 0x7f0684157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f0689020640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06841f5480 con 0x7f0684154220 2023-12-19T16:58:15.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f0689020640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0684159130 con 0x7f068406b3d0 2023-12-19T16:58:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f0689020640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0684159a40 con 0x7f06840cec40 2023-12-19T16:58:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f068359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0684154220 0x7f0684157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f068359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0684154220 0x7f0684157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46640/0 (socket says 172.21.15.26:46640) 2023-12-19T16:58:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f068359e640 1 -- 172.21.15.26:0/1337349340 learned_addr learned my addr 172.21.15.26:0/1337349340 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f067bfff640 1 --2- 172.21.15.26:0/1337349340 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06840cec40 0x7f0684159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f0682d9d640 1 --2- 172.21.15.26:0/1337349340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f068406b3d0 0x7f0684158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f068359e640 1 -- 172.21.15.26:0/1337349340 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06840cec40 msgr2=0x7f0684159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f068359e640 1 --2- 172.21.15.26:0/1337349340 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06840cec40 0x7f0684159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f068359e640 1 -- 172.21.15.26:0/1337349340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f068406b3d0 msgr2=0x7f0684158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f068359e640 1 --2- 172.21.15.26:0/1337349340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f068406b3d0 0x7f0684158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f068359e640 1 -- 172.21.15.26:0/1337349340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c009420 con 0x7f0684154220 2023-12-19T16:58:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f068359e640 1 --2- 172.21.15.26:0/1337349340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0684154220 0x7f0684157c20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f067c00cba0 tx=0x7f067c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:15.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f067b7fe640 1 -- 172.21.15.26:0/1337349340 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c00ace0 con 0x7f0684154220 2023-12-19T16:58:15.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f067b7fe640 1 -- 172.21.15.26:0/1337349340 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c00ae40 con 0x7f0684154220 2023-12-19T16:58:15.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f067b7fe640 1 -- 172.21.15.26:0/1337349340 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c012620 con 0x7f0684154220 2023-12-19T16:58:15.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f0689020640 1 -- 172.21.15.26:0/1337349340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068414d520 con 0x7f0684154220 2023-12-19T16:58:15.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f0689020640 1 -- 172.21.15.26:0/1337349340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068415dc40 con 0x7f0684154220 2023-12-19T16:58:15.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f0689020640 1 -- 172.21.15.26:0/1337349340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06841f55c0 con 0x7f0684154220 2023-12-19T16:58:15.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.184+0000 7f067b7fe640 1 -- 172.21.15.26:0/1337349340 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f067c007070 con 0x7f0684154220 2023-12-19T16:58:15.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.188+0000 7f067b7fe640 1 -- 172.21.15.26:0/1337349340 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f067c0831a0 con 0x7f0684154220 2023-12-19T16:58:15.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.192+0000 7f067b7fe640 1 -- 172.21.15.26:0/1337349340 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f067c050a30 con 0x7f0684154220 2023-12-19T16:58:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.348+0000 7f0689020640 1 -- 172.21.15.26:0/1337349340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0684074270 con 0x7f0684154220 2023-12-19T16:58:15.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.352+0000 7f067b7fe640 1 -- 172.21.15.26:0/1337349340 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+18077 (secure 0 0 0) 0x7f067c017090 con 0x7f0684154220 2023-12-19T16:58:15.353 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:15.353 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":95,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:58:14.389987+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:53.833233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:55.184912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/2447076551":"2023-12-20T16:58:14.389953+0000","172.21.15.160:0/1272901554":"2023-12-20T16:58:14.389953+0000","172.21.15.160:0/118825383":"2023-12-20T16:58:14.389953+0000","172.21.15.160:0/3624204704":"2023-12-20T16:58:14.389953+0000","172.21.15.160:0/3114672138":"2023-12-20T16:58:14.389953+0000","172.21.15.160:6833/38098":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/218363418":"2023-12-20T16:58:14.389953+0000","172.21.15.160:0/2084078801":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/385148557":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/3582055562":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/99434118":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/3106091363":"2023-12-20T16:58:08.323203+0000","172.21.15.160:6832/38098":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/3155542875":"2023-12-20T16:58:08.323203+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.352+0000 7f0689020640 1 -- 172.21.15.26:0/1337349340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0684154220 msgr2=0x7f0684157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.352+0000 7f0689020640 1 --2- 172.21.15.26:0/1337349340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0684154220 0x7f0684157c20 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f067c00cba0 tx=0x7f067c00a430 comp rx=0 tx=0).stop 2023-12-19T16:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.352+0000 7f0689020640 1 -- 172.21.15.26:0/1337349340 shutdown_connections 2023-12-19T16:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.352+0000 7f0689020640 1 --2- 172.21.15.26:0/1337349340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0684154220 0x7f0684157c20 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.352+0000 7f0689020640 1 --2- 172.21.15.26:0/1337349340 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06840cec40 0x7f0684159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.352+0000 7f0689020640 1 --2- 172.21.15.26:0/1337349340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f068406b3d0 0x7f0684158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.352+0000 7f0689020640 1 -- 172.21.15.26:0/1337349340 >> 172.21.15.26:0/1337349340 conn(0x7f06840658e0 msgr2=0x7f06840ac850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.352+0000 7f0689020640 1 -- 172.21.15.26:0/1337349340 shutdown_connections 2023-12-19T16:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.352+0000 7f0689020640 1 -- 172.21.15.26:0/1337349340 wait complete. 2023-12-19T16:58:15.374 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T16:58:15.374 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:58:15.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe77796d640 1 Processor -- start 2023-12-19T16:58:15.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe77796d640 1 -- start start 2023-12-19T16:58:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe77796d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe7700cb1d0 0x7fe7700cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe77796d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe77006bee0 0x7fe77006c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe77796d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe770164c90 0x7fe770165120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe77796d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7700cc030 con 0x7fe77006bee0 2023-12-19T16:58:15.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe77796d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe77006d4b0 con 0x7fe7700cb1d0 2023-12-19T16:58:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe77796d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700cbaf0 con 0x7fe770164c90 2023-12-19T16:58:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe774f08640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe77006bee0 0x7fe77006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe774f08640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe77006bee0 0x7fe77006c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46652/0 (socket says 172.21.15.26:46652) 2023-12-19T16:58:15.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe774f08640 1 -- 172.21.15.26:0/1391931898 learned_addr learned my addr 172.21.15.26:0/1391931898 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:15.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe775f0a640 1 --2- 172.21.15.26:0/1391931898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe770164c90 0x7fe770165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe775709640 1 --2- 172.21.15.26:0/1391931898 >> v2:172.21.15.160:3300/0 conn(0x7fe7700cb1d0 0x7fe7700cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe774f08640 1 -- 172.21.15.26:0/1391931898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe770164c90 msgr2=0x7fe770165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:15.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe774f08640 1 --2- 172.21.15.26:0/1391931898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe770164c90 0x7fe770165120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe774f08640 1 -- 172.21.15.26:0/1391931898 >> v2:172.21.15.160:3300/0 conn(0x7fe7700cb1d0 msgr2=0x7fe7700cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:15.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe774f08640 1 --2- 172.21.15.26:0/1391931898 >> v2:172.21.15.160:3300/0 conn(0x7fe7700cb1d0 0x7fe7700cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe774f08640 1 -- 172.21.15.26:0/1391931898 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77006cc10 con 0x7fe77006bee0 2023-12-19T16:58:15.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe775709640 1 --2- 172.21.15.26:0/1391931898 >> v2:172.21.15.160:3300/0 conn(0x7fe7700cb1d0 0x7fe7700cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:15.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.520+0000 7fe775f0a640 1 --2- 172.21.15.26:0/1391931898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe770164c90 0x7fe770165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.524+0000 7fe774f08640 1 --2- 172.21.15.26:0/1391931898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77006bee0 0x7fe77006c340 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe7700ccb20 tx=0x7fe76001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d92167daddee20 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.524+0000 7fe767fff640 1 -- 172.21.15.26:0/1391931898 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7600ab600 con 0x7fe77006bee0 2023-12-19T16:58:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.524+0000 7fe767fff640 1 -- 172.21.15.26:0/1391931898 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7600ab760 con 0x7fe77006bee0 2023-12-19T16:58:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.524+0000 7fe767fff640 1 -- 172.21.15.26:0/1391931898 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7600abaf0 con 0x7fe77006bee0 2023-12-19T16:58:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.524+0000 7fe77796d640 1 -- 172.21.15.26:0/1391931898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77006bee0 msgr2=0x7fe77006c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.524+0000 7fe77796d640 1 --2- 172.21.15.26:0/1391931898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77006bee0 0x7fe77006c340 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fe7700ccb20 tx=0x7fe76001c440 comp rx=0 tx=0).stop 2023-12-19T16:58:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.524+0000 7fe77796d640 1 -- 172.21.15.26:0/1391931898 shutdown_connections 2023-12-19T16:58:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.524+0000 7fe77796d640 1 --2- 172.21.15.26:0/1391931898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe770164c90 0x7fe770165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.524+0000 7fe77796d640 1 --2- 172.21.15.26:0/1391931898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe77006bee0 0x7fe77006c340 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.524+0000 7fe77796d640 1 --2- 172.21.15.26:0/1391931898 >> v2:172.21.15.160:3300/0 conn(0x7fe7700cb1d0 0x7fe7700cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.524+0000 7fe77796d640 1 -- 172.21.15.26:0/1391931898 >> 172.21.15.26:0/1391931898 conn(0x7fe7700658e0 msgr2=0x7fe770065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:15.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.524+0000 7fe77796d640 1 -- 172.21.15.26:0/1391931898 shutdown_connections 2023-12-19T16:58:15.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.524+0000 7fe77796d640 1 -- 172.21.15.26:0/1391931898 wait complete. 2023-12-19T16:58:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe77796d640 1 Processor -- start 2023-12-19T16:58:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe77796d640 1 -- start start 2023-12-19T16:58:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe77796d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe77006bee0 0x7fe770157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe77796d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7700cb1d0 0x7fe770156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe77796d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe770164c90 0x7fe770163370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe77796d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe770156d70 con 0x7fe770164c90 2023-12-19T16:58:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe77796d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe770155ab0 con 0x7fe7700cb1d0 2023-12-19T16:58:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe77796d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe770155d50 con 0x7fe77006bee0 2023-12-19T16:58:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe775f0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe770164c90 0x7fe770163370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe775f0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe770164c90 0x7fe770163370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46668/0 (socket says 172.21.15.26:46668) 2023-12-19T16:58:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe775f0a640 1 -- 172.21.15.26:0/4284268889 learned_addr learned my addr 172.21.15.26:0/4284268889 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe775709640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe77006bee0 0x7fe770157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe774f08640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7700cb1d0 0x7fe770156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe775f0a640 1 -- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe77006bee0 msgr2=0x7fe770157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe775f0a640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe77006bee0 0x7fe770157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe775f0a640 1 -- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7700cb1d0 msgr2=0x7fe770156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe775f0a640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7700cb1d0 0x7fe770156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe775f0a640 1 -- 172.21.15.26:0/4284268889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7600094d0 con 0x7fe770164c90 2023-12-19T16:58:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe775f0a640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe770164c90 0x7fe770163370 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe76800cef0 tx=0x7fe76800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe765ffb640 1 -- 172.21.15.26:0/4284268889 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76800d840 con 0x7fe770164c90 2023-12-19T16:58:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe765ffb640 1 -- 172.21.15.26:0/4284268889 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76800d9c0 con 0x7fe770164c90 2023-12-19T16:58:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe77796d640 1 -- 172.21.15.26:0/4284268889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe770156030 con 0x7fe770164c90 2023-12-19T16:58:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe765ffb640 1 -- 172.21.15.26:0/4284268889 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76800dd70 con 0x7fe770164c90 2023-12-19T16:58:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.528+0000 7fe77796d640 1 -- 172.21.15.26:0/4284268889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7700709e0 con 0x7fe770164c90 2023-12-19T16:58:15.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.532+0000 7fe77796d640 1 -- 172.21.15.26:0/4284268889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7701626b0 con 0x7fe770164c90 2023-12-19T16:58:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.532+0000 7fe765ffb640 1 -- 172.21.15.26:0/4284268889 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 44) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fe76801e3b0 con 0x7fe770164c90 2023-12-19T16:58:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.532+0000 7fe765ffb640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe7540658b0 0x7fe754067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.532+0000 7fe765ffb640 1 -- 172.21.15.26:0/4284268889 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fe768083dd0 con 0x7fe770164c90 2023-12-19T16:58:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.532+0000 7fe775709640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe7540658b0 0x7fe754067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.536+0000 7fe775709640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe7540658b0 0x7fe754067d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe7700ada00 tx=0x7fe76c008040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.536+0000 7fe765ffb640 1 -- 172.21.15.26:0/4284268889 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe768051630 con 0x7fe770164c90 2023-12-19T16:58:15.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.728+0000 7fe77796d640 1 -- 172.21.15.26:0/4284268889 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe770074270 con 0x7fe770164c90 2023-12-19T16:58:15.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.732+0000 7fe765ffb640 1 -- 172.21.15.26:0/4284268889 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4711 (secure 0 0 0) 0x7fe76801a090 con 0x7fe770164c90 2023-12-19T16:58:15.732 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:15.733 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/426534014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":426534014},{"type":"v1","addr":"172.21.15.160:6837","nonce":426534014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/1088174367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":1088174367},{"type":"v1","addr":"172.21.15.160:6836","nonce":1088174367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6981,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/343942943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":343942943},{"type":"v1","addr":"172.21.15.26:6834","nonce":343942943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:57:56.187532+0000","modified":"2023-12-19T16:57:59.206380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3421559564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3421559564},{"type":"v1","addr":"172.21.15.26:6835","nonce":3421559564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-19T16:58:15.733 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 41 2023-12-19T16:58:15.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.732+0000 7fe77796d640 1 -- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe7540658b0 msgr2=0x7fe754067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.732+0000 7fe77796d640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe7540658b0 0x7fe754067d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe7700ada00 tx=0x7fe76c008040 comp rx=0 tx=0).stop 2023-12-19T16:58:15.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.732+0000 7fe77796d640 1 -- 172.21.15.26:0/4284268889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe770164c90 msgr2=0x7fe770163370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.732+0000 7fe77796d640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe770164c90 0x7fe770163370 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fe76800cef0 tx=0x7fe76800a710 comp rx=0 tx=0).stop 2023-12-19T16:58:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.732+0000 7fe77796d640 1 -- 172.21.15.26:0/4284268889 shutdown_connections 2023-12-19T16:58:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.732+0000 7fe77796d640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe770164c90 0x7fe770163370 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.732+0000 7fe77796d640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7700cb1d0 0x7fe770156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.732+0000 7fe77796d640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe7540658b0 0x7fe754067d70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.732+0000 7fe77796d640 1 --2- 172.21.15.26:0/4284268889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe77006bee0 0x7fe770157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.732+0000 7fe77796d640 1 -- 172.21.15.26:0/4284268889 >> 172.21.15.26:0/4284268889 conn(0x7fe7700658e0 msgr2=0x7fe7700b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.732+0000 7fe77796d640 1 -- 172.21.15.26:0/4284268889 shutdown_connections 2023-12-19T16:58:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.732+0000 7fe77796d640 1 -- 172.21.15.26:0/4284268889 wait complete. 2023-12-19T16:58:15.754 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:58:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.896+0000 7f3f0276d640 1 Processor -- start 2023-12-19T16:58:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.896+0000 7f3f0276d640 1 -- start start 2023-12-19T16:58:15.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.896+0000 7f3f0276d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3efc154220 0x7f3efc15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.896+0000 7f3f0276d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc06bc30 con 0x7f3efc0cee10 2023-12-19T16:58:15.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.896+0000 7f3f0276d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3efc0cc6e0 con 0x7f3efc06b3d0 2023-12-19T16:58:15.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.896+0000 7f3f0276d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3efc15afc0 con 0x7f3efc154220 2023-12-19T16:58:15.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.896+0000 7f3efbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3efc154220 0x7f3efc15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.896+0000 7f3f00d0a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3efc0cee10 0x7f3efc15b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56786/0 (socket says 172.21.15.26:56786) 2023-12-19T16:58:15.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.896+0000 7f3f00d0a640 1 -- 172.21.15.26:0/2860991021 learned_addr learned my addr 172.21.15.26:0/2860991021 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:15.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3efbfff640 1 -- 172.21.15.26:0/2860991021 >> v1:172.21.15.160:6789/0 conn(0x7f3efc06b3d0 legacy=0x7f3efc0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:15.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3efbfff640 1 -- 172.21.15.26:0/2860991021 >> v1:172.21.15.26:6789/0 conn(0x7f3efc0cee10 legacy=0x7f3efc15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3efbfff640 1 -- 172.21.15.26:0/2860991021 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3efc06b950 con 0x7f3efc154220 2023-12-19T16:58:15.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3efaffd640 1 -- 172.21.15.26:0/2860991021 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368893118 0 0) 0x7f3efc06bc30 con 0x7f3efc0cee10 2023-12-19T16:58:15.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3efbfff640 1 --2- 172.21.15.26:0/2860991021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc154220 0x7f3efc15aa80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0097c0 tx=0x7f3eec01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5806cb38ec303ef server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3efaffd640 1 -- 172.21.15.26:0/2860991021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0ab610 con 0x7f3efc154220 2023-12-19T16:58:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3efaffd640 1 -- 172.21.15.26:0/2860991021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec0ab770 con 0x7f3efc154220 2023-12-19T16:58:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3efaffd640 1 -- 172.21.15.26:0/2860991021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0abb00 con 0x7f3efc154220 2023-12-19T16:58:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3f0276d640 1 -- 172.21.15.26:0/2860991021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc154220 msgr2=0x7f3efc15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3f0276d640 1 --2- 172.21.15.26:0/2860991021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc154220 0x7f3efc15aa80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0097c0 tx=0x7f3eec01c410 comp rx=0 tx=0).stop 2023-12-19T16:58:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3f0276d640 1 -- 172.21.15.26:0/2860991021 shutdown_connections 2023-12-19T16:58:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3f0276d640 1 --2- 172.21.15.26:0/2860991021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc154220 0x7f3efc15aa80 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3f0276d640 1 -- 172.21.15.26:0/2860991021 >> 172.21.15.26:0/2860991021 conn(0x7f3efc0658e0 msgr2=0x7f3efc065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3f0276d640 1 -- 172.21.15.26:0/2860991021 shutdown_connections 2023-12-19T16:58:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.900+0000 7f3f0276d640 1 -- 172.21.15.26:0/2860991021 wait complete. 2023-12-19T16:58:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f0276d640 1 Processor -- start 2023-12-19T16:58:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f0276d640 1 -- start start 2023-12-19T16:58:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f0276d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3efc06b3d0 0x7f3efc14a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f0276d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3efc0cee10 0x7f3efc14a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f0276d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc154220 0x7f3efc157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f0276d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc14a740 con 0x7f3efc154220 2023-12-19T16:58:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f0276d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc1538e0 con 0x7f3efc0cee10 2023-12-19T16:58:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f0276d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc0b1f10 con 0x7f3efc06b3d0 2023-12-19T16:58:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f00d0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc154220 0x7f3efc157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f00d0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc154220 0x7f3efc157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44822/0 (socket says 172.21.15.26:44822) 2023-12-19T16:58:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f00d0a640 1 -- 172.21.15.26:0/3621022806 learned_addr learned my addr 172.21.15.26:0/3621022806 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:15.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3efbfff640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3efc06b3d0 0x7f3efc14a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3efb7fe640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3efc0cee10 0x7f3efc14a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f00d0a640 1 -- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3efc06b3d0 msgr2=0x7f3efc14a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f00d0a640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3efc06b3d0 0x7f3efc14a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f00d0a640 1 -- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3efc0cee10 msgr2=0x7f3efc14a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:15.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f00d0a640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3efc0cee10 0x7f3efc14a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:15.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f00d0a640 1 -- 172.21.15.26:0/3621022806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec009420 con 0x7f3efc154220 2023-12-19T16:58:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.904+0000 7f3f00d0a640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc154220 0x7f3efc157c20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4002240 tx=0x7f3ef400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.908+0000 7f3ef8ff9640 1 -- 172.21.15.26:0/3621022806 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef400cea0 con 0x7f3efc154220 2023-12-19T16:58:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.908+0000 7f3ef8ff9640 1 -- 172.21.15.26:0/3621022806 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef400a640 con 0x7f3efc154220 2023-12-19T16:58:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.908+0000 7f3ef8ff9640 1 -- 172.21.15.26:0/3621022806 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef400a9f0 con 0x7f3efc154220 2023-12-19T16:58:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.908+0000 7f3f0276d640 1 -- 172.21.15.26:0/3621022806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3efc0b21f0 con 0x7f3efc154220 2023-12-19T16:58:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.908+0000 7f3f0276d640 1 -- 172.21.15.26:0/3621022806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3efc0b1c00 con 0x7f3efc154220 2023-12-19T16:58:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.908+0000 7f3f0276d640 1 -- 172.21.15.26:0/3621022806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3efc0b2710 con 0x7f3efc154220 2023-12-19T16:58:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.908+0000 7f3ef8ff9640 1 -- 172.21.15.26:0/3621022806 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 44) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f3ef400ab50 con 0x7f3efc154220 2023-12-19T16:58:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.908+0000 7f3ef8ff9640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3ee80658b0 0x7f3ee8067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.908+0000 7f3ef8ff9640 1 -- 172.21.15.26:0/3621022806 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f3ef40827e0 con 0x7f3efc154220 2023-12-19T16:58:15.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.908+0000 7f3efbfff640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3ee80658b0 0x7f3ee8067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:15.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.908+0000 7f3efbfff640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3ee80658b0 0x7f3ee8067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0af000 tx=0x7f3eec0a7000 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:15.912+0000 7f3ef8ff9640 1 -- 172.21.15.26:0/3621022806 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3efc0b2710 con 0x7f3efc154220 2023-12-19T16:58:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.100+0000 7f3f0276d640 1 -- 172.21.15.26:0/3621022806 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3efc0b2710 con 0x7f3efc154220 2023-12-19T16:58:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.100+0000 7f3ef8ff9640 1 -- 172.21.15.26:0/3621022806 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f3efc0b2710 con 0x7f3efc154220 2023-12-19T16:58:16.103 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:16.104 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/426534014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":426534014},{"type":"v1","addr":"172.21.15.160:6837","nonce":426534014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/1088174367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":1088174367},{"type":"v1","addr":"172.21.15.160:6836","nonce":1088174367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6981,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/343942943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":343942943},{"type":"v1","addr":"172.21.15.26:6834","nonce":343942943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:57:56.187532+0000","modified":"2023-12-19T16:57:59.206380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6990":{"gid":6990,"name":"a","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3421559564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3421559564},{"type":"v1","addr":"172.21.15.26:6835","nonce":3421559564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-19T16:58:16.104 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 41 2023-12-19T16:58:16.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.104+0000 7f3f0276d640 1 -- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3ee80658b0 msgr2=0x7f3ee8067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.104+0000 7f3f0276d640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3ee80658b0 0x7f3ee8067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0af000 tx=0x7f3eec0a7000 comp rx=0 tx=0).stop 2023-12-19T16:58:16.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.104+0000 7f3f0276d640 1 -- 172.21.15.26:0/3621022806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc154220 msgr2=0x7f3efc157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.104+0000 7f3f0276d640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc154220 0x7f3efc157c20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4002240 tx=0x7f3ef400cc50 comp rx=0 tx=0).stop 2023-12-19T16:58:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.104+0000 7f3f0276d640 1 -- 172.21.15.26:0/3621022806 shutdown_connections 2023-12-19T16:58:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.104+0000 7f3f0276d640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3efc154220 0x7f3efc157c20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.104+0000 7f3f0276d640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3efc0cee10 0x7f3efc14a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.104+0000 7f3f0276d640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3ee80658b0 0x7f3ee8067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.104+0000 7f3f0276d640 1 --2- 172.21.15.26:0/3621022806 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3efc06b3d0 0x7f3efc14a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.104+0000 7f3f0276d640 1 -- 172.21.15.26:0/3621022806 >> 172.21.15.26:0/3621022806 conn(0x7f3efc0658e0 msgr2=0x7f3efc065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.104+0000 7f3f0276d640 1 -- 172.21.15.26:0/3621022806 shutdown_connections 2023-12-19T16:58:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.104+0000 7f3f0276d640 1 -- 172.21.15.26:0/3621022806 wait complete. 2023-12-19T16:58:16.125 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:58:16.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa1471f0640 1 Processor -- start 2023-12-19T16:58:16.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa1471f0640 1 -- start start 2023-12-19T16:58:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa1471f0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa14015cf60 0x7fa1400d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa1471f0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa1400d1420 0x7fa1400b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa1471f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1400b88e0 0x7fa1400b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa1471f0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1400d2960 con 0x7fa14015cf60 2023-12-19T16:58:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa1471f0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1400d18c0 con 0x7fa1400d1420 2023-12-19T16:58:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa1471f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400d1ba0 con 0x7fa1400b88e0 2023-12-19T16:58:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa144f8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa14015cf60 0x7fa1400d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa144f8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa14015cf60 0x7fa1400d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44826/0 (socket says 172.21.15.26:44826) 2023-12-19T16:58:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa144f8c640 1 -- 172.21.15.26:0/2904090075 learned_addr learned my addr 172.21.15.26:0/2904090075 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa13ffff640 1 --2- 172.21.15.26:0/2904090075 >> v2:172.21.15.160:3300/0 conn(0x7fa1400d1420 0x7fa1400b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa14578d640 1 --2- 172.21.15.26:0/2904090075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1400b88e0 0x7fa1400b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa144f8c640 1 -- 172.21.15.26:0/2904090075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1400b88e0 msgr2=0x7fa1400b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa144f8c640 1 --2- 172.21.15.26:0/2904090075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1400b88e0 0x7fa1400b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa144f8c640 1 -- 172.21.15.26:0/2904090075 >> v2:172.21.15.160:3300/0 conn(0x7fa1400d1420 msgr2=0x7fa1400b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa144f8c640 1 --2- 172.21.15.26:0/2904090075 >> v2:172.21.15.160:3300/0 conn(0x7fa1400d1420 0x7fa1400b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.268+0000 7fa144f8c640 1 -- 172.21.15.26:0/2904090075 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1400d1e80 con 0x7fa14015cf60 2023-12-19T16:58:16.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.272+0000 7fa144f8c640 1 --2- 172.21.15.26:0/2904090075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa14015cf60 0x7fa1400d0ee0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa1400d06b0 tx=0x7fa13401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2e76dc0a42fbf1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:16.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.272+0000 7fa13f7fe640 1 -- 172.21.15.26:0/2904090075 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1340a89b0 con 0x7fa14015cf60 2023-12-19T16:58:16.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.272+0000 7fa13f7fe640 1 -- 172.21.15.26:0/2904090075 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1340a8b10 con 0x7fa14015cf60 2023-12-19T16:58:16.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.272+0000 7fa13f7fe640 1 -- 172.21.15.26:0/2904090075 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1340a8ea0 con 0x7fa14015cf60 2023-12-19T16:58:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.272+0000 7fa1471f0640 1 -- 172.21.15.26:0/2904090075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa14015cf60 msgr2=0x7fa1400d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.272+0000 7fa1471f0640 1 --2- 172.21.15.26:0/2904090075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa14015cf60 0x7fa1400d0ee0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fa1400d06b0 tx=0x7fa13401ecf0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.272+0000 7fa1471f0640 1 -- 172.21.15.26:0/2904090075 shutdown_connections 2023-12-19T16:58:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.272+0000 7fa1471f0640 1 --2- 172.21.15.26:0/2904090075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1400b88e0 0x7fa1400b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.272+0000 7fa1471f0640 1 --2- 172.21.15.26:0/2904090075 >> v2:172.21.15.160:3300/0 conn(0x7fa1400d1420 0x7fa1400b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.272+0000 7fa1471f0640 1 --2- 172.21.15.26:0/2904090075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa14015cf60 0x7fa1400d0ee0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.272+0000 7fa1471f0640 1 -- 172.21.15.26:0/2904090075 >> 172.21.15.26:0/2904090075 conn(0x7fa140065ee0 msgr2=0x7fa1400cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.272+0000 7fa1471f0640 1 -- 172.21.15.26:0/2904090075 shutdown_connections 2023-12-19T16:58:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.272+0000 7fa1471f0640 1 -- 172.21.15.26:0/2904090075 wait complete. 2023-12-19T16:58:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa1471f0640 1 Processor -- start 2023-12-19T16:58:16.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa1471f0640 1 -- start start 2023-12-19T16:58:16.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa1471f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1400b88e0 0x7fa14015dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa1471f0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1400d1420 0x7fa1401f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa1471f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa14015cf60 0x7fa14015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa1471f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa140158c00 con 0x7fa1400b88e0 2023-12-19T16:58:16.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa1471f0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1401602a0 con 0x7fa1400d1420 2023-12-19T16:58:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa1471f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14015f690 con 0x7fa14015cf60 2023-12-19T16:58:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa144f8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1400b88e0 0x7fa14015dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa144f8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1400b88e0 0x7fa14015dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44838/0 (socket says 172.21.15.26:44838) 2023-12-19T16:58:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa144f8c640 1 -- 172.21.15.26:0/1555806080 learned_addr learned my addr 172.21.15.26:0/1555806080 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa13ffff640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1400d1420 0x7fa1401f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa14578d640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa14015cf60 0x7fa14015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa144f8c640 1 -- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa14015cf60 msgr2=0x7fa14015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa144f8c640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa14015cf60 0x7fa14015a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa144f8c640 1 -- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1400d1420 msgr2=0x7fa1401f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa144f8c640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1400d1420 0x7fa1401f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa144f8c640 1 -- 172.21.15.26:0/1555806080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1340094d0 con 0x7fa1400b88e0 2023-12-19T16:58:16.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa14578d640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa14015cf60 0x7fa14015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa144f8c640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1400b88e0 0x7fa14015dd50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa13401c920 tx=0x7fa13401ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa13d7fa640 1 -- 172.21.15.26:0/1555806080 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1340b1db0 con 0x7fa1400b88e0 2023-12-19T16:58:16.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa13d7fa640 1 -- 172.21.15.26:0/1555806080 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1340b9440 con 0x7fa1400b88e0 2023-12-19T16:58:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa13d7fa640 1 -- 172.21.15.26:0/1555806080 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1340b97d0 con 0x7fa1400b88e0 2023-12-19T16:58:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa1471f0640 1 -- 172.21.15.26:0/1555806080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14015f970 con 0x7fa1400b88e0 2023-12-19T16:58:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.276+0000 7fa1471f0640 1 -- 172.21.15.26:0/1555806080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14015ff30 con 0x7fa1400b88e0 2023-12-19T16:58:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.280+0000 7fa1471f0640 1 -- 172.21.15.26:0/1555806080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa140074270 con 0x7fa1400b88e0 2023-12-19T16:58:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.280+0000 7fa13d7fa640 1 -- 172.21.15.26:0/1555806080 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 44) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fa1340b7070 con 0x7fa1400b88e0 2023-12-19T16:58:16.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.280+0000 7fa13d7fa640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa1240658b0 0x7fa124067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.280+0000 7fa13d7fa640 1 -- 172.21.15.26:0/1555806080 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fa134120cb0 con 0x7fa1400b88e0 2023-12-19T16:58:16.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.280+0000 7fa13ffff640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa1240658b0 0x7fa124067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.280+0000 7fa13ffff640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa1240658b0 0x7fa124067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa14015ed70 tx=0x7fa12c008040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:16.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.284+0000 7fa13d7fa640 1 -- 172.21.15.26:0/1555806080 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa1340ee3a0 con 0x7fa1400b88e0 2023-12-19T16:58:16.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.444+0000 7fa1471f0640 1 -- 172.21.15.26:0/1555806080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1400cdd30 con 0x7fa1400b88e0 2023-12-19T16:58:16.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.448+0000 7fa13d7fa640 1 -- 172.21.15.26:0/1555806080 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+18077 (secure 0 0 0) 0x7fa1340f19c0 con 0x7fa1400b88e0 2023-12-19T16:58:16.448 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:16.448 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":95,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:58:14.389987+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:57:53.833233+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T16:57:55.184912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/2447076551":"2023-12-20T16:58:14.389953+0000","172.21.15.160:0/1272901554":"2023-12-20T16:58:14.389953+0000","172.21.15.160:0/118825383":"2023-12-20T16:58:14.389953+0000","172.21.15.160:0/3624204704":"2023-12-20T16:58:14.389953+0000","172.21.15.160:0/3114672138":"2023-12-20T16:58:14.389953+0000","172.21.15.160:6833/38098":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/218363418":"2023-12-20T16:58:14.389953+0000","172.21.15.160:0/2084078801":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/385148557":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/3582055562":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/99434118":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/3106091363":"2023-12-20T16:58:08.323203+0000","172.21.15.160:6832/38098":"2023-12-20T16:58:08.323203+0000","172.21.15.160:0/3155542875":"2023-12-20T16:58:08.323203+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:58:16.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.448+0000 7fa1471f0640 1 -- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa1240658b0 msgr2=0x7fa124067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.448+0000 7fa1471f0640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa1240658b0 0x7fa124067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa14015ed70 tx=0x7fa12c008040 comp rx=0 tx=0).stop 2023-12-19T16:58:16.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.448+0000 7fa1471f0640 1 -- 172.21.15.26:0/1555806080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1400b88e0 msgr2=0x7fa14015dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.448+0000 7fa1471f0640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1400b88e0 0x7fa14015dd50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa13401c920 tx=0x7fa13401ba90 comp rx=0 tx=0).stop 2023-12-19T16:58:16.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.448+0000 7fa1471f0640 1 -- 172.21.15.26:0/1555806080 shutdown_connections 2023-12-19T16:58:16.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.448+0000 7fa1471f0640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa1240658b0 0x7fa124067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.448+0000 7fa1471f0640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa14015cf60 0x7fa14015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.448+0000 7fa1471f0640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1400d1420 0x7fa1401f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.448+0000 7fa1471f0640 1 --2- 172.21.15.26:0/1555806080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1400b88e0 0x7fa14015dd50 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.448+0000 7fa1471f0640 1 -- 172.21.15.26:0/1555806080 >> 172.21.15.26:0/1555806080 conn(0x7fa140065ee0 msgr2=0x7fa14006a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:16.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.448+0000 7fa1471f0640 1 -- 172.21.15.26:0/1555806080 shutdown_connections 2023-12-19T16:58:16.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.448+0000 7fa1471f0640 1 -- 172.21.15.26:0/1555806080 wait complete. 2023-12-19T16:58:16.470 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T16:58:16.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.616+0000 7f56adc73640 1 Processor -- start 2023-12-19T16:58:16.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.616+0000 7f56adc73640 1 -- start start 2023-12-19T16:58:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.616+0000 7f56adc73640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a815f370 0x7f56a815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.616+0000 7f56adc73640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a806c970 con 0x7f56a8155030 2023-12-19T16:58:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.616+0000 7f56adc73640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56a815b1f0 con 0x7f56a80ce740 2023-12-19T16:58:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.616+0000 7f56adc73640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a806c790 con 0x7f56a815f370 2023-12-19T16:58:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.616+0000 7f56a77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f56a8155030 0x7f56a806c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56800/0 (socket says 172.21.15.26:56800) 2023-12-19T16:58:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.616+0000 7f56a77fe640 1 -- 172.21.15.26:0/2178575047 learned_addr learned my addr 172.21.15.26:0/2178575047 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:16.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a6ffd640 1 --2- 172.21.15.26:0/2178575047 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a815f370 0x7f56a815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837279383 0 0) 0x7f56a806c970 con 0x7f56a8155030 2023-12-19T16:58:16.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5690003530 con 0x7f56a8155030 2023-12-19T16:58:16.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237751875 0 0) 0x7f56a815b1f0 con 0x7f56a80ce740 2023-12-19T16:58:16.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a806c970 con 0x7f56a80ce740 2023-12-19T16:58:16.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1497133502 0 0) 0x7f5690003530 con 0x7f56a8155030 2023-12-19T16:58:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a815f370 msgr2=0x7f56a815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 --2- 172.21.15.26:0/2178575047 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a815f370 0x7f56a815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 >> v1:172.21.15.160:6789/0 conn(0x7f56a80ce740 legacy=0x7f56a80ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a815b4d0 con 0x7f56a8155030 2023-12-19T16:58:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f5698001680 con 0x7f56a8155030 2023-12-19T16:58:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 >> v1:172.21.15.26:6789/0 conn(0x7f56a8155030 legacy=0x7f56a806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 --2- 172.21.15.26:0/2178575047 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5690003b40 0x7f5690003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 --2- 172.21.15.26:0/2178575047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56900047d0 0x7f5690004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 --2- 172.21.15.26:0/2178575047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5690005190 0x7f569000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a806c790 con 0x7f5690005190 2023-12-19T16:58:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569000bf80 con 0x7f56900047d0 2023-12-19T16:58:16.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569000c140 con 0x7f5690003b40 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a6ffd640 1 --2- 172.21.15.26:0/2178575047 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a815f370 0x7f56a815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a7fff640 1 --2- 172.21.15.26:0/2178575047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5690005190 0x7f569000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a77fe640 1 --2- 172.21.15.26:0/2178575047 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5690003b40 0x7f5690003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a6ffd640 1 --2- 172.21.15.26:0/2178575047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56900047d0 0x7f5690004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a7fff640 1 -- 172.21.15.26:0/2178575047 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5690003b40 msgr2=0x7f5690003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a7fff640 1 --2- 172.21.15.26:0/2178575047 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5690003b40 0x7f5690003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a7fff640 1 -- 172.21.15.26:0/2178575047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56900047d0 msgr2=0x7f5690004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a7fff640 1 --2- 172.21.15.26:0/2178575047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56900047d0 0x7f5690004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.620+0000 7f56a7fff640 1 -- 172.21.15.26:0/2178575047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5690003530 con 0x7f5690005190 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56a7fff640 1 --2- 172.21.15.26:0/2178575047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5690005190 0x7f569000b990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f56a0001fc0 tx=0x7f56a001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4644dd358da13c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0003b80 con 0x7f5690005190 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a0003ce0 con 0x7f5690005190 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56a67fc640 1 -- 172.21.15.26:0/2178575047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a00052f0 con 0x7f5690005190 2023-12-19T16:58:16.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56adc73640 1 -- 172.21.15.26:0/2178575047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5690005190 msgr2=0x7f569000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56adc73640 1 --2- 172.21.15.26:0/2178575047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5690005190 0x7f569000b990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f56a0001fc0 tx=0x7f56a001c8e0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56a7fff640 1 -- 172.21.15.26:0/2178575047 reap_dead start 2023-12-19T16:58:16.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56adc73640 1 -- 172.21.15.26:0/2178575047 shutdown_connections 2023-12-19T16:58:16.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56adc73640 1 -- 172.21.15.26:0/2178575047 >> 172.21.15.26:0/2178575047 conn(0x7f56a80658e0 msgr2=0x7f56a8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:16.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56adc73640 1 -- 172.21.15.26:0/2178575047 shutdown_connections 2023-12-19T16:58:16.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56adc73640 1 -- 172.21.15.26:0/2178575047 wait complete. 2023-12-19T16:58:16.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56adc73640 1 Processor -- start 2023-12-19T16:58:16.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56adc73640 1 -- start start 2023-12-19T16:58:16.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56adc73640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5690005190 0x7f56a81e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.624+0000 7f56adc73640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a80707a0 0x7f56a8142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56adc73640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a8070c20 0x7f56a8143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56adc73640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81e6c60 con 0x7f56a80707a0 2023-12-19T16:58:16.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56adc73640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81e5fe0 con 0x7f5690005190 2023-12-19T16:58:16.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56adc73640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a81e62c0 con 0x7f56a8070c20 2023-12-19T16:58:16.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56a6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a80707a0 0x7f56a8142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56a6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a80707a0 0x7f56a8142e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44852/0 (socket says 172.21.15.26:44852) 2023-12-19T16:58:16.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56a6ffd640 1 -- 172.21.15.26:0/994457372 learned_addr learned my addr 172.21.15.26:0/994457372 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:16.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56a7fff640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a8070c20 0x7f56a8143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56a77fe640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5690005190 0x7f56a81e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56a6ffd640 1 -- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a8070c20 msgr2=0x7f56a8143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56a6ffd640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a8070c20 0x7f56a8143380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56a6ffd640 1 -- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5690005190 msgr2=0x7f56a81e7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56a6ffd640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5690005190 0x7f56a81e7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56a6ffd640 1 -- 172.21.15.26:0/994457372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a81e65a0 con 0x7f56a80707a0 2023-12-19T16:58:16.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56a77fe640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5690005190 0x7f56a81e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:16.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56a6ffd640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a80707a0 0x7f56a8142e40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f569c0115f0 tx=0x7f569c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:16.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f568ffff640 1 -- 172.21.15.26:0/994457372 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c00f8f0 con 0x7f56a80707a0 2023-12-19T16:58:16.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f568ffff640 1 -- 172.21.15.26:0/994457372 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569c00fa50 con 0x7f56a80707a0 2023-12-19T16:58:16.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f568ffff640 1 -- 172.21.15.26:0/994457372 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569c00fdc0 con 0x7f56a80707a0 2023-12-19T16:58:16.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56adc73640 1 -- 172.21.15.26:0/994457372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a81e6880 con 0x7f56a80707a0 2023-12-19T16:58:16.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56adc73640 1 -- 172.21.15.26:0/994457372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a8143a70 con 0x7f56a80707a0 2023-12-19T16:58:16.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f56adc73640 1 -- 172.21.15.26:0/994457372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a8074270 con 0x7f56a80707a0 2023-12-19T16:58:16.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.628+0000 7f568ffff640 1 -- 172.21.15.26:0/994457372 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f569c015460 con 0x7f56a80707a0 2023-12-19T16:58:16.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.632+0000 7f568ffff640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5694065af0 0x7f5694067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:16.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.632+0000 7f568ffff640 1 -- 172.21.15.26:0/994457372 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f569c083ad0 con 0x7f56a80707a0 2023-12-19T16:58:16.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.632+0000 7f56a77fe640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5694065af0 0x7f5694067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:16.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.632+0000 7f56a77fe640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5694065af0 0x7f5694067fb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5698005260 tx=0x7f5698009040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:16.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.636+0000 7f568ffff640 1 -- 172.21.15.26:0/994457372 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f569c050ff0 con 0x7f56a80707a0 2023-12-19T16:58:16.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.824+0000 7f56adc73640 1 -- 172.21.15.26:0/994457372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f56a815a870 con 0x7f56a80707a0 2023-12-19T16:58:16.827 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T16:58:16.824+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T16:58:16.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.836+0000 7f568ffff640 1 -- 172.21.15.26:0/994457372 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7f569c054610 con 0x7f56a80707a0 2023-12-19T16:58:16.837 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T16:58:16.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.840+0000 7f56adc73640 1 -- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5694065af0 msgr2=0x7f5694067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.840+0000 7f56adc73640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5694065af0 0x7f5694067fb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5698005260 tx=0x7f5698009040 comp rx=0 tx=0).stop 2023-12-19T16:58:16.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.840+0000 7f56adc73640 1 -- 172.21.15.26:0/994457372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a80707a0 msgr2=0x7f56a8142e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:16.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.840+0000 7f56adc73640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a80707a0 0x7f56a8142e40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f569c0115f0 tx=0x7f569c011ab0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.840+0000 7f56adc73640 1 -- 172.21.15.26:0/994457372 shutdown_connections 2023-12-19T16:58:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.840+0000 7f56adc73640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5694065af0 0x7f5694067fb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.840+0000 7f56adc73640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56a8070c20 0x7f56a8143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.840+0000 7f56adc73640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a80707a0 0x7f56a8142e40 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.840+0000 7f56adc73640 1 --2- 172.21.15.26:0/994457372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5690005190 0x7f56a81e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.840+0000 7f56adc73640 1 -- 172.21.15.26:0/994457372 >> 172.21.15.26:0/994457372 conn(0x7f56a80658e0 msgr2=0x7f56a8154410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.840+0000 7f56adc73640 1 -- 172.21.15.26:0/994457372 shutdown_connections 2023-12-19T16:58:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:16.840+0000 7f56adc73640 1 -- 172.21.15.26:0/994457372 wait complete. 2023-12-19T16:58:16.846 INFO:tasks.ceph.mds.a.smithi026.stderr: -2519> 2023-12-19T16:58:01.757+0000 7f9901dad640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T16:58:16.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T16:58:16.899 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T16:58:17.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bd102d640 1 Processor -- start 2023-12-19T16:58:17.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bd102d640 1 -- start start 2023-12-19T16:58:17.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bd102d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bcc1585f0 0x7f0bcc0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:17.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bd102d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0bcc0cb5d0 0x7f0bcc0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:17.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bd102d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bcc15b1b0 con 0x7f0bcc0cb150 2023-12-19T16:58:17.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bd102d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bcc0b47a0 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bd102d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc15b490 con 0x7f0bcc1585f0 2023-12-19T16:58:17.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bcb59e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0bcc0cb5d0 0x7f0bcc0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:17.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bcad9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bcc1585f0 0x7f0bcc0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:17.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bca59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f0bcc0cb150 0x7f0bcc0cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:38120/0 (socket says 172.21.15.26:38120) 2023-12-19T16:58:17.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bca59c640 1 -- 172.21.15.26:0/2509420621 learned_addr learned my addr 172.21.15.26:0/2509420621 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bc9d9b640 1 -- 172.21.15.26:0/2509420621 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789502816 0 0) 0x7f0bcc15b1b0 con 0x7f0bcc0cb150 2023-12-19T16:58:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bc9d9b640 1 -- 172.21.15.26:0/2509420621 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bb4003530 con 0x7f0bcc0cb150 2023-12-19T16:58:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.012+0000 7f0bcb59e640 1 -- 172.21.15.26:0/2509420621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bcc1585f0 msgr2=0x7f0bcc0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bcb59e640 1 --2- 172.21.15.26:0/2509420621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bcc1585f0 0x7f0bcc0cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bcb59e640 1 -- 172.21.15.26:0/2509420621 >> v1:172.21.15.160:6789/0 conn(0x7f0bcc0cb150 legacy=0x7f0bcc0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bcb59e640 1 -- 172.21.15.26:0/2509420621 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc15b770 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bcad9d640 1 --2- 172.21.15.26:0/2509420621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bcc1585f0 0x7f0bcc0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:17.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bcb59e640 1 --2- 172.21.15.26:0/2509420621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0cb5d0 0x7f0bcc0cca20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009d90 tx=0x7f0bc401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c243ef1d6b024d23 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:17.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bc9d9b640 1 -- 172.21.15.26:0/2509420621 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40a5970 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bc9d9b640 1 -- 172.21.15.26:0/2509420621 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc40a5ad0 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bc9d9b640 1 -- 172.21.15.26:0/2509420621 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40a5e40 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bd102d640 1 -- 172.21.15.26:0/2509420621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0cb5d0 msgr2=0x7f0bcc0cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bd102d640 1 --2- 172.21.15.26:0/2509420621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0cb5d0 0x7f0bcc0cca20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009d90 tx=0x7f0bc401c890 comp rx=0 tx=0).stop 2023-12-19T16:58:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bd102d640 1 -- 172.21.15.26:0/2509420621 shutdown_connections 2023-12-19T16:58:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bd102d640 1 --2- 172.21.15.26:0/2509420621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0cb5d0 0x7f0bcc0cca20 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bd102d640 1 --2- 172.21.15.26:0/2509420621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bcc1585f0 0x7f0bcc0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bd102d640 1 -- 172.21.15.26:0/2509420621 >> 172.21.15.26:0/2509420621 conn(0x7f0bcc066160 msgr2=0x7f0bcc0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bd102d640 1 -- 172.21.15.26:0/2509420621 shutdown_connections 2023-12-19T16:58:17.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.016+0000 7f0bd102d640 1 -- 172.21.15.26:0/2509420621 wait complete. 2023-12-19T16:58:17.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bd102d640 1 Processor -- start 2023-12-19T16:58:17.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bd102d640 1 -- start start 2023-12-19T16:58:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bd102d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0cb150 0x7f0bcc0af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bd102d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bcc0cb5d0 0x7f0bcc0cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bd102d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bcc1585f0 0x7f0bcc14dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bd102d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0ccf40 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bd102d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0afb90 con 0x7f0bcc0cb150 2023-12-19T16:58:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bd102d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0cca30 con 0x7f0bcc1585f0 2023-12-19T16:58:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bca59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bcc0cb5d0 0x7f0bcc0cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bca59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bcc0cb5d0 0x7f0bcc0cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44872/0 (socket says 172.21.15.26:44872) 2023-12-19T16:58:17.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bca59c640 1 -- 172.21.15.26:0/2738236766 learned_addr learned my addr 172.21.15.26:0/2738236766 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:17.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bcad9d640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0cb150 0x7f0bcc0af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:17.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bcb59e640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bcc1585f0 0x7f0bcc14dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:17.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bca59c640 1 -- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bcc1585f0 msgr2=0x7f0bcc14dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:17.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bca59c640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bcc1585f0 0x7f0bcc14dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:17.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bca59c640 1 -- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0cb150 msgr2=0x7f0bcc0af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:17.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bca59c640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0cb150 0x7f0bcc0af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:17.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bca59c640 1 -- 172.21.15.26:0/2738236766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc009420 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.020+0000 7f0bca59c640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bcc0cb5d0 0x7f0bcc0cc4f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000c3c0 tx=0x7f0bc000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:17.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.024+0000 7f0bb37fe640 1 -- 172.21.15.26:0/2738236766 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0005cf0 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.024+0000 7f0bb37fe640 1 -- 172.21.15.26:0/2738236766 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc0005e50 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.024+0000 7f0bb37fe640 1 -- 172.21.15.26:0/2738236766 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc00072f0 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.024+0000 7f0bd102d640 1 -- 172.21.15.26:0/2738236766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc4009a70 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.024+0000 7f0bd102d640 1 -- 172.21.15.26:0/2738236766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc15c3e0 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.024+0000 7f0bd102d640 1 -- 172.21.15.26:0/2738236766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bcc14e170 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.024+0000 7f0bb37fe640 1 -- 172.21.15.26:0/2738236766 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f0bc0007450 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.024+0000 7f0bb37fe640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bb8065af0 0x7f0bb8067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:17.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.024+0000 7f0bb37fe640 1 -- 172.21.15.26:0/2738236766 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f0bc0082c20 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.024+0000 7f0bcad9d640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bb8065af0 0x7f0bb8067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:17.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.028+0000 7f0bcad9d640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bb8065af0 0x7f0bb8067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0cd160 tx=0x7f0bbc009190 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:17.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.028+0000 7f0bb37fe640 1 -- 172.21.15.26:0/2738236766 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0bc0050110 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.220+0000 7f0bd102d640 1 -- 172.21.15.26:0/2738236766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0bcc074270 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.840+0000 7f0bb37fe640 1 -- 172.21.15.26:0/2738236766 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0bc0017070 con 0x7f0bcc0cb5d0 2023-12-19T16:58:17.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.840+0000 7f0bd102d640 1 -- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bb8065af0 msgr2=0x7f0bb8067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:17.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.840+0000 7f0bd102d640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bb8065af0 0x7f0bb8067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0cd160 tx=0x7f0bbc009190 comp rx=0 tx=0).stop 2023-12-19T16:58:17.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.840+0000 7f0bd102d640 1 -- 172.21.15.26:0/2738236766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bcc0cb5d0 msgr2=0x7f0bcc0cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:17.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.840+0000 7f0bd102d640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bcc0cb5d0 0x7f0bcc0cc4f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000c3c0 tx=0x7f0bc000c880 comp rx=0 tx=0).stop 2023-12-19T16:58:17.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.840+0000 7f0bd102d640 1 -- 172.21.15.26:0/2738236766 shutdown_connections 2023-12-19T16:58:17.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.840+0000 7f0bd102d640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bb8065af0 0x7f0bb8067fb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:17.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.840+0000 7f0bd102d640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bcc1585f0 0x7f0bcc14dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:17.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.840+0000 7f0bd102d640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bcc0cb5d0 0x7f0bcc0cc4f0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:17.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.840+0000 7f0bd102d640 1 --2- 172.21.15.26:0/2738236766 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bcc0cb150 0x7f0bcc0af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:17.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.840+0000 7f0bd102d640 1 -- 172.21.15.26:0/2738236766 >> 172.21.15.26:0/2738236766 conn(0x7f0bcc066160 msgr2=0x7f0bcc150c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:17.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.840+0000 7f0bd102d640 1 -- 172.21.15.26:0/2738236766 shutdown_connections 2023-12-19T16:58:17.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:17.840+0000 7f0bd102d640 1 -- 172.21.15.26:0/2738236766 wait complete. 2023-12-19T16:58:17.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T16:58:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.052+0000 7f819358f640 1 Processor -- start 2023-12-19T16:58:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.052+0000 7f819358f640 1 -- start start 2023-12-19T16:58:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f819358f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f818c0ada30 0x7f818c06c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f819358f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c0c1a50 0x7f818c0ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f819358f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818c0cce50 con 0x7f818c0cd2e0 2023-12-19T16:58:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f819358f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c161800 con 0x7f818c0ada30 2023-12-19T16:58:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f819358f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0cc310 con 0x7f818c0c1a50 2023-12-19T16:58:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8190b2a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f818c0cd2e0 0x7f818c0c1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56816/0 (socket says 172.21.15.26:56816) 2023-12-19T16:58:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8190b2a640 1 -- 172.21.15.26:0/439162743 learned_addr learned my addr 172.21.15.26:0/439162743 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8191b2c640 1 --2- 172.21.15.26:0/439162743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c0c1a50 0x7f818c0ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f819132b640 1 --2- 172.21.15.26:0/439162743 >> v2:172.21.15.160:3300/0 conn(0x7f818c0ada30 0x7f818c06c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195132919 0 0) 0x7f818c0cce50 con 0x7f818c0cd2e0 2023-12-19T16:58:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8178003530 con 0x7f818c0cd2e0 2023-12-19T16:58:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1520637453 0 0) 0x7f8178003530 con 0x7f818c0cd2e0 2023-12-19T16:58:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c0c1a50 msgr2=0x7f818c0ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 --2- 172.21.15.26:0/439162743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c0c1a50 0x7f818c0ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 >> v2:172.21.15.160:3300/0 conn(0x7f818c0ada30 msgr2=0x7f818c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 --2- 172.21.15.26:0/439162743 >> v2:172.21.15.160:3300/0 conn(0x7f818c0ada30 0x7f818c06c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c0cd740 con 0x7f818c0cd2e0 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f817c001680 con 0x7f818c0cd2e0 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 >> v1:172.21.15.26:6789/0 conn(0x7f818c0cd2e0 legacy=0x7f818c0c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 --2- 172.21.15.26:0/439162743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c0c1a50 0x7f8178003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 --2- 172.21.15.26:0/439162743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8178003f00 0x7f81780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 --2- 172.21.15.26:0/439162743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8178004d10 0x7f8178005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c161800 con 0x7f8178003f00 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0cc310 con 0x7f8178004d10 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817800b990 con 0x7f818c0c1a50 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8190b2a640 1 --2- 172.21.15.26:0/439162743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8178003f00 0x7f81780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f819132b640 1 --2- 172.21.15.26:0/439162743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c0c1a50 0x7f8178003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8191b2c640 1 --2- 172.21.15.26:0/439162743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8178004d10 0x7f8178005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8190b2a640 1 -- 172.21.15.26:0/439162743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c0c1a50 msgr2=0x7f8178003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8190b2a640 1 --2- 172.21.15.26:0/439162743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c0c1a50 0x7f8178003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8190b2a640 1 -- 172.21.15.26:0/439162743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8178004d10 msgr2=0x7f8178005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8190b2a640 1 --2- 172.21.15.26:0/439162743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8178004d10 0x7f8178005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.056+0000 7f8190b2a640 1 -- 172.21.15.26:0/439162743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8178003530 con 0x7f8178003f00 2023-12-19T16:58:18.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.060+0000 7f8190b2a640 1 --2- 172.21.15.26:0/439162743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8178003f00 0x7f81780047d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f817c00c210 tx=0x7f817c01f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7627a0db6ec69364 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.060+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c005ae0 con 0x7f8178003f00 2023-12-19T16:58:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.060+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c0ab070 con 0x7f8178003f00 2023-12-19T16:58:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.060+0000 7f8183fff640 1 -- 172.21.15.26:0/439162743 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c005e80 con 0x7f8178003f00 2023-12-19T16:58:18.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.060+0000 7f819358f640 1 -- 172.21.15.26:0/439162743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8178003f00 msgr2=0x7f81780047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.060+0000 7f819358f640 1 --2- 172.21.15.26:0/439162743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8178003f00 0x7f81780047d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f817c00c210 tx=0x7f817c01f330 comp rx=0 tx=0).stop 2023-12-19T16:58:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.060+0000 7f8191b2c640 1 -- 172.21.15.26:0/439162743 reap_dead start 2023-12-19T16:58:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.060+0000 7f819358f640 1 -- 172.21.15.26:0/439162743 shutdown_connections 2023-12-19T16:58:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.060+0000 7f819358f640 1 -- 172.21.15.26:0/439162743 >> 172.21.15.26:0/439162743 conn(0x7f818c0b0270 msgr2=0x7f818c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.060+0000 7f819358f640 1 -- 172.21.15.26:0/439162743 shutdown_connections 2023-12-19T16:58:18.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.060+0000 7f819358f640 1 -- 172.21.15.26:0/439162743 wait complete. 2023-12-19T16:58:18.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819358f640 1 Processor -- start 2023-12-19T16:58:18.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819358f640 1 -- start start 2023-12-19T16:58:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819358f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8178003f00 0x7f818c1f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819358f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f818c0b2310 0x7f818c0c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819358f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c1f3ef0 0x7f818c0c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819358f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0b2790 con 0x7f8178003f00 2023-12-19T16:58:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819358f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c0c65d0 con 0x7f818c0b2310 2023-12-19T16:58:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819358f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c1626e0 con 0x7f818c1f3ef0 2023-12-19T16:58:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819132b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8178003f00 0x7f818c1f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819132b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8178003f00 0x7f818c1f75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44884/0 (socket says 172.21.15.26:44884) 2023-12-19T16:58:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819132b640 1 -- 172.21.15.26:0/4088205722 learned_addr learned my addr 172.21.15.26:0/4088205722 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f8190b2a640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f818c0b2310 0x7f818c0c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f8191b2c640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c1f3ef0 0x7f818c0c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819132b640 1 -- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c1f3ef0 msgr2=0x7f818c0c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:18.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819132b640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c1f3ef0 0x7f818c0c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:18.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819132b640 1 -- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f818c0b2310 msgr2=0x7f818c0c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:18.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819132b640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f818c0b2310 0x7f818c0c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:18.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819132b640 1 -- 172.21.15.26:0/4088205722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c1642f0 con 0x7f8178003f00 2023-12-19T16:58:18.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f8190b2a640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f818c0b2310 0x7f818c0c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:18.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f8191b2c640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c1f3ef0 0x7f818c0c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:18.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819132b640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8178003f00 0x7f818c1f75f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8188011a00 tx=0x7f8188011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:18.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f8181ffb640 1 -- 172.21.15.26:0/4088205722 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818800fb50 con 0x7f8178003f00 2023-12-19T16:58:18.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f8181ffb640 1 -- 172.21.15.26:0/4088205722 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818800fcb0 con 0x7f8178003f00 2023-12-19T16:58:18.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f8181ffb640 1 -- 172.21.15.26:0/4088205722 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188014440 con 0x7f8178003f00 2023-12-19T16:58:18.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819358f640 1 -- 172.21.15.26:0/4088205722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c0c6800 con 0x7f8178003f00 2023-12-19T16:58:18.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.064+0000 7f819358f640 1 -- 172.21.15.26:0/4088205722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c164590 con 0x7f8178003f00 2023-12-19T16:58:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.068+0000 7f819358f640 1 -- 172.21.15.26:0/4088205722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818c074270 con 0x7f8178003f00 2023-12-19T16:58:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.068+0000 7f8181ffb640 1 -- 172.21.15.26:0/4088205722 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f81880145a0 con 0x7f8178003f00 2023-12-19T16:58:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.068+0000 7f8181ffb640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8170065b20 0x7f8170067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.068+0000 7f8181ffb640 1 -- 172.21.15.26:0/4088205722 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f8188084040 con 0x7f8178003f00 2023-12-19T16:58:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.068+0000 7f8190b2a640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8170065b20 0x7f8170067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:18.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.068+0000 7f8190b2a640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8170065b20 0x7f8170067fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f817c0ae000 tx=0x7f817c000ff0 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:18.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.072+0000 7f8181ffb640 1 -- 172.21.15.26:0/4088205722 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8188051550 con 0x7f8178003f00 2023-12-19T16:58:18.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.232+0000 7f819358f640 1 -- 172.21.15.26:0/4088205722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f818c06c130 con 0x7f8178003f00 2023-12-19T16:58:18.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.848+0000 7f8181ffb640 1 -- 172.21.15.26:0/4088205722 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8188051920 con 0x7f8178003f00 2023-12-19T16:58:18.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.852+0000 7f8181ffb640 1 -- 172.21.15.26:0/4088205722 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8188054fe0 con 0x7f8178003f00 2023-12-19T16:58:18.852 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T16:58:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.852+0000 7f819358f640 1 -- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8170065b20 msgr2=0x7f8170067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.852+0000 7f819358f640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8170065b20 0x7f8170067fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f817c0ae000 tx=0x7f817c000ff0 comp rx=0 tx=0).stop 2023-12-19T16:58:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.852+0000 7f819358f640 1 -- 172.21.15.26:0/4088205722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8178003f00 msgr2=0x7f818c1f75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.852+0000 7f819358f640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8178003f00 0x7f818c1f75f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f8188011a00 tx=0x7f8188011ec0 comp rx=0 tx=0).stop 2023-12-19T16:58:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.852+0000 7f819358f640 1 -- 172.21.15.26:0/4088205722 shutdown_connections 2023-12-19T16:58:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.852+0000 7f819358f640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8170065b20 0x7f8170067fe0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.852+0000 7f819358f640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f818c1f3ef0 0x7f818c0c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.852+0000 7f819358f640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f818c0b2310 0x7f818c0c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.852+0000 7f819358f640 1 --2- 172.21.15.26:0/4088205722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8178003f00 0x7f818c1f75f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:18.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.852+0000 7f819358f640 1 -- 172.21.15.26:0/4088205722 >> 172.21.15.26:0/4088205722 conn(0x7f818c0b0270 msgr2=0x7f818c06f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.852+0000 7f819358f640 1 -- 172.21.15.26:0/4088205722 shutdown_connections 2023-12-19T16:58:18.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:18.856+0000 7f819358f640 1 -- 172.21.15.26:0/4088205722 wait complete. 2023-12-19T16:58:18.878 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T16:58:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.028+0000 7fa4cedf3640 1 Processor -- start 2023-12-19T16:58:19.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.028+0000 7fa4cedf3640 1 -- start start 2023-12-19T16:58:19.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4cedf3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa4c80b3f90 0x7fa4c806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:19.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4cedf3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c806cbf0 con 0x7fa4c80b1590 2023-12-19T16:58:19.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4cedf3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c80ca8b0 con 0x7fa4c815aad0 2023-12-19T16:58:19.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4cedf3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c806b6f0 con 0x7fa4c80b3f90 2023-12-19T16:58:19.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa4c80b1590 0x7fa4c80cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56830/0 (socket says 172.21.15.26:56830) 2023-12-19T16:58:19.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c7fff640 1 -- 172.21.15.26:0/2672705130 learned_addr learned my addr 172.21.15.26:0/2672705130 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:19.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4cd390640 1 --2- 172.21.15.26:0/2672705130 >> v2:172.21.15.160:3300/0 conn(0x7fa4c80b3f90 0x7fa4c806c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:19.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123897066 0 0) 0x7fa4b4001120 con 0x7fa4c80b1590 2023-12-19T16:58:19.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4b8003530 con 0x7fa4c80b1590 2023-12-19T16:58:19.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401981851 0 0) 0x7fa4c80ca8b0 con 0x7fa4c815aad0 2023-12-19T16:58:19.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4b4001120 con 0x7fa4c815aad0 2023-12-19T16:58:19.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1357963502 0 0) 0x7fa4b8003530 con 0x7fa4c80b1590 2023-12-19T16:58:19.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 >> v2:172.21.15.160:3300/0 conn(0x7fa4c80b3f90 msgr2=0x7fa4c806c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:19.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 --2- 172.21.15.26:0/2672705130 >> v2:172.21.15.160:3300/0 conn(0x7fa4c80b3f90 0x7fa4c806c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:19.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 >> v1:172.21.15.160:6789/0 conn(0x7fa4c815aad0 legacy=0x7fa4c8070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:19.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c806b9d0 con 0x7fa4c80b1590 2023-12-19T16:58:19.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fa4b4001810 con 0x7fa4c80b1590 2023-12-19T16:58:19.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 >> v1:172.21.15.26:6789/0 conn(0x7fa4c80b1590 legacy=0x7fa4c80cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:19.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 --2- 172.21.15.26:0/2672705130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b8004b70 0x7fa4b8004f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:19.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 --2- 172.21.15.26:0/2672705130 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4b8003770 0x7fa4b8003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:19.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 --2- 172.21.15.26:0/2672705130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b8004130 0x7fa4b800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:19.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c806b6f0 con 0x7fa4b8004b70 2023-12-19T16:58:19.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b800c010 con 0x7fa4b8004130 2023-12-19T16:58:19.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b800c1d0 con 0x7fa4b8003770 2023-12-19T16:58:19.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4ccb8f640 1 --2- 172.21.15.26:0/2672705130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b8004b70 0x7fa4b8004f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:19.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4c7fff640 1 --2- 172.21.15.26:0/2672705130 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4b8003770 0x7fa4b8003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:19.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4cd390640 1 --2- 172.21.15.26:0/2672705130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b8004130 0x7fa4b800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:19.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4ccb8f640 1 -- 172.21.15.26:0/2672705130 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4b8003770 msgr2=0x7fa4b8003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:19.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4ccb8f640 1 --2- 172.21.15.26:0/2672705130 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4b8003770 0x7fa4b8003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:19.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4ccb8f640 1 -- 172.21.15.26:0/2672705130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b8004130 msgr2=0x7fa4b800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:19.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4ccb8f640 1 --2- 172.21.15.26:0/2672705130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b8004130 0x7fa4b800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:19.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4ccb8f640 1 -- 172.21.15.26:0/2672705130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b8003530 con 0x7fa4b8004b70 2023-12-19T16:58:19.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4cd390640 1 --2- 172.21.15.26:0/2672705130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b8004130 0x7fa4b800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:19.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.032+0000 7fa4cd390640 1 -- 172.21.15.26:0/2672705130 reap_dead start 2023-12-19T16:58:19.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.036+0000 7fa4ccb8f640 1 --2- 172.21.15.26:0/2672705130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b8004b70 0x7fa4b8004f70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0018d0 tx=0x7fa4bc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12d21922f3d2350 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:19.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.036+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0a6040 con 0x7fa4b8004b70 2023-12-19T16:58:19.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.036+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4bc0052f0 con 0x7fa4b8004b70 2023-12-19T16:58:19.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.036+0000 7fa4c77fe640 1 -- 172.21.15.26:0/2672705130 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc003600 con 0x7fa4b8004b70 2023-12-19T16:58:19.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.036+0000 7fa4cedf3640 1 -- 172.21.15.26:0/2672705130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b8004b70 msgr2=0x7fa4b8004f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:19.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.036+0000 7fa4cedf3640 1 --2- 172.21.15.26:0/2672705130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b8004b70 0x7fa4b8004f70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0018d0 tx=0x7fa4bc01c4d0 comp rx=0 tx=0).stop 2023-12-19T16:58:19.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.036+0000 7fa4cedf3640 1 -- 172.21.15.26:0/2672705130 shutdown_connections 2023-12-19T16:58:19.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.036+0000 7fa4cedf3640 1 --2- 172.21.15.26:0/2672705130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b8004b70 0x7fa4b8004f70 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:19.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.036+0000 7fa4cedf3640 1 -- 172.21.15.26:0/2672705130 >> 172.21.15.26:0/2672705130 conn(0x7fa4c80b0270 msgr2=0x7fa4c80b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:19.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.036+0000 7fa4cedf3640 1 -- 172.21.15.26:0/2672705130 shutdown_connections 2023-12-19T16:58:19.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.036+0000 7fa4cedf3640 1 -- 172.21.15.26:0/2672705130 wait complete. 2023-12-19T16:58:19.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.036+0000 7fa4cedf3640 1 Processor -- start 2023-12-19T16:58:19.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4cedf3640 1 -- start start 2023-12-19T16:58:19.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4cedf3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4b8004b70 0x7fa4c80c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:19.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4cedf3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4c8157b50 0x7fa4c81580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:19.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4cedf3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4c81585e0 0x7fa4c80c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:19.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4cedf3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c8158a60 con 0x7fa4c8157b50 2023-12-19T16:58:19.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4cedf3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80c8dc0 con 0x7fa4c81585e0 2023-12-19T16:58:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4cedf3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80c9600 con 0x7fa4b8004b70 2023-12-19T16:58:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4c8157b50 0x7fa4c81580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4c8157b50 0x7fa4c81580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44900/0 (socket says 172.21.15.26:44900) 2023-12-19T16:58:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4c7fff640 1 -- 172.21.15.26:0/3477102887 learned_addr learned my addr 172.21.15.26:0/3477102887 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4ccb8f640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4b8004b70 0x7fa4c80c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4cd390640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4c81585e0 0x7fa4c80c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4c7fff640 1 -- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4b8004b70 msgr2=0x7fa4c80c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4c7fff640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4b8004b70 0x7fa4c80c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4c7fff640 1 -- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4c81585e0 msgr2=0x7fa4c80c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4c7fff640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4c81585e0 0x7fa4c80c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4c7fff640 1 -- 172.21.15.26:0/3477102887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c80c9040 con 0x7fa4c8157b50 2023-12-19T16:58:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4c7fff640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4c8157b50 0x7fa4c81580a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4007340 tx=0x7fa4b4007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:19.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4c57fa640 1 -- 172.21.15.26:0/3477102887 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b400fa20 con 0x7fa4c8157b50 2023-12-19T16:58:19.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4c57fa640 1 -- 172.21.15.26:0/3477102887 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b400fb80 con 0x7fa4c8157b50 2023-12-19T16:58:19.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4cedf3640 1 -- 172.21.15.26:0/3477102887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c80c9320 con 0x7fa4c8157b50 2023-12-19T16:58:19.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4c57fa640 1 -- 172.21.15.26:0/3477102887 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b40133f0 con 0x7fa4c8157b50 2023-12-19T16:58:19.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4cedf3640 1 -- 172.21.15.26:0/3477102887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c80c65a0 con 0x7fa4c8157b50 2023-12-19T16:58:19.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.040+0000 7fa4cedf3640 1 -- 172.21.15.26:0/3477102887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c8074270 con 0x7fa4c8157b50 2023-12-19T16:58:19.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.044+0000 7fa4c57fa640 1 -- 172.21.15.26:0/3477102887 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa4b4013550 con 0x7fa4c8157b50 2023-12-19T16:58:19.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.044+0000 7fa4c57fa640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4ac065b80 0x7fa4ac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:19.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.044+0000 7fa4c57fa640 1 -- 172.21.15.26:0/3477102887 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6691+0+0 (secure 0 0 0) 0x7fa4b40839f0 con 0x7fa4c8157b50 2023-12-19T16:58:19.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.044+0000 7fa4ccb8f640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4ac065b80 0x7fa4ac068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:19.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.044+0000 7fa4ccb8f640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4ac065b80 0x7fa4ac068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0ac000 tx=0x7fa4bc0a7000 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:19.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.048+0000 7fa4c57fa640 1 -- 172.21.15.26:0/3477102887 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa4b4050d90 con 0x7fa4c8157b50 2023-12-19T16:58:19.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.204+0000 7fa4cedf3640 1 -- 172.21.15.26:0/3477102887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa4c806cbf0 con 0x7fa4c8157b50 2023-12-19T16:58:19.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.848+0000 7fa4c57fa640 1 -- 172.21.15.26:0/3477102887 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa4c806cbf0 con 0x7fa4c8157b50 2023-12-19T16:58:19.850 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T16:58:19.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.852+0000 7fa4cedf3640 1 -- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4ac065b80 msgr2=0x7fa4ac068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:19.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.852+0000 7fa4cedf3640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4ac065b80 0x7fa4ac068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0ac000 tx=0x7fa4bc0a7000 comp rx=0 tx=0).stop 2023-12-19T16:58:19.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.852+0000 7fa4cedf3640 1 -- 172.21.15.26:0/3477102887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4c8157b50 msgr2=0x7fa4c81580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:19.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.852+0000 7fa4cedf3640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4c8157b50 0x7fa4c81580a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fa4b4007340 tx=0x7fa4b4007800 comp rx=0 tx=0).stop 2023-12-19T16:58:19.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.852+0000 7fa4cedf3640 1 -- 172.21.15.26:0/3477102887 shutdown_connections 2023-12-19T16:58:19.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.852+0000 7fa4cedf3640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4c81585e0 0x7fa4c80c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:19.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.852+0000 7fa4cedf3640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4c8157b50 0x7fa4c81580a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:19.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.852+0000 7fa4cedf3640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa4ac065b80 0x7fa4ac068040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:19.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.852+0000 7fa4cedf3640 1 --2- 172.21.15.26:0/3477102887 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4b8004b70 0x7fa4c80c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:19.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.852+0000 7fa4cedf3640 1 -- 172.21.15.26:0/3477102887 >> 172.21.15.26:0/3477102887 conn(0x7fa4c80b0270 msgr2=0x7fa4c8159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:19.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.852+0000 7fa4cedf3640 1 -- 172.21.15.26:0/3477102887 shutdown_connections 2023-12-19T16:58:19.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:19.852+0000 7fa4cedf3640 1 -- 172.21.15.26:0/3477102887 wait complete. 2023-12-19T16:58:19.878 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-19T16:58:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f52848b6640 1 Processor -- start 2023-12-19T16:58:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f52848b6640 1 -- start start 2023-12-19T16:58:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f52848b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52800cb860 0x7f528006f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f52848b6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f528015d530 0x7f528015d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f52848b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528015e240 con 0x7f52801599e0 2023-12-19T16:58:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f52848b6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52800c9520 con 0x7f528015d530 2023-12-19T16:58:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f52848b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52800c9800 con 0x7f52800cb860 2023-12-19T16:58:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f527e59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f52801599e0 0x7f52800cb150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56842/0 (socket says 172.21.15.26:56842) 2023-12-19T16:58:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f527e59c640 1 -- 172.21.15.26:0/2571219683 learned_addr learned my addr 172.21.15.26:0/2571219683 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f527ed9d640 1 --2- 172.21.15.26:0/2571219683 >> v2:172.21.15.160:3300/0 conn(0x7f528015d530 0x7f528015d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f527dd9b640 1 --2- 172.21.15.26:0/2571219683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52800cb860 0x7f528006f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f527d59a640 1 -- 172.21.15.26:0/2571219683 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507715805 0 0) 0x7f528015e240 con 0x7f52801599e0 2023-12-19T16:58:20.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f527d59a640 1 -- 172.21.15.26:0/2571219683 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5268005740 con 0x7f52801599e0 2023-12-19T16:58:20.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f527dd9b640 1 -- 172.21.15.26:0/2571219683 >> v2:172.21.15.160:3300/0 conn(0x7f528015d530 msgr2=0x7f528015d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:20.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f527dd9b640 1 --2- 172.21.15.26:0/2571219683 >> v2:172.21.15.160:3300/0 conn(0x7f528015d530 0x7f528015d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f527dd9b640 1 -- 172.21.15.26:0/2571219683 >> v1:172.21.15.26:6789/0 conn(0x7f52801599e0 legacy=0x7f52800cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f527dd9b640 1 -- 172.21.15.26:0/2571219683 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52800c9ae0 con 0x7f52800cb860 2023-12-19T16:58:20.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.028+0000 7f527ed9d640 1 --2- 172.21.15.26:0/2571219683 >> v2:172.21.15.160:3300/0 conn(0x7f528015d530 0x7f528015d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:20.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.032+0000 7f527dd9b640 1 --2- 172.21.15.26:0/2571219683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52800cb860 0x7f528006f6d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f52740097c0 tx=0x7f527401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c2160c6562d440a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.032+0000 7f527d59a640 1 -- 172.21.15.26:0/2571219683 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52740a6040 con 0x7f52800cb860 2023-12-19T16:58:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.032+0000 7f527d59a640 1 -- 172.21.15.26:0/2571219683 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52740aa600 con 0x7f52800cb860 2023-12-19T16:58:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.032+0000 7f527d59a640 1 -- 172.21.15.26:0/2571219683 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52740aa990 con 0x7f52800cb860 2023-12-19T16:58:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.032+0000 7f52848b6640 1 -- 172.21.15.26:0/2571219683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52800cb860 msgr2=0x7f528006f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.032+0000 7f52848b6640 1 --2- 172.21.15.26:0/2571219683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52800cb860 0x7f528006f6d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f52740097c0 tx=0x7f527401c480 comp rx=0 tx=0).stop 2023-12-19T16:58:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.032+0000 7f52848b6640 1 -- 172.21.15.26:0/2571219683 shutdown_connections 2023-12-19T16:58:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.032+0000 7f52848b6640 1 --2- 172.21.15.26:0/2571219683 >> v2:172.21.15.160:3300/0 conn(0x7f528015d530 0x7f528015d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.032+0000 7f52848b6640 1 --2- 172.21.15.26:0/2571219683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52800cb860 0x7f528006f6d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.032+0000 7f52848b6640 1 -- 172.21.15.26:0/2571219683 >> 172.21.15.26:0/2571219683 conn(0x7f5280066160 msgr2=0x7f52800b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.032+0000 7f52848b6640 1 -- 172.21.15.26:0/2571219683 shutdown_connections 2023-12-19T16:58:20.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.032+0000 7f52848b6640 1 -- 172.21.15.26:0/2571219683 wait complete. 2023-12-19T16:58:20.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f52848b6640 1 Processor -- start 2023-12-19T16:58:20.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f52848b6640 1 -- start start 2023-12-19T16:58:20.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f52848b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52800cb860 0x7f528015b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f52848b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52801599e0 0x7f528014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f52848b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f528015d530 0x7f528015c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f52848b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528015b6e0 con 0x7f52801599e0 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f52848b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528015b9c0 con 0x7f52800cb860 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f52848b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528015bca0 con 0x7f528015d530 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f527dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52801599e0 0x7f528014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f527dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52801599e0 0x7f528014fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44902/0 (socket says 172.21.15.26:44902) 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f527dd9b640 1 -- 172.21.15.26:0/4262765594 learned_addr learned my addr 172.21.15.26:0/4262765594 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f527ed9d640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f528015d530 0x7f528015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f527e59c640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52800cb860 0x7f528015b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f527dd9b640 1 -- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f528015d530 msgr2=0x7f528015c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f527dd9b640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f528015d530 0x7f528015c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f527dd9b640 1 -- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52800cb860 msgr2=0x7f528015b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f527dd9b640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52800cb860 0x7f528015b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f527dd9b640 1 -- 172.21.15.26:0/4262765594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5278009a70 con 0x7f52801599e0 2023-12-19T16:58:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f527ed9d640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f528015d530 0x7f528015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:20.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f527e59c640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52800cb860 0x7f528015b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:20.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f527dd9b640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52801599e0 0x7f528014fbe0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f527401a0b0 tx=0x7f5274019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:20.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f526effd640 1 -- 172.21.15.26:0/4262765594 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52740b3600 con 0x7f52801599e0 2023-12-19T16:58:20.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f526effd640 1 -- 172.21.15.26:0/4262765594 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52740b3760 con 0x7f52801599e0 2023-12-19T16:58:20.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f52848b6640 1 -- 172.21.15.26:0/4262765594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5274009420 con 0x7f52801599e0 2023-12-19T16:58:20.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f526effd640 1 -- 172.21.15.26:0/4262765594 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52740b3af0 con 0x7f52801599e0 2023-12-19T16:58:20.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.036+0000 7f52848b6640 1 -- 172.21.15.26:0/4262765594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528015c260 con 0x7f52801599e0 2023-12-19T16:58:20.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.040+0000 7f52848b6640 1 -- 172.21.15.26:0/4262765594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528015c450 con 0x7f52801599e0 2023-12-19T16:58:20.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.040+0000 7f526effd640 1 -- 172.21.15.26:0/4262765594 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f52740b3c50 con 0x7f52801599e0 2023-12-19T16:58:20.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.040+0000 7f526effd640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5260065b80 0x7f5260068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.040+0000 7f526effd640 1 -- 172.21.15.26:0/4262765594 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f527411dcf0 con 0x7f52801599e0 2023-12-19T16:58:20.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.040+0000 7f527e59c640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5260065b80 0x7f5260068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.040+0000 7f527e59c640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5260065b80 0x7f5260068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5270005890 tx=0x7f5270009040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.044+0000 7f526effd640 1 -- 172.21.15.26:0/4262765594 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f52740eb240 con 0x7f52801599e0 2023-12-19T16:58:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.204+0000 7f52848b6640 1 -- 172.21.15.26:0/4262765594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f5280074270 con 0x7f52801599e0 2023-12-19T16:58:20.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.208+0000 7f526effd640 1 -- 172.21.15.26:0/4262765594 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v31) v1 ==== 120+0+0 (secure 0 0 0) 0x7f52740ee860 con 0x7f52801599e0 2023-12-19T16:58:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.212+0000 7f52848b6640 1 -- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5260065b80 msgr2=0x7f5260068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.212+0000 7f52848b6640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5260065b80 0x7f5260068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5270005890 tx=0x7f5270009040 comp rx=0 tx=0).stop 2023-12-19T16:58:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.212+0000 7f52848b6640 1 -- 172.21.15.26:0/4262765594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52801599e0 msgr2=0x7f528014fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.212+0000 7f52848b6640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52801599e0 0x7f528014fbe0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f527401a0b0 tx=0x7f5274019220 comp rx=0 tx=0).stop 2023-12-19T16:58:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.212+0000 7f52848b6640 1 -- 172.21.15.26:0/4262765594 shutdown_connections 2023-12-19T16:58:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.212+0000 7f52848b6640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5260065b80 0x7f5260068040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.212+0000 7f52848b6640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f528015d530 0x7f528015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.212+0000 7f52848b6640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52801599e0 0x7f528014fbe0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.212+0000 7f52848b6640 1 --2- 172.21.15.26:0/4262765594 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52800cb860 0x7f528015b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.212+0000 7f52848b6640 1 -- 172.21.15.26:0/4262765594 >> 172.21.15.26:0/4262765594 conn(0x7f5280066160 msgr2=0x7f52800cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.212+0000 7f52848b6640 1 -- 172.21.15.26:0/4262765594 shutdown_connections 2023-12-19T16:58:20.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.212+0000 7f52848b6640 1 -- 172.21.15.26:0/4262765594 wait complete. 2023-12-19T16:58:20.247 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T16:58:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0e301f640 1 Processor -- start 2023-12-19T16:58:20.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0e301f640 1 -- start start 2023-12-19T16:58:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0e301f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc1585f0 0x7fe0dc0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0e301f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe0dc0cb150 0x7fe0dc0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0e301f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0dc15b1b0 con 0x7fe0dc0cc310 2023-12-19T16:58:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0e301f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0dc0cbb10 con 0x7fe0dc0cb150 2023-12-19T16:58:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0e301f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0b47a0 con 0x7fe0dc1585f0 2023-12-19T16:58:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0dbfff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe0dc0cb150 0x7fe0dc0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0e0dbb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc1585f0 0x7fe0dc0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0e15bc640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe0dc0cc310 0x7fe0dc0cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:38128/0 (socket says 172.21.15.26:38128) 2023-12-19T16:58:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0e15bc640 1 -- 172.21.15.26:0/1128340934 learned_addr learned my addr 172.21.15.26:0/1128340934 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:20.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0dbfff640 1 -- 172.21.15.26:0/1128340934 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc1585f0 msgr2=0x7fe0dc0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0dbfff640 1 --2- 172.21.15.26:0/1128340934 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc1585f0 0x7fe0dc0cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0dbfff640 1 -- 172.21.15.26:0/1128340934 >> v1:172.21.15.160:6789/0 conn(0x7fe0dc0cc310 legacy=0x7fe0dc0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.392+0000 7fe0dbfff640 1 -- 172.21.15.26:0/1128340934 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc15b490 con 0x7fe0dc0cb150 2023-12-19T16:58:20.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.396+0000 7fe0dbfff640 1 --2- 172.21.15.26:0/1128340934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0cb150 0x7fe0dc0cb5d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80097c0 tx=0x7fe0c801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691c89637c8f5553 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.396+0000 7fe0db7fe640 1 -- 172.21.15.26:0/1128340934 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80ac620 con 0x7fe0dc0cb150 2023-12-19T16:58:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.396+0000 7fe0db7fe640 1 -- 172.21.15.26:0/1128340934 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c80ac780 con 0x7fe0dc0cb150 2023-12-19T16:58:20.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.396+0000 7fe0db7fe640 1 -- 172.21.15.26:0/1128340934 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80acb10 con 0x7fe0dc0cb150 2023-12-19T16:58:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.396+0000 7fe0e301f640 1 -- 172.21.15.26:0/1128340934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0cb150 msgr2=0x7fe0dc0cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.396+0000 7fe0e301f640 1 --2- 172.21.15.26:0/1128340934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0cb150 0x7fe0dc0cb5d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80097c0 tx=0x7fe0c801c3c0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.396+0000 7fe0e301f640 1 -- 172.21.15.26:0/1128340934 shutdown_connections 2023-12-19T16:58:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.396+0000 7fe0e301f640 1 --2- 172.21.15.26:0/1128340934 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0cb150 0x7fe0dc0cb5d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.396+0000 7fe0e301f640 1 --2- 172.21.15.26:0/1128340934 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc1585f0 0x7fe0dc0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.396+0000 7fe0e301f640 1 -- 172.21.15.26:0/1128340934 >> 172.21.15.26:0/1128340934 conn(0x7fe0dc066160 msgr2=0x7fe0dc0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.396+0000 7fe0e301f640 1 -- 172.21.15.26:0/1128340934 shutdown_connections 2023-12-19T16:58:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.396+0000 7fe0e301f640 1 -- 172.21.15.26:0/1128340934 wait complete. 2023-12-19T16:58:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e301f640 1 Processor -- start 2023-12-19T16:58:20.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e301f640 1 -- start start 2023-12-19T16:58:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e301f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc0cb150 0x7fe0dc14e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e301f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0cc310 0x7fe0dc0c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e301f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc1585f0 0x7fe0dc0c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e301f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0c7ef0 con 0x7fe0dc1585f0 2023-12-19T16:58:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e301f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0c72e0 con 0x7fe0dc0cc310 2023-12-19T16:58:20.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e301f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc0c75c0 con 0x7fe0dc0cb150 2023-12-19T16:58:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e15bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc1585f0 0x7fe0dc0c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e15bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc1585f0 0x7fe0dc0c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44914/0 (socket says 172.21.15.26:44914) 2023-12-19T16:58:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e15bc640 1 -- 172.21.15.26:0/1389067359 learned_addr learned my addr 172.21.15.26:0/1389067359 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0dbfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0cc310 0x7fe0dc0c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e0dbb640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc0cb150 0x7fe0dc14e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e15bc640 1 -- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc0cb150 msgr2=0x7fe0dc14e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e15bc640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc0cb150 0x7fe0dc14e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e15bc640 1 -- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0cc310 msgr2=0x7fe0dc0c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e15bc640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0cc310 0x7fe0dc0c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e15bc640 1 -- 172.21.15.26:0/1389067359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c8009420 con 0x7fe0dc1585f0 2023-12-19T16:58:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0dbfff640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0cc310 0x7fe0dc0c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e15bc640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc1585f0 0x7fe0dc0c6240 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400ca80 tx=0x7fe0d400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0d97fa640 1 -- 172.21.15.26:0/1389067359 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400ac30 con 0x7fe0dc1585f0 2023-12-19T16:58:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0d97fa640 1 -- 172.21.15.26:0/1389067359 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d400ad90 con 0x7fe0dc1585f0 2023-12-19T16:58:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0d97fa640 1 -- 172.21.15.26:0/1389067359 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4012600 con 0x7fe0dc1585f0 2023-12-19T16:58:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e301f640 1 -- 172.21.15.26:0/1389067359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc15c3e0 con 0x7fe0dc1585f0 2023-12-19T16:58:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.400+0000 7fe0e301f640 1 -- 172.21.15.26:0/1389067359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc0c7b80 con 0x7fe0dc1585f0 2023-12-19T16:58:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.404+0000 7fe0e301f640 1 -- 172.21.15.26:0/1389067359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc0c8060 con 0x7fe0dc1585f0 2023-12-19T16:58:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.404+0000 7fe0d97fa640 1 -- 172.21.15.26:0/1389067359 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe0d4007070 con 0x7fe0dc1585f0 2023-12-19T16:58:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.404+0000 7fe0d97fa640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe0c0065b80 0x7fe0c0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.404+0000 7fe0d97fa640 1 -- 172.21.15.26:0/1389067359 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fe0d4083440 con 0x7fe0dc1585f0 2023-12-19T16:58:20.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.404+0000 7fe0e0dbb640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe0c0065b80 0x7fe0c0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.404+0000 7fe0e0dbb640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe0c0065b80 0x7fe0c0068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc15eff0 tx=0x7fe0d0009190 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:20.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.408+0000 7fe0d97fa640 1 -- 172.21.15.26:0/1389067359 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe0d4050ac0 con 0x7fe0dc1585f0 2023-12-19T16:58:20.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.568+0000 7fe0e301f640 1 -- 172.21.15.26:0/1389067359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe0dc074270 con 0x7fe0dc1585f0 2023-12-19T16:58:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.572+0000 7fe0d97fa640 1 -- 172.21.15.26:0/1389067359 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe0d40540e0 con 0x7fe0dc1585f0 2023-12-19T16:58:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.576+0000 7fe0e301f640 1 -- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe0c0065b80 msgr2=0x7fe0c0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.576+0000 7fe0e301f640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe0c0065b80 0x7fe0c0068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc15eff0 tx=0x7fe0d0009190 comp rx=0 tx=0).stop 2023-12-19T16:58:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.576+0000 7fe0e301f640 1 -- 172.21.15.26:0/1389067359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc1585f0 msgr2=0x7fe0dc0c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.576+0000 7fe0e301f640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc1585f0 0x7fe0dc0c6240 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400ca80 tx=0x7fe0d400cf40 comp rx=0 tx=0).stop 2023-12-19T16:58:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.576+0000 7fe0e301f640 1 -- 172.21.15.26:0/1389067359 shutdown_connections 2023-12-19T16:58:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.576+0000 7fe0e301f640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0dc1585f0 0x7fe0dc0c6240 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.576+0000 7fe0e301f640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0dc0cc310 0x7fe0dc0c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.576+0000 7fe0e301f640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe0c0065b80 0x7fe0c0068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.576+0000 7fe0e301f640 1 --2- 172.21.15.26:0/1389067359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0dc0cb150 0x7fe0dc14e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.576+0000 7fe0e301f640 1 -- 172.21.15.26:0/1389067359 >> 172.21.15.26:0/1389067359 conn(0x7fe0dc066160 msgr2=0x7fe0dc154ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.576+0000 7fe0e301f640 1 -- 172.21.15.26:0/1389067359 shutdown_connections 2023-12-19T16:58:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.576+0000 7fe0e301f640 1 -- 172.21.15.26:0/1389067359 wait complete. 2023-12-19T16:58:20.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-19T16:58:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.740+0000 7fb048ad1640 1 Processor -- start 2023-12-19T16:58:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.740+0000 7fb048ad1640 1 -- start start 2023-12-19T16:58:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb048ad1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0440cb860 0x7fb04406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb048ad1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb04415d530 0x7fb04415d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb048ad1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04415e240 con 0x7fb0441599e0 2023-12-19T16:58:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb048ad1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0440c9520 con 0x7fb04415d530 2023-12-19T16:58:20.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb048ad1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440c9800 con 0x7fb0440cb860 2023-12-19T16:58:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb042d9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb04415d530 0x7fb04415d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb04259c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fb0441599e0 0x7fb0440cb150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:38134/0 (socket says 172.21.15.26:38134) 2023-12-19T16:58:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb04259c640 1 -- 172.21.15.26:0/2631263755 learned_addr learned my addr 172.21.15.26:0/2631263755 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0440cb860 0x7fb04406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb042d9d640 1 -- 172.21.15.26:0/2631263755 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0440cb860 msgr2=0x7fb04406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb042d9d640 1 --2- 172.21.15.26:0/2631263755 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0440cb860 0x7fb04406f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb042d9d640 1 -- 172.21.15.26:0/2631263755 >> v1:172.21.15.160:6789/0 conn(0x7fb0441599e0 legacy=0x7fb0440cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb042d9d640 1 -- 172.21.15.26:0/2631263755 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0440c9ae0 con 0x7fb04415d530 2023-12-19T16:58:20.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.744+0000 7fb041d9b640 1 -- 172.21.15.26:0/2631263755 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901082021 0 0) 0x7fb04415e240 con 0x7fb0441599e0 2023-12-19T16:58:20.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb042d9d640 1 --2- 172.21.15.26:0/2631263755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb04415d530 0x7fb04415d980 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009d90 tx=0x7fb03c01c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24f3c9aaeeab75a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:20.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb041d9b640 1 -- 172.21.15.26:0/2631263755 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0a5a30 con 0x7fb04415d530 2023-12-19T16:58:20.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb041d9b640 1 -- 172.21.15.26:0/2631263755 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c0a5b90 con 0x7fb04415d530 2023-12-19T16:58:20.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb041d9b640 1 -- 172.21.15.26:0/2631263755 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0ad440 con 0x7fb04415d530 2023-12-19T16:58:20.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb048ad1640 1 -- 172.21.15.26:0/2631263755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb04415d530 msgr2=0x7fb04415d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb048ad1640 1 --2- 172.21.15.26:0/2631263755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb04415d530 0x7fb04415d980 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009d90 tx=0x7fb03c01c950 comp rx=0 tx=0).stop 2023-12-19T16:58:20.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb048ad1640 1 -- 172.21.15.26:0/2631263755 shutdown_connections 2023-12-19T16:58:20.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb048ad1640 1 --2- 172.21.15.26:0/2631263755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb04415d530 0x7fb04415d980 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb048ad1640 1 --2- 172.21.15.26:0/2631263755 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0440cb860 0x7fb04406f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb048ad1640 1 -- 172.21.15.26:0/2631263755 >> 172.21.15.26:0/2631263755 conn(0x7fb044066160 msgr2=0x7fb0440b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:20.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb048ad1640 1 -- 172.21.15.26:0/2631263755 shutdown_connections 2023-12-19T16:58:20.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb048ad1640 1 -- 172.21.15.26:0/2631263755 wait complete. 2023-12-19T16:58:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb048ad1640 1 Processor -- start 2023-12-19T16:58:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.748+0000 7fb048ad1640 1 -- start start 2023-12-19T16:58:20.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb048ad1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0440cb860 0x7fb0440b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb048ad1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0441599e0 0x7fb0440b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb048ad1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb04415d530 0x7fb0440c24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb048ad1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb044155640 con 0x7fb0441599e0 2023-12-19T16:58:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb048ad1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440b42d0 con 0x7fb04415d530 2023-12-19T16:58:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb048ad1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440b2a50 con 0x7fb0440cb860 2023-12-19T16:58:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0441599e0 0x7fb0440b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb04259c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0440cb860 0x7fb0440b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb03bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0441599e0 0x7fb0440b2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44928/0 (socket says 172.21.15.26:44928) 2023-12-19T16:58:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb03bfff640 1 -- 172.21.15.26:0/2239413440 learned_addr learned my addr 172.21.15.26:0/2239413440 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:20.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb042d9d640 1 --2- 172.21.15.26:0/2239413440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb04415d530 0x7fb0440c24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb03bfff640 1 -- 172.21.15.26:0/2239413440 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0440cb860 msgr2=0x7fb0440b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb03bfff640 1 --2- 172.21.15.26:0/2239413440 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0440cb860 0x7fb0440b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb03bfff640 1 -- 172.21.15.26:0/2239413440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb04415d530 msgr2=0x7fb0440c24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb03bfff640 1 --2- 172.21.15.26:0/2239413440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb04415d530 0x7fb0440c24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb03bfff640 1 -- 172.21.15.26:0/2239413440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c009a70 con 0x7fb0441599e0 2023-12-19T16:58:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb03bfff640 1 --2- 172.21.15.26:0/2239413440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0441599e0 0x7fb0440b2510 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e930 tx=0x7fb02c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb03affd640 1 -- 172.21.15.26:0/2239413440 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c009690 con 0x7fb0441599e0 2023-12-19T16:58:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb03affd640 1 -- 172.21.15.26:0/2239413440 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c00ca00 con 0x7fb0441599e0 2023-12-19T16:58:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb03affd640 1 -- 172.21.15.26:0/2239413440 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c00cd70 con 0x7fb0441599e0 2023-12-19T16:58:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb048ad1640 1 -- 172.21.15.26:0/2239413440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0440b2cf0 con 0x7fb0441599e0 2023-12-19T16:58:20.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb048ad1640 1 -- 172.21.15.26:0/2239413440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0440b32b0 con 0x7fb0441599e0 2023-12-19T16:58:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb048ad1640 1 -- 172.21.15.26:0/2239413440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb044074270 con 0x7fb0441599e0 2023-12-19T16:58:20.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.752+0000 7fb03affd640 1 -- 172.21.15.26:0/2239413440 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb02c013470 con 0x7fb0441599e0 2023-12-19T16:58:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.756+0000 7fb03affd640 1 --2- 172.21.15.26:0/2239413440 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb024065b80 0x7fb024068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.756+0000 7fb03affd640 1 -- 172.21.15.26:0/2239413440 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fb02c083700 con 0x7fb0441599e0 2023-12-19T16:58:20.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.756+0000 7fb04259c640 1 --2- 172.21.15.26:0/2239413440 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb024065b80 0x7fb024068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:20.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.756+0000 7fb04259c640 1 --2- 172.21.15.26:0/2239413440 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb024065b80 0x7fb024068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb034007fd0 tx=0x7fb034007d60 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:20.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.760+0000 7fb03affd640 1 -- 172.21.15.26:0/2239413440 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb02c050f70 con 0x7fb0441599e0 2023-12-19T16:58:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.916+0000 7fb048ad1640 1 -- 172.21.15.26:0/2239413440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fb0440c1af0 con 0x7fb0441599e0 2023-12-19T16:58:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.924+0000 7fb03affd640 1 -- 172.21.15.26:0/2239413440 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v33) v1 ==== 115+0+0 (secure 0 0 0) 0x7fb02c054590 con 0x7fb0441599e0 2023-12-19T16:58:20.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.924+0000 7fb048ad1640 1 -- 172.21.15.26:0/2239413440 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb024065b80 msgr2=0x7fb024068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.924+0000 7fb048ad1640 1 --2- 172.21.15.26:0/2239413440 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb024065b80 0x7fb024068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb034007fd0 tx=0x7fb034007d60 comp rx=0 tx=0).stop 2023-12-19T16:58:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.924+0000 7fb048ad1640 1 -- 172.21.15.26:0/2239413440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0441599e0 msgr2=0x7fb0440b2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.924+0000 7fb048ad1640 1 --2- 172.21.15.26:0/2239413440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0441599e0 0x7fb0440b2510 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e930 tx=0x7fb02c00edf0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.924+0000 7fb048ad1640 1 -- 172.21.15.26:0/2239413440 shutdown_connections 2023-12-19T16:58:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.924+0000 7fb048ad1640 1 --2- 172.21.15.26:0/2239413440 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb04415d530 0x7fb0440c24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.924+0000 7fb048ad1640 1 --2- 172.21.15.26:0/2239413440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0441599e0 0x7fb0440b2510 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.924+0000 7fb048ad1640 1 --2- 172.21.15.26:0/2239413440 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb024065b80 0x7fb024068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.924+0000 7fb048ad1640 1 --2- 172.21.15.26:0/2239413440 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0440cb860 0x7fb0440b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.924+0000 7fb048ad1640 1 -- 172.21.15.26:0/2239413440 >> 172.21.15.26:0/2239413440 conn(0x7fb044066160 msgr2=0x7fb0440ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.924+0000 7fb048ad1640 1 -- 172.21.15.26:0/2239413440 shutdown_connections 2023-12-19T16:58:20.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:20.924+0000 7fb048ad1640 1 -- 172.21.15.26:0/2239413440 wait complete. 2023-12-19T16:58:20.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-12-19T16:58:21.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.088+0000 7f9324fb3640 1 Processor -- start 2023-12-19T16:58:21.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.088+0000 7f9324fb3640 1 -- start start 2023-12-19T16:58:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.092+0000 7f9324fb3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f93200cee10 0x7f9320164980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.092+0000 7f9324fb3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932006bc30 con 0x7f9320154220 2023-12-19T16:58:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.092+0000 7f9324fb3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93200cc6e0 con 0x7f932006b3d0 2023-12-19T16:58:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.092+0000 7f9324fb3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932006b950 con 0x7f93200cee10 2023-12-19T16:58:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.092+0000 7f931ed9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f93200cee10 0x7f9320164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.092+0000 7f931e59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9320154220 0x7f932015aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56848/0 (socket says 172.21.15.26:56848) 2023-12-19T16:58:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.092+0000 7f931e59c640 1 -- 172.21.15.26:0/2041661277 learned_addr learned my addr 172.21.15.26:0/2041661277 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.092+0000 7f931ed9d640 1 -- 172.21.15.26:0/2041661277 >> v1:172.21.15.160:6789/0 conn(0x7f932006b3d0 legacy=0x7f93200ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.092+0000 7f931ed9d640 1 -- 172.21.15.26:0/2041661277 >> v1:172.21.15.26:6789/0 conn(0x7f9320154220 legacy=0x7f932015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.092+0000 7f931ed9d640 1 -- 172.21.15.26:0/2041661277 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93200cc930 con 0x7f93200cee10 2023-12-19T16:58:21.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.092+0000 7f931dd9b640 1 -- 172.21.15.26:0/2041661277 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2456027569 0 0) 0x7f932006bc30 con 0x7f9320154220 2023-12-19T16:58:21.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.096+0000 7f931ed9d640 1 --2- 172.21.15.26:0/2041661277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93200cee10 0x7f9320164980 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9318009d90 tx=0x7f931801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50555d776782840b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:21.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.096+0000 7f931dd9b640 1 -- 172.21.15.26:0/2041661277 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93180a59e0 con 0x7f93200cee10 2023-12-19T16:58:21.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.096+0000 7f931dd9b640 1 -- 172.21.15.26:0/2041661277 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93180a5b40 con 0x7f93200cee10 2023-12-19T16:58:21.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.096+0000 7f931dd9b640 1 -- 172.21.15.26:0/2041661277 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93180ad440 con 0x7f93200cee10 2023-12-19T16:58:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.096+0000 7f9324fb3640 1 -- 172.21.15.26:0/2041661277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93200cee10 msgr2=0x7f9320164980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.096+0000 7f9324fb3640 1 --2- 172.21.15.26:0/2041661277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93200cee10 0x7f9320164980 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9318009d90 tx=0x7f931801c920 comp rx=0 tx=0).stop 2023-12-19T16:58:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.096+0000 7f9324fb3640 1 -- 172.21.15.26:0/2041661277 shutdown_connections 2023-12-19T16:58:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.096+0000 7f9324fb3640 1 --2- 172.21.15.26:0/2041661277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93200cee10 0x7f9320164980 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.096+0000 7f9324fb3640 1 -- 172.21.15.26:0/2041661277 >> 172.21.15.26:0/2041661277 conn(0x7f93200658e0 msgr2=0x7f9320065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.096+0000 7f9324fb3640 1 -- 172.21.15.26:0/2041661277 shutdown_connections 2023-12-19T16:58:21.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.096+0000 7f9324fb3640 1 -- 172.21.15.26:0/2041661277 wait complete. 2023-12-19T16:58:21.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.096+0000 7f9324fb3640 1 Processor -- start 2023-12-19T16:58:21.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.096+0000 7f9324fb3640 1 -- start start 2023-12-19T16:58:21.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9324fb3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932006b3d0 0x7f9320164190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:21.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9324fb3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93200cee10 0x7f9320152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:21.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9324fb3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9320154220 0x7f9320158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:21.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9324fb3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9320158790 con 0x7f9320154220 2023-12-19T16:58:21.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9324fb3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9320157690 con 0x7f93200cee10 2023-12-19T16:58:21.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9324fb3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9320155840 con 0x7f932006b3d0 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f931ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9320154220 0x7f9320158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f931ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9320154220 0x7f9320158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44942/0 (socket says 172.21.15.26:44942) 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f931ed9d640 1 -- 172.21.15.26:0/1474522736 learned_addr learned my addr 172.21.15.26:0/1474522736 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f931e59c640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932006b3d0 0x7f9320164190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f931ed9d640 1 -- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932006b3d0 msgr2=0x7f9320164190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9317fff640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93200cee10 0x7f9320152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f931ed9d640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932006b3d0 0x7f9320164190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f931ed9d640 1 -- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93200cee10 msgr2=0x7f9320152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f931ed9d640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93200cee10 0x7f9320152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f931ed9d640 1 -- 172.21.15.26:0/1474522736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9318009a70 con 0x7f9320154220 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f931e59c640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932006b3d0 0x7f9320164190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9317fff640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93200cee10 0x7f9320152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f931ed9d640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9320154220 0x7f9320158df0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9318009d90 tx=0x7f93180196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:21.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9316ffd640 1 -- 172.21.15.26:0/1474522736 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93180ade40 con 0x7f9320154220 2023-12-19T16:58:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9316ffd640 1 -- 172.21.15.26:0/1474522736 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93180b9450 con 0x7f9320154220 2023-12-19T16:58:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9316ffd640 1 -- 172.21.15.26:0/1474522736 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93180b9800 con 0x7f9320154220 2023-12-19T16:58:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9324fb3640 1 -- 172.21.15.26:0/1474522736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93201583b0 con 0x7f9320154220 2023-12-19T16:58:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9324fb3640 1 -- 172.21.15.26:0/1474522736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9320155b20 con 0x7f9320154220 2023-12-19T16:58:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9324fb3640 1 -- 172.21.15.26:0/1474522736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93201588d0 con 0x7f9320154220 2023-12-19T16:58:21.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.100+0000 7f9316ffd640 1 -- 172.21.15.26:0/1474522736 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f93180b9960 con 0x7f9320154220 2023-12-19T16:58:21.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.104+0000 7f9316ffd640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9300065b80 0x7f9300068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:21.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.104+0000 7f9316ffd640 1 -- 172.21.15.26:0/1474522736 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f931811ee40 con 0x7f9320154220 2023-12-19T16:58:21.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.104+0000 7f931e59c640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9300065b80 0x7f9300068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:21.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.104+0000 7f931e59c640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9300065b80 0x7f9300068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9310000f70 tx=0x7f9310009040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:21.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.108+0000 7f9316ffd640 1 -- 172.21.15.26:0/1474522736 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f93201588d0 con 0x7f9320154220 2023-12-19T16:58:21.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.316+0000 7f9324fb3640 1 -- 172.21.15.26:0/1474522736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f93201588d0 con 0x7f9320154220 2023-12-19T16:58:21.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.852+0000 7f9316ffd640 1 -- 172.21.15.26:0/1474522736 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v595) v1 ==== 166+0+0 (secure 0 0 0) 0x7f93201588d0 con 0x7f9320154220 2023-12-19T16:58:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.856+0000 7f9324fb3640 1 -- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9300065b80 msgr2=0x7f9300068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.856+0000 7f9324fb3640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9300065b80 0x7f9300068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9310000f70 tx=0x7f9310009040 comp rx=0 tx=0).stop 2023-12-19T16:58:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.856+0000 7f9324fb3640 1 -- 172.21.15.26:0/1474522736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9320154220 msgr2=0x7f9320158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.856+0000 7f9324fb3640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9320154220 0x7f9320158df0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9318009d90 tx=0x7f93180196c0 comp rx=0 tx=0).stop 2023-12-19T16:58:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.856+0000 7f9324fb3640 1 -- 172.21.15.26:0/1474522736 shutdown_connections 2023-12-19T16:58:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.856+0000 7f9324fb3640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9320154220 0x7f9320158df0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.856+0000 7f9324fb3640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93200cee10 0x7f9320152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.856+0000 7f9324fb3640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9300065b80 0x7f9300068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.856+0000 7f9324fb3640 1 --2- 172.21.15.26:0/1474522736 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932006b3d0 0x7f9320164190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:21.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.856+0000 7f9324fb3640 1 -- 172.21.15.26:0/1474522736 >> 172.21.15.26:0/1474522736 conn(0x7f93200658e0 msgr2=0x7f9320065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.856+0000 7f9324fb3640 1 -- 172.21.15.26:0/1474522736 shutdown_connections 2023-12-19T16:58:21.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:21.856+0000 7f9324fb3640 1 -- 172.21.15.26:0/1474522736 wait complete. 2023-12-19T16:58:21.878 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-12-19T16:58:21.878 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-12-19T16:58:21.878 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-19T16:58:21.879 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-12-19T16:58:22.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa73088b640 1 Processor -- start 2023-12-19T16:58:22.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa73088b640 1 -- start start 2023-12-19T16:58:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa73088b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa72c0b8760 0x7fa72c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa73088b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c0cdff0 con 0x7fa72c15cca0 2023-12-19T16:58:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa73088b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c0b6cf0 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa73088b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72c0b7c30 con 0x7fa72c0b8760 2023-12-19T16:58:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa72ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa72c15cca0 0x7fa72c0b65e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56850/0 (socket says 172.21.15.26:56850) 2023-12-19T16:58:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa72ad9d640 1 -- 172.21.15.26:0/2287919564 learned_addr learned my addr 172.21.15.26:0/2287919564 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa72a59c640 1 --2- 172.21.15.26:0/2287919564 >> v2:172.21.15.26:3300/0 conn(0x7fa72c0b8760 0x7fa72c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa729d9b640 1 -- 172.21.15.26:0/2287919564 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477282526 0 0) 0x7fa72c0cdff0 con 0x7fa72c15cca0 2023-12-19T16:58:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa729d9b640 1 -- 172.21.15.26:0/2287919564 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa714003530 con 0x7fa72c15cca0 2023-12-19T16:58:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa72a59c640 1 -- 172.21.15.26:0/2287919564 >> v1:172.21.15.160:6789/0 conn(0x7fa72c0cdbb0 legacy=0x7fa72c0d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:22.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa72a59c640 1 -- 172.21.15.26:0/2287919564 >> v1:172.21.15.26:6789/0 conn(0x7fa72c15cca0 legacy=0x7fa72c0b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:22.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.024+0000 7fa72a59c640 1 -- 172.21.15.26:0/2287919564 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c0b6fb0 con 0x7fa72c0b8760 2023-12-19T16:58:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.028+0000 7fa72a59c640 1 --2- 172.21.15.26:0/2287919564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa72c0b8760 0x7fa72c0cd670 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa7200097c0 tx=0x7fa72001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc4df144f13ced7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.028+0000 7fa729d9b640 1 -- 172.21.15.26:0/2287919564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7200ab650 con 0x7fa72c0b8760 2023-12-19T16:58:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.028+0000 7fa729d9b640 1 -- 172.21.15.26:0/2287919564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7200ab7b0 con 0x7fa72c0b8760 2023-12-19T16:58:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.028+0000 7fa729d9b640 1 -- 172.21.15.26:0/2287919564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7200abb20 con 0x7fa72c0b8760 2023-12-19T16:58:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.028+0000 7fa73088b640 1 -- 172.21.15.26:0/2287919564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa72c0b8760 msgr2=0x7fa72c0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.028+0000 7fa73088b640 1 --2- 172.21.15.26:0/2287919564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa72c0b8760 0x7fa72c0cd670 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fa7200097c0 tx=0x7fa72001c450 comp rx=0 tx=0).stop 2023-12-19T16:58:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.028+0000 7fa73088b640 1 -- 172.21.15.26:0/2287919564 shutdown_connections 2023-12-19T16:58:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.028+0000 7fa73088b640 1 --2- 172.21.15.26:0/2287919564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa72c0b8760 0x7fa72c0cd670 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.028+0000 7fa73088b640 1 -- 172.21.15.26:0/2287919564 >> 172.21.15.26:0/2287919564 conn(0x7fa72c06a120 msgr2=0x7fa72c06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.028+0000 7fa73088b640 1 -- 172.21.15.26:0/2287919564 shutdown_connections 2023-12-19T16:58:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.028+0000 7fa73088b640 1 -- 172.21.15.26:0/2287919564 wait complete. 2023-12-19T16:58:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa73088b640 1 Processor -- start 2023-12-19T16:58:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa73088b640 1 -- start start 2023-12-19T16:58:22.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa73088b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa72c0b8760 0x7fa72c1f9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa73088b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa72c0cdbb0 0x7fa72c15dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa73088b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa72c15cca0 0x7fa72c14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa73088b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c154c10 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa73088b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c155380 con 0x7fa72c0b8760 2023-12-19T16:58:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa73088b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c156c90 con 0x7fa72c15cca0 2023-12-19T16:58:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa72a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa72c0cdbb0 0x7fa72c15dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa72a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa72c0cdbb0 0x7fa72c15dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44960/0 (socket says 172.21.15.26:44960) 2023-12-19T16:58:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa72a59c640 1 -- 172.21.15.26:0/3868325218 learned_addr learned my addr 172.21.15.26:0/3868325218 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa72ad9d640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa72c0b8760 0x7fa72c1f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa72b59e640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa72c15cca0 0x7fa72c14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa72a59c640 1 -- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa72c15cca0 msgr2=0x7fa72c14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa72a59c640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa72c15cca0 0x7fa72c14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa72a59c640 1 -- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa72c0b8760 msgr2=0x7fa72c1f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa72a59c640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa72c0b8760 0x7fa72c1f9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa72a59c640 1 -- 172.21.15.26:0/3868325218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa720009420 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa72b59e640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa72c15cca0 0x7fa72c14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa72a59c640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa72c0cdbb0 0x7fa72c15dc10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fa7200097c0 tx=0x7fa72001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa7137fe640 1 -- 172.21.15.26:0/3868325218 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720004220 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa7137fe640 1 -- 172.21.15.26:0/3868325218 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7200b9070 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa73088b640 1 -- 172.21.15.26:0/3868325218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c156f70 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa7137fe640 1 -- 172.21.15.26:0/3868325218 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720003620 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa73088b640 1 -- 172.21.15.26:0/3868325218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c14b510 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.032+0000 7fa73088b640 1 -- 172.21.15.26:0/3868325218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72c067400 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.036+0000 7fa7137fe640 1 -- 172.21.15.26:0/3868325218 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa720003780 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.036+0000 7fa7137fe640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa718065b80 0x7fa718068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.036+0000 7fa7137fe640 1 -- 172.21.15.26:0/3868325218 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fa7200bb070 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.036+0000 7fa72ad9d640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa718065b80 0x7fa718068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.036+0000 7fa72ad9d640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa718065b80 0x7fa718068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0011f0 tx=0x7fa71c007d40 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.040+0000 7fa7137fe640 1 -- 172.21.15.26:0/3868325218 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa720121030 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.248+0000 7fa73088b640 1 -- 172.21.15.26:0/3868325218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7fa72c0d0ea0 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.860+0000 7fa7137fe640 1 -- 172.21.15.26:0/3868325218 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v596) v1 ==== 186+0+0 (secure 0 0 0) 0x7fa7200c4020 con 0x7fa72c0cdbb0 2023-12-19T16:58:22.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.860+0000 7fa73088b640 1 -- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa718065b80 msgr2=0x7fa718068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.860+0000 7fa73088b640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa718065b80 0x7fa718068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0011f0 tx=0x7fa71c007d40 comp rx=0 tx=0).stop 2023-12-19T16:58:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.860+0000 7fa73088b640 1 -- 172.21.15.26:0/3868325218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa72c0cdbb0 msgr2=0x7fa72c15dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.860+0000 7fa73088b640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa72c0cdbb0 0x7fa72c15dc10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fa7200097c0 tx=0x7fa72001ac90 comp rx=0 tx=0).stop 2023-12-19T16:58:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.860+0000 7fa73088b640 1 -- 172.21.15.26:0/3868325218 shutdown_connections 2023-12-19T16:58:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.860+0000 7fa73088b640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa718065b80 0x7fa718068040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.860+0000 7fa73088b640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa72c15cca0 0x7fa72c14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.860+0000 7fa73088b640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa72c0cdbb0 0x7fa72c15dc10 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.860+0000 7fa73088b640 1 --2- 172.21.15.26:0/3868325218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa72c0b8760 0x7fa72c1f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.860+0000 7fa73088b640 1 -- 172.21.15.26:0/3868325218 >> 172.21.15.26:0/3868325218 conn(0x7fa72c06a120 msgr2=0x7fa72c0c6300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.860+0000 7fa73088b640 1 -- 172.21.15.26:0/3868325218 shutdown_connections 2023-12-19T16:58:22.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:22.860+0000 7fa73088b640 1 -- 172.21.15.26:0/3868325218 wait complete. 2023-12-19T16:58:22.886 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T16:58:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd05102d640 1 Processor -- start 2023-12-19T16:58:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd05102d640 1 -- start start 2023-12-19T16:58:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd05102d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd04c15a9c0 0x7fd04c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd05102d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd04c06ac80 0x7fd04c0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd05102d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c070d90 con 0x7fd04c15ada0 2023-12-19T16:58:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd05102d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c0cc270 con 0x7fd04c15a9c0 2023-12-19T16:58:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd05102d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c071070 con 0x7fd04c06ac80 2023-12-19T16:58:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd04ad9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd04c15a9c0 0x7fd04c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd04b59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd04c06ac80 0x7fd04c0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd04a59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd04c15ada0 0x7fd04c0b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:38168/0 (socket says 172.21.15.26:38168) 2023-12-19T16:58:23.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd04a59c640 1 -- 172.21.15.26:0/808623653 learned_addr learned my addr 172.21.15.26:0/808623653 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:23.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd04ad9d640 1 -- 172.21.15.26:0/808623653 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd04c06ac80 msgr2=0x7fd04c0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:23.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd04ad9d640 1 --2- 172.21.15.26:0/808623653 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd04c06ac80 0x7fd04c0b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd04ad9d640 1 -- 172.21.15.26:0/808623653 >> v1:172.21.15.160:6789/0 conn(0x7fd04c15ada0 legacy=0x7fd04c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:23.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd04ad9d640 1 -- 172.21.15.26:0/808623653 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c071350 con 0x7fd04c15a9c0 2023-12-19T16:58:23.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.036+0000 7fd049d9b640 1 -- 172.21.15.26:0/808623653 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1271513030 0 0) 0x7fd04c070d90 con 0x7fd04c15ada0 2023-12-19T16:58:23.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.040+0000 7fd04ad9d640 1 --2- 172.21.15.26:0/808623653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd04c15a9c0 0x7fd04c0706d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0097c0 tx=0x7fd03c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23bda335df5635ec server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.040+0000 7fd049d9b640 1 -- 172.21.15.26:0/808623653 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c0ab600 con 0x7fd04c15a9c0 2023-12-19T16:58:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.040+0000 7fd049d9b640 1 -- 172.21.15.26:0/808623653 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c0a7050 con 0x7fd04c15a9c0 2023-12-19T16:58:23.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.040+0000 7fd049d9b640 1 -- 172.21.15.26:0/808623653 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c0ab980 con 0x7fd04c15a9c0 2023-12-19T16:58:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.040+0000 7fd05102d640 1 -- 172.21.15.26:0/808623653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd04c15a9c0 msgr2=0x7fd04c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.040+0000 7fd05102d640 1 --2- 172.21.15.26:0/808623653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd04c15a9c0 0x7fd04c0706d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0097c0 tx=0x7fd03c01c3c0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.040+0000 7fd05102d640 1 -- 172.21.15.26:0/808623653 shutdown_connections 2023-12-19T16:58:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.040+0000 7fd05102d640 1 --2- 172.21.15.26:0/808623653 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd04c06ac80 0x7fd04c0b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.040+0000 7fd05102d640 1 --2- 172.21.15.26:0/808623653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd04c15a9c0 0x7fd04c0706d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.040+0000 7fd05102d640 1 -- 172.21.15.26:0/808623653 >> 172.21.15.26:0/808623653 conn(0x7fd04c0ce730 msgr2=0x7fd04c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.040+0000 7fd05102d640 1 -- 172.21.15.26:0/808623653 shutdown_connections 2023-12-19T16:58:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd05102d640 1 -- 172.21.15.26:0/808623653 wait complete. 2023-12-19T16:58:23.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd05102d640 1 Processor -- start 2023-12-19T16:58:23.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd05102d640 1 -- start start 2023-12-19T16:58:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd05102d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd04c06ac80 0x7fd04c159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd05102d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd04c15a9c0 0x7fd04c149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:23.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd05102d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd04c15ada0 0x7fd04c0c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd05102d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c152ef0 con 0x7fd04c06ac80 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd05102d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c14a340 con 0x7fd04c15a9c0 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd05102d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0c8e90 con 0x7fd04c15ada0 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd04ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd04c06ac80 0x7fd04c159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd04ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd04c06ac80 0x7fd04c159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44964/0 (socket says 172.21.15.26:44964) 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd04ad9d640 1 -- 172.21.15.26:0/2847472731 learned_addr learned my addr 172.21.15.26:0/2847472731 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd04b59e640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd04c15ada0 0x7fd04c0c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd04a59c640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd04c15a9c0 0x7fd04c149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd04ad9d640 1 -- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd04c15ada0 msgr2=0x7fd04c0c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd04ad9d640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd04c15ada0 0x7fd04c0c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd04ad9d640 1 -- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd04c15a9c0 msgr2=0x7fd04c149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd04ad9d640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd04c15a9c0 0x7fd04c149e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.044+0000 7fd04ad9d640 1 -- 172.21.15.26:0/2847472731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c009420 con 0x7fd04c06ac80 2023-12-19T16:58:23.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.048+0000 7fd04ad9d640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd04c06ac80 0x7fd04c159b00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd03c01aa70 tx=0x7fd03c019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:23.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.048+0000 7fd0337fe640 1 -- 172.21.15.26:0/2847472731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c004230 con 0x7fd04c06ac80 2023-12-19T16:58:23.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.048+0000 7fd0337fe640 1 -- 172.21.15.26:0/2847472731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c004390 con 0x7fd04c06ac80 2023-12-19T16:58:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.048+0000 7fd0337fe640 1 -- 172.21.15.26:0/2847472731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c003820 con 0x7fd04c06ac80 2023-12-19T16:58:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.048+0000 7fd05102d640 1 -- 172.21.15.26:0/2847472731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c152c10 con 0x7fd04c06ac80 2023-12-19T16:58:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.048+0000 7fd05102d640 1 -- 172.21.15.26:0/2847472731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c0c9410 con 0x7fd04c06ac80 2023-12-19T16:58:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.048+0000 7fd05102d640 1 -- 172.21.15.26:0/2847472731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c0c8cf0 con 0x7fd04c06ac80 2023-12-19T16:58:23.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.048+0000 7fd0337fe640 1 -- 172.21.15.26:0/2847472731 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd03c0b2040 con 0x7fd04c06ac80 2023-12-19T16:58:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.048+0000 7fd0337fe640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd038065b80 0x7fd038068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.048+0000 7fd0337fe640 1 -- 172.21.15.26:0/2847472731 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fd03c11d680 con 0x7fd04c06ac80 2023-12-19T16:58:23.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.048+0000 7fd04a59c640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd038065b80 0x7fd038068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:23.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.052+0000 7fd04a59c640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd038065b80 0x7fd038068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd040007fd0 tx=0x7fd040007ba0 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:23.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.052+0000 7fd0337fe640 1 -- 172.21.15.26:0/2847472731 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd04c0c8cf0 con 0x7fd04c06ac80 2023-12-19T16:58:23.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.208+0000 7fd05102d640 1 -- 172.21.15.26:0/2847472731 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd04c0c8cf0 con 0x7fd04c06ac80 2023-12-19T16:58:23.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.216+0000 7fd0337fe640 1 -- 172.21.15.26:0/2847472731 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd04c0c8cf0 con 0x7fd04c06ac80 2023-12-19T16:58:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.216+0000 7fd05102d640 1 -- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd038065b80 msgr2=0x7fd038068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.216+0000 7fd05102d640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd038065b80 0x7fd038068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd040007fd0 tx=0x7fd040007ba0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.216+0000 7fd05102d640 1 -- 172.21.15.26:0/2847472731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd04c06ac80 msgr2=0x7fd04c159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.216+0000 7fd05102d640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd04c06ac80 0x7fd04c159b00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd03c01aa70 tx=0x7fd03c019160 comp rx=0 tx=0).stop 2023-12-19T16:58:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.216+0000 7fd05102d640 1 -- 172.21.15.26:0/2847472731 shutdown_connections 2023-12-19T16:58:23.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.216+0000 7fd05102d640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd038065b80 0x7fd038068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.216+0000 7fd05102d640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd04c15ada0 0x7fd04c0c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.216+0000 7fd05102d640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd04c15a9c0 0x7fd04c149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.216+0000 7fd05102d640 1 --2- 172.21.15.26:0/2847472731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd04c06ac80 0x7fd04c159b00 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.216+0000 7fd05102d640 1 -- 172.21.15.26:0/2847472731 >> 172.21.15.26:0/2847472731 conn(0x7fd04c0ce730 msgr2=0x7fd04c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:23.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.216+0000 7fd05102d640 1 -- 172.21.15.26:0/2847472731 shutdown_connections 2023-12-19T16:58:23.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.216+0000 7fd05102d640 1 -- 172.21.15.26:0/2847472731 wait complete. 2023-12-19T16:58:23.238 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T16:58:23.239 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:58:23.330 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:58:23.396 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T16:58:23.396 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:58:23.396 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:58:23.407 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:58:23.407 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:58:23.407 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:58:23.417 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:58:23.418 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:58:23.418 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:58:23.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:58:23.461 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:58:23.461 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:58:23.470 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:58:23.470 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T16:58:23.470 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:58:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.644+0000 7fb1b968c640 1 Processor -- start 2023-12-19T16:58:23.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.644+0000 7fb1b968c640 1 -- start start 2023-12-19T16:58:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.644+0000 7fb1b968c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1b415c7f0 0x7fb1b406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.644+0000 7fb1b968c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1b40adb80 0x7fb1b40cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.644+0000 7fb1b968c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b406ced0 con 0x7fb1b40ccc20 2023-12-19T16:58:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.644+0000 7fb1b968c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b406d770 con 0x7fb1b40adb80 2023-12-19T16:58:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.644+0000 7fb1b968c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b406d1b0 con 0x7fb1b415c7f0 2023-12-19T16:58:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.644+0000 7fb1abfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb1b40adb80 0x7fb1b40cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.644+0000 7fb1b37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb1b40ccc20 0x7fb1b40b30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56862/0 (socket says 172.21.15.26:56862) 2023-12-19T16:58:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.644+0000 7fb1b37fe640 1 -- 172.21.15.26:0/1281957957 learned_addr learned my addr 172.21.15.26:0/1281957957 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.644+0000 7fb1b2ffd640 1 --2- 172.21.15.26:0/1281957957 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1b415c7f0 0x7fb1b406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.644+0000 7fb1b27fc640 1 -- 172.21.15.26:0/1281957957 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525888518 0 0) 0x7fb1b406ced0 con 0x7fb1b40ccc20 2023-12-19T16:58:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1abfff640 1 -- 172.21.15.26:0/1281957957 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1b415c7f0 msgr2=0x7fb1b406c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:23.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1abfff640 1 --2- 172.21.15.26:0/1281957957 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1b415c7f0 0x7fb1b406c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1abfff640 1 -- 172.21.15.26:0/1281957957 >> v1:172.21.15.26:6789/0 conn(0x7fb1b40ccc20 legacy=0x7fb1b40b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:23.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1abfff640 1 -- 172.21.15.26:0/1281957957 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b406d490 con 0x7fb1b40adb80 2023-12-19T16:58:23.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1b2ffd640 1 --2- 172.21.15.26:0/1281957957 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1b415c7f0 0x7fb1b406c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1abfff640 1 --2- 172.21.15.26:0/1281957957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1b40adb80 0x7fb1b40cc6e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0097c0 tx=0x7fb19c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e0498d0478bfcc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1b27fc640 1 -- 172.21.15.26:0/1281957957 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c0ac640 con 0x7fb1b40adb80 2023-12-19T16:58:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1b27fc640 1 -- 172.21.15.26:0/1281957957 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b40ae1f0 con 0x7fb1b40adb80 2023-12-19T16:58:23.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1b27fc640 1 -- 172.21.15.26:0/1281957957 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19c0ac9c0 con 0x7fb1b40adb80 2023-12-19T16:58:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1b968c640 1 -- 172.21.15.26:0/1281957957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1b40adb80 msgr2=0x7fb1b40cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1b968c640 1 --2- 172.21.15.26:0/1281957957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1b40adb80 0x7fb1b40cc6e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb19c0097c0 tx=0x7fb19c01c480 comp rx=0 tx=0).stop 2023-12-19T16:58:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1b968c640 1 -- 172.21.15.26:0/1281957957 shutdown_connections 2023-12-19T16:58:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1b968c640 1 --2- 172.21.15.26:0/1281957957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1b40adb80 0x7fb1b40cc6e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1b968c640 1 --2- 172.21.15.26:0/1281957957 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1b415c7f0 0x7fb1b406c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1b968c640 1 -- 172.21.15.26:0/1281957957 >> 172.21.15.26:0/1281957957 conn(0x7fb1b40658e0 msgr2=0x7fb1b4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1b968c640 1 -- 172.21.15.26:0/1281957957 shutdown_connections 2023-12-19T16:58:23.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.648+0000 7fb1b968c640 1 -- 172.21.15.26:0/1281957957 wait complete. 2023-12-19T16:58:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b968c640 1 Processor -- start 2023-12-19T16:58:23.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b968c640 1 -- start start 2023-12-19T16:58:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b968c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1b40adb80 0x7fb1b41f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b968c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1b40ccc20 0x7fb1b4149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:23.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b968c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1b415c7f0 0x7fb1b4159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:23.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b968c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b41f6360 con 0x7fb1b415c7f0 2023-12-19T16:58:23.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b968c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b414df40 con 0x7fb1b40adb80 2023-12-19T16:58:23.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b968c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b4153e90 con 0x7fb1b40ccc20 2023-12-19T16:58:23.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1b415c7f0 0x7fb1b4159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:23.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1b415c7f0 0x7fb1b4159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44976/0 (socket says 172.21.15.26:44976) 2023-12-19T16:58:23.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b37fe640 1 -- 172.21.15.26:0/1252881328 learned_addr learned my addr 172.21.15.26:0/1252881328 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:23.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b37fe640 1 -- 172.21.15.26:0/1252881328 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1b40ccc20 msgr2=0x7fb1b4149980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:23.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b37fe640 1 --2- 172.21.15.26:0/1252881328 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1b40ccc20 0x7fb1b4149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b37fe640 1 -- 172.21.15.26:0/1252881328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1b40adb80 msgr2=0x7fb1b41f5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:23.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b37fe640 1 --2- 172.21.15.26:0/1252881328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1b40adb80 0x7fb1b41f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b37fe640 1 -- 172.21.15.26:0/1252881328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c009420 con 0x7fb1b415c7f0 2023-12-19T16:58:23.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b37fe640 1 --2- 172.21.15.26:0/1252881328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1b415c7f0 0x7fb1b4159250 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00c780 tx=0x7fb1ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:23.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1ab7fe640 1 -- 172.21.15.26:0/1252881328 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00ce90 con 0x7fb1b415c7f0 2023-12-19T16:58:23.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1ab7fe640 1 -- 172.21.15.26:0/1252881328 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac00a640 con 0x7fb1b415c7f0 2023-12-19T16:58:23.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1ab7fe640 1 -- 172.21.15.26:0/1252881328 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00aa70 con 0x7fb1b415c7f0 2023-12-19T16:58:23.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b968c640 1 -- 172.21.15.26:0/1252881328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b4154170 con 0x7fb1b415c7f0 2023-12-19T16:58:23.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.652+0000 7fb1b968c640 1 -- 172.21.15.26:0/1252881328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b4149ec0 con 0x7fb1b415c7f0 2023-12-19T16:58:23.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.656+0000 7fb1b968c640 1 -- 172.21.15.26:0/1252881328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b4154690 con 0x7fb1b415c7f0 2023-12-19T16:58:23.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.656+0000 7fb1ab7fe640 1 -- 172.21.15.26:0/1252881328 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb1ac00abd0 con 0x7fb1b415c7f0 2023-12-19T16:58:23.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.656+0000 7fb1ab7fe640 1 --2- 172.21.15.26:0/1252881328 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb194065bb0 0x7fb194068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:23.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.656+0000 7fb1ab7fe640 1 -- 172.21.15.26:0/1252881328 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fb1ac082970 con 0x7fb1b415c7f0 2023-12-19T16:58:23.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.656+0000 7fb1b2ffd640 1 --2- 172.21.15.26:0/1252881328 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb194065bb0 0x7fb194068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:23.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.656+0000 7fb1b2ffd640 1 --2- 172.21.15.26:0/1252881328 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb194065bb0 0x7fb194068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400a890 tx=0x7fb1a4008040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:23.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.660+0000 7fb1ab7fe640 1 -- 172.21.15.26:0/1252881328 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb1b4154690 con 0x7fb1b415c7f0 2023-12-19T16:58:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1b968c640 1 -- 172.21.15.26:0/1252881328 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1b4154690 con 0x7fb1b415c7f0 2023-12-19T16:58:23.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1ab7fe640 1 -- 172.21.15.26:0/1252881328 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fb1b4154690 con 0x7fb1b415c7f0 2023-12-19T16:58:23.852 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:23.853 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6131,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/426534014","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":426534014},{"type":"v1","addr":"172.21.15.160:6837","nonce":426534014}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6730,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/1088174367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":1088174367},{"type":"v1","addr":"172.21.15.160:6836","nonce":1088174367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":6981,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/343942943","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":343942943},{"type":"v1","addr":"172.21.15.26:6834","nonce":343942943}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":7398,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2198559353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2198559353},{"type":"v1","addr":"172.21.15.26:6835","nonce":2198559353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[]} 2023-12-19T16:58:23.853 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 43 2023-12-19T16:58:23.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1b968c640 1 -- 172.21.15.26:0/1252881328 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb194065bb0 msgr2=0x7fb194068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:23.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1b968c640 1 --2- 172.21.15.26:0/1252881328 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb194065bb0 0x7fb194068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400a890 tx=0x7fb1a4008040 comp rx=0 tx=0).stop 2023-12-19T16:58:23.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1b968c640 1 -- 172.21.15.26:0/1252881328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1b415c7f0 msgr2=0x7fb1b4159250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:23.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1b968c640 1 --2- 172.21.15.26:0/1252881328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1b415c7f0 0x7fb1b4159250 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00c780 tx=0x7fb1ac00cc40 comp rx=0 tx=0).stop 2023-12-19T16:58:23.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1b968c640 1 -- 172.21.15.26:0/1252881328 shutdown_connections 2023-12-19T16:58:23.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1b968c640 1 --2- 172.21.15.26:0/1252881328 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1b415c7f0 0x7fb1b4159250 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1b968c640 1 --2- 172.21.15.26:0/1252881328 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb194065bb0 0x7fb194068070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1b968c640 1 --2- 172.21.15.26:0/1252881328 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1b40ccc20 0x7fb1b4149980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1b968c640 1 --2- 172.21.15.26:0/1252881328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1b40adb80 0x7fb1b41f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:23.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1b968c640 1 -- 172.21.15.26:0/1252881328 >> 172.21.15.26:0/1252881328 conn(0x7fb1b40658e0 msgr2=0x7fb1b406f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:23.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1b968c640 1 -- 172.21.15.26:0/1252881328 shutdown_connections 2023-12-19T16:58:23.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:23.852+0000 7fb1b968c640 1 -- 172.21.15.26:0/1252881328 wait complete. 2023-12-19T16:58:23.875 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T16:58:23.875 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T16:58:23.875 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T16:58:23.875 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:58:23.875 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T16:58:23.875 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T16:58:23.876 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T16:58:23.876 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:58:23.876 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T16:58:23.876 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T16:58:23.876 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T16:58:23.876 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:58:23.876 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T16:58:23.877 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T16:58:23.877 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T16:58:23.877 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:58:23.888 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T16:58:23.888 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T16:58:23.889 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T16:58:23.889 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T16:58:23.891 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T16:58:23.891 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T16:58:23.892 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T16:58:23.892 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T16:58:23.935 INFO:tasks.ceph.mds.a:Started 2023-12-19T16:58:23.935 DEBUG:teuthology.parallel:result is None 2023-12-19T16:58:23.935 INFO:tasks.ceph.mds.b:Started 2023-12-19T16:58:23.936 DEBUG:teuthology.parallel:result is None 2023-12-19T16:58:23.937 INFO:tasks.ceph.mds.c:Started 2023-12-19T16:58:23.937 DEBUG:teuthology.parallel:result is None 2023-12-19T16:58:23.938 INFO:tasks.ceph.mds.d:Started 2023-12-19T16:58:23.938 DEBUG:teuthology.parallel:result is None 2023-12-19T16:58:23.938 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T16:58:24.027 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T16:58:24.027 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T16:58:24.028 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T16:58:24.028 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T16:58:24.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.088+0000 7f9443266640 1 Processor -- start 2023-12-19T16:58:24.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.088+0000 7f9443266640 1 -- start start 2023-12-19T16:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.088+0000 7f9443266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f943c155030 0x7f943c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.088+0000 7f9443266640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f943c15f370 0x7f943c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.088+0000 7f9443266640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f943c06c970 con 0x7f943c0ce740 2023-12-19T16:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.088+0000 7f9443266640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943c15b1f0 con 0x7f943c15f370 2023-12-19T16:58:24.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.088+0000 7f9443266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c15b4d0 con 0x7f943c155030 2023-12-19T16:58:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.088+0000 7f9441002640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f943c155030 0x7f943c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.088+0000 7f9441803640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f943c0ce740 0x7f943c0ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:38188/0 (socket says 172.21.15.26:38188) 2023-12-19T16:58:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.088+0000 7f9440801640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f943c15f370 0x7f943c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.088+0000 7f9441803640 1 -- 172.21.15.26:0/342249971 learned_addr learned my addr 172.21.15.26:0/342249971 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.092+0000 7f9433fff640 1 -- 172.21.15.26:0/342249971 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543901292 0 0) 0x7f943c06c970 con 0x7f943c0ce740 2023-12-19T16:58:24.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.092+0000 7f9433fff640 1 -- 172.21.15.26:0/342249971 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9428003530 con 0x7f943c0ce740 2023-12-19T16:58:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.092+0000 7f9441002640 1 -- 172.21.15.26:0/342249971 >> v2:172.21.15.160:3300/0 conn(0x7f943c15f370 msgr2=0x7f943c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.092+0000 7f9441002640 1 --2- 172.21.15.26:0/342249971 >> v2:172.21.15.160:3300/0 conn(0x7f943c15f370 0x7f943c06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.092+0000 7f9441002640 1 -- 172.21.15.26:0/342249971 >> v1:172.21.15.160:6789/0 conn(0x7f943c0ce740 legacy=0x7f943c0ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.092+0000 7f9441002640 1 -- 172.21.15.26:0/342249971 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c15b7b0 con 0x7f943c155030 2023-12-19T16:58:24.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.092+0000 7f9440801640 1 --2- 172.21.15.26:0/342249971 >> v2:172.21.15.160:3300/0 conn(0x7f943c15f370 0x7f943c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:24.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.092+0000 7f9441002640 1 --2- 172.21.15.26:0/342249971 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f943c155030 0x7f943c15bcd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f94380097c0 tx=0x7f943801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2961cb9d019ee15 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:24.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.092+0000 7f9433fff640 1 -- 172.21.15.26:0/342249971 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94380a6040 con 0x7f943c155030 2023-12-19T16:58:24.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.092+0000 7f9433fff640 1 -- 172.21.15.26:0/342249971 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94380aa600 con 0x7f943c155030 2023-12-19T16:58:24.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.092+0000 7f9433fff640 1 -- 172.21.15.26:0/342249971 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94380aa990 con 0x7f943c155030 2023-12-19T16:58:24.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 -- 172.21.15.26:0/342249971 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f943c155030 msgr2=0x7f943c15bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:24.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 --2- 172.21.15.26:0/342249971 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f943c155030 0x7f943c15bcd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f94380097c0 tx=0x7f943801c480 comp rx=0 tx=0).stop 2023-12-19T16:58:24.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 -- 172.21.15.26:0/342249971 shutdown_connections 2023-12-19T16:58:24.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 --2- 172.21.15.26:0/342249971 >> v2:172.21.15.160:3300/0 conn(0x7f943c15f370 0x7f943c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:24.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 --2- 172.21.15.26:0/342249971 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f943c155030 0x7f943c15bcd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:24.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 -- 172.21.15.26:0/342249971 >> 172.21.15.26:0/342249971 conn(0x7f943c0658e0 msgr2=0x7f943c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:24.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 -- 172.21.15.26:0/342249971 shutdown_connections 2023-12-19T16:58:24.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 -- 172.21.15.26:0/342249971 wait complete. 2023-12-19T16:58:24.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 Processor -- start 2023-12-19T16:58:24.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 -- start start 2023-12-19T16:58:24.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f943c0ce740 0x7f943c0ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:24.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c155030 0x7f943c154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f943c15f370 0x7f943c14b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0b0910 con 0x7f943c155030 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0ae4e0 con 0x7f943c0ce740 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9443266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943c0b0bb0 con 0x7f943c15f370 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9440801640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c155030 0x7f943c154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9440801640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c155030 0x7f943c154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45006/0 (socket says 172.21.15.26:45006) 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9440801640 1 -- 172.21.15.26:0/113049042 learned_addr learned my addr 172.21.15.26:0/113049042 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9441002640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f943c0ce740 0x7f943c0ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.096+0000 7f9441803640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f943c15f370 0x7f943c14b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9441002640 1 -- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f943c15f370 msgr2=0x7f943c14b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9441002640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f943c15f370 0x7f943c14b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9441002640 1 -- 172.21.15.26:0/113049042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c155030 msgr2=0x7f943c154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9441002640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c155030 0x7f943c154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9441002640 1 -- 172.21.15.26:0/113049042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c009420 con 0x7f943c0ce740 2023-12-19T16:58:24.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9440801640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c155030 0x7f943c154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:24.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9441002640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f943c0ce740 0x7f943c0ad010 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f94380ab000 tx=0x7f943801acc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9431ffb640 1 -- 172.21.15.26:0/113049042 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94380b3600 con 0x7f943c0ce740 2023-12-19T16:58:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9431ffb640 1 -- 172.21.15.26:0/113049042 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94380b3760 con 0x7f943c0ce740 2023-12-19T16:58:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9443266640 1 -- 172.21.15.26:0/113049042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9438009420 con 0x7f943c0ce740 2023-12-19T16:58:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9431ffb640 1 -- 172.21.15.26:0/113049042 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94380b3b10 con 0x7f943c0ce740 2023-12-19T16:58:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9443266640 1 -- 172.21.15.26:0/113049042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c06c740 con 0x7f943c0ce740 2023-12-19T16:58:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9443266640 1 -- 172.21.15.26:0/113049042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c074270 con 0x7f943c0ce740 2023-12-19T16:58:24.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9431ffb640 1 -- 172.21.15.26:0/113049042 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f94380b3c70 con 0x7f943c0ce740 2023-12-19T16:58:24.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9431ffb640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9420065b80 0x7f9420068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:24.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9431ffb640 1 -- 172.21.15.26:0/113049042 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f943811dda0 con 0x7f943c0ce740 2023-12-19T16:58:24.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.100+0000 7f9440801640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9420065b80 0x7f9420068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:24.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.104+0000 7f9440801640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9420065b80 0x7f9420068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f943c0ace50 tx=0x7f942c009190 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:24.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.104+0000 7f9431ffb640 1 -- 172.21.15.26:0/113049042 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f94380eb3a0 con 0x7f943c0ce740 2023-12-19T16:58:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:24.272+0000 7f9443266640 1 -- 172.21.15.26:0/113049042 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f943c0ccc30 con 0x7f943c0ce740 2023-12-19T16:58:25.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.080+0000 7f9431ffb640 1 -- 172.21.15.26:0/113049042 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v103) v1 ==== 97+0+0 (secure 0 0 0) 0x7f94380ee9c0 con 0x7f943c0ce740 2023-12-19T16:58:25.080 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2023-12-19T16:58:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.080+0000 7f9443266640 1 -- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9420065b80 msgr2=0x7f9420068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.080+0000 7f9443266640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9420065b80 0x7f9420068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f943c0ace50 tx=0x7f942c009190 comp rx=0 tx=0).stop 2023-12-19T16:58:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.080+0000 7f9443266640 1 -- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f943c0ce740 msgr2=0x7f943c0ad010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.080+0000 7f9443266640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f943c0ce740 0x7f943c0ad010 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f94380ab000 tx=0x7f943801acc0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.080+0000 7f9443266640 1 -- 172.21.15.26:0/113049042 shutdown_connections 2023-12-19T16:58:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.080+0000 7f9443266640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9420065b80 0x7f9420068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.080+0000 7f9443266640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f943c15f370 0x7f943c14b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.080+0000 7f9443266640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f943c155030 0x7f943c154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.080+0000 7f9443266640 1 --2- 172.21.15.26:0/113049042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f943c0ce740 0x7f943c0ad010 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.080+0000 7f9443266640 1 -- 172.21.15.26:0/113049042 >> 172.21.15.26:0/113049042 conn(0x7f943c0658e0 msgr2=0x7f943c0b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:25.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.080+0000 7f9443266640 1 -- 172.21.15.26:0/113049042 shutdown_connections 2023-12-19T16:58:25.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.080+0000 7f9443266640 1 -- 172.21.15.26:0/113049042 wait complete. 2023-12-19T16:58:25.106 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T16:58:25.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f6573012640 1 Processor -- start 2023-12-19T16:58:25.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f6573012640 1 -- start start 2023-12-19T16:58:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f6573012640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f656c155030 0x7f656c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f6573012640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f656c15f370 0x7f656c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f6573012640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c06c970 con 0x7f656c0ce740 2023-12-19T16:58:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f6573012640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656c15b1f0 con 0x7f656c15f370 2023-12-19T16:58:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f6573012640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c15b4d0 con 0x7f656c155030 2023-12-19T16:58:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f656bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f656c15f370 0x7f656c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f656bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f656c15f370 0x7f656c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45020/0 (socket says 172.21.15.26:45020) 2023-12-19T16:58:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f656bfff640 1 -- 172.21.15.26:0/1384368165 learned_addr learned my addr 172.21.15.26:0/1384368165 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f65715af640 1 --1- 172.21.15.26:0/1384368165 >> v1:172.21.15.26:6789/0 conn(0x7f656c0ce740 0x7f656c0ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56888/0 (socket says 172.21.15.26:56888) 2023-12-19T16:58:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f6570dae640 1 --2- 172.21.15.26:0/1384368165 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f656c155030 0x7f656c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f656bfff640 1 -- 172.21.15.26:0/1384368165 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f656c155030 msgr2=0x7f656c15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f656bfff640 1 --2- 172.21.15.26:0/1384368165 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f656c155030 0x7f656c15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f656bfff640 1 -- 172.21.15.26:0/1384368165 >> v1:172.21.15.26:6789/0 conn(0x7f656c0ce740 legacy=0x7f656c0ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.256+0000 7f656bfff640 1 -- 172.21.15.26:0/1384368165 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c15b7b0 con 0x7f656c15f370 2023-12-19T16:58:25.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.260+0000 7f656bfff640 1 --2- 172.21.15.26:0/1384368165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c15f370 0x7f656c06c080 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f6558009870 tx=0x7f655801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f294d2ddc6f05923 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:25.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.260+0000 7f656b7fe640 1 -- 172.21.15.26:0/1384368165 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65580a9620 con 0x7f656c15f370 2023-12-19T16:58:25.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.260+0000 7f656b7fe640 1 -- 172.21.15.26:0/1384368165 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65580a9780 con 0x7f656c15f370 2023-12-19T16:58:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.260+0000 7f656b7fe640 1 -- 172.21.15.26:0/1384368165 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65580a9b10 con 0x7f656c15f370 2023-12-19T16:58:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.260+0000 7f6573012640 1 -- 172.21.15.26:0/1384368165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c15f370 msgr2=0x7f656c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.260+0000 7f6573012640 1 --2- 172.21.15.26:0/1384368165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c15f370 0x7f656c06c080 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f6558009870 tx=0x7f655801c540 comp rx=0 tx=0).stop 2023-12-19T16:58:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.260+0000 7f6573012640 1 -- 172.21.15.26:0/1384368165 shutdown_connections 2023-12-19T16:58:25.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.260+0000 7f6573012640 1 --2- 172.21.15.26:0/1384368165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c15f370 0x7f656c06c080 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.260+0000 7f6573012640 1 --2- 172.21.15.26:0/1384368165 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f656c155030 0x7f656c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.260+0000 7f6573012640 1 -- 172.21.15.26:0/1384368165 >> 172.21.15.26:0/1384368165 conn(0x7f656c0658e0 msgr2=0x7f656c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.260+0000 7f6573012640 1 -- 172.21.15.26:0/1384368165 shutdown_connections 2023-12-19T16:58:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.260+0000 7f6573012640 1 -- 172.21.15.26:0/1384368165 wait complete. 2023-12-19T16:58:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6573012640 1 Processor -- start 2023-12-19T16:58:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6573012640 1 -- start start 2023-12-19T16:58:25.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6573012640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c0ce740 0x7f656c150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:25.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6573012640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f656c155030 0x7f656c15a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:25.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6573012640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f656c15f370 0x7f656c159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:25.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6573012640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c151280 con 0x7f656c0ce740 2023-12-19T16:58:25.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6573012640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c0b23a0 con 0x7f656c15f370 2023-12-19T16:58:25.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6573012640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c0b2680 con 0x7f656c155030 2023-12-19T16:58:25.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6570dae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c0ce740 0x7f656c150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:25.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6570dae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c0ce740 0x7f656c150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45036/0 (socket says 172.21.15.26:45036) 2023-12-19T16:58:25.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6570dae640 1 -- 172.21.15.26:0/3426882683 learned_addr learned my addr 172.21.15.26:0/3426882683 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:25.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f656bfff640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f656c155030 0x7f656c15a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:25.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f65715af640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f656c15f370 0x7f656c159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:25.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6570dae640 1 -- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f656c155030 msgr2=0x7f656c15a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6570dae640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f656c155030 0x7f656c15a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6570dae640 1 -- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f656c15f370 msgr2=0x7f656c159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6570dae640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f656c15f370 0x7f656c159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6570dae640 1 -- 172.21.15.26:0/3426882683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65580094d0 con 0x7f656c0ce740 2023-12-19T16:58:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6570dae640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c0ce740 0x7f656c150d40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f656000d800 tx=0x7f656000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f65697fa640 1 -- 172.21.15.26:0/3426882683 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656000b6c0 con 0x7f656c0ce740 2023-12-19T16:58:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f65697fa640 1 -- 172.21.15.26:0/3426882683 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656000b820 con 0x7f656c0ce740 2023-12-19T16:58:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f65697fa640 1 -- 172.21.15.26:0/3426882683 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656000bb90 con 0x7f656c0ce740 2023-12-19T16:58:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6573012640 1 -- 172.21.15.26:0/3426882683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656c06c740 con 0x7f656c0ce740 2023-12-19T16:58:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6573012640 1 -- 172.21.15.26:0/3426882683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c0b2c40 con 0x7f656c0ce740 2023-12-19T16:58:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.264+0000 7f6573012640 1 -- 172.21.15.26:0/3426882683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656c074270 con 0x7f656c0ce740 2023-12-19T16:58:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.268+0000 7f65697fa640 1 -- 172.21.15.26:0/3426882683 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f656000bcf0 con 0x7f656c0ce740 2023-12-19T16:58:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.268+0000 7f65697fa640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6550065b80 0x7f6550068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.268+0000 7f65697fa640 1 -- 172.21.15.26:0/3426882683 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f6560083840 con 0x7f656c0ce740 2023-12-19T16:58:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.268+0000 7f656bfff640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6550065b80 0x7f6550068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:25.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.268+0000 7f656bfff640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6550065b80 0x7f6550068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f65580ae000 tx=0x7f65580b3040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:25.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.272+0000 7f65697fa640 1 -- 172.21.15.26:0/3426882683 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6560051150 con 0x7f656c0ce740 2023-12-19T16:58:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f6573012640 1 -- 172.21.15.26:0/3426882683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f656c0ccc30 con 0x7f656c0ce740 2023-12-19T16:58:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f65697fa640 1 -- 172.21.15.26:0/3426882683 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6560011340 con 0x7f656c0ce740 2023-12-19T16:58:25.486 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:25.486 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T16:58:25.486 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2023-12-19T16:58:25.486 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.486 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2023-12-19T16:58:25.486 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg==", 2023-12-19T16:58:25.486 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.486 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:58:25.486 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:58:25.486 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:58:25.486 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.486 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg==", 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2023-12-19T16:58:25.487 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg==", 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw==", 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:58:25.488 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig==", 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2023-12-19T16:58:25.489 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA==", 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ==", 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.490 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ==", 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A==", 2023-12-19T16:58:25.491 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ==", 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg==", 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg==", 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.493 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ==", 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.494 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ==", 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ==", 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:58:25.495 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ==", 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlT5ynDxAA7Sjbdwrc1RyAXJboysB54Q==", 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA==", 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.497 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg==", 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:25.498 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-12-19T16:58:25.499 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T16:58:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f6573012640 1 -- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6550065b80 msgr2=0x7f6550068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f6573012640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6550065b80 0x7f6550068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f65580ae000 tx=0x7f65580b3040 comp rx=0 tx=0).stop 2023-12-19T16:58:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f6573012640 1 -- 172.21.15.26:0/3426882683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c0ce740 msgr2=0x7f656c150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f6573012640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c0ce740 0x7f656c150d40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f656000d800 tx=0x7f656000dcc0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f6573012640 1 -- 172.21.15.26:0/3426882683 shutdown_connections 2023-12-19T16:58:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f6573012640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f656c15f370 0x7f656c159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f6573012640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6550065b80 0x7f6550068040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f6573012640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f656c155030 0x7f656c15a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f6573012640 1 --2- 172.21.15.26:0/3426882683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f656c0ce740 0x7f656c150d40 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f6573012640 1 -- 172.21.15.26:0/3426882683 >> 172.21.15.26:0/3426882683 conn(0x7f656c0658e0 msgr2=0x7f656c15da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f6573012640 1 -- 172.21.15.26:0/3426882683 shutdown_connections 2023-12-19T16:58:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.484+0000 7f6573012640 1 -- 172.21.15.26:0/3426882683 wait complete. 2023-12-19T16:58:25.509 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:58:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b9412640 1 Processor -- start 2023-12-19T16:58:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b9412640 1 -- start start 2023-12-19T16:58:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b9412640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff6b40cfbe0 0x7ff6b415de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b9412640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff6b40b8340 0x7ff6b40d0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b9412640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b40cd240 con 0x7ff6b415ccb0 2023-12-19T16:58:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b9412640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b40b8ac0 con 0x7ff6b40cfbe0 2023-12-19T16:58:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b9412640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b40cdbe0 con 0x7ff6b40b8340 2023-12-19T16:58:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b27fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff6b40cfbe0 0x7ff6b415de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b27fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff6b40cfbe0 0x7ff6b415de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45040/0 (socket says 172.21.15.26:45040) 2023-12-19T16:58:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b27fc640 1 -- 172.21.15.26:0/1025513932 learned_addr learned my addr 172.21.15.26:0/1025513932 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b37fe640 1 --2- 172.21.15.26:0/1025513932 >> v2:172.21.15.160:3300/0 conn(0x7ff6b40b8340 0x7ff6b40d0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b27fc640 1 -- 172.21.15.26:0/1025513932 >> v2:172.21.15.160:3300/0 conn(0x7ff6b40b8340 msgr2=0x7ff6b40d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b27fc640 1 --2- 172.21.15.26:0/1025513932 >> v2:172.21.15.160:3300/0 conn(0x7ff6b40b8340 0x7ff6b40d0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b27fc640 1 -- 172.21.15.26:0/1025513932 >> v1:172.21.15.160:6789/0 conn(0x7ff6b415ccb0 legacy=0x7ff6b40cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.652+0000 7ff6b27fc640 1 -- 172.21.15.26:0/1025513932 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b40cd520 con 0x7ff6b40cfbe0 2023-12-19T16:58:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.656+0000 7ff6b27fc640 1 --2- 172.21.15.26:0/1025513932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b40cfbe0 0x7ff6b415de70 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4066ec0 tx=0x7ff6a801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518259bb439a3ce8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.656+0000 7ff6b1ffb640 1 -- 172.21.15.26:0/1025513932 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a80aa650 con 0x7ff6b40cfbe0 2023-12-19T16:58:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.656+0000 7ff6b1ffb640 1 -- 172.21.15.26:0/1025513932 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a80aa7b0 con 0x7ff6b40cfbe0 2023-12-19T16:58:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.656+0000 7ff6b1ffb640 1 -- 172.21.15.26:0/1025513932 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a80aab40 con 0x7ff6b40cfbe0 2023-12-19T16:58:25.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.656+0000 7ff6b9412640 1 -- 172.21.15.26:0/1025513932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b40cfbe0 msgr2=0x7ff6b415de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.656+0000 7ff6b9412640 1 --2- 172.21.15.26:0/1025513932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b40cfbe0 0x7ff6b415de70 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4066ec0 tx=0x7ff6a801c4d0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.656+0000 7ff6b9412640 1 -- 172.21.15.26:0/1025513932 shutdown_connections 2023-12-19T16:58:25.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.656+0000 7ff6b9412640 1 --2- 172.21.15.26:0/1025513932 >> v2:172.21.15.160:3300/0 conn(0x7ff6b40b8340 0x7ff6b40d0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.656+0000 7ff6b9412640 1 --2- 172.21.15.26:0/1025513932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b40cfbe0 0x7ff6b415de70 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.656+0000 7ff6b9412640 1 -- 172.21.15.26:0/1025513932 >> 172.21.15.26:0/1025513932 conn(0x7ff6b40658e0 msgr2=0x7ff6b4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.656+0000 7ff6b9412640 1 -- 172.21.15.26:0/1025513932 shutdown_connections 2023-12-19T16:58:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.656+0000 7ff6b9412640 1 -- 172.21.15.26:0/1025513932 wait complete. 2023-12-19T16:58:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.656+0000 7ff6b9412640 1 Processor -- start 2023-12-19T16:58:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b9412640 1 -- start start 2023-12-19T16:58:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b9412640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6b40b8340 0x7ff6b40b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b9412640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6b40cfbe0 0x7ff6b414e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b9412640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b415ccb0 0x7ff6b4155a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b9412640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b4149910 con 0x7ff6b415ccb0 2023-12-19T16:58:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b9412640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b4156a50 con 0x7ff6b40cfbe0 2023-12-19T16:58:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b9412640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b4156d30 con 0x7ff6b40b8340 2023-12-19T16:58:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b415ccb0 0x7ff6b4155a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b415ccb0 0x7ff6b4155a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45050/0 (socket says 172.21.15.26:45050) 2023-12-19T16:58:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b37fe640 1 -- 172.21.15.26:0/1795420554 learned_addr learned my addr 172.21.15.26:0/1795420554 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b27fc640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6b40cfbe0 0x7ff6b414e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b2ffd640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6b40b8340 0x7ff6b40b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b37fe640 1 -- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6b40b8340 msgr2=0x7ff6b40b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b37fe640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6b40b8340 0x7ff6b40b7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b37fe640 1 -- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6b40cfbe0 msgr2=0x7ff6b414e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b37fe640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6b40cfbe0 0x7ff6b414e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b37fe640 1 -- 172.21.15.26:0/1795420554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a80094d0 con 0x7ff6b415ccb0 2023-12-19T16:58:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b37fe640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b415ccb0 0x7ff6b4155a40 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00efc0 tx=0x7ff6ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff69b7fe640 1 -- 172.21.15.26:0/1795420554 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac014480 con 0x7ff6b415ccb0 2023-12-19T16:58:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff69b7fe640 1 -- 172.21.15.26:0/1795420554 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6ac00f040 con 0x7ff6b415ccb0 2023-12-19T16:58:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b9412640 1 -- 172.21.15.26:0/1795420554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b4157010 con 0x7ff6b415ccb0 2023-12-19T16:58:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff69b7fe640 1 -- 172.21.15.26:0/1795420554 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6ac014820 con 0x7ff6b415ccb0 2023-12-19T16:58:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b9412640 1 -- 172.21.15.26:0/1795420554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b414ee90 con 0x7ff6b415ccb0 2023-12-19T16:58:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.660+0000 7ff6b9412640 1 -- 172.21.15.26:0/1795420554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b414f140 con 0x7ff6b415ccb0 2023-12-19T16:58:25.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.664+0000 7ff69b7fe640 1 -- 172.21.15.26:0/1795420554 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff6ac014980 con 0x7ff6b415ccb0 2023-12-19T16:58:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.664+0000 7ff69b7fe640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff6a0065b80 0x7ff6a0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.664+0000 7ff69b7fe640 1 -- 172.21.15.26:0/1795420554 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5589+0+0 (secure 0 0 0) 0x7ff6ac083f70 con 0x7ff6b415ccb0 2023-12-19T16:58:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.664+0000 7ff6b2ffd640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff6a0065b80 0x7ff6a0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.664+0000 7ff6b2ffd640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff6a0065b80 0x7ff6a0068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4007fd0 tx=0x7ff6a4009040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:25.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.668+0000 7ff69b7fe640 1 -- 172.21.15.26:0/1795420554 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff6b414f140 con 0x7ff6b415ccb0 2023-12-19T16:58:25.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.824+0000 7ff6b9412640 1 -- 172.21.15.26:0/1795420554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6b414f140 con 0x7ff6b415ccb0 2023-12-19T16:58:25.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.828+0000 7ff69b7fe640 1 -- 172.21.15.26:0/1795420554 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+13496 (secure 0 0 0) 0x7ff6b414f140 con 0x7ff6b415ccb0 2023-12-19T16:58:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:25.828 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":103,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:58:25.065498+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:58:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.828+0000 7ff6b9412640 1 -- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff6a0065b80 msgr2=0x7ff6a0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.828+0000 7ff6b9412640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff6a0065b80 0x7ff6a0068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4007fd0 tx=0x7ff6a4009040 comp rx=0 tx=0).stop 2023-12-19T16:58:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.828+0000 7ff6b9412640 1 -- 172.21.15.26:0/1795420554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b415ccb0 msgr2=0x7ff6b4155a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.828+0000 7ff6b9412640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b415ccb0 0x7ff6b4155a40 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac00efc0 tx=0x7ff6ac00c490 comp rx=0 tx=0).stop 2023-12-19T16:58:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.828+0000 7ff6b9412640 1 -- 172.21.15.26:0/1795420554 shutdown_connections 2023-12-19T16:58:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.828+0000 7ff6b9412640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6b415ccb0 0x7ff6b4155a40 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.828+0000 7ff6b9412640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6b40cfbe0 0x7ff6b414e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.828+0000 7ff6b9412640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff6a0065b80 0x7ff6a0068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.828+0000 7ff6b9412640 1 --2- 172.21.15.26:0/1795420554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6b40b8340 0x7ff6b40b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.828+0000 7ff6b9412640 1 -- 172.21.15.26:0/1795420554 >> 172.21.15.26:0/1795420554 conn(0x7ff6b40658e0 msgr2=0x7ff6b41544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.828+0000 7ff6b9412640 1 -- 172.21.15.26:0/1795420554 shutdown_connections 2023-12-19T16:58:25.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.828+0000 7ff6b9412640 1 -- 172.21.15.26:0/1795420554 wait complete. 2023-12-19T16:58:25.850 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:58:25.850 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:58:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f9932560640 1 Processor -- start 2023-12-19T16:58:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f9932560640 1 -- start start 2023-12-19T16:58:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f9932560640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f992c159140 0x7f992c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f9932560640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f992c06b390 0x7f992c06f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f9932560640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992c06f490 con 0x7f992c0cd190 2023-12-19T16:58:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f9932560640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992c159520 con 0x7f992c159140 2023-12-19T16:58:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f9932560640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992c0cd690 con 0x7f992c06b390 2023-12-19T16:58:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f992bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f992c159140 0x7f992c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f992bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f992c159140 0x7f992c0ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46494/0 (socket says 172.21.15.26:46494) 2023-12-19T16:58:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f992bfff640 1 -- 172.21.15.26:0/1751623904 learned_addr learned my addr 172.21.15.26:0/1751623904 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f9930afd640 1 --2- 172.21.15.26:0/1751623904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f992c06b390 0x7f992c06f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f992bfff640 1 -- 172.21.15.26:0/1751623904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f992c06b390 msgr2=0x7f992c06f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f992bfff640 1 --2- 172.21.15.26:0/1751623904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f992c06b390 0x7f992c06f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f992bfff640 1 -- 172.21.15.26:0/1751623904 >> v1:172.21.15.160:6789/0 conn(0x7f992c0cd190 legacy=0x7f992c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f992bfff640 1 -- 172.21.15.26:0/1751623904 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c0cd970 con 0x7f992c159140 2023-12-19T16:58:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:25.996+0000 7f9930afd640 1 --2- 172.21.15.26:0/1751623904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f992c06b390 0x7f992c06f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:26.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.000+0000 7f992bfff640 1 --2- 172.21.15.26:0/1751623904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992c159140 0x7f992c0ccc50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f991c009870 tx=0x7f991c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4cfc2e7f4d0d92 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.000+0000 7f992affd640 1 -- 172.21.15.26:0/1751623904 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c0ab600 con 0x7f992c159140 2023-12-19T16:58:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.000+0000 7f992affd640 1 -- 172.21.15.26:0/1751623904 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991c0a7050 con 0x7f992c159140 2023-12-19T16:58:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.000+0000 7f992affd640 1 -- 172.21.15.26:0/1751623904 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c0ab980 con 0x7f992c159140 2023-12-19T16:58:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.000+0000 7f9932560640 1 -- 172.21.15.26:0/1751623904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992c159140 msgr2=0x7f992c0ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.000+0000 7f9932560640 1 --2- 172.21.15.26:0/1751623904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992c159140 0x7f992c0ccc50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f991c009870 tx=0x7f991c01c4d0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.000+0000 7f9932560640 1 -- 172.21.15.26:0/1751623904 shutdown_connections 2023-12-19T16:58:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.000+0000 7f9932560640 1 --2- 172.21.15.26:0/1751623904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f992c06b390 0x7f992c06f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.000+0000 7f9932560640 1 --2- 172.21.15.26:0/1751623904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992c159140 0x7f992c0ccc50 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.000+0000 7f9932560640 1 -- 172.21.15.26:0/1751623904 >> 172.21.15.26:0/1751623904 conn(0x7f992c0ce730 msgr2=0x7f992c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.000+0000 7f9932560640 1 -- 172.21.15.26:0/1751623904 shutdown_connections 2023-12-19T16:58:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.000+0000 7f9932560640 1 -- 172.21.15.26:0/1751623904 wait complete. 2023-12-19T16:58:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.000+0000 7f9932560640 1 Processor -- start 2023-12-19T16:58:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9932560640 1 -- start start 2023-12-19T16:58:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9932560640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992c06b390 0x7f992c0c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9932560640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f992c0cd190 0x7f992c0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9932560640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f992c159140 0x7f992c0acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9932560640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c168210 con 0x7f992c06b390 2023-12-19T16:58:26.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9932560640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c167690 con 0x7f992c159140 2023-12-19T16:58:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9932560640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992c167970 con 0x7f992c0cd190 2023-12-19T16:58:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f992bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992c06b390 0x7f992c0c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f992bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992c06b390 0x7f992c0c3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46506/0 (socket says 172.21.15.26:46506) 2023-12-19T16:58:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f992bfff640 1 -- 172.21.15.26:0/3892979878 learned_addr learned my addr 172.21.15.26:0/3892979878 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9930afd640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f992c159140 0x7f992c0acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f992b7fe640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f992c0cd190 0x7f992c0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f992bfff640 1 -- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f992c0cd190 msgr2=0x7f992c0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f992bfff640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f992c0cd190 0x7f992c0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f992bfff640 1 -- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f992c159140 msgr2=0x7f992c0acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f992bfff640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f992c159140 0x7f992c0acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f992bfff640 1 -- 172.21.15.26:0/3892979878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c0094d0 con 0x7f992c06b390 2023-12-19T16:58:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f992bfff640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992c06b390 0x7f992c0c3d00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f991c0ae040 tx=0x7f991c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:26.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9928ff9640 1 -- 172.21.15.26:0/3892979878 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c0b4640 con 0x7f992c06b390 2023-12-19T16:58:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9928ff9640 1 -- 172.21.15.26:0/3892979878 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991c0b47a0 con 0x7f992c06b390 2023-12-19T16:58:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9928ff9640 1 -- 172.21.15.26:0/3892979878 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c0b4b30 con 0x7f992c06b390 2023-12-19T16:58:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9932560640 1 -- 172.21.15.26:0/3892979878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992c167c50 con 0x7f992c06b390 2023-12-19T16:58:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9932560640 1 -- 172.21.15.26:0/3892979878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992c0c4240 con 0x7f992c06b390 2023-12-19T16:58:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.004+0000 7f9932560640 1 -- 172.21.15.26:0/3892979878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992c074270 con 0x7f992c06b390 2023-12-19T16:58:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.008+0000 7f9928ff9640 1 -- 172.21.15.26:0/3892979878 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f991c0b2070 con 0x7f992c06b390 2023-12-19T16:58:26.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.008+0000 7f9928ff9640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9918065b80 0x7f9918068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.008+0000 7f9928ff9640 1 -- 172.21.15.26:0/3892979878 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f991c11d480 con 0x7f992c06b390 2023-12-19T16:58:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.008+0000 7f992b7fe640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9918065b80 0x7f9918068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.008+0000 7f992b7fe640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9918065b80 0x7f9918068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f992c0c3280 tx=0x7f9920009040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:26.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.012+0000 7f9928ff9640 1 -- 172.21.15.26:0/3892979878 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f991c0eab60 con 0x7f992c06b390 2023-12-19T16:58:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.168+0000 7f9932560640 1 -- 172.21.15.26:0/3892979878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f992c0711b0 con 0x7f992c06b390 2023-12-19T16:58:26.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.168+0000 7f9928ff9640 1 -- 172.21.15.26:0/3892979878 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f991c0b0020 con 0x7f992c06b390 2023-12-19T16:58:26.170 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:58:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.172+0000 7f9932560640 1 -- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9918065b80 msgr2=0x7f9918068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.172+0000 7f9932560640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9918065b80 0x7f9918068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f992c0c3280 tx=0x7f9920009040 comp rx=0 tx=0).stop 2023-12-19T16:58:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.172+0000 7f9932560640 1 -- 172.21.15.26:0/3892979878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992c06b390 msgr2=0x7f992c0c3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.172+0000 7f9932560640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992c06b390 0x7f992c0c3d00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f991c0ae040 tx=0x7f991c019270 comp rx=0 tx=0).stop 2023-12-19T16:58:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.172+0000 7f9932560640 1 -- 172.21.15.26:0/3892979878 shutdown_connections 2023-12-19T16:58:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.172+0000 7f9932560640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f992c159140 0x7f992c0acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.172+0000 7f9932560640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9918065b80 0x7f9918068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.172+0000 7f9932560640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f992c0cd190 0x7f992c0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.172+0000 7f9932560640 1 --2- 172.21.15.26:0/3892979878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f992c06b390 0x7f992c0c3d00 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.172+0000 7f9932560640 1 -- 172.21.15.26:0/3892979878 >> 172.21.15.26:0/3892979878 conn(0x7f992c0ce730 msgr2=0x7f992c0c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.172+0000 7f9932560640 1 -- 172.21.15.26:0/3892979878 shutdown_connections 2023-12-19T16:58:26.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.172+0000 7f9932560640 1 -- 172.21.15.26:0/3892979878 wait complete. 2023-12-19T16:58:26.192 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T16:58:26.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.332+0000 7f6f2f798640 1 Processor -- start 2023-12-19T16:58:26.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.332+0000 7f6f2f798640 1 -- start start 2023-12-19T16:58:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2f798640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6f280c9620 0x7f6f280c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2f798640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f280cefa0 0x7f6f280cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2f798640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f280cba80 con 0x7f6f28159260 2023-12-19T16:58:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2f798640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f280cf970 con 0x7f6f280c9620 2023-12-19T16:58:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2f798640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28159640 con 0x7f6f280cefa0 2023-12-19T16:58:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2d534640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6f28159260 0x7f6f280c3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45264/0 (socket says 172.21.15.26:45264) 2023-12-19T16:58:26.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2d534640 1 -- 172.21.15.26:0/615597695 learned_addr learned my addr 172.21.15.26:0/615597695 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2cd33640 1 --2- 172.21.15.26:0/615597695 >> v2:172.21.15.160:3300/0 conn(0x7f6f280c9620 0x7f6f280c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2dd35640 1 --2- 172.21.15.26:0/615597695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f280cefa0 0x7f6f280cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388359494 0 0) 0x7f6f280cba80 con 0x7f6f28159260 2023-12-19T16:58:26.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f14003530 con 0x7f6f28159260 2023-12-19T16:58:26.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1169498777 0 0) 0x7f6f14003530 con 0x7f6f28159260 2023-12-19T16:58:26.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f280cefa0 msgr2=0x7f6f280cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:26.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 --2- 172.21.15.26:0/615597695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f280cefa0 0x7f6f280cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 >> v2:172.21.15.160:3300/0 conn(0x7f6f280c9620 msgr2=0x7f6f280c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:26.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 --2- 172.21.15.26:0/615597695 >> v2:172.21.15.160:3300/0 conn(0x7f6f280c9620 0x7f6f280c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f280c4be0 con 0x7f6f28159260 2023-12-19T16:58:26.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f6f24001800 con 0x7f6f28159260 2023-12-19T16:58:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 >> v1:172.21.15.26:6789/0 conn(0x7f6f28159260 legacy=0x7f6f280c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 --2- 172.21.15.26:0/615597695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f14003980 0x7f6f14003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 --2- 172.21.15.26:0/615597695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f280cefa0 0x7f6f140047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 --2- 172.21.15.26:0/615597695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f14004d10 0x7f6f14005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f280cf970 con 0x7f6f14004d10 2023-12-19T16:58:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28159640 con 0x7f6f14003980 2023-12-19T16:58:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1400b990 con 0x7f6f280cefa0 2023-12-19T16:58:26.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2dd35640 1 --2- 172.21.15.26:0/615597695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f14004d10 0x7f6f14005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2d534640 1 --2- 172.21.15.26:0/615597695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f14003980 0x7f6f14003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2cd33640 1 --2- 172.21.15.26:0/615597695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f280cefa0 0x7f6f140047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2dd35640 1 -- 172.21.15.26:0/615597695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f280cefa0 msgr2=0x7f6f140047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:26.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2dd35640 1 --2- 172.21.15.26:0/615597695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f280cefa0 0x7f6f140047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2dd35640 1 -- 172.21.15.26:0/615597695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f14003980 msgr2=0x7f6f14003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:26.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2dd35640 1 --2- 172.21.15.26:0/615597695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f14003980 0x7f6f14003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.336+0000 7f6f2dd35640 1 -- 172.21.15.26:0/615597695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f14003530 con 0x7f6f14004d10 2023-12-19T16:58:26.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.340+0000 7f6f2dd35640 1 --2- 172.21.15.26:0/615597695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f14004d10 0x7f6f14005190 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009fc0 tx=0x7f6f2001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1c299ac770b625 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:26.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.340+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f200a5c70 con 0x7f6f14004d10 2023-12-19T16:58:26.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.340+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2000f040 con 0x7f6f14004d10 2023-12-19T16:58:26.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.340+0000 7f6f1ffff640 1 -- 172.21.15.26:0/615597695 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f200ad440 con 0x7f6f14004d10 2023-12-19T16:58:26.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.340+0000 7f6f2f798640 1 -- 172.21.15.26:0/615597695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f14004d10 msgr2=0x7f6f14005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.340+0000 7f6f2f798640 1 --2- 172.21.15.26:0/615597695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f14004d10 0x7f6f14005190 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009fc0 tx=0x7f6f2001cc60 comp rx=0 tx=0).stop 2023-12-19T16:58:26.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.340+0000 7f6f2dd35640 1 -- 172.21.15.26:0/615597695 reap_dead start 2023-12-19T16:58:26.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.340+0000 7f6f2f798640 1 -- 172.21.15.26:0/615597695 shutdown_connections 2023-12-19T16:58:26.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.340+0000 7f6f2f798640 1 -- 172.21.15.26:0/615597695 >> 172.21.15.26:0/615597695 conn(0x7f6f2806d5d0 msgr2=0x7f6f28069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:26.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.340+0000 7f6f2f798640 1 -- 172.21.15.26:0/615597695 shutdown_connections 2023-12-19T16:58:26.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.340+0000 7f6f2f798640 1 -- 172.21.15.26:0/615597695 wait complete. 2023-12-19T16:58:26.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.340+0000 7f6f2f798640 1 Processor -- start 2023-12-19T16:58:26.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.340+0000 7f6f2f798640 1 -- start start 2023-12-19T16:58:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2f798640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f14004d10 0x7f6f281662a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2f798640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f28144870 0x7f6f281679d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2f798640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f28167f10 0x7f6f280b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2f798640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28167540 con 0x7f6f28144870 2023-12-19T16:58:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2f798640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28141910 con 0x7f6f14004d10 2023-12-19T16:58:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2f798640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f28166930 con 0x7f6f28167f10 2023-12-19T16:58:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2cd33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f28144870 0x7f6f281679d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2cd33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f28144870 0x7f6f281679d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46512/0 (socket says 172.21.15.26:46512) 2023-12-19T16:58:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2cd33640 1 -- 172.21.15.26:0/851423694 learned_addr learned my addr 172.21.15.26:0/851423694 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2d534640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f14004d10 0x7f6f281662a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2dd35640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f28167f10 0x7f6f280b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2cd33640 1 -- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f28167f10 msgr2=0x7f6f280b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2cd33640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f28167f10 0x7f6f280b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2cd33640 1 -- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f14004d10 msgr2=0x7f6f281662a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2cd33640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f14004d10 0x7f6f281662a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2cd33640 1 -- 172.21.15.26:0/851423694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f28166c10 con 0x7f6f28144870 2023-12-19T16:58:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2d534640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f14004d10 0x7f6f281662a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2cd33640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f28144870 0x7f6f281679d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6f18011aa0 tx=0x7f6f18011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f1dffb640 1 -- 172.21.15.26:0/851423694 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1800fb80 con 0x7f6f28144870 2023-12-19T16:58:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f1dffb640 1 -- 172.21.15.26:0/851423694 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1800fce0 con 0x7f6f28144870 2023-12-19T16:58:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2f798640 1 -- 172.21.15.26:0/851423694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f28166ef0 con 0x7f6f28144870 2023-12-19T16:58:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f1dffb640 1 -- 172.21.15.26:0/851423694 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f18014680 con 0x7f6f28144870 2023-12-19T16:58:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2f798640 1 -- 172.21.15.26:0/851423694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f280b24a0 con 0x7f6f28144870 2023-12-19T16:58:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f2f798640 1 -- 172.21.15.26:0/851423694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f281682f0 con 0x7f6f28144870 2023-12-19T16:58:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.344+0000 7f6f1dffb640 1 -- 172.21.15.26:0/851423694 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6f1801d400 con 0x7f6f28144870 2023-12-19T16:58:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.348+0000 7f6f1dffb640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f0c065b80 0x7f6f0c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.348+0000 7f6f1dffb640 1 -- 172.21.15.26:0/851423694 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f6f18083cd0 con 0x7f6f28144870 2023-12-19T16:58:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.348+0000 7f6f2d534640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f0c065b80 0x7f6f0c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.348+0000 7f6f2d534640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f0c065b80 0x7f6f0c068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6f24000c10 tx=0x7f6f24009040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.352+0000 7f6f1dffb640 1 -- 172.21.15.26:0/851423694 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6f18051550 con 0x7f6f28144870 2023-12-19T16:58:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.508+0000 7f6f2f798640 1 -- 172.21.15.26:0/851423694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6f28074270 con 0x7f6f28144870 2023-12-19T16:58:26.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.508+0000 7f6f1dffb640 1 -- 172.21.15.26:0/851423694 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6f18054b70 con 0x7f6f28144870 2023-12-19T16:58:26.511 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:26.512 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T16:58:26.512 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:26.512 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T16:58:26.512 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T16:58:26.512 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:26.512 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T16:58:26.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.512+0000 7f6f2f798640 1 -- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f0c065b80 msgr2=0x7f6f0c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.512+0000 7f6f2f798640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f0c065b80 0x7f6f0c068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6f24000c10 tx=0x7f6f24009040 comp rx=0 tx=0).stop 2023-12-19T16:58:26.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.512+0000 7f6f2f798640 1 -- 172.21.15.26:0/851423694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f28144870 msgr2=0x7f6f281679d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.512+0000 7f6f2f798640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f28144870 0x7f6f281679d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f6f18011aa0 tx=0x7f6f18011f60 comp rx=0 tx=0).stop 2023-12-19T16:58:26.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.512+0000 7f6f2f798640 1 -- 172.21.15.26:0/851423694 shutdown_connections 2023-12-19T16:58:26.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.512+0000 7f6f2f798640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f0c065b80 0x7f6f0c068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.512+0000 7f6f2f798640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f28167f10 0x7f6f280b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.512+0000 7f6f2f798640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f28144870 0x7f6f281679d0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.512+0000 7f6f2f798640 1 --2- 172.21.15.26:0/851423694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f14004d10 0x7f6f281662a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.512+0000 7f6f2f798640 1 -- 172.21.15.26:0/851423694 >> 172.21.15.26:0/851423694 conn(0x7f6f2806d5d0 msgr2=0x7f6f280c3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:26.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.512+0000 7f6f2f798640 1 -- 172.21.15.26:0/851423694 shutdown_connections 2023-12-19T16:58:26.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.512+0000 7f6f2f798640 1 -- 172.21.15.26:0/851423694 wait complete. 2023-12-19T16:58:26.532 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T16:58:26.533 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T16:58:26.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f915c640 1 Processor -- start 2023-12-19T16:58:26.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f915c640 1 -- start start 2023-12-19T16:58:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f915c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f37f415c620 0x7f37f40b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f915c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37f406b3b0 0x7f37f415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f915c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f415cab0 con 0x7f37f40b37f0 2023-12-19T16:58:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f915c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37f40b26f0 con 0x7f37f415c620 2023-12-19T16:58:26.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f915c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40b1e50 con 0x7f37f406b3b0 2023-12-19T16:58:26.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f895b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37f406b3b0 0x7f37f415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f37fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f37f415c620 0x7f37f40b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f895b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37f406b3b0 0x7f37f415cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:55584/0 (socket says 172.21.15.26:55584) 2023-12-19T16:58:26.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f895b640 1 -- 172.21.15.26:0/85925690 learned_addr learned my addr 172.21.15.26:0/85925690 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:26.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f37fe640 1 -- 172.21.15.26:0/85925690 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37f406b3b0 msgr2=0x7f37f415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f37fe640 1 --2- 172.21.15.26:0/85925690 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37f406b3b0 0x7f37f415cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f37fe640 1 -- 172.21.15.26:0/85925690 >> v1:172.21.15.160:6789/0 conn(0x7f37f40b37f0 legacy=0x7f37f415a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f37fe640 1 -- 172.21.15.26:0/85925690 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f40b2130 con 0x7f37f415c620 2023-12-19T16:58:26.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.680+0000 7f37f895b640 1 --2- 172.21.15.26:0/85925690 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37f406b3b0 0x7f37f415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:26.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.684+0000 7f37f37fe640 1 --2- 172.21.15.26:0/85925690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37f415c620 0x7f37f40b15c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f37e80097c0 tx=0x7f37e801e5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfaa804126ed693a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.684+0000 7f37f2ffd640 1 -- 172.21.15.26:0/85925690 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e80b2600 con 0x7f37f415c620 2023-12-19T16:58:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.684+0000 7f37f2ffd640 1 -- 172.21.15.26:0/85925690 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e80b2760 con 0x7f37f415c620 2023-12-19T16:58:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.684+0000 7f37f2ffd640 1 -- 172.21.15.26:0/85925690 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e80b2af0 con 0x7f37f415c620 2023-12-19T16:58:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.684+0000 7f37f915c640 1 -- 172.21.15.26:0/85925690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37f415c620 msgr2=0x7f37f40b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.684+0000 7f37f915c640 1 --2- 172.21.15.26:0/85925690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37f415c620 0x7f37f40b15c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f37e80097c0 tx=0x7f37e801e5d0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.684+0000 7f37f915c640 1 -- 172.21.15.26:0/85925690 shutdown_connections 2023-12-19T16:58:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.684+0000 7f37f915c640 1 --2- 172.21.15.26:0/85925690 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37f406b3b0 0x7f37f415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.684+0000 7f37f915c640 1 --2- 172.21.15.26:0/85925690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37f415c620 0x7f37f40b15c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.684+0000 7f37f915c640 1 -- 172.21.15.26:0/85925690 >> 172.21.15.26:0/85925690 conn(0x7f37f40658e0 msgr2=0x7f37f4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:26.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.684+0000 7f37f915c640 1 -- 172.21.15.26:0/85925690 shutdown_connections 2023-12-19T16:58:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.684+0000 7f37f915c640 1 -- 172.21.15.26:0/85925690 wait complete. 2023-12-19T16:58:26.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f915c640 1 Processor -- start 2023-12-19T16:58:26.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f915c640 1 -- start start 2023-12-19T16:58:26.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f915c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37f406b3b0 0x7f37f4149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f915c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37f40b37f0 0x7f37f40cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f915c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37f415c620 0x7f37f406ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f915c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f41fe920 con 0x7f37f40b37f0 2023-12-19T16:58:26.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f915c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f41fa3a0 con 0x7f37f415c620 2023-12-19T16:58:26.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f915c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40cf4d0 con 0x7f37f406b3b0 2023-12-19T16:58:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37f40b37f0 0x7f37f40cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37f40b37f0 0x7f37f40cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46516/0 (socket says 172.21.15.26:46516) 2023-12-19T16:58:26.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f37fe640 1 -- 172.21.15.26:0/3982318066 learned_addr learned my addr 172.21.15.26:0/3982318066 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f895b640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37f415c620 0x7f37f406ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f37fe640 1 -- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37f406b3b0 msgr2=0x7f37f4149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f3fff640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37f406b3b0 0x7f37f4149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f37fe640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37f406b3b0 0x7f37f4149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f37fe640 1 -- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37f415c620 msgr2=0x7f37f406ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f37fe640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37f415c620 0x7f37f406ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f37fe640 1 -- 172.21.15.26:0/3982318066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ec009b30 con 0x7f37f40b37f0 2023-12-19T16:58:26.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f895b640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37f415c620 0x7f37f406ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f3fff640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37f406b3b0 0x7f37f4149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f37fe640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37f40b37f0 0x7f37f40cd6f0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f37e801cc80 tx=0x7f37e801b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f0ff9640 1 -- 172.21.15.26:0/3982318066 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e80b2da0 con 0x7f37f40b37f0 2023-12-19T16:58:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f0ff9640 1 -- 172.21.15.26:0/3982318066 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e80aa400 con 0x7f37f40b37f0 2023-12-19T16:58:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f0ff9640 1 -- 172.21.15.26:0/3982318066 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e80aa7b0 con 0x7f37f40b37f0 2023-12-19T16:58:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f915c640 1 -- 172.21.15.26:0/3982318066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e8009420 con 0x7f37f40b37f0 2023-12-19T16:58:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f915c640 1 -- 172.21.15.26:0/3982318066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f41fe080 con 0x7f37f40b37f0 2023-12-19T16:58:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.688+0000 7f37f915c640 1 -- 172.21.15.26:0/3982318066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f41fa4e0 con 0x7f37f40b37f0 2023-12-19T16:58:26.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.692+0000 7f37f0ff9640 1 -- 172.21.15.26:0/3982318066 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f37e80aa910 con 0x7f37f40b37f0 2023-12-19T16:58:26.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.692+0000 7f37f0ff9640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f37e0065b80 0x7f37e0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:26.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.692+0000 7f37f0ff9640 1 -- 172.21.15.26:0/3982318066 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f37e811d660 con 0x7f37f40b37f0 2023-12-19T16:58:26.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.692+0000 7f37f3fff640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f37e0065b80 0x7f37e0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:26.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.692+0000 7f37f3fff640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f37e0065b80 0x7f37e0068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f37e4000f70 tx=0x7f37e4009040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:26.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.696+0000 7f37f0ff9640 1 -- 172.21.15.26:0/3982318066 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f37e80eadd0 con 0x7f37f40b37f0 2023-12-19T16:58:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.856+0000 7f37f915c640 1 -- 172.21.15.26:0/3982318066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f37f404e850 con 0x7f37f40b37f0 2023-12-19T16:58:26.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.960+0000 7f37f0ff9640 1 -- 172.21.15.26:0/3982318066 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v104) v1 ==== 138+0+0 (secure 0 0 0) 0x7f37e80ee3f0 con 0x7f37f40b37f0 2023-12-19T16:58:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T16:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.964+0000 7f37f915c640 1 -- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f37e0065b80 msgr2=0x7f37e0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.964+0000 7f37f915c640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f37e0065b80 0x7f37e0068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f37e4000f70 tx=0x7f37e4009040 comp rx=0 tx=0).stop 2023-12-19T16:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.964+0000 7f37f915c640 1 -- 172.21.15.26:0/3982318066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37f40b37f0 msgr2=0x7f37f40cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.964+0000 7f37f915c640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37f40b37f0 0x7f37f40cd6f0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f37e801cc80 tx=0x7f37e801b370 comp rx=0 tx=0).stop 2023-12-19T16:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.964+0000 7f37f915c640 1 -- 172.21.15.26:0/3982318066 shutdown_connections 2023-12-19T16:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.964+0000 7f37f915c640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37f415c620 0x7f37f406ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.964+0000 7f37f915c640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37f40b37f0 0x7f37f40cd6f0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.964+0000 7f37f915c640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f37e0065b80 0x7f37e0068040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.964+0000 7f37f915c640 1 --2- 172.21.15.26:0/3982318066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37f406b3b0 0x7f37f4149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.964+0000 7f37f915c640 1 -- 172.21.15.26:0/3982318066 >> 172.21.15.26:0/3982318066 conn(0x7f37f40658e0 msgr2=0x7f37f415af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.964+0000 7f37f915c640 1 -- 172.21.15.26:0/3982318066 shutdown_connections 2023-12-19T16:58:26.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:26.964+0000 7f37f915c640 1 -- 172.21.15.26:0/3982318066 wait complete. 2023-12-19T16:58:26.987 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T16:58:27.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc552841640 1 Processor -- start 2023-12-19T16:58:27.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc552841640 1 -- start start 2023-12-19T16:58:27.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc552841640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc54c0adac0 0x7fc54c0c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:27.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc552841640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc54c15a480 0x7fc54c06fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:27.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc552841640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c070640 con 0x7fc54c0701f0 2023-12-19T16:58:27.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc552841640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c06b3d0 con 0x7fc54c0adac0 2023-12-19T16:58:27.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc552841640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c0ca5e0 con 0x7fc54c15a480 2023-12-19T16:58:27.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc54bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc54c0adac0 0x7fc54c0c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:27.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc550dde640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc54c0701f0 0x7fc54c15cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45280/0 (socket says 172.21.15.26:45280) 2023-12-19T16:58:27.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc550dde640 1 -- 172.21.15.26:0/1967149040 learned_addr learned my addr 172.21.15.26:0/1967149040 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:27.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc54bfff640 1 -- 172.21.15.26:0/1967149040 >> v2:172.21.15.160:3300/0 conn(0x7fc54c15a480 msgr2=0x7fc54c06fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:27.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc54bfff640 1 --2- 172.21.15.26:0/1967149040 >> v2:172.21.15.160:3300/0 conn(0x7fc54c15a480 0x7fc54c06fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:27.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc54bfff640 1 -- 172.21.15.26:0/1967149040 >> v1:172.21.15.26:6789/0 conn(0x7fc54c0701f0 legacy=0x7fc54c15cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:27.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.136+0000 7fc54bfff640 1 -- 172.21.15.26:0/1967149040 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c0ca8c0 con 0x7fc54c0adac0 2023-12-19T16:58:27.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.140+0000 7fc54bfff640 1 --2- 172.21.15.26:0/1967149040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54c0adac0 0x7fc54c0c9d90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc54c071700 tx=0x7fc53c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09baacf956a406a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.140+0000 7fc54affd640 1 -- 172.21.15.26:0/1967149040 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c0ab600 con 0x7fc54c0adac0 2023-12-19T16:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.140+0000 7fc54affd640 1 -- 172.21.15.26:0/1967149040 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c0a7050 con 0x7fc54c0adac0 2023-12-19T16:58:27.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.140+0000 7fc54affd640 1 -- 172.21.15.26:0/1967149040 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c0ab980 con 0x7fc54c0adac0 2023-12-19T16:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.140+0000 7fc552841640 1 -- 172.21.15.26:0/1967149040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54c0adac0 msgr2=0x7fc54c0c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.140+0000 7fc552841640 1 --2- 172.21.15.26:0/1967149040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54c0adac0 0x7fc54c0c9d90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc54c071700 tx=0x7fc53c01c430 comp rx=0 tx=0).stop 2023-12-19T16:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.140+0000 7fc552841640 1 -- 172.21.15.26:0/1967149040 shutdown_connections 2023-12-19T16:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.140+0000 7fc552841640 1 --2- 172.21.15.26:0/1967149040 >> v2:172.21.15.160:3300/0 conn(0x7fc54c15a480 0x7fc54c06fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.140+0000 7fc552841640 1 --2- 172.21.15.26:0/1967149040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54c0adac0 0x7fc54c0c9d90 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.140+0000 7fc552841640 1 -- 172.21.15.26:0/1967149040 >> 172.21.15.26:0/1967149040 conn(0x7fc54c0660d0 msgr2=0x7fc54c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.140+0000 7fc552841640 1 -- 172.21.15.26:0/1967149040 shutdown_connections 2023-12-19T16:58:27.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.140+0000 7fc552841640 1 -- 172.21.15.26:0/1967149040 wait complete. 2023-12-19T16:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc552841640 1 Processor -- start 2023-12-19T16:58:27.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc552841640 1 -- start start 2023-12-19T16:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc552841640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc54c0701f0 0x7fc54c14b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc552841640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c0adac0 0x7fc54c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc552841640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54c15a480 0x7fc54c0b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc552841640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0c5b90 con 0x7fc54c15a480 2023-12-19T16:58:27.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc552841640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c1662a0 con 0x7fc54c0adac0 2023-12-19T16:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc552841640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c167020 con 0x7fc54c0701f0 2023-12-19T16:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc550dde640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54c15a480 0x7fc54c0b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc550dde640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54c15a480 0x7fc54c0b3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46534/0 (socket says 172.21.15.26:46534) 2023-12-19T16:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc550dde640 1 -- 172.21.15.26:0/4049323205 learned_addr learned my addr 172.21.15.26:0/4049323205 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc54b7fe640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c0adac0 0x7fc54c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc54bfff640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc54c0701f0 0x7fc54c14b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc550dde640 1 -- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc54c0701f0 msgr2=0x7fc54c14b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc550dde640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc54c0701f0 0x7fc54c14b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc550dde640 1 -- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c0adac0 msgr2=0x7fc54c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc550dde640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c0adac0 0x7fc54c0b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:27.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc550dde640 1 -- 172.21.15.26:0/4049323205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c009420 con 0x7fc54c15a480 2023-12-19T16:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc550dde640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54c15a480 0x7fc54c0b3500 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc54400ca80 tx=0x7fc54400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc548ff9640 1 -- 172.21.15.26:0/4049323205 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54400ac30 con 0x7fc54c15a480 2023-12-19T16:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc548ff9640 1 -- 172.21.15.26:0/4049323205 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54400ad90 con 0x7fc54c15a480 2023-12-19T16:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc548ff9640 1 -- 172.21.15.26:0/4049323205 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544012600 con 0x7fc54c15a480 2023-12-19T16:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc552841640 1 -- 172.21.15.26:0/4049323205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c15d380 con 0x7fc54c15a480 2023-12-19T16:58:27.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.144+0000 7fc552841640 1 -- 172.21.15.26:0/4049323205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c166820 con 0x7fc54c15a480 2023-12-19T16:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.148+0000 7fc552841640 1 -- 172.21.15.26:0/4049323205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c074270 con 0x7fc54c15a480 2023-12-19T16:58:27.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.148+0000 7fc548ff9640 1 -- 172.21.15.26:0/4049323205 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc544007070 con 0x7fc54c15a480 2023-12-19T16:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.148+0000 7fc548ff9640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc538065b80 0x7fc538068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:27.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.148+0000 7fc548ff9640 1 -- 172.21.15.26:0/4049323205 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5959+0+0 (secure 0 0 0) 0x7fc544083480 con 0x7fc54c15a480 2023-12-19T16:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.148+0000 7fc54bfff640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc538065b80 0x7fc538068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:27.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.148+0000 7fc54bfff640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc538065b80 0x7fc538068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0ad000 tx=0x7fc53c008040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:27.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.152+0000 7fc548ff9640 1 -- 172.21.15.26:0/4049323205 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc544050dc0 con 0x7fc54c15a480 2023-12-19T16:58:27.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.308+0000 7fc552841640 1 -- 172.21.15.26:0/4049323205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc54c070640 con 0x7fc54c15a480 2023-12-19T16:58:27.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.964+0000 7fc548ff9640 1 -- 172.21.15.26:0/4049323205 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v105) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc54c070640 con 0x7fc54c15a480 2023-12-19T16:58:27.965 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T16:58:27.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.968+0000 7fc552841640 1 -- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc538065b80 msgr2=0x7fc538068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:27.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.968+0000 7fc552841640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc538065b80 0x7fc538068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0ad000 tx=0x7fc53c008040 comp rx=0 tx=0).stop 2023-12-19T16:58:27.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.968+0000 7fc552841640 1 -- 172.21.15.26:0/4049323205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54c15a480 msgr2=0x7fc54c0b3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:27.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.968+0000 7fc552841640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54c15a480 0x7fc54c0b3500 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc54400ca80 tx=0x7fc54400cf40 comp rx=0 tx=0).stop 2023-12-19T16:58:27.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.972+0000 7fc552841640 1 -- 172.21.15.26:0/4049323205 shutdown_connections 2023-12-19T16:58:27.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.972+0000 7fc552841640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc54c15a480 0x7fc54c0b3500 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:27.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.972+0000 7fc552841640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc54c0adac0 0x7fc54c0b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:27.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.972+0000 7fc552841640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc538065b80 0x7fc538068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:27.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.972+0000 7fc552841640 1 --2- 172.21.15.26:0/4049323205 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc54c0701f0 0x7fc54c14b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:27.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.972+0000 7fc552841640 1 -- 172.21.15.26:0/4049323205 >> 172.21.15.26:0/4049323205 conn(0x7fc54c0660d0 msgr2=0x7fc54c1519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:27.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.976+0000 7fc552841640 1 -- 172.21.15.26:0/4049323205 shutdown_connections 2023-12-19T16:58:27.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:27.976+0000 7fc552841640 1 -- 172.21.15.26:0/4049323205 wait complete. 2023-12-19T16:58:28.005 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T16:58:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.148+0000 7faabbbf3640 1 Processor -- start 2023-12-19T16:58:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.148+0000 7faabbbf3640 1 -- start start 2023-12-19T16:58:28.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faabbbf3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faab4159820 0x7faab415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faabbbf3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faab406f6c0 0x7faab40c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faabbbf3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab406fb60 con 0x7faab4159c00 2023-12-19T16:58:28.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faabbbf3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab415e470 con 0x7faab406f6c0 2023-12-19T16:58:28.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faabbbf3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab406fe40 con 0x7faab4159820 2023-12-19T16:58:28.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faaba190640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faab4159c00 0x7faab4070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45290/0 (socket says 172.21.15.26:45290) 2023-12-19T16:58:28.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faaba190640 1 -- 172.21.15.26:0/3231606848 learned_addr learned my addr 172.21.15.26:0/3231606848 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:28.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faab918e640 1 --2- 172.21.15.26:0/3231606848 >> v2:172.21.15.160:3300/0 conn(0x7faab406f6c0 0x7faab40c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faab998f640 1 --2- 172.21.15.26:0/3231606848 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faab4159820 0x7faab415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faab898d640 1 -- 172.21.15.26:0/3231606848 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224016287 0 0) 0x7faab406fb60 con 0x7faab4159c00 2023-12-19T16:58:28.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faab898d640 1 -- 172.21.15.26:0/3231606848 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa0003530 con 0x7faab4159c00 2023-12-19T16:58:28.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faab918e640 1 -- 172.21.15.26:0/3231606848 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faab4159820 msgr2=0x7faab415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:28.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faab918e640 1 --2- 172.21.15.26:0/3231606848 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faab4159820 0x7faab415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faab918e640 1 -- 172.21.15.26:0/3231606848 >> v1:172.21.15.26:6789/0 conn(0x7faab4159c00 legacy=0x7faab4070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faab898d640 1 -- 172.21.15.26:0/3231606848 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669092997 0 0) 0x7faaa0003530 con 0x7faab4159c00 2023-12-19T16:58:28.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faab918e640 1 -- 172.21.15.26:0/3231606848 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab4070120 con 0x7faab406f6c0 2023-12-19T16:58:28.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.152+0000 7faab998f640 1 --2- 172.21.15.26:0/3231606848 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faab4159820 0x7faab415df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:28.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faab918e640 1 --2- 172.21.15.26:0/3231606848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faab406f6c0 0x7faab40c8e80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faaa40097c0 tx=0x7faaa401c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53acc7961a14cfc9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:28.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faab898d640 1 -- 172.21.15.26:0/3231606848 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaac001a00 con 0x7faab406f6c0 2023-12-19T16:58:28.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faab898d640 1 -- 172.21.15.26:0/3231606848 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa40a7050 con 0x7faab406f6c0 2023-12-19T16:58:28.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faab898d640 1 -- 172.21.15.26:0/3231606848 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40ab620 con 0x7faab406f6c0 2023-12-19T16:58:28.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faabbbf3640 1 -- 172.21.15.26:0/3231606848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faab406f6c0 msgr2=0x7faab40c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faabbbf3640 1 --2- 172.21.15.26:0/3231606848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faab406f6c0 0x7faab40c8e80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7faaa40097c0 tx=0x7faaa401c370 comp rx=0 tx=0).stop 2023-12-19T16:58:28.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faabbbf3640 1 -- 172.21.15.26:0/3231606848 shutdown_connections 2023-12-19T16:58:28.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faabbbf3640 1 --2- 172.21.15.26:0/3231606848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faab406f6c0 0x7faab40c8e80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faabbbf3640 1 --2- 172.21.15.26:0/3231606848 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faab4159820 0x7faab415df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faabbbf3640 1 -- 172.21.15.26:0/3231606848 >> 172.21.15.26:0/3231606848 conn(0x7faab40ce730 msgr2=0x7faab40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:28.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faabbbf3640 1 -- 172.21.15.26:0/3231606848 shutdown_connections 2023-12-19T16:58:28.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faabbbf3640 1 -- 172.21.15.26:0/3231606848 wait complete. 2023-12-19T16:58:28.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faabbbf3640 1 Processor -- start 2023-12-19T16:58:28.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.156+0000 7faabbbf3640 1 -- start start 2023-12-19T16:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faabbbf3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faab406f6c0 0x7faab4152b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faabbbf3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faab4159820 0x7faab414f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faabbbf3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faab4159c00 0x7faab414fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faabbbf3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab4154bd0 con 0x7faab4159820 2023-12-19T16:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faabbbf3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab4154eb0 con 0x7faab406f6c0 2023-12-19T16:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faabbbf3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab4155190 con 0x7faab4159c00 2023-12-19T16:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faab918e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faab4159820 0x7faab414f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faab918e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faab4159820 0x7faab414f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46548/0 (socket says 172.21.15.26:46548) 2023-12-19T16:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faab918e640 1 -- 172.21.15.26:0/551121481 learned_addr learned my addr 172.21.15.26:0/551121481 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faab998f640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faab406f6c0 0x7faab4152b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faaba190640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faab4159c00 0x7faab414fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faab918e640 1 -- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faab4159c00 msgr2=0x7faab414fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faab918e640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faab4159c00 0x7faab414fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faab918e640 1 -- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faab406f6c0 msgr2=0x7faab4152b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faab918e640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faab406f6c0 0x7faab4152b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faab918e640 1 -- 172.21.15.26:0/551121481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab0009420 con 0x7faab4159820 2023-12-19T16:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faab918e640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faab4159820 0x7faab414f520 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7faaa401aa20 tx=0x7faaa4019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faaaa7fc640 1 -- 172.21.15.26:0/551121481 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40b4410 con 0x7faab4159820 2023-12-19T16:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faaaa7fc640 1 -- 172.21.15.26:0/551121481 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa40b4590 con 0x7faab4159820 2023-12-19T16:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faabbbf3640 1 -- 172.21.15.26:0/551121481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa4009420 con 0x7faab4159820 2023-12-19T16:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faaaa7fc640 1 -- 172.21.15.26:0/551121481 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40b4940 con 0x7faab4159820 2023-12-19T16:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faabbbf3640 1 -- 172.21.15.26:0/551121481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab406b880 con 0x7faab4159820 2023-12-19T16:58:28.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.160+0000 7faabbbf3640 1 -- 172.21.15.26:0/551121481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab4150120 con 0x7faab4159820 2023-12-19T16:58:28.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.164+0000 7faaaa7fc640 1 -- 172.21.15.26:0/551121481 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faaa40b2040 con 0x7faab4159820 2023-12-19T16:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.164+0000 7faaaa7fc640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa98065b80 0x7faa98068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.164+0000 7faaaa7fc640 1 -- 172.21.15.26:0/551121481 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6341+0+0 (secure 0 0 0) 0x7faaa411d730 con 0x7faab4159820 2023-12-19T16:58:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.164+0000 7faab998f640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa98065b80 0x7faa98068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.164+0000 7faab998f640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa98065b80 0x7faa98068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faab0009cf0 tx=0x7faab0009190 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:28.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.168+0000 7faaaa7fc640 1 -- 172.21.15.26:0/551121481 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faaa40ead70 con 0x7faab4159820 2023-12-19T16:58:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.360+0000 7faabbbf3640 1 -- 172.21.15.26:0/551121481 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7faab4150120 con 0x7faab4159820 2023-12-19T16:58:28.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.368+0000 7faaaa7fc640 1 -- 172.21.15.26:0/551121481 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v45) v1 ==== 174+0+0 (secure 0 0 0) 0x7faab4150120 con 0x7faab4159820 2023-12-19T16:58:28.371 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 12 and data pool 13 2023-12-19T16:58:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.372+0000 7faabbbf3640 1 -- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa98065b80 msgr2=0x7faa98068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.372+0000 7faabbbf3640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa98065b80 0x7faa98068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faab0009cf0 tx=0x7faab0009190 comp rx=0 tx=0).stop 2023-12-19T16:58:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.372+0000 7faabbbf3640 1 -- 172.21.15.26:0/551121481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faab4159820 msgr2=0x7faab414f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.372+0000 7faabbbf3640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faab4159820 0x7faab414f520 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7faaa401aa20 tx=0x7faaa4019110 comp rx=0 tx=0).stop 2023-12-19T16:58:28.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.376+0000 7faabbbf3640 1 -- 172.21.15.26:0/551121481 shutdown_connections 2023-12-19T16:58:28.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.376+0000 7faabbbf3640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa98065b80 0x7faa98068040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.376+0000 7faabbbf3640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faab4159c00 0x7faab414fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.376+0000 7faabbbf3640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faab4159820 0x7faab414f520 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.376+0000 7faabbbf3640 1 --2- 172.21.15.26:0/551121481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faab406f6c0 0x7faab4152b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.376+0000 7faabbbf3640 1 -- 172.21.15.26:0/551121481 >> 172.21.15.26:0/551121481 conn(0x7faab40ce730 msgr2=0x7faab40c9860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:28.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.376+0000 7faabbbf3640 1 -- 172.21.15.26:0/551121481 shutdown_connections 2023-12-19T16:58:28.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.376+0000 7faabbbf3640 1 -- 172.21.15.26:0/551121481 wait complete. 2023-12-19T16:58:28.417 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:58:28.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.556+0000 7fb84f939640 1 Processor -- start 2023-12-19T16:58:28.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.556+0000 7fb84f939640 1 -- start start 2023-12-19T16:58:28.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb84f939640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb84806bee0 0x7fb84806c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb84f939640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb8480b1d40 0x7fb8480b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb84f939640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb84806d4b0 con 0x7fb8480cb1d0 2023-12-19T16:58:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb84f939640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8480cc030 con 0x7fb8480b1d40 2023-12-19T16:58:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb84f939640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84806cc10 con 0x7fb84806bee0 2023-12-19T16:58:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb84d6d5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb8480cb1d0 0x7fb8480cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45302/0 (socket says 172.21.15.26:45302) 2023-12-19T16:58:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb84d6d5640 1 -- 172.21.15.26:0/1769744486 learned_addr learned my addr 172.21.15.26:0/1769744486 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb84ced4640 1 --2- 172.21.15.26:0/1769744486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb84806bee0 0x7fb84806c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb84ded6640 1 --2- 172.21.15.26:0/1769744486 >> v2:172.21.15.160:3300/0 conn(0x7fb8480b1d40 0x7fb8480b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb83ffff640 1 -- 172.21.15.26:0/1769744486 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249812551 0 0) 0x7fb84806d4b0 con 0x7fb8480cb1d0 2023-12-19T16:58:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb83ffff640 1 -- 172.21.15.26:0/1769744486 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb834003530 con 0x7fb8480cb1d0 2023-12-19T16:58:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb84ced4640 1 -- 172.21.15.26:0/1769744486 >> v2:172.21.15.160:3300/0 conn(0x7fb8480b1d40 msgr2=0x7fb8480b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb84ced4640 1 --2- 172.21.15.26:0/1769744486 >> v2:172.21.15.160:3300/0 conn(0x7fb8480b1d40 0x7fb8480b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb84ced4640 1 -- 172.21.15.26:0/1769744486 >> v1:172.21.15.26:6789/0 conn(0x7fb8480cb1d0 legacy=0x7fb8480cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.560+0000 7fb84ced4640 1 -- 172.21.15.26:0/1769744486 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84806cef0 con 0x7fb84806bee0 2023-12-19T16:58:28.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb84ced4640 1 --2- 172.21.15.26:0/1769744486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb84806bee0 0x7fb84806c340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb8380097c0 tx=0x7fb83801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=666007323b725bb5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:28.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb83ffff640 1 -- 172.21.15.26:0/1769744486 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8380ac620 con 0x7fb84806bee0 2023-12-19T16:58:28.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb83ffff640 1 -- 172.21.15.26:0/1769744486 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8380ac780 con 0x7fb84806bee0 2023-12-19T16:58:28.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb83ffff640 1 -- 172.21.15.26:0/1769744486 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8380acaf0 con 0x7fb84806bee0 2023-12-19T16:58:28.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb84f939640 1 -- 172.21.15.26:0/1769744486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb84806bee0 msgr2=0x7fb84806c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb84f939640 1 --2- 172.21.15.26:0/1769744486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb84806bee0 0x7fb84806c340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb8380097c0 tx=0x7fb83801c3c0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb84f939640 1 -- 172.21.15.26:0/1769744486 shutdown_connections 2023-12-19T16:58:28.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb84f939640 1 --2- 172.21.15.26:0/1769744486 >> v2:172.21.15.160:3300/0 conn(0x7fb8480b1d40 0x7fb8480b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb84f939640 1 --2- 172.21.15.26:0/1769744486 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb84806bee0 0x7fb84806c340 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb84f939640 1 -- 172.21.15.26:0/1769744486 >> 172.21.15.26:0/1769744486 conn(0x7fb8480658e0 msgr2=0x7fb848065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:28.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb84f939640 1 -- 172.21.15.26:0/1769744486 shutdown_connections 2023-12-19T16:58:28.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb84f939640 1 -- 172.21.15.26:0/1769744486 wait complete. 2023-12-19T16:58:28.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb84f939640 1 Processor -- start 2023-12-19T16:58:28.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.564+0000 7fb84f939640 1 -- start start 2023-12-19T16:58:28.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84f939640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb84806bee0 0x7fb8480cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84f939640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8480b1d40 0x7fb848159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84f939640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8480cb1d0 0x7fb84814f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84f939640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84815a280 con 0x7fb8480b1d40 2023-12-19T16:58:28.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84f939640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84815a560 con 0x7fb8480cb1d0 2023-12-19T16:58:28.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84f939640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84815a840 con 0x7fb84806bee0 2023-12-19T16:58:28.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84ced4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8480b1d40 0x7fb848159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84d6d5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb84806bee0 0x7fb8480cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84ced4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8480b1d40 0x7fb848159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46552/0 (socket says 172.21.15.26:46552) 2023-12-19T16:58:28.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84ced4640 1 -- 172.21.15.26:0/867512562 learned_addr learned my addr 172.21.15.26:0/867512562 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:28.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84ded6640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8480cb1d0 0x7fb84814f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84ced4640 1 -- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb84806bee0 msgr2=0x7fb8480cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84ced4640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb84806bee0 0x7fb8480cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84ced4640 1 -- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8480cb1d0 msgr2=0x7fb84814f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84ced4640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8480cb1d0 0x7fb84814f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84ced4640 1 -- 172.21.15.26:0/867512562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb838009420 con 0x7fb8480b1d40 2023-12-19T16:58:28.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84d6d5640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb84806bee0 0x7fb8480cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:28.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84ced4640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8480b1d40 0x7fb848159a50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb83801aa70 tx=0x7fb838019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:28.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb83dffb640 1 -- 172.21.15.26:0/867512562 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8380bb070 con 0x7fb8480b1d40 2023-12-19T16:58:28.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb83dffb640 1 -- 172.21.15.26:0/867512562 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8380ac420 con 0x7fb8480b1d40 2023-12-19T16:58:28.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84f939640 1 -- 172.21.15.26:0/867512562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84815ab20 con 0x7fb8480b1d40 2023-12-19T16:58:28.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb83dffb640 1 -- 172.21.15.26:0/867512562 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8380b5600 con 0x7fb8480b1d40 2023-12-19T16:58:28.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84f939640 1 -- 172.21.15.26:0/867512562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84814fda0 con 0x7fb8480b1d40 2023-12-19T16:58:28.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.568+0000 7fb84f939640 1 -- 172.21.15.26:0/867512562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb848074270 con 0x7fb8480b1d40 2023-12-19T16:58:28.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.572+0000 7fb83dffb640 1 -- 172.21.15.26:0/867512562 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb8380b3040 con 0x7fb8480b1d40 2023-12-19T16:58:28.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.572+0000 7fb83dffb640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb82c065b80 0x7fb82c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.572+0000 7fb83dffb640 1 -- 172.21.15.26:0/867512562 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fb83811cf00 con 0x7fb8480b1d40 2023-12-19T16:58:28.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.572+0000 7fb84d6d5640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb82c065b80 0x7fb82c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.572+0000 7fb84d6d5640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb82c065b80 0x7fb82c068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb844005890 tx=0x7fb844009040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:28.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.576+0000 7fb83dffb640 1 -- 172.21.15.26:0/867512562 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb8380ea4f0 con 0x7fb8480b1d40 2023-12-19T16:58:28.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.736+0000 7fb84f939640 1 -- 172.21.15.26:0/867512562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8480ae040 con 0x7fb8480b1d40 2023-12-19T16:58:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.740+0000 7fb83dffb640 1 -- 172.21.15.26:0/867512562 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17221 (secure 0 0 0) 0x7fb8380edb10 con 0x7fb8480b1d40 2023-12-19T16:58:28.740 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:28.740 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":106,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:58:28.363998+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:58:26.906185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T16:58:27.358583+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:58:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.740+0000 7fb84f939640 1 -- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb82c065b80 msgr2=0x7fb82c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.740+0000 7fb84f939640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb82c065b80 0x7fb82c068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb844005890 tx=0x7fb844009040 comp rx=0 tx=0).stop 2023-12-19T16:58:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.740+0000 7fb84f939640 1 -- 172.21.15.26:0/867512562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8480b1d40 msgr2=0x7fb848159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.740+0000 7fb84f939640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8480b1d40 0x7fb848159a50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fb83801aa70 tx=0x7fb838019160 comp rx=0 tx=0).stop 2023-12-19T16:58:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.740+0000 7fb84f939640 1 -- 172.21.15.26:0/867512562 shutdown_connections 2023-12-19T16:58:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.740+0000 7fb84f939640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8480cb1d0 0x7fb84814f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.740+0000 7fb84f939640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8480b1d40 0x7fb848159a50 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.740+0000 7fb84f939640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb82c065b80 0x7fb82c068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.740+0000 7fb84f939640 1 --2- 172.21.15.26:0/867512562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb84806bee0 0x7fb8480cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.740+0000 7fb84f939640 1 -- 172.21.15.26:0/867512562 >> 172.21.15.26:0/867512562 conn(0x7fb8480658e0 msgr2=0x7fb848065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:28.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.740+0000 7fb84f939640 1 -- 172.21.15.26:0/867512562 shutdown_connections 2023-12-19T16:58:28.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.740+0000 7fb84f939640 1 -- 172.21.15.26:0/867512562 wait complete. 2023-12-19T16:58:28.762 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:58:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.904+0000 7f4996796640 1 Processor -- start 2023-12-19T16:58:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.904+0000 7f4996796640 1 -- start start 2023-12-19T16:58:28.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.904+0000 7f4996796640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f499015a7c0 0x7f499015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.904+0000 7f4996796640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f499006f9d0 0x7f49900ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.904+0000 7f4996796640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49900c4070 0x7f49900cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.904+0000 7f4996796640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499015e2f0 con 0x7f499015a7c0 2023-12-19T16:58:28.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.904+0000 7f4996796640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49900c5b40 con 0x7f499006f9d0 2023-12-19T16:58:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.904+0000 7f4996796640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49900c5e20 con 0x7f49900c4070 2023-12-19T16:58:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f499015a7c0 0x7f499015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f499015a7c0 0x7f499015ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46560/0 (socket says 172.21.15.26:46560) 2023-12-19T16:58:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498ffff640 1 -- 172.21.15.26:0/2561128929 learned_addr learned my addr 172.21.15.26:0/2561128929 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498f7fe640 1 --2- 172.21.15.26:0/2561128929 >> v2:172.21.15.160:3300/0 conn(0x7f499006f9d0 0x7f49900ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f4994d33640 1 --2- 172.21.15.26:0/2561128929 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49900c4070 0x7f49900cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498ffff640 1 -- 172.21.15.26:0/2561128929 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49900c4070 msgr2=0x7f49900cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498ffff640 1 --2- 172.21.15.26:0/2561128929 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49900c4070 0x7f49900cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498ffff640 1 -- 172.21.15.26:0/2561128929 >> v2:172.21.15.160:3300/0 conn(0x7f499006f9d0 msgr2=0x7f49900ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498ffff640 1 --2- 172.21.15.26:0/2561128929 >> v2:172.21.15.160:3300/0 conn(0x7f499006f9d0 0x7f49900ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498ffff640 1 -- 172.21.15.26:0/2561128929 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900ce770 con 0x7f499015a7c0 2023-12-19T16:58:28.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498f7fe640 1 --2- 172.21.15.26:0/2561128929 >> v2:172.21.15.160:3300/0 conn(0x7f499006f9d0 0x7f49900ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498ffff640 1 --2- 172.21.15.26:0/2561128929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499015a7c0 0x7f499015ddb0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f499006ab70 tx=0x7f498001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4f4492490292a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498effd640 1 -- 172.21.15.26:0/2561128929 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49800aa5f0 con 0x7f499015a7c0 2023-12-19T16:58:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498effd640 1 -- 172.21.15.26:0/2561128929 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49800aa750 con 0x7f499015a7c0 2023-12-19T16:58:28.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.908+0000 7f498effd640 1 -- 172.21.15.26:0/2561128929 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49800aaae0 con 0x7f499015a7c0 2023-12-19T16:58:28.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 -- 172.21.15.26:0/2561128929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499015a7c0 msgr2=0x7f499015ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 --2- 172.21.15.26:0/2561128929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499015a7c0 0x7f499015ddb0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f499006ab70 tx=0x7f498001c4d0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 -- 172.21.15.26:0/2561128929 shutdown_connections 2023-12-19T16:58:28.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 --2- 172.21.15.26:0/2561128929 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49900c4070 0x7f49900cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 --2- 172.21.15.26:0/2561128929 >> v2:172.21.15.160:3300/0 conn(0x7f499006f9d0 0x7f49900ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 --2- 172.21.15.26:0/2561128929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499015a7c0 0x7f499015ddb0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 -- 172.21.15.26:0/2561128929 >> 172.21.15.26:0/2561128929 conn(0x7f499006d5d0 msgr2=0x7f4990069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:28.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 -- 172.21.15.26:0/2561128929 shutdown_connections 2023-12-19T16:58:28.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 -- 172.21.15.26:0/2561128929 wait complete. 2023-12-19T16:58:28.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 Processor -- start 2023-12-19T16:58:28.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 -- start start 2023-12-19T16:58:28.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f499006f9d0 0x7f4990152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49900c4070 0x7f49900ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499015a7c0 0x7f499015f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49900cbd90 con 0x7f499015a7c0 2023-12-19T16:58:28.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49900cc1b0 con 0x7f49900c4070 2023-12-19T16:58:28.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4996796640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49900cb110 con 0x7f499006f9d0 2023-12-19T16:58:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4994d33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499015a7c0 0x7f499015f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4994d33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499015a7c0 0x7f499015f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46574/0 (socket says 172.21.15.26:46574) 2023-12-19T16:58:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4994d33640 1 -- 172.21.15.26:0/3353115275 learned_addr learned my addr 172.21.15.26:0/3353115275 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f498ffff640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f499006f9d0 0x7f4990152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f498f7fe640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49900c4070 0x7f49900ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4994d33640 1 -- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f499006f9d0 msgr2=0x7f4990152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4994d33640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f499006f9d0 0x7f4990152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4994d33640 1 -- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49900c4070 msgr2=0x7f49900ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4994d33640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49900c4070 0x7f49900ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.912+0000 7f4994d33640 1 -- 172.21.15.26:0/3353115275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4980009480 con 0x7f499015a7c0 2023-12-19T16:58:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.916+0000 7f4994d33640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499015a7c0 0x7f499015f8e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f4988007e40 tx=0x7f498800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.916+0000 7f498cff9640 1 -- 172.21.15.26:0/3353115275 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988014620 con 0x7f499015a7c0 2023-12-19T16:58:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.916+0000 7f498cff9640 1 -- 172.21.15.26:0/3353115275 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498800ee90 con 0x7f499015a7c0 2023-12-19T16:58:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.916+0000 7f4996796640 1 -- 172.21.15.26:0/3353115275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49900cb3f0 con 0x7f499015a7c0 2023-12-19T16:58:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.916+0000 7f498cff9640 1 -- 172.21.15.26:0/3353115275 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49880149e0 con 0x7f499015a7c0 2023-12-19T16:58:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.916+0000 7f4996796640 1 -- 172.21.15.26:0/3353115275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49900cb9b0 con 0x7f499015a7c0 2023-12-19T16:58:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.916+0000 7f4996796640 1 -- 172.21.15.26:0/3353115275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49900cbed0 con 0x7f499015a7c0 2023-12-19T16:58:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.916+0000 7f498cff9640 1 -- 172.21.15.26:0/3353115275 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4988014b40 con 0x7f499015a7c0 2023-12-19T16:58:28.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.916+0000 7f498cff9640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f497c065b80 0x7f497c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:28.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.916+0000 7f498cff9640 1 -- 172.21.15.26:0/3353115275 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f4988083db0 con 0x7f499015a7c0 2023-12-19T16:58:28.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.916+0000 7f498ffff640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f497c065b80 0x7f497c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:28.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.920+0000 7f498ffff640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f497c065b80 0x7f497c068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f49800ae000 tx=0x7f49800a7000 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:28.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:28.920+0000 7f498cff9640 1 -- 172.21.15.26:0/3353115275 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4988088070 con 0x7f499015a7c0 2023-12-19T16:58:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.084+0000 7f4996796640 1 -- 172.21.15.26:0/3353115275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4990074270 con 0x7f499015a7c0 2023-12-19T16:58:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.084+0000 7f498cff9640 1 -- 172.21.15.26:0/3353115275 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17221 (secure 0 0 0) 0x7f4988051420 con 0x7f499015a7c0 2023-12-19T16:58:29.086 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:29.086 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":106,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:58:28.363998+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:58:26.906185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T16:58:27.358583+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:58:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.088+0000 7f4996796640 1 -- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f497c065b80 msgr2=0x7f497c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.088+0000 7f4996796640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f497c065b80 0x7f497c068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f49800ae000 tx=0x7f49800a7000 comp rx=0 tx=0).stop 2023-12-19T16:58:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.088+0000 7f4996796640 1 -- 172.21.15.26:0/3353115275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499015a7c0 msgr2=0x7f499015f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.088+0000 7f4996796640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499015a7c0 0x7f499015f8e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f4988007e40 tx=0x7f498800e4b0 comp rx=0 tx=0).stop 2023-12-19T16:58:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.088+0000 7f4996796640 1 -- 172.21.15.26:0/3353115275 shutdown_connections 2023-12-19T16:58:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.088+0000 7f4996796640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f499015a7c0 0x7f499015f8e0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.088+0000 7f4996796640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49900c4070 0x7f49900ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.088+0000 7f4996796640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f497c065b80 0x7f497c068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.088+0000 7f4996796640 1 --2- 172.21.15.26:0/3353115275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f499006f9d0 0x7f4990152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.088+0000 7f4996796640 1 -- 172.21.15.26:0/3353115275 >> 172.21.15.26:0/3353115275 conn(0x7f499006d5d0 msgr2=0x7f49900b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.088+0000 7f4996796640 1 -- 172.21.15.26:0/3353115275 shutdown_connections 2023-12-19T16:58:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.088+0000 7f4996796640 1 -- 172.21.15.26:0/3353115275 wait complete. 2023-12-19T16:58:29.108 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T16:58:29.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.252+0000 7f846cc66640 1 Processor -- start 2023-12-19T16:58:29.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.252+0000 7f846cc66640 1 -- start start 2023-12-19T16:58:29.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.252+0000 7f846cc66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f846806f6c0 0x7f846815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:29.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.252+0000 7f846cc66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846806fb60 con 0x7f8468159c00 2023-12-19T16:58:29.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f846cc66640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f846806fe40 con 0x7f8468159820 2023-12-19T16:58:29.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f846cc66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846815e470 con 0x7f846806f6c0 2023-12-19T16:58:29.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8466d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8468159c00 0x7f8468070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45308/0 (socket says 172.21.15.26:45308) 2023-12-19T16:58:29.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8466d9d640 1 -- 172.21.15.26:0/2367805080 learned_addr learned my addr 172.21.15.26:0/2367805080 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f845ffff640 1 --2- 172.21.15.26:0/2367805080 >> v2:172.21.15.160:3300/0 conn(0x7f846806f6c0 0x7f846815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725985537 0 0) 0x7f846806fb60 con 0x7f8468159c00 2023-12-19T16:58:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8454003530 con 0x7f8468159c00 2023-12-19T16:58:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1365981694 0 0) 0x7f8454003530 con 0x7f8468159c00 2023-12-19T16:58:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 >> v2:172.21.15.160:3300/0 conn(0x7f846806f6c0 msgr2=0x7f846815df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 --2- 172.21.15.26:0/2367805080 >> v2:172.21.15.160:3300/0 conn(0x7f846806f6c0 0x7f846815df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 >> v1:172.21.15.160:6789/0 conn(0x7f8468159820 legacy=0x7f84680c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8468070120 con 0x7f8468159c00 2023-12-19T16:58:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f8460001a00 con 0x7f8468159c00 2023-12-19T16:58:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 >> v1:172.21.15.26:6789/0 conn(0x7f8468159c00 legacy=0x7f8468070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 --2- 172.21.15.26:0/2367805080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8454003980 0x7f8454003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:29.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 --2- 172.21.15.26:0/2367805080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84540047d0 0x7f8454004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 --2- 172.21.15.26:0/2367805080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8454005190 0x7f845400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846815e470 con 0x7f84540047d0 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845400c030 con 0x7f8454005190 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845400c1f0 con 0x7f8454003980 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f845ffff640 1 --2- 172.21.15.26:0/2367805080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84540047d0 0x7f8454004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f846659c640 1 --2- 172.21.15.26:0/2367805080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8454003980 0x7f8454003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8466d9d640 1 --2- 172.21.15.26:0/2367805080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8454005190 0x7f845400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f845ffff640 1 -- 172.21.15.26:0/2367805080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8454003980 msgr2=0x7f8454003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f845ffff640 1 --2- 172.21.15.26:0/2367805080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8454003980 0x7f8454003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f845ffff640 1 -- 172.21.15.26:0/2367805080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8454005190 msgr2=0x7f845400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f845ffff640 1 --2- 172.21.15.26:0/2367805080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8454005190 0x7f845400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f845ffff640 1 -- 172.21.15.26:0/2367805080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8454003530 con 0x7f84540047d0 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f846659c640 1 --2- 172.21.15.26:0/2367805080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8454003980 0x7f8454003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:29.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8466d9d640 1 --2- 172.21.15.26:0/2367805080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8454005190 0x7f845400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:29.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.256+0000 7f8466d9d640 1 -- 172.21.15.26:0/2367805080 reap_dead start 2023-12-19T16:58:29.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.260+0000 7f845ffff640 1 --2- 172.21.15.26:0/2367805080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84540047d0 0x7f8454004c50 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f8450009670 tx=0x7f845001ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bd5155c759425c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:29.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.260+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84500a7840 con 0x7f84540047d0 2023-12-19T16:58:29.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.260+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84500a79a0 con 0x7f84540047d0 2023-12-19T16:58:29.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.260+0000 7f8465d9b640 1 -- 172.21.15.26:0/2367805080 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84500a7d50 con 0x7f84540047d0 2023-12-19T16:58:29.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.260+0000 7f846cc66640 1 -- 172.21.15.26:0/2367805080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84540047d0 msgr2=0x7f8454004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:29.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.260+0000 7f846cc66640 1 --2- 172.21.15.26:0/2367805080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84540047d0 0x7f8454004c50 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f8450009670 tx=0x7f845001ec40 comp rx=0 tx=0).stop 2023-12-19T16:58:29.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.260+0000 7f846cc66640 1 -- 172.21.15.26:0/2367805080 shutdown_connections 2023-12-19T16:58:29.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.260+0000 7f846cc66640 1 --2- 172.21.15.26:0/2367805080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84540047d0 0x7f8454004c50 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:29.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.260+0000 7f846cc66640 1 -- 172.21.15.26:0/2367805080 >> 172.21.15.26:0/2367805080 conn(0x7f84680ce730 msgr2=0x7f84680ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:29.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.260+0000 7f846cc66640 1 -- 172.21.15.26:0/2367805080 shutdown_connections 2023-12-19T16:58:29.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.260+0000 7f846cc66640 1 -- 172.21.15.26:0/2367805080 wait complete. 2023-12-19T16:58:29.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.260+0000 7f846cc66640 1 Processor -- start 2023-12-19T16:58:29.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.260+0000 7f846cc66640 1 -- start start 2023-12-19T16:58:29.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f846cc66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84540047d0 0x7f84680ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:29.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f846cc66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8468155dc0 0x7f84681f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:29.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f846cc66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84680b06e0 0x7f84680b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:29.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f846cc66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8468156240 con 0x7f8468155dc0 2023-12-19T16:58:29.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f846cc66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84680b0250 con 0x7f84540047d0 2023-12-19T16:58:29.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f846cc66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84680af080 con 0x7f84680b06e0 2023-12-19T16:58:29.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f845ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8468155dc0 0x7f84681f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:29.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f845ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8468155dc0 0x7f84681f6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46592/0 (socket says 172.21.15.26:46592) 2023-12-19T16:58:29.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f845ffff640 1 -- 172.21.15.26:0/2221263871 learned_addr learned my addr 172.21.15.26:0/2221263871 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:29.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f846659c640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84540047d0 0x7f84680ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:29.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f8466d9d640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84680b06e0 0x7f84680b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:29.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f845ffff640 1 -- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84680b06e0 msgr2=0x7f84680b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:29.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f845ffff640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84680b06e0 0x7f84680b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:29.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f845ffff640 1 -- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84540047d0 msgr2=0x7f84680ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:29.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f845ffff640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84540047d0 0x7f84680ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:29.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f845ffff640 1 -- 172.21.15.26:0/2221263871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84680ae7e0 con 0x7f8468155dc0 2023-12-19T16:58:29.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f8466d9d640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84680b06e0 0x7f84680b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f845ffff640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8468155dc0 0x7f84681f6300 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f845009a660 tx=0x7f845001b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f845effd640 1 -- 172.21.15.26:0/2221263871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450004230 con 0x7f8468155dc0 2023-12-19T16:58:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f845effd640 1 -- 172.21.15.26:0/2221263871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8450004390 con 0x7f8468155dc0 2023-12-19T16:58:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f846cc66640 1 -- 172.21.15.26:0/2221263871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84680aeac0 con 0x7f8468155dc0 2023-12-19T16:58:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f845effd640 1 -- 172.21.15.26:0/2221263871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84500af970 con 0x7f8468155dc0 2023-12-19T16:58:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f846cc66640 1 -- 172.21.15.26:0/2221263871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84680c21d0 con 0x7f8468155dc0 2023-12-19T16:58:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f846cc66640 1 -- 172.21.15.26:0/2221263871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8468074270 con 0x7f8468155dc0 2023-12-19T16:58:29.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.264+0000 7f845effd640 1 -- 172.21.15.26:0/2221263871 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f84500afad0 con 0x7f8468155dc0 2023-12-19T16:58:29.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.268+0000 7f845effd640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8448065bb0 0x7f8448068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:29.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.268+0000 7f845effd640 1 -- 172.21.15.26:0/2221263871 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f845011d520 con 0x7f8468155dc0 2023-12-19T16:58:29.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.268+0000 7f846659c640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8448065bb0 0x7f8448068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:29.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.268+0000 7f846659c640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8448065bb0 0x7f8448068070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8458007fd0 tx=0x7f8458009040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:29.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.272+0000 7f845effd640 1 -- 172.21.15.26:0/2221263871 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f84500eab90 con 0x7f8468155dc0 2023-12-19T16:58:29.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:29.464+0000 7f846cc66640 1 -- 172.21.15.26:0/2221263871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f84680b46b0 con 0x7f8468155dc0 2023-12-19T16:58:29.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:58:29.628 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:58:29.880 INFO:tasks.ceph.mds.c.smithi026.stderr:2023-12-19T16:58:29.880+0000 7f606171c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T16:58:30.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.384+0000 7f845effd640 1 -- 172.21.15.26:0/2221263871 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) v1 ==== 118+0+0 (secure 0 0 0) 0x7f84500ee1b0 con 0x7f8468155dc0 2023-12-19T16:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.388+0000 7f846cc66640 1 -- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8448065bb0 msgr2=0x7f8448068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.388+0000 7f846cc66640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8448065bb0 0x7f8448068070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8458007fd0 tx=0x7f8458009040 comp rx=0 tx=0).stop 2023-12-19T16:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.388+0000 7f846cc66640 1 -- 172.21.15.26:0/2221263871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8468155dc0 msgr2=0x7f84681f6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.388+0000 7f846cc66640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8468155dc0 0x7f84681f6300 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f845009a660 tx=0x7f845001b9e0 comp rx=0 tx=0).stop 2023-12-19T16:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.388+0000 7f846cc66640 1 -- 172.21.15.26:0/2221263871 shutdown_connections 2023-12-19T16:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.388+0000 7f846cc66640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8448065bb0 0x7f8448068070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.388+0000 7f846cc66640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84680b06e0 0x7f84680b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.388+0000 7f846cc66640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8468155dc0 0x7f84681f6300 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.388+0000 7f846cc66640 1 --2- 172.21.15.26:0/2221263871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84540047d0 0x7f84680ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.388+0000 7f846cc66640 1 -- 172.21.15.26:0/2221263871 >> 172.21.15.26:0/2221263871 conn(0x7f84680ce730 msgr2=0x7f8468157830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.388+0000 7f846cc66640 1 -- 172.21.15.26:0/2221263871 shutdown_connections 2023-12-19T16:58:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.388+0000 7f846cc66640 1 -- 172.21.15.26:0/2221263871 wait complete. 2023-12-19T16:58:30.414 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T16:58:30.414 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T16:58:30.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f504132c640 1 Processor -- start 2023-12-19T16:58:30.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f504132c640 1 -- start start 2023-12-19T16:58:30.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f504132c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0cb260 0x7f503c0ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:30.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f504132c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c0ae260 con 0x7f503c145ed0 2023-12-19T16:58:30.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f504132c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f503c0adf80 con 0x7f503c0cd2d0 2023-12-19T16:58:30.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f504132c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503c0c9770 con 0x7f503c0cb260 2023-12-19T16:58:30.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f503c145ed0 0x7f503c0ccbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45312/0 (socket says 172.21.15.26:45312) 2023-12-19T16:58:30.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503affd640 1 -- 172.21.15.26:0/3759208359 learned_addr learned my addr 172.21.15.26:0/3759208359 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:30.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503b7fe640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0cb260 0x7f503c0ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:30.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183626495 0 0) 0x7f503c0ae260 con 0x7f503c145ed0 2023-12-19T16:58:30.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5028003530 con 0x7f503c145ed0 2023-12-19T16:58:30.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1345152793 0 0) 0x7f5028003530 con 0x7f503c145ed0 2023-12-19T16:58:30.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0cb260 msgr2=0x7f503c0ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:30.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0cb260 0x7f503c0ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:30.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 >> v1:172.21.15.160:6789/0 conn(0x7f503c0cd2d0 legacy=0x7f503c0ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:30.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c0c9a50 con 0x7f503c145ed0 2023-12-19T16:58:30.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f502c001680 con 0x7f503c145ed0 2023-12-19T16:58:30.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 >> v1:172.21.15.26:6789/0 conn(0x7f503c145ed0 legacy=0x7f503c0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:30.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50280039b0 0x7f5028003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:30.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0cb260 0x7f50280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:30.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5028004d10 0x7f5028005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:30.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c0c9770 con 0x7f50280039b0 2023-12-19T16:58:30.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502800bb20 con 0x7f5028004d10 2023-12-19T16:58:30.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502800bcb0 con 0x7f503c0cb260 2023-12-19T16:58:30.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503affd640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50280039b0 0x7f5028003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:30.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f5033fff640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0cb260 0x7f50280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:30.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.564+0000 7f503b7fe640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5028004d10 0x7f5028005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:30.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f503affd640 1 -- 172.21.15.26:0/3759208359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0cb260 msgr2=0x7f50280047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:30.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f503affd640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0cb260 0x7f50280047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:30.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f503affd640 1 -- 172.21.15.26:0/3759208359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5028004d10 msgr2=0x7f5028005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:30.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f503affd640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5028004d10 0x7f5028005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:30.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f503affd640 1 -- 172.21.15.26:0/3759208359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5028003530 con 0x7f50280039b0 2023-12-19T16:58:30.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f5033fff640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0cb260 0x7f50280047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:30.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f503b7fe640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5028004d10 0x7f5028005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:30.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f503affd640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50280039b0 0x7f5028003d90 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f502c00c240 tx=0x7f502c01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff365cbb172e989f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:30.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c005ae0 con 0x7f50280039b0 2023-12-19T16:58:30.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c005c60 con 0x7f50280039b0 2023-12-19T16:58:30.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f503a7fc640 1 -- 172.21.15.26:0/3759208359 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c0a5800 con 0x7f50280039b0 2023-12-19T16:58:30.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f504132c640 1 -- 172.21.15.26:0/3759208359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50280039b0 msgr2=0x7f5028003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:30.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f504132c640 1 --2- 172.21.15.26:0/3759208359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50280039b0 0x7f5028003d90 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f502c00c240 tx=0x7f502c01cc10 comp rx=0 tx=0).stop 2023-12-19T16:58:30.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f503b7fe640 1 -- 172.21.15.26:0/3759208359 reap_dead start 2023-12-19T16:58:30.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f504132c640 1 -- 172.21.15.26:0/3759208359 shutdown_connections 2023-12-19T16:58:30.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f504132c640 1 -- 172.21.15.26:0/3759208359 >> 172.21.15.26:0/3759208359 conn(0x7f503c0b0b60 msgr2=0x7f503c0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:30.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f504132c640 1 -- 172.21.15.26:0/3759208359 shutdown_connections 2023-12-19T16:58:30.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.568+0000 7f504132c640 1 -- 172.21.15.26:0/3759208359 wait complete. 2023-12-19T16:58:30.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f504132c640 1 Processor -- start 2023-12-19T16:58:30.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f504132c640 1 -- start start 2023-12-19T16:58:30.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f504132c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50280039b0 0x7f503c15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:30.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f504132c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0c4070 0x7f503c0c3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:30.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f504132c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503c0c7590 0x7f503c1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:30.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f504132c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c159d70 con 0x7f503c0c7590 2023-12-19T16:58:30.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f504132c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503c157100 con 0x7f50280039b0 2023-12-19T16:58:30.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f504132c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503c1597b0 con 0x7f503c0c4070 2023-12-19T16:58:30.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f503b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503c0c7590 0x7f503c1573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:30.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f503b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503c0c7590 0x7f503c1573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46596/0 (socket says 172.21.15.26:46596) 2023-12-19T16:58:30.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f503b7fe640 1 -- 172.21.15.26:0/1646744597 learned_addr learned my addr 172.21.15.26:0/1646744597 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:30.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f5033fff640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0c4070 0x7f503c0c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:30.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f503affd640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50280039b0 0x7f503c15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:30.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f503b7fe640 1 -- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0c4070 msgr2=0x7f503c0c3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:30.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f503b7fe640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0c4070 0x7f503c0c3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:30.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f503b7fe640 1 -- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50280039b0 msgr2=0x7f503c15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:30.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f503b7fe640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50280039b0 0x7f503c15a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:30.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f503b7fe640 1 -- 172.21.15.26:0/1646744597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f503c159a90 con 0x7f503c0c7590 2023-12-19T16:58:30.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f5033fff640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0c4070 0x7f503c0c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:30.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f503affd640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50280039b0 0x7f503c15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:30.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f503b7fe640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503c0c7590 0x7f503c1573a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f503c0cca80 tx=0x7f5034010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:30.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f50337fe640 1 -- 172.21.15.26:0/1646744597 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503401b070 con 0x7f503c0c7590 2023-12-19T16:58:30.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f50337fe640 1 -- 172.21.15.26:0/1646744597 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503400f040 con 0x7f503c0c7590 2023-12-19T16:58:30.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f50337fe640 1 -- 172.21.15.26:0/1646744597 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503400e050 con 0x7f503c0c7590 2023-12-19T16:58:30.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f504132c640 1 -- 172.21.15.26:0/1646744597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503c154ff0 con 0x7f503c0c7590 2023-12-19T16:58:30.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.572+0000 7f504132c640 1 -- 172.21.15.26:0/1646744597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503c156d40 con 0x7f503c0c7590 2023-12-19T16:58:30.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.576+0000 7f504132c640 1 -- 172.21.15.26:0/1646744597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503c159ee0 con 0x7f503c0c7590 2023-12-19T16:58:30.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.576+0000 7f50337fe640 1 -- 172.21.15.26:0/1646744597 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5034015440 con 0x7f503c0c7590 2023-12-19T16:58:30.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.576+0000 7f50337fe640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f501c065b80 0x7f501c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:30.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.576+0000 7f50337fe640 1 -- 172.21.15.26:0/1646744597 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f5034083a40 con 0x7f503c0c7590 2023-12-19T16:58:30.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.576+0000 7f503affd640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f501c065b80 0x7f501c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:30.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.580+0000 7f503affd640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f501c065b80 0x7f501c068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f502c0ab000 tx=0x7f502c0a7070 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:30.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.580+0000 7f50337fe640 1 -- 172.21.15.26:0/1646744597 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5034050ef0 con 0x7f503c0c7590 2023-12-19T16:58:30.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:30.776+0000 7f504132c640 1 -- 172.21.15.26:0/1646744597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f503c1551e0 con 0x7f503c0c7590 2023-12-19T16:58:31.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.392+0000 7f50337fe640 1 -- 172.21.15.26:0/1646744597 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5034054510 con 0x7f503c0c7590 2023-12-19T16:58:31.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.396+0000 7f504132c640 1 -- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f501c065b80 msgr2=0x7f501c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:31.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.396+0000 7f504132c640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f501c065b80 0x7f501c068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f502c0ab000 tx=0x7f502c0a7070 comp rx=0 tx=0).stop 2023-12-19T16:58:31.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.396+0000 7f504132c640 1 -- 172.21.15.26:0/1646744597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503c0c7590 msgr2=0x7f503c1573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:31.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.396+0000 7f504132c640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503c0c7590 0x7f503c1573a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f503c0cca80 tx=0x7f5034010590 comp rx=0 tx=0).stop 2023-12-19T16:58:31.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.396+0000 7f504132c640 1 -- 172.21.15.26:0/1646744597 shutdown_connections 2023-12-19T16:58:31.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.396+0000 7f504132c640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503c0c7590 0x7f503c1573a0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.396+0000 7f504132c640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f501c065b80 0x7f501c068040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.396+0000 7f504132c640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503c0c4070 0x7f503c0c3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.396+0000 7f504132c640 1 --2- 172.21.15.26:0/1646744597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50280039b0 0x7f503c15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.396+0000 7f504132c640 1 -- 172.21.15.26:0/1646744597 >> 172.21.15.26:0/1646744597 conn(0x7f503c0b0b60 msgr2=0x7f503c0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:31.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.396+0000 7f504132c640 1 -- 172.21.15.26:0/1646744597 shutdown_connections 2023-12-19T16:58:31.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.396+0000 7f504132c640 1 -- 172.21.15.26:0/1646744597 wait complete. 2023-12-19T16:58:31.418 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T16:58:31.418 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:58:31.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcae2fce640 1 Processor -- start 2023-12-19T16:58:31.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcae2fce640 1 -- start start 2023-12-19T16:58:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcae2fce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcadc15cda0 0x7fcadc160920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcae2fce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcadc0b85e0 0x7fcadc0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcae2fce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcadc0cde60 0x7fcadc0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcae2fce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcadc0ce2f0 con 0x7fcadc0b85e0 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcae2fce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcadc0d0d40 con 0x7fcadc0cde60 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcae2fce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc161150 con 0x7fcadc15cda0 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcadbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcadc0b85e0 0x7fcadc0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcadbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcadc0b85e0 0x7fcadc0cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46608/0 (socket says 172.21.15.26:46608) 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcadbfff640 1 -- 172.21.15.26:0/2039403405 learned_addr learned my addr 172.21.15.26:0/2039403405 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcae0d6a640 1 --2- 172.21.15.26:0/2039403405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcadc15cda0 0x7fcadc160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcae156b640 1 --2- 172.21.15.26:0/2039403405 >> v2:172.21.15.160:3300/0 conn(0x7fcadc0cde60 0x7fcadc0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcadbfff640 1 -- 172.21.15.26:0/2039403405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcadc15cda0 msgr2=0x7fcadc160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcadbfff640 1 --2- 172.21.15.26:0/2039403405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcadc15cda0 0x7fcadc160920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcadbfff640 1 -- 172.21.15.26:0/2039403405 >> v2:172.21.15.160:3300/0 conn(0x7fcadc0cde60 msgr2=0x7fcadc0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:31.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcadbfff640 1 --2- 172.21.15.26:0/2039403405 >> v2:172.21.15.160:3300/0 conn(0x7fcadc0cde60 0x7fcadc0b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.564+0000 7fcadbfff640 1 -- 172.21.15.26:0/2039403405 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc161430 con 0x7fcadc0b85e0 2023-12-19T16:58:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.568+0000 7fcadbfff640 1 --2- 172.21.15.26:0/2039403405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcadc0b85e0 0x7fcadc0cd920 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fcadc08c070 tx=0x7fcac801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d73938267ede5a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.568+0000 7fcadb7fe640 1 -- 172.21.15.26:0/2039403405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac80ab600 con 0x7fcadc0b85e0 2023-12-19T16:58:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.568+0000 7fcadb7fe640 1 -- 172.21.15.26:0/2039403405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac80a7050 con 0x7fcadc0b85e0 2023-12-19T16:58:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.568+0000 7fcadb7fe640 1 -- 172.21.15.26:0/2039403405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac80ab980 con 0x7fcadc0b85e0 2023-12-19T16:58:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.568+0000 7fcae2fce640 1 -- 172.21.15.26:0/2039403405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcadc0b85e0 msgr2=0x7fcadc0cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:31.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.568+0000 7fcae2fce640 1 --2- 172.21.15.26:0/2039403405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcadc0b85e0 0x7fcadc0cd920 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fcadc08c070 tx=0x7fcac801c210 comp rx=0 tx=0).stop 2023-12-19T16:58:31.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.568+0000 7fcae2fce640 1 -- 172.21.15.26:0/2039403405 shutdown_connections 2023-12-19T16:58:31.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.568+0000 7fcae2fce640 1 --2- 172.21.15.26:0/2039403405 >> v2:172.21.15.160:3300/0 conn(0x7fcadc0cde60 0x7fcadc0b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.568+0000 7fcae2fce640 1 --2- 172.21.15.26:0/2039403405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcadc0b85e0 0x7fcadc0cd920 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.568+0000 7fcae2fce640 1 --2- 172.21.15.26:0/2039403405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcadc15cda0 0x7fcadc160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.568+0000 7fcae2fce640 1 -- 172.21.15.26:0/2039403405 >> 172.21.15.26:0/2039403405 conn(0x7fcadc0658e0 msgr2=0x7fcadc065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:31.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.568+0000 7fcae2fce640 1 -- 172.21.15.26:0/2039403405 shutdown_connections 2023-12-19T16:58:31.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.568+0000 7fcae2fce640 1 -- 172.21.15.26:0/2039403405 wait complete. 2023-12-19T16:58:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae2fce640 1 Processor -- start 2023-12-19T16:58:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae2fce640 1 -- start start 2023-12-19T16:58:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae2fce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcadc0b85e0 0x7fcadc147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae2fce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcadc0cde60 0x7fcadc151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae2fce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcadc15cda0 0x7fcadc150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae2fce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0c74a0 con 0x7fcadc15cda0 2023-12-19T16:58:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae2fce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc1484c0 con 0x7fcadc0cde60 2023-12-19T16:58:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae2fce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc151bb0 con 0x7fcadc0b85e0 2023-12-19T16:58:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae156b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcadc15cda0 0x7fcadc150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcadbfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcadc0cde60 0x7fcadc151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae156b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcadc15cda0 0x7fcadc150a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46620/0 (socket says 172.21.15.26:46620) 2023-12-19T16:58:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae156b640 1 -- 172.21.15.26:0/3026797023 learned_addr learned my addr 172.21.15.26:0/3026797023 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae0d6a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcadc0b85e0 0x7fcadc147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae156b640 1 -- 172.21.15.26:0/3026797023 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcadc0b85e0 msgr2=0x7fcadc147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae156b640 1 --2- 172.21.15.26:0/3026797023 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcadc0b85e0 0x7fcadc147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae156b640 1 -- 172.21.15.26:0/3026797023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcadc0cde60 msgr2=0x7fcadc151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:31.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae156b640 1 --2- 172.21.15.26:0/3026797023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcadc0cde60 0x7fcadc151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae156b640 1 -- 172.21.15.26:0/3026797023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac8009480 con 0x7fcadc15cda0 2023-12-19T16:58:31.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae156b640 1 --2- 172.21.15.26:0/3026797023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcadc15cda0 0x7fcadc150a90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fcad4007e40 tx=0x7fcad400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:31.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcad97fa640 1 -- 172.21.15.26:0/3026797023 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4014620 con 0x7fcadc15cda0 2023-12-19T16:58:31.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcad97fa640 1 -- 172.21.15.26:0/3026797023 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad400ee90 con 0x7fcadc15cda0 2023-12-19T16:58:31.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae2fce640 1 -- 172.21.15.26:0/3026797023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc0c6920 con 0x7fcadc15cda0 2023-12-19T16:58:31.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcad97fa640 1 -- 172.21.15.26:0/3026797023 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad40149e0 con 0x7fcadc15cda0 2023-12-19T16:58:31.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.572+0000 7fcae2fce640 1 -- 172.21.15.26:0/3026797023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc0c6ee0 con 0x7fcadc15cda0 2023-12-19T16:58:31.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.576+0000 7fcae2fce640 1 -- 172.21.15.26:0/3026797023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcadc074270 con 0x7fcadc15cda0 2023-12-19T16:58:31.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.576+0000 7fcad97fa640 1 -- 172.21.15.26:0/3026797023 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fcad4014b40 con 0x7fcadc15cda0 2023-12-19T16:58:31.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.576+0000 7fcad97fa640 1 --2- 172.21.15.26:0/3026797023 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcac0065b80 0x7fcac0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:31.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.576+0000 7fcad97fa640 1 -- 172.21.15.26:0/3026797023 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fcad4083e70 con 0x7fcadc15cda0 2023-12-19T16:58:31.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.576+0000 7fcae0d6a640 1 --2- 172.21.15.26:0/3026797023 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcac0065b80 0x7fcac0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:31.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.576+0000 7fcae0d6a640 1 --2- 172.21.15.26:0/3026797023 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcac0065b80 0x7fcac0068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcadc151df0 tx=0x7fcad0008040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:31.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.580+0000 7fcad97fa640 1 -- 172.21.15.26:0/3026797023 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcad40514e0 con 0x7fcadc15cda0 2023-12-19T16:58:31.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcae2fce640 1 -- 172.21.15.26:0/3026797023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcadc0c5e50 con 0x7fcadc15cda0 2023-12-19T16:58:31.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcad97fa640 1 -- 172.21.15.26:0/3026797023 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fcad4054b00 con 0x7fcadc15cda0 2023-12-19T16:58:31.773 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:31.773 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3393501439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3393501439},{"type":"v1","addr":"172.21.15.160:6836","nonce":3393501439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1517280064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1517280064},{"type":"v1","addr":"172.21.15.160:6837","nonce":1517280064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/2282339695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2282339695},{"type":"v1","addr":"172.21.15.26:6834","nonce":2282339695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:58:28.364346+0000","modified":"2023-12-19T16:58:31.386135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2693922436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2693922436},{"type":"v1","addr":"172.21.15.26:6835","nonce":2693922436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T16:58:31.774 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2023-12-19T16:58:31.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcae2fce640 1 -- 172.21.15.26:0/3026797023 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcac0065b80 msgr2=0x7fcac0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:31.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcae2fce640 1 --2- 172.21.15.26:0/3026797023 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcac0065b80 0x7fcac0068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcadc151df0 tx=0x7fcad0008040 comp rx=0 tx=0).stop 2023-12-19T16:58:31.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcae2fce640 1 -- 172.21.15.26:0/3026797023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcadc15cda0 msgr2=0x7fcadc150a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:31.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcae2fce640 1 --2- 172.21.15.26:0/3026797023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcadc15cda0 0x7fcadc150a90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fcad4007e40 tx=0x7fcad400e4b0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcae2fce640 1 -- 172.21.15.26:0/3026797023 shutdown_connections 2023-12-19T16:58:31.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcae2fce640 1 --2- 172.21.15.26:0/3026797023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcadc15cda0 0x7fcadc150a90 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcae2fce640 1 --2- 172.21.15.26:0/3026797023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcadc0cde60 0x7fcadc151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcae2fce640 1 --2- 172.21.15.26:0/3026797023 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcac0065b80 0x7fcac0068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcae2fce640 1 --2- 172.21.15.26:0/3026797023 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcadc0b85e0 0x7fcadc147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcae2fce640 1 -- 172.21.15.26:0/3026797023 >> 172.21.15.26:0/3026797023 conn(0x7fcadc0658e0 msgr2=0x7fcadc0b78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:31.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcae2fce640 1 -- 172.21.15.26:0/3026797023 shutdown_connections 2023-12-19T16:58:31.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.772+0000 7fcae2fce640 1 -- 172.21.15.26:0/3026797023 wait complete. 2023-12-19T16:58:31.795 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:58:31.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.936+0000 7fd114b7b640 1 Processor -- start 2023-12-19T16:58:31.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.936+0000 7fd114b7b640 1 -- start start 2023-12-19T16:58:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd114b7b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1100cb1d0 0x7fd1100cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd114b7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd11006bee0 0x7fd11006c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd114b7b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd11006d4b0 con 0x7fd110164c90 2023-12-19T16:58:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd114b7b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1100cc030 con 0x7fd1100cb1d0 2023-12-19T16:58:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd114b7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1100cbaf0 con 0x7fd11006bee0 2023-12-19T16:58:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd10e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1100cb1d0 0x7fd1100cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd10ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd110164c90 0x7fd110165120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45326/0 (socket says 172.21.15.26:45326) 2023-12-19T16:58:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd10ed9d640 1 -- 172.21.15.26:0/1322300685 learned_addr learned my addr 172.21.15.26:0/1322300685 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd107fff640 1 --2- 172.21.15.26:0/1322300685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd11006bee0 0x7fd11006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd10e59c640 1 -- 172.21.15.26:0/1322300685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd11006bee0 msgr2=0x7fd11006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd10e59c640 1 --2- 172.21.15.26:0/1322300685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd11006bee0 0x7fd11006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd10e59c640 1 -- 172.21.15.26:0/1322300685 >> v1:172.21.15.26:6789/0 conn(0x7fd110164c90 legacy=0x7fd110165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd10dd9b640 1 -- 172.21.15.26:0/1322300685 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351680519 0 0) 0x7fd11006d4b0 con 0x7fd110164c90 2023-12-19T16:58:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd10e59c640 1 -- 172.21.15.26:0/1322300685 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11006cc10 con 0x7fd1100cb1d0 2023-12-19T16:58:31.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd10e59c640 1 --2- 172.21.15.26:0/1322300685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100cb1d0 0x7fd1100cb5b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd1000097c0 tx=0x7fd10001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c5a4b7fa70c94 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:31.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd10dd9b640 1 -- 172.21.15.26:0/1322300685 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1000ac620 con 0x7fd1100cb1d0 2023-12-19T16:58:31.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd10dd9b640 1 -- 172.21.15.26:0/1322300685 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1000ac780 con 0x7fd1100cb1d0 2023-12-19T16:58:31.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.940+0000 7fd10dd9b640 1 -- 172.21.15.26:0/1322300685 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1000acb10 con 0x7fd1100cb1d0 2023-12-19T16:58:31.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 -- 172.21.15.26:0/1322300685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100cb1d0 msgr2=0x7fd1100cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:31.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 --2- 172.21.15.26:0/1322300685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100cb1d0 0x7fd1100cb5b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fd1000097c0 tx=0x7fd10001c3c0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 -- 172.21.15.26:0/1322300685 shutdown_connections 2023-12-19T16:58:31.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 --2- 172.21.15.26:0/1322300685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd11006bee0 0x7fd11006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 --2- 172.21.15.26:0/1322300685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100cb1d0 0x7fd1100cb5b0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 -- 172.21.15.26:0/1322300685 >> 172.21.15.26:0/1322300685 conn(0x7fd1100658e0 msgr2=0x7fd110065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:31.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 -- 172.21.15.26:0/1322300685 shutdown_connections 2023-12-19T16:58:31.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 -- 172.21.15.26:0/1322300685 wait complete. 2023-12-19T16:58:31.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 Processor -- start 2023-12-19T16:58:31.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 -- start start 2023-12-19T16:58:31.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd11006bee0 0x7fd110149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:31.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100cb1d0 0x7fd1100afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:31.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd110164c90 0x7fd1100ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:31.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1100af8f0 con 0x7fd1100cb1d0 2023-12-19T16:58:31.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11014f980 con 0x7fd11006bee0 2023-12-19T16:58:31.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd114b7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1100af230 con 0x7fd110164c90 2023-12-19T16:58:31.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd107fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100cb1d0 0x7fd1100afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:31.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd107fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100cb1d0 0x7fd1100afd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46642/0 (socket says 172.21.15.26:46642) 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd107fff640 1 -- 172.21.15.26:0/3376026656 learned_addr learned my addr 172.21.15.26:0/3376026656 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd10e59c640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd11006bee0 0x7fd110149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.944+0000 7fd10ed9d640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd110164c90 0x7fd1100ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd107fff640 1 -- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd110164c90 msgr2=0x7fd1100ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd107fff640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd110164c90 0x7fd1100ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd107fff640 1 -- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd11006bee0 msgr2=0x7fd110149ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd107fff640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd11006bee0 0x7fd110149ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd107fff640 1 -- 172.21.15.26:0/3376026656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd100009420 con 0x7fd1100cb1d0 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd107fff640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100cb1d0 0x7fd1100afd10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800e870 tx=0x7fd0f800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd106ffd640 1 -- 172.21.15.26:0/3376026656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8009510 con 0x7fd1100cb1d0 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd106ffd640 1 -- 172.21.15.26:0/3376026656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f800eec0 con 0x7fd1100cb1d0 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd114b7b640 1 -- 172.21.15.26:0/3376026656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1100af510 con 0x7fd1100cb1d0 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd106ffd640 1 -- 172.21.15.26:0/3376026656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f800cb40 con 0x7fd1100cb1d0 2023-12-19T16:58:31.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd114b7b640 1 -- 172.21.15.26:0/3376026656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11014f050 con 0x7fd1100cb1d0 2023-12-19T16:58:31.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd114b7b640 1 -- 172.21.15.26:0/3376026656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1100af090 con 0x7fd1100cb1d0 2023-12-19T16:58:31.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd106ffd640 1 -- 172.21.15.26:0/3376026656 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd0f800cca0 con 0x7fd1100cb1d0 2023-12-19T16:58:31.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd106ffd640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd0f0065b80 0x7fd0f0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:31.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.948+0000 7fd106ffd640 1 -- 172.21.15.26:0/3376026656 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fd0f8083660 con 0x7fd1100cb1d0 2023-12-19T16:58:31.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.952+0000 7fd10e59c640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd0f0065b80 0x7fd0f0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:31.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.952+0000 7fd10e59c640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd0f0065b80 0x7fd0f0068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1000ae000 tx=0x7fd1000a7000 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:31.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:31.952+0000 7fd106ffd640 1 -- 172.21.15.26:0/3376026656 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd1100af090 con 0x7fd1100cb1d0 2023-12-19T16:58:32.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.112+0000 7fd114b7b640 1 -- 172.21.15.26:0/3376026656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1100af090 con 0x7fd1100cb1d0 2023-12-19T16:58:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.112+0000 7fd106ffd640 1 -- 172.21.15.26:0/3376026656 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+17208 (secure 0 0 0) 0x7fd1100af090 con 0x7fd1100cb1d0 2023-12-19T16:58:32.115 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:32.115 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":107,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:58:29.367211+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:58:26.906185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T16:58:27.358583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:58:32.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.116+0000 7fd114b7b640 1 -- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd0f0065b80 msgr2=0x7fd0f0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.116+0000 7fd114b7b640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd0f0065b80 0x7fd0f0068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd1000ae000 tx=0x7fd1000a7000 comp rx=0 tx=0).stop 2023-12-19T16:58:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.116+0000 7fd114b7b640 1 -- 172.21.15.26:0/3376026656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100cb1d0 msgr2=0x7fd1100afd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.116+0000 7fd114b7b640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100cb1d0 0x7fd1100afd10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800e870 tx=0x7fd0f800ec70 comp rx=0 tx=0).stop 2023-12-19T16:58:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.116+0000 7fd114b7b640 1 -- 172.21.15.26:0/3376026656 shutdown_connections 2023-12-19T16:58:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.116+0000 7fd114b7b640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd0f0065b80 0x7fd0f0068040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.116+0000 7fd114b7b640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd110164c90 0x7fd1100ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.116+0000 7fd114b7b640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1100cb1d0 0x7fd1100afd10 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.116+0000 7fd114b7b640 1 --2- 172.21.15.26:0/3376026656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd11006bee0 0x7fd110149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.116+0000 7fd114b7b640 1 -- 172.21.15.26:0/3376026656 >> 172.21.15.26:0/3376026656 conn(0x7fd1100658e0 msgr2=0x7fd1101650e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:32.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.116+0000 7fd114b7b640 1 -- 172.21.15.26:0/3376026656 shutdown_connections 2023-12-19T16:58:32.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.116+0000 7fd114b7b640 1 -- 172.21.15.26:0/3376026656 wait complete. 2023-12-19T16:58:32.137 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T16:58:32.137 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:58:32.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.280+0000 7faa5f12c640 1 Processor -- start 2023-12-19T16:58:32.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.280+0000 7faa5f12c640 1 -- start start 2023-12-19T16:58:32.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.280+0000 7faa5f12c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa5806b3d0 0x7faa580ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:32.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.280+0000 7faa5f12c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5806bc30 con 0x7faa58154220 2023-12-19T16:58:32.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.280+0000 7faa5f12c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa580cc6e0 con 0x7faa580cec40 2023-12-19T16:58:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.280+0000 7faa5f12c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa580cf0d0 con 0x7faa5806b3d0 2023-12-19T16:58:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.280+0000 7faa57fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa5806b3d0 0x7faa580ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.280+0000 7faa57fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa5806b3d0 0x7faa580ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46646/0 (socket says 172.21.15.26:46646) 2023-12-19T16:58:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.280+0000 7faa57fff640 1 -- 172.21.15.26:0/1288240318 learned_addr learned my addr 172.21.15.26:0/1288240318 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.280+0000 7faa5cec8640 1 --1- 172.21.15.26:0/1288240318 >> v1:172.21.15.26:6789/0 conn(0x7faa58154220 0x7faa5815aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45330/0 (socket says 172.21.15.26:45330) 2023-12-19T16:58:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.280+0000 7faa57fff640 1 -- 172.21.15.26:0/1288240318 >> v1:172.21.15.160:6789/0 conn(0x7faa580cec40 legacy=0x7faa5815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.280+0000 7faa57fff640 1 -- 172.21.15.26:0/1288240318 >> v1:172.21.15.26:6789/0 conn(0x7faa58154220 legacy=0x7faa5815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.280+0000 7faa57fff640 1 -- 172.21.15.26:0/1288240318 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5806b950 con 0x7faa5806b3d0 2023-12-19T16:58:32.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.284+0000 7faa57fff640 1 --2- 172.21.15.26:0/1288240318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa5806b3d0 0x7faa580ce700 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7faa44009870 tx=0x7faa4401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a9be7ee18d7983 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:32.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.284+0000 7faa577fe640 1 -- 172.21.15.26:0/1288240318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa440aa670 con 0x7faa5806b3d0 2023-12-19T16:58:32.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.284+0000 7faa577fe640 1 -- 172.21.15.26:0/1288240318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa440aa7d0 con 0x7faa5806b3d0 2023-12-19T16:58:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.284+0000 7faa577fe640 1 -- 172.21.15.26:0/1288240318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa440aab60 con 0x7faa5806b3d0 2023-12-19T16:58:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.284+0000 7faa5f12c640 1 -- 172.21.15.26:0/1288240318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa5806b3d0 msgr2=0x7faa580ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.284+0000 7faa5f12c640 1 --2- 172.21.15.26:0/1288240318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa5806b3d0 0x7faa580ce700 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7faa44009870 tx=0x7faa4401c510 comp rx=0 tx=0).stop 2023-12-19T16:58:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.284+0000 7faa5f12c640 1 -- 172.21.15.26:0/1288240318 shutdown_connections 2023-12-19T16:58:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.284+0000 7faa5f12c640 1 --2- 172.21.15.26:0/1288240318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa5806b3d0 0x7faa580ce700 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.284+0000 7faa5f12c640 1 -- 172.21.15.26:0/1288240318 >> 172.21.15.26:0/1288240318 conn(0x7faa580658e0 msgr2=0x7faa58065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.284+0000 7faa5f12c640 1 -- 172.21.15.26:0/1288240318 shutdown_connections 2023-12-19T16:58:32.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.284+0000 7faa5f12c640 1 -- 172.21.15.26:0/1288240318 wait complete. 2023-12-19T16:58:32.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5f12c640 1 Processor -- start 2023-12-19T16:58:32.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5f12c640 1 -- start start 2023-12-19T16:58:32.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5f12c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa5806b3d0 0x7faa58149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5f12c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa580cec40 0x7faa580b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5f12c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa58154220 0x7faa58154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5f12c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa580b3530 con 0x7faa5806b3d0 2023-12-19T16:58:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5f12c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa58155070 con 0x7faa58154220 2023-12-19T16:58:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5f12c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5814b590 con 0x7faa580cec40 2023-12-19T16:58:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5cec8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa5806b3d0 0x7faa58149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5cec8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa5806b3d0 0x7faa58149d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46650/0 (socket says 172.21.15.26:46650) 2023-12-19T16:58:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5cec8640 1 -- 172.21.15.26:0/3750739455 learned_addr learned my addr 172.21.15.26:0/3750739455 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5d6c9640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa58154220 0x7faa58154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa57fff640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa580cec40 0x7faa580b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5cec8640 1 -- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa580cec40 msgr2=0x7faa580b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5cec8640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa580cec40 0x7faa580b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5cec8640 1 -- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa58154220 msgr2=0x7faa58154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5cec8640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa58154220 0x7faa58154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5cec8640 1 -- 172.21.15.26:0/3750739455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa440094d0 con 0x7faa5806b3d0 2023-12-19T16:58:32.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5cec8640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa5806b3d0 0x7faa58149d90 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00e6b0 tx=0x7faa4c00eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:32.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa557fa640 1 -- 172.21.15.26:0/3750739455 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c006c00 con 0x7faa5806b3d0 2023-12-19T16:58:32.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa557fa640 1 -- 172.21.15.26:0/3750739455 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4c006d60 con 0x7faa5806b3d0 2023-12-19T16:58:32.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa557fa640 1 -- 172.21.15.26:0/3750739455 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4c00c860 con 0x7faa5806b3d0 2023-12-19T16:58:32.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5f12c640 1 -- 172.21.15.26:0/3750739455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5814b870 con 0x7faa5806b3d0 2023-12-19T16:58:32.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5f12c640 1 -- 172.21.15.26:0/3750739455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5814be30 con 0x7faa5806b3d0 2023-12-19T16:58:32.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.288+0000 7faa5f12c640 1 -- 172.21.15.26:0/3750739455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5814c020 con 0x7faa5806b3d0 2023-12-19T16:58:32.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.292+0000 7faa557fa640 1 -- 172.21.15.26:0/3750739455 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faa4c00c9c0 con 0x7faa5806b3d0 2023-12-19T16:58:32.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.292+0000 7faa557fa640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa3c065b80 0x7faa3c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:32.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.292+0000 7faa557fa640 1 -- 172.21.15.26:0/3750739455 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7faa4c084920 con 0x7faa5806b3d0 2023-12-19T16:58:32.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.292+0000 7faa57fff640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa3c065b80 0x7faa3c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:32.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.292+0000 7faa57fff640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa3c065b80 0x7faa3c068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa440ae000 tx=0x7faa440b5040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:32.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.296+0000 7faa557fa640 1 -- 172.21.15.26:0/3750739455 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faa4c051f00 con 0x7faa5806b3d0 2023-12-19T16:58:32.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.448+0000 7faa5f12c640 1 -- 172.21.15.26:0/3750739455 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faa580ccef0 con 0x7faa3c065b80 2023-12-19T16:58:32.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.460+0000 7faa557fa640 1 -- 172.21.15.26:0/3750739455 <== mgr.6323 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289517 (secure 0 0 0) 0x7faa440aa940 con 0x7faa3c065b80 2023-12-19T16:58:32.465 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:32.467 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:58:32.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.464+0000 7faa5f12c640 1 -- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa3c065b80 msgr2=0x7faa3c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.464+0000 7faa5f12c640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa3c065b80 0x7faa3c068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7faa440ae000 tx=0x7faa440b5040 comp rx=0 tx=0).stop 2023-12-19T16:58:32.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.464+0000 7faa5f12c640 1 -- 172.21.15.26:0/3750739455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa5806b3d0 msgr2=0x7faa58149d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.464+0000 7faa5f12c640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa5806b3d0 0x7faa58149d90 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00e6b0 tx=0x7faa4c00eb70 comp rx=0 tx=0).stop 2023-12-19T16:58:32.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.464+0000 7faa5f12c640 1 -- 172.21.15.26:0/3750739455 shutdown_connections 2023-12-19T16:58:32.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.464+0000 7faa5f12c640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa58154220 0x7faa58154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.464+0000 7faa5f12c640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa3c065b80 0x7faa3c068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.464+0000 7faa5f12c640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa580cec40 0x7faa580b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.464+0000 7faa5f12c640 1 --2- 172.21.15.26:0/3750739455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa5806b3d0 0x7faa58149d90 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.464+0000 7faa5f12c640 1 -- 172.21.15.26:0/3750739455 >> 172.21.15.26:0/3750739455 conn(0x7faa580658e0 msgr2=0x7faa5815d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:32.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.464+0000 7faa5f12c640 1 -- 172.21.15.26:0/3750739455 shutdown_connections 2023-12-19T16:58:32.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.464+0000 7faa5f12c640 1 -- 172.21.15.26:0/3750739455 wait complete. 2023-12-19T16:58:32.485 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-19T16:58:32.413348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":866,"num_read_kb":1733,"num_write":359,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":241,"ondisk_log_size":241,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":320428,"kb_used_data":11372,"kb_used_omap":0,"kb_used_meta":308992,"kb_avail":754654292,"statfs":{"total":773094113280,"available":772765995008,"internally_reserved":0,"allocated":11644928,"data_stored":6392772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316407808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.450921"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400927+0000","last_change":"2023-12-19T16:58:28.395463+0000","last_active":"2023-12-19T16:58:28.400927+0000","last_peered":"2023-12-19T16:58:28.400927+0000","last_clean":"2023-12-19T16:58:28.400927+0000","last_became_active":"2023-12-19T16:58:28.391282+0000","last_became_peered":"2023-12-19T16:58:28.391282+0000","last_unstale":"2023-12-19T16:58:28.400927+0000","last_undegraded":"2023-12-19T16:58:28.400927+0000","last_fullsized":"2023-12-19T16:58:28.400927+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:53:26.875252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101126+0000","last_change":"2023-12-19T16:58:28.397187+0000","last_active":"2023-12-19T16:58:30.101126+0000","last_peered":"2023-12-19T16:58:30.101126+0000","last_clean":"2023-12-19T16:58:30.101126+0000","last_became_active":"2023-12-19T16:58:28.396108+0000","last_became_peered":"2023-12-19T16:58:28.396108+0000","last_unstale":"2023-12-19T16:58:30.101126+0000","last_undegraded":"2023-12-19T16:58:30.101126+0000","last_fullsized":"2023-12-19T16:58:30.101126+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:34.398222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400498+0000","last_change":"2023-12-19T16:58:28.393814+0000","last_active":"2023-12-19T16:58:28.400498+0000","last_peered":"2023-12-19T16:58:28.400498+0000","last_clean":"2023-12-19T16:58:28.400498+0000","last_became_active":"2023-12-19T16:58:28.393532+0000","last_became_peered":"2023-12-19T16:58:28.393532+0000","last_unstale":"2023-12-19T16:58:28.400498+0000","last_undegraded":"2023-12-19T16:58:28.400498+0000","last_fullsized":"2023-12-19T16:58:28.400498+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:05.089743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.399067+0000","last_change":"2023-12-19T16:58:28.389174+0000","last_active":"2023-12-19T16:58:28.399067+0000","last_peered":"2023-12-19T16:58:28.399067+0000","last_clean":"2023-12-19T16:58:28.399067+0000","last_became_active":"2023-12-19T16:58:28.388913+0000","last_became_peered":"2023-12-19T16:58:28.388913+0000","last_unstale":"2023-12-19T16:58:28.399067+0000","last_undegraded":"2023-12-19T16:58:28.399067+0000","last_fullsized":"2023-12-19T16:58:28.399067+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:00.122204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379928+0000","last_change":"2023-12-19T16:58:28.398965+0000","last_active":"2023-12-19T16:58:29.379928+0000","last_peered":"2023-12-19T16:58:29.379928+0000","last_clean":"2023-12-19T16:58:29.379928+0000","last_became_active":"2023-12-19T16:58:28.398195+0000","last_became_peered":"2023-12-19T16:58:28.398195+0000","last_unstale":"2023-12-19T16:58:29.379928+0000","last_undegraded":"2023-12-19T16:58:29.379928+0000","last_fullsized":"2023-12-19T16:58:29.379928+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:37.217016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380829+0000","last_change":"2023-12-19T16:58:28.396445+0000","last_active":"2023-12-19T16:58:29.380829+0000","last_peered":"2023-12-19T16:58:29.380829+0000","last_clean":"2023-12-19T16:58:29.380829+0000","last_became_active":"2023-12-19T16:58:28.396156+0000","last_became_peered":"2023-12-19T16:58:28.396156+0000","last_unstale":"2023-12-19T16:58:29.380829+0000","last_undegraded":"2023-12-19T16:58:29.380829+0000","last_fullsized":"2023-12-19T16:58:29.380829+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:44.255710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400611+0000","last_change":"2023-12-19T16:58:28.393012+0000","last_active":"2023-12-19T16:58:28.400611+0000","last_peered":"2023-12-19T16:58:28.400611+0000","last_clean":"2023-12-19T16:58:28.400611+0000","last_became_active":"2023-12-19T16:58:28.392767+0000","last_became_peered":"2023-12-19T16:58:28.392767+0000","last_unstale":"2023-12-19T16:58:28.400611+0000","last_undegraded":"2023-12-19T16:58:28.400611+0000","last_fullsized":"2023-12-19T16:58:28.400611+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:19:22.689573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380734+0000","last_change":"2023-12-19T16:58:28.389536+0000","last_active":"2023-12-19T16:58:29.380734+0000","last_peered":"2023-12-19T16:58:29.380734+0000","last_clean":"2023-12-19T16:58:29.380734+0000","last_became_active":"2023-12-19T16:58:28.389259+0000","last_became_peered":"2023-12-19T16:58:28.389259+0000","last_unstale":"2023-12-19T16:58:29.380734+0000","last_undegraded":"2023-12-19T16:58:29.380734+0000","last_fullsized":"2023-12-19T16:58:29.380734+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:42:03.472728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.381091+0000","last_change":"2023-12-19T16:58:28.380203+0000","last_active":"2023-12-19T16:58:29.381091+0000","last_peered":"2023-12-19T16:58:29.381091+0000","last_clean":"2023-12-19T16:58:29.381091+0000","last_became_active":"2023-12-19T16:58:28.379839+0000","last_became_peered":"2023-12-19T16:58:28.379839+0000","last_unstale":"2023-12-19T16:58:29.381091+0000","last_undegraded":"2023-12-19T16:58:29.381091+0000","last_fullsized":"2023-12-19T16:58:29.381091+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:13:13.020419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.402645+0000","last_change":"2023-12-19T16:58:28.398265+0000","last_active":"2023-12-19T16:58:28.402645+0000","last_peered":"2023-12-19T16:58:28.402645+0000","last_clean":"2023-12-19T16:58:28.402645+0000","last_became_active":"2023-12-19T16:58:28.397983+0000","last_became_peered":"2023-12-19T16:58:28.397983+0000","last_unstale":"2023-12-19T16:58:28.402645+0000","last_undegraded":"2023-12-19T16:58:28.402645+0000","last_fullsized":"2023-12-19T16:58:28.402645+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:47.949356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101004+0000","last_change":"2023-12-19T16:58:28.402109+0000","last_active":"2023-12-19T16:58:30.101004+0000","last_peered":"2023-12-19T16:58:30.101004+0000","last_clean":"2023-12-19T16:58:30.101004+0000","last_became_active":"2023-12-19T16:58:28.401836+0000","last_became_peered":"2023-12-19T16:58:28.401836+0000","last_unstale":"2023-12-19T16:58:30.101004+0000","last_undegraded":"2023-12-19T16:58:30.101004+0000","last_fullsized":"2023-12-19T16:58:30.101004+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:13:13.020419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.402706+0000","last_change":"2023-12-19T16:58:28.399027+0000","last_active":"2023-12-19T16:58:28.402706+0000","last_peered":"2023-12-19T16:58:28.402706+0000","last_clean":"2023-12-19T16:58:28.402706+0000","last_became_active":"2023-12-19T16:58:28.397008+0000","last_became_peered":"2023-12-19T16:58:28.397008+0000","last_unstale":"2023-12-19T16:58:28.402706+0000","last_undegraded":"2023-12-19T16:58:28.402706+0000","last_fullsized":"2023-12-19T16:58:28.402706+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:54.099479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.385999+0000","last_change":"2023-12-19T16:58:28.383779+0000","last_active":"2023-12-19T16:58:28.385999+0000","last_peered":"2023-12-19T16:58:28.385999+0000","last_clean":"2023-12-19T16:58:28.385999+0000","last_became_active":"2023-12-19T16:58:28.383481+0000","last_became_peered":"2023-12-19T16:58:28.383481+0000","last_unstale":"2023-12-19T16:58:28.385999+0000","last_undegraded":"2023-12-19T16:58:28.385999+0000","last_fullsized":"2023-12-19T16:58:28.385999+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:41.887964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.408981+0000","last_change":"2023-12-19T16:58:28.407502+0000","last_active":"2023-12-19T16:58:28.408981+0000","last_peered":"2023-12-19T16:58:28.408981+0000","last_clean":"2023-12-19T16:58:28.408981+0000","last_became_active":"2023-12-19T16:58:28.406786+0000","last_became_peered":"2023-12-19T16:58:28.406786+0000","last_unstale":"2023-12-19T16:58:28.408981+0000","last_undegraded":"2023-12-19T16:58:28.408981+0000","last_fullsized":"2023-12-19T16:58:28.408981+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:38:53.835503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.377983+0000","last_change":"2023-12-19T16:58:28.392334+0000","last_active":"2023-12-19T16:58:29.377983+0000","last_peered":"2023-12-19T16:58:29.377983+0000","last_clean":"2023-12-19T16:58:29.377983+0000","last_became_active":"2023-12-19T16:58:28.392072+0000","last_became_peered":"2023-12-19T16:58:28.392072+0000","last_unstale":"2023-12-19T16:58:29.377983+0000","last_undegraded":"2023-12-19T16:58:29.377983+0000","last_fullsized":"2023-12-19T16:58:29.377983+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:18:17.210947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379471+0000","last_change":"2023-12-19T16:58:28.392678+0000","last_active":"2023-12-19T16:58:29.379471+0000","last_peered":"2023-12-19T16:58:29.379471+0000","last_clean":"2023-12-19T16:58:29.379471+0000","last_became_active":"2023-12-19T16:58:28.392408+0000","last_became_peered":"2023-12-19T16:58:28.392408+0000","last_unstale":"2023-12-19T16:58:29.379471+0000","last_undegraded":"2023-12-19T16:58:29.379471+0000","last_fullsized":"2023-12-19T16:58:29.379471+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:11.231928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380306+0000","last_change":"2023-12-19T16:58:28.390549+0000","last_active":"2023-12-19T16:58:29.380306+0000","last_peered":"2023-12-19T16:58:29.380306+0000","last_clean":"2023-12-19T16:58:29.380306+0000","last_became_active":"2023-12-19T16:58:28.389375+0000","last_became_peered":"2023-12-19T16:58:28.389375+0000","last_unstale":"2023-12-19T16:58:29.380306+0000","last_undegraded":"2023-12-19T16:58:29.380306+0000","last_fullsized":"2023-12-19T16:58:29.380306+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:44.512004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400907+0000","last_change":"2023-12-19T16:58:28.396171+0000","last_active":"2023-12-19T16:58:28.400907+0000","last_peered":"2023-12-19T16:58:28.400907+0000","last_clean":"2023-12-19T16:58:28.400907+0000","last_became_active":"2023-12-19T16:58:28.395228+0000","last_became_peered":"2023-12-19T16:58:28.395228+0000","last_unstale":"2023-12-19T16:58:28.400907+0000","last_undegraded":"2023-12-19T16:58:28.400907+0000","last_fullsized":"2023-12-19T16:58:28.400907+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:14:33.253382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.405243+0000","last_change":"2023-12-19T16:58:28.400541+0000","last_active":"2023-12-19T16:58:28.405243+0000","last_peered":"2023-12-19T16:58:28.405243+0000","last_clean":"2023-12-19T16:58:28.405243+0000","last_became_active":"2023-12-19T16:58:28.400276+0000","last_became_peered":"2023-12-19T16:58:28.400276+0000","last_unstale":"2023-12-19T16:58:28.405243+0000","last_undegraded":"2023-12-19T16:58:28.405243+0000","last_fullsized":"2023-12-19T16:58:28.405243+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:46:06.611832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.399049+0000","last_change":"2023-12-19T16:58:28.387006+0000","last_active":"2023-12-19T16:58:28.399049+0000","last_peered":"2023-12-19T16:58:28.399049+0000","last_clean":"2023-12-19T16:58:28.399049+0000","last_became_active":"2023-12-19T16:58:28.385859+0000","last_became_peered":"2023-12-19T16:58:28.385859+0000","last_unstale":"2023-12-19T16:58:28.399049+0000","last_undegraded":"2023-12-19T16:58:28.399049+0000","last_fullsized":"2023-12-19T16:58:28.399049+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:48:00.307695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400027+0000","last_change":"2023-12-19T16:58:28.391296+0000","last_active":"2023-12-19T16:58:28.400027+0000","last_peered":"2023-12-19T16:58:28.400027+0000","last_clean":"2023-12-19T16:58:28.400027+0000","last_became_active":"2023-12-19T16:58:28.390770+0000","last_became_peered":"2023-12-19T16:58:28.390770+0000","last_unstale":"2023-12-19T16:58:28.400027+0000","last_undegraded":"2023-12-19T16:58:28.400027+0000","last_fullsized":"2023-12-19T16:58:28.400027+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:19:03.557277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100821+0000","last_change":"2023-12-19T16:58:28.404204+0000","last_active":"2023-12-19T16:58:30.100821+0000","last_peered":"2023-12-19T16:58:30.100821+0000","last_clean":"2023-12-19T16:58:30.100821+0000","last_became_active":"2023-12-19T16:58:28.403611+0000","last_became_peered":"2023-12-19T16:58:28.403611+0000","last_unstale":"2023-12-19T16:58:30.100821+0000","last_undegraded":"2023-12-19T16:58:30.100821+0000","last_fullsized":"2023-12-19T16:58:30.100821+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:44.255710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.395536+0000","last_change":"2023-12-19T16:58:28.392341+0000","last_active":"2023-12-19T16:58:28.395536+0000","last_peered":"2023-12-19T16:58:28.395536+0000","last_clean":"2023-12-19T16:58:28.395536+0000","last_became_active":"2023-12-19T16:58:28.392072+0000","last_became_peered":"2023-12-19T16:58:28.392072+0000","last_unstale":"2023-12-19T16:58:28.395536+0000","last_undegraded":"2023-12-19T16:58:28.395536+0000","last_fullsized":"2023-12-19T16:58:28.395536+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:13:13.020419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.397827+0000","last_change":"2023-12-19T16:58:28.385578+0000","last_active":"2023-12-19T16:58:28.397827+0000","last_peered":"2023-12-19T16:58:28.397827+0000","last_clean":"2023-12-19T16:58:28.397827+0000","last_became_active":"2023-12-19T16:58:28.385284+0000","last_became_peered":"2023-12-19T16:58:28.385284+0000","last_unstale":"2023-12-19T16:58:28.397827+0000","last_undegraded":"2023-12-19T16:58:28.397827+0000","last_fullsized":"2023-12-19T16:58:28.397827+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:20:14.883721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101101+0000","last_change":"2023-12-19T16:58:28.398690+0000","last_active":"2023-12-19T16:58:30.101101+0000","last_peered":"2023-12-19T16:58:30.101101+0000","last_clean":"2023-12-19T16:58:30.101101+0000","last_became_active":"2023-12-19T16:58:28.398435+0000","last_became_peered":"2023-12-19T16:58:28.398435+0000","last_unstale":"2023-12-19T16:58:30.101101+0000","last_undegraded":"2023-12-19T16:58:30.101101+0000","last_fullsized":"2023-12-19T16:58:30.101101+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:05:44.910475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.401399+0000","last_change":"2023-12-19T16:58:28.399416+0000","last_active":"2023-12-19T16:58:28.401399+0000","last_peered":"2023-12-19T16:58:28.401399+0000","last_clean":"2023-12-19T16:58:28.401399+0000","last_became_active":"2023-12-19T16:58:28.399103+0000","last_became_peered":"2023-12-19T16:58:28.399103+0000","last_unstale":"2023-12-19T16:58:28.401399+0000","last_undegraded":"2023-12-19T16:58:28.401399+0000","last_fullsized":"2023-12-19T16:58:28.401399+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:37:17.302378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379680+0000","last_change":"2023-12-19T16:58:28.397705+0000","last_active":"2023-12-19T16:58:29.379680+0000","last_peered":"2023-12-19T16:58:29.379680+0000","last_clean":"2023-12-19T16:58:29.379680+0000","last_became_active":"2023-12-19T16:58:28.397399+0000","last_became_peered":"2023-12-19T16:58:28.397399+0000","last_unstale":"2023-12-19T16:58:29.379680+0000","last_undegraded":"2023-12-19T16:58:29.379680+0000","last_fullsized":"2023-12-19T16:58:29.379680+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:57:04.690547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.395031+0000","last_change":"2023-12-19T16:58:28.389935+0000","last_active":"2023-12-19T16:58:28.395031+0000","last_peered":"2023-12-19T16:58:28.395031+0000","last_clean":"2023-12-19T16:58:28.395031+0000","last_became_active":"2023-12-19T16:58:28.389656+0000","last_became_peered":"2023-12-19T16:58:28.389656+0000","last_unstale":"2023-12-19T16:58:28.395031+0000","last_undegraded":"2023-12-19T16:58:28.395031+0000","last_fullsized":"2023-12-19T16:58:28.395031+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:24:38.193238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100886+0000","last_change":"2023-12-19T16:58:28.406975+0000","last_active":"2023-12-19T16:58:30.100886+0000","last_peered":"2023-12-19T16:58:30.100886+0000","last_clean":"2023-12-19T16:58:30.100886+0000","last_became_active":"2023-12-19T16:58:28.406504+0000","last_became_peered":"2023-12-19T16:58:28.406504+0000","last_unstale":"2023-12-19T16:58:30.100886+0000","last_undegraded":"2023-12-19T16:58:30.100886+0000","last_fullsized":"2023-12-19T16:58:30.100886+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:11:04.726712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101118+0000","last_change":"2023-12-19T16:58:28.404040+0000","last_active":"2023-12-19T16:58:30.101118+0000","last_peered":"2023-12-19T16:58:30.101118+0000","last_clean":"2023-12-19T16:58:30.101118+0000","last_became_active":"2023-12-19T16:58:28.403252+0000","last_became_peered":"2023-12-19T16:58:28.403252+0000","last_unstale":"2023-12-19T16:58:30.101118+0000","last_undegraded":"2023-12-19T16:58:30.101118+0000","last_fullsized":"2023-12-19T16:58:30.101118+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:37.217016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379634+0000","last_change":"2023-12-19T16:58:28.379743+0000","last_active":"2023-12-19T16:58:29.379634+0000","last_peered":"2023-12-19T16:58:29.379634+0000","last_clean":"2023-12-19T16:58:29.379634+0000","last_became_active":"2023-12-19T16:58:28.379081+0000","last_became_peered":"2023-12-19T16:58:28.379081+0000","last_unstale":"2023-12-19T16:58:29.379634+0000","last_undegraded":"2023-12-19T16:58:29.379634+0000","last_fullsized":"2023-12-19T16:58:29.379634+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:36.607132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.404139+0000","last_change":"2023-12-19T16:58:28.402332+0000","last_active":"2023-12-19T16:58:28.404139+0000","last_peered":"2023-12-19T16:58:28.404139+0000","last_clean":"2023-12-19T16:58:28.404139+0000","last_became_active":"2023-12-19T16:58:28.402016+0000","last_became_peered":"2023-12-19T16:58:28.402016+0000","last_unstale":"2023-12-19T16:58:28.404139+0000","last_undegraded":"2023-12-19T16:58:28.404139+0000","last_fullsized":"2023-12-19T16:58:28.404139+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:56.095278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379682+0000","last_change":"2023-12-19T16:58:28.391791+0000","last_active":"2023-12-19T16:58:29.379682+0000","last_peered":"2023-12-19T16:58:29.379682+0000","last_clean":"2023-12-19T16:58:29.379682+0000","last_became_active":"2023-12-19T16:58:28.391488+0000","last_became_peered":"2023-12-19T16:58:28.391488+0000","last_unstale":"2023-12-19T16:58:29.379682+0000","last_undegraded":"2023-12-19T16:58:29.379682+0000","last_fullsized":"2023-12-19T16:58:29.379682+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:42:03.472728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"106'4","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.404300+0000","last_change":"2023-12-19T16:58:27.980325+0000","last_active":"2023-12-19T16:58:28.404300+0000","last_peered":"2023-12-19T16:58:28.404300+0000","last_clean":"2023-12-19T16:58:28.404300+0000","last_became_active":"2023-12-19T16:58:27.980009+0000","last_became_peered":"2023-12-19T16:58:27.980009+0000","last_unstale":"2023-12-19T16:58:28.404300+0000","last_undegraded":"2023-12-19T16:58:28.404300+0000","last_fullsized":"2023-12-19T16:58:28.404300+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:11.988639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.398858+0000","last_change":"2023-12-19T16:58:28.395189+0000","last_active":"2023-12-19T16:58:28.398858+0000","last_peered":"2023-12-19T16:58:28.398858+0000","last_clean":"2023-12-19T16:58:28.398858+0000","last_became_active":"2023-12-19T16:58:28.394538+0000","last_became_peered":"2023-12-19T16:58:28.394538+0000","last_unstale":"2023-12-19T16:58:28.398858+0000","last_undegraded":"2023-12-19T16:58:28.398858+0000","last_fullsized":"2023-12-19T16:58:28.398858+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:43:46.183766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379966+0000","last_change":"2023-12-19T16:58:27.982211+0000","last_active":"2023-12-19T16:58:29.379966+0000","last_peered":"2023-12-19T16:58:29.379966+0000","last_clean":"2023-12-19T16:58:29.379966+0000","last_became_active":"2023-12-19T16:58:27.981947+0000","last_became_peered":"2023-12-19T16:58:27.981947+0000","last_unstale":"2023-12-19T16:58:29.379966+0000","last_undegraded":"2023-12-19T16:58:29.379966+0000","last_fullsized":"2023-12-19T16:58:29.379966+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:12:09.054363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.402749+0000","last_change":"2023-12-19T16:58:28.400320+0000","last_active":"2023-12-19T16:58:28.402749+0000","last_peered":"2023-12-19T16:58:28.402749+0000","last_clean":"2023-12-19T16:58:28.402749+0000","last_became_active":"2023-12-19T16:58:28.398806+0000","last_became_peered":"2023-12-19T16:58:28.398806+0000","last_unstale":"2023-12-19T16:58:28.402749+0000","last_undegraded":"2023-12-19T16:58:28.402749+0000","last_fullsized":"2023-12-19T16:58:28.402749+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:29.117499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.401413+0000","last_change":"2023-12-19T16:58:27.981164+0000","last_active":"2023-12-19T16:58:28.401413+0000","last_peered":"2023-12-19T16:58:28.401413+0000","last_clean":"2023-12-19T16:58:28.401413+0000","last_became_active":"2023-12-19T16:58:27.979348+0000","last_became_peered":"2023-12-19T16:58:27.979348+0000","last_unstale":"2023-12-19T16:58:28.401413+0000","last_undegraded":"2023-12-19T16:58:28.401413+0000","last_fullsized":"2023-12-19T16:58:28.401413+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:35.349198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.398113+0000","last_change":"2023-12-19T16:58:28.391133+0000","last_active":"2023-12-19T16:58:28.398113+0000","last_peered":"2023-12-19T16:58:28.398113+0000","last_clean":"2023-12-19T16:58:28.398113+0000","last_became_active":"2023-12-19T16:58:28.390866+0000","last_became_peered":"2023-12-19T16:58:28.390866+0000","last_unstale":"2023-12-19T16:58:28.398113+0000","last_undegraded":"2023-12-19T16:58:28.398113+0000","last_fullsized":"2023-12-19T16:58:28.398113+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:24.692744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.402413+0000","last_change":"2023-12-19T16:58:27.978882+0000","last_active":"2023-12-19T16:58:28.402413+0000","last_peered":"2023-12-19T16:58:28.402413+0000","last_clean":"2023-12-19T16:58:28.402413+0000","last_became_active":"2023-12-19T16:58:27.978520+0000","last_became_peered":"2023-12-19T16:58:27.978520+0000","last_unstale":"2023-12-19T16:58:28.402413+0000","last_undegraded":"2023-12-19T16:58:28.402413+0000","last_fullsized":"2023-12-19T16:58:28.402413+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:36:39.412312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100964+0000","last_change":"2023-12-19T16:58:28.403876+0000","last_active":"2023-12-19T16:58:30.100964+0000","last_peered":"2023-12-19T16:58:30.100964+0000","last_clean":"2023-12-19T16:58:30.100964+0000","last_became_active":"2023-12-19T16:58:28.402906+0000","last_became_peered":"2023-12-19T16:58:28.402906+0000","last_unstale":"2023-12-19T16:58:30.100964+0000","last_undegraded":"2023-12-19T16:58:30.100964+0000","last_fullsized":"2023-12-19T16:58:30.100964+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:57:04.690547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380106+0000","last_change":"2023-12-19T16:58:27.987378+0000","last_active":"2023-12-19T16:58:29.380106+0000","last_peered":"2023-12-19T16:58:29.380106+0000","last_clean":"2023-12-19T16:58:29.380106+0000","last_became_active":"2023-12-19T16:58:27.986991+0000","last_became_peered":"2023-12-19T16:58:27.986991+0000","last_unstale":"2023-12-19T16:58:29.380106+0000","last_undegraded":"2023-12-19T16:58:29.380106+0000","last_fullsized":"2023-12-19T16:58:29.380106+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:36:27.751276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379996+0000","last_change":"2023-12-19T16:58:28.388593+0000","last_active":"2023-12-19T16:58:29.379996+0000","last_peered":"2023-12-19T16:58:29.379996+0000","last_clean":"2023-12-19T16:58:29.379996+0000","last_became_active":"2023-12-19T16:58:28.388331+0000","last_became_peered":"2023-12-19T16:58:28.388331+0000","last_unstale":"2023-12-19T16:58:29.379996+0000","last_undegraded":"2023-12-19T16:58:29.379996+0000","last_fullsized":"2023-12-19T16:58:29.379996+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:05:44.910475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380226+0000","last_change":"2023-12-19T16:58:27.982923+0000","last_active":"2023-12-19T16:58:29.380226+0000","last_peered":"2023-12-19T16:58:29.380226+0000","last_clean":"2023-12-19T16:58:29.380226+0000","last_became_active":"2023-12-19T16:58:27.982618+0000","last_became_peered":"2023-12-19T16:58:27.982618+0000","last_unstale":"2023-12-19T16:58:29.380226+0000","last_undegraded":"2023-12-19T16:58:29.380226+0000","last_fullsized":"2023-12-19T16:58:29.380226+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:24.714136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400018+0000","last_change":"2023-12-19T16:58:28.392878+0000","last_active":"2023-12-19T16:58:28.400018+0000","last_peered":"2023-12-19T16:58:28.400018+0000","last_clean":"2023-12-19T16:58:28.400018+0000","last_became_active":"2023-12-19T16:58:28.392131+0000","last_became_peered":"2023-12-19T16:58:28.392131+0000","last_unstale":"2023-12-19T16:58:28.400018+0000","last_undegraded":"2023-12-19T16:58:28.400018+0000","last_fullsized":"2023-12-19T16:58:28.400018+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:24.245385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.369574+0000","last_change":"2023-12-19T16:58:27.982469+0000","last_active":"2023-12-19T16:58:28.369574+0000","last_peered":"2023-12-19T16:58:28.369574+0000","last_clean":"2023-12-19T16:58:28.369574+0000","last_became_active":"2023-12-19T16:58:27.982219+0000","last_became_peered":"2023-12-19T16:58:27.982219+0000","last_unstale":"2023-12-19T16:58:28.369574+0000","last_undegraded":"2023-12-19T16:58:28.369574+0000","last_fullsized":"2023-12-19T16:58:28.369574+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:16:49.807412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.377982+0000","last_change":"2023-12-19T16:58:28.383391+0000","last_active":"2023-12-19T16:58:29.377982+0000","last_peered":"2023-12-19T16:58:29.377982+0000","last_clean":"2023-12-19T16:58:29.377982+0000","last_became_active":"2023-12-19T16:58:28.383145+0000","last_became_peered":"2023-12-19T16:58:28.383145+0000","last_unstale":"2023-12-19T16:58:29.377982+0000","last_undegraded":"2023-12-19T16:58:29.377982+0000","last_fullsized":"2023-12-19T16:58:29.377982+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:35.704685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100770+0000","last_change":"2023-12-19T16:58:27.986696+0000","last_active":"2023-12-19T16:58:30.100770+0000","last_peered":"2023-12-19T16:58:30.100770+0000","last_clean":"2023-12-19T16:58:30.100770+0000","last_became_active":"2023-12-19T16:58:27.985791+0000","last_became_peered":"2023-12-19T16:58:27.985791+0000","last_unstale":"2023-12-19T16:58:30.100770+0000","last_undegraded":"2023-12-19T16:58:30.100770+0000","last_fullsized":"2023-12-19T16:58:30.100770+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:08.124546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.378348+0000","last_change":"2023-12-19T16:58:28.387976+0000","last_active":"2023-12-19T16:58:29.378348+0000","last_peered":"2023-12-19T16:58:29.378348+0000","last_clean":"2023-12-19T16:58:29.378348+0000","last_became_active":"2023-12-19T16:58:28.387663+0000","last_became_peered":"2023-12-19T16:58:28.387663+0000","last_unstale":"2023-12-19T16:58:29.378348+0000","last_undegraded":"2023-12-19T16:58:29.378348+0000","last_fullsized":"2023-12-19T16:58:29.378348+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:01:07.088699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.380765+0000","last_change":"2023-12-19T16:58:27.983171+0000","last_active":"2023-12-19T16:58:28.380765+0000","last_peered":"2023-12-19T16:58:28.380765+0000","last_clean":"2023-12-19T16:58:28.380765+0000","last_became_active":"2023-12-19T16:58:27.982909+0000","last_became_peered":"2023-12-19T16:58:27.982909+0000","last_unstale":"2023-12-19T16:58:28.380765+0000","last_undegraded":"2023-12-19T16:58:28.380765+0000","last_fullsized":"2023-12-19T16:58:28.380765+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:27:44.415340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.381111+0000","last_change":"2023-12-19T16:58:28.388446+0000","last_active":"2023-12-19T16:58:29.381111+0000","last_peered":"2023-12-19T16:58:29.381111+0000","last_clean":"2023-12-19T16:58:29.381111+0000","last_became_active":"2023-12-19T16:58:28.388129+0000","last_became_peered":"2023-12-19T16:58:28.388129+0000","last_unstale":"2023-12-19T16:58:29.381111+0000","last_undegraded":"2023-12-19T16:58:29.381111+0000","last_fullsized":"2023-12-19T16:58:29.381111+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:36.607132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.381455+0000","last_change":"2023-12-19T16:58:27.982711+0000","last_active":"2023-12-19T16:58:28.381455+0000","last_peered":"2023-12-19T16:58:28.381455+0000","last_clean":"2023-12-19T16:58:28.381455+0000","last_became_active":"2023-12-19T16:58:27.982437+0000","last_became_peered":"2023-12-19T16:58:27.982437+0000","last_unstale":"2023-12-19T16:58:28.381455+0000","last_undegraded":"2023-12-19T16:58:28.381455+0000","last_fullsized":"2023-12-19T16:58:28.381455+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:24.714136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.401057+0000","last_change":"2023-12-19T16:58:28.394629+0000","last_active":"2023-12-19T16:58:28.401057+0000","last_peered":"2023-12-19T16:58:28.401057+0000","last_clean":"2023-12-19T16:58:28.401057+0000","last_became_active":"2023-12-19T16:58:28.393543+0000","last_became_peered":"2023-12-19T16:58:28.393543+0000","last_unstale":"2023-12-19T16:58:28.401057+0000","last_undegraded":"2023-12-19T16:58:28.401057+0000","last_fullsized":"2023-12-19T16:58:28.401057+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:24:01.098046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"106'3","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400012+0000","last_change":"2023-12-19T16:58:27.984876+0000","last_active":"2023-12-19T16:58:28.400012+0000","last_peered":"2023-12-19T16:58:28.400012+0000","last_clean":"2023-12-19T16:58:28.400012+0000","last_became_active":"2023-12-19T16:58:27.984601+0000","last_became_peered":"2023-12-19T16:58:27.984601+0000","last_unstale":"2023-12-19T16:58:28.400012+0000","last_undegraded":"2023-12-19T16:58:28.400012+0000","last_fullsized":"2023-12-19T16:58:28.400012+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:48.685656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379757+0000","last_change":"2023-12-19T16:58:28.387850+0000","last_active":"2023-12-19T16:58:29.379757+0000","last_peered":"2023-12-19T16:58:29.379757+0000","last_clean":"2023-12-19T16:58:29.379757+0000","last_became_active":"2023-12-19T16:58:28.387571+0000","last_became_peered":"2023-12-19T16:58:28.387571+0000","last_unstale":"2023-12-19T16:58:29.379757+0000","last_undegraded":"2023-12-19T16:58:29.379757+0000","last_fullsized":"2023-12-19T16:58:29.379757+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:47.949356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"106'6","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380926+0000","last_change":"2023-12-19T16:58:27.983185+0000","last_active":"2023-12-19T16:58:29.380926+0000","last_peered":"2023-12-19T16:58:29.380926+0000","last_clean":"2023-12-19T16:58:29.380926+0000","last_became_active":"2023-12-19T16:58:27.981664+0000","last_became_peered":"2023-12-19T16:58:27.981664+0000","last_unstale":"2023-12-19T16:58:29.380926+0000","last_undegraded":"2023-12-19T16:58:29.380926+0000","last_fullsized":"2023-12-19T16:58:29.380926+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:11.988639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.393082+0000","last_change":"2023-12-19T16:58:28.382566+0000","last_active":"2023-12-19T16:58:28.393082+0000","last_peered":"2023-12-19T16:58:28.393082+0000","last_clean":"2023-12-19T16:58:28.393082+0000","last_became_active":"2023-12-19T16:58:28.382305+0000","last_became_peered":"2023-12-19T16:58:28.382305+0000","last_unstale":"2023-12-19T16:58:28.393082+0000","last_undegraded":"2023-12-19T16:58:28.393082+0000","last_fullsized":"2023-12-19T16:58:28.393082+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:47:50.538756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100986+0000","last_change":"2023-12-19T16:58:27.982846+0000","last_active":"2023-12-19T16:58:30.100986+0000","last_peered":"2023-12-19T16:58:30.100986+0000","last_clean":"2023-12-19T16:58:30.100986+0000","last_became_active":"2023-12-19T16:58:27.982549+0000","last_became_peered":"2023-12-19T16:58:27.982549+0000","last_unstale":"2023-12-19T16:58:30.100986+0000","last_undegraded":"2023-12-19T16:58:30.100986+0000","last_fullsized":"2023-12-19T16:58:30.100986+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:36:27.751276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400877+0000","last_change":"2023-12-19T16:58:28.397981+0000","last_active":"2023-12-19T16:58:28.400877+0000","last_peered":"2023-12-19T16:58:28.400877+0000","last_clean":"2023-12-19T16:58:28.400877+0000","last_became_active":"2023-12-19T16:58:28.397696+0000","last_became_peered":"2023-12-19T16:58:28.397696+0000","last_unstale":"2023-12-19T16:58:28.400877+0000","last_undegraded":"2023-12-19T16:58:28.400877+0000","last_fullsized":"2023-12-19T16:58:28.400877+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:34:28.248975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.378157+0000","last_change":"2023-12-19T16:58:27.987869+0000","last_active":"2023-12-19T16:58:29.378157+0000","last_peered":"2023-12-19T16:58:29.378157+0000","last_clean":"2023-12-19T16:58:29.378157+0000","last_became_active":"2023-12-19T16:58:27.987525+0000","last_became_peered":"2023-12-19T16:58:27.987525+0000","last_unstale":"2023-12-19T16:58:29.378157+0000","last_undegraded":"2023-12-19T16:58:29.378157+0000","last_fullsized":"2023-12-19T16:58:29.378157+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:25:23.743802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100708+0000","last_change":"2023-12-19T16:58:28.391359+0000","last_active":"2023-12-19T16:58:30.100708+0000","last_peered":"2023-12-19T16:58:30.100708+0000","last_clean":"2023-12-19T16:58:30.100708+0000","last_became_active":"2023-12-19T16:58:28.391056+0000","last_became_peered":"2023-12-19T16:58:28.391056+0000","last_unstale":"2023-12-19T16:58:30.100708+0000","last_undegraded":"2023-12-19T16:58:30.100708+0000","last_fullsized":"2023-12-19T16:58:30.100708+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:41.622888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380717+0000","last_change":"2023-12-19T16:58:27.984801+0000","last_active":"2023-12-19T16:58:29.380717+0000","last_peered":"2023-12-19T16:58:29.380717+0000","last_clean":"2023-12-19T16:58:29.380717+0000","last_became_active":"2023-12-19T16:58:27.984513+0000","last_became_peered":"2023-12-19T16:58:27.984513+0000","last_unstale":"2023-12-19T16:58:29.380717+0000","last_undegraded":"2023-12-19T16:58:29.380717+0000","last_fullsized":"2023-12-19T16:58:29.380717+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:48:20.013396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100651+0000","last_change":"2023-12-19T16:58:28.396661+0000","last_active":"2023-12-19T16:58:30.100651+0000","last_peered":"2023-12-19T16:58:30.100651+0000","last_clean":"2023-12-19T16:58:30.100651+0000","last_became_active":"2023-12-19T16:58:28.396352+0000","last_became_peered":"2023-12-19T16:58:28.396352+0000","last_unstale":"2023-12-19T16:58:30.100651+0000","last_undegraded":"2023-12-19T16:58:30.100651+0000","last_fullsized":"2023-12-19T16:58:30.100651+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:44.512004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"106'8","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.404723+0000","last_change":"2023-12-19T16:58:27.986085+0000","last_active":"2023-12-19T16:58:28.404723+0000","last_peered":"2023-12-19T16:58:28.404723+0000","last_clean":"2023-12-19T16:58:28.404723+0000","last_became_active":"2023-12-19T16:58:27.985783+0000","last_became_peered":"2023-12-19T16:58:27.985783+0000","last_unstale":"2023-12-19T16:58:28.404723+0000","last_undegraded":"2023-12-19T16:58:28.404723+0000","last_fullsized":"2023-12-19T16:58:28.404723+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:43.497884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.393588+0000","last_change":"2023-12-19T16:58:28.388033+0000","last_active":"2023-12-19T16:58:28.393588+0000","last_peered":"2023-12-19T16:58:28.393588+0000","last_clean":"2023-12-19T16:58:28.393588+0000","last_became_active":"2023-12-19T16:58:28.387744+0000","last_became_peered":"2023-12-19T16:58:28.387744+0000","last_unstale":"2023-12-19T16:58:28.393588+0000","last_undegraded":"2023-12-19T16:58:28.393588+0000","last_fullsized":"2023-12-19T16:58:28.393588+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:25:24.746600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.395735+0000","last_change":"2023-12-19T16:58:27.986462+0000","last_active":"2023-12-19T16:58:28.395735+0000","last_peered":"2023-12-19T16:58:28.395735+0000","last_clean":"2023-12-19T16:58:28.395735+0000","last_became_active":"2023-12-19T16:58:27.985925+0000","last_became_peered":"2023-12-19T16:58:27.985925+0000","last_unstale":"2023-12-19T16:58:28.395735+0000","last_undegraded":"2023-12-19T16:58:28.395735+0000","last_fullsized":"2023-12-19T16:58:28.395735+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:04:06.694458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.395808+0000","last_change":"2023-12-19T16:58:28.390457+0000","last_active":"2023-12-19T16:58:28.395808+0000","last_peered":"2023-12-19T16:58:28.395808+0000","last_clean":"2023-12-19T16:58:28.395808+0000","last_became_active":"2023-12-19T16:58:28.390098+0000","last_became_peered":"2023-12-19T16:58:28.390098+0000","last_unstale":"2023-12-19T16:58:28.395808+0000","last_undegraded":"2023-12-19T16:58:28.395808+0000","last_fullsized":"2023-12-19T16:58:28.395808+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:09:54.894372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.397397+0000","last_change":"2023-12-19T16:58:27.984105+0000","last_active":"2023-12-19T16:58:28.397397+0000","last_peered":"2023-12-19T16:58:28.397397+0000","last_clean":"2023-12-19T16:58:28.397397+0000","last_became_active":"2023-12-19T16:58:27.983826+0000","last_became_peered":"2023-12-19T16:58:27.983826+0000","last_unstale":"2023-12-19T16:58:28.397397+0000","last_undegraded":"2023-12-19T16:58:28.397397+0000","last_fullsized":"2023-12-19T16:58:28.397397+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:28:17.238941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379794+0000","last_change":"2023-12-19T16:58:28.396774+0000","last_active":"2023-12-19T16:58:29.379794+0000","last_peered":"2023-12-19T16:58:29.379794+0000","last_clean":"2023-12-19T16:58:29.379794+0000","last_became_active":"2023-12-19T16:58:28.396493+0000","last_became_peered":"2023-12-19T16:58:28.396493+0000","last_unstale":"2023-12-19T16:58:29.379794+0000","last_undegraded":"2023-12-19T16:58:29.379794+0000","last_fullsized":"2023-12-19T16:58:29.379794+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:13:13.020419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.370266+0000","last_change":"2023-12-19T16:58:27.982910+0000","last_active":"2023-12-19T16:58:28.370266+0000","last_peered":"2023-12-19T16:58:28.370266+0000","last_clean":"2023-12-19T16:58:28.370266+0000","last_became_active":"2023-12-19T16:58:27.982166+0000","last_became_peered":"2023-12-19T16:58:27.982166+0000","last_unstale":"2023-12-19T16:58:28.370266+0000","last_undegraded":"2023-12-19T16:58:28.370266+0000","last_fullsized":"2023-12-19T16:58:28.370266+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:51:55.444146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.409017+0000","last_change":"2023-12-19T16:58:28.407661+0000","last_active":"2023-12-19T16:58:28.409017+0000","last_peered":"2023-12-19T16:58:28.409017+0000","last_clean":"2023-12-19T16:58:28.409017+0000","last_became_active":"2023-12-19T16:58:28.407216+0000","last_became_peered":"2023-12-19T16:58:28.407216+0000","last_unstale":"2023-12-19T16:58:28.409017+0000","last_undegraded":"2023-12-19T16:58:28.409017+0000","last_fullsized":"2023-12-19T16:58:28.409017+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:20:08.229107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.378337+0000","last_change":"2023-12-19T16:58:27.979783+0000","last_active":"2023-12-19T16:58:29.378337+0000","last_peered":"2023-12-19T16:58:29.378337+0000","last_clean":"2023-12-19T16:58:29.378337+0000","last_became_active":"2023-12-19T16:58:27.979474+0000","last_became_peered":"2023-12-19T16:58:27.979474+0000","last_unstale":"2023-12-19T16:58:29.378337+0000","last_undegraded":"2023-12-19T16:58:29.378337+0000","last_fullsized":"2023-12-19T16:58:29.378337+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:04.086945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380784+0000","last_change":"2023-12-19T16:58:28.396048+0000","last_active":"2023-12-19T16:58:29.380784+0000","last_peered":"2023-12-19T16:58:29.380784+0000","last_clean":"2023-12-19T16:58:29.380784+0000","last_became_active":"2023-12-19T16:58:28.395726+0000","last_became_peered":"2023-12-19T16:58:28.395726+0000","last_unstale":"2023-12-19T16:58:29.380784+0000","last_undegraded":"2023-12-19T16:58:29.380784+0000","last_fullsized":"2023-12-19T16:58:29.380784+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:11.231928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.380446+0000","last_change":"2023-12-19T16:58:27.977554+0000","last_active":"2023-12-19T16:58:28.380446+0000","last_peered":"2023-12-19T16:58:28.380446+0000","last_clean":"2023-12-19T16:58:28.380446+0000","last_became_active":"2023-12-19T16:58:27.977121+0000","last_became_peered":"2023-12-19T16:58:27.977121+0000","last_unstale":"2023-12-19T16:58:28.380446+0000","last_undegraded":"2023-12-19T16:58:28.380446+0000","last_fullsized":"2023-12-19T16:58:28.380446+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:38.524842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.398928+0000","last_change":"2023-12-19T16:58:28.390022+0000","last_active":"2023-12-19T16:58:28.398928+0000","last_peered":"2023-12-19T16:58:28.398928+0000","last_clean":"2023-12-19T16:58:28.398928+0000","last_became_active":"2023-12-19T16:58:28.389744+0000","last_became_peered":"2023-12-19T16:58:28.389744+0000","last_unstale":"2023-12-19T16:58:28.398928+0000","last_undegraded":"2023-12-19T16:58:28.398928+0000","last_fullsized":"2023-12-19T16:58:28.398928+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:45.967982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.380292+0000","last_change":"2023-12-19T16:58:27.983065+0000","last_active":"2023-12-19T16:58:28.380292+0000","last_peered":"2023-12-19T16:58:28.380292+0000","last_clean":"2023-12-19T16:58:28.380292+0000","last_became_active":"2023-12-19T16:58:27.982805+0000","last_became_peered":"2023-12-19T16:58:27.982805+0000","last_unstale":"2023-12-19T16:58:28.380292+0000","last_undegraded":"2023-12-19T16:58:28.380292+0000","last_fullsized":"2023-12-19T16:58:28.380292+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:41:24.316368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380714+0000","last_change":"2023-12-19T16:58:28.384439+0000","last_active":"2023-12-19T16:58:29.380714+0000","last_peered":"2023-12-19T16:58:29.380714+0000","last_clean":"2023-12-19T16:58:29.380714+0000","last_became_active":"2023-12-19T16:58:28.383397+0000","last_became_peered":"2023-12-19T16:58:28.383397+0000","last_unstale":"2023-12-19T16:58:29.380714+0000","last_undegraded":"2023-12-19T16:58:29.380714+0000","last_fullsized":"2023-12-19T16:58:29.380714+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:57:04.690547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.377942+0000","last_change":"2023-12-19T16:58:27.980579+0000","last_active":"2023-12-19T16:58:29.377942+0000","last_peered":"2023-12-19T16:58:29.377942+0000","last_clean":"2023-12-19T16:58:29.377942+0000","last_became_active":"2023-12-19T16:58:27.980324+0000","last_became_peered":"2023-12-19T16:58:27.980324+0000","last_unstale":"2023-12-19T16:58:29.377942+0000","last_undegraded":"2023-12-19T16:58:29.377942+0000","last_fullsized":"2023-12-19T16:58:29.377942+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:40.885166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379846+0000","last_change":"2023-12-19T16:58:28.381233+0000","last_active":"2023-12-19T16:58:29.379846+0000","last_peered":"2023-12-19T16:58:29.379846+0000","last_clean":"2023-12-19T16:58:29.379846+0000","last_became_active":"2023-12-19T16:58:28.380924+0000","last_became_peered":"2023-12-19T16:58:28.380924+0000","last_unstale":"2023-12-19T16:58:29.379846+0000","last_undegraded":"2023-12-19T16:58:29.379846+0000","last_fullsized":"2023-12-19T16:58:29.379846+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:44.255710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101099+0000","last_change":"2023-12-19T16:58:27.988714+0000","last_active":"2023-12-19T16:58:30.101099+0000","last_peered":"2023-12-19T16:58:30.101099+0000","last_clean":"2023-12-19T16:58:30.101099+0000","last_became_active":"2023-12-19T16:58:27.988432+0000","last_became_peered":"2023-12-19T16:58:27.988432+0000","last_unstale":"2023-12-19T16:58:30.101099+0000","last_undegraded":"2023-12-19T16:58:30.101099+0000","last_fullsized":"2023-12-19T16:58:30.101099+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:12:09.054363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.399907+0000","last_change":"2023-12-19T16:58:28.392218+0000","last_active":"2023-12-19T16:58:28.399907+0000","last_peered":"2023-12-19T16:58:28.399907+0000","last_clean":"2023-12-19T16:58:28.399907+0000","last_became_active":"2023-12-19T16:58:28.391792+0000","last_became_peered":"2023-12-19T16:58:28.391792+0000","last_unstale":"2023-12-19T16:58:28.399907+0000","last_undegraded":"2023-12-19T16:58:28.399907+0000","last_fullsized":"2023-12-19T16:58:28.399907+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:33:04.099843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101111+0000","last_change":"2023-12-19T16:58:27.984423+0000","last_active":"2023-12-19T16:58:30.101111+0000","last_peered":"2023-12-19T16:58:30.101111+0000","last_clean":"2023-12-19T16:58:30.101111+0000","last_became_active":"2023-12-19T16:58:27.984122+0000","last_became_peered":"2023-12-19T16:58:27.984122+0000","last_unstale":"2023-12-19T16:58:30.101111+0000","last_undegraded":"2023-12-19T16:58:30.101111+0000","last_fullsized":"2023-12-19T16:58:30.101111+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:24.714136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.402774+0000","last_change":"2023-12-19T16:58:28.399913+0000","last_active":"2023-12-19T16:58:28.402774+0000","last_peered":"2023-12-19T16:58:28.402774+0000","last_clean":"2023-12-19T16:58:28.402774+0000","last_became_active":"2023-12-19T16:58:28.399490+0000","last_became_peered":"2023-12-19T16:58:28.399490+0000","last_unstale":"2023-12-19T16:58:28.402774+0000","last_undegraded":"2023-12-19T16:58:28.402774+0000","last_fullsized":"2023-12-19T16:58:28.402774+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:42:03.472728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.381945+0000","last_change":"2023-12-19T16:58:27.979794+0000","last_active":"2023-12-19T16:58:28.381945+0000","last_peered":"2023-12-19T16:58:28.381945+0000","last_clean":"2023-12-19T16:58:28.381945+0000","last_became_active":"2023-12-19T16:58:27.979446+0000","last_became_peered":"2023-12-19T16:58:27.979446+0000","last_unstale":"2023-12-19T16:58:28.381945+0000","last_undegraded":"2023-12-19T16:58:28.381945+0000","last_fullsized":"2023-12-19T16:58:28.381945+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:48:20.013396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379875+0000","last_change":"2023-12-19T16:58:28.384646+0000","last_active":"2023-12-19T16:58:29.379875+0000","last_peered":"2023-12-19T16:58:29.379875+0000","last_clean":"2023-12-19T16:58:29.379875+0000","last_became_active":"2023-12-19T16:58:28.384382+0000","last_became_peered":"2023-12-19T16:58:28.384382+0000","last_unstale":"2023-12-19T16:58:29.379875+0000","last_undegraded":"2023-12-19T16:58:29.379875+0000","last_fullsized":"2023-12-19T16:58:29.379875+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:43:46.183766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.381858+0000","last_change":"2023-12-19T16:58:27.978599+0000","last_active":"2023-12-19T16:58:28.381858+0000","last_peered":"2023-12-19T16:58:28.381858+0000","last_clean":"2023-12-19T16:58:28.381858+0000","last_became_active":"2023-12-19T16:58:27.978232+0000","last_became_peered":"2023-12-19T16:58:27.978232+0000","last_unstale":"2023-12-19T16:58:28.381858+0000","last_undegraded":"2023-12-19T16:58:28.381858+0000","last_fullsized":"2023-12-19T16:58:28.381858+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:59:51.105534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380871+0000","last_change":"2023-12-19T16:58:28.395430+0000","last_active":"2023-12-19T16:58:29.380871+0000","last_peered":"2023-12-19T16:58:29.380871+0000","last_clean":"2023-12-19T16:58:29.380871+0000","last_became_active":"2023-12-19T16:58:28.395103+0000","last_became_peered":"2023-12-19T16:58:28.395103+0000","last_unstale":"2023-12-19T16:58:29.380871+0000","last_undegraded":"2023-12-19T16:58:29.380871+0000","last_fullsized":"2023-12-19T16:58:29.380871+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:37.217016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.369795+0000","last_change":"2023-12-19T16:58:27.989285+0000","last_active":"2023-12-19T16:58:28.369795+0000","last_peered":"2023-12-19T16:58:28.369795+0000","last_clean":"2023-12-19T16:58:28.369795+0000","last_became_active":"2023-12-19T16:58:27.988986+0000","last_became_peered":"2023-12-19T16:58:27.988986+0000","last_unstale":"2023-12-19T16:58:28.369795+0000","last_undegraded":"2023-12-19T16:58:28.369795+0000","last_fullsized":"2023-12-19T16:58:28.369795+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:51:40.636904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.391742+0000","last_change":"2023-12-19T16:58:28.383556+0000","last_active":"2023-12-19T16:58:28.391742+0000","last_peered":"2023-12-19T16:58:28.391742+0000","last_clean":"2023-12-19T16:58:28.391742+0000","last_became_active":"2023-12-19T16:58:28.383270+0000","last_became_peered":"2023-12-19T16:58:28.383270+0000","last_unstale":"2023-12-19T16:58:28.391742+0000","last_undegraded":"2023-12-19T16:58:28.391742+0000","last_fullsized":"2023-12-19T16:58:28.391742+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:08.073433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380226+0000","last_change":"2023-12-19T16:58:27.982943+0000","last_active":"2023-12-19T16:58:29.380226+0000","last_peered":"2023-12-19T16:58:29.380226+0000","last_clean":"2023-12-19T16:58:29.380226+0000","last_became_active":"2023-12-19T16:58:27.982695+0000","last_became_peered":"2023-12-19T16:58:27.982695+0000","last_unstale":"2023-12-19T16:58:29.380226+0000","last_undegraded":"2023-12-19T16:58:29.380226+0000","last_fullsized":"2023-12-19T16:58:29.380226+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:38.848518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.403638+0000","last_change":"2023-12-19T16:58:28.402429+0000","last_active":"2023-12-19T16:58:28.403638+0000","last_peered":"2023-12-19T16:58:28.403638+0000","last_clean":"2023-12-19T16:58:28.403638+0000","last_became_active":"2023-12-19T16:58:28.402068+0000","last_became_peered":"2023-12-19T16:58:28.402068+0000","last_unstale":"2023-12-19T16:58:28.403638+0000","last_undegraded":"2023-12-19T16:58:28.403638+0000","last_fullsized":"2023-12-19T16:58:28.403638+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:49:39.660527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.380729+0000","last_change":"2023-12-19T16:58:27.977143+0000","last_active":"2023-12-19T16:58:28.380729+0000","last_peered":"2023-12-19T16:58:28.380729+0000","last_clean":"2023-12-19T16:58:28.380729+0000","last_became_active":"2023-12-19T16:58:27.976879+0000","last_became_peered":"2023-12-19T16:58:27.976879+0000","last_unstale":"2023-12-19T16:58:28.380729+0000","last_undegraded":"2023-12-19T16:58:28.380729+0000","last_fullsized":"2023-12-19T16:58:28.380729+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:38:04.181014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.402675+0000","last_change":"2023-12-19T16:58:28.398342+0000","last_active":"2023-12-19T16:58:28.402675+0000","last_peered":"2023-12-19T16:58:28.402675+0000","last_clean":"2023-12-19T16:58:28.402675+0000","last_became_active":"2023-12-19T16:58:28.396953+0000","last_became_peered":"2023-12-19T16:58:28.396953+0000","last_unstale":"2023-12-19T16:58:28.402675+0000","last_undegraded":"2023-12-19T16:58:28.402675+0000","last_fullsized":"2023-12-19T16:58:28.402675+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:11.231928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379728+0000","last_change":"2023-12-19T16:58:27.982173+0000","last_active":"2023-12-19T16:58:29.379728+0000","last_peered":"2023-12-19T16:58:29.379728+0000","last_clean":"2023-12-19T16:58:29.379728+0000","last_became_active":"2023-12-19T16:58:27.980350+0000","last_became_peered":"2023-12-19T16:58:27.980350+0000","last_unstale":"2023-12-19T16:58:29.379728+0000","last_undegraded":"2023-12-19T16:58:29.379728+0000","last_fullsized":"2023-12-19T16:58:29.379728+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:08.124546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101057+0000","last_change":"2023-12-19T16:58:28.392286+0000","last_active":"2023-12-19T16:58:30.101057+0000","last_peered":"2023-12-19T16:58:30.101057+0000","last_clean":"2023-12-19T16:58:30.101057+0000","last_became_active":"2023-12-19T16:58:28.392037+0000","last_became_peered":"2023-12-19T16:58:28.392037+0000","last_unstale":"2023-12-19T16:58:30.101057+0000","last_undegraded":"2023-12-19T16:58:30.101057+0000","last_fullsized":"2023-12-19T16:58:30.101057+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:36.607132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.399191+0000","last_change":"2023-12-19T16:58:27.987854+0000","last_active":"2023-12-19T16:58:28.399191+0000","last_peered":"2023-12-19T16:58:28.399191+0000","last_clean":"2023-12-19T16:58:28.399191+0000","last_became_active":"2023-12-19T16:58:27.987485+0000","last_became_peered":"2023-12-19T16:58:27.987485+0000","last_unstale":"2023-12-19T16:58:28.399191+0000","last_undegraded":"2023-12-19T16:58:28.399191+0000","last_fullsized":"2023-12-19T16:58:28.399191+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:42:02.469929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"95'203","reported_seq":547,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-12-19T16:58:27.975099+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:58:27.975099+0000","last_peered":"2023-12-19T16:58:27.975099+0000","last_clean":"2023-12-19T16:58:27.975099+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:58:27.975099+0000","last_undegraded":"2023-12-19T16:58:27.975099+0000","last_fullsized":"2023-12-19T16:58:27.975099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":866,"num_read_kb":1733,"num_write":315,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":866,"num_read_kb":1733,"num_write":315,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509635,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36992,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334848,"statfs":{"total":96636764160,"available":96598884352,"internally_reserved":0,"allocated":1892352,"data_stored":1238715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.30199999999999999},"max":{"1min":1.0169999999999999,"5min":1.3460000000000001,"15min":2.6389999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.72899999999999998,"15min":0.71499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.32700000000000001},"max":{"1min":0.875,"5min":1.6930000000000001,"15min":2.532},"last":0.54400000000000004}]},{"osd":1,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.73299999999999998,"15min":0.71599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.36399999999999999},"max":{"1min":0.875,"5min":1.212,"15min":1.212},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.72999999999999998,"15min":0.72799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":0.95899999999999996,"5min":1.74,"15min":1.74},"last":0.64100000000000001}]},{"osd":2,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.436,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89500000000000002,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75900000000000001,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36599999999999999},"max":{"1min":1.048,"5min":1.395,"15min":1.395},"last":0.5}]},{"osd":3,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.73299999999999998,"15min":0.72099999999999997},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.44600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97899999999999998,"5min":1.266,"15min":1.266},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.28499999999999998},"max":{"1min":0.91000000000000003,"5min":1.841,"15min":1.841},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.44,"5min":0.40600000000000003,"15min":0.36599999999999999},"max":{"1min":0.80800000000000005,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.69499999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.35699999999999998},"max":{"1min":0.85799999999999998,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98299999999999998,"5min":1.784,"15min":1.784},"last":0.59799999999999998}]},{"osd":6,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70399999999999996,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.45700000000000002,"15min":0.39800000000000002},"max":{"1min":0.94599999999999995,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.50800000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.84599999999999997,"5min":2.105,"15min":2.105},"last":0.72699999999999998}]}]},{"osd":6,"up_from":15,"seq":64424509635,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1306624,"data_stored":648971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69499999999999995,"15min":0.69999999999999996},"min":{"1min":0.51600000000000001,"5min":0.38700000000000001,"15min":0.309},"max":{"1min":0.93700000000000006,"5min":0.999,"15min":1.8600000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70999999999999996,"15min":0.70099999999999996},"min":{"1min":0.46200000000000002,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":0.93300000000000005,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.501,"5min":0.41399999999999998,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":0.99399999999999999,"15min":4.3799999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.52100000000000002,"5min":0.47799999999999998,"15min":0.39500000000000002},"max":{"1min":0.95199999999999996,"5min":1.0189999999999999,"15min":1.583},"last":0.74399999999999999}]},{"osd":2,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.751,"15min":0.74299999999999999},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.35199999999999998},"max":{"1min":0.89200000000000002,"5min":1.087,"15min":1.3180000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0549999999999999,"15min":1.1879999999999999},"last":0.76400000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.753,"15min":0.77800000000000002},"min":{"1min":0.51100000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97699999999999998,"5min":1.1020000000000001,"15min":4.673},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71499999999999997,"15min":0.72499999999999998},"min":{"1min":0.44900000000000001,"5min":0.42699999999999999,"15min":0.41399999999999998},"max":{"1min":0.84299999999999997,"5min":1.0740000000000001,"15min":1.962},"last":0.53500000000000003}]},{"osd":4,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.88400000000000001,"15min":0.96999999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.35699999999999998},"max":{"1min":0.83699999999999997,"5min":0.93500000000000005,"15min":1.006},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.33300000000000002},"max":{"1min":0.83999999999999997,"5min":0.90300000000000002,"15min":1.2050000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.68000000000000005,"15min":0.65600000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41499999999999998},"max":{"1min":0.89900000000000002,"5min":1.046,"15min":1.046},"last":0.71699999999999997}]},{"osd":7,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70499999999999996,"15min":0.69599999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.438},"max":{"1min":0.92300000000000004,"5min":1.006,"15min":1.0620000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71299999999999997,"15min":0.70099999999999996},"min":{"1min":0.47399999999999998,"5min":0.46600000000000003,"15min":0.371},"max":{"1min":0.90300000000000002,"5min":1.032,"15min":1.1220000000000001},"last":0.69999999999999996}]}]},{"osd":5,"up_from":15,"seq":64424509635,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36436,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335404,"statfs":{"total":96636764160,"available":96599453696,"internally_reserved":0,"allocated":1323008,"data_stored":656895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68100000000000005,"15min":0.69899999999999995},"min":{"1min":0.46899999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.84099999999999997,"5min":0.88400000000000001,"15min":0.95799999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90400000000000003,"5min":0.93999999999999995,"15min":1.0189999999999999},"last":0.72199999999999998}]},{"osd":1,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.48299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95099999999999996,"5min":0.995,"15min":0.995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.495,"5min":0.374,"15min":0.374},"max":{"1min":0.96699999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.5,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.89500000000000002,"5min":1.0409999999999999,"15min":1.1120000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.46400000000000002},"max":{"1min":0.84899999999999998,"5min":1.036,"15min":1.087},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.72899999999999998},"min":{"1min":0.51700000000000002,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":0.92000000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.77000000000000002,"15min":0.76100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.998,"5min":1.089,"15min":1.089},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54200000000000004,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77800000000000002,"5min":0.83999999999999997,"15min":1.165},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.32400000000000001},"max":{"1min":0.91100000000000003,"5min":0.92600000000000005,"15min":1.0649999999999999},"last":0.79700000000000004}]},{"osd":6,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61199999999999999,"15min":0.60899999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.34699999999999998},"max":{"1min":0.88800000000000001,"5min":0.98799999999999999,"15min":1.153},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.94199999999999995,"15min":0.98999999999999999},"last":0.59799999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.133},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.67700000000000005},"min":{"1min":0.45500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1080000000000001,"5min":1.131,"15min":1.131},"last":0.69599999999999995}]}]},{"osd":4,"up_from":15,"seq":64424509635,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54468,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317372,"statfs":{"total":96636764160,"available":96580988928,"internally_reserved":0,"allocated":1306624,"data_stored":655309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.39500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0740000000000001,"5min":1.877,"15min":1.877},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92800000000000005,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64500000000000002}]},{"osd":1,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.44900000000000001,"5min":0.44400000000000001,"15min":0.34799999999999998},"max":{"1min":0.85799999999999998,"5min":1.0209999999999999,"15min":1.8700000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":2.02,"15min":2.02},"last":0.69499999999999995}]},{"osd":2,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.54700000000000004,"5min":0.47099999999999997,"15min":0.38700000000000001},"max":{"1min":0.97199999999999998,"5min":1.375,"15min":1.375},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76700000000000002,"15min":0.753},"min":{"1min":0.439,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":1.927,"15min":1.927},"last":0.746}]},{"osd":3,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88500000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.74099999999999999,"15min":0.75600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92100000000000004,"5min":0.97599999999999998,"15min":1.3129999999999999},"last":0.77100000000000002}]},{"osd":5,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.379,"15min":0.35899999999999999},"max":{"1min":0.97699999999999998,"5min":1.99,"15min":1.99},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.42099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82299999999999995,"5min":0.89200000000000002,"15min":0.92800000000000005},"last":0.82999999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59799999999999998,"15min":0.61599999999999999},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.0820000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.50700000000000001,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":1.042,"5min":1.96,"15min":1.96},"last":0.93500000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63300000000000001,"15min":0.65200000000000002},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.80500000000000005,"5min":0.89700000000000002,"15min":1.0429999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.54100000000000004,"5min":0.46400000000000002,"15min":0.44900000000000001},"max":{"1min":1.008,"5min":1.103,"15min":1.107},"last":0.873}]}]},{"osd":3,"up_from":15,"seq":64424509635,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36432,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335408,"statfs":{"total":96636764160,"available":96599457792,"internally_reserved":0,"allocated":1318912,"data_stored":656873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.44900000000000001},"min":{"1min":0.28000000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63300000000000001,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.496},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.32200000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70199999999999996,"5min":1.163,"15min":1.738},"last":0.41299999999999998}]},{"osd":1,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45500000000000002,"15min":0.46500000000000002},"min":{"1min":0.29899999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":1.141},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75900000000000001,"5min":1.952,"15min":2.4910000000000001},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.53000000000000003,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.85399999999999998,"15min":1.821},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.39700000000000002,"5min":0.372,"15min":0.35399999999999998},"max":{"1min":0.78100000000000003,"5min":2.855,"15min":5.1219999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.50600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.081,"5min":4.234,"15min":4.234},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.755,"15min":0.78500000000000003},"min":{"1min":0.56399999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0649999999999999,"5min":1.111,"15min":2.2690000000000001},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.78600000000000003},"min":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.52800000000000002},"max":{"1min":1.071,"5min":1.071,"15min":2.5350000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77000000000000002,"15min":0.78800000000000003},"min":{"1min":0.54400000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.042,"5min":1.8300000000000001,"15min":1.9079999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.77600000000000002,"15min":0.80000000000000004},"min":{"1min":0.51200000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98499999999999999,"5min":1.599,"15min":1.956},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76900000000000002,"15min":0.79800000000000004},"min":{"1min":0.54500000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0569999999999999,"5min":1.0780000000000001,"15min":1.974},"last":0.63}]},{"osd":7,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.80600000000000005,"15min":0.83099999999999996},"min":{"1min":0.63,"5min":0.54700000000000004,"15min":0.51900000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.9279999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.82999999999999996,"15min":0.86599999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.48799999999999999},"max":{"1min":0.97199999999999998,"5min":1.623,"15min":5.0759999999999996},"last":0.72199999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509634,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40508,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331332,"statfs":{"total":96636764160,"available":96595283968,"internally_reserved":0,"allocated":1298432,"data_stored":648859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.49399999999999999,"15min":0.47099999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28199999999999997},"max":{"1min":0.59399999999999997,"5min":5.4870000000000001,"15min":5.4870000000000001},"last":0.313},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.59299999999999997,"15min":0.53300000000000003},"min":{"1min":0.30599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.58899999999999997,"5min":10.161,"15min":10.161},"last":0.54500000000000004}]},{"osd":1,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.625,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.41099999999999998,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":2.4740000000000002,"15min":5.7590000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70799999999999996,"5min":2.911,"15min":5.8070000000000004},"last":0.435}]},{"osd":3,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":1.1990000000000001,"15min":2.7599999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70099999999999996,"5min":2.7999999999999998,"15min":5.7190000000000003},"last":0.46500000000000002}]},{"osd":4,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78900000000000003,"15min":0.79800000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3,"5min":2.4449999999999998,"15min":3.8940000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81599999999999995,"15min":0.79300000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.42999999999999999},"max":{"1min":1.226,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":0.53400000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.84799999999999998,"15min":0.80900000000000005},"min":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.4890000000000001,"5min":5.601,"15min":5.601},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81100000000000005,"15min":0.82599999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.268,"5min":2.569,"15min":2.9260000000000002},"last":0.68400000000000005}]},{"osd":6,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83399999999999996,"15min":0.83299999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.52400000000000002},"max":{"1min":1.4319999999999999,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.76500000000000001,"15min":0.79800000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":2.121},"last":0.72499999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.87,"15min":0.82799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52300000000000002},"max":{"1min":1.3340000000000001,"5min":5.673,"15min":5.673},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82199999999999995,"15min":0.84599999999999997},"min":{"1min":0.624,"5min":0.60599999999999998,"15min":0.57799999999999996},"max":{"1min":1.369,"5min":1.429,"15min":2.0529999999999999},"last":0.71399999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509635,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50372,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321468,"statfs":{"total":96636764160,"available":96585183232,"internally_reserved":0,"allocated":1306624,"data_stored":648471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.433,"15min":0.44700000000000001},"min":{"1min":0.27900000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":2.3050000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.50600000000000001,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65900000000000003,"5min":0.84199999999999997,"15min":0.89800000000000002},"last":0.47399999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.51200000000000001},"min":{"1min":0.32900000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66000000000000003,"5min":0.80100000000000005,"15min":1.992},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.29799999999999999},"max":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.97299999999999998},"last":0.38300000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.61799999999999999},"min":{"1min":0.43099999999999999,"5min":0.36899999999999999,"15min":0.34799999999999998},"max":{"1min":0.76200000000000001,"5min":0.871,"15min":4.133},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33700000000000002},"max":{"1min":0.746,"5min":0.76400000000000001,"15min":0.91900000000000004},"last":0.55800000000000005}]},{"osd":4,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.745,"15min":0.76400000000000001},"min":{"1min":0.45300000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.107,"5min":1.202,"15min":2.1949999999999998},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.78100000000000003,"15min":0.78800000000000003},"min":{"1min":0.56499999999999995,"5min":0.44400000000000001,"15min":0.42399999999999999},"max":{"1min":1.01,"5min":1.181,"15min":2.048},"last":0.79300000000000004}]},{"osd":5,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77200000000000002,"15min":0.78500000000000003},"min":{"1min":0.56200000000000006,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.0589999999999999,"5min":1.288,"15min":2.1269999999999998},"last":0.872},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.59499999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.04,"5min":1.3380000000000001,"15min":4.0460000000000003},"last":0.81699999999999995}]},{"osd":6,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76200000000000001,"15min":0.79500000000000004},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.41999999999999998},"max":{"1min":0.98599999999999999,"5min":1.0740000000000001,"15min":2.1499999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79300000000000004,"15min":0.80900000000000005},"min":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1659999999999999,"5min":1.391,"15min":2.0880000000000001},"last":0.82899999999999996}]},{"osd":7,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82699999999999996,"15min":0.84199999999999997},"min":{"1min":0.63,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.024,"5min":1.2230000000000001,"15min":4.093},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80000000000000004,"15min":0.83599999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.53800000000000003},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":2.2349999999999999},"last":0.88400000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542340,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1892352,"data_stored":1238679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.434,"15min":0.42899999999999999},"min":{"1min":0.314,"5min":0.246,"15min":0.246},"max":{"1min":0.65600000000000003,"5min":0.752,"15min":0.752},"last":0.439},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.49299999999999999},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.26300000000000001},"max":{"1min":0.73299999999999998,"5min":0.81699999999999995,"15min":0.85499999999999998},"last":0.45400000000000001}]},{"osd":2,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.89700000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39800000000000002,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57299999999999995}]},{"osd":3,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.84699999999999998,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.628,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.40899999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.503,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.149,"5min":1.444,"15min":1.512},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.78700000000000003},"min":{"1min":0.56000000000000005,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0349999999999999,"5min":1.1020000000000001,"15min":1.52},"last":0.62}]},{"osd":5,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.80200000000000005,"15min":0.80000000000000004},"min":{"1min":0.55700000000000005,"5min":0.502,"15min":0.41499999999999998},"max":{"1min":1.125,"5min":1.294,"15min":1.748},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81299999999999994,"15min":0.79600000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.51600000000000001},"max":{"1min":1.1830000000000001,"5min":1.256,"15min":1.649},"last":0.70099999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81100000000000005,"15min":0.79600000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46999999999999997},"max":{"1min":1.2050000000000001,"5min":1.429,"15min":1.429},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.82099999999999995,"15min":0.81499999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.44700000000000001},"max":{"1min":1.224,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.59099999999999997}]},{"osd":7,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82999999999999996,"15min":0.82199999999999995},"min":{"1min":0.58599999999999997,"5min":0.505,"15min":0.48999999999999999},"max":{"1min":1.077,"5min":1.377,"15min":1.377},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.81699999999999995},"min":{"1min":0.53200000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0580000000000001,"5min":1.3300000000000001,"15min":1.661},"last":0.66800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:58:32.494 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T16:58:32.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.672+0000 7f8b13a2d640 1 Processor -- start 2023-12-19T16:58:32.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b13a2d640 1 -- start start 2023-12-19T16:58:32.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b13a2d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8b0c06b3d0 0x7f8b0c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:32.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b13a2d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0c06bc30 con 0x7f8b0c0cec40 2023-12-19T16:58:32.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b13a2d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0c0cc6e0 con 0x7f8b0c154220 2023-12-19T16:58:32.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b13a2d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0c0cf0d0 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b11fca640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8b0c0cec40 0x7f8b0c15b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45338/0 (socket says 172.21.15.26:45338) 2023-12-19T16:58:32.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b11fca640 1 -- 172.21.15.26:0/3712368427 learned_addr learned my addr 172.21.15.26:0/3712368427 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:32.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b10fc8640 1 --2- 172.21.15.26:0/3712368427 >> v2:172.21.15.160:3300/0 conn(0x7f8b0c06b3d0 0x7f8b0c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:32.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b03fff640 1 -- 172.21.15.26:0/3712368427 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071520085 0 0) 0x7f8b0c06bc30 con 0x7f8b0c0cec40 2023-12-19T16:58:32.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b03fff640 1 -- 172.21.15.26:0/3712368427 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8af8003530 con 0x7f8b0c0cec40 2023-12-19T16:58:32.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b10fc8640 1 -- 172.21.15.26:0/3712368427 >> v1:172.21.15.160:6789/0 conn(0x7f8b0c154220 legacy=0x7f8b0c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b10fc8640 1 -- 172.21.15.26:0/3712368427 >> v1:172.21.15.26:6789/0 conn(0x7f8b0c0cec40 legacy=0x7f8b0c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b03fff640 1 -- 172.21.15.26:0/3712368427 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4083043664 0 0) 0x7f8af8003530 con 0x7f8b0c0cec40 2023-12-19T16:58:32.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.676+0000 7f8b10fc8640 1 -- 172.21.15.26:0/3712368427 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c06b950 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.680+0000 7f8b10fc8640 1 --2- 172.21.15.26:0/3712368427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b0c06b3d0 0x7f8b0c0ce700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0097c0 tx=0x7f8afc01c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ec8513a67b3438 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:32.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.680+0000 7f8b03fff640 1 -- 172.21.15.26:0/3712368427 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04001a00 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.680+0000 7f8b03fff640 1 -- 172.21.15.26:0/3712368427 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8afc0a7050 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.680+0000 7f8b03fff640 1 -- 172.21.15.26:0/3712368427 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc0ab620 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.680+0000 7f8b13a2d640 1 -- 172.21.15.26:0/3712368427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b0c06b3d0 msgr2=0x7f8b0c0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.680+0000 7f8b13a2d640 1 --2- 172.21.15.26:0/3712368427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b0c06b3d0 0x7f8b0c0ce700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0097c0 tx=0x7f8afc01c370 comp rx=0 tx=0).stop 2023-12-19T16:58:32.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.680+0000 7f8b13a2d640 1 -- 172.21.15.26:0/3712368427 shutdown_connections 2023-12-19T16:58:32.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.680+0000 7f8b13a2d640 1 --2- 172.21.15.26:0/3712368427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b0c06b3d0 0x7f8b0c0ce700 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.680+0000 7f8b13a2d640 1 -- 172.21.15.26:0/3712368427 >> 172.21.15.26:0/3712368427 conn(0x7f8b0c0658e0 msgr2=0x7f8b0c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:32.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.680+0000 7f8b13a2d640 1 -- 172.21.15.26:0/3712368427 shutdown_connections 2023-12-19T16:58:32.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.680+0000 7f8b13a2d640 1 -- 172.21.15.26:0/3712368427 wait complete. 2023-12-19T16:58:32.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.680+0000 7f8b13a2d640 1 Processor -- start 2023-12-19T16:58:32.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b13a2d640 1 -- start start 2023-12-19T16:58:32.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b13a2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0c06b3d0 0x7f8b0c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:32.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b13a2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b0c0cec40 0x7f8b0c070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:32.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b13a2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b0c154220 0x7f8b0c14d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:32.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b13a2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c1525a0 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b13a2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c1527f0 con 0x7f8b0c0cec40 2023-12-19T16:58:32.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b13a2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c152ad0 con 0x7f8b0c154220 2023-12-19T16:58:32.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b117c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0c06b3d0 0x7f8b0c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:32.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b117c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0c06b3d0 0x7f8b0c0af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46654/0 (socket says 172.21.15.26:46654) 2023-12-19T16:58:32.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b117c9640 1 -- 172.21.15.26:0/644929145 learned_addr learned my addr 172.21.15.26:0/644929145 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:32.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b11fca640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b0c154220 0x7f8b0c14d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:32.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b10fc8640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b0c0cec40 0x7f8b0c070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:32.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b117c9640 1 -- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b0c154220 msgr2=0x7f8b0c14d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b117c9640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b0c154220 0x7f8b0c14d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b117c9640 1 -- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b0c0cec40 msgr2=0x7f8b0c070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b117c9640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b0c0cec40 0x7f8b0c070c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b117c9640 1 -- 172.21.15.26:0/644929145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc009420 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b117c9640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0c06b3d0 0x7f8b0c0af9c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800c3c0 tx=0x7f8b0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:32.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b01ffb640 1 -- 172.21.15.26:0/644929145 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080052d0 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b01ffb640 1 -- 172.21.15.26:0/644929145 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b08005430 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b01ffb640 1 -- 172.21.15.26:0/644929145 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b08003840 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b13a2d640 1 -- 172.21.15.26:0/644929145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0c152db0 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.684+0000 7f8b13a2d640 1 -- 172.21.15.26:0/644929145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c0711d0 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.688+0000 7f8b13a2d640 1 -- 172.21.15.26:0/644929145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0c074270 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.688+0000 7f8b01ffb640 1 -- 172.21.15.26:0/644929145 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8b080039a0 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.688+0000 7f8b01ffb640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8af0065b80 0x7f8af0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:32.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.688+0000 7f8b01ffb640 1 -- 172.21.15.26:0/644929145 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f8b08082aa0 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.688+0000 7f8b10fc8640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8af0065b80 0x7f8af0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:32.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.688+0000 7f8b10fc8640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8af0065b80 0x7f8af0068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0ad000 tx=0x7f8afc008040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:32.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.692+0000 7f8b01ffb640 1 -- 172.21.15.26:0/644929145 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8b08050100 con 0x7f8b0c06b3d0 2023-12-19T16:58:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.848+0000 7f8b13a2d640 1 -- 172.21.15.26:0/644929145 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b0c152960 con 0x7f8af0065b80 2023-12-19T16:58:32.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.856+0000 7f8b01ffb640 1 -- 172.21.15.26:0/644929145 <== mgr.6323 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289517 (secure 0 0 0) 0x7f8b0c152960 con 0x7f8af0065b80 2023-12-19T16:58:32.862 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T16:58:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.860+0000 7f8b13a2d640 1 -- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8af0065b80 msgr2=0x7f8af0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.860+0000 7f8b13a2d640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8af0065b80 0x7f8af0068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8afc0ad000 tx=0x7f8afc008040 comp rx=0 tx=0).stop 2023-12-19T16:58:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.860+0000 7f8b13a2d640 1 -- 172.21.15.26:0/644929145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0c06b3d0 msgr2=0x7f8b0c0af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.860+0000 7f8b13a2d640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0c06b3d0 0x7f8b0c0af9c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800c3c0 tx=0x7f8b0800c880 comp rx=0 tx=0).stop 2023-12-19T16:58:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.860+0000 7f8b13a2d640 1 -- 172.21.15.26:0/644929145 shutdown_connections 2023-12-19T16:58:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.860+0000 7f8b13a2d640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8af0065b80 0x7f8af0068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.860+0000 7f8b13a2d640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b0c154220 0x7f8b0c14d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.860+0000 7f8b13a2d640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b0c0cec40 0x7f8b0c070c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.860+0000 7f8b13a2d640 1 --2- 172.21.15.26:0/644929145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0c06b3d0 0x7f8b0c0af9c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.860+0000 7f8b13a2d640 1 -- 172.21.15.26:0/644929145 >> 172.21.15.26:0/644929145 conn(0x7f8b0c0658e0 msgr2=0x7f8b0c0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.860+0000 7f8b13a2d640 1 -- 172.21.15.26:0/644929145 shutdown_connections 2023-12-19T16:58:32.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:32.860+0000 7f8b13a2d640 1 -- 172.21.15.26:0/644929145 wait complete. 2023-12-19T16:58:32.882 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-19T16:58:32.413348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":866,"num_read_kb":1733,"num_write":359,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":241,"ondisk_log_size":241,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":320428,"kb_used_data":11372,"kb_used_omap":0,"kb_used_meta":308992,"kb_avail":754654292,"statfs":{"total":773094113280,"available":772765995008,"internally_reserved":0,"allocated":11644928,"data_stored":6392772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316407808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.450921"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400927+0000","last_change":"2023-12-19T16:58:28.395463+0000","last_active":"2023-12-19T16:58:28.400927+0000","last_peered":"2023-12-19T16:58:28.400927+0000","last_clean":"2023-12-19T16:58:28.400927+0000","last_became_active":"2023-12-19T16:58:28.391282+0000","last_became_peered":"2023-12-19T16:58:28.391282+0000","last_unstale":"2023-12-19T16:58:28.400927+0000","last_undegraded":"2023-12-19T16:58:28.400927+0000","last_fullsized":"2023-12-19T16:58:28.400927+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:53:26.875252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101126+0000","last_change":"2023-12-19T16:58:28.397187+0000","last_active":"2023-12-19T16:58:30.101126+0000","last_peered":"2023-12-19T16:58:30.101126+0000","last_clean":"2023-12-19T16:58:30.101126+0000","last_became_active":"2023-12-19T16:58:28.396108+0000","last_became_peered":"2023-12-19T16:58:28.396108+0000","last_unstale":"2023-12-19T16:58:30.101126+0000","last_undegraded":"2023-12-19T16:58:30.101126+0000","last_fullsized":"2023-12-19T16:58:30.101126+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:34.398222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400498+0000","last_change":"2023-12-19T16:58:28.393814+0000","last_active":"2023-12-19T16:58:28.400498+0000","last_peered":"2023-12-19T16:58:28.400498+0000","last_clean":"2023-12-19T16:58:28.400498+0000","last_became_active":"2023-12-19T16:58:28.393532+0000","last_became_peered":"2023-12-19T16:58:28.393532+0000","last_unstale":"2023-12-19T16:58:28.400498+0000","last_undegraded":"2023-12-19T16:58:28.400498+0000","last_fullsized":"2023-12-19T16:58:28.400498+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:05.089743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.399067+0000","last_change":"2023-12-19T16:58:28.389174+0000","last_active":"2023-12-19T16:58:28.399067+0000","last_peered":"2023-12-19T16:58:28.399067+0000","last_clean":"2023-12-19T16:58:28.399067+0000","last_became_active":"2023-12-19T16:58:28.388913+0000","last_became_peered":"2023-12-19T16:58:28.388913+0000","last_unstale":"2023-12-19T16:58:28.399067+0000","last_undegraded":"2023-12-19T16:58:28.399067+0000","last_fullsized":"2023-12-19T16:58:28.399067+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:00.122204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379928+0000","last_change":"2023-12-19T16:58:28.398965+0000","last_active":"2023-12-19T16:58:29.379928+0000","last_peered":"2023-12-19T16:58:29.379928+0000","last_clean":"2023-12-19T16:58:29.379928+0000","last_became_active":"2023-12-19T16:58:28.398195+0000","last_became_peered":"2023-12-19T16:58:28.398195+0000","last_unstale":"2023-12-19T16:58:29.379928+0000","last_undegraded":"2023-12-19T16:58:29.379928+0000","last_fullsized":"2023-12-19T16:58:29.379928+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:37.217016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380829+0000","last_change":"2023-12-19T16:58:28.396445+0000","last_active":"2023-12-19T16:58:29.380829+0000","last_peered":"2023-12-19T16:58:29.380829+0000","last_clean":"2023-12-19T16:58:29.380829+0000","last_became_active":"2023-12-19T16:58:28.396156+0000","last_became_peered":"2023-12-19T16:58:28.396156+0000","last_unstale":"2023-12-19T16:58:29.380829+0000","last_undegraded":"2023-12-19T16:58:29.380829+0000","last_fullsized":"2023-12-19T16:58:29.380829+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:44.255710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400611+0000","last_change":"2023-12-19T16:58:28.393012+0000","last_active":"2023-12-19T16:58:28.400611+0000","last_peered":"2023-12-19T16:58:28.400611+0000","last_clean":"2023-12-19T16:58:28.400611+0000","last_became_active":"2023-12-19T16:58:28.392767+0000","last_became_peered":"2023-12-19T16:58:28.392767+0000","last_unstale":"2023-12-19T16:58:28.400611+0000","last_undegraded":"2023-12-19T16:58:28.400611+0000","last_fullsized":"2023-12-19T16:58:28.400611+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:19:22.689573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380734+0000","last_change":"2023-12-19T16:58:28.389536+0000","last_active":"2023-12-19T16:58:29.380734+0000","last_peered":"2023-12-19T16:58:29.380734+0000","last_clean":"2023-12-19T16:58:29.380734+0000","last_became_active":"2023-12-19T16:58:28.389259+0000","last_became_peered":"2023-12-19T16:58:28.389259+0000","last_unstale":"2023-12-19T16:58:29.380734+0000","last_undegraded":"2023-12-19T16:58:29.380734+0000","last_fullsized":"2023-12-19T16:58:29.380734+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:42:03.472728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.381091+0000","last_change":"2023-12-19T16:58:28.380203+0000","last_active":"2023-12-19T16:58:29.381091+0000","last_peered":"2023-12-19T16:58:29.381091+0000","last_clean":"2023-12-19T16:58:29.381091+0000","last_became_active":"2023-12-19T16:58:28.379839+0000","last_became_peered":"2023-12-19T16:58:28.379839+0000","last_unstale":"2023-12-19T16:58:29.381091+0000","last_undegraded":"2023-12-19T16:58:29.381091+0000","last_fullsized":"2023-12-19T16:58:29.381091+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:13:13.020419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.402645+0000","last_change":"2023-12-19T16:58:28.398265+0000","last_active":"2023-12-19T16:58:28.402645+0000","last_peered":"2023-12-19T16:58:28.402645+0000","last_clean":"2023-12-19T16:58:28.402645+0000","last_became_active":"2023-12-19T16:58:28.397983+0000","last_became_peered":"2023-12-19T16:58:28.397983+0000","last_unstale":"2023-12-19T16:58:28.402645+0000","last_undegraded":"2023-12-19T16:58:28.402645+0000","last_fullsized":"2023-12-19T16:58:28.402645+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:47.949356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101004+0000","last_change":"2023-12-19T16:58:28.402109+0000","last_active":"2023-12-19T16:58:30.101004+0000","last_peered":"2023-12-19T16:58:30.101004+0000","last_clean":"2023-12-19T16:58:30.101004+0000","last_became_active":"2023-12-19T16:58:28.401836+0000","last_became_peered":"2023-12-19T16:58:28.401836+0000","last_unstale":"2023-12-19T16:58:30.101004+0000","last_undegraded":"2023-12-19T16:58:30.101004+0000","last_fullsized":"2023-12-19T16:58:30.101004+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:13:13.020419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.402706+0000","last_change":"2023-12-19T16:58:28.399027+0000","last_active":"2023-12-19T16:58:28.402706+0000","last_peered":"2023-12-19T16:58:28.402706+0000","last_clean":"2023-12-19T16:58:28.402706+0000","last_became_active":"2023-12-19T16:58:28.397008+0000","last_became_peered":"2023-12-19T16:58:28.397008+0000","last_unstale":"2023-12-19T16:58:28.402706+0000","last_undegraded":"2023-12-19T16:58:28.402706+0000","last_fullsized":"2023-12-19T16:58:28.402706+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:54.099479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.385999+0000","last_change":"2023-12-19T16:58:28.383779+0000","last_active":"2023-12-19T16:58:28.385999+0000","last_peered":"2023-12-19T16:58:28.385999+0000","last_clean":"2023-12-19T16:58:28.385999+0000","last_became_active":"2023-12-19T16:58:28.383481+0000","last_became_peered":"2023-12-19T16:58:28.383481+0000","last_unstale":"2023-12-19T16:58:28.385999+0000","last_undegraded":"2023-12-19T16:58:28.385999+0000","last_fullsized":"2023-12-19T16:58:28.385999+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:41.887964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.408981+0000","last_change":"2023-12-19T16:58:28.407502+0000","last_active":"2023-12-19T16:58:28.408981+0000","last_peered":"2023-12-19T16:58:28.408981+0000","last_clean":"2023-12-19T16:58:28.408981+0000","last_became_active":"2023-12-19T16:58:28.406786+0000","last_became_peered":"2023-12-19T16:58:28.406786+0000","last_unstale":"2023-12-19T16:58:28.408981+0000","last_undegraded":"2023-12-19T16:58:28.408981+0000","last_fullsized":"2023-12-19T16:58:28.408981+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:38:53.835503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.377983+0000","last_change":"2023-12-19T16:58:28.392334+0000","last_active":"2023-12-19T16:58:29.377983+0000","last_peered":"2023-12-19T16:58:29.377983+0000","last_clean":"2023-12-19T16:58:29.377983+0000","last_became_active":"2023-12-19T16:58:28.392072+0000","last_became_peered":"2023-12-19T16:58:28.392072+0000","last_unstale":"2023-12-19T16:58:29.377983+0000","last_undegraded":"2023-12-19T16:58:29.377983+0000","last_fullsized":"2023-12-19T16:58:29.377983+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:18:17.210947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379471+0000","last_change":"2023-12-19T16:58:28.392678+0000","last_active":"2023-12-19T16:58:29.379471+0000","last_peered":"2023-12-19T16:58:29.379471+0000","last_clean":"2023-12-19T16:58:29.379471+0000","last_became_active":"2023-12-19T16:58:28.392408+0000","last_became_peered":"2023-12-19T16:58:28.392408+0000","last_unstale":"2023-12-19T16:58:29.379471+0000","last_undegraded":"2023-12-19T16:58:29.379471+0000","last_fullsized":"2023-12-19T16:58:29.379471+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:11.231928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380306+0000","last_change":"2023-12-19T16:58:28.390549+0000","last_active":"2023-12-19T16:58:29.380306+0000","last_peered":"2023-12-19T16:58:29.380306+0000","last_clean":"2023-12-19T16:58:29.380306+0000","last_became_active":"2023-12-19T16:58:28.389375+0000","last_became_peered":"2023-12-19T16:58:28.389375+0000","last_unstale":"2023-12-19T16:58:29.380306+0000","last_undegraded":"2023-12-19T16:58:29.380306+0000","last_fullsized":"2023-12-19T16:58:29.380306+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:44.512004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400907+0000","last_change":"2023-12-19T16:58:28.396171+0000","last_active":"2023-12-19T16:58:28.400907+0000","last_peered":"2023-12-19T16:58:28.400907+0000","last_clean":"2023-12-19T16:58:28.400907+0000","last_became_active":"2023-12-19T16:58:28.395228+0000","last_became_peered":"2023-12-19T16:58:28.395228+0000","last_unstale":"2023-12-19T16:58:28.400907+0000","last_undegraded":"2023-12-19T16:58:28.400907+0000","last_fullsized":"2023-12-19T16:58:28.400907+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:14:33.253382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.405243+0000","last_change":"2023-12-19T16:58:28.400541+0000","last_active":"2023-12-19T16:58:28.405243+0000","last_peered":"2023-12-19T16:58:28.405243+0000","last_clean":"2023-12-19T16:58:28.405243+0000","last_became_active":"2023-12-19T16:58:28.400276+0000","last_became_peered":"2023-12-19T16:58:28.400276+0000","last_unstale":"2023-12-19T16:58:28.405243+0000","last_undegraded":"2023-12-19T16:58:28.405243+0000","last_fullsized":"2023-12-19T16:58:28.405243+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:46:06.611832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.399049+0000","last_change":"2023-12-19T16:58:28.387006+0000","last_active":"2023-12-19T16:58:28.399049+0000","last_peered":"2023-12-19T16:58:28.399049+0000","last_clean":"2023-12-19T16:58:28.399049+0000","last_became_active":"2023-12-19T16:58:28.385859+0000","last_became_peered":"2023-12-19T16:58:28.385859+0000","last_unstale":"2023-12-19T16:58:28.399049+0000","last_undegraded":"2023-12-19T16:58:28.399049+0000","last_fullsized":"2023-12-19T16:58:28.399049+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:48:00.307695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400027+0000","last_change":"2023-12-19T16:58:28.391296+0000","last_active":"2023-12-19T16:58:28.400027+0000","last_peered":"2023-12-19T16:58:28.400027+0000","last_clean":"2023-12-19T16:58:28.400027+0000","last_became_active":"2023-12-19T16:58:28.390770+0000","last_became_peered":"2023-12-19T16:58:28.390770+0000","last_unstale":"2023-12-19T16:58:28.400027+0000","last_undegraded":"2023-12-19T16:58:28.400027+0000","last_fullsized":"2023-12-19T16:58:28.400027+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:19:03.557277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100821+0000","last_change":"2023-12-19T16:58:28.404204+0000","last_active":"2023-12-19T16:58:30.100821+0000","last_peered":"2023-12-19T16:58:30.100821+0000","last_clean":"2023-12-19T16:58:30.100821+0000","last_became_active":"2023-12-19T16:58:28.403611+0000","last_became_peered":"2023-12-19T16:58:28.403611+0000","last_unstale":"2023-12-19T16:58:30.100821+0000","last_undegraded":"2023-12-19T16:58:30.100821+0000","last_fullsized":"2023-12-19T16:58:30.100821+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:44.255710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.395536+0000","last_change":"2023-12-19T16:58:28.392341+0000","last_active":"2023-12-19T16:58:28.395536+0000","last_peered":"2023-12-19T16:58:28.395536+0000","last_clean":"2023-12-19T16:58:28.395536+0000","last_became_active":"2023-12-19T16:58:28.392072+0000","last_became_peered":"2023-12-19T16:58:28.392072+0000","last_unstale":"2023-12-19T16:58:28.395536+0000","last_undegraded":"2023-12-19T16:58:28.395536+0000","last_fullsized":"2023-12-19T16:58:28.395536+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:13:13.020419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.397827+0000","last_change":"2023-12-19T16:58:28.385578+0000","last_active":"2023-12-19T16:58:28.397827+0000","last_peered":"2023-12-19T16:58:28.397827+0000","last_clean":"2023-12-19T16:58:28.397827+0000","last_became_active":"2023-12-19T16:58:28.385284+0000","last_became_peered":"2023-12-19T16:58:28.385284+0000","last_unstale":"2023-12-19T16:58:28.397827+0000","last_undegraded":"2023-12-19T16:58:28.397827+0000","last_fullsized":"2023-12-19T16:58:28.397827+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:20:14.883721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101101+0000","last_change":"2023-12-19T16:58:28.398690+0000","last_active":"2023-12-19T16:58:30.101101+0000","last_peered":"2023-12-19T16:58:30.101101+0000","last_clean":"2023-12-19T16:58:30.101101+0000","last_became_active":"2023-12-19T16:58:28.398435+0000","last_became_peered":"2023-12-19T16:58:28.398435+0000","last_unstale":"2023-12-19T16:58:30.101101+0000","last_undegraded":"2023-12-19T16:58:30.101101+0000","last_fullsized":"2023-12-19T16:58:30.101101+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:05:44.910475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.401399+0000","last_change":"2023-12-19T16:58:28.399416+0000","last_active":"2023-12-19T16:58:28.401399+0000","last_peered":"2023-12-19T16:58:28.401399+0000","last_clean":"2023-12-19T16:58:28.401399+0000","last_became_active":"2023-12-19T16:58:28.399103+0000","last_became_peered":"2023-12-19T16:58:28.399103+0000","last_unstale":"2023-12-19T16:58:28.401399+0000","last_undegraded":"2023-12-19T16:58:28.401399+0000","last_fullsized":"2023-12-19T16:58:28.401399+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:37:17.302378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379680+0000","last_change":"2023-12-19T16:58:28.397705+0000","last_active":"2023-12-19T16:58:29.379680+0000","last_peered":"2023-12-19T16:58:29.379680+0000","last_clean":"2023-12-19T16:58:29.379680+0000","last_became_active":"2023-12-19T16:58:28.397399+0000","last_became_peered":"2023-12-19T16:58:28.397399+0000","last_unstale":"2023-12-19T16:58:29.379680+0000","last_undegraded":"2023-12-19T16:58:29.379680+0000","last_fullsized":"2023-12-19T16:58:29.379680+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:57:04.690547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.395031+0000","last_change":"2023-12-19T16:58:28.389935+0000","last_active":"2023-12-19T16:58:28.395031+0000","last_peered":"2023-12-19T16:58:28.395031+0000","last_clean":"2023-12-19T16:58:28.395031+0000","last_became_active":"2023-12-19T16:58:28.389656+0000","last_became_peered":"2023-12-19T16:58:28.389656+0000","last_unstale":"2023-12-19T16:58:28.395031+0000","last_undegraded":"2023-12-19T16:58:28.395031+0000","last_fullsized":"2023-12-19T16:58:28.395031+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:24:38.193238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100886+0000","last_change":"2023-12-19T16:58:28.406975+0000","last_active":"2023-12-19T16:58:30.100886+0000","last_peered":"2023-12-19T16:58:30.100886+0000","last_clean":"2023-12-19T16:58:30.100886+0000","last_became_active":"2023-12-19T16:58:28.406504+0000","last_became_peered":"2023-12-19T16:58:28.406504+0000","last_unstale":"2023-12-19T16:58:30.100886+0000","last_undegraded":"2023-12-19T16:58:30.100886+0000","last_fullsized":"2023-12-19T16:58:30.100886+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:11:04.726712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101118+0000","last_change":"2023-12-19T16:58:28.404040+0000","last_active":"2023-12-19T16:58:30.101118+0000","last_peered":"2023-12-19T16:58:30.101118+0000","last_clean":"2023-12-19T16:58:30.101118+0000","last_became_active":"2023-12-19T16:58:28.403252+0000","last_became_peered":"2023-12-19T16:58:28.403252+0000","last_unstale":"2023-12-19T16:58:30.101118+0000","last_undegraded":"2023-12-19T16:58:30.101118+0000","last_fullsized":"2023-12-19T16:58:30.101118+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:37.217016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379634+0000","last_change":"2023-12-19T16:58:28.379743+0000","last_active":"2023-12-19T16:58:29.379634+0000","last_peered":"2023-12-19T16:58:29.379634+0000","last_clean":"2023-12-19T16:58:29.379634+0000","last_became_active":"2023-12-19T16:58:28.379081+0000","last_became_peered":"2023-12-19T16:58:28.379081+0000","last_unstale":"2023-12-19T16:58:29.379634+0000","last_undegraded":"2023-12-19T16:58:29.379634+0000","last_fullsized":"2023-12-19T16:58:29.379634+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:36.607132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.404139+0000","last_change":"2023-12-19T16:58:28.402332+0000","last_active":"2023-12-19T16:58:28.404139+0000","last_peered":"2023-12-19T16:58:28.404139+0000","last_clean":"2023-12-19T16:58:28.404139+0000","last_became_active":"2023-12-19T16:58:28.402016+0000","last_became_peered":"2023-12-19T16:58:28.402016+0000","last_unstale":"2023-12-19T16:58:28.404139+0000","last_undegraded":"2023-12-19T16:58:28.404139+0000","last_fullsized":"2023-12-19T16:58:28.404139+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:56.095278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379682+0000","last_change":"2023-12-19T16:58:28.391791+0000","last_active":"2023-12-19T16:58:29.379682+0000","last_peered":"2023-12-19T16:58:29.379682+0000","last_clean":"2023-12-19T16:58:29.379682+0000","last_became_active":"2023-12-19T16:58:28.391488+0000","last_became_peered":"2023-12-19T16:58:28.391488+0000","last_unstale":"2023-12-19T16:58:29.379682+0000","last_undegraded":"2023-12-19T16:58:29.379682+0000","last_fullsized":"2023-12-19T16:58:29.379682+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:42:03.472728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"106'4","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.404300+0000","last_change":"2023-12-19T16:58:27.980325+0000","last_active":"2023-12-19T16:58:28.404300+0000","last_peered":"2023-12-19T16:58:28.404300+0000","last_clean":"2023-12-19T16:58:28.404300+0000","last_became_active":"2023-12-19T16:58:27.980009+0000","last_became_peered":"2023-12-19T16:58:27.980009+0000","last_unstale":"2023-12-19T16:58:28.404300+0000","last_undegraded":"2023-12-19T16:58:28.404300+0000","last_fullsized":"2023-12-19T16:58:28.404300+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:11.988639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.398858+0000","last_change":"2023-12-19T16:58:28.395189+0000","last_active":"2023-12-19T16:58:28.398858+0000","last_peered":"2023-12-19T16:58:28.398858+0000","last_clean":"2023-12-19T16:58:28.398858+0000","last_became_active":"2023-12-19T16:58:28.394538+0000","last_became_peered":"2023-12-19T16:58:28.394538+0000","last_unstale":"2023-12-19T16:58:28.398858+0000","last_undegraded":"2023-12-19T16:58:28.398858+0000","last_fullsized":"2023-12-19T16:58:28.398858+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:43:46.183766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379966+0000","last_change":"2023-12-19T16:58:27.982211+0000","last_active":"2023-12-19T16:58:29.379966+0000","last_peered":"2023-12-19T16:58:29.379966+0000","last_clean":"2023-12-19T16:58:29.379966+0000","last_became_active":"2023-12-19T16:58:27.981947+0000","last_became_peered":"2023-12-19T16:58:27.981947+0000","last_unstale":"2023-12-19T16:58:29.379966+0000","last_undegraded":"2023-12-19T16:58:29.379966+0000","last_fullsized":"2023-12-19T16:58:29.379966+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:12:09.054363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.402749+0000","last_change":"2023-12-19T16:58:28.400320+0000","last_active":"2023-12-19T16:58:28.402749+0000","last_peered":"2023-12-19T16:58:28.402749+0000","last_clean":"2023-12-19T16:58:28.402749+0000","last_became_active":"2023-12-19T16:58:28.398806+0000","last_became_peered":"2023-12-19T16:58:28.398806+0000","last_unstale":"2023-12-19T16:58:28.402749+0000","last_undegraded":"2023-12-19T16:58:28.402749+0000","last_fullsized":"2023-12-19T16:58:28.402749+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:29.117499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.401413+0000","last_change":"2023-12-19T16:58:27.981164+0000","last_active":"2023-12-19T16:58:28.401413+0000","last_peered":"2023-12-19T16:58:28.401413+0000","last_clean":"2023-12-19T16:58:28.401413+0000","last_became_active":"2023-12-19T16:58:27.979348+0000","last_became_peered":"2023-12-19T16:58:27.979348+0000","last_unstale":"2023-12-19T16:58:28.401413+0000","last_undegraded":"2023-12-19T16:58:28.401413+0000","last_fullsized":"2023-12-19T16:58:28.401413+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:35.349198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.398113+0000","last_change":"2023-12-19T16:58:28.391133+0000","last_active":"2023-12-19T16:58:28.398113+0000","last_peered":"2023-12-19T16:58:28.398113+0000","last_clean":"2023-12-19T16:58:28.398113+0000","last_became_active":"2023-12-19T16:58:28.390866+0000","last_became_peered":"2023-12-19T16:58:28.390866+0000","last_unstale":"2023-12-19T16:58:28.398113+0000","last_undegraded":"2023-12-19T16:58:28.398113+0000","last_fullsized":"2023-12-19T16:58:28.398113+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:24.692744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.402413+0000","last_change":"2023-12-19T16:58:27.978882+0000","last_active":"2023-12-19T16:58:28.402413+0000","last_peered":"2023-12-19T16:58:28.402413+0000","last_clean":"2023-12-19T16:58:28.402413+0000","last_became_active":"2023-12-19T16:58:27.978520+0000","last_became_peered":"2023-12-19T16:58:27.978520+0000","last_unstale":"2023-12-19T16:58:28.402413+0000","last_undegraded":"2023-12-19T16:58:28.402413+0000","last_fullsized":"2023-12-19T16:58:28.402413+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:36:39.412312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100964+0000","last_change":"2023-12-19T16:58:28.403876+0000","last_active":"2023-12-19T16:58:30.100964+0000","last_peered":"2023-12-19T16:58:30.100964+0000","last_clean":"2023-12-19T16:58:30.100964+0000","last_became_active":"2023-12-19T16:58:28.402906+0000","last_became_peered":"2023-12-19T16:58:28.402906+0000","last_unstale":"2023-12-19T16:58:30.100964+0000","last_undegraded":"2023-12-19T16:58:30.100964+0000","last_fullsized":"2023-12-19T16:58:30.100964+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:57:04.690547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380106+0000","last_change":"2023-12-19T16:58:27.987378+0000","last_active":"2023-12-19T16:58:29.380106+0000","last_peered":"2023-12-19T16:58:29.380106+0000","last_clean":"2023-12-19T16:58:29.380106+0000","last_became_active":"2023-12-19T16:58:27.986991+0000","last_became_peered":"2023-12-19T16:58:27.986991+0000","last_unstale":"2023-12-19T16:58:29.380106+0000","last_undegraded":"2023-12-19T16:58:29.380106+0000","last_fullsized":"2023-12-19T16:58:29.380106+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:36:27.751276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379996+0000","last_change":"2023-12-19T16:58:28.388593+0000","last_active":"2023-12-19T16:58:29.379996+0000","last_peered":"2023-12-19T16:58:29.379996+0000","last_clean":"2023-12-19T16:58:29.379996+0000","last_became_active":"2023-12-19T16:58:28.388331+0000","last_became_peered":"2023-12-19T16:58:28.388331+0000","last_unstale":"2023-12-19T16:58:29.379996+0000","last_undegraded":"2023-12-19T16:58:29.379996+0000","last_fullsized":"2023-12-19T16:58:29.379996+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:05:44.910475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380226+0000","last_change":"2023-12-19T16:58:27.982923+0000","last_active":"2023-12-19T16:58:29.380226+0000","last_peered":"2023-12-19T16:58:29.380226+0000","last_clean":"2023-12-19T16:58:29.380226+0000","last_became_active":"2023-12-19T16:58:27.982618+0000","last_became_peered":"2023-12-19T16:58:27.982618+0000","last_unstale":"2023-12-19T16:58:29.380226+0000","last_undegraded":"2023-12-19T16:58:29.380226+0000","last_fullsized":"2023-12-19T16:58:29.380226+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:24.714136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400018+0000","last_change":"2023-12-19T16:58:28.392878+0000","last_active":"2023-12-19T16:58:28.400018+0000","last_peered":"2023-12-19T16:58:28.400018+0000","last_clean":"2023-12-19T16:58:28.400018+0000","last_became_active":"2023-12-19T16:58:28.392131+0000","last_became_peered":"2023-12-19T16:58:28.392131+0000","last_unstale":"2023-12-19T16:58:28.400018+0000","last_undegraded":"2023-12-19T16:58:28.400018+0000","last_fullsized":"2023-12-19T16:58:28.400018+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:24.245385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.369574+0000","last_change":"2023-12-19T16:58:27.982469+0000","last_active":"2023-12-19T16:58:28.369574+0000","last_peered":"2023-12-19T16:58:28.369574+0000","last_clean":"2023-12-19T16:58:28.369574+0000","last_became_active":"2023-12-19T16:58:27.982219+0000","last_became_peered":"2023-12-19T16:58:27.982219+0000","last_unstale":"2023-12-19T16:58:28.369574+0000","last_undegraded":"2023-12-19T16:58:28.369574+0000","last_fullsized":"2023-12-19T16:58:28.369574+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:16:49.807412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.377982+0000","last_change":"2023-12-19T16:58:28.383391+0000","last_active":"2023-12-19T16:58:29.377982+0000","last_peered":"2023-12-19T16:58:29.377982+0000","last_clean":"2023-12-19T16:58:29.377982+0000","last_became_active":"2023-12-19T16:58:28.383145+0000","last_became_peered":"2023-12-19T16:58:28.383145+0000","last_unstale":"2023-12-19T16:58:29.377982+0000","last_undegraded":"2023-12-19T16:58:29.377982+0000","last_fullsized":"2023-12-19T16:58:29.377982+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:35.704685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100770+0000","last_change":"2023-12-19T16:58:27.986696+0000","last_active":"2023-12-19T16:58:30.100770+0000","last_peered":"2023-12-19T16:58:30.100770+0000","last_clean":"2023-12-19T16:58:30.100770+0000","last_became_active":"2023-12-19T16:58:27.985791+0000","last_became_peered":"2023-12-19T16:58:27.985791+0000","last_unstale":"2023-12-19T16:58:30.100770+0000","last_undegraded":"2023-12-19T16:58:30.100770+0000","last_fullsized":"2023-12-19T16:58:30.100770+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:08.124546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.378348+0000","last_change":"2023-12-19T16:58:28.387976+0000","last_active":"2023-12-19T16:58:29.378348+0000","last_peered":"2023-12-19T16:58:29.378348+0000","last_clean":"2023-12-19T16:58:29.378348+0000","last_became_active":"2023-12-19T16:58:28.387663+0000","last_became_peered":"2023-12-19T16:58:28.387663+0000","last_unstale":"2023-12-19T16:58:29.378348+0000","last_undegraded":"2023-12-19T16:58:29.378348+0000","last_fullsized":"2023-12-19T16:58:29.378348+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:01:07.088699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.380765+0000","last_change":"2023-12-19T16:58:27.983171+0000","last_active":"2023-12-19T16:58:28.380765+0000","last_peered":"2023-12-19T16:58:28.380765+0000","last_clean":"2023-12-19T16:58:28.380765+0000","last_became_active":"2023-12-19T16:58:27.982909+0000","last_became_peered":"2023-12-19T16:58:27.982909+0000","last_unstale":"2023-12-19T16:58:28.380765+0000","last_undegraded":"2023-12-19T16:58:28.380765+0000","last_fullsized":"2023-12-19T16:58:28.380765+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:27:44.415340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.381111+0000","last_change":"2023-12-19T16:58:28.388446+0000","last_active":"2023-12-19T16:58:29.381111+0000","last_peered":"2023-12-19T16:58:29.381111+0000","last_clean":"2023-12-19T16:58:29.381111+0000","last_became_active":"2023-12-19T16:58:28.388129+0000","last_became_peered":"2023-12-19T16:58:28.388129+0000","last_unstale":"2023-12-19T16:58:29.381111+0000","last_undegraded":"2023-12-19T16:58:29.381111+0000","last_fullsized":"2023-12-19T16:58:29.381111+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:36.607132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.381455+0000","last_change":"2023-12-19T16:58:27.982711+0000","last_active":"2023-12-19T16:58:28.381455+0000","last_peered":"2023-12-19T16:58:28.381455+0000","last_clean":"2023-12-19T16:58:28.381455+0000","last_became_active":"2023-12-19T16:58:27.982437+0000","last_became_peered":"2023-12-19T16:58:27.982437+0000","last_unstale":"2023-12-19T16:58:28.381455+0000","last_undegraded":"2023-12-19T16:58:28.381455+0000","last_fullsized":"2023-12-19T16:58:28.381455+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:24.714136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.401057+0000","last_change":"2023-12-19T16:58:28.394629+0000","last_active":"2023-12-19T16:58:28.401057+0000","last_peered":"2023-12-19T16:58:28.401057+0000","last_clean":"2023-12-19T16:58:28.401057+0000","last_became_active":"2023-12-19T16:58:28.393543+0000","last_became_peered":"2023-12-19T16:58:28.393543+0000","last_unstale":"2023-12-19T16:58:28.401057+0000","last_undegraded":"2023-12-19T16:58:28.401057+0000","last_fullsized":"2023-12-19T16:58:28.401057+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:24:01.098046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"106'3","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400012+0000","last_change":"2023-12-19T16:58:27.984876+0000","last_active":"2023-12-19T16:58:28.400012+0000","last_peered":"2023-12-19T16:58:28.400012+0000","last_clean":"2023-12-19T16:58:28.400012+0000","last_became_active":"2023-12-19T16:58:27.984601+0000","last_became_peered":"2023-12-19T16:58:27.984601+0000","last_unstale":"2023-12-19T16:58:28.400012+0000","last_undegraded":"2023-12-19T16:58:28.400012+0000","last_fullsized":"2023-12-19T16:58:28.400012+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:17:48.685656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379757+0000","last_change":"2023-12-19T16:58:28.387850+0000","last_active":"2023-12-19T16:58:29.379757+0000","last_peered":"2023-12-19T16:58:29.379757+0000","last_clean":"2023-12-19T16:58:29.379757+0000","last_became_active":"2023-12-19T16:58:28.387571+0000","last_became_peered":"2023-12-19T16:58:28.387571+0000","last_unstale":"2023-12-19T16:58:29.379757+0000","last_undegraded":"2023-12-19T16:58:29.379757+0000","last_fullsized":"2023-12-19T16:58:29.379757+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:47.949356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"106'6","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380926+0000","last_change":"2023-12-19T16:58:27.983185+0000","last_active":"2023-12-19T16:58:29.380926+0000","last_peered":"2023-12-19T16:58:29.380926+0000","last_clean":"2023-12-19T16:58:29.380926+0000","last_became_active":"2023-12-19T16:58:27.981664+0000","last_became_peered":"2023-12-19T16:58:27.981664+0000","last_unstale":"2023-12-19T16:58:29.380926+0000","last_undegraded":"2023-12-19T16:58:29.380926+0000","last_fullsized":"2023-12-19T16:58:29.380926+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:11.988639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.393082+0000","last_change":"2023-12-19T16:58:28.382566+0000","last_active":"2023-12-19T16:58:28.393082+0000","last_peered":"2023-12-19T16:58:28.393082+0000","last_clean":"2023-12-19T16:58:28.393082+0000","last_became_active":"2023-12-19T16:58:28.382305+0000","last_became_peered":"2023-12-19T16:58:28.382305+0000","last_unstale":"2023-12-19T16:58:28.393082+0000","last_undegraded":"2023-12-19T16:58:28.393082+0000","last_fullsized":"2023-12-19T16:58:28.393082+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:47:50.538756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100986+0000","last_change":"2023-12-19T16:58:27.982846+0000","last_active":"2023-12-19T16:58:30.100986+0000","last_peered":"2023-12-19T16:58:30.100986+0000","last_clean":"2023-12-19T16:58:30.100986+0000","last_became_active":"2023-12-19T16:58:27.982549+0000","last_became_peered":"2023-12-19T16:58:27.982549+0000","last_unstale":"2023-12-19T16:58:30.100986+0000","last_undegraded":"2023-12-19T16:58:30.100986+0000","last_fullsized":"2023-12-19T16:58:30.100986+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:36:27.751276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.400877+0000","last_change":"2023-12-19T16:58:28.397981+0000","last_active":"2023-12-19T16:58:28.400877+0000","last_peered":"2023-12-19T16:58:28.400877+0000","last_clean":"2023-12-19T16:58:28.400877+0000","last_became_active":"2023-12-19T16:58:28.397696+0000","last_became_peered":"2023-12-19T16:58:28.397696+0000","last_unstale":"2023-12-19T16:58:28.400877+0000","last_undegraded":"2023-12-19T16:58:28.400877+0000","last_fullsized":"2023-12-19T16:58:28.400877+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:34:28.248975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.378157+0000","last_change":"2023-12-19T16:58:27.987869+0000","last_active":"2023-12-19T16:58:29.378157+0000","last_peered":"2023-12-19T16:58:29.378157+0000","last_clean":"2023-12-19T16:58:29.378157+0000","last_became_active":"2023-12-19T16:58:27.987525+0000","last_became_peered":"2023-12-19T16:58:27.987525+0000","last_unstale":"2023-12-19T16:58:29.378157+0000","last_undegraded":"2023-12-19T16:58:29.378157+0000","last_fullsized":"2023-12-19T16:58:29.378157+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:25:23.743802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100708+0000","last_change":"2023-12-19T16:58:28.391359+0000","last_active":"2023-12-19T16:58:30.100708+0000","last_peered":"2023-12-19T16:58:30.100708+0000","last_clean":"2023-12-19T16:58:30.100708+0000","last_became_active":"2023-12-19T16:58:28.391056+0000","last_became_peered":"2023-12-19T16:58:28.391056+0000","last_unstale":"2023-12-19T16:58:30.100708+0000","last_undegraded":"2023-12-19T16:58:30.100708+0000","last_fullsized":"2023-12-19T16:58:30.100708+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:41.622888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380717+0000","last_change":"2023-12-19T16:58:27.984801+0000","last_active":"2023-12-19T16:58:29.380717+0000","last_peered":"2023-12-19T16:58:29.380717+0000","last_clean":"2023-12-19T16:58:29.380717+0000","last_became_active":"2023-12-19T16:58:27.984513+0000","last_became_peered":"2023-12-19T16:58:27.984513+0000","last_unstale":"2023-12-19T16:58:29.380717+0000","last_undegraded":"2023-12-19T16:58:29.380717+0000","last_fullsized":"2023-12-19T16:58:29.380717+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:48:20.013396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.100651+0000","last_change":"2023-12-19T16:58:28.396661+0000","last_active":"2023-12-19T16:58:30.100651+0000","last_peered":"2023-12-19T16:58:30.100651+0000","last_clean":"2023-12-19T16:58:30.100651+0000","last_became_active":"2023-12-19T16:58:28.396352+0000","last_became_peered":"2023-12-19T16:58:28.396352+0000","last_unstale":"2023-12-19T16:58:30.100651+0000","last_undegraded":"2023-12-19T16:58:30.100651+0000","last_fullsized":"2023-12-19T16:58:30.100651+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:44.512004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"106'8","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.404723+0000","last_change":"2023-12-19T16:58:27.986085+0000","last_active":"2023-12-19T16:58:28.404723+0000","last_peered":"2023-12-19T16:58:28.404723+0000","last_clean":"2023-12-19T16:58:28.404723+0000","last_became_active":"2023-12-19T16:58:27.985783+0000","last_became_peered":"2023-12-19T16:58:27.985783+0000","last_unstale":"2023-12-19T16:58:28.404723+0000","last_undegraded":"2023-12-19T16:58:28.404723+0000","last_fullsized":"2023-12-19T16:58:28.404723+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:43.497884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.393588+0000","last_change":"2023-12-19T16:58:28.388033+0000","last_active":"2023-12-19T16:58:28.393588+0000","last_peered":"2023-12-19T16:58:28.393588+0000","last_clean":"2023-12-19T16:58:28.393588+0000","last_became_active":"2023-12-19T16:58:28.387744+0000","last_became_peered":"2023-12-19T16:58:28.387744+0000","last_unstale":"2023-12-19T16:58:28.393588+0000","last_undegraded":"2023-12-19T16:58:28.393588+0000","last_fullsized":"2023-12-19T16:58:28.393588+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:25:24.746600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.395735+0000","last_change":"2023-12-19T16:58:27.986462+0000","last_active":"2023-12-19T16:58:28.395735+0000","last_peered":"2023-12-19T16:58:28.395735+0000","last_clean":"2023-12-19T16:58:28.395735+0000","last_became_active":"2023-12-19T16:58:27.985925+0000","last_became_peered":"2023-12-19T16:58:27.985925+0000","last_unstale":"2023-12-19T16:58:28.395735+0000","last_undegraded":"2023-12-19T16:58:28.395735+0000","last_fullsized":"2023-12-19T16:58:28.395735+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:04:06.694458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.395808+0000","last_change":"2023-12-19T16:58:28.390457+0000","last_active":"2023-12-19T16:58:28.395808+0000","last_peered":"2023-12-19T16:58:28.395808+0000","last_clean":"2023-12-19T16:58:28.395808+0000","last_became_active":"2023-12-19T16:58:28.390098+0000","last_became_peered":"2023-12-19T16:58:28.390098+0000","last_unstale":"2023-12-19T16:58:28.395808+0000","last_undegraded":"2023-12-19T16:58:28.395808+0000","last_fullsized":"2023-12-19T16:58:28.395808+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:09:54.894372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.397397+0000","last_change":"2023-12-19T16:58:27.984105+0000","last_active":"2023-12-19T16:58:28.397397+0000","last_peered":"2023-12-19T16:58:28.397397+0000","last_clean":"2023-12-19T16:58:28.397397+0000","last_became_active":"2023-12-19T16:58:27.983826+0000","last_became_peered":"2023-12-19T16:58:27.983826+0000","last_unstale":"2023-12-19T16:58:28.397397+0000","last_undegraded":"2023-12-19T16:58:28.397397+0000","last_fullsized":"2023-12-19T16:58:28.397397+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:28:17.238941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379794+0000","last_change":"2023-12-19T16:58:28.396774+0000","last_active":"2023-12-19T16:58:29.379794+0000","last_peered":"2023-12-19T16:58:29.379794+0000","last_clean":"2023-12-19T16:58:29.379794+0000","last_became_active":"2023-12-19T16:58:28.396493+0000","last_became_peered":"2023-12-19T16:58:28.396493+0000","last_unstale":"2023-12-19T16:58:29.379794+0000","last_undegraded":"2023-12-19T16:58:29.379794+0000","last_fullsized":"2023-12-19T16:58:29.379794+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:13:13.020419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.370266+0000","last_change":"2023-12-19T16:58:27.982910+0000","last_active":"2023-12-19T16:58:28.370266+0000","last_peered":"2023-12-19T16:58:28.370266+0000","last_clean":"2023-12-19T16:58:28.370266+0000","last_became_active":"2023-12-19T16:58:27.982166+0000","last_became_peered":"2023-12-19T16:58:27.982166+0000","last_unstale":"2023-12-19T16:58:28.370266+0000","last_undegraded":"2023-12-19T16:58:28.370266+0000","last_fullsized":"2023-12-19T16:58:28.370266+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:51:55.444146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.409017+0000","last_change":"2023-12-19T16:58:28.407661+0000","last_active":"2023-12-19T16:58:28.409017+0000","last_peered":"2023-12-19T16:58:28.409017+0000","last_clean":"2023-12-19T16:58:28.409017+0000","last_became_active":"2023-12-19T16:58:28.407216+0000","last_became_peered":"2023-12-19T16:58:28.407216+0000","last_unstale":"2023-12-19T16:58:28.409017+0000","last_undegraded":"2023-12-19T16:58:28.409017+0000","last_fullsized":"2023-12-19T16:58:28.409017+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:20:08.229107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.378337+0000","last_change":"2023-12-19T16:58:27.979783+0000","last_active":"2023-12-19T16:58:29.378337+0000","last_peered":"2023-12-19T16:58:29.378337+0000","last_clean":"2023-12-19T16:58:29.378337+0000","last_became_active":"2023-12-19T16:58:27.979474+0000","last_became_peered":"2023-12-19T16:58:27.979474+0000","last_unstale":"2023-12-19T16:58:29.378337+0000","last_undegraded":"2023-12-19T16:58:29.378337+0000","last_fullsized":"2023-12-19T16:58:29.378337+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:04.086945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380784+0000","last_change":"2023-12-19T16:58:28.396048+0000","last_active":"2023-12-19T16:58:29.380784+0000","last_peered":"2023-12-19T16:58:29.380784+0000","last_clean":"2023-12-19T16:58:29.380784+0000","last_became_active":"2023-12-19T16:58:28.395726+0000","last_became_peered":"2023-12-19T16:58:28.395726+0000","last_unstale":"2023-12-19T16:58:29.380784+0000","last_undegraded":"2023-12-19T16:58:29.380784+0000","last_fullsized":"2023-12-19T16:58:29.380784+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:11.231928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.380446+0000","last_change":"2023-12-19T16:58:27.977554+0000","last_active":"2023-12-19T16:58:28.380446+0000","last_peered":"2023-12-19T16:58:28.380446+0000","last_clean":"2023-12-19T16:58:28.380446+0000","last_became_active":"2023-12-19T16:58:27.977121+0000","last_became_peered":"2023-12-19T16:58:27.977121+0000","last_unstale":"2023-12-19T16:58:28.380446+0000","last_undegraded":"2023-12-19T16:58:28.380446+0000","last_fullsized":"2023-12-19T16:58:28.380446+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:38.524842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.398928+0000","last_change":"2023-12-19T16:58:28.390022+0000","last_active":"2023-12-19T16:58:28.398928+0000","last_peered":"2023-12-19T16:58:28.398928+0000","last_clean":"2023-12-19T16:58:28.398928+0000","last_became_active":"2023-12-19T16:58:28.389744+0000","last_became_peered":"2023-12-19T16:58:28.389744+0000","last_unstale":"2023-12-19T16:58:28.398928+0000","last_undegraded":"2023-12-19T16:58:28.398928+0000","last_fullsized":"2023-12-19T16:58:28.398928+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:45.967982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.380292+0000","last_change":"2023-12-19T16:58:27.983065+0000","last_active":"2023-12-19T16:58:28.380292+0000","last_peered":"2023-12-19T16:58:28.380292+0000","last_clean":"2023-12-19T16:58:28.380292+0000","last_became_active":"2023-12-19T16:58:27.982805+0000","last_became_peered":"2023-12-19T16:58:27.982805+0000","last_unstale":"2023-12-19T16:58:28.380292+0000","last_undegraded":"2023-12-19T16:58:28.380292+0000","last_fullsized":"2023-12-19T16:58:28.380292+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:41:24.316368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380714+0000","last_change":"2023-12-19T16:58:28.384439+0000","last_active":"2023-12-19T16:58:29.380714+0000","last_peered":"2023-12-19T16:58:29.380714+0000","last_clean":"2023-12-19T16:58:29.380714+0000","last_became_active":"2023-12-19T16:58:28.383397+0000","last_became_peered":"2023-12-19T16:58:28.383397+0000","last_unstale":"2023-12-19T16:58:29.380714+0000","last_undegraded":"2023-12-19T16:58:29.380714+0000","last_fullsized":"2023-12-19T16:58:29.380714+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:57:04.690547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.377942+0000","last_change":"2023-12-19T16:58:27.980579+0000","last_active":"2023-12-19T16:58:29.377942+0000","last_peered":"2023-12-19T16:58:29.377942+0000","last_clean":"2023-12-19T16:58:29.377942+0000","last_became_active":"2023-12-19T16:58:27.980324+0000","last_became_peered":"2023-12-19T16:58:27.980324+0000","last_unstale":"2023-12-19T16:58:29.377942+0000","last_undegraded":"2023-12-19T16:58:29.377942+0000","last_fullsized":"2023-12-19T16:58:29.377942+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:40.885166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379846+0000","last_change":"2023-12-19T16:58:28.381233+0000","last_active":"2023-12-19T16:58:29.379846+0000","last_peered":"2023-12-19T16:58:29.379846+0000","last_clean":"2023-12-19T16:58:29.379846+0000","last_became_active":"2023-12-19T16:58:28.380924+0000","last_became_peered":"2023-12-19T16:58:28.380924+0000","last_unstale":"2023-12-19T16:58:29.379846+0000","last_undegraded":"2023-12-19T16:58:29.379846+0000","last_fullsized":"2023-12-19T16:58:29.379846+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:44.255710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101099+0000","last_change":"2023-12-19T16:58:27.988714+0000","last_active":"2023-12-19T16:58:30.101099+0000","last_peered":"2023-12-19T16:58:30.101099+0000","last_clean":"2023-12-19T16:58:30.101099+0000","last_became_active":"2023-12-19T16:58:27.988432+0000","last_became_peered":"2023-12-19T16:58:27.988432+0000","last_unstale":"2023-12-19T16:58:30.101099+0000","last_undegraded":"2023-12-19T16:58:30.101099+0000","last_fullsized":"2023-12-19T16:58:30.101099+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:12:09.054363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.399907+0000","last_change":"2023-12-19T16:58:28.392218+0000","last_active":"2023-12-19T16:58:28.399907+0000","last_peered":"2023-12-19T16:58:28.399907+0000","last_clean":"2023-12-19T16:58:28.399907+0000","last_became_active":"2023-12-19T16:58:28.391792+0000","last_became_peered":"2023-12-19T16:58:28.391792+0000","last_unstale":"2023-12-19T16:58:28.399907+0000","last_undegraded":"2023-12-19T16:58:28.399907+0000","last_fullsized":"2023-12-19T16:58:28.399907+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:33:04.099843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101111+0000","last_change":"2023-12-19T16:58:27.984423+0000","last_active":"2023-12-19T16:58:30.101111+0000","last_peered":"2023-12-19T16:58:30.101111+0000","last_clean":"2023-12-19T16:58:30.101111+0000","last_became_active":"2023-12-19T16:58:27.984122+0000","last_became_peered":"2023-12-19T16:58:27.984122+0000","last_unstale":"2023-12-19T16:58:30.101111+0000","last_undegraded":"2023-12-19T16:58:30.101111+0000","last_fullsized":"2023-12-19T16:58:30.101111+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:24.714136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.402774+0000","last_change":"2023-12-19T16:58:28.399913+0000","last_active":"2023-12-19T16:58:28.402774+0000","last_peered":"2023-12-19T16:58:28.402774+0000","last_clean":"2023-12-19T16:58:28.402774+0000","last_became_active":"2023-12-19T16:58:28.399490+0000","last_became_peered":"2023-12-19T16:58:28.399490+0000","last_unstale":"2023-12-19T16:58:28.402774+0000","last_undegraded":"2023-12-19T16:58:28.402774+0000","last_fullsized":"2023-12-19T16:58:28.402774+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:42:03.472728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.381945+0000","last_change":"2023-12-19T16:58:27.979794+0000","last_active":"2023-12-19T16:58:28.381945+0000","last_peered":"2023-12-19T16:58:28.381945+0000","last_clean":"2023-12-19T16:58:28.381945+0000","last_became_active":"2023-12-19T16:58:27.979446+0000","last_became_peered":"2023-12-19T16:58:27.979446+0000","last_unstale":"2023-12-19T16:58:28.381945+0000","last_undegraded":"2023-12-19T16:58:28.381945+0000","last_fullsized":"2023-12-19T16:58:28.381945+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:48:20.013396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379875+0000","last_change":"2023-12-19T16:58:28.384646+0000","last_active":"2023-12-19T16:58:29.379875+0000","last_peered":"2023-12-19T16:58:29.379875+0000","last_clean":"2023-12-19T16:58:29.379875+0000","last_became_active":"2023-12-19T16:58:28.384382+0000","last_became_peered":"2023-12-19T16:58:28.384382+0000","last_unstale":"2023-12-19T16:58:29.379875+0000","last_undegraded":"2023-12-19T16:58:29.379875+0000","last_fullsized":"2023-12-19T16:58:29.379875+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:43:46.183766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.381858+0000","last_change":"2023-12-19T16:58:27.978599+0000","last_active":"2023-12-19T16:58:28.381858+0000","last_peered":"2023-12-19T16:58:28.381858+0000","last_clean":"2023-12-19T16:58:28.381858+0000","last_became_active":"2023-12-19T16:58:27.978232+0000","last_became_peered":"2023-12-19T16:58:27.978232+0000","last_unstale":"2023-12-19T16:58:28.381858+0000","last_undegraded":"2023-12-19T16:58:28.381858+0000","last_fullsized":"2023-12-19T16:58:28.381858+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T16:59:51.105534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380871+0000","last_change":"2023-12-19T16:58:28.395430+0000","last_active":"2023-12-19T16:58:29.380871+0000","last_peered":"2023-12-19T16:58:29.380871+0000","last_clean":"2023-12-19T16:58:29.380871+0000","last_became_active":"2023-12-19T16:58:28.395103+0000","last_became_peered":"2023-12-19T16:58:28.395103+0000","last_unstale":"2023-12-19T16:58:29.380871+0000","last_undegraded":"2023-12-19T16:58:29.380871+0000","last_fullsized":"2023-12-19T16:58:29.380871+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:37.217016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.369795+0000","last_change":"2023-12-19T16:58:27.989285+0000","last_active":"2023-12-19T16:58:28.369795+0000","last_peered":"2023-12-19T16:58:28.369795+0000","last_clean":"2023-12-19T16:58:28.369795+0000","last_became_active":"2023-12-19T16:58:27.988986+0000","last_became_peered":"2023-12-19T16:58:27.988986+0000","last_unstale":"2023-12-19T16:58:28.369795+0000","last_undegraded":"2023-12-19T16:58:28.369795+0000","last_fullsized":"2023-12-19T16:58:28.369795+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:51:40.636904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.391742+0000","last_change":"2023-12-19T16:58:28.383556+0000","last_active":"2023-12-19T16:58:28.391742+0000","last_peered":"2023-12-19T16:58:28.391742+0000","last_clean":"2023-12-19T16:58:28.391742+0000","last_became_active":"2023-12-19T16:58:28.383270+0000","last_became_peered":"2023-12-19T16:58:28.383270+0000","last_unstale":"2023-12-19T16:58:28.391742+0000","last_undegraded":"2023-12-19T16:58:28.391742+0000","last_fullsized":"2023-12-19T16:58:28.391742+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:08.073433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.380226+0000","last_change":"2023-12-19T16:58:27.982943+0000","last_active":"2023-12-19T16:58:29.380226+0000","last_peered":"2023-12-19T16:58:29.380226+0000","last_clean":"2023-12-19T16:58:29.380226+0000","last_became_active":"2023-12-19T16:58:27.982695+0000","last_became_peered":"2023-12-19T16:58:27.982695+0000","last_unstale":"2023-12-19T16:58:29.380226+0000","last_undegraded":"2023-12-19T16:58:29.380226+0000","last_fullsized":"2023-12-19T16:58:29.380226+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:38.848518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.403638+0000","last_change":"2023-12-19T16:58:28.402429+0000","last_active":"2023-12-19T16:58:28.403638+0000","last_peered":"2023-12-19T16:58:28.403638+0000","last_clean":"2023-12-19T16:58:28.403638+0000","last_became_active":"2023-12-19T16:58:28.402068+0000","last_became_peered":"2023-12-19T16:58:28.402068+0000","last_unstale":"2023-12-19T16:58:28.403638+0000","last_undegraded":"2023-12-19T16:58:28.403638+0000","last_fullsized":"2023-12-19T16:58:28.403638+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:49:39.660527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.380729+0000","last_change":"2023-12-19T16:58:27.977143+0000","last_active":"2023-12-19T16:58:28.380729+0000","last_peered":"2023-12-19T16:58:28.380729+0000","last_clean":"2023-12-19T16:58:28.380729+0000","last_became_active":"2023-12-19T16:58:27.976879+0000","last_became_peered":"2023-12-19T16:58:27.976879+0000","last_unstale":"2023-12-19T16:58:28.380729+0000","last_undegraded":"2023-12-19T16:58:28.380729+0000","last_fullsized":"2023-12-19T16:58:28.380729+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:38:04.181014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.402675+0000","last_change":"2023-12-19T16:58:28.398342+0000","last_active":"2023-12-19T16:58:28.402675+0000","last_peered":"2023-12-19T16:58:28.402675+0000","last_clean":"2023-12-19T16:58:28.402675+0000","last_became_active":"2023-12-19T16:58:28.396953+0000","last_became_peered":"2023-12-19T16:58:28.396953+0000","last_unstale":"2023-12-19T16:58:28.402675+0000","last_undegraded":"2023-12-19T16:58:28.402675+0000","last_fullsized":"2023-12-19T16:58:28.402675+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:11.231928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:29.379728+0000","last_change":"2023-12-19T16:58:27.982173+0000","last_active":"2023-12-19T16:58:29.379728+0000","last_peered":"2023-12-19T16:58:29.379728+0000","last_clean":"2023-12-19T16:58:29.379728+0000","last_became_active":"2023-12-19T16:58:27.980350+0000","last_became_peered":"2023-12-19T16:58:27.980350+0000","last_unstale":"2023-12-19T16:58:29.379728+0000","last_undegraded":"2023-12-19T16:58:29.379728+0000","last_fullsized":"2023-12-19T16:58:29.379728+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:25:08.124546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-19T16:58:30.101057+0000","last_change":"2023-12-19T16:58:28.392286+0000","last_active":"2023-12-19T16:58:30.101057+0000","last_peered":"2023-12-19T16:58:30.101057+0000","last_clean":"2023-12-19T16:58:30.101057+0000","last_became_active":"2023-12-19T16:58:28.392037+0000","last_became_peered":"2023-12-19T16:58:28.392037+0000","last_unstale":"2023-12-19T16:58:30.101057+0000","last_undegraded":"2023-12-19T16:58:30.101057+0000","last_fullsized":"2023-12-19T16:58:30.101057+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:27.959201+0000","last_clean_scrub_stamp":"2023-12-19T16:58:27.959201+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:36.607132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-19T16:58:28.399191+0000","last_change":"2023-12-19T16:58:27.987854+0000","last_active":"2023-12-19T16:58:28.399191+0000","last_peered":"2023-12-19T16:58:28.399191+0000","last_clean":"2023-12-19T16:58:28.399191+0000","last_became_active":"2023-12-19T16:58:27.987485+0000","last_became_peered":"2023-12-19T16:58:27.987485+0000","last_unstale":"2023-12-19T16:58:28.399191+0000","last_undegraded":"2023-12-19T16:58:28.399191+0000","last_fullsized":"2023-12-19T16:58:28.399191+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:58:26.956403+0000","last_clean_scrub_stamp":"2023-12-19T16:58:26.956403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:42:02.469929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"95'203","reported_seq":547,"reported_epoch":105,"state":"active+clean","last_fresh":"2023-12-19T16:58:27.975099+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T16:58:27.975099+0000","last_peered":"2023-12-19T16:58:27.975099+0000","last_clean":"2023-12-19T16:58:27.975099+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T16:58:27.975099+0000","last_undegraded":"2023-12-19T16:58:27.975099+0000","last_fullsized":"2023-12-19T16:58:27.975099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":866,"num_read_kb":1733,"num_write":315,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":866,"num_read_kb":1733,"num_write":315,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":203,"ondisk_log_size":203,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509635,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36992,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334848,"statfs":{"total":96636764160,"available":96598884352,"internally_reserved":0,"allocated":1892352,"data_stored":1238715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.69199999999999995,"15min":0.68999999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.30199999999999999},"max":{"1min":1.0169999999999999,"5min":1.3460000000000001,"15min":2.6389999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.72899999999999998,"15min":0.71499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.32700000000000001},"max":{"1min":0.875,"5min":1.6930000000000001,"15min":2.532},"last":0.54400000000000004}]},{"osd":1,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.73299999999999998,"15min":0.71599999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.36399999999999999},"max":{"1min":0.875,"5min":1.212,"15min":1.212},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.72999999999999998,"15min":0.72799999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":0.95899999999999996,"5min":1.74,"15min":1.74},"last":0.64100000000000001}]},{"osd":2,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.436,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89500000000000002,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.75900000000000001,"15min":0.73599999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.36599999999999999},"max":{"1min":1.048,"5min":1.395,"15min":1.395},"last":0.5}]},{"osd":3,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.73299999999999998,"15min":0.72099999999999997},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.44600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97899999999999998,"5min":1.266,"15min":1.266},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59199999999999997,"15min":0.57699999999999996},"min":{"1min":0.308,"5min":0.308,"15min":0.28499999999999998},"max":{"1min":0.91000000000000003,"5min":1.841,"15min":1.841},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61199999999999999,"15min":0.60499999999999998},"min":{"1min":0.44,"5min":0.40600000000000003,"15min":0.36599999999999999},"max":{"1min":0.80800000000000005,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.69499999999999995}]},{"osd":5,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61899999999999999,"15min":0.60599999999999998},"min":{"1min":0.376,"5min":0.36899999999999999,"15min":0.35699999999999998},"max":{"1min":0.85799999999999998,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65000000000000002,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.98299999999999998,"5min":1.784,"15min":1.784},"last":0.59799999999999998}]},{"osd":6,"last update":"Tue Dec 19 16:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70399999999999996,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.45700000000000002,"15min":0.39800000000000002},"max":{"1min":0.94599999999999995,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66200000000000003,"15min":0.65700000000000003},"min":{"1min":0.50800000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.84599999999999997,"5min":2.105,"15min":2.105},"last":0.72699999999999998}]}]},{"osd":6,"up_from":15,"seq":64424509635,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1306624,"data_stored":648971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69499999999999995,"15min":0.69999999999999996},"min":{"1min":0.51600000000000001,"5min":0.38700000000000001,"15min":0.309},"max":{"1min":0.93700000000000006,"5min":0.999,"15min":1.8600000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.70999999999999996,"15min":0.70099999999999996},"min":{"1min":0.46200000000000002,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":0.93300000000000005,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.501,"5min":0.41399999999999998,"15min":0.33400000000000002},"max":{"1min":0.85899999999999999,"5min":0.99399999999999999,"15min":4.3799999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.52100000000000002,"5min":0.47799999999999998,"15min":0.39500000000000002},"max":{"1min":0.95199999999999996,"5min":1.0189999999999999,"15min":1.583},"last":0.74399999999999999}]},{"osd":2,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.751,"15min":0.74299999999999999},"min":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.35199999999999998},"max":{"1min":0.89200000000000002,"5min":1.087,"15min":1.3180000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73299999999999998,"15min":0.73199999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0549999999999999,"15min":1.1879999999999999},"last":0.76400000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.753,"15min":0.77800000000000002},"min":{"1min":0.51100000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.97699999999999998,"5min":1.1020000000000001,"15min":4.673},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71499999999999997,"15min":0.72499999999999998},"min":{"1min":0.44900000000000001,"5min":0.42699999999999999,"15min":0.41399999999999998},"max":{"1min":0.84299999999999997,"5min":1.0740000000000001,"15min":1.962},"last":0.53500000000000003}]},{"osd":4,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.35899999999999999},"max":{"1min":0.84699999999999998,"5min":0.88400000000000001,"15min":0.96999999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.35699999999999998},"max":{"1min":0.83699999999999997,"5min":0.93500000000000005,"15min":1.006},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.33300000000000002},"max":{"1min":0.83999999999999997,"5min":0.90300000000000002,"15min":1.2050000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.68000000000000005,"15min":0.65600000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41499999999999998},"max":{"1min":0.89900000000000002,"5min":1.046,"15min":1.046},"last":0.71699999999999997}]},{"osd":7,"last update":"Tue Dec 19 16:57:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70499999999999996,"15min":0.69599999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.438},"max":{"1min":0.92300000000000004,"5min":1.006,"15min":1.0620000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71299999999999997,"15min":0.70099999999999996},"min":{"1min":0.47399999999999998,"5min":0.46600000000000003,"15min":0.371},"max":{"1min":0.90300000000000002,"5min":1.032,"15min":1.1220000000000001},"last":0.69999999999999996}]}]},{"osd":5,"up_from":15,"seq":64424509635,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36436,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335404,"statfs":{"total":96636764160,"available":96599453696,"internally_reserved":0,"allocated":1323008,"data_stored":656895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.68100000000000005,"15min":0.69899999999999995},"min":{"1min":0.46899999999999997,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.84099999999999997,"5min":0.88400000000000001,"15min":0.95799999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.90400000000000003,"5min":0.93999999999999995,"15min":1.0189999999999999},"last":0.72199999999999998}]},{"osd":1,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.48299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95099999999999996,"5min":0.995,"15min":0.995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.495,"5min":0.374,"15min":0.374},"max":{"1min":0.96699999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.5,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.89500000000000002,"5min":1.0409999999999999,"15min":1.1120000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.46400000000000002},"max":{"1min":0.84899999999999998,"5min":1.036,"15min":1.087},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.72899999999999998},"min":{"1min":0.51700000000000002,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":0.92000000000000004,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.77000000000000002,"15min":0.76100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.998,"5min":1.089,"15min":1.089},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54200000000000004,"15min":0.55800000000000005},"min":{"1min":0.35699999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77800000000000002,"5min":0.83999999999999997,"15min":1.165},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59299999999999997,"15min":0.59499999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.32400000000000001},"max":{"1min":0.91100000000000003,"5min":0.92600000000000005,"15min":1.0649999999999999},"last":0.79700000000000004}]},{"osd":6,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61199999999999999,"15min":0.60899999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.34699999999999998},"max":{"1min":0.88800000000000001,"5min":0.98799999999999999,"15min":1.153},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61099999999999999,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86599999999999999,"5min":0.94199999999999995,"15min":0.98999999999999999},"last":0.59799999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":1.133},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.67700000000000005},"min":{"1min":0.45500000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1080000000000001,"5min":1.131,"15min":1.131},"last":0.69599999999999995}]}]},{"osd":4,"up_from":15,"seq":64424509635,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54468,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317372,"statfs":{"total":96636764160,"available":96580988928,"internally_reserved":0,"allocated":1306624,"data_stored":655309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.39500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.0740000000000001,"5min":1.877,"15min":1.877},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92800000000000005,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.64500000000000002}]},{"osd":1,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.71299999999999997,"15min":0.71599999999999997},"min":{"1min":0.44900000000000001,"5min":0.44400000000000001,"15min":0.34799999999999998},"max":{"1min":0.85799999999999998,"5min":1.0209999999999999,"15min":1.8700000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98999999999999999,"5min":2.02,"15min":2.02},"last":0.69499999999999995}]},{"osd":2,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.54700000000000004,"5min":0.47099999999999997,"15min":0.38700000000000001},"max":{"1min":0.97199999999999998,"5min":1.375,"15min":1.375},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.76700000000000002,"15min":0.753},"min":{"1min":0.439,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":1.927,"15min":1.927},"last":0.746}]},{"osd":3,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88500000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.74099999999999999,"15min":0.75600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92100000000000004,"5min":0.97599999999999998,"15min":1.3129999999999999},"last":0.77100000000000002}]},{"osd":5,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.42099999999999999,"5min":0.379,"15min":0.35899999999999999},"max":{"1min":0.97699999999999998,"5min":1.99,"15min":1.99},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.42099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82299999999999995,"5min":0.89200000000000002,"15min":0.92800000000000005},"last":0.82999999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.59799999999999998,"15min":0.61599999999999999},"min":{"1min":0.41399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.0820000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65500000000000003,"15min":0.66100000000000003},"min":{"1min":0.50700000000000001,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":1.042,"5min":1.96,"15min":1.96},"last":0.93500000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:57:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.63300000000000001,"15min":0.65200000000000002},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.80500000000000005,"5min":0.89700000000000002,"15min":1.0429999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.54100000000000004,"5min":0.46400000000000002,"15min":0.44900000000000001},"max":{"1min":1.008,"5min":1.103,"15min":1.107},"last":0.873}]}]},{"osd":3,"up_from":15,"seq":64424509635,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36432,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335408,"statfs":{"total":96636764160,"available":96599457792,"internally_reserved":0,"allocated":1318912,"data_stored":656873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45100000000000001,"15min":0.44900000000000001},"min":{"1min":0.28000000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63300000000000001,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.496},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.32200000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70199999999999996,"5min":1.163,"15min":1.738},"last":0.41299999999999998}]},{"osd":1,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45500000000000002,"15min":0.46500000000000002},"min":{"1min":0.29899999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.76300000000000001,"5min":0.77800000000000002,"15min":1.141},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.75900000000000001,"5min":1.952,"15min":2.4910000000000001},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.53000000000000003,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.66400000000000003,"5min":0.85399999999999998,"15min":1.821},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.57099999999999995,"15min":0.58199999999999996},"min":{"1min":0.39700000000000002,"5min":0.372,"15min":0.35399999999999998},"max":{"1min":0.78100000000000003,"5min":2.855,"15min":5.1219999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.50600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.081,"5min":4.234,"15min":4.234},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.755,"15min":0.78500000000000003},"min":{"1min":0.56399999999999995,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0649999999999999,"5min":1.111,"15min":2.2690000000000001},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76400000000000001,"15min":0.78600000000000003},"min":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.52800000000000002},"max":{"1min":1.071,"5min":1.071,"15min":2.5350000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77000000000000002,"15min":0.78800000000000003},"min":{"1min":0.54400000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.042,"5min":1.8300000000000001,"15min":1.9079999999999999},"last":0.70599999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.77600000000000002,"15min":0.80000000000000004},"min":{"1min":0.51200000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98499999999999999,"5min":1.599,"15min":1.956},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76900000000000002,"15min":0.79800000000000004},"min":{"1min":0.54500000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0569999999999999,"5min":1.0780000000000001,"15min":1.974},"last":0.63}]},{"osd":7,"last update":"Tue Dec 19 16:57:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.80600000000000005,"15min":0.83099999999999996},"min":{"1min":0.63,"5min":0.54700000000000004,"15min":0.51900000000000002},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.9279999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.82999999999999996,"15min":0.86599999999999999},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.48799999999999999},"max":{"1min":0.97199999999999998,"5min":1.623,"15min":5.0759999999999996},"last":0.72199999999999998}]}]},{"osd":2,"up_from":15,"seq":64424509634,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40508,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331332,"statfs":{"total":96636764160,"available":96595283968,"internally_reserved":0,"allocated":1298432,"data_stored":648859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.49399999999999999,"15min":0.47099999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.28199999999999997},"max":{"1min":0.59399999999999997,"5min":5.4870000000000001,"15min":5.4870000000000001},"last":0.313},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.59299999999999997,"15min":0.53300000000000003},"min":{"1min":0.30599999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.58899999999999997,"5min":10.161,"15min":10.161},"last":0.54500000000000004}]},{"osd":1,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.625,"15min":0.63},"min":{"1min":0.42299999999999999,"5min":0.41099999999999998,"15min":0.39900000000000002},"max":{"1min":0.747,"5min":2.4740000000000002,"15min":5.7590000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.70799999999999996,"5min":2.911,"15min":5.8070000000000004},"last":0.435}]},{"osd":3,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58599999999999997,"15min":0.59699999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":1.1990000000000001,"15min":2.7599999999999998},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.58599999999999997,"15min":0.59199999999999997},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70099999999999996,"5min":2.7999999999999998,"15min":5.7190000000000003},"last":0.46500000000000002}]},{"osd":4,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78900000000000003,"15min":0.79800000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3,"5min":2.4449999999999998,"15min":3.8940000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81599999999999995,"15min":0.79300000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.42999999999999999},"max":{"1min":1.226,"5min":5.5430000000000001,"15min":5.5430000000000001},"last":0.53400000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.84799999999999998,"15min":0.80900000000000005},"min":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.4890000000000001,"5min":5.601,"15min":5.601},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81100000000000005,"15min":0.82599999999999996},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.268,"5min":2.569,"15min":2.9260000000000002},"last":0.68400000000000005}]},{"osd":6,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83399999999999996,"15min":0.83299999999999996},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.52400000000000002},"max":{"1min":1.4319999999999999,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.76500000000000001,"15min":0.79800000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":2.121},"last":0.72499999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.87,"15min":0.82799999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52300000000000002},"max":{"1min":1.3340000000000001,"5min":5.673,"15min":5.673},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82199999999999995,"15min":0.84599999999999997},"min":{"1min":0.624,"5min":0.60599999999999998,"15min":0.57799999999999996},"max":{"1min":1.369,"5min":1.429,"15min":2.0529999999999999},"last":0.71399999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509635,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50372,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321468,"statfs":{"total":96636764160,"available":96585183232,"internally_reserved":0,"allocated":1306624,"data_stored":648471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.433,"15min":0.44700000000000001},"min":{"1min":0.27900000000000003,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":2.3050000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.50600000000000001,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.65900000000000003,"5min":0.84199999999999997,"15min":0.89800000000000002},"last":0.47399999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.49399999999999999,"15min":0.51200000000000001},"min":{"1min":0.32900000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.66000000000000003,"5min":0.80100000000000005,"15min":1.992},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.29799999999999999},"max":{"1min":0.80300000000000005,"5min":0.81899999999999995,"15min":0.97299999999999998},"last":0.38300000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.61799999999999999},"min":{"1min":0.43099999999999999,"5min":0.36899999999999999,"15min":0.34799999999999998},"max":{"1min":0.76200000000000001,"5min":0.871,"15min":4.133},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52900000000000003,"15min":0.54200000000000004},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33700000000000002},"max":{"1min":0.746,"5min":0.76400000000000001,"15min":0.91900000000000004},"last":0.55800000000000005}]},{"osd":4,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.745,"15min":0.76400000000000001},"min":{"1min":0.45300000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.107,"5min":1.202,"15min":2.1949999999999998},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.78100000000000003,"15min":0.78800000000000003},"min":{"1min":0.56499999999999995,"5min":0.44400000000000001,"15min":0.42399999999999999},"max":{"1min":1.01,"5min":1.181,"15min":2.048},"last":0.79300000000000004}]},{"osd":5,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77200000000000002,"15min":0.78500000000000003},"min":{"1min":0.56200000000000006,"5min":0.46500000000000002,"15min":0.45800000000000002},"max":{"1min":1.0589999999999999,"5min":1.288,"15min":2.1269999999999998},"last":0.872},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79300000000000004,"15min":0.81100000000000005},"min":{"1min":0.59499999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.04,"5min":1.3380000000000001,"15min":4.0460000000000003},"last":0.81699999999999995}]},{"osd":6,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76200000000000001,"15min":0.79500000000000004},"min":{"1min":0.54900000000000004,"5min":0.48299999999999998,"15min":0.41999999999999998},"max":{"1min":0.98599999999999999,"5min":1.0740000000000001,"15min":2.1499999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79300000000000004,"15min":0.80900000000000005},"min":{"1min":0.56999999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1659999999999999,"5min":1.391,"15min":2.0880000000000001},"last":0.82899999999999996}]},{"osd":7,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82699999999999996,"15min":0.84199999999999997},"min":{"1min":0.63,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.024,"5min":1.2230000000000001,"15min":4.093},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80000000000000004,"15min":0.83599999999999997},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.53800000000000003},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":2.2349999999999999},"last":0.88400000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542340,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32896,"kb_used_data":1848,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338944,"statfs":{"total":96636764160,"available":96603078656,"internally_reserved":0,"allocated":1892352,"data_stored":1238679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.434,"15min":0.42899999999999999},"min":{"1min":0.314,"5min":0.246,"15min":0.246},"max":{"1min":0.65600000000000003,"5min":0.752,"15min":0.752},"last":0.439},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.49299999999999999},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.26300000000000001},"max":{"1min":0.73299999999999998,"5min":0.81699999999999995,"15min":0.85499999999999998},"last":0.45400000000000001}]},{"osd":2,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.29299999999999998},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.89700000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55200000000000005,"15min":0.53600000000000003},"min":{"1min":0.39800000000000002,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":0.78400000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57299999999999995}]},{"osd":3,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.52500000000000002},"min":{"1min":0.32600000000000001,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.84699999999999998,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.628,"15min":0.61899999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.40899999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.503,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.149,"5min":1.444,"15min":1.512},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.78700000000000003},"min":{"1min":0.56000000000000005,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0349999999999999,"5min":1.1020000000000001,"15min":1.52},"last":0.62}]},{"osd":5,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.80200000000000005,"15min":0.80000000000000004},"min":{"1min":0.55700000000000005,"5min":0.502,"15min":0.41499999999999998},"max":{"1min":1.125,"5min":1.294,"15min":1.748},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81299999999999994,"15min":0.79600000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.51600000000000001},"max":{"1min":1.1830000000000001,"5min":1.256,"15min":1.649},"last":0.70099999999999996}]},{"osd":6,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81100000000000005,"15min":0.79600000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46999999999999997},"max":{"1min":1.2050000000000001,"5min":1.429,"15min":1.429},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.82099999999999995,"15min":0.81499999999999995},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.44700000000000001},"max":{"1min":1.224,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.59099999999999997}]},{"osd":7,"last update":"Tue Dec 19 16:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82999999999999996,"15min":0.82199999999999995},"min":{"1min":0.58599999999999997,"5min":0.505,"15min":0.48999999999999999},"max":{"1min":1.077,"5min":1.377,"15min":1.377},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82399999999999995,"15min":0.81699999999999995},"min":{"1min":0.53200000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0580000000000001,"5min":1.3300000000000001,"15min":1.661},"last":0.66800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T16:58:32.887 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T16:58:32.887 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T16:58:33.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.072+0000 7fbce6fa6640 1 Processor -- start 2023-12-19T16:58:33.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.072+0000 7fbce6fa6640 1 -- start start 2023-12-19T16:58:33.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.072+0000 7fbce6fa6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbce015ada0 0x7fbce00ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:33.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.072+0000 7fbce6fa6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbce0070db0 0x7fbce00b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:33.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.072+0000 7fbce6fa6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce006b500 con 0x7fbce00b0a60 2023-12-19T16:58:33.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.072+0000 7fbce6fa6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce00715b0 con 0x7fbce015ada0 2023-12-19T16:58:33.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.072+0000 7fbce6fa6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00adfd0 con 0x7fbce0070db0 2023-12-19T16:58:33.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.076+0000 7fbcdffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbce0070db0 0x7fbce00b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:33.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.076+0000 7fbce4d42640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbce015ada0 0x7fbce00ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:33.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.076+0000 7fbcdffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbce0070db0 0x7fbce00b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:55746/0 (socket says 172.21.15.26:55746) 2023-12-19T16:58:33.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.076+0000 7fbcdffff640 1 -- 172.21.15.26:0/3532985684 learned_addr learned my addr 172.21.15.26:0/3532985684 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:33.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.076+0000 7fbce4d42640 1 -- 172.21.15.26:0/3532985684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbce0070db0 msgr2=0x7fbce00b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:33.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.076+0000 7fbce4d42640 1 --2- 172.21.15.26:0/3532985684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbce0070db0 0x7fbce00b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.076+0000 7fbce4d42640 1 -- 172.21.15.26:0/3532985684 >> v1:172.21.15.160:6789/0 conn(0x7fbce00b0a60 legacy=0x7fbce00ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:33.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.076+0000 7fbce4d42640 1 -- 172.21.15.26:0/3532985684 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce0070b60 con 0x7fbce015ada0 2023-12-19T16:58:33.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.076+0000 7fbcdffff640 1 --2- 172.21.15.26:0/3532985684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbce0070db0 0x7fbce00b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:33.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.076+0000 7fbce4d42640 1 --2- 172.21.15.26:0/3532985684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbce015ada0 0x7fbce00ada90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009820 tx=0x7fbcd401c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64fc6e365d5f2d70 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:33.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbcdf7fe640 1 -- 172.21.15.26:0/3532985684 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40ab600 con 0x7fbce015ada0 2023-12-19T16:58:33.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbcdf7fe640 1 -- 172.21.15.26:0/3532985684 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd40a7050 con 0x7fbce015ada0 2023-12-19T16:58:33.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbcdf7fe640 1 -- 172.21.15.26:0/3532985684 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40ab980 con 0x7fbce015ada0 2023-12-19T16:58:33.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3532985684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbce015ada0 msgr2=0x7fbce00ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:33.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 --2- 172.21.15.26:0/3532985684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbce015ada0 0x7fbce00ada90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009820 tx=0x7fbcd401c430 comp rx=0 tx=0).stop 2023-12-19T16:58:33.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3532985684 shutdown_connections 2023-12-19T16:58:33.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 --2- 172.21.15.26:0/3532985684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbce0070db0 0x7fbce00b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 --2- 172.21.15.26:0/3532985684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbce015ada0 0x7fbce00ada90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3532985684 >> 172.21.15.26:0/3532985684 conn(0x7fbce00660d0 msgr2=0x7fbce00ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:33.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3532985684 shutdown_connections 2023-12-19T16:58:33.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3532985684 wait complete. 2023-12-19T16:58:33.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 Processor -- start 2023-12-19T16:58:33.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 -- start start 2023-12-19T16:58:33.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce0070db0 0x7fbce014fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:33.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbce00b0a60 0x7fbce013c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:33.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbce015ada0 0x7fbce00cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:33.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce0150420 con 0x7fbce0070db0 2023-12-19T16:58:33.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce0155ee0 con 0x7fbce015ada0 2023-12-19T16:58:33.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce6fa6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce01561c0 con 0x7fbce00b0a60 2023-12-19T16:58:33.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce4d42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce0070db0 0x7fbce014fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:33.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce4d42640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce0070db0 0x7fbce014fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46660/0 (socket says 172.21.15.26:46660) 2023-12-19T16:58:33.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce4d42640 1 -- 172.21.15.26:0/3870718762 learned_addr learned my addr 172.21.15.26:0/3870718762 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:33.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbcdffff640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbce00b0a60 0x7fbce013c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:33.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.080+0000 7fbce5543640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbce015ada0 0x7fbce00cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:33.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbce4d42640 1 -- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbce00b0a60 msgr2=0x7fbce013c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:33.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbce4d42640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbce00b0a60 0x7fbce013c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbce4d42640 1 -- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbce015ada0 msgr2=0x7fbce00cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:33.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbce4d42640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbce015ada0 0x7fbce00cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbce4d42640 1 -- 172.21.15.26:0/3870718762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc0094d0 con 0x7fbce0070db0 2023-12-19T16:58:33.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbce4d42640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce0070db0 0x7fbce014fee0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fbcd401a060 tx=0x7fbcd40191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:33.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbcdd7fa640 1 -- 172.21.15.26:0/3870718762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40b4400 con 0x7fbce0070db0 2023-12-19T16:58:33.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbcdd7fa640 1 -- 172.21.15.26:0/3870718762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd40b4560 con 0x7fbce0070db0 2023-12-19T16:58:33.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbcdd7fa640 1 -- 172.21.15.26:0/3870718762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40b4910 con 0x7fbce0070db0 2023-12-19T16:58:33.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3870718762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd4009420 con 0x7fbce0070db0 2023-12-19T16:58:33.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3870718762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce01506c0 con 0x7fbce0070db0 2023-12-19T16:58:33.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3870718762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce0074270 con 0x7fbce0070db0 2023-12-19T16:58:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbcdd7fa640 1 -- 172.21.15.26:0/3870718762 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbcd40b2040 con 0x7fbce0070db0 2023-12-19T16:58:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.084+0000 7fbcdd7fa640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbcc4065b80 0x7fbcc4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:33.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.088+0000 7fbcdd7fa640 1 -- 172.21.15.26:0/3870718762 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fbcd411d570 con 0x7fbce0070db0 2023-12-19T16:58:33.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.088+0000 7fbcdffff640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbcc4065b80 0x7fbcc4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:33.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.088+0000 7fbcdffff640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbcc4065b80 0x7fbcc4068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbce0152380 tx=0x7fbccc009240 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:33.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.088+0000 7fbcdd7fa640 1 -- 172.21.15.26:0/3870718762 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbcd40eaae0 con 0x7fbce0070db0 2023-12-19T16:58:33.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.304+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3870718762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbce00ae660 con 0x7fbce0070db0 2023-12-19T16:58:33.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.308+0000 7fbcdd7fa640 1 -- 172.21.15.26:0/3870718762 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7fbcd40ee100 con 0x7fbce0070db0 2023-12-19T16:58:33.310 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2023-12-19T16:58:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.308+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbcc4065b80 msgr2=0x7fbcc4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.308+0000 7fbce6fa6640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbcc4065b80 0x7fbcc4068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbce0152380 tx=0x7fbccc009240 comp rx=0 tx=0).stop 2023-12-19T16:58:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.308+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3870718762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce0070db0 msgr2=0x7fbce014fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.308+0000 7fbce6fa6640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce0070db0 0x7fbce014fee0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fbcd401a060 tx=0x7fbcd40191d0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.308+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3870718762 shutdown_connections 2023-12-19T16:58:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.308+0000 7fbce6fa6640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbce015ada0 0x7fbce00cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.308+0000 7fbce6fa6640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbcc4065b80 0x7fbcc4068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.308+0000 7fbce6fa6640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbce00b0a60 0x7fbce013c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.308+0000 7fbce6fa6640 1 --2- 172.21.15.26:0/3870718762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbce0070db0 0x7fbce014fee0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.308+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3870718762 >> 172.21.15.26:0/3870718762 conn(0x7fbce00660d0 msgr2=0x7fbce0159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.308+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3870718762 shutdown_connections 2023-12-19T16:58:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.308+0000 7fbce6fa6640 1 -- 172.21.15.26:0/3870718762 wait complete. 2023-12-19T16:58:33.331 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:58:33.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.472+0000 7f88aeef2640 1 Processor -- start 2023-12-19T16:58:33.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.472+0000 7f88aeef2640 1 -- start start 2023-12-19T16:58:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.476+0000 7f88aeef2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88a8154220 0x7f88a815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.476+0000 7f88aeef2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f88a80cee10 0x7f88a815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.476+0000 7f88aeef2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a806bc30 con 0x7f88a806b3d0 2023-12-19T16:58:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.476+0000 7f88aeef2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a80cc6e0 con 0x7f88a8154220 2023-12-19T16:58:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.476+0000 7f88aeef2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a815afc0 con 0x7f88a80cee10 2023-12-19T16:58:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.476+0000 7f88acc8e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88a8154220 0x7f88a815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.476+0000 7f88acc8e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88a8154220 0x7f88a815aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46674/0 (socket says 172.21.15.26:46674) 2023-12-19T16:58:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.476+0000 7f88acc8e640 1 -- 172.21.15.26:0/3266474899 learned_addr learned my addr 172.21.15.26:0/3266474899 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.476+0000 7f88ad48f640 1 --2- 172.21.15.26:0/3266474899 >> v2:172.21.15.160:3300/0 conn(0x7f88a80cee10 0x7f88a815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.476+0000 7f88acc8e640 1 -- 172.21.15.26:0/3266474899 >> v2:172.21.15.160:3300/0 conn(0x7f88a80cee10 msgr2=0x7f88a815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.476+0000 7f88acc8e640 1 --2- 172.21.15.26:0/3266474899 >> v2:172.21.15.160:3300/0 conn(0x7f88a80cee10 0x7f88a815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.476+0000 7f88acc8e640 1 -- 172.21.15.26:0/3266474899 >> v1:172.21.15.160:6789/0 conn(0x7f88a806b3d0 legacy=0x7f88a80ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:33.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.476+0000 7f88acc8e640 1 -- 172.21.15.26:0/3266474899 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a806b950 con 0x7f88a8154220 2023-12-19T16:58:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88acc8e640 1 --2- 172.21.15.26:0/3266474899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a8154220 0x7f88a815aa80 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f88a80cdf30 tx=0x7f889c01e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780b8a611b27666b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88a77fe640 1 -- 172.21.15.26:0/3266474899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c0a8670 con 0x7f88a8154220 2023-12-19T16:58:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88a77fe640 1 -- 172.21.15.26:0/3266474899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889c0a87d0 con 0x7f88a8154220 2023-12-19T16:58:33.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88a77fe640 1 -- 172.21.15.26:0/3266474899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c0a8b60 con 0x7f88a8154220 2023-12-19T16:58:33.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88aeef2640 1 -- 172.21.15.26:0/3266474899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a8154220 msgr2=0x7f88a815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:33.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88aeef2640 1 --2- 172.21.15.26:0/3266474899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a8154220 0x7f88a815aa80 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f88a80cdf30 tx=0x7f889c01e6a0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88aeef2640 1 -- 172.21.15.26:0/3266474899 shutdown_connections 2023-12-19T16:58:33.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88aeef2640 1 --2- 172.21.15.26:0/3266474899 >> v2:172.21.15.160:3300/0 conn(0x7f88a80cee10 0x7f88a815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88aeef2640 1 --2- 172.21.15.26:0/3266474899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a8154220 0x7f88a815aa80 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88aeef2640 1 -- 172.21.15.26:0/3266474899 >> 172.21.15.26:0/3266474899 conn(0x7f88a80658e0 msgr2=0x7f88a8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:33.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88aeef2640 1 -- 172.21.15.26:0/3266474899 shutdown_connections 2023-12-19T16:58:33.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88aeef2640 1 -- 172.21.15.26:0/3266474899 wait complete. 2023-12-19T16:58:33.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88aeef2640 1 Processor -- start 2023-12-19T16:58:33.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.480+0000 7f88aeef2640 1 -- start start 2023-12-19T16:58:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88aeef2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806b3d0 0x7f88a814afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88aeef2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88a80cee10 0x7f88a814cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88aeef2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a8154220 0x7f88a80aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88aeef2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a81f8990 con 0x7f88a806b3d0 2023-12-19T16:58:33.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88aeef2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a81f59a0 con 0x7f88a8154220 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88aeef2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a81f5c80 con 0x7f88a80cee10 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88acc8e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806b3d0 0x7f88a814afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88acc8e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806b3d0 0x7f88a814afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46676/0 (socket says 172.21.15.26:46676) 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88acc8e640 1 -- 172.21.15.26:0/454507110 learned_addr learned my addr 172.21.15.26:0/454507110 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88a7fff640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88a80cee10 0x7f88a814cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88ad48f640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a8154220 0x7f88a80aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88acc8e640 1 -- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88a80cee10 msgr2=0x7f88a814cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88acc8e640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88a80cee10 0x7f88a814cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88acc8e640 1 -- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a8154220 msgr2=0x7f88a80aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88acc8e640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a8154220 0x7f88a80aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88acc8e640 1 -- 172.21.15.26:0/454507110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c0094d0 con 0x7f88a806b3d0 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88acc8e640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806b3d0 0x7f88a814afc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f889c01cd50 tx=0x7f889c01b440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88a57fa640 1 -- 172.21.15.26:0/454507110 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c0a8670 con 0x7f88a806b3d0 2023-12-19T16:58:33.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88a57fa640 1 -- 172.21.15.26:0/454507110 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889c0b3860 con 0x7f88a806b3d0 2023-12-19T16:58:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88aeef2640 1 -- 172.21.15.26:0/454507110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a81f5f60 con 0x7f88a806b3d0 2023-12-19T16:58:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88a57fa640 1 -- 172.21.15.26:0/454507110 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c0b3bd0 con 0x7f88a806b3d0 2023-12-19T16:58:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88aeef2640 1 -- 172.21.15.26:0/454507110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a814b500 con 0x7f88a806b3d0 2023-12-19T16:58:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88aeef2640 1 -- 172.21.15.26:0/454507110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a8074270 con 0x7f88a806b3d0 2023-12-19T16:58:33.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.484+0000 7f88a57fa640 1 -- 172.21.15.26:0/454507110 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f889c0b3d30 con 0x7f88a806b3d0 2023-12-19T16:58:33.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.488+0000 7f88a57fa640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f888c065b80 0x7f888c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:33.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.488+0000 7f88a57fa640 1 -- 172.21.15.26:0/454507110 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f889c11def0 con 0x7f88a806b3d0 2023-12-19T16:58:33.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.488+0000 7f88a7fff640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f888c065b80 0x7f888c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:33.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.488+0000 7f88a7fff640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f888c065b80 0x7f888c068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88a81533d0 tx=0x7f8894008040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:33.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.492+0000 7f88a57fa640 1 -- 172.21.15.26:0/454507110 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f889c0eb420 con 0x7f88a806b3d0 2023-12-19T16:58:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88aeef2640 1 -- 172.21.15.26:0/454507110 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88a80cdf60 con 0x7f88a806b3d0 2023-12-19T16:58:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88a57fa640 1 -- 172.21.15.26:0/454507110 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f889c0ae090 con 0x7f88a806b3d0 2023-12-19T16:58:33.685 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:33.685 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3393501439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3393501439},{"type":"v1","addr":"172.21.15.160:6836","nonce":3393501439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1517280064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1517280064},{"type":"v1","addr":"172.21.15.160:6837","nonce":1517280064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/2282339695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2282339695},{"type":"v1","addr":"172.21.15.26:6834","nonce":2282339695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:58:28.364346+0000","modified":"2023-12-19T16:58:31.386135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2693922436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2693922436},{"type":"v1","addr":"172.21.15.26:6835","nonce":2693922436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T16:58:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2023-12-19T16:58:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88aeef2640 1 -- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f888c065b80 msgr2=0x7f888c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88aeef2640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f888c065b80 0x7f888c068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88a81533d0 tx=0x7f8894008040 comp rx=0 tx=0).stop 2023-12-19T16:58:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88aeef2640 1 -- 172.21.15.26:0/454507110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806b3d0 msgr2=0x7f88a814afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88aeef2640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806b3d0 0x7f88a814afc0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f889c01cd50 tx=0x7f889c01b440 comp rx=0 tx=0).stop 2023-12-19T16:58:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88aeef2640 1 -- 172.21.15.26:0/454507110 shutdown_connections 2023-12-19T16:58:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88aeef2640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88a8154220 0x7f88a80aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88aeef2640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f888c065b80 0x7f888c068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88aeef2640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f88a80cee10 0x7f88a814cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88aeef2640 1 --2- 172.21.15.26:0/454507110 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88a806b3d0 0x7f88a814afc0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88aeef2640 1 -- 172.21.15.26:0/454507110 >> 172.21.15.26:0/454507110 conn(0x7f88a80658e0 msgr2=0x7f88a815d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88aeef2640 1 -- 172.21.15.26:0/454507110 shutdown_connections 2023-12-19T16:58:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:33.684+0000 7f88aeef2640 1 -- 172.21.15.26:0/454507110 wait complete. 2023-12-19T16:58:34.108 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T16:58:28.364346+0000', 'modified': '2023-12-19T16:58:31.386135+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7479}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7479': {'gid': 7479, 'name': 'c', 'rank': 0, 'incarnation': 45, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/2693922436', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 2693922436}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 2693922436}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T16:58:34.108 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T16:58:34.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-19T16:58:34.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.252+0000 7f737542d640 1 Processor -- start 2023-12-19T16:58:34.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.252+0000 7f737542d640 1 -- start start 2023-12-19T16:58:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f737542d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f737006b3d0 0x7f73700ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f737542d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73700cec40 0x7f737015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f737542d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f737006bc30 con 0x7f7370154220 2023-12-19T16:58:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f737542d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73700cf0d0 con 0x7f737006b3d0 2023-12-19T16:58:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f737542d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73700cc6e0 con 0x7f73700cec40 2023-12-19T16:58:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7370154220 0x7f737015aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45352/0 (socket says 172.21.15.26:45352) 2023-12-19T16:58:34.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736effd640 1 -- 172.21.15.26:0/4269473617 learned_addr learned my addr 172.21.15.26:0/4269473617 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:34.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736f7fe640 1 --2- 172.21.15.26:0/4269473617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73700cec40 0x7f737015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f7367fff640 1 --2- 172.21.15.26:0/4269473617 >> v2:172.21.15.160:3300/0 conn(0x7f737006b3d0 0x7f73700ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515861034 0 0) 0x7f737006bc30 con 0x7f7370154220 2023-12-19T16:58:34.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f735c003530 con 0x7f7370154220 2023-12-19T16:58:34.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1568735955 0 0) 0x7f735c003530 con 0x7f7370154220 2023-12-19T16:58:34.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73700cec40 msgr2=0x7f737015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:34.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 --2- 172.21.15.26:0/4269473617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73700cec40 0x7f737015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 >> v2:172.21.15.160:3300/0 conn(0x7f737006b3d0 msgr2=0x7f73700ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:34.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 --2- 172.21.15.26:0/4269473617 >> v2:172.21.15.160:3300/0 conn(0x7f737006b3d0 0x7f73700ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737006b950 con 0x7f7370154220 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f7360001680 con 0x7f7370154220 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 >> v1:172.21.15.26:6789/0 conn(0x7f7370154220 legacy=0x7f737015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 --2- 172.21.15.26:0/4269473617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c003980 0x7f735c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 --2- 172.21.15.26:0/4269473617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c0047d0 0x7f735c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 --2- 172.21.15.26:0/4269473617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73700cec40 0x7f735c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73700cf0d0 con 0x7f735c003980 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73700cc6e0 con 0x7f735c0047d0 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c00b990 con 0x7f73700cec40 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736effd640 1 --2- 172.21.15.26:0/4269473617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c003980 0x7f735c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f7367fff640 1 --2- 172.21.15.26:0/4269473617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c0047d0 0x7f735c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736f7fe640 1 --2- 172.21.15.26:0/4269473617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73700cec40 0x7f735c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736effd640 1 -- 172.21.15.26:0/4269473617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73700cec40 msgr2=0x7f735c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:34.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736effd640 1 --2- 172.21.15.26:0/4269473617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73700cec40 0x7f735c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736effd640 1 -- 172.21.15.26:0/4269473617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c0047d0 msgr2=0x7f735c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:34.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736effd640 1 --2- 172.21.15.26:0/4269473617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c0047d0 0x7f735c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f736effd640 1 -- 172.21.15.26:0/4269473617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c003530 con 0x7f735c003980 2023-12-19T16:58:34.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.256+0000 7f7367fff640 1 --2- 172.21.15.26:0/4269473617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c0047d0 0x7f735c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:34.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.260+0000 7f736effd640 1 --2- 172.21.15.26:0/4269473617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c003980 0x7f735c003da0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f736000c290 tx=0x7f736001ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07fe5bdb78e99d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:34.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.260+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73600058e0 con 0x7f735c003980 2023-12-19T16:58:34.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.260+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7360005a40 con 0x7f735c003980 2023-12-19T16:58:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.260+0000 7f736e7fc640 1 -- 172.21.15.26:0/4269473617 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7360005df0 con 0x7f735c003980 2023-12-19T16:58:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.260+0000 7f737542d640 1 -- 172.21.15.26:0/4269473617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c003980 msgr2=0x7f735c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.260+0000 7f737542d640 1 --2- 172.21.15.26:0/4269473617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f735c003980 0x7f735c003da0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f736000c290 tx=0x7f736001ccb0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.260+0000 7f736f7fe640 1 -- 172.21.15.26:0/4269473617 reap_dead start 2023-12-19T16:58:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.260+0000 7f737542d640 1 -- 172.21.15.26:0/4269473617 shutdown_connections 2023-12-19T16:58:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.260+0000 7f737542d640 1 -- 172.21.15.26:0/4269473617 >> 172.21.15.26:0/4269473617 conn(0x7f73700658e0 msgr2=0x7f7370065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.260+0000 7f737542d640 1 -- 172.21.15.26:0/4269473617 shutdown_connections 2023-12-19T16:58:34.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.260+0000 7f737542d640 1 -- 172.21.15.26:0/4269473617 wait complete. 2023-12-19T16:58:34.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f737542d640 1 Processor -- start 2023-12-19T16:58:34.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f737542d640 1 -- start start 2023-12-19T16:58:34.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f737542d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c003980 0x7f737014ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f737542d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73701f8400 0x7f73701f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f737542d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7370158390 0x7f7370158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f737542d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7370158d50 con 0x7f7370158390 2023-12-19T16:58:34.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f737542d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737014d3d0 con 0x7f735c003980 2023-12-19T16:58:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f737542d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7370165990 con 0x7f73701f8400 2023-12-19T16:58:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7370158390 0x7f7370158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7370158390 0x7f7370158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46688/0 (socket says 172.21.15.26:46688) 2023-12-19T16:58:34.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f736f7fe640 1 -- 172.21.15.26:0/4148157408 learned_addr learned my addr 172.21.15.26:0/4148157408 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f736f7fe640 1 -- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73701f8400 msgr2=0x7f73701f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f7367fff640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73701f8400 0x7f73701f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f736effd640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c003980 0x7f737014ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f736f7fe640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73701f8400 0x7f73701f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f736f7fe640 1 -- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c003980 msgr2=0x7f737014ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f736f7fe640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c003980 0x7f737014ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f736f7fe640 1 -- 172.21.15.26:0/4148157408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73701f8ec0 con 0x7f7370158390 2023-12-19T16:58:34.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f736effd640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c003980 0x7f737014ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f7367fff640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73701f8400 0x7f73701f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f736f7fe640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7370158390 0x7f7370158810 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f73700cc400 tx=0x7f73680115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f73677fe640 1 -- 172.21.15.26:0/4148157408 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736800e070 con 0x7f7370158390 2023-12-19T16:58:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f73677fe640 1 -- 172.21.15.26:0/4148157408 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7368011ed0 con 0x7f7370158390 2023-12-19T16:58:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f73677fe640 1 -- 172.21.15.26:0/4148157408 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73680039a0 con 0x7f7370158390 2023-12-19T16:58:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f737542d640 1 -- 172.21.15.26:0/4148157408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73701f91a0 con 0x7f7370158390 2023-12-19T16:58:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.264+0000 7f737542d640 1 -- 172.21.15.26:0/4148157408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7370165f30 con 0x7f7370158390 2023-12-19T16:58:34.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f73677fe640 1 -- 172.21.15.26:0/4148157408 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7368003b20 con 0x7f7370158390 2023-12-19T16:58:34.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f73677fe640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7350065b80 0x7f7350068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f73677fe640 1 -- 172.21.15.26:0/4148157408 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f7368083ed0 con 0x7f7370158390 2023-12-19T16:58:34.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f736effd640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7350065b80 0x7f7350068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f7378795000 1 Processor -- start 2023-12-19T16:58:34.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f7378795000 1 -- start start 2023-12-19T16:58:34.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f736effd640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7350065b80 0x7f7350068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f73600aa000 tx=0x7f7360001a10 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:34.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f7378795000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fe5070 0x5640e8fe5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f7378795000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8fe5990 0x5640e8f9ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f7378795000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8f9b1c0 0x5640e8f9b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f7378795000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8dbec90 con 0x5640e8f9b1c0 2023-12-19T16:58:34.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f7378795000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8df0220 con 0x5640e8fe5990 2023-12-19T16:58:34.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f7378795000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640e8ded7f0 con 0x5640e8fe5070 2023-12-19T16:58:34.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.268+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8f9b1c0 0x5640e8f9b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8f9b1c0 0x5640e8f9b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46702/0 (socket says 172.21.15.26:46702) 2023-12-19T16:58:34.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f736f7fe640 1 -- 172.21.15.26:0/503172134 learned_addr learned my addr 172.21.15.26:0/503172134 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:34.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7367fff640 1 --2- 172.21.15.26:0/503172134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8fe5990 0x5640e8f9ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f736effd640 1 --2- 172.21.15.26:0/503172134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fe5070 0x5640e8fe5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f736f7fe640 1 -- 172.21.15.26:0/503172134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fe5070 msgr2=0x5640e8fe5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:34.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f736f7fe640 1 --2- 172.21.15.26:0/503172134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fe5070 0x5640e8fe5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f736f7fe640 1 -- 172.21.15.26:0/503172134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8fe5990 msgr2=0x5640e8f9ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:34.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f736f7fe640 1 --2- 172.21.15.26:0/503172134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8fe5990 0x5640e8f9ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f736f7fe640 1 -- 172.21.15.26:0/503172134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640e8ae5890 con 0x5640e8f9b1c0 2023-12-19T16:58:34.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f736f7fe640 1 --2- 172.21.15.26:0/503172134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8f9b1c0 0x5640e8f9b5a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7368009fd0 tx=0x7f7368055070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a1ab29f953abb6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:34.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7364ff9640 1 -- 172.21.15.26:0/503172134 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736805cf90 con 0x5640e8f9b1c0 2023-12-19T16:58:34.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7364ff9640 1 -- 172.21.15.26:0/503172134 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7368053350 con 0x5640e8f9b1c0 2023-12-19T16:58:34.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7364ff9640 1 -- 172.21.15.26:0/503172134 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73680516c0 con 0x5640e8f9b1c0 2023-12-19T16:58:34.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 -- 172.21.15.26:0/503172134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8f9b1c0 msgr2=0x5640e8f9b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 --2- 172.21.15.26:0/503172134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8f9b1c0 0x5640e8f9b5a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7368009fd0 tx=0x7f7368055070 comp rx=0 tx=0).stop 2023-12-19T16:58:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 -- 172.21.15.26:0/503172134 shutdown_connections 2023-12-19T16:58:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 --2- 172.21.15.26:0/503172134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8f9b1c0 0x5640e8f9b5a0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 --2- 172.21.15.26:0/503172134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8fe5990 0x5640e8f9ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 --2- 172.21.15.26:0/503172134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fe5070 0x5640e8fe5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 -- 172.21.15.26:0/503172134 >> 172.21.15.26:0/503172134 conn(0x5640e8fa1950 msgr2=0x5640e8f99560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 -- 172.21.15.26:0/503172134 shutdown_connections 2023-12-19T16:58:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 -- 172.21.15.26:0/503172134 wait complete. 2023-12-19T16:58:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 -- start start 2023-12-19T16:58:34.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 Processor -- start 2023-12-19T16:58:34.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8f9b1c0 0x5640e8fa0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e8fa7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8fe5990 0x5640e8fa75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8de90e0 con 0x5640e8fe5070 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8d88310 con 0x5640e8fe5990 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640e8da6770 con 0x5640e8f9b1c0 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7378795000 10 client.0 _ll_register_callbacks cb 0x5640e8f86b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.272+0000 7f7367fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e8fa7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7367fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e8fa7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46706/0 (socket says 172.21.15.26:46706) 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7367fff640 1 -- 172.21.15.26:0/3500794643 learned_addr learned my addr 172.21.15.26:0/3500794643 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f736effd640 1 --2- 172.21.15.26:0/3500794643 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8f9b1c0 0x5640e8fa0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f736f7fe640 1 --2- 172.21.15.26:0/3500794643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8fe5990 0x5640e8fa75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7367fff640 1 -- 172.21.15.26:0/3500794643 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8f9b1c0 msgr2=0x5640e8fa0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7367fff640 1 --2- 172.21.15.26:0/3500794643 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8f9b1c0 0x5640e8fa0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7367fff640 1 -- 172.21.15.26:0/3500794643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8fe5990 msgr2=0x5640e8fa75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7367fff640 1 --2- 172.21.15.26:0/3500794643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8fe5990 0x5640e8fa75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7367fff640 1 -- 172.21.15.26:0/3500794643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7370165f30 con 0x5640e8fe5070 2023-12-19T16:58:34.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7367fff640 1 --2- 172.21.15.26:0/3500794643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e8fa7080 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f7358014420 tx=0x7f73580148e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b974ee8471dd49a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f734dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f734dffb640 1 -- 172.21.15.26:0/3500794643 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358016900 con 0x5640e8fe5070 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f734dffb640 1 -- 172.21.15.26:0/3500794643 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f7358016a60 con 0x5640e8fe5070 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 -- 172.21.15.26:0/3500794643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640e8db8c10 con 0x5640e8fe5070 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f734dffb640 1 -- 172.21.15.26:0/3500794643 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7358016c20 con 0x5640e8fe5070 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 10 client.7605 fetch_fsmap learned FSMap version 49 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 -- 172.21.15.26:0/3500794643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5640e8dae260 con 0x5640e8fe5070 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f734dffb640 1 -- 172.21.15.26:0/3500794643 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640e8db8c10 con 0x5640e8fe5070 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f734dffb640 1 -- 172.21.15.26:0/3500794643 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f73580059a0 con 0x5640e8fe5070 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 10 client.7605 fetch_fsmap finished waiting for FSMap version 49 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 10 client.7605 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 20 client.7605 populate_metadata read hostname 'smithi026' 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 --2- 172.21.15.26:0/3500794643 >> [v2:172.21.15.26:6833/2693922436,v1:172.21.15.26:6835/2693922436] conn(0x5640e8fec8d0 0x5640e8feccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 4 client.7605 mds_command: new command op to 7479 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T16:58:34.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 -- 172.21.15.26:0/3500794643 --> [v2:172.21.15.26:6833/2693922436,v1:172.21.15.26:6835/2693922436] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640e8de7a30 con 0x5640e8fec8d0 2023-12-19T16:58:34.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f736effd640 1 --2- 172.21.15.26:0/3500794643 >> [v2:172.21.15.26:6833/2693922436,v1:172.21.15.26:6835/2693922436] conn(0x5640e8fec8d0 0x5640e8feccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f736effd640 1 --2- 172.21.15.26:0/3500794643 >> [v2:172.21.15.26:6833/2693922436,v1:172.21.15.26:6835/2693922436] conn(0x5640e8fec8d0 0x5640e8feccb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a76d07bf3091b3c server_cookie=48f1a2a4bc148278 in_seq=0 out_seq=0 2023-12-19T16:58:34.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f734dffb640 10 client.7605 ms_handle_connect on v2:172.21.15.26:6833/2693922436 2023-12-19T16:58:34.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f734dffb640 1 -- 172.21.15.26:0/3500794643 <== mds.0 v2:172.21.15.26:6833/2693922436 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5640e8de7a30 con 0x5640e8fec8d0 2023-12-19T16:58:34.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f734dffb640 10 client.7605 handle_command_reply: tid=0 2023-12-19T16:58:34.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 client.7605 shutdown 2023-12-19T16:58:34.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 -- 172.21.15.26:0/3500794643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 msgr2=0x5640e8fa7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 --2- 172.21.15.26:0/3500794643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e8fa7080 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f7358014420 tx=0x7f73580148e0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 -- 172.21.15.26:0/3500794643 shutdown_connections 2023-12-19T16:58:34.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 --2- 172.21.15.26:0/3500794643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8fe5990 0x5640e8fa75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 --2- 172.21.15.26:0/3500794643 >> [v2:172.21.15.26:6833/2693922436,v1:172.21.15.26:6835/2693922436] conn(0x5640e8fec8d0 0x5640e8feccb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 --2- 172.21.15.26:0/3500794643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e8fa7080 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 --2- 172.21.15.26:0/3500794643 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8f9b1c0 0x5640e8fa0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 -- 172.21.15.26:0/3500794643 >> 172.21.15.26:0/3500794643 conn(0x5640e8fa1950 msgr2=0x5640e8f99c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f734dffb640 0 client.7605 ms_handle_reset on v2:172.21.15.26:6833/2693922436 2023-12-19T16:58:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 -- 172.21.15.26:0/3500794643 shutdown_connections 2023-12-19T16:58:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 1 -- 172.21.15.26:0/3500794643 wait complete. 2023-12-19T16:58:34.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.276+0000 7f7378795000 20 client.7605 trim_cache size 0 max 16384 2023-12-19T16:58:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.292+0000 7f7378795000 1 Processor -- start 2023-12-19T16:58:34.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.292+0000 7f7378795000 1 -- start start 2023-12-19T16:58:34.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.292+0000 7f7378795000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fa1950 0x5640e903ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.292+0000 7f7378795000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8f9b1c0 0x5640e903d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.292+0000 7f7378795000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e903d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.292+0000 7f7378795000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8aa6940 con 0x5640e8fe5070 2023-12-19T16:58:34.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.292+0000 7f7378795000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8db6a80 con 0x5640e8f9b1c0 2023-12-19T16:58:34.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.292+0000 7f7378795000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640e8d8e770 con 0x5640e8fa1950 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.292+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e903d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.292+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e903d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46710/0 (socket says 172.21.15.26:46710) 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.292+0000 7f736f7fe640 1 -- 172.21.15.26:0/2304198788 learned_addr learned my addr 172.21.15.26:0/2304198788 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7367fff640 1 --2- 172.21.15.26:0/2304198788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8f9b1c0 0x5640e903d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736f7fe640 1 -- 172.21.15.26:0/2304198788 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fa1950 msgr2=0x5640e903ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736effd640 1 --2- 172.21.15.26:0/2304198788 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fa1950 0x5640e903ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736f7fe640 1 --2- 172.21.15.26:0/2304198788 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fa1950 0x5640e903ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736f7fe640 1 -- 172.21.15.26:0/2304198788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8f9b1c0 msgr2=0x5640e903d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736f7fe640 1 --2- 172.21.15.26:0/2304198788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8f9b1c0 0x5640e903d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736f7fe640 1 -- 172.21.15.26:0/2304198788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640e8d943b0 con 0x5640e8fe5070 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7367fff640 1 --2- 172.21.15.26:0/2304198788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8f9b1c0 0x5640e903d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736effd640 1 --2- 172.21.15.26:0/2304198788 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fa1950 0x5640e903ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736f7fe640 1 --2- 172.21.15.26:0/2304198788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e903d950 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f7368054e00 tx=0x7f7368021030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460de0b2d9ff998d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:34.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f73657fa640 1 -- 172.21.15.26:0/2304198788 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736805cf90 con 0x5640e8fe5070 2023-12-19T16:58:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f73657fa640 1 -- 172.21.15.26:0/2304198788 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7368053350 con 0x5640e8fe5070 2023-12-19T16:58:34.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f73657fa640 1 -- 172.21.15.26:0/2304198788 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368082070 con 0x5640e8fe5070 2023-12-19T16:58:34.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 -- 172.21.15.26:0/2304198788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 msgr2=0x5640e903d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 --2- 172.21.15.26:0/2304198788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e903d950 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f7368054e00 tx=0x7f7368021030 comp rx=0 tx=0).stop 2023-12-19T16:58:34.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 -- 172.21.15.26:0/2304198788 shutdown_connections 2023-12-19T16:58:34.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 --2- 172.21.15.26:0/2304198788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e903d950 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 --2- 172.21.15.26:0/2304198788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8f9b1c0 0x5640e903d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 --2- 172.21.15.26:0/2304198788 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fa1950 0x5640e903ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 -- 172.21.15.26:0/2304198788 >> 172.21.15.26:0/2304198788 conn(0x5640e8fa0d60 msgr2=0x5640e8fa6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:34.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 -- 172.21.15.26:0/2304198788 shutdown_connections 2023-12-19T16:58:34.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 -- 172.21.15.26:0/2304198788 wait complete. 2023-12-19T16:58:34.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 -- start start 2023-12-19T16:58:34.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 Processor -- start 2023-12-19T16:58:34.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8f9b1c0 0x5640e9025d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fa1950 0x5640e9026270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e90267b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8df14b0 con 0x5640e8fe5070 2023-12-19T16:58:34.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e8de64e0 con 0x5640e8f9b1c0 2023-12-19T16:58:34.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640e8de4a50 con 0x5640e8fa1950 2023-12-19T16:58:34.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7378795000 10 client.0 _ll_register_callbacks cb 0x5640e8d97640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T16:58:34.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e90267b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e90267b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46716/0 (socket says 172.21.15.26:46716) 2023-12-19T16:58:34.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736f7fe640 1 -- 172.21.15.26:0/2732645565 learned_addr learned my addr 172.21.15.26:0/2732645565 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:34.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736effd640 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8f9b1c0 0x5640e9025d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736f7fe640 1 -- 172.21.15.26:0/2732645565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fa1950 msgr2=0x5640e9026270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:34.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f7367fff640 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fa1950 0x5640e9026270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736f7fe640 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fa1950 0x5640e9026270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736f7fe640 1 -- 172.21.15.26:0/2732645565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8f9b1c0 msgr2=0x5640e9025d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:34.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.296+0000 7f736f7fe640 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8f9b1c0 0x5640e9025d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f736f7fe640 1 -- 172.21.15.26:0/2732645565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640e8ae5890 con 0x5640e8fe5070 2023-12-19T16:58:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f736effd640 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8f9b1c0 0x5640e9025d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7367fff640 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fa1950 0x5640e9026270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:58:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f736f7fe640 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e90267b0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7368054f40 tx=0x7f7368057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc30b8c694ba2e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f734dffb640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T16:58:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f734dffb640 1 -- 172.21.15.26:0/2732645565 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368082270 con 0x5640e8fe5070 2023-12-19T16:58:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f734dffb640 1 -- 172.21.15.26:0/2732645565 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f7368080070 con 0x5640e8fe5070 2023-12-19T16:58:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 -- 172.21.15.26:0/2732645565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640e8daee00 con 0x5640e8fe5070 2023-12-19T16:58:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f734dffb640 1 -- 172.21.15.26:0/2732645565 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73680544a0 con 0x5640e8fe5070 2023-12-19T16:58:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 10 client.7611 fetch_fsmap learned FSMap version 49 2023-12-19T16:58:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 -- 172.21.15.26:0/2732645565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5640e8ae5cc0 con 0x5640e8fe5070 2023-12-19T16:58:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f734dffb640 1 -- 172.21.15.26:0/2732645565 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640e8daee00 con 0x5640e8fe5070 2023-12-19T16:58:34.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f734dffb640 1 -- 172.21.15.26:0/2732645565 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 49) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7368005d20 con 0x5640e8fe5070 2023-12-19T16:58:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 10 client.7611 fetch_fsmap finished waiting for FSMap version 49 2023-12-19T16:58:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 10 client.7611 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T16:58:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 20 client.7611 populate_metadata read hostname 'smithi026' 2023-12-19T16:58:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.26:6833/2693922436,v1:172.21.15.26:6835/2693922436] conn(0x5640e8fe5990 0x5640e9026cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 4 client.7611 mds_command: new command op to 7479 tid=0[{"prefix": "status"}] 2023-12-19T16:58:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 -- 172.21.15.26:0/2732645565 --> [v2:172.21.15.26:6833/2693922436,v1:172.21.15.26:6835/2693922436] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5640e8de2160 con 0x5640e8fe5990 2023-12-19T16:58:34.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f736effd640 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.26:6833/2693922436,v1:172.21.15.26:6835/2693922436] conn(0x5640e8fe5990 0x5640e9026cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f736effd640 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.26:6833/2693922436,v1:172.21.15.26:6835/2693922436] conn(0x5640e8fe5990 0x5640e9026cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11384f334ace92b4 server_cookie=5a3e2313e1eac2f2 in_seq=0 out_seq=0 2023-12-19T16:58:34.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f734dffb640 10 client.7611 ms_handle_connect on v2:172.21.15.26:6833/2693922436 2023-12-19T16:58:34.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f734dffb640 1 -- 172.21.15.26:0/2732645565 <== mds.0 v2:172.21.15.26:6833/2693922436 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5640e8de2160 con 0x5640e8fe5990 2023-12-19T16:58:34.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f734dffb640 10 client.7611 handle_command_reply: tid=0 2023-12-19T16:58:34.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 client.7611 shutdown 2023-12-19T16:58:34.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 -- 172.21.15.26:0/2732645565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 msgr2=0x5640e90267b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e90267b0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7368054f40 tx=0x7f7368057040 comp rx=0 tx=0).stop 2023-12-19T16:58:34.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 -- 172.21.15.26:0/2732645565 shutdown_connections 2023-12-19T16:58:34.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.26:6833/2693922436,v1:172.21.15.26:6835/2693922436] conn(0x5640e8fe5990 0x5640e9026cf0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5640e8fe5070 0x5640e90267b0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5640e8fa1950 0x5640e9026270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 --2- 172.21.15.26:0/2732645565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5640e8f9b1c0 0x5640e9025d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f734dffb640 0 client.7611 ms_handle_reset on v2:172.21.15.26:6833/2693922436 2023-12-19T16:58:34.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 -- 172.21.15.26:0/2732645565 >> 172.21.15.26:0/2732645565 conn(0x5640e8fa0d60 msgr2=0x5640e8fe2060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:34.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 -- 172.21.15.26:0/2732645565 shutdown_connections 2023-12-19T16:58:34.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 1 -- 172.21.15.26:0/2732645565 wait complete. 2023-12-19T16:58:34.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f7378795000 20 client.7611 trim_cache size 0 max 16384 2023-12-19T16:58:34.308 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T16:58:34.308 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T16:58:34.308 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T16:58:34.308 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7479, 2023-12-19T16:58:34.308 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T16:58:34.308 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T16:58:34.308 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T16:58:34.309 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.935369133, 2023-12-19T16:58:34.309 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 49, 2023-12-19T16:58:34.309 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 107, 2023-12-19T16:58:34.309 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 107, 2023-12-19T16:58:34.309 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 10.270356233999999 2023-12-19T16:58:34.309 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T16:58:34.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f737542d640 1 -- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7350065b80 msgr2=0x7f7350068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f737542d640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7350065b80 0x7f7350068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f73600aa000 tx=0x7f7360001a10 comp rx=0 tx=0).stop 2023-12-19T16:58:34.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f737542d640 1 -- 172.21.15.26:0/4148157408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7370158390 msgr2=0x7f7370158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f737542d640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7370158390 0x7f7370158810 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f73700cc400 tx=0x7f73680115a0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f737542d640 1 -- 172.21.15.26:0/4148157408 shutdown_connections 2023-12-19T16:58:34.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f737542d640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7370158390 0x7f7370158810 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f737542d640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7350065b80 0x7f7350068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f737542d640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73701f8400 0x7f73701f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f737542d640 1 --2- 172.21.15.26:0/4148157408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c003980 0x7f737014ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f737542d640 1 -- 172.21.15.26:0/4148157408 >> 172.21.15.26:0/4148157408 conn(0x7f73700658e0 msgr2=0x7f73701591b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:34.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f737542d640 1 -- 172.21.15.26:0/4148157408 shutdown_connections 2023-12-19T16:58:34.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.300+0000 7f737542d640 1 -- 172.21.15.26:0/4148157408 wait complete. 2023-12-19T16:58:34.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T16:58:34.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15f6511640 1 Processor -- start 2023-12-19T16:58:34.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15f6511640 1 -- start start 2023-12-19T16:58:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15f6511640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15f015c7f0 0x7f15f006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15f6511640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f15f00adb80 0x7f15f00cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15f6511640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f006ced0 con 0x7f15f00ccc20 2023-12-19T16:58:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15f6511640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f006d770 con 0x7f15f015c7f0 2023-12-19T16:58:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15f6511640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f006d1b0 con 0x7f15f00adb80 2023-12-19T16:58:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15effff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15f015c7f0 0x7f15f006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15effff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15f015c7f0 0x7f15f006c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46718/0 (socket says 172.21.15.26:46718) 2023-12-19T16:58:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15effff640 1 -- 172.21.15.26:0/419200100 learned_addr learned my addr 172.21.15.26:0/419200100 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15ef7fe640 1 --2- 172.21.15.26:0/419200100 >> v2:172.21.15.160:3300/0 conn(0x7f15f00adb80 0x7f15f00cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15effff640 1 -- 172.21.15.26:0/419200100 >> v2:172.21.15.160:3300/0 conn(0x7f15f00adb80 msgr2=0x7f15f00cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15effff640 1 --2- 172.21.15.26:0/419200100 >> v2:172.21.15.160:3300/0 conn(0x7f15f00adb80 0x7f15f00cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15effff640 1 -- 172.21.15.26:0/419200100 >> v1:172.21.15.160:6789/0 conn(0x7f15f00ccc20 legacy=0x7f15f00b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.464+0000 7f15effff640 1 -- 172.21.15.26:0/419200100 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f006d490 con 0x7f15f015c7f0 2023-12-19T16:58:34.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.468+0000 7f15effff640 1 --2- 172.21.15.26:0/419200100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f015c7f0 0x7f15f006c7f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f15f00667b0 tx=0x7f15e001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae3474c333035da server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:34.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.468+0000 7f15eeffd640 1 -- 172.21.15.26:0/419200100 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00a8050 con 0x7f15f015c7f0 2023-12-19T16:58:34.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.468+0000 7f15eeffd640 1 -- 172.21.15.26:0/419200100 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e00ac5f0 con 0x7f15f015c7f0 2023-12-19T16:58:34.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.468+0000 7f15eeffd640 1 -- 172.21.15.26:0/419200100 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00ac980 con 0x7f15f015c7f0 2023-12-19T16:58:34.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.468+0000 7f15f6511640 1 -- 172.21.15.26:0/419200100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f015c7f0 msgr2=0x7f15f006c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.468+0000 7f15f6511640 1 --2- 172.21.15.26:0/419200100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f015c7f0 0x7f15f006c7f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f15f00667b0 tx=0x7f15e001c500 comp rx=0 tx=0).stop 2023-12-19T16:58:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.468+0000 7f15f6511640 1 -- 172.21.15.26:0/419200100 shutdown_connections 2023-12-19T16:58:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.468+0000 7f15f6511640 1 --2- 172.21.15.26:0/419200100 >> v2:172.21.15.160:3300/0 conn(0x7f15f00adb80 0x7f15f00cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.468+0000 7f15f6511640 1 --2- 172.21.15.26:0/419200100 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f015c7f0 0x7f15f006c7f0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.468+0000 7f15f6511640 1 -- 172.21.15.26:0/419200100 >> 172.21.15.26:0/419200100 conn(0x7f15f00658e0 msgr2=0x7f15f0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.468+0000 7f15f6511640 1 -- 172.21.15.26:0/419200100 shutdown_connections 2023-12-19T16:58:34.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.468+0000 7f15f6511640 1 -- 172.21.15.26:0/419200100 wait complete. 2023-12-19T16:58:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f6511640 1 Processor -- start 2023-12-19T16:58:34.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f6511640 1 -- start start 2023-12-19T16:58:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f6511640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15f00adb80 0x7f15f014ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f6511640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15f00ccc20 0x7f15f01f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f6511640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f015c7f0 0x7f15f01f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f6511640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f014a370 con 0x7f15f015c7f0 2023-12-19T16:58:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f6511640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f0149db0 con 0x7f15f00adb80 2023-12-19T16:58:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f6511640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f014a090 con 0x7f15f00ccc20 2023-12-19T16:58:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f4aae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f015c7f0 0x7f15f01f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f4aae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f015c7f0 0x7f15f01f75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46728/0 (socket says 172.21.15.26:46728) 2023-12-19T16:58:34.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f4aae640 1 -- 172.21.15.26:0/624080901 learned_addr learned my addr 172.21.15.26:0/624080901 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:34.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15ef7fe640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15f00ccc20 0x7f15f01f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15effff640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15f00adb80 0x7f15f014ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f4aae640 1 -- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15f00ccc20 msgr2=0x7f15f01f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f4aae640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15f00ccc20 0x7f15f01f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f4aae640 1 -- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15f00adb80 msgr2=0x7f15f014ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f4aae640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15f00adb80 0x7f15f014ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f4aae640 1 -- 172.21.15.26:0/624080901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e00094d0 con 0x7f15f015c7f0 2023-12-19T16:58:34.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15f4aae640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f015c7f0 0x7f15f01f75c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f15e800c7a0 tx=0x7f15e800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:34.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.472+0000 7f15ecff9640 1 -- 172.21.15.26:0/624080901 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e8003ba0 con 0x7f15f015c7f0 2023-12-19T16:58:34.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.476+0000 7f15ecff9640 1 -- 172.21.15.26:0/624080901 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e8003d00 con 0x7f15f015c7f0 2023-12-19T16:58:34.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.476+0000 7f15ecff9640 1 -- 172.21.15.26:0/624080901 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e800aa30 con 0x7f15f015c7f0 2023-12-19T16:58:34.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.476+0000 7f15f6511640 1 -- 172.21.15.26:0/624080901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f0150a50 con 0x7f15f015c7f0 2023-12-19T16:58:34.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.476+0000 7f15f6511640 1 -- 172.21.15.26:0/624080901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f0157b10 con 0x7f15f015c7f0 2023-12-19T16:58:34.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.476+0000 7f15f6511640 1 -- 172.21.15.26:0/624080901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f01508b0 con 0x7f15f015c7f0 2023-12-19T16:58:34.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.476+0000 7f15ecff9640 1 -- 172.21.15.26:0/624080901 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f15e800ab90 con 0x7f15f015c7f0 2023-12-19T16:58:34.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.476+0000 7f15ecff9640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f15dc065b80 0x7f15dc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.476+0000 7f15ecff9640 1 -- 172.21.15.26:0/624080901 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f15e80833f0 con 0x7f15f015c7f0 2023-12-19T16:58:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.476+0000 7f15effff640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f15dc065b80 0x7f15dc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:34.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.480+0000 7f15effff640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f15dc065b80 0x7f15dc068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f15e00ae000 tx=0x7f15e00a7000 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:34.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.480+0000 7f15ecff9640 1 -- 172.21.15.26:0/624080901 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f15f01508b0 con 0x7f15f015c7f0 2023-12-19T16:58:34.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.672+0000 7f15f6511640 1 -- 172.21.15.26:0/624080901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f15f01508b0 con 0x7f15f015c7f0 2023-12-19T16:58:34.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.672+0000 7f15ecff9640 1 -- 172.21.15.26:0/624080901 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7f15f01508b0 con 0x7f15f015c7f0 2023-12-19T16:58:34.675 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T16:58:34.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.676+0000 7f15f6511640 1 -- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f15dc065b80 msgr2=0x7f15dc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.676+0000 7f15f6511640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f15dc065b80 0x7f15dc068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f15e00ae000 tx=0x7f15e00a7000 comp rx=0 tx=0).stop 2023-12-19T16:58:34.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.676+0000 7f15f6511640 1 -- 172.21.15.26:0/624080901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f015c7f0 msgr2=0x7f15f01f75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:34.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.676+0000 7f15f6511640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f015c7f0 0x7f15f01f75c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f15e800c7a0 tx=0x7f15e800cc60 comp rx=0 tx=0).stop 2023-12-19T16:58:34.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.676+0000 7f15f6511640 1 -- 172.21.15.26:0/624080901 shutdown_connections 2023-12-19T16:58:34.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.676+0000 7f15f6511640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15f015c7f0 0x7f15f01f75c0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.676+0000 7f15f6511640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f15dc065b80 0x7f15dc068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.676+0000 7f15f6511640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15f00ccc20 0x7f15f01f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.676+0000 7f15f6511640 1 --2- 172.21.15.26:0/624080901 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15f00adb80 0x7f15f014ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:34.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.676+0000 7f15f6511640 1 -- 172.21.15.26:0/624080901 >> 172.21.15.26:0/624080901 conn(0x7f15f00658e0 msgr2=0x7f15f006fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:34.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.676+0000 7f15f6511640 1 -- 172.21.15.26:0/624080901 shutdown_connections 2023-12-19T16:58:34.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:34.676+0000 7f15f6511640 1 -- 172.21.15.26:0/624080901 wait complete. 2023-12-19T16:58:34.697 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-19T16:58:34.697 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T16:58:34.698 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T16:58:34.698 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T16:58:34.698 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T16:58:34.698 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-19T16:58:34.698 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-19T16:58:34.698 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T16:58:34.698 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:58:34.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:34.698 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T16:58:34.708 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T16:58:34.709 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T16:58:34.709 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T16:58:34.709 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:34.709 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T16:58:34.709 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:34.709 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:58:34.709 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T16:58:34.709 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T16:58:34.709 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:58:34.709 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T16:58:34.709 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T16:58:34.709 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 41426sec preferred_lft 41426sec 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T16:58:34.710 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T16:58:34.711 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T16:58:34.711 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:34.711 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-19T16:58:34.711 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:34.711 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T16:58:34.711 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T16:58:34.711 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-19T16:58:34.711 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:34.711 INFO:teuthology.orchestra.run.smithi026.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T16:58:34.711 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T16:58:34.711 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-19T16:58:34.711 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T16:58:34.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:34.712 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T16:58:34.770 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T16:58:34.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:34.773 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T16:58:34.825 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T16:58:34.825 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T16:58:34.825 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T16:58:34.825 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T16:58:34.825 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T16:58:34.825 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T16:58:34.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:34.826 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T16:58:34.826 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T16:58:34.826 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T16:58:34.826 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T16:58:34.826 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T16:58:34.826 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T16:58:34.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:34.920 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T16:58:34.929 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T16:58:34.930 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T16:58:34.930 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:58:34.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:34.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T16:58:34.992 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:58:34.992 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T16:58:35.044 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T16:58:35.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T16:58:35.044 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T16:58:35.096 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T16:58:35.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:35.096 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:58:35.191 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:58:35.191 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:58:35.204 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T16:58:35.204 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T16:58:35.204 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T16:58:35.204 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T16:58:35.204 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T16:58:35.208 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T16:58:35.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:35.208 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T16:58:35.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:58:35.230 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T16:58:35.289 INFO:teuthology.orchestra.run.smithi026.stdout:client7620 2023-12-19T16:58:35.289 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:58:35.289 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:58:35.340 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-19T16:58:35.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.528+0000 7f9c6a695640 1 Processor -- start 2023-12-19T16:58:35.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.528+0000 7f9c6a695640 1 -- start start 2023-12-19T16:58:35.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c6a695640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9c6406b3d0 0x7f9c640ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:35.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c6a695640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c640cec40 0x7f9c6415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:35.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c6a695640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c6406bc30 con 0x7f9c64154220 2023-12-19T16:58:35.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c6a695640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c640cf0d0 con 0x7f9c6406b3d0 2023-12-19T16:58:35.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c6a695640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640cc6e0 con 0x7f9c640cec40 2023-12-19T16:58:35.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c637fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9c6406b3d0 0x7f9c640ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:35.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c68c32640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c640cec40 0x7f9c6415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:35.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c68c32640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c640cec40 0x7f9c6415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:55844/0 (socket says 172.21.15.26:55844) 2023-12-19T16:58:35.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c637fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9c6406b3d0 0x7f9c640ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:42172/0 (socket says 172.21.15.26:42172) 2023-12-19T16:58:35.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c68c32640 1 -- 172.21.15.26:0/4209937405 learned_addr learned my addr 172.21.15.26:0/4209937405 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:35.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c63fff640 1 --1- 172.21.15.26:0/4209937405 >> v1:172.21.15.160:6789/0 conn(0x7f9c64154220 0x7f9c6415aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46404/0 (socket says 172.21.15.26:46404) 2023-12-19T16:58:35.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c637fe640 1 -- 172.21.15.26:0/4209937405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c640cec40 msgr2=0x7f9c6415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:35.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c637fe640 1 --2- 172.21.15.26:0/4209937405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c640cec40 0x7f9c6415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c637fe640 1 -- 172.21.15.26:0/4209937405 >> v1:172.21.15.160:6789/0 conn(0x7f9c64154220 legacy=0x7f9c6415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:35.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.532+0000 7f9c637fe640 1 -- 172.21.15.26:0/4209937405 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6406b950 con 0x7f9c6406b3d0 2023-12-19T16:58:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c637fe640 1 --2- 172.21.15.26:0/4209937405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c6406b3d0 0x7f9c640ce700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9c580097c0 tx=0x7f9c5801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71229f3e6b80bfdc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c62ffd640 1 -- 172.21.15.26:0/4209937405 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c580ab650 con 0x7f9c6406b3d0 2023-12-19T16:58:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c62ffd640 1 -- 172.21.15.26:0/4209937405 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c580ab7b0 con 0x7f9c6406b3d0 2023-12-19T16:58:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c62ffd640 1 -- 172.21.15.26:0/4209937405 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c580abb40 con 0x7f9c6406b3d0 2023-12-19T16:58:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c6a695640 1 -- 172.21.15.26:0/4209937405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c6406b3d0 msgr2=0x7f9c640ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c6a695640 1 --2- 172.21.15.26:0/4209937405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c6406b3d0 0x7f9c640ce700 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9c580097c0 tx=0x7f9c5801c450 comp rx=0 tx=0).stop 2023-12-19T16:58:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c6a695640 1 -- 172.21.15.26:0/4209937405 shutdown_connections 2023-12-19T16:58:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c6a695640 1 --2- 172.21.15.26:0/4209937405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c640cec40 0x7f9c6415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c6a695640 1 --2- 172.21.15.26:0/4209937405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c6406b3d0 0x7f9c640ce700 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c6a695640 1 -- 172.21.15.26:0/4209937405 >> 172.21.15.26:0/4209937405 conn(0x7f9c640658e0 msgr2=0x7f9c64065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c6a695640 1 -- 172.21.15.26:0/4209937405 shutdown_connections 2023-12-19T16:58:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c6a695640 1 -- 172.21.15.26:0/4209937405 wait complete. 2023-12-19T16:58:35.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c6a695640 1 Processor -- start 2023-12-19T16:58:35.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.536+0000 7f9c6a695640 1 -- start start 2023-12-19T16:58:35.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c6a695640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c6406b3d0 0x7f9c64070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:35.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c6a695640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c640cec40 0x7f9c64167190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:35.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c6a695640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c64154220 0x7f9c640b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:35.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c6a695640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c64071680 con 0x7f9c64154220 2023-12-19T16:58:35.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c6a695640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c64154690 con 0x7f9c6406b3d0 2023-12-19T16:58:35.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c6a695640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6414ddc0 con 0x7f9c640cec40 2023-12-19T16:58:35.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c68c32640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c64154220 0x7f9c640b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c68c32640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c64154220 0x7f9c640b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46734/0 (socket says 172.21.15.26:46734) 2023-12-19T16:58:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c68c32640 1 -- 172.21.15.26:0/555517995 learned_addr learned my addr 172.21.15.26:0/555517995 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c637fe640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c640cec40 0x7f9c64167190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c63fff640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c6406b3d0 0x7f9c64070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c68c32640 1 -- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c640cec40 msgr2=0x7f9c64167190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c68c32640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c640cec40 0x7f9c64167190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c68c32640 1 -- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c6406b3d0 msgr2=0x7f9c64070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c68c32640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c6406b3d0 0x7f9c64070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c68c32640 1 -- 172.21.15.26:0/555517995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c58009420 con 0x7f9c64154220 2023-12-19T16:58:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c68c32640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c64154220 0x7f9c640b1460 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00efc0 tx=0x7f9c5c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c60ff9640 1 -- 172.21.15.26:0/555517995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c00cd20 con 0x7f9c64154220 2023-12-19T16:58:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c60ff9640 1 -- 172.21.15.26:0/555517995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c00f040 con 0x7f9c64154220 2023-12-19T16:58:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c60ff9640 1 -- 172.21.15.26:0/555517995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c003be0 con 0x7f9c64154220 2023-12-19T16:58:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c6a695640 1 -- 172.21.15.26:0/555517995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c640713a0 con 0x7f9c64154220 2023-12-19T16:58:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c6a695640 1 -- 172.21.15.26:0/555517995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6414d5c0 con 0x7f9c64154220 2023-12-19T16:58:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c6a695640 1 -- 172.21.15.26:0/555517995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c64074270 con 0x7f9c64154220 2023-12-19T16:58:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.540+0000 7f9c60ff9640 1 -- 172.21.15.26:0/555517995 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9c5c003d40 con 0x7f9c64154220 2023-12-19T16:58:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.544+0000 7f9c60ff9640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c50065bb0 0x7f9c50068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.544+0000 7f9c60ff9640 1 -- 172.21.15.26:0/555517995 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f9c5c083c50 con 0x7f9c64154220 2023-12-19T16:58:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.544+0000 7f9c63fff640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c50065bb0 0x7f9c50068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:35.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.544+0000 7f9c63fff640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c50065bb0 0x7f9c50068070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9c54000f70 tx=0x7f9c54007e50 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:35.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.548+0000 7f9c60ff9640 1 -- 172.21.15.26:0/555517995 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9c5c089020 con 0x7f9c64154220 2023-12-19T16:58:35.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.728+0000 7f9c6a695640 1 -- 172.21.15.26:0/555517995 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c640ccec0 con 0x7f9c50065bb0 2023-12-19T16:58:35.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.732+0000 7f9c60ff9640 1 -- 172.21.15.26:0/555517995 <== mgr.6323 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f9c640ccec0 con 0x7f9c50065bb0 2023-12-19T16:58:35.733 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T16:58:35.733 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:58:35.733 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2023-12-19T16:58:35.733 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:58:35.734 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T16:58:35.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.732+0000 7f9c6a695640 1 -- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c50065bb0 msgr2=0x7f9c50068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:35.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.732+0000 7f9c6a695640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c50065bb0 0x7f9c50068070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9c54000f70 tx=0x7f9c54007e50 comp rx=0 tx=0).stop 2023-12-19T16:58:35.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.732+0000 7f9c6a695640 1 -- 172.21.15.26:0/555517995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c64154220 msgr2=0x7f9c640b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.732+0000 7f9c6a695640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c64154220 0x7f9c640b1460 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00efc0 tx=0x7f9c5c00c490 comp rx=0 tx=0).stop 2023-12-19T16:58:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.732+0000 7f9c6a695640 1 -- 172.21.15.26:0/555517995 shutdown_connections 2023-12-19T16:58:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.732+0000 7f9c6a695640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c64154220 0x7f9c640b1460 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.732+0000 7f9c6a695640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9c50065bb0 0x7f9c50068070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.732+0000 7f9c6a695640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9c640cec40 0x7f9c64167190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.732+0000 7f9c6a695640 1 --2- 172.21.15.26:0/555517995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c6406b3d0 0x7f9c64070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.732+0000 7f9c6a695640 1 -- 172.21.15.26:0/555517995 >> 172.21.15.26:0/555517995 conn(0x7f9c640658e0 msgr2=0x7f9c64065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.732+0000 7f9c6a695640 1 -- 172.21.15.26:0/555517995 shutdown_connections 2023-12-19T16:58:35.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.732+0000 7f9c6a695640 1 -- 172.21.15.26:0/555517995 wait complete. 2023-12-19T16:58:35.755 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-19T16:58:35.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28fb60640 1 Processor -- start 2023-12-19T16:58:35.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28fb60640 1 -- start start 2023-12-19T16:58:35.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28fb60640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2880cb860 0x7fc28806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:35.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28fb60640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc28815d530 0x7fc28815d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:35.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28fb60640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28815e240 con 0x7fc2881599e0 2023-12-19T16:58:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28fb60640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2880c9520 con 0x7fc2880cb860 2023-12-19T16:58:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28fb60640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2880c9800 con 0x7fc28815d530 2023-12-19T16:58:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28d8fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc2881599e0 0x7fc2880cb150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38102/0 (socket says 172.21.15.26:38102) 2023-12-19T16:58:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28d8fc640 1 -- 172.21.15.26:0/3269515708 learned_addr learned my addr 172.21.15.26:0/3269515708 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28d0fb640 1 --2- 172.21.15.26:0/3269515708 >> v2:172.21.15.26:3300/0 conn(0x7fc2880cb860 0x7fc28806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28e0fd640 1 --2- 172.21.15.26:0/3269515708 >> v2:172.21.15.160:3300/0 conn(0x7fc28815d530 0x7fc28815d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28c8fa640 1 -- 172.21.15.26:0/3269515708 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7216203 0 0) 0x7fc28815e240 con 0x7fc2881599e0 2023-12-19T16:58:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28c8fa640 1 -- 172.21.15.26:0/3269515708 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc274003530 con 0x7fc2881599e0 2023-12-19T16:58:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28d0fb640 1 -- 172.21.15.26:0/3269515708 >> v2:172.21.15.160:3300/0 conn(0x7fc28815d530 msgr2=0x7fc28815d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28d0fb640 1 --2- 172.21.15.26:0/3269515708 >> v2:172.21.15.160:3300/0 conn(0x7fc28815d530 0x7fc28815d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28d0fb640 1 -- 172.21.15.26:0/3269515708 >> v1:172.21.15.26:6789/0 conn(0x7fc2881599e0 legacy=0x7fc2880cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.900+0000 7fc28d0fb640 1 -- 172.21.15.26:0/3269515708 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880c9ae0 con 0x7fc2880cb860 2023-12-19T16:58:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.904+0000 7fc28d0fb640 1 --2- 172.21.15.26:0/3269515708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2880cb860 0x7fc28806f6d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc2880c9150 tx=0x7fc27801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ed6ded77e5b800 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:35.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.904+0000 7fc28c8fa640 1 -- 172.21.15.26:0/3269515708 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780ab680 con 0x7fc2880cb860 2023-12-19T16:58:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.904+0000 7fc28c8fa640 1 -- 172.21.15.26:0/3269515708 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2780ab7e0 con 0x7fc2880cb860 2023-12-19T16:58:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.904+0000 7fc28c8fa640 1 -- 172.21.15.26:0/3269515708 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780abb70 con 0x7fc2880cb860 2023-12-19T16:58:35.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.904+0000 7fc28fb60640 1 -- 172.21.15.26:0/3269515708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2880cb860 msgr2=0x7fc28806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.904+0000 7fc28fb60640 1 --2- 172.21.15.26:0/3269515708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2880cb860 0x7fc28806f6d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc2880c9150 tx=0x7fc27801c460 comp rx=0 tx=0).stop 2023-12-19T16:58:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.904+0000 7fc28fb60640 1 -- 172.21.15.26:0/3269515708 shutdown_connections 2023-12-19T16:58:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.904+0000 7fc28fb60640 1 --2- 172.21.15.26:0/3269515708 >> v2:172.21.15.160:3300/0 conn(0x7fc28815d530 0x7fc28815d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.904+0000 7fc28fb60640 1 --2- 172.21.15.26:0/3269515708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2880cb860 0x7fc28806f6d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.904+0000 7fc28fb60640 1 -- 172.21.15.26:0/3269515708 >> 172.21.15.26:0/3269515708 conn(0x7fc288066160 msgr2=0x7fc2880b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.904+0000 7fc28fb60640 1 -- 172.21.15.26:0/3269515708 shutdown_connections 2023-12-19T16:58:35.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.904+0000 7fc28fb60640 1 -- 172.21.15.26:0/3269515708 wait complete. 2023-12-19T16:58:35.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28fb60640 1 Processor -- start 2023-12-19T16:58:35.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28fb60640 1 -- start start 2023-12-19T16:58:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28fb60640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2880cb860 0x7fc288153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28fb60640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2881599e0 0x7fc288155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28fb60640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc28815d530 0x7fc28806c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28fb60640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288156df0 con 0x7fc28815d530 2023-12-19T16:58:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28fb60640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880c33f0 con 0x7fc2880cb860 2023-12-19T16:58:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28fb60640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc288156830 con 0x7fc2881599e0 2023-12-19T16:58:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28e0fd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc28815d530 0x7fc28806c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:35.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28e0fd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc28815d530 0x7fc28806c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56482/0 (socket says 172.21.15.26:56482) 2023-12-19T16:58:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28e0fd640 1 -- 172.21.15.26:0/3550657276 learned_addr learned my addr 172.21.15.26:0/3550657276 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28d0fb640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2881599e0 0x7fc288155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28d8fc640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2880cb860 0x7fc288153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28e0fd640 1 -- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2881599e0 msgr2=0x7fc288155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28e0fd640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2881599e0 0x7fc288155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28e0fd640 1 -- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2880cb860 msgr2=0x7fc288153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28e0fd640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2880cb860 0x7fc288153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28e0fd640 1 -- 172.21.15.26:0/3550657276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc278009420 con 0x7fc28815d530 2023-12-19T16:58:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28e0fd640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc28815d530 0x7fc28806c180 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc280007e60 tx=0x7fc28000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc27e7fc640 1 -- 172.21.15.26:0/3550657276 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280014420 con 0x7fc28815d530 2023-12-19T16:58:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc27e7fc640 1 -- 172.21.15.26:0/3550657276 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2800145a0 con 0x7fc28815d530 2023-12-19T16:58:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc27e7fc640 1 -- 172.21.15.26:0/3550657276 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280014950 con 0x7fc28815d530 2023-12-19T16:58:35.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28fb60640 1 -- 172.21.15.26:0/3550657276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc288156b10 con 0x7fc28815d530 2023-12-19T16:58:35.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28fb60640 1 -- 172.21.15.26:0/3550657276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2880c3920 con 0x7fc28815d530 2023-12-19T16:58:35.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.908+0000 7fc28fb60640 1 -- 172.21.15.26:0/3550657276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2880c3b10 con 0x7fc28815d530 2023-12-19T16:58:35.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.912+0000 7fc27e7fc640 1 -- 172.21.15.26:0/3550657276 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc280014ab0 con 0x7fc28815d530 2023-12-19T16:58:35.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.912+0000 7fc27e7fc640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc26c065b80 0x7fc26c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:35.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.912+0000 7fc27e7fc640 1 -- 172.21.15.26:0/3550657276 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fc280083f90 con 0x7fc28815d530 2023-12-19T16:58:35.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.912+0000 7fc28d8fc640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc26c065b80 0x7fc26c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:35.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.912+0000 7fc28d8fc640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc26c065b80 0x7fc26c068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc2840011f0 tx=0x7fc284009040 comp rx=0 tx=0).ready entity=mgr.6323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:35.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:35.916+0000 7fc27e7fc640 1 -- 172.21.15.26:0/3550657276 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc2800514c0 con 0x7fc28815d530 2023-12-19T16:58:36.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.112+0000 7fc28fb60640 1 -- 172.21.15.26:0/3550657276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc288074270 con 0x7fc28815d530 2023-12-19T16:58:36.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.564+0000 7fc27e7fc640 1 -- 172.21.15.26:0/3550657276 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 47) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fc280051890 con 0x7fc28815d530 2023-12-19T16:58:36.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.568+0000 7fc27e7fc640 1 -- 172.21.15.26:0/3550657276 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v47) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc28007f810 con 0x7fc28815d530 2023-12-19T16:58:36.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.568+0000 7fc28fb60640 1 -- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc26c065b80 msgr2=0x7fc26c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:36.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.568+0000 7fc28fb60640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc26c065b80 0x7fc26c068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc2840011f0 tx=0x7fc284009040 comp rx=0 tx=0).stop 2023-12-19T16:58:36.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.568+0000 7fc28fb60640 1 -- 172.21.15.26:0/3550657276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc28815d530 msgr2=0x7fc28806c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:36.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.568+0000 7fc28fb60640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc28815d530 0x7fc28806c180 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc280007e60 tx=0x7fc28000e4b0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.568+0000 7fc28fb60640 1 -- 172.21.15.26:0/3550657276 shutdown_connections 2023-12-19T16:58:36.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.568+0000 7fc28fb60640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc28815d530 0x7fc28806c180 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.568+0000 7fc28fb60640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc26c065b80 0x7fc26c068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.568+0000 7fc28fb60640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2881599e0 0x7fc288155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.568+0000 7fc28fb60640 1 --2- 172.21.15.26:0/3550657276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc2880cb860 0x7fc288153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.568+0000 7fc28fb60640 1 -- 172.21.15.26:0/3550657276 >> 172.21.15.26:0/3550657276 conn(0x7fc288066160 msgr2=0x7fc2880cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:36.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.568+0000 7fc28fb60640 1 -- 172.21.15.26:0/3550657276 shutdown_connections 2023-12-19T16:58:36.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.568+0000 7fc28fb60640 1 -- 172.21.15.26:0/3550657276 wait complete. 2023-12-19T16:58:36.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-19T16:58:36.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.740+0000 7fe5a908c640 1 Processor -- start 2023-12-19T16:58:36.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.740+0000 7fe5a908c640 1 -- start start 2023-12-19T16:58:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe5a908c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5a40b46c0 0x7fe5a40b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe5a908c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe5a40b1600 0x7fe5a40b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe5a908c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a40b5c00 con 0x7fe5a415cd20 2023-12-19T16:58:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe5a908c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a40b5080 con 0x7fe5a40b1600 2023-12-19T16:58:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe5a908c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a415d440 con 0x7fe5a40b46c0 2023-12-19T16:58:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe5a359e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe5a40b1600 0x7fe5a40b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe59bfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5a40b46c0 0x7fe5a40b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe5a359e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe5a40b1600 0x7fe5a40b1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:41688/0 (socket says 172.21.15.26:41688) 2023-12-19T16:58:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe5a359e640 1 -- 172.21.15.26:0/2385650007 learned_addr learned my addr 172.21.15.26:0/2385650007 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:36.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe5a359e640 1 -- 172.21.15.26:0/2385650007 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5a40b46c0 msgr2=0x7fe5a40b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:36.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe5a359e640 1 --2- 172.21.15.26:0/2385650007 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5a40b46c0 0x7fe5a40b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe5a359e640 1 -- 172.21.15.26:0/2385650007 >> v1:172.21.15.160:6789/0 conn(0x7fe5a415cd20 legacy=0x7fe5a40b8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe5a359e640 1 -- 172.21.15.26:0/2385650007 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a415d6e0 con 0x7fe5a40b1600 2023-12-19T16:58:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe59bfff640 1 --2- 172.21.15.26:0/2385650007 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5a40b46c0 0x7fe5a40b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.744+0000 7fe5a359e640 1 --2- 172.21.15.26:0/2385650007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5a40b1600 0x7fe5a40b1a90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009e50 tx=0x7fe59c01f1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c79562c322bbb2a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a259c640 1 -- 172.21.15.26:0/2385650007 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0af400 con 0x7fe5a40b1600 2023-12-19T16:58:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a259c640 1 -- 172.21.15.26:0/2385650007 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59c0ab050 con 0x7fe5a40b1600 2023-12-19T16:58:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a259c640 1 -- 172.21.15.26:0/2385650007 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0af780 con 0x7fe5a40b1600 2023-12-19T16:58:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 -- 172.21.15.26:0/2385650007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5a40b1600 msgr2=0x7fe5a40b1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 --2- 172.21.15.26:0/2385650007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5a40b1600 0x7fe5a40b1a90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009e50 tx=0x7fe59c01f1f0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 -- 172.21.15.26:0/2385650007 shutdown_connections 2023-12-19T16:58:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 --2- 172.21.15.26:0/2385650007 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5a40b1600 0x7fe5a40b1a90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 --2- 172.21.15.26:0/2385650007 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5a40b46c0 0x7fe5a40b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 -- 172.21.15.26:0/2385650007 >> 172.21.15.26:0/2385650007 conn(0x7fe5a406a120 msgr2=0x7fe5a406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 -- 172.21.15.26:0/2385650007 shutdown_connections 2023-12-19T16:58:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 -- 172.21.15.26:0/2385650007 wait complete. 2023-12-19T16:58:36.749 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:36.748+0000 7ff24156e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:58:36.750 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:36.746+0000 7f74e55d4180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:58:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 Processor -- start 2023-12-19T16:58:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 -- start start 2023-12-19T16:58:36.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5a40b1600 0x7fe5a40c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:36.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5a40b46c0 0x7fe5a4156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:36.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a415cd20 0x7fe5a4157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:36.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a40cacc0 con 0x7fe5a415cd20 2023-12-19T16:58:36.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a40ca380 con 0x7fe5a40b46c0 2023-12-19T16:58:36.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.748+0000 7fe5a908c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a40c79a0 con 0x7fe5a40b1600 2023-12-19T16:58:36.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a415cd20 0x7fe5a4157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:36.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a415cd20 0x7fe5a4157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56500/0 (socket says 172.21.15.26:56500) 2023-12-19T16:58:36.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a359e640 1 -- 172.21.15.26:0/4068581763 learned_addr learned my addr 172.21.15.26:0/4068581763 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a2d9d640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5a40b1600 0x7fe5a40c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe59bfff640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5a40b46c0 0x7fe5a4156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a2d9d640 1 -- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5a40b46c0 msgr2=0x7fe5a4156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a2d9d640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5a40b46c0 0x7fe5a4156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a2d9d640 1 -- 172.21.15.26:0/4068581763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a415cd20 msgr2=0x7fe5a4157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a2d9d640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a415cd20 0x7fe5a4157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a2d9d640 1 -- 172.21.15.26:0/4068581763 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe58c009420 con 0x7fe5a40b1600 2023-12-19T16:58:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a359e640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a415cd20 0x7fe5a4157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a2d9d640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5a40b1600 0x7fe5a40c80b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe594000f70 tx=0x7fe59400c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe59b7fe640 1 -- 172.21.15.26:0/4068581763 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5940052d0 con 0x7fe5a40b1600 2023-12-19T16:58:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe59b7fe640 1 -- 172.21.15.26:0/4068581763 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe594005430 con 0x7fe5a40b1600 2023-12-19T16:58:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a908c640 1 -- 172.21.15.26:0/4068581763 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59c009b30 con 0x7fe5a40b1600 2023-12-19T16:58:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe59b7fe640 1 -- 172.21.15.26:0/4068581763 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe594003840 con 0x7fe5a40b1600 2023-12-19T16:58:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a908c640 1 -- 172.21.15.26:0/4068581763 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a40ca8e0 con 0x7fe5a40b1600 2023-12-19T16:58:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe5a908c640 1 -- 172.21.15.26:0/4068581763 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a4074270 con 0x7fe5a40b1600 2023-12-19T16:58:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.752+0000 7fe59b7fe640 1 -- 172.21.15.26:0/4068581763 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 47) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fe5940039a0 con 0x7fe5a40b1600 2023-12-19T16:58:36.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.756+0000 7fe59b7fe640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe584065bd0 0x7fe584068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:36.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.756+0000 7fe59b7fe640 1 -- 172.21.15.26:0/4068581763 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fe594082ac0 con 0x7fe5a40b1600 2023-12-19T16:58:36.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.756+0000 7fe59bfff640 1 -- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe584065bd0 msgr2=0x7fe584068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:36.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.756+0000 7fe59bfff640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe584065bd0 0x7fe584068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.760+0000 7fe59b7fe640 1 -- 172.21.15.26:0/4068581763 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe5940501d0 con 0x7fe5a40b1600 2023-12-19T16:58:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.920+0000 7fe5a908c640 1 -- 172.21.15.26:0/4068581763 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fe5a40b7b60 con 0x7fe5a40b1600 2023-12-19T16:58:36.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.928+0000 7fe59b7fe640 1 -- 172.21.15.26:0/4068581763 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe5940170d0 con 0x7fe5a40b1600 2023-12-19T16:58:36.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.928+0000 7fe5a908c640 1 -- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe584065bd0 msgr2=0x7fe584068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:36.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.928+0000 7fe5a908c640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe584065bd0 0x7fe584068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.928+0000 7fe5a908c640 1 -- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5a40b1600 msgr2=0x7fe5a40c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:36.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.928+0000 7fe5a908c640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5a40b1600 0x7fe5a40c80b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe594000f70 tx=0x7fe59400c820 comp rx=0 tx=0).stop 2023-12-19T16:58:36.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.928+0000 7fe5a908c640 1 -- 172.21.15.26:0/4068581763 shutdown_connections 2023-12-19T16:58:36.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.928+0000 7fe5a908c640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5a415cd20 0x7fe5a4157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.928+0000 7fe5a908c640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5a40b46c0 0x7fe5a4156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.928+0000 7fe5a908c640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe584065bd0 0x7fe584068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.928+0000 7fe5a908c640 1 --2- 172.21.15.26:0/4068581763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5a40b1600 0x7fe5a40c80b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:36.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.928+0000 7fe5a908c640 1 -- 172.21.15.26:0/4068581763 >> 172.21.15.26:0/4068581763 conn(0x7fe5a406a120 msgr2=0x7fe5a40b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:36.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.928+0000 7fe5a908c640 1 -- 172.21.15.26:0/4068581763 shutdown_connections 2023-12-19T16:58:36.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:36.928+0000 7fe5a908c640 1 -- 172.21.15.26:0/4068581763 wait complete. 2023-12-19T16:58:36.951 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-19T16:58:37.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.100+0000 7effc542d640 1 Processor -- start 2023-12-19T16:58:37.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.100+0000 7effc542d640 1 -- start start 2023-12-19T16:58:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.100+0000 7effc542d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7effc0159820 0x7effc015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.100+0000 7effc542d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effc0159c00 0x7effc0070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.100+0000 7effc542d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc006fb60 con 0x7effc006f6c0 2023-12-19T16:58:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.100+0000 7effc542d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc015e470 con 0x7effc0159820 2023-12-19T16:58:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.100+0000 7effc542d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc006fe40 con 0x7effc0159c00 2023-12-19T16:58:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.100+0000 7effbeffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7effc0159820 0x7effc015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.100+0000 7effb7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7effc006f6c0 0x7effc00c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38120/0 (socket says 172.21.15.26:38120) 2023-12-19T16:58:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.100+0000 7effb7fff640 1 -- 172.21.15.26:0/1441799141 learned_addr learned my addr 172.21.15.26:0/1441799141 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.100+0000 7effbf7fe640 1 --2- 172.21.15.26:0/1441799141 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effc0159c00 0x7effc0070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.104+0000 7effbeffd640 1 -- 172.21.15.26:0/1441799141 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effc0159c00 msgr2=0x7effc0070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.104+0000 7effbeffd640 1 --2- 172.21.15.26:0/1441799141 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effc0159c00 0x7effc0070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.104+0000 7effbeffd640 1 -- 172.21.15.26:0/1441799141 >> v1:172.21.15.26:6789/0 conn(0x7effc006f6c0 legacy=0x7effc00c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:37.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.104+0000 7effbeffd640 1 -- 172.21.15.26:0/1441799141 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc0070120 con 0x7effc0159820 2023-12-19T16:58:37.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.104+0000 7effbeffd640 1 --2- 172.21.15.26:0/1441799141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc0159820 0x7effc015df30 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7effb00097c0 tx=0x7effb001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e555718503d06f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:37.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.104+0000 7effbe7fc640 1 -- 172.21.15.26:0/1441799141 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb00ad660 con 0x7effc0159820 2023-12-19T16:58:37.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.104+0000 7effbe7fc640 1 -- 172.21.15.26:0/1441799141 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb00ad7c0 con 0x7effc0159820 2023-12-19T16:58:37.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.104+0000 7effbe7fc640 1 -- 172.21.15.26:0/1441799141 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb00adb50 con 0x7effc0159820 2023-12-19T16:58:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 -- 172.21.15.26:0/1441799141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc0159820 msgr2=0x7effc015df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 --2- 172.21.15.26:0/1441799141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc0159820 0x7effc015df30 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7effb00097c0 tx=0x7effb001c450 comp rx=0 tx=0).stop 2023-12-19T16:58:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 -- 172.21.15.26:0/1441799141 shutdown_connections 2023-12-19T16:58:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 --2- 172.21.15.26:0/1441799141 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effc0159c00 0x7effc0070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 --2- 172.21.15.26:0/1441799141 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc0159820 0x7effc015df30 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 -- 172.21.15.26:0/1441799141 >> 172.21.15.26:0/1441799141 conn(0x7effc00ce730 msgr2=0x7effc00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 -- 172.21.15.26:0/1441799141 shutdown_connections 2023-12-19T16:58:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 -- 172.21.15.26:0/1441799141 wait complete. 2023-12-19T16:58:37.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 Processor -- start 2023-12-19T16:58:37.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 -- start start 2023-12-19T16:58:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effc006f6c0 0x7effc0143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc0159820 0x7effc00c2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc0159c00 0x7effc006ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc0156180 con 0x7effc0159820 2023-12-19T16:58:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc0156460 con 0x7effc0159c00 2023-12-19T16:58:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effc542d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc0156740 con 0x7effc006f6c0 2023-12-19T16:58:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc0159820 0x7effc00c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effbeffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effc006f6c0 0x7effc0143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effb7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc0159820 0x7effc00c2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56514/0 (socket says 172.21.15.26:56514) 2023-12-19T16:58:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effb7fff640 1 -- 172.21.15.26:0/4170832457 learned_addr learned my addr 172.21.15.26:0/4170832457 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effbf7fe640 1 --2- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc0159c00 0x7effc006ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effbeffd640 1 -- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc0159c00 msgr2=0x7effc006ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effbeffd640 1 --2- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc0159c00 0x7effc006ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effbeffd640 1 -- 172.21.15.26:0/4170832457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc0159820 msgr2=0x7effc00c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effbeffd640 1 --2- 172.21.15.26:0/4170832457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc0159820 0x7effc00c2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effbeffd640 1 -- 172.21.15.26:0/4170832457 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb0009420 con 0x7effc006f6c0 2023-12-19T16:58:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.108+0000 7effb7fff640 1 --2- 172.21.15.26:0/4170832457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc0159820 0x7effc00c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:58:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.112+0000 7effbeffd640 1 --2- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effc006f6c0 0x7effc0143d30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7effb00ac000 tx=0x7effb00191f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.112+0000 7effb77fe640 1 -- 172.21.15.26:0/4170832457 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb00b6610 con 0x7effc006f6c0 2023-12-19T16:58:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.112+0000 7effb77fe640 1 -- 172.21.15.26:0/4170832457 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb00b67b0 con 0x7effc006f6c0 2023-12-19T16:58:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.112+0000 7effb77fe640 1 -- 172.21.15.26:0/4170832457 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb00b6b60 con 0x7effc006f6c0 2023-12-19T16:58:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.112+0000 7effc542d640 1 -- 172.21.15.26:0/4170832457 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc006b880 con 0x7effc006f6c0 2023-12-19T16:58:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.112+0000 7effc542d640 1 -- 172.21.15.26:0/4170832457 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc0156d00 con 0x7effc006f6c0 2023-12-19T16:58:37.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.112+0000 7effc542d640 1 -- 172.21.15.26:0/4170832457 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc0074270 con 0x7effc006f6c0 2023-12-19T16:58:37.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.112+0000 7effb77fe640 1 -- 172.21.15.26:0/4170832457 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 47) v1 ==== 85347+0+0 (secure 0 0 0) 0x7effb00b4040 con 0x7effc006f6c0 2023-12-19T16:58:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.112+0000 7effb77fe640 1 --2- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7effa0065bd0 0x7effa0068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.112+0000 7effb77fe640 1 -- 172.21.15.26:0/4170832457 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7effb011da60 con 0x7effc006f6c0 2023-12-19T16:58:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.112+0000 7effb7fff640 1 -- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7effa0065bd0 msgr2=0x7effa0068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.112+0000 7effb7fff640 1 --2- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7effa0065bd0 0x7effa0068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:37.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.116+0000 7effb77fe640 1 -- 172.21.15.26:0/4170832457 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7effb00eb0e0 con 0x7effc006f6c0 2023-12-19T16:58:37.165 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:37.162+0000 7f74e55d4180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:58:37.199 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:37.196+0000 7ff24156e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:58:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.280+0000 7effc542d640 1 -- 172.21.15.26:0/4170832457 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7effc00cbf70 con 0x7effc006f6c0 2023-12-19T16:58:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.284+0000 7effb77fe640 1 -- 172.21.15.26:0/4170832457 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7effb00ee700 con 0x7effc006f6c0 2023-12-19T16:58:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.288+0000 7effc542d640 1 -- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7effa0065bd0 msgr2=0x7effa0068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.288+0000 7effc542d640 1 --2- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7effa0065bd0 0x7effa0068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.288+0000 7effc542d640 1 -- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effc006f6c0 msgr2=0x7effc0143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.288+0000 7effc542d640 1 --2- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effc006f6c0 0x7effc0143d30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7effb00ac000 tx=0x7effb00191f0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.288+0000 7effc542d640 1 -- 172.21.15.26:0/4170832457 shutdown_connections 2023-12-19T16:58:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.288+0000 7effc542d640 1 --2- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effc0159c00 0x7effc006ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.288+0000 7effc542d640 1 --2- 172.21.15.26:0/4170832457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effc0159820 0x7effc00c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.288+0000 7effc542d640 1 --2- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7effa0065bd0 0x7effa0068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.288+0000 7effc542d640 1 --2- 172.21.15.26:0/4170832457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effc006f6c0 0x7effc0143d30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.288+0000 7effc542d640 1 -- 172.21.15.26:0/4170832457 >> 172.21.15.26:0/4170832457 conn(0x7effc00ce730 msgr2=0x7effc00ad100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:37.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.288+0000 7effc542d640 1 -- 172.21.15.26:0/4170832457 shutdown_connections 2023-12-19T16:58:37.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.288+0000 7effc542d640 1 -- 172.21.15.26:0/4170832457 wait complete. 2023-12-19T16:58:37.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /bad-path 1M 2023-12-19T16:58:37.315 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:37.310+0000 7f74e55d4180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:58:37.360 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:37.360+0000 7ff24156e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:58:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.456+0000 7f0ac497a640 1 Processor -- start 2023-12-19T16:58:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.456+0000 7f0ac497a640 1 -- start start 2023-12-19T16:58:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.456+0000 7f0ac497a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ac015d070 0x7f0ac00cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.456+0000 7f0ac497a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ac0160a20 0x7f0ac00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.456+0000 7f0ac497a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac00cdb20 con 0x7f0ac015cc90 2023-12-19T16:58:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.456+0000 7f0ac497a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac00cfe40 con 0x7f0ac0160a20 2023-12-19T16:58:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.456+0000 7f0ac497a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00cde00 con 0x7f0ac015d070 2023-12-19T16:58:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.456+0000 7f0abf59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ac0160a20 0x7f0ac00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:37.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.456+0000 7f0abe59c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ac015d070 0x7f0ac00cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:37.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.456+0000 7f0abf59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ac0160a20 0x7f0ac00cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56528/0 (socket says 172.21.15.26:56528) 2023-12-19T16:58:37.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.456+0000 7f0abf59e640 1 -- 172.21.15.26:0/1186539462 learned_addr learned my addr 172.21.15.26:0/1186539462 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:37.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.460+0000 7f0abf59e640 1 -- 172.21.15.26:0/1186539462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ac015d070 msgr2=0x7f0ac00cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:37.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.460+0000 7f0abf59e640 1 --2- 172.21.15.26:0/1186539462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ac015d070 0x7f0ac00cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.460+0000 7f0abf59e640 1 -- 172.21.15.26:0/1186539462 >> v1:172.21.15.160:6789/0 conn(0x7f0ac015cc90 legacy=0x7f0ac00cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:37.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.460+0000 7f0abf59e640 1 -- 172.21.15.26:0/1186539462 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac00ce0e0 con 0x7f0ac0160a20 2023-12-19T16:58:37.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.460+0000 7f0abdd9b640 1 -- 172.21.15.26:0/1186539462 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468710369 0 0) 0x7f0ac00cdb20 con 0x7f0ac015cc90 2023-12-19T16:58:37.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.464+0000 7f0abf59e640 1 --2- 172.21.15.26:0/1186539462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac0160a20 0x7f0ac00cf900 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009e50 tx=0x7f0ab801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba39894fc5e86d56 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:37.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.464+0000 7f0abdd9b640 1 -- 172.21.15.26:0/1186539462 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab80a58a0 con 0x7f0ac0160a20 2023-12-19T16:58:37.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.464+0000 7f0abdd9b640 1 -- 172.21.15.26:0/1186539462 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab80a5a00 con 0x7f0ac0160a20 2023-12-19T16:58:37.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.464+0000 7f0abdd9b640 1 -- 172.21.15.26:0/1186539462 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab80a5d90 con 0x7f0ac0160a20 2023-12-19T16:58:37.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.464+0000 7f0ac497a640 1 -- 172.21.15.26:0/1186539462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac0160a20 msgr2=0x7f0ac00cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:37.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.464+0000 7f0ac497a640 1 --2- 172.21.15.26:0/1186539462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac0160a20 0x7f0ac00cf900 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009e50 tx=0x7f0ab801c9e0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.464+0000 7f0ac497a640 1 -- 172.21.15.26:0/1186539462 shutdown_connections 2023-12-19T16:58:37.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.464+0000 7f0ac497a640 1 --2- 172.21.15.26:0/1186539462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac0160a20 0x7f0ac00cf900 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.464+0000 7f0ac497a640 1 --2- 172.21.15.26:0/1186539462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ac015d070 0x7f0ac00cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.464+0000 7f0ac497a640 1 -- 172.21.15.26:0/1186539462 >> 172.21.15.26:0/1186539462 conn(0x7f0ac006a320 msgr2=0x7f0ac00658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:37.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.464+0000 7f0ac497a640 1 -- 172.21.15.26:0/1186539462 shutdown_connections 2023-12-19T16:58:37.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.464+0000 7f0ac497a640 1 -- 172.21.15.26:0/1186539462 wait complete. 2023-12-19T16:58:37.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0ac497a640 1 Processor -- start 2023-12-19T16:58:37.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0ac497a640 1 -- start start 2023-12-19T16:58:37.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0ac497a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac015cc90 0x7f0ac015d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:37.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0ac497a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ac015d070 0x7f0ac00c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:37.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0ac497a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ac0160a20 0x7f0ac00c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:37.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0ac497a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00b2be0 con 0x7f0ac015cc90 2023-12-19T16:58:37.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0ac497a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac015a070 con 0x7f0ac0160a20 2023-12-19T16:58:37.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0ac497a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac015a350 con 0x7f0ac015d070 2023-12-19T16:58:37.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0abed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac015cc90 0x7f0ac015d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:37.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0abed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac015cc90 0x7f0ac015d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56542/0 (socket says 172.21.15.26:56542) 2023-12-19T16:58:37.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0abed9d640 1 -- 172.21.15.26:0/4231446325 learned_addr learned my addr 172.21.15.26:0/4231446325 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:37.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0abe59c640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ac015d070 0x7f0ac00c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:37.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0abf59e640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ac0160a20 0x7f0ac00c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:37.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0abed9d640 1 -- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ac015d070 msgr2=0x7f0ac00c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0abed9d640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ac015d070 0x7f0ac00c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0abed9d640 1 -- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ac0160a20 msgr2=0x7f0ac00c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0abed9d640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ac0160a20 0x7f0ac00c69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0abed9d640 1 -- 172.21.15.26:0/4231446325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab8009b30 con 0x7f0ac015cc90 2023-12-19T16:58:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0abed9d640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac015cc90 0x7f0ac015d970 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c4d0 tx=0x7f0ab000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0aa77fe640 1 -- 172.21.15.26:0/4231446325 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0005480 con 0x7f0ac015cc90 2023-12-19T16:58:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0aa77fe640 1 -- 172.21.15.26:0/4231446325 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab0003600 con 0x7f0ac015cc90 2023-12-19T16:58:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0aa77fe640 1 -- 172.21.15.26:0/4231446325 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00039b0 con 0x7f0ac015cc90 2023-12-19T16:58:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0ac497a640 1 -- 172.21.15.26:0/4231446325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac015a630 con 0x7f0ac015cc90 2023-12-19T16:58:37.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.468+0000 7f0ac497a640 1 -- 172.21.15.26:0/4231446325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac00c77d0 con 0x7f0ac015cc90 2023-12-19T16:58:37.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.472+0000 7f0ac497a640 1 -- 172.21.15.26:0/4231446325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac015aae0 con 0x7f0ac015cc90 2023-12-19T16:58:37.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.472+0000 7f0aa77fe640 1 -- 172.21.15.26:0/4231446325 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 47) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f0ab0003b10 con 0x7f0ac015cc90 2023-12-19T16:58:37.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.472+0000 7f0aa77fe640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 0x7f0aac068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:37.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.472+0000 7f0aa77fe640 1 -- 172.21.15.26:0/4231446325 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f0ab0082c50 con 0x7f0ac015cc90 2023-12-19T16:58:37.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.472+0000 7f0abe59c640 1 -- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 msgr2=0x7f0aac068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:37.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.472+0000 7f0abe59c640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 0x7f0aac068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:58:37.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.476+0000 7f0aa77fe640 1 -- 172.21.15.26:0/4231446325 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0ab00502f0 con 0x7f0ac015cc90 2023-12-19T16:58:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.652+0000 7f0ac497a640 1 -- 172.21.15.26:0/4231446325 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ac0061930 con 0x7f0aac065bd0 2023-12-19T16:58:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.676+0000 7f0abe59c640 1 -- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 msgr2=0x7f0aac068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:37.676+0000 7f0abe59c640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 0x7f0aac068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T16:58:37.795 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:37.790+0000 7f74e55d4180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:58:37.854 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:37.850+0000 7f74e55d4180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:58:37.877 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:37.876+0000 7ff24156e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:58:37.909 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:37.906+0000 7f74e55d4180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:58:37.939 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:37.936+0000 7ff24156e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:58:37.994 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:37.990+0000 7f74e55d4180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:58:37.997 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:37.996+0000 7ff24156e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:58:38.052 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:38.046+0000 7f74e55d4180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:58:38.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:38.080+0000 7f0abe59c640 1 -- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 msgr2=0x7f0aac068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:38.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:38.080+0000 7f0abe59c640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 0x7f0aac068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-19T16:58:38.089 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:38.088+0000 7ff24156e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:58:38.135 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:38.130+0000 7f74e55d4180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:58:38.148 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:38.148+0000 7ff24156e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:58:38.191 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:38.186+0000 7f74e55d4180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:58:38.236 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:38.236+0000 7ff24156e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:58:38.295 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:38.296+0000 7ff24156e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:58:38.332 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:38.326+0000 7f74e55d4180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:58:38.390 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:38.386+0000 7f74e55d4180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:58:38.447 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:38.444+0000 7ff24156e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:58:38.508 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:38.508+0000 7ff24156e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:58:38.628 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:58:38.628 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:58:38.628 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:58:38.636 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:38.630+0000 7f74e55d4180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:58:38.758 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:58:38.758 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:58:38.758 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:58:38.767 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:38.764+0000 7ff24156e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:58:38.770 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:38.766+0000 7f74e55d4180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:58:38.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:38.884+0000 7f0abe59c640 1 -- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 msgr2=0x7f0aac068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:58:38.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:38.884+0000 7f0abe59c640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 0x7f0aac068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-19T16:58:38.894 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:38.890+0000 7f74e55d4180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:58:38.910 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:38.908+0000 7ff24156e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:58:38.964 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:38.958+0000 7f74e55d4180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:58:39.042 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:39.040+0000 7ff24156e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:58:39.099 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:39.094+0000 7f74e55d4180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:58:39.117 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:39.116+0000 7ff24156e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:58:39.259 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:39.260+0000 7ff24156e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:58:39.632 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:39.626+0000 7f74e55d4180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:58:39.718 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:39.714+0000 7f74e55d4180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:58:39.829 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:39.828+0000 7ff24156e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:58:39.866 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:39.862+0000 7f74e55d4180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:58:39.918 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:39.916+0000 7ff24156e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:58:39.948 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:39.946+0000 7f74e55d4180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:58:40.075 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:40.072+0000 7ff24156e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:58:40.163 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:40.160+0000 7ff24156e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:58:40.264 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:40.258+0000 7f74e55d4180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:58:40.324 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:40.318+0000 7f74e55d4180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:58:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:40.336+0000 7f0aa77fe640 1 -- 172.21.15.26:0/4231446325 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 48) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f0ab00506c0 con 0x7f0ac015cc90 2023-12-19T16:58:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:40.336+0000 7f0aa77fe640 1 -- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 msgr2=0x7f0aac068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:58:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:40.336+0000 7f0aa77fe640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 0x7f0aac068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:40.458 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:40.454+0000 7f746844b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:40.458 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:40.454+0000 7f746844b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:40.458 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:40.454+0000 7f746844b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:40.458 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:40.454+0000 7f746844b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:40.458 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:40.454+0000 7f746844b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:40.459 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:40.454+0000 7f746b451640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:40.459 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:40.454+0000 7f746b451640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:40.459 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:40.454+0000 7f746b451640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:40.459 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:40.454+0000 7f746b451640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:40.459 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:58:40.454+0000 7f746b451640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:58:40.510 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:40.508+0000 7ff24156e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:58:40.574 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:58:40.572+0000 7ff24156e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:58:41.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.340+0000 7f0aa77fe640 1 -- 172.21.15.26:0/4231446325 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 49) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f0ab0080df0 con 0x7f0ac015cc90 2023-12-19T16:58:41.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.340+0000 7f0aa77fe640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 0x7f0aac068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:41.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.340+0000 7f0aa77fe640 1 -- 172.21.15.26:0/4231446325 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ac0061930 con 0x7f0aac065bd0 2023-12-19T16:58:41.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.344+0000 7f0abe59c640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 0x7f0aac068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:41.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.348+0000 7f0abe59c640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 0x7f0aac068090 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0160590 tx=0x7f0ab4009190 comp rx=0 tx=0).ready entity=mgr.6608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.412+0000 7f0aa77fe640 1 -- 172.21.15.26:0/4231446325 <== mgr.6608 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f0ac0061930 con 0x7f0aac065bd0 2023-12-19T16:58:41.414 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path /bad-path 2023-12-19T16:58:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.416+0000 7f0ac497a640 1 -- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 msgr2=0x7f0aac068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.416+0000 7f0ac497a640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 0x7f0aac068090 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0160590 tx=0x7f0ab4009190 comp rx=0 tx=0).stop 2023-12-19T16:58:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.416+0000 7f0ac497a640 1 -- 172.21.15.26:0/4231446325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac015cc90 msgr2=0x7f0ac015d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.416+0000 7f0ac497a640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac015cc90 0x7f0ac015d970 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000c4d0 tx=0x7f0ab000c990 comp rx=0 tx=0).stop 2023-12-19T16:58:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.416+0000 7f0ac497a640 1 -- 172.21.15.26:0/4231446325 shutdown_connections 2023-12-19T16:58:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.416+0000 7f0ac497a640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ac0160a20 0x7f0ac00c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.416+0000 7f0ac497a640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0aac065bd0 0x7f0aac068090 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.416+0000 7f0ac497a640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ac015d070 0x7f0ac00c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.416+0000 7f0ac497a640 1 --2- 172.21.15.26:0/4231446325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac015cc90 0x7f0ac015d970 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.416+0000 7f0ac497a640 1 -- 172.21.15.26:0/4231446325 >> 172.21.15.26:0/4231446325 conn(0x7f0ac006a320 msgr2=0x7f0ac00ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.416+0000 7f0ac497a640 1 -- 172.21.15.26:0/4231446325 shutdown_connections 2023-12-19T16:58:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.416+0000 7f0ac497a640 1 -- 172.21.15.26:0/4231446325 wait complete. 2023-12-19T16:58:41.437 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-12-19T16:58:41.438 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-12-19T16:58:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa6549bb640 1 Processor -- start 2023-12-19T16:58:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa6549bb640 1 -- start start 2023-12-19T16:58:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa6549bb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa650159d20 0x7fa6500ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa6549bb640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa6500cdee0 0x7fa6500b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa6549bb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6500adde0 con 0x7fa6500cc0a0 2023-12-19T16:58:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa6549bb640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6500c9c20 con 0x7fa6500cdee0 2023-12-19T16:58:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa6549bb640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500b28d0 con 0x7fa650159d20 2023-12-19T16:58:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa64ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa6500cc0a0 0x7fa6500cc480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38132/0 (socket says 172.21.15.26:38132) 2023-12-19T16:58:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa64ed9d640 1 -- 172.21.15.26:0/3270438420 learned_addr learned my addr 172.21.15.26:0/3270438420 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:58:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa64f59e640 1 --2- 172.21.15.26:0/3270438420 >> v2:172.21.15.160:3300/0 conn(0x7fa6500cdee0 0x7fa6500b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa647fff640 1 --2- 172.21.15.26:0/3270438420 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa650159d20 0x7fa6500ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa64e59c640 1 -- 172.21.15.26:0/3270438420 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2274426315 0 0) 0x7fa6500adde0 con 0x7fa6500cc0a0 2023-12-19T16:58:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa64e59c640 1 -- 172.21.15.26:0/3270438420 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa63c003530 con 0x7fa6500cc0a0 2023-12-19T16:58:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa64f59e640 1 -- 172.21.15.26:0/3270438420 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa650159d20 msgr2=0x7fa6500ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:58:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa64f59e640 1 --2- 172.21.15.26:0/3270438420 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa650159d20 0x7fa6500ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa64f59e640 1 -- 172.21.15.26:0/3270438420 >> v1:172.21.15.26:6789/0 conn(0x7fa6500cc0a0 legacy=0x7fa6500cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.584+0000 7fa64f59e640 1 -- 172.21.15.26:0/3270438420 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6500adb00 con 0x7fa6500cdee0 2023-12-19T16:58:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.588+0000 7fa64f59e640 1 --2- 172.21.15.26:0/3270438420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6500cdee0 0x7fa6500b2db0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa648009d90 tx=0x7fa64801c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f6ae9baaa958c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.588+0000 7fa64e59c640 1 -- 172.21.15.26:0/3270438420 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6480a5950 con 0x7fa6500cdee0 2023-12-19T16:58:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.588+0000 7fa64e59c640 1 -- 172.21.15.26:0/3270438420 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6480a5ab0 con 0x7fa6500cdee0 2023-12-19T16:58:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.588+0000 7fa64e59c640 1 -- 172.21.15.26:0/3270438420 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6480a5e20 con 0x7fa6500cdee0 2023-12-19T16:58:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.588+0000 7fa6549bb640 1 -- 172.21.15.26:0/3270438420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6500cdee0 msgr2=0x7fa6500b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.588+0000 7fa6549bb640 1 --2- 172.21.15.26:0/3270438420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6500cdee0 0x7fa6500b2db0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa648009d90 tx=0x7fa64801c890 comp rx=0 tx=0).stop 2023-12-19T16:58:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.588+0000 7fa6549bb640 1 -- 172.21.15.26:0/3270438420 shutdown_connections 2023-12-19T16:58:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.588+0000 7fa6549bb640 1 --2- 172.21.15.26:0/3270438420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6500cdee0 0x7fa6500b2db0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.588+0000 7fa6549bb640 1 --2- 172.21.15.26:0/3270438420 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa650159d20 0x7fa6500ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.588+0000 7fa6549bb640 1 -- 172.21.15.26:0/3270438420 >> 172.21.15.26:0/3270438420 conn(0x7fa65006b0f0 msgr2=0x7fa65006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.588+0000 7fa6549bb640 1 -- 172.21.15.26:0/3270438420 shutdown_connections 2023-12-19T16:58:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.588+0000 7fa6549bb640 1 -- 172.21.15.26:0/3270438420 wait complete. 2023-12-19T16:58:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa6549bb640 1 Processor -- start 2023-12-19T16:58:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa6549bb640 1 -- start start 2023-12-19T16:58:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa6549bb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6500cc0a0 0x7fa6501552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa6549bb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6500cdee0 0x7fa650153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa6549bb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa650159d20 0x7fa6500af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa6549bb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6501557e0 con 0x7fa6500cdee0 2023-12-19T16:58:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa6549bb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65006fa30 con 0x7fa6500cc0a0 2023-12-19T16:58:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa6549bb640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6500c3ab0 con 0x7fa650159d20 2023-12-19T16:58:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa647fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6500cdee0 0x7fa650153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa647fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6500cdee0 0x7fa650153f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56550/0 (socket says 172.21.15.26:56550) 2023-12-19T16:58:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa647fff640 1 -- 172.21.15.26:0/2487883501 learned_addr learned my addr 172.21.15.26:0/2487883501 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:58:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa64ed9d640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6500cc0a0 0x7fa6501552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa64f59e640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa650159d20 0x7fa6500af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa647fff640 1 -- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa650159d20 msgr2=0x7fa6500af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa647fff640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa650159d20 0x7fa6500af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa647fff640 1 -- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6500cc0a0 msgr2=0x7fa6501552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa647fff640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6500cc0a0 0x7fa6501552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa647fff640 1 -- 172.21.15.26:0/2487883501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa648009a70 con 0x7fa6500cdee0 2023-12-19T16:58:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa64f59e640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa650159d20 0x7fa6500af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:58:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa647fff640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6500cdee0 0x7fa650153f80 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa63800e900 tx=0x7fa63800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa6477fe640 1 -- 172.21.15.26:0/2487883501 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638009690 con 0x7fa6500cdee0 2023-12-19T16:58:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa6477fe640 1 -- 172.21.15.26:0/2487883501 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63800ca00 con 0x7fa6500cdee0 2023-12-19T16:58:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa6477fe640 1 -- 172.21.15.26:0/2487883501 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63800cdb0 con 0x7fa6500cdee0 2023-12-19T16:58:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa6549bb640 1 -- 172.21.15.26:0/2487883501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6500c3d90 con 0x7fa6500cdee0 2023-12-19T16:58:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.592+0000 7fa6549bb640 1 -- 172.21.15.26:0/2487883501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6500c4350 con 0x7fa6500cdee0 2023-12-19T16:58:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.596+0000 7fa6549bb640 1 -- 172.21.15.26:0/2487883501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65004e850 con 0x7fa6500cdee0 2023-12-19T16:58:41.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.596+0000 7fa6477fe640 1 -- 172.21.15.26:0/2487883501 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 49) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fa638013470 con 0x7fa6500cdee0 2023-12-19T16:58:41.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.596+0000 7fa6477fe640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa630065900 0x7fa630067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:58:41.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.596+0000 7fa6477fe640 1 -- 172.21.15.26:0/2487883501 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fa638083390 con 0x7fa6500cdee0 2023-12-19T16:58:41.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.596+0000 7fa64ed9d640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa630065900 0x7fa630067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:58:41.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.596+0000 7fa64ed9d640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa630065900 0x7fa630067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa640007fd0 tx=0x7fa640009040 comp rx=0 tx=0).ready entity=mgr.6608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:58:41.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.600+0000 7fa6477fe640 1 -- 172.21.15.26:0/2487883501 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa638050d00 con 0x7fa6500cdee0 2023-12-19T16:58:41.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.780+0000 7fa6549bb640 1 -- 172.21.15.26:0/2487883501 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6500cdc80 con 0x7fa630065900 2023-12-19T16:58:41.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.788+0000 7fa6477fe640 1 -- 172.21.15.26:0/2487883501 <== mgr.6608 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7fa640001480 con 0x7fa630065900 2023-12-19T16:58:41.789 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:58:41.789 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:58:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-19T16:58:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.788+0000 7fa6549bb640 1 -- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa630065900 msgr2=0x7fa630067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:41.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.788+0000 7fa6549bb640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa630065900 0x7fa630067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa640007fd0 tx=0x7fa640009040 comp rx=0 tx=0).stop 2023-12-19T16:58:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.788+0000 7fa6549bb640 1 -- 172.21.15.26:0/2487883501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6500cdee0 msgr2=0x7fa650153f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:58:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.788+0000 7fa6549bb640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6500cdee0 0x7fa650153f80 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa63800e900 tx=0x7fa63800edc0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.788+0000 7fa6549bb640 1 -- 172.21.15.26:0/2487883501 shutdown_connections 2023-12-19T16:58:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.788+0000 7fa6549bb640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa630065900 0x7fa630067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.788+0000 7fa6549bb640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa650159d20 0x7fa6500af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.788+0000 7fa6549bb640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6500cdee0 0x7fa650153f80 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.788+0000 7fa6549bb640 1 --2- 172.21.15.26:0/2487883501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6500cc0a0 0x7fa6501552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:58:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.788+0000 7fa6549bb640 1 -- 172.21.15.26:0/2487883501 >> 172.21.15.26:0/2487883501 conn(0x7fa65006b0f0 msgr2=0x7fa6500ce2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:58:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.788+0000 7fa6549bb640 1 -- 172.21.15.26:0/2487883501 shutdown_connections 2023-12-19T16:58:41.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:58:41.788+0000 7fa6549bb640 1 -- 172.21.15.26:0/2487883501 wait complete. 2023-12-19T16:58:41.810 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:58:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-19T16:58:59.689 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:58:59.693 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:59:00.425 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:00.422+0000 7f745ac30640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:00.425 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:00.422+0000 7f745ac30640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:00.425 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:00.422+0000 7f745ac30640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:00.425 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:00.422+0000 7f745ac30640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:00.425 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:00.422+0000 7f745ac30640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:29.767 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:59:29.772 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:59:41.811 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-12-19T16:59:41.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00f1473640 1 Processor -- start 2023-12-19T16:59:41.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00f1473640 1 -- start start 2023-12-19T16:59:41.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00f1473640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f00ec1599e0 0x7f00ec06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:41.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00f1473640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00ec0cb150 0x7f00ec0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:41.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00f1473640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00ec0cbaf0 con 0x7f00ec15d530 2023-12-19T16:59:41.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00f1473640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ec15e050 con 0x7f00ec1599e0 2023-12-19T16:59:41.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00f1473640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec15e240 con 0x7f00ec0cb150 2023-12-19T16:59:41.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00eaffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f00ec1599e0 0x7f00ec06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:41.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00eb7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f00ec15d530 0x7f00ec15d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55900/0 (socket says 172.21.15.26:55900) 2023-12-19T16:59:41.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00eb7fe640 1 -- 172.21.15.26:0/4123752067 learned_addr learned my addr 172.21.15.26:0/4123752067 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:41.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00e3fff640 1 --2- 172.21.15.26:0/4123752067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00ec0cb150 0x7f00ec0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:41.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00eaffd640 1 -- 172.21.15.26:0/4123752067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00ec0cb150 msgr2=0x7f00ec0cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:41.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00eaffd640 1 --2- 172.21.15.26:0/4123752067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00ec0cb150 0x7f00ec0cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:41.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00eaffd640 1 -- 172.21.15.26:0/4123752067 >> v1:172.21.15.26:6789/0 conn(0x7f00ec15d530 legacy=0x7f00ec15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00eaffd640 1 -- 172.21.15.26:0/4123752067 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec0c9520 con 0x7f00ec1599e0 2023-12-19T16:59:41.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.967+0000 7f00ea7fc640 1 -- 172.21.15.26:0/4123752067 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3738975533 0 0) 0x7f00ec0cbaf0 con 0x7f00ec15d530 2023-12-19T16:59:41.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.971+0000 7f00eaffd640 1 --2- 172.21.15.26:0/4123752067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ec1599e0 0x7f00ec06f6d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0097c0 tx=0x7f00dc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368c1aca76361949 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:41.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.971+0000 7f00ea7fc640 1 -- 172.21.15.26:0/4123752067 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0ab680 con 0x7f00ec1599e0 2023-12-19T16:59:41.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.971+0000 7f00ea7fc640 1 -- 172.21.15.26:0/4123752067 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00dc0ab7e0 con 0x7f00ec1599e0 2023-12-19T16:59:41.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.971+0000 7f00ea7fc640 1 -- 172.21.15.26:0/4123752067 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0abb70 con 0x7f00ec1599e0 2023-12-19T16:59:41.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.971+0000 7f00f1473640 1 -- 172.21.15.26:0/4123752067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ec1599e0 msgr2=0x7f00ec06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:41.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.971+0000 7f00f1473640 1 --2- 172.21.15.26:0/4123752067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ec1599e0 0x7f00ec06f6d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0097c0 tx=0x7f00dc01c480 comp rx=0 tx=0).stop 2023-12-19T16:59:41.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.971+0000 7f00f1473640 1 -- 172.21.15.26:0/4123752067 shutdown_connections 2023-12-19T16:59:41.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.971+0000 7f00f1473640 1 --2- 172.21.15.26:0/4123752067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00ec0cb150 0x7f00ec0cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:41.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.971+0000 7f00f1473640 1 --2- 172.21.15.26:0/4123752067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ec1599e0 0x7f00ec06f6d0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:41.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.971+0000 7f00f1473640 1 -- 172.21.15.26:0/4123752067 >> 172.21.15.26:0/4123752067 conn(0x7f00ec066160 msgr2=0x7f00ec0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:41.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.971+0000 7f00f1473640 1 -- 172.21.15.26:0/4123752067 shutdown_connections 2023-12-19T16:59:41.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.971+0000 7f00f1473640 1 -- 172.21.15.26:0/4123752067 wait complete. 2023-12-19T16:59:41.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.971+0000 7f00f1473640 1 Processor -- start 2023-12-19T16:59:41.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00f1473640 1 -- start start 2023-12-19T16:59:41.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00f1473640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00ec0cb150 0x7f00ec1f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:41.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00f1473640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ec1599e0 0x7f00ec157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:41.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00f1473640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00ec15d530 0x7f00ec0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:41.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00f1473640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec1f6660 con 0x7f00ec1599e0 2023-12-19T16:59:41.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00f1473640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec1575d0 con 0x7f00ec15d530 2023-12-19T16:59:41.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00f1473640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec157800 con 0x7f00ec0cb150 2023-12-19T16:59:41.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00e3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ec1599e0 0x7f00ec157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:41.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00e3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ec1599e0 0x7f00ec157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49542/0 (socket says 172.21.15.26:49542) 2023-12-19T16:59:41.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00e3fff640 1 -- 172.21.15.26:0/318437669 learned_addr learned my addr 172.21.15.26:0/318437669 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:41.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00eb7fe640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00ec15d530 0x7f00ec0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:41.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00eaffd640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00ec0cb150 0x7f00ec1f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:41.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00e3fff640 1 -- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00ec0cb150 msgr2=0x7f00ec1f94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:41.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00e3fff640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00ec0cb150 0x7f00ec1f94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:41.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00e3fff640 1 -- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00ec15d530 msgr2=0x7f00ec0afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:41.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00e3fff640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00ec15d530 0x7f00ec0afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:41.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00e3fff640 1 -- 172.21.15.26:0/318437669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc009420 con 0x7f00ec1599e0 2023-12-19T16:59:41.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00eaffd640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00ec0cb150 0x7f00ec1f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:59:41.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00eb7fe640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00ec15d530 0x7f00ec0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:59:41.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00e3fff640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ec1599e0 0x7f00ec157090 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f00d400d800 tx=0x7f00d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:41.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00e37fe640 1 -- 172.21.15.26:0/318437669 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d400b6c0 con 0x7f00ec1599e0 2023-12-19T16:59:41.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00e37fe640 1 -- 172.21.15.26:0/318437669 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d400b820 con 0x7f00ec1599e0 2023-12-19T16:59:41.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00f1473640 1 -- 172.21.15.26:0/318437669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ec157ae0 con 0x7f00ec1599e0 2023-12-19T16:59:41.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00e37fe640 1 -- 172.21.15.26:0/318437669 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d400bb90 con 0x7f00ec1599e0 2023-12-19T16:59:41.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00f1473640 1 -- 172.21.15.26:0/318437669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ec1580a0 con 0x7f00ec1599e0 2023-12-19T16:59:41.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.975+0000 7f00f1473640 1 -- 172.21.15.26:0/318437669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ec1f68d0 con 0x7f00ec1599e0 2023-12-19T16:59:41.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.979+0000 7f00e37fe640 1 -- 172.21.15.26:0/318437669 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 52) v1 ==== 85461+0+0 (secure 0 0 0) 0x7f00d400bcf0 con 0x7f00ec1599e0 2023-12-19T16:59:41.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.979+0000 7f00e37fe640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f00cc065cf0 0x7f00cc0681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:41.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.979+0000 7f00e37fe640 1 -- 172.21.15.26:0/318437669 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f00d4083fe0 con 0x7f00ec1599e0 2023-12-19T16:59:41.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.979+0000 7f00eaffd640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f00cc065cf0 0x7f00cc0681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:41.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.979+0000 7f00eaffd640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f00cc065cf0 0x7f00cc0681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0af000 tx=0x7f00dc004740 comp rx=0 tx=0).ready entity=mgr.6608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:41.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:41.983+0000 7f00e37fe640 1 -- 172.21.15.26:0/318437669 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f00d4051220 con 0x7f00ec1599e0 2023-12-19T16:59:42.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.159+0000 7f00f1473640 1 -- 172.21.15.26:0/318437669 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00ec159dc0 con 0x7f00cc065cf0 2023-12-19T16:59:42.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.171+0000 7f00e37fe640 1 -- 172.21.15.26:0/318437669 <== mgr.6608 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f00dc0ab950 con 0x7f00cc065cf0 2023-12-19T16:59:42.174 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:59:42.174 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:58:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-12-19T16:59:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.175+0000 7f00f1473640 1 -- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f00cc065cf0 msgr2=0x7f00cc0681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.175+0000 7f00f1473640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f00cc065cf0 0x7f00cc0681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0af000 tx=0x7f00dc004740 comp rx=0 tx=0).stop 2023-12-19T16:59:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.175+0000 7f00f1473640 1 -- 172.21.15.26:0/318437669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ec1599e0 msgr2=0x7f00ec157090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.175+0000 7f00f1473640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ec1599e0 0x7f00ec157090 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f00d400d800 tx=0x7f00d400dcc0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.175+0000 7f00f1473640 1 -- 172.21.15.26:0/318437669 shutdown_connections 2023-12-19T16:59:42.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.175+0000 7f00f1473640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00ec15d530 0x7f00ec0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.175+0000 7f00f1473640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00ec1599e0 0x7f00ec157090 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.175+0000 7f00f1473640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f00cc065cf0 0x7f00cc0681b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.175+0000 7f00f1473640 1 --2- 172.21.15.26:0/318437669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00ec0cb150 0x7f00ec1f94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.175+0000 7f00f1473640 1 -- 172.21.15.26:0/318437669 >> 172.21.15.26:0/318437669 conn(0x7f00ec066160 msgr2=0x7f00ec0cebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.175+0000 7f00f1473640 1 -- 172.21.15.26:0/318437669 shutdown_connections 2023-12-19T16:59:42.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.175+0000 7f00f1473640 1 -- 172.21.15.26:0/318437669 wait complete. 2023-12-19T16:59:42.196 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T16:58:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-12-19T16:59:42.196 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /bad-path 2023-12-19T16:59:42.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.339+0000 7f296ac66640 1 Processor -- start 2023-12-19T16:59:42.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.339+0000 7f296ac66640 1 -- start start 2023-12-19T16:59:42.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.339+0000 7f296ac66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29641585f0 0x7f29640cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:42.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.339+0000 7f296ac66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f29640cb150 0x7f29640cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:42.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.339+0000 7f296ac66640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296415b1b0 con 0x7f29640cc310 2023-12-19T16:59:42.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.339+0000 7f296ac66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29640cbb10 con 0x7f29640cb150 2023-12-19T16:59:42.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.339+0000 7f296ac66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29640b47a0 con 0x7f29641585f0 2023-12-19T16:59:42.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.339+0000 7f2968a02640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29641585f0 0x7f29640cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:42.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.339+0000 7f2963fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f29640cb150 0x7f29640cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:42.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.343+0000 7f2969203640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f29640cc310 0x7f29640cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:40256/0 (socket says 172.21.15.26:40256) 2023-12-19T16:59:42.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.343+0000 7f2969203640 1 -- 172.21.15.26:0/3093075230 learned_addr learned my addr 172.21.15.26:0/3093075230 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:42.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.343+0000 7f2963fff640 1 -- 172.21.15.26:0/3093075230 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29641585f0 msgr2=0x7f29640cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:42.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.343+0000 7f2963fff640 1 --2- 172.21.15.26:0/3093075230 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29641585f0 0x7f29640cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.343+0000 7f2963fff640 1 -- 172.21.15.26:0/3093075230 >> v1:172.21.15.160:6789/0 conn(0x7f29640cc310 legacy=0x7f29640cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:42.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.343+0000 7f2963fff640 1 -- 172.21.15.26:0/3093075230 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296415b490 con 0x7f29640cb150 2023-12-19T16:59:42.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.343+0000 7f29637fe640 1 -- 172.21.15.26:0/3093075230 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351465079 0 0) 0x7f296415b1b0 con 0x7f29640cc310 2023-12-19T16:59:42.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.343+0000 7f2963fff640 1 --2- 172.21.15.26:0/3093075230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29640cb150 0x7f29640cb5d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f29500097c0 tx=0x7f295001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a5aeaf7448605bc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:42.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.343+0000 7f29637fe640 1 -- 172.21.15.26:0/3093075230 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29500ac620 con 0x7f29640cb150 2023-12-19T16:59:42.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.343+0000 7f29637fe640 1 -- 172.21.15.26:0/3093075230 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29500ac780 con 0x7f29640cb150 2023-12-19T16:59:42.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.343+0000 7f29637fe640 1 -- 172.21.15.26:0/3093075230 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29500acb10 con 0x7f29640cb150 2023-12-19T16:59:42.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 -- 172.21.15.26:0/3093075230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29640cb150 msgr2=0x7f29640cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:42.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 --2- 172.21.15.26:0/3093075230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29640cb150 0x7f29640cb5d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f29500097c0 tx=0x7f295001c3c0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 -- 172.21.15.26:0/3093075230 shutdown_connections 2023-12-19T16:59:42.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 --2- 172.21.15.26:0/3093075230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29640cb150 0x7f29640cb5d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 --2- 172.21.15.26:0/3093075230 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29641585f0 0x7f29640cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 -- 172.21.15.26:0/3093075230 >> 172.21.15.26:0/3093075230 conn(0x7f2964066160 msgr2=0x7f29640b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:42.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 -- 172.21.15.26:0/3093075230 shutdown_connections 2023-12-19T16:59:42.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 -- 172.21.15.26:0/3093075230 wait complete. 2023-12-19T16:59:42.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 Processor -- start 2023-12-19T16:59:42.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 -- start start 2023-12-19T16:59:42.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29640cb150 0x7f29641f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:42.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29640cc310 0x7f29641459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:42.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29641585f0 0x7f296406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:42.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29641f9e30 con 0x7f29641585f0 2023-12-19T16:59:42.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29640c81b0 con 0x7f29640cc310 2023-12-19T16:59:42.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f296ac66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29640c8490 con 0x7f29640cb150 2023-12-19T16:59:42.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f2969203640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29641585f0 0x7f296406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:42.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f2969203640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29641585f0 0x7f296406c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49558/0 (socket says 172.21.15.26:49558) 2023-12-19T16:59:42.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.347+0000 7f2969203640 1 -- 172.21.15.26:0/438075015 learned_addr learned my addr 172.21.15.26:0/438075015 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:42.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f2968a02640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29640cb150 0x7f29641f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:42.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f2963fff640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29640cc310 0x7f29641459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:42.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f2969203640 1 -- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29640cb150 msgr2=0x7f29641f98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:42.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f2969203640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29640cb150 0x7f29641f98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f2969203640 1 -- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29640cc310 msgr2=0x7f29641459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:42.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f2969203640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29640cc310 0x7f29641459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f2969203640 1 -- 172.21.15.26:0/438075015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2950009420 con 0x7f29641585f0 2023-12-19T16:59:42.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f2963fff640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29640cc310 0x7f29641459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:59:42.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f2968a02640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29640cb150 0x7f29641f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:59:42.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f2969203640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29641585f0 0x7f296406c360 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f295c00c780 tx=0x7f295c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:42.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f29617fa640 1 -- 172.21.15.26:0/438075015 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c00ce90 con 0x7f29641585f0 2023-12-19T16:59:42.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f29617fa640 1 -- 172.21.15.26:0/438075015 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c00a640 con 0x7f29641585f0 2023-12-19T16:59:42.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f296ac66640 1 -- 172.21.15.26:0/438075015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296415c3e0 con 0x7f29641585f0 2023-12-19T16:59:42.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f29617fa640 1 -- 172.21.15.26:0/438075015 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c00a9d0 con 0x7f29641585f0 2023-12-19T16:59:42.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f296ac66640 1 -- 172.21.15.26:0/438075015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296415e170 con 0x7f29641585f0 2023-12-19T16:59:42.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f296ac66640 1 -- 172.21.15.26:0/438075015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296415e360 con 0x7f29641585f0 2023-12-19T16:59:42.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f29617fa640 1 -- 172.21.15.26:0/438075015 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 52) v1 ==== 85461+0+0 (secure 0 0 0) 0x7f295c00ab30 con 0x7f29641585f0 2023-12-19T16:59:42.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f29617fa640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2948065cf0 0x7f29480681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:42.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.351+0000 7f29617fa640 1 -- 172.21.15.26:0/438075015 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f295c083080 con 0x7f29641585f0 2023-12-19T16:59:42.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.355+0000 7f2968a02640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2948065cf0 0x7f29480681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:42.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.355+0000 7f2968a02640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2948065cf0 0x7f29480681b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2958009b10 tx=0x7f2958008040 comp rx=0 tx=0).ready entity=mgr.6608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:42.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.355+0000 7f29617fa640 1 -- 172.21.15.26:0/438075015 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f295c0b40a0 con 0x7f29641585f0 2023-12-19T16:59:42.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.535+0000 7f296ac66640 1 -- 172.21.15.26:0/438075015 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f296415c260 con 0x7f2948065cf0 2023-12-19T16:59:42.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.603+0000 7f29617fa640 1 -- 172.21.15.26:0/438075015 <== mgr.6608 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f2958009760 con 0x7f2948065cf0 2023-12-19T16:59:42.606 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path /bad-path 2023-12-19T16:59:42.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.603+0000 7f296ac66640 1 -- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2948065cf0 msgr2=0x7f29480681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.603+0000 7f296ac66640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2948065cf0 0x7f29480681b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2958009b10 tx=0x7f2958008040 comp rx=0 tx=0).stop 2023-12-19T16:59:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.603+0000 7f296ac66640 1 -- 172.21.15.26:0/438075015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29641585f0 msgr2=0x7f296406c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.603+0000 7f296ac66640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29641585f0 0x7f296406c360 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f295c00c780 tx=0x7f295c00cc40 comp rx=0 tx=0).stop 2023-12-19T16:59:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.607+0000 7f296ac66640 1 -- 172.21.15.26:0/438075015 shutdown_connections 2023-12-19T16:59:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.607+0000 7f296ac66640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29641585f0 0x7f296406c360 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.607+0000 7f296ac66640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29640cc310 0x7f29641459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.607+0000 7f296ac66640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2948065cf0 0x7f29480681b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.607+0000 7f296ac66640 1 --2- 172.21.15.26:0/438075015 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f29640cb150 0x7f29641f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.607+0000 7f296ac66640 1 -- 172.21.15.26:0/438075015 >> 172.21.15.26:0/438075015 conn(0x7f2964066160 msgr2=0x7f29640ae0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.607+0000 7f296ac66640 1 -- 172.21.15.26:0/438075015 shutdown_connections 2023-12-19T16:59:42.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.607+0000 7f296ac66640 1 -- 172.21.15.26:0/438075015 wait complete. 2023-12-19T16:59:42.626 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-12-19T16:59:42.627 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-19T16:59:42.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.771+0000 7fc12ad39640 1 Processor -- start 2023-12-19T16:59:42.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.771+0000 7fc12ad39640 1 -- start start 2023-12-19T16:59:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.771+0000 7fc12ad39640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc1240cb150 0x7fc1240cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.771+0000 7fc12ad39640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1240ce3f0 con 0x7fc1241585f0 2023-12-19T16:59:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.771+0000 7fc12ad39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc12415b1b0 con 0x7fc12415e690 2023-12-19T16:59:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.771+0000 7fc12ad39640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1240cbb40 con 0x7fc1240cb150 2023-12-19T16:59:42.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.771+0000 7fc128ad5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc1241585f0 0x7fc1240cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55902/0 (socket says 172.21.15.26:55902) 2023-12-19T16:59:42.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.771+0000 7fc128ad5640 1 -- 172.21.15.26:0/1336008488 learned_addr learned my addr 172.21.15.26:0/1336008488 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:42.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.771+0000 7fc123fff640 1 --2- 172.21.15.26:0/1336008488 >> v2:172.21.15.160:3300/0 conn(0x7fc1240cb150 0x7fc1240cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:42.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.771+0000 7fc1237fe640 1 -- 172.21.15.26:0/1336008488 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196545401 0 0) 0x7fc1240ce3f0 con 0x7fc1241585f0 2023-12-19T16:59:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.771+0000 7fc1237fe640 1 -- 172.21.15.26:0/1336008488 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc114003530 con 0x7fc1241585f0 2023-12-19T16:59:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.771+0000 7fc1237fe640 1 -- 172.21.15.26:0/1336008488 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428061323 0 0) 0x7fc12415b1b0 con 0x7fc12415e690 2023-12-19T16:59:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.775+0000 7fc1237fe640 1 -- 172.21.15.26:0/1336008488 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1240ce3f0 con 0x7fc12415e690 2023-12-19T16:59:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.775+0000 7fc123fff640 1 -- 172.21.15.26:0/1336008488 >> v1:172.21.15.160:6789/0 conn(0x7fc12415e690 legacy=0x7fc12415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.775+0000 7fc123fff640 1 -- 172.21.15.26:0/1336008488 >> v1:172.21.15.26:6789/0 conn(0x7fc1241585f0 legacy=0x7fc1240cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:42.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.775+0000 7fc123fff640 1 -- 172.21.15.26:0/1336008488 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12415b490 con 0x7fc1240cb150 2023-12-19T16:59:42.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.775+0000 7fc123fff640 1 --2- 172.21.15.26:0/1336008488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1240cb150 0x7fc1240cb600 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc1100097c0 tx=0x7fc11001c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bb0f22b0d125f7c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:42.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.775+0000 7fc1237fe640 1 -- 172.21.15.26:0/1336008488 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1100ab600 con 0x7fc1240cb150 2023-12-19T16:59:42.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.775+0000 7fc1237fe640 1 -- 172.21.15.26:0/1336008488 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1100a7050 con 0x7fc1240cb150 2023-12-19T16:59:42.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.775+0000 7fc1237fe640 1 -- 172.21.15.26:0/1336008488 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1100ab980 con 0x7fc1240cb150 2023-12-19T16:59:42.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.775+0000 7fc12ad39640 1 -- 172.21.15.26:0/1336008488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1240cb150 msgr2=0x7fc1240cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:42.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.775+0000 7fc12ad39640 1 --2- 172.21.15.26:0/1336008488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1240cb150 0x7fc1240cb600 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc1100097c0 tx=0x7fc11001c3b0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.775+0000 7fc12ad39640 1 -- 172.21.15.26:0/1336008488 shutdown_connections 2023-12-19T16:59:42.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.775+0000 7fc12ad39640 1 --2- 172.21.15.26:0/1336008488 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1240cb150 0x7fc1240cb600 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.775+0000 7fc12ad39640 1 -- 172.21.15.26:0/1336008488 >> 172.21.15.26:0/1336008488 conn(0x7fc124066160 msgr2=0x7fc1240b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:42.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc12ad39640 1 -- 172.21.15.26:0/1336008488 shutdown_connections 2023-12-19T16:59:42.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc12ad39640 1 -- 172.21.15.26:0/1336008488 wait complete. 2023-12-19T16:59:42.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc12ad39640 1 Processor -- start 2023-12-19T16:59:42.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc12ad39640 1 -- start start 2023-12-19T16:59:42.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc12ad39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1240cb150 0x7fc1240c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:42.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc12ad39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1241585f0 0x7fc12406ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:42.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc12ad39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc12415e690 0x7fc124148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:42.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc12ad39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1240c6ae0 con 0x7fc12415e690 2023-12-19T16:59:42.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc12ad39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12406d1f0 con 0x7fc1241585f0 2023-12-19T16:59:42.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc12ad39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc124149360 con 0x7fc1240cb150 2023-12-19T16:59:42.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc1292d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc12415e690 0x7fc124148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:42.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc1292d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc12415e690 0x7fc124148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49564/0 (socket says 172.21.15.26:49564) 2023-12-19T16:59:42.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc1292d6640 1 -- 172.21.15.26:0/1591933770 learned_addr learned my addr 172.21.15.26:0/1591933770 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:42.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc123fff640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1241585f0 0x7fc12406ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:42.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc128ad5640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1240cb150 0x7fc1240c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc1292d6640 1 -- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1240cb150 msgr2=0x7fc1240c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc1292d6640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1240cb150 0x7fc1240c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc1292d6640 1 -- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1241585f0 msgr2=0x7fc12406ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc1292d6640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1241585f0 0x7fc12406ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc1292d6640 1 -- 172.21.15.26:0/1591933770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110009420 con 0x7fc12415e690 2023-12-19T16:59:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.779+0000 7fc1292d6640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc12415e690 0x7fc124148e20 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00c7e0 tx=0x7fc11c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.783+0000 7fc1217fa640 1 -- 172.21.15.26:0/1591933770 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c003db0 con 0x7fc12415e690 2023-12-19T16:59:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.783+0000 7fc1217fa640 1 -- 172.21.15.26:0/1591933770 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11c005070 con 0x7fc12415e690 2023-12-19T16:59:42.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.783+0000 7fc1217fa640 1 -- 172.21.15.26:0/1591933770 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11c005400 con 0x7fc12415e690 2023-12-19T16:59:42.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.783+0000 7fc12ad39640 1 -- 172.21.15.26:0/1591933770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12415c3e0 con 0x7fc12415e690 2023-12-19T16:59:42.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.783+0000 7fc12ad39640 1 -- 172.21.15.26:0/1591933770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1240c61b0 con 0x7fc12415e690 2023-12-19T16:59:42.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.783+0000 7fc12ad39640 1 -- 172.21.15.26:0/1591933770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1240c6c90 con 0x7fc12415e690 2023-12-19T16:59:42.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.783+0000 7fc1217fa640 1 -- 172.21.15.26:0/1591933770 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 52) v1 ==== 85461+0+0 (secure 0 0 0) 0x7fc11c005560 con 0x7fc12415e690 2023-12-19T16:59:42.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.783+0000 7fc1217fa640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc108065cf0 0x7fc1080681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:42.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.783+0000 7fc1217fa640 1 -- 172.21.15.26:0/1591933770 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6814+0+0 (secure 0 0 0) 0x7fc11c083300 con 0x7fc12415e690 2023-12-19T16:59:42.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.783+0000 7fc128ad5640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc108065cf0 0x7fc1080681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:42.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.787+0000 7fc128ad5640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc108065cf0 0x7fc1080681b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc1180011f0 tx=0x7fc118007d40 comp rx=0 tx=0).ready entity=mgr.6608 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:42.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.787+0000 7fc1217fa640 1 -- 172.21.15.26:0/1591933770 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc1240c6c90 con 0x7fc12415e690 2023-12-19T16:59:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:42.983+0000 7fc12ad39640 1 -- 172.21.15.26:0/1591933770 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fc1240c6c90 con 0x7fc12415e690 2023-12-19T16:59:43.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.703+0000 7fc1217fa640 1 -- 172.21.15.26:0/1591933770 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 53) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fc11c0508f0 con 0x7fc12415e690 2023-12-19T16:59:43.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.703+0000 7fc1217fa640 1 -- 172.21.15.26:0/1591933770 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v53) v1 ==== 93+0+0 (secure 0 0 0) 0x7fc11c06d270 con 0x7fc12415e690 2023-12-19T16:59:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.707+0000 7fc12ad39640 1 -- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc108065cf0 msgr2=0x7fc1080681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.707+0000 7fc12ad39640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc108065cf0 0x7fc1080681b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc1180011f0 tx=0x7fc118007d40 comp rx=0 tx=0).stop 2023-12-19T16:59:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.707+0000 7fc12ad39640 1 -- 172.21.15.26:0/1591933770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc12415e690 msgr2=0x7fc124148e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.707+0000 7fc12ad39640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc12415e690 0x7fc124148e20 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc11c00c7e0 tx=0x7fc11c00cca0 comp rx=0 tx=0).stop 2023-12-19T16:59:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.707+0000 7fc12ad39640 1 -- 172.21.15.26:0/1591933770 shutdown_connections 2023-12-19T16:59:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.707+0000 7fc12ad39640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc12415e690 0x7fc124148e20 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.707+0000 7fc12ad39640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1241585f0 0x7fc12406ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.707+0000 7fc12ad39640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc108065cf0 0x7fc1080681b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:43.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.707+0000 7fc12ad39640 1 --2- 172.21.15.26:0/1591933770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1240cb150 0x7fc1240c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.707+0000 7fc12ad39640 1 -- 172.21.15.26:0/1591933770 >> 172.21.15.26:0/1591933770 conn(0x7fc124066160 msgr2=0x7fc1240cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.707+0000 7fc12ad39640 1 -- 172.21.15.26:0/1591933770 shutdown_connections 2023-12-19T16:59:43.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:43.707+0000 7fc12ad39640 1 -- 172.21.15.26:0/1591933770 wait complete. 2023-12-19T16:59:43.733 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T16:59:43.733 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:59:43.838 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:59:43.886 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:43.883+0000 7fb51358e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:59:43.889 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:43.885+0000 7f386e457180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T16:59:43.935 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T16:59:43.935 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:59:43.935 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:59:43.945 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:59:43.945 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:59:43.996 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:59:44.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94ff290640 1 Processor -- start 2023-12-19T16:59:44.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94ff290640 1 -- start start 2023-12-19T16:59:44.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94ff290640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f80cc0a0 0x7f94f80cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94ff290640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f94f8159d20 0x7f94f80cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94ff290640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94f80cdee0 0x7f94f815d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94ff290640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f80c9c20 con 0x7f94f80cdee0 2023-12-19T16:59:44.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94ff290640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f80adde0 con 0x7f94f8159d20 2023-12-19T16:59:44.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94ff290640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f80b28d0 con 0x7f94f80cc0a0 2023-12-19T16:59:44.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94fe28e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f80cc0a0 0x7f94f80cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94fea8f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94f80cdee0 0x7f94f815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94fda8d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f94f8159d20 0x7f94f80cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94fe28e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f80cc0a0 0x7f94f80cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:51488/0 (socket says 172.21.15.26:51488) 2023-12-19T16:59:44.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94fe28e640 1 -- 172.21.15.26:0/3412690964 learned_addr learned my addr 172.21.15.26:0/3412690964 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:44.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94fe28e640 1 -- 172.21.15.26:0/3412690964 >> v2:172.21.15.160:3300/0 conn(0x7f94f8159d20 msgr2=0x7f94f80cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:44.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94fe28e640 1 --2- 172.21.15.26:0/3412690964 >> v2:172.21.15.160:3300/0 conn(0x7f94f8159d20 0x7f94f80cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94fe28e640 1 -- 172.21.15.26:0/3412690964 >> v2:172.21.15.26:3300/0 conn(0x7f94f80cdee0 msgr2=0x7f94f815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:44.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94fe28e640 1 --2- 172.21.15.26:0/3412690964 >> v2:172.21.15.26:3300/0 conn(0x7f94f80cdee0 0x7f94f815d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94fe28e640 1 -- 172.21.15.26:0/3412690964 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f80ccf00 con 0x7f94f80cc0a0 2023-12-19T16:59:44.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.187+0000 7f94fea8f640 1 --2- 172.21.15.26:0/3412690964 >> v2:172.21.15.26:3300/0 conn(0x7f94f80cdee0 0x7f94f815d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:59:44.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.191+0000 7f94fe28e640 1 --2- 172.21.15.26:0/3412690964 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f80cc0a0 0x7f94f80cc480 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94f806cc20 tx=0x7f94f401c180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77938aaf109f667c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:44.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.191+0000 7f94fd28c640 1 -- 172.21.15.26:0/3412690964 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f40ab600 con 0x7f94f80cc0a0 2023-12-19T16:59:44.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.191+0000 7f94fd28c640 1 -- 172.21.15.26:0/3412690964 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f40a7050 con 0x7f94f80cc0a0 2023-12-19T16:59:44.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.191+0000 7f94fd28c640 1 -- 172.21.15.26:0/3412690964 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f40ab980 con 0x7f94f80cc0a0 2023-12-19T16:59:44.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.191+0000 7f94ff290640 1 -- 172.21.15.26:0/3412690964 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f80cc0a0 msgr2=0x7f94f80cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:44.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.191+0000 7f94ff290640 1 --2- 172.21.15.26:0/3412690964 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f80cc0a0 0x7f94f80cc480 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94f806cc20 tx=0x7f94f401c180 comp rx=0 tx=0).stop 2023-12-19T16:59:44.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.191+0000 7f94ff290640 1 -- 172.21.15.26:0/3412690964 shutdown_connections 2023-12-19T16:59:44.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.191+0000 7f94ff290640 1 --2- 172.21.15.26:0/3412690964 >> v2:172.21.15.26:3300/0 conn(0x7f94f80cdee0 0x7f94f815d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.191+0000 7f94ff290640 1 --2- 172.21.15.26:0/3412690964 >> v2:172.21.15.160:3300/0 conn(0x7f94f8159d20 0x7f94f80cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.191+0000 7f94ff290640 1 --2- 172.21.15.26:0/3412690964 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f80cc0a0 0x7f94f80cc480 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.191+0000 7f94ff290640 1 -- 172.21.15.26:0/3412690964 >> 172.21.15.26:0/3412690964 conn(0x7f94f806b0f0 msgr2=0x7f94f806b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:44.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.191+0000 7f94ff290640 1 -- 172.21.15.26:0/3412690964 shutdown_connections 2023-12-19T16:59:44.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.191+0000 7f94ff290640 1 -- 172.21.15.26:0/3412690964 wait complete. 2023-12-19T16:59:44.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94ff290640 1 Processor -- start 2023-12-19T16:59:44.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94ff290640 1 -- start start 2023-12-19T16:59:44.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94ff290640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94f80cc0a0 0x7f94f80b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94ff290640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f80cdee0 0x7f94f8154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94ff290640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94f8159d20 0x7f94f815a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94ff290640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f815d0f0 con 0x7f94f80cc0a0 2023-12-19T16:59:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94ff290640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f815c470 con 0x7f94f8159d20 2023-12-19T16:59:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94ff290640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f815c750 con 0x7f94f80cdee0 2023-12-19T16:59:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94fe28e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94f80cc0a0 0x7f94f80b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94fe28e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94f80cc0a0 0x7f94f80b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49598/0 (socket says 172.21.15.26:49598) 2023-12-19T16:59:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94fe28e640 1 -- 172.21.15.26:0/677774597 learned_addr learned my addr 172.21.15.26:0/677774597 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94fda8d640 1 --2- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f80cdee0 0x7f94f8154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94fea8f640 1 --2- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94f8159d20 0x7f94f815a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94fe28e640 1 -- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f80cdee0 msgr2=0x7f94f8154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94fe28e640 1 --2- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f80cdee0 0x7f94f8154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94fe28e640 1 -- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94f8159d20 msgr2=0x7f94f815a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94fe28e640 1 --2- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94f8159d20 0x7f94f815a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94fe28e640 1 -- 172.21.15.26:0/677774597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f0009a70 con 0x7f94f80cc0a0 2023-12-19T16:59:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94fe28e640 1 --2- 172.21.15.26:0/677774597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94f80cc0a0 0x7f94f80b0b60 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f94f40ae040 tx=0x7f94f401a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94eeffd640 1 -- 172.21.15.26:0/677774597 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f40abe70 con 0x7f94f80cc0a0 2023-12-19T16:59:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94eeffd640 1 -- 172.21.15.26:0/677774597 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f40b6420 con 0x7f94f80cc0a0 2023-12-19T16:59:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94eeffd640 1 -- 172.21.15.26:0/677774597 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f40b67f0 con 0x7f94f80cc0a0 2023-12-19T16:59:44.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94ff290640 1 -- 172.21.15.26:0/677774597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f40094d0 con 0x7f94f80cc0a0 2023-12-19T16:59:44.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.195+0000 7f94ff290640 1 -- 172.21.15.26:0/677774597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f815cd10 con 0x7f94f80cc0a0 2023-12-19T16:59:44.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.199+0000 7f94ff290640 1 -- 172.21.15.26:0/677774597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f8067400 con 0x7f94f80cc0a0 2023-12-19T16:59:44.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.199+0000 7f94eeffd640 1 -- 172.21.15.26:0/677774597 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 53) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f94f40b2040 con 0x7f94f80cc0a0 2023-12-19T16:59:44.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.199+0000 7f94eeffd640 1 --2- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f94dc065cd0 0x7f94dc068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.199+0000 7f94eeffd640 1 -- 172.21.15.26:0/677774597 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f94f411dcd0 con 0x7f94f80cc0a0 2023-12-19T16:59:44.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.199+0000 7f94fda8d640 1 -- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f94dc065cd0 msgr2=0x7f94dc068190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:59:44.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.199+0000 7f94fda8d640 1 --2- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f94dc065cd0 0x7f94dc068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:59:44.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.207+0000 7f94eeffd640 1 -- 172.21.15.26:0/677774597 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f94f40eaeb0 con 0x7f94f80cc0a0 2023-12-19T16:59:44.304 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:44.301+0000 7f386e457180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:59:44.330 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:44.327+0000 7fb51358e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T16:59:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.363+0000 7f94ff290640 1 -- 172.21.15.26:0/677774597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94f80cd3c0 con 0x7f94f80cc0a0 2023-12-19T16:59:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.367+0000 7f94eeffd640 1 -- 172.21.15.26:0/677774597 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+17754 (secure 0 0 0) 0x7f94f40ba090 con 0x7f94f80cc0a0 2023-12-19T16:59:44.368 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:59:44.368 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":110,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:59:41.676863+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:58:26.906185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T16:58:27.358583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1a","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1480778234":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/3399946745":"2023-12-20T16:58:40.327736+0000","172.21.15.160:6833/38098":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/128716506":"2023-12-20T16:58:40.327736+0000","172.21.15.160:6832/38098":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/1184346492":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/1271544430":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/16297988":"2023-12-20T16:58:40.327736+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:59:44.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.371+0000 7f94ff290640 1 -- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f94dc065cd0 msgr2=0x7f94dc068190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:59:44.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.371+0000 7f94ff290640 1 --2- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f94dc065cd0 0x7f94dc068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.371+0000 7f94ff290640 1 -- 172.21.15.26:0/677774597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94f80cc0a0 msgr2=0x7f94f80b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:44.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.371+0000 7f94ff290640 1 --2- 172.21.15.26:0/677774597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94f80cc0a0 0x7f94f80b0b60 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f94f40ae040 tx=0x7f94f401a9c0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.371+0000 7f94ff290640 1 -- 172.21.15.26:0/677774597 shutdown_connections 2023-12-19T16:59:44.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.371+0000 7f94ff290640 1 --2- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94f8159d20 0x7f94f815a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.371+0000 7f94ff290640 1 --2- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f94dc065cd0 0x7f94dc068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.371+0000 7f94ff290640 1 --2- 172.21.15.26:0/677774597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94f80cdee0 0x7f94f8154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.371+0000 7f94ff290640 1 --2- 172.21.15.26:0/677774597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94f80cc0a0 0x7f94f80b0b60 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.371+0000 7f94ff290640 1 -- 172.21.15.26:0/677774597 >> 172.21.15.26:0/677774597 conn(0x7f94f806b0f0 msgr2=0x7f94f80cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:44.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.371+0000 7f94ff290640 1 -- 172.21.15.26:0/677774597 shutdown_connections 2023-12-19T16:59:44.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.371+0000 7f94ff290640 1 -- 172.21.15.26:0/677774597 wait complete. 2023-12-19T16:59:44.392 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:58:26.906185+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:58:27.358583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:59:44.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:59:44.454 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:44.449+0000 7f386e457180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:59:44.489 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:44.487+0000 7fb51358e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T16:59:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ed172c640 1 Processor -- start 2023-12-19T16:59:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ed172c640 1 -- start start 2023-12-19T16:59:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ed172c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ecc15ada0 0x7f6ecc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ed172c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6ecc06ac80 0x7f6ecc06b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ed172c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ecc070d90 con 0x7f6ecc15a9c0 2023-12-19T16:59:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ed172c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ecc0cc270 con 0x7f6ecc15ada0 2023-12-19T16:59:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ed172c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ecc071070 con 0x7f6ecc06ac80 2023-12-19T16:59:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ec3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ecc15ada0 0x7f6ecc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ec3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ecc15ada0 0x7f6ecc0706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49608/0 (socket says 172.21.15.26:49608) 2023-12-19T16:59:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ec3fff640 1 -- 172.21.15.26:0/2476192866 learned_addr learned my addr 172.21.15.26:0/2476192866 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ecaffd640 1 --1- 172.21.15.26:0/2476192866 >> v1:172.21.15.26:6789/0 conn(0x7f6ecc15a9c0 0x7f6ecc0b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55912/0 (socket says 172.21.15.26:55912) 2023-12-19T16:59:44.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ecb7fe640 1 --2- 172.21.15.26:0/2476192866 >> v2:172.21.15.160:3300/0 conn(0x7f6ecc06ac80 0x7f6ecc06b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ec3fff640 1 -- 172.21.15.26:0/2476192866 >> v2:172.21.15.160:3300/0 conn(0x7f6ecc06ac80 msgr2=0x7f6ecc06b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ec3fff640 1 --2- 172.21.15.26:0/2476192866 >> v2:172.21.15.160:3300/0 conn(0x7f6ecc06ac80 0x7f6ecc06b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ec3fff640 1 -- 172.21.15.26:0/2476192866 >> v1:172.21.15.26:6789/0 conn(0x7f6ecc15a9c0 legacy=0x7f6ecc0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6ec3fff640 1 -- 172.21.15.26:0/2476192866 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc071350 con 0x7f6ecc15ada0 2023-12-19T16:59:44.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.539+0000 7f6eca7fc640 1 -- 172.21.15.26:0/2476192866 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568524266 0 0) 0x7f6ecc070d90 con 0x7f6ecc15a9c0 2023-12-19T16:59:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.543+0000 7f6ec3fff640 1 --2- 172.21.15.26:0/2476192866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ecc15ada0 0x7f6ecc0706d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0cc3b0 tx=0x7f6eb401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa69be16feda8aa5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.543+0000 7f6eca7fc640 1 -- 172.21.15.26:0/2476192866 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb40ab660 con 0x7f6ecc15ada0 2023-12-19T16:59:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.543+0000 7f6eca7fc640 1 -- 172.21.15.26:0/2476192866 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb40ab7c0 con 0x7f6ecc15ada0 2023-12-19T16:59:44.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.543+0000 7f6eca7fc640 1 -- 172.21.15.26:0/2476192866 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb40abb50 con 0x7f6ecc15ada0 2023-12-19T16:59:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.543+0000 7f6ed172c640 1 -- 172.21.15.26:0/2476192866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ecc15ada0 msgr2=0x7f6ecc0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.543+0000 7f6ed172c640 1 --2- 172.21.15.26:0/2476192866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ecc15ada0 0x7f6ecc0706d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0cc3b0 tx=0x7f6eb401c440 comp rx=0 tx=0).stop 2023-12-19T16:59:44.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 -- 172.21.15.26:0/2476192866 shutdown_connections 2023-12-19T16:59:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 --2- 172.21.15.26:0/2476192866 >> v2:172.21.15.160:3300/0 conn(0x7f6ecc06ac80 0x7f6ecc06b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 --2- 172.21.15.26:0/2476192866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ecc15ada0 0x7f6ecc0706d0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 -- 172.21.15.26:0/2476192866 >> 172.21.15.26:0/2476192866 conn(0x7f6ecc0ce730 msgr2=0x7f6ecc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 -- 172.21.15.26:0/2476192866 shutdown_connections 2023-12-19T16:59:44.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 -- 172.21.15.26:0/2476192866 wait complete. 2023-12-19T16:59:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 Processor -- start 2023-12-19T16:59:44.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 -- start start 2023-12-19T16:59:44.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ecc06ac80 0x7f6ecc13c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ecc15a9c0 0x7f6ecc151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ecc15ada0 0x7f6ecc158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc13cbf0 con 0x7f6ecc15ada0 2023-12-19T16:59:44.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc151550 con 0x7f6ecc15a9c0 2023-12-19T16:59:44.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ed172c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc13a820 con 0x7f6ecc06ac80 2023-12-19T16:59:44.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ecb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ecc15ada0 0x7f6ecc158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ecb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ecc15ada0 0x7f6ecc158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49622/0 (socket says 172.21.15.26:49622) 2023-12-19T16:59:44.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ecb7fe640 1 -- 172.21.15.26:0/3560747181 learned_addr learned my addr 172.21.15.26:0/3560747181 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:44.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ecaffd640 1 --2- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ecc06ac80 0x7f6ecc13c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ec3fff640 1 --2- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ecc15a9c0 0x7f6ecc151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ecb7fe640 1 -- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ecc06ac80 msgr2=0x7f6ecc13c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:44.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ecb7fe640 1 --2- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ecc06ac80 0x7f6ecc13c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ecb7fe640 1 -- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ecc15a9c0 msgr2=0x7f6ecc151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:44.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ecb7fe640 1 --2- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ecc15a9c0 0x7f6ecc151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ecb7fe640 1 -- 172.21.15.26:0/3560747181 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb40094d0 con 0x7f6ecc15ada0 2023-12-19T16:59:44.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.547+0000 7f6ecb7fe640 1 --2- 172.21.15.26:0/3560747181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ecc15ada0 0x7f6ecc158160 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4007e70 tx=0x7f6ec400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:44.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.551+0000 7f6ec37fe640 1 -- 172.21.15.26:0/3560747181 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4014620 con 0x7f6ecc15ada0 2023-12-19T16:59:44.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.551+0000 7f6ec37fe640 1 -- 172.21.15.26:0/3560747181 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec40147a0 con 0x7f6ecc15ada0 2023-12-19T16:59:44.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.551+0000 7f6ed172c640 1 -- 172.21.15.26:0/3560747181 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ecc13d670 con 0x7f6ecc15ada0 2023-12-19T16:59:44.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.551+0000 7f6ec37fe640 1 -- 172.21.15.26:0/3560747181 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4014b50 con 0x7f6ecc15ada0 2023-12-19T16:59:44.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.551+0000 7f6ed172c640 1 -- 172.21.15.26:0/3560747181 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ecc13cdd0 con 0x7f6ecc15ada0 2023-12-19T16:59:44.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.551+0000 7f6ed172c640 1 -- 172.21.15.26:0/3560747181 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ecc157c20 con 0x7f6ecc15ada0 2023-12-19T16:59:44.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.551+0000 7f6ec37fe640 1 -- 172.21.15.26:0/3560747181 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 53) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f6ec4014cb0 con 0x7f6ecc15ada0 2023-12-19T16:59:44.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.551+0000 7f6ec37fe640 1 --2- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6eac065ca0 0x7f6eac068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.551+0000 7f6ec37fe640 1 -- 172.21.15.26:0/3560747181 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f6ec4084210 con 0x7f6ecc15ada0 2023-12-19T16:59:44.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.555+0000 7f6ecaffd640 1 -- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6eac065ca0 msgr2=0x7f6eac068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:59:44.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.555+0000 7f6ecaffd640 1 --2- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6eac065ca0 0x7f6eac068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:59:44.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.555+0000 7f6ec37fe640 1 -- 172.21.15.26:0/3560747181 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6ec4051510 con 0x7f6ecc15ada0 2023-12-19T16:59:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.715+0000 7f6ed172c640 1 -- 172.21.15.26:0/3560747181 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ecc074270 con 0x7f6ecc15ada0 2023-12-19T16:59:44.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.715+0000 7f6ec37fe640 1 -- 172.21.15.26:0/3560747181 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ec4054b30 con 0x7f6ecc15ada0 2023-12-19T16:59:44.718 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:59:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.719+0000 7f6ed172c640 1 -- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6eac065ca0 msgr2=0x7f6eac068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:59:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.719+0000 7f6ed172c640 1 --2- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6eac065ca0 0x7f6eac068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.719+0000 7f6ed172c640 1 -- 172.21.15.26:0/3560747181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ecc15ada0 msgr2=0x7f6ecc158160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.719+0000 7f6ed172c640 1 --2- 172.21.15.26:0/3560747181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ecc15ada0 0x7f6ecc158160 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4007e70 tx=0x7f6ec400e4b0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.719+0000 7f6ed172c640 1 -- 172.21.15.26:0/3560747181 shutdown_connections 2023-12-19T16:59:44.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.719+0000 7f6ed172c640 1 --2- 172.21.15.26:0/3560747181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ecc15ada0 0x7f6ecc158160 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.719+0000 7f6ed172c640 1 --2- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ecc15a9c0 0x7f6ecc151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.719+0000 7f6ed172c640 1 --2- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6eac065ca0 0x7f6eac068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.719+0000 7f6ed172c640 1 --2- 172.21.15.26:0/3560747181 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ecc06ac80 0x7f6ecc13c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.719+0000 7f6ed172c640 1 -- 172.21.15.26:0/3560747181 >> 172.21.15.26:0/3560747181 conn(0x7f6ecc0ce730 msgr2=0x7f6ecc0b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.719+0000 7f6ed172c640 1 -- 172.21.15.26:0/3560747181 shutdown_connections 2023-12-19T16:59:44.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.719+0000 7f6ed172c640 1 -- 172.21.15.26:0/3560747181 wait complete. 2023-12-19T16:59:44.740 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:59:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.883+0000 7f7189620640 1 Processor -- start 2023-12-19T16:59:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f7189620640 1 -- start start 2023-12-19T16:59:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f7189620640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f718415ada0 0x7f71840706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f7189620640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f718406ac80 0x7f718406b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f7189620640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7184070d90 con 0x7f718415a9c0 2023-12-19T16:59:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f7189620640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71840cc270 con 0x7f718415ada0 2023-12-19T16:59:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f7189620640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7184071070 con 0x7f718406ac80 2023-12-19T16:59:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f717bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f718415ada0 0x7f71840706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f717bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f718415ada0 0x7f71840706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49634/0 (socket says 172.21.15.26:49634) 2023-12-19T16:59:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f717bfff640 1 -- 172.21.15.26:0/66195193 learned_addr learned my addr 172.21.15.26:0/66195193 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f71837fe640 1 --2- 172.21.15.26:0/66195193 >> v2:172.21.15.160:3300/0 conn(0x7f718406ac80 0x7f718406b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f717bfff640 1 -- 172.21.15.26:0/66195193 >> v2:172.21.15.160:3300/0 conn(0x7f718406ac80 msgr2=0x7f718406b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f717bfff640 1 --2- 172.21.15.26:0/66195193 >> v2:172.21.15.160:3300/0 conn(0x7f718406ac80 0x7f718406b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f717bfff640 1 -- 172.21.15.26:0/66195193 >> v1:172.21.15.160:6789/0 conn(0x7f718415a9c0 legacy=0x7f71840b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f717bfff640 1 -- 172.21.15.26:0/66195193 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7184071350 con 0x7f718415ada0 2023-12-19T16:59:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.887+0000 7f71837fe640 1 --2- 172.21.15.26:0/66195193 >> v2:172.21.15.160:3300/0 conn(0x7f718406ac80 0x7f718406b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:59:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f717bfff640 1 --2- 172.21.15.26:0/66195193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718415ada0 0x7f71840706d0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f71840cc3b0 tx=0x7f716c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33944907ca677e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f71827fc640 1 -- 172.21.15.26:0/66195193 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c0ab620 con 0x7f718415ada0 2023-12-19T16:59:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f71827fc640 1 -- 172.21.15.26:0/66195193 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c0ab780 con 0x7f718415ada0 2023-12-19T16:59:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f71827fc640 1 -- 172.21.15.26:0/66195193 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c0abb10 con 0x7f718415ada0 2023-12-19T16:59:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f7189620640 1 -- 172.21.15.26:0/66195193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718415ada0 msgr2=0x7f71840706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f7189620640 1 --2- 172.21.15.26:0/66195193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718415ada0 0x7f71840706d0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f71840cc3b0 tx=0x7f716c01c400 comp rx=0 tx=0).stop 2023-12-19T16:59:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f7189620640 1 -- 172.21.15.26:0/66195193 shutdown_connections 2023-12-19T16:59:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f7189620640 1 --2- 172.21.15.26:0/66195193 >> v2:172.21.15.160:3300/0 conn(0x7f718406ac80 0x7f718406b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f7189620640 1 --2- 172.21.15.26:0/66195193 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718415ada0 0x7f71840706d0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f7189620640 1 -- 172.21.15.26:0/66195193 >> 172.21.15.26:0/66195193 conn(0x7f71840ce730 msgr2=0x7f71840ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f7189620640 1 -- 172.21.15.26:0/66195193 shutdown_connections 2023-12-19T16:59:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f7189620640 1 -- 172.21.15.26:0/66195193 wait complete. 2023-12-19T16:59:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f7189620640 1 Processor -- start 2023-12-19T16:59:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.891+0000 7f7189620640 1 -- start start 2023-12-19T16:59:44.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7189620640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718406ac80 0x7f7184151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7189620640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f718415a9c0 0x7f7184152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7189620640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f718415ada0 0x7f718414cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7189620640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7184153450 con 0x7f718406ac80 2023-12-19T16:59:44.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7189620640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718414e9b0 con 0x7f718415ada0 2023-12-19T16:59:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7189620640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7184153680 con 0x7f718415a9c0 2023-12-19T16:59:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7182ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718406ac80 0x7f7184151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7182ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718406ac80 0x7f7184151270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49640/0 (socket says 172.21.15.26:49640) 2023-12-19T16:59:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7182ffd640 1 -- 172.21.15.26:0/3914581078 learned_addr learned my addr 172.21.15.26:0/3914581078 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f71837fe640 1 --2- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f718415ada0 0x7f718414cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f717bfff640 1 --2- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f718415a9c0 0x7f7184152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7182ffd640 1 -- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f718415a9c0 msgr2=0x7f7184152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7182ffd640 1 --2- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f718415a9c0 0x7f7184152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7182ffd640 1 -- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f718415ada0 msgr2=0x7f718414cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7182ffd640 1 --2- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f718415ada0 0x7f718414cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7182ffd640 1 -- 172.21.15.26:0/3914581078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c0094d0 con 0x7f718406ac80 2023-12-19T16:59:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7182ffd640 1 --2- 172.21.15.26:0/3914581078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718406ac80 0x7f7184151270 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f717400c450 tx=0x7f717400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f717b7fe640 1 -- 172.21.15.26:0/3914581078 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174012830 con 0x7f718406ac80 2023-12-19T16:59:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f717b7fe640 1 -- 172.21.15.26:0/3914581078 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71740129b0 con 0x7f718406ac80 2023-12-19T16:59:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f717b7fe640 1 -- 172.21.15.26:0/3914581078 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174012d60 con 0x7f718406ac80 2023-12-19T16:59:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7189620640 1 -- 172.21.15.26:0/3914581078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718414e110 con 0x7f718406ac80 2023-12-19T16:59:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7189620640 1 -- 172.21.15.26:0/3914581078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718414e6d0 con 0x7f718406ac80 2023-12-19T16:59:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.895+0000 7f7189620640 1 -- 172.21.15.26:0/3914581078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7184074270 con 0x7f718406ac80 2023-12-19T16:59:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.899+0000 7f717b7fe640 1 -- 172.21.15.26:0/3914581078 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 53) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f717401c460 con 0x7f718406ac80 2023-12-19T16:59:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.899+0000 7f717b7fe640 1 --2- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7164065ca0 0x7f7164068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.899+0000 7f717b7fe640 1 -- 172.21.15.26:0/3914581078 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f7174083770 con 0x7f718406ac80 2023-12-19T16:59:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.899+0000 7f717bfff640 1 -- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7164065ca0 msgr2=0x7f7164068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:59:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.899+0000 7f717bfff640 1 --2- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7164065ca0 0x7f7164068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:59:44.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:44.903+0000 7f717b7fe640 1 -- 172.21.15.26:0/3914581078 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7174050a20 con 0x7f718406ac80 2023-12-19T16:59:44.934 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:44.929+0000 7f386e457180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:59:44.992 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:44.989+0000 7f386e457180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:59:45.008 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:45.007+0000 7fb51358e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T16:59:45.048 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:45.041+0000 7f386e457180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:59:45.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.059+0000 7f7189620640 1 -- 172.21.15.26:0/3914581078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f718414f480 con 0x7f718406ac80 2023-12-19T16:59:45.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.059+0000 7f717b7fe640 1 -- 172.21.15.26:0/3914581078 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7174054040 con 0x7f718406ac80 2023-12-19T16:59:45.061 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:59:45.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.063+0000 7f7189620640 1 -- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7164065ca0 msgr2=0x7f7164068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:59:45.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.063+0000 7f7189620640 1 --2- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7164065ca0 0x7f7164068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.063+0000 7f7189620640 1 -- 172.21.15.26:0/3914581078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718406ac80 msgr2=0x7f7184151270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:45.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.063+0000 7f7189620640 1 --2- 172.21.15.26:0/3914581078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718406ac80 0x7f7184151270 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f717400c450 tx=0x7f717400c910 comp rx=0 tx=0).stop 2023-12-19T16:59:45.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.063+0000 7f7189620640 1 -- 172.21.15.26:0/3914581078 shutdown_connections 2023-12-19T16:59:45.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.063+0000 7f7189620640 1 --2- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f718415ada0 0x7f718414cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.063+0000 7f7189620640 1 --2- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7164065ca0 0x7f7164068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.063+0000 7f7189620640 1 --2- 172.21.15.26:0/3914581078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f718415a9c0 0x7f7184152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.063+0000 7f7189620640 1 --2- 172.21.15.26:0/3914581078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f718406ac80 0x7f7184151270 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.063+0000 7f7189620640 1 -- 172.21.15.26:0/3914581078 >> 172.21.15.26:0/3914581078 conn(0x7f71840ce730 msgr2=0x7f71840ae160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:45.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.063+0000 7f7189620640 1 -- 172.21.15.26:0/3914581078 shutdown_connections 2023-12-19T16:59:45.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.063+0000 7f7189620640 1 -- 172.21.15.26:0/3914581078 wait complete. 2023-12-19T16:59:45.070 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:45.067+0000 7fb51358e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T16:59:45.084 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:59:45.130 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:45.127+0000 7fb51358e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T16:59:45.133 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:45.129+0000 7f386e457180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:59:45.190 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:45.185+0000 7f386e457180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:59:45.221 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:45.219+0000 7fb51358e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T16:59:45.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.231+0000 7f6f50a8c640 1 Processor -- start 2023-12-19T16:59:45.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.231+0000 7f6f50a8c640 1 -- start start 2023-12-19T16:59:45.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.231+0000 7f6f50a8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f4c15a9c0 0x7f6f4c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:45.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.231+0000 7f6f50a8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4c070d90 con 0x7f6f4c06ac80 2023-12-19T16:59:45.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.231+0000 7f6f50a8c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4c071070 con 0x7f6f4c15ada0 2023-12-19T16:59:45.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.231+0000 7f6f50a8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f4c0cc270 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.231+0000 7f6f4a59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f4c15a9c0 0x7f6f4c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:45.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.231+0000 7f6f4a59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f4c15a9c0 0x7f6f4c0706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49644/0 (socket says 172.21.15.26:49644) 2023-12-19T16:59:45.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.231+0000 7f6f4a59c640 1 -- 172.21.15.26:0/2706874699 learned_addr learned my addr 172.21.15.26:0/2706874699 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:45.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.231+0000 7f6f4ad9d640 1 --1- 172.21.15.26:0/2706874699 >> v1:172.21.15.26:6789/0 conn(0x7f6f4c06ac80 0x7f6f4c0b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55916/0 (socket says 172.21.15.26:55916) 2023-12-19T16:59:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.231+0000 7f6f4a59c640 1 -- 172.21.15.26:0/2706874699 >> v1:172.21.15.160:6789/0 conn(0x7f6f4c15ada0 legacy=0x7f6f4c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.231+0000 7f6f4a59c640 1 -- 172.21.15.26:0/2706874699 >> v1:172.21.15.26:6789/0 conn(0x7f6f4c06ac80 legacy=0x7f6f4c0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.231+0000 7f6f4a59c640 1 -- 172.21.15.26:0/2706874699 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c071350 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.235+0000 7f6f4a59c640 1 --2- 172.21.15.26:0/2706874699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c15a9c0 0x7f6f4c0706d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009870 tx=0x7f6f3c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da6f17dbfd7118e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.235+0000 7f6f4959a640 1 -- 172.21.15.26:0/2706874699 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c0a6040 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.235+0000 7f6f4959a640 1 -- 172.21.15.26:0/2706874699 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c0aa600 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.235+0000 7f6f4959a640 1 -- 172.21.15.26:0/2706874699 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c0aa990 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.235+0000 7f6f50a8c640 1 -- 172.21.15.26:0/2706874699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c15a9c0 msgr2=0x7f6f4c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:45.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.235+0000 7f6f50a8c640 1 --2- 172.21.15.26:0/2706874699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c15a9c0 0x7f6f4c0706d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009870 tx=0x7f6f3c01c480 comp rx=0 tx=0).stop 2023-12-19T16:59:45.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.235+0000 7f6f50a8c640 1 -- 172.21.15.26:0/2706874699 shutdown_connections 2023-12-19T16:59:45.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.235+0000 7f6f50a8c640 1 --2- 172.21.15.26:0/2706874699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c15a9c0 0x7f6f4c0706d0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.235+0000 7f6f50a8c640 1 -- 172.21.15.26:0/2706874699 >> 172.21.15.26:0/2706874699 conn(0x7f6f4c0ce730 msgr2=0x7f6f4c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:45.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.235+0000 7f6f50a8c640 1 -- 172.21.15.26:0/2706874699 shutdown_connections 2023-12-19T16:59:45.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.235+0000 7f6f50a8c640 1 -- 172.21.15.26:0/2706874699 wait complete. 2023-12-19T16:59:45.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f50a8c640 1 Processor -- start 2023-12-19T16:59:45.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f50a8c640 1 -- start start 2023-12-19T16:59:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f50a8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c06ac80 0x7f6f4c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f50a8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c15a9c0 0x7f6f4c0c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f50a8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f4c15ada0 0x7f6f4c1567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f50a8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0c5910 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f50a8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0c5bf0 con 0x7f6f4c06ac80 2023-12-19T16:59:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f50a8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0c5ed0 con 0x7f6f4c15ada0 2023-12-19T16:59:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f49d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c15a9c0 0x7f6f4c0c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f4a59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c06ac80 0x7f6f4c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f49d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c15a9c0 0x7f6f4c0c7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49658/0 (socket says 172.21.15.26:49658) 2023-12-19T16:59:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f49d9b640 1 -- 172.21.15.26:0/3892663344 learned_addr learned my addr 172.21.15.26:0/3892663344 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f4ad9d640 1 --2- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f4c15ada0 0x7f6f4c1567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f49d9b640 1 -- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f4c15ada0 msgr2=0x7f6f4c1567c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f49d9b640 1 --2- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f4c15ada0 0x7f6f4c1567c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f49d9b640 1 -- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c06ac80 msgr2=0x7f6f4c15d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f49d9b640 1 --2- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c06ac80 0x7f6f4c15d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f49d9b640 1 -- 172.21.15.26:0/3892663344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c0094d0 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f4a59c640 1 --2- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c06ac80 0x7f6f4c15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:59:45.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f49d9b640 1 --2- 172.21.15.26:0/3892663344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c15a9c0 0x7f6f4c0c7e50 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000c3c0 tx=0x7f6f4000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:45.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f3affd640 1 -- 172.21.15.26:0/3892663344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f400052d0 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f3affd640 1 -- 172.21.15.26:0/3892663344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f40005430 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f3affd640 1 -- 172.21.15.26:0/3892663344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f40003840 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f50a8c640 1 -- 172.21.15.26:0/3892663344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4c0c61b0 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.239+0000 7f6f50a8c640 1 -- 172.21.15.26:0/3892663344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4c0c8390 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.243+0000 7f6f50a8c640 1 -- 172.21.15.26:0/3892663344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4c074270 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.243+0000 7f6f3affd640 1 -- 172.21.15.26:0/3892663344 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 53) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f6f400039a0 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.243+0000 7f6f3affd640 1 --2- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f2c065ca0 0x7f6f2c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:45.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.243+0000 7f6f3affd640 1 -- 172.21.15.26:0/3892663344 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f6f40082df0 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.243+0000 7f6f4a59c640 1 -- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f2c065ca0 msgr2=0x7f6f2c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:59:45.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.243+0000 7f6f4a59c640 1 --2- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f2c065ca0 0x7f6f2c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:59:45.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.247+0000 7f6f3affd640 1 -- 172.21.15.26:0/3892663344 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6f40050170 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.274 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:45.269+0000 7f386e457180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:59:45.283 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:45.283+0000 7fb51358e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T16:59:45.330 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:45.325+0000 7f386e457180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:59:45.373 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:45.371+0000 7fb51358e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T16:59:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.403+0000 7f6f50a8c640 1 -- 172.21.15.26:0/3892663344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f4c0cd7b0 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.403+0000 7f6f3affd640 1 -- 172.21.15.26:0/3892663344 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f400170a0 con 0x7f6f4c15a9c0 2023-12-19T16:59:45.407 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.407+0000 7f6f50a8c640 1 -- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f2c065ca0 msgr2=0x7f6f2c068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.407+0000 7f6f50a8c640 1 --2- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f2c065ca0 0x7f6f2c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.407+0000 7f6f50a8c640 1 -- 172.21.15.26:0/3892663344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c15a9c0 msgr2=0x7f6f4c0c7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.407+0000 7f6f50a8c640 1 --2- 172.21.15.26:0/3892663344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c15a9c0 0x7f6f4c0c7e50 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000c3c0 tx=0x7f6f4000c880 comp rx=0 tx=0).stop 2023-12-19T16:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.407+0000 7f6f50a8c640 1 -- 172.21.15.26:0/3892663344 shutdown_connections 2023-12-19T16:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.407+0000 7f6f50a8c640 1 --2- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6f2c065ca0 0x7f6f2c068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.407+0000 7f6f50a8c640 1 --2- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6f4c15ada0 0x7f6f4c1567c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.407+0000 7f6f50a8c640 1 --2- 172.21.15.26:0/3892663344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f4c15a9c0 0x7f6f4c0c7e50 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.407+0000 7f6f50a8c640 1 --2- 172.21.15.26:0/3892663344 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f4c06ac80 0x7f6f4c15d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.407+0000 7f6f50a8c640 1 -- 172.21.15.26:0/3892663344 >> 172.21.15.26:0/3892663344 conn(0x7f6f4c0ce730 msgr2=0x7f6f4c0c1460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.407+0000 7f6f50a8c640 1 -- 172.21.15.26:0/3892663344 shutdown_connections 2023-12-19T16:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.407+0000 7f6f50a8c640 1 -- 172.21.15.26:0/3892663344 wait complete. 2023-12-19T16:59:45.429 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T16:59:45.434 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:45.431+0000 7fb51358e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T16:59:45.472 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:45.465+0000 7f386e457180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:59:45.531 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:45.525+0000 7f386e457180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:59:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.575+0000 7fd302da2640 1 Processor -- start 2023-12-19T16:59:45.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd302da2640 1 -- start start 2023-12-19T16:59:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd302da2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd2fc15cca0 0x7fd2fc0b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd302da2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2fc0b6b20 0x7fd2fc0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd302da2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2fc0b6fb0 con 0x7fd2fc0b8760 2023-12-19T16:59:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd302da2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2fc0b7c30 con 0x7fd2fc0b6b20 2023-12-19T16:59:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd302da2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2fc0b7290 con 0x7fd2fc15cca0 2023-12-19T16:59:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd3025a1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2fc0b6b20 0x7fd2fc0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd3025a1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2fc0b6b20 0x7fd2fc0d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49660/0 (socket says 172.21.15.26:49660) 2023-12-19T16:59:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd3025a1640 1 -- 172.21.15.26:0/3542663786 learned_addr learned my addr 172.21.15.26:0/3542663786 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd301da0640 1 --2- 172.21.15.26:0/3542663786 >> v2:172.21.15.160:3300/0 conn(0x7fd2fc15cca0 0x7fd2fc0b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd3025a1640 1 -- 172.21.15.26:0/3542663786 >> v2:172.21.15.160:3300/0 conn(0x7fd2fc15cca0 msgr2=0x7fd2fc0b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd3025a1640 1 --2- 172.21.15.26:0/3542663786 >> v2:172.21.15.160:3300/0 conn(0x7fd2fc15cca0 0x7fd2fc0b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd3025a1640 1 -- 172.21.15.26:0/3542663786 >> v1:172.21.15.160:6789/0 conn(0x7fd2fc0b8760 legacy=0x7fd2fc0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:45.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.579+0000 7fd3025a1640 1 -- 172.21.15.26:0/3542663786 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc0b7570 con 0x7fd2fc0b6b20 2023-12-19T16:59:45.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.583+0000 7fd3025a1640 1 --2- 172.21.15.26:0/3542663786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2fc0b6b20 0x7fd2fc0d09e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0b5500 tx=0x7fd2f401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b584aba15ecf23 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:45.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.583+0000 7fd300d9e640 1 -- 172.21.15.26:0/3542663786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f40a5880 con 0x7fd2fc0b6b20 2023-12-19T16:59:45.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.583+0000 7fd300d9e640 1 -- 172.21.15.26:0/3542663786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f40a59e0 con 0x7fd2fc0b6b20 2023-12-19T16:59:45.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.583+0000 7fd300d9e640 1 -- 172.21.15.26:0/3542663786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f40a5d70 con 0x7fd2fc0b6b20 2023-12-19T16:59:45.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.583+0000 7fd302da2640 1 -- 172.21.15.26:0/3542663786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2fc0b6b20 msgr2=0x7fd2fc0d09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:45.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.583+0000 7fd302da2640 1 --2- 172.21.15.26:0/3542663786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2fc0b6b20 0x7fd2fc0d09e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0b5500 tx=0x7fd2f401c9a0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.583+0000 7fd302da2640 1 -- 172.21.15.26:0/3542663786 shutdown_connections 2023-12-19T16:59:45.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.583+0000 7fd302da2640 1 --2- 172.21.15.26:0/3542663786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2fc0b6b20 0x7fd2fc0d09e0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.583+0000 7fd302da2640 1 --2- 172.21.15.26:0/3542663786 >> v2:172.21.15.160:3300/0 conn(0x7fd2fc15cca0 0x7fd2fc0b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.583+0000 7fd302da2640 1 -- 172.21.15.26:0/3542663786 >> 172.21.15.26:0/3542663786 conn(0x7fd2fc06a120 msgr2=0x7fd2fc06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:45.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.583+0000 7fd302da2640 1 -- 172.21.15.26:0/3542663786 shutdown_connections 2023-12-19T16:59:45.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.583+0000 7fd302da2640 1 -- 172.21.15.26:0/3542663786 wait complete. 2023-12-19T16:59:45.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd302da2640 1 Processor -- start 2023-12-19T16:59:45.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd302da2640 1 -- start start 2023-12-19T16:59:45.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd302da2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2fc0b8760 0x7fd2fc14ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:45.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd302da2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2fc15cca0 0x7fd2fc0cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:45.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd302da2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2fc0ca160 0x7fd2fc0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:45.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd302da2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc14ab30 con 0x7fd2fc0b8760 2023-12-19T16:59:45.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd302da2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0cab30 con 0x7fd2fc0ca160 2023-12-19T16:59:45.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd302da2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc14c020 con 0x7fd2fc15cca0 2023-12-19T16:59:45.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd301da0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2fc0b8760 0x7fd2fc14ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:45.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd301da0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2fc0b8760 0x7fd2fc14ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49664/0 (socket says 172.21.15.26:49664) 2023-12-19T16:59:45.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd301da0640 1 -- 172.21.15.26:0/3085106259 learned_addr learned my addr 172.21.15.26:0/3085106259 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:45.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd30159f640 1 --2- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2fc15cca0 0x7fd2fc0cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:45.590 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fb51358e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T16:59:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd3025a1640 1 --2- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2fc0ca160 0x7fd2fc0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd301da0640 1 -- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2fc15cca0 msgr2=0x7fd2fc0cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd301da0640 1 --2- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2fc15cca0 0x7fd2fc0cc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd301da0640 1 -- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2fc0ca160 msgr2=0x7fd2fc0ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd301da0640 1 --2- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2fc0ca160 0x7fd2fc0ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd301da0640 1 -- 172.21.15.26:0/3085106259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4009b30 con 0x7fd2fc0b8760 2023-12-19T16:59:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd301da0640 1 --2- 172.21.15.26:0/3085106259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2fc0b8760 0x7fd2fc14ec10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800e8a0 tx=0x7fd2f800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd2f27fc640 1 -- 172.21.15.26:0/3085106259 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8009510 con 0x7fd2fc0b8760 2023-12-19T16:59:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd2f27fc640 1 -- 172.21.15.26:0/3085106259 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f800c940 con 0x7fd2fc0b8760 2023-12-19T16:59:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd2f27fc640 1 -- 172.21.15.26:0/3085106259 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f800ccb0 con 0x7fd2fc0b8760 2023-12-19T16:59:45.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd302da2640 1 -- 172.21.15.26:0/3085106259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc154940 con 0x7fd2fc0b8760 2023-12-19T16:59:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd302da2640 1 -- 172.21.15.26:0/3085106259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc154380 con 0x7fd2fc0b8760 2023-12-19T16:59:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.587+0000 7fd302da2640 1 -- 172.21.15.26:0/3085106259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc067400 con 0x7fd2fc0b8760 2023-12-19T16:59:45.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.591+0000 7fd2f27fc640 1 -- 172.21.15.26:0/3085106259 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 53) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fd2f8013470 con 0x7fd2fc0b8760 2023-12-19T16:59:45.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.591+0000 7fd2f27fc640 1 --2- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd2e0065ca0 0x7fd2e0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:45.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.591+0000 7fd2f27fc640 1 -- 172.21.15.26:0/3085106259 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6814+0+0 (secure 0 0 0) 0x7fd2f8083a10 con 0x7fd2fc0b8760 2023-12-19T16:59:45.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.591+0000 7fd30159f640 1 -- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd2e0065ca0 msgr2=0x7fd2e0068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:59:45.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.591+0000 7fd30159f640 1 --2- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd2e0065ca0 0x7fd2e0068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:59:45.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.595+0000 7fd2f27fc640 1 -- 172.21.15.26:0/3085106259 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd2f8050d60 con 0x7fd2fc0b8760 2023-12-19T16:59:45.653 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:45.651+0000 7fb51358e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T16:59:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.763+0000 7fd302da2640 1 -- 172.21.15.26:0/3085106259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd2fc06a9a0 con 0x7fd2fc0b8760 2023-12-19T16:59:45.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.767+0000 7fd2f27fc640 1 -- 172.21.15.26:0/3085106259 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v110) v1 ==== 80+0+465 (secure 0 0 0) 0x7fd2f8054380 con 0x7fd2fc0b8760 2023-12-19T16:59:45.768 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1480778234 2023-12-20T16:58:40.327736+0000 2023-12-19T16:59:45.768 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3399946745 2023-12-20T16:58:40.327736+0000 2023-12-19T16:59:45.768 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6833/38098 2023-12-20T16:58:40.327736+0000 2023-12-19T16:59:45.768 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/128716506 2023-12-20T16:58:40.327736+0000 2023-12-19T16:59:45.768 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6832/38098 2023-12-20T16:58:40.327736+0000 2023-12-19T16:59:45.768 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1184346492 2023-12-20T16:58:40.327736+0000 2023-12-19T16:59:45.768 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1271544430 2023-12-20T16:58:40.327736+0000 2023-12-19T16:59:45.768 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/16297988 2023-12-20T16:58:40.327736+0000 2023-12-19T16:59:45.769 INFO:teuthology.orchestra.run.smithi026.stderr:listed 8 entries 2023-12-19T16:59:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.767+0000 7fd302da2640 1 -- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd2e0065ca0 msgr2=0x7fd2e0068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:59:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.767+0000 7fd302da2640 1 --2- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd2e0065ca0 0x7fd2e0068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.767+0000 7fd302da2640 1 -- 172.21.15.26:0/3085106259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2fc0b8760 msgr2=0x7fd2fc14ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.767+0000 7fd302da2640 1 --2- 172.21.15.26:0/3085106259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2fc0b8760 0x7fd2fc14ec10 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800e8a0 tx=0x7fd2f800eca0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.767+0000 7fd302da2640 1 -- 172.21.15.26:0/3085106259 shutdown_connections 2023-12-19T16:59:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.767+0000 7fd302da2640 1 --2- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2fc0ca160 0x7fd2fc0ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.767+0000 7fd302da2640 1 --2- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd2e0065ca0 0x7fd2e0068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.767+0000 7fd302da2640 1 --2- 172.21.15.26:0/3085106259 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2fc15cca0 0x7fd2fc0cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.767+0000 7fd302da2640 1 --2- 172.21.15.26:0/3085106259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2fc0b8760 0x7fd2fc14ec10 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:45.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.767+0000 7fd302da2640 1 -- 172.21.15.26:0/3085106259 >> 172.21.15.26:0/3085106259 conn(0x7fd2fc06a120 msgr2=0x7fd2fc0c7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:45.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.767+0000 7fd302da2640 1 -- 172.21.15.26:0/3085106259 shutdown_connections 2023-12-19T16:59:45.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:45.767+0000 7fd302da2640 1 -- 172.21.15.26:0/3085106259 wait complete. 2023-12-19T16:59:45.771 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:59:45.771 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:59:45.771 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:59:45.778 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:45.773+0000 7f386e457180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:59:45.790 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T16:59:45.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:59:45.790 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-19T16:59:45.842 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T16:59:45.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T16:59:45.843 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T16:59:45.868 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:59:45.868 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:59:45.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:59:45.879 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:59:45.879 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:59:45.905 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T16:59:45.906 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T16:59:45.906 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T16:59:45.916 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:59:45.916 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:59:45.916 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:59:45.917 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:45.909+0000 7f386e457180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:59:45.917 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:45.915+0000 7fb51358e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T16:59:45.927 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:59:45.927 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:59:46.044 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:46.037+0000 7f386e457180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:59:46.063 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:46.059+0000 7fb51358e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:59:46.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.071+0000 7f0bcdf39640 1 Processor -- start 2023-12-19T16:59:46.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.071+0000 7f0bcdf39640 1 -- start start 2023-12-19T16:59:46.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.071+0000 7f0bcdf39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0bc80cc770 0x7f0bc80cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.071+0000 7f0bcdf39640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0bc80afd70 0x7f0bc80cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.071+0000 7f0bcdf39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bc80ce520 0x7f0bc8069d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.071+0000 7f0bcdf39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc806b7c0 con 0x7f0bc80cc770 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.071+0000 7f0bcdf39640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bc80b3240 con 0x7f0bc80afd70 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.071+0000 7f0bcdf39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc8161730 con 0x7f0bc80ce520 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.071+0000 7f0bc77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0bc80cc770 0x7f0bc80cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.071+0000 7f0bc77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0bc80cc770 0x7f0bc80cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47796/0 (socket says 172.21.15.26:47796) 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.071+0000 7f0bc77fe640 1 -- 172.21.15.26:0/685293200 learned_addr learned my addr 172.21.15.26:0/685293200 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.071+0000 7f0bc7fff640 1 --2- 172.21.15.26:0/685293200 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bc80ce520 0x7f0bc8069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.071+0000 7f0bc6ffd640 1 --2- 172.21.15.26:0/685293200 >> v2:172.21.15.160:3300/0 conn(0x7f0bc80afd70 0x7f0bc80cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.075+0000 7f0bc77fe640 1 -- 172.21.15.26:0/685293200 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bc80ce520 msgr2=0x7f0bc8069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.075+0000 7f0bc77fe640 1 --2- 172.21.15.26:0/685293200 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bc80ce520 0x7f0bc8069d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.075+0000 7f0bc77fe640 1 -- 172.21.15.26:0/685293200 >> v2:172.21.15.160:3300/0 conn(0x7f0bc80afd70 msgr2=0x7f0bc80cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.075+0000 7f0bc77fe640 1 --2- 172.21.15.26:0/685293200 >> v2:172.21.15.160:3300/0 conn(0x7f0bc80afd70 0x7f0bc80cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.075+0000 7f0bc77fe640 1 -- 172.21.15.26:0/685293200 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc8161a10 con 0x7f0bc80cc770 2023-12-19T16:59:46.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.075+0000 7f0bc77fe640 1 --2- 172.21.15.26:0/685293200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc80cc770 0x7f0bc80cd130 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80b3520 tx=0x7f0bb801c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90bb335ba19debc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:46.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.075+0000 7f0bc67fc640 1 -- 172.21.15.26:0/685293200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80ab600 con 0x7f0bc80cc770 2023-12-19T16:59:46.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.075+0000 7f0bc67fc640 1 -- 172.21.15.26:0/685293200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb80a7050 con 0x7f0bc80cc770 2023-12-19T16:59:46.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.075+0000 7f0bc67fc640 1 -- 172.21.15.26:0/685293200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80ab980 con 0x7f0bc80cc770 2023-12-19T16:59:46.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 -- 172.21.15.26:0/685293200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc80cc770 msgr2=0x7f0bc80cd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 --2- 172.21.15.26:0/685293200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc80cc770 0x7f0bc80cd130 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80b3520 tx=0x7f0bb801c240 comp rx=0 tx=0).stop 2023-12-19T16:59:46.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 -- 172.21.15.26:0/685293200 shutdown_connections 2023-12-19T16:59:46.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 --2- 172.21.15.26:0/685293200 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bc80ce520 0x7f0bc8069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 --2- 172.21.15.26:0/685293200 >> v2:172.21.15.160:3300/0 conn(0x7f0bc80afd70 0x7f0bc80cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 --2- 172.21.15.26:0/685293200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc80cc770 0x7f0bc80cd130 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 -- 172.21.15.26:0/685293200 >> 172.21.15.26:0/685293200 conn(0x7f0bc806d570 msgr2=0x7f0bc806d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:46.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 -- 172.21.15.26:0/685293200 shutdown_connections 2023-12-19T16:59:46.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 -- 172.21.15.26:0/685293200 wait complete. 2023-12-19T16:59:46.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 Processor -- start 2023-12-19T16:59:46.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 -- start start 2023-12-19T16:59:46.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bc80afd70 0x7f0bc8158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc80cc770 0x7f0bc815e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bc80ce520 0x7f0bc815e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc815bd00 con 0x7f0bc80cc770 2023-12-19T16:59:46.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc8160710 con 0x7f0bc80ce520 2023-12-19T16:59:46.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bcdf39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc815b460 con 0x7f0bc80afd70 2023-12-19T16:59:46.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bc6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc80cc770 0x7f0bc815e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bc77fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bc80afd70 0x7f0bc8158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bc77fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bc80afd70 0x7f0bc8158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:50538/0 (socket says 172.21.15.26:50538) 2023-12-19T16:59:46.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bc77fe640 1 -- 172.21.15.26:0/3019445189 learned_addr learned my addr 172.21.15.26:0/3019445189 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:46.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.079+0000 7f0bc7fff640 1 --2- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bc80ce520 0x7f0bc815e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0bc77fe640 1 -- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bc80ce520 msgr2=0x7f0bc815e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0bc77fe640 1 --2- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bc80ce520 0x7f0bc815e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0bc77fe640 1 -- 172.21.15.26:0/3019445189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc80cc770 msgr2=0x7f0bc815e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0bc77fe640 1 --2- 172.21.15.26:0/3019445189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc80cc770 0x7f0bc815e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0bc77fe640 1 -- 172.21.15.26:0/3019445189 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb80094d0 con 0x7f0bc80afd70 2023-12-19T16:59:46.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0bc77fe640 1 --2- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bc80afd70 0x7f0bc8158a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0bb801cae0 tx=0x7f0bb8018fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:46.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0baffff640 1 -- 172.21.15.26:0/3019445189 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80abe30 con 0x7f0bc80afd70 2023-12-19T16:59:46.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0baffff640 1 -- 172.21.15.26:0/3019445189 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb80b5440 con 0x7f0bc80afd70 2023-12-19T16:59:46.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0bcdf39640 1 -- 172.21.15.26:0/3019445189 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc815b740 con 0x7f0bc80afd70 2023-12-19T16:59:46.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0baffff640 1 -- 172.21.15.26:0/3019445189 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80b57f0 con 0x7f0bc80afd70 2023-12-19T16:59:46.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0bcdf39640 1 -- 172.21.15.26:0/3019445189 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc8158fb0 con 0x7f0bc80afd70 2023-12-19T16:59:46.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0bcdf39640 1 -- 172.21.15.26:0/3019445189 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bc81591a0 con 0x7f0bc80afd70 2023-12-19T16:59:46.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0baffff640 1 -- 172.21.15.26:0/3019445189 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 53) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f0bb80b5950 con 0x7f0bc80afd70 2023-12-19T16:59:46.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0baffff640 1 --2- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bb4065ca0 0x7f0bb4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0baffff640 1 -- 172.21.15.26:0/3019445189 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f0bb811d9c0 con 0x7f0bc80afd70 2023-12-19T16:59:46.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0bc6ffd640 1 -- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bb4065ca0 msgr2=0x7f0bb4068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:59:46.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.083+0000 7f0bc6ffd640 1 --2- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bb4065ca0 0x7f0bb4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:59:46.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.087+0000 7f0baffff640 1 -- 172.21.15.26:0/3019445189 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0bc81591a0 con 0x7f0bc80afd70 2023-12-19T16:59:46.116 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:46.109+0000 7f386e457180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:59:46.197 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:46.195+0000 7fb51358e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T16:59:46.251 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:46.245+0000 7f386e457180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:59:46.271 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:46.271+0000 7fb51358e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T16:59:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.279+0000 7f0bcdf39640 1 -- 172.21.15.26:0/3019445189 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0bc81591a0 con 0x7f0bc80afd70 2023-12-19T16:59:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.279+0000 7f0baffff640 1 -- 172.21.15.26:0/3019445189 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f0bc81591a0 con 0x7f0bc80afd70 2023-12-19T16:59:46.283 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:59:46.283 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3393501439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3393501439},{"type":"v1","addr":"172.21.15.160:6836","nonce":3393501439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1517280064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1517280064},{"type":"v1","addr":"172.21.15.160:6837","nonce":1517280064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/2282339695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2282339695},{"type":"v1","addr":"172.21.15.26:6834","nonce":2282339695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:58:28.364346+0000","modified":"2023-12-19T16:58:31.386135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2693922436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2693922436},{"type":"v1","addr":"172.21.15.26:6835","nonce":2693922436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T16:59:46.283 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2023-12-19T16:59:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.283+0000 7f0bcdf39640 1 -- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bb4065ca0 msgr2=0x7f0bb4068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:59:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.283+0000 7f0bcdf39640 1 --2- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bb4065ca0 0x7f0bb4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.283+0000 7f0bcdf39640 1 -- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bc80afd70 msgr2=0x7f0bc8158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.283+0000 7f0bcdf39640 1 --2- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bc80afd70 0x7f0bc8158a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0bb801cae0 tx=0x7f0bb8018fe0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.283+0000 7f0bcdf39640 1 -- 172.21.15.26:0/3019445189 shutdown_connections 2023-12-19T16:59:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.283+0000 7f0bcdf39640 1 --2- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bc80ce520 0x7f0bc815e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.283+0000 7f0bcdf39640 1 --2- 172.21.15.26:0/3019445189 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bc80cc770 0x7f0bc815e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.283+0000 7f0bcdf39640 1 --2- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bb4065ca0 0x7f0bb4068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.283+0000 7f0bcdf39640 1 --2- 172.21.15.26:0/3019445189 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bc80afd70 0x7f0bc8158a70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.283+0000 7f0bcdf39640 1 -- 172.21.15.26:0/3019445189 >> 172.21.15.26:0/3019445189 conn(0x7f0bc806d570 msgr2=0x7f0bc8156770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.283+0000 7f0bcdf39640 1 -- 172.21.15.26:0/3019445189 shutdown_connections 2023-12-19T16:59:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.283+0000 7f0bcdf39640 1 -- 172.21.15.26:0/3019445189 wait complete. 2023-12-19T16:59:46.306 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:59:46.419 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:46.415+0000 7fb51358e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T16:59:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.447+0000 7f697852c640 1 Processor -- start 2023-12-19T16:59:46.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.447+0000 7f697852c640 1 -- start start 2023-12-19T16:59:46.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.447+0000 7f697852c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6970154220 0x7f697015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.447+0000 7f697852c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f697006b3d0 0x7f69700ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.447+0000 7f697852c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697006bc30 con 0x7f69700cec40 2023-12-19T16:59:46.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.447+0000 7f697852c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69700cf0d0 con 0x7f697006b3d0 2023-12-19T16:59:46.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.447+0000 7f697852c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69700cc6e0 con 0x7f6970154220 2023-12-19T16:59:46.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.451+0000 7f6975ac7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f697006b3d0 0x7f69700ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.451+0000 7f6975ac7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f697006b3d0 0x7f69700ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47808/0 (socket says 172.21.15.26:47808) 2023-12-19T16:59:46.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.451+0000 7f6975ac7640 1 -- 172.21.15.26:0/1898398037 learned_addr learned my addr 172.21.15.26:0/1898398037 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:46.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.451+0000 7f69762c8640 1 --2- 172.21.15.26:0/1898398037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6970154220 0x7f697015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.451+0000 7f6975ac7640 1 -- 172.21.15.26:0/1898398037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6970154220 msgr2=0x7f697015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:46.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.451+0000 7f6975ac7640 1 --2- 172.21.15.26:0/1898398037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6970154220 0x7f697015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.451+0000 7f6975ac7640 1 -- 172.21.15.26:0/1898398037 >> v1:172.21.15.160:6789/0 conn(0x7f69700cec40 legacy=0x7f697015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.451+0000 7f6975ac7640 1 -- 172.21.15.26:0/1898398037 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697015afc0 con 0x7f697006b3d0 2023-12-19T16:59:46.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.451+0000 7f6975ac7640 1 --2- 172.21.15.26:0/1898398037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f697006b3d0 0x7f69700ce700 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f696000bb10 tx=0x7f696001ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7dd41c5febadae server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:46.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.451+0000 7f69752c6640 1 -- 172.21.15.26:0/1898398037 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69600a7bd0 con 0x7f697006b3d0 2023-12-19T16:59:46.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.451+0000 7f69752c6640 1 -- 172.21.15.26:0/1898398037 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69600a7d30 con 0x7f697006b3d0 2023-12-19T16:59:46.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.451+0000 7f69752c6640 1 -- 172.21.15.26:0/1898398037 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69600a7ea0 con 0x7f697006b3d0 2023-12-19T16:59:46.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 -- 172.21.15.26:0/1898398037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f697006b3d0 msgr2=0x7f69700ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 --2- 172.21.15.26:0/1898398037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f697006b3d0 0x7f69700ce700 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f696000bb10 tx=0x7f696001ebe0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 -- 172.21.15.26:0/1898398037 shutdown_connections 2023-12-19T16:59:46.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 --2- 172.21.15.26:0/1898398037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f697006b3d0 0x7f69700ce700 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 --2- 172.21.15.26:0/1898398037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6970154220 0x7f697015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 -- 172.21.15.26:0/1898398037 >> 172.21.15.26:0/1898398037 conn(0x7f69700658e0 msgr2=0x7f6970065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 -- 172.21.15.26:0/1898398037 shutdown_connections 2023-12-19T16:59:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 -- 172.21.15.26:0/1898398037 wait complete. 2023-12-19T16:59:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 Processor -- start 2023-12-19T16:59:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 -- start start 2023-12-19T16:59:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f697006b3d0 0x7f697014ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69700cec40 0x7f69701486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6970154220 0x7f6970147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69701484a0 con 0x7f6970154220 2023-12-19T16:59:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6970070a70 con 0x7f697006b3d0 2023-12-19T16:59:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.455+0000 7f697852c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69700b1dd0 con 0x7f69700cec40 2023-12-19T16:59:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f69762c8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f697006b3d0 0x7f697014ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f6976ac9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6970154220 0x7f6970147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f69762c8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f697006b3d0 0x7f697014ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:40010/0 (socket says 172.21.15.26:40010) 2023-12-19T16:59:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f69762c8640 1 -- 172.21.15.26:0/2341068707 learned_addr learned my addr 172.21.15.26:0/2341068707 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f6975ac7640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69700cec40 0x7f69701486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f69762c8640 1 -- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69700cec40 msgr2=0x7f69701486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f69762c8640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69700cec40 0x7f69701486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f69762c8640 1 -- 172.21.15.26:0/2341068707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6970154220 msgr2=0x7f6970147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f69762c8640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6970154220 0x7f6970147f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f69762c8640 1 -- 172.21.15.26:0/2341068707 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f696000b770 con 0x7f697006b3d0 2023-12-19T16:59:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f6976ac9640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6970154220 0x7f6970147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:59:46.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f6975ac7640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69700cec40 0x7f69701486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:59:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f69762c8640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f697006b3d0 0x7f697014ac00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f696c00e930 tx=0x7f696c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f6966ffd640 1 -- 172.21.15.26:0/2341068707 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c009690 con 0x7f697006b3d0 2023-12-19T16:59:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f6966ffd640 1 -- 172.21.15.26:0/2341068707 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c00ca00 con 0x7f697006b3d0 2023-12-19T16:59:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f6966ffd640 1 -- 172.21.15.26:0/2341068707 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c00cd70 con 0x7f697006b3d0 2023-12-19T16:59:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f697852c640 1 -- 172.21.15.26:0/2341068707 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69700b20b0 con 0x7f697006b3d0 2023-12-19T16:59:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f697852c640 1 -- 172.21.15.26:0/2341068707 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69700b2390 con 0x7f697006b3d0 2023-12-19T16:59:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f697852c640 1 -- 172.21.15.26:0/2341068707 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6970074270 con 0x7f697006b3d0 2023-12-19T16:59:46.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.459+0000 7f6966ffd640 1 -- 172.21.15.26:0/2341068707 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 53) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f696c0059a0 con 0x7f697006b3d0 2023-12-19T16:59:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.463+0000 7f6966ffd640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6954065cd0 0x7f6954068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.463+0000 7f6966ffd640 1 -- 172.21.15.26:0/2341068707 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f696c083fe0 con 0x7f697006b3d0 2023-12-19T16:59:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.463+0000 7f6975ac7640 1 -- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6954065cd0 msgr2=0x7f6954068190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:59:46.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.463+0000 7f6975ac7640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6954065cd0 0x7f6954068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:59:46.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.467+0000 7f6966ffd640 1 -- 172.21.15.26:0/2341068707 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f696c051380 con 0x7f697006b3d0 2023-12-19T16:59:46.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.627+0000 7f697852c640 1 -- 172.21.15.26:0/2341068707 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69700ccbe0 con 0x7f697006b3d0 2023-12-19T16:59:46.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.631+0000 7f6966ffd640 1 -- 172.21.15.26:0/2341068707 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+17754 (secure 0 0 0) 0x7f696c017090 con 0x7f697006b3d0 2023-12-19T16:59:46.633 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:59:46.633 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":110,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:59:41.676863+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:58:26.906185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T16:58:27.358583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1a","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1480778234":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/3399946745":"2023-12-20T16:58:40.327736+0000","172.21.15.160:6833/38098":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/128716506":"2023-12-20T16:58:40.327736+0000","172.21.15.160:6832/38098":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/1184346492":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/1271544430":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/16297988":"2023-12-20T16:58:40.327736+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:59:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.631+0000 7f697852c640 1 -- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6954065cd0 msgr2=0x7f6954068190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:59:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.631+0000 7f697852c640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6954065cd0 0x7f6954068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.631+0000 7f697852c640 1 -- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f697006b3d0 msgr2=0x7f697014ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.631+0000 7f697852c640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f697006b3d0 0x7f697014ac00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f696c00e930 tx=0x7f696c00edf0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.631+0000 7f697852c640 1 -- 172.21.15.26:0/2341068707 shutdown_connections 2023-12-19T16:59:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.631+0000 7f697852c640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6970154220 0x7f6970147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.631+0000 7f697852c640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6954065cd0 0x7f6954068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.631+0000 7f697852c640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69700cec40 0x7f69701486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.631+0000 7f697852c640 1 --2- 172.21.15.26:0/2341068707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f697006b3d0 0x7f697014ac00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.631+0000 7f697852c640 1 -- 172.21.15.26:0/2341068707 >> 172.21.15.26:0/2341068707 conn(0x7f69700658e0 msgr2=0x7f6970065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:46.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.631+0000 7f697852c640 1 -- 172.21.15.26:0/2341068707 shutdown_connections 2023-12-19T16:59:46.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.631+0000 7f697852c640 1 -- 172.21.15.26:0/2341068707 wait complete. 2023-12-19T16:59:46.655 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:58:26.906185+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T16:58:27.358583+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:59:46.655 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:59:46.795 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:46.789+0000 7f386e457180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:59:46.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.803+0000 7f82b3a2d640 1 Processor -- start 2023-12-19T16:59:46.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.803+0000 7f82b3a2d640 1 -- start start 2023-12-19T16:59:46.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.803+0000 7f82b3a2d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f82ac15cde0 0x7f82ac0b8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.803+0000 7f82b3a2d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82ac15d650 0x7f82ac0d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.803+0000 7f82b3a2d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82ac0b5bd0 con 0x7f82ac15d1c0 2023-12-19T16:59:46.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.803+0000 7f82b3a2d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ac15e380 con 0x7f82ac15d650 2023-12-19T16:59:46.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.803+0000 7f82b3a2d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ac0b8b40 con 0x7f82ac15cde0 2023-12-19T16:59:46.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.803+0000 7f82b1fca640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82ac15d650 0x7f82ac0d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.803+0000 7f82b1fca640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82ac15d650 0x7f82ac0d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47826/0 (socket says 172.21.15.26:47826) 2023-12-19T16:59:46.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.803+0000 7f82b1fca640 1 -- 172.21.15.26:0/4104139303 learned_addr learned my addr 172.21.15.26:0/4104139303 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:46.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.803+0000 7f82b17c9640 1 --2- 172.21.15.26:0/4104139303 >> v2:172.21.15.160:3300/0 conn(0x7f82ac15cde0 0x7f82ac0b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.807+0000 7f82b1fca640 1 -- 172.21.15.26:0/4104139303 >> v2:172.21.15.160:3300/0 conn(0x7f82ac15cde0 msgr2=0x7f82ac0b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:46.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.807+0000 7f82b1fca640 1 --2- 172.21.15.26:0/4104139303 >> v2:172.21.15.160:3300/0 conn(0x7f82ac15cde0 0x7f82ac0b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.807+0000 7f82b1fca640 1 -- 172.21.15.26:0/4104139303 >> v1:172.21.15.160:6789/0 conn(0x7f82ac15d1c0 legacy=0x7f82ac0b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.807+0000 7f82b1fca640 1 -- 172.21.15.26:0/4104139303 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac0d0ee0 con 0x7f82ac15d650 2023-12-19T16:59:46.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.807+0000 7f82b1fca640 1 --2- 172.21.15.26:0/4104139303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15d650 0x7f82ac0d1370 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0d0660 tx=0x7f82a401f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f7bda90dada447 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:46.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.807+0000 7f82a3fff640 1 -- 172.21.15.26:0/4104139303 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40af400 con 0x7f82ac15d650 2023-12-19T16:59:46.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.807+0000 7f82a3fff640 1 -- 172.21.15.26:0/4104139303 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a40ab050 con 0x7f82ac15d650 2023-12-19T16:59:46.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.807+0000 7f82a3fff640 1 -- 172.21.15.26:0/4104139303 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40af780 con 0x7f82ac15d650 2023-12-19T16:59:46.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 -- 172.21.15.26:0/4104139303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15d650 msgr2=0x7f82ac0d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 --2- 172.21.15.26:0/4104139303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15d650 0x7f82ac0d1370 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0d0660 tx=0x7f82a401f1c0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 -- 172.21.15.26:0/4104139303 shutdown_connections 2023-12-19T16:59:46.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 --2- 172.21.15.26:0/4104139303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15d650 0x7f82ac0d1370 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 --2- 172.21.15.26:0/4104139303 >> v2:172.21.15.160:3300/0 conn(0x7f82ac15cde0 0x7f82ac0b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 -- 172.21.15.26:0/4104139303 >> 172.21.15.26:0/4104139303 conn(0x7f82ac065ee0 msgr2=0x7f82ac0d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:46.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 -- 172.21.15.26:0/4104139303 shutdown_connections 2023-12-19T16:59:46.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 -- 172.21.15.26:0/4104139303 wait complete. 2023-12-19T16:59:46.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 Processor -- start 2023-12-19T16:59:46.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 -- start start 2023-12-19T16:59:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15cde0 0x7f82ac0b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82ac15d1c0 0x7f82ac156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f82ac156e50 0x7f82ac155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac157300 con 0x7f82ac15cde0 2023-12-19T16:59:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac1577d0 con 0x7f82ac15d1c0 2023-12-19T16:59:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b3a2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac1f5fe0 con 0x7f82ac156e50 2023-12-19T16:59:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b17c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15cde0 0x7f82ac0b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b17c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15cde0 0x7f82ac0b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47830/0 (socket says 172.21.15.26:47830) 2023-12-19T16:59:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b17c9640 1 -- 172.21.15.26:0/2908106929 learned_addr learned my addr 172.21.15.26:0/2908106929 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b0fc8640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82ac15d1c0 0x7f82ac156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b1fca640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f82ac156e50 0x7f82ac155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b17c9640 1 -- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f82ac156e50 msgr2=0x7f82ac155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b17c9640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f82ac156e50 0x7f82ac155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b17c9640 1 -- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82ac15d1c0 msgr2=0x7f82ac156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b17c9640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82ac15d1c0 0x7f82ac156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b17c9640 1 -- 172.21.15.26:0/2908106929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4009b30 con 0x7f82ac15cde0 2023-12-19T16:59:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.811+0000 7f82b1fca640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f82ac156e50 0x7f82ac155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:59:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.815+0000 7f82b17c9640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15cde0 0x7f82ac0b3370 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f82a800e960 tx=0x7f82a800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.815+0000 7f82a1ffb640 1 -- 172.21.15.26:0/2908106929 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8009690 con 0x7f82ac15cde0 2023-12-19T16:59:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.815+0000 7f82a1ffb640 1 -- 172.21.15.26:0/2908106929 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a800ca00 con 0x7f82ac15cde0 2023-12-19T16:59:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.815+0000 7f82a1ffb640 1 -- 172.21.15.26:0/2908106929 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800cd70 con 0x7f82ac15cde0 2023-12-19T16:59:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.815+0000 7f82b3a2d640 1 -- 172.21.15.26:0/2908106929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ac1574f0 con 0x7f82ac15cde0 2023-12-19T16:59:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.815+0000 7f82b3a2d640 1 -- 172.21.15.26:0/2908106929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ac0c9fc0 con 0x7f82ac15cde0 2023-12-19T16:59:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.815+0000 7f82b3a2d640 1 -- 172.21.15.26:0/2908106929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ac074270 con 0x7f82ac15cde0 2023-12-19T16:59:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.815+0000 7f82a1ffb640 1 -- 172.21.15.26:0/2908106929 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 53) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f82a8013470 con 0x7f82ac15cde0 2023-12-19T16:59:46.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.815+0000 7f82a1ffb640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8290065cd0 0x7f8290068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:46.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.815+0000 7f82a1ffb640 1 -- 172.21.15.26:0/2908106929 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f82a8083c10 con 0x7f82ac15cde0 2023-12-19T16:59:46.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.815+0000 7f82b0fc8640 1 -- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8290065cd0 msgr2=0x7f8290068190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:59:46.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.815+0000 7f82b0fc8640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8290065cd0 0x7f8290068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:59:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.819+0000 7f82a1ffb640 1 -- 172.21.15.26:0/2908106929 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f82a8050ec0 con 0x7f82ac15cde0 2023-12-19T16:59:46.879 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:46.873+0000 7f386e457180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:59:46.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82b3a2d640 1 -- 172.21.15.26:0/2908106929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82ac15fb20 con 0x7f82ac15cde0 2023-12-19T16:59:46.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82a1ffb640 1 -- 172.21.15.26:0/2908106929 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82a80170a0 con 0x7f82ac15cde0 2023-12-19T16:59:46.980 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:59:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82b3a2d640 1 -- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8290065cd0 msgr2=0x7f8290068190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:59:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82b3a2d640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8290065cd0 0x7f8290068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82b3a2d640 1 -- 172.21.15.26:0/2908106929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15cde0 msgr2=0x7f82ac0b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82b3a2d640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15cde0 0x7f82ac0b3370 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f82a800e960 tx=0x7f82a800ee20 comp rx=0 tx=0).stop 2023-12-19T16:59:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82b3a2d640 1 -- 172.21.15.26:0/2908106929 shutdown_connections 2023-12-19T16:59:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82b3a2d640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8290065cd0 0x7f8290068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82b3a2d640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f82ac156e50 0x7f82ac155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82b3a2d640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82ac15d1c0 0x7f82ac156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82b3a2d640 1 --2- 172.21.15.26:0/2908106929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82ac15cde0 0x7f82ac0b3370 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82b3a2d640 1 -- 172.21.15.26:0/2908106929 >> 172.21.15.26:0/2908106929 conn(0x7f82ac065ee0 msgr2=0x7f82ac0d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82b3a2d640 1 -- 172.21.15.26:0/2908106929 shutdown_connections 2023-12-19T16:59:46.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:46.979+0000 7f82b3a2d640 1 -- 172.21.15.26:0/2908106929 wait complete. 2023-12-19T16:59:47.002 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:46.999+0000 7fb51358e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T16:59:47.003 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T16:59:47.028 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.021+0000 7f386e457180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:59:47.092 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:47.091+0000 7fb51358e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T16:59:47.111 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.105+0000 7f386e457180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:59:47.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.147+0000 7f7a429bc640 1 Processor -- start 2023-12-19T16:59:47.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.147+0000 7f7a429bc640 1 -- start start 2023-12-19T16:59:47.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.147+0000 7f7a429bc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7a3c15a9c0 0x7f7a3c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.147+0000 7f7a429bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7a3c15ada0 0x7f7a3c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.147+0000 7f7a429bc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3c0b46c0 con 0x7f7a3c06ac80 2023-12-19T16:59:47.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.147+0000 7f7a429bc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c0cc270 con 0x7f7a3c15ada0 2023-12-19T16:59:47.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.147+0000 7f7a429bc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3c070d90 con 0x7f7a3c15a9c0 2023-12-19T16:59:47.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.151+0000 7f7a3b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7a3c15ada0 0x7f7a3c0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.151+0000 7f7a3b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7a3c15ada0 0x7f7a3c0b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47836/0 (socket says 172.21.15.26:47836) 2023-12-19T16:59:47.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.151+0000 7f7a3b7fe640 1 -- 172.21.15.26:0/1785513962 learned_addr learned my addr 172.21.15.26:0/1785513962 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:47.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.151+0000 7f7a3bfff640 1 --2- 172.21.15.26:0/1785513962 >> v2:172.21.15.160:3300/0 conn(0x7f7a3c15a9c0 0x7f7a3c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.151+0000 7f7a3b7fe640 1 -- 172.21.15.26:0/1785513962 >> v2:172.21.15.160:3300/0 conn(0x7f7a3c15a9c0 msgr2=0x7f7a3c0706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:47.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.151+0000 7f7a3b7fe640 1 --2- 172.21.15.26:0/1785513962 >> v2:172.21.15.160:3300/0 conn(0x7f7a3c15a9c0 0x7f7a3c0706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.151+0000 7f7a3b7fe640 1 -- 172.21.15.26:0/1785513962 >> v1:172.21.15.26:6789/0 conn(0x7f7a3c06ac80 legacy=0x7f7a3c0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.151+0000 7f7a3b7fe640 1 -- 172.21.15.26:0/1785513962 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c071070 con 0x7f7a3c15ada0 2023-12-19T16:59:47.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.151+0000 7f7a3b7fe640 1 --2- 172.21.15.26:0/1785513962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3c15ada0 0x7f7a3c0b4180 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0cf590 tx=0x7f7a3001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aae2b0c76618cc9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:47.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.151+0000 7f7a3affd640 1 -- 172.21.15.26:0/1785513962 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a300ab640 con 0x7f7a3c15ada0 2023-12-19T16:59:47.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.151+0000 7f7a3affd640 1 -- 172.21.15.26:0/1785513962 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a300a7070 con 0x7f7a3c15ada0 2023-12-19T16:59:47.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.151+0000 7f7a3affd640 1 -- 172.21.15.26:0/1785513962 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a300ab9c0 con 0x7f7a3c15ada0 2023-12-19T16:59:47.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 -- 172.21.15.26:0/1785513962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3c15ada0 msgr2=0x7f7a3c0b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 --2- 172.21.15.26:0/1785513962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3c15ada0 0x7f7a3c0b4180 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0cf590 tx=0x7f7a3001c440 comp rx=0 tx=0).stop 2023-12-19T16:59:47.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 -- 172.21.15.26:0/1785513962 shutdown_connections 2023-12-19T16:59:47.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 --2- 172.21.15.26:0/1785513962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3c15ada0 0x7f7a3c0b4180 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 --2- 172.21.15.26:0/1785513962 >> v2:172.21.15.160:3300/0 conn(0x7f7a3c15a9c0 0x7f7a3c0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 -- 172.21.15.26:0/1785513962 >> 172.21.15.26:0/1785513962 conn(0x7f7a3c0ce730 msgr2=0x7f7a3c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:47.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 -- 172.21.15.26:0/1785513962 shutdown_connections 2023-12-19T16:59:47.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 -- 172.21.15.26:0/1785513962 wait complete. 2023-12-19T16:59:47.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 Processor -- start 2023-12-19T16:59:47.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 -- start start 2023-12-19T16:59:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3c06ac80 0x7f7a3c14ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a3c15a9c0 0x7f7a3c1542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a3c15ada0 0x7f7a3c154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c0c8e20 con 0x7f7a3c06ac80 2023-12-19T16:59:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c14cfb0 con 0x7f7a3c15ada0 2023-12-19T16:59:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a429bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c155730 con 0x7f7a3c15a9c0 2023-12-19T16:59:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a3bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3c06ac80 0x7f7a3c14ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a3bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3c06ac80 0x7f7a3c14ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47838/0 (socket says 172.21.15.26:47838) 2023-12-19T16:59:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.155+0000 7f7a3bfff640 1 -- 172.21.15.26:0/4213933736 learned_addr learned my addr 172.21.15.26:0/4213933736 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a3b7fe640 1 --2- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a3c15a9c0 0x7f7a3c1542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a40f59640 1 --2- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a3c15ada0 0x7f7a3c154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a3bfff640 1 -- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a3c15a9c0 msgr2=0x7f7a3c1542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a3bfff640 1 --2- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a3c15a9c0 0x7f7a3c1542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a3bfff640 1 -- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a3c15ada0 msgr2=0x7f7a3c154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a3bfff640 1 --2- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a3c15ada0 0x7f7a3c154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a3bfff640 1 -- 172.21.15.26:0/4213933736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a300094d0 con 0x7f7a3c06ac80 2023-12-19T16:59:47.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a3bfff640 1 --2- 172.21.15.26:0/4213933736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3c06ac80 0x7f7a3c14ca70 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00e960 tx=0x7f7a2c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:47.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a38ff9640 1 -- 172.21.15.26:0/4213933736 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c009690 con 0x7f7a3c06ac80 2023-12-19T16:59:47.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a38ff9640 1 -- 172.21.15.26:0/4213933736 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c00ca00 con 0x7f7a3c06ac80 2023-12-19T16:59:47.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a429bc640 1 -- 172.21.15.26:0/4213933736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3c154e90 con 0x7f7a3c06ac80 2023-12-19T16:59:47.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a38ff9640 1 -- 172.21.15.26:0/4213933736 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00cd70 con 0x7f7a3c06ac80 2023-12-19T16:59:47.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a429bc640 1 -- 172.21.15.26:0/4213933736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3c155450 con 0x7f7a3c06ac80 2023-12-19T16:59:47.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a429bc640 1 -- 172.21.15.26:0/4213933736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3c0b3c90 con 0x7f7a3c06ac80 2023-12-19T16:59:47.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a38ff9640 1 -- 172.21.15.26:0/4213933736 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 53) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f7a2c013470 con 0x7f7a3c06ac80 2023-12-19T16:59:47.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.159+0000 7f7a38ff9640 1 --2- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a28065ca0 0x7f7a28068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.163+0000 7f7a38ff9640 1 -- 172.21.15.26:0/4213933736 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f7a2c083b30 con 0x7f7a3c06ac80 2023-12-19T16:59:47.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.163+0000 7f7a3b7fe640 1 -- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a28065ca0 msgr2=0x7f7a28068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T16:59:47.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.163+0000 7f7a3b7fe640 1 --2- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a28065ca0 0x7f7a28068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T16:59:47.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.163+0000 7f7a38ff9640 1 -- 172.21.15.26:0/4213933736 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7a2c050e80 con 0x7f7a3c06ac80 2023-12-19T16:59:47.253 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:47.251+0000 7fb51358e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T16:59:47.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.323+0000 7f7a429bc640 1 -- 172.21.15.26:0/4213933736 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a3c074270 con 0x7f7a3c06ac80 2023-12-19T16:59:47.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.323+0000 7f7a38ff9640 1 -- 172.21.15.26:0/4213933736 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a2c0544a0 con 0x7f7a3c06ac80 2023-12-19T16:59:47.327 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T16:59:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.327+0000 7f7a429bc640 1 -- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a28065ca0 msgr2=0x7f7a28068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T16:59:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.327+0000 7f7a429bc640 1 --2- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a28065ca0 0x7f7a28068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.327+0000 7f7a429bc640 1 -- 172.21.15.26:0/4213933736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3c06ac80 msgr2=0x7f7a3c14ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.327+0000 7f7a429bc640 1 --2- 172.21.15.26:0/4213933736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3c06ac80 0x7f7a3c14ca70 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00e960 tx=0x7f7a2c00ee20 comp rx=0 tx=0).stop 2023-12-19T16:59:47.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.327+0000 7f7a429bc640 1 -- 172.21.15.26:0/4213933736 shutdown_connections 2023-12-19T16:59:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.327+0000 7f7a429bc640 1 --2- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a3c15ada0 0x7f7a3c154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.327+0000 7f7a429bc640 1 --2- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a28065ca0 0x7f7a28068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.327+0000 7f7a429bc640 1 --2- 172.21.15.26:0/4213933736 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7a3c15a9c0 0x7f7a3c1542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.327+0000 7f7a429bc640 1 --2- 172.21.15.26:0/4213933736 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a3c06ac80 0x7f7a3c14ca70 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.327+0000 7f7a429bc640 1 -- 172.21.15.26:0/4213933736 >> 172.21.15.26:0/4213933736 conn(0x7f7a3c0ce730 msgr2=0x7f7a3c159d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.327+0000 7f7a429bc640 1 -- 172.21.15.26:0/4213933736 shutdown_connections 2023-12-19T16:59:47.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.327+0000 7f7a429bc640 1 -- 172.21.15.26:0/4213933736 wait complete. 2023-12-19T16:59:47.343 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:47.339+0000 7fb51358e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T16:59:47.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T16:59:47.426 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.421+0000 7f386e457180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:59:47.486 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.481+0000 7f386e457180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:59:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343fa55640 1 Processor -- start 2023-12-19T16:59:47.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343fa55640 1 -- start start 2023-12-19T16:59:47.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343fa55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343815ccf0 0x7f34380d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343fa55640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f34380b4600 0x7f34380b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343fa55640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34380b4f70 con 0x7f343815d0d0 2023-12-19T16:59:47.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343fa55640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343815e200 con 0x7f34380b4600 2023-12-19T16:59:47.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343fa55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34380d0aa0 con 0x7f343815ccf0 2023-12-19T16:59:47.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343d7f1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343815ccf0 0x7f34380d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343dff2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f34380b4600 0x7f34380b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343d7f1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343815ccf0 0x7f34380d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:50584/0 (socket says 172.21.15.26:50584) 2023-12-19T16:59:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343d7f1640 1 -- 172.21.15.26:0/1016468068 learned_addr learned my addr 172.21.15.26:0/1016468068 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343d7f1640 1 -- 172.21.15.26:0/1016468068 >> v2:172.21.15.160:3300/0 conn(0x7f34380b4600 msgr2=0x7f34380b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343d7f1640 1 --2- 172.21.15.26:0/1016468068 >> v2:172.21.15.160:3300/0 conn(0x7f34380b4600 0x7f34380b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343d7f1640 1 -- 172.21.15.26:0/1016468068 >> v1:172.21.15.160:6789/0 conn(0x7f343815d0d0 legacy=0x7f34380b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.499+0000 7f343d7f1640 1 -- 172.21.15.26:0/1016468068 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34380d1380 con 0x7f343815ccf0 2023-12-19T16:59:47.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.503+0000 7f343d7f1640 1 --2- 172.21.15.26:0/1016468068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343815ccf0 0x7f34380d0e40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f34380b5100 tx=0x7f343401c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b772e1ced0c8935 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.503+0000 7f342ffff640 1 -- 172.21.15.26:0/1016468068 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34340a5790 con 0x7f343815ccf0 2023-12-19T16:59:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.503+0000 7f342ffff640 1 -- 172.21.15.26:0/1016468068 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34340a58f0 con 0x7f343815ccf0 2023-12-19T16:59:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.503+0000 7f342ffff640 1 -- 172.21.15.26:0/1016468068 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34340a5c80 con 0x7f343815ccf0 2023-12-19T16:59:47.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 -- 172.21.15.26:0/1016468068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343815ccf0 msgr2=0x7f34380d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 --2- 172.21.15.26:0/1016468068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343815ccf0 0x7f34380d0e40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f34380b5100 tx=0x7f343401c9c0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 -- 172.21.15.26:0/1016468068 shutdown_connections 2023-12-19T16:59:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 --2- 172.21.15.26:0/1016468068 >> v2:172.21.15.160:3300/0 conn(0x7f34380b4600 0x7f34380b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 --2- 172.21.15.26:0/1016468068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343815ccf0 0x7f34380d0e40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 -- 172.21.15.26:0/1016468068 >> 172.21.15.26:0/1016468068 conn(0x7f3438065ee0 msgr2=0x7f34380cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 -- 172.21.15.26:0/1016468068 shutdown_connections 2023-12-19T16:59:47.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 -- 172.21.15.26:0/1016468068 wait complete. 2023-12-19T16:59:47.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 Processor -- start 2023-12-19T16:59:47.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 -- start start 2023-12-19T16:59:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34380b4600 0x7f3438159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343815ccf0 0x7f3438157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343815d0d0 0x7f3438156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3438159fb0 con 0x7f34380b4600 2023-12-19T16:59:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343814d100 con 0x7f343815d0d0 2023-12-19T16:59:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343fa55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343815f580 con 0x7f343815ccf0 2023-12-19T16:59:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343d7f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34380b4600 0x7f3438159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343d7f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34380b4600 0x7f3438159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47846/0 (socket says 172.21.15.26:47846) 2023-12-19T16:59:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343d7f1640 1 -- 172.21.15.26:0/1386426946 learned_addr learned my addr 172.21.15.26:0/1386426946 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343cff0640 1 --2- 172.21.15.26:0/1386426946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343815ccf0 0x7f3438157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.507+0000 7f343dff2640 1 --2- 172.21.15.26:0/1386426946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343815d0d0 0x7f3438156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.511+0000 7f343d7f1640 1 -- 172.21.15.26:0/1386426946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343815ccf0 msgr2=0x7f3438157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.511+0000 7f343d7f1640 1 --2- 172.21.15.26:0/1386426946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343815ccf0 0x7f3438157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.511+0000 7f343d7f1640 1 -- 172.21.15.26:0/1386426946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343815d0d0 msgr2=0x7f3438156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.511+0000 7f343d7f1640 1 --2- 172.21.15.26:0/1386426946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343815d0d0 0x7f3438156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.511+0000 7f343d7f1640 1 -- 172.21.15.26:0/1386426946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3434009b30 con 0x7f34380b4600 2023-12-19T16:59:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.511+0000 7f343d7f1640 1 --2- 172.21.15.26:0/1386426946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34380b4600 0x7f3438159a70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f34340af040 tx=0x7f3434019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.511+0000 7f342dffb640 1 -- 172.21.15.26:0/1386426946 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34340b8670 con 0x7f34380b4600 2023-12-19T16:59:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.511+0000 7f342dffb640 1 -- 172.21.15.26:0/1386426946 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34340b8810 con 0x7f34380b4600 2023-12-19T16:59:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.511+0000 7f342dffb640 1 -- 172.21.15.26:0/1386426946 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34340b8bc0 con 0x7f34380b4600 2023-12-19T16:59:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.511+0000 7f343fa55640 1 -- 172.21.15.26:0/1386426946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34380cc8b0 con 0x7f34380b4600 2023-12-19T16:59:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.511+0000 7f343fa55640 1 -- 172.21.15.26:0/1386426946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343815fb00 con 0x7f34380b4600 2023-12-19T16:59:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.511+0000 7f343fa55640 1 -- 172.21.15.26:0/1386426946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343815fcf0 con 0x7f34380b4600 2023-12-19T16:59:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.515+0000 7f342dffb640 1 -- 172.21.15.26:0/1386426946 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 54) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f34340b8d20 con 0x7f34380b4600 2023-12-19T16:59:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.515+0000 7f342dffb640 1 -- 172.21.15.26:0/1386426946 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7158+0+0 (secure 0 0 0) 0x7f343411e190 con 0x7f34380b4600 2023-12-19T16:59:47.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.519+0000 7f342dffb640 1 -- 172.21.15.26:0/1386426946 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f34340ebb60 con 0x7f34380b4600 2023-12-19T16:59:47.561 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.557+0000 7f37f6039640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:47.562 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.557+0000 7f37f6039640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:47.562 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.557+0000 7f37f6039640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:47.562 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.557+0000 7f37f6039640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:47.562 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.557+0000 7f37f6039640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:47.562 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.557+0000 7f37f903f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:47.562 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.557+0000 7f37f903f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:47.562 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.557+0000 7f37f903f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:47.562 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.557+0000 7f37f903f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:47.562 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T16:59:47.557+0000 7f37f903f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T16:59:47.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.679+0000 7f343fa55640 1 -- 172.21.15.26:0/1386426946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f343815def0 con 0x7f34380b4600 2023-12-19T16:59:47.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.679+0000 7f342dffb640 1 -- 172.21.15.26:0/1386426946 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+11 (secure 0 0 0) 0x7f34340ef180 con 0x7f34380b4600 2023-12-19T16:59:47.682 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T16:59:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.683+0000 7f343fa55640 1 -- 172.21.15.26:0/1386426946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34380b4600 msgr2=0x7f3438159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.683+0000 7f343fa55640 1 --2- 172.21.15.26:0/1386426946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34380b4600 0x7f3438159a70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f34340af040 tx=0x7f3434019760 comp rx=0 tx=0).stop 2023-12-19T16:59:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.683+0000 7f343fa55640 1 -- 172.21.15.26:0/1386426946 shutdown_connections 2023-12-19T16:59:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.683+0000 7f343fa55640 1 --2- 172.21.15.26:0/1386426946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343815d0d0 0x7f3438156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.683+0000 7f343fa55640 1 --2- 172.21.15.26:0/1386426946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343815ccf0 0x7f3438157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.683+0000 7f343fa55640 1 --2- 172.21.15.26:0/1386426946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34380b4600 0x7f3438159a70 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.683+0000 7f343fa55640 1 -- 172.21.15.26:0/1386426946 >> 172.21.15.26:0/1386426946 conn(0x7f3438065ee0 msgr2=0x7f34380b81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:47.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.683+0000 7f343fa55640 1 -- 172.21.15.26:0/1386426946 shutdown_connections 2023-12-19T16:59:47.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.683+0000 7f343fa55640 1 -- 172.21.15.26:0/1386426946 wait complete. 2023-12-19T16:59:47.698 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:47.695+0000 7fb51358e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T16:59:47.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:59:47.765 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T16:59:47.763+0000 7fb51358e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T16:59:47.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd03562d640 1 Processor -- start 2023-12-19T16:59:47.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd03562d640 1 -- start start 2023-12-19T16:59:47.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd03562d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd030155030 0x7fd03015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd03562d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03015f370 0x7fd03006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd03562d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03006c970 con 0x7fd0300ce740 2023-12-19T16:59:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd03562d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03015b1f0 con 0x7fd030155030 2023-12-19T16:59:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd03562d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03015b4d0 con 0x7fd03015f370 2023-12-19T16:59:47.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd027fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03015f370 0x7fd03006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd02f7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd0300ce740 0x7fd0300ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:47878/0 (socket says 172.21.15.26:47878) 2023-12-19T16:59:47.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd02f7fe640 1 -- 172.21.15.26:0/1812897897 learned_addr learned my addr 172.21.15.26:0/1812897897 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:47.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd02effd640 1 --2- 172.21.15.26:0/1812897897 >> v2:172.21.15.160:3300/0 conn(0x7fd030155030 0x7fd03015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd027fff640 1 -- 172.21.15.26:0/1812897897 >> v2:172.21.15.160:3300/0 conn(0x7fd030155030 msgr2=0x7fd03015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:47.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd027fff640 1 --2- 172.21.15.26:0/1812897897 >> v2:172.21.15.160:3300/0 conn(0x7fd030155030 0x7fd03015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd027fff640 1 -- 172.21.15.26:0/1812897897 >> v1:172.21.15.160:6789/0 conn(0x7fd0300ce740 legacy=0x7fd0300ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.847+0000 7fd027fff640 1 -- 172.21.15.26:0/1812897897 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03015b7b0 con 0x7fd03015f370 2023-12-19T16:59:47.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.851+0000 7fd027fff640 1 --2- 172.21.15.26:0/1812897897 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03015f370 0x7fd03006c080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd0180097c0 tx=0x7fd01801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d986a6a2c83b2b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.851+0000 7fd02e7fc640 1 -- 172.21.15.26:0/1812897897 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0180a6040 con 0x7fd03015f370 2023-12-19T16:59:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.851+0000 7fd02e7fc640 1 -- 172.21.15.26:0/1812897897 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0180aa600 con 0x7fd03015f370 2023-12-19T16:59:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.851+0000 7fd02e7fc640 1 -- 172.21.15.26:0/1812897897 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0180aa990 con 0x7fd03015f370 2023-12-19T16:59:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.851+0000 7fd03562d640 1 -- 172.21.15.26:0/1812897897 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03015f370 msgr2=0x7fd03006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.851+0000 7fd03562d640 1 --2- 172.21.15.26:0/1812897897 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03015f370 0x7fd03006c080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd0180097c0 tx=0x7fd01801c480 comp rx=0 tx=0).stop 2023-12-19T16:59:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.851+0000 7fd03562d640 1 -- 172.21.15.26:0/1812897897 shutdown_connections 2023-12-19T16:59:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.851+0000 7fd03562d640 1 --2- 172.21.15.26:0/1812897897 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03015f370 0x7fd03006c080 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.851+0000 7fd03562d640 1 --2- 172.21.15.26:0/1812897897 >> v2:172.21.15.160:3300/0 conn(0x7fd030155030 0x7fd03015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.851+0000 7fd03562d640 1 -- 172.21.15.26:0/1812897897 >> 172.21.15.26:0/1812897897 conn(0x7fd0300658e0 msgr2=0x7fd030065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.851+0000 7fd03562d640 1 -- 172.21.15.26:0/1812897897 shutdown_connections 2023-12-19T16:59:47.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.851+0000 7fd03562d640 1 -- 172.21.15.26:0/1812897897 wait complete. 2023-12-19T16:59:47.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd03562d640 1 Processor -- start 2023-12-19T16:59:47.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd03562d640 1 -- start start 2023-12-19T16:59:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd03562d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0300ce740 0x7fd03014c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd03562d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd030155030 0x7fd0300b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd03562d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd03015f370 0x7fd03014b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd03562d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300b0ea0 con 0x7fd03015f370 2023-12-19T16:59:47.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd03562d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300af8c0 con 0x7fd0300ce740 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd03562d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300afba0 con 0x7fd030155030 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd02f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd03015f370 0x7fd03014b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd02f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd03015f370 0x7fd03014b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47860/0 (socket says 172.21.15.26:47860) 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd02f7fe640 1 -- 172.21.15.26:0/601608757 learned_addr learned my addr 172.21.15.26:0/601608757 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd02effd640 1 --2- 172.21.15.26:0/601608757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0300ce740 0x7fd03014c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd027fff640 1 --2- 172.21.15.26:0/601608757 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd030155030 0x7fd0300b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd02f7fe640 1 -- 172.21.15.26:0/601608757 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd030155030 msgr2=0x7fd0300b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd02f7fe640 1 --2- 172.21.15.26:0/601608757 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd030155030 0x7fd0300b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd02f7fe640 1 -- 172.21.15.26:0/601608757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0300ce740 msgr2=0x7fd03014c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd02f7fe640 1 --2- 172.21.15.26:0/601608757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0300ce740 0x7fd03014c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd02f7fe640 1 -- 172.21.15.26:0/601608757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd018009420 con 0x7fd03015f370 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd02effd640 1 --2- 172.21.15.26:0/601608757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0300ce740 0x7fd03014c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd02f7fe640 1 --2- 172.21.15.26:0/601608757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd03015f370 0x7fd03014b450 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd02800cbc0 tx=0x7fd02800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:47.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd0277fe640 1 -- 172.21.15.26:0/601608757 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02800ad40 con 0x7fd03015f370 2023-12-19T16:59:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd0277fe640 1 -- 172.21.15.26:0/601608757 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd028012400 con 0x7fd03015f370 2023-12-19T16:59:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd0277fe640 1 -- 172.21.15.26:0/601608757 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02800aea0 con 0x7fd03015f370 2023-12-19T16:59:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd03562d640 1 -- 172.21.15.26:0/601608757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03006c740 con 0x7fd03015f370 2023-12-19T16:59:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.855+0000 7fd03562d640 1 -- 172.21.15.26:0/601608757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0300b0160 con 0x7fd03015f370 2023-12-19T16:59:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.859+0000 7fd03562d640 1 -- 172.21.15.26:0/601608757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd030074270 con 0x7fd03015f370 2023-12-19T16:59:47.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.859+0000 7fd0277fe640 1 -- 172.21.15.26:0/601608757 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 54) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fd028007070 con 0x7fd03015f370 2023-12-19T16:59:47.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.859+0000 7fd0277fe640 1 -- 172.21.15.26:0/601608757 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7158+0+0 (secure 0 0 0) 0x7fd028083000 con 0x7fd03015f370 2023-12-19T16:59:47.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:47.863+0000 7fd0277fe640 1 -- 172.21.15.26:0/601608757 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd028050910 con 0x7fd03015f370 2023-12-19T16:59:48.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.055+0000 7fd03562d640 1 -- 172.21.15.26:0/601608757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0300ccc30 con 0x7fd03015f370 2023-12-19T16:59:48.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.055+0000 7fd0277fe640 1 -- 172.21.15.26:0/601608757 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fd028017020 con 0x7fd03015f370 2023-12-19T16:59:48.058 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:59:48.058 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3393501439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3393501439},{"type":"v1","addr":"172.21.15.160:6836","nonce":3393501439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1517280064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1517280064},{"type":"v1","addr":"172.21.15.160:6837","nonce":1517280064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/2282339695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2282339695},{"type":"v1","addr":"172.21.15.26:6834","nonce":2282339695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:58:28.364346+0000","modified":"2023-12-19T16:58:31.386135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2693922436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2693922436},{"type":"v1","addr":"172.21.15.26:6835","nonce":2693922436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T16:59:48.059 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2023-12-19T16:59:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.059+0000 7fd03562d640 1 -- 172.21.15.26:0/601608757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd03015f370 msgr2=0x7fd03014b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.059+0000 7fd03562d640 1 --2- 172.21.15.26:0/601608757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd03015f370 0x7fd03014b450 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd02800cbc0 tx=0x7fd02800a430 comp rx=0 tx=0).stop 2023-12-19T16:59:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.059+0000 7fd03562d640 1 -- 172.21.15.26:0/601608757 shutdown_connections 2023-12-19T16:59:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.059+0000 7fd03562d640 1 --2- 172.21.15.26:0/601608757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd03015f370 0x7fd03014b450 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.059+0000 7fd03562d640 1 --2- 172.21.15.26:0/601608757 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd030155030 0x7fd0300b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.059+0000 7fd03562d640 1 --2- 172.21.15.26:0/601608757 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0300ce740 0x7fd03014c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.059+0000 7fd03562d640 1 -- 172.21.15.26:0/601608757 >> 172.21.15.26:0/601608757 conn(0x7fd0300658e0 msgr2=0x7fd0300b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.059+0000 7fd03562d640 1 -- 172.21.15.26:0/601608757 shutdown_connections 2023-12-19T16:59:48.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.059+0000 7fd03562d640 1 -- 172.21.15.26:0/601608757 wait complete. 2023-12-19T16:59:48.079 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:59:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03e8ca640 1 Processor -- start 2023-12-19T16:59:48.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03e8ca640 1 -- start start 2023-12-19T16:59:48.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03e8ca640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff03815cbf0 0x7ff0380b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03e8ca640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0380d1a60 0x7ff0380b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03e8ca640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff03815cfd0 con 0x7ff0380b4420 2023-12-19T16:59:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03e8ca640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0380b4df0 con 0x7ff03815cbf0 2023-12-19T16:59:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03e8ca640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0380b6fc0 con 0x7ff0380d1a60 2023-12-19T16:59:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03d8c8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff03815cbf0 0x7ff0380b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03d8c8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff03815cbf0 0x7ff0380b82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47870/0 (socket says 172.21.15.26:47870) 2023-12-19T16:59:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03d8c8640 1 -- 172.21.15.26:0/2433369404 learned_addr learned my addr 172.21.15.26:0/2433369404 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03e0c9640 1 --2- 172.21.15.26:0/2433369404 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0380d1a60 0x7ff0380b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03d8c8640 1 -- 172.21.15.26:0/2433369404 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0380d1a60 msgr2=0x7ff0380b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03d8c8640 1 --2- 172.21.15.26:0/2433369404 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0380d1a60 0x7ff0380b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03d8c8640 1 -- 172.21.15.26:0/2433369404 >> v1:172.21.15.160:6789/0 conn(0x7ff0380b4420 legacy=0x7ff0380b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:48.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.223+0000 7ff03d8c8640 1 -- 172.21.15.26:0/2433369404 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0380b72a0 con 0x7ff03815cbf0 2023-12-19T16:59:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.227+0000 7ff03d8c8640 1 --2- 172.21.15.26:0/2433369404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff03815cbf0 0x7ff0380b82c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff034009870 tx=0x7ff03401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0dee6a090a8863 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:48.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.227+0000 7ff03c8c6640 1 -- 172.21.15.26:0/2433369404 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0340aa610 con 0x7ff03815cbf0 2023-12-19T16:59:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.227+0000 7ff03c8c6640 1 -- 172.21.15.26:0/2433369404 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0340a6040 con 0x7ff03815cbf0 2023-12-19T16:59:48.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.227+0000 7ff03c8c6640 1 -- 172.21.15.26:0/2433369404 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0340aa990 con 0x7ff03815cbf0 2023-12-19T16:59:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.227+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2433369404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff03815cbf0 msgr2=0x7ff0380b82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.227+0000 7ff03e8ca640 1 --2- 172.21.15.26:0/2433369404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff03815cbf0 0x7ff0380b82c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff034009870 tx=0x7ff03401c4b0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.227+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2433369404 shutdown_connections 2023-12-19T16:59:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.227+0000 7ff03e8ca640 1 --2- 172.21.15.26:0/2433369404 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0380d1a60 0x7ff0380b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.227+0000 7ff03e8ca640 1 --2- 172.21.15.26:0/2433369404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff03815cbf0 0x7ff0380b82c0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.227+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2433369404 >> 172.21.15.26:0/2433369404 conn(0x7ff03806a120 msgr2=0x7ff03806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.227+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2433369404 shutdown_connections 2023-12-19T16:59:48.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.227+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2433369404 wait complete. 2023-12-19T16:59:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.227+0000 7ff03e8ca640 1 Processor -- start 2023-12-19T16:59:48.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03e8ca640 1 -- start start 2023-12-19T16:59:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03e8ca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0380b4420 0x7ff03815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03e8ca640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0380d1a60 0x7ff03814dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03e8ca640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff03815cbf0 0x7ff0380c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03e8ca640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03814bf20 con 0x7ff0380b4420 2023-12-19T16:59:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03e8ca640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0380ca5f0 con 0x7ff03815cbf0 2023-12-19T16:59:48.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03e8ca640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03814b3a0 con 0x7ff0380d1a60 2023-12-19T16:59:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03d8c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0380b4420 0x7ff03815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03d8c8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0380b4420 0x7ff03815a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47880/0 (socket says 172.21.15.26:47880) 2023-12-19T16:59:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03d8c8640 1 -- 172.21.15.26:0/2137442107 learned_addr learned my addr 172.21.15.26:0/2137442107 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03d0c7640 1 --2- 172.21.15.26:0/2137442107 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0380d1a60 0x7ff03814dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03e0c9640 1 --2- 172.21.15.26:0/2137442107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff03815cbf0 0x7ff0380c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03d8c8640 1 -- 172.21.15.26:0/2137442107 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0380d1a60 msgr2=0x7ff03814dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03d8c8640 1 --2- 172.21.15.26:0/2137442107 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0380d1a60 0x7ff03814dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03d8c8640 1 -- 172.21.15.26:0/2137442107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff03815cbf0 msgr2=0x7ff0380c4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03d8c8640 1 --2- 172.21.15.26:0/2137442107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff03815cbf0 0x7ff0380c4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03d8c8640 1 -- 172.21.15.26:0/2137442107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0340094d0 con 0x7ff0380b4420 2023-12-19T16:59:48.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03d8c8640 1 --2- 172.21.15.26:0/2137442107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0380b4420 0x7ff03815a0f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff0340ac040 tx=0x7ff03401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff02e7fc640 1 -- 172.21.15.26:0/2137442107 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0340bd5f0 con 0x7ff0380b4420 2023-12-19T16:59:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff02e7fc640 1 -- 172.21.15.26:0/2137442107 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0340bd750 con 0x7ff0380b4420 2023-12-19T16:59:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff02e7fc640 1 -- 172.21.15.26:0/2137442107 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0340bdb00 con 0x7ff0380b4420 2023-12-19T16:59:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2137442107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03814b680 con 0x7ff0380b4420 2023-12-19T16:59:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2137442107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03814bc40 con 0x7ff0380b4420 2023-12-19T16:59:48.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.231+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2137442107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0380ca760 con 0x7ff0380b4420 2023-12-19T16:59:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.235+0000 7ff02e7fc640 1 -- 172.21.15.26:0/2137442107 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 54) v1 ==== 84939+0+0 (secure 0 0 0) 0x7ff0340b33f0 con 0x7ff0380b4420 2023-12-19T16:59:48.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.235+0000 7ff02e7fc640 1 -- 172.21.15.26:0/2137442107 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7158+0+0 (secure 0 0 0) 0x7ff03411dce0 con 0x7ff0380b4420 2023-12-19T16:59:48.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.239+0000 7ff02e7fc640 1 -- 172.21.15.26:0/2137442107 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff0380ca760 con 0x7ff0380b4420 2023-12-19T16:59:48.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.399+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2137442107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0380ca760 con 0x7ff0380b4420 2023-12-19T16:59:48.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.399+0000 7ff02e7fc640 1 -- 172.21.15.26:0/2137442107 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+18255 (secure 0 0 0) 0x7ff0380ca760 con 0x7ff0380b4420 2023-12-19T16:59:48.402 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:59:48.403 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":111,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:59:47.490266+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:58:26.906185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T16:58:27.358583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1a","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3959082448":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/317719705":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/809716033":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/4115775017":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/1480778234":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/3399946745":"2023-12-20T16:58:40.327736+0000","172.21.15.160:6833/38098":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/128716506":"2023-12-20T16:58:40.327736+0000","172.21.15.160:6832/38098":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/3774111672":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/1184346492":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/591216982":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/1271544430":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/1072519326":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/3323047182":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/16297988":"2023-12-20T16:58:40.327736+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:59:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.403+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2137442107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0380b4420 msgr2=0x7ff03815a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.403+0000 7ff03e8ca640 1 --2- 172.21.15.26:0/2137442107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0380b4420 0x7ff03815a0f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff0340ac040 tx=0x7ff03401acf0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.403+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2137442107 shutdown_connections 2023-12-19T16:59:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.403+0000 7ff03e8ca640 1 --2- 172.21.15.26:0/2137442107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff03815cbf0 0x7ff0380c4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.403+0000 7ff03e8ca640 1 --2- 172.21.15.26:0/2137442107 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0380d1a60 0x7ff03814dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.403+0000 7ff03e8ca640 1 --2- 172.21.15.26:0/2137442107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0380b4420 0x7ff03815a0f0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.403+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2137442107 >> 172.21.15.26:0/2137442107 conn(0x7ff03806a120 msgr2=0x7ff0380ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.403+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2137442107 shutdown_connections 2023-12-19T16:59:48.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.403+0000 7ff03e8ca640 1 -- 172.21.15.26:0/2137442107 wait complete. 2023-12-19T16:59:48.424 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T16:59:48.424 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:59:48.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.567+0000 7f4763420640 1 Processor -- start 2023-12-19T16:59:48.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.567+0000 7f4763420640 1 -- start start 2023-12-19T16:59:48.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f4763420640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475c155030 0x7f475c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f4763420640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f475c15f370 0x7f475c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f4763420640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f475c06c970 con 0x7f475c0ce740 2023-12-19T16:59:48.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f4763420640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475c15b1f0 con 0x7f475c15f370 2023-12-19T16:59:48.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f4763420640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475c15b4d0 con 0x7f475c155030 2023-12-19T16:59:48.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f47609bb640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f475c15f370 0x7f475c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f47611bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475c155030 0x7f475c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f47609bb640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f475c15f370 0x7f475c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:40104/0 (socket says 172.21.15.26:40104) 2023-12-19T16:59:48.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f47609bb640 1 -- 172.21.15.26:0/669731581 learned_addr learned my addr 172.21.15.26:0/669731581 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:48.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f47611bc640 1 -- 172.21.15.26:0/669731581 >> v2:172.21.15.160:3300/0 conn(0x7f475c15f370 msgr2=0x7f475c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:48.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f47611bc640 1 --2- 172.21.15.26:0/669731581 >> v2:172.21.15.160:3300/0 conn(0x7f475c15f370 0x7f475c06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f47611bc640 1 -- 172.21.15.26:0/669731581 >> v1:172.21.15.160:6789/0 conn(0x7f475c0ce740 legacy=0x7f475c0ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f47611bc640 1 -- 172.21.15.26:0/669731581 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c15b7b0 con 0x7f475c155030 2023-12-19T16:59:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.571+0000 7f47609bb640 1 --2- 172.21.15.26:0/669731581 >> v2:172.21.15.160:3300/0 conn(0x7f475c15f370 0x7f475c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:59:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f47611bc640 1 --2- 172.21.15.26:0/669731581 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475c155030 0x7f475c15bcd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f47580097c0 tx=0x7f475801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0f18a24bfc1a22 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f4753fff640 1 -- 172.21.15.26:0/669731581 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580a6040 con 0x7f475c155030 2023-12-19T16:59:48.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f4753fff640 1 -- 172.21.15.26:0/669731581 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47580aa600 con 0x7f475c155030 2023-12-19T16:59:48.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f4753fff640 1 -- 172.21.15.26:0/669731581 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580aa990 con 0x7f475c155030 2023-12-19T16:59:48.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f4763420640 1 -- 172.21.15.26:0/669731581 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475c155030 msgr2=0x7f475c15bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f4763420640 1 --2- 172.21.15.26:0/669731581 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475c155030 0x7f475c15bcd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f47580097c0 tx=0x7f475801c480 comp rx=0 tx=0).stop 2023-12-19T16:59:48.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f4763420640 1 -- 172.21.15.26:0/669731581 shutdown_connections 2023-12-19T16:59:48.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f4763420640 1 --2- 172.21.15.26:0/669731581 >> v2:172.21.15.160:3300/0 conn(0x7f475c15f370 0x7f475c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f4763420640 1 --2- 172.21.15.26:0/669731581 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475c155030 0x7f475c15bcd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f4763420640 1 -- 172.21.15.26:0/669731581 >> 172.21.15.26:0/669731581 conn(0x7f475c0658e0 msgr2=0x7f475c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:48.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f4763420640 1 -- 172.21.15.26:0/669731581 shutdown_connections 2023-12-19T16:59:48.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f4763420640 1 -- 172.21.15.26:0/669731581 wait complete. 2023-12-19T16:59:48.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f4763420640 1 Processor -- start 2023-12-19T16:59:48.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.575+0000 7f4763420640 1 -- start start 2023-12-19T16:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f4763420640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f475c0ce740 0x7f475c14c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f4763420640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c155030 0x7f475c148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f4763420640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475c15f370 0x7f475c0ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f4763420640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c14cdb0 con 0x7f475c0ce740 2023-12-19T16:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f4763420640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c14c590 con 0x7f475c155030 2023-12-19T16:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f4763420640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475c15d3b0 con 0x7f475c15f370 2023-12-19T16:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f47611bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f475c0ce740 0x7f475c14c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f47611bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f475c0ce740 0x7f475c14c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47894/0 (socket says 172.21.15.26:47894) 2023-12-19T16:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f47611bc640 1 -- 172.21.15.26:0/622960679 learned_addr learned my addr 172.21.15.26:0/622960679 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f47609bb640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c155030 0x7f475c148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f47619bd640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475c15f370 0x7f475c0ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f47611bc640 1 -- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475c15f370 msgr2=0x7f475c0ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f47611bc640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475c15f370 0x7f475c0ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f47611bc640 1 -- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c155030 msgr2=0x7f475c148200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f47611bc640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c155030 0x7f475c148200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f47611bc640 1 -- 172.21.15.26:0/622960679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c0094d0 con 0x7f475c0ce740 2023-12-19T16:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f47619bd640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475c15f370 0x7f475c0ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f47611bc640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f475c0ce740 0x7f475c14c870 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f475801a0b0 tx=0x7f475801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f4751ffb640 1 -- 172.21.15.26:0/622960679 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580b3800 con 0x7f475c0ce740 2023-12-19T16:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f4751ffb640 1 -- 172.21.15.26:0/622960679 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47580b3980 con 0x7f475c0ce740 2023-12-19T16:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f4763420640 1 -- 172.21.15.26:0/622960679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4758009420 con 0x7f475c0ce740 2023-12-19T16:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f4751ffb640 1 -- 172.21.15.26:0/622960679 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580b3d30 con 0x7f475c0ce740 2023-12-19T16:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f4763420640 1 -- 172.21.15.26:0/622960679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475c06c740 con 0x7f475c0ce740 2023-12-19T16:59:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f4763420640 1 -- 172.21.15.26:0/622960679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475c074270 con 0x7f475c0ce740 2023-12-19T16:59:48.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.579+0000 7f4751ffb640 1 -- 172.21.15.26:0/622960679 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 55) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f47580b2380 con 0x7f475c0ce740 2023-12-19T16:59:48.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.583+0000 7f4751ffb640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f47400658b0 0x7f4740067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.583+0000 7f4751ffb640 1 -- 172.21.15.26:0/622960679 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7158+0+0 (secure 0 0 0) 0x7f475811da20 con 0x7f475c0ce740 2023-12-19T16:59:48.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.583+0000 7f47609bb640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f47400658b0 0x7f4740067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.583+0000 7f47609bb640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f47400658b0 0x7f4740067d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b39d0 tx=0x7f474c009240 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:48.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.587+0000 7f4751ffb640 1 -- 172.21.15.26:0/622960679 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f47580eb270 con 0x7f475c0ce740 2023-12-19T16:59:48.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.775+0000 7f4763420640 1 -- 172.21.15.26:0/622960679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f475c0ccc30 con 0x7f475c0ce740 2023-12-19T16:59:48.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.775+0000 7f4751ffb640 1 -- 172.21.15.26:0/622960679 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f47580ee890 con 0x7f475c0ce740 2023-12-19T16:59:48.779 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:59:48.780 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3393501439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3393501439},{"type":"v1","addr":"172.21.15.160:6836","nonce":3393501439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1517280064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1517280064},{"type":"v1","addr":"172.21.15.160:6837","nonce":1517280064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/2282339695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2282339695},{"type":"v1","addr":"172.21.15.26:6834","nonce":2282339695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:58:28.364346+0000","modified":"2023-12-19T16:58:31.386135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2693922436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2693922436},{"type":"v1","addr":"172.21.15.26:6835","nonce":2693922436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T16:59:48.780 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2023-12-19T16:59:48.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.779+0000 7f4763420640 1 -- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f47400658b0 msgr2=0x7f4740067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.779+0000 7f4763420640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f47400658b0 0x7f4740067d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b39d0 tx=0x7f474c009240 comp rx=0 tx=0).stop 2023-12-19T16:59:48.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.779+0000 7f4763420640 1 -- 172.21.15.26:0/622960679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f475c0ce740 msgr2=0x7f475c14c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.779+0000 7f4763420640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f475c0ce740 0x7f475c14c870 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f475801a0b0 tx=0x7f475801acc0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.779+0000 7f4763420640 1 -- 172.21.15.26:0/622960679 shutdown_connections 2023-12-19T16:59:48.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.779+0000 7f4763420640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f47400658b0 0x7f4740067d70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.779+0000 7f4763420640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f475c15f370 0x7f475c0ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.779+0000 7f4763420640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f475c155030 0x7f475c148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.779+0000 7f4763420640 1 --2- 172.21.15.26:0/622960679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f475c0ce740 0x7f475c14c870 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.779+0000 7f4763420640 1 -- 172.21.15.26:0/622960679 >> 172.21.15.26:0/622960679 conn(0x7f475c0658e0 msgr2=0x7f475c06fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:48.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.779+0000 7f4763420640 1 -- 172.21.15.26:0/622960679 shutdown_connections 2023-12-19T16:59:48.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.779+0000 7f4763420640 1 -- 172.21.15.26:0/622960679 wait complete. 2023-12-19T16:59:48.800 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:59:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a6da2640 1 Processor -- start 2023-12-19T16:59:48.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a6da2640 1 -- start start 2023-12-19T16:59:48.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a6da2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00cdd80 0x7f20a01676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a6da2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a00b6cf0 con 0x7f20a00b8760 2023-12-19T16:59:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a6da2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a00b6fb0 con 0x7f20a015cca0 2023-12-19T16:59:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a6da2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00b7c30 con 0x7f20a00cdd80 2023-12-19T16:59:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a559f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f20a00b8760 0x7f20a00cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43736/0 (socket says 172.21.15.26:43736) 2023-12-19T16:59:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a559f640 1 -- 172.21.15.26:0/926483805 learned_addr learned my addr 172.21.15.26:0/926483805 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:48.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a65a1640 1 --2- 172.21.15.26:0/926483805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00cdd80 0x7f20a01676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a4d9e640 1 -- 172.21.15.26:0/926483805 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131390073 0 0) 0x7f20a00b6cf0 con 0x7f20a00b8760 2023-12-19T16:59:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a4d9e640 1 -- 172.21.15.26:0/926483805 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f208c003530 con 0x7f20a00b8760 2023-12-19T16:59:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a4d9e640 1 -- 172.21.15.26:0/926483805 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244421561 0 0) 0x7f20a00b6fb0 con 0x7f20a015cca0 2023-12-19T16:59:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a4d9e640 1 -- 172.21.15.26:0/926483805 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a00b6cf0 con 0x7f20a015cca0 2023-12-19T16:59:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a65a1640 1 -- 172.21.15.26:0/926483805 >> v1:172.21.15.160:6789/0 conn(0x7f20a015cca0 legacy=0x7f20a00b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a65a1640 1 -- 172.21.15.26:0/926483805 >> v1:172.21.15.26:6789/0 conn(0x7f20a00b8760 legacy=0x7f20a00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.943+0000 7f20a65a1640 1 -- 172.21.15.26:0/926483805 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00b7290 con 0x7f20a00cdd80 2023-12-19T16:59:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.947+0000 7f20a65a1640 1 --2- 172.21.15.26:0/926483805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00cdd80 0x7f20a01676b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2098009dc0 tx=0x7f209801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0d82325b8791bed server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.947+0000 7f20a4d9e640 1 -- 172.21.15.26:0/926483805 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20980a5a00 con 0x7f20a00cdd80 2023-12-19T16:59:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.947+0000 7f20a4d9e640 1 -- 172.21.15.26:0/926483805 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20980a5b60 con 0x7f20a00cdd80 2023-12-19T16:59:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.947+0000 7f20a4d9e640 1 -- 172.21.15.26:0/926483805 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20980ad440 con 0x7f20a00cdd80 2023-12-19T16:59:48.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.947+0000 7f20a6da2640 1 -- 172.21.15.26:0/926483805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00cdd80 msgr2=0x7f20a01676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.947+0000 7f20a6da2640 1 --2- 172.21.15.26:0/926483805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00cdd80 0x7f20a01676b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2098009dc0 tx=0x7f209801c940 comp rx=0 tx=0).stop 2023-12-19T16:59:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.947+0000 7f20a6da2640 1 -- 172.21.15.26:0/926483805 shutdown_connections 2023-12-19T16:59:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.947+0000 7f20a6da2640 1 --2- 172.21.15.26:0/926483805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00cdd80 0x7f20a01676b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.947+0000 7f20a6da2640 1 -- 172.21.15.26:0/926483805 >> 172.21.15.26:0/926483805 conn(0x7f20a006a120 msgr2=0x7f20a006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.947+0000 7f20a6da2640 1 -- 172.21.15.26:0/926483805 shutdown_connections 2023-12-19T16:59:48.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.947+0000 7f20a6da2640 1 -- 172.21.15.26:0/926483805 wait complete. 2023-12-19T16:59:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a6da2640 1 Processor -- start 2023-12-19T16:59:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a6da2640 1 -- start start 2023-12-19T16:59:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a6da2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a00b8760 0x7f20a0153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a6da2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00cdd80 0x7f20a0152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a6da2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20a015cca0 0x7f20a00c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a6da2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00c9ef0 con 0x7f20a015cca0 2023-12-19T16:59:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a6da2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01533e0 con 0x7f20a00b8760 2023-12-19T16:59:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a6da2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0154f10 con 0x7f20a00cdd80 2023-12-19T16:59:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a65a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20a015cca0 0x7f20a00c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a5da0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a00b8760 0x7f20a0153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a65a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20a015cca0 0x7f20a00c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47902/0 (socket says 172.21.15.26:47902) 2023-12-19T16:59:48.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a65a1640 1 -- 172.21.15.26:0/1359633556 learned_addr learned my addr 172.21.15.26:0/1359633556 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a559f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00cdd80 0x7f20a0152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a65a1640 1 -- 172.21.15.26:0/1359633556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00cdd80 msgr2=0x7f20a0152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a65a1640 1 --2- 172.21.15.26:0/1359633556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00cdd80 0x7f20a0152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a65a1640 1 -- 172.21.15.26:0/1359633556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a00b8760 msgr2=0x7f20a0153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a65a1640 1 --2- 172.21.15.26:0/1359633556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a00b8760 0x7f20a0153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a65a1640 1 -- 172.21.15.26:0/1359633556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2098009a70 con 0x7f20a015cca0 2023-12-19T16:59:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a65a1640 1 --2- 172.21.15.26:0/1359633556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20a015cca0 0x7f20a00c6e20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2098019550 tx=0x7f20980196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20967fc640 1 -- 172.21.15.26:0/1359633556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20980adcc0 con 0x7f20a015cca0 2023-12-19T16:59:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20967fc640 1 -- 172.21.15.26:0/1359633556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20980ade40 con 0x7f20a015cca0 2023-12-19T16:59:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a6da2640 1 -- 172.21.15.26:0/1359633556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a00c7b20 con 0x7f20a015cca0 2023-12-19T16:59:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20967fc640 1 -- 172.21.15.26:0/1359633556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20980b7650 con 0x7f20a015cca0 2023-12-19T16:59:48.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a6da2640 1 -- 172.21.15.26:0/1359633556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a0154950 con 0x7f20a015cca0 2023-12-19T16:59:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.951+0000 7f20a6da2640 1 -- 172.21.15.26:0/1359633556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a00ca030 con 0x7f20a015cca0 2023-12-19T16:59:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.955+0000 7f20967fc640 1 -- 172.21.15.26:0/1359633556 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 55) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f20980b4050 con 0x7f20a015cca0 2023-12-19T16:59:48.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.955+0000 7f20967fc640 1 --2- 172.21.15.26:0/1359633556 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f20840658b0 0x7f2084067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:48.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.955+0000 7f20967fc640 1 -- 172.21.15.26:0/1359633556 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7158+0+0 (secure 0 0 0) 0x7f209811e3d0 con 0x7f20a015cca0 2023-12-19T16:59:48.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.955+0000 7f20a5da0640 1 --2- 172.21.15.26:0/1359633556 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f20840658b0 0x7f2084067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:48.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.955+0000 7f20a5da0640 1 --2- 172.21.15.26:0/1359633556 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f20840658b0 0x7f2084067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f209c0011f0 tx=0x7f209c009040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:48.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:48.959+0000 7f20967fc640 1 -- 172.21.15.26:0/1359633556 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f20a00ca030 con 0x7f20a015cca0 2023-12-19T16:59:49.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20a6da2640 1 -- 172.21.15.26:0/1359633556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20a00ca030 con 0x7f20a015cca0 2023-12-19T16:59:49.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20967fc640 1 -- 172.21.15.26:0/1359633556 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f20a00ca030 con 0x7f20a015cca0 2023-12-19T16:59:49.158 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:59:49.158 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3393501439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3393501439},{"type":"v1","addr":"172.21.15.160:6836","nonce":3393501439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1517280064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1517280064},{"type":"v1","addr":"172.21.15.160:6837","nonce":1517280064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/2282339695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2282339695},{"type":"v1","addr":"172.21.15.26:6834","nonce":2282339695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T16:58:28.364346+0000","modified":"2023-12-19T16:58:31.386135+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7479},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7479":{"gid":7479,"name":"c","rank":0,"incarnation":45,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2693922436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2693922436},{"type":"v1","addr":"172.21.15.26:6835","nonce":2693922436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T16:59:49.159 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 49 2023-12-19T16:59:49.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20a6da2640 1 -- 172.21.15.26:0/1359633556 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f20840658b0 msgr2=0x7f2084067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20a6da2640 1 --2- 172.21.15.26:0/1359633556 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f20840658b0 0x7f2084067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f209c0011f0 tx=0x7f209c009040 comp rx=0 tx=0).stop 2023-12-19T16:59:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20a6da2640 1 -- 172.21.15.26:0/1359633556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20a015cca0 msgr2=0x7f20a00c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20a6da2640 1 --2- 172.21.15.26:0/1359633556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20a015cca0 0x7f20a00c6e20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f2098019550 tx=0x7f20980196e0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20a6da2640 1 -- 172.21.15.26:0/1359633556 shutdown_connections 2023-12-19T16:59:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20a6da2640 1 --2- 172.21.15.26:0/1359633556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20a015cca0 0x7f20a00c6e20 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20a6da2640 1 --2- 172.21.15.26:0/1359633556 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f20840658b0 0x7f2084067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20a6da2640 1 --2- 172.21.15.26:0/1359633556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20a00cdd80 0x7f20a0152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20a6da2640 1 --2- 172.21.15.26:0/1359633556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a00b8760 0x7f20a0153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20a6da2640 1 -- 172.21.15.26:0/1359633556 >> 172.21.15.26:0/1359633556 conn(0x7f20a006a120 msgr2=0x7f20a00d2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20a6da2640 1 -- 172.21.15.26:0/1359633556 shutdown_connections 2023-12-19T16:59:49.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.155+0000 7f20a6da2640 1 -- 172.21.15.26:0/1359633556 wait complete. 2023-12-19T16:59:49.178 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:59:49.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379c0f2640 1 Processor -- start 2023-12-19T16:59:49.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379c0f2640 1 -- start start 2023-12-19T16:59:49.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379c0f2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f379406b3d0 0x7f37940ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379c0f2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37940cec40 0x7f379415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379c0f2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f379406bc30 con 0x7f3794154220 2023-12-19T16:59:49.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379c0f2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37940cf0d0 con 0x7f379406b3d0 2023-12-19T16:59:49.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379c0f2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37940cc6e0 con 0x7f37940cec40 2023-12-19T16:59:49.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379968d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f379406b3d0 0x7f37940ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379a68f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37940cec40 0x7f379415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379968d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f379406b3d0 0x7f37940ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:40132/0 (socket says 172.21.15.26:40132) 2023-12-19T16:59:49.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379968d640 1 -- 172.21.15.26:0/762060711 learned_addr learned my addr 172.21.15.26:0/762060711 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:49.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f3799e8e640 1 --1- 172.21.15.26:0/762060711 >> v1:172.21.15.160:6789/0 conn(0x7f3794154220 0x7f379415aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:47908/0 (socket says 172.21.15.26:47908) 2023-12-19T16:59:49.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379a68f640 1 -- 172.21.15.26:0/762060711 >> v2:172.21.15.160:3300/0 conn(0x7f379406b3d0 msgr2=0x7f37940ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:49.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379a68f640 1 --2- 172.21.15.26:0/762060711 >> v2:172.21.15.160:3300/0 conn(0x7f379406b3d0 0x7f37940ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379a68f640 1 -- 172.21.15.26:0/762060711 >> v1:172.21.15.160:6789/0 conn(0x7f3794154220 legacy=0x7f379415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.319+0000 7f379a68f640 1 -- 172.21.15.26:0/762060711 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379406b950 con 0x7f37940cec40 2023-12-19T16:59:49.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.323+0000 7f379a68f640 1 --2- 172.21.15.26:0/762060711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37940cec40 0x7f379415b1c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f378c009d90 tx=0x7f378c01eb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd4649912df987e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.323+0000 7f3798e8c640 1 -- 172.21.15.26:0/762060711 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c0a7c00 con 0x7f37940cec40 2023-12-19T16:59:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.323+0000 7f3798e8c640 1 -- 172.21.15.26:0/762060711 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c0a7d60 con 0x7f37940cec40 2023-12-19T16:59:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.323+0000 7f3798e8c640 1 -- 172.21.15.26:0/762060711 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c0af660 con 0x7f37940cec40 2023-12-19T16:59:49.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.323+0000 7f379c0f2640 1 -- 172.21.15.26:0/762060711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37940cec40 msgr2=0x7f379415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.323+0000 7f379c0f2640 1 --2- 172.21.15.26:0/762060711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37940cec40 0x7f379415b1c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f378c009d90 tx=0x7f378c01eb30 comp rx=0 tx=0).stop 2023-12-19T16:59:49.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.323+0000 7f379c0f2640 1 -- 172.21.15.26:0/762060711 shutdown_connections 2023-12-19T16:59:49.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.323+0000 7f379c0f2640 1 --2- 172.21.15.26:0/762060711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f37940cec40 0x7f379415b1c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.323+0000 7f379c0f2640 1 --2- 172.21.15.26:0/762060711 >> v2:172.21.15.160:3300/0 conn(0x7f379406b3d0 0x7f37940ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.323+0000 7f379c0f2640 1 -- 172.21.15.26:0/762060711 >> 172.21.15.26:0/762060711 conn(0x7f37940658e0 msgr2=0x7f3794065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:49.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.323+0000 7f379c0f2640 1 -- 172.21.15.26:0/762060711 shutdown_connections 2023-12-19T16:59:49.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.323+0000 7f379c0f2640 1 -- 172.21.15.26:0/762060711 wait complete. 2023-12-19T16:59:49.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379c0f2640 1 Processor -- start 2023-12-19T16:59:49.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379c0f2640 1 -- start start 2023-12-19T16:59:49.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379c0f2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f379406b3d0 0x7f379414b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379c0f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37940cec40 0x7f37940ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379c0f2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3794154220 0x7f37940acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379c0f2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3794155570 con 0x7f37940cec40 2023-12-19T16:59:49.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379c0f2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37941f63d0 con 0x7f379406b3d0 2023-12-19T16:59:49.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379c0f2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3794154c40 con 0x7f3794154220 2023-12-19T16:59:49.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379968d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37940cec40 0x7f37940ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379968d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37940cec40 0x7f37940ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47908/0 (socket says 172.21.15.26:47908) 2023-12-19T16:59:49.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379968d640 1 -- 172.21.15.26:0/2816909676 learned_addr learned my addr 172.21.15.26:0/2816909676 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:49.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379a68f640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3794154220 0x7f37940acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f3799e8e640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f379406b3d0 0x7f379414b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379968d640 1 -- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3794154220 msgr2=0x7f37940acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379968d640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3794154220 0x7f37940acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379968d640 1 -- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f379406b3d0 msgr2=0x7f379414b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379968d640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f379406b3d0 0x7f379414b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379968d640 1 -- 172.21.15.26:0/2816909676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c009a70 con 0x7f37940cec40 2023-12-19T16:59:49.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379a68f640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3794154220 0x7f37940acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:59:49.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.327+0000 7f379968d640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37940cec40 0x7f37940ac850 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f378400e7f0 tx=0x7f378400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:49.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.331+0000 7f378a7fc640 1 -- 172.21.15.26:0/2816909676 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784009580 con 0x7f37940cec40 2023-12-19T16:59:49.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.331+0000 7f378a7fc640 1 -- 172.21.15.26:0/2816909676 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378400c940 con 0x7f37940cec40 2023-12-19T16:59:49.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.331+0000 7f378a7fc640 1 -- 172.21.15.26:0/2816909676 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400ccf0 con 0x7f37940cec40 2023-12-19T16:59:49.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.331+0000 7f379c0f2640 1 -- 172.21.15.26:0/2816909676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3794154f20 con 0x7f37940cec40 2023-12-19T16:59:49.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.331+0000 7f379c0f2640 1 -- 172.21.15.26:0/2816909676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379414b900 con 0x7f37940cec40 2023-12-19T16:59:49.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.331+0000 7f379c0f2640 1 -- 172.21.15.26:0/2816909676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37941556e0 con 0x7f37940cec40 2023-12-19T16:59:49.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.331+0000 7f378a7fc640 1 -- 172.21.15.26:0/2816909676 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 55) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f3784013470 con 0x7f37940cec40 2023-12-19T16:59:49.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.331+0000 7f378a7fc640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f37780658b0 0x7f3778067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.331+0000 7f378a7fc640 1 -- 172.21.15.26:0/2816909676 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7158+0+0 (secure 0 0 0) 0x7f37840835b0 con 0x7f37940cec40 2023-12-19T16:59:49.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.331+0000 7f3799e8e640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f37780658b0 0x7f3778067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.335+0000 7f3799e8e640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f37780658b0 0x7f3778067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3790000f70 tx=0x7f3790007cd0 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:49.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.335+0000 7f378a7fc640 1 -- 172.21.15.26:0/2816909676 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3784050c30 con 0x7f37940cec40 2023-12-19T16:59:49.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.495+0000 7f379c0f2640 1 -- 172.21.15.26:0/2816909676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3794074270 con 0x7f37940cec40 2023-12-19T16:59:49.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.495+0000 7f378a7fc640 1 -- 172.21.15.26:0/2816909676 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+18255 (secure 0 0 0) 0x7f3784054250 con 0x7f37940cec40 2023-12-19T16:59:49.498 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:59:49.498 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":111,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:59:47.490266+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:58:26.906185+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T16:58:27.358583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1a","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3959082448":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/317719705":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/809716033":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/4115775017":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/1480778234":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/3399946745":"2023-12-20T16:58:40.327736+0000","172.21.15.160:6833/38098":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/128716506":"2023-12-20T16:58:40.327736+0000","172.21.15.160:6832/38098":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/3774111672":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/1184346492":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/591216982":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/1271544430":"2023-12-20T16:58:40.327736+0000","172.21.15.160:0/1072519326":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/3323047182":"2023-12-20T16:59:47.490228+0000","172.21.15.160:0/16297988":"2023-12-20T16:58:40.327736+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:59:49.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.499+0000 7f379c0f2640 1 -- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f37780658b0 msgr2=0x7f3778067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.499+0000 7f379c0f2640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f37780658b0 0x7f3778067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3790000f70 tx=0x7f3790007cd0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.499+0000 7f379c0f2640 1 -- 172.21.15.26:0/2816909676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37940cec40 msgr2=0x7f37940ac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.499+0000 7f379c0f2640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37940cec40 0x7f37940ac850 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f378400e7f0 tx=0x7f378400ecb0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.499+0000 7f379c0f2640 1 -- 172.21.15.26:0/2816909676 shutdown_connections 2023-12-19T16:59:49.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.499+0000 7f379c0f2640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f37780658b0 0x7f3778067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.499+0000 7f379c0f2640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3794154220 0x7f37940acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.499+0000 7f379c0f2640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37940cec40 0x7f37940ac850 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.499+0000 7f379c0f2640 1 --2- 172.21.15.26:0/2816909676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f379406b3d0 0x7f379414b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.499+0000 7f379c0f2640 1 -- 172.21.15.26:0/2816909676 >> 172.21.15.26:0/2816909676 conn(0x7f37940658e0 msgr2=0x7f379415d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:49.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.499+0000 7f379c0f2640 1 -- 172.21.15.26:0/2816909676 shutdown_connections 2023-12-19T16:59:49.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.499+0000 7f379c0f2640 1 -- 172.21.15.26:0/2816909676 wait complete. 2023-12-19T16:59:49.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T16:59:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.663+0000 7f3f7b48c640 1 Processor -- start 2023-12-19T16:59:49.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.663+0000 7f3f7b48c640 1 -- start start 2023-12-19T16:59:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.663+0000 7f3f7b48c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3f740b2d50 0x7f3f7416c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.663+0000 7f3f7b48c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f7406d4b0 con 0x7f3f740cb1d0 2023-12-19T16:59:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.663+0000 7f3f7b48c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f7406cc10 con 0x7f3f7406bee0 2023-12-19T16:59:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.663+0000 7f3f7b48c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f740cc030 con 0x7f3f740b2d50 2023-12-19T16:59:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.663+0000 7f3f79228640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3f740cb1d0 0x7f3f740cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43748/0 (socket says 172.21.15.26:43748) 2023-12-19T16:59:49.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.663+0000 7f3f79228640 1 -- 172.21.15.26:0/4140225228 learned_addr learned my addr 172.21.15.26:0/4140225228 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f79a29640 1 --2- 172.21.15.26:0/4140225228 >> v2:172.21.15.160:3300/0 conn(0x7f3f740b2d50 0x7f3f7416c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148271097 0 0) 0x7f3f7406d4b0 con 0x7f3f740cb1d0 2023-12-19T16:59:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f60003530 con 0x7f3f740cb1d0 2023-12-19T16:59:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3819690234 0 0) 0x7f3f60003530 con 0x7f3f740cb1d0 2023-12-19T16:59:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 >> v2:172.21.15.160:3300/0 conn(0x7f3f740b2d50 msgr2=0x7f3f7416c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 --2- 172.21.15.26:0/4140225228 >> v2:172.21.15.160:3300/0 conn(0x7f3f740b2d50 0x7f3f7416c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 >> v1:172.21.15.160:6789/0 conn(0x7f3f7406bee0 legacy=0x7f3f7406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7406cef0 con 0x7f3f740cb1d0 2023-12-19T16:59:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f3f70001680 con 0x7f3f740cb1d0 2023-12-19T16:59:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 >> v1:172.21.15.26:6789/0 conn(0x7f3f740cb1d0 legacy=0x7f3f740cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 --2- 172.21.15.26:0/4140225228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f60003980 0x7f3f60003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 --2- 172.21.15.26:0/4140225228 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f600047d0 0x7f3f60004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 --2- 172.21.15.26:0/4140225228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f60005190 0x7f3f6000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f740cc030 con 0x7f3f60005190 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6000c130 con 0x7f3f60003980 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6000c2f0 con 0x7f3f600047d0 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f79a29640 1 --2- 172.21.15.26:0/4140225228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f60005190 0x7f3f6000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f79228640 1 --2- 172.21.15.26:0/4140225228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f60003980 0x7f3f60003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f78a27640 1 --2- 172.21.15.26:0/4140225228 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f600047d0 0x7f3f60004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f79a29640 1 -- 172.21.15.26:0/4140225228 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f600047d0 msgr2=0x7f3f60004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f79a29640 1 --2- 172.21.15.26:0/4140225228 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f600047d0 0x7f3f60004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f79a29640 1 -- 172.21.15.26:0/4140225228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f60003980 msgr2=0x7f3f60003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f79a29640 1 --2- 172.21.15.26:0/4140225228 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f60003980 0x7f3f60003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f79a29640 1 -- 172.21.15.26:0/4140225228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f60003530 con 0x7f3f60005190 2023-12-19T16:59:49.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f78a27640 1 --2- 172.21.15.26:0/4140225228 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f600047d0 0x7f3f60004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:59:49.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.667+0000 7f3f79a29640 1 -- 172.21.15.26:0/4140225228 reap_dead start 2023-12-19T16:59:49.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f79a29640 1 --2- 172.21.15.26:0/4140225228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f60005190 0x7f3f6000b990 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009f90 tx=0x7f3f6c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36fbde82c7df605 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:49.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0a5b70 con 0x7f3f60005190 2023-12-19T16:59:49.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c00f040 con 0x7f3f60005190 2023-12-19T16:59:49.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f6bfff640 1 -- 172.21.15.26:0/4140225228 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0ad440 con 0x7f3f60005190 2023-12-19T16:59:49.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f7b48c640 1 -- 172.21.15.26:0/4140225228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f60005190 msgr2=0x7f3f6000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f7b48c640 1 --2- 172.21.15.26:0/4140225228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f60005190 0x7f3f6000b990 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009f90 tx=0x7f3f6c01cc60 comp rx=0 tx=0).stop 2023-12-19T16:59:49.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f7b48c640 1 -- 172.21.15.26:0/4140225228 shutdown_connections 2023-12-19T16:59:49.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f7b48c640 1 --2- 172.21.15.26:0/4140225228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f60005190 0x7f3f6000b990 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f7b48c640 1 -- 172.21.15.26:0/4140225228 >> 172.21.15.26:0/4140225228 conn(0x7f3f740658e0 msgr2=0x7f3f74065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:49.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f7b48c640 1 -- 172.21.15.26:0/4140225228 shutdown_connections 2023-12-19T16:59:49.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f7b48c640 1 -- 172.21.15.26:0/4140225228 wait complete. 2023-12-19T16:59:49.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f7b48c640 1 Processor -- start 2023-12-19T16:59:49.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f7b48c640 1 -- start start 2023-12-19T16:59:49.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f7b48c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f60005190 0x7f3f74158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.671+0000 7f3f7b48c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f741f6540 0x7f3f7415c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f7b48c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f740afee0 0x7f3f7415fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f7b48c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f740ae260 con 0x7f3f740afee0 2023-12-19T16:59:49.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f7b48c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f740ae540 con 0x7f3f741f6540 2023-12-19T16:59:49.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f7b48c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7415cb60 con 0x7f3f60005190 2023-12-19T16:59:49.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f79a29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f740afee0 0x7f3f7415fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f79a29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f740afee0 0x7f3f7415fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47938/0 (socket says 172.21.15.26:47938) 2023-12-19T16:59:49.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f79a29640 1 -- 172.21.15.26:0/3279179871 learned_addr learned my addr 172.21.15.26:0/3279179871 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:49.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f79228640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f60005190 0x7f3f74158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f78a27640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f741f6540 0x7f3f7415c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f79a29640 1 -- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f60005190 msgr2=0x7f3f74158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f79a29640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f60005190 0x7f3f74158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f79a29640 1 -- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f741f6540 msgr2=0x7f3f7415c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f79a29640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f741f6540 0x7f3f7415c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f79a29640 1 -- 172.21.15.26:0/3279179871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7415ce40 con 0x7f3f740afee0 2023-12-19T16:59:49.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f79a29640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f740afee0 0x7f3f7415fed0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c01d500 tx=0x7f3f6c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:49.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f69ffb640 1 -- 172.21.15.26:0/3279179871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0bd070 con 0x7f3f740afee0 2023-12-19T16:59:49.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f69ffb640 1 -- 172.21.15.26:0/3279179871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0b8650 con 0x7f3f740afee0 2023-12-19T16:59:49.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f69ffb640 1 -- 172.21.15.26:0/3279179871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0b8a00 con 0x7f3f740afee0 2023-12-19T16:59:49.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f7b48c640 1 -- 172.21.15.26:0/3279179871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7415d120 con 0x7f3f740afee0 2023-12-19T16:59:49.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f7b48c640 1 -- 172.21.15.26:0/3279179871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7415c730 con 0x7f3f740afee0 2023-12-19T16:59:49.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f7b48c640 1 -- 172.21.15.26:0/3279179871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7415d2b0 con 0x7f3f740afee0 2023-12-19T16:59:49.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.675+0000 7f3f69ffb640 1 -- 172.21.15.26:0/3279179871 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 55) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f3f6c0b4040 con 0x7f3f740afee0 2023-12-19T16:59:49.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.679+0000 7f3f69ffb640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f580658b0 0x7f3f58067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:49.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.679+0000 7f3f69ffb640 1 -- 172.21.15.26:0/3279179871 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 7158+0+0 (secure 0 0 0) 0x7f3f6c11d5a0 con 0x7f3f740afee0 2023-12-19T16:59:49.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.679+0000 7f3f79228640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f580658b0 0x7f3f58067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:49.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.679+0000 7f3f79228640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f580658b0 0x7f3f58067d70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f70001060 tx=0x7f3f70000ff0 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:49.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.683+0000 7f3f69ffb640 1 -- 172.21.15.26:0/3279179871 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3f6c0eada0 con 0x7f3f740afee0 2023-12-19T16:59:49.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.723+0000 7f3f69ffb640 1 -- 172.21.15.26:0/3279179871 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 56) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f3f6c0eb170 con 0x7f3f740afee0 2023-12-19T16:59:49.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.875+0000 7f3f7b48c640 1 -- 172.21.15.26:0/3279179871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3f740ae6b0 con 0x7f3f740afee0 2023-12-19T16:59:49.876 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T16:59:49.875+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T16:59:49.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.883+0000 7f3f69ffb640 1 -- 172.21.15.26:0/3279179871 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3f6c0f1630 con 0x7f3f740afee0 2023-12-19T16:59:49.885 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T16:59:49.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.887+0000 7f3f7b48c640 1 -- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f580658b0 msgr2=0x7f3f58067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.887+0000 7f3f7b48c640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f580658b0 0x7f3f58067d70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3f70001060 tx=0x7f3f70000ff0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.887+0000 7f3f7b48c640 1 -- 172.21.15.26:0/3279179871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f740afee0 msgr2=0x7f3f7415fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:49.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.887+0000 7f3f7b48c640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f740afee0 0x7f3f7415fed0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c01d500 tx=0x7f3f6c009750 comp rx=0 tx=0).stop 2023-12-19T16:59:49.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.887+0000 7f3f7b48c640 1 -- 172.21.15.26:0/3279179871 shutdown_connections 2023-12-19T16:59:49.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.887+0000 7f3f7b48c640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f740afee0 0x7f3f7415fed0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.887+0000 7f3f7b48c640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f741f6540 0x7f3f7415c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.887+0000 7f3f7b48c640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f580658b0 0x7f3f58067d70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.887+0000 7f3f7b48c640 1 --2- 172.21.15.26:0/3279179871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f60005190 0x7f3f74158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:49.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.887+0000 7f3f7b48c640 1 -- 172.21.15.26:0/3279179871 >> 172.21.15.26:0/3279179871 conn(0x7f3f740658e0 msgr2=0x7f3f74065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:49.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.887+0000 7f3f7b48c640 1 -- 172.21.15.26:0/3279179871 shutdown_connections 2023-12-19T16:59:49.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:49.887+0000 7f3f7b48c640 1 -- 172.21.15.26:0/3279179871 wait complete. 2023-12-19T16:59:49.897 INFO:tasks.ceph.mds.c.smithi026.stderr: -7347> 2023-12-19T16:58:29.880+0000 7f606171c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T16:59:49.908 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T16:59:49.949 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T16:59:50.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.055+0000 7f91b30e5640 1 Processor -- start 2023-12-19T16:59:50.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.055+0000 7f91b30e5640 1 -- start start 2023-12-19T16:59:50.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91b30e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac15cc70 0x7f91ac0d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:50.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91b30e5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f91ac0b88b0 0x7f91ac0b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:50.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91b30e5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91ac0b71f0 con 0x7f91ac0b6140 2023-12-19T16:59:50.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91b30e5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91ac0b65e0 con 0x7f91ac0b88b0 2023-12-19T16:59:50.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91b30e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac0b68c0 con 0x7f91ac15cc70 2023-12-19T16:59:50.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91b1682640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f91ac0b6140 0x7f91ac0cfbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43768/0 (socket says 172.21.15.26:43768) 2023-12-19T16:59:50.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91b1682640 1 -- 172.21.15.26:0/2793861715 learned_addr learned my addr 172.21.15.26:0/2793861715 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:50.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91abfff640 1 --2- 172.21.15.26:0/2793861715 >> v2:172.21.15.160:3300/0 conn(0x7f91ac0b88b0 0x7f91ac0b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:50.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91b0e81640 1 --2- 172.21.15.26:0/2793861715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac15cc70 0x7f91ac0d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91ab7fe640 1 -- 172.21.15.26:0/2793861715 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693621773 0 0) 0x7f91ac0b71f0 con 0x7f91ac0b6140 2023-12-19T16:59:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91ab7fe640 1 -- 172.21.15.26:0/2793861715 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f919c003530 con 0x7f91ac0b6140 2023-12-19T16:59:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91abfff640 1 -- 172.21.15.26:0/2793861715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac15cc70 msgr2=0x7f91ac0d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91abfff640 1 --2- 172.21.15.26:0/2793861715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac15cc70 0x7f91ac0d0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91abfff640 1 -- 172.21.15.26:0/2793861715 >> v1:172.21.15.26:6789/0 conn(0x7f91ac0b6140 legacy=0x7f91ac0cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91abfff640 1 -- 172.21.15.26:0/2793861715 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac0b6ba0 con 0x7f91ac0b88b0 2023-12-19T16:59:50.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.059+0000 7f91b0e81640 1 --2- 172.21.15.26:0/2793861715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac15cc70 0x7f91ac0d0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:59:50.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.063+0000 7f91abfff640 1 --2- 172.21.15.26:0/2793861715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0b88b0 0x7f91ac0b5c00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f91980097c0 tx=0x7f919801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93a516aba854b310 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:50.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.063+0000 7f91ab7fe640 1 -- 172.21.15.26:0/2793861715 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91980a8970 con 0x7f91ac0b88b0 2023-12-19T16:59:50.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.063+0000 7f91ab7fe640 1 -- 172.21.15.26:0/2793861715 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91980a8ad0 con 0x7f91ac0b88b0 2023-12-19T16:59:50.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.063+0000 7f91ab7fe640 1 -- 172.21.15.26:0/2793861715 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91980a8e40 con 0x7f91ac0b88b0 2023-12-19T16:59:50.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.063+0000 7f91b30e5640 1 -- 172.21.15.26:0/2793861715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0b88b0 msgr2=0x7f91ac0b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:50.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.063+0000 7f91b30e5640 1 --2- 172.21.15.26:0/2793861715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0b88b0 0x7f91ac0b5c00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f91980097c0 tx=0x7f919801ec70 comp rx=0 tx=0).stop 2023-12-19T16:59:50.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.063+0000 7f91b30e5640 1 -- 172.21.15.26:0/2793861715 shutdown_connections 2023-12-19T16:59:50.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.063+0000 7f91b30e5640 1 --2- 172.21.15.26:0/2793861715 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0b88b0 0x7f91ac0b5c00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:50.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.063+0000 7f91b30e5640 1 --2- 172.21.15.26:0/2793861715 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac15cc70 0x7f91ac0d0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:50.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.063+0000 7f91b30e5640 1 -- 172.21.15.26:0/2793861715 >> 172.21.15.26:0/2793861715 conn(0x7f91ac06a120 msgr2=0x7f91ac06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:50.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.063+0000 7f91b30e5640 1 -- 172.21.15.26:0/2793861715 shutdown_connections 2023-12-19T16:59:50.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.063+0000 7f91b30e5640 1 -- 172.21.15.26:0/2793861715 wait complete. 2023-12-19T16:59:50.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b30e5640 1 Processor -- start 2023-12-19T16:59:50.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b30e5640 1 -- start start 2023-12-19T16:59:50.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b30e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac0b6140 0x7f91ac147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:50.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b30e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0b88b0 0x7f91ac158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:50.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b30e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac15cc70 0x7f91ac1f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:50.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b30e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac15ed60 con 0x7f91ac15cc70 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b30e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac15f040 con 0x7f91ac0b88b0 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b30e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac15f320 con 0x7f91ac0b6140 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b1682640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac15cc70 0x7f91ac1f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91abfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0b88b0 0x7f91ac158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b1682640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac15cc70 0x7f91ac1f26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47970/0 (socket says 172.21.15.26:47970) 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b1682640 1 -- 172.21.15.26:0/3480355237 learned_addr learned my addr 172.21.15.26:0/3480355237 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b0e81640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac0b6140 0x7f91ac147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91abfff640 1 -- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac0b6140 msgr2=0x7f91ac147310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91abfff640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac0b6140 0x7f91ac147310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91abfff640 1 -- 172.21.15.26:0/3480355237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac15cc70 msgr2=0x7f91ac1f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91abfff640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac15cc70 0x7f91ac1f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91abfff640 1 -- 172.21.15.26:0/3480355237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a0009420 con 0x7f91ac0b88b0 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b1682640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac15cc70 0x7f91ac1f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:59:50.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91abfff640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0b88b0 0x7f91ac158a20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f91980af000 tx=0x7f919801ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:50.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91a97fa640 1 -- 172.21.15.26:0/3480355237 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91980bf070 con 0x7f91ac0b88b0 2023-12-19T16:59:50.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91a97fa640 1 -- 172.21.15.26:0/3480355237 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91980b1bb0 con 0x7f91ac0b88b0 2023-12-19T16:59:50.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91a97fa640 1 -- 172.21.15.26:0/3480355237 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91980b9410 con 0x7f91ac0b88b0 2023-12-19T16:59:50.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b30e5640 1 -- 172.21.15.26:0/3480355237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9198009420 con 0x7f91ac0b88b0 2023-12-19T16:59:50.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.067+0000 7f91b30e5640 1 -- 172.21.15.26:0/3480355237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ac15f8e0 con 0x7f91ac0b88b0 2023-12-19T16:59:50.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.071+0000 7f91b30e5640 1 -- 172.21.15.26:0/3480355237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ac15fad0 con 0x7f91ac0b88b0 2023-12-19T16:59:50.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.071+0000 7f91a97fa640 1 -- 172.21.15.26:0/3480355237 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 56) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f91980b7040 con 0x7f91ac0b88b0 2023-12-19T16:59:50.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.071+0000 7f91a97fa640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9190065af0 0x7f9190067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:50.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.071+0000 7f91a97fa640 1 -- 172.21.15.26:0/3480355237 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7244+0+0 (secure 0 0 0) 0x7f9198120850 con 0x7f91ac0b88b0 2023-12-19T16:59:50.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.071+0000 7f91b0e81640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9190065af0 0x7f9190067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:50.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.071+0000 7f91b0e81640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9190065af0 0x7f9190067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009cf0 tx=0x7f91a0009190 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:50.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.075+0000 7f91a97fa640 1 -- 172.21.15.26:0/3480355237 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f91980edde0 con 0x7f91ac0b88b0 2023-12-19T16:59:50.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.271+0000 7f91b30e5640 1 -- 172.21.15.26:0/3480355237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f91ac15f180 con 0x7f91ac0b88b0 2023-12-19T16:59:50.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.887+0000 7f91a97fa640 1 -- 172.21.15.26:0/3480355237 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f91ac15f180 con 0x7f91ac0b88b0 2023-12-19T16:59:50.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.887+0000 7f91b30e5640 1 -- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9190065af0 msgr2=0x7f9190067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:50.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.887+0000 7f91b30e5640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9190065af0 0x7f9190067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009cf0 tx=0x7f91a0009190 comp rx=0 tx=0).stop 2023-12-19T16:59:50.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.887+0000 7f91b30e5640 1 -- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0b88b0 msgr2=0x7f91ac158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:50.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.887+0000 7f91b30e5640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0b88b0 0x7f91ac158a20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f91980af000 tx=0x7f919801ba10 comp rx=0 tx=0).stop 2023-12-19T16:59:50.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.891+0000 7f91b30e5640 1 -- 172.21.15.26:0/3480355237 shutdown_connections 2023-12-19T16:59:50.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.891+0000 7f91b30e5640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac15cc70 0x7f91ac1f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:50.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.891+0000 7f91b30e5640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0b88b0 0x7f91ac158a20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:50.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.891+0000 7f91b30e5640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9190065af0 0x7f9190067fb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:50.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.891+0000 7f91b30e5640 1 --2- 172.21.15.26:0/3480355237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac0b6140 0x7f91ac147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:50.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.891+0000 7f91b30e5640 1 -- 172.21.15.26:0/3480355237 >> 172.21.15.26:0/3480355237 conn(0x7f91ac06a120 msgr2=0x7f91ac0d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:50.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.891+0000 7f91b30e5640 1 -- 172.21.15.26:0/3480355237 shutdown_connections 2023-12-19T16:59:50.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:50.891+0000 7f91b30e5640 1 -- 172.21.15.26:0/3480355237 wait complete. 2023-12-19T16:59:50.914 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T16:59:51.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9957a8c640 1 Processor -- start 2023-12-19T16:59:51.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9957a8c640 1 -- start start 2023-12-19T16:59:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9957a8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9950159830 0x7f995015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9957a8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99500c9310 0x7f99500cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9957a8c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f99500cb110 0x7f99500cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9957a8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995006f7c0 con 0x7f9950159830 2023-12-19T16:59:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9957a8c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99500ca2b0 con 0x7f99500cb110 2023-12-19T16:59:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9957a8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995015e3d0 con 0x7f99500c9310 2023-12-19T16:59:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9955828640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9950159830 0x7f995015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9955828640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9950159830 0x7f995015d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47976/0 (socket says 172.21.15.26:47976) 2023-12-19T16:59:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9955828640 1 -- 172.21.15.26:0/1508871963 learned_addr learned my addr 172.21.15.26:0/1508871963 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:51.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9956029640 1 --2- 172.21.15.26:0/1508871963 >> v2:172.21.15.160:3300/0 conn(0x7f99500cb110 0x7f99500cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:51.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9955027640 1 --2- 172.21.15.26:0/1508871963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99500c9310 0x7f99500cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:51.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9955828640 1 -- 172.21.15.26:0/1508871963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99500c9310 msgr2=0x7f99500cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:51.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9955828640 1 --2- 172.21.15.26:0/1508871963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99500c9310 0x7f99500cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:51.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9955828640 1 -- 172.21.15.26:0/1508871963 >> v2:172.21.15.160:3300/0 conn(0x7f99500cb110 msgr2=0x7f99500cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:51.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9955828640 1 --2- 172.21.15.26:0/1508871963 >> v2:172.21.15.160:3300/0 conn(0x7f99500cb110 0x7f99500cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:51.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9955828640 1 -- 172.21.15.26:0/1508871963 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99500cba80 con 0x7f9950159830 2023-12-19T16:59:51.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.103+0000 7f9956029640 1 --2- 172.21.15.26:0/1508871963 >> v2:172.21.15.160:3300/0 conn(0x7f99500cb110 0x7f99500cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:59:51.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.107+0000 7f9955828640 1 --2- 172.21.15.26:0/1508871963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9950159830 0x7f995015d4e0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f99500c21d0 tx=0x7f994c01e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3845e22786ee52f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:51.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.107+0000 7f9954826640 1 -- 172.21.15.26:0/1508871963 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c0ac670 con 0x7f9950159830 2023-12-19T16:59:51.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.107+0000 7f9954826640 1 -- 172.21.15.26:0/1508871963 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994c0ac7d0 con 0x7f9950159830 2023-12-19T16:59:51.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.107+0000 7f9954826640 1 -- 172.21.15.26:0/1508871963 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c0acb60 con 0x7f9950159830 2023-12-19T16:59:51.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.107+0000 7f9957a8c640 1 -- 172.21.15.26:0/1508871963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9950159830 msgr2=0x7f995015d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:51.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.107+0000 7f9957a8c640 1 --2- 172.21.15.26:0/1508871963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9950159830 0x7f995015d4e0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f99500c21d0 tx=0x7f994c01e650 comp rx=0 tx=0).stop 2023-12-19T16:59:51.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.107+0000 7f9957a8c640 1 -- 172.21.15.26:0/1508871963 shutdown_connections 2023-12-19T16:59:51.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.107+0000 7f9957a8c640 1 --2- 172.21.15.26:0/1508871963 >> v2:172.21.15.160:3300/0 conn(0x7f99500cb110 0x7f99500cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:51.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.107+0000 7f9957a8c640 1 --2- 172.21.15.26:0/1508871963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99500c9310 0x7f99500cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:51.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.107+0000 7f9957a8c640 1 --2- 172.21.15.26:0/1508871963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9950159830 0x7f995015d4e0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:51.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.107+0000 7f9957a8c640 1 -- 172.21.15.26:0/1508871963 >> 172.21.15.26:0/1508871963 conn(0x7f99500b0270 msgr2=0x7f99500b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:51.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.107+0000 7f9957a8c640 1 -- 172.21.15.26:0/1508871963 shutdown_connections 2023-12-19T16:59:51.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.107+0000 7f9957a8c640 1 -- 172.21.15.26:0/1508871963 wait complete. 2023-12-19T16:59:51.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9957a8c640 1 Processor -- start 2023-12-19T16:59:51.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9957a8c640 1 -- start start 2023-12-19T16:59:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9957a8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99500c9310 0x7f99500b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9957a8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99500cb110 0x7f99501f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9957a8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9950159830 0x7f995015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9957a8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99500b43a0 con 0x7f9950159830 2023-12-19T16:59:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9957a8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99501f53a0 con 0x7f99500cb110 2023-12-19T16:59:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9957a8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9950159fd0 con 0x7f99500c9310 2023-12-19T16:59:51.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9956029640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9950159830 0x7f995015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9956029640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9950159830 0x7f995015b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47986/0 (socket says 172.21.15.26:47986) 2023-12-19T16:59:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9956029640 1 -- 172.21.15.26:0/424372905 learned_addr learned my addr 172.21.15.26:0/424372905 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9956029640 1 -- 172.21.15.26:0/424372905 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99500c9310 msgr2=0x7f99500b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9956029640 1 --2- 172.21.15.26:0/424372905 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99500c9310 0x7f99500b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9956029640 1 -- 172.21.15.26:0/424372905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99500cb110 msgr2=0x7f99501f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9956029640 1 --2- 172.21.15.26:0/424372905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99500cb110 0x7f99501f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9956029640 1 -- 172.21.15.26:0/424372905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c0094d0 con 0x7f9950159830 2023-12-19T16:59:51.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9956029640 1 --2- 172.21.15.26:0/424372905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9950159830 0x7f995015b5e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9948009fa0 tx=0x7f994800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f99467fc640 1 -- 172.21.15.26:0/424372905 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948003ba0 con 0x7f9950159830 2023-12-19T16:59:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f99467fc640 1 -- 172.21.15.26:0/424372905 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994800f040 con 0x7f9950159830 2023-12-19T16:59:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f99467fc640 1 -- 172.21.15.26:0/424372905 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948004030 con 0x7f9950159830 2023-12-19T16:59:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9957a8c640 1 -- 172.21.15.26:0/424372905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995006fa80 con 0x7f9950159830 2023-12-19T16:59:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.111+0000 7f9957a8c640 1 -- 172.21.15.26:0/424372905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995015c7d0 con 0x7f9950159830 2023-12-19T16:59:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.115+0000 7f9957a8c640 1 -- 172.21.15.26:0/424372905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9950074270 con 0x7f9950159830 2023-12-19T16:59:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.115+0000 7f99467fc640 1 -- 172.21.15.26:0/424372905 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 56) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f9948004190 con 0x7f9950159830 2023-12-19T16:59:51.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.115+0000 7f99467fc640 1 --2- 172.21.15.26:0/424372905 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9934065af0 0x7f9934067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.115+0000 7f99467fc640 1 -- 172.21.15.26:0/424372905 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7244+0+0 (secure 0 0 0) 0x7f9948083d20 con 0x7f9950159830 2023-12-19T16:59:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.115+0000 7f9955828640 1 --2- 172.21.15.26:0/424372905 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9934065af0 0x7f9934067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:51.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.115+0000 7f9955828640 1 --2- 172.21.15.26:0/424372905 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9934065af0 0x7f9934067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f994c0b0000 tx=0x7f994c008040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.119+0000 7f99467fc640 1 -- 172.21.15.26:0/424372905 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9948050ee0 con 0x7f9950159830 2023-12-19T16:59:51.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.275+0000 7f9957a8c640 1 -- 172.21.15.26:0/424372905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f99500b46c0 con 0x7f9950159830 2023-12-19T16:59:51.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.899+0000 7f99467fc640 1 -- 172.21.15.26:0/424372905 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f99480b50a0 con 0x7f9950159830 2023-12-19T16:59:51.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.899+0000 7f99467fc640 1 -- 172.21.15.26:0/424372905 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v113) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9948080020 con 0x7f9950159830 2023-12-19T16:59:51.901 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T16:59:51.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.903+0000 7f9957a8c640 1 -- 172.21.15.26:0/424372905 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9934065af0 msgr2=0x7f9934067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:51.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.903+0000 7f9957a8c640 1 --2- 172.21.15.26:0/424372905 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9934065af0 0x7f9934067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f994c0b0000 tx=0x7f994c008040 comp rx=0 tx=0).stop 2023-12-19T16:59:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.903+0000 7f9957a8c640 1 -- 172.21.15.26:0/424372905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9950159830 msgr2=0x7f995015b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.903+0000 7f9957a8c640 1 --2- 172.21.15.26:0/424372905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9950159830 0x7f995015b5e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f9948009fa0 tx=0x7f994800d560 comp rx=0 tx=0).stop 2023-12-19T16:59:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.903+0000 7f9957a8c640 1 -- 172.21.15.26:0/424372905 shutdown_connections 2023-12-19T16:59:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.903+0000 7f9957a8c640 1 --2- 172.21.15.26:0/424372905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9950159830 0x7f995015b5e0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.903+0000 7f9957a8c640 1 --2- 172.21.15.26:0/424372905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99500cb110 0x7f99501f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:51.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.903+0000 7f9957a8c640 1 --2- 172.21.15.26:0/424372905 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9934065af0 0x7f9934067fb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:51.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.903+0000 7f9957a8c640 1 --2- 172.21.15.26:0/424372905 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99500c9310 0x7f99500b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:51.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.903+0000 7f9957a8c640 1 -- 172.21.15.26:0/424372905 >> 172.21.15.26:0/424372905 conn(0x7f99500b0270 msgr2=0x7f99500afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:51.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.903+0000 7f9957a8c640 1 -- 172.21.15.26:0/424372905 shutdown_connections 2023-12-19T16:59:51.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:51.903+0000 7f9957a8c640 1 -- 172.21.15.26:0/424372905 wait complete. 2023-12-19T16:59:51.931 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T16:59:52.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.079+0000 7fe09dc8c640 1 Processor -- start 2023-12-19T16:59:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.079+0000 7fe09dc8c640 1 -- start start 2023-12-19T16:59:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.079+0000 7fe09dc8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0980c9310 0x7fe0980cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.079+0000 7fe09dc8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe0980cb110 0x7fe0980cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:52.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.079+0000 7fe09dc8c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe09815e3d0 con 0x7fe098159830 2023-12-19T16:59:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.079+0000 7fe09dc8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09806f7c0 con 0x7fe0980cb110 2023-12-19T16:59:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.079+0000 7fe09dc8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980ca2b0 con 0x7fe0980c9310 2023-12-19T16:59:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.079+0000 7fe097fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe0980cb110 0x7fe0980cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.079+0000 7fe097fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe0980cb110 0x7fe0980cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48002/0 (socket says 172.21.15.26:48002) 2023-12-19T16:59:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.079+0000 7fe097fff640 1 -- 172.21.15.26:0/1272269841 learned_addr learned my addr 172.21.15.26:0/1272269841 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.079+0000 7fe096ffd640 1 --2- 172.21.15.26:0/1272269841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0980c9310 0x7fe0980cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.079+0000 7fe097fff640 1 -- 172.21.15.26:0/1272269841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0980c9310 msgr2=0x7fe0980cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.079+0000 7fe097fff640 1 --2- 172.21.15.26:0/1272269841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0980c9310 0x7fe0980cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe097fff640 1 -- 172.21.15.26:0/1272269841 >> v1:172.21.15.160:6789/0 conn(0x7fe098159830 legacy=0x7fe09815d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:52.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe097fff640 1 -- 172.21.15.26:0/1272269841 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0980cba80 con 0x7fe0980cb110 2023-12-19T16:59:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe097fff640 1 --2- 172.21.15.26:0/1272269841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0980cb110 0x7fe0980cb540 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe09000d150 tx=0x7fe09001eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62383bcdaadd1ee5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe0967fc640 1 -- 172.21.15.26:0/1272269841 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09000b2d0 con 0x7fe0980cb110 2023-12-19T16:59:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe0967fc640 1 -- 172.21.15.26:0/1272269841 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09000b430 con 0x7fe0980cb110 2023-12-19T16:59:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe0967fc640 1 -- 172.21.15.26:0/1272269841 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0900a7b60 con 0x7fe0980cb110 2023-12-19T16:59:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe09dc8c640 1 -- 172.21.15.26:0/1272269841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0980cb110 msgr2=0x7fe0980cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe09dc8c640 1 --2- 172.21.15.26:0/1272269841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0980cb110 0x7fe0980cb540 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fe09000d150 tx=0x7fe09001eb80 comp rx=0 tx=0).stop 2023-12-19T16:59:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe09dc8c640 1 -- 172.21.15.26:0/1272269841 shutdown_connections 2023-12-19T16:59:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe09dc8c640 1 --2- 172.21.15.26:0/1272269841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0980cb110 0x7fe0980cb540 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe09dc8c640 1 --2- 172.21.15.26:0/1272269841 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0980c9310 0x7fe0980cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe09dc8c640 1 -- 172.21.15.26:0/1272269841 >> 172.21.15.26:0/1272269841 conn(0x7fe0980b0270 msgr2=0x7fe0980b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe09dc8c640 1 -- 172.21.15.26:0/1272269841 shutdown_connections 2023-12-19T16:59:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.083+0000 7fe09dc8c640 1 -- 172.21.15.26:0/1272269841 wait complete. 2023-12-19T16:59:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe09dc8c640 1 Processor -- start 2023-12-19T16:59:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe09dc8c640 1 -- start start 2023-12-19T16:59:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe09dc8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0980c9310 0x7fe09815c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe09dc8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0980cb110 0x7fe0980c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe09dc8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe098159830 0x7fe0980c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe09dc8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980b1360 con 0x7fe098159830 2023-12-19T16:59:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe09dc8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980b15b0 con 0x7fe0980c9310 2023-12-19T16:59:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe09dc8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0980b1890 con 0x7fe0980cb110 2023-12-19T16:59:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe097fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe098159830 0x7fe0980c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe097fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe098159830 0x7fe0980c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48018/0 (socket says 172.21.15.26:48018) 2023-12-19T16:59:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe097fff640 1 -- 172.21.15.26:0/2946827264 learned_addr learned my addr 172.21.15.26:0/2946827264 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe096ffd640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0980cb110 0x7fe0980c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe0977fe640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0980c9310 0x7fe09815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe097fff640 1 -- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0980cb110 msgr2=0x7fe0980c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe097fff640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0980cb110 0x7fe0980c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe097fff640 1 -- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0980c9310 msgr2=0x7fe09815c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe097fff640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0980c9310 0x7fe09815c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe097fff640 1 -- 172.21.15.26:0/2946827264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09000bd40 con 0x7fe098159830 2023-12-19T16:59:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe097fff640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe098159830 0x7fe0980c5d50 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fe09001c7b0 tx=0x7fe09001b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe07ffff640 1 -- 172.21.15.26:0/2946827264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090009800 con 0x7fe098159830 2023-12-19T16:59:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe07ffff640 1 -- 172.21.15.26:0/2946827264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe090009980 con 0x7fe098159830 2023-12-19T16:59:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe09dc8c640 1 -- 172.21.15.26:0/2946827264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0980b1b70 con 0x7fe098159830 2023-12-19T16:59:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe07ffff640 1 -- 172.21.15.26:0/2946827264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe090009d30 con 0x7fe098159830 2023-12-19T16:59:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.087+0000 7fe09dc8c640 1 -- 172.21.15.26:0/2946827264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0980b2130 con 0x7fe098159830 2023-12-19T16:59:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.091+0000 7fe09dc8c640 1 -- 172.21.15.26:0/2946827264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0980b2320 con 0x7fe098159830 2023-12-19T16:59:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.091+0000 7fe07ffff640 1 -- 172.21.15.26:0/2946827264 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe0900b9430 con 0x7fe098159830 2023-12-19T16:59:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.091+0000 7fe07ffff640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe084065bb0 0x7fe084068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.091+0000 7fe07ffff640 1 -- 172.21.15.26:0/2946827264 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6806+0+0 (secure 0 0 0) 0x7fe09011e860 con 0x7fe098159830 2023-12-19T16:59:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.091+0000 7fe0977fe640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe084065bb0 0x7fe084068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.095+0000 7fe0977fe640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe084065bb0 0x7fe084068070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe088000f70 tx=0x7fe088009040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:52.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.095+0000 7fe07ffff640 1 -- 172.21.15.26:0/2946827264 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe0900ebd40 con 0x7fe098159830 2023-12-19T16:59:52.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.255+0000 7fe09dc8c640 1 -- 172.21.15.26:0/2946827264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe098074270 con 0x7fe098159830 2023-12-19T16:59:52.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.899+0000 7fe07ffff640 1 -- 172.21.15.26:0/2946827264 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v114) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe0900ef360 con 0x7fe098159830 2023-12-19T16:59:52.900 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T16:59:52.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.899+0000 7fe09dc8c640 1 -- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe084065bb0 msgr2=0x7fe084068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:52.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.899+0000 7fe09dc8c640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe084065bb0 0x7fe084068070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe088000f70 tx=0x7fe088009040 comp rx=0 tx=0).stop 2023-12-19T16:59:52.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.899+0000 7fe09dc8c640 1 -- 172.21.15.26:0/2946827264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe098159830 msgr2=0x7fe0980c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:52.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.899+0000 7fe09dc8c640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe098159830 0x7fe0980c5d50 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fe09001c7b0 tx=0x7fe09001b920 comp rx=0 tx=0).stop 2023-12-19T16:59:52.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.903+0000 7fe09dc8c640 1 -- 172.21.15.26:0/2946827264 shutdown_connections 2023-12-19T16:59:52.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.903+0000 7fe09dc8c640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe098159830 0x7fe0980c5d50 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:52.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.903+0000 7fe09dc8c640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe084065bb0 0x7fe084068070 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:52.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.903+0000 7fe09dc8c640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0980cb110 0x7fe0980c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:52.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.903+0000 7fe09dc8c640 1 --2- 172.21.15.26:0/2946827264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0980c9310 0x7fe09815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:52.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.903+0000 7fe09dc8c640 1 -- 172.21.15.26:0/2946827264 >> 172.21.15.26:0/2946827264 conn(0x7fe0980b0270 msgr2=0x7fe0980af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:52.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.903+0000 7fe09dc8c640 1 -- 172.21.15.26:0/2946827264 shutdown_connections 2023-12-19T16:59:52.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:52.903+0000 7fe09dc8c640 1 -- 172.21.15.26:0/2946827264 wait complete. 2023-12-19T16:59:52.929 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-19T16:59:53.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.075+0000 7f47207e9640 1 Processor -- start 2023-12-19T16:59:53.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.075+0000 7f47207e9640 1 -- start start 2023-12-19T16:59:53.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f47207e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47180cb860 0x7f471806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f47207e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f471815d530 0x7f471815d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f47207e9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471815e240 con 0x7f47181599e0 2023-12-19T16:59:53.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f47207e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47180c9520 con 0x7f471815d530 2023-12-19T16:59:53.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f47207e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47180c9800 con 0x7f47180cb860 2023-12-19T16:59:53.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f471ed86640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f471815d530 0x7f471815d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f471ed86640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f471815d530 0x7f471815d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48028/0 (socket says 172.21.15.26:48028) 2023-12-19T16:59:53.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f471ed86640 1 -- 172.21.15.26:0/3310385428 learned_addr learned my addr 172.21.15.26:0/3310385428 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:53.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f471dd84640 1 --2- 172.21.15.26:0/3310385428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47180cb860 0x7f471806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f471ed86640 1 -- 172.21.15.26:0/3310385428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47180cb860 msgr2=0x7f471806f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:53.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f471ed86640 1 --2- 172.21.15.26:0/3310385428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47180cb860 0x7f471806f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f471ed86640 1 -- 172.21.15.26:0/3310385428 >> v1:172.21.15.160:6789/0 conn(0x7f47181599e0 legacy=0x7f47180cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f471ed86640 1 -- 172.21.15.26:0/3310385428 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47180c9ae0 con 0x7f471815d530 2023-12-19T16:59:53.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.079+0000 7f471ed86640 1 --2- 172.21.15.26:0/3310385428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f471815d530 0x7f471815d980 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f4710009e50 tx=0x7f471001ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5284378b82a5cc10 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:53.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f471d583640 1 -- 172.21.15.26:0/3310385428 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100a7b10 con 0x7f471815d530 2023-12-19T16:59:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f471d583640 1 -- 172.21.15.26:0/3310385428 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47100a7c70 con 0x7f471815d530 2023-12-19T16:59:53.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f471d583640 1 -- 172.21.15.26:0/3310385428 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100af3f0 con 0x7f471815d530 2023-12-19T16:59:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 -- 172.21.15.26:0/3310385428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f471815d530 msgr2=0x7f471815d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 --2- 172.21.15.26:0/3310385428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f471815d530 0x7f471815d980 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f4710009e50 tx=0x7f471001ec20 comp rx=0 tx=0).stop 2023-12-19T16:59:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 -- 172.21.15.26:0/3310385428 shutdown_connections 2023-12-19T16:59:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 --2- 172.21.15.26:0/3310385428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f471815d530 0x7f471815d980 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 --2- 172.21.15.26:0/3310385428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47180cb860 0x7f471806f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 -- 172.21.15.26:0/3310385428 >> 172.21.15.26:0/3310385428 conn(0x7f4718066160 msgr2=0x7f47180b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 -- 172.21.15.26:0/3310385428 shutdown_connections 2023-12-19T16:59:53.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 -- 172.21.15.26:0/3310385428 wait complete. 2023-12-19T16:59:53.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 Processor -- start 2023-12-19T16:59:53.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 -- start start 2023-12-19T16:59:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47180cb860 0x7f4718150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47181599e0 0x7f47180c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f471815d530 0x7f47180ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47180c6a30 con 0x7f47180cb860 2023-12-19T16:59:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47180c6190 con 0x7f471815d530 2023-12-19T16:59:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.083+0000 7f47207e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47180c6470 con 0x7f47181599e0 2023-12-19T16:59:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f471e585640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47180cb860 0x7f4718150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f471dd84640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47181599e0 0x7f47180c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f471e585640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47180cb860 0x7f4718150cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48036/0 (socket says 172.21.15.26:48036) 2023-12-19T16:59:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f471e585640 1 -- 172.21.15.26:0/3011786387 learned_addr learned my addr 172.21.15.26:0/3011786387 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f471ed86640 1 --2- 172.21.15.26:0/3011786387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f471815d530 0x7f47180ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f471e585640 1 -- 172.21.15.26:0/3011786387 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47181599e0 msgr2=0x7f47180c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f471e585640 1 --2- 172.21.15.26:0/3011786387 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47181599e0 0x7f47180c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f471e585640 1 -- 172.21.15.26:0/3011786387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f471815d530 msgr2=0x7f47180ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f471e585640 1 --2- 172.21.15.26:0/3011786387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f471815d530 0x7f47180ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f471e585640 1 -- 172.21.15.26:0/3011786387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4710009b30 con 0x7f47180cb860 2023-12-19T16:59:53.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f471e585640 1 --2- 172.21.15.26:0/3011786387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47180cb860 0x7f4718150cd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f471400c500 tx=0x7f471400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:53.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f470effd640 1 -- 172.21.15.26:0/3011786387 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714005220 con 0x7f47180cb860 2023-12-19T16:59:53.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f470effd640 1 -- 172.21.15.26:0/3011786387 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4714005380 con 0x7f47180cb860 2023-12-19T16:59:53.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f470effd640 1 -- 172.21.15.26:0/3011786387 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714007270 con 0x7f47180cb860 2023-12-19T16:59:53.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f47207e9640 1 -- 172.21.15.26:0/3011786387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47180c6750 con 0x7f47180cb860 2023-12-19T16:59:53.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f47207e9640 1 -- 172.21.15.26:0/3011786387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4718151210 con 0x7f47180cb860 2023-12-19T16:59:53.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f47207e9640 1 -- 172.21.15.26:0/3011786387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471815aa40 con 0x7f47180cb860 2023-12-19T16:59:53.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f470effd640 1 -- 172.21.15.26:0/3011786387 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f47140073d0 con 0x7f47180cb860 2023-12-19T16:59:53.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.087+0000 7f470effd640 1 --2- 172.21.15.26:0/3011786387 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f46fc065b80 0x7f46fc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.091+0000 7f470effd640 1 -- 172.21.15.26:0/3011786387 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f4714082e60 con 0x7f47180cb860 2023-12-19T16:59:53.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.091+0000 7f471dd84640 1 --2- 172.21.15.26:0/3011786387 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f46fc065b80 0x7f46fc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.091+0000 7f471dd84640 1 --2- 172.21.15.26:0/3011786387 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f46fc065b80 0x7f46fc068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f47180b0440 tx=0x7f4708008040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:53.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.095+0000 7f470effd640 1 -- 172.21.15.26:0/3011786387 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f471815aa40 con 0x7f47180cb860 2023-12-19T16:59:53.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.251+0000 7f47207e9640 1 -- 172.21.15.26:0/3011786387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f471815aa40 con 0x7f47180cb860 2023-12-19T16:59:53.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.255+0000 7f470effd640 1 -- 172.21.15.26:0/3011786387 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v37) v1 ==== 120+0+0 (secure 0 0 0) 0x7f471815aa40 con 0x7f47180cb860 2023-12-19T16:59:53.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.259+0000 7f47207e9640 1 -- 172.21.15.26:0/3011786387 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f46fc065b80 msgr2=0x7f46fc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.259+0000 7f47207e9640 1 --2- 172.21.15.26:0/3011786387 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f46fc065b80 0x7f46fc068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f47180b0440 tx=0x7f4708008040 comp rx=0 tx=0).stop 2023-12-19T16:59:53.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.259+0000 7f47207e9640 1 -- 172.21.15.26:0/3011786387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47180cb860 msgr2=0x7f4718150cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.259+0000 7f47207e9640 1 --2- 172.21.15.26:0/3011786387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47180cb860 0x7f4718150cd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f471400c500 tx=0x7f471400c9c0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.259+0000 7f47207e9640 1 -- 172.21.15.26:0/3011786387 shutdown_connections 2023-12-19T16:59:53.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.259+0000 7f47207e9640 1 --2- 172.21.15.26:0/3011786387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f471815d530 0x7f47180ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.259+0000 7f47207e9640 1 --2- 172.21.15.26:0/3011786387 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f46fc065b80 0x7f46fc068040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.259+0000 7f47207e9640 1 --2- 172.21.15.26:0/3011786387 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f47181599e0 0x7f47180c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.259+0000 7f47207e9640 1 --2- 172.21.15.26:0/3011786387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47180cb860 0x7f4718150cd0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.259+0000 7f47207e9640 1 -- 172.21.15.26:0/3011786387 >> 172.21.15.26:0/3011786387 conn(0x7f4718066160 msgr2=0x7f471806cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:53.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.259+0000 7f47207e9640 1 -- 172.21.15.26:0/3011786387 shutdown_connections 2023-12-19T16:59:53.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.259+0000 7f47207e9640 1 -- 172.21.15.26:0/3011786387 wait complete. 2023-12-19T16:59:53.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T16:59:53.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.423+0000 7f417abf3640 1 Processor -- start 2023-12-19T16:59:53.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.423+0000 7f417abf3640 1 -- start start 2023-12-19T16:59:53.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.423+0000 7f417abf3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41741585f0 0x7f41740cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.427+0000 7f417abf3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41740cb5d0 0x7f41740cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.427+0000 7f417abf3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417415b1b0 con 0x7f41740cb150 2023-12-19T16:59:53.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.427+0000 7f417abf3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41740b47a0 con 0x7f41740cb5d0 2023-12-19T16:59:53.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.427+0000 7f417abf3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417415b490 con 0x7f41741585f0 2023-12-19T16:59:53.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.427+0000 7f4179190640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f41740cb5d0 0x7f41740cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.427+0000 7f4173fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f41740cb150 0x7f41740cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43770/0 (socket says 172.21.15.26:43770) 2023-12-19T16:59:53.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.427+0000 7f4173fff640 1 -- 172.21.15.26:0/187731203 learned_addr learned my addr 172.21.15.26:0/187731203 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:53.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.427+0000 7f417898f640 1 --2- 172.21.15.26:0/187731203 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41741585f0 0x7f41740cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.427+0000 7f4179190640 1 -- 172.21.15.26:0/187731203 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41741585f0 msgr2=0x7f41740cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:53.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.427+0000 7f4179190640 1 --2- 172.21.15.26:0/187731203 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41741585f0 0x7f41740cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.427+0000 7f4179190640 1 -- 172.21.15.26:0/187731203 >> v1:172.21.15.26:6789/0 conn(0x7f41740cb150 legacy=0x7f41740cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.427+0000 7f41737fe640 1 -- 172.21.15.26:0/187731203 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910456922 0 0) 0x7f417415b1b0 con 0x7f41740cb150 2023-12-19T16:59:53.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.427+0000 7f4179190640 1 -- 172.21.15.26:0/187731203 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417415b770 con 0x7f41740cb5d0 2023-12-19T16:59:53.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.431+0000 7f4179190640 1 --2- 172.21.15.26:0/187731203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41740cb5d0 0x7f41740cca20 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f416c009d90 tx=0x7f416c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d20eac823ada5f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:53.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.431+0000 7f41737fe640 1 -- 172.21.15.26:0/187731203 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c0a5970 con 0x7f41740cb5d0 2023-12-19T16:59:53.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.431+0000 7f41737fe640 1 -- 172.21.15.26:0/187731203 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c0a5ad0 con 0x7f41740cb5d0 2023-12-19T16:59:53.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.431+0000 7f41737fe640 1 -- 172.21.15.26:0/187731203 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c0a5e60 con 0x7f41740cb5d0 2023-12-19T16:59:53.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.431+0000 7f417abf3640 1 -- 172.21.15.26:0/187731203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41740cb5d0 msgr2=0x7f41740cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.431+0000 7f417abf3640 1 --2- 172.21.15.26:0/187731203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41740cb5d0 0x7f41740cca20 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f416c009d90 tx=0x7f416c01c890 comp rx=0 tx=0).stop 2023-12-19T16:59:53.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.431+0000 7f417abf3640 1 -- 172.21.15.26:0/187731203 shutdown_connections 2023-12-19T16:59:53.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.431+0000 7f417abf3640 1 --2- 172.21.15.26:0/187731203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41740cb5d0 0x7f41740cca20 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.431+0000 7f417abf3640 1 --2- 172.21.15.26:0/187731203 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41741585f0 0x7f41740cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.431+0000 7f417abf3640 1 -- 172.21.15.26:0/187731203 >> 172.21.15.26:0/187731203 conn(0x7f4174066160 msgr2=0x7f41740b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:53.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.431+0000 7f417abf3640 1 -- 172.21.15.26:0/187731203 shutdown_connections 2023-12-19T16:59:53.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.431+0000 7f417abf3640 1 -- 172.21.15.26:0/187731203 wait complete. 2023-12-19T16:59:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.435+0000 7f417abf3640 1 Processor -- start 2023-12-19T16:59:53.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.435+0000 7f417abf3640 1 -- start start 2023-12-19T16:59:53.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f417abf3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41740cb150 0x7f417414d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f417abf3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb5d0 0x7f41740ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f417abf3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41741585f0 0x7f41740acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f417abf3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41741f55b0 con 0x7f41741585f0 2023-12-19T16:59:53.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f417abf3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417414d9e0 con 0x7f41740cb150 2023-12-19T16:59:53.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f417abf3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4174153c20 con 0x7f41740cb5d0 2023-12-19T16:59:53.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f417898f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41740cb150 0x7f417414d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f417898f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41740cb150 0x7f417414d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:40256/0 (socket says 172.21.15.26:40256) 2023-12-19T16:59:53.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f417898f640 1 -- 172.21.15.26:0/657145042 learned_addr learned my addr 172.21.15.26:0/657145042 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:53.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f4179190640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41741585f0 0x7f41740acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f4173fff640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb5d0 0x7f41740ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f4179190640 1 -- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb5d0 msgr2=0x7f41740ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f4179190640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb5d0 0x7f41740ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f4179190640 1 -- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41740cb150 msgr2=0x7f417414d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f4179190640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41740cb150 0x7f417414d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f4179190640 1 -- 172.21.15.26:0/657145042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c009a70 con 0x7f41741585f0 2023-12-19T16:59:53.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f4173fff640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb5d0 0x7f41740ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:59:53.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f417898f640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41740cb150 0x7f417414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:59:53.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f4179190640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41741585f0 0x7f41740acda0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f416c0ac000 tx=0x7f416c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:53.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f41717fa640 1 -- 172.21.15.26:0/657145042 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c0aedc0 con 0x7f41741585f0 2023-12-19T16:59:53.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f41717fa640 1 -- 172.21.15.26:0/657145042 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c009450 con 0x7f41741585f0 2023-12-19T16:59:53.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f417abf3640 1 -- 172.21.15.26:0/657145042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417415c3e0 con 0x7f41741585f0 2023-12-19T16:59:53.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f41717fa640 1 -- 172.21.15.26:0/657145042 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c003ba0 con 0x7f41741585f0 2023-12-19T16:59:53.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.439+0000 7f417abf3640 1 -- 172.21.15.26:0/657145042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41741f5a90 con 0x7f41741585f0 2023-12-19T16:59:53.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.443+0000 7f417abf3640 1 -- 172.21.15.26:0/657145042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4174074270 con 0x7f41741585f0 2023-12-19T16:59:53.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.443+0000 7f41717fa640 1 -- 172.21.15.26:0/657145042 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f416c0b4050 con 0x7f41741585f0 2023-12-19T16:59:53.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.443+0000 7f41717fa640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4158065bb0 0x7f4158068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.443+0000 7f41717fa640 1 -- 172.21.15.26:0/657145042 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f416c11e620 con 0x7f41741585f0 2023-12-19T16:59:53.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.443+0000 7f417898f640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4158065bb0 0x7f4158068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.447+0000 7f417898f640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4158065bb0 0x7f4158068070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f417414e4a0 tx=0x7f4168009190 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:53.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.451+0000 7f41717fa640 1 -- 172.21.15.26:0/657145042 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f416c0ebc50 con 0x7f41741585f0 2023-12-19T16:59:53.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.607+0000 7f417abf3640 1 -- 172.21.15.26:0/657145042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f41740c2050 con 0x7f41741585f0 2023-12-19T16:59:53.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.611+0000 7f41717fa640 1 -- 172.21.15.26:0/657145042 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7f416c0ef270 con 0x7f41741585f0 2023-12-19T16:59:53.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.615+0000 7f417abf3640 1 -- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4158065bb0 msgr2=0x7f4158068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.615+0000 7f417abf3640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4158065bb0 0x7f4158068070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f417414e4a0 tx=0x7f4168009190 comp rx=0 tx=0).stop 2023-12-19T16:59:53.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.615+0000 7f417abf3640 1 -- 172.21.15.26:0/657145042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41741585f0 msgr2=0x7f41740acda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.615+0000 7f417abf3640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41741585f0 0x7f41740acda0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f416c0ac000 tx=0x7f416c019630 comp rx=0 tx=0).stop 2023-12-19T16:59:53.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.615+0000 7f417abf3640 1 -- 172.21.15.26:0/657145042 shutdown_connections 2023-12-19T16:59:53.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.615+0000 7f417abf3640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41741585f0 0x7f41740acda0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.615+0000 7f417abf3640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4158065bb0 0x7f4158068070 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.615+0000 7f417abf3640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41740cb5d0 0x7f41740ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.615+0000 7f417abf3640 1 --2- 172.21.15.26:0/657145042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41740cb150 0x7f417414d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.615+0000 7f417abf3640 1 -- 172.21.15.26:0/657145042 >> 172.21.15.26:0/657145042 conn(0x7f4174066160 msgr2=0x7f41740ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:53.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.615+0000 7f417abf3640 1 -- 172.21.15.26:0/657145042 shutdown_connections 2023-12-19T16:59:53.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.615+0000 7f417abf3640 1 -- 172.21.15.26:0/657145042 wait complete. 2023-12-19T16:59:53.637 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-19T16:59:53.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.779+0000 7fe91ab6d640 1 Processor -- start 2023-12-19T16:59:53.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.779+0000 7fe91ab6d640 1 -- start start 2023-12-19T16:59:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.783+0000 7fe91ab6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe9140cb860 0x7fe91406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.783+0000 7fe91ab6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe91415e240 con 0x7fe9141599e0 2023-12-19T16:59:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.783+0000 7fe91ab6d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9140c9520 con 0x7fe91415d530 2023-12-19T16:59:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.783+0000 7fe91ab6d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9140c9800 con 0x7fe9140cb860 2023-12-19T16:59:53.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.783+0000 7fe913fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe9140cb860 0x7fe91406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.783+0000 7fe913fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe9140cb860 0x7fe91406f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48068/0 (socket says 172.21.15.26:48068) 2023-12-19T16:59:53.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.783+0000 7fe913fff640 1 -- 172.21.15.26:0/3544720998 learned_addr learned my addr 172.21.15.26:0/3544720998 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:53.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.783+0000 7fe913fff640 1 -- 172.21.15.26:0/3544720998 >> v1:172.21.15.160:6789/0 conn(0x7fe91415d530 legacy=0x7fe91415d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:53.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.783+0000 7fe913fff640 1 -- 172.21.15.26:0/3544720998 >> v1:172.21.15.26:6789/0 conn(0x7fe9141599e0 legacy=0x7fe9140cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.783+0000 7fe913fff640 1 -- 172.21.15.26:0/3544720998 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9140c9ae0 con 0x7fe9140cb860 2023-12-19T16:59:53.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe913fff640 1 --2- 172.21.15.26:0/3544720998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9140cb860 0x7fe91406f6d0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe900009870 tx=0x7fe90001e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e252bf4dc911f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:53.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe9137fe640 1 -- 172.21.15.26:0/3544720998 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9000ab5f0 con 0x7fe9140cb860 2023-12-19T16:59:53.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe9137fe640 1 -- 172.21.15.26:0/3544720998 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9000ab750 con 0x7fe9140cb860 2023-12-19T16:59:53.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe9137fe640 1 -- 172.21.15.26:0/3544720998 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9000abae0 con 0x7fe9140cb860 2023-12-19T16:59:53.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 -- 172.21.15.26:0/3544720998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9140cb860 msgr2=0x7fe91406f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 --2- 172.21.15.26:0/3544720998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9140cb860 0x7fe91406f6d0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe900009870 tx=0x7fe90001e710 comp rx=0 tx=0).stop 2023-12-19T16:59:53.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 -- 172.21.15.26:0/3544720998 shutdown_connections 2023-12-19T16:59:53.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 --2- 172.21.15.26:0/3544720998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9140cb860 0x7fe91406f6d0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 -- 172.21.15.26:0/3544720998 >> 172.21.15.26:0/3544720998 conn(0x7fe914066160 msgr2=0x7fe9140b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:53.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 -- 172.21.15.26:0/3544720998 shutdown_connections 2023-12-19T16:59:53.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 -- 172.21.15.26:0/3544720998 wait complete. 2023-12-19T16:59:53.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 Processor -- start 2023-12-19T16:59:53.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 -- start start 2023-12-19T16:59:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9140cb860 0x7fe9140af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9141599e0 0x7fe91414f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe91415d530 0x7fe9140af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91414faa0 con 0x7fe9140cb860 2023-12-19T16:59:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91415a0d0 con 0x7fe91415d530 2023-12-19T16:59:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.787+0000 7fe91ab6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91415a370 con 0x7fe9141599e0 2023-12-19T16:59:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe918909640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9140cb860 0x7fe9140af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe918909640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9140cb860 0x7fe9140af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48074/0 (socket says 172.21.15.26:48074) 2023-12-19T16:59:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe918909640 1 -- 172.21.15.26:0/2190234012 learned_addr learned my addr 172.21.15.26:0/2190234012 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:53.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe913fff640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9141599e0 0x7fe91414f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe91910a640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe91415d530 0x7fe9140af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe918909640 1 -- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9141599e0 msgr2=0x7fe91414f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe918909640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9141599e0 0x7fe91414f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe918909640 1 -- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe91415d530 msgr2=0x7fe9140af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe918909640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe91415d530 0x7fe9140af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe918909640 1 -- 172.21.15.26:0/2190234012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9000094d0 con 0x7fe9140cb860 2023-12-19T16:59:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe918909640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9140cb860 0x7fe9140af0b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe90800c500 tx=0x7fe90800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe9117fa640 1 -- 172.21.15.26:0/2190234012 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe908005220 con 0x7fe9140cb860 2023-12-19T16:59:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe9117fa640 1 -- 172.21.15.26:0/2190234012 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe908005380 con 0x7fe9140cb860 2023-12-19T16:59:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe9117fa640 1 -- 172.21.15.26:0/2190234012 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe908007270 con 0x7fe9140cb860 2023-12-19T16:59:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe91ab6d640 1 -- 172.21.15.26:0/2190234012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91415a650 con 0x7fe9140cb860 2023-12-19T16:59:53.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe91ab6d640 1 -- 172.21.15.26:0/2190234012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9140b07c0 con 0x7fe9140cb860 2023-12-19T16:59:53.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe91ab6d640 1 -- 172.21.15.26:0/2190234012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe914074270 con 0x7fe9140cb860 2023-12-19T16:59:53.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.791+0000 7fe9117fa640 1 -- 172.21.15.26:0/2190234012 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe9080073d0 con 0x7fe9140cb860 2023-12-19T16:59:53.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.795+0000 7fe9117fa640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe8f8065b80 0x7fe8f8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:53.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.795+0000 7fe9117fa640 1 -- 172.21.15.26:0/2190234012 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6363+0+0 (secure 0 0 0) 0x7fe908082e60 con 0x7fe9140cb860 2023-12-19T16:59:53.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.795+0000 7fe913fff640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe8f8065b80 0x7fe8f8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:53.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.795+0000 7fe913fff640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe8f8065b80 0x7fe8f8068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe9000b0000 tx=0x7fe9000b5040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:53.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.799+0000 7fe9117fa640 1 -- 172.21.15.26:0/2190234012 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe9080504a0 con 0x7fe9140cb860 2023-12-19T16:59:53.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.955+0000 7fe91ab6d640 1 -- 172.21.15.26:0/2190234012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fe9140ad8d0 con 0x7fe9140cb860 2023-12-19T16:59:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.959+0000 7fe9117fa640 1 -- 172.21.15.26:0/2190234012 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v39) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe908053ac0 con 0x7fe9140cb860 2023-12-19T16:59:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.963+0000 7fe91ab6d640 1 -- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe8f8065b80 msgr2=0x7fe8f8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.963+0000 7fe91ab6d640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe8f8065b80 0x7fe8f8068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe9000b0000 tx=0x7fe9000b5040 comp rx=0 tx=0).stop 2023-12-19T16:59:53.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.963+0000 7fe91ab6d640 1 -- 172.21.15.26:0/2190234012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9140cb860 msgr2=0x7fe9140af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.963+0000 7fe91ab6d640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9140cb860 0x7fe9140af0b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe90800c500 tx=0x7fe90800c9c0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.963+0000 7fe91ab6d640 1 -- 172.21.15.26:0/2190234012 shutdown_connections 2023-12-19T16:59:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.963+0000 7fe91ab6d640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe91415d530 0x7fe9140af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.963+0000 7fe91ab6d640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe8f8065b80 0x7fe8f8068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.963+0000 7fe91ab6d640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe9141599e0 0x7fe91414f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.963+0000 7fe91ab6d640 1 --2- 172.21.15.26:0/2190234012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9140cb860 0x7fe9140af0b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.963+0000 7fe91ab6d640 1 -- 172.21.15.26:0/2190234012 >> 172.21.15.26:0/2190234012 conn(0x7fe914066160 msgr2=0x7fe914158c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.963+0000 7fe91ab6d640 1 -- 172.21.15.26:0/2190234012 shutdown_connections 2023-12-19T16:59:53.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:53.963+0000 7fe91ab6d640 1 -- 172.21.15.26:0/2190234012 wait complete. 2023-12-19T16:59:53.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-12-19T16:59:54.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.131+0000 7f6178a66640 1 Processor -- start 2023-12-19T16:59:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.131+0000 7f6178a66640 1 -- start start 2023-12-19T16:59:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.131+0000 7f6178a66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6174155030 0x7f617415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.131+0000 7f6178a66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61740ce740 0x7f61740cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.131+0000 7f6178a66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f617406c970 con 0x7f617415f370 2023-12-19T16:59:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.131+0000 7f6178a66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f617406c790 con 0x7f6174155030 2023-12-19T16:59:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.131+0000 7f6178a66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61740cf0f0 con 0x7f61740ce740 2023-12-19T16:59:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.131+0000 7f617259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6174155030 0x7f617415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.131+0000 7f617259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6174155030 0x7f617415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48084/0 (socket says 172.21.15.26:48084) 2023-12-19T16:59:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.131+0000 7f617259c640 1 -- 172.21.15.26:0/1865017702 learned_addr learned my addr 172.21.15.26:0/1865017702 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.135+0000 7f6172d9d640 1 --2- 172.21.15.26:0/1865017702 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61740ce740 0x7f61740cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.135+0000 7f617259c640 1 -- 172.21.15.26:0/1865017702 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61740ce740 msgr2=0x7f61740cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.135+0000 7f617259c640 1 --2- 172.21.15.26:0/1865017702 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61740ce740 0x7f61740cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.135+0000 7f617259c640 1 -- 172.21.15.26:0/1865017702 >> v1:172.21.15.26:6789/0 conn(0x7f617415f370 legacy=0x7f617406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:54.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.135+0000 7f617259c640 1 -- 172.21.15.26:0/1865017702 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617415b1f0 con 0x7f6174155030 2023-12-19T16:59:54.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.135+0000 7f617259c640 1 --2- 172.21.15.26:0/1865017702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6174155030 0x7f617415bcd0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6164009870 tx=0x7f616401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3889ca56da5fbf71 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:54.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.135+0000 7f6171d9b640 1 -- 172.21.15.26:0/1865017702 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61640aa670 con 0x7f6174155030 2023-12-19T16:59:54.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.135+0000 7f6171d9b640 1 -- 172.21.15.26:0/1865017702 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61640aa7d0 con 0x7f6174155030 2023-12-19T16:59:54.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.135+0000 7f6171d9b640 1 -- 172.21.15.26:0/1865017702 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61640aab60 con 0x7f6174155030 2023-12-19T16:59:54.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.135+0000 7f6178a66640 1 -- 172.21.15.26:0/1865017702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6174155030 msgr2=0x7f617415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:54.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.135+0000 7f6178a66640 1 --2- 172.21.15.26:0/1865017702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6174155030 0x7f617415bcd0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6164009870 tx=0x7f616401c510 comp rx=0 tx=0).stop 2023-12-19T16:59:54.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 -- 172.21.15.26:0/1865017702 shutdown_connections 2023-12-19T16:59:54.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 --2- 172.21.15.26:0/1865017702 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61740ce740 0x7f61740cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:54.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 --2- 172.21.15.26:0/1865017702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6174155030 0x7f617415bcd0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:54.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 -- 172.21.15.26:0/1865017702 >> 172.21.15.26:0/1865017702 conn(0x7f61740658e0 msgr2=0x7f6174065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:54.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 -- 172.21.15.26:0/1865017702 shutdown_connections 2023-12-19T16:59:54.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 -- 172.21.15.26:0/1865017702 wait complete. 2023-12-19T16:59:54.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 Processor -- start 2023-12-19T16:59:54.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 -- start start 2023-12-19T16:59:54.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61740ce740 0x7f6174159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:54.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6174155030 0x7f61740b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:54.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f617415f370 0x7f6174150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:54.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617415af70 con 0x7f61740ce740 2023-12-19T16:59:54.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61740b0b60 con 0x7f6174155030 2023-12-19T16:59:54.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f6178a66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6174151150 con 0x7f617415f370 2023-12-19T16:59:54.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f617259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61740ce740 0x7f6174159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:54.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f617259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61740ce740 0x7f6174159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48086/0 (socket says 172.21.15.26:48086) 2023-12-19T16:59:54.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f617259c640 1 -- 172.21.15.26:0/135500307 learned_addr learned my addr 172.21.15.26:0/135500307 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:54.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f617259c640 1 -- 172.21.15.26:0/135500307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f617415f370 msgr2=0x7f6174150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:54.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f617259c640 1 --2- 172.21.15.26:0/135500307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f617415f370 0x7f6174150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:54.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f617259c640 1 -- 172.21.15.26:0/135500307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6174155030 msgr2=0x7f61740b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:54.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f617259c640 1 --2- 172.21.15.26:0/135500307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6174155030 0x7f61740b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:54.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f617259c640 1 -- 172.21.15.26:0/135500307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640094d0 con 0x7f61740ce740 2023-12-19T16:59:54.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.139+0000 7f617259c640 1 --2- 172.21.15.26:0/135500307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61740ce740 0x7f6174159dc0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f616401a140 tx=0x7f616401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.143+0000 7f616affd640 1 -- 172.21.15.26:0/135500307 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61640b3800 con 0x7f61740ce740 2023-12-19T16:59:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.143+0000 7f616affd640 1 -- 172.21.15.26:0/135500307 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61640b3980 con 0x7f61740ce740 2023-12-19T16:59:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.143+0000 7f616affd640 1 -- 172.21.15.26:0/135500307 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61640b3d30 con 0x7f61740ce740 2023-12-19T16:59:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.143+0000 7f6178a66640 1 -- 172.21.15.26:0/135500307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6174158dc0 con 0x7f61740ce740 2023-12-19T16:59:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.143+0000 7f6178a66640 1 -- 172.21.15.26:0/135500307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61741595c0 con 0x7f61740ce740 2023-12-19T16:59:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.143+0000 7f6178a66640 1 -- 172.21.15.26:0/135500307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6174074270 con 0x7f61740ce740 2023-12-19T16:59:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.143+0000 7f616affd640 1 -- 172.21.15.26:0/135500307 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f61640b2380 con 0x7f61740ce740 2023-12-19T16:59:54.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.143+0000 7f616affd640 1 --2- 172.21.15.26:0/135500307 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6154065b80 0x7f6154068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:54.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.143+0000 7f616affd640 1 -- 172.21.15.26:0/135500307 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6363+0+0 (secure 0 0 0) 0x7f616411de80 con 0x7f61740ce740 2023-12-19T16:59:54.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.143+0000 7f616bfff640 1 --2- 172.21.15.26:0/135500307 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6154065b80 0x7f6154068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:54.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.147+0000 7f616bfff640 1 --2- 172.21.15.26:0/135500307 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6154065b80 0x7f6154068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f615c000f70 tx=0x7f615c0079c0 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.147+0000 7f616affd640 1 -- 172.21.15.26:0/135500307 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f61640eb4b0 con 0x7f61740ce740 2023-12-19T16:59:54.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.363+0000 7f6178a66640 1 -- 172.21.15.26:0/135500307 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f6174158d60 con 0x7f61740ce740 2023-12-19T16:59:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.903+0000 7f616affd640 1 -- 172.21.15.26:0/135500307 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v657) v1 ==== 183+0+0 (secure 0 0 0) 0x7f61640c30f0 con 0x7f61740ce740 2023-12-19T16:59:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.903+0000 7f6178a66640 1 -- 172.21.15.26:0/135500307 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6154065b80 msgr2=0x7f6154068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.903+0000 7f6178a66640 1 --2- 172.21.15.26:0/135500307 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6154065b80 0x7f6154068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f615c000f70 tx=0x7f615c0079c0 comp rx=0 tx=0).stop 2023-12-19T16:59:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.903+0000 7f6178a66640 1 -- 172.21.15.26:0/135500307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61740ce740 msgr2=0x7f6174159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.903+0000 7f6178a66640 1 --2- 172.21.15.26:0/135500307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61740ce740 0x7f6174159dc0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f616401a140 tx=0x7f616401ad50 comp rx=0 tx=0).stop 2023-12-19T16:59:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.903+0000 7f6178a66640 1 -- 172.21.15.26:0/135500307 shutdown_connections 2023-12-19T16:59:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.903+0000 7f6178a66640 1 --2- 172.21.15.26:0/135500307 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6154065b80 0x7f6154068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.903+0000 7f6178a66640 1 --2- 172.21.15.26:0/135500307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f617415f370 0x7f6174150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.903+0000 7f6178a66640 1 --2- 172.21.15.26:0/135500307 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6174155030 0x7f61740b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.903+0000 7f6178a66640 1 --2- 172.21.15.26:0/135500307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61740ce740 0x7f6174159dc0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:54.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.903+0000 7f6178a66640 1 -- 172.21.15.26:0/135500307 >> 172.21.15.26:0/135500307 conn(0x7f61740658e0 msgr2=0x7f6174070f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:54.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.903+0000 7f6178a66640 1 -- 172.21.15.26:0/135500307 shutdown_connections 2023-12-19T16:59:54.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:54.903+0000 7f6178a66640 1 -- 172.21.15.26:0/135500307 wait complete. 2023-12-19T16:59:54.929 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-12-19T16:59:54.929 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-12-19T16:59:54.929 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-19T16:59:54.930 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-12-19T16:59:55.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.071+0000 7efe3d4df640 1 Processor -- start 2023-12-19T16:59:55.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.071+0000 7efe3d4df640 1 -- start start 2023-12-19T16:59:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe3d4df640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efe3815aca0 0x7efe380ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe3d4df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe380cba70 0x7efe380cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe3d4df640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe380b1b60 con 0x7efe380cc410 2023-12-19T16:59:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe3d4df640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3815b080 con 0x7efe3815aca0 2023-12-19T16:59:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe3d4df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe380cc890 con 0x7efe380cba70 2023-12-19T16:59:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe36ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efe3815aca0 0x7efe380ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:55.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe377fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efe380cc410 0x7efe3815e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43782/0 (socket says 172.21.15.26:43782) 2023-12-19T16:59:55.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe36ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efe3815aca0 0x7efe380ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48098/0 (socket says 172.21.15.26:48098) 2023-12-19T16:59:55.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe377fe640 1 -- 172.21.15.26:0/2732282127 learned_addr learned my addr 172.21.15.26:0/2732282127 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:55.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe367fc640 1 --2- 172.21.15.26:0/2732282127 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe380cba70 0x7efe380cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:55.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe36ffd640 1 -- 172.21.15.26:0/2732282127 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe380cba70 msgr2=0x7efe380cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:55.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe36ffd640 1 --2- 172.21.15.26:0/2732282127 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe380cba70 0x7efe380cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:55.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe36ffd640 1 -- 172.21.15.26:0/2732282127 >> v1:172.21.15.26:6789/0 conn(0x7efe380cc410 legacy=0x7efe3815e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:55.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe36ffd640 1 -- 172.21.15.26:0/2732282127 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe380ce1e0 con 0x7efe3815aca0 2023-12-19T16:59:55.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.075+0000 7efe367fc640 1 --2- 172.21.15.26:0/2732282127 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe380cba70 0x7efe380cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:59:55.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe36ffd640 1 --2- 172.21.15.26:0/2732282127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe3815aca0 0x7efe380ccf60 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7efe280097a0 tx=0x7efe2801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6dd07dfd7025e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:55.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe35ffb640 1 -- 172.21.15.26:0/2732282127 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe280ab660 con 0x7efe3815aca0 2023-12-19T16:59:55.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe35ffb640 1 -- 172.21.15.26:0/2732282127 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe280ab7c0 con 0x7efe3815aca0 2023-12-19T16:59:55.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe35ffb640 1 -- 172.21.15.26:0/2732282127 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe280abb50 con 0x7efe3815aca0 2023-12-19T16:59:55.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe3d4df640 1 -- 172.21.15.26:0/2732282127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe3815aca0 msgr2=0x7efe380ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:55.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe3d4df640 1 --2- 172.21.15.26:0/2732282127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe3815aca0 0x7efe380ccf60 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7efe280097a0 tx=0x7efe2801c460 comp rx=0 tx=0).stop 2023-12-19T16:59:55.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe3d4df640 1 -- 172.21.15.26:0/2732282127 shutdown_connections 2023-12-19T16:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe3d4df640 1 --2- 172.21.15.26:0/2732282127 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe380cba70 0x7efe380cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe3d4df640 1 --2- 172.21.15.26:0/2732282127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe3815aca0 0x7efe380ccf60 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe3d4df640 1 -- 172.21.15.26:0/2732282127 >> 172.21.15.26:0/2732282127 conn(0x7efe3806d5d0 msgr2=0x7efe38069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe3d4df640 1 -- 172.21.15.26:0/2732282127 shutdown_connections 2023-12-19T16:59:55.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe3d4df640 1 -- 172.21.15.26:0/2732282127 wait complete. 2023-12-19T16:59:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe3d4df640 1 Processor -- start 2023-12-19T16:59:55.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.079+0000 7efe3d4df640 1 -- start start 2023-12-19T16:59:55.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe3d4df640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe380cba70 0x7efe380c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:55.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe3d4df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe380cc410 0x7efe3814aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:55.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe3d4df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe3815aca0 0x7efe3814bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:55.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe3d4df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380c7740 con 0x7efe380cc410 2023-12-19T16:59:55.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe3d4df640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380c7990 con 0x7efe380cba70 2023-12-19T16:59:55.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe3d4df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe380c7c70 con 0x7efe3815aca0 2023-12-19T16:59:55.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe367fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe380cc410 0x7efe3814aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:55.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe367fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe380cc410 0x7efe3814aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48114/0 (socket says 172.21.15.26:48114) 2023-12-19T16:59:55.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe367fc640 1 -- 172.21.15.26:0/737687425 learned_addr learned my addr 172.21.15.26:0/737687425 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:55.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe377fe640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe3815aca0 0x7efe3814bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:55.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe36ffd640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe380cba70 0x7efe380c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:55.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe367fc640 1 -- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe3815aca0 msgr2=0x7efe3814bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:55.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe367fc640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe3815aca0 0x7efe3814bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:55.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe367fc640 1 -- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe380cba70 msgr2=0x7efe380c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:55.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe367fc640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe380cba70 0x7efe380c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:55.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe367fc640 1 -- 172.21.15.26:0/737687425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe280093b0 con 0x7efe380cc410 2023-12-19T16:59:55.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe367fc640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe380cc410 0x7efe3814aac0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c7f0 tx=0x7efe2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe1f7fe640 1 -- 172.21.15.26:0/737687425 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c007820 con 0x7efe380cc410 2023-12-19T16:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe1f7fe640 1 -- 172.21.15.26:0/737687425 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c007980 con 0x7efe380cc410 2023-12-19T16:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe1f7fe640 1 -- 172.21.15.26:0/737687425 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c007cf0 con 0x7efe380cc410 2023-12-19T16:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe3d4df640 1 -- 172.21.15.26:0/737687425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe380c7f50 con 0x7efe380cc410 2023-12-19T16:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe3d4df640 1 -- 172.21.15.26:0/737687425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe380c8510 con 0x7efe380cc410 2023-12-19T16:59:55.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.083+0000 7efe3d4df640 1 -- 172.21.15.26:0/737687425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe38074270 con 0x7efe380cc410 2023-12-19T16:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.087+0000 7efe1f7fe640 1 -- 172.21.15.26:0/737687425 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7efe2c0113f0 con 0x7efe380cc410 2023-12-19T16:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.087+0000 7efe1f7fe640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe24065b50 0x7efe24068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.087+0000 7efe1f7fe640 1 -- 172.21.15.26:0/737687425 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6363+0+0 (secure 0 0 0) 0x7efe2c083970 con 0x7efe380cc410 2023-12-19T16:59:55.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.087+0000 7efe36ffd640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe24065b50 0x7efe24068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:55.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.087+0000 7efe36ffd640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe24065b50 0x7efe24068010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efe280af000 tx=0x7efe280a7000 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:55.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.091+0000 7efe1f7fe640 1 -- 172.21.15.26:0/737687425 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efe2c050f10 con 0x7efe380cc410 2023-12-19T16:59:55.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.303+0000 7efe3d4df640 1 -- 172.21.15.26:0/737687425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7efe380cdbf0 con 0x7efe380cc410 2023-12-19T16:59:55.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.907+0000 7efe1f7fe640 1 -- 172.21.15.26:0/737687425 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v658) v1 ==== 151+0+0 (secure 0 0 0) 0x7efe2c01f0d0 con 0x7efe380cc410 2023-12-19T16:59:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.911+0000 7efe3d4df640 1 -- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe24065b50 msgr2=0x7efe24068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.911+0000 7efe3d4df640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe24065b50 0x7efe24068010 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efe280af000 tx=0x7efe280a7000 comp rx=0 tx=0).stop 2023-12-19T16:59:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.911+0000 7efe3d4df640 1 -- 172.21.15.26:0/737687425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe380cc410 msgr2=0x7efe3814aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.911+0000 7efe3d4df640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe380cc410 0x7efe3814aac0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c7f0 tx=0x7efe2c00ccb0 comp rx=0 tx=0).stop 2023-12-19T16:59:55.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.911+0000 7efe3d4df640 1 -- 172.21.15.26:0/737687425 shutdown_connections 2023-12-19T16:59:55.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.911+0000 7efe3d4df640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe24065b50 0x7efe24068010 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:55.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.911+0000 7efe3d4df640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efe3815aca0 0x7efe3814bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:55.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.911+0000 7efe3d4df640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efe380cc410 0x7efe3814aac0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:55.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.911+0000 7efe3d4df640 1 --2- 172.21.15.26:0/737687425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efe380cba70 0x7efe380c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:55.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.911+0000 7efe3d4df640 1 -- 172.21.15.26:0/737687425 >> 172.21.15.26:0/737687425 conn(0x7efe3806d5d0 msgr2=0x7efe380cb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:55.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.911+0000 7efe3d4df640 1 -- 172.21.15.26:0/737687425 shutdown_connections 2023-12-19T16:59:55.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:55.911+0000 7efe3d4df640 1 -- 172.21.15.26:0/737687425 wait complete. 2023-12-19T16:59:55.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T16:59:56.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.083+0000 7fa0e3f96640 1 Processor -- start 2023-12-19T16:59:56.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.083+0000 7fa0e3f96640 1 -- start start 2023-12-19T16:59:56.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e3f96640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa0dc0ccf10 0x7fa0dc0ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e3f96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0dc15d250 0x7fa0dc15d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e3f96640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0dc0ac860 con 0x7fa0dc0b01b0 2023-12-19T16:59:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e3f96640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0dc157730 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e3f96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc0ad7f0 con 0x7fa0dc15d250 2023-12-19T16:59:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e2533640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa0dc0b01b0 0x7fa0dc0c8cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46786/0 (socket says 172.21.15.26:46786) 2023-12-19T16:59:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e2533640 1 -- 172.21.15.26:0/3529388305 learned_addr learned my addr 172.21.15.26:0/3529388305 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e1531640 1 --2- 172.21.15.26:0/3529388305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0dc15d250 0x7fa0dc15d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e1d32640 1 --2- 172.21.15.26:0/3529388305 >> v2:172.21.15.160:3300/0 conn(0x7fa0dc0ccf10 0x7fa0dc0ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e0d30640 1 -- 172.21.15.26:0/3529388305 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120555234 0 0) 0x7fa0dc0ac860 con 0x7fa0dc0b01b0 2023-12-19T16:59:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e0d30640 1 -- 172.21.15.26:0/3529388305 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0c8003530 con 0x7fa0dc0b01b0 2023-12-19T16:59:56.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e1531640 1 -- 172.21.15.26:0/3529388305 >> v2:172.21.15.160:3300/0 conn(0x7fa0dc0ccf10 msgr2=0x7fa0dc0ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:56.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e1531640 1 --2- 172.21.15.26:0/3529388305 >> v2:172.21.15.160:3300/0 conn(0x7fa0dc0ccf10 0x7fa0dc0ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e1531640 1 -- 172.21.15.26:0/3529388305 >> v1:172.21.15.26:6789/0 conn(0x7fa0dc0b01b0 legacy=0x7fa0dc0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:56.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e1531640 1 -- 172.21.15.26:0/3529388305 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc15dd10 con 0x7fa0dc15d250 2023-12-19T16:59:56.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.087+0000 7fa0e1d32640 1 --2- 172.21.15.26:0/3529388305 >> v2:172.21.15.160:3300/0 conn(0x7fa0dc0ccf10 0x7fa0dc0ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:59:56.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e1531640 1 --2- 172.21.15.26:0/3529388305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0dc15d250 0x7fa0dc15d6d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0097c0 tx=0x7fa0cc01ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7939de452779952f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:56.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e0d30640 1 -- 172.21.15.26:0/3529388305 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0a8970 con 0x7fa0dc15d250 2023-12-19T16:59:56.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e0d30640 1 -- 172.21.15.26:0/3529388305 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc0a8ad0 con 0x7fa0dc15d250 2023-12-19T16:59:56.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e0d30640 1 -- 172.21.15.26:0/3529388305 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0a8e60 con 0x7fa0dc15d250 2023-12-19T16:59:56.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e3f96640 1 -- 172.21.15.26:0/3529388305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0dc15d250 msgr2=0x7fa0dc15d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:56.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e3f96640 1 --2- 172.21.15.26:0/3529388305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0dc15d250 0x7fa0dc15d6d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc0097c0 tx=0x7fa0cc01ec70 comp rx=0 tx=0).stop 2023-12-19T16:59:56.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e3f96640 1 -- 172.21.15.26:0/3529388305 shutdown_connections 2023-12-19T16:59:56.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e3f96640 1 --2- 172.21.15.26:0/3529388305 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0dc15d250 0x7fa0dc15d6d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e3f96640 1 --2- 172.21.15.26:0/3529388305 >> v2:172.21.15.160:3300/0 conn(0x7fa0dc0ccf10 0x7fa0dc0ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e3f96640 1 -- 172.21.15.26:0/3529388305 >> 172.21.15.26:0/3529388305 conn(0x7fa0dc0ce730 msgr2=0x7fa0dc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:56.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e3f96640 1 -- 172.21.15.26:0/3529388305 shutdown_connections 2023-12-19T16:59:56.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e3f96640 1 -- 172.21.15.26:0/3529388305 wait complete. 2023-12-19T16:59:56.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e3f96640 1 Processor -- start 2023-12-19T16:59:56.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.091+0000 7fa0e3f96640 1 -- start start 2023-12-19T16:59:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e3f96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0dc0b01b0 0x7fa0dc0c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e3f96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0dc0ccf10 0x7fa0dc15fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e3f96640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0dc15d250 0x7fa0dc1f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e3f96640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc0c3220 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e3f96640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc160100 con 0x7fa0dc15d250 2023-12-19T16:59:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e3f96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc1f2300 con 0x7fa0dc0b01b0 2023-12-19T16:59:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e1531640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0dc0ccf10 0x7fa0dc15fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e1531640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0dc0ccf10 0x7fa0dc15fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45668/0 (socket says 172.21.15.26:45668) 2023-12-19T16:59:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e1531640 1 -- 172.21.15.26:0/2328068209 learned_addr learned my addr 172.21.15.26:0/2328068209 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e1d32640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0dc0b01b0 0x7fa0dc0c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:56.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e2533640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0dc15d250 0x7fa0dc1f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e1531640 1 -- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0dc0b01b0 msgr2=0x7fa0dc0c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e1531640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0dc0b01b0 0x7fa0dc0c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e1531640 1 -- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0dc15d250 msgr2=0x7fa0dc1f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e1531640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0dc15d250 0x7fa0dc1f1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e1531640 1 -- 172.21.15.26:0/2328068209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d8009420 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e1531640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0dc0ccf10 0x7fa0dc15fbc0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc01d320 tx=0x7fa0cc01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0d27fc640 1 -- 172.21.15.26:0/2328068209 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0b0d60 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0d27fc640 1 -- 172.21.15.26:0/2328068209 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc0bb440 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0d27fc640 1 -- 172.21.15.26:0/2328068209 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0bb7f0 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e3f96640 1 -- 172.21.15.26:0/2328068209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc009420 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e3f96640 1 -- 172.21.15.26:0/2328068209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0dc0c3430 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0e3f96640 1 -- 172.21.15.26:0/2328068209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0dc074270 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.095+0000 7fa0d27fc640 1 -- 172.21.15.26:0/2328068209 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa0cc0b7040 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.099+0000 7fa0d27fc640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa0c0065b80 0x7fa0c0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:56.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.099+0000 7fa0d27fc640 1 -- 172.21.15.26:0/2328068209 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6363+0+0 (secure 0 0 0) 0x7fa0cc1213c0 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.099+0000 7fa0e1d32640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa0c0065b80 0x7fa0c0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:56.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.099+0000 7fa0e1d32640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa0c0065b80 0x7fa0c0068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009c80 tx=0x7fa0d8008040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:56.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.103+0000 7fa0d27fc640 1 -- 172.21.15.26:0/2328068209 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa0cc0eeaf0 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.259+0000 7fa0e3f96640 1 -- 172.21.15.26:0/2328068209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa0dc0b46b0 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.263+0000 7fa0d27fc640 1 -- 172.21.15.26:0/2328068209 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa0cc0f2110 con 0x7fa0dc0ccf10 2023-12-19T16:59:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.263+0000 7fa0e3f96640 1 -- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa0c0065b80 msgr2=0x7fa0c0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.263+0000 7fa0e3f96640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa0c0065b80 0x7fa0c0068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009c80 tx=0x7fa0d8008040 comp rx=0 tx=0).stop 2023-12-19T16:59:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.263+0000 7fa0e3f96640 1 -- 172.21.15.26:0/2328068209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0dc0ccf10 msgr2=0x7fa0dc15fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.263+0000 7fa0e3f96640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0dc0ccf10 0x7fa0dc15fbc0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc01d320 tx=0x7fa0cc01ba10 comp rx=0 tx=0).stop 2023-12-19T16:59:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.267+0000 7fa0e3f96640 1 -- 172.21.15.26:0/2328068209 shutdown_connections 2023-12-19T16:59:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.267+0000 7fa0e3f96640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0dc15d250 0x7fa0dc1f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.267+0000 7fa0e3f96640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0dc0ccf10 0x7fa0dc15fbc0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.267+0000 7fa0e3f96640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa0c0065b80 0x7fa0c0068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.267+0000 7fa0e3f96640 1 --2- 172.21.15.26:0/2328068209 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0dc0b01b0 0x7fa0dc0c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.267+0000 7fa0e3f96640 1 -- 172.21.15.26:0/2328068209 >> 172.21.15.26:0/2328068209 conn(0x7fa0dc0ce730 msgr2=0x7fa0dc0cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.267+0000 7fa0e3f96640 1 -- 172.21.15.26:0/2328068209 shutdown_connections 2023-12-19T16:59:56.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.267+0000 7fa0e3f96640 1 -- 172.21.15.26:0/2328068209 wait complete. 2023-12-19T16:59:56.287 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T16:59:56.287 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:59:56.374 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T16:59:56.463 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T16:59:56.463 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:59:56.463 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:59:56.472 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:59:56.473 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:59:56.473 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:59:56.482 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:59:56.483 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T16:59:56.483 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:59:56.523 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T16:59:56.524 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T16:59:56.524 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T16:59:56.533 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T16:59:56.534 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T16:59:56.534 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T16:59:56.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.707+0000 7ffbcb0a7640 1 Processor -- start 2023-12-19T16:59:56.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.707+0000 7ffbcb0a7640 1 -- start start 2023-12-19T16:59:56.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.707+0000 7ffbcb0a7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffbc40b6ae0 0x7ffbc40b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:56.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.707+0000 7ffbcb0a7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbc40b4960 0x7ffbc40d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:56.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.707+0000 7ffbcb0a7640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc415cfd0 con 0x7ffbc415cbf0 2023-12-19T16:59:56.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.707+0000 7ffbcb0a7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc40b4df0 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.707+0000 7ffbcb0a7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40b6fc0 con 0x7ffbc40b4960 2023-12-19T16:59:56.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.707+0000 7ffbc98a4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffbc40b6ae0 0x7ffbc40b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:56.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.707+0000 7ffbc98a4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffbc40b6ae0 0x7ffbc40b4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45674/0 (socket says 172.21.15.26:45674) 2023-12-19T16:59:56.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.707+0000 7ffbc98a4640 1 -- 172.21.15.26:0/342500564 learned_addr learned my addr 172.21.15.26:0/342500564 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:56.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.707+0000 7ffbca8a6640 1 --2- 172.21.15.26:0/342500564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbc40b4960 0x7ffbc40d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:56.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.707+0000 7ffbc98a4640 1 -- 172.21.15.26:0/342500564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbc40b4960 msgr2=0x7ffbc40d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:56.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.707+0000 7ffbc98a4640 1 --2- 172.21.15.26:0/342500564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbc40b4960 0x7ffbc40d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.711+0000 7ffbc98a4640 1 -- 172.21.15.26:0/342500564 >> v1:172.21.15.160:6789/0 conn(0x7ffbc415cbf0 legacy=0x7ffbc40b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:56.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.711+0000 7ffbc98a4640 1 -- 172.21.15.26:0/342500564 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40b72a0 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.711+0000 7ffbc90a3640 1 -- 172.21.15.26:0/342500564 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714717153 0 0) 0x7ffbc415cfd0 con 0x7ffbc415cbf0 2023-12-19T16:59:56.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.711+0000 7ffbc98a4640 1 --2- 172.21.15.26:0/342500564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbc40b6ae0 0x7ffbc40b4420 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009870 tx=0x7ffbb401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c952e9079f3991aa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:56.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.711+0000 7ffbc90a3640 1 -- 172.21.15.26:0/342500564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40aa430 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.711+0000 7ffbc90a3640 1 -- 172.21.15.26:0/342500564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb40aa590 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.711+0000 7ffbc90a3640 1 -- 172.21.15.26:0/342500564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40aa920 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.711+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/342500564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbc40b6ae0 msgr2=0x7ffbc40b4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:56.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.711+0000 7ffbcb0a7640 1 --2- 172.21.15.26:0/342500564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbc40b6ae0 0x7ffbc40b4420 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4009870 tx=0x7ffbb401c4b0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.711+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/342500564 shutdown_connections 2023-12-19T16:59:56.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.711+0000 7ffbcb0a7640 1 --2- 172.21.15.26:0/342500564 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbc40b4960 0x7ffbc40d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.711+0000 7ffbcb0a7640 1 --2- 172.21.15.26:0/342500564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbc40b6ae0 0x7ffbc40b4420 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.711+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/342500564 >> 172.21.15.26:0/342500564 conn(0x7ffbc406a120 msgr2=0x7ffbc406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:56.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/342500564 shutdown_connections 2023-12-19T16:59:56.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/342500564 wait complete. 2023-12-19T16:59:56.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbcb0a7640 1 Processor -- start 2023-12-19T16:59:56.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbcb0a7640 1 -- start start 2023-12-19T16:59:56.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbcb0a7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbc40b4960 0x7ffbc40c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:56.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbcb0a7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbc40b6ae0 0x7ffbc4151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:56.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbcb0a7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbc415cbf0 0x7ffbc40cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:56.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbcb0a7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc4146720 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbcb0a7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc41522b0 con 0x7ffbc415cbf0 2023-12-19T16:59:56.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbcb0a7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc4152010 con 0x7ffbc40b4960 2023-12-19T16:59:56.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbc98a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbc40b6ae0 0x7ffbc4151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:56.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbc98a4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbc40b6ae0 0x7ffbc4151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45676/0 (socket says 172.21.15.26:45676) 2023-12-19T16:59:56.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbc98a4640 1 -- 172.21.15.26:0/1029865872 learned_addr learned my addr 172.21.15.26:0/1029865872 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:56.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbca0a5640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbc40b4960 0x7ffbc40c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:56.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbca8a6640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbc415cbf0 0x7ffbc40cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:56.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbc98a4640 1 -- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbc40b4960 msgr2=0x7ffbc40c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:56.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbc98a4640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbc40b4960 0x7ffbc40c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbc98a4640 1 -- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbc415cbf0 msgr2=0x7ffbc40cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbc98a4640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbc415cbf0 0x7ffbc40cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbc98a4640 1 -- 172.21.15.26:0/1029865872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb40094d0 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbca8a6640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbc415cbf0 0x7ffbc40cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T16:59:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbc98a4640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbc40b6ae0 0x7ffbc4151ad0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40ac040 tx=0x7ffbb401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbbaffd640 1 -- 172.21.15.26:0/1029865872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40aa430 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbbaffd640 1 -- 172.21.15.26:0/1029865872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb40b3820 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbbaffd640 1 -- 172.21.15.26:0/1029865872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40b3bd0 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.715+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/1029865872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc4145d80 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.719+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/1029865872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc4146340 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.719+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/1029865872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc4146860 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.719+0000 7ffbbaffd640 1 -- 172.21.15.26:0/1029865872 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ffbb40b3d30 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.719+0000 7ffbbaffd640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffba8065b80 0x7ffba8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.719+0000 7ffbbaffd640 1 -- 172.21.15.26:0/1029865872 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6363+0+0 (secure 0 0 0) 0x7ffbb411ded0 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.719+0000 7ffbca0a5640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffba8065b80 0x7ffba8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:56.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.719+0000 7ffbca0a5640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffba8065b80 0x7ffba8068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0007fd0 tx=0x7ffbc0009040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:56.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.723+0000 7ffbbaffd640 1 -- 172.21.15.26:0/1029865872 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffbc4146860 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.915+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/1029865872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbc4146860 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.915+0000 7ffbbaffd640 1 -- 172.21.15.26:0/1029865872 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3535 (secure 0 0 0) 0x7ffbc4146860 con 0x7ffbc40b6ae0 2023-12-19T16:59:56.918 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:59:56.918 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3393501439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3393501439},{"type":"v1","addr":"172.21.15.160:6836","nonce":3393501439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1517280064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1517280064},{"type":"v1","addr":"172.21.15.160:6837","nonce":1517280064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":7473,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/2282339695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2282339695},{"type":"v1","addr":"172.21.15.26:6834","nonce":2282339695}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":44},{"gid":7884,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1387305673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1387305673},{"type":"v1","addr":"172.21.15.26:6835","nonce":1387305673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[]} 2023-12-19T16:59:56.919 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 51 2023-12-19T16:59:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.919+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffba8065b80 msgr2=0x7ffba8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.919+0000 7ffbcb0a7640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffba8065b80 0x7ffba8068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0007fd0 tx=0x7ffbc0009040 comp rx=0 tx=0).stop 2023-12-19T16:59:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.919+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/1029865872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbc40b6ae0 msgr2=0x7ffbc4151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.919+0000 7ffbcb0a7640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbc40b6ae0 0x7ffbc4151ad0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40ac040 tx=0x7ffbb401acf0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.919+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/1029865872 shutdown_connections 2023-12-19T16:59:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.919+0000 7ffbcb0a7640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbc415cbf0 0x7ffbc40cc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.919+0000 7ffbcb0a7640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbc40b6ae0 0x7ffbc4151ad0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.919+0000 7ffbcb0a7640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffba8065b80 0x7ffba8068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.919+0000 7ffbcb0a7640 1 --2- 172.21.15.26:0/1029865872 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbc40b4960 0x7ffbc40c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.919+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/1029865872 >> 172.21.15.26:0/1029865872 conn(0x7ffbc406a120 msgr2=0x7ffbc415bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.919+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/1029865872 shutdown_connections 2023-12-19T16:59:56.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:56.919+0000 7ffbcb0a7640 1 -- 172.21.15.26:0/1029865872 wait complete. 2023-12-19T16:59:56.939 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T16:59:56.940 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T16:59:56.940 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T16:59:56.940 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:59:56.940 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T16:59:56.940 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T16:59:56.940 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T16:59:56.941 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:59:56.941 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T16:59:56.941 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T16:59:56.941 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T16:59:56.941 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:59:56.945 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T16:59:56.945 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T16:59:56.945 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T16:59:56.945 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T16:59:56.951 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T16:59:56.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T16:59:56.952 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T16:59:56.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T16:59:56.957 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T16:59:56.957 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T16:59:56.961 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T16:59:56.961 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T16:59:57.002 INFO:tasks.ceph.mds.c:Started 2023-12-19T16:59:57.003 DEBUG:teuthology.parallel:result is None 2023-12-19T16:59:57.004 INFO:tasks.ceph.mds.a:Started 2023-12-19T16:59:57.004 DEBUG:teuthology.parallel:result is None 2023-12-19T16:59:57.006 INFO:tasks.ceph.mds.b:Started 2023-12-19T16:59:57.007 DEBUG:teuthology.parallel:result is None 2023-12-19T16:59:57.009 INFO:tasks.ceph.mds.d:Started 2023-12-19T16:59:57.009 DEBUG:teuthology.parallel:result is None 2023-12-19T16:59:57.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T16:59:57.093 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T16:59:57.094 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T16:59:57.097 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T16:59:57.098 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T16:59:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.159+0000 7ff347b6d640 1 Processor -- start 2023-12-19T16:59:57.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.159+0000 7ff347b6d640 1 -- start start 2023-12-19T16:59:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.159+0000 7ff347b6d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff34006b3d0 0x7ff3400ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff347b6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3400cec40 0x7ff34015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:57.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff347b6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34006bc30 con 0x7ff340154220 2023-12-19T16:59:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff347b6d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3400cf0d0 con 0x7ff34006b3d0 2023-12-19T16:59:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff347b6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400cc6e0 con 0x7ff3400cec40 2023-12-19T16:59:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff345909640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff340154220 0x7ff34015aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46796/0 (socket says 172.21.15.26:46796) 2023-12-19T16:59:57.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff345909640 1 -- 172.21.15.26:0/2750335944 learned_addr learned my addr 172.21.15.26:0/2750335944 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff34610a640 1 --2- 172.21.15.26:0/2750335944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3400cec40 0x7ff34015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff345108640 1 --2- 172.21.15.26:0/2750335944 >> v2:172.21.15.160:3300/0 conn(0x7ff34006b3d0 0x7ff3400ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff344907640 1 -- 172.21.15.26:0/2750335944 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91461191 0 0) 0x7ff34006bc30 con 0x7ff340154220 2023-12-19T16:59:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff344907640 1 -- 172.21.15.26:0/2750335944 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff32c003530 con 0x7ff340154220 2023-12-19T16:59:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff34610a640 1 -- 172.21.15.26:0/2750335944 >> v2:172.21.15.160:3300/0 conn(0x7ff34006b3d0 msgr2=0x7ff3400ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff34610a640 1 --2- 172.21.15.26:0/2750335944 >> v2:172.21.15.160:3300/0 conn(0x7ff34006b3d0 0x7ff3400ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff34610a640 1 -- 172.21.15.26:0/2750335944 >> v1:172.21.15.26:6789/0 conn(0x7ff340154220 legacy=0x7ff34015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:57.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff34610a640 1 -- 172.21.15.26:0/2750335944 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34006b950 con 0x7ff3400cec40 2023-12-19T16:59:57.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.163+0000 7ff34610a640 1 --2- 172.21.15.26:0/2750335944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3400cec40 0x7ff34015b1c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff338009d90 tx=0x7ff33801eb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3abc390b4b9a1f5f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:57.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff344907640 1 -- 172.21.15.26:0/2750335944 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3380a7c00 con 0x7ff3400cec40 2023-12-19T16:59:57.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff344907640 1 -- 172.21.15.26:0/2750335944 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3380a7d60 con 0x7ff3400cec40 2023-12-19T16:59:57.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff344907640 1 -- 172.21.15.26:0/2750335944 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3380af640 con 0x7ff3400cec40 2023-12-19T16:59:57.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 -- 172.21.15.26:0/2750335944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3400cec40 msgr2=0x7ff34015b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:57.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 --2- 172.21.15.26:0/2750335944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3400cec40 0x7ff34015b1c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff338009d90 tx=0x7ff33801eb30 comp rx=0 tx=0).stop 2023-12-19T16:59:57.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 -- 172.21.15.26:0/2750335944 shutdown_connections 2023-12-19T16:59:57.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 --2- 172.21.15.26:0/2750335944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3400cec40 0x7ff34015b1c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:57.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 --2- 172.21.15.26:0/2750335944 >> v2:172.21.15.160:3300/0 conn(0x7ff34006b3d0 0x7ff3400ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:57.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 -- 172.21.15.26:0/2750335944 >> 172.21.15.26:0/2750335944 conn(0x7ff3400658e0 msgr2=0x7ff340065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:57.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 -- 172.21.15.26:0/2750335944 shutdown_connections 2023-12-19T16:59:57.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 -- 172.21.15.26:0/2750335944 wait complete. 2023-12-19T16:59:57.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 Processor -- start 2023-12-19T16:59:57.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 -- start start 2023-12-19T16:59:57.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff34006b3d0 0x7ff34014b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:57.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3400cec40 0x7ff340153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:57.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff340154220 0x7ff3401501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:57.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff340153a40 con 0x7ff340154220 2023-12-19T16:59:57.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400b1f60 con 0x7ff34006b3d0 2023-12-19T16:59:57.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff347b6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400b2240 con 0x7ff3400cec40 2023-12-19T16:59:57.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff34610a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff340154220 0x7ff3401501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:57.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff345108640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3400cec40 0x7ff340153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:57.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff345909640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff34006b3d0 0x7ff34014b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:57.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff345108640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3400cec40 0x7ff340153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:60232/0 (socket says 172.21.15.26:60232) 2023-12-19T16:59:57.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff345108640 1 -- 172.21.15.26:0/1287395443 learned_addr learned my addr 172.21.15.26:0/1287395443 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:57.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.167+0000 7ff345909640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff34006b3d0 0x7ff34014b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:33784/0 (socket says 172.21.15.26:33784) 2023-12-19T16:59:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff345108640 1 -- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff34006b3d0 msgr2=0x7ff34014b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff345108640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff34006b3d0 0x7ff34014b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff345108640 1 -- 172.21.15.26:0/1287395443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff340154220 msgr2=0x7ff3401501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff345108640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff340154220 0x7ff3401501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff345108640 1 -- 172.21.15.26:0/1287395443 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff338009a70 con 0x7ff3400cec40 2023-12-19T16:59:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff34610a640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff340154220 0x7ff3401501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T16:59:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff345108640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3400cec40 0x7ff340153500 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff33000e870 tx=0x7ff33000ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff3367fc640 1 -- 172.21.15.26:0/1287395443 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330009510 con 0x7ff3400cec40 2023-12-19T16:59:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff347b6d640 1 -- 172.21.15.26:0/1287395443 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3400b2520 con 0x7ff3400cec40 2023-12-19T16:59:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff3367fc640 1 -- 172.21.15.26:0/1287395443 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33000eec0 con 0x7ff3400cec40 2023-12-19T16:59:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff3367fc640 1 -- 172.21.15.26:0/1287395443 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33000cb40 con 0x7ff3400cec40 2023-12-19T16:59:57.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff347b6d640 1 -- 172.21.15.26:0/1287395443 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3400b2ae0 con 0x7ff3400cec40 2023-12-19T16:59:57.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff347b6d640 1 -- 172.21.15.26:0/1287395443 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff340074270 con 0x7ff3400cec40 2023-12-19T16:59:57.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.171+0000 7ff3367fc640 1 -- 172.21.15.26:0/1287395443 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff33000cca0 con 0x7ff3400cec40 2023-12-19T16:59:57.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.175+0000 7ff3367fc640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff324065bb0 0x7ff324068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:57.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.175+0000 7ff3367fc640 1 -- 172.21.15.26:0/1287395443 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6363+0+0 (secure 0 0 0) 0x7ff330083890 con 0x7ff3400cec40 2023-12-19T16:59:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.175+0000 7ff345909640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff324065bb0 0x7ff324068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:57.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.175+0000 7ff345909640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff324065bb0 0x7ff324068070 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff33c007fd0 tx=0x7ff33c009040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:57.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.175+0000 7ff3367fc640 1 -- 172.21.15.26:0/1287395443 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff330050ec0 con 0x7ff3400cec40 2023-12-19T16:59:57.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:57.347+0000 7ff347b6d640 1 -- 172.21.15.26:0/1287395443 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff3400b2380 con 0x7ff3400cec40 2023-12-19T16:59:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.151+0000 7ff3367fc640 1 -- 172.21.15.26:0/1287395443 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v119) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff3400b2380 con 0x7ff3400cec40 2023-12-19T16:59:58.152 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2023-12-19T16:59:58.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.151+0000 7ff347b6d640 1 -- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff324065bb0 msgr2=0x7ff324068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.151+0000 7ff347b6d640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff324065bb0 0x7ff324068070 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff33c007fd0 tx=0x7ff33c009040 comp rx=0 tx=0).stop 2023-12-19T16:59:58.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.151+0000 7ff347b6d640 1 -- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3400cec40 msgr2=0x7ff340153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.151+0000 7ff347b6d640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3400cec40 0x7ff340153500 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff33000e870 tx=0x7ff33000ec70 comp rx=0 tx=0).stop 2023-12-19T16:59:58.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.151+0000 7ff347b6d640 1 -- 172.21.15.26:0/1287395443 shutdown_connections 2023-12-19T16:59:58.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.151+0000 7ff347b6d640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff340154220 0x7ff3401501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.151+0000 7ff347b6d640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff324065bb0 0x7ff324068070 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.151+0000 7ff347b6d640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff3400cec40 0x7ff340153500 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.151+0000 7ff347b6d640 1 --2- 172.21.15.26:0/1287395443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff34006b3d0 0x7ff34014b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.151+0000 7ff347b6d640 1 -- 172.21.15.26:0/1287395443 >> 172.21.15.26:0/1287395443 conn(0x7ff3400658e0 msgr2=0x7ff34015c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:58.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.151+0000 7ff347b6d640 1 -- 172.21.15.26:0/1287395443 shutdown_connections 2023-12-19T16:59:58.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.151+0000 7ff347b6d640 1 -- 172.21.15.26:0/1287395443 wait complete. 2023-12-19T16:59:58.176 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T16:59:58.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.319+0000 7fcd33b39640 1 Processor -- start 2023-12-19T16:59:58.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.319+0000 7fcd33b39640 1 -- start start 2023-12-19T16:59:58.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.319+0000 7fcd33b39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd2c06b3d0 0x7fcd2c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:58.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.319+0000 7fcd33b39640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcd2c0cec40 0x7fcd2c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:58.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.319+0000 7fcd33b39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2c06bc30 con 0x7fcd2c154220 2023-12-19T16:59:58.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.319+0000 7fcd33b39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2c0cf0d0 con 0x7fcd2c06b3d0 2023-12-19T16:59:58.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.319+0000 7fcd33b39640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2c0cc6e0 con 0x7fcd2c0cec40 2023-12-19T16:59:58.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.319+0000 7fcd310d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd2c06b3d0 0x7fcd2c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:58.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.319+0000 7fcd310d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd2c06b3d0 0x7fcd2c0ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45704/0 (socket says 172.21.15.26:45704) 2023-12-19T16:59:58.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.319+0000 7fcd310d4640 1 -- 172.21.15.26:0/3953884656 learned_addr learned my addr 172.21.15.26:0/3953884656 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:58.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd320d6640 1 --2- 172.21.15.26:0/3953884656 >> v2:172.21.15.160:3300/0 conn(0x7fcd2c0cec40 0x7fcd2c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:58.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd310d4640 1 -- 172.21.15.26:0/3953884656 >> v2:172.21.15.160:3300/0 conn(0x7fcd2c0cec40 msgr2=0x7fcd2c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:58.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd310d4640 1 --2- 172.21.15.26:0/3953884656 >> v2:172.21.15.160:3300/0 conn(0x7fcd2c0cec40 0x7fcd2c15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd310d4640 1 -- 172.21.15.26:0/3953884656 >> v1:172.21.15.160:6789/0 conn(0x7fcd2c154220 legacy=0x7fcd2c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd310d4640 1 -- 172.21.15.26:0/3953884656 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c06b950 con 0x7fcd2c06b3d0 2023-12-19T16:59:58.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd310d4640 1 --2- 172.21.15.26:0/3953884656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2c06b3d0 0x7fcd2c0ce700 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c157bd0 tx=0x7fcd1c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320dafb3f8de9cac server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:58.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd308d3640 1 -- 172.21.15.26:0/3953884656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0aa430 con 0x7fcd2c06b3d0 2023-12-19T16:59:58.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd308d3640 1 -- 172.21.15.26:0/3953884656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1c0aa590 con 0x7fcd2c06b3d0 2023-12-19T16:59:58.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd308d3640 1 -- 172.21.15.26:0/3953884656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0aa920 con 0x7fcd2c06b3d0 2023-12-19T16:59:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd33b39640 1 -- 172.21.15.26:0/3953884656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2c06b3d0 msgr2=0x7fcd2c0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd33b39640 1 --2- 172.21.15.26:0/3953884656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2c06b3d0 0x7fcd2c0ce700 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c157bd0 tx=0x7fcd1c01c4d0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd33b39640 1 -- 172.21.15.26:0/3953884656 shutdown_connections 2023-12-19T16:59:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd33b39640 1 --2- 172.21.15.26:0/3953884656 >> v2:172.21.15.160:3300/0 conn(0x7fcd2c0cec40 0x7fcd2c15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd33b39640 1 --2- 172.21.15.26:0/3953884656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2c06b3d0 0x7fcd2c0ce700 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.323+0000 7fcd33b39640 1 -- 172.21.15.26:0/3953884656 >> 172.21.15.26:0/3953884656 conn(0x7fcd2c0658e0 msgr2=0x7fcd2c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd33b39640 1 -- 172.21.15.26:0/3953884656 shutdown_connections 2023-12-19T16:59:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd33b39640 1 -- 172.21.15.26:0/3953884656 wait complete. 2023-12-19T16:59:58.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd33b39640 1 Processor -- start 2023-12-19T16:59:58.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd33b39640 1 -- start start 2023-12-19T16:59:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd33b39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd2c06b3d0 0x7fcd2c0b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd33b39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd2c0cec40 0x7fcd2c152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd33b39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2c154220 0x7fcd2c14dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd33b39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c151000 con 0x7fcd2c154220 2023-12-19T16:59:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd33b39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c152550 con 0x7fcd2c0cec40 2023-12-19T16:59:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd33b39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c14fc10 con 0x7fcd2c06b3d0 2023-12-19T16:59:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd320d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2c154220 0x7fcd2c14dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd320d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2c154220 0x7fcd2c14dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45710/0 (socket says 172.21.15.26:45710) 2023-12-19T16:59:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd320d6640 1 -- 172.21.15.26:0/278401634 learned_addr learned my addr 172.21.15.26:0/278401634 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd318d5640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd2c06b3d0 0x7fcd2c0b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd310d4640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd2c0cec40 0x7fcd2c152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:58.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd320d6640 1 -- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd2c06b3d0 msgr2=0x7fcd2c0b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd320d6640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd2c06b3d0 0x7fcd2c0b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd320d6640 1 -- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd2c0cec40 msgr2=0x7fcd2c152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd320d6640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd2c0cec40 0x7fcd2c152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd320d6640 1 -- 172.21.15.26:0/278401634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c0094d0 con 0x7fcd2c154220 2023-12-19T16:59:58.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd320d6640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2c154220 0x7fcd2c14dce0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fcd24007e40 tx=0x7fcd2400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:58.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.327+0000 7fcd227fc640 1 -- 172.21.15.26:0/278401634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24014620 con 0x7fcd2c154220 2023-12-19T16:59:58.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.331+0000 7fcd227fc640 1 -- 172.21.15.26:0/278401634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2400ee90 con 0x7fcd2c154220 2023-12-19T16:59:58.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.331+0000 7fcd227fc640 1 -- 172.21.15.26:0/278401634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd240149e0 con 0x7fcd2c154220 2023-12-19T16:59:58.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.331+0000 7fcd33b39640 1 -- 172.21.15.26:0/278401634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2c151250 con 0x7fcd2c154220 2023-12-19T16:59:58.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.331+0000 7fcd33b39640 1 -- 172.21.15.26:0/278401634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2c151810 con 0x7fcd2c154220 2023-12-19T16:59:58.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.331+0000 7fcd33b39640 1 -- 172.21.15.26:0/278401634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2c074270 con 0x7fcd2c154220 2023-12-19T16:59:58.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.331+0000 7fcd227fc640 1 -- 172.21.15.26:0/278401634 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fcd24014b40 con 0x7fcd2c154220 2023-12-19T16:59:58.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.331+0000 7fcd227fc640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd10065b80 0x7fcd10068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:58.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.331+0000 7fcd227fc640 1 -- 172.21.15.26:0/278401634 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fcd24083aa0 con 0x7fcd2c154220 2023-12-19T16:59:58.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.331+0000 7fcd318d5640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd10065b80 0x7fcd10068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:58.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.335+0000 7fcd318d5640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd10065b80 0x7fcd10068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c152e00 tx=0x7fcd28009040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:58.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.335+0000 7fcd227fc640 1 -- 172.21.15.26:0/278401634 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcd24051460 con 0x7fcd2c154220 2023-12-19T16:59:58.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.547+0000 7fcd33b39640 1 -- 172.21.15.26:0/278401634 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcd2c0cdf60 con 0x7fcd2c154220 2023-12-19T16:59:58.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.547+0000 7fcd227fc640 1 -- 172.21.15.26:0/278401634 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fcd24054a80 con 0x7fcd2c154220 2023-12-19T16:59:58.550 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:59:58.550 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T16:59:58.550 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg==", 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg==", 2023-12-19T16:59:58.551 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg==", 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:59:58.552 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw==", 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.553 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig==", 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA==", 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:59:58.554 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ==", 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2023-12-19T16:59:58.555 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ==", 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A==", 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.556 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.557 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2023-12-19T16:59:58.557 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ==", 2023-12-19T16:59:58.557 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.557 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:59:58.557 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:59:58.557 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.557 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.557 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.557 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.557 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2023-12-19T16:59:58.557 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg==", 2023-12-19T16:59:58.557 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg==", 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.559 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2023-12-19T16:59:58.559 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ==", 2023-12-19T16:59:58.559 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.559 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:59:58.559 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2023-12-19T16:59:58.559 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T16:59:58.559 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.559 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2023-12-19T16:59:58.559 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ==", 2023-12-19T16:59:58.559 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ==", 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:59:58.560 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ==", 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2023-12-19T16:59:58.561 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlT5ynDxAA7Sjbdwrc1RyAXJboysB54Q==", 2023-12-19T16:59:58.562 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.562 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:59:58.562 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T16:59:58.562 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T16:59:58.562 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.562 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.562 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2023-12-19T16:59:58.562 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA==", 2023-12-19T16:59:58.562 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.562 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:59:58.562 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg==", 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-12-19T16:59:58.563 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T16:59:58.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.551+0000 7fcd33b39640 1 -- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd10065b80 msgr2=0x7fcd10068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.551+0000 7fcd33b39640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd10065b80 0x7fcd10068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c152e00 tx=0x7fcd28009040 comp rx=0 tx=0).stop 2023-12-19T16:59:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.551+0000 7fcd33b39640 1 -- 172.21.15.26:0/278401634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2c154220 msgr2=0x7fcd2c14dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.551+0000 7fcd33b39640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2c154220 0x7fcd2c14dce0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fcd24007e40 tx=0x7fcd2400e4b0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.551+0000 7fcd33b39640 1 -- 172.21.15.26:0/278401634 shutdown_connections 2023-12-19T16:59:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.551+0000 7fcd33b39640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd2c154220 0x7fcd2c14dce0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.551+0000 7fcd33b39640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd2c0cec40 0x7fcd2c152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.551+0000 7fcd33b39640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcd10065b80 0x7fcd10068040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.551+0000 7fcd33b39640 1 --2- 172.21.15.26:0/278401634 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd2c06b3d0 0x7fcd2c0b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.551+0000 7fcd33b39640 1 -- 172.21.15.26:0/278401634 >> 172.21.15.26:0/278401634 conn(0x7fcd2c0658e0 msgr2=0x7fcd2c0cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.551+0000 7fcd33b39640 1 -- 172.21.15.26:0/278401634 shutdown_connections 2023-12-19T16:59:58.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.551+0000 7fcd33b39640 1 -- 172.21.15.26:0/278401634 wait complete. 2023-12-19T16:59:58.571 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T16:59:58.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb7266640 1 Processor -- start 2023-12-19T16:59:58.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb7266640 1 -- start start 2023-12-19T16:59:58.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb7266640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9bb006b3d0 0x7f9bb00ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:58.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb7266640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9bb00cec40 0x7f9bb015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:58.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb7266640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb006bc30 con 0x7f9bb0154220 2023-12-19T16:59:58.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb7266640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb00cf0d0 con 0x7f9bb006b3d0 2023-12-19T16:59:58.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb7266640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb00cc6e0 con 0x7f9bb00cec40 2023-12-19T16:59:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb4801640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9bb006b3d0 0x7f9bb00ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb5002640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9bb0154220 0x7f9bb015aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46804/0 (socket says 172.21.15.26:46804) 2023-12-19T16:59:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb5002640 1 -- 172.21.15.26:0/4056787377 learned_addr learned my addr 172.21.15.26:0/4056787377 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb5803640 1 --2- 172.21.15.26:0/4056787377 >> v2:172.21.15.160:3300/0 conn(0x7f9bb00cec40 0x7f9bb015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb4801640 1 -- 172.21.15.26:0/4056787377 >> v2:172.21.15.160:3300/0 conn(0x7f9bb00cec40 msgr2=0x7f9bb015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T16:59:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb4801640 1 --2- 172.21.15.26:0/4056787377 >> v2:172.21.15.160:3300/0 conn(0x7f9bb00cec40 0x7f9bb015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb4801640 1 -- 172.21.15.26:0/4056787377 >> v1:172.21.15.26:6789/0 conn(0x7f9bb0154220 legacy=0x7f9bb015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9bb4801640 1 -- 172.21.15.26:0/4056787377 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb006b950 con 0x7f9bb006b3d0 2023-12-19T16:59:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.715+0000 7f9ba7fff640 1 -- 172.21.15.26:0/4056787377 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316787826 0 0) 0x7f9bb006bc30 con 0x7f9bb0154220 2023-12-19T16:59:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.719+0000 7f9bb4801640 1 --2- 172.21.15.26:0/4056787377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb006b3d0 0x7f9bb00ce700 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0157bd0 tx=0x7f9ba001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91e09db61bdfd9e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.719+0000 7f9ba7fff640 1 -- 172.21.15.26:0/4056787377 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba00a7070 con 0x7f9bb006b3d0 2023-12-19T16:59:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.719+0000 7f9ba7fff640 1 -- 172.21.15.26:0/4056787377 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba00ab430 con 0x7f9bb006b3d0 2023-12-19T16:59:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.719+0000 7f9ba7fff640 1 -- 172.21.15.26:0/4056787377 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba00ab7c0 con 0x7f9bb006b3d0 2023-12-19T16:59:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.719+0000 7f9bb7266640 1 -- 172.21.15.26:0/4056787377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb006b3d0 msgr2=0x7f9bb00ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.719+0000 7f9bb7266640 1 --2- 172.21.15.26:0/4056787377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb006b3d0 0x7f9bb00ce700 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0157bd0 tx=0x7f9ba001c430 comp rx=0 tx=0).stop 2023-12-19T16:59:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.719+0000 7f9bb7266640 1 -- 172.21.15.26:0/4056787377 shutdown_connections 2023-12-19T16:59:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.719+0000 7f9bb7266640 1 --2- 172.21.15.26:0/4056787377 >> v2:172.21.15.160:3300/0 conn(0x7f9bb00cec40 0x7f9bb015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.719+0000 7f9bb7266640 1 --2- 172.21.15.26:0/4056787377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb006b3d0 0x7f9bb00ce700 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.719+0000 7f9bb7266640 1 -- 172.21.15.26:0/4056787377 >> 172.21.15.26:0/4056787377 conn(0x7f9bb00658e0 msgr2=0x7f9bb0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.719+0000 7f9bb7266640 1 -- 172.21.15.26:0/4056787377 shutdown_connections 2023-12-19T16:59:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.719+0000 7f9bb7266640 1 -- 172.21.15.26:0/4056787377 wait complete. 2023-12-19T16:59:58.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb7266640 1 Processor -- start 2023-12-19T16:59:58.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb7266640 1 -- start start 2023-12-19T16:59:58.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb7266640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bb006b3d0 0x7f9bb0139ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:58.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb7266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9bb00cec40 0x7f9bb015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:58.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb7266640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0154220 0x7f9bb015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:58.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb7266640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb013aaa0 con 0x7f9bb0154220 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb7266640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb013ad80 con 0x7f9bb006b3d0 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb7266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb013b060 con 0x7f9bb00cec40 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb5803640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0154220 0x7f9bb015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb5803640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0154220 0x7f9bb015cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45730/0 (socket says 172.21.15.26:45730) 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb5803640 1 -- 172.21.15.26:0/1687044847 learned_addr learned my addr 172.21.15.26:0/1687044847 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb5002640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bb006b3d0 0x7f9bb0139ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb4801640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9bb00cec40 0x7f9bb015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb5803640 1 -- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9bb00cec40 msgr2=0x7f9bb015c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb5803640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9bb00cec40 0x7f9bb015c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb5803640 1 -- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bb006b3d0 msgr2=0x7f9bb0139ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb5803640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bb006b3d0 0x7f9bb0139ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb5803640 1 -- 172.21.15.26:0/1687044847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba0009420 con 0x7f9bb0154220 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb5002640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bb006b3d0 0x7f9bb0139ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:59:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb4801640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9bb00cec40 0x7f9bb015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T16:59:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb5803640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0154220 0x7f9bb015cb30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800efc0 tx=0x7f9ba800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9ba5ffb640 1 -- 172.21.15.26:0/1687044847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba800ced0 con 0x7f9bb0154220 2023-12-19T16:59:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9ba5ffb640 1 -- 172.21.15.26:0/1687044847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba800f040 con 0x7f9bb0154220 2023-12-19T16:59:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9ba5ffb640 1 -- 172.21.15.26:0/1687044847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba80146c0 con 0x7f9bb0154220 2023-12-19T16:59:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb7266640 1 -- 172.21.15.26:0/1687044847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb013b340 con 0x7f9bb0154220 2023-12-19T16:59:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb7266640 1 -- 172.21.15.26:0/1687044847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb006f370 con 0x7f9bb0154220 2023-12-19T16:59:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.723+0000 7f9bb7266640 1 -- 172.21.15.26:0/1687044847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb0074270 con 0x7f9bb0154220 2023-12-19T16:59:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.727+0000 7f9ba5ffb640 1 -- 172.21.15.26:0/1687044847 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9ba8014820 con 0x7f9bb0154220 2023-12-19T16:59:58.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.727+0000 7f9ba5ffb640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9b94065bb0 0x7f9b94068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:58.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.727+0000 7f9ba5ffb640 1 -- 172.21.15.26:0/1687044847 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f9ba8083f20 con 0x7f9bb0154220 2023-12-19T16:59:58.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.727+0000 7f9bb5002640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9b94065bb0 0x7f9b94068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:58.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.727+0000 7f9bb5002640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9b94065bb0 0x7f9b94068070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9bac000f70 tx=0x7f9bac009040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:58.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.731+0000 7f9ba5ffb640 1 -- 172.21.15.26:0/1687044847 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9ba8051720 con 0x7f9bb0154220 2023-12-19T16:59:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.891+0000 7f9bb7266640 1 -- 172.21.15.26:0/1687044847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bb013abe0 con 0x7f9bb0154220 2023-12-19T16:59:58.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.891+0000 7f9ba5ffb640 1 -- 172.21.15.26:0/1687044847 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+13504 (secure 0 0 0) 0x7f9bb013abe0 con 0x7f9bb0154220 2023-12-19T16:59:58.893 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:59:58.893 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":119,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T16:59:58.144023+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T16:59:58.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.891+0000 7f9bb7266640 1 -- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9b94065bb0 msgr2=0x7f9b94068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.891+0000 7f9bb7266640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9b94065bb0 0x7f9b94068070 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9bac000f70 tx=0x7f9bac009040 comp rx=0 tx=0).stop 2023-12-19T16:59:58.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.891+0000 7f9bb7266640 1 -- 172.21.15.26:0/1687044847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0154220 msgr2=0x7f9bb015cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:58.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.891+0000 7f9bb7266640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0154220 0x7f9bb015cb30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800efc0 tx=0x7f9ba800c490 comp rx=0 tx=0).stop 2023-12-19T16:59:58.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.891+0000 7f9bb7266640 1 -- 172.21.15.26:0/1687044847 shutdown_connections 2023-12-19T16:59:58.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.891+0000 7f9bb7266640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb0154220 0x7f9bb015cb30 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.891+0000 7f9bb7266640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9b94065bb0 0x7f9b94068070 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.891+0000 7f9bb7266640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9bb00cec40 0x7f9bb015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.891+0000 7f9bb7266640 1 --2- 172.21.15.26:0/1687044847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9bb006b3d0 0x7f9bb0139ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:58.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.891+0000 7f9bb7266640 1 -- 172.21.15.26:0/1687044847 >> 172.21.15.26:0/1687044847 conn(0x7f9bb00658e0 msgr2=0x7f9bb00cbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:58.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.895+0000 7f9bb7266640 1 -- 172.21.15.26:0/1687044847 shutdown_connections 2023-12-19T16:59:58.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:58.895+0000 7f9bb7266640 1 -- 172.21.15.26:0/1687044847 wait complete. 2023-12-19T16:59:58.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T16:59:58.915 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T16:59:59.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.059+0000 7fd8e698f640 1 Processor -- start 2023-12-19T16:59:59.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.059+0000 7fd8e698f640 1 -- start start 2023-12-19T16:59:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.063+0000 7fd8e698f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd8e00ccf10 0x7fd8e00ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.063+0000 7fd8e698f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e00ac860 con 0x7fd8e00b08c0 2023-12-19T16:59:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.063+0000 7fd8e698f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e00ad7f0 con 0x7fd8e015d250 2023-12-19T16:59:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.063+0000 7fd8e698f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e0157730 con 0x7fd8e00ccf10 2023-12-19T16:59:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.063+0000 7fd8dffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd8e00ccf10 0x7fd8e00ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.063+0000 7fd8e4f2c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd8e00b08c0 0x7fd8e00c8cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46818/0 (socket says 172.21.15.26:46818) 2023-12-19T16:59:59.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.063+0000 7fd8e4f2c640 1 -- 172.21.15.26:0/1036944119 learned_addr learned my addr 172.21.15.26:0/1036944119 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:59.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.063+0000 7fd8dffff640 1 -- 172.21.15.26:0/1036944119 >> v1:172.21.15.160:6789/0 conn(0x7fd8e015d250 legacy=0x7fd8e00b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.063+0000 7fd8dffff640 1 -- 172.21.15.26:0/1036944119 >> v1:172.21.15.26:6789/0 conn(0x7fd8e00b08c0 legacy=0x7fd8e00c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.063+0000 7fd8dffff640 1 -- 172.21.15.26:0/1036944119 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e015dd10 con 0x7fd8e00ccf10 2023-12-19T16:59:59.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.063+0000 7fd8deffd640 1 -- 172.21.15.26:0/1036944119 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254622066 0 0) 0x7fd8e00ac860 con 0x7fd8e00b08c0 2023-12-19T16:59:59.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8dffff640 1 --2- 172.21.15.26:0/1036944119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e00ccf10 0x7fd8e00ad2b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00097c0 tx=0x7fd8d001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb05002c1eb08cd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8deffd640 1 -- 172.21.15.26:0/1036944119 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00a8970 con 0x7fd8e00ccf10 2023-12-19T16:59:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8deffd640 1 -- 172.21.15.26:0/1036944119 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d00a8ad0 con 0x7fd8e00ccf10 2023-12-19T16:59:59.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8deffd640 1 -- 172.21.15.26:0/1036944119 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00a8e60 con 0x7fd8e00ccf10 2023-12-19T16:59:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8e698f640 1 -- 172.21.15.26:0/1036944119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e00ccf10 msgr2=0x7fd8e00ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8e698f640 1 --2- 172.21.15.26:0/1036944119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e00ccf10 0x7fd8e00ad2b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00097c0 tx=0x7fd8d001ec70 comp rx=0 tx=0).stop 2023-12-19T16:59:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8e698f640 1 -- 172.21.15.26:0/1036944119 shutdown_connections 2023-12-19T16:59:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8e698f640 1 --2- 172.21.15.26:0/1036944119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e00ccf10 0x7fd8e00ad2b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8e698f640 1 -- 172.21.15.26:0/1036944119 >> 172.21.15.26:0/1036944119 conn(0x7fd8e00ce730 msgr2=0x7fd8e00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8e698f640 1 -- 172.21.15.26:0/1036944119 shutdown_connections 2023-12-19T16:59:59.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8e698f640 1 -- 172.21.15.26:0/1036944119 wait complete. 2023-12-19T16:59:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8e698f640 1 Processor -- start 2023-12-19T16:59:59.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8e698f640 1 -- start start 2023-12-19T16:59:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.067+0000 7fd8e698f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8e00b08c0 0x7fd8e0156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8e698f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e00ccf10 0x7fd8e00c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8e698f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8e015d250 0x7fd8e00c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8e698f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00c5a90 con 0x7fd8e00ccf10 2023-12-19T16:59:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8e698f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00c2db0 con 0x7fd8e015d250 2023-12-19T16:59:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8e698f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e00c3090 con 0x7fd8e00b08c0 2023-12-19T16:59:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8df7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e00ccf10 0x7fd8e00c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8df7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e00ccf10 0x7fd8e00c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45752/0 (socket says 172.21.15.26:45752) 2023-12-19T16:59:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8df7fe640 1 -- 172.21.15.26:0/2216469289 learned_addr learned my addr 172.21.15.26:0/2216469289 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8e4f2c640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8e015d250 0x7fd8e00c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8dffff640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8e00b08c0 0x7fd8e0156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8df7fe640 1 -- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8e00b08c0 msgr2=0x7fd8e0156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8df7fe640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8e00b08c0 0x7fd8e0156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8df7fe640 1 -- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8e015d250 msgr2=0x7fd8e00c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8df7fe640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8e015d250 0x7fd8e00c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8df7fe640 1 -- 172.21.15.26:0/2216469289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d0009420 con 0x7fd8e00ccf10 2023-12-19T16:59:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8df7fe640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e00ccf10 0x7fd8e00c6fe0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400c3c0 tx=0x7fd8d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8dcff9640 1 -- 172.21.15.26:0/2216469289 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d40052d0 con 0x7fd8e00ccf10 2023-12-19T16:59:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8dcff9640 1 -- 172.21.15.26:0/2216469289 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d4005430 con 0x7fd8e00ccf10 2023-12-19T16:59:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8e698f640 1 -- 172.21.15.26:0/2216469289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e00c3370 con 0x7fd8e00ccf10 2023-12-19T16:59:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8dcff9640 1 -- 172.21.15.26:0/2216469289 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4003840 con 0x7fd8e00ccf10 2023-12-19T16:59:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8e698f640 1 -- 172.21.15.26:0/2216469289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e00c3930 con 0x7fd8e00ccf10 2023-12-19T16:59:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8e698f640 1 -- 172.21.15.26:0/2216469289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e00c3b50 con 0x7fd8e00ccf10 2023-12-19T16:59:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.071+0000 7fd8dcff9640 1 -- 172.21.15.26:0/2216469289 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd8d40039a0 con 0x7fd8e00ccf10 2023-12-19T16:59:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.075+0000 7fd8dcff9640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd8cc065b80 0x7fd8cc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.075+0000 7fd8dcff9640 1 -- 172.21.15.26:0/2216469289 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fd8d4082720 con 0x7fd8e00ccf10 2023-12-19T16:59:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.075+0000 7fd8dffff640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd8cc065b80 0x7fd8cc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.075+0000 7fd8dffff640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd8cc065b80 0x7fd8cc068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00b2000 tx=0x7fd8d0008040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.079+0000 7fd8dcff9640 1 -- 172.21.15.26:0/2216469289 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd8d40500e0 con 0x7fd8e00ccf10 2023-12-19T16:59:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.231+0000 7fd8e698f640 1 -- 172.21.15.26:0/2216469289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8e0074270 con 0x7fd8e00ccf10 2023-12-19T16:59:59.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.235+0000 7fd8dcff9640 1 -- 172.21.15.26:0/2216469289 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8d4053700 con 0x7fd8e00ccf10 2023-12-19T16:59:59.235 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T16:59:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.235+0000 7fd8e698f640 1 -- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd8cc065b80 msgr2=0x7fd8cc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.235+0000 7fd8e698f640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd8cc065b80 0x7fd8cc068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00b2000 tx=0x7fd8d0008040 comp rx=0 tx=0).stop 2023-12-19T16:59:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.235+0000 7fd8e698f640 1 -- 172.21.15.26:0/2216469289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e00ccf10 msgr2=0x7fd8e00c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.235+0000 7fd8e698f640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e00ccf10 0x7fd8e00c6fe0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd8d400c3c0 tx=0x7fd8d400c880 comp rx=0 tx=0).stop 2023-12-19T16:59:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.235+0000 7fd8e698f640 1 -- 172.21.15.26:0/2216469289 shutdown_connections 2023-12-19T16:59:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.235+0000 7fd8e698f640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8e015d250 0x7fd8e00c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.235+0000 7fd8e698f640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8e00ccf10 0x7fd8e00c6fe0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.235+0000 7fd8e698f640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd8cc065b80 0x7fd8cc068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.235+0000 7fd8e698f640 1 --2- 172.21.15.26:0/2216469289 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8e00b08c0 0x7fd8e0156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.235+0000 7fd8e698f640 1 -- 172.21.15.26:0/2216469289 >> 172.21.15.26:0/2216469289 conn(0x7fd8e00ce730 msgr2=0x7fd8e00ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.235+0000 7fd8e698f640 1 -- 172.21.15.26:0/2216469289 shutdown_connections 2023-12-19T16:59:59.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.235+0000 7fd8e698f640 1 -- 172.21.15.26:0/2216469289 wait complete. 2023-12-19T16:59:59.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T16:59:59.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa1a42a640 1 Processor -- start 2023-12-19T16:59:59.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa1a42a640 1 -- start start 2023-12-19T16:59:59.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa1a42a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faa140cdd80 0x7faa141676b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa1a42a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa140b6cf0 con 0x7faa1415cca0 2023-12-19T16:59:59.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa1a42a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa140b6fb0 con 0x7faa140b8760 2023-12-19T16:59:59.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa1a42a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa140b7c30 con 0x7faa140cdd80 2023-12-19T16:59:59.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa19428640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faa1415cca0 0x7faa140b65e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46822/0 (socket says 172.21.15.26:46822) 2023-12-19T16:59:59.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa19428640 1 -- 172.21.15.26:0/3003781831 learned_addr learned my addr 172.21.15.26:0/3003781831 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:59.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa19c29640 1 --2- 172.21.15.26:0/3003781831 >> v2:172.21.15.160:3300/0 conn(0x7faa140cdd80 0x7faa141676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:59.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa0bfff640 1 -- 172.21.15.26:0/3003781831 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491910582 0 0) 0x7faa140b6cf0 con 0x7faa1415cca0 2023-12-19T16:59:59.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa0bfff640 1 -- 172.21.15.26:0/3003781831 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa00003530 con 0x7faa1415cca0 2023-12-19T16:59:59.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa0bfff640 1 -- 172.21.15.26:0/3003781831 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 204768125 0 0) 0x7faa00003530 con 0x7faa1415cca0 2023-12-19T16:59:59.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa19c29640 1 -- 172.21.15.26:0/3003781831 >> v1:172.21.15.160:6789/0 conn(0x7faa140b8760 legacy=0x7faa140cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa19c29640 1 -- 172.21.15.26:0/3003781831 >> v1:172.21.15.26:6789/0 conn(0x7faa1415cca0 legacy=0x7faa140b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.399+0000 7faa19c29640 1 -- 172.21.15.26:0/3003781831 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa140b7290 con 0x7faa140cdd80 2023-12-19T16:59:59.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.403+0000 7faa19c29640 1 --2- 172.21.15.26:0/3003781831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa140cdd80 0x7faa141676b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009d90 tx=0x7faa0c01c6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90d21875df41d434 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:59.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.403+0000 7faa0bfff640 1 -- 172.21.15.26:0/3003781831 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10001680 con 0x7faa140cdd80 2023-12-19T16:59:59.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.403+0000 7faa0bfff640 1 -- 172.21.15.26:0/3003781831 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c0a6500 con 0x7faa140cdd80 2023-12-19T16:59:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.403+0000 7faa0bfff640 1 -- 172.21.15.26:0/3003781831 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0a6890 con 0x7faa140cdd80 2023-12-19T16:59:59.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.403+0000 7faa1a42a640 1 -- 172.21.15.26:0/3003781831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa140cdd80 msgr2=0x7faa141676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.403+0000 7faa1a42a640 1 --2- 172.21.15.26:0/3003781831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa140cdd80 0x7faa141676b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009d90 tx=0x7faa0c01c6f0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.403+0000 7faa1a42a640 1 -- 172.21.15.26:0/3003781831 shutdown_connections 2023-12-19T16:59:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.403+0000 7faa1a42a640 1 --2- 172.21.15.26:0/3003781831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa140cdd80 0x7faa141676b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.403+0000 7faa1a42a640 1 -- 172.21.15.26:0/3003781831 >> 172.21.15.26:0/3003781831 conn(0x7faa1406a120 msgr2=0x7faa1406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.403+0000 7faa1a42a640 1 -- 172.21.15.26:0/3003781831 shutdown_connections 2023-12-19T16:59:59.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.403+0000 7faa1a42a640 1 -- 172.21.15.26:0/3003781831 wait complete. 2023-12-19T16:59:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa1a42a640 1 Processor -- start 2023-12-19T16:59:59.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa1a42a640 1 -- start start 2023-12-19T16:59:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa1a42a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa140b8760 0x7faa1415e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa1a42a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa140cdd80 0x7faa141580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa1a42a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa1415cca0 0x7faa140b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa1a42a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1415d850 con 0x7faa140cdd80 2023-12-19T16:59:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa1a42a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1415db30 con 0x7faa140b8760 2023-12-19T16:59:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa1a42a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1415de10 con 0x7faa1415cca0 2023-12-19T16:59:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa18c27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa140cdd80 0x7faa141580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa18c27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa140cdd80 0x7faa141580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45760/0 (socket says 172.21.15.26:45760) 2023-12-19T16:59:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa18c27640 1 -- 172.21.15.26:0/4138425611 learned_addr learned my addr 172.21.15.26:0/4138425611 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa18c27640 1 -- 172.21.15.26:0/4138425611 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa1415cca0 msgr2=0x7faa140b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa18c27640 1 --2- 172.21.15.26:0/4138425611 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa1415cca0 0x7faa140b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa18c27640 1 -- 172.21.15.26:0/4138425611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa140b8760 msgr2=0x7faa1415e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa18c27640 1 --2- 172.21.15.26:0/4138425611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa140b8760 0x7faa1415e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa18c27640 1 -- 172.21.15.26:0/4138425611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c009a70 con 0x7faa140cdd80 2023-12-19T16:59:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa18c27640 1 --2- 172.21.15.26:0/4138425611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa140cdd80 0x7faa141580e0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7faa0400c780 tx=0x7faa0400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa09ffb640 1 -- 172.21.15.26:0/4138425611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04005270 con 0x7faa140cdd80 2023-12-19T16:59:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa09ffb640 1 -- 172.21.15.26:0/4138425611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa040053d0 con 0x7faa140cdd80 2023-12-19T16:59:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa09ffb640 1 -- 172.21.15.26:0/4138425611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0400a880 con 0x7faa140cdd80 2023-12-19T16:59:59.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa1a42a640 1 -- 172.21.15.26:0/4138425611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1415e0f0 con 0x7faa140cdd80 2023-12-19T16:59:59.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa1a42a640 1 -- 172.21.15.26:0/4138425611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1415ee30 con 0x7faa140cdd80 2023-12-19T16:59:59.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.407+0000 7faa1a42a640 1 -- 172.21.15.26:0/4138425611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1415e610 con 0x7faa140cdd80 2023-12-19T16:59:59.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.411+0000 7faa09ffb640 1 -- 172.21.15.26:0/4138425611 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faa0400a9e0 con 0x7faa140cdd80 2023-12-19T16:59:59.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.411+0000 7faa09ffb640 1 --2- 172.21.15.26:0/4138425611 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa9f8065b80 0x7fa9f8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.411+0000 7faa09ffb640 1 -- 172.21.15.26:0/4138425611 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5589+0+0 (secure 0 0 0) 0x7faa040829f0 con 0x7faa140cdd80 2023-12-19T16:59:59.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.411+0000 7faa19428640 1 --2- 172.21.15.26:0/4138425611 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa9f8065b80 0x7fa9f8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:59.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.411+0000 7faa19428640 1 --2- 172.21.15.26:0/4138425611 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa9f8065b80 0x7fa9f8068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faa10000c10 tx=0x7faa10001a10 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:59.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.415+0000 7faa09ffb640 1 -- 172.21.15.26:0/4138425611 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faa1415e610 con 0x7faa140cdd80 2023-12-19T16:59:59.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.571+0000 7faa1a42a640 1 -- 172.21.15.26:0/4138425611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7faa1415e610 con 0x7faa140cdd80 2023-12-19T16:59:59.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.575+0000 7faa09ffb640 1 -- 172.21.15.26:0/4138425611 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v119) v1 ==== 84+0+65 (secure 0 0 0) 0x7faa1415e610 con 0x7faa140cdd80 2023-12-19T16:59:59.576 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T16:59:59.576 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T16:59:59.576 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T16:59:59.576 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T16:59:59.576 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T16:59:59.576 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T16:59:59.576 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T16:59:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.575+0000 7faa1a42a640 1 -- 172.21.15.26:0/4138425611 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa9f8065b80 msgr2=0x7fa9f8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.575+0000 7faa1a42a640 1 --2- 172.21.15.26:0/4138425611 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa9f8065b80 0x7fa9f8068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faa10000c10 tx=0x7faa10001a10 comp rx=0 tx=0).stop 2023-12-19T16:59:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.575+0000 7faa1a42a640 1 -- 172.21.15.26:0/4138425611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa140cdd80 msgr2=0x7faa141580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.575+0000 7faa1a42a640 1 --2- 172.21.15.26:0/4138425611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa140cdd80 0x7faa141580e0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7faa0400c780 tx=0x7faa0400cc40 comp rx=0 tx=0).stop 2023-12-19T16:59:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.575+0000 7faa1a42a640 1 -- 172.21.15.26:0/4138425611 shutdown_connections 2023-12-19T16:59:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.575+0000 7faa1a42a640 1 --2- 172.21.15.26:0/4138425611 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa9f8065b80 0x7fa9f8068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.575+0000 7faa1a42a640 1 --2- 172.21.15.26:0/4138425611 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa1415cca0 0x7faa140b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.575+0000 7faa1a42a640 1 --2- 172.21.15.26:0/4138425611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa140cdd80 0x7faa141580e0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.575+0000 7faa1a42a640 1 --2- 172.21.15.26:0/4138425611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa140b8760 0x7faa1415e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.575+0000 7faa1a42a640 1 -- 172.21.15.26:0/4138425611 >> 172.21.15.26:0/4138425611 conn(0x7faa1406a120 msgr2=0x7faa140b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.575+0000 7faa1a42a640 1 -- 172.21.15.26:0/4138425611 shutdown_connections 2023-12-19T16:59:59.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.575+0000 7faa1a42a640 1 -- 172.21.15.26:0/4138425611 wait complete. 2023-12-19T16:59:59.596 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T16:59:59.597 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T16:59:59.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.743+0000 7f939b59e640 1 Processor -- start 2023-12-19T16:59:59.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.743+0000 7f939b59e640 1 -- start start 2023-12-19T16:59:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.743+0000 7f939b59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f939c06bac0 0x7f939c0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.743+0000 7f939b59e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939c158050 con 0x7f939c153f20 2023-12-19T16:59:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.743+0000 7f939b59e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f939c15a2b0 con 0x7f939c157ba0 2023-12-19T16:59:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.743+0000 7f939b59e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939c0b0600 con 0x7f939c06bac0 2023-12-19T16:59:59.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.743+0000 7f939ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f939c06bac0 0x7f939c0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.743+0000 7f939a59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f939c153f20 0x7f939c0b3270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46826/0 (socket says 172.21.15.26:46826) 2023-12-19T16:59:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.743+0000 7f939a59c640 1 -- 172.21.15.26:0/419973575 learned_addr learned my addr 172.21.15.26:0/419973575 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T16:59:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.743+0000 7f939ad9d640 1 -- 172.21.15.26:0/419973575 >> v1:172.21.15.160:6789/0 conn(0x7f939c157ba0 legacy=0x7f939c06b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.743+0000 7f939ad9d640 1 -- 172.21.15.26:0/419973575 >> v1:172.21.15.26:6789/0 conn(0x7f939c153f20 legacy=0x7f939c0b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.743+0000 7f939ad9d640 1 -- 172.21.15.26:0/419973575 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c15a590 con 0x7f939c06bac0 2023-12-19T16:59:59.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.747+0000 7f939ad9d640 1 --2- 172.21.15.26:0/419973575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c06bac0 0x7f939c0ae5a0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9394009d90 tx=0x7f939401eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f56e60226d33d07 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:59.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.747+0000 7f9399d9b640 1 -- 172.21.15.26:0/419973575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940a7b70 con 0x7f939c06bac0 2023-12-19T16:59:59.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.747+0000 7f9399d9b640 1 -- 172.21.15.26:0/419973575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93940a7cd0 con 0x7f939c06bac0 2023-12-19T16:59:59.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.747+0000 7f9399d9b640 1 -- 172.21.15.26:0/419973575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940af660 con 0x7f939c06bac0 2023-12-19T16:59:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.747+0000 7f939b59e640 1 -- 172.21.15.26:0/419973575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c06bac0 msgr2=0x7f939c0ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.747+0000 7f939b59e640 1 --2- 172.21.15.26:0/419973575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c06bac0 0x7f939c0ae5a0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f9394009d90 tx=0x7f939401eaa0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.747+0000 7f939b59e640 1 -- 172.21.15.26:0/419973575 shutdown_connections 2023-12-19T16:59:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.747+0000 7f939b59e640 1 --2- 172.21.15.26:0/419973575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c06bac0 0x7f939c0ae5a0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.747+0000 7f939b59e640 1 -- 172.21.15.26:0/419973575 >> 172.21.15.26:0/419973575 conn(0x7f939c0658e0 msgr2=0x7f939c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T16:59:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.747+0000 7f939b59e640 1 -- 172.21.15.26:0/419973575 shutdown_connections 2023-12-19T16:59:59.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.747+0000 7f939b59e640 1 -- 172.21.15.26:0/419973575 wait complete. 2023-12-19T16:59:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.747+0000 7f939b59e640 1 Processor -- start 2023-12-19T16:59:59.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939b59e640 1 -- start start 2023-12-19T16:59:59.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939b59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f939c06bac0 0x7f939c15c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939b59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f939c153f20 0x7f939c13b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939b59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c157ba0 0x7f939c152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939b59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c06fb80 con 0x7f939c157ba0 2023-12-19T16:59:59.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939b59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939c14e0b0 con 0x7f939c06bac0 2023-12-19T16:59:59.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939b59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939c14e390 con 0x7f939c153f20 2023-12-19T16:59:59.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c157ba0 0x7f939c152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:59.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c157ba0 0x7f939c152a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45774/0 (socket says 172.21.15.26:45774) 2023-12-19T16:59:59.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939a59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f939c06bac0 0x7f939c15c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:59.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939ad9d640 1 -- 172.21.15.26:0/24030523 learned_addr learned my addr 172.21.15.26:0/24030523 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T16:59:59.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f9393fff640 1 --2- 172.21.15.26:0/24030523 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f939c153f20 0x7f939c13b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939ad9d640 1 -- 172.21.15.26:0/24030523 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f939c153f20 msgr2=0x7f939c13b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939ad9d640 1 --2- 172.21.15.26:0/24030523 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f939c153f20 0x7f939c13b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939ad9d640 1 -- 172.21.15.26:0/24030523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f939c06bac0 msgr2=0x7f939c15c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T16:59:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939ad9d640 1 --2- 172.21.15.26:0/24030523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f939c06bac0 0x7f939c15c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T16:59:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939ad9d640 1 -- 172.21.15.26:0/24030523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9394009a70 con 0x7f939c157ba0 2023-12-19T16:59:59.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939ad9d640 1 --2- 172.21.15.26:0/24030523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c157ba0 0x7f939c152a60 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f939401c6d0 tx=0x7f939401b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f9392ffd640 1 -- 172.21.15.26:0/24030523 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940afe00 con 0x7f939c157ba0 2023-12-19T16:59:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f9392ffd640 1 -- 172.21.15.26:0/24030523 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939400e140 con 0x7f939c157ba0 2023-12-19T16:59:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939b59e640 1 -- 172.21.15.26:0/24030523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939c14e670 con 0x7f939c157ba0 2023-12-19T16:59:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f9392ffd640 1 -- 172.21.15.26:0/24030523 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940b8640 con 0x7f939c157ba0 2023-12-19T16:59:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939b59e640 1 -- 172.21.15.26:0/24030523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939c06f7a0 con 0x7f939c157ba0 2023-12-19T16:59:59.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.751+0000 7f939b59e640 1 -- 172.21.15.26:0/24030523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939c159f00 con 0x7f939c157ba0 2023-12-19T16:59:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.755+0000 7f9392ffd640 1 -- 172.21.15.26:0/24030523 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f93940b6070 con 0x7f939c157ba0 2023-12-19T16:59:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.755+0000 7f9392ffd640 1 --2- 172.21.15.26:0/24030523 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f937c065b80 0x7f937c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T16:59:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.755+0000 7f9392ffd640 1 -- 172.21.15.26:0/24030523 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f93940a7e40 con 0x7f939c157ba0 2023-12-19T16:59:59.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.755+0000 7f939a59c640 1 --2- 172.21.15.26:0/24030523 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f937c065b80 0x7f937c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T16:59:59.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.755+0000 7f939a59c640 1 --2- 172.21.15.26:0/24030523 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f937c065b80 0x7f937c068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f938c005890 tx=0x7f938c009040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T16:59:59.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.759+0000 7f9392ffd640 1 -- 172.21.15.26:0/24030523 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f939c159f00 con 0x7f939c157ba0 2023-12-19T16:59:59.825 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:59:59.830 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T16:59:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T16:59:59.915+0000 7f939b59e640 1 -- 172.21.15.26:0/24030523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f939c159f00 con 0x7f939c157ba0 2023-12-19T17:00:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.019+0000 7f9392ffd640 1 -- 172.21.15.26:0/24030523 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v120) v1 ==== 138+0+0 (secure 0 0 0) 0x7f939c159f00 con 0x7f939c157ba0 2023-12-19T17:00:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T17:00:00.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.023+0000 7f939b59e640 1 -- 172.21.15.26:0/24030523 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f937c065b80 msgr2=0x7f937c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:00.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.023+0000 7f939b59e640 1 --2- 172.21.15.26:0/24030523 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f937c065b80 0x7f937c068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f938c005890 tx=0x7f938c009040 comp rx=0 tx=0).stop 2023-12-19T17:00:00.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.023+0000 7f939b59e640 1 -- 172.21.15.26:0/24030523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c157ba0 msgr2=0x7f939c152a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:00.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.023+0000 7f939b59e640 1 --2- 172.21.15.26:0/24030523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c157ba0 0x7f939c152a60 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f939401c6d0 tx=0x7f939401b840 comp rx=0 tx=0).stop 2023-12-19T17:00:00.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.023+0000 7f939b59e640 1 -- 172.21.15.26:0/24030523 shutdown_connections 2023-12-19T17:00:00.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.023+0000 7f939b59e640 1 --2- 172.21.15.26:0/24030523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f939c157ba0 0x7f939c152a60 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:00.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.023+0000 7f939b59e640 1 --2- 172.21.15.26:0/24030523 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f937c065b80 0x7f937c068040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:00.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.023+0000 7f939b59e640 1 --2- 172.21.15.26:0/24030523 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f939c153f20 0x7f939c13b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:00.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.023+0000 7f939b59e640 1 --2- 172.21.15.26:0/24030523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f939c06bac0 0x7f939c15c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:00.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.023+0000 7f939b59e640 1 -- 172.21.15.26:0/24030523 >> 172.21.15.26:0/24030523 conn(0x7f939c0658e0 msgr2=0x7f939c065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.023+0000 7f939b59e640 1 -- 172.21.15.26:0/24030523 shutdown_connections 2023-12-19T17:00:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.023+0000 7f939b59e640 1 -- 172.21.15.26:0/24030523 wait complete. 2023-12-19T17:00:00.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T17:00:00.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.195+0000 7fa034ad1640 1 Processor -- start 2023-12-19T17:00:00.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.195+0000 7fa034ad1640 1 -- start start 2023-12-19T17:00:00.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.195+0000 7fa034ad1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa03006c740 0x7fa03006f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:00.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.195+0000 7fa034ad1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03006f790 con 0x7fa03015a3c0 2023-12-19T17:00:00.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.195+0000 7fa034ad1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa03015d5a0 con 0x7fa03006cb20 2023-12-19T17:00:00.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.195+0000 7fa034ad1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa03015d880 con 0x7fa03006c740 2023-12-19T17:00:00.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.199+0000 7fa02e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa03006c740 0x7fa03006f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:00.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.199+0000 7fa02e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa03006c740 0x7fa03006f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45776/0 (socket says 172.21.15.26:45776) 2023-12-19T17:00:00.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.199+0000 7fa02ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa03015a3c0 0x7fa03015a810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46828/0 (socket says 172.21.15.26:46828) 2023-12-19T17:00:00.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.199+0000 7fa02e59c640 1 -- 172.21.15.26:0/793949816 learned_addr learned my addr 172.21.15.26:0/793949816 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:00.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.199+0000 7fa02e59c640 1 -- 172.21.15.26:0/793949816 >> v1:172.21.15.160:6789/0 conn(0x7fa03006cb20 legacy=0x7fa0300ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:00.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.199+0000 7fa02e59c640 1 -- 172.21.15.26:0/793949816 >> v1:172.21.15.26:6789/0 conn(0x7fa03015a3c0 legacy=0x7fa03015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:00.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.199+0000 7fa02e59c640 1 -- 172.21.15.26:0/793949816 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03015db60 con 0x7fa03006c740 2023-12-19T17:00:00.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.199+0000 7fa02dd9b640 1 -- 172.21.15.26:0/793949816 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503945229 0 0) 0x7fa03015d5a0 con 0x7fa03006cb20 2023-12-19T17:00:00.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.199+0000 7fa02e59c640 1 --2- 172.21.15.26:0/793949816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa03006c740 0x7fa03006f250 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa020009870 tx=0x7fa02001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0605fc62ce0b2ee server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:00.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.199+0000 7fa02dd9b640 1 -- 172.21.15.26:0/793949816 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0200ab600 con 0x7fa03006c740 2023-12-19T17:00:00.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.199+0000 7fa02dd9b640 1 -- 172.21.15.26:0/793949816 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0200a7050 con 0x7fa03006c740 2023-12-19T17:00:00.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.199+0000 7fa02dd9b640 1 -- 172.21.15.26:0/793949816 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0200ab980 con 0x7fa03006c740 2023-12-19T17:00:00.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 -- 172.21.15.26:0/793949816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa03006c740 msgr2=0x7fa03006f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:00.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 --2- 172.21.15.26:0/793949816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa03006c740 0x7fa03006f250 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fa020009870 tx=0x7fa02001c3d0 comp rx=0 tx=0).stop 2023-12-19T17:00:00.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 -- 172.21.15.26:0/793949816 shutdown_connections 2023-12-19T17:00:00.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 --2- 172.21.15.26:0/793949816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa03006c740 0x7fa03006f250 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:00.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 -- 172.21.15.26:0/793949816 >> 172.21.15.26:0/793949816 conn(0x7fa0300660d0 msgr2=0x7fa0300ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:00.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 -- 172.21.15.26:0/793949816 shutdown_connections 2023-12-19T17:00:00.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 -- 172.21.15.26:0/793949816 wait complete. 2023-12-19T17:00:00.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 Processor -- start 2023-12-19T17:00:00.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 -- start start 2023-12-19T17:00:00.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa03006c740 0x7fa030153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:00.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa03006cb20 0x7fa030153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:00.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa03015a3c0 0x7fa0301e4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:00.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa030158090 con 0x7fa03006cb20 2023-12-19T17:00:00.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0301e4d70 con 0x7fa03006c740 2023-12-19T17:00:00.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa034ad1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0301e5040 con 0x7fa03015a3c0 2023-12-19T17:00:00.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa027fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa03006cb20 0x7fa030153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:00.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa027fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa03006cb20 0x7fa030153390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45778/0 (socket says 172.21.15.26:45778) 2023-12-19T17:00:00.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa027fff640 1 -- 172.21.15.26:0/2177884945 learned_addr learned my addr 172.21.15.26:0/2177884945 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:00.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.203+0000 7fa02e59c640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa03006c740 0x7fa030153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:00.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa02ed9d640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa03015a3c0 0x7fa0301e4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:00.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa027fff640 1 -- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa03015a3c0 msgr2=0x7fa0301e4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:00.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa027fff640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa03015a3c0 0x7fa0301e4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:00.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa027fff640 1 -- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa03006c740 msgr2=0x7fa030153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:00.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa027fff640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa03006c740 0x7fa030153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:00.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa027fff640 1 -- 172.21.15.26:0/2177884945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0200094d0 con 0x7fa03006cb20 2023-12-19T17:00:00.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa027fff640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa03006cb20 0x7fa030153390 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa01800c500 tx=0x7fa01800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:00.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa026ffd640 1 -- 172.21.15.26:0/2177884945 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018005480 con 0x7fa03006cb20 2023-12-19T17:00:00.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa026ffd640 1 -- 172.21.15.26:0/2177884945 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa018003600 con 0x7fa03006cb20 2023-12-19T17:00:00.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa026ffd640 1 -- 172.21.15.26:0/2177884945 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0180039b0 con 0x7fa03006cb20 2023-12-19T17:00:00.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa034ad1640 1 -- 172.21.15.26:0/2177884945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0301e5320 con 0x7fa03006cb20 2023-12-19T17:00:00.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa034ad1640 1 -- 172.21.15.26:0/2177884945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0301e88d0 con 0x7fa03006cb20 2023-12-19T17:00:00.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa034ad1640 1 -- 172.21.15.26:0/2177884945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa030158200 con 0x7fa03006cb20 2023-12-19T17:00:00.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.207+0000 7fa026ffd640 1 -- 172.21.15.26:0/2177884945 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa018003b10 con 0x7fa03006cb20 2023-12-19T17:00:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.211+0000 7fa026ffd640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa010065b80 0x7fa010068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.211+0000 7fa026ffd640 1 -- 172.21.15.26:0/2177884945 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5959+0+0 (secure 0 0 0) 0x7fa018082d10 con 0x7fa03006cb20 2023-12-19T17:00:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.211+0000 7fa02e59c640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa010065b80 0x7fa010068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:00.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.211+0000 7fa02e59c640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa010065b80 0x7fa010068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa0200ad000 tx=0x7fa0200a6000 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.211+0000 7fa026ffd640 1 -- 172.21.15.26:0/2177884945 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa0180506e0 con 0x7fa03006cb20 2023-12-19T17:00:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:00.371+0000 7fa034ad1640 1 -- 172.21.15.26:0/2177884945 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa0301e57d0 con 0x7fa03006cb20 2023-12-19T17:00:01.026 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T17:00:01.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.023+0000 7fa026ffd640 1 -- 172.21.15.26:0/2177884945 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v121) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa018058e60 con 0x7fa03006cb20 2023-12-19T17:00:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.027+0000 7fa034ad1640 1 -- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa010065b80 msgr2=0x7fa010068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.027+0000 7fa034ad1640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa010065b80 0x7fa010068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa0200ad000 tx=0x7fa0200a6000 comp rx=0 tx=0).stop 2023-12-19T17:00:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.027+0000 7fa034ad1640 1 -- 172.21.15.26:0/2177884945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa03006cb20 msgr2=0x7fa030153390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.027+0000 7fa034ad1640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa03006cb20 0x7fa030153390 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fa01800c500 tx=0x7fa01800c9c0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.031+0000 7fa034ad1640 1 -- 172.21.15.26:0/2177884945 shutdown_connections 2023-12-19T17:00:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.031+0000 7fa034ad1640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa010065b80 0x7fa010068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.031+0000 7fa034ad1640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa03015a3c0 0x7fa0301e4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.031+0000 7fa034ad1640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa03006cb20 0x7fa030153390 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.031+0000 7fa034ad1640 1 --2- 172.21.15.26:0/2177884945 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa03006c740 0x7fa030153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.031+0000 7fa034ad1640 1 -- 172.21.15.26:0/2177884945 >> 172.21.15.26:0/2177884945 conn(0x7fa0300660d0 msgr2=0x7fa0300ca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.035+0000 7fa034ad1640 1 -- 172.21.15.26:0/2177884945 shutdown_connections 2023-12-19T17:00:01.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.035+0000 7fa034ad1640 1 -- 172.21.15.26:0/2177884945 wait complete. 2023-12-19T17:00:01.070 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T17:00:01.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.215+0000 7f5caa9a2640 1 Processor -- start 2023-12-19T17:00:01.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.215+0000 7f5caa9a2640 1 -- start start 2023-12-19T17:00:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.215+0000 7f5caa9a2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5ca415ab40 0x7f5ca406b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.215+0000 7f5caa9a2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca415af20 con 0x7f5ca406f0a0 2023-12-19T17:00:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.215+0000 7f5caa9a2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca415f420 con 0x7f5ca4071890 2023-12-19T17:00:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.215+0000 7f5caa9a2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca40aea90 con 0x7f5ca415ab40 2023-12-19T17:00:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.215+0000 7f5ca919f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5ca406f0a0 0x7f5ca415e600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46838/0 (socket says 172.21.15.26:46838) 2023-12-19T17:00:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.215+0000 7f5ca919f640 1 -- 172.21.15.26:0/2741711633 learned_addr learned my addr 172.21.15.26:0/2741711633 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:00:01.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.215+0000 7f5ca99a0640 1 --2- 172.21.15.26:0/2741711633 >> v2:172.21.15.160:3300/0 conn(0x7f5ca415ab40 0x7f5ca406b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.219+0000 7f5ca899e640 1 -- 172.21.15.26:0/2741711633 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576390602 0 0) 0x7f5ca415af20 con 0x7f5ca406f0a0 2023-12-19T17:00:01.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.219+0000 7f5ca899e640 1 -- 172.21.15.26:0/2741711633 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c90003530 con 0x7f5ca406f0a0 2023-12-19T17:00:01.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.219+0000 7f5ca99a0640 1 -- 172.21.15.26:0/2741711633 >> v1:172.21.15.160:6789/0 conn(0x7f5ca4071890 legacy=0x7f5ca415ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.219+0000 7f5ca899e640 1 -- 172.21.15.26:0/2741711633 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870806498 0 0) 0x7f5ca415f420 con 0x7f5ca4071890 2023-12-19T17:00:01.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.219+0000 7f5ca99a0640 1 -- 172.21.15.26:0/2741711633 >> v1:172.21.15.26:6789/0 conn(0x7f5ca406f0a0 legacy=0x7f5ca415e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.219+0000 7f5ca99a0640 1 -- 172.21.15.26:0/2741711633 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca40aed70 con 0x7f5ca415ab40 2023-12-19T17:00:01.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.219+0000 7f5ca99a0640 1 --2- 172.21.15.26:0/2741711633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ca415ab40 0x7f5ca406b3b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00097c0 tx=0x7f5ca001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=893777c70c55ccd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:01.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.219+0000 7f5ca899e640 1 -- 172.21.15.26:0/2741711633 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca00ab600 con 0x7f5ca415ab40 2023-12-19T17:00:01.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.219+0000 7f5ca899e640 1 -- 172.21.15.26:0/2741711633 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca00a7050 con 0x7f5ca415ab40 2023-12-19T17:00:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.219+0000 7f5ca899e640 1 -- 172.21.15.26:0/2741711633 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca00ab960 con 0x7f5ca415ab40 2023-12-19T17:00:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 -- 172.21.15.26:0/2741711633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ca415ab40 msgr2=0x7f5ca406b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 --2- 172.21.15.26:0/2741711633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ca415ab40 0x7f5ca406b3b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00097c0 tx=0x7f5ca001c480 comp rx=0 tx=0).stop 2023-12-19T17:00:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 -- 172.21.15.26:0/2741711633 shutdown_connections 2023-12-19T17:00:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 --2- 172.21.15.26:0/2741711633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ca415ab40 0x7f5ca406b3b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 -- 172.21.15.26:0/2741711633 >> 172.21.15.26:0/2741711633 conn(0x7f5ca40658e0 msgr2=0x7f5ca4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 -- 172.21.15.26:0/2741711633 shutdown_connections 2023-12-19T17:00:01.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 -- 172.21.15.26:0/2741711633 wait complete. 2023-12-19T17:00:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 Processor -- start 2023-12-19T17:00:01.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 -- start start 2023-12-19T17:00:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5ca406f0a0 0x7f5ca40cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ca4071890 0x7f5ca41482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca415ab40 0x7f5ca414ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca4156260 con 0x7f5ca415ab40 2023-12-19T17:00:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca40cbed0 con 0x7f5ca4071890 2023-12-19T17:00:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa9a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca41559c0 con 0x7f5ca406f0a0 2023-12-19T17:00:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa1a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca415ab40 0x7f5ca414ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa1a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca415ab40 0x7f5ca414ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45792/0 (socket says 172.21.15.26:45792) 2023-12-19T17:00:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa1a1640 1 -- 172.21.15.26:0/3857770562 learned_addr learned my addr 172.21.15.26:0/3857770562 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5ca99a0640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5ca406f0a0 0x7f5ca40cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5ca919f640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ca4071890 0x7f5ca41482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa1a1640 1 -- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5ca406f0a0 msgr2=0x7f5ca40cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa1a1640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5ca406f0a0 0x7f5ca40cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa1a1640 1 -- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ca4071890 msgr2=0x7f5ca41482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa1a1640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ca4071890 0x7f5ca41482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.223+0000 7f5caa1a1640 1 -- 172.21.15.26:0/3857770562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca0009420 con 0x7f5ca415ab40 2023-12-19T17:00:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.227+0000 7f5caa1a1640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca415ab40 0x7f5ca414ffa0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00c780 tx=0x7f5c9c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.227+0000 7f5c9a7fc640 1 -- 172.21.15.26:0/3857770562 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c003e00 con 0x7f5ca415ab40 2023-12-19T17:00:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.227+0000 7f5c9a7fc640 1 -- 172.21.15.26:0/3857770562 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c00ce90 con 0x7f5ca415ab40 2023-12-19T17:00:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.227+0000 7f5c9a7fc640 1 -- 172.21.15.26:0/3857770562 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0052b0 con 0x7f5ca415ab40 2023-12-19T17:00:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.227+0000 7f5caa9a2640 1 -- 172.21.15.26:0/3857770562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca4155ca0 con 0x7f5ca415ab40 2023-12-19T17:00:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.227+0000 7f5caa9a2640 1 -- 172.21.15.26:0/3857770562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca414f7a0 con 0x7f5ca415ab40 2023-12-19T17:00:01.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.227+0000 7f5caa9a2640 1 -- 172.21.15.26:0/3857770562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca414f600 con 0x7f5ca415ab40 2023-12-19T17:00:01.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.227+0000 7f5c9a7fc640 1 -- 172.21.15.26:0/3857770562 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5c9c005410 con 0x7f5ca415ab40 2023-12-19T17:00:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.227+0000 7f5c9a7fc640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5c88065b80 0x7f5c88068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.227+0000 7f5c9a7fc640 1 -- 172.21.15.26:0/3857770562 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f5c9c083d90 con 0x7f5ca415ab40 2023-12-19T17:00:01.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.231+0000 7f5ca99a0640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5c88065b80 0x7f5c88068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.231+0000 7f5ca99a0640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5c88065b80 0x7f5c88068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00ad000 tx=0x7f5ca00a6000 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:01.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.231+0000 7f5c9a7fc640 1 -- 172.21.15.26:0/3857770562 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5c9c0513a0 con 0x7f5ca415ab40 2023-12-19T17:00:01.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.419+0000 7f5caa9a2640 1 -- 172.21.15.26:0/3857770562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5ca414f600 con 0x7f5ca415ab40 2023-12-19T17:00:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.431+0000 7f5c9a7fc640 1 -- 172.21.15.26:0/3857770562 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v53) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5ca414f600 con 0x7f5ca415ab40 2023-12-19T17:00:01.437 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 14 and data pool 15 2023-12-19T17:00:01.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.439+0000 7f5caa9a2640 1 -- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5c88065b80 msgr2=0x7f5c88068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.439+0000 7f5caa9a2640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5c88065b80 0x7f5c88068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00ad000 tx=0x7f5ca00a6000 comp rx=0 tx=0).stop 2023-12-19T17:00:01.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.439+0000 7f5caa9a2640 1 -- 172.21.15.26:0/3857770562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca415ab40 msgr2=0x7f5ca414ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.439+0000 7f5caa9a2640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca415ab40 0x7f5ca414ffa0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00c780 tx=0x7f5c9c00cc40 comp rx=0 tx=0).stop 2023-12-19T17:00:01.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.439+0000 7f5caa9a2640 1 -- 172.21.15.26:0/3857770562 shutdown_connections 2023-12-19T17:00:01.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.439+0000 7f5caa9a2640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ca415ab40 0x7f5ca414ffa0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.439+0000 7f5caa9a2640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ca4071890 0x7f5ca41482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.439+0000 7f5caa9a2640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5c88065b80 0x7f5c88068040 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00ad000 tx=0x7f5ca00a6000 comp rx=0 tx=0).stop 2023-12-19T17:00:01.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.439+0000 7f5caa9a2640 1 --2- 172.21.15.26:0/3857770562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5ca406f0a0 0x7f5ca40cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.439+0000 7f5caa9a2640 1 -- 172.21.15.26:0/3857770562 >> 172.21.15.26:0/3857770562 conn(0x7f5ca40658e0 msgr2=0x7f5ca40b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:01.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.439+0000 7f5caa9a2640 1 -- 172.21.15.26:0/3857770562 shutdown_connections 2023-12-19T17:00:01.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.439+0000 7f5caa9a2640 1 -- 172.21.15.26:0/3857770562 wait complete. 2023-12-19T17:00:01.478 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:00:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.619+0000 7efd5ce95640 1 Processor -- start 2023-12-19T17:00:01.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.619+0000 7efd5ce95640 1 -- start start 2023-12-19T17:00:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5ce95640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd58155030 0x7efd5815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5ce95640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efd5815f370 0x7efd5806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5ce95640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd580ce740 0x7efd580cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5ce95640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd580cf0f0 con 0x7efd58155030 2023-12-19T17:00:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5ce95640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5806c970 con 0x7efd5815f370 2023-12-19T17:00:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5ce95640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5815b1f0 con 0x7efd580ce740 2023-12-19T17:00:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd58155030 0x7efd5815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd58155030 0x7efd5815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45806/0 (socket says 172.21.15.26:45806) 2023-12-19T17:00:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5659c640 1 -- 172.21.15.26:0/758865509 learned_addr learned my addr 172.21.15.26:0/758865509 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5659c640 1 -- 172.21.15.26:0/758865509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd580ce740 msgr2=0x7efd580cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5659c640 1 --2- 172.21.15.26:0/758865509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd580ce740 0x7efd580cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5659c640 1 -- 172.21.15.26:0/758865509 >> v2:172.21.15.160:3300/0 conn(0x7efd5815f370 msgr2=0x7efd5806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5659c640 1 --2- 172.21.15.26:0/758865509 >> v2:172.21.15.160:3300/0 conn(0x7efd5815f370 0x7efd5806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5659c640 1 -- 172.21.15.26:0/758865509 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5815b4d0 con 0x7efd58155030 2023-12-19T17:00:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.623+0000 7efd5659c640 1 --2- 172.21.15.26:0/758865509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd58155030 0x7efd5815bcd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7efd580b1780 tx=0x7efd4801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3d21c43df567ff server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd55d9b640 1 -- 172.21.15.26:0/758865509 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd480aa610 con 0x7efd58155030 2023-12-19T17:00:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd55d9b640 1 -- 172.21.15.26:0/758865509 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd480a6040 con 0x7efd58155030 2023-12-19T17:00:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd55d9b640 1 -- 172.21.15.26:0/758865509 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd480aa990 con 0x7efd58155030 2023-12-19T17:00:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 -- 172.21.15.26:0/758865509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd58155030 msgr2=0x7efd5815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 --2- 172.21.15.26:0/758865509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd58155030 0x7efd5815bcd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7efd580b1780 tx=0x7efd4801c4d0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 -- 172.21.15.26:0/758865509 shutdown_connections 2023-12-19T17:00:01.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 --2- 172.21.15.26:0/758865509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd580ce740 0x7efd580cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 --2- 172.21.15.26:0/758865509 >> v2:172.21.15.160:3300/0 conn(0x7efd5815f370 0x7efd5806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 --2- 172.21.15.26:0/758865509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd58155030 0x7efd5815bcd0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 -- 172.21.15.26:0/758865509 >> 172.21.15.26:0/758865509 conn(0x7efd580658e0 msgr2=0x7efd58065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:01.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 -- 172.21.15.26:0/758865509 shutdown_connections 2023-12-19T17:00:01.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 -- 172.21.15.26:0/758865509 wait complete. 2023-12-19T17:00:01.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 Processor -- start 2023-12-19T17:00:01.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 -- start start 2023-12-19T17:00:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd580ce740 0x7efd58147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd58155030 0x7efd580b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5815f370 0x7efd58147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5814f510 con 0x7efd580ce740 2023-12-19T17:00:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5814cba0 con 0x7efd58155030 2023-12-19T17:00:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.627+0000 7efd5ce95640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5814ef50 con 0x7efd5815f370 2023-12-19T17:00:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd5659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd580ce740 0x7efd58147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd5659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd580ce740 0x7efd58147ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45818/0 (socket says 172.21.15.26:45818) 2023-12-19T17:00:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd5659c640 1 -- 172.21.15.26:0/3452667740 learned_addr learned my addr 172.21.15.26:0/3452667740 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd56d9d640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5815f370 0x7efd58147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd4ffff640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd58155030 0x7efd580b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd5659c640 1 -- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5815f370 msgr2=0x7efd58147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd5659c640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5815f370 0x7efd58147280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd5659c640 1 -- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd58155030 msgr2=0x7efd580b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd5659c640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd58155030 0x7efd580b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd5659c640 1 -- 172.21.15.26:0/3452667740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd480094d0 con 0x7efd580ce740 2023-12-19T17:00:01.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd5659c640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd580ce740 0x7efd58147ab0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7efd480ac040 tx=0x7efd4801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:01.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd4effd640 1 -- 172.21.15.26:0/3452667740 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd480bd5f0 con 0x7efd580ce740 2023-12-19T17:00:01.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd4effd640 1 -- 172.21.15.26:0/3452667740 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd480bd770 con 0x7efd580ce740 2023-12-19T17:00:01.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd4effd640 1 -- 172.21.15.26:0/3452667740 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd480bdb20 con 0x7efd580ce740 2023-12-19T17:00:01.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd5ce95640 1 -- 172.21.15.26:0/3452667740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5806c740 con 0x7efd580ce740 2023-12-19T17:00:01.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd5ce95640 1 -- 172.21.15.26:0/3452667740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5814c830 con 0x7efd580ce740 2023-12-19T17:00:01.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd5ce95640 1 -- 172.21.15.26:0/3452667740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5814c690 con 0x7efd580ce740 2023-12-19T17:00:01.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.631+0000 7efd4effd640 1 -- 172.21.15.26:0/3452667740 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7efd480b33f0 con 0x7efd580ce740 2023-12-19T17:00:01.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.635+0000 7efd4effd640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd38065b80 0x7efd38068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.635+0000 7efd4effd640 1 -- 172.21.15.26:0/3452667740 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6441+0+0 (secure 0 0 0) 0x7efd4811e410 con 0x7efd580ce740 2023-12-19T17:00:01.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.635+0000 7efd4ffff640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd38065b80 0x7efd38068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.635+0000 7efd4ffff640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd38065b80 0x7efd38068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efd40006fd0 tx=0x7efd40008040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.639+0000 7efd4effd640 1 -- 172.21.15.26:0/3452667740 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efd5814c690 con 0x7efd580ce740 2023-12-19T17:00:01.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.795+0000 7efd5ce95640 1 -- 172.21.15.26:0/3452667740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd5814c690 con 0x7efd580ce740 2023-12-19T17:00:01.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.799+0000 7efd4effd640 1 -- 172.21.15.26:0/3452667740 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17169 (secure 0 0 0) 0x7efd5814c690 con 0x7efd580ce740 2023-12-19T17:00:01.800 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:00:01.800 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":122,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:00:01.424890+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:59:59.967175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T17:00:00.420191+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:00:01.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.799+0000 7efd5ce95640 1 -- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd38065b80 msgr2=0x7efd38068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.799+0000 7efd5ce95640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd38065b80 0x7efd38068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efd40006fd0 tx=0x7efd40008040 comp rx=0 tx=0).stop 2023-12-19T17:00:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.799+0000 7efd5ce95640 1 -- 172.21.15.26:0/3452667740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd580ce740 msgr2=0x7efd58147ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.799+0000 7efd5ce95640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd580ce740 0x7efd58147ab0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7efd480ac040 tx=0x7efd4801ad10 comp rx=0 tx=0).stop 2023-12-19T17:00:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.799+0000 7efd5ce95640 1 -- 172.21.15.26:0/3452667740 shutdown_connections 2023-12-19T17:00:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.799+0000 7efd5ce95640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd38065b80 0x7efd38068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.799+0000 7efd5ce95640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd5815f370 0x7efd58147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.799+0000 7efd5ce95640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd58155030 0x7efd580b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.799+0000 7efd5ce95640 1 --2- 172.21.15.26:0/3452667740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd580ce740 0x7efd58147ab0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.799+0000 7efd5ce95640 1 -- 172.21.15.26:0/3452667740 >> 172.21.15.26:0/3452667740 conn(0x7efd580658e0 msgr2=0x7efd5814be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.799+0000 7efd5ce95640 1 -- 172.21.15.26:0/3452667740 shutdown_connections 2023-12-19T17:00:01.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.799+0000 7efd5ce95640 1 -- 172.21.15.26:0/3452667740 wait complete. 2023-12-19T17:00:01.822 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:00:01.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.963+0000 7f98a99a6640 1 Processor -- start 2023-12-19T17:00:01.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.963+0000 7f98a99a6640 1 -- start start 2023-12-19T17:00:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a99a6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f98a415c7f0 0x7f98a406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a99a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a40b30d0 0x7f98a40b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a99a6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a40b3a60 con 0x7f98a40adb80 2023-12-19T17:00:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a99a6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a406d770 con 0x7f98a415c7f0 2023-12-19T17:00:01.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a99a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a406ced0 con 0x7f98a40b30d0 2023-12-19T17:00:01.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f989bfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f98a40adb80 0x7f98a40cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:51498/0 (socket says 172.21.15.26:51498) 2023-12-19T17:00:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f989bfff640 1 -- 172.21.15.26:0/953703734 learned_addr learned my addr 172.21.15.26:0/953703734 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:00:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a37fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a40b30d0 0x7f98a40b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a2ffd640 1 --2- 172.21.15.26:0/953703734 >> v2:172.21.15.160:3300/0 conn(0x7f98a415c7f0 0x7f98a406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a27fc640 1 -- 172.21.15.26:0/953703734 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100889466 0 0) 0x7f98a40b3a60 con 0x7f98a40adb80 2023-12-19T17:00:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a27fc640 1 -- 172.21.15.26:0/953703734 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9890003530 con 0x7f98a40adb80 2023-12-19T17:00:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a2ffd640 1 -- 172.21.15.26:0/953703734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a40b30d0 msgr2=0x7f98a40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a2ffd640 1 --2- 172.21.15.26:0/953703734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a40b30d0 0x7f98a40b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a2ffd640 1 -- 172.21.15.26:0/953703734 >> v1:172.21.15.160:6789/0 conn(0x7f98a40adb80 legacy=0x7f98a40cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a2ffd640 1 -- 172.21.15.26:0/953703734 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a406d1b0 con 0x7f98a415c7f0 2023-12-19T17:00:01.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.967+0000 7f98a37fe640 1 --2- 172.21.15.26:0/953703734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a40b30d0 0x7f98a40b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:00:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a2ffd640 1 --2- 172.21.15.26:0/953703734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98a415c7f0 0x7f98a406c7f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f989400a960 tx=0x7f989401d850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a1735f6eb546168 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a27fc640 1 -- 172.21.15.26:0/953703734 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940a6770 con 0x7f98a415c7f0 2023-12-19T17:00:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a27fc640 1 -- 172.21.15.26:0/953703734 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98940a68d0 con 0x7f98a415c7f0 2023-12-19T17:00:01.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a27fc640 1 -- 172.21.15.26:0/953703734 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940a6c60 con 0x7f98a415c7f0 2023-12-19T17:00:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a99a6640 1 -- 172.21.15.26:0/953703734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98a415c7f0 msgr2=0x7f98a406c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a99a6640 1 --2- 172.21.15.26:0/953703734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98a415c7f0 0x7f98a406c7f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f989400a960 tx=0x7f989401d850 comp rx=0 tx=0).stop 2023-12-19T17:00:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a99a6640 1 -- 172.21.15.26:0/953703734 shutdown_connections 2023-12-19T17:00:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a99a6640 1 --2- 172.21.15.26:0/953703734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a40b30d0 0x7f98a40b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a99a6640 1 --2- 172.21.15.26:0/953703734 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98a415c7f0 0x7f98a406c7f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a99a6640 1 -- 172.21.15.26:0/953703734 >> 172.21.15.26:0/953703734 conn(0x7f98a40658e0 msgr2=0x7f98a4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a99a6640 1 -- 172.21.15.26:0/953703734 shutdown_connections 2023-12-19T17:00:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a99a6640 1 -- 172.21.15.26:0/953703734 wait complete. 2023-12-19T17:00:01.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a99a6640 1 Processor -- start 2023-12-19T17:00:01.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.971+0000 7f98a99a6640 1 -- start start 2023-12-19T17:00:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a99a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a40adb80 0x7f98a4148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a99a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a40b30d0 0x7f98a41544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a99a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98a415c7f0 0x7f98a414ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a99a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a414b670 con 0x7f98a40adb80 2023-12-19T17:00:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a99a6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a414b950 con 0x7f98a415c7f0 2023-12-19T17:00:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a99a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a414bc30 con 0x7f98a40b30d0 2023-12-19T17:00:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a40adb80 0x7f98a4148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a40adb80 0x7f98a4148620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45834/0 (socket says 172.21.15.26:45834) 2023-12-19T17:00:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a2ffd640 1 -- 172.21.15.26:0/3498684140 learned_addr learned my addr 172.21.15.26:0/3498684140 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a37fe640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98a415c7f0 0x7f98a414ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f989bfff640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a40b30d0 0x7f98a41544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a2ffd640 1 -- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a40b30d0 msgr2=0x7f98a41544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a2ffd640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a40b30d0 0x7f98a41544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a2ffd640 1 -- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98a415c7f0 msgr2=0x7f98a414ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a2ffd640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98a415c7f0 0x7f98a414ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a2ffd640 1 -- 172.21.15.26:0/3498684140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c009a70 con 0x7f98a40adb80 2023-12-19T17:00:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f989bfff640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a40b30d0 0x7f98a41544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:00:01.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a2ffd640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a40adb80 0x7f98a4148620 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f98940b1040 tx=0x7f989401a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a0ff9640 1 -- 172.21.15.26:0/3498684140 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940b9640 con 0x7f98a40adb80 2023-12-19T17:00:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a0ff9640 1 -- 172.21.15.26:0/3498684140 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98940b97a0 con 0x7f98a40adb80 2023-12-19T17:00:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a0ff9640 1 -- 172.21.15.26:0/3498684140 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98940b9b50 con 0x7f98a40adb80 2023-12-19T17:00:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a99a6640 1 -- 172.21.15.26:0/3498684140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f989400a5c0 con 0x7f98a40adb80 2023-12-19T17:00:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a99a6640 1 -- 172.21.15.26:0/3498684140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a41549e0 con 0x7f98a40adb80 2023-12-19T17:00:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a99a6640 1 -- 172.21.15.26:0/3498684140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a414c0e0 con 0x7f98a40adb80 2023-12-19T17:00:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.975+0000 7f98a0ff9640 1 -- 172.21.15.26:0/3498684140 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f98940b5040 con 0x7f98a40adb80 2023-12-19T17:00:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.979+0000 7f98a0ff9640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9884065b80 0x7f9884068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.979+0000 7f98a0ff9640 1 -- 172.21.15.26:0/3498684140 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f989411dc80 con 0x7f98a40adb80 2023-12-19T17:00:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.979+0000 7f989bfff640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9884065b80 0x7f9884068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.979+0000 7f989bfff640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9884065b80 0x7f9884068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f98a41f4380 tx=0x7f988c009040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:01.983+0000 7f98a0ff9640 1 -- 172.21.15.26:0/3498684140 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f98940eb2e0 con 0x7f98a40adb80 2023-12-19T17:00:02.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.139+0000 7f98a99a6640 1 -- 172.21.15.26:0/3498684140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98a4074270 con 0x7f98a40adb80 2023-12-19T17:00:02.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.143+0000 7f98a0ff9640 1 -- 172.21.15.26:0/3498684140 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17169 (secure 0 0 0) 0x7f98940ee900 con 0x7f98a40adb80 2023-12-19T17:00:02.144 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:00:02.144 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":122,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:00:01.424890+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:59:59.967175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T17:00:00.420191+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:00:02.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.143+0000 7f98a99a6640 1 -- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9884065b80 msgr2=0x7f9884068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:02.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.143+0000 7f98a99a6640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9884065b80 0x7f9884068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f98a41f4380 tx=0x7f988c009040 comp rx=0 tx=0).stop 2023-12-19T17:00:02.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.143+0000 7f98a99a6640 1 -- 172.21.15.26:0/3498684140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a40adb80 msgr2=0x7f98a4148620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:02.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.143+0000 7f98a99a6640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a40adb80 0x7f98a4148620 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f98940b1040 tx=0x7f989401a5f0 comp rx=0 tx=0).stop 2023-12-19T17:00:02.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.143+0000 7f98a99a6640 1 -- 172.21.15.26:0/3498684140 shutdown_connections 2023-12-19T17:00:02.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.143+0000 7f98a99a6640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98a415c7f0 0x7f98a414ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.143+0000 7f98a99a6640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9884065b80 0x7f9884068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.143+0000 7f98a99a6640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98a40b30d0 0x7f98a41544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.143+0000 7f98a99a6640 1 --2- 172.21.15.26:0/3498684140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98a40adb80 0x7f98a4148620 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.143+0000 7f98a99a6640 1 -- 172.21.15.26:0/3498684140 >> 172.21.15.26:0/3498684140 conn(0x7f98a40658e0 msgr2=0x7f98a40b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.143+0000 7f98a99a6640 1 -- 172.21.15.26:0/3498684140 shutdown_connections 2023-12-19T17:00:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.143+0000 7f98a99a6640 1 -- 172.21.15.26:0/3498684140 wait complete. 2023-12-19T17:00:02.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T17:00:02.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e3ee5640 1 Processor -- start 2023-12-19T17:00:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e3ee5640 1 -- start start 2023-12-19T17:00:02.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e3ee5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80dc161230 0x7f80dc0ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e3ee5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80dc06c3f0 0x7f80dc06c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e3ee5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80dc0b01b0 con 0x7f80dc164e50 2023-12-19T17:00:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e3ee5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80dc0c9e00 con 0x7f80dc161230 2023-12-19T17:00:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e3ee5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc0adfb0 con 0x7f80dc06c3f0 2023-12-19T17:00:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e1c81640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80dc161230 0x7f80dc0ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e1c81640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80dc161230 0x7f80dc0ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45846/0 (socket says 172.21.15.26:45846) 2023-12-19T17:00:02.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e1c81640 1 -- 172.21.15.26:0/3780787201 learned_addr learned my addr 172.21.15.26:0/3780787201 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e1480640 1 --2- 172.21.15.26:0/3780787201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80dc06c3f0 0x7f80dc06c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e1c81640 1 -- 172.21.15.26:0/3780787201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80dc06c3f0 msgr2=0x7f80dc06c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e1c81640 1 --2- 172.21.15.26:0/3780787201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80dc06c3f0 0x7f80dc06c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e1c81640 1 -- 172.21.15.26:0/3780787201 >> v1:172.21.15.160:6789/0 conn(0x7f80dc164e50 legacy=0x7f80dc1652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e1c81640 1 -- 172.21.15.26:0/3780787201 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80dc0ca4e0 con 0x7f80dc161230 2023-12-19T17:00:02.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.311+0000 7f80e1480640 1 --2- 172.21.15.26:0/3780787201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80dc06c3f0 0x7f80dc06c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:00:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.315+0000 7f80e1c81640 1 --2- 172.21.15.26:0/3780787201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80dc161230 0x7f80dc0ada70 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009870 tx=0x7f80d801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354fed404f835021 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:02.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.315+0000 7f80e0c7f640 1 -- 172.21.15.26:0/3780787201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80ab600 con 0x7f80dc161230 2023-12-19T17:00:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.315+0000 7f80e0c7f640 1 -- 172.21.15.26:0/3780787201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d80ab760 con 0x7f80dc161230 2023-12-19T17:00:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.315+0000 7f80e0c7f640 1 -- 172.21.15.26:0/3780787201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d80abaf0 con 0x7f80dc161230 2023-12-19T17:00:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.315+0000 7f80e3ee5640 1 -- 172.21.15.26:0/3780787201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80dc161230 msgr2=0x7f80dc0ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.315+0000 7f80e3ee5640 1 --2- 172.21.15.26:0/3780787201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80dc161230 0x7f80dc0ada70 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f80d8009870 tx=0x7f80d801c440 comp rx=0 tx=0).stop 2023-12-19T17:00:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.315+0000 7f80e3ee5640 1 -- 172.21.15.26:0/3780787201 shutdown_connections 2023-12-19T17:00:02.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.315+0000 7f80e3ee5640 1 --2- 172.21.15.26:0/3780787201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80dc06c3f0 0x7f80dc06c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.315+0000 7f80e3ee5640 1 --2- 172.21.15.26:0/3780787201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80dc161230 0x7f80dc0ada70 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.315+0000 7f80e3ee5640 1 -- 172.21.15.26:0/3780787201 >> 172.21.15.26:0/3780787201 conn(0x7f80dc0ce730 msgr2=0x7f80dc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.315+0000 7f80e3ee5640 1 -- 172.21.15.26:0/3780787201 shutdown_connections 2023-12-19T17:00:02.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.315+0000 7f80e3ee5640 1 -- 172.21.15.26:0/3780787201 wait complete. 2023-12-19T17:00:02.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e3ee5640 1 Processor -- start 2023-12-19T17:00:02.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e3ee5640 1 -- start start 2023-12-19T17:00:02.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e3ee5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80dc06c3f0 0x7f80dc160110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:02.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e3ee5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80dc161230 0x7f80dc0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:02.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e3ee5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80dc164e50 0x7f80dc1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:02.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e3ee5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc0cd830 con 0x7f80dc164e50 2023-12-19T17:00:02.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e3ee5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc158ef0 con 0x7f80dc161230 2023-12-19T17:00:02.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e3ee5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc0cc270 con 0x7f80dc06c3f0 2023-12-19T17:00:02.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e2482640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80dc164e50 0x7f80dc1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:02.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e2482640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80dc164e50 0x7f80dc1589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45862/0 (socket says 172.21.15.26:45862) 2023-12-19T17:00:02.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e2482640 1 -- 172.21.15.26:0/300736967 learned_addr learned my addr 172.21.15.26:0/300736967 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:02.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e1480640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80dc161230 0x7f80dc0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:02.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e1c81640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80dc06c3f0 0x7f80dc160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:02.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e2482640 1 -- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80dc06c3f0 msgr2=0x7f80dc160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e2482640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80dc06c3f0 0x7f80dc160110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e2482640 1 -- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80dc161230 msgr2=0x7f80dc0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e2482640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80dc161230 0x7f80dc0cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e2482640 1 -- 172.21.15.26:0/300736967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d80094d0 con 0x7f80dc164e50 2023-12-19T17:00:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e1480640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80dc161230 0x7f80dc0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:00:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e2482640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80dc164e50 0x7f80dc1589b0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f80d400b9a0 tx=0x7f80d400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80d27fc640 1 -- 172.21.15.26:0/300736967 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d400cba0 con 0x7f80dc164e50 2023-12-19T17:00:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80d27fc640 1 -- 172.21.15.26:0/300736967 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d400cd00 con 0x7f80dc164e50 2023-12-19T17:00:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80d27fc640 1 -- 172.21.15.26:0/300736967 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d4014680 con 0x7f80dc164e50 2023-12-19T17:00:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e3ee5640 1 -- 172.21.15.26:0/300736967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80dc0cc530 con 0x7f80dc164e50 2023-12-19T17:00:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e3ee5640 1 -- 172.21.15.26:0/300736967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80dc0ccaf0 con 0x7f80dc164e50 2023-12-19T17:00:02.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.319+0000 7f80e3ee5640 1 -- 172.21.15.26:0/300736967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80dc074270 con 0x7f80dc164e50 2023-12-19T17:00:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.323+0000 7f80d27fc640 1 -- 172.21.15.26:0/300736967 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f80d401d3b0 con 0x7f80dc164e50 2023-12-19T17:00:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.323+0000 7f80d27fc640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f80c0065b80 0x7f80c0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:02.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.323+0000 7f80d27fc640 1 -- 172.21.15.26:0/300736967 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f80d40833c0 con 0x7f80dc164e50 2023-12-19T17:00:02.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.323+0000 7f80e1c81640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f80c0065b80 0x7f80c0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:02.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.323+0000 7f80e1c81640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f80c0065b80 0x7f80c0068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f80d80af000 tx=0x7f80d80a7000 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:02.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.327+0000 7f80d27fc640 1 -- 172.21.15.26:0/300736967 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f80d4050990 con 0x7f80dc164e50 2023-12-19T17:00:02.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:02.519+0000 7f80e3ee5640 1 -- 172.21.15.26:0/300736967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f80dc160a60 con 0x7f80dc164e50 2023-12-19T17:00:02.924 INFO:tasks.ceph.mds.a.smithi026.stderr:2023-12-19T17:00:02.923+0000 7fd4a789e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T17:00:03.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.443+0000 7f80d27fc640 1 -- 172.21.15.26:0/300736967 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) v1 ==== 118+0+0 (secure 0 0 0) 0x7f80d4053fb0 con 0x7f80dc164e50 2023-12-19T17:00:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.447+0000 7f80e3ee5640 1 -- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f80c0065b80 msgr2=0x7f80c0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.447+0000 7f80e3ee5640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f80c0065b80 0x7f80c0068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f80d80af000 tx=0x7f80d80a7000 comp rx=0 tx=0).stop 2023-12-19T17:00:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.447+0000 7f80e3ee5640 1 -- 172.21.15.26:0/300736967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80dc164e50 msgr2=0x7f80dc1589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.447+0000 7f80e3ee5640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80dc164e50 0x7f80dc1589b0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f80d400b9a0 tx=0x7f80d400be60 comp rx=0 tx=0).stop 2023-12-19T17:00:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.447+0000 7f80e3ee5640 1 -- 172.21.15.26:0/300736967 shutdown_connections 2023-12-19T17:00:03.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.447+0000 7f80e3ee5640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80dc164e50 0x7f80dc1589b0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:03.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.447+0000 7f80e3ee5640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80dc161230 0x7f80dc0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:03.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.447+0000 7f80e3ee5640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f80c0065b80 0x7f80c0068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:03.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.447+0000 7f80e3ee5640 1 --2- 172.21.15.26:0/300736967 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80dc06c3f0 0x7f80dc160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:03.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.447+0000 7f80e3ee5640 1 -- 172.21.15.26:0/300736967 >> 172.21.15.26:0/300736967 conn(0x7f80dc0ce730 msgr2=0x7f80dc0ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:03.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.447+0000 7f80e3ee5640 1 -- 172.21.15.26:0/300736967 shutdown_connections 2023-12-19T17:00:03.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.447+0000 7f80e3ee5640 1 -- 172.21.15.26:0/300736967 wait complete. 2023-12-19T17:00:03.471 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T17:00:03.471 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T17:00:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.619+0000 7ff4b110a640 1 Processor -- start 2023-12-19T17:00:03.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.619+0000 7ff4b110a640 1 -- start start 2023-12-19T17:00:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4b110a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff4ac15a9c0 0x7ff4ac0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4b110a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff4ac06ac80 0x7ff4ac06b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4b110a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ac070d90 con 0x7ff4ac15ada0 2023-12-19T17:00:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4b110a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ac0cc270 con 0x7ff4ac06ac80 2023-12-19T17:00:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4b110a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ac071070 con 0x7ff4ac15a9c0 2023-12-19T17:00:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4ab59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff4ac06ac80 0x7ff4ac06b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4ab59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff4ac06ac80 0x7ff4ac06b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45874/0 (socket says 172.21.15.26:45874) 2023-12-19T17:00:03.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4ab59e640 1 -- 172.21.15.26:0/1737540948 learned_addr learned my addr 172.21.15.26:0/1737540948 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4ab59e640 1 -- 172.21.15.26:0/1737540948 >> v2:172.21.15.160:3300/0 conn(0x7ff4ac15a9c0 msgr2=0x7ff4ac0706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4ab59e640 1 --2- 172.21.15.26:0/1737540948 >> v2:172.21.15.160:3300/0 conn(0x7ff4ac15a9c0 0x7ff4ac0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4ab59e640 1 -- 172.21.15.26:0/1737540948 >> v1:172.21.15.160:6789/0 conn(0x7ff4ac15ada0 legacy=0x7ff4ac0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:03.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4ab59e640 1 -- 172.21.15.26:0/1737540948 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac071350 con 0x7ff4ac06ac80 2023-12-19T17:00:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4ab59e640 1 --2- 172.21.15.26:0/1737540948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac06ac80 0x7ff4ac06b090 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0cc3b0 tx=0x7ff4a401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f885ef44602e1c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4a9d9b640 1 -- 172.21.15.26:0/1737540948 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40a57b0 con 0x7ff4ac06ac80 2023-12-19T17:00:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4a9d9b640 1 -- 172.21.15.26:0/1737540948 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a40a5910 con 0x7ff4ac06ac80 2023-12-19T17:00:03.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.623+0000 7ff4a9d9b640 1 -- 172.21.15.26:0/1737540948 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40a5ca0 con 0x7ff4ac06ac80 2023-12-19T17:00:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 -- 172.21.15.26:0/1737540948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac06ac80 msgr2=0x7ff4ac06b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 --2- 172.21.15.26:0/1737540948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac06ac80 0x7ff4ac06b090 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac0cc3b0 tx=0x7ff4a401c8d0 comp rx=0 tx=0).stop 2023-12-19T17:00:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 -- 172.21.15.26:0/1737540948 shutdown_connections 2023-12-19T17:00:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 --2- 172.21.15.26:0/1737540948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac06ac80 0x7ff4ac06b090 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:03.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 --2- 172.21.15.26:0/1737540948 >> v2:172.21.15.160:3300/0 conn(0x7ff4ac15a9c0 0x7ff4ac0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 -- 172.21.15.26:0/1737540948 >> 172.21.15.26:0/1737540948 conn(0x7ff4ac0ce730 msgr2=0x7ff4ac0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 -- 172.21.15.26:0/1737540948 shutdown_connections 2023-12-19T17:00:03.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 -- 172.21.15.26:0/1737540948 wait complete. 2023-12-19T17:00:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 Processor -- start 2023-12-19T17:00:03.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 -- start start 2023-12-19T17:00:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac06ac80 0x7ff4ac0c1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4ac15a9c0 0x7ff4ac148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:03.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4ac15ada0 0x7ff4ac1557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac159b60 con 0x7ff4ac06ac80 2023-12-19T17:00:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac0c8cf0 con 0x7ff4ac15ada0 2023-12-19T17:00:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4b110a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac153040 con 0x7ff4ac15a9c0 2023-12-19T17:00:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.627+0000 7ff4aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac06ac80 0x7ff4ac0c1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac06ac80 0x7ff4ac0c1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45890/0 (socket says 172.21.15.26:45890) 2023-12-19T17:00:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4aad9d640 1 -- 172.21.15.26:0/2786875980 learned_addr learned my addr 172.21.15.26:0/2786875980 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4aa59c640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4ac15a9c0 0x7ff4ac148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4ab59e640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4ac15ada0 0x7ff4ac1557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4aad9d640 1 -- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4ac15a9c0 msgr2=0x7ff4ac148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4aad9d640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4ac15a9c0 0x7ff4ac148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4aad9d640 1 -- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4ac15ada0 msgr2=0x7ff4ac1557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4aad9d640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4ac15ada0 0x7ff4ac1557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:03.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4aad9d640 1 -- 172.21.15.26:0/2786875980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4009b30 con 0x7ff4ac06ac80 2023-12-19T17:00:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4ab59e640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4ac15ada0 0x7ff4ac1557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:00:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4aad9d640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac06ac80 0x7ff4ac0c1650 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00b580 tx=0x7ff49c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4937fe640 1 -- 172.21.15.26:0/2786875980 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c010870 con 0x7ff4ac06ac80 2023-12-19T17:00:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4937fe640 1 -- 172.21.15.26:0/2786875980 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c0109f0 con 0x7ff4ac06ac80 2023-12-19T17:00:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4937fe640 1 -- 172.21.15.26:0/2786875980 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c010da0 con 0x7ff4ac06ac80 2023-12-19T17:00:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4b110a640 1 -- 172.21.15.26:0/2786875980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ac159d70 con 0x7ff4ac06ac80 2023-12-19T17:00:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4b110a640 1 -- 172.21.15.26:0/2786875980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ac0c8f40 con 0x7ff4ac06ac80 2023-12-19T17:00:03.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4b110a640 1 -- 172.21.15.26:0/2786875980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ac074270 con 0x7ff4ac06ac80 2023-12-19T17:00:03.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.631+0000 7ff4937fe640 1 -- 172.21.15.26:0/2786875980 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff49c00f3c0 con 0x7ff4ac06ac80 2023-12-19T17:00:03.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.635+0000 7ff4937fe640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff498065b80 0x7ff498068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:03.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.635+0000 7ff4937fe640 1 -- 172.21.15.26:0/2786875980 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7ff49c083520 con 0x7ff4ac06ac80 2023-12-19T17:00:03.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.635+0000 7ff4aa59c640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff498065b80 0x7ff498068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:03.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.635+0000 7ff4aa59c640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff498065b80 0x7ff498068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac152520 tx=0x7ff4a0009060 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:03.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.639+0000 7ff4937fe640 1 -- 172.21.15.26:0/2786875980 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff49c050bb0 con 0x7ff4ac06ac80 2023-12-19T17:00:03.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:03.831+0000 7ff4b110a640 1 -- 172.21.15.26:0/2786875980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff4ac0cd7b0 con 0x7ff4ac06ac80 2023-12-19T17:00:04.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.451+0000 7ff4937fe640 1 -- 172.21.15.26:0/2786875980 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff49c0541d0 con 0x7ff4ac06ac80 2023-12-19T17:00:04.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.451+0000 7ff4b110a640 1 -- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff498065b80 msgr2=0x7ff498068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:04.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.451+0000 7ff4b110a640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff498065b80 0x7ff498068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac152520 tx=0x7ff4a0009060 comp rx=0 tx=0).stop 2023-12-19T17:00:04.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.451+0000 7ff4b110a640 1 -- 172.21.15.26:0/2786875980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac06ac80 msgr2=0x7ff4ac0c1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:04.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.451+0000 7ff4b110a640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac06ac80 0x7ff4ac0c1650 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00b580 tx=0x7ff49c00ba40 comp rx=0 tx=0).stop 2023-12-19T17:00:04.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.451+0000 7ff4b110a640 1 -- 172.21.15.26:0/2786875980 shutdown_connections 2023-12-19T17:00:04.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.451+0000 7ff4b110a640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4ac15ada0 0x7ff4ac1557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:04.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.451+0000 7ff4b110a640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff498065b80 0x7ff498068040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:04.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.451+0000 7ff4b110a640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4ac15a9c0 0x7ff4ac148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:04.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.451+0000 7ff4b110a640 1 --2- 172.21.15.26:0/2786875980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4ac06ac80 0x7ff4ac0c1650 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:04.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.451+0000 7ff4b110a640 1 -- 172.21.15.26:0/2786875980 >> 172.21.15.26:0/2786875980 conn(0x7ff4ac0ce730 msgr2=0x7ff4ac153640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:04.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.455+0000 7ff4b110a640 1 -- 172.21.15.26:0/2786875980 shutdown_connections 2023-12-19T17:00:04.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.455+0000 7ff4b110a640 1 -- 172.21.15.26:0/2786875980 wait complete. 2023-12-19T17:00:04.479 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T17:00:04.479 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:00:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.623+0000 7f551d266640 1 Processor -- start 2023-12-19T17:00:04.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.623+0000 7f551d266640 1 -- start start 2023-12-19T17:00:04.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f551d266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5518155030 0x7f551815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:04.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f551d266640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551806c970 con 0x7f551815f370 2023-12-19T17:00:04.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f551d266640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551815b1f0 con 0x7f55180ce740 2023-12-19T17:00:04.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f551d266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551806c790 con 0x7f5518155030 2023-12-19T17:00:04.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f550ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f551815f370 0x7f551806c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46854/0 (socket says 172.21.15.26:46854) 2023-12-19T17:00:04.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f550ffff640 1 -- 172.21.15.26:0/3277026062 learned_addr learned my addr 172.21.15.26:0/3277026062 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:00:04.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f5516ffd640 1 --2- 172.21.15.26:0/3277026062 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5518155030 0x7f551815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:04.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f55167fc640 1 -- 172.21.15.26:0/3277026062 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385474713 0 0) 0x7f551806c970 con 0x7f551815f370 2023-12-19T17:00:04.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f55167fc640 1 -- 172.21.15.26:0/3277026062 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5504003530 con 0x7f551815f370 2023-12-19T17:00:04.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f55167fc640 1 -- 172.21.15.26:0/3277026062 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573336952 0 0) 0x7f551815b1f0 con 0x7f55180ce740 2023-12-19T17:00:04.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f55167fc640 1 -- 172.21.15.26:0/3277026062 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f551806c970 con 0x7f55180ce740 2023-12-19T17:00:04.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f55167fc640 1 -- 172.21.15.26:0/3277026062 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 254064734 0 0) 0x7f5504003530 con 0x7f551815f370 2023-12-19T17:00:04.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f5516ffd640 1 -- 172.21.15.26:0/3277026062 >> v1:172.21.15.160:6789/0 conn(0x7f55180ce740 legacy=0x7f55180ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:04.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f5516ffd640 1 -- 172.21.15.26:0/3277026062 >> v1:172.21.15.26:6789/0 conn(0x7f551815f370 legacy=0x7f551806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:04.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.627+0000 7f5516ffd640 1 -- 172.21.15.26:0/3277026062 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551815b4d0 con 0x7f5518155030 2023-12-19T17:00:04.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.631+0000 7f5516ffd640 1 --2- 172.21.15.26:0/3277026062 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5518155030 0x7f551815bcd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f55080097f0 tx=0x7f550801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24bdca11bd44068b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:04.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.631+0000 7f55167fc640 1 -- 172.21.15.26:0/3277026062 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500001680 con 0x7f5518155030 2023-12-19T17:00:04.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.631+0000 7f55167fc640 1 -- 172.21.15.26:0/3277026062 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55080a7050 con 0x7f5518155030 2023-12-19T17:00:04.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.631+0000 7f55167fc640 1 -- 172.21.15.26:0/3277026062 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508004250 con 0x7f5518155030 2023-12-19T17:00:04.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.631+0000 7f551d266640 1 -- 172.21.15.26:0/3277026062 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5518155030 msgr2=0x7f551815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:04.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.631+0000 7f551d266640 1 --2- 172.21.15.26:0/3277026062 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5518155030 0x7f551815bcd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f55080097f0 tx=0x7f550801c3b0 comp rx=0 tx=0).stop 2023-12-19T17:00:04.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.631+0000 7f551d266640 1 -- 172.21.15.26:0/3277026062 shutdown_connections 2023-12-19T17:00:04.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.631+0000 7f551d266640 1 --2- 172.21.15.26:0/3277026062 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5518155030 0x7f551815bcd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:04.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.631+0000 7f551d266640 1 -- 172.21.15.26:0/3277026062 >> 172.21.15.26:0/3277026062 conn(0x7f55180658e0 msgr2=0x7f5518065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:04.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.631+0000 7f551d266640 1 -- 172.21.15.26:0/3277026062 shutdown_connections 2023-12-19T17:00:04.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.631+0000 7f551d266640 1 -- 172.21.15.26:0/3277026062 wait complete. 2023-12-19T17:00:04.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.631+0000 7f551d266640 1 Processor -- start 2023-12-19T17:00:04.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f551d266640 1 -- start start 2023-12-19T17:00:04.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f551d266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55180ce740 0x7f551814aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:04.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f551d266640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5518155030 0x7f55180b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:04.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f551d266640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551815f370 0x7f5518148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:04.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f551d266640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55180adfb0 con 0x7f551815f370 2023-12-19T17:00:04.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f551d266640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55180ae200 con 0x7f5518155030 2023-12-19T17:00:04.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f551d266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55180ae4e0 con 0x7f55180ce740 2023-12-19T17:00:04.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f55177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551815f370 0x7f5518148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f55177fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551815f370 0x7f5518148fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45902/0 (socket says 172.21.15.26:45902) 2023-12-19T17:00:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f5516ffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55180ce740 0x7f551814aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f55177fe640 1 -- 172.21.15.26:0/3952564832 learned_addr learned my addr 172.21.15.26:0/3952564832 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f550ffff640 1 --2- 172.21.15.26:0/3952564832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5518155030 0x7f55180b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f55177fe640 1 -- 172.21.15.26:0/3952564832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55180ce740 msgr2=0x7f551814aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f55177fe640 1 --2- 172.21.15.26:0/3952564832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55180ce740 0x7f551814aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f55177fe640 1 -- 172.21.15.26:0/3952564832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5518155030 msgr2=0x7f55180b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f55177fe640 1 --2- 172.21.15.26:0/3952564832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5518155030 0x7f55180b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f55177fe640 1 -- 172.21.15.26:0/3952564832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5508009420 con 0x7f551815f370 2023-12-19T17:00:04.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f55177fe640 1 --2- 172.21.15.26:0/3952564832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551815f370 0x7f5518148fa0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f551000c790 tx=0x7f551000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:04.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f550f7fe640 1 -- 172.21.15.26:0/3952564832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551000cea0 con 0x7f551815f370 2023-12-19T17:00:04.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f550f7fe640 1 -- 172.21.15.26:0/3952564832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5510003b80 con 0x7f551815f370 2023-12-19T17:00:04.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f551d266640 1 -- 172.21.15.26:0/3952564832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55180ae7c0 con 0x7f551815f370 2023-12-19T17:00:04.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f550f7fe640 1 -- 172.21.15.26:0/3952564832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551000a850 con 0x7f551815f370 2023-12-19T17:00:04.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f551d266640 1 -- 172.21.15.26:0/3952564832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551814afa0 con 0x7f551815f370 2023-12-19T17:00:04.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.635+0000 7f551d266640 1 -- 172.21.15.26:0/3952564832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5518074270 con 0x7f551815f370 2023-12-19T17:00:04.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.639+0000 7f550f7fe640 1 -- 172.21.15.26:0/3952564832 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f551000a9b0 con 0x7f551815f370 2023-12-19T17:00:04.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.639+0000 7f550f7fe640 1 --2- 172.21.15.26:0/3952564832 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f8065b80 0x7f54f8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:04.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.639+0000 7f550f7fe640 1 -- 172.21.15.26:0/3952564832 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f55100832f0 con 0x7f551815f370 2023-12-19T17:00:04.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.639+0000 7f5516ffd640 1 --2- 172.21.15.26:0/3952564832 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f8065b80 0x7f54f8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:04.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.639+0000 7f5516ffd640 1 --2- 172.21.15.26:0/3952564832 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f8065b80 0x7f54f8068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f55080ad000 tx=0x7f55080a6000 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:04.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.643+0000 7f550f7fe640 1 -- 172.21.15.26:0/3952564832 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f55100509e0 con 0x7f551815f370 2023-12-19T17:00:04.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.831+0000 7f551d266640 1 -- 172.21.15.26:0/3952564832 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55180ccc30 con 0x7f551815f370 2023-12-19T17:00:04.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.831+0000 7f550f7fe640 1 -- 172.21.15.26:0/3952564832 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f5510054000 con 0x7f551815f370 2023-12-19T17:00:04.835 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:00:04.835 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6884,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1869411616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1869411616},{"type":"v1","addr":"172.21.15.26:6835","nonce":1869411616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3129570713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3129570713},{"type":"v1","addr":"172.21.15.160:6836","nonce":3129570713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4063618989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":4063618989},{"type":"v1","addr":"172.21.15.160:6837","nonce":4063618989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:00:01.425222+0000","modified":"2023-12-19T17:00:04.444345+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7974":{"gid":7974,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/2837068623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2837068623},{"type":"v1","addr":"172.21.15.26:6834","nonce":2837068623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T17:00:04.835 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 57 2023-12-19T17:00:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.835+0000 7f551d266640 1 -- 172.21.15.26:0/3952564832 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f8065b80 msgr2=0x7f54f8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.835+0000 7f551d266640 1 --2- 172.21.15.26:0/3952564832 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f8065b80 0x7f54f8068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f55080ad000 tx=0x7f55080a6000 comp rx=0 tx=0).stop 2023-12-19T17:00:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.835+0000 7f551d266640 1 -- 172.21.15.26:0/3952564832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551815f370 msgr2=0x7f5518148fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.835+0000 7f551d266640 1 --2- 172.21.15.26:0/3952564832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551815f370 0x7f5518148fa0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f551000c790 tx=0x7f551000cc50 comp rx=0 tx=0).stop 2023-12-19T17:00:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.835+0000 7f551d266640 1 -- 172.21.15.26:0/3952564832 shutdown_connections 2023-12-19T17:00:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.835+0000 7f551d266640 1 --2- 172.21.15.26:0/3952564832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f551815f370 0x7f5518148fa0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.835+0000 7f551d266640 1 --2- 172.21.15.26:0/3952564832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5518155030 0x7f55180b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.835+0000 7f551d266640 1 --2- 172.21.15.26:0/3952564832 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f8065b80 0x7f54f8068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.835+0000 7f551d266640 1 --2- 172.21.15.26:0/3952564832 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55180ce740 0x7f551814aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.835+0000 7f551d266640 1 -- 172.21.15.26:0/3952564832 >> 172.21.15.26:0/3952564832 conn(0x7f55180658e0 msgr2=0x7f551806b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.835+0000 7f551d266640 1 -- 172.21.15.26:0/3952564832 shutdown_connections 2023-12-19T17:00:04.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.835+0000 7f551d266640 1 -- 172.21.15.26:0/3952564832 wait complete. 2023-12-19T17:00:04.856 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:00:04.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.995+0000 7faaa968c640 1 Processor -- start 2023-12-19T17:00:04.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.995+0000 7faaa968c640 1 -- start start 2023-12-19T17:00:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faaa968c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faaa415a7c0 0x7faaa415ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faaa968c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faaa40cf700 0x7faaa40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faaa968c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa415e2f0 con 0x7faaa406f9d0 2023-12-19T17:00:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faaa968c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa40c5b40 con 0x7faaa415a7c0 2023-12-19T17:00:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faaa968c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa40c5e20 con 0x7faaa40cf700 2023-12-19T17:00:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faaa2ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faaa415a7c0 0x7faaa415ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faa9bfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7faaa406f9d0 0x7faaa40ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:51546/0 (socket says 172.21.15.26:51546) 2023-12-19T17:00:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faa9bfff640 1 -- 172.21.15.26:0/2326175631 learned_addr learned my addr 172.21.15.26:0/2326175631 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:00:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faaa37fe640 1 --2- 172.21.15.26:0/2326175631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faaa40cf700 0x7faaa40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faaa27fc640 1 -- 172.21.15.26:0/2326175631 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930282573 0 0) 0x7faaa415e2f0 con 0x7faaa406f9d0 2023-12-19T17:00:05.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faaa37fe640 1 -- 172.21.15.26:0/2326175631 >> v2:172.21.15.160:3300/0 conn(0x7faaa415a7c0 msgr2=0x7faaa415ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:05.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faaa37fe640 1 --2- 172.21.15.26:0/2326175631 >> v2:172.21.15.160:3300/0 conn(0x7faaa415a7c0 0x7faaa415ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faaa37fe640 1 -- 172.21.15.26:0/2326175631 >> v1:172.21.15.160:6789/0 conn(0x7faaa406f9d0 legacy=0x7faaa40ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:04.999+0000 7faaa37fe640 1 -- 172.21.15.26:0/2326175631 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa40ce770 con 0x7faaa40cf700 2023-12-19T17:00:05.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa37fe640 1 --2- 172.21.15.26:0/2326175631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faaa40cf700 0x7faaa40cd240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009d90 tx=0x7faa9c01c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6578bd8c1de71ecc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:05.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa27fc640 1 -- 172.21.15.26:0/2326175631 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c0a59e0 con 0x7faaa40cf700 2023-12-19T17:00:05.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa27fc640 1 -- 172.21.15.26:0/2326175631 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9c0a5b40 con 0x7faaa40cf700 2023-12-19T17:00:05.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa27fc640 1 -- 172.21.15.26:0/2326175631 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c0ad440 con 0x7faaa40cf700 2023-12-19T17:00:05.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa968c640 1 -- 172.21.15.26:0/2326175631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faaa40cf700 msgr2=0x7faaa40cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa968c640 1 --2- 172.21.15.26:0/2326175631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faaa40cf700 0x7faaa40cd240 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009d90 tx=0x7faa9c01c920 comp rx=0 tx=0).stop 2023-12-19T17:00:05.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa968c640 1 -- 172.21.15.26:0/2326175631 shutdown_connections 2023-12-19T17:00:05.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa968c640 1 --2- 172.21.15.26:0/2326175631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faaa40cf700 0x7faaa40cd240 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa968c640 1 --2- 172.21.15.26:0/2326175631 >> v2:172.21.15.160:3300/0 conn(0x7faaa415a7c0 0x7faaa415ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa968c640 1 -- 172.21.15.26:0/2326175631 >> 172.21.15.26:0/2326175631 conn(0x7faaa406d5d0 msgr2=0x7faaa4069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:05.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa968c640 1 -- 172.21.15.26:0/2326175631 shutdown_connections 2023-12-19T17:00:05.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa968c640 1 -- 172.21.15.26:0/2326175631 wait complete. 2023-12-19T17:00:05.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa968c640 1 Processor -- start 2023-12-19T17:00:05.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.003+0000 7faaa968c640 1 -- start start 2023-12-19T17:00:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa968c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa406f9d0 0x7faaa40ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa968c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faaa40cf700 0x7faaa40b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa968c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faaa415a7c0 0x7faaa4151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa968c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa40b4c00 con 0x7faaa406f9d0 2023-12-19T17:00:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa968c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa4152c30 con 0x7faaa40cf700 2023-12-19T17:00:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa968c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa40b4640 con 0x7faaa415a7c0 2023-12-19T17:00:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa406f9d0 0x7faaa40ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa406f9d0 0x7faaa40ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45908/0 (socket says 172.21.15.26:45908) 2023-12-19T17:00:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa2ffd640 1 -- 172.21.15.26:0/320534424 learned_addr learned my addr 172.21.15.26:0/320534424 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faa9bfff640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faaa40cf700 0x7faaa40b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa37fe640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faaa415a7c0 0x7faaa4151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa2ffd640 1 -- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faaa415a7c0 msgr2=0x7faaa4151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa2ffd640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faaa415a7c0 0x7faaa4151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa2ffd640 1 -- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faaa40cf700 msgr2=0x7faaa40b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa2ffd640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faaa40cf700 0x7faaa40b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa2ffd640 1 -- 172.21.15.26:0/320534424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c009a70 con 0x7faaa406f9d0 2023-12-19T17:00:05.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faa9bfff640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faaa40cf700 0x7faaa40b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:00:05.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa2ffd640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa406f9d0 0x7faaa40ac850 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7faa9400e870 tx=0x7faa9400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:05.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faa9b7fe640 1 -- 172.21.15.26:0/320534424 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa940095d0 con 0x7faaa406f9d0 2023-12-19T17:00:05.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa968c640 1 -- 172.21.15.26:0/320534424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa40b4920 con 0x7faaa406f9d0 2023-12-19T17:00:05.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faa9b7fe640 1 -- 172.21.15.26:0/320534424 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9400c940 con 0x7faaa406f9d0 2023-12-19T17:00:05.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faa9b7fe640 1 -- 172.21.15.26:0/320534424 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9400ccb0 con 0x7faaa406f9d0 2023-12-19T17:00:05.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa968c640 1 -- 172.21.15.26:0/320534424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa40b4230 con 0x7faaa406f9d0 2023-12-19T17:00:05.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.007+0000 7faaa968c640 1 -- 172.21.15.26:0/320534424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa4152720 con 0x7faaa406f9d0 2023-12-19T17:00:05.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.011+0000 7faa9b7fe640 1 -- 172.21.15.26:0/320534424 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faa94013470 con 0x7faaa406f9d0 2023-12-19T17:00:05.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.011+0000 7faa9b7fe640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa84065b80 0x7faa84068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.011+0000 7faa9b7fe640 1 -- 172.21.15.26:0/320534424 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7faa94083b00 con 0x7faaa406f9d0 2023-12-19T17:00:05.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.011+0000 7faa9bfff640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa84065b80 0x7faa84068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.011+0000 7faa9bfff640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa84065b80 0x7faa84068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faaa41609d0 tx=0x7faa8c007d60 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:05.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.015+0000 7faa9b7fe640 1 -- 172.21.15.26:0/320534424 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faaa4152720 con 0x7faaa406f9d0 2023-12-19T17:00:05.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.175+0000 7faaa968c640 1 -- 172.21.15.26:0/320534424 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faaa4152720 con 0x7faaa406f9d0 2023-12-19T17:00:05.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.175+0000 7faa9b7fe640 1 -- 172.21.15.26:0/320534424 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17156 (secure 0 0 0) 0x7faaa4152720 con 0x7faaa406f9d0 2023-12-19T17:00:05.178 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:00:05.178 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":123,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:00:02.428229+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:59:59.967175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T17:00:00.420191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:00:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.175+0000 7faaa968c640 1 -- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa84065b80 msgr2=0x7faa84068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.175+0000 7faaa968c640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa84065b80 0x7faa84068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faaa41609d0 tx=0x7faa8c007d60 comp rx=0 tx=0).stop 2023-12-19T17:00:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.175+0000 7faaa968c640 1 -- 172.21.15.26:0/320534424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa406f9d0 msgr2=0x7faaa40ac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.175+0000 7faaa968c640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa406f9d0 0x7faaa40ac850 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7faa9400e870 tx=0x7faa9400ed30 comp rx=0 tx=0).stop 2023-12-19T17:00:05.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.175+0000 7faaa968c640 1 -- 172.21.15.26:0/320534424 shutdown_connections 2023-12-19T17:00:05.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.175+0000 7faaa968c640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa84065b80 0x7faa84068040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.175+0000 7faaa968c640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faaa415a7c0 0x7faaa4151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.175+0000 7faaa968c640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faaa40cf700 0x7faaa40b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.175+0000 7faaa968c640 1 --2- 172.21.15.26:0/320534424 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faaa406f9d0 0x7faaa40ac850 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.175+0000 7faaa968c640 1 -- 172.21.15.26:0/320534424 >> 172.21.15.26:0/320534424 conn(0x7faaa406d5d0 msgr2=0x7faaa40ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:05.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.179+0000 7faaa968c640 1 -- 172.21.15.26:0/320534424 shutdown_connections 2023-12-19T17:00:05.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.179+0000 7faaa968c640 1 -- 172.21.15.26:0/320534424 wait complete. 2023-12-19T17:00:05.199 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T17:00:05.199 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T17:00:05.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.339+0000 7ff2b30e5640 1 Processor -- start 2023-12-19T17:00:05.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.339+0000 7ff2b30e5640 1 -- start start 2023-12-19T17:00:05.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2b30e5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff2ac0658e0 0x7ff2ac06e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2b30e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff2ac15d260 0x7ff2ac065cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2b30e5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ac0b3060 con 0x7ff2ac0703a0 2023-12-19T17:00:05.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2b30e5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2ac066200 con 0x7ff2ac0658e0 2023-12-19T17:00:05.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2b30e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac15d820 con 0x7ff2ac15d260 2023-12-19T17:00:05.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2b1682640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff2ac0703a0 0x7ff2ac070810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46862/0 (socket says 172.21.15.26:46862) 2023-12-19T17:00:05.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2b1682640 1 -- 172.21.15.26:0/3206583273 learned_addr learned my addr 172.21.15.26:0/3206583273 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:00:05.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2abfff640 1 --2- 172.21.15.26:0/3206583273 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff2ac15d260 0x7ff2ac065cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2b0e81640 1 --2- 172.21.15.26:0/3206583273 >> v2:172.21.15.160:3300/0 conn(0x7ff2ac0658e0 0x7ff2ac06e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2ab7fe640 1 -- 172.21.15.26:0/3206583273 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607257762 0 0) 0x7ff2ac0b3060 con 0x7ff2ac0703a0 2023-12-19T17:00:05.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2ab7fe640 1 -- 172.21.15.26:0/3206583273 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff29c003530 con 0x7ff2ac0703a0 2023-12-19T17:00:05.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2ab7fe640 1 -- 172.21.15.26:0/3206583273 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3315790724 0 0) 0x7ff29c003530 con 0x7ff2ac0703a0 2023-12-19T17:00:05.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2abfff640 1 -- 172.21.15.26:0/3206583273 >> v2:172.21.15.160:3300/0 conn(0x7ff2ac0658e0 msgr2=0x7ff2ac06e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2abfff640 1 --2- 172.21.15.26:0/3206583273 >> v2:172.21.15.160:3300/0 conn(0x7ff2ac0658e0 0x7ff2ac06e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2abfff640 1 -- 172.21.15.26:0/3206583273 >> v1:172.21.15.26:6789/0 conn(0x7ff2ac0703a0 legacy=0x7ff2ac070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.343+0000 7ff2abfff640 1 -- 172.21.15.26:0/3206583273 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac15db00 con 0x7ff2ac15d260 2023-12-19T17:00:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2abfff640 1 --2- 172.21.15.26:0/3206583273 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff2ac15d260 0x7ff2ac065cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff2980097c0 tx=0x7ff29801c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a31d722737efa8c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2ab7fe640 1 -- 172.21.15.26:0/3206583273 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4001a00 con 0x7ff2ac15d260 2023-12-19T17:00:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2ab7fe640 1 -- 172.21.15.26:0/3206583273 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2980a7050 con 0x7ff2ac15d260 2023-12-19T17:00:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2ab7fe640 1 -- 172.21.15.26:0/3206583273 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2980ab620 con 0x7ff2ac15d260 2023-12-19T17:00:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2b30e5640 1 -- 172.21.15.26:0/3206583273 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff2ac15d260 msgr2=0x7ff2ac065cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2b30e5640 1 --2- 172.21.15.26:0/3206583273 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff2ac15d260 0x7ff2ac065cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff2980097c0 tx=0x7ff29801c370 comp rx=0 tx=0).stop 2023-12-19T17:00:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2b30e5640 1 -- 172.21.15.26:0/3206583273 shutdown_connections 2023-12-19T17:00:05.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2b30e5640 1 --2- 172.21.15.26:0/3206583273 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff2ac15d260 0x7ff2ac065cc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2b30e5640 1 --2- 172.21.15.26:0/3206583273 >> v2:172.21.15.160:3300/0 conn(0x7ff2ac0658e0 0x7ff2ac06e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2b30e5640 1 -- 172.21.15.26:0/3206583273 >> 172.21.15.26:0/3206583273 conn(0x7ff2ac06a320 msgr2=0x7ff2ac0b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:05.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2b30e5640 1 -- 172.21.15.26:0/3206583273 shutdown_connections 2023-12-19T17:00:05.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2b30e5640 1 -- 172.21.15.26:0/3206583273 wait complete. 2023-12-19T17:00:05.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2b30e5640 1 Processor -- start 2023-12-19T17:00:05.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.347+0000 7ff2b30e5640 1 -- start start 2023-12-19T17:00:05.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b30e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff2ac0658e0 0x7ff2ac13a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b30e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2ac0703a0 0x7ff2ac15eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b30e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac15d260 0x7ff2ac137e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b30e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac13bba0 con 0x7ff2ac15d260 2023-12-19T17:00:05.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b30e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac1601b0 con 0x7ff2ac0703a0 2023-12-19T17:00:05.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b30e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac13af20 con 0x7ff2ac0658e0 2023-12-19T17:00:05.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b1682640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac15d260 0x7ff2ac137e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b1682640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac15d260 0x7ff2ac137e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45910/0 (socket says 172.21.15.26:45910) 2023-12-19T17:00:05.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b1682640 1 -- 172.21.15.26:0/1875941273 learned_addr learned my addr 172.21.15.26:0/1875941273 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2abfff640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2ac0703a0 0x7ff2ac15eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b0e81640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff2ac0658e0 0x7ff2ac13a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b1682640 1 -- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff2ac0658e0 msgr2=0x7ff2ac13a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b1682640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff2ac0658e0 0x7ff2ac13a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b1682640 1 -- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2ac0703a0 msgr2=0x7ff2ac15eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b1682640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2ac0703a0 0x7ff2ac15eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b1682640 1 -- 172.21.15.26:0/1875941273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff298009420 con 0x7ff2ac15d260 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b0e81640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff2ac0658e0 0x7ff2ac13a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b1682640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac15d260 0x7ff2ac137e00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40018a0 tx=0x7ff2a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2a97fa640 1 -- 172.21.15.26:0/1875941273 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4012400 con 0x7ff2ac15d260 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2a97fa640 1 -- 172.21.15.26:0/1875941273 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a4012560 con 0x7ff2ac15d260 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2a97fa640 1 -- 172.21.15.26:0/1875941273 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4012910 con 0x7ff2ac15d260 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b30e5640 1 -- 172.21.15.26:0/1875941273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac13b200 con 0x7ff2ac15d260 2023-12-19T17:00:05.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b30e5640 1 -- 172.21.15.26:0/1875941273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac13b7c0 con 0x7ff2ac15d260 2023-12-19T17:00:05.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2b30e5640 1 -- 172.21.15.26:0/1875941273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac15ad10 con 0x7ff2ac15d260 2023-12-19T17:00:05.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.351+0000 7ff2a97fa640 1 -- 172.21.15.26:0/1875941273 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff2a4012a70 con 0x7ff2ac15d260 2023-12-19T17:00:05.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.355+0000 7ff2a97fa640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff290065b80 0x7ff290068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.355+0000 7ff2a97fa640 1 -- 172.21.15.26:0/1875941273 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7ff2a4083e60 con 0x7ff2ac15d260 2023-12-19T17:00:05.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.355+0000 7ff2b0e81640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff290065b80 0x7ff290068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.355+0000 7ff2b0e81640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff290065b80 0x7ff290068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009bb0 tx=0x7ff2a0008040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:05.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.359+0000 7ff2a97fa640 1 -- 172.21.15.26:0/1875941273 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff2a4051390 con 0x7ff2ac15d260 2023-12-19T17:00:05.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.515+0000 7ff2b30e5640 1 -- 172.21.15.26:0/1875941273 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2ac146740 con 0x7ff290065b80 2023-12-19T17:00:05.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.523+0000 7ff2a97fa640 1 -- 172.21.15.26:0/1875941273 <== mgr.6707 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289515 (secure 0 0 0) 0x7ff2a0011a70 con 0x7ff290065b80 2023-12-19T17:00:05.531 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:00:05.533 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T17:00:05.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.527+0000 7ff2b30e5640 1 -- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff290065b80 msgr2=0x7ff290068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.527+0000 7ff2b30e5640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff290065b80 0x7ff290068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009bb0 tx=0x7ff2a0008040 comp rx=0 tx=0).stop 2023-12-19T17:00:05.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.527+0000 7ff2b30e5640 1 -- 172.21.15.26:0/1875941273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac15d260 msgr2=0x7ff2ac137e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.527+0000 7ff2b30e5640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac15d260 0x7ff2ac137e00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40018a0 tx=0x7ff2a400a430 comp rx=0 tx=0).stop 2023-12-19T17:00:05.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.527+0000 7ff2b30e5640 1 -- 172.21.15.26:0/1875941273 shutdown_connections 2023-12-19T17:00:05.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.527+0000 7ff2b30e5640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac15d260 0x7ff2ac137e00 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.527+0000 7ff2b30e5640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2ac0703a0 0x7ff2ac15eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.527+0000 7ff2b30e5640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff290065b80 0x7ff290068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.527+0000 7ff2b30e5640 1 --2- 172.21.15.26:0/1875941273 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff2ac0658e0 0x7ff2ac13a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.527+0000 7ff2b30e5640 1 -- 172.21.15.26:0/1875941273 >> 172.21.15.26:0/1875941273 conn(0x7ff2ac06a320 msgr2=0x7ff2ac1403b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:05.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.531+0000 7ff2b30e5640 1 -- 172.21.15.26:0/1875941273 shutdown_connections 2023-12-19T17:00:05.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.531+0000 7ff2b30e5640 1 -- 172.21.15.26:0/1875941273 wait complete. 2023-12-19T17:00:05.551 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-19T17:00:05.516966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1058,"num_read_kb":2145,"num_write":399,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":263,"ondisk_log_size":263,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":340112,"kb_used_data":13008,"kb_used_omap":0,"kb_used_meta":327040,"kb_avail":754634608,"statfs":{"total":773094113280,"available":772745838592,"internally_reserved":0,"allocated":13320192,"data_stored":7309324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":334888960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.493446"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733747+0000","last_change":"2023-12-19T17:00:01.450171+0000","last_active":"2023-12-19T17:00:02.733747+0000","last_peered":"2023-12-19T17:00:02.733747+0000","last_clean":"2023-12-19T17:00:02.733747+0000","last_became_active":"2023-12-19T17:00:01.449898+0000","last_became_peered":"2023-12-19T17:00:01.449898+0000","last_unstale":"2023-12-19T17:00:02.733747+0000","last_undegraded":"2023-12-19T17:00:02.733747+0000","last_fullsized":"2023-12-19T17:00:02.733747+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:35.314161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438108+0000","last_change":"2023-12-19T17:00:01.449283+0000","last_active":"2023-12-19T17:00:02.438108+0000","last_peered":"2023-12-19T17:00:02.438108+0000","last_clean":"2023-12-19T17:00:02.438108+0000","last_became_active":"2023-12-19T17:00:01.449014+0000","last_became_peered":"2023-12-19T17:00:01.449014+0000","last_unstale":"2023-12-19T17:00:02.438108+0000","last_undegraded":"2023-12-19T17:00:02.438108+0000","last_fullsized":"2023-12-19T17:00:02.438108+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:23:00.185954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733766+0000","last_change":"2023-12-19T17:00:01.450395+0000","last_active":"2023-12-19T17:00:02.733766+0000","last_peered":"2023-12-19T17:00:02.733766+0000","last_clean":"2023-12-19T17:00:02.733766+0000","last_became_active":"2023-12-19T17:00:01.450032+0000","last_became_peered":"2023-12-19T17:00:01.450032+0000","last_unstale":"2023-12-19T17:00:02.733766+0000","last_undegraded":"2023-12-19T17:00:02.733766+0000","last_fullsized":"2023-12-19T17:00:02.733766+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:41.057366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733979+0000","last_change":"2023-12-19T17:00:01.466863+0000","last_active":"2023-12-19T17:00:02.733979+0000","last_peered":"2023-12-19T17:00:02.733979+0000","last_clean":"2023-12-19T17:00:02.733979+0000","last_became_active":"2023-12-19T17:00:01.466580+0000","last_became_peered":"2023-12-19T17:00:01.466580+0000","last_unstale":"2023-12-19T17:00:02.733979+0000","last_undegraded":"2023-12-19T17:00:02.733979+0000","last_fullsized":"2023-12-19T17:00:02.733979+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:05:36.465953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439120+0000","last_change":"2023-12-19T17:00:01.453915+0000","last_active":"2023-12-19T17:00:02.439120+0000","last_peered":"2023-12-19T17:00:02.439120+0000","last_clean":"2023-12-19T17:00:02.439120+0000","last_became_active":"2023-12-19T17:00:01.453639+0000","last_became_peered":"2023-12-19T17:00:01.453639+0000","last_unstale":"2023-12-19T17:00:02.439120+0000","last_undegraded":"2023-12-19T17:00:02.439120+0000","last_fullsized":"2023-12-19T17:00:02.439120+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:15:28.811743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734113+0000","last_change":"2023-12-19T17:00:01.459472+0000","last_active":"2023-12-19T17:00:02.734113+0000","last_peered":"2023-12-19T17:00:02.734113+0000","last_clean":"2023-12-19T17:00:02.734113+0000","last_became_active":"2023-12-19T17:00:01.459114+0000","last_became_peered":"2023-12-19T17:00:01.459114+0000","last_unstale":"2023-12-19T17:00:02.734113+0000","last_undegraded":"2023-12-19T17:00:02.734113+0000","last_fullsized":"2023-12-19T17:00:02.734113+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:47:45.647622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734357+0000","last_change":"2023-12-19T17:00:01.454499+0000","last_active":"2023-12-19T17:00:02.734357+0000","last_peered":"2023-12-19T17:00:02.734357+0000","last_clean":"2023-12-19T17:00:02.734357+0000","last_became_active":"2023-12-19T17:00:01.454217+0000","last_became_peered":"2023-12-19T17:00:01.454217+0000","last_unstale":"2023-12-19T17:00:02.734357+0000","last_undegraded":"2023-12-19T17:00:02.734357+0000","last_fullsized":"2023-12-19T17:00:02.734357+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:44.662350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437762+0000","last_change":"2023-12-19T17:00:01.453847+0000","last_active":"2023-12-19T17:00:02.437762+0000","last_peered":"2023-12-19T17:00:02.437762+0000","last_clean":"2023-12-19T17:00:02.437762+0000","last_became_active":"2023-12-19T17:00:01.453352+0000","last_became_peered":"2023-12-19T17:00:01.453352+0000","last_unstale":"2023-12-19T17:00:02.437762+0000","last_undegraded":"2023-12-19T17:00:02.437762+0000","last_fullsized":"2023-12-19T17:00:02.437762+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:27:30.538250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734522+0000","last_change":"2023-12-19T17:00:01.455767+0000","last_active":"2023-12-19T17:00:02.734522+0000","last_peered":"2023-12-19T17:00:02.734522+0000","last_clean":"2023-12-19T17:00:02.734522+0000","last_became_active":"2023-12-19T17:00:01.455202+0000","last_became_peered":"2023-12-19T17:00:01.455202+0000","last_unstale":"2023-12-19T17:00:02.734522+0000","last_undegraded":"2023-12-19T17:00:02.734522+0000","last_fullsized":"2023-12-19T17:00:02.734522+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:35.314161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439436+0000","last_change":"2023-12-19T17:00:01.448229+0000","last_active":"2023-12-19T17:00:02.439436+0000","last_peered":"2023-12-19T17:00:02.439436+0000","last_clean":"2023-12-19T17:00:02.439436+0000","last_became_active":"2023-12-19T17:00:01.447606+0000","last_became_peered":"2023-12-19T17:00:01.447606+0000","last_unstale":"2023-12-19T17:00:02.439436+0000","last_undegraded":"2023-12-19T17:00:02.439436+0000","last_fullsized":"2023-12-19T17:00:02.439436+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:11:08.259813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734010+0000","last_change":"2023-12-19T17:00:01.457240+0000","last_active":"2023-12-19T17:00:02.734010+0000","last_peered":"2023-12-19T17:00:02.734010+0000","last_clean":"2023-12-19T17:00:02.734010+0000","last_became_active":"2023-12-19T17:00:01.456975+0000","last_became_peered":"2023-12-19T17:00:01.456975+0000","last_unstale":"2023-12-19T17:00:02.734010+0000","last_undegraded":"2023-12-19T17:00:02.734010+0000","last_fullsized":"2023-12-19T17:00:02.734010+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:53.251492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733982+0000","last_change":"2023-12-19T17:00:01.447688+0000","last_active":"2023-12-19T17:00:02.733982+0000","last_peered":"2023-12-19T17:00:02.733982+0000","last_clean":"2023-12-19T17:00:02.733982+0000","last_became_active":"2023-12-19T17:00:01.446300+0000","last_became_peered":"2023-12-19T17:00:01.446300+0000","last_unstale":"2023-12-19T17:00:02.733982+0000","last_undegraded":"2023-12-19T17:00:02.733982+0000","last_fullsized":"2023-12-19T17:00:02.733982+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:41.057366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733909+0000","last_change":"2023-12-19T17:00:01.456109+0000","last_active":"2023-12-19T17:00:02.733909+0000","last_peered":"2023-12-19T17:00:02.733909+0000","last_clean":"2023-12-19T17:00:02.733909+0000","last_became_active":"2023-12-19T17:00:01.455860+0000","last_became_peered":"2023-12-19T17:00:01.455860+0000","last_unstale":"2023-12-19T17:00:02.733909+0000","last_undegraded":"2023-12-19T17:00:02.733909+0000","last_fullsized":"2023-12-19T17:00:02.733909+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:13.426008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734182+0000","last_change":"2023-12-19T17:00:01.452323+0000","last_active":"2023-12-19T17:00:02.734182+0000","last_peered":"2023-12-19T17:00:02.734182+0000","last_clean":"2023-12-19T17:00:02.734182+0000","last_became_active":"2023-12-19T17:00:01.452059+0000","last_became_peered":"2023-12-19T17:00:01.452059+0000","last_unstale":"2023-12-19T17:00:02.734182+0000","last_undegraded":"2023-12-19T17:00:02.734182+0000","last_fullsized":"2023-12-19T17:00:02.734182+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:31.653516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733496+0000","last_change":"2023-12-19T17:00:01.459010+0000","last_active":"2023-12-19T17:00:02.733496+0000","last_peered":"2023-12-19T17:00:02.733496+0000","last_clean":"2023-12-19T17:00:02.733496+0000","last_became_active":"2023-12-19T17:00:01.458738+0000","last_became_peered":"2023-12-19T17:00:01.458738+0000","last_unstale":"2023-12-19T17:00:02.733496+0000","last_undegraded":"2023-12-19T17:00:02.733496+0000","last_fullsized":"2023-12-19T17:00:02.733496+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:15:53.444387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733623+0000","last_change":"2023-12-19T17:00:01.463960+0000","last_active":"2023-12-19T17:00:02.733623+0000","last_peered":"2023-12-19T17:00:02.733623+0000","last_clean":"2023-12-19T17:00:02.733623+0000","last_became_active":"2023-12-19T17:00:01.463664+0000","last_became_peered":"2023-12-19T17:00:01.463664+0000","last_unstale":"2023-12-19T17:00:02.733623+0000","last_undegraded":"2023-12-19T17:00:02.733623+0000","last_fullsized":"2023-12-19T17:00:02.733623+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:34.250172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438223+0000","last_change":"2023-12-19T17:00:01.456608+0000","last_active":"2023-12-19T17:00:02.438223+0000","last_peered":"2023-12-19T17:00:02.438223+0000","last_clean":"2023-12-19T17:00:02.438223+0000","last_became_active":"2023-12-19T17:00:01.455916+0000","last_became_peered":"2023-12-19T17:00:01.455916+0000","last_unstale":"2023-12-19T17:00:02.438223+0000","last_undegraded":"2023-12-19T17:00:02.438223+0000","last_fullsized":"2023-12-19T17:00:02.438223+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:49:54.041124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734036+0000","last_change":"2023-12-19T17:00:01.463663+0000","last_active":"2023-12-19T17:00:02.734036+0000","last_peered":"2023-12-19T17:00:02.734036+0000","last_clean":"2023-12-19T17:00:02.734036+0000","last_became_active":"2023-12-19T17:00:01.463393+0000","last_became_peered":"2023-12-19T17:00:01.463393+0000","last_unstale":"2023-12-19T17:00:02.734036+0000","last_undegraded":"2023-12-19T17:00:02.734036+0000","last_fullsized":"2023-12-19T17:00:02.734036+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:21:33.482019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734521+0000","last_change":"2023-12-19T17:00:01.445233+0000","last_active":"2023-12-19T17:00:02.734521+0000","last_peered":"2023-12-19T17:00:02.734521+0000","last_clean":"2023-12-19T17:00:02.734521+0000","last_became_active":"2023-12-19T17:00:01.444790+0000","last_became_peered":"2023-12-19T17:00:01.444790+0000","last_unstale":"2023-12-19T17:00:02.734521+0000","last_undegraded":"2023-12-19T17:00:02.734521+0000","last_fullsized":"2023-12-19T17:00:02.734521+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:06:07.601962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734009+0000","last_change":"2023-12-19T17:00:01.451995+0000","last_active":"2023-12-19T17:00:02.734009+0000","last_peered":"2023-12-19T17:00:02.734009+0000","last_clean":"2023-12-19T17:00:02.734009+0000","last_became_active":"2023-12-19T17:00:01.451725+0000","last_became_peered":"2023-12-19T17:00:01.451725+0000","last_unstale":"2023-12-19T17:00:02.734009+0000","last_undegraded":"2023-12-19T17:00:02.734009+0000","last_fullsized":"2023-12-19T17:00:02.734009+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:19:01.646545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734415+0000","last_change":"2023-12-19T17:00:01.452559+0000","last_active":"2023-12-19T17:00:02.734415+0000","last_peered":"2023-12-19T17:00:02.734415+0000","last_clean":"2023-12-19T17:00:02.734415+0000","last_became_active":"2023-12-19T17:00:01.452291+0000","last_became_peered":"2023-12-19T17:00:01.452291+0000","last_unstale":"2023-12-19T17:00:02.734415+0000","last_undegraded":"2023-12-19T17:00:02.734415+0000","last_fullsized":"2023-12-19T17:00:02.734415+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:04:46.518597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437595+0000","last_change":"2023-12-19T17:00:01.442182+0000","last_active":"2023-12-19T17:00:02.437595+0000","last_peered":"2023-12-19T17:00:02.437595+0000","last_clean":"2023-12-19T17:00:02.437595+0000","last_became_active":"2023-12-19T17:00:01.441932+0000","last_became_peered":"2023-12-19T17:00:01.441932+0000","last_unstale":"2023-12-19T17:00:02.437595+0000","last_undegraded":"2023-12-19T17:00:02.437595+0000","last_fullsized":"2023-12-19T17:00:02.437595+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:41:37.856086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437543+0000","last_change":"2023-12-19T17:00:01.442454+0000","last_active":"2023-12-19T17:00:02.437543+0000","last_peered":"2023-12-19T17:00:02.437543+0000","last_clean":"2023-12-19T17:00:02.437543+0000","last_became_active":"2023-12-19T17:00:01.442208+0000","last_became_peered":"2023-12-19T17:00:01.442208+0000","last_unstale":"2023-12-19T17:00:02.437543+0000","last_undegraded":"2023-12-19T17:00:02.437543+0000","last_fullsized":"2023-12-19T17:00:02.437543+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:20.704143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733779+0000","last_change":"2023-12-19T17:00:01.457198+0000","last_active":"2023-12-19T17:00:02.733779+0000","last_peered":"2023-12-19T17:00:02.733779+0000","last_clean":"2023-12-19T17:00:02.733779+0000","last_became_active":"2023-12-19T17:00:01.456899+0000","last_became_peered":"2023-12-19T17:00:01.456899+0000","last_unstale":"2023-12-19T17:00:02.733779+0000","last_undegraded":"2023-12-19T17:00:02.733779+0000","last_fullsized":"2023-12-19T17:00:02.733779+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:24.025937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439503+0000","last_change":"2023-12-19T17:00:01.461224+0000","last_active":"2023-12-19T17:00:02.439503+0000","last_peered":"2023-12-19T17:00:02.439503+0000","last_clean":"2023-12-19T17:00:02.439503+0000","last_became_active":"2023-12-19T17:00:01.460962+0000","last_became_peered":"2023-12-19T17:00:01.460962+0000","last_unstale":"2023-12-19T17:00:02.439503+0000","last_undegraded":"2023-12-19T17:00:02.439503+0000","last_fullsized":"2023-12-19T17:00:02.439503+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:23.855329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438152+0000","last_change":"2023-12-19T17:00:01.445537+0000","last_active":"2023-12-19T17:00:02.438152+0000","last_peered":"2023-12-19T17:00:02.438152+0000","last_clean":"2023-12-19T17:00:02.438152+0000","last_became_active":"2023-12-19T17:00:01.445251+0000","last_became_peered":"2023-12-19T17:00:01.445251+0000","last_unstale":"2023-12-19T17:00:02.438152+0000","last_undegraded":"2023-12-19T17:00:02.438152+0000","last_fullsized":"2023-12-19T17:00:02.438152+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:21.544283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734115+0000","last_change":"2023-12-19T17:00:01.468834+0000","last_active":"2023-12-19T17:00:02.734115+0000","last_peered":"2023-12-19T17:00:02.734115+0000","last_clean":"2023-12-19T17:00:02.734115+0000","last_became_active":"2023-12-19T17:00:01.468472+0000","last_became_peered":"2023-12-19T17:00:01.468472+0000","last_unstale":"2023-12-19T17:00:02.734115+0000","last_undegraded":"2023-12-19T17:00:02.734115+0000","last_fullsized":"2023-12-19T17:00:02.734115+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:30:41.998190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439353+0000","last_change":"2023-12-19T17:00:01.457034+0000","last_active":"2023-12-19T17:00:02.439353+0000","last_peered":"2023-12-19T17:00:02.439353+0000","last_clean":"2023-12-19T17:00:02.439353+0000","last_became_active":"2023-12-19T17:00:01.454694+0000","last_became_peered":"2023-12-19T17:00:01.454694+0000","last_unstale":"2023-12-19T17:00:02.439353+0000","last_undegraded":"2023-12-19T17:00:02.439353+0000","last_fullsized":"2023-12-19T17:00:02.439353+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:16.412408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437627+0000","last_change":"2023-12-19T17:00:01.445931+0000","last_active":"2023-12-19T17:00:02.437627+0000","last_peered":"2023-12-19T17:00:02.437627+0000","last_clean":"2023-12-19T17:00:02.437627+0000","last_became_active":"2023-12-19T17:00:01.445229+0000","last_became_peered":"2023-12-19T17:00:01.445229+0000","last_unstale":"2023-12-19T17:00:02.437627+0000","last_undegraded":"2023-12-19T17:00:02.437627+0000","last_fullsized":"2023-12-19T17:00:02.437627+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:21:23.742043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734288+0000","last_change":"2023-12-19T17:00:01.447323+0000","last_active":"2023-12-19T17:00:02.734288+0000","last_peered":"2023-12-19T17:00:02.734288+0000","last_clean":"2023-12-19T17:00:02.734288+0000","last_became_active":"2023-12-19T17:00:01.447051+0000","last_became_peered":"2023-12-19T17:00:01.447051+0000","last_unstale":"2023-12-19T17:00:02.734288+0000","last_undegraded":"2023-12-19T17:00:02.734288+0000","last_fullsized":"2023-12-19T17:00:02.734288+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:46:40.430536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733547+0000","last_change":"2023-12-19T17:00:01.455037+0000","last_active":"2023-12-19T17:00:02.733547+0000","last_peered":"2023-12-19T17:00:02.733547+0000","last_clean":"2023-12-19T17:00:02.733547+0000","last_became_active":"2023-12-19T17:00:01.453925+0000","last_became_peered":"2023-12-19T17:00:01.453925+0000","last_unstale":"2023-12-19T17:00:02.733547+0000","last_undegraded":"2023-12-19T17:00:02.733547+0000","last_fullsized":"2023-12-19T17:00:02.733547+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:52.817577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733321+0000","last_change":"2023-12-19T17:00:01.450713+0000","last_active":"2023-12-19T17:00:02.733321+0000","last_peered":"2023-12-19T17:00:02.733321+0000","last_clean":"2023-12-19T17:00:02.733321+0000","last_became_active":"2023-12-19T17:00:01.450454+0000","last_became_peered":"2023-12-19T17:00:01.450454+0000","last_unstale":"2023-12-19T17:00:02.733321+0000","last_undegraded":"2023-12-19T17:00:02.733321+0000","last_fullsized":"2023-12-19T17:00:02.733321+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:34.348541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734484+0000","last_change":"2023-12-19T17:00:01.452401+0000","last_active":"2023-12-19T17:00:02.734484+0000","last_peered":"2023-12-19T17:00:02.734484+0000","last_clean":"2023-12-19T17:00:02.734484+0000","last_became_active":"2023-12-19T17:00:01.452129+0000","last_became_peered":"2023-12-19T17:00:01.452129+0000","last_unstale":"2023-12-19T17:00:02.734484+0000","last_undegraded":"2023-12-19T17:00:02.734484+0000","last_fullsized":"2023-12-19T17:00:02.734484+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:02.080480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"122'4","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438256+0000","last_change":"2023-12-19T17:00:01.047133+0000","last_active":"2023-12-19T17:00:02.438256+0000","last_peered":"2023-12-19T17:00:02.438256+0000","last_clean":"2023-12-19T17:00:02.438256+0000","last_became_active":"2023-12-19T17:00:01.046831+0000","last_became_peered":"2023-12-19T17:00:01.046831+0000","last_unstale":"2023-12-19T17:00:02.438256+0000","last_undegraded":"2023-12-19T17:00:02.438256+0000","last_fullsized":"2023-12-19T17:00:02.438256+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:15:35.770496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734269+0000","last_change":"2023-12-19T17:00:01.453768+0000","last_active":"2023-12-19T17:00:02.734269+0000","last_peered":"2023-12-19T17:00:02.734269+0000","last_clean":"2023-12-19T17:00:02.734269+0000","last_became_active":"2023-12-19T17:00:01.453444+0000","last_became_peered":"2023-12-19T17:00:01.453444+0000","last_unstale":"2023-12-19T17:00:02.734269+0000","last_undegraded":"2023-12-19T17:00:02.734269+0000","last_fullsized":"2023-12-19T17:00:02.734269+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:59.365491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"122'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733998+0000","last_change":"2023-12-19T17:00:01.050160+0000","last_active":"2023-12-19T17:00:02.733998+0000","last_peered":"2023-12-19T17:00:02.733998+0000","last_clean":"2023-12-19T17:00:02.733998+0000","last_became_active":"2023-12-19T17:00:01.049851+0000","last_became_peered":"2023-12-19T17:00:01.049851+0000","last_unstale":"2023-12-19T17:00:02.733998+0000","last_undegraded":"2023-12-19T17:00:02.733998+0000","last_fullsized":"2023-12-19T17:00:02.733998+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:33.346149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733522+0000","last_change":"2023-12-19T17:00:01.442685+0000","last_active":"2023-12-19T17:00:02.733522+0000","last_peered":"2023-12-19T17:00:02.733522+0000","last_clean":"2023-12-19T17:00:02.733522+0000","last_became_active":"2023-12-19T17:00:01.442363+0000","last_became_peered":"2023-12-19T17:00:01.442363+0000","last_unstale":"2023-12-19T17:00:02.733522+0000","last_undegraded":"2023-12-19T17:00:02.733522+0000","last_fullsized":"2023-12-19T17:00:02.733522+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:04:18.408128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438059+0000","last_change":"2023-12-19T17:00:01.053355+0000","last_active":"2023-12-19T17:00:02.438059+0000","last_peered":"2023-12-19T17:00:02.438059+0000","last_clean":"2023-12-19T17:00:02.438059+0000","last_became_active":"2023-12-19T17:00:01.053008+0000","last_became_peered":"2023-12-19T17:00:01.053008+0000","last_unstale":"2023-12-19T17:00:02.438059+0000","last_undegraded":"2023-12-19T17:00:02.438059+0000","last_fullsized":"2023-12-19T17:00:02.438059+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:12.935403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734243+0000","last_change":"2023-12-19T17:00:01.453056+0000","last_active":"2023-12-19T17:00:02.734243+0000","last_peered":"2023-12-19T17:00:02.734243+0000","last_clean":"2023-12-19T17:00:02.734243+0000","last_became_active":"2023-12-19T17:00:01.452756+0000","last_became_peered":"2023-12-19T17:00:01.452756+0000","last_unstale":"2023-12-19T17:00:02.734243+0000","last_undegraded":"2023-12-19T17:00:02.734243+0000","last_fullsized":"2023-12-19T17:00:02.734243+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:42:36.359289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437904+0000","last_change":"2023-12-19T17:00:01.041483+0000","last_active":"2023-12-19T17:00:02.437904+0000","last_peered":"2023-12-19T17:00:02.437904+0000","last_clean":"2023-12-19T17:00:02.437904+0000","last_became_active":"2023-12-19T17:00:01.041219+0000","last_became_peered":"2023-12-19T17:00:01.041219+0000","last_unstale":"2023-12-19T17:00:02.437904+0000","last_undegraded":"2023-12-19T17:00:02.437904+0000","last_fullsized":"2023-12-19T17:00:02.437904+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:42:38.232078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438375+0000","last_change":"2023-12-19T17:00:01.448953+0000","last_active":"2023-12-19T17:00:02.438375+0000","last_peered":"2023-12-19T17:00:02.438375+0000","last_clean":"2023-12-19T17:00:02.438375+0000","last_became_active":"2023-12-19T17:00:01.448660+0000","last_became_peered":"2023-12-19T17:00:01.448660+0000","last_unstale":"2023-12-19T17:00:02.438375+0000","last_undegraded":"2023-12-19T17:00:02.438375+0000","last_fullsized":"2023-12-19T17:00:02.438375+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:41:06.018409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"122'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733622+0000","last_change":"2023-12-19T17:00:01.045175+0000","last_active":"2023-12-19T17:00:02.733622+0000","last_peered":"2023-12-19T17:00:02.733622+0000","last_clean":"2023-12-19T17:00:02.733622+0000","last_became_active":"2023-12-19T17:00:01.044877+0000","last_became_peered":"2023-12-19T17:00:01.044877+0000","last_unstale":"2023-12-19T17:00:02.733622+0000","last_undegraded":"2023-12-19T17:00:02.733622+0000","last_fullsized":"2023-12-19T17:00:02.733622+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:42:45.648537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733836+0000","last_change":"2023-12-19T17:00:01.457482+0000","last_active":"2023-12-19T17:00:02.733836+0000","last_peered":"2023-12-19T17:00:02.733836+0000","last_clean":"2023-12-19T17:00:02.733836+0000","last_became_active":"2023-12-19T17:00:01.456496+0000","last_became_peered":"2023-12-19T17:00:01.456496+0000","last_unstale":"2023-12-19T17:00:02.733836+0000","last_undegraded":"2023-12-19T17:00:02.733836+0000","last_fullsized":"2023-12-19T17:00:02.733836+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:19:19.027263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"122'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437621+0000","last_change":"2023-12-19T17:00:01.046428+0000","last_active":"2023-12-19T17:00:02.437621+0000","last_peered":"2023-12-19T17:00:02.437621+0000","last_clean":"2023-12-19T17:00:02.437621+0000","last_became_active":"2023-12-19T17:00:01.046039+0000","last_became_peered":"2023-12-19T17:00:01.046039+0000","last_unstale":"2023-12-19T17:00:02.437621+0000","last_undegraded":"2023-12-19T17:00:02.437621+0000","last_fullsized":"2023-12-19T17:00:02.437621+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:56.299107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733428+0000","last_change":"2023-12-19T17:00:01.458952+0000","last_active":"2023-12-19T17:00:02.733428+0000","last_peered":"2023-12-19T17:00:02.733428+0000","last_clean":"2023-12-19T17:00:02.733428+0000","last_became_active":"2023-12-19T17:00:01.458691+0000","last_became_peered":"2023-12-19T17:00:01.458691+0000","last_unstale":"2023-12-19T17:00:02.733428+0000","last_undegraded":"2023-12-19T17:00:02.733428+0000","last_fullsized":"2023-12-19T17:00:02.733428+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:16.058767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438928+0000","last_change":"2023-12-19T17:00:01.048430+0000","last_active":"2023-12-19T17:00:02.438928+0000","last_peered":"2023-12-19T17:00:02.438928+0000","last_clean":"2023-12-19T17:00:02.438928+0000","last_became_active":"2023-12-19T17:00:01.048154+0000","last_became_peered":"2023-12-19T17:00:01.048154+0000","last_unstale":"2023-12-19T17:00:02.438928+0000","last_undegraded":"2023-12-19T17:00:02.438928+0000","last_fullsized":"2023-12-19T17:00:02.438928+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:25:34.382256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733479+0000","last_change":"2023-12-19T17:00:01.446763+0000","last_active":"2023-12-19T17:00:02.733479+0000","last_peered":"2023-12-19T17:00:02.733479+0000","last_clean":"2023-12-19T17:00:02.733479+0000","last_became_active":"2023-12-19T17:00:01.446409+0000","last_became_peered":"2023-12-19T17:00:01.446409+0000","last_unstale":"2023-12-19T17:00:02.733479+0000","last_undegraded":"2023-12-19T17:00:02.733479+0000","last_fullsized":"2023-12-19T17:00:02.733479+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:17.713011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733408+0000","last_change":"2023-12-19T17:00:01.044270+0000","last_active":"2023-12-19T17:00:02.733408+0000","last_peered":"2023-12-19T17:00:02.733408+0000","last_clean":"2023-12-19T17:00:02.733408+0000","last_became_active":"2023-12-19T17:00:01.042770+0000","last_became_peered":"2023-12-19T17:00:01.042770+0000","last_unstale":"2023-12-19T17:00:02.733408+0000","last_undegraded":"2023-12-19T17:00:02.733408+0000","last_fullsized":"2023-12-19T17:00:02.733408+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:41:48.345922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438189+0000","last_change":"2023-12-19T17:00:01.437736+0000","last_active":"2023-12-19T17:00:02.438189+0000","last_peered":"2023-12-19T17:00:02.438189+0000","last_clean":"2023-12-19T17:00:02.438189+0000","last_became_active":"2023-12-19T17:00:01.437469+0000","last_became_peered":"2023-12-19T17:00:01.437469+0000","last_unstale":"2023-12-19T17:00:02.438189+0000","last_undegraded":"2023-12-19T17:00:02.438189+0000","last_fullsized":"2023-12-19T17:00:02.438189+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:08:30.255976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733929+0000","last_change":"2023-12-19T17:00:01.037812+0000","last_active":"2023-12-19T17:00:02.733929+0000","last_peered":"2023-12-19T17:00:02.733929+0000","last_clean":"2023-12-19T17:00:02.733929+0000","last_became_active":"2023-12-19T17:00:01.037549+0000","last_became_peered":"2023-12-19T17:00:01.037549+0000","last_unstale":"2023-12-19T17:00:02.733929+0000","last_undegraded":"2023-12-19T17:00:02.733929+0000","last_fullsized":"2023-12-19T17:00:02.733929+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:20:48.883394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734519+0000","last_change":"2023-12-19T17:00:01.447194+0000","last_active":"2023-12-19T17:00:02.734519+0000","last_peered":"2023-12-19T17:00:02.734519+0000","last_clean":"2023-12-19T17:00:02.734519+0000","last_became_active":"2023-12-19T17:00:01.446882+0000","last_became_peered":"2023-12-19T17:00:01.446882+0000","last_unstale":"2023-12-19T17:00:02.734519+0000","last_undegraded":"2023-12-19T17:00:02.734519+0000","last_fullsized":"2023-12-19T17:00:02.734519+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:04:18.408128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734135+0000","last_change":"2023-12-19T17:00:01.043355+0000","last_active":"2023-12-19T17:00:02.734135+0000","last_peered":"2023-12-19T17:00:02.734135+0000","last_clean":"2023-12-19T17:00:02.734135+0000","last_became_active":"2023-12-19T17:00:01.040660+0000","last_became_peered":"2023-12-19T17:00:01.040660+0000","last_unstale":"2023-12-19T17:00:02.734135+0000","last_undegraded":"2023-12-19T17:00:02.734135+0000","last_fullsized":"2023-12-19T17:00:02.734135+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:04:05.450288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439504+0000","last_change":"2023-12-19T17:00:01.459130+0000","last_active":"2023-12-19T17:00:02.439504+0000","last_peered":"2023-12-19T17:00:02.439504+0000","last_clean":"2023-12-19T17:00:02.439504+0000","last_became_active":"2023-12-19T17:00:01.458735+0000","last_became_peered":"2023-12-19T17:00:01.458735+0000","last_unstale":"2023-12-19T17:00:02.439504+0000","last_undegraded":"2023-12-19T17:00:02.439504+0000","last_fullsized":"2023-12-19T17:00:02.439504+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:43:15.214527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"122'3","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734335+0000","last_change":"2023-12-19T17:00:01.045639+0000","last_active":"2023-12-19T17:00:02.734335+0000","last_peered":"2023-12-19T17:00:02.734335+0000","last_clean":"2023-12-19T17:00:02.734335+0000","last_became_active":"2023-12-19T17:00:01.045370+0000","last_became_peered":"2023-12-19T17:00:01.045370+0000","last_unstale":"2023-12-19T17:00:02.734335+0000","last_undegraded":"2023-12-19T17:00:02.734335+0000","last_fullsized":"2023-12-19T17:00:02.734335+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:34.311769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734196+0000","last_change":"2023-12-19T17:00:01.464531+0000","last_active":"2023-12-19T17:00:02.734196+0000","last_peered":"2023-12-19T17:00:02.734196+0000","last_clean":"2023-12-19T17:00:02.734196+0000","last_became_active":"2023-12-19T17:00:01.464220+0000","last_became_peered":"2023-12-19T17:00:01.464220+0000","last_unstale":"2023-12-19T17:00:02.734196+0000","last_undegraded":"2023-12-19T17:00:02.734196+0000","last_fullsized":"2023-12-19T17:00:02.734196+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:19.044976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"122'6","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734068+0000","last_change":"2023-12-19T17:00:01.045307+0000","last_active":"2023-12-19T17:00:02.734068+0000","last_peered":"2023-12-19T17:00:02.734068+0000","last_clean":"2023-12-19T17:00:02.734068+0000","last_became_active":"2023-12-19T17:00:01.045019+0000","last_became_peered":"2023-12-19T17:00:01.045019+0000","last_unstale":"2023-12-19T17:00:02.734068+0000","last_undegraded":"2023-12-19T17:00:02.734068+0000","last_fullsized":"2023-12-19T17:00:02.734068+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:42:45.648537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733680+0000","last_change":"2023-12-19T17:00:01.451874+0000","last_active":"2023-12-19T17:00:02.733680+0000","last_peered":"2023-12-19T17:00:02.733680+0000","last_clean":"2023-12-19T17:00:02.733680+0000","last_became_active":"2023-12-19T17:00:01.451583+0000","last_became_peered":"2023-12-19T17:00:01.451583+0000","last_unstale":"2023-12-19T17:00:02.733680+0000","last_undegraded":"2023-12-19T17:00:02.733680+0000","last_fullsized":"2023-12-19T17:00:02.733680+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:56.037615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734623+0000","last_change":"2023-12-19T17:00:01.045633+0000","last_active":"2023-12-19T17:00:02.734623+0000","last_peered":"2023-12-19T17:00:02.734623+0000","last_clean":"2023-12-19T17:00:02.734623+0000","last_became_active":"2023-12-19T17:00:01.045311+0000","last_became_peered":"2023-12-19T17:00:01.045311+0000","last_unstale":"2023-12-19T17:00:02.734623+0000","last_undegraded":"2023-12-19T17:00:02.734623+0000","last_fullsized":"2023-12-19T17:00:02.734623+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:17:31.980824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733560+0000","last_change":"2023-12-19T17:00:01.454203+0000","last_active":"2023-12-19T17:00:02.733560+0000","last_peered":"2023-12-19T17:00:02.733560+0000","last_clean":"2023-12-19T17:00:02.733560+0000","last_became_active":"2023-12-19T17:00:01.453940+0000","last_became_peered":"2023-12-19T17:00:01.453940+0000","last_unstale":"2023-12-19T17:00:02.733560+0000","last_undegraded":"2023-12-19T17:00:02.733560+0000","last_fullsized":"2023-12-19T17:00:02.733560+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:17.713011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734438+0000","last_change":"2023-12-19T17:00:01.047398+0000","last_active":"2023-12-19T17:00:02.734438+0000","last_peered":"2023-12-19T17:00:02.734438+0000","last_clean":"2023-12-19T17:00:02.734438+0000","last_became_active":"2023-12-19T17:00:01.047119+0000","last_became_peered":"2023-12-19T17:00:01.047119+0000","last_unstale":"2023-12-19T17:00:02.734438+0000","last_undegraded":"2023-12-19T17:00:02.734438+0000","last_fullsized":"2023-12-19T17:00:02.734438+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:04.654653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733453+0000","last_change":"2023-12-19T17:00:01.442043+0000","last_active":"2023-12-19T17:00:02.733453+0000","last_peered":"2023-12-19T17:00:02.733453+0000","last_clean":"2023-12-19T17:00:02.733453+0000","last_became_active":"2023-12-19T17:00:01.441759+0000","last_became_peered":"2023-12-19T17:00:01.441759+0000","last_unstale":"2023-12-19T17:00:02.733453+0000","last_undegraded":"2023-12-19T17:00:02.733453+0000","last_fullsized":"2023-12-19T17:00:02.733453+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:12:00.585820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733539+0000","last_change":"2023-12-19T17:00:01.041719+0000","last_active":"2023-12-19T17:00:02.733539+0000","last_peered":"2023-12-19T17:00:02.733539+0000","last_clean":"2023-12-19T17:00:02.733539+0000","last_became_active":"2023-12-19T17:00:01.041318+0000","last_became_peered":"2023-12-19T17:00:01.041318+0000","last_unstale":"2023-12-19T17:00:02.733539+0000","last_undegraded":"2023-12-19T17:00:02.733539+0000","last_fullsized":"2023-12-19T17:00:02.733539+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:52:11.763337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734172+0000","last_change":"2023-12-19T17:00:01.448648+0000","last_active":"2023-12-19T17:00:02.734172+0000","last_peered":"2023-12-19T17:00:02.734172+0000","last_clean":"2023-12-19T17:00:02.734172+0000","last_became_active":"2023-12-19T17:00:01.448163+0000","last_became_peered":"2023-12-19T17:00:01.448163+0000","last_unstale":"2023-12-19T17:00:02.734172+0000","last_undegraded":"2023-12-19T17:00:02.734172+0000","last_fullsized":"2023-12-19T17:00:02.734172+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:25.056126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"122'8","reported_seq":19,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733521+0000","last_change":"2023-12-19T17:00:01.046175+0000","last_active":"2023-12-19T17:00:02.733521+0000","last_peered":"2023-12-19T17:00:02.733521+0000","last_clean":"2023-12-19T17:00:02.733521+0000","last_became_active":"2023-12-19T17:00:01.045397+0000","last_became_peered":"2023-12-19T17:00:01.045397+0000","last_unstale":"2023-12-19T17:00:02.733521+0000","last_undegraded":"2023-12-19T17:00:02.733521+0000","last_fullsized":"2023-12-19T17:00:02.733521+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:19:52.426515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439690+0000","last_change":"2023-12-19T17:00:01.449596+0000","last_active":"2023-12-19T17:00:02.439690+0000","last_peered":"2023-12-19T17:00:02.439690+0000","last_clean":"2023-12-19T17:00:02.439690+0000","last_became_active":"2023-12-19T17:00:01.449172+0000","last_became_peered":"2023-12-19T17:00:01.449172+0000","last_unstale":"2023-12-19T17:00:02.439690+0000","last_undegraded":"2023-12-19T17:00:02.439690+0000","last_fullsized":"2023-12-19T17:00:02.439690+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:34.348541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734389+0000","last_change":"2023-12-19T17:00:01.044427+0000","last_active":"2023-12-19T17:00:02.734389+0000","last_peered":"2023-12-19T17:00:02.734389+0000","last_clean":"2023-12-19T17:00:02.734389+0000","last_became_active":"2023-12-19T17:00:01.044072+0000","last_became_peered":"2023-12-19T17:00:01.044072+0000","last_unstale":"2023-12-19T17:00:02.734389+0000","last_undegraded":"2023-12-19T17:00:02.734389+0000","last_fullsized":"2023-12-19T17:00:02.734389+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:12:24.444051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438572+0000","last_change":"2023-12-19T17:00:01.455551+0000","last_active":"2023-12-19T17:00:02.438572+0000","last_peered":"2023-12-19T17:00:02.438572+0000","last_clean":"2023-12-19T17:00:02.438572+0000","last_became_active":"2023-12-19T17:00:01.455284+0000","last_became_peered":"2023-12-19T17:00:01.455284+0000","last_unstale":"2023-12-19T17:00:02.438572+0000","last_undegraded":"2023-12-19T17:00:02.438572+0000","last_fullsized":"2023-12-19T17:00:02.438572+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:01:22.807158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"122'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439455+0000","last_change":"2023-12-19T17:00:01.040987+0000","last_active":"2023-12-19T17:00:02.439455+0000","last_peered":"2023-12-19T17:00:02.439455+0000","last_clean":"2023-12-19T17:00:02.439455+0000","last_became_active":"2023-12-19T17:00:01.040684+0000","last_became_peered":"2023-12-19T17:00:01.040684+0000","last_unstale":"2023-12-19T17:00:02.439455+0000","last_undegraded":"2023-12-19T17:00:02.439455+0000","last_fullsized":"2023-12-19T17:00:02.439455+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:55:15.222559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734211+0000","last_change":"2023-12-19T17:00:01.461410+0000","last_active":"2023-12-19T17:00:02.734211+0000","last_peered":"2023-12-19T17:00:02.734211+0000","last_clean":"2023-12-19T17:00:02.734211+0000","last_became_active":"2023-12-19T17:00:01.461125+0000","last_became_peered":"2023-12-19T17:00:01.461125+0000","last_unstale":"2023-12-19T17:00:02.734211+0000","last_undegraded":"2023-12-19T17:00:02.734211+0000","last_fullsized":"2023-12-19T17:00:02.734211+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:45:54.781258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438008+0000","last_change":"2023-12-19T17:00:01.040204+0000","last_active":"2023-12-19T17:00:02.438008+0000","last_peered":"2023-12-19T17:00:02.438008+0000","last_clean":"2023-12-19T17:00:02.438008+0000","last_became_active":"2023-12-19T17:00:01.039889+0000","last_became_peered":"2023-12-19T17:00:01.039889+0000","last_unstale":"2023-12-19T17:00:02.438008+0000","last_undegraded":"2023-12-19T17:00:02.438008+0000","last_fullsized":"2023-12-19T17:00:02.438008+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:24:17.214776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734351+0000","last_change":"2023-12-19T17:00:01.456618+0000","last_active":"2023-12-19T17:00:02.734351+0000","last_peered":"2023-12-19T17:00:02.734351+0000","last_clean":"2023-12-19T17:00:02.734351+0000","last_became_active":"2023-12-19T17:00:01.456320+0000","last_became_peered":"2023-12-19T17:00:01.456320+0000","last_unstale":"2023-12-19T17:00:02.734351+0000","last_undegraded":"2023-12-19T17:00:02.734351+0000","last_fullsized":"2023-12-19T17:00:02.734351+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:06.671287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437723+0000","last_change":"2023-12-19T17:00:01.048293+0000","last_active":"2023-12-19T17:00:02.437723+0000","last_peered":"2023-12-19T17:00:02.437723+0000","last_clean":"2023-12-19T17:00:02.437723+0000","last_became_active":"2023-12-19T17:00:01.047807+0000","last_became_peered":"2023-12-19T17:00:01.047807+0000","last_unstale":"2023-12-19T17:00:02.437723+0000","last_undegraded":"2023-12-19T17:00:02.437723+0000","last_fullsized":"2023-12-19T17:00:02.437723+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:14:26.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734526+0000","last_change":"2023-12-19T17:00:01.446512+0000","last_active":"2023-12-19T17:00:02.734526+0000","last_peered":"2023-12-19T17:00:02.734526+0000","last_clean":"2023-12-19T17:00:02.734526+0000","last_became_active":"2023-12-19T17:00:01.446250+0000","last_became_peered":"2023-12-19T17:00:01.446250+0000","last_unstale":"2023-12-19T17:00:02.734526+0000","last_undegraded":"2023-12-19T17:00:02.734526+0000","last_fullsized":"2023-12-19T17:00:02.734526+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:57:49.332518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438505+0000","last_change":"2023-12-19T17:00:01.044948+0000","last_active":"2023-12-19T17:00:02.438505+0000","last_peered":"2023-12-19T17:00:02.438505+0000","last_clean":"2023-12-19T17:00:02.438505+0000","last_became_active":"2023-12-19T17:00:01.044501+0000","last_became_peered":"2023-12-19T17:00:01.044501+0000","last_unstale":"2023-12-19T17:00:02.438505+0000","last_undegraded":"2023-12-19T17:00:02.438505+0000","last_fullsized":"2023-12-19T17:00:02.438505+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:03:33.414677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733877+0000","last_change":"2023-12-19T17:00:01.459066+0000","last_active":"2023-12-19T17:00:02.733877+0000","last_peered":"2023-12-19T17:00:02.733877+0000","last_clean":"2023-12-19T17:00:02.733877+0000","last_became_active":"2023-12-19T17:00:01.457362+0000","last_became_peered":"2023-12-19T17:00:01.457362+0000","last_unstale":"2023-12-19T17:00:02.733877+0000","last_undegraded":"2023-12-19T17:00:02.733877+0000","last_fullsized":"2023-12-19T17:00:02.733877+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:31.360615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438370+0000","last_change":"2023-12-19T17:00:01.046842+0000","last_active":"2023-12-19T17:00:02.438370+0000","last_peered":"2023-12-19T17:00:02.438370+0000","last_clean":"2023-12-19T17:00:02.438370+0000","last_became_active":"2023-12-19T17:00:01.046493+0000","last_became_peered":"2023-12-19T17:00:01.046493+0000","last_unstale":"2023-12-19T17:00:02.438370+0000","last_undegraded":"2023-12-19T17:00:02.438370+0000","last_fullsized":"2023-12-19T17:00:02.438370+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:20:48.883394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438729+0000","last_change":"2023-12-19T17:00:01.448890+0000","last_active":"2023-12-19T17:00:02.438729+0000","last_peered":"2023-12-19T17:00:02.438729+0000","last_clean":"2023-12-19T17:00:02.438729+0000","last_became_active":"2023-12-19T17:00:01.447594+0000","last_became_peered":"2023-12-19T17:00:01.447594+0000","last_unstale":"2023-12-19T17:00:02.438729+0000","last_undegraded":"2023-12-19T17:00:02.438729+0000","last_fullsized":"2023-12-19T17:00:02.438729+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:18:24.878752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437562+0000","last_change":"2023-12-19T17:00:01.046453+0000","last_active":"2023-12-19T17:00:02.437562+0000","last_peered":"2023-12-19T17:00:02.437562+0000","last_clean":"2023-12-19T17:00:02.437562+0000","last_became_active":"2023-12-19T17:00:01.046170+0000","last_became_peered":"2023-12-19T17:00:01.046170+0000","last_unstale":"2023-12-19T17:00:02.437562+0000","last_undegraded":"2023-12-19T17:00:02.437562+0000","last_fullsized":"2023-12-19T17:00:02.437562+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:43:01.663266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438275+0000","last_change":"2023-12-19T17:00:01.448980+0000","last_active":"2023-12-19T17:00:02.438275+0000","last_peered":"2023-12-19T17:00:02.438275+0000","last_clean":"2023-12-19T17:00:02.438275+0000","last_became_active":"2023-12-19T17:00:01.448727+0000","last_became_peered":"2023-12-19T17:00:01.448727+0000","last_unstale":"2023-12-19T17:00:02.438275+0000","last_undegraded":"2023-12-19T17:00:02.438275+0000","last_fullsized":"2023-12-19T17:00:02.438275+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:59:55.634822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439547+0000","last_change":"2023-12-19T17:00:01.037313+0000","last_active":"2023-12-19T17:00:02.439547+0000","last_peered":"2023-12-19T17:00:02.439547+0000","last_clean":"2023-12-19T17:00:02.439547+0000","last_became_active":"2023-12-19T17:00:01.036914+0000","last_became_peered":"2023-12-19T17:00:01.036914+0000","last_unstale":"2023-12-19T17:00:02.439547+0000","last_undegraded":"2023-12-19T17:00:02.439547+0000","last_fullsized":"2023-12-19T17:00:02.439547+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:41:48.345922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733844+0000","last_change":"2023-12-19T17:00:01.446677+0000","last_active":"2023-12-19T17:00:02.733844+0000","last_peered":"2023-12-19T17:00:02.733844+0000","last_clean":"2023-12-19T17:00:02.733844+0000","last_became_active":"2023-12-19T17:00:01.446402+0000","last_became_peered":"2023-12-19T17:00:01.446402+0000","last_unstale":"2023-12-19T17:00:02.733844+0000","last_undegraded":"2023-12-19T17:00:02.733844+0000","last_fullsized":"2023-12-19T17:00:02.733844+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:13:24.059700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438240+0000","last_change":"2023-12-19T17:00:01.047171+0000","last_active":"2023-12-19T17:00:02.438240+0000","last_peered":"2023-12-19T17:00:02.438240+0000","last_clean":"2023-12-19T17:00:02.438240+0000","last_became_active":"2023-12-19T17:00:01.046910+0000","last_became_peered":"2023-12-19T17:00:01.046910+0000","last_unstale":"2023-12-19T17:00:02.438240+0000","last_undegraded":"2023-12-19T17:00:02.438240+0000","last_fullsized":"2023-12-19T17:00:02.438240+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:38:56.371491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733628+0000","last_change":"2023-12-19T17:00:01.456285+0000","last_active":"2023-12-19T17:00:02.733628+0000","last_peered":"2023-12-19T17:00:02.733628+0000","last_clean":"2023-12-19T17:00:02.733628+0000","last_became_active":"2023-12-19T17:00:01.456005+0000","last_became_peered":"2023-12-19T17:00:01.456005+0000","last_unstale":"2023-12-19T17:00:02.733628+0000","last_undegraded":"2023-12-19T17:00:02.733628+0000","last_fullsized":"2023-12-19T17:00:02.733628+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:19:01.646545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438094+0000","last_change":"2023-12-19T17:00:01.041852+0000","last_active":"2023-12-19T17:00:02.438094+0000","last_peered":"2023-12-19T17:00:02.438094+0000","last_clean":"2023-12-19T17:00:02.438094+0000","last_became_active":"2023-12-19T17:00:01.039894+0000","last_became_peered":"2023-12-19T17:00:01.039894+0000","last_unstale":"2023-12-19T17:00:02.438094+0000","last_undegraded":"2023-12-19T17:00:02.438094+0000","last_fullsized":"2023-12-19T17:00:02.438094+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:27.050279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439343+0000","last_change":"2023-12-19T17:00:01.453472+0000","last_active":"2023-12-19T17:00:02.439343+0000","last_peered":"2023-12-19T17:00:02.439343+0000","last_clean":"2023-12-19T17:00:02.439343+0000","last_became_active":"2023-12-19T17:00:01.453172+0000","last_became_peered":"2023-12-19T17:00:01.453172+0000","last_unstale":"2023-12-19T17:00:02.439343+0000","last_undegraded":"2023-12-19T17:00:02.439343+0000","last_fullsized":"2023-12-19T17:00:02.439343+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:21.252414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439591+0000","last_change":"2023-12-19T17:00:01.043008+0000","last_active":"2023-12-19T17:00:02.439591+0000","last_peered":"2023-12-19T17:00:02.439591+0000","last_clean":"2023-12-19T17:00:02.439591+0000","last_became_active":"2023-12-19T17:00:01.042702+0000","last_became_peered":"2023-12-19T17:00:01.042702+0000","last_unstale":"2023-12-19T17:00:02.439591+0000","last_undegraded":"2023-12-19T17:00:02.439591+0000","last_fullsized":"2023-12-19T17:00:02.439591+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:11.270666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734398+0000","last_change":"2023-12-19T17:00:01.450048+0000","last_active":"2023-12-19T17:00:02.734398+0000","last_peered":"2023-12-19T17:00:02.734398+0000","last_clean":"2023-12-19T17:00:02.734398+0000","last_became_active":"2023-12-19T17:00:01.448266+0000","last_became_peered":"2023-12-19T17:00:01.448266+0000","last_unstale":"2023-12-19T17:00:02.734398+0000","last_undegraded":"2023-12-19T17:00:02.734398+0000","last_fullsized":"2023-12-19T17:00:02.734398+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:42.373477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437855+0000","last_change":"2023-12-19T17:00:01.048091+0000","last_active":"2023-12-19T17:00:02.437855+0000","last_peered":"2023-12-19T17:00:02.437855+0000","last_clean":"2023-12-19T17:00:02.437855+0000","last_became_active":"2023-12-19T17:00:01.047443+0000","last_became_peered":"2023-12-19T17:00:01.047443+0000","last_unstale":"2023-12-19T17:00:02.437855+0000","last_undegraded":"2023-12-19T17:00:02.437855+0000","last_fullsized":"2023-12-19T17:00:02.437855+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:24:44.265597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733802+0000","last_change":"2023-12-19T17:00:01.447116+0000","last_active":"2023-12-19T17:00:02.733802+0000","last_peered":"2023-12-19T17:00:02.733802+0000","last_clean":"2023-12-19T17:00:02.733802+0000","last_became_active":"2023-12-19T17:00:01.445512+0000","last_became_peered":"2023-12-19T17:00:01.445512+0000","last_unstale":"2023-12-19T17:00:02.733802+0000","last_undegraded":"2023-12-19T17:00:02.733802+0000","last_fullsized":"2023-12-19T17:00:02.733802+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:13:24.059700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438859+0000","last_change":"2023-12-19T17:00:01.050200+0000","last_active":"2023-12-19T17:00:02.438859+0000","last_peered":"2023-12-19T17:00:02.438859+0000","last_clean":"2023-12-19T17:00:02.438859+0000","last_became_active":"2023-12-19T17:00:01.049811+0000","last_became_peered":"2023-12-19T17:00:01.049811+0000","last_unstale":"2023-12-19T17:00:02.438859+0000","last_undegraded":"2023-12-19T17:00:02.438859+0000","last_fullsized":"2023-12-19T17:00:02.438859+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:44:47.234295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733983+0000","last_change":"2023-12-19T17:00:01.449051+0000","last_active":"2023-12-19T17:00:02.733983+0000","last_peered":"2023-12-19T17:00:02.733983+0000","last_clean":"2023-12-19T17:00:02.733983+0000","last_became_active":"2023-12-19T17:00:01.448269+0000","last_became_peered":"2023-12-19T17:00:01.448269+0000","last_unstale":"2023-12-19T17:00:02.733983+0000","last_undegraded":"2023-12-19T17:00:02.733983+0000","last_fullsized":"2023-12-19T17:00:02.733983+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:51:42.827617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733646+0000","last_change":"2023-12-19T17:00:01.049007+0000","last_active":"2023-12-19T17:00:02.733646+0000","last_peered":"2023-12-19T17:00:02.733646+0000","last_clean":"2023-12-19T17:00:02.733646+0000","last_became_active":"2023-12-19T17:00:01.048650+0000","last_became_peered":"2023-12-19T17:00:01.048650+0000","last_unstale":"2023-12-19T17:00:02.733646+0000","last_undegraded":"2023-12-19T17:00:02.733646+0000","last_fullsized":"2023-12-19T17:00:02.733646+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:52.236059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438836+0000","last_change":"2023-12-19T17:00:01.451719+0000","last_active":"2023-12-19T17:00:02.438836+0000","last_peered":"2023-12-19T17:00:02.438836+0000","last_clean":"2023-12-19T17:00:02.438836+0000","last_became_active":"2023-12-19T17:00:01.450749+0000","last_became_peered":"2023-12-19T17:00:01.450749+0000","last_unstale":"2023-12-19T17:00:02.438836+0000","last_undegraded":"2023-12-19T17:00:02.438836+0000","last_fullsized":"2023-12-19T17:00:02.438836+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:54.706706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734034+0000","last_change":"2023-12-19T17:00:01.046499+0000","last_active":"2023-12-19T17:00:02.734034+0000","last_peered":"2023-12-19T17:00:02.734034+0000","last_clean":"2023-12-19T17:00:02.734034+0000","last_became_active":"2023-12-19T17:00:01.043118+0000","last_became_peered":"2023-12-19T17:00:01.043118+0000","last_unstale":"2023-12-19T17:00:02.734034+0000","last_undegraded":"2023-12-19T17:00:02.734034+0000","last_fullsized":"2023-12-19T17:00:02.734034+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:54.763702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734585+0000","last_change":"2023-12-19T17:00:01.456952+0000","last_active":"2023-12-19T17:00:02.734585+0000","last_peered":"2023-12-19T17:00:02.734585+0000","last_clean":"2023-12-19T17:00:02.734585+0000","last_became_active":"2023-12-19T17:00:01.456431+0000","last_became_peered":"2023-12-19T17:00:01.456431+0000","last_unstale":"2023-12-19T17:00:02.734585+0000","last_undegraded":"2023-12-19T17:00:02.734585+0000","last_fullsized":"2023-12-19T17:00:02.734585+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:02:56.620726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"122'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734485+0000","last_change":"2023-12-19T17:00:01.045830+0000","last_active":"2023-12-19T17:00:02.734485+0000","last_peered":"2023-12-19T17:00:02.734485+0000","last_clean":"2023-12-19T17:00:02.734485+0000","last_became_active":"2023-12-19T17:00:01.045145+0000","last_became_peered":"2023-12-19T17:00:01.045145+0000","last_unstale":"2023-12-19T17:00:02.734485+0000","last_undegraded":"2023-12-19T17:00:02.734485+0000","last_fullsized":"2023-12-19T17:00:02.734485+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:43:14.212135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"111'225","reported_seq":639,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-19T17:00:01.444297+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T17:00:01.444297+0000","last_peered":"2023-12-19T17:00:01.444297+0000","last_clean":"2023-12-19T17:00:01.444297+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T17:00:01.444297+0000","last_undegraded":"2023-12-19T17:00:01.444297+0000","last_fullsized":"2023-12-19T17:00:01.444297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1058,"num_read_kb":2145,"num_write":355,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1058,"num_read_kb":2145,"num_write":355,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509653,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":2101248,"data_stored":1350689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.504,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":1.04,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.57999999999999996,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":0.95499999999999996,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69099999999999995,"15min":0.71099999999999997},"min":{"1min":0.56999999999999995,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0489999999999999,"5min":1.212,"15min":1.212},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.70899999999999996,"15min":0.73099999999999998},"min":{"1min":0.53600000000000003,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":1.016,"5min":1.74,"15min":1.74},"last":0.74399999999999999}]},{"osd":2,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.52600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.503,"5min":0.5,"15min":0.44600000000000001},"max":{"1min":0.97199999999999998,"5min":1.395,"15min":1.395},"last":0.75600000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98399999999999999,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.74299999999999999},"min":{"1min":0.55600000000000005,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.012,"5min":1.266,"15min":1.266},"last":0.72999999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.45700000000000002,"5min":0.308,"15min":0.28499999999999998},"max":{"1min":0.94399999999999995,"5min":1.841,"15min":1.841},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61499999999999999,"15min":0.60399999999999998},"min":{"1min":0.441,"5min":0.40600000000000003,"15min":0.36599999999999999},"max":{"1min":1.0089999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.62,"15min":0.60699999999999998},"min":{"1min":0.44600000000000001,"5min":0.376,"15min":0.36899999999999999},"max":{"1min":1.0700000000000001,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.629},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64800000000000002,"15min":0.64400000000000002},"min":{"1min":0.50900000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0760000000000001,"5min":1.784,"15min":1.784},"last":0.65200000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.59899999999999998,"5min":0.45200000000000001,"15min":0.39800000000000002},"max":{"1min":1.0109999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96499999999999997,"5min":2.105,"15min":2.105},"last":0.59899999999999998}]}]},{"osd":6,"up_from":15,"seq":64424509653,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1519616,"data_stored":767893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.68100000000000005,"15min":0.70899999999999996},"min":{"1min":0.60099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90300000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.68700000000000006,"15min":0.70999999999999996},"min":{"1min":0.56499999999999995,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":0.94299999999999995,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.61799999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68000000000000005,"15min":0.70799999999999996},"min":{"1min":0.621,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.84699999999999998,"5min":1.135,"15min":1.135},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.54600000000000004,"5min":0.42699999999999999,"15min":0.39500000000000002},"max":{"1min":0.96699999999999997,"5min":1.238,"15min":1.238},"last":0.73299999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72799999999999998,"15min":0.745},"min":{"1min":0.61099999999999999,"5min":0.47599999999999998,"15min":0.35199999999999998},"max":{"1min":0.92800000000000005,"5min":1.264,"15min":1.264},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.73299999999999998},"min":{"1min":0.56799999999999995,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92200000000000004,"5min":1.0049999999999999,"15min":1.1879999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73599999999999999,"15min":0.755},"min":{"1min":0.59199999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96499999999999997,"5min":1.413,"15min":1.413},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68400000000000005,"15min":0.71699999999999997},"min":{"1min":0.53400000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89100000000000001,"5min":0.94199999999999995,"15min":1.0740000000000001},"last":0.77500000000000002}]},{"osd":4,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.443,"5min":0.36699999999999999,"15min":0.35899999999999999},"max":{"1min":0.76300000000000001,"5min":0.95399999999999996,"15min":0.96999999999999997},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.47599999999999998,"5min":0.40600000000000003,"15min":0.35699999999999998},"max":{"1min":0.75600000000000001,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.33300000000000002},"max":{"1min":0.79100000000000004,"5min":0.85999999999999999,"15min":0.96699999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66800000000000004,"15min":0.65400000000000003},"min":{"1min":0.51900000000000002,"5min":0.42499999999999999,"15min":0.41499999999999998},"max":{"1min":0.82299999999999995,"5min":1.036,"15min":1.046},"last":0.79300000000000004}]},{"osd":7,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68899999999999995,"15min":0.69099999999999995},"min":{"1min":0.49199999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90600000000000003,"5min":1.006,"15min":1.0249999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.371},"max":{"1min":0.84999999999999998,"5min":0.94799999999999995,"15min":1.032},"last":0.82099999999999995}]}]},{"osd":5,"up_from":15,"seq":64424509653,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36624,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335216,"statfs":{"total":96636764160,"available":96599261184,"internally_reserved":0,"allocated":1515520,"data_stored":767311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68100000000000005,"15min":0.70199999999999996},"min":{"1min":0.57099999999999995,"5min":0.34399999999999997,"15min":0.29499999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.95799999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.59199999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.0189999999999999},"last":0.84299999999999997}]},{"osd":1,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.57499999999999996,"5min":0.39400000000000002,"15min":0.38600000000000001},"max":{"1min":0.91900000000000004,"5min":0.995,"15min":0.995},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.59799999999999998,"5min":0.40200000000000002,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.67700000000000005}]},{"osd":2,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72999999999999998,"15min":0.73699999999999999},"min":{"1min":0.56399999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96999999999999997,"5min":1.026,"15min":1.0409999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.58399999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93600000000000005,"5min":1.036,"15min":1.036},"last":0.68899999999999995}]},{"osd":3,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.63100000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.98699999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.75900000000000001,"15min":0.76100000000000001},"min":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0169999999999999,"5min":1.085,"15min":1.089},"last":0.86099999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.33300000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":0.92600000000000005,"15min":0.97199999999999998},"last":0.46700000000000003}]},{"osd":6,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":0.89500000000000002,"15min":0.98799999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88700000000000001,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55900000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.54100000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.877,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68400000000000005,"15min":0.67300000000000004},"min":{"1min":0.50600000000000001,"5min":0.45500000000000002,"15min":0.42099999999999999},"max":{"1min":0.874,"5min":1.1080000000000001,"15min":1.131},"last":0.58699999999999997}]}]},{"osd":4,"up_from":15,"seq":64424509653,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54688,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317152,"statfs":{"total":96636764160,"available":96580763648,"internally_reserved":0,"allocated":1531904,"data_stored":768929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71399999999999997,"15min":0.71099999999999997},"min":{"1min":0.40899999999999997,"5min":0.32600000000000001,"15min":0.316},"max":{"1min":1.0309999999999999,"5min":1.877,"15min":1.877},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68999999999999995,"15min":0.70199999999999996},"min":{"1min":0.45500000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88400000000000001,"5min":0.94899999999999995,"15min":1.0680000000000001},"last":0.41399999999999998}]},{"osd":1,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68999999999999995,"15min":0.71099999999999997},"min":{"1min":0.42699999999999999,"5min":0.39000000000000001,"15min":0.34799999999999998},"max":{"1min":0.872,"5min":0.93799999999999994,"15min":1.0209999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74299999999999999,"15min":0.73599999999999999},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.35599999999999998},"max":{"1min":0.89200000000000002,"5min":2.02,"15min":2.02},"last":0.56699999999999995}]},{"osd":2,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71299999999999997,"15min":0.72899999999999998},"min":{"1min":0.57599999999999996,"5min":0.46800000000000003,"15min":0.38700000000000001},"max":{"1min":0.93500000000000005,"5min":0.97199999999999998,"15min":1.375},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.749,"15min":0.749},"min":{"1min":0.44900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95099999999999996,"5min":1.927,"15min":1.927},"last":0.48099999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.71099999999999997},"min":{"1min":0.437,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91600000000000004,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73499999999999999,"15min":0.751},"min":{"1min":0.502,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":1.0049999999999999},"last":0.65800000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.63300000000000001,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":1.99,"15min":1.99},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.72799999999999998}]},{"osd":6,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.61599999999999999},"min":{"1min":0.438,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79300000000000004,"5min":0.96299999999999997,"15min":0.97199999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68100000000000005,"15min":0.66300000000000003},"min":{"1min":0.44,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":0.88800000000000001,"5min":1.96,"15min":1.96},"last":0.68799999999999994}]},{"osd":7,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66400000000000003,"15min":0.65600000000000003},"min":{"1min":0.437,"5min":0.376,"15min":0.376},"max":{"1min":0.97099999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0189999999999999,"5min":1.103,"15min":1.103},"last":0.748}]}]},{"osd":3,"up_from":15,"seq":64424509653,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36624,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335216,"statfs":{"total":96636764160,"available":96599261184,"internally_reserved":0,"allocated":1515520,"data_stored":761795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29399999999999998,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":0.68200000000000005,"5min":0.69199999999999995,"15min":1.2509999999999999},"last":0.307},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.502,"15min":0.50700000000000001},"min":{"1min":0.31,"5min":0.308,"15min":0.27100000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.163},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.46300000000000002},"min":{"1min":0.35599999999999998,"5min":0.29899999999999999,"15min":0.25900000000000001},"max":{"1min":0.66800000000000004,"5min":0.76300000000000001,"15min":0.77800000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.29699999999999999},"max":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":1.952},"last":0.34899999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.34499999999999997,"15min":0.30099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.45300000000000001,"5min":0.39500000000000002,"15min":0.36699999999999999},"max":{"1min":1.157,"5min":1.157,"15min":2.855},"last":0.40400000000000003}]},{"osd":4,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73999999999999999,"15min":0.78000000000000003},"min":{"1min":0.56999999999999995,"5min":0.50600000000000001,"15min":0.432},"max":{"1min":1.0309999999999999,"5min":1.081,"15min":4.234},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.749,"15min":0.77600000000000002},"min":{"1min":0.624,"5min":0.56100000000000005,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":1.111,"15min":1.3069999999999999},"last":0.63600000000000001}]},{"osd":5,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.752,"15min":0.77800000000000002},"min":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.1399999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.748,"15min":0.77700000000000002},"min":{"1min":0.58799999999999997,"5min":0.53900000000000003,"15min":0.43099999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.8300000000000001},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.749,"15min":0.78700000000000003},"min":{"1min":0.51500000000000001,"5min":0.51200000000000001,"15min":0.45800000000000002},"max":{"1min":1.054,"5min":1.0589999999999999,"15min":1.599},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.76900000000000002,"15min":0.79600000000000004},"min":{"1min":0.56299999999999994,"5min":0.54400000000000004,"15min":0.49199999999999999},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.80500000000000005,"15min":0.82499999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.79900000000000004,"15min":0.84499999999999997},"min":{"1min":0.67600000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.623},"last":0.68400000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509652,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58768,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313072,"statfs":{"total":96636764160,"available":96576585728,"internally_reserved":0,"allocated":1515520,"data_stored":767275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44,"15min":0.46000000000000002},"min":{"1min":0.36299999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":5.4870000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.48499999999999999,"15min":0.53100000000000003},"min":{"1min":0.41299999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":10.161},"last":0.27100000000000002}]},{"osd":1,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60199999999999998,"15min":0.61199999999999999},"min":{"1min":0.53400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76100000000000001,"5min":1.53,"15min":2.4740000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.42899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68799999999999994,"5min":1.673,"15min":2.911},"last":0.36399999999999999}]},{"osd":3,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57899999999999996,"15min":0.58999999999999997},"min":{"1min":0.504,"5min":0.378,"15min":0.378},"max":{"1min":0.755,"5min":0.79200000000000004,"15min":1.1990000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56000000000000005,"15min":0.57199999999999995},"min":{"1min":0.495,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.751,"5min":1.6279999999999999,"15min":2.7999999999999998},"last":0.441}]},{"osd":4,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.78300000000000003,"15min":0.78200000000000003},"min":{"1min":0.69399999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0069999999999999,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.75800000000000001,"15min":0.78500000000000003},"min":{"1min":0.63900000000000001,"5min":0.45200000000000001,"15min":0.42999999999999999},"max":{"1min":1.105,"5min":1.3049999999999999,"15min":5.5430000000000001},"last":0.88600000000000001}]},{"osd":5,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.58599999999999997,"5min":0.5,"15min":0.49299999999999999},"max":{"1min":1.048,"5min":1.4890000000000001,"15min":5.601},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.79300000000000004,"15min":0.80800000000000005},"min":{"1min":0.70099999999999996,"5min":0.504,"15min":0.504},"max":{"1min":0.97699999999999998,"5min":2.569,"15min":2.569},"last":0.93100000000000005}]},{"osd":6,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82799999999999996,"15min":0.82399999999999995},"min":{"1min":0.77100000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0780000000000001,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.754,"15min":0.78500000000000003},"min":{"1min":0.69299999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94499999999999995,"5min":1.1950000000000001,"15min":1.4379999999999999},"last":0.98399999999999999}]},{"osd":7,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.71499999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.099,"5min":2.5409999999999999,"15min":5.673},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.872,"5min":0.80800000000000005,"15min":0.83299999999999996},"min":{"1min":0.72099999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.028,"5min":1.369,"15min":1.429},"last":1.024}]}]},{"osd":1,"up_from":15,"seq":64424509653,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50572,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321268,"statfs":{"total":96636764160,"available":96584978432,"internally_reserved":0,"allocated":1511424,"data_stored":767241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.441,"15min":0.44600000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.22500000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.84899999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.501,"15min":0.51700000000000002},"min":{"1min":0.30099999999999999,"5min":0.29699999999999999,"15min":0.26400000000000001},"max":{"1min":0.79500000000000004,"5min":1.071,"15min":1.071},"last":0.48899999999999999}]},{"osd":2,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.498,"15min":0.50900000000000001},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.30299999999999999},"max":{"1min":0.67000000000000004,"5min":0.78800000000000003,"15min":0.80400000000000005},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52800000000000002,"15min":0.55100000000000005},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.317},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.36199999999999999}]},{"osd":3,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58299999999999996,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81100000000000005,"5min":0.871,"15min":0.91500000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.54000000000000004},"min":{"1min":0.41899999999999998,"5min":0.34499999999999997,"15min":0.33700000000000002},"max":{"1min":0.73999999999999999,"5min":0.755,"15min":0.91900000000000004},"last":0.505}]},{"osd":4,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.73099999999999998,"15min":0.755},"min":{"1min":0.51900000000000002,"5min":0.45300000000000001,"15min":0.43099999999999999},"max":{"1min":1.0509999999999999,"5min":1.107,"15min":1.321},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73999999999999999,"15min":0.77800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.002,"5min":1.0580000000000001,"15min":1.4179999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.746,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.46500000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.4450000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.75800000000000001,"15min":0.79200000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.46500000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.3380000000000001},"last":0.59599999999999997}]},{"osd":6,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73699999999999999,"15min":0.78200000000000003},"min":{"1min":0.58899999999999997,"5min":0.496,"15min":0.41999999999999998},"max":{"1min":1.0089999999999999,"5min":1.048,"15min":1.3939999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76000000000000001,"15min":0.79800000000000004},"min":{"1min":0.58299999999999996,"5min":0.56200000000000006,"15min":0.52700000000000002},"max":{"1min":1.0940000000000001,"5min":1.1659999999999999,"15min":1.391},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.79700000000000004,"15min":0.82399999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.2589999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77800000000000002,"15min":0.82199999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.53800000000000003},"max":{"1min":1.129,"5min":1.2050000000000001,"15min":1.4219999999999999},"last":0.69199999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542358,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":2109440,"data_stored":1358191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29699999999999999,"5min":0.26000000000000001,"15min":0.246},"max":{"1min":0.72699999999999998,"5min":0.752,"15min":0.752},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48499999999999999,"15min":0.49299999999999999},"min":{"1min":0.34100000000000003,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.72399999999999998,"5min":0.81699999999999995,"15min":0.85499999999999998},"last":0.69299999999999995}]},{"osd":2,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.496,"15min":0.497},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.31900000000000001},"max":{"1min":0.64100000000000001,"5min":0.71499999999999997,"15min":0.89700000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.41099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76600000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.59699999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.93600000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.495,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91300000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.71299999999999997}]},{"osd":4,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77100000000000002,"15min":0.78600000000000003},"min":{"1min":0.59399999999999997,"5min":0.503,"15min":0.45900000000000002},"max":{"1min":1.113,"5min":1.444,"15min":1.444},"last":1.145},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78100000000000003,"15min":0.79300000000000004},"min":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":0.95399999999999996,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.871}]},{"osd":5,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.80500000000000005,"15min":0.80300000000000005},"min":{"1min":0.53200000000000003,"5min":0.502,"15min":0.41499999999999998},"max":{"1min":1.0940000000000001,"5min":1.294,"15min":1.294},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.54600000000000004,"5min":0.54400000000000004,"15min":0.52300000000000002},"max":{"1min":1.014,"5min":1.256,"15min":1.256},"last":1.0529999999999999}]},{"osd":6,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80300000000000005,"15min":0.80100000000000005},"min":{"1min":0.60099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.075,"5min":1.429,"15min":1.429},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.64900000000000002,"5min":0.53100000000000003,"15min":0.44700000000000001},"max":{"1min":1.0580000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.91900000000000004}]},{"osd":7,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81899999999999995,"15min":0.82899999999999996},"min":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.48999999999999999},"max":{"1min":1.1639999999999999,"5min":1.377,"15min":1.377},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.83199999999999996,"15min":0.82399999999999995},"min":{"1min":0.58499999999999996,"5min":0.51400000000000001,"15min":0.48099999999999998},"max":{"1min":1.135,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.96599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T17:00:05.559 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T17:00:05.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.739+0000 7f088fce5640 1 Processor -- start 2023-12-19T17:00:05.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.739+0000 7f088fce5640 1 -- start start 2023-12-19T17:00:05.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.739+0000 7f088fce5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08880cb1d0 0x7f08880cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.739+0000 7f088fce5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f088806bee0 0x7f088806c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.739+0000 7f088fce5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088806d4b0 con 0x7f0888164c90 2023-12-19T17:00:05.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.739+0000 7f088fce5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08880cc030 con 0x7f08880cb1d0 2023-12-19T17:00:05.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.739+0000 7f088fce5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08880cbaf0 con 0x7f088806bee0 2023-12-19T17:00:05.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.739+0000 7f088da81640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08880cb1d0 0x7f08880cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.739+0000 7f088da81640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f08880cb1d0 0x7f08880cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45918/0 (socket says 172.21.15.26:45918) 2023-12-19T17:00:05.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.739+0000 7f088da81640 1 -- 172.21.15.26:0/1277968076 learned_addr learned my addr 172.21.15.26:0/1277968076 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:05.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.739+0000 7f088e282640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0888164c90 0x7f0888165120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46868/0 (socket says 172.21.15.26:46868) 2023-12-19T17:00:05.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.739+0000 7f088d280640 1 --2- 172.21.15.26:0/1277968076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f088806bee0 0x7f088806c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.743+0000 7f088da81640 1 -- 172.21.15.26:0/1277968076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f088806bee0 msgr2=0x7f088806c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:05.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.743+0000 7f088da81640 1 --2- 172.21.15.26:0/1277968076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f088806bee0 0x7f088806c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.743+0000 7f088da81640 1 -- 172.21.15.26:0/1277968076 >> v1:172.21.15.26:6789/0 conn(0x7f0888164c90 legacy=0x7f0888165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.743+0000 7f088da81640 1 -- 172.21.15.26:0/1277968076 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088806cc10 con 0x7f08880cb1d0 2023-12-19T17:00:05.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.743+0000 7f088d280640 1 --2- 172.21.15.26:0/1277968076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f088806bee0 0x7f088806c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:00:05.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.743+0000 7f088da81640 1 --2- 172.21.15.26:0/1277968076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08880cb1d0 0x7f08880cb5b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f0884009870 tx=0x7f088401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf2bc8cb4ad3f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.743+0000 7f088ca7f640 1 -- 172.21.15.26:0/1277968076 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840ab680 con 0x7f08880cb1d0 2023-12-19T17:00:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.743+0000 7f088ca7f640 1 -- 172.21.15.26:0/1277968076 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08840ab7e0 con 0x7f08880cb1d0 2023-12-19T17:00:05.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.743+0000 7f088ca7f640 1 -- 172.21.15.26:0/1277968076 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840abb70 con 0x7f08880cb1d0 2023-12-19T17:00:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.743+0000 7f088fce5640 1 -- 172.21.15.26:0/1277968076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08880cb1d0 msgr2=0x7f08880cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.743+0000 7f088fce5640 1 --2- 172.21.15.26:0/1277968076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08880cb1d0 0x7f08880cb5b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f0884009870 tx=0x7f088401c480 comp rx=0 tx=0).stop 2023-12-19T17:00:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 -- 172.21.15.26:0/1277968076 shutdown_connections 2023-12-19T17:00:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 --2- 172.21.15.26:0/1277968076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f088806bee0 0x7f088806c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 --2- 172.21.15.26:0/1277968076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08880cb1d0 0x7f08880cb5b0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 -- 172.21.15.26:0/1277968076 >> 172.21.15.26:0/1277968076 conn(0x7f08880658e0 msgr2=0x7f0888065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 -- 172.21.15.26:0/1277968076 shutdown_connections 2023-12-19T17:00:05.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 -- 172.21.15.26:0/1277968076 wait complete. 2023-12-19T17:00:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 Processor -- start 2023-12-19T17:00:05.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 -- start start 2023-12-19T17:00:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088806bee0 0x7f08880af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08880cb1d0 0x7f088814bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0888164c90 0x7f08880ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088814c8a0 con 0x7f088806bee0 2023-12-19T17:00:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0888148810 con 0x7f0888164c90 2023-12-19T17:00:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088fce5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088814c2e0 con 0x7f08880cb1d0 2023-12-19T17:00:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088da81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088806bee0 0x7f08880af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088da81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088806bee0 0x7f08880af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45932/0 (socket says 172.21.15.26:45932) 2023-12-19T17:00:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088da81640 1 -- 172.21.15.26:0/1045538668 learned_addr learned my addr 172.21.15.26:0/1045538668 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088da81640 1 -- 172.21.15.26:0/1045538668 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08880cb1d0 msgr2=0x7f088814bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088da81640 1 --2- 172.21.15.26:0/1045538668 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08880cb1d0 0x7f088814bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088da81640 1 -- 172.21.15.26:0/1045538668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0888164c90 msgr2=0x7f08880ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088da81640 1 --2- 172.21.15.26:0/1045538668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0888164c90 0x7f08880ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088da81640 1 -- 172.21.15.26:0/1045538668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08840094d0 con 0x7f088806bee0 2023-12-19T17:00:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f088da81640 1 --2- 172.21.15.26:0/1045538668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088806bee0 0x7f08880af4b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f08840ad040 tx=0x7f088401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.747+0000 7f087e7fc640 1 -- 172.21.15.26:0/1045538668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840b4800 con 0x7f088806bee0 2023-12-19T17:00:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.751+0000 7f087e7fc640 1 -- 172.21.15.26:0/1045538668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08840b49a0 con 0x7f088806bee0 2023-12-19T17:00:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.751+0000 7f087e7fc640 1 -- 172.21.15.26:0/1045538668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840b4d50 con 0x7f088806bee0 2023-12-19T17:00:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.751+0000 7f088fce5640 1 -- 172.21.15.26:0/1045538668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088814c5c0 con 0x7f088806bee0 2023-12-19T17:00:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.751+0000 7f088fce5640 1 -- 172.21.15.26:0/1045538668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08880afe00 con 0x7f088806bee0 2023-12-19T17:00:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.751+0000 7f088fce5640 1 -- 172.21.15.26:0/1045538668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088806c860 con 0x7f088806bee0 2023-12-19T17:00:05.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.751+0000 7f087e7fc640 1 -- 172.21.15.26:0/1045538668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f08840c7440 con 0x7f088806bee0 2023-12-19T17:00:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.751+0000 7f087e7fc640 1 --2- 172.21.15.26:0/1045538668 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f086c065b80 0x7f086c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.751+0000 7f087e7fc640 1 -- 172.21.15.26:0/1045538668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f088411e590 con 0x7f088806bee0 2023-12-19T17:00:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.751+0000 7f088d280640 1 --2- 172.21.15.26:0/1045538668 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f086c065b80 0x7f086c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.755+0000 7f088d280640 1 --2- 172.21.15.26:0/1045538668 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f086c065b80 0x7f086c068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0878006f10 tx=0x7f0878008040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:05.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.755+0000 7f087e7fc640 1 -- 172.21.15.26:0/1045538668 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f088806c860 con 0x7f088806bee0 2023-12-19T17:00:05.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.911+0000 7f088fce5640 1 -- 172.21.15.26:0/1045538668 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f08880b1820 con 0x7f086c065b80 2023-12-19T17:00:05.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.919+0000 7f087e7fc640 1 -- 172.21.15.26:0/1045538668 <== mgr.6707 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289515 (secure 0 0 0) 0x7f0878009e00 con 0x7f086c065b80 2023-12-19T17:00:05.924 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:00:05.926 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T17:00:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.923+0000 7f088fce5640 1 -- 172.21.15.26:0/1045538668 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f086c065b80 msgr2=0x7f086c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.923+0000 7f088fce5640 1 --2- 172.21.15.26:0/1045538668 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f086c065b80 0x7f086c068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0878006f10 tx=0x7f0878008040 comp rx=0 tx=0).stop 2023-12-19T17:00:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.923+0000 7f088fce5640 1 -- 172.21.15.26:0/1045538668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088806bee0 msgr2=0x7f08880af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.923+0000 7f088fce5640 1 --2- 172.21.15.26:0/1045538668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088806bee0 0x7f08880af4b0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f08840ad040 tx=0x7f088401acc0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.923+0000 7f088fce5640 1 -- 172.21.15.26:0/1045538668 shutdown_connections 2023-12-19T17:00:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.923+0000 7f088fce5640 1 --2- 172.21.15.26:0/1045538668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0888164c90 0x7f08880ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.923+0000 7f088fce5640 1 --2- 172.21.15.26:0/1045538668 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f086c065b80 0x7f086c068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.923+0000 7f088fce5640 1 --2- 172.21.15.26:0/1045538668 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08880cb1d0 0x7f088814bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.923+0000 7f088fce5640 1 --2- 172.21.15.26:0/1045538668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f088806bee0 0x7f08880af4b0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.923+0000 7f088fce5640 1 -- 172.21.15.26:0/1045538668 >> 172.21.15.26:0/1045538668 conn(0x7f08880658e0 msgr2=0x7f08880b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.923+0000 7f088fce5640 1 -- 172.21.15.26:0/1045538668 shutdown_connections 2023-12-19T17:00:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:05.923+0000 7f088fce5640 1 -- 172.21.15.26:0/1045538668 wait complete. 2023-12-19T17:00:05.944 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-19T17:00:05.516966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1058,"num_read_kb":2145,"num_write":399,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":263,"ondisk_log_size":263,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":340112,"kb_used_data":13008,"kb_used_omap":0,"kb_used_meta":327040,"kb_avail":754634608,"statfs":{"total":773094113280,"available":772745838592,"internally_reserved":0,"allocated":13320192,"data_stored":7309324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":334888960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.493446"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733747+0000","last_change":"2023-12-19T17:00:01.450171+0000","last_active":"2023-12-19T17:00:02.733747+0000","last_peered":"2023-12-19T17:00:02.733747+0000","last_clean":"2023-12-19T17:00:02.733747+0000","last_became_active":"2023-12-19T17:00:01.449898+0000","last_became_peered":"2023-12-19T17:00:01.449898+0000","last_unstale":"2023-12-19T17:00:02.733747+0000","last_undegraded":"2023-12-19T17:00:02.733747+0000","last_fullsized":"2023-12-19T17:00:02.733747+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:35.314161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438108+0000","last_change":"2023-12-19T17:00:01.449283+0000","last_active":"2023-12-19T17:00:02.438108+0000","last_peered":"2023-12-19T17:00:02.438108+0000","last_clean":"2023-12-19T17:00:02.438108+0000","last_became_active":"2023-12-19T17:00:01.449014+0000","last_became_peered":"2023-12-19T17:00:01.449014+0000","last_unstale":"2023-12-19T17:00:02.438108+0000","last_undegraded":"2023-12-19T17:00:02.438108+0000","last_fullsized":"2023-12-19T17:00:02.438108+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:23:00.185954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733766+0000","last_change":"2023-12-19T17:00:01.450395+0000","last_active":"2023-12-19T17:00:02.733766+0000","last_peered":"2023-12-19T17:00:02.733766+0000","last_clean":"2023-12-19T17:00:02.733766+0000","last_became_active":"2023-12-19T17:00:01.450032+0000","last_became_peered":"2023-12-19T17:00:01.450032+0000","last_unstale":"2023-12-19T17:00:02.733766+0000","last_undegraded":"2023-12-19T17:00:02.733766+0000","last_fullsized":"2023-12-19T17:00:02.733766+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:41.057366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733979+0000","last_change":"2023-12-19T17:00:01.466863+0000","last_active":"2023-12-19T17:00:02.733979+0000","last_peered":"2023-12-19T17:00:02.733979+0000","last_clean":"2023-12-19T17:00:02.733979+0000","last_became_active":"2023-12-19T17:00:01.466580+0000","last_became_peered":"2023-12-19T17:00:01.466580+0000","last_unstale":"2023-12-19T17:00:02.733979+0000","last_undegraded":"2023-12-19T17:00:02.733979+0000","last_fullsized":"2023-12-19T17:00:02.733979+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:05:36.465953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439120+0000","last_change":"2023-12-19T17:00:01.453915+0000","last_active":"2023-12-19T17:00:02.439120+0000","last_peered":"2023-12-19T17:00:02.439120+0000","last_clean":"2023-12-19T17:00:02.439120+0000","last_became_active":"2023-12-19T17:00:01.453639+0000","last_became_peered":"2023-12-19T17:00:01.453639+0000","last_unstale":"2023-12-19T17:00:02.439120+0000","last_undegraded":"2023-12-19T17:00:02.439120+0000","last_fullsized":"2023-12-19T17:00:02.439120+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:15:28.811743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734113+0000","last_change":"2023-12-19T17:00:01.459472+0000","last_active":"2023-12-19T17:00:02.734113+0000","last_peered":"2023-12-19T17:00:02.734113+0000","last_clean":"2023-12-19T17:00:02.734113+0000","last_became_active":"2023-12-19T17:00:01.459114+0000","last_became_peered":"2023-12-19T17:00:01.459114+0000","last_unstale":"2023-12-19T17:00:02.734113+0000","last_undegraded":"2023-12-19T17:00:02.734113+0000","last_fullsized":"2023-12-19T17:00:02.734113+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:47:45.647622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734357+0000","last_change":"2023-12-19T17:00:01.454499+0000","last_active":"2023-12-19T17:00:02.734357+0000","last_peered":"2023-12-19T17:00:02.734357+0000","last_clean":"2023-12-19T17:00:02.734357+0000","last_became_active":"2023-12-19T17:00:01.454217+0000","last_became_peered":"2023-12-19T17:00:01.454217+0000","last_unstale":"2023-12-19T17:00:02.734357+0000","last_undegraded":"2023-12-19T17:00:02.734357+0000","last_fullsized":"2023-12-19T17:00:02.734357+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:44.662350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437762+0000","last_change":"2023-12-19T17:00:01.453847+0000","last_active":"2023-12-19T17:00:02.437762+0000","last_peered":"2023-12-19T17:00:02.437762+0000","last_clean":"2023-12-19T17:00:02.437762+0000","last_became_active":"2023-12-19T17:00:01.453352+0000","last_became_peered":"2023-12-19T17:00:01.453352+0000","last_unstale":"2023-12-19T17:00:02.437762+0000","last_undegraded":"2023-12-19T17:00:02.437762+0000","last_fullsized":"2023-12-19T17:00:02.437762+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:27:30.538250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734522+0000","last_change":"2023-12-19T17:00:01.455767+0000","last_active":"2023-12-19T17:00:02.734522+0000","last_peered":"2023-12-19T17:00:02.734522+0000","last_clean":"2023-12-19T17:00:02.734522+0000","last_became_active":"2023-12-19T17:00:01.455202+0000","last_became_peered":"2023-12-19T17:00:01.455202+0000","last_unstale":"2023-12-19T17:00:02.734522+0000","last_undegraded":"2023-12-19T17:00:02.734522+0000","last_fullsized":"2023-12-19T17:00:02.734522+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:35.314161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439436+0000","last_change":"2023-12-19T17:00:01.448229+0000","last_active":"2023-12-19T17:00:02.439436+0000","last_peered":"2023-12-19T17:00:02.439436+0000","last_clean":"2023-12-19T17:00:02.439436+0000","last_became_active":"2023-12-19T17:00:01.447606+0000","last_became_peered":"2023-12-19T17:00:01.447606+0000","last_unstale":"2023-12-19T17:00:02.439436+0000","last_undegraded":"2023-12-19T17:00:02.439436+0000","last_fullsized":"2023-12-19T17:00:02.439436+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:11:08.259813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734010+0000","last_change":"2023-12-19T17:00:01.457240+0000","last_active":"2023-12-19T17:00:02.734010+0000","last_peered":"2023-12-19T17:00:02.734010+0000","last_clean":"2023-12-19T17:00:02.734010+0000","last_became_active":"2023-12-19T17:00:01.456975+0000","last_became_peered":"2023-12-19T17:00:01.456975+0000","last_unstale":"2023-12-19T17:00:02.734010+0000","last_undegraded":"2023-12-19T17:00:02.734010+0000","last_fullsized":"2023-12-19T17:00:02.734010+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:53.251492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733982+0000","last_change":"2023-12-19T17:00:01.447688+0000","last_active":"2023-12-19T17:00:02.733982+0000","last_peered":"2023-12-19T17:00:02.733982+0000","last_clean":"2023-12-19T17:00:02.733982+0000","last_became_active":"2023-12-19T17:00:01.446300+0000","last_became_peered":"2023-12-19T17:00:01.446300+0000","last_unstale":"2023-12-19T17:00:02.733982+0000","last_undegraded":"2023-12-19T17:00:02.733982+0000","last_fullsized":"2023-12-19T17:00:02.733982+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:41.057366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733909+0000","last_change":"2023-12-19T17:00:01.456109+0000","last_active":"2023-12-19T17:00:02.733909+0000","last_peered":"2023-12-19T17:00:02.733909+0000","last_clean":"2023-12-19T17:00:02.733909+0000","last_became_active":"2023-12-19T17:00:01.455860+0000","last_became_peered":"2023-12-19T17:00:01.455860+0000","last_unstale":"2023-12-19T17:00:02.733909+0000","last_undegraded":"2023-12-19T17:00:02.733909+0000","last_fullsized":"2023-12-19T17:00:02.733909+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:13.426008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734182+0000","last_change":"2023-12-19T17:00:01.452323+0000","last_active":"2023-12-19T17:00:02.734182+0000","last_peered":"2023-12-19T17:00:02.734182+0000","last_clean":"2023-12-19T17:00:02.734182+0000","last_became_active":"2023-12-19T17:00:01.452059+0000","last_became_peered":"2023-12-19T17:00:01.452059+0000","last_unstale":"2023-12-19T17:00:02.734182+0000","last_undegraded":"2023-12-19T17:00:02.734182+0000","last_fullsized":"2023-12-19T17:00:02.734182+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:31.653516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733496+0000","last_change":"2023-12-19T17:00:01.459010+0000","last_active":"2023-12-19T17:00:02.733496+0000","last_peered":"2023-12-19T17:00:02.733496+0000","last_clean":"2023-12-19T17:00:02.733496+0000","last_became_active":"2023-12-19T17:00:01.458738+0000","last_became_peered":"2023-12-19T17:00:01.458738+0000","last_unstale":"2023-12-19T17:00:02.733496+0000","last_undegraded":"2023-12-19T17:00:02.733496+0000","last_fullsized":"2023-12-19T17:00:02.733496+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:15:53.444387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733623+0000","last_change":"2023-12-19T17:00:01.463960+0000","last_active":"2023-12-19T17:00:02.733623+0000","last_peered":"2023-12-19T17:00:02.733623+0000","last_clean":"2023-12-19T17:00:02.733623+0000","last_became_active":"2023-12-19T17:00:01.463664+0000","last_became_peered":"2023-12-19T17:00:01.463664+0000","last_unstale":"2023-12-19T17:00:02.733623+0000","last_undegraded":"2023-12-19T17:00:02.733623+0000","last_fullsized":"2023-12-19T17:00:02.733623+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:34.250172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438223+0000","last_change":"2023-12-19T17:00:01.456608+0000","last_active":"2023-12-19T17:00:02.438223+0000","last_peered":"2023-12-19T17:00:02.438223+0000","last_clean":"2023-12-19T17:00:02.438223+0000","last_became_active":"2023-12-19T17:00:01.455916+0000","last_became_peered":"2023-12-19T17:00:01.455916+0000","last_unstale":"2023-12-19T17:00:02.438223+0000","last_undegraded":"2023-12-19T17:00:02.438223+0000","last_fullsized":"2023-12-19T17:00:02.438223+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:49:54.041124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734036+0000","last_change":"2023-12-19T17:00:01.463663+0000","last_active":"2023-12-19T17:00:02.734036+0000","last_peered":"2023-12-19T17:00:02.734036+0000","last_clean":"2023-12-19T17:00:02.734036+0000","last_became_active":"2023-12-19T17:00:01.463393+0000","last_became_peered":"2023-12-19T17:00:01.463393+0000","last_unstale":"2023-12-19T17:00:02.734036+0000","last_undegraded":"2023-12-19T17:00:02.734036+0000","last_fullsized":"2023-12-19T17:00:02.734036+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:21:33.482019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734521+0000","last_change":"2023-12-19T17:00:01.445233+0000","last_active":"2023-12-19T17:00:02.734521+0000","last_peered":"2023-12-19T17:00:02.734521+0000","last_clean":"2023-12-19T17:00:02.734521+0000","last_became_active":"2023-12-19T17:00:01.444790+0000","last_became_peered":"2023-12-19T17:00:01.444790+0000","last_unstale":"2023-12-19T17:00:02.734521+0000","last_undegraded":"2023-12-19T17:00:02.734521+0000","last_fullsized":"2023-12-19T17:00:02.734521+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:06:07.601962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734009+0000","last_change":"2023-12-19T17:00:01.451995+0000","last_active":"2023-12-19T17:00:02.734009+0000","last_peered":"2023-12-19T17:00:02.734009+0000","last_clean":"2023-12-19T17:00:02.734009+0000","last_became_active":"2023-12-19T17:00:01.451725+0000","last_became_peered":"2023-12-19T17:00:01.451725+0000","last_unstale":"2023-12-19T17:00:02.734009+0000","last_undegraded":"2023-12-19T17:00:02.734009+0000","last_fullsized":"2023-12-19T17:00:02.734009+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:19:01.646545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734415+0000","last_change":"2023-12-19T17:00:01.452559+0000","last_active":"2023-12-19T17:00:02.734415+0000","last_peered":"2023-12-19T17:00:02.734415+0000","last_clean":"2023-12-19T17:00:02.734415+0000","last_became_active":"2023-12-19T17:00:01.452291+0000","last_became_peered":"2023-12-19T17:00:01.452291+0000","last_unstale":"2023-12-19T17:00:02.734415+0000","last_undegraded":"2023-12-19T17:00:02.734415+0000","last_fullsized":"2023-12-19T17:00:02.734415+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:04:46.518597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437595+0000","last_change":"2023-12-19T17:00:01.442182+0000","last_active":"2023-12-19T17:00:02.437595+0000","last_peered":"2023-12-19T17:00:02.437595+0000","last_clean":"2023-12-19T17:00:02.437595+0000","last_became_active":"2023-12-19T17:00:01.441932+0000","last_became_peered":"2023-12-19T17:00:01.441932+0000","last_unstale":"2023-12-19T17:00:02.437595+0000","last_undegraded":"2023-12-19T17:00:02.437595+0000","last_fullsized":"2023-12-19T17:00:02.437595+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:41:37.856086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437543+0000","last_change":"2023-12-19T17:00:01.442454+0000","last_active":"2023-12-19T17:00:02.437543+0000","last_peered":"2023-12-19T17:00:02.437543+0000","last_clean":"2023-12-19T17:00:02.437543+0000","last_became_active":"2023-12-19T17:00:01.442208+0000","last_became_peered":"2023-12-19T17:00:01.442208+0000","last_unstale":"2023-12-19T17:00:02.437543+0000","last_undegraded":"2023-12-19T17:00:02.437543+0000","last_fullsized":"2023-12-19T17:00:02.437543+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:20.704143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733779+0000","last_change":"2023-12-19T17:00:01.457198+0000","last_active":"2023-12-19T17:00:02.733779+0000","last_peered":"2023-12-19T17:00:02.733779+0000","last_clean":"2023-12-19T17:00:02.733779+0000","last_became_active":"2023-12-19T17:00:01.456899+0000","last_became_peered":"2023-12-19T17:00:01.456899+0000","last_unstale":"2023-12-19T17:00:02.733779+0000","last_undegraded":"2023-12-19T17:00:02.733779+0000","last_fullsized":"2023-12-19T17:00:02.733779+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:24.025937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439503+0000","last_change":"2023-12-19T17:00:01.461224+0000","last_active":"2023-12-19T17:00:02.439503+0000","last_peered":"2023-12-19T17:00:02.439503+0000","last_clean":"2023-12-19T17:00:02.439503+0000","last_became_active":"2023-12-19T17:00:01.460962+0000","last_became_peered":"2023-12-19T17:00:01.460962+0000","last_unstale":"2023-12-19T17:00:02.439503+0000","last_undegraded":"2023-12-19T17:00:02.439503+0000","last_fullsized":"2023-12-19T17:00:02.439503+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:23.855329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438152+0000","last_change":"2023-12-19T17:00:01.445537+0000","last_active":"2023-12-19T17:00:02.438152+0000","last_peered":"2023-12-19T17:00:02.438152+0000","last_clean":"2023-12-19T17:00:02.438152+0000","last_became_active":"2023-12-19T17:00:01.445251+0000","last_became_peered":"2023-12-19T17:00:01.445251+0000","last_unstale":"2023-12-19T17:00:02.438152+0000","last_undegraded":"2023-12-19T17:00:02.438152+0000","last_fullsized":"2023-12-19T17:00:02.438152+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:21.544283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734115+0000","last_change":"2023-12-19T17:00:01.468834+0000","last_active":"2023-12-19T17:00:02.734115+0000","last_peered":"2023-12-19T17:00:02.734115+0000","last_clean":"2023-12-19T17:00:02.734115+0000","last_became_active":"2023-12-19T17:00:01.468472+0000","last_became_peered":"2023-12-19T17:00:01.468472+0000","last_unstale":"2023-12-19T17:00:02.734115+0000","last_undegraded":"2023-12-19T17:00:02.734115+0000","last_fullsized":"2023-12-19T17:00:02.734115+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:30:41.998190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439353+0000","last_change":"2023-12-19T17:00:01.457034+0000","last_active":"2023-12-19T17:00:02.439353+0000","last_peered":"2023-12-19T17:00:02.439353+0000","last_clean":"2023-12-19T17:00:02.439353+0000","last_became_active":"2023-12-19T17:00:01.454694+0000","last_became_peered":"2023-12-19T17:00:01.454694+0000","last_unstale":"2023-12-19T17:00:02.439353+0000","last_undegraded":"2023-12-19T17:00:02.439353+0000","last_fullsized":"2023-12-19T17:00:02.439353+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:16.412408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437627+0000","last_change":"2023-12-19T17:00:01.445931+0000","last_active":"2023-12-19T17:00:02.437627+0000","last_peered":"2023-12-19T17:00:02.437627+0000","last_clean":"2023-12-19T17:00:02.437627+0000","last_became_active":"2023-12-19T17:00:01.445229+0000","last_became_peered":"2023-12-19T17:00:01.445229+0000","last_unstale":"2023-12-19T17:00:02.437627+0000","last_undegraded":"2023-12-19T17:00:02.437627+0000","last_fullsized":"2023-12-19T17:00:02.437627+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:21:23.742043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734288+0000","last_change":"2023-12-19T17:00:01.447323+0000","last_active":"2023-12-19T17:00:02.734288+0000","last_peered":"2023-12-19T17:00:02.734288+0000","last_clean":"2023-12-19T17:00:02.734288+0000","last_became_active":"2023-12-19T17:00:01.447051+0000","last_became_peered":"2023-12-19T17:00:01.447051+0000","last_unstale":"2023-12-19T17:00:02.734288+0000","last_undegraded":"2023-12-19T17:00:02.734288+0000","last_fullsized":"2023-12-19T17:00:02.734288+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:46:40.430536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733547+0000","last_change":"2023-12-19T17:00:01.455037+0000","last_active":"2023-12-19T17:00:02.733547+0000","last_peered":"2023-12-19T17:00:02.733547+0000","last_clean":"2023-12-19T17:00:02.733547+0000","last_became_active":"2023-12-19T17:00:01.453925+0000","last_became_peered":"2023-12-19T17:00:01.453925+0000","last_unstale":"2023-12-19T17:00:02.733547+0000","last_undegraded":"2023-12-19T17:00:02.733547+0000","last_fullsized":"2023-12-19T17:00:02.733547+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:52.817577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733321+0000","last_change":"2023-12-19T17:00:01.450713+0000","last_active":"2023-12-19T17:00:02.733321+0000","last_peered":"2023-12-19T17:00:02.733321+0000","last_clean":"2023-12-19T17:00:02.733321+0000","last_became_active":"2023-12-19T17:00:01.450454+0000","last_became_peered":"2023-12-19T17:00:01.450454+0000","last_unstale":"2023-12-19T17:00:02.733321+0000","last_undegraded":"2023-12-19T17:00:02.733321+0000","last_fullsized":"2023-12-19T17:00:02.733321+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:34.348541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734484+0000","last_change":"2023-12-19T17:00:01.452401+0000","last_active":"2023-12-19T17:00:02.734484+0000","last_peered":"2023-12-19T17:00:02.734484+0000","last_clean":"2023-12-19T17:00:02.734484+0000","last_became_active":"2023-12-19T17:00:01.452129+0000","last_became_peered":"2023-12-19T17:00:01.452129+0000","last_unstale":"2023-12-19T17:00:02.734484+0000","last_undegraded":"2023-12-19T17:00:02.734484+0000","last_fullsized":"2023-12-19T17:00:02.734484+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:02.080480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"122'4","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438256+0000","last_change":"2023-12-19T17:00:01.047133+0000","last_active":"2023-12-19T17:00:02.438256+0000","last_peered":"2023-12-19T17:00:02.438256+0000","last_clean":"2023-12-19T17:00:02.438256+0000","last_became_active":"2023-12-19T17:00:01.046831+0000","last_became_peered":"2023-12-19T17:00:01.046831+0000","last_unstale":"2023-12-19T17:00:02.438256+0000","last_undegraded":"2023-12-19T17:00:02.438256+0000","last_fullsized":"2023-12-19T17:00:02.438256+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:15:35.770496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734269+0000","last_change":"2023-12-19T17:00:01.453768+0000","last_active":"2023-12-19T17:00:02.734269+0000","last_peered":"2023-12-19T17:00:02.734269+0000","last_clean":"2023-12-19T17:00:02.734269+0000","last_became_active":"2023-12-19T17:00:01.453444+0000","last_became_peered":"2023-12-19T17:00:01.453444+0000","last_unstale":"2023-12-19T17:00:02.734269+0000","last_undegraded":"2023-12-19T17:00:02.734269+0000","last_fullsized":"2023-12-19T17:00:02.734269+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:59.365491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"122'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733998+0000","last_change":"2023-12-19T17:00:01.050160+0000","last_active":"2023-12-19T17:00:02.733998+0000","last_peered":"2023-12-19T17:00:02.733998+0000","last_clean":"2023-12-19T17:00:02.733998+0000","last_became_active":"2023-12-19T17:00:01.049851+0000","last_became_peered":"2023-12-19T17:00:01.049851+0000","last_unstale":"2023-12-19T17:00:02.733998+0000","last_undegraded":"2023-12-19T17:00:02.733998+0000","last_fullsized":"2023-12-19T17:00:02.733998+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:33.346149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733522+0000","last_change":"2023-12-19T17:00:01.442685+0000","last_active":"2023-12-19T17:00:02.733522+0000","last_peered":"2023-12-19T17:00:02.733522+0000","last_clean":"2023-12-19T17:00:02.733522+0000","last_became_active":"2023-12-19T17:00:01.442363+0000","last_became_peered":"2023-12-19T17:00:01.442363+0000","last_unstale":"2023-12-19T17:00:02.733522+0000","last_undegraded":"2023-12-19T17:00:02.733522+0000","last_fullsized":"2023-12-19T17:00:02.733522+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:04:18.408128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438059+0000","last_change":"2023-12-19T17:00:01.053355+0000","last_active":"2023-12-19T17:00:02.438059+0000","last_peered":"2023-12-19T17:00:02.438059+0000","last_clean":"2023-12-19T17:00:02.438059+0000","last_became_active":"2023-12-19T17:00:01.053008+0000","last_became_peered":"2023-12-19T17:00:01.053008+0000","last_unstale":"2023-12-19T17:00:02.438059+0000","last_undegraded":"2023-12-19T17:00:02.438059+0000","last_fullsized":"2023-12-19T17:00:02.438059+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:12.935403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734243+0000","last_change":"2023-12-19T17:00:01.453056+0000","last_active":"2023-12-19T17:00:02.734243+0000","last_peered":"2023-12-19T17:00:02.734243+0000","last_clean":"2023-12-19T17:00:02.734243+0000","last_became_active":"2023-12-19T17:00:01.452756+0000","last_became_peered":"2023-12-19T17:00:01.452756+0000","last_unstale":"2023-12-19T17:00:02.734243+0000","last_undegraded":"2023-12-19T17:00:02.734243+0000","last_fullsized":"2023-12-19T17:00:02.734243+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:42:36.359289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437904+0000","last_change":"2023-12-19T17:00:01.041483+0000","last_active":"2023-12-19T17:00:02.437904+0000","last_peered":"2023-12-19T17:00:02.437904+0000","last_clean":"2023-12-19T17:00:02.437904+0000","last_became_active":"2023-12-19T17:00:01.041219+0000","last_became_peered":"2023-12-19T17:00:01.041219+0000","last_unstale":"2023-12-19T17:00:02.437904+0000","last_undegraded":"2023-12-19T17:00:02.437904+0000","last_fullsized":"2023-12-19T17:00:02.437904+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:42:38.232078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438375+0000","last_change":"2023-12-19T17:00:01.448953+0000","last_active":"2023-12-19T17:00:02.438375+0000","last_peered":"2023-12-19T17:00:02.438375+0000","last_clean":"2023-12-19T17:00:02.438375+0000","last_became_active":"2023-12-19T17:00:01.448660+0000","last_became_peered":"2023-12-19T17:00:01.448660+0000","last_unstale":"2023-12-19T17:00:02.438375+0000","last_undegraded":"2023-12-19T17:00:02.438375+0000","last_fullsized":"2023-12-19T17:00:02.438375+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:41:06.018409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"122'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733622+0000","last_change":"2023-12-19T17:00:01.045175+0000","last_active":"2023-12-19T17:00:02.733622+0000","last_peered":"2023-12-19T17:00:02.733622+0000","last_clean":"2023-12-19T17:00:02.733622+0000","last_became_active":"2023-12-19T17:00:01.044877+0000","last_became_peered":"2023-12-19T17:00:01.044877+0000","last_unstale":"2023-12-19T17:00:02.733622+0000","last_undegraded":"2023-12-19T17:00:02.733622+0000","last_fullsized":"2023-12-19T17:00:02.733622+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:42:45.648537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733836+0000","last_change":"2023-12-19T17:00:01.457482+0000","last_active":"2023-12-19T17:00:02.733836+0000","last_peered":"2023-12-19T17:00:02.733836+0000","last_clean":"2023-12-19T17:00:02.733836+0000","last_became_active":"2023-12-19T17:00:01.456496+0000","last_became_peered":"2023-12-19T17:00:01.456496+0000","last_unstale":"2023-12-19T17:00:02.733836+0000","last_undegraded":"2023-12-19T17:00:02.733836+0000","last_fullsized":"2023-12-19T17:00:02.733836+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:19:19.027263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"122'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437621+0000","last_change":"2023-12-19T17:00:01.046428+0000","last_active":"2023-12-19T17:00:02.437621+0000","last_peered":"2023-12-19T17:00:02.437621+0000","last_clean":"2023-12-19T17:00:02.437621+0000","last_became_active":"2023-12-19T17:00:01.046039+0000","last_became_peered":"2023-12-19T17:00:01.046039+0000","last_unstale":"2023-12-19T17:00:02.437621+0000","last_undegraded":"2023-12-19T17:00:02.437621+0000","last_fullsized":"2023-12-19T17:00:02.437621+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:56.299107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733428+0000","last_change":"2023-12-19T17:00:01.458952+0000","last_active":"2023-12-19T17:00:02.733428+0000","last_peered":"2023-12-19T17:00:02.733428+0000","last_clean":"2023-12-19T17:00:02.733428+0000","last_became_active":"2023-12-19T17:00:01.458691+0000","last_became_peered":"2023-12-19T17:00:01.458691+0000","last_unstale":"2023-12-19T17:00:02.733428+0000","last_undegraded":"2023-12-19T17:00:02.733428+0000","last_fullsized":"2023-12-19T17:00:02.733428+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:16.058767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438928+0000","last_change":"2023-12-19T17:00:01.048430+0000","last_active":"2023-12-19T17:00:02.438928+0000","last_peered":"2023-12-19T17:00:02.438928+0000","last_clean":"2023-12-19T17:00:02.438928+0000","last_became_active":"2023-12-19T17:00:01.048154+0000","last_became_peered":"2023-12-19T17:00:01.048154+0000","last_unstale":"2023-12-19T17:00:02.438928+0000","last_undegraded":"2023-12-19T17:00:02.438928+0000","last_fullsized":"2023-12-19T17:00:02.438928+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:25:34.382256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733479+0000","last_change":"2023-12-19T17:00:01.446763+0000","last_active":"2023-12-19T17:00:02.733479+0000","last_peered":"2023-12-19T17:00:02.733479+0000","last_clean":"2023-12-19T17:00:02.733479+0000","last_became_active":"2023-12-19T17:00:01.446409+0000","last_became_peered":"2023-12-19T17:00:01.446409+0000","last_unstale":"2023-12-19T17:00:02.733479+0000","last_undegraded":"2023-12-19T17:00:02.733479+0000","last_fullsized":"2023-12-19T17:00:02.733479+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:17.713011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733408+0000","last_change":"2023-12-19T17:00:01.044270+0000","last_active":"2023-12-19T17:00:02.733408+0000","last_peered":"2023-12-19T17:00:02.733408+0000","last_clean":"2023-12-19T17:00:02.733408+0000","last_became_active":"2023-12-19T17:00:01.042770+0000","last_became_peered":"2023-12-19T17:00:01.042770+0000","last_unstale":"2023-12-19T17:00:02.733408+0000","last_undegraded":"2023-12-19T17:00:02.733408+0000","last_fullsized":"2023-12-19T17:00:02.733408+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:41:48.345922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438189+0000","last_change":"2023-12-19T17:00:01.437736+0000","last_active":"2023-12-19T17:00:02.438189+0000","last_peered":"2023-12-19T17:00:02.438189+0000","last_clean":"2023-12-19T17:00:02.438189+0000","last_became_active":"2023-12-19T17:00:01.437469+0000","last_became_peered":"2023-12-19T17:00:01.437469+0000","last_unstale":"2023-12-19T17:00:02.438189+0000","last_undegraded":"2023-12-19T17:00:02.438189+0000","last_fullsized":"2023-12-19T17:00:02.438189+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:08:30.255976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733929+0000","last_change":"2023-12-19T17:00:01.037812+0000","last_active":"2023-12-19T17:00:02.733929+0000","last_peered":"2023-12-19T17:00:02.733929+0000","last_clean":"2023-12-19T17:00:02.733929+0000","last_became_active":"2023-12-19T17:00:01.037549+0000","last_became_peered":"2023-12-19T17:00:01.037549+0000","last_unstale":"2023-12-19T17:00:02.733929+0000","last_undegraded":"2023-12-19T17:00:02.733929+0000","last_fullsized":"2023-12-19T17:00:02.733929+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:20:48.883394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734519+0000","last_change":"2023-12-19T17:00:01.447194+0000","last_active":"2023-12-19T17:00:02.734519+0000","last_peered":"2023-12-19T17:00:02.734519+0000","last_clean":"2023-12-19T17:00:02.734519+0000","last_became_active":"2023-12-19T17:00:01.446882+0000","last_became_peered":"2023-12-19T17:00:01.446882+0000","last_unstale":"2023-12-19T17:00:02.734519+0000","last_undegraded":"2023-12-19T17:00:02.734519+0000","last_fullsized":"2023-12-19T17:00:02.734519+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:04:18.408128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734135+0000","last_change":"2023-12-19T17:00:01.043355+0000","last_active":"2023-12-19T17:00:02.734135+0000","last_peered":"2023-12-19T17:00:02.734135+0000","last_clean":"2023-12-19T17:00:02.734135+0000","last_became_active":"2023-12-19T17:00:01.040660+0000","last_became_peered":"2023-12-19T17:00:01.040660+0000","last_unstale":"2023-12-19T17:00:02.734135+0000","last_undegraded":"2023-12-19T17:00:02.734135+0000","last_fullsized":"2023-12-19T17:00:02.734135+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:04:05.450288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439504+0000","last_change":"2023-12-19T17:00:01.459130+0000","last_active":"2023-12-19T17:00:02.439504+0000","last_peered":"2023-12-19T17:00:02.439504+0000","last_clean":"2023-12-19T17:00:02.439504+0000","last_became_active":"2023-12-19T17:00:01.458735+0000","last_became_peered":"2023-12-19T17:00:01.458735+0000","last_unstale":"2023-12-19T17:00:02.439504+0000","last_undegraded":"2023-12-19T17:00:02.439504+0000","last_fullsized":"2023-12-19T17:00:02.439504+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:43:15.214527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"122'3","reported_seq":14,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734335+0000","last_change":"2023-12-19T17:00:01.045639+0000","last_active":"2023-12-19T17:00:02.734335+0000","last_peered":"2023-12-19T17:00:02.734335+0000","last_clean":"2023-12-19T17:00:02.734335+0000","last_became_active":"2023-12-19T17:00:01.045370+0000","last_became_peered":"2023-12-19T17:00:01.045370+0000","last_unstale":"2023-12-19T17:00:02.734335+0000","last_undegraded":"2023-12-19T17:00:02.734335+0000","last_fullsized":"2023-12-19T17:00:02.734335+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:34.311769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734196+0000","last_change":"2023-12-19T17:00:01.464531+0000","last_active":"2023-12-19T17:00:02.734196+0000","last_peered":"2023-12-19T17:00:02.734196+0000","last_clean":"2023-12-19T17:00:02.734196+0000","last_became_active":"2023-12-19T17:00:01.464220+0000","last_became_peered":"2023-12-19T17:00:01.464220+0000","last_unstale":"2023-12-19T17:00:02.734196+0000","last_undegraded":"2023-12-19T17:00:02.734196+0000","last_fullsized":"2023-12-19T17:00:02.734196+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:19.044976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"122'6","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734068+0000","last_change":"2023-12-19T17:00:01.045307+0000","last_active":"2023-12-19T17:00:02.734068+0000","last_peered":"2023-12-19T17:00:02.734068+0000","last_clean":"2023-12-19T17:00:02.734068+0000","last_became_active":"2023-12-19T17:00:01.045019+0000","last_became_peered":"2023-12-19T17:00:01.045019+0000","last_unstale":"2023-12-19T17:00:02.734068+0000","last_undegraded":"2023-12-19T17:00:02.734068+0000","last_fullsized":"2023-12-19T17:00:02.734068+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:42:45.648537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733680+0000","last_change":"2023-12-19T17:00:01.451874+0000","last_active":"2023-12-19T17:00:02.733680+0000","last_peered":"2023-12-19T17:00:02.733680+0000","last_clean":"2023-12-19T17:00:02.733680+0000","last_became_active":"2023-12-19T17:00:01.451583+0000","last_became_peered":"2023-12-19T17:00:01.451583+0000","last_unstale":"2023-12-19T17:00:02.733680+0000","last_undegraded":"2023-12-19T17:00:02.733680+0000","last_fullsized":"2023-12-19T17:00:02.733680+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:56.037615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734623+0000","last_change":"2023-12-19T17:00:01.045633+0000","last_active":"2023-12-19T17:00:02.734623+0000","last_peered":"2023-12-19T17:00:02.734623+0000","last_clean":"2023-12-19T17:00:02.734623+0000","last_became_active":"2023-12-19T17:00:01.045311+0000","last_became_peered":"2023-12-19T17:00:01.045311+0000","last_unstale":"2023-12-19T17:00:02.734623+0000","last_undegraded":"2023-12-19T17:00:02.734623+0000","last_fullsized":"2023-12-19T17:00:02.734623+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:17:31.980824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733560+0000","last_change":"2023-12-19T17:00:01.454203+0000","last_active":"2023-12-19T17:00:02.733560+0000","last_peered":"2023-12-19T17:00:02.733560+0000","last_clean":"2023-12-19T17:00:02.733560+0000","last_became_active":"2023-12-19T17:00:01.453940+0000","last_became_peered":"2023-12-19T17:00:01.453940+0000","last_unstale":"2023-12-19T17:00:02.733560+0000","last_undegraded":"2023-12-19T17:00:02.733560+0000","last_fullsized":"2023-12-19T17:00:02.733560+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:17.713011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734438+0000","last_change":"2023-12-19T17:00:01.047398+0000","last_active":"2023-12-19T17:00:02.734438+0000","last_peered":"2023-12-19T17:00:02.734438+0000","last_clean":"2023-12-19T17:00:02.734438+0000","last_became_active":"2023-12-19T17:00:01.047119+0000","last_became_peered":"2023-12-19T17:00:01.047119+0000","last_unstale":"2023-12-19T17:00:02.734438+0000","last_undegraded":"2023-12-19T17:00:02.734438+0000","last_fullsized":"2023-12-19T17:00:02.734438+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:04.654653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733453+0000","last_change":"2023-12-19T17:00:01.442043+0000","last_active":"2023-12-19T17:00:02.733453+0000","last_peered":"2023-12-19T17:00:02.733453+0000","last_clean":"2023-12-19T17:00:02.733453+0000","last_became_active":"2023-12-19T17:00:01.441759+0000","last_became_peered":"2023-12-19T17:00:01.441759+0000","last_unstale":"2023-12-19T17:00:02.733453+0000","last_undegraded":"2023-12-19T17:00:02.733453+0000","last_fullsized":"2023-12-19T17:00:02.733453+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:12:00.585820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733539+0000","last_change":"2023-12-19T17:00:01.041719+0000","last_active":"2023-12-19T17:00:02.733539+0000","last_peered":"2023-12-19T17:00:02.733539+0000","last_clean":"2023-12-19T17:00:02.733539+0000","last_became_active":"2023-12-19T17:00:01.041318+0000","last_became_peered":"2023-12-19T17:00:01.041318+0000","last_unstale":"2023-12-19T17:00:02.733539+0000","last_undegraded":"2023-12-19T17:00:02.733539+0000","last_fullsized":"2023-12-19T17:00:02.733539+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:52:11.763337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734172+0000","last_change":"2023-12-19T17:00:01.448648+0000","last_active":"2023-12-19T17:00:02.734172+0000","last_peered":"2023-12-19T17:00:02.734172+0000","last_clean":"2023-12-19T17:00:02.734172+0000","last_became_active":"2023-12-19T17:00:01.448163+0000","last_became_peered":"2023-12-19T17:00:01.448163+0000","last_unstale":"2023-12-19T17:00:02.734172+0000","last_undegraded":"2023-12-19T17:00:02.734172+0000","last_fullsized":"2023-12-19T17:00:02.734172+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:25.056126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"122'8","reported_seq":19,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733521+0000","last_change":"2023-12-19T17:00:01.046175+0000","last_active":"2023-12-19T17:00:02.733521+0000","last_peered":"2023-12-19T17:00:02.733521+0000","last_clean":"2023-12-19T17:00:02.733521+0000","last_became_active":"2023-12-19T17:00:01.045397+0000","last_became_peered":"2023-12-19T17:00:01.045397+0000","last_unstale":"2023-12-19T17:00:02.733521+0000","last_undegraded":"2023-12-19T17:00:02.733521+0000","last_fullsized":"2023-12-19T17:00:02.733521+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:19:52.426515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439690+0000","last_change":"2023-12-19T17:00:01.449596+0000","last_active":"2023-12-19T17:00:02.439690+0000","last_peered":"2023-12-19T17:00:02.439690+0000","last_clean":"2023-12-19T17:00:02.439690+0000","last_became_active":"2023-12-19T17:00:01.449172+0000","last_became_peered":"2023-12-19T17:00:01.449172+0000","last_unstale":"2023-12-19T17:00:02.439690+0000","last_undegraded":"2023-12-19T17:00:02.439690+0000","last_fullsized":"2023-12-19T17:00:02.439690+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:34.348541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734389+0000","last_change":"2023-12-19T17:00:01.044427+0000","last_active":"2023-12-19T17:00:02.734389+0000","last_peered":"2023-12-19T17:00:02.734389+0000","last_clean":"2023-12-19T17:00:02.734389+0000","last_became_active":"2023-12-19T17:00:01.044072+0000","last_became_peered":"2023-12-19T17:00:01.044072+0000","last_unstale":"2023-12-19T17:00:02.734389+0000","last_undegraded":"2023-12-19T17:00:02.734389+0000","last_fullsized":"2023-12-19T17:00:02.734389+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:12:24.444051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438572+0000","last_change":"2023-12-19T17:00:01.455551+0000","last_active":"2023-12-19T17:00:02.438572+0000","last_peered":"2023-12-19T17:00:02.438572+0000","last_clean":"2023-12-19T17:00:02.438572+0000","last_became_active":"2023-12-19T17:00:01.455284+0000","last_became_peered":"2023-12-19T17:00:01.455284+0000","last_unstale":"2023-12-19T17:00:02.438572+0000","last_undegraded":"2023-12-19T17:00:02.438572+0000","last_fullsized":"2023-12-19T17:00:02.438572+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:01:22.807158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"122'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439455+0000","last_change":"2023-12-19T17:00:01.040987+0000","last_active":"2023-12-19T17:00:02.439455+0000","last_peered":"2023-12-19T17:00:02.439455+0000","last_clean":"2023-12-19T17:00:02.439455+0000","last_became_active":"2023-12-19T17:00:01.040684+0000","last_became_peered":"2023-12-19T17:00:01.040684+0000","last_unstale":"2023-12-19T17:00:02.439455+0000","last_undegraded":"2023-12-19T17:00:02.439455+0000","last_fullsized":"2023-12-19T17:00:02.439455+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:55:15.222559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734211+0000","last_change":"2023-12-19T17:00:01.461410+0000","last_active":"2023-12-19T17:00:02.734211+0000","last_peered":"2023-12-19T17:00:02.734211+0000","last_clean":"2023-12-19T17:00:02.734211+0000","last_became_active":"2023-12-19T17:00:01.461125+0000","last_became_peered":"2023-12-19T17:00:01.461125+0000","last_unstale":"2023-12-19T17:00:02.734211+0000","last_undegraded":"2023-12-19T17:00:02.734211+0000","last_fullsized":"2023-12-19T17:00:02.734211+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:45:54.781258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438008+0000","last_change":"2023-12-19T17:00:01.040204+0000","last_active":"2023-12-19T17:00:02.438008+0000","last_peered":"2023-12-19T17:00:02.438008+0000","last_clean":"2023-12-19T17:00:02.438008+0000","last_became_active":"2023-12-19T17:00:01.039889+0000","last_became_peered":"2023-12-19T17:00:01.039889+0000","last_unstale":"2023-12-19T17:00:02.438008+0000","last_undegraded":"2023-12-19T17:00:02.438008+0000","last_fullsized":"2023-12-19T17:00:02.438008+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:24:17.214776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734351+0000","last_change":"2023-12-19T17:00:01.456618+0000","last_active":"2023-12-19T17:00:02.734351+0000","last_peered":"2023-12-19T17:00:02.734351+0000","last_clean":"2023-12-19T17:00:02.734351+0000","last_became_active":"2023-12-19T17:00:01.456320+0000","last_became_peered":"2023-12-19T17:00:01.456320+0000","last_unstale":"2023-12-19T17:00:02.734351+0000","last_undegraded":"2023-12-19T17:00:02.734351+0000","last_fullsized":"2023-12-19T17:00:02.734351+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:06.671287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437723+0000","last_change":"2023-12-19T17:00:01.048293+0000","last_active":"2023-12-19T17:00:02.437723+0000","last_peered":"2023-12-19T17:00:02.437723+0000","last_clean":"2023-12-19T17:00:02.437723+0000","last_became_active":"2023-12-19T17:00:01.047807+0000","last_became_peered":"2023-12-19T17:00:01.047807+0000","last_unstale":"2023-12-19T17:00:02.437723+0000","last_undegraded":"2023-12-19T17:00:02.437723+0000","last_fullsized":"2023-12-19T17:00:02.437723+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:14:26.525327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734526+0000","last_change":"2023-12-19T17:00:01.446512+0000","last_active":"2023-12-19T17:00:02.734526+0000","last_peered":"2023-12-19T17:00:02.734526+0000","last_clean":"2023-12-19T17:00:02.734526+0000","last_became_active":"2023-12-19T17:00:01.446250+0000","last_became_peered":"2023-12-19T17:00:01.446250+0000","last_unstale":"2023-12-19T17:00:02.734526+0000","last_undegraded":"2023-12-19T17:00:02.734526+0000","last_fullsized":"2023-12-19T17:00:02.734526+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:57:49.332518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438505+0000","last_change":"2023-12-19T17:00:01.044948+0000","last_active":"2023-12-19T17:00:02.438505+0000","last_peered":"2023-12-19T17:00:02.438505+0000","last_clean":"2023-12-19T17:00:02.438505+0000","last_became_active":"2023-12-19T17:00:01.044501+0000","last_became_peered":"2023-12-19T17:00:01.044501+0000","last_unstale":"2023-12-19T17:00:02.438505+0000","last_undegraded":"2023-12-19T17:00:02.438505+0000","last_fullsized":"2023-12-19T17:00:02.438505+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:03:33.414677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733877+0000","last_change":"2023-12-19T17:00:01.459066+0000","last_active":"2023-12-19T17:00:02.733877+0000","last_peered":"2023-12-19T17:00:02.733877+0000","last_clean":"2023-12-19T17:00:02.733877+0000","last_became_active":"2023-12-19T17:00:01.457362+0000","last_became_peered":"2023-12-19T17:00:01.457362+0000","last_unstale":"2023-12-19T17:00:02.733877+0000","last_undegraded":"2023-12-19T17:00:02.733877+0000","last_fullsized":"2023-12-19T17:00:02.733877+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:31.360615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438370+0000","last_change":"2023-12-19T17:00:01.046842+0000","last_active":"2023-12-19T17:00:02.438370+0000","last_peered":"2023-12-19T17:00:02.438370+0000","last_clean":"2023-12-19T17:00:02.438370+0000","last_became_active":"2023-12-19T17:00:01.046493+0000","last_became_peered":"2023-12-19T17:00:01.046493+0000","last_unstale":"2023-12-19T17:00:02.438370+0000","last_undegraded":"2023-12-19T17:00:02.438370+0000","last_fullsized":"2023-12-19T17:00:02.438370+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:20:48.883394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438729+0000","last_change":"2023-12-19T17:00:01.448890+0000","last_active":"2023-12-19T17:00:02.438729+0000","last_peered":"2023-12-19T17:00:02.438729+0000","last_clean":"2023-12-19T17:00:02.438729+0000","last_became_active":"2023-12-19T17:00:01.447594+0000","last_became_peered":"2023-12-19T17:00:01.447594+0000","last_unstale":"2023-12-19T17:00:02.438729+0000","last_undegraded":"2023-12-19T17:00:02.438729+0000","last_fullsized":"2023-12-19T17:00:02.438729+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:18:24.878752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437562+0000","last_change":"2023-12-19T17:00:01.046453+0000","last_active":"2023-12-19T17:00:02.437562+0000","last_peered":"2023-12-19T17:00:02.437562+0000","last_clean":"2023-12-19T17:00:02.437562+0000","last_became_active":"2023-12-19T17:00:01.046170+0000","last_became_peered":"2023-12-19T17:00:01.046170+0000","last_unstale":"2023-12-19T17:00:02.437562+0000","last_undegraded":"2023-12-19T17:00:02.437562+0000","last_fullsized":"2023-12-19T17:00:02.437562+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:43:01.663266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438275+0000","last_change":"2023-12-19T17:00:01.448980+0000","last_active":"2023-12-19T17:00:02.438275+0000","last_peered":"2023-12-19T17:00:02.438275+0000","last_clean":"2023-12-19T17:00:02.438275+0000","last_became_active":"2023-12-19T17:00:01.448727+0000","last_became_peered":"2023-12-19T17:00:01.448727+0000","last_unstale":"2023-12-19T17:00:02.438275+0000","last_undegraded":"2023-12-19T17:00:02.438275+0000","last_fullsized":"2023-12-19T17:00:02.438275+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:59:55.634822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439547+0000","last_change":"2023-12-19T17:00:01.037313+0000","last_active":"2023-12-19T17:00:02.439547+0000","last_peered":"2023-12-19T17:00:02.439547+0000","last_clean":"2023-12-19T17:00:02.439547+0000","last_became_active":"2023-12-19T17:00:01.036914+0000","last_became_peered":"2023-12-19T17:00:01.036914+0000","last_unstale":"2023-12-19T17:00:02.439547+0000","last_undegraded":"2023-12-19T17:00:02.439547+0000","last_fullsized":"2023-12-19T17:00:02.439547+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:41:48.345922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733844+0000","last_change":"2023-12-19T17:00:01.446677+0000","last_active":"2023-12-19T17:00:02.733844+0000","last_peered":"2023-12-19T17:00:02.733844+0000","last_clean":"2023-12-19T17:00:02.733844+0000","last_became_active":"2023-12-19T17:00:01.446402+0000","last_became_peered":"2023-12-19T17:00:01.446402+0000","last_unstale":"2023-12-19T17:00:02.733844+0000","last_undegraded":"2023-12-19T17:00:02.733844+0000","last_fullsized":"2023-12-19T17:00:02.733844+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:13:24.059700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438240+0000","last_change":"2023-12-19T17:00:01.047171+0000","last_active":"2023-12-19T17:00:02.438240+0000","last_peered":"2023-12-19T17:00:02.438240+0000","last_clean":"2023-12-19T17:00:02.438240+0000","last_became_active":"2023-12-19T17:00:01.046910+0000","last_became_peered":"2023-12-19T17:00:01.046910+0000","last_unstale":"2023-12-19T17:00:02.438240+0000","last_undegraded":"2023-12-19T17:00:02.438240+0000","last_fullsized":"2023-12-19T17:00:02.438240+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:38:56.371491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733628+0000","last_change":"2023-12-19T17:00:01.456285+0000","last_active":"2023-12-19T17:00:02.733628+0000","last_peered":"2023-12-19T17:00:02.733628+0000","last_clean":"2023-12-19T17:00:02.733628+0000","last_became_active":"2023-12-19T17:00:01.456005+0000","last_became_peered":"2023-12-19T17:00:01.456005+0000","last_unstale":"2023-12-19T17:00:02.733628+0000","last_undegraded":"2023-12-19T17:00:02.733628+0000","last_fullsized":"2023-12-19T17:00:02.733628+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:19:01.646545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438094+0000","last_change":"2023-12-19T17:00:01.041852+0000","last_active":"2023-12-19T17:00:02.438094+0000","last_peered":"2023-12-19T17:00:02.438094+0000","last_clean":"2023-12-19T17:00:02.438094+0000","last_became_active":"2023-12-19T17:00:01.039894+0000","last_became_peered":"2023-12-19T17:00:01.039894+0000","last_unstale":"2023-12-19T17:00:02.438094+0000","last_undegraded":"2023-12-19T17:00:02.438094+0000","last_fullsized":"2023-12-19T17:00:02.438094+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:27.050279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439343+0000","last_change":"2023-12-19T17:00:01.453472+0000","last_active":"2023-12-19T17:00:02.439343+0000","last_peered":"2023-12-19T17:00:02.439343+0000","last_clean":"2023-12-19T17:00:02.439343+0000","last_became_active":"2023-12-19T17:00:01.453172+0000","last_became_peered":"2023-12-19T17:00:01.453172+0000","last_unstale":"2023-12-19T17:00:02.439343+0000","last_undegraded":"2023-12-19T17:00:02.439343+0000","last_fullsized":"2023-12-19T17:00:02.439343+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:21.252414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.439591+0000","last_change":"2023-12-19T17:00:01.043008+0000","last_active":"2023-12-19T17:00:02.439591+0000","last_peered":"2023-12-19T17:00:02.439591+0000","last_clean":"2023-12-19T17:00:02.439591+0000","last_became_active":"2023-12-19T17:00:01.042702+0000","last_became_peered":"2023-12-19T17:00:01.042702+0000","last_unstale":"2023-12-19T17:00:02.439591+0000","last_undegraded":"2023-12-19T17:00:02.439591+0000","last_fullsized":"2023-12-19T17:00:02.439591+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:11.270666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734398+0000","last_change":"2023-12-19T17:00:01.450048+0000","last_active":"2023-12-19T17:00:02.734398+0000","last_peered":"2023-12-19T17:00:02.734398+0000","last_clean":"2023-12-19T17:00:02.734398+0000","last_became_active":"2023-12-19T17:00:01.448266+0000","last_became_peered":"2023-12-19T17:00:01.448266+0000","last_unstale":"2023-12-19T17:00:02.734398+0000","last_undegraded":"2023-12-19T17:00:02.734398+0000","last_fullsized":"2023-12-19T17:00:02.734398+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:42.373477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.437855+0000","last_change":"2023-12-19T17:00:01.048091+0000","last_active":"2023-12-19T17:00:02.437855+0000","last_peered":"2023-12-19T17:00:02.437855+0000","last_clean":"2023-12-19T17:00:02.437855+0000","last_became_active":"2023-12-19T17:00:01.047443+0000","last_became_peered":"2023-12-19T17:00:01.047443+0000","last_unstale":"2023-12-19T17:00:02.437855+0000","last_undegraded":"2023-12-19T17:00:02.437855+0000","last_fullsized":"2023-12-19T17:00:02.437855+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:24:44.265597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733802+0000","last_change":"2023-12-19T17:00:01.447116+0000","last_active":"2023-12-19T17:00:02.733802+0000","last_peered":"2023-12-19T17:00:02.733802+0000","last_clean":"2023-12-19T17:00:02.733802+0000","last_became_active":"2023-12-19T17:00:01.445512+0000","last_became_peered":"2023-12-19T17:00:01.445512+0000","last_unstale":"2023-12-19T17:00:02.733802+0000","last_undegraded":"2023-12-19T17:00:02.733802+0000","last_fullsized":"2023-12-19T17:00:02.733802+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:13:24.059700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438859+0000","last_change":"2023-12-19T17:00:01.050200+0000","last_active":"2023-12-19T17:00:02.438859+0000","last_peered":"2023-12-19T17:00:02.438859+0000","last_clean":"2023-12-19T17:00:02.438859+0000","last_became_active":"2023-12-19T17:00:01.049811+0000","last_became_peered":"2023-12-19T17:00:01.049811+0000","last_unstale":"2023-12-19T17:00:02.438859+0000","last_undegraded":"2023-12-19T17:00:02.438859+0000","last_fullsized":"2023-12-19T17:00:02.438859+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:44:47.234295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733983+0000","last_change":"2023-12-19T17:00:01.449051+0000","last_active":"2023-12-19T17:00:02.733983+0000","last_peered":"2023-12-19T17:00:02.733983+0000","last_clean":"2023-12-19T17:00:02.733983+0000","last_became_active":"2023-12-19T17:00:01.448269+0000","last_became_peered":"2023-12-19T17:00:01.448269+0000","last_unstale":"2023-12-19T17:00:02.733983+0000","last_undegraded":"2023-12-19T17:00:02.733983+0000","last_fullsized":"2023-12-19T17:00:02.733983+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:51:42.827617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.733646+0000","last_change":"2023-12-19T17:00:01.049007+0000","last_active":"2023-12-19T17:00:02.733646+0000","last_peered":"2023-12-19T17:00:02.733646+0000","last_clean":"2023-12-19T17:00:02.733646+0000","last_became_active":"2023-12-19T17:00:01.048650+0000","last_became_peered":"2023-12-19T17:00:01.048650+0000","last_unstale":"2023-12-19T17:00:02.733646+0000","last_undegraded":"2023-12-19T17:00:02.733646+0000","last_fullsized":"2023-12-19T17:00:02.733646+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:52.236059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.438836+0000","last_change":"2023-12-19T17:00:01.451719+0000","last_active":"2023-12-19T17:00:02.438836+0000","last_peered":"2023-12-19T17:00:02.438836+0000","last_clean":"2023-12-19T17:00:02.438836+0000","last_became_active":"2023-12-19T17:00:01.450749+0000","last_became_peered":"2023-12-19T17:00:01.450749+0000","last_unstale":"2023-12-19T17:00:02.438836+0000","last_undegraded":"2023-12-19T17:00:02.438836+0000","last_fullsized":"2023-12-19T17:00:02.438836+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:54.706706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734034+0000","last_change":"2023-12-19T17:00:01.046499+0000","last_active":"2023-12-19T17:00:02.734034+0000","last_peered":"2023-12-19T17:00:02.734034+0000","last_clean":"2023-12-19T17:00:02.734034+0000","last_became_active":"2023-12-19T17:00:01.043118+0000","last_became_peered":"2023-12-19T17:00:01.043118+0000","last_unstale":"2023-12-19T17:00:02.734034+0000","last_undegraded":"2023-12-19T17:00:02.734034+0000","last_fullsized":"2023-12-19T17:00:02.734034+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:54.763702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734585+0000","last_change":"2023-12-19T17:00:01.456952+0000","last_active":"2023-12-19T17:00:02.734585+0000","last_peered":"2023-12-19T17:00:02.734585+0000","last_clean":"2023-12-19T17:00:02.734585+0000","last_became_active":"2023-12-19T17:00:01.456431+0000","last_became_peered":"2023-12-19T17:00:01.456431+0000","last_unstale":"2023-12-19T17:00:02.734585+0000","last_undegraded":"2023-12-19T17:00:02.734585+0000","last_fullsized":"2023-12-19T17:00:02.734585+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:01.019735+0000","last_clean_scrub_stamp":"2023-12-19T17:00:01.019735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:02:56.620726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"122'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-19T17:00:02.734485+0000","last_change":"2023-12-19T17:00:01.045830+0000","last_active":"2023-12-19T17:00:02.734485+0000","last_peered":"2023-12-19T17:00:02.734485+0000","last_clean":"2023-12-19T17:00:02.734485+0000","last_became_active":"2023-12-19T17:00:01.045145+0000","last_became_peered":"2023-12-19T17:00:01.045145+0000","last_unstale":"2023-12-19T17:00:02.734485+0000","last_undegraded":"2023-12-19T17:00:02.734485+0000","last_fullsized":"2023-12-19T17:00:02.734485+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:00:00.017343+0000","last_clean_scrub_stamp":"2023-12-19T17:00:00.017343+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:43:14.212135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"111'225","reported_seq":639,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-19T17:00:01.444297+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T17:00:01.444297+0000","last_peered":"2023-12-19T17:00:01.444297+0000","last_clean":"2023-12-19T17:00:01.444297+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T17:00:01.444297+0000","last_undegraded":"2023-12-19T17:00:01.444297+0000","last_fullsized":"2023-12-19T17:00:01.444297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1058,"num_read_kb":2145,"num_write":355,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1058,"num_read_kb":2145,"num_write":355,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":225,"ondisk_log_size":225,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509653,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37196,"kb_used_data":2052,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334644,"statfs":{"total":96636764160,"available":96598675456,"internally_reserved":0,"allocated":2101248,"data_stored":1350689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.504,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":1.04,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.69599999999999995,"15min":0.70999999999999996},"min":{"1min":0.57999999999999996,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":0.95499999999999996,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69099999999999995,"15min":0.71099999999999997},"min":{"1min":0.56999999999999995,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0489999999999999,"5min":1.212,"15min":1.212},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.70899999999999996,"15min":0.73099999999999998},"min":{"1min":0.53600000000000003,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":1.016,"5min":1.74,"15min":1.74},"last":0.74399999999999999}]},{"osd":2,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.52600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88,"5min":1.9990000000000001,"15min":1.9990000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.503,"5min":0.5,"15min":0.44600000000000001},"max":{"1min":0.97199999999999998,"5min":1.395,"15min":1.395},"last":0.75600000000000001}]},{"osd":3,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98399999999999999,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.71999999999999997,"15min":0.74299999999999999},"min":{"1min":0.55600000000000005,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.012,"5min":1.266,"15min":1.266},"last":0.72999999999999998}]},{"osd":4,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.45700000000000002,"5min":0.308,"15min":0.28499999999999998},"max":{"1min":0.94399999999999995,"5min":1.841,"15min":1.841},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.61499999999999999,"15min":0.60399999999999998},"min":{"1min":0.441,"5min":0.40600000000000003,"15min":0.36599999999999999},"max":{"1min":1.0089999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.54700000000000004}]},{"osd":5,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.62,"15min":0.60699999999999998},"min":{"1min":0.44600000000000001,"5min":0.376,"15min":0.36899999999999999},"max":{"1min":1.0700000000000001,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.629},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64800000000000002,"15min":0.64400000000000002},"min":{"1min":0.50900000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0760000000000001,"5min":1.784,"15min":1.784},"last":0.65200000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.70599999999999996,"15min":0.69199999999999995},"min":{"1min":0.59899999999999998,"5min":0.45200000000000001,"15min":0.39800000000000002},"max":{"1min":1.0109999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.96499999999999997,"5min":2.105,"15min":2.105},"last":0.59899999999999998}]}]},{"osd":6,"up_from":15,"seq":64424509653,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1519616,"data_stored":767893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.68100000000000005,"15min":0.70899999999999996},"min":{"1min":0.60099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90300000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.68700000000000006,"15min":0.70999999999999996},"min":{"1min":0.56499999999999995,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":0.94299999999999995,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.61799999999999999}]},{"osd":1,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68000000000000005,"15min":0.70799999999999996},"min":{"1min":0.621,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.84699999999999998,"5min":1.135,"15min":1.135},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.54600000000000004,"5min":0.42699999999999999,"15min":0.39500000000000002},"max":{"1min":0.96699999999999997,"5min":1.238,"15min":1.238},"last":0.73299999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72799999999999998,"15min":0.745},"min":{"1min":0.61099999999999999,"5min":0.47599999999999998,"15min":0.35199999999999998},"max":{"1min":0.92800000000000005,"5min":1.264,"15min":1.264},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70199999999999996,"15min":0.73299999999999998},"min":{"1min":0.56799999999999995,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92200000000000004,"5min":1.0049999999999999,"15min":1.1879999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73599999999999999,"15min":0.755},"min":{"1min":0.59199999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.96499999999999997,"5min":1.413,"15min":1.413},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68400000000000005,"15min":0.71699999999999997},"min":{"1min":0.53400000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89100000000000001,"5min":0.94199999999999995,"15min":1.0740000000000001},"last":0.77500000000000002}]},{"osd":4,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58599999999999997,"15min":0.57499999999999996},"min":{"1min":0.443,"5min":0.36699999999999999,"15min":0.35899999999999999},"max":{"1min":0.76300000000000001,"5min":0.95399999999999996,"15min":0.96999999999999997},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.47599999999999998,"5min":0.40600000000000003,"15min":0.35699999999999998},"max":{"1min":0.75600000000000001,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71899999999999997}]},{"osd":5,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.33300000000000002},"max":{"1min":0.79100000000000004,"5min":0.85999999999999999,"15min":0.96699999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66800000000000004,"15min":0.65400000000000003},"min":{"1min":0.51900000000000002,"5min":0.42499999999999999,"15min":0.41499999999999998},"max":{"1min":0.82299999999999995,"5min":1.036,"15min":1.046},"last":0.79300000000000004}]},{"osd":7,"last update":"Tue Dec 19 16:59:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68899999999999995,"15min":0.69099999999999995},"min":{"1min":0.49199999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90600000000000003,"5min":1.006,"15min":1.0249999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.371},"max":{"1min":0.84999999999999998,"5min":0.94799999999999995,"15min":1.032},"last":0.82099999999999995}]}]},{"osd":5,"up_from":15,"seq":64424509653,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36624,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335216,"statfs":{"total":96636764160,"available":96599261184,"internally_reserved":0,"allocated":1515520,"data_stored":767311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.68100000000000005,"15min":0.70199999999999996},"min":{"1min":0.57099999999999995,"5min":0.34399999999999997,"15min":0.29499999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.95799999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73699999999999999,"15min":0.72999999999999998},"min":{"1min":0.59199999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.0189999999999999},"last":0.84299999999999997}]},{"osd":1,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.57499999999999996,"5min":0.39400000000000002,"15min":0.38600000000000001},"max":{"1min":0.91900000000000004,"5min":0.995,"15min":0.995},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72699999999999998,"15min":0.73099999999999998},"min":{"1min":0.59799999999999998,"5min":0.40200000000000002,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.67700000000000005}]},{"osd":2,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72999999999999998,"15min":0.73699999999999999},"min":{"1min":0.56399999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96999999999999997,"5min":1.026,"15min":1.0409999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70999999999999996,"15min":0.71999999999999997},"min":{"1min":0.58399999999999996,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.93600000000000005,"5min":1.036,"15min":1.036},"last":0.68899999999999995}]},{"osd":3,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.63100000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.98699999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.75900000000000001,"15min":0.76100000000000001},"min":{"1min":0.51700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0169999999999999,"5min":1.085,"15min":1.089},"last":0.86099999999999999}]},{"osd":4,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.33300000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83599999999999997,"5min":0.92600000000000005,"15min":0.97199999999999998},"last":0.46700000000000003}]},{"osd":6,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":0.89500000000000002,"15min":0.98799999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88700000000000001,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55900000000000005}]},{"osd":7,"last update":"Tue Dec 19 16:59:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.54100000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.877,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68400000000000005,"15min":0.67300000000000004},"min":{"1min":0.50600000000000001,"5min":0.45500000000000002,"15min":0.42099999999999999},"max":{"1min":0.874,"5min":1.1080000000000001,"15min":1.131},"last":0.58699999999999997}]}]},{"osd":4,"up_from":15,"seq":64424509653,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54688,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317152,"statfs":{"total":96636764160,"available":96580763648,"internally_reserved":0,"allocated":1531904,"data_stored":768929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.71399999999999997,"15min":0.71099999999999997},"min":{"1min":0.40899999999999997,"5min":0.32600000000000001,"15min":0.316},"max":{"1min":1.0309999999999999,"5min":1.877,"15min":1.877},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68999999999999995,"15min":0.70199999999999996},"min":{"1min":0.45500000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88400000000000001,"5min":0.94899999999999995,"15min":1.0680000000000001},"last":0.41399999999999998}]},{"osd":1,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68999999999999995,"15min":0.71099999999999997},"min":{"1min":0.42699999999999999,"5min":0.39000000000000001,"15min":0.34799999999999998},"max":{"1min":0.872,"5min":0.93799999999999994,"15min":1.0209999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.74299999999999999,"15min":0.73599999999999999},"min":{"1min":0.46300000000000002,"5min":0.37,"15min":0.35599999999999998},"max":{"1min":0.89200000000000002,"5min":2.02,"15min":2.02},"last":0.56699999999999995}]},{"osd":2,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71299999999999997,"15min":0.72899999999999998},"min":{"1min":0.57599999999999996,"5min":0.46800000000000003,"15min":0.38700000000000001},"max":{"1min":0.93500000000000005,"5min":0.97199999999999998,"15min":1.375},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.749,"15min":0.749},"min":{"1min":0.44900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95099999999999996,"5min":1.927,"15min":1.927},"last":0.48099999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.71099999999999997},"min":{"1min":0.437,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91600000000000004,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73499999999999999,"15min":0.751},"min":{"1min":0.502,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":1.0049999999999999},"last":0.65800000000000003}]},{"osd":5,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.63300000000000001,"15min":0.60699999999999998},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.73899999999999999,"5min":1.99,"15min":1.99},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.621,"15min":0.60299999999999998},"min":{"1min":0.40799999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.72799999999999998}]},{"osd":6,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.61599999999999999},"min":{"1min":0.438,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79300000000000004,"5min":0.96299999999999997,"15min":0.97199999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68100000000000005,"15min":0.66300000000000003},"min":{"1min":0.44,"5min":0.39800000000000002,"15min":0.376},"max":{"1min":0.88800000000000001,"5min":1.96,"15min":1.96},"last":0.68799999999999994}]},{"osd":7,"last update":"Tue Dec 19 16:59:50 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66400000000000003,"15min":0.65600000000000003},"min":{"1min":0.437,"5min":0.376,"15min":0.376},"max":{"1min":0.97099999999999997,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.69699999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0189999999999999,"5min":1.103,"15min":1.103},"last":0.748}]}]},{"osd":3,"up_from":15,"seq":64424509653,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36624,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335216,"statfs":{"total":96636764160,"available":96599261184,"internally_reserved":0,"allocated":1515520,"data_stored":761795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.29399999999999998,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":0.68200000000000005,"5min":0.69199999999999995,"15min":1.2509999999999999},"last":0.307},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.502,"15min":0.50700000000000001},"min":{"1min":0.31,"5min":0.308,"15min":0.27100000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.163},"last":0.46200000000000002}]},{"osd":1,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.46500000000000002,"15min":0.46300000000000002},"min":{"1min":0.35599999999999998,"5min":0.29899999999999999,"15min":0.25900000000000001},"max":{"1min":0.66800000000000004,"5min":0.76300000000000001,"15min":0.77800000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.29699999999999999},"max":{"1min":0.73099999999999998,"5min":0.75900000000000001,"15min":1.952},"last":0.34899999999999998}]},{"osd":2,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53500000000000003,"15min":0.54500000000000004},"min":{"1min":0.44,"5min":0.34499999999999997,"15min":0.30099999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.45300000000000001,"5min":0.39500000000000002,"15min":0.36699999999999999},"max":{"1min":1.157,"5min":1.157,"15min":2.855},"last":0.40400000000000003}]},{"osd":4,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73999999999999999,"15min":0.78000000000000003},"min":{"1min":0.56999999999999995,"5min":0.50600000000000001,"15min":0.432},"max":{"1min":1.0309999999999999,"5min":1.081,"15min":4.234},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.749,"15min":0.77600000000000002},"min":{"1min":0.624,"5min":0.56100000000000005,"15min":0.46100000000000002},"max":{"1min":0.96699999999999997,"5min":1.111,"15min":1.3069999999999999},"last":0.63600000000000001}]},{"osd":5,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.752,"15min":0.77800000000000002},"min":{"1min":0.53000000000000003,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.1399999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.748,"15min":0.77700000000000002},"min":{"1min":0.58799999999999997,"5min":0.53900000000000003,"15min":0.43099999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.8300000000000001},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.749,"15min":0.78700000000000003},"min":{"1min":0.51500000000000001,"5min":0.51200000000000001,"15min":0.45800000000000002},"max":{"1min":1.054,"5min":1.0589999999999999,"15min":1.599},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.76900000000000002,"15min":0.79600000000000004},"min":{"1min":0.56299999999999994,"5min":0.54400000000000004,"15min":0.49199999999999999},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:59:55 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.80500000000000005,"15min":0.82499999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.385,"5min":1.385,"15min":1.385},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.79900000000000004,"15min":0.84499999999999997},"min":{"1min":0.67600000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.623},"last":0.68400000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509652,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58768,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313072,"statfs":{"total":96636764160,"available":96576585728,"internally_reserved":0,"allocated":1515520,"data_stored":767275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.44,"15min":0.46000000000000002},"min":{"1min":0.36299999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":5.4870000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.48499999999999999,"15min":0.53100000000000003},"min":{"1min":0.41299999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":10.161},"last":0.27100000000000002}]},{"osd":1,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.60199999999999998,"15min":0.61199999999999999},"min":{"1min":0.53400000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.76100000000000001,"5min":1.53,"15min":2.4740000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.42899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68799999999999994,"5min":1.673,"15min":2.911},"last":0.36399999999999999}]},{"osd":3,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57899999999999996,"15min":0.58999999999999997},"min":{"1min":0.504,"5min":0.378,"15min":0.378},"max":{"1min":0.755,"5min":0.79200000000000004,"15min":1.1990000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56000000000000005,"15min":0.57199999999999995},"min":{"1min":0.495,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.751,"5min":1.6279999999999999,"15min":2.7999999999999998},"last":0.441}]},{"osd":4,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.78300000000000003,"15min":0.78200000000000003},"min":{"1min":0.69399999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0069999999999999,"5min":2.4449999999999998,"15min":2.4449999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.75800000000000001,"15min":0.78500000000000003},"min":{"1min":0.63900000000000001,"5min":0.45200000000000001,"15min":0.42999999999999999},"max":{"1min":1.105,"5min":1.3049999999999999,"15min":5.5430000000000001},"last":0.88600000000000001}]},{"osd":5,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.58599999999999997,"5min":0.5,"15min":0.49299999999999999},"max":{"1min":1.048,"5min":1.4890000000000001,"15min":5.601},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.79300000000000004,"15min":0.80800000000000005},"min":{"1min":0.70099999999999996,"5min":0.504,"15min":0.504},"max":{"1min":0.97699999999999998,"5min":2.569,"15min":2.569},"last":0.93100000000000005}]},{"osd":6,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82799999999999996,"15min":0.82399999999999995},"min":{"1min":0.77100000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0780000000000001,"5min":2.3889999999999998,"15min":2.3889999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.754,"15min":0.78500000000000003},"min":{"1min":0.69299999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.94499999999999995,"5min":1.1950000000000001,"15min":1.4379999999999999},"last":0.98399999999999999}]},{"osd":7,"last update":"Tue Dec 19 16:59:41 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.71499999999999997,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.099,"5min":2.5409999999999999,"15min":5.673},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.872,"5min":0.80800000000000005,"15min":0.83299999999999996},"min":{"1min":0.72099999999999997,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.028,"5min":1.369,"15min":1.429},"last":1.024}]}]},{"osd":1,"up_from":15,"seq":64424509653,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50572,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321268,"statfs":{"total":96636764160,"available":96584978432,"internally_reserved":0,"allocated":1511424,"data_stored":767241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.441,"15min":0.44600000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.22500000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.84899999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.501,"15min":0.51700000000000002},"min":{"1min":0.30099999999999999,"5min":0.29699999999999999,"15min":0.26400000000000001},"max":{"1min":0.79500000000000004,"5min":1.071,"15min":1.071},"last":0.48899999999999999}]},{"osd":2,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.498,"15min":0.50900000000000001},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.30299999999999999},"max":{"1min":0.67000000000000004,"5min":0.78800000000000003,"15min":0.80400000000000005},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52800000000000002,"15min":0.55100000000000005},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.317},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.36199999999999999}]},{"osd":3,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58299999999999996,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81100000000000005,"5min":0.871,"15min":0.91500000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.50900000000000001,"15min":0.54000000000000004},"min":{"1min":0.41899999999999998,"5min":0.34499999999999997,"15min":0.33700000000000002},"max":{"1min":0.73999999999999999,"5min":0.755,"15min":0.91900000000000004},"last":0.505}]},{"osd":4,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.73099999999999998,"15min":0.755},"min":{"1min":0.51900000000000002,"5min":0.45300000000000001,"15min":0.43099999999999999},"max":{"1min":1.0509999999999999,"5min":1.107,"15min":1.321},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73999999999999999,"15min":0.77800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.002,"5min":1.0580000000000001,"15min":1.4179999999999999},"last":0.68100000000000005}]},{"osd":5,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.746,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.46500000000000002},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.4450000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.75800000000000001,"15min":0.79200000000000004},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.46500000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.3380000000000001},"last":0.59599999999999997}]},{"osd":6,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73699999999999999,"15min":0.78200000000000003},"min":{"1min":0.58899999999999997,"5min":0.496,"15min":0.41999999999999998},"max":{"1min":1.0089999999999999,"5min":1.048,"15min":1.3939999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76000000000000001,"15min":0.79800000000000004},"min":{"1min":0.58299999999999996,"5min":0.56200000000000006,"15min":0.52700000000000002},"max":{"1min":1.0940000000000001,"5min":1.1659999999999999,"15min":1.391},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Dec 19 16:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.79700000000000004,"15min":0.82399999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.2589999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77800000000000002,"15min":0.82199999999999995},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.53800000000000003},"max":{"1min":1.129,"5min":1.2050000000000001,"15min":1.4219999999999999},"last":0.69199999999999995}]}]},{"osd":0,"up_from":14,"seq":60129542358,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":2109440,"data_stored":1358191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.29699999999999999,"5min":0.26000000000000001,"15min":0.246},"max":{"1min":0.72699999999999998,"5min":0.752,"15min":0.752},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48499999999999999,"15min":0.49299999999999999},"min":{"1min":0.34100000000000003,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.72399999999999998,"5min":0.81699999999999995,"15min":0.85499999999999998},"last":0.69299999999999995}]},{"osd":2,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.496,"15min":0.497},"min":{"1min":0.36899999999999999,"5min":0.33300000000000002,"15min":0.31900000000000001},"max":{"1min":0.64100000000000001,"5min":0.71499999999999997,"15min":0.89700000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.41099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76600000000000001,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.59699999999999998}]},{"osd":3,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.93600000000000005,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.495,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91300000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.71299999999999997}]},{"osd":4,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.77100000000000002,"15min":0.78600000000000003},"min":{"1min":0.59399999999999997,"5min":0.503,"15min":0.45900000000000002},"max":{"1min":1.113,"5min":1.444,"15min":1.444},"last":1.145},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78100000000000003,"15min":0.79300000000000004},"min":{"1min":0.54600000000000004,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":0.95399999999999996,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.871}]},{"osd":5,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.80500000000000005,"15min":0.80300000000000005},"min":{"1min":0.53200000000000003,"5min":0.502,"15min":0.41499999999999998},"max":{"1min":1.0940000000000001,"5min":1.294,"15min":1.294},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80600000000000005,"15min":0.80100000000000005},"min":{"1min":0.54600000000000004,"5min":0.54400000000000004,"15min":0.52300000000000002},"max":{"1min":1.014,"5min":1.256,"15min":1.256},"last":1.0529999999999999}]},{"osd":6,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80300000000000005,"15min":0.80100000000000005},"min":{"1min":0.60099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.075,"5min":1.429,"15min":1.429},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.64900000000000002,"5min":0.53100000000000003,"15min":0.44700000000000001},"max":{"1min":1.0580000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.91900000000000004}]},{"osd":7,"last update":"Tue Dec 19 16:59:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81899999999999995,"15min":0.82899999999999996},"min":{"1min":0.56899999999999995,"5min":0.56499999999999995,"15min":0.48999999999999999},"max":{"1min":1.1639999999999999,"5min":1.377,"15min":1.377},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.83199999999999996,"15min":0.82399999999999995},"min":{"1min":0.58499999999999996,"5min":0.51400000000000001,"15min":0.48099999999999998},"max":{"1min":1.135,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.96599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T17:00:05.951 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T17:00:05.951 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T17:00:06.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.135+0000 7f1ba61a6640 1 Processor -- start 2023-12-19T17:00:06.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.135+0000 7f1ba61a6640 1 -- start start 2023-12-19T17:00:06.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1ba61a6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1ba015ada0 0x7f1ba00ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1ba61a6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba006b500 con 0x7f1ba00b0c30 2023-12-19T17:00:06.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1ba61a6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba0070b60 con 0x7f1ba0070db0 2023-12-19T17:00:06.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1ba61a6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba00715b0 con 0x7f1ba015ada0 2023-12-19T17:00:06.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1ba00b0c30 0x7f1ba006b990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42468/0 (socket says 172.21.15.26:42468) 2023-12-19T17:00:06.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9ffff640 1 -- 172.21.15.26:0/2540382888 learned_addr learned my addr 172.21.15.26:0/2540382888 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:00:06.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9f7fe640 1 --2- 172.21.15.26:0/2540382888 >> v2:172.21.15.160:3300/0 conn(0x7f1ba015ada0 0x7f1ba00ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892105570 0 0) 0x7f1ba006b500 con 0x7f1ba00b0c30 2023-12-19T17:00:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b88003530 con 0x7f1ba00b0c30 2023-12-19T17:00:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235452590 0 0) 0x7f1ba0070b60 con 0x7f1ba0070db0 2023-12-19T17:00:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ba006b500 con 0x7f1ba0070db0 2023-12-19T17:00:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2247797740 0 0) 0x7f1b88003530 con 0x7f1ba00b0c30 2023-12-19T17:00:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 >> v2:172.21.15.160:3300/0 conn(0x7f1ba015ada0 msgr2=0x7f1ba00ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 --2- 172.21.15.26:0/2540382888 >> v2:172.21.15.160:3300/0 conn(0x7f1ba015ada0 0x7f1ba00ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 >> v1:172.21.15.160:6789/0 conn(0x7f1ba0070db0 legacy=0x7f1ba00b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba00702c0 con 0x7f1ba00b0c30 2023-12-19T17:00:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f1b98001a00 con 0x7f1ba00b0c30 2023-12-19T17:00:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9f7fe640 1 --2- 172.21.15.26:0/2540382888 >> v2:172.21.15.160:3300/0 conn(0x7f1ba015ada0 0x7f1ba00ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:00:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 >> v1:172.21.15.26:6789/0 conn(0x7f1ba00b0c30 legacy=0x7f1ba006b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 --2- 172.21.15.26:0/2540382888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b88003b10 0x7f1b88003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 --2- 172.21.15.26:0/2540382888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b880047d0 0x7f1b88004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 --2- 172.21.15.26:0/2540382888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b88005190 0x7f1b8800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba00715b0 con 0x7f1b880047d0 2023-12-19T17:00:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8800c0c0 con 0x7f1b88005190 2023-12-19T17:00:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8800c280 con 0x7f1b88003b10 2023-12-19T17:00:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9effd640 1 --2- 172.21.15.26:0/2540382888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b880047d0 0x7f1b88004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9ffff640 1 --2- 172.21.15.26:0/2540382888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b88005190 0x7f1b8800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9f7fe640 1 --2- 172.21.15.26:0/2540382888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b88003b10 0x7f1b88003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9effd640 1 -- 172.21.15.26:0/2540382888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b88003b10 msgr2=0x7f1b88003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:06.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9effd640 1 --2- 172.21.15.26:0/2540382888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1b88003b10 0x7f1b88003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9effd640 1 -- 172.21.15.26:0/2540382888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b88005190 msgr2=0x7f1b8800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:06.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9effd640 1 --2- 172.21.15.26:0/2540382888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b88005190 0x7f1b8800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9effd640 1 -- 172.21.15.26:0/2540382888 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b88003530 con 0x7f1b880047d0 2023-12-19T17:00:06.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.139+0000 7f1b9ffff640 1 -- 172.21.15.26:0/2540382888 reap_dead start 2023-12-19T17:00:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.143+0000 7f1b9effd640 1 --2- 172.21.15.26:0/2540382888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b880047d0 0x7f1b88004c50 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f1b940018d0 tx=0x7f1b9401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62e6bc68a9516a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.143+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b940052d0 con 0x7f1b880047d0 2023-12-19T17:00:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.143+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b940a7050 con 0x7f1b880047d0 2023-12-19T17:00:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.143+0000 7f1b9e7fc640 1 -- 172.21.15.26:0/2540382888 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94003600 con 0x7f1b880047d0 2023-12-19T17:00:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.143+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2540382888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b880047d0 msgr2=0x7f1b88004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.143+0000 7f1ba61a6640 1 --2- 172.21.15.26:0/2540382888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b880047d0 0x7f1b88004c50 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f1b940018d0 tx=0x7f1b9401c510 comp rx=0 tx=0).stop 2023-12-19T17:00:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.143+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2540382888 shutdown_connections 2023-12-19T17:00:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.143+0000 7f1ba61a6640 1 --2- 172.21.15.26:0/2540382888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b880047d0 0x7f1b88004c50 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.143+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2540382888 >> 172.21.15.26:0/2540382888 conn(0x7f1ba00660d0 msgr2=0x7f1ba00ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.143+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2540382888 shutdown_connections 2023-12-19T17:00:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.143+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2540382888 wait complete. 2023-12-19T17:00:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.143+0000 7f1ba61a6640 1 Processor -- start 2023-12-19T17:00:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1ba61a6640 1 -- start start 2023-12-19T17:00:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1ba61a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b880047d0 0x7f1ba01f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1ba61a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba00c8e80 0x7f1ba014b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1ba61a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba015d890 0x7f1ba015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1ba61a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba014b720 con 0x7f1ba015d890 2023-12-19T17:00:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1ba61a6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba015c7f0 con 0x7f1b880047d0 2023-12-19T17:00:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1ba61a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba015cad0 con 0x7f1ba00c8e80 2023-12-19T17:00:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b9ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba015d890 0x7f1ba015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b9ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba015d890 0x7f1ba015dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56610/0 (socket says 172.21.15.26:56610) 2023-12-19T17:00:06.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b9ffff640 1 -- 172.21.15.26:0/2308068968 learned_addr learned my addr 172.21.15.26:0/2308068968 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b9f7fe640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b880047d0 0x7f1ba01f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b9effd640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba00c8e80 0x7f1ba014b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b9ffff640 1 -- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba00c8e80 msgr2=0x7f1ba014b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b9ffff640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba00c8e80 0x7f1ba014b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b9ffff640 1 -- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b880047d0 msgr2=0x7f1ba01f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b9ffff640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b880047d0 0x7f1ba01f4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b9ffff640 1 -- 172.21.15.26:0/2308068968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba015cdb0 con 0x7f1ba015d890 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b9ffff640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba015d890 0x7f1ba015dd10 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00cb0c0 tx=0x7f1b98007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b87fff640 1 -- 172.21.15.26:0/2308068968 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9800aa90 con 0x7f1ba015d890 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b87fff640 1 -- 172.21.15.26:0/2308068968 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9800ac10 con 0x7f1ba015d890 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2308068968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba015d090 con 0x7f1ba015d890 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1b87fff640 1 -- 172.21.15.26:0/2308068968 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b980133f0 con 0x7f1ba015d890 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2308068968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba014bdc0 con 0x7f1ba015d890 2023-12-19T17:00:06.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.147+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2308068968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba0074270 con 0x7f1ba015d890 2023-12-19T17:00:06.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.151+0000 7f1b87fff640 1 -- 172.21.15.26:0/2308068968 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1b98013550 con 0x7f1ba015d890 2023-12-19T17:00:06.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.151+0000 7f1b87fff640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b8c065b80 0x7f1b8c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.151+0000 7f1b87fff640 1 -- 172.21.15.26:0/2308068968 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f1b980839f0 con 0x7f1ba015d890 2023-12-19T17:00:06.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.151+0000 7f1b9f7fe640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b8c065b80 0x7f1b8c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.151+0000 7f1b9f7fe640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b8c065b80 0x7f1b8c068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0066870 tx=0x7f1b90009190 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:06.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.155+0000 7f1b87fff640 1 -- 172.21.15.26:0/2308068968 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1b98051060 con 0x7f1ba015d890 2023-12-19T17:00:06.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.367+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2308068968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1ba006ba30 con 0x7f1ba015d890 2023-12-19T17:00:06.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.371+0000 7f1b87fff640 1 -- 172.21.15.26:0/2308068968 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1b98054680 con 0x7f1ba015d890 2023-12-19T17:00:06.374 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2023-12-19T17:00:06.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.375+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b8c065b80 msgr2=0x7f1b8c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:06.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.375+0000 7f1ba61a6640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b8c065b80 0x7f1b8c068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0066870 tx=0x7f1b90009190 comp rx=0 tx=0).stop 2023-12-19T17:00:06.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.375+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2308068968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba015d890 msgr2=0x7f1ba015dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:06.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.375+0000 7f1ba61a6640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba015d890 0x7f1ba015dd10 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00cb0c0 tx=0x7f1b98007ae0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.375+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2308068968 shutdown_connections 2023-12-19T17:00:06.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.375+0000 7f1ba61a6640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ba015d890 0x7f1ba015dd10 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.375+0000 7f1ba61a6640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1b8c065b80 0x7f1b8c068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.375+0000 7f1ba61a6640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ba00c8e80 0x7f1ba014b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.375+0000 7f1ba61a6640 1 --2- 172.21.15.26:0/2308068968 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b880047d0 0x7f1ba01f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.375+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2308068968 >> 172.21.15.26:0/2308068968 conn(0x7f1ba00660d0 msgr2=0x7f1ba00af260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:06.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.375+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2308068968 shutdown_connections 2023-12-19T17:00:06.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.375+0000 7f1ba61a6640 1 -- 172.21.15.26:0/2308068968 wait complete. 2023-12-19T17:00:06.396 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:00:06.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfd0ee5640 1 Processor -- start 2023-12-19T17:00:06.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfd0ee5640 1 -- start start 2023-12-19T17:00:06.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfd0ee5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfcc15c7f0 0x7fdfcc06c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfd0ee5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdfcc0adb80 0x7fdfcc0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfd0ee5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfcc06ced0 con 0x7fdfcc0ccc20 2023-12-19T17:00:06.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfd0ee5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfcc06d770 con 0x7fdfcc0adb80 2023-12-19T17:00:06.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfd0ee5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc06d1b0 con 0x7fdfcc15c7f0 2023-12-19T17:00:06.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfcad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdfcc0ccc20 0x7fdfcc0b30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42472/0 (socket says 172.21.15.26:42472) 2023-12-19T17:00:06.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfcad9d640 1 -- 172.21.15.26:0/1267976443 learned_addr learned my addr 172.21.15.26:0/1267976443 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:00:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfca59c640 1 --2- 172.21.15.26:0/1267976443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfcc15c7f0 0x7fdfcc06c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc3fff640 1 --2- 172.21.15.26:0/1267976443 >> v2:172.21.15.160:3300/0 conn(0x7fdfcc0adb80 0x7fdfcc0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526070767 0 0) 0x7fdfcc06ced0 con 0x7fdfcc0ccc20 2023-12-19T17:00:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb8003530 con 0x7fdfcc0ccc20 2023-12-19T17:00:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 745407357 0 0) 0x7fdfb8003530 con 0x7fdfcc0ccc20 2023-12-19T17:00:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfcc15c7f0 msgr2=0x7fdfcc06c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:06.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 --2- 172.21.15.26:0/1267976443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfcc15c7f0 0x7fdfcc06c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 >> v2:172.21.15.160:3300/0 conn(0x7fdfcc0adb80 msgr2=0x7fdfcc0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:06.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 --2- 172.21.15.26:0/1267976443 >> v2:172.21.15.160:3300/0 conn(0x7fdfcc0adb80 0x7fdfcc0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc06d490 con 0x7fdfcc0ccc20 2023-12-19T17:00:06.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fdfc4001a00 con 0x7fdfcc0ccc20 2023-12-19T17:00:06.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 >> v1:172.21.15.26:6789/0 conn(0x7fdfcc0ccc20 legacy=0x7fdfcc0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:06.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 --2- 172.21.15.26:0/1267976443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfb8003980 0x7fdfb8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 --2- 172.21.15.26:0/1267976443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfcc15c7f0 0x7fdfb80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 --2- 172.21.15.26:0/1267976443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb8004d10 0x7fdfb8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc06d770 con 0x7fdfb8003980 2023-12-19T17:00:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc06d1b0 con 0x7fdfb8004d10 2023-12-19T17:00:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb800dba0 con 0x7fdfcc15c7f0 2023-12-19T17:00:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfca59c640 1 --2- 172.21.15.26:0/1267976443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfb8003980 0x7fdfb8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfc3fff640 1 --2- 172.21.15.26:0/1267976443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfcc15c7f0 0x7fdfb80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfcad9d640 1 --2- 172.21.15.26:0/1267976443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb8004d10 0x7fdfb8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfca59c640 1 -- 172.21.15.26:0/1267976443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfcc15c7f0 msgr2=0x7fdfb80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfca59c640 1 --2- 172.21.15.26:0/1267976443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfcc15c7f0 0x7fdfb80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfca59c640 1 -- 172.21.15.26:0/1267976443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb8004d10 msgr2=0x7fdfb8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfca59c640 1 --2- 172.21.15.26:0/1267976443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb8004d10 0x7fdfb8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfca59c640 1 -- 172.21.15.26:0/1267976443 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb8003530 con 0x7fdfb8003980 2023-12-19T17:00:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.539+0000 7fdfcad9d640 1 --2- 172.21.15.26:0/1267976443 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb8004d10 0x7fdfb8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:00:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.543+0000 7fdfca59c640 1 --2- 172.21.15.26:0/1267976443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfb8003980 0x7fdfb8003da0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009ae0 tx=0x7fdfbc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450e27df347ff0e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.543+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0a5530 con 0x7fdfb8003980 2023-12-19T17:00:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.543+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc0a5690 con 0x7fdfb8003980 2023-12-19T17:00:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.543+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/1267976443 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0a5a80 con 0x7fdfb8003980 2023-12-19T17:00:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.543+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/1267976443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfb8003980 msgr2=0x7fdfb8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.543+0000 7fdfd0ee5640 1 --2- 172.21.15.26:0/1267976443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfb8003980 0x7fdfb8003da0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009ae0 tx=0x7fdfbc01ca50 comp rx=0 tx=0).stop 2023-12-19T17:00:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.543+0000 7fdfcad9d640 1 -- 172.21.15.26:0/1267976443 reap_dead start 2023-12-19T17:00:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.543+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/1267976443 shutdown_connections 2023-12-19T17:00:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.543+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/1267976443 >> 172.21.15.26:0/1267976443 conn(0x7fdfcc0658e0 msgr2=0x7fdfcc065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.543+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/1267976443 shutdown_connections 2023-12-19T17:00:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.543+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/1267976443 wait complete. 2023-12-19T17:00:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfd0ee5640 1 Processor -- start 2023-12-19T17:00:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfd0ee5640 1 -- start start 2023-12-19T17:00:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfd0ee5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfb8003980 0x7fdfcc0acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfd0ee5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc0c82a0 0x7fdfcc0c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfd0ee5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfcc0c6a20 0x7fdfcc14d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfd0ee5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc14ddd0 con 0x7fdfcc0c82a0 2023-12-19T17:00:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfd0ee5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc0c6ea0 con 0x7fdfcc0c6a20 2023-12-19T17:00:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfd0ee5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc1f5630 con 0x7fdfb8003980 2023-12-19T17:00:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc0c82a0 0x7fdfcc0c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc0c82a0 0x7fdfcc0c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56616/0 (socket says 172.21.15.26:56616) 2023-12-19T17:00:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfc3fff640 1 -- 172.21.15.26:0/672476948 learned_addr learned my addr 172.21.15.26:0/672476948 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfcad9d640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfcc0c6a20 0x7fdfcc14d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfca59c640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfb8003980 0x7fdfcc0acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfc3fff640 1 -- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfb8003980 msgr2=0x7fdfcc0acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfc3fff640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfb8003980 0x7fdfcc0acab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfc3fff640 1 -- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfcc0c6a20 msgr2=0x7fdfcc14d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfc3fff640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfcc0c6a20 0x7fdfcc14d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfc3fff640 1 -- 172.21.15.26:0/672476948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc14af40 con 0x7fdfcc0c82a0 2023-12-19T17:00:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfcad9d640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfcc0c6a20 0x7fdfcc14d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:00:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfca59c640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfb8003980 0x7fdfcc0acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:00:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfc3fff640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc0c82a0 0x7fdfcc0c64e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4011ad0 tx=0x7fdfb4011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfc2ffd640 1 -- 172.21.15.26:0/672476948 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb400fbb0 con 0x7fdfcc0c82a0 2023-12-19T17:00:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfc2ffd640 1 -- 172.21.15.26:0/672476948 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb400fd10 con 0x7fdfcc0c82a0 2023-12-19T17:00:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfc2ffd640 1 -- 172.21.15.26:0/672476948 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4014680 con 0x7fdfcc0c82a0 2023-12-19T17:00:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/672476948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfcc14b220 con 0x7fdfcc0c82a0 2023-12-19T17:00:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.547+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/672476948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfcc14b7e0 con 0x7fdfcc0c82a0 2023-12-19T17:00:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.551+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/672476948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfcc074270 con 0x7fdfcc0c82a0 2023-12-19T17:00:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.551+0000 7fdfc2ffd640 1 -- 172.21.15.26:0/672476948 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fdfb40147e0 con 0x7fdfcc0c82a0 2023-12-19T17:00:06.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.551+0000 7fdfc2ffd640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfac065bb0 0x7fdfac068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.551+0000 7fdfc2ffd640 1 -- 172.21.15.26:0/672476948 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fdfb4083f00 con 0x7fdfcc0c82a0 2023-12-19T17:00:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.551+0000 7fdfca59c640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfac065bb0 0x7fdfac068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:06.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.551+0000 7fdfca59c640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfac065bb0 0x7fdfac068070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0af000 tx=0x7fdfbc0046d0 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:06.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.555+0000 7fdfc2ffd640 1 -- 172.21.15.26:0/672476948 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdfb4051440 con 0x7fdfcc0c82a0 2023-12-19T17:00:06.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/672476948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfcc070fd0 con 0x7fdfcc0c82a0 2023-12-19T17:00:06.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfc2ffd640 1 -- 172.21.15.26:0/672476948 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fdfb4054a60 con 0x7fdfcc0c82a0 2023-12-19T17:00:06.749 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:00:06.749 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6884,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1869411616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1869411616},{"type":"v1","addr":"172.21.15.26:6835","nonce":1869411616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3129570713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3129570713},{"type":"v1","addr":"172.21.15.160:6836","nonce":3129570713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4063618989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":4063618989},{"type":"v1","addr":"172.21.15.160:6837","nonce":4063618989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:00:01.425222+0000","modified":"2023-12-19T17:00:04.444345+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7974":{"gid":7974,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/2837068623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2837068623},{"type":"v1","addr":"172.21.15.26:6834","nonce":2837068623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T17:00:06.749 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 57 2023-12-19T17:00:06.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfac065bb0 msgr2=0x7fdfac068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfd0ee5640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfac065bb0 0x7fdfac068070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0af000 tx=0x7fdfbc0046d0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/672476948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc0c82a0 msgr2=0x7fdfcc0c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfd0ee5640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc0c82a0 0x7fdfcc0c64e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4011ad0 tx=0x7fdfb4011f90 comp rx=0 tx=0).stop 2023-12-19T17:00:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/672476948 shutdown_connections 2023-12-19T17:00:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfd0ee5640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfcc0c6a20 0x7fdfcc14d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfd0ee5640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc0c82a0 0x7fdfcc0c64e0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfd0ee5640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfac065bb0 0x7fdfac068070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfd0ee5640 1 --2- 172.21.15.26:0/672476948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfb8003980 0x7fdfcc0acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/672476948 >> 172.21.15.26:0/672476948 conn(0x7fdfcc0658e0 msgr2=0x7fdfcc15b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/672476948 shutdown_connections 2023-12-19T17:00:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:06.747+0000 7fdfd0ee5640 1 -- 172.21.15.26:0/672476948 wait complete. 2023-12-19T17:00:07.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T17:00:01.425222+0000', 'modified': '2023-12-19T17:00:04.444345+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7974}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7974': {'gid': 7974, 'name': 'a', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6834/2837068623', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 2837068623}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 2837068623}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T17:00:07.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T17:00:07.173 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-19T17:00:07.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.319+0000 7f33bf066640 1 Processor -- start 2023-12-19T17:00:07.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.319+0000 7f33bf066640 1 -- start start 2023-12-19T17:00:07.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.319+0000 7f33bf066640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b806b3d0 0x7f33b80ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.319+0000 7f33bf066640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b806bc30 con 0x7f33b8154220 2023-12-19T17:00:07.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.319+0000 7f33bf066640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b80cc6e0 con 0x7f33b80cec40 2023-12-19T17:00:07.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.319+0000 7f33bf066640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80cf0d0 con 0x7f33b806b3d0 2023-12-19T17:00:07.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.319+0000 7f33bce02640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f33b8154220 0x7f33b815aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42484/0 (socket says 172.21.15.26:42484) 2023-12-19T17:00:07.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.319+0000 7f33bce02640 1 -- 172.21.15.26:0/4123088897 learned_addr learned my addr 172.21.15.26:0/4123088897 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:00:07.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.319+0000 7f33b7fff640 1 --2- 172.21.15.26:0/4123088897 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b806b3d0 0x7f33b80ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.323+0000 7f33b77fe640 1 -- 172.21.15.26:0/4123088897 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340464174 0 0) 0x7f33b806bc30 con 0x7f33b8154220 2023-12-19T17:00:07.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.323+0000 7f33b77fe640 1 -- 172.21.15.26:0/4123088897 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33a8003530 con 0x7f33b8154220 2023-12-19T17:00:07.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.323+0000 7f33b77fe640 1 -- 172.21.15.26:0/4123088897 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627886089 0 0) 0x7f33b80cc6e0 con 0x7f33b80cec40 2023-12-19T17:00:07.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.323+0000 7f33b7fff640 1 -- 172.21.15.26:0/4123088897 >> v1:172.21.15.160:6789/0 conn(0x7f33b80cec40 legacy=0x7f33b815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.323+0000 7f33b7fff640 1 -- 172.21.15.26:0/4123088897 >> v1:172.21.15.26:6789/0 conn(0x7f33b8154220 legacy=0x7f33b815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.323+0000 7f33b7fff640 1 -- 172.21.15.26:0/4123088897 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b806b950 con 0x7f33b806b3d0 2023-12-19T17:00:07.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.323+0000 7f33b7fff640 1 --2- 172.21.15.26:0/4123088897 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b806b3d0 0x7f33b80ce700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f33a40097f0 tx=0x7f33a401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80bf4814192f6163 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.323+0000 7f33b77fe640 1 -- 172.21.15.26:0/4123088897 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a40aa5f0 con 0x7f33b806b3d0 2023-12-19T17:00:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.323+0000 7f33b77fe640 1 -- 172.21.15.26:0/4123088897 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a40aa750 con 0x7f33b806b3d0 2023-12-19T17:00:07.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.323+0000 7f33b77fe640 1 -- 172.21.15.26:0/4123088897 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a40aaac0 con 0x7f33b806b3d0 2023-12-19T17:00:07.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 -- 172.21.15.26:0/4123088897 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b806b3d0 msgr2=0x7f33b80ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 --2- 172.21.15.26:0/4123088897 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b806b3d0 0x7f33b80ce700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f33a40097f0 tx=0x7f33a401c490 comp rx=0 tx=0).stop 2023-12-19T17:00:07.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 -- 172.21.15.26:0/4123088897 shutdown_connections 2023-12-19T17:00:07.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 --2- 172.21.15.26:0/4123088897 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b806b3d0 0x7f33b80ce700 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 -- 172.21.15.26:0/4123088897 >> 172.21.15.26:0/4123088897 conn(0x7f33b80658e0 msgr2=0x7f33b8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:07.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 -- 172.21.15.26:0/4123088897 shutdown_connections 2023-12-19T17:00:07.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 -- 172.21.15.26:0/4123088897 wait complete. 2023-12-19T17:00:07.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 Processor -- start 2023-12-19T17:00:07.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 -- start start 2023-12-19T17:00:07.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b806b3d0 0x7f33b81531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33b80cec40 0x7f33b813e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b8154220 0x7f33b80af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b813e7a0 con 0x7f33b8154220 2023-12-19T17:00:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80afbe0 con 0x7f33b80cec40 2023-12-19T17:00:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bf066640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b806faf0 con 0x7f33b806b3d0 2023-12-19T17:00:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bd603640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b8154220 0x7f33b80af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bd603640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b8154220 0x7f33b80af6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56630/0 (socket says 172.21.15.26:56630) 2023-12-19T17:00:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bd603640 1 -- 172.21.15.26:0/2506568501 learned_addr learned my addr 172.21.15.26:0/2506568501 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33bce02640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b806b3d0 0x7f33b81531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.327+0000 7f33b7fff640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33b80cec40 0x7f33b813e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33bd603640 1 -- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b806b3d0 msgr2=0x7f33b81531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33bd603640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b806b3d0 0x7f33b81531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33bd603640 1 -- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33b80cec40 msgr2=0x7f33b813e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33bd603640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33b80cec40 0x7f33b813e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33bd603640 1 -- 172.21.15.26:0/2506568501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a4009420 con 0x7f33b8154220 2023-12-19T17:00:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33bd603640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b8154220 0x7f33b80af6a0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c890 tx=0x7f33b000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33b57fa640 1 -- 172.21.15.26:0/2506568501 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0003db0 con 0x7f33b8154220 2023-12-19T17:00:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33b57fa640 1 -- 172.21.15.26:0/2506568501 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b0005070 con 0x7f33b8154220 2023-12-19T17:00:07.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33b57fa640 1 -- 172.21.15.26:0/2506568501 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0005420 con 0x7f33b8154220 2023-12-19T17:00:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33bf066640 1 -- 172.21.15.26:0/2506568501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b8153ab0 con 0x7f33b8154220 2023-12-19T17:00:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33bf066640 1 -- 172.21.15.26:0/2506568501 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b81537d0 con 0x7f33b8154220 2023-12-19T17:00:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33b57fa640 1 -- 172.21.15.26:0/2506568501 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f33b0005580 con 0x7f33b8154220 2023-12-19T17:00:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33b57fa640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f339c065b80 0x7f339c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33b57fa640 1 -- 172.21.15.26:0/2506568501 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f33b0082eb0 con 0x7f33b8154220 2023-12-19T17:00:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.331+0000 7f33bce02640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f339c065b80 0x7f339c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 Processor -- start 2023-12-19T17:00:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 -- start start 2023-12-19T17:00:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33bce02640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f339c065b80 0x7f339c068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f33ac005890 tx=0x7f33ac009040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5070 0x55a256ee5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256ee5990 0x55a256e9ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256e9b1c0 0x55a256e9b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a256cbec90 con 0x55a256ee5070 2023-12-19T17:00:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a256cf0220 con 0x55a256e9b1c0 2023-12-19T17:00:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a256ced7f0 con 0x55a256ee5990 2023-12-19T17:00:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33bce02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5070 0x55a256ee5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33bce02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5070 0x55a256ee5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56634/0 (socket says 172.21.15.26:56634) 2023-12-19T17:00:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33bce02640 1 -- 172.21.15.26:0/483401778 learned_addr learned my addr 172.21.15.26:0/483401778 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33b7fff640 1 --2- 172.21.15.26:0/483401778 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256ee5990 0x55a256e9ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33bd603640 1 --2- 172.21.15.26:0/483401778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256e9b1c0 0x55a256e9b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33bce02640 1 -- 172.21.15.26:0/483401778 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256ee5990 msgr2=0x55a256e9ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33bce02640 1 --2- 172.21.15.26:0/483401778 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256ee5990 0x55a256e9ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33bce02640 1 -- 172.21.15.26:0/483401778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256e9b1c0 msgr2=0x55a256e9b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33bce02640 1 --2- 172.21.15.26:0/483401778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256e9b1c0 0x55a256e9b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33bce02640 1 -- 172.21.15.26:0/483401778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2569e5890 con 0x55a256ee5070 2023-12-19T17:00:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33bd603640 1 --2- 172.21.15.26:0/483401778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256e9b1c0 0x55a256e9b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:00:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33bce02640 1 --2- 172.21.15.26:0/483401778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5070 0x55a256ee5450 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f33ac001540 tx=0x7f33ac011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6f14736394cdd2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33a27fc640 1 -- 172.21.15.26:0/483401778 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac005e10 con 0x55a256ee5070 2023-12-19T17:00:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33a27fc640 1 -- 172.21.15.26:0/483401778 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac00e750 con 0x55a256ee5070 2023-12-19T17:00:07.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33a27fc640 1 -- 172.21.15.26:0/483401778 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac00eae0 con 0x55a256ee5070 2023-12-19T17:00:07.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 -- 172.21.15.26:0/483401778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5070 msgr2=0x55a256ee5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 --2- 172.21.15.26:0/483401778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5070 0x55a256ee5450 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f33ac001540 tx=0x7f33ac011000 comp rx=0 tx=0).stop 2023-12-19T17:00:07.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 -- 172.21.15.26:0/483401778 shutdown_connections 2023-12-19T17:00:07.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 --2- 172.21.15.26:0/483401778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256e9b1c0 0x55a256e9b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 --2- 172.21.15.26:0/483401778 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256ee5990 0x55a256e9ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 --2- 172.21.15.26:0/483401778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5070 0x55a256ee5450 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 -- 172.21.15.26:0/483401778 >> 172.21.15.26:0/483401778 conn(0x55a256ea1950 msgr2=0x55a256e99560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:07.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 -- 172.21.15.26:0/483401778 shutdown_connections 2023-12-19T17:00:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.335+0000 7f33c23d0000 1 -- 172.21.15.26:0/483401778 wait complete. 2023-12-19T17:00:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 1 -- start start 2023-12-19T17:00:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 1 Processor -- start 2023-12-19T17:00:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256e9b1c0 0x55a256ea0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5070 0x55a256ea7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 0x55a256ea75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a256ce90e0 con 0x55a256ee5990 2023-12-19T17:00:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a256c88310 con 0x55a256ee5070 2023-12-19T17:00:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a256ca6770 con 0x55a256e9b1c0 2023-12-19T17:00:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 10 client.0 _ll_register_callbacks cb 0x55a256e86b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T17:00:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33bd603640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 0x55a256ea75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33bd603640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 0x55a256ea75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56650/0 (socket says 172.21.15.26:56650) 2023-12-19T17:00:07.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33bd603640 1 -- 172.21.15.26:0/842828789 learned_addr learned my addr 172.21.15.26:0/842828789 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33b7fff640 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5070 0x55a256ea7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33bce02640 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256e9b1c0 0x55a256ea0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33bd603640 1 -- 172.21.15.26:0/842828789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256e9b1c0 msgr2=0x55a256ea0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33bd603640 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256e9b1c0 0x55a256ea0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33bd603640 1 -- 172.21.15.26:0/842828789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5070 msgr2=0x55a256ea7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33bd603640 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5070 0x55a256ea7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33bd603640 1 -- 172.21.15.26:0/842828789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac00f040 con 0x55a256ee5990 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33b7fff640 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5070 0x55a256ea7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33bd603640 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 0x55a256ea75c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f33b005bca0 tx=0x7f33b0053c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7032f92b843f0468 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33977fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33977fe640 1 -- 172.21.15.26:0/842828789 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0058020 con 0x55a256ee5990 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33977fe640 1 -- 172.21.15.26:0/842828789 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f33b005c1d0 con 0x55a256ee5990 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 1 -- 172.21.15.26:0/842828789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a256cb8c10 con 0x55a256ee5990 2023-12-19T17:00:07.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33977fe640 1 -- 172.21.15.26:0/842828789 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33b00536e0 con 0x55a256ee5990 2023-12-19T17:00:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 10 client.8112 fetch_fsmap learned FSMap version 57 2023-12-19T17:00:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 1 -- 172.21.15.26:0/842828789 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55a256cae260 con 0x55a256ee5990 2023-12-19T17:00:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33977fe640 1 -- 172.21.15.26:0/842828789 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a256cb8c10 con 0x55a256ee5990 2023-12-19T17:00:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33977fe640 1 -- 172.21.15.26:0/842828789 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f33b005bd40 con 0x55a256ee5990 2023-12-19T17:00:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 10 client.8112 fetch_fsmap finished waiting for FSMap version 57 2023-12-19T17:00:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 10 client.8112 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T17:00:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 20 client.8112 populate_metadata read hostname 'smithi026' 2023-12-19T17:00:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.26:6832/2837068623,v1:172.21.15.26:6834/2837068623] conn(0x55a256eec8d0 0x55a256eeccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 4 client.8112 mds_command: new command op to 7974 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T17:00:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33c23d0000 1 -- 172.21.15.26:0/842828789 --> [v2:172.21.15.26:6832/2837068623,v1:172.21.15.26:6834/2837068623] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a256ce7a30 con 0x55a256eec8d0 2023-12-19T17:00:07.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.339+0000 7f33bce02640 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.26:6832/2837068623,v1:172.21.15.26:6834/2837068623] conn(0x55a256eec8d0 0x55a256eeccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33bce02640 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.26:6832/2837068623,v1:172.21.15.26:6834/2837068623] conn(0x55a256eec8d0 0x55a256eeccb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1fadc8f3163a1ad server_cookie=55af5e9d021d0edf in_seq=0 out_seq=0 2023-12-19T17:00:07.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33977fe640 10 client.8112 ms_handle_connect on v2:172.21.15.26:6832/2837068623 2023-12-19T17:00:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33977fe640 1 -- 172.21.15.26:0/842828789 <== mds.0 v2:172.21.15.26:6832/2837068623 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a256ce7a30 con 0x55a256eec8d0 2023-12-19T17:00:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33977fe640 10 client.8112 handle_command_reply: tid=0 2023-12-19T17:00:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33c23d0000 1 client.8112 shutdown 2023-12-19T17:00:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33c23d0000 1 -- 172.21.15.26:0/842828789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 msgr2=0x55a256ea75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33c23d0000 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 0x55a256ea75c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f33b005bca0 tx=0x7f33b0053c10 comp rx=0 tx=0).stop 2023-12-19T17:00:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33c23d0000 1 -- 172.21.15.26:0/842828789 shutdown_connections 2023-12-19T17:00:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33c23d0000 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 0x55a256ea75c0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33c23d0000 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.26:6832/2837068623,v1:172.21.15.26:6834/2837068623] conn(0x55a256eec8d0 0x55a256eeccb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33c23d0000 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5070 0x55a256ea7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33c23d0000 1 --2- 172.21.15.26:0/842828789 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256e9b1c0 0x55a256ea0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33c23d0000 1 -- 172.21.15.26:0/842828789 >> 172.21.15.26:0/842828789 conn(0x55a256ea1950 msgr2=0x55a256e99c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33977fe640 0 client.8112 ms_handle_reset on v2:172.21.15.26:6832/2837068623 2023-12-19T17:00:07.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33c23d0000 1 -- 172.21.15.26:0/842828789 shutdown_connections 2023-12-19T17:00:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33c23d0000 1 -- 172.21.15.26:0/842828789 wait complete. 2023-12-19T17:00:07.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.343+0000 7f33c23d0000 20 client.8112 trim_cache size 0 max 16384 2023-12-19T17:00:07.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 Processor -- start 2023-12-19T17:00:07.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 -- start start 2023-12-19T17:00:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256e9b1c0 0x55a256f3d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5070 0x55a256f3d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 0x55a256f3db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2569a6940 con 0x55a256ee5990 2023-12-19T17:00:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a256ce7f70 con 0x55a256ee5070 2023-12-19T17:00:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a256c8e770 con 0x55a256e9b1c0 2023-12-19T17:00:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33bd603640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 0x55a256f3db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33bd603640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 0x55a256f3db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56658/0 (socket says 172.21.15.26:56658) 2023-12-19T17:00:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33bd603640 1 -- 172.21.15.26:0/287682815 learned_addr learned my addr 172.21.15.26:0/287682815 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33bce02640 1 --2- 172.21.15.26:0/287682815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256e9b1c0 0x55a256f3d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33b7fff640 1 --2- 172.21.15.26:0/287682815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5070 0x55a256f3d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33bd603640 1 -- 172.21.15.26:0/287682815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256e9b1c0 msgr2=0x55a256f3d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33bd603640 1 --2- 172.21.15.26:0/287682815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256e9b1c0 0x55a256f3d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33bd603640 1 -- 172.21.15.26:0/287682815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5070 msgr2=0x55a256f3d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33bd603640 1 --2- 172.21.15.26:0/287682815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5070 0x55a256f3d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33bd603640 1 -- 172.21.15.26:0/287682815 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a256c943b0 con 0x55a256ee5990 2023-12-19T17:00:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33bd603640 1 --2- 172.21.15.26:0/287682815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 0x55a256f3db60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f33b0054180 tx=0x7f33b0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a90099b22a08f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33a2ffd640 1 -- 172.21.15.26:0/287682815 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0077070 con 0x55a256ee5990 2023-12-19T17:00:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33a2ffd640 1 -- 172.21.15.26:0/287682815 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b0040e90 con 0x55a256ee5990 2023-12-19T17:00:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33a2ffd640 1 -- 172.21.15.26:0/287682815 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b00536e0 con 0x55a256ee5990 2023-12-19T17:00:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 -- 172.21.15.26:0/287682815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 msgr2=0x55a256f3db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 --2- 172.21.15.26:0/287682815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 0x55a256f3db60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f33b0054180 tx=0x7f33b0014000 comp rx=0 tx=0).stop 2023-12-19T17:00:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 -- 172.21.15.26:0/287682815 shutdown_connections 2023-12-19T17:00:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 --2- 172.21.15.26:0/287682815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256ee5990 0x55a256f3db60 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 --2- 172.21.15.26:0/287682815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5070 0x55a256f3d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 --2- 172.21.15.26:0/287682815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256e9b1c0 0x55a256f3d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 -- 172.21.15.26:0/287682815 >> 172.21.15.26:0/287682815 conn(0x55a256ea1950 msgr2=0x55a256ea6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 -- 172.21.15.26:0/287682815 shutdown_connections 2023-12-19T17:00:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.359+0000 7f33c23d0000 1 -- 172.21.15.26:0/287682815 wait complete. 2023-12-19T17:00:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 1 -- start start 2023-12-19T17:00:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 1 Processor -- start 2023-12-19T17:00:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256e9b1c0 0x55a256f3d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256ee5070 0x55a256f25940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5990 0x55a256f25e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a256cea670 con 0x55a256e9b1c0 2023-12-19T17:00:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a256ce5cf0 con 0x55a256ee5990 2023-12-19T17:00:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a256ca2170 con 0x55a256ee5070 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 10 client.0 _ll_register_callbacks cb 0x55a256c97640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33bce02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256e9b1c0 0x55a256f3d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33bce02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256e9b1c0 0x55a256f3d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56662/0 (socket says 172.21.15.26:56662) 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33bce02640 1 -- 172.21.15.26:0/3306211346 learned_addr learned my addr 172.21.15.26:0/3306211346 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33b7fff640 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256ee5070 0x55a256f25940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33bd603640 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5990 0x55a256f25e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33bce02640 1 -- 172.21.15.26:0/3306211346 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256ee5070 msgr2=0x55a256f25940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33bce02640 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256ee5070 0x55a256f25940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33bce02640 1 -- 172.21.15.26:0/3306211346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5990 msgr2=0x55a256f25e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33bce02640 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5990 0x55a256f25e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33bce02640 1 -- 172.21.15.26:0/3306211346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a256cae260 con 0x55a256e9b1c0 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33bd603640 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5990 0x55a256f25e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33bce02640 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256e9b1c0 0x55a256f3d6e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f33ac014000 tx=0x7f33ac0010b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81d687c24fa617a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33977fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T17:00:07.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33977fe640 1 -- 172.21.15.26:0/3306211346 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac028040 con 0x55a256e9b1c0 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33977fe640 1 -- 172.21.15.26:0/3306211346 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f33ac02ea00 con 0x55a256e9b1c0 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 1 -- 172.21.15.26:0/3306211346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a256caee00 con 0x55a256e9b1c0 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33977fe640 1 -- 172.21.15.26:0/3306211346 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33ac007860 con 0x55a256e9b1c0 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 10 client.8118 fetch_fsmap learned FSMap version 57 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 1 -- 172.21.15.26:0/3306211346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55a2569e5cc0 con 0x55a256e9b1c0 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33977fe640 1 -- 172.21.15.26:0/3306211346 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a256caee00 con 0x55a256e9b1c0 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33977fe640 1 -- 172.21.15.26:0/3306211346 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 57) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f33ac02ebc0 con 0x55a256e9b1c0 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 10 client.8118 fetch_fsmap finished waiting for FSMap version 57 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 10 client.8118 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 20 client.8118 populate_metadata read hostname 'smithi026' 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.26:6832/2837068623,v1:172.21.15.26:6834/2837068623] conn(0x55a256f41950 0x55a256f41d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 4 client.8118 mds_command: new command op to 7974 tid=0[{"prefix": "status"}] 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33c23d0000 1 -- 172.21.15.26:0/3306211346 --> [v2:172.21.15.26:6832/2837068623,v1:172.21.15.26:6834/2837068623] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a256ce2160 con 0x55a256f41950 2023-12-19T17:00:07.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.363+0000 7f33b7fff640 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.26:6832/2837068623,v1:172.21.15.26:6834/2837068623] conn(0x55a256f41950 0x55a256f41d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33b7fff640 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.26:6832/2837068623,v1:172.21.15.26:6834/2837068623] conn(0x55a256f41950 0x55a256f41d30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0fcdbba8dc8002 server_cookie=a5c727f7d507eb47 in_seq=0 out_seq=0 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33977fe640 10 client.8118 ms_handle_connect on v2:172.21.15.26:6832/2837068623 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33977fe640 1 -- 172.21.15.26:0/3306211346 <== mds.0 v2:172.21.15.26:6832/2837068623 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55a256ce2160 con 0x55a256f41950 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33977fe640 10 client.8118 handle_command_reply: tid=0 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33c23d0000 1 client.8118 shutdown 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33c23d0000 1 -- 172.21.15.26:0/3306211346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256e9b1c0 msgr2=0x55a256f3d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33c23d0000 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256e9b1c0 0x55a256f3d6e0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f33ac014000 tx=0x7f33ac0010b0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33c23d0000 1 -- 172.21.15.26:0/3306211346 shutdown_connections 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33c23d0000 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.26:6832/2837068623,v1:172.21.15.26:6834/2837068623] conn(0x55a256f41950 0x55a256f41d30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33c23d0000 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a256ee5990 0x55a256f25e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33c23d0000 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55a256ee5070 0x55a256f25940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33c23d0000 1 --2- 172.21.15.26:0/3306211346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55a256e9b1c0 0x55a256f3d6e0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33977fe640 0 client.8118 ms_handle_reset on v2:172.21.15.26:6832/2837068623 2023-12-19T17:00:07.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33c23d0000 1 -- 172.21.15.26:0/3306211346 >> 172.21.15.26:0/3306211346 conn(0x55a256ea1950 msgr2=0x55a256ee2000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33c23d0000 1 -- 172.21.15.26:0/3306211346 shutdown_connections 2023-12-19T17:00:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33c23d0000 1 -- 172.21.15.26:0/3306211346 wait complete. 2023-12-19T17:00:07.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33c23d0000 20 client.8118 trim_cache size 0 max 16384 2023-12-19T17:00:07.373 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T17:00:07.373 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T17:00:07.373 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T17:00:07.373 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7974, 2023-12-19T17:00:07.373 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T17:00:07.373 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T17:00:07.373 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T17:00:07.374 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.9409739879999997, 2023-12-19T17:00:07.374 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 57, 2023-12-19T17:00:07.374 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 123, 2023-12-19T17:00:07.374 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 123, 2023-12-19T17:00:07.374 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 10.270979304999999 2023-12-19T17:00:07.374 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T17:00:07.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33bf066640 1 -- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f339c065b80 msgr2=0x7f339c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33bf066640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f339c065b80 0x7f339c068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f33ac005890 tx=0x7f33ac009040 comp rx=0 tx=0).stop 2023-12-19T17:00:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33bf066640 1 -- 172.21.15.26:0/2506568501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b8154220 msgr2=0x7f33b80af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33bf066640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b8154220 0x7f33b80af6a0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f33b000c890 tx=0x7f33b000cd50 comp rx=0 tx=0).stop 2023-12-19T17:00:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33bf066640 1 -- 172.21.15.26:0/2506568501 shutdown_connections 2023-12-19T17:00:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33bf066640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33b8154220 0x7f33b80af6a0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33bf066640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f33b80cec40 0x7f33b813e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33bf066640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f339c065b80 0x7f339c068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33bf066640 1 --2- 172.21.15.26:0/2506568501 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f33b806b3d0 0x7f33b81531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33bf066640 1 -- 172.21.15.26:0/2506568501 >> 172.21.15.26:0/2506568501 conn(0x7f33b80658e0 msgr2=0x7f33b815c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33bf066640 1 -- 172.21.15.26:0/2506568501 shutdown_connections 2023-12-19T17:00:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.367+0000 7f33bf066640 1 -- 172.21.15.26:0/2506568501 wait complete. 2023-12-19T17:00:07.389 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T17:00:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.531+0000 7f461b466640 1 Processor -- start 2023-12-19T17:00:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f461b466640 1 -- start start 2023-12-19T17:00:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f461b466640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f46140b88e0 0x7f46140b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f461b466640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46140d2960 con 0x7f46140b4b40 2023-12-19T17:00:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f461b466640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46140b4f40 con 0x7f461415cf60 2023-12-19T17:00:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f461b466640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46140d15f0 con 0x7f46140b88e0 2023-12-19T17:00:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f4619a03640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f46140b4b40 0x7f46140b5cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42490/0 (socket says 172.21.15.26:42490) 2023-12-19T17:00:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f4619a03640 1 -- 172.21.15.26:0/569903247 learned_addr learned my addr 172.21.15.26:0/569903247 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:00:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f4618a01640 1 --2- 172.21.15.26:0/569903247 >> v2:172.21.15.160:3300/0 conn(0x7f46140b88e0 0x7f46140b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f460bfff640 1 -- 172.21.15.26:0/569903247 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146847317 0 0) 0x7f46140d2960 con 0x7f46140b4b40 2023-12-19T17:00:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f460bfff640 1 -- 172.21.15.26:0/569903247 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4600003530 con 0x7f46140b4b40 2023-12-19T17:00:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f460bfff640 1 -- 172.21.15.26:0/569903247 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564839690 0 0) 0x7f46140b4f40 con 0x7f461415cf60 2023-12-19T17:00:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f460bfff640 1 -- 172.21.15.26:0/569903247 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46140d2960 con 0x7f461415cf60 2023-12-19T17:00:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f4618a01640 1 -- 172.21.15.26:0/569903247 >> v1:172.21.15.160:6789/0 conn(0x7f461415cf60 legacy=0x7f46140d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f4618a01640 1 -- 172.21.15.26:0/569903247 >> v1:172.21.15.26:6789/0 conn(0x7f46140b4b40 legacy=0x7f46140b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f460bfff640 1 -- 172.21.15.26:0/569903247 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3962873695 0 0) 0x7f4600003530 con 0x7f46140b4b40 2023-12-19T17:00:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.535+0000 7f4618a01640 1 -- 172.21.15.26:0/569903247 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46140d18c0 con 0x7f46140b88e0 2023-12-19T17:00:07.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.539+0000 7f4618a01640 1 --2- 172.21.15.26:0/569903247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46140b88e0 0x7f46140b4600 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f46040097c0 tx=0x7f460401eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d35cb9868bc0ddac server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:07.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.539+0000 7f460bfff640 1 -- 172.21.15.26:0/569903247 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c001a00 con 0x7f46140b88e0 2023-12-19T17:00:07.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.539+0000 7f460bfff640 1 -- 172.21.15.26:0/569903247 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46040a8610 con 0x7f46140b88e0 2023-12-19T17:00:07.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.539+0000 7f460bfff640 1 -- 172.21.15.26:0/569903247 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040a89a0 con 0x7f46140b88e0 2023-12-19T17:00:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.539+0000 7f461b466640 1 -- 172.21.15.26:0/569903247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46140b88e0 msgr2=0x7f46140b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.539+0000 7f461b466640 1 --2- 172.21.15.26:0/569903247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46140b88e0 0x7f46140b4600 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f46040097c0 tx=0x7f460401eb90 comp rx=0 tx=0).stop 2023-12-19T17:00:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.539+0000 7f461b466640 1 -- 172.21.15.26:0/569903247 shutdown_connections 2023-12-19T17:00:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.539+0000 7f461b466640 1 --2- 172.21.15.26:0/569903247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46140b88e0 0x7f46140b4600 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.539+0000 7f461b466640 1 -- 172.21.15.26:0/569903247 >> 172.21.15.26:0/569903247 conn(0x7f4614065ee0 msgr2=0x7f46140cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.539+0000 7f461b466640 1 -- 172.21.15.26:0/569903247 shutdown_connections 2023-12-19T17:00:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.539+0000 7f461b466640 1 -- 172.21.15.26:0/569903247 wait complete. 2023-12-19T17:00:07.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f461b466640 1 Processor -- start 2023-12-19T17:00:07.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f461b466640 1 -- start start 2023-12-19T17:00:07.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f461b466640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46140b4b40 0x7f46140c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f461b466640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46140b88e0 0x7f46140ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f461b466640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f461415cf60 0x7f46140c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f461b466640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46140cb0b0 con 0x7f46140b4b40 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f461b466640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46140c67d0 con 0x7f461415cf60 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f461b466640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46140c6ab0 con 0x7f46140b88e0 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4619202640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46140b4b40 0x7f46140c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4619202640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46140b4b40 0x7f46140c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56676/0 (socket says 172.21.15.26:56676) 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4619202640 1 -- 172.21.15.26:0/3798861431 learned_addr learned my addr 172.21.15.26:0/3798861431 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4618a01640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46140b88e0 0x7f46140ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4619a03640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f461415cf60 0x7f46140c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4619202640 1 -- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46140b88e0 msgr2=0x7f46140ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4619202640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46140b88e0 0x7f46140ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4619202640 1 -- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f461415cf60 msgr2=0x7f46140c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4619202640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f461415cf60 0x7f46140c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4619202640 1 -- 172.21.15.26:0/3798861431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4604009420 con 0x7f46140b4b40 2023-12-19T17:00:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4619202640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46140b4b40 0x7f46140c8d10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f461000c490 tx=0x7f461000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4609ffb640 1 -- 172.21.15.26:0/3798861431 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100052d0 con 0x7f46140b4b40 2023-12-19T17:00:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4609ffb640 1 -- 172.21.15.26:0/3798861431 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4610005430 con 0x7f46140b4b40 2023-12-19T17:00:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f4609ffb640 1 -- 172.21.15.26:0/3798861431 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610012870 con 0x7f46140b4b40 2023-12-19T17:00:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f461b466640 1 -- 172.21.15.26:0/3798861431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46140c6d90 con 0x7f46140b4b40 2023-12-19T17:00:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.543+0000 7f461b466640 1 -- 172.21.15.26:0/3798861431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46140cbc40 con 0x7f46140b4b40 2023-12-19T17:00:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.547+0000 7f461b466640 1 -- 172.21.15.26:0/3798861431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4614074270 con 0x7f46140b4b40 2023-12-19T17:00:07.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.547+0000 7f4609ffb640 1 -- 172.21.15.26:0/3798861431 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f46100129d0 con 0x7f46140b4b40 2023-12-19T17:00:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.547+0000 7f4609ffb640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f45f8065bb0 0x7f45f8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:07.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.547+0000 7f4609ffb640 1 -- 172.21.15.26:0/3798861431 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f4610083060 con 0x7f46140b4b40 2023-12-19T17:00:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.547+0000 7f4618a01640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f45f8065bb0 0x7f45f8068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.547+0000 7f4618a01640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f45f8065bb0 0x7f45f8068070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f46040b2000 tx=0x7f4604008040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.551+0000 7f4609ffb640 1 -- 172.21.15.26:0/3798861431 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f46100b40a0 con 0x7f46140b4b40 2023-12-19T17:00:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.747+0000 7f461b466640 1 -- 172.21.15.26:0/3798861431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f46140b1be0 con 0x7f46140b4b40 2023-12-19T17:00:07.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.747+0000 7f4609ffb640 1 -- 172.21.15.26:0/3798861431 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7f46100506d0 con 0x7f46140b4b40 2023-12-19T17:00:07.750 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T17:00:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.751+0000 7f461b466640 1 -- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f45f8065bb0 msgr2=0x7f45f8068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.751+0000 7f461b466640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f45f8065bb0 0x7f45f8068070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f46040b2000 tx=0x7f4604008040 comp rx=0 tx=0).stop 2023-12-19T17:00:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.751+0000 7f461b466640 1 -- 172.21.15.26:0/3798861431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46140b4b40 msgr2=0x7f46140c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:07.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.751+0000 7f461b466640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46140b4b40 0x7f46140c8d10 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f461000c490 tx=0x7f461000c950 comp rx=0 tx=0).stop 2023-12-19T17:00:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.751+0000 7f461b466640 1 -- 172.21.15.26:0/3798861431 shutdown_connections 2023-12-19T17:00:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.751+0000 7f461b466640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f461415cf60 0x7f46140c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.751+0000 7f461b466640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f45f8065bb0 0x7f45f8068070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.751+0000 7f461b466640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46140b88e0 0x7f46140ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.751+0000 7f461b466640 1 --2- 172.21.15.26:0/3798861431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46140b4b40 0x7f46140c8d10 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.751+0000 7f461b466640 1 -- 172.21.15.26:0/3798861431 >> 172.21.15.26:0/3798861431 conn(0x7f4614065ee0 msgr2=0x7f46140d2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.751+0000 7f461b466640 1 -- 172.21.15.26:0/3798861431 shutdown_connections 2023-12-19T17:00:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:07.751+0000 7f461b466640 1 -- 172.21.15.26:0/3798861431 wait complete. 2023-12-19T17:00:07.772 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-19T17:00:07.772 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T17:00:07.772 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T17:00:07.772 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T17:00:07.772 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T17:00:07.772 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-19T17:00:07.772 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-19T17:00:07.772 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T17:00:07.772 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T17:00:07.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:07.773 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T17:00:07.783 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T17:00:07.783 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T17:00:07.783 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T17:00:07.783 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:00:07.783 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T17:00:07.783 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:00:07.783 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T17:00:07.783 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 41333sec preferred_lft 41333sec 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T17:00:07.784 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-19T17:00:07.785 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:00:07.786 INFO:teuthology.orchestra.run.smithi026.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T17:00:07.786 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T17:00:07.786 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-19T17:00:07.786 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:00:07.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:07.786 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T17:00:07.851 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T17:00:07.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:07.853 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T17:00:07.909 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T17:00:07.909 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T17:00:07.909 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T17:00:07.909 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T17:00:07.909 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T17:00:07.909 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T17:00:07.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:07.910 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T17:00:07.910 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T17:00:07.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T17:00:07.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T17:00:07.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T17:00:07.910 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T17:00:08.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:08.004 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T17:00:08.014 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T17:00:08.014 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T17:00:08.014 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T17:00:08.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:08.014 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T17:00:08.076 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T17:00:08.076 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T17:00:08.128 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T17:00:08.128 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T17:00:08.128 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T17:00:08.183 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T17:00:08.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:08.183 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T17:00:08.278 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T17:00:08.278 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T17:00:08.292 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T17:00:08.292 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T17:00:08.292 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T17:00:08.292 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T17:00:08.292 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T17:00:08.296 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T17:00:08.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:08.296 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T17:00:08.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:08.318 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T17:00:08.376 INFO:teuthology.orchestra.run.smithi026.stdout:client7762 2023-12-19T17:00:08.377 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:00:08.377 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:00:08.428 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-19T17:00:08.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cfde73640 1 Processor -- start 2023-12-19T17:00:08.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cfde73640 1 -- start start 2023-12-19T17:00:08.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cfde73640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1cf8155030 0x7f1cf815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cfde73640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cf80ce740 0x7f1cf80ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cfde73640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf80cf0b0 con 0x7f1cf815f370 2023-12-19T17:00:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cfde73640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf806c790 con 0x7f1cf80ce740 2023-12-19T17:00:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cfde73640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf806c970 con 0x7f1cf8155030 2023-12-19T17:00:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cf7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cf80ce740 0x7f1cf80ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cf7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cf80ce740 0x7f1cf80ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56690/0 (socket says 172.21.15.26:56690) 2023-12-19T17:00:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cf7fff640 1 -- 172.21.15.26:0/4094978349 learned_addr learned my addr 172.21.15.26:0/4094978349 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:08.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cf77fe640 1 --2- 172.21.15.26:0/4094978349 >> v2:172.21.15.160:3300/0 conn(0x7f1cf8155030 0x7f1cf815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cf7fff640 1 -- 172.21.15.26:0/4094978349 >> v2:172.21.15.160:3300/0 conn(0x7f1cf8155030 msgr2=0x7f1cf815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cf7fff640 1 --2- 172.21.15.26:0/4094978349 >> v2:172.21.15.160:3300/0 conn(0x7f1cf8155030 0x7f1cf815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cf7fff640 1 -- 172.21.15.26:0/4094978349 >> v1:172.21.15.160:6789/0 conn(0x7f1cf815f370 legacy=0x7f1cf806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cf7fff640 1 -- 172.21.15.26:0/4094978349 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf815b1f0 con 0x7f1cf80ce740 2023-12-19T17:00:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.615+0000 7f1cf77fe640 1 --2- 172.21.15.26:0/4094978349 >> v2:172.21.15.160:3300/0 conn(0x7f1cf8155030 0x7f1cf815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:00:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.619+0000 7f1cf7fff640 1 --2- 172.21.15.26:0/4094978349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf80ce740 0x7f1cf80ceb70 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80cf4a0 tx=0x7f1cf001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d99244f0bfbba server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:08.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.619+0000 7f1cf67fc640 1 -- 172.21.15.26:0/4094978349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00a5860 con 0x7f1cf80ce740 2023-12-19T17:00:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.619+0000 7f1cf67fc640 1 -- 172.21.15.26:0/4094978349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf00a59c0 con 0x7f1cf80ce740 2023-12-19T17:00:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.619+0000 7f1cf67fc640 1 -- 172.21.15.26:0/4094978349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00a5d50 con 0x7f1cf80ce740 2023-12-19T17:00:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.619+0000 7f1cfde73640 1 -- 172.21.15.26:0/4094978349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf80ce740 msgr2=0x7f1cf80ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.619+0000 7f1cfde73640 1 --2- 172.21.15.26:0/4094978349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf80ce740 0x7f1cf80ceb70 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80cf4a0 tx=0x7f1cf001c9a0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.619+0000 7f1cfde73640 1 -- 172.21.15.26:0/4094978349 shutdown_connections 2023-12-19T17:00:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.619+0000 7f1cfde73640 1 --2- 172.21.15.26:0/4094978349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf80ce740 0x7f1cf80ceb70 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.619+0000 7f1cfde73640 1 --2- 172.21.15.26:0/4094978349 >> v2:172.21.15.160:3300/0 conn(0x7f1cf8155030 0x7f1cf815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.619+0000 7f1cfde73640 1 -- 172.21.15.26:0/4094978349 >> 172.21.15.26:0/4094978349 conn(0x7f1cf80658e0 msgr2=0x7f1cf8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.619+0000 7f1cfde73640 1 -- 172.21.15.26:0/4094978349 shutdown_connections 2023-12-19T17:00:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.619+0000 7f1cfde73640 1 -- 172.21.15.26:0/4094978349 wait complete. 2023-12-19T17:00:08.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.619+0000 7f1cfde73640 1 Processor -- start 2023-12-19T17:00:08.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cfde73640 1 -- start start 2023-12-19T17:00:08.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cfde73640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf80ce740 0x7f1cf814eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:08.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cfde73640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf8155030 0x7f1cf80ac510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:08.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cfde73640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cf815f370 0x7f1cf80aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cfde73640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf8152070 con 0x7f1cf8155030 2023-12-19T17:00:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cfde73640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf8151d20 con 0x7f1cf80ce740 2023-12-19T17:00:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cfde73640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80acbd0 con 0x7f1cf815f370 2023-12-19T17:00:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cf6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf8155030 0x7f1cf80ac510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cf6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf8155030 0x7f1cf80ac510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56702/0 (socket says 172.21.15.26:56702) 2023-12-19T17:00:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cf6ffd640 1 -- 172.21.15.26:0/3100430362 learned_addr learned my addr 172.21.15.26:0/3100430362 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cf77fe640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf80ce740 0x7f1cf814eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cf7fff640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cf815f370 0x7f1cf80aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cf6ffd640 1 -- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cf815f370 msgr2=0x7f1cf80aa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cf6ffd640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cf815f370 0x7f1cf80aa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cf6ffd640 1 -- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf80ce740 msgr2=0x7f1cf814eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cf6ffd640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf80ce740 0x7f1cf814eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cf6ffd640 1 -- 172.21.15.26:0/3100430362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf0009b30 con 0x7f1cf8155030 2023-12-19T17:00:08.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cf6ffd640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf8155030 0x7f1cf80ac510 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00c3c0 tx=0x7f1cec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:08.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cdffff640 1 -- 172.21.15.26:0/3100430362 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0052d0 con 0x7f1cf8155030 2023-12-19T17:00:08.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cdffff640 1 -- 172.21.15.26:0/3100430362 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec005430 con 0x7f1cf8155030 2023-12-19T17:00:08.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cfde73640 1 -- 172.21.15.26:0/3100430362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf80aceb0 con 0x7f1cf8155030 2023-12-19T17:00:08.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cdffff640 1 -- 172.21.15.26:0/3100430362 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec003840 con 0x7f1cf8155030 2023-12-19T17:00:08.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cfde73640 1 -- 172.21.15.26:0/3100430362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf814f750 con 0x7f1cf8155030 2023-12-19T17:00:08.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.623+0000 7f1cfde73640 1 -- 172.21.15.26:0/3100430362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf81521e0 con 0x7f1cf8155030 2023-12-19T17:00:08.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.627+0000 7f1cdffff640 1 -- 172.21.15.26:0/3100430362 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1cec0039a0 con 0x7f1cf8155030 2023-12-19T17:00:08.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.627+0000 7f1cdffff640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ce4065b80 0x7f1ce4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:08.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.627+0000 7f1cdffff640 1 -- 172.21.15.26:0/3100430362 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f1cec082b20 con 0x7f1cf8155030 2023-12-19T17:00:08.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.627+0000 7f1cf77fe640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ce4065b80 0x7f1ce4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:08.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.627+0000 7f1cf77fe640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ce4065b80 0x7f1ce4068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8006e90 tx=0x7f1ce8008040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:08.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.631+0000 7f1cdffff640 1 -- 172.21.15.26:0/3100430362 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1cec050100 con 0x7f1cf8155030 2023-12-19T17:00:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.811+0000 7f1cfde73640 1 -- 172.21.15.26:0/3100430362 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cf806cc50 con 0x7f1ce4065b80 2023-12-19T17:00:08.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.811+0000 7f1cdffff640 1 -- 172.21.15.26:0/3100430362 <== mgr.6707 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f1ce80118e0 con 0x7f1ce4065b80 2023-12-19T17:00:08.814 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T17:00:08.814 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:00:08.814 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2023-12-19T17:00:08.814 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:00:08.814 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T17:00:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.815+0000 7f1cfde73640 1 -- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ce4065b80 msgr2=0x7f1ce4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.815+0000 7f1cfde73640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ce4065b80 0x7f1ce4068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8006e90 tx=0x7f1ce8008040 comp rx=0 tx=0).stop 2023-12-19T17:00:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.815+0000 7f1cfde73640 1 -- 172.21.15.26:0/3100430362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf8155030 msgr2=0x7f1cf80ac510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.815+0000 7f1cfde73640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf8155030 0x7f1cf80ac510 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00c3c0 tx=0x7f1cec00c880 comp rx=0 tx=0).stop 2023-12-19T17:00:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.815+0000 7f1cfde73640 1 -- 172.21.15.26:0/3100430362 shutdown_connections 2023-12-19T17:00:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.815+0000 7f1cfde73640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1ce4065b80 0x7f1ce4068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.815+0000 7f1cfde73640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cf815f370 0x7f1cf80aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.815+0000 7f1cfde73640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cf8155030 0x7f1cf80ac510 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.815+0000 7f1cfde73640 1 --2- 172.21.15.26:0/3100430362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf80ce740 0x7f1cf814eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.815+0000 7f1cfde73640 1 -- 172.21.15.26:0/3100430362 >> 172.21.15.26:0/3100430362 conn(0x7f1cf80658e0 msgr2=0x7f1cf80cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.815+0000 7f1cfde73640 1 -- 172.21.15.26:0/3100430362 shutdown_connections 2023-12-19T17:00:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.815+0000 7f1cfde73640 1 -- 172.21.15.26:0/3100430362 wait complete. 2023-12-19T17:00:08.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-19T17:00:08.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.979+0000 7f0b8f497640 1 Processor -- start 2023-12-19T17:00:08.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.979+0000 7f0b8f497640 1 -- start start 2023-12-19T17:00:08.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.979+0000 7f0b8f497640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b881595d0 0x7f0b880b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:08.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.979+0000 7f0b8f497640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0b880affe0 0x7f0b880b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:08.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.979+0000 7f0b8f497640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b880cdd80 con 0x7f0b881591f0 2023-12-19T17:00:08.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.979+0000 7f0b8f497640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b880ce060 con 0x7f0b880affe0 2023-12-19T17:00:08.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.979+0000 7f0b8f497640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b880b09b0 con 0x7f0b881595d0 2023-12-19T17:00:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.979+0000 7f0b8ec96640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0b880affe0 0x7f0b880b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.979+0000 7f0b8dc94640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b881595d0 0x7f0b880b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.979+0000 7f0b8ec96640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0b880affe0 0x7f0b880b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:47052/0 (socket says 172.21.15.26:47052) 2023-12-19T17:00:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.979+0000 7f0b8ec96640 1 -- 172.21.15.26:0/2745368200 learned_addr learned my addr 172.21.15.26:0/2745368200 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.979+0000 7f0b8e495640 1 --1- 172.21.15.26:0/2745368200 >> v1:172.21.15.160:6789/0 conn(0x7f0b881591f0 0x7f0b880cd2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:59880/0 (socket says 172.21.15.26:59880) 2023-12-19T17:00:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.983+0000 7f0b8dc94640 1 -- 172.21.15.26:0/2745368200 >> v2:172.21.15.160:3300/0 conn(0x7f0b880affe0 msgr2=0x7f0b880b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.983+0000 7f0b8dc94640 1 --2- 172.21.15.26:0/2745368200 >> v2:172.21.15.160:3300/0 conn(0x7f0b880affe0 0x7f0b880b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.983+0000 7f0b8dc94640 1 -- 172.21.15.26:0/2745368200 >> v1:172.21.15.160:6789/0 conn(0x7f0b881591f0 legacy=0x7f0b880cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.983+0000 7f0b8dc94640 1 -- 172.21.15.26:0/2745368200 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b880c8cc0 con 0x7f0b881595d0 2023-12-19T17:00:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.983+0000 7f0b8dc94640 1 --2- 172.21.15.26:0/2745368200 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b881595d0 0x7f0b880b1fc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800a960 tx=0x7f0b7801d790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32a7f98999403581 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.983+0000 7f0b8d493640 1 -- 172.21.15.26:0/2745368200 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b780a67c0 con 0x7f0b881595d0 2023-12-19T17:00:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.983+0000 7f0b8d493640 1 -- 172.21.15.26:0/2745368200 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b780a6920 con 0x7f0b881595d0 2023-12-19T17:00:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.983+0000 7f0b8d493640 1 -- 172.21.15.26:0/2745368200 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b780a6cb0 con 0x7f0b881595d0 2023-12-19T17:00:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 -- 172.21.15.26:0/2745368200 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b881595d0 msgr2=0x7f0b880b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:08.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 --2- 172.21.15.26:0/2745368200 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b881595d0 0x7f0b880b1fc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800a960 tx=0x7f0b7801d790 comp rx=0 tx=0).stop 2023-12-19T17:00:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 -- 172.21.15.26:0/2745368200 shutdown_connections 2023-12-19T17:00:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 --2- 172.21.15.26:0/2745368200 >> v2:172.21.15.160:3300/0 conn(0x7f0b880affe0 0x7f0b880b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 --2- 172.21.15.26:0/2745368200 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b881595d0 0x7f0b880b1fc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 -- 172.21.15.26:0/2745368200 >> 172.21.15.26:0/2745368200 conn(0x7f0b8806b0f0 msgr2=0x7f0b8806b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 -- 172.21.15.26:0/2745368200 shutdown_connections 2023-12-19T17:00:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 -- 172.21.15.26:0/2745368200 wait complete. 2023-12-19T17:00:08.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 Processor -- start 2023-12-19T17:00:08.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 -- start start 2023-12-19T17:00:08.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b880affe0 0x7f0b88147c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:08.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b881591f0 0x7f0b88154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:08.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b881595d0 0x7f0b88154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:08.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b880722c0 con 0x7f0b880affe0 2023-12-19T17:00:08.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b880716b0 con 0x7f0b881595d0 2023-12-19T17:00:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8f497640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b88071990 con 0x7f0b881591f0 2023-12-19T17:00:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8e495640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b880affe0 0x7f0b88147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8e495640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b880affe0 0x7f0b88147c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56712/0 (socket says 172.21.15.26:56712) 2023-12-19T17:00:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8e495640 1 -- 172.21.15.26:0/3906195042 learned_addr learned my addr 172.21.15.26:0/3906195042 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8dc94640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b881591f0 0x7f0b88154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8ec96640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b881595d0 0x7f0b88154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8e495640 1 -- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b881591f0 msgr2=0x7f0b88154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8e495640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b881591f0 0x7f0b88154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8e495640 1 -- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b881595d0 msgr2=0x7f0b88154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8e495640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b881595d0 0x7f0b88154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.987+0000 7f0b8e495640 1 -- 172.21.15.26:0/3906195042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7800a5c0 con 0x7f0b880affe0 2023-12-19T17:00:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.991+0000 7f0b8e495640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b880affe0 0x7f0b88147c00 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400c490 tx=0x7f0b8400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.991+0000 7f0b7effd640 1 -- 172.21.15.26:0/3906195042 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b84007480 con 0x7f0b880affe0 2023-12-19T17:00:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.991+0000 7f0b7effd640 1 -- 172.21.15.26:0/3906195042 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b840075e0 con 0x7f0b880affe0 2023-12-19T17:00:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.991+0000 7f0b8f497640 1 -- 172.21.15.26:0/3906195042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b880cdb50 con 0x7f0b880affe0 2023-12-19T17:00:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.991+0000 7f0b7effd640 1 -- 172.21.15.26:0/3906195042 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b84004a10 con 0x7f0b880affe0 2023-12-19T17:00:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.991+0000 7f0b8f497640 1 -- 172.21.15.26:0/3906195042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b88071f50 con 0x7f0b880affe0 2023-12-19T17:00:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.991+0000 7f0b8f497640 1 -- 172.21.15.26:0/3906195042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8804e850 con 0x7f0b880affe0 2023-12-19T17:00:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.991+0000 7f0b7effd640 1 -- 172.21.15.26:0/3906195042 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 57) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0b84004b70 con 0x7f0b880affe0 2023-12-19T17:00:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.991+0000 7f0b7effd640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0b6c065bb0 0x7f0b6c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.991+0000 7f0b7effd640 1 -- 172.21.15.26:0/3906195042 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f0b840826a0 con 0x7f0b880affe0 2023-12-19T17:00:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.991+0000 7f0b8dc94640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0b6c065bb0 0x7f0b6c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.995+0000 7f0b8dc94640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0b6c065bb0 0x7f0b6c068070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b780b0000 tx=0x7f0b78008040 comp rx=0 tx=0).ready entity=mgr.6707 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:08.995+0000 7f0b7effd640 1 -- 172.21.15.26:0/3906195042 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0b8404fc90 con 0x7f0b880affe0 2023-12-19T17:00:09.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.195+0000 7f0b8f497640 1 -- 172.21.15.26:0/3906195042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f0b880cbb10 con 0x7f0b880affe0 2023-12-19T17:00:09.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.599+0000 7f0b7effd640 1 -- 172.21.15.26:0/3906195042 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 58) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f0b84050060 con 0x7f0b880affe0 2023-12-19T17:00:09.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.599+0000 7f0b7effd640 1 -- 172.21.15.26:0/3906195042 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v58) v1 ==== 92+0+0 (secure 0 0 0) 0x7f0b8407c2e0 con 0x7f0b880affe0 2023-12-19T17:00:09.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.603+0000 7f0b8f497640 1 -- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0b6c065bb0 msgr2=0x7f0b6c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:09.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.603+0000 7f0b8f497640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0b6c065bb0 0x7f0b6c068070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0b780b0000 tx=0x7f0b78008040 comp rx=0 tx=0).stop 2023-12-19T17:00:09.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.603+0000 7f0b8f497640 1 -- 172.21.15.26:0/3906195042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b880affe0 msgr2=0x7f0b88147c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:09.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.603+0000 7f0b8f497640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b880affe0 0x7f0b88147c00 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400c490 tx=0x7f0b8400c950 comp rx=0 tx=0).stop 2023-12-19T17:00:09.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.603+0000 7f0b8f497640 1 -- 172.21.15.26:0/3906195042 shutdown_connections 2023-12-19T17:00:09.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.603+0000 7f0b8f497640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0b881595d0 0x7f0b88154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.603+0000 7f0b8f497640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0b6c065bb0 0x7f0b6c068070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.603+0000 7f0b8f497640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0b881591f0 0x7f0b88154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.603+0000 7f0b8f497640 1 --2- 172.21.15.26:0/3906195042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b880affe0 0x7f0b88147c00 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.603+0000 7f0b8f497640 1 -- 172.21.15.26:0/3906195042 >> 172.21.15.26:0/3906195042 conn(0x7f0b8806b0f0 msgr2=0x7f0b880ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:09.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.603+0000 7f0b8f497640 1 -- 172.21.15.26:0/3906195042 shutdown_connections 2023-12-19T17:00:09.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.603+0000 7f0b8f497640 1 -- 172.21.15.26:0/3906195042 wait complete. 2023-12-19T17:00:09.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-19T17:00:09.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.771+0000 7f4f3de55640 1 Processor -- start 2023-12-19T17:00:09.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f3de55640 1 -- start start 2023-12-19T17:00:09.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f3de55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f3815ccf0 0x7f4f380d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:09.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f3de55640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4f3815d0d0 0x7f4f380b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:09.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f3de55640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f3815e200 con 0x7f4f380b4600 2023-12-19T17:00:09.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f3de55640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f380d0aa0 con 0x7f4f3815d0d0 2023-12-19T17:00:09.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f3de55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380d1380 con 0x7f4f3815ccf0 2023-12-19T17:00:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f37fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f4f380b4600 0x7f4f38160c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:59890/0 (socket says 172.21.15.26:59890) 2023-12-19T17:00:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f37fff640 1 -- 172.21.15.26:0/98571256 learned_addr learned my addr 172.21.15.26:0/98571256 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:00:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f36ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4f3815d0d0 0x7f4f380b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f377fe640 1 --2- 172.21.15.26:0/98571256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f3815ccf0 0x7f4f380d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f367fc640 1 -- 172.21.15.26:0/98571256 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357199094 0 0) 0x7f4f3815e200 con 0x7f4f380b4600 2023-12-19T17:00:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f367fc640 1 -- 172.21.15.26:0/98571256 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f20003530 con 0x7f4f380b4600 2023-12-19T17:00:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f36ffd640 1 -- 172.21.15.26:0/98571256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f3815ccf0 msgr2=0x7f4f380d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f36ffd640 1 --2- 172.21.15.26:0/98571256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f3815ccf0 0x7f4f380d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f36ffd640 1 -- 172.21.15.26:0/98571256 >> v1:172.21.15.160:6789/0 conn(0x7f4f380b4600 legacy=0x7f4f38160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f36ffd640 1 -- 172.21.15.26:0/98571256 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3815d680 con 0x7f4f3815d0d0 2023-12-19T17:00:09.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.775+0000 7f4f377fe640 1 --2- 172.21.15.26:0/98571256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f3815ccf0 0x7f4f380d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:00:09.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.779+0000 7f4f36ffd640 1 --2- 172.21.15.26:0/98571256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f3815d0d0 0x7f4f380b8400 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0097c0 tx=0x7f4f2c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba6799f565bdeddc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:09.781 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:09.781+0000 7f2da3f97180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:00:09.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.779+0000 7f4f367fc640 1 -- 172.21.15.26:0/98571256 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c0ab650 con 0x7f4f3815d0d0 2023-12-19T17:00:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.779+0000 7f4f367fc640 1 -- 172.21.15.26:0/98571256 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2c0ab7b0 con 0x7f4f3815d0d0 2023-12-19T17:00:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.779+0000 7f4f367fc640 1 -- 172.21.15.26:0/98571256 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c0abb40 con 0x7f4f3815d0d0 2023-12-19T17:00:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.779+0000 7f4f3de55640 1 -- 172.21.15.26:0/98571256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f3815d0d0 msgr2=0x7f4f380b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.779+0000 7f4f3de55640 1 --2- 172.21.15.26:0/98571256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f3815d0d0 0x7f4f380b8400 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0097c0 tx=0x7f4f2c01c450 comp rx=0 tx=0).stop 2023-12-19T17:00:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.779+0000 7f4f3de55640 1 -- 172.21.15.26:0/98571256 shutdown_connections 2023-12-19T17:00:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.779+0000 7f4f3de55640 1 --2- 172.21.15.26:0/98571256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f3815d0d0 0x7f4f380b8400 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.779+0000 7f4f3de55640 1 --2- 172.21.15.26:0/98571256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f3815ccf0 0x7f4f380d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.779+0000 7f4f3de55640 1 -- 172.21.15.26:0/98571256 >> 172.21.15.26:0/98571256 conn(0x7f4f38065ee0 msgr2=0x7f4f380cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.779+0000 7f4f3de55640 1 -- 172.21.15.26:0/98571256 shutdown_connections 2023-12-19T17:00:09.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.779+0000 7f4f3de55640 1 -- 172.21.15.26:0/98571256 wait complete. 2023-12-19T17:00:09.783 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:09.779+0000 7f9be9857180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:00:09.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f3de55640 1 Processor -- start 2023-12-19T17:00:09.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f3de55640 1 -- start start 2023-12-19T17:00:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f3de55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f380b4600 0x7f4f380cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f3de55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3815ccf0 0x7f4f3815f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f3de55640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f3815d0d0 0x7f4f3815fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f3de55640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f38158610 con 0x7f4f3815ccf0 2023-12-19T17:00:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f3de55640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380ccc10 con 0x7f4f3815d0d0 2023-12-19T17:00:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f3de55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f38157990 con 0x7f4f380b4600 2023-12-19T17:00:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f36ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3815ccf0 0x7f4f3815f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f377fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f380b4600 0x7f4f380cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:09.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f377fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f380b4600 0x7f4f380cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:39378/0 (socket says 172.21.15.26:39378) 2023-12-19T17:00:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f377fe640 1 -- 172.21.15.26:0/2443489495 learned_addr learned my addr 172.21.15.26:0/2443489495 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f37fff640 1 --2- 172.21.15.26:0/2443489495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f3815d0d0 0x7f4f3815fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:09.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f36ffd640 1 -- 172.21.15.26:0/2443489495 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f380b4600 msgr2=0x7f4f380cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:09.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f36ffd640 1 --2- 172.21.15.26:0/2443489495 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f380b4600 0x7f4f380cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f36ffd640 1 -- 172.21.15.26:0/2443489495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f3815d0d0 msgr2=0x7f4f3815fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:09.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f36ffd640 1 --2- 172.21.15.26:0/2443489495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f3815d0d0 0x7f4f3815fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f36ffd640 1 -- 172.21.15.26:0/2443489495 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f28009420 con 0x7f4f3815ccf0 2023-12-19T17:00:09.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f36ffd640 1 --2- 172.21.15.26:0/2443489495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3815ccf0 0x7f4f3815f5f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0ac000 tx=0x7f4f2c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:09.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f1ffff640 1 -- 172.21.15.26:0/2443489495 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c0abdf0 con 0x7f4f3815ccf0 2023-12-19T17:00:09.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f1ffff640 1 -- 172.21.15.26:0/2443489495 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2c0be3f0 con 0x7f4f3815ccf0 2023-12-19T17:00:09.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f1ffff640 1 -- 172.21.15.26:0/2443489495 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c0be7a0 con 0x7f4f3815ccf0 2023-12-19T17:00:09.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f3de55640 1 -- 172.21.15.26:0/2443489495 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2c009420 con 0x7f4f3815ccf0 2023-12-19T17:00:09.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f3de55640 1 -- 172.21.15.26:0/2443489495 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f38157f50 con 0x7f4f3815ccf0 2023-12-19T17:00:09.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.783+0000 7f4f3de55640 1 -- 172.21.15.26:0/2443489495 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f38158750 con 0x7f4f3815ccf0 2023-12-19T17:00:09.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.787+0000 7f4f1ffff640 1 -- 172.21.15.26:0/2443489495 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 58) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f4f2c0b43f0 con 0x7f4f3815ccf0 2023-12-19T17:00:09.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.787+0000 7f4f1ffff640 1 --2- 172.21.15.26:0/2443489495 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f24065c00 0x7f4f240680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:09.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.787+0000 7f4f1ffff640 1 -- 172.21.15.26:0/2443489495 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f4f2c11dcc0 con 0x7f4f3815ccf0 2023-12-19T17:00:09.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.787+0000 7f4f377fe640 1 -- 172.21.15.26:0/2443489495 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f24065c00 msgr2=0x7f4f240680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:00:09.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.787+0000 7f4f377fe640 1 --2- 172.21.15.26:0/2443489495 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f24065c00 0x7f4f240680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:00:09.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.791+0000 7f4f1ffff640 1 -- 172.21.15.26:0/2443489495 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4f2c124080 con 0x7f4f3815ccf0 2023-12-19T17:00:09.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.951+0000 7f4f3de55640 1 -- 172.21.15.26:0/2443489495 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f4f38074270 con 0x7f4f3815ccf0 2023-12-19T17:00:09.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.955+0000 7f4f1ffff640 1 -- 172.21.15.26:0/2443489495 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4f2c0c3030 con 0x7f4f3815ccf0 2023-12-19T17:00:09.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.959+0000 7f4f3de55640 1 -- 172.21.15.26:0/2443489495 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f24065c00 msgr2=0x7f4f240680c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:00:09.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.959+0000 7f4f3de55640 1 --2- 172.21.15.26:0/2443489495 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f24065c00 0x7f4f240680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.959+0000 7f4f3de55640 1 -- 172.21.15.26:0/2443489495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3815ccf0 msgr2=0x7f4f3815f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:09.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.959+0000 7f4f3de55640 1 --2- 172.21.15.26:0/2443489495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3815ccf0 0x7f4f3815f5f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c0ac000 tx=0x7f4f2c0191f0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.959+0000 7f4f3de55640 1 -- 172.21.15.26:0/2443489495 shutdown_connections 2023-12-19T17:00:09.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.959+0000 7f4f3de55640 1 --2- 172.21.15.26:0/2443489495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f3815d0d0 0x7f4f3815fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.959+0000 7f4f3de55640 1 --2- 172.21.15.26:0/2443489495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f3815ccf0 0x7f4f3815f5f0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.959+0000 7f4f3de55640 1 --2- 172.21.15.26:0/2443489495 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4f24065c00 0x7f4f240680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.959+0000 7f4f3de55640 1 --2- 172.21.15.26:0/2443489495 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4f380b4600 0x7f4f380cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.959+0000 7f4f3de55640 1 -- 172.21.15.26:0/2443489495 >> 172.21.15.26:0/2443489495 conn(0x7f4f38065ee0 msgr2=0x7f4f3815e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.959+0000 7f4f3de55640 1 -- 172.21.15.26:0/2443489495 shutdown_connections 2023-12-19T17:00:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:09.959+0000 7f4f3de55640 1 -- 172.21.15.26:0/2443489495 wait complete. 2023-12-19T17:00:09.981 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-19T17:00:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f3132709640 1 Processor -- start 2023-12-19T17:00:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f3132709640 1 -- start start 2023-12-19T17:00:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f3132709640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312c06f6c0 0x7f312c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f3132709640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f312c159c00 0x7f312c070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f3132709640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312c06fb60 con 0x7f312c159820 2023-12-19T17:00:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f3132709640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312c15e470 con 0x7f312c159c00 2023-12-19T17:00:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f3132709640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c06fe40 con 0x7f312c06f6c0 2023-12-19T17:00:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f312bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f312c159820 0x7f312c0c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42518/0 (socket says 172.21.15.26:42518) 2023-12-19T17:00:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f312bfff640 1 -- 172.21.15.26:0/3357725824 learned_addr learned my addr 172.21.15.26:0/3357725824 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:00:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f312b7fe640 1 --2- 172.21.15.26:0/3357725824 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312c06f6c0 0x7f312c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f3130ca6640 1 --2- 172.21.15.26:0/3357725824 >> v2:172.21.15.160:3300/0 conn(0x7f312c159c00 0x7f312c070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f312affd640 1 -- 172.21.15.26:0/3357725824 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387552742 0 0) 0x7f312c06fb60 con 0x7f312c159820 2023-12-19T17:00:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f312affd640 1 -- 172.21.15.26:0/3357725824 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3114005740 con 0x7f312c159820 2023-12-19T17:00:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f312b7fe640 1 -- 172.21.15.26:0/3357725824 >> v2:172.21.15.160:3300/0 conn(0x7f312c159c00 msgr2=0x7f312c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f312b7fe640 1 --2- 172.21.15.26:0/3357725824 >> v2:172.21.15.160:3300/0 conn(0x7f312c159c00 0x7f312c070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f312b7fe640 1 -- 172.21.15.26:0/3357725824 >> v1:172.21.15.26:6789/0 conn(0x7f312c159820 legacy=0x7f312c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f312b7fe640 1 -- 172.21.15.26:0/3357725824 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c070120 con 0x7f312c06f6c0 2023-12-19T17:00:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.127+0000 7f3130ca6640 1 --2- 172.21.15.26:0/3357725824 >> v2:172.21.15.160:3300/0 conn(0x7f312c159c00 0x7f312c070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:00:10.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.131+0000 7f312b7fe640 1 --2- 172.21.15.26:0/3357725824 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312c06f6c0 0x7f312c15df30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f31200097c0 tx=0x7f312001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c4488043d6c0ab server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:10.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.131+0000 7f312affd640 1 -- 172.21.15.26:0/3357725824 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31200ad660 con 0x7f312c06f6c0 2023-12-19T17:00:10.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.131+0000 7f312affd640 1 -- 172.21.15.26:0/3357725824 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31200ad7c0 con 0x7f312c06f6c0 2023-12-19T17:00:10.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.131+0000 7f312affd640 1 -- 172.21.15.26:0/3357725824 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31200adb50 con 0x7f312c06f6c0 2023-12-19T17:00:10.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.131+0000 7f3132709640 1 -- 172.21.15.26:0/3357725824 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312c06f6c0 msgr2=0x7f312c15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:10.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.131+0000 7f3132709640 1 --2- 172.21.15.26:0/3357725824 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312c06f6c0 0x7f312c15df30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f31200097c0 tx=0x7f312001c450 comp rx=0 tx=0).stop 2023-12-19T17:00:10.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.131+0000 7f3132709640 1 -- 172.21.15.26:0/3357725824 shutdown_connections 2023-12-19T17:00:10.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.131+0000 7f3132709640 1 --2- 172.21.15.26:0/3357725824 >> v2:172.21.15.160:3300/0 conn(0x7f312c159c00 0x7f312c070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.131+0000 7f3132709640 1 --2- 172.21.15.26:0/3357725824 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312c06f6c0 0x7f312c15df30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.131+0000 7f3132709640 1 -- 172.21.15.26:0/3357725824 >> 172.21.15.26:0/3357725824 conn(0x7f312c0ce730 msgr2=0x7f312c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:10.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.131+0000 7f3132709640 1 -- 172.21.15.26:0/3357725824 shutdown_connections 2023-12-19T17:00:10.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.131+0000 7f3132709640 1 -- 172.21.15.26:0/3357725824 wait complete. 2023-12-19T17:00:10.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3132709640 1 Processor -- start 2023-12-19T17:00:10.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3132709640 1 -- start start 2023-12-19T17:00:10.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3132709640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f312c06f6c0 0x7f312c0b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:10.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3132709640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312c159820 0x7f312c0c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:10.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3132709640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c159c00 0x7f312c0c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:10.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3132709640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c15ab00 con 0x7f312c159c00 2023-12-19T17:00:10.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3132709640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c15a160 con 0x7f312c06f6c0 2023-12-19T17:00:10.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3132709640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c15a440 con 0x7f312c159820 2023-12-19T17:00:10.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3130ca6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c159c00 0x7f312c0c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:10.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3130ca6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c159c00 0x7f312c0c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56730/0 (socket says 172.21.15.26:56730) 2023-12-19T17:00:10.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3130ca6640 1 -- 172.21.15.26:0/2522101540 learned_addr learned my addr 172.21.15.26:0/2522101540 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:10.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f312b7fe640 1 --2- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312c159820 0x7f312c0c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:10.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f312bfff640 1 --2- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f312c06f6c0 0x7f312c0b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:10.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3130ca6640 1 -- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312c159820 msgr2=0x7f312c0c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:10.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3130ca6640 1 --2- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312c159820 0x7f312c0c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3130ca6640 1 -- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f312c06f6c0 msgr2=0x7f312c0b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:10.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3130ca6640 1 --2- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f312c06f6c0 0x7f312c0b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3130ca6640 1 -- 172.21.15.26:0/2522101540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3124009a70 con 0x7f312c159c00 2023-12-19T17:00:10.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3130ca6640 1 --2- 172.21.15.26:0/2522101540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c159c00 0x7f312c0c6e40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f312400efc0 tx=0x7f312400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:10.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3128ff9640 1 -- 172.21.15.26:0/2522101540 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3124009cd0 con 0x7f312c159c00 2023-12-19T17:00:10.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3128ff9640 1 -- 172.21.15.26:0/2522101540 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312400f040 con 0x7f312c159c00 2023-12-19T17:00:10.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3128ff9640 1 -- 172.21.15.26:0/2522101540 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31240039a0 con 0x7f312c159c00 2023-12-19T17:00:10.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3132709640 1 -- 172.21.15.26:0/2522101540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3120009420 con 0x7f312c159c00 2023-12-19T17:00:10.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.135+0000 7f3132709640 1 -- 172.21.15.26:0/2522101540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312c06b880 con 0x7f312c159c00 2023-12-19T17:00:10.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.139+0000 7f3132709640 1 -- 172.21.15.26:0/2522101540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312c15ac40 con 0x7f312c159c00 2023-12-19T17:00:10.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.139+0000 7f3128ff9640 1 -- 172.21.15.26:0/2522101540 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 58) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f3124014480 con 0x7f312c159c00 2023-12-19T17:00:10.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.139+0000 7f3128ff9640 1 --2- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3118065bd0 0x7f3118068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:10.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.139+0000 7f3128ff9640 1 -- 172.21.15.26:0/2522101540 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f3124084250 con 0x7f312c159c00 2023-12-19T17:00:10.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.139+0000 7f312bfff640 1 -- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3118065bd0 msgr2=0x7f3118068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:00:10.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.139+0000 7f312bfff640 1 --2- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3118065bd0 0x7f3118068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:00:10.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.147+0000 7f3128ff9640 1 -- 172.21.15.26:0/2522101540 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f312c15ac40 con 0x7f312c159c00 2023-12-19T17:00:10.200 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:10.197+0000 7f2da3f97180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:00:10.236 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:10.235+0000 7f9be9857180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:00:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.307+0000 7f3132709640 1 -- 172.21.15.26:0/2522101540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f312c15ac40 con 0x7f312c159c00 2023-12-19T17:00:10.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.311+0000 7f3128ff9640 1 -- 172.21.15.26:0/2522101540 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7f312c15ac40 con 0x7f312c159c00 2023-12-19T17:00:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.311+0000 7f3132709640 1 -- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3118065bd0 msgr2=0x7f3118068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:00:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.311+0000 7f3132709640 1 --2- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3118065bd0 0x7f3118068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.311+0000 7f3132709640 1 -- 172.21.15.26:0/2522101540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c159c00 msgr2=0x7f312c0c6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.311+0000 7f3132709640 1 --2- 172.21.15.26:0/2522101540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c159c00 0x7f312c0c6e40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f312400efc0 tx=0x7f312400c490 comp rx=0 tx=0).stop 2023-12-19T17:00:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.315+0000 7f3132709640 1 -- 172.21.15.26:0/2522101540 shutdown_connections 2023-12-19T17:00:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.315+0000 7f3132709640 1 --2- 172.21.15.26:0/2522101540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312c159c00 0x7f312c0c6e40 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.315+0000 7f3132709640 1 --2- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3118065bd0 0x7f3118068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.315+0000 7f3132709640 1 --2- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312c159820 0x7f312c0c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.315+0000 7f3132709640 1 --2- 172.21.15.26:0/2522101540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f312c06f6c0 0x7f312c0b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.315+0000 7f3132709640 1 -- 172.21.15.26:0/2522101540 >> 172.21.15.26:0/2522101540 conn(0x7f312c0ce730 msgr2=0x7f312c0af930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.315+0000 7f3132709640 1 -- 172.21.15.26:0/2522101540 shutdown_connections 2023-12-19T17:00:10.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.315+0000 7f3132709640 1 -- 172.21.15.26:0/2522101540 wait complete. 2023-12-19T17:00:10.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:10.336 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-19T17:00:10.351 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:10.349+0000 7f2da3f97180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:00:10.352 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1M 2023-12-19T17:00:10.399 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:10.395+0000 7f9be9857180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:00:10.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.539+0000 7f51a0073640 1 Processor -- start 2023-12-19T17:00:10.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.539+0000 7f51a0073640 1 -- start start 2023-12-19T17:00:10.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f51a0073640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f519815a9c0 0x7f51980706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f51a0073640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f519815ada0 0x7f51980b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f51a0073640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51980b46c0 con 0x7f519806ac80 2023-12-19T17:00:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f51a0073640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51980cc270 con 0x7f519815a9c0 2023-12-19T17:00:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f51a0073640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5198070d90 con 0x7f519815ada0 2023-12-19T17:00:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f519de0f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f519815a9c0 0x7f51980706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f519e610640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f519806ac80 0x7f51980b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42534/0 (socket says 172.21.15.26:42534) 2023-12-19T17:00:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f519de0f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f519815a9c0 0x7f51980706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56732/0 (socket says 172.21.15.26:56732) 2023-12-19T17:00:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f519e610640 1 -- 172.21.15.26:0/2801662771 learned_addr learned my addr 172.21.15.26:0/2801662771 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:00:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f519d60e640 1 --2- 172.21.15.26:0/2801662771 >> v2:172.21.15.160:3300/0 conn(0x7f519815ada0 0x7f51980b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f519de0f640 1 -- 172.21.15.26:0/2801662771 >> v2:172.21.15.160:3300/0 conn(0x7f519815ada0 msgr2=0x7f51980b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:10.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f519de0f640 1 --2- 172.21.15.26:0/2801662771 >> v2:172.21.15.160:3300/0 conn(0x7f519815ada0 0x7f51980b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f519de0f640 1 -- 172.21.15.26:0/2801662771 >> v1:172.21.15.26:6789/0 conn(0x7f519806ac80 legacy=0x7f51980b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f519de0f640 1 -- 172.21.15.26:0/2801662771 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5198071070 con 0x7f519815a9c0 2023-12-19T17:00:10.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.543+0000 7f519ce0d640 1 -- 172.21.15.26:0/2801662771 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581902010 0 0) 0x7f51980b46c0 con 0x7f519806ac80 2023-12-19T17:00:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.547+0000 7f519de0f640 1 --2- 172.21.15.26:0/2801662771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519815a9c0 0x7f51980706d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5198070160 tx=0x7f519401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0249838e482b91 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:10.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.547+0000 7f519ce0d640 1 -- 172.21.15.26:0/2801662771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940ac600 con 0x7f519815a9c0 2023-12-19T17:00:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.547+0000 7f519ce0d640 1 -- 172.21.15.26:0/2801662771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51940a8050 con 0x7f519815a9c0 2023-12-19T17:00:10.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.547+0000 7f519ce0d640 1 -- 172.21.15.26:0/2801662771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940ac980 con 0x7f519815a9c0 2023-12-19T17:00:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.547+0000 7f51a0073640 1 -- 172.21.15.26:0/2801662771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519815a9c0 msgr2=0x7f51980706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.547+0000 7f51a0073640 1 --2- 172.21.15.26:0/2801662771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519815a9c0 0x7f51980706d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5198070160 tx=0x7f519401c3a0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.547+0000 7f51a0073640 1 -- 172.21.15.26:0/2801662771 shutdown_connections 2023-12-19T17:00:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.547+0000 7f51a0073640 1 --2- 172.21.15.26:0/2801662771 >> v2:172.21.15.160:3300/0 conn(0x7f519815ada0 0x7f51980b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.547+0000 7f51a0073640 1 --2- 172.21.15.26:0/2801662771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519815a9c0 0x7f51980706d0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.547+0000 7f51a0073640 1 -- 172.21.15.26:0/2801662771 >> 172.21.15.26:0/2801662771 conn(0x7f51980ce730 msgr2=0x7f51980ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.547+0000 7f51a0073640 1 -- 172.21.15.26:0/2801662771 shutdown_connections 2023-12-19T17:00:10.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.547+0000 7f51a0073640 1 -- 172.21.15.26:0/2801662771 wait complete. 2023-12-19T17:00:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.547+0000 7f51a0073640 1 Processor -- start 2023-12-19T17:00:10.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f51a0073640 1 -- start start 2023-12-19T17:00:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f51a0073640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f519806ac80 0x7f5198157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:10.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f51a0073640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519815a9c0 0x7f5198143b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f51a0073640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f519815ada0 0x7f51980c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f51a0073640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51980c7290 con 0x7f519815a9c0 2023-12-19T17:00:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f51a0073640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51980c6680 con 0x7f519806ac80 2023-12-19T17:00:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f51a0073640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51980c6960 con 0x7f519815ada0 2023-12-19T17:00:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f519d60e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519815a9c0 0x7f5198143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:10.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f519d60e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519815a9c0 0x7f5198143b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56734/0 (socket says 172.21.15.26:56734) 2023-12-19T17:00:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f519d60e640 1 -- 172.21.15.26:0/2688808338 learned_addr learned my addr 172.21.15.26:0/2688808338 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f519de0f640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f519806ac80 0x7f5198157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f519e610640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f519815ada0 0x7f51980c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f519d60e640 1 -- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f519815ada0 msgr2=0x7f51980c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f519d60e640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f519815ada0 0x7f51980c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f519d60e640 1 -- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f519806ac80 msgr2=0x7f5198157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:10.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f519d60e640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f519806ac80 0x7f5198157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f519d60e640 1 -- 172.21.15.26:0/2688808338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5194009420 con 0x7f519815a9c0 2023-12-19T17:00:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f519d60e640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519815a9c0 0x7f5198143b80 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f518800e870 tx=0x7f518800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f518e7fc640 1 -- 172.21.15.26:0/2688808338 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188009510 con 0x7f519815a9c0 2023-12-19T17:00:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f518e7fc640 1 -- 172.21.15.26:0/2688808338 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518800eec0 con 0x7f519815a9c0 2023-12-19T17:00:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f51a0073640 1 -- 172.21.15.26:0/2688808338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51980c6c40 con 0x7f519815a9c0 2023-12-19T17:00:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f518e7fc640 1 -- 172.21.15.26:0/2688808338 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518800cb40 con 0x7f519815a9c0 2023-12-19T17:00:10.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.551+0000 7f51a0073640 1 -- 172.21.15.26:0/2688808338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5198157c80 con 0x7f519815a9c0 2023-12-19T17:00:10.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.555+0000 7f51a0073640 1 -- 172.21.15.26:0/2688808338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51980c7400 con 0x7f519815a9c0 2023-12-19T17:00:10.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.555+0000 7f518e7fc640 1 -- 172.21.15.26:0/2688808338 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 58) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f518800cca0 con 0x7f519815a9c0 2023-12-19T17:00:10.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.555+0000 7f518e7fc640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 0x7f517c0680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:10.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.555+0000 7f518e7fc640 1 -- 172.21.15.26:0/2688808338 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f5188083a20 con 0x7f519815a9c0 2023-12-19T17:00:10.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.555+0000 7f519de0f640 1 -- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 msgr2=0x7f517c0680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:00:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.555+0000 7f519de0f640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 0x7f517c0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:00:10.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.559+0000 7f518e7fc640 1 -- 172.21.15.26:0/2688808338 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f51880510a0 con 0x7f519815a9c0 2023-12-19T17:00:10.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.759+0000 7f519de0f640 1 -- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 msgr2=0x7f517c0680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:00:10.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.759+0000 7f519de0f640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 0x7f517c0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T17:00:10.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:10.759+0000 7f51a0073640 1 -- 172.21.15.26:0/2688808338 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f51980b08c0 con 0x7f517c065c00 2023-12-19T17:00:10.832 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:10.829+0000 7f2da3f97180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:00:10.891 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:10.889+0000 7f2da3f97180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:00:10.936 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:10.935+0000 7f9be9857180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:00:10.946 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:10.945+0000 7f2da3f97180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:00:10.998 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:10.995+0000 7f9be9857180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:00:11.032 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:11.029+0000 7f2da3f97180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:00:11.056 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:11.055+0000 7f9be9857180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:00:11.090 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:11.089+0000 7f2da3f97180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:00:11.148 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:11.147+0000 7f9be9857180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:00:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:11.163+0000 7f519de0f640 1 -- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 msgr2=0x7f517c0680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:00:11.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:11.163+0000 7f519de0f640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 0x7f517c0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-19T17:00:11.173 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:11.173+0000 7f2da3f97180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:00:11.208 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:11.207+0000 7f9be9857180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:00:11.229 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:11.229+0000 7f2da3f97180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:00:11.297 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:11.295+0000 7f9be9857180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:00:11.355 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:11.355+0000 7f9be9857180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:00:11.371 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:11.369+0000 7f2da3f97180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:00:11.430 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:11.429+0000 7f2da3f97180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:00:11.507 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:11.503+0000 7f9be9857180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:00:11.569 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:11.567+0000 7f9be9857180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:00:11.670 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:00:11.671 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:00:11.671 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:00:11.678 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:11.677+0000 7f2da3f97180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:00:11.814 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:11.813+0000 7f2da3f97180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:00:11.819 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:00:11.819 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:00:11.819 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:00:11.829 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:11.827+0000 7f9be9857180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:00:11.938 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:11.937+0000 7f2da3f97180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:00:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:11.967+0000 7f519de0f640 1 -- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 msgr2=0x7f517c0680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:00:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:11.967+0000 7f519de0f640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 0x7f517c0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-19T17:00:11.970 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:11.967+0000 7f9be9857180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:00:12.007 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:12.005+0000 7f2da3f97180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:00:12.103 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:12.099+0000 7f9be9857180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:00:12.142 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:12.141+0000 7f2da3f97180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:00:12.177 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:12.175+0000 7f9be9857180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:00:12.321 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:12.319+0000 7f9be9857180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:00:12.679 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:12.677+0000 7f2da3f97180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:00:12.763 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:12.761+0000 7f2da3f97180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:00:12.892 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:12.891+0000 7f9be9857180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:00:12.912 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:12.909+0000 7f2da3f97180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:00:12.981 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:12.979+0000 7f9be9857180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:00:12.995 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:12.993+0000 7f2da3f97180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:00:13.139 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:13.135+0000 7f9be9857180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:00:13.228 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:13.227+0000 7f9be9857180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:00:13.311 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:13.309+0000 7f2da3f97180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:00:13.371 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:13.369+0000 7f2da3f97180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:00:13.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:13.379+0000 7f518e7fc640 1 -- 172.21.15.26:0/2688808338 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 59) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f5188081c60 con 0x7f519815a9c0 2023-12-19T17:00:13.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:13.379+0000 7f518e7fc640 1 -- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 msgr2=0x7f517c0680c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:00:13.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:13.379+0000 7f518e7fc640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 0x7f517c0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:13.502 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:13.501+0000 7f2d29e00640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:00:13.502 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:13.501+0000 7f2d29e00640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:00:13.502 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:13.501+0000 7f2d29e00640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:00:13.503 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:13.501+0000 7f2d29e00640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:00:13.503 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:13.501+0000 7f2d29e00640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:00:13.503 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:13.501+0000 7f2d26dfa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:00:13.503 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:13.501+0000 7f2d26dfa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:00:13.503 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:13.501+0000 7f2d26dfa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:00:13.503 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:13.501+0000 7f2d26dfa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:00:13.503 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:00:13.501+0000 7f2d26dfa640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:00:13.583 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:13.579+0000 7f9be9857180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:00:13.648 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:00:13.647+0000 7f9be9857180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:00:14.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.387+0000 7f518e7fc640 1 -- 172.21.15.26:0/2688808338 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 60) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f5188051470 con 0x7f519815a9c0 2023-12-19T17:00:14.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.387+0000 7f518e7fc640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 0x7f517c0680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:14.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.387+0000 7f518e7fc640 1 -- 172.21.15.26:0/2688808338 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f517c06bc00 con 0x7f517c065c00 2023-12-19T17:00:14.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.391+0000 7f519de0f640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 0x7f517c0680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:14.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.391+0000 7f519de0f640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 0x7f517c0680c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f51940ae000 tx=0x7f51940a7000 comp rx=0 tx=0).ready entity=mgr.7792 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:14.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.463+0000 7f518e7fc640 1 -- 172.21.15.26:0/2688808338 <== mgr.7792 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f517c06bc00 con 0x7f517c065c00 2023-12-19T17:00:14.464 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path snap_test_dir1 2023-12-19T17:00:14.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.463+0000 7f51a0073640 1 -- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 msgr2=0x7f517c0680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:14.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.463+0000 7f51a0073640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 0x7f517c0680c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f51940ae000 tx=0x7f51940a7000 comp rx=0 tx=0).stop 2023-12-19T17:00:14.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.463+0000 7f51a0073640 1 -- 172.21.15.26:0/2688808338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519815a9c0 msgr2=0x7f5198143b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:14.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.463+0000 7f51a0073640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519815a9c0 0x7f5198143b80 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f518800e870 tx=0x7f518800ec70 comp rx=0 tx=0).stop 2023-12-19T17:00:14.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.463+0000 7f51a0073640 1 -- 172.21.15.26:0/2688808338 shutdown_connections 2023-12-19T17:00:14.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.463+0000 7f51a0073640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f517c065c00 0x7f517c0680c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:14.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.463+0000 7f51a0073640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f519815ada0 0x7f51980c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:14.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.463+0000 7f51a0073640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f519815a9c0 0x7f5198143b80 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:14.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.463+0000 7f51a0073640 1 --2- 172.21.15.26:0/2688808338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f519806ac80 0x7f5198157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:14.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.463+0000 7f51a0073640 1 -- 172.21.15.26:0/2688808338 >> 172.21.15.26:0/2688808338 conn(0x7f51980ce730 msgr2=0x7f51980af980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:14.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.463+0000 7f51a0073640 1 -- 172.21.15.26:0/2688808338 shutdown_connections 2023-12-19T17:00:14.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.463+0000 7f51a0073640 1 -- 172.21.15.26:0/2688808338 wait complete. 2023-12-19T17:00:14.487 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-19T17:00:14.488 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-19-17_01_00 in ~46s... 2023-12-19T17:00:14.488 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-12-19T17:00:14.488 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-12-19T17:00:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd47e6e5640 1 Processor -- start 2023-12-19T17:00:14.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd47e6e5640 1 -- start start 2023-12-19T17:00:14.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd47e6e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd4780ca820 0x7fd47806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:14.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd47e6e5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd47806b850 0x7fd4780ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:14.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd47e6e5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4780cc310 con 0x7fd478159090 2023-12-19T17:00:14.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd47e6e5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd47806bd00 con 0x7fd47806b850 2023-12-19T17:00:14.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd47e6e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4780c9eb0 con 0x7fd4780ca820 2023-12-19T17:00:14.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd46ffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd4780ca820 0x7fd47806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:14.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd47cc82640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd47806b850 0x7fd4780ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:14.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd477fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd478159090 0x7fd4780c97a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:59902/0 (socket says 172.21.15.26:59902) 2023-12-19T17:00:14.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd477fff640 1 -- 172.21.15.26:0/3400934955 learned_addr learned my addr 172.21.15.26:0/3400934955 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:00:14.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd46ffff640 1 -- 172.21.15.26:0/3400934955 >> v2:172.21.15.160:3300/0 conn(0x7fd47806b850 msgr2=0x7fd4780ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:00:14.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd46ffff640 1 --2- 172.21.15.26:0/3400934955 >> v2:172.21.15.160:3300/0 conn(0x7fd47806b850 0x7fd4780ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:14.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd46ffff640 1 -- 172.21.15.26:0/3400934955 >> v1:172.21.15.160:6789/0 conn(0x7fd478159090 legacy=0x7fd4780c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:14.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd4777fe640 1 -- 172.21.15.26:0/3400934955 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865758534 0 0) 0x7fd4780cc310 con 0x7fd478159090 2023-12-19T17:00:14.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.635+0000 7fd46ffff640 1 -- 172.21.15.26:0/3400934955 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4780ca110 con 0x7fd4780ca820 2023-12-19T17:00:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.639+0000 7fd46ffff640 1 --2- 172.21.15.26:0/3400934955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd4780ca820 0x7fd47806b310 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd4600097c0 tx=0x7fd46001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe677022784be890 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.639+0000 7fd4777fe640 1 -- 172.21.15.26:0/3400934955 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600ac630 con 0x7fd4780ca820 2023-12-19T17:00:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.639+0000 7fd4777fe640 1 -- 172.21.15.26:0/3400934955 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4600ac790 con 0x7fd4780ca820 2023-12-19T17:00:14.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.639+0000 7fd4777fe640 1 -- 172.21.15.26:0/3400934955 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600acb20 con 0x7fd4780ca820 2023-12-19T17:00:14.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.639+0000 7fd47e6e5640 1 -- 172.21.15.26:0/3400934955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd4780ca820 msgr2=0x7fd47806b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:14.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.639+0000 7fd47e6e5640 1 --2- 172.21.15.26:0/3400934955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd4780ca820 0x7fd47806b310 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd4600097c0 tx=0x7fd46001c450 comp rx=0 tx=0).stop 2023-12-19T17:00:14.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.639+0000 7fd47e6e5640 1 -- 172.21.15.26:0/3400934955 shutdown_connections 2023-12-19T17:00:14.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.639+0000 7fd47e6e5640 1 --2- 172.21.15.26:0/3400934955 >> v2:172.21.15.160:3300/0 conn(0x7fd47806b850 0x7fd4780ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:14.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.639+0000 7fd47e6e5640 1 --2- 172.21.15.26:0/3400934955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd4780ca820 0x7fd47806b310 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:14.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.639+0000 7fd47e6e5640 1 -- 172.21.15.26:0/3400934955 >> 172.21.15.26:0/3400934955 conn(0x7fd478066160 msgr2=0x7fd4780b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.639+0000 7fd47e6e5640 1 -- 172.21.15.26:0/3400934955 shutdown_connections 2023-12-19T17:00:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.639+0000 7fd47e6e5640 1 -- 172.21.15.26:0/3400934955 wait complete. 2023-12-19T17:00:14.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd47e6e5640 1 Processor -- start 2023-12-19T17:00:14.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd47e6e5640 1 -- start start 2023-12-19T17:00:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd47e6e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd47806b850 0x7fd4780c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd47e6e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4780ca820 0x7fd478157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd47e6e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd478159090 0x7fd4780c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd47e6e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd478158360 con 0x7fd4780ca820 2023-12-19T17:00:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd47e6e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4780c7510 con 0x7fd47806b850 2023-12-19T17:00:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd47e6e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4780c6900 con 0x7fd478159090 2023-12-19T17:00:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd46ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4780ca820 0x7fd478157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd46ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4780ca820 0x7fd478157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56748/0 (socket says 172.21.15.26:56748) 2023-12-19T17:00:14.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd46ffff640 1 -- 172.21.15.26:0/2368773256 learned_addr learned my addr 172.21.15.26:0/2368773256 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:00:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd47cc82640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd478159090 0x7fd4780c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd477fff640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd47806b850 0x7fd4780c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd46ffff640 1 -- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd478159090 msgr2=0x7fd4780c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd46ffff640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd478159090 0x7fd4780c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd46ffff640 1 -- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd47806b850 msgr2=0x7fd4780c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd46ffff640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd47806b850 0x7fd4780c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd46ffff640 1 -- 172.21.15.26:0/2368773256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd460009420 con 0x7fd4780ca820 2023-12-19T17:00:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd46ffff640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4780ca820 0x7fd478157e20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd46001a080 tx=0x7fd46001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd4757fa640 1 -- 172.21.15.26:0/2368773256 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600b8650 con 0x7fd4780ca820 2023-12-19T17:00:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd4757fa640 1 -- 172.21.15.26:0/2368773256 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4600b87b0 con 0x7fd4780ca820 2023-12-19T17:00:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd4757fa640 1 -- 172.21.15.26:0/2368773256 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600b8b60 con 0x7fd4780ca820 2023-12-19T17:00:14.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.643+0000 7fd47e6e5640 1 -- 172.21.15.26:0/2368773256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4780c6be0 con 0x7fd4780ca820 2023-12-19T17:00:14.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.647+0000 7fd47e6e5640 1 -- 172.21.15.26:0/2368773256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4780c71a0 con 0x7fd4780ca820 2023-12-19T17:00:14.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.647+0000 7fd47e6e5640 1 -- 172.21.15.26:0/2368773256 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4780c76a0 con 0x7fd4780ca820 2023-12-19T17:00:14.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.647+0000 7fd4757fa640 1 -- 172.21.15.26:0/2368773256 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 60) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fd4600b8cc0 con 0x7fd4780ca820 2023-12-19T17:00:14.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.647+0000 7fd4757fa640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd458065900 0x7fd458067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:00:14.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.647+0000 7fd4757fa640 1 -- 172.21.15.26:0/2368773256 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fd46011dc60 con 0x7fd4780ca820 2023-12-19T17:00:14.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.647+0000 7fd477fff640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd458065900 0x7fd458067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:00:14.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.651+0000 7fd477fff640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd458065900 0x7fd458067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd468000f70 tx=0x7fd468007c10 comp rx=0 tx=0).ready entity=mgr.7792 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:00:14.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.651+0000 7fd4757fa640 1 -- 172.21.15.26:0/2368773256 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd4600eb520 con 0x7fd4780ca820 2023-12-19T17:00:14.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.827+0000 7fd47e6e5640 1 -- 172.21.15.26:0/2368773256 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd478159470 con 0x7fd458065900 2023-12-19T17:00:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.835+0000 7fd4757fa640 1 -- 172.21.15.26:0/2368773256 <== mgr.7792 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7fd468007da0 con 0x7fd458065900 2023-12-19T17:00:14.839 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:00:14.839 INFO:teuthology.orchestra.run.smithi026.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-12-19T17:00:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.839+0000 7fd47e6e5640 1 -- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd458065900 msgr2=0x7fd458067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.839+0000 7fd47e6e5640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd458065900 0x7fd458067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd468000f70 tx=0x7fd468007c10 comp rx=0 tx=0).stop 2023-12-19T17:00:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.839+0000 7fd47e6e5640 1 -- 172.21.15.26:0/2368773256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4780ca820 msgr2=0x7fd478157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:00:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.839+0000 7fd47e6e5640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4780ca820 0x7fd478157e20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd46001a080 tx=0x7fd46001ac90 comp rx=0 tx=0).stop 2023-12-19T17:00:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.839+0000 7fd47e6e5640 1 -- 172.21.15.26:0/2368773256 shutdown_connections 2023-12-19T17:00:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.839+0000 7fd47e6e5640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd458065900 0x7fd458067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.839+0000 7fd47e6e5640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd478159090 0x7fd4780c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.839+0000 7fd47e6e5640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4780ca820 0x7fd478157e20 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.839+0000 7fd47e6e5640 1 --2- 172.21.15.26:0/2368773256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd47806b850 0x7fd4780c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:00:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.839+0000 7fd47e6e5640 1 -- 172.21.15.26:0/2368773256 >> 172.21.15.26:0/2368773256 conn(0x7fd478066160 msgr2=0x7fd478070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:00:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.839+0000 7fd47e6e5640 1 -- 172.21.15.26:0/2368773256 shutdown_connections 2023-12-19T17:00:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:00:14.839+0000 7fd47e6e5640 1 -- 172.21.15.26:0/2368773256 wait complete. 2023-12-19T17:00:14.861 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-12-19T17:00:14.861 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{}]} 2023-12-19T17:00:14.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:14.861 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:14.872 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:14.872 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:14.872 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:15.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:15.873 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:15.885 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:15.885 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:15.885 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:16.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:16.886 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:16.898 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:16.898 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:16.898 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:17.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:17.900 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:17.912 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:17.912 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:17.912 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:18.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:18.913 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:18.926 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:18.926 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:18.926 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:19.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:19.927 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:19.939 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:19.939 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:19.939 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:20.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:20.940 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:20.952 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:20.952 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:20.952 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:21.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:21.954 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:21.966 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:21.966 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:21.966 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:22.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:22.967 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:22.979 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:22.979 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:22.979 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:23.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:23.980 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:23.992 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:23.992 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:23.992 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:24.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:24.994 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:25.006 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:25.006 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:25.006 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:26.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:26.007 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:26.019 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:26.019 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:26.019 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:27.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:27.020 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:27.032 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:27.032 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:27.032 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:28.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:28.034 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:28.046 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:28.046 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:28.046 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:29.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:29.047 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:29.059 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:29.059 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:29.059 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:29.914 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:00:29.918 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:00:30.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:30.061 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:30.073 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:30.073 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:30.073 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:31.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:31.074 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:31.087 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:31.087 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:31.087 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:32.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:32.088 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:32.100 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:32.100 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:32.100 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:33.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:33.101 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:33.113 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:33.113 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:33.113 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:34.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:34.115 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:34.127 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:34.127 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:34.127 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:35.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:35.128 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:35.141 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:35.141 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:35.141 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:36.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:36.142 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:36.154 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:36.154 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:36.154 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:37.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:37.155 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:37.167 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:37.167 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:37.168 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:38.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:38.168 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:38.181 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:38.181 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:38.181 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:39.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:39.182 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:39.194 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:39.194 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:39.194 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:40.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:40.195 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:40.207 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:40.207 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:40.207 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:41.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:41.209 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:41.222 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:41.222 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:41.223 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:42.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:42.223 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:42.236 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:42.236 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:42.236 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:43.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:43.237 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:43.249 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:43.249 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:43.249 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:44.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:44.249 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:44.262 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:44.262 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:44.262 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:45.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:45.264 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:45.277 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:45.277 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:45.278 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:46.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:46.279 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:46.291 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:46.291 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:46.292 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:47.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:47.292 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:47.304 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:47.304 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:47.305 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:48.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:48.306 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:48.319 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:48.319 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:48.319 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:49.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:49.320 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:49.332 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:49.332 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:49.332 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:50.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:50.334 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:50.346 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:50.346 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:50.346 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:51.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:51.347 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:51.359 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:51.359 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:51.359 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:52.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:52.360 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:52.372 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:52.372 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:52.372 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:53.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:53.373 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:53.386 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:53.386 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:53.386 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:54.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:54.388 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:54.400 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:54.400 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:54.400 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:55.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:55.401 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:55.413 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:55.413 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:55.414 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:56.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:56.414 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:56.427 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:56.427 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:56.427 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:57.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:57.428 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:57.440 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:57.440 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:57.440 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:58.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:58.441 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:58.453 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:58.453 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:58.453 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:59.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:00:59.455 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:00:59.467 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:00:59.467 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:00:59.467 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:00:59.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:00:59.982 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:01:00.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:00.468 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:01:00.477 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:00.476+0000 7f2d195df640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:00.478 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:00.476+0000 7f2d195df640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:00.478 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:00.476+0000 7f2d195df640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:00.478 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:00.476+0000 7f2d195df640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:00.478 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:00.476+0000 7f2d195df640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:00.481 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T17:01:00.481 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T17:01:00.481 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:01:01.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:01.482 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:01:01.493 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_01_00_UTC 2023-12-19T17:01:01.494 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-17_01_00_UTC'} 2023-12-19T17:01:01.494 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-19-17_01_00_UTC'} 2023-12-19T17:01:01.494 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T17:01:01.494 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-19-17_01_00_UTC'] 2023-12-19T17:01:01.494 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703005214.4880524, now = 1703005261.494626, timo = 46 2023-12-19T17:01:02.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-19T17:01:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb70df3640 1 Processor -- start 2023-12-19T17:01:02.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb70df3640 1 -- start start 2023-12-19T17:01:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb70df3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbb6c0cb860 0x7fbb6c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:02.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb70df3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbb6c15d530 0x7fbb6c15d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb70df3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb6c15e240 con 0x7fbb6c1599e0 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb70df3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6c0c9520 con 0x7fbb6c0cb860 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb70df3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb6c0c9800 con 0x7fbb6c15d530 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb6a59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbb6c1599e0 0x7fbb6c0cb150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46330/0 (socket says 172.21.15.26:46330) 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb6a59c640 1 -- 172.21.15.26:0/1485552463 learned_addr learned my addr 172.21.15.26:0/1485552463 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb69d9b640 1 --2- 172.21.15.26:0/1485552463 >> v2:172.21.15.26:3300/0 conn(0x7fbb6c0cb860 0x7fbb6c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb6959a640 1 -- 172.21.15.26:0/1485552463 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038906452 0 0) 0x7fbb6c15e240 con 0x7fbb6c1599e0 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb6ad9d640 1 --2- 172.21.15.26:0/1485552463 >> v2:172.21.15.160:3300/0 conn(0x7fbb6c15d530 0x7fbb6c15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb6959a640 1 -- 172.21.15.26:0/1485552463 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb54003530 con 0x7fbb6c1599e0 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb69d9b640 1 -- 172.21.15.26:0/1485552463 >> v2:172.21.15.160:3300/0 conn(0x7fbb6c15d530 msgr2=0x7fbb6c15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb69d9b640 1 --2- 172.21.15.26:0/1485552463 >> v2:172.21.15.160:3300/0 conn(0x7fbb6c15d530 0x7fbb6c15d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb69d9b640 1 -- 172.21.15.26:0/1485552463 >> v1:172.21.15.26:6789/0 conn(0x7fbb6c1599e0 legacy=0x7fbb6c0cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb69d9b640 1 -- 172.21.15.26:0/1485552463 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c0c9ae0 con 0x7fbb6c0cb860 2023-12-19T17:01:02.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.650+0000 7fbb6ad9d640 1 --2- 172.21.15.26:0/1485552463 >> v2:172.21.15.160:3300/0 conn(0x7fbb6c15d530 0x7fbb6c15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:02.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.654+0000 7fbb69d9b640 1 --2- 172.21.15.26:0/1485552463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c0cb860 0x7fbb6c06f6d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0c9150 tx=0x7fbb6001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5d97fea3d8ed9a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:02.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.654+0000 7fbb6959a640 1 -- 172.21.15.26:0/1485552463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb600ab680 con 0x7fbb6c0cb860 2023-12-19T17:01:02.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.654+0000 7fbb6959a640 1 -- 172.21.15.26:0/1485552463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb600ab7e0 con 0x7fbb6c0cb860 2023-12-19T17:01:02.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.654+0000 7fbb6959a640 1 -- 172.21.15.26:0/1485552463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb600abb70 con 0x7fbb6c0cb860 2023-12-19T17:01:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.654+0000 7fbb70df3640 1 -- 172.21.15.26:0/1485552463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c0cb860 msgr2=0x7fbb6c06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.654+0000 7fbb70df3640 1 --2- 172.21.15.26:0/1485552463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c0cb860 0x7fbb6c06f6d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c0c9150 tx=0x7fbb6001c460 comp rx=0 tx=0).stop 2023-12-19T17:01:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.654+0000 7fbb70df3640 1 -- 172.21.15.26:0/1485552463 shutdown_connections 2023-12-19T17:01:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.654+0000 7fbb70df3640 1 --2- 172.21.15.26:0/1485552463 >> v2:172.21.15.160:3300/0 conn(0x7fbb6c15d530 0x7fbb6c15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.654+0000 7fbb70df3640 1 --2- 172.21.15.26:0/1485552463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c0cb860 0x7fbb6c06f6d0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.654+0000 7fbb70df3640 1 -- 172.21.15.26:0/1485552463 >> 172.21.15.26:0/1485552463 conn(0x7fbb6c066160 msgr2=0x7fbb6c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.654+0000 7fbb70df3640 1 -- 172.21.15.26:0/1485552463 shutdown_connections 2023-12-19T17:01:02.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.654+0000 7fbb70df3640 1 -- 172.21.15.26:0/1485552463 wait complete. 2023-12-19T17:01:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb70df3640 1 Processor -- start 2023-12-19T17:01:02.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb70df3640 1 -- start start 2023-12-19T17:01:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb70df3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c0cb860 0x7fbb6c153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb70df3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb6c1599e0 0x7fbb6c156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb70df3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb6c15d530 0x7fbb6c06c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb70df3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0c36d0 con 0x7fbb6c0cb860 2023-12-19T17:01:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb70df3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0c39b0 con 0x7fbb6c15d530 2023-12-19T17:01:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb70df3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0c3c90 con 0x7fbb6c1599e0 2023-12-19T17:01:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb6a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c0cb860 0x7fbb6c153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb6a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c0cb860 0x7fbb6c153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35260/0 (socket says 172.21.15.26:35260) 2023-12-19T17:01:02.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb6a59c640 1 -- 172.21.15.26:0/281495673 learned_addr learned my addr 172.21.15.26:0/281495673 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb6ad9d640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb6c15d530 0x7fbb6c06c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb69d9b640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb6c1599e0 0x7fbb6c156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb6a59c640 1 -- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb6c1599e0 msgr2=0x7fbb6c156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb6a59c640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb6c1599e0 0x7fbb6c156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb6a59c640 1 -- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb6c15d530 msgr2=0x7fbb6c06c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb6a59c640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb6c15d530 0x7fbb6c06c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb6a59c640 1 -- 172.21.15.26:0/281495673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb60009420 con 0x7fbb6c0cb860 2023-12-19T17:01:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb6ad9d640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb6c15d530 0x7fbb6c06c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb69d9b640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb6c1599e0 0x7fbb6c156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb6a59c640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c0cb860 0x7fbb6c153b90 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c4d0 tx=0x7fbb5c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:02.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb5affd640 1 -- 172.21.15.26:0/281495673 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c005cf0 con 0x7fbb6c0cb860 2023-12-19T17:01:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb5affd640 1 -- 172.21.15.26:0/281495673 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c005e50 con 0x7fbb6c0cb860 2023-12-19T17:01:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb5affd640 1 -- 172.21.15.26:0/281495673 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c007270 con 0x7fbb6c0cb860 2023-12-19T17:01:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb70df3640 1 -- 172.21.15.26:0/281495673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c0c3f70 con 0x7fbb6c0cb860 2023-12-19T17:01:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.658+0000 7fbb70df3640 1 -- 172.21.15.26:0/281495673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c1540d0 con 0x7fbb6c0cb860 2023-12-19T17:01:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.662+0000 7fbb70df3640 1 -- 172.21.15.26:0/281495673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6c0c4490 con 0x7fbb6c0cb860 2023-12-19T17:01:02.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.662+0000 7fbb5affd640 1 -- 172.21.15.26:0/281495673 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 63) v1 ==== 85461+0+0 (secure 0 0 0) 0x7fbb5c0073d0 con 0x7fbb6c0cb860 2023-12-19T17:01:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.662+0000 7fbb5affd640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbb4c065cf0 0x7fbb4c0681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.662+0000 7fbb5affd640 1 -- 172.21.15.26:0/281495673 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fbb5c082ff0 con 0x7fbb6c0cb860 2023-12-19T17:01:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.662+0000 7fbb69d9b640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbb4c065cf0 0x7fbb4c0681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:02.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.662+0000 7fbb69d9b640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbb4c065cf0 0x7fbb4c0681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb600af000 tx=0x7fbb60008040 comp rx=0 tx=0).ready entity=mgr.7792 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:02.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.666+0000 7fbb5affd640 1 -- 172.21.15.26:0/281495673 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbb6c0c4490 con 0x7fbb6c0cb860 2023-12-19T17:01:02.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.842+0000 7fbb70df3640 1 -- 172.21.15.26:0/281495673 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb6c0c4490 con 0x7fbb4c065cf0 2023-12-19T17:01:02.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.914+0000 7fbb5affd640 1 -- 172.21.15.26:0/281495673 <== mgr.7792 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fbb6c0c4490 con 0x7fbb4c065cf0 2023-12-19T17:01:02.918 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path snap_test_dir1 2023-12-19T17:01:02.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.914+0000 7fbb70df3640 1 -- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbb4c065cf0 msgr2=0x7fbb4c0681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:02.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.914+0000 7fbb70df3640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbb4c065cf0 0x7fbb4c0681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb600af000 tx=0x7fbb60008040 comp rx=0 tx=0).stop 2023-12-19T17:01:02.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.914+0000 7fbb70df3640 1 -- 172.21.15.26:0/281495673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c0cb860 msgr2=0x7fbb6c153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:02.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.914+0000 7fbb70df3640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c0cb860 0x7fbb6c153b90 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c4d0 tx=0x7fbb5c00c990 comp rx=0 tx=0).stop 2023-12-19T17:01:02.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.914+0000 7fbb70df3640 1 -- 172.21.15.26:0/281495673 shutdown_connections 2023-12-19T17:01:02.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.914+0000 7fbb70df3640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb6c15d530 0x7fbb6c06c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:02.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.914+0000 7fbb70df3640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbb4c065cf0 0x7fbb4c0681b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:02.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.914+0000 7fbb70df3640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbb6c1599e0 0x7fbb6c156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:02.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.914+0000 7fbb70df3640 1 --2- 172.21.15.26:0/281495673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb6c0cb860 0x7fbb6c153b90 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:02.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.914+0000 7fbb70df3640 1 -- 172.21.15.26:0/281495673 >> 172.21.15.26:0/281495673 conn(0x7fbb6c066160 msgr2=0x7fbb6c0cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:02.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.918+0000 7fbb70df3640 1 -- 172.21.15.26:0/281495673 shutdown_connections 2023-12-19T17:01:02.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:02.918+0000 7fbb70df3640 1 -- 172.21.15.26:0/281495673 wait complete. 2023-12-19T17:01:02.938 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-19T17:01:02.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:02.939 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:01:02.948 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_01_00_UTC 2023-12-19T17:01:02.949 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_01_00_UTC 2023-12-19T17:01:02.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:02.949 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_01_00_UTC) 2023-12-19T17:01:03.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:03.010 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-19T17:01:03.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-19T17:01:03.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51c5c63640 1 Processor -- start 2023-12-19T17:01:03.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51c5c63640 1 -- start start 2023-12-19T17:01:03.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51c5c63640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51c00adc60 0x7f51c00ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:03.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51c5c63640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c0071560 con 0x7f51c006d1e0 2023-12-19T17:01:03.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51c5c63640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c01615b0 con 0x7f51c0071100 2023-12-19T17:01:03.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51c5c63640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00c8d90 con 0x7f51c00adc60 2023-12-19T17:01:03.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51bffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f51c006d1e0 0x7f51c006f250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46344/0 (socket says 172.21.15.26:46344) 2023-12-19T17:01:03.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51bffff640 1 -- 172.21.15.26:0/3046725243 learned_addr learned my addr 172.21.15.26:0/3046725243 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51bf7fe640 1 --2- 172.21.15.26:0/3046725243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51c00adc60 0x7f51c00ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850682924 0 0) 0x7f51c0071560 con 0x7f51c006d1e0 2023-12-19T17:01:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51a8003530 con 0x7f51c006d1e0 2023-12-19T17:01:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736388081 0 0) 0x7f51c01615b0 con 0x7f51c0071100 2023-12-19T17:01:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51c0071560 con 0x7f51c0071100 2023-12-19T17:01:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 576230496 0 0) 0x7f51a8003530 con 0x7f51c006d1e0 2023-12-19T17:01:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51c00adc60 msgr2=0x7f51c00ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 --2- 172.21.15.26:0/3046725243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51c00adc60 0x7f51c00ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 >> v1:172.21.15.160:6789/0 conn(0x7f51c0071100 legacy=0x7f51c006cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:03.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00c9070 con 0x7f51c006d1e0 2023-12-19T17:01:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f51b8001a00 con 0x7f51c006d1e0 2023-12-19T17:01:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 >> v1:172.21.15.26:6789/0 conn(0x7f51c006d1e0 legacy=0x7f51c006f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 --2- 172.21.15.26:0/3046725243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a8003b40 0x7f51a8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 --2- 172.21.15.26:0/3046725243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51a80047d0 0x7f51a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 --2- 172.21.15.26:0/3046725243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51c00adc60 0x7f51a8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00c8d90 con 0x7f51a8003b40 2023-12-19T17:01:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a800ba40 con 0x7f51a80047d0 2023-12-19T17:01:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a800bbd0 con 0x7f51c00adc60 2023-12-19T17:01:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51bf7fe640 1 --2- 172.21.15.26:0/3046725243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a8003b40 0x7f51a8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51bffff640 1 --2- 172.21.15.26:0/3046725243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51c00adc60 0x7f51a8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51beffd640 1 --2- 172.21.15.26:0/3046725243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51a80047d0 0x7f51a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51bf7fe640 1 -- 172.21.15.26:0/3046725243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51c00adc60 msgr2=0x7f51a8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:03.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51bf7fe640 1 --2- 172.21.15.26:0/3046725243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51c00adc60 0x7f51a8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51bf7fe640 1 -- 172.21.15.26:0/3046725243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51a80047d0 msgr2=0x7f51a8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51bf7fe640 1 --2- 172.21.15.26:0/3046725243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51a80047d0 0x7f51a8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.254+0000 7f51bf7fe640 1 -- 172.21.15.26:0/3046725243 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a8003530 con 0x7f51a8003b40 2023-12-19T17:01:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.258+0000 7f51bf7fe640 1 --2- 172.21.15.26:0/3046725243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a8003b40 0x7f51a8003f40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009ae0 tx=0x7f51b001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb622e9b68433b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.258+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b00a64b0 con 0x7f51a8003b40 2023-12-19T17:01:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.258+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b00a6610 con 0x7f51a8003b40 2023-12-19T17:01:03.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.258+0000 7f51be7fc640 1 -- 172.21.15.26:0/3046725243 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b00a69c0 con 0x7f51a8003b40 2023-12-19T17:01:03.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.258+0000 7f51c5c63640 1 -- 172.21.15.26:0/3046725243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a8003b40 msgr2=0x7f51a8003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:03.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.258+0000 7f51c5c63640 1 --2- 172.21.15.26:0/3046725243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a8003b40 0x7f51a8003f40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009ae0 tx=0x7f51b001c9c0 comp rx=0 tx=0).stop 2023-12-19T17:01:03.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.258+0000 7f51bffff640 1 -- 172.21.15.26:0/3046725243 reap_dead start 2023-12-19T17:01:03.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.258+0000 7f51c5c63640 1 -- 172.21.15.26:0/3046725243 shutdown_connections 2023-12-19T17:01:03.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.258+0000 7f51c5c63640 1 -- 172.21.15.26:0/3046725243 >> 172.21.15.26:0/3046725243 conn(0x7f51c0066160 msgr2=0x7f51c00b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:03.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.258+0000 7f51c5c63640 1 -- 172.21.15.26:0/3046725243 shutdown_connections 2023-12-19T17:01:03.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.258+0000 7f51c5c63640 1 -- 172.21.15.26:0/3046725243 wait complete. 2023-12-19T17:01:03.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51c5c63640 1 Processor -- start 2023-12-19T17:01:03.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51c5c63640 1 -- start start 2023-12-19T17:01:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51c5c63640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a8003b40 0x7f51c01f9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51c5c63640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51c00c6550 0x7f51c0163030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51c5c63640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51c00b3410 0x7f51c00b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51c5c63640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c01f9a80 con 0x7f51a8003b40 2023-12-19T17:01:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51c5c63640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00b3dd0 con 0x7f51c00c6550 2023-12-19T17:01:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51c5c63640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c01645b0 con 0x7f51c00b3410 2023-12-19T17:01:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51bf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a8003b40 0x7f51c01f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51bf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a8003b40 0x7f51c01f9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35264/0 (socket says 172.21.15.26:35264) 2023-12-19T17:01:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51bf7fe640 1 -- 172.21.15.26:0/1213537342 learned_addr learned my addr 172.21.15.26:0/1213537342 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51beffd640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51c00c6550 0x7f51c0163030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51bffff640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51c00b3410 0x7f51c00b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:03.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51bf7fe640 1 -- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51c00b3410 msgr2=0x7f51c00b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51bf7fe640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51c00b3410 0x7f51c00b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51bf7fe640 1 -- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51c00c6550 msgr2=0x7f51c0163030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51bf7fe640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51c00c6550 0x7f51c0163030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:03.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51bf7fe640 1 -- 172.21.15.26:0/1213537342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c01639a0 con 0x7f51a8003b40 2023-12-19T17:01:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51bf7fe640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a8003b40 0x7f51c01f9540 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f51b001c7f0 tx=0x7f51a8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.262+0000 7f51a7fff640 1 -- 172.21.15.26:0/1213537342 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0005c60 con 0x7f51a8003b40 2023-12-19T17:01:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.266+0000 7f51a7fff640 1 -- 172.21.15.26:0/1213537342 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b0005de0 con 0x7f51a8003b40 2023-12-19T17:01:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.266+0000 7f51c5c63640 1 -- 172.21.15.26:0/1213537342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c0163c80 con 0x7f51a8003b40 2023-12-19T17:01:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.266+0000 7f51a7fff640 1 -- 172.21.15.26:0/1213537342 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b00a69c0 con 0x7f51a8003b40 2023-12-19T17:01:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.266+0000 7f51c5c63640 1 -- 172.21.15.26:0/1213537342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c0164240 con 0x7f51a8003b40 2023-12-19T17:01:03.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.266+0000 7f51c5c63640 1 -- 172.21.15.26:0/1213537342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c00c2030 con 0x7f51a8003b40 2023-12-19T17:01:03.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.266+0000 7f51a7fff640 1 -- 172.21.15.26:0/1213537342 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 63) v1 ==== 85461+0+0 (secure 0 0 0) 0x7f51b00a6b20 con 0x7f51a8003b40 2023-12-19T17:01:03.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.266+0000 7f51a7fff640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51ac065d20 0x7f51ac0681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:03.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.266+0000 7f51a7fff640 1 -- 172.21.15.26:0/1213537342 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f51b00aa070 con 0x7f51a8003b40 2023-12-19T17:01:03.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.266+0000 7f51beffd640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51ac065d20 0x7f51ac0681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:03.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.270+0000 7f51beffd640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51ac065d20 0x7f51ac0681e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f51b4007fd0 tx=0x7f51b4009040 comp rx=0 tx=0).ready entity=mgr.7792 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:03.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.270+0000 7f51a7fff640 1 -- 172.21.15.26:0/1213537342 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f51b00eb610 con 0x7f51a8003b40 2023-12-19T17:01:03.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.466+0000 7f51c5c63640 1 -- 172.21.15.26:0/1213537342 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f51c0074270 con 0x7f51a8003b40 2023-12-19T17:01:03.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.606+0000 7f51a7fff640 1 -- 172.21.15.26:0/1213537342 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 64) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f51b00eb9e0 con 0x7f51a8003b40 2023-12-19T17:01:03.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.606+0000 7f51a7fff640 1 -- 172.21.15.26:0/1213537342 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v64) v1 ==== 93+0+0 (secure 0 0 0) 0x7f51b00bcd60 con 0x7f51a8003b40 2023-12-19T17:01:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.610+0000 7f51c5c63640 1 -- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51ac065d20 msgr2=0x7f51ac0681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.610+0000 7f51c5c63640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51ac065d20 0x7f51ac0681e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f51b4007fd0 tx=0x7f51b4009040 comp rx=0 tx=0).stop 2023-12-19T17:01:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.610+0000 7f51c5c63640 1 -- 172.21.15.26:0/1213537342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a8003b40 msgr2=0x7f51c01f9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.610+0000 7f51c5c63640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a8003b40 0x7f51c01f9540 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f51b001c7f0 tx=0x7f51a8005800 comp rx=0 tx=0).stop 2023-12-19T17:01:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.610+0000 7f51c5c63640 1 -- 172.21.15.26:0/1213537342 shutdown_connections 2023-12-19T17:01:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.610+0000 7f51c5c63640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51ac065d20 0x7f51ac0681e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.610+0000 7f51c5c63640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51c00b3410 0x7f51c00b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.610+0000 7f51c5c63640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51c00c6550 0x7f51c0163030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.610+0000 7f51c5c63640 1 --2- 172.21.15.26:0/1213537342 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51a8003b40 0x7f51c01f9540 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.610+0000 7f51c5c63640 1 -- 172.21.15.26:0/1213537342 >> 172.21.15.26:0/1213537342 conn(0x7f51c0066160 msgr2=0x7f51c00cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.610+0000 7f51c5c63640 1 -- 172.21.15.26:0/1213537342 shutdown_connections 2023-12-19T17:01:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:03.610+0000 7f51c5c63640 1 -- 172.21.15.26:0/1213537342 wait complete. 2023-12-19T17:01:03.645 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T17:01:03.645 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T17:01:03.730 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T17:01:03.802 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:03.798+0000 7f2672276180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:01:03.803 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:03.800+0000 7fb4d93f6180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:01:03.823 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T17:01:03.823 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:01:03.823 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:01:03.833 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:01:03.833 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:01:03.885 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:01:04.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.066+0000 7fd979d8f640 1 Processor -- start 2023-12-19T17:01:04.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.066+0000 7fd979d8f640 1 -- start start 2023-12-19T17:01:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd979d8f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd974155030 0x7fd97415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd979d8f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9740ce740 0x7fd9740ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd979d8f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9740cf0b0 con 0x7fd97415f370 2023-12-19T17:01:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd979d8f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97406c790 con 0x7fd9740ce740 2023-12-19T17:01:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd979d8f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97406c970 con 0x7fd974155030 2023-12-19T17:01:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd973fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9740ce740 0x7fd9740ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd973fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9740ce740 0x7fd9740ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35272/0 (socket says 172.21.15.26:35272) 2023-12-19T17:01:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd973fff640 1 -- 172.21.15.26:0/2740773603 learned_addr learned my addr 172.21.15.26:0/2740773603 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd972ffd640 1 --1- 172.21.15.26:0/2740773603 >> v1:172.21.15.26:6789/0 conn(0x7fd97415f370 0x7fd97406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46352/0 (socket says 172.21.15.26:46352) 2023-12-19T17:01:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd9737fe640 1 --2- 172.21.15.26:0/2740773603 >> v2:172.21.15.160:3300/0 conn(0x7fd974155030 0x7fd97415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd973fff640 1 -- 172.21.15.26:0/2740773603 >> v2:172.21.15.160:3300/0 conn(0x7fd974155030 msgr2=0x7fd97415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd973fff640 1 --2- 172.21.15.26:0/2740773603 >> v2:172.21.15.160:3300/0 conn(0x7fd974155030 0x7fd97415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd973fff640 1 -- 172.21.15.26:0/2740773603 >> v1:172.21.15.26:6789/0 conn(0x7fd97415f370 legacy=0x7fd97406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd973fff640 1 -- 172.21.15.26:0/2740773603 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97415b1f0 con 0x7fd9740ce740 2023-12-19T17:01:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.070+0000 7fd9727fc640 1 -- 172.21.15.26:0/2740773603 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654187957 0 0) 0x7fd9740cf0b0 con 0x7fd97415f370 2023-12-19T17:01:04.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.074+0000 7fd973fff640 1 --2- 172.21.15.26:0/2740773603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9740ce740 0x7fd9740ceb70 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd9740cf4a0 tx=0x7fd96c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35beca882affb16 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:04.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.074+0000 7fd9727fc640 1 -- 172.21.15.26:0/2740773603 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0a5860 con 0x7fd9740ce740 2023-12-19T17:01:04.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.074+0000 7fd9727fc640 1 -- 172.21.15.26:0/2740773603 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c0a59c0 con 0x7fd9740ce740 2023-12-19T17:01:04.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.074+0000 7fd9727fc640 1 -- 172.21.15.26:0/2740773603 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0a5d50 con 0x7fd9740ce740 2023-12-19T17:01:04.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.074+0000 7fd979d8f640 1 -- 172.21.15.26:0/2740773603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9740ce740 msgr2=0x7fd9740ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.074+0000 7fd979d8f640 1 --2- 172.21.15.26:0/2740773603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9740ce740 0x7fd9740ceb70 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd9740cf4a0 tx=0x7fd96c01c9a0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.074+0000 7fd979d8f640 1 -- 172.21.15.26:0/2740773603 shutdown_connections 2023-12-19T17:01:04.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.074+0000 7fd979d8f640 1 --2- 172.21.15.26:0/2740773603 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9740ce740 0x7fd9740ceb70 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.074+0000 7fd979d8f640 1 --2- 172.21.15.26:0/2740773603 >> v2:172.21.15.160:3300/0 conn(0x7fd974155030 0x7fd97415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.074+0000 7fd979d8f640 1 -- 172.21.15.26:0/2740773603 >> 172.21.15.26:0/2740773603 conn(0x7fd9740658e0 msgr2=0x7fd974065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:04.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd979d8f640 1 -- 172.21.15.26:0/2740773603 shutdown_connections 2023-12-19T17:01:04.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd979d8f640 1 -- 172.21.15.26:0/2740773603 wait complete. 2023-12-19T17:01:04.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd979d8f640 1 Processor -- start 2023-12-19T17:01:04.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd979d8f640 1 -- start start 2023-12-19T17:01:04.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd979d8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9740ce740 0x7fd974150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd979d8f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd974155030 0x7fd974150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd979d8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97415f370 0x7fd9740af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd979d8f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97415a4b0 con 0x7fd97415f370 2023-12-19T17:01:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd979d8f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd974155b10 con 0x7fd974155030 2023-12-19T17:01:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd979d8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd974155df0 con 0x7fd9740ce740 2023-12-19T17:01:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd973fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97415f370 0x7fd9740af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd973fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97415f370 0x7fd9740af0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35280/0 (socket says 172.21.15.26:35280) 2023-12-19T17:01:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd973fff640 1 -- 172.21.15.26:0/3974989818 learned_addr learned my addr 172.21.15.26:0/3974989818 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd9737fe640 1 --2- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9740ce740 0x7fd974150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd972ffd640 1 --2- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd974155030 0x7fd974150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd973fff640 1 -- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9740ce740 msgr2=0x7fd974150940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd973fff640 1 --2- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9740ce740 0x7fd974150940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd973fff640 1 -- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd974155030 msgr2=0x7fd974150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd973fff640 1 --2- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd974155030 0x7fd974150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd973fff640 1 -- 172.21.15.26:0/3974989818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c009b30 con 0x7fd97415f370 2023-12-19T17:01:04.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.078+0000 7fd973fff640 1 --2- 172.21.15.26:0/3974989818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97415f370 0x7fd9740af0a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0af040 tx=0x7fd96c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:04.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.082+0000 7fd95bfff640 1 -- 172.21.15.26:0/3974989818 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0b8670 con 0x7fd97415f370 2023-12-19T17:01:04.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.082+0000 7fd95bfff640 1 -- 172.21.15.26:0/3974989818 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c0b87f0 con 0x7fd97415f370 2023-12-19T17:01:04.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.082+0000 7fd95bfff640 1 -- 172.21.15.26:0/3974989818 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c0b8ba0 con 0x7fd97415f370 2023-12-19T17:01:04.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.082+0000 7fd979d8f640 1 -- 172.21.15.26:0/3974989818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9741585d0 con 0x7fd97415f370 2023-12-19T17:01:04.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.082+0000 7fd979d8f640 1 -- 172.21.15.26:0/3974989818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9741582c0 con 0x7fd97415f370 2023-12-19T17:01:04.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.082+0000 7fd979d8f640 1 -- 172.21.15.26:0/3974989818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd974074270 con 0x7fd97415f370 2023-12-19T17:01:04.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.082+0000 7fd95bfff640 1 -- 172.21.15.26:0/3974989818 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 64) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fd96c0b8d00 con 0x7fd97415f370 2023-12-19T17:01:04.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.082+0000 7fd95bfff640 1 --2- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd960065ca0 0x7fd960068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.082+0000 7fd95bfff640 1 -- 172.21.15.26:0/3974989818 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fd96c11eed0 con 0x7fd97415f370 2023-12-19T17:01:04.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.082+0000 7fd9737fe640 1 -- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd960065ca0 msgr2=0x7fd960068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:04.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.082+0000 7fd9737fe640 1 --2- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd960065ca0 0x7fd960068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:01:04.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.086+0000 7fd95bfff640 1 -- 172.21.15.26:0/3974989818 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd96c0ec2b0 con 0x7fd97415f370 2023-12-19T17:01:04.221 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:04.220+0000 7fb4d93f6180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:01:04.246 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:04.241+0000 7f2672276180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:01:04.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.245+0000 7fd979d8f640 1 -- 172.21.15.26:0/3974989818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd974158a80 con 0x7fd97415f370 2023-12-19T17:01:04.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.249+0000 7fd95bfff640 1 -- 172.21.15.26:0/3974989818 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+17653 (secure 0 0 0) 0x7fd96c0bc090 con 0x7fd97415f370 2023-12-19T17:01:04.253 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:04.253 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":124,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:00:13.374923+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:59:59.967175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T17:00:00.420191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6833/38098":"2023-12-20T17:00:13.374887+0000","172.21.15.160:6832/38098":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/4135162200":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/378405713":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/520438069":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/2978385192":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/3723046945":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/3946479365":"2023-12-20T17:00:13.374887+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:01:04.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.249+0000 7fd979d8f640 1 -- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd960065ca0 msgr2=0x7fd960068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:01:04.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.249+0000 7fd979d8f640 1 --2- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd960065ca0 0x7fd960068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.249+0000 7fd979d8f640 1 -- 172.21.15.26:0/3974989818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97415f370 msgr2=0x7fd9740af0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.249+0000 7fd979d8f640 1 --2- 172.21.15.26:0/3974989818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97415f370 0x7fd9740af0a0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0af040 tx=0x7fd96c019740 comp rx=0 tx=0).stop 2023-12-19T17:01:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.249+0000 7fd979d8f640 1 -- 172.21.15.26:0/3974989818 shutdown_connections 2023-12-19T17:01:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.249+0000 7fd979d8f640 1 --2- 172.21.15.26:0/3974989818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd97415f370 0x7fd9740af0a0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.249+0000 7fd979d8f640 1 --2- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd974155030 0x7fd974150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.249+0000 7fd979d8f640 1 --2- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd960065ca0 0x7fd960068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.249+0000 7fd979d8f640 1 --2- 172.21.15.26:0/3974989818 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd9740ce740 0x7fd974150940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.249+0000 7fd979d8f640 1 -- 172.21.15.26:0/3974989818 >> 172.21.15.26:0/3974989818 conn(0x7fd9740658e0 msgr2=0x7fd9740706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.249+0000 7fd979d8f640 1 -- 172.21.15.26:0/3974989818 shutdown_connections 2023-12-19T17:01:04.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.249+0000 7fd979d8f640 1 -- 172.21.15.26:0/3974989818 wait complete. 2023-12-19T17:01:04.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:59:59.967175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T17:00:00.420191+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T17:01:04.276 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T17:01:04.370 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:04.368+0000 7fb4d93f6180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:01:04.406 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:04.401+0000 7f2672276180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:01:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa37a841640 1 Processor -- start 2023-12-19T17:01:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa37a841640 1 -- start start 2023-12-19T17:01:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa37a841640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa37415ada0 0x7fa3740706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa37a841640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa37406ac80 0x7fa3740b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa37a841640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa374070d90 con 0x7fa37415a9c0 2023-12-19T17:01:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa37a841640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3740cc270 con 0x7fa37415ada0 2023-12-19T17:01:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa37a841640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa374071070 con 0x7fa37406ac80 2023-12-19T17:01:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa3737fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa37415ada0 0x7fa3740706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa373fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa37415a9c0 0x7fa3740b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46354/0 (socket says 172.21.15.26:46354) 2023-12-19T17:01:04.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa373fff640 1 -- 172.21.15.26:0/39307404 learned_addr learned my addr 172.21.15.26:0/39307404 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:04.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa3737fe640 1 --2- 172.21.15.26:0/39307404 >> v2:172.21.15.26:3300/0 conn(0x7fa37415ada0 0x7fa3740706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35290/0 (socket says 172.21.15.26:35290) 2023-12-19T17:01:04.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa378dde640 1 --2- 172.21.15.26:0/39307404 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa37406ac80 0x7fa3740b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa3737fe640 1 -- 172.21.15.26:0/39307404 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa37406ac80 msgr2=0x7fa3740b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:04.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa3737fe640 1 --2- 172.21.15.26:0/39307404 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa37406ac80 0x7fa3740b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa3737fe640 1 -- 172.21.15.26:0/39307404 >> v1:172.21.15.26:6789/0 conn(0x7fa37415a9c0 legacy=0x7fa3740b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa3737fe640 1 -- 172.21.15.26:0/39307404 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa374071350 con 0x7fa37415ada0 2023-12-19T17:01:04.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.421+0000 7fa378dde640 1 --2- 172.21.15.26:0/39307404 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa37406ac80 0x7fa3740b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:04.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.425+0000 7fa3737fe640 1 --2- 172.21.15.26:0/39307404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa37415ada0 0x7fa3740706d0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa3680097c0 tx=0x7fa36801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152ed63f172c8297 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:04.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.425+0000 7fa372ffd640 1 -- 172.21.15.26:0/39307404 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3680ab600 con 0x7fa37415ada0 2023-12-19T17:01:04.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.425+0000 7fa372ffd640 1 -- 172.21.15.26:0/39307404 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3680a7050 con 0x7fa37415ada0 2023-12-19T17:01:04.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.425+0000 7fa372ffd640 1 -- 172.21.15.26:0/39307404 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3680ab980 con 0x7fa37415ada0 2023-12-19T17:01:04.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.425+0000 7fa37a841640 1 -- 172.21.15.26:0/39307404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa37415ada0 msgr2=0x7fa3740706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.425+0000 7fa37a841640 1 --2- 172.21.15.26:0/39307404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa37415ada0 0x7fa3740706d0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa3680097c0 tx=0x7fa36801c3c0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.425+0000 7fa37a841640 1 -- 172.21.15.26:0/39307404 shutdown_connections 2023-12-19T17:01:04.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.425+0000 7fa37a841640 1 --2- 172.21.15.26:0/39307404 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa37406ac80 0x7fa3740b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.425+0000 7fa37a841640 1 --2- 172.21.15.26:0/39307404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa37415ada0 0x7fa3740706d0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.425+0000 7fa37a841640 1 -- 172.21.15.26:0/39307404 >> 172.21.15.26:0/39307404 conn(0x7fa3740ce730 msgr2=0x7fa3740ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:04.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.425+0000 7fa37a841640 1 -- 172.21.15.26:0/39307404 shutdown_connections 2023-12-19T17:01:04.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.425+0000 7fa37a841640 1 -- 172.21.15.26:0/39307404 wait complete. 2023-12-19T17:01:04.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa37a841640 1 Processor -- start 2023-12-19T17:01:04.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa37a841640 1 -- start start 2023-12-19T17:01:04.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa37a841640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa37406ac80 0x7fa3740b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa37a841640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa37415a9c0 0x7fa3740b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa37a841640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa37415ada0 0x7fa374156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa37a841640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740b2200 con 0x7fa37415ada0 2023-12-19T17:01:04.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa37a841640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740b24e0 con 0x7fa37415a9c0 2023-12-19T17:01:04.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa37a841640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3740b27c0 con 0x7fa37406ac80 2023-12-19T17:01:04.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa378dde640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa37415ada0 0x7fa374156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa3737fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa37415a9c0 0x7fa3740b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa378dde640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa37415ada0 0x7fa374156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35306/0 (socket says 172.21.15.26:35306) 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa378dde640 1 -- 172.21.15.26:0/1854615074 learned_addr learned my addr 172.21.15.26:0/1854615074 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa373fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa37406ac80 0x7fa3740b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa378dde640 1 -- 172.21.15.26:0/1854615074 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa37406ac80 msgr2=0x7fa3740b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa378dde640 1 --2- 172.21.15.26:0/1854615074 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa37406ac80 0x7fa3740b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa378dde640 1 -- 172.21.15.26:0/1854615074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa37415a9c0 msgr2=0x7fa3740b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa378dde640 1 --2- 172.21.15.26:0/1854615074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa37415a9c0 0x7fa3740b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa378dde640 1 -- 172.21.15.26:0/1854615074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa368009420 con 0x7fa37415ada0 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa378dde640 1 --2- 172.21.15.26:0/1854615074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa37415ada0 0x7fa374156530 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009fa0 tx=0x7fa36c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa370ff9640 1 -- 172.21.15.26:0/1854615074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c003ba0 con 0x7fa37415ada0 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa370ff9640 1 -- 172.21.15.26:0/1854615074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36c00f040 con 0x7fa37415ada0 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa37a841640 1 -- 172.21.15.26:0/1854615074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3740b2aa0 con 0x7fa37415ada0 2023-12-19T17:01:04.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa370ff9640 1 -- 172.21.15.26:0/1854615074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36c004030 con 0x7fa37415ada0 2023-12-19T17:01:04.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.429+0000 7fa37a841640 1 -- 172.21.15.26:0/1854615074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa374156a70 con 0x7fa37415ada0 2023-12-19T17:01:04.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.433+0000 7fa37a841640 1 -- 172.21.15.26:0/1854615074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa374074270 con 0x7fa37415ada0 2023-12-19T17:01:04.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.433+0000 7fa370ff9640 1 -- 172.21.15.26:0/1854615074 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 64) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fa36c004190 con 0x7fa37415ada0 2023-12-19T17:01:04.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.433+0000 7fa370ff9640 1 --2- 172.21.15.26:0/1854615074 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa360065cd0 0x7fa360068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.433+0000 7fa370ff9640 1 -- 172.21.15.26:0/1854615074 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fa36c083dc0 con 0x7fa37415ada0 2023-12-19T17:01:04.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.433+0000 7fa373fff640 1 -- 172.21.15.26:0/1854615074 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa360065cd0 msgr2=0x7fa360068190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:04.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.433+0000 7fa373fff640 1 --2- 172.21.15.26:0/1854615074 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa360065cd0 0x7fa360068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:01:04.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.437+0000 7fa370ff9640 1 -- 172.21.15.26:0/1854615074 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa36c050f80 con 0x7fa37415ada0 2023-12-19T17:01:04.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.597+0000 7fa37a841640 1 -- 172.21.15.26:0/1854615074 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3740cd7b0 con 0x7fa37415ada0 2023-12-19T17:01:04.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.597+0000 7fa370ff9640 1 -- 172.21.15.26:0/1854615074 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa36c01a0a0 con 0x7fa37415ada0 2023-12-19T17:01:04.602 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T17:01:04.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.601+0000 7fa37a841640 1 -- 172.21.15.26:0/1854615074 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa360065cd0 msgr2=0x7fa360068190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:01:04.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.601+0000 7fa37a841640 1 --2- 172.21.15.26:0/1854615074 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa360065cd0 0x7fa360068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.601+0000 7fa37a841640 1 -- 172.21.15.26:0/1854615074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa37415ada0 msgr2=0x7fa374156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.601+0000 7fa37a841640 1 --2- 172.21.15.26:0/1854615074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa37415ada0 0x7fa374156530 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa36c009fa0 tx=0x7fa36c00d560 comp rx=0 tx=0).stop 2023-12-19T17:01:04.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.601+0000 7fa37a841640 1 -- 172.21.15.26:0/1854615074 shutdown_connections 2023-12-19T17:01:04.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.601+0000 7fa37a841640 1 --2- 172.21.15.26:0/1854615074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa37415ada0 0x7fa374156530 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.601+0000 7fa37a841640 1 --2- 172.21.15.26:0/1854615074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa37415a9c0 0x7fa3740b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.601+0000 7fa37a841640 1 --2- 172.21.15.26:0/1854615074 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa360065cd0 0x7fa360068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.601+0000 7fa37a841640 1 --2- 172.21.15.26:0/1854615074 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa37406ac80 0x7fa3740b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.601+0000 7fa37a841640 1 -- 172.21.15.26:0/1854615074 >> 172.21.15.26:0/1854615074 conn(0x7fa3740ce730 msgr2=0x7fa3740cb890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:04.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.601+0000 7fa37a841640 1 -- 172.21.15.26:0/1854615074 shutdown_connections 2023-12-19T17:01:04.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.601+0000 7fa37a841640 1 -- 172.21.15.26:0/1854615074 wait complete. 2023-12-19T17:01:04.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T17:01:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8806f09640 1 Processor -- start 2023-12-19T17:01:04.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8806f09640 1 -- start start 2023-12-19T17:01:04.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8806f09640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f880015a9c0 0x7f88000706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8806f09640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f880006ac80 0x7f880006b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8806f09640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8800070d90 con 0x7f880015ada0 2023-12-19T17:01:04.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8806f09640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88000cc270 con 0x7f880015a9c0 2023-12-19T17:01:04.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8806f09640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8800071070 con 0x7f880006ac80 2023-12-19T17:01:04.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8804ca5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f880015a9c0 0x7f88000706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8804ca5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f880015a9c0 0x7f88000706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35312/0 (socket says 172.21.15.26:35312) 2023-12-19T17:01:04.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f87fffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f880015ada0 0x7f88000b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46358/0 (socket says 172.21.15.26:46358) 2023-12-19T17:01:04.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8804ca5640 1 -- 172.21.15.26:0/2021878481 learned_addr learned my addr 172.21.15.26:0/2021878481 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f88054a6640 1 --2- 172.21.15.26:0/2021878481 >> v2:172.21.15.160:3300/0 conn(0x7f880006ac80 0x7f880006b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8804ca5640 1 -- 172.21.15.26:0/2021878481 >> v2:172.21.15.160:3300/0 conn(0x7f880006ac80 msgr2=0x7f880006b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8804ca5640 1 --2- 172.21.15.26:0/2021878481 >> v2:172.21.15.160:3300/0 conn(0x7f880006ac80 0x7f880006b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8804ca5640 1 -- 172.21.15.26:0/2021878481 >> v1:172.21.15.26:6789/0 conn(0x7f880015ada0 legacy=0x7f88000b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f8804ca5640 1 -- 172.21.15.26:0/2021878481 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8800071350 con 0x7f880015a9c0 2023-12-19T17:01:04.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.769+0000 7f87ff7fe640 1 -- 172.21.15.26:0/2021878481 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199430662 0 0) 0x7f8800070d90 con 0x7f880015ada0 2023-12-19T17:01:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.773+0000 7f8804ca5640 1 --2- 172.21.15.26:0/2021878481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880015a9c0 0x7f88000706d0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f88000cc3b0 tx=0x7f87f401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1228416d591e466e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.773+0000 7f87ff7fe640 1 -- 172.21.15.26:0/2021878481 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f40ab660 con 0x7f880015a9c0 2023-12-19T17:01:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.773+0000 7f87ff7fe640 1 -- 172.21.15.26:0/2021878481 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f40ab7c0 con 0x7f880015a9c0 2023-12-19T17:01:04.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.773+0000 7f87ff7fe640 1 -- 172.21.15.26:0/2021878481 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f40abb50 con 0x7f880015a9c0 2023-12-19T17:01:04.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.773+0000 7f8806f09640 1 -- 172.21.15.26:0/2021878481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880015a9c0 msgr2=0x7f88000706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.773+0000 7f8806f09640 1 --2- 172.21.15.26:0/2021878481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880015a9c0 0x7f88000706d0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f88000cc3b0 tx=0x7f87f401c440 comp rx=0 tx=0).stop 2023-12-19T17:01:04.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.773+0000 7f8806f09640 1 -- 172.21.15.26:0/2021878481 shutdown_connections 2023-12-19T17:01:04.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.773+0000 7f8806f09640 1 --2- 172.21.15.26:0/2021878481 >> v2:172.21.15.160:3300/0 conn(0x7f880006ac80 0x7f880006b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.773+0000 7f8806f09640 1 --2- 172.21.15.26:0/2021878481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880015a9c0 0x7f88000706d0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.773+0000 7f8806f09640 1 -- 172.21.15.26:0/2021878481 >> 172.21.15.26:0/2021878481 conn(0x7f88000ce730 msgr2=0x7f88000ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:04.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.773+0000 7f8806f09640 1 -- 172.21.15.26:0/2021878481 shutdown_connections 2023-12-19T17:01:04.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.773+0000 7f8806f09640 1 -- 172.21.15.26:0/2021878481 wait complete. 2023-12-19T17:01:04.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f8806f09640 1 Processor -- start 2023-12-19T17:01:04.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f8806f09640 1 -- start start 2023-12-19T17:01:04.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f8806f09640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f880006ac80 0x7f8800145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f8806f09640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f880015a9c0 0x7f88000c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f8806f09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880015ada0 0x7f88000c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f8806f09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88000c24e0 con 0x7f880015ada0 2023-12-19T17:01:04.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f8806f09640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88001460e0 con 0x7f880015a9c0 2023-12-19T17:01:04.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f8806f09640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88000c2190 con 0x7f880006ac80 2023-12-19T17:01:04.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f88054a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880015ada0 0x7f88000c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f88054a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880015ada0 0x7f88000c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35314/0 (socket says 172.21.15.26:35314) 2023-12-19T17:01:04.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f88054a6640 1 -- 172.21.15.26:0/1114879346 learned_addr learned my addr 172.21.15.26:0/1114879346 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:04.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f8804ca5640 1 --2- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f880006ac80 0x7f8800145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f87fffff640 1 --2- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f880015a9c0 0x7f88000c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:04.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f88054a6640 1 -- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f880006ac80 msgr2=0x7f8800145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f88054a6640 1 --2- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f880006ac80 0x7f8800145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f88054a6640 1 -- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f880015a9c0 msgr2=0x7f88000c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f88054a6640 1 --2- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f880015a9c0 0x7f88000c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f88054a6640 1 -- 172.21.15.26:0/1114879346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f40094d0 con 0x7f880015ada0 2023-12-19T17:01:04.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f88054a6640 1 --2- 172.21.15.26:0/1114879346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880015ada0 0x7f88000c6d00 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009dd0 tx=0x7f87f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:04.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f87fd7fa640 1 -- 172.21.15.26:0/1114879346 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8014480 con 0x7f880015ada0 2023-12-19T17:01:04.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f87fd7fa640 1 -- 172.21.15.26:0/1114879346 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f800f040 con 0x7f880015ada0 2023-12-19T17:01:04.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f8806f09640 1 -- 172.21.15.26:0/1114879346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88000c7970 con 0x7f880015ada0 2023-12-19T17:01:04.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f87fd7fa640 1 -- 172.21.15.26:0/1114879346 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8014840 con 0x7f880015ada0 2023-12-19T17:01:04.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.777+0000 7f8806f09640 1 -- 172.21.15.26:0/1114879346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88000c7f30 con 0x7f880015ada0 2023-12-19T17:01:04.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.781+0000 7f8806f09640 1 -- 172.21.15.26:0/1114879346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88000c2620 con 0x7f880015ada0 2023-12-19T17:01:04.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.781+0000 7f87fd7fa640 1 -- 172.21.15.26:0/1114879346 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 64) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f87f80149a0 con 0x7f880015ada0 2023-12-19T17:01:04.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.781+0000 7f87fd7fa640 1 --2- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f87e4065ca0 0x7f87e4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:04.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.781+0000 7f87fd7fa640 1 -- 172.21.15.26:0/1114879346 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f87f80847b0 con 0x7f880015ada0 2023-12-19T17:01:04.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.781+0000 7f8804ca5640 1 -- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f87e4065ca0 msgr2=0x7f87e4068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:04.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.781+0000 7f8804ca5640 1 --2- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f87e4065ca0 0x7f87e4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:01:04.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.785+0000 7f87fd7fa640 1 -- 172.21.15.26:0/1114879346 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f88000c2620 con 0x7f880015ada0 2023-12-19T17:01:04.856 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:04.856+0000 7fb4d93f6180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:01:04.915 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:04.912+0000 7fb4d93f6180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:01:04.926 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:04.921+0000 7f2672276180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:01:04.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f8806f09640 1 -- 172.21.15.26:0/1114879346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88000c2620 con 0x7f880015ada0 2023-12-19T17:01:04.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f87fd7fa640 1 -- 172.21.15.26:0/1114879346 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7f88000c2620 con 0x7f880015ada0 2023-12-19T17:01:04.949 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T17:01:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f8806f09640 1 -- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f87e4065ca0 msgr2=0x7f87e4068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:01:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f8806f09640 1 --2- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f87e4065ca0 0x7f87e4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f8806f09640 1 -- 172.21.15.26:0/1114879346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880015ada0 msgr2=0x7f88000c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f8806f09640 1 --2- 172.21.15.26:0/1114879346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880015ada0 0x7f88000c6d00 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009dd0 tx=0x7f87f800c490 comp rx=0 tx=0).stop 2023-12-19T17:01:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f8806f09640 1 -- 172.21.15.26:0/1114879346 shutdown_connections 2023-12-19T17:01:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f8806f09640 1 --2- 172.21.15.26:0/1114879346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f880015ada0 0x7f88000c6d00 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f8806f09640 1 --2- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f880015a9c0 0x7f88000c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f8806f09640 1 --2- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f87e4065ca0 0x7f87e4068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f8806f09640 1 --2- 172.21.15.26:0/1114879346 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f880006ac80 0x7f8800145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f8806f09640 1 -- 172.21.15.26:0/1114879346 >> 172.21.15.26:0/1114879346 conn(0x7f88000ce730 msgr2=0x7f880006c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f8806f09640 1 -- 172.21.15.26:0/1114879346 shutdown_connections 2023-12-19T17:01:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:04.945+0000 7f8806f09640 1 -- 172.21.15.26:0/1114879346 wait complete. 2023-12-19T17:01:04.971 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T17:01:04.972 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:04.968+0000 7fb4d93f6180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:01:04.989 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:04.985+0000 7f2672276180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:01:05.052 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:05.049+0000 7f2672276180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:01:05.060 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:05.056+0000 7fb4d93f6180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:01:05.120 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:05.116+0000 7fb4d93f6180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:01:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.117+0000 7fdce67be640 1 Processor -- start 2023-12-19T17:01:05.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.117+0000 7fdce67be640 1 -- start start 2023-12-19T17:01:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdce67be640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdce00b8600 0x7fdce00b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdce67be640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce00b5bd0 con 0x7fdce015cde0 2023-12-19T17:01:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdce67be640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce00d0ee0 con 0x7fdce00b4b30 2023-12-19T17:01:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdce67be640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce015e380 con 0x7fdce00b8600 2023-12-19T17:01:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdcdffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdce015cde0 0x7fdce015d1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46374/0 (socket says 172.21.15.26:46374) 2023-12-19T17:01:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdcdffff640 1 -- 172.21.15.26:0/1707222031 learned_addr learned my addr 172.21.15.26:0/1707222031 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdcdf7fe640 1 --2- 172.21.15.26:0/1707222031 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdce00b8600 0x7fdce00b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdcdeffd640 1 -- 172.21.15.26:0/1707222031 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441488070 0 0) 0x7fdce00b5bd0 con 0x7fdce015cde0 2023-12-19T17:01:05.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdcdeffd640 1 -- 172.21.15.26:0/1707222031 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc8003530 con 0x7fdce015cde0 2023-12-19T17:01:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdcdeffd640 1 -- 172.21.15.26:0/1707222031 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038544270 0 0) 0x7fdce00d0ee0 con 0x7fdce00b4b30 2023-12-19T17:01:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdcdeffd640 1 -- 172.21.15.26:0/1707222031 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce00b5bd0 con 0x7fdce00b4b30 2023-12-19T17:01:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdcdf7fe640 1 -- 172.21.15.26:0/1707222031 >> v1:172.21.15.160:6789/0 conn(0x7fdce00b4b30 legacy=0x7fdce00d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdcdf7fe640 1 -- 172.21.15.26:0/1707222031 >> v1:172.21.15.26:6789/0 conn(0x7fdce015cde0 legacy=0x7fdce015d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:05.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.121+0000 7fdcdf7fe640 1 -- 172.21.15.26:0/1707222031 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce015dae0 con 0x7fdce00b8600 2023-12-19T17:01:05.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.125+0000 7fdcdf7fe640 1 --2- 172.21.15.26:0/1707222031 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdce00b8600 0x7fdce00b45f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40097f0 tx=0x7fdcd401ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ab6e2f08f007123 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:05.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.125+0000 7fdcdeffd640 1 -- 172.21.15.26:0/1707222031 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40a89b0 con 0x7fdce00b8600 2023-12-19T17:01:05.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.125+0000 7fdcdeffd640 1 -- 172.21.15.26:0/1707222031 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd40a8b10 con 0x7fdce00b8600 2023-12-19T17:01:05.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.125+0000 7fdcdeffd640 1 -- 172.21.15.26:0/1707222031 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40a8ea0 con 0x7fdce00b8600 2023-12-19T17:01:05.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.125+0000 7fdce67be640 1 -- 172.21.15.26:0/1707222031 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdce00b8600 msgr2=0x7fdce00b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:05.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.125+0000 7fdce67be640 1 --2- 172.21.15.26:0/1707222031 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdce00b8600 0x7fdce00b45f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40097f0 tx=0x7fdcd401ecb0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.125+0000 7fdce67be640 1 -- 172.21.15.26:0/1707222031 shutdown_connections 2023-12-19T17:01:05.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.125+0000 7fdce67be640 1 --2- 172.21.15.26:0/1707222031 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdce00b8600 0x7fdce00b45f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.125+0000 7fdce67be640 1 -- 172.21.15.26:0/1707222031 >> 172.21.15.26:0/1707222031 conn(0x7fdce0065ee0 msgr2=0x7fdce00d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:05.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.125+0000 7fdce67be640 1 -- 172.21.15.26:0/1707222031 shutdown_connections 2023-12-19T17:01:05.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.125+0000 7fdce67be640 1 -- 172.21.15.26:0/1707222031 wait complete. 2023-12-19T17:01:05.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.125+0000 7fdce67be640 1 Processor -- start 2023-12-19T17:01:05.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.125+0000 7fdce67be640 1 -- start start 2023-12-19T17:01:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdce67be640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce00b4b30 0x7fdce015aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdce67be640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdce00b8600 0x7fdce00cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdce67be640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdce015cde0 0x7fdce00cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdce67be640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce006bf30 con 0x7fdce00b4b30 2023-12-19T17:01:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdce67be640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce006c180 con 0x7fdce015cde0 2023-12-19T17:01:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdce67be640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce006c460 con 0x7fdce00b8600 2023-12-19T17:01:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce00b4b30 0x7fdce015aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce00b4b30 0x7fdce015aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35322/0 (socket says 172.21.15.26:35322) 2023-12-19T17:01:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdffff640 1 -- 172.21.15.26:0/84886264 learned_addr learned my addr 172.21.15.26:0/84886264 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:05.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdce4d5b640 1 --2- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdce015cde0 0x7fdce00cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:05.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdf7fe640 1 --2- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdce00b8600 0x7fdce00cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:05.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdffff640 1 -- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdce00b8600 msgr2=0x7fdce00cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:05.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdffff640 1 --2- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdce00b8600 0x7fdce00cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdffff640 1 -- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdce015cde0 msgr2=0x7fdce00cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:05.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdffff640 1 --2- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdce015cde0 0x7fdce00cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdffff640 1 -- 172.21.15.26:0/84886264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd4009420 con 0x7fdce00b4b30 2023-12-19T17:01:05.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdffff640 1 --2- 172.21.15.26:0/84886264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce00b4b30 0x7fdce015aba0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000c4d0 tx=0x7fdcd000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:05.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdcff9640 1 -- 172.21.15.26:0/84886264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0005cf0 con 0x7fdce00b4b30 2023-12-19T17:01:05.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdcff9640 1 -- 172.21.15.26:0/84886264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd0005e50 con 0x7fdce00b4b30 2023-12-19T17:01:05.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdce67be640 1 -- 172.21.15.26:0/84886264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce006c740 con 0x7fdce00b4b30 2023-12-19T17:01:05.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdcff9640 1 -- 172.21.15.26:0/84886264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd00072b0 con 0x7fdce00b4b30 2023-12-19T17:01:05.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdce67be640 1 -- 172.21.15.26:0/84886264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce006cf40 con 0x7fdce00b4b30 2023-12-19T17:01:05.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdce67be640 1 -- 172.21.15.26:0/84886264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce0074270 con 0x7fdce00b4b30 2023-12-19T17:01:05.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.129+0000 7fdcdcff9640 1 -- 172.21.15.26:0/84886264 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 64) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fdcd0007410 con 0x7fdce00b4b30 2023-12-19T17:01:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.133+0000 7fdcdcff9640 1 --2- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdccc065ca0 0x7fdccc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.133+0000 7fdcdcff9640 1 -- 172.21.15.26:0/84886264 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fdcd0082fa0 con 0x7fdce00b4b30 2023-12-19T17:01:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.133+0000 7fdcdf7fe640 1 -- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdccc065ca0 msgr2=0x7fdccc068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:05.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.133+0000 7fdcdf7fe640 1 --2- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdccc065ca0 0x7fdccc068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:01:05.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.137+0000 7fdcdcff9640 1 -- 172.21.15.26:0/84886264 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdcd0050260 con 0x7fdce00b4b30 2023-12-19T17:01:05.146 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:05.141+0000 7f2672276180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:01:05.206 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:05.201+0000 7f2672276180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:01:05.206 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:05.204+0000 7fb4d93f6180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:01:05.262 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:05.260+0000 7fb4d93f6180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:01:05.295 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:05.289+0000 7f2672276180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:01:05.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdce67be640 1 -- 172.21.15.26:0/84886264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdce015c5f0 con 0x7fdce00b4b30 2023-12-19T17:01:05.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdcdcff9640 1 -- 172.21.15.26:0/84886264 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdcd0017020 con 0x7fdce00b4b30 2023-12-19T17:01:05.301 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T17:01:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdce67be640 1 -- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdccc065ca0 msgr2=0x7fdccc068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:01:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdce67be640 1 --2- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdccc065ca0 0x7fdccc068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdce67be640 1 -- 172.21.15.26:0/84886264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce00b4b30 msgr2=0x7fdce015aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdce67be640 1 --2- 172.21.15.26:0/84886264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce00b4b30 0x7fdce015aba0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000c4d0 tx=0x7fdcd000c990 comp rx=0 tx=0).stop 2023-12-19T17:01:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdce67be640 1 -- 172.21.15.26:0/84886264 shutdown_connections 2023-12-19T17:01:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdce67be640 1 --2- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdce015cde0 0x7fdce00cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdce67be640 1 --2- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdccc065ca0 0x7fdccc068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdce67be640 1 --2- 172.21.15.26:0/84886264 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdce00b8600 0x7fdce00cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdce67be640 1 --2- 172.21.15.26:0/84886264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdce00b4b30 0x7fdce015aba0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdce67be640 1 -- 172.21.15.26:0/84886264 >> 172.21.15.26:0/84886264 conn(0x7fdce0065ee0 msgr2=0x7fdce00d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdce67be640 1 -- 172.21.15.26:0/84886264 shutdown_connections 2023-12-19T17:01:05.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.297+0000 7fdce67be640 1 -- 172.21.15.26:0/84886264 wait complete. 2023-12-19T17:01:05.324 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T17:01:05.355 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:05.349+0000 7f2672276180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:01:05.406 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:05.404+0000 7fb4d93f6180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:01:05.469 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:05.468+0000 7fb4d93f6180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:01:05.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61d1ce5640 1 Processor -- start 2023-12-19T17:01:05.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61d1ce5640 1 -- start start 2023-12-19T17:01:05.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61d1ce5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61cc15cd40 0x7f61cc0ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61d1ce5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f61cc0d3300 0x7f61cc160d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61d1ce5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61cc0d29d0 con 0x7f61cc160900 2023-12-19T17:01:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61d1ce5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61cc15d680 con 0x7f61cc15cd40 2023-12-19T17:01:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61d1ce5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61cc161750 con 0x7f61cc0d3300 2023-12-19T17:01:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61cb7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61cc15cd40 0x7f61cc0ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61cb7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f61cc15cd40 0x7f61cc0ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35332/0 (socket says 172.21.15.26:35332) 2023-12-19T17:01:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61cb7fe640 1 -- 172.21.15.26:0/3509635909 learned_addr learned my addr 172.21.15.26:0/3509635909 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61cbfff640 1 --2- 172.21.15.26:0/3509635909 >> v2:172.21.15.160:3300/0 conn(0x7f61cc0d3300 0x7f61cc160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61cb7fe640 1 -- 172.21.15.26:0/3509635909 >> v2:172.21.15.160:3300/0 conn(0x7f61cc0d3300 msgr2=0x7f61cc160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61cb7fe640 1 --2- 172.21.15.26:0/3509635909 >> v2:172.21.15.160:3300/0 conn(0x7f61cc0d3300 0x7f61cc160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61cb7fe640 1 -- 172.21.15.26:0/3509635909 >> v1:172.21.15.160:6789/0 conn(0x7f61cc160900 legacy=0x7f61cc0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:05.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.501+0000 7f61cb7fe640 1 -- 172.21.15.26:0/3509635909 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc161470 con 0x7f61cc15cd40 2023-12-19T17:01:05.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.505+0000 7f61cb7fe640 1 --2- 172.21.15.26:0/3509635909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc15cd40 0x7f61cc0ce040 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0d01d0 tx=0x7f61bc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579d3fce4b1c7da9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:05.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.505+0000 7f61caffd640 1 -- 172.21.15.26:0/3509635909 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0ab610 con 0x7f61cc15cd40 2023-12-19T17:01:05.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.505+0000 7f61caffd640 1 -- 172.21.15.26:0/3509635909 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc0ab770 con 0x7f61cc15cd40 2023-12-19T17:01:05.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.505+0000 7f61caffd640 1 -- 172.21.15.26:0/3509635909 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0abb00 con 0x7f61cc15cd40 2023-12-19T17:01:05.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.505+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3509635909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc15cd40 msgr2=0x7f61cc0ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:05.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.505+0000 7f61d1ce5640 1 --2- 172.21.15.26:0/3509635909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc15cd40 0x7f61cc0ce040 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0d01d0 tx=0x7f61bc01c500 comp rx=0 tx=0).stop 2023-12-19T17:01:05.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.505+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3509635909 shutdown_connections 2023-12-19T17:01:05.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.505+0000 7f61d1ce5640 1 --2- 172.21.15.26:0/3509635909 >> v2:172.21.15.160:3300/0 conn(0x7f61cc0d3300 0x7f61cc160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.505+0000 7f61d1ce5640 1 --2- 172.21.15.26:0/3509635909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc15cd40 0x7f61cc0ce040 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.505+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3509635909 >> 172.21.15.26:0/3509635909 conn(0x7f61cc06a120 msgr2=0x7f61cc06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:05.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.505+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3509635909 shutdown_connections 2023-12-19T17:01:05.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.505+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3509635909 wait complete. 2023-12-19T17:01:05.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61d1ce5640 1 Processor -- start 2023-12-19T17:01:05.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61d1ce5640 1 -- start start 2023-12-19T17:01:05.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61d1ce5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc0d3300 0x7f61cc154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:05.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61d1ce5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc15cd40 0x7f61cc15bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:05.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61d1ce5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc160900 0x7f61cc155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:05.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61d1ce5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc15c970 con 0x7f61cc15cd40 2023-12-19T17:01:05.514 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f2672276180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:01:05.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61d1ce5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc158db0 con 0x7f61cc160900 2023-12-19T17:01:05.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61d1ce5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc158480 con 0x7f61cc0d3300 2023-12-19T17:01:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61c3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc15cd40 0x7f61cc15bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61c3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc15cd40 0x7f61cc15bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35344/0 (socket says 172.21.15.26:35344) 2023-12-19T17:01:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61c3fff640 1 -- 172.21.15.26:0/3369334776 learned_addr learned my addr 172.21.15.26:0/3369334776 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61cbfff640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc160900 0x7f61cc155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61cb7fe640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc0d3300 0x7f61cc154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61c3fff640 1 -- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc0d3300 msgr2=0x7f61cc154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61c3fff640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc0d3300 0x7f61cc154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61c3fff640 1 -- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc160900 msgr2=0x7f61cc155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61c3fff640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc160900 0x7f61cc155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61c3fff640 1 -- 172.21.15.26:0/3369334776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc009480 con 0x7f61cc15cd40 2023-12-19T17:01:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61cbfff640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc160900 0x7f61cc155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:01:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61c3fff640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc15cd40 0x7f61cc15bf40 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c3c0 tx=0x7f61b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61c8ff9640 1 -- 172.21.15.26:0/3369334776 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b40052d0 con 0x7f61cc15cd40 2023-12-19T17:01:05.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61c8ff9640 1 -- 172.21.15.26:0/3369334776 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b4005430 con 0x7f61cc15cd40 2023-12-19T17:01:05.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3369334776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61cc158760 con 0x7f61cc15cd40 2023-12-19T17:01:05.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61c8ff9640 1 -- 172.21.15.26:0/3369334776 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4003840 con 0x7f61cc15cd40 2023-12-19T17:01:05.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.509+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3369334776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61cc1f6290 con 0x7f61cc15cd40 2023-12-19T17:01:05.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.513+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3369334776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61cc158f20 con 0x7f61cc15cd40 2023-12-19T17:01:05.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.513+0000 7f61c8ff9640 1 -- 172.21.15.26:0/3369334776 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 64) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f61b40039a0 con 0x7f61cc15cd40 2023-12-19T17:01:05.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.513+0000 7f61c8ff9640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f61ac065ca0 0x7f61ac068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:05.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.513+0000 7f61c8ff9640 1 -- 172.21.15.26:0/3369334776 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f61b4082c50 con 0x7f61cc15cd40 2023-12-19T17:01:05.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.513+0000 7f61cb7fe640 1 -- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f61ac065ca0 msgr2=0x7f61ac068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:05.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.513+0000 7f61cb7fe640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f61ac065ca0 0x7f61ac068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:01:05.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.517+0000 7f61c8ff9640 1 -- 172.21.15.26:0/3369334776 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f61b4088020 con 0x7f61cc15cd40 2023-12-19T17:01:05.577 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:05.573+0000 7f2672276180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:01:05.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3369334776 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f61cc074270 con 0x7f61cc15cd40 2023-12-19T17:01:05.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61c8ff9640 1 -- 172.21.15.26:0/3369334776 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v124) v1 ==== 80+0+466 (secure 0 0 0) 0x7f61b4017080 con 0x7f61cc15cd40 2023-12-19T17:01:05.694 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6833/38098 2023-12-20T17:00:13.374887+0000 2023-12-19T17:01:05.694 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6832/38098 2023-12-20T17:00:13.374887+0000 2023-12-19T17:01:05.694 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/4135162200 2023-12-20T17:00:13.374887+0000 2023-12-19T17:01:05.694 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/378405713 2023-12-20T17:00:13.374887+0000 2023-12-19T17:01:05.694 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/520438069 2023-12-20T17:00:13.374887+0000 2023-12-19T17:01:05.694 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2978385192 2023-12-20T17:00:13.374887+0000 2023-12-19T17:01:05.694 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3723046945 2023-12-20T17:00:13.374887+0000 2023-12-19T17:01:05.694 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3946479365 2023-12-20T17:00:13.374887+0000 2023-12-19T17:01:05.695 INFO:teuthology.orchestra.run.smithi026.stderr:listed 8 entries 2023-12-19T17:01:05.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f61ac065ca0 msgr2=0x7f61ac068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:01:05.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61d1ce5640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f61ac065ca0 0x7f61ac068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3369334776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc15cd40 msgr2=0x7f61cc15bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:05.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61d1ce5640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc15cd40 0x7f61cc15bf40 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f61b400c3c0 tx=0x7f61b400c880 comp rx=0 tx=0).stop 2023-12-19T17:01:05.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3369334776 shutdown_connections 2023-12-19T17:01:05.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61d1ce5640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61cc160900 0x7f61cc155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61d1ce5640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61cc15cd40 0x7f61cc15bf40 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61d1ce5640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f61ac065ca0 0x7f61ac068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61d1ce5640 1 --2- 172.21.15.26:0/3369334776 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61cc0d3300 0x7f61cc154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:05.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3369334776 >> 172.21.15.26:0/3369334776 conn(0x7f61cc06a120 msgr2=0x7f61cc0b7850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:05.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3369334776 shutdown_connections 2023-12-19T17:01:05.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:05.689+0000 7f61d1ce5640 1 -- 172.21.15.26:0/3369334776 wait complete. 2023-12-19T17:01:05.713 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:01:05.713 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:01:05.713 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:01:05.715 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T17:01:05.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:05.716 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-19T17:01:05.721 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:05.720+0000 7fb4d93f6180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:01:05.791 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T17:01:05.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:05.791 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T17:01:05.804 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T17:01:05.805 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:01:05.814 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T17:01:05.814 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:01:05.814 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:01:05.834 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:01:05.834 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:01:05.834 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:01:05.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T17:01:05.843 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T17:01:05.843 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:01:05.843 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:05.837+0000 7f2672276180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:01:05.860 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:05.856+0000 7fb4d93f6180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:01:05.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T17:01:05.865 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:01:05.986 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:05.984+0000 7fb4d93f6180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:01:05.987 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:05.981+0000 7f2672276180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:01:06.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.029+0000 7f25af30b640 1 Processor -- start 2023-12-19T17:01:06.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.029+0000 7f25af30b640 1 -- start start 2023-12-19T17:01:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.029+0000 7f25af30b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f25a80b06f0 0x7f25a80b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.029+0000 7f25af30b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a80cdd80 con 0x7f25a81591f0 2023-12-19T17:01:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.029+0000 7f25af30b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a80c8cc0 con 0x7f25a81595d0 2023-12-19T17:01:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.029+0000 7f25af30b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a80ce060 con 0x7f25a80b06f0 2023-12-19T17:01:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.029+0000 7f25aeb0a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f25a80b06f0 0x7f25a80b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.029+0000 7f25aeb0a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f25a80b06f0 0x7f25a80b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60370/0 (socket says 172.21.15.26:60370) 2023-12-19T17:01:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.029+0000 7f25aeb0a640 1 -- 172.21.15.26:0/3806253695 learned_addr learned my addr 172.21.15.26:0/3806253695 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.029+0000 7f25ae309640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f25a81591f0 0x7f25a80cd2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41432/0 (socket says 172.21.15.26:41432) 2023-12-19T17:01:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.029+0000 7f25aeb0a640 1 -- 172.21.15.26:0/3806253695 >> v1:172.21.15.160:6789/0 conn(0x7f25a81595d0 legacy=0x7f25a80affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.029+0000 7f25aeb0a640 1 -- 172.21.15.26:0/3806253695 >> v1:172.21.15.26:6789/0 conn(0x7f25a81591f0 legacy=0x7f25a80cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.029+0000 7f25aeb0a640 1 -- 172.21.15.26:0/3806253695 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80c8fa0 con 0x7f25a80b06f0 2023-12-19T17:01:06.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.033+0000 7f25aeb0a640 1 --2- 172.21.15.26:0/3806253695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a80b06f0 0x7f25a80b1fc0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009e50 tx=0x7f25a001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ad76d615f42e9e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:06.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.033+0000 7f25ad307640 1 -- 172.21.15.26:0/3806253695 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a00a57b0 con 0x7f25a80b06f0 2023-12-19T17:01:06.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.033+0000 7f25ad307640 1 -- 172.21.15.26:0/3806253695 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a00a5910 con 0x7f25a80b06f0 2023-12-19T17:01:06.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.033+0000 7f25ad307640 1 -- 172.21.15.26:0/3806253695 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a00a5ca0 con 0x7f25a80b06f0 2023-12-19T17:01:06.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.033+0000 7f25af30b640 1 -- 172.21.15.26:0/3806253695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a80b06f0 msgr2=0x7f25a80b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.033+0000 7f25af30b640 1 --2- 172.21.15.26:0/3806253695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a80b06f0 0x7f25a80b1fc0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009e50 tx=0x7f25a001c950 comp rx=0 tx=0).stop 2023-12-19T17:01:06.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.033+0000 7f25af30b640 1 -- 172.21.15.26:0/3806253695 shutdown_connections 2023-12-19T17:01:06.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.033+0000 7f25af30b640 1 --2- 172.21.15.26:0/3806253695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a80b06f0 0x7f25a80b1fc0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.033+0000 7f25af30b640 1 -- 172.21.15.26:0/3806253695 >> 172.21.15.26:0/3806253695 conn(0x7f25a806b0f0 msgr2=0x7f25a806b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:06.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.033+0000 7f25af30b640 1 -- 172.21.15.26:0/3806253695 shutdown_connections 2023-12-19T17:01:06.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.033+0000 7f25af30b640 1 -- 172.21.15.26:0/3806253695 wait complete. 2023-12-19T17:01:06.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25af30b640 1 Processor -- start 2023-12-19T17:01:06.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25af30b640 1 -- start start 2023-12-19T17:01:06.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25af30b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a80b06f0 0x7f25a8151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25af30b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a81591f0 0x7f25a8154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25af30b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25a81595d0 0x7f25a80ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25af30b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a8159c80 con 0x7f25a81591f0 2023-12-19T17:01:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25af30b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a8152400 con 0x7f25a80b06f0 2023-12-19T17:01:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25af30b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a8154bc0 con 0x7f25a81595d0 2023-12-19T17:01:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25adb08640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a81591f0 0x7f25a8154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25adb08640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a81591f0 0x7f25a8154140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60374/0 (socket says 172.21.15.26:60374) 2023-12-19T17:01:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25adb08640 1 -- 172.21.15.26:0/2674992981 learned_addr learned my addr 172.21.15.26:0/2674992981 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25aeb0a640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25a81595d0 0x7f25a80ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25ae309640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a80b06f0 0x7f25a8151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25adb08640 1 -- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25a81595d0 msgr2=0x7f25a80ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25adb08640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25a81595d0 0x7f25a80ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25adb08640 1 -- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a80b06f0 msgr2=0x7f25a8151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25adb08640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a80b06f0 0x7f25a8151ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25adb08640 1 -- 172.21.15.26:0/2674992981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0009b30 con 0x7f25a81591f0 2023-12-19T17:01:06.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25adb08640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a81591f0 0x7f25a8154140 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f259800c390 tx=0x7f259800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:06.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f259effd640 1 -- 172.21.15.26:0/2674992981 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980052d0 con 0x7f25a81591f0 2023-12-19T17:01:06.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f259effd640 1 -- 172.21.15.26:0/2674992981 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2598005430 con 0x7f25a81591f0 2023-12-19T17:01:06.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f259effd640 1 -- 172.21.15.26:0/2674992981 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598003820 con 0x7f25a81591f0 2023-12-19T17:01:06.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25af30b640 1 -- 172.21.15.26:0/2674992981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a80cdb50 con 0x7f25a81591f0 2023-12-19T17:01:06.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25af30b640 1 -- 172.21.15.26:0/2674992981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a8155180 con 0x7f25a81591f0 2023-12-19T17:01:06.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.037+0000 7f25af30b640 1 -- 172.21.15.26:0/2674992981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a80670b0 con 0x7f25a81591f0 2023-12-19T17:01:06.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.041+0000 7f259effd640 1 -- 172.21.15.26:0/2674992981 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 64) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f2598003980 con 0x7f25a81591f0 2023-12-19T17:01:06.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.041+0000 7f259effd640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f258c065ca0 0x7f258c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.041+0000 7f259effd640 1 -- 172.21.15.26:0/2674992981 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f2598082e30 con 0x7f25a81591f0 2023-12-19T17:01:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.041+0000 7f25ae309640 1 -- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f258c065ca0 msgr2=0x7f258c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.041+0000 7f25ae309640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f258c065ca0 0x7f258c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:01:06.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.049+0000 7f259effd640 1 -- 172.21.15.26:0/2674992981 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2598050130 con 0x7f25a81591f0 2023-12-19T17:01:06.057 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:06.056+0000 7fb4d93f6180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:01:06.124 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:06.121+0000 7f2672276180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:01:06.193 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:06.192+0000 7fb4d93f6180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:01:06.200 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:06.197+0000 7f2672276180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:01:06.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25af30b640 1 -- 172.21.15.26:0/2674992981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25a8067400 con 0x7f25a81591f0 2023-12-19T17:01:06.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25ae309640 1 -- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f258c065ca0 msgr2=0x7f258c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:06.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25ae309640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f258c065ca0 0x7f258c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T17:01:06.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f259effd640 1 -- 172.21.15.26:0/2674992981 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f2598017090 con 0x7f25a81591f0 2023-12-19T17:01:06.249 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:06.249 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6884,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1869411616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1869411616},{"type":"v1","addr":"172.21.15.26:6835","nonce":1869411616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3129570713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3129570713},{"type":"v1","addr":"172.21.15.160:6836","nonce":3129570713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4063618989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":4063618989},{"type":"v1","addr":"172.21.15.160:6837","nonce":4063618989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:00:01.425222+0000","modified":"2023-12-19T17:00:04.444345+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7974":{"gid":7974,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/2837068623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2837068623},{"type":"v1","addr":"172.21.15.26:6834","nonce":2837068623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T17:01:06.249 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 57 2023-12-19T17:01:06.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25af30b640 1 -- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f258c065ca0 msgr2=0x7f258c068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:01:06.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25af30b640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f258c065ca0 0x7f258c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25af30b640 1 -- 172.21.15.26:0/2674992981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a81591f0 msgr2=0x7f25a8154140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25af30b640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a81591f0 0x7f25a8154140 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f259800c390 tx=0x7f259800c850 comp rx=0 tx=0).stop 2023-12-19T17:01:06.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25af30b640 1 -- 172.21.15.26:0/2674992981 shutdown_connections 2023-12-19T17:01:06.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25af30b640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f258c065ca0 0x7f258c068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25af30b640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25a81595d0 0x7f25a80ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25af30b640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25a81591f0 0x7f25a8154140 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25af30b640 1 --2- 172.21.15.26:0/2674992981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a80b06f0 0x7f25a8151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25af30b640 1 -- 172.21.15.26:0/2674992981 >> 172.21.15.26:0/2674992981 conn(0x7f25a806b0f0 msgr2=0x7f25a8156de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:06.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25af30b640 1 -- 172.21.15.26:0/2674992981 shutdown_connections 2023-12-19T17:01:06.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.245+0000 7f25af30b640 1 -- 172.21.15.26:0/2674992981 wait complete. 2023-12-19T17:01:06.271 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:01:06.345 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:06.341+0000 7f2672276180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:01:06.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f92a0be9640 1 Processor -- start 2023-12-19T17:01:06.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f92a0be9640 1 -- start start 2023-12-19T17:01:06.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f92a0be9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f929c15c7f0 0x7f929c06c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f92a0be9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f929c0b30d0 0x7f929c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f92a0be9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f929c0b3a60 con 0x7f929c0adb80 2023-12-19T17:01:06.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f92a0be9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f929c06d770 con 0x7f929c15c7f0 2023-12-19T17:01:06.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f92a0be9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929c06ced0 con 0x7f929c0b30d0 2023-12-19T17:01:06.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f929ad9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f929c0b30d0 0x7f929c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f929ad9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f929c0b30d0 0x7f929c0b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:57200/0 (socket says 172.21.15.26:57200) 2023-12-19T17:01:06.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f929ad9d640 1 -- 172.21.15.26:0/3016094173 learned_addr learned my addr 172.21.15.26:0/3016094173 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:06.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f929a59c640 1 --2- 172.21.15.26:0/3016094173 >> v2:172.21.15.160:3300/0 conn(0x7f929c15c7f0 0x7f929c06c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f9299d9b640 1 --1- 172.21.15.26:0/3016094173 >> v1:172.21.15.160:6789/0 conn(0x7f929c0adb80 0x7f929c0cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:50490/0 (socket says 172.21.15.26:50490) 2023-12-19T17:01:06.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f929ad9d640 1 -- 172.21.15.26:0/3016094173 >> v2:172.21.15.160:3300/0 conn(0x7f929c15c7f0 msgr2=0x7f929c06c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:06.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f929ad9d640 1 --2- 172.21.15.26:0/3016094173 >> v2:172.21.15.160:3300/0 conn(0x7f929c15c7f0 0x7f929c06c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f929ad9d640 1 -- 172.21.15.26:0/3016094173 >> v1:172.21.15.160:6789/0 conn(0x7f929c0adb80 legacy=0x7f929c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f929ad9d640 1 -- 172.21.15.26:0/3016094173 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c06d1b0 con 0x7f929c0b30d0 2023-12-19T17:01:06.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f929959a640 1 -- 172.21.15.26:0/3016094173 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935016673 0 0) 0x7f929c0b3a60 con 0x7f929c0adb80 2023-12-19T17:01:06.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.413+0000 7f929a59c640 1 --2- 172.21.15.26:0/3016094173 >> v2:172.21.15.160:3300/0 conn(0x7f929c15c7f0 0x7f929c06c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:01:06.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.417+0000 7f929ad9d640 1 --2- 172.21.15.26:0/3016094173 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f929c0b30d0 0x7f929c0b3520 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f929400d150 tx=0x7f929401ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a29ad6fc3cc7ccd3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:06.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.417+0000 7f929959a640 1 -- 172.21.15.26:0/3016094173 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929400b2d0 con 0x7f929c0b30d0 2023-12-19T17:01:06.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.417+0000 7f929959a640 1 -- 172.21.15.26:0/3016094173 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929400b430 con 0x7f929c0b30d0 2023-12-19T17:01:06.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.417+0000 7f929959a640 1 -- 172.21.15.26:0/3016094173 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92940a7b70 con 0x7f929c0b30d0 2023-12-19T17:01:06.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.417+0000 7f92a0be9640 1 -- 172.21.15.26:0/3016094173 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f929c0b30d0 msgr2=0x7f929c0b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.417+0000 7f92a0be9640 1 --2- 172.21.15.26:0/3016094173 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f929c0b30d0 0x7f929c0b3520 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f929400d150 tx=0x7f929401ec10 comp rx=0 tx=0).stop 2023-12-19T17:01:06.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.417+0000 7f92a0be9640 1 -- 172.21.15.26:0/3016094173 shutdown_connections 2023-12-19T17:01:06.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.417+0000 7f92a0be9640 1 --2- 172.21.15.26:0/3016094173 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f929c0b30d0 0x7f929c0b3520 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.417+0000 7f92a0be9640 1 --2- 172.21.15.26:0/3016094173 >> v2:172.21.15.160:3300/0 conn(0x7f929c15c7f0 0x7f929c06c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.417+0000 7f92a0be9640 1 -- 172.21.15.26:0/3016094173 >> 172.21.15.26:0/3016094173 conn(0x7f929c0658e0 msgr2=0x7f929c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:06.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.417+0000 7f92a0be9640 1 -- 172.21.15.26:0/3016094173 shutdown_connections 2023-12-19T17:01:06.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.417+0000 7f92a0be9640 1 -- 172.21.15.26:0/3016094173 wait complete. 2023-12-19T17:01:06.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f92a0be9640 1 Processor -- start 2023-12-19T17:01:06.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f92a0be9640 1 -- start start 2023-12-19T17:01:06.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f92a0be9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f929c0adb80 0x7f929c14fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f92a0be9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f929c0b30d0 0x7f929c14a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f92a0be9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c15c7f0 0x7f929c138da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f92a0be9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c151fd0 con 0x7f929c15c7f0 2023-12-19T17:01:06.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f92a0be9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929c1513c0 con 0x7f929c0adb80 2023-12-19T17:01:06.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f92a0be9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929c1516a0 con 0x7f929c0b30d0 2023-12-19T17:01:06.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f929ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c15c7f0 0x7f929c138da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f9299d9b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f929c0b30d0 0x7f929c14a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f929a59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f929c0adb80 0x7f929c14fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f9299d9b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f929c0b30d0 0x7f929c14a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:57212/0 (socket says 172.21.15.26:57212) 2023-12-19T17:01:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f9299d9b640 1 -- 172.21.15.26:0/56151237 learned_addr learned my addr 172.21.15.26:0/56151237 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f929a59c640 1 -- 172.21.15.26:0/56151237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f929c0b30d0 msgr2=0x7f929c14a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f929a59c640 1 --2- 172.21.15.26:0/56151237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f929c0b30d0 0x7f929c14a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f929a59c640 1 -- 172.21.15.26:0/56151237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c15c7f0 msgr2=0x7f929c138da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f929a59c640 1 --2- 172.21.15.26:0/56151237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c15c7f0 0x7f929c138da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f929a59c640 1 -- 172.21.15.26:0/56151237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c009420 con 0x7f929c0adb80 2023-12-19T17:01:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f929a59c640 1 --2- 172.21.15.26:0/56151237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f929c0adb80 0x7f929c14fcc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f928c00e8e0 tx=0x7f928c00eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f928affd640 1 -- 172.21.15.26:0/56151237 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c00c8e0 con 0x7f929c0adb80 2023-12-19T17:01:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f928affd640 1 -- 172.21.15.26:0/56151237 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928c00ca40 con 0x7f929c0adb80 2023-12-19T17:01:06.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f92a0be9640 1 -- 172.21.15.26:0/56151237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929400bd40 con 0x7f929c0adb80 2023-12-19T17:01:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f928affd640 1 -- 172.21.15.26:0/56151237 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f928c00cdf0 con 0x7f929c0adb80 2023-12-19T17:01:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.421+0000 7f92a0be9640 1 -- 172.21.15.26:0/56151237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929c151c60 con 0x7f929c0adb80 2023-12-19T17:01:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.425+0000 7f92a0be9640 1 -- 172.21.15.26:0/56151237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f929c074270 con 0x7f929c0adb80 2023-12-19T17:01:06.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.425+0000 7f928affd640 1 -- 172.21.15.26:0/56151237 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 64) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f928c013470 con 0x7f929c0adb80 2023-12-19T17:01:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.425+0000 7f928affd640 1 --2- 172.21.15.26:0/56151237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f927c065ca0 0x7f927c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.425+0000 7f928affd640 1 -- 172.21.15.26:0/56151237 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f928c083b90 con 0x7f929c0adb80 2023-12-19T17:01:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.425+0000 7f9299d9b640 1 -- 172.21.15.26:0/56151237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f927c065ca0 msgr2=0x7f927c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:06.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.425+0000 7f9299d9b640 1 --2- 172.21.15.26:0/56151237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f927c065ca0 0x7f927c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:01:06.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.429+0000 7f928affd640 1 -- 172.21.15.26:0/56151237 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f928c050e90 con 0x7f929c0adb80 2023-12-19T17:01:06.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.589+0000 7f92a0be9640 1 -- 172.21.15.26:0/56151237 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f929c070fd0 con 0x7f929c0adb80 2023-12-19T17:01:06.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.589+0000 7f928affd640 1 -- 172.21.15.26:0/56151237 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+17653 (secure 0 0 0) 0x7f928c0544b0 con 0x7f929c0adb80 2023-12-19T17:01:06.595 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:06.596 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":124,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:00:13.374923+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:59:59.967175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T17:00:00.420191+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6833/38098":"2023-12-20T17:00:13.374887+0000","172.21.15.160:6832/38098":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/4135162200":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/378405713":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/520438069":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/2978385192":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/3723046945":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/3946479365":"2023-12-20T17:00:13.374887+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:01:06.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.593+0000 7f92a0be9640 1 -- 172.21.15.26:0/56151237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f927c065ca0 msgr2=0x7f927c068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:01:06.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.593+0000 7f92a0be9640 1 --2- 172.21.15.26:0/56151237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f927c065ca0 0x7f927c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.593+0000 7f92a0be9640 1 -- 172.21.15.26:0/56151237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f929c0adb80 msgr2=0x7f929c14fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.593+0000 7f92a0be9640 1 --2- 172.21.15.26:0/56151237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f929c0adb80 0x7f929c14fcc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f928c00e8e0 tx=0x7f928c00eda0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.593+0000 7f92a0be9640 1 -- 172.21.15.26:0/56151237 shutdown_connections 2023-12-19T17:01:06.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.593+0000 7f92a0be9640 1 --2- 172.21.15.26:0/56151237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f929c15c7f0 0x7f929c138da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.593+0000 7f92a0be9640 1 --2- 172.21.15.26:0/56151237 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f927c065ca0 0x7f927c068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.593+0000 7f92a0be9640 1 --2- 172.21.15.26:0/56151237 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f929c0b30d0 0x7f929c14a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.593+0000 7f92a0be9640 1 --2- 172.21.15.26:0/56151237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f929c0adb80 0x7f929c14fcc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.593+0000 7f92a0be9640 1 -- 172.21.15.26:0/56151237 >> 172.21.15.26:0/56151237 conn(0x7f929c0658e0 msgr2=0x7f929c0cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:06.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.593+0000 7f92a0be9640 1 -- 172.21.15.26:0/56151237 shutdown_connections 2023-12-19T17:01:06.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.593+0000 7f92a0be9640 1 -- 172.21.15.26:0/56151237 wait complete. 2023-12-19T17:01:06.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T16:59:59.967175+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T17:00:00.420191+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T17:01:06.618 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T17:01:06.738 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:06.736+0000 7fb4d93f6180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:01:06.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.765+0000 7fbef7925640 1 Processor -- start 2023-12-19T17:01:06.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.765+0000 7fbef7925640 1 -- start start 2023-12-19T17:01:06.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.765+0000 7fbef7925640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbef015d0d0 0x7fbef00b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.765+0000 7fbef7925640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbef00b4600 0x7fbef00b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.765+0000 7fbef7925640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef015e200 con 0x7fbef015ccf0 2023-12-19T17:01:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.765+0000 7fbef7925640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef00b4f20 con 0x7fbef00b4600 2023-12-19T17:01:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.765+0000 7fbef7925640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef00d0aa0 con 0x7fbef015d0d0 2023-12-19T17:01:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.765+0000 7fbef5ec2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbef00b4600 0x7fbef00b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.765+0000 7fbef56c1640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbef015ccf0 0x7fbef00d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41446/0 (socket says 172.21.15.26:41446) 2023-12-19T17:01:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.765+0000 7fbef56c1640 1 -- 172.21.15.26:0/2819798627 learned_addr learned my addr 172.21.15.26:0/2819798627 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.765+0000 7fbef4ec0640 1 --2- 172.21.15.26:0/2819798627 >> v2:172.21.15.160:3300/0 conn(0x7fbef015d0d0 0x7fbef00b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.769+0000 7fbef5ec2640 1 -- 172.21.15.26:0/2819798627 >> v2:172.21.15.160:3300/0 conn(0x7fbef015d0d0 msgr2=0x7fbef00b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.769+0000 7fbef5ec2640 1 --2- 172.21.15.26:0/2819798627 >> v2:172.21.15.160:3300/0 conn(0x7fbef015d0d0 0x7fbef00b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.769+0000 7fbef5ec2640 1 -- 172.21.15.26:0/2819798627 >> v1:172.21.15.26:6789/0 conn(0x7fbef015ccf0 legacy=0x7fbef00d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.769+0000 7fbee7fff640 1 -- 172.21.15.26:0/2819798627 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668357850 0 0) 0x7fbef015e200 con 0x7fbef015ccf0 2023-12-19T17:01:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.769+0000 7fbef5ec2640 1 -- 172.21.15.26:0/2819798627 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef015d680 con 0x7fbef00b4600 2023-12-19T17:01:06.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.773+0000 7fbef5ec2640 1 --2- 172.21.15.26:0/2819798627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef00b4600 0x7fbef00b49e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fbef00d0400 tx=0x7fbee801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf6b1e31c5735a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.773+0000 7fbee7fff640 1 -- 172.21.15.26:0/2819798627 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee80a59a0 con 0x7fbef00b4600 2023-12-19T17:01:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.773+0000 7fbee7fff640 1 -- 172.21.15.26:0/2819798627 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee80a5b00 con 0x7fbef00b4600 2023-12-19T17:01:06.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.773+0000 7fbee7fff640 1 -- 172.21.15.26:0/2819798627 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee80a5e90 con 0x7fbef00b4600 2023-12-19T17:01:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.773+0000 7fbef7925640 1 -- 172.21.15.26:0/2819798627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef00b4600 msgr2=0x7fbef00b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.773+0000 7fbef7925640 1 --2- 172.21.15.26:0/2819798627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef00b4600 0x7fbef00b49e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fbef00d0400 tx=0x7fbee801c8e0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.773+0000 7fbef7925640 1 -- 172.21.15.26:0/2819798627 shutdown_connections 2023-12-19T17:01:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.773+0000 7fbef7925640 1 --2- 172.21.15.26:0/2819798627 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef00b4600 0x7fbef00b49e0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.773+0000 7fbef7925640 1 --2- 172.21.15.26:0/2819798627 >> v2:172.21.15.160:3300/0 conn(0x7fbef015d0d0 0x7fbef00b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.773+0000 7fbef7925640 1 -- 172.21.15.26:0/2819798627 >> 172.21.15.26:0/2819798627 conn(0x7fbef0065ee0 msgr2=0x7fbef00cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.773+0000 7fbef7925640 1 -- 172.21.15.26:0/2819798627 shutdown_connections 2023-12-19T17:01:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.773+0000 7fbef7925640 1 -- 172.21.15.26:0/2819798627 wait complete. 2023-12-19T17:01:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef7925640 1 Processor -- start 2023-12-19T17:01:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef7925640 1 -- start start 2023-12-19T17:01:06.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef7925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef00b4600 0x7fbef0158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef7925640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbef015ccf0 0x7fbef0154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef7925640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbef015d0d0 0x7fbef015bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef7925640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01566f0 con 0x7fbef00b4600 2023-12-19T17:01:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef7925640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01569d0 con 0x7fbef015ccf0 2023-12-19T17:01:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef7925640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0156cb0 con 0x7fbef015d0d0 2023-12-19T17:01:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef56c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef00b4600 0x7fbef0158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef56c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef00b4600 0x7fbef0158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60390/0 (socket says 172.21.15.26:60390) 2023-12-19T17:01:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef4ec0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbef015ccf0 0x7fbef0154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef56c1640 1 -- 172.21.15.26:0/1123980276 learned_addr learned my addr 172.21.15.26:0/1123980276 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef5ec2640 1 --2- 172.21.15.26:0/1123980276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbef015d0d0 0x7fbef015bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef56c1640 1 -- 172.21.15.26:0/1123980276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbef015d0d0 msgr2=0x7fbef015bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef56c1640 1 --2- 172.21.15.26:0/1123980276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbef015d0d0 0x7fbef015bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef56c1640 1 -- 172.21.15.26:0/1123980276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbef015ccf0 msgr2=0x7fbef0154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef56c1640 1 --2- 172.21.15.26:0/1123980276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbef015ccf0 0x7fbef0154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef56c1640 1 -- 172.21.15.26:0/1123980276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee8009a70 con 0x7fbef00b4600 2023-12-19T17:01:06.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef56c1640 1 --2- 172.21.15.26:0/1123980276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef00b4600 0x7fbef0158670 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0011f0 tx=0x7fbeec00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:06.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbee5ffb640 1 -- 172.21.15.26:0/1123980276 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0074e0 con 0x7fbef00b4600 2023-12-19T17:01:06.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbee5ffb640 1 -- 172.21.15.26:0/1123980276 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec007640 con 0x7fbef00b4600 2023-12-19T17:01:06.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbee5ffb640 1 -- 172.21.15.26:0/1123980276 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec004a10 con 0x7fbef00b4600 2023-12-19T17:01:06.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef7925640 1 -- 172.21.15.26:0/1123980276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef0156f90 con 0x7fbef00b4600 2023-12-19T17:01:06.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef7925640 1 -- 172.21.15.26:0/1123980276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef0157790 con 0x7fbef00b4600 2023-12-19T17:01:06.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.777+0000 7fbef7925640 1 -- 172.21.15.26:0/1123980276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef0074270 con 0x7fbef00b4600 2023-12-19T17:01:06.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.781+0000 7fbee5ffb640 1 -- 172.21.15.26:0/1123980276 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 64) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fbeec004b70 con 0x7fbef00b4600 2023-12-19T17:01:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.781+0000 7fbee5ffb640 1 --2- 172.21.15.26:0/1123980276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbed4065ca0 0x7fbed4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.781+0000 7fbee5ffb640 1 -- 172.21.15.26:0/1123980276 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fbeec082cf0 con 0x7fbef00b4600 2023-12-19T17:01:06.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.781+0000 7fbef4ec0640 1 -- 172.21.15.26:0/1123980276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbed4065ca0 msgr2=0x7fbed4068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:06.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.781+0000 7fbef4ec0640 1 --2- 172.21.15.26:0/1123980276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbed4065ca0 0x7fbed4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:01:06.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.785+0000 7fbee5ffb640 1 -- 172.21.15.26:0/1123980276 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbeec04ff20 con 0x7fbef00b4600 2023-12-19T17:01:06.824 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:06.820+0000 7fb4d93f6180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:01:06.931 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:06.925+0000 7f2672276180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:01:06.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.941+0000 7fbef7925640 1 -- 172.21.15.26:0/1123980276 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbef0157440 con 0x7fbef00b4600 2023-12-19T17:01:06.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.941+0000 7fbee5ffb640 1 -- 172.21.15.26:0/1123980276 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbeec018070 con 0x7fbef00b4600 2023-12-19T17:01:06.947 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T17:01:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.945+0000 7fbef7925640 1 -- 172.21.15.26:0/1123980276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbed4065ca0 msgr2=0x7fbed4068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:01:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.945+0000 7fbef7925640 1 --2- 172.21.15.26:0/1123980276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbed4065ca0 0x7fbed4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.945+0000 7fbef7925640 1 -- 172.21.15.26:0/1123980276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef00b4600 msgr2=0x7fbef0158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.945+0000 7fbef7925640 1 --2- 172.21.15.26:0/1123980276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef00b4600 0x7fbef0158670 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0011f0 tx=0x7fbeec00c9b0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.945+0000 7fbef7925640 1 -- 172.21.15.26:0/1123980276 shutdown_connections 2023-12-19T17:01:06.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.945+0000 7fbef7925640 1 --2- 172.21.15.26:0/1123980276 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbed4065ca0 0x7fbed4068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.945+0000 7fbef7925640 1 --2- 172.21.15.26:0/1123980276 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbef015d0d0 0x7fbef015bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.945+0000 7fbef7925640 1 --2- 172.21.15.26:0/1123980276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbef015ccf0 0x7fbef0154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.945+0000 7fbef7925640 1 --2- 172.21.15.26:0/1123980276 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbef00b4600 0x7fbef0158670 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:06.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.945+0000 7fbef7925640 1 -- 172.21.15.26:0/1123980276 >> 172.21.15.26:0/1123980276 conn(0x7fbef0065ee0 msgr2=0x7fbef0166ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:06.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.945+0000 7fbef7925640 1 -- 172.21.15.26:0/1123980276 shutdown_connections 2023-12-19T17:01:06.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:06.945+0000 7fbef7925640 1 -- 172.21.15.26:0/1123980276 wait complete. 2023-12-19T17:01:06.968 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T17:01:06.974 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:06.972+0000 7fb4d93f6180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:01:07.023 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:07.017+0000 7f2672276180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:01:07.058 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:07.056+0000 7fb4d93f6180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:01:07.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be7f18640 1 Processor -- start 2023-12-19T17:01:07.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be7f18640 1 -- start start 2023-12-19T17:01:07.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be7f18640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0be00b4d00 0x7f0be01608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be7f18640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0be00b85e0 0x7f0be00d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be7f18640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be01617f0 con 0x7f0be015cee0 2023-12-19T17:01:07.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be7f18640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be00b8a90 con 0x7f0be00b4d00 2023-12-19T17:01:07.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be7f18640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be0160f50 con 0x7f0be00b85e0 2023-12-19T17:01:07.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be54b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0be00b4d00 0x7f0be01608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be54b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0be00b4d00 0x7f0be01608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60398/0 (socket says 172.21.15.26:60398) 2023-12-19T17:01:07.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be54b3640 1 -- 172.21.15.26:0/2644253423 learned_addr learned my addr 172.21.15.26:0/2644253423 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:07.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be5cb4640 1 --1- 172.21.15.26:0/2644253423 >> v1:172.21.15.26:6789/0 conn(0x7f0be015cee0 0x7f0be00b45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41456/0 (socket says 172.21.15.26:41456) 2023-12-19T17:01:07.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be64b5640 1 --2- 172.21.15.26:0/2644253423 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0be00b85e0 0x7f0be00d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be54b3640 1 -- 172.21.15.26:0/2644253423 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0be00b85e0 msgr2=0x7f0be00d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:07.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be54b3640 1 --2- 172.21.15.26:0/2644253423 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0be00b85e0 0x7f0be00d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be54b3640 1 -- 172.21.15.26:0/2644253423 >> v1:172.21.15.26:6789/0 conn(0x7f0be015cee0 legacy=0x7f0be00b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be54b3640 1 -- 172.21.15.26:0/2644253423 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be0161230 con 0x7f0be00b4d00 2023-12-19T17:01:07.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.113+0000 7f0be64b5640 1 --2- 172.21.15.26:0/2644253423 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0be00b85e0 0x7f0be00d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:07.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.117+0000 7f0be54b3640 1 --2- 172.21.15.26:0/2644253423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be00b4d00 0x7f0be01608e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009870 tx=0x7f0bd001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07b59e5e2ac14d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:07.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.117+0000 7f0be4cb2640 1 -- 172.21.15.26:0/2644253423 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00ab620 con 0x7f0be00b4d00 2023-12-19T17:01:07.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.117+0000 7f0be4cb2640 1 -- 172.21.15.26:0/2644253423 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd00ab780 con 0x7f0be00b4d00 2023-12-19T17:01:07.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.117+0000 7f0be4cb2640 1 -- 172.21.15.26:0/2644253423 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00abb10 con 0x7f0be00b4d00 2023-12-19T17:01:07.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.117+0000 7f0be7f18640 1 -- 172.21.15.26:0/2644253423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be00b4d00 msgr2=0x7f0be01608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.117+0000 7f0be7f18640 1 --2- 172.21.15.26:0/2644253423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be00b4d00 0x7f0be01608e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009870 tx=0x7f0bd001c420 comp rx=0 tx=0).stop 2023-12-19T17:01:07.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.117+0000 7f0be7f18640 1 -- 172.21.15.26:0/2644253423 shutdown_connections 2023-12-19T17:01:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.117+0000 7f0be7f18640 1 --2- 172.21.15.26:0/2644253423 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0be00b85e0 0x7f0be00d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.117+0000 7f0be7f18640 1 --2- 172.21.15.26:0/2644253423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be00b4d00 0x7f0be01608e0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.117+0000 7f0be7f18640 1 -- 172.21.15.26:0/2644253423 >> 172.21.15.26:0/2644253423 conn(0x7f0be00cf5f0 msgr2=0x7f0be00cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.117+0000 7f0be7f18640 1 -- 172.21.15.26:0/2644253423 shutdown_connections 2023-12-19T17:01:07.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.117+0000 7f0be7f18640 1 -- 172.21.15.26:0/2644253423 wait complete. 2023-12-19T17:01:07.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be7f18640 1 Processor -- start 2023-12-19T17:01:07.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be7f18640 1 -- start start 2023-12-19T17:01:07.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be7f18640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0be00b4d00 0x7f0be01540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be7f18640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be00b85e0 0x7f0be015e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be7f18640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be015cee0 0x7f0be00ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be7f18640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be0154b30 con 0x7f0be015cee0 2023-12-19T17:01:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be7f18640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be0155f30 con 0x7f0be00b85e0 2023-12-19T17:01:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be7f18640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be00caea0 con 0x7f0be00b4d00 2023-12-19T17:01:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be64b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be015cee0 0x7f0be00ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be64b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be015cee0 0x7f0be00ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60402/0 (socket says 172.21.15.26:60402) 2023-12-19T17:01:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be64b5640 1 -- 172.21.15.26:0/3764983364 learned_addr learned my addr 172.21.15.26:0/3764983364 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be5cb4640 1 --2- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0be00b4d00 0x7f0be01540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be54b3640 1 --2- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be00b85e0 0x7f0be015e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be64b5640 1 -- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0be00b4d00 msgr2=0x7f0be01540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be64b5640 1 --2- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0be00b4d00 0x7f0be01540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be64b5640 1 -- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be00b85e0 msgr2=0x7f0be015e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be64b5640 1 --2- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be00b85e0 0x7f0be015e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be64b5640 1 -- 172.21.15.26:0/3764983364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd00094d0 con 0x7f0be015cee0 2023-12-19T17:01:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be64b5640 1 --2- 172.21.15.26:0/3764983364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be015cee0 0x7f0be00ca330 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8007ef0 tx=0x7f0bd800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:07.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0bd67fc640 1 -- 172.21.15.26:0/3764983364 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8003ba0 con 0x7f0be015cee0 2023-12-19T17:01:07.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0bd67fc640 1 -- 172.21.15.26:0/3764983364 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd800f040 con 0x7f0be015cee0 2023-12-19T17:01:07.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0bd67fc640 1 -- 172.21.15.26:0/3764983364 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd800de80 con 0x7f0be015cee0 2023-12-19T17:01:07.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be7f18640 1 -- 172.21.15.26:0/3764983364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be00cb180 con 0x7f0be015cee0 2023-12-19T17:01:07.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.121+0000 7f0be7f18640 1 -- 172.21.15.26:0/3764983364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be01547c0 con 0x7f0be015cee0 2023-12-19T17:01:07.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.125+0000 7f0be7f18640 1 -- 172.21.15.26:0/3764983364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be0154ca0 con 0x7f0be015cee0 2023-12-19T17:01:07.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.125+0000 7f0bd67fc640 1 -- 172.21.15.26:0/3764983364 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 64) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f0bd8004030 con 0x7f0be015cee0 2023-12-19T17:01:07.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.125+0000 7f0bd67fc640 1 --2- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bc4065ca0 0x7f0bc4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.125+0000 7f0bd67fc640 1 -- 172.21.15.26:0/3764983364 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f0bd8083e40 con 0x7f0be015cee0 2023-12-19T17:01:07.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.125+0000 7f0be5cb4640 1 -- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bc4065ca0 msgr2=0x7f0bc4068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:07.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.125+0000 7f0be5cb4640 1 --2- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bc4065ca0 0x7f0bc4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:01:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.129+0000 7f0bd67fc640 1 -- 172.21.15.26:0/3764983364 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0bd8050f60 con 0x7f0be015cee0 2023-12-19T17:01:07.186 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:07.181+0000 7f2672276180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:01:07.274 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:07.269+0000 7f2672276180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:01:07.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0be7f18640 1 -- 172.21.15.26:0/3764983364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0be0074270 con 0x7f0be015cee0 2023-12-19T17:01:07.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0bd67fc640 1 -- 172.21.15.26:0/3764983364 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0bd801a020 con 0x7f0be015cee0 2023-12-19T17:01:07.293 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T17:01:07.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0be7f18640 1 -- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bc4065ca0 msgr2=0x7f0bc4068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:01:07.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0be7f18640 1 --2- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bc4065ca0 0x7f0bc4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0be7f18640 1 -- 172.21.15.26:0/3764983364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be015cee0 msgr2=0x7f0be00ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0be7f18640 1 --2- 172.21.15.26:0/3764983364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be015cee0 0x7f0be00ca330 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8007ef0 tx=0x7f0bd800d4a0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0be7f18640 1 -- 172.21.15.26:0/3764983364 shutdown_connections 2023-12-19T17:01:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0be7f18640 1 --2- 172.21.15.26:0/3764983364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be015cee0 0x7f0be00ca330 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0be7f18640 1 --2- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0be00b85e0 0x7f0be015e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0be7f18640 1 --2- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0bc4065ca0 0x7f0bc4068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0be7f18640 1 --2- 172.21.15.26:0/3764983364 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0be00b4d00 0x7f0be01540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0be7f18640 1 -- 172.21.15.26:0/3764983364 >> 172.21.15.26:0/3764983364 conn(0x7f0be00cf5f0 msgr2=0x7f0be00b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0be7f18640 1 -- 172.21.15.26:0/3764983364 shutdown_connections 2023-12-19T17:01:07.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.289+0000 7f0be7f18640 1 -- 172.21.15.26:0/3764983364 wait complete. 2023-12-19T17:01:07.316 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T17:01:07.377 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:07.376+0000 7fb4d93f6180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:01:07.437 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:07.436+0000 7fb4d93f6180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:01:07.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.465+0000 7f3919e73640 1 Processor -- start 2023-12-19T17:01:07.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.465+0000 7f3919e73640 1 -- start start 2023-12-19T17:01:07.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.469+0000 7f3919e73640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f39140b4600 0x7f39140b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.469+0000 7f3919e73640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f391415e200 con 0x7f391415ccf0 2023-12-19T17:01:07.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.469+0000 7f3919e73640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39140b4f20 con 0x7f391415d0d0 2023-12-19T17:01:07.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.469+0000 7f3919e73640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39140d0aa0 con 0x7f39140b4600 2023-12-19T17:01:07.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.469+0000 7f3913fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f39140b4600 0x7f39140b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.469+0000 7f39137fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f391415ccf0 0x7f39140d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41464/0 (socket says 172.21.15.26:41464) 2023-12-19T17:01:07.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.469+0000 7f39137fe640 1 -- 172.21.15.26:0/3183487877 learned_addr learned my addr 172.21.15.26:0/3183487877 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:07.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.469+0000 7f3912ffd640 1 --1- 172.21.15.26:0/3183487877 >> v1:172.21.15.160:6789/0 conn(0x7f391415d0d0 0x7f39140b8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:50494/0 (socket says 172.21.15.26:50494) 2023-12-19T17:01:07.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.469+0000 7f3913fff640 1 -- 172.21.15.26:0/3183487877 >> v1:172.21.15.160:6789/0 conn(0x7f391415d0d0 legacy=0x7f39140b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.469+0000 7f39127fc640 1 -- 172.21.15.26:0/3183487877 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982006401 0 0) 0x7f39140b4f20 con 0x7f391415d0d0 2023-12-19T17:01:07.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.469+0000 7f3913fff640 1 -- 172.21.15.26:0/3183487877 >> v1:172.21.15.26:6789/0 conn(0x7f391415ccf0 legacy=0x7f39140d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.469+0000 7f3913fff640 1 -- 172.21.15.26:0/3183487877 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391415d680 con 0x7f39140b4600 2023-12-19T17:01:07.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.473+0000 7f3913fff640 1 --2- 172.21.15.26:0/3183487877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39140b4600 0x7f39140b49e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f390c009d90 tx=0x7f390c01c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3908df46070f226 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:07.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.473+0000 7f39127fc640 1 -- 172.21.15.26:0/3183487877 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c0a57f0 con 0x7f39140b4600 2023-12-19T17:01:07.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.473+0000 7f39127fc640 1 -- 172.21.15.26:0/3183487877 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c0a5950 con 0x7f39140b4600 2023-12-19T17:01:07.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.473+0000 7f39127fc640 1 -- 172.21.15.26:0/3183487877 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c0a5ce0 con 0x7f39140b4600 2023-12-19T17:01:07.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.473+0000 7f3919e73640 1 -- 172.21.15.26:0/3183487877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39140b4600 msgr2=0x7f39140b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.473+0000 7f3919e73640 1 --2- 172.21.15.26:0/3183487877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39140b4600 0x7f39140b49e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f390c009d90 tx=0x7f390c01c7b0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.473+0000 7f3919e73640 1 -- 172.21.15.26:0/3183487877 shutdown_connections 2023-12-19T17:01:07.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.473+0000 7f3919e73640 1 --2- 172.21.15.26:0/3183487877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39140b4600 0x7f39140b49e0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.473+0000 7f3919e73640 1 -- 172.21.15.26:0/3183487877 >> 172.21.15.26:0/3183487877 conn(0x7f3914065ee0 msgr2=0x7f39140cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:07.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.473+0000 7f3919e73640 1 -- 172.21.15.26:0/3183487877 shutdown_connections 2023-12-19T17:01:07.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.473+0000 7f3919e73640 1 -- 172.21.15.26:0/3183487877 wait complete. 2023-12-19T17:01:07.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.473+0000 7f3919e73640 1 Processor -- start 2023-12-19T17:01:07.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.473+0000 7f3919e73640 1 -- start start 2023-12-19T17:01:07.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3919e73640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39140b4600 0x7f391415ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3919e73640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f391415ccf0 0x7f3914151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3919e73640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f391415d0d0 0x7f39141f8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3919e73640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391415b3d0 con 0x7f391415ccf0 2023-12-19T17:01:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3919e73640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3914155a40 con 0x7f39140b4600 2023-12-19T17:01:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3919e73640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3914152310 con 0x7f391415d0d0 2023-12-19T17:01:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3912ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f391415ccf0 0x7f3914151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3912ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f391415ccf0 0x7f3914151dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60418/0 (socket says 172.21.15.26:60418) 2023-12-19T17:01:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3912ffd640 1 -- 172.21.15.26:0/4160327477 learned_addr learned my addr 172.21.15.26:0/4160327477 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f39137fe640 1 --2- 172.21.15.26:0/4160327477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39140b4600 0x7f391415ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3913fff640 1 --2- 172.21.15.26:0/4160327477 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f391415d0d0 0x7f39141f8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3912ffd640 1 -- 172.21.15.26:0/4160327477 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f391415d0d0 msgr2=0x7f39141f8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3912ffd640 1 --2- 172.21.15.26:0/4160327477 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f391415d0d0 0x7f39141f8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3912ffd640 1 -- 172.21.15.26:0/4160327477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39140b4600 msgr2=0x7f391415ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3912ffd640 1 --2- 172.21.15.26:0/4160327477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39140b4600 0x7f391415ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3912ffd640 1 -- 172.21.15.26:0/4160327477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c009a70 con 0x7f391415ccf0 2023-12-19T17:01:07.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3912ffd640 1 --2- 172.21.15.26:0/4160327477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f391415ccf0 0x7f3914151dd0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f390800c420 tx=0x7f390800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:07.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f38fbfff640 1 -- 172.21.15.26:0/4160327477 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39080074e0 con 0x7f391415ccf0 2023-12-19T17:01:07.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f38fbfff640 1 -- 172.21.15.26:0/4160327477 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3908007640 con 0x7f391415ccf0 2023-12-19T17:01:07.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f38fbfff640 1 -- 172.21.15.26:0/4160327477 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908004a30 con 0x7f391415ccf0 2023-12-19T17:01:07.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3919e73640 1 -- 172.21.15.26:0/4160327477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39141f94c0 con 0x7f391415ccf0 2023-12-19T17:01:07.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3919e73640 1 -- 172.21.15.26:0/4160327477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3914155c70 con 0x7f391415ccf0 2023-12-19T17:01:07.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f3919e73640 1 -- 172.21.15.26:0/4160327477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39140b8b90 con 0x7f391415ccf0 2023-12-19T17:01:07.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.477+0000 7f38fbfff640 1 -- 172.21.15.26:0/4160327477 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 65) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f3908004b90 con 0x7f391415ccf0 2023-12-19T17:01:07.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.481+0000 7f38fbfff640 1 -- 172.21.15.26:0/4160327477 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7185+0+0 (secure 0 0 0) 0x7f3908082f60 con 0x7f391415ccf0 2023-12-19T17:01:07.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.485+0000 7f38fbfff640 1 -- 172.21.15.26:0/4160327477 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f39080507c0 con 0x7f391415ccf0 2023-12-19T17:01:07.520 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:07.516+0000 7fb464c6e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:07.520 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:07.516+0000 7fb464c6e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:07.520 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:07.516+0000 7fb464c6e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:07.520 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:07.516+0000 7fb464c6e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:07.520 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:07.516+0000 7fb464c6e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:07.521 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:07.520+0000 7fb460c66640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:07.521 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:07.520+0000 7fb460c66640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:07.521 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:07.520+0000 7fb460c66640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:07.521 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:07.520+0000 7fb460c66640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:07.521 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:07.520+0000 7fb460c66640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:07.648 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:07.645+0000 7f2672276180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:01:07.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.665+0000 7f3919e73640 1 -- 172.21.15.26:0/4160327477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3914074270 con 0x7f391415ccf0 2023-12-19T17:01:07.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.665+0000 7f38fbfff640 1 -- 172.21.15.26:0/4160327477 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3908053de0 con 0x7f391415ccf0 2023-12-19T17:01:07.670 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T17:01:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.665+0000 7f3919e73640 1 -- 172.21.15.26:0/4160327477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f391415ccf0 msgr2=0x7f3914151dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.665+0000 7f3919e73640 1 --2- 172.21.15.26:0/4160327477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f391415ccf0 0x7f3914151dd0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f390800c420 tx=0x7f390800c8e0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.669+0000 7f3919e73640 1 -- 172.21.15.26:0/4160327477 shutdown_connections 2023-12-19T17:01:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.669+0000 7f3919e73640 1 --2- 172.21.15.26:0/4160327477 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f391415d0d0 0x7f39141f8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.669+0000 7f3919e73640 1 --2- 172.21.15.26:0/4160327477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f391415ccf0 0x7f3914151dd0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.669+0000 7f3919e73640 1 --2- 172.21.15.26:0/4160327477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39140b4600 0x7f391415ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.669+0000 7f3919e73640 1 -- 172.21.15.26:0/4160327477 >> 172.21.15.26:0/4160327477 conn(0x7f3914065ee0 msgr2=0x7f39140b8400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.669+0000 7f3919e73640 1 -- 172.21.15.26:0/4160327477 shutdown_connections 2023-12-19T17:01:07.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.669+0000 7f3919e73640 1 -- 172.21.15.26:0/4160327477 wait complete. 2023-12-19T17:01:07.692 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:01:07.713 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:07.709+0000 7f2672276180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:01:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178b59e640 1 Processor -- start 2023-12-19T17:01:07.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178b59e640 1 -- start start 2023-12-19T17:01:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178b59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f178c15cca0 0x7f178c0b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178b59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f178c0b8760 0x7f178c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178b59e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178c0b6fb0 con 0x7f178c0b6b20 2023-12-19T17:01:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178b59e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f178c0b7c30 con 0x7f178c15cca0 2023-12-19T17:01:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178b59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0b7290 con 0x7f178c0b8760 2023-12-19T17:01:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178a59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f178c15cca0 0x7f178c0b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178a59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f178c15cca0 0x7f178c0b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60422/0 (socket says 172.21.15.26:60422) 2023-12-19T17:01:07.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178a59c640 1 -- 172.21.15.26:0/1503937538 learned_addr learned my addr 172.21.15.26:0/1503937538 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f1789d9b640 1 --2- 172.21.15.26:0/1503937538 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f178c0b8760 0x7f178c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178a59c640 1 -- 172.21.15.26:0/1503937538 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f178c0b8760 msgr2=0x7f178c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178a59c640 1 --2- 172.21.15.26:0/1503937538 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f178c0b8760 0x7f178c0cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178a59c640 1 -- 172.21.15.26:0/1503937538 >> v1:172.21.15.160:6789/0 conn(0x7f178c0b6b20 legacy=0x7f178c0b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.833+0000 7f178a59c640 1 -- 172.21.15.26:0/1503937538 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c0b7570 con 0x7f178c15cca0 2023-12-19T17:01:07.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.837+0000 7f178a59c640 1 --2- 172.21.15.26:0/1503937538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178c15cca0 0x7f178c0b65e0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f177c009870 tx=0x7f177c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d457708222a0bf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:07.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.837+0000 7f178959a640 1 -- 172.21.15.26:0/1503937538 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c0a95f0 con 0x7f178c15cca0 2023-12-19T17:01:07.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.837+0000 7f178959a640 1 -- 172.21.15.26:0/1503937538 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177c0a9750 con 0x7f178c15cca0 2023-12-19T17:01:07.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.837+0000 7f178959a640 1 -- 172.21.15.26:0/1503937538 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177c0a9ae0 con 0x7f178c15cca0 2023-12-19T17:01:07.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.837+0000 7f178b59e640 1 -- 172.21.15.26:0/1503937538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178c15cca0 msgr2=0x7f178c0b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.837+0000 7f178b59e640 1 --2- 172.21.15.26:0/1503937538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178c15cca0 0x7f178c0b65e0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f177c009870 tx=0x7f177c01c510 comp rx=0 tx=0).stop 2023-12-19T17:01:07.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.837+0000 7f178b59e640 1 -- 172.21.15.26:0/1503937538 shutdown_connections 2023-12-19T17:01:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.837+0000 7f178b59e640 1 --2- 172.21.15.26:0/1503937538 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f178c0b8760 0x7f178c0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.837+0000 7f178b59e640 1 --2- 172.21.15.26:0/1503937538 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178c15cca0 0x7f178c0b65e0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.837+0000 7f178b59e640 1 -- 172.21.15.26:0/1503937538 >> 172.21.15.26:0/1503937538 conn(0x7f178c06a120 msgr2=0x7f178c06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.837+0000 7f178b59e640 1 -- 172.21.15.26:0/1503937538 shutdown_connections 2023-12-19T17:01:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.837+0000 7f178b59e640 1 -- 172.21.15.26:0/1503937538 wait complete. 2023-12-19T17:01:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f178b59e640 1 Processor -- start 2023-12-19T17:01:07.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f178b59e640 1 -- start start 2023-12-19T17:01:07.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f178b59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f178c0b8760 0x7f178c153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f178b59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178c15cca0 0x7f178c15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f178b59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f178c15b430 0x7f178c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:07.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f178b59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c1406b0 con 0x7f178c15cca0 2023-12-19T17:01:07.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f178b59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c153640 con 0x7f178c0b8760 2023-12-19T17:01:07.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f178b59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c13f750 con 0x7f178c15b430 2023-12-19T17:01:07.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f1789d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178c15cca0 0x7f178c15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f1789d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178c15cca0 0x7f178c15aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60426/0 (socket says 172.21.15.26:60426) 2023-12-19T17:01:07.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f1789d9b640 1 -- 172.21.15.26:0/3114891500 learned_addr learned my addr 172.21.15.26:0/3114891500 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:07.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f178a59c640 1 --2- 172.21.15.26:0/3114891500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f178c0b8760 0x7f178c153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f178ad9d640 1 --2- 172.21.15.26:0/3114891500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f178c15b430 0x7f178c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:07.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f1789d9b640 1 -- 172.21.15.26:0/3114891500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f178c15b430 msgr2=0x7f178c1537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f1789d9b640 1 --2- 172.21.15.26:0/3114891500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f178c15b430 0x7f178c1537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f1789d9b640 1 -- 172.21.15.26:0/3114891500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f178c0b8760 msgr2=0x7f178c153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:07.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f1789d9b640 1 --2- 172.21.15.26:0/3114891500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f178c0b8760 0x7f178c153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:07.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f1789d9b640 1 -- 172.21.15.26:0/3114891500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177c0094d0 con 0x7f178c15cca0 2023-12-19T17:01:07.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f1789d9b640 1 --2- 172.21.15.26:0/3114891500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178c15cca0 0x7f178c15aef0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f178000d800 tx=0x7f178000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:07.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f177affd640 1 -- 172.21.15.26:0/3114891500 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178000b6c0 con 0x7f178c15cca0 2023-12-19T17:01:07.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f177affd640 1 -- 172.21.15.26:0/3114891500 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178000b820 con 0x7f178c15cca0 2023-12-19T17:01:07.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f177affd640 1 -- 172.21.15.26:0/3114891500 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178000bb90 con 0x7f178c15cca0 2023-12-19T17:01:07.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f178b59e640 1 -- 172.21.15.26:0/3114891500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178c13fa30 con 0x7f178c15cca0 2023-12-19T17:01:07.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f178b59e640 1 -- 172.21.15.26:0/3114891500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c13fff0 con 0x7f178c15cca0 2023-12-19T17:01:07.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.841+0000 7f178b59e640 1 -- 172.21.15.26:0/3114891500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178c067400 con 0x7f178c15cca0 2023-12-19T17:01:07.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.845+0000 7f177affd640 1 -- 172.21.15.26:0/3114891500 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 65) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f178000bcf0 con 0x7f178c15cca0 2023-12-19T17:01:07.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.845+0000 7f177affd640 1 -- 172.21.15.26:0/3114891500 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7185+0+0 (secure 0 0 0) 0x7f17800836a0 con 0x7f178c15cca0 2023-12-19T17:01:07.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:07.849+0000 7f177affd640 1 -- 172.21.15.26:0/3114891500 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1780050ee0 con 0x7f178c15cca0 2023-12-19T17:01:08.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.037+0000 7f178b59e640 1 -- 172.21.15.26:0/3114891500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f178c06a880 con 0x7f178c15cca0 2023-12-19T17:01:08.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.037+0000 7f177affd640 1 -- 172.21.15.26:0/3114891500 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f1780054500 con 0x7f178c15cca0 2023-12-19T17:01:08.043 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:08.043 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6884,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1869411616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1869411616},{"type":"v1","addr":"172.21.15.26:6835","nonce":1869411616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3129570713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3129570713},{"type":"v1","addr":"172.21.15.160:6836","nonce":3129570713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4063618989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":4063618989},{"type":"v1","addr":"172.21.15.160:6837","nonce":4063618989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:00:01.425222+0000","modified":"2023-12-19T17:00:04.444345+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7974":{"gid":7974,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/2837068623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2837068623},{"type":"v1","addr":"172.21.15.26:6834","nonce":2837068623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T17:01:08.043 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 57 2023-12-19T17:01:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.041+0000 7f178b59e640 1 -- 172.21.15.26:0/3114891500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178c15cca0 msgr2=0x7f178c15aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.041+0000 7f178b59e640 1 --2- 172.21.15.26:0/3114891500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178c15cca0 0x7f178c15aef0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f178000d800 tx=0x7f178000dcc0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.041+0000 7f178b59e640 1 -- 172.21.15.26:0/3114891500 shutdown_connections 2023-12-19T17:01:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.041+0000 7f178b59e640 1 --2- 172.21.15.26:0/3114891500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f178c15b430 0x7f178c1537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.041+0000 7f178b59e640 1 --2- 172.21.15.26:0/3114891500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f178c15cca0 0x7f178c15aef0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.041+0000 7f178b59e640 1 --2- 172.21.15.26:0/3114891500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f178c0b8760 0x7f178c153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.041+0000 7f178b59e640 1 -- 172.21.15.26:0/3114891500 >> 172.21.15.26:0/3114891500 conn(0x7f178c06a120 msgr2=0x7f178c0b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.041+0000 7f178b59e640 1 -- 172.21.15.26:0/3114891500 shutdown_connections 2023-12-19T17:01:08.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.041+0000 7f178b59e640 1 -- 172.21.15.26:0/3114891500 wait complete. 2023-12-19T17:01:08.064 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:01:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.245+0000 7fcd365a6640 1 Processor -- start 2023-12-19T17:01:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.245+0000 7fcd365a6640 1 -- start start 2023-12-19T17:01:08.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.245+0000 7fcd365a6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcd3006b3d0 0x7fcd300ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd365a6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3006bc30 con 0x7fcd30154220 2023-12-19T17:01:08.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd365a6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd300cc6e0 con 0x7fcd300cec40 2023-12-19T17:01:08.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd365a6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd300cf0d0 con 0x7fcd3006b3d0 2023-12-19T17:01:08.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcd30154220 0x7fcd3015aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41472/0 (socket says 172.21.15.26:41472) 2023-12-19T17:01:08.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2ffff640 1 -- 172.21.15.26:0/92660877 learned_addr learned my addr 172.21.15.26:0/92660877 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:08.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2f7fe640 1 --2- 172.21.15.26:0/92660877 >> v2:172.21.15.160:3300/0 conn(0x7fcd3006b3d0 0x7fcd300ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764422792 0 0) 0x7fcd3006bc30 con 0x7fcd30154220 2023-12-19T17:01:08.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd18003530 con 0x7fcd30154220 2023-12-19T17:01:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2692241693 0 0) 0x7fcd18003530 con 0x7fcd30154220 2023-12-19T17:01:08.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 >> v2:172.21.15.160:3300/0 conn(0x7fcd3006b3d0 msgr2=0x7fcd300ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 --2- 172.21.15.26:0/92660877 >> v2:172.21.15.160:3300/0 conn(0x7fcd3006b3d0 0x7fcd300ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 >> v1:172.21.15.160:6789/0 conn(0x7fcd300cec40 legacy=0x7fcd3015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3006b950 con 0x7fcd30154220 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fcd20001680 con 0x7fcd30154220 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 >> v1:172.21.15.26:6789/0 conn(0x7fcd30154220 legacy=0x7fcd3015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 --2- 172.21.15.26:0/92660877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd18003980 0x7fcd18003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 --2- 172.21.15.26:0/92660877 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd18004380 0x7fcd18004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 --2- 172.21.15.26:0/92660877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd18005050 0x7fcd1800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300cf0d0 con 0x7fcd18003980 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1800bfb0 con 0x7fcd18005050 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1800c170 con 0x7fcd18004380 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2ffff640 1 --2- 172.21.15.26:0/92660877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd18003980 0x7fcd18003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2f7fe640 1 --2- 172.21.15.26:0/92660877 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd18004380 0x7fcd18004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd34b43640 1 --2- 172.21.15.26:0/92660877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd18005050 0x7fcd1800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2ffff640 1 -- 172.21.15.26:0/92660877 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd18004380 msgr2=0x7fcd18004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2ffff640 1 --2- 172.21.15.26:0/92660877 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd18004380 0x7fcd18004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2ffff640 1 -- 172.21.15.26:0/92660877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd18005050 msgr2=0x7fcd1800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2ffff640 1 --2- 172.21.15.26:0/92660877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd18005050 0x7fcd1800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd2ffff640 1 -- 172.21.15.26:0/92660877 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd18003530 con 0x7fcd18003980 2023-12-19T17:01:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd34b43640 1 --2- 172.21.15.26:0/92660877 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd18005050 0x7fcd1800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.249+0000 7fcd34b43640 1 -- 172.21.15.26:0/92660877 reap_dead start 2023-12-19T17:01:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.253+0000 7fcd2ffff640 1 --2- 172.21.15.26:0/92660877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd18003980 0x7fcd18003d60 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c190 tx=0x7fcd2001cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0cee236d9bc07c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.253+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20005ae0 con 0x7fcd18003980 2023-12-19T17:01:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.253+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd20005c60 con 0x7fcd18003980 2023-12-19T17:01:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.253+0000 7fcd2effd640 1 -- 172.21.15.26:0/92660877 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd200a59e0 con 0x7fcd18003980 2023-12-19T17:01:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.253+0000 7fcd365a6640 1 -- 172.21.15.26:0/92660877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd18003980 msgr2=0x7fcd18003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.253+0000 7fcd365a6640 1 --2- 172.21.15.26:0/92660877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd18003980 0x7fcd18003d60 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c190 tx=0x7fcd2001cbb0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.253+0000 7fcd365a6640 1 -- 172.21.15.26:0/92660877 shutdown_connections 2023-12-19T17:01:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.253+0000 7fcd365a6640 1 --2- 172.21.15.26:0/92660877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd18003980 0x7fcd18003d60 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.253+0000 7fcd365a6640 1 -- 172.21.15.26:0/92660877 >> 172.21.15.26:0/92660877 conn(0x7fcd300658e0 msgr2=0x7fcd30065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.253+0000 7fcd365a6640 1 -- 172.21.15.26:0/92660877 shutdown_connections 2023-12-19T17:01:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.253+0000 7fcd365a6640 1 -- 172.21.15.26:0/92660877 wait complete. 2023-12-19T17:01:08.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.253+0000 7fcd365a6640 1 Processor -- start 2023-12-19T17:01:08.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.253+0000 7fcd365a6640 1 -- start start 2023-12-19T17:01:08.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd365a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd18003980 0x7fcd3013a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd365a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd300a8440 0x7fcd300a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd365a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd30147fb0 0x7fcd30148430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd365a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd30148970 con 0x7fcd300a8440 2023-12-19T17:01:08.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd365a6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3014b400 con 0x7fcd18003980 2023-12-19T17:01:08.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd365a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300af140 con 0x7fcd30147fb0 2023-12-19T17:01:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd300a8440 0x7fcd300a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd300a8440 0x7fcd300a88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60436/0 (socket says 172.21.15.26:60436) 2023-12-19T17:01:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2f7fe640 1 -- 172.21.15.26:0/664106528 learned_addr learned my addr 172.21.15.26:0/664106528 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2ffff640 1 --2- 172.21.15.26:0/664106528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd18003980 0x7fcd3013a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd34b43640 1 --2- 172.21.15.26:0/664106528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd30147fb0 0x7fcd30148430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2f7fe640 1 -- 172.21.15.26:0/664106528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd30147fb0 msgr2=0x7fcd30148430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2f7fe640 1 --2- 172.21.15.26:0/664106528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd30147fb0 0x7fcd30148430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2f7fe640 1 -- 172.21.15.26:0/664106528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd18003980 msgr2=0x7fcd3013a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2f7fe640 1 --2- 172.21.15.26:0/664106528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd18003980 0x7fcd3013a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2f7fe640 1 -- 172.21.15.26:0/664106528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3015b010 con 0x7fcd300a8440 2023-12-19T17:01:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd34b43640 1 --2- 172.21.15.26:0/664106528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd30147fb0 0x7fcd30148430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2ffff640 1 --2- 172.21.15.26:0/664106528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd18003980 0x7fcd3013a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2f7fe640 1 --2- 172.21.15.26:0/664106528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd300a8440 0x7fcd300a88c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fcd24011aa0 tx=0x7fcd24011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2cff9640 1 -- 172.21.15.26:0/664106528 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2400fb80 con 0x7fcd300a8440 2023-12-19T17:01:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2cff9640 1 -- 172.21.15.26:0/664106528 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2400fce0 con 0x7fcd300a8440 2023-12-19T17:01:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd365a6640 1 -- 172.21.15.26:0/664106528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd300af420 con 0x7fcd300a8440 2023-12-19T17:01:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd2cff9640 1 -- 172.21.15.26:0/664106528 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24014680 con 0x7fcd300a8440 2023-12-19T17:01:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd365a6640 1 -- 172.21.15.26:0/664106528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3014b6a0 con 0x7fcd300a8440 2023-12-19T17:01:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.257+0000 7fcd365a6640 1 -- 172.21.15.26:0/664106528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd30074270 con 0x7fcd300a8440 2023-12-19T17:01:08.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.261+0000 7fcd2cff9640 1 -- 172.21.15.26:0/664106528 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 65) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fcd2401d400 con 0x7fcd300a8440 2023-12-19T17:01:08.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.261+0000 7fcd2cff9640 1 -- 172.21.15.26:0/664106528 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7185+0+0 (secure 0 0 0) 0x7fcd24083b10 con 0x7fcd300a8440 2023-12-19T17:01:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.265+0000 7fcd2cff9640 1 -- 172.21.15.26:0/664106528 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcd24051260 con 0x7fcd300a8440 2023-12-19T17:01:08.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.425+0000 7fcd365a6640 1 -- 172.21.15.26:0/664106528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd300ce270 con 0x7fcd300a8440 2023-12-19T17:01:08.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.425+0000 7fcd2cff9640 1 -- 172.21.15.26:0/664106528 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+18267 (secure 0 0 0) 0x7fcd24054880 con 0x7fcd300a8440 2023-12-19T17:01:08.431 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:08.431 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":125,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:01:07.441391+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:59:59.967175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T17:00:00.420191+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":125,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1860853944":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/338696599":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/2051635732":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/2964407315":"2023-12-20T17:01:07.441331+0000","172.21.15.160:6833/38098":"2023-12-20T17:00:13.374887+0000","172.21.15.160:6832/38098":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/4135162200":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/378405713":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/885849037":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/520438069":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/3653397836":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/2978385192":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/2885865829":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/3723046945":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/1687941241":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/3946479365":"2023-12-20T17:00:13.374887+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:01:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.429+0000 7fcd365a6640 1 -- 172.21.15.26:0/664106528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd300a8440 msgr2=0x7fcd300a88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.429+0000 7fcd365a6640 1 --2- 172.21.15.26:0/664106528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd300a8440 0x7fcd300a88c0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fcd24011aa0 tx=0x7fcd24011f60 comp rx=0 tx=0).stop 2023-12-19T17:01:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.429+0000 7fcd365a6640 1 -- 172.21.15.26:0/664106528 shutdown_connections 2023-12-19T17:01:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.429+0000 7fcd365a6640 1 --2- 172.21.15.26:0/664106528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd30147fb0 0x7fcd30148430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.429+0000 7fcd365a6640 1 --2- 172.21.15.26:0/664106528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd300a8440 0x7fcd300a88c0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.429+0000 7fcd365a6640 1 --2- 172.21.15.26:0/664106528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd18003980 0x7fcd3013a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.429+0000 7fcd365a6640 1 -- 172.21.15.26:0/664106528 >> 172.21.15.26:0/664106528 conn(0x7fcd300658e0 msgr2=0x7fcd300cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.429+0000 7fcd365a6640 1 -- 172.21.15.26:0/664106528 shutdown_connections 2023-12-19T17:01:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.429+0000 7fcd365a6640 1 -- 172.21.15.26:0/664106528 wait complete. 2023-12-19T17:01:08.455 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T17:01:08.455 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:01:08.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.593+0000 7efeaa1a2640 1 Processor -- start 2023-12-19T17:01:08.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.593+0000 7efeaa1a2640 1 -- start start 2023-12-19T17:01:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.597+0000 7efeaa1a2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efea4164f00 0x7efea40cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.597+0000 7efeaa1a2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea40ade10 con 0x7efea40cb3a0 2023-12-19T17:01:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.597+0000 7efeaa1a2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea40cc9d0 con 0x7efea40ada30 2023-12-19T17:01:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.597+0000 7efeaa1a2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea40b16e0 con 0x7efea4164f00 2023-12-19T17:01:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.597+0000 7efea899f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efea4164f00 0x7efea40cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.597+0000 7efea99a1640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efea40cb3a0 0x7efea40cce60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41482/0 (socket says 172.21.15.26:41482) 2023-12-19T17:01:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.597+0000 7efea99a1640 1 -- 172.21.15.26:0/705249158 learned_addr learned my addr 172.21.15.26:0/705249158 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.597+0000 7efea899f640 1 -- 172.21.15.26:0/705249158 >> v1:172.21.15.160:6789/0 conn(0x7efea40ada30 legacy=0x7efea40c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.597+0000 7efea899f640 1 -- 172.21.15.26:0/705249158 >> v1:172.21.15.26:6789/0 conn(0x7efea40cb3a0 legacy=0x7efea40cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.597+0000 7efea899f640 1 -- 172.21.15.26:0/705249158 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea4161830 con 0x7efea4164f00 2023-12-19T17:01:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.597+0000 7efe9bfff640 1 -- 172.21.15.26:0/705249158 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048514205 0 0) 0x7efea40ade10 con 0x7efea40cb3a0 2023-12-19T17:01:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efea899f640 1 --2- 172.21.15.26:0/705249158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea4164f00 0x7efea40cae60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7efe940097c0 tx=0x7efe9401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3816e609ee9154e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efe9bfff640 1 -- 172.21.15.26:0/705249158 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe940ab640 con 0x7efea4164f00 2023-12-19T17:01:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efe9bfff640 1 -- 172.21.15.26:0/705249158 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe940a7070 con 0x7efea4164f00 2023-12-19T17:01:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efe9bfff640 1 -- 172.21.15.26:0/705249158 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe940ab9c0 con 0x7efea4164f00 2023-12-19T17:01:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 -- 172.21.15.26:0/705249158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea4164f00 msgr2=0x7efea40cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 --2- 172.21.15.26:0/705249158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea4164f00 0x7efea40cae60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7efe940097c0 tx=0x7efe9401c440 comp rx=0 tx=0).stop 2023-12-19T17:01:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 -- 172.21.15.26:0/705249158 shutdown_connections 2023-12-19T17:01:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 --2- 172.21.15.26:0/705249158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea4164f00 0x7efea40cae60 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 -- 172.21.15.26:0/705249158 >> 172.21.15.26:0/705249158 conn(0x7efea406b0f0 msgr2=0x7efea406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 -- 172.21.15.26:0/705249158 shutdown_connections 2023-12-19T17:01:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 -- 172.21.15.26:0/705249158 wait complete. 2023-12-19T17:01:08.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 Processor -- start 2023-12-19T17:01:08.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 -- start start 2023-12-19T17:01:08.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efea40ada30 0x7efea415bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea40cb3a0 0x7efea4161f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efea4164f00 0x7efea40af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea415c410 con 0x7efea40cb3a0 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea40c2fb0 con 0x7efea40ada30 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efeaa1a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea40c3290 con 0x7efea4164f00 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efea899f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea40cb3a0 0x7efea4161f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efea899f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea40cb3a0 0x7efea4161f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60456/0 (socket says 172.21.15.26:60456) 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.601+0000 7efea899f640 1 -- 172.21.15.26:0/3751987332 learned_addr learned my addr 172.21.15.26:0/3751987332 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efea91a0640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efea40ada30 0x7efea415bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efea99a1640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efea4164f00 0x7efea40af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efea899f640 1 -- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efea4164f00 msgr2=0x7efea40af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efea899f640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efea4164f00 0x7efea40af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efea899f640 1 -- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efea40ada30 msgr2=0x7efea415bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efea899f640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efea40ada30 0x7efea415bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efea899f640 1 -- 172.21.15.26:0/3751987332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94009420 con 0x7efea40cb3a0 2023-12-19T17:01:08.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efea99a1640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efea4164f00 0x7efea40af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:08.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efea91a0640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efea40ada30 0x7efea415bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:08.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efea899f640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea40cb3a0 0x7efea4161f10 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7efe940097c0 tx=0x7efe9401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:08.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efe99ffb640 1 -- 172.21.15.26:0/3751987332 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94004030 con 0x7efea40cb3a0 2023-12-19T17:01:08.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efe99ffb640 1 -- 172.21.15.26:0/3751987332 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe940b9070 con 0x7efea40cb3a0 2023-12-19T17:01:08.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efeaa1a2640 1 -- 172.21.15.26:0/3751987332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea40c3570 con 0x7efea40cb3a0 2023-12-19T17:01:08.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efe99ffb640 1 -- 172.21.15.26:0/3751987332 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94004390 con 0x7efea40cb3a0 2023-12-19T17:01:08.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efeaa1a2640 1 -- 172.21.15.26:0/3751987332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea40c3b30 con 0x7efea40cb3a0 2023-12-19T17:01:08.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efeaa1a2640 1 -- 172.21.15.26:0/3751987332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea40c3d20 con 0x7efea40cb3a0 2023-12-19T17:01:08.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.605+0000 7efe99ffb640 1 -- 172.21.15.26:0/3751987332 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 66) v1 ==== 85066+0+0 (secure 0 0 0) 0x7efe94003620 con 0x7efea40cb3a0 2023-12-19T17:01:08.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.609+0000 7efe99ffb640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe880658b0 0x7efe88067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.609+0000 7efe99ffb640 1 -- 172.21.15.26:0/3751987332 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7185+0+0 (secure 0 0 0) 0x7efe940bb070 con 0x7efea40cb3a0 2023-12-19T17:01:08.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.609+0000 7efea91a0640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe880658b0 0x7efe88067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.609+0000 7efea91a0640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe880658b0 0x7efe88067d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efea0003ec0 tx=0x7efea0009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:08.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.613+0000 7efe99ffb640 1 -- 172.21.15.26:0/3751987332 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efe940eb590 con 0x7efea40cb3a0 2023-12-19T17:01:08.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.805+0000 7efeaa1a2640 1 -- 172.21.15.26:0/3751987332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efea4067400 con 0x7efea40cb3a0 2023-12-19T17:01:08.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.809+0000 7efe99ffb640 1 -- 172.21.15.26:0/3751987332 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4717 (secure 0 0 0) 0x7efe940044f0 con 0x7efea40cb3a0 2023-12-19T17:01:08.813 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:08.813 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6884,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1869411616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1869411616},{"type":"v1","addr":"172.21.15.26:6835","nonce":1869411616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3129570713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3129570713},{"type":"v1","addr":"172.21.15.160:6836","nonce":3129570713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4063618989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":4063618989},{"type":"v1","addr":"172.21.15.160:6837","nonce":4063618989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:00:01.425222+0000","modified":"2023-12-19T17:00:04.444345+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7974":{"gid":7974,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/2837068623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2837068623},{"type":"v1","addr":"172.21.15.26:6834","nonce":2837068623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T17:01:08.814 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 57 2023-12-19T17:01:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.809+0000 7efeaa1a2640 1 -- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe880658b0 msgr2=0x7efe88067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.809+0000 7efeaa1a2640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe880658b0 0x7efe88067d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efea0003ec0 tx=0x7efea0009040 comp rx=0 tx=0).stop 2023-12-19T17:01:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.809+0000 7efeaa1a2640 1 -- 172.21.15.26:0/3751987332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea40cb3a0 msgr2=0x7efea4161f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.809+0000 7efeaa1a2640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea40cb3a0 0x7efea4161f10 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7efe940097c0 tx=0x7efe9401ac80 comp rx=0 tx=0).stop 2023-12-19T17:01:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.809+0000 7efeaa1a2640 1 -- 172.21.15.26:0/3751987332 shutdown_connections 2023-12-19T17:01:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.809+0000 7efeaa1a2640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe880658b0 0x7efe88067d70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.809+0000 7efeaa1a2640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efea4164f00 0x7efea40af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.809+0000 7efeaa1a2640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efea40cb3a0 0x7efea4161f10 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.809+0000 7efeaa1a2640 1 --2- 172.21.15.26:0/3751987332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efea40ada30 0x7efea415bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.809+0000 7efeaa1a2640 1 -- 172.21.15.26:0/3751987332 >> 172.21.15.26:0/3751987332 conn(0x7efea406b0f0 msgr2=0x7efea406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.809+0000 7efeaa1a2640 1 -- 172.21.15.26:0/3751987332 shutdown_connections 2023-12-19T17:01:08.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.809+0000 7efeaa1a2640 1 -- 172.21.15.26:0/3751987332 wait complete. 2023-12-19T17:01:08.833 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:01:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e83004640 1 Processor -- start 2023-12-19T17:01:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e83004640 1 -- start start 2023-12-19T17:01:08.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e83004640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e7c15cca0 0x7f6e7c0b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e83004640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6e7c0b6b20 0x7f6e7c0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e83004640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c0b6fb0 con 0x7f6e7c0b8760 2023-12-19T17:01:08.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e83004640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c0b7c30 con 0x7f6e7c0b6b20 2023-12-19T17:01:08.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e83004640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0b7290 con 0x7f6e7c15cca0 2023-12-19T17:01:08.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e82002640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e7c15cca0 0x7f6e7c0b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e81801640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6e7c0b8760 0x7f6e7c0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41490/0 (socket says 172.21.15.26:41490) 2023-12-19T17:01:08.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e82002640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e7c15cca0 0x7f6e7c0b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:57290/0 (socket says 172.21.15.26:57290) 2023-12-19T17:01:08.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e81801640 1 -- 172.21.15.26:0/2733236658 learned_addr learned my addr 172.21.15.26:0/2733236658 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:08.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e82803640 1 --2- 172.21.15.26:0/2733236658 >> v2:172.21.15.160:3300/0 conn(0x7f6e7c0b6b20 0x7f6e7c0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e81000640 1 -- 172.21.15.26:0/2733236658 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400741703 0 0) 0x7f6e7c0b6fb0 con 0x7f6e7c0b8760 2023-12-19T17:01:08.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e82002640 1 -- 172.21.15.26:0/2733236658 >> v2:172.21.15.160:3300/0 conn(0x7f6e7c0b6b20 msgr2=0x7f6e7c0d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:08.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e82002640 1 --2- 172.21.15.26:0/2733236658 >> v2:172.21.15.160:3300/0 conn(0x7f6e7c0b6b20 0x7f6e7c0d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e82002640 1 -- 172.21.15.26:0/2733236658 >> v1:172.21.15.26:6789/0 conn(0x7f6e7c0b8760 legacy=0x7f6e7c0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.973+0000 7f6e82002640 1 -- 172.21.15.26:0/2733236658 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c0b7570 con 0x7f6e7c15cca0 2023-12-19T17:01:08.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.977+0000 7f6e82002640 1 --2- 172.21.15.26:0/2733236658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e7c15cca0 0x7f6e7c0b65e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e780097c0 tx=0x7f6e7801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f281a0e6eed282a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:08.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.977+0000 7f6e81000640 1 -- 172.21.15.26:0/2733236658 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e780ab670 con 0x7f6e7c15cca0 2023-12-19T17:01:08.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.977+0000 7f6e81000640 1 -- 172.21.15.26:0/2733236658 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e780ab7d0 con 0x7f6e7c15cca0 2023-12-19T17:01:08.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.977+0000 7f6e81000640 1 -- 172.21.15.26:0/2733236658 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e780abb60 con 0x7f6e7c15cca0 2023-12-19T17:01:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.977+0000 7f6e83004640 1 -- 172.21.15.26:0/2733236658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e7c15cca0 msgr2=0x7f6e7c0b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.977+0000 7f6e83004640 1 --2- 172.21.15.26:0/2733236658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e7c15cca0 0x7f6e7c0b65e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e780097c0 tx=0x7f6e7801c450 comp rx=0 tx=0).stop 2023-12-19T17:01:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 -- 172.21.15.26:0/2733236658 shutdown_connections 2023-12-19T17:01:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 --2- 172.21.15.26:0/2733236658 >> v2:172.21.15.160:3300/0 conn(0x7f6e7c0b6b20 0x7f6e7c0d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 --2- 172.21.15.26:0/2733236658 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e7c15cca0 0x7f6e7c0b65e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 -- 172.21.15.26:0/2733236658 >> 172.21.15.26:0/2733236658 conn(0x7f6e7c06a120 msgr2=0x7f6e7c06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 -- 172.21.15.26:0/2733236658 shutdown_connections 2023-12-19T17:01:08.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 -- 172.21.15.26:0/2733236658 wait complete. 2023-12-19T17:01:08.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 Processor -- start 2023-12-19T17:01:08.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 -- start start 2023-12-19T17:01:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e7c0b8760 0x7f6e7c0ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c15cca0 0x7f6e7c0c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e7c0c6840 0x7f6e7c15a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0cab30 con 0x7f6e7c15cca0 2023-12-19T17:01:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0c70c0 con 0x7f6e7c0b8760 2023-12-19T17:01:08.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e83004640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0c6de0 con 0x7f6e7c0c6840 2023-12-19T17:01:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e81801640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c15cca0 0x7f6e7c0c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e81801640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c15cca0 0x7f6e7c0c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60460/0 (socket says 172.21.15.26:60460) 2023-12-19T17:01:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e81801640 1 -- 172.21.15.26:0/3346903230 learned_addr learned my addr 172.21.15.26:0/3346903230 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e82803640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e7c0c6840 0x7f6e7c15a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e82002640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e7c0b8760 0x7f6e7c0ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e81801640 1 -- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e7c0c6840 msgr2=0x7f6e7c15a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e81801640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e7c0c6840 0x7f6e7c15a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e81801640 1 -- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e7c0b8760 msgr2=0x7f6e7c0ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e81801640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e7c0b8760 0x7f6e7c0ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:08.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.981+0000 7f6e81801640 1 -- 172.21.15.26:0/3346903230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e78009420 con 0x7f6e7c15cca0 2023-12-19T17:01:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.985+0000 7f6e81801640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c15cca0 0x7f6e7c0c6300 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0011f0 tx=0x7f6e6c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.985+0000 7f6e727fc640 1 -- 172.21.15.26:0/3346903230 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c0074e0 con 0x7f6e7c15cca0 2023-12-19T17:01:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.985+0000 7f6e727fc640 1 -- 172.21.15.26:0/3346903230 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6c007640 con 0x7f6e7c15cca0 2023-12-19T17:01:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.985+0000 7f6e727fc640 1 -- 172.21.15.26:0/3346903230 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c004a10 con 0x7f6e7c15cca0 2023-12-19T17:01:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.985+0000 7f6e83004640 1 -- 172.21.15.26:0/3346903230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c0c9b10 con 0x7f6e7c15cca0 2023-12-19T17:01:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.985+0000 7f6e83004640 1 -- 172.21.15.26:0/3346903230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c0ca0d0 con 0x7f6e7c15cca0 2023-12-19T17:01:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.985+0000 7f6e83004640 1 -- 172.21.15.26:0/3346903230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c0ca2c0 con 0x7f6e7c15cca0 2023-12-19T17:01:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.985+0000 7f6e727fc640 1 -- 172.21.15.26:0/3346903230 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 66) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f6e6c004b70 con 0x7f6e7c15cca0 2023-12-19T17:01:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.985+0000 7f6e727fc640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6e600658b0 0x7f6e60067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.985+0000 7f6e727fc640 1 -- 172.21.15.26:0/3346903230 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7185+0+0 (secure 0 0 0) 0x7f6e6c0825e0 con 0x7f6e7c15cca0 2023-12-19T17:01:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.985+0000 7f6e82002640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6e600658b0 0x7f6e60067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.989+0000 7f6e82002640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6e600658b0 0x7f6e60067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6e780af000 tx=0x7f6e780a7000 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:08.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:08.989+0000 7f6e727fc640 1 -- 172.21.15.26:0/3346903230 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6e6c04fdc0 con 0x7f6e7c15cca0 2023-12-19T17:01:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.181+0000 7f6e83004640 1 -- 172.21.15.26:0/3346903230 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e7c067400 con 0x7f6e7c15cca0 2023-12-19T17:01:09.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.181+0000 7f6e727fc640 1 -- 172.21.15.26:0/3346903230 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f6e6c018020 con 0x7f6e7c15cca0 2023-12-19T17:01:09.188 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:09.188 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6884,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1869411616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1869411616},{"type":"v1","addr":"172.21.15.26:6835","nonce":1869411616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3129570713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3129570713},{"type":"v1","addr":"172.21.15.160:6836","nonce":3129570713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4063618989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":4063618989},{"type":"v1","addr":"172.21.15.160:6837","nonce":4063618989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:00:01.425222+0000","modified":"2023-12-19T17:00:04.444345+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7974":{"gid":7974,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/2837068623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2837068623},{"type":"v1","addr":"172.21.15.26:6834","nonce":2837068623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T17:01:09.188 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 57 2023-12-19T17:01:09.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.185+0000 7f6e83004640 1 -- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6e600658b0 msgr2=0x7f6e60067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.185+0000 7f6e83004640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6e600658b0 0x7f6e60067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6e780af000 tx=0x7f6e780a7000 comp rx=0 tx=0).stop 2023-12-19T17:01:09.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.185+0000 7f6e83004640 1 -- 172.21.15.26:0/3346903230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c15cca0 msgr2=0x7f6e7c0c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.185+0000 7f6e83004640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c15cca0 0x7f6e7c0c6300 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0011f0 tx=0x7f6e6c00c9b0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.185+0000 7f6e83004640 1 -- 172.21.15.26:0/3346903230 shutdown_connections 2023-12-19T17:01:09.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.185+0000 7f6e83004640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6e600658b0 0x7f6e60067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.185+0000 7f6e83004640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e7c0c6840 0x7f6e7c15a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.185+0000 7f6e83004640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c15cca0 0x7f6e7c0c6300 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.185+0000 7f6e83004640 1 --2- 172.21.15.26:0/3346903230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e7c0b8760 0x7f6e7c0ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.185+0000 7f6e83004640 1 -- 172.21.15.26:0/3346903230 >> 172.21.15.26:0/3346903230 conn(0x7f6e7c06a120 msgr2=0x7f6e7c0d1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:09.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.185+0000 7f6e83004640 1 -- 172.21.15.26:0/3346903230 shutdown_connections 2023-12-19T17:01:09.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.185+0000 7f6e83004640 1 -- 172.21.15.26:0/3346903230 wait complete. 2023-12-19T17:01:09.208 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:01:09.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.345+0000 7f20927f3640 1 Processor -- start 2023-12-19T17:01:09.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.345+0000 7f20927f3640 1 -- start start 2023-12-19T17:01:09.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f20927f3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f208c154220 0x7f208c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:09.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f20927f3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f208c06b3d0 0x7f208c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:09.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f20927f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c0cec40 0x7f208c15d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:09.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f20927f3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208c0cf0d0 con 0x7f208c154220 2023-12-19T17:01:09.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f20927f3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208c06bc30 con 0x7f208c06b3d0 2023-12-19T17:01:09.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f20927f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0cc6e0 con 0x7f208c0cec40 2023-12-19T17:01:09.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f208bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f208c154220 0x7f208c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:09.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f208bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f208c154220 0x7f208c15aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60474/0 (socket says 172.21.15.26:60474) 2023-12-19T17:01:09.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f208bfff640 1 -- 172.21.15.26:0/1144050900 learned_addr learned my addr 172.21.15.26:0/1144050900 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:09.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f208b7fe640 1 --2- 172.21.15.26:0/1144050900 >> v2:172.21.15.160:3300/0 conn(0x7f208c06b3d0 0x7f208c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:09.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f2090d90640 1 --2- 172.21.15.26:0/1144050900 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c0cec40 0x7f208c15d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:09.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f208bfff640 1 -- 172.21.15.26:0/1144050900 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c0cec40 msgr2=0x7f208c15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:09.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f208bfff640 1 --2- 172.21.15.26:0/1144050900 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c0cec40 0x7f208c15d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f208bfff640 1 -- 172.21.15.26:0/1144050900 >> v2:172.21.15.160:3300/0 conn(0x7f208c06b3d0 msgr2=0x7f208c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:09.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f208bfff640 1 --2- 172.21.15.26:0/1144050900 >> v2:172.21.15.160:3300/0 conn(0x7f208c06b3d0 0x7f208c0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f208bfff640 1 -- 172.21.15.26:0/1144050900 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c15afc0 con 0x7f208c154220 2023-12-19T17:01:09.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.349+0000 7f2090d90640 1 --2- 172.21.15.26:0/1144050900 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c0cec40 0x7f208c15d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:09.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f208bfff640 1 --2- 172.21.15.26:0/1144050900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c154220 0x7f208c15aa80 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f208c0cf880 tx=0x7f207c01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98888ee278999cfa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:09.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f208affd640 1 -- 172.21.15.26:0/1144050900 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c0ab600 con 0x7f208c154220 2023-12-19T17:01:09.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f208affd640 1 -- 172.21.15.26:0/1144050900 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c0a7050 con 0x7f208c154220 2023-12-19T17:01:09.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f208affd640 1 -- 172.21.15.26:0/1144050900 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c0ab980 con 0x7f208c154220 2023-12-19T17:01:09.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f20927f3640 1 -- 172.21.15.26:0/1144050900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c154220 msgr2=0x7f208c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f20927f3640 1 --2- 172.21.15.26:0/1144050900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c154220 0x7f208c15aa80 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f208c0cf880 tx=0x7f207c01c210 comp rx=0 tx=0).stop 2023-12-19T17:01:09.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f20927f3640 1 -- 172.21.15.26:0/1144050900 shutdown_connections 2023-12-19T17:01:09.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f20927f3640 1 --2- 172.21.15.26:0/1144050900 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c0cec40 0x7f208c15d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f20927f3640 1 --2- 172.21.15.26:0/1144050900 >> v2:172.21.15.160:3300/0 conn(0x7f208c06b3d0 0x7f208c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f20927f3640 1 --2- 172.21.15.26:0/1144050900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c154220 0x7f208c15aa80 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f20927f3640 1 -- 172.21.15.26:0/1144050900 >> 172.21.15.26:0/1144050900 conn(0x7f208c0658e0 msgr2=0x7f208c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:09.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f20927f3640 1 -- 172.21.15.26:0/1144050900 shutdown_connections 2023-12-19T17:01:09.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f20927f3640 1 -- 172.21.15.26:0/1144050900 wait complete. 2023-12-19T17:01:09.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f20927f3640 1 Processor -- start 2023-12-19T17:01:09.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.353+0000 7f20927f3640 1 -- start start 2023-12-19T17:01:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f20927f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b3d0 0x7f208c152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f20927f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c0cec40 0x7f208c14c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f20927f3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f208c154220 0x7f208c1f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f20927f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c14c910 con 0x7f208c06b3d0 2023-12-19T17:01:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f20927f3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0b1bf0 con 0x7f208c154220 2023-12-19T17:01:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f20927f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0b1ed0 con 0x7f208c0cec40 2023-12-19T17:01:09.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f208bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b3d0 0x7f208c152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:09.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f208bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b3d0 0x7f208c152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60490/0 (socket says 172.21.15.26:60490) 2023-12-19T17:01:09.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f208bfff640 1 -- 172.21.15.26:0/791102496 learned_addr learned my addr 172.21.15.26:0/791102496 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:09.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f208b7fe640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c0cec40 0x7f208c14c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:09.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f2090d90640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f208c154220 0x7f208c1f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:09.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f208bfff640 1 -- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c0cec40 msgr2=0x7f208c14c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f208bfff640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c0cec40 0x7f208c14c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f208bfff640 1 -- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f208c154220 msgr2=0x7f208c1f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f208bfff640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f208c154220 0x7f208c1f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f208bfff640 1 -- 172.21.15.26:0/791102496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c0094d0 con 0x7f208c06b3d0 2023-12-19T17:01:09.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f208bfff640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b3d0 0x7f208c152d30 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f207c01cab0 tx=0x7f207c01aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:09.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f2088ff9640 1 -- 172.21.15.26:0/791102496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c0b6620 con 0x7f208c06b3d0 2023-12-19T17:01:09.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f2088ff9640 1 -- 172.21.15.26:0/791102496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c0abe90 con 0x7f208c06b3d0 2023-12-19T17:01:09.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f20927f3640 1 -- 172.21.15.26:0/791102496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208c1f8e90 con 0x7f208c06b3d0 2023-12-19T17:01:09.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f2088ff9640 1 -- 172.21.15.26:0/791102496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c0b69e0 con 0x7f208c06b3d0 2023-12-19T17:01:09.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f20927f3640 1 -- 172.21.15.26:0/791102496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208c1f9410 con 0x7f208c06b3d0 2023-12-19T17:01:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f20927f3640 1 -- 172.21.15.26:0/791102496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208c074270 con 0x7f208c06b3d0 2023-12-19T17:01:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.357+0000 7f2088ff9640 1 -- 172.21.15.26:0/791102496 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 66) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f207c0b2040 con 0x7f208c06b3d0 2023-12-19T17:01:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.361+0000 7f2088ff9640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f20780658b0 0x7f2078067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.361+0000 7f2088ff9640 1 -- 172.21.15.26:0/791102496 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7185+0+0 (secure 0 0 0) 0x7f207c11d4f0 con 0x7f208c06b3d0 2023-12-19T17:01:09.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.361+0000 7f208b7fe640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f20780658b0 0x7f2078067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:09.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.361+0000 7f208b7fe640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f20780658b0 0x7f2078067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f208c14ed80 tx=0x7f2080008040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:09.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.365+0000 7f2088ff9640 1 -- 172.21.15.26:0/791102496 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f207c0eac00 con 0x7f208c06b3d0 2023-12-19T17:01:09.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.525+0000 7f20927f3640 1 -- 172.21.15.26:0/791102496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f208c0b3930 con 0x7f208c06b3d0 2023-12-19T17:01:09.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.529+0000 7f2088ff9640 1 -- 172.21.15.26:0/791102496 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+18267 (secure 0 0 0) 0x7f207c0ee220 con 0x7f208c06b3d0 2023-12-19T17:01:09.533 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:09.533 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":125,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:01:07.441391+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T16:59:59.967175+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T17:00:00.420191+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":125,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1860853944":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/338696599":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/2051635732":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/2964407315":"2023-12-20T17:01:07.441331+0000","172.21.15.160:6833/38098":"2023-12-20T17:00:13.374887+0000","172.21.15.160:6832/38098":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/4135162200":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/378405713":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/885849037":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/520438069":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/3653397836":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/2978385192":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/2885865829":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/3723046945":"2023-12-20T17:00:13.374887+0000","172.21.15.160:0/1687941241":"2023-12-20T17:01:07.441331+0000","172.21.15.160:0/3946479365":"2023-12-20T17:00:13.374887+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:01:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.529+0000 7f20927f3640 1 -- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f20780658b0 msgr2=0x7f2078067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.529+0000 7f20927f3640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f20780658b0 0x7f2078067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f208c14ed80 tx=0x7f2080008040 comp rx=0 tx=0).stop 2023-12-19T17:01:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.529+0000 7f20927f3640 1 -- 172.21.15.26:0/791102496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b3d0 msgr2=0x7f208c152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.529+0000 7f20927f3640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b3d0 0x7f208c152d30 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f207c01cab0 tx=0x7f207c01aa50 comp rx=0 tx=0).stop 2023-12-19T17:01:09.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.529+0000 7f20927f3640 1 -- 172.21.15.26:0/791102496 shutdown_connections 2023-12-19T17:01:09.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.529+0000 7f20927f3640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f208c154220 0x7f208c1f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.529+0000 7f20927f3640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f20780658b0 0x7f2078067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.529+0000 7f20927f3640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f208c0cec40 0x7f208c14c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.529+0000 7f20927f3640 1 --2- 172.21.15.26:0/791102496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f208c06b3d0 0x7f208c152d30 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.529+0000 7f20927f3640 1 -- 172.21.15.26:0/791102496 >> 172.21.15.26:0/791102496 conn(0x7f208c0658e0 msgr2=0x7f208c0cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.529+0000 7f20927f3640 1 -- 172.21.15.26:0/791102496 shutdown_connections 2023-12-19T17:01:09.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.529+0000 7f20927f3640 1 -- 172.21.15.26:0/791102496 wait complete. 2023-12-19T17:01:09.554 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T17:01:09.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdd54d7640 1 Processor -- start 2023-12-19T17:01:09.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdd54d7640 1 -- start start 2023-12-19T17:01:09.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdd54d7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdd00cc0a0 0x7efdd00cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:09.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdd54d7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd00adde0 con 0x7efdd0159d20 2023-12-19T17:01:09.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdd54d7640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd00b28d0 con 0x7efdd00cdee0 2023-12-19T17:01:09.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdd54d7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd00c9c20 con 0x7efdd00cc0a0 2023-12-19T17:01:09.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdcf7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efdd0159d20 0x7efdd00cc9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41506/0 (socket says 172.21.15.26:41506) 2023-12-19T17:01:09.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdcf7fe640 1 -- 172.21.15.26:0/834412120 learned_addr learned my addr 172.21.15.26:0/834412120 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:09.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdcffff640 1 --2- 172.21.15.26:0/834412120 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdd00cc0a0 0x7efdd00cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:09.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdceffd640 1 -- 172.21.15.26:0/834412120 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292343537 0 0) 0x7efdd00adde0 con 0x7efdd0159d20 2023-12-19T17:01:09.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdceffd640 1 -- 172.21.15.26:0/834412120 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdb8003530 con 0x7efdd0159d20 2023-12-19T17:01:09.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdceffd640 1 -- 172.21.15.26:0/834412120 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863387938 0 0) 0x7efdd00b28d0 con 0x7efdd00cdee0 2023-12-19T17:01:09.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdceffd640 1 -- 172.21.15.26:0/834412120 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdd00adde0 con 0x7efdd00cdee0 2023-12-19T17:01:09.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdceffd640 1 -- 172.21.15.26:0/834412120 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3290016129 0 0) 0x7efdb8003530 con 0x7efdd0159d20 2023-12-19T17:01:09.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdcffff640 1 -- 172.21.15.26:0/834412120 >> v1:172.21.15.160:6789/0 conn(0x7efdd00cdee0 legacy=0x7efdd00b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdcffff640 1 -- 172.21.15.26:0/834412120 >> v1:172.21.15.26:6789/0 conn(0x7efdd0159d20 legacy=0x7efdd00cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.697+0000 7efdcffff640 1 -- 172.21.15.26:0/834412120 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd00adb00 con 0x7efdd00cc0a0 2023-12-19T17:01:09.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.701+0000 7efdcffff640 1 --2- 172.21.15.26:0/834412120 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdd00cc0a0 0x7efdd00cc480 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efdc00097f0 tx=0x7efdc001c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad8ee8b04c6453bd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:09.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.701+0000 7efdceffd640 1 -- 172.21.15.26:0/834412120 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc4001680 con 0x7efdd00cc0a0 2023-12-19T17:01:09.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.701+0000 7efdceffd640 1 -- 172.21.15.26:0/834412120 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc00a7050 con 0x7efdd00cc0a0 2023-12-19T17:01:09.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.701+0000 7efdceffd640 1 -- 172.21.15.26:0/834412120 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc00ab620 con 0x7efdd00cc0a0 2023-12-19T17:01:09.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.701+0000 7efdd54d7640 1 -- 172.21.15.26:0/834412120 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdd00cc0a0 msgr2=0x7efdd00cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.701+0000 7efdd54d7640 1 --2- 172.21.15.26:0/834412120 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdd00cc0a0 0x7efdd00cc480 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efdc00097f0 tx=0x7efdc001c320 comp rx=0 tx=0).stop 2023-12-19T17:01:09.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.701+0000 7efdd54d7640 1 -- 172.21.15.26:0/834412120 shutdown_connections 2023-12-19T17:01:09.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.701+0000 7efdd54d7640 1 --2- 172.21.15.26:0/834412120 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdd00cc0a0 0x7efdd00cc480 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.701+0000 7efdd54d7640 1 -- 172.21.15.26:0/834412120 >> 172.21.15.26:0/834412120 conn(0x7efdd006b0f0 msgr2=0x7efdd006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:09.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.701+0000 7efdd54d7640 1 -- 172.21.15.26:0/834412120 shutdown_connections 2023-12-19T17:01:09.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.701+0000 7efdd54d7640 1 -- 172.21.15.26:0/834412120 wait complete. 2023-12-19T17:01:09.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdd54d7640 1 Processor -- start 2023-12-19T17:01:09.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdd54d7640 1 -- start start 2023-12-19T17:01:09.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdd54d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd00cc0a0 0x7efdd01f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:09.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdd54d7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdd00cdee0 0x7efdd01f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:09.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdd54d7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdd0159d20 0x7efdd00c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:09.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdd54d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd01f5960 con 0x7efdd00cc0a0 2023-12-19T17:01:09.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdd54d7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd00c7050 con 0x7efdd00cdee0 2023-12-19T17:01:09.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdd54d7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd00c6a90 con 0x7efdd0159d20 2023-12-19T17:01:09.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdcffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd00cc0a0 0x7efdd01f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:09.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdcffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd00cc0a0 0x7efdd01f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60494/0 (socket says 172.21.15.26:60494) 2023-12-19T17:01:09.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdcffff640 1 -- 172.21.15.26:0/3241035506 learned_addr learned my addr 172.21.15.26:0/3241035506 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:09.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdd4cd6640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdd0159d20 0x7efdd00c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:09.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdcffff640 1 -- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdd0159d20 msgr2=0x7efdd00c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdcf7fe640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdd00cdee0 0x7efdd01f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:09.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdcffff640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdd0159d20 0x7efdd00c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdcffff640 1 -- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdd00cdee0 msgr2=0x7efdd01f5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdcffff640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdd00cdee0 0x7efdd01f5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdcffff640 1 -- 172.21.15.26:0/3241035506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc0009420 con 0x7efdd00cc0a0 2023-12-19T17:01:09.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdd4cd6640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdd0159d20 0x7efdd00c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:09.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdcf7fe640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdd00cdee0 0x7efdd01f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:09.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdcffff640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd00cc0a0 0x7efdd01f4f80 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7efdc00ae040 tx=0x7efdc001ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:09.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdccff9640 1 -- 172.21.15.26:0/3241035506 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc00abcd0 con 0x7efdd00cc0a0 2023-12-19T17:01:09.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdccff9640 1 -- 172.21.15.26:0/3241035506 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdc00abe50 con 0x7efdd00cc0a0 2023-12-19T17:01:09.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdd54d7640 1 -- 172.21.15.26:0/3241035506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd00c6d70 con 0x7efdd00cc0a0 2023-12-19T17:01:09.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdccff9640 1 -- 172.21.15.26:0/3241035506 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdc0004260 con 0x7efdd00cc0a0 2023-12-19T17:01:09.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.705+0000 7efdd54d7640 1 -- 172.21.15.26:0/3241035506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd00b11e0 con 0x7efdd00cc0a0 2023-12-19T17:01:09.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.709+0000 7efdd54d7640 1 -- 172.21.15.26:0/3241035506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd0067400 con 0x7efdd00cc0a0 2023-12-19T17:01:09.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.709+0000 7efdccff9640 1 -- 172.21.15.26:0/3241035506 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 67) v1 ==== 85274+0+0 (secure 0 0 0) 0x7efdc00043c0 con 0x7efdd00cc0a0 2023-12-19T17:01:09.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.709+0000 7efdccff9640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efdbc065af0 0x7efdbc067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:09.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.709+0000 7efdccff9640 1 -- 172.21.15.26:0/3241035506 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7185+0+0 (secure 0 0 0) 0x7efdc00a4e40 con 0x7efdd00cc0a0 2023-12-19T17:01:09.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.709+0000 7efdcf7fe640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efdbc065af0 0x7efdbc067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:09.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.709+0000 7efdcf7fe640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efdbc065af0 0x7efdbc067fb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efdd00b3280 tx=0x7efdc4009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:09.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.713+0000 7efdccff9640 1 -- 172.21.15.26:0/3241035506 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efdc00eb380 con 0x7efdd00cc0a0 2023-12-19T17:01:09.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.901+0000 7efdd54d7640 1 -- 172.21.15.26:0/3241035506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efdd006b6e0 con 0x7efdd00cc0a0 2023-12-19T17:01:09.908 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T17:01:09.905+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T17:01:09.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.913+0000 7efdccff9640 1 -- 172.21.15.26:0/3241035506 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7efdc00bfe90 con 0x7efdd00cc0a0 2023-12-19T17:01:09.918 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T17:01:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.917+0000 7efdd54d7640 1 -- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efdbc065af0 msgr2=0x7efdbc067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.917+0000 7efdd54d7640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efdbc065af0 0x7efdbc067fb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efdd00b3280 tx=0x7efdc4009040 comp rx=0 tx=0).stop 2023-12-19T17:01:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.917+0000 7efdd54d7640 1 -- 172.21.15.26:0/3241035506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd00cc0a0 msgr2=0x7efdd01f4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.917+0000 7efdd54d7640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd00cc0a0 0x7efdd01f4f80 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7efdc00ae040 tx=0x7efdc001ab60 comp rx=0 tx=0).stop 2023-12-19T17:01:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.917+0000 7efdd54d7640 1 -- 172.21.15.26:0/3241035506 shutdown_connections 2023-12-19T17:01:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.917+0000 7efdd54d7640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efdbc065af0 0x7efdbc067fb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.917+0000 7efdd54d7640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efdd0159d20 0x7efdd00c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.917+0000 7efdd54d7640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efdd00cdee0 0x7efdd01f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.917+0000 7efdd54d7640 1 --2- 172.21.15.26:0/3241035506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efdd00cc0a0 0x7efdd01f4f80 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:09.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.917+0000 7efdd54d7640 1 -- 172.21.15.26:0/3241035506 >> 172.21.15.26:0/3241035506 conn(0x7efdd006b0f0 msgr2=0x7efdd0154b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.917+0000 7efdd54d7640 1 -- 172.21.15.26:0/3241035506 shutdown_connections 2023-12-19T17:01:09.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:09.917+0000 7efdd54d7640 1 -- 172.21.15.26:0/3241035506 wait complete. 2023-12-19T17:01:09.929 INFO:tasks.ceph.mds.a.smithi026.stderr: -9690> 2023-12-19T17:00:02.923+0000 7fd4a789e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T17:01:09.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T17:01:09.983 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T17:01:10.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.085+0000 7fd8d9d8f640 1 Processor -- start 2023-12-19T17:01:10.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.085+0000 7fd8d9d8f640 1 -- start start 2023-12-19T17:01:10.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d9d8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8d40cb150 0x7fd8d40cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:10.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d9d8f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd8d415e690 0x7fd8d415eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:10.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d9d8f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d40ce3f0 con 0x7fd8d41585f0 2023-12-19T17:01:10.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d9d8f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d40cbb10 con 0x7fd8d415e690 2023-12-19T17:01:10.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d9d8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40b47a0 con 0x7fd8d40cb150 2023-12-19T17:01:10.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d3fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd8d415e690 0x7fd8d415eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:10.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d2ffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8d40cb150 0x7fd8d40cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:10.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d3fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd8d415e690 0x7fd8d415eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:38542/0 (socket says 172.21.15.26:38542) 2023-12-19T17:01:10.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d2ffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8d40cb150 0x7fd8d40cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:57370/0 (socket says 172.21.15.26:57370) 2023-12-19T17:01:10.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d3fff640 1 -- 172.21.15.26:0/2365156796 learned_addr learned my addr 172.21.15.26:0/2365156796 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:10.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d37fe640 1 --1- 172.21.15.26:0/2365156796 >> v1:172.21.15.160:6789/0 conn(0x7fd8d41585f0 0x7fd8d40cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:50546/0 (socket says 172.21.15.26:50546) 2023-12-19T17:01:10.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d2ffd640 1 -- 172.21.15.26:0/2365156796 >> v2:172.21.15.160:3300/0 conn(0x7fd8d415e690 msgr2=0x7fd8d415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:10.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d2ffd640 1 --2- 172.21.15.26:0/2365156796 >> v2:172.21.15.160:3300/0 conn(0x7fd8d415e690 0x7fd8d415eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:10.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d2ffd640 1 -- 172.21.15.26:0/2365156796 >> v1:172.21.15.160:6789/0 conn(0x7fd8d41585f0 legacy=0x7fd8d40cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:10.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d2ffd640 1 -- 172.21.15.26:0/2365156796 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d415b1b0 con 0x7fd8d40cb150 2023-12-19T17:01:10.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.089+0000 7fd8d3fff640 1 --2- 172.21.15.26:0/2365156796 >> v2:172.21.15.160:3300/0 conn(0x7fd8d415e690 0x7fd8d415eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:01:10.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d2ffd640 1 --2- 172.21.15.26:0/2365156796 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8d40cb150 0x7fd8d40cb5d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80097c0 tx=0x7fd8c801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab4b2d6afc22162 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d27fc640 1 -- 172.21.15.26:0/2365156796 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c80ac620 con 0x7fd8d40cb150 2023-12-19T17:01:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d27fc640 1 -- 172.21.15.26:0/2365156796 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c80ac780 con 0x7fd8d40cb150 2023-12-19T17:01:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d27fc640 1 -- 172.21.15.26:0/2365156796 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c80acb10 con 0x7fd8d40cb150 2023-12-19T17:01:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/2365156796 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8d40cb150 msgr2=0x7fd8d40cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d9d8f640 1 --2- 172.21.15.26:0/2365156796 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8d40cb150 0x7fd8d40cb5d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80097c0 tx=0x7fd8c801c3c0 comp rx=0 tx=0).stop 2023-12-19T17:01:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/2365156796 shutdown_connections 2023-12-19T17:01:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d9d8f640 1 --2- 172.21.15.26:0/2365156796 >> v2:172.21.15.160:3300/0 conn(0x7fd8d415e690 0x7fd8d415eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d9d8f640 1 --2- 172.21.15.26:0/2365156796 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8d40cb150 0x7fd8d40cb5d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/2365156796 >> 172.21.15.26:0/2365156796 conn(0x7fd8d4066160 msgr2=0x7fd8d40b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/2365156796 shutdown_connections 2023-12-19T17:01:10.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/2365156796 wait complete. 2023-12-19T17:01:10.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d9d8f640 1 Processor -- start 2023-12-19T17:01:10.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.093+0000 7fd8d9d8f640 1 -- start start 2023-12-19T17:01:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d9d8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8d40cb150 0x7fd8d4151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d9d8f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d41585f0 0x7fd8d40ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d9d8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d415e690 0x7fd8d4159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d9d8f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d4152460 con 0x7fd8d415e690 2023-12-19T17:01:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d9d8f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d415a450 con 0x7fd8d41585f0 2023-12-19T17:01:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d9d8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d415a730 con 0x7fd8d40cb150 2023-12-19T17:01:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d415e690 0x7fd8d4159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d415e690 0x7fd8d4159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60500/0 (socket says 172.21.15.26:60500) 2023-12-19T17:01:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d3fff640 1 -- 172.21.15.26:0/654720455 learned_addr learned my addr 172.21.15.26:0/654720455 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d2ffd640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d41585f0 0x7fd8d40ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d37fe640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8d40cb150 0x7fd8d4151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d3fff640 1 -- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8d40cb150 msgr2=0x7fd8d4151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d3fff640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8d40cb150 0x7fd8d4151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d3fff640 1 -- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d41585f0 msgr2=0x7fd8d40ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d3fff640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d41585f0 0x7fd8d40ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d3fff640 1 -- 172.21.15.26:0/654720455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc009b30 con 0x7fd8d415e690 2023-12-19T17:01:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d2ffd640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d41585f0 0x7fd8d40ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d3fff640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d415e690 0x7fd8d4159350 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00efc0 tx=0x7fd8cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:10.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8bbfff640 1 -- 172.21.15.26:0/654720455 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc018070 con 0x7fd8d415e690 2023-12-19T17:01:10.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8bbfff640 1 -- 172.21.15.26:0/654720455 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8cc00f040 con 0x7fd8d415e690 2023-12-19T17:01:10.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8bbfff640 1 -- 172.21.15.26:0/654720455 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8cc00ce00 con 0x7fd8d415e690 2023-12-19T17:01:10.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/654720455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c8009420 con 0x7fd8d415e690 2023-12-19T17:01:10.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/654720455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d4158b50 con 0x7fd8d415e690 2023-12-19T17:01:10.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/654720455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d4074270 con 0x7fd8d415e690 2023-12-19T17:01:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.097+0000 7fd8bbfff640 1 -- 172.21.15.26:0/654720455 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 67) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fd8cc014480 con 0x7fd8d415e690 2023-12-19T17:01:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.101+0000 7fd8bbfff640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd8c0065af0 0x7fd8c0067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.101+0000 7fd8bbfff640 1 -- 172.21.15.26:0/654720455 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 7243+0+0 (secure 0 0 0) 0x7fd8cc083bd0 con 0x7fd8d415e690 2023-12-19T17:01:10.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.101+0000 7fd8d37fe640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd8c0065af0 0x7fd8c0067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:10.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.101+0000 7fd8d37fe640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd8c0065af0 0x7fd8c0067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4000f70 tx=0x7fd8c4007cb0 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:10.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.105+0000 7fd8bbfff640 1 -- 172.21.15.26:0/654720455 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd8cc050e20 con 0x7fd8d415e690 2023-12-19T17:01:10.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.301+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/654720455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd8d40c2050 con 0x7fd8d415e690 2023-12-19T17:01:10.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.917+0000 7fd8bbfff640 1 -- 172.21.15.26:0/654720455 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd8cc054440 con 0x7fd8d415e690 2023-12-19T17:01:10.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.917+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd8c0065af0 msgr2=0x7fd8c0067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:10.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.917+0000 7fd8d9d8f640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd8c0065af0 0x7fd8c0067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4000f70 tx=0x7fd8c4007cb0 comp rx=0 tx=0).stop 2023-12-19T17:01:10.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.917+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/654720455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d415e690 msgr2=0x7fd8d4159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:10.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.917+0000 7fd8d9d8f640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d415e690 0x7fd8d4159350 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fd8cc00efc0 tx=0x7fd8cc00c490 comp rx=0 tx=0).stop 2023-12-19T17:01:10.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.917+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/654720455 shutdown_connections 2023-12-19T17:01:10.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.917+0000 7fd8d9d8f640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8d415e690 0x7fd8d4159350 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:10.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.917+0000 7fd8d9d8f640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8d41585f0 0x7fd8d40ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:10.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.917+0000 7fd8d9d8f640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd8c0065af0 0x7fd8c0067fb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:10.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.917+0000 7fd8d9d8f640 1 --2- 172.21.15.26:0/654720455 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd8d40cb150 0x7fd8d4151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:10.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.917+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/654720455 >> 172.21.15.26:0/654720455 conn(0x7fd8d4066160 msgr2=0x7fd8d40cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:10.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.917+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/654720455 shutdown_connections 2023-12-19T17:01:10.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:10.917+0000 7fd8d9d8f640 1 -- 172.21.15.26:0/654720455 wait complete. 2023-12-19T17:01:10.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T17:01:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb06622a640 1 Processor -- start 2023-12-19T17:01:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb06622a640 1 -- start start 2023-12-19T17:01:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb06622a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb06015e5b0 0x7fb06015d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb06622a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0600cfc30 0x7fb06015da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb06622a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0600d0dc0 con 0x7fb060157410 2023-12-19T17:01:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb06622a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb06015e240 con 0x7fb06015e5b0 2023-12-19T17:01:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb06622a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600d1010 con 0x7fb0600cfc30 2023-12-19T17:01:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb065228640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb060157410 0x7fb0600cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41528/0 (socket says 172.21.15.26:41528) 2023-12-19T17:01:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb065228640 1 -- 172.21.15.26:0/916984547 learned_addr learned my addr 172.21.15.26:0/916984547 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb064a27640 1 --2- 172.21.15.26:0/916984547 >> v2:172.21.15.26:3300/0 conn(0x7fb06015e5b0 0x7fb06015d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb065a29640 1 --2- 172.21.15.26:0/916984547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0600cfc30 0x7fb06015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb057fff640 1 -- 172.21.15.26:0/916984547 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2169799624 0 0) 0x7fb0600d0dc0 con 0x7fb060157410 2023-12-19T17:01:11.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb057fff640 1 -- 172.21.15.26:0/916984547 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb04c003530 con 0x7fb060157410 2023-12-19T17:01:11.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb064a27640 1 -- 172.21.15.26:0/916984547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0600cfc30 msgr2=0x7fb06015da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:11.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb064a27640 1 --2- 172.21.15.26:0/916984547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0600cfc30 0x7fb06015da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:11.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb064a27640 1 -- 172.21.15.26:0/916984547 >> v1:172.21.15.26:6789/0 conn(0x7fb060157410 legacy=0x7fb0600cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:11.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.137+0000 7fb064a27640 1 -- 172.21.15.26:0/916984547 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0600d12f0 con 0x7fb06015e5b0 2023-12-19T17:01:11.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.141+0000 7fb064a27640 1 --2- 172.21.15.26:0/916984547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06015e5b0 0x7fb06015d4d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb0500097c0 tx=0x7fb05001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb6ec8ff1f1e802 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:11.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.141+0000 7fb057fff640 1 -- 172.21.15.26:0/916984547 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0500ac600 con 0x7fb06015e5b0 2023-12-19T17:01:11.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.141+0000 7fb057fff640 1 -- 172.21.15.26:0/916984547 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0500a8050 con 0x7fb06015e5b0 2023-12-19T17:01:11.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.141+0000 7fb057fff640 1 -- 172.21.15.26:0/916984547 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0500ac980 con 0x7fb06015e5b0 2023-12-19T17:01:11.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.141+0000 7fb06622a640 1 -- 172.21.15.26:0/916984547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06015e5b0 msgr2=0x7fb06015d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:11.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.141+0000 7fb06622a640 1 --2- 172.21.15.26:0/916984547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06015e5b0 0x7fb06015d4d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fb0500097c0 tx=0x7fb05001c3c0 comp rx=0 tx=0).stop 2023-12-19T17:01:11.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.141+0000 7fb06622a640 1 -- 172.21.15.26:0/916984547 shutdown_connections 2023-12-19T17:01:11.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.141+0000 7fb06622a640 1 --2- 172.21.15.26:0/916984547 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0600cfc30 0x7fb06015da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:11.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.141+0000 7fb06622a640 1 --2- 172.21.15.26:0/916984547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb06015e5b0 0x7fb06015d4d0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:11.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.141+0000 7fb06622a640 1 -- 172.21.15.26:0/916984547 >> 172.21.15.26:0/916984547 conn(0x7fb06006a120 msgr2=0x7fb06006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:11.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.141+0000 7fb06622a640 1 -- 172.21.15.26:0/916984547 shutdown_connections 2023-12-19T17:01:11.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.141+0000 7fb06622a640 1 -- 172.21.15.26:0/916984547 wait complete. 2023-12-19T17:01:11.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb06622a640 1 Processor -- start 2023-12-19T17:01:11.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb06622a640 1 -- start start 2023-12-19T17:01:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb06622a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0600cfc30 0x7fb06014fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb06622a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb060157410 0x7fb06015ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb06622a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb06015e5b0 0x7fb06015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb06622a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb060157970 con 0x7fb0600cfc30 2023-12-19T17:01:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb06622a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600cc620 con 0x7fb06015e5b0 2023-12-19T17:01:11.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb06622a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600cc900 con 0x7fb060157410 2023-12-19T17:01:11.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb065228640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0600cfc30 0x7fb06014fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:11.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb065228640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0600cfc30 0x7fb06014fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60528/0 (socket says 172.21.15.26:60528) 2023-12-19T17:01:11.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb065228640 1 -- 172.21.15.26:0/560348078 learned_addr learned my addr 172.21.15.26:0/560348078 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:11.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb064a27640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb060157410 0x7fb06015ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:11.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb065a29640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb06015e5b0 0x7fb06015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:11.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb065228640 1 -- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb060157410 msgr2=0x7fb06015ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb065228640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb060157410 0x7fb06015ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb065228640 1 -- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb06015e5b0 msgr2=0x7fb06015f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb065228640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb06015e5b0 0x7fb06015f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb065228640 1 -- 172.21.15.26:0/560348078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb050009420 con 0x7fb0600cfc30 2023-12-19T17:01:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb064a27640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb060157410 0x7fb06015ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:11.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb065228640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0600cfc30 0x7fb06014fef0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00c4d0 tx=0x7fb05c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb055ffb640 1 -- 172.21.15.26:0/560348078 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c005cf0 con 0x7fb0600cfc30 2023-12-19T17:01:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb055ffb640 1 -- 172.21.15.26:0/560348078 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05c005e50 con 0x7fb0600cfc30 2023-12-19T17:01:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb055ffb640 1 -- 172.21.15.26:0/560348078 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c007270 con 0x7fb0600cfc30 2023-12-19T17:01:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb06622a640 1 -- 172.21.15.26:0/560348078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb060157bc0 con 0x7fb0600cfc30 2023-12-19T17:01:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb06622a640 1 -- 172.21.15.26:0/560348078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb060150430 con 0x7fb0600cfc30 2023-12-19T17:01:11.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.145+0000 7fb06622a640 1 -- 172.21.15.26:0/560348078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb060150620 con 0x7fb0600cfc30 2023-12-19T17:01:11.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.149+0000 7fb055ffb640 1 -- 172.21.15.26:0/560348078 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 67) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fb05c0073d0 con 0x7fb0600cfc30 2023-12-19T17:01:11.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.149+0000 7fb055ffb640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb044065b20 0x7fb044067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:11.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.149+0000 7fb055ffb640 1 -- 172.21.15.26:0/560348078 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 7243+0+0 (secure 0 0 0) 0x7fb05c082ce0 con 0x7fb0600cfc30 2023-12-19T17:01:11.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.149+0000 7fb064a27640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb044065b20 0x7fb044067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:11.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.149+0000 7fb064a27640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb044065b20 0x7fb044067fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb0500ae000 tx=0x7fb050008040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:11.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.153+0000 7fb055ffb640 1 -- 172.21.15.26:0/560348078 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb05c0500f0 con 0x7fb0600cfc30 2023-12-19T17:01:11.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.309+0000 7fb06622a640 1 -- 172.21.15.26:0/560348078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb0600ccac0 con 0x7fb0600cfc30 2023-12-19T17:01:11.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.929+0000 7fb055ffb640 1 -- 172.21.15.26:0/560348078 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb05c0504c0 con 0x7fb0600cfc30 2023-12-19T17:01:11.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.929+0000 7fb055ffb640 1 -- 172.21.15.26:0/560348078 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v127) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb05c06c4f0 con 0x7fb0600cfc30 2023-12-19T17:01:11.935 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T17:01:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.933+0000 7fb06622a640 1 -- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb044065b20 msgr2=0x7fb044067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.933+0000 7fb06622a640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb044065b20 0x7fb044067fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb0500ae000 tx=0x7fb050008040 comp rx=0 tx=0).stop 2023-12-19T17:01:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.933+0000 7fb06622a640 1 -- 172.21.15.26:0/560348078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0600cfc30 msgr2=0x7fb06014fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.933+0000 7fb06622a640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0600cfc30 0x7fb06014fef0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00c4d0 tx=0x7fb05c00c990 comp rx=0 tx=0).stop 2023-12-19T17:01:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.933+0000 7fb06622a640 1 -- 172.21.15.26:0/560348078 shutdown_connections 2023-12-19T17:01:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.933+0000 7fb06622a640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb06015e5b0 0x7fb06015f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.933+0000 7fb06622a640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb044065b20 0x7fb044067fe0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.933+0000 7fb06622a640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb060157410 0x7fb06015ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:11.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.933+0000 7fb06622a640 1 --2- 172.21.15.26:0/560348078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0600cfc30 0x7fb06014fef0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:11.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.933+0000 7fb06622a640 1 -- 172.21.15.26:0/560348078 >> 172.21.15.26:0/560348078 conn(0x7fb06006a120 msgr2=0x7fb0600b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:11.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.933+0000 7fb06622a640 1 -- 172.21.15.26:0/560348078 shutdown_connections 2023-12-19T17:01:11.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:11.933+0000 7fb06622a640 1 -- 172.21.15.26:0/560348078 wait complete. 2023-12-19T17:01:11.960 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T17:01:12.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.105+0000 7f992f641640 1 Processor -- start 2023-12-19T17:01:12.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.105+0000 7f992f641640 1 -- start start 2023-12-19T17:01:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.105+0000 7f992f641640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f99280ada30 0x7f992806c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.105+0000 7f992f641640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99280c1a50 0x7f99280ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.105+0000 7f992f641640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99280cce50 con 0x7f99280cd2e0 2023-12-19T17:01:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.105+0000 7f992f641640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9928161800 con 0x7f99280ada30 2023-12-19T17:01:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.105+0000 7f992f641640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99280cc310 con 0x7f99280c1a50 2023-12-19T17:01:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.105+0000 7f992cbdc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f99280cd2e0 0x7f99280c1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41530/0 (socket says 172.21.15.26:41530) 2023-12-19T17:01:12.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.105+0000 7f992cbdc640 1 -- 172.21.15.26:0/546531368 learned_addr learned my addr 172.21.15.26:0/546531368 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.105+0000 7f992dbde640 1 --2- 172.21.15.26:0/546531368 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99280c1a50 0x7f99280ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.105+0000 7f992d3dd640 1 --2- 172.21.15.26:0/546531368 >> v2:172.21.15.160:3300/0 conn(0x7f99280ada30 0x7f992806c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.105+0000 7f991ffff640 1 -- 172.21.15.26:0/546531368 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2179604453 0 0) 0x7f99280cce50 con 0x7f99280cd2e0 2023-12-19T17:01:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.105+0000 7f991ffff640 1 -- 172.21.15.26:0/546531368 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9914003530 con 0x7f99280cd2e0 2023-12-19T17:01:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.109+0000 7f992dbde640 1 -- 172.21.15.26:0/546531368 >> v2:172.21.15.160:3300/0 conn(0x7f99280ada30 msgr2=0x7f992806c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:12.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.109+0000 7f992dbde640 1 --2- 172.21.15.26:0/546531368 >> v2:172.21.15.160:3300/0 conn(0x7f99280ada30 0x7f992806c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.109+0000 7f992dbde640 1 -- 172.21.15.26:0/546531368 >> v1:172.21.15.26:6789/0 conn(0x7f99280cd2e0 legacy=0x7f99280c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.109+0000 7f991ffff640 1 -- 172.21.15.26:0/546531368 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1558615615 0 0) 0x7f9914003530 con 0x7f99280cd2e0 2023-12-19T17:01:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.109+0000 7f992dbde640 1 -- 172.21.15.26:0/546531368 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99280cd740 con 0x7f99280c1a50 2023-12-19T17:01:12.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.109+0000 7f992d3dd640 1 --2- 172.21.15.26:0/546531368 >> v2:172.21.15.160:3300/0 conn(0x7f99280ada30 0x7f992806c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:01:12.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.109+0000 7f992dbde640 1 --2- 172.21.15.26:0/546531368 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99280c1a50 0x7f99280ca570 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9920009d90 tx=0x7f992001c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b139df6fd82c47a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:12.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.109+0000 7f991ffff640 1 -- 172.21.15.26:0/546531368 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918001680 con 0x7f99280c1a50 2023-12-19T17:01:12.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.109+0000 7f991ffff640 1 -- 172.21.15.26:0/546531368 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99200a5640 con 0x7f99280c1a50 2023-12-19T17:01:12.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.109+0000 7f991ffff640 1 -- 172.21.15.26:0/546531368 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99200a59b0 con 0x7f99280c1a50 2023-12-19T17:01:12.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 -- 172.21.15.26:0/546531368 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99280c1a50 msgr2=0x7f99280ca570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:12.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 --2- 172.21.15.26:0/546531368 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99280c1a50 0x7f99280ca570 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9920009d90 tx=0x7f992001c840 comp rx=0 tx=0).stop 2023-12-19T17:01:12.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 -- 172.21.15.26:0/546531368 shutdown_connections 2023-12-19T17:01:12.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 --2- 172.21.15.26:0/546531368 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99280c1a50 0x7f99280ca570 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 --2- 172.21.15.26:0/546531368 >> v2:172.21.15.160:3300/0 conn(0x7f99280ada30 0x7f992806c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 -- 172.21.15.26:0/546531368 >> 172.21.15.26:0/546531368 conn(0x7f99280b0270 msgr2=0x7f99280b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:12.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 -- 172.21.15.26:0/546531368 shutdown_connections 2023-12-19T17:01:12.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 -- 172.21.15.26:0/546531368 wait complete. 2023-12-19T17:01:12.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 Processor -- start 2023-12-19T17:01:12.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 -- start start 2023-12-19T17:01:12.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99280ada30 0x7f99280c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:12.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99280c1a50 0x7f99281f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:12.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99280cd2e0 0x7f99280c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99280c86c0 con 0x7f99280ada30 2023-12-19T17:01:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99281fa380 con 0x7f99280c1a50 2023-12-19T17:01:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992f641640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99281f9130 con 0x7f99280cd2e0 2023-12-19T17:01:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992d3dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99280ada30 0x7f99280c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992d3dd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99280ada30 0x7f99280c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60534/0 (socket says 172.21.15.26:60534) 2023-12-19T17:01:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992d3dd640 1 -- 172.21.15.26:0/4071890192 learned_addr learned my addr 172.21.15.26:0/4071890192 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992dbde640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99280cd2e0 0x7f99280c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992cbdc640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99280c1a50 0x7f99281f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992d3dd640 1 -- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99280cd2e0 msgr2=0x7f99280c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992d3dd640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99280cd2e0 0x7f99280c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992d3dd640 1 -- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99280c1a50 msgr2=0x7f99281f9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992d3dd640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99280c1a50 0x7f99281f9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992d3dd640 1 -- 172.21.15.26:0/4071890192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9924009420 con 0x7f99280ada30 2023-12-19T17:01:12.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.113+0000 7f992cbdc640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99280c1a50 0x7f99281f9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:12.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.117+0000 7f992d3dd640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99280ada30 0x7f99280c5fc0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f992400ec10 tx=0x7f992400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:12.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.117+0000 7f991dffb640 1 -- 172.21.15.26:0/4071890192 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992400cc30 con 0x7f99280ada30 2023-12-19T17:01:12.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.117+0000 7f991dffb640 1 -- 172.21.15.26:0/4071890192 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992400cd90 con 0x7f99280ada30 2023-12-19T17:01:12.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.117+0000 7f992f641640 1 -- 172.21.15.26:0/4071890192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9920009a70 con 0x7f99280ada30 2023-12-19T17:01:12.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.117+0000 7f991dffb640 1 -- 172.21.15.26:0/4071890192 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924013690 con 0x7f99280ada30 2023-12-19T17:01:12.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.117+0000 7f992f641640 1 -- 172.21.15.26:0/4071890192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99280c83e0 con 0x7f99280ada30 2023-12-19T17:01:12.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.117+0000 7f992f641640 1 -- 172.21.15.26:0/4071890192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9928074270 con 0x7f99280ada30 2023-12-19T17:01:12.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.117+0000 7f991dffb640 1 -- 172.21.15.26:0/4071890192 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f99240137f0 con 0x7f99280ada30 2023-12-19T17:01:12.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.117+0000 7f991dffb640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f990c065b80 0x7f990c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:12.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.117+0000 7f991dffb640 1 -- 172.21.15.26:0/4071890192 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6805+0+0 (secure 0 0 0) 0x7f9924083a90 con 0x7f99280ada30 2023-12-19T17:01:12.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.117+0000 7f992cbdc640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f990c065b80 0x7f990c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:12.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.121+0000 7f992cbdc640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f990c065b80 0x7f990c068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9928161b10 tx=0x7f9918009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:12.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.121+0000 7f991dffb640 1 -- 172.21.15.26:0/4071890192 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9924050e80 con 0x7f99280ada30 2023-12-19T17:01:12.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.281+0000 7f992f641640 1 -- 172.21.15.26:0/4071890192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f992806c130 con 0x7f99280ada30 2023-12-19T17:01:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.597+0000 7f991dffb640 1 -- 172.21.15.26:0/4071890192 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v128) v1 ==== 169+0+0 (secure 0 0 0) 0x7f99240544a0 con 0x7f99280ada30 2023-12-19T17:01:12.604 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T17:01:12.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.601+0000 7f992f641640 1 -- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f990c065b80 msgr2=0x7f990c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:12.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.601+0000 7f992f641640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f990c065b80 0x7f990c068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9928161b10 tx=0x7f9918009040 comp rx=0 tx=0).stop 2023-12-19T17:01:12.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.601+0000 7f992f641640 1 -- 172.21.15.26:0/4071890192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99280ada30 msgr2=0x7f99280c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:12.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.601+0000 7f992f641640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99280ada30 0x7f99280c5fc0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f992400ec10 tx=0x7f992400c520 comp rx=0 tx=0).stop 2023-12-19T17:01:12.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.601+0000 7f992f641640 1 -- 172.21.15.26:0/4071890192 shutdown_connections 2023-12-19T17:01:12.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.601+0000 7f992f641640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f990c065b80 0x7f990c068040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.601+0000 7f992f641640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99280cd2e0 0x7f99280c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.601+0000 7f992f641640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99280c1a50 0x7f99281f9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.601+0000 7f992f641640 1 --2- 172.21.15.26:0/4071890192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99280ada30 0x7f99280c5fc0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.601+0000 7f992f641640 1 -- 172.21.15.26:0/4071890192 >> 172.21.15.26:0/4071890192 conn(0x7f99280b0270 msgr2=0x7f9928159800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:12.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.605+0000 7f992f641640 1 -- 172.21.15.26:0/4071890192 shutdown_connections 2023-12-19T17:01:12.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.605+0000 7f992f641640 1 -- 172.21.15.26:0/4071890192 wait complete. 2023-12-19T17:01:12.632 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-19T17:01:12.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.789+0000 7f3171b8f640 1 Processor -- start 2023-12-19T17:01:12.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.789+0000 7f3171b8f640 1 -- start start 2023-12-19T17:01:12.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.789+0000 7f3171b8f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f316c1599e0 0x7f316c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:12.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.789+0000 7f3171b8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f316c0cb150 0x7f316c0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:12.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f3171b8f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316c0cbaf0 con 0x7f316c15d530 2023-12-19T17:01:12.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f3171b8f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316c15e050 con 0x7f316c1599e0 2023-12-19T17:01:12.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f3171b8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316c15e240 con 0x7f316c0cb150 2023-12-19T17:01:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f316bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f316c15d530 0x7f316c15d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41542/0 (socket says 172.21.15.26:41542) 2023-12-19T17:01:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f316bfff640 1 -- 172.21.15.26:0/1885397223 learned_addr learned my addr 172.21.15.26:0/1885397223 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f316b7fe640 1 --2- 172.21.15.26:0/1885397223 >> v2:172.21.15.160:3300/0 conn(0x7f316c1599e0 0x7f316c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f316affd640 1 --2- 172.21.15.26:0/1885397223 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f316c0cb150 0x7f316c0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f316a7fc640 1 -- 172.21.15.26:0/1885397223 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438008156 0 0) 0x7f316c0cbaf0 con 0x7f316c15d530 2023-12-19T17:01:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f316a7fc640 1 -- 172.21.15.26:0/1885397223 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3154003530 con 0x7f316c15d530 2023-12-19T17:01:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f316affd640 1 -- 172.21.15.26:0/1885397223 >> v2:172.21.15.160:3300/0 conn(0x7f316c1599e0 msgr2=0x7f316c06f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:12.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f316affd640 1 --2- 172.21.15.26:0/1885397223 >> v2:172.21.15.160:3300/0 conn(0x7f316c1599e0 0x7f316c06f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f316affd640 1 -- 172.21.15.26:0/1885397223 >> v1:172.21.15.26:6789/0 conn(0x7f316c15d530 legacy=0x7f316c15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:12.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f316affd640 1 -- 172.21.15.26:0/1885397223 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c0c9520 con 0x7f316c0cb150 2023-12-19T17:01:12.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.793+0000 7f316affd640 1 --2- 172.21.15.26:0/1885397223 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f316c0cb150 0x7f316c0cb5b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f31600097c0 tx=0x7f316001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa53d1a820864e77 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:12.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f316a7fc640 1 -- 172.21.15.26:0/1885397223 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31600ab680 con 0x7f316c0cb150 2023-12-19T17:01:12.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f316a7fc640 1 -- 172.21.15.26:0/1885397223 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31600ab7e0 con 0x7f316c0cb150 2023-12-19T17:01:12.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f316a7fc640 1 -- 172.21.15.26:0/1885397223 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31600abb50 con 0x7f316c0cb150 2023-12-19T17:01:12.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 -- 172.21.15.26:0/1885397223 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f316c0cb150 msgr2=0x7f316c0cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:12.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 --2- 172.21.15.26:0/1885397223 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f316c0cb150 0x7f316c0cb5b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f31600097c0 tx=0x7f316001c480 comp rx=0 tx=0).stop 2023-12-19T17:01:12.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 -- 172.21.15.26:0/1885397223 shutdown_connections 2023-12-19T17:01:12.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 --2- 172.21.15.26:0/1885397223 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f316c0cb150 0x7f316c0cb5b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 --2- 172.21.15.26:0/1885397223 >> v2:172.21.15.160:3300/0 conn(0x7f316c1599e0 0x7f316c06f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 -- 172.21.15.26:0/1885397223 >> 172.21.15.26:0/1885397223 conn(0x7f316c066160 msgr2=0x7f316c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:12.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 -- 172.21.15.26:0/1885397223 shutdown_connections 2023-12-19T17:01:12.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 -- 172.21.15.26:0/1885397223 wait complete. 2023-12-19T17:01:12.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 Processor -- start 2023-12-19T17:01:12.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 -- start start 2023-12-19T17:01:12.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f316c0cb150 0x7f316c14ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:12.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f316c1599e0 0x7f316c1f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:12.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f316c15d530 0x7f316c0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:12.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c152550 con 0x7f316c1599e0 2023-12-19T17:01:12.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c1514b0 con 0x7f316c0cb150 2023-12-19T17:01:12.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.797+0000 7f3171b8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316c151790 con 0x7f316c15d530 2023-12-19T17:01:12.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f316affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f316c1599e0 0x7f316c1f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:12.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f316affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f316c1599e0 0x7f316c1f5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60536/0 (socket says 172.21.15.26:60536) 2023-12-19T17:01:12.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f316affd640 1 -- 172.21.15.26:0/1372940562 learned_addr learned my addr 172.21.15.26:0/1372940562 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:12.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f316b7fe640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f316c0cb150 0x7f316c14ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f316bfff640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f316c15d530 0x7f316c0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f316affd640 1 -- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f316c15d530 msgr2=0x7f316c0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f316affd640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f316c15d530 0x7f316c0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f316affd640 1 -- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f316c0cb150 msgr2=0x7f316c14ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f316affd640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f316c0cb150 0x7f316c14ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f316affd640 1 -- 172.21.15.26:0/1372940562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3160009420 con 0x7f316c1599e0 2023-12-19T17:01:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f316b7fe640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f316c0cb150 0x7f316c14ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f316affd640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f316c1599e0 0x7f316c1f5c40 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f31600ad040 tx=0x7f3160019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f3153fff640 1 -- 172.21.15.26:0/1372940562 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31600bf610 con 0x7f316c1599e0 2023-12-19T17:01:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f3153fff640 1 -- 172.21.15.26:0/1372940562 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31600bf790 con 0x7f316c1599e0 2023-12-19T17:01:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f3153fff640 1 -- 172.21.15.26:0/1372940562 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31600bfb40 con 0x7f316c1599e0 2023-12-19T17:01:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f3171b8f640 1 -- 172.21.15.26:0/1372940562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c151a70 con 0x7f316c1599e0 2023-12-19T17:01:12.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f3171b8f640 1 -- 172.21.15.26:0/1372940562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c152030 con 0x7f316c1599e0 2023-12-19T17:01:12.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f3171b8f640 1 -- 172.21.15.26:0/1372940562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c152220 con 0x7f316c1599e0 2023-12-19T17:01:12.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.801+0000 7f3153fff640 1 -- 172.21.15.26:0/1372940562 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f31600ca430 con 0x7f316c1599e0 2023-12-19T17:01:12.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.805+0000 7f3153fff640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3158065b80 0x7f3158068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:12.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.805+0000 7f3153fff640 1 -- 172.21.15.26:0/1372940562 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f316011e3f0 con 0x7f316c1599e0 2023-12-19T17:01:12.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.805+0000 7f316b7fe640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3158065b80 0x7f3158068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:12.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.805+0000 7f316b7fe640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3158065b80 0x7f3158068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f315c009bb0 tx=0x7f315c008040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:12.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.805+0000 7f3153fff640 1 -- 172.21.15.26:0/1372940562 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f31600eb9c0 con 0x7f316c1599e0 2023-12-19T17:01:12.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.965+0000 7f3171b8f640 1 -- 172.21.15.26:0/1372940562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f316c074270 con 0x7f316c1599e0 2023-12-19T17:01:12.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.969+0000 7f3153fff640 1 -- 172.21.15.26:0/1372940562 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v43) v1 ==== 120+0+0 (secure 0 0 0) 0x7f31600eefe0 con 0x7f316c1599e0 2023-12-19T17:01:12.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.969+0000 7f3171b8f640 1 -- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3158065b80 msgr2=0x7f3158068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:12.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.969+0000 7f3171b8f640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3158065b80 0x7f3158068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f315c009bb0 tx=0x7f315c008040 comp rx=0 tx=0).stop 2023-12-19T17:01:12.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.969+0000 7f3171b8f640 1 -- 172.21.15.26:0/1372940562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f316c1599e0 msgr2=0x7f316c1f5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:12.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.969+0000 7f3171b8f640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f316c1599e0 0x7f316c1f5c40 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f31600ad040 tx=0x7f3160019220 comp rx=0 tx=0).stop 2023-12-19T17:01:12.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.969+0000 7f3171b8f640 1 -- 172.21.15.26:0/1372940562 shutdown_connections 2023-12-19T17:01:12.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.969+0000 7f3171b8f640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3158065b80 0x7f3158068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.969+0000 7f3171b8f640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f316c15d530 0x7f316c0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.969+0000 7f3171b8f640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f316c1599e0 0x7f316c1f5c40 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.969+0000 7f3171b8f640 1 --2- 172.21.15.26:0/1372940562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f316c0cb150 0x7f316c14ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.969+0000 7f3171b8f640 1 -- 172.21.15.26:0/1372940562 >> 172.21.15.26:0/1372940562 conn(0x7f316c066160 msgr2=0x7f316c1539f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.973+0000 7f3171b8f640 1 -- 172.21.15.26:0/1372940562 shutdown_connections 2023-12-19T17:01:12.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:12.973+0000 7f3171b8f640 1 -- 172.21.15.26:0/1372940562 wait complete. 2023-12-19T17:01:12.995 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T17:01:13.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b8709640 1 Processor -- start 2023-12-19T17:01:13.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b8709640 1 -- start start 2023-12-19T17:01:13.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b8709640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09b015cc70 0x7f09b00d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b8709640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f09b00b88b0 0x7f09b00b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b8709640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b00b71f0 con 0x7f09b00b6140 2023-12-19T17:01:13.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b8709640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b00b65e0 con 0x7f09b015cc70 2023-12-19T17:01:13.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b8709640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b00b68c0 con 0x7f09b00b88b0 2023-12-19T17:01:13.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b64a5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09b015cc70 0x7f09b00d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b64a5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09b015cc70 0x7f09b00d0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60540/0 (socket says 172.21.15.26:60540) 2023-12-19T17:01:13.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b64a5640 1 -- 172.21.15.26:0/3016887368 learned_addr learned my addr 172.21.15.26:0/3016887368 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:13.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b5ca4640 1 --2- 172.21.15.26:0/3016887368 >> v2:172.21.15.160:3300/0 conn(0x7f09b00b88b0 0x7f09b00b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b64a5640 1 -- 172.21.15.26:0/3016887368 >> v2:172.21.15.160:3300/0 conn(0x7f09b00b88b0 msgr2=0x7f09b00b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:13.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b64a5640 1 --2- 172.21.15.26:0/3016887368 >> v2:172.21.15.160:3300/0 conn(0x7f09b00b88b0 0x7f09b00b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b64a5640 1 -- 172.21.15.26:0/3016887368 >> v1:172.21.15.160:6789/0 conn(0x7f09b00b6140 legacy=0x7f09b00cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b64a5640 1 -- 172.21.15.26:0/3016887368 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b00b6ba0 con 0x7f09b015cc70 2023-12-19T17:01:13.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.137+0000 7f09b5ca4640 1 --2- 172.21.15.26:0/3016887368 >> v2:172.21.15.160:3300/0 conn(0x7f09b00b88b0 0x7f09b00b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:13.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.141+0000 7f09b64a5640 1 --2- 172.21.15.26:0/3016887368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b015cc70 0x7f09b00d0c60 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f09b00d3ce0 tx=0x7f09ac01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a58dee857ec77e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:13.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.141+0000 7f09b54a3640 1 -- 172.21.15.26:0/3016887368 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac0a89f0 con 0x7f09b015cc70 2023-12-19T17:01:13.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.141+0000 7f09b54a3640 1 -- 172.21.15.26:0/3016887368 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ac0a8b50 con 0x7f09b015cc70 2023-12-19T17:01:13.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.141+0000 7f09b54a3640 1 -- 172.21.15.26:0/3016887368 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac0b0440 con 0x7f09b015cc70 2023-12-19T17:01:13.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.141+0000 7f09b8709640 1 -- 172.21.15.26:0/3016887368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b015cc70 msgr2=0x7f09b00d0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.141+0000 7f09b8709640 1 --2- 172.21.15.26:0/3016887368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b015cc70 0x7f09b00d0c60 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f09b00d3ce0 tx=0x7f09ac01ecf0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.141+0000 7f09b8709640 1 -- 172.21.15.26:0/3016887368 shutdown_connections 2023-12-19T17:01:13.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.141+0000 7f09b8709640 1 --2- 172.21.15.26:0/3016887368 >> v2:172.21.15.160:3300/0 conn(0x7f09b00b88b0 0x7f09b00b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.141+0000 7f09b8709640 1 --2- 172.21.15.26:0/3016887368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b015cc70 0x7f09b00d0c60 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.141+0000 7f09b8709640 1 -- 172.21.15.26:0/3016887368 >> 172.21.15.26:0/3016887368 conn(0x7f09b006a120 msgr2=0x7f09b006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:13.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.141+0000 7f09b8709640 1 -- 172.21.15.26:0/3016887368 shutdown_connections 2023-12-19T17:01:13.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.141+0000 7f09b8709640 1 -- 172.21.15.26:0/3016887368 wait complete. 2023-12-19T17:01:13.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b8709640 1 Processor -- start 2023-12-19T17:01:13.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b8709640 1 -- start start 2023-12-19T17:01:13.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b8709640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b00b6140 0x7f09b00c7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b8709640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09b00b88b0 0x7f09b015f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b8709640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b015cc70 0x7f09b015f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b8709640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00ca530 con 0x7f09b00b6140 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b8709640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00ca1e0 con 0x7f09b00b88b0 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b8709640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00c7a80 con 0x7f09b015cc70 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b64a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b00b6140 0x7f09b00c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b64a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b00b6140 0x7f09b00c7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60552/0 (socket says 172.21.15.26:60552) 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b64a5640 1 -- 172.21.15.26:0/836844191 learned_addr learned my addr 172.21.15.26:0/836844191 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b5ca4640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09b00b88b0 0x7f09b015f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b6ca6640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b015cc70 0x7f09b015f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b64a5640 1 -- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b015cc70 msgr2=0x7f09b015f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b64a5640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b015cc70 0x7f09b015f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b64a5640 1 -- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09b00b88b0 msgr2=0x7f09b015f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b64a5640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09b00b88b0 0x7f09b015f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b64a5640 1 -- 172.21.15.26:0/836844191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0094d0 con 0x7f09b00b6140 2023-12-19T17:01:13.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b64a5640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b00b6140 0x7f09b00c7540 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f09ac01b900 tx=0x7f09ac01ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:13.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09a6ffd640 1 -- 172.21.15.26:0/836844191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac0ba600 con 0x7f09b00b6140 2023-12-19T17:01:13.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09a6ffd640 1 -- 172.21.15.26:0/836844191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ac0ba760 con 0x7f09b00b6140 2023-12-19T17:01:13.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09a6ffd640 1 -- 172.21.15.26:0/836844191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac0bab10 con 0x7f09b00b6140 2023-12-19T17:01:13.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b8709640 1 -- 172.21.15.26:0/836844191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b0148850 con 0x7f09b00b6140 2023-12-19T17:01:13.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.145+0000 7f09b8709640 1 -- 172.21.15.26:0/836844191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b00c8000 con 0x7f09b00b6140 2023-12-19T17:01:13.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.149+0000 7f09b8709640 1 -- 172.21.15.26:0/836844191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b0074270 con 0x7f09b00b6140 2023-12-19T17:01:13.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.149+0000 7f09a6ffd640 1 -- 172.21.15.26:0/836844191 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f09ac0bac70 con 0x7f09b00b6140 2023-12-19T17:01:13.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.149+0000 7f09a6ffd640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0994065b80 0x7f0994068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.149+0000 7f09a6ffd640 1 -- 172.21.15.26:0/836844191 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f09ac121950 con 0x7f09b00b6140 2023-12-19T17:01:13.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.149+0000 7f09b5ca4640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0994065b80 0x7f0994068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.149+0000 7f09b5ca4640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0994065b80 0x7f0994068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f09b01550e0 tx=0x7f09a0008040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:13.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.153+0000 7f09a6ffd640 1 -- 172.21.15.26:0/836844191 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f09ac0eeff0 con 0x7f09b00b6140 2023-12-19T17:01:13.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.313+0000 7f09b8709640 1 -- 172.21.15.26:0/836844191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f09b00c9fc0 con 0x7f09b00b6140 2023-12-19T17:01:13.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.317+0000 7f09a6ffd640 1 -- 172.21.15.26:0/836844191 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f09ac0f2610 con 0x7f09b00b6140 2023-12-19T17:01:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.317+0000 7f09b8709640 1 -- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0994065b80 msgr2=0x7f0994068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.317+0000 7f09b8709640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0994065b80 0x7f0994068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f09b01550e0 tx=0x7f09a0008040 comp rx=0 tx=0).stop 2023-12-19T17:01:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.317+0000 7f09b8709640 1 -- 172.21.15.26:0/836844191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b00b6140 msgr2=0x7f09b00c7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.317+0000 7f09b8709640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b00b6140 0x7f09b00c7540 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f09ac01b900 tx=0x7f09ac01ba90 comp rx=0 tx=0).stop 2023-12-19T17:01:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.317+0000 7f09b8709640 1 -- 172.21.15.26:0/836844191 shutdown_connections 2023-12-19T17:01:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.317+0000 7f09b8709640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0994065b80 0x7f0994068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.317+0000 7f09b8709640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b015cc70 0x7f09b015f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.317+0000 7f09b8709640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09b00b88b0 0x7f09b015f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.317+0000 7f09b8709640 1 --2- 172.21.15.26:0/836844191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b00b6140 0x7f09b00c7540 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.317+0000 7f09b8709640 1 -- 172.21.15.26:0/836844191 >> 172.21.15.26:0/836844191 conn(0x7f09b006a120 msgr2=0x7f09b00cdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:13.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.317+0000 7f09b8709640 1 -- 172.21.15.26:0/836844191 shutdown_connections 2023-12-19T17:01:13.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.317+0000 7f09b8709640 1 -- 172.21.15.26:0/836844191 wait complete. 2023-12-19T17:01:13.343 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-19T17:01:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae492c4640 1 Processor -- start 2023-12-19T17:01:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae492c4640 1 -- start start 2023-12-19T17:01:13.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae492c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae440ccbc0 0x7fae440d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae492c4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae440cd330 con 0x7fae44160d00 2023-12-19T17:01:13.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae492c4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae440cd610 con 0x7fae440d1330 2023-12-19T17:01:13.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae492c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae440cd8f0 con 0x7fae440ccbc0 2023-12-19T17:01:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae42ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae44160d00 0x7fae441610e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41556/0 (socket says 172.21.15.26:41556) 2023-12-19T17:01:13.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae42ffd640 1 -- 172.21.15.26:0/3343944341 learned_addr learned my addr 172.21.15.26:0/3343944341 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae3bfff640 1 --2- 172.21.15.26:0/3343944341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae440ccbc0 0x7fae440d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191621026 0 0) 0x7fae440cd330 con 0x7fae44160d00 2023-12-19T17:01:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae30003530 con 0x7fae44160d00 2023-12-19T17:01:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1307129351 0 0) 0x7fae30003530 con 0x7fae44160d00 2023-12-19T17:01:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae440ccbc0 msgr2=0x7fae440d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 --2- 172.21.15.26:0/3343944341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae440ccbc0 0x7fae440d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 >> v1:172.21.15.160:6789/0 conn(0x7fae440d1330 legacy=0x7fae4415c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae440cdbd0 con 0x7fae44160d00 2023-12-19T17:01:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fae340016a0 con 0x7fae44160d00 2023-12-19T17:01:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 >> v1:172.21.15.26:6789/0 conn(0x7fae44160d00 legacy=0x7fae441610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 --2- 172.21.15.26:0/3343944341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae300039b0 0x7fae30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 --2- 172.21.15.26:0/3343944341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae300047d0 0x7fae30004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 --2- 172.21.15.26:0/3343944341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae440ccbc0 0x7fae30005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440cd8f0 con 0x7fae300039b0 2023-12-19T17:01:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3000ba40 con 0x7fae300047d0 2023-12-19T17:01:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3000bc00 con 0x7fae440ccbc0 2023-12-19T17:01:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae42ffd640 1 --2- 172.21.15.26:0/3343944341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae300039b0 0x7fae30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae3bfff640 1 --2- 172.21.15.26:0/3343944341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae300047d0 0x7fae30004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.485+0000 7fae437fe640 1 --2- 172.21.15.26:0/3343944341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae440ccbc0 0x7fae30005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae42ffd640 1 -- 172.21.15.26:0/3343944341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae440ccbc0 msgr2=0x7fae30005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae42ffd640 1 --2- 172.21.15.26:0/3343944341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae440ccbc0 0x7fae30005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae42ffd640 1 -- 172.21.15.26:0/3343944341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae300047d0 msgr2=0x7fae30004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae42ffd640 1 --2- 172.21.15.26:0/3343944341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae300047d0 0x7fae30004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae42ffd640 1 -- 172.21.15.26:0/3343944341 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae30003530 con 0x7fae300039b0 2023-12-19T17:01:13.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae437fe640 1 --2- 172.21.15.26:0/3343944341 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae440ccbc0 0x7fae30005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:13.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae42ffd640 1 --2- 172.21.15.26:0/3343944341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae300039b0 0x7fae30003dd0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fae34001ec0 tx=0x7fae3401cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c134b7fad365b494 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:13.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34005ae0 con 0x7fae300039b0 2023-12-19T17:01:13.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae34005c40 con 0x7fae300039b0 2023-12-19T17:01:13.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae427fc640 1 -- 172.21.15.26:0/3343944341 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae340a58b0 con 0x7fae300039b0 2023-12-19T17:01:13.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae492c4640 1 -- 172.21.15.26:0/3343944341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae300039b0 msgr2=0x7fae30003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae492c4640 1 --2- 172.21.15.26:0/3343944341 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae300039b0 0x7fae30003dd0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fae34001ec0 tx=0x7fae3401cbb0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae437fe640 1 -- 172.21.15.26:0/3343944341 reap_dead start 2023-12-19T17:01:13.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae492c4640 1 -- 172.21.15.26:0/3343944341 shutdown_connections 2023-12-19T17:01:13.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae492c4640 1 -- 172.21.15.26:0/3343944341 >> 172.21.15.26:0/3343944341 conn(0x7fae4406ef60 msgr2=0x7fae4406f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:13.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae492c4640 1 -- 172.21.15.26:0/3343944341 shutdown_connections 2023-12-19T17:01:13.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.489+0000 7fae492c4640 1 -- 172.21.15.26:0/3343944341 wait complete. 2023-12-19T17:01:13.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae492c4640 1 Processor -- start 2023-12-19T17:01:13.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae492c4640 1 -- start start 2023-12-19T17:01:13.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae492c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae300039b0 0x7fae440af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae492c4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4415a620 0x7fae440ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae492c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4415b610 0x7fae44155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae492c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440ad2f0 con 0x7fae300039b0 2023-12-19T17:01:13.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae492c4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4415c1c0 con 0x7fae4415a620 2023-12-19T17:01:13.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae492c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4415aaa0 con 0x7fae4415b610 2023-12-19T17:01:13.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae42ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae300039b0 0x7fae440af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae42ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae300039b0 0x7fae440af180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60556/0 (socket says 172.21.15.26:60556) 2023-12-19T17:01:13.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae42ffd640 1 -- 172.21.15.26:0/297899973 learned_addr learned my addr 172.21.15.26:0/297899973 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:13.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae437fe640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4415b610 0x7fae44155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae3bfff640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4415a620 0x7fae440ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae42ffd640 1 -- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4415b610 msgr2=0x7fae44155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae42ffd640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4415b610 0x7fae44155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae42ffd640 1 -- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4415a620 msgr2=0x7fae440ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae42ffd640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4415a620 0x7fae440ad6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae42ffd640 1 -- 172.21.15.26:0/297899973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4415ad20 con 0x7fae300039b0 2023-12-19T17:01:13.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae42ffd640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae300039b0 0x7fae440af180 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fae3401c9e0 tx=0x7fae30005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:13.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae3b7fe640 1 -- 172.21.15.26:0/297899973 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae340b4070 con 0x7fae300039b0 2023-12-19T17:01:13.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae3b7fe640 1 -- 172.21.15.26:0/297899973 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae34005db0 con 0x7fae300039b0 2023-12-19T17:01:13.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae492c4640 1 -- 172.21.15.26:0/297899973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae440ac990 con 0x7fae300039b0 2023-12-19T17:01:13.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae3b7fe640 1 -- 172.21.15.26:0/297899973 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae340016a0 con 0x7fae300039b0 2023-12-19T17:01:13.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.493+0000 7fae492c4640 1 -- 172.21.15.26:0/297899973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae440acf10 con 0x7fae300039b0 2023-12-19T17:01:13.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.497+0000 7fae492c4640 1 -- 172.21.15.26:0/297899973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae44074270 con 0x7fae300039b0 2023-12-19T17:01:13.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.497+0000 7fae3b7fe640 1 -- 172.21.15.26:0/297899973 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fae340a58b0 con 0x7fae300039b0 2023-12-19T17:01:13.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.497+0000 7fae3b7fe640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae24065bb0 0x7fae24068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.497+0000 7fae3b7fe640 1 -- 172.21.15.26:0/297899973 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6391+0+0 (secure 0 0 0) 0x7fae3411d2d0 con 0x7fae300039b0 2023-12-19T17:01:13.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.497+0000 7fae3bfff640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae24065bb0 0x7fae24068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.497+0000 7fae3bfff640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae24065bb0 0x7fae24068070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fae4415c500 tx=0x7fae2c008040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:13.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.501+0000 7fae3b7fe640 1 -- 172.21.15.26:0/297899973 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fae340ea8c0 con 0x7fae300039b0 2023-12-19T17:01:13.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.657+0000 7fae492c4640 1 -- 172.21.15.26:0/297899973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fae440b2680 con 0x7fae300039b0 2023-12-19T17:01:13.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.661+0000 7fae3b7fe640 1 -- 172.21.15.26:0/297899973 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v45) v1 ==== 115+0+0 (secure 0 0 0) 0x7fae340edee0 con 0x7fae300039b0 2023-12-19T17:01:13.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.665+0000 7fae492c4640 1 -- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae24065bb0 msgr2=0x7fae24068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.665+0000 7fae492c4640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae24065bb0 0x7fae24068070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fae4415c500 tx=0x7fae2c008040 comp rx=0 tx=0).stop 2023-12-19T17:01:13.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.665+0000 7fae492c4640 1 -- 172.21.15.26:0/297899973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae300039b0 msgr2=0x7fae440af180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.665+0000 7fae492c4640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae300039b0 0x7fae440af180 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fae3401c9e0 tx=0x7fae30005800 comp rx=0 tx=0).stop 2023-12-19T17:01:13.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.665+0000 7fae492c4640 1 -- 172.21.15.26:0/297899973 shutdown_connections 2023-12-19T17:01:13.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.665+0000 7fae492c4640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae24065bb0 0x7fae24068070 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.665+0000 7fae492c4640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4415b610 0x7fae44155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.665+0000 7fae492c4640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4415a620 0x7fae440ad6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.665+0000 7fae492c4640 1 --2- 172.21.15.26:0/297899973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae300039b0 0x7fae440af180 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.665+0000 7fae492c4640 1 -- 172.21.15.26:0/297899973 >> 172.21.15.26:0/297899973 conn(0x7fae4406ef60 msgr2=0x7fae4415fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:13.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.665+0000 7fae492c4640 1 -- 172.21.15.26:0/297899973 shutdown_connections 2023-12-19T17:01:13.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.665+0000 7fae492c4640 1 -- 172.21.15.26:0/297899973 wait complete. 2023-12-19T17:01:13.689 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-12-19T17:01:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c862d640 1 Processor -- start 2023-12-19T17:01:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c862d640 1 -- start start 2023-12-19T17:01:13.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c862d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f58c006cb30 0x7f58c00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c862d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58c015d2b0 0x7f58c015d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c862d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c00b2dd0 0x7f58c00b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c862d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c015ddc0 con 0x7f58c015d2b0 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c862d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c015e0a0 con 0x7f58c006cb30 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c862d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00cb450 con 0x7f58c00b2dd0 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c5bc8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58c015d2b0 0x7f58c015d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c5bc8640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58c015d2b0 0x7f58c015d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60570/0 (socket says 172.21.15.26:60570) 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c5bc8640 1 -- 172.21.15.26:0/2014999227 learned_addr learned my addr 172.21.15.26:0/2014999227 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c6bca640 1 --2- 172.21.15.26:0/2014999227 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c00b2dd0 0x7f58c00b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c5bc8640 1 -- 172.21.15.26:0/2014999227 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c00b2dd0 msgr2=0x7f58c00b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c63c9640 1 --2- 172.21.15.26:0/2014999227 >> v2:172.21.15.160:3300/0 conn(0x7f58c006cb30 0x7f58c00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c5bc8640 1 --2- 172.21.15.26:0/2014999227 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c00b2dd0 0x7f58c00b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c5bc8640 1 -- 172.21.15.26:0/2014999227 >> v2:172.21.15.160:3300/0 conn(0x7f58c006cb30 msgr2=0x7f58c00cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c5bc8640 1 --2- 172.21.15.26:0/2014999227 >> v2:172.21.15.160:3300/0 conn(0x7f58c006cb30 0x7f58c00cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c5bc8640 1 -- 172.21.15.26:0/2014999227 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c00cb730 con 0x7f58c015d2b0 2023-12-19T17:01:13.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c6bca640 1 --2- 172.21.15.26:0/2014999227 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c00b2dd0 0x7f58c00b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:13.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.829+0000 7f58c63c9640 1 --2- 172.21.15.26:0/2014999227 >> v2:172.21.15.160:3300/0 conn(0x7f58c006cb30 0x7f58c00cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:13.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.833+0000 7f58c5bc8640 1 --2- 172.21.15.26:0/2014999227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c015d2b0 0x7f58c015d730 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f58c00cf6a0 tx=0x7f58b001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6eb61c0f3136152 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:13.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.833+0000 7f58c53c7640 1 -- 172.21.15.26:0/2014999227 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b00ab640 con 0x7f58c015d2b0 2023-12-19T17:01:13.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.833+0000 7f58c53c7640 1 -- 172.21.15.26:0/2014999227 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b00a7070 con 0x7f58c015d2b0 2023-12-19T17:01:13.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.833+0000 7f58c53c7640 1 -- 172.21.15.26:0/2014999227 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b00ab9c0 con 0x7f58c015d2b0 2023-12-19T17:01:13.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.833+0000 7f58c862d640 1 -- 172.21.15.26:0/2014999227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c015d2b0 msgr2=0x7f58c015d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.833+0000 7f58c862d640 1 --2- 172.21.15.26:0/2014999227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c015d2b0 0x7f58c015d730 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f58c00cf6a0 tx=0x7f58b001c500 comp rx=0 tx=0).stop 2023-12-19T17:01:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.833+0000 7f58c862d640 1 -- 172.21.15.26:0/2014999227 shutdown_connections 2023-12-19T17:01:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.833+0000 7f58c862d640 1 --2- 172.21.15.26:0/2014999227 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c00b2dd0 0x7f58c00b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.833+0000 7f58c862d640 1 --2- 172.21.15.26:0/2014999227 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c015d2b0 0x7f58c015d730 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.833+0000 7f58c862d640 1 --2- 172.21.15.26:0/2014999227 >> v2:172.21.15.160:3300/0 conn(0x7f58c006cb30 0x7f58c00cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.833+0000 7f58c862d640 1 -- 172.21.15.26:0/2014999227 >> 172.21.15.26:0/2014999227 conn(0x7f58c00658e0 msgr2=0x7f58c0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.833+0000 7f58c862d640 1 -- 172.21.15.26:0/2014999227 shutdown_connections 2023-12-19T17:01:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.833+0000 7f58c862d640 1 -- 172.21.15.26:0/2014999227 wait complete. 2023-12-19T17:01:13.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c862d640 1 Processor -- start 2023-12-19T17:01:13.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c862d640 1 -- start start 2023-12-19T17:01:13.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c862d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c006cb30 0x7f58c0145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c862d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c00b2dd0 0x7f58c0168080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c862d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58c015d2b0 0x7f58c013e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c862d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c013e3c0 con 0x7f58c006cb30 2023-12-19T17:01:13.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c862d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00cd240 con 0x7f58c015d2b0 2023-12-19T17:01:13.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c862d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c01f7650 con 0x7f58c00b2dd0 2023-12-19T17:01:13.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c63c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c006cb30 0x7f58c0145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c63c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c006cb30 0x7f58c0145430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60572/0 (socket says 172.21.15.26:60572) 2023-12-19T17:01:13.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c63c9640 1 -- 172.21.15.26:0/3353517744 learned_addr learned my addr 172.21.15.26:0/3353517744 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:13.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c5bc8640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c00b2dd0 0x7f58c0168080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c6bca640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58c015d2b0 0x7f58c013e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c63c9640 1 -- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c00b2dd0 msgr2=0x7f58c0168080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c63c9640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c00b2dd0 0x7f58c0168080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c63c9640 1 -- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58c015d2b0 msgr2=0x7f58c013e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:13.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c63c9640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58c015d2b0 0x7f58c013e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:13.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c63c9640 1 -- 172.21.15.26:0/3353517744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b00094d0 con 0x7f58c006cb30 2023-12-19T17:01:13.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c6bca640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58c015d2b0 0x7f58c013e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:13.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c63c9640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c006cb30 0x7f58c0145430 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00c7f0 tx=0x7f58bc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:13.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58b6ffd640 1 -- 172.21.15.26:0/3353517744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00cea0 con 0x7f58c006cb30 2023-12-19T17:01:13.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58b6ffd640 1 -- 172.21.15.26:0/3353517744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc007a30 con 0x7f58c006cb30 2023-12-19T17:01:13.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c862d640 1 -- 172.21.15.26:0/3353517744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c01f7930 con 0x7f58c006cb30 2023-12-19T17:01:13.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58b6ffd640 1 -- 172.21.15.26:0/3353517744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc007de0 con 0x7f58c006cb30 2023-12-19T17:01:13.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c862d640 1 -- 172.21.15.26:0/3353517744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c01f7ef0 con 0x7f58c006cb30 2023-12-19T17:01:13.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.837+0000 7f58c862d640 1 -- 172.21.15.26:0/3353517744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c0074270 con 0x7f58c006cb30 2023-12-19T17:01:13.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.841+0000 7f58b6ffd640 1 -- 172.21.15.26:0/3353517744 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f58bc0113f0 con 0x7f58c006cb30 2023-12-19T17:01:13.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.841+0000 7f58b6ffd640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f58a4065b80 0x7f58a4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:13.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.841+0000 7f58b6ffd640 1 -- 172.21.15.26:0/3353517744 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f58bc083740 con 0x7f58c006cb30 2023-12-19T17:01:13.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.841+0000 7f58c5bc8640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f58a4065b80 0x7f58a4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:13.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.841+0000 7f58c5bc8640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f58a4065b80 0x7f58a4068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f58b00af000 tx=0x7f58b00a7000 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:13.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:13.845+0000 7f58b6ffd640 1 -- 172.21.15.26:0/3353517744 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f58bc050cd0 con 0x7f58c006cb30 2023-12-19T17:01:14.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.057+0000 7f58c862d640 1 -- 172.21.15.26:0/3353517744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f58c00acf60 con 0x7f58c006cb30 2023-12-19T17:01:14.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.637+0000 7f58b6ffd640 1 -- 172.21.15.26:0/3353517744 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v711) v1 ==== 148+0+0 (secure 0 0 0) 0x7f58bc0542f0 con 0x7f58c006cb30 2023-12-19T17:01:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.637+0000 7f58c862d640 1 -- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f58a4065b80 msgr2=0x7f58a4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.637+0000 7f58c862d640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f58a4065b80 0x7f58a4068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f58b00af000 tx=0x7f58b00a7000 comp rx=0 tx=0).stop 2023-12-19T17:01:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.637+0000 7f58c862d640 1 -- 172.21.15.26:0/3353517744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c006cb30 msgr2=0x7f58c0145430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.637+0000 7f58c862d640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c006cb30 0x7f58c0145430 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00c7f0 tx=0x7f58bc00ccb0 comp rx=0 tx=0).stop 2023-12-19T17:01:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.637+0000 7f58c862d640 1 -- 172.21.15.26:0/3353517744 shutdown_connections 2023-12-19T17:01:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.637+0000 7f58c862d640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58c015d2b0 0x7f58c013e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.637+0000 7f58c862d640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f58a4065b80 0x7f58a4068040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.637+0000 7f58c862d640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c00b2dd0 0x7f58c0168080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.637+0000 7f58c862d640 1 --2- 172.21.15.26:0/3353517744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c006cb30 0x7f58c0145430 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.637+0000 7f58c862d640 1 -- 172.21.15.26:0/3353517744 >> 172.21.15.26:0/3353517744 conn(0x7f58c00658e0 msgr2=0x7f58c00cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:14.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.641+0000 7f58c862d640 1 -- 172.21.15.26:0/3353517744 shutdown_connections 2023-12-19T17:01:14.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.641+0000 7f58c862d640 1 -- 172.21.15.26:0/3353517744 wait complete. 2023-12-19T17:01:14.667 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-12-19T17:01:14.667 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-12-19T17:01:14.667 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-19T17:01:14.668 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-12-19T17:01:14.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.809+0000 7f7ab4855640 1 Processor -- start 2023-12-19T17:01:14.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.809+0000 7f7ab4855640 1 -- start start 2023-12-19T17:01:14.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.813+0000 7f7ab4855640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7ab0154220 0x7f7ab015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:14.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.813+0000 7f7ab4855640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab006bc30 con 0x7f7ab006b3d0 2023-12-19T17:01:14.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.813+0000 7f7ab4855640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab00cc6e0 con 0x7f7ab00cee10 2023-12-19T17:01:14.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.813+0000 7f7ab4855640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab015afc0 con 0x7f7ab0154220 2023-12-19T17:01:14.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.813+0000 7f7aae59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7ab0154220 0x7f7ab015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:14.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.813+0000 7f7aa7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7ab006b3d0 0x7f7ab00ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41568/0 (socket says 172.21.15.26:41568) 2023-12-19T17:01:14.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.813+0000 7f7aa7fff640 1 -- 172.21.15.26:0/839597640 learned_addr learned my addr 172.21.15.26:0/839597640 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:14.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.813+0000 7f7aae59c640 1 -- 172.21.15.26:0/839597640 >> v1:172.21.15.160:6789/0 conn(0x7f7ab00cee10 legacy=0x7f7ab015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:14.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.813+0000 7f7aae59c640 1 -- 172.21.15.26:0/839597640 >> v1:172.21.15.26:6789/0 conn(0x7f7ab006b3d0 legacy=0x7f7ab00ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:14.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.813+0000 7f7aae59c640 1 -- 172.21.15.26:0/839597640 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab006b950 con 0x7f7ab0154220 2023-12-19T17:01:14.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.817+0000 7f7aae59c640 1 --2- 172.21.15.26:0/839597640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab0154220 0x7f7ab015aa80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00097c0 tx=0x7f7aa001e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9885f2c4a53400 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:14.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.817+0000 7f7aadd9b640 1 -- 172.21.15.26:0/839597640 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00a8630 con 0x7f7ab0154220 2023-12-19T17:01:14.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.817+0000 7f7aadd9b640 1 -- 172.21.15.26:0/839597640 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa00a8790 con 0x7f7ab0154220 2023-12-19T17:01:14.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.817+0000 7f7aadd9b640 1 -- 172.21.15.26:0/839597640 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00a8b20 con 0x7f7ab0154220 2023-12-19T17:01:14.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.817+0000 7f7ab4855640 1 -- 172.21.15.26:0/839597640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab0154220 msgr2=0x7f7ab015aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:14.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.817+0000 7f7ab4855640 1 --2- 172.21.15.26:0/839597640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab0154220 0x7f7ab015aa80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00097c0 tx=0x7f7aa001e660 comp rx=0 tx=0).stop 2023-12-19T17:01:14.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.817+0000 7f7ab4855640 1 -- 172.21.15.26:0/839597640 shutdown_connections 2023-12-19T17:01:14.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.817+0000 7f7ab4855640 1 --2- 172.21.15.26:0/839597640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab0154220 0x7f7ab015aa80 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:14.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.817+0000 7f7ab4855640 1 -- 172.21.15.26:0/839597640 >> 172.21.15.26:0/839597640 conn(0x7f7ab00658e0 msgr2=0x7f7ab0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:14.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.817+0000 7f7ab4855640 1 -- 172.21.15.26:0/839597640 shutdown_connections 2023-12-19T17:01:14.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.817+0000 7f7ab4855640 1 -- 172.21.15.26:0/839597640 wait complete. 2023-12-19T17:01:14.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.817+0000 7f7ab4855640 1 Processor -- start 2023-12-19T17:01:14.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.817+0000 7f7ab4855640 1 -- start start 2023-12-19T17:01:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7ab4855640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab006b3d0 0x7f7ab0157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7ab4855640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab00cee10 0x7f7ab014d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7ab4855640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ab0154220 0x7f7ab014a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7ab4855640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab014ce10 con 0x7f7ab006b3d0 2023-12-19T17:01:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7ab4855640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab014dd40 con 0x7f7ab00cee10 2023-12-19T17:01:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7ab4855640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab01575a0 con 0x7f7ab0154220 2023-12-19T17:01:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aae59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab006b3d0 0x7f7ab0157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aae59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab006b3d0 0x7f7ab0157060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60582/0 (socket says 172.21.15.26:60582) 2023-12-19T17:01:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aae59c640 1 -- 172.21.15.26:0/1664683958 learned_addr learned my addr 172.21.15.26:0/1664683958 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aae59c640 1 -- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ab0154220 msgr2=0x7f7ab014a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aaed9d640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ab0154220 0x7f7ab014a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aa7fff640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab00cee10 0x7f7ab014d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aae59c640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ab0154220 0x7f7ab014a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aae59c640 1 -- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab00cee10 msgr2=0x7f7ab014d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aae59c640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab00cee10 0x7f7ab014d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aae59c640 1 -- 172.21.15.26:0/1664683958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa0009420 con 0x7f7ab006b3d0 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aaed9d640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ab0154220 0x7f7ab014a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aa7fff640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab00cee10 0x7f7ab014d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aae59c640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab006b3d0 0x7f7ab0157060 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7aa001c290 tx=0x7f7aa001b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aa6ffd640 1 -- 172.21.15.26:0/1664683958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00a9820 con 0x7f7ab006b3d0 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aa6ffd640 1 -- 172.21.15.26:0/1664683958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa00a99a0 con 0x7f7ab006b3d0 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7aa6ffd640 1 -- 172.21.15.26:0/1664683958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00a9d50 con 0x7f7ab006b3d0 2023-12-19T17:01:14.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7ab4855640 1 -- 172.21.15.26:0/1664683958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab00b25c0 con 0x7f7ab006b3d0 2023-12-19T17:01:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7ab4855640 1 -- 172.21.15.26:0/1664683958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab014c7c0 con 0x7f7ab006b3d0 2023-12-19T17:01:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.821+0000 7f7ab4855640 1 -- 172.21.15.26:0/1664683958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab014c620 con 0x7f7ab006b3d0 2023-12-19T17:01:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.825+0000 7f7aa6ffd640 1 -- 172.21.15.26:0/1664683958 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7aa00b2400 con 0x7f7ab006b3d0 2023-12-19T17:01:14.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.825+0000 7f7aa6ffd640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a90065b80 0x7f7a90068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:14.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.825+0000 7f7aa6ffd640 1 -- 172.21.15.26:0/1664683958 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f7aa011dba0 con 0x7f7ab006b3d0 2023-12-19T17:01:14.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.825+0000 7f7aa7fff640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a90065b80 0x7f7a90068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:14.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.825+0000 7f7aa7fff640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a90065b80 0x7f7a90068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7ab01f88b0 tx=0x7f7a98009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:14.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:14.829+0000 7f7aa6ffd640 1 -- 172.21.15.26:0/1664683958 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7ab014c620 con 0x7f7ab006b3d0 2023-12-19T17:01:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.037+0000 7f7ab4855640 1 -- 172.21.15.26:0/1664683958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f7ab014c620 con 0x7f7ab006b3d0 2023-12-19T17:01:15.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.649+0000 7f7aa6ffd640 1 -- 172.21.15.26:0/1664683958 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v712) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7ab014c620 con 0x7f7ab006b3d0 2023-12-19T17:01:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.649+0000 7f7ab4855640 1 -- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a90065b80 msgr2=0x7f7a90068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.649+0000 7f7ab4855640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a90065b80 0x7f7a90068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7ab01f88b0 tx=0x7f7a98009040 comp rx=0 tx=0).stop 2023-12-19T17:01:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.649+0000 7f7ab4855640 1 -- 172.21.15.26:0/1664683958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab006b3d0 msgr2=0x7f7ab0157060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.649+0000 7f7ab4855640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab006b3d0 0x7f7ab0157060 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f7aa001c290 tx=0x7f7aa001b400 comp rx=0 tx=0).stop 2023-12-19T17:01:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.649+0000 7f7ab4855640 1 -- 172.21.15.26:0/1664683958 shutdown_connections 2023-12-19T17:01:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.649+0000 7f7ab4855640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7a90065b80 0x7f7a90068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.649+0000 7f7ab4855640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ab0154220 0x7f7ab014a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.649+0000 7f7ab4855640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab00cee10 0x7f7ab014d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.649+0000 7f7ab4855640 1 --2- 172.21.15.26:0/1664683958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab006b3d0 0x7f7ab0157060 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.649+0000 7f7ab4855640 1 -- 172.21.15.26:0/1664683958 >> 172.21.15.26:0/1664683958 conn(0x7f7ab00658e0 msgr2=0x7f7ab00cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.649+0000 7f7ab4855640 1 -- 172.21.15.26:0/1664683958 shutdown_connections 2023-12-19T17:01:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.649+0000 7f7ab4855640 1 -- 172.21.15.26:0/1664683958 wait complete. 2023-12-19T17:01:15.677 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T17:01:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.821+0000 7f56fa82a640 1 Processor -- start 2023-12-19T17:01:15.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.821+0000 7f56fa82a640 1 -- start start 2023-12-19T17:01:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56fa82a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56f415ab40 0x7f56f406b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56fa82a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f56f406f0a0 0x7f56f406b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56fa82a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f415af20 con 0x7f56f40afe70 2023-12-19T17:01:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56fa82a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f40aea90 con 0x7f56f415ab40 2023-12-19T17:01:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56fa82a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f40aed70 con 0x7f56f406f0a0 2023-12-19T17:01:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56fa029640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f56f40afe70 0x7f56f415e600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41576/0 (socket says 172.21.15.26:41576) 2023-12-19T17:01:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56fa029640 1 -- 172.21.15.26:0/3722549892 learned_addr learned my addr 172.21.15.26:0/3722549892 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56f9828640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56f415ab40 0x7f56f406b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56f9027640 1 --2- 172.21.15.26:0/3722549892 >> v2:172.21.15.160:3300/0 conn(0x7f56f406f0a0 0x7f56f406b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56f8826640 1 -- 172.21.15.26:0/3722549892 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746094301 0 0) 0x7f56f415af20 con 0x7f56f40afe70 2023-12-19T17:01:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56f8826640 1 -- 172.21.15.26:0/3722549892 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56e0003530 con 0x7f56f40afe70 2023-12-19T17:01:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56f9828640 1 -- 172.21.15.26:0/3722549892 >> v2:172.21.15.160:3300/0 conn(0x7f56f406f0a0 msgr2=0x7f56f406b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56f9828640 1 --2- 172.21.15.26:0/3722549892 >> v2:172.21.15.160:3300/0 conn(0x7f56f406f0a0 0x7f56f406b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56f9828640 1 -- 172.21.15.26:0/3722549892 >> v1:172.21.15.26:6789/0 conn(0x7f56f40afe70 legacy=0x7f56f415e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.825+0000 7f56f9828640 1 -- 172.21.15.26:0/3722549892 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f40af050 con 0x7f56f415ab40 2023-12-19T17:01:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56f9828640 1 --2- 172.21.15.26:0/3722549892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f415ab40 0x7f56f406b3b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f56f4066860 tx=0x7f56f001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc6fda6e3573a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56f8826640 1 -- 172.21.15.26:0/3722549892 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0004230 con 0x7f56f415ab40 2023-12-19T17:01:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56f8826640 1 -- 172.21.15.26:0/3722549892 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f00a7050 con 0x7f56f415ab40 2023-12-19T17:01:15.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56f8826640 1 -- 172.21.15.26:0/3722549892 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f0003620 con 0x7f56f415ab40 2023-12-19T17:01:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56fa82a640 1 -- 172.21.15.26:0/3722549892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f415ab40 msgr2=0x7f56f406b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56fa82a640 1 --2- 172.21.15.26:0/3722549892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f415ab40 0x7f56f406b3b0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f56f4066860 tx=0x7f56f001c460 comp rx=0 tx=0).stop 2023-12-19T17:01:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56fa82a640 1 -- 172.21.15.26:0/3722549892 shutdown_connections 2023-12-19T17:01:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56fa82a640 1 --2- 172.21.15.26:0/3722549892 >> v2:172.21.15.160:3300/0 conn(0x7f56f406f0a0 0x7f56f406b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56fa82a640 1 --2- 172.21.15.26:0/3722549892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f415ab40 0x7f56f406b3b0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56fa82a640 1 -- 172.21.15.26:0/3722549892 >> 172.21.15.26:0/3722549892 conn(0x7f56f40658e0 msgr2=0x7f56f4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56fa82a640 1 -- 172.21.15.26:0/3722549892 shutdown_connections 2023-12-19T17:01:15.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56fa82a640 1 -- 172.21.15.26:0/3722549892 wait complete. 2023-12-19T17:01:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56fa82a640 1 Processor -- start 2023-12-19T17:01:15.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.829+0000 7f56fa82a640 1 -- start start 2023-12-19T17:01:15.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56fa82a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56f406f0a0 0x7f56f415c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:15.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56fa82a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f40afe70 0x7f56f415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:15.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56fa82a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56f415ab40 0x7f56f4152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:15.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56fa82a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f4154480 con 0x7f56f40afe70 2023-12-19T17:01:15.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56fa82a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f40a8770 con 0x7f56f415ab40 2023-12-19T17:01:15.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56fa82a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f40a8440 con 0x7f56f406f0a0 2023-12-19T17:01:15.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56f9027640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f40afe70 0x7f56f415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:15.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56f9027640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f40afe70 0x7f56f415bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60596/0 (socket says 172.21.15.26:60596) 2023-12-19T17:01:15.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56f9027640 1 -- 172.21.15.26:0/2191583308 learned_addr learned my addr 172.21.15.26:0/2191583308 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:15.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56f9828640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56f406f0a0 0x7f56f415c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:15.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56fa029640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56f415ab40 0x7f56f4152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56f9027640 1 -- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56f406f0a0 msgr2=0x7f56f415c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56f9027640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56f406f0a0 0x7f56f415c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56f9027640 1 -- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56f415ab40 msgr2=0x7f56f4152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56f9027640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56f415ab40 0x7f56f4152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56f9027640 1 -- 172.21.15.26:0/2191583308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f0009420 con 0x7f56f40afe70 2023-12-19T17:01:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56fa029640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56f415ab40 0x7f56f4152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56f9828640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56f406f0a0 0x7f56f415c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56f9027640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f40afe70 0x7f56f415bdb0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f56e400e8a0 tx=0x7f56e400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56ea7fc640 1 -- 172.21.15.26:0/2191583308 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e4009510 con 0x7f56f40afe70 2023-12-19T17:01:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56ea7fc640 1 -- 172.21.15.26:0/2191583308 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e400c940 con 0x7f56f40afe70 2023-12-19T17:01:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56ea7fc640 1 -- 172.21.15.26:0/2191583308 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e400ccb0 con 0x7f56f40afe70 2023-12-19T17:01:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56fa82a640 1 -- 172.21.15.26:0/2191583308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f41546d0 con 0x7f56f40afe70 2023-12-19T17:01:15.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56fa82a640 1 -- 172.21.15.26:0/2191583308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f4154c90 con 0x7f56f40afe70 2023-12-19T17:01:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56fa82a640 1 -- 172.21.15.26:0/2191583308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f40a8900 con 0x7f56f40afe70 2023-12-19T17:01:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.833+0000 7f56ea7fc640 1 -- 172.21.15.26:0/2191583308 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f56e4013470 con 0x7f56f40afe70 2023-12-19T17:01:15.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.837+0000 7f56ea7fc640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f56d8065bb0 0x7f56d8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.837+0000 7f56ea7fc640 1 -- 172.21.15.26:0/2191583308 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f56e40835b0 con 0x7f56f40afe70 2023-12-19T17:01:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.837+0000 7f56f9828640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f56d8065bb0 0x7f56d8068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:15.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.837+0000 7f56f9828640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f56d8065bb0 0x7f56d8068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f56f00ad000 tx=0x7f56f00a6000 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:15.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.841+0000 7f56ea7fc640 1 -- 172.21.15.26:0/2191583308 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f56e4050c40 con 0x7f56f40afe70 2023-12-19T17:01:16.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:15.997+0000 7f56fa82a640 1 -- 172.21.15.26:0/2191583308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f56f4061930 con 0x7f56f40afe70 2023-12-19T17:01:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.005+0000 7f56ea7fc640 1 -- 172.21.15.26:0/2191583308 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f56e4054260 con 0x7f56f40afe70 2023-12-19T17:01:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.005+0000 7f56fa82a640 1 -- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f56d8065bb0 msgr2=0x7f56d8068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.005+0000 7f56fa82a640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f56d8065bb0 0x7f56d8068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f56f00ad000 tx=0x7f56f00a6000 comp rx=0 tx=0).stop 2023-12-19T17:01:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.005+0000 7f56fa82a640 1 -- 172.21.15.26:0/2191583308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f40afe70 msgr2=0x7f56f415bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.005+0000 7f56fa82a640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f40afe70 0x7f56f415bdb0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f56e400e8a0 tx=0x7f56e400eca0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.005+0000 7f56fa82a640 1 -- 172.21.15.26:0/2191583308 shutdown_connections 2023-12-19T17:01:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.005+0000 7f56fa82a640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56f415ab40 0x7f56f4152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.005+0000 7f56fa82a640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56f40afe70 0x7f56f415bdb0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.005+0000 7f56fa82a640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f56d8065bb0 0x7f56d8068070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.005+0000 7f56fa82a640 1 --2- 172.21.15.26:0/2191583308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56f406f0a0 0x7f56f415c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.005+0000 7f56fa82a640 1 -- 172.21.15.26:0/2191583308 >> 172.21.15.26:0/2191583308 conn(0x7f56f40658e0 msgr2=0x7f56f4065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.005+0000 7f56fa82a640 1 -- 172.21.15.26:0/2191583308 shutdown_connections 2023-12-19T17:01:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.005+0000 7f56fa82a640 1 -- 172.21.15.26:0/2191583308 wait complete. 2023-12-19T17:01:16.029 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T17:01:16.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T17:01:16.118 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T17:01:16.179 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T17:01:16.179 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:01:16.179 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:01:16.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T17:01:16.189 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T17:01:16.189 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:01:16.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T17:01:16.199 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:01:16.199 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:01:16.240 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T17:01:16.240 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T17:01:16.240 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:01:16.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T17:01:16.249 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T17:01:16.250 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:01:16.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.421+0000 7fe232bb3640 1 Processor -- start 2023-12-19T17:01:16.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.421+0000 7fe232bb3640 1 -- start start 2023-12-19T17:01:16.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe232bb3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe22c154220 0x7fe22c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:16.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe232bb3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe22c06b3d0 0x7fe22c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:16.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe232bb3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c06bc30 con 0x7fe22c0cec40 2023-12-19T17:01:16.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe232bb3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22c0cf0d0 con 0x7fe22c154220 2023-12-19T17:01:16.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe232bb3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c0cc6e0 con 0x7fe22c06b3d0 2023-12-19T17:01:16.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe23094f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe22c154220 0x7fe22c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:16.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe231150640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe22c0cec40 0x7fe22c15d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33244/0 (socket says 172.21.15.26:33244) 2023-12-19T17:01:16.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe231150640 1 -- 172.21.15.26:0/268871708 learned_addr learned my addr 172.21.15.26:0/268871708 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:16.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe22bfff640 1 --2- 172.21.15.26:0/268871708 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe22c06b3d0 0x7fe22c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:16.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe22b7fe640 1 -- 172.21.15.26:0/268871708 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025394426 0 0) 0x7fe22c06bc30 con 0x7fe22c0cec40 2023-12-19T17:01:16.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe22b7fe640 1 -- 172.21.15.26:0/268871708 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe21c003530 con 0x7fe22c0cec40 2023-12-19T17:01:16.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe23094f640 1 -- 172.21.15.26:0/268871708 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe22c06b3d0 msgr2=0x7fe22c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:16.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe23094f640 1 --2- 172.21.15.26:0/268871708 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe22c06b3d0 0x7fe22c0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe23094f640 1 -- 172.21.15.26:0/268871708 >> v1:172.21.15.26:6789/0 conn(0x7fe22c0cec40 legacy=0x7fe22c15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:16.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.425+0000 7fe23094f640 1 -- 172.21.15.26:0/268871708 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c15afc0 con 0x7fe22c154220 2023-12-19T17:01:16.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe23094f640 1 --2- 172.21.15.26:0/268871708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c154220 0x7fe22c15aa80 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fe2200097c0 tx=0x7fe22001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaaaa1e9a6310cd7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:16.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe22b7fe640 1 -- 172.21.15.26:0/268871708 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200ab600 con 0x7fe22c154220 2023-12-19T17:01:16.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe22b7fe640 1 -- 172.21.15.26:0/268871708 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2200a7050 con 0x7fe22c154220 2023-12-19T17:01:16.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe22b7fe640 1 -- 172.21.15.26:0/268871708 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200ab960 con 0x7fe22c154220 2023-12-19T17:01:16.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe232bb3640 1 -- 172.21.15.26:0/268871708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c154220 msgr2=0x7fe22c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:16.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe232bb3640 1 --2- 172.21.15.26:0/268871708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c154220 0x7fe22c15aa80 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fe2200097c0 tx=0x7fe22001c450 comp rx=0 tx=0).stop 2023-12-19T17:01:16.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe232bb3640 1 -- 172.21.15.26:0/268871708 shutdown_connections 2023-12-19T17:01:16.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe232bb3640 1 --2- 172.21.15.26:0/268871708 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe22c06b3d0 0x7fe22c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe232bb3640 1 --2- 172.21.15.26:0/268871708 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c154220 0x7fe22c15aa80 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe232bb3640 1 -- 172.21.15.26:0/268871708 >> 172.21.15.26:0/268871708 conn(0x7fe22c0658e0 msgr2=0x7fe22c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe232bb3640 1 -- 172.21.15.26:0/268871708 shutdown_connections 2023-12-19T17:01:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe232bb3640 1 -- 172.21.15.26:0/268871708 wait complete. 2023-12-19T17:01:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe232bb3640 1 Processor -- start 2023-12-19T17:01:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.429+0000 7fe232bb3640 1 -- start start 2023-12-19T17:01:16.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe232bb3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c06b3d0 0x7fe22c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:16.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe232bb3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe22c0cec40 0x7fe22c13b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:16.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe232bb3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe22c154220 0x7fe22c070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:16.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe232bb3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c13b770 con 0x7fe22c06b3d0 2023-12-19T17:01:16.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe232bb3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c0b2630 con 0x7fe22c154220 2023-12-19T17:01:16.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe232bb3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c0b28d0 con 0x7fe22c0cec40 2023-12-19T17:01:16.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe23094f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c06b3d0 0x7fe22c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:16.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe23094f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c06b3d0 0x7fe22c0b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40708/0 (socket says 172.21.15.26:40708) 2023-12-19T17:01:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe23094f640 1 -- 172.21.15.26:0/4287270010 learned_addr learned my addr 172.21.15.26:0/4287270010 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe22bfff640 1 --2- 172.21.15.26:0/4287270010 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe22c0cec40 0x7fe22c13b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe23094f640 1 -- 172.21.15.26:0/4287270010 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe22c0cec40 msgr2=0x7fe22c13b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe23094f640 1 --2- 172.21.15.26:0/4287270010 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe22c0cec40 0x7fe22c13b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe23094f640 1 -- 172.21.15.26:0/4287270010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe22c154220 msgr2=0x7fe22c070dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe23094f640 1 --2- 172.21.15.26:0/4287270010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe22c154220 0x7fe22c070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe23094f640 1 -- 172.21.15.26:0/4287270010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe220009420 con 0x7fe22c06b3d0 2023-12-19T17:01:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe23094f640 1 --2- 172.21.15.26:0/4287270010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c06b3d0 0x7fe22c0b3690 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fe22001ab00 tx=0x7fe2200191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe2297fa640 1 -- 172.21.15.26:0/4287270010 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200b4400 con 0x7fe22c06b3d0 2023-12-19T17:01:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe2297fa640 1 -- 172.21.15.26:0/4287270010 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2200b4560 con 0x7fe22c06b3d0 2023-12-19T17:01:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe2297fa640 1 -- 172.21.15.26:0/4287270010 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200b48f0 con 0x7fe22c06b3d0 2023-12-19T17:01:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe232bb3640 1 -- 172.21.15.26:0/4287270010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22c0b2bb0 con 0x7fe22c06b3d0 2023-12-19T17:01:16.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe232bb3640 1 -- 172.21.15.26:0/4287270010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22c0b3170 con 0x7fe22c06b3d0 2023-12-19T17:01:16.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe232bb3640 1 -- 172.21.15.26:0/4287270010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22c074270 con 0x7fe22c06b3d0 2023-12-19T17:01:16.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.433+0000 7fe2297fa640 1 -- 172.21.15.26:0/4287270010 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe2200b2040 con 0x7fe22c06b3d0 2023-12-19T17:01:16.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.437+0000 7fe2297fa640 1 --2- 172.21.15.26:0/4287270010 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe210065b80 0x7fe210068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:16.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.437+0000 7fe2297fa640 1 -- 172.21.15.26:0/4287270010 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6391+0+0 (secure 0 0 0) 0x7fe22011d470 con 0x7fe22c06b3d0 2023-12-19T17:01:16.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.437+0000 7fe22bfff640 1 --2- 172.21.15.26:0/4287270010 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe210065b80 0x7fe210068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:16.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.437+0000 7fe22bfff640 1 --2- 172.21.15.26:0/4287270010 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe210065b80 0x7fe210068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe22c067bf0 tx=0x7fe218008040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:16.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.441+0000 7fe2297fa640 1 -- 172.21.15.26:0/4287270010 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe2200ea960 con 0x7fe22c06b3d0 2023-12-19T17:01:16.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.629+0000 7fe232bb3640 1 -- 172.21.15.26:0/4287270010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe22c0b3330 con 0x7fe22c06b3d0 2023-12-19T17:01:16.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.633+0000 7fe2297fa640 1 -- 172.21.15.26:0/4287270010 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fe2200b0de0 con 0x7fe22c06b3d0 2023-12-19T17:01:16.636 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:16.637 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":59,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6884,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1869411616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1869411616},{"type":"v1","addr":"172.21.15.26:6835","nonce":1869411616}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7630,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3129570713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3129570713},{"type":"v1","addr":"172.21.15.160:6836","nonce":3129570713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7633,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4063618989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":4063618989},{"type":"v1","addr":"172.21.15.160:6837","nonce":4063618989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":8400,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/799618323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":799618323},{"type":"v1","addr":"172.21.15.26:6834","nonce":799618323}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[]} 2023-12-19T17:01:16.637 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 59 2023-12-19T17:01:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.633+0000 7fe232bb3640 1 -- 172.21.15.26:0/4287270010 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe210065b80 msgr2=0x7fe210068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.633+0000 7fe232bb3640 1 --2- 172.21.15.26:0/4287270010 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe210065b80 0x7fe210068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe22c067bf0 tx=0x7fe218008040 comp rx=0 tx=0).stop 2023-12-19T17:01:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.633+0000 7fe232bb3640 1 -- 172.21.15.26:0/4287270010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c06b3d0 msgr2=0x7fe22c0b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.633+0000 7fe232bb3640 1 --2- 172.21.15.26:0/4287270010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c06b3d0 0x7fe22c0b3690 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fe22001ab00 tx=0x7fe2200191f0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.633+0000 7fe232bb3640 1 -- 172.21.15.26:0/4287270010 shutdown_connections 2023-12-19T17:01:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.633+0000 7fe232bb3640 1 --2- 172.21.15.26:0/4287270010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe22c154220 0x7fe22c070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.633+0000 7fe232bb3640 1 --2- 172.21.15.26:0/4287270010 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe210065b80 0x7fe210068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.633+0000 7fe232bb3640 1 --2- 172.21.15.26:0/4287270010 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe22c0cec40 0x7fe22c13b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.633+0000 7fe232bb3640 1 --2- 172.21.15.26:0/4287270010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe22c06b3d0 0x7fe22c0b3690 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.633+0000 7fe232bb3640 1 -- 172.21.15.26:0/4287270010 >> 172.21.15.26:0/4287270010 conn(0x7fe22c0658e0 msgr2=0x7fe22c0adbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:16.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.633+0000 7fe232bb3640 1 -- 172.21.15.26:0/4287270010 shutdown_connections 2023-12-19T17:01:16.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.633+0000 7fe232bb3640 1 -- 172.21.15.26:0/4287270010 wait complete. 2023-12-19T17:01:16.658 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T17:01:16.658 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T17:01:16.658 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T17:01:16.658 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T17:01:16.659 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T17:01:16.659 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T17:01:16.659 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T17:01:16.659 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T17:01:16.659 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T17:01:16.659 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T17:01:16.659 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T17:01:16.660 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T17:01:16.660 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T17:01:16.660 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T17:01:16.660 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T17:01:16.660 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T17:01:16.670 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T17:01:16.670 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T17:01:16.671 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T17:01:16.672 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T17:01:16.674 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T17:01:16.675 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T17:01:16.675 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T17:01:16.675 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T17:01:16.715 INFO:tasks.ceph.mds.c:Started 2023-12-19T17:01:16.715 DEBUG:teuthology.parallel:result is None 2023-12-19T17:01:16.717 INFO:tasks.ceph.mds.a:Started 2023-12-19T17:01:16.717 DEBUG:teuthology.parallel:result is None 2023-12-19T17:01:16.719 INFO:tasks.ceph.mds.d:Started 2023-12-19T17:01:16.719 DEBUG:teuthology.parallel:result is None 2023-12-19T17:01:16.721 INFO:tasks.ceph.mds.b:Started 2023-12-19T17:01:16.721 DEBUG:teuthology.parallel:result is None 2023-12-19T17:01:16.722 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T17:01:16.803 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T17:01:16.805 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T17:01:16.819 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T17:01:16.822 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T17:01:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.869+0000 7ff200bf0640 1 Processor -- start 2023-12-19T17:01:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.869+0000 7ff200bf0640 1 -- start start 2023-12-19T17:01:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.869+0000 7ff200bf0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1fc0cee50 0x7ff1fc15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:16.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.869+0000 7ff200bf0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1fc06c970 con 0x7ff1fc15f370 2023-12-19T17:01:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.869+0000 7ff200bf0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1fc15b1f0 con 0x7ff1fc155030 2023-12-19T17:01:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.869+0000 7ff200bf0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1fc06c790 con 0x7ff1fc0cee50 2023-12-19T17:01:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.869+0000 7ff1f3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff1fc15f370 0x7ff1fc0ce740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33284/0 (socket says 172.21.15.26:33284) 2023-12-19T17:01:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.869+0000 7ff1f3fff640 1 -- 172.21.15.26:0/2881428522 learned_addr learned my addr 172.21.15.26:0/2881428522 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.869+0000 7ff1fad9d640 1 --2- 172.21.15.26:0/2881428522 >> v2:172.21.15.26:3300/0 conn(0x7ff1fc0cee50 0x7ff1fc15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:16.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.873+0000 7ff1f9d9b640 1 -- 172.21.15.26:0/2881428522 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648955891 0 0) 0x7ff1fc06c970 con 0x7ff1fc15f370 2023-12-19T17:01:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.873+0000 7ff1f9d9b640 1 -- 172.21.15.26:0/2881428522 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1e8003530 con 0x7ff1fc15f370 2023-12-19T17:01:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.873+0000 7ff1fad9d640 1 -- 172.21.15.26:0/2881428522 >> v1:172.21.15.160:6789/0 conn(0x7ff1fc155030 legacy=0x7ff1fc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.873+0000 7ff1fad9d640 1 -- 172.21.15.26:0/2881428522 >> v1:172.21.15.26:6789/0 conn(0x7ff1fc15f370 legacy=0x7ff1fc0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.873+0000 7ff1fad9d640 1 -- 172.21.15.26:0/2881428522 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1fc15b4d0 con 0x7ff1fc0cee50 2023-12-19T17:01:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.873+0000 7ff1fad9d640 1 --2- 172.21.15.26:0/2881428522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1fc0cee50 0x7ff1fc15bcd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009d90 tx=0x7ff1f401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86af6d92b3cb003 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.873+0000 7ff1f9d9b640 1 -- 172.21.15.26:0/2881428522 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f40a5a00 con 0x7ff1fc0cee50 2023-12-19T17:01:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.873+0000 7ff1f9d9b640 1 -- 172.21.15.26:0/2881428522 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f40a5b60 con 0x7ff1fc0cee50 2023-12-19T17:01:16.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.873+0000 7ff1f9d9b640 1 -- 172.21.15.26:0/2881428522 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f40ad440 con 0x7ff1fc0cee50 2023-12-19T17:01:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 -- 172.21.15.26:0/2881428522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1fc0cee50 msgr2=0x7ff1fc15bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 --2- 172.21.15.26:0/2881428522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1fc0cee50 0x7ff1fc15bcd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009d90 tx=0x7ff1f401c920 comp rx=0 tx=0).stop 2023-12-19T17:01:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 -- 172.21.15.26:0/2881428522 shutdown_connections 2023-12-19T17:01:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 --2- 172.21.15.26:0/2881428522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1fc0cee50 0x7ff1fc15bcd0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 -- 172.21.15.26:0/2881428522 >> 172.21.15.26:0/2881428522 conn(0x7ff1fc0658e0 msgr2=0x7ff1fc065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 -- 172.21.15.26:0/2881428522 shutdown_connections 2023-12-19T17:01:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 -- 172.21.15.26:0/2881428522 wait complete. 2023-12-19T17:01:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 Processor -- start 2023-12-19T17:01:16.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 -- start start 2023-12-19T17:01:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1fc0cee50 0x7ff1fc14bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:16.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1fc155030 0x7ff1fc0b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1fc15f370 0x7ff1fc0b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc14c100 con 0x7ff1fc15f370 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc0b12d0 con 0x7ff1fc0cee50 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff200bf0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1fc0ae210 con 0x7ff1fc155030 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff1fad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1fc15f370 0x7ff1fc0b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff1fad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1fc15f370 0x7ff1fc0b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40748/0 (socket says 172.21.15.26:40748) 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff1fad9d640 1 -- 172.21.15.26:0/363555837 learned_addr learned my addr 172.21.15.26:0/363555837 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff1fa59c640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1fc0cee50 0x7ff1fc14bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff1f3fff640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1fc155030 0x7ff1fc0b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff1fad9d640 1 -- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1fc155030 msgr2=0x7ff1fc0b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff1fad9d640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1fc155030 0x7ff1fc0b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff1fad9d640 1 -- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1fc0cee50 msgr2=0x7ff1fc14bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff1fad9d640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1fc0cee50 0x7ff1fc14bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:16.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff1fad9d640 1 -- 172.21.15.26:0/363555837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f4009a70 con 0x7ff1fc15f370 2023-12-19T17:01:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff1fa59c640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1fc0cee50 0x7ff1fc14bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.877+0000 7ff1f3fff640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1fc155030 0x7ff1fc0b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.881+0000 7ff1fad9d640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1fc15f370 0x7ff1fc0b2400 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4019530 tx=0x7ff1f40196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.881+0000 7ff1f2ffd640 1 -- 172.21.15.26:0/363555837 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f40aee00 con 0x7ff1fc15f370 2023-12-19T17:01:16.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.881+0000 7ff1f2ffd640 1 -- 172.21.15.26:0/363555837 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f4009450 con 0x7ff1fc15f370 2023-12-19T17:01:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.881+0000 7ff1f2ffd640 1 -- 172.21.15.26:0/363555837 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4003bc0 con 0x7ff1fc15f370 2023-12-19T17:01:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.881+0000 7ff200bf0640 1 -- 172.21.15.26:0/363555837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1fc152cc0 con 0x7ff1fc15f370 2023-12-19T17:01:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.881+0000 7ff200bf0640 1 -- 172.21.15.26:0/363555837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1fc153280 con 0x7ff1fc15f370 2023-12-19T17:01:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.881+0000 7ff200bf0640 1 -- 172.21.15.26:0/363555837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1fc0ae3a0 con 0x7ff1fc15f370 2023-12-19T17:01:16.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.881+0000 7ff1f2ffd640 1 -- 172.21.15.26:0/363555837 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff1f40b4050 con 0x7ff1fc15f370 2023-12-19T17:01:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.881+0000 7ff1f2ffd640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff1dc065b80 0x7ff1dc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.881+0000 7ff1f2ffd640 1 -- 172.21.15.26:0/363555837 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6363+0+0 (secure 0 0 0) 0x7ff1f411dc60 con 0x7ff1fc15f370 2023-12-19T17:01:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.881+0000 7ff1fa59c640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff1dc065b80 0x7ff1dc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:16.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.885+0000 7ff1fa59c640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff1dc065b80 0x7ff1dc068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec001710 tx=0x7ff1ec009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:16.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:16.885+0000 7ff1f2ffd640 1 -- 172.21.15.26:0/363555837 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff1f40eb1e0 con 0x7ff1fc15f370 2023-12-19T17:01:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.053+0000 7ff200bf0640 1 -- 172.21.15.26:0/363555837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff1fc074270 con 0x7ff1fc15f370 2023-12-19T17:01:17.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.861+0000 7ff1f2ffd640 1 -- 172.21.15.26:0/363555837 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v133) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff1f40b2080 con 0x7ff1fc15f370 2023-12-19T17:01:17.865 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2023-12-19T17:01:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.861+0000 7ff200bf0640 1 -- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff1dc065b80 msgr2=0x7ff1dc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.861+0000 7ff200bf0640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff1dc065b80 0x7ff1dc068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec001710 tx=0x7ff1ec009040 comp rx=0 tx=0).stop 2023-12-19T17:01:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.861+0000 7ff200bf0640 1 -- 172.21.15.26:0/363555837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1fc15f370 msgr2=0x7ff1fc0b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.861+0000 7ff200bf0640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1fc15f370 0x7ff1fc0b2400 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4019530 tx=0x7ff1f40196c0 comp rx=0 tx=0).stop 2023-12-19T17:01:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.865+0000 7ff200bf0640 1 -- 172.21.15.26:0/363555837 shutdown_connections 2023-12-19T17:01:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.865+0000 7ff200bf0640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1fc15f370 0x7ff1fc0b2400 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.865+0000 7ff200bf0640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff1dc065b80 0x7ff1dc068040 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.865+0000 7ff200bf0640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1fc155030 0x7ff1fc0b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.865+0000 7ff200bf0640 1 --2- 172.21.15.26:0/363555837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1fc0cee50 0x7ff1fc14bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.865+0000 7ff200bf0640 1 -- 172.21.15.26:0/363555837 >> 172.21.15.26:0/363555837 conn(0x7ff1fc0658e0 msgr2=0x7ff1fc065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.865+0000 7ff200bf0640 1 -- 172.21.15.26:0/363555837 shutdown_connections 2023-12-19T17:01:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:17.865+0000 7ff200bf0640 1 -- 172.21.15.26:0/363555837 wait complete. 2023-12-19T17:01:17.892 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T17:01:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.033+0000 7f2f243ce640 1 Processor -- start 2023-12-19T17:01:18.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.033+0000 7f2f243ce640 1 -- start start 2023-12-19T17:01:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f243ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2f1c154220 0x7f2f1c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f243ce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2f1c06b3d0 0x7f2f1c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f243ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f1c0cec40 0x7f2f1c15d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f243ce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1c0cf0d0 con 0x7f2f1c154220 2023-12-19T17:01:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f243ce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f1c06bc30 con 0x7f2f1c06b3d0 2023-12-19T17:01:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f243ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0cc6e0 con 0x7f2f1c0cec40 2023-12-19T17:01:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f2216a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2f1c154220 0x7f2f1c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f2216a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2f1c154220 0x7f2f1c15aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40754/0 (socket says 172.21.15.26:40754) 2023-12-19T17:01:18.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f2216a640 1 -- 172.21.15.26:0/3450640105 learned_addr learned my addr 172.21.15.26:0/3450640105 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:18.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f2296b640 1 --2- 172.21.15.26:0/3450640105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f1c0cec40 0x7f2f1c15d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f21969640 1 --2- 172.21.15.26:0/3450640105 >> v2:172.21.15.160:3300/0 conn(0x7f2f1c06b3d0 0x7f2f1c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f2216a640 1 -- 172.21.15.26:0/3450640105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f1c0cec40 msgr2=0x7f2f1c15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:18.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f2216a640 1 --2- 172.21.15.26:0/3450640105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f1c0cec40 0x7f2f1c15d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f2216a640 1 -- 172.21.15.26:0/3450640105 >> v2:172.21.15.160:3300/0 conn(0x7f2f1c06b3d0 msgr2=0x7f2f1c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:18.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f2216a640 1 --2- 172.21.15.26:0/3450640105 >> v2:172.21.15.160:3300/0 conn(0x7f2f1c06b3d0 0x7f2f1c0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f2216a640 1 -- 172.21.15.26:0/3450640105 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c15afc0 con 0x7f2f1c154220 2023-12-19T17:01:18.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.037+0000 7f2f2216a640 1 --2- 172.21.15.26:0/3450640105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f1c154220 0x7f2f1c15aa80 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0cf880 tx=0x7f2f1801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c91ac91eda57f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:18.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f21168640 1 -- 172.21.15.26:0/3450640105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180ab600 con 0x7f2f1c154220 2023-12-19T17:01:18.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f21168640 1 -- 172.21.15.26:0/3450640105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f180a7050 con 0x7f2f1c154220 2023-12-19T17:01:18.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f21168640 1 -- 172.21.15.26:0/3450640105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180ab980 con 0x7f2f1c154220 2023-12-19T17:01:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 -- 172.21.15.26:0/3450640105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f1c154220 msgr2=0x7f2f1c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 --2- 172.21.15.26:0/3450640105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f1c154220 0x7f2f1c15aa80 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0cf880 tx=0x7f2f1801c210 comp rx=0 tx=0).stop 2023-12-19T17:01:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 -- 172.21.15.26:0/3450640105 shutdown_connections 2023-12-19T17:01:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 --2- 172.21.15.26:0/3450640105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f1c0cec40 0x7f2f1c15d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 --2- 172.21.15.26:0/3450640105 >> v2:172.21.15.160:3300/0 conn(0x7f2f1c06b3d0 0x7f2f1c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 --2- 172.21.15.26:0/3450640105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f1c154220 0x7f2f1c15aa80 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 -- 172.21.15.26:0/3450640105 >> 172.21.15.26:0/3450640105 conn(0x7f2f1c0658e0 msgr2=0x7f2f1c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 -- 172.21.15.26:0/3450640105 shutdown_connections 2023-12-19T17:01:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 -- 172.21.15.26:0/3450640105 wait complete. 2023-12-19T17:01:18.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 Processor -- start 2023-12-19T17:01:18.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 -- start start 2023-12-19T17:01:18.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f1c06b3d0 0x7f2f1c14c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f1c0cec40 0x7f2f1c1f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f1c154220 0x7f2f1c13a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c15bbe0 con 0x7f2f1c0cec40 2023-12-19T17:01:18.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c14c7e0 con 0x7f2f1c06b3d0 2023-12-19T17:01:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f243ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c1f9730 con 0x7f2f1c154220 2023-12-19T17:01:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.041+0000 7f2f21969640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f1c0cec40 0x7f2f1c1f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f21969640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f1c0cec40 0x7f2f1c1f91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40756/0 (socket says 172.21.15.26:40756) 2023-12-19T17:01:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f21969640 1 -- 172.21.15.26:0/1635470009 learned_addr learned my addr 172.21.15.26:0/1635470009 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f2216a640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f1c06b3d0 0x7f2f1c14c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f2296b640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f1c154220 0x7f2f1c13a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f21969640 1 -- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f1c154220 msgr2=0x7f2f1c13a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f21969640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f1c154220 0x7f2f1c13a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f21969640 1 -- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f1c06b3d0 msgr2=0x7f2f1c14c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f21969640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f1c06b3d0 0x7f2f1c14c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f21969640 1 -- 172.21.15.26:0/1635470009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f180094d0 con 0x7f2f1c0cec40 2023-12-19T17:01:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f21969640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f1c0cec40 0x7f2f1c1f91f0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00d800 tx=0x7f2f0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:18.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f12ffd640 1 -- 172.21.15.26:0/1635470009 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0096a0 con 0x7f2f1c0cec40 2023-12-19T17:01:18.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f12ffd640 1 -- 172.21.15.26:0/1635470009 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0c009800 con 0x7f2f1c0cec40 2023-12-19T17:01:18.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f243ce640 1 -- 172.21.15.26:0/1635470009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c13a760 con 0x7f2f1c0cec40 2023-12-19T17:01:18.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f12ffd640 1 -- 172.21.15.26:0/1635470009 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c009b70 con 0x7f2f1c0cec40 2023-12-19T17:01:18.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f243ce640 1 -- 172.21.15.26:0/1635470009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c15b520 con 0x7f2f1c0cec40 2023-12-19T17:01:18.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f243ce640 1 -- 172.21.15.26:0/1635470009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c15bd20 con 0x7f2f1c0cec40 2023-12-19T17:01:18.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f12ffd640 1 -- 172.21.15.26:0/1635470009 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2f0c009cd0 con 0x7f2f1c0cec40 2023-12-19T17:01:18.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.045+0000 7f2f12ffd640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2f00065b80 0x7f2f00068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.049+0000 7f2f12ffd640 1 -- 172.21.15.26:0/1635470009 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f2f0c083830 con 0x7f2f1c0cec40 2023-12-19T17:01:18.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.049+0000 7f2f2216a640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2f00065b80 0x7f2f00068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.049+0000 7f2f2216a640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2f00065b80 0x7f2f00068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2f180ad000 tx=0x7f2f180a6000 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.053+0000 7f2f12ffd640 1 -- 172.21.15.26:0/1635470009 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2f1c15bd20 con 0x7f2f1c0cec40 2023-12-19T17:01:18.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.265+0000 7f2f243ce640 1 -- 172.21.15.26:0/1635470009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2f1c15bd20 con 0x7f2f1c0cec40 2023-12-19T17:01:18.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.265+0000 7f2f12ffd640 1 -- 172.21.15.26:0/1635470009 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2f1c15bd20 con 0x7f2f1c0cec40 2023-12-19T17:01:18.271 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:18.271 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T17:01:18.271 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2023-12-19T17:01:18.271 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg==", 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg==", 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.272 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg==", 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.273 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.274 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2023-12-19T17:01:18.274 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw==", 2023-12-19T17:01:18.274 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.274 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:01:18.274 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:01:18.274 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:01:18.274 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.274 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.274 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.274 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.274 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2023-12-19T17:01:18.275 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig==", 2023-12-19T17:01:18.275 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.275 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:01:18.275 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:01:18.275 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.275 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.275 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.275 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.275 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2023-12-19T17:01:18.275 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA==", 2023-12-19T17:01:18.275 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.275 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:01:18.275 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ==", 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.276 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2023-12-19T17:01:18.277 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ==", 2023-12-19T17:01:18.277 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.277 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:01:18.277 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:01:18.277 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.277 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.277 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.277 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.277 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2023-12-19T17:01:18.277 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A==", 2023-12-19T17:01:18.277 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.277 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:01:18.277 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:01:18.278 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.278 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.278 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2023-12-19T17:01:18.278 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ==", 2023-12-19T17:01:18.278 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.278 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:01:18.278 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:01:18.278 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.278 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.278 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.278 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg==", 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg==", 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:01:18.279 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ==", 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.280 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ==", 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ==", 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:01:18.281 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ==", 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlT5ynDxAA7Sjbdwrc1RyAXJboysB54Q==", 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA==", 2023-12-19T17:01:18.283 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg==", 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.284 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:18.285 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-12-19T17:01:18.285 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T17:01:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.269+0000 7f2f243ce640 1 -- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2f00065b80 msgr2=0x7f2f00068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.269+0000 7f2f243ce640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2f00065b80 0x7f2f00068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2f180ad000 tx=0x7f2f180a6000 comp rx=0 tx=0).stop 2023-12-19T17:01:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.269+0000 7f2f243ce640 1 -- 172.21.15.26:0/1635470009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f1c0cec40 msgr2=0x7f2f1c1f91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.269+0000 7f2f243ce640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f1c0cec40 0x7f2f1c1f91f0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00d800 tx=0x7f2f0c00dcc0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.269+0000 7f2f243ce640 1 -- 172.21.15.26:0/1635470009 shutdown_connections 2023-12-19T17:01:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.269+0000 7f2f243ce640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2f00065b80 0x7f2f00068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.269+0000 7f2f243ce640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f1c154220 0x7f2f1c13a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.269+0000 7f2f243ce640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f1c0cec40 0x7f2f1c1f91f0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.269+0000 7f2f243ce640 1 --2- 172.21.15.26:0/1635470009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f1c06b3d0 0x7f2f1c14c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.269+0000 7f2f243ce640 1 -- 172.21.15.26:0/1635470009 >> 172.21.15.26:0/1635470009 conn(0x7f2f1c0658e0 msgr2=0x7f2f1c0ac850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.269+0000 7f2f243ce640 1 -- 172.21.15.26:0/1635470009 shutdown_connections 2023-12-19T17:01:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.269+0000 7f2f243ce640 1 -- 172.21.15.26:0/1635470009 wait complete. 2023-12-19T17:01:18.293 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:01:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6bb130b640 1 Processor -- start 2023-12-19T17:01:18.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6bb130b640 1 -- start start 2023-12-19T17:01:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6bb130b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6bac0cc590 0x7f6bac0cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6bb130b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac0cceb0 con 0x7f6bac1606c0 2023-12-19T17:01:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6bb130b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac15d9b0 con 0x7f6bac15d3f0 2023-12-19T17:01:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6bb130b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bac15dc90 con 0x7f6bac0cc590 2023-12-19T17:01:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6bab7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6bac1606c0 0x7f6bac15cce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33286/0 (socket says 172.21.15.26:33286) 2023-12-19T17:01:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6bab7fe640 1 -- 172.21.15.26:0/1234757888 learned_addr learned my addr 172.21.15.26:0/1234757888 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:18.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6babfff640 1 --2- 172.21.15.26:0/1234757888 >> v2:172.21.15.160:3300/0 conn(0x7f6bac0cc590 0x7f6bac0cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6baaffd640 1 -- 172.21.15.26:0/1234757888 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933252036 0 0) 0x7f6bac0cceb0 con 0x7f6bac1606c0 2023-12-19T17:01:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6baaffd640 1 -- 172.21.15.26:0/1234757888 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b94003530 con 0x7f6bac1606c0 2023-12-19T17:01:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6babfff640 1 -- 172.21.15.26:0/1234757888 >> v1:172.21.15.160:6789/0 conn(0x7f6bac15d3f0 legacy=0x7f6bac06b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6babfff640 1 -- 172.21.15.26:0/1234757888 >> v1:172.21.15.26:6789/0 conn(0x7f6bac1606c0 legacy=0x7f6bac15cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.433+0000 7f6babfff640 1 -- 172.21.15.26:0/1234757888 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac15df70 con 0x7f6bac0cc590 2023-12-19T17:01:18.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.437+0000 7f6babfff640 1 --2- 172.21.15.26:0/1234757888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac0cc590 0x7f6bac0cc970 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0097c0 tx=0x7f6b9c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f08e8d8efba8e139 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.437+0000 7f6baaffd640 1 -- 172.21.15.26:0/1234757888 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0ad660 con 0x7f6bac0cc590 2023-12-19T17:01:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.437+0000 7f6baaffd640 1 -- 172.21.15.26:0/1234757888 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c0ad7c0 con 0x7f6bac0cc590 2023-12-19T17:01:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.437+0000 7f6baaffd640 1 -- 172.21.15.26:0/1234757888 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0adb50 con 0x7f6bac0cc590 2023-12-19T17:01:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.437+0000 7f6bb130b640 1 -- 172.21.15.26:0/1234757888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac0cc590 msgr2=0x7f6bac0cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.437+0000 7f6bb130b640 1 --2- 172.21.15.26:0/1234757888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac0cc590 0x7f6bac0cc970 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0097c0 tx=0x7f6b9c01c450 comp rx=0 tx=0).stop 2023-12-19T17:01:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.437+0000 7f6bb130b640 1 -- 172.21.15.26:0/1234757888 shutdown_connections 2023-12-19T17:01:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.437+0000 7f6bb130b640 1 --2- 172.21.15.26:0/1234757888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac0cc590 0x7f6bac0cc970 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.437+0000 7f6bb130b640 1 -- 172.21.15.26:0/1234757888 >> 172.21.15.26:0/1234757888 conn(0x7f6bac0b04f0 msgr2=0x7f6bac0b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.437+0000 7f6bb130b640 1 -- 172.21.15.26:0/1234757888 shutdown_connections 2023-12-19T17:01:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.437+0000 7f6bb130b640 1 -- 172.21.15.26:0/1234757888 wait complete. 2023-12-19T17:01:18.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bb130b640 1 Processor -- start 2023-12-19T17:01:18.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bb130b640 1 -- start start 2023-12-19T17:01:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bb130b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac0cc590 0x7f6bac0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bb130b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15d3f0 0x7f6bac142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bb130b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac1606c0 0x7f6bac158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bb130b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac143010 con 0x7f6bac15d3f0 2023-12-19T17:01:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bb130b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac1f85c0 con 0x7f6bac1606c0 2023-12-19T17:01:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bb130b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac1f88a0 con 0x7f6bac0cc590 2023-12-19T17:01:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15d3f0 0x7f6bac142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bab7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15d3f0 0x7f6bac142ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40762/0 (socket says 172.21.15.26:40762) 2023-12-19T17:01:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bab7fe640 1 -- 172.21.15.26:0/2612680665 learned_addr learned my addr 172.21.15.26:0/2612680665 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6babfff640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac0cc590 0x7f6bac0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bb0b0a640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac1606c0 0x7f6bac158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bab7fe640 1 -- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac0cc590 msgr2=0x7f6bac0b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bab7fe640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac0cc590 0x7f6bac0b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bab7fe640 1 -- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac1606c0 msgr2=0x7f6bac158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bab7fe640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac1606c0 0x7f6bac158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bab7fe640 1 -- 172.21.15.26:0/2612680665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c009420 con 0x7f6bac15d3f0 2023-12-19T17:01:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bab7fe640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15d3f0 0x7f6bac142ad0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c400 tx=0x7f6ba000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6ba8ff9640 1 -- 172.21.15.26:0/2612680665 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0005cf0 con 0x7f6bac15d3f0 2023-12-19T17:01:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6ba8ff9640 1 -- 172.21.15.26:0/2612680665 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba0005e50 con 0x7f6bac15d3f0 2023-12-19T17:01:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bb130b640 1 -- 172.21.15.26:0/2612680665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac1f8b80 con 0x7f6bac15d3f0 2023-12-19T17:01:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6ba8ff9640 1 -- 172.21.15.26:0/2612680665 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba00072d0 con 0x7f6bac15d3f0 2023-12-19T17:01:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bb130b640 1 -- 172.21.15.26:0/2612680665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac1592a0 con 0x7f6bac15d3f0 2023-12-19T17:01:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.441+0000 7f6bb130b640 1 -- 172.21.15.26:0/2612680665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac067400 con 0x7f6bac15d3f0 2023-12-19T17:01:18.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.445+0000 7f6ba8ff9640 1 -- 172.21.15.26:0/2612680665 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6ba0007430 con 0x7f6bac15d3f0 2023-12-19T17:01:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.445+0000 7f6ba8ff9640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b98065b80 0x7f6b98068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.445+0000 7f6ba8ff9640 1 -- 172.21.15.26:0/2612680665 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f6ba0082530 con 0x7f6bac15d3f0 2023-12-19T17:01:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.445+0000 7f6babfff640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b98065b80 0x7f6b98068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.445+0000 7f6babfff640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b98065b80 0x7f6b98068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0af000 tx=0x7f6b9c008040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.449+0000 7f6ba8ff9640 1 -- 172.21.15.26:0/2612680665 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6ba004fea0 con 0x7f6bac15d3f0 2023-12-19T17:01:18.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.605+0000 7f6bb130b640 1 -- 172.21.15.26:0/2612680665 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bac0adb80 con 0x7f6bac15d3f0 2023-12-19T17:01:18.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.609+0000 7f6ba8ff9640 1 -- 172.21.15.26:0/2612680665 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+13504 (secure 0 0 0) 0x7f6ba00534c0 con 0x7f6bac15d3f0 2023-12-19T17:01:18.613 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:18.613 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":133,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:01:17.855499+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:01:18.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.609+0000 7f6bb130b640 1 -- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b98065b80 msgr2=0x7f6b98068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.609+0000 7f6bb130b640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b98065b80 0x7f6b98068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0af000 tx=0x7f6b9c008040 comp rx=0 tx=0).stop 2023-12-19T17:01:18.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.609+0000 7f6bb130b640 1 -- 172.21.15.26:0/2612680665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15d3f0 msgr2=0x7f6bac142ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.609+0000 7f6bb130b640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15d3f0 0x7f6bac142ad0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c400 tx=0x7f6ba000c8c0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.609+0000 7f6bb130b640 1 -- 172.21.15.26:0/2612680665 shutdown_connections 2023-12-19T17:01:18.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.609+0000 7f6bb130b640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac1606c0 0x7f6bac158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.609+0000 7f6bb130b640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15d3f0 0x7f6bac142ad0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.609+0000 7f6bb130b640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b98065b80 0x7f6b98068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.609+0000 7f6bb130b640 1 --2- 172.21.15.26:0/2612680665 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac0cc590 0x7f6bac0b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.609+0000 7f6bb130b640 1 -- 172.21.15.26:0/2612680665 >> 172.21.15.26:0/2612680665 conn(0x7f6bac0b04f0 msgr2=0x7f6bac06f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:18.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.609+0000 7f6bb130b640 1 -- 172.21.15.26:0/2612680665 shutdown_connections 2023-12-19T17:01:18.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.609+0000 7f6bb130b640 1 -- 172.21.15.26:0/2612680665 wait complete. 2023-12-19T17:01:18.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T17:01:18.634 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T17:01:18.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bc2339640 1 Processor -- start 2023-12-19T17:01:18.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bc2339640 1 -- start start 2023-12-19T17:01:18.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bc2339640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0bbc06f6c0 0x7f0bbc15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bc2339640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bbc06fb60 con 0x7f0bbc159c00 2023-12-19T17:01:18.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bc2339640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bbc06fe40 con 0x7f0bbc159820 2023-12-19T17:01:18.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bc2339640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bbc15e470 con 0x7f0bbc06f6c0 2023-12-19T17:01:18.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bbb7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0bbc06f6c0 0x7f0bbc15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bbb7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0bbc06f6c0 0x7f0bbc15df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40768/0 (socket says 172.21.15.26:40768) 2023-12-19T17:01:18.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bbb7fe640 1 -- 172.21.15.26:0/3795720959 learned_addr learned my addr 172.21.15.26:0/3795720959 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:18.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bc08d6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0bbc159c00 0x7f0bbc070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33290/0 (socket says 172.21.15.26:33290) 2023-12-19T17:01:18.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bbb7fe640 1 -- 172.21.15.26:0/3795720959 >> v1:172.21.15.160:6789/0 conn(0x7f0bbc159820 legacy=0x7f0bbc0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:18.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bbb7fe640 1 -- 172.21.15.26:0/3795720959 >> v1:172.21.15.26:6789/0 conn(0x7f0bbc159c00 legacy=0x7f0bbc070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bbaffd640 1 -- 172.21.15.26:0/3795720959 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992644883 0 0) 0x7f0bbc06fb60 con 0x7f0bbc159c00 2023-12-19T17:01:18.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.777+0000 7f0bbb7fe640 1 -- 172.21.15.26:0/3795720959 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc070120 con 0x7f0bbc06f6c0 2023-12-19T17:01:18.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.781+0000 7f0bbb7fe640 1 --2- 172.21.15.26:0/3795720959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc06f6c0 0x7f0bbc15df30 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009870 tx=0x7f0bb001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefe4834b4767d5d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.781+0000 7f0bbaffd640 1 -- 172.21.15.26:0/3795720959 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00aa670 con 0x7f0bbc06f6c0 2023-12-19T17:01:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.781+0000 7f0bbaffd640 1 -- 172.21.15.26:0/3795720959 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb00aa7d0 con 0x7f0bbc06f6c0 2023-12-19T17:01:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.781+0000 7f0bbaffd640 1 -- 172.21.15.26:0/3795720959 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00aab60 con 0x7f0bbc06f6c0 2023-12-19T17:01:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.781+0000 7f0bc2339640 1 -- 172.21.15.26:0/3795720959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc06f6c0 msgr2=0x7f0bbc15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.781+0000 7f0bc2339640 1 --2- 172.21.15.26:0/3795720959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc06f6c0 0x7f0bbc15df30 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009870 tx=0x7f0bb001c510 comp rx=0 tx=0).stop 2023-12-19T17:01:18.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.781+0000 7f0bc2339640 1 -- 172.21.15.26:0/3795720959 shutdown_connections 2023-12-19T17:01:18.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.781+0000 7f0bc2339640 1 --2- 172.21.15.26:0/3795720959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc06f6c0 0x7f0bbc15df30 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.781+0000 7f0bc2339640 1 -- 172.21.15.26:0/3795720959 >> 172.21.15.26:0/3795720959 conn(0x7f0bbc0ce730 msgr2=0x7f0bbc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:18.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.781+0000 7f0bc2339640 1 -- 172.21.15.26:0/3795720959 shutdown_connections 2023-12-19T17:01:18.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.781+0000 7f0bc2339640 1 -- 172.21.15.26:0/3795720959 wait complete. 2023-12-19T17:01:18.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bc2339640 1 Processor -- start 2023-12-19T17:01:18.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bc2339640 1 -- start start 2023-12-19T17:01:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bc2339640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bbc06f6c0 0x7f0bbc0ac670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bc2339640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc159820 0x7f0bbc0c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bc2339640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bbc159c00 0x7f0bbc159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bc2339640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc150750 con 0x7f0bbc159820 2023-12-19T17:01:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bc2339640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc15a520 con 0x7f0bbc159c00 2023-12-19T17:01:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bc2339640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc15a7c0 con 0x7f0bbc06f6c0 2023-12-19T17:01:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bbb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc159820 0x7f0bbc0c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bbb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc159820 0x7f0bbc0c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40772/0 (socket says 172.21.15.26:40772) 2023-12-19T17:01:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bbb7fe640 1 -- 172.21.15.26:0/2405979786 learned_addr learned my addr 172.21.15.26:0/2405979786 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bc08d6640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bbc159c00 0x7f0bbc159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bbbfff640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bbc06f6c0 0x7f0bbc0ac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bbb7fe640 1 -- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bbc06f6c0 msgr2=0x7f0bbc0ac670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bbb7fe640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bbc06f6c0 0x7f0bbc0ac670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bbb7fe640 1 -- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bbc159c00 msgr2=0x7f0bbc159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bbb7fe640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bbc159c00 0x7f0bbc159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bbb7fe640 1 -- 172.21.15.26:0/2405979786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00094d0 con 0x7f0bbc159820 2023-12-19T17:01:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bbb7fe640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc159820 0x7f0bbc0c7e40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0019120 tx=0x7f0bb001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bb8ff9640 1 -- 172.21.15.26:0/2405979786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00b3800 con 0x7f0bbc159820 2023-12-19T17:01:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bb8ff9640 1 -- 172.21.15.26:0/2405979786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb00b3980 con 0x7f0bbc159820 2023-12-19T17:01:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bc2339640 1 -- 172.21.15.26:0/2405979786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bbc06b880 con 0x7f0bbc159820 2023-12-19T17:01:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bb8ff9640 1 -- 172.21.15.26:0/2405979786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00b3d30 con 0x7f0bbc159820 2023-12-19T17:01:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.785+0000 7f0bc2339640 1 -- 172.21.15.26:0/2405979786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bbc0c2bd0 con 0x7f0bbc159820 2023-12-19T17:01:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.789+0000 7f0bc2339640 1 -- 172.21.15.26:0/2405979786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bbc074270 con 0x7f0bbc159820 2023-12-19T17:01:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.789+0000 7f0bb8ff9640 1 -- 172.21.15.26:0/2405979786 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0bb00c6390 con 0x7f0bbc159820 2023-12-19T17:01:18.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.789+0000 7f0bb8ff9640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0ba8065b80 0x7f0ba8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:18.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.789+0000 7f0bb8ff9640 1 -- 172.21.15.26:0/2405979786 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f0bb011dfa0 con 0x7f0bbc159820 2023-12-19T17:01:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.789+0000 7f0bbbfff640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0ba8065b80 0x7f0ba8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.789+0000 7f0bbbfff640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0ba8065b80 0x7f0ba8068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc15c0d0 tx=0x7f0bac009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:18.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.793+0000 7f0bb8ff9640 1 -- 172.21.15.26:0/2405979786 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0bb00eb7b0 con 0x7f0bbc159820 2023-12-19T17:01:18.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.949+0000 7f0bc2339640 1 -- 172.21.15.26:0/2405979786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0bbc0b4690 con 0x7f0bbc159820 2023-12-19T17:01:18.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.949+0000 7f0bb8ff9640 1 -- 172.21.15.26:0/2405979786 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0bb00eedd0 con 0x7f0bbc159820 2023-12-19T17:01:18.954 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T17:01:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.949+0000 7f0bc2339640 1 -- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0ba8065b80 msgr2=0x7f0ba8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.949+0000 7f0bc2339640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0ba8065b80 0x7f0ba8068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc15c0d0 tx=0x7f0bac009040 comp rx=0 tx=0).stop 2023-12-19T17:01:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.949+0000 7f0bc2339640 1 -- 172.21.15.26:0/2405979786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc159820 msgr2=0x7f0bbc0c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.949+0000 7f0bc2339640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc159820 0x7f0bbc0c7e40 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0019120 tx=0x7f0bb001ad50 comp rx=0 tx=0).stop 2023-12-19T17:01:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.949+0000 7f0bc2339640 1 -- 172.21.15.26:0/2405979786 shutdown_connections 2023-12-19T17:01:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.949+0000 7f0bc2339640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0bbc159c00 0x7f0bbc159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.949+0000 7f0bc2339640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bbc159820 0x7f0bbc0c7e40 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.949+0000 7f0bc2339640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0ba8065b80 0x7f0ba8068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.949+0000 7f0bc2339640 1 --2- 172.21.15.26:0/2405979786 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0bbc06f6c0 0x7f0bbc0ac670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.949+0000 7f0bc2339640 1 -- 172.21.15.26:0/2405979786 >> 172.21.15.26:0/2405979786 conn(0x7f0bbc0ce730 msgr2=0x7f0bbc15db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:18.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.953+0000 7f0bc2339640 1 -- 172.21.15.26:0/2405979786 shutdown_connections 2023-12-19T17:01:18.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:18.953+0000 7f0bc2339640 1 -- 172.21.15.26:0/2405979786 wait complete. 2023-12-19T17:01:18.975 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T17:01:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.113+0000 7fb0e33be640 1 Processor -- start 2023-12-19T17:01:19.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.113+0000 7fb0e33be640 1 -- start start 2023-12-19T17:01:19.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e33be640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0dc072120 0x7fb0dc071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e33be640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0dc06baa0 0x7fb0dc15c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e33be640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0dc0d0490 con 0x7fb0dc071d40 2023-12-19T17:01:19.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e33be640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0dc0cfb60 con 0x7fb0dc06baa0 2023-12-19T17:01:19.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e33be640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0cfe40 con 0x7fb0dc072120 2023-12-19T17:01:19.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e23bc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb0dc071d40 0x7fb0dc06b390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33302/0 (socket says 172.21.15.26:33302) 2023-12-19T17:01:19.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e23bc640 1 -- 172.21.15.26:0/1840122185 learned_addr learned my addr 172.21.15.26:0/1840122185 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:19.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e2bbd640 1 --2- 172.21.15.26:0/1840122185 >> v2:172.21.15.26:3300/0 conn(0x7fb0dc06baa0 0x7fb0dc15c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e2bbd640 1 -- 172.21.15.26:0/1840122185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0dc072120 msgr2=0x7fb0dc071230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-19T17:01:19.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e2bbd640 1 --2- 172.21.15.26:0/1840122185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0dc072120 0x7fb0dc071230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e2bbd640 1 -- 172.21.15.26:0/1840122185 >> v1:172.21.15.26:6789/0 conn(0x7fb0dc071d40 legacy=0x7fb0dc06b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e2bbd640 1 -- 172.21.15.26:0/1840122185 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0dc0d0120 con 0x7fb0dc06baa0 2023-12-19T17:01:19.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e13ba640 1 -- 172.21.15.26:0/1840122185 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2032951804 0 0) 0x7fb0dc0d0490 con 0x7fb0dc071d40 2023-12-19T17:01:19.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.117+0000 7fb0e2bbd640 1 --2- 172.21.15.26:0/1840122185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc06baa0 0x7fb0dc15c400 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4009d90 tx=0x7fb0d401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fad5c45de56ed91 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:19.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e13ba640 1 -- 172.21.15.26:0/1840122185 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc0cd080 con 0x7fb0dc06baa0 2023-12-19T17:01:19.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e13ba640 1 -- 172.21.15.26:0/1840122185 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d40a59c0 con 0x7fb0dc06baa0 2023-12-19T17:01:19.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e13ba640 1 -- 172.21.15.26:0/1840122185 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d40a5d50 con 0x7fb0dc06baa0 2023-12-19T17:01:19.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 -- 172.21.15.26:0/1840122185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc06baa0 msgr2=0x7fb0dc15c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 --2- 172.21.15.26:0/1840122185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc06baa0 0x7fb0dc15c400 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4009d90 tx=0x7fb0d401c8e0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 -- 172.21.15.26:0/1840122185 shutdown_connections 2023-12-19T17:01:19.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 --2- 172.21.15.26:0/1840122185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc06baa0 0x7fb0dc15c400 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 --2- 172.21.15.26:0/1840122185 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0dc072120 0x7fb0dc071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 -- 172.21.15.26:0/1840122185 >> 172.21.15.26:0/1840122185 conn(0x7fb0dc0cd210 msgr2=0x7fb0dc0cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:19.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 -- 172.21.15.26:0/1840122185 shutdown_connections 2023-12-19T17:01:19.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 -- 172.21.15.26:0/1840122185 wait complete. 2023-12-19T17:01:19.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 Processor -- start 2023-12-19T17:01:19.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 -- start start 2023-12-19T17:01:19.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0dc06baa0 0x7fb0dc0ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc071d40 0x7fb0dc0c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0dc072120 0x7fb0dc15d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0c6e20 con 0x7fb0dc071d40 2023-12-19T17:01:19.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0adf20 con 0x7fb0dc06baa0 2023-12-19T17:01:19.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e33be640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0dc0ad310 con 0x7fb0dc072120 2023-12-19T17:01:19.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e1bbb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc071d40 0x7fb0dc0c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e1bbb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc071d40 0x7fb0dc0c62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40796/0 (socket says 172.21.15.26:40796) 2023-12-19T17:01:19.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e1bbb640 1 -- 172.21.15.26:0/2070051612 learned_addr learned my addr 172.21.15.26:0/2070051612 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:19.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e2bbd640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0dc072120 0x7fb0dc15d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.121+0000 7fb0e23bc640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0dc06baa0 0x7fb0dc0ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0e1bbb640 1 -- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0dc072120 msgr2=0x7fb0dc15d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0e1bbb640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0dc072120 0x7fb0dc15d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0e1bbb640 1 -- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0dc06baa0 msgr2=0x7fb0dc0ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0e1bbb640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0dc06baa0 0x7fb0dc0ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0e1bbb640 1 -- 172.21.15.26:0/2070051612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d4009a70 con 0x7fb0dc071d40 2023-12-19T17:01:19.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0e1bbb640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc071d40 0x7fb0dc0c62a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00b3e0 tx=0x7fb0cc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:19.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0d2ffd640 1 -- 172.21.15.26:0/2070051612 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc011830 con 0x7fb0dc071d40 2023-12-19T17:01:19.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0d2ffd640 1 -- 172.21.15.26:0/2070051612 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0cc011990 con 0x7fb0dc071d40 2023-12-19T17:01:19.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0d2ffd640 1 -- 172.21.15.26:0/2070051612 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc011d60 con 0x7fb0dc071d40 2023-12-19T17:01:19.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0e33be640 1 -- 172.21.15.26:0/2070051612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0dc0ad5f0 con 0x7fb0dc071d40 2023-12-19T17:01:19.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0e33be640 1 -- 172.21.15.26:0/2070051612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc0adbb0 con 0x7fb0dc071d40 2023-12-19T17:01:19.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0e33be640 1 -- 172.21.15.26:0/2070051612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0dc0ae8f0 con 0x7fb0dc071d40 2023-12-19T17:01:19.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0d2ffd640 1 -- 172.21.15.26:0/2070051612 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb0cc01a400 con 0x7fb0dc071d40 2023-12-19T17:01:19.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0d2ffd640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb0c0065bb0 0x7fb0c0068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0d2ffd640 1 -- 172.21.15.26:0/2070051612 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fb0cc0833a0 con 0x7fb0dc071d40 2023-12-19T17:01:19.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.125+0000 7fb0e23bc640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb0c0065bb0 0x7fb0c0068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.129+0000 7fb0e23bc640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb0c0065bb0 0x7fb0c0068070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8000f70 tx=0x7fb0d8007b60 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:19.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.129+0000 7fb0d2ffd640 1 -- 172.21.15.26:0/2070051612 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb0cc050cc0 con 0x7fb0dc071d40 2023-12-19T17:01:19.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0e33be640 1 -- 172.21.15.26:0/2070051612 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb0dc0ae0d0 con 0x7fb0dc071d40 2023-12-19T17:01:19.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0d2ffd640 1 -- 172.21.15.26:0/2070051612 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v133) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb0cc0542e0 con 0x7fb0dc071d40 2023-12-19T17:01:19.294 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:19.294 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T17:01:19.294 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:19.294 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T17:01:19.294 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T17:01:19.294 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:19.294 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T17:01:19.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0e33be640 1 -- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb0c0065bb0 msgr2=0x7fb0c0068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0e33be640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb0c0065bb0 0x7fb0c0068070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8000f70 tx=0x7fb0d8007b60 comp rx=0 tx=0).stop 2023-12-19T17:01:19.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0e33be640 1 -- 172.21.15.26:0/2070051612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc071d40 msgr2=0x7fb0dc0c62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0e33be640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc071d40 0x7fb0dc0c62a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc00b3e0 tx=0x7fb0cc00b8a0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0e33be640 1 -- 172.21.15.26:0/2070051612 shutdown_connections 2023-12-19T17:01:19.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0e33be640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb0c0065bb0 0x7fb0c0068070 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0e33be640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0dc072120 0x7fb0dc15d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0e33be640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0dc071d40 0x7fb0dc0c62a0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0e33be640 1 --2- 172.21.15.26:0/2070051612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0dc06baa0 0x7fb0dc0ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0e33be640 1 -- 172.21.15.26:0/2070051612 >> 172.21.15.26:0/2070051612 conn(0x7fb0dc0cd210 msgr2=0x7fb0dc0cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:19.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0e33be640 1 -- 172.21.15.26:0/2070051612 shutdown_connections 2023-12-19T17:01:19.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.289+0000 7fb0e33be640 1 -- 172.21.15.26:0/2070051612 wait complete. 2023-12-19T17:01:19.315 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T17:01:19.315 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T17:01:19.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.457+0000 7fdfc2266640 1 Processor -- start 2023-12-19T17:01:19.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.457+0000 7fdfc2266640 1 -- start start 2023-12-19T17:01:19.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.457+0000 7fdfc2266640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdfbc15ccf0 0x7fdfbc0ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.457+0000 7fdfc2266640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdfbc06c0f0 0x7fdfbc0ac960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.457+0000 7fdfc2266640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfbc0cd100 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.457+0000 7fdfc2266640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfbc06e000 con 0x7fdfbc15ccf0 2023-12-19T17:01:19.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.457+0000 7fdfc2266640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfbc06da40 con 0x7fdfbc06c0f0 2023-12-19T17:01:19.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.457+0000 7fdfbbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdfbc15ccf0 0x7fdfbc0ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.457+0000 7fdfbbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdfbc15ccf0 0x7fdfbc0ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40810/0 (socket says 172.21.15.26:40810) 2023-12-19T17:01:19.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.457+0000 7fdfbb7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdfbc0ce7f0 0x7fdfbc06b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33310/0 (socket says 172.21.15.26:33310) 2023-12-19T17:01:19.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.457+0000 7fdfbbfff640 1 -- 172.21.15.26:0/3165655352 learned_addr learned my addr 172.21.15.26:0/3165655352 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:19.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.457+0000 7fdfc0803640 1 --2- 172.21.15.26:0/3165655352 >> v2:172.21.15.160:3300/0 conn(0x7fdfbc06c0f0 0x7fdfbc0ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.461+0000 7fdfbbfff640 1 -- 172.21.15.26:0/3165655352 >> v2:172.21.15.160:3300/0 conn(0x7fdfbc06c0f0 msgr2=0x7fdfbc0ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:19.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.461+0000 7fdfbbfff640 1 --2- 172.21.15.26:0/3165655352 >> v2:172.21.15.160:3300/0 conn(0x7fdfbc06c0f0 0x7fdfbc0ac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.461+0000 7fdfbbfff640 1 -- 172.21.15.26:0/3165655352 >> v1:172.21.15.26:6789/0 conn(0x7fdfbc0ce7f0 legacy=0x7fdfbc06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.461+0000 7fdfbbfff640 1 -- 172.21.15.26:0/3165655352 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc06dd20 con 0x7fdfbc15ccf0 2023-12-19T17:01:19.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.461+0000 7fdfbbfff640 1 --2- 172.21.15.26:0/3165655352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc15ccf0 0x7fdfbc0ae3f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0cd070 tx=0x7fdfac01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b4e8102f10080 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:19.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.461+0000 7fdfbaffd640 1 -- 172.21.15.26:0/3165655352 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0ab610 con 0x7fdfbc15ccf0 2023-12-19T17:01:19.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.461+0000 7fdfbaffd640 1 -- 172.21.15.26:0/3165655352 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac0ab770 con 0x7fdfbc15ccf0 2023-12-19T17:01:19.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.461+0000 7fdfbaffd640 1 -- 172.21.15.26:0/3165655352 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0abb00 con 0x7fdfbc15ccf0 2023-12-19T17:01:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.461+0000 7fdfc2266640 1 -- 172.21.15.26:0/3165655352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc15ccf0 msgr2=0x7fdfbc0ae3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.461+0000 7fdfc2266640 1 --2- 172.21.15.26:0/3165655352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc15ccf0 0x7fdfbc0ae3f0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0cd070 tx=0x7fdfac01c470 comp rx=0 tx=0).stop 2023-12-19T17:01:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 -- 172.21.15.26:0/3165655352 shutdown_connections 2023-12-19T17:01:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 --2- 172.21.15.26:0/3165655352 >> v2:172.21.15.160:3300/0 conn(0x7fdfbc06c0f0 0x7fdfbc0ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 --2- 172.21.15.26:0/3165655352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc15ccf0 0x7fdfbc0ae3f0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 -- 172.21.15.26:0/3165655352 >> 172.21.15.26:0/3165655352 conn(0x7fdfbc0cf020 msgr2=0x7fdfbc0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 -- 172.21.15.26:0/3165655352 shutdown_connections 2023-12-19T17:01:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 -- 172.21.15.26:0/3165655352 wait complete. 2023-12-19T17:01:19.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 Processor -- start 2023-12-19T17:01:19.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 -- start start 2023-12-19T17:01:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfbc06c0f0 0x7fdfbc158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc0ce7f0 0x7fdfbc156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfbc15ccf0 0x7fdfbc14bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc0c2f30 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc158480 con 0x7fdfbc06c0f0 2023-12-19T17:01:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc2266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc157870 con 0x7fdfbc15ccf0 2023-12-19T17:01:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfbb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc0ce7f0 0x7fdfbc156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfbb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc0ce7f0 0x7fdfbc156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40816/0 (socket says 172.21.15.26:40816) 2023-12-19T17:01:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfbb7fe640 1 -- 172.21.15.26:0/395072643 learned_addr learned my addr 172.21.15.26:0/395072643 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfc0803640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfbc15ccf0 0x7fdfbc14bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfbbfff640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfbc06c0f0 0x7fdfbc158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfbb7fe640 1 -- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfbc15ccf0 msgr2=0x7fdfbc14bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfbb7fe640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfbc15ccf0 0x7fdfbc14bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfbb7fe640 1 -- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfbc06c0f0 msgr2=0x7fdfbc158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfbb7fe640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfbc06c0f0 0x7fdfbc158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfbb7fe640 1 -- 172.21.15.26:0/395072643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac009480 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.465+0000 7fdfbbfff640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfbc06c0f0 0x7fdfbc158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.469+0000 7fdfbb7fe640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc0ce7f0 0x7fdfbc156ea0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000c490 tx=0x7fdfb000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.469+0000 7fdfb8ff9640 1 -- 172.21.15.26:0/395072643 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0007480 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.469+0000 7fdfb8ff9640 1 -- 172.21.15.26:0/395072643 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb00075e0 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.469+0000 7fdfc2266640 1 -- 172.21.15.26:0/395072643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfbc157b50 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.469+0000 7fdfb8ff9640 1 -- 172.21.15.26:0/395072643 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0004a10 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.469+0000 7fdfc2266640 1 -- 172.21.15.26:0/395072643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfbc06ffa0 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.469+0000 7fdfc2266640 1 -- 172.21.15.26:0/395072643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfbc158610 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.469+0000 7fdfb8ff9640 1 -- 172.21.15.26:0/395072643 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fdfb0004b70 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.469+0000 7fdfb8ff9640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfa8065b80 0x7fdfa8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.469+0000 7fdfb8ff9640 1 -- 172.21.15.26:0/395072643 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fdfb00826c0 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.469+0000 7fdfbbfff640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfa8065b80 0x7fdfa8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.473+0000 7fdfbbfff640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfa8065b80 0x7fdfa8068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0af000 tx=0x7fdfac0a7000 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.473+0000 7fdfb8ff9640 1 -- 172.21.15.26:0/395072643 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdfb0050070 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.629+0000 7fdfc2266640 1 -- 172.21.15.26:0/395072643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdfbc0b4c00 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.733+0000 7fdfb8ff9640 1 -- 172.21.15.26:0/395072643 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v134) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdfb0053690 con 0x7fdfbc0ce7f0 2023-12-19T17:01:19.737 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T17:01:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.737+0000 7fdfc2266640 1 -- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfa8065b80 msgr2=0x7fdfa8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.737+0000 7fdfc2266640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfa8065b80 0x7fdfa8068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0af000 tx=0x7fdfac0a7000 comp rx=0 tx=0).stop 2023-12-19T17:01:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.737+0000 7fdfc2266640 1 -- 172.21.15.26:0/395072643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc0ce7f0 msgr2=0x7fdfbc156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.737+0000 7fdfc2266640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc0ce7f0 0x7fdfbc156ea0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fdfb000c490 tx=0x7fdfb000c950 comp rx=0 tx=0).stop 2023-12-19T17:01:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.737+0000 7fdfc2266640 1 -- 172.21.15.26:0/395072643 shutdown_connections 2023-12-19T17:01:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.737+0000 7fdfc2266640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfa8065b80 0x7fdfa8068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.737+0000 7fdfc2266640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfbc15ccf0 0x7fdfbc14bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.737+0000 7fdfc2266640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc0ce7f0 0x7fdfbc156ea0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.737+0000 7fdfc2266640 1 --2- 172.21.15.26:0/395072643 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfbc06c0f0 0x7fdfbc158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.737+0000 7fdfc2266640 1 -- 172.21.15.26:0/395072643 >> 172.21.15.26:0/395072643 conn(0x7fdfbc0cf020 msgr2=0x7fdfbc070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.737+0000 7fdfc2266640 1 -- 172.21.15.26:0/395072643 shutdown_connections 2023-12-19T17:01:19.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.737+0000 7fdfc2266640 1 -- 172.21.15.26:0/395072643 wait complete. 2023-12-19T17:01:19.764 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T17:01:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.909+0000 7f12bcda9640 1 Processor -- start 2023-12-19T17:01:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.909+0000 7f12bcda9640 1 -- start start 2023-12-19T17:01:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12bcda9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f12b815ab40 0x7f12b806b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12bcda9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12b8071890 0x7f12b806b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12bcda9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b815af20 con 0x7f12b806f0a0 2023-12-19T17:01:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12bcda9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b80aea90 con 0x7f12b815ab40 2023-12-19T17:01:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12bcda9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80aed70 con 0x7f12b8071890 2023-12-19T17:01:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f12b806f0a0 0x7f12b815e600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33322/0 (socket says 172.21.15.26:33322) 2023-12-19T17:01:19.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b6ffd640 1 -- 172.21.15.26:0/1178389674 learned_addr learned my addr 172.21.15.26:0/1178389674 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b77fe640 1 --2- 172.21.15.26:0/1178389674 >> v2:172.21.15.160:3300/0 conn(0x7f12b815ab40 0x7f12b806b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b7fff640 1 --2- 172.21.15.26:0/1178389674 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12b8071890 0x7f12b806b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133635845 0 0) 0x7f12b815af20 con 0x7f12b806f0a0 2023-12-19T17:01:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12a0003530 con 0x7f12b806f0a0 2023-12-19T17:01:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 921355 0 0) 0x7f12a0003530 con 0x7f12b806f0a0 2023-12-19T17:01:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12b8071890 msgr2=0x7f12b806b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 --2- 172.21.15.26:0/1178389674 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12b8071890 0x7f12b806b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 >> v2:172.21.15.160:3300/0 conn(0x7f12b815ab40 msgr2=0x7f12b806b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 --2- 172.21.15.26:0/1178389674 >> v2:172.21.15.160:3300/0 conn(0x7f12b815ab40 0x7f12b806b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b80af050 con 0x7f12b806f0a0 2023-12-19T17:01:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f12ac001680 con 0x7f12b806f0a0 2023-12-19T17:01:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 >> v1:172.21.15.26:6789/0 conn(0x7f12b806f0a0 legacy=0x7f12b815e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 --2- 172.21.15.26:0/1178389674 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a0003980 0x7f12a0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 --2- 172.21.15.26:0/1178389674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12a0004370 0x7f12a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 --2- 172.21.15.26:0/1178389674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a0004ec0 0x7f12a000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80aea90 con 0x7f12a0004ec0 2023-12-19T17:01:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80aed70 con 0x7f12a0004370 2023-12-19T17:01:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a000bed0 con 0x7f12a0003980 2023-12-19T17:01:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b7fff640 1 --2- 172.21.15.26:0/1178389674 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12b8071890 0x7f12b806b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:01:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b77fe640 1 --2- 172.21.15.26:0/1178389674 >> v2:172.21.15.160:3300/0 conn(0x7f12b815ab40 0x7f12b806b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:01:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b7fff640 1 --2- 172.21.15.26:0/1178389674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a0004ec0 0x7f12a000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b6ffd640 1 --2- 172.21.15.26:0/1178389674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12a0004370 0x7f12a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b77fe640 1 --2- 172.21.15.26:0/1178389674 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a0003980 0x7f12a0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b7fff640 1 -- 172.21.15.26:0/1178389674 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a0003980 msgr2=0x7f12a0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b7fff640 1 --2- 172.21.15.26:0/1178389674 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12a0003980 0x7f12a0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b7fff640 1 -- 172.21.15.26:0/1178389674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12a0004370 msgr2=0x7f12a0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b7fff640 1 --2- 172.21.15.26:0/1178389674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12a0004370 0x7f12a0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.913+0000 7f12b7fff640 1 -- 172.21.15.26:0/1178389674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a0003530 con 0x7f12a0004ec0 2023-12-19T17:01:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.917+0000 7f12b7fff640 1 --2- 172.21.15.26:0/1178389674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a0004ec0 0x7f12a000b990 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f12b000e1f0 tx=0x7f12b001cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ef77204f2f1874 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.917+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b00a5d50 con 0x7f12a0004ec0 2023-12-19T17:01:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.917+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b000f040 con 0x7f12a0004ec0 2023-12-19T17:01:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.917+0000 7f12b67fc640 1 -- 172.21.15.26:0/1178389674 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0003c10 con 0x7f12a0004ec0 2023-12-19T17:01:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.917+0000 7f12bcda9640 1 -- 172.21.15.26:0/1178389674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a0004ec0 msgr2=0x7f12a000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.917+0000 7f12bcda9640 1 --2- 172.21.15.26:0/1178389674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a0004ec0 0x7f12a000b990 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f12b000e1f0 tx=0x7f12b001cd00 comp rx=0 tx=0).stop 2023-12-19T17:01:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.917+0000 7f12b7fff640 1 -- 172.21.15.26:0/1178389674 reap_dead start 2023-12-19T17:01:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.917+0000 7f12bcda9640 1 -- 172.21.15.26:0/1178389674 shutdown_connections 2023-12-19T17:01:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.917+0000 7f12bcda9640 1 -- 172.21.15.26:0/1178389674 >> 172.21.15.26:0/1178389674 conn(0x7f12b80658e0 msgr2=0x7f12b8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.917+0000 7f12bcda9640 1 -- 172.21.15.26:0/1178389674 shutdown_connections 2023-12-19T17:01:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.917+0000 7f12bcda9640 1 -- 172.21.15.26:0/1178389674 wait complete. 2023-12-19T17:01:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.917+0000 7f12bcda9640 1 Processor -- start 2023-12-19T17:01:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12bcda9640 1 -- start start 2023-12-19T17:01:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12bcda9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a0004ec0 0x7f12b81e5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12bcda9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8151560 0x7f12b8151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12bcda9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12b81521b0 0x7f12b80b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12bcda9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b81e5090 con 0x7f12a0004ec0 2023-12-19T17:01:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12bcda9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b81e4480 con 0x7f12b8151560 2023-12-19T17:01:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12bcda9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b81e4760 con 0x7f12b81521b0 2023-12-19T17:01:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a0004ec0 0x7f12b81e5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a0004ec0 0x7f12b81e5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40822/0 (socket says 172.21.15.26:40822) 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12b77fe640 1 -- 172.21.15.26:0/314546695 learned_addr learned my addr 172.21.15.26:0/314546695 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12b7fff640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12b81521b0 0x7f12b80b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12b6ffd640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8151560 0x7f12b8151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12b77fe640 1 -- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12b81521b0 msgr2=0x7f12b80b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12b77fe640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12b81521b0 0x7f12b80b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12b77fe640 1 -- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8151560 msgr2=0x7f12b8151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12b77fe640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8151560 0x7f12b8151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12b77fe640 1 -- 172.21.15.26:0/314546695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b81e4a40 con 0x7f12a0004ec0 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12b77fe640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a0004ec0 0x7f12b81e5520 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f12a8011cd0 tx=0x7f12a800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f129ffff640 1 -- 172.21.15.26:0/314546695 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a800fd20 con 0x7f12a0004ec0 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f129ffff640 1 -- 172.21.15.26:0/314546695 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a800fe80 con 0x7f12a0004ec0 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12bcda9640 1 -- 172.21.15.26:0/314546695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b81e4d20 con 0x7f12a0004ec0 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f129ffff640 1 -- 172.21.15.26:0/314546695 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8004270 con 0x7f12a0004ec0 2023-12-19T17:01:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12bcda9640 1 -- 172.21.15.26:0/314546695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b80b27e0 con 0x7f12a0004ec0 2023-12-19T17:01:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.921+0000 7f12bcda9640 1 -- 172.21.15.26:0/314546695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b8061930 con 0x7f12a0004ec0 2023-12-19T17:01:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.925+0000 7f129ffff640 1 -- 172.21.15.26:0/314546695 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f12a80043d0 con 0x7f12a0004ec0 2023-12-19T17:01:19.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.925+0000 7f129ffff640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f12a4065b80 0x7f12a4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:19.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.925+0000 7f129ffff640 1 -- 172.21.15.26:0/314546695 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5959+0+0 (secure 0 0 0) 0x7f12a8083d20 con 0x7f12a0004ec0 2023-12-19T17:01:19.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.925+0000 7f12b6ffd640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f12a4065b80 0x7f12a4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:19.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.925+0000 7f12b6ffd640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f12a4065b80 0x7f12a4068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f12b80cd810 tx=0x7f12ac009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:19.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:19.929+0000 7f129ffff640 1 -- 172.21.15.26:0/314546695 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f12a8051430 con 0x7f12a0004ec0 2023-12-19T17:01:20.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.089+0000 7f12bcda9640 1 -- 172.21.15.26:0/314546695 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f12b81e48a0 con 0x7f12a0004ec0 2023-12-19T17:01:20.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.737+0000 7f129ffff640 1 -- 172.21.15.26:0/314546695 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v135) v1 ==== 170+0+0 (secure 0 0 0) 0x7f12b81e48a0 con 0x7f12a0004ec0 2023-12-19T17:01:20.745 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T17:01:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.745+0000 7f12bcda9640 1 -- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f12a4065b80 msgr2=0x7f12a4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.745+0000 7f12bcda9640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f12a4065b80 0x7f12a4068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f12b80cd810 tx=0x7f12ac009040 comp rx=0 tx=0).stop 2023-12-19T17:01:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.745+0000 7f12bcda9640 1 -- 172.21.15.26:0/314546695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a0004ec0 msgr2=0x7f12b81e5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.745+0000 7f12bcda9640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a0004ec0 0x7f12b81e5520 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f12a8011cd0 tx=0x7f12a800f610 comp rx=0 tx=0).stop 2023-12-19T17:01:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.745+0000 7f12bcda9640 1 -- 172.21.15.26:0/314546695 shutdown_connections 2023-12-19T17:01:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.745+0000 7f12bcda9640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f12a4065b80 0x7f12a4068040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:20.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.745+0000 7f12bcda9640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12b81521b0 0x7f12b80b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.745+0000 7f12bcda9640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12b8151560 0x7f12b8151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.745+0000 7f12bcda9640 1 --2- 172.21.15.26:0/314546695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12a0004ec0 0x7f12b81e5520 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.745+0000 7f12bcda9640 1 -- 172.21.15.26:0/314546695 >> 172.21.15.26:0/314546695 conn(0x7f12b80658e0 msgr2=0x7f12b8154000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.749+0000 7f12bcda9640 1 -- 172.21.15.26:0/314546695 shutdown_connections 2023-12-19T17:01:20.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.749+0000 7f12bcda9640 1 -- 172.21.15.26:0/314546695 wait complete. 2023-12-19T17:01:20.787 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T17:01:20.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.929+0000 7f215aa66640 1 Processor -- start 2023-12-19T17:01:20.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.929+0000 7f215aa66640 1 -- start start 2023-12-19T17:01:20.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.929+0000 7f215aa66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f215406f6c0 0x7f215415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:20.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.929+0000 7f215aa66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2154159c00 0x7f2154070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:20.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.929+0000 7f215aa66640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f215406fb60 con 0x7f2154159820 2023-12-19T17:01:20.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.929+0000 7f215aa66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215415e470 con 0x7f215406f6c0 2023-12-19T17:01:20.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.929+0000 7f215aa66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215406fe40 con 0x7f2154159c00 2023-12-19T17:01:20.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.929+0000 7f2153fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f215406f6c0 0x7f215415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:20.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.933+0000 7f2153fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f215406f6c0 0x7f215415df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40826/0 (socket says 172.21.15.26:40826) 2023-12-19T17:01:20.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.933+0000 7f2153fff640 1 -- 172.21.15.26:0/1756089978 learned_addr learned my addr 172.21.15.26:0/1756089978 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:20.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.933+0000 7f2159003640 1 --2- 172.21.15.26:0/1756089978 >> v2:172.21.15.160:3300/0 conn(0x7f2154159c00 0x7f2154070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:20.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.933+0000 7f2153fff640 1 -- 172.21.15.26:0/1756089978 >> v2:172.21.15.160:3300/0 conn(0x7f2154159c00 msgr2=0x7f2154070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:20.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.933+0000 7f2153fff640 1 --2- 172.21.15.26:0/1756089978 >> v2:172.21.15.160:3300/0 conn(0x7f2154159c00 0x7f2154070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:20.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.933+0000 7f2153fff640 1 -- 172.21.15.26:0/1756089978 >> v1:172.21.15.160:6789/0 conn(0x7f2154159820 legacy=0x7f21540c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:20.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.933+0000 7f2153fff640 1 -- 172.21.15.26:0/1756089978 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2154070120 con 0x7f215406f6c0 2023-12-19T17:01:20.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.933+0000 7f2153fff640 1 --2- 172.21.15.26:0/1756089978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f215406f6c0 0x7f215415df30 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f215406f200 tx=0x7f214001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55ced96fa74c077 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:20.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.933+0000 7f21537fe640 1 -- 172.21.15.26:0/1756089978 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21400ac600 con 0x7f215406f6c0 2023-12-19T17:01:20.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.933+0000 7f21537fe640 1 -- 172.21.15.26:0/1756089978 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21400a8050 con 0x7f215406f6c0 2023-12-19T17:01:20.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.933+0000 7f21537fe640 1 -- 172.21.15.26:0/1756089978 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21400ac980 con 0x7f215406f6c0 2023-12-19T17:01:20.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 -- 172.21.15.26:0/1756089978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f215406f6c0 msgr2=0x7f215415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:20.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 --2- 172.21.15.26:0/1756089978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f215406f6c0 0x7f215415df30 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f215406f200 tx=0x7f214001c490 comp rx=0 tx=0).stop 2023-12-19T17:01:20.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 -- 172.21.15.26:0/1756089978 shutdown_connections 2023-12-19T17:01:20.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 --2- 172.21.15.26:0/1756089978 >> v2:172.21.15.160:3300/0 conn(0x7f2154159c00 0x7f2154070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:20.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 --2- 172.21.15.26:0/1756089978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f215406f6c0 0x7f215415df30 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:20.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 -- 172.21.15.26:0/1756089978 >> 172.21.15.26:0/1756089978 conn(0x7f21540ce730 msgr2=0x7f21540ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:20.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 -- 172.21.15.26:0/1756089978 shutdown_connections 2023-12-19T17:01:20.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 -- 172.21.15.26:0/1756089978 wait complete. 2023-12-19T17:01:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 Processor -- start 2023-12-19T17:01:20.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 -- start start 2023-12-19T17:01:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f215406f6c0 0x7f2154150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2154159820 0x7f21541504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2154159c00 0x7f2154152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21540ca7d0 con 0x7f2154159c00 2023-12-19T17:01:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21541509f0 con 0x7f215406f6c0 2023-12-19T17:01:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f215aa66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21540ca4f0 con 0x7f2154159820 2023-12-19T17:01:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f2159003640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2154159c00 0x7f2154152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f2159003640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2154159c00 0x7f2154152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40840/0 (socket says 172.21.15.26:40840) 2023-12-19T17:01:20.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f2159003640 1 -- 172.21.15.26:0/1733641275 learned_addr learned my addr 172.21.15.26:0/1733641275 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:20.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f2153fff640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2154159820 0x7f21541504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:20.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.937+0000 7f2158802640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f215406f6c0 0x7f2154150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:20.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f2159003640 1 -- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2154159820 msgr2=0x7f21541504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:20.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f2159003640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2154159820 0x7f21541504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:20.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f2159003640 1 -- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f215406f6c0 msgr2=0x7f2154150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:20.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f2159003640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f215406f6c0 0x7f2154150b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:20.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f2159003640 1 -- 172.21.15.26:0/1733641275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21400094d0 con 0x7f2154159c00 2023-12-19T17:01:20.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f2158802640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f215406f6c0 0x7f2154150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:20.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f2159003640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2154159c00 0x7f2154152cc0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f214c009fd0 tx=0x7f214c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:20.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f21517fa640 1 -- 172.21.15.26:0/1733641275 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c014420 con 0x7f2154159c00 2023-12-19T17:01:20.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f21517fa640 1 -- 172.21.15.26:0/1733641275 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c014580 con 0x7f2154159c00 2023-12-19T17:01:20.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f21517fa640 1 -- 172.21.15.26:0/1733641275 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c014930 con 0x7f2154159c00 2023-12-19T17:01:20.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f215aa66640 1 -- 172.21.15.26:0/1733641275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215406b880 con 0x7f2154159c00 2023-12-19T17:01:20.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f215aa66640 1 -- 172.21.15.26:0/1733641275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21540b1f90 con 0x7f2154159c00 2023-12-19T17:01:20.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f215aa66640 1 -- 172.21.15.26:0/1733641275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21540b1b10 con 0x7f2154159c00 2023-12-19T17:01:20.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.941+0000 7f21517fa640 1 -- 172.21.15.26:0/1733641275 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f214c014a90 con 0x7f2154159c00 2023-12-19T17:01:20.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.945+0000 7f21517fa640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2138065bb0 0x7f2138068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:20.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.945+0000 7f21517fa640 1 -- 172.21.15.26:0/1733641275 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f214c084660 con 0x7f2154159c00 2023-12-19T17:01:20.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.945+0000 7f2158802640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2138065bb0 0x7f2138068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:20.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.945+0000 7f2158802640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2138065bb0 0x7f2138068070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2148003ec0 tx=0x7f2148009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:20.945+0000 7f21517fa640 1 -- 172.21.15.26:0/1733641275 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f214c051d70 con 0x7f2154159c00 2023-12-19T17:01:21.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.137+0000 7f215aa66640 1 -- 172.21.15.26:0/1733641275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f21540b1b10 con 0x7f2154159c00 2023-12-19T17:01:21.150 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 16 and data pool 17 2023-12-19T17:01:21.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.145+0000 7f21517fa640 1 -- 172.21.15.26:0/1733641275 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v61) v1 ==== 174+0+0 (secure 0 0 0) 0x7f21540b1b10 con 0x7f2154159c00 2023-12-19T17:01:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.153+0000 7f215aa66640 1 -- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2138065bb0 msgr2=0x7f2138068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.153+0000 7f215aa66640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2138065bb0 0x7f2138068070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2148003ec0 tx=0x7f2148009040 comp rx=0 tx=0).stop 2023-12-19T17:01:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.153+0000 7f215aa66640 1 -- 172.21.15.26:0/1733641275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2154159c00 msgr2=0x7f2154152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.153+0000 7f215aa66640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2154159c00 0x7f2154152cc0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f214c009fd0 tx=0x7f214c00e4b0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.157+0000 7f215aa66640 1 -- 172.21.15.26:0/1733641275 shutdown_connections 2023-12-19T17:01:21.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.157+0000 7f215aa66640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2154159c00 0x7f2154152cc0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.157+0000 7f215aa66640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2138065bb0 0x7f2138068070 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.157+0000 7f215aa66640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2154159820 0x7f21541504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.157+0000 7f215aa66640 1 --2- 172.21.15.26:0/1733641275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f215406f6c0 0x7f2154150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.157+0000 7f215aa66640 1 -- 172.21.15.26:0/1733641275 >> 172.21.15.26:0/1733641275 conn(0x7f21540ce730 msgr2=0x7f2154156670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:21.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.157+0000 7f215aa66640 1 -- 172.21.15.26:0/1733641275 shutdown_connections 2023-12-19T17:01:21.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.157+0000 7f215aa66640 1 -- 172.21.15.26:0/1733641275 wait complete. 2023-12-19T17:01:21.198 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:01:21.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9eab8f2640 1 Processor -- start 2023-12-19T17:01:21.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9eab8f2640 1 -- start start 2023-12-19T17:01:21.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9eab8f2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ea40ce040 0x7f9ea40d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9eab8f2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ea40d3130 0x7f9ea40d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9eab8f2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea40d29d0 con 0x7f9ea415cd40 2023-12-19T17:01:21.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9eab8f2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea40d3570 con 0x7f9ea40d3130 2023-12-19T17:01:21.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9eab8f2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea415d680 con 0x7f9ea40ce040 2023-12-19T17:01:21.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9ea9e8f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ea40d3130 0x7f9ea40d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9ea9e8f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ea40d3130 0x7f9ea40d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40846/0 (socket says 172.21.15.26:40846) 2023-12-19T17:01:21.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9ea9e8f640 1 -- 172.21.15.26:0/2852897083 learned_addr learned my addr 172.21.15.26:0/2852897083 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:21.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9ea8e8d640 1 --2- 172.21.15.26:0/2852897083 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ea40ce040 0x7f9ea40d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9ea9e8f640 1 -- 172.21.15.26:0/2852897083 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ea40ce040 msgr2=0x7f9ea40d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:21.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9ea9e8f640 1 --2- 172.21.15.26:0/2852897083 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ea40ce040 0x7f9ea40d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9ea9e8f640 1 -- 172.21.15.26:0/2852897083 >> v1:172.21.15.160:6789/0 conn(0x7f9ea415cd40 legacy=0x7f9ea4160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.337+0000 7f9ea9e8f640 1 -- 172.21.15.26:0/2852897083 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4161750 con 0x7f9ea40d3130 2023-12-19T17:01:21.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.341+0000 7f9ea9e8f640 1 --2- 172.21.15.26:0/2852897083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ea40d3130 0x7f9ea40d0e30 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c009e50 tx=0x7f9e9c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66f92e4f46f013b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:21.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.341+0000 7f9e9bfff640 1 -- 172.21.15.26:0/2852897083 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0a58b0 con 0x7f9ea40d3130 2023-12-19T17:01:21.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.341+0000 7f9e9bfff640 1 -- 172.21.15.26:0/2852897083 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c0a5a10 con 0x7f9ea40d3130 2023-12-19T17:01:21.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.341+0000 7f9e9bfff640 1 -- 172.21.15.26:0/2852897083 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0a5da0 con 0x7f9ea40d3130 2023-12-19T17:01:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.341+0000 7f9eab8f2640 1 -- 172.21.15.26:0/2852897083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ea40d3130 msgr2=0x7f9ea40d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.341+0000 7f9eab8f2640 1 --2- 172.21.15.26:0/2852897083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ea40d3130 0x7f9ea40d0e30 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c009e50 tx=0x7f9e9c01c9f0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.341+0000 7f9eab8f2640 1 -- 172.21.15.26:0/2852897083 shutdown_connections 2023-12-19T17:01:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.341+0000 7f9eab8f2640 1 --2- 172.21.15.26:0/2852897083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ea40d3130 0x7f9ea40d0e30 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.341+0000 7f9eab8f2640 1 --2- 172.21.15.26:0/2852897083 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ea40ce040 0x7f9ea40d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.341+0000 7f9eab8f2640 1 -- 172.21.15.26:0/2852897083 >> 172.21.15.26:0/2852897083 conn(0x7f9ea406a120 msgr2=0x7f9ea406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.341+0000 7f9eab8f2640 1 -- 172.21.15.26:0/2852897083 shutdown_connections 2023-12-19T17:01:21.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.341+0000 7f9eab8f2640 1 -- 172.21.15.26:0/2852897083 wait complete. 2023-12-19T17:01:21.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9eab8f2640 1 Processor -- start 2023-12-19T17:01:21.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9eab8f2640 1 -- start start 2023-12-19T17:01:21.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9eab8f2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ea40ce040 0x7f9ea4166440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9eab8f2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ea40d3130 0x7f9ea4154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9eab8f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ea415cd40 0x7f9ea415b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9eab8f2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea4157c80 con 0x7f9ea415cd40 2023-12-19T17:01:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9eab8f2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea4155570 con 0x7f9ea40d3130 2023-12-19T17:01:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9eab8f2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea41547b0 con 0x7f9ea40ce040 2023-12-19T17:01:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9ea9e8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ea415cd40 0x7f9ea415b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9ea9e8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ea415cd40 0x7f9ea415b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40848/0 (socket says 172.21.15.26:40848) 2023-12-19T17:01:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9ea9e8f640 1 -- 172.21.15.26:0/71795175 learned_addr learned my addr 172.21.15.26:0/71795175 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9ea968e640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ea40ce040 0x7f9ea4166440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9ea8e8d640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ea40d3130 0x7f9ea4154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9ea9e8f640 1 -- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ea40ce040 msgr2=0x7f9ea4166440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9ea9e8f640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ea40ce040 0x7f9ea4166440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9ea9e8f640 1 -- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ea40d3130 msgr2=0x7f9ea4154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9ea9e8f640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ea40d3130 0x7f9ea4154270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9ea9e8f640 1 -- 172.21.15.26:0/71795175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c009b30 con 0x7f9ea415cd40 2023-12-19T17:01:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9ea9e8f640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ea415cd40 0x7f9ea415b3a0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c01b0a0 tx=0x7f9e9c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9e99ffb640 1 -- 172.21.15.26:0/71795175 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0b8880 con 0x7f9ea415cd40 2023-12-19T17:01:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9e99ffb640 1 -- 172.21.15.26:0/71795175 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c0b89e0 con 0x7f9ea415cd40 2023-12-19T17:01:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9e99ffb640 1 -- 172.21.15.26:0/71795175 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0b8d90 con 0x7f9ea415cd40 2023-12-19T17:01:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9eab8f2640 1 -- 172.21.15.26:0/71795175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea4157090 con 0x7f9ea415cd40 2023-12-19T17:01:21.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.345+0000 7f9eab8f2640 1 -- 172.21.15.26:0/71795175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea4156ad0 con 0x7f9ea415cd40 2023-12-19T17:01:21.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.349+0000 7f9eab8f2640 1 -- 172.21.15.26:0/71795175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea41556b0 con 0x7f9ea415cd40 2023-12-19T17:01:21.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.349+0000 7f9e99ffb640 1 -- 172.21.15.26:0/71795175 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9e9c0b6440 con 0x7f9ea415cd40 2023-12-19T17:01:21.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.349+0000 7f9e99ffb640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9e88065b80 0x7f9e88068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.349+0000 7f9e99ffb640 1 -- 172.21.15.26:0/71795175 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f9e9c11ef30 con 0x7f9ea415cd40 2023-12-19T17:01:21.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.349+0000 7f9ea968e640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9e88065b80 0x7f9e88068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.349+0000 7f9ea968e640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9e88065b80 0x7f9e88068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0007fd0 tx=0x7f9ea0009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:21.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.353+0000 7f9e99ffb640 1 -- 172.21.15.26:0/71795175 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9ea41556b0 con 0x7f9ea415cd40 2023-12-19T17:01:21.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9eab8f2640 1 -- 172.21.15.26:0/71795175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ea41556b0 con 0x7f9ea415cd40 2023-12-19T17:01:21.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9e99ffb640 1 -- 172.21.15.26:0/71795175 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+17177 (secure 0 0 0) 0x7f9ea41556b0 con 0x7f9ea415cd40 2023-12-19T17:01:21.517 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:21.517 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":136,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:01:21.141752+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:01:19.682876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T17:01:20.140612+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:01:21.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9eab8f2640 1 -- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9e88065b80 msgr2=0x7f9e88068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9eab8f2640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9e88065b80 0x7f9e88068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0007fd0 tx=0x7f9ea0009040 comp rx=0 tx=0).stop 2023-12-19T17:01:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9eab8f2640 1 -- 172.21.15.26:0/71795175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ea415cd40 msgr2=0x7f9ea415b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9eab8f2640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ea415cd40 0x7f9ea415b3a0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c01b0a0 tx=0x7f9e9c019790 comp rx=0 tx=0).stop 2023-12-19T17:01:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9eab8f2640 1 -- 172.21.15.26:0/71795175 shutdown_connections 2023-12-19T17:01:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9eab8f2640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ea415cd40 0x7f9ea415b3a0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9eab8f2640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ea40d3130 0x7f9ea4154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9eab8f2640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9e88065b80 0x7f9e88068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9eab8f2640 1 --2- 172.21.15.26:0/71795175 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ea40ce040 0x7f9ea4166440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9eab8f2640 1 -- 172.21.15.26:0/71795175 >> 172.21.15.26:0/71795175 conn(0x7f9ea406a120 msgr2=0x7f9ea415d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9eab8f2640 1 -- 172.21.15.26:0/71795175 shutdown_connections 2023-12-19T17:01:21.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.513+0000 7f9eab8f2640 1 -- 172.21.15.26:0/71795175 wait complete. 2023-12-19T17:01:21.539 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:01:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa0e0f9640 1 Processor -- start 2023-12-19T17:01:21.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa0e0f9640 1 -- start start 2023-12-19T17:01:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa0e0f9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faa08154220 0x7faa0815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa0e0f9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0806bc30 con 0x7faa080cee10 2023-12-19T17:01:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa0e0f9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa080cc6e0 con 0x7faa0806b3d0 2023-12-19T17:01:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa0e0f9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0815afc0 con 0x7faa08154220 2023-12-19T17:01:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa07fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faa080cee10 0x7faa0815b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33336/0 (socket says 172.21.15.26:33336) 2023-12-19T17:01:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa07fff640 1 -- 172.21.15.26:0/3078358236 learned_addr learned my addr 172.21.15.26:0/3078358236 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa077fe640 1 --2- 172.21.15.26:0/3078358236 >> v2:172.21.15.160:3300/0 conn(0x7faa08154220 0x7faa0815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567573883 0 0) 0x7faa0806bc30 con 0x7faa080cee10 2023-12-19T17:01:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9f0003530 con 0x7faa080cee10 2023-12-19T17:01:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1446024688 0 0) 0x7fa9f0003530 con 0x7faa080cee10 2023-12-19T17:01:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 >> v2:172.21.15.160:3300/0 conn(0x7faa08154220 msgr2=0x7faa0815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa067fc640 1 --2- 172.21.15.26:0/3078358236 >> v2:172.21.15.160:3300/0 conn(0x7faa08154220 0x7faa0815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 >> v1:172.21.15.160:6789/0 conn(0x7faa0806b3d0 legacy=0x7faa080ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0806b950 con 0x7faa080cee10 2023-12-19T17:01:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7faa00001a00 con 0x7faa080cee10 2023-12-19T17:01:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.677+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 >> v1:172.21.15.26:6789/0 conn(0x7faa080cee10 legacy=0x7faa0815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa067fc640 1 --2- 172.21.15.26:0/3078358236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9f0003980 0x7fa9f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa067fc640 1 --2- 172.21.15.26:0/3078358236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa9f00047d0 0x7fa9f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa067fc640 1 --2- 172.21.15.26:0/3078358236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9f0005190 0x7fa9f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0815afc0 con 0x7fa9f0003980 2023-12-19T17:01:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f000c030 con 0x7fa9f0005190 2023-12-19T17:01:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f000c1f0 con 0x7fa9f00047d0 2023-12-19T17:01:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa077fe640 1 --2- 172.21.15.26:0/3078358236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9f0003980 0x7fa9f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa07fff640 1 --2- 172.21.15.26:0/3078358236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9f0005190 0x7fa9f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa06ffd640 1 --2- 172.21.15.26:0/3078358236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa9f00047d0 0x7fa9f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa077fe640 1 -- 172.21.15.26:0/3078358236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa9f00047d0 msgr2=0x7fa9f0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa077fe640 1 --2- 172.21.15.26:0/3078358236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa9f00047d0 0x7fa9f0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa077fe640 1 -- 172.21.15.26:0/3078358236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9f0005190 msgr2=0x7fa9f000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa077fe640 1 --2- 172.21.15.26:0/3078358236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9f0005190 0x7fa9f000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa077fe640 1 -- 172.21.15.26:0/3078358236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f0003530 con 0x7fa9f0003980 2023-12-19T17:01:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa07fff640 1 -- 172.21.15.26:0/3078358236 reap_dead start 2023-12-19T17:01:21.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa077fe640 1 --2- 172.21.15.26:0/3078358236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9f0003980 0x7fa9f0003da0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009ae0 tx=0x7fa9f801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c378d517561e96e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:21.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80a64e0 con 0x7fa9f0003980 2023-12-19T17:01:21.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f80a6640 con 0x7fa9f0003980 2023-12-19T17:01:21.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.681+0000 7faa067fc640 1 -- 172.21.15.26:0/3078358236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80a69f0 con 0x7fa9f0003980 2023-12-19T17:01:21.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 -- 172.21.15.26:0/3078358236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9f0003980 msgr2=0x7fa9f0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 --2- 172.21.15.26:0/3078358236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9f0003980 0x7fa9f0003da0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009ae0 tx=0x7fa9f801ca20 comp rx=0 tx=0).stop 2023-12-19T17:01:21.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 -- 172.21.15.26:0/3078358236 shutdown_connections 2023-12-19T17:01:21.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 --2- 172.21.15.26:0/3078358236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9f0003980 0x7fa9f0003da0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 -- 172.21.15.26:0/3078358236 >> 172.21.15.26:0/3078358236 conn(0x7faa080658e0 msgr2=0x7faa08065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:21.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 -- 172.21.15.26:0/3078358236 shutdown_connections 2023-12-19T17:01:21.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 -- 172.21.15.26:0/3078358236 wait complete. 2023-12-19T17:01:21.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 Processor -- start 2023-12-19T17:01:21.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 -- start start 2023-12-19T17:01:21.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9f0003980 0x7faa081590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa080b2c90 0x7faa0814c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa080b34b0 0x7faa08150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa080b1ed0 con 0x7fa9f0003980 2023-12-19T17:01:21.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa080b21b0 con 0x7faa080b34b0 2023-12-19T17:01:21.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa0e0f9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa080b2490 con 0x7faa080b2c90 2023-12-19T17:01:21.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9f0003980 0x7faa081590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9f0003980 0x7faa081590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40868/0 (socket says 172.21.15.26:40868) 2023-12-19T17:01:21.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa06ffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa080b2c90 0x7faa0814c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa077fe640 1 -- 172.21.15.26:0/2506564144 learned_addr learned my addr 172.21.15.26:0/2506564144 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:21.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.685+0000 7faa07fff640 1 --2- 172.21.15.26:0/2506564144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa080b34b0 0x7faa08150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7faa077fe640 1 -- 172.21.15.26:0/2506564144 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa080b2c90 msgr2=0x7faa0814c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7faa077fe640 1 --2- 172.21.15.26:0/2506564144 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa080b2c90 0x7faa0814c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7faa077fe640 1 -- 172.21.15.26:0/2506564144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa080b34b0 msgr2=0x7faa08150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7faa077fe640 1 --2- 172.21.15.26:0/2506564144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa080b34b0 0x7faa08150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7faa077fe640 1 -- 172.21.15.26:0/2506564144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa080b2770 con 0x7fa9f0003980 2023-12-19T17:01:21.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7faa077fe640 1 --2- 172.21.15.26:0/2506564144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9f0003980 0x7faa081590f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80b0040 tx=0x7fa9f0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:21.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7fa9effff640 1 -- 172.21.15.26:0/2506564144 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80a69b0 con 0x7fa9f0003980 2023-12-19T17:01:21.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7fa9effff640 1 -- 172.21.15.26:0/2506564144 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f80a6b10 con 0x7fa9f0003980 2023-12-19T17:01:21.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7fa9effff640 1 -- 172.21.15.26:0/2506564144 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f80a6e80 con 0x7fa9f0003980 2023-12-19T17:01:21.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7faa0e0f9640 1 -- 172.21.15.26:0/2506564144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0814c880 con 0x7fa9f0003980 2023-12-19T17:01:21.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7faa0e0f9640 1 -- 172.21.15.26:0/2506564144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0814dd30 con 0x7fa9f0003980 2023-12-19T17:01:21.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7faa0e0f9640 1 -- 172.21.15.26:0/2506564144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa08074270 con 0x7fa9f0003980 2023-12-19T17:01:21.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7fa9effff640 1 -- 172.21.15.26:0/2506564144 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa9f8006040 con 0x7fa9f0003980 2023-12-19T17:01:21.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7fa9effff640 1 --2- 172.21.15.26:0/2506564144 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa9f4065b80 0x7fa9f4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:21.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.689+0000 7fa9effff640 1 -- 172.21.15.26:0/2506564144 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fa9f80aa070 con 0x7fa9f0003980 2023-12-19T17:01:21.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.693+0000 7faa06ffd640 1 --2- 172.21.15.26:0/2506564144 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa9f4065b80 0x7fa9f4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:21.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.693+0000 7faa06ffd640 1 --2- 172.21.15.26:0/2506564144 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa9f4065b80 0x7fa9f4068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faa081f8e80 tx=0x7fa9fc009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:21.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.693+0000 7fa9effff640 1 -- 172.21.15.26:0/2506564144 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa9f80eb670 con 0x7fa9f0003980 2023-12-19T17:01:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.853+0000 7faa0e0f9640 1 -- 172.21.15.26:0/2506564144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa080ca6d0 con 0x7fa9f0003980 2023-12-19T17:01:21.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.857+0000 7fa9effff640 1 -- 172.21.15.26:0/2506564144 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+17177 (secure 0 0 0) 0x7fa9f80eec90 con 0x7fa9f0003980 2023-12-19T17:01:21.860 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:21.860 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":136,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:01:21.141752+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:01:19.682876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T17:01:20.140612+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:01:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.857+0000 7faa0e0f9640 1 -- 172.21.15.26:0/2506564144 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa9f4065b80 msgr2=0x7fa9f4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.857+0000 7faa0e0f9640 1 --2- 172.21.15.26:0/2506564144 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa9f4065b80 0x7fa9f4068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faa081f8e80 tx=0x7fa9fc009040 comp rx=0 tx=0).stop 2023-12-19T17:01:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.857+0000 7faa0e0f9640 1 -- 172.21.15.26:0/2506564144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9f0003980 msgr2=0x7faa081590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.857+0000 7faa0e0f9640 1 --2- 172.21.15.26:0/2506564144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9f0003980 0x7faa081590f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80b0040 tx=0x7fa9f0005800 comp rx=0 tx=0).stop 2023-12-19T17:01:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.857+0000 7faa0e0f9640 1 -- 172.21.15.26:0/2506564144 shutdown_connections 2023-12-19T17:01:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.857+0000 7faa0e0f9640 1 --2- 172.21.15.26:0/2506564144 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa080b34b0 0x7faa08150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.857+0000 7faa0e0f9640 1 --2- 172.21.15.26:0/2506564144 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa9f4065b80 0x7fa9f4068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.857+0000 7faa0e0f9640 1 --2- 172.21.15.26:0/2506564144 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa080b2c90 0x7faa0814c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.857+0000 7faa0e0f9640 1 --2- 172.21.15.26:0/2506564144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa9f0003980 0x7faa081590f0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.857+0000 7faa0e0f9640 1 -- 172.21.15.26:0/2506564144 >> 172.21.15.26:0/2506564144 conn(0x7faa080658e0 msgr2=0x7faa0815dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.857+0000 7faa0e0f9640 1 -- 172.21.15.26:0/2506564144 shutdown_connections 2023-12-19T17:01:21.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:21.857+0000 7faa0e0f9640 1 -- 172.21.15.26:0/2506564144 wait complete. 2023-12-19T17:01:21.882 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T17:01:22.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.021+0000 7f1c1a063640 1 Processor -- start 2023-12-19T17:01:22.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.021+0000 7f1c1a063640 1 -- start start 2023-12-19T17:01:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c1a063640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1c14161230 0x7f1c140ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c1a063640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c140b01b0 con 0x7f1c1406c3f0 2023-12-19T17:01:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c1a063640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c140adfb0 con 0x7f1c140c8df0 2023-12-19T17:01:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c1a063640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c140ca4e0 con 0x7f1c14161230 2023-12-19T17:01:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c12ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1c1406c3f0 0x7f1c1406c850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33342/0 (socket says 172.21.15.26:33342) 2023-12-19T17:01:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c12ffd640 1 -- 172.21.15.26:0/4139112222 learned_addr learned my addr 172.21.15.26:0/4139112222 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c137fe640 1 --2- 172.21.15.26:0/4139112222 >> v2:172.21.15.160:3300/0 conn(0x7f1c14161230 0x7f1c140ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415971827 0 0) 0x7f1c140b01b0 con 0x7f1c1406c3f0 2023-12-19T17:01:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bfc003530 con 0x7f1c1406c3f0 2023-12-19T17:01:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908561140 0 0) 0x7f1c140adfb0 con 0x7f1c140c8df0 2023-12-19T17:01:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c140b01b0 con 0x7f1c140c8df0 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 267364741 0 0) 0x7f1bfc003530 con 0x7f1c1406c3f0 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 >> v2:172.21.15.160:3300/0 conn(0x7f1c14161230 msgr2=0x7f1c140ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 --2- 172.21.15.26:0/4139112222 >> v2:172.21.15.160:3300/0 conn(0x7f1c14161230 0x7f1c140ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 >> v1:172.21.15.160:6789/0 conn(0x7f1c140c8df0 legacy=0x7f1c140c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c140b0400 con 0x7f1c1406c3f0 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f1c08001680 con 0x7f1c1406c3f0 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 >> v1:172.21.15.26:6789/0 conn(0x7f1c1406c3f0 legacy=0x7f1c1406c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 --2- 172.21.15.26:0/4139112222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bfc003b10 0x7f1bfc003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 --2- 172.21.15.26:0/4139112222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bfc0047d0 0x7f1bfc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 --2- 172.21.15.26:0/4139112222 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1bfc005190 0x7f1bfc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c140ca4e0 con 0x7f1bfc003b10 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc00c0c0 con 0x7f1bfc0047d0 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc00c280 con 0x7f1bfc005190 2023-12-19T17:01:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c137fe640 1 --2- 172.21.15.26:0/4139112222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bfc003b10 0x7f1bfc003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:22.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c12ffd640 1 --2- 172.21.15.26:0/4139112222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bfc0047d0 0x7f1bfc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:22.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c13fff640 1 --2- 172.21.15.26:0/4139112222 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1bfc005190 0x7f1bfc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:22.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c137fe640 1 -- 172.21.15.26:0/4139112222 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1bfc005190 msgr2=0x7f1bfc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:22.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c137fe640 1 --2- 172.21.15.26:0/4139112222 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1bfc005190 0x7f1bfc00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:22.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c137fe640 1 -- 172.21.15.26:0/4139112222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bfc0047d0 msgr2=0x7f1bfc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:22.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c137fe640 1 --2- 172.21.15.26:0/4139112222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bfc0047d0 0x7f1bfc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:22.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c137fe640 1 -- 172.21.15.26:0/4139112222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc003530 con 0x7f1bfc003b10 2023-12-19T17:01:22.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c13fff640 1 --2- 172.21.15.26:0/4139112222 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1bfc005190 0x7f1bfc00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:22.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.025+0000 7f1c13fff640 1 -- 172.21.15.26:0/4139112222 reap_dead start 2023-12-19T17:01:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.029+0000 7f1c137fe640 1 --2- 172.21.15.26:0/4139112222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bfc003b10 0x7f1bfc003f10 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009ae0 tx=0x7f1c0401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c23d61db8edffa9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.029+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c040a64d0 con 0x7f1bfc003b10 2023-12-19T17:01:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.029+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c040a6630 con 0x7f1bfc003b10 2023-12-19T17:01:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.029+0000 7f1c127fc640 1 -- 172.21.15.26:0/4139112222 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c040a69e0 con 0x7f1bfc003b10 2023-12-19T17:01:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.029+0000 7f1c1a063640 1 -- 172.21.15.26:0/4139112222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bfc003b10 msgr2=0x7f1bfc003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.029+0000 7f1c1a063640 1 --2- 172.21.15.26:0/4139112222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bfc003b10 0x7f1bfc003f10 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009ae0 tx=0x7f1c0401ca20 comp rx=0 tx=0).stop 2023-12-19T17:01:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.029+0000 7f1c1a063640 1 -- 172.21.15.26:0/4139112222 shutdown_connections 2023-12-19T17:01:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.029+0000 7f1c1a063640 1 --2- 172.21.15.26:0/4139112222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1bfc003b10 0x7f1bfc003f10 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.029+0000 7f1c1a063640 1 -- 172.21.15.26:0/4139112222 >> 172.21.15.26:0/4139112222 conn(0x7f1c140ce730 msgr2=0x7f1c140ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.029+0000 7f1c1a063640 1 -- 172.21.15.26:0/4139112222 shutdown_connections 2023-12-19T17:01:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.029+0000 7f1c1a063640 1 -- 172.21.15.26:0/4139112222 wait complete. 2023-12-19T17:01:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c1a063640 1 Processor -- start 2023-12-19T17:01:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c1a063640 1 -- start start 2023-12-19T17:01:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c1a063640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bfc003b10 0x7f1c1415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c1a063640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c140b35b0 0x7f1c140b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c1a063640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c140cb130 0x7f1c140cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c1a063640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c140b1dd0 con 0x7f1c140cb130 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c1a063640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c140cbaf0 con 0x7f1bfc003b10 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c1a063640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c140c4200 con 0x7f1c140b35b0 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c13fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c140cb130 0x7f1c140cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c13fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c140cb130 0x7f1c140cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40892/0 (socket says 172.21.15.26:40892) 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c13fff640 1 -- 172.21.15.26:0/3852695960 learned_addr learned my addr 172.21.15.26:0/3852695960 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c137fe640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bfc003b10 0x7f1c1415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c12ffd640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c140b35b0 0x7f1c140b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c13fff640 1 -- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c140b35b0 msgr2=0x7f1c140b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c13fff640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c140b35b0 0x7f1c140b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c13fff640 1 -- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bfc003b10 msgr2=0x7f1c1415b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c13fff640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bfc003b10 0x7f1c1415b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c13fff640 1 -- 172.21.15.26:0/3852695960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c140b4070 con 0x7f1c140cb130 2023-12-19T17:01:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c13fff640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c140cb130 0x7f1c140cb5b0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f1c140cf740 tx=0x7f1c0c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1bfbfff640 1 -- 172.21.15.26:0/3852695960 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c017070 con 0x7f1c140cb130 2023-12-19T17:01:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1bfbfff640 1 -- 172.21.15.26:0/3852695960 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0c00eec0 con 0x7f1c140cb130 2023-12-19T17:01:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1bfbfff640 1 -- 172.21.15.26:0/3852695960 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c003b90 con 0x7f1c140cb130 2023-12-19T17:01:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c1a063640 1 -- 172.21.15.26:0/3852695960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c140c38d0 con 0x7f1c140cb130 2023-12-19T17:01:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.033+0000 7f1c1a063640 1 -- 172.21.15.26:0/3852695960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c140c3e90 con 0x7f1c140cb130 2023-12-19T17:01:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.037+0000 7f1c1a063640 1 -- 172.21.15.26:0/3852695960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c14074270 con 0x7f1c140cb130 2023-12-19T17:01:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.037+0000 7f1bfbfff640 1 -- 172.21.15.26:0/3852695960 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1c0c003cf0 con 0x7f1c140cb130 2023-12-19T17:01:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.037+0000 7f1bfbfff640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1c00065b80 0x7f1c00068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.037+0000 7f1bfbfff640 1 -- 172.21.15.26:0/3852695960 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f1c0c082b90 con 0x7f1c140cb130 2023-12-19T17:01:22.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.037+0000 7f1c137fe640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1c00065b80 0x7f1c00068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:22.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.037+0000 7f1c137fe640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1c00065b80 0x7f1c00068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1c040af000 tx=0x7f1c040046d0 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:22.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.041+0000 7f1bfbfff640 1 -- 172.21.15.26:0/3852695960 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1c0c050170 con 0x7f1c140cb130 2023-12-19T17:01:22.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:22.233+0000 7f1c1a063640 1 -- 172.21.15.26:0/3852695960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1c140b46b0 con 0x7f1c140cb130 2023-12-19T17:01:22.667 INFO:tasks.ceph.mds.a.smithi026.stderr:2023-12-19T17:01:22.661+0000 7fb8475ec640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T17:01:23.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.165+0000 7f1bfbfff640 1 -- 172.21.15.26:0/3852695960 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1c0c053790 con 0x7f1c140cb130 2023-12-19T17:01:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.165+0000 7f1c1a063640 1 -- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1c00065b80 msgr2=0x7f1c00068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.165+0000 7f1c1a063640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1c00065b80 0x7f1c00068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1c040af000 tx=0x7f1c040046d0 comp rx=0 tx=0).stop 2023-12-19T17:01:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.165+0000 7f1c1a063640 1 -- 172.21.15.26:0/3852695960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c140cb130 msgr2=0x7f1c140cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.165+0000 7f1c1a063640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c140cb130 0x7f1c140cb5b0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f1c140cf740 tx=0x7f1c0c00ecd0 comp rx=0 tx=0).stop 2023-12-19T17:01:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.169+0000 7f1c1a063640 1 -- 172.21.15.26:0/3852695960 shutdown_connections 2023-12-19T17:01:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.169+0000 7f1c1a063640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c140cb130 0x7f1c140cb5b0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.169+0000 7f1c1a063640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1c00065b80 0x7f1c00068040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.169+0000 7f1c1a063640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c140b35b0 0x7f1c140b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.169+0000 7f1c1a063640 1 --2- 172.21.15.26:0/3852695960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1bfc003b10 0x7f1c1415b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:23.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.169+0000 7f1c1a063640 1 -- 172.21.15.26:0/3852695960 >> 172.21.15.26:0/3852695960 conn(0x7f1c140ce730 msgr2=0x7f1c140cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:23.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.169+0000 7f1c1a063640 1 -- 172.21.15.26:0/3852695960 shutdown_connections 2023-12-19T17:01:23.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.169+0000 7f1c1a063640 1 -- 172.21.15.26:0/3852695960 wait complete. 2023-12-19T17:01:23.196 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T17:01:23.196 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T17:01:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.341+0000 7f2b8a666640 1 Processor -- start 2023-12-19T17:01:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.341+0000 7f2b8a666640 1 -- start start 2023-12-19T17:01:23.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b8a666640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2b84159820 0x7f2b8415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b8a666640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8406fb60 con 0x7f2b8406f6c0 2023-12-19T17:01:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b8a666640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8406fe40 con 0x7f2b84159c00 2023-12-19T17:01:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b8a666640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8415e470 con 0x7f2b84159820 2023-12-19T17:01:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b837fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2b8406f6c0 0x7f2b840c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33356/0 (socket says 172.21.15.26:33356) 2023-12-19T17:01:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b837fe640 1 -- 172.21.15.26:0/930626470 learned_addr learned my addr 172.21.15.26:0/930626470 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b83fff640 1 --2- 172.21.15.26:0/930626470 >> v2:172.21.15.160:3300/0 conn(0x7f2b84159820 0x7f2b8415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1343990657 0 0) 0x7f2b8406fb60 con 0x7f2b8406f6c0 2023-12-19T17:01:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b6c003530 con 0x7f2b8406f6c0 2023-12-19T17:01:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 93252884 0 0) 0x7f2b6c003530 con 0x7f2b8406f6c0 2023-12-19T17:01:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 >> v2:172.21.15.160:3300/0 conn(0x7f2b84159820 msgr2=0x7f2b8415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/930626470 >> v2:172.21.15.160:3300/0 conn(0x7f2b84159820 0x7f2b8415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 >> v1:172.21.15.160:6789/0 conn(0x7f2b84159c00 legacy=0x7f2b84070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:23.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b84070120 con 0x7f2b8406f6c0 2023-12-19T17:01:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f2b78001680 con 0x7f2b8406f6c0 2023-12-19T17:01:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 >> v1:172.21.15.26:6789/0 conn(0x7f2b8406f6c0 legacy=0x7f2b840c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/930626470 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b6c003980 0x7f2b6c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/930626470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c004380 0x7f2b6c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 --2- 172.21.15.26:0/930626470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b6c005050 0x7f2b6c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8415e470 con 0x7f2b6c004380 2023-12-19T17:01:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c00bfb0 con 0x7f2b6c005050 2023-12-19T17:01:23.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c00c170 con 0x7f2b6c003980 2023-12-19T17:01:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b837fe640 1 --2- 172.21.15.26:0/930626470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c004380 0x7f2b6c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b83fff640 1 --2- 172.21.15.26:0/930626470 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b6c003980 0x7f2b6c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b837fe640 1 -- 172.21.15.26:0/930626470 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b6c003980 msgr2=0x7f2b6c003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b837fe640 1 --2- 172.21.15.26:0/930626470 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b6c003980 0x7f2b6c003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b837fe640 1 -- 172.21.15.26:0/930626470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b6c005050 msgr2=0x7f2b6c00b990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-19T17:01:23.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b837fe640 1 --2- 172.21.15.26:0/930626470 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b6c005050 0x7f2b6c00b990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b837fe640 1 -- 172.21.15.26:0/930626470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c003530 con 0x7f2b6c004380 2023-12-19T17:01:23.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.345+0000 7f2b88c03640 1 -- 172.21.15.26:0/930626470 reap_dead start 2023-12-19T17:01:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.349+0000 7f2b837fe640 1 --2- 172.21.15.26:0/930626470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c004380 0x7f2b6c004850 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f2b8415d480 tx=0x7f2b7801cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7616d625d2311a5c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.349+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78005ae0 con 0x7f2b6c004380 2023-12-19T17:01:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.349+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b78005c60 con 0x7f2b6c004380 2023-12-19T17:01:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.349+0000 7f2b82ffd640 1 -- 172.21.15.26:0/930626470 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b780a5a00 con 0x7f2b6c004380 2023-12-19T17:01:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.349+0000 7f2b8a666640 1 -- 172.21.15.26:0/930626470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c004380 msgr2=0x7f2b6c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:23.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.349+0000 7f2b8a666640 1 --2- 172.21.15.26:0/930626470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c004380 0x7f2b6c004850 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f2b8415d480 tx=0x7f2b7801cbd0 comp rx=0 tx=0).stop 2023-12-19T17:01:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.349+0000 7f2b8a666640 1 -- 172.21.15.26:0/930626470 shutdown_connections 2023-12-19T17:01:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.349+0000 7f2b8a666640 1 --2- 172.21.15.26:0/930626470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c004380 0x7f2b6c004850 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.349+0000 7f2b8a666640 1 -- 172.21.15.26:0/930626470 >> 172.21.15.26:0/930626470 conn(0x7f2b840ce730 msgr2=0x7f2b840ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.349+0000 7f2b8a666640 1 -- 172.21.15.26:0/930626470 shutdown_connections 2023-12-19T17:01:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.349+0000 7f2b8a666640 1 -- 172.21.15.26:0/930626470 wait complete. 2023-12-19T17:01:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b8a666640 1 Processor -- start 2023-12-19T17:01:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b8a666640 1 -- start start 2023-12-19T17:01:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b8a666640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c004380 0x7f2b840c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b8a666640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b840c5770 0x7f2b841f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:23.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b8a666640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b840ac860 0x7f2b840acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b8a666640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b840ad220 con 0x7f2b6c004380 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b8a666640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8415a020 con 0x7f2b840ac860 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b8a666640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b840caa00 con 0x7f2b840c5770 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b83fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c004380 0x7f2b840c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b83fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c004380 0x7f2b840c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40912/0 (socket says 172.21.15.26:40912) 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b83fff640 1 -- 172.21.15.26:0/2997710833 learned_addr learned my addr 172.21.15.26:0/2997710833 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b837fe640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b840c5770 0x7f2b841f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b88c03640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b840ac860 0x7f2b840acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b83fff640 1 -- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b840c5770 msgr2=0x7f2b841f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b83fff640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b840c5770 0x7f2b841f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b83fff640 1 -- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b840ac860 msgr2=0x7f2b840acce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b83fff640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b840ac860 0x7f2b840acce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b83fff640 1 -- 172.21.15.26:0/2997710833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8406b880 con 0x7f2b6c004380 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b88c03640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b840ac860 0x7f2b840acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:23.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b83fff640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c004380 0x7f2b840c9fd0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2b74011ad0 tx=0x7f2b74011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b80ff9640 1 -- 172.21.15.26:0/2997710833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7400fbb0 con 0x7f2b6c004380 2023-12-19T17:01:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b80ff9640 1 -- 172.21.15.26:0/2997710833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7400fd10 con 0x7f2b6c004380 2023-12-19T17:01:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b80ff9640 1 -- 172.21.15.26:0/2997710833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74014680 con 0x7f2b6c004380 2023-12-19T17:01:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b8a666640 1 -- 172.21.15.26:0/2997710833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b840c5ce0 con 0x7f2b6c004380 2023-12-19T17:01:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.353+0000 7f2b8a666640 1 -- 172.21.15.26:0/2997710833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8415a270 con 0x7f2b6c004380 2023-12-19T17:01:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.357+0000 7f2b8a666640 1 -- 172.21.15.26:0/2997710833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b840c6200 con 0x7f2b6c004380 2023-12-19T17:01:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.357+0000 7f2b80ff9640 1 -- 172.21.15.26:0/2997710833 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2b740147e0 con 0x7f2b6c004380 2023-12-19T17:01:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.357+0000 7f2b80ff9640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b70065bb0 0x7f2b70068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.357+0000 7f2b80ff9640 1 -- 172.21.15.26:0/2997710833 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f2b74083e60 con 0x7f2b6c004380 2023-12-19T17:01:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.357+0000 7f2b837fe640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b70065bb0 0x7f2b70068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:23.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.357+0000 7f2b837fe640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b70065bb0 0x7f2b70068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b780ab000 tx=0x7f2b780b1040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:23.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.361+0000 7f2b80ff9640 1 -- 172.21.15.26:0/2997710833 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2b74051430 con 0x7f2b6c004380 2023-12-19T17:01:23.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:23.553+0000 7f2b8a666640 1 -- 172.21.15.26:0/2997710833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2b84074270 con 0x7f2b6c004380 2023-12-19T17:01:24.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.173+0000 7f2b80ff9640 1 -- 172.21.15.26:0/2997710833 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2b740220c0 con 0x7f2b6c004380 2023-12-19T17:01:24.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.173+0000 7f2b8a666640 1 -- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b70065bb0 msgr2=0x7f2b70068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.173+0000 7f2b8a666640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b70065bb0 0x7f2b70068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b780ab000 tx=0x7f2b780b1040 comp rx=0 tx=0).stop 2023-12-19T17:01:24.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.173+0000 7f2b8a666640 1 -- 172.21.15.26:0/2997710833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c004380 msgr2=0x7f2b840c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.173+0000 7f2b8a666640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c004380 0x7f2b840c9fd0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2b74011ad0 tx=0x7f2b74011f90 comp rx=0 tx=0).stop 2023-12-19T17:01:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.173+0000 7f2b8a666640 1 -- 172.21.15.26:0/2997710833 shutdown_connections 2023-12-19T17:01:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.173+0000 7f2b8a666640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b840ac860 0x7f2b840acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.173+0000 7f2b8a666640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2b70065bb0 0x7f2b70068070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.173+0000 7f2b8a666640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b840c5770 0x7f2b841f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.173+0000 7f2b8a666640 1 --2- 172.21.15.26:0/2997710833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b6c004380 0x7f2b840c9fd0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.173+0000 7f2b8a666640 1 -- 172.21.15.26:0/2997710833 >> 172.21.15.26:0/2997710833 conn(0x7f2b840ce730 msgr2=0x7f2b84158d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.173+0000 7f2b8a666640 1 -- 172.21.15.26:0/2997710833 shutdown_connections 2023-12-19T17:01:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.173+0000 7f2b8a666640 1 -- 172.21.15.26:0/2997710833 wait complete. 2023-12-19T17:01:24.203 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T17:01:24.203 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:01:24.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.345+0000 7fcd18df0640 1 Processor -- start 2023-12-19T17:01:24.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.345+0000 7fcd18df0640 1 -- start start 2023-12-19T17:01:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.349+0000 7fcd18df0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd1415c7f0 0x7fcd1406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.349+0000 7fcd18df0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1406ced0 con 0x7fcd140adb80 2023-12-19T17:01:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.349+0000 7fcd18df0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1406d1b0 con 0x7fcd140b30d0 2023-12-19T17:01:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.349+0000 7fcd18df0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1406d770 con 0x7fcd1415c7f0 2023-12-19T17:01:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.349+0000 7fcd1259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd1415c7f0 0x7fcd1406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.349+0000 7fcd1259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd1415c7f0 0x7fcd1406c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40926/0 (socket says 172.21.15.26:40926) 2023-12-19T17:01:24.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.349+0000 7fcd1259c640 1 -- 172.21.15.26:0/1861245286 learned_addr learned my addr 172.21.15.26:0/1861245286 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:24.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.349+0000 7fcd1259c640 1 -- 172.21.15.26:0/1861245286 >> v1:172.21.15.160:6789/0 conn(0x7fcd140b30d0 legacy=0x7fcd140b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.349+0000 7fcd1259c640 1 -- 172.21.15.26:0/1861245286 >> v1:172.21.15.26:6789/0 conn(0x7fcd140adb80 legacy=0x7fcd140cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.349+0000 7fcd1259c640 1 -- 172.21.15.26:0/1861245286 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1406d490 con 0x7fcd1415c7f0 2023-12-19T17:01:24.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.349+0000 7fcd1159a640 1 -- 172.21.15.26:0/1861245286 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439701866 0 0) 0x7fcd1406ced0 con 0x7fcd140adb80 2023-12-19T17:01:24.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.353+0000 7fcd1259c640 1 --2- 172.21.15.26:0/1861245286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd1415c7f0 0x7fcd1406c7f0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009870 tx=0x7fcd0401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6547ca5a034e3c26 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:24.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.353+0000 7fcd1159a640 1 -- 172.21.15.26:0/1861245286 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd040ac630 con 0x7fcd1415c7f0 2023-12-19T17:01:24.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.353+0000 7fcd1159a640 1 -- 172.21.15.26:0/1861245286 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd040ac790 con 0x7fcd1415c7f0 2023-12-19T17:01:24.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.353+0000 7fcd1159a640 1 -- 172.21.15.26:0/1861245286 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd040acb20 con 0x7fcd1415c7f0 2023-12-19T17:01:24.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.353+0000 7fcd18df0640 1 -- 172.21.15.26:0/1861245286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd1415c7f0 msgr2=0x7fcd1406c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.353+0000 7fcd18df0640 1 --2- 172.21.15.26:0/1861245286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd1415c7f0 0x7fcd1406c7f0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009870 tx=0x7fcd0401c540 comp rx=0 tx=0).stop 2023-12-19T17:01:24.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.353+0000 7fcd18df0640 1 -- 172.21.15.26:0/1861245286 shutdown_connections 2023-12-19T17:01:24.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.353+0000 7fcd18df0640 1 --2- 172.21.15.26:0/1861245286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd1415c7f0 0x7fcd1406c7f0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.353+0000 7fcd18df0640 1 -- 172.21.15.26:0/1861245286 >> 172.21.15.26:0/1861245286 conn(0x7fcd140658e0 msgr2=0x7fcd14065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:24.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.353+0000 7fcd18df0640 1 -- 172.21.15.26:0/1861245286 shutdown_connections 2023-12-19T17:01:24.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.353+0000 7fcd18df0640 1 -- 172.21.15.26:0/1861245286 wait complete. 2023-12-19T17:01:24.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.353+0000 7fcd18df0640 1 Processor -- start 2023-12-19T17:01:24.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.353+0000 7fcd18df0640 1 -- start start 2023-12-19T17:01:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd18df0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd140adb80 0x7fcd1414cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd18df0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd140b30d0 0x7fcd1414e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd18df0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd1415c7f0 0x7fcd140aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd18df0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1414b880 con 0x7fcd1415c7f0 2023-12-19T17:01:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd18df0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd14149030 con 0x7fcd140b30d0 2023-12-19T17:01:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd18df0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1414b030 con 0x7fcd140adb80 2023-12-19T17:01:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd1415c7f0 0x7fcd140aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd1415c7f0 0x7fcd140aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40938/0 (socket says 172.21.15.26:40938) 2023-12-19T17:01:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd12d9d640 1 -- 172.21.15.26:0/3078508567 learned_addr learned my addr 172.21.15.26:0/3078508567 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd1259c640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd140adb80 0x7fcd1414cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd11d9b640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd140b30d0 0x7fcd1414e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd12d9d640 1 -- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd140adb80 msgr2=0x7fcd1414cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd12d9d640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd140adb80 0x7fcd1414cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd12d9d640 1 -- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd140b30d0 msgr2=0x7fcd1414e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd12d9d640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd140b30d0 0x7fcd1414e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd12d9d640 1 -- 172.21.15.26:0/3078508567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd040094d0 con 0x7fcd1415c7f0 2023-12-19T17:01:24.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd12d9d640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd1415c7f0 0x7fcd140aebd0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00c770 tx=0x7fcd0c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:24.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd02ffd640 1 -- 172.21.15.26:0/3078508567 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c003b80 con 0x7fcd1415c7f0 2023-12-19T17:01:24.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd02ffd640 1 -- 172.21.15.26:0/3078508567 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c003ce0 con 0x7fcd1415c7f0 2023-12-19T17:01:24.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd18df0640 1 -- 172.21.15.26:0/3078508567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1414b5a0 con 0x7fcd1415c7f0 2023-12-19T17:01:24.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd02ffd640 1 -- 172.21.15.26:0/3078508567 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c00aa90 con 0x7fcd1415c7f0 2023-12-19T17:01:24.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd18df0640 1 -- 172.21.15.26:0/3078508567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1414d1b0 con 0x7fcd1415c7f0 2023-12-19T17:01:24.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd18df0640 1 -- 172.21.15.26:0/3078508567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd14155850 con 0x7fcd1415c7f0 2023-12-19T17:01:24.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.357+0000 7fcd02ffd640 1 -- 172.21.15.26:0/3078508567 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fcd0c00abf0 con 0x7fcd1415c7f0 2023-12-19T17:01:24.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.361+0000 7fcd02ffd640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fccf4065b80 0x7fccf4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:24.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.361+0000 7fcd02ffd640 1 -- 172.21.15.26:0/3078508567 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fcd0c083370 con 0x7fcd1415c7f0 2023-12-19T17:01:24.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.361+0000 7fcd1259c640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fccf4065b80 0x7fccf4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:24.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.361+0000 7fcd1259c640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fccf4065b80 0x7fccf4068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcd040ae000 tx=0x7fcd040a7000 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:24.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.365+0000 7fcd02ffd640 1 -- 172.21.15.26:0/3078508567 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcd0c050950 con 0x7fcd1415c7f0 2023-12-19T17:01:24.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd18df0640 1 -- 172.21.15.26:0/3078508567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd14074270 con 0x7fcd1415c7f0 2023-12-19T17:01:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd02ffd640 1 -- 172.21.15.26:0/3078508567 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fcd0c053f70 con 0x7fcd1415c7f0 2023-12-19T17:01:24.557 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:24.557 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8092,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3217828830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3217828830},{"type":"v1","addr":"172.21.15.160:6835","nonce":3217828830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2758554130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2758554130},{"type":"v1","addr":"172.21.15.160:6837","nonce":2758554130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3510623843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3510623843},{"type":"v1","addr":"172.21.15.26:6833","nonce":3510623843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:01:21.142075+0000","modified":"2023-12-19T17:01:24.167832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8487":{"gid":8487,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3823167404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3823167404},{"type":"v1","addr":"172.21.15.26:6835","nonce":3823167404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T17:01:24.557 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 65 2023-12-19T17:01:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd18df0640 1 -- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fccf4065b80 msgr2=0x7fccf4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd18df0640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fccf4065b80 0x7fccf4068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcd040ae000 tx=0x7fcd040a7000 comp rx=0 tx=0).stop 2023-12-19T17:01:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd18df0640 1 -- 172.21.15.26:0/3078508567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd1415c7f0 msgr2=0x7fcd140aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd18df0640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd1415c7f0 0x7fcd140aebd0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00c770 tx=0x7fcd0c00cc30 comp rx=0 tx=0).stop 2023-12-19T17:01:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd18df0640 1 -- 172.21.15.26:0/3078508567 shutdown_connections 2023-12-19T17:01:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd18df0640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd1415c7f0 0x7fcd140aebd0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd18df0640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd140b30d0 0x7fcd1414e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd18df0640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fccf4065b80 0x7fccf4068040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd18df0640 1 --2- 172.21.15.26:0/3078508567 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd140adb80 0x7fcd1414cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd18df0640 1 -- 172.21.15.26:0/3078508567 >> 172.21.15.26:0/3078508567 conn(0x7fcd140658e0 msgr2=0x7fcd14067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd18df0640 1 -- 172.21.15.26:0/3078508567 shutdown_connections 2023-12-19T17:01:24.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.553+0000 7fcd18df0640 1 -- 172.21.15.26:0/3078508567 wait complete. 2023-12-19T17:01:24.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:01:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f312b939640 1 Processor -- start 2023-12-19T17:01:24.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f312b939640 1 -- start start 2023-12-19T17:01:24.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f312b939640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f312415c7f0 0x7f312406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f312b939640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f31240cc6e0 0x7f31240ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f312b939640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312406ced0 con 0x7f31240adb80 2023-12-19T17:01:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f312b939640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312406d770 con 0x7f312415c7f0 2023-12-19T17:01:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f312b939640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31240ceef0 con 0x7f31240cc6e0 2023-12-19T17:01:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f31296d5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f312415c7f0 0x7f312406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f3128ed4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f31240adb80 0x7f31240ce7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33362/0 (socket says 172.21.15.26:33362) 2023-12-19T17:01:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f3128ed4640 1 -- 172.21.15.26:0/2386637557 learned_addr learned my addr 172.21.15.26:0/2386637557 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:24.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f3129ed6640 1 --2- 172.21.15.26:0/2386637557 >> v2:172.21.15.160:3300/0 conn(0x7f31240cc6e0 0x7f31240ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f311bfff640 1 -- 172.21.15.26:0/2386637557 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925305447 0 0) 0x7f312406ced0 con 0x7f31240adb80 2023-12-19T17:01:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f311bfff640 1 -- 172.21.15.26:0/2386637557 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3110005740 con 0x7f31240adb80 2023-12-19T17:01:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f31296d5640 1 -- 172.21.15.26:0/2386637557 >> v2:172.21.15.160:3300/0 conn(0x7f31240cc6e0 msgr2=0x7f31240ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f31296d5640 1 --2- 172.21.15.26:0/2386637557 >> v2:172.21.15.160:3300/0 conn(0x7f31240cc6e0 0x7f31240ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f31296d5640 1 -- 172.21.15.26:0/2386637557 >> v1:172.21.15.26:6789/0 conn(0x7f31240adb80 legacy=0x7f31240ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f31296d5640 1 -- 172.21.15.26:0/2386637557 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312406d1b0 con 0x7f312415c7f0 2023-12-19T17:01:24.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.717+0000 7f3129ed6640 1 --2- 172.21.15.26:0/2386637557 >> v2:172.21.15.160:3300/0 conn(0x7f31240cc6e0 0x7f31240ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:01:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.721+0000 7f31296d5640 1 --2- 172.21.15.26:0/2386637557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312415c7f0 0x7f312406c7f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f31240cc420 tx=0x7f312001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8150356a6c86fed3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.721+0000 7f311bfff640 1 -- 172.21.15.26:0/2386637557 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31200ab600 con 0x7f312415c7f0 2023-12-19T17:01:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.721+0000 7f311bfff640 1 -- 172.21.15.26:0/2386637557 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31200a7050 con 0x7f312415c7f0 2023-12-19T17:01:24.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.721+0000 7f311bfff640 1 -- 172.21.15.26:0/2386637557 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31200ab980 con 0x7f312415c7f0 2023-12-19T17:01:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.721+0000 7f312b939640 1 -- 172.21.15.26:0/2386637557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312415c7f0 msgr2=0x7f312406c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.721+0000 7f312b939640 1 --2- 172.21.15.26:0/2386637557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312415c7f0 0x7f312406c7f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f31240cc420 tx=0x7f312001c440 comp rx=0 tx=0).stop 2023-12-19T17:01:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.721+0000 7f312b939640 1 -- 172.21.15.26:0/2386637557 shutdown_connections 2023-12-19T17:01:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.721+0000 7f312b939640 1 --2- 172.21.15.26:0/2386637557 >> v2:172.21.15.160:3300/0 conn(0x7f31240cc6e0 0x7f31240ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.721+0000 7f312b939640 1 --2- 172.21.15.26:0/2386637557 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f312415c7f0 0x7f312406c7f0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.721+0000 7f312b939640 1 -- 172.21.15.26:0/2386637557 >> 172.21.15.26:0/2386637557 conn(0x7f31240658e0 msgr2=0x7f3124065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.721+0000 7f312b939640 1 -- 172.21.15.26:0/2386637557 shutdown_connections 2023-12-19T17:01:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.721+0000 7f312b939640 1 -- 172.21.15.26:0/2386637557 wait complete. 2023-12-19T17:01:24.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f312b939640 1 Processor -- start 2023-12-19T17:01:24.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f312b939640 1 -- start start 2023-12-19T17:01:24.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f312b939640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31240adb80 0x7f3124151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:24.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f312b939640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31240cc6e0 0x7f31241465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:24.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f312b939640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312415c7f0 0x7f31241406f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:24.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f312b939640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3124146ae0 con 0x7f31240cc6e0 2023-12-19T17:01:24.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f312b939640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312414fcd0 con 0x7f31240adb80 2023-12-19T17:01:24.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f312b939640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312414ffb0 con 0x7f312415c7f0 2023-12-19T17:01:24.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f3128ed4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31240cc6e0 0x7f31241465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:24.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f3128ed4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31240cc6e0 0x7f31241465a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40956/0 (socket says 172.21.15.26:40956) 2023-12-19T17:01:24.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f3128ed4640 1 -- 172.21.15.26:0/185926870 learned_addr learned my addr 172.21.15.26:0/185926870 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:24.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f3128ed4640 1 -- 172.21.15.26:0/185926870 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312415c7f0 msgr2=0x7f31241406f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f3128ed4640 1 --2- 172.21.15.26:0/185926870 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312415c7f0 0x7f31241406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f3128ed4640 1 -- 172.21.15.26:0/185926870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31240adb80 msgr2=0x7f3124151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f3128ed4640 1 --2- 172.21.15.26:0/185926870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31240adb80 0x7f3124151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f3128ed4640 1 -- 172.21.15.26:0/185926870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c009a70 con 0x7f31240cc6e0 2023-12-19T17:01:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f3128ed4640 1 --2- 172.21.15.26:0/185926870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31240cc6e0 0x7f31241465a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f311400c4d0 tx=0x7f311400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f3119ffb640 1 -- 172.21.15.26:0/185926870 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114005cf0 con 0x7f31240cc6e0 2023-12-19T17:01:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f3119ffb640 1 -- 172.21.15.26:0/185926870 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3114005e50 con 0x7f31240cc6e0 2023-12-19T17:01:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f3119ffb640 1 -- 172.21.15.26:0/185926870 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114007270 con 0x7f31240cc6e0 2023-12-19T17:01:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f312b939640 1 -- 172.21.15.26:0/185926870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3120009420 con 0x7f31240cc6e0 2023-12-19T17:01:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.725+0000 7f312b939640 1 -- 172.21.15.26:0/185926870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3124150570 con 0x7f31240cc6e0 2023-12-19T17:01:24.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.729+0000 7f312b939640 1 -- 172.21.15.26:0/185926870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3124074270 con 0x7f31240cc6e0 2023-12-19T17:01:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.729+0000 7f3119ffb640 1 -- 172.21.15.26:0/185926870 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f31140073d0 con 0x7f31240cc6e0 2023-12-19T17:01:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.729+0000 7f3119ffb640 1 --2- 172.21.15.26:0/185926870 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3108065b80 0x7f3108068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.729+0000 7f3119ffb640 1 -- 172.21.15.26:0/185926870 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f3114082780 con 0x7f31240cc6e0 2023-12-19T17:01:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.729+0000 7f31296d5640 1 --2- 172.21.15.26:0/185926870 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3108065b80 0x7f3108068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.729+0000 7f31296d5640 1 --2- 172.21.15.26:0/185926870 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3108065b80 0x7f3108068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f31200ad000 tx=0x7f3120004740 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:24.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.733+0000 7f3119ffb640 1 -- 172.21.15.26:0/185926870 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f31140538b0 con 0x7f31240cc6e0 2023-12-19T17:01:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.893+0000 7f312b939640 1 -- 172.21.15.26:0/185926870 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3124070fd0 con 0x7f31240cc6e0 2023-12-19T17:01:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.893+0000 7f3119ffb640 1 -- 172.21.15.26:0/185926870 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+17164 (secure 0 0 0) 0x7f3114053440 con 0x7f31240cc6e0 2023-12-19T17:01:24.899 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:24.899 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":137,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:01:22.145249+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:01:19.682876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T17:01:20.140612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:01:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.897+0000 7f312b939640 1 -- 172.21.15.26:0/185926870 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3108065b80 msgr2=0x7f3108068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.897+0000 7f312b939640 1 --2- 172.21.15.26:0/185926870 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3108065b80 0x7f3108068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f31200ad000 tx=0x7f3120004740 comp rx=0 tx=0).stop 2023-12-19T17:01:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.897+0000 7f312b939640 1 -- 172.21.15.26:0/185926870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31240cc6e0 msgr2=0x7f31241465a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.897+0000 7f312b939640 1 --2- 172.21.15.26:0/185926870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31240cc6e0 0x7f31241465a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f311400c4d0 tx=0x7f311400c990 comp rx=0 tx=0).stop 2023-12-19T17:01:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.897+0000 7f312b939640 1 -- 172.21.15.26:0/185926870 shutdown_connections 2023-12-19T17:01:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.897+0000 7f312b939640 1 --2- 172.21.15.26:0/185926870 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3108065b80 0x7f3108068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.897+0000 7f312b939640 1 --2- 172.21.15.26:0/185926870 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f312415c7f0 0x7f31241406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.897+0000 7f312b939640 1 --2- 172.21.15.26:0/185926870 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31240cc6e0 0x7f31241465a0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.897+0000 7f312b939640 1 --2- 172.21.15.26:0/185926870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31240adb80 0x7f3124151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.897+0000 7f312b939640 1 -- 172.21.15.26:0/185926870 >> 172.21.15.26:0/185926870 conn(0x7f31240658e0 msgr2=0x7f312406f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.897+0000 7f312b939640 1 -- 172.21.15.26:0/185926870 shutdown_connections 2023-12-19T17:01:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:24.897+0000 7f312b939640 1 -- 172.21.15.26:0/185926870 wait complete. 2023-12-19T17:01:24.922 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T17:01:24.922 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T17:01:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.061+0000 7f7451dbc640 1 Processor -- start 2023-12-19T17:01:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.061+0000 7f7451dbc640 1 -- start start 2023-12-19T17:01:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.061+0000 7f7451dbc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f744c0cee10 0x7f744c164980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.061+0000 7f7451dbc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744c06bc30 con 0x7f744c154220 2023-12-19T17:01:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.061+0000 7f7451dbc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744c0cc6e0 con 0x7f744c06b3d0 2023-12-19T17:01:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.061+0000 7f7451dbc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f744c06b950 con 0x7f744c0cee10 2023-12-19T17:01:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.061+0000 7f744bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f744c0cee10 0x7f744c164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.061+0000 7f744bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f744c0cee10 0x7f744c164980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40966/0 (socket says 172.21.15.26:40966) 2023-12-19T17:01:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.061+0000 7f744bfff640 1 -- 172.21.15.26:0/959944887 learned_addr learned my addr 172.21.15.26:0/959944887 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.061+0000 7f744bfff640 1 -- 172.21.15.26:0/959944887 >> v1:172.21.15.160:6789/0 conn(0x7f744c06b3d0 legacy=0x7f744c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.061+0000 7f744bfff640 1 -- 172.21.15.26:0/959944887 >> v1:172.21.15.26:6789/0 conn(0x7f744c154220 legacy=0x7f744c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.061+0000 7f744bfff640 1 -- 172.21.15.26:0/959944887 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c0cc930 con 0x7f744c0cee10 2023-12-19T17:01:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.061+0000 7f744a7fc640 1 -- 172.21.15.26:0/959944887 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226196469 0 0) 0x7f744c06bc30 con 0x7f744c154220 2023-12-19T17:01:25.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.065+0000 7f744bfff640 1 --2- 172.21.15.26:0/959944887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f744c0cee10 0x7f744c164980 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f744400d150 tx=0x7f744401ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35abe9effc6829ba server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.065+0000 7f744a7fc640 1 -- 172.21.15.26:0/959944887 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744400b2d0 con 0x7f744c0cee10 2023-12-19T17:01:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.065+0000 7f744a7fc640 1 -- 172.21.15.26:0/959944887 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744400b430 con 0x7f744c0cee10 2023-12-19T17:01:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.065+0000 7f744a7fc640 1 -- 172.21.15.26:0/959944887 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74440a7bd0 con 0x7f744c0cee10 2023-12-19T17:01:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.065+0000 7f7451dbc640 1 -- 172.21.15.26:0/959944887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f744c0cee10 msgr2=0x7f744c164980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.065+0000 7f7451dbc640 1 --2- 172.21.15.26:0/959944887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f744c0cee10 0x7f744c164980 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f744400d150 tx=0x7f744401ebe0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.065+0000 7f7451dbc640 1 -- 172.21.15.26:0/959944887 shutdown_connections 2023-12-19T17:01:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.065+0000 7f7451dbc640 1 --2- 172.21.15.26:0/959944887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f744c0cee10 0x7f744c164980 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.065+0000 7f7451dbc640 1 -- 172.21.15.26:0/959944887 >> 172.21.15.26:0/959944887 conn(0x7f744c0658e0 msgr2=0x7f744c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.065+0000 7f7451dbc640 1 -- 172.21.15.26:0/959944887 shutdown_connections 2023-12-19T17:01:25.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.065+0000 7f7451dbc640 1 -- 172.21.15.26:0/959944887 wait complete. 2023-12-19T17:01:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f7451dbc640 1 Processor -- start 2023-12-19T17:01:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f7451dbc640 1 -- start start 2023-12-19T17:01:25.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f7451dbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f744c06b3d0 0x7f744c1422a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f7451dbc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f744c0cee10 0x7f744c070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f7451dbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f744c154220 0x7f744c070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f7451dbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c1427e0 con 0x7f744c154220 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f7451dbc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c1553e0 con 0x7f744c0cee10 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f7451dbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744c1556c0 con 0x7f744c06b3d0 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f744bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f744c154220 0x7f744c070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f744bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f744c154220 0x7f744c070eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40970/0 (socket says 172.21.15.26:40970) 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f744bfff640 1 -- 172.21.15.26:0/1735683427 learned_addr learned my addr 172.21.15.26:0/1735683427 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f744b7fe640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f744c06b3d0 0x7f744c1422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f744affd640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f744c0cee10 0x7f744c070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f744bfff640 1 -- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f744c06b3d0 msgr2=0x7f744c1422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f744bfff640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f744c06b3d0 0x7f744c1422a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f744bfff640 1 -- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f744c0cee10 msgr2=0x7f744c070970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f744bfff640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f744c0cee10 0x7f744c070970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f744bfff640 1 -- 172.21.15.26:0/1735683427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744400bd40 con 0x7f744c154220 2023-12-19T17:01:25.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f744bfff640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f744c154220 0x7f744c070eb0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f744400d150 tx=0x7f744401b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f7433fff640 1 -- 172.21.15.26:0/1735683427 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444009800 con 0x7f744c154220 2023-12-19T17:01:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f7433fff640 1 -- 172.21.15.26:0/1735683427 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7444009960 con 0x7f744c154220 2023-12-19T17:01:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f7451dbc640 1 -- 172.21.15.26:0/1735683427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744c1559a0 con 0x7f744c154220 2023-12-19T17:01:25.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f7433fff640 1 -- 172.21.15.26:0/1735683427 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7444009cf0 con 0x7f744c154220 2023-12-19T17:01:25.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.069+0000 7f7451dbc640 1 -- 172.21.15.26:0/1735683427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744c155f60 con 0x7f744c154220 2023-12-19T17:01:25.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.073+0000 7f7451dbc640 1 -- 172.21.15.26:0/1735683427 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744c074270 con 0x7f744c154220 2023-12-19T17:01:25.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.073+0000 7f7433fff640 1 -- 172.21.15.26:0/1735683427 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f74440af3f0 con 0x7f744c154220 2023-12-19T17:01:25.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.073+0000 7f7433fff640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7438065b80 0x7f7438068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.073+0000 7f7433fff640 1 -- 172.21.15.26:0/1735683427 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f744411e340 con 0x7f744c154220 2023-12-19T17:01:25.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.073+0000 7f744b7fe640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7438065b80 0x7f7438068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.073+0000 7f744b7fe640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7438065b80 0x7f7438068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f743c000f70 tx=0x7f743c009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:25.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.077+0000 7f7433fff640 1 -- 172.21.15.26:0/1735683427 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f74440eb870 con 0x7f744c154220 2023-12-19T17:01:25.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.233+0000 7f7451dbc640 1 -- 172.21.15.26:0/1735683427 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f744c0cba70 con 0x7f7438065b80 2023-12-19T17:01:25.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.245+0000 7f7433fff640 1 -- 172.21.15.26:0/1735683427 <== mgr.7082 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289360 (secure 0 0 0) 0x7f743c0050d0 con 0x7f7438065b80 2023-12-19T17:01:25.253 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:25.255 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T17:01:25.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.249+0000 7f7451dbc640 1 -- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7438065b80 msgr2=0x7f7438068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.249+0000 7f7451dbc640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7438065b80 0x7f7438068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f743c000f70 tx=0x7f743c009040 comp rx=0 tx=0).stop 2023-12-19T17:01:25.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.249+0000 7f7451dbc640 1 -- 172.21.15.26:0/1735683427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f744c154220 msgr2=0x7f744c070eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.249+0000 7f7451dbc640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f744c154220 0x7f744c070eb0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f744400d150 tx=0x7f744401b980 comp rx=0 tx=0).stop 2023-12-19T17:01:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.249+0000 7f7451dbc640 1 -- 172.21.15.26:0/1735683427 shutdown_connections 2023-12-19T17:01:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.249+0000 7f7451dbc640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f744c154220 0x7f744c070eb0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.249+0000 7f7451dbc640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f744c0cee10 0x7f744c070970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.249+0000 7f7451dbc640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7438065b80 0x7f7438068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.249+0000 7f7451dbc640 1 --2- 172.21.15.26:0/1735683427 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f744c06b3d0 0x7f744c1422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.249+0000 7f7451dbc640 1 -- 172.21.15.26:0/1735683427 >> 172.21.15.26:0/1735683427 conn(0x7f744c0658e0 msgr2=0x7f744c065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.249+0000 7f7451dbc640 1 -- 172.21.15.26:0/1735683427 shutdown_connections 2023-12-19T17:01:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.249+0000 7f7451dbc640 1 -- 172.21.15.26:0/1735683427 wait complete. 2023-12-19T17:01:25.273 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-19T17:01:23.467890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1250,"num_read_kb":2557,"num_write":439,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":285,"ondisk_log_size":285,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":345628,"kb_used_data":14428,"kb_used_omap":0,"kb_used_meta":331136,"kb_avail":754629092,"statfs":{"total":773094113280,"available":772740190208,"internally_reserved":0,"allocated":14774272,"data_stored":8103156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":339083264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000963"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509211+0000","last_change":"2023-12-19T17:01:21.167279+0000","last_active":"2023-12-19T17:01:22.509211+0000","last_peered":"2023-12-19T17:01:22.509211+0000","last_clean":"2023-12-19T17:01:22.509211+0000","last_became_active":"2023-12-19T17:01:21.167023+0000","last_became_peered":"2023-12-19T17:01:21.167023+0000","last_unstale":"2023-12-19T17:01:22.509211+0000","last_undegraded":"2023-12-19T17:01:22.509211+0000","last_fullsized":"2023-12-19T17:01:22.509211+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:09:33.899825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162479+0000","last_change":"2023-12-19T17:01:21.162772+0000","last_active":"2023-12-19T17:01:22.162479+0000","last_peered":"2023-12-19T17:01:22.162479+0000","last_clean":"2023-12-19T17:01:22.162479+0000","last_became_active":"2023-12-19T17:01:21.161491+0000","last_became_peered":"2023-12-19T17:01:21.161491+0000","last_unstale":"2023-12-19T17:01:22.162479+0000","last_undegraded":"2023-12-19T17:01:22.162479+0000","last_fullsized":"2023-12-19T17:01:22.162479+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:16:02.193853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508927+0000","last_change":"2023-12-19T17:01:21.178569+0000","last_active":"2023-12-19T17:01:22.508927+0000","last_peered":"2023-12-19T17:01:22.508927+0000","last_clean":"2023-12-19T17:01:22.508927+0000","last_became_active":"2023-12-19T17:01:21.178190+0000","last_became_peered":"2023-12-19T17:01:21.178190+0000","last_unstale":"2023-12-19T17:01:22.508927+0000","last_undegraded":"2023-12-19T17:01:22.508927+0000","last_fullsized":"2023-12-19T17:01:22.508927+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:26:19.065866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508750+0000","last_change":"2023-12-19T17:01:21.168709+0000","last_active":"2023-12-19T17:01:22.508750+0000","last_peered":"2023-12-19T17:01:22.508750+0000","last_clean":"2023-12-19T17:01:22.508750+0000","last_became_active":"2023-12-19T17:01:21.168181+0000","last_became_peered":"2023-12-19T17:01:21.168181+0000","last_unstale":"2023-12-19T17:01:22.508750+0000","last_undegraded":"2023-12-19T17:01:22.508750+0000","last_fullsized":"2023-12-19T17:01:22.508750+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:28:19.270066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481642+0000","last_change":"2023-12-19T17:01:21.173151+0000","last_active":"2023-12-19T17:01:22.481642+0000","last_peered":"2023-12-19T17:01:22.481642+0000","last_clean":"2023-12-19T17:01:22.481642+0000","last_became_active":"2023-12-19T17:01:21.172842+0000","last_became_peered":"2023-12-19T17:01:21.172842+0000","last_unstale":"2023-12-19T17:01:22.481642+0000","last_undegraded":"2023-12-19T17:01:22.481642+0000","last_fullsized":"2023-12-19T17:01:22.481642+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:40.001134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158109+0000","last_change":"2023-12-19T17:01:21.170905+0000","last_active":"2023-12-19T17:01:22.158109+0000","last_peered":"2023-12-19T17:01:22.158109+0000","last_clean":"2023-12-19T17:01:22.158109+0000","last_became_active":"2023-12-19T17:01:21.170639+0000","last_became_peered":"2023-12-19T17:01:21.170639+0000","last_unstale":"2023-12-19T17:01:22.158109+0000","last_undegraded":"2023-12-19T17:01:22.158109+0000","last_fullsized":"2023-12-19T17:01:22.158109+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:33.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163133+0000","last_change":"2023-12-19T17:01:21.164812+0000","last_active":"2023-12-19T17:01:22.163133+0000","last_peered":"2023-12-19T17:01:22.163133+0000","last_clean":"2023-12-19T17:01:22.163133+0000","last_became_active":"2023-12-19T17:01:21.164526+0000","last_became_peered":"2023-12-19T17:01:21.164526+0000","last_unstale":"2023-12-19T17:01:22.163133+0000","last_undegraded":"2023-12-19T17:01:22.163133+0000","last_fullsized":"2023-12-19T17:01:22.163133+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:12.213357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508959+0000","last_change":"2023-12-19T17:01:21.185785+0000","last_active":"2023-12-19T17:01:22.508959+0000","last_peered":"2023-12-19T17:01:22.508959+0000","last_clean":"2023-12-19T17:01:22.508959+0000","last_became_active":"2023-12-19T17:01:21.185526+0000","last_became_peered":"2023-12-19T17:01:21.185526+0000","last_unstale":"2023-12-19T17:01:22.508959+0000","last_undegraded":"2023-12-19T17:01:22.508959+0000","last_fullsized":"2023-12-19T17:01:22.508959+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:59:57.417117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509265+0000","last_change":"2023-12-19T17:01:21.170137+0000","last_active":"2023-12-19T17:01:22.509265+0000","last_peered":"2023-12-19T17:01:22.509265+0000","last_clean":"2023-12-19T17:01:22.509265+0000","last_became_active":"2023-12-19T17:01:21.169830+0000","last_became_peered":"2023-12-19T17:01:21.169830+0000","last_unstale":"2023-12-19T17:01:22.509265+0000","last_undegraded":"2023-12-19T17:01:22.509265+0000","last_fullsized":"2023-12-19T17:01:22.509265+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:38.425276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509107+0000","last_change":"2023-12-19T17:01:21.168575+0000","last_active":"2023-12-19T17:01:22.509107+0000","last_peered":"2023-12-19T17:01:22.509107+0000","last_clean":"2023-12-19T17:01:22.509107+0000","last_became_active":"2023-12-19T17:01:21.168298+0000","last_became_peered":"2023-12-19T17:01:21.168298+0000","last_unstale":"2023-12-19T17:01:22.509107+0000","last_undegraded":"2023-12-19T17:01:22.509107+0000","last_fullsized":"2023-12-19T17:01:22.509107+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:56:49.237027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162404+0000","last_change":"2023-12-19T17:01:21.167363+0000","last_active":"2023-12-19T17:01:22.162404+0000","last_peered":"2023-12-19T17:01:22.162404+0000","last_clean":"2023-12-19T17:01:22.162404+0000","last_became_active":"2023-12-19T17:01:21.167061+0000","last_became_peered":"2023-12-19T17:01:21.167061+0000","last_unstale":"2023-12-19T17:01:22.162404+0000","last_undegraded":"2023-12-19T17:01:22.162404+0000","last_fullsized":"2023-12-19T17:01:22.162404+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:02:02.542798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508844+0000","last_change":"2023-12-19T17:01:21.168531+0000","last_active":"2023-12-19T17:01:22.508844+0000","last_peered":"2023-12-19T17:01:22.508844+0000","last_clean":"2023-12-19T17:01:22.508844+0000","last_became_active":"2023-12-19T17:01:21.167885+0000","last_became_peered":"2023-12-19T17:01:21.167885+0000","last_unstale":"2023-12-19T17:01:22.508844+0000","last_undegraded":"2023-12-19T17:01:22.508844+0000","last_fullsized":"2023-12-19T17:01:22.508844+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:10:26.212444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509166+0000","last_change":"2023-12-19T17:01:21.182858+0000","last_active":"2023-12-19T17:01:22.509166+0000","last_peered":"2023-12-19T17:01:22.509166+0000","last_clean":"2023-12-19T17:01:22.509166+0000","last_became_active":"2023-12-19T17:01:21.182040+0000","last_became_peered":"2023-12-19T17:01:21.182040+0000","last_unstale":"2023-12-19T17:01:22.509166+0000","last_undegraded":"2023-12-19T17:01:22.509166+0000","last_fullsized":"2023-12-19T17:01:22.509166+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:51.742460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508675+0000","last_change":"2023-12-19T17:01:21.181107+0000","last_active":"2023-12-19T17:01:22.508675+0000","last_peered":"2023-12-19T17:01:22.508675+0000","last_clean":"2023-12-19T17:01:22.508675+0000","last_became_active":"2023-12-19T17:01:21.180021+0000","last_became_peered":"2023-12-19T17:01:21.180021+0000","last_unstale":"2023-12-19T17:01:22.508675+0000","last_undegraded":"2023-12-19T17:01:22.508675+0000","last_fullsized":"2023-12-19T17:01:22.508675+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:28.720065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.157928+0000","last_change":"2023-12-19T17:01:21.169449+0000","last_active":"2023-12-19T17:01:22.157928+0000","last_peered":"2023-12-19T17:01:22.157928+0000","last_clean":"2023-12-19T17:01:22.157928+0000","last_became_active":"2023-12-19T17:01:21.169207+0000","last_became_peered":"2023-12-19T17:01:21.169207+0000","last_unstale":"2023-12-19T17:01:22.157928+0000","last_undegraded":"2023-12-19T17:01:22.157928+0000","last_fullsized":"2023-12-19T17:01:22.157928+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:40.106699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508407+0000","last_change":"2023-12-19T17:01:21.174549+0000","last_active":"2023-12-19T17:01:22.508407+0000","last_peered":"2023-12-19T17:01:22.508407+0000","last_clean":"2023-12-19T17:01:22.508407+0000","last_became_active":"2023-12-19T17:01:21.174275+0000","last_became_peered":"2023-12-19T17:01:21.174275+0000","last_unstale":"2023-12-19T17:01:22.508407+0000","last_undegraded":"2023-12-19T17:01:22.508407+0000","last_fullsized":"2023-12-19T17:01:22.508407+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:56.423474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162421+0000","last_change":"2023-12-19T17:01:21.166560+0000","last_active":"2023-12-19T17:01:22.162421+0000","last_peered":"2023-12-19T17:01:22.162421+0000","last_clean":"2023-12-19T17:01:22.162421+0000","last_became_active":"2023-12-19T17:01:21.166300+0000","last_became_peered":"2023-12-19T17:01:21.166300+0000","last_unstale":"2023-12-19T17:01:22.162421+0000","last_undegraded":"2023-12-19T17:01:22.162421+0000","last_fullsized":"2023-12-19T17:01:22.162421+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:33.879282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509411+0000","last_change":"2023-12-19T17:01:21.165340+0000","last_active":"2023-12-19T17:01:22.509411+0000","last_peered":"2023-12-19T17:01:22.509411+0000","last_clean":"2023-12-19T17:01:22.509411+0000","last_became_active":"2023-12-19T17:01:21.165059+0000","last_became_peered":"2023-12-19T17:01:21.165059+0000","last_unstale":"2023-12-19T17:01:22.509411+0000","last_undegraded":"2023-12-19T17:01:22.509411+0000","last_fullsized":"2023-12-19T17:01:22.509411+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:36:28.680559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508935+0000","last_change":"2023-12-19T17:01:21.180898+0000","last_active":"2023-12-19T17:01:22.508935+0000","last_peered":"2023-12-19T17:01:22.508935+0000","last_clean":"2023-12-19T17:01:22.508935+0000","last_became_active":"2023-12-19T17:01:21.180585+0000","last_became_peered":"2023-12-19T17:01:21.180585+0000","last_unstale":"2023-12-19T17:01:22.508935+0000","last_undegraded":"2023-12-19T17:01:22.508935+0000","last_fullsized":"2023-12-19T17:01:22.508935+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:12:24.811047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162270+0000","last_change":"2023-12-19T17:01:21.174401+0000","last_active":"2023-12-19T17:01:22.162270+0000","last_peered":"2023-12-19T17:01:22.162270+0000","last_clean":"2023-12-19T17:01:22.162270+0000","last_became_active":"2023-12-19T17:01:21.173141+0000","last_became_peered":"2023-12-19T17:01:21.173141+0000","last_unstale":"2023-12-19T17:01:22.162270+0000","last_undegraded":"2023-12-19T17:01:22.162270+0000","last_fullsized":"2023-12-19T17:01:22.162270+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:11:26.427051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162273+0000","last_change":"2023-12-19T17:01:21.171120+0000","last_active":"2023-12-19T17:01:22.162273+0000","last_peered":"2023-12-19T17:01:22.162273+0000","last_clean":"2023-12-19T17:01:22.162273+0000","last_became_active":"2023-12-19T17:01:21.170859+0000","last_became_peered":"2023-12-19T17:01:21.170859+0000","last_unstale":"2023-12-19T17:01:22.162273+0000","last_undegraded":"2023-12-19T17:01:22.162273+0000","last_fullsized":"2023-12-19T17:01:22.162273+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:54:08.905754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508427+0000","last_change":"2023-12-19T17:01:21.166512+0000","last_active":"2023-12-19T17:01:22.508427+0000","last_peered":"2023-12-19T17:01:22.508427+0000","last_clean":"2023-12-19T17:01:22.508427+0000","last_became_active":"2023-12-19T17:01:21.166233+0000","last_became_peered":"2023-12-19T17:01:21.166233+0000","last_unstale":"2023-12-19T17:01:22.508427+0000","last_undegraded":"2023-12-19T17:01:22.508427+0000","last_fullsized":"2023-12-19T17:01:22.508427+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:30:00.737607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.161853+0000","last_change":"2023-12-19T17:01:21.170529+0000","last_active":"2023-12-19T17:01:22.161853+0000","last_peered":"2023-12-19T17:01:22.161853+0000","last_clean":"2023-12-19T17:01:22.161853+0000","last_became_active":"2023-12-19T17:01:21.169678+0000","last_became_peered":"2023-12-19T17:01:21.169678+0000","last_unstale":"2023-12-19T17:01:22.161853+0000","last_undegraded":"2023-12-19T17:01:22.161853+0000","last_fullsized":"2023-12-19T17:01:22.161853+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:15.142197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508191+0000","last_change":"2023-12-19T17:01:21.178360+0000","last_active":"2023-12-19T17:01:22.508191+0000","last_peered":"2023-12-19T17:01:22.508191+0000","last_clean":"2023-12-19T17:01:22.508191+0000","last_became_active":"2023-12-19T17:01:21.178092+0000","last_became_peered":"2023-12-19T17:01:21.178092+0000","last_unstale":"2023-12-19T17:01:22.508191+0000","last_undegraded":"2023-12-19T17:01:22.508191+0000","last_fullsized":"2023-12-19T17:01:22.508191+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:43.259541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162475+0000","last_change":"2023-12-19T17:01:21.165927+0000","last_active":"2023-12-19T17:01:22.162475+0000","last_peered":"2023-12-19T17:01:22.162475+0000","last_clean":"2023-12-19T17:01:22.162475+0000","last_became_active":"2023-12-19T17:01:21.165619+0000","last_became_peered":"2023-12-19T17:01:21.165619+0000","last_unstale":"2023-12-19T17:01:22.162475+0000","last_undegraded":"2023-12-19T17:01:22.162475+0000","last_fullsized":"2023-12-19T17:01:22.162475+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:36:28.680559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509176+0000","last_change":"2023-12-19T17:01:21.165757+0000","last_active":"2023-12-19T17:01:22.509176+0000","last_peered":"2023-12-19T17:01:22.509176+0000","last_clean":"2023-12-19T17:01:22.509176+0000","last_became_active":"2023-12-19T17:01:21.165487+0000","last_became_peered":"2023-12-19T17:01:21.165487+0000","last_unstale":"2023-12-19T17:01:22.509176+0000","last_undegraded":"2023-12-19T17:01:22.509176+0000","last_fullsized":"2023-12-19T17:01:22.509176+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:00:31.002290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158665+0000","last_change":"2023-12-19T17:01:21.169961+0000","last_active":"2023-12-19T17:01:22.158665+0000","last_peered":"2023-12-19T17:01:22.158665+0000","last_clean":"2023-12-19T17:01:22.158665+0000","last_became_active":"2023-12-19T17:01:21.169693+0000","last_became_peered":"2023-12-19T17:01:21.169693+0000","last_unstale":"2023-12-19T17:01:22.158665+0000","last_undegraded":"2023-12-19T17:01:22.158665+0000","last_fullsized":"2023-12-19T17:01:22.158665+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:54:08.905754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508514+0000","last_change":"2023-12-19T17:01:21.176635+0000","last_active":"2023-12-19T17:01:22.508514+0000","last_peered":"2023-12-19T17:01:22.508514+0000","last_clean":"2023-12-19T17:01:22.508514+0000","last_became_active":"2023-12-19T17:01:21.176360+0000","last_became_peered":"2023-12-19T17:01:21.176360+0000","last_unstale":"2023-12-19T17:01:22.508514+0000","last_undegraded":"2023-12-19T17:01:22.508514+0000","last_fullsized":"2023-12-19T17:01:22.508514+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:42.611360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.161967+0000","last_change":"2023-12-19T17:01:21.176706+0000","last_active":"2023-12-19T17:01:22.161967+0000","last_peered":"2023-12-19T17:01:22.161967+0000","last_clean":"2023-12-19T17:01:22.161967+0000","last_became_active":"2023-12-19T17:01:21.176115+0000","last_became_peered":"2023-12-19T17:01:21.176115+0000","last_unstale":"2023-12-19T17:01:22.161967+0000","last_undegraded":"2023-12-19T17:01:22.161967+0000","last_fullsized":"2023-12-19T17:01:22.161967+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:53:33.990500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163762+0000","last_change":"2023-12-19T17:01:21.167889+0000","last_active":"2023-12-19T17:01:22.163762+0000","last_peered":"2023-12-19T17:01:22.163762+0000","last_clean":"2023-12-19T17:01:22.163762+0000","last_became_active":"2023-12-19T17:01:21.167605+0000","last_became_peered":"2023-12-19T17:01:21.167605+0000","last_unstale":"2023-12-19T17:01:22.163762+0000","last_undegraded":"2023-12-19T17:01:22.163762+0000","last_fullsized":"2023-12-19T17:01:22.163762+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:47:26.481010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508675+0000","last_change":"2023-12-19T17:01:21.167703+0000","last_active":"2023-12-19T17:01:22.508675+0000","last_peered":"2023-12-19T17:01:22.508675+0000","last_clean":"2023-12-19T17:01:22.508675+0000","last_became_active":"2023-12-19T17:01:21.167445+0000","last_became_peered":"2023-12-19T17:01:21.167445+0000","last_unstale":"2023-12-19T17:01:22.508675+0000","last_undegraded":"2023-12-19T17:01:22.508675+0000","last_fullsized":"2023-12-19T17:01:22.508675+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:17:50.362533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.157928+0000","last_change":"2023-12-19T17:01:21.168392+0000","last_active":"2023-12-19T17:01:22.157928+0000","last_peered":"2023-12-19T17:01:22.157928+0000","last_clean":"2023-12-19T17:01:22.157928+0000","last_became_active":"2023-12-19T17:01:21.168124+0000","last_became_peered":"2023-12-19T17:01:21.168124+0000","last_unstale":"2023-12-19T17:01:22.157928+0000","last_undegraded":"2023-12-19T17:01:22.157928+0000","last_fullsized":"2023-12-19T17:01:22.157928+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:41:20.519158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158710+0000","last_change":"2023-12-19T17:01:21.159972+0000","last_active":"2023-12-19T17:01:22.158710+0000","last_peered":"2023-12-19T17:01:22.158710+0000","last_clean":"2023-12-19T17:01:22.158710+0000","last_became_active":"2023-12-19T17:01:21.159719+0000","last_became_peered":"2023-12-19T17:01:21.159719+0000","last_unstale":"2023-12-19T17:01:22.158710+0000","last_undegraded":"2023-12-19T17:01:22.158710+0000","last_fullsized":"2023-12-19T17:01:22.158710+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:14:52.849543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"136'4","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163268+0000","last_change":"2023-12-19T17:01:20.761392+0000","last_active":"2023-12-19T17:01:22.163268+0000","last_peered":"2023-12-19T17:01:22.163268+0000","last_clean":"2023-12-19T17:01:22.163268+0000","last_became_active":"2023-12-19T17:01:20.761089+0000","last_became_peered":"2023-12-19T17:01:20.761089+0000","last_unstale":"2023-12-19T17:01:22.163268+0000","last_undegraded":"2023-12-19T17:01:22.163268+0000","last_fullsized":"2023-12-19T17:01:22.163268+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:06.194676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509754+0000","last_change":"2023-12-19T17:01:21.174481+0000","last_active":"2023-12-19T17:01:22.509754+0000","last_peered":"2023-12-19T17:01:22.509754+0000","last_clean":"2023-12-19T17:01:22.509754+0000","last_became_active":"2023-12-19T17:01:21.174214+0000","last_became_peered":"2023-12-19T17:01:21.174214+0000","last_unstale":"2023-12-19T17:01:22.509754+0000","last_undegraded":"2023-12-19T17:01:22.509754+0000","last_fullsized":"2023-12-19T17:01:22.509754+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:02:02.542798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"136'1","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509001+0000","last_change":"2023-12-19T17:01:20.762211+0000","last_active":"2023-12-19T17:01:22.509001+0000","last_peered":"2023-12-19T17:01:22.509001+0000","last_clean":"2023-12-19T17:01:22.509001+0000","last_became_active":"2023-12-19T17:01:20.761905+0000","last_became_peered":"2023-12-19T17:01:20.761905+0000","last_unstale":"2023-12-19T17:01:22.509001+0000","last_undegraded":"2023-12-19T17:01:22.509001+0000","last_fullsized":"2023-12-19T17:01:22.509001+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:47:41.266771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158543+0000","last_change":"2023-12-19T17:01:21.171035+0000","last_active":"2023-12-19T17:01:22.158543+0000","last_peered":"2023-12-19T17:01:22.158543+0000","last_clean":"2023-12-19T17:01:22.158543+0000","last_became_active":"2023-12-19T17:01:21.170758+0000","last_became_peered":"2023-12-19T17:01:21.170758+0000","last_unstale":"2023-12-19T17:01:22.158543+0000","last_undegraded":"2023-12-19T17:01:22.158543+0000","last_fullsized":"2023-12-19T17:01:22.158543+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:16.978719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"136'2","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481647+0000","last_change":"2023-12-19T17:01:20.758467+0000","last_active":"2023-12-19T17:01:22.481647+0000","last_peered":"2023-12-19T17:01:22.481647+0000","last_clean":"2023-12-19T17:01:22.481647+0000","last_became_active":"2023-12-19T17:01:20.758207+0000","last_became_peered":"2023-12-19T17:01:20.758207+0000","last_unstale":"2023-12-19T17:01:22.481647+0000","last_undegraded":"2023-12-19T17:01:22.481647+0000","last_fullsized":"2023-12-19T17:01:22.481647+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:16:34.000691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509409+0000","last_change":"2023-12-19T17:01:21.181538+0000","last_active":"2023-12-19T17:01:22.509409+0000","last_peered":"2023-12-19T17:01:22.509409+0000","last_clean":"2023-12-19T17:01:22.509409+0000","last_became_active":"2023-12-19T17:01:21.181230+0000","last_became_peered":"2023-12-19T17:01:21.181230+0000","last_unstale":"2023-12-19T17:01:22.509409+0000","last_undegraded":"2023-12-19T17:01:22.509409+0000","last_fullsized":"2023-12-19T17:01:22.509409+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:37:05.457522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"136'2","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481531+0000","last_change":"2023-12-19T17:01:20.760542+0000","last_active":"2023-12-19T17:01:22.481531+0000","last_peered":"2023-12-19T17:01:22.481531+0000","last_clean":"2023-12-19T17:01:22.481531+0000","last_became_active":"2023-12-19T17:01:20.760282+0000","last_became_peered":"2023-12-19T17:01:20.760282+0000","last_unstale":"2023-12-19T17:01:22.481531+0000","last_undegraded":"2023-12-19T17:01:22.481531+0000","last_fullsized":"2023-12-19T17:01:22.481531+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:06.857504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508799+0000","last_change":"2023-12-19T17:01:21.179503+0000","last_active":"2023-12-19T17:01:22.508799+0000","last_peered":"2023-12-19T17:01:22.508799+0000","last_clean":"2023-12-19T17:01:22.508799+0000","last_became_active":"2023-12-19T17:01:21.179205+0000","last_became_peered":"2023-12-19T17:01:21.179205+0000","last_unstale":"2023-12-19T17:01:22.508799+0000","last_undegraded":"2023-12-19T17:01:22.508799+0000","last_fullsized":"2023-12-19T17:01:22.508799+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:47.078840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"136'1","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163221+0000","last_change":"2023-12-19T17:01:20.764974+0000","last_active":"2023-12-19T17:01:22.163221+0000","last_peered":"2023-12-19T17:01:22.163221+0000","last_clean":"2023-12-19T17:01:22.163221+0000","last_became_active":"2023-12-19T17:01:20.764671+0000","last_became_peered":"2023-12-19T17:01:20.764671+0000","last_unstale":"2023-12-19T17:01:22.163221+0000","last_undegraded":"2023-12-19T17:01:22.163221+0000","last_fullsized":"2023-12-19T17:01:22.163221+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:17:27.185290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.161964+0000","last_change":"2023-12-19T17:01:21.162892+0000","last_active":"2023-12-19T17:01:22.161964+0000","last_peered":"2023-12-19T17:01:22.161964+0000","last_clean":"2023-12-19T17:01:22.161964+0000","last_became_active":"2023-12-19T17:01:21.162305+0000","last_became_peered":"2023-12-19T17:01:21.162305+0000","last_unstale":"2023-12-19T17:01:22.161964+0000","last_undegraded":"2023-12-19T17:01:22.161964+0000","last_fullsized":"2023-12-19T17:01:22.161964+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:47:42.269522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"136'1","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508528+0000","last_change":"2023-12-19T17:01:20.758226+0000","last_active":"2023-12-19T17:01:22.508528+0000","last_peered":"2023-12-19T17:01:22.508528+0000","last_clean":"2023-12-19T17:01:22.508528+0000","last_became_active":"2023-12-19T17:01:20.757660+0000","last_became_peered":"2023-12-19T17:01:20.757660+0000","last_unstale":"2023-12-19T17:01:22.508528+0000","last_undegraded":"2023-12-19T17:01:22.508528+0000","last_fullsized":"2023-12-19T17:01:22.508528+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:37:04.454772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508320+0000","last_change":"2023-12-19T17:01:21.180268+0000","last_active":"2023-12-19T17:01:22.508320+0000","last_peered":"2023-12-19T17:01:22.508320+0000","last_clean":"2023-12-19T17:01:22.508320+0000","last_became_active":"2023-12-19T17:01:21.176971+0000","last_became_peered":"2023-12-19T17:01:21.176971+0000","last_unstale":"2023-12-19T17:01:22.508320+0000","last_undegraded":"2023-12-19T17:01:22.508320+0000","last_fullsized":"2023-12-19T17:01:22.508320+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:20:43.779315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.161920+0000","last_change":"2023-12-19T17:01:20.750004+0000","last_active":"2023-12-19T17:01:22.161920+0000","last_peered":"2023-12-19T17:01:22.161920+0000","last_clean":"2023-12-19T17:01:22.161920+0000","last_became_active":"2023-12-19T17:01:20.749674+0000","last_became_peered":"2023-12-19T17:01:20.749674+0000","last_unstale":"2023-12-19T17:01:22.161920+0000","last_undegraded":"2023-12-19T17:01:22.161920+0000","last_fullsized":"2023-12-19T17:01:22.161920+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:11.210607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163012+0000","last_change":"2023-12-19T17:01:21.166939+0000","last_active":"2023-12-19T17:01:22.163012+0000","last_peered":"2023-12-19T17:01:22.163012+0000","last_clean":"2023-12-19T17:01:22.163012+0000","last_became_active":"2023-12-19T17:01:21.166604+0000","last_became_peered":"2023-12-19T17:01:21.166604+0000","last_unstale":"2023-12-19T17:01:22.163012+0000","last_undegraded":"2023-12-19T17:01:22.163012+0000","last_fullsized":"2023-12-19T17:01:22.163012+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:58.423008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508407+0000","last_change":"2023-12-19T17:01:20.764982+0000","last_active":"2023-12-19T17:01:22.508407+0000","last_peered":"2023-12-19T17:01:22.508407+0000","last_clean":"2023-12-19T17:01:22.508407+0000","last_became_active":"2023-12-19T17:01:20.763688+0000","last_became_peered":"2023-12-19T17:01:20.763688+0000","last_unstale":"2023-12-19T17:01:22.508407+0000","last_undegraded":"2023-12-19T17:01:22.508407+0000","last_fullsized":"2023-12-19T17:01:22.508407+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:24.679053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163310+0000","last_change":"2023-12-19T17:01:21.165409+0000","last_active":"2023-12-19T17:01:22.163310+0000","last_peered":"2023-12-19T17:01:22.163310+0000","last_clean":"2023-12-19T17:01:22.163310+0000","last_became_active":"2023-12-19T17:01:21.165044+0000","last_became_peered":"2023-12-19T17:01:21.165044+0000","last_unstale":"2023-12-19T17:01:22.163310+0000","last_undegraded":"2023-12-19T17:01:22.163310+0000","last_fullsized":"2023-12-19T17:01:22.163310+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:20:35.663601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481661+0000","last_change":"2023-12-19T17:01:20.760021+0000","last_active":"2023-12-19T17:01:22.481661+0000","last_peered":"2023-12-19T17:01:22.481661+0000","last_clean":"2023-12-19T17:01:22.481661+0000","last_became_active":"2023-12-19T17:01:20.759760+0000","last_became_peered":"2023-12-19T17:01:20.759760+0000","last_unstale":"2023-12-19T17:01:22.481661+0000","last_undegraded":"2023-12-19T17:01:22.481661+0000","last_fullsized":"2023-12-19T17:01:22.481661+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:15:57.899075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509756+0000","last_change":"2023-12-19T17:01:21.171195+0000","last_active":"2023-12-19T17:01:22.509756+0000","last_peered":"2023-12-19T17:01:22.509756+0000","last_clean":"2023-12-19T17:01:22.509756+0000","last_became_active":"2023-12-19T17:01:21.170870+0000","last_became_peered":"2023-12-19T17:01:21.170870+0000","last_unstale":"2023-12-19T17:01:22.509756+0000","last_undegraded":"2023-12-19T17:01:22.509756+0000","last_fullsized":"2023-12-19T17:01:22.509756+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:12:49.539867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508719+0000","last_change":"2023-12-19T17:01:20.758357+0000","last_active":"2023-12-19T17:01:22.508719+0000","last_peered":"2023-12-19T17:01:22.508719+0000","last_clean":"2023-12-19T17:01:22.508719+0000","last_became_active":"2023-12-19T17:01:20.758063+0000","last_became_peered":"2023-12-19T17:01:20.758063+0000","last_unstale":"2023-12-19T17:01:22.508719+0000","last_undegraded":"2023-12-19T17:01:22.508719+0000","last_fullsized":"2023-12-19T17:01:22.508719+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:35.811938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508587+0000","last_change":"2023-12-19T17:01:21.167608+0000","last_active":"2023-12-19T17:01:22.508587+0000","last_peered":"2023-12-19T17:01:22.508587+0000","last_clean":"2023-12-19T17:01:22.508587+0000","last_became_active":"2023-12-19T17:01:21.167343+0000","last_became_peered":"2023-12-19T17:01:21.167343+0000","last_unstale":"2023-12-19T17:01:22.508587+0000","last_undegraded":"2023-12-19T17:01:22.508587+0000","last_fullsized":"2023-12-19T17:01:22.508587+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:02:13.132918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"136'3","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508915+0000","last_change":"2023-12-19T17:01:20.764005+0000","last_active":"2023-12-19T17:01:22.508915+0000","last_peered":"2023-12-19T17:01:22.508915+0000","last_clean":"2023-12-19T17:01:22.508915+0000","last_became_active":"2023-12-19T17:01:20.763666+0000","last_became_peered":"2023-12-19T17:01:20.763666+0000","last_unstale":"2023-12-19T17:01:22.508915+0000","last_undegraded":"2023-12-19T17:01:22.508915+0000","last_fullsized":"2023-12-19T17:01:22.508915+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:32:49.766097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163361+0000","last_change":"2023-12-19T17:01:21.176347+0000","last_active":"2023-12-19T17:01:22.163361+0000","last_peered":"2023-12-19T17:01:22.163361+0000","last_clean":"2023-12-19T17:01:22.163361+0000","last_became_active":"2023-12-19T17:01:21.176082+0000","last_became_peered":"2023-12-19T17:01:21.176082+0000","last_unstale":"2023-12-19T17:01:22.163361+0000","last_undegraded":"2023-12-19T17:01:22.163361+0000","last_fullsized":"2023-12-19T17:01:22.163361+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:27:13.839010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"136'6","reported_seq":17,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509012+0000","last_change":"2023-12-19T17:01:20.752058+0000","last_active":"2023-12-19T17:01:22.509012+0000","last_peered":"2023-12-19T17:01:22.509012+0000","last_clean":"2023-12-19T17:01:22.509012+0000","last_became_active":"2023-12-19T17:01:20.750865+0000","last_became_peered":"2023-12-19T17:01:20.750865+0000","last_unstale":"2023-12-19T17:01:22.509012+0000","last_undegraded":"2023-12-19T17:01:22.509012+0000","last_fullsized":"2023-12-19T17:01:22.509012+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:53:57.220194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508509+0000","last_change":"2023-12-19T17:01:21.166468+0000","last_active":"2023-12-19T17:01:22.508509+0000","last_peered":"2023-12-19T17:01:22.508509+0000","last_clean":"2023-12-19T17:01:22.508509+0000","last_became_active":"2023-12-19T17:01:21.166187+0000","last_became_peered":"2023-12-19T17:01:21.166187+0000","last_unstale":"2023-12-19T17:01:22.508509+0000","last_undegraded":"2023-12-19T17:01:22.508509+0000","last_fullsized":"2023-12-19T17:01:22.508509+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:13.881080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"136'2","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158408+0000","last_change":"2023-12-19T17:01:20.761730+0000","last_active":"2023-12-19T17:01:22.158408+0000","last_peered":"2023-12-19T17:01:22.158408+0000","last_clean":"2023-12-19T17:01:22.158408+0000","last_became_active":"2023-12-19T17:01:20.759271+0000","last_became_peered":"2023-12-19T17:01:20.759271+0000","last_unstale":"2023-12-19T17:01:22.158408+0000","last_undegraded":"2023-12-19T17:01:22.158408+0000","last_fullsized":"2023-12-19T17:01:22.158408+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:12:34.670243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508983+0000","last_change":"2023-12-19T17:01:21.178696+0000","last_active":"2023-12-19T17:01:22.508983+0000","last_peered":"2023-12-19T17:01:22.508983+0000","last_clean":"2023-12-19T17:01:22.508983+0000","last_became_active":"2023-12-19T17:01:21.178105+0000","last_became_peered":"2023-12-19T17:01:21.178105+0000","last_unstale":"2023-12-19T17:01:22.508983+0000","last_undegraded":"2023-12-19T17:01:22.508983+0000","last_fullsized":"2023-12-19T17:01:22.508983+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:33:25.066550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509165+0000","last_change":"2023-12-19T17:01:20.768570+0000","last_active":"2023-12-19T17:01:22.509165+0000","last_peered":"2023-12-19T17:01:22.509165+0000","last_clean":"2023-12-19T17:01:22.509165+0000","last_became_active":"2023-12-19T17:01:20.768218+0000","last_became_peered":"2023-12-19T17:01:20.768218+0000","last_unstale":"2023-12-19T17:01:22.509165+0000","last_undegraded":"2023-12-19T17:01:22.509165+0000","last_fullsized":"2023-12-19T17:01:22.509165+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:39.103948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508306+0000","last_change":"2023-12-19T17:01:21.166553+0000","last_active":"2023-12-19T17:01:22.508306+0000","last_peered":"2023-12-19T17:01:22.508306+0000","last_clean":"2023-12-19T17:01:22.508306+0000","last_became_active":"2023-12-19T17:01:21.166151+0000","last_became_peered":"2023-12-19T17:01:21.166151+0000","last_unstale":"2023-12-19T17:01:22.508306+0000","last_undegraded":"2023-12-19T17:01:22.508306+0000","last_fullsized":"2023-12-19T17:01:22.508306+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:48:34.666631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"136'8","reported_seq":19,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481428+0000","last_change":"2023-12-19T17:01:20.768850+0000","last_active":"2023-12-19T17:01:22.481428+0000","last_peered":"2023-12-19T17:01:22.481428+0000","last_clean":"2023-12-19T17:01:22.481428+0000","last_became_active":"2023-12-19T17:01:20.768183+0000","last_became_peered":"2023-12-19T17:01:20.768183+0000","last_unstale":"2023-12-19T17:01:22.481428+0000","last_undegraded":"2023-12-19T17:01:22.481428+0000","last_fullsized":"2023-12-19T17:01:22.481428+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:51:57.415839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481684+0000","last_change":"2023-12-19T17:01:21.171790+0000","last_active":"2023-12-19T17:01:22.481684+0000","last_peered":"2023-12-19T17:01:22.481684+0000","last_clean":"2023-12-19T17:01:22.481684+0000","last_became_active":"2023-12-19T17:01:21.171292+0000","last_became_peered":"2023-12-19T17:01:21.171292+0000","last_unstale":"2023-12-19T17:01:22.481684+0000","last_undegraded":"2023-12-19T17:01:22.481684+0000","last_fullsized":"2023-12-19T17:01:22.481684+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:13:33.472416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"136'1","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509354+0000","last_change":"2023-12-19T17:01:20.767391+0000","last_active":"2023-12-19T17:01:22.509354+0000","last_peered":"2023-12-19T17:01:22.509354+0000","last_clean":"2023-12-19T17:01:22.509354+0000","last_became_active":"2023-12-19T17:01:20.766442+0000","last_became_peered":"2023-12-19T17:01:20.766442+0000","last_unstale":"2023-12-19T17:01:22.509354+0000","last_undegraded":"2023-12-19T17:01:22.509354+0000","last_fullsized":"2023-12-19T17:01:22.509354+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:46:27.003043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158876+0000","last_change":"2023-12-19T17:01:21.172659+0000","last_active":"2023-12-19T17:01:22.158876+0000","last_peered":"2023-12-19T17:01:22.158876+0000","last_clean":"2023-12-19T17:01:22.158876+0000","last_became_active":"2023-12-19T17:01:21.172390+0000","last_became_peered":"2023-12-19T17:01:21.172390+0000","last_unstale":"2023-12-19T17:01:22.158876+0000","last_undegraded":"2023-12-19T17:01:22.158876+0000","last_fullsized":"2023-12-19T17:01:22.158876+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:16:02.193853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"136'2","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481712+0000","last_change":"2023-12-19T17:01:20.759071+0000","last_active":"2023-12-19T17:01:22.481712+0000","last_peered":"2023-12-19T17:01:22.481712+0000","last_clean":"2023-12-19T17:01:22.481712+0000","last_became_active":"2023-12-19T17:01:20.757647+0000","last_became_peered":"2023-12-19T17:01:20.757647+0000","last_unstale":"2023-12-19T17:01:22.481712+0000","last_undegraded":"2023-12-19T17:01:22.481712+0000","last_fullsized":"2023-12-19T17:01:22.481712+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:52:56.973652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508859+0000","last_change":"2023-12-19T17:01:21.179611+0000","last_active":"2023-12-19T17:01:22.508859+0000","last_peered":"2023-12-19T17:01:22.508859+0000","last_clean":"2023-12-19T17:01:22.508859+0000","last_became_active":"2023-12-19T17:01:21.179025+0000","last_became_peered":"2023-12-19T17:01:21.179025+0000","last_unstale":"2023-12-19T17:01:22.508859+0000","last_undegraded":"2023-12-19T17:01:22.508859+0000","last_fullsized":"2023-12-19T17:01:22.508859+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:12.354302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"136'1","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509401+0000","last_change":"2023-12-19T17:01:20.760598+0000","last_active":"2023-12-19T17:01:22.509401+0000","last_peered":"2023-12-19T17:01:22.509401+0000","last_clean":"2023-12-19T17:01:22.509401+0000","last_became_active":"2023-12-19T17:01:20.760267+0000","last_became_peered":"2023-12-19T17:01:20.760267+0000","last_unstale":"2023-12-19T17:01:22.509401+0000","last_undegraded":"2023-12-19T17:01:22.509401+0000","last_fullsized":"2023-12-19T17:01:22.509401+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:15.975968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509359+0000","last_change":"2023-12-19T17:01:21.180915+0000","last_active":"2023-12-19T17:01:22.509359+0000","last_peered":"2023-12-19T17:01:22.509359+0000","last_clean":"2023-12-19T17:01:22.509359+0000","last_became_active":"2023-12-19T17:01:21.180574+0000","last_became_peered":"2023-12-19T17:01:21.180574+0000","last_unstale":"2023-12-19T17:01:22.509359+0000","last_undegraded":"2023-12-19T17:01:22.509359+0000","last_fullsized":"2023-12-19T17:01:22.509359+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:33:36.026852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163327+0000","last_change":"2023-12-19T17:01:20.769315+0000","last_active":"2023-12-19T17:01:22.163327+0000","last_peered":"2023-12-19T17:01:22.163327+0000","last_clean":"2023-12-19T17:01:22.163327+0000","last_became_active":"2023-12-19T17:01:20.768969+0000","last_became_peered":"2023-12-19T17:01:20.768969+0000","last_unstale":"2023-12-19T17:01:22.163327+0000","last_undegraded":"2023-12-19T17:01:22.163327+0000","last_fullsized":"2023-12-19T17:01:22.163327+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:59.214738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508780+0000","last_change":"2023-12-19T17:01:21.178808+0000","last_active":"2023-12-19T17:01:22.508780+0000","last_peered":"2023-12-19T17:01:22.508780+0000","last_clean":"2023-12-19T17:01:22.508780+0000","last_became_active":"2023-12-19T17:01:21.178434+0000","last_became_peered":"2023-12-19T17:01:21.178434+0000","last_unstale":"2023-12-19T17:01:22.508780+0000","last_undegraded":"2023-12-19T17:01:22.508780+0000","last_fullsized":"2023-12-19T17:01:22.508780+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:53:36.243483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"136'2","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158427+0000","last_change":"2023-12-19T17:01:20.764438+0000","last_active":"2023-12-19T17:01:22.158427+0000","last_peered":"2023-12-19T17:01:22.158427+0000","last_clean":"2023-12-19T17:01:22.158427+0000","last_became_active":"2023-12-19T17:01:20.764157+0000","last_became_peered":"2023-12-19T17:01:20.764157+0000","last_unstale":"2023-12-19T17:01:22.158427+0000","last_undegraded":"2023-12-19T17:01:22.158427+0000","last_fullsized":"2023-12-19T17:01:22.158427+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:31:49.518342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508661+0000","last_change":"2023-12-19T17:01:21.181487+0000","last_active":"2023-12-19T17:01:22.508661+0000","last_peered":"2023-12-19T17:01:22.508661+0000","last_clean":"2023-12-19T17:01:22.508661+0000","last_became_active":"2023-12-19T17:01:21.181227+0000","last_became_peered":"2023-12-19T17:01:21.181227+0000","last_unstale":"2023-12-19T17:01:22.508661+0000","last_undegraded":"2023-12-19T17:01:22.508661+0000","last_fullsized":"2023-12-19T17:01:22.508661+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:26.258189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481409+0000","last_change":"2023-12-19T17:01:20.757946+0000","last_active":"2023-12-19T17:01:22.481409+0000","last_peered":"2023-12-19T17:01:22.481409+0000","last_clean":"2023-12-19T17:01:22.481409+0000","last_became_active":"2023-12-19T17:01:20.757645+0000","last_became_peered":"2023-12-19T17:01:20.757645+0000","last_unstale":"2023-12-19T17:01:22.481409+0000","last_undegraded":"2023-12-19T17:01:22.481409+0000","last_fullsized":"2023-12-19T17:01:22.481409+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:07.182292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509041+0000","last_change":"2023-12-19T17:01:21.181146+0000","last_active":"2023-12-19T17:01:22.509041+0000","last_peered":"2023-12-19T17:01:22.509041+0000","last_clean":"2023-12-19T17:01:22.509041+0000","last_became_active":"2023-12-19T17:01:21.180709+0000","last_became_peered":"2023-12-19T17:01:21.180709+0000","last_unstale":"2023-12-19T17:01:22.509041+0000","last_undegraded":"2023-12-19T17:01:22.509041+0000","last_fullsized":"2023-12-19T17:01:22.509041+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:36.814688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.157961+0000","last_change":"2023-12-19T17:01:20.764680+0000","last_active":"2023-12-19T17:01:22.157961+0000","last_peered":"2023-12-19T17:01:22.157961+0000","last_clean":"2023-12-19T17:01:22.157961+0000","last_became_active":"2023-12-19T17:01:20.764412+0000","last_became_peered":"2023-12-19T17:01:20.764412+0000","last_unstale":"2023-12-19T17:01:22.157961+0000","last_undegraded":"2023-12-19T17:01:22.157961+0000","last_fullsized":"2023-12-19T17:01:22.157961+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:09.971025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508678+0000","last_change":"2023-12-19T17:01:21.175298+0000","last_active":"2023-12-19T17:01:22.508678+0000","last_peered":"2023-12-19T17:01:22.508678+0000","last_clean":"2023-12-19T17:01:22.508678+0000","last_became_active":"2023-12-19T17:01:21.174897+0000","last_became_peered":"2023-12-19T17:01:21.174897+0000","last_unstale":"2023-12-19T17:01:22.508678+0000","last_undegraded":"2023-12-19T17:01:22.508678+0000","last_fullsized":"2023-12-19T17:01:22.508678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:30.004851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508505+0000","last_change":"2023-12-19T17:01:20.765848+0000","last_active":"2023-12-19T17:01:22.508505+0000","last_peered":"2023-12-19T17:01:22.508505+0000","last_clean":"2023-12-19T17:01:22.508505+0000","last_became_active":"2023-12-19T17:01:20.764435+0000","last_became_peered":"2023-12-19T17:01:20.764435+0000","last_unstale":"2023-12-19T17:01:22.508505+0000","last_undegraded":"2023-12-19T17:01:22.508505+0000","last_fullsized":"2023-12-19T17:01:22.508505+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:54:53.906874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509461+0000","last_change":"2023-12-19T17:01:21.169428+0000","last_active":"2023-12-19T17:01:22.509461+0000","last_peered":"2023-12-19T17:01:22.509461+0000","last_clean":"2023-12-19T17:01:22.509461+0000","last_became_active":"2023-12-19T17:01:21.168788+0000","last_became_peered":"2023-12-19T17:01:21.168788+0000","last_unstale":"2023-12-19T17:01:22.509461+0000","last_undegraded":"2023-12-19T17:01:22.509461+0000","last_fullsized":"2023-12-19T17:01:22.509461+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:15.142197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"136'2","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508881+0000","last_change":"2023-12-19T17:01:20.755864+0000","last_active":"2023-12-19T17:01:22.508881+0000","last_peered":"2023-12-19T17:01:22.508881+0000","last_clean":"2023-12-19T17:01:22.508881+0000","last_became_active":"2023-12-19T17:01:20.755586+0000","last_became_peered":"2023-12-19T17:01:20.755586+0000","last_unstale":"2023-12-19T17:01:22.508881+0000","last_undegraded":"2023-12-19T17:01:22.508881+0000","last_fullsized":"2023-12-19T17:01:22.508881+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:26:18.063116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509260+0000","last_change":"2023-12-19T17:01:21.166827+0000","last_active":"2023-12-19T17:01:22.509260+0000","last_peered":"2023-12-19T17:01:22.509260+0000","last_clean":"2023-12-19T17:01:22.509260+0000","last_became_active":"2023-12-19T17:01:21.166551+0000","last_became_peered":"2023-12-19T17:01:21.166551+0000","last_unstale":"2023-12-19T17:01:22.509260+0000","last_undegraded":"2023-12-19T17:01:22.509260+0000","last_fullsized":"2023-12-19T17:01:22.509260+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:31.155271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509572+0000","last_change":"2023-12-19T17:01:20.759224+0000","last_active":"2023-12-19T17:01:22.509572+0000","last_peered":"2023-12-19T17:01:22.509572+0000","last_clean":"2023-12-19T17:01:22.509572+0000","last_became_active":"2023-12-19T17:01:20.758939+0000","last_became_peered":"2023-12-19T17:01:20.758939+0000","last_unstale":"2023-12-19T17:01:22.509572+0000","last_undegraded":"2023-12-19T17:01:22.509572+0000","last_fullsized":"2023-12-19T17:01:22.509572+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:32.253014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508932+0000","last_change":"2023-12-19T17:01:21.173966+0000","last_active":"2023-12-19T17:01:22.508932+0000","last_peered":"2023-12-19T17:01:22.508932+0000","last_clean":"2023-12-19T17:01:22.508932+0000","last_became_active":"2023-12-19T17:01:21.171653+0000","last_became_peered":"2023-12-19T17:01:21.171653+0000","last_unstale":"2023-12-19T17:01:22.508932+0000","last_undegraded":"2023-12-19T17:01:22.508932+0000","last_fullsized":"2023-12-19T17:01:22.508932+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:13:21.266430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509067+0000","last_change":"2023-12-19T17:01:20.759526+0000","last_active":"2023-12-19T17:01:22.509067+0000","last_peered":"2023-12-19T17:01:22.509067+0000","last_clean":"2023-12-19T17:01:22.509067+0000","last_became_active":"2023-12-19T17:01:20.758696+0000","last_became_peered":"2023-12-19T17:01:20.758696+0000","last_unstale":"2023-12-19T17:01:22.509067+0000","last_undegraded":"2023-12-19T17:01:22.509067+0000","last_fullsized":"2023-12-19T17:01:22.509067+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:54:54.464876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509180+0000","last_change":"2023-12-19T17:01:21.171826+0000","last_active":"2023-12-19T17:01:22.509180+0000","last_peered":"2023-12-19T17:01:22.509180+0000","last_clean":"2023-12-19T17:01:22.509180+0000","last_became_active":"2023-12-19T17:01:21.171530+0000","last_became_peered":"2023-12-19T17:01:21.171530+0000","last_unstale":"2023-12-19T17:01:22.509180+0000","last_undegraded":"2023-12-19T17:01:22.509180+0000","last_fullsized":"2023-12-19T17:01:22.509180+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:33.879282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508678+0000","last_change":"2023-12-19T17:01:20.760149+0000","last_active":"2023-12-19T17:01:22.508678+0000","last_peered":"2023-12-19T17:01:22.508678+0000","last_clean":"2023-12-19T17:01:22.508678+0000","last_became_active":"2023-12-19T17:01:20.759859+0000","last_became_peered":"2023-12-19T17:01:20.759859+0000","last_unstale":"2023-12-19T17:01:22.508678+0000","last_undegraded":"2023-12-19T17:01:22.508678+0000","last_fullsized":"2023-12-19T17:01:22.508678+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:11.351551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.161928+0000","last_change":"2023-12-19T17:01:21.160734+0000","last_active":"2023-12-19T17:01:22.161928+0000","last_peered":"2023-12-19T17:01:22.161928+0000","last_clean":"2023-12-19T17:01:22.161928+0000","last_became_active":"2023-12-19T17:01:21.160403+0000","last_became_peered":"2023-12-19T17:01:21.160403+0000","last_unstale":"2023-12-19T17:01:22.161928+0000","last_undegraded":"2023-12-19T17:01:22.161928+0000","last_fullsized":"2023-12-19T17:01:22.161928+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:12:49.539867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163251+0000","last_change":"2023-12-19T17:01:20.762097+0000","last_active":"2023-12-19T17:01:22.163251+0000","last_peered":"2023-12-19T17:01:22.163251+0000","last_clean":"2023-12-19T17:01:22.163251+0000","last_became_active":"2023-12-19T17:01:20.761146+0000","last_became_peered":"2023-12-19T17:01:20.761146+0000","last_unstale":"2023-12-19T17:01:22.163251+0000","last_undegraded":"2023-12-19T17:01:22.163251+0000","last_fullsized":"2023-12-19T17:01:22.163251+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:48.285357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162102+0000","last_change":"2023-12-19T17:01:21.161038+0000","last_active":"2023-12-19T17:01:22.162102+0000","last_peered":"2023-12-19T17:01:22.162102+0000","last_clean":"2023-12-19T17:01:22.162102+0000","last_became_active":"2023-12-19T17:01:21.160277+0000","last_became_peered":"2023-12-19T17:01:21.160277+0000","last_unstale":"2023-12-19T17:01:22.162102+0000","last_undegraded":"2023-12-19T17:01:22.162102+0000","last_fullsized":"2023-12-19T17:01:22.162102+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:53:23.892183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162050+0000","last_change":"2023-12-19T17:01:20.756168+0000","last_active":"2023-12-19T17:01:22.162050+0000","last_peered":"2023-12-19T17:01:22.162050+0000","last_clean":"2023-12-19T17:01:22.162050+0000","last_became_active":"2023-12-19T17:01:20.755690+0000","last_became_peered":"2023-12-19T17:01:20.755690+0000","last_unstale":"2023-12-19T17:01:22.162050+0000","last_undegraded":"2023-12-19T17:01:22.162050+0000","last_fullsized":"2023-12-19T17:01:22.162050+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:12:34.670243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509232+0000","last_change":"2023-12-19T17:01:21.180434+0000","last_active":"2023-12-19T17:01:22.509232+0000","last_peered":"2023-12-19T17:01:22.509232+0000","last_clean":"2023-12-19T17:01:22.509232+0000","last_became_active":"2023-12-19T17:01:21.180030+0000","last_became_peered":"2023-12-19T17:01:21.180030+0000","last_unstale":"2023-12-19T17:01:22.509232+0000","last_undegraded":"2023-12-19T17:01:22.509232+0000","last_fullsized":"2023-12-19T17:01:22.509232+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:00.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481441+0000","last_change":"2023-12-19T17:01:20.758941+0000","last_active":"2023-12-19T17:01:22.481441+0000","last_peered":"2023-12-19T17:01:22.481441+0000","last_clean":"2023-12-19T17:01:22.481441+0000","last_became_active":"2023-12-19T17:01:20.758647+0000","last_became_peered":"2023-12-19T17:01:20.758647+0000","last_unstale":"2023-12-19T17:01:22.481441+0000","last_undegraded":"2023-12-19T17:01:22.481441+0000","last_fullsized":"2023-12-19T17:01:22.481441+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:04:24.307536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508775+0000","last_change":"2023-12-19T17:01:21.177636+0000","last_active":"2023-12-19T17:01:22.508775+0000","last_peered":"2023-12-19T17:01:22.508775+0000","last_clean":"2023-12-19T17:01:22.508775+0000","last_became_active":"2023-12-19T17:01:21.177301+0000","last_became_peered":"2023-12-19T17:01:21.177301+0000","last_unstale":"2023-12-19T17:01:22.508775+0000","last_undegraded":"2023-12-19T17:01:22.508775+0000","last_fullsized":"2023-12-19T17:01:22.508775+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:02.423935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508597+0000","last_change":"2023-12-19T17:01:20.765395+0000","last_active":"2023-12-19T17:01:22.508597+0000","last_peered":"2023-12-19T17:01:22.508597+0000","last_clean":"2023-12-19T17:01:22.508597+0000","last_became_active":"2023-12-19T17:01:20.764046+0000","last_became_peered":"2023-12-19T17:01:20.764046+0000","last_unstale":"2023-12-19T17:01:22.508597+0000","last_undegraded":"2023-12-19T17:01:22.508597+0000","last_fullsized":"2023-12-19T17:01:22.508597+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:42:34.362818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508305+0000","last_change":"2023-12-19T17:01:21.169058+0000","last_active":"2023-12-19T17:01:22.508305+0000","last_peered":"2023-12-19T17:01:22.508305+0000","last_clean":"2023-12-19T17:01:22.508305+0000","last_became_active":"2023-12-19T17:01:21.167387+0000","last_became_peered":"2023-12-19T17:01:21.167387+0000","last_unstale":"2023-12-19T17:01:22.508305+0000","last_undegraded":"2023-12-19T17:01:22.508305+0000","last_fullsized":"2023-12-19T17:01:22.508305+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:19:55.046386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163195+0000","last_change":"2023-12-19T17:01:20.767278+0000","last_active":"2023-12-19T17:01:22.163195+0000","last_peered":"2023-12-19T17:01:22.163195+0000","last_clean":"2023-12-19T17:01:22.163195+0000","last_became_active":"2023-12-19T17:01:20.767010+0000","last_became_peered":"2023-12-19T17:01:20.767010+0000","last_unstale":"2023-12-19T17:01:22.163195+0000","last_undegraded":"2023-12-19T17:01:22.163195+0000","last_fullsized":"2023-12-19T17:01:22.163195+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:06.730128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"125'247","reported_seq":730,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-12-19T17:01:21.169657+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T17:01:21.169657+0000","last_peered":"2023-12-19T17:01:21.169657+0000","last_clean":"2023-12-19T17:01:21.169657+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T17:01:21.169657+0000","last_undegraded":"2023-12-19T17:01:21.169657+0000","last_fullsized":"2023-12-19T17:01:21.169657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1250,"num_read_kb":2557,"num_write":395,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1250,"num_read_kb":2557,"num_write":395,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":247,"ondisk_log_size":247,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509668,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37384,"kb_used_data":2240,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2293760,"data_stored":1456575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67300000000000004,"15min":0.68600000000000005},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":0.98499999999999999,"5min":1.04,"15min":1.3460000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68899999999999995,"15min":0.70799999999999996},"min":{"1min":0.443,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.6930000000000001},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69199999999999995,"15min":0.70999999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.99199999999999999,"5min":1.0489999999999999,"15min":1.212},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.52000000000000002,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":1.008,"5min":1.016,"15min":1.74},"last":0.65000000000000002}]},{"osd":2,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.42199999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.9990000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.44600000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.395},"last":0.71999999999999997}]},{"osd":3,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.879,"5min":1.0209999999999999,"15min":2.0379999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.73099999999999998,"15min":0.74399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.38300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.266},"last":0.66900000000000004}]},{"osd":4,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.39100000000000001,"5min":0.308,"15min":0.28499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94399999999999995,"15min":1.841},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.38900000000000001},"max":{"1min":0.96299999999999997,"5min":1.0640000000000001,"15min":2.0710000000000002},"last":0.502}]},{"osd":5,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.39600000000000002,"5min":0.376,"15min":0.36899999999999999},"max":{"1min":0.94699999999999995,"5min":1.0700000000000001,"15min":1.4339999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63600000000000001,"15min":0.64300000000000002},"min":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84099999999999997,"5min":1.0760000000000001,"15min":1.784},"last":0.52800000000000002}]},{"osd":6,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.503,"5min":0.45200000000000001,"15min":0.39800000000000002},"max":{"1min":0.83399999999999996,"5min":1.0109999999999999,"15min":1.8779999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.66300000000000003,"15min":0.66200000000000003},"min":{"1min":0.46899999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93300000000000005,"5min":1.0940000000000001,"15min":2.105},"last":0.63500000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509668,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1656,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1695744,"data_stored":865661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.42999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97199999999999998,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70299999999999996,"15min":0.70999999999999996},"min":{"1min":0.51100000000000001,"5min":0.44400000000000001,"15min":0.34999999999999998},"max":{"1min":0.88800000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.0289999999999999}]},{"osd":1,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.70499999999999996},"min":{"1min":0.42899999999999999,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.871,"5min":1.135,"15min":1.135},"last":0.435},{"interface":"front","average":{"1min":0.746,"5min":0.73799999999999999,"15min":0.73899999999999999},"min":{"1min":0.56699999999999995,"5min":0.42699999999999999,"15min":0.39500000000000002},"max":{"1min":0.91200000000000003,"5min":1.238,"15min":1.238},"last":0.876}]},{"osd":2,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73599999999999999,"15min":0.746},"min":{"1min":0.55700000000000005,"5min":0.47599999999999998,"15min":0.35199999999999998},"max":{"1min":1.0029999999999999,"5min":1.264,"15min":1.264},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.52700000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98599999999999999,"5min":1.0049999999999999,"15min":1.1879999999999999},"last":0.84199999999999997}]},{"osd":3,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73899999999999999,"15min":0.753},"min":{"1min":0.54500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95699999999999996,"5min":1.413,"15min":1.413},"last":0.996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68500000000000005,"15min":0.71499999999999997},"min":{"1min":0.47599999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93300000000000005,"5min":0.94199999999999995,"15min":1.0740000000000001},"last":0.54900000000000004}]},{"osd":4,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.42799999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83399999999999996,"5min":0.95399999999999996,"15min":0.96999999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.35699999999999998},"max":{"1min":0.81899999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57799999999999996,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.378,"15min":0.33300000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.96699999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.65700000000000003},"min":{"1min":0.501,"5min":0.42499999999999999,"15min":0.41499999999999998},"max":{"1min":0.84199999999999997,"5min":1.036,"15min":1.046},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.59099999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":1.006,"15min":1.0249999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69199999999999995,"15min":0.69499999999999995},"min":{"1min":0.53400000000000003,"5min":0.46600000000000003,"15min":0.371},"max":{"1min":0.87,"5min":0.91100000000000003,"15min":1.032},"last":0.49199999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509668,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1699840,"data_stored":866195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69099999999999995,"15min":0.70499999999999996},"min":{"1min":0.56699999999999995,"5min":0.34399999999999997,"15min":0.29499999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.626},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.60199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98399999999999999,"5min":0.99199999999999999,"15min":1.0189999999999999},"last":0.83099999999999996}]},{"osd":1,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.45000000000000001,"5min":0.39400000000000002,"15min":0.38600000000000001},"max":{"1min":0.91100000000000003,"5min":0.995,"15min":0.995},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.56599999999999995,"5min":0.40200000000000002,"15min":0.374},"max":{"1min":1.0309999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.64900000000000002}]},{"osd":2,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.73099999999999998,"15min":0.74199999999999999},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.996,"5min":1.026,"15min":1.0409999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71099999999999997,"15min":0.72299999999999998},"min":{"1min":0.58699999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0249999999999999,"5min":1.036,"15min":1.036},"last":0.70899999999999996}]},{"osd":3,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.752,"15min":0.73599999999999999},"min":{"1min":0.63600000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.012,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.65900000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.054,"5min":1.085,"15min":1.089},"last":0.73399999999999999}]},{"osd":4,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.35599999999999998,"15min":0.33300000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90400000000000003,"5min":0.92600000000000005,"15min":0.97199999999999998},"last":0.67300000000000004}]},{"osd":6,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.49099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88400000000000001,"5min":0.89500000000000002,"15min":0.98799999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.41399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.5,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.747},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67300000000000004,"15min":0.67400000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.42099999999999999},"max":{"1min":0.89300000000000002,"5min":1.1080000000000001,"15min":1.131},"last":0.80300000000000005}]}]},{"osd":4,"up_from":15,"seq":64424509668,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54844,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316996,"statfs":{"total":96636764160,"available":96580603904,"internally_reserved":0,"allocated":1691648,"data_stored":865591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.316},"max":{"1min":0.93100000000000005,"5min":1.0740000000000001,"15min":1.877},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69499999999999995,"15min":0.69999999999999996},"min":{"1min":0.41399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0680000000000001},"last":0.85799999999999998}]},{"osd":1,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69499999999999995,"15min":0.70999999999999996},"min":{"1min":0.45400000000000001,"5min":0.39000000000000001,"15min":0.34799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0209999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.56699999999999995,"5min":0.44600000000000001,"15min":0.35599999999999998},"max":{"1min":0.91600000000000004,"5min":0.98999999999999999,"15min":2.02},"last":0.89600000000000002}]},{"osd":2,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.5,"5min":0.46800000000000003,"15min":0.38700000000000001},"max":{"1min":0.94499999999999995,"5min":0.97199999999999998,"15min":1.375},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.748},"min":{"1min":0.48099999999999998,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.927},"last":1.046}]},{"osd":3,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.49399999999999999,"5min":0.437,"15min":0.41199999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.2030000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.58299999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98199999999999998,"5min":0.98299999999999998,"15min":1.0049999999999999},"last":0.93999999999999995}]},{"osd":5,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.44900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.004,"5min":1.004,"15min":1.99},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.41999999999999998,"5min":0.40799999999999997,"15min":0.33500000000000002},"max":{"1min":0.92600000000000005,"5min":1.002,"15min":1.002},"last":0.55400000000000005}]},{"osd":6,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.41399999999999998,"15min":0.40300000000000002},"max":{"1min":0.90600000000000003,"5min":0.96299999999999997,"15min":0.97199999999999998},"last":0.502},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67500000000000004,"15min":0.66500000000000004},"min":{"1min":0.52000000000000002,"5min":0.44,"15min":0.376},"max":{"1min":0.97099999999999997,"5min":1.042,"15min":1.96},"last":0.58399999999999996}]},{"osd":7,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.434,"5min":0.42299999999999999,"15min":0.376},"max":{"1min":0.871,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70699999999999996,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95399999999999996,"5min":1.0189999999999999,"15min":1.103},"last":0.66700000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509668,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40908,"kb_used_data":1668,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330932,"statfs":{"total":96636764160,"available":96594874368,"internally_reserved":0,"allocated":1708032,"data_stored":867225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.443,"15min":0.45000000000000001},"min":{"1min":0.26900000000000002,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":0.58399999999999996,"5min":0.69199999999999995,"15min":1.2509999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.27100000000000002},"max":{"1min":0.70299999999999996,"5min":1.0620000000000001,"15min":1.163},"last":0.65000000000000002}]},{"osd":1,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.36699999999999999,"5min":0.29899999999999999,"15min":0.25900000000000001},"max":{"1min":0.67800000000000005,"5min":0.76300000000000001,"15min":0.77800000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.29699999999999999},"max":{"1min":0.69999999999999996,"5min":0.75900000000000001,"15min":1.952},"last":0.60899999999999999}]},{"osd":2,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54200000000000004,"15min":0.54700000000000004},"min":{"1min":0.44,"5min":0.371,"15min":0.30099999999999999},"max":{"1min":0.76400000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.36699999999999999},"max":{"1min":0.752,"5min":1.157,"15min":2.855},"last":0.63}]},{"osd":4,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.745,"15min":0.77800000000000002},"min":{"1min":0.58099999999999996,"5min":0.50600000000000001,"15min":0.432},"max":{"1min":0.98299999999999998,"5min":1.081,"15min":4.234},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.75900000000000001,"15min":0.77600000000000002},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.46100000000000002},"max":{"1min":1.0349999999999999,"5min":1.0649999999999999,"15min":1.3069999999999999},"last":1.369}]},{"osd":5,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.57399999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.266},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.751,"15min":0.77600000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.43099999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.8300000000000001},"last":1.121}]},{"osd":6,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76100000000000001,"15min":0.78800000000000003},"min":{"1min":0.54400000000000004,"5min":0.51200000000000001,"15min":0.45800000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.599},"last":1.218},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76900000000000002,"15min":0.79200000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.49199999999999999},"max":{"1min":1.0049999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.409}]},{"osd":7,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.60099999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0920000000000001,"5min":1.385,"15min":1.385},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.81100000000000005,"15min":0.84499999999999997},"min":{"1min":0.67500000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.204,"5min":1.204,"15min":1.623},"last":1.4239999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509667,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58928,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312912,"statfs":{"total":96636764160,"available":96576421888,"internally_reserved":0,"allocated":1679360,"data_stored":858559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.433,"15min":0.45800000000000002},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63900000000000001,"5min":0.66500000000000004,"15min":5.4870000000000001},"last":0.433},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47999999999999998,"15min":0.52800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.72499999999999998,"5min":0.76500000000000001,"15min":10.161},"last":0.503}]},{"osd":1,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57499999999999996,"15min":0.60499999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76500000000000001,"5min":0.82799999999999996,"15min":2.4740000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.504,"15min":0.51900000000000002},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":0.71499999999999997,"15min":2.911},"last":0.59999999999999998}]},{"osd":3,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56699999999999995,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.71599999999999997,"5min":0.77300000000000002,"15min":1.1990000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54600000000000004,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69599999999999995,"5min":0.81499999999999995,"15min":2.7999999999999998},"last":0.71299999999999997}]},{"osd":4,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.75600000000000001,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0369999999999999,"5min":1.3,"15min":2.4449999999999998},"last":0.877},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.78400000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.42999999999999999},"max":{"1min":0.97299999999999998,"5min":1.3049999999999999,"15min":5.5430000000000001},"last":0.76900000000000002}]},{"osd":5,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.79700000000000004},"min":{"1min":0.52200000000000002,"5min":0.5,"15min":0.49299999999999999},"max":{"1min":1.0600000000000001,"5min":1.4890000000000001,"15min":5.601},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76200000000000001,"15min":0.80400000000000005},"min":{"1min":0.54600000000000004,"5min":0.504,"15min":0.504},"max":{"1min":0.99099999999999999,"5min":1.268,"15min":2.569},"last":0.92800000000000005}]},{"osd":6,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79700000000000004,"15min":0.81899999999999995},"min":{"1min":0.58799999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0369999999999999,"5min":1.4319999999999999,"15min":2.3889999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.78100000000000003},"min":{"1min":0.53100000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98399999999999999,"5min":1.1950000000000001,"15min":1.4379999999999999},"last":0.81100000000000005}]},{"osd":7,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.77800000000000002,"15min":0.81499999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.01,"5min":1.3340000000000001,"15min":5.673},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80200000000000005,"15min":0.82699999999999996},"min":{"1min":0.59999999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.024,"5min":1.369,"15min":1.429},"last":0.78700000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509668,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50756,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321084,"statfs":{"total":96636764160,"available":96584790016,"internally_reserved":0,"allocated":1699840,"data_stored":865715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.439,"15min":0.44500000000000001},"min":{"1min":0.30099999999999999,"5min":0.26900000000000002,"15min":0.22500000000000001},"max":{"1min":0.60499999999999998,"5min":0.82799999999999996,"15min":0.84899999999999998},"last":0.313},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.496,"15min":0.51800000000000002},"min":{"1min":0.36399999999999999,"5min":0.29699999999999999,"15min":0.26400000000000001},"max":{"1min":0.92100000000000004,"5min":1.071,"15min":1.071},"last":0.45000000000000001}]},{"osd":2,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.30299999999999999},"max":{"1min":0.65000000000000002,"5min":0.67000000000000004,"15min":0.80400000000000005},"last":0.435},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52700000000000002,"15min":0.54800000000000004},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.317},"max":{"1min":0.80600000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49099999999999999}]},{"osd":3,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58099999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.504,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.34499999999999997,"15min":0.33700000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.91900000000000004},"last":0.36799999999999999}]},{"osd":4,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.74099999999999999,"15min":0.75700000000000001},"min":{"1min":0.46300000000000002,"5min":0.45300000000000001,"15min":0.43099999999999999},"max":{"1min":1.008,"5min":1.107,"15min":1.321},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.746,"15min":0.77600000000000002},"min":{"1min":0.56200000000000006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98499999999999999,"5min":1.0580000000000001,"15min":1.4179999999999999},"last":0.56299999999999994}]},{"osd":5,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.747,"15min":0.77400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.46500000000000002},"max":{"1min":0.97299999999999998,"5min":1.2010000000000001,"15min":1.4450000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.59599999999999997,"5min":0.51200000000000001,"15min":0.46500000000000002},"max":{"1min":1.0489999999999999,"5min":1.0700000000000001,"15min":1.3380000000000001},"last":0.68100000000000005}]},{"osd":6,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.74399999999999999,"15min":0.78000000000000003},"min":{"1min":0.51500000000000001,"5min":0.496,"15min":0.41999999999999998},"max":{"1min":0.95599999999999996,"5min":1.048,"15min":1.3939999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77200000000000002,"15min":0.79800000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.52700000000000002},"max":{"1min":1.032,"5min":1.1659999999999999,"15min":1.391},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79700000000000004,"15min":0.82399999999999995},"min":{"1min":0.58199999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.008,"5min":1.175,"15min":1.2589999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.79200000000000004,"15min":0.81999999999999995},"min":{"1min":0.55300000000000005,"5min":0.55000000000000004,"15min":0.53800000000000003},"max":{"1min":1.0720000000000001,"5min":1.2050000000000001,"15min":1.4219999999999999},"last":0.65300000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542373,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2252,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2306048,"data_stored":1457635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.443,"15min":0.434},"min":{"1min":0.29999999999999999,"5min":0.26000000000000001,"15min":0.246},"max":{"1min":0.67900000000000005,"5min":0.752,"15min":0.752},"last":0.502},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.498,"15min":0.49299999999999999},"min":{"1min":0.32800000000000001,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.69299999999999995,"5min":0.81699999999999995,"15min":0.85499999999999998},"last":0.38600000000000001}]},{"osd":2,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.40200000000000002,"5min":0.33300000000000002,"15min":0.31900000000000001},"max":{"1min":0.67000000000000004,"5min":0.71499999999999997,"15min":0.77800000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73999999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56100000000000005}]},{"osd":3,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.34300000000000003,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.74199999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.623,"15min":0.621},"min":{"1min":0.42999999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81000000000000005,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.61099999999999999}]},{"osd":4,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.79100000000000004,"15min":0.79200000000000004},"min":{"1min":0.53500000000000003,"5min":0.503,"15min":0.45900000000000002},"max":{"1min":1.145,"5min":1.2090000000000001,"15min":1.444},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79200000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.42599999999999999},"max":{"1min":1.0620000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.59199999999999997}]},{"osd":5,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.51400000000000001,"5min":0.502,"15min":0.41499999999999998},"max":{"1min":1.016,"5min":1.1599999999999999,"15min":1.294},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81299999999999994,"15min":0.80400000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.113,"5min":1.2549999999999999,"15min":1.256},"last":0.76500000000000001}]},{"osd":6,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81100000000000005,"15min":0.80700000000000005},"min":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.093,"5min":1.2789999999999999,"15min":1.429},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.81699999999999995},"min":{"1min":0.63500000000000001,"5min":0.53100000000000003,"15min":0.44700000000000001},"max":{"1min":1.046,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.94699999999999995}]},{"osd":7,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82999999999999996,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.48999999999999999},"max":{"1min":1.1870000000000001,"5min":1.3129999999999999,"15min":1.377},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.84099999999999997,"15min":0.82799999999999996},"min":{"1min":0.58399999999999996,"5min":0.51400000000000001,"15min":0.48099999999999998},"max":{"1min":1.0880000000000001,"5min":1.2649999999999999,"15min":1.3300000000000001},"last":0.67000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T17:01:25.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T17:01:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.457+0000 7f8f75a66640 1 Processor -- start 2023-12-19T17:01:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.457+0000 7f8f75a66640 1 -- start start 2023-12-19T17:01:25.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.457+0000 7f8f75a66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8f7015a7c0 0x7f8f7015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.457+0000 7f8f75a66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7015e2f0 con 0x7f8f700cf700 2023-12-19T17:01:25.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.457+0000 7f8f75a66640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f700c5b40 con 0x7f8f7006f9d0 2023-12-19T17:01:25.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.457+0000 7f8f75a66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f700c5e20 con 0x7f8f7015a7c0 2023-12-19T17:01:25.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.457+0000 7f8f6f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8f7015a7c0 0x7f8f7015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.457+0000 7f8f6ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8f700cf700 0x7f8f700cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33386/0 (socket says 172.21.15.26:33386) 2023-12-19T17:01:25.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.457+0000 7f8f6ffff640 1 -- 172.21.15.26:0/454148923 learned_addr learned my addr 172.21.15.26:0/454148923 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:25.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.457+0000 7f8f6f7fe640 1 -- 172.21.15.26:0/454148923 >> v1:172.21.15.160:6789/0 conn(0x7f8f7006f9d0 legacy=0x7f8f700ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:25.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.457+0000 7f8f6f7fe640 1 -- 172.21.15.26:0/454148923 >> v1:172.21.15.26:6789/0 conn(0x7f8f700cf700 legacy=0x7f8f700cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.457+0000 7f8f6f7fe640 1 -- 172.21.15.26:0/454148923 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f700ce770 con 0x7f8f7015a7c0 2023-12-19T17:01:25.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.461+0000 7f8f6f7fe640 1 --2- 172.21.15.26:0/454148923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f7015a7c0 0x7f8f7015ddb0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f8f600097a0 tx=0x7f8f6001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760e313e634ef8e8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:25.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.461+0000 7f8f6e7fc640 1 -- 172.21.15.26:0/454148923 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f600ab5f0 con 0x7f8f7015a7c0 2023-12-19T17:01:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.461+0000 7f8f6e7fc640 1 -- 172.21.15.26:0/454148923 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f600ab750 con 0x7f8f7015a7c0 2023-12-19T17:01:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.461+0000 7f8f6e7fc640 1 -- 172.21.15.26:0/454148923 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f600abae0 con 0x7f8f7015a7c0 2023-12-19T17:01:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.461+0000 7f8f75a66640 1 -- 172.21.15.26:0/454148923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f7015a7c0 msgr2=0x7f8f7015ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.461+0000 7f8f75a66640 1 --2- 172.21.15.26:0/454148923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f7015a7c0 0x7f8f7015ddb0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f8f600097a0 tx=0x7f8f6001c3f0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.461+0000 7f8f75a66640 1 -- 172.21.15.26:0/454148923 shutdown_connections 2023-12-19T17:01:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.461+0000 7f8f75a66640 1 --2- 172.21.15.26:0/454148923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f7015a7c0 0x7f8f7015ddb0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.461+0000 7f8f75a66640 1 -- 172.21.15.26:0/454148923 >> 172.21.15.26:0/454148923 conn(0x7f8f7006d5d0 msgr2=0x7f8f70069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.461+0000 7f8f75a66640 1 -- 172.21.15.26:0/454148923 shutdown_connections 2023-12-19T17:01:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.461+0000 7f8f75a66640 1 -- 172.21.15.26:0/454148923 wait complete. 2023-12-19T17:01:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f75a66640 1 Processor -- start 2023-12-19T17:01:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f75a66640 1 -- start start 2023-12-19T17:01:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f75a66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f7006f9d0 0x7f8f70155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f75a66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f700cf700 0x7f8f70152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f75a66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f7015a7c0 0x7f8f7014ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f75a66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f701564f0 con 0x7f8f700cf700 2023-12-19T17:01:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f75a66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f700c95e0 con 0x7f8f7006f9d0 2023-12-19T17:01:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f75a66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f700c97f0 con 0x7f8f7015a7c0 2023-12-19T17:01:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f6effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f700cf700 0x7f8f70152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f6effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f700cf700 0x7f8f70152b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40988/0 (socket says 172.21.15.26:40988) 2023-12-19T17:01:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f6effd640 1 -- 172.21.15.26:0/1095035683 learned_addr learned my addr 172.21.15.26:0/1095035683 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f6f7fe640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f7006f9d0 0x7f8f70155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f6ffff640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f7015a7c0 0x7f8f7014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f6effd640 1 -- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f7015a7c0 msgr2=0x7f8f7014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f6effd640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f7015a7c0 0x7f8f7014ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f6effd640 1 -- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f7006f9d0 msgr2=0x7f8f70155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f6effd640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f7006f9d0 0x7f8f70155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f6effd640 1 -- 172.21.15.26:0/1095035683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f600093b0 con 0x7f8f700cf700 2023-12-19T17:01:25.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f6effd640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f700cf700 0x7f8f70152b10 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400c450 tx=0x7f8f6400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:25.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f57fff640 1 -- 172.21.15.26:0/1095035683 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64012830 con 0x7f8f700cf700 2023-12-19T17:01:25.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f57fff640 1 -- 172.21.15.26:0/1095035683 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f640129b0 con 0x7f8f700cf700 2023-12-19T17:01:25.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f75a66640 1 -- 172.21.15.26:0/1095035683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f700c9ad0 con 0x7f8f700cf700 2023-12-19T17:01:25.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f57fff640 1 -- 172.21.15.26:0/1095035683 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64012d60 con 0x7f8f700cf700 2023-12-19T17:01:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.465+0000 7f8f75a66640 1 -- 172.21.15.26:0/1095035683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f70156110 con 0x7f8f700cf700 2023-12-19T17:01:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.469+0000 7f8f75a66640 1 -- 172.21.15.26:0/1095035683 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f70156630 con 0x7f8f700cf700 2023-12-19T17:01:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.469+0000 7f8f57fff640 1 -- 172.21.15.26:0/1095035683 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8f6401c460 con 0x7f8f700cf700 2023-12-19T17:01:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.469+0000 7f8f57fff640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8f5c065b80 0x7f8f5c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.469+0000 7f8f57fff640 1 -- 172.21.15.26:0/1095035683 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f8f64083470 con 0x7f8f700cf700 2023-12-19T17:01:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.469+0000 7f8f6f7fe640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8f5c065b80 0x7f8f5c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.469+0000 7f8f6f7fe640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8f5c065b80 0x7f8f5c068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f600af000 tx=0x7f8f600a7000 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:25.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.473+0000 7f8f57fff640 1 -- 172.21.15.26:0/1095035683 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8f70156630 con 0x7f8f700cf700 2023-12-19T17:01:25.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.625+0000 7f8f75a66640 1 -- 172.21.15.26:0/1095035683 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f700c57f0 con 0x7f8f5c065b80 2023-12-19T17:01:25.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.637+0000 7f8f57fff640 1 -- 172.21.15.26:0/1095035683 <== mgr.7082 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289360 (secure 0 0 0) 0x7f8f600ab8c0 con 0x7f8f5c065b80 2023-12-19T17:01:25.642 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:25.644 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T17:01:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.637+0000 7f8f75a66640 1 -- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8f5c065b80 msgr2=0x7f8f5c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.637+0000 7f8f75a66640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8f5c065b80 0x7f8f5c068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f600af000 tx=0x7f8f600a7000 comp rx=0 tx=0).stop 2023-12-19T17:01:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.637+0000 7f8f75a66640 1 -- 172.21.15.26:0/1095035683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f700cf700 msgr2=0x7f8f70152b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.637+0000 7f8f75a66640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f700cf700 0x7f8f70152b10 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f8f6400c450 tx=0x7f8f6400c910 comp rx=0 tx=0).stop 2023-12-19T17:01:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.637+0000 7f8f75a66640 1 -- 172.21.15.26:0/1095035683 shutdown_connections 2023-12-19T17:01:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.637+0000 7f8f75a66640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8f5c065b80 0x7f8f5c068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.637+0000 7f8f75a66640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f7015a7c0 0x7f8f7014ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.637+0000 7f8f75a66640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f700cf700 0x7f8f70152b10 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.637+0000 7f8f75a66640 1 --2- 172.21.15.26:0/1095035683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f7006f9d0 0x7f8f70155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.637+0000 7f8f75a66640 1 -- 172.21.15.26:0/1095035683 >> 172.21.15.26:0/1095035683 conn(0x7f8f7006d5d0 msgr2=0x7f8f700b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.637+0000 7f8f75a66640 1 -- 172.21.15.26:0/1095035683 shutdown_connections 2023-12-19T17:01:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.637+0000 7f8f75a66640 1 -- 172.21.15.26:0/1095035683 wait complete. 2023-12-19T17:01:25.664 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-19T17:01:25.468339+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1250,"num_read_kb":2557,"num_write":439,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":285,"ondisk_log_size":285,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":112,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":345628,"kb_used_data":14428,"kb_used_omap":0,"kb_used_meta":331136,"kb_avail":754629092,"statfs":{"total":773094113280,"available":772740190208,"internally_reserved":0,"allocated":14774272,"data_stored":8103156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":339083264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.731135"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509211+0000","last_change":"2023-12-19T17:01:21.167279+0000","last_active":"2023-12-19T17:01:22.509211+0000","last_peered":"2023-12-19T17:01:22.509211+0000","last_clean":"2023-12-19T17:01:22.509211+0000","last_became_active":"2023-12-19T17:01:21.167023+0000","last_became_peered":"2023-12-19T17:01:21.167023+0000","last_unstale":"2023-12-19T17:01:22.509211+0000","last_undegraded":"2023-12-19T17:01:22.509211+0000","last_fullsized":"2023-12-19T17:01:22.509211+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:09:33.899825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162479+0000","last_change":"2023-12-19T17:01:21.162772+0000","last_active":"2023-12-19T17:01:22.162479+0000","last_peered":"2023-12-19T17:01:22.162479+0000","last_clean":"2023-12-19T17:01:22.162479+0000","last_became_active":"2023-12-19T17:01:21.161491+0000","last_became_peered":"2023-12-19T17:01:21.161491+0000","last_unstale":"2023-12-19T17:01:22.162479+0000","last_undegraded":"2023-12-19T17:01:22.162479+0000","last_fullsized":"2023-12-19T17:01:22.162479+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:16:02.193853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508927+0000","last_change":"2023-12-19T17:01:21.178569+0000","last_active":"2023-12-19T17:01:22.508927+0000","last_peered":"2023-12-19T17:01:22.508927+0000","last_clean":"2023-12-19T17:01:22.508927+0000","last_became_active":"2023-12-19T17:01:21.178190+0000","last_became_peered":"2023-12-19T17:01:21.178190+0000","last_unstale":"2023-12-19T17:01:22.508927+0000","last_undegraded":"2023-12-19T17:01:22.508927+0000","last_fullsized":"2023-12-19T17:01:22.508927+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:26:19.065866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508750+0000","last_change":"2023-12-19T17:01:21.168709+0000","last_active":"2023-12-19T17:01:22.508750+0000","last_peered":"2023-12-19T17:01:22.508750+0000","last_clean":"2023-12-19T17:01:22.508750+0000","last_became_active":"2023-12-19T17:01:21.168181+0000","last_became_peered":"2023-12-19T17:01:21.168181+0000","last_unstale":"2023-12-19T17:01:22.508750+0000","last_undegraded":"2023-12-19T17:01:22.508750+0000","last_fullsized":"2023-12-19T17:01:22.508750+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:28:19.270066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481642+0000","last_change":"2023-12-19T17:01:21.173151+0000","last_active":"2023-12-19T17:01:22.481642+0000","last_peered":"2023-12-19T17:01:22.481642+0000","last_clean":"2023-12-19T17:01:22.481642+0000","last_became_active":"2023-12-19T17:01:21.172842+0000","last_became_peered":"2023-12-19T17:01:21.172842+0000","last_unstale":"2023-12-19T17:01:22.481642+0000","last_undegraded":"2023-12-19T17:01:22.481642+0000","last_fullsized":"2023-12-19T17:01:22.481642+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:40.001134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158109+0000","last_change":"2023-12-19T17:01:21.170905+0000","last_active":"2023-12-19T17:01:22.158109+0000","last_peered":"2023-12-19T17:01:22.158109+0000","last_clean":"2023-12-19T17:01:22.158109+0000","last_became_active":"2023-12-19T17:01:21.170639+0000","last_became_peered":"2023-12-19T17:01:21.170639+0000","last_unstale":"2023-12-19T17:01:22.158109+0000","last_undegraded":"2023-12-19T17:01:22.158109+0000","last_fullsized":"2023-12-19T17:01:22.158109+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:33.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163133+0000","last_change":"2023-12-19T17:01:21.164812+0000","last_active":"2023-12-19T17:01:22.163133+0000","last_peered":"2023-12-19T17:01:22.163133+0000","last_clean":"2023-12-19T17:01:22.163133+0000","last_became_active":"2023-12-19T17:01:21.164526+0000","last_became_peered":"2023-12-19T17:01:21.164526+0000","last_unstale":"2023-12-19T17:01:22.163133+0000","last_undegraded":"2023-12-19T17:01:22.163133+0000","last_fullsized":"2023-12-19T17:01:22.163133+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:12.213357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508959+0000","last_change":"2023-12-19T17:01:21.185785+0000","last_active":"2023-12-19T17:01:22.508959+0000","last_peered":"2023-12-19T17:01:22.508959+0000","last_clean":"2023-12-19T17:01:22.508959+0000","last_became_active":"2023-12-19T17:01:21.185526+0000","last_became_peered":"2023-12-19T17:01:21.185526+0000","last_unstale":"2023-12-19T17:01:22.508959+0000","last_undegraded":"2023-12-19T17:01:22.508959+0000","last_fullsized":"2023-12-19T17:01:22.508959+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:59:57.417117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509265+0000","last_change":"2023-12-19T17:01:21.170137+0000","last_active":"2023-12-19T17:01:22.509265+0000","last_peered":"2023-12-19T17:01:22.509265+0000","last_clean":"2023-12-19T17:01:22.509265+0000","last_became_active":"2023-12-19T17:01:21.169830+0000","last_became_peered":"2023-12-19T17:01:21.169830+0000","last_unstale":"2023-12-19T17:01:22.509265+0000","last_undegraded":"2023-12-19T17:01:22.509265+0000","last_fullsized":"2023-12-19T17:01:22.509265+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:38.425276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509107+0000","last_change":"2023-12-19T17:01:21.168575+0000","last_active":"2023-12-19T17:01:22.509107+0000","last_peered":"2023-12-19T17:01:22.509107+0000","last_clean":"2023-12-19T17:01:22.509107+0000","last_became_active":"2023-12-19T17:01:21.168298+0000","last_became_peered":"2023-12-19T17:01:21.168298+0000","last_unstale":"2023-12-19T17:01:22.509107+0000","last_undegraded":"2023-12-19T17:01:22.509107+0000","last_fullsized":"2023-12-19T17:01:22.509107+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:56:49.237027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162404+0000","last_change":"2023-12-19T17:01:21.167363+0000","last_active":"2023-12-19T17:01:22.162404+0000","last_peered":"2023-12-19T17:01:22.162404+0000","last_clean":"2023-12-19T17:01:22.162404+0000","last_became_active":"2023-12-19T17:01:21.167061+0000","last_became_peered":"2023-12-19T17:01:21.167061+0000","last_unstale":"2023-12-19T17:01:22.162404+0000","last_undegraded":"2023-12-19T17:01:22.162404+0000","last_fullsized":"2023-12-19T17:01:22.162404+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:02:02.542798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508844+0000","last_change":"2023-12-19T17:01:21.168531+0000","last_active":"2023-12-19T17:01:22.508844+0000","last_peered":"2023-12-19T17:01:22.508844+0000","last_clean":"2023-12-19T17:01:22.508844+0000","last_became_active":"2023-12-19T17:01:21.167885+0000","last_became_peered":"2023-12-19T17:01:21.167885+0000","last_unstale":"2023-12-19T17:01:22.508844+0000","last_undegraded":"2023-12-19T17:01:22.508844+0000","last_fullsized":"2023-12-19T17:01:22.508844+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:10:26.212444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509166+0000","last_change":"2023-12-19T17:01:21.182858+0000","last_active":"2023-12-19T17:01:22.509166+0000","last_peered":"2023-12-19T17:01:22.509166+0000","last_clean":"2023-12-19T17:01:22.509166+0000","last_became_active":"2023-12-19T17:01:21.182040+0000","last_became_peered":"2023-12-19T17:01:21.182040+0000","last_unstale":"2023-12-19T17:01:22.509166+0000","last_undegraded":"2023-12-19T17:01:22.509166+0000","last_fullsized":"2023-12-19T17:01:22.509166+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:51.742460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508675+0000","last_change":"2023-12-19T17:01:21.181107+0000","last_active":"2023-12-19T17:01:22.508675+0000","last_peered":"2023-12-19T17:01:22.508675+0000","last_clean":"2023-12-19T17:01:22.508675+0000","last_became_active":"2023-12-19T17:01:21.180021+0000","last_became_peered":"2023-12-19T17:01:21.180021+0000","last_unstale":"2023-12-19T17:01:22.508675+0000","last_undegraded":"2023-12-19T17:01:22.508675+0000","last_fullsized":"2023-12-19T17:01:22.508675+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:28.720065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.157928+0000","last_change":"2023-12-19T17:01:21.169449+0000","last_active":"2023-12-19T17:01:22.157928+0000","last_peered":"2023-12-19T17:01:22.157928+0000","last_clean":"2023-12-19T17:01:22.157928+0000","last_became_active":"2023-12-19T17:01:21.169207+0000","last_became_peered":"2023-12-19T17:01:21.169207+0000","last_unstale":"2023-12-19T17:01:22.157928+0000","last_undegraded":"2023-12-19T17:01:22.157928+0000","last_fullsized":"2023-12-19T17:01:22.157928+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:40.106699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508407+0000","last_change":"2023-12-19T17:01:21.174549+0000","last_active":"2023-12-19T17:01:22.508407+0000","last_peered":"2023-12-19T17:01:22.508407+0000","last_clean":"2023-12-19T17:01:22.508407+0000","last_became_active":"2023-12-19T17:01:21.174275+0000","last_became_peered":"2023-12-19T17:01:21.174275+0000","last_unstale":"2023-12-19T17:01:22.508407+0000","last_undegraded":"2023-12-19T17:01:22.508407+0000","last_fullsized":"2023-12-19T17:01:22.508407+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:56.423474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162421+0000","last_change":"2023-12-19T17:01:21.166560+0000","last_active":"2023-12-19T17:01:22.162421+0000","last_peered":"2023-12-19T17:01:22.162421+0000","last_clean":"2023-12-19T17:01:22.162421+0000","last_became_active":"2023-12-19T17:01:21.166300+0000","last_became_peered":"2023-12-19T17:01:21.166300+0000","last_unstale":"2023-12-19T17:01:22.162421+0000","last_undegraded":"2023-12-19T17:01:22.162421+0000","last_fullsized":"2023-12-19T17:01:22.162421+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:33.879282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509411+0000","last_change":"2023-12-19T17:01:21.165340+0000","last_active":"2023-12-19T17:01:22.509411+0000","last_peered":"2023-12-19T17:01:22.509411+0000","last_clean":"2023-12-19T17:01:22.509411+0000","last_became_active":"2023-12-19T17:01:21.165059+0000","last_became_peered":"2023-12-19T17:01:21.165059+0000","last_unstale":"2023-12-19T17:01:22.509411+0000","last_undegraded":"2023-12-19T17:01:22.509411+0000","last_fullsized":"2023-12-19T17:01:22.509411+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:36:28.680559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508935+0000","last_change":"2023-12-19T17:01:21.180898+0000","last_active":"2023-12-19T17:01:22.508935+0000","last_peered":"2023-12-19T17:01:22.508935+0000","last_clean":"2023-12-19T17:01:22.508935+0000","last_became_active":"2023-12-19T17:01:21.180585+0000","last_became_peered":"2023-12-19T17:01:21.180585+0000","last_unstale":"2023-12-19T17:01:22.508935+0000","last_undegraded":"2023-12-19T17:01:22.508935+0000","last_fullsized":"2023-12-19T17:01:22.508935+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:12:24.811047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162270+0000","last_change":"2023-12-19T17:01:21.174401+0000","last_active":"2023-12-19T17:01:22.162270+0000","last_peered":"2023-12-19T17:01:22.162270+0000","last_clean":"2023-12-19T17:01:22.162270+0000","last_became_active":"2023-12-19T17:01:21.173141+0000","last_became_peered":"2023-12-19T17:01:21.173141+0000","last_unstale":"2023-12-19T17:01:22.162270+0000","last_undegraded":"2023-12-19T17:01:22.162270+0000","last_fullsized":"2023-12-19T17:01:22.162270+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:11:26.427051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162273+0000","last_change":"2023-12-19T17:01:21.171120+0000","last_active":"2023-12-19T17:01:22.162273+0000","last_peered":"2023-12-19T17:01:22.162273+0000","last_clean":"2023-12-19T17:01:22.162273+0000","last_became_active":"2023-12-19T17:01:21.170859+0000","last_became_peered":"2023-12-19T17:01:21.170859+0000","last_unstale":"2023-12-19T17:01:22.162273+0000","last_undegraded":"2023-12-19T17:01:22.162273+0000","last_fullsized":"2023-12-19T17:01:22.162273+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:54:08.905754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508427+0000","last_change":"2023-12-19T17:01:21.166512+0000","last_active":"2023-12-19T17:01:22.508427+0000","last_peered":"2023-12-19T17:01:22.508427+0000","last_clean":"2023-12-19T17:01:22.508427+0000","last_became_active":"2023-12-19T17:01:21.166233+0000","last_became_peered":"2023-12-19T17:01:21.166233+0000","last_unstale":"2023-12-19T17:01:22.508427+0000","last_undegraded":"2023-12-19T17:01:22.508427+0000","last_fullsized":"2023-12-19T17:01:22.508427+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:30:00.737607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.161853+0000","last_change":"2023-12-19T17:01:21.170529+0000","last_active":"2023-12-19T17:01:22.161853+0000","last_peered":"2023-12-19T17:01:22.161853+0000","last_clean":"2023-12-19T17:01:22.161853+0000","last_became_active":"2023-12-19T17:01:21.169678+0000","last_became_peered":"2023-12-19T17:01:21.169678+0000","last_unstale":"2023-12-19T17:01:22.161853+0000","last_undegraded":"2023-12-19T17:01:22.161853+0000","last_fullsized":"2023-12-19T17:01:22.161853+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:15.142197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508191+0000","last_change":"2023-12-19T17:01:21.178360+0000","last_active":"2023-12-19T17:01:22.508191+0000","last_peered":"2023-12-19T17:01:22.508191+0000","last_clean":"2023-12-19T17:01:22.508191+0000","last_became_active":"2023-12-19T17:01:21.178092+0000","last_became_peered":"2023-12-19T17:01:21.178092+0000","last_unstale":"2023-12-19T17:01:22.508191+0000","last_undegraded":"2023-12-19T17:01:22.508191+0000","last_fullsized":"2023-12-19T17:01:22.508191+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:43.259541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162475+0000","last_change":"2023-12-19T17:01:21.165927+0000","last_active":"2023-12-19T17:01:22.162475+0000","last_peered":"2023-12-19T17:01:22.162475+0000","last_clean":"2023-12-19T17:01:22.162475+0000","last_became_active":"2023-12-19T17:01:21.165619+0000","last_became_peered":"2023-12-19T17:01:21.165619+0000","last_unstale":"2023-12-19T17:01:22.162475+0000","last_undegraded":"2023-12-19T17:01:22.162475+0000","last_fullsized":"2023-12-19T17:01:22.162475+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:36:28.680559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509176+0000","last_change":"2023-12-19T17:01:21.165757+0000","last_active":"2023-12-19T17:01:22.509176+0000","last_peered":"2023-12-19T17:01:22.509176+0000","last_clean":"2023-12-19T17:01:22.509176+0000","last_became_active":"2023-12-19T17:01:21.165487+0000","last_became_peered":"2023-12-19T17:01:21.165487+0000","last_unstale":"2023-12-19T17:01:22.509176+0000","last_undegraded":"2023-12-19T17:01:22.509176+0000","last_fullsized":"2023-12-19T17:01:22.509176+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:00:31.002290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158665+0000","last_change":"2023-12-19T17:01:21.169961+0000","last_active":"2023-12-19T17:01:22.158665+0000","last_peered":"2023-12-19T17:01:22.158665+0000","last_clean":"2023-12-19T17:01:22.158665+0000","last_became_active":"2023-12-19T17:01:21.169693+0000","last_became_peered":"2023-12-19T17:01:21.169693+0000","last_unstale":"2023-12-19T17:01:22.158665+0000","last_undegraded":"2023-12-19T17:01:22.158665+0000","last_fullsized":"2023-12-19T17:01:22.158665+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:54:08.905754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508514+0000","last_change":"2023-12-19T17:01:21.176635+0000","last_active":"2023-12-19T17:01:22.508514+0000","last_peered":"2023-12-19T17:01:22.508514+0000","last_clean":"2023-12-19T17:01:22.508514+0000","last_became_active":"2023-12-19T17:01:21.176360+0000","last_became_peered":"2023-12-19T17:01:21.176360+0000","last_unstale":"2023-12-19T17:01:22.508514+0000","last_undegraded":"2023-12-19T17:01:22.508514+0000","last_fullsized":"2023-12-19T17:01:22.508514+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:42.611360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.161967+0000","last_change":"2023-12-19T17:01:21.176706+0000","last_active":"2023-12-19T17:01:22.161967+0000","last_peered":"2023-12-19T17:01:22.161967+0000","last_clean":"2023-12-19T17:01:22.161967+0000","last_became_active":"2023-12-19T17:01:21.176115+0000","last_became_peered":"2023-12-19T17:01:21.176115+0000","last_unstale":"2023-12-19T17:01:22.161967+0000","last_undegraded":"2023-12-19T17:01:22.161967+0000","last_fullsized":"2023-12-19T17:01:22.161967+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:53:33.990500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163762+0000","last_change":"2023-12-19T17:01:21.167889+0000","last_active":"2023-12-19T17:01:22.163762+0000","last_peered":"2023-12-19T17:01:22.163762+0000","last_clean":"2023-12-19T17:01:22.163762+0000","last_became_active":"2023-12-19T17:01:21.167605+0000","last_became_peered":"2023-12-19T17:01:21.167605+0000","last_unstale":"2023-12-19T17:01:22.163762+0000","last_undegraded":"2023-12-19T17:01:22.163762+0000","last_fullsized":"2023-12-19T17:01:22.163762+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:47:26.481010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508675+0000","last_change":"2023-12-19T17:01:21.167703+0000","last_active":"2023-12-19T17:01:22.508675+0000","last_peered":"2023-12-19T17:01:22.508675+0000","last_clean":"2023-12-19T17:01:22.508675+0000","last_became_active":"2023-12-19T17:01:21.167445+0000","last_became_peered":"2023-12-19T17:01:21.167445+0000","last_unstale":"2023-12-19T17:01:22.508675+0000","last_undegraded":"2023-12-19T17:01:22.508675+0000","last_fullsized":"2023-12-19T17:01:22.508675+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:17:50.362533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.157928+0000","last_change":"2023-12-19T17:01:21.168392+0000","last_active":"2023-12-19T17:01:22.157928+0000","last_peered":"2023-12-19T17:01:22.157928+0000","last_clean":"2023-12-19T17:01:22.157928+0000","last_became_active":"2023-12-19T17:01:21.168124+0000","last_became_peered":"2023-12-19T17:01:21.168124+0000","last_unstale":"2023-12-19T17:01:22.157928+0000","last_undegraded":"2023-12-19T17:01:22.157928+0000","last_fullsized":"2023-12-19T17:01:22.157928+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:41:20.519158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158710+0000","last_change":"2023-12-19T17:01:21.159972+0000","last_active":"2023-12-19T17:01:22.158710+0000","last_peered":"2023-12-19T17:01:22.158710+0000","last_clean":"2023-12-19T17:01:22.158710+0000","last_became_active":"2023-12-19T17:01:21.159719+0000","last_became_peered":"2023-12-19T17:01:21.159719+0000","last_unstale":"2023-12-19T17:01:22.158710+0000","last_undegraded":"2023-12-19T17:01:22.158710+0000","last_fullsized":"2023-12-19T17:01:22.158710+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:14:52.849543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"136'4","reported_seq":15,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163268+0000","last_change":"2023-12-19T17:01:20.761392+0000","last_active":"2023-12-19T17:01:22.163268+0000","last_peered":"2023-12-19T17:01:22.163268+0000","last_clean":"2023-12-19T17:01:22.163268+0000","last_became_active":"2023-12-19T17:01:20.761089+0000","last_became_peered":"2023-12-19T17:01:20.761089+0000","last_unstale":"2023-12-19T17:01:22.163268+0000","last_undegraded":"2023-12-19T17:01:22.163268+0000","last_fullsized":"2023-12-19T17:01:22.163268+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:06.194676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509754+0000","last_change":"2023-12-19T17:01:21.174481+0000","last_active":"2023-12-19T17:01:22.509754+0000","last_peered":"2023-12-19T17:01:22.509754+0000","last_clean":"2023-12-19T17:01:22.509754+0000","last_became_active":"2023-12-19T17:01:21.174214+0000","last_became_peered":"2023-12-19T17:01:21.174214+0000","last_unstale":"2023-12-19T17:01:22.509754+0000","last_undegraded":"2023-12-19T17:01:22.509754+0000","last_fullsized":"2023-12-19T17:01:22.509754+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:02:02.542798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"136'1","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509001+0000","last_change":"2023-12-19T17:01:20.762211+0000","last_active":"2023-12-19T17:01:22.509001+0000","last_peered":"2023-12-19T17:01:22.509001+0000","last_clean":"2023-12-19T17:01:22.509001+0000","last_became_active":"2023-12-19T17:01:20.761905+0000","last_became_peered":"2023-12-19T17:01:20.761905+0000","last_unstale":"2023-12-19T17:01:22.509001+0000","last_undegraded":"2023-12-19T17:01:22.509001+0000","last_fullsized":"2023-12-19T17:01:22.509001+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:47:41.266771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158543+0000","last_change":"2023-12-19T17:01:21.171035+0000","last_active":"2023-12-19T17:01:22.158543+0000","last_peered":"2023-12-19T17:01:22.158543+0000","last_clean":"2023-12-19T17:01:22.158543+0000","last_became_active":"2023-12-19T17:01:21.170758+0000","last_became_peered":"2023-12-19T17:01:21.170758+0000","last_unstale":"2023-12-19T17:01:22.158543+0000","last_undegraded":"2023-12-19T17:01:22.158543+0000","last_fullsized":"2023-12-19T17:01:22.158543+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:16.978719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"136'2","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481647+0000","last_change":"2023-12-19T17:01:20.758467+0000","last_active":"2023-12-19T17:01:22.481647+0000","last_peered":"2023-12-19T17:01:22.481647+0000","last_clean":"2023-12-19T17:01:22.481647+0000","last_became_active":"2023-12-19T17:01:20.758207+0000","last_became_peered":"2023-12-19T17:01:20.758207+0000","last_unstale":"2023-12-19T17:01:22.481647+0000","last_undegraded":"2023-12-19T17:01:22.481647+0000","last_fullsized":"2023-12-19T17:01:22.481647+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:16:34.000691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509409+0000","last_change":"2023-12-19T17:01:21.181538+0000","last_active":"2023-12-19T17:01:22.509409+0000","last_peered":"2023-12-19T17:01:22.509409+0000","last_clean":"2023-12-19T17:01:22.509409+0000","last_became_active":"2023-12-19T17:01:21.181230+0000","last_became_peered":"2023-12-19T17:01:21.181230+0000","last_unstale":"2023-12-19T17:01:22.509409+0000","last_undegraded":"2023-12-19T17:01:22.509409+0000","last_fullsized":"2023-12-19T17:01:22.509409+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:37:05.457522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"136'2","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481531+0000","last_change":"2023-12-19T17:01:20.760542+0000","last_active":"2023-12-19T17:01:22.481531+0000","last_peered":"2023-12-19T17:01:22.481531+0000","last_clean":"2023-12-19T17:01:22.481531+0000","last_became_active":"2023-12-19T17:01:20.760282+0000","last_became_peered":"2023-12-19T17:01:20.760282+0000","last_unstale":"2023-12-19T17:01:22.481531+0000","last_undegraded":"2023-12-19T17:01:22.481531+0000","last_fullsized":"2023-12-19T17:01:22.481531+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:06:06.857504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508799+0000","last_change":"2023-12-19T17:01:21.179503+0000","last_active":"2023-12-19T17:01:22.508799+0000","last_peered":"2023-12-19T17:01:22.508799+0000","last_clean":"2023-12-19T17:01:22.508799+0000","last_became_active":"2023-12-19T17:01:21.179205+0000","last_became_peered":"2023-12-19T17:01:21.179205+0000","last_unstale":"2023-12-19T17:01:22.508799+0000","last_undegraded":"2023-12-19T17:01:22.508799+0000","last_fullsized":"2023-12-19T17:01:22.508799+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:47.078840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"136'1","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163221+0000","last_change":"2023-12-19T17:01:20.764974+0000","last_active":"2023-12-19T17:01:22.163221+0000","last_peered":"2023-12-19T17:01:22.163221+0000","last_clean":"2023-12-19T17:01:22.163221+0000","last_became_active":"2023-12-19T17:01:20.764671+0000","last_became_peered":"2023-12-19T17:01:20.764671+0000","last_unstale":"2023-12-19T17:01:22.163221+0000","last_undegraded":"2023-12-19T17:01:22.163221+0000","last_fullsized":"2023-12-19T17:01:22.163221+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:17:27.185290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.161964+0000","last_change":"2023-12-19T17:01:21.162892+0000","last_active":"2023-12-19T17:01:22.161964+0000","last_peered":"2023-12-19T17:01:22.161964+0000","last_clean":"2023-12-19T17:01:22.161964+0000","last_became_active":"2023-12-19T17:01:21.162305+0000","last_became_peered":"2023-12-19T17:01:21.162305+0000","last_unstale":"2023-12-19T17:01:22.161964+0000","last_undegraded":"2023-12-19T17:01:22.161964+0000","last_fullsized":"2023-12-19T17:01:22.161964+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:47:42.269522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"136'1","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508528+0000","last_change":"2023-12-19T17:01:20.758226+0000","last_active":"2023-12-19T17:01:22.508528+0000","last_peered":"2023-12-19T17:01:22.508528+0000","last_clean":"2023-12-19T17:01:22.508528+0000","last_became_active":"2023-12-19T17:01:20.757660+0000","last_became_peered":"2023-12-19T17:01:20.757660+0000","last_unstale":"2023-12-19T17:01:22.508528+0000","last_undegraded":"2023-12-19T17:01:22.508528+0000","last_fullsized":"2023-12-19T17:01:22.508528+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:37:04.454772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508320+0000","last_change":"2023-12-19T17:01:21.180268+0000","last_active":"2023-12-19T17:01:22.508320+0000","last_peered":"2023-12-19T17:01:22.508320+0000","last_clean":"2023-12-19T17:01:22.508320+0000","last_became_active":"2023-12-19T17:01:21.176971+0000","last_became_peered":"2023-12-19T17:01:21.176971+0000","last_unstale":"2023-12-19T17:01:22.508320+0000","last_undegraded":"2023-12-19T17:01:22.508320+0000","last_fullsized":"2023-12-19T17:01:22.508320+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:20:43.779315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.161920+0000","last_change":"2023-12-19T17:01:20.750004+0000","last_active":"2023-12-19T17:01:22.161920+0000","last_peered":"2023-12-19T17:01:22.161920+0000","last_clean":"2023-12-19T17:01:22.161920+0000","last_became_active":"2023-12-19T17:01:20.749674+0000","last_became_peered":"2023-12-19T17:01:20.749674+0000","last_unstale":"2023-12-19T17:01:22.161920+0000","last_undegraded":"2023-12-19T17:01:22.161920+0000","last_fullsized":"2023-12-19T17:01:22.161920+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:11.210607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163012+0000","last_change":"2023-12-19T17:01:21.166939+0000","last_active":"2023-12-19T17:01:22.163012+0000","last_peered":"2023-12-19T17:01:22.163012+0000","last_clean":"2023-12-19T17:01:22.163012+0000","last_became_active":"2023-12-19T17:01:21.166604+0000","last_became_peered":"2023-12-19T17:01:21.166604+0000","last_unstale":"2023-12-19T17:01:22.163012+0000","last_undegraded":"2023-12-19T17:01:22.163012+0000","last_fullsized":"2023-12-19T17:01:22.163012+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:58.423008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508407+0000","last_change":"2023-12-19T17:01:20.764982+0000","last_active":"2023-12-19T17:01:22.508407+0000","last_peered":"2023-12-19T17:01:22.508407+0000","last_clean":"2023-12-19T17:01:22.508407+0000","last_became_active":"2023-12-19T17:01:20.763688+0000","last_became_peered":"2023-12-19T17:01:20.763688+0000","last_unstale":"2023-12-19T17:01:22.508407+0000","last_undegraded":"2023-12-19T17:01:22.508407+0000","last_fullsized":"2023-12-19T17:01:22.508407+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:24.679053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163310+0000","last_change":"2023-12-19T17:01:21.165409+0000","last_active":"2023-12-19T17:01:22.163310+0000","last_peered":"2023-12-19T17:01:22.163310+0000","last_clean":"2023-12-19T17:01:22.163310+0000","last_became_active":"2023-12-19T17:01:21.165044+0000","last_became_peered":"2023-12-19T17:01:21.165044+0000","last_unstale":"2023-12-19T17:01:22.163310+0000","last_undegraded":"2023-12-19T17:01:22.163310+0000","last_fullsized":"2023-12-19T17:01:22.163310+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:20:35.663601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481661+0000","last_change":"2023-12-19T17:01:20.760021+0000","last_active":"2023-12-19T17:01:22.481661+0000","last_peered":"2023-12-19T17:01:22.481661+0000","last_clean":"2023-12-19T17:01:22.481661+0000","last_became_active":"2023-12-19T17:01:20.759760+0000","last_became_peered":"2023-12-19T17:01:20.759760+0000","last_unstale":"2023-12-19T17:01:22.481661+0000","last_undegraded":"2023-12-19T17:01:22.481661+0000","last_fullsized":"2023-12-19T17:01:22.481661+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:15:57.899075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509756+0000","last_change":"2023-12-19T17:01:21.171195+0000","last_active":"2023-12-19T17:01:22.509756+0000","last_peered":"2023-12-19T17:01:22.509756+0000","last_clean":"2023-12-19T17:01:22.509756+0000","last_became_active":"2023-12-19T17:01:21.170870+0000","last_became_peered":"2023-12-19T17:01:21.170870+0000","last_unstale":"2023-12-19T17:01:22.509756+0000","last_undegraded":"2023-12-19T17:01:22.509756+0000","last_fullsized":"2023-12-19T17:01:22.509756+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:12:49.539867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508719+0000","last_change":"2023-12-19T17:01:20.758357+0000","last_active":"2023-12-19T17:01:22.508719+0000","last_peered":"2023-12-19T17:01:22.508719+0000","last_clean":"2023-12-19T17:01:22.508719+0000","last_became_active":"2023-12-19T17:01:20.758063+0000","last_became_peered":"2023-12-19T17:01:20.758063+0000","last_unstale":"2023-12-19T17:01:22.508719+0000","last_undegraded":"2023-12-19T17:01:22.508719+0000","last_fullsized":"2023-12-19T17:01:22.508719+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:35.811938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508587+0000","last_change":"2023-12-19T17:01:21.167608+0000","last_active":"2023-12-19T17:01:22.508587+0000","last_peered":"2023-12-19T17:01:22.508587+0000","last_clean":"2023-12-19T17:01:22.508587+0000","last_became_active":"2023-12-19T17:01:21.167343+0000","last_became_peered":"2023-12-19T17:01:21.167343+0000","last_unstale":"2023-12-19T17:01:22.508587+0000","last_undegraded":"2023-12-19T17:01:22.508587+0000","last_fullsized":"2023-12-19T17:01:22.508587+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:02:13.132918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"136'3","reported_seq":14,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508915+0000","last_change":"2023-12-19T17:01:20.764005+0000","last_active":"2023-12-19T17:01:22.508915+0000","last_peered":"2023-12-19T17:01:22.508915+0000","last_clean":"2023-12-19T17:01:22.508915+0000","last_became_active":"2023-12-19T17:01:20.763666+0000","last_became_peered":"2023-12-19T17:01:20.763666+0000","last_unstale":"2023-12-19T17:01:22.508915+0000","last_undegraded":"2023-12-19T17:01:22.508915+0000","last_fullsized":"2023-12-19T17:01:22.508915+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:32:49.766097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163361+0000","last_change":"2023-12-19T17:01:21.176347+0000","last_active":"2023-12-19T17:01:22.163361+0000","last_peered":"2023-12-19T17:01:22.163361+0000","last_clean":"2023-12-19T17:01:22.163361+0000","last_became_active":"2023-12-19T17:01:21.176082+0000","last_became_peered":"2023-12-19T17:01:21.176082+0000","last_unstale":"2023-12-19T17:01:22.163361+0000","last_undegraded":"2023-12-19T17:01:22.163361+0000","last_fullsized":"2023-12-19T17:01:22.163361+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:27:13.839010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"136'6","reported_seq":17,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509012+0000","last_change":"2023-12-19T17:01:20.752058+0000","last_active":"2023-12-19T17:01:22.509012+0000","last_peered":"2023-12-19T17:01:22.509012+0000","last_clean":"2023-12-19T17:01:22.509012+0000","last_became_active":"2023-12-19T17:01:20.750865+0000","last_became_peered":"2023-12-19T17:01:20.750865+0000","last_unstale":"2023-12-19T17:01:22.509012+0000","last_undegraded":"2023-12-19T17:01:22.509012+0000","last_fullsized":"2023-12-19T17:01:22.509012+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:53:57.220194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508509+0000","last_change":"2023-12-19T17:01:21.166468+0000","last_active":"2023-12-19T17:01:22.508509+0000","last_peered":"2023-12-19T17:01:22.508509+0000","last_clean":"2023-12-19T17:01:22.508509+0000","last_became_active":"2023-12-19T17:01:21.166187+0000","last_became_peered":"2023-12-19T17:01:21.166187+0000","last_unstale":"2023-12-19T17:01:22.508509+0000","last_undegraded":"2023-12-19T17:01:22.508509+0000","last_fullsized":"2023-12-19T17:01:22.508509+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:13.881080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"136'2","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158408+0000","last_change":"2023-12-19T17:01:20.761730+0000","last_active":"2023-12-19T17:01:22.158408+0000","last_peered":"2023-12-19T17:01:22.158408+0000","last_clean":"2023-12-19T17:01:22.158408+0000","last_became_active":"2023-12-19T17:01:20.759271+0000","last_became_peered":"2023-12-19T17:01:20.759271+0000","last_unstale":"2023-12-19T17:01:22.158408+0000","last_undegraded":"2023-12-19T17:01:22.158408+0000","last_fullsized":"2023-12-19T17:01:22.158408+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:12:34.670243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508983+0000","last_change":"2023-12-19T17:01:21.178696+0000","last_active":"2023-12-19T17:01:22.508983+0000","last_peered":"2023-12-19T17:01:22.508983+0000","last_clean":"2023-12-19T17:01:22.508983+0000","last_became_active":"2023-12-19T17:01:21.178105+0000","last_became_peered":"2023-12-19T17:01:21.178105+0000","last_unstale":"2023-12-19T17:01:22.508983+0000","last_undegraded":"2023-12-19T17:01:22.508983+0000","last_fullsized":"2023-12-19T17:01:22.508983+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:33:25.066550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509165+0000","last_change":"2023-12-19T17:01:20.768570+0000","last_active":"2023-12-19T17:01:22.509165+0000","last_peered":"2023-12-19T17:01:22.509165+0000","last_clean":"2023-12-19T17:01:22.509165+0000","last_became_active":"2023-12-19T17:01:20.768218+0000","last_became_peered":"2023-12-19T17:01:20.768218+0000","last_unstale":"2023-12-19T17:01:22.509165+0000","last_undegraded":"2023-12-19T17:01:22.509165+0000","last_fullsized":"2023-12-19T17:01:22.509165+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:39.103948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508306+0000","last_change":"2023-12-19T17:01:21.166553+0000","last_active":"2023-12-19T17:01:22.508306+0000","last_peered":"2023-12-19T17:01:22.508306+0000","last_clean":"2023-12-19T17:01:22.508306+0000","last_became_active":"2023-12-19T17:01:21.166151+0000","last_became_peered":"2023-12-19T17:01:21.166151+0000","last_unstale":"2023-12-19T17:01:22.508306+0000","last_undegraded":"2023-12-19T17:01:22.508306+0000","last_fullsized":"2023-12-19T17:01:22.508306+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:48:34.666631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"136'8","reported_seq":19,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481428+0000","last_change":"2023-12-19T17:01:20.768850+0000","last_active":"2023-12-19T17:01:22.481428+0000","last_peered":"2023-12-19T17:01:22.481428+0000","last_clean":"2023-12-19T17:01:22.481428+0000","last_became_active":"2023-12-19T17:01:20.768183+0000","last_became_peered":"2023-12-19T17:01:20.768183+0000","last_unstale":"2023-12-19T17:01:22.481428+0000","last_undegraded":"2023-12-19T17:01:22.481428+0000","last_fullsized":"2023-12-19T17:01:22.481428+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:51:57.415839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481684+0000","last_change":"2023-12-19T17:01:21.171790+0000","last_active":"2023-12-19T17:01:22.481684+0000","last_peered":"2023-12-19T17:01:22.481684+0000","last_clean":"2023-12-19T17:01:22.481684+0000","last_became_active":"2023-12-19T17:01:21.171292+0000","last_became_peered":"2023-12-19T17:01:21.171292+0000","last_unstale":"2023-12-19T17:01:22.481684+0000","last_undegraded":"2023-12-19T17:01:22.481684+0000","last_fullsized":"2023-12-19T17:01:22.481684+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:13:33.472416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"136'1","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509354+0000","last_change":"2023-12-19T17:01:20.767391+0000","last_active":"2023-12-19T17:01:22.509354+0000","last_peered":"2023-12-19T17:01:22.509354+0000","last_clean":"2023-12-19T17:01:22.509354+0000","last_became_active":"2023-12-19T17:01:20.766442+0000","last_became_peered":"2023-12-19T17:01:20.766442+0000","last_unstale":"2023-12-19T17:01:22.509354+0000","last_undegraded":"2023-12-19T17:01:22.509354+0000","last_fullsized":"2023-12-19T17:01:22.509354+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:46:27.003043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158876+0000","last_change":"2023-12-19T17:01:21.172659+0000","last_active":"2023-12-19T17:01:22.158876+0000","last_peered":"2023-12-19T17:01:22.158876+0000","last_clean":"2023-12-19T17:01:22.158876+0000","last_became_active":"2023-12-19T17:01:21.172390+0000","last_became_peered":"2023-12-19T17:01:21.172390+0000","last_unstale":"2023-12-19T17:01:22.158876+0000","last_undegraded":"2023-12-19T17:01:22.158876+0000","last_fullsized":"2023-12-19T17:01:22.158876+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:16:02.193853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"136'2","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481712+0000","last_change":"2023-12-19T17:01:20.759071+0000","last_active":"2023-12-19T17:01:22.481712+0000","last_peered":"2023-12-19T17:01:22.481712+0000","last_clean":"2023-12-19T17:01:22.481712+0000","last_became_active":"2023-12-19T17:01:20.757647+0000","last_became_peered":"2023-12-19T17:01:20.757647+0000","last_unstale":"2023-12-19T17:01:22.481712+0000","last_undegraded":"2023-12-19T17:01:22.481712+0000","last_fullsized":"2023-12-19T17:01:22.481712+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:52:56.973652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508859+0000","last_change":"2023-12-19T17:01:21.179611+0000","last_active":"2023-12-19T17:01:22.508859+0000","last_peered":"2023-12-19T17:01:22.508859+0000","last_clean":"2023-12-19T17:01:22.508859+0000","last_became_active":"2023-12-19T17:01:21.179025+0000","last_became_peered":"2023-12-19T17:01:21.179025+0000","last_unstale":"2023-12-19T17:01:22.508859+0000","last_undegraded":"2023-12-19T17:01:22.508859+0000","last_fullsized":"2023-12-19T17:01:22.508859+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:12.354302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"136'1","reported_seq":12,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509401+0000","last_change":"2023-12-19T17:01:20.760598+0000","last_active":"2023-12-19T17:01:22.509401+0000","last_peered":"2023-12-19T17:01:22.509401+0000","last_clean":"2023-12-19T17:01:22.509401+0000","last_became_active":"2023-12-19T17:01:20.760267+0000","last_became_peered":"2023-12-19T17:01:20.760267+0000","last_unstale":"2023-12-19T17:01:22.509401+0000","last_undegraded":"2023-12-19T17:01:22.509401+0000","last_fullsized":"2023-12-19T17:01:22.509401+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:15.975968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509359+0000","last_change":"2023-12-19T17:01:21.180915+0000","last_active":"2023-12-19T17:01:22.509359+0000","last_peered":"2023-12-19T17:01:22.509359+0000","last_clean":"2023-12-19T17:01:22.509359+0000","last_became_active":"2023-12-19T17:01:21.180574+0000","last_became_peered":"2023-12-19T17:01:21.180574+0000","last_unstale":"2023-12-19T17:01:22.509359+0000","last_undegraded":"2023-12-19T17:01:22.509359+0000","last_fullsized":"2023-12-19T17:01:22.509359+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:33:36.026852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163327+0000","last_change":"2023-12-19T17:01:20.769315+0000","last_active":"2023-12-19T17:01:22.163327+0000","last_peered":"2023-12-19T17:01:22.163327+0000","last_clean":"2023-12-19T17:01:22.163327+0000","last_became_active":"2023-12-19T17:01:20.768969+0000","last_became_peered":"2023-12-19T17:01:20.768969+0000","last_unstale":"2023-12-19T17:01:22.163327+0000","last_undegraded":"2023-12-19T17:01:22.163327+0000","last_fullsized":"2023-12-19T17:01:22.163327+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:59.214738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508780+0000","last_change":"2023-12-19T17:01:21.178808+0000","last_active":"2023-12-19T17:01:22.508780+0000","last_peered":"2023-12-19T17:01:22.508780+0000","last_clean":"2023-12-19T17:01:22.508780+0000","last_became_active":"2023-12-19T17:01:21.178434+0000","last_became_peered":"2023-12-19T17:01:21.178434+0000","last_unstale":"2023-12-19T17:01:22.508780+0000","last_undegraded":"2023-12-19T17:01:22.508780+0000","last_fullsized":"2023-12-19T17:01:22.508780+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:53:36.243483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"136'2","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.158427+0000","last_change":"2023-12-19T17:01:20.764438+0000","last_active":"2023-12-19T17:01:22.158427+0000","last_peered":"2023-12-19T17:01:22.158427+0000","last_clean":"2023-12-19T17:01:22.158427+0000","last_became_active":"2023-12-19T17:01:20.764157+0000","last_became_peered":"2023-12-19T17:01:20.764157+0000","last_unstale":"2023-12-19T17:01:22.158427+0000","last_undegraded":"2023-12-19T17:01:22.158427+0000","last_fullsized":"2023-12-19T17:01:22.158427+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:31:49.518342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508661+0000","last_change":"2023-12-19T17:01:21.181487+0000","last_active":"2023-12-19T17:01:22.508661+0000","last_peered":"2023-12-19T17:01:22.508661+0000","last_clean":"2023-12-19T17:01:22.508661+0000","last_became_active":"2023-12-19T17:01:21.181227+0000","last_became_peered":"2023-12-19T17:01:21.181227+0000","last_unstale":"2023-12-19T17:01:22.508661+0000","last_undegraded":"2023-12-19T17:01:22.508661+0000","last_fullsized":"2023-12-19T17:01:22.508661+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:26.258189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481409+0000","last_change":"2023-12-19T17:01:20.757946+0000","last_active":"2023-12-19T17:01:22.481409+0000","last_peered":"2023-12-19T17:01:22.481409+0000","last_clean":"2023-12-19T17:01:22.481409+0000","last_became_active":"2023-12-19T17:01:20.757645+0000","last_became_peered":"2023-12-19T17:01:20.757645+0000","last_unstale":"2023-12-19T17:01:22.481409+0000","last_undegraded":"2023-12-19T17:01:22.481409+0000","last_fullsized":"2023-12-19T17:01:22.481409+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:07.182292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509041+0000","last_change":"2023-12-19T17:01:21.181146+0000","last_active":"2023-12-19T17:01:22.509041+0000","last_peered":"2023-12-19T17:01:22.509041+0000","last_clean":"2023-12-19T17:01:22.509041+0000","last_became_active":"2023-12-19T17:01:21.180709+0000","last_became_peered":"2023-12-19T17:01:21.180709+0000","last_unstale":"2023-12-19T17:01:22.509041+0000","last_undegraded":"2023-12-19T17:01:22.509041+0000","last_fullsized":"2023-12-19T17:01:22.509041+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:36.814688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.157961+0000","last_change":"2023-12-19T17:01:20.764680+0000","last_active":"2023-12-19T17:01:22.157961+0000","last_peered":"2023-12-19T17:01:22.157961+0000","last_clean":"2023-12-19T17:01:22.157961+0000","last_became_active":"2023-12-19T17:01:20.764412+0000","last_became_peered":"2023-12-19T17:01:20.764412+0000","last_unstale":"2023-12-19T17:01:22.157961+0000","last_undegraded":"2023-12-19T17:01:22.157961+0000","last_fullsized":"2023-12-19T17:01:22.157961+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:09.971025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508678+0000","last_change":"2023-12-19T17:01:21.175298+0000","last_active":"2023-12-19T17:01:22.508678+0000","last_peered":"2023-12-19T17:01:22.508678+0000","last_clean":"2023-12-19T17:01:22.508678+0000","last_became_active":"2023-12-19T17:01:21.174897+0000","last_became_peered":"2023-12-19T17:01:21.174897+0000","last_unstale":"2023-12-19T17:01:22.508678+0000","last_undegraded":"2023-12-19T17:01:22.508678+0000","last_fullsized":"2023-12-19T17:01:22.508678+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:30.004851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508505+0000","last_change":"2023-12-19T17:01:20.765848+0000","last_active":"2023-12-19T17:01:22.508505+0000","last_peered":"2023-12-19T17:01:22.508505+0000","last_clean":"2023-12-19T17:01:22.508505+0000","last_became_active":"2023-12-19T17:01:20.764435+0000","last_became_peered":"2023-12-19T17:01:20.764435+0000","last_unstale":"2023-12-19T17:01:22.508505+0000","last_undegraded":"2023-12-19T17:01:22.508505+0000","last_fullsized":"2023-12-19T17:01:22.508505+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:54:53.906874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509461+0000","last_change":"2023-12-19T17:01:21.169428+0000","last_active":"2023-12-19T17:01:22.509461+0000","last_peered":"2023-12-19T17:01:22.509461+0000","last_clean":"2023-12-19T17:01:22.509461+0000","last_became_active":"2023-12-19T17:01:21.168788+0000","last_became_peered":"2023-12-19T17:01:21.168788+0000","last_unstale":"2023-12-19T17:01:22.509461+0000","last_undegraded":"2023-12-19T17:01:22.509461+0000","last_fullsized":"2023-12-19T17:01:22.509461+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:15.142197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"136'2","reported_seq":13,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508881+0000","last_change":"2023-12-19T17:01:20.755864+0000","last_active":"2023-12-19T17:01:22.508881+0000","last_peered":"2023-12-19T17:01:22.508881+0000","last_clean":"2023-12-19T17:01:22.508881+0000","last_became_active":"2023-12-19T17:01:20.755586+0000","last_became_peered":"2023-12-19T17:01:20.755586+0000","last_unstale":"2023-12-19T17:01:22.508881+0000","last_undegraded":"2023-12-19T17:01:22.508881+0000","last_fullsized":"2023-12-19T17:01:22.508881+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:26:18.063116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509260+0000","last_change":"2023-12-19T17:01:21.166827+0000","last_active":"2023-12-19T17:01:22.509260+0000","last_peered":"2023-12-19T17:01:22.509260+0000","last_clean":"2023-12-19T17:01:22.509260+0000","last_became_active":"2023-12-19T17:01:21.166551+0000","last_became_peered":"2023-12-19T17:01:21.166551+0000","last_unstale":"2023-12-19T17:01:22.509260+0000","last_undegraded":"2023-12-19T17:01:22.509260+0000","last_fullsized":"2023-12-19T17:01:22.509260+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:31.155271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509572+0000","last_change":"2023-12-19T17:01:20.759224+0000","last_active":"2023-12-19T17:01:22.509572+0000","last_peered":"2023-12-19T17:01:22.509572+0000","last_clean":"2023-12-19T17:01:22.509572+0000","last_became_active":"2023-12-19T17:01:20.758939+0000","last_became_peered":"2023-12-19T17:01:20.758939+0000","last_unstale":"2023-12-19T17:01:22.509572+0000","last_undegraded":"2023-12-19T17:01:22.509572+0000","last_fullsized":"2023-12-19T17:01:22.509572+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:32.253014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508932+0000","last_change":"2023-12-19T17:01:21.173966+0000","last_active":"2023-12-19T17:01:22.508932+0000","last_peered":"2023-12-19T17:01:22.508932+0000","last_clean":"2023-12-19T17:01:22.508932+0000","last_became_active":"2023-12-19T17:01:21.171653+0000","last_became_peered":"2023-12-19T17:01:21.171653+0000","last_unstale":"2023-12-19T17:01:22.508932+0000","last_undegraded":"2023-12-19T17:01:22.508932+0000","last_fullsized":"2023-12-19T17:01:22.508932+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:13:21.266430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509067+0000","last_change":"2023-12-19T17:01:20.759526+0000","last_active":"2023-12-19T17:01:22.509067+0000","last_peered":"2023-12-19T17:01:22.509067+0000","last_clean":"2023-12-19T17:01:22.509067+0000","last_became_active":"2023-12-19T17:01:20.758696+0000","last_became_peered":"2023-12-19T17:01:20.758696+0000","last_unstale":"2023-12-19T17:01:22.509067+0000","last_undegraded":"2023-12-19T17:01:22.509067+0000","last_fullsized":"2023-12-19T17:01:22.509067+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:54:54.464876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509180+0000","last_change":"2023-12-19T17:01:21.171826+0000","last_active":"2023-12-19T17:01:22.509180+0000","last_peered":"2023-12-19T17:01:22.509180+0000","last_clean":"2023-12-19T17:01:22.509180+0000","last_became_active":"2023-12-19T17:01:21.171530+0000","last_became_peered":"2023-12-19T17:01:21.171530+0000","last_unstale":"2023-12-19T17:01:22.509180+0000","last_undegraded":"2023-12-19T17:01:22.509180+0000","last_fullsized":"2023-12-19T17:01:22.509180+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:33.879282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508678+0000","last_change":"2023-12-19T17:01:20.760149+0000","last_active":"2023-12-19T17:01:22.508678+0000","last_peered":"2023-12-19T17:01:22.508678+0000","last_clean":"2023-12-19T17:01:22.508678+0000","last_became_active":"2023-12-19T17:01:20.759859+0000","last_became_peered":"2023-12-19T17:01:20.759859+0000","last_unstale":"2023-12-19T17:01:22.508678+0000","last_undegraded":"2023-12-19T17:01:22.508678+0000","last_fullsized":"2023-12-19T17:01:22.508678+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:11.351551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.161928+0000","last_change":"2023-12-19T17:01:21.160734+0000","last_active":"2023-12-19T17:01:22.161928+0000","last_peered":"2023-12-19T17:01:22.161928+0000","last_clean":"2023-12-19T17:01:22.161928+0000","last_became_active":"2023-12-19T17:01:21.160403+0000","last_became_peered":"2023-12-19T17:01:21.160403+0000","last_unstale":"2023-12-19T17:01:22.161928+0000","last_undegraded":"2023-12-19T17:01:22.161928+0000","last_fullsized":"2023-12-19T17:01:22.161928+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:12:49.539867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163251+0000","last_change":"2023-12-19T17:01:20.762097+0000","last_active":"2023-12-19T17:01:22.163251+0000","last_peered":"2023-12-19T17:01:22.163251+0000","last_clean":"2023-12-19T17:01:22.163251+0000","last_became_active":"2023-12-19T17:01:20.761146+0000","last_became_peered":"2023-12-19T17:01:20.761146+0000","last_unstale":"2023-12-19T17:01:22.163251+0000","last_undegraded":"2023-12-19T17:01:22.163251+0000","last_fullsized":"2023-12-19T17:01:22.163251+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:48.285357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162102+0000","last_change":"2023-12-19T17:01:21.161038+0000","last_active":"2023-12-19T17:01:22.162102+0000","last_peered":"2023-12-19T17:01:22.162102+0000","last_clean":"2023-12-19T17:01:22.162102+0000","last_became_active":"2023-12-19T17:01:21.160277+0000","last_became_peered":"2023-12-19T17:01:21.160277+0000","last_unstale":"2023-12-19T17:01:22.162102+0000","last_undegraded":"2023-12-19T17:01:22.162102+0000","last_fullsized":"2023-12-19T17:01:22.162102+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:53:23.892183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.162050+0000","last_change":"2023-12-19T17:01:20.756168+0000","last_active":"2023-12-19T17:01:22.162050+0000","last_peered":"2023-12-19T17:01:22.162050+0000","last_clean":"2023-12-19T17:01:22.162050+0000","last_became_active":"2023-12-19T17:01:20.755690+0000","last_became_peered":"2023-12-19T17:01:20.755690+0000","last_unstale":"2023-12-19T17:01:22.162050+0000","last_undegraded":"2023-12-19T17:01:22.162050+0000","last_fullsized":"2023-12-19T17:01:22.162050+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:12:34.670243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.509232+0000","last_change":"2023-12-19T17:01:21.180434+0000","last_active":"2023-12-19T17:01:22.509232+0000","last_peered":"2023-12-19T17:01:22.509232+0000","last_clean":"2023-12-19T17:01:22.509232+0000","last_became_active":"2023-12-19T17:01:21.180030+0000","last_became_peered":"2023-12-19T17:01:21.180030+0000","last_unstale":"2023-12-19T17:01:22.509232+0000","last_undegraded":"2023-12-19T17:01:22.509232+0000","last_fullsized":"2023-12-19T17:01:22.509232+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:00.405409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.481441+0000","last_change":"2023-12-19T17:01:20.758941+0000","last_active":"2023-12-19T17:01:22.481441+0000","last_peered":"2023-12-19T17:01:22.481441+0000","last_clean":"2023-12-19T17:01:22.481441+0000","last_became_active":"2023-12-19T17:01:20.758647+0000","last_became_peered":"2023-12-19T17:01:20.758647+0000","last_unstale":"2023-12-19T17:01:22.481441+0000","last_undegraded":"2023-12-19T17:01:22.481441+0000","last_fullsized":"2023-12-19T17:01:22.481441+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T17:04:24.307536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508775+0000","last_change":"2023-12-19T17:01:21.177636+0000","last_active":"2023-12-19T17:01:22.508775+0000","last_peered":"2023-12-19T17:01:22.508775+0000","last_clean":"2023-12-19T17:01:22.508775+0000","last_became_active":"2023-12-19T17:01:21.177301+0000","last_became_peered":"2023-12-19T17:01:21.177301+0000","last_unstale":"2023-12-19T17:01:22.508775+0000","last_undegraded":"2023-12-19T17:01:22.508775+0000","last_fullsized":"2023-12-19T17:01:22.508775+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:02.423935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508597+0000","last_change":"2023-12-19T17:01:20.765395+0000","last_active":"2023-12-19T17:01:22.508597+0000","last_peered":"2023-12-19T17:01:22.508597+0000","last_clean":"2023-12-19T17:01:22.508597+0000","last_became_active":"2023-12-19T17:01:20.764046+0000","last_became_peered":"2023-12-19T17:01:20.764046+0000","last_unstale":"2023-12-19T17:01:22.508597+0000","last_undegraded":"2023-12-19T17:01:22.508597+0000","last_fullsized":"2023-12-19T17:01:22.508597+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:42:34.362818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.508305+0000","last_change":"2023-12-19T17:01:21.169058+0000","last_active":"2023-12-19T17:01:22.508305+0000","last_peered":"2023-12-19T17:01:22.508305+0000","last_clean":"2023-12-19T17:01:22.508305+0000","last_became_active":"2023-12-19T17:01:21.167387+0000","last_became_peered":"2023-12-19T17:01:21.167387+0000","last_unstale":"2023-12-19T17:01:22.508305+0000","last_undegraded":"2023-12-19T17:01:22.508305+0000","last_fullsized":"2023-12-19T17:01:22.508305+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:20.735809+0000","last_clean_scrub_stamp":"2023-12-19T17:01:20.735809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:19:55.046386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-19T17:01:22.163195+0000","last_change":"2023-12-19T17:01:20.767278+0000","last_active":"2023-12-19T17:01:22.163195+0000","last_peered":"2023-12-19T17:01:22.163195+0000","last_clean":"2023-12-19T17:01:22.163195+0000","last_became_active":"2023-12-19T17:01:20.767010+0000","last_became_peered":"2023-12-19T17:01:20.767010+0000","last_unstale":"2023-12-19T17:01:22.163195+0000","last_undegraded":"2023-12-19T17:01:22.163195+0000","last_fullsized":"2023-12-19T17:01:22.163195+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:01:19.733059+0000","last_clean_scrub_stamp":"2023-12-19T17:01:19.733059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:06.730128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"125'247","reported_seq":730,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-12-19T17:01:21.169657+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T17:01:21.169657+0000","last_peered":"2023-12-19T17:01:21.169657+0000","last_clean":"2023-12-19T17:01:21.169657+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T17:01:21.169657+0000","last_undegraded":"2023-12-19T17:01:21.169657+0000","last_fullsized":"2023-12-19T17:01:21.169657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1250,"num_read_kb":2557,"num_write":395,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1250,"num_read_kb":2557,"num_write":395,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":247,"ondisk_log_size":247,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509668,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37384,"kb_used_data":2240,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2293760,"data_stored":1456575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67300000000000004,"15min":0.68600000000000005},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.32100000000000001},"max":{"1min":0.98499999999999999,"5min":1.04,"15min":1.3460000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68899999999999995,"15min":0.70799999999999996},"min":{"1min":0.443,"5min":0.39600000000000002,"15min":0.32700000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":1.6930000000000001},"last":0.57099999999999995}]},{"osd":1,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69199999999999995,"15min":0.70999999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.99199999999999999,"5min":1.0489999999999999,"15min":1.212},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70599999999999996,"15min":0.72899999999999998},"min":{"1min":0.52000000000000002,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":1.008,"5min":1.016,"15min":1.74},"last":0.65000000000000002}]},{"osd":2,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.42199999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.9990000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.44600000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.395},"last":0.71999999999999997}]},{"osd":3,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.879,"5min":1.0209999999999999,"15min":2.0379999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.73099999999999998,"15min":0.74399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.38300000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.266},"last":0.66900000000000004}]},{"osd":4,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.57699999999999996},"min":{"1min":0.39100000000000001,"5min":0.308,"15min":0.28499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94399999999999995,"15min":1.841},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60599999999999998,"15min":0.60499999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.38900000000000001},"max":{"1min":0.96299999999999997,"5min":1.0640000000000001,"15min":2.0710000000000002},"last":0.502}]},{"osd":5,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.39600000000000002,"5min":0.376,"15min":0.36899999999999999},"max":{"1min":0.94699999999999995,"5min":1.0700000000000001,"15min":1.4339999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63600000000000001,"15min":0.64300000000000002},"min":{"1min":0.41699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84099999999999997,"5min":1.0760000000000001,"15min":1.784},"last":0.52800000000000002}]},{"osd":6,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.503,"5min":0.45200000000000001,"15min":0.39800000000000002},"max":{"1min":0.83399999999999996,"5min":1.0109999999999999,"15min":1.8779999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.66300000000000003,"15min":0.66200000000000003},"min":{"1min":0.46899999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93300000000000005,"5min":1.0940000000000001,"15min":2.105},"last":0.63500000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509668,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1656,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1695744,"data_stored":865661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69199999999999995,"15min":0.70599999999999996},"min":{"1min":0.42999999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97199999999999998,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70299999999999996,"15min":0.70999999999999996},"min":{"1min":0.51100000000000001,"5min":0.44400000000000001,"15min":0.34999999999999998},"max":{"1min":0.88800000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.0289999999999999}]},{"osd":1,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.70499999999999996},"min":{"1min":0.42899999999999999,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.871,"5min":1.135,"15min":1.135},"last":0.435},{"interface":"front","average":{"1min":0.746,"5min":0.73799999999999999,"15min":0.73899999999999999},"min":{"1min":0.56699999999999995,"5min":0.42699999999999999,"15min":0.39500000000000002},"max":{"1min":0.91200000000000003,"5min":1.238,"15min":1.238},"last":0.876}]},{"osd":2,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73599999999999999,"15min":0.746},"min":{"1min":0.55700000000000005,"5min":0.47599999999999998,"15min":0.35199999999999998},"max":{"1min":1.0029999999999999,"5min":1.264,"15min":1.264},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71399999999999997,"15min":0.73399999999999999},"min":{"1min":0.52700000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98599999999999999,"5min":1.0049999999999999,"15min":1.1879999999999999},"last":0.84199999999999997}]},{"osd":3,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73899999999999999,"15min":0.753},"min":{"1min":0.54500000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.95699999999999996,"5min":1.413,"15min":1.413},"last":0.996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68500000000000005,"15min":0.71499999999999997},"min":{"1min":0.47599999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93300000000000005,"5min":0.94199999999999995,"15min":1.0740000000000001},"last":0.54900000000000004}]},{"osd":4,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58399999999999996,"15min":0.57899999999999996},"min":{"1min":0.42799999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83399999999999996,"5min":0.95399999999999996,"15min":0.96999999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.35699999999999998},"max":{"1min":0.81899999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57799999999999996,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.378,"15min":0.33300000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.96699999999999997},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65800000000000003,"15min":0.65700000000000003},"min":{"1min":0.501,"5min":0.42499999999999999,"15min":0.41499999999999998},"max":{"1min":0.84199999999999997,"5min":1.036,"15min":1.046},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Dec 19 17:00:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.59099999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":1.006,"15min":1.0249999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69199999999999995,"15min":0.69499999999999995},"min":{"1min":0.53400000000000003,"5min":0.46600000000000003,"15min":0.371},"max":{"1min":0.87,"5min":0.91100000000000003,"15min":1.032},"last":0.49199999999999999}]}]},{"osd":5,"up_from":15,"seq":64424509668,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1699840,"data_stored":866195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.69099999999999995,"15min":0.70499999999999996},"min":{"1min":0.56699999999999995,"5min":0.34399999999999997,"15min":0.29499999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.626},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.60199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.98399999999999999,"5min":0.99199999999999999,"15min":1.0189999999999999},"last":0.83099999999999996}]},{"osd":1,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.45000000000000001,"5min":0.39400000000000002,"15min":0.38600000000000001},"max":{"1min":0.91100000000000003,"5min":0.995,"15min":0.995},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.56599999999999995,"5min":0.40200000000000002,"15min":0.374},"max":{"1min":1.0309999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.64900000000000002}]},{"osd":2,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.73099999999999998,"15min":0.74199999999999999},"min":{"1min":0.44800000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.996,"5min":1.026,"15min":1.0409999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71099999999999997,"15min":0.72299999999999998},"min":{"1min":0.58699999999999997,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0249999999999999,"5min":1.036,"15min":1.036},"last":0.70899999999999996}]},{"osd":3,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.752,"15min":0.73599999999999999},"min":{"1min":0.63600000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.012,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.65900000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.054,"5min":1.085,"15min":1.089},"last":0.73399999999999999}]},{"osd":4,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.55800000000000005},"min":{"1min":0.373,"5min":0.35599999999999998,"15min":0.33300000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90400000000000003,"5min":0.92600000000000005,"15min":0.97199999999999998},"last":0.67300000000000004}]},{"osd":6,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.49099999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88400000000000001,"5min":0.89500000000000002,"15min":0.98799999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61499999999999999,"15min":0.61599999999999999},"min":{"1min":0.41399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.66000000000000003,"15min":0.65000000000000002},"min":{"1min":0.5,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.747},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67300000000000004,"15min":0.67400000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.42099999999999999},"max":{"1min":0.89300000000000002,"5min":1.1080000000000001,"15min":1.131},"last":0.80300000000000005}]}]},{"osd":4,"up_from":15,"seq":64424509668,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54844,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316996,"statfs":{"total":96636764160,"available":96580603904,"internally_reserved":0,"allocated":1691648,"data_stored":865591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.42799999999999999,"5min":0.39500000000000002,"15min":0.316},"max":{"1min":0.93100000000000005,"5min":1.0740000000000001,"15min":1.877},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69499999999999995,"15min":0.69999999999999996},"min":{"1min":0.41399999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0680000000000001},"last":0.85799999999999998}]},{"osd":1,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69499999999999995,"15min":0.70999999999999996},"min":{"1min":0.45400000000000001,"5min":0.39000000000000001,"15min":0.34799999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0209999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.56699999999999995,"5min":0.44600000000000001,"15min":0.35599999999999998},"max":{"1min":0.91600000000000004,"5min":0.98999999999999999,"15min":2.02},"last":0.89600000000000002}]},{"osd":2,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71899999999999997,"15min":0.72699999999999998},"min":{"1min":0.5,"5min":0.46800000000000003,"15min":0.38700000000000001},"max":{"1min":0.94499999999999995,"5min":0.97199999999999998,"15min":1.375},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.748},"min":{"1min":0.48099999999999998,"5min":0.439,"15min":0.42099999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.927},"last":1.046}]},{"osd":3,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.49399999999999999,"5min":0.437,"15min":0.41199999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.2030000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.58299999999999996,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98199999999999998,"5min":0.98299999999999998,"15min":1.0049999999999999},"last":0.93999999999999995}]},{"osd":5,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61899999999999999,"15min":0.60899999999999999},"min":{"1min":0.44900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.004,"5min":1.004,"15min":1.99},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.41999999999999998,"5min":0.40799999999999997,"15min":0.33500000000000002},"max":{"1min":0.92600000000000005,"5min":1.002,"15min":1.002},"last":0.55400000000000005}]},{"osd":6,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.41399999999999998,"15min":0.40300000000000002},"max":{"1min":0.90600000000000003,"5min":0.96299999999999997,"15min":0.97199999999999998},"last":0.502},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67500000000000004,"15min":0.66500000000000004},"min":{"1min":0.52000000000000002,"5min":0.44,"15min":0.376},"max":{"1min":0.97099999999999997,"5min":1.042,"15min":1.96},"last":0.58399999999999996}]},{"osd":7,"last update":"Tue Dec 19 17:00:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66600000000000004,"15min":0.65500000000000003},"min":{"1min":0.434,"5min":0.42299999999999999,"15min":0.376},"max":{"1min":0.871,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70699999999999996,"15min":0.69799999999999995},"min":{"1min":0.50900000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95399999999999996,"5min":1.0189999999999999,"15min":1.103},"last":0.66700000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509668,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40908,"kb_used_data":1668,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330932,"statfs":{"total":96636764160,"available":96594874368,"internally_reserved":0,"allocated":1708032,"data_stored":867225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.443,"15min":0.45000000000000001},"min":{"1min":0.26900000000000002,"5min":0.26200000000000001,"15min":0.25900000000000001},"max":{"1min":0.58399999999999996,"5min":0.69199999999999995,"15min":1.2509999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.27100000000000002},"max":{"1min":0.70299999999999996,"5min":1.0620000000000001,"15min":1.163},"last":0.65000000000000002}]},{"osd":1,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.46500000000000002},"min":{"1min":0.36699999999999999,"5min":0.29899999999999999,"15min":0.25900000000000001},"max":{"1min":0.67800000000000005,"5min":0.76300000000000001,"15min":0.77800000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.29699999999999999},"max":{"1min":0.69999999999999996,"5min":0.75900000000000001,"15min":1.952},"last":0.60899999999999999}]},{"osd":2,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.54200000000000004,"15min":0.54700000000000004},"min":{"1min":0.44,"5min":0.371,"15min":0.30099999999999999},"max":{"1min":0.76400000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56000000000000005,"15min":0.56699999999999995},"min":{"1min":0.40400000000000003,"5min":0.39500000000000002,"15min":0.36699999999999999},"max":{"1min":0.752,"5min":1.157,"15min":2.855},"last":0.63}]},{"osd":4,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.745,"15min":0.77800000000000002},"min":{"1min":0.58099999999999996,"5min":0.50600000000000001,"15min":0.432},"max":{"1min":0.98299999999999998,"5min":1.081,"15min":4.234},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.75900000000000001,"15min":0.77600000000000002},"min":{"1min":0.58799999999999997,"5min":0.56399999999999995,"15min":0.46100000000000002},"max":{"1min":1.0349999999999999,"5min":1.0649999999999999,"15min":1.3069999999999999},"last":1.369}]},{"osd":5,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.76500000000000001,"15min":0.77900000000000003},"min":{"1min":0.57399999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.266},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.751,"15min":0.77600000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.43099999999999999},"max":{"1min":1.278,"5min":1.278,"15min":1.8300000000000001},"last":1.121}]},{"osd":6,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76100000000000001,"15min":0.78800000000000003},"min":{"1min":0.54400000000000004,"5min":0.51200000000000001,"15min":0.45800000000000002},"max":{"1min":1.147,"5min":1.147,"15min":1.599},"last":1.218},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.76900000000000002,"15min":0.79200000000000004},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.49199999999999999},"max":{"1min":1.0049999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.409}]},{"osd":7,"last update":"Tue Dec 19 17:00:56 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.81000000000000005,"15min":0.82399999999999995},"min":{"1min":0.60099999999999998,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0920000000000001,"5min":1.385,"15min":1.385},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.81100000000000005,"15min":0.84499999999999997},"min":{"1min":0.67500000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.204,"5min":1.204,"15min":1.623},"last":1.4239999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509667,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58928,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312912,"statfs":{"total":96636764160,"available":96576421888,"internally_reserved":0,"allocated":1679360,"data_stored":858559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.433,"15min":0.45800000000000002},"min":{"1min":0.29099999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63900000000000001,"5min":0.66500000000000004,"15min":5.4870000000000001},"last":0.433},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.47999999999999998,"15min":0.52800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.72499999999999998,"5min":0.76500000000000001,"15min":10.161},"last":0.503}]},{"osd":1,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.57499999999999996,"15min":0.60499999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76500000000000001,"5min":0.82799999999999996,"15min":2.4740000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.504,"15min":0.51900000000000002},"min":{"1min":0.33200000000000002,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.58799999999999997,"5min":0.71499999999999997,"15min":2.911},"last":0.59999999999999998}]},{"osd":3,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.56699999999999995,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.71599999999999997,"5min":0.77300000000000002,"15min":1.1990000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54600000000000004,"15min":0.56999999999999995},"min":{"1min":0.40500000000000003,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69599999999999995,"5min":0.81499999999999995,"15min":2.7999999999999998},"last":0.71299999999999997}]},{"osd":4,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.75600000000000001,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0369999999999999,"5min":1.3,"15min":2.4449999999999998},"last":0.877},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.78400000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.42999999999999999},"max":{"1min":0.97299999999999998,"5min":1.3049999999999999,"15min":5.5430000000000001},"last":0.76900000000000002}]},{"osd":5,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.77200000000000002,"15min":0.79700000000000004},"min":{"1min":0.52200000000000002,"5min":0.5,"15min":0.49299999999999999},"max":{"1min":1.0600000000000001,"5min":1.4890000000000001,"15min":5.601},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.76200000000000001,"15min":0.80400000000000005},"min":{"1min":0.54600000000000004,"5min":0.504,"15min":0.504},"max":{"1min":0.99099999999999999,"5min":1.268,"15min":2.569},"last":0.92800000000000005}]},{"osd":6,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79700000000000004,"15min":0.81899999999999995},"min":{"1min":0.58799999999999997,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0369999999999999,"5min":1.4319999999999999,"15min":2.3889999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.78100000000000003},"min":{"1min":0.53100000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98399999999999999,"5min":1.1950000000000001,"15min":1.4379999999999999},"last":0.81100000000000005}]},{"osd":7,"last update":"Tue Dec 19 17:00:44 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.77800000000000002,"15min":0.81499999999999995},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.01,"5min":1.3340000000000001,"15min":5.673},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.80200000000000005,"15min":0.82699999999999996},"min":{"1min":0.59999999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.024,"5min":1.369,"15min":1.429},"last":0.78700000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509668,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50756,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321084,"statfs":{"total":96636764160,"available":96584790016,"internally_reserved":0,"allocated":1699840,"data_stored":865715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.439,"15min":0.44500000000000001},"min":{"1min":0.30099999999999999,"5min":0.26900000000000002,"15min":0.22500000000000001},"max":{"1min":0.60499999999999998,"5min":0.82799999999999996,"15min":0.84899999999999998},"last":0.313},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.496,"15min":0.51800000000000002},"min":{"1min":0.36399999999999999,"5min":0.29699999999999999,"15min":0.26400000000000001},"max":{"1min":0.92100000000000004,"5min":1.071,"15min":1.071},"last":0.45000000000000001}]},{"osd":2,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.35399999999999998,"5min":0.32900000000000001,"15min":0.30299999999999999},"max":{"1min":0.65000000000000002,"5min":0.67000000000000004,"15min":0.80400000000000005},"last":0.435},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52700000000000002,"15min":0.54800000000000004},"min":{"1min":0.36199999999999999,"5min":0.34799999999999998,"15min":0.317},"max":{"1min":0.80600000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49099999999999999}]},{"osd":3,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.58099999999999996,"15min":0.60199999999999998},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.504,"15min":0.54000000000000004},"min":{"1min":0.35499999999999998,"5min":0.34499999999999997,"15min":0.33700000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.91900000000000004},"last":0.36799999999999999}]},{"osd":4,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.74099999999999999,"15min":0.75700000000000001},"min":{"1min":0.46300000000000002,"5min":0.45300000000000001,"15min":0.43099999999999999},"max":{"1min":1.008,"5min":1.107,"15min":1.321},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.746,"15min":0.77600000000000002},"min":{"1min":0.56200000000000006,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98499999999999999,"5min":1.0580000000000001,"15min":1.4179999999999999},"last":0.56299999999999994}]},{"osd":5,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.747,"15min":0.77400000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.46500000000000002},"max":{"1min":0.97299999999999998,"5min":1.2010000000000001,"15min":1.4450000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76000000000000001,"15min":0.79100000000000004},"min":{"1min":0.59599999999999997,"5min":0.51200000000000001,"15min":0.46500000000000002},"max":{"1min":1.0489999999999999,"5min":1.0700000000000001,"15min":1.3380000000000001},"last":0.68100000000000005}]},{"osd":6,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.74399999999999999,"15min":0.78000000000000003},"min":{"1min":0.51500000000000001,"5min":0.496,"15min":0.41999999999999998},"max":{"1min":0.95599999999999996,"5min":1.048,"15min":1.3939999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77200000000000002,"15min":0.79800000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.52700000000000002},"max":{"1min":1.032,"5min":1.1659999999999999,"15min":1.391},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Dec 19 17:00:59 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79700000000000004,"15min":0.82399999999999995},"min":{"1min":0.58199999999999996,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.008,"5min":1.175,"15min":1.2589999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.79200000000000004,"15min":0.81999999999999995},"min":{"1min":0.55300000000000005,"5min":0.55000000000000004,"15min":0.53800000000000003},"max":{"1min":1.0720000000000001,"5min":1.2050000000000001,"15min":1.4219999999999999},"last":0.65300000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542373,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2252,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2306048,"data_stored":1457635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.443,"15min":0.434},"min":{"1min":0.29999999999999999,"5min":0.26000000000000001,"15min":0.246},"max":{"1min":0.67900000000000005,"5min":0.752,"15min":0.752},"last":0.502},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.498,"15min":0.49299999999999999},"min":{"1min":0.32800000000000001,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.69299999999999995,"5min":0.81699999999999995,"15min":0.85499999999999998},"last":0.38600000000000001}]},{"osd":2,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.51000000000000001,"15min":0.498},"min":{"1min":0.40200000000000002,"5min":0.33300000000000002,"15min":0.31900000000000001},"max":{"1min":0.67000000000000004,"5min":0.71499999999999997,"15min":0.77800000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.371,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73999999999999999,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56100000000000005}]},{"osd":3,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.34300000000000003,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.74199999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.623,"15min":0.621},"min":{"1min":0.42999999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81000000000000005,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.61099999999999999}]},{"osd":4,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.79100000000000004,"15min":0.79200000000000004},"min":{"1min":0.53500000000000003,"5min":0.503,"15min":0.45900000000000002},"max":{"1min":1.145,"5min":1.2090000000000001,"15min":1.444},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79200000000000004,"15min":0.79300000000000004},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.42599999999999999},"max":{"1min":1.0620000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.59199999999999997}]},{"osd":5,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.51400000000000001,"5min":0.502,"15min":0.41499999999999998},"max":{"1min":1.016,"5min":1.1599999999999999,"15min":1.294},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81299999999999994,"15min":0.80400000000000005},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.113,"5min":1.2549999999999999,"15min":1.256},"last":0.76500000000000001}]},{"osd":6,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81100000000000005,"15min":0.80700000000000005},"min":{"1min":0.44800000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.093,"5min":1.2789999999999999,"15min":1.429},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.81699999999999995},"min":{"1min":0.63500000000000001,"5min":0.53100000000000003,"15min":0.44700000000000001},"max":{"1min":1.046,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.94699999999999995}]},{"osd":7,"last update":"Tue Dec 19 17:00:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82999999999999996,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.48999999999999999},"max":{"1min":1.1870000000000001,"5min":1.3129999999999999,"15min":1.377},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.84099999999999997,"15min":0.82799999999999996},"min":{"1min":0.58399999999999996,"5min":0.51400000000000001,"15min":0.48099999999999998},"max":{"1min":1.0880000000000001,"5min":1.2649999999999999,"15min":1.3300000000000001},"last":0.67000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T17:01:25.668 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T17:01:25.669 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T17:01:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.853+0000 7f90ae6e5640 1 Processor -- start 2023-12-19T17:01:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.853+0000 7f90ae6e5640 1 -- start start 2023-12-19T17:01:25.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.853+0000 7f90ae6e5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90a806cb20 0x7f90a806f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.853+0000 7f90ae6e5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a806f790 con 0x7f90a806c740 2023-12-19T17:01:25.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.853+0000 7f90ae6e5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a815d5a0 con 0x7f90a815a3c0 2023-12-19T17:01:25.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.853+0000 7f90ae6e5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a815d880 con 0x7f90a806cb20 2023-12-19T17:01:25.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.853+0000 7f90a7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f90a806c740 0x7f90a80ca800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33390/0 (socket says 172.21.15.26:33390) 2023-12-19T17:01:25.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.853+0000 7f90a7fff640 1 -- 172.21.15.26:0/3539918504 learned_addr learned my addr 172.21.15.26:0/3539918504 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:25.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.853+0000 7f90a77fe640 1 --2- 172.21.15.26:0/3539918504 >> v2:172.21.15.26:3300/0 conn(0x7f90a806cb20 0x7f90a806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.853+0000 7f90a6ffd640 1 -- 172.21.15.26:0/3539918504 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538207700 0 0) 0x7f90a806f790 con 0x7f90a806c740 2023-12-19T17:01:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.853+0000 7f90a77fe640 1 -- 172.21.15.26:0/3539918504 >> v1:172.21.15.160:6789/0 conn(0x7f90a815a3c0 legacy=0x7f90a815a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.853+0000 7f90a77fe640 1 -- 172.21.15.26:0/3539918504 >> v1:172.21.15.26:6789/0 conn(0x7f90a806c740 legacy=0x7f90a80ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.853+0000 7f90a77fe640 1 -- 172.21.15.26:0/3539918504 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a815db60 con 0x7f90a806cb20 2023-12-19T17:01:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.857+0000 7f90a77fe640 1 --2- 172.21.15.26:0/3539918504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90a806cb20 0x7f90a806f250 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f909c0097c0 tx=0x7f909c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f71975377463d03 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:25.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.857+0000 7f90a6ffd640 1 -- 172.21.15.26:0/3539918504 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c0ab600 con 0x7f90a806cb20 2023-12-19T17:01:25.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.857+0000 7f90a6ffd640 1 -- 172.21.15.26:0/3539918504 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c0a7050 con 0x7f90a806cb20 2023-12-19T17:01:25.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.857+0000 7f90a6ffd640 1 -- 172.21.15.26:0/3539918504 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c0ab980 con 0x7f90a806cb20 2023-12-19T17:01:25.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.857+0000 7f90ae6e5640 1 -- 172.21.15.26:0/3539918504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90a806cb20 msgr2=0x7f90a806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.857+0000 7f90ae6e5640 1 --2- 172.21.15.26:0/3539918504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90a806cb20 0x7f90a806f250 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f909c0097c0 tx=0x7f909c01c440 comp rx=0 tx=0).stop 2023-12-19T17:01:25.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.857+0000 7f90ae6e5640 1 -- 172.21.15.26:0/3539918504 shutdown_connections 2023-12-19T17:01:25.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.857+0000 7f90ae6e5640 1 --2- 172.21.15.26:0/3539918504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90a806cb20 0x7f90a806f250 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.857+0000 7f90ae6e5640 1 -- 172.21.15.26:0/3539918504 >> 172.21.15.26:0/3539918504 conn(0x7f90a80660d0 msgr2=0x7f90a80ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:25.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.857+0000 7f90ae6e5640 1 -- 172.21.15.26:0/3539918504 shutdown_connections 2023-12-19T17:01:25.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.857+0000 7f90ae6e5640 1 -- 172.21.15.26:0/3539918504 wait complete. 2023-12-19T17:01:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90ae6e5640 1 Processor -- start 2023-12-19T17:01:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90ae6e5640 1 -- start start 2023-12-19T17:01:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90ae6e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90a806c740 0x7f90a8152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90ae6e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90a806cb20 0x7f90a81537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90ae6e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90a815a3c0 0x7f90a8153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90ae6e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a8158930 con 0x7f90a815a3c0 2023-12-19T17:01:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90ae6e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a8155be0 con 0x7f90a806cb20 2023-12-19T17:01:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90ae6e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a8155ec0 con 0x7f90a806c740 2023-12-19T17:01:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90acc82640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90a815a3c0 0x7f90a8153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90acc82640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90a815a3c0 0x7f90a8153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41008/0 (socket says 172.21.15.26:41008) 2023-12-19T17:01:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90acc82640 1 -- 172.21.15.26:0/1008979651 learned_addr learned my addr 172.21.15.26:0/1008979651 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90a7fff640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90a806c740 0x7f90a8152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90a77fe640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90a806cb20 0x7f90a81537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90acc82640 1 -- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90a806c740 msgr2=0x7f90a8152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90acc82640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90a806c740 0x7f90a8152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90acc82640 1 -- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90a806cb20 msgr2=0x7f90a81537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90acc82640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90a806cb20 0x7f90a81537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90acc82640 1 -- 172.21.15.26:0/1008979651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c009420 con 0x7f90a815a3c0 2023-12-19T17:01:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90a77fe640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90a806cb20 0x7f90a81537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90acc82640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90a815a3c0 0x7f90a8153ce0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f90a000b970 tx=0x7f90a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.861+0000 7f90a4ff9640 1 -- 172.21.15.26:0/1008979651 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a000cb40 con 0x7f90a815a3c0 2023-12-19T17:01:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.865+0000 7f90a4ff9640 1 -- 172.21.15.26:0/1008979651 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a000cca0 con 0x7f90a815a3c0 2023-12-19T17:01:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.865+0000 7f90a4ff9640 1 -- 172.21.15.26:0/1008979651 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0014680 con 0x7f90a815a3c0 2023-12-19T17:01:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.865+0000 7f90ae6e5640 1 -- 172.21.15.26:0/1008979651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a81561a0 con 0x7f90a815a3c0 2023-12-19T17:01:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.865+0000 7f90ae6e5640 1 -- 172.21.15.26:0/1008979651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a81533f0 con 0x7f90a815a3c0 2023-12-19T17:01:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.865+0000 7f90ae6e5640 1 -- 172.21.15.26:0/1008979651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a8074270 con 0x7f90a815a3c0 2023-12-19T17:01:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.865+0000 7f90a4ff9640 1 -- 172.21.15.26:0/1008979651 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f90a001d3b0 con 0x7f90a815a3c0 2023-12-19T17:01:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.865+0000 7f90a4ff9640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9094065b80 0x7f9094068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.865+0000 7f90a4ff9640 1 -- 172.21.15.26:0/1008979651 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f90a0083430 con 0x7f90a815a3c0 2023-12-19T17:01:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.865+0000 7f90a7fff640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9094065b80 0x7f9094068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.869+0000 7f90a7fff640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9094065b80 0x7f9094068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9098000f70 tx=0x7f9098007b60 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:25.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:25.869+0000 7f90a4ff9640 1 -- 172.21.15.26:0/1008979651 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f90a00509f0 con 0x7f90a815a3c0 2023-12-19T17:01:26.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.085+0000 7f90ae6e5640 1 -- 172.21.15.26:0/1008979651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f90a815d310 con 0x7f90a815a3c0 2023-12-19T17:01:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.089+0000 7f90a4ff9640 1 -- 172.21.15.26:0/1008979651 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7f90a0054010 con 0x7f90a815a3c0 2023-12-19T17:01:26.093 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2023-12-19T17:01:26.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.089+0000 7f90ae6e5640 1 -- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9094065b80 msgr2=0x7f9094068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:26.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.089+0000 7f90ae6e5640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9094065b80 0x7f9094068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9098000f70 tx=0x7f9098007b60 comp rx=0 tx=0).stop 2023-12-19T17:01:26.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.089+0000 7f90ae6e5640 1 -- 172.21.15.26:0/1008979651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90a815a3c0 msgr2=0x7f90a8153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:26.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.089+0000 7f90ae6e5640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90a815a3c0 0x7f90a8153ce0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f90a000b970 tx=0x7f90a000be30 comp rx=0 tx=0).stop 2023-12-19T17:01:26.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.089+0000 7f90ae6e5640 1 -- 172.21.15.26:0/1008979651 shutdown_connections 2023-12-19T17:01:26.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.089+0000 7f90ae6e5640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90a815a3c0 0x7f90a8153ce0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.089+0000 7f90ae6e5640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90a806cb20 0x7f90a81537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.089+0000 7f90ae6e5640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9094065b80 0x7f9094068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.089+0000 7f90ae6e5640 1 --2- 172.21.15.26:0/1008979651 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90a806c740 0x7f90a8152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.089+0000 7f90ae6e5640 1 -- 172.21.15.26:0/1008979651 >> 172.21.15.26:0/1008979651 conn(0x7f90a80660d0 msgr2=0x7f90a80cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.093+0000 7f90ae6e5640 1 -- 172.21.15.26:0/1008979651 shutdown_connections 2023-12-19T17:01:26.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.093+0000 7f90ae6e5640 1 -- 172.21.15.26:0/1008979651 wait complete. 2023-12-19T17:01:26.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:01:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a2c41640 1 Processor -- start 2023-12-19T17:01:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a2c41640 1 -- start start 2023-12-19T17:01:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a2c41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f299c15cf60 0x7f299c0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a2c41640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f299c0b4d10 0x7f299c0b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a2c41640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f299c0d2960 con 0x7f299c0d1420 2023-12-19T17:01:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a2c41640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299c0d18c0 con 0x7f299c15cf60 2023-12-19T17:01:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a2c41640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299c0d1ba0 con 0x7f299c0b4d10 2023-12-19T17:01:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a09dd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f299c15cf60 0x7f299c0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a09dd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f299c15cf60 0x7f299c0d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34264/0 (socket says 172.21.15.26:34264) 2023-12-19T17:01:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a09dd640 1 -- 172.21.15.26:0/503874339 learned_addr learned my addr 172.21.15.26:0/503874339 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a11de640 1 --2- 172.21.15.26:0/503874339 >> v2:172.21.15.160:3300/0 conn(0x7f299c0b4d10 0x7f299c0b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a09dd640 1 -- 172.21.15.26:0/503874339 >> v2:172.21.15.160:3300/0 conn(0x7f299c0b4d10 msgr2=0x7f299c0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a09dd640 1 --2- 172.21.15.26:0/503874339 >> v2:172.21.15.160:3300/0 conn(0x7f299c0b4d10 0x7f299c0b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a09dd640 1 -- 172.21.15.26:0/503874339 >> v1:172.21.15.160:6789/0 conn(0x7f299c0d1420 legacy=0x7f299c0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:26.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.257+0000 7f29a09dd640 1 -- 172.21.15.26:0/503874339 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c0d1e80 con 0x7f299c15cf60 2023-12-19T17:01:26.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.261+0000 7f29a09dd640 1 --2- 172.21.15.26:0/503874339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f299c15cf60 0x7f299c0d0ee0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f299c1608c0 tx=0x7f299001ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d0df3f28a07862 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.261+0000 7f299b7fe640 1 -- 172.21.15.26:0/503874339 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29900a89b0 con 0x7f299c15cf60 2023-12-19T17:01:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.261+0000 7f299b7fe640 1 -- 172.21.15.26:0/503874339 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29900a8b10 con 0x7f299c15cf60 2023-12-19T17:01:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.261+0000 7f299b7fe640 1 -- 172.21.15.26:0/503874339 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29900a8ea0 con 0x7f299c15cf60 2023-12-19T17:01:26.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.261+0000 7f29a2c41640 1 -- 172.21.15.26:0/503874339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f299c15cf60 msgr2=0x7f299c0d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.261+0000 7f29a2c41640 1 --2- 172.21.15.26:0/503874339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f299c15cf60 0x7f299c0d0ee0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f299c1608c0 tx=0x7f299001ecb0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.261+0000 7f29a2c41640 1 -- 172.21.15.26:0/503874339 shutdown_connections 2023-12-19T17:01:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.261+0000 7f29a2c41640 1 --2- 172.21.15.26:0/503874339 >> v2:172.21.15.160:3300/0 conn(0x7f299c0b4d10 0x7f299c0b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.261+0000 7f29a2c41640 1 --2- 172.21.15.26:0/503874339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f299c15cf60 0x7f299c0d0ee0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.261+0000 7f29a2c41640 1 -- 172.21.15.26:0/503874339 >> 172.21.15.26:0/503874339 conn(0x7f299c065ee0 msgr2=0x7f299c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.261+0000 7f29a2c41640 1 -- 172.21.15.26:0/503874339 shutdown_connections 2023-12-19T17:01:26.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.261+0000 7f29a2c41640 1 -- 172.21.15.26:0/503874339 wait complete. 2023-12-19T17:01:26.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a2c41640 1 Processor -- start 2023-12-19T17:01:26.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a2c41640 1 -- start start 2023-12-19T17:01:26.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a2c41640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f299c0b4d10 0x7f299c0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:26.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a2c41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f299c0d1420 0x7f299c151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:26.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a2c41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f299c15cf60 0x7f299c0cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:26.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a2c41640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c1560f0 con 0x7f299c15cf60 2023-12-19T17:01:26.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a2c41640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299c155ac0 con 0x7f299c0b4d10 2023-12-19T17:01:26.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a2c41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299c0ca740 con 0x7f299c0d1420 2023-12-19T17:01:26.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a11de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f299c15cf60 0x7f299c0cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:26.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a11de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f299c15cf60 0x7f299c0cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34280/0 (socket says 172.21.15.26:34280) 2023-12-19T17:01:26.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a11de640 1 -- 172.21.15.26:0/4196711024 learned_addr learned my addr 172.21.15.26:0/4196711024 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:26.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a09dd640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f299c0b4d10 0x7f299c0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:26.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f299bfff640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f299c0d1420 0x7f299c151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:26.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a11de640 1 -- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f299c0d1420 msgr2=0x7f299c151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:26.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a11de640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f299c0d1420 0x7f299c151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a11de640 1 -- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f299c0b4d10 msgr2=0x7f299c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:26.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a11de640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f299c0b4d10 0x7f299c0c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a11de640 1 -- 172.21.15.26:0/4196711024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29900094d0 con 0x7f299c15cf60 2023-12-19T17:01:26.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.265+0000 7f29a11de640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f299c15cf60 0x7f299c0cc0e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f2994007e40 tx=0x7f299400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:26.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.269+0000 7f29997fa640 1 -- 172.21.15.26:0/4196711024 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994014620 con 0x7f299c15cf60 2023-12-19T17:01:26.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.269+0000 7f29997fa640 1 -- 172.21.15.26:0/4196711024 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299400ee90 con 0x7f299c15cf60 2023-12-19T17:01:26.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.269+0000 7f29997fa640 1 -- 172.21.15.26:0/4196711024 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29940149e0 con 0x7f299c15cf60 2023-12-19T17:01:26.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.269+0000 7f29a2c41640 1 -- 172.21.15.26:0/4196711024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299c0caa00 con 0x7f299c15cf60 2023-12-19T17:01:26.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.269+0000 7f29a2c41640 1 -- 172.21.15.26:0/4196711024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299c1520f0 con 0x7f299c15cf60 2023-12-19T17:01:26.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.269+0000 7f29a2c41640 1 -- 172.21.15.26:0/4196711024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299c074270 con 0x7f299c15cf60 2023-12-19T17:01:26.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.269+0000 7f29997fa640 1 -- 172.21.15.26:0/4196711024 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2994014b40 con 0x7f299c15cf60 2023-12-19T17:01:26.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.269+0000 7f29997fa640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2980065bb0 0x7f2980068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:26.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.269+0000 7f29997fa640 1 -- 172.21.15.26:0/4196711024 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f2994083d60 con 0x7f299c15cf60 2023-12-19T17:01:26.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.269+0000 7f29a09dd640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2980065bb0 0x7f2980068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:26.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.273+0000 7f29a09dd640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2980065bb0 0x7f2980068070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f29900b2000 tx=0x7f2990009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:26.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.273+0000 7f29997fa640 1 -- 172.21.15.26:0/4196711024 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2994051470 con 0x7f299c15cf60 2023-12-19T17:01:26.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29a2c41640 1 -- 172.21.15.26:0/4196711024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f299c160370 con 0x7f299c15cf60 2023-12-19T17:01:26.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29997fa640 1 -- 172.21.15.26:0/4196711024 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f2994054a90 con 0x7f299c15cf60 2023-12-19T17:01:26.469 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:26.469 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8092,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3217828830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3217828830},{"type":"v1","addr":"172.21.15.160:6835","nonce":3217828830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2758554130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2758554130},{"type":"v1","addr":"172.21.15.160:6837","nonce":2758554130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3510623843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3510623843},{"type":"v1","addr":"172.21.15.26:6833","nonce":3510623843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:01:21.142075+0000","modified":"2023-12-19T17:01:24.167832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8487":{"gid":8487,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3823167404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3823167404},{"type":"v1","addr":"172.21.15.26:6835","nonce":3823167404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T17:01:26.470 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 65 2023-12-19T17:01:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29a2c41640 1 -- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2980065bb0 msgr2=0x7f2980068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29a2c41640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2980065bb0 0x7f2980068070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f29900b2000 tx=0x7f2990009040 comp rx=0 tx=0).stop 2023-12-19T17:01:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29a2c41640 1 -- 172.21.15.26:0/4196711024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f299c15cf60 msgr2=0x7f299c0cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29a2c41640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f299c15cf60 0x7f299c0cc0e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f2994007e40 tx=0x7f299400e4b0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29a2c41640 1 -- 172.21.15.26:0/4196711024 shutdown_connections 2023-12-19T17:01:26.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29a2c41640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f299c15cf60 0x7f299c0cc0e0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29a2c41640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2980065bb0 0x7f2980068070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29a2c41640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f299c0d1420 0x7f299c151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29a2c41640 1 --2- 172.21.15.26:0/4196711024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f299c0b4d10 0x7f299c0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:26.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29a2c41640 1 -- 172.21.15.26:0/4196711024 >> 172.21.15.26:0/4196711024 conn(0x7f299c065ee0 msgr2=0x7f299c0b4320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:26.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29a2c41640 1 -- 172.21.15.26:0/4196711024 shutdown_connections 2023-12-19T17:01:26.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:26.465+0000 7f29a2c41640 1 -- 172.21.15.26:0/4196711024 wait complete. 2023-12-19T17:01:26.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T17:01:21.142075+0000', 'modified': '2023-12-19T17:01:24.167832+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8487': {'gid': 8487, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/3823167404', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 3823167404}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 3823167404}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T17:01:26.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T17:01:26.894 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-19T17:01:27.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.037+0000 7f3206725640 1 Processor -- start 2023-12-19T17:01:27.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.037+0000 7f3206725640 1 -- start start 2023-12-19T17:01:27.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.037+0000 7f3206725640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f320015f370 0x7f320015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.037+0000 7f3206725640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f320006c970 con 0x7f3200155030 2023-12-19T17:01:27.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.037+0000 7f3206725640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f320015b1f0 con 0x7f32000ce740 2023-12-19T17:01:27.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.037+0000 7f3206725640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f320006c790 con 0x7f320015f370 2023-12-19T17:01:27.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.037+0000 7f31fffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3200155030 0x7f320006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57324/0 (socket says 172.21.15.26:57324) 2023-12-19T17:01:27.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.037+0000 7f31fffff640 1 -- 172.21.15.26:0/2843409104 learned_addr learned my addr 172.21.15.26:0/2843409104 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:27.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.041+0000 7f31ff7fe640 1 --2- 172.21.15.26:0/2843409104 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f320015f370 0x7f320015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.041+0000 7f31feffd640 1 -- 172.21.15.26:0/2843409104 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262249 0 0) 0x7f320006c970 con 0x7f3200155030 2023-12-19T17:01:27.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.041+0000 7f31feffd640 1 -- 172.21.15.26:0/2843409104 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31e8003530 con 0x7f3200155030 2023-12-19T17:01:27.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.041+0000 7f31feffd640 1 -- 172.21.15.26:0/2843409104 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568067437 0 0) 0x7f320015b1f0 con 0x7f32000ce740 2023-12-19T17:01:27.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.041+0000 7f31ff7fe640 1 -- 172.21.15.26:0/2843409104 >> v1:172.21.15.160:6789/0 conn(0x7f32000ce740 legacy=0x7f32000ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.041+0000 7f31ff7fe640 1 -- 172.21.15.26:0/2843409104 >> v1:172.21.15.26:6789/0 conn(0x7f3200155030 legacy=0x7f320006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.041+0000 7f31ff7fe640 1 -- 172.21.15.26:0/2843409104 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320015b4d0 con 0x7f320015f370 2023-12-19T17:01:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.041+0000 7f31ff7fe640 1 --2- 172.21.15.26:0/2843409104 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f320015f370 0x7f320015bcd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f31f40097f0 tx=0x7f31f401c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df8aa2840f403d97 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.041+0000 7f31feffd640 1 -- 172.21.15.26:0/2843409104 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4004230 con 0x7f320015f370 2023-12-19T17:01:27.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.041+0000 7f31feffd640 1 -- 172.21.15.26:0/2843409104 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f40a7050 con 0x7f320015f370 2023-12-19T17:01:27.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.041+0000 7f31feffd640 1 -- 172.21.15.26:0/2843409104 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4003620 con 0x7f320015f370 2023-12-19T17:01:27.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 -- 172.21.15.26:0/2843409104 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f320015f370 msgr2=0x7f320015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 --2- 172.21.15.26:0/2843409104 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f320015f370 0x7f320015bcd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f31f40097f0 tx=0x7f31f401c3b0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 -- 172.21.15.26:0/2843409104 shutdown_connections 2023-12-19T17:01:27.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 --2- 172.21.15.26:0/2843409104 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f320015f370 0x7f320015bcd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 -- 172.21.15.26:0/2843409104 >> 172.21.15.26:0/2843409104 conn(0x7f32000658e0 msgr2=0x7f3200065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:27.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 -- 172.21.15.26:0/2843409104 shutdown_connections 2023-12-19T17:01:27.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 -- 172.21.15.26:0/2843409104 wait complete. 2023-12-19T17:01:27.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 Processor -- start 2023-12-19T17:01:27.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 -- start start 2023-12-19T17:01:27.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32000ce740 0x7f32001f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3200155030 0x7f320006b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015f370 0x7f320006be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32001f6e60 con 0x7f320015f370 2023-12-19T17:01:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32001f7140 con 0x7f32000ce740 2023-12-19T17:01:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3206725640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32001f7420 con 0x7f3200155030 2023-12-19T17:01:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3204cc2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015f370 0x7f320006be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3204cc2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015f370 0x7f320006be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34296/0 (socket says 172.21.15.26:34296) 2023-12-19T17:01:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.045+0000 7f3204cc2640 1 -- 172.21.15.26:0/3541200758 learned_addr learned my addr 172.21.15.26:0/3541200758 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f31ff7fe640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3200155030 0x7f320006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f31fffff640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32000ce740 0x7f32001f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f3204cc2640 1 -- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3200155030 msgr2=0x7f320006b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f3204cc2640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3200155030 0x7f320006b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f3204cc2640 1 -- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32000ce740 msgr2=0x7f32001f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f3204cc2640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32000ce740 0x7f32001f65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f3204cc2640 1 -- 172.21.15.26:0/3541200758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f4009420 con 0x7f320015f370 2023-12-19T17:01:27.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f31fffff640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32000ce740 0x7f32001f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f3204cc2640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015f370 0x7f320006be60 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f31f8001570 tx=0x7f31f800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f31fcff9640 1 -- 172.21.15.26:0/3541200758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8003db0 con 0x7f320015f370 2023-12-19T17:01:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f31fcff9640 1 -- 172.21.15.26:0/3541200758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f8005070 con 0x7f320015f370 2023-12-19T17:01:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f31fcff9640 1 -- 172.21.15.26:0/3541200758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8005420 con 0x7f320015f370 2023-12-19T17:01:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f3206725640 1 -- 172.21.15.26:0/3541200758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32001f7700 con 0x7f320015f370 2023-12-19T17:01:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f3206725640 1 -- 172.21.15.26:0/3541200758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32001f6b20 con 0x7f320015f370 2023-12-19T17:01:27.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f31fcff9640 1 -- 172.21.15.26:0/3541200758 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f31f8005580 con 0x7f320015f370 2023-12-19T17:01:27.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f31fcff9640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f31ec065b80 0x7f31ec068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.049+0000 7f31fcff9640 1 -- 172.21.15.26:0/3541200758 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f31f8082fa0 con 0x7f320015f370 2023-12-19T17:01:27.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31fffff640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f31ec065b80 0x7f31ec068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 Processor -- start 2023-12-19T17:01:27.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 -- start start 2023-12-19T17:01:27.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31fffff640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f31ec065b80 0x7f31ec068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f31f0000f70 tx=0x7f31f0009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:27.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b08e070 0x55d34b08e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b043c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b0441c0 0x55d34b0445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34ae67c90 con 0x55d34b08e990 2023-12-19T17:01:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34ae99220 con 0x55d34b0441c0 2023-12-19T17:01:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d34ae967f0 con 0x55d34b08e070 2023-12-19T17:01:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b043c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b043c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34306/0 (socket says 172.21.15.26:34306) 2023-12-19T17:01:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31ff7fe640 1 -- 172.21.15.26:0/941099973 learned_addr learned my addr 172.21.15.26:0/941099973 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31fffff640 1 --2- 172.21.15.26:0/941099973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b08e070 0x55d34b08e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3204cc2640 1 --2- 172.21.15.26:0/941099973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b0441c0 0x55d34b0445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31ff7fe640 1 -- 172.21.15.26:0/941099973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b08e070 msgr2=0x55d34b08e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31ff7fe640 1 --2- 172.21.15.26:0/941099973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b08e070 0x55d34b08e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31ff7fe640 1 -- 172.21.15.26:0/941099973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b0441c0 msgr2=0x55d34b0445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:27.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31ff7fe640 1 --2- 172.21.15.26:0/941099973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b0441c0 0x55d34b0445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31ff7fe640 1 -- 172.21.15.26:0/941099973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d34ab8e890 con 0x55d34b08e990 2023-12-19T17:01:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31ff7fe640 1 --2- 172.21.15.26:0/941099973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b043c80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f31f4019fe0 tx=0x7f31f4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c45d9bc8a9e5f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31e67fc640 1 -- 172.21.15.26:0/941099973 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4004430 con 0x55d34b08e990 2023-12-19T17:01:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31e67fc640 1 -- 172.21.15.26:0/941099973 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f40ba070 con 0x55d34b08e990 2023-12-19T17:01:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f31e67fc640 1 -- 172.21.15.26:0/941099973 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4003840 con 0x55d34b08e990 2023-12-19T17:01:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 -- 172.21.15.26:0/941099973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 msgr2=0x55d34b043c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 --2- 172.21.15.26:0/941099973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b043c80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f31f4019fe0 tx=0x7f31f4004740 comp rx=0 tx=0).stop 2023-12-19T17:01:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 -- 172.21.15.26:0/941099973 shutdown_connections 2023-12-19T17:01:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 --2- 172.21.15.26:0/941099973 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b0441c0 0x55d34b0445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 --2- 172.21.15.26:0/941099973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b043c80 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 --2- 172.21.15.26:0/941099973 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b08e070 0x55d34b08e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 -- 172.21.15.26:0/941099973 >> 172.21.15.26:0/941099973 conn(0x55d34b04a950 msgr2=0x55d34b042560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 -- 172.21.15.26:0/941099973 shutdown_connections 2023-12-19T17:01:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.053+0000 7f3209a8d000 1 -- 172.21.15.26:0/941099973 wait complete. 2023-12-19T17:01:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 1 -- start start 2023-12-19T17:01:27.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 1 Processor -- start 2023-12-19T17:01:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b0441c0 0x55d34b049750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b08e070 0x55d34b050080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b0505c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34ae920e0 con 0x55d34b08e990 2023-12-19T17:01:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34ae31310 con 0x55d34b08e070 2023-12-19T17:01:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d34ae4f770 con 0x55d34b0441c0 2023-12-19T17:01:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 10 client.0 _ll_register_callbacks cb 0x55d34b02fb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T17:01:27.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3204cc2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b0505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3204cc2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b0505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34310/0 (socket says 172.21.15.26:34310) 2023-12-19T17:01:27.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3204cc2640 1 -- 172.21.15.26:0/2157670296 learned_addr learned my addr 172.21.15.26:0/2157670296 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:27.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f31fffff640 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b0441c0 0x55d34b049750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f31ff7fe640 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b08e070 0x55d34b050080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3204cc2640 1 -- 172.21.15.26:0/2157670296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b0441c0 msgr2=0x55d34b049750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:27.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3204cc2640 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b0441c0 0x55d34b049750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3204cc2640 1 -- 172.21.15.26:0/2157670296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b08e070 msgr2=0x55d34b050080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:27.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3204cc2640 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b08e070 0x55d34b050080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3204cc2640 1 -- 172.21.15.26:0/2157670296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320015b4d0 con 0x55d34b08e990 2023-12-19T17:01:27.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f31fffff640 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b0441c0 0x55d34b049750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:27.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3204cc2640 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b0505c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f31f8005990 tx=0x7f31f8054890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54208fdafb35a8a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:27.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f31deffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T17:01:27.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f31deffd640 1 -- 172.21.15.26:0/2157670296 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f805c170 con 0x55d34b08e990 2023-12-19T17:01:27.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f31deffd640 1 -- 172.21.15.26:0/2157670296 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f31f8053680 con 0x55d34b08e990 2023-12-19T17:01:27.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 1 -- 172.21.15.26:0/2157670296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d34ae61c10 con 0x55d34b08e990 2023-12-19T17:01:27.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f31deffd640 1 -- 172.21.15.26:0/2157670296 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31f8053840 con 0x55d34b08e990 2023-12-19T17:01:27.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 10 client.8640 fetch_fsmap learned FSMap version 65 2023-12-19T17:01:27.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 1 -- 172.21.15.26:0/2157670296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55d34ae57260 con 0x55d34b08e990 2023-12-19T17:01:27.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f31deffd640 1 -- 172.21.15.26:0/2157670296 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f806f070 con 0x55d34b08e990 2023-12-19T17:01:27.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f31deffd640 1 -- 172.21.15.26:0/2157670296 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f31f805bce0 con 0x55d34b08e990 2023-12-19T17:01:27.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 10 client.8640 fetch_fsmap finished waiting for FSMap version 65 2023-12-19T17:01:27.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 10 client.8640 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T17:01:27.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 20 client.8640 populate_metadata read hostname 'smithi026' 2023-12-19T17:01:27.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.26:6834/3823167404,v1:172.21.15.26:6835/3823167404] conn(0x55d34b0958d0 0x55d34b095cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 4 client.8640 mds_command: new command op to 8487 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T17:01:27.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f3209a8d000 1 -- 172.21.15.26:0/2157670296 --> [v2:172.21.15.26:6834/3823167404,v1:172.21.15.26:6835/3823167404] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d34ae90a30 con 0x55d34b0958d0 2023-12-19T17:01:27.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.057+0000 7f31fffff640 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.26:6834/3823167404,v1:172.21.15.26:6835/3823167404] conn(0x55d34b0958d0 0x55d34b095cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f31fffff640 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.26:6834/3823167404,v1:172.21.15.26:6835/3823167404] conn(0x55d34b0958d0 0x55d34b095cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9cf0def09ff160d server_cookie=6dfa85b6fcd904f4 in_seq=0 out_seq=0 2023-12-19T17:01:27.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f31deffd640 10 client.8640 ms_handle_connect on v2:172.21.15.26:6834/3823167404 2023-12-19T17:01:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f31deffd640 1 -- 172.21.15.26:0/2157670296 <== mds.0 v2:172.21.15.26:6834/3823167404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d34ae90a30 con 0x55d34b0958d0 2023-12-19T17:01:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f31deffd640 10 client.8640 handle_command_reply: tid=0 2023-12-19T17:01:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f3209a8d000 1 client.8640 shutdown 2023-12-19T17:01:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f3209a8d000 1 -- 172.21.15.26:0/2157670296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 msgr2=0x55d34b0505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f3209a8d000 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b0505c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f31f8005990 tx=0x7f31f8054890 comp rx=0 tx=0).stop 2023-12-19T17:01:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f3209a8d000 1 -- 172.21.15.26:0/2157670296 shutdown_connections 2023-12-19T17:01:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f3209a8d000 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.26:6834/3823167404,v1:172.21.15.26:6835/3823167404] conn(0x55d34b0958d0 0x55d34b095cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f3209a8d000 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b0505c0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f3209a8d000 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b08e070 0x55d34b050080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f3209a8d000 1 --2- 172.21.15.26:0/2157670296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b0441c0 0x55d34b049750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f31deffd640 0 client.8640 ms_handle_reset on v2:172.21.15.26:6834/3823167404 2023-12-19T17:01:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f3209a8d000 1 -- 172.21.15.26:0/2157670296 >> 172.21.15.26:0/2157670296 conn(0x55d34b04a950 msgr2=0x55d34b042c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f3209a8d000 1 -- 172.21.15.26:0/2157670296 shutdown_connections 2023-12-19T17:01:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f3209a8d000 1 -- 172.21.15.26:0/2157670296 wait complete. 2023-12-19T17:01:27.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.061+0000 7f3209a8d000 20 client.8640 trim_cache size 0 max 16384 2023-12-19T17:01:27.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 Processor -- start 2023-12-19T17:01:27.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 -- start start 2023-12-19T17:01:27.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b0441c0 0x55d34b0e62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e070 0x55d34b0e6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b08e990 0x55d34b0e6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34ab4f940 con 0x55d34b08e070 2023-12-19T17:01:27.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34ae90f70 con 0x55d34b0441c0 2023-12-19T17:01:27.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d34b03b9d0 con 0x55d34b08e990 2023-12-19T17:01:27.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e070 0x55d34b0e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31ff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e070 0x55d34b0e6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34318/0 (socket says 172.21.15.26:34318) 2023-12-19T17:01:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31ff7fe640 1 -- 172.21.15.26:0/2003423468 learned_addr learned my addr 172.21.15.26:0/2003423468 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3204cc2640 1 --2- 172.21.15.26:0/2003423468 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b08e990 0x55d34b0e6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31fffff640 1 --2- 172.21.15.26:0/2003423468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b0441c0 0x55d34b0e62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31ff7fe640 1 -- 172.21.15.26:0/2003423468 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b08e990 msgr2=0x55d34b0e6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31ff7fe640 1 --2- 172.21.15.26:0/2003423468 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b08e990 0x55d34b0e6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31ff7fe640 1 -- 172.21.15.26:0/2003423468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b0441c0 msgr2=0x55d34b0e62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31ff7fe640 1 --2- 172.21.15.26:0/2003423468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b0441c0 0x55d34b0e62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31ff7fe640 1 -- 172.21.15.26:0/2003423468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d34ae3d3b0 con 0x55d34b08e070 2023-12-19T17:01:27.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31fffff640 1 --2- 172.21.15.26:0/2003423468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b0441c0 0x55d34b0e62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:01:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31ff7fe640 1 --2- 172.21.15.26:0/2003423468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e070 0x55d34b0e6830 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f31f4004780 tx=0x7f31f40a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8954879bf41327 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31e6ffd640 1 -- 172.21.15.26:0/2003423468 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4004430 con 0x55d34b08e070 2023-12-19T17:01:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31e6ffd640 1 -- 172.21.15.26:0/2003423468 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f40ba040 con 0x55d34b08e070 2023-12-19T17:01:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f31e6ffd640 1 -- 172.21.15.26:0/2003423468 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4003820 con 0x55d34b08e070 2023-12-19T17:01:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 -- 172.21.15.26:0/2003423468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e070 msgr2=0x55d34b0e6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 --2- 172.21.15.26:0/2003423468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e070 0x55d34b0e6830 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f31f4004780 tx=0x7f31f40a8000 comp rx=0 tx=0).stop 2023-12-19T17:01:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 -- 172.21.15.26:0/2003423468 shutdown_connections 2023-12-19T17:01:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 --2- 172.21.15.26:0/2003423468 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b08e990 0x55d34b0e6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 --2- 172.21.15.26:0/2003423468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e070 0x55d34b0e6830 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 --2- 172.21.15.26:0/2003423468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b0441c0 0x55d34b0e62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 -- 172.21.15.26:0/2003423468 >> 172.21.15.26:0/2003423468 conn(0x55d34b04a950 msgr2=0x55d34b04ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 -- 172.21.15.26:0/2003423468 shutdown_connections 2023-12-19T17:01:27.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 -- 172.21.15.26:0/2003423468 wait complete. 2023-12-19T17:01:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.077+0000 7f3209a8d000 1 -- start start 2023-12-19T17:01:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 Processor -- start 2023-12-19T17:01:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b0441c0 0x55d34b0ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b08e070 0x55d34b0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b0cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34ae871f0 con 0x55d34b08e990 2023-12-19T17:01:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d34ae9a4b0 con 0x55d34b08e070 2023-12-19T17:01:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d34ae960e0 con 0x55d34b0441c0 2023-12-19T17:01:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 10 client.0 _ll_register_callbacks cb 0x55d34ae40640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T17:01:27.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3204cc2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b0cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3204cc2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b0cf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34334/0 (socket says 172.21.15.26:34334) 2023-12-19T17:01:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3204cc2640 1 -- 172.21.15.26:0/3692753392 learned_addr learned my addr 172.21.15.26:0/3692753392 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f31fffff640 1 --2- 172.21.15.26:0/3692753392 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b0441c0 0x55d34b0ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f31ff7fe640 1 --2- 172.21.15.26:0/3692753392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b08e070 0x55d34b0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3204cc2640 1 -- 172.21.15.26:0/3692753392 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b0441c0 msgr2=0x55d34b0ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3204cc2640 1 --2- 172.21.15.26:0/3692753392 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b0441c0 0x55d34b0ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3204cc2640 1 -- 172.21.15.26:0/3692753392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b08e070 msgr2=0x55d34b0cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:27.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3204cc2640 1 --2- 172.21.15.26:0/3692753392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b08e070 0x55d34b0cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3204cc2640 1 -- 172.21.15.26:0/3692753392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d34ab8e890 con 0x55d34b08e990 2023-12-19T17:01:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3204cc2640 1 --2- 172.21.15.26:0/3692753392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b0cf5d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f31f8053e90 tx=0x7f31f8001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f675da438ef1eb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f31deffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T17:01:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f31deffd640 1 -- 172.21.15.26:0/3692753392 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f805c170 con 0x55d34b08e990 2023-12-19T17:01:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f31deffd640 1 -- 172.21.15.26:0/3692753392 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f31f8067dc0 con 0x55d34b08e990 2023-12-19T17:01:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 -- 172.21.15.26:0/3692753392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d34ae57e00 con 0x55d34b08e990 2023-12-19T17:01:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f31deffd640 1 -- 172.21.15.26:0/3692753392 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31f804ff80 con 0x55d34b08e990 2023-12-19T17:01:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 10 client.8646 fetch_fsmap learned FSMap version 65 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 -- 172.21.15.26:0/3692753392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55d34ab8ecc0 con 0x55d34b08e990 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f31deffd640 1 -- 172.21.15.26:0/3692753392 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d34ae57e00 con 0x55d34b08e990 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f31deffd640 1 -- 172.21.15.26:0/3692753392 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 65) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f31f80500e0 con 0x55d34b08e990 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 10 client.8646 fetch_fsmap finished waiting for FSMap version 65 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 10 client.8646 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 20 client.8646 populate_metadata read hostname 'smithi026' 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 --2- 172.21.15.26:0/3692753392 >> [v2:172.21.15.26:6834/3823167404,v1:172.21.15.26:6835/3823167404] conn(0x55d34b0cfb10 0x55d34b0cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 4 client.8646 mds_command: new command op to 8487 tid=0[{"prefix": "status"}] 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 -- 172.21.15.26:0/3692753392 --> [v2:172.21.15.26:6834/3823167404,v1:172.21.15.26:6835/3823167404] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d34ae8b160 con 0x55d34b0cfb10 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f31fffff640 1 --2- 172.21.15.26:0/3692753392 >> [v2:172.21.15.26:6834/3823167404,v1:172.21.15.26:6835/3823167404] conn(0x55d34b0cfb10 0x55d34b0cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f31fffff640 1 --2- 172.21.15.26:0/3692753392 >> [v2:172.21.15.26:6834/3823167404,v1:172.21.15.26:6835/3823167404] conn(0x55d34b0cfb10 0x55d34b0cfef0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4091ec665627d050 server_cookie=47e8d14e6c68fc8d in_seq=0 out_seq=0 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f31deffd640 10 client.8646 ms_handle_connect on v2:172.21.15.26:6834/3823167404 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f31deffd640 1 -- 172.21.15.26:0/3692753392 <== mds.0 v2:172.21.15.26:6834/3823167404 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55d34ae8b160 con 0x55d34b0cfb10 2023-12-19T17:01:27.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f31deffd640 10 client.8646 handle_command_reply: tid=0 2023-12-19T17:01:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 client.8646 shutdown 2023-12-19T17:01:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 -- 172.21.15.26:0/3692753392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 msgr2=0x55d34b0cf5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.081+0000 7f3209a8d000 1 --2- 172.21.15.26:0/3692753392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b0cf5d0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f31f8053e90 tx=0x7f31f8001260 comp rx=0 tx=0).stop 2023-12-19T17:01:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3209a8d000 1 -- 172.21.15.26:0/3692753392 shutdown_connections 2023-12-19T17:01:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3209a8d000 1 --2- 172.21.15.26:0/3692753392 >> [v2:172.21.15.26:6834/3823167404,v1:172.21.15.26:6835/3823167404] conn(0x55d34b0cfb10 0x55d34b0cfef0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3209a8d000 1 --2- 172.21.15.26:0/3692753392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55d34b08e990 0x55d34b0cf5d0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3209a8d000 1 --2- 172.21.15.26:0/3692753392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55d34b08e070 0x55d34b0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f31deffd640 0 client.8646 ms_handle_reset on v2:172.21.15.26:6834/3823167404 2023-12-19T17:01:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3209a8d000 1 --2- 172.21.15.26:0/3692753392 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55d34b0441c0 0x55d34b0ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3209a8d000 1 -- 172.21.15.26:0/3692753392 >> 172.21.15.26:0/3692753392 conn(0x55d34b04a950 msgr2=0x55d34b08b060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3209a8d000 1 -- 172.21.15.26:0/3692753392 shutdown_connections 2023-12-19T17:01:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3209a8d000 1 -- 172.21.15.26:0/3692753392 wait complete. 2023-12-19T17:01:27.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3209a8d000 20 client.8646 trim_cache size 0 max 16384 2023-12-19T17:01:27.096 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T17:01:27.096 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T17:01:27.096 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T17:01:27.096 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 8487, 2023-12-19T17:01:27.096 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T17:01:27.096 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T17:01:27.096 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T17:01:27.096 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.943178896, 2023-12-19T17:01:27.096 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 65, 2023-12-19T17:01:27.096 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 137, 2023-12-19T17:01:27.096 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 137, 2023-12-19T17:01:27.097 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 10.27725429 2023-12-19T17:01:27.097 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T17:01:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3206725640 1 -- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f31ec065b80 msgr2=0x7f31ec068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3206725640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f31ec065b80 0x7f31ec068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f31f0000f70 tx=0x7f31f0009040 comp rx=0 tx=0).stop 2023-12-19T17:01:27.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3206725640 1 -- 172.21.15.26:0/3541200758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015f370 msgr2=0x7f320006be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3206725640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015f370 0x7f320006be60 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f31f8001570 tx=0x7f31f800cd20 comp rx=0 tx=0).stop 2023-12-19T17:01:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3206725640 1 -- 172.21.15.26:0/3541200758 shutdown_connections 2023-12-19T17:01:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3206725640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f320015f370 0x7f320006be60 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3206725640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f31ec065b80 0x7f31ec068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3206725640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3200155030 0x7f320006b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3206725640 1 --2- 172.21.15.26:0/3541200758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32000ce740 0x7f32001f65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3206725640 1 -- 172.21.15.26:0/3541200758 >> 172.21.15.26:0/3541200758 conn(0x7f32000658e0 msgr2=0x7f320015bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3206725640 1 -- 172.21.15.26:0/3541200758 shutdown_connections 2023-12-19T17:01:27.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.085+0000 7f3206725640 1 -- 172.21.15.26:0/3541200758 wait complete. 2023-12-19T17:01:27.107 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T17:01:27.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.245+0000 7f9ab2cf2640 1 Processor -- start 2023-12-19T17:01:27.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.245+0000 7f9ab2cf2640 1 -- start start 2023-12-19T17:01:27.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9ab2cf2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9aac06cb30 0x7f9aac0cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9ab2cf2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aac15ddc0 con 0x7f9aac15d2b0 2023-12-19T17:01:27.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9ab2cf2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aac15e0a0 con 0x7f9aac0b34e0 2023-12-19T17:01:27.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9ab2cf2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0cb450 con 0x7f9aac06cb30 2023-12-19T17:01:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aabfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9aac15d2b0 0x7f9aac0b2dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57336/0 (socket says 172.21.15.26:57336) 2023-12-19T17:01:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aabfff640 1 -- 172.21.15.26:0/3323649485 learned_addr learned my addr 172.21.15.26:0/3323649485 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9ab0a8e640 1 --2- 172.21.15.26:0/3323649485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9aac06cb30 0x7f9aac0cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408215731 0 0) 0x7f9aac15ddc0 con 0x7f9aac15d2b0 2023-12-19T17:01:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a9c003530 con 0x7f9aac15d2b0 2023-12-19T17:01:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1600056741 0 0) 0x7f9a9c003530 con 0x7f9aac15d2b0 2023-12-19T17:01:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9aac06cb30 msgr2=0x7f9aac0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 --2- 172.21.15.26:0/3323649485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9aac06cb30 0x7f9aac0cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 >> v1:172.21.15.160:6789/0 conn(0x7f9aac0b34e0 legacy=0x7f9aac0ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac0cb730 con 0x7f9aac15d2b0 2023-12-19T17:01:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f9a98001680 con 0x7f9aac15d2b0 2023-12-19T17:01:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 >> v1:172.21.15.26:6789/0 conn(0x7f9aac15d2b0 legacy=0x7f9aac0b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 --2- 172.21.15.26:0/3323649485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9aac06cb30 0x7f9a9c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 --2- 172.21.15.26:0/3323649485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a9c003ef0 0x7f9a9c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 --2- 172.21.15.26:0/3323649485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a9c004d10 0x7f9a9c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0cb450 con 0x7f9a9c003ef0 2023-12-19T17:01:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c00b9f0 con 0x7f9a9c004d10 2023-12-19T17:01:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c00bbb0 con 0x7f9aac06cb30 2023-12-19T17:01:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aabfff640 1 --2- 172.21.15.26:0/3323649485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a9c003ef0 0x7f9a9c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9ab0a8e640 1 --2- 172.21.15.26:0/3323649485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9aac06cb30 0x7f9a9c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9ab128f640 1 --2- 172.21.15.26:0/3323649485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a9c004d10 0x7f9a9c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aabfff640 1 -- 172.21.15.26:0/3323649485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9aac06cb30 msgr2=0x7f9a9c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:27.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aabfff640 1 --2- 172.21.15.26:0/3323649485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9aac06cb30 0x7f9a9c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aabfff640 1 -- 172.21.15.26:0/3323649485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a9c004d10 msgr2=0x7f9a9c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:27.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aabfff640 1 --2- 172.21.15.26:0/3323649485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a9c004d10 0x7f9a9c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.249+0000 7f9aabfff640 1 -- 172.21.15.26:0/3323649485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c003530 con 0x7f9a9c003ef0 2023-12-19T17:01:27.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.253+0000 7f9aabfff640 1 --2- 172.21.15.26:0/3323649485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a9c003ef0 0x7f9a9c0047d0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800c210 tx=0x7f9a9801cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2497597209ed3c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:27.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.253+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98005ae0 con 0x7f9a9c003ef0 2023-12-19T17:01:27.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.253+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a98005c60 con 0x7f9a9c003ef0 2023-12-19T17:01:27.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.253+0000 7f9aab7fe640 1 -- 172.21.15.26:0/3323649485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a980a5ad0 con 0x7f9a9c003ef0 2023-12-19T17:01:27.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.253+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3323649485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a9c003ef0 msgr2=0x7f9a9c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.253+0000 7f9ab2cf2640 1 --2- 172.21.15.26:0/3323649485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a9c003ef0 0x7f9a9c0047d0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800c210 tx=0x7f9a9801cb30 comp rx=0 tx=0).stop 2023-12-19T17:01:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.253+0000 7f9ab128f640 1 -- 172.21.15.26:0/3323649485 reap_dead start 2023-12-19T17:01:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.253+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3323649485 shutdown_connections 2023-12-19T17:01:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.253+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3323649485 >> 172.21.15.26:0/3323649485 conn(0x7f9aac0658e0 msgr2=0x7f9aac065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.253+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3323649485 shutdown_connections 2023-12-19T17:01:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.253+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3323649485 wait complete. 2023-12-19T17:01:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab2cf2640 1 Processor -- start 2023-12-19T17:01:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab2cf2640 1 -- start start 2023-12-19T17:01:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab2cf2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a9c003ef0 0x7f9aac141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab2cf2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9aac0ceee0 0x7f9aac0b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab2cf2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9aac0af150 0x7f9aac0af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab2cf2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac143030 con 0x7f9aac0af150 2023-12-19T17:01:27.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab2cf2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac143310 con 0x7f9aac0ceee0 2023-12-19T17:01:27.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab2cf2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac1435f0 con 0x7f9a9c003ef0 2023-12-19T17:01:27.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab128f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9aac0af150 0x7f9aac0af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab128f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9aac0af150 0x7f9aac0af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34348/0 (socket says 172.21.15.26:34348) 2023-12-19T17:01:27.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab128f640 1 -- 172.21.15.26:0/3527674453 learned_addr learned my addr 172.21.15.26:0/3527674453 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:27.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9aabfff640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9aac0ceee0 0x7f9aac0b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab0a8e640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a9c003ef0 0x7f9aac141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab128f640 1 -- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a9c003ef0 msgr2=0x7f9aac141520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab128f640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a9c003ef0 0x7f9aac141520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab128f640 1 -- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9aac0ceee0 msgr2=0x7f9aac0b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab128f640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9aac0ceee0 0x7f9aac0b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab128f640 1 -- 172.21.15.26:0/3527674453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac1438d0 con 0x7f9aac0af150 2023-12-19T17:01:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab128f640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9aac0af150 0x7f9aac0af5d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9aac146b30 tx=0x7f9aa400eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9aa97fa640 1 -- 172.21.15.26:0/3527674453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4003980 con 0x7f9aac0af150 2023-12-19T17:01:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9aa97fa640 1 -- 172.21.15.26:0/3527674453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa4003ae0 con 0x7f9aac0af150 2023-12-19T17:01:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9aa97fa640 1 -- 172.21.15.26:0/3527674453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4003e90 con 0x7f9aac0af150 2023-12-19T17:01:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3527674453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aac143bb0 con 0x7f9aac0af150 2023-12-19T17:01:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.257+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3527674453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aac141a60 con 0x7f9aac0af150 2023-12-19T17:01:27.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.261+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3527674453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aac074270 con 0x7f9aac0af150 2023-12-19T17:01:27.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.261+0000 7f9aa97fa640 1 -- 172.21.15.26:0/3527674453 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9aa40050d0 con 0x7f9aac0af150 2023-12-19T17:01:27.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.261+0000 7f9aa97fa640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9a90065b80 0x7f9a90068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:27.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.261+0000 7f9aa97fa640 1 -- 172.21.15.26:0/3527674453 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f9aa4083550 con 0x7f9aac0af150 2023-12-19T17:01:27.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.261+0000 7f9ab0a8e640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9a90065b80 0x7f9a90068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:27.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.261+0000 7f9ab0a8e640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9a90065b80 0x7f9a90068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009a30 tx=0x7f9aa0008040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:27.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.265+0000 7f9aa97fa640 1 -- 172.21.15.26:0/3527674453 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9aa4050b20 con 0x7f9aac0af150 2023-12-19T17:01:27.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3527674453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9aac0acf60 con 0x7f9aac0af150 2023-12-19T17:01:27.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9aa97fa640 1 -- 172.21.15.26:0/3527674453 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9aa4054140 con 0x7f9aac0af150 2023-12-19T17:01:27.465 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T17:01:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9a90065b80 msgr2=0x7f9a90068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9ab2cf2640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9a90065b80 0x7f9a90068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009a30 tx=0x7f9aa0008040 comp rx=0 tx=0).stop 2023-12-19T17:01:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3527674453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9aac0af150 msgr2=0x7f9aac0af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9ab2cf2640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9aac0af150 0x7f9aac0af5d0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9aac146b30 tx=0x7f9aa400eb70 comp rx=0 tx=0).stop 2023-12-19T17:01:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3527674453 shutdown_connections 2023-12-19T17:01:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9ab2cf2640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9aac0af150 0x7f9aac0af5d0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9ab2cf2640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9aac0ceee0 0x7f9aac0b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9ab2cf2640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9a90065b80 0x7f9a90068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9ab2cf2640 1 --2- 172.21.15.26:0/3527674453 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a9c003ef0 0x7f9aac141520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3527674453 >> 172.21.15.26:0/3527674453 conn(0x7f9aac0658e0 msgr2=0x7f9aac160c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3527674453 shutdown_connections 2023-12-19T17:01:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:27.461+0000 7f9ab2cf2640 1 -- 172.21.15.26:0/3527674453 wait complete. 2023-12-19T17:01:27.487 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-19T17:01:27.487 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T17:01:27.487 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T17:01:27.487 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T17:01:27.487 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T17:01:27.487 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-19T17:01:27.487 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-19T17:01:27.487 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T17:01:27.487 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T17:01:27.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:27.488 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T17:01:27.498 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T17:01:27.498 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T17:01:27.498 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T17:01:27.498 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:01:27.498 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T17:01:27.498 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:01:27.498 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T17:01:27.498 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T17:01:27.498 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T17:01:27.498 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T17:01:27.498 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T17:01:27.498 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T17:01:27.499 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T17:01:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T17:01:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T17:01:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T17:01:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 41253sec preferred_lft 41253sec 2023-12-19T17:01:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T17:01:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:01:27.499 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T17:01:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T17:01:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T17:01:27.499 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T17:01:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T17:01:27.499 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T17:01:27.500 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-19T17:01:27.501 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:01:27.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:27.502 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T17:01:27.567 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T17:01:27.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:27.569 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T17:01:27.625 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T17:01:27.625 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T17:01:27.625 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T17:01:27.625 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T17:01:27.625 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T17:01:27.625 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T17:01:27.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:27.626 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T17:01:27.626 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T17:01:27.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T17:01:27.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T17:01:27.626 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T17:01:27.626 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T17:01:27.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:27.724 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T17:01:27.734 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T17:01:27.734 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T17:01:27.735 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T17:01:27.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:27.735 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T17:01:27.797 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T17:01:27.797 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T17:01:27.852 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T17:01:27.852 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T17:01:27.853 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T17:01:27.904 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T17:01:27.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:27.904 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T17:01:28.001 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T17:01:28.002 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T17:01:28.012 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T17:01:28.012 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T17:01:28.012 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T17:01:28.012 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T17:01:28.012 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T17:01:28.016 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T17:01:28.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:28.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T17:01:28.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:28.037 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T17:01:28.097 INFO:teuthology.orchestra.run.smithi026.stdout:client8655 2023-12-19T17:01:28.097 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:01:28.097 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:01:28.152 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-19T17:01:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f5445466640 1 Processor -- start 2023-12-19T17:01:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f5445466640 1 -- start start 2023-12-19T17:01:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f5445466640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54400cee50 0x7f544015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f5445466640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544006c970 con 0x7f5440155030 2023-12-19T17:01:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f5445466640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544015b1f0 con 0x7f544015f370 2023-12-19T17:01:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f5445466640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544006c790 con 0x7f54400cee50 2023-12-19T17:01:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5440155030 0x7f544006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57342/0 (socket says 172.21.15.26:57342) 2023-12-19T17:01:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543effd640 1 -- 172.21.15.26:0/3369632079 learned_addr learned my addr 172.21.15.26:0/3369632079 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543f7fe640 1 --2- 172.21.15.26:0/3369632079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54400cee50 0x7f544015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366029566 0 0) 0x7f544006c970 con 0x7f5440155030 2023-12-19T17:01:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f542c003530 con 0x7f5440155030 2023-12-19T17:01:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 172993025 0 0) 0x7f544015b1f0 con 0x7f544015f370 2023-12-19T17:01:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f544006c970 con 0x7f544015f370 2023-12-19T17:01:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1547510445 0 0) 0x7f542c003530 con 0x7f5440155030 2023-12-19T17:01:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54400cee50 msgr2=0x7f544015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 --2- 172.21.15.26:0/3369632079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54400cee50 0x7f544015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 >> v1:172.21.15.160:6789/0 conn(0x7f544015f370 legacy=0x7f54400ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544015b4d0 con 0x7f5440155030 2023-12-19T17:01:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f5430001680 con 0x7f5440155030 2023-12-19T17:01:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 >> v1:172.21.15.26:6789/0 conn(0x7f5440155030 legacy=0x7f544006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 --2- 172.21.15.26:0/3369632079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542c003b40 0x7f542c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 --2- 172.21.15.26:0/3369632079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f542c0047d0 0x7f542c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 --2- 172.21.15.26:0/3369632079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54400cee50 0x7f542c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544006c790 con 0x7f542c003b40 2023-12-19T17:01:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c00ba70 con 0x7f542c0047d0 2023-12-19T17:01:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c00bc30 con 0x7f54400cee50 2023-12-19T17:01:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543effd640 1 --2- 172.21.15.26:0/3369632079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542c003b40 0x7f542c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f5437fff640 1 --2- 172.21.15.26:0/3369632079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f542c0047d0 0x7f542c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543f7fe640 1 --2- 172.21.15.26:0/3369632079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54400cee50 0x7f542c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543effd640 1 -- 172.21.15.26:0/3369632079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54400cee50 msgr2=0x7f542c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543effd640 1 --2- 172.21.15.26:0/3369632079 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54400cee50 0x7f542c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543effd640 1 -- 172.21.15.26:0/3369632079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f542c0047d0 msgr2=0x7f542c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543effd640 1 --2- 172.21.15.26:0/3369632079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f542c0047d0 0x7f542c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.337+0000 7f543effd640 1 -- 172.21.15.26:0/3369632079 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c003530 con 0x7f542c003b40 2023-12-19T17:01:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.341+0000 7f543effd640 1 --2- 172.21.15.26:0/3369632079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542c003b40 0x7f542c003f40 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f5430007fb0 tx=0x7f543001cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fcccc2c5dd59a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.341+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430005ae0 con 0x7f542c003b40 2023-12-19T17:01:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.341+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5430005c60 con 0x7f542c003b40 2023-12-19T17:01:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.341+0000 7f543e7fc640 1 -- 172.21.15.26:0/3369632079 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430007070 con 0x7f542c003b40 2023-12-19T17:01:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.341+0000 7f5445466640 1 -- 172.21.15.26:0/3369632079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542c003b40 msgr2=0x7f542c003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.341+0000 7f5445466640 1 --2- 172.21.15.26:0/3369632079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f542c003b40 0x7f542c003f40 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f5430007fb0 tx=0x7f543001cad0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.341+0000 7f543f7fe640 1 -- 172.21.15.26:0/3369632079 reap_dead start 2023-12-19T17:01:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.341+0000 7f5445466640 1 -- 172.21.15.26:0/3369632079 shutdown_connections 2023-12-19T17:01:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.341+0000 7f5445466640 1 -- 172.21.15.26:0/3369632079 >> 172.21.15.26:0/3369632079 conn(0x7f54400658e0 msgr2=0x7f5440065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.341+0000 7f5445466640 1 -- 172.21.15.26:0/3369632079 shutdown_connections 2023-12-19T17:01:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.341+0000 7f5445466640 1 -- 172.21.15.26:0/3369632079 wait complete. 2023-12-19T17:01:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f5445466640 1 Processor -- start 2023-12-19T17:01:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f5445466640 1 -- start start 2023-12-19T17:01:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f5445466640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f542c003b40 0x7f5440144160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f5445466640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400ac850 0x7f54400accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f5445466640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5440148090 0x7f5440148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f5445466640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54400ad210 con 0x7f5440148090 2023-12-19T17:01:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f5445466640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5440148a50 con 0x7f54400ac850 2023-12-19T17:01:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f5445466640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54401446a0 con 0x7f542c003b40 2023-12-19T17:01:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f543f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5440148090 0x7f5440148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f543f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5440148090 0x7f5440148510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34366/0 (socket says 172.21.15.26:34366) 2023-12-19T17:01:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f543f7fe640 1 -- 172.21.15.26:0/1347870101 learned_addr learned my addr 172.21.15.26:0/1347870101 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f5437fff640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400ac850 0x7f54400accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f543effd640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f542c003b40 0x7f5440144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f543f7fe640 1 -- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f542c003b40 msgr2=0x7f5440144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f543f7fe640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f542c003b40 0x7f5440144160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f543f7fe640 1 -- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400ac850 msgr2=0x7f54400accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f543f7fe640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400ac850 0x7f54400accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f543f7fe640 1 -- 172.21.15.26:0/1347870101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54401f97e0 con 0x7f5440148090 2023-12-19T17:01:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f543f7fe640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5440148090 0x7f5440148510 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f544006d880 tx=0x7f54380115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f54377fe640 1 -- 172.21.15.26:0/1347870101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543800e070 con 0x7f5440148090 2023-12-19T17:01:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f54377fe640 1 -- 172.21.15.26:0/1347870101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5438011ed0 con 0x7f5440148090 2023-12-19T17:01:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f54377fe640 1 -- 172.21.15.26:0/1347870101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54380039a0 con 0x7f5440148090 2023-12-19T17:01:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f5445466640 1 -- 172.21.15.26:0/1347870101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54401f8a20 con 0x7f5440148090 2023-12-19T17:01:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.345+0000 7f5445466640 1 -- 172.21.15.26:0/1347870101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54401f8fe0 con 0x7f5440148090 2023-12-19T17:01:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.349+0000 7f5445466640 1 -- 172.21.15.26:0/1347870101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5440074270 con 0x7f5440148090 2023-12-19T17:01:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.349+0000 7f54377fe640 1 -- 172.21.15.26:0/1347870101 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5438003b00 con 0x7f5440148090 2023-12-19T17:01:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.349+0000 7f54377fe640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5420065b80 0x7f5420068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.349+0000 7f54377fe640 1 -- 172.21.15.26:0/1347870101 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f5438083e40 con 0x7f5440148090 2023-12-19T17:01:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.349+0000 7f543effd640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5420065b80 0x7f5420068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.349+0000 7f543effd640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5420065b80 0x7f5420068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54300ab000 tx=0x7f54300a7070 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.353+0000 7f54377fe640 1 -- 172.21.15.26:0/1347870101 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f54380512f0 con 0x7f5440148090 2023-12-19T17:01:28.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.533+0000 7f5445466640 1 -- 172.21.15.26:0/1347870101 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f54400adca0 con 0x7f5420065b80 2023-12-19T17:01:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.537+0000 7f54377fe640 1 -- 172.21.15.26:0/1347870101 <== mgr.7082 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f54300b5040 con 0x7f5420065b80 2023-12-19T17:01:28.541 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T17:01:28.541 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:01:28.541 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2023-12-19T17:01:28.541 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:01:28.541 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T17:01:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.537+0000 7f5445466640 1 -- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5420065b80 msgr2=0x7f5420068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.537+0000 7f5445466640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5420065b80 0x7f5420068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f54300ab000 tx=0x7f54300a7070 comp rx=0 tx=0).stop 2023-12-19T17:01:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.537+0000 7f5445466640 1 -- 172.21.15.26:0/1347870101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5440148090 msgr2=0x7f5440148510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.537+0000 7f5445466640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5440148090 0x7f5440148510 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f544006d880 tx=0x7f54380115a0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.537+0000 7f5445466640 1 -- 172.21.15.26:0/1347870101 shutdown_connections 2023-12-19T17:01:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.537+0000 7f5445466640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5440148090 0x7f5440148510 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.537+0000 7f5445466640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54400ac850 0x7f54400accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.537+0000 7f5445466640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5420065b80 0x7f5420068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.537+0000 7f5445466640 1 --2- 172.21.15.26:0/1347870101 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f542c003b40 0x7f5440144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.537+0000 7f5445466640 1 -- 172.21.15.26:0/1347870101 >> 172.21.15.26:0/1347870101 conn(0x7f54400658e0 msgr2=0x7f544014e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.537+0000 7f5445466640 1 -- 172.21.15.26:0/1347870101 shutdown_connections 2023-12-19T17:01:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.537+0000 7f5445466640 1 -- 172.21.15.26:0/1347870101 wait complete. 2023-12-19T17:01:28.563 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-19T17:01:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.705+0000 7f843c0b6640 1 Processor -- start 2023-12-19T17:01:28.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.705+0000 7f843c0b6640 1 -- start start 2023-12-19T17:01:28.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.709+0000 7f843c0b6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f84340ca640 0x7f8434071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.709+0000 7f843c0b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84341615b0 con 0x7f84340adc60 2023-12-19T17:01:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.709+0000 7f843c0b6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84340c8d90 con 0x7f843406d1e0 2023-12-19T17:01:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.709+0000 7f843c0b6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84340caa90 con 0x7f84340ca640 2023-12-19T17:01:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.709+0000 7f8439e52640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f84340adc60 0x7f843406cad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57354/0 (socket says 172.21.15.26:57354) 2023-12-19T17:01:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.709+0000 7f8439e52640 1 -- 172.21.15.26:0/3780036463 learned_addr learned my addr 172.21.15.26:0/3780036463 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.709+0000 7f843a653640 1 --2- 172.21.15.26:0/3780036463 >> v2:172.21.15.26:3300/0 conn(0x7f84340ca640 0x7f8434071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.709+0000 7f8438e50640 1 -- 172.21.15.26:0/3780036463 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311011382 0 0) 0x7f84341615b0 con 0x7f84340adc60 2023-12-19T17:01:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.709+0000 7f8438e50640 1 -- 172.21.15.26:0/3780036463 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8420003530 con 0x7f84340adc60 2023-12-19T17:01:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.709+0000 7f843a653640 1 -- 172.21.15.26:0/3780036463 >> v1:172.21.15.160:6789/0 conn(0x7f843406d1e0 legacy=0x7f843406f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.709+0000 7f843a653640 1 -- 172.21.15.26:0/3780036463 >> v1:172.21.15.26:6789/0 conn(0x7f84340adc60 legacy=0x7f843406cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:28.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.709+0000 7f843a653640 1 -- 172.21.15.26:0/3780036463 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84340c9070 con 0x7f84340ca640 2023-12-19T17:01:28.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.713+0000 7f843a653640 1 --2- 172.21.15.26:0/3780036463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84340ca640 0x7f8434071100 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f842c009d90 tx=0x7f842c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f13aabf4c112473 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:28.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.713+0000 7f8438e50640 1 -- 172.21.15.26:0/3780036463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0a5950 con 0x7f84340ca640 2023-12-19T17:01:28.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.713+0000 7f8438e50640 1 -- 172.21.15.26:0/3780036463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c0a5ab0 con 0x7f84340ca640 2023-12-19T17:01:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.713+0000 7f8438e50640 1 -- 172.21.15.26:0/3780036463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c0a5e40 con 0x7f84340ca640 2023-12-19T17:01:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.713+0000 7f843c0b6640 1 -- 172.21.15.26:0/3780036463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84340ca640 msgr2=0x7f8434071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.713+0000 7f843c0b6640 1 --2- 172.21.15.26:0/3780036463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84340ca640 0x7f8434071100 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f842c009d90 tx=0x7f842c01c890 comp rx=0 tx=0).stop 2023-12-19T17:01:28.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.713+0000 7f843c0b6640 1 -- 172.21.15.26:0/3780036463 shutdown_connections 2023-12-19T17:01:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.713+0000 7f843c0b6640 1 --2- 172.21.15.26:0/3780036463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84340ca640 0x7f8434071100 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.713+0000 7f843c0b6640 1 -- 172.21.15.26:0/3780036463 >> 172.21.15.26:0/3780036463 conn(0x7f8434066160 msgr2=0x7f84340b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.713+0000 7f843c0b6640 1 -- 172.21.15.26:0/3780036463 shutdown_connections 2023-12-19T17:01:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.713+0000 7f843c0b6640 1 -- 172.21.15.26:0/3780036463 wait complete. 2023-12-19T17:01:28.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.713+0000 7f843c0b6640 1 Processor -- start 2023-12-19T17:01:28.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.713+0000 7f843c0b6640 1 -- start start 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f843c0b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843406d1e0 0x7f843415b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f843c0b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84340adc60 0x7f84340b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f843c0b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84340ca640 0x7f84340cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f843c0b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843415b870 con 0x7f843406d1e0 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f843c0b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8434148190 con 0x7f84340ca640 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f843c0b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84340b3580 con 0x7f84340adc60 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f8439e52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843406d1e0 0x7f843415b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f8439e52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843406d1e0 0x7f843415b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34378/0 (socket says 172.21.15.26:34378) 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f8439e52640 1 -- 172.21.15.26:0/429515988 learned_addr learned my addr 172.21.15.26:0/429515988 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f8439651640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84340adc60 0x7f84340b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f8439e52640 1 -- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84340adc60 msgr2=0x7f84340b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f8439e52640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84340adc60 0x7f84340b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f843a653640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84340ca640 0x7f84340cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f8439e52640 1 -- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84340ca640 msgr2=0x7f84340cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:28.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f8439e52640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84340ca640 0x7f84340cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:28.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f8439e52640 1 -- 172.21.15.26:0/429515988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c009a70 con 0x7f843406d1e0 2023-12-19T17:01:28.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f8439651640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84340adc60 0x7f84340b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:28.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f843a653640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84340ca640 0x7f84340cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:28.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f8439e52640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843406d1e0 0x7f843415b330 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f843000c4d0 tx=0x7f843000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:28.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f842a7fc640 1 -- 172.21.15.26:0/429515988 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430005cf0 con 0x7f843406d1e0 2023-12-19T17:01:28.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f842a7fc640 1 -- 172.21.15.26:0/429515988 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8430005e50 con 0x7f843406d1e0 2023-12-19T17:01:28.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f843c0b6640 1 -- 172.21.15.26:0/429515988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84340b3860 con 0x7f843406d1e0 2023-12-19T17:01:28.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f842a7fc640 1 -- 172.21.15.26:0/429515988 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430007270 con 0x7f843406d1e0 2023-12-19T17:01:28.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f843c0b6640 1 -- 172.21.15.26:0/429515988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84340b3270 con 0x7f843406d1e0 2023-12-19T17:01:28.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f843c0b6640 1 -- 172.21.15.26:0/429515988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84340b3d80 con 0x7f843406d1e0 2023-12-19T17:01:28.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.717+0000 7f842a7fc640 1 -- 172.21.15.26:0/429515988 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 68) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f84300073d0 con 0x7f843406d1e0 2023-12-19T17:01:28.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.721+0000 7f842a7fc640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8418065b80 0x7f8418068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:28.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.721+0000 7f842a7fc640 1 -- 172.21.15.26:0/429515988 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f8430082b70 con 0x7f843406d1e0 2023-12-19T17:01:28.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.721+0000 7f8439651640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8418065b80 0x7f8418068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:28.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.721+0000 7f8439651640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8418065b80 0x7f8418068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f84340b0410 tx=0x7f8424009040 comp rx=0 tx=0).ready entity=mgr.7082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:28.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.725+0000 7f842a7fc640 1 -- 172.21.15.26:0/429515988 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8430050150 con 0x7f843406d1e0 2023-12-19T17:01:28.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:28.921+0000 7f843c0b6640 1 -- 172.21.15.26:0/429515988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f8434074270 con 0x7f843406d1e0 2023-12-19T17:01:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.193+0000 7f842a7fc640 1 -- 172.21.15.26:0/429515988 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 69) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f8430050520 con 0x7f843406d1e0 2023-12-19T17:01:29.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.197+0000 7f842a7fc640 1 -- 172.21.15.26:0/429515988 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v69) v1 ==== 92+0+0 (secure 0 0 0) 0x7f843006ca20 con 0x7f843406d1e0 2023-12-19T17:01:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.197+0000 7f843c0b6640 1 -- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8418065b80 msgr2=0x7f8418068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.197+0000 7f843c0b6640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8418065b80 0x7f8418068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f84340b0410 tx=0x7f8424009040 comp rx=0 tx=0).stop 2023-12-19T17:01:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.197+0000 7f843c0b6640 1 -- 172.21.15.26:0/429515988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843406d1e0 msgr2=0x7f843415b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.197+0000 7f843c0b6640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843406d1e0 0x7f843415b330 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f843000c4d0 tx=0x7f843000c990 comp rx=0 tx=0).stop 2023-12-19T17:01:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.197+0000 7f843c0b6640 1 -- 172.21.15.26:0/429515988 shutdown_connections 2023-12-19T17:01:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.197+0000 7f843c0b6640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84340ca640 0x7f84340cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.197+0000 7f843c0b6640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8418065b80 0x7f8418068040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.197+0000 7f843c0b6640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f84340adc60 0x7f84340b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.197+0000 7f843c0b6640 1 --2- 172.21.15.26:0/429515988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f843406d1e0 0x7f843415b330 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.197+0000 7f843c0b6640 1 -- 172.21.15.26:0/429515988 >> 172.21.15.26:0/429515988 conn(0x7f8434066160 msgr2=0x7f8434070bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.197+0000 7f843c0b6640 1 -- 172.21.15.26:0/429515988 shutdown_connections 2023-12-19T17:01:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.197+0000 7f843c0b6640 1 -- 172.21.15.26:0/429515988 wait complete. 2023-12-19T17:01:29.224 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-19T17:01:29.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.369+0000 7f48ff960640 1 Processor -- start 2023-12-19T17:01:29.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.369+0000 7f48ff960640 1 -- start start 2023-12-19T17:01:29.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48ff960640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48f8159820 0x7f48f815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:29.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48ff960640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f48f8159c00 0x7f48f8070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:29.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48ff960640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f806fb60 con 0x7f48f806f6c0 2023-12-19T17:01:29.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48ff960640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f815e470 con 0x7f48f8159c00 2023-12-19T17:01:29.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48ff960640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f806fe40 con 0x7f48f8159820 2023-12-19T17:01:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48fdefd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f48f8159c00 0x7f48f8070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48fcefb640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f48f806f6c0 0x7f48f80c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57356/0 (socket says 172.21.15.26:57356) 2023-12-19T17:01:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48fdefd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f48f8159c00 0x7f48f8070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34402/0 (socket says 172.21.15.26:34402) 2023-12-19T17:01:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48fcefb640 1 -- 172.21.15.26:0/1646951775 learned_addr learned my addr 172.21.15.26:0/1646951775 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:29.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48fd6fc640 1 --2- 172.21.15.26:0/1646951775 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48f8159820 0x7f48f815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:29.378 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:29.373+0000 7fa04464a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:01:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48fdefd640 1 -- 172.21.15.26:0/1646951775 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48f8159820 msgr2=0x7f48f815df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48fdefd640 1 --2- 172.21.15.26:0/1646951775 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48f8159820 0x7f48f815df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48fdefd640 1 -- 172.21.15.26:0/1646951775 >> v1:172.21.15.26:6789/0 conn(0x7f48f806f6c0 legacy=0x7f48f80c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48fdefd640 1 -- 172.21.15.26:0/1646951775 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f8070120 con 0x7f48f8159c00 2023-12-19T17:01:29.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.373+0000 7f48effff640 1 -- 172.21.15.26:0/1646951775 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227449591 0 0) 0x7f48f806fb60 con 0x7f48f806f6c0 2023-12-19T17:01:29.379 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:29.375+0000 7f039a84a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:01:29.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.377+0000 7f48fdefd640 1 --2- 172.21.15.26:0/1646951775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f8159c00 0x7f48f8070c00 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f48f000bfa0 tx=0x7f48f001eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249edfc2626d9cf0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.377+0000 7f48effff640 1 -- 172.21.15.26:0/1646951775 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f000b2d0 con 0x7f48f8159c00 2023-12-19T17:01:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.377+0000 7f48effff640 1 -- 172.21.15.26:0/1646951775 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f00a7900 con 0x7f48f8159c00 2023-12-19T17:01:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.377+0000 7f48effff640 1 -- 172.21.15.26:0/1646951775 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f00a7c90 con 0x7f48f8159c00 2023-12-19T17:01:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.377+0000 7f48ff960640 1 -- 172.21.15.26:0/1646951775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f8159c00 msgr2=0x7f48f8070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.377+0000 7f48ff960640 1 --2- 172.21.15.26:0/1646951775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f8159c00 0x7f48f8070c00 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f48f000bfa0 tx=0x7f48f001eb20 comp rx=0 tx=0).stop 2023-12-19T17:01:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.377+0000 7f48ff960640 1 -- 172.21.15.26:0/1646951775 shutdown_connections 2023-12-19T17:01:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.377+0000 7f48ff960640 1 --2- 172.21.15.26:0/1646951775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f8159c00 0x7f48f8070c00 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.377+0000 7f48ff960640 1 --2- 172.21.15.26:0/1646951775 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48f8159820 0x7f48f815df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.377+0000 7f48ff960640 1 -- 172.21.15.26:0/1646951775 >> 172.21.15.26:0/1646951775 conn(0x7f48f80ce730 msgr2=0x7f48f80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.377+0000 7f48ff960640 1 -- 172.21.15.26:0/1646951775 shutdown_connections 2023-12-19T17:01:29.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.377+0000 7f48ff960640 1 -- 172.21.15.26:0/1646951775 wait complete. 2023-12-19T17:01:29.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.377+0000 7f48ff960640 1 Processor -- start 2023-12-19T17:01:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48ff960640 1 -- start start 2023-12-19T17:01:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48ff960640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48f806f6c0 0x7f48f8155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:29.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48ff960640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f8159820 0x7f48f815c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:29.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48ff960640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f8159c00 0x7f48f8152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:29.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48ff960640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f8155e70 con 0x7f48f8159820 2023-12-19T17:01:29.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48ff960640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f815c9a0 con 0x7f48f8159c00 2023-12-19T17:01:29.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48ff960640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f81560a0 con 0x7f48f806f6c0 2023-12-19T17:01:29.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48fcefb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f8159820 0x7f48f815c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:29.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48fcefb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f8159820 0x7f48f815c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34412/0 (socket says 172.21.15.26:34412) 2023-12-19T17:01:29.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48fcefb640 1 -- 172.21.15.26:0/2790267638 learned_addr learned my addr 172.21.15.26:0/2790267638 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:29.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48fdefd640 1 --2- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f8159c00 0x7f48f8152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:29.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48fd6fc640 1 --2- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48f806f6c0 0x7f48f8155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:29.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48fcefb640 1 -- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48f806f6c0 msgr2=0x7f48f8155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:29.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48fcefb640 1 --2- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48f806f6c0 0x7f48f8155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48fcefb640 1 -- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f8159c00 msgr2=0x7f48f8152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:29.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48fcefb640 1 --2- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f8159c00 0x7f48f8152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48fcefb640 1 -- 172.21.15.26:0/2790267638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f000bc80 con 0x7f48f8159820 2023-12-19T17:01:29.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48fcefb640 1 --2- 172.21.15.26:0/2790267638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f8159820 0x7f48f815c460 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f48e800c4b0 tx=0x7f48e800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:29.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48edffb640 1 -- 172.21.15.26:0/2790267638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e80074e0 con 0x7f48f8159820 2023-12-19T17:01:29.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48edffb640 1 -- 172.21.15.26:0/2790267638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e8007660 con 0x7f48f8159820 2023-12-19T17:01:29.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48edffb640 1 -- 172.21.15.26:0/2790267638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8004a30 con 0x7f48f8159820 2023-12-19T17:01:29.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48ff960640 1 -- 172.21.15.26:0/2790267638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f806b880 con 0x7f48f8159820 2023-12-19T17:01:29.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48ff960640 1 -- 172.21.15.26:0/2790267638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f8156660 con 0x7f48f8159820 2023-12-19T17:01:29.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.381+0000 7f48ff960640 1 -- 172.21.15.26:0/2790267638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f8074270 con 0x7f48f8159820 2023-12-19T17:01:29.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.385+0000 7f48edffb640 1 -- 172.21.15.26:0/2790267638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 69) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f48e8004b90 con 0x7f48f8159820 2023-12-19T17:01:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.385+0000 7f48edffb640 1 --2- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f48dc065bd0 0x7f48dc068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.385+0000 7f48edffb640 1 -- 172.21.15.26:0/2790267638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f48e8082900 con 0x7f48f8159820 2023-12-19T17:01:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.385+0000 7f48fd6fc640 1 -- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f48dc065bd0 msgr2=0x7f48dc068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:29.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.385+0000 7f48fd6fc640 1 --2- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f48dc065bd0 0x7f48dc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:01:29.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.389+0000 7f48edffb640 1 -- 172.21.15.26:0/2790267638 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f48e804ffa0 con 0x7f48f8159820 2023-12-19T17:01:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.553+0000 7f48ff960640 1 -- 172.21.15.26:0/2790267638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f48f80b46b0 con 0x7f48f8159820 2023-12-19T17:01:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.557+0000 7f48edffb640 1 -- 172.21.15.26:0/2790267638 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v47)=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f48e8018020 con 0x7f48f8159820 2023-12-19T17:01:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.557+0000 7f48ff960640 1 -- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f48dc065bd0 msgr2=0x7f48dc068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:01:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.557+0000 7f48ff960640 1 --2- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f48dc065bd0 0x7f48dc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.557+0000 7f48ff960640 1 -- 172.21.15.26:0/2790267638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f8159820 msgr2=0x7f48f815c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.557+0000 7f48ff960640 1 --2- 172.21.15.26:0/2790267638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f8159820 0x7f48f815c460 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f48e800c4b0 tx=0x7f48e800c970 comp rx=0 tx=0).stop 2023-12-19T17:01:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.557+0000 7f48ff960640 1 -- 172.21.15.26:0/2790267638 shutdown_connections 2023-12-19T17:01:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.557+0000 7f48ff960640 1 --2- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f8159c00 0x7f48f8152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.557+0000 7f48ff960640 1 --2- 172.21.15.26:0/2790267638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f8159820 0x7f48f815c460 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.557+0000 7f48ff960640 1 --2- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f48dc065bd0 0x7f48dc068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.557+0000 7f48ff960640 1 --2- 172.21.15.26:0/2790267638 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48f806f6c0 0x7f48f8155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.557+0000 7f48ff960640 1 -- 172.21.15.26:0/2790267638 >> 172.21.15.26:0/2790267638 conn(0x7f48f80ce730 msgr2=0x7f48f80cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.557+0000 7f48ff960640 1 -- 172.21.15.26:0/2790267638 shutdown_connections 2023-12-19T17:01:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.557+0000 7f48ff960640 1 -- 172.21.15.26:0/2790267638 wait complete. 2023-12-19T17:01:29.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-19T17:01:29.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbe2a7640 1 Processor -- start 2023-12-19T17:01:29.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbe2a7640 1 -- start start 2023-12-19T17:01:29.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbe2a7640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7bb815cc90 0x7f7bb80cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:29.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbe2a7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bb8160a20 0x7f7bb80d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:29.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbe2a7640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb80cdb20 con 0x7f7bb815d070 2023-12-19T17:01:29.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbe2a7640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb80cde00 con 0x7f7bb815cc90 2023-12-19T17:01:29.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbe2a7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80ce0e0 con 0x7f7bb8160a20 2023-12-19T17:01:29.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbcaa4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7bb815d070 0x7f7bb80cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57368/0 (socket says 172.21.15.26:57368) 2023-12-19T17:01:29.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbcaa4640 1 -- 172.21.15.26:0/2826820707 learned_addr learned my addr 172.21.15.26:0/2826820707 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:29.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbd2a5640 1 --2- 172.21.15.26:0/2826820707 >> v2:172.21.15.160:3300/0 conn(0x7f7bb815cc90 0x7f7bb80cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:29.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbdaa6640 1 --2- 172.21.15.26:0/2826820707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bb8160a20 0x7f7bb80d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7baffff640 1 -- 172.21.15.26:0/2826820707 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079334401 0 0) 0x7f7bb80cdb20 con 0x7f7bb815d070 2023-12-19T17:01:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7baffff640 1 -- 172.21.15.26:0/2826820707 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba4003530 con 0x7f7bb815d070 2023-12-19T17:01:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbd2a5640 1 -- 172.21.15.26:0/2826820707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bb8160a20 msgr2=0x7f7bb80d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbd2a5640 1 --2- 172.21.15.26:0/2826820707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bb8160a20 0x7f7bb80d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbd2a5640 1 -- 172.21.15.26:0/2826820707 >> v1:172.21.15.26:6789/0 conn(0x7f7bb815d070 legacy=0x7f7bb80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbd2a5640 1 -- 172.21.15.26:0/2826820707 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb80ce3c0 con 0x7f7bb815cc90 2023-12-19T17:01:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.729+0000 7f7bbdaa6640 1 --2- 172.21.15.26:0/2826820707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bb8160a20 0x7f7bb80d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:01:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.733+0000 7f7bbd2a5640 1 --2- 172.21.15.26:0/2826820707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bb815cc90 0x7f7bb80cd2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40097c0 tx=0x7f7bb401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6414fd51abb39cf6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.733+0000 7f7baffff640 1 -- 172.21.15.26:0/2826820707 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb40a7070 con 0x7f7bb815cc90 2023-12-19T17:01:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.733+0000 7f7baffff640 1 -- 172.21.15.26:0/2826820707 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb40ab430 con 0x7f7bb815cc90 2023-12-19T17:01:29.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.733+0000 7f7baffff640 1 -- 172.21.15.26:0/2826820707 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb40ab7c0 con 0x7f7bb815cc90 2023-12-19T17:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.733+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/2826820707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bb815cc90 msgr2=0x7f7bb80cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.733+0000 7f7bbe2a7640 1 --2- 172.21.15.26:0/2826820707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bb815cc90 0x7f7bb80cd2f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7bb40097c0 tx=0x7f7bb401c450 comp rx=0 tx=0).stop 2023-12-19T17:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.733+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/2826820707 shutdown_connections 2023-12-19T17:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.733+0000 7f7bbe2a7640 1 --2- 172.21.15.26:0/2826820707 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bb8160a20 0x7f7bb80d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.733+0000 7f7bbe2a7640 1 --2- 172.21.15.26:0/2826820707 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bb815cc90 0x7f7bb80cd2f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.733+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/2826820707 >> 172.21.15.26:0/2826820707 conn(0x7f7bb806a320 msgr2=0x7f7bb80658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.733+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/2826820707 shutdown_connections 2023-12-19T17:01:29.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.733+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/2826820707 wait complete. 2023-12-19T17:01:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbe2a7640 1 Processor -- start 2023-12-19T17:01:29.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbe2a7640 1 -- start start 2023-12-19T17:01:29.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbe2a7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bb815cc90 0x7f7bb814cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbe2a7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb815d070 0x7f7bb80c2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbe2a7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bb8160a20 0x7f7bb815f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbe2a7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80b2440 con 0x7f7bb815d070 2023-12-19T17:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbe2a7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb8155310 con 0x7f7bb8160a20 2023-12-19T17:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbe2a7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb80b2ca0 con 0x7f7bb815cc90 2023-12-19T17:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbcaa4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb815d070 0x7f7bb80c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbd2a5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bb815cc90 0x7f7bb814cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbcaa4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb815d070 0x7f7bb80c2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34416/0 (socket says 172.21.15.26:34416) 2023-12-19T17:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbcaa4640 1 -- 172.21.15.26:0/3841135453 learned_addr learned my addr 172.21.15.26:0/3841135453 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbdaa6640 1 --2- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bb8160a20 0x7f7bb815f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbcaa4640 1 -- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bb815cc90 msgr2=0x7f7bb814cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:29.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbcaa4640 1 --2- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bb815cc90 0x7f7bb814cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbcaa4640 1 -- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bb8160a20 msgr2=0x7f7bb815f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbcaa4640 1 --2- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bb8160a20 0x7f7bb815f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbcaa4640 1 -- 172.21.15.26:0/3841135453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb0009a70 con 0x7f7bb815d070 2023-12-19T17:01:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbd2a5640 1 --2- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bb815cc90 0x7f7bb814cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:01:29.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbcaa4640 1 --2- 172.21.15.26:0/3841135453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb815d070 0x7f7bb80c2880 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800c4d0 tx=0x7f7ba800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7badffb640 1 -- 172.21.15.26:0/3841135453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8005cf0 con 0x7f7bb815d070 2023-12-19T17:01:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7badffb640 1 -- 172.21.15.26:0/3841135453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8005e50 con 0x7f7bb815d070 2023-12-19T17:01:29.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7badffb640 1 -- 172.21.15.26:0/3841135453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80072d0 con 0x7f7bb815d070 2023-12-19T17:01:29.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/3841135453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb4009420 con 0x7f7bb815d070 2023-12-19T17:01:29.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.737+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/3841135453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb80b29c0 con 0x7f7bb815d070 2023-12-19T17:01:29.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.741+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/3841135453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb8155170 con 0x7f7bb815d070 2023-12-19T17:01:29.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.741+0000 7f7badffb640 1 -- 172.21.15.26:0/3841135453 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 69) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f7ba8007430 con 0x7f7bb815d070 2023-12-19T17:01:29.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.741+0000 7f7badffb640 1 --2- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7b9c065bd0 0x7f7b9c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:29.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.741+0000 7f7badffb640 1 -- 172.21.15.26:0/3841135453 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f7ba8082b90 con 0x7f7bb815d070 2023-12-19T17:01:29.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.741+0000 7f7bbd2a5640 1 -- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7b9c065bd0 msgr2=0x7f7b9c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:29.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.741+0000 7f7bbd2a5640 1 --2- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7b9c065bd0 0x7f7b9c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:01:29.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.745+0000 7f7badffb640 1 -- 172.21.15.26:0/3841135453 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7bb8155170 con 0x7f7bb815d070 2023-12-19T17:01:29.795 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:29.791+0000 7f039a84a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:01:29.825 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:29.821+0000 7fa04464a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:01:29.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.905+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/3841135453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f7bb8155170 con 0x7f7bb815d070 2023-12-19T17:01:29.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.909+0000 7f7badffb640 1 -- 172.21.15.26:0/3841135453 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v48)=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7bb8155170 con 0x7f7bb815d070 2023-12-19T17:01:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.913+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7b9c065bd0 msgr2=0x7f7b9c068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:01:29.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.913+0000 7f7bbe2a7640 1 --2- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7b9c065bd0 0x7f7b9c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.913+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/3841135453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb815d070 msgr2=0x7f7bb80c2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.913+0000 7f7bbe2a7640 1 --2- 172.21.15.26:0/3841135453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb815d070 0x7f7bb80c2880 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800c4d0 tx=0x7f7ba800c8d0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.913+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/3841135453 shutdown_connections 2023-12-19T17:01:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.913+0000 7f7bbe2a7640 1 --2- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7bb8160a20 0x7f7bb815f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.913+0000 7f7bbe2a7640 1 --2- 172.21.15.26:0/3841135453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7bb815d070 0x7f7bb80c2880 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.913+0000 7f7bbe2a7640 1 --2- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7b9c065bd0 0x7f7b9c068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.913+0000 7f7bbe2a7640 1 --2- 172.21.15.26:0/3841135453 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7bb815cc90 0x7f7bb814cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.913+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/3841135453 >> 172.21.15.26:0/3841135453 conn(0x7f7bb806a320 msgr2=0x7f7bb80c6530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.913+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/3841135453 shutdown_connections 2023-12-19T17:01:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:29.913+0000 7f7bbe2a7640 1 -- 172.21.15.26:0/3841135453 wait complete. 2023-12-19T17:01:29.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:01:29.938 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-19T17:01:29.945 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:29.943+0000 7f039a84a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:01:29.953 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1 1M 2023-12-19T17:01:29.986 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:29.981+0000 7fa04464a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:01:30.034 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:01:30.038 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:01:30.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.129+0000 7f93ce325640 1 Processor -- start 2023-12-19T17:01:30.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.129+0000 7f93ce325640 1 -- start start 2023-12-19T17:01:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.129+0000 7f93ce325640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93c8159140 0x7f93c80ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.129+0000 7f93ce325640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f93c80cd190 0x7f93c806ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.129+0000 7f93ce325640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f93c806b1c0 0x7f93c815ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.129+0000 7f93ce325640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c806b650 con 0x7f93c806b1c0 2023-12-19T17:01:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.129+0000 7f93ce325640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c806f490 con 0x7f93c80cd190 2023-12-19T17:01:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.129+0000 7f93ce325640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c8159520 con 0x7f93c8159140 2023-12-19T17:01:30.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93c77fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f93c80cd190 0x7f93c806ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:30.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93c7fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93c8159140 0x7f93c80ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:30.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93cc8c2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f93c806b1c0 0x7f93c815ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:30.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93c77fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f93c80cd190 0x7f93c806ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:56464/0 (socket says 172.21.15.26:56464) 2023-12-19T17:01:30.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93c77fe640 1 -- 172.21.15.26:0/3640154323 learned_addr learned my addr 172.21.15.26:0/3640154323 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:30.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93c77fe640 1 -- 172.21.15.26:0/3640154323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93c8159140 msgr2=0x7f93c80ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:30.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93c77fe640 1 --2- 172.21.15.26:0/3640154323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93c8159140 0x7f93c80ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:30.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93c77fe640 1 -- 172.21.15.26:0/3640154323 >> v2:172.21.15.26:3300/0 conn(0x7f93c806b1c0 msgr2=0x7f93c815ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:30.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93c77fe640 1 --2- 172.21.15.26:0/3640154323 >> v2:172.21.15.26:3300/0 conn(0x7f93c806b1c0 0x7f93c815ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:30.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93c77fe640 1 -- 172.21.15.26:0/3640154323 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c80cd690 con 0x7f93c80cd190 2023-12-19T17:01:30.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93c77fe640 1 --2- 172.21.15.26:0/3640154323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93c80cd190 0x7f93c806ac80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f93c80b1320 tx=0x7f93bc01e980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be8bfac1b5ca92b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:30.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93c6ffd640 1 -- 172.21.15.26:0/3640154323 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc004230 con 0x7f93c80cd190 2023-12-19T17:01:30.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93c6ffd640 1 -- 172.21.15.26:0/3640154323 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc0ab070 con 0x7f93c80cd190 2023-12-19T17:01:30.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.133+0000 7f93c6ffd640 1 -- 172.21.15.26:0/3640154323 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0a7790 con 0x7f93c80cd190 2023-12-19T17:01:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 -- 172.21.15.26:0/3640154323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93c80cd190 msgr2=0x7f93c806ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 --2- 172.21.15.26:0/3640154323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93c80cd190 0x7f93c806ac80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f93c80b1320 tx=0x7f93bc01e980 comp rx=0 tx=0).stop 2023-12-19T17:01:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 -- 172.21.15.26:0/3640154323 shutdown_connections 2023-12-19T17:01:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 --2- 172.21.15.26:0/3640154323 >> v2:172.21.15.26:3300/0 conn(0x7f93c806b1c0 0x7f93c815ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 --2- 172.21.15.26:0/3640154323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93c80cd190 0x7f93c806ac80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 --2- 172.21.15.26:0/3640154323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93c8159140 0x7f93c80ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 -- 172.21.15.26:0/3640154323 >> 172.21.15.26:0/3640154323 conn(0x7f93c80ce730 msgr2=0x7f93c80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 -- 172.21.15.26:0/3640154323 shutdown_connections 2023-12-19T17:01:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 -- 172.21.15.26:0/3640154323 wait complete. 2023-12-19T17:01:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 Processor -- start 2023-12-19T17:01:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 -- start start 2023-12-19T17:01:30.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93c806b1c0 0x7f93c8153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:30.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93c80cd190 0x7f93c80c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:30.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93c8159140 0x7f93c80ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:30.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c8154250 con 0x7f93c80cd190 2023-12-19T17:01:30.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c80c5770 con 0x7f93c8159140 2023-12-19T17:01:30.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93ce325640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c80c65d0 con 0x7f93c806b1c0 2023-12-19T17:01:30.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93c77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93c80cd190 0x7f93c80c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:30.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93c77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93c80cd190 0x7f93c80c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34440/0 (socket says 172.21.15.26:34440) 2023-12-19T17:01:30.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93c77fe640 1 -- 172.21.15.26:0/2483644785 learned_addr learned my addr 172.21.15.26:0/2483644785 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:30.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93c7fff640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93c806b1c0 0x7f93c8153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:30.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.137+0000 7f93cc8c2640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93c8159140 0x7f93c80ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:30.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.141+0000 7f93c77fe640 1 -- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93c806b1c0 msgr2=0x7f93c8153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:30.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.141+0000 7f93c77fe640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93c806b1c0 0x7f93c8153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:30.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.141+0000 7f93c77fe640 1 -- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93c8159140 msgr2=0x7f93c80ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:30.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.141+0000 7f93c77fe640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93c8159140 0x7f93c80ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:30.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.141+0000 7f93c77fe640 1 -- 172.21.15.26:0/2483644785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc00b780 con 0x7f93c80cd190 2023-12-19T17:01:30.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.141+0000 7f93c7fff640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93c806b1c0 0x7f93c8153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:01:30.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.141+0000 7f93c77fe640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93c80cd190 0x7f93c80c7b40 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f93bc000c10 tx=0x7f93bc01b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:30.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.141+0000 7f93c4ff9640 1 -- 172.21.15.26:0/2483644785 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0a79a0 con 0x7f93c80cd190 2023-12-19T17:01:30.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.141+0000 7f93c4ff9640 1 -- 172.21.15.26:0/2483644785 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc0a7b20 con 0x7f93c80cd190 2023-12-19T17:01:30.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.141+0000 7f93c4ff9640 1 -- 172.21.15.26:0/2483644785 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0a7cd0 con 0x7f93c80cd190 2023-12-19T17:01:30.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.141+0000 7f93ce325640 1 -- 172.21.15.26:0/2483644785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c8157e50 con 0x7f93c80cd190 2023-12-19T17:01:30.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.141+0000 7f93ce325640 1 -- 172.21.15.26:0/2483644785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c80c59c0 con 0x7f93c80cd190 2023-12-19T17:01:30.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.141+0000 7f93ce325640 1 -- 172.21.15.26:0/2483644785 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c80c6780 con 0x7f93c80cd190 2023-12-19T17:01:30.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.145+0000 7f93c4ff9640 1 -- 172.21.15.26:0/2483644785 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 69) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f93bc003620 con 0x7f93c80cd190 2023-12-19T17:01:30.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.145+0000 7f93c4ff9640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 0x7f93b4068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:30.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.145+0000 7f93c4ff9640 1 -- 172.21.15.26:0/2483644785 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f93bc11d110 con 0x7f93c80cd190 2023-12-19T17:01:30.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.145+0000 7f93c7fff640 1 -- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 msgr2=0x7f93b4068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:30.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.145+0000 7f93c7fff640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 0x7f93b4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:01:30.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.149+0000 7f93c4ff9640 1 -- 172.21.15.26:0/2483644785 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f93c80c6780 con 0x7f93c80cd190 2023-12-19T17:01:30.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.325+0000 7f93ce325640 1 -- 172.21.15.26:0/2483644785 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f93c806f120 con 0x7f93b4065bd0 2023-12-19T17:01:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.349+0000 7f93c7fff640 1 -- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 msgr2=0x7f93b4068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:30.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.349+0000 7f93c7fff640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 0x7f93b4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T17:01:30.427 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:30.423+0000 7f039a84a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:01:30.485 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:30.483+0000 7f039a84a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:01:30.500 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:30.497+0000 7fa04464a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:01:30.540 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:30.535+0000 7f039a84a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:01:30.561 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:30.557+0000 7fa04464a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:01:30.619 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:30.613+0000 7fa04464a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:01:30.625 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:30.623+0000 7f039a84a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:01:30.682 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:30.679+0000 7f039a84a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:01:30.711 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:30.705+0000 7fa04464a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:01:30.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.753+0000 7f93c7fff640 1 -- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 msgr2=0x7f93b4068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:30.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:30.753+0000 7f93c7fff640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 0x7f93b4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-19T17:01:30.767 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:30.763+0000 7f039a84a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:01:30.771 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:30.765+0000 7fa04464a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:01:30.823 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:30.819+0000 7f039a84a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:01:30.861 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:30.853+0000 7fa04464a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:01:30.918 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:30.913+0000 7fa04464a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:01:30.965 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:30.963+0000 7f039a84a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:01:31.023 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:31.019+0000 7f039a84a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:01:31.069 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:31.065+0000 7fa04464a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:01:31.131 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:31.125+0000 7fa04464a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:01:31.262 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:01:31.262 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:01:31.262 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:01:31.271 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:31.267+0000 7f039a84a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:01:31.380 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:01:31.380 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:01:31.380 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:01:31.388 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:31.385+0000 7fa04464a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:01:31.405 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:31.403+0000 7f039a84a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:01:31.530 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:31.527+0000 7f039a84a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:01:31.531 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:31.525+0000 7fa04464a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:01:31.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:31.557+0000 7f93c7fff640 1 -- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 msgr2=0x7f93b4068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:01:31.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:31.557+0000 7f93c7fff640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 0x7f93b4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-19T17:01:31.600 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:31.595+0000 7f039a84a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:01:31.662 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:31.657+0000 7fa04464a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:01:31.736 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:31.731+0000 7f039a84a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:01:31.737 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:31.733+0000 7fa04464a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:01:31.880 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:31.877+0000 7fa04464a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:01:32.273 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:32.271+0000 7f039a84a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:01:32.357 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:32.355+0000 7f039a84a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:01:32.448 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:32.441+0000 7fa04464a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:01:32.505 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:32.503+0000 7f039a84a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:01:32.536 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:32.533+0000 7fa04464a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:01:32.589 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:32.587+0000 7f039a84a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:01:32.694 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:32.689+0000 7fa04464a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:01:32.782 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:32.777+0000 7fa04464a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:01:32.908 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:32.903+0000 7f039a84a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:01:32.968 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:32.963+0000 7f039a84a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:01:32.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:32.973+0000 7f93c4ff9640 1 -- 172.21.15.26:0/2483644785 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 70) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f93bc0eabf0 con 0x7f93c80cd190 2023-12-19T17:01:32.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:32.977+0000 7f93c4ff9640 1 -- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 msgr2=0x7f93b4068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:01:32.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:32.977+0000 7f93c4ff9640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 0x7f93b4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:33.093 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:33.091+0000 7f031cc20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:33.093 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:33.091+0000 7f031cc20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:33.093 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:33.091+0000 7f031cc20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:33.093 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:33.091+0000 7f031cc20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:33.094 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:33.091+0000 7f031cc20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:33.094 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:33.091+0000 7f031fc26640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:33.094 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:33.091+0000 7f031fc26640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:33.094 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:33.091+0000 7f031fc26640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:33.094 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:33.091+0000 7f031fc26640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:33.094 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:01:33.091+0000 7f031fc26640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:01:33.128 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:33.125+0000 7fa04464a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:01:33.192 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:01:33.189+0000 7fa04464a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:01:33.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:33.981+0000 7f93c4ff9640 1 -- 172.21.15.26:0/2483644785 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 71) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f93bc11b320 con 0x7f93c80cd190 2023-12-19T17:01:33.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:33.981+0000 7f93c4ff9640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 0x7f93b4068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:33.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:33.981+0000 7f93c4ff9640 1 -- 172.21.15.26:0/2483644785 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f93b406bbd0 con 0x7f93b4065bd0 2023-12-19T17:01:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:33.985+0000 7f93c7fff640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 0x7f93b4068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:33.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:33.989+0000 7f93c7fff640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 0x7f93b4068090 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f93c80c6eb0 tx=0x7f93b8009190 comp rx=0 tx=0).ready entity=mgr.8245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.049+0000 7f93c4ff9640 1 -- 172.21.15.26:0/2483644785 <== mgr.8245 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f93b406bbd0 con 0x7f93b4065bd0 2023-12-19T17:01:34.056 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path /snap_test_dir1 2023-12-19T17:01:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.053+0000 7f93ce325640 1 -- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 msgr2=0x7f93b4068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.053+0000 7f93ce325640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 0x7f93b4068090 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f93c80c6eb0 tx=0x7f93b8009190 comp rx=0 tx=0).stop 2023-12-19T17:01:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.053+0000 7f93ce325640 1 -- 172.21.15.26:0/2483644785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93c80cd190 msgr2=0x7f93c80c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.053+0000 7f93ce325640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93c80cd190 0x7f93c80c7b40 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f93bc000c10 tx=0x7f93bc01b720 comp rx=0 tx=0).stop 2023-12-19T17:01:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.053+0000 7f93ce325640 1 -- 172.21.15.26:0/2483644785 shutdown_connections 2023-12-19T17:01:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.053+0000 7f93ce325640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93c8159140 0x7f93c80ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.053+0000 7f93ce325640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93c80cd190 0x7f93c80c7b40 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.053+0000 7f93ce325640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f93b4065bd0 0x7f93b4068090 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.053+0000 7f93ce325640 1 --2- 172.21.15.26:0/2483644785 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93c806b1c0 0x7f93c8153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.053+0000 7f93ce325640 1 -- 172.21.15.26:0/2483644785 >> 172.21.15.26:0/2483644785 conn(0x7f93c80ce730 msgr2=0x7f93c80afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.053+0000 7f93ce325640 1 -- 172.21.15.26:0/2483644785 shutdown_connections 2023-12-19T17:01:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.053+0000 7f93ce325640 1 -- 172.21.15.26:0/2483644785 wait complete. 2023-12-19T17:01:34.081 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-12-19T17:01:34.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add /snap_test_dir1 55n 2023-12-19T17:01:34.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.225+0000 7faf16e6f640 1 Processor -- start 2023-12-19T17:01:34.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf16e6f640 1 -- start start 2023-12-19T17:01:34.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf16e6f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10161150 0x7faf10167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:34.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf16e6f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf10161550 con 0x7faf100cfcb0 2023-12-19T17:01:34.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf16e6f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf100cd490 con 0x7faf1015a770 2023-12-19T17:01:34.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf16e6f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf100cd770 con 0x7faf10161150 2023-12-19T17:01:34.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf1566c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faf100cfcb0 0x7faf10160a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57378/0 (socket says 172.21.15.26:57378) 2023-12-19T17:01:34.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf1566c640 1 -- 172.21.15.26:0/2917579191 learned_addr learned my addr 172.21.15.26:0/2917579191 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:01:34.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf1666e640 1 --2- 172.21.15.26:0/2917579191 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10161150 0x7faf10167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:34.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf14e6b640 1 -- 172.21.15.26:0/2917579191 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4210490800 0 0) 0x7faf00001120 con 0x7faf100cfcb0 2023-12-19T17:01:34.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf14e6b640 1 -- 172.21.15.26:0/2917579191 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faefc003530 con 0x7faf100cfcb0 2023-12-19T17:01:34.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf14e6b640 1 -- 172.21.15.26:0/2917579191 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470179271 0 0) 0x7faf100cd490 con 0x7faf1015a770 2023-12-19T17:01:34.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf14e6b640 1 -- 172.21.15.26:0/2917579191 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf00001120 con 0x7faf1015a770 2023-12-19T17:01:34.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf1666e640 1 -- 172.21.15.26:0/2917579191 >> v1:172.21.15.160:6789/0 conn(0x7faf1015a770 legacy=0x7faf100cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf1666e640 1 -- 172.21.15.26:0/2917579191 >> v1:172.21.15.26:6789/0 conn(0x7faf100cfcb0 legacy=0x7faf10160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.229+0000 7faf1666e640 1 -- 172.21.15.26:0/2917579191 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf100cda50 con 0x7faf10161150 2023-12-19T17:01:34.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.233+0000 7faf1666e640 1 --2- 172.21.15.26:0/2917579191 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10161150 0x7faf10167ac0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faf08009dc0 tx=0x7faf0801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6fb8f5bbf169f8b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:34.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.233+0000 7faf14e6b640 1 -- 172.21.15.26:0/2917579191 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf080a59a0 con 0x7faf10161150 2023-12-19T17:01:34.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.233+0000 7faf14e6b640 1 -- 172.21.15.26:0/2917579191 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf080a5b00 con 0x7faf10161150 2023-12-19T17:01:34.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.233+0000 7faf14e6b640 1 -- 172.21.15.26:0/2917579191 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf080a5e70 con 0x7faf10161150 2023-12-19T17:01:34.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.233+0000 7faf16e6f640 1 -- 172.21.15.26:0/2917579191 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10161150 msgr2=0x7faf10167ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.233+0000 7faf16e6f640 1 --2- 172.21.15.26:0/2917579191 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10161150 0x7faf10167ac0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faf08009dc0 tx=0x7faf0801c940 comp rx=0 tx=0).stop 2023-12-19T17:01:34.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.233+0000 7faf16e6f640 1 -- 172.21.15.26:0/2917579191 shutdown_connections 2023-12-19T17:01:34.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.233+0000 7faf16e6f640 1 --2- 172.21.15.26:0/2917579191 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf10161150 0x7faf10167ac0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.233+0000 7faf16e6f640 1 -- 172.21.15.26:0/2917579191 >> 172.21.15.26:0/2917579191 conn(0x7faf1006a150 msgr2=0x7faf1006a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:34.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.233+0000 7faf16e6f640 1 -- 172.21.15.26:0/2917579191 shutdown_connections 2023-12-19T17:01:34.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.233+0000 7faf16e6f640 1 -- 172.21.15.26:0/2917579191 wait complete. 2023-12-19T17:01:34.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf16e6f640 1 Processor -- start 2023-12-19T17:01:34.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf16e6f640 1 -- start start 2023-12-19T17:01:34.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf16e6f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf100cfcb0 0x7faf10150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:34.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf16e6f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf1015a770 0x7faf100cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:34.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf16e6f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10161150 0x7faf1015e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:34.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf16e6f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf10150f40 con 0x7faf10161150 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf16e6f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1015e0d0 con 0x7faf1015a770 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf16e6f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1015d3a0 con 0x7faf100cfcb0 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf1666e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10161150 0x7faf1015e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf1666e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10161150 0x7faf1015e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34456/0 (socket says 172.21.15.26:34456) 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf1666e640 1 -- 172.21.15.26:0/3340183682 learned_addr learned my addr 172.21.15.26:0/3340183682 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf15e6d640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf100cfcb0 0x7faf10150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf1566c640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf1015a770 0x7faf100cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf1666e640 1 -- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf100cfcb0 msgr2=0x7faf10150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf1666e640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf100cfcb0 0x7faf10150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf1666e640 1 -- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf1015a770 msgr2=0x7faf100cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf1666e640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf1015a770 0x7faf100cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf1666e640 1 -- 172.21.15.26:0/3340183682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf08009a70 con 0x7faf10161150 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf1566c640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf1015a770 0x7faf100cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:01:34.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf1666e640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10161150 0x7faf1015e7f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7faf08019550 tx=0x7faf080196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:34.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf067fc640 1 -- 172.21.15.26:0/3340183682 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08004240 con 0x7faf10161150 2023-12-19T17:01:34.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf067fc640 1 -- 172.21.15.26:0/3340183682 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf080043a0 con 0x7faf10161150 2023-12-19T17:01:34.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf16e6f640 1 -- 172.21.15.26:0/3340183682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf100cc840 con 0x7faf10161150 2023-12-19T17:01:34.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf067fc640 1 -- 172.21.15.26:0/3340183682 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08004710 con 0x7faf10161150 2023-12-19T17:01:34.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf16e6f640 1 -- 172.21.15.26:0/3340183682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1015d8d0 con 0x7faf10161150 2023-12-19T17:01:34.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.237+0000 7faf16e6f640 1 -- 172.21.15.26:0/3340183682 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf1004e850 con 0x7faf10161150 2023-12-19T17:01:34.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.241+0000 7faf067fc640 1 -- 172.21.15.26:0/3340183682 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 71) v1 ==== 85083+0+0 (secure 0 0 0) 0x7faf080b4030 con 0x7faf10161150 2023-12-19T17:01:34.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.241+0000 7faf067fc640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faef4065900 0x7faef4067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:34.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.241+0000 7faf067fc640 1 -- 172.21.15.26:0/3340183682 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6785+0+0 (secure 0 0 0) 0x7faf0811e4b0 con 0x7faf10161150 2023-12-19T17:01:34.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.241+0000 7faf15e6d640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faef4065900 0x7faef4067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:34.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.241+0000 7faf15e6d640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faef4065900 0x7faef4067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf0c007fd0 tx=0x7faf0c009040 comp rx=0 tx=0).ready entity=mgr.8245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:34.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.245+0000 7faf067fc640 1 -- 172.21.15.26:0/3340183682 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faf080ebdc0 con 0x7faf10161150 2023-12-19T17:01:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.421+0000 7faf16e6f640 1 -- 172.21.15.26:0/3340183682 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7faf10160470 con 0x7faef4065900 2023-12-19T17:01:34.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.473+0000 7faf067fc640 1 -- 172.21.15.26:0/3340183682 <== mgr.8245 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7faf10160470 con 0x7faef4065900 2023-12-19T17:01:34.479 INFO:teuthology.orchestra.run.smithi026.stdout:Retention added to path /snap_test_dir1 2023-12-19T17:01:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.477+0000 7faf16e6f640 1 -- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faef4065900 msgr2=0x7faef4067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.477+0000 7faf16e6f640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faef4065900 0x7faef4067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf0c007fd0 tx=0x7faf0c009040 comp rx=0 tx=0).stop 2023-12-19T17:01:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.477+0000 7faf16e6f640 1 -- 172.21.15.26:0/3340183682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10161150 msgr2=0x7faf1015e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.477+0000 7faf16e6f640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10161150 0x7faf1015e7f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7faf08019550 tx=0x7faf080196e0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.477+0000 7faf16e6f640 1 -- 172.21.15.26:0/3340183682 shutdown_connections 2023-12-19T17:01:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.477+0000 7faf16e6f640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf10161150 0x7faf1015e7f0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.477+0000 7faf16e6f640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf1015a770 0x7faf100cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.477+0000 7faf16e6f640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faef4065900 0x7faef4067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.477+0000 7faf16e6f640 1 --2- 172.21.15.26:0/3340183682 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf100cfcb0 0x7faf10150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.477+0000 7faf16e6f640 1 -- 172.21.15.26:0/3340183682 >> 172.21.15.26:0/3340183682 conn(0x7faf1006a150 msgr2=0x7faf1015f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:34.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.477+0000 7faf16e6f640 1 -- 172.21.15.26:0/3340183682 shutdown_connections 2023-12-19T17:01:34.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.477+0000 7faf16e6f640 1 -- 172.21.15.26:0/3340183682 wait complete. 2023-12-19T17:01:34.500 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-12-19T17:01:34.501 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-12-19T17:01:34.501 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json /snap_test_dir1 2023-12-19T17:01:34.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f9265f96640 1 Processor -- start 2023-12-19T17:01:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f9265f96640 1 -- start start 2023-12-19T17:01:34.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f9265f96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f926015cc70 0x7f92600d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f9265f96640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f92600b88b0 0x7f92600b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f9265f96640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92600b71f0 con 0x7f92600b6140 2023-12-19T17:01:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f9265f96640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92600b65e0 con 0x7f92600b88b0 2023-12-19T17:01:34.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f9265f96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92600b68c0 con 0x7f926015cc70 2023-12-19T17:01:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f925f7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f926015cc70 0x7f92600d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f925effd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f92600b88b0 0x7f92600b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f925f7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f926015cc70 0x7f92600d0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:58260/0 (socket says 172.21.15.26:58260) 2023-12-19T17:01:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f925f7fe640 1 -- 172.21.15.26:0/2629128565 learned_addr learned my addr 172.21.15.26:0/2629128565 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f925ffff640 1 --1- 172.21.15.26:0/2629128565 >> v1:172.21.15.160:6789/0 conn(0x7f92600b6140 0x7f92600cfbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:32994/0 (socket says 172.21.15.26:32994) 2023-12-19T17:01:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f925f7fe640 1 -- 172.21.15.26:0/2629128565 >> v2:172.21.15.160:3300/0 conn(0x7f92600b88b0 msgr2=0x7f92600b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:01:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f925f7fe640 1 --2- 172.21.15.26:0/2629128565 >> v2:172.21.15.160:3300/0 conn(0x7f92600b88b0 0x7f92600b5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f925f7fe640 1 -- 172.21.15.26:0/2629128565 >> v1:172.21.15.160:6789/0 conn(0x7f92600b6140 legacy=0x7f92600cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.641+0000 7f925f7fe640 1 -- 172.21.15.26:0/2629128565 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92600b6ba0 con 0x7f926015cc70 2023-12-19T17:01:34.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.645+0000 7f925f7fe640 1 --2- 172.21.15.26:0/2629128565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f926015cc70 0x7f92600d0c60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9250009870 tx=0x7f925001ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c1a867bb27cb253 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.645+0000 7f925e7fc640 1 -- 172.21.15.26:0/2629128565 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92500a8a30 con 0x7f926015cc70 2023-12-19T17:01:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.645+0000 7f925e7fc640 1 -- 172.21.15.26:0/2629128565 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92500a8b90 con 0x7f926015cc70 2023-12-19T17:01:34.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.645+0000 7f925e7fc640 1 -- 172.21.15.26:0/2629128565 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92500b0440 con 0x7f926015cc70 2023-12-19T17:01:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.645+0000 7f9265f96640 1 -- 172.21.15.26:0/2629128565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f926015cc70 msgr2=0x7f92600d0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.645+0000 7f9265f96640 1 --2- 172.21.15.26:0/2629128565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f926015cc70 0x7f92600d0c60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9250009870 tx=0x7f925001ed30 comp rx=0 tx=0).stop 2023-12-19T17:01:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.645+0000 7f9265f96640 1 -- 172.21.15.26:0/2629128565 shutdown_connections 2023-12-19T17:01:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.645+0000 7f9265f96640 1 --2- 172.21.15.26:0/2629128565 >> v2:172.21.15.160:3300/0 conn(0x7f92600b88b0 0x7f92600b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.645+0000 7f9265f96640 1 --2- 172.21.15.26:0/2629128565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f926015cc70 0x7f92600d0c60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.645+0000 7f9265f96640 1 -- 172.21.15.26:0/2629128565 >> 172.21.15.26:0/2629128565 conn(0x7f926006a120 msgr2=0x7f926006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.645+0000 7f9265f96640 1 -- 172.21.15.26:0/2629128565 shutdown_connections 2023-12-19T17:01:34.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f9265f96640 1 -- 172.21.15.26:0/2629128565 wait complete. 2023-12-19T17:01:34.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f9265f96640 1 Processor -- start 2023-12-19T17:01:34.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f9265f96640 1 -- start start 2023-12-19T17:01:34.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f9265f96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f92600b6140 0x7f92600cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:34.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f9265f96640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92600b88b0 0x7f92601570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:34.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f9265f96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f926015cc70 0x7f9260159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:34.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f9265f96640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92600cafb0 con 0x7f926015cc70 2023-12-19T17:01:34.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f9265f96640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92600cc070 con 0x7f92600b88b0 2023-12-19T17:01:34.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f9265f96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92600cb460 con 0x7f92600b6140 2023-12-19T17:01:34.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f925ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f926015cc70 0x7f9260159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:34.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f925f7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f92600b6140 0x7f92600cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:34.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f925effd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92600b88b0 0x7f92601570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:34.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f925f7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f92600b6140 0x7f92600cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:58272/0 (socket says 172.21.15.26:58272) 2023-12-19T17:01:34.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f925f7fe640 1 -- 172.21.15.26:0/2674480060 learned_addr learned my addr 172.21.15.26:0/2674480060 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:01:34.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f925f7fe640 1 -- 172.21.15.26:0/2674480060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92600b88b0 msgr2=0x7f92601570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f925f7fe640 1 --2- 172.21.15.26:0/2674480060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92600b88b0 0x7f92601570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f925f7fe640 1 -- 172.21.15.26:0/2674480060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f926015cc70 msgr2=0x7f9260159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f925f7fe640 1 --2- 172.21.15.26:0/2674480060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f926015cc70 0x7f9260159450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f925f7fe640 1 -- 172.21.15.26:0/2674480060 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92500094d0 con 0x7f92600b6140 2023-12-19T17:01:34.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.649+0000 7f925f7fe640 1 --2- 172.21.15.26:0/2674480060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f92600b6140 0x7f92600cc500 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f92500b3040 tx=0x7f925001bad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:34.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.653+0000 7f9247fff640 1 -- 172.21.15.26:0/2674480060 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92500b0dd0 con 0x7f92600b6140 2023-12-19T17:01:34.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.653+0000 7f9247fff640 1 -- 172.21.15.26:0/2674480060 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92500bb460 con 0x7f92600b6140 2023-12-19T17:01:34.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.653+0000 7f9247fff640 1 -- 172.21.15.26:0/2674480060 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92500bb810 con 0x7f92600b6140 2023-12-19T17:01:34.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.653+0000 7f9265f96640 1 -- 172.21.15.26:0/2674480060 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92600cb740 con 0x7f92600b6140 2023-12-19T17:01:34.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.653+0000 7f9265f96640 1 -- 172.21.15.26:0/2674480060 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92600cbd00 con 0x7f92600b6140 2023-12-19T17:01:34.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.653+0000 7f9265f96640 1 -- 172.21.15.26:0/2674480060 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9260074270 con 0x7f92600b6140 2023-12-19T17:01:34.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.653+0000 7f9247fff640 1 -- 172.21.15.26:0/2674480060 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 71) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f92500bb970 con 0x7f92600b6140 2023-12-19T17:01:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.653+0000 7f9247fff640 1 --2- 172.21.15.26:0/2674480060 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f924c065900 0x7f924c067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:01:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.653+0000 7f9247fff640 1 -- 172.21.15.26:0/2674480060 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f9250120c10 con 0x7f92600b6140 2023-12-19T17:01:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.653+0000 7f925effd640 1 --2- 172.21.15.26:0/2674480060 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f924c065900 0x7f924c067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:01:34.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.657+0000 7f925effd640 1 --2- 172.21.15.26:0/2674480060 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f924c065900 0x7f924c067dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9260155b20 tx=0x7f9254009190 comp rx=0 tx=0).ready entity=mgr.8245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:01:34.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.657+0000 7f9247fff640 1 -- 172.21.15.26:0/2674480060 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f92500ee530 con 0x7f92600b6140 2023-12-19T17:01:34.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.837+0000 7f9265f96640 1 -- 172.21.15.26:0/2674480060 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92600b6e80 con 0x7f924c065900 2023-12-19T17:01:34.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.845+0000 7f9247fff640 1 -- 172.21.15.26:0/2674480060 <== mgr.8245 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7f9254011a30 con 0x7f924c065900 2023-12-19T17:01:34.851 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:01:34.851 INFO:teuthology.orchestra.run.smithi026.stdout:{"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-12-19T17:01:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.849+0000 7f9265f96640 1 -- 172.21.15.26:0/2674480060 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f924c065900 msgr2=0x7f924c067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.849+0000 7f9265f96640 1 --2- 172.21.15.26:0/2674480060 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f924c065900 0x7f924c067dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9260155b20 tx=0x7f9254009190 comp rx=0 tx=0).stop 2023-12-19T17:01:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.849+0000 7f9265f96640 1 -- 172.21.15.26:0/2674480060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f92600b6140 msgr2=0x7f92600cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:01:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.849+0000 7f9265f96640 1 --2- 172.21.15.26:0/2674480060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f92600b6140 0x7f92600cc500 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f92500b3040 tx=0x7f925001bad0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.849+0000 7f9265f96640 1 -- 172.21.15.26:0/2674480060 shutdown_connections 2023-12-19T17:01:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.849+0000 7f9265f96640 1 --2- 172.21.15.26:0/2674480060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f926015cc70 0x7f9260159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.849+0000 7f9265f96640 1 --2- 172.21.15.26:0/2674480060 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92600b88b0 0x7f92601570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.849+0000 7f9265f96640 1 --2- 172.21.15.26:0/2674480060 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f924c065900 0x7f924c067dc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.849+0000 7f9265f96640 1 --2- 172.21.15.26:0/2674480060 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f92600b6140 0x7f92600cc500 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:01:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.849+0000 7f9265f96640 1 -- 172.21.15.26:0/2674480060 >> 172.21.15.26:0/2674480060 conn(0x7f926006a120 msgr2=0x7f92600d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:01:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.849+0000 7f9265f96640 1 -- 172.21.15.26:0/2674480060 shutdown_connections 2023-12-19T17:01:34.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:01:34.849+0000 7f9265f96640 1 -- 172.21.15.26:0/2674480060 wait complete. 2023-12-19T17:01:34.872 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-12-19T17:01:34.872 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1M'], 'retention': [{'n': 55}]} 2023-12-19T17:02:00.089 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:02:00.094 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:02:00.491 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:02:00.487+0000 7f030ec04640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:02:00.491 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:02:00.487+0000 7f030ec04640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:02:00.491 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:02:00.487+0000 7f030ec04640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:02:00.491 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:02:00.487+0000 7f030ec04640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:02:00.491 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:02:00.487+0000 7f030ec04640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:02:30.153 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:02:30.158 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:03:00.217 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:03:00.222 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:03:30.280 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:03:30.285 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:04:00.352 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:04:00.357 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:04:30.424 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:04:30.429 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:05:00.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:05:00.494 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:05:30.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:05:30.598 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:06:00.649 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:06:00.654 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:06:30.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:06:30.710 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:07:00.766 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:07:00.771 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:07:30.832 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:07:30.838 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:08:00.895 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:08:00.900 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:08:30.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:08:30.963 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:09:01.022 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:09:01.027 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:09:31.101 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:09:31.106 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:10:01.166 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:10:01.171 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:10:31.255 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:10:31.260 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:11:01.318 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:11:01.323 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:11:31.382 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:11:31.387 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:12:01.446 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:12:01.451 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:12:31.506 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:12:31.511 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:13:01.562 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:13:01.567 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:13:31.637 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:13:31.641 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:14:01.696 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:14:01.701 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:14:31.777 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:14:31.781 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:15:01.836 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:15:01.841 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:15:31.918 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:15:31.923 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:16:01.981 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:16:01.986 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:16:32.045 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:16:32.050 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:17:02.108 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:17:02.112 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:17:32.172 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:17:32.177 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:18:02.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:18:02.287 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:18:32.343 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:18:32.348 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:19:02.431 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:19:02.436 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:19:32.512 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:19:32.517 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:20:02.575 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:20:02.580 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:20:32.664 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:20:32.671 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:21:02.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:21:02.733 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:21:32.791 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:21:32.796 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:22:02.855 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:22:02.860 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:22:32.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:22:32.956 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:23:03.008 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:23:03.013 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:23:33.080 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:23:33.085 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:24:03.146 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:24:03.151 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:24:33.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:24:33.226 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:25:03.288 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:25:03.293 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:25:33.369 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:25:33.373 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:26:03.429 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:26:03.434 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:26:33.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:26:33.522 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:27:03.610 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:27:03.615 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:27:33.674 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:27:33.679 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:28:03.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:28:03.742 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:28:33.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:28:33.801 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:29:03.857 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:29:03.863 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:29:33.934 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:29:33.942 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:30:04.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:30:04.007 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:30:34.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:30:34.087 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:31:04.146 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:31:04.151 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:31:34.203 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:31:34.207 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:32:04.262 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:32:04.267 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:32:34.329 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:32:34.335 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:33:04.392 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:33:04.397 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:33:34.456 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:33:34.461 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:34:04.521 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:34:04.525 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:34:34.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:34:34.598 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:35:04.655 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:35:04.660 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:35:34.744 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:35:34.748 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:36:04.807 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:36:04.812 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:36:34.873 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:36:34.878 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:37:04.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:37:04.939 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:37:35.022 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:37:35.027 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:38:05.087 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:38:05.091 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:38:35.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:38:35.155 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:39:05.207 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:39:05.211 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:39:35.270 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:39:35.274 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:40:05.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:40:05.338 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:40:35.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:40:35.428 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:41:05.510 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:41:05.514 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:41:35.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:41:35.578 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:42:05.637 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:42:05.642 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:42:35.701 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:42:35.705 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:43:05.765 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:43:05.770 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:43:35.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:43:35.833 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:44:05.892 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:44:05.897 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:44:35.963 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:44:35.969 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:45:06.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:45:06.031 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:45:36.126 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:45:36.131 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:46:06.188 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:46:06.193 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:46:36.253 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:46:36.258 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:47:06.317 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:47:06.322 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:47:36.380 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:47:36.385 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:48:06.444 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:48:06.449 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:48:36.508 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:48:36.513 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:49:06.568 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:49:06.572 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:49:36.659 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:49:36.663 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:50:06.723 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:50:06.728 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:50:36.812 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:50:36.817 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:51:06.876 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:51:06.881 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:51:36.940 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:51:36.945 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:52:07.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:52:07.008 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:52:37.075 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:52:37.080 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:53:07.130 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:53:07.135 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:53:37.260 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:53:37.265 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:54:07.324 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:54:07.329 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:54:37.454 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:54:37.459 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:55:07.602 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:55:07.606 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:55:37.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:55:37.687 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:56:07.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:56:07.834 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:56:37.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:56:37.964 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:57:08.089 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:57:08.094 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:57:15.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:15.874 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:57:15.890 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_03_00_UTC 2023-12-19T17:57:15.890 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_04_00_UTC 2023-12-19T17:57:15.890 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_05_00_UTC 2023-12-19T17:57:15.890 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_06_00_UTC 2023-12-19T17:57:15.890 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_07_00_UTC 2023-12-19T17:57:15.890 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_08_00_UTC 2023-12-19T17:57:15.890 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_09_00_UTC 2023-12-19T17:57:15.890 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_10_00_UTC 2023-12-19T17:57:15.890 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_11_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_12_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_13_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_14_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_15_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_16_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_17_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_18_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_19_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_20_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_21_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_22_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_23_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_24_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_25_00_UTC 2023-12-19T17:57:15.891 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_26_00_UTC 2023-12-19T17:57:15.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_27_00_UTC 2023-12-19T17:57:15.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_28_00_UTC 2023-12-19T17:57:15.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_29_00_UTC 2023-12-19T17:57:15.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_30_00_UTC 2023-12-19T17:57:15.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_31_00_UTC 2023-12-19T17:57:15.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_32_00_UTC 2023-12-19T17:57:15.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_33_00_UTC 2023-12-19T17:57:15.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_34_00_UTC 2023-12-19T17:57:15.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_35_00_UTC 2023-12-19T17:57:15.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_36_00_UTC 2023-12-19T17:57:15.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_37_00_UTC 2023-12-19T17:57:15.892 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_38_00_UTC 2023-12-19T17:57:15.893 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_39_00_UTC 2023-12-19T17:57:15.893 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_40_00_UTC 2023-12-19T17:57:15.893 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_41_00_UTC 2023-12-19T17:57:15.893 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_42_00_UTC 2023-12-19T17:57:15.893 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_43_00_UTC 2023-12-19T17:57:15.893 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_44_00_UTC 2023-12-19T17:57:15.893 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_45_00_UTC 2023-12-19T17:57:15.893 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_46_00_UTC 2023-12-19T17:57:15.893 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_47_00_UTC 2023-12-19T17:57:15.893 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_48_00_UTC 2023-12-19T17:57:15.893 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_49_00_UTC 2023-12-19T17:57:15.893 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_50_00_UTC 2023-12-19T17:57:15.893 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_51_00_UTC 2023-12-19T17:57:15.894 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_52_00_UTC 2023-12-19T17:57:15.894 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_53_00_UTC 2023-12-19T17:57:15.894 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_54_00_UTC 2023-12-19T17:57:15.894 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_55_00_UTC 2023-12-19T17:57:15.894 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_56_00_UTC 2023-12-19T17:57:15.894 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_57_00_UTC 2023-12-19T17:57:15.895 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-19-17_03_00_UTC', 'scheduled-2023-12-19-17_04_00_UTC', 'scheduled-2023-12-19-17_05_00_UTC', 'scheduled-2023-12-19-17_06_00_UTC', 'scheduled-2023-12-19-17_07_00_UTC', 'scheduled-2023-12-19-17_08_00_UTC', 'scheduled-2023-12-19-17_09_00_UTC', 'scheduled-2023-12-19-17_10_00_UTC', 'scheduled-2023-12-19-17_11_00_UTC', 'scheduled-2023-12-19-17_12_00_UTC', 'scheduled-2023-12-19-17_13_00_UTC', 'scheduled-2023-12-19-17_14_00_UTC', 'scheduled-2023-12-19-17_15_00_UTC', 'scheduled-2023-12-19-17_16_00_UTC', 'scheduled-2023-12-19-17_17_00_UTC', 'scheduled-2023-12-19-17_18_00_UTC', 'scheduled-2023-12-19-17_19_00_UTC', 'scheduled-2023-12-19-17_20_00_UTC', 'scheduled-2023-12-19-17_21_00_UTC', 'scheduled-2023-12-19-17_22_00_UTC', 'scheduled-2023-12-19-17_23_00_UTC', 'scheduled-2023-12-19-17_24_00_UTC', 'scheduled-2023-12-19-17_25_00_UTC', 'scheduled-2023-12-19-17_26_00_UTC', 'scheduled-2023-12-19-17_27_00_UTC', 'scheduled-2023-12-19-17_28_00_UTC', 'scheduled-2023-12-19-17_29_00_UTC', 'scheduled-2023-12-19-17_30_00_UTC', 'scheduled-2023-12-19-17_31_00_UTC', 'scheduled-2023-12-19-17_32_00_UTC', 'scheduled-2023-12-19-17_33_00_UTC', 'scheduled-2023-12-19-17_34_00_UTC', 'scheduled-2023-12-19-17_35_00_UTC', 'scheduled-2023-12-19-17_36_00_UTC', 'scheduled-2023-12-19-17_37_00_UTC', 'scheduled-2023-12-19-17_38_00_UTC', 'scheduled-2023-12-19-17_39_00_UTC', 'scheduled-2023-12-19-17_40_00_UTC', 'scheduled-2023-12-19-17_41_00_UTC', 'scheduled-2023-12-19-17_42_00_UTC', 'scheduled-2023-12-19-17_43_00_UTC', 'scheduled-2023-12-19-17_44_00_UTC', 'scheduled-2023-12-19-17_45_00_UTC', 'scheduled-2023-12-19-17_46_00_UTC', 'scheduled-2023-12-19-17_47_00_UTC', 'scheduled-2023-12-19-17_48_00_UTC', 'scheduled-2023-12-19-17_49_00_UTC', 'scheduled-2023-12-19-17_50_00_UTC', 'scheduled-2023-12-19-17_51_00_UTC', 'scheduled-2023-12-19-17_52_00_UTC', 'scheduled-2023-12-19-17_53_00_UTC', 'scheduled-2023-12-19-17_54_00_UTC', 'scheduled-2023-12-19-17_55_00_UTC', 'scheduled-2023-12-19-17_56_00_UTC', 'scheduled-2023-12-19-17_57_00_UTC'] 2023-12-19T17:57:15.895 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1 2023-12-19T17:57:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.091+0000 7f76409a2640 1 Processor -- start 2023-12-19T17:57:16.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.091+0000 7f76409a2640 1 -- start start 2023-12-19T17:57:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.095+0000 7f76409a2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f763c06d100 0x7f763c0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.095+0000 7f76409a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f763c0ce1d0 0x7f763c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.095+0000 7f76409a2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763c0ceb60 con 0x7f763c0cad20 2023-12-19T17:57:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.095+0000 7f76409a2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763c0cd080 con 0x7f763c06d100 2023-12-19T17:57:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.095+0000 7f76409a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c072e90 con 0x7f763c0ce1d0 2023-12-19T17:57:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.095+0000 7f763affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f763c06d100 0x7f763c0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.095+0000 7f763affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f763c06d100 0x7f763c0ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42472/0 (socket says 172.21.15.26:42472) 2023-12-19T17:57:16.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.095+0000 7f763affd640 1 -- 172.21.15.26:0/2981883752 learned_addr learned my addr 172.21.15.26:0/2981883752 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.095+0000 7f763bfff640 1 --2- 172.21.15.26:0/2981883752 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f763c0ce1d0 0x7f763c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.095+0000 7f763affd640 1 -- 172.21.15.26:0/2981883752 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f763c0ce1d0 msgr2=0x7f763c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.095+0000 7f763affd640 1 --2- 172.21.15.26:0/2981883752 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f763c0ce1d0 0x7f763c0ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.095+0000 7f763affd640 1 -- 172.21.15.26:0/2981883752 >> v1:172.21.15.160:6789/0 conn(0x7f763c0cad20 legacy=0x7f763c15dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:16.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.095+0000 7f763affd640 1 -- 172.21.15.26:0/2981883752 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c073170 con 0x7f763c06d100 2023-12-19T17:57:16.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f763affd640 1 --2- 172.21.15.26:0/2981883752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f763c06d100 0x7f763c0ac860 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f7630009870 tx=0x7f763001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16a735601cdc028 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:16.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f763a7fc640 1 -- 172.21.15.26:0/2981883752 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76300ac600 con 0x7f763c06d100 2023-12-19T17:57:16.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f763a7fc640 1 -- 172.21.15.26:0/2981883752 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76300a8050 con 0x7f763c06d100 2023-12-19T17:57:16.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f763a7fc640 1 -- 172.21.15.26:0/2981883752 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76300ac980 con 0x7f763c06d100 2023-12-19T17:57:16.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f76409a2640 1 -- 172.21.15.26:0/2981883752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f763c06d100 msgr2=0x7f763c0ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:16.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f76409a2640 1 --2- 172.21.15.26:0/2981883752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f763c06d100 0x7f763c0ac860 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f7630009870 tx=0x7f763001c4b0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f76409a2640 1 -- 172.21.15.26:0/2981883752 shutdown_connections 2023-12-19T17:57:16.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f76409a2640 1 --2- 172.21.15.26:0/2981883752 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f763c0ce1d0 0x7f763c0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f76409a2640 1 --2- 172.21.15.26:0/2981883752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f763c06d100 0x7f763c0ac860 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f76409a2640 1 -- 172.21.15.26:0/2981883752 >> 172.21.15.26:0/2981883752 conn(0x7f763c0b0900 msgr2=0x7f763c0b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:16.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f76409a2640 1 -- 172.21.15.26:0/2981883752 shutdown_connections 2023-12-19T17:57:16.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f76409a2640 1 -- 172.21.15.26:0/2981883752 wait complete. 2023-12-19T17:57:16.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f76409a2640 1 Processor -- start 2023-12-19T17:57:16.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.099+0000 7f76409a2640 1 -- start start 2023-12-19T17:57:16.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f76409a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f763c06d100 0x7f763c06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:16.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f76409a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f763c0cad20 0x7f763c1588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:16.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f76409a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f763c0ce1d0 0x7f763c15ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:16.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f76409a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c159120 con 0x7f763c06d100 2023-12-19T17:57:16.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f76409a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c070070 con 0x7f763c0ce1d0 2023-12-19T17:57:16.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f76409a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c070280 con 0x7f763c0cad20 2023-12-19T17:57:16.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f763b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f763c06d100 0x7f763c06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:16.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f763b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f763c06d100 0x7f763c06fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42484/0 (socket says 172.21.15.26:42484) 2023-12-19T17:57:16.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f763b7fe640 1 -- 172.21.15.26:0/1908806526 learned_addr learned my addr 172.21.15.26:0/1908806526 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:16.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f763affd640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f763c0cad20 0x7f763c1588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:16.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f763bfff640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f763c0ce1d0 0x7f763c15ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f763b7fe640 1 -- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f763c0cad20 msgr2=0x7f763c1588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f763b7fe640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f763c0cad20 0x7f763c1588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f763b7fe640 1 -- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f763c0ce1d0 msgr2=0x7f763c15ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f763b7fe640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f763c0ce1d0 0x7f763c15ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f763b7fe640 1 -- 172.21.15.26:0/1908806526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76300094d0 con 0x7f763c06d100 2023-12-19T17:57:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f763b7fe640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f763c06d100 0x7f763c06fa70 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f762c00c3c0 tx=0x7f762c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f7623fff640 1 -- 172.21.15.26:0/1908806526 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c0052d0 con 0x7f763c06d100 2023-12-19T17:57:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f7623fff640 1 -- 172.21.15.26:0/1908806526 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762c005430 con 0x7f763c06d100 2023-12-19T17:57:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f7623fff640 1 -- 172.21.15.26:0/1908806526 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c003840 con 0x7f763c06d100 2023-12-19T17:57:16.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f76409a2640 1 -- 172.21.15.26:0/1908806526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c070560 con 0x7f763c06d100 2023-12-19T17:57:16.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f76409a2640 1 -- 172.21.15.26:0/1908806526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c070b20 con 0x7f763c06d100 2023-12-19T17:57:16.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f76409a2640 1 -- 172.21.15.26:0/1908806526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c070d10 con 0x7f763c06d100 2023-12-19T17:57:16.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.103+0000 7f7623fff640 1 -- 172.21.15.26:0/1908806526 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 75) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f762c0039a0 con 0x7f763c06d100 2023-12-19T17:57:16.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.107+0000 7f7623fff640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7628065bd0 0x7f7628068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:16.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.107+0000 7f7623fff640 1 -- 172.21.15.26:0/1908806526 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f762c082b30 con 0x7f763c06d100 2023-12-19T17:57:16.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.107+0000 7f763affd640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7628065bd0 0x7f7628068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:16.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.107+0000 7f763affd640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7628065bd0 0x7f7628068090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f76300ae000 tx=0x7f76300a7000 comp rx=0 tx=0).ready entity=mgr.8245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:16.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.111+0000 7f7623fff640 1 -- 172.21.15.26:0/1908806526 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f762c04ffe0 con 0x7f763c06d100 2023-12-19T17:57:16.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.287+0000 7f76409a2640 1 -- 172.21.15.26:0/1908806526 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f763c0703f0 con 0x7f7628065bd0 2023-12-19T17:57:16.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.295+0000 7f7623fff640 1 -- 172.21.15.26:0/1908806526 <== mgr.8245 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f763c0703f0 con 0x7f7628065bd0 2023-12-19T17:57:16.299 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:16.299 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T17:01:33", "first": "2023-12-19T17:02:00", "last": "2023-12-19T17:57:00", "last_pruned": "2023-12-19T17:57:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-12-19T17:57:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.299+0000 7f76409a2640 1 -- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7628065bd0 msgr2=0x7f7628068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.299+0000 7f76409a2640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7628065bd0 0x7f7628068090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f76300ae000 tx=0x7f76300a7000 comp rx=0 tx=0).stop 2023-12-19T17:57:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.299+0000 7f76409a2640 1 -- 172.21.15.26:0/1908806526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f763c06d100 msgr2=0x7f763c06fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.299+0000 7f76409a2640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f763c06d100 0x7f763c06fa70 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f762c00c3c0 tx=0x7f762c00c880 comp rx=0 tx=0).stop 2023-12-19T17:57:16.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.299+0000 7f76409a2640 1 -- 172.21.15.26:0/1908806526 shutdown_connections 2023-12-19T17:57:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.299+0000 7f76409a2640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f763c0ce1d0 0x7f763c15ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.299+0000 7f76409a2640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7628065bd0 0x7f7628068090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.299+0000 7f76409a2640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f763c0cad20 0x7f763c1588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.299+0000 7f76409a2640 1 --2- 172.21.15.26:0/1908806526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f763c06d100 0x7f763c06fa70 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.299+0000 7f76409a2640 1 -- 172.21.15.26:0/1908806526 >> 172.21.15.26:0/1908806526 conn(0x7f763c0b0900 msgr2=0x7f763c15e430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.299+0000 7f76409a2640 1 -- 172.21.15.26:0/1908806526 shutdown_connections 2023-12-19T17:57:16.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.299+0000 7f76409a2640 1 -- 172.21.15.26:0/1908806526 wait complete. 2023-12-19T17:57:16.320 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T17:01:33", "first": "2023-12-19T17:02:00", "last": "2023-12-19T17:57:00", "last_pruned": "2023-12-19T17:57:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-12-19T17:57:16.321 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1M', 'retention': {'n': 55}, 'start': '2023-12-19T00:00:00', 'created': '2023-12-19T17:01:33', 'first': '2023-12-19T17:02:00', 'last': '2023-12-19T17:57:00', 'last_pruned': '2023-12-19T17:57:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-12-19T17:57:16.321 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-12-19T17:57:16.321 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-12-19T17:57:16.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1 2023-12-19T17:57:16.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.463+0000 7f7edb441640 1 Processor -- start 2023-12-19T17:57:16.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.463+0000 7f7edb441640 1 -- start start 2023-12-19T17:57:16.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.467+0000 7f7edb441640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ed40adc60 0x7f7ed40ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.467+0000 7f7edb441640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7ed4071100 0x7f7ed406cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.467+0000 7f7edb441640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed4070260 con 0x7f7ed406d010 2023-12-19T17:57:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.467+0000 7f7edb441640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed4071560 con 0x7f7ed4071100 2023-12-19T17:57:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.467+0000 7f7edb441640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed41615b0 con 0x7f7ed40adc60 2023-12-19T17:57:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.467+0000 7f7ed91dd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ed40adc60 0x7f7ed40ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.467+0000 7f7ed91dd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ed40adc60 0x7f7ed40ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:38908/0 (socket says 172.21.15.26:38908) 2023-12-19T17:57:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.467+0000 7f7ed91dd640 1 -- 172.21.15.26:0/2883236605 learned_addr learned my addr 172.21.15.26:0/2883236605 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.467+0000 7f7ed89dc640 1 --2- 172.21.15.26:0/2883236605 >> v2:172.21.15.160:3300/0 conn(0x7f7ed4071100 0x7f7ed406cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.467+0000 7f7ed91dd640 1 -- 172.21.15.26:0/2883236605 >> v2:172.21.15.160:3300/0 conn(0x7f7ed4071100 msgr2=0x7f7ed406cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.467+0000 7f7ed91dd640 1 --2- 172.21.15.26:0/2883236605 >> v2:172.21.15.160:3300/0 conn(0x7f7ed4071100 0x7f7ed406cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.467+0000 7f7ed91dd640 1 -- 172.21.15.26:0/2883236605 >> v1:172.21.15.160:6789/0 conn(0x7f7ed406d010 legacy=0x7f7ed4164ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.467+0000 7f7ed91dd640 1 -- 172.21.15.26:0/2883236605 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed40c8d90 con 0x7f7ed40adc60 2023-12-19T17:57:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7ed91dd640 1 --2- 172.21.15.26:0/2883236605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ed40adc60 0x7f7ed40ca640 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009870 tx=0x7f7ed001c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79fbadfa0cc8fda2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7ecbfff640 1 -- 172.21.15.26:0/2883236605 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00a7070 con 0x7f7ed40adc60 2023-12-19T17:57:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7ecbfff640 1 -- 172.21.15.26:0/2883236605 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed00ab430 con 0x7f7ed40adc60 2023-12-19T17:57:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7ecbfff640 1 -- 172.21.15.26:0/2883236605 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00ab7c0 con 0x7f7ed40adc60 2023-12-19T17:57:16.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7edb441640 1 -- 172.21.15.26:0/2883236605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ed40adc60 msgr2=0x7f7ed40ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:16.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7edb441640 1 --2- 172.21.15.26:0/2883236605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ed40adc60 0x7f7ed40ca640 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009870 tx=0x7f7ed001c510 comp rx=0 tx=0).stop 2023-12-19T17:57:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7edb441640 1 -- 172.21.15.26:0/2883236605 shutdown_connections 2023-12-19T17:57:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7edb441640 1 --2- 172.21.15.26:0/2883236605 >> v2:172.21.15.160:3300/0 conn(0x7f7ed4071100 0x7f7ed406cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7edb441640 1 --2- 172.21.15.26:0/2883236605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ed40adc60 0x7f7ed40ca640 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7edb441640 1 -- 172.21.15.26:0/2883236605 >> 172.21.15.26:0/2883236605 conn(0x7f7ed4066160 msgr2=0x7f7ed40b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7edb441640 1 -- 172.21.15.26:0/2883236605 shutdown_connections 2023-12-19T17:57:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7edb441640 1 -- 172.21.15.26:0/2883236605 wait complete. 2023-12-19T17:57:16.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7edb441640 1 Processor -- start 2023-12-19T17:57:16.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.471+0000 7f7edb441640 1 -- start start 2023-12-19T17:57:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7edb441640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ed406d010 0x7f7ed4158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7edb441640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ed4071100 0x7f7ed40b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7edb441640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed40adc60 0x7f7ed415c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7edb441640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed40b3a10 con 0x7f7ed40adc60 2023-12-19T17:57:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7edb441640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed40b2e00 con 0x7f7ed4071100 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7edb441640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed40b30e0 con 0x7f7ed406d010 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ed99de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed40adc60 0x7f7ed415c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ed99de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed40adc60 0x7f7ed415c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42494/0 (socket says 172.21.15.26:42494) 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ed99de640 1 -- 172.21.15.26:0/3428031359 learned_addr learned my addr 172.21.15.26:0/3428031359 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ed89dc640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ed4071100 0x7f7ed40b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ed91dd640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ed406d010 0x7f7ed4158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ed99de640 1 -- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ed406d010 msgr2=0x7f7ed4158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ed99de640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ed406d010 0x7f7ed4158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ed99de640 1 -- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ed4071100 msgr2=0x7f7ed40b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ed99de640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ed4071100 0x7f7ed40b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ed99de640 1 -- 172.21.15.26:0/3428031359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed00094d0 con 0x7f7ed40adc60 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ed99de640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed40adc60 0x7f7ed415c6c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00c730 tx=0x7f7ecc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ec9ffb640 1 -- 172.21.15.26:0/3428031359 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc003b80 con 0x7f7ed40adc60 2023-12-19T17:57:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ec9ffb640 1 -- 172.21.15.26:0/3428031359 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ecc003ce0 con 0x7f7ed40adc60 2023-12-19T17:57:16.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7ec9ffb640 1 -- 172.21.15.26:0/3428031359 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc00aa30 con 0x7f7ed40adc60 2023-12-19T17:57:16.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7edb441640 1 -- 172.21.15.26:0/3428031359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed40b33c0 con 0x7f7ed40adc60 2023-12-19T17:57:16.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7edb441640 1 -- 172.21.15.26:0/3428031359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed415cc00 con 0x7f7ed40adc60 2023-12-19T17:57:16.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.475+0000 7f7edb441640 1 -- 172.21.15.26:0/3428031359 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed40b3b80 con 0x7f7ed40adc60 2023-12-19T17:57:16.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.479+0000 7f7ec9ffb640 1 -- 172.21.15.26:0/3428031359 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 75) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f7ecc00ab90 con 0x7f7ed40adc60 2023-12-19T17:57:16.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.479+0000 7f7ec9ffb640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7eb8065c00 0x7f7eb80680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:16.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.479+0000 7f7ec9ffb640 1 -- 172.21.15.26:0/3428031359 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6887+0+0 (secure 0 0 0) 0x7f7ecc083400 con 0x7f7ed40adc60 2023-12-19T17:57:16.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.479+0000 7f7ed91dd640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7eb8065c00 0x7f7eb80680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:16.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.479+0000 7f7ed91dd640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7eb8065c00 0x7f7eb80680c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00af000 tx=0x7f7ed00a7000 comp rx=0 tx=0).ready entity=mgr.8245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:16.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.483+0000 7f7ec9ffb640 1 -- 172.21.15.26:0/3428031359 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7ecc0508c0 con 0x7f7ed40adc60 2023-12-19T17:57:16.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.659+0000 7f7edb441640 1 -- 172.21.15.26:0/3428031359 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ed40c9910 con 0x7f7eb8065c00 2023-12-19T17:57:16.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.735+0000 7f7ec9ffb640 1 -- 172.21.15.26:0/3428031359 <== mgr.8245 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7f7ed00ab860 con 0x7f7eb8065c00 2023-12-19T17:57:16.739 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path /snap_test_dir1 2023-12-19T17:57:16.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.739+0000 7f7edb441640 1 -- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7eb8065c00 msgr2=0x7f7eb80680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:16.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.739+0000 7f7edb441640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7eb8065c00 0x7f7eb80680c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00af000 tx=0x7f7ed00a7000 comp rx=0 tx=0).stop 2023-12-19T17:57:16.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.739+0000 7f7edb441640 1 -- 172.21.15.26:0/3428031359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed40adc60 msgr2=0x7f7ed415c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:16.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.739+0000 7f7edb441640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed40adc60 0x7f7ed415c6c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00c730 tx=0x7f7ecc00cbf0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.739+0000 7f7edb441640 1 -- 172.21.15.26:0/3428031359 shutdown_connections 2023-12-19T17:57:16.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.739+0000 7f7edb441640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ed40adc60 0x7f7ed415c6c0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.739+0000 7f7edb441640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ed4071100 0x7f7ed40b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.739+0000 7f7edb441640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7eb8065c00 0x7f7eb80680c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.739+0000 7f7edb441640 1 --2- 172.21.15.26:0/3428031359 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ed406d010 0x7f7ed4158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:16.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.739+0000 7f7edb441640 1 -- 172.21.15.26:0/3428031359 >> 172.21.15.26:0/3428031359 conn(0x7f7ed4066160 msgr2=0x7f7ed40ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:16.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.739+0000 7f7edb441640 1 -- 172.21.15.26:0/3428031359 shutdown_connections 2023-12-19T17:57:16.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:16.739+0000 7f7edb441640 1 -- 172.21.15.26:0/3428031359 wait complete. 2023-12-19T17:57:16.760 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-12-19T17:57:16.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:16.760 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T17:57:16.773 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_03_00_UTC 2023-12-19T17:57:16.773 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_04_00_UTC 2023-12-19T17:57:16.773 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_05_00_UTC 2023-12-19T17:57:16.773 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_06_00_UTC 2023-12-19T17:57:16.773 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_07_00_UTC 2023-12-19T17:57:16.773 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_08_00_UTC 2023-12-19T17:57:16.773 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_09_00_UTC 2023-12-19T17:57:16.773 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_10_00_UTC 2023-12-19T17:57:16.773 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_11_00_UTC 2023-12-19T17:57:16.773 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_12_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_13_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_14_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_15_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_16_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_17_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_18_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_19_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_20_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_21_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_22_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_23_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_24_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_25_00_UTC 2023-12-19T17:57:16.774 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_26_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_27_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_28_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_29_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_30_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_31_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_32_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_33_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_34_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_35_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_36_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_37_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_38_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_39_00_UTC 2023-12-19T17:57:16.775 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_40_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_41_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_42_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_43_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_44_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_45_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_46_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_47_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_48_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_49_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_50_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_51_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_52_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_53_00_UTC 2023-12-19T17:57:16.776 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_54_00_UTC 2023-12-19T17:57:16.777 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_55_00_UTC 2023-12-19T17:57:16.777 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_56_00_UTC 2023-12-19T17:57:16.777 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_57_00_UTC 2023-12-19T17:57:16.778 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_03_00_UTC 2023-12-19T17:57:16.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:16.778 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_03_00_UTC) 2023-12-19T17:57:16.838 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_04_00_UTC 2023-12-19T17:57:16.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:16.838 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_04_00_UTC) 2023-12-19T17:57:16.893 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_05_00_UTC 2023-12-19T17:57:16.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:16.894 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_05_00_UTC) 2023-12-19T17:57:16.962 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_06_00_UTC 2023-12-19T17:57:16.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:16.962 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_06_00_UTC) 2023-12-19T17:57:17.021 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_07_00_UTC 2023-12-19T17:57:17.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.022 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_07_00_UTC) 2023-12-19T17:57:17.078 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_08_00_UTC 2023-12-19T17:57:17.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.078 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_08_00_UTC) 2023-12-19T17:57:17.133 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_09_00_UTC 2023-12-19T17:57:17.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.134 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_09_00_UTC) 2023-12-19T17:57:17.190 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_10_00_UTC 2023-12-19T17:57:17.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.190 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_10_00_UTC) 2023-12-19T17:57:17.245 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_11_00_UTC 2023-12-19T17:57:17.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.246 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_11_00_UTC) 2023-12-19T17:57:17.302 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_12_00_UTC 2023-12-19T17:57:17.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.302 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_12_00_UTC) 2023-12-19T17:57:17.360 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_13_00_UTC 2023-12-19T17:57:17.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.360 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_13_00_UTC) 2023-12-19T17:57:17.417 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_14_00_UTC 2023-12-19T17:57:17.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.418 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_14_00_UTC) 2023-12-19T17:57:17.474 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_15_00_UTC 2023-12-19T17:57:17.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.474 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_15_00_UTC) 2023-12-19T17:57:17.530 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_16_00_UTC 2023-12-19T17:57:17.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.530 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_16_00_UTC) 2023-12-19T17:57:17.586 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_17_00_UTC 2023-12-19T17:57:17.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.586 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_17_00_UTC) 2023-12-19T17:57:17.642 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_18_00_UTC 2023-12-19T17:57:17.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.642 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_18_00_UTC) 2023-12-19T17:57:17.698 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_19_00_UTC 2023-12-19T17:57:17.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.698 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_19_00_UTC) 2023-12-19T17:57:17.754 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_20_00_UTC 2023-12-19T17:57:17.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.754 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_20_00_UTC) 2023-12-19T17:57:17.809 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_21_00_UTC 2023-12-19T17:57:17.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.809 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_21_00_UTC) 2023-12-19T17:57:17.866 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_22_00_UTC 2023-12-19T17:57:17.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.866 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_22_00_UTC) 2023-12-19T17:57:17.921 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_23_00_UTC 2023-12-19T17:57:17.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.922 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_23_00_UTC) 2023-12-19T17:57:17.982 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_24_00_UTC 2023-12-19T17:57:17.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:17.983 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_24_00_UTC) 2023-12-19T17:57:18.038 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_25_00_UTC 2023-12-19T17:57:18.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.038 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_25_00_UTC) 2023-12-19T17:57:18.094 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_26_00_UTC 2023-12-19T17:57:18.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.094 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_26_00_UTC) 2023-12-19T17:57:18.149 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_27_00_UTC 2023-12-19T17:57:18.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.150 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_27_00_UTC) 2023-12-19T17:57:18.206 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_28_00_UTC 2023-12-19T17:57:18.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.206 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_28_00_UTC) 2023-12-19T17:57:18.261 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_29_00_UTC 2023-12-19T17:57:18.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.262 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_29_00_UTC) 2023-12-19T17:57:18.321 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_30_00_UTC 2023-12-19T17:57:18.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.321 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_30_00_UTC) 2023-12-19T17:57:18.378 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_31_00_UTC 2023-12-19T17:57:18.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.378 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_31_00_UTC) 2023-12-19T17:57:18.433 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_32_00_UTC 2023-12-19T17:57:18.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.434 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_32_00_UTC) 2023-12-19T17:57:18.490 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_33_00_UTC 2023-12-19T17:57:18.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.490 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_33_00_UTC) 2023-12-19T17:57:18.546 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_34_00_UTC 2023-12-19T17:57:18.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.546 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_34_00_UTC) 2023-12-19T17:57:18.601 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_35_00_UTC 2023-12-19T17:57:18.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.602 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_35_00_UTC) 2023-12-19T17:57:18.658 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_36_00_UTC 2023-12-19T17:57:18.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.658 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_36_00_UTC) 2023-12-19T17:57:18.714 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_37_00_UTC 2023-12-19T17:57:18.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.714 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_37_00_UTC) 2023-12-19T17:57:18.770 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_38_00_UTC 2023-12-19T17:57:18.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.770 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_38_00_UTC) 2023-12-19T17:57:18.826 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_39_00_UTC 2023-12-19T17:57:18.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.826 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_39_00_UTC) 2023-12-19T17:57:18.881 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_40_00_UTC 2023-12-19T17:57:18.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.882 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_40_00_UTC) 2023-12-19T17:57:18.938 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_41_00_UTC 2023-12-19T17:57:18.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:18.938 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_41_00_UTC) 2023-12-19T17:57:19.045 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_42_00_UTC 2023-12-19T17:57:19.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.045 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_42_00_UTC) 2023-12-19T17:57:19.079 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_43_00_UTC 2023-12-19T17:57:19.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.079 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_43_00_UTC) 2023-12-19T17:57:19.133 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_44_00_UTC 2023-12-19T17:57:19.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.134 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_44_00_UTC) 2023-12-19T17:57:19.205 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_45_00_UTC 2023-12-19T17:57:19.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.206 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_45_00_UTC) 2023-12-19T17:57:19.277 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_46_00_UTC 2023-12-19T17:57:19.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.278 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_46_00_UTC) 2023-12-19T17:57:19.334 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_47_00_UTC 2023-12-19T17:57:19.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.335 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_47_00_UTC) 2023-12-19T17:57:19.390 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_48_00_UTC 2023-12-19T17:57:19.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.390 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_48_00_UTC) 2023-12-19T17:57:19.445 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_49_00_UTC 2023-12-19T17:57:19.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.446 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_49_00_UTC) 2023-12-19T17:57:19.502 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_50_00_UTC 2023-12-19T17:57:19.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.502 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_50_00_UTC) 2023-12-19T17:57:19.558 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_51_00_UTC 2023-12-19T17:57:19.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.558 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_51_00_UTC) 2023-12-19T17:57:19.614 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_52_00_UTC 2023-12-19T17:57:19.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.614 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_52_00_UTC) 2023-12-19T17:57:19.670 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_53_00_UTC 2023-12-19T17:57:19.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.670 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_53_00_UTC) 2023-12-19T17:57:19.726 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_54_00_UTC 2023-12-19T17:57:19.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.726 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_54_00_UTC) 2023-12-19T17:57:19.782 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_55_00_UTC 2023-12-19T17:57:19.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.782 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_55_00_UTC) 2023-12-19T17:57:19.837 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_56_00_UTC 2023-12-19T17:57:19.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.837 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_56_00_UTC) 2023-12-19T17:57:19.893 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-17_57_00_UTC 2023-12-19T17:57:19.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.893 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-17_57_00_UTC) 2023-12-19T17:57:19.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:19.950 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-19T17:57:20.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-19T17:57:20.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.195+0000 7f00bbd18640 1 Processor -- start 2023-12-19T17:57:20.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.195+0000 7f00bbd18640 1 -- start start 2023-12-19T17:57:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.195+0000 7f00bbd18640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00b40cb150 0x7f00b40cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.195+0000 7f00bbd18640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b40ce3f0 con 0x7f00b415e690 2023-12-19T17:57:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.195+0000 7f00bbd18640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b415b1b0 con 0x7f00b41585f0 2023-12-19T17:57:20.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.195+0000 7f00bbd18640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b40cbb10 con 0x7f00b40cb150 2023-12-19T17:57:20.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.195+0000 7f00ba2b5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f00b415e690 0x7f00b415eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33212/0 (socket says 172.21.15.26:33212) 2023-12-19T17:57:20.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.195+0000 7f00ba2b5640 1 -- 172.21.15.26:0/3013920856 learned_addr learned my addr 172.21.15.26:0/3013920856 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.195+0000 7f00b92b3640 1 --2- 172.21.15.26:0/3013920856 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00b40cb150 0x7f00b40cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.195+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285545000 0 0) 0x7f00b40ce3f0 con 0x7f00b415e690 2023-12-19T17:57:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.195+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00a0003530 con 0x7f00b415e690 2023-12-19T17:57:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043796808 0 0) 0x7f00b415b1b0 con 0x7f00b41585f0 2023-12-19T17:57:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00b40ce3f0 con 0x7f00b41585f0 2023-12-19T17:57:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2370169935 0 0) 0x7f00a0003530 con 0x7f00b415e690 2023-12-19T17:57:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00b40cb150 msgr2=0x7f00b40cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 --2- 172.21.15.26:0/3013920856 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00b40cb150 0x7f00b40cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 >> v1:172.21.15.160:6789/0 conn(0x7f00b41585f0 legacy=0x7f00b40cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b415b490 con 0x7f00b415e690 2023-12-19T17:57:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f00ac001a00 con 0x7f00b415e690 2023-12-19T17:57:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 >> v1:172.21.15.26:6789/0 conn(0x7f00b415e690 legacy=0x7f00b415eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 --2- 172.21.15.26:0/3013920856 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00b40cb150 0x7f00a0003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 --2- 172.21.15.26:0/3013920856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00a00040a0 0x7f00a00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 --2- 172.21.15.26:0/3013920856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a0004d10 0x7f00a0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b40cbb10 con 0x7f00a00040a0 2023-12-19T17:57:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a000bb20 con 0x7f00a0004d10 2023-12-19T17:57:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a000bcb0 con 0x7f00b40cb150 2023-12-19T17:57:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b92b3640 1 --2- 172.21.15.26:0/3013920856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00a00040a0 0x7f00a00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b9ab4640 1 --2- 172.21.15.26:0/3013920856 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00b40cb150 0x7f00a0003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00ba2b5640 1 --2- 172.21.15.26:0/3013920856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a0004d10 0x7f00a0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b92b3640 1 -- 172.21.15.26:0/3013920856 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00b40cb150 msgr2=0x7f00a0003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b92b3640 1 --2- 172.21.15.26:0/3013920856 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00b40cb150 0x7f00a0003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b92b3640 1 -- 172.21.15.26:0/3013920856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a0004d10 msgr2=0x7f00a0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b92b3640 1 --2- 172.21.15.26:0/3013920856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a0004d10 0x7f00a0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:20.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.199+0000 7f00b92b3640 1 -- 172.21.15.26:0/3013920856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a0003530 con 0x7f00a00040a0 2023-12-19T17:57:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.203+0000 7f00b92b3640 1 --2- 172.21.15.26:0/3013920856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00a00040a0 0x7f00a00047d0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009ae0 tx=0x7f00a401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd907cc93ba1fe62 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.203+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a40a6430 con 0x7f00a00040a0 2023-12-19T17:57:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.203+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a40a6590 con 0x7f00a00040a0 2023-12-19T17:57:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.203+0000 7f00b8ab2640 1 -- 172.21.15.26:0/3013920856 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a40a6940 con 0x7f00a00040a0 2023-12-19T17:57:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.203+0000 7f00bbd18640 1 -- 172.21.15.26:0/3013920856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00a00040a0 msgr2=0x7f00a00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:20.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.203+0000 7f00bbd18640 1 --2- 172.21.15.26:0/3013920856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00a00040a0 0x7f00a00047d0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009ae0 tx=0x7f00a401ca50 comp rx=0 tx=0).stop 2023-12-19T17:57:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.203+0000 7f00ba2b5640 1 -- 172.21.15.26:0/3013920856 reap_dead start 2023-12-19T17:57:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.203+0000 7f00bbd18640 1 -- 172.21.15.26:0/3013920856 shutdown_connections 2023-12-19T17:57:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.203+0000 7f00bbd18640 1 -- 172.21.15.26:0/3013920856 >> 172.21.15.26:0/3013920856 conn(0x7f00b4066160 msgr2=0x7f00b40b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.203+0000 7f00bbd18640 1 -- 172.21.15.26:0/3013920856 shutdown_connections 2023-12-19T17:57:20.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.203+0000 7f00bbd18640 1 -- 172.21.15.26:0/3013920856 wait complete. 2023-12-19T17:57:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.203+0000 7f00bbd18640 1 Processor -- start 2023-12-19T17:57:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.203+0000 7f00bbd18640 1 -- start start 2023-12-19T17:57:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00bbd18640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a00040a0 0x7f00b40c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00bbd18640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00b40c80a0 0x7f00b40ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00bbd18640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00b4164a60 0x7f00b4164ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00bbd18640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b4165420 con 0x7f00b4164a60 2023-12-19T17:57:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00bbd18640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b40c77d0 con 0x7f00a00040a0 2023-12-19T17:57:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00bbd18640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b41fa540 con 0x7f00b40c80a0 2023-12-19T17:57:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00ba2b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00b4164a60 0x7f00b4164ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:20.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00ba2b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00b4164a60 0x7f00b4164ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42516/0 (socket says 172.21.15.26:42516) 2023-12-19T17:57:20.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00b9ab4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a00040a0 0x7f00b40c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:20.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00ba2b5640 1 -- 172.21.15.26:0/336192430 learned_addr learned my addr 172.21.15.26:0/336192430 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:20.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00b92b3640 1 --2- 172.21.15.26:0/336192430 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00b40c80a0 0x7f00b40ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00ba2b5640 1 -- 172.21.15.26:0/336192430 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00b40c80a0 msgr2=0x7f00b40ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00ba2b5640 1 --2- 172.21.15.26:0/336192430 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00b40c80a0 0x7f00b40ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00ba2b5640 1 -- 172.21.15.26:0/336192430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a00040a0 msgr2=0x7f00b40c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00ba2b5640 1 --2- 172.21.15.26:0/336192430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a00040a0 0x7f00b40c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00ba2b5640 1 -- 172.21.15.26:0/336192430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b41fa820 con 0x7f00b4164a60 2023-12-19T17:57:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00ba2b5640 1 --2- 172.21.15.26:0/336192430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00b4164a60 0x7f00b4164ee0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f00ac000f90 tx=0x7f00ac007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00aa7fc640 1 -- 172.21.15.26:0/336192430 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00ea80 con 0x7f00b4164a60 2023-12-19T17:57:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00aa7fc640 1 -- 172.21.15.26:0/336192430 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac00ebe0 con 0x7f00b4164a60 2023-12-19T17:57:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00aa7fc640 1 -- 172.21.15.26:0/336192430 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac013420 con 0x7f00b4164a60 2023-12-19T17:57:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00bbd18640 1 -- 172.21.15.26:0/336192430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b41fab00 con 0x7f00b4164a60 2023-12-19T17:57:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00bbd18640 1 -- 172.21.15.26:0/336192430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b41fb0c0 con 0x7f00b4164a60 2023-12-19T17:57:20.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.207+0000 7f00bbd18640 1 -- 172.21.15.26:0/336192430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b41fd850 con 0x7f00b4164a60 2023-12-19T17:57:20.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.211+0000 7f00aa7fc640 1 -- 172.21.15.26:0/336192430 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 75) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f00ac013580 con 0x7f00b4164a60 2023-12-19T17:57:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.211+0000 7f00aa7fc640 1 --2- 172.21.15.26:0/336192430 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0098065c00 0x7f00980680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:20.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.211+0000 7f00aa7fc640 1 -- 172.21.15.26:0/336192430 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f00ac083aa0 con 0x7f00b4164a60 2023-12-19T17:57:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.211+0000 7f00b9ab4640 1 --2- 172.21.15.26:0/336192430 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0098065c00 0x7f00980680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.211+0000 7f00b9ab4640 1 --2- 172.21.15.26:0/336192430 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0098065c00 0x7f00980680c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f00b0007af0 tx=0x7f00b0009040 comp rx=0 tx=0).ready entity=mgr.8245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:20.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.215+0000 7f00aa7fc640 1 -- 172.21.15.26:0/336192430 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f00ac050f30 con 0x7f00b4164a60 2023-12-19T17:57:20.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.511+0000 7f00bbd18640 1 -- 172.21.15.26:0/336192430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f00b4074270 con 0x7f00b4164a60 2023-12-19T17:57:20.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.979+0000 7f00aa7fc640 1 -- 172.21.15.26:0/336192430 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 76) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f00ac081af0 con 0x7f00b4164a60 2023-12-19T17:57:20.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.979+0000 7f00aa7fc640 1 -- 172.21.15.26:0/336192430 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v76) v1 ==== 93+0+0 (secure 0 0 0) 0x7f00ac07ffc0 con 0x7f00b4164a60 2023-12-19T17:57:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.983+0000 7f00bbd18640 1 -- 172.21.15.26:0/336192430 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0098065c00 msgr2=0x7f00980680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.983+0000 7f00bbd18640 1 --2- 172.21.15.26:0/336192430 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0098065c00 0x7f00980680c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f00b0007af0 tx=0x7f00b0009040 comp rx=0 tx=0).stop 2023-12-19T17:57:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.983+0000 7f00bbd18640 1 -- 172.21.15.26:0/336192430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00b4164a60 msgr2=0x7f00b4164ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.983+0000 7f00bbd18640 1 --2- 172.21.15.26:0/336192430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00b4164a60 0x7f00b4164ee0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f00ac000f90 tx=0x7f00ac007b90 comp rx=0 tx=0).stop 2023-12-19T17:57:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.983+0000 7f00bbd18640 1 -- 172.21.15.26:0/336192430 shutdown_connections 2023-12-19T17:57:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.983+0000 7f00bbd18640 1 --2- 172.21.15.26:0/336192430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00b4164a60 0x7f00b4164ee0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.983+0000 7f00bbd18640 1 --2- 172.21.15.26:0/336192430 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0098065c00 0x7f00980680c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.983+0000 7f00bbd18640 1 --2- 172.21.15.26:0/336192430 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00b40c80a0 0x7f00b40ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.983+0000 7f00bbd18640 1 --2- 172.21.15.26:0/336192430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a00040a0 0x7f00b40c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.983+0000 7f00bbd18640 1 -- 172.21.15.26:0/336192430 >> 172.21.15.26:0/336192430 conn(0x7f00b4066160 msgr2=0x7f00b415dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.983+0000 7f00bbd18640 1 -- 172.21.15.26:0/336192430 shutdown_connections 2023-12-19T17:57:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:20.983+0000 7f00bbd18640 1 -- 172.21.15.26:0/336192430 wait complete. 2023-12-19T17:57:21.009 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T17:57:21.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T17:57:21.114 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T17:57:21.172 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:21.171+0000 7f25cb48d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:57:21.193 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:21.190+0000 7f3ba6bda180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:57:21.224 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T17:57:21.224 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:57:21.225 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:21.235 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:57:21.236 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:21.288 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:57:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f75a2a2d640 1 Processor -- start 2023-12-19T17:57:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f75a2a2d640 1 -- start start 2023-12-19T17:57:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f75a2a2d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f759c06c7f0 0x7f759c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f75a2a2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759c0ccc20 0x7f759c0b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f75a2a2d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759c06cc80 con 0x7f759c15c7f0 2023-12-19T17:57:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f75a2a2d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759c06d770 con 0x7f759c06c7f0 2023-12-19T17:57:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f75a2a2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c06ced0 con 0x7f759c0ccc20 2023-12-19T17:57:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f759b7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f759c06c7f0 0x7f759c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f75a0fca640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759c0ccc20 0x7f759c0b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f759b7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f759c06c7f0 0x7f759c0cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:35354/0 (socket says 172.21.15.26:35354) 2023-12-19T17:57:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f759b7fe640 1 -- 172.21.15.26:0/2593108685 learned_addr learned my addr 172.21.15.26:0/2593108685 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f759bfff640 1 --1- 172.21.15.26:0/2593108685 >> v1:172.21.15.160:6789/0 conn(0x7f759c15c7f0 0x7f759c0adb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:40326/0 (socket says 172.21.15.26:40326) 2023-12-19T17:57:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f759b7fe640 1 -- 172.21.15.26:0/2593108685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759c0ccc20 msgr2=0x7f759c0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f759b7fe640 1 --2- 172.21.15.26:0/2593108685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759c0ccc20 0x7f759c0b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f759b7fe640 1 -- 172.21.15.26:0/2593108685 >> v1:172.21.15.160:6789/0 conn(0x7f759c15c7f0 legacy=0x7f759c0adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.499+0000 7f759b7fe640 1 -- 172.21.15.26:0/2593108685 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c06d1b0 con 0x7f759c06c7f0 2023-12-19T17:57:21.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.503+0000 7f759b7fe640 1 --2- 172.21.15.26:0/2593108685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f759c06c7f0 0x7f759c0cc6e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7590006fd0 tx=0x7f759001e9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77cfe38f038bb77a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:21.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.503+0000 7f759affd640 1 -- 172.21.15.26:0/2593108685 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590004230 con 0x7f759c06c7f0 2023-12-19T17:57:21.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.503+0000 7f759affd640 1 -- 172.21.15.26:0/2593108685 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75900ab070 con 0x7f759c06c7f0 2023-12-19T17:57:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.503+0000 7f759affd640 1 -- 172.21.15.26:0/2593108685 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590003620 con 0x7f759c06c7f0 2023-12-19T17:57:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.503+0000 7f75a2a2d640 1 -- 172.21.15.26:0/2593108685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f759c06c7f0 msgr2=0x7f759c0cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.503+0000 7f75a2a2d640 1 --2- 172.21.15.26:0/2593108685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f759c06c7f0 0x7f759c0cc6e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7590006fd0 tx=0x7f759001e9d0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.503+0000 7f75a2a2d640 1 -- 172.21.15.26:0/2593108685 shutdown_connections 2023-12-19T17:57:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.503+0000 7f75a2a2d640 1 --2- 172.21.15.26:0/2593108685 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759c0ccc20 0x7f759c0b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.503+0000 7f75a2a2d640 1 --2- 172.21.15.26:0/2593108685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f759c06c7f0 0x7f759c0cc6e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.503+0000 7f75a2a2d640 1 -- 172.21.15.26:0/2593108685 >> 172.21.15.26:0/2593108685 conn(0x7f759c0658e0 msgr2=0x7f759c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.503+0000 7f75a2a2d640 1 -- 172.21.15.26:0/2593108685 shutdown_connections 2023-12-19T17:57:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.503+0000 7f75a2a2d640 1 -- 172.21.15.26:0/2593108685 wait complete. 2023-12-19T17:57:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f75a2a2d640 1 Processor -- start 2023-12-19T17:57:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f75a2a2d640 1 -- start start 2023-12-19T17:57:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f75a2a2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f759c06c7f0 0x7f759c155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f75a2a2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f759c0ccc20 0x7f759c160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f75a2a2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759c15c7f0 0x7f759c158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f75a2a2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c15e9c0 con 0x7f759c0ccc20 2023-12-19T17:57:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f75a2a2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c156110 con 0x7f759c06c7f0 2023-12-19T17:57:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f75a2a2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c1613c0 con 0x7f759c15c7f0 2023-12-19T17:57:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f759b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f759c0ccc20 0x7f759c160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f759b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f759c0ccc20 0x7f759c160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42538/0 (socket says 172.21.15.26:42538) 2023-12-19T17:57:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f759b7fe640 1 -- 172.21.15.26:0/320722308 learned_addr learned my addr 172.21.15.26:0/320722308 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f759bfff640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f759c06c7f0 0x7f759c155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f75a0fca640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759c15c7f0 0x7f759c158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f759b7fe640 1 -- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759c15c7f0 msgr2=0x7f759c158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:21.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f759b7fe640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759c15c7f0 0x7f759c158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f759b7fe640 1 -- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f759c06c7f0 msgr2=0x7f759c155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f759b7fe640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f759c06c7f0 0x7f759c155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f759b7fe640 1 -- 172.21.15.26:0/320722308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759000b780 con 0x7f759c0ccc20 2023-12-19T17:57:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f75a0fca640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759c15c7f0 0x7f759c158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f759b7fe640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f759c0ccc20 0x7f759c160900 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f759001d080 tx=0x7f759001b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f7598ff9640 1 -- 172.21.15.26:0/320722308 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900a88c0 con 0x7f759c0ccc20 2023-12-19T17:57:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f7598ff9640 1 -- 172.21.15.26:0/320722308 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75900a8a20 con 0x7f759c0ccc20 2023-12-19T17:57:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f75a2a2d640 1 -- 172.21.15.26:0/320722308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759c1616a0 con 0x7f759c0ccc20 2023-12-19T17:57:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f7598ff9640 1 -- 172.21.15.26:0/320722308 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900a8d90 con 0x7f759c0ccc20 2023-12-19T17:57:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.507+0000 7f75a2a2d640 1 -- 172.21.15.26:0/320722308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c15e5e0 con 0x7f759c0ccc20 2023-12-19T17:57:21.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.511+0000 7f75a2a2d640 1 -- 172.21.15.26:0/320722308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759c074270 con 0x7f759c0ccc20 2023-12-19T17:57:21.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.511+0000 7f7598ff9640 1 -- 172.21.15.26:0/320722308 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 76) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f7590003620 con 0x7f759c0ccc20 2023-12-19T17:57:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.511+0000 7f7598ff9640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7588065b80 0x7f7588068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.511+0000 7f7598ff9640 1 -- 172.21.15.26:0/320722308 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f759011dd60 con 0x7f759c0ccc20 2023-12-19T17:57:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.511+0000 7f759bfff640 1 -- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7588065b80 msgr2=0x7f7588068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:21.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.511+0000 7f759bfff640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7588065b80 0x7f7588068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:21.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.515+0000 7f7598ff9640 1 -- 172.21.15.26:0/320722308 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f75900eb1e0 con 0x7f759c0ccc20 2023-12-19T17:57:21.659 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:21.654+0000 7f3ba6bda180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:57:21.711 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:21.707+0000 7f25cb48d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:57:21.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.715+0000 7f759bfff640 1 -- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7588065b80 msgr2=0x7f7588068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:21.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.715+0000 7f759bfff640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7588065b80 0x7f7588068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T17:57:21.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f75a2a2d640 1 -- 172.21.15.26:0/320722308 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f759c0ce1a0 con 0x7f759c0ccc20 2023-12-19T17:57:21.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f7598ff9640 1 -- 172.21.15.26:0/320722308 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+17980 (secure 0 0 0) 0x7f75900c0090 con 0x7f759c0ccc20 2023-12-19T17:57:21.729 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:21.730 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":147,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:57:20.951684+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:01:19.682876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T17:01:20.140612+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":147,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.f","mappings":[{"from":0,"to":2}]},{"pgid":"17.3f","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6833/38098":"2023-12-20T17:01:32.971594+0000","172.21.15.160:6832/38098":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/1067813784":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/1947760249":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/464310353":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/2140731666":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/870873437":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/950669571":"2023-12-20T17:01:32.971594+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":17,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":40,"length":17},{"begin":96,"length":17}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:57:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f75a2a2d640 1 -- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7588065b80 msgr2=0x7f7588068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f75a2a2d640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7588065b80 0x7f7588068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f75a2a2d640 1 -- 172.21.15.26:0/320722308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f759c0ccc20 msgr2=0x7f759c160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f75a2a2d640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f759c0ccc20 0x7f759c160900 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f759001d080 tx=0x7f759001b770 comp rx=0 tx=0).stop 2023-12-19T17:57:21.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f75a2a2d640 1 -- 172.21.15.26:0/320722308 shutdown_connections 2023-12-19T17:57:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f75a2a2d640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7588065b80 0x7f7588068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f75a2a2d640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f759c15c7f0 0x7f759c158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f75a2a2d640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f759c0ccc20 0x7f759c160900 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f75a2a2d640 1 --2- 172.21.15.26:0/320722308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f759c06c7f0 0x7f759c155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f75a2a2d640 1 -- 172.21.15.26:0/320722308 >> 172.21.15.26:0/320722308 conn(0x7f759c0658e0 msgr2=0x7f759c15b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f75a2a2d640 1 -- 172.21.15.26:0/320722308 shutdown_connections 2023-12-19T17:57:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.727+0000 7f75a2a2d640 1 -- 172.21.15.26:0/320722308 wait complete. 2023-12-19T17:57:21.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T17:01:19.682876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T17:01:20.140612+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 147, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T17:57:21.752 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T17:57:21.825 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:21.822+0000 7f3ba6bda180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:57:21.903 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:21.899+0000 7f25cb48d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:57:21.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf241f3640 1 Processor -- start 2023-12-19T17:57:21.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf241f3640 1 -- start start 2023-12-19T17:57:21.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf241f3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdf1c15a9c0 0x7fdf1c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:21.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf241f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf1c06ac80 0x7fdf1c0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:21.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf241f3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf1c070d90 con 0x7fdf1c15ada0 2023-12-19T17:57:21.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf241f3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1c0cc270 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf241f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c071070 con 0x7fdf1c06ac80 2023-12-19T17:57:21.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf2178e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdf1c15ada0 0x7fdf1c0b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33226/0 (socket says 172.21.15.26:33226) 2023-12-19T17:57:21.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf2178e640 1 -- 172.21.15.26:0/3080406648 learned_addr learned my addr 172.21.15.26:0/3080406648 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:21.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf22790640 1 --2- 172.21.15.26:0/3080406648 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf1c06ac80 0x7fdf1c0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:21.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf21f8f640 1 --2- 172.21.15.26:0/3080406648 >> v2:172.21.15.160:3300/0 conn(0x7fdf1c15a9c0 0x7fdf1c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:21.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf20f8d640 1 -- 172.21.15.26:0/3080406648 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619959816 0 0) 0x7fdf1c070d90 con 0x7fdf1c15ada0 2023-12-19T17:57:21.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf20f8d640 1 -- 172.21.15.26:0/3080406648 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf08003530 con 0x7fdf1c15ada0 2023-12-19T17:57:21.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf21f8f640 1 -- 172.21.15.26:0/3080406648 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf1c06ac80 msgr2=0x7fdf1c0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:21.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf21f8f640 1 --2- 172.21.15.26:0/3080406648 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf1c06ac80 0x7fdf1c0b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf21f8f640 1 -- 172.21.15.26:0/3080406648 >> v1:172.21.15.26:6789/0 conn(0x7fdf1c15ada0 legacy=0x7fdf1c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:21.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf21f8f640 1 -- 172.21.15.26:0/3080406648 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c071350 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.927+0000 7fdf22790640 1 --2- 172.21.15.26:0/3080406648 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf1c06ac80 0x7fdf1c0b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:21.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.931+0000 7fdf21f8f640 1 --2- 172.21.15.26:0/3080406648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c15a9c0 0x7fdf1c0706d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdf180097c0 tx=0x7fdf1801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcbcbe78e40a6ce3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:21.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.931+0000 7fdf20f8d640 1 -- 172.21.15.26:0/3080406648 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf180ab600 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.931+0000 7fdf20f8d640 1 -- 172.21.15.26:0/3080406648 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf180a7050 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.931+0000 7fdf20f8d640 1 -- 172.21.15.26:0/3080406648 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf180ab960 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.931+0000 7fdf241f3640 1 -- 172.21.15.26:0/3080406648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c15a9c0 msgr2=0x7fdf1c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:21.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.931+0000 7fdf241f3640 1 --2- 172.21.15.26:0/3080406648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c15a9c0 0x7fdf1c0706d0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdf180097c0 tx=0x7fdf1801c3c0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.931+0000 7fdf241f3640 1 -- 172.21.15.26:0/3080406648 shutdown_connections 2023-12-19T17:57:21.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.931+0000 7fdf241f3640 1 --2- 172.21.15.26:0/3080406648 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf1c06ac80 0x7fdf1c0b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.931+0000 7fdf241f3640 1 --2- 172.21.15.26:0/3080406648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c15a9c0 0x7fdf1c0706d0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.931+0000 7fdf241f3640 1 -- 172.21.15.26:0/3080406648 >> 172.21.15.26:0/3080406648 conn(0x7fdf1c0ce730 msgr2=0x7fdf1c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:21.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.935+0000 7fdf241f3640 1 -- 172.21.15.26:0/3080406648 shutdown_connections 2023-12-19T17:57:21.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.935+0000 7fdf241f3640 1 -- 172.21.15.26:0/3080406648 wait complete. 2023-12-19T17:57:21.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.935+0000 7fdf241f3640 1 Processor -- start 2023-12-19T17:57:21.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.935+0000 7fdf241f3640 1 -- start start 2023-12-19T17:57:21.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf241f3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c06ac80 0x7fdf1c0c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:21.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf241f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf1c15a9c0 0x7fdf1c154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:21.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf241f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf1c15ada0 0x7fdf1c0c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:21.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf241f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c155250 con 0x7fdf1c15ada0 2023-12-19T17:57:21.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf241f3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c155530 con 0x7fdf1c06ac80 2023-12-19T17:57:21.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf241f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c155810 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf22790640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf1c15ada0 0x7fdf1c0c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:21.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf2178e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf1c15a9c0 0x7fdf1c154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:21.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf21f8f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c06ac80 0x7fdf1c0c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:21.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf2178e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf1c15a9c0 0x7fdf1c154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:39002/0 (socket says 172.21.15.26:39002) 2023-12-19T17:57:21.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf2178e640 1 -- 172.21.15.26:0/3718026525 learned_addr learned my addr 172.21.15.26:0/3718026525 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:21.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf21f8f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c06ac80 0x7fdf1c0c4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:35378/0 (socket says 172.21.15.26:35378) 2023-12-19T17:57:21.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf22790640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf1c15ada0 0x7fdf1c0c1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42546/0 (socket says 172.21.15.26:42546) 2023-12-19T17:57:21.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf2178e640 1 -- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c06ac80 msgr2=0x7fdf1c0c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:21.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf2178e640 1 --2- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c06ac80 0x7fdf1c0c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf2178e640 1 -- 172.21.15.26:0/3718026525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf1c15ada0 msgr2=0x7fdf1c0c1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:21.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf2178e640 1 --2- 172.21.15.26:0/3718026525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf1c15ada0 0x7fdf1c0c1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:21.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf2178e640 1 -- 172.21.15.26:0/3718026525 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf14009a70 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf21f8f640 1 --2- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c06ac80 0x7fdf1c0c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:21.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf22790640 1 --2- 172.21.15.26:0/3718026525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf1c15ada0 0x7fdf1c0c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:21.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf2178e640 1 --2- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf1c15a9c0 0x7fdf1c154920 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00c450 tx=0x7fdf0c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:21.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf127fc640 1 -- 172.21.15.26:0/3718026525 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c005cf0 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf127fc640 1 -- 172.21.15.26:0/3718026525 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c005e50 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf127fc640 1 -- 172.21.15.26:0/3718026525 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0072f0 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf241f3640 1 -- 172.21.15.26:0/3718026525 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf18009420 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf241f3640 1 -- 172.21.15.26:0/3718026525 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1c155dd0 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.939+0000 7fdf241f3640 1 -- 172.21.15.26:0/3718026525 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1c0c89b0 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.943+0000 7fdf127fc640 1 -- 172.21.15.26:0/3718026525 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 76) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fdf0c007450 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.943+0000 7fdf127fc640 1 --2- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf00065b80 0x7fdf00068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:21.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.943+0000 7fdf127fc640 1 -- 172.21.15.26:0/3718026525 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6931+0+0 (secure 0 0 0) 0x7fdf0c082d80 con 0x7fdf1c15a9c0 2023-12-19T17:57:21.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.943+0000 7fdf21f8f640 1 -- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf00065b80 msgr2=0x7fdf00068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:21.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.943+0000 7fdf21f8f640 1 --2- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf00065b80 0x7fdf00068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:21.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:21.947+0000 7fdf127fc640 1 -- 172.21.15.26:0/3718026525 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdf0c050170 con 0x7fdf1c15a9c0 2023-12-19T17:57:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.107+0000 7fdf241f3640 1 -- 172.21.15.26:0/3718026525 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf1c155950 con 0x7fdf1c15a9c0 2023-12-19T17:57:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.107+0000 7fdf127fc640 1 -- 172.21.15.26:0/3718026525 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf1c155950 con 0x7fdf1c15a9c0 2023-12-19T17:57:22.110 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T17:57:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.111+0000 7fdf241f3640 1 -- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf00065b80 msgr2=0x7fdf00068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.111+0000 7fdf241f3640 1 --2- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf00065b80 0x7fdf00068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.111+0000 7fdf241f3640 1 -- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf1c15a9c0 msgr2=0x7fdf1c154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.111+0000 7fdf241f3640 1 --2- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf1c15a9c0 0x7fdf1c154920 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00c450 tx=0x7fdf0c00c850 comp rx=0 tx=0).stop 2023-12-19T17:57:22.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.111+0000 7fdf241f3640 1 -- 172.21.15.26:0/3718026525 shutdown_connections 2023-12-19T17:57:22.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.111+0000 7fdf241f3640 1 --2- 172.21.15.26:0/3718026525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf1c15ada0 0x7fdf1c0c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.111+0000 7fdf241f3640 1 --2- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdf00065b80 0x7fdf00068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.111+0000 7fdf241f3640 1 --2- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdf1c15a9c0 0x7fdf1c154920 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.111+0000 7fdf241f3640 1 --2- 172.21.15.26:0/3718026525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdf1c06ac80 0x7fdf1c0c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.111+0000 7fdf241f3640 1 -- 172.21.15.26:0/3718026525 >> 172.21.15.26:0/3718026525 conn(0x7fdf1c0ce730 msgr2=0x7fdf1c06bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:22.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.111+0000 7fdf241f3640 1 -- 172.21.15.26:0/3718026525 shutdown_connections 2023-12-19T17:57:22.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.111+0000 7fdf241f3640 1 -- 172.21.15.26:0/3718026525 wait complete. 2023-12-19T17:57:22.133 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T17:57:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.279+0000 7f6b3398f640 1 Processor -- start 2023-12-19T17:57:22.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.279+0000 7f6b3398f640 1 -- start start 2023-12-19T17:57:22.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.279+0000 7f6b3398f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b2c15d0d0 0x7f6b2c0b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:22.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.279+0000 7f6b3398f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b2c0b4600 0x7f6b2c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:22.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b3398f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c0b4f50 con 0x7f6b2c15ccf0 2023-12-19T17:57:22.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b3398f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b2c15e200 con 0x7f6b2c15d0d0 2023-12-19T17:57:22.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b3398f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0d0aa0 con 0x7f6b2c0b4600 2023-12-19T17:57:22.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b30f2a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b2c15d0d0 0x7f6b2c0b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:22.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b30f2a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b2c15d0d0 0x7f6b2c0b8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42558/0 (socket says 172.21.15.26:42558) 2023-12-19T17:57:22.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b30f2a640 1 -- 172.21.15.26:0/2137955242 learned_addr learned my addr 172.21.15.26:0/2137955242 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:22.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b3172b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6b2c15ccf0 0x7f6b2c0d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33242/0 (socket says 172.21.15.26:33242) 2023-12-19T17:57:22.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b31f2c640 1 --2- 172.21.15.26:0/2137955242 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b2c0b4600 0x7f6b2c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:22.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b30f2a640 1 -- 172.21.15.26:0/2137955242 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b2c0b4600 msgr2=0x7f6b2c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:22.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b30f2a640 1 --2- 172.21.15.26:0/2137955242 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b2c0b4600 0x7f6b2c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b30f2a640 1 -- 172.21.15.26:0/2137955242 >> v1:172.21.15.26:6789/0 conn(0x7f6b2c15ccf0 legacy=0x7f6b2c0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:22.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b30f2a640 1 -- 172.21.15.26:0/2137955242 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c15d680 con 0x7f6b2c15d0d0 2023-12-19T17:57:22.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b23fff640 1 -- 172.21.15.26:0/2137955242 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077140775 0 0) 0x7f6b2c0b4f50 con 0x7f6b2c15ccf0 2023-12-19T17:57:22.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b30f2a640 1 --2- 172.21.15.26:0/2137955242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b2c15d0d0 0x7f6b2c0b8400 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009870 tx=0x7f6b1c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478dfd2e579b094a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:22.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.283+0000 7f6b23fff640 1 -- 172.21.15.26:0/2137955242 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c0aa670 con 0x7f6b2c15d0d0 2023-12-19T17:57:22.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b23fff640 1 -- 172.21.15.26:0/2137955242 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1c0aa7d0 con 0x7f6b2c15d0d0 2023-12-19T17:57:22.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b23fff640 1 -- 172.21.15.26:0/2137955242 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c0aab60 con 0x7f6b2c15d0d0 2023-12-19T17:57:22.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 -- 172.21.15.26:0/2137955242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b2c15d0d0 msgr2=0x7f6b2c0b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:22.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 --2- 172.21.15.26:0/2137955242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b2c15d0d0 0x7f6b2c0b8400 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009870 tx=0x7f6b1c01c510 comp rx=0 tx=0).stop 2023-12-19T17:57:22.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 -- 172.21.15.26:0/2137955242 shutdown_connections 2023-12-19T17:57:22.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 --2- 172.21.15.26:0/2137955242 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b2c0b4600 0x7f6b2c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 --2- 172.21.15.26:0/2137955242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b2c15d0d0 0x7f6b2c0b8400 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 -- 172.21.15.26:0/2137955242 >> 172.21.15.26:0/2137955242 conn(0x7f6b2c065ee0 msgr2=0x7f6b2c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:22.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 -- 172.21.15.26:0/2137955242 shutdown_connections 2023-12-19T17:57:22.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 -- 172.21.15.26:0/2137955242 wait complete. 2023-12-19T17:57:22.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 Processor -- start 2023-12-19T17:57:22.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 -- start start 2023-12-19T17:57:22.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b2c0b4600 0x7f6b2c15b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:22.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2c15ccf0 0x7f6b2c0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:22.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b2c15d0d0 0x7f6b2c153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:22.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c158300 con 0x7f6b2c15d0d0 2023-12-19T17:57:22.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0c8940 con 0x7f6b2c15ccf0 2023-12-19T17:57:22.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3398f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0c8c20 con 0x7f6b2c0b4600 2023-12-19T17:57:22.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b31f2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b2c15d0d0 0x7f6b2c153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:22.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3172b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b2c0b4600 0x7f6b2c15b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:22.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b31f2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b2c15d0d0 0x7f6b2c153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42568/0 (socket says 172.21.15.26:42568) 2023-12-19T17:57:22.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b31f2c640 1 -- 172.21.15.26:0/339971767 learned_addr learned my addr 172.21.15.26:0/339971767 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:22.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.287+0000 7f6b3172b640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b2c0b4600 0x7f6b2c15b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:39018/0 (socket says 172.21.15.26:39018) 2023-12-19T17:57:22.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b30f2a640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2c15ccf0 0x7f6b2c0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:22.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b3172b640 1 -- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2c15ccf0 msgr2=0x7f6b2c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:22.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b3172b640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2c15ccf0 0x7f6b2c0b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b3172b640 1 -- 172.21.15.26:0/339971767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b2c15d0d0 msgr2=0x7f6b2c153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:22.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b3172b640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b2c15d0d0 0x7f6b2c153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b3172b640 1 -- 172.21.15.26:0/339971767 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c0094d0 con 0x7f6b2c0b4600 2023-12-19T17:57:22.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b31f2c640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b2c15d0d0 0x7f6b2c153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:22.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b3172b640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b2c0b4600 0x7f6b2c15b5e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800c430 tx=0x7f6b2800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:22.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b21ffb640 1 -- 172.21.15.26:0/339971767 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b280074e0 con 0x7f6b2c0b4600 2023-12-19T17:57:22.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b21ffb640 1 -- 172.21.15.26:0/339971767 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b28007660 con 0x7f6b2c0b4600 2023-12-19T17:57:22.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b3398f640 1 -- 172.21.15.26:0/339971767 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c157c40 con 0x7f6b2c0b4600 2023-12-19T17:57:22.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b21ffb640 1 -- 172.21.15.26:0/339971767 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b28004a30 con 0x7f6b2c0b4600 2023-12-19T17:57:22.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b3398f640 1 -- 172.21.15.26:0/339971767 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c15bb20 con 0x7f6b2c0b4600 2023-12-19T17:57:22.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b3398f640 1 -- 172.21.15.26:0/339971767 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c074270 con 0x7f6b2c0b4600 2023-12-19T17:57:22.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b21ffb640 1 -- 172.21.15.26:0/339971767 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 76) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f6b28004b90 con 0x7f6b2c0b4600 2023-12-19T17:57:22.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.291+0000 7f6b21ffb640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b10065b80 0x7f6b10068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:22.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.295+0000 7f6b21ffb640 1 -- 172.21.15.26:0/339971767 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f6b28083570 con 0x7f6b2c0b4600 2023-12-19T17:57:22.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.295+0000 7f6b30f2a640 1 -- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b10065b80 msgr2=0x7f6b10068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:22.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.295+0000 7f6b30f2a640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b10065b80 0x7f6b10068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:22.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.299+0000 7f6b21ffb640 1 -- 172.21.15.26:0/339971767 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6b28050a20 con 0x7f6b2c0b4600 2023-12-19T17:57:22.329 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:22.326+0000 7f3ba6bda180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:57:22.386 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:22.382+0000 7f3ba6bda180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:57:22.425 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:22.423+0000 7f25cb48d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:57:22.441 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:22.438+0000 7f3ba6bda180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:57:22.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.455+0000 7f6b3398f640 1 -- 172.21.15.26:0/339971767 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b2c0d19d0 con 0x7f6b2c0b4600 2023-12-19T17:57:22.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.455+0000 7f6b21ffb640 1 -- 172.21.15.26:0/339971767 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v147) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6b28054040 con 0x7f6b2c0b4600 2023-12-19T17:57:22.457 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T17:57:22.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.459+0000 7f6b3398f640 1 -- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b10065b80 msgr2=0x7f6b10068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:22.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.459+0000 7f6b3398f640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b10065b80 0x7f6b10068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.459+0000 7f6b3398f640 1 -- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b2c0b4600 msgr2=0x7f6b2c15b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:22.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.459+0000 7f6b3398f640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b2c0b4600 0x7f6b2c15b5e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800c430 tx=0x7f6b2800c8f0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.459+0000 7f6b3398f640 1 -- 172.21.15.26:0/339971767 shutdown_connections 2023-12-19T17:57:22.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.459+0000 7f6b3398f640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b2c15d0d0 0x7f6b2c153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.459+0000 7f6b3398f640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b2c15ccf0 0x7f6b2c0b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.459+0000 7f6b3398f640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6b10065b80 0x7f6b10068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.459+0000 7f6b3398f640 1 --2- 172.21.15.26:0/339971767 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b2c0b4600 0x7f6b2c15b5e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.459+0000 7f6b3398f640 1 -- 172.21.15.26:0/339971767 >> 172.21.15.26:0/339971767 conn(0x7f6b2c065ee0 msgr2=0x7f6b2c0b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:22.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.459+0000 7f6b3398f640 1 -- 172.21.15.26:0/339971767 shutdown_connections 2023-12-19T17:57:22.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.459+0000 7f6b3398f640 1 -- 172.21.15.26:0/339971767 wait complete. 2023-12-19T17:57:22.488 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T17:57:22.499 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:22.495+0000 7f25cb48d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:57:22.529 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:22.526+0000 7f3ba6bda180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:57:22.559 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:22.555+0000 7f25cb48d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:57:22.585 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:22.582+0000 7f3ba6bda180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:57:22.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7cd2866640 1 Processor -- start 2023-12-19T17:57:22.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7cd2866640 1 -- start start 2023-12-19T17:57:22.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7cd2866640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7ccc15a9c0 0x7f7ccc0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:22.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7cd2866640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ccc06ac80 0x7f7ccc0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:22.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7cd2866640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ccc070d90 con 0x7f7ccc15ada0 2023-12-19T17:57:22.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7cd2866640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc0cc270 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7cd2866640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc071070 con 0x7f7ccc06ac80 2023-12-19T17:57:22.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7ccbfff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7ccc15a9c0 0x7f7ccc0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:22.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7ccbfff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7ccc15a9c0 0x7f7ccc0706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:35402/0 (socket says 172.21.15.26:35402) 2023-12-19T17:57:22.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7ccbfff640 1 -- 172.21.15.26:0/1760119292 learned_addr learned my addr 172.21.15.26:0/1760119292 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:22.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7cd0e03640 1 --2- 172.21.15.26:0/1760119292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ccc06ac80 0x7f7ccc0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7cd0e03640 1 -- 172.21.15.26:0/1760119292 >> v2:172.21.15.160:3300/0 conn(0x7f7ccc15a9c0 msgr2=0x7f7ccc0706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7cd0e03640 1 --2- 172.21.15.26:0/1760119292 >> v2:172.21.15.160:3300/0 conn(0x7f7ccc15a9c0 0x7f7ccc0706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7cd0e03640 1 -- 172.21.15.26:0/1760119292 >> v1:172.21.15.160:6789/0 conn(0x7f7ccc15ada0 legacy=0x7f7ccc0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7cd0e03640 1 -- 172.21.15.26:0/1760119292 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc071350 con 0x7f7ccc06ac80 2023-12-19T17:57:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.635+0000 7f7ccbfff640 1 --2- 172.21.15.26:0/1760119292 >> v2:172.21.15.160:3300/0 conn(0x7f7ccc15a9c0 0x7f7ccc0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:22.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.639+0000 7f7cd0e03640 1 --2- 172.21.15.26:0/1760119292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ccc06ac80 0x7f7ccc0b01b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009d90 tx=0x7f7cc401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f2a03b0c20a6ed9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:22.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.639+0000 7f7ccaffd640 1 -- 172.21.15.26:0/1760119292 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40a5860 con 0x7f7ccc06ac80 2023-12-19T17:57:22.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.639+0000 7f7ccaffd640 1 -- 172.21.15.26:0/1760119292 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc40a59c0 con 0x7f7ccc06ac80 2023-12-19T17:57:22.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.639+0000 7f7ccaffd640 1 -- 172.21.15.26:0/1760119292 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40a5d50 con 0x7f7ccc06ac80 2023-12-19T17:57:22.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.639+0000 7f7cd2866640 1 -- 172.21.15.26:0/1760119292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ccc06ac80 msgr2=0x7f7ccc0b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:22.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.639+0000 7f7cd2866640 1 --2- 172.21.15.26:0/1760119292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ccc06ac80 0x7f7ccc0b01b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4009d90 tx=0x7f7cc401c890 comp rx=0 tx=0).stop 2023-12-19T17:57:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.639+0000 7f7cd2866640 1 -- 172.21.15.26:0/1760119292 shutdown_connections 2023-12-19T17:57:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.639+0000 7f7cd2866640 1 --2- 172.21.15.26:0/1760119292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ccc06ac80 0x7f7ccc0b01b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.639+0000 7f7cd2866640 1 --2- 172.21.15.26:0/1760119292 >> v2:172.21.15.160:3300/0 conn(0x7f7ccc15a9c0 0x7f7ccc0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.639+0000 7f7cd2866640 1 -- 172.21.15.26:0/1760119292 >> 172.21.15.26:0/1760119292 conn(0x7f7ccc0ce730 msgr2=0x7f7ccc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.639+0000 7f7cd2866640 1 -- 172.21.15.26:0/1760119292 shutdown_connections 2023-12-19T17:57:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.639+0000 7f7cd2866640 1 -- 172.21.15.26:0/1760119292 wait complete. 2023-12-19T17:57:22.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cd2866640 1 Processor -- start 2023-12-19T17:57:22.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cd2866640 1 -- start start 2023-12-19T17:57:22.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cd2866640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ccc06ac80 0x7f7ccc150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cd2866640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ccc15a9c0 0x7f7ccc15d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cd2866640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ccc15ada0 0x7f7ccc0c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cd2866640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc151240 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cd2866640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc15ccf0 con 0x7f7ccc15ada0 2023-12-19T17:57:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cd2866640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc15d6c0 con 0x7f7ccc06ac80 2023-12-19T17:57:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7ccb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ccc15a9c0 0x7f7ccc15d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7ccb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ccc15a9c0 0x7f7ccc15d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42580/0 (socket says 172.21.15.26:42580) 2023-12-19T17:57:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7ccb7fe640 1 -- 172.21.15.26:0/332600681 learned_addr learned my addr 172.21.15.26:0/332600681 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cd0e03640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ccc15ada0 0x7f7ccc0c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:22.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7ccbfff640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ccc06ac80 0x7f7ccc150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7ccb7fe640 1 -- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ccc06ac80 msgr2=0x7f7ccc150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7ccb7fe640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ccc06ac80 0x7f7ccc150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7ccb7fe640 1 -- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ccc15ada0 msgr2=0x7f7ccc0c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7ccb7fe640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ccc15ada0 0x7f7ccc0c69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7ccb7fe640 1 -- 172.21.15.26:0/332600681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc0094d0 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cd0e03640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ccc15ada0 0x7f7ccc0c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7ccb7fe640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ccc15a9c0 0x7f7ccc15d180 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000c500 tx=0x7f7cc000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cc8ff9640 1 -- 172.21.15.26:0/332600681 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0005220 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cc8ff9640 1 -- 172.21.15.26:0/332600681 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc0005380 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cc8ff9640 1 -- 172.21.15.26:0/332600681 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc0007270 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cd2866640 1 -- 172.21.15.26:0/332600681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc4009a70 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.643+0000 7f7cd2866640 1 -- 172.21.15.26:0/332600681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc15c190 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.647+0000 7f7cd2866640 1 -- 172.21.15.26:0/332600681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc15ce80 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.647+0000 7f7cc8ff9640 1 -- 172.21.15.26:0/332600681 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 76) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f7cc00073d0 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.647+0000 7f7cc8ff9640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cb8065b80 0x7f7cb8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:22.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.647+0000 7f7cc8ff9640 1 -- 172.21.15.26:0/332600681 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f7cc00830a0 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.647+0000 7f7ccbfff640 1 -- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cb8065b80 msgr2=0x7f7cb8068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:22.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.647+0000 7f7ccbfff640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cb8065b80 0x7f7cb8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:22.652 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:22.651+0000 7f25cb48d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:57:22.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.651+0000 7f7cc8ff9640 1 -- 172.21.15.26:0/332600681 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7cc00504a0 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.671 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:22.666+0000 7f3ba6bda180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:57:22.713 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:22.711+0000 7f25cb48d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:57:22.726 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:22.722+0000 7f3ba6bda180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:57:22.802 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:22.799+0000 7f25cb48d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:57:22.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cd2866640 1 -- 172.21.15.26:0/332600681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7ccc074270 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cc8ff9640 1 -- 172.21.15.26:0/332600681 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v147) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7cc0053ac0 con 0x7f7ccc15a9c0 2023-12-19T17:57:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T17:57:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cd2866640 1 -- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cb8065b80 msgr2=0x7f7cb8068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cd2866640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cb8065b80 0x7f7cb8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cd2866640 1 -- 172.21.15.26:0/332600681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ccc15a9c0 msgr2=0x7f7ccc15d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cd2866640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ccc15a9c0 0x7f7ccc15d180 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f7cc000c500 tx=0x7f7cc000c9c0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cd2866640 1 -- 172.21.15.26:0/332600681 shutdown_connections 2023-12-19T17:57:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cd2866640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ccc15ada0 0x7f7ccc0c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cd2866640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ccc15a9c0 0x7f7ccc15d180 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cd2866640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7cb8065b80 0x7f7cb8068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cd2866640 1 --2- 172.21.15.26:0/332600681 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ccc06ac80 0x7f7ccc150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cd2866640 1 -- 172.21.15.26:0/332600681 >> 172.21.15.26:0/332600681 conn(0x7f7ccc0ce730 msgr2=0x7f7ccc0c9650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cd2866640 1 -- 172.21.15.26:0/332600681 shutdown_connections 2023-12-19T17:57:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:22.811+0000 7f7cd2866640 1 -- 172.21.15.26:0/332600681 wait complete. 2023-12-19T17:57:22.836 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T17:57:22.863 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:22.859+0000 7f25cb48d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:57:22.868 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:22.866+0000 7f3ba6bda180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:57:22.926 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:22.922+0000 7f3ba6bda180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:57:23.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38f4854640 1 Processor -- start 2023-12-19T17:57:23.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38f4854640 1 -- start start 2023-12-19T17:57:23.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38f4854640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38f015cca0 0x7f38f00b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38f4854640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f00b6fb0 con 0x7f38f00b6b20 2023-12-19T17:57:23.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38f4854640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f00b7290 con 0x7f38f00b8760 2023-12-19T17:57:23.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38f4854640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f00b7c30 con 0x7f38f015cca0 2023-12-19T17:57:23.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38ef7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38f015cca0 0x7f38f00b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:23.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38ef7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38f015cca0 0x7f38f00b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42592/0 (socket says 172.21.15.26:42592) 2023-12-19T17:57:23.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38ef7fe640 1 -- 172.21.15.26:0/1021047877 learned_addr learned my addr 172.21.15.26:0/1021047877 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:23.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38effff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f38f00b6b20 0x7f38f00d09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33244/0 (socket says 172.21.15.26:33244) 2023-12-19T17:57:23.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38ef7fe640 1 -- 172.21.15.26:0/1021047877 >> v1:172.21.15.160:6789/0 conn(0x7f38f00b8760 legacy=0x7f38f00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38ef7fe640 1 -- 172.21.15.26:0/1021047877 >> v1:172.21.15.26:6789/0 conn(0x7f38f00b6b20 legacy=0x7f38f00d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38ef7fe640 1 -- 172.21.15.26:0/1021047877 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f00b7570 con 0x7f38f015cca0 2023-12-19T17:57:23.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.007+0000 7f38ee7fc640 1 -- 172.21.15.26:0/1021047877 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750950948 0 0) 0x7f38f00b6fb0 con 0x7f38f00b6b20 2023-12-19T17:57:23.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.011+0000 7f38ef7fe640 1 --2- 172.21.15.26:0/1021047877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f015cca0 0x7f38f00b65e0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009870 tx=0x7f38e001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886eeb95329faf8f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.011+0000 7f38ee7fc640 1 -- 172.21.15.26:0/1021047877 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e00a95f0 con 0x7f38f015cca0 2023-12-19T17:57:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.011+0000 7f38ee7fc640 1 -- 172.21.15.26:0/1021047877 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e00a9750 con 0x7f38f015cca0 2023-12-19T17:57:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.011+0000 7f38ee7fc640 1 -- 172.21.15.26:0/1021047877 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e00a9ae0 con 0x7f38f015cca0 2023-12-19T17:57:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.011+0000 7f38f4854640 1 -- 172.21.15.26:0/1021047877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f015cca0 msgr2=0x7f38f00b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.011+0000 7f38f4854640 1 --2- 172.21.15.26:0/1021047877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f015cca0 0x7f38f00b65e0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f38e0009870 tx=0x7f38e001c510 comp rx=0 tx=0).stop 2023-12-19T17:57:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.011+0000 7f38f4854640 1 -- 172.21.15.26:0/1021047877 shutdown_connections 2023-12-19T17:57:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.011+0000 7f38f4854640 1 --2- 172.21.15.26:0/1021047877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f015cca0 0x7f38f00b65e0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.011+0000 7f38f4854640 1 -- 172.21.15.26:0/1021047877 >> 172.21.15.26:0/1021047877 conn(0x7f38f006a120 msgr2=0x7f38f006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.011+0000 7f38f4854640 1 -- 172.21.15.26:0/1021047877 shutdown_connections 2023-12-19T17:57:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.011+0000 7f38f4854640 1 -- 172.21.15.26:0/1021047877 wait complete. 2023-12-19T17:57:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38f4854640 1 Processor -- start 2023-12-19T17:57:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38f4854640 1 -- start start 2023-12-19T17:57:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38f4854640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38f00b8760 0x7f38f014e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38f4854640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f015cca0 0x7f38f015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38f4854640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38f0151340 0x7f38f0152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38f4854640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f0150050 con 0x7f38f015cca0 2023-12-19T17:57:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38f4854640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f01502a0 con 0x7f38f0151340 2023-12-19T17:57:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38f4854640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f0150580 con 0x7f38f00b8760 2023-12-19T17:57:23.021 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f25cb48d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:57:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38eeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f015cca0 0x7f38f015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38effff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38f0151340 0x7f38f0152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38eeffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f015cca0 0x7f38f015b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42596/0 (socket says 172.21.15.26:42596) 2023-12-19T17:57:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38ef7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38f00b8760 0x7f38f014e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38eeffd640 1 -- 172.21.15.26:0/2636708646 learned_addr learned my addr 172.21.15.26:0/2636708646 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38effff640 1 -- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38f00b8760 msgr2=0x7f38f014e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38effff640 1 --2- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38f00b8760 0x7f38f014e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38effff640 1 -- 172.21.15.26:0/2636708646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f015cca0 msgr2=0x7f38f015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38effff640 1 --2- 172.21.15.26:0/2636708646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f015cca0 0x7f38f015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38effff640 1 -- 172.21.15.26:0/2636708646 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e00094d0 con 0x7f38f0151340 2023-12-19T17:57:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38eeffd640 1 --2- 172.21.15.26:0/2636708646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f015cca0 0x7f38f015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38ef7fe640 1 --2- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38f00b8760 0x7f38f014e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38effff640 1 --2- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38f0151340 0x7f38f0152f30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f38e8002240 tx=0x7f38e800cbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.015+0000 7f38d7fff640 1 -- 172.21.15.26:0/2636708646 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e800ce30 con 0x7f38f0151340 2023-12-19T17:57:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.019+0000 7f38d7fff640 1 -- 172.21.15.26:0/2636708646 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e800a640 con 0x7f38f0151340 2023-12-19T17:57:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.019+0000 7f38f4854640 1 -- 172.21.15.26:0/2636708646 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f0150860 con 0x7f38f0151340 2023-12-19T17:57:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.019+0000 7f38d7fff640 1 -- 172.21.15.26:0/2636708646 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e800a9f0 con 0x7f38f0151340 2023-12-19T17:57:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.019+0000 7f38f4854640 1 -- 172.21.15.26:0/2636708646 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f0150e20 con 0x7f38f0151340 2023-12-19T17:57:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.019+0000 7f38f4854640 1 -- 172.21.15.26:0/2636708646 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f0151010 con 0x7f38f0151340 2023-12-19T17:57:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.019+0000 7f38d7fff640 1 -- 172.21.15.26:0/2636708646 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 76) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f38e800ab50 con 0x7f38f0151340 2023-12-19T17:57:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.019+0000 7f38d7fff640 1 --2- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f38dc065bb0 0x7f38dc068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.019+0000 7f38d7fff640 1 -- 172.21.15.26:0/2636708646 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f38e8082e70 con 0x7f38f0151340 2023-12-19T17:57:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.019+0000 7f38ef7fe640 1 -- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f38dc065bb0 msgr2=0x7f38dc068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.019+0000 7f38ef7fe640 1 --2- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f38dc065bb0 0x7f38dc068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.023+0000 7f38d7fff640 1 -- 172.21.15.26:0/2636708646 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f38e80502f0 con 0x7f38f0151340 2023-12-19T17:57:23.081 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:23.079+0000 7f25cb48d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:57:23.170 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:57:23.170 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:57:23.170 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:57:23.178 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:23.174+0000 7f3ba6bda180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:57:23.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38f4854640 1 -- 172.21.15.26:0/2636708646 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f38f0067400 con 0x7f38f0151340 2023-12-19T17:57:23.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38d7fff640 1 -- 172.21.15.26:0/2636708646 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v147) v1 ==== 80+0+465 (secure 0 0 0) 0x7f38e8018020 con 0x7f38f0151340 2023-12-19T17:57:23.201 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6833/38098 2023-12-20T17:01:32.971594+0000 2023-12-19T17:57:23.201 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6832/38098 2023-12-20T17:01:32.971594+0000 2023-12-19T17:57:23.201 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1067813784 2023-12-20T17:01:32.971594+0000 2023-12-19T17:57:23.201 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1947760249 2023-12-20T17:01:32.971594+0000 2023-12-19T17:57:23.201 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/464310353 2023-12-20T17:01:32.971594+0000 2023-12-19T17:57:23.201 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2140731666 2023-12-20T17:01:32.971594+0000 2023-12-19T17:57:23.201 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/870873437 2023-12-20T17:01:32.971594+0000 2023-12-19T17:57:23.201 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/950669571 2023-12-20T17:01:32.971594+0000 2023-12-19T17:57:23.202 INFO:teuthology.orchestra.run.smithi026.stderr:listed 8 entries 2023-12-19T17:57:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38f4854640 1 -- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f38dc065bb0 msgr2=0x7f38dc068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38f4854640 1 --2- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f38dc065bb0 0x7f38dc068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38f4854640 1 -- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38f0151340 msgr2=0x7f38f0152f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38f4854640 1 --2- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38f0151340 0x7f38f0152f30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f38e8002240 tx=0x7f38e800cbe0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38f4854640 1 -- 172.21.15.26:0/2636708646 shutdown_connections 2023-12-19T17:57:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38f4854640 1 --2- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38f0151340 0x7f38f0152f30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38f4854640 1 --2- 172.21.15.26:0/2636708646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f015cca0 0x7f38f015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38f4854640 1 --2- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f38dc065bb0 0x7f38dc068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38f4854640 1 --2- 172.21.15.26:0/2636708646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38f00b8760 0x7f38f014e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38f4854640 1 -- 172.21.15.26:0/2636708646 >> 172.21.15.26:0/2636708646 conn(0x7f38f006a120 msgr2=0x7f38f00c7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38f4854640 1 -- 172.21.15.26:0/2636708646 shutdown_connections 2023-12-19T17:57:23.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.199+0000 7f38f4854640 1 -- 172.21.15.26:0/2636708646 wait complete. 2023-12-19T17:57:23.224 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T17:57:23.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:23.224 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-19T17:57:23.286 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T17:57:23.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:23.287 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T17:57:23.303 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T17:57:23.303 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:23.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T17:57:23.314 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:57:23.314 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:23.314 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:23.310+0000 7f3ba6bda180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:57:23.333 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:57:23.333 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:57:23.333 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:57:23.343 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T17:57:23.343 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T17:57:23.343 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:23.344 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:23.339+0000 7f25cb48d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:57:23.364 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T17:57:23.365 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:57:23.438 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:23.434+0000 7f3ba6bda180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:57:23.504 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:23.503+0000 7f25cb48d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:57:23.509 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:23.506+0000 7f3ba6bda180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:57:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01cd695640 1 Processor -- start 2023-12-19T17:57:23.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01cd695640 1 -- start start 2023-12-19T17:57:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01cd695640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01c815a7c0 0x7f01c815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01cd695640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c815e2f0 con 0x7f01c80cf700 2023-12-19T17:57:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01cd695640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c80c5b40 con 0x7f01c806f9d0 2023-12-19T17:57:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01cd695640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c80c5e20 con 0x7f01c815a7c0 2023-12-19T17:57:23.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01c77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f01c80cf700 0x7f01c80cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33246/0 (socket says 172.21.15.26:33246) 2023-12-19T17:57:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01c77fe640 1 -- 172.21.15.26:0/838027263 learned_addr learned my addr 172.21.15.26:0/838027263 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01c6ffd640 1 --2- 172.21.15.26:0/838027263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01c815a7c0 0x7f01c815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:23.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01c5ffb640 1 -- 172.21.15.26:0/838027263 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316317135 0 0) 0x7f01c815e2f0 con 0x7f01c80cf700 2023-12-19T17:57:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01c5ffb640 1 -- 172.21.15.26:0/838027263 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01b0003530 con 0x7f01c80cf700 2023-12-19T17:57:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01c5ffb640 1 -- 172.21.15.26:0/838027263 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3158855827 0 0) 0x7f01c80c5b40 con 0x7f01c806f9d0 2023-12-19T17:57:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01c5ffb640 1 -- 172.21.15.26:0/838027263 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01c815e2f0 con 0x7f01c806f9d0 2023-12-19T17:57:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01c6ffd640 1 -- 172.21.15.26:0/838027263 >> v1:172.21.15.160:6789/0 conn(0x7f01c806f9d0 legacy=0x7f01c80ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01c6ffd640 1 -- 172.21.15.26:0/838027263 >> v1:172.21.15.26:6789/0 conn(0x7f01c80cf700 legacy=0x7f01c80cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.531+0000 7f01c6ffd640 1 -- 172.21.15.26:0/838027263 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c80ce770 con 0x7f01c815a7c0 2023-12-19T17:57:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.535+0000 7f01c6ffd640 1 --2- 172.21.15.26:0/838027263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01c815a7c0 0x7f01c815ddb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01b80097d0 tx=0x7f01b801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=564b383d6f871964 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.535+0000 7f01c5ffb640 1 -- 172.21.15.26:0/838027263 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b80ab670 con 0x7f01c815a7c0 2023-12-19T17:57:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.535+0000 7f01c5ffb640 1 -- 172.21.15.26:0/838027263 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b80ab7d0 con 0x7f01c815a7c0 2023-12-19T17:57:23.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.535+0000 7f01c5ffb640 1 -- 172.21.15.26:0/838027263 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b80abb40 con 0x7f01c815a7c0 2023-12-19T17:57:23.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.535+0000 7f01cd695640 1 -- 172.21.15.26:0/838027263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01c815a7c0 msgr2=0x7f01c815ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.535+0000 7f01cd695640 1 --2- 172.21.15.26:0/838027263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01c815a7c0 0x7f01c815ddb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f01b80097d0 tx=0x7f01b801c470 comp rx=0 tx=0).stop 2023-12-19T17:57:23.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.535+0000 7f01cd695640 1 -- 172.21.15.26:0/838027263 shutdown_connections 2023-12-19T17:57:23.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.535+0000 7f01cd695640 1 --2- 172.21.15.26:0/838027263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01c815a7c0 0x7f01c815ddb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.535+0000 7f01cd695640 1 -- 172.21.15.26:0/838027263 >> 172.21.15.26:0/838027263 conn(0x7f01c806d5d0 msgr2=0x7f01c8069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:23.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.535+0000 7f01cd695640 1 -- 172.21.15.26:0/838027263 shutdown_connections 2023-12-19T17:57:23.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.535+0000 7f01cd695640 1 -- 172.21.15.26:0/838027263 wait complete. 2023-12-19T17:57:23.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01cd695640 1 Processor -- start 2023-12-19T17:57:23.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01cd695640 1 -- start start 2023-12-19T17:57:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01cd695640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c806f9d0 0x7f01c8148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01cd695640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01c80cf700 0x7f01c80b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01cd695640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01c815a7c0 0x7f01c81513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01cd695640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c80b4470 con 0x7f01c806f9d0 2023-12-19T17:57:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01cd695640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c80b3260 con 0x7f01c815a7c0 2023-12-19T17:57:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01cd695640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c8151920 con 0x7f01c80cf700 2023-12-19T17:57:23.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c806f9d0 0x7f01c8148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01c6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c806f9d0 0x7f01c8148b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42606/0 (socket says 172.21.15.26:42606) 2023-12-19T17:57:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01c6ffd640 1 -- 172.21.15.26:0/2672473957 learned_addr learned my addr 172.21.15.26:0/2672473957 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01c67fc640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01c80cf700 0x7f01c80b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:23.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01c77fe640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01c815a7c0 0x7f01c81513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:23.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01c6ffd640 1 -- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01c80cf700 msgr2=0x7f01c80b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01c6ffd640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01c80cf700 0x7f01c80b2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01c6ffd640 1 -- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01c815a7c0 msgr2=0x7f01c81513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01c6ffd640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01c815a7c0 0x7f01c81513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01c6ffd640 1 -- 172.21.15.26:0/2672473957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b80093b0 con 0x7f01c806f9d0 2023-12-19T17:57:23.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01c6ffd640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c806f9d0 0x7f01c8148b60 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f01b801ab20 tx=0x7f01b8019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:23.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01af7fe640 1 -- 172.21.15.26:0/2672473957 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b80abb40 con 0x7f01c806f9d0 2023-12-19T17:57:23.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01cd695640 1 -- 172.21.15.26:0/2672473957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c80b1c80 con 0x7f01c806f9d0 2023-12-19T17:57:23.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01af7fe640 1 -- 172.21.15.26:0/2672473957 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b80b9070 con 0x7f01c806f9d0 2023-12-19T17:57:23.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01af7fe640 1 -- 172.21.15.26:0/2672473957 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b80abea0 con 0x7f01c806f9d0 2023-12-19T17:57:23.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.539+0000 7f01cd695640 1 -- 172.21.15.26:0/2672473957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c80b2240 con 0x7f01c806f9d0 2023-12-19T17:57:23.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.543+0000 7f01cd695640 1 -- 172.21.15.26:0/2672473957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c8074270 con 0x7f01c806f9d0 2023-12-19T17:57:23.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.543+0000 7f01af7fe640 1 -- 172.21.15.26:0/2672473957 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 76) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f01b8003620 con 0x7f01c806f9d0 2023-12-19T17:57:23.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.543+0000 7f01af7fe640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b4065b80 0x7f01b4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.543+0000 7f01af7fe640 1 -- 172.21.15.26:0/2672473957 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f01b80bb070 con 0x7f01c806f9d0 2023-12-19T17:57:23.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.543+0000 7f01c67fc640 1 -- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b4065b80 msgr2=0x7f01b4068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:23.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.543+0000 7f01c67fc640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b4065b80 0x7f01b4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:23.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.551+0000 7f01af7fe640 1 -- 172.21.15.26:0/2672473957 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f01b80eb7c0 con 0x7f01c806f9d0 2023-12-19T17:57:23.644 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:23.642+0000 7f3ba6bda180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:57:23.650 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:23.647+0000 7f25cb48d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:57:23.727 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:23.727+0000 7f25cb48d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:57:23.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.747+0000 7f01c67fc640 1 -- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b4065b80 msgr2=0x7f01b4068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:23.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.747+0000 7f01c67fc640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b4065b80 0x7f01b4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T17:57:23.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.751+0000 7f01cd695640 1 -- 172.21.15.26:0/2672473957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01c80b0df0 con 0x7f01c806f9d0 2023-12-19T17:57:23.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.755+0000 7f01af7fe640 1 -- 172.21.15.26:0/2672473957 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f01b80eede0 con 0x7f01c806f9d0 2023-12-19T17:57:23.756 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:23.756 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8092,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3217828830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3217828830},{"type":"v1","addr":"172.21.15.160:6835","nonce":3217828830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2758554130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2758554130},{"type":"v1","addr":"172.21.15.160:6837","nonce":2758554130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3510623843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3510623843},{"type":"v1","addr":"172.21.15.26:6833","nonce":3510623843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:01:21.142075+0000","modified":"2023-12-19T17:01:24.167832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8487":{"gid":8487,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3823167404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3823167404},{"type":"v1","addr":"172.21.15.26:6835","nonce":3823167404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T17:57:23.756 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 65 2023-12-19T17:57:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.755+0000 7f01cd695640 1 -- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b4065b80 msgr2=0x7f01b4068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.755+0000 7f01cd695640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b4065b80 0x7f01b4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.755+0000 7f01cd695640 1 -- 172.21.15.26:0/2672473957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c806f9d0 msgr2=0x7f01c8148b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.755+0000 7f01cd695640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c806f9d0 0x7f01c8148b60 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f01b801ab20 tx=0x7f01b8019210 comp rx=0 tx=0).stop 2023-12-19T17:57:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.755+0000 7f01cd695640 1 -- 172.21.15.26:0/2672473957 shutdown_connections 2023-12-19T17:57:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.755+0000 7f01cd695640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01c815a7c0 0x7f01c81513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.755+0000 7f01cd695640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01b4065b80 0x7f01b4068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.755+0000 7f01cd695640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01c80cf700 0x7f01c80b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.755+0000 7f01cd695640 1 --2- 172.21.15.26:0/2672473957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01c806f9d0 0x7f01c8148b60 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.755+0000 7f01cd695640 1 -- 172.21.15.26:0/2672473957 >> 172.21.15.26:0/2672473957 conn(0x7f01c806d5d0 msgr2=0x7f01c815fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.755+0000 7f01cd695640 1 -- 172.21.15.26:0/2672473957 shutdown_connections 2023-12-19T17:57:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.755+0000 7f01cd695640 1 -- 172.21.15.26:0/2672473957 wait complete. 2023-12-19T17:57:23.779 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:57:23.881 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:23.879+0000 7f25cb48d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:57:23.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b2788c640 1 Processor -- start 2023-12-19T17:57:23.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b2788c640 1 -- start start 2023-12-19T17:57:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b2788c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3b2015f370 0x7f3b2015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b2788c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3b200ce740 0x7f3b200ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b2788c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b200cf0b0 con 0x7f3b20155030 2023-12-19T17:57:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b2788c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b2006c790 con 0x7f3b200ce740 2023-12-19T17:57:23.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b2788c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b2006c970 con 0x7f3b2015f370 2023-12-19T17:57:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b25e29640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3b200ce740 0x7f3b200ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b24e27640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3b2015f370 0x7f3b2015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b24e27640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3b2015f370 0x7f3b2015bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:35432/0 (socket says 172.21.15.26:35432) 2023-12-19T17:57:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b24e27640 1 -- 172.21.15.26:0/1627620846 learned_addr learned my addr 172.21.15.26:0/1627620846 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b24e27640 1 -- 172.21.15.26:0/1627620846 >> v2:172.21.15.26:3300/0 conn(0x7f3b200ce740 msgr2=0x7f3b200ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b24e27640 1 --2- 172.21.15.26:0/1627620846 >> v2:172.21.15.26:3300/0 conn(0x7f3b200ce740 0x7f3b200ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b24e27640 1 -- 172.21.15.26:0/1627620846 >> v1:172.21.15.160:6789/0 conn(0x7f3b20155030 legacy=0x7f3b2006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b24e27640 1 -- 172.21.15.26:0/1627620846 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2015b1f0 con 0x7f3b2015f370 2023-12-19T17:57:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b17fff640 1 -- 172.21.15.26:0/1627620846 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2459463912 0 0) 0x7f3b200cf0b0 con 0x7f3b20155030 2023-12-19T17:57:23.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.927+0000 7f3b25e29640 1 --2- 172.21.15.26:0/1627620846 >> v2:172.21.15.26:3300/0 conn(0x7f3b200ce740 0x7f3b200ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:23.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.931+0000 7f3b24e27640 1 --2- 172.21.15.26:0/1627620846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b2015f370 0x7f3b2015bcd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3b200b1780 tx=0x7f3b1001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d55d8efc8d5a03f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.931+0000 7f3b17fff640 1 -- 172.21.15.26:0/1627620846 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b100aa430 con 0x7f3b2015f370 2023-12-19T17:57:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.931+0000 7f3b17fff640 1 -- 172.21.15.26:0/1627620846 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b100aa590 con 0x7f3b2015f370 2023-12-19T17:57:23.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.931+0000 7f3b17fff640 1 -- 172.21.15.26:0/1627620846 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b100aa920 con 0x7f3b2015f370 2023-12-19T17:57:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.931+0000 7f3b2788c640 1 -- 172.21.15.26:0/1627620846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b2015f370 msgr2=0x7f3b2015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.931+0000 7f3b2788c640 1 --2- 172.21.15.26:0/1627620846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b2015f370 0x7f3b2015bcd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3b200b1780 tx=0x7f3b1001c4d0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.931+0000 7f3b2788c640 1 -- 172.21.15.26:0/1627620846 shutdown_connections 2023-12-19T17:57:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.931+0000 7f3b2788c640 1 --2- 172.21.15.26:0/1627620846 >> v2:172.21.15.26:3300/0 conn(0x7f3b200ce740 0x7f3b200ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.931+0000 7f3b2788c640 1 --2- 172.21.15.26:0/1627620846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b2015f370 0x7f3b2015bcd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.931+0000 7f3b2788c640 1 -- 172.21.15.26:0/1627620846 >> 172.21.15.26:0/1627620846 conn(0x7f3b200658e0 msgr2=0x7f3b20065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.931+0000 7f3b2788c640 1 -- 172.21.15.26:0/1627620846 shutdown_connections 2023-12-19T17:57:23.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.931+0000 7f3b2788c640 1 -- 172.21.15.26:0/1627620846 wait complete. 2023-12-19T17:57:23.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b2788c640 1 Processor -- start 2023-12-19T17:57:23.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b2788c640 1 -- start start 2023-12-19T17:57:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b2788c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b200ce740 0x7f3b2014b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b2788c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b20155030 0x7f3b2006b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b2788c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b2015f370 0x7f3b2006be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b2788c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2014f6e0 con 0x7f3b200ce740 2023-12-19T17:57:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b2788c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200ae880 con 0x7f3b20155030 2023-12-19T17:57:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b2788c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b200aeb60 con 0x7f3b2015f370 2023-12-19T17:57:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b25628640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b200ce740 0x7f3b2014b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b25628640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b200ce740 0x7f3b2014b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42622/0 (socket says 172.21.15.26:42622) 2023-12-19T17:57:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b25628640 1 -- 172.21.15.26:0/2522666337 learned_addr learned my addr 172.21.15.26:0/2522666337 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b25e29640 1 --2- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b2015f370 0x7f3b2006be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:23.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b24e27640 1 --2- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b20155030 0x7f3b2006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b25628640 1 -- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b2015f370 msgr2=0x7f3b2006be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b25628640 1 --2- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b2015f370 0x7f3b2006be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b25628640 1 -- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b20155030 msgr2=0x7f3b2006b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b25628640 1 --2- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b20155030 0x7f3b2006b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b25628640 1 -- 172.21.15.26:0/2522666337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b18009a70 con 0x7f3b200ce740 2023-12-19T17:57:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b25628640 1 --2- 172.21.15.26:0/2522666337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b200ce740 0x7f3b2014b210 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c480 tx=0x7f3b1c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.935+0000 7f3b15ffb640 1 -- 172.21.15.26:0/2522666337 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c0054e0 con 0x7f3b200ce740 2023-12-19T17:57:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.939+0000 7f3b15ffb640 1 -- 172.21.15.26:0/2522666337 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c003600 con 0x7f3b200ce740 2023-12-19T17:57:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.939+0000 7f3b15ffb640 1 -- 172.21.15.26:0/2522666337 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c0039d0 con 0x7f3b200ce740 2023-12-19T17:57:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.939+0000 7f3b2788c640 1 -- 172.21.15.26:0/2522666337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b100094d0 con 0x7f3b200ce740 2023-12-19T17:57:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.939+0000 7f3b2788c640 1 -- 172.21.15.26:0/2522666337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b20151dd0 con 0x7f3b200ce740 2023-12-19T17:57:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.939+0000 7f3b2788c640 1 -- 172.21.15.26:0/2522666337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b20074270 con 0x7f3b200ce740 2023-12-19T17:57:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.939+0000 7f3b15ffb640 1 -- 172.21.15.26:0/2522666337 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 76) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f3b1c003b30 con 0x7f3b200ce740 2023-12-19T17:57:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.943+0000 7f3b15ffb640 1 --2- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3b04065b80 0x7f3b04068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.943+0000 7f3b15ffb640 1 -- 172.21.15.26:0/2522666337 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f3b1c082d40 con 0x7f3b200ce740 2023-12-19T17:57:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.943+0000 7f3b24e27640 1 -- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3b04065b80 msgr2=0x7f3b04068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:23.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.943+0000 7f3b24e27640 1 --2- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3b04065b80 0x7f3b04068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:23.943+0000 7f3b15ffb640 1 -- 172.21.15.26:0/2522666337 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3b1c050130 con 0x7f3b200ce740 2023-12-19T17:57:24.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.107+0000 7f3b2788c640 1 -- 172.21.15.26:0/2522666337 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b20151fc0 con 0x7f3b200ce740 2023-12-19T17:57:24.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.107+0000 7f3b15ffb640 1 -- 172.21.15.26:0/2522666337 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+17977 (secure 0 0 0) 0x7f3b1c017020 con 0x7f3b200ce740 2023-12-19T17:57:24.111 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:24.111 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":148,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:57:23.679458+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:01:19.682876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T17:01:20.140612+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":147,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.f","mappings":[{"from":0,"to":2}]},{"pgid":"17.3f","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:6833/38098":"2023-12-20T17:01:32.971594+0000","172.21.15.160:6832/38098":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/1067813784":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/1947760249":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/464310353":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/2140731666":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/870873437":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/950669571":"2023-12-20T17:01:32.971594+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":17,"snaps":[{"begin":5,"length":52},{"begin":61,"length":52}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":3,"length":2},{"begin":59,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:57:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.111+0000 7f3b2788c640 1 -- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3b04065b80 msgr2=0x7f3b04068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.111+0000 7f3b2788c640 1 --2- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3b04065b80 0x7f3b04068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.111+0000 7f3b2788c640 1 -- 172.21.15.26:0/2522666337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b200ce740 msgr2=0x7f3b2014b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.111+0000 7f3b2788c640 1 --2- 172.21.15.26:0/2522666337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b200ce740 0x7f3b2014b210 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00c480 tx=0x7f3b1c00c880 comp rx=0 tx=0).stop 2023-12-19T17:57:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.111+0000 7f3b2788c640 1 -- 172.21.15.26:0/2522666337 shutdown_connections 2023-12-19T17:57:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.111+0000 7f3b2788c640 1 --2- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3b04065b80 0x7f3b04068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.111+0000 7f3b2788c640 1 --2- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3b2015f370 0x7f3b2006be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.111+0000 7f3b2788c640 1 --2- 172.21.15.26:0/2522666337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b20155030 0x7f3b2006b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.111+0000 7f3b2788c640 1 --2- 172.21.15.26:0/2522666337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b200ce740 0x7f3b2014b210 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.111+0000 7f3b2788c640 1 -- 172.21.15.26:0/2522666337 >> 172.21.15.26:0/2522666337 conn(0x7f3b200658e0 msgr2=0x7f3b2014bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.111+0000 7f3b2788c640 1 -- 172.21.15.26:0/2522666337 shutdown_connections 2023-12-19T17:57:24.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.111+0000 7f3b2788c640 1 -- 172.21.15.26:0/2522666337 wait complete. 2023-12-19T17:57:24.141 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T17:01:19.682876+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T17:01:20.140612+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 147, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T17:57:24.142 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T17:57:24.192 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.186+0000 7f3ba6bda180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:57:24.275 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.270+0000 7f3ba6bda180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:57:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd7b60640 1 Processor -- start 2023-12-19T17:57:24.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd7b60640 1 -- start start 2023-12-19T17:57:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd7b60640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd006b140 0x7fdfd0161520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd7b60640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdfd00cd610 0x7fdfd00cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:24.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd7b60640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd0161a60 con 0x7fdfd006ad60 2023-12-19T17:57:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd7b60640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd00ccbc0 con 0x7fdfd00cd610 2023-12-19T17:57:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd7b60640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd00cd180 con 0x7fdfd006b140 2023-12-19T17:57:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd60fd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdfd00cd610 0x7fdfd00cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd60fd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdfd00cd610 0x7fdfd00cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42626/0 (socket says 172.21.15.26:42626) 2023-12-19T17:57:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd60fd640 1 -- 172.21.15.26:0/1644236428 learned_addr learned my addr 172.21.15.26:0/1644236428 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd50fb640 1 --2- 172.21.15.26:0/1644236428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd006b140 0x7fdfd0161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd60fd640 1 -- 172.21.15.26:0/1644236428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd006b140 msgr2=0x7fdfd0161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd60fd640 1 --2- 172.21.15.26:0/1644236428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd006b140 0x7fdfd0161520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd60fd640 1 -- 172.21.15.26:0/1644236428 >> v1:172.21.15.26:6789/0 conn(0x7fdfd006ad60 legacy=0x7fdfd0069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd60fd640 1 -- 172.21.15.26:0/1644236428 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd006b8b0 con 0x7fdfd00cd610 2023-12-19T17:57:24.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.287+0000 7fdfd50fb640 1 --2- 172.21.15.26:0/1644236428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd006b140 0x7fdfd0161520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.291+0000 7fdfd60fd640 1 --2- 172.21.15.26:0/1644236428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfd00cd610 0x7fdfd00cdaa0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009e50 tx=0x7fdfc801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d63eb622057def6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.291+0000 7fdfd48fa640 1 -- 172.21.15.26:0/1644236428 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc80a5810 con 0x7fdfd00cd610 2023-12-19T17:57:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.291+0000 7fdfd48fa640 1 -- 172.21.15.26:0/1644236428 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc80a5970 con 0x7fdfd00cd610 2023-12-19T17:57:24.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.291+0000 7fdfd48fa640 1 -- 172.21.15.26:0/1644236428 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc80a5d00 con 0x7fdfd00cd610 2023-12-19T17:57:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.291+0000 7fdfd7b60640 1 -- 172.21.15.26:0/1644236428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfd00cd610 msgr2=0x7fdfd00cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:24.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.291+0000 7fdfd7b60640 1 --2- 172.21.15.26:0/1644236428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfd00cd610 0x7fdfd00cdaa0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009e50 tx=0x7fdfc801c950 comp rx=0 tx=0).stop 2023-12-19T17:57:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 -- 172.21.15.26:0/1644236428 shutdown_connections 2023-12-19T17:57:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 --2- 172.21.15.26:0/1644236428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfd00cd610 0x7fdfd00cdaa0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 --2- 172.21.15.26:0/1644236428 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd006b140 0x7fdfd0161520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 -- 172.21.15.26:0/1644236428 >> 172.21.15.26:0/1644236428 conn(0x7fdfd006ef60 msgr2=0x7fdfd006f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 -- 172.21.15.26:0/1644236428 shutdown_connections 2023-12-19T17:57:24.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 -- 172.21.15.26:0/1644236428 wait complete. 2023-12-19T17:57:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 Processor -- start 2023-12-19T17:57:24.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 -- start start 2023-12-19T17:57:24.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfd006ad60 0x7fdfd0147ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:24.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfd006b140 0x7fdfd00c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:24.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cd610 0x7fdfd00c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:24.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd00b6320 con 0x7fdfd006ad60 2023-12-19T17:57:24.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd015fd30 con 0x7fdfd006b140 2023-12-19T17:57:24.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd7b60640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd01483e0 con 0x7fdfd00cd610 2023-12-19T17:57:24.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd58fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfd006ad60 0x7fdfd0147ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:24.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd58fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfd006ad60 0x7fdfd0147ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42636/0 (socket says 172.21.15.26:42636) 2023-12-19T17:57:24.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd58fc640 1 -- 172.21.15.26:0/2551522459 learned_addr learned my addr 172.21.15.26:0/2551522459 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:24.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd60fd640 1 --2- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cd610 0x7fdfd00c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:24.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd50fb640 1 --2- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfd006b140 0x7fdfd00c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd58fc640 1 -- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cd610 msgr2=0x7fdfd00c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd58fc640 1 --2- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cd610 0x7fdfd00c6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd58fc640 1 -- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfd006b140 msgr2=0x7fdfd00c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd58fc640 1 --2- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfd006b140 0x7fdfd00c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.295+0000 7fdfd58fc640 1 -- 172.21.15.26:0/2551522459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc8009b30 con 0x7fdfd006ad60 2023-12-19T17:57:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.299+0000 7fdfd58fc640 1 --2- 172.21.15.26:0/2551522459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfd006ad60 0x7fdfd0147ea0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00c3c0 tx=0x7fdfcc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.299+0000 7fdfc67fc640 1 -- 172.21.15.26:0/2551522459 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0052d0 con 0x7fdfd006ad60 2023-12-19T17:57:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.299+0000 7fdfc67fc640 1 -- 172.21.15.26:0/2551522459 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfcc005430 con 0x7fdfd006ad60 2023-12-19T17:57:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.299+0000 7fdfd7b60640 1 -- 172.21.15.26:0/2551522459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd00b36c0 con 0x7fdfd006ad60 2023-12-19T17:57:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.299+0000 7fdfc67fc640 1 -- 172.21.15.26:0/2551522459 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc003840 con 0x7fdfd006ad60 2023-12-19T17:57:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.299+0000 7fdfd7b60640 1 -- 172.21.15.26:0/2551522459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd00b3c80 con 0x7fdfd006ad60 2023-12-19T17:57:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.299+0000 7fdfd7b60640 1 -- 172.21.15.26:0/2551522459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd0074270 con 0x7fdfd006ad60 2023-12-19T17:57:24.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.299+0000 7fdfc67fc640 1 -- 172.21.15.26:0/2551522459 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 76) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fdfcc0039a0 con 0x7fdfd006ad60 2023-12-19T17:57:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.303+0000 7fdfc67fc640 1 --2- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfb4065b80 0x7fdfb4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.303+0000 7fdfc67fc640 1 -- 172.21.15.26:0/2551522459 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6931+0+0 (secure 0 0 0) 0x7fdfcc082b90 con 0x7fdfd006ad60 2023-12-19T17:57:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.303+0000 7fdfd50fb640 1 -- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfb4065b80 msgr2=0x7fdfb4068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:24.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.303+0000 7fdfd50fb640 1 --2- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfb4065b80 0x7fdfb4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:24.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.311+0000 7fdfc67fc640 1 -- 172.21.15.26:0/2551522459 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdfcc04ff80 con 0x7fdfd006ad60 2023-12-19T17:57:24.424 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.418+0000 7f3ba6bda180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:57:24.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfd7b60640 1 -- 172.21.15.26:0/2551522459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdfd0161a60 con 0x7fdfd006ad60 2023-12-19T17:57:24.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfc67fc640 1 -- 172.21.15.26:0/2551522459 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdfcc017020 con 0x7fdfd006ad60 2023-12-19T17:57:24.474 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T17:57:24.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfd7b60640 1 -- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfb4065b80 msgr2=0x7fdfb4068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:24.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfd7b60640 1 --2- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfb4065b80 0x7fdfb4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfd7b60640 1 -- 172.21.15.26:0/2551522459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfd006ad60 msgr2=0x7fdfd0147ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:24.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfd7b60640 1 --2- 172.21.15.26:0/2551522459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfd006ad60 0x7fdfd0147ea0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00c3c0 tx=0x7fdfcc00c880 comp rx=0 tx=0).stop 2023-12-19T17:57:24.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfd7b60640 1 -- 172.21.15.26:0/2551522459 shutdown_connections 2023-12-19T17:57:24.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfd7b60640 1 --2- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fdfb4065b80 0x7fdfb4068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfd7b60640 1 --2- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfd00cd610 0x7fdfd00c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfd7b60640 1 --2- 172.21.15.26:0/2551522459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfd006b140 0x7fdfd00c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfd7b60640 1 --2- 172.21.15.26:0/2551522459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfd006ad60 0x7fdfd0147ea0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfd7b60640 1 -- 172.21.15.26:0/2551522459 >> 172.21.15.26:0/2551522459 conn(0x7fdfd006ef60 msgr2=0x7fdfd00c4820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:24.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfd7b60640 1 -- 172.21.15.26:0/2551522459 shutdown_connections 2023-12-19T17:57:24.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.467+0000 7fdfd7b60640 1 -- 172.21.15.26:0/2551522459 wait complete. 2023-12-19T17:57:24.488 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:24.475+0000 7f25cb48d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:57:24.491 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T17:57:24.507 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.502+0000 7f3ba6bda180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:57:24.568 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:24.567+0000 7f25cb48d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:57:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677f998640 1 Processor -- start 2023-12-19T17:57:24.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677f998640 1 -- start start 2023-12-19T17:57:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677f998640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f677815cf80 0x7f67780cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:24.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677f998640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67780cdae0 0x7f677806a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677f998640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67780ce750 con 0x7f67780ccbd0 2023-12-19T17:57:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677f998640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67780afaa0 con 0x7f677815cf80 2023-12-19T17:57:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677f998640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67780afd80 con 0x7f67780cdae0 2023-12-19T17:57:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677d734640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f677815cf80 0x7f67780cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677d734640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f677815cf80 0x7f67780cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:35460/0 (socket says 172.21.15.26:35460) 2023-12-19T17:57:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677d734640 1 -- 172.21.15.26:0/375178944 learned_addr learned my addr 172.21.15.26:0/375178944 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677cf33640 1 --1- 172.21.15.26:0/375178944 >> v1:172.21.15.160:6789/0 conn(0x7f67780ccbd0 0x7f67780cd3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:40362/0 (socket says 172.21.15.26:40362) 2023-12-19T17:57:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677df35640 1 --2- 172.21.15.26:0/375178944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67780cdae0 0x7f677806a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677d734640 1 -- 172.21.15.26:0/375178944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67780cdae0 msgr2=0x7f677806a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677d734640 1 --2- 172.21.15.26:0/375178944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67780cdae0 0x7f677806a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677d734640 1 -- 172.21.15.26:0/375178944 >> v1:172.21.15.160:6789/0 conn(0x7f67780ccbd0 legacy=0x7f67780cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:24.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.643+0000 7f677d734640 1 -- 172.21.15.26:0/375178944 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6778069d20 con 0x7f677815cf80 2023-12-19T17:57:24.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.647+0000 7f677d734640 1 --2- 172.21.15.26:0/375178944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677815cf80 0x7f67780cb010 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6774009870 tx=0x7f677401c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10459530cf758582 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:24.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.647+0000 7f676ffff640 1 -- 172.21.15.26:0/375178944 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740aa670 con 0x7f677815cf80 2023-12-19T17:57:24.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.647+0000 7f676ffff640 1 -- 172.21.15.26:0/375178944 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67740aa7d0 con 0x7f677815cf80 2023-12-19T17:57:24.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.647+0000 7f676ffff640 1 -- 172.21.15.26:0/375178944 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740aab60 con 0x7f677815cf80 2023-12-19T17:57:24.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.647+0000 7f677f998640 1 -- 172.21.15.26:0/375178944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677815cf80 msgr2=0x7f67780cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:24.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.647+0000 7f677f998640 1 --2- 172.21.15.26:0/375178944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677815cf80 0x7f67780cb010 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6774009870 tx=0x7f677401c510 comp rx=0 tx=0).stop 2023-12-19T17:57:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.647+0000 7f677f998640 1 -- 172.21.15.26:0/375178944 shutdown_connections 2023-12-19T17:57:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.647+0000 7f677f998640 1 --2- 172.21.15.26:0/375178944 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67780cdae0 0x7f677806a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.647+0000 7f677f998640 1 --2- 172.21.15.26:0/375178944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677815cf80 0x7f67780cb010 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.647+0000 7f677f998640 1 -- 172.21.15.26:0/375178944 >> 172.21.15.26:0/375178944 conn(0x7f6778071490 msgr2=0x7f67780718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.647+0000 7f677f998640 1 -- 172.21.15.26:0/375178944 shutdown_connections 2023-12-19T17:57:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.647+0000 7f677f998640 1 -- 172.21.15.26:0/375178944 wait complete. 2023-12-19T17:57:24.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677f998640 1 Processor -- start 2023-12-19T17:57:24.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677f998640 1 -- start start 2023-12-19T17:57:24.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677f998640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67780ccbd0 0x7f67780b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:24.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677f998640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67780cdae0 0x7f677806b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:24.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677f998640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677815cf80 0x7f6778160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:24.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677f998640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67780b6270 con 0x7f67780cdae0 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677f998640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67780b35e0 con 0x7f677815cf80 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677f998640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67780b38c0 con 0x7f67780ccbd0 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677cf33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67780cdae0 0x7f677806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677d734640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67780ccbd0 0x7f67780b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677df35640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677815cf80 0x7f6778160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677df35640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677815cf80 0x7f6778160aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:35462/0 (socket says 172.21.15.26:35462) 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677cf33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67780cdae0 0x7f677806b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42638/0 (socket says 172.21.15.26:42638) 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677d734640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67780ccbd0 0x7f67780b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:39096/0 (socket says 172.21.15.26:39096) 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677df35640 1 -- 172.21.15.26:0/608678080 learned_addr learned my addr 172.21.15.26:0/608678080 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677df35640 1 -- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67780ccbd0 msgr2=0x7f67780b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677df35640 1 --2- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67780ccbd0 0x7f67780b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677df35640 1 -- 172.21.15.26:0/608678080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67780cdae0 msgr2=0x7f677806b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677df35640 1 --2- 172.21.15.26:0/608678080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67780cdae0 0x7f677806b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677df35640 1 -- 172.21.15.26:0/608678080 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67740094d0 con 0x7f677815cf80 2023-12-19T17:57:24.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677d734640 1 --2- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67780ccbd0 0x7f67780b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:24.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677cf33640 1 --2- 172.21.15.26:0/608678080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67780cdae0 0x7f677806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:24.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677df35640 1 --2- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677815cf80 0x7f6778160aa0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f677000ef90 tx=0x7f677000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:24.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f676dffb640 1 -- 172.21.15.26:0/608678080 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770009ca0 con 0x7f677815cf80 2023-12-19T17:57:24.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f676dffb640 1 -- 172.21.15.26:0/608678080 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677000f040 con 0x7f677815cf80 2023-12-19T17:57:24.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f676dffb640 1 -- 172.21.15.26:0/608678080 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67700146c0 con 0x7f677815cf80 2023-12-19T17:57:24.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677f998640 1 -- 172.21.15.26:0/608678080 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67780b3ba0 con 0x7f677815cf80 2023-12-19T17:57:24.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.651+0000 7f677f998640 1 -- 172.21.15.26:0/608678080 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67780b4160 con 0x7f677815cf80 2023-12-19T17:57:24.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.655+0000 7f677f998640 1 -- 172.21.15.26:0/608678080 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6778074270 con 0x7f677815cf80 2023-12-19T17:57:24.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.655+0000 7f676dffb640 1 -- 172.21.15.26:0/608678080 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 76) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f6770014820 con 0x7f677815cf80 2023-12-19T17:57:24.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.655+0000 7f676dffb640 1 --2- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f675c065b80 0x7f675c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:24.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.655+0000 7f676dffb640 1 -- 172.21.15.26:0/608678080 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6931+0+0 (secure 0 0 0) 0x7f6770084760 con 0x7f677815cf80 2023-12-19T17:57:24.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.655+0000 7f677d734640 1 -- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f675c065b80 msgr2=0x7f675c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:24.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.655+0000 7f677d734640 1 --2- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f675c065b80 0x7f675c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:24.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.659+0000 7f676dffb640 1 -- 172.21.15.26:0/608678080 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6770051a40 con 0x7f677815cf80 2023-12-19T17:57:24.742 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:24.739+0000 7f25cb48d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:57:24.830 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:24.827+0000 7f25cb48d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:57:24.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f677f998640 1 -- 172.21.15.26:0/608678080 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f677806a700 con 0x7f677815cf80 2023-12-19T17:57:24.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f676dffb640 1 -- 172.21.15.26:0/608678080 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v149) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6770055060 con 0x7f677815cf80 2023-12-19T17:57:24.833 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T17:57:24.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f677f998640 1 -- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f675c065b80 msgr2=0x7f675c068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:24.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f677f998640 1 --2- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f675c065b80 0x7f675c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f677f998640 1 -- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677815cf80 msgr2=0x7f6778160aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:24.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f677f998640 1 --2- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677815cf80 0x7f6778160aa0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f677000ef90 tx=0x7f677000c490 comp rx=0 tx=0).stop 2023-12-19T17:57:24.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f677f998640 1 -- 172.21.15.26:0/608678080 shutdown_connections 2023-12-19T17:57:24.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f677f998640 1 --2- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f677815cf80 0x7f6778160aa0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f677f998640 1 --2- 172.21.15.26:0/608678080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67780cdae0 0x7f677806b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f677f998640 1 --2- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f675c065b80 0x7f675c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f677f998640 1 --2- 172.21.15.26:0/608678080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67780ccbd0 0x7f67780b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:24.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f677f998640 1 -- 172.21.15.26:0/608678080 >> 172.21.15.26:0/608678080 conn(0x7f6778071490 msgr2=0x7f67780b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:24.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f677f998640 1 -- 172.21.15.26:0/608678080 shutdown_connections 2023-12-19T17:57:24.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:24.831+0000 7f677f998640 1 -- 172.21.15.26:0/608678080 wait complete. 2023-12-19T17:57:24.837 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.830+0000 7f3ba6bda180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:57:24.855 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T17:57:24.896 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.890+0000 7f3ba6bda180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:57:24.973 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.970+0000 7f3b2f34c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:24.974 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.970+0000 7f3b2f34c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:24.974 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.970+0000 7f3b2f34c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:24.974 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.970+0000 7f3b2f34c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:24.974 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.970+0000 7f3b2f34c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:24.975 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.970+0000 7f3b31b51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:24.975 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.970+0000 7f3b31b51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:24.975 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.970+0000 7f3b31b51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:24.975 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.970+0000 7f3b31b51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:24.975 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:24.970+0000 7f3b31b51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:25.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f56007f0640 1 Processor -- start 2023-12-19T17:57:25.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f56007f0640 1 -- start start 2023-12-19T17:57:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f56007f0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f55f815d0d0 0x7f55f80b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f56007f0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f815e200 con 0x7f55f80b4600 2023-12-19T17:57:25.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f56007f0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f80d0aa0 con 0x7f55f815ccf0 2023-12-19T17:57:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f56007f0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f815d680 con 0x7f55f815d0d0 2023-12-19T17:57:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f55fdd8b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f55f815d0d0 0x7f55f80b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f55fdd8b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f55f815d0d0 0x7f55f80b8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42652/0 (socket says 172.21.15.26:42652) 2023-12-19T17:57:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f55fed8d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f55f80b4600 0x7f55f80b49e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33254/0 (socket says 172.21.15.26:33254) 2023-12-19T17:57:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f55fdd8b640 1 -- 172.21.15.26:0/3095938391 learned_addr learned my addr 172.21.15.26:0/3095938391 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f55fdd8b640 1 -- 172.21.15.26:0/3095938391 >> v1:172.21.15.160:6789/0 conn(0x7f55f815ccf0 legacy=0x7f55f80d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f55fdd8b640 1 -- 172.21.15.26:0/3095938391 >> v1:172.21.15.26:6789/0 conn(0x7f55f80b4600 legacy=0x7f55f80b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f55fd58a640 1 -- 172.21.15.26:0/3095938391 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739191610 0 0) 0x7f55f80d0aa0 con 0x7f55f815ccf0 2023-12-19T17:57:25.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.011+0000 7f55fdd8b640 1 -- 172.21.15.26:0/3095938391 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f815d960 con 0x7f55f815d0d0 2023-12-19T17:57:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.015+0000 7f55fdd8b640 1 --2- 172.21.15.26:0/3095938391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f815d0d0 0x7f55f80b8400 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009870 tx=0x7f55e801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fb846139743d0a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:25.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.015+0000 7f55fd58a640 1 -- 172.21.15.26:0/3095938391 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e80ab600 con 0x7f55f815d0d0 2023-12-19T17:57:25.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.015+0000 7f55fd58a640 1 -- 172.21.15.26:0/3095938391 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e80a7050 con 0x7f55f815d0d0 2023-12-19T17:57:25.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.015+0000 7f55fd58a640 1 -- 172.21.15.26:0/3095938391 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e80ab980 con 0x7f55f815d0d0 2023-12-19T17:57:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.015+0000 7f56007f0640 1 -- 172.21.15.26:0/3095938391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f815d0d0 msgr2=0x7f55f80b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.015+0000 7f56007f0640 1 --2- 172.21.15.26:0/3095938391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f815d0d0 0x7f55f80b8400 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009870 tx=0x7f55e801c3d0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.015+0000 7f56007f0640 1 -- 172.21.15.26:0/3095938391 shutdown_connections 2023-12-19T17:57:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.015+0000 7f56007f0640 1 --2- 172.21.15.26:0/3095938391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f815d0d0 0x7f55f80b8400 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.015+0000 7f56007f0640 1 -- 172.21.15.26:0/3095938391 >> 172.21.15.26:0/3095938391 conn(0x7f55f8065ee0 msgr2=0x7f55f80cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.015+0000 7f56007f0640 1 -- 172.21.15.26:0/3095938391 shutdown_connections 2023-12-19T17:57:25.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.015+0000 7f56007f0640 1 -- 172.21.15.26:0/3095938391 wait complete. 2023-12-19T17:57:25.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f56007f0640 1 Processor -- start 2023-12-19T17:57:25.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f56007f0640 1 -- start start 2023-12-19T17:57:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f56007f0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f80b4600 0x7f55f81f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f56007f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f815ccf0 0x7f55f80d1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f56007f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55f815d0d0 0x7f55f81f6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f56007f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80c9cf0 con 0x7f55f815ccf0 2023-12-19T17:57:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f56007f0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80c9fd0 con 0x7f55f80b4600 2023-12-19T17:57:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f56007f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80ca2b0 con 0x7f55f815d0d0 2023-12-19T17:57:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55fdd8b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f815ccf0 0x7f55f80d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55fdd8b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f815ccf0 0x7f55f80d1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42668/0 (socket says 172.21.15.26:42668) 2023-12-19T17:57:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55fdd8b640 1 -- 172.21.15.26:0/3100265898 learned_addr learned my addr 172.21.15.26:0/3100265898 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55fe58c640 1 --2- 172.21.15.26:0/3100265898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f80b4600 0x7f55f81f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55fed8d640 1 --2- 172.21.15.26:0/3100265898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55f815d0d0 0x7f55f81f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55fdd8b640 1 -- 172.21.15.26:0/3100265898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55f815d0d0 msgr2=0x7f55f81f6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55fdd8b640 1 --2- 172.21.15.26:0/3100265898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55f815d0d0 0x7f55f81f6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55fdd8b640 1 -- 172.21.15.26:0/3100265898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f80b4600 msgr2=0x7f55f81f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55fdd8b640 1 --2- 172.21.15.26:0/3100265898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f80b4600 0x7f55f81f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55fdd8b640 1 -- 172.21.15.26:0/3100265898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e80094d0 con 0x7f55f815ccf0 2023-12-19T17:57:25.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55fdd8b640 1 --2- 172.21.15.26:0/3100265898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f815ccf0 0x7f55f80d1150 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f55e80ae040 tx=0x7f55e801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:25.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55eeffd640 1 -- 172.21.15.26:0/3100265898 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e80aa970 con 0x7f55f815ccf0 2023-12-19T17:57:25.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55eeffd640 1 -- 172.21.15.26:0/3100265898 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e80aaad0 con 0x7f55f815ccf0 2023-12-19T17:57:25.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f55eeffd640 1 -- 172.21.15.26:0/3100265898 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e80aae80 con 0x7f55f815ccf0 2023-12-19T17:57:25.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f56007f0640 1 -- 172.21.15.26:0/3100265898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f81f5900 con 0x7f55f815ccf0 2023-12-19T17:57:25.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.019+0000 7f56007f0640 1 -- 172.21.15.26:0/3100265898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f80d1690 con 0x7f55f815ccf0 2023-12-19T17:57:25.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.023+0000 7f56007f0640 1 -- 172.21.15.26:0/3100265898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f81f5e20 con 0x7f55f815ccf0 2023-12-19T17:57:25.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.023+0000 7f55eeffd640 1 -- 172.21.15.26:0/3100265898 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 77) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f55e80b2040 con 0x7f55f815ccf0 2023-12-19T17:57:25.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.023+0000 7f55eeffd640 1 -- 172.21.15.26:0/3100265898 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 7145+0+0 (secure 0 0 0) 0x7f55e811db30 con 0x7f55f815ccf0 2023-12-19T17:57:25.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.027+0000 7f55eeffd640 1 -- 172.21.15.26:0/3100265898 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f55f81f5e20 con 0x7f55f815ccf0 2023-12-19T17:57:25.179 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:25.175+0000 7f25cb48d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:57:25.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.187+0000 7f56007f0640 1 -- 172.21.15.26:0/3100265898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f55f81f5e20 con 0x7f55f815ccf0 2023-12-19T17:57:25.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.187+0000 7f55eeffd640 1 -- 172.21.15.26:0/3100265898 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v150) v1 ==== 100+0+11 (secure 0 0 0) 0x7f55f81f5e20 con 0x7f55f815ccf0 2023-12-19T17:57:25.189 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T17:57:25.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.187+0000 7f56007f0640 1 -- 172.21.15.26:0/3100265898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f815ccf0 msgr2=0x7f55f80d1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.187+0000 7f56007f0640 1 --2- 172.21.15.26:0/3100265898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f815ccf0 0x7f55f80d1150 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f55e80ae040 tx=0x7f55e801ac10 comp rx=0 tx=0).stop 2023-12-19T17:57:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.187+0000 7f56007f0640 1 -- 172.21.15.26:0/3100265898 shutdown_connections 2023-12-19T17:57:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.187+0000 7f56007f0640 1 --2- 172.21.15.26:0/3100265898 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55f815d0d0 0x7f55f81f6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.187+0000 7f56007f0640 1 --2- 172.21.15.26:0/3100265898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55f815ccf0 0x7f55f80d1150 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.187+0000 7f56007f0640 1 --2- 172.21.15.26:0/3100265898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55f80b4600 0x7f55f81f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.187+0000 7f56007f0640 1 -- 172.21.15.26:0/3100265898 >> 172.21.15.26:0/3100265898 conn(0x7f55f8065ee0 msgr2=0x7f55f80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.187+0000 7f56007f0640 1 -- 172.21.15.26:0/3100265898 shutdown_connections 2023-12-19T17:57:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.187+0000 7f56007f0640 1 -- 172.21.15.26:0/3100265898 wait complete. 2023-12-19T17:57:25.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:57:25.246 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:25.243+0000 7f25cb48d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:57:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d800b640 1 Processor -- start 2023-12-19T17:57:25.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d800b640 1 -- start start 2023-12-19T17:57:25.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d800b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d00b88e0 0x7f32d00b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d800b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f32d00b4b40 0x7f32d00b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d800b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d00d2960 con 0x7f32d015cf60 2023-12-19T17:57:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d800b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d00b4fd0 con 0x7f32d00b4b40 2023-12-19T17:57:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d800b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00d15f0 con 0x7f32d00b88e0 2023-12-19T17:57:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d5da7640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f32d015cf60 0x7f32d00d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33270/0 (socket says 172.21.15.26:33270) 2023-12-19T17:57:25.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d5da7640 1 -- 172.21.15.26:0/1773040330 learned_addr learned my addr 172.21.15.26:0/1773040330 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d65a8640 1 --2- 172.21.15.26:0/1773040330 >> v2:172.21.15.160:3300/0 conn(0x7f32d00b4b40 0x7f32d00b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d55a6640 1 --2- 172.21.15.26:0/1773040330 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d00b88e0 0x7f32d00b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d4da5640 1 -- 172.21.15.26:0/1773040330 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271051254 0 0) 0x7f32d00d2960 con 0x7f32d015cf60 2023-12-19T17:57:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d4da5640 1 -- 172.21.15.26:0/1773040330 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32bc003530 con 0x7f32d015cf60 2023-12-19T17:57:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d65a8640 1 -- 172.21.15.26:0/1773040330 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d00b88e0 msgr2=0x7f32d00b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d65a8640 1 --2- 172.21.15.26:0/1773040330 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d00b88e0 0x7f32d00b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d65a8640 1 -- 172.21.15.26:0/1773040330 >> v1:172.21.15.26:6789/0 conn(0x7f32d015cf60 legacy=0x7f32d00d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.355+0000 7f32d65a8640 1 -- 172.21.15.26:0/1773040330 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d00d18c0 con 0x7f32d00b4b40 2023-12-19T17:57:25.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.359+0000 7f32d65a8640 1 --2- 172.21.15.26:0/1773040330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d00b4b40 0x7f32d00b5cb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009d90 tx=0x7f32c801f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b88d6d74b7e74d51 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.359+0000 7f32d4da5640 1 -- 172.21.15.26:0/1773040330 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c80af600 con 0x7f32d00b4b40 2023-12-19T17:57:25.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.359+0000 7f32d4da5640 1 -- 172.21.15.26:0/1773040330 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c80ab050 con 0x7f32d00b4b40 2023-12-19T17:57:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.359+0000 7f32d4da5640 1 -- 172.21.15.26:0/1773040330 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c80af980 con 0x7f32d00b4b40 2023-12-19T17:57:25.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.359+0000 7f32d800b640 1 -- 172.21.15.26:0/1773040330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d00b4b40 msgr2=0x7f32d00b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.359+0000 7f32d800b640 1 --2- 172.21.15.26:0/1773040330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d00b4b40 0x7f32d00b5cb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009d90 tx=0x7f32c801f130 comp rx=0 tx=0).stop 2023-12-19T17:57:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.359+0000 7f32d800b640 1 -- 172.21.15.26:0/1773040330 shutdown_connections 2023-12-19T17:57:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.359+0000 7f32d800b640 1 --2- 172.21.15.26:0/1773040330 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d00b4b40 0x7f32d00b5cb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.359+0000 7f32d800b640 1 --2- 172.21.15.26:0/1773040330 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d00b88e0 0x7f32d00b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.359+0000 7f32d800b640 1 -- 172.21.15.26:0/1773040330 >> 172.21.15.26:0/1773040330 conn(0x7f32d0065ee0 msgr2=0x7f32d00cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.359+0000 7f32d800b640 1 -- 172.21.15.26:0/1773040330 shutdown_connections 2023-12-19T17:57:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.359+0000 7f32d800b640 1 -- 172.21.15.26:0/1773040330 wait complete. 2023-12-19T17:57:25.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d800b640 1 Processor -- start 2023-12-19T17:57:25.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d800b640 1 -- start start 2023-12-19T17:57:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d800b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d00b4b40 0x7f32d015fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d800b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d00b88e0 0x7f32d00c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d800b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d015cf60 0x7f32d00b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d800b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d015f730 con 0x7f32d00b4b40 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d800b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00b45a0 con 0x7f32d00b88e0 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d800b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d00c7100 con 0x7f32d015cf60 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d5da7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d00b4b40 0x7f32d015fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d5da7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d00b4b40 0x7f32d015fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42682/0 (socket says 172.21.15.26:42682) 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d5da7640 1 -- 172.21.15.26:0/2273825943 learned_addr learned my addr 172.21.15.26:0/2273825943 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d55a6640 1 --2- 172.21.15.26:0/2273825943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d00b88e0 0x7f32d00c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d65a8640 1 --2- 172.21.15.26:0/2273825943 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d015cf60 0x7f32d00b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d5da7640 1 -- 172.21.15.26:0/2273825943 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d015cf60 msgr2=0x7f32d00b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d5da7640 1 --2- 172.21.15.26:0/2273825943 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d015cf60 0x7f32d00b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d5da7640 1 -- 172.21.15.26:0/2273825943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d00b88e0 msgr2=0x7f32d00c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d5da7640 1 --2- 172.21.15.26:0/2273825943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d00b88e0 0x7f32d00c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d5da7640 1 -- 172.21.15.26:0/2273825943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c8009a70 con 0x7f32d00b4b40 2023-12-19T17:57:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d55a6640 1 --2- 172.21.15.26:0/2273825943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d00b88e0 0x7f32d00c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d5da7640 1 --2- 172.21.15.26:0/2273825943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d00b4b40 0x7f32d015fdf0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00c4d0 tx=0x7f32cc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32c67fc640 1 -- 172.21.15.26:0/2273825943 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc005cf0 con 0x7f32d00b4b40 2023-12-19T17:57:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32c67fc640 1 -- 172.21.15.26:0/2273825943 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc005e50 con 0x7f32d00b4b40 2023-12-19T17:57:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d800b640 1 -- 172.21.15.26:0/2273825943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d00c6810 con 0x7f32d00b4b40 2023-12-19T17:57:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32c67fc640 1 -- 172.21.15.26:0/2273825943 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc007270 con 0x7f32d00b4b40 2023-12-19T17:57:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d800b640 1 -- 172.21.15.26:0/2273825943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d00c6d90 con 0x7f32d00b4b40 2023-12-19T17:57:25.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.363+0000 7f32d800b640 1 -- 172.21.15.26:0/2273825943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d0074270 con 0x7f32d00b4b40 2023-12-19T17:57:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.367+0000 7f32c67fc640 1 -- 172.21.15.26:0/2273825943 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 77) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f32cc0073d0 con 0x7f32d00b4b40 2023-12-19T17:57:25.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.367+0000 7f32c67fc640 1 -- 172.21.15.26:0/2273825943 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 7145+0+0 (secure 0 0 0) 0x7f32cc082760 con 0x7f32d00b4b40 2023-12-19T17:57:25.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.371+0000 7f32c67fc640 1 -- 172.21.15.26:0/2273825943 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f32cc04ff90 con 0x7f32d00b4b40 2023-12-19T17:57:25.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.559+0000 7f32d800b640 1 -- 172.21.15.26:0/2273825943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32d00d0400 con 0x7f32d00b4b40 2023-12-19T17:57:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.559+0000 7f32c67fc640 1 -- 172.21.15.26:0/2273825943 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f32cc017090 con 0x7f32d00b4b40 2023-12-19T17:57:25.563 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:25.563 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8092,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3217828830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3217828830},{"type":"v1","addr":"172.21.15.160:6835","nonce":3217828830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2758554130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2758554130},{"type":"v1","addr":"172.21.15.160:6837","nonce":2758554130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3510623843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3510623843},{"type":"v1","addr":"172.21.15.26:6833","nonce":3510623843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:01:21.142075+0000","modified":"2023-12-19T17:01:24.167832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8487":{"gid":8487,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3823167404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3823167404},{"type":"v1","addr":"172.21.15.26:6835","nonce":3823167404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T17:57:25.563 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 65 2023-12-19T17:57:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.563+0000 7f32d800b640 1 -- 172.21.15.26:0/2273825943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d00b4b40 msgr2=0x7f32d015fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.563+0000 7f32d800b640 1 --2- 172.21.15.26:0/2273825943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d00b4b40 0x7f32d015fdf0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00c4d0 tx=0x7f32cc00c990 comp rx=0 tx=0).stop 2023-12-19T17:57:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.563+0000 7f32d800b640 1 -- 172.21.15.26:0/2273825943 shutdown_connections 2023-12-19T17:57:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.563+0000 7f32d800b640 1 --2- 172.21.15.26:0/2273825943 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d015cf60 0x7f32d00b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.563+0000 7f32d800b640 1 --2- 172.21.15.26:0/2273825943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d00b88e0 0x7f32d00c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.563+0000 7f32d800b640 1 --2- 172.21.15.26:0/2273825943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d00b4b40 0x7f32d015fdf0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.563+0000 7f32d800b640 1 -- 172.21.15.26:0/2273825943 >> 172.21.15.26:0/2273825943 conn(0x7f32d0065ee0 msgr2=0x7f32d015b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:25.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.563+0000 7f32d800b640 1 -- 172.21.15.26:0/2273825943 shutdown_connections 2023-12-19T17:57:25.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.563+0000 7f32d800b640 1 -- 172.21.15.26:0/2273825943 wait complete. 2023-12-19T17:57:25.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:57:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.767+0000 7fd5ff37b640 1 Processor -- start 2023-12-19T17:57:25.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.767+0000 7fd5ff37b640 1 -- start start 2023-12-19T17:57:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.767+0000 7fd5ff37b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5f806b3d0 0x7fd5f80ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.767+0000 7fd5ff37b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd5f80cec40 0x7fd5f815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.767+0000 7fd5ff37b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f806bc30 con 0x7fd5f8154220 2023-12-19T17:57:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.767+0000 7fd5ff37b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f80cf0d0 con 0x7fd5f806b3d0 2023-12-19T17:57:25.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.767+0000 7fd5ff37b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f80cc6e0 con 0x7fd5f80cec40 2023-12-19T17:57:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.771+0000 7fd5fc916640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5f806b3d0 0x7fd5f80ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.771+0000 7fd5fc916640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5f806b3d0 0x7fd5f80ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42694/0 (socket says 172.21.15.26:42694) 2023-12-19T17:57:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.771+0000 7fd5fc916640 1 -- 172.21.15.26:0/3946660880 learned_addr learned my addr 172.21.15.26:0/3946660880 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.771+0000 7fd5fd918640 1 --2- 172.21.15.26:0/3946660880 >> v2:172.21.15.160:3300/0 conn(0x7fd5f80cec40 0x7fd5f815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.771+0000 7fd5fc916640 1 -- 172.21.15.26:0/3946660880 >> v2:172.21.15.160:3300/0 conn(0x7fd5f80cec40 msgr2=0x7fd5f815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:25.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.771+0000 7fd5fc916640 1 --2- 172.21.15.26:0/3946660880 >> v2:172.21.15.160:3300/0 conn(0x7fd5f80cec40 0x7fd5f815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.771+0000 7fd5fc916640 1 -- 172.21.15.26:0/3946660880 >> v1:172.21.15.160:6789/0 conn(0x7fd5f8154220 legacy=0x7fd5f815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.771+0000 7fd5fc916640 1 -- 172.21.15.26:0/3946660880 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f806b950 con 0x7fd5f806b3d0 2023-12-19T17:57:25.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.771+0000 7fd5fd918640 1 --2- 172.21.15.26:0/3946660880 >> v2:172.21.15.160:3300/0 conn(0x7fd5f80cec40 0x7fd5f815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:57:25.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.771+0000 7fd5fc916640 1 --2- 172.21.15.26:0/3946660880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f806b3d0 0x7fd5f80ce700 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8157bd0 tx=0x7fd5e801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8086818bfc29f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.771+0000 7fd5effff640 1 -- 172.21.15.26:0/3946660880 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e80aa5f0 con 0x7fd5f806b3d0 2023-12-19T17:57:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.771+0000 7fd5effff640 1 -- 172.21.15.26:0/3946660880 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e80aa750 con 0x7fd5f806b3d0 2023-12-19T17:57:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.771+0000 7fd5effff640 1 -- 172.21.15.26:0/3946660880 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e80aaae0 con 0x7fd5f806b3d0 2023-12-19T17:57:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 -- 172.21.15.26:0/3946660880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f806b3d0 msgr2=0x7fd5f80ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 --2- 172.21.15.26:0/3946660880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f806b3d0 0x7fd5f80ce700 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8157bd0 tx=0x7fd5e801c490 comp rx=0 tx=0).stop 2023-12-19T17:57:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 -- 172.21.15.26:0/3946660880 shutdown_connections 2023-12-19T17:57:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 --2- 172.21.15.26:0/3946660880 >> v2:172.21.15.160:3300/0 conn(0x7fd5f80cec40 0x7fd5f815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 --2- 172.21.15.26:0/3946660880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f806b3d0 0x7fd5f80ce700 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 -- 172.21.15.26:0/3946660880 >> 172.21.15.26:0/3946660880 conn(0x7fd5f80658e0 msgr2=0x7fd5f8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 -- 172.21.15.26:0/3946660880 shutdown_connections 2023-12-19T17:57:25.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 -- 172.21.15.26:0/3946660880 wait complete. 2023-12-19T17:57:25.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 Processor -- start 2023-12-19T17:57:25.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 -- start start 2023-12-19T17:57:25.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5f806b3d0 0x7fd5f80b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cec40 0x7fd5f80af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5f8154220 0x7fd5f814e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f80b19a0 con 0x7fd5f80cec40 2023-12-19T17:57:25.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f806f410 con 0x7fd5f806b3d0 2023-12-19T17:57:25.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5ff37b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f814f620 con 0x7fd5f8154220 2023-12-19T17:57:25.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5fc916640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cec40 0x7fd5f80af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5fd918640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5f8154220 0x7fd5f814e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5fc916640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cec40 0x7fd5f80af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42710/0 (socket says 172.21.15.26:42710) 2023-12-19T17:57:25.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5fc916640 1 -- 172.21.15.26:0/1154693623 learned_addr learned my addr 172.21.15.26:0/1154693623 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:25.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5fd117640 1 --2- 172.21.15.26:0/1154693623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5f806b3d0 0x7fd5f80b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5fc916640 1 -- 172.21.15.26:0/1154693623 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5f8154220 msgr2=0x7fd5f814e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5fc916640 1 --2- 172.21.15.26:0/1154693623 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5f8154220 0x7fd5f814e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5fc916640 1 -- 172.21.15.26:0/1154693623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5f806b3d0 msgr2=0x7fd5f80b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5fc916640 1 --2- 172.21.15.26:0/1154693623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5f806b3d0 0x7fd5f80b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.775+0000 7fd5fc916640 1 -- 172.21.15.26:0/1154693623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e80094d0 con 0x7fd5f80cec40 2023-12-19T17:57:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.779+0000 7fd5fc916640 1 --2- 172.21.15.26:0/1154693623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cec40 0x7fd5f80af6b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd5e801a0c0 tx=0x7fd5e801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.779+0000 7fd5edffb640 1 -- 172.21.15.26:0/1154693623 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e80b3800 con 0x7fd5f80cec40 2023-12-19T17:57:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.779+0000 7fd5edffb640 1 -- 172.21.15.26:0/1154693623 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e80b3980 con 0x7fd5f80cec40 2023-12-19T17:57:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.779+0000 7fd5edffb640 1 -- 172.21.15.26:0/1154693623 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e80b3d30 con 0x7fd5f80cec40 2023-12-19T17:57:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.779+0000 7fd5ff37b640 1 -- 172.21.15.26:0/1154693623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f806f6b0 con 0x7fd5f80cec40 2023-12-19T17:57:25.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.779+0000 7fd5ff37b640 1 -- 172.21.15.26:0/1154693623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f806fc70 con 0x7fd5f80cec40 2023-12-19T17:57:25.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.779+0000 7fd5ff37b640 1 -- 172.21.15.26:0/1154693623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f8074270 con 0x7fd5f80cec40 2023-12-19T17:57:25.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.779+0000 7fd5edffb640 1 -- 172.21.15.26:0/1154693623 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 77) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fd5e80b2380 con 0x7fd5f80cec40 2023-12-19T17:57:25.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.779+0000 7fd5edffb640 1 -- 172.21.15.26:0/1154693623 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 7145+0+0 (secure 0 0 0) 0x7fd5e80a7070 con 0x7fd5f80cec40 2023-12-19T17:57:25.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.783+0000 7fd5edffb640 1 -- 172.21.15.26:0/1154693623 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd5e80ead60 con 0x7fd5f80cec40 2023-12-19T17:57:25.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.911+0000 7fd5edffb640 1 -- 172.21.15.26:0/1154693623 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 78) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fd5e80ea810 con 0x7fd5f80cec40 2023-12-19T17:57:25.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.911+0000 7fd5edffb640 1 --2- 172.21.15.26:0/1154693623 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd5dc06bb90 0x7fd5dc06df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:25.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.915+0000 7fd5fd117640 1 --2- 172.21.15.26:0/1154693623 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd5dc06bb90 0x7fd5dc06df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:25.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.915+0000 7fd5fd117640 1 --2- 172.21.15.26:0/1154693623 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd5dc06bb90 0x7fd5dc06df80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4007fd0 tx=0x7fd5f4007e40 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:25.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.947+0000 7fd5ff37b640 1 -- 172.21.15.26:0/1154693623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5f80cdf60 con 0x7fd5f80cec40 2023-12-19T17:57:25.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.947+0000 7fd5edffb640 1 -- 172.21.15.26:0/1154693623 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+18285 (secure 0 0 0) 0x7fd5e80c08a0 con 0x7fd5f80cec40 2023-12-19T17:57:25.951 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:25.951 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":150,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:57:24.900124+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:01:19.682876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T17:01:20.140612+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":149,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.f","mappings":[{"from":0,"to":2}]},{"pgid":"17.3f","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/635672045":"2023-12-20T17:57:24.900091+0000","172.21.15.160:0/988541364":"2023-12-20T17:57:24.900091+0000","172.21.15.160:0/3538348635":"2023-12-20T17:57:24.900091+0000","172.21.15.160:0/4117035290":"2023-12-20T17:57:24.900091+0000","172.21.15.160:6833/38098":"2023-12-20T17:01:32.971594+0000","172.21.15.160:6832/38098":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/1746260591":"2023-12-20T17:57:24.900091+0000","172.21.15.160:0/1067813784":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/3376805359":"2023-12-20T17:57:24.900091+0000","172.21.15.160:0/1947760249":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/464310353":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/2140731666":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/870873437":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/950669571":"2023-12-20T17:01:32.971594+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":17,"snaps":[{"begin":5,"length":53},{"begin":61,"length":53}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:57:25.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.951+0000 7fd5ff37b640 1 -- 172.21.15.26:0/1154693623 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd5dc06bb90 msgr2=0x7fd5dc06df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.951+0000 7fd5ff37b640 1 --2- 172.21.15.26:0/1154693623 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd5dc06bb90 0x7fd5dc06df80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4007fd0 tx=0x7fd5f4007e40 comp rx=0 tx=0).stop 2023-12-19T17:57:25.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.951+0000 7fd5ff37b640 1 -- 172.21.15.26:0/1154693623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cec40 msgr2=0x7fd5f80af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.951+0000 7fd5ff37b640 1 --2- 172.21.15.26:0/1154693623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cec40 0x7fd5f80af6b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fd5e801a0c0 tx=0x7fd5e801acd0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.951+0000 7fd5ff37b640 1 -- 172.21.15.26:0/1154693623 shutdown_connections 2023-12-19T17:57:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.951+0000 7fd5ff37b640 1 --2- 172.21.15.26:0/1154693623 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fd5dc06bb90 0x7fd5dc06df80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.951+0000 7fd5ff37b640 1 --2- 172.21.15.26:0/1154693623 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5f8154220 0x7fd5f814e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.951+0000 7fd5ff37b640 1 --2- 172.21.15.26:0/1154693623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cec40 0x7fd5f80af6b0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.951+0000 7fd5ff37b640 1 --2- 172.21.15.26:0/1154693623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5f806b3d0 0x7fd5f80b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.951+0000 7fd5ff37b640 1 -- 172.21.15.26:0/1154693623 >> 172.21.15.26:0/1154693623 conn(0x7fd5f80658e0 msgr2=0x7fd5f815dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.951+0000 7fd5ff37b640 1 -- 172.21.15.26:0/1154693623 shutdown_connections 2023-12-19T17:57:25.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:25.951+0000 7fd5ff37b640 1 -- 172.21.15.26:0/1154693623 wait complete. 2023-12-19T17:57:25.973 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T17:57:25.973 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:57:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5bab39640 1 Processor -- start 2023-12-19T17:57:26.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5bab39640 1 -- start start 2023-12-19T17:57:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5bab39640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa5b4154220 0x7fa5b415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5bab39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5b40cee10 0x7fa5b415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5bab39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b406bc30 con 0x7fa5b406b3d0 2023-12-19T17:57:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5bab39640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b40cc6e0 con 0x7fa5b4154220 2023-12-19T17:57:26.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5bab39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b415afc0 con 0x7fa5b40cee10 2023-12-19T17:57:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5b88d5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa5b4154220 0x7fa5b415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5b90d6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5b40cee10 0x7fa5b415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5b88d5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa5b4154220 0x7fa5b415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:54390/0 (socket says 172.21.15.26:54390) 2023-12-19T17:57:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5b88d5640 1 -- 172.21.15.26:0/2641653691 learned_addr learned my addr 172.21.15.26:0/2641653691 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5b3fff640 1 --1- 172.21.15.26:0/2641653691 >> v1:172.21.15.160:6789/0 conn(0x7fa5b406b3d0 0x7fa5b40ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:42526/0 (socket says 172.21.15.26:42526) 2023-12-19T17:57:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5b88d5640 1 -- 172.21.15.26:0/2641653691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5b40cee10 msgr2=0x7fa5b415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5b88d5640 1 --2- 172.21.15.26:0/2641653691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5b40cee10 0x7fa5b415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5b88d5640 1 -- 172.21.15.26:0/2641653691 >> v1:172.21.15.160:6789/0 conn(0x7fa5b406b3d0 legacy=0x7fa5b40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.115+0000 7fa5b88d5640 1 -- 172.21.15.26:0/2641653691 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b406b950 con 0x7fa5b4154220 2023-12-19T17:57:26.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.119+0000 7fa5b88d5640 1 --2- 172.21.15.26:0/2641653691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5b4154220 0x7fa5b415aa80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009870 tx=0x7fa5a801e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fed701c1b0214ad server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:26.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.119+0000 7fa5b37fe640 1 -- 172.21.15.26:0/2641653691 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80a86f0 con 0x7fa5b4154220 2023-12-19T17:57:26.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.119+0000 7fa5b37fe640 1 -- 172.21.15.26:0/2641653691 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a80a8850 con 0x7fa5b4154220 2023-12-19T17:57:26.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.119+0000 7fa5b37fe640 1 -- 172.21.15.26:0/2641653691 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a80a8be0 con 0x7fa5b4154220 2023-12-19T17:57:26.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.119+0000 7fa5bab39640 1 -- 172.21.15.26:0/2641653691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5b4154220 msgr2=0x7fa5b415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.119+0000 7fa5bab39640 1 --2- 172.21.15.26:0/2641653691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5b4154220 0x7fa5b415aa80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8009870 tx=0x7fa5a801e720 comp rx=0 tx=0).stop 2023-12-19T17:57:26.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.119+0000 7fa5bab39640 1 -- 172.21.15.26:0/2641653691 shutdown_connections 2023-12-19T17:57:26.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.119+0000 7fa5bab39640 1 --2- 172.21.15.26:0/2641653691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5b40cee10 0x7fa5b415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.119+0000 7fa5bab39640 1 --2- 172.21.15.26:0/2641653691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5b4154220 0x7fa5b415aa80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.119+0000 7fa5bab39640 1 -- 172.21.15.26:0/2641653691 >> 172.21.15.26:0/2641653691 conn(0x7fa5b40658e0 msgr2=0x7fa5b4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:26.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.119+0000 7fa5bab39640 1 -- 172.21.15.26:0/2641653691 shutdown_connections 2023-12-19T17:57:26.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.119+0000 7fa5bab39640 1 -- 172.21.15.26:0/2641653691 wait complete. 2023-12-19T17:57:26.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5bab39640 1 Processor -- start 2023-12-19T17:57:26.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5bab39640 1 -- start start 2023-12-19T17:57:26.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5bab39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5b406b3d0 0x7fa5b414e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5bab39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5b40cee10 0x7fa5b4152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5bab39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b4154220 0x7fa5b406f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5bab39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b414ebc0 con 0x7fa5b4154220 2023-12-19T17:57:26.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5bab39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b414f7d0 con 0x7fa5b406b3d0 2023-12-19T17:57:26.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5bab39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b4153190 con 0x7fa5b40cee10 2023-12-19T17:57:26.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b90d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b4154220 0x7fa5b406f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b90d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b4154220 0x7fa5b406f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43660/0 (socket says 172.21.15.26:43660) 2023-12-19T17:57:26.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b90d6640 1 -- 172.21.15.26:0/1631444463 learned_addr learned my addr 172.21.15.26:0/1631444463 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:26.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b88d5640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5b406b3d0 0x7fa5b414e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b3fff640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5b40cee10 0x7fa5b4152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b90d6640 1 -- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5b40cee10 msgr2=0x7fa5b4152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b90d6640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5b40cee10 0x7fa5b4152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b90d6640 1 -- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5b406b3d0 msgr2=0x7fa5b414e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b90d6640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5b406b3d0 0x7fa5b414e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b90d6640 1 -- 172.21.15.26:0/1631444463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a80094d0 con 0x7fa5b4154220 2023-12-19T17:57:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b3fff640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5b40cee10 0x7fa5b4152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:57:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b90d6640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b4154220 0x7fa5b406f370 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00efc0 tx=0x7fa5ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b17fa640 1 -- 172.21.15.26:0/1631444463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac009ba0 con 0x7fa5b4154220 2023-12-19T17:57:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b17fa640 1 -- 172.21.15.26:0/1631444463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac00f040 con 0x7fa5b4154220 2023-12-19T17:57:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5b17fa640 1 -- 172.21.15.26:0/1631444463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac0146c0 con 0x7fa5b4154220 2023-12-19T17:57:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5bab39640 1 -- 172.21.15.26:0/1631444463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b414eea0 con 0x7fa5b4154220 2023-12-19T17:57:26.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.123+0000 7fa5bab39640 1 -- 172.21.15.26:0/1631444463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b414f460 con 0x7fa5b4154220 2023-12-19T17:57:26.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.127+0000 7fa5bab39640 1 -- 172.21.15.26:0/1631444463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b414f980 con 0x7fa5b4154220 2023-12-19T17:57:26.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.127+0000 7fa5b17fa640 1 -- 172.21.15.26:0/1631444463 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 78) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fa5ac014820 con 0x7fa5b4154220 2023-12-19T17:57:26.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.127+0000 7fa5b17fa640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa5980658b0 0x7fa598067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.127+0000 7fa5b17fa640 1 -- 172.21.15.26:0/1631444463 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 7145+0+0 (secure 0 0 0) 0x7fa5ac0841d0 con 0x7fa5b4154220 2023-12-19T17:57:26.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.127+0000 7fa5b88d5640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa5980658b0 0x7fa598067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.127+0000 7fa5b88d5640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa5980658b0 0x7fa598067d70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80b1000 tx=0x7fa5a80a9000 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:26.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.131+0000 7fa5b17fa640 1 -- 172.21.15.26:0/1631444463 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa5b414f980 con 0x7fa5b4154220 2023-12-19T17:57:26.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.323+0000 7fa5bab39640 1 -- 172.21.15.26:0/1631444463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5b414f620 con 0x7fa5b4154220 2023-12-19T17:57:26.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.323+0000 7fa5b17fa640 1 -- 172.21.15.26:0/1631444463 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fa5ac018090 con 0x7fa5b4154220 2023-12-19T17:57:26.325 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:26.326 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8092,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3217828830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3217828830},{"type":"v1","addr":"172.21.15.160:6835","nonce":3217828830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2758554130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2758554130},{"type":"v1","addr":"172.21.15.160:6837","nonce":2758554130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3510623843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3510623843},{"type":"v1","addr":"172.21.15.26:6833","nonce":3510623843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:01:21.142075+0000","modified":"2023-12-19T17:01:24.167832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8487":{"gid":8487,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3823167404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3823167404},{"type":"v1","addr":"172.21.15.26:6835","nonce":3823167404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T17:57:26.326 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 65 2023-12-19T17:57:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.323+0000 7fa5bab39640 1 -- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa5980658b0 msgr2=0x7fa598067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.323+0000 7fa5bab39640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa5980658b0 0x7fa598067d70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80b1000 tx=0x7fa5a80a9000 comp rx=0 tx=0).stop 2023-12-19T17:57:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.323+0000 7fa5bab39640 1 -- 172.21.15.26:0/1631444463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b4154220 msgr2=0x7fa5b406f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.323+0000 7fa5bab39640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b4154220 0x7fa5b406f370 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00efc0 tx=0x7fa5ac00c490 comp rx=0 tx=0).stop 2023-12-19T17:57:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.323+0000 7fa5bab39640 1 -- 172.21.15.26:0/1631444463 shutdown_connections 2023-12-19T17:57:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.323+0000 7fa5bab39640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5b4154220 0x7fa5b406f370 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.323+0000 7fa5bab39640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa5980658b0 0x7fa598067d70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.323+0000 7fa5bab39640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa5b40cee10 0x7fa5b4152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.327+0000 7fa5bab39640 1 --2- 172.21.15.26:0/1631444463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5b406b3d0 0x7fa5b414e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.327+0000 7fa5bab39640 1 -- 172.21.15.26:0/1631444463 >> 172.21.15.26:0/1631444463 conn(0x7fa5b40658e0 msgr2=0x7fa5b415c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.327+0000 7fa5bab39640 1 -- 172.21.15.26:0/1631444463 shutdown_connections 2023-12-19T17:57:26.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.327+0000 7fa5bab39640 1 -- 172.21.15.26:0/1631444463 wait complete. 2023-12-19T17:57:26.347 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:57:26.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.487+0000 7fa3fe3b3640 1 Processor -- start 2023-12-19T17:57:26.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.487+0000 7fa3fe3b3640 1 -- start start 2023-12-19T17:57:26.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fe3b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa3f80cb1d0 0x7fa3f80cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fe3b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3f806bee0 0x7fa3f806c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fe3b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3f8164c90 0x7fa3f8165120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fe3b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f80cc030 con 0x7fa3f8164c90 2023-12-19T17:57:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fe3b3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f806d4b0 con 0x7fa3f80cb1d0 2023-12-19T17:57:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fe3b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80cbaf0 con 0x7fa3f806bee0 2023-12-19T17:57:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fc950640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3f8164c90 0x7fa3f8165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fc950640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3f8164c90 0x7fa3f8165120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43672/0 (socket says 172.21.15.26:43672) 2023-12-19T17:57:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fc950640 1 -- 172.21.15.26:0/2235941242 learned_addr learned my addr 172.21.15.26:0/2235941242 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3f7fff640 1 --2- 172.21.15.26:0/2235941242 >> v2:172.21.15.160:3300/0 conn(0x7fa3f80cb1d0 0x7fa3f80cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3f77fe640 1 --2- 172.21.15.26:0/2235941242 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3f806bee0 0x7fa3f806c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fc950640 1 -- 172.21.15.26:0/2235941242 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3f806bee0 msgr2=0x7fa3f806c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fc950640 1 --2- 172.21.15.26:0/2235941242 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3f806bee0 0x7fa3f806c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fc950640 1 -- 172.21.15.26:0/2235941242 >> v2:172.21.15.160:3300/0 conn(0x7fa3f80cb1d0 msgr2=0x7fa3f80cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fc950640 1 --2- 172.21.15.26:0/2235941242 >> v2:172.21.15.160:3300/0 conn(0x7fa3f80cb1d0 0x7fa3f80cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fc950640 1 -- 172.21.15.26:0/2235941242 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f806cc10 con 0x7fa3f8164c90 2023-12-19T17:57:26.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.491+0000 7fa3fc950640 1 --2- 172.21.15.26:0/2235941242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f8164c90 0x7fa3f8165120 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fa3f80ccb20 tx=0x7fa3f001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74dc0a746123022 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:26.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3f6ffd640 1 -- 172.21.15.26:0/2235941242 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f00a5790 con 0x7fa3f8164c90 2023-12-19T17:57:26.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3f6ffd640 1 -- 172.21.15.26:0/2235941242 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f00a58f0 con 0x7fa3f8164c90 2023-12-19T17:57:26.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3f6ffd640 1 -- 172.21.15.26:0/2235941242 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f00a5c80 con 0x7fa3f8164c90 2023-12-19T17:57:26.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2235941242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f8164c90 msgr2=0x7fa3f8165120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 --2- 172.21.15.26:0/2235941242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f8164c90 0x7fa3f8165120 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fa3f80ccb20 tx=0x7fa3f001c910 comp rx=0 tx=0).stop 2023-12-19T17:57:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2235941242 shutdown_connections 2023-12-19T17:57:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 --2- 172.21.15.26:0/2235941242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f8164c90 0x7fa3f8165120 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 --2- 172.21.15.26:0/2235941242 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3f806bee0 0x7fa3f806c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 --2- 172.21.15.26:0/2235941242 >> v2:172.21.15.160:3300/0 conn(0x7fa3f80cb1d0 0x7fa3f80cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2235941242 >> 172.21.15.26:0/2235941242 conn(0x7fa3f80658e0 msgr2=0x7fa3f8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2235941242 shutdown_connections 2023-12-19T17:57:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2235941242 wait complete. 2023-12-19T17:57:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 Processor -- start 2023-12-19T17:57:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 -- start start 2023-12-19T17:57:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3f806bee0 0x7fa3f80cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3f80cb1d0 0x7fa3f815b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f8164c90 0x7fa3f815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f815bed0 con 0x7fa3f8164c90 2023-12-19T17:57:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f814c760 con 0x7fa3f806bee0 2023-12-19T17:57:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.495+0000 7fa3fe3b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f815d910 con 0x7fa3f80cb1d0 2023-12-19T17:57:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3fc950640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f8164c90 0x7fa3f815d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3fc950640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f8164c90 0x7fa3f815d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43688/0 (socket says 172.21.15.26:43688) 2023-12-19T17:57:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3fc950640 1 -- 172.21.15.26:0/2271035240 learned_addr learned my addr 172.21.15.26:0/2271035240 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3f77fe640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3f80cb1d0 0x7fa3f815b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3f7fff640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3f806bee0 0x7fa3f80cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3fc950640 1 -- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3f80cb1d0 msgr2=0x7fa3f815b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3fc950640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3f80cb1d0 0x7fa3f815b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3fc950640 1 -- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3f806bee0 msgr2=0x7fa3f80cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3fc950640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3f806bee0 0x7fa3f80cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3fc950640 1 -- 172.21.15.26:0/2271035240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0009b30 con 0x7fa3f8164c90 2023-12-19T17:57:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3fc950640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f8164c90 0x7fa3f815d3d0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00af040 tx=0x7fa3f00196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3f4ff9640 1 -- 172.21.15.26:0/2271035240 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f00b8670 con 0x7fa3f8164c90 2023-12-19T17:57:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3f4ff9640 1 -- 172.21.15.26:0/2271035240 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f00b87d0 con 0x7fa3f8164c90 2023-12-19T17:57:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2271035240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f80ce810 con 0x7fa3f8164c90 2023-12-19T17:57:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3f4ff9640 1 -- 172.21.15.26:0/2271035240 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f00b8b80 con 0x7fa3f8164c90 2023-12-19T17:57:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2271035240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f80ced90 con 0x7fa3f8164c90 2023-12-19T17:57:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2271035240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f8074270 con 0x7fa3f8164c90 2023-12-19T17:57:26.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.499+0000 7fa3f4ff9640 1 -- 172.21.15.26:0/2271035240 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 78) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fa3f00b8ce0 con 0x7fa3f8164c90 2023-12-19T17:57:26.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.503+0000 7fa3f4ff9640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa3e40658b0 0x7fa3e4067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.503+0000 7fa3f4ff9640 1 -- 172.21.15.26:0/2271035240 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 7145+0+0 (secure 0 0 0) 0x7fa3f011e7f0 con 0x7fa3f8164c90 2023-12-19T17:57:26.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.503+0000 7fa3f7fff640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa3e40658b0 0x7fa3e4067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.503+0000 7fa3f7fff640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa3e40658b0 0x7fa3e4067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009ac0 tx=0x7fa3e8009190 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.507+0000 7fa3f4ff9640 1 -- 172.21.15.26:0/2271035240 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa3f00ebf30 con 0x7fa3f8164c90 2023-12-19T17:57:26.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2271035240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3f80ccdf0 con 0x7fa3f8164c90 2023-12-19T17:57:26.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3f4ff9640 1 -- 172.21.15.26:0/2271035240 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fa3f0009510 con 0x7fa3f8164c90 2023-12-19T17:57:26.696 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:26.697 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8092,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3217828830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3217828830},{"type":"v1","addr":"172.21.15.160:6835","nonce":3217828830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2758554130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2758554130},{"type":"v1","addr":"172.21.15.160:6837","nonce":2758554130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3510623843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3510623843},{"type":"v1","addr":"172.21.15.26:6833","nonce":3510623843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:01:21.142075+0000","modified":"2023-12-19T17:01:24.167832+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8487":{"gid":8487,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3823167404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3823167404},{"type":"v1","addr":"172.21.15.26:6835","nonce":3823167404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T17:57:26.697 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 65 2023-12-19T17:57:26.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa3e40658b0 msgr2=0x7fa3e4067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3fe3b3640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa3e40658b0 0x7fa3e4067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009ac0 tx=0x7fa3e8009190 comp rx=0 tx=0).stop 2023-12-19T17:57:26.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2271035240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f8164c90 msgr2=0x7fa3f815d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3fe3b3640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f8164c90 0x7fa3f815d3d0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fa3f00af040 tx=0x7fa3f00196b0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2271035240 shutdown_connections 2023-12-19T17:57:26.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3fe3b3640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3f8164c90 0x7fa3f815d3d0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3fe3b3640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa3e40658b0 0x7fa3e4067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3fe3b3640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3f80cb1d0 0x7fa3f815b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3fe3b3640 1 --2- 172.21.15.26:0/2271035240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3f806bee0 0x7fa3f80cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2271035240 >> 172.21.15.26:0/2271035240 conn(0x7fa3f80658e0 msgr2=0x7fa3f80b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:26.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2271035240 shutdown_connections 2023-12-19T17:57:26.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.695+0000 7fa3fe3b3640 1 -- 172.21.15.26:0/2271035240 wait complete. 2023-12-19T17:57:26.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:57:26.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.859+0000 7efebf98f640 1 Processor -- start 2023-12-19T17:57:26.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.859+0000 7efebf98f640 1 -- start start 2023-12-19T17:57:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.863+0000 7efebf98f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efeb80b2d50 0x7efeb816c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.863+0000 7efebf98f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb806d4b0 con 0x7efeb806bee0 2023-12-19T17:57:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.863+0000 7efebf98f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb806cc10 con 0x7efeb80cb1d0 2023-12-19T17:57:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.863+0000 7efebf98f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb80cc030 con 0x7efeb80b2d50 2023-12-19T17:57:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.863+0000 7efebcf2a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efeb806bee0 0x7efeb806c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34408/0 (socket says 172.21.15.26:34408) 2023-12-19T17:57:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.863+0000 7efebcf2a640 1 -- 172.21.15.26:0/3183132399 learned_addr learned my addr 172.21.15.26:0/3183132399 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.863+0000 7efebdf2c640 1 --2- 172.21.15.26:0/3183132399 >> v2:172.21.15.160:3300/0 conn(0x7efeb80b2d50 0x7efeb816c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.863+0000 7efeaffff640 1 -- 172.21.15.26:0/3183132399 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151022998 0 0) 0x7efeb806d4b0 con 0x7efeb806bee0 2023-12-19T17:57:26.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.863+0000 7efeaffff640 1 -- 172.21.15.26:0/3183132399 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efea4003530 con 0x7efeb806bee0 2023-12-19T17:57:26.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.863+0000 7efebdf2c640 1 -- 172.21.15.26:0/3183132399 >> v1:172.21.15.160:6789/0 conn(0x7efeb80cb1d0 legacy=0x7efeb80cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.863+0000 7efebdf2c640 1 -- 172.21.15.26:0/3183132399 >> v1:172.21.15.26:6789/0 conn(0x7efeb806bee0 legacy=0x7efeb806c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.863+0000 7efebdf2c640 1 -- 172.21.15.26:0/3183132399 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb806cef0 con 0x7efeb80b2d50 2023-12-19T17:57:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebdf2c640 1 --2- 172.21.15.26:0/3183132399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb80b2d50 0x7efeb816c020 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009d90 tx=0x7efeb001c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e200ee4cce39c192 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efeaffff640 1 -- 172.21.15.26:0/3183132399 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb00a5980 con 0x7efeb80b2d50 2023-12-19T17:57:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efeaffff640 1 -- 172.21.15.26:0/3183132399 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb00a5ae0 con 0x7efeb80b2d50 2023-12-19T17:57:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efeaffff640 1 -- 172.21.15.26:0/3183132399 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb00a5e70 con 0x7efeb80b2d50 2023-12-19T17:57:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 -- 172.21.15.26:0/3183132399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb80b2d50 msgr2=0x7efeb816c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 --2- 172.21.15.26:0/3183132399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb80b2d50 0x7efeb816c020 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009d90 tx=0x7efeb001c8c0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 -- 172.21.15.26:0/3183132399 shutdown_connections 2023-12-19T17:57:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 --2- 172.21.15.26:0/3183132399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb80b2d50 0x7efeb816c020 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 -- 172.21.15.26:0/3183132399 >> 172.21.15.26:0/3183132399 conn(0x7efeb80658e0 msgr2=0x7efeb8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 -- 172.21.15.26:0/3183132399 shutdown_connections 2023-12-19T17:57:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 -- 172.21.15.26:0/3183132399 wait complete. 2023-12-19T17:57:26.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 Processor -- start 2023-12-19T17:57:26.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 -- start start 2023-12-19T17:57:26.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb806bee0 0x7efeb80cec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb80b2d50 0x7efeb8163970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb80cb1d0 0x7efeb814ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb80b0210 con 0x7efeb80b2d50 2023-12-19T17:57:26.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb81505b0 con 0x7efeb806bee0 2023-12-19T17:57:26.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.867+0000 7efebf98f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb80cd1b0 con 0x7efeb80cb1d0 2023-12-19T17:57:26.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebcf2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb80b2d50 0x7efeb8163970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebcf2a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb80b2d50 0x7efeb8163970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43692/0 (socket says 172.21.15.26:43692) 2023-12-19T17:57:26.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebcf2a640 1 -- 172.21.15.26:0/1913411128 learned_addr learned my addr 172.21.15.26:0/1913411128 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebd72b640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb806bee0 0x7efeb80cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebdf2c640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb80cb1d0 0x7efeb814ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebcf2a640 1 -- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb80cb1d0 msgr2=0x7efeb814ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebcf2a640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb80cb1d0 0x7efeb814ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebcf2a640 1 -- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb806bee0 msgr2=0x7efeb80cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebcf2a640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb806bee0 0x7efeb80cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebcf2a640 1 -- 172.21.15.26:0/1913411128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb0009a70 con 0x7efeb80b2d50 2023-12-19T17:57:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebd72b640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb806bee0 0x7efeb80cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebdf2c640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb80cb1d0 0x7efeb814ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:26.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebcf2a640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb80b2d50 0x7efeb8163970 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7efea800c4d0 tx=0x7efea800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efeadffb640 1 -- 172.21.15.26:0/1913411128 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8005cf0 con 0x7efeb80b2d50 2023-12-19T17:57:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efeadffb640 1 -- 172.21.15.26:0/1913411128 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea8005e50 con 0x7efeb80b2d50 2023-12-19T17:57:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efeadffb640 1 -- 172.21.15.26:0/1913411128 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8007270 con 0x7efeb80b2d50 2023-12-19T17:57:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebf98f640 1 -- 172.21.15.26:0/1913411128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb80cd490 con 0x7efeb80b2d50 2023-12-19T17:57:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebf98f640 1 -- 172.21.15.26:0/1913411128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb80cda50 con 0x7efeb80b2d50 2023-12-19T17:57:26.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efebf98f640 1 -- 172.21.15.26:0/1913411128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb8074270 con 0x7efeb80b2d50 2023-12-19T17:57:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efeadffb640 1 -- 172.21.15.26:0/1913411128 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 78) v1 ==== 85066+0+0 (secure 0 0 0) 0x7efea80073d0 con 0x7efeb80b2d50 2023-12-19T17:57:26.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.871+0000 7efeadffb640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe9c0658b0 0x7efe9c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:26.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.875+0000 7efeadffb640 1 -- 172.21.15.26:0/1913411128 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 7145+0+0 (secure 0 0 0) 0x7efea8082810 con 0x7efeb80b2d50 2023-12-19T17:57:26.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.875+0000 7efebd72b640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe9c0658b0 0x7efe9c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:26.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.875+0000 7efebd72b640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe9c0658b0 0x7efe9c067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efeb8160f60 tx=0x7efeb4009040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:26.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.875+0000 7efeadffb640 1 -- 172.21.15.26:0/1913411128 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efea8050020 con 0x7efeb80b2d50 2023-12-19T17:57:26.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:26.975+0000 7efeadffb640 1 -- 172.21.15.26:0/1913411128 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 79) v1 ==== 85274+0+0 (secure 0 0 0) 0x7efea80503f0 con 0x7efeb80b2d50 2023-12-19T17:57:27.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.039+0000 7efebf98f640 1 -- 172.21.15.26:0/1913411128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efeb80cdbe0 con 0x7efeb80b2d50 2023-12-19T17:57:27.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.043+0000 7efeadffb640 1 -- 172.21.15.26:0/1913411128 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+18285 (secure 0 0 0) 0x7efea806cdb0 con 0x7efeb80b2d50 2023-12-19T17:57:27.045 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:27.045 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":150,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:57:24.900124+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:01:19.682876+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T17:01:20.140612+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":149,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.f","mappings":[{"from":0,"to":2}]},{"pgid":"17.3f","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/635672045":"2023-12-20T17:57:24.900091+0000","172.21.15.160:0/988541364":"2023-12-20T17:57:24.900091+0000","172.21.15.160:0/3538348635":"2023-12-20T17:57:24.900091+0000","172.21.15.160:0/4117035290":"2023-12-20T17:57:24.900091+0000","172.21.15.160:6833/38098":"2023-12-20T17:01:32.971594+0000","172.21.15.160:6832/38098":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/1746260591":"2023-12-20T17:57:24.900091+0000","172.21.15.160:0/1067813784":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/3376805359":"2023-12-20T17:57:24.900091+0000","172.21.15.160:0/1947760249":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/464310353":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/2140731666":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/870873437":"2023-12-20T17:01:32.971594+0000","172.21.15.160:0/950669571":"2023-12-20T17:01:32.971594+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":17,"snaps":[{"begin":5,"length":53},{"begin":61,"length":53}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:57:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.043+0000 7efebf98f640 1 -- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe9c0658b0 msgr2=0x7efe9c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.043+0000 7efebf98f640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe9c0658b0 0x7efe9c067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efeb8160f60 tx=0x7efeb4009040 comp rx=0 tx=0).stop 2023-12-19T17:57:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.043+0000 7efebf98f640 1 -- 172.21.15.26:0/1913411128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb80b2d50 msgr2=0x7efeb8163970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.043+0000 7efebf98f640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb80b2d50 0x7efeb8163970 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7efea800c4d0 tx=0x7efea800c990 comp rx=0 tx=0).stop 2023-12-19T17:57:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.043+0000 7efebf98f640 1 -- 172.21.15.26:0/1913411128 shutdown_connections 2023-12-19T17:57:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.043+0000 7efebf98f640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efe9c0658b0 0x7efe9c067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.043+0000 7efebf98f640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb80cb1d0 0x7efeb814ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.043+0000 7efebf98f640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb80b2d50 0x7efeb8163970 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.043+0000 7efebf98f640 1 --2- 172.21.15.26:0/1913411128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb806bee0 0x7efeb80cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.043+0000 7efebf98f640 1 -- 172.21.15.26:0/1913411128 >> 172.21.15.26:0/1913411128 conn(0x7efeb80658e0 msgr2=0x7efeb80cb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.043+0000 7efebf98f640 1 -- 172.21.15.26:0/1913411128 shutdown_connections 2023-12-19T17:57:27.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.043+0000 7efebf98f640 1 -- 172.21.15.26:0/1913411128 wait complete. 2023-12-19T17:57:27.066 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T17:57:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.207+0000 7f55ac066640 1 Processor -- start 2023-12-19T17:57:27.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.207+0000 7f55ac066640 1 -- start start 2023-12-19T17:57:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.207+0000 7f55ac066640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a40cb1d0 0x7f55a40cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.207+0000 7f55ac066640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f55a406bee0 0x7f55a406c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.207+0000 7f55ac066640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55a406d4b0 con 0x7f55a4164c90 2023-12-19T17:57:27.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55ac066640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a40cc030 con 0x7f55a406bee0 2023-12-19T17:57:27.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55ac066640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a40cbaf0 con 0x7f55a40cb1d0 2023-12-19T17:57:27.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55a9e02640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a40cb1d0 0x7f55a40cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:27.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55a9601640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f55a406bee0 0x7f55a406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:27.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55aa603640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f55a4164c90 0x7f55a4165120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:42546/0 (socket says 172.21.15.26:42546) 2023-12-19T17:57:27.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55aa603640 1 -- 172.21.15.26:0/1541410312 learned_addr learned my addr 172.21.15.26:0/1541410312 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:27.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55a8e00640 1 -- 172.21.15.26:0/1541410312 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1037420295 0 0) 0x7f55a406d4b0 con 0x7f55a4164c90 2023-12-19T17:57:27.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55a8e00640 1 -- 172.21.15.26:0/1541410312 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5590003530 con 0x7f55a4164c90 2023-12-19T17:57:27.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55a9601640 1 -- 172.21.15.26:0/1541410312 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a40cb1d0 msgr2=0x7f55a40cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:27.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55a9601640 1 --2- 172.21.15.26:0/1541410312 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a40cb1d0 0x7f55a40cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55a9601640 1 -- 172.21.15.26:0/1541410312 >> v1:172.21.15.160:6789/0 conn(0x7f55a4164c90 legacy=0x7f55a4165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:27.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55a9601640 1 -- 172.21.15.26:0/1541410312 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a406cc10 con 0x7f55a406bee0 2023-12-19T17:57:27.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55a9e02640 1 --2- 172.21.15.26:0/1541410312 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a40cb1d0 0x7f55a40cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.211+0000 7f55a9601640 1 --2- 172.21.15.26:0/1541410312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a406bee0 0x7f55a406c340 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f55940097c0 tx=0x7f559401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8811d65a761200e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55a8e00640 1 -- 172.21.15.26:0/1541410312 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55940ac620 con 0x7f55a406bee0 2023-12-19T17:57:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55a8e00640 1 -- 172.21.15.26:0/1541410312 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55940ac780 con 0x7f55a406bee0 2023-12-19T17:57:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55a8e00640 1 -- 172.21.15.26:0/1541410312 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55940acaf0 con 0x7f55a406bee0 2023-12-19T17:57:27.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 -- 172.21.15.26:0/1541410312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a406bee0 msgr2=0x7f55a406c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 --2- 172.21.15.26:0/1541410312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a406bee0 0x7f55a406c340 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f55940097c0 tx=0x7f559401c3c0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 -- 172.21.15.26:0/1541410312 shutdown_connections 2023-12-19T17:57:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 --2- 172.21.15.26:0/1541410312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a406bee0 0x7f55a406c340 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 --2- 172.21.15.26:0/1541410312 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a40cb1d0 0x7f55a40cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 -- 172.21.15.26:0/1541410312 >> 172.21.15.26:0/1541410312 conn(0x7f55a40658e0 msgr2=0x7f55a4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 -- 172.21.15.26:0/1541410312 shutdown_connections 2023-12-19T17:57:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 -- 172.21.15.26:0/1541410312 wait complete. 2023-12-19T17:57:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 Processor -- start 2023-12-19T17:57:27.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 -- start start 2023-12-19T17:57:27.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a406bee0 0x7f55a4158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a40cb1d0 0x7f55a41541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a4164c90 0x7f55a40af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a41f9d90 con 0x7f55a406bee0 2023-12-19T17:57:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a4154aa0 con 0x7f55a40cb1d0 2023-12-19T17:57:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55ac066640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a4154d80 con 0x7f55a4164c90 2023-12-19T17:57:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.215+0000 7f55a9e02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a406bee0 0x7f55a4158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f55a9e02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a406bee0 0x7f55a4158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43720/0 (socket says 172.21.15.26:43720) 2023-12-19T17:57:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f55a9e02640 1 -- 172.21.15.26:0/3458817714 learned_addr learned my addr 172.21.15.26:0/3458817714 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f55aa603640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a4164c90 0x7f55a40af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f55a9601640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a40cb1d0 0x7f55a41541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f55a9e02640 1 -- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a4164c90 msgr2=0x7f55a40af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f55a9e02640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a4164c90 0x7f55a40af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f55a9e02640 1 -- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a40cb1d0 msgr2=0x7f55a41541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f55a9e02640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a40cb1d0 0x7f55a41541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f55a9e02640 1 -- 172.21.15.26:0/3458817714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a0009420 con 0x7f55a406bee0 2023-12-19T17:57:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f55a9e02640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a406bee0 0x7f55a4158210 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f55a000e980 tx=0x7f55a000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f559a7fc640 1 -- 172.21.15.26:0/3458817714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a000ca00 con 0x7f55a406bee0 2023-12-19T17:57:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f559a7fc640 1 -- 172.21.15.26:0/3458817714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a000cb60 con 0x7f55a406bee0 2023-12-19T17:57:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f55ac066640 1 -- 172.21.15.26:0/3458817714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5594009420 con 0x7f55a406bee0 2023-12-19T17:57:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f559a7fc640 1 -- 172.21.15.26:0/3458817714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a0013470 con 0x7f55a406bee0 2023-12-19T17:57:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f55ac066640 1 -- 172.21.15.26:0/3458817714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a41fa050 con 0x7f55a406bee0 2023-12-19T17:57:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f55ac066640 1 -- 172.21.15.26:0/3458817714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a4074270 con 0x7f55a406bee0 2023-12-19T17:57:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f559a7fc640 1 -- 172.21.15.26:0/3458817714 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 79) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f55a00135d0 con 0x7f55a406bee0 2023-12-19T17:57:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f559a7fc640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5588065af0 0x7f5588067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.219+0000 7f559a7fc640 1 -- 172.21.15.26:0/3458817714 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 7145+0+0 (secure 0 0 0) 0x7f55a0083a70 con 0x7f55a406bee0 2023-12-19T17:57:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.223+0000 7f55a9601640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5588065af0 0x7f5588067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:27.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.223+0000 7f55a9601640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5588065af0 0x7f5588067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f55940ae000 tx=0x7f55940a7000 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:27.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.223+0000 7f559a7fc640 1 -- 172.21.15.26:0/3458817714 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f55a0050e30 con 0x7f55a406bee0 2023-12-19T17:57:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.415+0000 7f55ac066640 1 -- 172.21.15.26:0/3458817714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f55a41655c0 con 0x7f55a406bee0 2023-12-19T17:57:27.417 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T17:57:27.415+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T17:57:27.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.427+0000 7f559a7fc640 1 -- 172.21.15.26:0/3458817714 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7f55a0021090 con 0x7f55a406bee0 2023-12-19T17:57:27.428 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T17:57:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.427+0000 7f55ac066640 1 -- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5588065af0 msgr2=0x7f5588067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.427+0000 7f55ac066640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5588065af0 0x7f5588067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f55940ae000 tx=0x7f55940a7000 comp rx=0 tx=0).stop 2023-12-19T17:57:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.427+0000 7f55ac066640 1 -- 172.21.15.26:0/3458817714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a406bee0 msgr2=0x7f55a4158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.427+0000 7f55ac066640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a406bee0 0x7f55a4158210 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f55a000e980 tx=0x7f55a000ee40 comp rx=0 tx=0).stop 2023-12-19T17:57:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.427+0000 7f55ac066640 1 -- 172.21.15.26:0/3458817714 shutdown_connections 2023-12-19T17:57:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.427+0000 7f55ac066640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5588065af0 0x7f5588067fb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.427+0000 7f55ac066640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f55a4164c90 0x7f55a40af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.427+0000 7f55ac066640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55a40cb1d0 0x7f55a41541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.427+0000 7f55ac066640 1 --2- 172.21.15.26:0/3458817714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55a406bee0 0x7f55a4158210 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.427+0000 7f55ac066640 1 -- 172.21.15.26:0/3458817714 >> 172.21.15.26:0/3458817714 conn(0x7f55a40658e0 msgr2=0x7f55a4065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.427+0000 7f55ac066640 1 -- 172.21.15.26:0/3458817714 shutdown_connections 2023-12-19T17:57:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.427+0000 7f55ac066640 1 -- 172.21.15.26:0/3458817714 wait complete. 2023-12-19T17:57:27.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T17:57:27.493 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T17:57:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f5676b2c640 1 Processor -- start 2023-12-19T17:57:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f5676b2c640 1 -- start start 2023-12-19T17:57:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f5676b2c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f56700cb860 0x7f567006f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f5676b2c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567015e240 con 0x7f567015d530 2023-12-19T17:57:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f5676b2c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56700c9520 con 0x7f56701599e0 2023-12-19T17:57:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f5676b2c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56700c9800 con 0x7f56700cb860 2023-12-19T17:57:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f56750c9640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f567015d530 0x7f567015d980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34420/0 (socket says 172.21.15.26:34420) 2023-12-19T17:57:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f56750c9640 1 -- 172.21.15.26:0/3782445854 learned_addr learned my addr 172.21.15.26:0/3782445854 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f566ffff640 1 --2- 172.21.15.26:0/3782445854 >> v2:172.21.15.160:3300/0 conn(0x7f56700cb860 0x7f567006f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f566f7fe640 1 -- 172.21.15.26:0/3782445854 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813304370 0 0) 0x7f567015e240 con 0x7f567015d530 2023-12-19T17:57:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f566f7fe640 1 -- 172.21.15.26:0/3782445854 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5660003530 con 0x7f567015d530 2023-12-19T17:57:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f566f7fe640 1 -- 172.21.15.26:0/3782445854 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342799858 0 0) 0x7f56700c9520 con 0x7f56701599e0 2023-12-19T17:57:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f566f7fe640 1 -- 172.21.15.26:0/3782445854 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f567015e240 con 0x7f56701599e0 2023-12-19T17:57:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f566ffff640 1 -- 172.21.15.26:0/3782445854 >> v1:172.21.15.160:6789/0 conn(0x7f56701599e0 legacy=0x7f56700cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f566f7fe640 1 -- 172.21.15.26:0/3782445854 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 635376403 0 0) 0x7f5660003530 con 0x7f567015d530 2023-12-19T17:57:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f566ffff640 1 -- 172.21.15.26:0/3782445854 >> v1:172.21.15.26:6789/0 conn(0x7f567015d530 legacy=0x7f567015d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.599+0000 7f566ffff640 1 -- 172.21.15.26:0/3782445854 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56700c9ae0 con 0x7f56700cb860 2023-12-19T17:57:27.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.603+0000 7f566ffff640 1 --2- 172.21.15.26:0/3782445854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56700cb860 0x7f567006f6d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f565c0097c0 tx=0x7f565c01c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c332498da830041d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:27.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.603+0000 7f566f7fe640 1 -- 172.21.15.26:0/3782445854 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668001a00 con 0x7f56700cb860 2023-12-19T17:57:27.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.603+0000 7f566f7fe640 1 -- 172.21.15.26:0/3782445854 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c0a7050 con 0x7f56700cb860 2023-12-19T17:57:27.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.603+0000 7f566f7fe640 1 -- 172.21.15.26:0/3782445854 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c0ab620 con 0x7f56700cb860 2023-12-19T17:57:27.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.603+0000 7f5676b2c640 1 -- 172.21.15.26:0/3782445854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56700cb860 msgr2=0x7f567006f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:27.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.603+0000 7f5676b2c640 1 --2- 172.21.15.26:0/3782445854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56700cb860 0x7f567006f6d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f565c0097c0 tx=0x7f565c01c3a0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.603+0000 7f5676b2c640 1 -- 172.21.15.26:0/3782445854 shutdown_connections 2023-12-19T17:57:27.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.603+0000 7f5676b2c640 1 --2- 172.21.15.26:0/3782445854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56700cb860 0x7f567006f6d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.603+0000 7f5676b2c640 1 -- 172.21.15.26:0/3782445854 >> 172.21.15.26:0/3782445854 conn(0x7f5670066160 msgr2=0x7f56700b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.603+0000 7f5676b2c640 1 -- 172.21.15.26:0/3782445854 shutdown_connections 2023-12-19T17:57:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.603+0000 7f5676b2c640 1 -- 172.21.15.26:0/3782445854 wait complete. 2023-12-19T17:57:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f5676b2c640 1 Processor -- start 2023-12-19T17:57:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f5676b2c640 1 -- start start 2023-12-19T17:57:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f5676b2c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56700cb860 0x7f56700c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f5676b2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56701599e0 0x7f567015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f5676b2c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f567015d530 0x7f56700c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f5676b2c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567015b250 con 0x7f56701599e0 2023-12-19T17:57:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f5676b2c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567015b530 con 0x7f567015d530 2023-12-19T17:57:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f5676b2c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567015b810 con 0x7f56700cb860 2023-12-19T17:57:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f566ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56701599e0 0x7f567015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f566ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56701599e0 0x7f567015c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43744/0 (socket says 172.21.15.26:43744) 2023-12-19T17:57:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f566ffff640 1 -- 172.21.15.26:0/655378931 learned_addr learned my addr 172.21.15.26:0/655378931 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f56748c8640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56700cb860 0x7f56700c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f56750c9640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f567015d530 0x7f56700c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:27.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f566ffff640 1 -- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56700cb860 msgr2=0x7f56700c28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:27.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f566ffff640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56700cb860 0x7f56700c28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f566ffff640 1 -- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f567015d530 msgr2=0x7f56700c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:27.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f566ffff640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f567015d530 0x7f56700c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:27.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f566ffff640 1 -- 172.21.15.26:0/655378931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c009420 con 0x7f56701599e0 2023-12-19T17:57:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f56748c8640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56700cb860 0x7f56700c28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f566ffff640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56701599e0 0x7f567015c5d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f565c019fd0 tx=0x7f565c019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f566d7fa640 1 -- 172.21.15.26:0/655378931 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c0b4810 con 0x7f56701599e0 2023-12-19T17:57:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f566d7fa640 1 -- 172.21.15.26:0/655378931 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c0b4990 con 0x7f56701599e0 2023-12-19T17:57:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f566d7fa640 1 -- 172.21.15.26:0/655378931 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c0b4d40 con 0x7f56701599e0 2023-12-19T17:57:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f5676b2c640 1 -- 172.21.15.26:0/655378931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567015baf0 con 0x7f56701599e0 2023-12-19T17:57:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.607+0000 7f5676b2c640 1 -- 172.21.15.26:0/655378931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567015c0b0 con 0x7f56701599e0 2023-12-19T17:57:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.611+0000 7f5676b2c640 1 -- 172.21.15.26:0/655378931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567015c2a0 con 0x7f56701599e0 2023-12-19T17:57:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.611+0000 7f566d7fa640 1 -- 172.21.15.26:0/655378931 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 79) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f565c0b2040 con 0x7f56701599e0 2023-12-19T17:57:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.611+0000 7f566d7fa640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5654065af0 0x7f5654067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.611+0000 7f566d7fa640 1 -- 172.21.15.26:0/655378931 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 7231+0+0 (secure 0 0 0) 0x7f565c11dc40 con 0x7f56701599e0 2023-12-19T17:57:27.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.611+0000 7f56748c8640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5654065af0 0x7f5654067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:27.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.611+0000 7f56748c8640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5654065af0 0x7f5654067fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f56640011f0 tx=0x7f5664009040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:27.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.615+0000 7f566d7fa640 1 -- 172.21.15.26:0/655378931 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f565c0eaf50 con 0x7f56701599e0 2023-12-19T17:57:27.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:27.807+0000 7f5676b2c640 1 -- 172.21.15.26:0/655378931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5670074270 con 0x7f56701599e0 2023-12-19T17:57:28.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.427+0000 7f566d7fa640 1 -- 172.21.15.26:0/655378931 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f565c0c20a0 con 0x7f56701599e0 2023-12-19T17:57:28.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.431+0000 7f5676b2c640 1 -- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5654065af0 msgr2=0x7f5654067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:28.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.431+0000 7f5676b2c640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5654065af0 0x7f5654067fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f56640011f0 tx=0x7f5664009040 comp rx=0 tx=0).stop 2023-12-19T17:57:28.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.431+0000 7f5676b2c640 1 -- 172.21.15.26:0/655378931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56701599e0 msgr2=0x7f567015c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:28.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.431+0000 7f5676b2c640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56701599e0 0x7f567015c5d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f565c019fd0 tx=0x7f565c019140 comp rx=0 tx=0).stop 2023-12-19T17:57:28.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.431+0000 7f5676b2c640 1 -- 172.21.15.26:0/655378931 shutdown_connections 2023-12-19T17:57:28.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.431+0000 7f5676b2c640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f567015d530 0x7f56700c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:28.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.431+0000 7f5676b2c640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56701599e0 0x7f567015c5d0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:28.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.431+0000 7f5676b2c640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5654065af0 0x7f5654067fb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:28.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.431+0000 7f5676b2c640 1 --2- 172.21.15.26:0/655378931 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56700cb860 0x7f56700c28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:28.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.431+0000 7f5676b2c640 1 -- 172.21.15.26:0/655378931 >> 172.21.15.26:0/655378931 conn(0x7f5670066160 msgr2=0x7f5670153990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:28.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.431+0000 7f5676b2c640 1 -- 172.21.15.26:0/655378931 shutdown_connections 2023-12-19T17:57:28.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.431+0000 7f5676b2c640 1 -- 172.21.15.26:0/655378931 wait complete. 2023-12-19T17:57:28.453 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T17:57:28.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a3462d640 1 Processor -- start 2023-12-19T17:57:28.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a3462d640 1 -- start start 2023-12-19T17:57:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a3462d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4a2c15aad0 0x7f4a2c06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a3462d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4a2c0b1590 0x7f4a2c070870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a3462d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2c06cbf0 con 0x7f4a2c0b3f90 2023-12-19T17:57:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a3462d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a2c0ca8b0 con 0x7f4a2c15aad0 2023-12-19T17:57:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a3462d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a2c06b6f0 con 0x7f4a2c0b1590 2023-12-19T17:57:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a32bca640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4a2c0b3f90 0x7f4a2c0c8d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34422/0 (socket says 172.21.15.26:34422) 2023-12-19T17:57:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a32bca640 1 -- 172.21.15.26:0/3746177288 learned_addr learned my addr 172.21.15.26:0/3746177288 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a323c9640 1 --2- 172.21.15.26:0/3746177288 >> v2:172.21.15.26:3300/0 conn(0x7f4a2c15aad0 0x7f4a2c06c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a31bc8640 1 --2- 172.21.15.26:0/3746177288 >> v2:172.21.15.160:3300/0 conn(0x7f4a2c0b1590 0x7f4a2c070870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a313c7640 1 -- 172.21.15.26:0/3746177288 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599718736 0 0) 0x7f4a240014a0 con 0x7f4a2c0b3f90 2023-12-19T17:57:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a313c7640 1 -- 172.21.15.26:0/3746177288 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a18003530 con 0x7f4a2c0b3f90 2023-12-19T17:57:28.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a323c9640 1 -- 172.21.15.26:0/3746177288 >> v2:172.21.15.160:3300/0 conn(0x7f4a2c0b1590 msgr2=0x7f4a2c070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:28.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a323c9640 1 --2- 172.21.15.26:0/3746177288 >> v2:172.21.15.160:3300/0 conn(0x7f4a2c0b1590 0x7f4a2c070870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:28.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a323c9640 1 -- 172.21.15.26:0/3746177288 >> v1:172.21.15.26:6789/0 conn(0x7f4a2c0b3f90 legacy=0x7f4a2c0c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:28.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.643+0000 7f4a323c9640 1 -- 172.21.15.26:0/3746177288 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2c06b9d0 con 0x7f4a2c15aad0 2023-12-19T17:57:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.647+0000 7f4a323c9640 1 --2- 172.21.15.26:0/3746177288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c15aad0 0x7f4a2c06c6b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0ce5b0 tx=0x7f4a2801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a139cc912e79018 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.647+0000 7f4a313c7640 1 -- 172.21.15.26:0/3746177288 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a280a7070 con 0x7f4a2c15aad0 2023-12-19T17:57:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.647+0000 7f4a313c7640 1 -- 172.21.15.26:0/3746177288 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a280ab430 con 0x7f4a2c15aad0 2023-12-19T17:57:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.647+0000 7f4a313c7640 1 -- 172.21.15.26:0/3746177288 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a280ab7c0 con 0x7f4a2c15aad0 2023-12-19T17:57:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.647+0000 7f4a3462d640 1 -- 172.21.15.26:0/3746177288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c15aad0 msgr2=0x7f4a2c06c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:28.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.647+0000 7f4a3462d640 1 --2- 172.21.15.26:0/3746177288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c15aad0 0x7f4a2c06c6b0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0ce5b0 tx=0x7f4a2801c430 comp rx=0 tx=0).stop 2023-12-19T17:57:28.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.647+0000 7f4a3462d640 1 -- 172.21.15.26:0/3746177288 shutdown_connections 2023-12-19T17:57:28.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.647+0000 7f4a3462d640 1 --2- 172.21.15.26:0/3746177288 >> v2:172.21.15.160:3300/0 conn(0x7f4a2c0b1590 0x7f4a2c070870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:28.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.647+0000 7f4a3462d640 1 --2- 172.21.15.26:0/3746177288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c15aad0 0x7f4a2c06c6b0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:28.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.647+0000 7f4a3462d640 1 -- 172.21.15.26:0/3746177288 >> 172.21.15.26:0/3746177288 conn(0x7f4a2c0b0270 msgr2=0x7f4a2c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:28.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.647+0000 7f4a3462d640 1 -- 172.21.15.26:0/3746177288 shutdown_connections 2023-12-19T17:57:28.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.647+0000 7f4a3462d640 1 -- 172.21.15.26:0/3746177288 wait complete. 2023-12-19T17:57:28.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a3462d640 1 Processor -- start 2023-12-19T17:57:28.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a3462d640 1 -- start start 2023-12-19T17:57:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a3462d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a2c0b1590 0x7f4a2c1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a3462d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a2c0b3f90 0x7f4a2c0cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a3462d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c15aad0 0x7f4a2c1541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a3462d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c156c00 con 0x7f4a2c15aad0 2023-12-19T17:57:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a3462d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c0cd6f0 con 0x7f4a2c0b3f90 2023-12-19T17:57:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a3462d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2c0ccae0 con 0x7f4a2c0b1590 2023-12-19T17:57:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a32bca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c15aad0 0x7f4a2c1541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a32bca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c15aad0 0x7f4a2c1541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43774/0 (socket says 172.21.15.26:43774) 2023-12-19T17:57:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a32bca640 1 -- 172.21.15.26:0/2931048797 learned_addr learned my addr 172.21.15.26:0/2931048797 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a323c9640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a2c0b1590 0x7f4a2c1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a31bc8640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a2c0b3f90 0x7f4a2c0cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a32bca640 1 -- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a2c0b1590 msgr2=0x7f4a2c1566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a32bca640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a2c0b1590 0x7f4a2c1566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a32bca640 1 -- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a2c0b3f90 msgr2=0x7f4a2c0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a32bca640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a2c0b3f90 0x7f4a2c0cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a32bca640 1 -- 172.21.15.26:0/2931048797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a28009420 con 0x7f4a2c15aad0 2023-12-19T17:57:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a32bca640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c15aad0 0x7f4a2c1541a0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400cb40 tx=0x7f4a2400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a22ffd640 1 -- 172.21.15.26:0/2931048797 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2400aca0 con 0x7f4a2c15aad0 2023-12-19T17:57:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a22ffd640 1 -- 172.21.15.26:0/2931048797 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2400ae00 con 0x7f4a2c15aad0 2023-12-19T17:57:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a22ffd640 1 -- 172.21.15.26:0/2931048797 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a24012620 con 0x7f4a2c15aad0 2023-12-19T17:57:28.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a3462d640 1 -- 172.21.15.26:0/2931048797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2c0ccdc0 con 0x7f4a2c15aad0 2023-12-19T17:57:28.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a3462d640 1 -- 172.21.15.26:0/2931048797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a2c0cd380 con 0x7f4a2c15aad0 2023-12-19T17:57:28.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.651+0000 7f4a3462d640 1 -- 172.21.15.26:0/2931048797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2c0cd880 con 0x7f4a2c15aad0 2023-12-19T17:57:28.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.655+0000 7f4a22ffd640 1 -- 172.21.15.26:0/2931048797 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 79) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f4a24007070 con 0x7f4a2c15aad0 2023-12-19T17:57:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.655+0000 7f4a22ffd640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4a10065af0 0x7f4a10067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.655+0000 7f4a22ffd640 1 -- 172.21.15.26:0/2931048797 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 7231+0+0 (secure 0 0 0) 0x7f4a24083670 con 0x7f4a2c15aad0 2023-12-19T17:57:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.655+0000 7f4a323c9640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4a10065af0 0x7f4a10067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:28.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.655+0000 7f4a323c9640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4a10065af0 0x7f4a10067fb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4a280af000 tx=0x7f4a280a7000 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.659+0000 7f4a22ffd640 1 -- 172.21.15.26:0/2931048797 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4a24050a00 con 0x7f4a2c15aad0 2023-12-19T17:57:28.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:28.827+0000 7f4a3462d640 1 -- 172.21.15.26:0/2931048797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4a2c06cbf0 con 0x7f4a2c15aad0 2023-12-19T17:57:29.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.707+0000 7f4a22ffd640 1 -- 172.21.15.26:0/2931048797 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4a24050dd0 con 0x7f4a2c15aad0 2023-12-19T17:57:29.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.707+0000 7f4a22ffd640 1 -- 172.21.15.26:0/2931048797 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v153) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4a2404cd80 con 0x7f4a2c15aad0 2023-12-19T17:57:29.708 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T17:57:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.707+0000 7f4a3462d640 1 -- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4a10065af0 msgr2=0x7f4a10067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.707+0000 7f4a3462d640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4a10065af0 0x7f4a10067fb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4a280af000 tx=0x7f4a280a7000 comp rx=0 tx=0).stop 2023-12-19T17:57:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.707+0000 7f4a3462d640 1 -- 172.21.15.26:0/2931048797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c15aad0 msgr2=0x7f4a2c1541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:29.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.707+0000 7f4a3462d640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c15aad0 0x7f4a2c1541a0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400cb40 tx=0x7f4a2400a060 comp rx=0 tx=0).stop 2023-12-19T17:57:29.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.707+0000 7f4a3462d640 1 -- 172.21.15.26:0/2931048797 shutdown_connections 2023-12-19T17:57:29.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.707+0000 7f4a3462d640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a2c15aad0 0x7f4a2c1541a0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:29.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.707+0000 7f4a3462d640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a2c0b3f90 0x7f4a2c0cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:29.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.707+0000 7f4a3462d640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4a10065af0 0x7f4a10067fb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:29.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.707+0000 7f4a3462d640 1 --2- 172.21.15.26:0/2931048797 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4a2c0b1590 0x7f4a2c1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:29.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.707+0000 7f4a3462d640 1 -- 172.21.15.26:0/2931048797 >> 172.21.15.26:0/2931048797 conn(0x7f4a2c0b0270 msgr2=0x7f4a2c0ae1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:29.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.711+0000 7f4a3462d640 1 -- 172.21.15.26:0/2931048797 shutdown_connections 2023-12-19T17:57:29.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.711+0000 7f4a3462d640 1 -- 172.21.15.26:0/2931048797 wait complete. 2023-12-19T17:57:29.733 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T17:57:29.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0559339640 1 Processor -- start 2023-12-19T17:57:29.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0559339640 1 -- start start 2023-12-19T17:57:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0559339640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05541603a0 0x7f05540d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0559339640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f05540b8640 0x7f05540cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0559339640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05540d29b0 con 0x7f05540d0130 2023-12-19T17:57:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0559339640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05540d0550 con 0x7f05541603a0 2023-12-19T17:57:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0559339640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05540d1910 con 0x7f05540b8640 2023-12-19T17:57:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0552ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05541603a0 0x7f05540d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0552ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f05541603a0 0x7f05540d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43782/0 (socket says 172.21.15.26:43782) 2023-12-19T17:57:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0552ffd640 1 -- 172.21.15.26:0/1993183866 learned_addr learned my addr 172.21.15.26:0/1993183866 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f054bfff640 1 --2- 172.21.15.26:0/1993183866 >> v2:172.21.15.160:3300/0 conn(0x7f05540b8640 0x7f05540cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:29.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0552ffd640 1 -- 172.21.15.26:0/1993183866 >> v2:172.21.15.160:3300/0 conn(0x7f05540b8640 msgr2=0x7f05540cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:29.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0552ffd640 1 --2- 172.21.15.26:0/1993183866 >> v2:172.21.15.160:3300/0 conn(0x7f05540b8640 0x7f05540cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:29.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0552ffd640 1 -- 172.21.15.26:0/1993183866 >> v1:172.21.15.160:6789/0 conn(0x7f05540d0130 legacy=0x7f05540d2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:29.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.879+0000 7f0552ffd640 1 -- 172.21.15.26:0/1993183866 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05540d1bf0 con 0x7f05541603a0 2023-12-19T17:57:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.883+0000 7f0552ffd640 1 --2- 172.21.15.26:0/1993183866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05541603a0 0x7f05540d1010 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f05540d3c70 tx=0x7f054401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd396a2813ef3f17 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.883+0000 7f05527fc640 1 -- 172.21.15.26:0/1993183866 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05440aa5f0 con 0x7f05541603a0 2023-12-19T17:57:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.883+0000 7f05527fc640 1 -- 172.21.15.26:0/1993183866 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05440aa750 con 0x7f05541603a0 2023-12-19T17:57:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.883+0000 7f05527fc640 1 -- 172.21.15.26:0/1993183866 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05440aaae0 con 0x7f05541603a0 2023-12-19T17:57:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.883+0000 7f0559339640 1 -- 172.21.15.26:0/1993183866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05541603a0 msgr2=0x7f05540d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.883+0000 7f0559339640 1 --2- 172.21.15.26:0/1993183866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05541603a0 0x7f05540d1010 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f05540d3c70 tx=0x7f054401c490 comp rx=0 tx=0).stop 2023-12-19T17:57:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.883+0000 7f0559339640 1 -- 172.21.15.26:0/1993183866 shutdown_connections 2023-12-19T17:57:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.883+0000 7f0559339640 1 --2- 172.21.15.26:0/1993183866 >> v2:172.21.15.160:3300/0 conn(0x7f05540b8640 0x7f05540cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.883+0000 7f0559339640 1 --2- 172.21.15.26:0/1993183866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05541603a0 0x7f05540d1010 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.883+0000 7f0559339640 1 -- 172.21.15.26:0/1993183866 >> 172.21.15.26:0/1993183866 conn(0x7f055406a120 msgr2=0x7f055406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.883+0000 7f0559339640 1 -- 172.21.15.26:0/1993183866 shutdown_connections 2023-12-19T17:57:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.883+0000 7f0559339640 1 -- 172.21.15.26:0/1993183866 wait complete. 2023-12-19T17:57:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0559339640 1 Processor -- start 2023-12-19T17:57:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0559339640 1 -- start start 2023-12-19T17:57:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0559339640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05540b8640 0x7f05541f8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0559339640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05540d0130 0x7f0554153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0559339640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05541603a0 0x7f055415f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0559339640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05540cad00 con 0x7f05540b8640 2023-12-19T17:57:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0559339640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05541f8b80 con 0x7f05541603a0 2023-12-19T17:57:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0559339640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05541f8e60 con 0x7f05540d0130 2023-12-19T17:57:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0552ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05540b8640 0x7f05541f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0552ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05540b8640 0x7f05541f8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43788/0 (socket says 172.21.15.26:43788) 2023-12-19T17:57:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0552ffd640 1 -- 172.21.15.26:0/1081010991 learned_addr learned my addr 172.21.15.26:0/1081010991 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f05537fe640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05541603a0 0x7f055415f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f054bfff640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05540d0130 0x7f0554153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0552ffd640 1 -- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05540d0130 msgr2=0x7f0554153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0552ffd640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05540d0130 0x7f0554153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0552ffd640 1 -- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05541603a0 msgr2=0x7f055415f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0552ffd640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05541603a0 0x7f055415f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0552ffd640 1 -- 172.21.15.26:0/1081010991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05440094d0 con 0x7f05540b8640 2023-12-19T17:57:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.887+0000 7f0552ffd640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05540b8640 0x7f05541f8330 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f054401a0c0 tx=0x7f054401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.891+0000 7f054b7fe640 1 -- 172.21.15.26:0/1081010991 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05440b3800 con 0x7f05540b8640 2023-12-19T17:57:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.891+0000 7f054b7fe640 1 -- 172.21.15.26:0/1081010991 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05440b3980 con 0x7f05540b8640 2023-12-19T17:57:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.891+0000 7f0559339640 1 -- 172.21.15.26:0/1081010991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05541f9140 con 0x7f05540b8640 2023-12-19T17:57:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.891+0000 7f054b7fe640 1 -- 172.21.15.26:0/1081010991 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05440b3d30 con 0x7f05540b8640 2023-12-19T17:57:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.891+0000 7f0559339640 1 -- 172.21.15.26:0/1081010991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05541f8870 con 0x7f05540b8640 2023-12-19T17:57:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.891+0000 7f0559339640 1 -- 172.21.15.26:0/1081010991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05540cb200 con 0x7f05540b8640 2023-12-19T17:57:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.891+0000 7f054b7fe640 1 -- 172.21.15.26:0/1081010991 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f05440b2380 con 0x7f05540b8640 2023-12-19T17:57:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.891+0000 7f054b7fe640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0534065b80 0x7f0534068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.891+0000 7f054b7fe640 1 -- 172.21.15.26:0/1081010991 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6749+0+0 (secure 0 0 0) 0x7f05440a7070 con 0x7f05540b8640 2023-12-19T17:57:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.891+0000 7f054bfff640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0534065b80 0x7f0534068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.895+0000 7f054bfff640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0534065b80 0x7f0534068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f05540b47d0 tx=0x7f053c008040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:29.895+0000 7f054b7fe640 1 -- 172.21.15.26:0/1081010991 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f05540cb200 con 0x7f05540b8640 2023-12-19T17:57:30.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.071+0000 7f0559339640 1 -- 172.21.15.26:0/1081010991 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f05540d29b0 con 0x7f05540b8640 2023-12-19T17:57:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T17:57:30.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.991+0000 7f054b7fe640 1 -- 172.21.15.26:0/1081010991 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v154) v1 ==== 169+0+0 (secure 0 0 0) 0x7f05540d29b0 con 0x7f05540b8640 2023-12-19T17:57:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.995+0000 7f0559339640 1 -- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0534065b80 msgr2=0x7f0534068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.995+0000 7f0559339640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0534065b80 0x7f0534068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f05540b47d0 tx=0x7f053c008040 comp rx=0 tx=0).stop 2023-12-19T17:57:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.995+0000 7f0559339640 1 -- 172.21.15.26:0/1081010991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05540b8640 msgr2=0x7f05541f8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:30.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.995+0000 7f0559339640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05540b8640 0x7f05541f8330 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f054401a0c0 tx=0x7f054401acd0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.999+0000 7f0559339640 1 -- 172.21.15.26:0/1081010991 shutdown_connections 2023-12-19T17:57:31.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.999+0000 7f0559339640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05541603a0 0x7f055415f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.999+0000 7f0559339640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0534065b80 0x7f0534068040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.999+0000 7f0559339640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f05540d0130 0x7f0554153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.999+0000 7f0559339640 1 --2- 172.21.15.26:0/1081010991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05540b8640 0x7f05541f8330 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.999+0000 7f0559339640 1 -- 172.21.15.26:0/1081010991 >> 172.21.15.26:0/1081010991 conn(0x7f055406a120 msgr2=0x7f05540cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.999+0000 7f0559339640 1 -- 172.21.15.26:0/1081010991 shutdown_connections 2023-12-19T17:57:31.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:30.999+0000 7f0559339640 1 -- 172.21.15.26:0/1081010991 wait complete. 2023-12-19T17:57:31.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-19T17:57:31.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5e2f09640 1 Processor -- start 2023-12-19T17:57:31.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5e2f09640 1 -- start start 2023-12-19T17:57:31.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5e2f09640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5dc0cb150 0x7ff5dc0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5e2f09640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5dc0ce3f0 con 0x7ff5dc1585f0 2023-12-19T17:57:31.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5e2f09640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5dc15b1b0 con 0x7ff5dc15e690 2023-12-19T17:57:31.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5e2f09640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0cbb10 con 0x7ff5dc0cb150 2023-12-19T17:57:31.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5e0ca5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff5dc1585f0 0x7ff5dc0cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34428/0 (socket says 172.21.15.26:34428) 2023-12-19T17:57:31.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5e0ca5640 1 -- 172.21.15.26:0/2393622041 learned_addr learned my addr 172.21.15.26:0/2393622041 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:31.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5dbfff640 1 --2- 172.21.15.26:0/2393622041 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5dc0cb150 0x7ff5dc0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5db7fe640 1 -- 172.21.15.26:0/2393622041 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840660908 0 0) 0x7ff5dc0ce3f0 con 0x7ff5dc1585f0 2023-12-19T17:57:31.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5db7fe640 1 -- 172.21.15.26:0/2393622041 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5cc003530 con 0x7ff5dc1585f0 2023-12-19T17:57:31.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5db7fe640 1 -- 172.21.15.26:0/2393622041 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132329296 0 0) 0x7ff5dc15b1b0 con 0x7ff5dc15e690 2023-12-19T17:57:31.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5dbfff640 1 -- 172.21.15.26:0/2393622041 >> v1:172.21.15.160:6789/0 conn(0x7ff5dc15e690 legacy=0x7ff5dc15eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5dbfff640 1 -- 172.21.15.26:0/2393622041 >> v1:172.21.15.26:6789/0 conn(0x7ff5dc1585f0 legacy=0x7ff5dc0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.175+0000 7ff5dbfff640 1 -- 172.21.15.26:0/2393622041 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc15b490 con 0x7ff5dc0cb150 2023-12-19T17:57:31.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.179+0000 7ff5dbfff640 1 --2- 172.21.15.26:0/2393622041 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5dc0cb150 0x7ff5dc0cb5d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80097f0 tx=0x7ff5c801c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fd688729bd7c38f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:31.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.179+0000 7ff5db7fe640 1 -- 172.21.15.26:0/2393622041 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80ab600 con 0x7ff5dc0cb150 2023-12-19T17:57:31.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.179+0000 7ff5db7fe640 1 -- 172.21.15.26:0/2393622041 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c80a7050 con 0x7ff5dc0cb150 2023-12-19T17:57:31.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.179+0000 7ff5db7fe640 1 -- 172.21.15.26:0/2393622041 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80ab980 con 0x7ff5dc0cb150 2023-12-19T17:57:31.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.179+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2393622041 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5dc0cb150 msgr2=0x7ff5dc0cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.179+0000 7ff5e2f09640 1 --2- 172.21.15.26:0/2393622041 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5dc0cb150 0x7ff5dc0cb5d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80097f0 tx=0x7ff5c801c430 comp rx=0 tx=0).stop 2023-12-19T17:57:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.179+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2393622041 shutdown_connections 2023-12-19T17:57:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.179+0000 7ff5e2f09640 1 --2- 172.21.15.26:0/2393622041 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5dc0cb150 0x7ff5dc0cb5d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.179+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2393622041 >> 172.21.15.26:0/2393622041 conn(0x7ff5dc066160 msgr2=0x7ff5dc0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.179+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2393622041 shutdown_connections 2023-12-19T17:57:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.179+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2393622041 wait complete. 2023-12-19T17:57:31.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e2f09640 1 Processor -- start 2023-12-19T17:57:31.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e2f09640 1 -- start start 2023-12-19T17:57:31.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e2f09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5dc0cb150 0x7ff5dc0af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e2f09640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5dc1585f0 0x7ff5dc155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e2f09640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5dc15e690 0x7ff5dc155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e2f09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0cae50 con 0x7ff5dc0cb150 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e2f09640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0ca920 con 0x7ff5dc1585f0 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e2f09640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0cab70 con 0x7ff5dc15e690 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e0ca5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5dc0cb150 0x7ff5dc0af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e0ca5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5dc0cb150 0x7ff5dc0af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43798/0 (socket says 172.21.15.26:43798) 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e0ca5640 1 -- 172.21.15.26:0/2033419649 learned_addr learned my addr 172.21.15.26:0/2033419649 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5dbfff640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5dc1585f0 0x7ff5dc155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e14a6640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5dc15e690 0x7ff5dc155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e0ca5640 1 -- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5dc15e690 msgr2=0x7ff5dc155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e0ca5640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5dc15e690 0x7ff5dc155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e0ca5640 1 -- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5dc1585f0 msgr2=0x7ff5dc155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e0ca5640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5dc1585f0 0x7ff5dc155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e0ca5640 1 -- 172.21.15.26:0/2033419649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8009420 con 0x7ff5dc0cb150 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e0ca5640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5dc0cb150 0x7ff5dc0af050 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000c420 tx=0x7ff5d000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5d97fa640 1 -- 172.21.15.26:0/2033419649 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d0005cf0 con 0x7ff5dc0cb150 2023-12-19T17:57:31.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5d97fa640 1 -- 172.21.15.26:0/2033419649 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d0005e50 con 0x7ff5dc0cb150 2023-12-19T17:57:31.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5d97fa640 1 -- 172.21.15.26:0/2033419649 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00072d0 con 0x7ff5dc0cb150 2023-12-19T17:57:31.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2033419649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5dc15c3e0 con 0x7ff5dc0cb150 2023-12-19T17:57:31.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.183+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2033419649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5dc0c9210 con 0x7ff5dc0cb150 2023-12-19T17:57:31.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.187+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2033419649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5dc0c8d90 con 0x7ff5dc0cb150 2023-12-19T17:57:31.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.187+0000 7ff5d97fa640 1 -- 172.21.15.26:0/2033419649 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff5d0007430 con 0x7ff5dc0cb150 2023-12-19T17:57:31.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.187+0000 7ff5d97fa640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff5c0065b80 0x7ff5c0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.187+0000 7ff5d97fa640 1 -- 172.21.15.26:0/2033419649 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6277+0+0 (secure 0 0 0) 0x7ff5d0082b50 con 0x7ff5dc0cb150 2023-12-19T17:57:31.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.187+0000 7ff5dbfff640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff5c0065b80 0x7ff5c0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.187+0000 7ff5dbfff640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff5c0065b80 0x7ff5c0068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80ad000 tx=0x7ff5c80a6000 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:31.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.191+0000 7ff5d97fa640 1 -- 172.21.15.26:0/2033419649 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff5dc0c8d90 con 0x7ff5dc0cb150 2023-12-19T17:57:31.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.347+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2033419649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7ff5dc0c8d90 con 0x7ff5dc0cb150 2023-12-19T17:57:31.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.355+0000 7ff5d97fa640 1 -- 172.21.15.26:0/2033419649 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v49) v1 ==== 120+0+0 (secure 0 0 0) 0x7ff5dc0c8d90 con 0x7ff5dc0cb150 2023-12-19T17:57:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.355+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff5c0065b80 msgr2=0x7ff5c0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.355+0000 7ff5e2f09640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff5c0065b80 0x7ff5c0068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80ad000 tx=0x7ff5c80a6000 comp rx=0 tx=0).stop 2023-12-19T17:57:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.355+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2033419649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5dc0cb150 msgr2=0x7ff5dc0af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.355+0000 7ff5e2f09640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5dc0cb150 0x7ff5dc0af050 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ff5d000c420 tx=0x7ff5d000c8e0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.355+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2033419649 shutdown_connections 2023-12-19T17:57:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.355+0000 7ff5e2f09640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff5c0065b80 0x7ff5c0068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.355+0000 7ff5e2f09640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5dc15e690 0x7ff5dc155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.355+0000 7ff5e2f09640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5dc1585f0 0x7ff5dc155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.355+0000 7ff5e2f09640 1 --2- 172.21.15.26:0/2033419649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5dc0cb150 0x7ff5dc0af050 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.355+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2033419649 >> 172.21.15.26:0/2033419649 conn(0x7ff5dc066160 msgr2=0x7ff5dc15cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:31.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.355+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2033419649 shutdown_connections 2023-12-19T17:57:31.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.355+0000 7ff5e2f09640 1 -- 172.21.15.26:0/2033419649 wait complete. 2023-12-19T17:57:31.377 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T17:57:31.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.523+0000 7fbda92b6640 1 Processor -- start 2023-12-19T17:57:31.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.523+0000 7fbda92b6640 1 -- start start 2023-12-19T17:57:31.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.523+0000 7fbda92b6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbda415cdf0 0x7fbda406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.523+0000 7fbda92b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda406f270 0x7fbda40cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.523+0000 7fbda92b6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbda40cfdf0 0x7fbda4071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.523+0000 7fbda92b6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda415d1d0 con 0x7fbda415cdf0 2023-12-19T17:57:31.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.523+0000 7fbda92b6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda406f6d0 con 0x7fbda40cfdf0 2023-12-19T17:57:31.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.523+0000 7fbda92b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40d0280 con 0x7fbda406f270 2023-12-19T17:57:31.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.523+0000 7fbda2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbda415cdf0 0x7fbda406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.527+0000 7fbda2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbda415cdf0 0x7fbda406ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43802/0 (socket says 172.21.15.26:43802) 2023-12-19T17:57:31.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.527+0000 7fbda2ffd640 1 -- 172.21.15.26:0/2622918656 learned_addr learned my addr 172.21.15.26:0/2622918656 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:31.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.527+0000 7fbda37fe640 1 --2- 172.21.15.26:0/2622918656 >> v2:172.21.15.160:3300/0 conn(0x7fbda40cfdf0 0x7fbda4071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.527+0000 7fbd9bfff640 1 --2- 172.21.15.26:0/2622918656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda406f270 0x7fbda40cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.527+0000 7fbda2ffd640 1 -- 172.21.15.26:0/2622918656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda406f270 msgr2=0x7fbda40cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:31.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.527+0000 7fbda2ffd640 1 --2- 172.21.15.26:0/2622918656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda406f270 0x7fbda40cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.527+0000 7fbda2ffd640 1 -- 172.21.15.26:0/2622918656 >> v2:172.21.15.160:3300/0 conn(0x7fbda40cfdf0 msgr2=0x7fbda4071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:31.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.527+0000 7fbda2ffd640 1 --2- 172.21.15.26:0/2622918656 >> v2:172.21.15.160:3300/0 conn(0x7fbda40cfdf0 0x7fbda4071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.527+0000 7fbda2ffd640 1 -- 172.21.15.26:0/2622918656 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4161140 con 0x7fbda415cdf0 2023-12-19T17:57:31.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.527+0000 7fbda2ffd640 1 --2- 172.21.15.26:0/2622918656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda415cdf0 0x7fbda406ed30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fbda40b22b0 tx=0x7fbd9401c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c56e279472c212 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:31.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.527+0000 7fbda27fc640 1 -- 172.21.15.26:0/2622918656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd940ab600 con 0x7fbda415cdf0 2023-12-19T17:57:31.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.527+0000 7fbda27fc640 1 -- 172.21.15.26:0/2622918656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd940a7050 con 0x7fbda415cdf0 2023-12-19T17:57:31.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.527+0000 7fbda27fc640 1 -- 172.21.15.26:0/2622918656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd940ab980 con 0x7fbda415cdf0 2023-12-19T17:57:31.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 -- 172.21.15.26:0/2622918656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda415cdf0 msgr2=0x7fbda406ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 --2- 172.21.15.26:0/2622918656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda415cdf0 0x7fbda406ed30 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fbda40b22b0 tx=0x7fbd9401c210 comp rx=0 tx=0).stop 2023-12-19T17:57:31.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 -- 172.21.15.26:0/2622918656 shutdown_connections 2023-12-19T17:57:31.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 --2- 172.21.15.26:0/2622918656 >> v2:172.21.15.160:3300/0 conn(0x7fbda40cfdf0 0x7fbda4071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 --2- 172.21.15.26:0/2622918656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda406f270 0x7fbda40cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 --2- 172.21.15.26:0/2622918656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda415cdf0 0x7fbda406ed30 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 -- 172.21.15.26:0/2622918656 >> 172.21.15.26:0/2622918656 conn(0x7fbda40d0830 msgr2=0x7fbda40d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:31.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 -- 172.21.15.26:0/2622918656 shutdown_connections 2023-12-19T17:57:31.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 -- 172.21.15.26:0/2622918656 wait complete. 2023-12-19T17:57:31.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 Processor -- start 2023-12-19T17:57:31.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 -- start start 2023-12-19T17:57:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda406f270 0x7fbda415be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda40cfdf0 0x7fbda4154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda415cdf0 0x7fbda40c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda4154010 con 0x7fbda415cdf0 2023-12-19T17:57:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda40afed0 con 0x7fbda406f270 2023-12-19T17:57:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda92b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda41536b0 con 0x7fbda40cfdf0 2023-12-19T17:57:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda415cdf0 0x7fbda40c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda415cdf0 0x7fbda40c6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43804/0 (socket says 172.21.15.26:43804) 2023-12-19T17:57:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda37fe640 1 -- 172.21.15.26:0/3440677476 learned_addr learned my addr 172.21.15.26:0/3440677476 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:31.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbda2ffd640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda406f270 0x7fbda415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.531+0000 7fbd9bfff640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda40cfdf0 0x7fbda4154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbda37fe640 1 -- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda40cfdf0 msgr2=0x7fbda4154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbda37fe640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda40cfdf0 0x7fbda4154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbda37fe640 1 -- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda406f270 msgr2=0x7fbda415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbda37fe640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda406f270 0x7fbda415be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbda37fe640 1 -- 172.21.15.26:0/3440677476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd940094d0 con 0x7fbda415cdf0 2023-12-19T17:57:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbda37fe640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda415cdf0 0x7fbda40c6850 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00efc0 tx=0x7fbd9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbd9b7fe640 1 -- 172.21.15.26:0/3440677476 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c009b10 con 0x7fbda415cdf0 2023-12-19T17:57:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbd9b7fe640 1 -- 172.21.15.26:0/3440677476 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd9c00f040 con 0x7fbda415cdf0 2023-12-19T17:57:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbd9b7fe640 1 -- 172.21.15.26:0/3440677476 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0146c0 con 0x7fbda415cdf0 2023-12-19T17:57:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbda92b6640 1 -- 172.21.15.26:0/3440677476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda4153950 con 0x7fbda415cdf0 2023-12-19T17:57:31.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbda92b6640 1 -- 172.21.15.26:0/3440677476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda40c6d90 con 0x7fbda415cdf0 2023-12-19T17:57:31.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbda92b6640 1 -- 172.21.15.26:0/3440677476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda4154150 con 0x7fbda415cdf0 2023-12-19T17:57:31.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbd9b7fe640 1 -- 172.21.15.26:0/3440677476 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbd9c014820 con 0x7fbda415cdf0 2023-12-19T17:57:31.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.535+0000 7fbd9b7fe640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd84065b80 0x7fbd84068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.539+0000 7fbd9b7fe640 1 -- 172.21.15.26:0/3440677476 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fbd9c084310 con 0x7fbda415cdf0 2023-12-19T17:57:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.539+0000 7fbda2ffd640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd84065b80 0x7fbd84068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.539+0000 7fbda2ffd640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd84065b80 0x7fbd84068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbd940ad000 tx=0x7fbd940a6000 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:31.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.539+0000 7fbd9b7fe640 1 -- 172.21.15.26:0/3440677476 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbda4154150 con 0x7fbda415cdf0 2023-12-19T17:57:31.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.695+0000 7fbda92b6640 1 -- 172.21.15.26:0/3440677476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbda4154150 con 0x7fbda415cdf0 2023-12-19T17:57:31.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.703+0000 7fbd9b7fe640 1 -- 172.21.15.26:0/3440677476 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbda4154150 con 0x7fbda415cdf0 2023-12-19T17:57:31.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.703+0000 7fbda92b6640 1 -- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd84065b80 msgr2=0x7fbd84068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.703+0000 7fbda92b6640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd84065b80 0x7fbd84068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbd940ad000 tx=0x7fbd940a6000 comp rx=0 tx=0).stop 2023-12-19T17:57:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.703+0000 7fbda92b6640 1 -- 172.21.15.26:0/3440677476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda415cdf0 msgr2=0x7fbda40c6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.703+0000 7fbda92b6640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda415cdf0 0x7fbda40c6850 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00efc0 tx=0x7fbd9c00c490 comp rx=0 tx=0).stop 2023-12-19T17:57:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.703+0000 7fbda92b6640 1 -- 172.21.15.26:0/3440677476 shutdown_connections 2023-12-19T17:57:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.703+0000 7fbda92b6640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda415cdf0 0x7fbda40c6850 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.703+0000 7fbda92b6640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd84065b80 0x7fbd84068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.703+0000 7fbda92b6640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda40cfdf0 0x7fbda4154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.703+0000 7fbda92b6640 1 --2- 172.21.15.26:0/3440677476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda406f270 0x7fbda415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.703+0000 7fbda92b6640 1 -- 172.21.15.26:0/3440677476 >> 172.21.15.26:0/3440677476 conn(0x7fbda40d0830 msgr2=0x7fbda40c4400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.703+0000 7fbda92b6640 1 -- 172.21.15.26:0/3440677476 shutdown_connections 2023-12-19T17:57:31.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.703+0000 7fbda92b6640 1 -- 172.21.15.26:0/3440677476 wait complete. 2023-12-19T17:57:31.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-19T17:57:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.867+0000 7f1339873640 1 Processor -- start 2023-12-19T17:57:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.867+0000 7f1339873640 1 -- start start 2023-12-19T17:57:31.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f1339873640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f133415d230 0x7f13340658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f1339873640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13341467c0 con 0x7f13341463e0 2023-12-19T17:57:31.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f1339873640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133415d780 con 0x7f1334065e20 2023-12-19T17:57:31.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f1339873640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13340662d0 con 0x7f133415d230 2023-12-19T17:57:31.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f1332ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f13341463e0 0x7f133406e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34440/0 (socket says 172.21.15.26:34440) 2023-12-19T17:57:31.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f1332ffd640 1 -- 172.21.15.26:0/1375628809 learned_addr learned my addr 172.21.15.26:0/1375628809 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:31.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574201054 0 0) 0x7f13341467c0 con 0x7f13341463e0 2023-12-19T17:57:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f132bfff640 1 --2- 172.21.15.26:0/1375628809 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f133415d230 0x7f13340658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1320003530 con 0x7f13341463e0 2023-12-19T17:57:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2334606059 0 0) 0x7f1320003530 con 0x7f13341463e0 2023-12-19T17:57:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f133415d230 msgr2=0x7f13340658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 --2- 172.21.15.26:0/1375628809 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f133415d230 0x7f13340658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 >> v1:172.21.15.160:6789/0 conn(0x7f1334065e20 legacy=0x7f13340b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133415da60 con 0x7f13341463e0 2023-12-19T17:57:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f1324001680 con 0x7f13341463e0 2023-12-19T17:57:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 >> v1:172.21.15.26:6789/0 conn(0x7f13341463e0 legacy=0x7f133406e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 --2- 172.21.15.26:0/1375628809 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f133415d230 0x7f13200039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 --2- 172.21.15.26:0/1375628809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1320003f30 0x7f13200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 --2- 172.21.15.26:0/1375628809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1320004d10 0x7f1320005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13340662d0 con 0x7f1320004d10 2023-12-19T17:57:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132000ba40 con 0x7f1320003f30 2023-12-19T17:57:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132000bc00 con 0x7f133415d230 2023-12-19T17:57:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13337fe640 1 --2- 172.21.15.26:0/1375628809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1320004d10 0x7f1320005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f1332ffd640 1 --2- 172.21.15.26:0/1375628809 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f133415d230 0x7f13200039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f132bfff640 1 --2- 172.21.15.26:0/1375628809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1320003f30 0x7f13200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13337fe640 1 -- 172.21.15.26:0/1375628809 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f133415d230 msgr2=0x7f13200039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13337fe640 1 --2- 172.21.15.26:0/1375628809 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f133415d230 0x7f13200039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13337fe640 1 -- 172.21.15.26:0/1375628809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1320003f30 msgr2=0x7f13200047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13337fe640 1 --2- 172.21.15.26:0/1375628809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1320003f30 0x7f13200047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f13337fe640 1 -- 172.21.15.26:0/1375628809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133415da60 con 0x7f1320004d10 2023-12-19T17:57:31.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.871+0000 7f1332ffd640 1 --2- 172.21.15.26:0/1375628809 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f133415d230 0x7f13200039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:31.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.875+0000 7f13337fe640 1 --2- 172.21.15.26:0/1375628809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1320004d10 0x7f1320005190 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f132c001e70 tx=0x7f132c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6982fd9307a139f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:31.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.875+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c003b80 con 0x7f1320004d10 2023-12-19T17:57:31.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.875+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c003ce0 con 0x7f1320004d10 2023-12-19T17:57:31.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.875+0000 7f13327fc640 1 -- 172.21.15.26:0/1375628809 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c0052c0 con 0x7f1320004d10 2023-12-19T17:57:31.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.875+0000 7f1339873640 1 -- 172.21.15.26:0/1375628809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1320004d10 msgr2=0x7f1320005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.875+0000 7f1339873640 1 --2- 172.21.15.26:0/1375628809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1320004d10 0x7f1320005190 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f132c001e70 tx=0x7f132c01c760 comp rx=0 tx=0).stop 2023-12-19T17:57:31.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.875+0000 7f13337fe640 1 -- 172.21.15.26:0/1375628809 reap_dead start 2023-12-19T17:57:31.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.875+0000 7f1339873640 1 -- 172.21.15.26:0/1375628809 shutdown_connections 2023-12-19T17:57:31.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.875+0000 7f1339873640 1 -- 172.21.15.26:0/1375628809 >> 172.21.15.26:0/1375628809 conn(0x7f133406a320 msgr2=0x7f13340b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:31.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.875+0000 7f1339873640 1 -- 172.21.15.26:0/1375628809 shutdown_connections 2023-12-19T17:57:31.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.875+0000 7f1339873640 1 -- 172.21.15.26:0/1375628809 wait complete. 2023-12-19T17:57:31.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1339873640 1 Processor -- start 2023-12-19T17:57:31.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1339873640 1 -- start start 2023-12-19T17:57:31.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1339873640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1320004d10 0x7f1334142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1339873640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f133413eef0 0x7f13340ad690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1339873640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13340adbd0 0x7f133414ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1339873640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13340ae050 con 0x7f133413eef0 2023-12-19T17:57:31.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1339873640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13341416b0 con 0x7f13340adbd0 2023-12-19T17:57:31.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1339873640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1334141990 con 0x7f1320004d10 2023-12-19T17:57:31.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f132bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f133413eef0 0x7f13340ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f132bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f133413eef0 0x7f13340ad690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43824/0 (socket says 172.21.15.26:43824) 2023-12-19T17:57:31.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f132bfff640 1 -- 172.21.15.26:0/383108631 learned_addr learned my addr 172.21.15.26:0/383108631 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f132bfff640 1 -- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1320004d10 msgr2=0x7f1334142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1332ffd640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1320004d10 0x7f1334142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f132bfff640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1320004d10 0x7f1334142470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f13337fe640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13340adbd0 0x7f133414ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f132bfff640 1 -- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13340adbd0 msgr2=0x7f133414ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f132bfff640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13340adbd0 0x7f133414ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f132bfff640 1 -- 172.21.15.26:0/383108631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1334141c70 con 0x7f133413eef0 2023-12-19T17:57:31.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1332ffd640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1320004d10 0x7f1334142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:57:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f13337fe640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13340adbd0 0x7f133414ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:57:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f132bfff640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f133413eef0 0x7f13340ad690 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f131c011930 tx=0x7f131c011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1330ff9640 1 -- 172.21.15.26:0/383108631 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c0095b0 con 0x7f133413eef0 2023-12-19T17:57:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1330ff9640 1 -- 172.21.15.26:0/383108631 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131c00faf0 con 0x7f133413eef0 2023-12-19T17:57:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1339873640 1 -- 172.21.15.26:0/383108631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1334141f50 con 0x7f133413eef0 2023-12-19T17:57:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1330ff9640 1 -- 172.21.15.26:0/383108631 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c00fe80 con 0x7f133413eef0 2023-12-19T17:57:31.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1339873640 1 -- 172.21.15.26:0/383108631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133414fa30 con 0x7f133413eef0 2023-12-19T17:57:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.879+0000 7f1339873640 1 -- 172.21.15.26:0/383108631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133415e000 con 0x7f133413eef0 2023-12-19T17:57:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.883+0000 7f1330ff9640 1 -- 172.21.15.26:0/383108631 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f131c014440 con 0x7f133413eef0 2023-12-19T17:57:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.883+0000 7f1330ff9640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1314065bb0 0x7f1314068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.883+0000 7f1330ff9640 1 -- 172.21.15.26:0/383108631 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f131c083e60 con 0x7f133413eef0 2023-12-19T17:57:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.883+0000 7f1332ffd640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1314065bb0 0x7f1314068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:31.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.883+0000 7f1332ffd640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1314065bb0 0x7f1314068070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1324000c10 tx=0x7f1324009040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:31.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:31.887+0000 7f1330ff9640 1 -- 172.21.15.26:0/383108631 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f133415e000 con 0x7f133413eef0 2023-12-19T17:57:32.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.055+0000 7f1339873640 1 -- 172.21.15.26:0/383108631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f133415e000 con 0x7f133413eef0 2023-12-19T17:57:32.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.059+0000 7f1330ff9640 1 -- 172.21.15.26:0/383108631 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v51) v1 ==== 115+0+0 (secure 0 0 0) 0x7f133415e000 con 0x7f133413eef0 2023-12-19T17:57:32.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.063+0000 7f1339873640 1 -- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1314065bb0 msgr2=0x7f1314068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:32.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.063+0000 7f1339873640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1314065bb0 0x7f1314068070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1324000c10 tx=0x7f1324009040 comp rx=0 tx=0).stop 2023-12-19T17:57:32.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.063+0000 7f1339873640 1 -- 172.21.15.26:0/383108631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f133413eef0 msgr2=0x7f13340ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:32.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.063+0000 7f1339873640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f133413eef0 0x7f13340ad690 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f131c011930 tx=0x7f131c011df0 comp rx=0 tx=0).stop 2023-12-19T17:57:32.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.063+0000 7f1339873640 1 -- 172.21.15.26:0/383108631 shutdown_connections 2023-12-19T17:57:32.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.063+0000 7f1339873640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13340adbd0 0x7f133414ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:32.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.063+0000 7f1339873640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f133413eef0 0x7f13340ad690 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.063+0000 7f1339873640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1314065bb0 0x7f1314068070 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.063+0000 7f1339873640 1 --2- 172.21.15.26:0/383108631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1320004d10 0x7f1334142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.063+0000 7f1339873640 1 -- 172.21.15.26:0/383108631 >> 172.21.15.26:0/383108631 conn(0x7f133406a320 msgr2=0x7f13340b4360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.063+0000 7f1339873640 1 -- 172.21.15.26:0/383108631 shutdown_connections 2023-12-19T17:57:32.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.063+0000 7f1339873640 1 -- 172.21.15.26:0/383108631 wait complete. 2023-12-19T17:57:32.084 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-12-19T17:57:32.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.223+0000 7fc07a0e5640 1 Processor -- start 2023-12-19T17:57:32.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.223+0000 7fc07a0e5640 1 -- start start 2023-12-19T17:57:32.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc07a0e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc07415c7f0 0x7fc07406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:32.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc07a0e5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc0740adb80 0x7fc0740ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:32.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc07a0e5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07406ced0 con 0x7fc0740cc6e0 2023-12-19T17:57:32.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc07a0e5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07406d770 con 0x7fc0740adb80 2023-12-19T17:57:32.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc07a0e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07406d1b0 con 0x7fc07415c7f0 2023-12-19T17:57:32.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc073fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc0740cc6e0 0x7fc0740b30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34454/0 (socket says 172.21.15.26:34454) 2023-12-19T17:57:32.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc073fff640 1 -- 172.21.15.26:0/1569283960 learned_addr learned my addr 172.21.15.26:0/1569283960 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:32.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0737fe640 1 --2- 172.21.15.26:0/1569283960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc07415c7f0 0x7fc07406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:32.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc072ffd640 1 --2- 172.21.15.26:0/1569283960 >> v2:172.21.15.160:3300/0 conn(0x7fc0740adb80 0x7fc0740ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:32.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 328172498 0 0) 0x7fc07406ced0 con 0x7fc0740cc6e0 2023-12-19T17:57:32.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc05c003530 con 0x7fc0740cc6e0 2023-12-19T17:57:32.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3493530168 0 0) 0x7fc05c003530 con 0x7fc0740cc6e0 2023-12-19T17:57:32.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc07415c7f0 msgr2=0x7fc07406c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:32.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 --2- 172.21.15.26:0/1569283960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc07415c7f0 0x7fc07406c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:32.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 >> v2:172.21.15.160:3300/0 conn(0x7fc0740adb80 msgr2=0x7fc0740ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 --2- 172.21.15.26:0/1569283960 >> v2:172.21.15.160:3300/0 conn(0x7fc0740adb80 0x7fc0740ce7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07406d490 con 0x7fc0740cc6e0 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fc06c001a00 con 0x7fc0740cc6e0 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 >> v1:172.21.15.26:6789/0 conn(0x7fc0740cc6e0 legacy=0x7fc0740b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 --2- 172.21.15.26:0/1569283960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc05c003980 0x7fc05c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 --2- 172.21.15.26:0/1569283960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc05c0047d0 0x7fc05c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 --2- 172.21.15.26:0/1569283960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05c005190 0x7fc05c00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07406d770 con 0x7fc05c005190 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07406d1b0 con 0x7fc05c003980 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c00e0e0 con 0x7fc05c0047d0 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0737fe640 1 --2- 172.21.15.26:0/1569283960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc07415c7f0 0x7fc07406c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc073fff640 1 --2- 172.21.15.26:0/1569283960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05c005190 0x7fc05c00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc072ffd640 1 --2- 172.21.15.26:0/1569283960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc05c0047d0 0x7fc05c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:32.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc0737fe640 1 --2- 172.21.15.26:0/1569283960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc05c003980 0x7fc05c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:32.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc073fff640 1 -- 172.21.15.26:0/1569283960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc05c0047d0 msgr2=0x7fc05c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:32.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc073fff640 1 --2- 172.21.15.26:0/1569283960 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc05c0047d0 0x7fc05c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:32.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc073fff640 1 -- 172.21.15.26:0/1569283960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc05c003980 msgr2=0x7fc05c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:32.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc073fff640 1 --2- 172.21.15.26:0/1569283960 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc05c003980 0x7fc05c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:32.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.227+0000 7fc073fff640 1 -- 172.21.15.26:0/1569283960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c003530 con 0x7fc05c005190 2023-12-19T17:57:32.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.231+0000 7fc073fff640 1 --2- 172.21.15.26:0/1569283960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05c005190 0x7fc05c00dba0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00a890 tx=0x7fc06c01d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ec9bf1c0356543 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:32.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.231+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0ad630 con 0x7fc05c005190 2023-12-19T17:57:32.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.231+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c0a8070 con 0x7fc05c005190 2023-12-19T17:57:32.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.231+0000 7fc0727fc640 1 -- 172.21.15.26:0/1569283960 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0ad9f0 con 0x7fc05c005190 2023-12-19T17:57:32.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.231+0000 7fc07a0e5640 1 -- 172.21.15.26:0/1569283960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05c005190 msgr2=0x7fc05c00dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:32.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.231+0000 7fc07a0e5640 1 --2- 172.21.15.26:0/1569283960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05c005190 0x7fc05c00dba0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00a890 tx=0x7fc06c01d360 comp rx=0 tx=0).stop 2023-12-19T17:57:32.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.231+0000 7fc073fff640 1 -- 172.21.15.26:0/1569283960 reap_dead start 2023-12-19T17:57:32.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.231+0000 7fc07a0e5640 1 -- 172.21.15.26:0/1569283960 shutdown_connections 2023-12-19T17:57:32.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.231+0000 7fc07a0e5640 1 -- 172.21.15.26:0/1569283960 >> 172.21.15.26:0/1569283960 conn(0x7fc0740658e0 msgr2=0x7fc074065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:32.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.231+0000 7fc07a0e5640 1 -- 172.21.15.26:0/1569283960 shutdown_connections 2023-12-19T17:57:32.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.231+0000 7fc07a0e5640 1 -- 172.21.15.26:0/1569283960 wait complete. 2023-12-19T17:57:32.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc07a0e5640 1 Processor -- start 2023-12-19T17:57:32.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc07a0e5640 1 -- start start 2023-12-19T17:57:32.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc07a0e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc05c005190 0x7fc0741e22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:32.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc07a0e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0741e19e0 0x7fc074145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:32.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc07a0e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0740cdb70 0x7fc074144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:32.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc07a0e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0741e28d0 con 0x7fc0740cdb70 2023-12-19T17:57:32.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc07a0e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0741e2bb0 con 0x7fc0741e19e0 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc07a0e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07415acf0 con 0x7fc05c005190 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc073fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0740cdb70 0x7fc074144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc073fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0740cdb70 0x7fc074144ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43844/0 (socket says 172.21.15.26:43844) 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc073fff640 1 -- 172.21.15.26:0/3650909025 learned_addr learned my addr 172.21.15.26:0/3650909025 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc0737fe640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc05c005190 0x7fc0741e22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc072ffd640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0741e19e0 0x7fc074145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc073fff640 1 -- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc05c005190 msgr2=0x7fc0741e22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc073fff640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc05c005190 0x7fc0741e22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc073fff640 1 -- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0741e19e0 msgr2=0x7fc074145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc073fff640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0741e19e0 0x7fc074145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc073fff640 1 -- 172.21.15.26:0/3650909025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07415afd0 con 0x7fc0740cdb70 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc072ffd640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0741e19e0 0x7fc074145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc073fff640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0740cdb70 0x7fc074144ed0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00a7b0 tx=0x7fc05c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:32.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc05bfff640 1 -- 172.21.15.26:0/3650909025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0017c0 con 0x7fc0740cdb70 2023-12-19T17:57:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc05bfff640 1 -- 172.21.15.26:0/3650909025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c001920 con 0x7fc0740cdb70 2023-12-19T17:57:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc05bfff640 1 -- 172.21.15.26:0/3650909025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c004d70 con 0x7fc0740cdb70 2023-12-19T17:57:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc07a0e5640 1 -- 172.21.15.26:0/3650909025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc074146290 con 0x7fc0740cdb70 2023-12-19T17:57:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.235+0000 7fc07a0e5640 1 -- 172.21.15.26:0/3650909025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0740cdff0 con 0x7fc0740cdb70 2023-12-19T17:57:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.239+0000 7fc07a0e5640 1 -- 172.21.15.26:0/3650909025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0741e2d70 con 0x7fc0740cdb70 2023-12-19T17:57:32.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.239+0000 7fc05bfff640 1 -- 172.21.15.26:0/3650909025 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc06c0aa030 con 0x7fc0740cdb70 2023-12-19T17:57:32.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.239+0000 7fc05bfff640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc060065bb0 0x7fc060068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:32.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.239+0000 7fc05bfff640 1 -- 172.21.15.26:0/3650909025 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fc06c11cbd0 con 0x7fc0740cdb70 2023-12-19T17:57:32.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.239+0000 7fc0737fe640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc060065bb0 0x7fc060068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:32.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.239+0000 7fc0737fe640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc060065bb0 0x7fc060068070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc0740670d0 tx=0x7fc064009190 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:32.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.243+0000 7fc05bfff640 1 -- 172.21.15.26:0/3650909025 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc06c0ea090 con 0x7fc0740cdb70 2023-12-19T17:57:32.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.455+0000 7fc07a0e5640 1 -- 172.21.15.26:0/3650909025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7fc0740ce240 con 0x7fc0740cdb70 2023-12-19T17:57:32.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.995+0000 7fc05bfff640 1 -- 172.21.15.26:0/3650909025 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2418) v1 ==== 178+0+0 (secure 0 0 0) 0x7fc06c0ba0f0 con 0x7fc0740cdb70 2023-12-19T17:57:33.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.999+0000 7fc07a0e5640 1 -- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc060065bb0 msgr2=0x7fc060068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:33.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.999+0000 7fc07a0e5640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc060065bb0 0x7fc060068070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc0740670d0 tx=0x7fc064009190 comp rx=0 tx=0).stop 2023-12-19T17:57:33.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.999+0000 7fc07a0e5640 1 -- 172.21.15.26:0/3650909025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0740cdb70 msgr2=0x7fc074144ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:33.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.999+0000 7fc07a0e5640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0740cdb70 0x7fc074144ed0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00a7b0 tx=0x7fc05c005800 comp rx=0 tx=0).stop 2023-12-19T17:57:33.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.999+0000 7fc07a0e5640 1 -- 172.21.15.26:0/3650909025 shutdown_connections 2023-12-19T17:57:33.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.999+0000 7fc07a0e5640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0740cdb70 0x7fc074144ed0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:33.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.999+0000 7fc07a0e5640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc0741e19e0 0x7fc074145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:33.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.999+0000 7fc07a0e5640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc060065bb0 0x7fc060068070 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:33.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.999+0000 7fc07a0e5640 1 --2- 172.21.15.26:0/3650909025 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc05c005190 0x7fc0741e22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:33.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.999+0000 7fc07a0e5640 1 -- 172.21.15.26:0/3650909025 >> 172.21.15.26:0/3650909025 conn(0x7fc0740658e0 msgr2=0x7fc0740b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:33.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.999+0000 7fc07a0e5640 1 -- 172.21.15.26:0/3650909025 shutdown_connections 2023-12-19T17:57:33.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:32.999+0000 7fc07a0e5640 1 -- 172.21.15.26:0/3650909025 wait complete. 2023-12-19T17:57:33.025 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-12-19T17:57:33.025 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-12-19T17:57:33.025 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-19T17:57:33.026 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-12-19T17:57:33.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.171+0000 7f4bdc095640 1 Processor -- start 2023-12-19T17:57:33.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.171+0000 7f4bdc095640 1 -- start start 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.171+0000 7f4bdc095640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd4154220 0x7f4bd415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.171+0000 7f4bdc095640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4bd406b3d0 0x7f4bd40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.171+0000 7f4bdc095640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4bd40cec40 0x7f4bd415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.171+0000 7f4bdc095640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd40cf0d0 con 0x7f4bd406b3d0 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.171+0000 7f4bdc095640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd406bc30 con 0x7f4bd40cec40 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.171+0000 7f4bdc095640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40cc6e0 con 0x7f4bd4154220 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.171+0000 7f4bd9630640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4bd406b3d0 0x7f4bd40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.171+0000 7f4bd9630640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4bd406b3d0 0x7f4bd40ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43854/0 (socket says 172.21.15.26:43854) 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.171+0000 7f4bd9630640 1 -- 172.21.15.26:0/3801941992 learned_addr learned my addr 172.21.15.26:0/3801941992 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.175+0000 7f4bd9e31640 1 --2- 172.21.15.26:0/3801941992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd4154220 0x7f4bd415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.175+0000 7f4bda632640 1 --2- 172.21.15.26:0/3801941992 >> v2:172.21.15.160:3300/0 conn(0x7f4bd40cec40 0x7f4bd415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.175+0000 7f4bd9630640 1 -- 172.21.15.26:0/3801941992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd4154220 msgr2=0x7f4bd415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.175+0000 7f4bd9630640 1 --2- 172.21.15.26:0/3801941992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd4154220 0x7f4bd415aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:33.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.175+0000 7f4bd9630640 1 -- 172.21.15.26:0/3801941992 >> v2:172.21.15.160:3300/0 conn(0x7f4bd40cec40 msgr2=0x7f4bd415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.175+0000 7f4bd9630640 1 --2- 172.21.15.26:0/3801941992 >> v2:172.21.15.160:3300/0 conn(0x7f4bd40cec40 0x7f4bd415d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:33.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.175+0000 7f4bd9630640 1 -- 172.21.15.26:0/3801941992 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd415afc0 con 0x7f4bd406b3d0 2023-12-19T17:57:33.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.175+0000 7f4bd9630640 1 --2- 172.21.15.26:0/3801941992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd406b3d0 0x7f4bd40ce700 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40ae310 tx=0x7f4bc401c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4bd6e908e91ce server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:33.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.175+0000 7f4bd8e2f640 1 -- 172.21.15.26:0/3801941992 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc40ab600 con 0x7f4bd406b3d0 2023-12-19T17:57:33.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.175+0000 7f4bd8e2f640 1 -- 172.21.15.26:0/3801941992 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc40a7050 con 0x7f4bd406b3d0 2023-12-19T17:57:33.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.175+0000 7f4bd8e2f640 1 -- 172.21.15.26:0/3801941992 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc40ab980 con 0x7f4bd406b3d0 2023-12-19T17:57:33.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.175+0000 7f4bdc095640 1 -- 172.21.15.26:0/3801941992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd406b3d0 msgr2=0x7f4bd40ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:33.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.175+0000 7f4bdc095640 1 --2- 172.21.15.26:0/3801941992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd406b3d0 0x7f4bd40ce700 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40ae310 tx=0x7f4bc401c210 comp rx=0 tx=0).stop 2023-12-19T17:57:33.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 -- 172.21.15.26:0/3801941992 shutdown_connections 2023-12-19T17:57:33.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 --2- 172.21.15.26:0/3801941992 >> v2:172.21.15.160:3300/0 conn(0x7f4bd40cec40 0x7f4bd415d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:33.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 --2- 172.21.15.26:0/3801941992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd406b3d0 0x7f4bd40ce700 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:33.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 --2- 172.21.15.26:0/3801941992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd4154220 0x7f4bd415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:33.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 -- 172.21.15.26:0/3801941992 >> 172.21.15.26:0/3801941992 conn(0x7f4bd40658e0 msgr2=0x7f4bd4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:33.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 -- 172.21.15.26:0/3801941992 shutdown_connections 2023-12-19T17:57:33.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 -- 172.21.15.26:0/3801941992 wait complete. 2023-12-19T17:57:33.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 Processor -- start 2023-12-19T17:57:33.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 -- start start 2023-12-19T17:57:33.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 0x7f4bd414a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:33.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd40cec40 0x7f4bd40ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:33.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd4154220 0x7f4bd40acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:33.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd414cac0 con 0x7f4bd4154220 2023-12-19T17:57:33.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40b19d0 con 0x7f4bd406b3d0 2023-12-19T17:57:33.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bdc095640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40b1cb0 con 0x7f4bd40cec40 2023-12-19T17:57:33.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bda632640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd4154220 0x7f4bd40acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:33.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bda632640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd4154220 0x7f4bd40acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43862/0 (socket says 172.21.15.26:43862) 2023-12-19T17:57:33.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bda632640 1 -- 172.21.15.26:0/1738527930 learned_addr learned my addr 172.21.15.26:0/1738527930 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:33.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bd9e31640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 0x7f4bd414a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:33.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bd9630640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd40cec40 0x7f4bd40ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bda632640 1 -- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd40cec40 msgr2=0x7f4bd40ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bda632640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd40cec40 0x7f4bd40ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bda632640 1 -- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 msgr2=0x7f4bd414a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bda632640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 0x7f4bd414a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.179+0000 7f4bda632640 1 -- 172.21.15.26:0/1738527930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc40094d0 con 0x7f4bd4154220 2023-12-19T17:57:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.183+0000 7f4bda632640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd4154220 0x7f4bd40acd90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc007e60 tx=0x7f4bcc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.183+0000 7f4bca7fc640 1 -- 172.21.15.26:0/1738527930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc014620 con 0x7f4bd4154220 2023-12-19T17:57:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.183+0000 7f4bca7fc640 1 -- 172.21.15.26:0/1738527930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bcc00ee90 con 0x7f4bd4154220 2023-12-19T17:57:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.183+0000 7f4bca7fc640 1 -- 172.21.15.26:0/1738527930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc0149e0 con 0x7f4bd4154220 2023-12-19T17:57:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.183+0000 7f4bdc095640 1 -- 172.21.15.26:0/1738527930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd40b1f90 con 0x7f4bd4154220 2023-12-19T17:57:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.183+0000 7f4bdc095640 1 -- 172.21.15.26:0/1738527930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd414ac10 con 0x7f4bd4154220 2023-12-19T17:57:33.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.183+0000 7f4bdc095640 1 -- 172.21.15.26:0/1738527930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd4074270 con 0x7f4bd4154220 2023-12-19T17:57:33.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.183+0000 7f4bca7fc640 1 -- 172.21.15.26:0/1738527930 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4bcc014b40 con 0x7f4bd4154220 2023-12-19T17:57:33.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.183+0000 7f4bca7fc640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4bb8065b80 0x7f4bb8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:33.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.183+0000 7f4bca7fc640 1 -- 172.21.15.26:0/1738527930 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f4bcc083dd0 con 0x7f4bd4154220 2023-12-19T17:57:33.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.183+0000 7f4bd9e31640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4bb8065b80 0x7f4bb8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:33.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.187+0000 7f4bd9e31640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4bb8065b80 0x7f4bb8068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40ad3a0 tx=0x7f4bd0008040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:33.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.187+0000 7f4bca7fc640 1 -- 172.21.15.26:0/1738527930 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4bcc051450 con 0x7f4bd4154220 2023-12-19T17:57:33.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.407+0000 7f4bdc095640 1 -- 172.21.15.26:0/1738527930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f4bd40b2440 con 0x7f4bd4154220 2023-12-19T17:57:34.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.999+0000 7f4bca7fc640 1 -- 172.21.15.26:0/1738527930 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2419) v1 ==== 177+0+0 (secure 0 0 0) 0x7f4bcc054a70 con 0x7f4bd4154220 2023-12-19T17:57:34.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.999+0000 7f4bdc095640 1 -- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4bb8065b80 msgr2=0x7f4bb8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.999+0000 7f4bdc095640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4bb8065b80 0x7f4bb8068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40ad3a0 tx=0x7f4bd0008040 comp rx=0 tx=0).stop 2023-12-19T17:57:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.999+0000 7f4bdc095640 1 -- 172.21.15.26:0/1738527930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd4154220 msgr2=0x7f4bd40acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.999+0000 7f4bdc095640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd4154220 0x7f4bd40acd90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc007e60 tx=0x7f4bcc00e4b0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.999+0000 7f4bdc095640 1 -- 172.21.15.26:0/1738527930 shutdown_connections 2023-12-19T17:57:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.999+0000 7f4bdc095640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd4154220 0x7f4bd40acd90 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.999+0000 7f4bdc095640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4bb8065b80 0x7f4bb8068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.999+0000 7f4bdc095640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd40cec40 0x7f4bd40ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.999+0000 7f4bdc095640 1 --2- 172.21.15.26:0/1738527930 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 0x7f4bd414a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:33.999+0000 7f4bdc095640 1 -- 172.21.15.26:0/1738527930 >> 172.21.15.26:0/1738527930 conn(0x7f4bd40658e0 msgr2=0x7f4bd4065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.003+0000 7f4bdc095640 1 -- 172.21.15.26:0/1738527930 shutdown_connections 2023-12-19T17:57:34.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.003+0000 7f4bdc095640 1 -- 172.21.15.26:0/1738527930 wait complete. 2023-12-19T17:57:34.026 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T17:57:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f406f196640 1 Processor -- start 2023-12-19T17:57:34.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f406f196640 1 -- start start 2023-12-19T17:57:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f406f196640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4068159820 0x7f406815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f406f196640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4068159c00 0x7f4068070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f406f196640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406806fb60 con 0x7f406806f6c0 2023-12-19T17:57:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f406f196640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406815e470 con 0x7f4068159820 2023-12-19T17:57:34.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f406f196640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406806fe40 con 0x7f4068159c00 2023-12-19T17:57:34.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f4067fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f406806f6c0 0x7f40680c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34466/0 (socket says 172.21.15.26:34466) 2023-12-19T17:57:34.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f4067fff640 1 -- 172.21.15.26:0/3567496918 learned_addr learned my addr 172.21.15.26:0/3567496918 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:34.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f406cf32640 1 --2- 172.21.15.26:0/3567496918 >> v2:172.21.15.26:3300/0 conn(0x7f4068159820 0x7f406815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:34.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f406d733640 1 --2- 172.21.15.26:0/3567496918 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4068159c00 0x7f4068070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:34.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f406cf32640 1 -- 172.21.15.26:0/3567496918 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4068159c00 msgr2=0x7f4068070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:34.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f406cf32640 1 --2- 172.21.15.26:0/3567496918 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4068159c00 0x7f4068070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f406cf32640 1 -- 172.21.15.26:0/3567496918 >> v1:172.21.15.26:6789/0 conn(0x7f406806f6c0 legacy=0x7f40680c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f40677fe640 1 -- 172.21.15.26:0/3567496918 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481199124 0 0) 0x7f406806fb60 con 0x7f406806f6c0 2023-12-19T17:57:34.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.175+0000 7f406cf32640 1 -- 172.21.15.26:0/3567496918 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4068070120 con 0x7f4068159820 2023-12-19T17:57:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.179+0000 7f406cf32640 1 --2- 172.21.15.26:0/3567496918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4068159820 0x7f406815df30 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f405c0097c0 tx=0x7f405c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8f8879945701f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.179+0000 7f40677fe640 1 -- 172.21.15.26:0/3567496918 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c0ad660 con 0x7f4068159820 2023-12-19T17:57:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.179+0000 7f40677fe640 1 -- 172.21.15.26:0/3567496918 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c0ad7c0 con 0x7f4068159820 2023-12-19T17:57:34.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.179+0000 7f40677fe640 1 -- 172.21.15.26:0/3567496918 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c0adb50 con 0x7f4068159820 2023-12-19T17:57:34.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.179+0000 7f406f196640 1 -- 172.21.15.26:0/3567496918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4068159820 msgr2=0x7f406815df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.179+0000 7f406f196640 1 --2- 172.21.15.26:0/3567496918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4068159820 0x7f406815df30 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f405c0097c0 tx=0x7f405c01c450 comp rx=0 tx=0).stop 2023-12-19T17:57:34.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.179+0000 7f406f196640 1 -- 172.21.15.26:0/3567496918 shutdown_connections 2023-12-19T17:57:34.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.179+0000 7f406f196640 1 --2- 172.21.15.26:0/3567496918 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4068159c00 0x7f4068070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.179+0000 7f406f196640 1 --2- 172.21.15.26:0/3567496918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4068159820 0x7f406815df30 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.179+0000 7f406f196640 1 -- 172.21.15.26:0/3567496918 >> 172.21.15.26:0/3567496918 conn(0x7f40680ce730 msgr2=0x7f40680ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:34.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.179+0000 7f406f196640 1 -- 172.21.15.26:0/3567496918 shutdown_connections 2023-12-19T17:57:34.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.179+0000 7f406f196640 1 -- 172.21.15.26:0/3567496918 wait complete. 2023-12-19T17:57:34.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406f196640 1 Processor -- start 2023-12-19T17:57:34.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406f196640 1 -- start start 2023-12-19T17:57:34.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406f196640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f406806f6c0 0x7f406814b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406f196640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4068159820 0x7f4068156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406f196640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4068159c00 0x7f40681570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406f196640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40680af8d0 con 0x7f4068159820 2023-12-19T17:57:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406f196640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40680b33d0 con 0x7f406806f6c0 2023-12-19T17:57:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406f196640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40680b30f0 con 0x7f4068159c00 2023-12-19T17:57:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f4067fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4068159820 0x7f4068156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f4067fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4068159820 0x7f4068156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43880/0 (socket says 172.21.15.26:43880) 2023-12-19T17:57:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f4067fff640 1 -- 172.21.15.26:0/1907743021 learned_addr learned my addr 172.21.15.26:0/1907743021 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406cf32640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f406806f6c0 0x7f406814b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406d733640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4068159c00 0x7f40681570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f4067fff640 1 -- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4068159c00 msgr2=0x7f40681570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f4067fff640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4068159c00 0x7f40681570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f4067fff640 1 -- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f406806f6c0 msgr2=0x7f406814b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f4067fff640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f406806f6c0 0x7f406814b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f4067fff640 1 -- 172.21.15.26:0/1907743021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c009420 con 0x7f4068159820 2023-12-19T17:57:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406d733640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4068159c00 0x7f40681570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406cf32640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f406806f6c0 0x7f406814b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:34.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f4067fff640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4068159820 0x7f4068156b90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f405400c450 tx=0x7f405400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f40657fa640 1 -- 172.21.15.26:0/1907743021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40540074e0 con 0x7f4068159820 2023-12-19T17:57:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f40657fa640 1 -- 172.21.15.26:0/1907743021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4054007640 con 0x7f4068159820 2023-12-19T17:57:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f40657fa640 1 -- 172.21.15.26:0/1907743021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40540049f0 con 0x7f4068159820 2023-12-19T17:57:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406f196640 1 -- 172.21.15.26:0/1907743021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406806b880 con 0x7f4068159820 2023-12-19T17:57:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406f196640 1 -- 172.21.15.26:0/1907743021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40680af560 con 0x7f4068159820 2023-12-19T17:57:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.183+0000 7f406f196640 1 -- 172.21.15.26:0/1907743021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4068074270 con 0x7f4068159820 2023-12-19T17:57:34.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.187+0000 7f40657fa640 1 -- 172.21.15.26:0/1907743021 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4054004b50 con 0x7f4068159820 2023-12-19T17:57:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.187+0000 7f40657fa640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f404c065b80 0x7f404c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.187+0000 7f40657fa640 1 -- 172.21.15.26:0/1907743021 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f4054082880 con 0x7f4068159820 2023-12-19T17:57:34.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.187+0000 7f406cf32640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f404c065b80 0x7f404c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:34.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.187+0000 7f406cf32640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f404c065b80 0x7f404c068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f405c0af000 tx=0x7f405c0a7000 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:34.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.191+0000 7f40657fa640 1 -- 172.21.15.26:0/1907743021 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4054053850 con 0x7f4068159820 2023-12-19T17:57:34.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.347+0000 7f406f196640 1 -- 172.21.15.26:0/1907743021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f40680b46b0 con 0x7f4068159820 2023-12-19T17:57:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.355+0000 7f40657fa640 1 -- 172.21.15.26:0/1907743021 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4054018020 con 0x7f4068159820 2023-12-19T17:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.355+0000 7f406f196640 1 -- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f404c065b80 msgr2=0x7f404c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.355+0000 7f406f196640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f404c065b80 0x7f404c068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f405c0af000 tx=0x7f405c0a7000 comp rx=0 tx=0).stop 2023-12-19T17:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.355+0000 7f406f196640 1 -- 172.21.15.26:0/1907743021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4068159820 msgr2=0x7f4068156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.355+0000 7f406f196640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4068159820 0x7f4068156b90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f405400c450 tx=0x7f405400c910 comp rx=0 tx=0).stop 2023-12-19T17:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.355+0000 7f406f196640 1 -- 172.21.15.26:0/1907743021 shutdown_connections 2023-12-19T17:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.355+0000 7f406f196640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f404c065b80 0x7f404c068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.355+0000 7f406f196640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4068159c00 0x7f40681570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.355+0000 7f406f196640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4068159820 0x7f4068156b90 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.355+0000 7f406f196640 1 --2- 172.21.15.26:0/1907743021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f406806f6c0 0x7f406814b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.355+0000 7f406f196640 1 -- 172.21.15.26:0/1907743021 >> 172.21.15.26:0/1907743021 conn(0x7f40680ce730 msgr2=0x7f40680c9860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.355+0000 7f406f196640 1 -- 172.21.15.26:0/1907743021 shutdown_connections 2023-12-19T17:57:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.355+0000 7f406f196640 1 -- 172.21.15.26:0/1907743021 wait complete. 2023-12-19T17:57:34.378 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T17:57:34.379 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T17:57:34.458 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T17:57:34.511 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T17:57:34.512 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:57:34.512 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:34.522 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T17:57:34.522 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T17:57:34.522 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:34.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T17:57:34.531 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:57:34.531 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:34.576 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T17:57:34.576 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T17:57:34.577 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:34.587 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T17:57:34.587 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T17:57:34.587 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:57:34.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.759+0000 7f9448dce640 1 Processor -- start 2023-12-19T17:57:34.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.759+0000 7f9448dce640 1 -- start start 2023-12-19T17:57:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f9448dce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9444155030 0x7f944415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f9448dce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f944415f370 0x7f944406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:34.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f9448dce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f944406c970 con 0x7f94440ce740 2023-12-19T17:57:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f9448dce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944415b1f0 con 0x7f944415f370 2023-12-19T17:57:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f9448dce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944415b4d0 con 0x7f9444155030 2023-12-19T17:57:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f943bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f944415f370 0x7f944406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f943bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f944415f370 0x7f944406c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43892/0 (socket says 172.21.15.26:43892) 2023-12-19T17:57:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f943bfff640 1 -- 172.21.15.26:0/2356308930 learned_addr learned my addr 172.21.15.26:0/2356308930 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f944259c640 1 --2- 172.21.15.26:0/2356308930 >> v2:172.21.15.160:3300/0 conn(0x7f9444155030 0x7f944415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f943bfff640 1 -- 172.21.15.26:0/2356308930 >> v2:172.21.15.160:3300/0 conn(0x7f9444155030 msgr2=0x7f944415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f943bfff640 1 --2- 172.21.15.26:0/2356308930 >> v2:172.21.15.160:3300/0 conn(0x7f9444155030 0x7f944415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f943bfff640 1 -- 172.21.15.26:0/2356308930 >> v1:172.21.15.160:6789/0 conn(0x7f94440ce740 legacy=0x7f94440ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f943bfff640 1 -- 172.21.15.26:0/2356308930 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944415b7b0 con 0x7f944415f370 2023-12-19T17:57:34.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.763+0000 7f943bfff640 1 --2- 172.21.15.26:0/2356308930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f944415f370 0x7f944406c080 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f94440cf4a0 tx=0x7f942c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c761840c08c5ac79 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:34.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9441d9b640 1 -- 172.21.15.26:0/2356308930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0ab620 con 0x7f944415f370 2023-12-19T17:57:34.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9441d9b640 1 -- 172.21.15.26:0/2356308930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c0ab780 con 0x7f944415f370 2023-12-19T17:57:34.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9441d9b640 1 -- 172.21.15.26:0/2356308930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c0abb10 con 0x7f944415f370 2023-12-19T17:57:34.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 -- 172.21.15.26:0/2356308930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f944415f370 msgr2=0x7f944406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 --2- 172.21.15.26:0/2356308930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f944415f370 0x7f944406c080 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f94440cf4a0 tx=0x7f942c01c500 comp rx=0 tx=0).stop 2023-12-19T17:57:34.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 -- 172.21.15.26:0/2356308930 shutdown_connections 2023-12-19T17:57:34.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 --2- 172.21.15.26:0/2356308930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f944415f370 0x7f944406c080 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 --2- 172.21.15.26:0/2356308930 >> v2:172.21.15.160:3300/0 conn(0x7f9444155030 0x7f944415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 -- 172.21.15.26:0/2356308930 >> 172.21.15.26:0/2356308930 conn(0x7f94440658e0 msgr2=0x7f9444065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:34.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 -- 172.21.15.26:0/2356308930 shutdown_connections 2023-12-19T17:57:34.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 -- 172.21.15.26:0/2356308930 wait complete. 2023-12-19T17:57:34.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 Processor -- start 2023-12-19T17:57:34.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 -- start start 2023-12-19T17:57:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94440ce740 0x7f9444148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9444155030 0x7f94440b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f944415f370 0x7f9444153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9444151e60 con 0x7f944415f370 2023-12-19T17:57:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944414af30 con 0x7f9444155030 2023-12-19T17:57:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9448dce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9444152070 con 0x7f94440ce740 2023-12-19T17:57:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.767+0000 7f9442d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f944415f370 0x7f9444153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f943bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9444155030 0x7f94440b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:34.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f9442d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f944415f370 0x7f9444153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43896/0 (socket says 172.21.15.26:43896) 2023-12-19T17:57:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f9442d9d640 1 -- 172.21.15.26:0/2001509416 learned_addr learned my addr 172.21.15.26:0/2001509416 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f944259c640 1 --2- 172.21.15.26:0/2001509416 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94440ce740 0x7f9444148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f9442d9d640 1 -- 172.21.15.26:0/2001509416 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94440ce740 msgr2=0x7f9444148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f9442d9d640 1 --2- 172.21.15.26:0/2001509416 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94440ce740 0x7f9444148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f9442d9d640 1 -- 172.21.15.26:0/2001509416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9444155030 msgr2=0x7f94440b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f9442d9d640 1 --2- 172.21.15.26:0/2001509416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9444155030 0x7f94440b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f9442d9d640 1 -- 172.21.15.26:0/2001509416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0094d0 con 0x7f944415f370 2023-12-19T17:57:34.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f9442d9d640 1 --2- 172.21.15.26:0/2001509416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f944415f370 0x7f9444153e60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f943c00c770 tx=0x7f943c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f943affd640 1 -- 172.21.15.26:0/2001509416 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c003b80 con 0x7f944415f370 2023-12-19T17:57:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f943affd640 1 -- 172.21.15.26:0/2001509416 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c003ce0 con 0x7f944415f370 2023-12-19T17:57:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f9448dce640 1 -- 172.21.15.26:0/2001509416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94440cf0e0 con 0x7f944415f370 2023-12-19T17:57:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f943affd640 1 -- 172.21.15.26:0/2001509416 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c00aa90 con 0x7f944415f370 2023-12-19T17:57:34.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f9448dce640 1 -- 172.21.15.26:0/2001509416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9444152630 con 0x7f944415f370 2023-12-19T17:57:34.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f9448dce640 1 -- 172.21.15.26:0/2001509416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f944414b0c0 con 0x7f944415f370 2023-12-19T17:57:34.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f943affd640 1 -- 172.21.15.26:0/2001509416 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f943c00abf0 con 0x7f944415f370 2023-12-19T17:57:34.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.771+0000 7f943affd640 1 --2- 172.21.15.26:0/2001509416 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9424065b80 0x7f9424068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:34.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.775+0000 7f943affd640 1 -- 172.21.15.26:0/2001509416 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f943c0832d0 con 0x7f944415f370 2023-12-19T17:57:34.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.775+0000 7f944259c640 1 --2- 172.21.15.26:0/2001509416 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9424065b80 0x7f9424068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:34.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.775+0000 7f944259c640 1 --2- 172.21.15.26:0/2001509416 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9424065b80 0x7f9424068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9434009b60 tx=0x7f9434009190 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:34.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.775+0000 7f943affd640 1 -- 172.21.15.26:0/2001509416 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f943c050950 con 0x7f944415f370 2023-12-19T17:57:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.967+0000 7f9448dce640 1 -- 172.21.15.26:0/2001509416 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9444074270 con 0x7f944415f370 2023-12-19T17:57:34.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.967+0000 7f943affd640 1 -- 172.21.15.26:0/2001509416 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f943c053f70 con 0x7f944415f370 2023-12-19T17:57:34.970 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:34.971 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8092,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3217828830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3217828830},{"type":"v1","addr":"172.21.15.160:6835","nonce":3217828830}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8095,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/2758554130","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":2758554130},{"type":"v1","addr":"172.21.15.160:6837","nonce":2758554130}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3510623843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3510623843},{"type":"v1","addr":"172.21.15.26:6833","nonce":3510623843}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60},{"gid":8922,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/892306171","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":892306171},{"type":"v1","addr":"172.21.15.26:6835","nonce":892306171}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[]} 2023-12-19T17:57:34.971 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 67 2023-12-19T17:57:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.971+0000 7f9448dce640 1 -- 172.21.15.26:0/2001509416 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9424065b80 msgr2=0x7f9424068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.971+0000 7f9448dce640 1 --2- 172.21.15.26:0/2001509416 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9424065b80 0x7f9424068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9434009b60 tx=0x7f9434009190 comp rx=0 tx=0).stop 2023-12-19T17:57:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.971+0000 7f9448dce640 1 -- 172.21.15.26:0/2001509416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f944415f370 msgr2=0x7f9444153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.971+0000 7f9448dce640 1 --2- 172.21.15.26:0/2001509416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f944415f370 0x7f9444153e60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f943c00c770 tx=0x7f943c00cc30 comp rx=0 tx=0).stop 2023-12-19T17:57:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.971+0000 7f9448dce640 1 -- 172.21.15.26:0/2001509416 shutdown_connections 2023-12-19T17:57:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.971+0000 7f9448dce640 1 --2- 172.21.15.26:0/2001509416 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f944415f370 0x7f9444153e60 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.971+0000 7f9448dce640 1 --2- 172.21.15.26:0/2001509416 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9444155030 0x7f94440b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.971+0000 7f9448dce640 1 --2- 172.21.15.26:0/2001509416 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9424065b80 0x7f9424068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.971+0000 7f9448dce640 1 --2- 172.21.15.26:0/2001509416 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94440ce740 0x7f9444148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.971+0000 7f9448dce640 1 -- 172.21.15.26:0/2001509416 >> 172.21.15.26:0/2001509416 conn(0x7f94440658e0 msgr2=0x7f94440cd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.971+0000 7f9448dce640 1 -- 172.21.15.26:0/2001509416 shutdown_connections 2023-12-19T17:57:34.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:34.971+0000 7f9448dce640 1 -- 172.21.15.26:0/2001509416 wait complete. 2023-12-19T17:57:34.992 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T17:57:34.992 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T17:57:34.992 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T17:57:34.992 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T17:57:34.993 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T17:57:34.993 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T17:57:34.993 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T17:57:34.993 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T17:57:34.993 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T17:57:34.993 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T17:57:34.994 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T17:57:34.994 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T17:57:34.994 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T17:57:34.994 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T17:57:34.994 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T17:57:34.994 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T17:57:35.005 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T17:57:35.005 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T17:57:35.006 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T17:57:35.006 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T17:57:35.010 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T17:57:35.010 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T17:57:35.011 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T17:57:35.011 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T17:57:35.051 INFO:tasks.ceph.mds.a:Started 2023-12-19T17:57:35.051 DEBUG:teuthology.parallel:result is None 2023-12-19T17:57:35.053 INFO:tasks.ceph.mds.c:Started 2023-12-19T17:57:35.053 DEBUG:teuthology.parallel:result is None 2023-12-19T17:57:35.054 INFO:tasks.ceph.mds.d:Started 2023-12-19T17:57:35.054 DEBUG:teuthology.parallel:result is None 2023-12-19T17:57:35.057 INFO:tasks.ceph.mds.b:Started 2023-12-19T17:57:35.057 DEBUG:teuthology.parallel:result is None 2023-12-19T17:57:35.057 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T17:57:35.140 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T17:57:35.140 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T17:57:35.145 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T17:57:35.145 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T17:57:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.207+0000 7fc1c8d6d640 1 Processor -- start 2023-12-19T17:57:35.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.207+0000 7fc1c8d6d640 1 -- start start 2023-12-19T17:57:35.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1c8d6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1c4154220 0x7fc1c415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:35.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1c8d6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c406bc30 con 0x7fc1c406b3d0 2023-12-19T17:57:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1c8d6d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c40cc6e0 con 0x7fc1c40cee10 2023-12-19T17:57:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1c8d6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c415afc0 con 0x7fc1c4154220 2023-12-19T17:57:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1bbfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc1c406b3d0 0x7fc1c40ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34492/0 (socket says 172.21.15.26:34492) 2023-12-19T17:57:35.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1bbfff640 1 -- 172.21.15.26:0/1954400169 learned_addr learned my addr 172.21.15.26:0/1954400169 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1c259c640 1 --2- 172.21.15.26:0/1954400169 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1c4154220 0x7fc1c415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1c1d9b640 1 -- 172.21.15.26:0/1954400169 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321094453 0 0) 0x7fc1c406bc30 con 0x7fc1c406b3d0 2023-12-19T17:57:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1c1d9b640 1 -- 172.21.15.26:0/1954400169 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1b0003530 con 0x7fc1c406b3d0 2023-12-19T17:57:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1c1d9b640 1 -- 172.21.15.26:0/1954400169 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507762816 0 0) 0x7fc1c40cc6e0 con 0x7fc1c40cee10 2023-12-19T17:57:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1c1d9b640 1 -- 172.21.15.26:0/1954400169 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1c406bc30 con 0x7fc1c40cee10 2023-12-19T17:57:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1c259c640 1 -- 172.21.15.26:0/1954400169 >> v1:172.21.15.160:6789/0 conn(0x7fc1c40cee10 legacy=0x7fc1c415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1c259c640 1 -- 172.21.15.26:0/1954400169 >> v1:172.21.15.26:6789/0 conn(0x7fc1c406b3d0 legacy=0x7fc1c40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:35.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.211+0000 7fc1c259c640 1 -- 172.21.15.26:0/1954400169 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c406b950 con 0x7fc1c4154220 2023-12-19T17:57:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.215+0000 7fc1c259c640 1 --2- 172.21.15.26:0/1954400169 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1c4154220 0x7fc1c415aa80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40097f0 tx=0x7fc1b401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4fedec01dab59ef server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:35.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.215+0000 7fc1c1d9b640 1 -- 172.21.15.26:0/1954400169 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40aa5f0 con 0x7fc1c4154220 2023-12-19T17:57:35.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.215+0000 7fc1c1d9b640 1 -- 172.21.15.26:0/1954400169 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b40aa750 con 0x7fc1c4154220 2023-12-19T17:57:35.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.215+0000 7fc1c1d9b640 1 -- 172.21.15.26:0/1954400169 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40aaae0 con 0x7fc1c4154220 2023-12-19T17:57:35.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.215+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/1954400169 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1c4154220 msgr2=0x7fc1c415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:35.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.215+0000 7fc1c8d6d640 1 --2- 172.21.15.26:0/1954400169 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1c4154220 0x7fc1c415aa80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40097f0 tx=0x7fc1b401c490 comp rx=0 tx=0).stop 2023-12-19T17:57:35.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.215+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/1954400169 shutdown_connections 2023-12-19T17:57:35.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.215+0000 7fc1c8d6d640 1 --2- 172.21.15.26:0/1954400169 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1c4154220 0x7fc1c415aa80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:35.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.215+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/1954400169 >> 172.21.15.26:0/1954400169 conn(0x7fc1c40658e0 msgr2=0x7fc1c4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:35.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.215+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/1954400169 shutdown_connections 2023-12-19T17:57:35.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.215+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/1954400169 wait complete. 2023-12-19T17:57:35.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.215+0000 7fc1c8d6d640 1 Processor -- start 2023-12-19T17:57:35.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.215+0000 7fc1c8d6d640 1 -- start start 2023-12-19T17:57:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c8d6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c406b3d0 0x7fc1c4147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c8d6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1c40cee10 0x7fc1c40adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c8d6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1c4154220 0x7fc1c4151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c8d6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c414f250 con 0x7fc1c406b3d0 2023-12-19T17:57:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c8d6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c414f510 con 0x7fc1c4154220 2023-12-19T17:57:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c8d6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40aee20 con 0x7fc1c40cee10 2023-12-19T17:57:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c406b3d0 0x7fc1c4147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c406b3d0 0x7fc1c4147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43938/0 (socket says 172.21.15.26:43938) 2023-12-19T17:57:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c259c640 1 -- 172.21.15.26:0/2278893669 learned_addr learned my addr 172.21.15.26:0/2278893669 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1bbfff640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1c40cee10 0x7fc1c40adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c2d9d640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1c4154220 0x7fc1c4151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c259c640 1 -- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1c40cee10 msgr2=0x7fc1c40adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c259c640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1c40cee10 0x7fc1c40adac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c259c640 1 -- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1c4154220 msgr2=0x7fc1c4151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c259c640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1c4154220 0x7fc1c4151b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c259c640 1 -- 172.21.15.26:0/2278893669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b4009420 con 0x7fc1c406b3d0 2023-12-19T17:57:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1bbfff640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1c40cee10 0x7fc1c40adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:35.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c259c640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c406b3d0 0x7fc1c4147e40 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40190a0 tx=0x7fc1b401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1baffd640 1 -- 172.21.15.26:0/2278893669 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40be610 con 0x7fc1c406b3d0 2023-12-19T17:57:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1baffd640 1 -- 172.21.15.26:0/2278893669 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b40be770 con 0x7fc1c406b3d0 2023-12-19T17:57:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1baffd640 1 -- 172.21.15.26:0/2278893669 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40beb20 con 0x7fc1c406b3d0 2023-12-19T17:57:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/2278893669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c40ae580 con 0x7fc1c406b3d0 2023-12-19T17:57:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/2278893669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c40aeb40 con 0x7fc1c406b3d0 2023-12-19T17:57:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.219+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/2278893669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c4074270 con 0x7fc1c406b3d0 2023-12-19T17:57:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.223+0000 7fc1baffd640 1 -- 172.21.15.26:0/2278893669 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc1b40bec80 con 0x7fc1c406b3d0 2023-12-19T17:57:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.223+0000 7fc1baffd640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc1a4065b80 0x7fc1a4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.223+0000 7fc1baffd640 1 -- 172.21.15.26:0/2278893669 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fc1b40a7070 con 0x7fc1c406b3d0 2023-12-19T17:57:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.223+0000 7fc1bbfff640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc1a4065b80 0x7fc1a4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:35.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.223+0000 7fc1bbfff640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc1a4065b80 0x7fc1a4068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40af200 tx=0x7fc1ac009040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:35.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.227+0000 7fc1baffd640 1 -- 172.21.15.26:0/2278893669 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc1b40eb2b0 con 0x7fc1c406b3d0 2023-12-19T17:57:35.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:35.399+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/2278893669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc1c40ad370 con 0x7fc1c406b3d0 2023-12-19T17:57:36.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.359+0000 7fc1baffd640 1 -- 172.21.15.26:0/2278893669 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v159) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc1b40ee8d0 con 0x7fc1c406b3d0 2023-12-19T17:57:36.362 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2023-12-19T17:57:36.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.363+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc1a4065b80 msgr2=0x7fc1a4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:36.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.363+0000 7fc1c8d6d640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc1a4065b80 0x7fc1a4068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40af200 tx=0x7fc1ac009040 comp rx=0 tx=0).stop 2023-12-19T17:57:36.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.363+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/2278893669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c406b3d0 msgr2=0x7fc1c4147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:36.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.363+0000 7fc1c8d6d640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c406b3d0 0x7fc1c4147e40 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40190a0 tx=0x7fc1b401acd0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.363+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/2278893669 shutdown_connections 2023-12-19T17:57:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.363+0000 7fc1c8d6d640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1c4154220 0x7fc1c4151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.363+0000 7fc1c8d6d640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc1a4065b80 0x7fc1a4068040 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.363+0000 7fc1c8d6d640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc1c40cee10 0x7fc1c40adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.363+0000 7fc1c8d6d640 1 --2- 172.21.15.26:0/2278893669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1c406b3d0 0x7fc1c4147e40 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.363+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/2278893669 >> 172.21.15.26:0/2278893669 conn(0x7fc1c40658e0 msgr2=0x7fc1c40ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.363+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/2278893669 shutdown_connections 2023-12-19T17:57:36.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.363+0000 7fc1c8d6d640 1 -- 172.21.15.26:0/2278893669 wait complete. 2023-12-19T17:57:36.388 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T17:57:36.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f906359e640 1 Processor -- start 2023-12-19T17:57:36.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f906359e640 1 -- start start 2023-12-19T17:57:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f906359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90640b8760 0x7f90640cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f906359e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90640cdbb0 0x7f90640d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f906359e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90640cdff0 con 0x7f906415cca0 2023-12-19T17:57:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f906359e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90640b7c30 con 0x7f90640b8760 2023-12-19T17:57:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f906359e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90640b6cf0 con 0x7f90640cdbb0 2023-12-19T17:57:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f905bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90640b8760 0x7f90640cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f905bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90640b8760 0x7f90640cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46066/0 (socket says 172.21.15.26:46066) 2023-12-19T17:57:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f905bfff640 1 -- 172.21.15.26:0/3494366835 learned_addr learned my addr 172.21.15.26:0/3494366835 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f905bfff640 1 -- 172.21.15.26:0/3494366835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90640cdbb0 msgr2=0x7f90640d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f9062d9d640 1 --2- 172.21.15.26:0/3494366835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90640cdbb0 0x7f90640d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f905bfff640 1 --2- 172.21.15.26:0/3494366835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90640cdbb0 0x7f90640d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f905bfff640 1 -- 172.21.15.26:0/3494366835 >> v1:172.21.15.26:6789/0 conn(0x7f906415cca0 legacy=0x7f90640b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:36.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f905bfff640 1 -- 172.21.15.26:0/3494366835 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90640b6fb0 con 0x7f90640b8760 2023-12-19T17:57:36.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.535+0000 7f9062d9d640 1 --2- 172.21.15.26:0/3494366835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90640cdbb0 0x7f90640d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:57:36.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.539+0000 7f905bfff640 1 --2- 172.21.15.26:0/3494366835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90640b8760 0x7f90640cd670 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f904c009870 tx=0x7f904c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd70371deea7bc4d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:36.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.539+0000 7f9061d9b640 1 -- 172.21.15.26:0/3494366835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0aa670 con 0x7f90640b8760 2023-12-19T17:57:36.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.539+0000 7f9061d9b640 1 -- 172.21.15.26:0/3494366835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c0aa7d0 con 0x7f90640b8760 2023-12-19T17:57:36.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.539+0000 7f9061d9b640 1 -- 172.21.15.26:0/3494366835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c0aab60 con 0x7f90640b8760 2023-12-19T17:57:36.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.539+0000 7f906359e640 1 -- 172.21.15.26:0/3494366835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90640b8760 msgr2=0x7f90640cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:36.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.539+0000 7f906359e640 1 --2- 172.21.15.26:0/3494366835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90640b8760 0x7f90640cd670 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f904c009870 tx=0x7f904c01c510 comp rx=0 tx=0).stop 2023-12-19T17:57:36.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.539+0000 7f906359e640 1 -- 172.21.15.26:0/3494366835 shutdown_connections 2023-12-19T17:57:36.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.539+0000 7f906359e640 1 --2- 172.21.15.26:0/3494366835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90640cdbb0 0x7f90640d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.539+0000 7f906359e640 1 --2- 172.21.15.26:0/3494366835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90640b8760 0x7f90640cd670 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.539+0000 7f906359e640 1 -- 172.21.15.26:0/3494366835 >> 172.21.15.26:0/3494366835 conn(0x7f906406a120 msgr2=0x7f906406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:36.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.539+0000 7f906359e640 1 -- 172.21.15.26:0/3494366835 shutdown_connections 2023-12-19T17:57:36.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.539+0000 7f906359e640 1 -- 172.21.15.26:0/3494366835 wait complete. 2023-12-19T17:57:36.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.539+0000 7f906359e640 1 Processor -- start 2023-12-19T17:57:36.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f906359e640 1 -- start start 2023-12-19T17:57:36.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f906359e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90640b8760 0x7f906414fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:36.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f906359e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90640cdbb0 0x7f90641570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:36.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f906359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906415cca0 0x7f9064157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:36.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f906359e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9064157b40 con 0x7f906415cca0 2023-12-19T17:57:36.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f906359e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90640c74d0 con 0x7f90640cdbb0 2023-12-19T17:57:36.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f906359e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90640c7720 con 0x7f90640b8760 2023-12-19T17:57:36.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f9062d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906415cca0 0x7f9064157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:36.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f9062d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906415cca0 0x7f9064157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46074/0 (socket says 172.21.15.26:46074) 2023-12-19T17:57:36.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f9062d9d640 1 -- 172.21.15.26:0/441076452 learned_addr learned my addr 172.21.15.26:0/441076452 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:36.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f906259c640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90640b8760 0x7f906414fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:36.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f905bfff640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90640cdbb0 0x7f90641570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:36.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f9062d9d640 1 -- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90640b8760 msgr2=0x7f906414fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:36.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f9062d9d640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90640b8760 0x7f906414fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f9062d9d640 1 -- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90640cdbb0 msgr2=0x7f90641570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:36.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f9062d9d640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90640cdbb0 0x7f90641570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f9062d9d640 1 -- 172.21.15.26:0/441076452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c0094d0 con 0x7f906415cca0 2023-12-19T17:57:36.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f9062d9d640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906415cca0 0x7f9064157600 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f905c00cef0 tx=0x7f905c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:36.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f905affd640 1 -- 172.21.15.26:0/441076452 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c00d840 con 0x7f906415cca0 2023-12-19T17:57:36.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f905affd640 1 -- 172.21.15.26:0/441076452 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905c00d9c0 con 0x7f906415cca0 2023-12-19T17:57:36.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f905affd640 1 -- 172.21.15.26:0/441076452 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905c00dd70 con 0x7f906415cca0 2023-12-19T17:57:36.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f906359e640 1 -- 172.21.15.26:0/441076452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90640c7a00 con 0x7f906415cca0 2023-12-19T17:57:36.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f906359e640 1 -- 172.21.15.26:0/441076452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90640c7fc0 con 0x7f906415cca0 2023-12-19T17:57:36.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.543+0000 7f906359e640 1 -- 172.21.15.26:0/441076452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90640c81b0 con 0x7f906415cca0 2023-12-19T17:57:36.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.547+0000 7f905affd640 1 -- 172.21.15.26:0/441076452 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f905c01e3b0 con 0x7f906415cca0 2023-12-19T17:57:36.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.547+0000 7f905affd640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9044065b80 0x7f9044068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:36.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.547+0000 7f905affd640 1 -- 172.21.15.26:0/441076452 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f905c083e00 con 0x7f906415cca0 2023-12-19T17:57:36.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.547+0000 7f906259c640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9044065b80 0x7f9044068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:36.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.547+0000 7f906259c640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9044065b80 0x7f9044068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9064158710 tx=0x7f9054009040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:36.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.551+0000 7f905affd640 1 -- 172.21.15.26:0/441076452 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f905c051740 con 0x7f906415cca0 2023-12-19T17:57:36.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.763+0000 7f906359e640 1 -- 172.21.15.26:0/441076452 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9064067400 con 0x7f906415cca0 2023-12-19T17:57:36.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.763+0000 7f905affd640 1 -- 172.21.15.26:0/441076452 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v41) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f905c01a090 con 0x7f906415cca0 2023-12-19T17:57:36.766 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:36.766 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T17:57:36.766 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2023-12-19T17:57:36.766 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.766 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2023-12-19T17:57:36.766 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg==", 2023-12-19T17:57:36.766 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.766 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:57:36.766 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:57:36.766 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:57:36.766 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.766 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.766 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg==", 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg==", 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.767 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw==", 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.768 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig==", 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA==", 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.769 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ==", 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ==", 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A==", 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:57:36.771 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2023-12-19T17:57:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ==", 2023-12-19T17:57:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:57:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:57:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.772 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2023-12-19T17:57:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg==", 2023-12-19T17:57:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:57:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:57:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg==", 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ==", 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:57:36.774 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ==", 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.775 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ==", 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ==", 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T17:57:36.776 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlT5ynDxAA7Sjbdwrc1RyAXJboysB54Q==", 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.777 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA==", 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg==", 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T17:57:36.778 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:57:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:36.779 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-12-19T17:57:36.779 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T17:57:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.767+0000 7f906359e640 1 -- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9044065b80 msgr2=0x7f9044068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.767+0000 7f906359e640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9044065b80 0x7f9044068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9064158710 tx=0x7f9054009040 comp rx=0 tx=0).stop 2023-12-19T17:57:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.767+0000 7f906359e640 1 -- 172.21.15.26:0/441076452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906415cca0 msgr2=0x7f9064157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.767+0000 7f906359e640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906415cca0 0x7f9064157600 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f905c00cef0 tx=0x7f905c00a710 comp rx=0 tx=0).stop 2023-12-19T17:57:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.767+0000 7f906359e640 1 -- 172.21.15.26:0/441076452 shutdown_connections 2023-12-19T17:57:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.767+0000 7f906359e640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f906415cca0 0x7f9064157600 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.767+0000 7f906359e640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90640cdbb0 0x7f90641570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.767+0000 7f906359e640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9044065b80 0x7f9044068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.767+0000 7f906359e640 1 --2- 172.21.15.26:0/441076452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90640b8760 0x7f906414fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.767+0000 7f906359e640 1 -- 172.21.15.26:0/441076452 >> 172.21.15.26:0/441076452 conn(0x7f906406a120 msgr2=0x7f90640b5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.767+0000 7f906359e640 1 -- 172.21.15.26:0/441076452 shutdown_connections 2023-12-19T17:57:36.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.767+0000 7f906359e640 1 -- 172.21.15.26:0/441076452 wait complete. 2023-12-19T17:57:36.787 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:57:36.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.927+0000 7f06e7df0640 1 Processor -- start 2023-12-19T17:57:36.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.927+0000 7f06e7df0640 1 -- start start 2023-12-19T17:57:36.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e7df0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f06e006bee0 0x7f06e006c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:36.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e7df0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06e00b1d40 0x7f06e00b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:36.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e7df0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e006d4b0 con 0x7f06e00cb1d0 2023-12-19T17:57:36.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e7df0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e00cc030 con 0x7f06e006bee0 2023-12-19T17:57:36.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e7df0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e006cc10 con 0x7f06e00b1d40 2023-12-19T17:57:36.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e638d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06e00b1d40 0x7f06e00b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:36.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e538b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f06e006bee0 0x7f06e006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:36.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e638d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06e00b1d40 0x7f06e00b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:35190/0 (socket says 172.21.15.26:35190) 2023-12-19T17:57:36.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e638d640 1 -- 172.21.15.26:0/1313147541 learned_addr learned my addr 172.21.15.26:0/1313147541 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:36.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e538b640 1 --2- 172.21.15.26:0/1313147541 >> v2:172.21.15.160:3300/0 conn(0x7f06e006bee0 0x7f06e006c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:58314/0 (socket says 172.21.15.26:58314) 2023-12-19T17:57:36.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e638d640 1 -- 172.21.15.26:0/1313147541 >> v2:172.21.15.160:3300/0 conn(0x7f06e006bee0 msgr2=0x7f06e006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:36.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e638d640 1 --2- 172.21.15.26:0/1313147541 >> v2:172.21.15.160:3300/0 conn(0x7f06e006bee0 0x7f06e006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e638d640 1 -- 172.21.15.26:0/1313147541 >> v1:172.21.15.160:6789/0 conn(0x7f06e00cb1d0 legacy=0x7f06e00cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:36.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e638d640 1 -- 172.21.15.26:0/1313147541 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e006cef0 con 0x7f06e00b1d40 2023-12-19T17:57:36.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.931+0000 7f06e538b640 1 --2- 172.21.15.26:0/1313147541 >> v2:172.21.15.160:3300/0 conn(0x7f06e006bee0 0x7f06e006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:36.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e638d640 1 --2- 172.21.15.26:0/1313147541 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06e00b1d40 0x7f06e00b21d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009e50 tx=0x7f06d801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3140c4206151c8d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:36.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e4b8a640 1 -- 172.21.15.26:0/1313147541 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d80a5810 con 0x7f06e00b1d40 2023-12-19T17:57:36.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e4b8a640 1 -- 172.21.15.26:0/1313147541 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d80a5970 con 0x7f06e00b1d40 2023-12-19T17:57:36.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e4b8a640 1 -- 172.21.15.26:0/1313147541 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d80a5d00 con 0x7f06e00b1d40 2023-12-19T17:57:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e7df0640 1 -- 172.21.15.26:0/1313147541 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06e00b1d40 msgr2=0x7f06e00b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e7df0640 1 --2- 172.21.15.26:0/1313147541 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06e00b1d40 0x7f06e00b21d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009e50 tx=0x7f06d801c950 comp rx=0 tx=0).stop 2023-12-19T17:57:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e7df0640 1 -- 172.21.15.26:0/1313147541 shutdown_connections 2023-12-19T17:57:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e7df0640 1 --2- 172.21.15.26:0/1313147541 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06e00b1d40 0x7f06e00b21d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e7df0640 1 --2- 172.21.15.26:0/1313147541 >> v2:172.21.15.160:3300/0 conn(0x7f06e006bee0 0x7f06e006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e7df0640 1 -- 172.21.15.26:0/1313147541 >> 172.21.15.26:0/1313147541 conn(0x7f06e00658e0 msgr2=0x7f06e0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e7df0640 1 -- 172.21.15.26:0/1313147541 shutdown_connections 2023-12-19T17:57:36.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e7df0640 1 -- 172.21.15.26:0/1313147541 wait complete. 2023-12-19T17:57:36.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e7df0640 1 Processor -- start 2023-12-19T17:57:36.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.935+0000 7f06e7df0640 1 -- start start 2023-12-19T17:57:36.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e7df0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06e006bee0 0x7f06e00ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:36.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e7df0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06e00b1d40 0x7f06e00af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:36.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e7df0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e00cb1d0 0x7f06e014fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:36.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e7df0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e0151650 con 0x7f06e00cb1d0 2023-12-19T17:57:36.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e7df0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e0151ae0 con 0x7f06e006bee0 2023-12-19T17:57:36.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e7df0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e0150760 con 0x7f06e00b1d40 2023-12-19T17:57:36.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e638d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e00cb1d0 0x7f06e014fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:36.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e638d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e00cb1d0 0x7f06e014fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46078/0 (socket says 172.21.15.26:46078) 2023-12-19T17:57:36.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e638d640 1 -- 172.21.15.26:0/1111716533 learned_addr learned my addr 172.21.15.26:0/1111716533 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:36.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e5b8c640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06e006bee0 0x7f06e00ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:36.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e538b640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06e00b1d40 0x7f06e00af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e638d640 1 -- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06e00b1d40 msgr2=0x7f06e00af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e638d640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06e00b1d40 0x7f06e00af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e638d640 1 -- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06e006bee0 msgr2=0x7f06e00ce980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e638d640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06e006bee0 0x7f06e00ce980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e638d640 1 -- 172.21.15.26:0/1111716533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d0009420 con 0x7f06e00cb1d0 2023-12-19T17:57:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e638d640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e00cb1d0 0x7f06e014fbc0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f06d801a580 tx=0x7f06d80196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:36.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06d67fc640 1 -- 172.21.15.26:0/1111716533 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d80b8670 con 0x7f06e00cb1d0 2023-12-19T17:57:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06d67fc640 1 -- 172.21.15.26:0/1111716533 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d80b87d0 con 0x7f06e00cb1d0 2023-12-19T17:57:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e7df0640 1 -- 172.21.15.26:0/1111716533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d8009b30 con 0x7f06e00cb1d0 2023-12-19T17:57:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06d67fc640 1 -- 172.21.15.26:0/1111716533 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d80b8b40 con 0x7f06e00cb1d0 2023-12-19T17:57:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e7df0640 1 -- 172.21.15.26:0/1111716533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e0150d20 con 0x7f06e00cb1d0 2023-12-19T17:57:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.939+0000 7f06e7df0640 1 -- 172.21.15.26:0/1111716533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e01517c0 con 0x7f06e00cb1d0 2023-12-19T17:57:36.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.943+0000 7f06d67fc640 1 -- 172.21.15.26:0/1111716533 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f06d80b8ca0 con 0x7f06e00cb1d0 2023-12-19T17:57:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.943+0000 7f06d67fc640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f06c4065b80 0x7f06c4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:36.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.943+0000 7f06d67fc640 1 -- 172.21.15.26:0/1111716533 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f06d811e770 con 0x7f06e00cb1d0 2023-12-19T17:57:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.943+0000 7f06e5b8c640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f06c4065b80 0x7f06c4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:36.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.943+0000 7f06e5b8c640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f06c4065b80 0x7f06c4068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f06dc007fd0 tx=0x7f06dc009040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:36.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:36.947+0000 7f06d67fc640 1 -- 172.21.15.26:0/1111716533 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f06d80ec250 con 0x7f06e00cb1d0 2023-12-19T17:57:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06e7df0640 1 -- 172.21.15.26:0/1111716533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06e0074270 con 0x7f06e00cb1d0 2023-12-19T17:57:37.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06d67fc640 1 -- 172.21.15.26:0/1111716533 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+13504 (secure 0 0 0) 0x7f06d80ef870 con 0x7f06e00cb1d0 2023-12-19T17:57:37.109 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:37.109 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":159,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:57:36.356583+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:57:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06e7df0640 1 -- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f06c4065b80 msgr2=0x7f06c4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06e7df0640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f06c4065b80 0x7f06c4068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f06dc007fd0 tx=0x7f06dc009040 comp rx=0 tx=0).stop 2023-12-19T17:57:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06e7df0640 1 -- 172.21.15.26:0/1111716533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e00cb1d0 msgr2=0x7f06e014fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06e7df0640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e00cb1d0 0x7f06e014fbc0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f06d801a580 tx=0x7f06d80196f0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06e7df0640 1 -- 172.21.15.26:0/1111716533 shutdown_connections 2023-12-19T17:57:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06e7df0640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e00cb1d0 0x7f06e014fbc0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06e7df0640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f06c4065b80 0x7f06c4068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06e7df0640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06e00b1d40 0x7f06e00af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06e7df0640 1 --2- 172.21.15.26:0/1111716533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06e006bee0 0x7f06e00ce980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06e7df0640 1 -- 172.21.15.26:0/1111716533 >> 172.21.15.26:0/1111716533 conn(0x7f06e00658e0 msgr2=0x7f06e0157e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06e7df0640 1 -- 172.21.15.26:0/1111716533 shutdown_connections 2023-12-19T17:57:37.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.107+0000 7f06e7df0640 1 -- 172.21.15.26:0/1111716533 wait complete. 2023-12-19T17:57:37.130 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T17:57:37.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T17:57:37.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09c5f0640 1 Processor -- start 2023-12-19T17:57:37.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09c5f0640 1 -- start start 2023-12-19T17:57:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09c5f0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa09415a9c0 0x7fa0940706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09c5f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa09415ada0 0x7fa0940b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09c5f0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0940b46c0 con 0x7fa09406ac80 2023-12-19T17:57:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09c5f0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0940cc270 con 0x7fa09415a9c0 2023-12-19T17:57:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09c5f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa094070d90 con 0x7fa09415ada0 2023-12-19T17:57:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09a38c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa09415a9c0 0x7fa0940706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09a38c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa09415a9c0 0x7fa0940706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46082/0 (socket says 172.21.15.26:46082) 2023-12-19T17:57:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09a38c640 1 -- 172.21.15.26:0/418259426 learned_addr learned my addr 172.21.15.26:0/418259426 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:37.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa099b8b640 1 --2- 172.21.15.26:0/418259426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa09415ada0 0x7fa0940b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09a38c640 1 -- 172.21.15.26:0/418259426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa09415ada0 msgr2=0x7fa0940b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09a38c640 1 --2- 172.21.15.26:0/418259426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa09415ada0 0x7fa0940b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09a38c640 1 -- 172.21.15.26:0/418259426 >> v1:172.21.15.160:6789/0 conn(0x7fa09406ac80 legacy=0x7fa0940b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.275+0000 7fa09a38c640 1 -- 172.21.15.26:0/418259426 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa094071070 con 0x7fa09415a9c0 2023-12-19T17:57:37.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.279+0000 7fa09a38c640 1 --2- 172.21.15.26:0/418259426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa09415a9c0 0x7fa0940706d0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fa090009870 tx=0x7fa09001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b3d76835398962 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:37.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.279+0000 7fa09938a640 1 -- 172.21.15.26:0/418259426 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0900ab680 con 0x7fa09415a9c0 2023-12-19T17:57:37.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.279+0000 7fa09938a640 1 -- 172.21.15.26:0/418259426 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0900ab7e0 con 0x7fa09415a9c0 2023-12-19T17:57:37.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.279+0000 7fa09938a640 1 -- 172.21.15.26:0/418259426 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0900abb70 con 0x7fa09415a9c0 2023-12-19T17:57:37.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.279+0000 7fa09c5f0640 1 -- 172.21.15.26:0/418259426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa09415a9c0 msgr2=0x7fa0940706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.279+0000 7fa09c5f0640 1 --2- 172.21.15.26:0/418259426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa09415a9c0 0x7fa0940706d0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fa090009870 tx=0x7fa09001c480 comp rx=0 tx=0).stop 2023-12-19T17:57:37.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.279+0000 7fa09c5f0640 1 -- 172.21.15.26:0/418259426 shutdown_connections 2023-12-19T17:57:37.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.279+0000 7fa09c5f0640 1 --2- 172.21.15.26:0/418259426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa09415ada0 0x7fa0940b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.279+0000 7fa09c5f0640 1 --2- 172.21.15.26:0/418259426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa09415a9c0 0x7fa0940706d0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.279+0000 7fa09c5f0640 1 -- 172.21.15.26:0/418259426 >> 172.21.15.26:0/418259426 conn(0x7fa0940ce730 msgr2=0x7fa0940ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:37.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.279+0000 7fa09c5f0640 1 -- 172.21.15.26:0/418259426 shutdown_connections 2023-12-19T17:57:37.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.279+0000 7fa09c5f0640 1 -- 172.21.15.26:0/418259426 wait complete. 2023-12-19T17:57:37.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09c5f0640 1 Processor -- start 2023-12-19T17:57:37.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09c5f0640 1 -- start start 2023-12-19T17:57:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09c5f0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa09406ac80 0x7fa0940b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09c5f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa09415a9c0 0x7fa0940b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09c5f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa09415ada0 0x7fa0940c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09c5f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940b29c0 con 0x7fa09415ada0 2023-12-19T17:57:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09c5f0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940c74e0 con 0x7fa09406ac80 2023-12-19T17:57:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09c5f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940c6220 con 0x7fa09415a9c0 2023-12-19T17:57:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09ab8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa09415ada0 0x7fa0940c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09ab8d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa09415ada0 0x7fa0940c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46098/0 (socket says 172.21.15.26:46098) 2023-12-19T17:57:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09ab8d640 1 -- 172.21.15.26:0/1215074271 learned_addr learned my addr 172.21.15.26:0/1215074271 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:37.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09a38c640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa09406ac80 0x7fa0940b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa099b8b640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa09415a9c0 0x7fa0940b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09ab8d640 1 -- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa09415a9c0 msgr2=0x7fa0940b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09ab8d640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa09415a9c0 0x7fa0940b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09ab8d640 1 -- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa09406ac80 msgr2=0x7fa0940b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09ab8d640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa09406ac80 0x7fa0940b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09ab8d640 1 -- 172.21.15.26:0/1215074271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0900094d0 con 0x7fa09415ada0 2023-12-19T17:57:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09ab8d640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa09415ada0 0x7fa0940c6fa0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00c730 tx=0x7fa08c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa08affd640 1 -- 172.21.15.26:0/1215074271 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c003b80 con 0x7fa09415ada0 2023-12-19T17:57:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa08affd640 1 -- 172.21.15.26:0/1215074271 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c003ce0 con 0x7fa09415ada0 2023-12-19T17:57:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09c5f0640 1 -- 172.21.15.26:0/1215074271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0940c64c0 con 0x7fa09415ada0 2023-12-19T17:57:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa08affd640 1 -- 172.21.15.26:0/1215074271 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c00aa30 con 0x7fa09415ada0 2023-12-19T17:57:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.283+0000 7fa09c5f0640 1 -- 172.21.15.26:0/1215074271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0940b2bd0 con 0x7fa09415ada0 2023-12-19T17:57:37.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.287+0000 7fa09c5f0640 1 -- 172.21.15.26:0/1215074271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0940c68b0 con 0x7fa09415ada0 2023-12-19T17:57:37.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.287+0000 7fa08affd640 1 -- 172.21.15.26:0/1215074271 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa08c00ab90 con 0x7fa09415ada0 2023-12-19T17:57:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.287+0000 7fa08affd640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa078065b80 0x7fa078068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.287+0000 7fa08affd640 1 -- 172.21.15.26:0/1215074271 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fa08c082f90 con 0x7fa09415ada0 2023-12-19T17:57:37.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.287+0000 7fa09a38c640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa078065b80 0x7fa078068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.287+0000 7fa09a38c640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa078065b80 0x7fa078068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa0900af000 tx=0x7fa0900b6040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:37.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.291+0000 7fa08affd640 1 -- 172.21.15.26:0/1215074271 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa08c0508c0 con 0x7fa09415ada0 2023-12-19T17:57:37.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.447+0000 7fa09c5f0640 1 -- 172.21.15.26:0/1215074271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa094074270 con 0x7fa09415ada0 2023-12-19T17:57:37.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.451+0000 7fa08affd640 1 -- 172.21.15.26:0/1215074271 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa08c053ee0 con 0x7fa09415ada0 2023-12-19T17:57:37.452 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T17:57:37.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.451+0000 7fa09c5f0640 1 -- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa078065b80 msgr2=0x7fa078068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.451+0000 7fa09c5f0640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa078065b80 0x7fa078068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa0900af000 tx=0x7fa0900b6040 comp rx=0 tx=0).stop 2023-12-19T17:57:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.451+0000 7fa09c5f0640 1 -- 172.21.15.26:0/1215074271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa09415ada0 msgr2=0x7fa0940c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.451+0000 7fa09c5f0640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa09415ada0 0x7fa0940c6fa0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00c730 tx=0x7fa08c00cbf0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.451+0000 7fa09c5f0640 1 -- 172.21.15.26:0/1215074271 shutdown_connections 2023-12-19T17:57:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.451+0000 7fa09c5f0640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa09415ada0 0x7fa0940c6fa0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.451+0000 7fa09c5f0640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa078065b80 0x7fa078068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.451+0000 7fa09c5f0640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa09415a9c0 0x7fa0940b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.451+0000 7fa09c5f0640 1 --2- 172.21.15.26:0/1215074271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa09406ac80 0x7fa0940b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.451+0000 7fa09c5f0640 1 -- 172.21.15.26:0/1215074271 >> 172.21.15.26:0/1215074271 conn(0x7fa0940ce730 msgr2=0x7fa0940ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.451+0000 7fa09c5f0640 1 -- 172.21.15.26:0/1215074271 shutdown_connections 2023-12-19T17:57:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.451+0000 7fa09c5f0640 1 -- 172.21.15.26:0/1215074271 wait complete. 2023-12-19T17:57:37.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T17:57:37.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0b56df640 1 Processor -- start 2023-12-19T17:57:37.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0b56df640 1 -- start start 2023-12-19T17:57:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0b56df640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0b015ccb0 0x7fb0b015de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0b56df640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb0b00b8340 0x7fb0b00cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0b56df640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b00d1630 con 0x7fb0b00cfa10 2023-12-19T17:57:37.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0b56df640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b00b8ac0 con 0x7fb0b015ccb0 2023-12-19T17:57:37.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0b56df640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b00cdbe0 con 0x7fb0b00b8340 2023-12-19T17:57:37.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0aeffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0b015ccb0 0x7fb0b015de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0aeffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0b015ccb0 0x7fb0b015de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46102/0 (socket says 172.21.15.26:46102) 2023-12-19T17:57:37.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0aeffd640 1 -- 172.21.15.26:0/2017080619 learned_addr learned my addr 172.21.15.26:0/2017080619 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:37.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0ae7fc640 1 --2- 172.21.15.26:0/2017080619 >> v2:172.21.15.160:3300/0 conn(0x7fb0b00b8340 0x7fb0b00cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0aeffd640 1 -- 172.21.15.26:0/2017080619 >> v2:172.21.15.160:3300/0 conn(0x7fb0b00b8340 msgr2=0x7fb0b00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:37.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0aeffd640 1 --2- 172.21.15.26:0/2017080619 >> v2:172.21.15.160:3300/0 conn(0x7fb0b00b8340 0x7fb0b00cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0aeffd640 1 -- 172.21.15.26:0/2017080619 >> v1:172.21.15.26:6789/0 conn(0x7fb0b00cfa10 legacy=0x7fb0b00b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.615+0000 7fb0aeffd640 1 -- 172.21.15.26:0/2017080619 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b00cfdf0 con 0x7fb0b015ccb0 2023-12-19T17:57:37.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.619+0000 7fb0aeffd640 1 --2- 172.21.15.26:0/2017080619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0b015ccb0 0x7fb0b015de70 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb0b00d1eb0 tx=0x7fb0a001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d742e0377227df8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:37.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.619+0000 7fb0adffb640 1 -- 172.21.15.26:0/2017080619 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00ab5f0 con 0x7fb0b015ccb0 2023-12-19T17:57:37.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.619+0000 7fb0adffb640 1 -- 172.21.15.26:0/2017080619 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a00ab750 con 0x7fb0b015ccb0 2023-12-19T17:57:37.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.619+0000 7fb0adffb640 1 -- 172.21.15.26:0/2017080619 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00abae0 con 0x7fb0b015ccb0 2023-12-19T17:57:37.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.619+0000 7fb0b56df640 1 -- 172.21.15.26:0/2017080619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0b015ccb0 msgr2=0x7fb0b015de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.619+0000 7fb0b56df640 1 --2- 172.21.15.26:0/2017080619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0b015ccb0 0x7fb0b015de70 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb0b00d1eb0 tx=0x7fb0a001c4d0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.619+0000 7fb0b56df640 1 -- 172.21.15.26:0/2017080619 shutdown_connections 2023-12-19T17:57:37.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.619+0000 7fb0b56df640 1 --2- 172.21.15.26:0/2017080619 >> v2:172.21.15.160:3300/0 conn(0x7fb0b00b8340 0x7fb0b00cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.619+0000 7fb0b56df640 1 --2- 172.21.15.26:0/2017080619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0b015ccb0 0x7fb0b015de70 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.619+0000 7fb0b56df640 1 -- 172.21.15.26:0/2017080619 >> 172.21.15.26:0/2017080619 conn(0x7fb0b00658e0 msgr2=0x7fb0b0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:37.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.619+0000 7fb0b56df640 1 -- 172.21.15.26:0/2017080619 shutdown_connections 2023-12-19T17:57:37.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.619+0000 7fb0b56df640 1 -- 172.21.15.26:0/2017080619 wait complete. 2023-12-19T17:57:37.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0b56df640 1 Processor -- start 2023-12-19T17:57:37.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0b56df640 1 -- start start 2023-12-19T17:57:37.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0b56df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b00b8340 0x7fb0b015b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0b56df640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0b00cfa10 0x7fb0b014ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0b56df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0b015ccb0 0x7fb0b0154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0b56df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b015a340 con 0x7fb0b015ccb0 2023-12-19T17:57:37.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0b56df640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b014f4a0 con 0x7fb0b00cfa10 2023-12-19T17:57:37.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0b56df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b015a530 con 0x7fb0b00b8340 2023-12-19T17:57:37.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0af7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0b015ccb0 0x7fb0b0154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0af7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0b015ccb0 0x7fb0b0154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46108/0 (socket says 172.21.15.26:46108) 2023-12-19T17:57:37.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0af7fe640 1 -- 172.21.15.26:0/3128451697 learned_addr learned my addr 172.21.15.26:0/3128451697 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:37.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0ae7fc640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0b00cfa10 0x7fb0b014ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0aeffd640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b00b8340 0x7fb0b015b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0af7fe640 1 -- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b00b8340 msgr2=0x7fb0b015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0af7fe640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b00b8340 0x7fb0b015b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0af7fe640 1 -- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0b00cfa10 msgr2=0x7fb0b014ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0af7fe640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0b00cfa10 0x7fb0b014ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0af7fe640 1 -- 172.21.15.26:0/3128451697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a00094d0 con 0x7fb0b015ccb0 2023-12-19T17:57:37.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0af7fe640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0b015ccb0 0x7fb0b0154470 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c7a0 tx=0x7fb0a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:37.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0977fe640 1 -- 172.21.15.26:0/3128451697 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8003b80 con 0x7fb0b015ccb0 2023-12-19T17:57:37.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0977fe640 1 -- 172.21.15.26:0/3128451697 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a8003ce0 con 0x7fb0b015ccb0 2023-12-19T17:57:37.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0977fe640 1 -- 172.21.15.26:0/3128451697 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800aa90 con 0x7fb0b015ccb0 2023-12-19T17:57:37.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0b56df640 1 -- 172.21.15.26:0/3128451697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b015a810 con 0x7fb0b015ccb0 2023-12-19T17:57:37.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0b56df640 1 -- 172.21.15.26:0/3128451697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b015add0 con 0x7fb0b015ccb0 2023-12-19T17:57:37.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.623+0000 7fb0b56df640 1 -- 172.21.15.26:0/3128451697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b004e850 con 0x7fb0b015ccb0 2023-12-19T17:57:37.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.627+0000 7fb0977fe640 1 -- 172.21.15.26:0/3128451697 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb0a800abf0 con 0x7fb0b015ccb0 2023-12-19T17:57:37.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.627+0000 7fb0977fe640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb09c065b80 0x7fb09c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.627+0000 7fb0977fe640 1 -- 172.21.15.26:0/3128451697 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fb0a8082fb0 con 0x7fb0b015ccb0 2023-12-19T17:57:37.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.627+0000 7fb0aeffd640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb09c065b80 0x7fb09c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.627+0000 7fb0aeffd640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb09c065b80 0x7fb09c068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00af000 tx=0x7fb0a00b6040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:37.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.631+0000 7fb0977fe640 1 -- 172.21.15.26:0/3128451697 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb0a8050970 con 0x7fb0b015ccb0 2023-12-19T17:57:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0b56df640 1 -- 172.21.15.26:0/3128451697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb0b0152c40 con 0x7fb0b015ccb0 2023-12-19T17:57:37.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0977fe640 1 -- 172.21.15.26:0/3128451697 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v159) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb0a8053f90 con 0x7fb0b015ccb0 2023-12-19T17:57:37.792 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:37.793 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T17:57:37.793 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:37.793 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T17:57:37.793 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T17:57:37.793 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:37.793 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T17:57:37.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0b56df640 1 -- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb09c065b80 msgr2=0x7fb09c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0b56df640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb09c065b80 0x7fb09c068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb0a00af000 tx=0x7fb0a00b6040 comp rx=0 tx=0).stop 2023-12-19T17:57:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0b56df640 1 -- 172.21.15.26:0/3128451697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0b015ccb0 msgr2=0x7fb0b0154470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0b56df640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0b015ccb0 0x7fb0b0154470 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c7a0 tx=0x7fb0a800cc60 comp rx=0 tx=0).stop 2023-12-19T17:57:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0b56df640 1 -- 172.21.15.26:0/3128451697 shutdown_connections 2023-12-19T17:57:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0b56df640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0b015ccb0 0x7fb0b0154470 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0b56df640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0b00cfa10 0x7fb0b014ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0b56df640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fb09c065b80 0x7fb09c068040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0b56df640 1 --2- 172.21.15.26:0/3128451697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb0b00b8340 0x7fb0b015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0b56df640 1 -- 172.21.15.26:0/3128451697 >> 172.21.15.26:0/3128451697 conn(0x7fb0b00658e0 msgr2=0x7fb0b015bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0b56df640 1 -- 172.21.15.26:0/3128451697 shutdown_connections 2023-12-19T17:57:37.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.791+0000 7fb0b56df640 1 -- 172.21.15.26:0/3128451697 wait complete. 2023-12-19T17:57:37.814 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T17:57:37.814 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T17:57:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.959+0000 7f9a20d39640 1 Processor -- start 2023-12-19T17:57:37.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.959+0000 7f9a20d39640 1 -- start start 2023-12-19T17:57:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.963+0000 7f9a20d39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a1c0b1590 0x7f9a1c06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.963+0000 7f9a20d39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a1c0b3f90 0x7f9a1c0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.963+0000 7f9a20d39640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1c06cbf0 con 0x7f9a1c15aad0 2023-12-19T17:57:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.963+0000 7f9a20d39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a1c0ca8b0 con 0x7f9a1c0b3f90 2023-12-19T17:57:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.963+0000 7f9a20d39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c06b6f0 con 0x7f9a1c0b1590 2023-12-19T17:57:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.963+0000 7f9a1ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a1c0b3f90 0x7f9a1c0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.963+0000 7f9a1ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a1c0b3f90 0x7f9a1c0cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46116/0 (socket says 172.21.15.26:46116) 2023-12-19T17:57:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.963+0000 7f9a1ad9d640 1 -- 172.21.15.26:0/1256044351 learned_addr learned my addr 172.21.15.26:0/1256044351 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.963+0000 7f9a13fff640 1 --2- 172.21.15.26:0/1256044351 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a1c0b1590 0x7f9a1c06c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.963+0000 7f9a1ad9d640 1 -- 172.21.15.26:0/1256044351 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a1c0b1590 msgr2=0x7f9a1c06c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.963+0000 7f9a1ad9d640 1 --2- 172.21.15.26:0/1256044351 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a1c0b1590 0x7f9a1c06c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.963+0000 7f9a1ad9d640 1 -- 172.21.15.26:0/1256044351 >> v1:172.21.15.26:6789/0 conn(0x7f9a1c15aad0 legacy=0x7f9a1c070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.963+0000 7f9a1ad9d640 1 -- 172.21.15.26:0/1256044351 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c06b9d0 con 0x7f9a1c0b3f90 2023-12-19T17:57:37.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a1ad9d640 1 --2- 172.21.15.26:0/1256044351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1c0b3f90 0x7f9a1c0cad40 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009e50 tx=0x7f9a1401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce63dd68c36062b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a19d9b640 1 -- 172.21.15.26:0/1256044351 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a140a57c0 con 0x7f9a1c0b3f90 2023-12-19T17:57:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a19d9b640 1 -- 172.21.15.26:0/1256044351 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a140a5920 con 0x7f9a1c0b3f90 2023-12-19T17:57:37.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a19d9b640 1 -- 172.21.15.26:0/1256044351 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a140a5cb0 con 0x7f9a1c0b3f90 2023-12-19T17:57:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a20d39640 1 -- 172.21.15.26:0/1256044351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1c0b3f90 msgr2=0x7f9a1c0cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a20d39640 1 --2- 172.21.15.26:0/1256044351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1c0b3f90 0x7f9a1c0cad40 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009e50 tx=0x7f9a1401c9e0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a20d39640 1 -- 172.21.15.26:0/1256044351 shutdown_connections 2023-12-19T17:57:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a20d39640 1 --2- 172.21.15.26:0/1256044351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1c0b3f90 0x7f9a1c0cad40 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a20d39640 1 --2- 172.21.15.26:0/1256044351 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a1c0b1590 0x7f9a1c06c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a20d39640 1 -- 172.21.15.26:0/1256044351 >> 172.21.15.26:0/1256044351 conn(0x7f9a1c0b0270 msgr2=0x7f9a1c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a20d39640 1 -- 172.21.15.26:0/1256044351 shutdown_connections 2023-12-19T17:57:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a20d39640 1 -- 172.21.15.26:0/1256044351 wait complete. 2023-12-19T17:57:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a20d39640 1 Processor -- start 2023-12-19T17:57:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.967+0000 7f9a20d39640 1 -- start start 2023-12-19T17:57:37.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a20d39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1c0b1590 0x7f9a1c14fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a20d39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a1c0b3f90 0x7f9a1c0c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a20d39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a1c15aad0 0x7f9a1c0c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a20d39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c0c2f60 con 0x7f9a1c0b1590 2023-12-19T17:57:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a20d39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c150220 con 0x7f9a1c0b3f90 2023-12-19T17:57:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a20d39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c0c39b0 con 0x7f9a1c15aad0 2023-12-19T17:57:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a1a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1c0b1590 0x7f9a1c14fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a1a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1c0b1590 0x7f9a1c14fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46124/0 (socket says 172.21.15.26:46124) 2023-12-19T17:57:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a1a59c640 1 -- 172.21.15.26:0/3759341491 learned_addr learned my addr 172.21.15.26:0/3759341491 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a13fff640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a1c0b3f90 0x7f9a1c0c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a1ad9d640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a1c15aad0 0x7f9a1c0c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a1a59c640 1 -- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a1c15aad0 msgr2=0x7f9a1c0c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a1a59c640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a1c15aad0 0x7f9a1c0c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a1a59c640 1 -- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a1c0b3f90 msgr2=0x7f9a1c0c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a1a59c640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a1c0b3f90 0x7f9a1c0c42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a1a59c640 1 -- 172.21.15.26:0/3759341491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a14009b30 con 0x7f9a1c0b1590 2023-12-19T17:57:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a13fff640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a1c0b3f90 0x7f9a1c0c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a1a59c640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1c0b1590 0x7f9a1c14fce0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00c420 tx=0x7f9a0c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a12ffd640 1 -- 172.21.15.26:0/3759341491 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c0052d0 con 0x7f9a1c0b1590 2023-12-19T17:57:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a12ffd640 1 -- 172.21.15.26:0/3759341491 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c005430 con 0x7f9a1c0b1590 2023-12-19T17:57:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a12ffd640 1 -- 172.21.15.26:0/3759341491 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c003840 con 0x7f9a1c0b1590 2023-12-19T17:57:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a20d39640 1 -- 172.21.15.26:0/3759341491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c0b2ed0 con 0x7f9a1c0b1590 2023-12-19T17:57:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a20d39640 1 -- 172.21.15.26:0/3759341491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c0c31b0 con 0x7f9a1c0b1590 2023-12-19T17:57:37.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a20d39640 1 -- 172.21.15.26:0/3759341491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c074270 con 0x7f9a1c0b1590 2023-12-19T17:57:37.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.971+0000 7f9a12ffd640 1 -- 172.21.15.26:0/3759341491 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9a0c0039a0 con 0x7f9a1c0b1590 2023-12-19T17:57:37.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.975+0000 7f9a12ffd640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f99fc065b80 0x7f99fc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:37.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.975+0000 7f9a12ffd640 1 -- 172.21.15.26:0/3759341491 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f9a0c082640 con 0x7f9a1c0b1590 2023-12-19T17:57:37.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.975+0000 7f9a13fff640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f99fc065b80 0x7f99fc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:37.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.975+0000 7f9a13fff640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f99fc065b80 0x7f99fc068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c155740 tx=0x7f9a04008040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:37.979+0000 7f9a12ffd640 1 -- 172.21.15.26:0/3759341491 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9a0c04ff50 con 0x7f9a1c0b1590 2023-12-19T17:57:38.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:38.135+0000 7f9a20d39640 1 -- 172.21.15.26:0/3759341491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9a1c06cbf0 con 0x7f9a1c0b1590 2023-12-19T17:57:38.162 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:57:38.165 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:57:39.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.023+0000 7f9a12ffd640 1 -- 172.21.15.26:0/3759341491 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v160) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9a1c06cbf0 con 0x7f9a1c0b1590 2023-12-19T17:57:39.027 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T17:57:39.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.031+0000 7f9a20d39640 1 -- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f99fc065b80 msgr2=0x7f99fc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:39.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.031+0000 7f9a20d39640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f99fc065b80 0x7f99fc068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c155740 tx=0x7f9a04008040 comp rx=0 tx=0).stop 2023-12-19T17:57:39.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.031+0000 7f9a20d39640 1 -- 172.21.15.26:0/3759341491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1c0b1590 msgr2=0x7f9a1c14fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:39.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.031+0000 7f9a20d39640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1c0b1590 0x7f9a1c14fce0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00c420 tx=0x7f9a0c00c8e0 comp rx=0 tx=0).stop 2023-12-19T17:57:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.031+0000 7f9a20d39640 1 -- 172.21.15.26:0/3759341491 shutdown_connections 2023-12-19T17:57:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.031+0000 7f9a20d39640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f99fc065b80 0x7f99fc068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.031+0000 7f9a20d39640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a1c15aad0 0x7f9a1c0c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.031+0000 7f9a20d39640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a1c0b3f90 0x7f9a1c0c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.031+0000 7f9a20d39640 1 --2- 172.21.15.26:0/3759341491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1c0b1590 0x7f9a1c14fce0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.031+0000 7f9a20d39640 1 -- 172.21.15.26:0/3759341491 >> 172.21.15.26:0/3759341491 conn(0x7f9a1c0b0270 msgr2=0x7f9a1c0af310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.031+0000 7f9a20d39640 1 -- 172.21.15.26:0/3759341491 shutdown_connections 2023-12-19T17:57:39.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.031+0000 7f9a20d39640 1 -- 172.21.15.26:0/3759341491 wait complete. 2023-12-19T17:57:39.059 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T17:57:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed55a41640 1 Processor -- start 2023-12-19T17:57:39.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed55a41640 1 -- start start 2023-12-19T17:57:39.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed55a41640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fed5006cb20 0x7fed5006f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:39.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed55a41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed5015a3c0 0x7fed5015a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed55a41640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5006f790 con 0x7fed5006c740 2023-12-19T17:57:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed55a41640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5015d5a0 con 0x7fed5006cb20 2023-12-19T17:57:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed55a41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5015d880 con 0x7fed5015a3c0 2023-12-19T17:57:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fed5006c740 0x7fed500ca800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35160/0 (socket says 172.21.15.26:35160) 2023-12-19T17:57:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4effd640 1 -- 172.21.15.26:0/3838016115 learned_addr learned my addr 172.21.15.26:0/3838016115 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed47fff640 1 --2- 172.21.15.26:0/3838016115 >> v2:172.21.15.160:3300/0 conn(0x7fed5006cb20 0x7fed5006f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4f7fe640 1 --2- 172.21.15.26:0/3838016115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed5015a3c0 0x7fed5015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477977309 0 0) 0x7fed5006f790 con 0x7fed5006c740 2023-12-19T17:57:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed3c003530 con 0x7fed5006c740 2023-12-19T17:57:39.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2362365566 0 0) 0x7fed3c003530 con 0x7fed5006c740 2023-12-19T17:57:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed5015a3c0 msgr2=0x7fed5015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:39.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 --2- 172.21.15.26:0/3838016115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed5015a3c0 0x7fed5015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 >> v2:172.21.15.160:3300/0 conn(0x7fed5006cb20 msgr2=0x7fed5006f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 --2- 172.21.15.26:0/3838016115 >> v2:172.21.15.160:3300/0 conn(0x7fed5006cb20 0x7fed5006f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5015db60 con 0x7fed5006c740 2023-12-19T17:57:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fed40001680 con 0x7fed5006c740 2023-12-19T17:57:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 >> v1:172.21.15.26:6789/0 conn(0x7fed5006c740 legacy=0x7fed500ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 --2- 172.21.15.26:0/3838016115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed3c003980 0x7fed3c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 --2- 172.21.15.26:0/3838016115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed5015a3c0 0x7fed3c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 --2- 172.21.15.26:0/3838016115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed3c004d10 0x7fed3c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5015d5a0 con 0x7fed3c003980 2023-12-19T17:57:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5015d880 con 0x7fed3c004d10 2023-12-19T17:57:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c00b990 con 0x7fed5015a3c0 2023-12-19T17:57:39.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.211+0000 7fed4effd640 1 --2- 172.21.15.26:0/3838016115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed3c003980 0x7fed3c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.215+0000 7fed47fff640 1 --2- 172.21.15.26:0/3838016115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed5015a3c0 0x7fed3c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.215+0000 7fed4f7fe640 1 --2- 172.21.15.26:0/3838016115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed3c004d10 0x7fed3c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.215+0000 7fed4effd640 1 -- 172.21.15.26:0/3838016115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed5015a3c0 msgr2=0x7fed3c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.215+0000 7fed4effd640 1 --2- 172.21.15.26:0/3838016115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed5015a3c0 0x7fed3c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.215+0000 7fed4effd640 1 -- 172.21.15.26:0/3838016115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed3c004d10 msgr2=0x7fed3c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.215+0000 7fed4effd640 1 --2- 172.21.15.26:0/3838016115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed3c004d10 0x7fed3c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:39.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.215+0000 7fed4effd640 1 -- 172.21.15.26:0/3838016115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c003530 con 0x7fed3c003980 2023-12-19T17:57:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.215+0000 7fed4effd640 1 --2- 172.21.15.26:0/3838016115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed3c003980 0x7fed3c003da0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fed4000c240 tx=0x7fed4001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb07c3f129865769 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.215+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed400a5910 con 0x7fed3c003980 2023-12-19T17:57:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.215+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed400a5a70 con 0x7fed3c003980 2023-12-19T17:57:39.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.215+0000 7fed4e7fc640 1 -- 172.21.15.26:0/3838016115 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed400a5e40 con 0x7fed3c003980 2023-12-19T17:57:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 -- 172.21.15.26:0/3838016115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed3c003980 msgr2=0x7fed3c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 --2- 172.21.15.26:0/3838016115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed3c003980 0x7fed3c003da0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fed4000c240 tx=0x7fed4001cc30 comp rx=0 tx=0).stop 2023-12-19T17:57:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed4f7fe640 1 -- 172.21.15.26:0/3838016115 reap_dead start 2023-12-19T17:57:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 -- 172.21.15.26:0/3838016115 shutdown_connections 2023-12-19T17:57:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 -- 172.21.15.26:0/3838016115 >> 172.21.15.26:0/3838016115 conn(0x7fed500660d0 msgr2=0x7fed500ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 -- 172.21.15.26:0/3838016115 shutdown_connections 2023-12-19T17:57:39.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 -- 172.21.15.26:0/3838016115 wait complete. 2023-12-19T17:57:39.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 Processor -- start 2023-12-19T17:57:39.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 -- start start 2023-12-19T17:57:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed3c003980 0x7fed50149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed500c19c0 0x7fed50158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed501552e0 0x7fed50155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed50155ca0 con 0x7fed3c003980 2023-12-19T17:57:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed501579e0 con 0x7fed500c19c0 2023-12-19T17:57:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed55a41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed50157cc0 con 0x7fed501552e0 2023-12-19T17:57:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed4effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed3c003980 0x7fed50149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed4effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed3c003980 0x7fed50149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46138/0 (socket says 172.21.15.26:46138) 2023-12-19T17:57:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed4effd640 1 -- 172.21.15.26:0/3978527324 learned_addr learned my addr 172.21.15.26:0/3978527324 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed4f7fe640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed501552e0 0x7fed50155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:39.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed47fff640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed500c19c0 0x7fed50158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed4effd640 1 -- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed501552e0 msgr2=0x7fed50155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed4effd640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed501552e0 0x7fed50155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed4effd640 1 -- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed500c19c0 msgr2=0x7fed50158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed4effd640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed500c19c0 0x7fed50158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed4effd640 1 -- 172.21.15.26:0/3978527324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed50157fa0 con 0x7fed3c003980 2023-12-19T17:57:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed4f7fe640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed501552e0 0x7fed50155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.219+0000 7fed47fff640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed500c19c0 0x7fed50158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.223+0000 7fed4effd640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed3c003980 0x7fed50149700 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fed4000c240 tx=0x7fed3c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.223+0000 7fed4cff9640 1 -- 172.21.15.26:0/3978527324 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40004ab0 con 0x7fed3c003980 2023-12-19T17:57:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.223+0000 7fed4cff9640 1 -- 172.21.15.26:0/3978527324 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed400b2070 con 0x7fed3c003980 2023-12-19T17:57:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.223+0000 7fed4cff9640 1 -- 172.21.15.26:0/3978527324 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed40004e70 con 0x7fed3c003980 2023-12-19T17:57:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.223+0000 7fed55a41640 1 -- 172.21.15.26:0/3978527324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed50158280 con 0x7fed3c003980 2023-12-19T17:57:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.223+0000 7fed55a41640 1 -- 172.21.15.26:0/3978527324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed50158840 con 0x7fed3c003980 2023-12-19T17:57:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.223+0000 7fed55a41640 1 -- 172.21.15.26:0/3978527324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed50074270 con 0x7fed3c003980 2023-12-19T17:57:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.223+0000 7fed4cff9640 1 -- 172.21.15.26:0/3978527324 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fed40007070 con 0x7fed3c003980 2023-12-19T17:57:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.223+0000 7fed4cff9640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed30065b70 0x7fed30068030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.223+0000 7fed4cff9640 1 -- 172.21.15.26:0/3978527324 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5959+0+0 (secure 0 0 0) 0x7fed4011d340 con 0x7fed3c003980 2023-12-19T17:57:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.223+0000 7fed47fff640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed30065b70 0x7fed30068030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:39.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.227+0000 7fed47fff640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed30065b70 0x7fed30068030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fed500b0010 tx=0x7fed38008040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:39.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.227+0000 7fed4cff9640 1 -- 172.21.15.26:0/3978527324 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fed400eabd0 con 0x7fed3c003980 2023-12-19T17:57:39.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:39.387+0000 7fed55a41640 1 -- 172.21.15.26:0/3978527324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fed5015d310 con 0x7fed3c003980 2023-12-19T17:57:40.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.039+0000 7fed4cff9640 1 -- 172.21.15.26:0/3978527324 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v161) v1 ==== 170+0+0 (secure 0 0 0) 0x7fed400b4020 con 0x7fed3c003980 2023-12-19T17:57:40.159 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T17:57:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.043+0000 7fed55a41640 1 -- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed30065b70 msgr2=0x7fed30068030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.043+0000 7fed55a41640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed30065b70 0x7fed30068030 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fed500b0010 tx=0x7fed38008040 comp rx=0 tx=0).stop 2023-12-19T17:57:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.043+0000 7fed55a41640 1 -- 172.21.15.26:0/3978527324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed3c003980 msgr2=0x7fed50149700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.043+0000 7fed55a41640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed3c003980 0x7fed50149700 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fed4000c240 tx=0x7fed3c005800 comp rx=0 tx=0).stop 2023-12-19T17:57:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.043+0000 7fed55a41640 1 -- 172.21.15.26:0/3978527324 shutdown_connections 2023-12-19T17:57:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.043+0000 7fed55a41640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed30065b70 0x7fed30068030 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fed500b0010 tx=0x7fed38008040 comp rx=0 tx=0).stop 2023-12-19T17:57:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.043+0000 7fed55a41640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed501552e0 0x7fed50155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.043+0000 7fed55a41640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed500c19c0 0x7fed50158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.043+0000 7fed55a41640 1 --2- 172.21.15.26:0/3978527324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed3c003980 0x7fed50149700 secure :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fed4000c240 tx=0x7fed3c005800 comp rx=0 tx=0).stop 2023-12-19T17:57:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.043+0000 7fed55a41640 1 -- 172.21.15.26:0/3978527324 >> 172.21.15.26:0/3978527324 conn(0x7fed500660d0 msgr2=0x7fed500cc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:40.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.047+0000 7fed55a41640 1 -- 172.21.15.26:0/3978527324 shutdown_connections 2023-12-19T17:57:40.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.047+0000 7fed55a41640 1 -- 172.21.15.26:0/3978527324 wait complete. 2023-12-19T17:57:40.164 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T17:57:40.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.307+0000 7f42c18fc640 1 Processor -- start 2023-12-19T17:57:40.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.307+0000 7f42c18fc640 1 -- start start 2023-12-19T17:57:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.307+0000 7f42c18fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42bc0ad960 0x7f42bc0addc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.307+0000 7f42c18fc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42bc0ad4d0 con 0x7f42bc15cdc0 2023-12-19T17:57:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.307+0000 7f42c18fc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42bc15d1a0 con 0x7f42bc0cad70 2023-12-19T17:57:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.307+0000 7f42c18fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0ac900 con 0x7f42bc0ad960 2023-12-19T17:57:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42baffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f42bc15cdc0 0x7f42bc06bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35172/0 (socket says 172.21.15.26:35172) 2023-12-19T17:57:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42baffd640 1 -- 172.21.15.26:0/2429091978 learned_addr learned my addr 172.21.15.26:0/2429091978 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42ba7fc640 1 --2- 172.21.15.26:0/2429091978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42bc0ad960 0x7f42bc0addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571628688 0 0) 0x7f42bc0ad4d0 con 0x7f42bc15cdc0 2023-12-19T17:57:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42a4003530 con 0x7f42bc15cdc0 2023-12-19T17:57:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 243644672 0 0) 0x7f42a4003530 con 0x7f42bc15cdc0 2023-12-19T17:57:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42bc0ad960 msgr2=0x7f42bc0addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 --2- 172.21.15.26:0/2429091978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42bc0ad960 0x7f42bc0addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 >> v1:172.21.15.160:6789/0 conn(0x7f42bc0cad70 legacy=0x7f42bc0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc0acba0 con 0x7f42bc15cdc0 2023-12-19T17:57:40.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f42ac001680 con 0x7f42bc15cdc0 2023-12-19T17:57:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 >> v1:172.21.15.26:6789/0 conn(0x7f42bc15cdc0 legacy=0x7f42bc06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 --2- 172.21.15.26:0/2429091978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42bc0ad960 0x7f42a40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 --2- 172.21.15.26:0/2429091978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4003f30 0x7f42a40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 --2- 172.21.15.26:0/2429091978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a4004d10 0x7f42a4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0ac900 con 0x7f42a4004d10 2023-12-19T17:57:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a400ba70 con 0x7f42a4003f30 2023-12-19T17:57:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a400bc30 con 0x7f42bc0ad960 2023-12-19T17:57:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42bb7fe640 1 --2- 172.21.15.26:0/2429091978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a4004d10 0x7f42a4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42baffd640 1 --2- 172.21.15.26:0/2429091978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42bc0ad960 0x7f42a40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42ba7fc640 1 --2- 172.21.15.26:0/2429091978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4003f30 0x7f42a40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42bb7fe640 1 -- 172.21.15.26:0/2429091978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42bc0ad960 msgr2=0x7f42a40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42bb7fe640 1 --2- 172.21.15.26:0/2429091978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42bc0ad960 0x7f42a40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42bb7fe640 1 -- 172.21.15.26:0/2429091978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4003f30 msgr2=0x7f42a40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42bb7fe640 1 --2- 172.21.15.26:0/2429091978 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42a4003f30 0x7f42a40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.311+0000 7f42bb7fe640 1 -- 172.21.15.26:0/2429091978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a4003530 con 0x7f42a4004d10 2023-12-19T17:57:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.315+0000 7f42bb7fe640 1 --2- 172.21.15.26:0/2429091978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a4004d10 0x7f42a4005190 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f42b4001e70 tx=0x7f42b401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934b076178796950 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.315+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4003b80 con 0x7f42a4004d10 2023-12-19T17:57:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.315+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b4003ce0 con 0x7f42a4004d10 2023-12-19T17:57:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.315+0000 7f42b9ffb640 1 -- 172.21.15.26:0/2429091978 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b40052c0 con 0x7f42a4004d10 2023-12-19T17:57:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.315+0000 7f42c18fc640 1 -- 172.21.15.26:0/2429091978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a4004d10 msgr2=0x7f42a4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.315+0000 7f42c18fc640 1 --2- 172.21.15.26:0/2429091978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a4004d10 0x7f42a4005190 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f42b4001e70 tx=0x7f42b401c760 comp rx=0 tx=0).stop 2023-12-19T17:57:40.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.315+0000 7f42bb7fe640 1 -- 172.21.15.26:0/2429091978 reap_dead start 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.315+0000 7f42c18fc640 1 -- 172.21.15.26:0/2429091978 shutdown_connections 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.315+0000 7f42c18fc640 1 -- 172.21.15.26:0/2429091978 >> 172.21.15.26:0/2429091978 conn(0x7f42bc0b0b60 msgr2=0x7f42bc0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.315+0000 7f42c18fc640 1 -- 172.21.15.26:0/2429091978 shutdown_connections 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.315+0000 7f42c18fc640 1 -- 172.21.15.26:0/2429091978 wait complete. 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.315+0000 7f42c18fc640 1 Processor -- start 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.315+0000 7f42c18fc640 1 -- start start 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42c18fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42a4004d10 0x7f42bc154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42c18fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42bc0c60c0 0x7f42bc0c6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42c18fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42bc0c8570 0x7f42bc0c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42c18fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0c6e20 con 0x7f42bc0c8570 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42c18fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0c8f30 con 0x7f42bc0c60c0 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42c18fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0c6b40 con 0x7f42a4004d10 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42bb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42bc0c8570 0x7f42bc0c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42bb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42bc0c8570 0x7f42bc0c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46158/0 (socket says 172.21.15.26:46158) 2023-12-19T17:57:40.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42bb7fe640 1 -- 172.21.15.26:0/834991702 learned_addr learned my addr 172.21.15.26:0/834991702 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42ba7fc640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42bc0c60c0 0x7f42bc0c6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42baffd640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42a4004d10 0x7f42bc154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42bb7fe640 1 -- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42a4004d10 msgr2=0x7f42bc154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42bb7fe640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42a4004d10 0x7f42bc154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42bb7fe640 1 -- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42bc0c60c0 msgr2=0x7f42bc0c6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42bb7fe640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42bc0c60c0 0x7f42bc0c6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42bb7fe640 1 -- 172.21.15.26:0/834991702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc06fb90 con 0x7f42bc0c8570 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42ba7fc640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42bc0c60c0 0x7f42bc0c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42bb7fe640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42bc0c8570 0x7f42bc0c89f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f42b4098170 tx=0x7f42a4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42a3fff640 1 -- 172.21.15.26:0/834991702 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4006d60 con 0x7f42bc0c8570 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42a3fff640 1 -- 172.21.15.26:0/834991702 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b4007dc0 con 0x7f42bc0c8570 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42a3fff640 1 -- 172.21.15.26:0/834991702 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4007230 con 0x7f42bc0c8570 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42c18fc640 1 -- 172.21.15.26:0/834991702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc06fe70 con 0x7f42bc0c8570 2023-12-19T17:57:40.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42c18fc640 1 -- 172.21.15.26:0/834991702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc147d70 con 0x7f42bc0c8570 2023-12-19T17:57:40.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.319+0000 7f42c18fc640 1 -- 172.21.15.26:0/834991702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42bc0c6e20 con 0x7f42bc0c8570 2023-12-19T17:57:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.323+0000 7f42a3fff640 1 -- 172.21.15.26:0/834991702 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f42b4007390 con 0x7f42bc0c8570 2023-12-19T17:57:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.323+0000 7f42a3fff640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f42a8065b80 0x7f42a8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.323+0000 7f42a3fff640 1 -- 172.21.15.26:0/834991702 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f42b411d2d0 con 0x7f42bc0c8570 2023-12-19T17:57:40.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.323+0000 7f42baffd640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f42a8065b80 0x7f42a8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.323+0000 7f42baffd640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f42a8065b80 0x7f42a8068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42ac000c10 tx=0x7f42ac001180 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:40.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.327+0000 7f42a3fff640 1 -- 172.21.15.26:0/834991702 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f42b40ea990 con 0x7f42bc0c8570 2023-12-19T17:57:40.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.511+0000 7f42c18fc640 1 -- 172.21.15.26:0/834991702 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f42bc0c6e20 con 0x7f42bc0c8570 2023-12-19T17:57:40.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.523+0000 7f42a3fff640 1 -- 172.21.15.26:0/834991702 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v69) v1 ==== 174+0+0 (secure 0 0 0) 0x7f42bc0c6e20 con 0x7f42bc0c8570 2023-12-19T17:57:40.526 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 18 and data pool 19 2023-12-19T17:57:40.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.527+0000 7f42c18fc640 1 -- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f42a8065b80 msgr2=0x7f42a8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.527+0000 7f42c18fc640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f42a8065b80 0x7f42a8068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f42ac000c10 tx=0x7f42ac001180 comp rx=0 tx=0).stop 2023-12-19T17:57:40.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.527+0000 7f42c18fc640 1 -- 172.21.15.26:0/834991702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42bc0c8570 msgr2=0x7f42bc0c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.527+0000 7f42c18fc640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42bc0c8570 0x7f42bc0c89f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f42b4098170 tx=0x7f42a4005800 comp rx=0 tx=0).stop 2023-12-19T17:57:40.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.527+0000 7f42c18fc640 1 -- 172.21.15.26:0/834991702 shutdown_connections 2023-12-19T17:57:40.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.527+0000 7f42c18fc640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42bc0c8570 0x7f42bc0c89f0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.527+0000 7f42c18fc640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42bc0c60c0 0x7f42bc0c6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.527+0000 7f42c18fc640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f42a8065b80 0x7f42a8068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.527+0000 7f42c18fc640 1 --2- 172.21.15.26:0/834991702 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42a4004d10 0x7f42bc154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.527+0000 7f42c18fc640 1 -- 172.21.15.26:0/834991702 >> 172.21.15.26:0/834991702 conn(0x7f42bc0b0b60 msgr2=0x7f42bc155530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:40.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.531+0000 7f42c18fc640 1 -- 172.21.15.26:0/834991702 shutdown_connections 2023-12-19T17:57:40.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.531+0000 7f42c18fc640 1 -- 172.21.15.26:0/834991702 wait complete. 2023-12-19T17:57:40.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:57:40.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7fedaabcb640 1 Processor -- start 2023-12-19T17:57:40.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7fedaabcb640 1 -- start start 2023-12-19T17:57:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7fedaabcb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feda415cca0 0x7feda40b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7fedaabcb640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7feda40b8760 0x7feda40cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7fedaabcb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feda40b6b20 0x7feda40cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7fedaabcb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda40b7c30 con 0x7feda415cca0 2023-12-19T17:57:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7fedaabcb640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda40b6fb0 con 0x7feda40b8760 2023-12-19T17:57:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7fedaabcb640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda40b7290 con 0x7feda40b6b20 2023-12-19T17:57:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7feda9bc9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feda415cca0 0x7feda40b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7feda9bc9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feda415cca0 0x7feda40b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46160/0 (socket says 172.21.15.26:46160) 2023-12-19T17:57:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7feda9bc9640 1 -- 172.21.15.26:0/744429862 learned_addr learned my addr 172.21.15.26:0/744429862 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7feda93c8640 1 --2- 172.21.15.26:0/744429862 >> v2:172.21.15.160:3300/0 conn(0x7feda40b8760 0x7feda40cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7fedaa3ca640 1 --2- 172.21.15.26:0/744429862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feda40b6b20 0x7feda40cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7feda9bc9640 1 -- 172.21.15.26:0/744429862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feda40b6b20 msgr2=0x7feda40cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7feda9bc9640 1 --2- 172.21.15.26:0/744429862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feda40b6b20 0x7feda40cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7feda9bc9640 1 -- 172.21.15.26:0/744429862 >> v2:172.21.15.160:3300/0 conn(0x7feda40b8760 msgr2=0x7feda40cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:40.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7feda9bc9640 1 --2- 172.21.15.26:0/744429862 >> v2:172.21.15.160:3300/0 conn(0x7feda40b8760 0x7feda40cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7feda9bc9640 1 -- 172.21.15.26:0/744429862 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda40b7570 con 0x7feda415cca0 2023-12-19T17:57:40.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.715+0000 7fedaa3ca640 1 --2- 172.21.15.26:0/744429862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feda40b6b20 0x7feda40cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:40.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7feda9bc9640 1 --2- 172.21.15.26:0/744429862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda415cca0 0x7feda40b65e0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7feda40b8050 tx=0x7feda001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd94feb62b6c25f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:40.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7feda8bc7640 1 -- 172.21.15.26:0/744429862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda00aa610 con 0x7feda415cca0 2023-12-19T17:57:40.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7feda8bc7640 1 -- 172.21.15.26:0/744429862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda00a6040 con 0x7feda415cca0 2023-12-19T17:57:40.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7feda8bc7640 1 -- 172.21.15.26:0/744429862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda00aa990 con 0x7feda415cca0 2023-12-19T17:57:40.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7fedaabcb640 1 -- 172.21.15.26:0/744429862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda415cca0 msgr2=0x7feda40b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7fedaabcb640 1 --2- 172.21.15.26:0/744429862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda415cca0 0x7feda40b65e0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7feda40b8050 tx=0x7feda001c4d0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7fedaabcb640 1 -- 172.21.15.26:0/744429862 shutdown_connections 2023-12-19T17:57:40.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7fedaabcb640 1 --2- 172.21.15.26:0/744429862 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feda40b6b20 0x7feda40cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7fedaabcb640 1 --2- 172.21.15.26:0/744429862 >> v2:172.21.15.160:3300/0 conn(0x7feda40b8760 0x7feda40cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7fedaabcb640 1 --2- 172.21.15.26:0/744429862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda415cca0 0x7feda40b65e0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7fedaabcb640 1 -- 172.21.15.26:0/744429862 >> 172.21.15.26:0/744429862 conn(0x7feda406a120 msgr2=0x7feda406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:40.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7fedaabcb640 1 -- 172.21.15.26:0/744429862 shutdown_connections 2023-12-19T17:57:40.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7fedaabcb640 1 -- 172.21.15.26:0/744429862 wait complete. 2023-12-19T17:57:40.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.719+0000 7fedaabcb640 1 Processor -- start 2023-12-19T17:57:40.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaabcb640 1 -- start start 2023-12-19T17:57:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaabcb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feda40b8760 0x7feda415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaabcb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feda415cca0 0x7feda4149950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaabcb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda414b750 0x7feda40d1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaabcb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda40cc640 con 0x7feda414b750 2023-12-19T17:57:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaabcb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda40cca60 con 0x7feda40b8760 2023-12-19T17:57:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaabcb640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda40ca730 con 0x7feda415cca0 2023-12-19T17:57:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaa3ca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda414b750 0x7feda40d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaa3ca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda414b750 0x7feda40d1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46166/0 (socket says 172.21.15.26:46166) 2023-12-19T17:57:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaa3ca640 1 -- 172.21.15.26:0/1992148053 learned_addr learned my addr 172.21.15.26:0/1992148053 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7feda93c8640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feda415cca0 0x7feda4149950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7feda9bc9640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feda40b8760 0x7feda415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaa3ca640 1 -- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feda415cca0 msgr2=0x7feda4149950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaa3ca640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feda415cca0 0x7feda4149950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaa3ca640 1 -- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feda40b8760 msgr2=0x7feda415b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaa3ca640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feda40b8760 0x7feda415b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaa3ca640 1 -- 172.21.15.26:0/1992148053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda00094d0 con 0x7feda414b750 2023-12-19T17:57:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaa3ca640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda414b750 0x7feda40d1810 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fed9c007e10 tx=0x7fed9c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fed9a7fc640 1 -- 172.21.15.26:0/1992148053 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c003ba0 con 0x7feda414b750 2023-12-19T17:57:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fed9a7fc640 1 -- 172.21.15.26:0/1992148053 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c00f040 con 0x7feda414b750 2023-12-19T17:57:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fed9a7fc640 1 -- 172.21.15.26:0/1992148053 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c00de80 con 0x7feda414b750 2023-12-19T17:57:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaabcb640 1 -- 172.21.15.26:0/1992148053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda415bd80 con 0x7feda414b750 2023-12-19T17:57:40.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaabcb640 1 -- 172.21.15.26:0/1992148053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda40cbf80 con 0x7feda414b750 2023-12-19T17:57:40.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.723+0000 7fedaabcb640 1 -- 172.21.15.26:0/1992148053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda4067400 con 0x7feda414b750 2023-12-19T17:57:40.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.727+0000 7fed9a7fc640 1 -- 172.21.15.26:0/1992148053 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fed9c004030 con 0x7feda414b750 2023-12-19T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.727+0000 7fed9a7fc640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed88065b80 0x7fed88068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.727+0000 7fed9a7fc640 1 -- 172.21.15.26:0/1992148053 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fed9c083da0 con 0x7feda414b750 2023-12-19T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.727+0000 7feda9bc9640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed88065b80 0x7fed88068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.727+0000 7feda9bc9640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed88065b80 0x7fed88068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7feda00ae000 tx=0x7feda00b5040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.731+0000 7fed9a7fc640 1 -- 172.21.15.26:0/1992148053 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fed9c051480 con 0x7feda414b750 2023-12-19T17:57:40.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.887+0000 7fedaabcb640 1 -- 172.21.15.26:0/1992148053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feda40c6ba0 con 0x7feda414b750 2023-12-19T17:57:40.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.887+0000 7fed9a7fc640 1 -- 172.21.15.26:0/1992148053 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+17229 (secure 0 0 0) 0x7fed9c01a090 con 0x7feda414b750 2023-12-19T17:57:40.890 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:40.890 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":162,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:57:40.517375+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:57:38.188932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T17:57:39.438347+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:57:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.891+0000 7fedaabcb640 1 -- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed88065b80 msgr2=0x7fed88068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.891+0000 7fedaabcb640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed88065b80 0x7fed88068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7feda00ae000 tx=0x7feda00b5040 comp rx=0 tx=0).stop 2023-12-19T17:57:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.891+0000 7fedaabcb640 1 -- 172.21.15.26:0/1992148053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda414b750 msgr2=0x7feda40d1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:40.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.891+0000 7fedaabcb640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda414b750 0x7feda40d1810 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fed9c007e10 tx=0x7fed9c00d4a0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.891+0000 7fedaabcb640 1 -- 172.21.15.26:0/1992148053 shutdown_connections 2023-12-19T17:57:40.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.891+0000 7fedaabcb640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feda414b750 0x7feda40d1810 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.891+0000 7fedaabcb640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fed88065b80 0x7fed88068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.891+0000 7fedaabcb640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feda415cca0 0x7feda4149950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.891+0000 7fedaabcb640 1 --2- 172.21.15.26:0/1992148053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feda40b8760 0x7feda415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:40.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.891+0000 7fedaabcb640 1 -- 172.21.15.26:0/1992148053 >> 172.21.15.26:0/1992148053 conn(0x7feda406a120 msgr2=0x7feda40c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:40.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.891+0000 7fedaabcb640 1 -- 172.21.15.26:0/1992148053 shutdown_connections 2023-12-19T17:57:40.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:40.891+0000 7fedaabcb640 1 -- 172.21.15.26:0/1992148053 wait complete. 2023-12-19T17:57:40.912 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:57:41.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb275db640 1 Processor -- start 2023-12-19T17:57:41.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb275db640 1 -- start start 2023-12-19T17:57:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb275db640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7feb2006bee0 0x7feb2006c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb275db640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2006d4b0 con 0x7feb200b1d40 2023-12-19T17:57:41.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb275db640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2006cc10 con 0x7feb200cb1d0 2023-12-19T17:57:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb275db640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb200cc030 con 0x7feb2006bee0 2023-12-19T17:57:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb25b78640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feb200b1d40 0x7feb200b21d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35180/0 (socket says 172.21.15.26:35180) 2023-12-19T17:57:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb25b78640 1 -- 172.21.15.26:0/3634273472 learned_addr learned my addr 172.21.15.26:0/3634273472 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:41.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb24b76640 1 --2- 172.21.15.26:0/3634273472 >> v2:172.21.15.160:3300/0 conn(0x7feb2006bee0 0x7feb2006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb17fff640 1 -- 172.21.15.26:0/3634273472 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253080009 0 0) 0x7feb2006d4b0 con 0x7feb200b1d40 2023-12-19T17:57:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb17fff640 1 -- 172.21.15.26:0/3634273472 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb0c003530 con 0x7feb200b1d40 2023-12-19T17:57:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb24b76640 1 -- 172.21.15.26:0/3634273472 >> v1:172.21.15.160:6789/0 conn(0x7feb200cb1d0 legacy=0x7feb200cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb24b76640 1 -- 172.21.15.26:0/3634273472 >> v1:172.21.15.26:6789/0 conn(0x7feb200b1d40 legacy=0x7feb200b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:41.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.055+0000 7feb24b76640 1 -- 172.21.15.26:0/3634273472 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2006cef0 con 0x7feb2006bee0 2023-12-19T17:57:41.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.059+0000 7feb24b76640 1 --2- 172.21.15.26:0/3634273472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb2006bee0 0x7feb2006c340 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7feb100097c0 tx=0x7feb1001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cf05fc89e804ce9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.059+0000 7feb17fff640 1 -- 172.21.15.26:0/3634273472 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100ac620 con 0x7feb2006bee0 2023-12-19T17:57:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.059+0000 7feb17fff640 1 -- 172.21.15.26:0/3634273472 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb100ac780 con 0x7feb2006bee0 2023-12-19T17:57:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.059+0000 7feb17fff640 1 -- 172.21.15.26:0/3634273472 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100acaf0 con 0x7feb2006bee0 2023-12-19T17:57:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.059+0000 7feb275db640 1 -- 172.21.15.26:0/3634273472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb2006bee0 msgr2=0x7feb2006c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:41.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.059+0000 7feb275db640 1 --2- 172.21.15.26:0/3634273472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb2006bee0 0x7feb2006c340 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7feb100097c0 tx=0x7feb1001c3c0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.059+0000 7feb275db640 1 -- 172.21.15.26:0/3634273472 shutdown_connections 2023-12-19T17:57:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.059+0000 7feb275db640 1 --2- 172.21.15.26:0/3634273472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb2006bee0 0x7feb2006c340 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.059+0000 7feb275db640 1 -- 172.21.15.26:0/3634273472 >> 172.21.15.26:0/3634273472 conn(0x7feb200658e0 msgr2=0x7feb20065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.059+0000 7feb275db640 1 -- 172.21.15.26:0/3634273472 shutdown_connections 2023-12-19T17:57:41.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.059+0000 7feb275db640 1 -- 172.21.15.26:0/3634273472 wait complete. 2023-12-19T17:57:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb275db640 1 Processor -- start 2023-12-19T17:57:41.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb275db640 1 -- start start 2023-12-19T17:57:41.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb275db640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb2006bee0 0x7feb2014dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:41.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb275db640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb200b1d40 0x7feb20147e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:41.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb275db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb200cb1d0 0x7feb20148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:41.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb275db640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2015a9e0 con 0x7feb200cb1d0 2023-12-19T17:57:41.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb275db640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2015a3b0 con 0x7feb200b1d40 2023-12-19T17:57:41.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb275db640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb20148370 con 0x7feb2006bee0 2023-12-19T17:57:41.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb25b78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb200cb1d0 0x7feb20148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb25b78640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb200cb1d0 0x7feb20148750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46170/0 (socket says 172.21.15.26:46170) 2023-12-19T17:57:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb25b78640 1 -- 172.21.15.26:0/952974916 learned_addr learned my addr 172.21.15.26:0/952974916 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb24b76640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb200b1d40 0x7feb20147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb25377640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb2006bee0 0x7feb2014dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb25b78640 1 -- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb2006bee0 msgr2=0x7feb2014dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb25b78640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb2006bee0 0x7feb2014dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb25b78640 1 -- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb200b1d40 msgr2=0x7feb20147e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb25b78640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb200b1d40 0x7feb20147e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb25b78640 1 -- 172.21.15.26:0/952974916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb10009420 con 0x7feb200cb1d0 2023-12-19T17:57:41.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb25377640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb2006bee0 0x7feb2014dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:41.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb25b78640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb200cb1d0 0x7feb20148750 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7feb1800cb40 tx=0x7feb1800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:41.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb15ffb640 1 -- 172.21.15.26:0/952974916 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1800aca0 con 0x7feb200cb1d0 2023-12-19T17:57:41.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb15ffb640 1 -- 172.21.15.26:0/952974916 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1800ae00 con 0x7feb200cb1d0 2023-12-19T17:57:41.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb15ffb640 1 -- 172.21.15.26:0/952974916 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18012620 con 0x7feb200cb1d0 2023-12-19T17:57:41.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb275db640 1 -- 172.21.15.26:0/952974916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2014e6b0 con 0x7feb200cb1d0 2023-12-19T17:57:41.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.063+0000 7feb275db640 1 -- 172.21.15.26:0/952974916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2015a600 con 0x7feb200cb1d0 2023-12-19T17:57:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.067+0000 7feb275db640 1 -- 172.21.15.26:0/952974916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2015ab20 con 0x7feb200cb1d0 2023-12-19T17:57:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.067+0000 7feb15ffb640 1 -- 172.21.15.26:0/952974916 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7feb18007070 con 0x7feb200cb1d0 2023-12-19T17:57:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.067+0000 7feb15ffb640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feb04065b80 0x7feb04068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.067+0000 7feb15ffb640 1 -- 172.21.15.26:0/952974916 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6441+0+0 (secure 0 0 0) 0x7feb18083500 con 0x7feb200cb1d0 2023-12-19T17:57:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.067+0000 7feb25377640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feb04065b80 0x7feb04068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:41.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.067+0000 7feb25377640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feb04065b80 0x7feb04068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feb1c000f70 tx=0x7feb1c009040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:41.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.071+0000 7feb15ffb640 1 -- 172.21.15.26:0/952974916 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feb2015ab20 con 0x7feb200cb1d0 2023-12-19T17:57:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.227+0000 7feb275db640 1 -- 172.21.15.26:0/952974916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb2015ab20 con 0x7feb200cb1d0 2023-12-19T17:57:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.231+0000 7feb15ffb640 1 -- 172.21.15.26:0/952974916 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+17229 (secure 0 0 0) 0x7feb2015ab20 con 0x7feb200cb1d0 2023-12-19T17:57:41.232 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:41.232 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":162,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:57:40.517375+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:57:38.188932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T17:57:39.438347+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:57:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.231+0000 7feb275db640 1 -- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feb04065b80 msgr2=0x7feb04068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.231+0000 7feb275db640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feb04065b80 0x7feb04068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feb1c000f70 tx=0x7feb1c009040 comp rx=0 tx=0).stop 2023-12-19T17:57:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.231+0000 7feb275db640 1 -- 172.21.15.26:0/952974916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb200cb1d0 msgr2=0x7feb20148750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.231+0000 7feb275db640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb200cb1d0 0x7feb20148750 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7feb1800cb40 tx=0x7feb1800a060 comp rx=0 tx=0).stop 2023-12-19T17:57:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.231+0000 7feb275db640 1 -- 172.21.15.26:0/952974916 shutdown_connections 2023-12-19T17:57:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.231+0000 7feb275db640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb200cb1d0 0x7feb20148750 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.231+0000 7feb275db640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb200b1d40 0x7feb20147e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.231+0000 7feb275db640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feb04065b80 0x7feb04068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.231+0000 7feb275db640 1 --2- 172.21.15.26:0/952974916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb2006bee0 0x7feb2014dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.231+0000 7feb275db640 1 -- 172.21.15.26:0/952974916 >> 172.21.15.26:0/952974916 conn(0x7feb200658e0 msgr2=0x7feb20065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.231+0000 7feb275db640 1 -- 172.21.15.26:0/952974916 shutdown_connections 2023-12-19T17:57:41.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.231+0000 7feb275db640 1 -- 172.21.15.26:0/952974916 wait complete. 2023-12-19T17:57:41.254 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T17:57:41.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c3598640 1 Processor -- start 2023-12-19T17:57:41.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c3598640 1 -- start start 2023-12-19T17:57:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c3598640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa7bc06f6c0 0x7fa7bc15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c3598640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7bc159c00 0x7fa7bc070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c3598640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc06fb60 con 0x7fa7bc159820 2023-12-19T17:57:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c3598640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7bc15e470 con 0x7fa7bc159c00 2023-12-19T17:57:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c3598640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7bc06fe40 con 0x7fa7bc06f6c0 2023-12-19T17:57:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c1b35640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7bc159c00 0x7fa7bc070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c1b35640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7bc159c00 0x7fa7bc070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46178/0 (socket says 172.21.15.26:46178) 2023-12-19T17:57:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c1b35640 1 -- 172.21.15.26:0/2720435467 learned_addr learned my addr 172.21.15.26:0/2720435467 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c0b33640 1 --2- 172.21.15.26:0/2720435467 >> v2:172.21.15.160:3300/0 conn(0x7fa7bc06f6c0 0x7fa7bc15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c1b35640 1 -- 172.21.15.26:0/2720435467 >> v2:172.21.15.160:3300/0 conn(0x7fa7bc06f6c0 msgr2=0x7fa7bc15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c1b35640 1 --2- 172.21.15.26:0/2720435467 >> v2:172.21.15.160:3300/0 conn(0x7fa7bc06f6c0 0x7fa7bc15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c1b35640 1 -- 172.21.15.26:0/2720435467 >> v1:172.21.15.26:6789/0 conn(0x7fa7bc159820 legacy=0x7fa7bc0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c1b35640 1 -- 172.21.15.26:0/2720435467 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc070120 con 0x7fa7bc159c00 2023-12-19T17:57:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.399+0000 7fa7c0b33640 1 --2- 172.21.15.26:0/2720435467 >> v2:172.21.15.160:3300/0 conn(0x7fa7bc06f6c0 0x7fa7bc15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:57:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.403+0000 7fa7c1b35640 1 --2- 172.21.15.26:0/2720435467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc159c00 0x7fa7bc070c00 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0caf30 tx=0x7fa7b401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c26efff9013cbb8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.403+0000 7fa7b3fff640 1 -- 172.21.15.26:0/2720435467 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40a5770 con 0x7fa7bc159c00 2023-12-19T17:57:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.403+0000 7fa7b3fff640 1 -- 172.21.15.26:0/2720435467 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b40a58d0 con 0x7fa7bc159c00 2023-12-19T17:57:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.403+0000 7fa7b3fff640 1 -- 172.21.15.26:0/2720435467 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40a5c60 con 0x7fa7bc159c00 2023-12-19T17:57:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.403+0000 7fa7c3598640 1 -- 172.21.15.26:0/2720435467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc159c00 msgr2=0x7fa7bc070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.403+0000 7fa7c3598640 1 --2- 172.21.15.26:0/2720435467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc159c00 0x7fa7bc070c00 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0caf30 tx=0x7fa7b401c9a0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.403+0000 7fa7c3598640 1 -- 172.21.15.26:0/2720435467 shutdown_connections 2023-12-19T17:57:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.403+0000 7fa7c3598640 1 --2- 172.21.15.26:0/2720435467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc159c00 0x7fa7bc070c00 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.403+0000 7fa7c3598640 1 --2- 172.21.15.26:0/2720435467 >> v2:172.21.15.160:3300/0 conn(0x7fa7bc06f6c0 0x7fa7bc15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.403+0000 7fa7c3598640 1 -- 172.21.15.26:0/2720435467 >> 172.21.15.26:0/2720435467 conn(0x7fa7bc0ce730 msgr2=0x7fa7bc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.403+0000 7fa7c3598640 1 -- 172.21.15.26:0/2720435467 shutdown_connections 2023-12-19T17:57:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.403+0000 7fa7c3598640 1 -- 172.21.15.26:0/2720435467 wait complete. 2023-12-19T17:57:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.403+0000 7fa7c3598640 1 Processor -- start 2023-12-19T17:57:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c3598640 1 -- start start 2023-12-19T17:57:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c3598640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc06f6c0 0x7fa7bc150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c3598640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7bc159820 0x7fa7bc150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c3598640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7bc159c00 0x7fa7bc0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c3598640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0c32a0 con 0x7fa7bc06f6c0 2023-12-19T17:57:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c3598640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0c2e10 con 0x7fa7bc159c00 2023-12-19T17:57:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c3598640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0c2ae0 con 0x7fa7bc159820 2023-12-19T17:57:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c1334640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc06f6c0 0x7fa7bc150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c1334640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc06f6c0 0x7fa7bc150500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46194/0 (socket says 172.21.15.26:46194) 2023-12-19T17:57:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c1334640 1 -- 172.21.15.26:0/2093487697 learned_addr learned my addr 172.21.15.26:0/2093487697 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c0b33640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7bc159820 0x7fa7bc150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c1b35640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7bc159c00 0x7fa7bc0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c1334640 1 -- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7bc159820 msgr2=0x7fa7bc150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c1334640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7bc159820 0x7fa7bc150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c1334640 1 -- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7bc159c00 msgr2=0x7fa7bc0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c1334640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7bc159c00 0x7fa7bc0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c1334640 1 -- 172.21.15.26:0/2093487697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b4009b30 con 0x7fa7bc06f6c0 2023-12-19T17:57:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c1334640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc06f6c0 0x7fa7bc150500 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800c500 tx=0x7fa7b800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7b1ffb640 1 -- 172.21.15.26:0/2093487697 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8005220 con 0x7fa7bc06f6c0 2023-12-19T17:57:41.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7b1ffb640 1 -- 172.21.15.26:0/2093487697 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b8005380 con 0x7fa7bc06f6c0 2023-12-19T17:57:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7b1ffb640 1 -- 172.21.15.26:0/2093487697 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8007270 con 0x7fa7bc06f6c0 2023-12-19T17:57:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c3598640 1 -- 172.21.15.26:0/2093487697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc06b880 con 0x7fa7bc06f6c0 2023-12-19T17:57:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c3598640 1 -- 172.21.15.26:0/2093487697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc0c2650 con 0x7fa7bc06f6c0 2023-12-19T17:57:41.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.407+0000 7fa7c3598640 1 -- 172.21.15.26:0/2093487697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc074270 con 0x7fa7bc06f6c0 2023-12-19T17:57:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.411+0000 7fa7b1ffb640 1 -- 172.21.15.26:0/2093487697 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa7b80073d0 con 0x7fa7bc06f6c0 2023-12-19T17:57:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.411+0000 7fa7b1ffb640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa7a0065bb0 0x7fa7a0068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.411+0000 7fa7b1ffb640 1 -- 172.21.15.26:0/2093487697 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fa7b8082e00 con 0x7fa7bc06f6c0 2023-12-19T17:57:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.411+0000 7fa7c0b33640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa7a0065bb0 0x7fa7a0068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.411+0000 7fa7c0b33640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa7a0065bb0 0x7fa7a0068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc150490 tx=0x7fa7ac008040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.415+0000 7fa7b1ffb640 1 -- 172.21.15.26:0/2093487697 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa7b80503f0 con 0x7fa7bc06f6c0 2023-12-19T17:57:41.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:41.607+0000 7fa7c3598640 1 -- 172.21.15.26:0/2093487697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa7bc158e80 con 0x7fa7bc06f6c0 2023-12-19T17:57:42.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.535+0000 7fa7b1ffb640 1 -- 172.21.15.26:0/2093487697 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v72) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa7b8053a10 con 0x7fa7bc06f6c0 2023-12-19T17:57:42.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.539+0000 7fa7c3598640 1 -- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa7a0065bb0 msgr2=0x7fa7a0068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:42.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.539+0000 7fa7c3598640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa7a0065bb0 0x7fa7a0068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc150490 tx=0x7fa7ac008040 comp rx=0 tx=0).stop 2023-12-19T17:57:42.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.539+0000 7fa7c3598640 1 -- 172.21.15.26:0/2093487697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc06f6c0 msgr2=0x7fa7bc150500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:42.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.539+0000 7fa7c3598640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc06f6c0 0x7fa7bc150500 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800c500 tx=0x7fa7b800c9c0 comp rx=0 tx=0).stop 2023-12-19T17:57:42.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.539+0000 7fa7c3598640 1 -- 172.21.15.26:0/2093487697 shutdown_connections 2023-12-19T17:57:42.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.539+0000 7fa7c3598640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa7bc159c00 0x7fa7bc0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:42.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.539+0000 7fa7c3598640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa7a0065bb0 0x7fa7a0068070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:42.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.539+0000 7fa7c3598640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa7bc159820 0x7fa7bc150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:42.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.539+0000 7fa7c3598640 1 --2- 172.21.15.26:0/2093487697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc06f6c0 0x7fa7bc150500 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:42.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.539+0000 7fa7c3598640 1 -- 172.21.15.26:0/2093487697 >> 172.21.15.26:0/2093487697 conn(0x7fa7bc0ce730 msgr2=0x7fa7bc0cb890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:42.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.539+0000 7fa7c3598640 1 -- 172.21.15.26:0/2093487697 shutdown_connections 2023-12-19T17:57:42.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.539+0000 7fa7c3598640 1 -- 172.21.15.26:0/2093487697 wait complete. 2023-12-19T17:57:42.564 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T17:57:42.565 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T17:57:42.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.715+0000 7f01b9996640 1 Processor -- start 2023-12-19T17:57:42.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.715+0000 7f01b9996640 1 -- start start 2023-12-19T17:57:42.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.715+0000 7f01b9996640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01b40cd3a0 0x7f01b406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.715+0000 7f01b9996640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b40ca360 con 0x7f01b40ad7f0 2023-12-19T17:57:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.715+0000 7f01b9996640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b40c9750 con 0x7f01b406db00 2023-12-19T17:57:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.715+0000 7f01b9996640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b40cee80 con 0x7f01b40cd3a0 2023-12-19T17:57:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.715+0000 7f01b27fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01b40cd3a0 0x7f01b406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.715+0000 7f01b27fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01b40cd3a0 0x7f01b406bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46204/0 (socket says 172.21.15.26:46204) 2023-12-19T17:57:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.715+0000 7f01b37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f01b40ad7f0 0x7f01b40adc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35192/0 (socket says 172.21.15.26:35192) 2023-12-19T17:57:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.715+0000 7f01b27fc640 1 -- 172.21.15.26:0/61152134 learned_addr learned my addr 172.21.15.26:0/61152134 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.715+0000 7f01b27fc640 1 -- 172.21.15.26:0/61152134 >> v1:172.21.15.160:6789/0 conn(0x7f01b406db00 legacy=0x7f01b40ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.715+0000 7f01b27fc640 1 -- 172.21.15.26:0/61152134 >> v1:172.21.15.26:6789/0 conn(0x7f01b40ad7f0 legacy=0x7f01b40adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:42.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.715+0000 7f01b27fc640 1 -- 172.21.15.26:0/61152134 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b40c9a30 con 0x7f01b40cd3a0 2023-12-19T17:57:42.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.719+0000 7f01b27fc640 1 --2- 172.21.15.26:0/61152134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b40cd3a0 0x7f01b406bcf0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009870 tx=0x7f01a801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f272b85cc99f168 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.719+0000 7f01b1ffb640 1 -- 172.21.15.26:0/61152134 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a80aa610 con 0x7f01b40cd3a0 2023-12-19T17:57:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.719+0000 7f01b1ffb640 1 -- 172.21.15.26:0/61152134 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a80a6040 con 0x7f01b40cd3a0 2023-12-19T17:57:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.719+0000 7f01b1ffb640 1 -- 172.21.15.26:0/61152134 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a80aa990 con 0x7f01b40cd3a0 2023-12-19T17:57:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.719+0000 7f01b9996640 1 -- 172.21.15.26:0/61152134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b40cd3a0 msgr2=0x7f01b406bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.719+0000 7f01b9996640 1 --2- 172.21.15.26:0/61152134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b40cd3a0 0x7f01b406bcf0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f01a8009870 tx=0x7f01a801c4b0 comp rx=0 tx=0).stop 2023-12-19T17:57:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.719+0000 7f01b9996640 1 -- 172.21.15.26:0/61152134 shutdown_connections 2023-12-19T17:57:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.719+0000 7f01b9996640 1 --2- 172.21.15.26:0/61152134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b40cd3a0 0x7f01b406bcf0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.719+0000 7f01b9996640 1 -- 172.21.15.26:0/61152134 >> 172.21.15.26:0/61152134 conn(0x7f01b40b0b60 msgr2=0x7f01b40b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.719+0000 7f01b9996640 1 -- 172.21.15.26:0/61152134 shutdown_connections 2023-12-19T17:57:42.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.719+0000 7f01b9996640 1 -- 172.21.15.26:0/61152134 wait complete. 2023-12-19T17:57:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.719+0000 7f01b9996640 1 Processor -- start 2023-12-19T17:57:42.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b9996640 1 -- start start 2023-12-19T17:57:42.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b9996640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b406db00 0x7f01b415c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b9996640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01b40ad7f0 0x7f01b40b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b9996640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01b40cd3a0 0x7f01b415f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b9996640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b415d490 con 0x7f01b406db00 2023-12-19T17:57:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b9996640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b4154390 con 0x7f01b40ad7f0 2023-12-19T17:57:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b9996640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40b3f70 con 0x7f01b40cd3a0 2023-12-19T17:57:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b406db00 0x7f01b415c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b406db00 0x7f01b415c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46218/0 (socket says 172.21.15.26:46218) 2023-12-19T17:57:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b2ffd640 1 -- 172.21.15.26:0/4209651105 learned_addr learned my addr 172.21.15.26:0/4209651105 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b27fc640 1 --2- 172.21.15.26:0/4209651105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01b40ad7f0 0x7f01b40b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b2ffd640 1 -- 172.21.15.26:0/4209651105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01b40cd3a0 msgr2=0x7f01b415f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b2ffd640 1 --2- 172.21.15.26:0/4209651105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01b40cd3a0 0x7f01b415f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b2ffd640 1 -- 172.21.15.26:0/4209651105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01b40ad7f0 msgr2=0x7f01b40b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:42.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b2ffd640 1 --2- 172.21.15.26:0/4209651105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01b40ad7f0 0x7f01b40b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:42.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b2ffd640 1 -- 172.21.15.26:0/4209651105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80094d0 con 0x7f01b406db00 2023-12-19T17:57:42.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b2ffd640 1 --2- 172.21.15.26:0/4209651105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b406db00 0x7f01b415c780 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f01a400c3c0 tx=0x7f01a400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:42.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f019bfff640 1 -- 172.21.15.26:0/4209651105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a40052d0 con 0x7f01b406db00 2023-12-19T17:57:42.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f019bfff640 1 -- 172.21.15.26:0/4209651105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a4005430 con 0x7f01b406db00 2023-12-19T17:57:42.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b9996640 1 -- 172.21.15.26:0/4209651105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b40b4250 con 0x7f01b406db00 2023-12-19T17:57:42.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f019bfff640 1 -- 172.21.15.26:0/4209651105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4003840 con 0x7f01b406db00 2023-12-19T17:57:42.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b9996640 1 -- 172.21.15.26:0/4209651105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b40b4810 con 0x7f01b406db00 2023-12-19T17:57:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.723+0000 7f01b9996640 1 -- 172.21.15.26:0/4209651105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b40735b0 con 0x7f01b406db00 2023-12-19T17:57:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.727+0000 7f019bfff640 1 -- 172.21.15.26:0/4209651105 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f01a40039a0 con 0x7f01b406db00 2023-12-19T17:57:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.727+0000 7f019bfff640 1 --2- 172.21.15.26:0/4209651105 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01a0065b80 0x7f01a0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.727+0000 7f019bfff640 1 -- 172.21.15.26:0/4209651105 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f01a4082b40 con 0x7f01b406db00 2023-12-19T17:57:42.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.727+0000 7f01b27fc640 1 --2- 172.21.15.26:0/4209651105 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01a0065b80 0x7f01a0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.727+0000 7f01b27fc640 1 --2- 172.21.15.26:0/4209651105 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01a0065b80 0x7f01a0068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f01a80ae000 tx=0x7f01a80a7000 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.731+0000 7f019bfff640 1 -- 172.21.15.26:0/4209651105 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f01a4050100 con 0x7f01b406db00 2023-12-19T17:57:42.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:42.919+0000 7f01b9996640 1 -- 172.21.15.26:0/4209651105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f01b415d5d0 con 0x7f01b406db00 2023-12-19T17:57:43.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.543+0000 7f019bfff640 1 -- 172.21.15.26:0/4209651105 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v73) v1 ==== 122+0+0 (secure 0 0 0) 0x7f01b415d5d0 con 0x7f01b406db00 2023-12-19T17:57:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.547+0000 7f01b9996640 1 -- 172.21.15.26:0/4209651105 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01a0065b80 msgr2=0x7f01a0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.547+0000 7f01b9996640 1 --2- 172.21.15.26:0/4209651105 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01a0065b80 0x7f01a0068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f01a80ae000 tx=0x7f01a80a7000 comp rx=0 tx=0).stop 2023-12-19T17:57:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.547+0000 7f01b9996640 1 -- 172.21.15.26:0/4209651105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b406db00 msgr2=0x7f01b415c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:43.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.547+0000 7f01b9996640 1 --2- 172.21.15.26:0/4209651105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b406db00 0x7f01b415c780 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f01a400c3c0 tx=0x7f01a400c880 comp rx=0 tx=0).stop 2023-12-19T17:57:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.547+0000 7f01b9996640 1 -- 172.21.15.26:0/4209651105 shutdown_connections 2023-12-19T17:57:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.547+0000 7f01b9996640 1 --2- 172.21.15.26:0/4209651105 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f01a0065b80 0x7f01a0068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.547+0000 7f01b9996640 1 --2- 172.21.15.26:0/4209651105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f01b40cd3a0 0x7f01b415f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.547+0000 7f01b9996640 1 --2- 172.21.15.26:0/4209651105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f01b40ad7f0 0x7f01b40b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.547+0000 7f01b9996640 1 --2- 172.21.15.26:0/4209651105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01b406db00 0x7f01b415c780 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.547+0000 7f01b9996640 1 -- 172.21.15.26:0/4209651105 >> 172.21.15.26:0/4209651105 conn(0x7f01b40b0b60 msgr2=0x7f01b40adc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.547+0000 7f01b9996640 1 -- 172.21.15.26:0/4209651105 shutdown_connections 2023-12-19T17:57:43.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.547+0000 7f01b9996640 1 -- 172.21.15.26:0/4209651105 wait complete. 2023-12-19T17:57:43.572 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T17:57:43.573 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:57:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17d6d25640 1 Processor -- start 2023-12-19T17:57:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17d6d25640 1 -- start start 2023-12-19T17:57:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17d6d25640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f17d00ce040 0x7f17d00d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17d6d25640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d00d29d0 con 0x7f17d015cd40 2023-12-19T17:57:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17d6d25640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d00d3570 con 0x7f17d00d3130 2023-12-19T17:57:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17d6d25640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d015d680 con 0x7f17d00ce040 2023-12-19T17:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17d4ac1640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f17d015cd40 0x7f17d0160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35208/0 (socket says 172.21.15.26:35208) 2023-12-19T17:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17d4ac1640 1 -- 172.21.15.26:0/210606090 learned_addr learned my addr 172.21.15.26:0/210606090 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cffff640 1 --2- 172.21.15.26:0/210606090 >> v2:172.21.15.160:3300/0 conn(0x7f17d00ce040 0x7f17d00d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3601923580 0 0) 0x7f17c40012a0 con 0x7f17d015cd40 2023-12-19T17:57:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17c0003530 con 0x7f17d015cd40 2023-12-19T17:57:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 422839455 0 0) 0x7f17c0003530 con 0x7f17d015cd40 2023-12-19T17:57:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 >> v2:172.21.15.160:3300/0 conn(0x7f17d00ce040 msgr2=0x7f17d00d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 --2- 172.21.15.26:0/210606090 >> v2:172.21.15.160:3300/0 conn(0x7f17d00ce040 0x7f17d00d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 >> v1:172.21.15.160:6789/0 conn(0x7f17d00d3130 legacy=0x7f17d00d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d00ce4b0 con 0x7f17d015cd40 2023-12-19T17:57:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f17c4001990 con 0x7f17d015cd40 2023-12-19T17:57:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 >> v1:172.21.15.26:6789/0 conn(0x7f17d015cd40 legacy=0x7f17d0160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 --2- 172.21.15.26:0/210606090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c0003980 0x7f17c0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 --2- 172.21.15.26:0/210606090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c00047d0 0x7f17c0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 --2- 172.21.15.26:0/210606090 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17c0005190 0x7f17c000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d015d680 con 0x7f17c0003980 2023-12-19T17:57:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c000c030 con 0x7f17c00047d0 2023-12-19T17:57:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.719+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c000c1f0 con 0x7f17c0005190 2023-12-19T17:57:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17d4ac1640 1 --2- 172.21.15.26:0/210606090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c0003980 0x7f17c0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17cffff640 1 --2- 172.21.15.26:0/210606090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c00047d0 0x7f17c0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17d52c2640 1 --2- 172.21.15.26:0/210606090 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17c0005190 0x7f17c000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17d4ac1640 1 -- 172.21.15.26:0/210606090 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17c0005190 msgr2=0x7f17c000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17d4ac1640 1 --2- 172.21.15.26:0/210606090 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17c0005190 0x7f17c000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17d4ac1640 1 -- 172.21.15.26:0/210606090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c00047d0 msgr2=0x7f17c0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:43.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17d4ac1640 1 --2- 172.21.15.26:0/210606090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c00047d0 0x7f17c0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17d4ac1640 1 -- 172.21.15.26:0/210606090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c0003530 con 0x7f17c0003980 2023-12-19T17:57:43.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17d52c2640 1 -- 172.21.15.26:0/210606090 reap_dead start 2023-12-19T17:57:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17d4ac1640 1 --2- 172.21.15.26:0/210606090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c0003980 0x7f17c0003da0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f17c400c190 tx=0x7f17c401cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158a3179f1441303 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4007270 con 0x7f17c0003980 2023-12-19T17:57:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c40073d0 con 0x7f17c0003980 2023-12-19T17:57:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17cf7fe640 1 -- 172.21.15.26:0/210606090 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40a57e0 con 0x7f17c0003980 2023-12-19T17:57:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17d6d25640 1 -- 172.21.15.26:0/210606090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c0003980 msgr2=0x7f17c0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:43.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.723+0000 7f17d6d25640 1 --2- 172.21.15.26:0/210606090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c0003980 0x7f17c0003da0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f17c400c190 tx=0x7f17c401cb60 comp rx=0 tx=0).stop 2023-12-19T17:57:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d6d25640 1 -- 172.21.15.26:0/210606090 shutdown_connections 2023-12-19T17:57:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d6d25640 1 --2- 172.21.15.26:0/210606090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c0003980 0x7f17c0003da0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d6d25640 1 -- 172.21.15.26:0/210606090 >> 172.21.15.26:0/210606090 conn(0x7f17d006a120 msgr2=0x7f17d006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d6d25640 1 -- 172.21.15.26:0/210606090 shutdown_connections 2023-12-19T17:57:43.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d6d25640 1 -- 172.21.15.26:0/210606090 wait complete. 2023-12-19T17:57:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d6d25640 1 Processor -- start 2023-12-19T17:57:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d6d25640 1 -- start start 2023-12-19T17:57:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d6d25640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c0003980 0x7f17d0157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d6d25640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17d00c6790 0x7f17d0146d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d6d25640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17d00c6c10 0x7f17d0147270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d6d25640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d00c7df0 con 0x7f17c0003980 2023-12-19T17:57:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d6d25640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d01f59b0 con 0x7f17d00c6c10 2023-12-19T17:57:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d6d25640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d00c8280 con 0x7f17d00c6790 2023-12-19T17:57:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d4ac1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c0003980 0x7f17d0157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d4ac1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c0003980 0x7f17d0157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46234/0 (socket says 172.21.15.26:46234) 2023-12-19T17:57:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d4ac1640 1 -- 172.21.15.26:0/1772757625 learned_addr learned my addr 172.21.15.26:0/1772757625 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17cffff640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17d00c6790 0x7f17d0146d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d52c2640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17d00c6c10 0x7f17d0147270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d4ac1640 1 -- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17d00c6790 msgr2=0x7f17d0146d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d4ac1640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17d00c6790 0x7f17d0146d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d4ac1640 1 -- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17d00c6c10 msgr2=0x7f17d0147270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d4ac1640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17d00c6c10 0x7f17d0147270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d4ac1640 1 -- 172.21.15.26:0/1772757625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d00c71e0 con 0x7f17c0003980 2023-12-19T17:57:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.727+0000 7f17d4ac1640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c0003980 0x7f17d0157890 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f17c400c190 tx=0x7f17c0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.731+0000 7f17cd7fa640 1 -- 172.21.15.26:0/1772757625 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c40b4070 con 0x7f17c0003980 2023-12-19T17:57:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.731+0000 7f17cd7fa640 1 -- 172.21.15.26:0/1772757625 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c40012a0 con 0x7f17c0003980 2023-12-19T17:57:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.731+0000 7f17cd7fa640 1 -- 172.21.15.26:0/1772757625 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4001750 con 0x7f17c0003980 2023-12-19T17:57:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.731+0000 7f17d6d25640 1 -- 172.21.15.26:0/1772757625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d00c74c0 con 0x7f17c0003980 2023-12-19T17:57:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.731+0000 7f17d6d25640 1 -- 172.21.15.26:0/1772757625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d00c7a80 con 0x7f17c0003980 2023-12-19T17:57:43.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.731+0000 7f17d6d25640 1 -- 172.21.15.26:0/1772757625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d0074270 con 0x7f17c0003980 2023-12-19T17:57:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.731+0000 7f17cd7fa640 1 -- 172.21.15.26:0/1772757625 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f17c4007030 con 0x7f17c0003980 2023-12-19T17:57:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.731+0000 7f17cd7fa640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f17b4065b80 0x7f17b4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.731+0000 7f17cd7fa640 1 -- 172.21.15.26:0/1772757625 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f17c411cb50 con 0x7f17c0003980 2023-12-19T17:57:43.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.731+0000 7f17cffff640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f17b4065b80 0x7f17b4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:43.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.735+0000 7f17cffff640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f17b4065b80 0x7f17b4068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f17d01fa140 tx=0x7f17bc008040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:43.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.735+0000 7f17cd7fa640 1 -- 172.21.15.26:0/1772757625 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f17c40ea080 con 0x7f17c0003980 2023-12-19T17:57:43.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.927+0000 7f17d6d25640 1 -- 172.21.15.26:0/1772757625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17d00d29d0 con 0x7f17c0003980 2023-12-19T17:57:43.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.927+0000 7f17cd7fa640 1 -- 172.21.15.26:0/1772757625 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f17c40ed6a0 con 0x7f17c0003980 2023-12-19T17:57:43.931 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:43.931 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2850013418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2850013418},{"type":"v1","addr":"172.21.15.26:6835","nonce":2850013418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2213513866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2213513866},{"type":"v1","addr":"172.21.15.160:6836","nonce":2213513866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4292779848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4292779848},{"type":"v1","addr":"172.21.15.160:6837","nonce":4292779848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:57:40.517697+0000","modified":"2023-12-19T17:57:43.536734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8575":{"gid":8575,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/2838422258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2838422258},{"type":"v1","addr":"172.21.15.26:6834","nonce":2838422258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-19T17:57:43.931 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 73 2023-12-19T17:57:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.931+0000 7f17d6d25640 1 -- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f17b4065b80 msgr2=0x7f17b4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.931+0000 7f17d6d25640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f17b4065b80 0x7f17b4068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f17d01fa140 tx=0x7f17bc008040 comp rx=0 tx=0).stop 2023-12-19T17:57:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.931+0000 7f17d6d25640 1 -- 172.21.15.26:0/1772757625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c0003980 msgr2=0x7f17d0157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.931+0000 7f17d6d25640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c0003980 0x7f17d0157890 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f17c400c190 tx=0x7f17c0005800 comp rx=0 tx=0).stop 2023-12-19T17:57:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.931+0000 7f17d6d25640 1 -- 172.21.15.26:0/1772757625 shutdown_connections 2023-12-19T17:57:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.931+0000 7f17d6d25640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17d00c6c10 0x7f17d0147270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.931+0000 7f17d6d25640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f17b4065b80 0x7f17b4068040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.931+0000 7f17d6d25640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17d00c6790 0x7f17d0146d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.931+0000 7f17d6d25640 1 --2- 172.21.15.26:0/1772757625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17c0003980 0x7f17d0157890 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.931+0000 7f17d6d25640 1 -- 172.21.15.26:0/1772757625 >> 172.21.15.26:0/1772757625 conn(0x7f17d006a120 msgr2=0x7f17d00b8320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:43.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.931+0000 7f17d6d25640 1 -- 172.21.15.26:0/1772757625 shutdown_connections 2023-12-19T17:57:43.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:43.931+0000 7f17d6d25640 1 -- 172.21.15.26:0/1772757625 wait complete. 2023-12-19T17:57:43.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:57:44.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58ca6e5640 1 Processor -- start 2023-12-19T17:57:44.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58ca6e5640 1 -- start start 2023-12-19T17:57:44.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58ca6e5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58c415c7f0 0x7f58c406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58ca6e5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f58c40cc6e0 0x7f58c40ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58ca6e5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c406ced0 con 0x7f58c40adb80 2023-12-19T17:57:44.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58ca6e5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c406d770 con 0x7f58c415c7f0 2023-12-19T17:57:44.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58ca6e5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c40ceef0 con 0x7f58c40cc6e0 2023-12-19T17:57:44.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58c3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58c415c7f0 0x7f58c406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58c3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58c415c7f0 0x7f58c406c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46248/0 (socket says 172.21.15.26:46248) 2023-12-19T17:57:44.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58c3fff640 1 -- 172.21.15.26:0/4265336343 learned_addr learned my addr 172.21.15.26:0/4265336343 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:44.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58c8c82640 1 --2- 172.21.15.26:0/4265336343 >> v2:172.21.15.160:3300/0 conn(0x7f58c40cc6e0 0x7f58c40ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58c3fff640 1 -- 172.21.15.26:0/4265336343 >> v2:172.21.15.160:3300/0 conn(0x7f58c40cc6e0 msgr2=0x7f58c40ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:44.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58c3fff640 1 --2- 172.21.15.26:0/4265336343 >> v2:172.21.15.160:3300/0 conn(0x7f58c40cc6e0 0x7f58c40ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58c3fff640 1 -- 172.21.15.26:0/4265336343 >> v1:172.21.15.26:6789/0 conn(0x7f58c40adb80 legacy=0x7f58c40ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58c3fff640 1 -- 172.21.15.26:0/4265336343 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c406d1b0 con 0x7f58c415c7f0 2023-12-19T17:57:44.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.095+0000 7f58c8c82640 1 --2- 172.21.15.26:0/4265336343 >> v2:172.21.15.160:3300/0 conn(0x7f58c40cc6e0 0x7f58c40ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:57:44.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.099+0000 7f58c3fff640 1 --2- 172.21.15.26:0/4265336343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c415c7f0 0x7f58c406c7f0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f58c40cc420 tx=0x7f58b401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa311c2053cc86c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:44.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.099+0000 7f58c37fe640 1 -- 172.21.15.26:0/4265336343 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40ab5f0 con 0x7f58c415c7f0 2023-12-19T17:57:44.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.099+0000 7f58c37fe640 1 -- 172.21.15.26:0/4265336343 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b40ab750 con 0x7f58c415c7f0 2023-12-19T17:57:44.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.099+0000 7f58c37fe640 1 -- 172.21.15.26:0/4265336343 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40abae0 con 0x7f58c415c7f0 2023-12-19T17:57:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.099+0000 7f58ca6e5640 1 -- 172.21.15.26:0/4265336343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c415c7f0 msgr2=0x7f58c406c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.099+0000 7f58ca6e5640 1 --2- 172.21.15.26:0/4265336343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c415c7f0 0x7f58c406c7f0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f58c40cc420 tx=0x7f58b401c500 comp rx=0 tx=0).stop 2023-12-19T17:57:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.099+0000 7f58ca6e5640 1 -- 172.21.15.26:0/4265336343 shutdown_connections 2023-12-19T17:57:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.099+0000 7f58ca6e5640 1 --2- 172.21.15.26:0/4265336343 >> v2:172.21.15.160:3300/0 conn(0x7f58c40cc6e0 0x7f58c40ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.099+0000 7f58ca6e5640 1 --2- 172.21.15.26:0/4265336343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c415c7f0 0x7f58c406c7f0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.099+0000 7f58ca6e5640 1 -- 172.21.15.26:0/4265336343 >> 172.21.15.26:0/4265336343 conn(0x7f58c40658e0 msgr2=0x7f58c4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.099+0000 7f58ca6e5640 1 -- 172.21.15.26:0/4265336343 shutdown_connections 2023-12-19T17:57:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.099+0000 7f58ca6e5640 1 -- 172.21.15.26:0/4265336343 wait complete. 2023-12-19T17:57:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58ca6e5640 1 Processor -- start 2023-12-19T17:57:44.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58ca6e5640 1 -- start start 2023-12-19T17:57:44.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58ca6e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c40adb80 0x7f58c40c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58ca6e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58c40cc6e0 0x7f58c41583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58ca6e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c415c7f0 0x7f58c40b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58ca6e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c41588e0 con 0x7f58c415c7f0 2023-12-19T17:57:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58ca6e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c4146a00 con 0x7f58c40cc6e0 2023-12-19T17:57:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58ca6e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c40c4850 con 0x7f58c40adb80 2023-12-19T17:57:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58c8c82640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c415c7f0 0x7f58c40b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58c8c82640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c415c7f0 0x7f58c40b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46256/0 (socket says 172.21.15.26:46256) 2023-12-19T17:57:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58c8c82640 1 -- 172.21.15.26:0/1455687212 learned_addr learned my addr 172.21.15.26:0/1455687212 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58c3fff640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c40adb80 0x7f58c40c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58bbfff640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58c40cc6e0 0x7f58c41583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58c8c82640 1 -- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c40adb80 msgr2=0x7f58c40c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58c8c82640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c40adb80 0x7f58c40c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58c8c82640 1 -- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58c40cc6e0 msgr2=0x7f58c41583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58c8c82640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58c40cc6e0 0x7f58c41583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58c8c82640 1 -- 172.21.15.26:0/1455687212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b40094d0 con 0x7f58c415c7f0 2023-12-19T17:57:44.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58c8c82640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c415c7f0 0x7f58c40b1460 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00cef0 tx=0x7f58bc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:44.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58c17fa640 1 -- 172.21.15.26:0/1455687212 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00d840 con 0x7f58c415c7f0 2023-12-19T17:57:44.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58c17fa640 1 -- 172.21.15.26:0/1455687212 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58bc00d9c0 con 0x7f58c415c7f0 2023-12-19T17:57:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58c17fa640 1 -- 172.21.15.26:0/1455687212 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00dd70 con 0x7f58c415c7f0 2023-12-19T17:57:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58ca6e5640 1 -- 172.21.15.26:0/1455687212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c40c4570 con 0x7f58c415c7f0 2023-12-19T17:57:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58ca6e5640 1 -- 172.21.15.26:0/1455687212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c4145a20 con 0x7f58c415c7f0 2023-12-19T17:57:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.103+0000 7f58ca6e5640 1 -- 172.21.15.26:0/1455687212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c41455a0 con 0x7f58c415c7f0 2023-12-19T17:57:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.107+0000 7f58c17fa640 1 -- 172.21.15.26:0/1455687212 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f58bc01e3b0 con 0x7f58c415c7f0 2023-12-19T17:57:44.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.107+0000 7f58c17fa640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f58a4065b80 0x7f58a4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.107+0000 7f58c17fa640 1 -- 172.21.15.26:0/1455687212 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f58bc0840b0 con 0x7f58c415c7f0 2023-12-19T17:57:44.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.107+0000 7f58c3fff640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f58a4065b80 0x7f58a4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.107+0000 7f58c3fff640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f58a4065b80 0x7f58a4068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f58b40af000 tx=0x7f58b40a7000 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:44.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.111+0000 7f58c17fa640 1 -- 172.21.15.26:0/1455687212 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f58c41455a0 con 0x7f58c415c7f0 2023-12-19T17:57:44.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.267+0000 7f58ca6e5640 1 -- 172.21.15.26:0/1455687212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58c41455a0 con 0x7f58c415c7f0 2023-12-19T17:57:44.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.271+0000 7f58c17fa640 1 -- 172.21.15.26:0/1455687212 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v163) v1 ==== 74+0+17216 (secure 0 0 0) 0x7f58c41455a0 con 0x7f58c415c7f0 2023-12-19T17:57:44.272 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:44.273 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":163,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:57:41.520826+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:57:38.188932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T17:57:39.438347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:57:44.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.271+0000 7f58ca6e5640 1 -- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f58a4065b80 msgr2=0x7f58a4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.271+0000 7f58ca6e5640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f58a4065b80 0x7f58a4068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f58b40af000 tx=0x7f58b40a7000 comp rx=0 tx=0).stop 2023-12-19T17:57:44.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.271+0000 7f58ca6e5640 1 -- 172.21.15.26:0/1455687212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c415c7f0 msgr2=0x7f58c40b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.271+0000 7f58ca6e5640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c415c7f0 0x7f58c40b1460 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00cef0 tx=0x7f58bc00a710 comp rx=0 tx=0).stop 2023-12-19T17:57:44.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.271+0000 7f58ca6e5640 1 -- 172.21.15.26:0/1455687212 shutdown_connections 2023-12-19T17:57:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.271+0000 7f58ca6e5640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58c415c7f0 0x7f58c40b1460 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.271+0000 7f58ca6e5640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58c40cc6e0 0x7f58c41583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.271+0000 7f58ca6e5640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f58a4065b80 0x7f58a4068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.271+0000 7f58ca6e5640 1 --2- 172.21.15.26:0/1455687212 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58c40adb80 0x7f58c40c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.271+0000 7f58ca6e5640 1 -- 172.21.15.26:0/1455687212 >> 172.21.15.26:0/1455687212 conn(0x7f58c40658e0 msgr2=0x7f58c41550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.271+0000 7f58ca6e5640 1 -- 172.21.15.26:0/1455687212 shutdown_connections 2023-12-19T17:57:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.271+0000 7f58ca6e5640 1 -- 172.21.15.26:0/1455687212 wait complete. 2023-12-19T17:57:44.294 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T17:57:44.295 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T17:57:44.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.435+0000 7ff226552640 1 Processor -- start 2023-12-19T17:57:44.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.435+0000 7ff226552640 1 -- start start 2023-12-19T17:57:44.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.435+0000 7ff226552640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff22006b3d0 0x7ff2200ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.435+0000 7ff226552640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2200cec40 0x7ff22015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.435+0000 7ff226552640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22006bc30 con 0x7ff220154220 2023-12-19T17:57:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.435+0000 7ff226552640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2200cf0d0 con 0x7ff2200cec40 2023-12-19T17:57:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.435+0000 7ff226552640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2200cc6e0 con 0x7ff22006b3d0 2023-12-19T17:57:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.439+0000 7ff224aef640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2200cec40 0x7ff22015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.439+0000 7ff224aef640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2200cec40 0x7ff22015b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46264/0 (socket says 172.21.15.26:46264) 2023-12-19T17:57:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.439+0000 7ff224aef640 1 -- 172.21.15.26:0/4040575745 learned_addr learned my addr 172.21.15.26:0/4040575745 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.439+0000 7ff21f7fe640 1 --2- 172.21.15.26:0/4040575745 >> v2:172.21.15.160:3300/0 conn(0x7ff22006b3d0 0x7ff2200ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.439+0000 7ff224aef640 1 -- 172.21.15.26:0/4040575745 >> v2:172.21.15.160:3300/0 conn(0x7ff22006b3d0 msgr2=0x7ff2200ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.439+0000 7ff224aef640 1 --2- 172.21.15.26:0/4040575745 >> v2:172.21.15.160:3300/0 conn(0x7ff22006b3d0 0x7ff2200ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.439+0000 7ff224aef640 1 -- 172.21.15.26:0/4040575745 >> v1:172.21.15.160:6789/0 conn(0x7ff220154220 legacy=0x7ff22015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.439+0000 7ff224aef640 1 -- 172.21.15.26:0/4040575745 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22006b950 con 0x7ff2200cec40 2023-12-19T17:57:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.439+0000 7ff224aef640 1 --2- 172.21.15.26:0/4040575745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2200cec40 0x7ff22015b1c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7ff2200cf880 tx=0x7ff21801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbb1da72c461ac7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:44.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.439+0000 7ff21effd640 1 -- 172.21.15.26:0/4040575745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2180a5860 con 0x7ff2200cec40 2023-12-19T17:57:44.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.439+0000 7ff21effd640 1 -- 172.21.15.26:0/4040575745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2180a59c0 con 0x7ff2200cec40 2023-12-19T17:57:44.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.439+0000 7ff21effd640 1 -- 172.21.15.26:0/4040575745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2180a5d50 con 0x7ff2200cec40 2023-12-19T17:57:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 -- 172.21.15.26:0/4040575745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2200cec40 msgr2=0x7ff22015b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 --2- 172.21.15.26:0/4040575745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2200cec40 0x7ff22015b1c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7ff2200cf880 tx=0x7ff21801c9a0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 -- 172.21.15.26:0/4040575745 shutdown_connections 2023-12-19T17:57:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 --2- 172.21.15.26:0/4040575745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2200cec40 0x7ff22015b1c0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 --2- 172.21.15.26:0/4040575745 >> v2:172.21.15.160:3300/0 conn(0x7ff22006b3d0 0x7ff2200ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 -- 172.21.15.26:0/4040575745 >> 172.21.15.26:0/4040575745 conn(0x7ff2200658e0 msgr2=0x7ff220065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 -- 172.21.15.26:0/4040575745 shutdown_connections 2023-12-19T17:57:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 -- 172.21.15.26:0/4040575745 wait complete. 2023-12-19T17:57:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 Processor -- start 2023-12-19T17:57:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 -- start start 2023-12-19T17:57:44.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff22006b3d0 0x7ff220070f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2200cec40 0x7ff22014e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff220154220 0x7ff22014c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22014ed40 con 0x7ff220154220 2023-12-19T17:57:44.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200ad420 con 0x7ff2200cec40 2023-12-19T17:57:44.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff226552640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2201508d0 con 0x7ff22006b3d0 2023-12-19T17:57:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff224aef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff220154220 0x7ff22014c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff224aef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff220154220 0x7ff22014c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46280/0 (socket says 172.21.15.26:46280) 2023-12-19T17:57:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff224aef640 1 -- 172.21.15.26:0/735519333 learned_addr learned my addr 172.21.15.26:0/735519333 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.443+0000 7ff21ffff640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff22006b3d0 0x7ff220070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff21f7fe640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2200cec40 0x7ff22014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff224aef640 1 -- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff22006b3d0 msgr2=0x7ff220070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff224aef640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff22006b3d0 0x7ff220070f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff224aef640 1 -- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2200cec40 msgr2=0x7ff22014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff224aef640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2200cec40 0x7ff22014e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff224aef640 1 -- 172.21.15.26:0/735519333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff218009b30 con 0x7ff220154220 2023-12-19T17:57:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff224aef640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff220154220 0x7ff22014c460 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff2180af040 tx=0x7ff218019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff21cff9640 1 -- 172.21.15.26:0/735519333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2180b8670 con 0x7ff220154220 2023-12-19T17:57:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff21cff9640 1 -- 172.21.15.26:0/735519333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2180b87f0 con 0x7ff220154220 2023-12-19T17:57:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff21cff9640 1 -- 172.21.15.26:0/735519333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2180b8ba0 con 0x7ff220154220 2023-12-19T17:57:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff226552640 1 -- 172.21.15.26:0/735519333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2200ad6c0 con 0x7ff220154220 2023-12-19T17:57:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff226552640 1 -- 172.21.15.26:0/735519333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22014c9a0 con 0x7ff220154220 2023-12-19T17:57:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff226552640 1 -- 172.21.15.26:0/735519333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff220074270 con 0x7ff220154220 2023-12-19T17:57:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff21cff9640 1 -- 172.21.15.26:0/735519333 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff2180b8d00 con 0x7ff220154220 2023-12-19T17:57:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff21cff9640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff20c065b80 0x7ff20c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.447+0000 7ff21cff9640 1 -- 172.21.15.26:0/735519333 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7ff21811ec10 con 0x7ff220154220 2023-12-19T17:57:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.451+0000 7ff21ffff640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff20c065b80 0x7ff20c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.451+0000 7ff21ffff640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff20c065b80 0x7ff20c068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff210000f70 tx=0x7ff210009040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.455+0000 7ff21cff9640 1 -- 172.21.15.26:0/735519333 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff2180ec1f0 con 0x7ff220154220 2023-12-19T17:57:44.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.607+0000 7ff226552640 1 -- 172.21.15.26:0/735519333 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2200ccef0 con 0x7ff20c065b80 2023-12-19T17:57:44.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.619+0000 7ff21cff9640 1 -- 172.21.15.26:0/735519333 <== mgr.8359 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289729 (secure 0 0 0) 0x7ff210001370 con 0x7ff20c065b80 2023-12-19T17:57:44.623 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:44.625 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T17:57:44.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.623+0000 7ff226552640 1 -- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff20c065b80 msgr2=0x7ff20c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.623+0000 7ff226552640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff20c065b80 0x7ff20c068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff210000f70 tx=0x7ff210009040 comp rx=0 tx=0).stop 2023-12-19T17:57:44.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.623+0000 7ff226552640 1 -- 172.21.15.26:0/735519333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff220154220 msgr2=0x7ff22014c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.623+0000 7ff226552640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff220154220 0x7ff22014c460 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7ff2180af040 tx=0x7ff218019740 comp rx=0 tx=0).stop 2023-12-19T17:57:44.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.623+0000 7ff226552640 1 -- 172.21.15.26:0/735519333 shutdown_connections 2023-12-19T17:57:44.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.623+0000 7ff226552640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff220154220 0x7ff22014c460 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.623+0000 7ff226552640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff2200cec40 0x7ff22014e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.623+0000 7ff226552640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff20c065b80 0x7ff20c068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.623+0000 7ff226552640 1 --2- 172.21.15.26:0/735519333 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff22006b3d0 0x7ff220070f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.623+0000 7ff226552640 1 -- 172.21.15.26:0/735519333 >> 172.21.15.26:0/735519333 conn(0x7ff2200658e0 msgr2=0x7ff22015c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:44.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.623+0000 7ff226552640 1 -- 172.21.15.26:0/735519333 shutdown_connections 2023-12-19T17:57:44.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.623+0000 7ff226552640 1 -- 172.21.15.26:0/735519333 wait complete. 2023-12-19T17:57:44.643 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-19T17:57:42.925463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1442,"num_read_kb":2969,"num_write":479,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":404600,"kb_used_data":16824,"kb_used_omap":0,"kb_used_meta":387712,"kb_avail":754570120,"statfs":{"total":773094113280,"available":772679802880,"internally_reserved":0,"allocated":17227776,"data_stored":9510534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":397017088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000873"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558796+0000","last_change":"2023-12-19T17:57:40.553913+0000","last_active":"2023-12-19T17:57:40.558796+0000","last_peered":"2023-12-19T17:57:40.558796+0000","last_clean":"2023-12-19T17:57:40.558796+0000","last_became_active":"2023-12-19T17:57:40.553650+0000","last_became_peered":"2023-12-19T17:57:40.553650+0000","last_unstale":"2023-12-19T17:57:40.558796+0000","last_undegraded":"2023-12-19T17:57:40.558796+0000","last_fullsized":"2023-12-19T17:57:40.558796+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:01:15.868496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.545806+0000","last_change":"2023-12-19T17:57:40.537346+0000","last_active":"2023-12-19T17:57:40.545806+0000","last_peered":"2023-12-19T17:57:40.545806+0000","last_clean":"2023-12-19T17:57:40.545806+0000","last_became_active":"2023-12-19T17:57:40.535223+0000","last_became_peered":"2023-12-19T17:57:40.535223+0000","last_unstale":"2023-12-19T17:57:40.545806+0000","last_undegraded":"2023-12-19T17:57:40.545806+0000","last_fullsized":"2023-12-19T17:57:40.545806+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:26.339164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.547274+0000","last_change":"2023-12-19T17:57:40.540455+0000","last_active":"2023-12-19T17:57:40.547274+0000","last_peered":"2023-12-19T17:57:40.547274+0000","last_clean":"2023-12-19T17:57:40.547274+0000","last_became_active":"2023-12-19T17:57:40.540176+0000","last_became_peered":"2023-12-19T17:57:40.540176+0000","last_unstale":"2023-12-19T17:57:40.547274+0000","last_undegraded":"2023-12-19T17:57:40.547274+0000","last_fullsized":"2023-12-19T17:57:40.547274+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:08:31.123349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558520+0000","last_change":"2023-12-19T17:57:40.555239+0000","last_active":"2023-12-19T17:57:40.558520+0000","last_peered":"2023-12-19T17:57:40.558520+0000","last_clean":"2023-12-19T17:57:40.558520+0000","last_became_active":"2023-12-19T17:57:40.554913+0000","last_became_peered":"2023-12-19T17:57:40.554913+0000","last_unstale":"2023-12-19T17:57:40.558520+0000","last_undegraded":"2023-12-19T17:57:40.558520+0000","last_fullsized":"2023-12-19T17:57:40.558520+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:12.715283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.554151+0000","last_change":"2023-12-19T17:57:40.547627+0000","last_active":"2023-12-19T17:57:40.554151+0000","last_peered":"2023-12-19T17:57:40.554151+0000","last_clean":"2023-12-19T17:57:40.554151+0000","last_became_active":"2023-12-19T17:57:40.547346+0000","last_became_peered":"2023-12-19T17:57:40.547346+0000","last_unstale":"2023-12-19T17:57:40.554151+0000","last_undegraded":"2023-12-19T17:57:40.554151+0000","last_fullsized":"2023-12-19T17:57:40.554151+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:49.866445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552666+0000","last_change":"2023-12-19T17:57:40.545255+0000","last_active":"2023-12-19T17:57:40.552666+0000","last_peered":"2023-12-19T17:57:40.552666+0000","last_clean":"2023-12-19T17:57:40.552666+0000","last_became_active":"2023-12-19T17:57:40.544935+0000","last_became_peered":"2023-12-19T17:57:40.544935+0000","last_unstale":"2023-12-19T17:57:40.552666+0000","last_undegraded":"2023-12-19T17:57:40.552666+0000","last_fullsized":"2023-12-19T17:57:40.552666+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:15.573540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.551567+0000","last_change":"2023-12-19T17:57:40.541873+0000","last_active":"2023-12-19T17:57:40.551567+0000","last_peered":"2023-12-19T17:57:40.551567+0000","last_clean":"2023-12-19T17:57:40.551567+0000","last_became_active":"2023-12-19T17:57:40.541543+0000","last_became_peered":"2023-12-19T17:57:40.541543+0000","last_unstale":"2023-12-19T17:57:40.551567+0000","last_undegraded":"2023-12-19T17:57:40.551567+0000","last_fullsized":"2023-12-19T17:57:40.551567+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:24.741840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.553038+0000","last_change":"2023-12-19T17:57:40.542470+0000","last_active":"2023-12-19T17:57:40.553038+0000","last_peered":"2023-12-19T17:57:40.553038+0000","last_clean":"2023-12-19T17:57:40.553038+0000","last_became_active":"2023-12-19T17:57:40.542196+0000","last_became_peered":"2023-12-19T17:57:40.542196+0000","last_unstale":"2023-12-19T17:57:40.553038+0000","last_undegraded":"2023-12-19T17:57:40.553038+0000","last_fullsized":"2023-12-19T17:57:40.553038+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:34.938988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.550136+0000","last_change":"2023-12-19T17:57:40.541758+0000","last_active":"2023-12-19T17:57:40.550136+0000","last_peered":"2023-12-19T17:57:40.550136+0000","last_clean":"2023-12-19T17:57:40.550136+0000","last_became_active":"2023-12-19T17:57:40.541441+0000","last_became_peered":"2023-12-19T17:57:40.541441+0000","last_unstale":"2023-12-19T17:57:40.550136+0000","last_undegraded":"2023-12-19T17:57:40.550136+0000","last_fullsized":"2023-12-19T17:57:40.550136+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:28.187346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.545787+0000","last_change":"2023-12-19T17:57:40.535288+0000","last_active":"2023-12-19T17:57:40.545787+0000","last_peered":"2023-12-19T17:57:40.545787+0000","last_clean":"2023-12-19T17:57:40.545787+0000","last_became_active":"2023-12-19T17:57:40.534872+0000","last_became_peered":"2023-12-19T17:57:40.534872+0000","last_unstale":"2023-12-19T17:57:40.545787+0000","last_undegraded":"2023-12-19T17:57:40.545787+0000","last_fullsized":"2023-12-19T17:57:40.545787+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:12:43.624773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.538550+0000","last_change":"2023-12-19T17:57:40.537490+0000","last_active":"2023-12-19T17:57:40.538550+0000","last_peered":"2023-12-19T17:57:40.538550+0000","last_clean":"2023-12-19T17:57:40.538550+0000","last_became_active":"2023-12-19T17:57:40.536529+0000","last_became_peered":"2023-12-19T17:57:40.536529+0000","last_unstale":"2023-12-19T17:57:40.538550+0000","last_undegraded":"2023-12-19T17:57:40.538550+0000","last_fullsized":"2023-12-19T17:57:40.538550+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:31:36.716597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.547856+0000","last_change":"2023-12-19T17:57:40.540398+0000","last_active":"2023-12-19T17:57:40.547856+0000","last_peered":"2023-12-19T17:57:40.547856+0000","last_clean":"2023-12-19T17:57:40.547856+0000","last_became_active":"2023-12-19T17:57:40.540079+0000","last_became_peered":"2023-12-19T17:57:40.540079+0000","last_unstale":"2023-12-19T17:57:40.547856+0000","last_undegraded":"2023-12-19T17:57:40.547856+0000","last_fullsized":"2023-12-19T17:57:40.547856+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:46.267225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558470+0000","last_change":"2023-12-19T17:57:40.554375+0000","last_active":"2023-12-19T17:57:40.558470+0000","last_peered":"2023-12-19T17:57:40.558470+0000","last_clean":"2023-12-19T17:57:40.558470+0000","last_became_active":"2023-12-19T17:57:40.554119+0000","last_became_peered":"2023-12-19T17:57:40.554119+0000","last_unstale":"2023-12-19T17:57:40.558470+0000","last_undegraded":"2023-12-19T17:57:40.558470+0000","last_fullsized":"2023-12-19T17:57:40.558470+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:43:23.839334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557283+0000","last_change":"2023-12-19T17:57:40.549550+0000","last_active":"2023-12-19T17:57:40.557283+0000","last_peered":"2023-12-19T17:57:40.557283+0000","last_clean":"2023-12-19T17:57:40.557283+0000","last_became_active":"2023-12-19T17:57:40.549263+0000","last_became_peered":"2023-12-19T17:57:40.549263+0000","last_unstale":"2023-12-19T17:57:40.557283+0000","last_undegraded":"2023-12-19T17:57:40.557283+0000","last_fullsized":"2023-12-19T17:57:40.557283+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:10.867122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558239+0000","last_change":"2023-12-19T17:57:40.553519+0000","last_active":"2023-12-19T17:57:40.558239+0000","last_peered":"2023-12-19T17:57:40.558239+0000","last_clean":"2023-12-19T17:57:40.558239+0000","last_became_active":"2023-12-19T17:57:40.552994+0000","last_became_peered":"2023-12-19T17:57:40.552994+0000","last_unstale":"2023-12-19T17:57:40.558239+0000","last_undegraded":"2023-12-19T17:57:40.558239+0000","last_fullsized":"2023-12-19T17:57:40.558239+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:09.991187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.551724+0000","last_change":"2023-12-19T17:57:40.548553+0000","last_active":"2023-12-19T17:57:40.551724+0000","last_peered":"2023-12-19T17:57:40.551724+0000","last_clean":"2023-12-19T17:57:40.551724+0000","last_became_active":"2023-12-19T17:57:40.547224+0000","last_became_peered":"2023-12-19T17:57:40.547224+0000","last_unstale":"2023-12-19T17:57:40.551724+0000","last_undegraded":"2023-12-19T17:57:40.551724+0000","last_fullsized":"2023-12-19T17:57:40.551724+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:49:41.869352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552351+0000","last_change":"2023-12-19T17:57:40.542681+0000","last_active":"2023-12-19T17:57:40.552351+0000","last_peered":"2023-12-19T17:57:40.552351+0000","last_clean":"2023-12-19T17:57:40.552351+0000","last_became_active":"2023-12-19T17:57:40.541340+0000","last_became_peered":"2023-12-19T17:57:40.541340+0000","last_unstale":"2023-12-19T17:57:40.552351+0000","last_undegraded":"2023-12-19T17:57:40.552351+0000","last_fullsized":"2023-12-19T17:57:40.552351+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:08:31.123349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.537686+0000","last_change":"2023-12-19T17:57:40.531453+0000","last_active":"2023-12-19T17:57:40.537686+0000","last_peered":"2023-12-19T17:57:40.537686+0000","last_clean":"2023-12-19T17:57:40.537686+0000","last_became_active":"2023-12-19T17:57:40.531187+0000","last_became_peered":"2023-12-19T17:57:40.531187+0000","last_unstale":"2023-12-19T17:57:40.537686+0000","last_undegraded":"2023-12-19T17:57:40.537686+0000","last_fullsized":"2023-12-19T17:57:40.537686+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:45:09.414432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.537347+0000","last_change":"2023-12-19T17:57:40.536450+0000","last_active":"2023-12-19T17:57:40.537347+0000","last_peered":"2023-12-19T17:57:40.537347+0000","last_clean":"2023-12-19T17:57:40.537347+0000","last_became_active":"2023-12-19T17:57:40.533713+0000","last_became_peered":"2023-12-19T17:57:40.533713+0000","last_unstale":"2023-12-19T17:57:40.537347+0000","last_undegraded":"2023-12-19T17:57:40.537347+0000","last_fullsized":"2023-12-19T17:57:40.537347+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:43:03.209078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.544357+0000","last_change":"2023-12-19T17:57:40.541446+0000","last_active":"2023-12-19T17:57:40.544357+0000","last_peered":"2023-12-19T17:57:40.544357+0000","last_clean":"2023-12-19T17:57:40.544357+0000","last_became_active":"2023-12-19T17:57:40.541174+0000","last_became_peered":"2023-12-19T17:57:40.541174+0000","last_unstale":"2023-12-19T17:57:40.544357+0000","last_undegraded":"2023-12-19T17:57:40.544357+0000","last_fullsized":"2023-12-19T17:57:40.544357+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:59.428689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555512+0000","last_change":"2023-12-19T17:57:40.552259+0000","last_active":"2023-12-19T17:57:40.555512+0000","last_peered":"2023-12-19T17:57:40.555512+0000","last_clean":"2023-12-19T17:57:40.555512+0000","last_became_active":"2023-12-19T17:57:40.547786+0000","last_became_peered":"2023-12-19T17:57:40.547786+0000","last_unstale":"2023-12-19T17:57:40.555512+0000","last_undegraded":"2023-12-19T17:57:40.555512+0000","last_fullsized":"2023-12-19T17:57:40.555512+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:33.503868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.536641+0000","last_change":"2023-12-19T17:57:40.535867+0000","last_active":"2023-12-19T17:57:40.536641+0000","last_peered":"2023-12-19T17:57:40.536641+0000","last_clean":"2023-12-19T17:57:40.536641+0000","last_became_active":"2023-12-19T17:57:40.535608+0000","last_became_peered":"2023-12-19T17:57:40.535608+0000","last_unstale":"2023-12-19T17:57:40.536641+0000","last_undegraded":"2023-12-19T17:57:40.536641+0000","last_fullsized":"2023-12-19T17:57:40.536641+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:55:01.298850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557229+0000","last_change":"2023-12-19T17:57:40.550030+0000","last_active":"2023-12-19T17:57:40.557229+0000","last_peered":"2023-12-19T17:57:40.557229+0000","last_clean":"2023-12-19T17:57:40.557229+0000","last_became_active":"2023-12-19T17:57:40.547042+0000","last_became_peered":"2023-12-19T17:57:40.547042+0000","last_unstale":"2023-12-19T17:57:40.557229+0000","last_undegraded":"2023-12-19T17:57:40.557229+0000","last_fullsized":"2023-12-19T17:57:40.557229+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:50:55.497595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.536491+0000","last_change":"2023-12-19T17:57:40.534296+0000","last_active":"2023-12-19T17:57:40.536491+0000","last_peered":"2023-12-19T17:57:40.536491+0000","last_clean":"2023-12-19T17:57:40.536491+0000","last_became_active":"2023-12-19T17:57:40.534018+0000","last_became_peered":"2023-12-19T17:57:40.534018+0000","last_unstale":"2023-12-19T17:57:40.536491+0000","last_undegraded":"2023-12-19T17:57:40.536491+0000","last_fullsized":"2023-12-19T17:57:40.536491+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:08:52.714270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555374+0000","last_change":"2023-12-19T17:57:40.543166+0000","last_active":"2023-12-19T17:57:40.555374+0000","last_peered":"2023-12-19T17:57:40.555374+0000","last_clean":"2023-12-19T17:57:40.555374+0000","last_became_active":"2023-12-19T17:57:40.542918+0000","last_became_peered":"2023-12-19T17:57:40.542918+0000","last_unstale":"2023-12-19T17:57:40.555374+0000","last_undegraded":"2023-12-19T17:57:40.555374+0000","last_fullsized":"2023-12-19T17:57:40.555374+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:33.503868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.551141+0000","last_change":"2023-12-19T17:57:40.547916+0000","last_active":"2023-12-19T17:57:40.551141+0000","last_peered":"2023-12-19T17:57:40.551141+0000","last_clean":"2023-12-19T17:57:40.551141+0000","last_became_active":"2023-12-19T17:57:40.547146+0000","last_became_peered":"2023-12-19T17:57:40.547146+0000","last_unstale":"2023-12-19T17:57:40.551141+0000","last_undegraded":"2023-12-19T17:57:40.551141+0000","last_fullsized":"2023-12-19T17:57:40.551141+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:06.321345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555144+0000","last_change":"2023-12-19T17:57:40.547229+0000","last_active":"2023-12-19T17:57:40.555144+0000","last_peered":"2023-12-19T17:57:40.555144+0000","last_clean":"2023-12-19T17:57:40.555144+0000","last_became_active":"2023-12-19T17:57:40.546886+0000","last_became_peered":"2023-12-19T17:57:40.546886+0000","last_unstale":"2023-12-19T17:57:40.555144+0000","last_undegraded":"2023-12-19T17:57:40.555144+0000","last_fullsized":"2023-12-19T17:57:40.555144+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:05:16.354884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.533991+0000","last_change":"2023-12-19T17:57:40.530714+0000","last_active":"2023-12-19T17:57:40.533991+0000","last_peered":"2023-12-19T17:57:40.533991+0000","last_clean":"2023-12-19T17:57:40.533991+0000","last_became_active":"2023-12-19T17:57:40.530349+0000","last_became_peered":"2023-12-19T17:57:40.530349+0000","last_unstale":"2023-12-19T17:57:40.533991+0000","last_undegraded":"2023-12-19T17:57:40.533991+0000","last_fullsized":"2023-12-19T17:57:40.533991+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:23:48.874061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558450+0000","last_change":"2023-12-19T17:57:40.552318+0000","last_active":"2023-12-19T17:57:40.558450+0000","last_peered":"2023-12-19T17:57:40.558450+0000","last_clean":"2023-12-19T17:57:40.558450+0000","last_became_active":"2023-12-19T17:57:40.552014+0000","last_became_peered":"2023-12-19T17:57:40.552014+0000","last_unstale":"2023-12-19T17:57:40.558450+0000","last_undegraded":"2023-12-19T17:57:40.558450+0000","last_fullsized":"2023-12-19T17:57:40.558450+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:57:19.937681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557833+0000","last_change":"2023-12-19T17:57:40.551079+0000","last_active":"2023-12-19T17:57:40.557833+0000","last_peered":"2023-12-19T17:57:40.557833+0000","last_clean":"2023-12-19T17:57:40.557833+0000","last_became_active":"2023-12-19T17:57:40.550831+0000","last_became_peered":"2023-12-19T17:57:40.550831+0000","last_unstale":"2023-12-19T17:57:40.557833+0000","last_undegraded":"2023-12-19T17:57:40.557833+0000","last_fullsized":"2023-12-19T17:57:40.557833+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:49:10.047104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.554643+0000","last_change":"2023-12-19T17:57:40.550962+0000","last_active":"2023-12-19T17:57:40.554643+0000","last_peered":"2023-12-19T17:57:40.554643+0000","last_clean":"2023-12-19T17:57:40.554643+0000","last_became_active":"2023-12-19T17:57:40.550690+0000","last_became_peered":"2023-12-19T17:57:40.550690+0000","last_unstale":"2023-12-19T17:57:40.554643+0000","last_undegraded":"2023-12-19T17:57:40.554643+0000","last_fullsized":"2023-12-19T17:57:40.554643+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:55:06.915659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.533947+0000","last_change":"2023-12-19T17:57:40.529128+0000","last_active":"2023-12-19T17:57:40.533947+0000","last_peered":"2023-12-19T17:57:40.533947+0000","last_clean":"2023-12-19T17:57:40.533947+0000","last_became_active":"2023-12-19T17:57:40.528831+0000","last_became_peered":"2023-12-19T17:57:40.528831+0000","last_unstale":"2023-12-19T17:57:40.533947+0000","last_undegraded":"2023-12-19T17:57:40.533947+0000","last_fullsized":"2023-12-19T17:57:40.533947+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:34.938988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.538565+0000","last_change":"2023-12-19T17:57:40.534086+0000","last_active":"2023-12-19T17:57:40.538565+0000","last_peered":"2023-12-19T17:57:40.538565+0000","last_clean":"2023-12-19T17:57:40.538565+0000","last_became_active":"2023-12-19T17:57:40.533839+0000","last_became_peered":"2023-12-19T17:57:40.533839+0000","last_unstale":"2023-12-19T17:57:40.538565+0000","last_undegraded":"2023-12-19T17:57:40.538565+0000","last_fullsized":"2023-12-19T17:57:40.538565+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:13.176396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"162'4","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557836+0000","last_change":"2023-12-19T17:57:40.066265+0000","last_active":"2023-12-19T17:57:40.557836+0000","last_peered":"2023-12-19T17:57:40.557836+0000","last_clean":"2023-12-19T17:57:40.557836+0000","last_became_active":"2023-12-19T17:57:40.065975+0000","last_became_peered":"2023-12-19T17:57:40.065975+0000","last_unstale":"2023-12-19T17:57:40.557836+0000","last_undegraded":"2023-12-19T17:57:40.557836+0000","last_fullsized":"2023-12-19T17:57:40.557836+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:03.592110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.559086+0000","last_change":"2023-12-19T17:57:40.556008+0000","last_active":"2023-12-19T17:57:40.559086+0000","last_peered":"2023-12-19T17:57:40.559086+0000","last_clean":"2023-12-19T17:57:40.559086+0000","last_became_active":"2023-12-19T17:57:40.553963+0000","last_became_peered":"2023-12-19T17:57:40.553963+0000","last_unstale":"2023-12-19T17:57:40.559086+0000","last_undegraded":"2023-12-19T17:57:40.559086+0000","last_fullsized":"2023-12-19T17:57:40.559086+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:58.489960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552764+0000","last_change":"2023-12-19T17:57:40.059089+0000","last_active":"2023-12-19T17:57:40.552764+0000","last_peered":"2023-12-19T17:57:40.552764+0000","last_clean":"2023-12-19T17:57:40.552764+0000","last_became_active":"2023-12-19T17:57:40.058725+0000","last_became_peered":"2023-12-19T17:57:40.058725+0000","last_unstale":"2023-12-19T17:57:40.552764+0000","last_undegraded":"2023-12-19T17:57:40.552764+0000","last_fullsized":"2023-12-19T17:57:40.552764+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:01:14.857081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558173+0000","last_change":"2023-12-19T17:57:40.555051+0000","last_active":"2023-12-19T17:57:40.558173+0000","last_peered":"2023-12-19T17:57:40.558173+0000","last_clean":"2023-12-19T17:57:40.558173+0000","last_became_active":"2023-12-19T17:57:40.554537+0000","last_became_peered":"2023-12-19T17:57:40.554537+0000","last_unstale":"2023-12-19T17:57:40.558173+0000","last_undegraded":"2023-12-19T17:57:40.558173+0000","last_fullsized":"2023-12-19T17:57:40.558173+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:12:43.624773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.550457+0000","last_change":"2023-12-19T17:57:40.053267+0000","last_active":"2023-12-19T17:57:40.550457+0000","last_peered":"2023-12-19T17:57:40.550457+0000","last_clean":"2023-12-19T17:57:40.550457+0000","last_became_active":"2023-12-19T17:57:40.052945+0000","last_became_peered":"2023-12-19T17:57:40.052945+0000","last_unstale":"2023-12-19T17:57:40.550457+0000","last_undegraded":"2023-12-19T17:57:40.550457+0000","last_fullsized":"2023-12-19T17:57:40.550457+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:56:55.139037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558160+0000","last_change":"2023-12-19T17:57:40.556132+0000","last_active":"2023-12-19T17:57:40.558160+0000","last_peered":"2023-12-19T17:57:40.558160+0000","last_clean":"2023-12-19T17:57:40.558160+0000","last_became_active":"2023-12-19T17:57:40.555792+0000","last_became_peered":"2023-12-19T17:57:40.555792+0000","last_unstale":"2023-12-19T17:57:40.558160+0000","last_undegraded":"2023-12-19T17:57:40.558160+0000","last_fullsized":"2023-12-19T17:57:40.558160+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:26.339164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557205+0000","last_change":"2023-12-19T17:57:40.055429+0000","last_active":"2023-12-19T17:57:40.557205+0000","last_peered":"2023-12-19T17:57:40.557205+0000","last_clean":"2023-12-19T17:57:40.557205+0000","last_became_active":"2023-12-19T17:57:40.055148+0000","last_became_peered":"2023-12-19T17:57:40.055148+0000","last_unstale":"2023-12-19T17:57:40.557205+0000","last_undegraded":"2023-12-19T17:57:40.557205+0000","last_fullsized":"2023-12-19T17:57:40.557205+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:14.562125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.538014+0000","last_change":"2023-12-19T17:57:40.532278+0000","last_active":"2023-12-19T17:57:40.538014+0000","last_peered":"2023-12-19T17:57:40.538014+0000","last_clean":"2023-12-19T17:57:40.538014+0000","last_became_active":"2023-12-19T17:57:40.531964+0000","last_became_peered":"2023-12-19T17:57:40.531964+0000","last_unstale":"2023-12-19T17:57:40.538014+0000","last_undegraded":"2023-12-19T17:57:40.538014+0000","last_fullsized":"2023-12-19T17:57:40.538014+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:14.500977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.556680+0000","last_change":"2023-12-19T17:57:40.060446+0000","last_active":"2023-12-19T17:57:40.556680+0000","last_peered":"2023-12-19T17:57:40.556680+0000","last_clean":"2023-12-19T17:57:40.556680+0000","last_became_active":"2023-12-19T17:57:40.060197+0000","last_became_peered":"2023-12-19T17:57:40.060197+0000","last_unstale":"2023-12-19T17:57:40.556680+0000","last_undegraded":"2023-12-19T17:57:40.556680+0000","last_fullsized":"2023-12-19T17:57:40.556680+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:41:42.776082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.536814+0000","last_change":"2023-12-19T17:57:40.534304+0000","last_active":"2023-12-19T17:57:40.536814+0000","last_peered":"2023-12-19T17:57:40.536814+0000","last_clean":"2023-12-19T17:57:40.536814+0000","last_became_active":"2023-12-19T17:57:40.533988+0000","last_became_peered":"2023-12-19T17:57:40.533988+0000","last_unstale":"2023-12-19T17:57:40.536814+0000","last_undegraded":"2023-12-19T17:57:40.536814+0000","last_fullsized":"2023-12-19T17:57:40.536814+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:29:12.843009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555095+0000","last_change":"2023-12-19T17:57:40.059110+0000","last_active":"2023-12-19T17:57:40.555095+0000","last_peered":"2023-12-19T17:57:40.555095+0000","last_clean":"2023-12-19T17:57:40.555095+0000","last_became_active":"2023-12-19T17:57:40.058830+0000","last_became_peered":"2023-12-19T17:57:40.058830+0000","last_unstale":"2023-12-19T17:57:40.555095+0000","last_undegraded":"2023-12-19T17:57:40.555095+0000","last_fullsized":"2023-12-19T17:57:40.555095+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.816414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.547416+0000","last_change":"2023-12-19T17:57:40.540067+0000","last_active":"2023-12-19T17:57:40.547416+0000","last_peered":"2023-12-19T17:57:40.547416+0000","last_clean":"2023-12-19T17:57:40.547416+0000","last_became_active":"2023-12-19T17:57:40.539701+0000","last_became_peered":"2023-12-19T17:57:40.539701+0000","last_unstale":"2023-12-19T17:57:40.547416+0000","last_undegraded":"2023-12-19T17:57:40.547416+0000","last_fullsized":"2023-12-19T17:57:40.547416+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:29:12.843009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.527444+0000","last_change":"2023-12-19T17:57:40.063570+0000","last_active":"2023-12-19T17:57:40.527444+0000","last_peered":"2023-12-19T17:57:40.527444+0000","last_clean":"2023-12-19T17:57:40.527444+0000","last_became_active":"2023-12-19T17:57:40.063286+0000","last_became_peered":"2023-12-19T17:57:40.063286+0000","last_unstale":"2023-12-19T17:57:40.527444+0000","last_undegraded":"2023-12-19T17:57:40.527444+0000","last_fullsized":"2023-12-19T17:57:40.527444+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:34:33.680079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.553208+0000","last_change":"2023-12-19T17:57:40.539328+0000","last_active":"2023-12-19T17:57:40.553208+0000","last_peered":"2023-12-19T17:57:40.553208+0000","last_clean":"2023-12-19T17:57:40.553208+0000","last_became_active":"2023-12-19T17:57:40.539027+0000","last_became_peered":"2023-12-19T17:57:40.539027+0000","last_unstale":"2023-12-19T17:57:40.553208+0000","last_undegraded":"2023-12-19T17:57:40.553208+0000","last_fullsized":"2023-12-19T17:57:40.553208+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:23.973532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.531517+0000","last_change":"2023-12-19T17:57:40.061729+0000","last_active":"2023-12-19T17:57:40.531517+0000","last_peered":"2023-12-19T17:57:40.531517+0000","last_clean":"2023-12-19T17:57:40.531517+0000","last_became_active":"2023-12-19T17:57:40.061340+0000","last_became_peered":"2023-12-19T17:57:40.061340+0000","last_unstale":"2023-12-19T17:57:40.531517+0000","last_undegraded":"2023-12-19T17:57:40.531517+0000","last_fullsized":"2023-12-19T17:57:40.531517+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:23.730425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557315+0000","last_change":"2023-12-19T17:57:40.550421+0000","last_active":"2023-12-19T17:57:40.557315+0000","last_peered":"2023-12-19T17:57:40.557315+0000","last_clean":"2023-12-19T17:57:40.557315+0000","last_became_active":"2023-12-19T17:57:40.550104+0000","last_became_peered":"2023-12-19T17:57:40.550104+0000","last_unstale":"2023-12-19T17:57:40.557315+0000","last_undegraded":"2023-12-19T17:57:40.557315+0000","last_fullsized":"2023-12-19T17:57:40.557315+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:50:39.392090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.526732+0000","last_change":"2023-12-19T17:57:40.058636+0000","last_active":"2023-12-19T17:57:40.526732+0000","last_peered":"2023-12-19T17:57:40.526732+0000","last_clean":"2023-12-19T17:57:40.526732+0000","last_became_active":"2023-12-19T17:57:40.058366+0000","last_became_peered":"2023-12-19T17:57:40.058366+0000","last_unstale":"2023-12-19T17:57:40.526732+0000","last_undegraded":"2023-12-19T17:57:40.526732+0000","last_fullsized":"2023-12-19T17:57:40.526732+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:11.703868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.539952+0000","last_change":"2023-12-19T17:57:40.535337+0000","last_active":"2023-12-19T17:57:40.539952+0000","last_peered":"2023-12-19T17:57:40.539952+0000","last_clean":"2023-12-19T17:57:40.539952+0000","last_became_active":"2023-12-19T17:57:40.535059+0000","last_became_peered":"2023-12-19T17:57:40.535059+0000","last_unstale":"2023-12-19T17:57:40.539952+0000","last_undegraded":"2023-12-19T17:57:40.539952+0000","last_fullsized":"2023-12-19T17:57:40.539952+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:13:06.277412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.524341+0000","last_change":"2023-12-19T17:57:40.057956+0000","last_active":"2023-12-19T17:57:40.524341+0000","last_peered":"2023-12-19T17:57:40.524341+0000","last_clean":"2023-12-19T17:57:40.524341+0000","last_became_active":"2023-12-19T17:57:40.057689+0000","last_became_peered":"2023-12-19T17:57:40.057689+0000","last_unstale":"2023-12-19T17:57:40.524341+0000","last_undegraded":"2023-12-19T17:57:40.524341+0000","last_fullsized":"2023-12-19T17:57:40.524341+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:55:44.482375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552513+0000","last_change":"2023-12-19T17:57:40.547732+0000","last_active":"2023-12-19T17:57:40.552513+0000","last_peered":"2023-12-19T17:57:40.552513+0000","last_clean":"2023-12-19T17:57:40.552513+0000","last_became_active":"2023-12-19T17:57:40.547451+0000","last_became_peered":"2023-12-19T17:57:40.547451+0000","last_unstale":"2023-12-19T17:57:40.552513+0000","last_undegraded":"2023-12-19T17:57:40.552513+0000","last_fullsized":"2023-12-19T17:57:40.552513+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:10.867122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"162'3","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.548429+0000","last_change":"2023-12-19T17:57:40.061646+0000","last_active":"2023-12-19T17:57:40.548429+0000","last_peered":"2023-12-19T17:57:40.548429+0000","last_clean":"2023-12-19T17:57:40.548429+0000","last_became_active":"2023-12-19T17:57:40.061357+0000","last_became_peered":"2023-12-19T17:57:40.061357+0000","last_unstale":"2023-12-19T17:57:40.548429+0000","last_undegraded":"2023-12-19T17:57:40.548429+0000","last_fullsized":"2023-12-19T17:57:40.548429+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:05:15.343469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.556692+0000","last_change":"2023-12-19T17:57:40.551317+0000","last_active":"2023-12-19T17:57:40.556692+0000","last_peered":"2023-12-19T17:57:40.556692+0000","last_clean":"2023-12-19T17:57:40.556692+0000","last_became_active":"2023-12-19T17:57:40.551061+0000","last_became_peered":"2023-12-19T17:57:40.551061+0000","last_unstale":"2023-12-19T17:57:40.556692+0000","last_undegraded":"2023-12-19T17:57:40.556692+0000","last_fullsized":"2023-12-19T17:57:40.556692+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:48:38.782646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"162'6","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.561151+0000","last_change":"2023-12-19T17:57:40.067315+0000","last_active":"2023-12-19T17:57:40.561151+0000","last_peered":"2023-12-19T17:57:40.561151+0000","last_clean":"2023-12-19T17:57:40.561151+0000","last_became_active":"2023-12-19T17:57:40.066941+0000","last_became_peered":"2023-12-19T17:57:40.066941+0000","last_unstale":"2023-12-19T17:57:40.561151+0000","last_undegraded":"2023-12-19T17:57:40.561151+0000","last_fullsized":"2023-12-19T17:57:40.561151+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:43:53.860436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557806+0000","last_change":"2023-12-19T17:57:40.550484+0000","last_active":"2023-12-19T17:57:40.557806+0000","last_peered":"2023-12-19T17:57:40.557806+0000","last_clean":"2023-12-19T17:57:40.557806+0000","last_became_active":"2023-12-19T17:57:40.549105+0000","last_became_peered":"2023-12-19T17:57:40.549105+0000","last_unstale":"2023-12-19T17:57:40.557806+0000","last_undegraded":"2023-12-19T17:57:40.557806+0000","last_fullsized":"2023-12-19T17:57:40.557806+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:29:59.039217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.545499+0000","last_change":"2023-12-19T17:57:40.052924+0000","last_active":"2023-12-19T17:57:40.545499+0000","last_peered":"2023-12-19T17:57:40.545499+0000","last_clean":"2023-12-19T17:57:40.545499+0000","last_became_active":"2023-12-19T17:57:40.052663+0000","last_became_peered":"2023-12-19T17:57:40.052663+0000","last_unstale":"2023-12-19T17:57:40.545499+0000","last_undegraded":"2023-12-19T17:57:40.545499+0000","last_fullsized":"2023-12-19T17:57:40.545499+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:48:37.899407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.556642+0000","last_change":"2023-12-19T17:57:40.546595+0000","last_active":"2023-12-19T17:57:40.556642+0000","last_peered":"2023-12-19T17:57:40.556642+0000","last_clean":"2023-12-19T17:57:40.556642+0000","last_became_active":"2023-12-19T17:57:40.546142+0000","last_became_peered":"2023-12-19T17:57:40.546142+0000","last_unstale":"2023-12-19T17:57:40.556642+0000","last_undegraded":"2023-12-19T17:57:40.556642+0000","last_fullsized":"2023-12-19T17:57:40.556642+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:49.866445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.531436+0000","last_change":"2023-12-19T17:57:40.065628+0000","last_active":"2023-12-19T17:57:40.531436+0000","last_peered":"2023-12-19T17:57:40.531436+0000","last_clean":"2023-12-19T17:57:40.531436+0000","last_became_active":"2023-12-19T17:57:40.065313+0000","last_became_peered":"2023-12-19T17:57:40.065313+0000","last_unstale":"2023-12-19T17:57:40.531436+0000","last_undegraded":"2023-12-19T17:57:40.531436+0000","last_fullsized":"2023-12-19T17:57:40.531436+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:03.592110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.556852+0000","last_change":"2023-12-19T17:57:40.550414+0000","last_active":"2023-12-19T17:57:40.556852+0000","last_peered":"2023-12-19T17:57:40.556852+0000","last_clean":"2023-12-19T17:57:40.556852+0000","last_became_active":"2023-12-19T17:57:40.547679+0000","last_became_peered":"2023-12-19T17:57:40.547679+0000","last_unstale":"2023-12-19T17:57:40.556852+0000","last_undegraded":"2023-12-19T17:57:40.556852+0000","last_fullsized":"2023-12-19T17:57:40.556852+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:34.746775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"162'8","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558862+0000","last_change":"2023-12-19T17:57:40.062566+0000","last_active":"2023-12-19T17:57:40.558862+0000","last_peered":"2023-12-19T17:57:40.558862+0000","last_clean":"2023-12-19T17:57:40.558862+0000","last_became_active":"2023-12-19T17:57:40.061288+0000","last_became_peered":"2023-12-19T17:57:40.061288+0000","last_unstale":"2023-12-19T17:57:40.558862+0000","last_undegraded":"2023-12-19T17:57:40.558862+0000","last_fullsized":"2023-12-19T17:57:40.558862+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:47.281313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557596+0000","last_change":"2023-12-19T17:57:40.552976+0000","last_active":"2023-12-19T17:57:40.557596+0000","last_peered":"2023-12-19T17:57:40.557596+0000","last_clean":"2023-12-19T17:57:40.557596+0000","last_became_active":"2023-12-19T17:57:40.552691+0000","last_became_peered":"2023-12-19T17:57:40.552691+0000","last_unstale":"2023-12-19T17:57:40.557596+0000","last_undegraded":"2023-12-19T17:57:40.557596+0000","last_fullsized":"2023-12-19T17:57:40.557596+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:15.573540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.554597+0000","last_change":"2023-12-19T17:57:40.067096+0000","last_active":"2023-12-19T17:57:40.554597+0000","last_peered":"2023-12-19T17:57:40.554597+0000","last_clean":"2023-12-19T17:57:40.554597+0000","last_became_active":"2023-12-19T17:57:40.066562+0000","last_became_peered":"2023-12-19T17:57:40.066562+0000","last_unstale":"2023-12-19T17:57:40.554597+0000","last_undegraded":"2023-12-19T17:57:40.554597+0000","last_fullsized":"2023-12-19T17:57:40.554597+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:08.979772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.539969+0000","last_change":"2023-12-19T17:57:40.536577+0000","last_active":"2023-12-19T17:57:40.539969+0000","last_peered":"2023-12-19T17:57:40.539969+0000","last_clean":"2023-12-19T17:57:40.539969+0000","last_became_active":"2023-12-19T17:57:40.536260+0000","last_became_peered":"2023-12-19T17:57:40.536260+0000","last_unstale":"2023-12-19T17:57:40.539969+0000","last_undegraded":"2023-12-19T17:57:40.539969+0000","last_fullsized":"2023-12-19T17:57:40.539969+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:34:24.820518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.554222+0000","last_change":"2023-12-19T17:57:40.057376+0000","last_active":"2023-12-19T17:57:40.554222+0000","last_peered":"2023-12-19T17:57:40.554222+0000","last_clean":"2023-12-19T17:57:40.554222+0000","last_became_active":"2023-12-19T17:57:40.057086+0000","last_became_peered":"2023-12-19T17:57:40.057086+0000","last_unstale":"2023-12-19T17:57:40.554222+0000","last_undegraded":"2023-12-19T17:57:40.554222+0000","last_fullsized":"2023-12-19T17:57:40.554222+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:27:55.404240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.551140+0000","last_change":"2023-12-19T17:57:40.547510+0000","last_active":"2023-12-19T17:57:40.551140+0000","last_peered":"2023-12-19T17:57:40.551140+0000","last_clean":"2023-12-19T17:57:40.551140+0000","last_became_active":"2023-12-19T17:57:40.547204+0000","last_became_peered":"2023-12-19T17:57:40.547204+0000","last_unstale":"2023-12-19T17:57:40.551140+0000","last_undegraded":"2023-12-19T17:57:40.551140+0000","last_fullsized":"2023-12-19T17:57:40.551140+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:34:40.964262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552504+0000","last_change":"2023-12-19T17:57:40.055466+0000","last_active":"2023-12-19T17:57:40.552504+0000","last_peered":"2023-12-19T17:57:40.552504+0000","last_clean":"2023-12-19T17:57:40.552504+0000","last_became_active":"2023-12-19T17:57:40.055145+0000","last_became_peered":"2023-12-19T17:57:40.055145+0000","last_unstale":"2023-12-19T17:57:40.552504+0000","last_undegraded":"2023-12-19T17:57:40.552504+0000","last_fullsized":"2023-12-19T17:57:40.552504+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:32:20.433509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.553312+0000","last_change":"2023-12-19T17:57:40.540229+0000","last_active":"2023-12-19T17:57:40.553312+0000","last_peered":"2023-12-19T17:57:40.553312+0000","last_clean":"2023-12-19T17:57:40.553312+0000","last_became_active":"2023-12-19T17:57:40.539948+0000","last_became_peered":"2023-12-19T17:57:40.539948+0000","last_unstale":"2023-12-19T17:57:40.553312+0000","last_undegraded":"2023-12-19T17:57:40.553312+0000","last_fullsized":"2023-12-19T17:57:40.553312+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:55:45.493789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.525668+0000","last_change":"2023-12-19T17:57:40.058276+0000","last_active":"2023-12-19T17:57:40.525668+0000","last_peered":"2023-12-19T17:57:40.525668+0000","last_clean":"2023-12-19T17:57:40.525668+0000","last_became_active":"2023-12-19T17:57:40.057982+0000","last_became_peered":"2023-12-19T17:57:40.057982+0000","last_unstale":"2023-12-19T17:57:40.525668+0000","last_undegraded":"2023-12-19T17:57:40.525668+0000","last_fullsized":"2023-12-19T17:57:40.525668+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:49:09.035689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.556469+0000","last_change":"2023-12-19T17:57:40.548292+0000","last_active":"2023-12-19T17:57:40.556469+0000","last_peered":"2023-12-19T17:57:40.556469+0000","last_clean":"2023-12-19T17:57:40.556469+0000","last_became_active":"2023-12-19T17:57:40.547740+0000","last_became_peered":"2023-12-19T17:57:40.547740+0000","last_unstale":"2023-12-19T17:57:40.556469+0000","last_undegraded":"2023-12-19T17:57:40.556469+0000","last_fullsized":"2023-12-19T17:57:40.556469+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:59.244547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.556960+0000","last_change":"2023-12-19T17:57:40.059000+0000","last_active":"2023-12-19T17:57:40.556960+0000","last_peered":"2023-12-19T17:57:40.556960+0000","last_clean":"2023-12-19T17:57:40.556960+0000","last_became_active":"2023-12-19T17:57:40.058583+0000","last_became_peered":"2023-12-19T17:57:40.058583+0000","last_unstale":"2023-12-19T17:57:40.556960+0000","last_undegraded":"2023-12-19T17:57:40.556960+0000","last_fullsized":"2023-12-19T17:57:40.556960+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:57:18.926267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552592+0000","last_change":"2023-12-19T17:57:40.546220+0000","last_active":"2023-12-19T17:57:40.552592+0000","last_peered":"2023-12-19T17:57:40.552592+0000","last_clean":"2023-12-19T17:57:40.552592+0000","last_became_active":"2023-12-19T17:57:40.545944+0000","last_became_peered":"2023-12-19T17:57:40.545944+0000","last_unstale":"2023-12-19T17:57:40.552592+0000","last_undegraded":"2023-12-19T17:57:40.552592+0000","last_fullsized":"2023-12-19T17:57:40.552592+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:41:43.787497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.523199+0000","last_change":"2023-12-19T17:57:40.058009+0000","last_active":"2023-12-19T17:57:40.523199+0000","last_peered":"2023-12-19T17:57:40.523199+0000","last_clean":"2023-12-19T17:57:40.523199+0000","last_became_active":"2023-12-19T17:57:40.057735+0000","last_became_peered":"2023-12-19T17:57:40.057735+0000","last_unstale":"2023-12-19T17:57:40.523199+0000","last_undegraded":"2023-12-19T17:57:40.523199+0000","last_fullsized":"2023-12-19T17:57:40.523199+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:55:05.904244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552616+0000","last_change":"2023-12-19T17:57:40.539348+0000","last_active":"2023-12-19T17:57:40.552616+0000","last_peered":"2023-12-19T17:57:40.552616+0000","last_clean":"2023-12-19T17:57:40.552616+0000","last_became_active":"2023-12-19T17:57:40.539081+0000","last_became_peered":"2023-12-19T17:57:40.539081+0000","last_unstale":"2023-12-19T17:57:40.552616+0000","last_undegraded":"2023-12-19T17:57:40.552616+0000","last_fullsized":"2023-12-19T17:57:40.552616+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:50.827829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.522645+0000","last_change":"2023-12-19T17:57:40.063416+0000","last_active":"2023-12-19T17:57:40.522645+0000","last_peered":"2023-12-19T17:57:40.522645+0000","last_clean":"2023-12-19T17:57:40.522645+0000","last_became_active":"2023-12-19T17:57:40.062526+0000","last_became_peered":"2023-12-19T17:57:40.062526+0000","last_unstale":"2023-12-19T17:57:40.522645+0000","last_undegraded":"2023-12-19T17:57:40.522645+0000","last_fullsized":"2023-12-19T17:57:40.522645+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:24.484854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555318+0000","last_change":"2023-12-19T17:57:40.552043+0000","last_active":"2023-12-19T17:57:40.555318+0000","last_peered":"2023-12-19T17:57:40.555318+0000","last_clean":"2023-12-19T17:57:40.555318+0000","last_became_active":"2023-12-19T17:57:40.549303+0000","last_became_peered":"2023-12-19T17:57:40.549303+0000","last_unstale":"2023-12-19T17:57:40.555318+0000","last_undegraded":"2023-12-19T17:57:40.555318+0000","last_fullsized":"2023-12-19T17:57:40.555318+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:55:45.493789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.527606+0000","last_change":"2023-12-19T17:57:40.061890+0000","last_active":"2023-12-19T17:57:40.527606+0000","last_peered":"2023-12-19T17:57:40.527606+0000","last_clean":"2023-12-19T17:57:40.527606+0000","last_became_active":"2023-12-19T17:57:40.060827+0000","last_became_peered":"2023-12-19T17:57:40.060827+0000","last_unstale":"2023-12-19T17:57:40.527606+0000","last_undegraded":"2023-12-19T17:57:40.527606+0000","last_fullsized":"2023-12-19T17:57:40.527606+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:41:22.464097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.550776+0000","last_change":"2023-12-19T17:57:40.539806+0000","last_active":"2023-12-19T17:57:40.550776+0000","last_peered":"2023-12-19T17:57:40.550776+0000","last_clean":"2023-12-19T17:57:40.550776+0000","last_became_active":"2023-12-19T17:57:40.539544+0000","last_became_peered":"2023-12-19T17:57:40.539544+0000","last_unstale":"2023-12-19T17:57:40.550776+0000","last_undegraded":"2023-12-19T17:57:40.550776+0000","last_fullsized":"2023-12-19T17:57:40.550776+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:28.187346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558862+0000","last_change":"2023-12-19T17:57:40.062974+0000","last_active":"2023-12-19T17:57:40.558862+0000","last_peered":"2023-12-19T17:57:40.558862+0000","last_clean":"2023-12-19T17:57:40.558862+0000","last_became_active":"2023-12-19T17:57:40.062312+0000","last_became_peered":"2023-12-19T17:57:40.062312+0000","last_unstale":"2023-12-19T17:57:40.558862+0000","last_undegraded":"2023-12-19T17:57:40.558862+0000","last_fullsized":"2023-12-19T17:57:40.558862+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:21.078553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.559402+0000","last_change":"2023-12-19T17:57:40.558668+0000","last_active":"2023-12-19T17:57:40.559402+0000","last_peered":"2023-12-19T17:57:40.559402+0000","last_clean":"2023-12-19T17:57:40.559402+0000","last_became_active":"2023-12-19T17:57:40.558367+0000","last_became_peered":"2023-12-19T17:57:40.558367+0000","last_unstale":"2023-12-19T17:57:40.559402+0000","last_undegraded":"2023-12-19T17:57:40.559402+0000","last_fullsized":"2023-12-19T17:57:40.559402+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:24.881677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.537273+0000","last_change":"2023-12-19T17:57:40.065674+0000","last_active":"2023-12-19T17:57:40.537273+0000","last_peered":"2023-12-19T17:57:40.537273+0000","last_clean":"2023-12-19T17:57:40.537273+0000","last_became_active":"2023-12-19T17:57:40.065320+0000","last_became_peered":"2023-12-19T17:57:40.065320+0000","last_unstale":"2023-12-19T17:57:40.537273+0000","last_undegraded":"2023-12-19T17:57:40.537273+0000","last_fullsized":"2023-12-19T17:57:40.537273+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:23.730425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557061+0000","last_change":"2023-12-19T17:57:40.548642+0000","last_active":"2023-12-19T17:57:40.557061+0000","last_peered":"2023-12-19T17:57:40.557061+0000","last_clean":"2023-12-19T17:57:40.557061+0000","last_became_active":"2023-12-19T17:57:40.548153+0000","last_became_peered":"2023-12-19T17:57:40.548153+0000","last_unstale":"2023-12-19T17:57:40.557061+0000","last_undegraded":"2023-12-19T17:57:40.557061+0000","last_fullsized":"2023-12-19T17:57:40.557061+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:14.500977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.531965+0000","last_change":"2023-12-19T17:57:40.061558+0000","last_active":"2023-12-19T17:57:40.531965+0000","last_peered":"2023-12-19T17:57:40.531965+0000","last_clean":"2023-12-19T17:57:40.531965+0000","last_became_active":"2023-12-19T17:57:40.061289+0000","last_became_peered":"2023-12-19T17:57:40.061289+0000","last_unstale":"2023-12-19T17:57:40.531965+0000","last_undegraded":"2023-12-19T17:57:40.531965+0000","last_fullsized":"2023-12-19T17:57:40.531965+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:59.357090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552556+0000","last_change":"2023-12-19T17:57:40.547683+0000","last_active":"2023-12-19T17:57:40.552556+0000","last_peered":"2023-12-19T17:57:40.552556+0000","last_clean":"2023-12-19T17:57:40.552556+0000","last_became_active":"2023-12-19T17:57:40.547412+0000","last_became_peered":"2023-12-19T17:57:40.547412+0000","last_unstale":"2023-12-19T17:57:40.552556+0000","last_undegraded":"2023-12-19T17:57:40.552556+0000","last_fullsized":"2023-12-19T17:57:40.552556+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:58.489960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.528324+0000","last_change":"2023-12-19T17:57:40.055721+0000","last_active":"2023-12-19T17:57:40.528324+0000","last_peered":"2023-12-19T17:57:40.528324+0000","last_clean":"2023-12-19T17:57:40.528324+0000","last_became_active":"2023-12-19T17:57:40.055382+0000","last_became_peered":"2023-12-19T17:57:40.055382+0000","last_unstale":"2023-12-19T17:57:40.528324+0000","last_undegraded":"2023-12-19T17:57:40.528324+0000","last_fullsized":"2023-12-19T17:57:40.528324+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:47:20.484481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555834+0000","last_change":"2023-12-19T17:57:40.552580+0000","last_active":"2023-12-19T17:57:40.555834+0000","last_peered":"2023-12-19T17:57:40.555834+0000","last_clean":"2023-12-19T17:57:40.555834+0000","last_became_active":"2023-12-19T17:57:40.552322+0000","last_became_peered":"2023-12-19T17:57:40.552322+0000","last_unstale":"2023-12-19T17:57:40.555834+0000","last_undegraded":"2023-12-19T17:57:40.555834+0000","last_fullsized":"2023-12-19T17:57:40.555834+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:07.348734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.531955+0000","last_change":"2023-12-19T17:57:40.064222+0000","last_active":"2023-12-19T17:57:40.531955+0000","last_peered":"2023-12-19T17:57:40.531955+0000","last_clean":"2023-12-19T17:57:40.531955+0000","last_became_active":"2023-12-19T17:57:40.063957+0000","last_became_peered":"2023-12-19T17:57:40.063957+0000","last_unstale":"2023-12-19T17:57:40.531955+0000","last_undegraded":"2023-12-19T17:57:40.531955+0000","last_fullsized":"2023-12-19T17:57:40.531955+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:38:35.525981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.554690+0000","last_change":"2023-12-19T17:57:40.551811+0000","last_active":"2023-12-19T17:57:40.554690+0000","last_peered":"2023-12-19T17:57:40.554690+0000","last_clean":"2023-12-19T17:57:40.554690+0000","last_became_active":"2023-12-19T17:57:40.551354+0000","last_became_peered":"2023-12-19T17:57:40.551354+0000","last_unstale":"2023-12-19T17:57:40.554690+0000","last_undegraded":"2023-12-19T17:57:40.554690+0000","last_fullsized":"2023-12-19T17:57:40.554690+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:20:00.841561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.537836+0000","last_change":"2023-12-19T17:57:40.067008+0000","last_active":"2023-12-19T17:57:40.537836+0000","last_peered":"2023-12-19T17:57:40.537836+0000","last_clean":"2023-12-19T17:57:40.537836+0000","last_became_active":"2023-12-19T17:57:40.066730+0000","last_became_peered":"2023-12-19T17:57:40.066730+0000","last_unstale":"2023-12-19T17:57:40.537836+0000","last_undegraded":"2023-12-19T17:57:40.537836+0000","last_fullsized":"2023-12-19T17:57:40.537836+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:41:42.776082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.545805+0000","last_change":"2023-12-19T17:57:40.542049+0000","last_active":"2023-12-19T17:57:40.545805+0000","last_peered":"2023-12-19T17:57:40.545805+0000","last_clean":"2023-12-19T17:57:40.545805+0000","last_became_active":"2023-12-19T17:57:40.541728+0000","last_became_peered":"2023-12-19T17:57:40.541728+0000","last_unstale":"2023-12-19T17:57:40.545805+0000","last_undegraded":"2023-12-19T17:57:40.545805+0000","last_fullsized":"2023-12-19T17:57:40.545805+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:00:44.145016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.522661+0000","last_change":"2023-12-19T17:57:40.056628+0000","last_active":"2023-12-19T17:57:40.522661+0000","last_peered":"2023-12-19T17:57:40.522661+0000","last_clean":"2023-12-19T17:57:40.522661+0000","last_became_active":"2023-12-19T17:57:40.056340+0000","last_became_peered":"2023-12-19T17:57:40.056340+0000","last_unstale":"2023-12-19T17:57:40.522661+0000","last_undegraded":"2023-12-19T17:57:40.522661+0000","last_fullsized":"2023-12-19T17:57:40.522661+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:46:58.173739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555349+0000","last_change":"2023-12-19T17:57:40.542871+0000","last_active":"2023-12-19T17:57:40.555349+0000","last_peered":"2023-12-19T17:57:40.555349+0000","last_clean":"2023-12-19T17:57:40.555349+0000","last_became_active":"2023-12-19T17:57:40.542533+0000","last_became_peered":"2023-12-19T17:57:40.542533+0000","last_unstale":"2023-12-19T17:57:40.555349+0000","last_undegraded":"2023-12-19T17:57:40.555349+0000","last_fullsized":"2023-12-19T17:57:40.555349+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:48:38.782646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.527697+0000","last_change":"2023-12-19T17:57:40.051875+0000","last_active":"2023-12-19T17:57:40.527697+0000","last_peered":"2023-12-19T17:57:40.527697+0000","last_clean":"2023-12-19T17:57:40.527697+0000","last_became_active":"2023-12-19T17:57:40.051571+0000","last_became_peered":"2023-12-19T17:57:40.051571+0000","last_unstale":"2023-12-19T17:57:40.527697+0000","last_undegraded":"2023-12-19T17:57:40.527697+0000","last_fullsized":"2023-12-19T17:57:40.527697+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:11:23.185297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.553447+0000","last_change":"2023-12-19T17:57:40.550829+0000","last_active":"2023-12-19T17:57:40.553447+0000","last_peered":"2023-12-19T17:57:40.553447+0000","last_clean":"2023-12-19T17:57:40.553447+0000","last_became_active":"2023-12-19T17:57:40.550300+0000","last_became_peered":"2023-12-19T17:57:40.550300+0000","last_unstale":"2023-12-19T17:57:40.553447+0000","last_undegraded":"2023-12-19T17:57:40.553447+0000","last_fullsized":"2023-12-19T17:57:40.553447+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:25.416611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557659+0000","last_change":"2023-12-19T17:57:40.065786+0000","last_active":"2023-12-19T17:57:40.557659+0000","last_peered":"2023-12-19T17:57:40.557659+0000","last_clean":"2023-12-19T17:57:40.557659+0000","last_became_active":"2023-12-19T17:57:40.065390+0000","last_became_peered":"2023-12-19T17:57:40.065390+0000","last_unstale":"2023-12-19T17:57:40.557659+0000","last_undegraded":"2023-12-19T17:57:40.557659+0000","last_fullsized":"2023-12-19T17:57:40.557659+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:59.357090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"150'269","reported_seq":859,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.057992+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T17:57:40.057992+0000","last_peered":"2023-12-19T17:57:40.057992+0000","last_clean":"2023-12-19T17:57:40.057992+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T17:57:40.057992+0000","last_undegraded":"2023-12-19T17:57:40.057992+0000","last_fullsized":"2023-12-19T17:57:40.057992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1442,"num_read_kb":2969,"num_write":435,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1442,"num_read_kb":2969,"num_write":435,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510343,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37672,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334168,"statfs":{"total":96636764160,"available":96598188032,"internally_reserved":0,"allocated":2588672,"data_stored":1627538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.51100000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88600000000000001,"5min":1.012,"15min":1.0209999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73899999999999999,"15min":0.73799999999999999},"min":{"1min":0.56100000000000005,"5min":0.41299999999999998,"15min":0.41199999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.77200000000000002}]},{"osd":1,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75800000000000001,"15min":0.746},"min":{"1min":0.57399999999999995,"5min":0.442,"15min":0.442},"max":{"1min":0.96699999999999997,"5min":0.997,"15min":0.997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.53600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.96899999999999997,"5min":1.0169999999999999,"15min":1.05},"last":0.99399999999999999}]},{"osd":2,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73199999999999998,"15min":0.72799999999999998},"min":{"1min":0.59599999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89100000000000001,"5min":0.92400000000000004,"15min":0.97999999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79000000000000004,"15min":0.78500000000000003},"min":{"1min":0.56699999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.87}]},{"osd":3,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.54800000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76900000000000002,"15min":0.77700000000000002},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0109999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.41999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.93899999999999995,"15min":0.94299999999999995},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61399999999999999,"15min":0.60799999999999998},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.89000000000000001}]},{"osd":5,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.64000000000000001,"15min":0.626},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.36799999999999999},"max":{"1min":0.875,"5min":1.044,"15min":1.044},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.47199999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.0980000000000001}]},{"osd":6,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.54100000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.89100000000000001,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":1.071},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.52000000000000002,"5min":0.45900000000000002,"15min":0.44500000000000001},"max":{"1min":0.91000000000000003,"5min":1.075,"15min":1.075},"last":1.016}]}]},{"osd":6,"up_from":15,"seq":64424510343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41212,"kb_used_data":1972,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330628,"statfs":{"total":96636764160,"available":96594563072,"internally_reserved":0,"allocated":2019328,"data_stored":1046241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.48799999999999999,"5min":0.40300000000000002,"15min":0.36799999999999999},"max":{"1min":0.995,"5min":0.995,"15min":1.02},"last":0.878},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.747,"15min":0.73799999999999999},"min":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92900000000000005,"5min":0.95699999999999996,"15min":1.329},"last":0.71999999999999997}]},{"osd":1,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.75,"15min":0.73799999999999999},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.42899999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.1419999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.77200000000000002,"15min":0.75700000000000001},"min":{"1min":0.65600000000000003,"5min":0.47799999999999998,"15min":0.45200000000000001},"max":{"1min":0.96499999999999997,"5min":1.026,"15min":1.026},"last":0.78300000000000003}]},{"osd":2,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78200000000000003,"15min":0.76300000000000001},"min":{"1min":0.61299999999999999,"5min":0.497,"15min":0.48799999999999999},"max":{"1min":0.97099999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76000000000000001,"15min":0.76900000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.497},"max":{"1min":0.97999999999999998,"5min":1.0029999999999999,"15min":1.853},"last":0.68899999999999995}]},{"osd":3,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80100000000000005,"15min":0.79400000000000004},"min":{"1min":0.63700000000000001,"5min":0.59899999999999998,"15min":0.45400000000000001},"max":{"1min":0.93799999999999994,"5min":1.0149999999999999,"15min":1.542},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75800000000000001,"15min":0.73699999999999999},"min":{"1min":0.57099999999999995,"5min":0.54700000000000004,"15min":0.48799999999999999},"max":{"1min":0.94699999999999995,"5min":0.98699999999999999,"15min":1.04},"last":0.73199999999999998}]},{"osd":4,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.36799999999999999},"max":{"1min":0.85399999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.499,"5min":0.41499999999999998,"15min":0.372},"max":{"1min":1.131,"5min":1.131,"15min":1.3919999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.46600000000000003,"5min":0.442,"15min":0.34100000000000003},"max":{"1min":0.86399999999999999,"5min":0.91000000000000003,"15min":1.0920000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.54700000000000004,"5min":0.44700000000000001,"15min":0.41399999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.59399999999999997}]},{"osd":7,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.57899999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1,"5min":1.0580000000000001,"15min":1.4259999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.71599999999999997},"min":{"1min":0.59499999999999997,"5min":0.503,"15min":0.42699999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0369999999999999},"last":0.65700000000000003}]}]},{"osd":5,"up_from":15,"seq":64424510343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45284,"kb_used_data":1948,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94326556,"statfs":{"total":96636764160,"available":96590393344,"internally_reserved":0,"allocated":1994752,"data_stored":1037170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.40799999999999997},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.1120000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.753,"15min":0.748},"min":{"1min":0.52200000000000002,"5min":0.48899999999999999,"15min":0.374},"max":{"1min":0.95799999999999996,"5min":1.0289999999999999,"15min":1.073},"last":0.73899999999999999}]},{"osd":1,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77000000000000002,"15min":0.76700000000000002},"min":{"1min":0.69199999999999995,"5min":0.497,"15min":0.497},"max":{"1min":0.89600000000000002,"5min":1.0029999999999999,"15min":1.1659999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.75,"15min":0.746},"min":{"1min":0.57999999999999996,"5min":0.51800000000000002,"15min":0.50800000000000001},"max":{"1min":1.012,"5min":1.0249999999999999,"15min":1.0469999999999999},"last":0.85799999999999998}]},{"osd":2,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78400000000000003,"15min":0.78300000000000003},"min":{"1min":0.53000000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0409999999999999,"5min":1.05,"15min":1.1040000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.59299999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91600000000000004,"5min":0.95899999999999996,"15min":1.0349999999999999},"last":0.77000000000000002}]},{"osd":3,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.64500000000000002,"5min":0.56399999999999995,"15min":0.52700000000000002},"max":{"1min":0.93100000000000005,"5min":0.97099999999999997,"15min":1.131},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.71399999999999997,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.98799999999999999,"5min":0.99299999999999999,"15min":1.125},"last":0.67000000000000004}]},{"osd":4,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.434,"5min":0.38900000000000001,"15min":0.35699999999999998},"max":{"1min":0.83299999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.88},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.41399999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.92200000000000004},"last":0.92600000000000005}]},{"osd":6,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.48999999999999999,"5min":0.46600000000000003,"15min":0.41199999999999998},"max":{"1min":0.86199999999999999,"5min":0.875,"15min":1.048},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65300000000000002,"15min":0.64600000000000002},"min":{"1min":0.53300000000000003,"5min":0.44400000000000001,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":0.85499999999999998,"15min":0.97699999999999998},"last":0.82699999999999996}]},{"osd":7,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.66100000000000003,"15min":0.66800000000000004},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":0.97399999999999998,"15min":1.016},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.55000000000000004,"5min":0.49399999999999999,"15min":0.41299999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.0289999999999999},"last":0.71499999999999997}]}]},{"osd":4,"up_from":15,"seq":64424510343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59252,"kb_used_data":1964,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312588,"statfs":{"total":96636764160,"available":96576090112,"internally_reserved":0,"allocated":2011136,"data_stored":1044747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73199999999999998,"15min":0.74199999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.38300000000000001},"max":{"1min":0.95599999999999996,"5min":1.036,"15min":1.056},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.73899999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.40400000000000003},"max":{"1min":0.92500000000000004,"5min":0.95199999999999996,"15min":1.0620000000000001},"last":0.78100000000000003}]},{"osd":1,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.755,"15min":0.76000000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":0.93999999999999995,"5min":0.995,"15min":1.0980000000000001},"last":0.75},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.753,"15min":0.755},"min":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.46400000000000002},"max":{"1min":0.92900000000000005,"5min":1.0489999999999999,"15min":1.0549999999999999},"last":0.82699999999999996}]},{"osd":2,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.76400000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.94399999999999995,"5min":1.026,"15min":1.042},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78400000000000003,"15min":0.77300000000000002},"min":{"1min":0.63300000000000001,"5min":0.496,"15min":0.46100000000000002},"max":{"1min":0.93000000000000005,"5min":1.0149999999999999,"15min":1.018},"last":0.80000000000000004}]},{"osd":3,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75,"15min":0.748},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.94399999999999995,"5min":0.96799999999999997,"15min":1.0229999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.78300000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.499},"max":{"1min":0.97099999999999997,"5min":0.997,"15min":1.0049999999999999},"last":0.81499999999999995}]},{"osd":5,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.51100000000000001,"5min":0.40799999999999997,"15min":0.36299999999999999},"max":{"1min":0.88400000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63700000000000001,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.379},"max":{"1min":0.88900000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72099999999999997}]},{"osd":6,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65200000000000002,"15min":0.64100000000000001},"min":{"1min":0.46500000000000002,"5min":0.44700000000000001,"15min":0.41399999999999998},"max":{"1min":0.93500000000000005,"5min":1.03,"15min":1.079},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70799999999999996,"15min":0.68100000000000005},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.34599999999999997},"max":{"1min":0.91900000000000004,"5min":0.98899999999999999,"15min":0.99099999999999999},"last":0.70099999999999996}]},{"osd":7,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68999999999999995,"15min":0.68200000000000005},"min":{"1min":0.54600000000000004,"5min":0.44800000000000001,"15min":0.40400000000000003},"max":{"1min":0.97799999999999998,"5min":1.075,"15min":1.075},"last":0.629},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.71799999999999997},"min":{"1min":0.59799999999999998,"5min":0.51300000000000001,"15min":0.45500000000000002},"max":{"1min":0.97699999999999998,"5min":1.0629999999999999,"15min":1.1160000000000001},"last":0.67800000000000005}]}]},{"osd":3,"up_from":15,"seq":64424510343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46968,"kb_used_data":1968,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324872,"statfs":{"total":96636764160,"available":96588668928,"internally_reserved":0,"allocated":2015232,"data_stored":1044837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.317,"5min":0.30599999999999999,"15min":0.28000000000000003},"max":{"1min":0.68899999999999995,"5min":0.70799999999999996,"15min":0.74399999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.52100000000000002},"min":{"1min":0.42399999999999999,"5min":0.313,"15min":0.29499999999999998},"max":{"1min":0.64900000000000002,"5min":0.76700000000000002,"15min":1.01},"last":0.42499999999999999}]},{"osd":1,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49399999999999999,"15min":0.48299999999999998},"min":{"1min":0.36799999999999999,"5min":0.32800000000000001,"15min":0.307},"max":{"1min":0.67700000000000005,"5min":0.72199999999999998,"15min":0.83899999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66600000000000004,"5min":0.78300000000000003,"15min":0.873},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76500000000000001,"5min":0.82999999999999996,"15min":0.92100000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.51200000000000001,"5min":0.41699999999999998,"15min":0.38600000000000001},"max":{"1min":0.80300000000000005,"5min":0.93000000000000005,"15min":0.98699999999999999},"last":0.51300000000000001}]},{"osd":4,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.82299999999999995},"min":{"1min":0.69399999999999995,"5min":0.60199999999999998,"15min":0.52700000000000002},"max":{"1min":1.1140000000000001,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.83899999999999997,"15min":0.83199999999999996},"min":{"1min":0.68100000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.135,"5min":1.268,"15min":1.371},"last":0.64000000000000001}]},{"osd":5,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.83699999999999997,"15min":0.83299999999999996},"min":{"1min":0.70499999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0189999999999999,"5min":1.296,"15min":1.296},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83299999999999996,"15min":0.82199999999999995},"min":{"1min":0.65300000000000002,"5min":0.61299999999999999,"15min":0.52800000000000002},"max":{"1min":1.1200000000000001,"5min":1.141,"15min":1.2290000000000001},"last":0.67100000000000004}]},{"osd":6,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.86699999999999999,"15min":0.85199999999999998},"min":{"1min":0.68100000000000005,"5min":0.61399999999999999,"15min":0.58799999999999997},"max":{"1min":1.0840000000000001,"5min":1.359,"15min":1.359},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.86799999999999999,"15min":0.84899999999999998},"min":{"1min":0.71999999999999997,"5min":0.60499999999999998,"15min":0.59599999999999997},"max":{"1min":1.175,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91700000000000004,"15min":0.89700000000000002},"min":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.60599999999999998},"max":{"1min":1.21,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.78900000000000003,"5min":0.63600000000000001,"15min":0.59599999999999997},"max":{"1min":1.101,"5min":1.288,"15min":1.3069999999999999},"last":0.71699999999999997}]}]},{"osd":2,"up_from":15,"seq":64424510342,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63352,"kb_used_data":1968,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308488,"statfs":{"total":96636764160,"available":96571891712,"internally_reserved":0,"allocated":2015232,"data_stored":1045293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44700000000000001,"15min":0.45800000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59999999999999998,"5min":0.71499999999999997,"15min":0.80800000000000005},"last":0.373},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32000000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88600000000000001},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.62,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.36499999999999999},"max":{"1min":0.80200000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.51900000000000002},"min":{"1min":0.375,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.67900000000000005,"5min":0.79600000000000004,"15min":0.93300000000000005},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.60799999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82699999999999996,"5min":0.88,"15min":0.93300000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.38400000000000001,"15min":0.35599999999999998},"max":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.93300000000000005},"last":0.55000000000000004}]},{"osd":4,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.80100000000000005,"15min":0.80600000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.54000000000000004},"max":{"1min":1.1020000000000001,"5min":1.1639999999999999,"15min":1.3360000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79800000000000004,"15min":0.79000000000000004},"min":{"1min":0.53600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.1180000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80300000000000005,"15min":0.80500000000000005},"min":{"1min":0.60499999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.071,"5min":1.339,"15min":1.339},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.083,"5min":1.351,"15min":1.351},"last":0.85599999999999998}]},{"osd":6,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84199999999999997,"15min":0.84299999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.55800000000000005},"max":{"1min":1.0349999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.46600000000000003},"max":{"1min":1.1399999999999999,"5min":1.1619999999999999,"15min":1.171},"last":0.83899999999999997}]},{"osd":7,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.82699999999999996},"min":{"1min":0.69399999999999995,"5min":0.58699999999999997,"15min":0.53500000000000003},"max":{"1min":1.21,"5min":1.2629999999999999,"15min":1.3580000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.64900000000000002,"5min":0.63300000000000001,"15min":0.61899999999999999},"max":{"1min":1.1279999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.81399999999999995}]}]},{"osd":1,"up_from":15,"seq":64424510343,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55140,"kb_used_data":1948,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316700,"statfs":{"total":96636764160,"available":96580300800,"internally_reserved":0,"allocated":1994752,"data_stored":1037170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.45300000000000001,"15min":0.45800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.627,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53200000000000003,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.34799999999999998,"15min":0.34499999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.89200000000000002},"last":0.68999999999999995}]},{"osd":2,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53100000000000003,"15min":0.52100000000000002},"min":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.308},"max":{"1min":0.71999999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34599999999999997},"max":{"1min":0.80000000000000004,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.58299999999999996}]},{"osd":3,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.49199999999999999,"5min":0.45600000000000002,"15min":0.42299999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.42499999999999999,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.85199999999999998},"last":0.53200000000000003}]},{"osd":4,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81699999999999995,"15min":0.81000000000000005},"min":{"1min":0.60499999999999998,"5min":0.54500000000000004,"15min":0.53200000000000003},"max":{"1min":1.091,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80200000000000005,"15min":0.80700000000000005},"min":{"1min":0.56999999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0369999999999999,"5min":1.355,"15min":1.355},"last":0.71199999999999997}]},{"osd":5,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.82499999999999996,"15min":0.81200000000000006},"min":{"1min":0.57599999999999996,"5min":0.55600000000000005,"15min":0.54600000000000004},"max":{"1min":1.214,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.82699999999999996,"15min":0.81999999999999995},"min":{"1min":0.623,"5min":0.61899999999999999,"15min":0.59699999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.74299999999999999}]},{"osd":6,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.82199999999999995,"15min":0.82099999999999995},"min":{"1min":0.64800000000000002,"5min":0.59099999999999997,"15min":0.52200000000000002},"max":{"1min":1.026,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.83899999999999997,"15min":0.82999999999999996},"min":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.157,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.88300000000000001,"15min":0.86399999999999999},"min":{"1min":0.71799999999999997,"5min":0.64500000000000002,"15min":0.625},"max":{"1min":1.1950000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.84299999999999997,"15min":0.85399999999999998},"min":{"1min":0.63300000000000001,"5min":0.628,"15min":0.59499999999999997},"max":{"1min":1.177,"5min":1.258,"15min":1.258},"last":0.81499999999999995}]}]},{"osd":0,"up_from":14,"seq":60129543048,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55720,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316120,"statfs":{"total":96636764160,"available":96579706880,"internally_reserved":0,"allocated":2588672,"data_stored":1627538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46000000000000002,"15min":0.45500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64100000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.29599999999999999},"max":{"1min":0.71499999999999997,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.309}]},{"osd":2,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.35699999999999998,"15min":0.34300000000000003},"max":{"1min":0.60899999999999999,"5min":0.78500000000000003,"15min":0.83299999999999996},"last":0.437},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65900000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.35499999999999998}]},{"osd":3,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.65000000000000002,"5min":0.84799999999999998,"15min":0.90400000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.501,"5min":0.497,"15min":0.45000000000000001},"max":{"1min":0.71299999999999997,"5min":0.90000000000000002,"15min":0.93500000000000005},"last":0.55000000000000004}]},{"osd":4,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.44700000000000001},"max":{"1min":1.1080000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52400000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80000000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.50700000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.169},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81799999999999995,"15min":0.82499999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.351},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1619999999999999,"5min":1.173,"15min":1.2350000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82699999999999996,"15min":0.83099999999999996},"min":{"1min":0.63300000000000001,"5min":0.60299999999999998,"15min":0.505},"max":{"1min":1.079,"5min":1.1020000000000001,"15min":1.329},"last":0.63300000000000001}]},{"osd":7,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84999999999999998,"15min":0.85599999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2909999999999999},"last":0.64300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T17:57:44.649 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T17:57:44.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.831+0000 7efde1fa2640 1 Processor -- start 2023-12-19T17:57:44.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.831+0000 7efde1fa2640 1 -- start start 2023-12-19T17:57:44.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde1fa2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efddc0cc0a0 0x7efddc0cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde1fa2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc159d20 0x7efddc0cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde1fa2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efddc0cdee0 0x7efddc15d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde1fa2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc0c9c20 con 0x7efddc0cc0a0 2023-12-19T17:57:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde1fa2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc0adde0 con 0x7efddc0cdee0 2023-12-19T17:57:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde1fa2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0b28d0 con 0x7efddc159d20 2023-12-19T17:57:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde0fa0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efddc0cc0a0 0x7efddc0cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde0fa0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efddc0cc0a0 0x7efddc0cc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46290/0 (socket says 172.21.15.26:46290) 2023-12-19T17:57:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde0fa0640 1 -- 172.21.15.26:0/2876537747 learned_addr learned my addr 172.21.15.26:0/2876537747 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde17a1640 1 --2- 172.21.15.26:0/2876537747 >> v2:172.21.15.160:3300/0 conn(0x7efddc0cdee0 0x7efddc15d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efddbfff640 1 --2- 172.21.15.26:0/2876537747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc159d20 0x7efddc0cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde0fa0640 1 -- 172.21.15.26:0/2876537747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc159d20 msgr2=0x7efddc0cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde0fa0640 1 --2- 172.21.15.26:0/2876537747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc159d20 0x7efddc0cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde0fa0640 1 -- 172.21.15.26:0/2876537747 >> v2:172.21.15.160:3300/0 conn(0x7efddc0cdee0 msgr2=0x7efddc15d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde0fa0640 1 --2- 172.21.15.26:0/2876537747 >> v2:172.21.15.160:3300/0 conn(0x7efddc0cdee0 0x7efddc15d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde0fa0640 1 -- 172.21.15.26:0/2876537747 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc0ccf00 con 0x7efddc0cc0a0 2023-12-19T17:57:44.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efddbfff640 1 --2- 172.21.15.26:0/2876537747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc159d20 0x7efddc0cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:57:44.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.835+0000 7efde0fa0640 1 --2- 172.21.15.26:0/2876537747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cc0a0 0x7efddc0cc480 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7efddc06c9f0 tx=0x7efdd001c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31da99f5c5fa198c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:44.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efddb7fe640 1 -- 172.21.15.26:0/2876537747 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd00ab600 con 0x7efddc0cc0a0 2023-12-19T17:57:44.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efddb7fe640 1 -- 172.21.15.26:0/2876537747 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd00a7050 con 0x7efddc0cc0a0 2023-12-19T17:57:44.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efddb7fe640 1 -- 172.21.15.26:0/2876537747 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd00ab980 con 0x7efddc0cc0a0 2023-12-19T17:57:44.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 -- 172.21.15.26:0/2876537747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cc0a0 msgr2=0x7efddc0cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 --2- 172.21.15.26:0/2876537747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cc0a0 0x7efddc0cc480 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7efddc06c9f0 tx=0x7efdd001c180 comp rx=0 tx=0).stop 2023-12-19T17:57:44.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 -- 172.21.15.26:0/2876537747 shutdown_connections 2023-12-19T17:57:44.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 --2- 172.21.15.26:0/2876537747 >> v2:172.21.15.160:3300/0 conn(0x7efddc0cdee0 0x7efddc15d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 --2- 172.21.15.26:0/2876537747 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc159d20 0x7efddc0cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 --2- 172.21.15.26:0/2876537747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cc0a0 0x7efddc0cc480 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 -- 172.21.15.26:0/2876537747 >> 172.21.15.26:0/2876537747 conn(0x7efddc06b0f0 msgr2=0x7efddc06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:44.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 -- 172.21.15.26:0/2876537747 shutdown_connections 2023-12-19T17:57:44.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 -- 172.21.15.26:0/2876537747 wait complete. 2023-12-19T17:57:44.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 Processor -- start 2023-12-19T17:57:44.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 -- start start 2023-12-19T17:57:44.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cc0a0 0x7efddc152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc0cdee0 0x7efddc151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc159d20 0x7efddc153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc15c9e0 con 0x7efddc0cc0a0 2023-12-19T17:57:44.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc15c550 con 0x7efddc159d20 2023-12-19T17:57:44.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde1fa2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc154bd0 con 0x7efddc0cdee0 2023-12-19T17:57:44.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde0fa0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cc0a0 0x7efddc152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.839+0000 7efde0fa0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cc0a0 0x7efddc152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46304/0 (socket says 172.21.15.26:46304) 2023-12-19T17:57:44.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efde0fa0640 1 -- 172.21.15.26:0/2539858224 learned_addr learned my addr 172.21.15.26:0/2539858224 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:44.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efde17a1640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc159d20 0x7efddc153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efddbfff640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc0cdee0 0x7efddc151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efde0fa0640 1 -- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc0cdee0 msgr2=0x7efddc151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efde0fa0640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc0cdee0 0x7efddc151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efde0fa0640 1 -- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc159d20 msgr2=0x7efddc153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:44.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efde0fa0640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc159d20 0x7efddc153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:44.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efde0fa0640 1 -- 172.21.15.26:0/2539858224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd00094d0 con 0x7efddc0cc0a0 2023-12-19T17:57:44.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efde17a1640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc159d20 0x7efddc153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:44.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efddbfff640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc0cdee0 0x7efddc151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:44.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efde0fa0640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cc0a0 0x7efddc152f20 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ae040 tx=0x7efdd001a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:44.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efdd97fa640 1 -- 172.21.15.26:0/2539858224 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd00b6620 con 0x7efddc0cc0a0 2023-12-19T17:57:44.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efdd97fa640 1 -- 172.21.15.26:0/2539858224 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd00b67a0 con 0x7efddc0cc0a0 2023-12-19T17:57:44.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efdd97fa640 1 -- 172.21.15.26:0/2539858224 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd00b6b50 con 0x7efddc0cc0a0 2023-12-19T17:57:44.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efde1fa2640 1 -- 172.21.15.26:0/2539858224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc15b940 con 0x7efddc0cc0a0 2023-12-19T17:57:44.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efde1fa2640 1 -- 172.21.15.26:0/2539858224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc15bf00 con 0x7efddc0cc0a0 2023-12-19T17:57:44.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efde1fa2640 1 -- 172.21.15.26:0/2539858224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc0b16c0 con 0x7efddc0cc0a0 2023-12-19T17:57:44.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.843+0000 7efdd97fa640 1 -- 172.21.15.26:0/2539858224 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7efdd00b2040 con 0x7efddc0cc0a0 2023-12-19T17:57:44.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.847+0000 7efdd97fa640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efdc0065b80 0x7efdc0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:44.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.847+0000 7efdd97fa640 1 -- 172.21.15.26:0/2539858224 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7efdd011db00 con 0x7efddc0cc0a0 2023-12-19T17:57:44.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.847+0000 7efddbfff640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efdc0065b80 0x7efdc0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:44.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.847+0000 7efddbfff640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efdc0065b80 0x7efdc0068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efdc8006fd0 tx=0x7efdc8008040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:44.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:44.851+0000 7efdd97fa640 1 -- 172.21.15.26:0/2539858224 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efdd00eb050 con 0x7efddc0cc0a0 2023-12-19T17:57:45.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.003+0000 7efde1fa2640 1 -- 172.21.15.26:0/2539858224 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efddc0cdc80 con 0x7efdc0065b80 2023-12-19T17:57:45.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.011+0000 7efdd97fa640 1 -- 172.21.15.26:0/2539858224 <== mgr.8359 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289729 (secure 0 0 0) 0x7efdc800f830 con 0x7efdc0065b80 2023-12-19T17:57:45.016 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:45.018 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T17:57:45.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.015+0000 7efde1fa2640 1 -- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efdc0065b80 msgr2=0x7efdc0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.015+0000 7efde1fa2640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efdc0065b80 0x7efdc0068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efdc8006fd0 tx=0x7efdc8008040 comp rx=0 tx=0).stop 2023-12-19T17:57:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.015+0000 7efde1fa2640 1 -- 172.21.15.26:0/2539858224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cc0a0 msgr2=0x7efddc152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.015+0000 7efde1fa2640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cc0a0 0x7efddc152f20 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ae040 tx=0x7efdd001a9c0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.015+0000 7efde1fa2640 1 -- 172.21.15.26:0/2539858224 shutdown_connections 2023-12-19T17:57:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.015+0000 7efde1fa2640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efddc159d20 0x7efddc153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.015+0000 7efde1fa2640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efdc0065b80 0x7efdc0068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.015+0000 7efde1fa2640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efddc0cdee0 0x7efddc151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.015+0000 7efde1fa2640 1 --2- 172.21.15.26:0/2539858224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cc0a0 0x7efddc152f20 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.015+0000 7efde1fa2640 1 -- 172.21.15.26:0/2539858224 >> 172.21.15.26:0/2539858224 conn(0x7efddc06b0f0 msgr2=0x7efddc159010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.015+0000 7efde1fa2640 1 -- 172.21.15.26:0/2539858224 shutdown_connections 2023-12-19T17:57:45.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.015+0000 7efde1fa2640 1 -- 172.21.15.26:0/2539858224 wait complete. 2023-12-19T17:57:45.038 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-19T17:57:44.925906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1442,"num_read_kb":2969,"num_write":479,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":46,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":404600,"kb_used_data":16824,"kb_used_omap":0,"kb_used_meta":387712,"kb_avail":754570120,"statfs":{"total":773094113280,"available":772679802880,"internally_reserved":0,"allocated":17227776,"data_stored":9510534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":397017088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.895186"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558796+0000","last_change":"2023-12-19T17:57:40.553913+0000","last_active":"2023-12-19T17:57:40.558796+0000","last_peered":"2023-12-19T17:57:40.558796+0000","last_clean":"2023-12-19T17:57:40.558796+0000","last_became_active":"2023-12-19T17:57:40.553650+0000","last_became_peered":"2023-12-19T17:57:40.553650+0000","last_unstale":"2023-12-19T17:57:40.558796+0000","last_undegraded":"2023-12-19T17:57:40.558796+0000","last_fullsized":"2023-12-19T17:57:40.558796+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:01:15.868496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.545806+0000","last_change":"2023-12-19T17:57:40.537346+0000","last_active":"2023-12-19T17:57:40.545806+0000","last_peered":"2023-12-19T17:57:40.545806+0000","last_clean":"2023-12-19T17:57:40.545806+0000","last_became_active":"2023-12-19T17:57:40.535223+0000","last_became_peered":"2023-12-19T17:57:40.535223+0000","last_unstale":"2023-12-19T17:57:40.545806+0000","last_undegraded":"2023-12-19T17:57:40.545806+0000","last_fullsized":"2023-12-19T17:57:40.545806+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:26.339164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.547274+0000","last_change":"2023-12-19T17:57:40.540455+0000","last_active":"2023-12-19T17:57:40.547274+0000","last_peered":"2023-12-19T17:57:40.547274+0000","last_clean":"2023-12-19T17:57:40.547274+0000","last_became_active":"2023-12-19T17:57:40.540176+0000","last_became_peered":"2023-12-19T17:57:40.540176+0000","last_unstale":"2023-12-19T17:57:40.547274+0000","last_undegraded":"2023-12-19T17:57:40.547274+0000","last_fullsized":"2023-12-19T17:57:40.547274+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:08:31.123349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558520+0000","last_change":"2023-12-19T17:57:40.555239+0000","last_active":"2023-12-19T17:57:40.558520+0000","last_peered":"2023-12-19T17:57:40.558520+0000","last_clean":"2023-12-19T17:57:40.558520+0000","last_became_active":"2023-12-19T17:57:40.554913+0000","last_became_peered":"2023-12-19T17:57:40.554913+0000","last_unstale":"2023-12-19T17:57:40.558520+0000","last_undegraded":"2023-12-19T17:57:40.558520+0000","last_fullsized":"2023-12-19T17:57:40.558520+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:12.715283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.554151+0000","last_change":"2023-12-19T17:57:40.547627+0000","last_active":"2023-12-19T17:57:40.554151+0000","last_peered":"2023-12-19T17:57:40.554151+0000","last_clean":"2023-12-19T17:57:40.554151+0000","last_became_active":"2023-12-19T17:57:40.547346+0000","last_became_peered":"2023-12-19T17:57:40.547346+0000","last_unstale":"2023-12-19T17:57:40.554151+0000","last_undegraded":"2023-12-19T17:57:40.554151+0000","last_fullsized":"2023-12-19T17:57:40.554151+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:49.866445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552666+0000","last_change":"2023-12-19T17:57:40.545255+0000","last_active":"2023-12-19T17:57:40.552666+0000","last_peered":"2023-12-19T17:57:40.552666+0000","last_clean":"2023-12-19T17:57:40.552666+0000","last_became_active":"2023-12-19T17:57:40.544935+0000","last_became_peered":"2023-12-19T17:57:40.544935+0000","last_unstale":"2023-12-19T17:57:40.552666+0000","last_undegraded":"2023-12-19T17:57:40.552666+0000","last_fullsized":"2023-12-19T17:57:40.552666+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:15.573540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.551567+0000","last_change":"2023-12-19T17:57:40.541873+0000","last_active":"2023-12-19T17:57:40.551567+0000","last_peered":"2023-12-19T17:57:40.551567+0000","last_clean":"2023-12-19T17:57:40.551567+0000","last_became_active":"2023-12-19T17:57:40.541543+0000","last_became_peered":"2023-12-19T17:57:40.541543+0000","last_unstale":"2023-12-19T17:57:40.551567+0000","last_undegraded":"2023-12-19T17:57:40.551567+0000","last_fullsized":"2023-12-19T17:57:40.551567+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:24.741840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.553038+0000","last_change":"2023-12-19T17:57:40.542470+0000","last_active":"2023-12-19T17:57:40.553038+0000","last_peered":"2023-12-19T17:57:40.553038+0000","last_clean":"2023-12-19T17:57:40.553038+0000","last_became_active":"2023-12-19T17:57:40.542196+0000","last_became_peered":"2023-12-19T17:57:40.542196+0000","last_unstale":"2023-12-19T17:57:40.553038+0000","last_undegraded":"2023-12-19T17:57:40.553038+0000","last_fullsized":"2023-12-19T17:57:40.553038+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:34.938988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.550136+0000","last_change":"2023-12-19T17:57:40.541758+0000","last_active":"2023-12-19T17:57:40.550136+0000","last_peered":"2023-12-19T17:57:40.550136+0000","last_clean":"2023-12-19T17:57:40.550136+0000","last_became_active":"2023-12-19T17:57:40.541441+0000","last_became_peered":"2023-12-19T17:57:40.541441+0000","last_unstale":"2023-12-19T17:57:40.550136+0000","last_undegraded":"2023-12-19T17:57:40.550136+0000","last_fullsized":"2023-12-19T17:57:40.550136+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:28.187346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.545787+0000","last_change":"2023-12-19T17:57:40.535288+0000","last_active":"2023-12-19T17:57:40.545787+0000","last_peered":"2023-12-19T17:57:40.545787+0000","last_clean":"2023-12-19T17:57:40.545787+0000","last_became_active":"2023-12-19T17:57:40.534872+0000","last_became_peered":"2023-12-19T17:57:40.534872+0000","last_unstale":"2023-12-19T17:57:40.545787+0000","last_undegraded":"2023-12-19T17:57:40.545787+0000","last_fullsized":"2023-12-19T17:57:40.545787+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:12:43.624773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.538550+0000","last_change":"2023-12-19T17:57:40.537490+0000","last_active":"2023-12-19T17:57:40.538550+0000","last_peered":"2023-12-19T17:57:40.538550+0000","last_clean":"2023-12-19T17:57:40.538550+0000","last_became_active":"2023-12-19T17:57:40.536529+0000","last_became_peered":"2023-12-19T17:57:40.536529+0000","last_unstale":"2023-12-19T17:57:40.538550+0000","last_undegraded":"2023-12-19T17:57:40.538550+0000","last_fullsized":"2023-12-19T17:57:40.538550+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:31:36.716597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.547856+0000","last_change":"2023-12-19T17:57:40.540398+0000","last_active":"2023-12-19T17:57:40.547856+0000","last_peered":"2023-12-19T17:57:40.547856+0000","last_clean":"2023-12-19T17:57:40.547856+0000","last_became_active":"2023-12-19T17:57:40.540079+0000","last_became_peered":"2023-12-19T17:57:40.540079+0000","last_unstale":"2023-12-19T17:57:40.547856+0000","last_undegraded":"2023-12-19T17:57:40.547856+0000","last_fullsized":"2023-12-19T17:57:40.547856+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:46.267225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558470+0000","last_change":"2023-12-19T17:57:40.554375+0000","last_active":"2023-12-19T17:57:40.558470+0000","last_peered":"2023-12-19T17:57:40.558470+0000","last_clean":"2023-12-19T17:57:40.558470+0000","last_became_active":"2023-12-19T17:57:40.554119+0000","last_became_peered":"2023-12-19T17:57:40.554119+0000","last_unstale":"2023-12-19T17:57:40.558470+0000","last_undegraded":"2023-12-19T17:57:40.558470+0000","last_fullsized":"2023-12-19T17:57:40.558470+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:43:23.839334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557283+0000","last_change":"2023-12-19T17:57:40.549550+0000","last_active":"2023-12-19T17:57:40.557283+0000","last_peered":"2023-12-19T17:57:40.557283+0000","last_clean":"2023-12-19T17:57:40.557283+0000","last_became_active":"2023-12-19T17:57:40.549263+0000","last_became_peered":"2023-12-19T17:57:40.549263+0000","last_unstale":"2023-12-19T17:57:40.557283+0000","last_undegraded":"2023-12-19T17:57:40.557283+0000","last_fullsized":"2023-12-19T17:57:40.557283+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:10.867122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558239+0000","last_change":"2023-12-19T17:57:40.553519+0000","last_active":"2023-12-19T17:57:40.558239+0000","last_peered":"2023-12-19T17:57:40.558239+0000","last_clean":"2023-12-19T17:57:40.558239+0000","last_became_active":"2023-12-19T17:57:40.552994+0000","last_became_peered":"2023-12-19T17:57:40.552994+0000","last_unstale":"2023-12-19T17:57:40.558239+0000","last_undegraded":"2023-12-19T17:57:40.558239+0000","last_fullsized":"2023-12-19T17:57:40.558239+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:09.991187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.551724+0000","last_change":"2023-12-19T17:57:40.548553+0000","last_active":"2023-12-19T17:57:40.551724+0000","last_peered":"2023-12-19T17:57:40.551724+0000","last_clean":"2023-12-19T17:57:40.551724+0000","last_became_active":"2023-12-19T17:57:40.547224+0000","last_became_peered":"2023-12-19T17:57:40.547224+0000","last_unstale":"2023-12-19T17:57:40.551724+0000","last_undegraded":"2023-12-19T17:57:40.551724+0000","last_fullsized":"2023-12-19T17:57:40.551724+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:49:41.869352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552351+0000","last_change":"2023-12-19T17:57:40.542681+0000","last_active":"2023-12-19T17:57:40.552351+0000","last_peered":"2023-12-19T17:57:40.552351+0000","last_clean":"2023-12-19T17:57:40.552351+0000","last_became_active":"2023-12-19T17:57:40.541340+0000","last_became_peered":"2023-12-19T17:57:40.541340+0000","last_unstale":"2023-12-19T17:57:40.552351+0000","last_undegraded":"2023-12-19T17:57:40.552351+0000","last_fullsized":"2023-12-19T17:57:40.552351+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:08:31.123349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.537686+0000","last_change":"2023-12-19T17:57:40.531453+0000","last_active":"2023-12-19T17:57:40.537686+0000","last_peered":"2023-12-19T17:57:40.537686+0000","last_clean":"2023-12-19T17:57:40.537686+0000","last_became_active":"2023-12-19T17:57:40.531187+0000","last_became_peered":"2023-12-19T17:57:40.531187+0000","last_unstale":"2023-12-19T17:57:40.537686+0000","last_undegraded":"2023-12-19T17:57:40.537686+0000","last_fullsized":"2023-12-19T17:57:40.537686+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:45:09.414432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.537347+0000","last_change":"2023-12-19T17:57:40.536450+0000","last_active":"2023-12-19T17:57:40.537347+0000","last_peered":"2023-12-19T17:57:40.537347+0000","last_clean":"2023-12-19T17:57:40.537347+0000","last_became_active":"2023-12-19T17:57:40.533713+0000","last_became_peered":"2023-12-19T17:57:40.533713+0000","last_unstale":"2023-12-19T17:57:40.537347+0000","last_undegraded":"2023-12-19T17:57:40.537347+0000","last_fullsized":"2023-12-19T17:57:40.537347+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:43:03.209078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.544357+0000","last_change":"2023-12-19T17:57:40.541446+0000","last_active":"2023-12-19T17:57:40.544357+0000","last_peered":"2023-12-19T17:57:40.544357+0000","last_clean":"2023-12-19T17:57:40.544357+0000","last_became_active":"2023-12-19T17:57:40.541174+0000","last_became_peered":"2023-12-19T17:57:40.541174+0000","last_unstale":"2023-12-19T17:57:40.544357+0000","last_undegraded":"2023-12-19T17:57:40.544357+0000","last_fullsized":"2023-12-19T17:57:40.544357+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:59.428689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555512+0000","last_change":"2023-12-19T17:57:40.552259+0000","last_active":"2023-12-19T17:57:40.555512+0000","last_peered":"2023-12-19T17:57:40.555512+0000","last_clean":"2023-12-19T17:57:40.555512+0000","last_became_active":"2023-12-19T17:57:40.547786+0000","last_became_peered":"2023-12-19T17:57:40.547786+0000","last_unstale":"2023-12-19T17:57:40.555512+0000","last_undegraded":"2023-12-19T17:57:40.555512+0000","last_fullsized":"2023-12-19T17:57:40.555512+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:33.503868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.536641+0000","last_change":"2023-12-19T17:57:40.535867+0000","last_active":"2023-12-19T17:57:40.536641+0000","last_peered":"2023-12-19T17:57:40.536641+0000","last_clean":"2023-12-19T17:57:40.536641+0000","last_became_active":"2023-12-19T17:57:40.535608+0000","last_became_peered":"2023-12-19T17:57:40.535608+0000","last_unstale":"2023-12-19T17:57:40.536641+0000","last_undegraded":"2023-12-19T17:57:40.536641+0000","last_fullsized":"2023-12-19T17:57:40.536641+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:55:01.298850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557229+0000","last_change":"2023-12-19T17:57:40.550030+0000","last_active":"2023-12-19T17:57:40.557229+0000","last_peered":"2023-12-19T17:57:40.557229+0000","last_clean":"2023-12-19T17:57:40.557229+0000","last_became_active":"2023-12-19T17:57:40.547042+0000","last_became_peered":"2023-12-19T17:57:40.547042+0000","last_unstale":"2023-12-19T17:57:40.557229+0000","last_undegraded":"2023-12-19T17:57:40.557229+0000","last_fullsized":"2023-12-19T17:57:40.557229+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:50:55.497595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.536491+0000","last_change":"2023-12-19T17:57:40.534296+0000","last_active":"2023-12-19T17:57:40.536491+0000","last_peered":"2023-12-19T17:57:40.536491+0000","last_clean":"2023-12-19T17:57:40.536491+0000","last_became_active":"2023-12-19T17:57:40.534018+0000","last_became_peered":"2023-12-19T17:57:40.534018+0000","last_unstale":"2023-12-19T17:57:40.536491+0000","last_undegraded":"2023-12-19T17:57:40.536491+0000","last_fullsized":"2023-12-19T17:57:40.536491+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:08:52.714270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555374+0000","last_change":"2023-12-19T17:57:40.543166+0000","last_active":"2023-12-19T17:57:40.555374+0000","last_peered":"2023-12-19T17:57:40.555374+0000","last_clean":"2023-12-19T17:57:40.555374+0000","last_became_active":"2023-12-19T17:57:40.542918+0000","last_became_peered":"2023-12-19T17:57:40.542918+0000","last_unstale":"2023-12-19T17:57:40.555374+0000","last_undegraded":"2023-12-19T17:57:40.555374+0000","last_fullsized":"2023-12-19T17:57:40.555374+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:33.503868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.551141+0000","last_change":"2023-12-19T17:57:40.547916+0000","last_active":"2023-12-19T17:57:40.551141+0000","last_peered":"2023-12-19T17:57:40.551141+0000","last_clean":"2023-12-19T17:57:40.551141+0000","last_became_active":"2023-12-19T17:57:40.547146+0000","last_became_peered":"2023-12-19T17:57:40.547146+0000","last_unstale":"2023-12-19T17:57:40.551141+0000","last_undegraded":"2023-12-19T17:57:40.551141+0000","last_fullsized":"2023-12-19T17:57:40.551141+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:06.321345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555144+0000","last_change":"2023-12-19T17:57:40.547229+0000","last_active":"2023-12-19T17:57:40.555144+0000","last_peered":"2023-12-19T17:57:40.555144+0000","last_clean":"2023-12-19T17:57:40.555144+0000","last_became_active":"2023-12-19T17:57:40.546886+0000","last_became_peered":"2023-12-19T17:57:40.546886+0000","last_unstale":"2023-12-19T17:57:40.555144+0000","last_undegraded":"2023-12-19T17:57:40.555144+0000","last_fullsized":"2023-12-19T17:57:40.555144+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:05:16.354884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.533991+0000","last_change":"2023-12-19T17:57:40.530714+0000","last_active":"2023-12-19T17:57:40.533991+0000","last_peered":"2023-12-19T17:57:40.533991+0000","last_clean":"2023-12-19T17:57:40.533991+0000","last_became_active":"2023-12-19T17:57:40.530349+0000","last_became_peered":"2023-12-19T17:57:40.530349+0000","last_unstale":"2023-12-19T17:57:40.533991+0000","last_undegraded":"2023-12-19T17:57:40.533991+0000","last_fullsized":"2023-12-19T17:57:40.533991+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:23:48.874061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558450+0000","last_change":"2023-12-19T17:57:40.552318+0000","last_active":"2023-12-19T17:57:40.558450+0000","last_peered":"2023-12-19T17:57:40.558450+0000","last_clean":"2023-12-19T17:57:40.558450+0000","last_became_active":"2023-12-19T17:57:40.552014+0000","last_became_peered":"2023-12-19T17:57:40.552014+0000","last_unstale":"2023-12-19T17:57:40.558450+0000","last_undegraded":"2023-12-19T17:57:40.558450+0000","last_fullsized":"2023-12-19T17:57:40.558450+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:57:19.937681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557833+0000","last_change":"2023-12-19T17:57:40.551079+0000","last_active":"2023-12-19T17:57:40.557833+0000","last_peered":"2023-12-19T17:57:40.557833+0000","last_clean":"2023-12-19T17:57:40.557833+0000","last_became_active":"2023-12-19T17:57:40.550831+0000","last_became_peered":"2023-12-19T17:57:40.550831+0000","last_unstale":"2023-12-19T17:57:40.557833+0000","last_undegraded":"2023-12-19T17:57:40.557833+0000","last_fullsized":"2023-12-19T17:57:40.557833+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:49:10.047104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.554643+0000","last_change":"2023-12-19T17:57:40.550962+0000","last_active":"2023-12-19T17:57:40.554643+0000","last_peered":"2023-12-19T17:57:40.554643+0000","last_clean":"2023-12-19T17:57:40.554643+0000","last_became_active":"2023-12-19T17:57:40.550690+0000","last_became_peered":"2023-12-19T17:57:40.550690+0000","last_unstale":"2023-12-19T17:57:40.554643+0000","last_undegraded":"2023-12-19T17:57:40.554643+0000","last_fullsized":"2023-12-19T17:57:40.554643+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:55:06.915659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.533947+0000","last_change":"2023-12-19T17:57:40.529128+0000","last_active":"2023-12-19T17:57:40.533947+0000","last_peered":"2023-12-19T17:57:40.533947+0000","last_clean":"2023-12-19T17:57:40.533947+0000","last_became_active":"2023-12-19T17:57:40.528831+0000","last_became_peered":"2023-12-19T17:57:40.528831+0000","last_unstale":"2023-12-19T17:57:40.533947+0000","last_undegraded":"2023-12-19T17:57:40.533947+0000","last_fullsized":"2023-12-19T17:57:40.533947+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:34.938988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.538565+0000","last_change":"2023-12-19T17:57:40.534086+0000","last_active":"2023-12-19T17:57:40.538565+0000","last_peered":"2023-12-19T17:57:40.538565+0000","last_clean":"2023-12-19T17:57:40.538565+0000","last_became_active":"2023-12-19T17:57:40.533839+0000","last_became_peered":"2023-12-19T17:57:40.533839+0000","last_unstale":"2023-12-19T17:57:40.538565+0000","last_undegraded":"2023-12-19T17:57:40.538565+0000","last_fullsized":"2023-12-19T17:57:40.538565+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:13.176396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"162'4","reported_seq":14,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557836+0000","last_change":"2023-12-19T17:57:40.066265+0000","last_active":"2023-12-19T17:57:40.557836+0000","last_peered":"2023-12-19T17:57:40.557836+0000","last_clean":"2023-12-19T17:57:40.557836+0000","last_became_active":"2023-12-19T17:57:40.065975+0000","last_became_peered":"2023-12-19T17:57:40.065975+0000","last_unstale":"2023-12-19T17:57:40.557836+0000","last_undegraded":"2023-12-19T17:57:40.557836+0000","last_fullsized":"2023-12-19T17:57:40.557836+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:03.592110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.559086+0000","last_change":"2023-12-19T17:57:40.556008+0000","last_active":"2023-12-19T17:57:40.559086+0000","last_peered":"2023-12-19T17:57:40.559086+0000","last_clean":"2023-12-19T17:57:40.559086+0000","last_became_active":"2023-12-19T17:57:40.553963+0000","last_became_peered":"2023-12-19T17:57:40.553963+0000","last_unstale":"2023-12-19T17:57:40.559086+0000","last_undegraded":"2023-12-19T17:57:40.559086+0000","last_fullsized":"2023-12-19T17:57:40.559086+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:58.489960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552764+0000","last_change":"2023-12-19T17:57:40.059089+0000","last_active":"2023-12-19T17:57:40.552764+0000","last_peered":"2023-12-19T17:57:40.552764+0000","last_clean":"2023-12-19T17:57:40.552764+0000","last_became_active":"2023-12-19T17:57:40.058725+0000","last_became_peered":"2023-12-19T17:57:40.058725+0000","last_unstale":"2023-12-19T17:57:40.552764+0000","last_undegraded":"2023-12-19T17:57:40.552764+0000","last_fullsized":"2023-12-19T17:57:40.552764+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:01:14.857081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558173+0000","last_change":"2023-12-19T17:57:40.555051+0000","last_active":"2023-12-19T17:57:40.558173+0000","last_peered":"2023-12-19T17:57:40.558173+0000","last_clean":"2023-12-19T17:57:40.558173+0000","last_became_active":"2023-12-19T17:57:40.554537+0000","last_became_peered":"2023-12-19T17:57:40.554537+0000","last_unstale":"2023-12-19T17:57:40.558173+0000","last_undegraded":"2023-12-19T17:57:40.558173+0000","last_fullsized":"2023-12-19T17:57:40.558173+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:12:43.624773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.550457+0000","last_change":"2023-12-19T17:57:40.053267+0000","last_active":"2023-12-19T17:57:40.550457+0000","last_peered":"2023-12-19T17:57:40.550457+0000","last_clean":"2023-12-19T17:57:40.550457+0000","last_became_active":"2023-12-19T17:57:40.052945+0000","last_became_peered":"2023-12-19T17:57:40.052945+0000","last_unstale":"2023-12-19T17:57:40.550457+0000","last_undegraded":"2023-12-19T17:57:40.550457+0000","last_fullsized":"2023-12-19T17:57:40.550457+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:56:55.139037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558160+0000","last_change":"2023-12-19T17:57:40.556132+0000","last_active":"2023-12-19T17:57:40.558160+0000","last_peered":"2023-12-19T17:57:40.558160+0000","last_clean":"2023-12-19T17:57:40.558160+0000","last_became_active":"2023-12-19T17:57:40.555792+0000","last_became_peered":"2023-12-19T17:57:40.555792+0000","last_unstale":"2023-12-19T17:57:40.558160+0000","last_undegraded":"2023-12-19T17:57:40.558160+0000","last_fullsized":"2023-12-19T17:57:40.558160+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:26.339164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557205+0000","last_change":"2023-12-19T17:57:40.055429+0000","last_active":"2023-12-19T17:57:40.557205+0000","last_peered":"2023-12-19T17:57:40.557205+0000","last_clean":"2023-12-19T17:57:40.557205+0000","last_became_active":"2023-12-19T17:57:40.055148+0000","last_became_peered":"2023-12-19T17:57:40.055148+0000","last_unstale":"2023-12-19T17:57:40.557205+0000","last_undegraded":"2023-12-19T17:57:40.557205+0000","last_fullsized":"2023-12-19T17:57:40.557205+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:14.562125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.538014+0000","last_change":"2023-12-19T17:57:40.532278+0000","last_active":"2023-12-19T17:57:40.538014+0000","last_peered":"2023-12-19T17:57:40.538014+0000","last_clean":"2023-12-19T17:57:40.538014+0000","last_became_active":"2023-12-19T17:57:40.531964+0000","last_became_peered":"2023-12-19T17:57:40.531964+0000","last_unstale":"2023-12-19T17:57:40.538014+0000","last_undegraded":"2023-12-19T17:57:40.538014+0000","last_fullsized":"2023-12-19T17:57:40.538014+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:14.500977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.556680+0000","last_change":"2023-12-19T17:57:40.060446+0000","last_active":"2023-12-19T17:57:40.556680+0000","last_peered":"2023-12-19T17:57:40.556680+0000","last_clean":"2023-12-19T17:57:40.556680+0000","last_became_active":"2023-12-19T17:57:40.060197+0000","last_became_peered":"2023-12-19T17:57:40.060197+0000","last_unstale":"2023-12-19T17:57:40.556680+0000","last_undegraded":"2023-12-19T17:57:40.556680+0000","last_fullsized":"2023-12-19T17:57:40.556680+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:41:42.776082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.536814+0000","last_change":"2023-12-19T17:57:40.534304+0000","last_active":"2023-12-19T17:57:40.536814+0000","last_peered":"2023-12-19T17:57:40.536814+0000","last_clean":"2023-12-19T17:57:40.536814+0000","last_became_active":"2023-12-19T17:57:40.533988+0000","last_became_peered":"2023-12-19T17:57:40.533988+0000","last_unstale":"2023-12-19T17:57:40.536814+0000","last_undegraded":"2023-12-19T17:57:40.536814+0000","last_fullsized":"2023-12-19T17:57:40.536814+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:29:12.843009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555095+0000","last_change":"2023-12-19T17:57:40.059110+0000","last_active":"2023-12-19T17:57:40.555095+0000","last_peered":"2023-12-19T17:57:40.555095+0000","last_clean":"2023-12-19T17:57:40.555095+0000","last_became_active":"2023-12-19T17:57:40.058830+0000","last_became_peered":"2023-12-19T17:57:40.058830+0000","last_unstale":"2023-12-19T17:57:40.555095+0000","last_undegraded":"2023-12-19T17:57:40.555095+0000","last_fullsized":"2023-12-19T17:57:40.555095+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:49.816414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.547416+0000","last_change":"2023-12-19T17:57:40.540067+0000","last_active":"2023-12-19T17:57:40.547416+0000","last_peered":"2023-12-19T17:57:40.547416+0000","last_clean":"2023-12-19T17:57:40.547416+0000","last_became_active":"2023-12-19T17:57:40.539701+0000","last_became_peered":"2023-12-19T17:57:40.539701+0000","last_unstale":"2023-12-19T17:57:40.547416+0000","last_undegraded":"2023-12-19T17:57:40.547416+0000","last_fullsized":"2023-12-19T17:57:40.547416+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:29:12.843009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.527444+0000","last_change":"2023-12-19T17:57:40.063570+0000","last_active":"2023-12-19T17:57:40.527444+0000","last_peered":"2023-12-19T17:57:40.527444+0000","last_clean":"2023-12-19T17:57:40.527444+0000","last_became_active":"2023-12-19T17:57:40.063286+0000","last_became_peered":"2023-12-19T17:57:40.063286+0000","last_unstale":"2023-12-19T17:57:40.527444+0000","last_undegraded":"2023-12-19T17:57:40.527444+0000","last_fullsized":"2023-12-19T17:57:40.527444+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:34:33.680079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.553208+0000","last_change":"2023-12-19T17:57:40.539328+0000","last_active":"2023-12-19T17:57:40.553208+0000","last_peered":"2023-12-19T17:57:40.553208+0000","last_clean":"2023-12-19T17:57:40.553208+0000","last_became_active":"2023-12-19T17:57:40.539027+0000","last_became_peered":"2023-12-19T17:57:40.539027+0000","last_unstale":"2023-12-19T17:57:40.553208+0000","last_undegraded":"2023-12-19T17:57:40.553208+0000","last_fullsized":"2023-12-19T17:57:40.553208+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:23.973532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.531517+0000","last_change":"2023-12-19T17:57:40.061729+0000","last_active":"2023-12-19T17:57:40.531517+0000","last_peered":"2023-12-19T17:57:40.531517+0000","last_clean":"2023-12-19T17:57:40.531517+0000","last_became_active":"2023-12-19T17:57:40.061340+0000","last_became_peered":"2023-12-19T17:57:40.061340+0000","last_unstale":"2023-12-19T17:57:40.531517+0000","last_undegraded":"2023-12-19T17:57:40.531517+0000","last_fullsized":"2023-12-19T17:57:40.531517+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:23.730425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557315+0000","last_change":"2023-12-19T17:57:40.550421+0000","last_active":"2023-12-19T17:57:40.557315+0000","last_peered":"2023-12-19T17:57:40.557315+0000","last_clean":"2023-12-19T17:57:40.557315+0000","last_became_active":"2023-12-19T17:57:40.550104+0000","last_became_peered":"2023-12-19T17:57:40.550104+0000","last_unstale":"2023-12-19T17:57:40.557315+0000","last_undegraded":"2023-12-19T17:57:40.557315+0000","last_fullsized":"2023-12-19T17:57:40.557315+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:50:39.392090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.526732+0000","last_change":"2023-12-19T17:57:40.058636+0000","last_active":"2023-12-19T17:57:40.526732+0000","last_peered":"2023-12-19T17:57:40.526732+0000","last_clean":"2023-12-19T17:57:40.526732+0000","last_became_active":"2023-12-19T17:57:40.058366+0000","last_became_peered":"2023-12-19T17:57:40.058366+0000","last_unstale":"2023-12-19T17:57:40.526732+0000","last_undegraded":"2023-12-19T17:57:40.526732+0000","last_fullsized":"2023-12-19T17:57:40.526732+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:11.703868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.539952+0000","last_change":"2023-12-19T17:57:40.535337+0000","last_active":"2023-12-19T17:57:40.539952+0000","last_peered":"2023-12-19T17:57:40.539952+0000","last_clean":"2023-12-19T17:57:40.539952+0000","last_became_active":"2023-12-19T17:57:40.535059+0000","last_became_peered":"2023-12-19T17:57:40.535059+0000","last_unstale":"2023-12-19T17:57:40.539952+0000","last_undegraded":"2023-12-19T17:57:40.539952+0000","last_fullsized":"2023-12-19T17:57:40.539952+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:13:06.277412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.524341+0000","last_change":"2023-12-19T17:57:40.057956+0000","last_active":"2023-12-19T17:57:40.524341+0000","last_peered":"2023-12-19T17:57:40.524341+0000","last_clean":"2023-12-19T17:57:40.524341+0000","last_became_active":"2023-12-19T17:57:40.057689+0000","last_became_peered":"2023-12-19T17:57:40.057689+0000","last_unstale":"2023-12-19T17:57:40.524341+0000","last_undegraded":"2023-12-19T17:57:40.524341+0000","last_fullsized":"2023-12-19T17:57:40.524341+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:55:44.482375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552513+0000","last_change":"2023-12-19T17:57:40.547732+0000","last_active":"2023-12-19T17:57:40.552513+0000","last_peered":"2023-12-19T17:57:40.552513+0000","last_clean":"2023-12-19T17:57:40.552513+0000","last_became_active":"2023-12-19T17:57:40.547451+0000","last_became_peered":"2023-12-19T17:57:40.547451+0000","last_unstale":"2023-12-19T17:57:40.552513+0000","last_undegraded":"2023-12-19T17:57:40.552513+0000","last_fullsized":"2023-12-19T17:57:40.552513+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:10.867122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"162'3","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.548429+0000","last_change":"2023-12-19T17:57:40.061646+0000","last_active":"2023-12-19T17:57:40.548429+0000","last_peered":"2023-12-19T17:57:40.548429+0000","last_clean":"2023-12-19T17:57:40.548429+0000","last_became_active":"2023-12-19T17:57:40.061357+0000","last_became_peered":"2023-12-19T17:57:40.061357+0000","last_unstale":"2023-12-19T17:57:40.548429+0000","last_undegraded":"2023-12-19T17:57:40.548429+0000","last_fullsized":"2023-12-19T17:57:40.548429+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:05:15.343469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.556692+0000","last_change":"2023-12-19T17:57:40.551317+0000","last_active":"2023-12-19T17:57:40.556692+0000","last_peered":"2023-12-19T17:57:40.556692+0000","last_clean":"2023-12-19T17:57:40.556692+0000","last_became_active":"2023-12-19T17:57:40.551061+0000","last_became_peered":"2023-12-19T17:57:40.551061+0000","last_unstale":"2023-12-19T17:57:40.556692+0000","last_undegraded":"2023-12-19T17:57:40.556692+0000","last_fullsized":"2023-12-19T17:57:40.556692+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:48:38.782646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"162'6","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.561151+0000","last_change":"2023-12-19T17:57:40.067315+0000","last_active":"2023-12-19T17:57:40.561151+0000","last_peered":"2023-12-19T17:57:40.561151+0000","last_clean":"2023-12-19T17:57:40.561151+0000","last_became_active":"2023-12-19T17:57:40.066941+0000","last_became_peered":"2023-12-19T17:57:40.066941+0000","last_unstale":"2023-12-19T17:57:40.561151+0000","last_undegraded":"2023-12-19T17:57:40.561151+0000","last_fullsized":"2023-12-19T17:57:40.561151+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:43:53.860436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557806+0000","last_change":"2023-12-19T17:57:40.550484+0000","last_active":"2023-12-19T17:57:40.557806+0000","last_peered":"2023-12-19T17:57:40.557806+0000","last_clean":"2023-12-19T17:57:40.557806+0000","last_became_active":"2023-12-19T17:57:40.549105+0000","last_became_peered":"2023-12-19T17:57:40.549105+0000","last_unstale":"2023-12-19T17:57:40.557806+0000","last_undegraded":"2023-12-19T17:57:40.557806+0000","last_fullsized":"2023-12-19T17:57:40.557806+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:29:59.039217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.545499+0000","last_change":"2023-12-19T17:57:40.052924+0000","last_active":"2023-12-19T17:57:40.545499+0000","last_peered":"2023-12-19T17:57:40.545499+0000","last_clean":"2023-12-19T17:57:40.545499+0000","last_became_active":"2023-12-19T17:57:40.052663+0000","last_became_peered":"2023-12-19T17:57:40.052663+0000","last_unstale":"2023-12-19T17:57:40.545499+0000","last_undegraded":"2023-12-19T17:57:40.545499+0000","last_fullsized":"2023-12-19T17:57:40.545499+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:48:37.899407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.556642+0000","last_change":"2023-12-19T17:57:40.546595+0000","last_active":"2023-12-19T17:57:40.556642+0000","last_peered":"2023-12-19T17:57:40.556642+0000","last_clean":"2023-12-19T17:57:40.556642+0000","last_became_active":"2023-12-19T17:57:40.546142+0000","last_became_peered":"2023-12-19T17:57:40.546142+0000","last_unstale":"2023-12-19T17:57:40.556642+0000","last_undegraded":"2023-12-19T17:57:40.556642+0000","last_fullsized":"2023-12-19T17:57:40.556642+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:49.866445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.531436+0000","last_change":"2023-12-19T17:57:40.065628+0000","last_active":"2023-12-19T17:57:40.531436+0000","last_peered":"2023-12-19T17:57:40.531436+0000","last_clean":"2023-12-19T17:57:40.531436+0000","last_became_active":"2023-12-19T17:57:40.065313+0000","last_became_peered":"2023-12-19T17:57:40.065313+0000","last_unstale":"2023-12-19T17:57:40.531436+0000","last_undegraded":"2023-12-19T17:57:40.531436+0000","last_fullsized":"2023-12-19T17:57:40.531436+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:03.592110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.556852+0000","last_change":"2023-12-19T17:57:40.550414+0000","last_active":"2023-12-19T17:57:40.556852+0000","last_peered":"2023-12-19T17:57:40.556852+0000","last_clean":"2023-12-19T17:57:40.556852+0000","last_became_active":"2023-12-19T17:57:40.547679+0000","last_became_peered":"2023-12-19T17:57:40.547679+0000","last_unstale":"2023-12-19T17:57:40.556852+0000","last_undegraded":"2023-12-19T17:57:40.556852+0000","last_fullsized":"2023-12-19T17:57:40.556852+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:34.746775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"162'8","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558862+0000","last_change":"2023-12-19T17:57:40.062566+0000","last_active":"2023-12-19T17:57:40.558862+0000","last_peered":"2023-12-19T17:57:40.558862+0000","last_clean":"2023-12-19T17:57:40.558862+0000","last_became_active":"2023-12-19T17:57:40.061288+0000","last_became_peered":"2023-12-19T17:57:40.061288+0000","last_unstale":"2023-12-19T17:57:40.558862+0000","last_undegraded":"2023-12-19T17:57:40.558862+0000","last_fullsized":"2023-12-19T17:57:40.558862+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:47.281313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557596+0000","last_change":"2023-12-19T17:57:40.552976+0000","last_active":"2023-12-19T17:57:40.557596+0000","last_peered":"2023-12-19T17:57:40.557596+0000","last_clean":"2023-12-19T17:57:40.557596+0000","last_became_active":"2023-12-19T17:57:40.552691+0000","last_became_peered":"2023-12-19T17:57:40.552691+0000","last_unstale":"2023-12-19T17:57:40.557596+0000","last_undegraded":"2023-12-19T17:57:40.557596+0000","last_fullsized":"2023-12-19T17:57:40.557596+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:15.573540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.554597+0000","last_change":"2023-12-19T17:57:40.067096+0000","last_active":"2023-12-19T17:57:40.554597+0000","last_peered":"2023-12-19T17:57:40.554597+0000","last_clean":"2023-12-19T17:57:40.554597+0000","last_became_active":"2023-12-19T17:57:40.066562+0000","last_became_peered":"2023-12-19T17:57:40.066562+0000","last_unstale":"2023-12-19T17:57:40.554597+0000","last_undegraded":"2023-12-19T17:57:40.554597+0000","last_fullsized":"2023-12-19T17:57:40.554597+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:08.979772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.539969+0000","last_change":"2023-12-19T17:57:40.536577+0000","last_active":"2023-12-19T17:57:40.539969+0000","last_peered":"2023-12-19T17:57:40.539969+0000","last_clean":"2023-12-19T17:57:40.539969+0000","last_became_active":"2023-12-19T17:57:40.536260+0000","last_became_peered":"2023-12-19T17:57:40.536260+0000","last_unstale":"2023-12-19T17:57:40.539969+0000","last_undegraded":"2023-12-19T17:57:40.539969+0000","last_fullsized":"2023-12-19T17:57:40.539969+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:34:24.820518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.554222+0000","last_change":"2023-12-19T17:57:40.057376+0000","last_active":"2023-12-19T17:57:40.554222+0000","last_peered":"2023-12-19T17:57:40.554222+0000","last_clean":"2023-12-19T17:57:40.554222+0000","last_became_active":"2023-12-19T17:57:40.057086+0000","last_became_peered":"2023-12-19T17:57:40.057086+0000","last_unstale":"2023-12-19T17:57:40.554222+0000","last_undegraded":"2023-12-19T17:57:40.554222+0000","last_fullsized":"2023-12-19T17:57:40.554222+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:27:55.404240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.551140+0000","last_change":"2023-12-19T17:57:40.547510+0000","last_active":"2023-12-19T17:57:40.551140+0000","last_peered":"2023-12-19T17:57:40.551140+0000","last_clean":"2023-12-19T17:57:40.551140+0000","last_became_active":"2023-12-19T17:57:40.547204+0000","last_became_peered":"2023-12-19T17:57:40.547204+0000","last_unstale":"2023-12-19T17:57:40.551140+0000","last_undegraded":"2023-12-19T17:57:40.551140+0000","last_fullsized":"2023-12-19T17:57:40.551140+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:34:40.964262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"162'1","reported_seq":11,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552504+0000","last_change":"2023-12-19T17:57:40.055466+0000","last_active":"2023-12-19T17:57:40.552504+0000","last_peered":"2023-12-19T17:57:40.552504+0000","last_clean":"2023-12-19T17:57:40.552504+0000","last_became_active":"2023-12-19T17:57:40.055145+0000","last_became_peered":"2023-12-19T17:57:40.055145+0000","last_unstale":"2023-12-19T17:57:40.552504+0000","last_undegraded":"2023-12-19T17:57:40.552504+0000","last_fullsized":"2023-12-19T17:57:40.552504+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:32:20.433509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.553312+0000","last_change":"2023-12-19T17:57:40.540229+0000","last_active":"2023-12-19T17:57:40.553312+0000","last_peered":"2023-12-19T17:57:40.553312+0000","last_clean":"2023-12-19T17:57:40.553312+0000","last_became_active":"2023-12-19T17:57:40.539948+0000","last_became_peered":"2023-12-19T17:57:40.539948+0000","last_unstale":"2023-12-19T17:57:40.553312+0000","last_undegraded":"2023-12-19T17:57:40.553312+0000","last_fullsized":"2023-12-19T17:57:40.553312+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:55:45.493789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.525668+0000","last_change":"2023-12-19T17:57:40.058276+0000","last_active":"2023-12-19T17:57:40.525668+0000","last_peered":"2023-12-19T17:57:40.525668+0000","last_clean":"2023-12-19T17:57:40.525668+0000","last_became_active":"2023-12-19T17:57:40.057982+0000","last_became_peered":"2023-12-19T17:57:40.057982+0000","last_unstale":"2023-12-19T17:57:40.525668+0000","last_undegraded":"2023-12-19T17:57:40.525668+0000","last_fullsized":"2023-12-19T17:57:40.525668+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:49:09.035689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.556469+0000","last_change":"2023-12-19T17:57:40.548292+0000","last_active":"2023-12-19T17:57:40.556469+0000","last_peered":"2023-12-19T17:57:40.556469+0000","last_clean":"2023-12-19T17:57:40.556469+0000","last_became_active":"2023-12-19T17:57:40.547740+0000","last_became_peered":"2023-12-19T17:57:40.547740+0000","last_unstale":"2023-12-19T17:57:40.556469+0000","last_undegraded":"2023-12-19T17:57:40.556469+0000","last_fullsized":"2023-12-19T17:57:40.556469+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:59.244547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.556960+0000","last_change":"2023-12-19T17:57:40.059000+0000","last_active":"2023-12-19T17:57:40.556960+0000","last_peered":"2023-12-19T17:57:40.556960+0000","last_clean":"2023-12-19T17:57:40.556960+0000","last_became_active":"2023-12-19T17:57:40.058583+0000","last_became_peered":"2023-12-19T17:57:40.058583+0000","last_unstale":"2023-12-19T17:57:40.556960+0000","last_undegraded":"2023-12-19T17:57:40.556960+0000","last_fullsized":"2023-12-19T17:57:40.556960+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:57:18.926267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552592+0000","last_change":"2023-12-19T17:57:40.546220+0000","last_active":"2023-12-19T17:57:40.552592+0000","last_peered":"2023-12-19T17:57:40.552592+0000","last_clean":"2023-12-19T17:57:40.552592+0000","last_became_active":"2023-12-19T17:57:40.545944+0000","last_became_peered":"2023-12-19T17:57:40.545944+0000","last_unstale":"2023-12-19T17:57:40.552592+0000","last_undegraded":"2023-12-19T17:57:40.552592+0000","last_fullsized":"2023-12-19T17:57:40.552592+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:41:43.787497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.523199+0000","last_change":"2023-12-19T17:57:40.058009+0000","last_active":"2023-12-19T17:57:40.523199+0000","last_peered":"2023-12-19T17:57:40.523199+0000","last_clean":"2023-12-19T17:57:40.523199+0000","last_became_active":"2023-12-19T17:57:40.057735+0000","last_became_peered":"2023-12-19T17:57:40.057735+0000","last_unstale":"2023-12-19T17:57:40.523199+0000","last_undegraded":"2023-12-19T17:57:40.523199+0000","last_fullsized":"2023-12-19T17:57:40.523199+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:55:05.904244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552616+0000","last_change":"2023-12-19T17:57:40.539348+0000","last_active":"2023-12-19T17:57:40.552616+0000","last_peered":"2023-12-19T17:57:40.552616+0000","last_clean":"2023-12-19T17:57:40.552616+0000","last_became_active":"2023-12-19T17:57:40.539081+0000","last_became_peered":"2023-12-19T17:57:40.539081+0000","last_unstale":"2023-12-19T17:57:40.552616+0000","last_undegraded":"2023-12-19T17:57:40.552616+0000","last_fullsized":"2023-12-19T17:57:40.552616+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:50.827829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.522645+0000","last_change":"2023-12-19T17:57:40.063416+0000","last_active":"2023-12-19T17:57:40.522645+0000","last_peered":"2023-12-19T17:57:40.522645+0000","last_clean":"2023-12-19T17:57:40.522645+0000","last_became_active":"2023-12-19T17:57:40.062526+0000","last_became_peered":"2023-12-19T17:57:40.062526+0000","last_unstale":"2023-12-19T17:57:40.522645+0000","last_undegraded":"2023-12-19T17:57:40.522645+0000","last_fullsized":"2023-12-19T17:57:40.522645+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:24.484854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555318+0000","last_change":"2023-12-19T17:57:40.552043+0000","last_active":"2023-12-19T17:57:40.555318+0000","last_peered":"2023-12-19T17:57:40.555318+0000","last_clean":"2023-12-19T17:57:40.555318+0000","last_became_active":"2023-12-19T17:57:40.549303+0000","last_became_peered":"2023-12-19T17:57:40.549303+0000","last_unstale":"2023-12-19T17:57:40.555318+0000","last_undegraded":"2023-12-19T17:57:40.555318+0000","last_fullsized":"2023-12-19T17:57:40.555318+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:55:45.493789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.527606+0000","last_change":"2023-12-19T17:57:40.061890+0000","last_active":"2023-12-19T17:57:40.527606+0000","last_peered":"2023-12-19T17:57:40.527606+0000","last_clean":"2023-12-19T17:57:40.527606+0000","last_became_active":"2023-12-19T17:57:40.060827+0000","last_became_peered":"2023-12-19T17:57:40.060827+0000","last_unstale":"2023-12-19T17:57:40.527606+0000","last_undegraded":"2023-12-19T17:57:40.527606+0000","last_fullsized":"2023-12-19T17:57:40.527606+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:41:22.464097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.550776+0000","last_change":"2023-12-19T17:57:40.539806+0000","last_active":"2023-12-19T17:57:40.550776+0000","last_peered":"2023-12-19T17:57:40.550776+0000","last_clean":"2023-12-19T17:57:40.550776+0000","last_became_active":"2023-12-19T17:57:40.539544+0000","last_became_peered":"2023-12-19T17:57:40.539544+0000","last_unstale":"2023-12-19T17:57:40.550776+0000","last_undegraded":"2023-12-19T17:57:40.550776+0000","last_fullsized":"2023-12-19T17:57:40.550776+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:28.187346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.558862+0000","last_change":"2023-12-19T17:57:40.062974+0000","last_active":"2023-12-19T17:57:40.558862+0000","last_peered":"2023-12-19T17:57:40.558862+0000","last_clean":"2023-12-19T17:57:40.558862+0000","last_became_active":"2023-12-19T17:57:40.062312+0000","last_became_peered":"2023-12-19T17:57:40.062312+0000","last_unstale":"2023-12-19T17:57:40.558862+0000","last_undegraded":"2023-12-19T17:57:40.558862+0000","last_fullsized":"2023-12-19T17:57:40.558862+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:21.078553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.559402+0000","last_change":"2023-12-19T17:57:40.558668+0000","last_active":"2023-12-19T17:57:40.559402+0000","last_peered":"2023-12-19T17:57:40.559402+0000","last_clean":"2023-12-19T17:57:40.559402+0000","last_became_active":"2023-12-19T17:57:40.558367+0000","last_became_peered":"2023-12-19T17:57:40.558367+0000","last_unstale":"2023-12-19T17:57:40.559402+0000","last_undegraded":"2023-12-19T17:57:40.559402+0000","last_fullsized":"2023-12-19T17:57:40.559402+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:24.881677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.537273+0000","last_change":"2023-12-19T17:57:40.065674+0000","last_active":"2023-12-19T17:57:40.537273+0000","last_peered":"2023-12-19T17:57:40.537273+0000","last_clean":"2023-12-19T17:57:40.537273+0000","last_became_active":"2023-12-19T17:57:40.065320+0000","last_became_peered":"2023-12-19T17:57:40.065320+0000","last_unstale":"2023-12-19T17:57:40.537273+0000","last_undegraded":"2023-12-19T17:57:40.537273+0000","last_fullsized":"2023-12-19T17:57:40.537273+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:48:23.730425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557061+0000","last_change":"2023-12-19T17:57:40.548642+0000","last_active":"2023-12-19T17:57:40.557061+0000","last_peered":"2023-12-19T17:57:40.557061+0000","last_clean":"2023-12-19T17:57:40.557061+0000","last_became_active":"2023-12-19T17:57:40.548153+0000","last_became_peered":"2023-12-19T17:57:40.548153+0000","last_unstale":"2023-12-19T17:57:40.557061+0000","last_undegraded":"2023-12-19T17:57:40.557061+0000","last_fullsized":"2023-12-19T17:57:40.557061+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:14.500977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.531965+0000","last_change":"2023-12-19T17:57:40.061558+0000","last_active":"2023-12-19T17:57:40.531965+0000","last_peered":"2023-12-19T17:57:40.531965+0000","last_clean":"2023-12-19T17:57:40.531965+0000","last_became_active":"2023-12-19T17:57:40.061289+0000","last_became_peered":"2023-12-19T17:57:40.061289+0000","last_unstale":"2023-12-19T17:57:40.531965+0000","last_undegraded":"2023-12-19T17:57:40.531965+0000","last_fullsized":"2023-12-19T17:57:40.531965+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:59.357090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.552556+0000","last_change":"2023-12-19T17:57:40.547683+0000","last_active":"2023-12-19T17:57:40.552556+0000","last_peered":"2023-12-19T17:57:40.552556+0000","last_clean":"2023-12-19T17:57:40.552556+0000","last_became_active":"2023-12-19T17:57:40.547412+0000","last_became_peered":"2023-12-19T17:57:40.547412+0000","last_unstale":"2023-12-19T17:57:40.552556+0000","last_undegraded":"2023-12-19T17:57:40.552556+0000","last_fullsized":"2023-12-19T17:57:40.552556+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:58.489960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.528324+0000","last_change":"2023-12-19T17:57:40.055721+0000","last_active":"2023-12-19T17:57:40.528324+0000","last_peered":"2023-12-19T17:57:40.528324+0000","last_clean":"2023-12-19T17:57:40.528324+0000","last_became_active":"2023-12-19T17:57:40.055382+0000","last_became_peered":"2023-12-19T17:57:40.055382+0000","last_unstale":"2023-12-19T17:57:40.528324+0000","last_undegraded":"2023-12-19T17:57:40.528324+0000","last_fullsized":"2023-12-19T17:57:40.528324+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:47:20.484481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555834+0000","last_change":"2023-12-19T17:57:40.552580+0000","last_active":"2023-12-19T17:57:40.555834+0000","last_peered":"2023-12-19T17:57:40.555834+0000","last_clean":"2023-12-19T17:57:40.555834+0000","last_became_active":"2023-12-19T17:57:40.552322+0000","last_became_peered":"2023-12-19T17:57:40.552322+0000","last_unstale":"2023-12-19T17:57:40.555834+0000","last_undegraded":"2023-12-19T17:57:40.555834+0000","last_fullsized":"2023-12-19T17:57:40.555834+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:07.348734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.531955+0000","last_change":"2023-12-19T17:57:40.064222+0000","last_active":"2023-12-19T17:57:40.531955+0000","last_peered":"2023-12-19T17:57:40.531955+0000","last_clean":"2023-12-19T17:57:40.531955+0000","last_became_active":"2023-12-19T17:57:40.063957+0000","last_became_peered":"2023-12-19T17:57:40.063957+0000","last_unstale":"2023-12-19T17:57:40.531955+0000","last_undegraded":"2023-12-19T17:57:40.531955+0000","last_fullsized":"2023-12-19T17:57:40.531955+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:38:35.525981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.554690+0000","last_change":"2023-12-19T17:57:40.551811+0000","last_active":"2023-12-19T17:57:40.554690+0000","last_peered":"2023-12-19T17:57:40.554690+0000","last_clean":"2023-12-19T17:57:40.554690+0000","last_became_active":"2023-12-19T17:57:40.551354+0000","last_became_peered":"2023-12-19T17:57:40.551354+0000","last_unstale":"2023-12-19T17:57:40.554690+0000","last_undegraded":"2023-12-19T17:57:40.554690+0000","last_fullsized":"2023-12-19T17:57:40.554690+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:20:00.841561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.537836+0000","last_change":"2023-12-19T17:57:40.067008+0000","last_active":"2023-12-19T17:57:40.537836+0000","last_peered":"2023-12-19T17:57:40.537836+0000","last_clean":"2023-12-19T17:57:40.537836+0000","last_became_active":"2023-12-19T17:57:40.066730+0000","last_became_peered":"2023-12-19T17:57:40.066730+0000","last_unstale":"2023-12-19T17:57:40.537836+0000","last_undegraded":"2023-12-19T17:57:40.537836+0000","last_fullsized":"2023-12-19T17:57:40.537836+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:41:42.776082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.545805+0000","last_change":"2023-12-19T17:57:40.542049+0000","last_active":"2023-12-19T17:57:40.545805+0000","last_peered":"2023-12-19T17:57:40.545805+0000","last_clean":"2023-12-19T17:57:40.545805+0000","last_became_active":"2023-12-19T17:57:40.541728+0000","last_became_peered":"2023-12-19T17:57:40.541728+0000","last_unstale":"2023-12-19T17:57:40.545805+0000","last_undegraded":"2023-12-19T17:57:40.545805+0000","last_fullsized":"2023-12-19T17:57:40.545805+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:00:44.145016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.522661+0000","last_change":"2023-12-19T17:57:40.056628+0000","last_active":"2023-12-19T17:57:40.522661+0000","last_peered":"2023-12-19T17:57:40.522661+0000","last_clean":"2023-12-19T17:57:40.522661+0000","last_became_active":"2023-12-19T17:57:40.056340+0000","last_became_peered":"2023-12-19T17:57:40.056340+0000","last_unstale":"2023-12-19T17:57:40.522661+0000","last_undegraded":"2023-12-19T17:57:40.522661+0000","last_fullsized":"2023-12-19T17:57:40.522661+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:46:58.173739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.555349+0000","last_change":"2023-12-19T17:57:40.542871+0000","last_active":"2023-12-19T17:57:40.555349+0000","last_peered":"2023-12-19T17:57:40.555349+0000","last_clean":"2023-12-19T17:57:40.555349+0000","last_became_active":"2023-12-19T17:57:40.542533+0000","last_became_peered":"2023-12-19T17:57:40.542533+0000","last_unstale":"2023-12-19T17:57:40.555349+0000","last_undegraded":"2023-12-19T17:57:40.555349+0000","last_fullsized":"2023-12-19T17:57:40.555349+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:48:38.782646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.527697+0000","last_change":"2023-12-19T17:57:40.051875+0000","last_active":"2023-12-19T17:57:40.527697+0000","last_peered":"2023-12-19T17:57:40.527697+0000","last_clean":"2023-12-19T17:57:40.527697+0000","last_became_active":"2023-12-19T17:57:40.051571+0000","last_became_peered":"2023-12-19T17:57:40.051571+0000","last_unstale":"2023-12-19T17:57:40.527697+0000","last_undegraded":"2023-12-19T17:57:40.527697+0000","last_fullsized":"2023-12-19T17:57:40.527697+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:11:23.185297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.553447+0000","last_change":"2023-12-19T17:57:40.550829+0000","last_active":"2023-12-19T17:57:40.553447+0000","last_peered":"2023-12-19T17:57:40.553447+0000","last_clean":"2023-12-19T17:57:40.553447+0000","last_became_active":"2023-12-19T17:57:40.550300+0000","last_became_peered":"2023-12-19T17:57:40.550300+0000","last_unstale":"2023-12-19T17:57:40.553447+0000","last_undegraded":"2023-12-19T17:57:40.553447+0000","last_fullsized":"2023-12-19T17:57:40.553447+0000","mapping_epoch":161,"log_start":"0'0","ondisk_log_start":"0'0","created":161,"last_epoch_clean":162,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:40.025533+0000","last_clean_scrub_stamp":"2023-12-19T17:57:40.025533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:25.416611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"162'2","reported_seq":12,"reported_epoch":162,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.557659+0000","last_change":"2023-12-19T17:57:40.065786+0000","last_active":"2023-12-19T17:57:40.557659+0000","last_peered":"2023-12-19T17:57:40.557659+0000","last_clean":"2023-12-19T17:57:40.557659+0000","last_became_active":"2023-12-19T17:57:40.065390+0000","last_became_peered":"2023-12-19T17:57:40.065390+0000","last_unstale":"2023-12-19T17:57:40.557659+0000","last_undegraded":"2023-12-19T17:57:40.557659+0000","last_fullsized":"2023-12-19T17:57:40.557659+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:57:39.014118+0000","last_clean_scrub_stamp":"2023-12-19T17:57:39.014118+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:59.357090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"150'269","reported_seq":859,"reported_epoch":161,"state":"active+clean","last_fresh":"2023-12-19T17:57:40.057992+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T17:57:40.057992+0000","last_peered":"2023-12-19T17:57:40.057992+0000","last_clean":"2023-12-19T17:57:40.057992+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T17:57:40.057992+0000","last_undegraded":"2023-12-19T17:57:40.057992+0000","last_fullsized":"2023-12-19T17:57:40.057992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":269,"log_dups_size":0,"ondisk_log_size":269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1442,"num_read_kb":2969,"num_write":435,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1442,"num_read_kb":2969,"num_write":435,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510343,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37672,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334168,"statfs":{"total":96636764160,"available":96598188032,"internally_reserved":0,"allocated":2588672,"data_stored":1627538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.51100000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.88600000000000001,"5min":1.012,"15min":1.0209999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73899999999999999,"15min":0.73799999999999999},"min":{"1min":0.56100000000000005,"5min":0.41299999999999998,"15min":0.41199999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.77200000000000002}]},{"osd":1,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75800000000000001,"15min":0.746},"min":{"1min":0.57399999999999995,"5min":0.442,"15min":0.442},"max":{"1min":0.96699999999999997,"5min":0.997,"15min":0.997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.53600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":0.96899999999999997,"5min":1.0169999999999999,"15min":1.05},"last":0.99399999999999999}]},{"osd":2,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73199999999999998,"15min":0.72799999999999998},"min":{"1min":0.59599999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89100000000000001,"5min":0.92400000000000004,"15min":0.97999999999999998},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.79000000000000004,"15min":0.78500000000000003},"min":{"1min":0.56699999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.87}]},{"osd":3,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.75800000000000001,"15min":0.75},"min":{"1min":0.54800000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76900000000000002,"15min":0.77700000000000002},"min":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0109999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59799999999999998,"15min":0.58599999999999997},"min":{"1min":0.41999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.93899999999999995,"15min":0.94299999999999995},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61399999999999999,"15min":0.60799999999999998},"min":{"1min":0.432,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.89000000000000001}]},{"osd":5,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.64000000000000001,"15min":0.626},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.36799999999999999},"max":{"1min":0.875,"5min":1.044,"15min":1.044},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.66400000000000003,"15min":0.66000000000000003},"min":{"1min":0.47199999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":1.0980000000000001}]},{"osd":6,"last update":"Tue Dec 19 17:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72699999999999998,"15min":0.71599999999999997},"min":{"1min":0.54100000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.89100000000000001,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":1.071},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.52000000000000002,"5min":0.45900000000000002,"15min":0.44500000000000001},"max":{"1min":0.91000000000000003,"5min":1.075,"15min":1.075},"last":1.016}]}]},{"osd":6,"up_from":15,"seq":64424510343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41212,"kb_used_data":1972,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330628,"statfs":{"total":96636764160,"available":96594563072,"internally_reserved":0,"allocated":2019328,"data_stored":1046241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.48799999999999999,"5min":0.40300000000000002,"15min":0.36799999999999999},"max":{"1min":0.995,"5min":0.995,"15min":1.02},"last":0.878},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.747,"15min":0.73799999999999999},"min":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92900000000000005,"5min":0.95699999999999996,"15min":1.329},"last":0.71999999999999997}]},{"osd":1,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.75,"15min":0.73799999999999999},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.42899999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.1419999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.77200000000000002,"15min":0.75700000000000001},"min":{"1min":0.65600000000000003,"5min":0.47799999999999998,"15min":0.45200000000000001},"max":{"1min":0.96499999999999997,"5min":1.026,"15min":1.026},"last":0.78300000000000003}]},{"osd":2,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78200000000000003,"15min":0.76300000000000001},"min":{"1min":0.61299999999999999,"5min":0.497,"15min":0.48799999999999999},"max":{"1min":0.97099999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76000000000000001,"15min":0.76900000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.497},"max":{"1min":0.97999999999999998,"5min":1.0029999999999999,"15min":1.853},"last":0.68899999999999995}]},{"osd":3,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80100000000000005,"15min":0.79400000000000004},"min":{"1min":0.63700000000000001,"5min":0.59899999999999998,"15min":0.45400000000000001},"max":{"1min":0.93799999999999994,"5min":1.0149999999999999,"15min":1.542},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75800000000000001,"15min":0.73699999999999999},"min":{"1min":0.57099999999999995,"5min":0.54700000000000004,"15min":0.48799999999999999},"max":{"1min":0.94699999999999995,"5min":0.98699999999999999,"15min":1.04},"last":0.73199999999999998}]},{"osd":4,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.36799999999999999},"max":{"1min":0.85399999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.499,"5min":0.41499999999999998,"15min":0.372},"max":{"1min":1.131,"5min":1.131,"15min":1.3919999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.46600000000000003,"5min":0.442,"15min":0.34100000000000003},"max":{"1min":0.86399999999999999,"5min":0.91000000000000003,"15min":1.0920000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.54700000000000004,"5min":0.44700000000000001,"15min":0.41399999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.59399999999999997}]},{"osd":7,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.57899999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1,"5min":1.0580000000000001,"15min":1.4259999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.71599999999999997},"min":{"1min":0.59499999999999997,"5min":0.503,"15min":0.42699999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0369999999999999},"last":0.65700000000000003}]}]},{"osd":5,"up_from":15,"seq":64424510343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45284,"kb_used_data":1948,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94326556,"statfs":{"total":96636764160,"available":96590393344,"internally_reserved":0,"allocated":1994752,"data_stored":1037170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.40799999999999997},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.1120000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.753,"15min":0.748},"min":{"1min":0.52200000000000002,"5min":0.48899999999999999,"15min":0.374},"max":{"1min":0.95799999999999996,"5min":1.0289999999999999,"15min":1.073},"last":0.73899999999999999}]},{"osd":1,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.77000000000000002,"15min":0.76700000000000002},"min":{"1min":0.69199999999999995,"5min":0.497,"15min":0.497},"max":{"1min":0.89600000000000002,"5min":1.0029999999999999,"15min":1.1659999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.75,"15min":0.746},"min":{"1min":0.57999999999999996,"5min":0.51800000000000002,"15min":0.50800000000000001},"max":{"1min":1.012,"5min":1.0249999999999999,"15min":1.0469999999999999},"last":0.85799999999999998}]},{"osd":2,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.78400000000000003,"15min":0.78300000000000003},"min":{"1min":0.53000000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0409999999999999,"5min":1.05,"15min":1.1040000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.59299999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91600000000000004,"5min":0.95899999999999996,"15min":1.0349999999999999},"last":0.77000000000000002}]},{"osd":3,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.64500000000000002,"5min":0.56399999999999995,"15min":0.52700000000000002},"max":{"1min":0.93100000000000005,"5min":0.97099999999999997,"15min":1.131},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.71399999999999997,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.98799999999999999,"5min":0.99299999999999999,"15min":1.125},"last":0.67000000000000004}]},{"osd":4,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.434,"5min":0.38900000000000001,"15min":0.35699999999999998},"max":{"1min":0.83299999999999996,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.88},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.41399999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.92200000000000004},"last":0.92600000000000005}]},{"osd":6,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.48999999999999999,"5min":0.46600000000000003,"15min":0.41199999999999998},"max":{"1min":0.86199999999999999,"5min":0.875,"15min":1.048},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.65300000000000002,"15min":0.64600000000000002},"min":{"1min":0.53300000000000003,"5min":0.44400000000000001,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":0.85499999999999998,"15min":0.97699999999999998},"last":0.82699999999999996}]},{"osd":7,"last update":"Tue Dec 19 17:56:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.66100000000000003,"15min":0.66800000000000004},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":0.97399999999999998,"15min":1.016},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.55000000000000004,"5min":0.49399999999999999,"15min":0.41299999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":1.0289999999999999},"last":0.71499999999999997}]}]},{"osd":4,"up_from":15,"seq":64424510343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59252,"kb_used_data":1964,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312588,"statfs":{"total":96636764160,"available":96576090112,"internally_reserved":0,"allocated":2011136,"data_stored":1044747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73199999999999998,"15min":0.74199999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.38300000000000001},"max":{"1min":0.95599999999999996,"5min":1.036,"15min":1.056},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.73899999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.40400000000000003},"max":{"1min":0.92500000000000004,"5min":0.95199999999999996,"15min":1.0620000000000001},"last":0.78100000000000003}]},{"osd":1,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.755,"15min":0.76000000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":0.93999999999999995,"5min":0.995,"15min":1.0980000000000001},"last":0.75},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.753,"15min":0.755},"min":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.46400000000000002},"max":{"1min":0.92900000000000005,"5min":1.0489999999999999,"15min":1.0549999999999999},"last":0.82699999999999996}]},{"osd":2,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.752,"15min":0.76400000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.94399999999999995,"5min":1.026,"15min":1.042},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78400000000000003,"15min":0.77300000000000002},"min":{"1min":0.63300000000000001,"5min":0.496,"15min":0.46100000000000002},"max":{"1min":0.93000000000000005,"5min":1.0149999999999999,"15min":1.018},"last":0.80000000000000004}]},{"osd":3,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.75,"15min":0.748},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.94399999999999995,"5min":0.96799999999999997,"15min":1.0229999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78800000000000003,"15min":0.78300000000000003},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.499},"max":{"1min":0.97099999999999997,"5min":0.997,"15min":1.0049999999999999},"last":0.81499999999999995}]},{"osd":5,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.51100000000000001,"5min":0.40799999999999997,"15min":0.36299999999999999},"max":{"1min":0.88400000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63700000000000001,"15min":0.622},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.379},"max":{"1min":0.88900000000000001,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72099999999999997}]},{"osd":6,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.65200000000000002,"15min":0.64100000000000001},"min":{"1min":0.46500000000000002,"5min":0.44700000000000001,"15min":0.41399999999999998},"max":{"1min":0.93500000000000005,"5min":1.03,"15min":1.079},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70799999999999996,"15min":0.68100000000000005},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.34599999999999997},"max":{"1min":0.91900000000000004,"5min":0.98899999999999999,"15min":0.99099999999999999},"last":0.70099999999999996}]},{"osd":7,"last update":"Tue Dec 19 17:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68999999999999995,"15min":0.68200000000000005},"min":{"1min":0.54600000000000004,"5min":0.44800000000000001,"15min":0.40400000000000003},"max":{"1min":0.97799999999999998,"5min":1.075,"15min":1.075},"last":0.629},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.71799999999999997},"min":{"1min":0.59799999999999998,"5min":0.51300000000000001,"15min":0.45500000000000002},"max":{"1min":0.97699999999999998,"5min":1.0629999999999999,"15min":1.1160000000000001},"last":0.67800000000000005}]}]},{"osd":3,"up_from":15,"seq":64424510343,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46968,"kb_used_data":1968,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324872,"statfs":{"total":96636764160,"available":96588668928,"internally_reserved":0,"allocated":2015232,"data_stored":1044837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.317,"5min":0.30599999999999999,"15min":0.28000000000000003},"max":{"1min":0.68899999999999995,"5min":0.70799999999999996,"15min":0.74399999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.52100000000000002},"min":{"1min":0.42399999999999999,"5min":0.313,"15min":0.29499999999999998},"max":{"1min":0.64900000000000002,"5min":0.76700000000000002,"15min":1.01},"last":0.42499999999999999}]},{"osd":1,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49399999999999999,"15min":0.48299999999999998},"min":{"1min":0.36799999999999999,"5min":0.32800000000000001,"15min":0.307},"max":{"1min":0.67700000000000005,"5min":0.72199999999999998,"15min":0.83899999999999997},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.39300000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66600000000000004,"5min":0.78300000000000003,"15min":0.873},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.57699999999999996,"15min":0.57099999999999995},"min":{"1min":0.41099999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76500000000000001,"5min":0.82999999999999996,"15min":0.92100000000000004},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.60599999999999998,"15min":0.59199999999999997},"min":{"1min":0.51200000000000001,"5min":0.41699999999999998,"15min":0.38600000000000001},"max":{"1min":0.80300000000000005,"5min":0.93000000000000005,"15min":0.98699999999999999},"last":0.51300000000000001}]},{"osd":4,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.82299999999999995},"min":{"1min":0.69399999999999995,"5min":0.60199999999999998,"15min":0.52700000000000002},"max":{"1min":1.1140000000000001,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.83899999999999997,"15min":0.83199999999999996},"min":{"1min":0.68100000000000005,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.135,"5min":1.268,"15min":1.371},"last":0.64000000000000001}]},{"osd":5,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.83699999999999997,"15min":0.83299999999999996},"min":{"1min":0.70499999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0189999999999999,"5min":1.296,"15min":1.296},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83299999999999996,"15min":0.82199999999999995},"min":{"1min":0.65300000000000002,"5min":0.61299999999999999,"15min":0.52800000000000002},"max":{"1min":1.1200000000000001,"5min":1.141,"15min":1.2290000000000001},"last":0.67100000000000004}]},{"osd":6,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.86699999999999999,"15min":0.85199999999999998},"min":{"1min":0.68100000000000005,"5min":0.61399999999999999,"15min":0.58799999999999997},"max":{"1min":1.0840000000000001,"5min":1.359,"15min":1.359},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.86799999999999999,"15min":0.84899999999999998},"min":{"1min":0.71999999999999997,"5min":0.60499999999999998,"15min":0.59599999999999997},"max":{"1min":1.175,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Dec 19 17:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.91700000000000004,"15min":0.89700000000000002},"min":{"1min":0.71399999999999997,"5min":0.65100000000000002,"15min":0.60599999999999998},"max":{"1min":1.21,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.78900000000000003,"5min":0.63600000000000001,"15min":0.59599999999999997},"max":{"1min":1.101,"5min":1.288,"15min":1.3069999999999999},"last":0.71699999999999997}]}]},{"osd":2,"up_from":15,"seq":64424510342,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63352,"kb_used_data":1968,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308488,"statfs":{"total":96636764160,"available":96571891712,"internally_reserved":0,"allocated":2015232,"data_stored":1045293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44700000000000001,"15min":0.45800000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59999999999999998,"5min":0.71499999999999997,"15min":0.80800000000000005},"last":0.373},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32000000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88600000000000001},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.62,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.36499999999999999},"max":{"1min":0.80200000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.51900000000000002},"min":{"1min":0.375,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.67900000000000005,"5min":0.79600000000000004,"15min":0.93300000000000005},"last":0.42699999999999999}]},{"osd":3,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.60799999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82699999999999996,"5min":0.88,"15min":0.93300000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.38400000000000001,"15min":0.35599999999999998},"max":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.93300000000000005},"last":0.55000000000000004}]},{"osd":4,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.80100000000000005,"15min":0.80600000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.54000000000000004},"max":{"1min":1.1020000000000001,"5min":1.1639999999999999,"15min":1.3360000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79800000000000004,"15min":0.79000000000000004},"min":{"1min":0.53600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.1180000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80300000000000005,"15min":0.80500000000000005},"min":{"1min":0.60499999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.071,"5min":1.339,"15min":1.339},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.083,"5min":1.351,"15min":1.351},"last":0.85599999999999998}]},{"osd":6,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84199999999999997,"15min":0.84299999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.55800000000000005},"max":{"1min":1.0349999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.46600000000000003},"max":{"1min":1.1399999999999999,"5min":1.1619999999999999,"15min":1.171},"last":0.83899999999999997}]},{"osd":7,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.82699999999999996},"min":{"1min":0.69399999999999995,"5min":0.58699999999999997,"15min":0.53500000000000003},"max":{"1min":1.21,"5min":1.2629999999999999,"15min":1.3580000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.64900000000000002,"5min":0.63300000000000001,"15min":0.61899999999999999},"max":{"1min":1.1279999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.81399999999999995}]}]},{"osd":1,"up_from":15,"seq":64424510343,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55140,"kb_used_data":1948,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316700,"statfs":{"total":96636764160,"available":96580300800,"internally_reserved":0,"allocated":1994752,"data_stored":1037170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.45300000000000001,"15min":0.45800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.627,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53200000000000003,"15min":0.52700000000000002},"min":{"1min":0.36499999999999999,"5min":0.34799999999999998,"15min":0.34499999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.89200000000000002},"last":0.68999999999999995}]},{"osd":2,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53100000000000003,"15min":0.52100000000000002},"min":{"1min":0.38700000000000001,"5min":0.36699999999999999,"15min":0.308},"max":{"1min":0.71999999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34599999999999997},"max":{"1min":0.80000000000000004,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.58299999999999996}]},{"osd":3,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.49199999999999999,"5min":0.45600000000000002,"15min":0.42299999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.55200000000000005},"min":{"1min":0.42499999999999999,"5min":0.35799999999999998,"15min":0.35499999999999998},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.85199999999999998},"last":0.53200000000000003}]},{"osd":4,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.81699999999999995,"15min":0.81000000000000005},"min":{"1min":0.60499999999999998,"5min":0.54500000000000004,"15min":0.53200000000000003},"max":{"1min":1.091,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80200000000000005,"15min":0.80700000000000005},"min":{"1min":0.56999999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0369999999999999,"5min":1.355,"15min":1.355},"last":0.71199999999999997}]},{"osd":5,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.82499999999999996,"15min":0.81200000000000006},"min":{"1min":0.57599999999999996,"5min":0.55600000000000005,"15min":0.54600000000000004},"max":{"1min":1.214,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.82699999999999996,"15min":0.81999999999999995},"min":{"1min":0.623,"5min":0.61899999999999999,"15min":0.59699999999999998},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.74299999999999999}]},{"osd":6,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.82199999999999995,"15min":0.82099999999999995},"min":{"1min":0.64800000000000002,"5min":0.59099999999999997,"15min":0.52200000000000002},"max":{"1min":1.026,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.83899999999999997,"15min":0.82999999999999996},"min":{"1min":0.56999999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.157,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue Dec 19 17:56:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.88300000000000001,"15min":0.86399999999999999},"min":{"1min":0.71799999999999997,"5min":0.64500000000000002,"15min":0.625},"max":{"1min":1.1950000000000001,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.84299999999999997,"15min":0.85399999999999998},"min":{"1min":0.63300000000000001,"5min":0.628,"15min":0.59499999999999997},"max":{"1min":1.177,"5min":1.258,"15min":1.258},"last":0.81499999999999995}]}]},{"osd":0,"up_from":14,"seq":60129543048,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55720,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316120,"statfs":{"total":96636764160,"available":96579706880,"internally_reserved":0,"allocated":2588672,"data_stored":1627538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46000000000000002,"15min":0.45500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64100000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.29599999999999999},"max":{"1min":0.71499999999999997,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.309}]},{"osd":2,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.35699999999999998,"15min":0.34300000000000003},"max":{"1min":0.60899999999999999,"5min":0.78500000000000003,"15min":0.83299999999999996},"last":0.437},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65900000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.35499999999999998}]},{"osd":3,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.65000000000000002,"5min":0.84799999999999998,"15min":0.90400000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.501,"5min":0.497,"15min":0.45000000000000001},"max":{"1min":0.71299999999999997,"5min":0.90000000000000002,"15min":0.93500000000000005},"last":0.55000000000000004}]},{"osd":4,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.44700000000000001},"max":{"1min":1.1080000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52400000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80000000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.50700000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.169},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81799999999999995,"15min":0.82499999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.351},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1619999999999999,"5min":1.173,"15min":1.2350000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82699999999999996,"15min":0.83099999999999996},"min":{"1min":0.63300000000000001,"5min":0.60299999999999998,"15min":0.505},"max":{"1min":1.079,"5min":1.1020000000000001,"15min":1.329},"last":0.63300000000000001}]},{"osd":7,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84999999999999998,"15min":0.85599999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2909999999999999},"last":0.64300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T17:57:45.043 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T17:57:45.044 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T17:57:45.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.231+0000 7f51d9404640 1 Processor -- start 2023-12-19T17:57:45.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.231+0000 7f51d9404640 1 -- start start 2023-12-19T17:57:45.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d9404640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f51d40ac850 0x7f51d406b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d9404640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51d406bed0 0x7f51d415d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d9404640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d40b1940 con 0x7f51d4153d30 2023-12-19T17:57:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d9404640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d40ade30 con 0x7f51d406bed0 2023-12-19T17:57:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d9404640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d40acd50 con 0x7f51d40ac850 2023-12-19T17:57:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51d406bed0 0x7f51d415d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51d406bed0 0x7f51d415d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46310/0 (socket says 172.21.15.26:46310) 2023-12-19T17:57:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d37fe640 1 -- 172.21.15.26:0/2281030683 learned_addr learned my addr 172.21.15.26:0/2281030683 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51cbfff640 1 --2- 172.21.15.26:0/2281030683 >> v2:172.21.15.160:3300/0 conn(0x7f51d40ac850 0x7f51d406b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:45.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d37fe640 1 -- 172.21.15.26:0/2281030683 >> v2:172.21.15.160:3300/0 conn(0x7f51d40ac850 msgr2=0x7f51d406b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:45.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d37fe640 1 --2- 172.21.15.26:0/2281030683 >> v2:172.21.15.160:3300/0 conn(0x7f51d40ac850 0x7f51d406b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d37fe640 1 -- 172.21.15.26:0/2281030683 >> v1:172.21.15.26:6789/0 conn(0x7f51d4153d30 legacy=0x7f51d40cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d37fe640 1 -- 172.21.15.26:0/2281030683 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d40ad030 con 0x7f51d406bed0 2023-12-19T17:57:45.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d37fe640 1 --2- 172.21.15.26:0/2281030683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51d406bed0 0x7f51d415d2a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f51d406c650 tx=0x7f51cc01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817fa298bf1b59f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d27fc640 1 -- 172.21.15.26:0/2281030683 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0af400 con 0x7f51d406bed0 2023-12-19T17:57:45.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.235+0000 7f51d27fc640 1 -- 172.21.15.26:0/2281030683 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc0ab050 con 0x7f51d406bed0 2023-12-19T17:57:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d27fc640 1 -- 172.21.15.26:0/2281030683 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0af780 con 0x7f51d406bed0 2023-12-19T17:57:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 -- 172.21.15.26:0/2281030683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51d406bed0 msgr2=0x7f51d415d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 --2- 172.21.15.26:0/2281030683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51d406bed0 0x7f51d415d2a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f51d406c650 tx=0x7f51cc01f1c0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 -- 172.21.15.26:0/2281030683 shutdown_connections 2023-12-19T17:57:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 --2- 172.21.15.26:0/2281030683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51d406bed0 0x7f51d415d2a0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 --2- 172.21.15.26:0/2281030683 >> v2:172.21.15.160:3300/0 conn(0x7f51d40ac850 0x7f51d406b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 -- 172.21.15.26:0/2281030683 >> 172.21.15.26:0/2281030683 conn(0x7f51d40658e0 msgr2=0x7f51d4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:45.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 -- 172.21.15.26:0/2281030683 shutdown_connections 2023-12-19T17:57:45.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 -- 172.21.15.26:0/2281030683 wait complete. 2023-12-19T17:57:45.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 Processor -- start 2023-12-19T17:57:45.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 -- start start 2023-12-19T17:57:45.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51d406bed0 0x7f51d41507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:45.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51d40ac850 0x7f51d40b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:45.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51d4153d30 0x7f51d414c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:45.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d414f810 con 0x7f51d406bed0 2023-12-19T17:57:45.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d4150350 con 0x7f51d40ac850 2023-12-19T17:57:45.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d9404640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d414fa20 con 0x7f51d4153d30 2023-12-19T17:57:45.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51d406bed0 0x7f51d41507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:45.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51d406bed0 0x7f51d41507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46314/0 (socket says 172.21.15.26:46314) 2023-12-19T17:57:45.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d2ffd640 1 -- 172.21.15.26:0/3942783843 learned_addr learned my addr 172.21.15.26:0/3942783843 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:45.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51cbfff640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51d40ac850 0x7f51d40b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:45.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.239+0000 7f51d37fe640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51d4153d30 0x7f51d414c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:45.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51d2ffd640 1 -- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51d4153d30 msgr2=0x7f51d414c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51d2ffd640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51d4153d30 0x7f51d414c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51d2ffd640 1 -- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51d40ac850 msgr2=0x7f51d40b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51d2ffd640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51d40ac850 0x7f51d40b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51d2ffd640 1 -- 172.21.15.26:0/3942783843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51cc009b30 con 0x7f51d406bed0 2023-12-19T17:57:45.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51d2ffd640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51d406bed0 0x7f51d41507e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f51c400c3c0 tx=0x7f51c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:45.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51cb7fe640 1 -- 172.21.15.26:0/3942783843 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c40052d0 con 0x7f51d406bed0 2023-12-19T17:57:45.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51cb7fe640 1 -- 172.21.15.26:0/3942783843 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c4005430 con 0x7f51d406bed0 2023-12-19T17:57:45.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51cb7fe640 1 -- 172.21.15.26:0/3942783843 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4003840 con 0x7f51d406bed0 2023-12-19T17:57:45.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51d9404640 1 -- 172.21.15.26:0/3942783843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d414fd00 con 0x7f51d406bed0 2023-12-19T17:57:45.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51d9404640 1 -- 172.21.15.26:0/3942783843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d40b3360 con 0x7f51d406bed0 2023-12-19T17:57:45.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51d9404640 1 -- 172.21.15.26:0/3942783843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d404e850 con 0x7f51d406bed0 2023-12-19T17:57:45.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51cb7fe640 1 -- 172.21.15.26:0/3942783843 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f51c40039a0 con 0x7f51d406bed0 2023-12-19T17:57:45.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51cb7fe640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51b4065bb0 0x7f51b4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:45.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51cb7fe640 1 -- 172.21.15.26:0/3942783843 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f51c4082a00 con 0x7f51d406bed0 2023-12-19T17:57:45.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.243+0000 7f51cbfff640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51b4065bb0 0x7f51b4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:45.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.247+0000 7f51cbfff640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51b4065bb0 0x7f51b4068070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f51d4150e40 tx=0x7f51bc008040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:45.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.247+0000 7f51cb7fe640 1 -- 172.21.15.26:0/3942783843 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f51c4050080 con 0x7f51d406bed0 2023-12-19T17:57:45.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.459+0000 7f51d9404640 1 -- 172.21.15.26:0/3942783843 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f51d40cb850 con 0x7f51d406bed0 2023-12-19T17:57:45.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.467+0000 7f51cb7fe640 1 -- 172.21.15.26:0/3942783843 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v42) v1 ==== 190+0+0 (secure 0 0 0) 0x7f51c40536a0 con 0x7f51d406bed0 2023-12-19T17:57:45.467 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2023-12-19T17:57:45.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.467+0000 7f51d9404640 1 -- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51b4065bb0 msgr2=0x7f51b4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.467+0000 7f51d9404640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51b4065bb0 0x7f51b4068070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f51d4150e40 tx=0x7f51bc008040 comp rx=0 tx=0).stop 2023-12-19T17:57:45.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.467+0000 7f51d9404640 1 -- 172.21.15.26:0/3942783843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51d406bed0 msgr2=0x7f51d41507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.467+0000 7f51d9404640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51d406bed0 0x7f51d41507e0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f51c400c3c0 tx=0x7f51c400c880 comp rx=0 tx=0).stop 2023-12-19T17:57:45.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.467+0000 7f51d9404640 1 -- 172.21.15.26:0/3942783843 shutdown_connections 2023-12-19T17:57:45.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.467+0000 7f51d9404640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f51b4065bb0 0x7f51b4068070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.467+0000 7f51d9404640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51d4153d30 0x7f51d414c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.467+0000 7f51d9404640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51d40ac850 0x7f51d40b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.467+0000 7f51d9404640 1 --2- 172.21.15.26:0/3942783843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51d406bed0 0x7f51d41507e0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.467+0000 7f51d9404640 1 -- 172.21.15.26:0/3942783843 >> 172.21.15.26:0/3942783843 conn(0x7f51d40658e0 msgr2=0x7f51d40af540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:45.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.467+0000 7f51d9404640 1 -- 172.21.15.26:0/3942783843 shutdown_connections 2023-12-19T17:57:45.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.467+0000 7f51d9404640 1 -- 172.21.15.26:0/3942783843 wait complete. 2023-12-19T17:57:45.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:57:45.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.631+0000 7f216efa2640 1 Processor -- start 2023-12-19T17:57:45.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.631+0000 7f216efa2640 1 -- start start 2023-12-19T17:57:45.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.631+0000 7f216efa2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2168164f00 0x7f21680cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.631+0000 7f216efa2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21680ade10 con 0x7f21680ada30 2023-12-19T17:57:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.631+0000 7f216efa2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21680cc9d0 con 0x7f21680cb3a0 2023-12-19T17:57:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.631+0000 7f216efa2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21680b16e0 con 0x7f2168164f00 2023-12-19T17:57:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.631+0000 7f216dfa0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f21680ada30 0x7f21680c9860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35240/0 (socket says 172.21.15.26:35240) 2023-12-19T17:57:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.631+0000 7f216dfa0640 1 -- 172.21.15.26:0/3926824580 learned_addr learned my addr 172.21.15.26:0/3926824580 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.631+0000 7f216d79f640 1 --2- 172.21.15.26:0/3926824580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2168164f00 0x7f21680cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.631+0000 7f216cf9e640 1 -- 172.21.15.26:0/3926824580 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724896662 0 0) 0x7f21680ade10 con 0x7f21680ada30 2023-12-19T17:57:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.635+0000 7f216cf9e640 1 -- 172.21.15.26:0/3926824580 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2154003530 con 0x7f21680ada30 2023-12-19T17:57:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.635+0000 7f216cf9e640 1 -- 172.21.15.26:0/3926824580 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301285804 0 0) 0x7f21680cc9d0 con 0x7f21680cb3a0 2023-12-19T17:57:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.635+0000 7f216cf9e640 1 -- 172.21.15.26:0/3926824580 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21680ade10 con 0x7f21680cb3a0 2023-12-19T17:57:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.635+0000 7f216cf9e640 1 -- 172.21.15.26:0/3926824580 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3242240209 0 0) 0x7f2154003530 con 0x7f21680ada30 2023-12-19T17:57:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.635+0000 7f216d79f640 1 -- 172.21.15.26:0/3926824580 >> v1:172.21.15.160:6789/0 conn(0x7f21680cb3a0 legacy=0x7f21680cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.635+0000 7f216d79f640 1 -- 172.21.15.26:0/3926824580 >> v1:172.21.15.26:6789/0 conn(0x7f21680ada30 legacy=0x7f21680c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.635+0000 7f216d79f640 1 -- 172.21.15.26:0/3926824580 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21680cb7f0 con 0x7f2168164f00 2023-12-19T17:57:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.635+0000 7f216d79f640 1 --2- 172.21.15.26:0/3926824580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2168164f00 0x7f21680cae60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f21580097f0 tx=0x7f215801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69e671da88e16d88 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.635+0000 7f216cf9e640 1 -- 172.21.15.26:0/3926824580 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164001680 con 0x7f2168164f00 2023-12-19T17:57:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.635+0000 7f216cf9e640 1 -- 172.21.15.26:0/3926824580 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21580a7050 con 0x7f2168164f00 2023-12-19T17:57:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.635+0000 7f216cf9e640 1 -- 172.21.15.26:0/3926824580 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21580ab620 con 0x7f2168164f00 2023-12-19T17:57:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 -- 172.21.15.26:0/3926824580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2168164f00 msgr2=0x7f21680cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 --2- 172.21.15.26:0/3926824580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2168164f00 0x7f21680cae60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f21580097f0 tx=0x7f215801c3e0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 -- 172.21.15.26:0/3926824580 shutdown_connections 2023-12-19T17:57:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 --2- 172.21.15.26:0/3926824580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2168164f00 0x7f21680cae60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 -- 172.21.15.26:0/3926824580 >> 172.21.15.26:0/3926824580 conn(0x7f216806b0f0 msgr2=0x7f216806b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 -- 172.21.15.26:0/3926824580 shutdown_connections 2023-12-19T17:57:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 -- 172.21.15.26:0/3926824580 wait complete. 2023-12-19T17:57:45.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 Processor -- start 2023-12-19T17:57:45.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 -- start start 2023-12-19T17:57:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21680ada30 0x7f216815bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21680cb3a0 0x7f2168161f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2168164f00 0x7f21680c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21680c4350 con 0x7f21680cb3a0 2023-12-19T17:57:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216815d840 con 0x7f21680ada30 2023-12-19T17:57:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216efa2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216815db20 con 0x7f2168164f00 2023-12-19T17:57:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216d79f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21680cb3a0 0x7f2168161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216d79f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21680cb3a0 0x7f2168161f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46316/0 (socket says 172.21.15.26:46316) 2023-12-19T17:57:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216d79f640 1 -- 172.21.15.26:0/3348852635 learned_addr learned my addr 172.21.15.26:0/3348852635 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216e7a1640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2168164f00 0x7f21680c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216dfa0640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21680ada30 0x7f216815bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216d79f640 1 -- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2168164f00 msgr2=0x7f21680c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216d79f640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2168164f00 0x7f21680c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216d79f640 1 -- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21680ada30 msgr2=0x7f216815bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216d79f640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21680ada30 0x7f216815bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.639+0000 7f216d79f640 1 -- 172.21.15.26:0/3348852635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2158009420 con 0x7f21680cb3a0 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.643+0000 7f216dfa0640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21680ada30 0x7f216815bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.643+0000 7f216d79f640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21680cb3a0 0x7f2168161f20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f2158009740 tx=0x7f2158019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.643+0000 7f215e7fc640 1 -- 172.21.15.26:0/3348852635 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158004020 con 0x7f21680cb3a0 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.643+0000 7f215e7fc640 1 -- 172.21.15.26:0/3348852635 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2158004180 con 0x7f21680cb3a0 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.643+0000 7f216efa2640 1 -- 172.21.15.26:0/3348852635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216815de00 con 0x7f21680cb3a0 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.643+0000 7f215e7fc640 1 -- 172.21.15.26:0/3348852635 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21580044f0 con 0x7f21680cb3a0 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.643+0000 7f216efa2640 1 -- 172.21.15.26:0/3348852635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21680c2e40 con 0x7f21680cb3a0 2023-12-19T17:57:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.643+0000 7f216efa2640 1 -- 172.21.15.26:0/3348852635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2168067400 con 0x7f21680cb3a0 2023-12-19T17:57:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.643+0000 7f215e7fc640 1 -- 172.21.15.26:0/3348852635 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f21580b2040 con 0x7f21680cb3a0 2023-12-19T17:57:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.643+0000 7f215e7fc640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f214c065bb0 0x7f214c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.643+0000 7f215e7fc640 1 -- 172.21.15.26:0/3348852635 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f215811ca40 con 0x7f21680cb3a0 2023-12-19T17:57:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.643+0000 7f216dfa0640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f214c065bb0 0x7f214c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.647+0000 7f216dfa0640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f214c065bb0 0x7f214c068070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2164005890 tx=0x7f2164009040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.647+0000 7f215e7fc640 1 -- 172.21.15.26:0/3348852635 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f21580e9e60 con 0x7f21680cb3a0 2023-12-19T17:57:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.839+0000 7f216efa2640 1 -- 172.21.15.26:0/3348852635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f216815e2b0 con 0x7f21680cb3a0 2023-12-19T17:57:45.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.843+0000 7f215e7fc640 1 -- 172.21.15.26:0/3348852635 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f21580b0070 con 0x7f21680cb3a0 2023-12-19T17:57:45.844 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:45.844 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2850013418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2850013418},{"type":"v1","addr":"172.21.15.26:6835","nonce":2850013418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2213513866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2213513866},{"type":"v1","addr":"172.21.15.160:6836","nonce":2213513866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4292779848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4292779848},{"type":"v1","addr":"172.21.15.160:6837","nonce":4292779848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:57:40.517697+0000","modified":"2023-12-19T17:57:43.536734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8575":{"gid":8575,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/2838422258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2838422258},{"type":"v1","addr":"172.21.15.26:6834","nonce":2838422258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-19T17:57:45.844 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 73 2023-12-19T17:57:45.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.843+0000 7f216efa2640 1 -- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f214c065bb0 msgr2=0x7f214c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.843+0000 7f216efa2640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f214c065bb0 0x7f214c068070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2164005890 tx=0x7f2164009040 comp rx=0 tx=0).stop 2023-12-19T17:57:45.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.843+0000 7f216efa2640 1 -- 172.21.15.26:0/3348852635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21680cb3a0 msgr2=0x7f2168161f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:45.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.843+0000 7f216efa2640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21680cb3a0 0x7f2168161f20 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f2158009740 tx=0x7f2158019180 comp rx=0 tx=0).stop 2023-12-19T17:57:45.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.843+0000 7f216efa2640 1 -- 172.21.15.26:0/3348852635 shutdown_connections 2023-12-19T17:57:45.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.843+0000 7f216efa2640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f214c065bb0 0x7f214c068070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.843+0000 7f216efa2640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2168164f00 0x7f21680c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.843+0000 7f216efa2640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21680cb3a0 0x7f2168161f20 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.843+0000 7f216efa2640 1 --2- 172.21.15.26:0/3348852635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21680ada30 0x7f216815bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:45.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.843+0000 7f216efa2640 1 -- 172.21.15.26:0/3348852635 >> 172.21.15.26:0/3348852635 conn(0x7f216806b0f0 msgr2=0x7f21680b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:45.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.843+0000 7f216efa2640 1 -- 172.21.15.26:0/3348852635 shutdown_connections 2023-12-19T17:57:45.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:45.843+0000 7f216efa2640 1 -- 172.21.15.26:0/3348852635 wait complete. 2023-12-19T17:57:46.015 INFO:tasks.ceph.mds.c.smithi026.stderr:2023-12-19T17:57:46.011+0000 7fd100fdf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T17:57:46.268 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T17:57:40.517697+0000', 'modified': '2023-12-19T17:57:43.536734+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8575}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8575': {'gid': 8575, 'name': 'c', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6834/2838422258', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 2838422258}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 2838422258}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T17:57:46.268 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T17:57:46.268 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-19T17:57:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.415+0000 7fcae6c55640 1 Processor -- start 2023-12-19T17:57:46.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.415+0000 7fcae6c55640 1 -- start start 2023-12-19T17:57:46.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.415+0000 7fcae6c55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcae006b3d0 0x7fcae00ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.415+0000 7fcae6c55640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae006bc30 con 0x7fcae0154220 2023-12-19T17:57:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.415+0000 7fcae6c55640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae00cc6e0 con 0x7fcae00cec40 2023-12-19T17:57:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.415+0000 7fcae6c55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00cf0d0 con 0x7fcae006b3d0 2023-12-19T17:57:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.415+0000 7fcae49f1640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcae0154220 0x7fcae015aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59552/0 (socket says 172.21.15.26:59552) 2023-12-19T17:57:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.415+0000 7fcae49f1640 1 -- 172.21.15.26:0/621531352 learned_addr learned my addr 172.21.15.26:0/621531352 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.415+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808412852 0 0) 0x7fcae006bc30 con 0x7fcae0154220 2023-12-19T17:57:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.415+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcad0003530 con 0x7fcae0154220 2023-12-19T17:57:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.415+0000 7fcadffff640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcae006b3d0 0x7fcae00ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.415+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1779162153 0 0) 0x7fcad0003530 con 0x7fcae0154220 2023-12-19T17:57:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.415+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcae006b3d0 msgr2=0x7fcae00ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcae006b3d0 0x7fcae00ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 >> v1:172.21.15.160:6789/0 conn(0x7fcae00cec40 legacy=0x7fcae015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae006b950 con 0x7fcae0154220 2023-12-19T17:57:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fcad4001680 con 0x7fcae0154220 2023-12-19T17:57:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 >> v1:172.21.15.26:6789/0 conn(0x7fcae0154220 legacy=0x7fcae015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad00039b0 0x7fcad0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcae006b3d0 0x7fcad00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcad0004d10 0x7fcad0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00cf0d0 con 0x7fcad00039b0 2023-12-19T17:57:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad000ba40 con 0x7fcad0004d10 2023-12-19T17:57:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad000bc00 con 0x7fcae006b3d0 2023-12-19T17:57:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcae49f1640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad00039b0 0x7fcad0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadffff640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcae006b3d0 0x7fcad00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcae51f2640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcad0004d10 0x7fcad0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcae49f1640 1 -- 172.21.15.26:0/621531352 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcae006b3d0 msgr2=0x7fcad00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcae49f1640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcae006b3d0 0x7fcad00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcae49f1640 1 -- 172.21.15.26:0/621531352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcad0004d10 msgr2=0x7fcad0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcae49f1640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcad0004d10 0x7fcad0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcae49f1640 1 -- 172.21.15.26:0/621531352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad0003530 con 0x7fcad00039b0 2023-12-19T17:57:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcae51f2640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcad0004d10 0x7fcad0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadffff640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcae006b3d0 0x7fcad00047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:46.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcae49f1640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad00039b0 0x7fcad0003dd0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fcad4001ec0 tx=0x7fcad401cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145fb3345c5e013a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:46.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4005ae0 con 0x7fcad00039b0 2023-12-19T17:57:46.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad4005c40 con 0x7fcad00039b0 2023-12-19T17:57:46.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.419+0000 7fcadf7fe640 1 -- 172.21.15.26:0/621531352 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad40a57e0 con 0x7fcad00039b0 2023-12-19T17:57:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 -- 172.21.15.26:0/621531352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad00039b0 msgr2=0x7fcad0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 --2- 172.21.15.26:0/621531352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcad00039b0 0x7fcad0003dd0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fcad4001ec0 tx=0x7fcad401cbf0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae51f2640 1 -- 172.21.15.26:0/621531352 reap_dead start 2023-12-19T17:57:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 -- 172.21.15.26:0/621531352 shutdown_connections 2023-12-19T17:57:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 -- 172.21.15.26:0/621531352 >> 172.21.15.26:0/621531352 conn(0x7fcae00658e0 msgr2=0x7fcae0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 -- 172.21.15.26:0/621531352 shutdown_connections 2023-12-19T17:57:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 -- 172.21.15.26:0/621531352 wait complete. 2023-12-19T17:57:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 Processor -- start 2023-12-19T17:57:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 -- start start 2023-12-19T17:57:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcad00039b0 0x7fcae015c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae01f6af0 0x7fcae01f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae01f6f70 0x7fcae01f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae01f31b0 con 0x7fcae01f6af0 2023-12-19T17:57:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae006f410 con 0x7fcae01f6f70 2023-12-19T17:57:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcae6c55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae01560b0 con 0x7fcad00039b0 2023-12-19T17:57:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcadffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae01f6af0 0x7fcae01f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcadffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae01f6af0 0x7fcae01f2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57732/0 (socket says 172.21.15.26:57732) 2023-12-19T17:57:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.423+0000 7fcadffff640 1 -- 172.21.15.26:0/4104591147 learned_addr learned my addr 172.21.15.26:0/4104591147 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcae49f1640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcad00039b0 0x7fcae015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcae51f2640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae01f6f70 0x7fcae01f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcadffff640 1 -- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcad00039b0 msgr2=0x7fcae015c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcadffff640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcad00039b0 0x7fcae015c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcadffff640 1 -- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae01f6f70 msgr2=0x7fcae01f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcadffff640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae01f6f70 0x7fcae01f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcadffff640 1 -- 172.21.15.26:0/4104591147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae0156390 con 0x7fcae01f6af0 2023-12-19T17:57:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcae51f2640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae01f6f70 0x7fcae01f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcadffff640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae01f6af0 0x7fcae01f2730 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009fd0 tx=0x7fcacc011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcadd7fa640 1 -- 172.21.15.26:0/4104591147 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc004220 con 0x7fcae01f6af0 2023-12-19T17:57:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcadd7fa640 1 -- 172.21.15.26:0/4104591147 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc004380 con 0x7fcae01f6af0 2023-12-19T17:57:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcadd7fa640 1 -- 172.21.15.26:0/4104591147 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc00f9e0 con 0x7fcae01f6af0 2023-12-19T17:57:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcae6c55640 1 -- 172.21.15.26:0/4104591147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae006f6b0 con 0x7fcae01f6af0 2023-12-19T17:57:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcae6c55640 1 -- 172.21.15.26:0/4104591147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae0143ad0 con 0x7fcae01f6af0 2023-12-19T17:57:46.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcadd7fa640 1 -- 172.21.15.26:0/4104591147 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fcacc00fb40 con 0x7fcae01f6af0 2023-12-19T17:57:46.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.427+0000 7fcadd7fa640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcac4065b80 0x7fcac4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcadd7fa640 1 -- 172.21.15.26:0/4104591147 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fcacc083af0 con 0x7fcae01f6af0 2023-12-19T17:57:46.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 Processor -- start 2023-12-19T17:57:46.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae49f1640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcac4065b80 0x7fcac4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 -- start start 2023-12-19T17:57:46.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae49f1640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcac4065b80 0x7fcac4068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcad40ab000 tx=0x7fcad40a7070 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:46.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e129b8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 0x555e1296dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 0x555e1296e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e12791c90 con 0x555e129b8990 2023-12-19T17:57:46.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e127c3220 con 0x555e129b8070 2023-12-19T17:57:46.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e127c07f0 con 0x555e1296e1c0 2023-12-19T17:57:46.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcadffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 0x555e1296dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcadffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 0x555e1296dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57748/0 (socket says 172.21.15.26:57748) 2023-12-19T17:57:46.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcadffff640 1 -- 172.21.15.26:0/3081177857 learned_addr learned my addr 172.21.15.26:0/3081177857 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:46.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcadffff640 1 -- 172.21.15.26:0/3081177857 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 msgr2=0x555e1296e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:46.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcadffff640 1 --2- 172.21.15.26:0/3081177857 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 0x555e1296e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae49f1640 1 --2- 172.21.15.26:0/3081177857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e129b8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcadffff640 1 -- 172.21.15.26:0/3081177857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 msgr2=0x555e129b8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:46.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcadffff640 1 --2- 172.21.15.26:0/3081177857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e129b8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcadffff640 1 -- 172.21.15.26:0/3081177857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e124b8890 con 0x555e129b8990 2023-12-19T17:57:46.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae49f1640 1 --2- 172.21.15.26:0/3081177857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e129b8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:57:46.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcadffff640 1 --2- 172.21.15.26:0/3081177857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 0x555e1296dc80 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fcacc05c790 tx=0x7fcacc0546f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c8a460101d5a1d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:46.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcaca7fc640 1 -- 172.21.15.26:0/3081177857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc05ccc0 con 0x555e129b8990 2023-12-19T17:57:46.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcaca7fc640 1 -- 172.21.15.26:0/3081177857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc053130 con 0x555e129b8990 2023-12-19T17:57:46.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcaca7fc640 1 -- 172.21.15.26:0/3081177857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc081c90 con 0x555e129b8990 2023-12-19T17:57:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 -- 172.21.15.26:0/3081177857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 msgr2=0x555e1296dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/3081177857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 0x555e1296dc80 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fcacc05c790 tx=0x7fcacc0546f0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 -- 172.21.15.26:0/3081177857 shutdown_connections 2023-12-19T17:57:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/3081177857 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 0x555e1296e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/3081177857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 0x555e1296dc80 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/3081177857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e129b8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 -- 172.21.15.26:0/3081177857 >> 172.21.15.26:0/3081177857 conn(0x555e12974950 msgr2=0x555e1296c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 -- 172.21.15.26:0/3081177857 shutdown_connections 2023-12-19T17:57:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 -- 172.21.15.26:0/3081177857 wait complete. 2023-12-19T17:57:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.431+0000 7fcae9fb6000 1 -- start start 2023-12-19T17:57:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 1 Processor -- start 2023-12-19T17:57:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 0x555e129bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e129bfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 0x555e129a6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e127bc0e0 con 0x555e129b8990 2023-12-19T17:57:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e1275b310 con 0x555e129b8070 2023-12-19T17:57:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e12779770 con 0x555e1296e1c0 2023-12-19T17:57:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 10 client.0 _ll_register_callbacks cb 0x555e12959b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T17:57:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae51f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 0x555e129a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae51f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 0x555e129a6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57760/0 (socket says 172.21.15.26:57760) 2023-12-19T17:57:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae51f2640 1 -- 172.21.15.26:0/3914076280 learned_addr learned my addr 172.21.15.26:0/3914076280 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae49f1640 1 --2- 172.21.15.26:0/3914076280 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 0x555e129bf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcadffff640 1 --2- 172.21.15.26:0/3914076280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e129bfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae51f2640 1 -- 172.21.15.26:0/3914076280 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 msgr2=0x555e129bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae51f2640 1 --2- 172.21.15.26:0/3914076280 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 0x555e129bf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae51f2640 1 -- 172.21.15.26:0/3914076280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 msgr2=0x555e129bfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae51f2640 1 --2- 172.21.15.26:0/3914076280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e129bfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae51f2640 1 -- 172.21.15.26:0/3914076280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae0143ad0 con 0x555e129b8990 2023-12-19T17:57:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae51f2640 1 --2- 172.21.15.26:0/3914076280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 0x555e129a6030 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fcae006c230 tx=0x7fcad80019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c6110805fb5fab server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcabf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T17:57:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcabf7fe640 1 -- 172.21.15.26:0/3914076280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8005860 con 0x555e129b8990 2023-12-19T17:57:46.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcabf7fe640 1 -- 172.21.15.26:0/3914076280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fcad8003af0 con 0x555e129b8990 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 1 -- 172.21.15.26:0/3914076280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e1278bc10 con 0x555e129b8990 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcabf7fe640 1 -- 172.21.15.26:0/3914076280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcad8014400 con 0x555e129b8990 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 10 client.9147 fetch_fsmap learned FSMap version 73 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 1 -- 172.21.15.26:0/3914076280 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x555e12781260 con 0x555e129b8990 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcabf7fe640 1 -- 172.21.15.26:0/3914076280 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555e1278bc10 con 0x555e129b8990 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcabf7fe640 1 -- 172.21.15.26:0/3914076280 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 73) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcad801c350 con 0x555e129b8990 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 10 client.9147 fetch_fsmap finished waiting for FSMap version 73 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 10 client.9147 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 20 client.9147 populate_metadata read hostname 'smithi026' 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/3914076280 >> [v2:172.21.15.26:6832/2838422258,v1:172.21.15.26:6834/2838422258] conn(0x555e129a6dd0 0x555e129a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 4 client.9147 mds_command: new command op to 8575 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae9fb6000 1 -- 172.21.15.26:0/3914076280 --> [v2:172.21.15.26:6832/2838422258,v1:172.21.15.26:6834/2838422258] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e127baa30 con 0x555e129a6dd0 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae49f1640 1 --2- 172.21.15.26:0/3914076280 >> [v2:172.21.15.26:6832/2838422258,v1:172.21.15.26:6834/2838422258] conn(0x555e129a6dd0 0x555e129a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcabf7fe640 10 client.9147 ms_handle_connect on v2:172.21.15.26:6832/2838422258 2023-12-19T17:57:46.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.435+0000 7fcae49f1640 1 --2- 172.21.15.26:0/3914076280 >> [v2:172.21.15.26:6832/2838422258,v1:172.21.15.26:6834/2838422258] conn(0x555e129a6dd0 0x555e129a71b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be981ac96dafab70 server_cookie=22a371c9267965ee in_seq=0 out_seq=0 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcabf7fe640 1 -- 172.21.15.26:0/3914076280 <== mds.0 v2:172.21.15.26:6832/2838422258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555e127baa30 con 0x555e129a6dd0 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcabf7fe640 10 client.9147 handle_command_reply: tid=0 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcae9fb6000 1 client.9147 shutdown 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcae9fb6000 1 -- 172.21.15.26:0/3914076280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 msgr2=0x555e129a6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/3914076280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 0x555e129a6030 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fcae006c230 tx=0x7fcad80019f0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcae9fb6000 1 -- 172.21.15.26:0/3914076280 shutdown_connections 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/3914076280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e129b8990 0x555e129a6030 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/3914076280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e129bfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/3914076280 >> [v2:172.21.15.26:6832/2838422258,v1:172.21.15.26:6834/2838422258] conn(0x555e129a6dd0 0x555e129a71b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/3914076280 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 0x555e129bf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcabf7fe640 0 client.9147 ms_handle_reset on v2:172.21.15.26:6832/2838422258 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcae9fb6000 1 -- 172.21.15.26:0/3914076280 >> 172.21.15.26:0/3914076280 conn(0x555e12974950 msgr2=0x555e1296cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcae9fb6000 1 -- 172.21.15.26:0/3914076280 shutdown_connections 2023-12-19T17:57:46.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcae9fb6000 1 -- 172.21.15.26:0/3914076280 wait complete. 2023-12-19T17:57:46.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.439+0000 7fcae9fb6000 20 client.9147 trim_cache size 0 max 16384 2023-12-19T17:57:46.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 Processor -- start 2023-12-19T17:57:46.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 -- start start 2023-12-19T17:57:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e12974950 0x555e12a14b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 0x555e12a150b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e12a155f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e12479940 con 0x555e12974950 2023-12-19T17:57:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e129739f0 con 0x555e129b8070 2023-12-19T17:57:46.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e127b7a50 con 0x555e1296e1c0 2023-12-19T17:57:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae49f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e12974950 0x555e12a14b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae49f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e12974950 0x555e12a14b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57764/0 (socket says 172.21.15.26:57764) 2023-12-19T17:57:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae49f1640 1 -- 172.21.15.26:0/2815903529 learned_addr learned my addr 172.21.15.26:0/2815903529 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcadffff640 1 --2- 172.21.15.26:0/2815903529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 0x555e12a150b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae51f2640 1 --2- 172.21.15.26:0/2815903529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e12a155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae49f1640 1 -- 172.21.15.26:0/2815903529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 msgr2=0x555e12a150b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae49f1640 1 --2- 172.21.15.26:0/2815903529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 0x555e12a150b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae49f1640 1 -- 172.21.15.26:0/2815903529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 msgr2=0x555e12a155f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae49f1640 1 --2- 172.21.15.26:0/2815903529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e12a155f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae49f1640 1 -- 172.21.15.26:0/2815903529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e127673b0 con 0x555e12974950 2023-12-19T17:57:46.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae51f2640 1 --2- 172.21.15.26:0/2815903529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e12a155f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae49f1640 1 --2- 172.21.15.26:0/2815903529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e12974950 0x555e12a14b70 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fcad401c660 tx=0x7fcad4001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1876e804eb19fb31 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcacaffd640 1 -- 172.21.15.26:0/2815903529 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad40bf030 con 0x555e12974950 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcacaffd640 1 -- 172.21.15.26:0/2815903529 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad40c0040 con 0x555e12974950 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcacaffd640 1 -- 172.21.15.26:0/2815903529 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4004e80 con 0x555e12974950 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 -- 172.21.15.26:0/2815903529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e12974950 msgr2=0x555e12a14b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/2815903529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e12974950 0x555e12a14b70 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fcad401c660 tx=0x7fcad4001a10 comp rx=0 tx=0).stop 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 -- 172.21.15.26:0/2815903529 shutdown_connections 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/2815903529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e129b8070 0x555e12a155f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/2815903529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e1296e1c0 0x555e12a150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/2815903529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e12974950 0x555e12a14b70 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 -- 172.21.15.26:0/2815903529 >> 172.21.15.26:0/2815903529 conn(0x555e129b77f0 msgr2=0x555e129a3180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 -- 172.21.15.26:0/2815903529 shutdown_connections 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 -- 172.21.15.26:0/2815903529 wait complete. 2023-12-19T17:57:46.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.455+0000 7fcae9fb6000 1 -- start start 2023-12-19T17:57:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 1 Processor -- start 2023-12-19T17:57:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e1296e1c0 0x555e129a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e12974950 0x555e129fd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e129b8070 0x555e129fdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e12789a80 con 0x555e1296e1c0 2023-12-19T17:57:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e127bd670 con 0x555e12974950 2023-12-19T17:57:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e12764860 con 0x555e129b8070 2023-12-19T17:57:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 10 client.0 _ll_register_callbacks cb 0x555e1276a640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T17:57:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae49f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e1296e1c0 0x555e129a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae49f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e1296e1c0 0x555e129a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57780/0 (socket says 172.21.15.26:57780) 2023-12-19T17:57:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae49f1640 1 -- 172.21.15.26:0/2599509500 learned_addr learned my addr 172.21.15.26:0/2599509500 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae51f2640 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e129b8070 0x555e129fdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcadffff640 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e12974950 0x555e129fd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae49f1640 1 -- 172.21.15.26:0/2599509500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e129b8070 msgr2=0x555e129fdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae49f1640 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e129b8070 0x555e129fdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae49f1640 1 -- 172.21.15.26:0/2599509500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e12974950 msgr2=0x555e129fd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae49f1640 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e12974950 0x555e129fd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae49f1640 1 -- 172.21.15.26:0/2599509500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad0003530 con 0x555e1296e1c0 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae51f2640 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e129b8070 0x555e129fdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae49f1640 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e1296e1c0 0x555e129a5a90 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fcad40ca040 tx=0x7fcad4019990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df6456d4eb8d331 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcabf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcabf7fe640 1 -- 172.21.15.26:0/2599509500 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad40bf070 con 0x555e1296e1c0 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcabf7fe640 1 -- 172.21.15.26:0/2599509500 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fcad40c7250 con 0x555e1296e1c0 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 1 -- 172.21.15.26:0/2599509500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e12781e00 con 0x555e1296e1c0 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcabf7fe640 1 -- 172.21.15.26:0/2599509500 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcad4001650 con 0x555e1296e1c0 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 10 client.9153 fetch_fsmap learned FSMap version 73 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 1 -- 172.21.15.26:0/2599509500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x555e124b8cc0 con 0x555e1296e1c0 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcabf7fe640 1 -- 172.21.15.26:0/2599509500 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555e12781e00 con 0x555e1296e1c0 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcabf7fe640 1 -- 172.21.15.26:0/2599509500 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 73) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcad40ce070 con 0x555e1296e1c0 2023-12-19T17:57:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 10 client.9153 fetch_fsmap finished waiting for FSMap version 73 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 10 client.9153 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 20 client.9153 populate_metadata read hostname 'smithi026' 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.26:6832/2838422258,v1:172.21.15.26:6834/2838422258] conn(0x555e129b8990 0x555e129fe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 4 client.9153 mds_command: new command op to 8575 tid=0[{"prefix": "status"}] 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcae9fb6000 1 -- 172.21.15.26:0/2599509500 --> [v2:172.21.15.26:6832/2838422258,v1:172.21.15.26:6834/2838422258] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555e127b5160 con 0x555e129b8990 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcadffff640 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.26:6832/2838422258,v1:172.21.15.26:6834/2838422258] conn(0x555e129b8990 0x555e129fe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcadffff640 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.26:6832/2838422258,v1:172.21.15.26:6834/2838422258] conn(0x555e129b8990 0x555e129fe450 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f528ca533d57043f server_cookie=de9175fdc5708a36 in_seq=0 out_seq=0 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.459+0000 7fcabf7fe640 10 client.9153 ms_handle_connect on v2:172.21.15.26:6832/2838422258 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcabf7fe640 1 -- 172.21.15.26:0/2599509500 <== mds.0 v2:172.21.15.26:6832/2838422258 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x555e127b5160 con 0x555e129b8990 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcabf7fe640 10 client.9153 handle_command_reply: tid=0 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae9fb6000 1 client.9153 shutdown 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae9fb6000 1 -- 172.21.15.26:0/2599509500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e1296e1c0 msgr2=0x555e129a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e1296e1c0 0x555e129a5a90 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fcad40ca040 tx=0x7fcad4019990 comp rx=0 tx=0).stop 2023-12-19T17:57:46.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae9fb6000 1 -- 172.21.15.26:0/2599509500 shutdown_connections 2023-12-19T17:57:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.26:6832/2838422258,v1:172.21.15.26:6834/2838422258] conn(0x555e129b8990 0x555e129fe450 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x555e129b8070 0x555e129fdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555e12974950 0x555e129fd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae9fb6000 1 --2- 172.21.15.26:0/2599509500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x555e1296e1c0 0x555e129a5a90 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcabf7fe640 0 client.9153 ms_handle_reset on v2:172.21.15.26:6832/2838422258 2023-12-19T17:57:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae9fb6000 1 -- 172.21.15.26:0/2599509500 >> 172.21.15.26:0/2599509500 conn(0x555e129b77f0 msgr2=0x555e129cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae9fb6000 1 -- 172.21.15.26:0/2599509500 shutdown_connections 2023-12-19T17:57:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae9fb6000 1 -- 172.21.15.26:0/2599509500 wait complete. 2023-12-19T17:57:46.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae9fb6000 20 client.9153 trim_cache size 0 max 16384 2023-12-19T17:57:46.470 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T17:57:46.470 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T17:57:46.470 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T17:57:46.470 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 8575, 2023-12-19T17:57:46.470 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T17:57:46.470 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T17:57:46.470 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T17:57:46.470 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.9426245250000003, 2023-12-19T17:57:46.470 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 73, 2023-12-19T17:57:46.470 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 163, 2023-12-19T17:57:46.471 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 163, 2023-12-19T17:57:46.471 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 11.320455406000001 2023-12-19T17:57:46.471 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T17:57:46.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae6c55640 1 -- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcac4065b80 msgr2=0x7fcac4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae6c55640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcac4065b80 0x7fcac4068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcad40ab000 tx=0x7fcad40a7070 comp rx=0 tx=0).stop 2023-12-19T17:57:46.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae6c55640 1 -- 172.21.15.26:0/4104591147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae01f6af0 msgr2=0x7fcae01f2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae6c55640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae01f6af0 0x7fcae01f2730 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009fd0 tx=0x7fcacc011b30 comp rx=0 tx=0).stop 2023-12-19T17:57:46.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae6c55640 1 -- 172.21.15.26:0/4104591147 shutdown_connections 2023-12-19T17:57:46.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae6c55640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae01f6f70 0x7fcae01f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae6c55640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcae01f6af0 0x7fcae01f2730 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae6c55640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcac4065b80 0x7fcac4068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae6c55640 1 --2- 172.21.15.26:0/4104591147 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcad00039b0 0x7fcae015c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae6c55640 1 -- 172.21.15.26:0/4104591147 >> 172.21.15.26:0/4104591147 conn(0x7fcae00658e0 msgr2=0x7fcae0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:46.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae6c55640 1 -- 172.21.15.26:0/4104591147 shutdown_connections 2023-12-19T17:57:46.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.463+0000 7fcae6c55640 1 -- 172.21.15.26:0/4104591147 wait complete. 2023-12-19T17:57:46.484 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T17:57:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f888dcc4640 1 Processor -- start 2023-12-19T17:57:46.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f888dcc4640 1 -- start start 2023-12-19T17:57:46.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f888dcc4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f888815cd40 0x7f88880ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f888dcc4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8888160900 0x7f8888160d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f888dcc4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88880d29d0 con 0x7f88880d2bf0 2023-12-19T17:57:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f888dcc4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8888161280 con 0x7f8888160900 2023-12-19T17:57:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f888dcc4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888815d680 con 0x7f888815cd40 2023-12-19T17:57:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f8886ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8888160900 0x7f8888160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f8886ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8888160900 0x7f8888160d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57796/0 (socket says 172.21.15.26:57796) 2023-12-19T17:57:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f8886ffd640 1 -- 172.21.15.26:0/2410693554 learned_addr learned my addr 172.21.15.26:0/2410693554 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f88877fe640 1 --2- 172.21.15.26:0/2410693554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f888815cd40 0x7f88880ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f8886ffd640 1 -- 172.21.15.26:0/2410693554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f888815cd40 msgr2=0x7f88880ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f8886ffd640 1 --2- 172.21.15.26:0/2410693554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f888815cd40 0x7f88880ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f8886ffd640 1 -- 172.21.15.26:0/2410693554 >> v1:172.21.15.26:6789/0 conn(0x7f88880d2bf0 legacy=0x7f88880d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.627+0000 7f8886ffd640 1 -- 172.21.15.26:0/2410693554 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8888161750 con 0x7f8888160900 2023-12-19T17:57:46.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.631+0000 7f8886ffd640 1 --2- 172.21.15.26:0/2410693554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8888160900 0x7f8888160d40 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f887c009870 tx=0x7f887c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f25017ab093238 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:46.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.631+0000 7f88867fc640 1 -- 172.21.15.26:0/2410693554 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c0a7070 con 0x7f8888160900 2023-12-19T17:57:46.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.631+0000 7f88867fc640 1 -- 172.21.15.26:0/2410693554 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c0ab430 con 0x7f8888160900 2023-12-19T17:57:46.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.631+0000 7f88867fc640 1 -- 172.21.15.26:0/2410693554 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c0ab7c0 con 0x7f8888160900 2023-12-19T17:57:46.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.631+0000 7f888dcc4640 1 -- 172.21.15.26:0/2410693554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8888160900 msgr2=0x7f8888160d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.631+0000 7f888dcc4640 1 --2- 172.21.15.26:0/2410693554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8888160900 0x7f8888160d40 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f887c009870 tx=0x7f887c01c540 comp rx=0 tx=0).stop 2023-12-19T17:57:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.631+0000 7f888dcc4640 1 -- 172.21.15.26:0/2410693554 shutdown_connections 2023-12-19T17:57:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.631+0000 7f888dcc4640 1 --2- 172.21.15.26:0/2410693554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8888160900 0x7f8888160d40 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.631+0000 7f888dcc4640 1 --2- 172.21.15.26:0/2410693554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f888815cd40 0x7f88880ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.631+0000 7f888dcc4640 1 -- 172.21.15.26:0/2410693554 >> 172.21.15.26:0/2410693554 conn(0x7f888806a120 msgr2=0x7f888806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.631+0000 7f888dcc4640 1 -- 172.21.15.26:0/2410693554 shutdown_connections 2023-12-19T17:57:46.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.631+0000 7f888dcc4640 1 -- 172.21.15.26:0/2410693554 wait complete. 2023-12-19T17:57:46.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f888dcc4640 1 Processor -- start 2023-12-19T17:57:46.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f888dcc4640 1 -- start start 2023-12-19T17:57:46.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f888dcc4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88880d2bf0 0x7f8888159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f888dcc4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f888815cd40 0x7f888815d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f888dcc4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8888160900 0x7f888815ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f888dcc4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88881f6730 con 0x7f8888160900 2023-12-19T17:57:46.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f888dcc4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88881f68e0 con 0x7f88880d2bf0 2023-12-19T17:57:46.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f888dcc4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88881f6450 con 0x7f888815cd40 2023-12-19T17:57:46.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f8887fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8888160900 0x7f888815ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f8887fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8888160900 0x7f888815ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57798/0 (socket says 172.21.15.26:57798) 2023-12-19T17:57:46.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f8887fff640 1 -- 172.21.15.26:0/2683452405 learned_addr learned my addr 172.21.15.26:0/2683452405 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:46.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f8886ffd640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f888815cd40 0x7f888815d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f88877fe640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88880d2bf0 0x7f8888159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f8887fff640 1 -- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f888815cd40 msgr2=0x7f888815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f8887fff640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f888815cd40 0x7f888815d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f8887fff640 1 -- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88880d2bf0 msgr2=0x7f8888159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f8887fff640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88880d2bf0 0x7f8888159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f8887fff640 1 -- 172.21.15.26:0/2683452405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c0094d0 con 0x7f8888160900 2023-12-19T17:57:46.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f8887fff640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8888160900 0x7f888815ddc0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f888000c770 tx=0x7f888000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:46.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f886ffff640 1 -- 172.21.15.26:0/2683452405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880003b80 con 0x7f8888160900 2023-12-19T17:57:46.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f886ffff640 1 -- 172.21.15.26:0/2683452405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8880003ce0 con 0x7f8888160900 2023-12-19T17:57:46.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f886ffff640 1 -- 172.21.15.26:0/2683452405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888000aa90 con 0x7f8888160900 2023-12-19T17:57:46.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f888dcc4640 1 -- 172.21.15.26:0/2683452405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8888143bf0 con 0x7f8888160900 2023-12-19T17:57:46.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.635+0000 7f888dcc4640 1 -- 172.21.15.26:0/2683452405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88881441b0 con 0x7f8888160900 2023-12-19T17:57:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.639+0000 7f888dcc4640 1 -- 172.21.15.26:0/2683452405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8888143a50 con 0x7f8888160900 2023-12-19T17:57:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.639+0000 7f886ffff640 1 -- 172.21.15.26:0/2683452405 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f888000abf0 con 0x7f8888160900 2023-12-19T17:57:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.639+0000 7f886ffff640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8874065b80 0x7f8874068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.639+0000 7f886ffff640 1 -- 172.21.15.26:0/2683452405 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f8880083150 con 0x7f8888160900 2023-12-19T17:57:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.639+0000 7f88877fe640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8874065b80 0x7f8874068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.639+0000 7f88877fe640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8874065b80 0x7f8874068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f88881f6890 tx=0x7f8878006f90 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.643+0000 7f886ffff640 1 -- 172.21.15.26:0/2683452405 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8888143a50 con 0x7f8888160900 2023-12-19T17:57:46.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.851+0000 7f888dcc4640 1 -- 172.21.15.26:0/2683452405 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8888143a50 con 0x7f8888160900 2023-12-19T17:57:46.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.851+0000 7f886ffff640 1 -- 172.21.15.26:0/2683452405 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8888143a50 con 0x7f8888160900 2023-12-19T17:57:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T17:57:46.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.851+0000 7f888dcc4640 1 -- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8874065b80 msgr2=0x7f8874068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.851+0000 7f888dcc4640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8874065b80 0x7f8874068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f88881f6890 tx=0x7f8878006f90 comp rx=0 tx=0).stop 2023-12-19T17:57:46.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.851+0000 7f888dcc4640 1 -- 172.21.15.26:0/2683452405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8888160900 msgr2=0x7f888815ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:46.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.851+0000 7f888dcc4640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8888160900 0x7f888815ddc0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f888000c770 tx=0x7f888000cc30 comp rx=0 tx=0).stop 2023-12-19T17:57:46.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.855+0000 7f888dcc4640 1 -- 172.21.15.26:0/2683452405 shutdown_connections 2023-12-19T17:57:46.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.855+0000 7f888dcc4640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8888160900 0x7f888815ddc0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.855+0000 7f888dcc4640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8874065b80 0x7f8874068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.855+0000 7f888dcc4640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f888815cd40 0x7f888815d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.855+0000 7f888dcc4640 1 --2- 172.21.15.26:0/2683452405 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f88880d2bf0 0x7f8888159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:46.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.855+0000 7f888dcc4640 1 -- 172.21.15.26:0/2683452405 >> 172.21.15.26:0/2683452405 conn(0x7f888806a120 msgr2=0x7f888815ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:46.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.855+0000 7f888dcc4640 1 -- 172.21.15.26:0/2683452405 shutdown_connections 2023-12-19T17:57:46.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:46.855+0000 7f888dcc4640 1 -- 172.21.15.26:0/2683452405 wait complete. 2023-12-19T17:57:46.875 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-19T17:57:46.875 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T17:57:46.875 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T17:57:46.875 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T17:57:46.875 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T17:57:46.875 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-19T17:57:46.875 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-19T17:57:46.875 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T17:57:46.875 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T17:57:46.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:46.876 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T17:57:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T17:57:46.886 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T17:57:46.886 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T17:57:46.887 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:57:46.887 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T17:57:46.887 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:57:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T17:57:46.887 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T17:57:46.887 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T17:57:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T17:57:46.887 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T17:57:46.887 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T17:57:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T17:57:46.887 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T17:57:46.887 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T17:57:46.887 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 37874sec preferred_lft 37874sec 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T17:57:46.888 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:57:46.889 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-19T17:57:46.889 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:57:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T17:57:46.889 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T17:57:46.889 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-19T17:57:46.889 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:57:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T17:57:46.889 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T17:57:46.889 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-19T17:57:46.889 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:57:46.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:46.890 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T17:57:46.947 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T17:57:46.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:46.949 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T17:57:47.002 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T17:57:47.002 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T17:57:47.002 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T17:57:47.002 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T17:57:47.002 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T17:57:47.002 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T17:57:47.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:47.003 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T17:57:47.003 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T17:57:47.003 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T17:57:47.003 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T17:57:47.003 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T17:57:47.003 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T17:57:47.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:47.100 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T17:57:47.110 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T17:57:47.110 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T17:57:47.110 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T17:57:47.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:47.111 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T17:57:47.176 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T17:57:47.177 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T17:57:47.232 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T17:57:47.232 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T17:57:47.233 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T17:57:47.287 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T17:57:47.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:47.288 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T17:57:47.387 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T17:57:47.387 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T17:57:47.401 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T17:57:47.401 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T17:57:47.401 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T17:57:47.401 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T17:57:47.401 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T17:57:47.405 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T17:57:47.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:47.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T17:57:47.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:47.426 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T17:57:47.489 INFO:teuthology.orchestra.run.smithi026.stdout:client7730 2023-12-19T17:57:47.490 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:57:47.490 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:47.545 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-19T17:57:47.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46fe666640 1 Processor -- start 2023-12-19T17:57:47.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46fe666640 1 -- start start 2023-12-19T17:57:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46fe666640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46f806b3d0 0x7f46f80ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46fe666640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f46f80cec40 0x7f46f815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46fe666640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f806bc30 con 0x7f46f8154220 2023-12-19T17:57:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46fe666640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f80cf0d0 con 0x7f46f806b3d0 2023-12-19T17:57:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46fe666640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f80cc6e0 con 0x7f46f80cec40 2023-12-19T17:57:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46f77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46f806b3d0 0x7f46f80ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46f77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46f806b3d0 0x7f46f80ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57808/0 (socket says 172.21.15.26:57808) 2023-12-19T17:57:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46f77fe640 1 -- 172.21.15.26:0/1892154238 learned_addr learned my addr 172.21.15.26:0/1892154238 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46fcc03640 1 --2- 172.21.15.26:0/1892154238 >> v2:172.21.15.160:3300/0 conn(0x7f46f80cec40 0x7f46f815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46f77fe640 1 -- 172.21.15.26:0/1892154238 >> v2:172.21.15.160:3300/0 conn(0x7f46f80cec40 msgr2=0x7f46f815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46f77fe640 1 --2- 172.21.15.26:0/1892154238 >> v2:172.21.15.160:3300/0 conn(0x7f46f80cec40 0x7f46f815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46f77fe640 1 -- 172.21.15.26:0/1892154238 >> v1:172.21.15.160:6789/0 conn(0x7f46f8154220 legacy=0x7f46f815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.735+0000 7f46f77fe640 1 -- 172.21.15.26:0/1892154238 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f806b950 con 0x7f46f806b3d0 2023-12-19T17:57:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.739+0000 7f46f77fe640 1 --2- 172.21.15.26:0/1892154238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f806b3d0 0x7f46f80ce700 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f46f8157bd0 tx=0x7f46ec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5db6388b55c5646 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.739+0000 7f46f6ffd640 1 -- 172.21.15.26:0/1892154238 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0aa5f0 con 0x7f46f806b3d0 2023-12-19T17:57:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.739+0000 7f46f6ffd640 1 -- 172.21.15.26:0/1892154238 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec0aa750 con 0x7f46f806b3d0 2023-12-19T17:57:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.739+0000 7f46f6ffd640 1 -- 172.21.15.26:0/1892154238 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0aaae0 con 0x7f46f806b3d0 2023-12-19T17:57:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.739+0000 7f46fe666640 1 -- 172.21.15.26:0/1892154238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f806b3d0 msgr2=0x7f46f80ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.739+0000 7f46fe666640 1 --2- 172.21.15.26:0/1892154238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f806b3d0 0x7f46f80ce700 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f46f8157bd0 tx=0x7f46ec01c490 comp rx=0 tx=0).stop 2023-12-19T17:57:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.739+0000 7f46fe666640 1 -- 172.21.15.26:0/1892154238 shutdown_connections 2023-12-19T17:57:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.739+0000 7f46fe666640 1 --2- 172.21.15.26:0/1892154238 >> v2:172.21.15.160:3300/0 conn(0x7f46f80cec40 0x7f46f815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.739+0000 7f46fe666640 1 --2- 172.21.15.26:0/1892154238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f806b3d0 0x7f46f80ce700 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.739+0000 7f46fe666640 1 -- 172.21.15.26:0/1892154238 >> 172.21.15.26:0/1892154238 conn(0x7f46f80658e0 msgr2=0x7f46f8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.739+0000 7f46fe666640 1 -- 172.21.15.26:0/1892154238 shutdown_connections 2023-12-19T17:57:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.739+0000 7f46fe666640 1 -- 172.21.15.26:0/1892154238 wait complete. 2023-12-19T17:57:47.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.739+0000 7f46fe666640 1 Processor -- start 2023-12-19T17:57:47.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fe666640 1 -- start start 2023-12-19T17:57:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fe666640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46f806b3d0 0x7f46f8148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fe666640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46f80cec40 0x7f46f8148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fe666640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f8154220 0x7f46f80b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fe666640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f8148860 con 0x7f46f8154220 2023-12-19T17:57:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fe666640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f8149130 con 0x7f46f80cec40 2023-12-19T17:57:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fe666640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f8149410 con 0x7f46f806b3d0 2023-12-19T17:57:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fcc03640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f8154220 0x7f46f80b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fcc03640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f8154220 0x7f46f80b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57820/0 (socket says 172.21.15.26:57820) 2023-12-19T17:57:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fcc03640 1 -- 172.21.15.26:0/767618748 learned_addr learned my addr 172.21.15.26:0/767618748 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46f7fff640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46f806b3d0 0x7f46f8148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46f77fe640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46f80cec40 0x7f46f8148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fcc03640 1 -- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46f806b3d0 msgr2=0x7f46f8148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fcc03640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46f806b3d0 0x7f46f8148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fcc03640 1 -- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46f80cec40 msgr2=0x7f46f8148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fcc03640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46f80cec40 0x7f46f8148320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fcc03640 1 -- 172.21.15.26:0/767618748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec0094d0 con 0x7f46f8154220 2023-12-19T17:57:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fcc03640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f8154220 0x7f46f80b2660 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f46f0007e40 tx=0x7f46f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46f4ff9640 1 -- 172.21.15.26:0/767618748 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0014620 con 0x7f46f8154220 2023-12-19T17:57:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46f4ff9640 1 -- 172.21.15.26:0/767618748 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f000ee90 con 0x7f46f8154220 2023-12-19T17:57:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fe666640 1 -- 172.21.15.26:0/767618748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f81496f0 con 0x7f46f8154220 2023-12-19T17:57:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46f4ff9640 1 -- 172.21.15.26:0/767618748 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00149e0 con 0x7f46f8154220 2023-12-19T17:57:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fe666640 1 -- 172.21.15.26:0/767618748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f8149cb0 con 0x7f46f8154220 2023-12-19T17:57:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.743+0000 7f46fe666640 1 -- 172.21.15.26:0/767618748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f8074270 con 0x7f46f8154220 2023-12-19T17:57:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.747+0000 7f46f4ff9640 1 -- 172.21.15.26:0/767618748 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f46f0014b40 con 0x7f46f8154220 2023-12-19T17:57:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.747+0000 7f46f4ff9640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f46e4065b80 0x7f46e4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.747+0000 7f46f4ff9640 1 -- 172.21.15.26:0/767618748 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f46f0083e70 con 0x7f46f8154220 2023-12-19T17:57:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.747+0000 7f46f7fff640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f46e4065b80 0x7f46e4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:47.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.747+0000 7f46f7fff640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f46e4065b80 0x7f46e4068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f46e8007fd0 tx=0x7f46e8007e40 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:47.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.751+0000 7f46f4ff9640 1 -- 172.21.15.26:0/767618748 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f46f00514e0 con 0x7f46f8154220 2023-12-19T17:57:47.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.927+0000 7f46fe666640 1 -- 172.21.15.26:0/767618748 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f46f80ccef0 con 0x7f46e4065b80 2023-12-19T17:57:47.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.931+0000 7f46f4ff9640 1 -- 172.21.15.26:0/767618748 <== mgr.8359 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f46e8004c00 con 0x7f46e4065b80 2023-12-19T17:57:47.932 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T17:57:47.932 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:57:47.932 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2023-12-19T17:57:47.932 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:57:47.932 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T17:57:47.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.931+0000 7f46fe666640 1 -- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f46e4065b80 msgr2=0x7f46e4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:47.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.931+0000 7f46fe666640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f46e4065b80 0x7f46e4068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f46e8007fd0 tx=0x7f46e8007e40 comp rx=0 tx=0).stop 2023-12-19T17:57:47.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.931+0000 7f46fe666640 1 -- 172.21.15.26:0/767618748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f8154220 msgr2=0x7f46f80b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:47.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.931+0000 7f46fe666640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f8154220 0x7f46f80b2660 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f46f0007e40 tx=0x7f46f000e4b0 comp rx=0 tx=0).stop 2023-12-19T17:57:47.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.931+0000 7f46fe666640 1 -- 172.21.15.26:0/767618748 shutdown_connections 2023-12-19T17:57:47.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.931+0000 7f46fe666640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f8154220 0x7f46f80b2660 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:47.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.931+0000 7f46fe666640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46f80cec40 0x7f46f8148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:47.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.931+0000 7f46fe666640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f46e4065b80 0x7f46e4068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:47.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.931+0000 7f46fe666640 1 --2- 172.21.15.26:0/767618748 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46f806b3d0 0x7f46f8148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:47.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.931+0000 7f46fe666640 1 -- 172.21.15.26:0/767618748 >> 172.21.15.26:0/767618748 conn(0x7f46f80658e0 msgr2=0x7f46f80cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:47.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.931+0000 7f46fe666640 1 -- 172.21.15.26:0/767618748 shutdown_connections 2023-12-19T17:57:47.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:47.931+0000 7f46fe666640 1 -- 172.21.15.26:0/767618748 wait complete. 2023-12-19T17:57:47.954 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-19T17:57:48.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.099+0000 7fe1e2725640 1 Processor -- start 2023-12-19T17:57:48.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.099+0000 7fe1e2725640 1 -- start start 2023-12-19T17:57:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.099+0000 7fe1e2725640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe1dc15cd10 0x7fe1dc0cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:48.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.099+0000 7fe1e2725640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1dc06d3e0 0x7fe1dc06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.099+0000 7fe1e2725640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1dc15d0f0 con 0x7fe1dc06c130 2023-12-19T17:57:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.099+0000 7fe1e2725640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc06c5a0 con 0x7fe1dc06d3e0 2023-12-19T17:57:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.099+0000 7fe1e2725640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc0c98b0 con 0x7fe1dc15cd10 2023-12-19T17:57:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.099+0000 7fe1db7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1dc06d3e0 0x7fe1dc06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.099+0000 7fe1db7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1dc06d3e0 0x7fe1dc06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57824/0 (socket says 172.21.15.26:57824) 2023-12-19T17:57:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.099+0000 7fe1db7fe640 1 -- 172.21.15.26:0/3985580504 learned_addr learned my addr 172.21.15.26:0/3985580504 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.099+0000 7fe1dbfff640 1 --2- 172.21.15.26:0/3985580504 >> v2:172.21.15.160:3300/0 conn(0x7fe1dc15cd10 0x7fe1dc0cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1db7fe640 1 -- 172.21.15.26:0/3985580504 >> v2:172.21.15.160:3300/0 conn(0x7fe1dc15cd10 msgr2=0x7fe1dc0cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1db7fe640 1 --2- 172.21.15.26:0/3985580504 >> v2:172.21.15.160:3300/0 conn(0x7fe1dc15cd10 0x7fe1dc0cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1db7fe640 1 -- 172.21.15.26:0/3985580504 >> v1:172.21.15.160:6789/0 conn(0x7fe1dc06c130 legacy=0x7fe1dc160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:48.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1db7fe640 1 -- 172.21.15.26:0/3985580504 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0cd0e0 con 0x7fe1dc06d3e0 2023-12-19T17:57:48.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1db7fe640 1 --2- 172.21.15.26:0/3985580504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc06d3e0 0x7fe1dc06bbf0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0cff10 tx=0x7fe1d001ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3462547b75a8573b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:48.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1daffd640 1 -- 172.21.15.26:0/3985580504 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00a89b0 con 0x7fe1dc06d3e0 2023-12-19T17:57:48.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1daffd640 1 -- 172.21.15.26:0/3985580504 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d00a8b10 con 0x7fe1dc06d3e0 2023-12-19T17:57:48.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1daffd640 1 -- 172.21.15.26:0/3985580504 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00a8ea0 con 0x7fe1dc06d3e0 2023-12-19T17:57:48.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1e2725640 1 -- 172.21.15.26:0/3985580504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc06d3e0 msgr2=0x7fe1dc06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:48.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1e2725640 1 --2- 172.21.15.26:0/3985580504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc06d3e0 0x7fe1dc06bbf0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0cff10 tx=0x7fe1d001ecb0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1e2725640 1 -- 172.21.15.26:0/3985580504 shutdown_connections 2023-12-19T17:57:48.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1e2725640 1 --2- 172.21.15.26:0/3985580504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc06d3e0 0x7fe1dc06bbf0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1e2725640 1 --2- 172.21.15.26:0/3985580504 >> v2:172.21.15.160:3300/0 conn(0x7fe1dc15cd10 0x7fe1dc0cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1e2725640 1 -- 172.21.15.26:0/3985580504 >> 172.21.15.26:0/3985580504 conn(0x7fe1dc0cf260 msgr2=0x7fe1dc0cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:48.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1e2725640 1 -- 172.21.15.26:0/3985580504 shutdown_connections 2023-12-19T17:57:48.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.103+0000 7fe1e2725640 1 -- 172.21.15.26:0/3985580504 wait complete. 2023-12-19T17:57:48.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e2725640 1 Processor -- start 2023-12-19T17:57:48.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e2725640 1 -- start start 2023-12-19T17:57:48.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e2725640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1dc06c130 0x7fe1dc155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:48.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e2725640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1dc06d3e0 0x7fe1dc157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:48.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e2725640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc15cd10 0x7fe1dc158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:48.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e2725640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0b1ac0 con 0x7fe1dc15cd10 2023-12-19T17:57:48.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e2725640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0aedf0 con 0x7fe1dc06c130 2023-12-19T17:57:48.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e2725640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0af0d0 con 0x7fe1dc06d3e0 2023-12-19T17:57:48.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e0cc2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc15cd10 0x7fe1dc158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:48.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e0cc2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc15cd10 0x7fe1dc158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57832/0 (socket says 172.21.15.26:57832) 2023-12-19T17:57:48.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e0cc2640 1 -- 172.21.15.26:0/3944372624 learned_addr learned my addr 172.21.15.26:0/3944372624 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:48.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1db7fe640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1dc06d3e0 0x7fe1dc157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:48.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1dbfff640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1dc06c130 0x7fe1dc155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:48.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e0cc2640 1 -- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1dc06d3e0 msgr2=0x7fe1dc157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:48.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e0cc2640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1dc06d3e0 0x7fe1dc157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e0cc2640 1 -- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1dc06c130 msgr2=0x7fe1dc155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:48.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e0cc2640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1dc06c130 0x7fe1dc155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e0cc2640 1 -- 172.21.15.26:0/3944372624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d00094d0 con 0x7fe1dc15cd10 2023-12-19T17:57:48.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1dbfff640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1dc06c130 0x7fe1dc155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:48.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.107+0000 7fe1e0cc2640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc15cd10 0x7fe1dc158040 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400b970 tx=0x7fe1d400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:48.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.111+0000 7fe1d8ff9640 1 -- 172.21.15.26:0/3944372624 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d400cb40 con 0x7fe1dc15cd10 2023-12-19T17:57:48.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.111+0000 7fe1d8ff9640 1 -- 172.21.15.26:0/3944372624 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d400cca0 con 0x7fe1dc15cd10 2023-12-19T17:57:48.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.111+0000 7fe1d8ff9640 1 -- 172.21.15.26:0/3944372624 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4014680 con 0x7fe1dc15cd10 2023-12-19T17:57:48.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.111+0000 7fe1e2725640 1 -- 172.21.15.26:0/3944372624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc0af3b0 con 0x7fe1dc15cd10 2023-12-19T17:57:48.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.111+0000 7fe1e2725640 1 -- 172.21.15.26:0/3944372624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc1557b0 con 0x7fe1dc15cd10 2023-12-19T17:57:48.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.111+0000 7fe1e2725640 1 -- 172.21.15.26:0/3944372624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc074270 con 0x7fe1dc15cd10 2023-12-19T17:57:48.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.111+0000 7fe1d8ff9640 1 -- 172.21.15.26:0/3944372624 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 80) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe1d401d3b0 con 0x7fe1dc15cd10 2023-12-19T17:57:48.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.111+0000 7fe1d8ff9640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe1c8065b80 0x7fe1c8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:48.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.111+0000 7fe1d8ff9640 1 -- 172.21.15.26:0/3944372624 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fe1d4083350 con 0x7fe1dc15cd10 2023-12-19T17:57:48.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.111+0000 7fe1dbfff640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe1c8065b80 0x7fe1c8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:48.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.115+0000 7fe1dbfff640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe1c8065b80 0x7fe1c8068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc1f59f0 tx=0x7fe1cc008040 comp rx=0 tx=0).ready entity=mgr.8359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:48.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.115+0000 7fe1d8ff9640 1 -- 172.21.15.26:0/3944372624 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe1d40509c0 con 0x7fe1dc15cd10 2023-12-19T17:57:48.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.315+0000 7fe1e2725640 1 -- 172.21.15.26:0/3944372624 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fe1dc0af860 con 0x7fe1dc15cd10 2023-12-19T17:57:48.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.567+0000 7fe1d8ff9640 1 -- 172.21.15.26:0/3944372624 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 81) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fe1d4050d90 con 0x7fe1dc15cd10 2023-12-19T17:57:48.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.567+0000 7fe1d8ff9640 1 -- 172.21.15.26:0/3944372624 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v81) v1 ==== 92+0+0 (secure 0 0 0) 0x7fe1d406cdc0 con 0x7fe1dc15cd10 2023-12-19T17:57:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.571+0000 7fe1e2725640 1 -- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe1c8065b80 msgr2=0x7fe1c8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.571+0000 7fe1e2725640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe1c8065b80 0x7fe1c8068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc1f59f0 tx=0x7fe1cc008040 comp rx=0 tx=0).stop 2023-12-19T17:57:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.571+0000 7fe1e2725640 1 -- 172.21.15.26:0/3944372624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc15cd10 msgr2=0x7fe1dc158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.571+0000 7fe1e2725640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc15cd10 0x7fe1dc158040 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400b970 tx=0x7fe1d400be30 comp rx=0 tx=0).stop 2023-12-19T17:57:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.571+0000 7fe1e2725640 1 -- 172.21.15.26:0/3944372624 shutdown_connections 2023-12-19T17:57:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.571+0000 7fe1e2725640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc15cd10 0x7fe1dc158040 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.571+0000 7fe1e2725640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe1c8065b80 0x7fe1c8068040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.571+0000 7fe1e2725640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1dc06d3e0 0x7fe1dc157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.571+0000 7fe1e2725640 1 --2- 172.21.15.26:0/3944372624 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1dc06c130 0x7fe1dc155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.571+0000 7fe1e2725640 1 -- 172.21.15.26:0/3944372624 >> 172.21.15.26:0/3944372624 conn(0x7fe1dc0cf260 msgr2=0x7fe1dc0ae1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:48.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.571+0000 7fe1e2725640 1 -- 172.21.15.26:0/3944372624 shutdown_connections 2023-12-19T17:57:48.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.571+0000 7fe1e2725640 1 -- 172.21.15.26:0/3944372624 wait complete. 2023-12-19T17:57:48.603 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-19T17:57:48.751 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:48.746+0000 7f82c4249180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:57:48.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.751+0000 7f7123bce640 1 Processor -- start 2023-12-19T17:57:48.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.751+0000 7f7123bce640 1 -- start start 2023-12-19T17:57:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.751+0000 7f7123bce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f711c15a9c0 0x7f711c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.751+0000 7f7123bce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f711c06ac80 0x7f711c0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.751+0000 7f7123bce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711c070d90 con 0x7f711c15ada0 2023-12-19T17:57:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.751+0000 7f7123bce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711c0cc270 con 0x7f711c15a9c0 2023-12-19T17:57:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.751+0000 7f7123bce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c071070 con 0x7f711c06ac80 2023-12-19T17:57:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.751+0000 7f712196a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f711c15a9c0 0x7f711c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.751+0000 7f712196a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f711c15a9c0 0x7f711c0706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57860/0 (socket says 172.21.15.26:57860) 2023-12-19T17:57:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.751+0000 7f712196a640 1 -- 172.21.15.26:0/2697952472 learned_addr learned my addr 172.21.15.26:0/2697952472 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.751+0000 7f712216b640 1 --2- 172.21.15.26:0/2697952472 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f711c06ac80 0x7f711c0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.755+0000 7f712196a640 1 -- 172.21.15.26:0/2697952472 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f711c06ac80 msgr2=0x7f711c0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.755+0000 7f712196a640 1 --2- 172.21.15.26:0/2697952472 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f711c06ac80 0x7f711c0b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.755+0000 7f712196a640 1 -- 172.21.15.26:0/2697952472 >> v1:172.21.15.160:6789/0 conn(0x7f711c15ada0 legacy=0x7f711c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.755+0000 7f712196a640 1 -- 172.21.15.26:0/2697952472 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c071350 con 0x7f711c15a9c0 2023-12-19T17:57:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.755+0000 7f7120968640 1 -- 172.21.15.26:0/2697952472 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142610752 0 0) 0x7f711c070d90 con 0x7f711c15ada0 2023-12-19T17:57:48.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.755+0000 7f712196a640 1 --2- 172.21.15.26:0/2697952472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c15a9c0 0x7f711c0706d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7118009870 tx=0x7f711801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14939017f56445b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:48.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.755+0000 7f7120968640 1 -- 172.21.15.26:0/2697952472 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71180ad640 con 0x7f711c15a9c0 2023-12-19T17:57:48.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.755+0000 7f7120968640 1 -- 172.21.15.26:0/2697952472 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71180ad7a0 con 0x7f711c15a9c0 2023-12-19T17:57:48.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.755+0000 7f7120968640 1 -- 172.21.15.26:0/2697952472 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71180adb30 con 0x7f711c15a9c0 2023-12-19T17:57:48.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 -- 172.21.15.26:0/2697952472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c15a9c0 msgr2=0x7f711c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 --2- 172.21.15.26:0/2697952472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c15a9c0 0x7f711c0706d0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f7118009870 tx=0x7f711801c480 comp rx=0 tx=0).stop 2023-12-19T17:57:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 -- 172.21.15.26:0/2697952472 shutdown_connections 2023-12-19T17:57:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 --2- 172.21.15.26:0/2697952472 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f711c06ac80 0x7f711c0b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 --2- 172.21.15.26:0/2697952472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c15a9c0 0x7f711c0706d0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 -- 172.21.15.26:0/2697952472 >> 172.21.15.26:0/2697952472 conn(0x7f711c0ce730 msgr2=0x7f711c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 -- 172.21.15.26:0/2697952472 shutdown_connections 2023-12-19T17:57:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 -- 172.21.15.26:0/2697952472 wait complete. 2023-12-19T17:57:48.760 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:48.759+0000 7ff1fd10b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:57:48.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 Processor -- start 2023-12-19T17:57:48.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 -- start start 2023-12-19T17:57:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f711c06ac80 0x7f711c155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c15a9c0 0x7f711c152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f711c15ada0 0x7f711c152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c15a2b0 con 0x7f711c15a9c0 2023-12-19T17:57:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c1f5570 con 0x7f711c15ada0 2023-12-19T17:57:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7123bce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c152dc0 con 0x7f711c06ac80 2023-12-19T17:57:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7121169640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c15a9c0 0x7f711c152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7121169640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c15a9c0 0x7f711c152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57864/0 (socket says 172.21.15.26:57864) 2023-12-19T17:57:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f7121169640 1 -- 172.21.15.26:0/3448065019 learned_addr learned my addr 172.21.15.26:0/3448065019 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f712196a640 1 --2- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f711c06ac80 0x7f711c155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:48.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.759+0000 7f712216b640 1 --2- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f711c15ada0 0x7f711c152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f7121169640 1 -- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f711c06ac80 msgr2=0x7f711c155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f7121169640 1 --2- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f711c06ac80 0x7f711c155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f7121169640 1 -- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f711c15ada0 msgr2=0x7f711c152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f7121169640 1 --2- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f711c15ada0 0x7f711c152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f7121169640 1 -- 172.21.15.26:0/3448065019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71180094d0 con 0x7f711c15a9c0 2023-12-19T17:57:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f7121169640 1 --2- 172.21.15.26:0/3448065019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c15a9c0 0x7f711c152340 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c4d0 tx=0x7f710c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f71127fc640 1 -- 172.21.15.26:0/3448065019 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c005480 con 0x7f711c15a9c0 2023-12-19T17:57:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f71127fc640 1 -- 172.21.15.26:0/3448065019 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c003600 con 0x7f711c15a9c0 2023-12-19T17:57:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f71127fc640 1 -- 172.21.15.26:0/3448065019 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c0039b0 con 0x7f711c15a9c0 2023-12-19T17:57:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f7123bce640 1 -- 172.21.15.26:0/3448065019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711c1f5780 con 0x7f711c15a9c0 2023-12-19T17:57:48.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f7123bce640 1 -- 172.21.15.26:0/3448065019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711c1f5d40 con 0x7f711c15a9c0 2023-12-19T17:57:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f7123bce640 1 -- 172.21.15.26:0/3448065019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711c074270 con 0x7f711c15a9c0 2023-12-19T17:57:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f71127fc640 1 -- 172.21.15.26:0/3448065019 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 81) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f710c003b10 con 0x7f711c15a9c0 2023-12-19T17:57:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f71127fc640 1 --2- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7100065bd0 0x7f7100068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f71127fc640 1 -- 172.21.15.26:0/3448065019 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f710c082bb0 con 0x7f711c15a9c0 2023-12-19T17:57:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.763+0000 7f712196a640 1 -- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7100065bd0 msgr2=0x7f7100068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.767+0000 7f712196a640 1 --2- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7100065bd0 0x7f7100068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:48.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.767+0000 7f71127fc640 1 -- 172.21.15.26:0/3448065019 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f710c0502f0 con 0x7f711c15a9c0 2023-12-19T17:57:48.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.927+0000 7f7123bce640 1 -- 172.21.15.26:0/3448065019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f711c0cd7b0 con 0x7f711c15a9c0 2023-12-19T17:57:48.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.935+0000 7f71127fc640 1 -- 172.21.15.26:0/3448065019 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7f710c017070 con 0x7f711c15a9c0 2023-12-19T17:57:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.935+0000 7f7123bce640 1 -- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7100065bd0 msgr2=0x7f7100068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.935+0000 7f7123bce640 1 --2- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7100065bd0 0x7f7100068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.935+0000 7f7123bce640 1 -- 172.21.15.26:0/3448065019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c15a9c0 msgr2=0x7f711c152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.935+0000 7f7123bce640 1 --2- 172.21.15.26:0/3448065019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c15a9c0 0x7f711c152340 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c4d0 tx=0x7f710c00c990 comp rx=0 tx=0).stop 2023-12-19T17:57:48.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.935+0000 7f7123bce640 1 -- 172.21.15.26:0/3448065019 shutdown_connections 2023-12-19T17:57:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.935+0000 7f7123bce640 1 --2- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f711c15ada0 0x7f711c152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.935+0000 7f7123bce640 1 --2- 172.21.15.26:0/3448065019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711c15a9c0 0x7f711c152340 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.935+0000 7f7123bce640 1 --2- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f7100065bd0 0x7f7100068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.935+0000 7f7123bce640 1 --2- 172.21.15.26:0/3448065019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f711c06ac80 0x7f711c155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.935+0000 7f7123bce640 1 -- 172.21.15.26:0/3448065019 >> 172.21.15.26:0/3448065019 conn(0x7f711c0ce730 msgr2=0x7f711c0ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.935+0000 7f7123bce640 1 -- 172.21.15.26:0/3448065019 shutdown_connections 2023-12-19T17:57:48.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:48.935+0000 7f7123bce640 1 -- 172.21.15.26:0/3448065019 wait complete. 2023-12-19T17:57:48.958 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-19T17:57:49.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.103+0000 7f55158b6640 1 Processor -- start 2023-12-19T17:57:49.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.103+0000 7f55158b6640 1 -- start start 2023-12-19T17:57:49.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.103+0000 7f55158b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5510159140 0x7f55100ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:49.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.103+0000 7f55158b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551006f490 con 0x7f55100cd190 2023-12-19T17:57:49.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.103+0000 7f55158b6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5510159520 con 0x7f551006b390 2023-12-19T17:57:49.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.103+0000 7f55158b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55100cd690 con 0x7f5510159140 2023-12-19T17:57:49.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f5507fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f55100cd190 0x7f551006ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59578/0 (socket says 172.21.15.26:59578) 2023-12-19T17:57:49.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f5507fff640 1 -- 172.21.15.26:0/745717980 learned_addr learned my addr 172.21.15.26:0/745717980 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:49.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f550effd640 1 --2- 172.21.15.26:0/745717980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5510159140 0x7f55100ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f550e7fc640 1 -- 172.21.15.26:0/745717980 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853359696 0 0) 0x7f551006f490 con 0x7f55100cd190 2023-12-19T17:57:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f550e7fc640 1 -- 172.21.15.26:0/745717980 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54fc003530 con 0x7f55100cd190 2023-12-19T17:57:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f550e7fc640 1 -- 172.21.15.26:0/745717980 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2176521913 0 0) 0x7f5510159520 con 0x7f551006b390 2023-12-19T17:57:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f550e7fc640 1 -- 172.21.15.26:0/745717980 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f551006f490 con 0x7f551006b390 2023-12-19T17:57:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f550effd640 1 -- 172.21.15.26:0/745717980 >> v1:172.21.15.160:6789/0 conn(0x7f551006b390 legacy=0x7f551006f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f550effd640 1 -- 172.21.15.26:0/745717980 >> v1:172.21.15.26:6789/0 conn(0x7f55100cd190 legacy=0x7f551006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:49.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f550effd640 1 -- 172.21.15.26:0/745717980 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55100cd970 con 0x7f5510159140 2023-12-19T17:57:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f550effd640 1 --2- 172.21.15.26:0/745717980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5510159140 0x7f55100ccc50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f55000097f0 tx=0x7f550001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10d7ef0b5dbe4ec1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f550e7fc640 1 -- 172.21.15.26:0/745717980 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55000ab610 con 0x7f5510159140 2023-12-19T17:57:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f550e7fc640 1 -- 172.21.15.26:0/745717980 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55000ab770 con 0x7f5510159140 2023-12-19T17:57:49.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.107+0000 7f550e7fc640 1 -- 172.21.15.26:0/745717980 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55000abb00 con 0x7f5510159140 2023-12-19T17:57:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 -- 172.21.15.26:0/745717980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5510159140 msgr2=0x7f55100ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 --2- 172.21.15.26:0/745717980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5510159140 0x7f55100ccc50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f55000097f0 tx=0x7f550001c490 comp rx=0 tx=0).stop 2023-12-19T17:57:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 -- 172.21.15.26:0/745717980 shutdown_connections 2023-12-19T17:57:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 --2- 172.21.15.26:0/745717980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5510159140 0x7f55100ccc50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 -- 172.21.15.26:0/745717980 >> 172.21.15.26:0/745717980 conn(0x7f55100ce730 msgr2=0x7f55100ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 -- 172.21.15.26:0/745717980 shutdown_connections 2023-12-19T17:57:49.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 -- 172.21.15.26:0/745717980 wait complete. 2023-12-19T17:57:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 Processor -- start 2023-12-19T17:57:49.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 -- start start 2023-12-19T17:57:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f551006b390 0x7f55100c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55100cd190 0x7f55100c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5510159140 0x7f55100c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55101533a0 con 0x7f55100cd190 2023-12-19T17:57:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5510145ce0 con 0x7f5510159140 2023-12-19T17:57:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f55158b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55100c5170 con 0x7f551006b390 2023-12-19T17:57:49.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f5507fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55100cd190 0x7f55100c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f550effd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f551006b390 0x7f55100c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f5507fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55100cd190 0x7f55100c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57880/0 (socket says 172.21.15.26:57880) 2023-12-19T17:57:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f550f7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5510159140 0x7f55100c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f550effd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f551006b390 0x7f55100c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:44066/0 (socket says 172.21.15.26:44066) 2023-12-19T17:57:49.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.111+0000 7f5507fff640 1 -- 172.21.15.26:0/2122736323 learned_addr learned my addr 172.21.15.26:0/2122736323 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f550effd640 1 -- 172.21.15.26:0/2122736323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5510159140 msgr2=0x7f55100c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f550effd640 1 --2- 172.21.15.26:0/2122736323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5510159140 0x7f55100c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f550effd640 1 -- 172.21.15.26:0/2122736323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55100cd190 msgr2=0x7f55100c7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f550effd640 1 --2- 172.21.15.26:0/2122736323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55100cd190 0x7f55100c7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f550effd640 1 -- 172.21.15.26:0/2122736323 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5500009420 con 0x7f551006b390 2023-12-19T17:57:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f550effd640 1 --2- 172.21.15.26:0/2122736323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f551006b390 0x7f55100c4c30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f55000ac000 tx=0x7f5500019230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f550cff9640 1 -- 172.21.15.26:0/2122736323 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55000abdb0 con 0x7f551006b390 2023-12-19T17:57:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f550cff9640 1 -- 172.21.15.26:0/2122736323 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55000bf450 con 0x7f551006b390 2023-12-19T17:57:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f550cff9640 1 -- 172.21.15.26:0/2122736323 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55000bf800 con 0x7f551006b390 2023-12-19T17:57:49.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f55158b6640 1 -- 172.21.15.26:0/2122736323 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55101450d0 con 0x7f551006b390 2023-12-19T17:57:49.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f55158b6640 1 -- 172.21.15.26:0/2122736323 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5510145690 con 0x7f551006b390 2023-12-19T17:57:49.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f55158b6640 1 -- 172.21.15.26:0/2122736323 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5510074270 con 0x7f551006b390 2023-12-19T17:57:49.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f550cff9640 1 -- 172.21.15.26:0/2122736323 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 81) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f55000bf960 con 0x7f551006b390 2023-12-19T17:57:49.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f550cff9640 1 --2- 172.21.15.26:0/2122736323 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f0065c00 0x7f54f00680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:49.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f550cff9640 1 -- 172.21.15.26:0/2122736323 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f550011e210 con 0x7f551006b390 2023-12-19T17:57:49.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f5507fff640 1 -- 172.21.15.26:0/2122736323 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f0065c00 msgr2=0x7f54f00680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:49.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.115+0000 7f5507fff640 1 --2- 172.21.15.26:0/2122736323 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f0065c00 0x7f54f00680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:49.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.123+0000 7f550cff9640 1 -- 172.21.15.26:0/2122736323 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f55000eb800 con 0x7f551006b390 2023-12-19T17:57:49.170 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:49.166+0000 7f82c4249180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:57:49.209 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:49.207+0000 7ff1fd10b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:57:49.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.283+0000 7f55158b6640 1 -- 172.21.15.26:0/2122736323 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f55100711b0 con 0x7f551006b390 2023-12-19T17:57:49.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.291+0000 7f550cff9640 1 -- 172.21.15.26:0/2122736323 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7f55000eee20 con 0x7f551006b390 2023-12-19T17:57:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.295+0000 7f55158b6640 1 -- 172.21.15.26:0/2122736323 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f0065c00 msgr2=0x7f54f00680c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.295+0000 7f55158b6640 1 --2- 172.21.15.26:0/2122736323 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f0065c00 0x7f54f00680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.295+0000 7f55158b6640 1 -- 172.21.15.26:0/2122736323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f551006b390 msgr2=0x7f55100c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.295+0000 7f55158b6640 1 --2- 172.21.15.26:0/2122736323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f551006b390 0x7f55100c4c30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f55000ac000 tx=0x7f5500019230 comp rx=0 tx=0).stop 2023-12-19T17:57:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.295+0000 7f55158b6640 1 -- 172.21.15.26:0/2122736323 shutdown_connections 2023-12-19T17:57:49.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.295+0000 7f55158b6640 1 --2- 172.21.15.26:0/2122736323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5510159140 0x7f55100c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.295+0000 7f55158b6640 1 --2- 172.21.15.26:0/2122736323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55100cd190 0x7f55100c7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.295+0000 7f55158b6640 1 --2- 172.21.15.26:0/2122736323 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f54f0065c00 0x7f54f00680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.295+0000 7f55158b6640 1 --2- 172.21.15.26:0/2122736323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f551006b390 0x7f55100c4c30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.295+0000 7f55158b6640 1 -- 172.21.15.26:0/2122736323 >> 172.21.15.26:0/2122736323 conn(0x7f55100ce730 msgr2=0x7f55100c9640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.295+0000 7f55158b6640 1 -- 172.21.15.26:0/2122736323 shutdown_connections 2023-12-19T17:57:49.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.295+0000 7f55158b6640 1 -- 172.21.15.26:0/2122736323 wait complete. 2023-12-19T17:57:49.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:49.316 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-19T17:57:49.320 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:49.318+0000 7f82c4249180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:57:49.332 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1h 2023-12-19T17:57:49.371 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:49.367+0000 7ff1fd10b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:57:49.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.515+0000 7f28ef59e640 1 Processor -- start 2023-12-19T17:57:49.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.515+0000 7f28ef59e640 1 -- start start 2023-12-19T17:57:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.519+0000 7f28ef59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28f0161150 0x7f28f0167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.519+0000 7f28ef59e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f00cd490 con 0x7f28f00cfcb0 2023-12-19T17:57:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.519+0000 7f28ef59e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f00cd770 con 0x7f28f015a770 2023-12-19T17:57:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.519+0000 7f28ef59e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f0161580 con 0x7f28f0161150 2023-12-19T17:57:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.519+0000 7f28e7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f28f00cfcb0 0x7f28f0160a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59594/0 (socket says 172.21.15.26:59594) 2023-12-19T17:57:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.519+0000 7f28e7fff640 1 -- 172.21.15.26:0/620354160 learned_addr learned my addr 172.21.15.26:0/620354160 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.519+0000 7f28eed9d640 1 --2- 172.21.15.26:0/620354160 >> v2:172.21.15.26:3300/0 conn(0x7f28f0161150 0x7f28f0167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.519+0000 7f28edd9b640 1 -- 172.21.15.26:0/620354160 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029246347 0 0) 0x7f28f00cd490 con 0x7f28f00cfcb0 2023-12-19T17:57:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.519+0000 7f28edd9b640 1 -- 172.21.15.26:0/620354160 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28dc003530 con 0x7f28f00cfcb0 2023-12-19T17:57:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.519+0000 7f28edd9b640 1 -- 172.21.15.26:0/620354160 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72085775 0 0) 0x7f28f00cd770 con 0x7f28f015a770 2023-12-19T17:57:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.519+0000 7f28eed9d640 1 -- 172.21.15.26:0/620354160 >> v1:172.21.15.160:6789/0 conn(0x7f28f015a770 legacy=0x7f28f00cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.519+0000 7f28eed9d640 1 -- 172.21.15.26:0/620354160 >> v1:172.21.15.26:6789/0 conn(0x7f28f00cfcb0 legacy=0x7f28f0160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.519+0000 7f28eed9d640 1 -- 172.21.15.26:0/620354160 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f00cda50 con 0x7f28f0161150 2023-12-19T17:57:49.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.523+0000 7f28eed9d640 1 --2- 172.21.15.26:0/620354160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0161150 0x7f28f0167ac0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009d90 tx=0x7f28e801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81be32dc55039838 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:49.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.523+0000 7f28edd9b640 1 -- 172.21.15.26:0/620354160 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80a5980 con 0x7f28f0161150 2023-12-19T17:57:49.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.523+0000 7f28edd9b640 1 -- 172.21.15.26:0/620354160 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e80a5ae0 con 0x7f28f0161150 2023-12-19T17:57:49.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.523+0000 7f28edd9b640 1 -- 172.21.15.26:0/620354160 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80a5e70 con 0x7f28f0161150 2023-12-19T17:57:49.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.523+0000 7f28ef59e640 1 -- 172.21.15.26:0/620354160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0161150 msgr2=0x7f28f0167ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.523+0000 7f28ef59e640 1 --2- 172.21.15.26:0/620354160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0161150 0x7f28f0167ac0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009d90 tx=0x7f28e801c920 comp rx=0 tx=0).stop 2023-12-19T17:57:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.523+0000 7f28ef59e640 1 -- 172.21.15.26:0/620354160 shutdown_connections 2023-12-19T17:57:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.523+0000 7f28ef59e640 1 --2- 172.21.15.26:0/620354160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0161150 0x7f28f0167ac0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.523+0000 7f28ef59e640 1 -- 172.21.15.26:0/620354160 >> 172.21.15.26:0/620354160 conn(0x7f28f006a150 msgr2=0x7f28f006a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.523+0000 7f28ef59e640 1 -- 172.21.15.26:0/620354160 shutdown_connections 2023-12-19T17:57:49.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.523+0000 7f28ef59e640 1 -- 172.21.15.26:0/620354160 wait complete. 2023-12-19T17:57:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28ef59e640 1 Processor -- start 2023-12-19T17:57:49.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28ef59e640 1 -- start start 2023-12-19T17:57:49.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28ef59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28f00cfcb0 0x7f28f013c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:49.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28ef59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28f015a770 0x7f28f0150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:49.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28ef59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0161150 0x7f28f00c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28ef59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00cc4a0 con 0x7f28f0161150 2023-12-19T17:57:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28ef59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00cb8f0 con 0x7f28f00cfcb0 2023-12-19T17:57:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28ef59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00ca470 con 0x7f28f015a770 2023-12-19T17:57:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28eed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0161150 0x7f28f00c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28e7fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28f015a770 0x7f28f0150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28eed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0161150 0x7f28f00c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57898/0 (socket says 172.21.15.26:57898) 2023-12-19T17:57:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28eed9d640 1 -- 172.21.15.26:0/2975931245 learned_addr learned my addr 172.21.15.26:0/2975931245 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28ee59c640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28f00cfcb0 0x7f28f013c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28eed9d640 1 -- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28f015a770 msgr2=0x7f28f0150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28eed9d640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28f015a770 0x7f28f0150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28eed9d640 1 -- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28f00cfcb0 msgr2=0x7f28f013c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28eed9d640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28f00cfcb0 0x7f28f013c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:49.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28eed9d640 1 -- 172.21.15.26:0/2975931245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e8009a70 con 0x7f28f0161150 2023-12-19T17:57:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28eed9d640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0161150 0x7f28f00c9bd0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f28e80b0040 tx=0x7f28e80196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:49.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28e6ffd640 1 -- 172.21.15.26:0/2975931245 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8003ba0 con 0x7f28f0161150 2023-12-19T17:57:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28e6ffd640 1 -- 172.21.15.26:0/2975931245 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e8003d00 con 0x7f28f0161150 2023-12-19T17:57:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28e6ffd640 1 -- 172.21.15.26:0/2975931245 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80ad890 con 0x7f28f0161150 2023-12-19T17:57:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28ef59e640 1 -- 172.21.15.26:0/2975931245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f00cbb70 con 0x7f28f0161150 2023-12-19T17:57:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.527+0000 7f28ef59e640 1 -- 172.21.15.26:0/2975931245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f00cc130 con 0x7f28f0161150 2023-12-19T17:57:49.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.531+0000 7f28ef59e640 1 -- 172.21.15.26:0/2975931245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f0061930 con 0x7f28f0161150 2023-12-19T17:57:49.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.531+0000 7f28e6ffd640 1 -- 172.21.15.26:0/2975931245 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 81) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f28e80b4050 con 0x7f28f0161150 2023-12-19T17:57:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.531+0000 7f28e6ffd640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 0x7f28d00680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.531+0000 7f28e6ffd640 1 -- 172.21.15.26:0/2975931245 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f28e811e270 con 0x7f28f0161150 2023-12-19T17:57:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.531+0000 7f28ee59c640 1 -- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 msgr2=0x7f28d00680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:49.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.531+0000 7f28ee59c640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 0x7f28d00680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:49.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.535+0000 7f28e6ffd640 1 -- 172.21.15.26:0/2975931245 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f28e80ebaa0 con 0x7f28f0161150 2023-12-19T17:57:49.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.715+0000 7f28ef59e640 1 -- 172.21.15.26:0/2975931245 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f28f00d1db0 con 0x7f28d0065c00 2023-12-19T17:57:49.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.735+0000 7f28ee59c640 1 -- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 msgr2=0x7f28d00680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:49.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:49.735+0000 7f28ee59c640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 0x7f28d00680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T17:57:49.803 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:49.798+0000 7f82c4249180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:57:49.862 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:49.858+0000 7f82c4249180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:57:49.889 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:49.887+0000 7ff1fd10b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:57:49.918 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:49.914+0000 7f82c4249180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:57:49.951 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:49.947+0000 7ff1fd10b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:57:50.006 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:50.002+0000 7f82c4249180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:57:50.010 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:50.007+0000 7ff1fd10b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:57:50.100 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:50.099+0000 7ff1fd10b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:57:50.104 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:50.102+0000 7f82c4249180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:57:50.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:50.139+0000 7f28ee59c640 1 -- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 msgr2=0x7f28d00680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:50.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:50.139+0000 7f28ee59c640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 0x7f28d00680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-19T17:57:50.160 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:50.159+0000 7ff1fd10b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:57:50.193 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:50.190+0000 7f82c4249180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:57:50.249 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:50.246+0000 7f82c4249180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:57:50.250 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:50.247+0000 7ff1fd10b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:57:50.310 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:50.307+0000 7ff1fd10b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:57:50.392 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:50.386+0000 7f82c4249180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:57:50.450 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:50.446+0000 7f82c4249180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:57:50.462 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:50.459+0000 7ff1fd10b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:57:50.521 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:50.519+0000 7ff1fd10b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:57:50.691 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:57:50.692 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:57:50.692 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:57:50.701 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:50.694+0000 7f82c4249180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:57:50.773 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:57:50.773 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:57:50.774 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:57:50.781 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:50.779+0000 7ff1fd10b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:57:50.834 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:50.830+0000 7f82c4249180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:57:50.923 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:50.919+0000 7ff1fd10b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:57:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:50.943+0000 7f28ee59c640 1 -- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 msgr2=0x7f28d00680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:50.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:50.943+0000 7f28ee59c640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 0x7f28d00680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-19T17:57:50.957 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:50.954+0000 7f82c4249180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:57:51.027 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:51.022+0000 7f82c4249180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:57:51.055 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:51.051+0000 7ff1fd10b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:57:51.129 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:51.126+0000 7ff1fd10b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:57:51.162 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:51.158+0000 7f82c4249180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:57:51.273 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:51.270+0000 7ff1fd10b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:57:51.702 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:51.698+0000 7f82c4249180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:57:51.785 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:51.782+0000 7f82c4249180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:57:51.841 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:51.838+0000 7ff1fd10b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:57:51.930 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:51.926+0000 7ff1fd10b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:57:51.933 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:51.930+0000 7f82c4249180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:57:52.017 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:52.014+0000 7f82c4249180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:57:52.087 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:52.082+0000 7ff1fd10b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:57:52.177 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:52.174+0000 7ff1fd10b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:57:52.334 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:52.330+0000 7f82c4249180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:57:52.393 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:52.390+0000 7f82c4249180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:57:52.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:52.402+0000 7f28e6ffd640 1 -- 172.21.15.26:0/2975931245 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 82) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f28e80ebe70 con 0x7f28f0161150 2023-12-19T17:57:52.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:52.402+0000 7f28e6ffd640 1 -- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 msgr2=0x7f28d00680c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:52.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:52.402+0000 7f28e6ffd640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 0x7f28d00680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:52.520 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:52.518+0000 7ff1fd10b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:57:52.527 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:52.522+0000 7f8249713640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:52.527 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:52.522+0000 7f8249713640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:52.528 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:52.522+0000 7f8249713640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:52.528 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:52.522+0000 7f8249713640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:52.528 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:52.522+0000 7f8249713640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:52.529 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:52.526+0000 7f824670d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:52.529 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:52.526+0000 7f824670d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:52.529 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:52.526+0000 7f824670d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:52.529 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:52.526+0000 7f824670d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:52.529 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:52.526+0000 7f824670d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:52.585 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:52.582+0000 7ff1fd10b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:57:53.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.406+0000 7f28e6ffd640 1 -- 172.21.15.26:0/2975931245 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 83) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f28e80f52c0 con 0x7f28f0161150 2023-12-19T17:57:53.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.406+0000 7f28e6ffd640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 0x7f28d00680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:53.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.406+0000 7f28e6ffd640 1 -- 172.21.15.26:0/2975931245 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f28d006bc00 con 0x7f28d0065c00 2023-12-19T17:57:53.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.406+0000 7f28ee59c640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 0x7f28d00680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:53.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.414+0000 7f28ee59c640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 0x7f28d00680c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f28e0000f70 tx=0x7f28e0007d40 comp rx=0 tx=0).ready entity=mgr.7742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:53.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.478+0000 7f28e6ffd640 1 -- 172.21.15.26:0/2975931245 <== mgr.7742 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f28d006bc00 con 0x7f28d0065c00 2023-12-19T17:57:53.480 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path snap_test_dir1 2023-12-19T17:57:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.478+0000 7f28ef59e640 1 -- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 msgr2=0x7f28d00680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.478+0000 7f28ef59e640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 0x7f28d00680c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f28e0000f70 tx=0x7f28e0007d40 comp rx=0 tx=0).stop 2023-12-19T17:57:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.478+0000 7f28ef59e640 1 -- 172.21.15.26:0/2975931245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0161150 msgr2=0x7f28f00c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.478+0000 7f28ef59e640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0161150 0x7f28f00c9bd0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f28e80b0040 tx=0x7f28e80196c0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.478+0000 7f28ef59e640 1 -- 172.21.15.26:0/2975931245 shutdown_connections 2023-12-19T17:57:53.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.478+0000 7f28ef59e640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f0161150 0x7f28f00c9bd0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.478+0000 7f28ef59e640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f28d0065c00 0x7f28d00680c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.478+0000 7f28ef59e640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f28f015a770 0x7f28f0150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.478+0000 7f28ef59e640 1 --2- 172.21.15.26:0/2975931245 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f28f00cfcb0 0x7f28f013c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.478+0000 7f28ef59e640 1 -- 172.21.15.26:0/2975931245 >> 172.21.15.26:0/2975931245 conn(0x7f28f006a150 msgr2=0x7f28f006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.478+0000 7f28ef59e640 1 -- 172.21.15.26:0/2975931245 shutdown_connections 2023-12-19T17:57:53.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.478+0000 7f28ef59e640 1 -- 172.21.15.26:0/2975931245 wait complete. 2023-12-19T17:57:53.503 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-19T17:57:53.503 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-19T17:57:53.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.646+0000 7f62994c3640 1 Processor -- start 2023-12-19T17:57:53.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.646+0000 7f62994c3640 1 -- start start 2023-12-19T17:57:53.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.646+0000 7f62994c3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6294072120 0x7f6294071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:53.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.646+0000 7f62994c3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629406baa0 0x7f629415c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:53.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.646+0000 7f62994c3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62940d0490 con 0x7f6294071d40 2023-12-19T17:57:53.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.646+0000 7f62994c3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62940cfb60 con 0x7f6294072120 2023-12-19T17:57:53.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.646+0000 7f62994c3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62940cfe40 con 0x7f629406baa0 2023-12-19T17:57:53.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f62937fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6294072120 0x7f6294071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:53.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f6293fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f6294071d40 0x7f629406b390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:50336/0 (socket says 172.21.15.26:50336) 2023-12-19T17:57:53.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f6293fff640 1 -- 172.21.15.26:0/1285526706 learned_addr learned my addr 172.21.15.26:0/1285526706 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:53.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f6298cc2640 1 --2- 172.21.15.26:0/1285526706 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629406baa0 0x7f629415c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:53.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f62937fe640 1 -- 172.21.15.26:0/1285526706 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629406baa0 msgr2=0x7f629415c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:53.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f62937fe640 1 --2- 172.21.15.26:0/1285526706 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629406baa0 0x7f629415c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f62937fe640 1 -- 172.21.15.26:0/1285526706 >> v1:172.21.15.160:6789/0 conn(0x7f6294071d40 legacy=0x7f629406b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:53.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f62937fe640 1 -- 172.21.15.26:0/1285526706 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62940d0120 con 0x7f6294072120 2023-12-19T17:57:53.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f6292ffd640 1 -- 172.21.15.26:0/1285526706 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525420518 0 0) 0x7f62940d0490 con 0x7f6294071d40 2023-12-19T17:57:53.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f6298cc2640 1 --2- 172.21.15.26:0/1285526706 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629406baa0 0x7f629415c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:53.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f62937fe640 1 --2- 172.21.15.26:0/1285526706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6294072120 0x7f6294071230 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f62880097c0 tx=0x7f628801c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c1f4557a1877888 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:53.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f6292ffd640 1 -- 172.21.15.26:0/1285526706 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62940cd080 con 0x7f6294072120 2023-12-19T17:57:53.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f6292ffd640 1 -- 172.21.15.26:0/1285526706 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62880ab610 con 0x7f6294072120 2023-12-19T17:57:53.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.650+0000 7f6292ffd640 1 -- 172.21.15.26:0/1285526706 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62880ab9a0 con 0x7f6294072120 2023-12-19T17:57:53.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 -- 172.21.15.26:0/1285526706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6294072120 msgr2=0x7f6294071230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:53.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 --2- 172.21.15.26:0/1285526706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6294072120 0x7f6294071230 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f62880097c0 tx=0x7f628801c3f0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 -- 172.21.15.26:0/1285526706 shutdown_connections 2023-12-19T17:57:53.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 --2- 172.21.15.26:0/1285526706 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629406baa0 0x7f629415c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 --2- 172.21.15.26:0/1285526706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6294072120 0x7f6294071230 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 -- 172.21.15.26:0/1285526706 >> 172.21.15.26:0/1285526706 conn(0x7f62940cd210 msgr2=0x7f62940cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:53.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 -- 172.21.15.26:0/1285526706 shutdown_connections 2023-12-19T17:57:53.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 -- 172.21.15.26:0/1285526706 wait complete. 2023-12-19T17:57:53.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 Processor -- start 2023-12-19T17:57:53.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 -- start start 2023-12-19T17:57:53.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629406baa0 0x7f6294158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:53.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6294071d40 0x7f62940ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:53.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6294072120 0x7f62940b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:53.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629415eb80 con 0x7f629406baa0 2023-12-19T17:57:53.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6294158b20 con 0x7f6294071d40 2023-12-19T17:57:53.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62994c3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62940c8960 con 0x7f6294072120 2023-12-19T17:57:53.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f6293fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629406baa0 0x7f6294158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:53.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f6293fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629406baa0 0x7f6294158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57908/0 (socket says 172.21.15.26:57908) 2023-12-19T17:57:53.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f6293fff640 1 -- 172.21.15.26:0/738343150 learned_addr learned my addr 172.21.15.26:0/738343150 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:53.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f62937fe640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6294071d40 0x7f62940ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:53.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.654+0000 7f6298cc2640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6294072120 0x7f62940b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:53.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.658+0000 7f6293fff640 1 -- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6294072120 msgr2=0x7f62940b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:53.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.658+0000 7f6293fff640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6294072120 0x7f62940b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.658+0000 7f6293fff640 1 -- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6294071d40 msgr2=0x7f62940ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:53.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.658+0000 7f6293fff640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6294071d40 0x7f62940ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.658+0000 7f6293fff640 1 -- 172.21.15.26:0/738343150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c009a70 con 0x7f629406baa0 2023-12-19T17:57:53.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.658+0000 7f6293fff640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629406baa0 0x7f6294158020 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f628400c520 tx=0x7f628400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:53.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.658+0000 7f6290ff9640 1 -- 172.21.15.26:0/738343150 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62840074e0 con 0x7f629406baa0 2023-12-19T17:57:53.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.658+0000 7f6290ff9640 1 -- 172.21.15.26:0/738343150 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6284007640 con 0x7f629406baa0 2023-12-19T17:57:53.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.658+0000 7f6290ff9640 1 -- 172.21.15.26:0/738343150 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62840049f0 con 0x7f629406baa0 2023-12-19T17:57:53.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.658+0000 7f62994c3640 1 -- 172.21.15.26:0/738343150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6288009420 con 0x7f629406baa0 2023-12-19T17:57:53.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.658+0000 7f62994c3640 1 -- 172.21.15.26:0/738343150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629415e7a0 con 0x7f629406baa0 2023-12-19T17:57:53.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.658+0000 7f62994c3640 1 -- 172.21.15.26:0/738343150 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629404e850 con 0x7f629406baa0 2023-12-19T17:57:53.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.658+0000 7f6290ff9640 1 -- 172.21.15.26:0/738343150 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 83) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f6284004b50 con 0x7f629406baa0 2023-12-19T17:57:53.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.662+0000 7f6290ff9640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6280065900 0x7f6280067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:53.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.662+0000 7f6290ff9640 1 -- 172.21.15.26:0/738343150 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f62840823d0 con 0x7f629406baa0 2023-12-19T17:57:53.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.662+0000 7f62937fe640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6280065900 0x7f6280067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:53.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.662+0000 7f62937fe640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6280065900 0x7f6280067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f62880af000 tx=0x7f62880b6040 comp rx=0 tx=0).ready entity=mgr.7742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:53.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.666+0000 7f6290ff9640 1 -- 172.21.15.26:0/738343150 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f628404fc70 con 0x7f629406baa0 2023-12-19T17:57:53.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.842+0000 7f62994c3640 1 -- 172.21.15.26:0/738343150 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f62940cf110 con 0x7f6280065900 2023-12-19T17:57:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.906+0000 7f6290ff9640 1 -- 172.21.15.26:0/738343150 <== mgr.7742 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f62940cf110 con 0x7f6280065900 2023-12-19T17:57:53.910 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path snap_test_dir1 2023-12-19T17:57:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.906+0000 7f62994c3640 1 -- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6280065900 msgr2=0x7f6280067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.906+0000 7f62994c3640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6280065900 0x7f6280067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f62880af000 tx=0x7f62880b6040 comp rx=0 tx=0).stop 2023-12-19T17:57:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.906+0000 7f62994c3640 1 -- 172.21.15.26:0/738343150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629406baa0 msgr2=0x7f6294158020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.906+0000 7f62994c3640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629406baa0 0x7f6294158020 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f628400c520 tx=0x7f628400c920 comp rx=0 tx=0).stop 2023-12-19T17:57:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.910+0000 7f62994c3640 1 -- 172.21.15.26:0/738343150 shutdown_connections 2023-12-19T17:57:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.910+0000 7f62994c3640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6280065900 0x7f6280067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.910+0000 7f62994c3640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6294072120 0x7f62940b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.910+0000 7f62994c3640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6294071d40 0x7f62940ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.910+0000 7f62994c3640 1 --2- 172.21.15.26:0/738343150 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629406baa0 0x7f6294158020 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.910+0000 7f62994c3640 1 -- 172.21.15.26:0/738343150 >> 172.21.15.26:0/738343150 conn(0x7f62940cd210 msgr2=0x7f62940cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:53.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.910+0000 7f62994c3640 1 -- 172.21.15.26:0/738343150 shutdown_connections 2023-12-19T17:57:53.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:53.910+0000 7f62994c3640 1 -- 172.21.15.26:0/738343150 wait complete. 2023-12-19T17:57:53.931 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-19T17:57:53.932 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-12-19T17:57:54.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f074d48c640 1 Processor -- start 2023-12-19T17:57:54.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f074d48c640 1 -- start start 2023-12-19T17:57:54.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f074d48c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07481599e0 0x7f074806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:54.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f074d48c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07480cb150 0x7f07480cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f074d48c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f074815d530 0x7f074815d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f074d48c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07480cbaf0 con 0x7f07480cb150 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f074d48c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074815e240 con 0x7f074815d530 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f074d48c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07480c9520 con 0x7f07481599e0 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f073ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07480cb150 0x7f07480cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f073ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07480cb150 0x7f07480cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57924/0 (socket says 172.21.15.26:57924) 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f073ffff640 1 -- 172.21.15.26:0/188886143 learned_addr learned my addr 172.21.15.26:0/188886143 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f0746ffd640 1 --2- 172.21.15.26:0/188886143 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07481599e0 0x7f074806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f073ffff640 1 -- 172.21.15.26:0/188886143 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07481599e0 msgr2=0x7f074806f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f07477fe640 1 --2- 172.21.15.26:0/188886143 >> v2:172.21.15.160:3300/0 conn(0x7f074815d530 0x7f074815d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f073ffff640 1 --2- 172.21.15.26:0/188886143 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07481599e0 0x7f074806f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f073ffff640 1 -- 172.21.15.26:0/188886143 >> v2:172.21.15.160:3300/0 conn(0x7f074815d530 msgr2=0x7f074815d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f073ffff640 1 --2- 172.21.15.26:0/188886143 >> v2:172.21.15.160:3300/0 conn(0x7f074815d530 0x7f074815d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:54.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f073ffff640 1 -- 172.21.15.26:0/188886143 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07480c9800 con 0x7f07480cb150 2023-12-19T17:57:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f0746ffd640 1 --2- 172.21.15.26:0/188886143 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07481599e0 0x7f074806f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:57:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.074+0000 7f07477fe640 1 --2- 172.21.15.26:0/188886143 >> v2:172.21.15.160:3300/0 conn(0x7f074815d530 0x7f074815d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:57:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.078+0000 7f073ffff640 1 --2- 172.21.15.26:0/188886143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07480cb150 0x7f07480cb5b0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f07480cc220 tx=0x7f073001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba6c276d3b9869a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.078+0000 7f07467fc640 1 -- 172.21.15.26:0/188886143 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300ab680 con 0x7f07480cb150 2023-12-19T17:57:54.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.078+0000 7f07467fc640 1 -- 172.21.15.26:0/188886143 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07300ab7e0 con 0x7f07480cb150 2023-12-19T17:57:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.078+0000 7f07467fc640 1 -- 172.21.15.26:0/188886143 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300abb70 con 0x7f07480cb150 2023-12-19T17:57:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.078+0000 7f074d48c640 1 -- 172.21.15.26:0/188886143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07480cb150 msgr2=0x7f07480cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.078+0000 7f074d48c640 1 --2- 172.21.15.26:0/188886143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07480cb150 0x7f07480cb5b0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f07480cc220 tx=0x7f073001c500 comp rx=0 tx=0).stop 2023-12-19T17:57:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.078+0000 7f074d48c640 1 -- 172.21.15.26:0/188886143 shutdown_connections 2023-12-19T17:57:54.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.078+0000 7f074d48c640 1 --2- 172.21.15.26:0/188886143 >> v2:172.21.15.160:3300/0 conn(0x7f074815d530 0x7f074815d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:54.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.078+0000 7f074d48c640 1 --2- 172.21.15.26:0/188886143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07480cb150 0x7f07480cb5b0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:54.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.078+0000 7f074d48c640 1 --2- 172.21.15.26:0/188886143 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07481599e0 0x7f074806f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:54.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.078+0000 7f074d48c640 1 -- 172.21.15.26:0/188886143 >> 172.21.15.26:0/188886143 conn(0x7f0748066160 msgr2=0x7f07480b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:54.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.078+0000 7f074d48c640 1 -- 172.21.15.26:0/188886143 shutdown_connections 2023-12-19T17:57:54.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.078+0000 7f074d48c640 1 -- 172.21.15.26:0/188886143 wait complete. 2023-12-19T17:57:54.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f074d48c640 1 Processor -- start 2023-12-19T17:57:54.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f074d48c640 1 -- start start 2023-12-19T17:57:54.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f074d48c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07480cb150 0x7f07481484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:54.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f074d48c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07481599e0 0x7f074815be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:54.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f074d48c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f074815d530 0x7f0748150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:54.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f074d48c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074815cb10 con 0x7f07481599e0 2023-12-19T17:57:54.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f074d48c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07480c45c0 con 0x7f07480cb150 2023-12-19T17:57:54.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f074d48c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07480c48a0 con 0x7f074815d530 2023-12-19T17:57:54.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f073ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07481599e0 0x7f074815be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:54.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f073ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07481599e0 0x7f074815be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57934/0 (socket says 172.21.15.26:57934) 2023-12-19T17:57:54.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f073ffff640 1 -- 172.21.15.26:0/2494657698 learned_addr learned my addr 172.21.15.26:0/2494657698 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:54.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f0746ffd640 1 --2- 172.21.15.26:0/2494657698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07480cb150 0x7f07481484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:54.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f07477fe640 1 --2- 172.21.15.26:0/2494657698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f074815d530 0x7f0748150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:54.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f073ffff640 1 -- 172.21.15.26:0/2494657698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f074815d530 msgr2=0x7f0748150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:54.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f073ffff640 1 --2- 172.21.15.26:0/2494657698 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f074815d530 0x7f0748150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:54.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f073ffff640 1 -- 172.21.15.26:0/2494657698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07480cb150 msgr2=0x7f07481484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:54.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f073ffff640 1 --2- 172.21.15.26:0/2494657698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07480cb150 0x7f07481484f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:54.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f073ffff640 1 -- 172.21.15.26:0/2494657698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07300094d0 con 0x7f07481599e0 2023-12-19T17:57:54.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f073ffff640 1 --2- 172.21.15.26:0/2494657698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07481599e0 0x7f074815be10 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f073001abb0 tx=0x7f073001ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:54.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f073f7fe640 1 -- 172.21.15.26:0/2494657698 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300b48c0 con 0x7f07481599e0 2023-12-19T17:57:54.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f073f7fe640 1 -- 172.21.15.26:0/2494657698 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07300b4a40 con 0x7f07481599e0 2023-12-19T17:57:54.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f074d48c640 1 -- 172.21.15.26:0/2494657698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0748148c60 con 0x7f07481599e0 2023-12-19T17:57:54.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f073f7fe640 1 -- 172.21.15.26:0/2494657698 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300b4df0 con 0x7f07481599e0 2023-12-19T17:57:54.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f074d48c640 1 -- 172.21.15.26:0/2494657698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074815c350 con 0x7f07481599e0 2023-12-19T17:57:54.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.082+0000 7f074d48c640 1 -- 172.21.15.26:0/2494657698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0748074270 con 0x7f07481599e0 2023-12-19T17:57:54.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.086+0000 7f073f7fe640 1 -- 172.21.15.26:0/2494657698 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 83) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f07300b3380 con 0x7f07481599e0 2023-12-19T17:57:54.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.086+0000 7f073f7fe640 1 --2- 172.21.15.26:0/2494657698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0728065900 0x7f0728067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:54.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.086+0000 7f073f7fe640 1 -- 172.21.15.26:0/2494657698 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f073011dd50 con 0x7f07481599e0 2023-12-19T17:57:54.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.086+0000 7f0746ffd640 1 --2- 172.21.15.26:0/2494657698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0728065900 0x7f0728067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:54.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.086+0000 7f0746ffd640 1 --2- 172.21.15.26:0/2494657698 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0728065900 0x7f0728067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f074814d970 tx=0x7f0738008040 comp rx=0 tx=0).ready entity=mgr.7742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:54.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.090+0000 7f073f7fe640 1 -- 172.21.15.26:0/2494657698 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f07300eb710 con 0x7f07481599e0 2023-12-19T17:57:54.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.266+0000 7f074d48c640 1 -- 172.21.15.26:0/2494657698 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0748148ae0 con 0x7f0728065900 2023-12-19T17:57:54.276 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:54.274+0000 7f8250721640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-12-19T17:57:54.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.274+0000 7f073f7fe640 1 -- 172.21.15.26:0/2494657698 <== mgr.7742 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f073800a670 con 0x7f0728065900 2023-12-19T17:57:54.276 INFO:teuthology.orchestra.run.smithi026.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-12-19T17:57:54.285 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-12-19T17:57:54.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:54.285 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-19T17:57:54.297 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-19T17:57:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.486+0000 7f344bcb6640 1 Processor -- start 2023-12-19T17:57:54.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.486+0000 7f344bcb6640 1 -- start start 2023-12-19T17:57:54.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.486+0000 7f344bcb6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34440ca640 0x7f3444071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:54.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.486+0000 7f344bcb6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34441615b0 con 0x7f344406d1e0 2023-12-19T17:57:54.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.486+0000 7f344bcb6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34440c8d90 con 0x7f34440adc60 2023-12-19T17:57:54.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.486+0000 7f344bcb6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34440caa90 con 0x7f34440ca640 2023-12-19T17:57:54.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.486+0000 7f3449251640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f344406d1e0 0x7f344406f250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59604/0 (socket says 172.21.15.26:59604) 2023-12-19T17:57:54.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.486+0000 7f344a253640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34440ca640 0x7f3444071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:54.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.486+0000 7f3449251640 1 -- 172.21.15.26:0/1832705315 learned_addr learned my addr 172.21.15.26:0/1832705315 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:54.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.486+0000 7f344a253640 1 -- 172.21.15.26:0/1832705315 >> v1:172.21.15.160:6789/0 conn(0x7f34440adc60 legacy=0x7f344406cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:54.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.486+0000 7f3448a50640 1 -- 172.21.15.26:0/1832705315 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50494261 0 0) 0x7f34441615b0 con 0x7f344406d1e0 2023-12-19T17:57:54.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.486+0000 7f344a253640 1 -- 172.21.15.26:0/1832705315 >> v1:172.21.15.26:6789/0 conn(0x7f344406d1e0 legacy=0x7f344406f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:54.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.486+0000 7f344a253640 1 -- 172.21.15.26:0/1832705315 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34440c9070 con 0x7f34440ca640 2023-12-19T17:57:54.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.490+0000 7f344a253640 1 --2- 172.21.15.26:0/1832705315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440ca640 0x7f3444071100 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f343c009d90 tx=0x7f343c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec15f9e17d58995 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:54.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.490+0000 7f3448a50640 1 -- 172.21.15.26:0/1832705315 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0a5950 con 0x7f34440ca640 2023-12-19T17:57:54.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.490+0000 7f3448a50640 1 -- 172.21.15.26:0/1832705315 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c0a5ab0 con 0x7f34440ca640 2023-12-19T17:57:54.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.490+0000 7f3448a50640 1 -- 172.21.15.26:0/1832705315 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0a5e40 con 0x7f34440ca640 2023-12-19T17:57:54.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.490+0000 7f344bcb6640 1 -- 172.21.15.26:0/1832705315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440ca640 msgr2=0x7f3444071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:54.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.490+0000 7f344bcb6640 1 --2- 172.21.15.26:0/1832705315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440ca640 0x7f3444071100 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f343c009d90 tx=0x7f343c01c890 comp rx=0 tx=0).stop 2023-12-19T17:57:54.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.490+0000 7f344bcb6640 1 -- 172.21.15.26:0/1832705315 shutdown_connections 2023-12-19T17:57:54.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.490+0000 7f344bcb6640 1 --2- 172.21.15.26:0/1832705315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440ca640 0x7f3444071100 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:54.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.490+0000 7f344bcb6640 1 -- 172.21.15.26:0/1832705315 >> 172.21.15.26:0/1832705315 conn(0x7f3444066160 msgr2=0x7f34440b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:54.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.490+0000 7f344bcb6640 1 -- 172.21.15.26:0/1832705315 shutdown_connections 2023-12-19T17:57:54.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.490+0000 7f344bcb6640 1 -- 172.21.15.26:0/1832705315 wait complete. 2023-12-19T17:57:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f344bcb6640 1 Processor -- start 2023-12-19T17:57:54.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f344bcb6640 1 -- start start 2023-12-19T17:57:54.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f344bcb6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f344406d1e0 0x7f344415c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f344bcb6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440adc60 0x7f34440b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f344bcb6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34440ca640 0x7f344415af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f344bcb6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344415f430 con 0x7f34440adc60 2023-12-19T17:57:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f344bcb6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344415bf80 con 0x7f34440ca640 2023-12-19T17:57:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f344bcb6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344415f9d0 con 0x7f344406d1e0 2023-12-19T17:57:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f3449251640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440adc60 0x7f34440b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f3449251640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440adc60 0x7f34440b1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57960/0 (socket says 172.21.15.26:57960) 2023-12-19T17:57:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f3449251640 1 -- 172.21.15.26:0/3640081484 learned_addr learned my addr 172.21.15.26:0/3640081484 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f3449a52640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f344406d1e0 0x7f344415c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f344a253640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34440ca640 0x7f344415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f3449251640 1 -- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f344406d1e0 msgr2=0x7f344415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f3449251640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f344406d1e0 0x7f344415c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:54.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f3449251640 1 -- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34440ca640 msgr2=0x7f344415af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:54.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f3449251640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34440ca640 0x7f344415af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:54.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f3449251640 1 -- 172.21.15.26:0/3640081484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c009a70 con 0x7f34440adc60 2023-12-19T17:57:54.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f3449251640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440adc60 0x7f34440b1780 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f343400c4a0 tx=0x7f343400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:54.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f343a7fc640 1 -- 172.21.15.26:0/3640081484 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34340074e0 con 0x7f34440adc60 2023-12-19T17:57:54.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f343a7fc640 1 -- 172.21.15.26:0/3640081484 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434007640 con 0x7f34440adc60 2023-12-19T17:57:54.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f344bcb6640 1 -- 172.21.15.26:0/3640081484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344415b9c0 con 0x7f34440adc60 2023-12-19T17:57:54.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f343a7fc640 1 -- 172.21.15.26:0/3640081484 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434004a30 con 0x7f34440adc60 2023-12-19T17:57:54.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.494+0000 7f344bcb6640 1 -- 172.21.15.26:0/3640081484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344415f660 con 0x7f34440adc60 2023-12-19T17:57:54.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.498+0000 7f344bcb6640 1 -- 172.21.15.26:0/3640081484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344415fb40 con 0x7f34440adc60 2023-12-19T17:57:54.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.498+0000 7f343a7fc640 1 -- 172.21.15.26:0/3640081484 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 83) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f3434004b90 con 0x7f34440adc60 2023-12-19T17:57:54.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.498+0000 7f343a7fc640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3428065900 0x7f3428067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:54.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.498+0000 7f343a7fc640 1 -- 172.21.15.26:0/3640081484 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f3434082430 con 0x7f34440adc60 2023-12-19T17:57:54.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.498+0000 7f3449a52640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3428065900 0x7f3428067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:54.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.498+0000 7f3449a52640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3428065900 0x7f3428067dc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3440007fd0 tx=0x7f3440009040 comp rx=0 tx=0).ready entity=mgr.7742 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:54.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.502+0000 7f343a7fc640 1 -- 172.21.15.26:0/3640081484 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f343404fda0 con 0x7f34440adc60 2023-12-19T17:57:54.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.694+0000 7f343a7fc640 1 -- 172.21.15.26:0/3640081484 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 84) v1 ==== 85348+0+0 (secure 0 0 0) 0x7f3434050170 con 0x7f34440adc60 2023-12-19T17:57:54.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:54.706+0000 7f344bcb6640 1 -- 172.21.15.26:0/3640081484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f3444074270 con 0x7f34440adc60 2023-12-19T17:57:55.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.698+0000 7f343a7fc640 1 -- 172.21.15.26:0/3640081484 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 85) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f3434071460 con 0x7f34440adc60 2023-12-19T17:57:55.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.702+0000 7f343a7fc640 1 -- 172.21.15.26:0/3640081484 <== mon.0 v2:172.21.15.26:3300/0 9 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v85) v1 ==== 93+0+0 (secure 0 0 0) 0x7f3434050db0 con 0x7f34440adc60 2023-12-19T17:57:55.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.702+0000 7f344bcb6640 1 -- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3428065900 msgr2=0x7f3428067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:55.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.702+0000 7f344bcb6640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3428065900 0x7f3428067dc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3440007fd0 tx=0x7f3440009040 comp rx=0 tx=0).stop 2023-12-19T17:57:55.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.702+0000 7f344bcb6640 1 -- 172.21.15.26:0/3640081484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440adc60 msgr2=0x7f34440b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:55.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.702+0000 7f344bcb6640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440adc60 0x7f34440b1780 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f343400c4a0 tx=0x7f343400c960 comp rx=0 tx=0).stop 2023-12-19T17:57:55.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.702+0000 7f344bcb6640 1 -- 172.21.15.26:0/3640081484 shutdown_connections 2023-12-19T17:57:55.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.702+0000 7f344bcb6640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f34440ca640 0x7f344415af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:55.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.702+0000 7f344bcb6640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440adc60 0x7f34440b1780 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:55.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.702+0000 7f344bcb6640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3428065900 0x7f3428067dc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:55.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.702+0000 7f344bcb6640 1 --2- 172.21.15.26:0/3640081484 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f344406d1e0 0x7f344415c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:55.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.702+0000 7f344bcb6640 1 -- 172.21.15.26:0/3640081484 >> 172.21.15.26:0/3640081484 conn(0x7f3444066160 msgr2=0x7f344406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:55.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.702+0000 7f344bcb6640 1 -- 172.21.15.26:0/3640081484 shutdown_connections 2023-12-19T17:57:55.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:55.702+0000 7f344bcb6640 1 -- 172.21.15.26:0/3640081484 wait complete. 2023-12-19T17:57:55.731 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T17:57:55.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T17:57:55.810 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T17:57:55.882 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:55.878+0000 7fb3e6e4d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:57:55.886 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:55.882+0000 7f81531f6180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:57:55.890 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T17:57:55.890 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:57:55.890 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:55.900 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:57:55.900 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:55.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:57:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f408b77b640 1 Processor -- start 2023-12-19T17:57:56.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f408b77b640 1 -- start start 2023-12-19T17:57:56.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f408b77b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f408415cf60 0x7f40840d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f408b77b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40840d1420 0x7f40840b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f408b77b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f40840b88e0 0x7f40840b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f408b77b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40840d2960 con 0x7f408415cf60 2023-12-19T17:57:56.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f408b77b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40840d18c0 con 0x7f40840b88e0 2023-12-19T17:57:56.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f408b77b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40840d1ba0 con 0x7f40840d1420 2023-12-19T17:57:56.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f4089517640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f408415cf60 0x7f40840d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f4089517640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f408415cf60 0x7f40840d0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39758/0 (socket says 172.21.15.26:39758) 2023-12-19T17:57:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f4089517640 1 -- 172.21.15.26:0/1921712161 learned_addr learned my addr 172.21.15.26:0/1921712161 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f4089d18640 1 --2- 172.21.15.26:0/1921712161 >> v2:172.21.15.160:3300/0 conn(0x7f40840b88e0 0x7f40840b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f4088d16640 1 --2- 172.21.15.26:0/1921712161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40840d1420 0x7f40840b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f4089517640 1 -- 172.21.15.26:0/1921712161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40840d1420 msgr2=0x7f40840b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f4089517640 1 --2- 172.21.15.26:0/1921712161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40840d1420 0x7f40840b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f4089517640 1 -- 172.21.15.26:0/1921712161 >> v2:172.21.15.160:3300/0 conn(0x7f40840b88e0 msgr2=0x7f40840b4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f4089517640 1 --2- 172.21.15.26:0/1921712161 >> v2:172.21.15.160:3300/0 conn(0x7f40840b88e0 0x7f40840b4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f4089517640 1 -- 172.21.15.26:0/1921712161 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40840d1e80 con 0x7f408415cf60 2023-12-19T17:57:56.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.138+0000 7f4089d18640 1 --2- 172.21.15.26:0/1921712161 >> v2:172.21.15.160:3300/0 conn(0x7f40840b88e0 0x7f40840b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:56.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.142+0000 7f4089517640 1 --2- 172.21.15.26:0/1921712161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f408415cf60 0x7f40840d0ee0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f408408be50 tx=0x7f408001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9963e3e674bbb4e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:56.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.142+0000 7f407bfff640 1 -- 172.21.15.26:0/1921712161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40800a89b0 con 0x7f408415cf60 2023-12-19T17:57:56.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.142+0000 7f407bfff640 1 -- 172.21.15.26:0/1921712161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40800a8b10 con 0x7f408415cf60 2023-12-19T17:57:56.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.142+0000 7f407bfff640 1 -- 172.21.15.26:0/1921712161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40800a8ea0 con 0x7f408415cf60 2023-12-19T17:57:56.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.142+0000 7f408b77b640 1 -- 172.21.15.26:0/1921712161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f408415cf60 msgr2=0x7f40840d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:56.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.142+0000 7f408b77b640 1 --2- 172.21.15.26:0/1921712161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f408415cf60 0x7f40840d0ee0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f408408be50 tx=0x7f408001ecf0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.142+0000 7f408b77b640 1 -- 172.21.15.26:0/1921712161 shutdown_connections 2023-12-19T17:57:56.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.142+0000 7f408b77b640 1 --2- 172.21.15.26:0/1921712161 >> v2:172.21.15.160:3300/0 conn(0x7f40840b88e0 0x7f40840b4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.142+0000 7f408b77b640 1 --2- 172.21.15.26:0/1921712161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40840d1420 0x7f40840b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.142+0000 7f408b77b640 1 --2- 172.21.15.26:0/1921712161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f408415cf60 0x7f40840d0ee0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.142+0000 7f408b77b640 1 -- 172.21.15.26:0/1921712161 >> 172.21.15.26:0/1921712161 conn(0x7f4084065ee0 msgr2=0x7f40840cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:56.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.142+0000 7f408b77b640 1 -- 172.21.15.26:0/1921712161 shutdown_connections 2023-12-19T17:57:56.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.142+0000 7f408b77b640 1 -- 172.21.15.26:0/1921712161 wait complete. 2023-12-19T17:57:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f408b77b640 1 Processor -- start 2023-12-19T17:57:56.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f408b77b640 1 -- start start 2023-12-19T17:57:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f408b77b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40840b88e0 0x7f40840c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f408b77b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40840d1420 0x7f4084142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f408b77b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f408415cf60 0x7f408415f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f408b77b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408415e530 con 0x7f408415cf60 2023-12-19T17:57:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f408b77b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408415e810 con 0x7f40840b88e0 2023-12-19T17:57:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f408b77b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408415eaf0 con 0x7f40840d1420 2023-12-19T17:57:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f4089d18640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f408415cf60 0x7f408415f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f4089517640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40840b88e0 0x7f40840c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f4088d16640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40840d1420 0x7f4084142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f4089d18640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f408415cf60 0x7f408415f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39774/0 (socket says 172.21.15.26:39774) 2023-12-19T17:57:56.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f4089d18640 1 -- 172.21.15.26:0/3561813032 learned_addr learned my addr 172.21.15.26:0/3561813032 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:56.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f4089d18640 1 -- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40840d1420 msgr2=0x7f4084142d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:56.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f4089d18640 1 --2- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40840d1420 0x7f4084142d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f4089d18640 1 -- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40840b88e0 msgr2=0x7f40840c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:56.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f4089d18640 1 --2- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40840b88e0 0x7f40840c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f4089d18640 1 -- 172.21.15.26:0/3561813032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40800094d0 con 0x7f408415cf60 2023-12-19T17:57:56.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f4089517640 1 --2- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40840b88e0 0x7f40840c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:56.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f4088d16640 1 --2- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40840d1420 0x7f4084142d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.146+0000 7f4089d18640 1 --2- 172.21.15.26:0/3561813032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f408415cf60 0x7f408415f2f0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f407c007db0 tx=0x7f407c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.150+0000 7f4079ffb640 1 -- 172.21.15.26:0/3561813032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c0039a0 con 0x7f408415cf60 2023-12-19T17:57:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.150+0000 7f4079ffb640 1 -- 172.21.15.26:0/3561813032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c00f040 con 0x7f408415cf60 2023-12-19T17:57:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.150+0000 7f4079ffb640 1 -- 172.21.15.26:0/3561813032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c003d40 con 0x7f408415cf60 2023-12-19T17:57:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.150+0000 7f408b77b640 1 -- 172.21.15.26:0/3561813032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408415edd0 con 0x7f408415cf60 2023-12-19T17:57:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.150+0000 7f408b77b640 1 -- 172.21.15.26:0/3561813032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40840ca4c0 con 0x7f408415cf60 2023-12-19T17:57:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.150+0000 7f408b77b640 1 -- 172.21.15.26:0/3561813032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408415eff0 con 0x7f408415cf60 2023-12-19T17:57:56.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.150+0000 7f4079ffb640 1 -- 172.21.15.26:0/3561813032 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 85) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f407c004030 con 0x7f408415cf60 2023-12-19T17:57:56.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.150+0000 7f4079ffb640 1 --2- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4068065b80 0x7f4068068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.150+0000 7f4079ffb640 1 -- 172.21.15.26:0/3561813032 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f407c083a80 con 0x7f408415cf60 2023-12-19T17:57:56.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.150+0000 7f4089517640 1 -- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4068065b80 msgr2=0x7f4068068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:56.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.150+0000 7f4089517640 1 --2- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4068065b80 0x7f4068068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:56.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.154+0000 7f4079ffb640 1 -- 172.21.15.26:0/3561813032 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f407c050e90 con 0x7f408415cf60 2023-12-19T17:57:56.309 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:56.306+0000 7f81531f6180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:57:56.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.314+0000 7f408b77b640 1 -- 172.21.15.26:0/3561813032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f408415e670 con 0x7f408415cf60 2023-12-19T17:57:56.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.318+0000 7f4079ffb640 1 -- 172.21.15.26:0/3561813032 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+17713 (secure 0 0 0) 0x7f408415e670 con 0x7f408415cf60 2023-12-19T17:57:56.320 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:56.321 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":164,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:57:52.397105+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:57:38.188932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T17:57:39.438347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1221875420":"2023-12-20T17:57:52.397071+0000","172.21.15.160:6833/38098":"2023-12-20T17:57:52.397071+0000","172.21.15.160:6832/38098":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/1668583613":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/1848495251":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/92169616":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/2374756181":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/1748726321":"2023-12-20T17:57:52.397071+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:57:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.318+0000 7f408b77b640 1 -- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4068065b80 msgr2=0x7f4068068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.318+0000 7f408b77b640 1 --2- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4068065b80 0x7f4068068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.318+0000 7f408b77b640 1 -- 172.21.15.26:0/3561813032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f408415cf60 msgr2=0x7f408415f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.318+0000 7f408b77b640 1 --2- 172.21.15.26:0/3561813032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f408415cf60 0x7f408415f2f0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f407c007db0 tx=0x7f407c00d4a0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.318+0000 7f408b77b640 1 -- 172.21.15.26:0/3561813032 shutdown_connections 2023-12-19T17:57:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.318+0000 7f408b77b640 1 --2- 172.21.15.26:0/3561813032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f408415cf60 0x7f408415f2f0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.318+0000 7f408b77b640 1 --2- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4068065b80 0x7f4068068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.318+0000 7f408b77b640 1 --2- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40840d1420 0x7f4084142d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.318+0000 7f408b77b640 1 --2- 172.21.15.26:0/3561813032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40840b88e0 0x7f40840c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.318+0000 7f408b77b640 1 -- 172.21.15.26:0/3561813032 >> 172.21.15.26:0/3561813032 conn(0x7f4084065ee0 msgr2=0x7f40840cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.318+0000 7f408b77b640 1 -- 172.21.15.26:0/3561813032 shutdown_connections 2023-12-19T17:57:56.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.318+0000 7f408b77b640 1 -- 172.21.15.26:0/3561813032 wait complete. 2023-12-19T17:57:56.329 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:56.326+0000 7fb3e6e4d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:57:56.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T17:57:38.188932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T17:57:39.438347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '163', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T17:57:56.344 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T17:57:56.460 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:56.454+0000 7f81531f6180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:57:56.490 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:56.486+0000 7fb3e6e4d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:57:56.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.490+0000 7fcc806fc640 1 Processor -- start 2023-12-19T17:57:56.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.490+0000 7fcc806fc640 1 -- start start 2023-12-19T17:57:56.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.490+0000 7fcc806fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcc7815ccf0 0x7fcc780d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.490+0000 7fcc806fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc7815d0d0 0x7fcc780b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.490+0000 7fcc806fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc780b4600 0x7fcc780b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.490+0000 7fcc806fc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc7815e200 con 0x7fcc7815d0d0 2023-12-19T17:57:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.490+0000 7fcc806fc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc780b4f20 con 0x7fcc7815ccf0 2023-12-19T17:57:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.490+0000 7fcc806fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc780d0aa0 con 0x7fcc780b4600 2023-12-19T17:57:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.490+0000 7fcc7dc97640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc7815d0d0 0x7fcc780b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.490+0000 7fcc7dc97640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc7815d0d0 0x7fcc780b8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39782/0 (socket says 172.21.15.26:39782) 2023-12-19T17:57:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.490+0000 7fcc7dc97640 1 -- 172.21.15.26:0/159741076 learned_addr learned my addr 172.21.15.26:0/159741076 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.490+0000 7fcc7ec99640 1 --2- 172.21.15.26:0/159741076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc780b4600 0x7fcc780b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.490+0000 7fcc7e498640 1 --2- 172.21.15.26:0/159741076 >> v2:172.21.15.160:3300/0 conn(0x7fcc7815ccf0 0x7fcc780d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.494+0000 7fcc7dc97640 1 -- 172.21.15.26:0/159741076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc780b4600 msgr2=0x7fcc780b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:56.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.494+0000 7fcc7dc97640 1 --2- 172.21.15.26:0/159741076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc780b4600 0x7fcc780b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.494+0000 7fcc7dc97640 1 -- 172.21.15.26:0/159741076 >> v2:172.21.15.160:3300/0 conn(0x7fcc7815ccf0 msgr2=0x7fcc780d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:56.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.494+0000 7fcc7dc97640 1 --2- 172.21.15.26:0/159741076 >> v2:172.21.15.160:3300/0 conn(0x7fcc7815ccf0 0x7fcc780d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.494+0000 7fcc7dc97640 1 -- 172.21.15.26:0/159741076 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc780d1380 con 0x7fcc7815d0d0 2023-12-19T17:57:56.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.494+0000 7fcc7e498640 1 --2- 172.21.15.26:0/159741076 >> v2:172.21.15.160:3300/0 conn(0x7fcc7815ccf0 0x7fcc780d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.498+0000 7fcc7dc97640 1 --2- 172.21.15.26:0/159741076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc7815d0d0 0x7fcc780b8400 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcc780d3c20 tx=0x7fcc6801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4b15deb7e633af server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.498+0000 7fcc7d496640 1 -- 172.21.15.26:0/159741076 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc680aa5f0 con 0x7fcc7815d0d0 2023-12-19T17:57:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.498+0000 7fcc7d496640 1 -- 172.21.15.26:0/159741076 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc680aa750 con 0x7fcc7815d0d0 2023-12-19T17:57:56.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.498+0000 7fcc7d496640 1 -- 172.21.15.26:0/159741076 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc680aaae0 con 0x7fcc7815d0d0 2023-12-19T17:57:56.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.498+0000 7fcc806fc640 1 -- 172.21.15.26:0/159741076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc7815d0d0 msgr2=0x7fcc780b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:56.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.498+0000 7fcc806fc640 1 --2- 172.21.15.26:0/159741076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc7815d0d0 0x7fcc780b8400 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcc780d3c20 tx=0x7fcc6801c4d0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.498+0000 7fcc806fc640 1 -- 172.21.15.26:0/159741076 shutdown_connections 2023-12-19T17:57:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.498+0000 7fcc806fc640 1 --2- 172.21.15.26:0/159741076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc780b4600 0x7fcc780b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.498+0000 7fcc806fc640 1 --2- 172.21.15.26:0/159741076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc7815d0d0 0x7fcc780b8400 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.498+0000 7fcc806fc640 1 --2- 172.21.15.26:0/159741076 >> v2:172.21.15.160:3300/0 conn(0x7fcc7815ccf0 0x7fcc780d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.498+0000 7fcc806fc640 1 -- 172.21.15.26:0/159741076 >> 172.21.15.26:0/159741076 conn(0x7fcc78065ee0 msgr2=0x7fcc780cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.498+0000 7fcc806fc640 1 -- 172.21.15.26:0/159741076 shutdown_connections 2023-12-19T17:57:56.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.498+0000 7fcc806fc640 1 -- 172.21.15.26:0/159741076 wait complete. 2023-12-19T17:57:56.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc806fc640 1 Processor -- start 2023-12-19T17:57:56.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc806fc640 1 -- start start 2023-12-19T17:57:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc806fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc780b4600 0x7fcc780c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc806fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc7815ccf0 0x7fcc780b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc806fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc7815d0d0 0x7fcc78159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc806fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc781579f0 con 0x7fcc7815d0d0 2023-12-19T17:57:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc806fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc78156de0 con 0x7fcc780b4600 2023-12-19T17:57:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc806fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc781570c0 con 0x7fcc7815ccf0 2023-12-19T17:57:56.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc7ec99640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc7815d0d0 0x7fcc78159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc7e498640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc780b4600 0x7fcc780c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc7ec99640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc7815d0d0 0x7fcc78159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39790/0 (socket says 172.21.15.26:39790) 2023-12-19T17:57:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc7ec99640 1 -- 172.21.15.26:0/2798227129 learned_addr learned my addr 172.21.15.26:0/2798227129 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc7dc97640 1 --2- 172.21.15.26:0/2798227129 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc7815ccf0 0x7fcc780b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc7ec99640 1 -- 172.21.15.26:0/2798227129 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc7815ccf0 msgr2=0x7fcc780b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc7ec99640 1 --2- 172.21.15.26:0/2798227129 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc7815ccf0 0x7fcc780b6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc7ec99640 1 -- 172.21.15.26:0/2798227129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc780b4600 msgr2=0x7fcc780c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc7ec99640 1 --2- 172.21.15.26:0/2798227129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc780b4600 0x7fcc780c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc7ec99640 1 -- 172.21.15.26:0/2798227129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc74009420 con 0x7fcc7815d0d0 2023-12-19T17:57:56.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc7ec99640 1 --2- 172.21.15.26:0/2798227129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc7815d0d0 0x7fcc78159c60 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000eed0 tx=0x7fcc7000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc6effd640 1 -- 172.21.15.26:0/2798227129 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70010040 con 0x7fcc7815d0d0 2023-12-19T17:57:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc6effd640 1 -- 172.21.15.26:0/2798227129 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7000bd20 con 0x7fcc7815d0d0 2023-12-19T17:57:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc806fc640 1 -- 172.21.15.26:0/2798227129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc680094d0 con 0x7fcc7815d0d0 2023-12-19T17:57:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc6effd640 1 -- 172.21.15.26:0/2798227129 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc70014640 con 0x7fcc7815d0d0 2023-12-19T17:57:56.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.502+0000 7fcc806fc640 1 -- 172.21.15.26:0/2798227129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc78157680 con 0x7fcc7815d0d0 2023-12-19T17:57:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.506+0000 7fcc806fc640 1 -- 172.21.15.26:0/2798227129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc78074270 con 0x7fcc7815d0d0 2023-12-19T17:57:56.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.506+0000 7fcc6effd640 1 -- 172.21.15.26:0/2798227129 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 85) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fcc700147a0 con 0x7fcc7815d0d0 2023-12-19T17:57:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.506+0000 7fcc6effd640 1 --2- 172.21.15.26:0/2798227129 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcc5c065b80 0x7fcc5c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.506+0000 7fcc6effd640 1 -- 172.21.15.26:0/2798227129 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fcc70083ec0 con 0x7fcc7815d0d0 2023-12-19T17:57:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.506+0000 7fcc7e498640 1 -- 172.21.15.26:0/2798227129 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcc5c065b80 msgr2=0x7fcc5c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:56.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.506+0000 7fcc7e498640 1 --2- 172.21.15.26:0/2798227129 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcc5c065b80 0x7fcc5c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:56.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.510+0000 7fcc6effd640 1 -- 172.21.15.26:0/2798227129 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcc70051230 con 0x7fcc7815d0d0 2023-12-19T17:57:56.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc806fc640 1 -- 172.21.15.26:0/2798227129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc78157b30 con 0x7fcc7815d0d0 2023-12-19T17:57:56.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc6effd640 1 -- 172.21.15.26:0/2798227129 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v164) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc70018020 con 0x7fcc7815d0d0 2023-12-19T17:57:56.676 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T17:57:56.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc806fc640 1 -- 172.21.15.26:0/2798227129 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcc5c065b80 msgr2=0x7fcc5c068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:56.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc806fc640 1 --2- 172.21.15.26:0/2798227129 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcc5c065b80 0x7fcc5c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc806fc640 1 -- 172.21.15.26:0/2798227129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc7815d0d0 msgr2=0x7fcc78159c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc806fc640 1 --2- 172.21.15.26:0/2798227129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc7815d0d0 0x7fcc78159c60 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fcc7000eed0 tx=0x7fcc7000b480 comp rx=0 tx=0).stop 2023-12-19T17:57:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc806fc640 1 -- 172.21.15.26:0/2798227129 shutdown_connections 2023-12-19T17:57:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc806fc640 1 --2- 172.21.15.26:0/2798227129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc7815d0d0 0x7fcc78159c60 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc806fc640 1 --2- 172.21.15.26:0/2798227129 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcc5c065b80 0x7fcc5c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc806fc640 1 --2- 172.21.15.26:0/2798227129 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcc7815ccf0 0x7fcc780b6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc806fc640 1 --2- 172.21.15.26:0/2798227129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcc780b4600 0x7fcc780c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc806fc640 1 -- 172.21.15.26:0/2798227129 >> 172.21.15.26:0/2798227129 conn(0x7fcc78065ee0 msgr2=0x7fcc780b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc806fc640 1 -- 172.21.15.26:0/2798227129 shutdown_connections 2023-12-19T17:57:56.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.674+0000 7fcc806fc640 1 -- 172.21.15.26:0/2798227129 wait complete. 2023-12-19T17:57:56.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T17:57:56.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95c0118640 1 Processor -- start 2023-12-19T17:57:56.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95c0118640 1 -- start start 2023-12-19T17:57:56.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95c0118640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f95b806c3f0 0x7f95b806c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95c0118640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95b80c8df0 0x7f95b80c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95c0118640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b80b01b0 con 0x7f95b8161230 2023-12-19T17:57:56.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95c0118640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b80ca4e0 con 0x7f95b806c3f0 2023-12-19T17:57:56.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95c0118640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80b0400 con 0x7f95b80c8df0 2023-12-19T17:57:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95be6b5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95b80c8df0 0x7f95b80c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95bd6b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f95b806c3f0 0x7f95b806c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95bdeb4640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f95b8161230 0x7f95b80ada70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:52650/0 (socket says 172.21.15.26:52650) 2023-12-19T17:57:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95bdeb4640 1 -- 172.21.15.26:0/1063722634 learned_addr learned my addr 172.21.15.26:0/1063722634 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95be6b5640 1 -- 172.21.15.26:0/1063722634 >> v2:172.21.15.160:3300/0 conn(0x7f95b806c3f0 msgr2=0x7f95b806c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95be6b5640 1 --2- 172.21.15.26:0/1063722634 >> v2:172.21.15.160:3300/0 conn(0x7f95b806c3f0 0x7f95b806c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95be6b5640 1 -- 172.21.15.26:0/1063722634 >> v1:172.21.15.160:6789/0 conn(0x7f95b8161230 legacy=0x7f95b80ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95be6b5640 1 -- 172.21.15.26:0/1063722634 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b80b06e0 con 0x7f95b80c8df0 2023-12-19T17:57:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95bceb2640 1 -- 172.21.15.26:0/1063722634 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378047335 0 0) 0x7f95b80b01b0 con 0x7f95b8161230 2023-12-19T17:57:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.846+0000 7f95bd6b3640 1 --2- 172.21.15.26:0/1063722634 >> v2:172.21.15.160:3300/0 conn(0x7f95b806c3f0 0x7f95b806c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:56.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.850+0000 7f95be6b5640 1 --2- 172.21.15.26:0/1063722634 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95b80c8df0 0x7f95b80c9280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009d90 tx=0x7f95b001c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53edde537b885b53 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:56.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.850+0000 7f95bceb2640 1 -- 172.21.15.26:0/1063722634 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b00a5970 con 0x7f95b80c8df0 2023-12-19T17:57:56.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.850+0000 7f95bceb2640 1 -- 172.21.15.26:0/1063722634 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b00a5ad0 con 0x7f95b80c8df0 2023-12-19T17:57:56.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.850+0000 7f95bceb2640 1 -- 172.21.15.26:0/1063722634 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b00a5e60 con 0x7f95b80c8df0 2023-12-19T17:57:56.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.850+0000 7f95c0118640 1 -- 172.21.15.26:0/1063722634 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95b80c8df0 msgr2=0x7f95b80c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:56.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.850+0000 7f95c0118640 1 --2- 172.21.15.26:0/1063722634 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95b80c8df0 0x7f95b80c9280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009d90 tx=0x7f95b001c890 comp rx=0 tx=0).stop 2023-12-19T17:57:56.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.850+0000 7f95c0118640 1 -- 172.21.15.26:0/1063722634 shutdown_connections 2023-12-19T17:57:56.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.850+0000 7f95c0118640 1 --2- 172.21.15.26:0/1063722634 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95b80c8df0 0x7f95b80c9280 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.850+0000 7f95c0118640 1 --2- 172.21.15.26:0/1063722634 >> v2:172.21.15.160:3300/0 conn(0x7f95b806c3f0 0x7f95b806c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.850+0000 7f95c0118640 1 -- 172.21.15.26:0/1063722634 >> 172.21.15.26:0/1063722634 conn(0x7f95b80ce730 msgr2=0x7f95b80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:56.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.850+0000 7f95c0118640 1 -- 172.21.15.26:0/1063722634 shutdown_connections 2023-12-19T17:57:56.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.850+0000 7f95c0118640 1 -- 172.21.15.26:0/1063722634 wait complete. 2023-12-19T17:57:56.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95c0118640 1 Processor -- start 2023-12-19T17:57:56.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95c0118640 1 -- start start 2023-12-19T17:57:56.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95c0118640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b806c3f0 0x7f95b806fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95c0118640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95b80c8df0 0x7f95b815cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95c0118640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95b8161230 0x7f95b81562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95c0118640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80c3900 con 0x7f95b806c3f0 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95c0118640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80c3be0 con 0x7f95b80c8df0 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95c0118640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80c3ec0 con 0x7f95b8161230 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95bdeb4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b806c3f0 0x7f95b806fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95bdeb4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b806c3f0 0x7f95b806fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39800/0 (socket says 172.21.15.26:39800) 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95bdeb4640 1 -- 172.21.15.26:0/209309710 learned_addr learned my addr 172.21.15.26:0/209309710 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95be6b5640 1 --2- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95b8161230 0x7f95b81562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95bd6b3640 1 --2- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95b80c8df0 0x7f95b815cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95bdeb4640 1 -- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95b8161230 msgr2=0x7f95b81562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95bdeb4640 1 --2- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95b8161230 0x7f95b81562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95bdeb4640 1 -- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95b80c8df0 msgr2=0x7f95b815cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:56.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95bdeb4640 1 --2- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95b80c8df0 0x7f95b815cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:56.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95bdeb4640 1 -- 172.21.15.26:0/209309710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a8009420 con 0x7f95b806c3f0 2023-12-19T17:57:56.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95bdeb4640 1 --2- 172.21.15.26:0/209309710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b806c3f0 0x7f95b806fae0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f95b400c480 tx=0x7f95b400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:56.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95ae7fc640 1 -- 172.21.15.26:0/209309710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b40074e0 con 0x7f95b806c3f0 2023-12-19T17:57:56.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95ae7fc640 1 -- 172.21.15.26:0/209309710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b4007660 con 0x7f95b806c3f0 2023-12-19T17:57:56.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95ae7fc640 1 -- 172.21.15.26:0/209309710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b4004a30 con 0x7f95b806c3f0 2023-12-19T17:57:56.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95c0118640 1 -- 172.21.15.26:0/209309710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b0009a70 con 0x7f95b806c3f0 2023-12-19T17:57:56.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.854+0000 7f95c0118640 1 -- 172.21.15.26:0/209309710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b8151180 con 0x7f95b806c3f0 2023-12-19T17:57:56.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.858+0000 7f95c0118640 1 -- 172.21.15.26:0/209309710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b80c46c0 con 0x7f95b806c3f0 2023-12-19T17:57:56.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.858+0000 7f95ae7fc640 1 -- 172.21.15.26:0/209309710 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 85) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f95b4004b90 con 0x7f95b806c3f0 2023-12-19T17:57:56.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.858+0000 7f95ae7fc640 1 --2- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f959c065b80 0x7f959c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:56.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.858+0000 7f95ae7fc640 1 -- 172.21.15.26:0/209309710 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f95b40827d0 con 0x7f95b806c3f0 2023-12-19T17:57:56.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.858+0000 7f95bd6b3640 1 -- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f959c065b80 msgr2=0x7f959c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:56.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.858+0000 7f95bd6b3640 1 --2- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f959c065b80 0x7f959c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:56.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:56.862+0000 7f95ae7fc640 1 -- 172.21.15.26:0/209309710 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f95b80c46c0 con 0x7f95b806c3f0 2023-12-19T17:57:56.955 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:56.950+0000 7f81531f6180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:57:57.014 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:57.010+0000 7fb3e6e4d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:57:57.014 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:57.010+0000 7f81531f6180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:57:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.022+0000 7f95c0118640 1 -- 172.21.15.26:0/209309710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95b80c46c0 con 0x7f95b806c3f0 2023-12-19T17:57:57.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.022+0000 7f95ae7fc640 1 -- 172.21.15.26:0/209309710 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v164) v1 ==== 104+0+11 (secure 0 0 0) 0x7f95b4018020 con 0x7f95b806c3f0 2023-12-19T17:57:57.025 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T17:57:57.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.022+0000 7f95c0118640 1 -- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f959c065b80 msgr2=0x7f959c068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:57.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.022+0000 7f95c0118640 1 --2- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f959c065b80 0x7f959c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.022+0000 7f95c0118640 1 -- 172.21.15.26:0/209309710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b806c3f0 msgr2=0x7f95b806fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.022+0000 7f95c0118640 1 --2- 172.21.15.26:0/209309710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b806c3f0 0x7f95b806fae0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f95b400c480 tx=0x7f95b400c940 comp rx=0 tx=0).stop 2023-12-19T17:57:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.022+0000 7f95c0118640 1 -- 172.21.15.26:0/209309710 shutdown_connections 2023-12-19T17:57:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.022+0000 7f95c0118640 1 --2- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f959c065b80 0x7f959c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.022+0000 7f95c0118640 1 --2- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95b8161230 0x7f95b81562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.022+0000 7f95c0118640 1 --2- 172.21.15.26:0/209309710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95b80c8df0 0x7f95b815cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.022+0000 7f95c0118640 1 --2- 172.21.15.26:0/209309710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95b806c3f0 0x7f95b806fae0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.022+0000 7f95c0118640 1 -- 172.21.15.26:0/209309710 >> 172.21.15.26:0/209309710 conn(0x7f95b80ce730 msgr2=0x7f95b80ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.026+0000 7f95c0118640 1 -- 172.21.15.26:0/209309710 shutdown_connections 2023-12-19T17:57:57.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.026+0000 7f95c0118640 1 -- 172.21.15.26:0/209309710 wait complete. 2023-12-19T17:57:57.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T17:57:57.071 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:57.066+0000 7f81531f6180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:57:57.079 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:57.078+0000 7fb3e6e4d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:57:57.139 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:57.138+0000 7fb3e6e4d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:57:57.158 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:57.154+0000 7f81531f6180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:57:57.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.194+0000 7fae449bc640 1 Processor -- start 2023-12-19T17:57:57.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae449bc640 1 -- start start 2023-12-19T17:57:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae449bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4015a9c0 0x7fae400706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae449bc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fae4015ada0 0x7fae400b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae449bc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae400b46c0 con 0x7fae4006ac80 2023-12-19T17:57:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae449bc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae400cc270 con 0x7fae4015ada0 2023-12-19T17:57:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae449bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae40070d90 con 0x7fae4015a9c0 2023-12-19T17:57:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae3ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae4006ac80 0x7fae400b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44748/0 (socket says 172.21.15.26:44748) 2023-12-19T17:57:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae3ed9d640 1 -- 172.21.15.26:0/569251543 learned_addr learned my addr 172.21.15.26:0/569251543 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae3dd9b640 1 --2- 172.21.15.26:0/569251543 >> v2:172.21.15.160:3300/0 conn(0x7fae4015ada0 0x7fae400b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:57.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae3e59c640 1 --2- 172.21.15.26:0/569251543 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4015a9c0 0x7fae400706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae3d59a640 1 -- 172.21.15.26:0/569251543 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096342068 0 0) 0x7fae400b46c0 con 0x7fae4006ac80 2023-12-19T17:57:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae3d59a640 1 -- 172.21.15.26:0/569251543 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae28003530 con 0x7fae4006ac80 2023-12-19T17:57:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae3dd9b640 1 -- 172.21.15.26:0/569251543 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4015a9c0 msgr2=0x7fae400706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae3dd9b640 1 --2- 172.21.15.26:0/569251543 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4015a9c0 0x7fae400706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae3dd9b640 1 -- 172.21.15.26:0/569251543 >> v1:172.21.15.26:6789/0 conn(0x7fae4006ac80 legacy=0x7fae400b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:57.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.198+0000 7fae3dd9b640 1 -- 172.21.15.26:0/569251543 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae40071070 con 0x7fae4015ada0 2023-12-19T17:57:57.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.202+0000 7fae3dd9b640 1 --2- 172.21.15.26:0/569251543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4015ada0 0x7fae400b4180 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fae340097c0 tx=0x7fae3401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9381b8b685072b1a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.202+0000 7fae3d59a640 1 -- 172.21.15.26:0/569251543 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae340ac620 con 0x7fae4015ada0 2023-12-19T17:57:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.202+0000 7fae3d59a640 1 -- 172.21.15.26:0/569251543 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae340ac780 con 0x7fae4015ada0 2023-12-19T17:57:57.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.202+0000 7fae3d59a640 1 -- 172.21.15.26:0/569251543 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae340acaf0 con 0x7fae4015ada0 2023-12-19T17:57:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.202+0000 7fae449bc640 1 -- 172.21.15.26:0/569251543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4015ada0 msgr2=0x7fae400b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.202+0000 7fae449bc640 1 --2- 172.21.15.26:0/569251543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4015ada0 0x7fae400b4180 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fae340097c0 tx=0x7fae3401c3c0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.202+0000 7fae449bc640 1 -- 172.21.15.26:0/569251543 shutdown_connections 2023-12-19T17:57:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.202+0000 7fae449bc640 1 --2- 172.21.15.26:0/569251543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4015ada0 0x7fae400b4180 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.202+0000 7fae449bc640 1 --2- 172.21.15.26:0/569251543 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4015a9c0 0x7fae400706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.202+0000 7fae449bc640 1 -- 172.21.15.26:0/569251543 >> 172.21.15.26:0/569251543 conn(0x7fae400ce730 msgr2=0x7fae400ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.202+0000 7fae449bc640 1 -- 172.21.15.26:0/569251543 shutdown_connections 2023-12-19T17:57:57.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.202+0000 7fae449bc640 1 -- 172.21.15.26:0/569251543 wait complete. 2023-12-19T17:57:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae449bc640 1 Processor -- start 2023-12-19T17:57:57.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae449bc640 1 -- start start 2023-12-19T17:57:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae449bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4006ac80 0x7fae401513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae449bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4015a9c0 0x7fae400ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae449bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4015ada0 0x7fae400aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae449bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae401532d0 con 0x7fae4015a9c0 2023-12-19T17:57:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae449bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae401535b0 con 0x7fae4015ada0 2023-12-19T17:57:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae449bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae40153890 con 0x7fae4006ac80 2023-12-19T17:57:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae3e59c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4006ac80 0x7fae401513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae3e59c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4006ac80 0x7fae401513c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:43420/0 (socket says 172.21.15.26:43420) 2023-12-19T17:57:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae3e59c640 1 -- 172.21.15.26:0/578312107 learned_addr learned my addr 172.21.15.26:0/578312107 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae3dd9b640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4015a9c0 0x7fae400ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:57.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae3ed9d640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4015ada0 0x7fae400aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:57.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae3e59c640 1 -- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4015ada0 msgr2=0x7fae400aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:57.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae3e59c640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4015ada0 0x7fae400aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae3e59c640 1 -- 172.21.15.26:0/578312107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4015a9c0 msgr2=0x7fae400ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:57.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae3e59c640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4015a9c0 0x7fae400ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae3e59c640 1 -- 172.21.15.26:0/578312107 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae34009420 con 0x7fae4006ac80 2023-12-19T17:57:57.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae3dd9b640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4015a9c0 0x7fae400ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:57.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae3e59c640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4006ac80 0x7fae401513c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fae3000e930 tx=0x7fae3000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:57.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae2effd640 1 -- 172.21.15.26:0/578312107 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30009690 con 0x7fae4006ac80 2023-12-19T17:57:57.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae2effd640 1 -- 172.21.15.26:0/578312107 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3000ca00 con 0x7fae4006ac80 2023-12-19T17:57:57.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae2effd640 1 -- 172.21.15.26:0/578312107 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3000cd70 con 0x7fae4006ac80 2023-12-19T17:57:57.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae449bc640 1 -- 172.21.15.26:0/578312107 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae40153b70 con 0x7fae4006ac80 2023-12-19T17:57:57.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae449bc640 1 -- 172.21.15.26:0/578312107 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae40151900 con 0x7fae4006ac80 2023-12-19T17:57:57.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.206+0000 7fae449bc640 1 -- 172.21.15.26:0/578312107 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae40074270 con 0x7fae4006ac80 2023-12-19T17:57:57.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.210+0000 7fae2effd640 1 -- 172.21.15.26:0/578312107 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 85) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fae300059a0 con 0x7fae4006ac80 2023-12-19T17:57:57.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.210+0000 7fae2effd640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae20065b80 0x7fae20068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:57.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.210+0000 7fae2effd640 1 -- 172.21.15.26:0/578312107 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fae30083980 con 0x7fae4006ac80 2023-12-19T17:57:57.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.210+0000 7fae3dd9b640 1 -- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae20065b80 msgr2=0x7fae20068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:57.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.210+0000 7fae3dd9b640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae20065b80 0x7fae20068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:57.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.218+0000 7fae2effd640 1 -- 172.21.15.26:0/578312107 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fae300510d0 con 0x7fae4006ac80 2023-12-19T17:57:57.221 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:57.218+0000 7f81531f6180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:57:57.234 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:57.230+0000 7fb3e6e4d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:57:57.295 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:57.290+0000 7fb3e6e4d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:57:57.307 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:57.302+0000 7f81531f6180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:57:57.363 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:57.358+0000 7f81531f6180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:57:57.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae449bc640 1 -- 172.21.15.26:0/578312107 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae400cd7b0 con 0x7fae4006ac80 2023-12-19T17:57:57.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae2effd640 1 -- 172.21.15.26:0/578312107 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v164) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae30017070 con 0x7fae4006ac80 2023-12-19T17:57:57.380 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T17:57:57.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae449bc640 1 -- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae20065b80 msgr2=0x7fae20068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:57.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae449bc640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae20065b80 0x7fae20068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae449bc640 1 -- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4006ac80 msgr2=0x7fae401513c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:57.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae449bc640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4006ac80 0x7fae401513c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fae3000e930 tx=0x7fae3000edf0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae449bc640 1 -- 172.21.15.26:0/578312107 shutdown_connections 2023-12-19T17:57:57.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae449bc640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae4015ada0 0x7fae400aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae449bc640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae4015a9c0 0x7fae400ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae449bc640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fae20065b80 0x7fae20068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae449bc640 1 --2- 172.21.15.26:0/578312107 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae4006ac80 0x7fae401513c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae449bc640 1 -- 172.21.15.26:0/578312107 >> 172.21.15.26:0/578312107 conn(0x7fae400ce730 msgr2=0x7fae4006b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:57.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae449bc640 1 -- 172.21.15.26:0/578312107 shutdown_connections 2023-12-19T17:57:57.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.378+0000 7fae449bc640 1 -- 172.21.15.26:0/578312107 wait complete. 2023-12-19T17:57:57.384 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:57.382+0000 7fb3e6e4d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:57:57.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T17:57:57.446 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:57.442+0000 7fb3e6e4d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:57:57.511 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:57.506+0000 7f81531f6180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:57:57.570 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:57.566+0000 7f81531f6180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:57:57.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1e11640 1 Processor -- start 2023-12-19T17:57:57.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1e11640 1 -- start start 2023-12-19T17:57:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1e11640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f38ec15cbf0 0x7f38ec0b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1e11640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38ec0d1a60 0x7f38ec0b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1e11640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec15cfd0 con 0x7f38ec0b4420 2023-12-19T17:57:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1e11640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec0b4df0 con 0x7f38ec0d1a60 2023-12-19T17:57:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1e11640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec0b6fc0 con 0x7f38ec15cbf0 2023-12-19T17:57:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1610640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38ec0d1a60 0x7f38ec0b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1610640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38ec0d1a60 0x7f38ec0b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39824/0 (socket says 172.21.15.26:39824) 2023-12-19T17:57:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1610640 1 -- 172.21.15.26:0/183282076 learned_addr learned my addr 172.21.15.26:0/183282076 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f0e0f640 1 --2- 172.21.15.26:0/183282076 >> v2:172.21.15.160:3300/0 conn(0x7f38ec15cbf0 0x7f38ec0b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1610640 1 -- 172.21.15.26:0/183282076 >> v2:172.21.15.160:3300/0 conn(0x7f38ec15cbf0 msgr2=0x7f38ec0b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1610640 1 --2- 172.21.15.26:0/183282076 >> v2:172.21.15.160:3300/0 conn(0x7f38ec15cbf0 0x7f38ec0b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1610640 1 -- 172.21.15.26:0/183282076 >> v1:172.21.15.26:6789/0 conn(0x7f38ec0b4420 legacy=0x7f38ec0b5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:57.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.590+0000 7f38f1610640 1 -- 172.21.15.26:0/183282076 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec0b72a0 con 0x7f38ec0d1a60 2023-12-19T17:57:57.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.594+0000 7f38f1610640 1 --2- 172.21.15.26:0/183282076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38ec0d1a60 0x7f38ec0b48b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f38ec073840 tx=0x7f38e401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df30ddd905a9e2cd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:57.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.594+0000 7f38eb7fe640 1 -- 172.21.15.26:0/183282076 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e40a5800 con 0x7f38ec0d1a60 2023-12-19T17:57:57.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.594+0000 7f38eb7fe640 1 -- 172.21.15.26:0/183282076 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e40a5960 con 0x7f38ec0d1a60 2023-12-19T17:57:57.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.594+0000 7f38eb7fe640 1 -- 172.21.15.26:0/183282076 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e40a5cf0 con 0x7f38ec0d1a60 2023-12-19T17:57:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.594+0000 7f38f1e11640 1 -- 172.21.15.26:0/183282076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38ec0d1a60 msgr2=0x7f38ec0b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.594+0000 7f38f1e11640 1 --2- 172.21.15.26:0/183282076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38ec0d1a60 0x7f38ec0b48b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f38ec073840 tx=0x7f38e401c940 comp rx=0 tx=0).stop 2023-12-19T17:57:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.594+0000 7f38f1e11640 1 -- 172.21.15.26:0/183282076 shutdown_connections 2023-12-19T17:57:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.594+0000 7f38f1e11640 1 --2- 172.21.15.26:0/183282076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38ec0d1a60 0x7f38ec0b48b0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.594+0000 7f38f1e11640 1 --2- 172.21.15.26:0/183282076 >> v2:172.21.15.160:3300/0 conn(0x7f38ec15cbf0 0x7f38ec0b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.594+0000 7f38f1e11640 1 -- 172.21.15.26:0/183282076 >> 172.21.15.26:0/183282076 conn(0x7f38ec06a120 msgr2=0x7f38ec06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.594+0000 7f38f1e11640 1 -- 172.21.15.26:0/183282076 shutdown_connections 2023-12-19T17:57:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.594+0000 7f38f1e11640 1 -- 172.21.15.26:0/183282076 wait complete. 2023-12-19T17:57:57.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38f1e11640 1 Processor -- start 2023-12-19T17:57:57.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38f1e11640 1 -- start start 2023-12-19T17:57:57.601 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:57.598+0000 7fb3e6e4d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:57:57.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38f1e11640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38ec0b4420 0x7f38ec0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38f1e11640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38ec0d1a60 0x7f38ec1f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38f1e11640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38ec15cbf0 0x7f38ec14c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38f1e11640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0c9050 con 0x7f38ec0d1a60 2023-12-19T17:57:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38f1e11640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0ca9f0 con 0x7f38ec15cbf0 2023-12-19T17:57:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38f1e11640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0c9e70 con 0x7f38ec0b4420 2023-12-19T17:57:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38ebfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38ec0d1a60 0x7f38ec1f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38f1610640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38ec15cbf0 0x7f38ec14c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38f0e0f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38ec0b4420 0x7f38ec0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38ebfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38ec0d1a60 0x7f38ec1f8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39838/0 (socket says 172.21.15.26:39838) 2023-12-19T17:57:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38ebfff640 1 -- 172.21.15.26:0/2766188014 learned_addr learned my addr 172.21.15.26:0/2766188014 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38ebfff640 1 -- 172.21.15.26:0/2766188014 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38ec0b4420 msgr2=0x7f38ec0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38ebfff640 1 --2- 172.21.15.26:0/2766188014 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38ec0b4420 0x7f38ec0c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38ebfff640 1 -- 172.21.15.26:0/2766188014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38ec15cbf0 msgr2=0x7f38ec14c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38ebfff640 1 --2- 172.21.15.26:0/2766188014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38ec15cbf0 0x7f38ec14c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38ebfff640 1 -- 172.21.15.26:0/2766188014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e4009b30 con 0x7f38ec0d1a60 2023-12-19T17:57:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38f1610640 1 --2- 172.21.15.26:0/2766188014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38ec15cbf0 0x7f38ec14c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:57.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38ebfff640 1 --2- 172.21.15.26:0/2766188014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38ec0d1a60 0x7f38ec1f8be0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f38d800c500 tx=0x7f38d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:57.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38e97fa640 1 -- 172.21.15.26:0/2766188014 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8005220 con 0x7f38ec0d1a60 2023-12-19T17:57:57.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.602+0000 7f38f1e11640 1 -- 172.21.15.26:0/2766188014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec0ca150 con 0x7f38ec0d1a60 2023-12-19T17:57:57.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.598+0000 7f38e97fa640 1 -- 172.21.15.26:0/2766188014 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8005380 con 0x7f38ec0d1a60 2023-12-19T17:57:57.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.602+0000 7f38e97fa640 1 -- 172.21.15.26:0/2766188014 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8007270 con 0x7f38ec0d1a60 2023-12-19T17:57:57.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.602+0000 7f38f1e11640 1 -- 172.21.15.26:0/2766188014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec0ca710 con 0x7f38ec0d1a60 2023-12-19T17:57:57.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.602+0000 7f38f1e11640 1 -- 172.21.15.26:0/2766188014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec067400 con 0x7f38ec0d1a60 2023-12-19T17:57:57.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.602+0000 7f38e97fa640 1 -- 172.21.15.26:0/2766188014 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 85) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f38d80073d0 con 0x7f38ec0d1a60 2023-12-19T17:57:57.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.602+0000 7f38e97fa640 1 --2- 172.21.15.26:0/2766188014 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f38d0065bb0 0x7f38d0068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:57.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.606+0000 7f38e97fa640 1 -- 172.21.15.26:0/2766188014 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f38d8082f00 con 0x7f38ec0d1a60 2023-12-19T17:57:57.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.606+0000 7f38f0e0f640 1 -- 172.21.15.26:0/2766188014 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f38d0065bb0 msgr2=0x7f38d0068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:57.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.606+0000 7f38f0e0f640 1 --2- 172.21.15.26:0/2766188014 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f38d0065bb0 0x7f38d0068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:57.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.610+0000 7f38e97fa640 1 -- 172.21.15.26:0/2766188014 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f38d8050390 con 0x7f38ec0d1a60 2023-12-19T17:57:57.669 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:57.666+0000 7fb3e6e4d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:57:57.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.782+0000 7f38f1e11640 1 -- 172.21.15.26:0/2766188014 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f38ec0d0ee0 con 0x7f38ec0d1a60 2023-12-19T17:57:57.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.786+0000 7f38e97fa640 1 -- 172.21.15.26:0/2766188014 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v164) v1 ==== 80+0+466 (secure 0 0 0) 0x7f38d80539b0 con 0x7f38ec0d1a60 2023-12-19T17:57:57.788 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1221875420 2023-12-20T17:57:52.397071+0000 2023-12-19T17:57:57.789 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6833/38098 2023-12-20T17:57:52.397071+0000 2023-12-19T17:57:57.789 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6832/38098 2023-12-20T17:57:52.397071+0000 2023-12-19T17:57:57.789 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1668583613 2023-12-20T17:57:52.397071+0000 2023-12-19T17:57:57.789 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1848495251 2023-12-20T17:57:52.397071+0000 2023-12-19T17:57:57.789 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/92169616 2023-12-20T17:57:52.397071+0000 2023-12-19T17:57:57.789 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2374756181 2023-12-20T17:57:52.397071+0000 2023-12-19T17:57:57.789 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1748726321 2023-12-20T17:57:52.397071+0000 2023-12-19T17:57:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:listed 8 entries 2023-12-19T17:57:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.786+0000 7f38f1e11640 1 -- 172.21.15.26:0/2766188014 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f38d0065bb0 msgr2=0x7f38d0068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.786+0000 7f38f1e11640 1 --2- 172.21.15.26:0/2766188014 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f38d0065bb0 0x7f38d0068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.786+0000 7f38f1e11640 1 -- 172.21.15.26:0/2766188014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38ec0d1a60 msgr2=0x7f38ec1f8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.786+0000 7f38f1e11640 1 --2- 172.21.15.26:0/2766188014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38ec0d1a60 0x7f38ec1f8be0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f38d800c500 tx=0x7f38d800c9c0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.786+0000 7f38f1e11640 1 -- 172.21.15.26:0/2766188014 shutdown_connections 2023-12-19T17:57:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.786+0000 7f38f1e11640 1 --2- 172.21.15.26:0/2766188014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38ec15cbf0 0x7f38ec14c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.786+0000 7f38f1e11640 1 --2- 172.21.15.26:0/2766188014 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38ec0d1a60 0x7f38ec1f8be0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.786+0000 7f38f1e11640 1 --2- 172.21.15.26:0/2766188014 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f38d0065bb0 0x7f38d0068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.786+0000 7f38f1e11640 1 --2- 172.21.15.26:0/2766188014 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38ec0b4420 0x7f38ec0c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:57.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.786+0000 7f38f1e11640 1 -- 172.21.15.26:0/2766188014 >> 172.21.15.26:0/2766188014 conn(0x7f38ec06a120 msgr2=0x7f38ec0d1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:57.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.786+0000 7f38f1e11640 1 -- 172.21.15.26:0/2766188014 shutdown_connections 2023-12-19T17:57:57.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:57.786+0000 7f38f1e11640 1 -- 172.21.15.26:0/2766188014 wait complete. 2023-12-19T17:57:57.810 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T17:57:57.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:57.811 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-19T17:57:57.816 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:57:57.816 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:57:57.816 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:57:57.824 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:57.822+0000 7f81531f6180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:57:57.870 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T17:57:57.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:57:57.870 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T17:57:57.893 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T17:57:57.893 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:57.902 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T17:57:57.902 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:57:57.902 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:57.921 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:57:57.921 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:57:57.922 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:57:57.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T17:57:57.931 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T17:57:57.931 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:57:57.931 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:57.926+0000 7fb3e6e4d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:57:57.952 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T17:57:57.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:57:57.963 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:57.958+0000 7f81531f6180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:57:58.075 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:58.070+0000 7fb3e6e4d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:57:58.090 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:58.086+0000 7f81531f6180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:57:58.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd898b6640 1 Processor -- start 2023-12-19T17:57:58.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd898b6640 1 -- start start 2023-12-19T17:57:58.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd898b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd8415d260 0x7fbd8406e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd898b6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd840b34f0 0x7fbd840b3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd898b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd840b3060 con 0x7fbd840658e0 2023-12-19T17:57:58.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd898b6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd8415d820 con 0x7fbd840b34f0 2023-12-19T17:57:58.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd898b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8415db00 con 0x7fbd8415d260 2023-12-19T17:57:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd82ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbd840658e0 0x7fbd84065cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44776/0 (socket says 172.21.15.26:44776) 2023-12-19T17:57:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd82ffd640 1 -- 172.21.15.26:0/204678701 learned_addr learned my addr 172.21.15.26:0/204678701 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd837fe640 1 --2- 172.21.15.26:0/204678701 >> v2:172.21.15.26:3300/0 conn(0x7fbd840b34f0 0x7fbd840b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd827fc640 1 --2- 172.21.15.26:0/204678701 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd8415d260 0x7fbd8406e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd81ffb640 1 -- 172.21.15.26:0/204678701 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211235593 0 0) 0x7fbd840b3060 con 0x7fbd840658e0 2023-12-19T17:57:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd81ffb640 1 -- 172.21.15.26:0/204678701 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd6c003530 con 0x7fbd840658e0 2023-12-19T17:57:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd837fe640 1 -- 172.21.15.26:0/204678701 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd8415d260 msgr2=0x7fbd8406e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd837fe640 1 --2- 172.21.15.26:0/204678701 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd8415d260 0x7fbd8406e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd837fe640 1 -- 172.21.15.26:0/204678701 >> v1:172.21.15.26:6789/0 conn(0x7fbd840658e0 legacy=0x7fbd84065cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.114+0000 7fbd837fe640 1 -- 172.21.15.26:0/204678701 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8415dde0 con 0x7fbd840b34f0 2023-12-19T17:57:58.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.118+0000 7fbd837fe640 1 --2- 172.21.15.26:0/204678701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840b34f0 0x7fbd840b3980 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009d90 tx=0x7fbd7c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51f6dbfb042649a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:58.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.118+0000 7fbd81ffb640 1 -- 172.21.15.26:0/204678701 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0a58f0 con 0x7fbd840b34f0 2023-12-19T17:57:58.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.118+0000 7fbd81ffb640 1 -- 172.21.15.26:0/204678701 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c0a5a50 con 0x7fbd840b34f0 2023-12-19T17:57:58.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.118+0000 7fbd81ffb640 1 -- 172.21.15.26:0/204678701 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0a5dc0 con 0x7fbd840b34f0 2023-12-19T17:57:58.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.118+0000 7fbd898b6640 1 -- 172.21.15.26:0/204678701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840b34f0 msgr2=0x7fbd840b3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.118+0000 7fbd898b6640 1 --2- 172.21.15.26:0/204678701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840b34f0 0x7fbd840b3980 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009d90 tx=0x7fbd7c01c920 comp rx=0 tx=0).stop 2023-12-19T17:57:58.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.118+0000 7fbd898b6640 1 -- 172.21.15.26:0/204678701 shutdown_connections 2023-12-19T17:57:58.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.118+0000 7fbd898b6640 1 --2- 172.21.15.26:0/204678701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840b34f0 0x7fbd840b3980 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.118+0000 7fbd898b6640 1 --2- 172.21.15.26:0/204678701 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd8415d260 0x7fbd8406e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.118+0000 7fbd898b6640 1 -- 172.21.15.26:0/204678701 >> 172.21.15.26:0/204678701 conn(0x7fbd8406a320 msgr2=0x7fbd840b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.118+0000 7fbd898b6640 1 -- 172.21.15.26:0/204678701 shutdown_connections 2023-12-19T17:57:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.118+0000 7fbd898b6640 1 -- 172.21.15.26:0/204678701 wait complete. 2023-12-19T17:57:58.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd898b6640 1 Processor -- start 2023-12-19T17:57:58.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd898b6640 1 -- start start 2023-12-19T17:57:58.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd898b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd840658e0 0x7fbd84070850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd898b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840b34f0 0x7fbd8415f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd898b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd8415d260 0x7fbd8415fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd898b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd841447a0 con 0x7fbd840b34f0 2023-12-19T17:57:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd898b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd84144a80 con 0x7fbd840658e0 2023-12-19T17:57:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd898b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd84144d60 con 0x7fbd8415d260 2023-12-19T17:57:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840b34f0 0x7fbd8415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840b34f0 0x7fbd8415f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39852/0 (socket says 172.21.15.26:39852) 2023-12-19T17:57:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd827fc640 1 -- 172.21.15.26:0/2131799895 learned_addr learned my addr 172.21.15.26:0/2131799895 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd82ffd640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd840658e0 0x7fbd84070850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd837fe640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd8415d260 0x7fbd8415fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd827fc640 1 -- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd8415d260 msgr2=0x7fbd8415fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd827fc640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd8415d260 0x7fbd8415fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd827fc640 1 -- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd840658e0 msgr2=0x7fbd84070850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd827fc640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd840658e0 0x7fbd84070850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd827fc640 1 -- 172.21.15.26:0/2131799895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c009a70 con 0x7fbd840b34f0 2023-12-19T17:57:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd837fe640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd8415d260 0x7fbd8415fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd827fc640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840b34f0 0x7fbd8415f790 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800e870 tx=0x7fbd7800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd6bfff640 1 -- 172.21.15.26:0/2131799895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78009510 con 0x7fbd840b34f0 2023-12-19T17:57:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd6bfff640 1 -- 172.21.15.26:0/2131799895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7800eec0 con 0x7fbd840b34f0 2023-12-19T17:57:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd898b6640 1 -- 172.21.15.26:0/2131799895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd84145040 con 0x7fbd840b34f0 2023-12-19T17:57:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd6bfff640 1 -- 172.21.15.26:0/2131799895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7800cb40 con 0x7fbd840b34f0 2023-12-19T17:57:58.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.122+0000 7fbd898b6640 1 -- 172.21.15.26:0/2131799895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd84145840 con 0x7fbd840b34f0 2023-12-19T17:57:58.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.126+0000 7fbd898b6640 1 -- 172.21.15.26:0/2131799895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd84074270 con 0x7fbd840b34f0 2023-12-19T17:57:58.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.126+0000 7fbd6bfff640 1 -- 172.21.15.26:0/2131799895 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 85) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fbd7800cca0 con 0x7fbd840b34f0 2023-12-19T17:57:58.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.126+0000 7fbd6bfff640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd70065b80 0x7fbd70068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.126+0000 7fbd6bfff640 1 -- 172.21.15.26:0/2131799895 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6785+0+0 (secure 0 0 0) 0x7fbd78083850 con 0x7fbd840b34f0 2023-12-19T17:57:58.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.126+0000 7fbd82ffd640 1 -- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd70065b80 msgr2=0x7fbd70068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:58.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.126+0000 7fbd82ffd640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd70065b80 0x7fbd70068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:58.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.130+0000 7fbd6bfff640 1 -- 172.21.15.26:0/2131799895 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbd78050cd0 con 0x7fbd840b34f0 2023-12-19T17:57:58.162 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:58.158+0000 7f81531f6180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:57:58.210 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:58.206+0000 7fb3e6e4d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:57:58.284 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:58.282+0000 7fb3e6e4d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:57:58.299 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:58.294+0000 7f81531f6180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:57:58.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.322+0000 7fbd898b6640 1 -- 172.21.15.26:0/2131799895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd840b12a0 con 0x7fbd840b34f0 2023-12-19T17:57:58.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.322+0000 7fbd6bfff640 1 -- 172.21.15.26:0/2131799895 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fbd780542f0 con 0x7fbd840b34f0 2023-12-19T17:57:58.326 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:58.326 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2850013418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2850013418},{"type":"v1","addr":"172.21.15.26:6835","nonce":2850013418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2213513866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2213513866},{"type":"v1","addr":"172.21.15.160:6836","nonce":2213513866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4292779848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4292779848},{"type":"v1","addr":"172.21.15.160:6837","nonce":4292779848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:57:40.517697+0000","modified":"2023-12-19T17:57:43.536734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8575":{"gid":8575,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/2838422258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2838422258},{"type":"v1","addr":"172.21.15.26:6834","nonce":2838422258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-19T17:57:58.326 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 73 2023-12-19T17:57:58.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.326+0000 7fbd898b6640 1 -- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd70065b80 msgr2=0x7fbd70068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.326+0000 7fbd898b6640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd70065b80 0x7fbd70068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.326+0000 7fbd898b6640 1 -- 172.21.15.26:0/2131799895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840b34f0 msgr2=0x7fbd8415f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.326+0000 7fbd898b6640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840b34f0 0x7fbd8415f790 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800e870 tx=0x7fbd7800ec70 comp rx=0 tx=0).stop 2023-12-19T17:57:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.326+0000 7fbd898b6640 1 -- 172.21.15.26:0/2131799895 shutdown_connections 2023-12-19T17:57:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.326+0000 7fbd898b6640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fbd70065b80 0x7fbd70068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.326+0000 7fbd898b6640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbd8415d260 0x7fbd8415fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.326+0000 7fbd898b6640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840b34f0 0x7fbd8415f790 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.326+0000 7fbd898b6640 1 --2- 172.21.15.26:0/2131799895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd840658e0 0x7fbd84070850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.326+0000 7fbd898b6640 1 -- 172.21.15.26:0/2131799895 >> 172.21.15.26:0/2131799895 conn(0x7fbd8406a320 msgr2=0x7fbd840ac850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:58.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.326+0000 7fbd898b6640 1 -- 172.21.15.26:0/2131799895 shutdown_connections 2023-12-19T17:57:58.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.326+0000 7fbd898b6640 1 -- 172.21.15.26:0/2131799895 wait complete. 2023-12-19T17:57:58.348 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:57:58.430 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:58.426+0000 7fb3e6e4d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:57:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.490+0000 7f9df53b3640 1 Processor -- start 2023-12-19T17:57:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.490+0000 7f9df53b3640 1 -- start start 2023-12-19T17:57:58.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.490+0000 7f9df53b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9df0155030 0x7f9df015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.490+0000 7f9df53b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9df00ce740 0x7f9df00ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.490+0000 7f9df53b3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df00cf0b0 con 0x7f9df015f370 2023-12-19T17:57:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.490+0000 7f9df53b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df006c790 con 0x7f9df0155030 2023-12-19T17:57:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.490+0000 7f9df53b3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df006c970 con 0x7f9df00ce740 2023-12-19T17:57:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.490+0000 7f9deeffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9df0155030 0x7f9df015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.490+0000 7f9deeffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9df0155030 0x7f9df015bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39868/0 (socket says 172.21.15.26:39868) 2023-12-19T17:57:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.490+0000 7f9dee7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9df015f370 0x7f9df006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44780/0 (socket says 172.21.15.26:44780) 2023-12-19T17:57:58.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.490+0000 7f9deeffd640 1 -- 172.21.15.26:0/355524483 learned_addr learned my addr 172.21.15.26:0/355524483 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.490+0000 7f9def7fe640 1 --2- 172.21.15.26:0/355524483 >> v2:172.21.15.160:3300/0 conn(0x7f9df00ce740 0x7f9df00ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.494+0000 7f9deeffd640 1 -- 172.21.15.26:0/355524483 >> v2:172.21.15.160:3300/0 conn(0x7f9df00ce740 msgr2=0x7f9df00ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.494+0000 7f9deeffd640 1 --2- 172.21.15.26:0/355524483 >> v2:172.21.15.160:3300/0 conn(0x7f9df00ce740 0x7f9df00ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.494+0000 7f9deeffd640 1 -- 172.21.15.26:0/355524483 >> v1:172.21.15.26:6789/0 conn(0x7f9df015f370 legacy=0x7f9df006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.494+0000 7f9deeffd640 1 -- 172.21.15.26:0/355524483 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df015b1f0 con 0x7f9df0155030 2023-12-19T17:57:58.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.494+0000 7f9dedffb640 1 -- 172.21.15.26:0/355524483 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996930850 0 0) 0x7f9df00cf0b0 con 0x7f9df015f370 2023-12-19T17:57:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.494+0000 7f9deeffd640 1 --2- 172.21.15.26:0/355524483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df0155030 0x7f9df015bcd0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f9df00cc390 tx=0x7f9de001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77ad5f064feb26a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.494+0000 7f9dedffb640 1 -- 172.21.15.26:0/355524483 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00aa430 con 0x7f9df0155030 2023-12-19T17:57:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.494+0000 7f9dedffb640 1 -- 172.21.15.26:0/355524483 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de00aa590 con 0x7f9df0155030 2023-12-19T17:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.494+0000 7f9dedffb640 1 -- 172.21.15.26:0/355524483 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00aa920 con 0x7f9df0155030 2023-12-19T17:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.494+0000 7f9df53b3640 1 -- 172.21.15.26:0/355524483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df0155030 msgr2=0x7f9df015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.494+0000 7f9df53b3640 1 --2- 172.21.15.26:0/355524483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df0155030 0x7f9df015bcd0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f9df00cc390 tx=0x7f9de001c4d0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 -- 172.21.15.26:0/355524483 shutdown_connections 2023-12-19T17:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 --2- 172.21.15.26:0/355524483 >> v2:172.21.15.160:3300/0 conn(0x7f9df00ce740 0x7f9df00ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 --2- 172.21.15.26:0/355524483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df0155030 0x7f9df015bcd0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 -- 172.21.15.26:0/355524483 >> 172.21.15.26:0/355524483 conn(0x7f9df00658e0 msgr2=0x7f9df0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 -- 172.21.15.26:0/355524483 shutdown_connections 2023-12-19T17:57:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 -- 172.21.15.26:0/355524483 wait complete. 2023-12-19T17:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 Processor -- start 2023-12-19T17:57:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 -- start start 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9df00ce740 0x7f9df0145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df0155030 0x7f9df00ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9df015f370 0x7f9df00acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00adca0 con 0x7f9df0155030 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df014c010 con 0x7f9df00ce740 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9df53b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df014b2c0 con 0x7f9df015f370 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9dee7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df0155030 0x7f9df00ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9deeffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9df00ce740 0x7f9df0145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9def7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9df015f370 0x7f9df00acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9dee7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df0155030 0x7f9df00ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39874/0 (socket says 172.21.15.26:39874) 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9dee7fc640 1 -- 172.21.15.26:0/555385770 learned_addr learned my addr 172.21.15.26:0/555385770 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.498+0000 7f9deeffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9df00ce740 0x7f9df0145940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:51562/0 (socket says 172.21.15.26:51562) 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9deeffd640 1 -- 172.21.15.26:0/555385770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9df015f370 msgr2=0x7f9df00acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9deeffd640 1 --2- 172.21.15.26:0/555385770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9df015f370 0x7f9df00acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9deeffd640 1 -- 172.21.15.26:0/555385770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df0155030 msgr2=0x7f9df00ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9deeffd640 1 --2- 172.21.15.26:0/555385770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df0155030 0x7f9df00ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9deeffd640 1 -- 172.21.15.26:0/555385770 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00094d0 con 0x7f9df00ce740 2023-12-19T17:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9deeffd640 1 --2- 172.21.15.26:0/555385770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9df00ce740 0x7f9df0145940 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9df0066830 tx=0x7f9de001ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9dd77fe640 1 -- 172.21.15.26:0/555385770 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00aa430 con 0x7f9df00ce740 2023-12-19T17:57:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9dd77fe640 1 -- 172.21.15.26:0/555385770 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de00b3820 con 0x7f9df00ce740 2023-12-19T17:57:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9dd77fe640 1 -- 172.21.15.26:0/555385770 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de00b3bd0 con 0x7f9df00ce740 2023-12-19T17:57:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9df53b3640 1 -- 172.21.15.26:0/555385770 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df00b3800 con 0x7f9df00ce740 2023-12-19T17:57:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9df53b3640 1 -- 172.21.15.26:0/555385770 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df014b810 con 0x7f9df00ce740 2023-12-19T17:57:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9df53b3640 1 -- 172.21.15.26:0/555385770 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df0074270 con 0x7f9df00ce740 2023-12-19T17:57:58.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9dd77fe640 1 -- 172.21.15.26:0/555385770 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 85) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f9de00b3d30 con 0x7f9df00ce740 2023-12-19T17:57:58.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9dd77fe640 1 --2- 172.21.15.26:0/555385770 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9ddc065b80 0x7f9ddc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.502+0000 7f9dd77fe640 1 -- 172.21.15.26:0/555385770 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f9de011d950 con 0x7f9df00ce740 2023-12-19T17:57:58.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.506+0000 7f9dee7fc640 1 -- 172.21.15.26:0/555385770 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9ddc065b80 msgr2=0x7f9ddc068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:58.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.506+0000 7f9dee7fc640 1 --2- 172.21.15.26:0/555385770 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9ddc065b80 0x7f9ddc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:58.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.510+0000 7f9dd77fe640 1 -- 172.21.15.26:0/555385770 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9de00ead50 con 0x7f9df00ce740 2023-12-19T17:57:58.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.674+0000 7f9df53b3640 1 -- 172.21.15.26:0/555385770 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9df015abb0 con 0x7f9df00ce740 2023-12-19T17:57:58.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.674+0000 7f9dd77fe640 1 -- 172.21.15.26:0/555385770 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+17713 (secure 0 0 0) 0x7f9de00ee370 con 0x7f9df00ce740 2023-12-19T17:57:58.678 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:57:58.678 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":164,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:57:52.397105+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:57:38.188932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T17:57:39.438347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1221875420":"2023-12-20T17:57:52.397071+0000","172.21.15.160:6833/38098":"2023-12-20T17:57:52.397071+0000","172.21.15.160:6832/38098":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/1668583613":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/1848495251":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/92169616":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/2374756181":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/1748726321":"2023-12-20T17:57:52.397071+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:57:58.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.678+0000 7f9df53b3640 1 -- 172.21.15.26:0/555385770 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9ddc065b80 msgr2=0x7f9ddc068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:58.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.678+0000 7f9df53b3640 1 --2- 172.21.15.26:0/555385770 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9ddc065b80 0x7f9ddc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.678+0000 7f9df53b3640 1 -- 172.21.15.26:0/555385770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9df00ce740 msgr2=0x7f9df0145940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.678+0000 7f9df53b3640 1 --2- 172.21.15.26:0/555385770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9df00ce740 0x7f9df0145940 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f9df0066830 tx=0x7f9de001ad10 comp rx=0 tx=0).stop 2023-12-19T17:57:58.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.678+0000 7f9df53b3640 1 -- 172.21.15.26:0/555385770 shutdown_connections 2023-12-19T17:57:58.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.678+0000 7f9df53b3640 1 --2- 172.21.15.26:0/555385770 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9ddc065b80 0x7f9ddc068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.678+0000 7f9df53b3640 1 --2- 172.21.15.26:0/555385770 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9df015f370 0x7f9df00acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.678+0000 7f9df53b3640 1 --2- 172.21.15.26:0/555385770 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df0155030 0x7f9df00ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.678+0000 7f9df53b3640 1 --2- 172.21.15.26:0/555385770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9df00ce740 0x7f9df0145940 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.678+0000 7f9df53b3640 1 -- 172.21.15.26:0/555385770 >> 172.21.15.26:0/555385770 conn(0x7f9df00658e0 msgr2=0x7f9df015ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:58.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.678+0000 7f9df53b3640 1 -- 172.21.15.26:0/555385770 shutdown_connections 2023-12-19T17:57:58.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.678+0000 7f9df53b3640 1 -- 172.21.15.26:0/555385770 wait complete. 2023-12-19T17:57:58.701 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T17:57:38.188932+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T17:57:39.438347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '163', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T17:57:58.701 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T17:57:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.846+0000 7ffb88eb6640 1 Processor -- start 2023-12-19T17:57:58.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb88eb6640 1 -- start start 2023-12-19T17:57:58.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb88eb6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb8415ccf0 0x7ffb840d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb88eb6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb8415d0d0 0x7ffb840b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb88eb6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8415e200 con 0x7ffb840b4600 2023-12-19T17:57:58.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb88eb6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb840d0aa0 con 0x7ffb8415ccf0 2023-12-19T17:57:58.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb88eb6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb840d1380 con 0x7ffb8415d0d0 2023-12-19T17:57:58.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb8259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb8415ccf0 0x7ffb840d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb8259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb8415ccf0 0x7ffb840d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39882/0 (socket says 172.21.15.26:39882) 2023-12-19T17:57:58.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb8259c640 1 -- 172.21.15.26:0/2923777268 learned_addr learned my addr 172.21.15.26:0/2923777268 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:58.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb81d9b640 1 --2- 172.21.15.26:0/2923777268 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb8415d0d0 0x7ffb840b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.854 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:58.850+0000 7f81531f6180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:57:58.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb8259c640 1 -- 172.21.15.26:0/2923777268 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb8415d0d0 msgr2=0x7ffb840b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:58.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb8259c640 1 --2- 172.21.15.26:0/2923777268 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb8415d0d0 0x7ffb840b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb8259c640 1 -- 172.21.15.26:0/2923777268 >> v1:172.21.15.160:6789/0 conn(0x7ffb840b4600 legacy=0x7ffb84160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.850+0000 7ffb8259c640 1 -- 172.21.15.26:0/2923777268 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8415d680 con 0x7ffb8415ccf0 2023-12-19T17:57:58.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.854+0000 7ffb8259c640 1 --2- 172.21.15.26:0/2923777268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb8415ccf0 0x7ffb840d0e40 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009870 tx=0x7ffb7401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf4f871eff49278 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:58.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.854+0000 7ffb8159a640 1 -- 172.21.15.26:0/2923777268 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb740aa670 con 0x7ffb8415ccf0 2023-12-19T17:57:58.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.854+0000 7ffb8159a640 1 -- 172.21.15.26:0/2923777268 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb740aa7d0 con 0x7ffb8415ccf0 2023-12-19T17:57:58.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.854+0000 7ffb8159a640 1 -- 172.21.15.26:0/2923777268 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb740aab60 con 0x7ffb8415ccf0 2023-12-19T17:57:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.854+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2923777268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb8415ccf0 msgr2=0x7ffb840d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.854+0000 7ffb88eb6640 1 --2- 172.21.15.26:0/2923777268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb8415ccf0 0x7ffb840d0e40 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ffb74009870 tx=0x7ffb7401c510 comp rx=0 tx=0).stop 2023-12-19T17:57:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.854+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2923777268 shutdown_connections 2023-12-19T17:57:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.854+0000 7ffb88eb6640 1 --2- 172.21.15.26:0/2923777268 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb8415d0d0 0x7ffb840b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.854+0000 7ffb88eb6640 1 --2- 172.21.15.26:0/2923777268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb8415ccf0 0x7ffb840d0e40 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.854+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2923777268 >> 172.21.15.26:0/2923777268 conn(0x7ffb84065ee0 msgr2=0x7ffb840cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.854+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2923777268 shutdown_connections 2023-12-19T17:57:58.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.854+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2923777268 wait complete. 2023-12-19T17:57:58.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb88eb6640 1 Processor -- start 2023-12-19T17:57:58.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb88eb6640 1 -- start start 2023-12-19T17:57:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb88eb6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb840b4600 0x7ffb840cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb88eb6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb8415ccf0 0x7ffb84153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb88eb6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8415d0d0 0x7ffb8415a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb88eb6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb840c9d00 con 0x7ffb8415ccf0 2023-12-19T17:57:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb88eb6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb84155260 con 0x7ffb8415d0d0 2023-12-19T17:57:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb88eb6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb84155540 con 0x7ffb840b4600 2023-12-19T17:57:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb81d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb8415ccf0 0x7ffb84153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb8259c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb840b4600 0x7ffb840cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb82d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8415d0d0 0x7ffb8415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb8259c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb840b4600 0x7ffb840cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:43480/0 (socket says 172.21.15.26:43480) 2023-12-19T17:57:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb8259c640 1 -- 172.21.15.26:0/2223031062 learned_addr learned my addr 172.21.15.26:0/2223031062 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb82d9d640 1 -- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb840b4600 msgr2=0x7ffb840cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb82d9d640 1 --2- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb840b4600 0x7ffb840cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb82d9d640 1 -- 172.21.15.26:0/2223031062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb8415ccf0 msgr2=0x7ffb84153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb82d9d640 1 --2- 172.21.15.26:0/2223031062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb8415ccf0 0x7ffb84153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb82d9d640 1 -- 172.21.15.26:0/2223031062 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb740094d0 con 0x7ffb8415d0d0 2023-12-19T17:57:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb8259c640 1 --2- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb840b4600 0x7ffb840cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb82d9d640 1 --2- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8415d0d0 0x7ffb8415a300 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00c730 tx=0x7ffb7c00cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb72ffd640 1 -- 172.21.15.26:0/2223031062 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c003b80 con 0x7ffb8415d0d0 2023-12-19T17:57:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb72ffd640 1 -- 172.21.15.26:0/2223031062 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c003ce0 con 0x7ffb8415d0d0 2023-12-19T17:57:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2223031062 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb84155820 con 0x7ffb8415d0d0 2023-12-19T17:57:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb72ffd640 1 -- 172.21.15.26:0/2223031062 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c005310 con 0x7ffb8415d0d0 2023-12-19T17:57:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.858+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2223031062 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb840c7fc0 con 0x7ffb8415d0d0 2023-12-19T17:57:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.862+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2223031062 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb840c7e20 con 0x7ffb8415d0d0 2023-12-19T17:57:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.862+0000 7ffb72ffd640 1 -- 172.21.15.26:0/2223031062 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 85) v1 ==== 85331+0+0 (secure 0 0 0) 0x7ffb7c005680 con 0x7ffb8415d0d0 2023-12-19T17:57:58.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.862+0000 7ffb72ffd640 1 --2- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb64065b80 0x7ffb64068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:58.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.862+0000 7ffb72ffd640 1 -- 172.21.15.26:0/2223031062 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6785+0+0 (secure 0 0 0) 0x7ffb7c083590 con 0x7ffb8415d0d0 2023-12-19T17:57:58.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.862+0000 7ffb8259c640 1 -- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb64065b80 msgr2=0x7ffb64068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:58.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.862+0000 7ffb8259c640 1 --2- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb64065b80 0x7ffb64068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:58.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:58.866+0000 7ffb72ffd640 1 -- 172.21.15.26:0/2223031062 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffb840c7e20 con 0x7ffb8415d0d0 2023-12-19T17:57:58.940 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:58.938+0000 7f81531f6180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:57:59.017 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:59.014+0000 7fb3e6e4d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:57:59.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.022+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2223031062 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb840c7e20 con 0x7ffb8415d0d0 2023-12-19T17:57:59.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.022+0000 7ffb72ffd640 1 -- 172.21.15.26:0/2223031062 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v164) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb840c7e20 con 0x7ffb8415d0d0 2023-12-19T17:57:59.027 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T17:57:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.026+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb64065b80 msgr2=0x7ffb64068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.026+0000 7ffb88eb6640 1 --2- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb64065b80 0x7ffb64068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.026+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8415d0d0 msgr2=0x7ffb8415a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.026+0000 7ffb88eb6640 1 --2- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8415d0d0 0x7ffb8415a300 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00c730 tx=0x7ffb7c00cbf0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.026+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2223031062 shutdown_connections 2023-12-19T17:57:59.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.026+0000 7ffb88eb6640 1 --2- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb8415d0d0 0x7ffb8415a300 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.026+0000 7ffb88eb6640 1 --2- 172.21.15.26:0/2223031062 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb8415ccf0 0x7ffb84153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.026+0000 7ffb88eb6640 1 --2- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ffb64065b80 0x7ffb64068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.026+0000 7ffb88eb6640 1 --2- 172.21.15.26:0/2223031062 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb840b4600 0x7ffb840cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.026+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2223031062 >> 172.21.15.26:0/2223031062 conn(0x7ffb84065ee0 msgr2=0x7ffb84067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:59.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.026+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2223031062 shutdown_connections 2023-12-19T17:57:59.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.026+0000 7ffb88eb6640 1 -- 172.21.15.26:0/2223031062 wait complete. 2023-12-19T17:57:59.049 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T17:57:59.090 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.086+0000 7f81531f6180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:57:59.110 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:59.106+0000 7fb3e6e4d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:57:59.175 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.170+0000 7f81531f6180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:57:59.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.194+0000 7f91b22df640 1 Processor -- start 2023-12-19T17:57:59.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.194+0000 7f91b22df640 1 -- start start 2023-12-19T17:57:59.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91b22df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac159140 0x7f91ac0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91b22df640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f91ac0cd190 0x7f91ac06ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91b22df640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91ac06f490 con 0x7f91ac06b1c0 2023-12-19T17:57:59.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91b22df640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91ac06b650 con 0x7f91ac0cd190 2023-12-19T17:57:59.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91b22df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac0b0930 con 0x7f91ac159140 2023-12-19T17:57:59.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91ab7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f91ac0cd190 0x7f91ac06ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91abfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac159140 0x7f91ac0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91b087c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f91ac06b1c0 0x7f91ac0b0220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:52676/0 (socket says 172.21.15.26:52676) 2023-12-19T17:57:59.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91b087c640 1 -- 172.21.15.26:0/2368789292 learned_addr learned my addr 172.21.15.26:0/2368789292 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:59.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91ab7fe640 1 -- 172.21.15.26:0/2368789292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac159140 msgr2=0x7f91ac0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:59.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91ab7fe640 1 --2- 172.21.15.26:0/2368789292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac159140 0x7f91ac0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91ab7fe640 1 -- 172.21.15.26:0/2368789292 >> v1:172.21.15.160:6789/0 conn(0x7f91ac06b1c0 legacy=0x7f91ac0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91ab7fe640 1 -- 172.21.15.26:0/2368789292 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac159520 con 0x7f91ac0cd190 2023-12-19T17:57:59.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.198+0000 7f91aaffd640 1 -- 172.21.15.26:0/2368789292 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838381723 0 0) 0x7f91ac06f490 con 0x7f91ac06b1c0 2023-12-19T17:57:59.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91ab7fe640 1 --2- 172.21.15.26:0/2368789292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0cd190 0x7f91ac06ac80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f91a00097f0 tx=0x7f91a001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de1297d6b5165d6e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:59.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91aaffd640 1 -- 172.21.15.26:0/2368789292 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a00ab650 con 0x7f91ac0cd190 2023-12-19T17:57:59.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91aaffd640 1 -- 172.21.15.26:0/2368789292 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a00ab7b0 con 0x7f91ac0cd190 2023-12-19T17:57:59.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91aaffd640 1 -- 172.21.15.26:0/2368789292 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a00abb40 con 0x7f91ac0cd190 2023-12-19T17:57:59.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91b22df640 1 -- 172.21.15.26:0/2368789292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0cd190 msgr2=0x7f91ac06ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91b22df640 1 --2- 172.21.15.26:0/2368789292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0cd190 0x7f91ac06ac80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f91a00097f0 tx=0x7f91a001c450 comp rx=0 tx=0).stop 2023-12-19T17:57:59.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91b22df640 1 -- 172.21.15.26:0/2368789292 shutdown_connections 2023-12-19T17:57:59.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91b22df640 1 --2- 172.21.15.26:0/2368789292 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0cd190 0x7f91ac06ac80 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91b22df640 1 --2- 172.21.15.26:0/2368789292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac159140 0x7f91ac0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91b22df640 1 -- 172.21.15.26:0/2368789292 >> 172.21.15.26:0/2368789292 conn(0x7f91ac0ce730 msgr2=0x7f91ac0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:59.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91b22df640 1 -- 172.21.15.26:0/2368789292 shutdown_connections 2023-12-19T17:57:59.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91b22df640 1 -- 172.21.15.26:0/2368789292 wait complete. 2023-12-19T17:57:59.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91b22df640 1 Processor -- start 2023-12-19T17:57:59.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.202+0000 7f91b22df640 1 -- start start 2023-12-19T17:57:59.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91b22df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac06b1c0 0x7f91ac167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91b22df640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0cd190 0x7f91ac167ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91b22df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac159140 0x7f91ac0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91b22df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac0c70e0 con 0x7f91ac06b1c0 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91b22df640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac0c6ee0 con 0x7f91ac0cd190 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91b22df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac13d140 con 0x7f91ac159140 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac06b1c0 0x7f91ac167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac06b1c0 0x7f91ac167ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39892/0 (socket says 172.21.15.26:39892) 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91abfff640 1 -- 172.21.15.26:0/818186078 learned_addr learned my addr 172.21.15.26:0/818186078 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91ab7fe640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0cd190 0x7f91ac167ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91b087c640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac159140 0x7f91ac0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91abfff640 1 -- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac159140 msgr2=0x7f91ac0c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91abfff640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac159140 0x7f91ac0c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91abfff640 1 -- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0cd190 msgr2=0x7f91ac167ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91abfff640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0cd190 0x7f91ac167ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91abfff640 1 -- 172.21.15.26:0/818186078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a0009420 con 0x7f91ac06b1c0 2023-12-19T17:57:59.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91ab7fe640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0cd190 0x7f91ac167ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:59.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91abfff640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac06b1c0 0x7f91ac167ab0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f919c00e870 tx=0x7f919c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:59.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91a8ff9640 1 -- 172.21.15.26:0/818186078 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c009510 con 0x7f91ac06b1c0 2023-12-19T17:57:59.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91a8ff9640 1 -- 172.21.15.26:0/818186078 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919c00eec0 con 0x7f91ac06b1c0 2023-12-19T17:57:59.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91a8ff9640 1 -- 172.21.15.26:0/818186078 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c00cb40 con 0x7f91ac06b1c0 2023-12-19T17:57:59.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91b22df640 1 -- 172.21.15.26:0/818186078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ac0b3b20 con 0x7f91ac06b1c0 2023-12-19T17:57:59.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91b22df640 1 -- 172.21.15.26:0/818186078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ac0b40e0 con 0x7f91ac06b1c0 2023-12-19T17:57:59.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91b22df640 1 -- 172.21.15.26:0/818186078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ac0c7350 con 0x7f91ac06b1c0 2023-12-19T17:57:59.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.206+0000 7f91a8ff9640 1 -- 172.21.15.26:0/818186078 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 85) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f919c00cca0 con 0x7f91ac06b1c0 2023-12-19T17:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.210+0000 7f91a8ff9640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9198065b80 0x7f9198068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.210+0000 7f91a8ff9640 1 -- 172.21.15.26:0/818186078 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f919c083850 con 0x7f91ac06b1c0 2023-12-19T17:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.210+0000 7f91ab7fe640 1 -- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9198065b80 msgr2=0x7f9198068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:59.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.210+0000 7f91ab7fe640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9198065b80 0x7f9198068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:59.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.214+0000 7f91a8ff9640 1 -- 172.21.15.26:0/818186078 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f919c050cd0 con 0x7f91ac06b1c0 2023-12-19T17:57:59.272 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:59.270+0000 7fb3e6e4d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:57:59.361 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:59.358+0000 7fb3e6e4d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:57:59.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.370+0000 7f91b22df640 1 -- 172.21.15.26:0/818186078 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f91ac0cafb0 con 0x7f91ac06b1c0 2023-12-19T17:57:59.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.370+0000 7f91a8ff9640 1 -- 172.21.15.26:0/818186078 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v164) v1 ==== 104+0+11 (secure 0 0 0) 0x7f919c0542f0 con 0x7f91ac06b1c0 2023-12-19T17:57:59.375 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T17:57:59.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.374+0000 7f91b22df640 1 -- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9198065b80 msgr2=0x7f9198068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.374+0000 7f91b22df640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9198065b80 0x7f9198068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.374+0000 7f91b22df640 1 -- 172.21.15.26:0/818186078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac06b1c0 msgr2=0x7f91ac167ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.374+0000 7f91b22df640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac06b1c0 0x7f91ac167ab0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f919c00e870 tx=0x7f919c00ec70 comp rx=0 tx=0).stop 2023-12-19T17:57:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.374+0000 7f91b22df640 1 -- 172.21.15.26:0/818186078 shutdown_connections 2023-12-19T17:57:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.374+0000 7f91b22df640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f9198065b80 0x7f9198068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.374+0000 7f91b22df640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f91ac159140 0x7f91ac0c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.374+0000 7f91b22df640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91ac0cd190 0x7f91ac167ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.374+0000 7f91b22df640 1 --2- 172.21.15.26:0/818186078 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91ac06b1c0 0x7f91ac167ab0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.374+0000 7f91b22df640 1 -- 172.21.15.26:0/818186078 >> 172.21.15.26:0/818186078 conn(0x7f91ac0ce730 msgr2=0x7f91ac157e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.374+0000 7f91b22df640 1 -- 172.21.15.26:0/818186078 shutdown_connections 2023-12-19T17:57:59.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.374+0000 7f91b22df640 1 -- 172.21.15.26:0/818186078 wait complete. 2023-12-19T17:57:59.398 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T17:57:59.495 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.490+0000 7f81531f6180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:57:59.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.542+0000 7f43c202a640 1 Processor -- start 2023-12-19T17:57:59.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.542+0000 7f43c202a640 1 -- start start 2023-12-19T17:57:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43c202a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f43bc15cc90 0x7f43bc0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43c202a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43bc0cdb20 con 0x7f43bc160a20 2023-12-19T17:57:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43c202a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43bc0cde00 con 0x7f43bc15d070 2023-12-19T17:57:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43c202a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43bc0ce0e0 con 0x7f43bc15cc90 2023-12-19T17:57:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43c1829640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f43bc160a20 0x7f43bc0d1760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44784/0 (socket says 172.21.15.26:44784) 2023-12-19T17:57:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43c1829640 1 -- 172.21.15.26:0/1150108419 learned_addr learned my addr 172.21.15.26:0/1150108419 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:59.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43c1028640 1 --2- 172.21.15.26:0/1150108419 >> v2:172.21.15.160:3300/0 conn(0x7f43bc15cc90 0x7f43bc0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43b3fff640 1 -- 172.21.15.26:0/1150108419 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467000755 0 0) 0x7f43bc0cdb20 con 0x7f43bc160a20 2023-12-19T17:57:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43b3fff640 1 -- 172.21.15.26:0/1150108419 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43a8003530 con 0x7f43bc160a20 2023-12-19T17:57:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43b3fff640 1 -- 172.21.15.26:0/1150108419 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931442441 0 0) 0x7f43bc0cde00 con 0x7f43bc15d070 2023-12-19T17:57:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43b3fff640 1 -- 172.21.15.26:0/1150108419 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43bc0cdb20 con 0x7f43bc15d070 2023-12-19T17:57:59.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43c1028640 1 -- 172.21.15.26:0/1150108419 >> v1:172.21.15.160:6789/0 conn(0x7f43bc15d070 legacy=0x7f43bc0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43c1028640 1 -- 172.21.15.26:0/1150108419 >> v1:172.21.15.26:6789/0 conn(0x7f43bc160a20 legacy=0x7f43bc0d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.546+0000 7f43c1028640 1 -- 172.21.15.26:0/1150108419 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43bc0ce3c0 con 0x7f43bc15cc90 2023-12-19T17:57:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.550+0000 7f43c1028640 1 --2- 172.21.15.26:0/1150108419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43bc15cc90 0x7f43bc0cd2f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f43b80097c0 tx=0x7f43b801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3246d98ec9fbe2c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.550+0000 7f43b3fff640 1 -- 172.21.15.26:0/1150108419 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b80a7070 con 0x7f43bc15cc90 2023-12-19T17:57:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.550+0000 7f43b3fff640 1 -- 172.21.15.26:0/1150108419 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b80ab430 con 0x7f43bc15cc90 2023-12-19T17:57:59.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.550+0000 7f43b3fff640 1 -- 172.21.15.26:0/1150108419 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b80ab7a0 con 0x7f43bc15cc90 2023-12-19T17:57:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.550+0000 7f43c202a640 1 -- 172.21.15.26:0/1150108419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43bc15cc90 msgr2=0x7f43bc0cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.550+0000 7f43c202a640 1 --2- 172.21.15.26:0/1150108419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43bc15cc90 0x7f43bc0cd2f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f43b80097c0 tx=0x7f43b801c450 comp rx=0 tx=0).stop 2023-12-19T17:57:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.550+0000 7f43c202a640 1 -- 172.21.15.26:0/1150108419 shutdown_connections 2023-12-19T17:57:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.550+0000 7f43c202a640 1 --2- 172.21.15.26:0/1150108419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43bc15cc90 0x7f43bc0cd2f0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.550+0000 7f43c202a640 1 -- 172.21.15.26:0/1150108419 >> 172.21.15.26:0/1150108419 conn(0x7f43bc06a320 msgr2=0x7f43bc0658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.550+0000 7f43c202a640 1 -- 172.21.15.26:0/1150108419 shutdown_connections 2023-12-19T17:57:59.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.550+0000 7f43c202a640 1 -- 172.21.15.26:0/1150108419 wait complete. 2023-12-19T17:57:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.550+0000 7f43c202a640 1 Processor -- start 2023-12-19T17:57:59.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.550+0000 7f43c202a640 1 -- start start 2023-12-19T17:57:59.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c202a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43bc15cc90 0x7f43bc0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c202a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43bc15d070 0x7f43bc14f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c202a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43bc160a20 0x7f43bc15e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c202a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0ca570 con 0x7f43bc15cc90 2023-12-19T17:57:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c202a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc0cb3d0 con 0x7f43bc160a20 2023-12-19T17:57:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c202a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc14f760 con 0x7f43bc15d070 2023-12-19T17:57:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c1028640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43bc15cc90 0x7f43bc0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c1028640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43bc15cc90 0x7f43bc0cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39898/0 (socket says 172.21.15.26:39898) 2023-12-19T17:57:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c1028640 1 -- 172.21.15.26:0/1593668990 learned_addr learned my addr 172.21.15.26:0/1593668990 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c0827640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43bc15d070 0x7f43bc14f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c1829640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43bc160a20 0x7f43bc15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c1028640 1 -- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43bc15d070 msgr2=0x7f43bc14f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c1028640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43bc15d070 0x7f43bc14f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c1028640 1 -- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43bc160a20 msgr2=0x7f43bc15e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c1028640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43bc160a20 0x7f43bc15e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c1028640 1 -- 172.21.15.26:0/1593668990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b8009420 con 0x7f43bc15cc90 2023-12-19T17:57:59.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c1829640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43bc160a20 0x7f43bc15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:59.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c0827640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43bc15d070 0x7f43bc14f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:57:59.558 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.554+0000 7f81531f6180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:57:59.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c1028640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43bc15cc90 0x7f43bc0cb860 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009710 tx=0x7f43b80191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:59.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43b1ffb640 1 -- 172.21.15.26:0/1593668990 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8003820 con 0x7f43bc15cc90 2023-12-19T17:57:59.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43b1ffb640 1 -- 172.21.15.26:0/1593668990 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b8003980 con 0x7f43bc15cc90 2023-12-19T17:57:59.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43b1ffb640 1 -- 172.21.15.26:0/1593668990 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b8003cf0 con 0x7f43bc15cc90 2023-12-19T17:57:59.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c202a640 1 -- 172.21.15.26:0/1593668990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc0ca7c0 con 0x7f43bc15cc90 2023-12-19T17:57:59.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c202a640 1 -- 172.21.15.26:0/1593668990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc0cad80 con 0x7f43bc15cc90 2023-12-19T17:57:59.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43c202a640 1 -- 172.21.15.26:0/1593668990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc061930 con 0x7f43bc15cc90 2023-12-19T17:57:59.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.554+0000 7f43b1ffb640 1 -- 172.21.15.26:0/1593668990 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 85) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f43b80ab5a0 con 0x7f43bc15cc90 2023-12-19T17:57:59.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.558+0000 7f43b1ffb640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f43a0065b80 0x7f43a0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.558+0000 7f43b1ffb640 1 -- 172.21.15.26:0/1593668990 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f43b811d770 con 0x7f43bc15cc90 2023-12-19T17:57:59.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.558+0000 7f43c0827640 1 -- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f43a0065b80 msgr2=0x7f43a0068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:57:59.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.558+0000 7f43c0827640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f43a0065b80 0x7f43a0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:57:59.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.562+0000 7f43b1ffb640 1 -- 172.21.15.26:0/1593668990 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f43b80eaba0 con 0x7f43bc15cc90 2023-12-19T17:57:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.570+0000 7f43b1ffb640 1 -- 172.21.15.26:0/1593668990 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 86) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f43b80eaf70 con 0x7f43bc15cc90 2023-12-19T17:57:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.570+0000 7f43b1ffb640 1 -- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f43a0065b80 msgr2=0x7f43a0068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:57:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.570+0000 7f43b1ffb640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f43a0065b80 0x7f43a0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.640 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.638+0000 7f80db963640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:59.640 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.638+0000 7f80db963640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:59.640 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.638+0000 7f80db963640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:59.640 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.638+0000 7f80db963640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:59.640 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.638+0000 7f80db963640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:59.641 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.638+0000 7f80de969640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:59.641 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.638+0000 7f80de969640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:59.641 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.638+0000 7f80de969640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:59.641 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.638+0000 7f80de969640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:59.641 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:57:59.638+0000 7f80de969640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:57:59.715 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:59.714+0000 7fb3e6e4d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:57:59.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.730+0000 7f43c202a640 1 -- 172.21.15.26:0/1593668990 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43bc1618e0 con 0x7f43bc15cc90 2023-12-19T17:57:59.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.730+0000 7f43b1ffb640 1 -- 172.21.15.26:0/1593668990 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v165) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43b8103bb0 con 0x7f43bc15cc90 2023-12-19T17:57:59.733 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T17:57:59.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.730+0000 7f43c202a640 1 -- 172.21.15.26:0/1593668990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43bc15cc90 msgr2=0x7f43bc0cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.730+0000 7f43c202a640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43bc15cc90 0x7f43bc0cb860 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f43b8009710 tx=0x7f43b80191f0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.730+0000 7f43c202a640 1 -- 172.21.15.26:0/1593668990 shutdown_connections 2023-12-19T17:57:59.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.730+0000 7f43c202a640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43bc160a20 0x7f43bc15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.730+0000 7f43c202a640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f43a0065b80 0x7f43a0068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.730+0000 7f43c202a640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f43bc15d070 0x7f43bc14f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.730+0000 7f43c202a640 1 --2- 172.21.15.26:0/1593668990 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43bc15cc90 0x7f43bc0cb860 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.730+0000 7f43c202a640 1 -- 172.21.15.26:0/1593668990 >> 172.21.15.26:0/1593668990 conn(0x7f43bc06a320 msgr2=0x7f43bc157f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:59.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.730+0000 7f43c202a640 1 -- 172.21.15.26:0/1593668990 shutdown_connections 2023-12-19T17:57:59.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.730+0000 7f43c202a640 1 -- 172.21.15.26:0/1593668990 wait complete. 2023-12-19T17:57:59.756 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:57:59.785 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:57:59.782+0000 7fb3e6e4d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:57:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.898+0000 7f34428c4640 1 Processor -- start 2023-12-19T17:57:59.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.898+0000 7f34428c4640 1 -- start start 2023-12-19T17:57:59.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.898+0000 7f34428c4640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f343c15c7f0 0x7f343c06c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.898+0000 7f34428c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343c0adb80 0x7f343c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.898+0000 7f34428c4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f343c06ced0 con 0x7f343c0ccc20 2023-12-19T17:57:59.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.898+0000 7f34428c4640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343c06d770 con 0x7f343c15c7f0 2023-12-19T17:57:59.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.898+0000 7f34428c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343c06d1b0 con 0x7f343c0adb80 2023-12-19T17:57:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.898+0000 7f3440e61640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f343c0ccc20 0x7f343c0b30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44790/0 (socket says 172.21.15.26:44790) 2023-12-19T17:57:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.898+0000 7f3440e61640 1 -- 172.21.15.26:0/3735883743 learned_addr learned my addr 172.21.15.26:0/3735883743 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:57:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.898+0000 7f343bfff640 1 --2- 172.21.15.26:0/3735883743 >> v2:172.21.15.160:3300/0 conn(0x7f343c15c7f0 0x7f343c06c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.902+0000 7f343b7fe640 1 --2- 172.21.15.26:0/3735883743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343c0adb80 0x7f343c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.902+0000 7f343affd640 1 -- 172.21.15.26:0/3735883743 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535830964 0 0) 0x7f343c06ced0 con 0x7f343c0ccc20 2023-12-19T17:57:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.902+0000 7f343affd640 1 -- 172.21.15.26:0/3735883743 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3424003530 con 0x7f343c0ccc20 2023-12-19T17:57:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.902+0000 7f343bfff640 1 -- 172.21.15.26:0/3735883743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343c0adb80 msgr2=0x7f343c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:57:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.902+0000 7f343bfff640 1 --2- 172.21.15.26:0/3735883743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343c0adb80 0x7f343c0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.902+0000 7f343bfff640 1 -- 172.21.15.26:0/3735883743 >> v1:172.21.15.26:6789/0 conn(0x7f343c0ccc20 legacy=0x7f343c0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.902+0000 7f343bfff640 1 -- 172.21.15.26:0/3735883743 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c06d490 con 0x7f343c15c7f0 2023-12-19T17:57:59.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.902+0000 7f343b7fe640 1 --2- 172.21.15.26:0/3735883743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343c0adb80 0x7f343c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:57:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.902+0000 7f343bfff640 1 --2- 172.21.15.26:0/3735883743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343c15c7f0 0x7f343c06c7f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f342c0097c0 tx=0x7f342c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69b122da320b55ac server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.902+0000 7f343affd640 1 -- 172.21.15.26:0/3735883743 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c0ac640 con 0x7f343c15c7f0 2023-12-19T17:57:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.902+0000 7f343affd640 1 -- 172.21.15.26:0/3735883743 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342c0ac7a0 con 0x7f343c15c7f0 2023-12-19T17:57:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.902+0000 7f343affd640 1 -- 172.21.15.26:0/3735883743 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c0acb30 con 0x7f343c15c7f0 2023-12-19T17:57:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 -- 172.21.15.26:0/3735883743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343c15c7f0 msgr2=0x7f343c06c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 --2- 172.21.15.26:0/3735883743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343c15c7f0 0x7f343c06c7f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f342c0097c0 tx=0x7f342c01c480 comp rx=0 tx=0).stop 2023-12-19T17:57:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 -- 172.21.15.26:0/3735883743 shutdown_connections 2023-12-19T17:57:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 --2- 172.21.15.26:0/3735883743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343c0adb80 0x7f343c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 --2- 172.21.15.26:0/3735883743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343c15c7f0 0x7f343c06c7f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 -- 172.21.15.26:0/3735883743 >> 172.21.15.26:0/3735883743 conn(0x7f343c0658e0 msgr2=0x7f343c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:57:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 -- 172.21.15.26:0/3735883743 shutdown_connections 2023-12-19T17:57:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 -- 172.21.15.26:0/3735883743 wait complete. 2023-12-19T17:57:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 Processor -- start 2023-12-19T17:57:59.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 -- start start 2023-12-19T17:57:59.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343c0adb80 0x7f343c15b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343c0ccc20 0x7f343c154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f343c15c7f0 0x7f343c1f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:57:59.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c155fc0 con 0x7f343c15c7f0 2023-12-19T17:57:59.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c15b740 con 0x7f343c0adb80 2023-12-19T17:57:59.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f34428c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343c15ba20 con 0x7f343c0ccc20 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f3440e61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f343c15c7f0 0x7f343c1f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f3440e61640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f343c15c7f0 0x7f343c1f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39910/0 (socket says 172.21.15.26:39910) 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f3440e61640 1 -- 172.21.15.26:0/2674557406 learned_addr learned my addr 172.21.15.26:0/2674557406 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f343b7fe640 1 --2- 172.21.15.26:0/2674557406 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343c0ccc20 0x7f343c154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.906+0000 7f343bfff640 1 --2- 172.21.15.26:0/2674557406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343c0adb80 0x7f343c15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f3440e61640 1 -- 172.21.15.26:0/2674557406 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343c0ccc20 msgr2=0x7f343c154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f3440e61640 1 --2- 172.21.15.26:0/2674557406 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343c0ccc20 0x7f343c154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f3440e61640 1 -- 172.21.15.26:0/2674557406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343c0adb80 msgr2=0x7f343c15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f3440e61640 1 --2- 172.21.15.26:0/2674557406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343c0adb80 0x7f343c15b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f3440e61640 1 -- 172.21.15.26:0/2674557406 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3430009420 con 0x7f343c15c7f0 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f3440e61640 1 --2- 172.21.15.26:0/2674557406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f343c15c7f0 0x7f343c1f4f30 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f343400cbb0 tx=0x7f343400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f3438ff9640 1 -- 172.21.15.26:0/2674557406 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343400ad40 con 0x7f343c15c7f0 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f3438ff9640 1 -- 172.21.15.26:0/2674557406 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434012400 con 0x7f343c15c7f0 2023-12-19T17:57:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f34428c4640 1 -- 172.21.15.26:0/2674557406 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c009420 con 0x7f343c15c7f0 2023-12-19T17:57:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f3438ff9640 1 -- 172.21.15.26:0/2674557406 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343400aea0 con 0x7f343c15c7f0 2023-12-19T17:57:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f34428c4640 1 -- 172.21.15.26:0/2674557406 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c156210 con 0x7f343c15c7f0 2023-12-19T17:57:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f34428c4640 1 -- 172.21.15.26:0/2674557406 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c156400 con 0x7f343c15c7f0 2023-12-19T17:57:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f3438ff9640 1 -- 172.21.15.26:0/2674557406 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 86) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f3434007070 con 0x7f343c15c7f0 2023-12-19T17:57:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.910+0000 7f3438ff9640 1 -- 172.21.15.26:0/2674557406 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f34340831d0 con 0x7f343c15c7f0 2023-12-19T17:57:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:57:59.914+0000 7f3438ff9640 1 -- 172.21.15.26:0/2674557406 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f34340509e0 con 0x7f343c15c7f0 2023-12-19T17:58:00.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.106+0000 7f34428c4640 1 -- 172.21.15.26:0/2674557406 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f343c074270 con 0x7f343c15c7f0 2023-12-19T17:58:00.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.106+0000 7f3438ff9640 1 -- 172.21.15.26:0/2674557406 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f3434054000 con 0x7f343c15c7f0 2023-12-19T17:58:00.110 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:00.111 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2850013418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2850013418},{"type":"v1","addr":"172.21.15.26:6835","nonce":2850013418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2213513866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2213513866},{"type":"v1","addr":"172.21.15.160:6836","nonce":2213513866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4292779848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4292779848},{"type":"v1","addr":"172.21.15.160:6837","nonce":4292779848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:57:40.517697+0000","modified":"2023-12-19T17:57:43.536734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8575":{"gid":8575,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/2838422258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2838422258},{"type":"v1","addr":"172.21.15.26:6834","nonce":2838422258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-19T17:58:00.111 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 73 2023-12-19T17:58:00.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.110+0000 7f34428c4640 1 -- 172.21.15.26:0/2674557406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f343c15c7f0 msgr2=0x7f343c1f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:00.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.110+0000 7f34428c4640 1 --2- 172.21.15.26:0/2674557406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f343c15c7f0 0x7f343c1f4f30 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f343400cbb0 tx=0x7f343400a430 comp rx=0 tx=0).stop 2023-12-19T17:58:00.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.110+0000 7f34428c4640 1 -- 172.21.15.26:0/2674557406 shutdown_connections 2023-12-19T17:58:00.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.110+0000 7f34428c4640 1 --2- 172.21.15.26:0/2674557406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f343c15c7f0 0x7f343c1f4f30 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.110+0000 7f34428c4640 1 --2- 172.21.15.26:0/2674557406 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f343c0ccc20 0x7f343c154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.110+0000 7f34428c4640 1 --2- 172.21.15.26:0/2674557406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f343c0adb80 0x7f343c15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.110+0000 7f34428c4640 1 -- 172.21.15.26:0/2674557406 >> 172.21.15.26:0/2674557406 conn(0x7f343c0658e0 msgr2=0x7f343c1557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:00.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.110+0000 7f34428c4640 1 -- 172.21.15.26:0/2674557406 shutdown_connections 2023-12-19T17:58:00.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.110+0000 7f34428c4640 1 -- 172.21.15.26:0/2674557406 wait complete. 2023-12-19T17:58:00.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:58:00.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09b676d640 1 Processor -- start 2023-12-19T17:58:00.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09b676d640 1 -- start start 2023-12-19T17:58:00.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09b676d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09b015cd40 0x7f09b00ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:00.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09b676d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b00d3300 0x7f09b0160d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:00.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09b676d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b00d29d0 con 0x7f09b0160900 2023-12-19T17:58:00.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09b676d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b015d680 con 0x7f09b015cd40 2023-12-19T17:58:00.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09b676d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0161750 con 0x7f09b00d3300 2023-12-19T17:58:00.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09affff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09b015cd40 0x7f09b00ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:00.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09affff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09b015cd40 0x7f09b00ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39916/0 (socket says 172.21.15.26:39916) 2023-12-19T17:58:00.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09affff640 1 -- 172.21.15.26:0/2582869337 learned_addr learned my addr 172.21.15.26:0/2582869337 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:00.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09b4d0a640 1 --2- 172.21.15.26:0/2582869337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b00d3300 0x7f09b0160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:00.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09affff640 1 -- 172.21.15.26:0/2582869337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b00d3300 msgr2=0x7f09b0160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:00.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09affff640 1 --2- 172.21.15.26:0/2582869337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b00d3300 0x7f09b0160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09affff640 1 -- 172.21.15.26:0/2582869337 >> v1:172.21.15.160:6789/0 conn(0x7f09b0160900 legacy=0x7f09b00d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:00.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09affff640 1 -- 172.21.15.26:0/2582869337 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b0161470 con 0x7f09b015cd40 2023-12-19T17:58:00.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.318+0000 7f09b4d0a640 1 --2- 172.21.15.26:0/2582869337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b00d3300 0x7f09b0160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:58:00.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.322+0000 7f09affff640 1 --2- 172.21.15.26:0/2582869337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b015cd40 0x7f09b00ce040 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f09a0006fd0 tx=0x7f09a001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74177c3013583533 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:00.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.322+0000 7f09aeffd640 1 -- 172.21.15.26:0/2582869337 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00ac410 con 0x7f09b015cd40 2023-12-19T17:58:00.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.322+0000 7f09aeffd640 1 -- 172.21.15.26:0/2582869337 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a00ac570 con 0x7f09b015cd40 2023-12-19T17:58:00.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.322+0000 7f09aeffd640 1 -- 172.21.15.26:0/2582869337 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00ac900 con 0x7f09b015cd40 2023-12-19T17:58:00.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.322+0000 7f09b676d640 1 -- 172.21.15.26:0/2582869337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b015cd40 msgr2=0x7f09b00ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:00.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.322+0000 7f09b676d640 1 --2- 172.21.15.26:0/2582869337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b015cd40 0x7f09b00ce040 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f09a0006fd0 tx=0x7f09a001c4e0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.322+0000 7f09b676d640 1 -- 172.21.15.26:0/2582869337 shutdown_connections 2023-12-19T17:58:00.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.322+0000 7f09b676d640 1 --2- 172.21.15.26:0/2582869337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b00d3300 0x7f09b0160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.322+0000 7f09b676d640 1 --2- 172.21.15.26:0/2582869337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b015cd40 0x7f09b00ce040 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.322+0000 7f09b676d640 1 -- 172.21.15.26:0/2582869337 >> 172.21.15.26:0/2582869337 conn(0x7f09b006a120 msgr2=0x7f09b006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:00.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.322+0000 7f09b676d640 1 -- 172.21.15.26:0/2582869337 shutdown_connections 2023-12-19T17:58:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.322+0000 7f09b676d640 1 -- 172.21.15.26:0/2582869337 wait complete. 2023-12-19T17:58:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b676d640 1 Processor -- start 2023-12-19T17:58:00.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b676d640 1 -- start start 2023-12-19T17:58:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b676d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b00d3300 0x7f09b015f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b676d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09b015cd40 0x7f09b00c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b676d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b0160900 0x7f09b00c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b676d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0156170 con 0x7f09b0160900 2023-12-19T17:58:00.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b676d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0155560 con 0x7f09b015cd40 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b676d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0155840 con 0x7f09b00d3300 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b4d0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b0160900 0x7f09b00c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09affff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b00d3300 0x7f09b015f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09affff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b00d3300 0x7f09b015f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:43546/0 (socket says 172.21.15.26:43546) 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09affff640 1 -- 172.21.15.26:0/2228926801 learned_addr learned my addr 172.21.15.26:0/2228926801 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09af7fe640 1 --2- 172.21.15.26:0/2228926801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09b015cd40 0x7f09b00c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b4d0a640 1 -- 172.21.15.26:0/2228926801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b00d3300 msgr2=0x7f09b015f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b4d0a640 1 --2- 172.21.15.26:0/2228926801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b00d3300 0x7f09b015f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b4d0a640 1 -- 172.21.15.26:0/2228926801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09b015cd40 msgr2=0x7f09b00c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b4d0a640 1 --2- 172.21.15.26:0/2228926801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09b015cd40 0x7f09b00c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b4d0a640 1 -- 172.21.15.26:0/2228926801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a0009480 con 0x7f09b0160900 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09affff640 1 --2- 172.21.15.26:0/2228926801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b00d3300 0x7f09b015f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09af7fe640 1 --2- 172.21.15.26:0/2228926801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09b015cd40 0x7f09b00c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:58:00.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b4d0a640 1 --2- 172.21.15.26:0/2228926801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b0160900 0x7f09b00c6fb0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f09a800cdf0 tx=0x7f09a800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09acff9640 1 -- 172.21.15.26:0/2228926801 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800d840 con 0x7f09b0160900 2023-12-19T17:58:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09acff9640 1 -- 172.21.15.26:0/2228926801 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a800d9a0 con 0x7f09b0160900 2023-12-19T17:58:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b676d640 1 -- 172.21.15.26:0/2228926801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b0155b20 con 0x7f09b0160900 2023-12-19T17:58:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09acff9640 1 -- 172.21.15.26:0/2228926801 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a800dd90 con 0x7f09b0160900 2023-12-19T17:58:00.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.326+0000 7f09b676d640 1 -- 172.21.15.26:0/2228926801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b0156be0 con 0x7f09b0160900 2023-12-19T17:58:00.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.330+0000 7f09b676d640 1 -- 172.21.15.26:0/2228926801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b00c6790 con 0x7f09b0160900 2023-12-19T17:58:00.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.330+0000 7f09acff9640 1 -- 172.21.15.26:0/2228926801 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 86) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f09a801e3b0 con 0x7f09b0160900 2023-12-19T17:58:00.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.330+0000 7f09acff9640 1 -- 172.21.15.26:0/2228926801 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f09a8083b50 con 0x7f09b0160900 2023-12-19T17:58:00.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.334+0000 7f09acff9640 1 -- 172.21.15.26:0/2228926801 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f09b00c6790 con 0x7f09b0160900 2023-12-19T17:58:00.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.494+0000 7f09b676d640 1 -- 172.21.15.26:0/2228926801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09b00c6790 con 0x7f09b0160900 2023-12-19T17:58:00.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.494+0000 7f09acff9640 1 -- 172.21.15.26:0/2228926801 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+18090 (secure 0 0 0) 0x7f09b00c6790 con 0x7f09b0160900 2023-12-19T17:58:00.498 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:00.498 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":165,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:57:59.565382+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:57:38.188932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T17:57:39.438347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3171778537":"2023-12-20T17:57:59.565333+0000","172.21.15.160:0/752855480":"2023-12-20T17:57:59.565333+0000","172.21.15.160:0/2457683375":"2023-12-20T17:57:59.565333+0000","172.21.15.160:0/1089394066":"2023-12-20T17:57:59.565333+0000","172.21.15.160:0/1221875420":"2023-12-20T17:57:52.397071+0000","172.21.15.160:6833/38098":"2023-12-20T17:57:52.397071+0000","172.21.15.160:6832/38098":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/1668583613":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/1848495251":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/2804752681":"2023-12-20T17:57:59.565333+0000","172.21.15.160:0/3041540379":"2023-12-20T17:57:59.565333+0000","172.21.15.160:0/92169616":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/2374756181":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/1748726321":"2023-12-20T17:57:52.397071+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:58:00.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.498+0000 7f09b676d640 1 -- 172.21.15.26:0/2228926801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b0160900 msgr2=0x7f09b00c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:00.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.498+0000 7f09b676d640 1 --2- 172.21.15.26:0/2228926801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b0160900 0x7f09b00c6fb0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f09a800cdf0 tx=0x7f09a800a710 comp rx=0 tx=0).stop 2023-12-19T17:58:00.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.498+0000 7f09b676d640 1 -- 172.21.15.26:0/2228926801 shutdown_connections 2023-12-19T17:58:00.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.498+0000 7f09b676d640 1 --2- 172.21.15.26:0/2228926801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09b0160900 0x7f09b00c6fb0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.498+0000 7f09b676d640 1 --2- 172.21.15.26:0/2228926801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f09b015cd40 0x7f09b00c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.498+0000 7f09b676d640 1 --2- 172.21.15.26:0/2228926801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f09b00d3300 0x7f09b015f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.498+0000 7f09b676d640 1 -- 172.21.15.26:0/2228926801 >> 172.21.15.26:0/2228926801 conn(0x7f09b006a120 msgr2=0x7f09b00b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:00.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.498+0000 7f09b676d640 1 -- 172.21.15.26:0/2228926801 shutdown_connections 2023-12-19T17:58:00.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.498+0000 7f09b676d640 1 -- 172.21.15.26:0/2228926801 wait complete. 2023-12-19T17:58:00.519 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T17:58:00.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:58:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cbabb3640 1 Processor -- start 2023-12-19T17:58:00.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cbabb3640 1 -- start start 2023-12-19T17:58:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cbabb3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb415f370 0x7f0cb415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cbabb3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0cb40ce740 0x7f0cb40cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cbabb3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb406c970 con 0x7f0cb4155030 2023-12-19T17:58:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cbabb3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb406c790 con 0x7f0cb40ce740 2023-12-19T17:58:00.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cbabb3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40cf130 con 0x7f0cb415f370 2023-12-19T17:58:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cb894f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0cb4155030 0x7f0cb406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44806/0 (socket says 172.21.15.26:44806) 2023-12-19T17:58:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cb894f640 1 -- 172.21.15.26:0/1627548556 learned_addr learned my addr 172.21.15.26:0/1627548556 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cb9150640 1 --2- 172.21.15.26:0/1627548556 >> v2:172.21.15.26:3300/0 conn(0x7f0cb40ce740 0x7f0cb40cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cb3fff640 1 --2- 172.21.15.26:0/1627548556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb415f370 0x7f0cb415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cb37fe640 1 -- 172.21.15.26:0/1627548556 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788645320 0 0) 0x7f0cb406c970 con 0x7f0cb4155030 2023-12-19T17:58:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cb37fe640 1 -- 172.21.15.26:0/1627548556 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ca4003530 con 0x7f0cb4155030 2023-12-19T17:58:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cb9150640 1 -- 172.21.15.26:0/1627548556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb415f370 msgr2=0x7f0cb415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cb9150640 1 --2- 172.21.15.26:0/1627548556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb415f370 0x7f0cb415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cb9150640 1 -- 172.21.15.26:0/1627548556 >> v1:172.21.15.26:6789/0 conn(0x7f0cb4155030 legacy=0x7f0cb406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cb9150640 1 -- 172.21.15.26:0/1627548556 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb415b1f0 con 0x7f0cb40ce740 2023-12-19T17:58:00.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.666+0000 7f0cb3fff640 1 --2- 172.21.15.26:0/1627548556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb415f370 0x7f0cb415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:00.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.670+0000 7f0cb9150640 1 --2- 172.21.15.26:0/1627548556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb40ce740 0x7f0cb40cebf0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009d90 tx=0x7f0cac01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d515374cf371ce51 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:00.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.670+0000 7f0cb37fe640 1 -- 172.21.15.26:0/1627548556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0a59e0 con 0x7f0cb40ce740 2023-12-19T17:58:00.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.670+0000 7f0cb37fe640 1 -- 172.21.15.26:0/1627548556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac0a5b40 con 0x7f0cb40ce740 2023-12-19T17:58:00.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.670+0000 7f0cb37fe640 1 -- 172.21.15.26:0/1627548556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0ad440 con 0x7f0cb40ce740 2023-12-19T17:58:00.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.670+0000 7f0cbabb3640 1 -- 172.21.15.26:0/1627548556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb40ce740 msgr2=0x7f0cb40cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:00.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.670+0000 7f0cbabb3640 1 --2- 172.21.15.26:0/1627548556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb40ce740 0x7f0cb40cebf0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009d90 tx=0x7f0cac01c920 comp rx=0 tx=0).stop 2023-12-19T17:58:00.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.670+0000 7f0cbabb3640 1 -- 172.21.15.26:0/1627548556 shutdown_connections 2023-12-19T17:58:00.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.670+0000 7f0cbabb3640 1 --2- 172.21.15.26:0/1627548556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb40ce740 0x7f0cb40cebf0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.670+0000 7f0cbabb3640 1 --2- 172.21.15.26:0/1627548556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb415f370 0x7f0cb415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.670+0000 7f0cbabb3640 1 -- 172.21.15.26:0/1627548556 >> 172.21.15.26:0/1627548556 conn(0x7f0cb40658e0 msgr2=0x7f0cb4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:00.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.670+0000 7f0cbabb3640 1 -- 172.21.15.26:0/1627548556 shutdown_connections 2023-12-19T17:58:00.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.670+0000 7f0cbabb3640 1 -- 172.21.15.26:0/1627548556 wait complete. 2023-12-19T17:58:00.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cbabb3640 1 Processor -- start 2023-12-19T17:58:00.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cbabb3640 1 -- start start 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cbabb3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cb40ce740 0x7f0cb41450c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cbabb3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb4155030 0x7f0cb40ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cbabb3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb415f370 0x7f0cb40acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cbabb3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4146670 con 0x7f0cb4155030 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cbabb3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40b3890 con 0x7f0cb40ce740 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cbabb3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb41459f0 con 0x7f0cb415f370 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb4155030 0x7f0cb40ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb4155030 0x7f0cb40ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39958/0 (socket says 172.21.15.26:39958) 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb3fff640 1 -- 172.21.15.26:0/3168706333 learned_addr learned my addr 172.21.15.26:0/3168706333 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb9150640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb415f370 0x7f0cb40acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb894f640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cb40ce740 0x7f0cb41450c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb3fff640 1 -- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb415f370 msgr2=0x7f0cb40acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb3fff640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb415f370 0x7f0cb40acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb3fff640 1 -- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cb40ce740 msgr2=0x7f0cb41450c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:00.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb3fff640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cb40ce740 0x7f0cb41450c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb3fff640 1 -- 172.21.15.26:0/3168706333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac009a70 con 0x7f0cb4155030 2023-12-19T17:58:00.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb3fff640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb4155030 0x7f0cb40ac850 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000d7d0 tx=0x7f0ca000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:00.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb17fa640 1 -- 172.21.15.26:0/3168706333 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca000b6c0 con 0x7f0cb4155030 2023-12-19T17:58:00.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb17fa640 1 -- 172.21.15.26:0/3168706333 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca000b820 con 0x7f0cb4155030 2023-12-19T17:58:00.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cb17fa640 1 -- 172.21.15.26:0/3168706333 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca000bbd0 con 0x7f0cb4155030 2023-12-19T17:58:00.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cbabb3640 1 -- 172.21.15.26:0/3168706333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb4145cd0 con 0x7f0cb4155030 2023-12-19T17:58:00.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cbabb3640 1 -- 172.21.15.26:0/3168706333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb4146290 con 0x7f0cb4155030 2023-12-19T17:58:00.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.674+0000 7f0cbabb3640 1 -- 172.21.15.26:0/3168706333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb4074270 con 0x7f0cb4155030 2023-12-19T17:58:00.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.678+0000 7f0cb17fa640 1 -- 172.21.15.26:0/3168706333 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 87) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f0ca000bd30 con 0x7f0cb4155030 2023-12-19T17:58:00.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.678+0000 7f0cb17fa640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0c980658b0 0x7f0c98067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:00.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.678+0000 7f0cb17fa640 1 -- 172.21.15.26:0/3168706333 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f0ca0083890 con 0x7f0cb4155030 2023-12-19T17:58:00.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.678+0000 7f0cb894f640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0c980658b0 0x7f0c98067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:00.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.678+0000 7f0cb894f640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0c980658b0 0x7f0c98067d70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80011f0 tx=0x7f0ca8009040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:00.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.682+0000 7f0cb17fa640 1 -- 172.21.15.26:0/3168706333 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0ca0050f80 con 0x7f0cb4155030 2023-12-19T17:58:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cbabb3640 1 -- 172.21.15.26:0/3168706333 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cb4149d70 con 0x7f0cb4155030 2023-12-19T17:58:00.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cb17fa640 1 -- 172.21.15.26:0/3168706333 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f0ca0011340 con 0x7f0cb4155030 2023-12-19T17:58:00.877 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:00.877 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2850013418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2850013418},{"type":"v1","addr":"172.21.15.26:6835","nonce":2850013418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2213513866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2213513866},{"type":"v1","addr":"172.21.15.160:6836","nonce":2213513866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4292779848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4292779848},{"type":"v1","addr":"172.21.15.160:6837","nonce":4292779848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:57:40.517697+0000","modified":"2023-12-19T17:57:43.536734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8575":{"gid":8575,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/2838422258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2838422258},{"type":"v1","addr":"172.21.15.26:6834","nonce":2838422258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-19T17:58:00.878 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 73 2023-12-19T17:58:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cbabb3640 1 -- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0c980658b0 msgr2=0x7f0c98067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cbabb3640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0c980658b0 0x7f0c98067d70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0ca80011f0 tx=0x7f0ca8009040 comp rx=0 tx=0).stop 2023-12-19T17:58:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cbabb3640 1 -- 172.21.15.26:0/3168706333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb4155030 msgr2=0x7f0cb40ac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cbabb3640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb4155030 0x7f0cb40ac850 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f0ca000d7d0 tx=0x7f0ca000dc90 comp rx=0 tx=0).stop 2023-12-19T17:58:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cbabb3640 1 -- 172.21.15.26:0/3168706333 shutdown_connections 2023-12-19T17:58:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cbabb3640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f0c980658b0 0x7f0c98067d70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cbabb3640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0cb415f370 0x7f0cb40acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cbabb3640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cb4155030 0x7f0cb40ac850 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cbabb3640 1 --2- 172.21.15.26:0/3168706333 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cb40ce740 0x7f0cb41450c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:00.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cbabb3640 1 -- 172.21.15.26:0/3168706333 >> 172.21.15.26:0/3168706333 conn(0x7f0cb40658e0 msgr2=0x7f0cb414f980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cbabb3640 1 -- 172.21.15.26:0/3168706333 shutdown_connections 2023-12-19T17:58:00.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:00.874+0000 7f0cbabb3640 1 -- 172.21.15.26:0/3168706333 wait complete. 2023-12-19T17:58:00.898 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:58:01.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.046+0000 7f1ce6c2d640 1 Processor -- start 2023-12-19T17:58:01.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.046+0000 7f1ce6c2d640 1 -- start start 2023-12-19T17:58:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1ce6c2d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ce015c7f0 0x7f1ce006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1ce6c2d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1ce00b30d0 0x7f1ce00b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1ce6c2d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce00b3a20 con 0x7f1ce00adb80 2023-12-19T17:58:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1ce6c2d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce006d770 con 0x7f1ce015c7f0 2023-12-19T17:58:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1ce6c2d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce006ced0 con 0x7f1ce00b30d0 2023-12-19T17:58:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1ce49c9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ce015c7f0 0x7f1ce006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1ce49c9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ce015c7f0 0x7f1ce006c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39968/0 (socket says 172.21.15.26:39968) 2023-12-19T17:58:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1cdffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1ce00adb80 0x7f1ce00cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44810/0 (socket says 172.21.15.26:44810) 2023-12-19T17:58:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1ce49c9640 1 -- 172.21.15.26:0/1261248407 learned_addr learned my addr 172.21.15.26:0/1261248407 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1ce51ca640 1 --2- 172.21.15.26:0/1261248407 >> v2:172.21.15.160:3300/0 conn(0x7f1ce00b30d0 0x7f1ce00b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1ce49c9640 1 -- 172.21.15.26:0/1261248407 >> v2:172.21.15.160:3300/0 conn(0x7f1ce00b30d0 msgr2=0x7f1ce00b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1ce49c9640 1 --2- 172.21.15.26:0/1261248407 >> v2:172.21.15.160:3300/0 conn(0x7f1ce00b30d0 0x7f1ce00b34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1ce49c9640 1 -- 172.21.15.26:0/1261248407 >> v1:172.21.15.26:6789/0 conn(0x7f1ce00adb80 legacy=0x7f1ce00cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1cdf7fe640 1 -- 172.21.15.26:0/1261248407 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739013212 0 0) 0x7f1ce00b3a20 con 0x7f1ce00adb80 2023-12-19T17:58:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.050+0000 7f1ce49c9640 1 -- 172.21.15.26:0/1261248407 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce006d1b0 con 0x7f1ce015c7f0 2023-12-19T17:58:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1ce49c9640 1 --2- 172.21.15.26:0/1261248407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce015c7f0 0x7f1ce006c7f0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00cc420 tx=0x7f1cd401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae369f47cf4ea2b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1cdf7fe640 1 -- 172.21.15.26:0/1261248407 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40ab600 con 0x7f1ce015c7f0 2023-12-19T17:58:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1cdf7fe640 1 -- 172.21.15.26:0/1261248407 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd40a7050 con 0x7f1ce015c7f0 2023-12-19T17:58:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1cdf7fe640 1 -- 172.21.15.26:0/1261248407 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40ab980 con 0x7f1ce015c7f0 2023-12-19T17:58:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1261248407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce015c7f0 msgr2=0x7f1ce006c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1ce6c2d640 1 --2- 172.21.15.26:0/1261248407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce015c7f0 0x7f1ce006c7f0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00cc420 tx=0x7f1cd401c500 comp rx=0 tx=0).stop 2023-12-19T17:58:01.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1261248407 shutdown_connections 2023-12-19T17:58:01.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1ce6c2d640 1 --2- 172.21.15.26:0/1261248407 >> v2:172.21.15.160:3300/0 conn(0x7f1ce00b30d0 0x7f1ce00b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1ce6c2d640 1 --2- 172.21.15.26:0/1261248407 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce015c7f0 0x7f1ce006c7f0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1261248407 >> 172.21.15.26:0/1261248407 conn(0x7f1ce00658e0 msgr2=0x7f1ce0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:01.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1261248407 shutdown_connections 2023-12-19T17:58:01.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1261248407 wait complete. 2023-12-19T17:58:01.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1ce6c2d640 1 Processor -- start 2023-12-19T17:58:01.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.054+0000 7f1ce6c2d640 1 -- start start 2023-12-19T17:58:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1ce6c2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ce00adb80 0x7f1ce0151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1ce6c2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce00b30d0 0x7f1ce0165e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1ce6c2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce015c7f0 0x7f1ce00ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1ce6c2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce013a790 con 0x7f1ce00b30d0 2023-12-19T17:58:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1ce6c2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce0151dc0 con 0x7f1ce00adb80 2023-12-19T17:58:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1ce6c2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce0139b80 con 0x7f1ce015c7f0 2023-12-19T17:58:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1cdffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce00b30d0 0x7f1ce0165e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1cdffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce00b30d0 0x7f1ce0165e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39970/0 (socket says 172.21.15.26:39970) 2023-12-19T17:58:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1cdffff640 1 -- 172.21.15.26:0/1628906232 learned_addr learned my addr 172.21.15.26:0/1628906232 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1ce49c9640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ce00adb80 0x7f1ce0151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1ce51ca640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce015c7f0 0x7f1ce00ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1cdffff640 1 -- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce015c7f0 msgr2=0x7f1ce00ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1cdffff640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce015c7f0 0x7f1ce00ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1cdffff640 1 -- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ce00adb80 msgr2=0x7f1ce0151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1cdffff640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ce00adb80 0x7f1ce0151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1cdffff640 1 -- 172.21.15.26:0/1628906232 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd40094d0 con 0x7f1ce00b30d0 2023-12-19T17:58:01.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1cdffff640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce00b30d0 0x7f1ce0165e60 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00c520 tx=0x7f1ccc00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:01.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1cdd7fa640 1 -- 172.21.15.26:0/1628906232 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0074e0 con 0x7f1ce00b30d0 2023-12-19T17:58:01.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1cdd7fa640 1 -- 172.21.15.26:0/1628906232 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc007640 con 0x7f1ce00b30d0 2023-12-19T17:58:01.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1628906232 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce0139e60 con 0x7f1ce00b30d0 2023-12-19T17:58:01.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1cdd7fa640 1 -- 172.21.15.26:0/1628906232 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0049f0 con 0x7f1ce00b30d0 2023-12-19T17:58:01.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1628906232 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce013a420 con 0x7f1ce00b30d0 2023-12-19T17:58:01.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1628906232 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce013a940 con 0x7f1ce00b30d0 2023-12-19T17:58:01.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.058+0000 7f1cdd7fa640 1 -- 172.21.15.26:0/1628906232 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 87) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f1ccc004b50 con 0x7f1ce00b30d0 2023-12-19T17:58:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.062+0000 7f1cdd7fa640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1cc40658e0 0x7f1cc4067da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.062+0000 7f1cdd7fa640 1 -- 172.21.15.26:0/1628906232 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f1ccc082950 con 0x7f1ce00b30d0 2023-12-19T17:58:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.062+0000 7f1ce49c9640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1cc40658e0 0x7f1cc4067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.062+0000 7f1ce49c9640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1cc40658e0 0x7f1cc4067da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40ad000 tx=0x7f1cd40a6000 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.066+0000 7f1cdd7fa640 1 -- 172.21.15.26:0/1628906232 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1ce013a940 con 0x7f1ce00b30d0 2023-12-19T17:58:01.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.254+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1628906232 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ce013a940 con 0x7f1ce00b30d0 2023-12-19T17:58:01.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.254+0000 7f1cdd7fa640 1 -- 172.21.15.26:0/1628906232 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f1ce013a940 con 0x7f1ce00b30d0 2023-12-19T17:58:01.260 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:01.260 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":73,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2850013418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2850013418},{"type":"v1","addr":"172.21.15.26:6835","nonce":2850013418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2213513866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2213513866},{"type":"v1","addr":"172.21.15.160:6836","nonce":2213513866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4292779848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4292779848},{"type":"v1","addr":"172.21.15.160:6837","nonce":4292779848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:57:40.517697+0000","modified":"2023-12-19T17:57:43.536734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8575},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8575":{"gid":8575,"name":"c","rank":0,"incarnation":69,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/2838422258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2838422258},{"type":"v1","addr":"172.21.15.26:6834","nonce":2838422258}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-19T17:58:01.260 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 73 2023-12-19T17:58:01.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.258+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1cc40658e0 msgr2=0x7f1cc4067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.258+0000 7f1ce6c2d640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1cc40658e0 0x7f1cc4067da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40ad000 tx=0x7f1cd40a6000 comp rx=0 tx=0).stop 2023-12-19T17:58:01.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.258+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1628906232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce00b30d0 msgr2=0x7f1ce0165e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.258+0000 7f1ce6c2d640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce00b30d0 0x7f1ce0165e60 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00c520 tx=0x7f1ccc00c9e0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.258+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1628906232 shutdown_connections 2023-12-19T17:58:01.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.258+0000 7f1ce6c2d640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1cc40658e0 0x7f1cc4067da0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.258+0000 7f1ce6c2d640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ce015c7f0 0x7f1ce00ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.258+0000 7f1ce6c2d640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ce00b30d0 0x7f1ce0165e60 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.258+0000 7f1ce6c2d640 1 --2- 172.21.15.26:0/1628906232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ce00adb80 0x7f1ce0151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.258+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1628906232 >> 172.21.15.26:0/1628906232 conn(0x7f1ce00658e0 msgr2=0x7f1ce0065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:01.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.258+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1628906232 shutdown_connections 2023-12-19T17:58:01.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.258+0000 7f1ce6c2d640 1 -- 172.21.15.26:0/1628906232 wait complete. 2023-12-19T17:58:01.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:58:01.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.422+0000 7fee9f59e640 1 Processor -- start 2023-12-19T17:58:01.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.422+0000 7fee9f59e640 1 -- start start 2023-12-19T17:58:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee9f59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feea015ca10 0x7feea00b8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee9f59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feea00b4b50 0x7feea00b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee9f59e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea00b5200 con 0x7feea00b8740 2023-12-19T17:58:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee9f59e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea00b5cb0 con 0x7feea015ca10 2023-12-19T17:58:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee9f59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea00d0cc0 con 0x7feea00b4b50 2023-12-19T17:58:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee9e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feea015ca10 0x7feea00b8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee97fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feea00b8740 0x7feea00b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44812/0 (socket says 172.21.15.26:44812) 2023-12-19T17:58:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee97fff640 1 -- 172.21.15.26:0/2254504176 learned_addr learned my addr 172.21.15.26:0/2254504176 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee9e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feea015ca10 0x7feea00b8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39976/0 (socket says 172.21.15.26:39976) 2023-12-19T17:58:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee9ed9d640 1 --2- 172.21.15.26:0/2254504176 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feea00b4b50 0x7feea00b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee9e59c640 1 -- 172.21.15.26:0/2254504176 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feea00b4b50 msgr2=0x7feea00b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee9e59c640 1 --2- 172.21.15.26:0/2254504176 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feea00b4b50 0x7feea00b6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee9e59c640 1 -- 172.21.15.26:0/2254504176 >> v1:172.21.15.26:6789/0 conn(0x7feea00b8740 legacy=0x7feea00b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee9e59c640 1 -- 172.21.15.26:0/2254504176 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea00b5f50 con 0x7feea015ca10 2023-12-19T17:58:01.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.426+0000 7fee9dd9b640 1 -- 172.21.15.26:0/2254504176 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276696807 0 0) 0x7feea00b5200 con 0x7feea00b8740 2023-12-19T17:58:01.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9e59c640 1 --2- 172.21.15.26:0/2254504176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea015ca10 0x7feea00b8200 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fee900097c0 tx=0x7fee9001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145d88983fefdc20 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9dd9b640 1 -- 172.21.15.26:0/2254504176 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee900ab650 con 0x7feea015ca10 2023-12-19T17:58:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9dd9b640 1 -- 172.21.15.26:0/2254504176 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee900ab7b0 con 0x7feea015ca10 2023-12-19T17:58:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9dd9b640 1 -- 172.21.15.26:0/2254504176 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee900abb40 con 0x7feea015ca10 2023-12-19T17:58:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9f59e640 1 -- 172.21.15.26:0/2254504176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea015ca10 msgr2=0x7feea00b8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9f59e640 1 --2- 172.21.15.26:0/2254504176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea015ca10 0x7feea00b8200 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fee900097c0 tx=0x7fee9001c450 comp rx=0 tx=0).stop 2023-12-19T17:58:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9f59e640 1 -- 172.21.15.26:0/2254504176 shutdown_connections 2023-12-19T17:58:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9f59e640 1 --2- 172.21.15.26:0/2254504176 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feea00b4b50 0x7feea00b6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9f59e640 1 --2- 172.21.15.26:0/2254504176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea015ca10 0x7feea00b8200 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9f59e640 1 -- 172.21.15.26:0/2254504176 >> 172.21.15.26:0/2254504176 conn(0x7feea006a120 msgr2=0x7feea006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9f59e640 1 -- 172.21.15.26:0/2254504176 shutdown_connections 2023-12-19T17:58:01.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9f59e640 1 -- 172.21.15.26:0/2254504176 wait complete. 2023-12-19T17:58:01.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9f59e640 1 Processor -- start 2023-12-19T17:58:01.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.430+0000 7fee9f59e640 1 -- start start 2023-12-19T17:58:01.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea00b4b50 0x7feea00c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9f59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feea00b8740 0x7feea0147b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9f59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feea015ca10 0x7feea00c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9f59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea0148d70 con 0x7feea00b4b50 2023-12-19T17:58:01.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9f59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea00d1860 con 0x7feea015ca10 2023-12-19T17:58:01.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9f59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea00d1b40 con 0x7feea00b8740 2023-12-19T17:58:01.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea00b4b50 0x7feea00c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea00b4b50 0x7feea00c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39982/0 (socket says 172.21.15.26:39982) 2023-12-19T17:58:01.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9e59c640 1 -- 172.21.15.26:0/4214325142 learned_addr learned my addr 172.21.15.26:0/4214325142 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:01.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee97fff640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feea00b8740 0x7feea0147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9ed9d640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feea015ca10 0x7feea00c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9e59c640 1 -- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feea00b8740 msgr2=0x7feea0147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9e59c640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feea00b8740 0x7feea0147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9e59c640 1 -- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feea015ca10 msgr2=0x7feea00c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9e59c640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feea015ca10 0x7feea00c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9e59c640 1 -- 172.21.15.26:0/4214325142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee90009420 con 0x7feea00b4b50 2023-12-19T17:58:01.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9ed9d640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feea015ca10 0x7feea00c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:01.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9e59c640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea00b4b50 0x7feea00c8d10 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fee900ad040 tx=0x7fee9001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:01.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee96ffd640 1 -- 172.21.15.26:0/4214325142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee900bf610 con 0x7feea00b4b50 2023-12-19T17:58:01.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee96ffd640 1 -- 172.21.15.26:0/4214325142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee900bf770 con 0x7feea00b4b50 2023-12-19T17:58:01.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee96ffd640 1 -- 172.21.15.26:0/4214325142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee900bfb20 con 0x7feea00b4b50 2023-12-19T17:58:01.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9f59e640 1 -- 172.21.15.26:0/4214325142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea00d1e20 con 0x7feea00b4b50 2023-12-19T17:58:01.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9f59e640 1 -- 172.21.15.26:0/4214325142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea00c9250 con 0x7feea00b4b50 2023-12-19T17:58:01.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee9f59e640 1 -- 172.21.15.26:0/4214325142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea00d2010 con 0x7feea00b4b50 2023-12-19T17:58:01.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.434+0000 7fee96ffd640 1 -- 172.21.15.26:0/4214325142 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 87) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fee900b5400 con 0x7feea00b4b50 2023-12-19T17:58:01.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.438+0000 7fee96ffd640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fee800658b0 0x7fee80067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.438+0000 7fee96ffd640 1 -- 172.21.15.26:0/4214325142 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 7043+0+0 (secure 0 0 0) 0x7fee9011e2f0 con 0x7feea00b4b50 2023-12-19T17:58:01.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.438+0000 7fee97fff640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fee800658b0 0x7fee80067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.438+0000 7fee97fff640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fee800658b0 0x7fee80067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7feea00d0ba0 tx=0x7fee88007b60 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:01.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.442+0000 7fee96ffd640 1 -- 172.21.15.26:0/4214325142 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fee900ebb00 con 0x7feea00b4b50 2023-12-19T17:58:01.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.598+0000 7fee9f59e640 1 -- 172.21.15.26:0/4214325142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feea0067400 con 0x7feea00b4b50 2023-12-19T17:58:01.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.602+0000 7fee96ffd640 1 -- 172.21.15.26:0/4214325142 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+18090 (secure 0 0 0) 0x7fee900ef120 con 0x7feea00b4b50 2023-12-19T17:58:01.604 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:01.604 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":165,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:57:59.565382+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:57:38.188932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T17:57:39.438347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"163","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3171778537":"2023-12-20T17:57:59.565333+0000","172.21.15.160:0/752855480":"2023-12-20T17:57:59.565333+0000","172.21.15.160:0/2457683375":"2023-12-20T17:57:59.565333+0000","172.21.15.160:0/1089394066":"2023-12-20T17:57:59.565333+0000","172.21.15.160:0/1221875420":"2023-12-20T17:57:52.397071+0000","172.21.15.160:6833/38098":"2023-12-20T17:57:52.397071+0000","172.21.15.160:6832/38098":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/1668583613":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/1848495251":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/2804752681":"2023-12-20T17:57:59.565333+0000","172.21.15.160:0/3041540379":"2023-12-20T17:57:59.565333+0000","172.21.15.160:0/92169616":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/2374756181":"2023-12-20T17:57:52.397071+0000","172.21.15.160:0/1748726321":"2023-12-20T17:57:52.397071+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:58:01.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.602+0000 7fee9f59e640 1 -- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fee800658b0 msgr2=0x7fee80067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.602+0000 7fee9f59e640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fee800658b0 0x7fee80067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7feea00d0ba0 tx=0x7fee88007b60 comp rx=0 tx=0).stop 2023-12-19T17:58:01.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.602+0000 7fee9f59e640 1 -- 172.21.15.26:0/4214325142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea00b4b50 msgr2=0x7feea00c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.602+0000 7fee9f59e640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea00b4b50 0x7feea00c8d10 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fee900ad040 tx=0x7fee9001ac90 comp rx=0 tx=0).stop 2023-12-19T17:58:01.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.602+0000 7fee9f59e640 1 -- 172.21.15.26:0/4214325142 shutdown_connections 2023-12-19T17:58:01.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.602+0000 7fee9f59e640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feea015ca10 0x7feea00c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.602+0000 7fee9f59e640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fee800658b0 0x7fee80067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.602+0000 7fee9f59e640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feea00b8740 0x7feea0147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.602+0000 7fee9f59e640 1 --2- 172.21.15.26:0/4214325142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feea00b4b50 0x7feea00c8d10 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.602+0000 7fee9f59e640 1 -- 172.21.15.26:0/4214325142 >> 172.21.15.26:0/4214325142 conn(0x7feea006a120 msgr2=0x7feea00cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:01.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.602+0000 7fee9f59e640 1 -- 172.21.15.26:0/4214325142 shutdown_connections 2023-12-19T17:58:01.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.602+0000 7fee9f59e640 1 -- 172.21.15.26:0/4214325142 wait complete. 2023-12-19T17:58:01.627 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T17:58:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.770+0000 7f8773466640 1 Processor -- start 2023-12-19T17:58:01.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.770+0000 7f8773466640 1 -- start start 2023-12-19T17:58:01.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8773466640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f876c155030 0x7f876c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8773466640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f876c15f370 0x7f876c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8773466640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c06c970 con 0x7f876c0ce740 2023-12-19T17:58:01.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8773466640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f876c15b1f0 con 0x7f876c15f370 2023-12-19T17:58:01.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8773466640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f876c15b4d0 con 0x7f876c155030 2023-12-19T17:58:01.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8770a01640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f876c15f370 0x7f876c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8770a01640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f876c15f370 0x7f876c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39998/0 (socket says 172.21.15.26:39998) 2023-12-19T17:58:01.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8770a01640 1 -- 172.21.15.26:0/1150255456 learned_addr learned my addr 172.21.15.26:0/1150255456 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:01.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8771202640 1 --2- 172.21.15.26:0/1150255456 >> v2:172.21.15.160:3300/0 conn(0x7f876c155030 0x7f876c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8770a01640 1 -- 172.21.15.26:0/1150255456 >> v2:172.21.15.160:3300/0 conn(0x7f876c155030 msgr2=0x7f876c15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8770a01640 1 --2- 172.21.15.26:0/1150255456 >> v2:172.21.15.160:3300/0 conn(0x7f876c155030 0x7f876c15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8770a01640 1 -- 172.21.15.26:0/1150255456 >> v1:172.21.15.26:6789/0 conn(0x7f876c0ce740 legacy=0x7f876c0ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8770a01640 1 -- 172.21.15.26:0/1150255456 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c15b7b0 con 0x7f876c15f370 2023-12-19T17:58:01.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8763fff640 1 -- 172.21.15.26:0/1150255456 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616383265 0 0) 0x7f876c06c970 con 0x7f876c0ce740 2023-12-19T17:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.774+0000 7f8770a01640 1 --2- 172.21.15.26:0/1150255456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f876c15f370 0x7f876c06c080 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f876c0cf4a0 tx=0x7f875c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65fdf13c6e186a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8763fff640 1 -- 172.21.15.26:0/1150255456 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c0ab620 con 0x7f876c15f370 2023-12-19T17:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8763fff640 1 -- 172.21.15.26:0/1150255456 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c0ab780 con 0x7f876c15f370 2023-12-19T17:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8763fff640 1 -- 172.21.15.26:0/1150255456 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c0abb10 con 0x7f876c15f370 2023-12-19T17:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 -- 172.21.15.26:0/1150255456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f876c15f370 msgr2=0x7f876c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 --2- 172.21.15.26:0/1150255456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f876c15f370 0x7f876c06c080 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f876c0cf4a0 tx=0x7f875c01c500 comp rx=0 tx=0).stop 2023-12-19T17:58:01.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 -- 172.21.15.26:0/1150255456 shutdown_connections 2023-12-19T17:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 --2- 172.21.15.26:0/1150255456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f876c15f370 0x7f876c06c080 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 --2- 172.21.15.26:0/1150255456 >> v2:172.21.15.160:3300/0 conn(0x7f876c155030 0x7f876c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 -- 172.21.15.26:0/1150255456 >> 172.21.15.26:0/1150255456 conn(0x7f876c0658e0 msgr2=0x7f876c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 -- 172.21.15.26:0/1150255456 shutdown_connections 2023-12-19T17:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 -- 172.21.15.26:0/1150255456 wait complete. 2023-12-19T17:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 Processor -- start 2023-12-19T17:58:01.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 -- start start 2023-12-19T17:58:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f876c0ce740 0x7f876c149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f876c155030 0x7f876c148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f876c15f370 0x7f876c146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c14b790 con 0x7f876c0ce740 2023-12-19T17:58:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c0b1890 con 0x7f876c15f370 2023-12-19T17:58:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.778+0000 7f8773466640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876c0b1b70 con 0x7f876c155030 2023-12-19T17:58:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8771202640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f876c0ce740 0x7f876c149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8771202640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f876c0ce740 0x7f876c149c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40012/0 (socket says 172.21.15.26:40012) 2023-12-19T17:58:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8771202640 1 -- 172.21.15.26:0/2369522805 learned_addr learned my addr 172.21.15.26:0/2369522805 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:01.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8771a03640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f876c15f370 0x7f876c146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8770a01640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f876c155030 0x7f876c148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8771202640 1 -- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f876c155030 msgr2=0x7f876c148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8771202640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f876c155030 0x7f876c148220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8771202640 1 -- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f876c15f370 msgr2=0x7f876c146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8771202640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f876c15f370 0x7f876c146ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8771202640 1 -- 172.21.15.26:0/2369522805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c0094d0 con 0x7f876c0ce740 2023-12-19T17:58:01.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8771202640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f876c0ce740 0x7f876c149c30 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f876800e930 tx=0x7f876800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8761ffb640 1 -- 172.21.15.26:0/2369522805 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768009690 con 0x7f876c0ce740 2023-12-19T17:58:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8761ffb640 1 -- 172.21.15.26:0/2369522805 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876800ca00 con 0x7f876c0ce740 2023-12-19T17:58:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8761ffb640 1 -- 172.21.15.26:0/2369522805 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876800cd70 con 0x7f876c0ce740 2023-12-19T17:58:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8773466640 1 -- 172.21.15.26:0/2369522805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876c0cf0e0 con 0x7f876c0ce740 2023-12-19T17:58:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8773466640 1 -- 172.21.15.26:0/2369522805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876c14ba70 con 0x7f876c0ce740 2023-12-19T17:58:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8773466640 1 -- 172.21.15.26:0/2369522805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876c074270 con 0x7f876c0ce740 2023-12-19T17:58:01.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.782+0000 7f8761ffb640 1 -- 172.21.15.26:0/2369522805 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 88) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f8768013470 con 0x7f876c0ce740 2023-12-19T17:58:01.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.786+0000 7f8761ffb640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8750065af0 0x7f8750067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:01.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.786+0000 7f8761ffb640 1 -- 172.21.15.26:0/2369522805 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 7043+0+0 (secure 0 0 0) 0x7f87680839e0 con 0x7f876c0ce740 2023-12-19T17:58:01.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.786+0000 7f8770a01640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8750065af0 0x7f8750067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:01.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.786+0000 7f8770a01640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8750065af0 0x7f8750067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f875c0af000 tx=0x7f875c0a7000 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:01.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.790+0000 7f8761ffb640 1 -- 172.21.15.26:0/2369522805 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8768050ec0 con 0x7f876c0ce740 2023-12-19T17:58:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.978+0000 7f8773466640 1 -- 172.21.15.26:0/2369522805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f876c06c3a0 con 0x7f876c0ce740 2023-12-19T17:58:01.984 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T17:58:01.982+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T17:58:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.990+0000 7f8761ffb640 1 -- 172.21.15.26:0/2369522805 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8768021090 con 0x7f876c0ce740 2023-12-19T17:58:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T17:58:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.994+0000 7f8773466640 1 -- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8750065af0 msgr2=0x7f8750067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.994+0000 7f8773466640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8750065af0 0x7f8750067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f875c0af000 tx=0x7f875c0a7000 comp rx=0 tx=0).stop 2023-12-19T17:58:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.994+0000 7f8773466640 1 -- 172.21.15.26:0/2369522805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f876c0ce740 msgr2=0x7f876c149c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.994+0000 7f8773466640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f876c0ce740 0x7f876c149c30 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f876800e930 tx=0x7f876800edf0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.994+0000 7f8773466640 1 -- 172.21.15.26:0/2369522805 shutdown_connections 2023-12-19T17:58:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.994+0000 7f8773466640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f876c15f370 0x7f876c146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.994+0000 7f8773466640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8750065af0 0x7f8750067fb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.994+0000 7f8773466640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f876c155030 0x7f876c148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.994+0000 7f8773466640 1 --2- 172.21.15.26:0/2369522805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f876c0ce740 0x7f876c149c30 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.994+0000 7f8773466640 1 -- 172.21.15.26:0/2369522805 >> 172.21.15.26:0/2369522805 conn(0x7f876c0658e0 msgr2=0x7f876c15ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.994+0000 7f8773466640 1 -- 172.21.15.26:0/2369522805 shutdown_connections 2023-12-19T17:58:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:01.994+0000 7f8773466640 1 -- 172.21.15.26:0/2369522805 wait complete. 2023-12-19T17:58:02.003 INFO:tasks.ceph.mds.c.smithi026.stderr: -3874> 2023-12-19T17:57:46.011+0000 7fd100fdf640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T17:58:02.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T17:58:02.053 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T17:58:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.158+0000 7f670d1c0640 1 Processor -- start 2023-12-19T17:58:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.158+0000 7f670d1c0640 1 -- start start 2023-12-19T17:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.162+0000 7f670d1c0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f67081585f0 0x7f67080cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.162+0000 7f670d1c0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67080cb600 0x7f67080cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.162+0000 7f670d1c0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67080cba50 con 0x7f67080cb150 2023-12-19T17:58:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.162+0000 7f670d1c0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67080b47a0 con 0x7f67081585f0 2023-12-19T17:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.162+0000 7f670d1c0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670815b1b0 con 0x7f67080cb600 2023-12-19T17:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.162+0000 7f6706d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f67081585f0 0x7f67080cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.162+0000 7f6706d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f67081585f0 0x7f67080cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40032/0 (socket says 172.21.15.26:40032) 2023-12-19T17:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.162+0000 7f6706d9d640 1 -- 172.21.15.26:0/314727683 learned_addr learned my addr 172.21.15.26:0/314727683 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.162+0000 7f6706d9d640 1 -- 172.21.15.26:0/314727683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67080cb600 msgr2=0x7f67080cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.162+0000 7f6706d9d640 1 --2- 172.21.15.26:0/314727683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67080cb600 0x7f67080cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.162+0000 7f6706d9d640 1 -- 172.21.15.26:0/314727683 >> v1:172.21.15.26:6789/0 conn(0x7f67080cb150 legacy=0x7f67080cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:02.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.162+0000 7f6706d9d640 1 -- 172.21.15.26:0/314727683 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670815b490 con 0x7f67081585f0 2023-12-19T17:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f6706d9d640 1 --2- 172.21.15.26:0/314727683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67081585f0 0x7f67080cdce0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009870 tx=0x7f66f801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f8b6ae5560711b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f6705d9b640 1 -- 172.21.15.26:0/314727683 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f80ab680 con 0x7f67081585f0 2023-12-19T17:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f6705d9b640 1 -- 172.21.15.26:0/314727683 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f80ab7e0 con 0x7f67081585f0 2023-12-19T17:58:02.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f6705d9b640 1 -- 172.21.15.26:0/314727683 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f80abb70 con 0x7f67081585f0 2023-12-19T17:58:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f670d1c0640 1 -- 172.21.15.26:0/314727683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67081585f0 msgr2=0x7f67080cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f670d1c0640 1 --2- 172.21.15.26:0/314727683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67081585f0 0x7f67080cdce0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009870 tx=0x7f66f801c480 comp rx=0 tx=0).stop 2023-12-19T17:58:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f670d1c0640 1 -- 172.21.15.26:0/314727683 shutdown_connections 2023-12-19T17:58:02.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f670d1c0640 1 --2- 172.21.15.26:0/314727683 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67080cb600 0x7f67080cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f670d1c0640 1 --2- 172.21.15.26:0/314727683 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67081585f0 0x7f67080cdce0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f670d1c0640 1 -- 172.21.15.26:0/314727683 >> 172.21.15.26:0/314727683 conn(0x7f6708066160 msgr2=0x7f67080b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f670d1c0640 1 -- 172.21.15.26:0/314727683 shutdown_connections 2023-12-19T17:58:02.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f670d1c0640 1 -- 172.21.15.26:0/314727683 wait complete. 2023-12-19T17:58:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f670d1c0640 1 Processor -- start 2023-12-19T17:58:02.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.166+0000 7f670d1c0640 1 -- start start 2023-12-19T17:58:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f670d1c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67080cb150 0x7f670814ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f670d1c0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67080cb600 0x7f6708157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f670d1c0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67081585f0 0x7f6708157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f670d1c0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67080b3390 con 0x7f67080cb150 2023-12-19T17:58:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f670d1c0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67080b22f0 con 0x7f67080cb600 2023-12-19T17:58:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f670d1c0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67080b25d0 con 0x7f67081585f0 2023-12-19T17:58:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f6706d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67080cb150 0x7f670814ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f6706d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67080cb150 0x7f670814ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40046/0 (socket says 172.21.15.26:40046) 2023-12-19T17:58:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f6706d9d640 1 -- 172.21.15.26:0/3877712598 learned_addr learned my addr 172.21.15.26:0/3877712598 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f670759e640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67081585f0 0x7f6708157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f670659c640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67080cb600 0x7f6708157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f6706d9d640 1 -- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67081585f0 msgr2=0x7f6708157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f6706d9d640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67081585f0 0x7f6708157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f6706d9d640 1 -- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67080cb600 msgr2=0x7f6708157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f6706d9d640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67080cb600 0x7f6708157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f6706d9d640 1 -- 172.21.15.26:0/3877712598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f80094d0 con 0x7f67080cb150 2023-12-19T17:58:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f6706d9d640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67080cb150 0x7f670814ebd0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f66f80ad040 tx=0x7f66f801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f66ef7fe640 1 -- 172.21.15.26:0/3877712598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f80b4800 con 0x7f67080cb150 2023-12-19T17:58:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f66ef7fe640 1 -- 172.21.15.26:0/3877712598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f80b4960 con 0x7f67080cb150 2023-12-19T17:58:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f66ef7fe640 1 -- 172.21.15.26:0/3877712598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f80b4d10 con 0x7f67080cb150 2023-12-19T17:58:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f670d1c0640 1 -- 172.21.15.26:0/3877712598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670815c3e0 con 0x7f67080cb150 2023-12-19T17:58:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f670d1c0640 1 -- 172.21.15.26:0/3877712598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67080b2b90 con 0x7f67080cb150 2023-12-19T17:58:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.170+0000 7f670d1c0640 1 -- 172.21.15.26:0/3877712598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6708074270 con 0x7f67080cb150 2023-12-19T17:58:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.174+0000 7f66ef7fe640 1 -- 172.21.15.26:0/3877712598 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 88) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f66f80c7440 con 0x7f67080cb150 2023-12-19T17:58:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.174+0000 7f66ef7fe640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f66f4065b20 0x7f66f4067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.174+0000 7f66ef7fe640 1 -- 172.21.15.26:0/3877712598 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 7129+0+0 (secure 0 0 0) 0x7f66f811e860 con 0x7f67080cb150 2023-12-19T17:58:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.174+0000 7f670659c640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f66f4065b20 0x7f66f4067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.174+0000 7f670659c640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f66f4065b20 0x7f66f4067fe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6708156670 tx=0x7f66fc009040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.178+0000 7f66ef7fe640 1 -- 172.21.15.26:0/3877712598 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f66f80ebb20 con 0x7f67080cb150 2023-12-19T17:58:02.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.366+0000 7f670d1c0640 1 -- 172.21.15.26:0/3877712598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f67080b2710 con 0x7f67080cb150 2023-12-19T17:58:02.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.994+0000 7f66ef7fe640 1 -- 172.21.15.26:0/3877712598 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7f67080b2710 con 0x7f67080cb150 2023-12-19T17:58:03.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.994+0000 7f670d1c0640 1 -- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f66f4065b20 msgr2=0x7f66f4067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:03.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.994+0000 7f670d1c0640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f66f4065b20 0x7f66f4067fe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6708156670 tx=0x7f66fc009040 comp rx=0 tx=0).stop 2023-12-19T17:58:03.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.998+0000 7f670d1c0640 1 -- 172.21.15.26:0/3877712598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67080cb150 msgr2=0x7f670814ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:03.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.998+0000 7f670d1c0640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67080cb150 0x7f670814ebd0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f66f80ad040 tx=0x7f66f801acc0 comp rx=0 tx=0).stop 2023-12-19T17:58:03.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.998+0000 7f670d1c0640 1 -- 172.21.15.26:0/3877712598 shutdown_connections 2023-12-19T17:58:03.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.998+0000 7f670d1c0640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f66f4065b20 0x7f66f4067fe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:03.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.998+0000 7f670d1c0640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67081585f0 0x7f6708157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:03.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.998+0000 7f670d1c0640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67080cb600 0x7f6708157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:03.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.998+0000 7f670d1c0640 1 --2- 172.21.15.26:0/3877712598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67080cb150 0x7f670814ebd0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:03.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.998+0000 7f670d1c0640 1 -- 172.21.15.26:0/3877712598 >> 172.21.15.26:0/3877712598 conn(0x7f6708066160 msgr2=0x7f670815a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:03.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.998+0000 7f670d1c0640 1 -- 172.21.15.26:0/3877712598 shutdown_connections 2023-12-19T17:58:03.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:02.998+0000 7f670d1c0640 1 -- 172.21.15.26:0/3877712598 wait complete. 2023-12-19T17:58:03.023 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T17:58:03.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.210+0000 7fe35ed6d640 1 Processor -- start 2023-12-19T17:58:03.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.210+0000 7fe35ed6d640 1 -- start start 2023-12-19T17:58:03.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.210+0000 7fe35ed6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe358159830 0x7fe35815d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:03.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.210+0000 7fe35ed6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35815e3d0 con 0x7fe3580c9310 2023-12-19T17:58:03.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.210+0000 7fe35ed6d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35815de10 con 0x7fe3580cb110 2023-12-19T17:58:03.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.210+0000 7fe35ed6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35806f7c0 con 0x7fe358159830 2023-12-19T17:58:03.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.210+0000 7fe357fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe3580c9310 0x7fe3580cc430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44830/0 (socket says 172.21.15.26:44830) 2023-12-19T17:58:03.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.210+0000 7fe357fff640 1 -- 172.21.15.26:0/1631039667 learned_addr learned my addr 172.21.15.26:0/1631039667 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:03.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.214+0000 7fe35cb09640 1 --2- 172.21.15.26:0/1631039667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe358159830 0x7fe35815d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:03.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.214+0000 7fe3577fe640 1 -- 172.21.15.26:0/1631039667 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278143771 0 0) 0x7fe35815e3d0 con 0x7fe3580c9310 2023-12-19T17:58:03.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.214+0000 7fe3577fe640 1 -- 172.21.15.26:0/1631039667 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe348005740 con 0x7fe3580c9310 2023-12-19T17:58:03.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.214+0000 7fe3577fe640 1 -- 172.21.15.26:0/1631039667 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2977299928 0 0) 0x7fe35815de10 con 0x7fe3580cb110 2023-12-19T17:58:03.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.214+0000 7fe3577fe640 1 -- 172.21.15.26:0/1631039667 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe35815e3d0 con 0x7fe3580cb110 2023-12-19T17:58:03.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.214+0000 7fe35cb09640 1 -- 172.21.15.26:0/1631039667 >> v1:172.21.15.160:6789/0 conn(0x7fe3580cb110 legacy=0x7fe3580701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:03.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.214+0000 7fe35cb09640 1 -- 172.21.15.26:0/1631039667 >> v1:172.21.15.26:6789/0 conn(0x7fe3580c9310 legacy=0x7fe3580cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:03.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.214+0000 7fe3577fe640 1 -- 172.21.15.26:0/1631039667 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1964723794 0 0) 0x7fe348005740 con 0x7fe3580c9310 2023-12-19T17:58:03.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.214+0000 7fe35cb09640 1 -- 172.21.15.26:0/1631039667 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35815e0f0 con 0x7fe358159830 2023-12-19T17:58:03.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.214+0000 7fe35cb09640 1 --2- 172.21.15.26:0/1631039667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe358159830 0x7fe35815d4e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0097f0 tx=0x7fe34c01c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3ae22a9ec167d2c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:03.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe3577fe640 1 -- 172.21.15.26:0/1631039667 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe344001680 con 0x7fe358159830 2023-12-19T17:58:03.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe3577fe640 1 -- 172.21.15.26:0/1631039667 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c0a7050 con 0x7fe358159830 2023-12-19T17:58:03.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe3577fe640 1 -- 172.21.15.26:0/1631039667 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0ab600 con 0x7fe358159830 2023-12-19T17:58:03.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1631039667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe358159830 msgr2=0x7fe35815d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:03.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 --2- 172.21.15.26:0/1631039667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe358159830 0x7fe35815d4e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0097f0 tx=0x7fe34c01c350 comp rx=0 tx=0).stop 2023-12-19T17:58:03.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1631039667 shutdown_connections 2023-12-19T17:58:03.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 --2- 172.21.15.26:0/1631039667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe358159830 0x7fe35815d4e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:03.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1631039667 >> 172.21.15.26:0/1631039667 conn(0x7fe3580b0270 msgr2=0x7fe3580b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:03.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1631039667 shutdown_connections 2023-12-19T17:58:03.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1631039667 wait complete. 2023-12-19T17:58:03.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 Processor -- start 2023-12-19T17:58:03.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 -- start start 2023-12-19T17:58:03.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3580c9310 0x7fe35814df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:03.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3580cb110 0x7fe35815b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:03.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe358159830 0x7fe35814b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:03.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35815baa0 con 0x7fe3580c9310 2023-12-19T17:58:03.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35815bd80 con 0x7fe3580cb110 2023-12-19T17:58:03.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.218+0000 7fe35ed6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35815c060 con 0x7fe358159830 2023-12-19T17:58:03.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe35cb09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3580c9310 0x7fe35814df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:03.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe35cb09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3580c9310 0x7fe35814df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40054/0 (socket says 172.21.15.26:40054) 2023-12-19T17:58:03.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe35cb09640 1 -- 172.21.15.26:0/1733400378 learned_addr learned my addr 172.21.15.26:0/1733400378 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:03.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe35d30a640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe358159830 0x7fe35814b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:03.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe357fff640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3580cb110 0x7fe35815b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:03.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe35cb09640 1 -- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe358159830 msgr2=0x7fe35814b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:03.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe35cb09640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe358159830 0x7fe35814b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:03.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe35cb09640 1 -- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3580cb110 msgr2=0x7fe35815b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:03.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe35cb09640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3580cb110 0x7fe35815b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:03.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe35cb09640 1 -- 172.21.15.26:0/1733400378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c009420 con 0x7fe3580c9310 2023-12-19T17:58:03.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe35cb09640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3580c9310 0x7fe35814df10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fe34c019f80 tx=0x7fe34c0190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:03.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe3557fa640 1 -- 172.21.15.26:0/1733400378 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c003a30 con 0x7fe3580c9310 2023-12-19T17:58:03.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe3557fa640 1 -- 172.21.15.26:0/1733400378 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c003b90 con 0x7fe3580c9310 2023-12-19T17:58:03.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1733400378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3580cbab0 con 0x7fe3580c9310 2023-12-19T17:58:03.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe3557fa640 1 -- 172.21.15.26:0/1733400378 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0ab400 con 0x7fe3580c9310 2023-12-19T17:58:03.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1733400378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35815c620 con 0x7fe3580c9310 2023-12-19T17:58:03.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1733400378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35815c810 con 0x7fe3580c9310 2023-12-19T17:58:03.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.222+0000 7fe3557fa640 1 -- 172.21.15.26:0/1733400378 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 88) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fe34c0b2040 con 0x7fe3580c9310 2023-12-19T17:58:03.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.226+0000 7fe3557fa640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe33c065af0 0x7fe33c067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:03.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.226+0000 7fe3557fa640 1 -- 172.21.15.26:0/1733400378 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 7129+0+0 (secure 0 0 0) 0x7fe34c11d5d0 con 0x7fe3580c9310 2023-12-19T17:58:03.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.226+0000 7fe357fff640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe33c065af0 0x7fe33c067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:03.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.226+0000 7fe357fff640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe33c065af0 0x7fe33c067fb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe3580c4e70 tx=0x7fe3440016a0 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:03.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.230+0000 7fe3557fa640 1 -- 172.21.15.26:0/1733400378 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe34c0ea9c0 con 0x7fe3580c9310 2023-12-19T17:58:03.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:03.386+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1733400378 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe358074270 con 0x7fe3580c9310 2023-12-19T17:58:04.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.002+0000 7fe3557fa640 1 -- 172.21.15.26:0/1733400378 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe34c0ead90 con 0x7fe3580c9310 2023-12-19T17:58:04.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.006+0000 7fe3557fa640 1 -- 172.21.15.26:0/1733400378 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v167) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe34c0ea9c0 con 0x7fe3580c9310 2023-12-19T17:58:04.008 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T17:58:04.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.006+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe33c065af0 msgr2=0x7fe33c067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:04.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.010+0000 7fe35ed6d640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe33c065af0 0x7fe33c067fb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe3580c4e70 tx=0x7fe3440016a0 comp rx=0 tx=0).stop 2023-12-19T17:58:04.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.010+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1733400378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3580c9310 msgr2=0x7fe35814df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:04.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.010+0000 7fe35ed6d640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3580c9310 0x7fe35814df10 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fe34c019f80 tx=0x7fe34c0190f0 comp rx=0 tx=0).stop 2023-12-19T17:58:04.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.010+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1733400378 shutdown_connections 2023-12-19T17:58:04.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.010+0000 7fe35ed6d640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe33c065af0 0x7fe33c067fb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:04.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.010+0000 7fe35ed6d640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe358159830 0x7fe35814b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:04.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.010+0000 7fe35ed6d640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3580cb110 0x7fe35815b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:04.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.010+0000 7fe35ed6d640 1 --2- 172.21.15.26:0/1733400378 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3580c9310 0x7fe35814df10 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:04.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.010+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1733400378 >> 172.21.15.26:0/1733400378 conn(0x7fe3580b0270 msgr2=0x7fe35814ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:04.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.010+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1733400378 shutdown_connections 2023-12-19T17:58:04.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.010+0000 7fe35ed6d640 1 -- 172.21.15.26:0/1733400378 wait complete. 2023-12-19T17:58:04.043 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T17:58:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd3718640 1 Processor -- start 2023-12-19T17:58:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd3718640 1 -- start start 2023-12-19T17:58:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd3718640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3bcc159830 0x7f3bcc15d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd3718640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3bcc0cb110 0x7f3bcc0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:04.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd3718640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bcc15e3d0 con 0x7f3bcc0c9310 2023-12-19T17:58:04.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd3718640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc06f7c0 con 0x7f3bcc0cb110 2023-12-19T17:58:04.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd3718640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0ca2b0 con 0x7f3bcc159830 2023-12-19T17:58:04.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd0cb3640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3bcc0c9310 0x7f3bcc0cc430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44844/0 (socket says 172.21.15.26:44844) 2023-12-19T17:58:04.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd0cb3640 1 -- 172.21.15.26:0/3314243320 learned_addr learned my addr 172.21.15.26:0/3314243320 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:04.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd14b4640 1 --2- 172.21.15.26:0/3314243320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3bcc159830 0x7f3bcc15d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:04.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd1cb5640 1 --2- 172.21.15.26:0/3314243320 >> v2:172.21.15.160:3300/0 conn(0x7f3bcc0cb110 0x7f3bcc0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:04.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bc3fff640 1 -- 172.21.15.26:0/3314243320 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598342188 0 0) 0x7f3bcc15e3d0 con 0x7f3bcc0c9310 2023-12-19T17:58:04.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bc3fff640 1 -- 172.21.15.26:0/3314243320 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bb8003530 con 0x7f3bcc0c9310 2023-12-19T17:58:04.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd1cb5640 1 -- 172.21.15.26:0/3314243320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3bcc159830 msgr2=0x7f3bcc15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:04.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd1cb5640 1 --2- 172.21.15.26:0/3314243320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3bcc159830 0x7f3bcc15d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:04.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd1cb5640 1 -- 172.21.15.26:0/3314243320 >> v1:172.21.15.26:6789/0 conn(0x7f3bcc0c9310 legacy=0x7f3bcc0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:04.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd1cb5640 1 -- 172.21.15.26:0/3314243320 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc0cba80 con 0x7f3bcc0cb110 2023-12-19T17:58:04.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.202+0000 7f3bd14b4640 1 --2- 172.21.15.26:0/3314243320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3bcc159830 0x7f3bcc15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:58:04.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.206+0000 7f3bd1cb5640 1 --2- 172.21.15.26:0/3314243320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bcc0cb110 0x7f3bcc0cb540 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009d90 tx=0x7f3bc401c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ef70e971fe2822e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:04.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.206+0000 7f3bc3fff640 1 -- 172.21.15.26:0/3314243320 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc40a5870 con 0x7f3bcc0cb110 2023-12-19T17:58:04.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.206+0000 7f3bc3fff640 1 -- 172.21.15.26:0/3314243320 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc40a59d0 con 0x7f3bcc0cb110 2023-12-19T17:58:04.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.206+0000 7f3bc3fff640 1 -- 172.21.15.26:0/3314243320 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc40a5d40 con 0x7f3bcc0cb110 2023-12-19T17:58:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.206+0000 7f3bd3718640 1 -- 172.21.15.26:0/3314243320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bcc0cb110 msgr2=0x7f3bcc0cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.206+0000 7f3bd3718640 1 --2- 172.21.15.26:0/3314243320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bcc0cb110 0x7f3bcc0cb540 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009d90 tx=0x7f3bc401c8c0 comp rx=0 tx=0).stop 2023-12-19T17:58:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.206+0000 7f3bd3718640 1 -- 172.21.15.26:0/3314243320 shutdown_connections 2023-12-19T17:58:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.206+0000 7f3bd3718640 1 --2- 172.21.15.26:0/3314243320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bcc0cb110 0x7f3bcc0cb540 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.206+0000 7f3bd3718640 1 --2- 172.21.15.26:0/3314243320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3bcc159830 0x7f3bcc15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.206+0000 7f3bd3718640 1 -- 172.21.15.26:0/3314243320 >> 172.21.15.26:0/3314243320 conn(0x7f3bcc0b0270 msgr2=0x7f3bcc0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.206+0000 7f3bd3718640 1 -- 172.21.15.26:0/3314243320 shutdown_connections 2023-12-19T17:58:04.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.206+0000 7f3bd3718640 1 -- 172.21.15.26:0/3314243320 wait complete. 2023-12-19T17:58:04.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd3718640 1 Processor -- start 2023-12-19T17:58:04.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd3718640 1 -- start start 2023-12-19T17:58:04.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd3718640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3bcc0c9310 0x7f3bcc150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd3718640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bcc0cb110 0x7f3bcc0cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd3718640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bcc159830 0x7f3bcc0ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd3718640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc1f3ef0 con 0x7f3bcc159830 2023-12-19T17:58:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd3718640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc1f48b0 con 0x7f3bcc0cb110 2023-12-19T17:58:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd3718640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc150f70 con 0x7f3bcc0c9310 2023-12-19T17:58:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd1cb5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bcc159830 0x7f3bcc0ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd1cb5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bcc159830 0x7f3bcc0ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40070/0 (socket says 172.21.15.26:40070) 2023-12-19T17:58:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd1cb5640 1 -- 172.21.15.26:0/2542320275 learned_addr learned my addr 172.21.15.26:0/2542320275 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd0cb3640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bcc0cb110 0x7f3bcc0cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd14b4640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3bcc0c9310 0x7f3bcc150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd1cb5640 1 -- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3bcc0c9310 msgr2=0x7f3bcc150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd1cb5640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3bcc0c9310 0x7f3bcc150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:04.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd1cb5640 1 -- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bcc0cb110 msgr2=0x7f3bcc0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:04.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd1cb5640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bcc0cb110 0x7f3bcc0cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:04.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd1cb5640 1 -- 172.21.15.26:0/2542320275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8009420 con 0x7f3bcc159830 2023-12-19T17:58:04.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd1cb5640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bcc159830 0x7f3bcc0ce4d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f3bc401af70 tx=0x7f3bc4019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bc1ffb640 1 -- 172.21.15.26:0/2542320275 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4003ba0 con 0x7f3bcc159830 2023-12-19T17:58:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bc1ffb640 1 -- 172.21.15.26:0/2542320275 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc4003d00 con 0x7f3bcc159830 2023-12-19T17:58:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bc1ffb640 1 -- 172.21.15.26:0/2542320275 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc4004240 con 0x7f3bcc159830 2023-12-19T17:58:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.210+0000 7f3bd3718640 1 -- 172.21.15.26:0/2542320275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc4009a70 con 0x7f3bcc159830 2023-12-19T17:58:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.214+0000 7f3bd3718640 1 -- 172.21.15.26:0/2542320275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc1f41f0 con 0x7f3bcc159830 2023-12-19T17:58:04.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.214+0000 7f3bd3718640 1 -- 172.21.15.26:0/2542320275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc1f4050 con 0x7f3bcc159830 2023-12-19T17:58:04.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.214+0000 7f3bc1ffb640 1 -- 172.21.15.26:0/2542320275 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3bc40b4070 con 0x7f3bcc159830 2023-12-19T17:58:04.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.214+0000 7f3bc1ffb640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3bb0065bd0 0x7f3bb0068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:04.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.214+0000 7f3bc1ffb640 1 -- 172.21.15.26:0/2542320275 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 6691+0+0 (secure 0 0 0) 0x7f3bc411e620 con 0x7f3bcc159830 2023-12-19T17:58:04.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.214+0000 7f3bd14b4640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3bb0065bd0 0x7f3bb0068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:04.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.214+0000 7f3bd14b4640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3bb0065bd0 0x7f3bb0068090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009c50 tx=0x7f3bc8009190 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:04.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.218+0000 7f3bc1ffb640 1 -- 172.21.15.26:0/2542320275 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3bcc1f4050 con 0x7f3bcc159830 2023-12-19T17:58:04.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:04.378+0000 7f3bd3718640 1 -- 172.21.15.26:0/2542320275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3bcc1f4050 con 0x7f3bcc159830 2023-12-19T17:58:05.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.022+0000 7f3bc1ffb640 1 -- 172.21.15.26:0/2542320275 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v168) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3bcc1f4050 con 0x7f3bcc159830 2023-12-19T17:58:05.029 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T17:58:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.026+0000 7f3bd3718640 1 -- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3bb0065bd0 msgr2=0x7f3bb0068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.026+0000 7f3bd3718640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3bb0065bd0 0x7f3bb0068090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3bc8009c50 tx=0x7f3bc8009190 comp rx=0 tx=0).stop 2023-12-19T17:58:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.026+0000 7f3bd3718640 1 -- 172.21.15.26:0/2542320275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bcc159830 msgr2=0x7f3bcc0ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.026+0000 7f3bd3718640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bcc159830 0x7f3bcc0ce4d0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f3bc401af70 tx=0x7f3bc4019660 comp rx=0 tx=0).stop 2023-12-19T17:58:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.026+0000 7f3bd3718640 1 -- 172.21.15.26:0/2542320275 shutdown_connections 2023-12-19T17:58:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.026+0000 7f3bd3718640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3bcc159830 0x7f3bcc0ce4d0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.026+0000 7f3bd3718640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bcc0cb110 0x7f3bcc0cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.026+0000 7f3bd3718640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3bb0065bd0 0x7f3bb0068090 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.026+0000 7f3bd3718640 1 --2- 172.21.15.26:0/2542320275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3bcc0c9310 0x7f3bcc150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.026+0000 7f3bd3718640 1 -- 172.21.15.26:0/2542320275 >> 172.21.15.26:0/2542320275 conn(0x7f3bcc0b0270 msgr2=0x7f3bcc154910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:05.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.026+0000 7f3bd3718640 1 -- 172.21.15.26:0/2542320275 shutdown_connections 2023-12-19T17:58:05.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.026+0000 7f3bd3718640 1 -- 172.21.15.26:0/2542320275 wait complete. 2023-12-19T17:58:05.053 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-19T17:58:05.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.194+0000 7f2a1a1e9640 1 Processor -- start 2023-12-19T17:58:05.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.194+0000 7f2a1a1e9640 1 -- start start 2023-12-19T17:58:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.198+0000 7f2a1a1e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a141585f0 0x7f2a140cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.198+0000 7f2a1a1e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a140cb150 0x7f2a140cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.198+0000 7f2a1a1e9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a1415b1b0 con 0x7f2a140cc310 2023-12-19T17:58:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.198+0000 7f2a1a1e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a140cbb10 con 0x7f2a140cb150 2023-12-19T17:58:05.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.198+0000 7f2a1a1e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140b47a0 con 0x7f2a141585f0 2023-12-19T17:58:05.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.198+0000 7f2a12ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a140cb150 0x7f2a140cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.198+0000 7f2a12ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a140cb150 0x7f2a140cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40074/0 (socket says 172.21.15.26:40074) 2023-12-19T17:58:05.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.198+0000 7f2a12ffd640 1 -- 172.21.15.26:0/3549559247 learned_addr learned my addr 172.21.15.26:0/3549559247 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:05.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.198+0000 7f2a137fe640 1 --2- 172.21.15.26:0/3549559247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a141585f0 0x7f2a140cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.198+0000 7f2a12ffd640 1 -- 172.21.15.26:0/3549559247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a141585f0 msgr2=0x7f2a140cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:05.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.198+0000 7f2a12ffd640 1 --2- 172.21.15.26:0/3549559247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a141585f0 0x7f2a140cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.198+0000 7f2a12ffd640 1 -- 172.21.15.26:0/3549559247 >> v1:172.21.15.160:6789/0 conn(0x7f2a140cc310 legacy=0x7f2a140cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.198+0000 7f2a12ffd640 1 -- 172.21.15.26:0/3549559247 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1415b490 con 0x7f2a140cb150 2023-12-19T17:58:05.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a12ffd640 1 --2- 172.21.15.26:0/3549559247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a140cb150 0x7f2a140cb5d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2a08009870 tx=0x7f2a0801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca30096086b27747 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:05.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a127fc640 1 -- 172.21.15.26:0/3549559247 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a080ab680 con 0x7f2a140cb150 2023-12-19T17:58:05.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a127fc640 1 -- 172.21.15.26:0/3549559247 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a080ab7e0 con 0x7f2a140cb150 2023-12-19T17:58:05.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a127fc640 1 -- 172.21.15.26:0/3549559247 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a080abb70 con 0x7f2a140cb150 2023-12-19T17:58:05.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/3549559247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a140cb150 msgr2=0x7f2a140cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a1a1e9640 1 --2- 172.21.15.26:0/3549559247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a140cb150 0x7f2a140cb5d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2a08009870 tx=0x7f2a0801c480 comp rx=0 tx=0).stop 2023-12-19T17:58:05.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/3549559247 shutdown_connections 2023-12-19T17:58:05.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a1a1e9640 1 --2- 172.21.15.26:0/3549559247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a140cb150 0x7f2a140cb5d0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a1a1e9640 1 --2- 172.21.15.26:0/3549559247 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a141585f0 0x7f2a140cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/3549559247 >> 172.21.15.26:0/3549559247 conn(0x7f2a14066160 msgr2=0x7f2a140b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:05.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/3549559247 shutdown_connections 2023-12-19T17:58:05.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/3549559247 wait complete. 2023-12-19T17:58:05.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a1a1e9640 1 Processor -- start 2023-12-19T17:58:05.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.202+0000 7f2a1a1e9640 1 -- start start 2023-12-19T17:58:05.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a1a1e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a140cb150 0x7f2a14159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a1a1e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a140cc310 0x7f2a14143a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a1a1e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a141585f0 0x7f2a1406c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a1a1e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140c7180 con 0x7f2a141585f0 2023-12-19T17:58:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a1a1e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140c6220 con 0x7f2a140cc310 2023-12-19T17:58:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a1a1e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a140c6500 con 0x7f2a140cb150 2023-12-19T17:58:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a137fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a140cb150 0x7f2a14159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a13fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a141585f0 0x7f2a1406c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a137fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a140cb150 0x7f2a14159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:43680/0 (socket says 172.21.15.26:43680) 2023-12-19T17:58:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a137fe640 1 -- 172.21.15.26:0/409218639 learned_addr learned my addr 172.21.15.26:0/409218639 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a12ffd640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a140cc310 0x7f2a14143a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a13fff640 1 -- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a140cb150 msgr2=0x7f2a14159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a13fff640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a140cb150 0x7f2a14159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a13fff640 1 -- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a140cc310 msgr2=0x7f2a14143a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a13fff640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a140cc310 0x7f2a14143a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a13fff640 1 -- 172.21.15.26:0/409218639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a080094d0 con 0x7f2a141585f0 2023-12-19T17:58:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a137fe640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a140cb150 0x7f2a14159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:58:05.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a13fff640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a141585f0 0x7f2a1406c490 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c001dc0 tx=0x7f2a0c010420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:05.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f29fbfff640 1 -- 172.21.15.26:0/409218639 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c013070 con 0x7f2a141585f0 2023-12-19T17:58:05.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f29fbfff640 1 -- 172.21.15.26:0/409218639 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0c00e2e0 con 0x7f2a141585f0 2023-12-19T17:58:05.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/409218639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1415c3e0 con 0x7f2a141585f0 2023-12-19T17:58:05.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f29fbfff640 1 -- 172.21.15.26:0/409218639 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0020d0 con 0x7f2a141585f0 2023-12-19T17:58:05.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/409218639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a140c6ac0 con 0x7f2a141585f0 2023-12-19T17:58:05.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/409218639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a140c72c0 con 0x7f2a141585f0 2023-12-19T17:58:05.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.206+0000 7f29fbfff640 1 -- 172.21.15.26:0/409218639 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2a0c010b30 con 0x7f2a141585f0 2023-12-19T17:58:05.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.210+0000 7f29fbfff640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2a00065b80 0x7f2a00068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.210+0000 7f29fbfff640 1 -- 172.21.15.26:0/409218639 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f2a0c0870c0 con 0x7f2a141585f0 2023-12-19T17:58:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.210+0000 7f2a137fe640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2a00065b80 0x7f2a00068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.210+0000 7f2a137fe640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2a00065b80 0x7f2a00068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2a04007e60 tx=0x7f2a04008040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.214+0000 7f29fbfff640 1 -- 172.21.15.26:0/409218639 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2a140c72c0 con 0x7f2a141585f0 2023-12-19T17:58:05.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.370+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/409218639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f2a140c72c0 con 0x7f2a141585f0 2023-12-19T17:58:05.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.378+0000 7f29fbfff640 1 -- 172.21.15.26:0/409218639 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v55) v1 ==== 120+0+0 (secure 0 0 0) 0x7f2a140c72c0 con 0x7f2a141585f0 2023-12-19T17:58:05.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.378+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2a00065b80 msgr2=0x7f2a00068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.378+0000 7f2a1a1e9640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2a00065b80 0x7f2a00068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2a04007e60 tx=0x7f2a04008040 comp rx=0 tx=0).stop 2023-12-19T17:58:05.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.378+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/409218639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a141585f0 msgr2=0x7f2a1406c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.378+0000 7f2a1a1e9640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a141585f0 0x7f2a1406c490 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c001dc0 tx=0x7f2a0c010420 comp rx=0 tx=0).stop 2023-12-19T17:58:05.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.378+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/409218639 shutdown_connections 2023-12-19T17:58:05.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.378+0000 7f2a1a1e9640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a141585f0 0x7f2a1406c490 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.378+0000 7f2a1a1e9640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a140cc310 0x7f2a14143a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.378+0000 7f2a1a1e9640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f2a00065b80 0x7f2a00068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.378+0000 7f2a1a1e9640 1 --2- 172.21.15.26:0/409218639 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2a140cb150 0x7f2a14159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.378+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/409218639 >> 172.21.15.26:0/409218639 conn(0x7f2a14066160 msgr2=0x7f2a14155010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.378+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/409218639 shutdown_connections 2023-12-19T17:58:05.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.378+0000 7f2a1a1e9640 1 -- 172.21.15.26:0/409218639 wait complete. 2023-12-19T17:58:05.402 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T17:58:05.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.542+0000 7f30b0cd1640 1 Processor -- start 2023-12-19T17:58:05.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.542+0000 7f30b0cd1640 1 -- start start 2023-12-19T17:58:05.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30b0cd1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30ac1599e0 0x7f30ac06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30b0cd1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f30ac15dc40 0x7f30ac0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30b0cd1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ac15e240 con 0x7f30ac0cb150 2023-12-19T17:58:05.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30b0cd1640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ac15e050 con 0x7f30ac1599e0 2023-12-19T17:58:05.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30b0cd1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ac0cbb20 con 0x7f30ac15dc40 2023-12-19T17:58:05.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30aa59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30ac1599e0 0x7f30ac06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30aa59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30ac1599e0 0x7f30ac06f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40104/0 (socket says 172.21.15.26:40104) 2023-12-19T17:58:05.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30aa59c640 1 -- 172.21.15.26:0/3661522895 learned_addr learned my addr 172.21.15.26:0/3661522895 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:05.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30aad9d640 1 --2- 172.21.15.26:0/3661522895 >> v2:172.21.15.160:3300/0 conn(0x7f30ac15dc40 0x7f30ac0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30aa59c640 1 -- 172.21.15.26:0/3661522895 >> v2:172.21.15.160:3300/0 conn(0x7f30ac15dc40 msgr2=0x7f30ac0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:05.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30aa59c640 1 --2- 172.21.15.26:0/3661522895 >> v2:172.21.15.160:3300/0 conn(0x7f30ac15dc40 0x7f30ac0cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30aa59c640 1 -- 172.21.15.26:0/3661522895 >> v1:172.21.15.160:6789/0 conn(0x7f30ac0cb150 legacy=0x7f30ac15d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:05.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30aa59c640 1 -- 172.21.15.26:0/3661522895 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac0c9520 con 0x7f30ac1599e0 2023-12-19T17:58:05.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30aad9d640 1 --2- 172.21.15.26:0/3661522895 >> v2:172.21.15.160:3300/0 conn(0x7f30ac15dc40 0x7f30ac0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:58:05.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.546+0000 7f30aa59c640 1 --2- 172.21.15.26:0/3661522895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac1599e0 0x7f30ac06f6d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0cbcf0 tx=0x7f309c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac231798373a57b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:05.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30a9d9b640 1 -- 172.21.15.26:0/3661522895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c0aa620 con 0x7f30ac1599e0 2023-12-19T17:58:05.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30a9d9b640 1 -- 172.21.15.26:0/3661522895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c0aa780 con 0x7f30ac1599e0 2023-12-19T17:58:05.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30a9d9b640 1 -- 172.21.15.26:0/3661522895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c0aab10 con 0x7f30ac1599e0 2023-12-19T17:58:05.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 -- 172.21.15.26:0/3661522895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac1599e0 msgr2=0x7f30ac06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 --2- 172.21.15.26:0/3661522895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac1599e0 0x7f30ac06f6d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0cbcf0 tx=0x7f309c01c4c0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 -- 172.21.15.26:0/3661522895 shutdown_connections 2023-12-19T17:58:05.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 --2- 172.21.15.26:0/3661522895 >> v2:172.21.15.160:3300/0 conn(0x7f30ac15dc40 0x7f30ac0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 --2- 172.21.15.26:0/3661522895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac1599e0 0x7f30ac06f6d0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 -- 172.21.15.26:0/3661522895 >> 172.21.15.26:0/3661522895 conn(0x7f30ac066160 msgr2=0x7f30ac0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:05.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 -- 172.21.15.26:0/3661522895 shutdown_connections 2023-12-19T17:58:05.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 -- 172.21.15.26:0/3661522895 wait complete. 2023-12-19T17:58:05.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 Processor -- start 2023-12-19T17:58:05.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 -- start start 2023-12-19T17:58:05.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30ac0cb150 0x7f30ac0c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac1599e0 0x7f30ac152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30ac15dc40 0x7f30ac152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac152b90 con 0x7f30ac1599e0 2023-12-19T17:58:05.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac0b31f0 con 0x7f30ac15dc40 2023-12-19T17:58:05.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.550+0000 7f30b0cd1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac0b34d0 con 0x7f30ac0cb150 2023-12-19T17:58:05.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac1599e0 0x7f30ac152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac1599e0 0x7f30ac152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40106/0 (socket says 172.21.15.26:40106) 2023-12-19T17:58:05.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30a3fff640 1 -- 172.21.15.26:0/1671911652 learned_addr learned my addr 172.21.15.26:0/1671911652 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:05.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30aa59c640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30ac0cb150 0x7f30ac0c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30aad9d640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30ac15dc40 0x7f30ac152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30a3fff640 1 -- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30ac0cb150 msgr2=0x7f30ac0c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30a3fff640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30ac0cb150 0x7f30ac0c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30a3fff640 1 -- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30ac15dc40 msgr2=0x7f30ac152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30a3fff640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30ac15dc40 0x7f30ac152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30a3fff640 1 -- 172.21.15.26:0/1671911652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c0094d0 con 0x7f30ac1599e0 2023-12-19T17:58:05.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30a3fff640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac1599e0 0x7f30ac152110 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f309400c450 tx=0x7f309400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:05.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30a2ffd640 1 -- 172.21.15.26:0/1671911652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094012830 con 0x7f30ac1599e0 2023-12-19T17:58:05.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30a2ffd640 1 -- 172.21.15.26:0/1671911652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30940129b0 con 0x7f30ac1599e0 2023-12-19T17:58:05.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30a2ffd640 1 -- 172.21.15.26:0/1671911652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094012d60 con 0x7f30ac1599e0 2023-12-19T17:58:05.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30b0cd1640 1 -- 172.21.15.26:0/1671911652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ac0b37b0 con 0x7f30ac1599e0 2023-12-19T17:58:05.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30b0cd1640 1 -- 172.21.15.26:0/1671911652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ac0b3d70 con 0x7f30ac1599e0 2023-12-19T17:58:05.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30b0cd1640 1 -- 172.21.15.26:0/1671911652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ac074270 con 0x7f30ac1599e0 2023-12-19T17:58:05.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.554+0000 7f30a2ffd640 1 -- 172.21.15.26:0/1671911652 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f309401c460 con 0x7f30ac1599e0 2023-12-19T17:58:05.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.558+0000 7f30a2ffd640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f308c065b80 0x7f308c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.558+0000 7f30a2ffd640 1 -- 172.21.15.26:0/1671911652 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f3094083290 con 0x7f30ac1599e0 2023-12-19T17:58:05.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.558+0000 7f30aa59c640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f308c065b80 0x7f308c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.558+0000 7f30aa59c640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f308c065b80 0x7f308c068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f309c0ae000 tx=0x7f309c0b5040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:05.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.562+0000 7f30a2ffd640 1 -- 172.21.15.26:0/1671911652 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f30940509b0 con 0x7f30ac1599e0 2023-12-19T17:58:05.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.718+0000 7f30b0cd1640 1 -- 172.21.15.26:0/1671911652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f30ac0c2050 con 0x7f30ac1599e0 2023-12-19T17:58:05.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.722+0000 7f30a2ffd640 1 -- 172.21.15.26:0/1671911652 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3094053fd0 con 0x7f30ac1599e0 2023-12-19T17:58:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.726+0000 7f30b0cd1640 1 -- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f308c065b80 msgr2=0x7f308c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.726+0000 7f30b0cd1640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f308c065b80 0x7f308c068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f309c0ae000 tx=0x7f309c0b5040 comp rx=0 tx=0).stop 2023-12-19T17:58:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.726+0000 7f30b0cd1640 1 -- 172.21.15.26:0/1671911652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac1599e0 msgr2=0x7f30ac152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.726+0000 7f30b0cd1640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac1599e0 0x7f30ac152110 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f309400c450 tx=0x7f309400c910 comp rx=0 tx=0).stop 2023-12-19T17:58:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.726+0000 7f30b0cd1640 1 -- 172.21.15.26:0/1671911652 shutdown_connections 2023-12-19T17:58:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.726+0000 7f30b0cd1640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30ac15dc40 0x7f30ac152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.726+0000 7f30b0cd1640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30ac1599e0 0x7f30ac152110 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.726+0000 7f30b0cd1640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f308c065b80 0x7f308c068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.726+0000 7f30b0cd1640 1 --2- 172.21.15.26:0/1671911652 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30ac0cb150 0x7f30ac0c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.726+0000 7f30b0cd1640 1 -- 172.21.15.26:0/1671911652 >> 172.21.15.26:0/1671911652 conn(0x7f30ac066160 msgr2=0x7f30ac0701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.726+0000 7f30b0cd1640 1 -- 172.21.15.26:0/1671911652 shutdown_connections 2023-12-19T17:58:05.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.726+0000 7f30b0cd1640 1 -- 172.21.15.26:0/1671911652 wait complete. 2023-12-19T17:58:05.748 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-19T17:58:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.890+0000 7f1999641640 1 Processor -- start 2023-12-19T17:58:05.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.890+0000 7f1999641640 1 -- start start 2023-12-19T17:58:05.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.890+0000 7f1999641640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f199406c300 0x7f19940cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.890+0000 7f1999641640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199415d0f0 con 0x7f199406d3e0 2023-12-19T17:58:05.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.890+0000 7f1999641640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19940c98b0 con 0x7f199415cd10 2023-12-19T17:58:05.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.890+0000 7f1999641640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19940cd0e0 con 0x7f199406c300 2023-12-19T17:58:05.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.890+0000 7f19937fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f199406d3e0 0x7f19940c9d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44852/0 (socket says 172.21.15.26:44852) 2023-12-19T17:58:05.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.890+0000 7f19937fe640 1 -- 172.21.15.26:0/2201374578 learned_addr learned my addr 172.21.15.26:0/2201374578 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:05.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.890+0000 7f198bfff640 1 --2- 172.21.15.26:0/2201374578 >> v2:172.21.15.160:3300/0 conn(0x7f199406c300 0x7f19940cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946502036 0 0) 0x7f198c0014a0 con 0x7f199406d3e0 2023-12-19T17:58:05.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1980003530 con 0x7f199406d3e0 2023-12-19T17:58:05.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3950116253 0 0) 0x7f1980003530 con 0x7f199406d3e0 2023-12-19T17:58:05.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 >> v2:172.21.15.160:3300/0 conn(0x7f199406c300 msgr2=0x7f19940cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:05.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 --2- 172.21.15.26:0/2201374578 >> v2:172.21.15.160:3300/0 conn(0x7f199406c300 0x7f19940cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 >> v1:172.21.15.160:6789/0 conn(0x7f199415cd10 legacy=0x7f199406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199406d950 con 0x7f199406d3e0 2023-12-19T17:58:05.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f198c001b90 con 0x7f199406d3e0 2023-12-19T17:58:05.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 >> v1:172.21.15.26:6789/0 conn(0x7f199406d3e0 legacy=0x7f19940c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 --2- 172.21.15.26:0/2201374578 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1980003980 0x7f1980003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 --2- 172.21.15.26:0/2201374578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1980004380 0x7f1980004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 --2- 172.21.15.26:0/2201374578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1980005050 0x7f198000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19940cd0e0 con 0x7f1980004380 2023-12-19T17:58:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198000bf00 con 0x7f1980005050 2023-12-19T17:58:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19800148d0 con 0x7f1980003980 2023-12-19T17:58:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f198bfff640 1 --2- 172.21.15.26:0/2201374578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1980004380 0x7f1980004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f1992ffd640 1 --2- 172.21.15.26:0/2201374578 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1980003980 0x7f1980003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19937fe640 1 --2- 172.21.15.26:0/2201374578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1980005050 0x7f198000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f198bfff640 1 -- 172.21.15.26:0/2201374578 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1980003980 msgr2=0x7f1980003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f198bfff640 1 --2- 172.21.15.26:0/2201374578 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1980003980 0x7f1980003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f198bfff640 1 -- 172.21.15.26:0/2201374578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1980005050 msgr2=0x7f198000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f198bfff640 1 --2- 172.21.15.26:0/2201374578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1980005050 0x7f198000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f198bfff640 1 -- 172.21.15.26:0/2201374578 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980003530 con 0x7f1980004380 2023-12-19T17:58:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19937fe640 1 --2- 172.21.15.26:0/2201374578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1980005050 0x7f198000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:05.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.894+0000 7f19937fe640 1 -- 172.21.15.26:0/2201374578 reap_dead start 2023-12-19T17:58:05.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f198bfff640 1 --2- 172.21.15.26:0/2201374578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1980004380 0x7f1980004850 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f197c009ae0 tx=0x7f197c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370f73a152a3ccf6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:05.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c0a5530 con 0x7f1980004380 2023-12-19T17:58:05.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c0a5690 con 0x7f1980004380 2023-12-19T17:58:05.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f19927fc640 1 -- 172.21.15.26:0/2201374578 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c0a5a80 con 0x7f1980004380 2023-12-19T17:58:05.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 -- 172.21.15.26:0/2201374578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1980004380 msgr2=0x7f1980004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 --2- 172.21.15.26:0/2201374578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1980004380 0x7f1980004850 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f197c009ae0 tx=0x7f197c01ca50 comp rx=0 tx=0).stop 2023-12-19T17:58:05.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 -- 172.21.15.26:0/2201374578 shutdown_connections 2023-12-19T17:58:05.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 --2- 172.21.15.26:0/2201374578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1980004380 0x7f1980004850 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 -- 172.21.15.26:0/2201374578 >> 172.21.15.26:0/2201374578 conn(0x7f19940cf260 msgr2=0x7f19940cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:05.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 -- 172.21.15.26:0/2201374578 shutdown_connections 2023-12-19T17:58:05.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 -- 172.21.15.26:0/2201374578 wait complete. 2023-12-19T17:58:05.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 Processor -- start 2023-12-19T17:58:05.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 -- start start 2023-12-19T17:58:05.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1980004380 0x7f19940c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19940b2800 0x7f19940b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1994158690 0x7f1994158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19940c7d10 con 0x7f1994158690 2023-12-19T17:58:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1994159050 con 0x7f19940b2800 2023-12-19T17:58:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f1999641640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19940c7f40 con 0x7f1980004380 2023-12-19T17:58:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.898+0000 7f19937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1994158690 0x7f1994158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f19937fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1994158690 0x7f1994158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50974/0 (socket says 172.21.15.26:50974) 2023-12-19T17:58:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f19937fe640 1 -- 172.21.15.26:0/2083030720 learned_addr learned my addr 172.21.15.26:0/2083030720 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f1992ffd640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1980004380 0x7f19940c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f198bfff640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19940b2800 0x7f19940b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f19937fe640 1 -- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1980004380 msgr2=0x7f19940c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f19937fe640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1980004380 0x7f19940c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f19937fe640 1 -- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19940b2800 msgr2=0x7f19940b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f19937fe640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19940b2800 0x7f19940b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f19937fe640 1 -- 172.21.15.26:0/2083030720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19940c8220 con 0x7f1994158690 2023-12-19T17:58:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f19937fe640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1994158690 0x7f1994158b10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f19940cf190 tx=0x7f198c010910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f198b7fe640 1 -- 172.21.15.26:0/2083030720 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c007070 con 0x7f1994158690 2023-12-19T17:58:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f198b7fe640 1 -- 172.21.15.26:0/2083030720 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198c00d070 con 0x7f1994158690 2023-12-19T17:58:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f198b7fe640 1 -- 172.21.15.26:0/2083030720 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c007430 con 0x7f1994158690 2023-12-19T17:58:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f1999641640 1 -- 172.21.15.26:0/2083030720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19940c8500 con 0x7f1994158690 2023-12-19T17:58:05.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f1999641640 1 -- 172.21.15.26:0/2083030720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19940c8ac0 con 0x7f1994158690 2023-12-19T17:58:05.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f1999641640 1 -- 172.21.15.26:0/2083030720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19940c8ce0 con 0x7f1994158690 2023-12-19T17:58:05.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f198b7fe640 1 -- 172.21.15.26:0/2083030720 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f198c007590 con 0x7f1994158690 2023-12-19T17:58:05.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f198b7fe640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1974065bb0 0x7f1974068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:05.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.902+0000 7f198b7fe640 1 -- 172.21.15.26:0/2083030720 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f198c085310 con 0x7f1994158690 2023-12-19T17:58:05.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.906+0000 7f1992ffd640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1974065bb0 0x7f1974068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:05.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.906+0000 7f1992ffd640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1974065bb0 0x7f1974068070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1984007fd0 tx=0x7f1984007f60 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:05.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:05.906+0000 7f198b7fe640 1 -- 172.21.15.26:0/2083030720 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f198c052880 con 0x7f1994158690 2023-12-19T17:58:06.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.150+0000 7f1999641640 1 -- 172.21.15.26:0/2083030720 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f199415d0f0 con 0x7f1994158690 2023-12-19T17:58:06.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.154+0000 7f198b7fe640 1 -- 172.21.15.26:0/2083030720 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v57) v1 ==== 115+0+0 (secure 0 0 0) 0x7f199415d0f0 con 0x7f1994158690 2023-12-19T17:58:06.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.154+0000 7f1999641640 1 -- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1974065bb0 msgr2=0x7f1974068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:06.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.154+0000 7f1999641640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1974065bb0 0x7f1974068070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1984007fd0 tx=0x7f1984007f60 comp rx=0 tx=0).stop 2023-12-19T17:58:06.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.154+0000 7f1999641640 1 -- 172.21.15.26:0/2083030720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1994158690 msgr2=0x7f1994158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:06.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.154+0000 7f1999641640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1994158690 0x7f1994158b10 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f19940cf190 tx=0x7f198c010910 comp rx=0 tx=0).stop 2023-12-19T17:58:06.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.154+0000 7f1999641640 1 -- 172.21.15.26:0/2083030720 shutdown_connections 2023-12-19T17:58:06.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.154+0000 7f1999641640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1994158690 0x7f1994158b10 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:06.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.154+0000 7f1999641640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19940b2800 0x7f19940b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:06.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.154+0000 7f1999641640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1974065bb0 0x7f1974068070 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:06.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.154+0000 7f1999641640 1 --2- 172.21.15.26:0/2083030720 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1980004380 0x7f19940c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:06.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.154+0000 7f1999641640 1 -- 172.21.15.26:0/2083030720 >> 172.21.15.26:0/2083030720 conn(0x7f19940cf260 msgr2=0x7f199406fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:06.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.154+0000 7f1999641640 1 -- 172.21.15.26:0/2083030720 shutdown_connections 2023-12-19T17:58:06.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.154+0000 7f1999641640 1 -- 172.21.15.26:0/2083030720 wait complete. 2023-12-19T17:58:06.181 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-12-19T17:58:06.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d1a28c640 1 Processor -- start 2023-12-19T17:58:06.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d1a28c640 1 -- start start 2023-12-19T17:58:06.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d1a28c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6d1406b3d0 0x7f6d140ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:06.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d1a28c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6d140cec40 0x7f6d1415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d1a28c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1406bc30 con 0x7f6d14154220 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d1a28c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d140cf0d0 con 0x7f6d1406b3d0 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d1a28c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d140cc6e0 con 0x7f6d140cec40 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d13fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6d14154220 0x7f6d1415aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50138/0 (socket says 172.21.15.26:50138) 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d13fff640 1 -- 172.21.15.26:0/3751857816 learned_addr learned my addr 172.21.15.26:0/3751857816 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d137fe640 1 --2- 172.21.15.26:0/3751857816 >> v2:172.21.15.26:3300/0 conn(0x7f6d1406b3d0 0x7f6d140ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d18829640 1 --2- 172.21.15.26:0/3751857816 >> v2:172.21.15.160:3300/0 conn(0x7f6d140cec40 0x7f6d1415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d12ffd640 1 -- 172.21.15.26:0/3751857816 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476339231 0 0) 0x7f6d1406bc30 con 0x7f6d14154220 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d12ffd640 1 -- 172.21.15.26:0/3751857816 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cfc003530 con 0x7f6d14154220 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d137fe640 1 -- 172.21.15.26:0/3751857816 >> v2:172.21.15.160:3300/0 conn(0x7f6d140cec40 msgr2=0x7f6d1415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d137fe640 1 --2- 172.21.15.26:0/3751857816 >> v2:172.21.15.160:3300/0 conn(0x7f6d140cec40 0x7f6d1415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d137fe640 1 -- 172.21.15.26:0/3751857816 >> v1:172.21.15.26:6789/0 conn(0x7f6d14154220 legacy=0x7f6d1415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d137fe640 1 -- 172.21.15.26:0/3751857816 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1406b950 con 0x7f6d1406b3d0 2023-12-19T17:58:06.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.330+0000 7f6d18829640 1 --2- 172.21.15.26:0/3751857816 >> v2:172.21.15.160:3300/0 conn(0x7f6d140cec40 0x7f6d1415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:58:06.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.334+0000 7f6d137fe640 1 --2- 172.21.15.26:0/3751857816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d1406b3d0 0x7f6d140ce700 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6d14157bd0 tx=0x7f6d0801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682140b91dbd1388 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.334+0000 7f6d12ffd640 1 -- 172.21.15.26:0/3751857816 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080a7070 con 0x7f6d1406b3d0 2023-12-19T17:58:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.334+0000 7f6d12ffd640 1 -- 172.21.15.26:0/3751857816 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d080ab430 con 0x7f6d1406b3d0 2023-12-19T17:58:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.334+0000 7f6d12ffd640 1 -- 172.21.15.26:0/3751857816 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080ab7c0 con 0x7f6d1406b3d0 2023-12-19T17:58:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.334+0000 7f6d1a28c640 1 -- 172.21.15.26:0/3751857816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d1406b3d0 msgr2=0x7f6d140ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.334+0000 7f6d1a28c640 1 --2- 172.21.15.26:0/3751857816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d1406b3d0 0x7f6d140ce700 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6d14157bd0 tx=0x7f6d0801c430 comp rx=0 tx=0).stop 2023-12-19T17:58:06.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.334+0000 7f6d1a28c640 1 -- 172.21.15.26:0/3751857816 shutdown_connections 2023-12-19T17:58:06.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.334+0000 7f6d1a28c640 1 --2- 172.21.15.26:0/3751857816 >> v2:172.21.15.160:3300/0 conn(0x7f6d140cec40 0x7f6d1415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:06.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.334+0000 7f6d1a28c640 1 --2- 172.21.15.26:0/3751857816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d1406b3d0 0x7f6d140ce700 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:06.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.334+0000 7f6d1a28c640 1 -- 172.21.15.26:0/3751857816 >> 172.21.15.26:0/3751857816 conn(0x7f6d140658e0 msgr2=0x7f6d14065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:06.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.334+0000 7f6d1a28c640 1 -- 172.21.15.26:0/3751857816 shutdown_connections 2023-12-19T17:58:06.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.334+0000 7f6d1a28c640 1 -- 172.21.15.26:0/3751857816 wait complete. 2023-12-19T17:58:06.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d1a28c640 1 Processor -- start 2023-12-19T17:58:06.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d1a28c640 1 -- start start 2023-12-19T17:58:06.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d1a28c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d1406b3d0 0x7f6d1414d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:06.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d1a28c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d140cec40 0x7f6d1413ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:06.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d1a28c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d14154220 0x7f6d140ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:06.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d1a28c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d14153480 con 0x7f6d140cec40 2023-12-19T17:58:06.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d1a28c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1414f780 con 0x7f6d14154220 2023-12-19T17:58:06.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d1a28c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1414fa60 con 0x7f6d1406b3d0 2023-12-19T17:58:06.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d137fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d140cec40 0x7f6d1413ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:06.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d137fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d140cec40 0x7f6d1413ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50988/0 (socket says 172.21.15.26:50988) 2023-12-19T17:58:06.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d137fe640 1 -- 172.21.15.26:0/2003892361 learned_addr learned my addr 172.21.15.26:0/2003892361 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:06.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d137fe640 1 -- 172.21.15.26:0/2003892361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d1406b3d0 msgr2=0x7f6d1414d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:06.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d137fe640 1 --2- 172.21.15.26:0/2003892361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d1406b3d0 0x7f6d1414d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:06.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d137fe640 1 -- 172.21.15.26:0/2003892361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d14154220 msgr2=0x7f6d140ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:06.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d137fe640 1 --2- 172.21.15.26:0/2003892361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d14154220 0x7f6d140ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:06.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d137fe640 1 -- 172.21.15.26:0/2003892361 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d08009420 con 0x7f6d140cec40 2023-12-19T17:58:06.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d137fe640 1 --2- 172.21.15.26:0/2003892361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d140cec40 0x7f6d1413ff50 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f6d08019040 tx=0x7f6d0801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:06.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d10ff9640 1 -- 172.21.15.26:0/2003892361 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080b5610 con 0x7f6d140cec40 2023-12-19T17:58:06.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d10ff9640 1 -- 172.21.15.26:0/2003892361 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d080b5790 con 0x7f6d140cec40 2023-12-19T17:58:06.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d10ff9640 1 -- 172.21.15.26:0/2003892361 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080b5b40 con 0x7f6d140cec40 2023-12-19T17:58:06.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d1a28c640 1 -- 172.21.15.26:0/2003892361 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d14152e30 con 0x7f6d140cec40 2023-12-19T17:58:06.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.338+0000 7f6d1a28c640 1 -- 172.21.15.26:0/2003892361 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1414fd40 con 0x7f6d140cec40 2023-12-19T17:58:06.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.342+0000 7f6d1a28c640 1 -- 172.21.15.26:0/2003892361 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d141532e0 con 0x7f6d140cec40 2023-12-19T17:58:06.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.342+0000 7f6d10ff9640 1 -- 172.21.15.26:0/2003892361 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6d080b5ca0 con 0x7f6d140cec40 2023-12-19T17:58:06.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.342+0000 7f6d10ff9640 1 --2- 172.21.15.26:0/2003892361 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6d00065bd0 0x7f6d00068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:06.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.342+0000 7f6d10ff9640 1 -- 172.21.15.26:0/2003892361 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f6d0811e040 con 0x7f6d140cec40 2023-12-19T17:58:06.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.342+0000 7f6d13fff640 1 --2- 172.21.15.26:0/2003892361 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6d00065bd0 0x7f6d00068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:06.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.342+0000 7f6d13fff640 1 --2- 172.21.15.26:0/2003892361 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6d00065bd0 0x7f6d00068090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6d04007fd0 tx=0x7f6d04007940 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:06.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.346+0000 7f6d10ff9640 1 -- 172.21.15.26:0/2003892361 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6d141532e0 con 0x7f6d140cec40 2023-12-19T17:58:06.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:06.562+0000 7f6d1a28c640 1 -- 172.21.15.26:0/2003892361 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f6d141532e0 con 0x7f6d140cec40 2023-12-19T17:58:07.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.026+0000 7f6d10ff9640 1 -- 172.21.15.26:0/2003892361 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2449) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6d141532e0 con 0x7f6d140cec40 2023-12-19T17:58:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.026+0000 7f6d1a28c640 1 -- 172.21.15.26:0/2003892361 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6d00065bd0 msgr2=0x7f6d00068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.026+0000 7f6d1a28c640 1 --2- 172.21.15.26:0/2003892361 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6d00065bd0 0x7f6d00068090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6d04007fd0 tx=0x7f6d04007940 comp rx=0 tx=0).stop 2023-12-19T17:58:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.026+0000 7f6d1a28c640 1 -- 172.21.15.26:0/2003892361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d140cec40 msgr2=0x7f6d1413ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.026+0000 7f6d1a28c640 1 --2- 172.21.15.26:0/2003892361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d140cec40 0x7f6d1413ff50 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f6d08019040 tx=0x7f6d0801ac70 comp rx=0 tx=0).stop 2023-12-19T17:58:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.026+0000 7f6d1a28c640 1 -- 172.21.15.26:0/2003892361 shutdown_connections 2023-12-19T17:58:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.026+0000 7f6d1a28c640 1 --2- 172.21.15.26:0/2003892361 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d14154220 0x7f6d140ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.026+0000 7f6d1a28c640 1 --2- 172.21.15.26:0/2003892361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d140cec40 0x7f6d1413ff50 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.026+0000 7f6d1a28c640 1 --2- 172.21.15.26:0/2003892361 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6d00065bd0 0x7f6d00068090 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.026+0000 7f6d1a28c640 1 --2- 172.21.15.26:0/2003892361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d1406b3d0 0x7f6d1414d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.026+0000 7f6d1a28c640 1 -- 172.21.15.26:0/2003892361 >> 172.21.15.26:0/2003892361 conn(0x7f6d140658e0 msgr2=0x7f6d140cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.026+0000 7f6d1a28c640 1 -- 172.21.15.26:0/2003892361 shutdown_connections 2023-12-19T17:58:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.026+0000 7f6d1a28c640 1 -- 172.21.15.26:0/2003892361 wait complete. 2023-12-19T17:58:07.054 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-12-19T17:58:07.054 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-12-19T17:58:07.054 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-19T17:58:07.055 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-12-19T17:58:07.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.198+0000 7fa5225ce640 1 Processor -- start 2023-12-19T17:58:07.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.198+0000 7fa5225ce640 1 -- start start 2023-12-19T17:58:07.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa5225ce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa51c06bee0 0x7fa51c06c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:07.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa5225ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa51c0b1d40 0x7fa51c0b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa5225ce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa51c06d4b0 con 0x7fa51c0cb1d0 2023-12-19T17:58:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa5225ce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51c0cc030 con 0x7fa51c0b1d40 2023-12-19T17:58:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa5225ce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51c06cc10 con 0x7fa51c06bee0 2023-12-19T17:58:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa51bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa51c0cb1d0 0x7fa51c0cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50140/0 (socket says 172.21.15.26:50140) 2023-12-19T17:58:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa51bfff640 1 -- 172.21.15.26:0/2359836280 learned_addr learned my addr 172.21.15.26:0/2359836280 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa520b6b640 1 --2- 172.21.15.26:0/2359836280 >> v2:172.21.15.26:3300/0 conn(0x7fa51c0b1d40 0x7fa51c0b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa51b7fe640 1 --2- 172.21.15.26:0/2359836280 >> v2:172.21.15.160:3300/0 conn(0x7fa51c06bee0 0x7fa51c06c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa520b6b640 1 -- 172.21.15.26:0/2359836280 >> v2:172.21.15.160:3300/0 conn(0x7fa51c06bee0 msgr2=0x7fa51c06c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa520b6b640 1 --2- 172.21.15.26:0/2359836280 >> v2:172.21.15.160:3300/0 conn(0x7fa51c06bee0 0x7fa51c06c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa520b6b640 1 -- 172.21.15.26:0/2359836280 >> v1:172.21.15.26:6789/0 conn(0x7fa51c0cb1d0 legacy=0x7fa51c0cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa520b6b640 1 -- 172.21.15.26:0/2359836280 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa51c06cef0 con 0x7fa51c0b1d40 2023-12-19T17:58:07.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa51affd640 1 -- 172.21.15.26:0/2359836280 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281985051 0 0) 0x7fa51c06d4b0 con 0x7fa51c0cb1d0 2023-12-19T17:58:07.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.202+0000 7fa520b6b640 1 --2- 172.21.15.26:0/2359836280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c0b1d40 0x7fa51c0b21d0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0667b0 tx=0x7fa51401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaabf04c51ae1cd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa51affd640 1 -- 172.21.15.26:0/2359836280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5140a5910 con 0x7fa51c0b1d40 2023-12-19T17:58:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa51affd640 1 -- 172.21.15.26:0/2359836280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5140a5a70 con 0x7fa51c0b1d40 2023-12-19T17:58:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa51affd640 1 -- 172.21.15.26:0/2359836280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5140a5e00 con 0x7fa51c0b1d40 2023-12-19T17:58:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 -- 172.21.15.26:0/2359836280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c0b1d40 msgr2=0x7fa51c0b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 --2- 172.21.15.26:0/2359836280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c0b1d40 0x7fa51c0b21d0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0667b0 tx=0x7fa51401c850 comp rx=0 tx=0).stop 2023-12-19T17:58:07.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 -- 172.21.15.26:0/2359836280 shutdown_connections 2023-12-19T17:58:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 --2- 172.21.15.26:0/2359836280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c0b1d40 0x7fa51c0b21d0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 --2- 172.21.15.26:0/2359836280 >> v2:172.21.15.160:3300/0 conn(0x7fa51c06bee0 0x7fa51c06c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 -- 172.21.15.26:0/2359836280 >> 172.21.15.26:0/2359836280 conn(0x7fa51c0658e0 msgr2=0x7fa51c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 -- 172.21.15.26:0/2359836280 shutdown_connections 2023-12-19T17:58:07.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 -- 172.21.15.26:0/2359836280 wait complete. 2023-12-19T17:58:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 Processor -- start 2023-12-19T17:58:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 -- start start 2023-12-19T17:58:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c06bee0 0x7fa51c157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa51c0b1d40 0x7fa51c163890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa51c0cb1d0 0x7fa51c163dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:07.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c15b280 con 0x7fa51c06bee0 2023-12-19T17:58:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c0afa60 con 0x7fa51c0b1d40 2023-12-19T17:58:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa5225ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51c15acc0 con 0x7fa51c0cb1d0 2023-12-19T17:58:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa51bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c06bee0 0x7fa51c157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa51bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c06bee0 0x7fa51c157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51014/0 (socket says 172.21.15.26:51014) 2023-12-19T17:58:07.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.206+0000 7fa51bfff640 1 -- 172.21.15.26:0/2358912298 learned_addr learned my addr 172.21.15.26:0/2358912298 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa51b7fe640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa51c0b1d40 0x7fa51c163890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa51bfff640 1 -- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa51c0cb1d0 msgr2=0x7fa51c163dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa51bfff640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa51c0cb1d0 0x7fa51c163dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa51bfff640 1 -- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa51c0b1d40 msgr2=0x7fa51c163890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa51bfff640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa51c0b1d40 0x7fa51c163890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa51bfff640 1 -- 172.21.15.26:0/2358912298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa514009a70 con 0x7fa51c06bee0 2023-12-19T17:58:07.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa51b7fe640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa51c0b1d40 0x7fa51c163890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:58:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa51bfff640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c06bee0 0x7fa51c157550 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c480 tx=0x7fa50c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa518ff9640 1 -- 172.21.15.26:0/2358912298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c0074e0 con 0x7fa51c06bee0 2023-12-19T17:58:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa518ff9640 1 -- 172.21.15.26:0/2358912298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50c007660 con 0x7fa51c06bee0 2023-12-19T17:58:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa518ff9640 1 -- 172.21.15.26:0/2358912298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50c004a30 con 0x7fa51c06bee0 2023-12-19T17:58:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa5225ce640 1 -- 172.21.15.26:0/2358912298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51c15afa0 con 0x7fa51c06bee0 2023-12-19T17:58:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa5225ce640 1 -- 172.21.15.26:0/2358912298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51c155c10 con 0x7fa51c06bee0 2023-12-19T17:58:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa5225ce640 1 -- 172.21.15.26:0/2358912298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa51c158d70 con 0x7fa51c06bee0 2023-12-19T17:58:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.210+0000 7fa518ff9640 1 -- 172.21.15.26:0/2358912298 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa50c004b90 con 0x7fa51c06bee0 2023-12-19T17:58:07.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.214+0000 7fa518ff9640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa508065b80 0x7fa508068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.214+0000 7fa518ff9640 1 -- 172.21.15.26:0/2358912298 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fa50c082680 con 0x7fa51c06bee0 2023-12-19T17:58:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.214+0000 7fa51b7fe640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa508065b80 0x7fa508068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.214+0000 7fa51b7fe640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa508065b80 0x7fa508068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa51c160600 tx=0x7fa510008040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.214+0000 7fa518ff9640 1 -- 172.21.15.26:0/2358912298 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa50c053810 con 0x7fa51c06bee0 2023-12-19T17:58:07.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:07.430+0000 7fa5225ce640 1 -- 172.21.15.26:0/2358912298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7fa51c074270 con 0x7fa51c06bee0 2023-12-19T17:58:08.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.034+0000 7fa518ff9640 1 -- 172.21.15.26:0/2358912298 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2450) v1 ==== 168+0+0 (secure 0 0 0) 0x7fa50c0533a0 con 0x7fa51c06bee0 2023-12-19T17:58:08.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.034+0000 7fa5225ce640 1 -- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa508065b80 msgr2=0x7fa508068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:08.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.034+0000 7fa5225ce640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa508065b80 0x7fa508068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa51c160600 tx=0x7fa510008040 comp rx=0 tx=0).stop 2023-12-19T17:58:08.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.034+0000 7fa5225ce640 1 -- 172.21.15.26:0/2358912298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c06bee0 msgr2=0x7fa51c157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:08.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.034+0000 7fa5225ce640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c06bee0 0x7fa51c157550 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00c480 tx=0x7fa50c00c940 comp rx=0 tx=0).stop 2023-12-19T17:58:08.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.034+0000 7fa5225ce640 1 -- 172.21.15.26:0/2358912298 shutdown_connections 2023-12-19T17:58:08.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.034+0000 7fa5225ce640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fa508065b80 0x7fa508068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.034+0000 7fa5225ce640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa51c0cb1d0 0x7fa51c163dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.034+0000 7fa5225ce640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa51c0b1d40 0x7fa51c163890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.034+0000 7fa5225ce640 1 --2- 172.21.15.26:0/2358912298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa51c06bee0 0x7fa51c157550 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.034+0000 7fa5225ce640 1 -- 172.21.15.26:0/2358912298 >> 172.21.15.26:0/2358912298 conn(0x7fa51c0658e0 msgr2=0x7fa51c06fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:08.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.034+0000 7fa5225ce640 1 -- 172.21.15.26:0/2358912298 shutdown_connections 2023-12-19T17:58:08.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.034+0000 7fa5225ce640 1 -- 172.21.15.26:0/2358912298 wait complete. 2023-12-19T17:58:08.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T17:58:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.210+0000 7fedc9666640 1 Processor -- start 2023-12-19T17:58:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.210+0000 7fedc9666640 1 -- start start 2023-12-19T17:58:08.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc9666640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fedc415cdb0 0x7fedc406d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc9666640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedc40ca060 0x7fedc4161280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc9666640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fedc40658e0 0x7fedc4065d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc9666640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc40662b0 con 0x7fedc40658e0 2023-12-19T17:58:08.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc9666640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc415d190 con 0x7fedc415cdb0 2023-12-19T17:58:08.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc9666640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc40cd940 con 0x7fedc40ca060 2023-12-19T17:58:08.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fedc40658e0 0x7fedc4065d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc37fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fedc40658e0 0x7fedc4065d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51022/0 (socket says 172.21.15.26:51022) 2023-12-19T17:58:08.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc37fe640 1 -- 172.21.15.26:0/2176759582 learned_addr learned my addr 172.21.15.26:0/2176759582 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:08.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedbbfff640 1 --2- 172.21.15.26:0/2176759582 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedc40ca060 0x7fedc4161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc2ffd640 1 --2- 172.21.15.26:0/2176759582 >> v2:172.21.15.160:3300/0 conn(0x7fedc415cdb0 0x7fedc406d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc37fe640 1 -- 172.21.15.26:0/2176759582 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedc40ca060 msgr2=0x7fedc4161280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:08.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc37fe640 1 --2- 172.21.15.26:0/2176759582 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedc40ca060 0x7fedc4161280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc37fe640 1 -- 172.21.15.26:0/2176759582 >> v2:172.21.15.160:3300/0 conn(0x7fedc415cdb0 msgr2=0x7fedc406d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:08.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc37fe640 1 --2- 172.21.15.26:0/2176759582 >> v2:172.21.15.160:3300/0 conn(0x7fedc415cdb0 0x7fedc406d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc37fe640 1 -- 172.21.15.26:0/2176759582 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc40cdf50 con 0x7fedc40658e0 2023-12-19T17:58:08.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.214+0000 7fedc2ffd640 1 --2- 172.21.15.26:0/2176759582 >> v2:172.21.15.160:3300/0 conn(0x7fedc415cdb0 0x7fedc406d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc37fe640 1 --2- 172.21.15.26:0/2176759582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedc40658e0 0x7fedc4065d70 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fedc40c6950 tx=0x7fedbc01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4c4ebb9859581a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc27fc640 1 -- 172.21.15.26:0/2176759582 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0a5530 con 0x7fedc40658e0 2023-12-19T17:58:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc27fc640 1 -- 172.21.15.26:0/2176759582 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc0a5690 con 0x7fedc40658e0 2023-12-19T17:58:08.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc27fc640 1 -- 172.21.15.26:0/2176759582 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc0a5a20 con 0x7fedc40658e0 2023-12-19T17:58:08.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc9666640 1 -- 172.21.15.26:0/2176759582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedc40658e0 msgr2=0x7fedc4065d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:08.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc9666640 1 --2- 172.21.15.26:0/2176759582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedc40658e0 0x7fedc4065d70 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fedc40c6950 tx=0x7fedbc01c8b0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc9666640 1 -- 172.21.15.26:0/2176759582 shutdown_connections 2023-12-19T17:58:08.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc9666640 1 --2- 172.21.15.26:0/2176759582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedc40658e0 0x7fedc4065d70 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc9666640 1 --2- 172.21.15.26:0/2176759582 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedc40ca060 0x7fedc4161280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc9666640 1 --2- 172.21.15.26:0/2176759582 >> v2:172.21.15.160:3300/0 conn(0x7fedc415cdb0 0x7fedc406d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc9666640 1 -- 172.21.15.26:0/2176759582 >> 172.21.15.26:0/2176759582 conn(0x7fedc4069d20 msgr2=0x7fedc406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:08.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc9666640 1 -- 172.21.15.26:0/2176759582 shutdown_connections 2023-12-19T17:58:08.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc9666640 1 -- 172.21.15.26:0/2176759582 wait complete. 2023-12-19T17:58:08.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc9666640 1 Processor -- start 2023-12-19T17:58:08.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.218+0000 7fedc9666640 1 -- start start 2023-12-19T17:58:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedc9666640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedc40658e0 0x7fedc415ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedc9666640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedc40ca060 0x7fedc415d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedc9666640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedc415cdb0 0x7fedc40c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedc9666640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc415dcd0 con 0x7fedc40ca060 2023-12-19T17:58:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedc9666640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc415dfb0 con 0x7fedc415cdb0 2023-12-19T17:58:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedc9666640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc415e290 con 0x7fedc40658e0 2023-12-19T17:58:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedbbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedc40ca060 0x7fedc415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedbbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedc40ca060 0x7fedc415d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51028/0 (socket says 172.21.15.26:51028) 2023-12-19T17:58:08.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedbbfff640 1 -- 172.21.15.26:0/1752356996 learned_addr learned my addr 172.21.15.26:0/1752356996 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedc2ffd640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedc40658e0 0x7fedc415ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedc37fe640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedc415cdb0 0x7fedc40c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedbbfff640 1 -- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedc40658e0 msgr2=0x7fedc415ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedbbfff640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedc40658e0 0x7fedc415ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedbbfff640 1 -- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedc415cdb0 msgr2=0x7fedc40c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedbbfff640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedc415cdb0 0x7fedc40c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedbbfff640 1 -- 172.21.15.26:0/1752356996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc009b30 con 0x7fedc40ca060 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedc37fe640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedc415cdb0 0x7fedc40c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedbbfff640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedc40ca060 0x7fedc415d310 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fedac00e930 tx=0x7fedac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedbb7fe640 1 -- 172.21.15.26:0/1752356996 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac009690 con 0x7fedc40ca060 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedbb7fe640 1 -- 172.21.15.26:0/1752356996 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac00ca00 con 0x7fedc40ca060 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedc9666640 1 -- 172.21.15.26:0/1752356996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc415e570 con 0x7fedc40ca060 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedbb7fe640 1 -- 172.21.15.26:0/1752356996 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac00cd70 con 0x7fedc40ca060 2023-12-19T17:58:08.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedc9666640 1 -- 172.21.15.26:0/1752356996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc415d850 con 0x7fedc40ca060 2023-12-19T17:58:08.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:58:08.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedc9666640 1 -- 172.21.15.26:0/1752356996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc4074270 con 0x7fedc40ca060 2023-12-19T17:58:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.222+0000 7fedbb7fe640 1 -- 172.21.15.26:0/1752356996 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fedac013470 con 0x7fedc40ca060 2023-12-19T17:58:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.226+0000 7fedbb7fe640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feda4065b80 0x7feda4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.226+0000 7fedbb7fe640 1 -- 172.21.15.26:0/1752356996 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6277+0+0 (secure 0 0 0) 0x7fedac0837c0 con 0x7fedc40ca060 2023-12-19T17:58:08.231 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:58:08.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.226+0000 7fedc2ffd640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feda4065b80 0x7feda4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.226+0000 7fedc2ffd640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feda4065b80 0x7feda4068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fedb400d8d0 tx=0x7fedb4008040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:08.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.230+0000 7fedbb7fe640 1 -- 172.21.15.26:0/1752356996 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fedac050ee0 con 0x7fedc40ca060 2023-12-19T17:58:08.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.386+0000 7fedc9666640 1 -- 172.21.15.26:0/1752356996 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fedc4070c80 con 0x7fedc40ca060 2023-12-19T17:58:08.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.394+0000 7fedbb7fe640 1 -- 172.21.15.26:0/1752356996 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7fedac054500 con 0x7fedc40ca060 2023-12-19T17:58:08.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.394+0000 7fedc9666640 1 -- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feda4065b80 msgr2=0x7feda4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:08.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.394+0000 7fedc9666640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feda4065b80 0x7feda4068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fedb400d8d0 tx=0x7fedb4008040 comp rx=0 tx=0).stop 2023-12-19T17:58:08.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.394+0000 7fedc9666640 1 -- 172.21.15.26:0/1752356996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedc40ca060 msgr2=0x7fedc415d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:08.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.394+0000 7fedc9666640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedc40ca060 0x7fedc415d310 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fedac00e930 tx=0x7fedac00edf0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.394+0000 7fedc9666640 1 -- 172.21.15.26:0/1752356996 shutdown_connections 2023-12-19T17:58:08.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.394+0000 7fedc9666640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fedc415cdb0 0x7fedc40c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.394+0000 7fedc9666640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fedc40ca060 0x7fedc415d310 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.394+0000 7fedc9666640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7feda4065b80 0x7feda4068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.394+0000 7fedc9666640 1 --2- 172.21.15.26:0/1752356996 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fedc40658e0 0x7fedc415ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.394+0000 7fedc9666640 1 -- 172.21.15.26:0/1752356996 >> 172.21.15.26:0/1752356996 conn(0x7fedc4069d20 msgr2=0x7fedc40c5600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:08.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.394+0000 7fedc9666640 1 -- 172.21.15.26:0/1752356996 shutdown_connections 2023-12-19T17:58:08.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.394+0000 7fedc9666640 1 -- 172.21.15.26:0/1752356996 wait complete. 2023-12-19T17:58:08.419 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T17:58:08.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T17:58:08.478 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T17:58:08.531 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T17:58:08.531 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:58:08.531 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:58:08.541 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T17:58:08.542 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T17:58:08.542 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:58:08.552 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T17:58:08.553 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:58:08.553 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:58:08.591 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T17:58:08.591 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T17:58:08.592 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:58:08.608 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T17:58:08.608 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T17:58:08.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:58:08.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.774+0000 7f1567ec4640 1 Processor -- start 2023-12-19T17:58:08.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.774+0000 7f1567ec4640 1 -- start start 2023-12-19T17:58:08.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1567ec4640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f15600b85e0 0x7f15600cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1567ec4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600cde60 0x7f15600d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1567ec4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15600d0d40 con 0x7f156015cda0 2023-12-19T17:58:08.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1567ec4640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15600ce2f0 con 0x7f15600b85e0 2023-12-19T17:58:08.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1567ec4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1560161150 con 0x7f15600cde60 2023-12-19T17:58:08.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1565c60640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f156015cda0 0x7f1560160920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50142/0 (socket says 172.21.15.26:50142) 2023-12-19T17:58:08.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1565c60640 1 -- 172.21.15.26:0/3032906546 learned_addr learned my addr 172.21.15.26:0/3032906546 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:08.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1566461640 1 --2- 172.21.15.26:0/3032906546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600cde60 0x7f15600d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f156545f640 1 --2- 172.21.15.26:0/3032906546 >> v2:172.21.15.160:3300/0 conn(0x7f15600b85e0 0x7f15600cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977308621 0 0) 0x7f15600d0d40 con 0x7f156015cda0 2023-12-19T17:58:08.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f154c003530 con 0x7f156015cda0 2023-12-19T17:58:08.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 524827047 0 0) 0x7f154c003530 con 0x7f156015cda0 2023-12-19T17:58:08.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600cde60 msgr2=0x7f15600d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:08.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 --2- 172.21.15.26:0/3032906546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600cde60 0x7f15600d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 >> v2:172.21.15.160:3300/0 conn(0x7f15600b85e0 msgr2=0x7f15600cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:08.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 --2- 172.21.15.26:0/3032906546 >> v2:172.21.15.160:3300/0 conn(0x7f15600b85e0 0x7f15600cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1560161430 con 0x7f156015cda0 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f155c001680 con 0x7f156015cda0 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 >> v1:172.21.15.26:6789/0 conn(0x7f156015cda0 legacy=0x7f1560160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 --2- 172.21.15.26:0/3032906546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f154c003980 0x7f154c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 --2- 172.21.15.26:0/3032906546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600cde60 0x7f154c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 --2- 172.21.15.26:0/3032906546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154c004d10 0x7f154c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15600ce2f0 con 0x7f154c004d10 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1560161150 con 0x7f154c003980 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f154c00b990 con 0x7f15600cde60 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1566461640 1 --2- 172.21.15.26:0/3032906546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154c004d10 0x7f154c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1565c60640 1 --2- 172.21.15.26:0/3032906546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f154c003980 0x7f154c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f156545f640 1 --2- 172.21.15.26:0/3032906546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600cde60 0x7f154c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1566461640 1 -- 172.21.15.26:0/3032906546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600cde60 msgr2=0x7f154c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1566461640 1 --2- 172.21.15.26:0/3032906546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600cde60 0x7f154c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1566461640 1 -- 172.21.15.26:0/3032906546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f154c003980 msgr2=0x7f154c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:08.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1566461640 1 --2- 172.21.15.26:0/3032906546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f154c003980 0x7f154c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f1566461640 1 -- 172.21.15.26:0/3032906546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c003530 con 0x7f154c004d10 2023-12-19T17:58:08.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.778+0000 7f156545f640 1 --2- 172.21.15.26:0/3032906546 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600cde60 0x7f154c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:08.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.782+0000 7f1566461640 1 --2- 172.21.15.26:0/3032906546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154c004d10 0x7f154c005190 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f1558009fc0 tx=0x7f155801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa974450d7b30598 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:08.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.782+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580a5c70 con 0x7f154c004d10 2023-12-19T17:58:08.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.782+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155800f040 con 0x7f154c004d10 2023-12-19T17:58:08.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.782+0000 7f1564c5e640 1 -- 172.21.15.26:0/3032906546 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580ad440 con 0x7f154c004d10 2023-12-19T17:58:08.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.782+0000 7f1567ec4640 1 -- 172.21.15.26:0/3032906546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154c004d10 msgr2=0x7f154c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:08.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.782+0000 7f1567ec4640 1 --2- 172.21.15.26:0/3032906546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154c004d10 0x7f154c005190 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f1558009fc0 tx=0x7f155801cc60 comp rx=0 tx=0).stop 2023-12-19T17:58:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.782+0000 7f1566461640 1 -- 172.21.15.26:0/3032906546 reap_dead start 2023-12-19T17:58:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.782+0000 7f1567ec4640 1 -- 172.21.15.26:0/3032906546 shutdown_connections 2023-12-19T17:58:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.782+0000 7f1567ec4640 1 -- 172.21.15.26:0/3032906546 >> 172.21.15.26:0/3032906546 conn(0x7f15600658e0 msgr2=0x7f1560065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.782+0000 7f1567ec4640 1 -- 172.21.15.26:0/3032906546 shutdown_connections 2023-12-19T17:58:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.782+0000 7f1567ec4640 1 -- 172.21.15.26:0/3032906546 wait complete. 2023-12-19T17:58:08.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1567ec4640 1 Processor -- start 2023-12-19T17:58:08.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1567ec4640 1 -- start start 2023-12-19T17:58:08.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1567ec4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154c004d10 0x7f156014f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1567ec4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f156015d4d0 0x7f15600c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1567ec4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600b30c0 0x7f15600c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1567ec4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15600c9660 con 0x7f154c004d10 2023-12-19T17:58:08.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1567ec4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15600b4640 con 0x7f156015d4d0 2023-12-19T17:58:08.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1567ec4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156014f970 con 0x7f15600b30c0 2023-12-19T17:58:08.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1565c60640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154c004d10 0x7f156014f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1565c60640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154c004d10 0x7f156014f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51056/0 (socket says 172.21.15.26:51056) 2023-12-19T17:58:08.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1565c60640 1 -- 172.21.15.26:0/2790184391 learned_addr learned my addr 172.21.15.26:0/2790184391 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:08.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f156545f640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f156015d4d0 0x7f15600c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1566461640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600b30c0 0x7f15600c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1565c60640 1 -- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600b30c0 msgr2=0x7f15600c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:08.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1565c60640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600b30c0 0x7f15600c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1565c60640 1 -- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f156015d4d0 msgr2=0x7f15600c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:08.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1565c60640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f156015d4d0 0x7f15600c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1565c60640 1 -- 172.21.15.26:0/2790184391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15600b3a30 con 0x7f154c004d10 2023-12-19T17:58:08.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1565c60640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154c004d10 0x7f156014f430 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f155c007340 tx=0x7f155c007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:08.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f15567fc640 1 -- 172.21.15.26:0/2790184391 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c00f9c0 con 0x7f154c004d10 2023-12-19T17:58:08.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f15567fc640 1 -- 172.21.15.26:0/2790184391 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c00fb40 con 0x7f154c004d10 2023-12-19T17:58:08.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1567ec4640 1 -- 172.21.15.26:0/2790184391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15600b3d10 con 0x7f154c004d10 2023-12-19T17:58:08.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f15567fc640 1 -- 172.21.15.26:0/2790184391 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c012350 con 0x7f154c004d10 2023-12-19T17:58:08.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.786+0000 7f1567ec4640 1 -- 172.21.15.26:0/2790184391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15600b42d0 con 0x7f154c004d10 2023-12-19T17:58:08.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.790+0000 7f1567ec4640 1 -- 172.21.15.26:0/2790184391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15600b4490 con 0x7f154c004d10 2023-12-19T17:58:08.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.790+0000 7f15567fc640 1 -- 172.21.15.26:0/2790184391 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f155c0124b0 con 0x7f154c004d10 2023-12-19T17:58:08.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.790+0000 7f15567fc640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1544065bb0 0x7f1544068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:08.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.790+0000 7f15567fc640 1 -- 172.21.15.26:0/2790184391 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6277+0+0 (secure 0 0 0) 0x7f155c083080 con 0x7f154c004d10 2023-12-19T17:58:08.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.794+0000 7f156545f640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1544065bb0 0x7f1544068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:08.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.794+0000 7f156545f640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1544065bb0 0x7f1544068070 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15600b7ef0 tx=0x7f1550008040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:08.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.798+0000 7f15567fc640 1 -- 172.21.15.26:0/2790184391 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f155c0507a0 con 0x7f154c004d10 2023-12-19T17:58:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.986+0000 7f1567ec4640 1 -- 172.21.15.26:0/2790184391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15600cfc20 con 0x7f154c004d10 2023-12-19T17:58:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.986+0000 7f15567fc640 1 -- 172.21.15.26:0/2790184391 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f155c01ade0 con 0x7f154c004d10 2023-12-19T17:58:08.991 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:08.991 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7625,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2850013418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2850013418},{"type":"v1","addr":"172.21.15.26:6835","nonce":2850013418}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2213513866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":2213513866},{"type":"v1","addr":"172.21.15.160:6836","nonce":2213513866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":7631,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/4292779848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":4292779848},{"type":"v1","addr":"172.21.15.160:6837","nonce":4292779848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":68},{"gid":9432,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3210345789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3210345789},{"type":"v1","addr":"172.21.15.26:6834","nonce":3210345789}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":75}],"filesystems":[]} 2023-12-19T17:58:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 75 2023-12-19T17:58:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.990+0000 7f1567ec4640 1 -- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1544065bb0 msgr2=0x7f1544068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.990+0000 7f1567ec4640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1544065bb0 0x7f1544068070 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f15600b7ef0 tx=0x7f1550008040 comp rx=0 tx=0).stop 2023-12-19T17:58:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.990+0000 7f1567ec4640 1 -- 172.21.15.26:0/2790184391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154c004d10 msgr2=0x7f156014f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.990+0000 7f1567ec4640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154c004d10 0x7f156014f430 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f155c007340 tx=0x7f155c007740 comp rx=0 tx=0).stop 2023-12-19T17:58:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.990+0000 7f1567ec4640 1 -- 172.21.15.26:0/2790184391 shutdown_connections 2023-12-19T17:58:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.990+0000 7f1567ec4640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1544065bb0 0x7f1544068070 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.990+0000 7f1567ec4640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15600b30c0 0x7f15600c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.990+0000 7f1567ec4640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f156015d4d0 0x7f15600c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.990+0000 7f1567ec4640 1 --2- 172.21.15.26:0/2790184391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f154c004d10 0x7f156014f430 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.990+0000 7f1567ec4640 1 -- 172.21.15.26:0/2790184391 >> 172.21.15.26:0/2790184391 conn(0x7f15600658e0 msgr2=0x7f15600d1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.990+0000 7f1567ec4640 1 -- 172.21.15.26:0/2790184391 shutdown_connections 2023-12-19T17:58:08.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:08.990+0000 7f1567ec4640 1 -- 172.21.15.26:0/2790184391 wait complete. 2023-12-19T17:58:09.012 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T17:58:09.012 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T17:58:09.012 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T17:58:09.013 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T17:58:09.013 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T17:58:09.013 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T17:58:09.013 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T17:58:09.013 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T17:58:09.013 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T17:58:09.013 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T17:58:09.014 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T17:58:09.014 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T17:58:09.014 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T17:58:09.014 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T17:58:09.014 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T17:58:09.014 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T17:58:09.025 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T17:58:09.025 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T17:58:09.026 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T17:58:09.027 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T17:58:09.028 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T17:58:09.028 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T17:58:09.029 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T17:58:09.029 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T17:58:09.075 INFO:tasks.ceph.mds.a:Started 2023-12-19T17:58:09.075 DEBUG:teuthology.parallel:result is None 2023-12-19T17:58:09.076 INFO:tasks.ceph.mds.c:Started 2023-12-19T17:58:09.077 DEBUG:teuthology.parallel:result is None 2023-12-19T17:58:09.079 INFO:tasks.ceph.mds.b:Started 2023-12-19T17:58:09.079 DEBUG:teuthology.parallel:result is None 2023-12-19T17:58:09.081 INFO:tasks.ceph.mds.d:Started 2023-12-19T17:58:09.081 DEBUG:teuthology.parallel:result is None 2023-12-19T17:58:09.081 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T17:58:09.165 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T17:58:09.165 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T17:58:09.165 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T17:58:09.166 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T17:58:09.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa2488c640 1 Processor -- start 2023-12-19T17:58:09.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa2488c640 1 -- start start 2023-12-19T17:58:09.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa2488c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa20155030 0x7faa2015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:09.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa2488c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2006c970 con 0x7faa200ce740 2023-12-19T17:58:09.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa2488c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2015b1f0 con 0x7faa2015f370 2023-12-19T17:58:09.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa2488c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2006c790 con 0x7faa20155030 2023-12-19T17:58:09.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa1ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faa200ce740 0x7faa200ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50152/0 (socket says 172.21.15.26:50152) 2023-12-19T17:58:09.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa1ed9d640 1 -- 172.21.15.26:0/1170898170 learned_addr learned my addr 172.21.15.26:0/1170898170 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:09.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa1e59c640 1 --2- 172.21.15.26:0/1170898170 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa20155030 0x7faa2015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:09.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa1d59a640 1 -- 172.21.15.26:0/1170898170 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86655638 0 0) 0x7faa2006c970 con 0x7faa200ce740 2023-12-19T17:58:09.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa1d59a640 1 -- 172.21.15.26:0/1170898170 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa08003530 con 0x7faa200ce740 2023-12-19T17:58:09.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa1d59a640 1 -- 172.21.15.26:0/1170898170 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828164187 0 0) 0x7faa2015b1f0 con 0x7faa2015f370 2023-12-19T17:58:09.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa1d59a640 1 -- 172.21.15.26:0/1170898170 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa2006c970 con 0x7faa2015f370 2023-12-19T17:58:09.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa1e59c640 1 -- 172.21.15.26:0/1170898170 >> v1:172.21.15.160:6789/0 conn(0x7faa2015f370 legacy=0x7faa2006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:09.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa1d59a640 1 -- 172.21.15.26:0/1170898170 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3404125813 0 0) 0x7faa08003530 con 0x7faa200ce740 2023-12-19T17:58:09.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa1e59c640 1 -- 172.21.15.26:0/1170898170 >> v1:172.21.15.26:6789/0 conn(0x7faa200ce740 legacy=0x7faa200ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:09.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.230+0000 7faa1e59c640 1 -- 172.21.15.26:0/1170898170 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2015b4d0 con 0x7faa20155030 2023-12-19T17:58:09.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.234+0000 7faa1e59c640 1 --2- 172.21.15.26:0/1170898170 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa20155030 0x7faa2015bcd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faa100097f0 tx=0x7faa1001c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93eeb061e743859d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:09.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.234+0000 7faa1d59a640 1 -- 172.21.15.26:0/1170898170 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18001a00 con 0x7faa20155030 2023-12-19T17:58:09.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.234+0000 7faa1d59a640 1 -- 172.21.15.26:0/1170898170 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa100a7050 con 0x7faa20155030 2023-12-19T17:58:09.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.234+0000 7faa1d59a640 1 -- 172.21.15.26:0/1170898170 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa100ab620 con 0x7faa20155030 2023-12-19T17:58:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.234+0000 7faa2488c640 1 -- 172.21.15.26:0/1170898170 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa20155030 msgr2=0x7faa2015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.234+0000 7faa2488c640 1 --2- 172.21.15.26:0/1170898170 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa20155030 0x7faa2015bcd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faa100097f0 tx=0x7faa1001c3b0 comp rx=0 tx=0).stop 2023-12-19T17:58:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.234+0000 7faa2488c640 1 -- 172.21.15.26:0/1170898170 shutdown_connections 2023-12-19T17:58:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.234+0000 7faa2488c640 1 --2- 172.21.15.26:0/1170898170 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa20155030 0x7faa2015bcd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.234+0000 7faa2488c640 1 -- 172.21.15.26:0/1170898170 >> 172.21.15.26:0/1170898170 conn(0x7faa200658e0 msgr2=0x7faa20065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.234+0000 7faa2488c640 1 -- 172.21.15.26:0/1170898170 shutdown_connections 2023-12-19T17:58:09.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.234+0000 7faa2488c640 1 -- 172.21.15.26:0/1170898170 wait complete. 2023-12-19T17:58:09.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa2488c640 1 Processor -- start 2023-12-19T17:58:09.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa2488c640 1 -- start start 2023-12-19T17:58:09.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa2488c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa200ce740 0x7faa20146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:09.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa2488c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa20155030 0x7faa2014d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:09.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa2488c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa2015f370 0x7faa200b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:09.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa2488c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa201481a0 con 0x7faa2015f370 2023-12-19T17:58:09.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa2488c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa20147310 con 0x7faa20155030 2023-12-19T17:58:09.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa2488c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa20147590 con 0x7faa200ce740 2023-12-19T17:58:09.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa1dd9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa20155030 0x7faa2014d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa1dd9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa20155030 0x7faa2014d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:33922/0 (socket says 172.21.15.26:33922) 2023-12-19T17:58:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa1dd9b640 1 -- 172.21.15.26:0/2673229456 learned_addr learned my addr 172.21.15.26:0/2673229456 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa1e59c640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa200ce740 0x7faa20146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa1ed9d640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa2015f370 0x7faa200b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa1dd9b640 1 -- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa200ce740 msgr2=0x7faa20146dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa1dd9b640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa200ce740 0x7faa20146dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa1dd9b640 1 -- 172.21.15.26:0/2673229456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa2015f370 msgr2=0x7faa200b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa1dd9b640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa2015f370 0x7faa200b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa1dd9b640 1 -- 172.21.15.26:0/2673229456 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa10009420 con 0x7faa20155030 2023-12-19T17:58:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.238+0000 7faa1dd9b640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa20155030 0x7faa2014d760 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c500 tx=0x7faa1400c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.242+0000 7faa0effd640 1 -- 172.21.15.26:0/2673229456 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140052d0 con 0x7faa20155030 2023-12-19T17:58:09.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.242+0000 7faa0effd640 1 -- 172.21.15.26:0/2673229456 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14005430 con 0x7faa20155030 2023-12-19T17:58:09.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.242+0000 7faa2488c640 1 -- 172.21.15.26:0/2673229456 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa20147870 con 0x7faa20155030 2023-12-19T17:58:09.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.242+0000 7faa0effd640 1 -- 172.21.15.26:0/2673229456 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14012870 con 0x7faa20155030 2023-12-19T17:58:09.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.242+0000 7faa2488c640 1 -- 172.21.15.26:0/2673229456 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa20147e30 con 0x7faa20155030 2023-12-19T17:58:09.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.242+0000 7faa2488c640 1 -- 172.21.15.26:0/2673229456 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa20148350 con 0x7faa20155030 2023-12-19T17:58:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.242+0000 7faa0effd640 1 -- 172.21.15.26:0/2673229456 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faa140129d0 con 0x7faa20155030 2023-12-19T17:58:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.242+0000 7faa0effd640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa00065b80 0x7faa00068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.242+0000 7faa0effd640 1 -- 172.21.15.26:0/2673229456 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6277+0+0 (secure 0 0 0) 0x7faa14083190 con 0x7faa20155030 2023-12-19T17:58:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.242+0000 7faa1e59c640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa00065b80 0x7faa00068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.246+0000 7faa1e59c640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa00065b80 0x7faa00068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faa100ad000 tx=0x7faa100a6000 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.246+0000 7faa0effd640 1 -- 172.21.15.26:0/2673229456 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faa20148350 con 0x7faa20155030 2023-12-19T17:58:09.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:09.418+0000 7faa2488c640 1 -- 172.21.15.26:0/2673229456 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7faa20148310 con 0x7faa20155030 2023-12-19T17:58:10.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.202+0000 7faa0effd640 1 -- 172.21.15.26:0/2673229456 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v173) v1 ==== 97+0+0 (secure 0 0 0) 0x7faa14050770 con 0x7faa20155030 2023-12-19T17:58:10.204 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2023-12-19T17:58:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.202+0000 7faa2488c640 1 -- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa00065b80 msgr2=0x7faa00068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.202+0000 7faa2488c640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa00065b80 0x7faa00068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faa100ad000 tx=0x7faa100a6000 comp rx=0 tx=0).stop 2023-12-19T17:58:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.202+0000 7faa2488c640 1 -- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa20155030 msgr2=0x7faa2014d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.202+0000 7faa2488c640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa20155030 0x7faa2014d760 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7faa1400c500 tx=0x7faa1400c9c0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.202+0000 7faa2488c640 1 -- 172.21.15.26:0/2673229456 shutdown_connections 2023-12-19T17:58:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.202+0000 7faa2488c640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa2015f370 0x7faa200b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.202+0000 7faa2488c640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa20155030 0x7faa2014d760 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.202+0000 7faa2488c640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7faa00065b80 0x7faa00068040 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.202+0000 7faa2488c640 1 --2- 172.21.15.26:0/2673229456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa200ce740 0x7faa20146dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.202+0000 7faa2488c640 1 -- 172.21.15.26:0/2673229456 >> 172.21.15.26:0/2673229456 conn(0x7faa200658e0 msgr2=0x7faa2014e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.202+0000 7faa2488c640 1 -- 172.21.15.26:0/2673229456 shutdown_connections 2023-12-19T17:58:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.202+0000 7faa2488c640 1 -- 172.21.15.26:0/2673229456 wait complete. 2023-12-19T17:58:10.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T17:58:10.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.374+0000 7ff75790a640 1 Processor -- start 2023-12-19T17:58:10.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.374+0000 7ff75790a640 1 -- start start 2023-12-19T17:58:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff75790a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff750154220 0x7ff75015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff75790a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff7500cee10 0x7ff75015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff75790a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75006bc30 con 0x7ff75006b3d0 2023-12-19T17:58:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff75790a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7500cc6e0 con 0x7ff750154220 2023-12-19T17:58:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff75790a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75015afc0 con 0x7ff7500cee10 2023-12-19T17:58:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff7556a6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff750154220 0x7ff75015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff7556a6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff750154220 0x7ff75015aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51106/0 (socket says 172.21.15.26:51106) 2023-12-19T17:58:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff7556a6640 1 -- 172.21.15.26:0/3700481168 learned_addr learned my addr 172.21.15.26:0/3700481168 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff755ea7640 1 --2- 172.21.15.26:0/3700481168 >> v2:172.21.15.160:3300/0 conn(0x7ff7500cee10 0x7ff75015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff7556a6640 1 -- 172.21.15.26:0/3700481168 >> v2:172.21.15.160:3300/0 conn(0x7ff7500cee10 msgr2=0x7ff75015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff7556a6640 1 --2- 172.21.15.26:0/3700481168 >> v2:172.21.15.160:3300/0 conn(0x7ff7500cee10 0x7ff75015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff7556a6640 1 -- 172.21.15.26:0/3700481168 >> v1:172.21.15.160:6789/0 conn(0x7ff75006b3d0 legacy=0x7ff7500ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff7556a6640 1 -- 172.21.15.26:0/3700481168 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75006b950 con 0x7ff750154220 2023-12-19T17:58:10.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.378+0000 7ff755ea7640 1 --2- 172.21.15.26:0/3700481168 >> v2:172.21.15.160:3300/0 conn(0x7ff7500cee10 0x7ff75015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff7556a6640 1 --2- 172.21.15.26:0/3700481168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff750154220 0x7ff75015aa80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7ff7500cdf30 tx=0x7ff74c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d3aa1537389f7a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff747fff640 1 -- 172.21.15.26:0/3700481168 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c0aa5f0 con 0x7ff750154220 2023-12-19T17:58:10.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff747fff640 1 -- 172.21.15.26:0/3700481168 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c0aa750 con 0x7ff750154220 2023-12-19T17:58:10.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff747fff640 1 -- 172.21.15.26:0/3700481168 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c0aaae0 con 0x7ff750154220 2023-12-19T17:58:10.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff75790a640 1 -- 172.21.15.26:0/3700481168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff750154220 msgr2=0x7ff75015aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff75790a640 1 --2- 172.21.15.26:0/3700481168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff750154220 0x7ff75015aa80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7ff7500cdf30 tx=0x7ff74c01c490 comp rx=0 tx=0).stop 2023-12-19T17:58:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff75790a640 1 -- 172.21.15.26:0/3700481168 shutdown_connections 2023-12-19T17:58:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff75790a640 1 --2- 172.21.15.26:0/3700481168 >> v2:172.21.15.160:3300/0 conn(0x7ff7500cee10 0x7ff75015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff75790a640 1 --2- 172.21.15.26:0/3700481168 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff750154220 0x7ff75015aa80 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff75790a640 1 -- 172.21.15.26:0/3700481168 >> 172.21.15.26:0/3700481168 conn(0x7ff7500658e0 msgr2=0x7ff750065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff75790a640 1 -- 172.21.15.26:0/3700481168 shutdown_connections 2023-12-19T17:58:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff75790a640 1 -- 172.21.15.26:0/3700481168 wait complete. 2023-12-19T17:58:10.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff75790a640 1 Processor -- start 2023-12-19T17:58:10.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.382+0000 7ff75790a640 1 -- start start 2023-12-19T17:58:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff75790a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff75006b3d0 0x7ff750145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff75790a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff7500cee10 0x7ff75015c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff75790a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff750154220 0x7ff750158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff75790a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff750152a20 con 0x7ff750154220 2023-12-19T17:58:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff75790a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff750152d00 con 0x7ff75006b3d0 2023-12-19T17:58:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff75790a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff750152fe0 con 0x7ff7500cee10 2023-12-19T17:58:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff755ea7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff750154220 0x7ff750158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff755ea7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff750154220 0x7ff750158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51116/0 (socket says 172.21.15.26:51116) 2023-12-19T17:58:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff755ea7640 1 -- 172.21.15.26:0/3388536804 learned_addr learned my addr 172.21.15.26:0/3388536804 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff7556a6640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff75006b3d0 0x7ff750145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff754ea5640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff7500cee10 0x7ff75015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff755ea7640 1 -- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff7500cee10 msgr2=0x7ff75015c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff755ea7640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff7500cee10 0x7ff75015c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff755ea7640 1 -- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff75006b3d0 msgr2=0x7ff750145bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff755ea7640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff75006b3d0 0x7ff750145bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff755ea7640 1 -- 172.21.15.26:0/3388536804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c0094d0 con 0x7ff750154220 2023-12-19T17:58:10.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff755ea7640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff750154220 0x7ff750158aa0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7ff748007e30 tx=0x7ff74800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:10.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff745ffb640 1 -- 172.21.15.26:0/3388536804 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff748003ba0 con 0x7ff750154220 2023-12-19T17:58:10.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff745ffb640 1 -- 172.21.15.26:0/3388536804 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74800f040 con 0x7ff750154220 2023-12-19T17:58:10.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff75790a640 1 -- 172.21.15.26:0/3388536804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7501532c0 con 0x7ff750154220 2023-12-19T17:58:10.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff745ffb640 1 -- 172.21.15.26:0/3388536804 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74800de80 con 0x7ff750154220 2023-12-19T17:58:10.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff75790a640 1 -- 172.21.15.26:0/3388536804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff750152660 con 0x7ff750154220 2023-12-19T17:58:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff75790a640 1 -- 172.21.15.26:0/3388536804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff750074270 con 0x7ff750154220 2023-12-19T17:58:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.386+0000 7ff745ffb640 1 -- 172.21.15.26:0/3388536804 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff748004030 con 0x7ff750154220 2023-12-19T17:58:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.390+0000 7ff745ffb640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff734065b80 0x7ff734068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.390+0000 7ff745ffb640 1 -- 172.21.15.26:0/3388536804 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5589+0+0 (secure 0 0 0) 0x7ff7480836f0 con 0x7ff750154220 2023-12-19T17:58:10.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.390+0000 7ff7556a6640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff734065b80 0x7ff734068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.390+0000 7ff7556a6640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff734065b80 0x7ff734068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0ae000 tx=0x7ff74c0b5040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:10.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.394+0000 7ff745ffb640 1 -- 172.21.15.26:0/3388536804 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff748050f10 con 0x7ff750154220 2023-12-19T17:58:10.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff75790a640 1 -- 172.21.15.26:0/3388536804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff750153770 con 0x7ff750154220 2023-12-19T17:58:10.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff745ffb640 1 -- 172.21.15.26:0/3388536804 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v43) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff748054530 con 0x7ff750154220 2023-12-19T17:58:10.609 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:10.609 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T17:58:10.609 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2023-12-19T17:58:10.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg==", 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg==", 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.610 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg==", 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.612 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2023-12-19T17:58:10.612 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw==", 2023-12-19T17:58:10.612 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.612 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:58:10.612 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:58:10.612 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:58:10.612 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.612 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.612 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.612 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2023-12-19T17:58:10.613 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig==", 2023-12-19T17:58:10.613 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:58:10.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:58:10.613 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.613 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA==", 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ==", 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.614 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ==", 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.615 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A==", 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ==", 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.616 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg==", 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg==", 2023-12-19T17:58:10.617 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ==", 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.618 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ==", 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ==", 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.619 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ==", 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.620 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlT5ynDxAA7Sjbdwrc1RyAXJboysB54Q==", 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA==", 2023-12-19T17:58:10.621 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg==", 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.622 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:10.623 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-12-19T17:58:10.623 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T17:58:10.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff75790a640 1 -- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff734065b80 msgr2=0x7ff734068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff75790a640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff734065b80 0x7ff734068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0ae000 tx=0x7ff74c0b5040 comp rx=0 tx=0).stop 2023-12-19T17:58:10.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff75790a640 1 -- 172.21.15.26:0/3388536804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff750154220 msgr2=0x7ff750158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff75790a640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff750154220 0x7ff750158aa0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7ff748007e30 tx=0x7ff74800d4a0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff75790a640 1 -- 172.21.15.26:0/3388536804 shutdown_connections 2023-12-19T17:58:10.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff75790a640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff750154220 0x7ff750158aa0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff75790a640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7ff734065b80 0x7ff734068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff75790a640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff7500cee10 0x7ff75015c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff75790a640 1 --2- 172.21.15.26:0/3388536804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff75006b3d0 0x7ff750145bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff75790a640 1 -- 172.21.15.26:0/3388536804 >> 172.21.15.26:0/3388536804 conn(0x7ff7500658e0 msgr2=0x7ff7500ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:10.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff75790a640 1 -- 172.21.15.26:0/3388536804 shutdown_connections 2023-12-19T17:58:10.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.606+0000 7ff75790a640 1 -- 172.21.15.26:0/3388536804 wait complete. 2023-12-19T17:58:10.631 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:58:10.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.770+0000 7f4910b6d640 1 Processor -- start 2023-12-19T17:58:10.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.770+0000 7f4910b6d640 1 -- start start 2023-12-19T17:58:10.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f4910b6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c06b3d0 0x7f490c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f4910b6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c06bc30 con 0x7f490c154220 2023-12-19T17:58:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f4910b6d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c0cc6e0 con 0x7f490c0cec40 2023-12-19T17:58:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f4910b6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0cf0d0 con 0x7f490c06b3d0 2023-12-19T17:58:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490a59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f490c154220 0x7f490c15aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50168/0 (socket says 172.21.15.26:50168) 2023-12-19T17:58:10.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490a59c640 1 -- 172.21.15.26:0/3493638949 learned_addr learned my addr 172.21.15.26:0/3493638949 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f4909d9b640 1 --2- 172.21.15.26:0/3493638949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c06b3d0 0x7f490c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696139041 0 0) 0x7f490c06bc30 con 0x7f490c154220 2023-12-19T17:58:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48f4003530 con 0x7f490c154220 2023-12-19T17:58:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127798780 0 0) 0x7f490c0cc6e0 con 0x7f490c0cec40 2023-12-19T17:58:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f490c06bc30 con 0x7f490c0cec40 2023-12-19T17:58:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1922111870 0 0) 0x7f48f4003530 con 0x7f490c154220 2023-12-19T17:58:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c06b3d0 msgr2=0x7f490c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 --2- 172.21.15.26:0/3493638949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c06b3d0 0x7f490c0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 >> v1:172.21.15.160:6789/0 conn(0x7f490c0cec40 legacy=0x7f490c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c06b950 con 0x7f490c154220 2023-12-19T17:58:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f48fc001680 con 0x7f490c154220 2023-12-19T17:58:10.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f4909d9b640 1 --2- 172.21.15.26:0/3493638949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c06b3d0 0x7f490c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 >> v1:172.21.15.26:6789/0 conn(0x7f490c154220 legacy=0x7f490c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 --2- 172.21.15.26:0/3493638949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f4003b40 0x7f48f4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 --2- 172.21.15.26:0/3493638949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f40047d0 0x7f48f4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 --2- 172.21.15.26:0/3493638949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c06b3d0 0x7f48f4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0cf0d0 con 0x7f48f4003b40 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f400b9c0 con 0x7f48f40047d0 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f400bb80 con 0x7f490c06b3d0 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490a59c640 1 --2- 172.21.15.26:0/3493638949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f4003b40 0x7f48f4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f4909d9b640 1 --2- 172.21.15.26:0/3493638949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f40047d0 0x7f48f4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490ad9d640 1 --2- 172.21.15.26:0/3493638949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c06b3d0 0x7f48f4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490a59c640 1 -- 172.21.15.26:0/3493638949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c06b3d0 msgr2=0x7f48f4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490a59c640 1 --2- 172.21.15.26:0/3493638949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c06b3d0 0x7f48f4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490a59c640 1 -- 172.21.15.26:0/3493638949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f40047d0 msgr2=0x7f48f4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:10.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490a59c640 1 --2- 172.21.15.26:0/3493638949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f40047d0 0x7f48f4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.774+0000 7f490a59c640 1 -- 172.21.15.26:0/3493638949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4900009420 con 0x7f48f4003b40 2023-12-19T17:58:10.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.778+0000 7f490a59c640 1 --2- 172.21.15.26:0/3493638949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f4003b40 0x7f48f4003f40 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f48fc001ec0 tx=0x7f48fc01cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c344a3ee93d268 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:10.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.778+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc0058e0 con 0x7f48f4003b40 2023-12-19T17:58:10.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.778+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc005a60 con 0x7f48f4003b40 2023-12-19T17:58:10.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.778+0000 7f490959a640 1 -- 172.21.15.26:0/3493638949 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc005e10 con 0x7f48f4003b40 2023-12-19T17:58:10.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.778+0000 7f4910b6d640 1 -- 172.21.15.26:0/3493638949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f4003b40 msgr2=0x7f48f4003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.778+0000 7f4910b6d640 1 --2- 172.21.15.26:0/3493638949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48f4003b40 0x7f48f4003f40 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f48fc001ec0 tx=0x7f48fc01cb50 comp rx=0 tx=0).stop 2023-12-19T17:58:10.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.778+0000 7f490ad9d640 1 -- 172.21.15.26:0/3493638949 reap_dead start 2023-12-19T17:58:10.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.778+0000 7f4910b6d640 1 -- 172.21.15.26:0/3493638949 shutdown_connections 2023-12-19T17:58:10.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.778+0000 7f4910b6d640 1 -- 172.21.15.26:0/3493638949 >> 172.21.15.26:0/3493638949 conn(0x7f490c0658e0 msgr2=0x7f490c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:10.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.778+0000 7f4910b6d640 1 -- 172.21.15.26:0/3493638949 shutdown_connections 2023-12-19T17:58:10.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.778+0000 7f4910b6d640 1 -- 172.21.15.26:0/3493638949 wait complete. 2023-12-19T17:58:10.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f4910b6d640 1 Processor -- start 2023-12-19T17:58:10.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f4910b6d640 1 -- start start 2023-12-19T17:58:10.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f4910b6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f4003b40 0x7f490c1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f4910b6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c1f56c0 0x7f490c155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f4910b6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c156320 0x7f490c0af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f4910b6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c1f6bf0 con 0x7f490c156320 2023-12-19T17:58:10.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f4910b6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c1f6760 con 0x7f48f4003b40 2023-12-19T17:58:10.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f4910b6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c1f5b50 con 0x7f490c1f56c0 2023-12-19T17:58:10.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f490ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c156320 0x7f490c0af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f490ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c156320 0x7f490c0af020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51136/0 (socket says 172.21.15.26:51136) 2023-12-19T17:58:10.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f490ad9d640 1 -- 172.21.15.26:0/3537806836 learned_addr learned my addr 172.21.15.26:0/3537806836 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:10.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f490a59c640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f4003b40 0x7f490c1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f4909d9b640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c1f56c0 0x7f490c155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f490ad9d640 1 -- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c1f56c0 msgr2=0x7f490c155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f490ad9d640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c1f56c0 0x7f490c155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f490ad9d640 1 -- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f4003b40 msgr2=0x7f490c1534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f490ad9d640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f4003b40 0x7f490c1534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f490ad9d640 1 -- 172.21.15.26:0/3537806836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c1f5e30 con 0x7f490c156320 2023-12-19T17:58:10.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f490ad9d640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c156320 0x7f490c0af020 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f490c06f1c0 tx=0x7f490400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:10.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f48faffd640 1 -- 172.21.15.26:0/3537806836 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904006d70 con 0x7f490c156320 2023-12-19T17:58:10.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f48faffd640 1 -- 172.21.15.26:0/3537806836 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4904003980 con 0x7f490c156320 2023-12-19T17:58:10.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f48faffd640 1 -- 172.21.15.26:0/3537806836 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4904003d30 con 0x7f490c156320 2023-12-19T17:58:10.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f4910b6d640 1 -- 172.21.15.26:0/3537806836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c1f6110 con 0x7f490c156320 2023-12-19T17:58:10.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f4910b6d640 1 -- 172.21.15.26:0/3537806836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c1539f0 con 0x7f490c156320 2023-12-19T17:58:10.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.782+0000 7f4910b6d640 1 -- 172.21.15.26:0/3537806836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c1f6910 con 0x7f490c156320 2023-12-19T17:58:10.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.786+0000 7f48faffd640 1 -- 172.21.15.26:0/3537806836 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f49040050d0 con 0x7f490c156320 2023-12-19T17:58:10.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.786+0000 7f48faffd640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f48ec065b80 0x7f48ec068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:10.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.786+0000 7f48faffd640 1 -- 172.21.15.26:0/3537806836 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f49040825f0 con 0x7f490c156320 2023-12-19T17:58:10.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.786+0000 7f490a59c640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f48ec065b80 0x7f48ec068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:10.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.786+0000 7f490a59c640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f48ec065b80 0x7f48ec068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0aa000 tx=0x7f48fc001a00 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:10.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.790+0000 7f48faffd640 1 -- 172.21.15.26:0/3537806836 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f490c1f6910 con 0x7f490c156320 2023-12-19T17:58:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.950+0000 7f4910b6d640 1 -- 172.21.15.26:0/3537806836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f490c1f68d0 con 0x7f490c156320 2023-12-19T17:58:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.950+0000 7f48faffd640 1 -- 172.21.15.26:0/3537806836 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+13504 (secure 0 0 0) 0x7f4904050050 con 0x7f490c156320 2023-12-19T17:58:10.955 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:10.956 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":173,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:58:10.195229+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":161,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:58:10.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.954+0000 7f4910b6d640 1 -- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f48ec065b80 msgr2=0x7f48ec068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.954+0000 7f4910b6d640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f48ec065b80 0x7f48ec068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0aa000 tx=0x7f48fc001a00 comp rx=0 tx=0).stop 2023-12-19T17:58:10.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.954+0000 7f4910b6d640 1 -- 172.21.15.26:0/3537806836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c156320 msgr2=0x7f490c0af020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:10.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.954+0000 7f4910b6d640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c156320 0x7f490c0af020 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f490c06f1c0 tx=0x7f490400ff70 comp rx=0 tx=0).stop 2023-12-19T17:58:10.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.954+0000 7f4910b6d640 1 -- 172.21.15.26:0/3537806836 shutdown_connections 2023-12-19T17:58:10.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.954+0000 7f4910b6d640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f490c156320 0x7f490c0af020 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.954+0000 7f4910b6d640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f48ec065b80 0x7f48ec068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.954+0000 7f4910b6d640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f490c1f56c0 0x7f490c155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.954+0000 7f4910b6d640 1 --2- 172.21.15.26:0/3537806836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48f4003b40 0x7f490c1534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:10.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.954+0000 7f4910b6d640 1 -- 172.21.15.26:0/3537806836 >> 172.21.15.26:0/3537806836 conn(0x7f490c0658e0 msgr2=0x7f490c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:10.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.954+0000 7f4910b6d640 1 -- 172.21.15.26:0/3537806836 shutdown_connections 2023-12-19T17:58:10.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:10.954+0000 7f4910b6d640 1 -- 172.21.15.26:0/3537806836 wait complete. 2023-12-19T17:58:10.978 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T17:58:10.979 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T17:58:11.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.122+0000 7f6fd6e54640 1 Processor -- start 2023-12-19T17:58:11.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.122+0000 7f6fd6e54640 1 -- start start 2023-12-19T17:58:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd6e54640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fd015ab40 0x7f6fd006b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd6e54640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6fd006f0a0 0x7f6fd006b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd6e54640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd015af20 con 0x7f6fd00afe70 2023-12-19T17:58:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd6e54640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd00aea90 con 0x7f6fd006f0a0 2023-12-19T17:58:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd6e54640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00aed70 con 0x7f6fd015ab40 2023-12-19T17:58:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd5651640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6fd006f0a0 0x7f6fd006b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd5651640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6fd006f0a0 0x7f6fd006b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51142/0 (socket says 172.21.15.26:51142) 2023-12-19T17:58:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd5651640 1 -- 172.21.15.26:0/2420156186 learned_addr learned my addr 172.21.15.26:0/2420156186 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd5651640 1 -- 172.21.15.26:0/2420156186 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fd015ab40 msgr2=0x7f6fd006b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd5e52640 1 --2- 172.21.15.26:0/2420156186 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fd015ab40 0x7f6fd006b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd5651640 1 --2- 172.21.15.26:0/2420156186 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fd015ab40 0x7f6fd006b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd5651640 1 -- 172.21.15.26:0/2420156186 >> v1:172.21.15.160:6789/0 conn(0x7f6fd00afe70 legacy=0x7f6fd015e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd5651640 1 -- 172.21.15.26:0/2420156186 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00af050 con 0x7f6fd006f0a0 2023-12-19T17:58:11.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.126+0000 7f6fd5e52640 1 --2- 172.21.15.26:0/2420156186 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fd015ab40 0x7f6fd006b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:58:11.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd5651640 1 --2- 172.21.15.26:0/2420156186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd006f0a0 0x7f6fd006b8f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009870 tx=0x7f6fc001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af37e50fc061e5e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:11.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd4e50640 1 -- 172.21.15.26:0/2420156186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00ac410 con 0x7f6fd006f0a0 2023-12-19T17:58:11.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd4e50640 1 -- 172.21.15.26:0/2420156186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc00ac570 con 0x7f6fd006f0a0 2023-12-19T17:58:11.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd4e50640 1 -- 172.21.15.26:0/2420156186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00ac900 con 0x7f6fd006f0a0 2023-12-19T17:58:11.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 -- 172.21.15.26:0/2420156186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd006f0a0 msgr2=0x7f6fd006b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 --2- 172.21.15.26:0/2420156186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd006f0a0 0x7f6fd006b8f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0009870 tx=0x7f6fc001c500 comp rx=0 tx=0).stop 2023-12-19T17:58:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 -- 172.21.15.26:0/2420156186 shutdown_connections 2023-12-19T17:58:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 --2- 172.21.15.26:0/2420156186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd006f0a0 0x7f6fd006b8f0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 --2- 172.21.15.26:0/2420156186 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fd015ab40 0x7f6fd006b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 -- 172.21.15.26:0/2420156186 >> 172.21.15.26:0/2420156186 conn(0x7f6fd00658e0 msgr2=0x7f6fd0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 -- 172.21.15.26:0/2420156186 shutdown_connections 2023-12-19T17:58:11.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 -- 172.21.15.26:0/2420156186 wait complete. 2023-12-19T17:58:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 Processor -- start 2023-12-19T17:58:11.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 -- start start 2023-12-19T17:58:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fd006f0a0 0x7f6fd0146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd00afe70 0x7f6fd00ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fd015ab40 0x7f6fd00cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00cdbd0 con 0x7f6fd00afe70 2023-12-19T17:58:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd0146640 con 0x7f6fd006f0a0 2023-12-19T17:58:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.130+0000 7f6fd6e54640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00ccfc0 con 0x7f6fd015ab40 2023-12-19T17:58:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd5651640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd00afe70 0x7f6fd00ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd5651640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd00afe70 0x7f6fd00ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51156/0 (socket says 172.21.15.26:51156) 2023-12-19T17:58:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd5651640 1 -- 172.21.15.26:0/3418410244 learned_addr learned my addr 172.21.15.26:0/3418410244 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd6653640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fd015ab40 0x7f6fd00cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd5e52640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fd006f0a0 0x7f6fd0146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd5651640 1 -- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fd015ab40 msgr2=0x7f6fd00cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd5651640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fd015ab40 0x7f6fd00cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd5651640 1 -- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fd006f0a0 msgr2=0x7f6fd0146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd5651640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fd006f0a0 0x7f6fd0146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd5651640 1 -- 172.21.15.26:0/3418410244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc00094d0 con 0x7f6fd00afe70 2023-12-19T17:58:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd5651640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd00afe70 0x7f6fd00ce060 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00af040 tx=0x7f6fc00192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fc67fc640 1 -- 172.21.15.26:0/3418410244 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00bb070 con 0x7f6fd00afe70 2023-12-19T17:58:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fc67fc640 1 -- 172.21.15.26:0/3418410244 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc00b65f0 con 0x7f6fd00afe70 2023-12-19T17:58:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fc67fc640 1 -- 172.21.15.26:0/3418410244 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00ac410 con 0x7f6fd00afe70 2023-12-19T17:58:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd6e54640 1 -- 172.21.15.26:0/3418410244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd00cd2a0 con 0x7f6fd00afe70 2023-12-19T17:58:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd6e54640 1 -- 172.21.15.26:0/3418410244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd00cd860 con 0x7f6fd00afe70 2023-12-19T17:58:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fd6e54640 1 -- 172.21.15.26:0/3418410244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd0061930 con 0x7f6fd00afe70 2023-12-19T17:58:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.134+0000 7f6fc67fc640 1 -- 172.21.15.26:0/3418410244 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6fc00b67c0 con 0x7f6fd00afe70 2023-12-19T17:58:11.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.138+0000 7f6fc67fc640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6fb4065b80 0x7f6fb4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.138+0000 7f6fc67fc640 1 -- 172.21.15.26:0/3418410244 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f6fc011d000 con 0x7f6fd00afe70 2023-12-19T17:58:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.138+0000 7f6fd5e52640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6fb4065b80 0x7f6fb4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.138+0000 7f6fd5e52640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6fb4065b80 0x7f6fb4068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6fd015a230 tx=0x7f6fcc008040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.142+0000 7f6fc67fc640 1 -- 172.21.15.26:0/3418410244 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6fc00eaa60 con 0x7f6fd00afe70 2023-12-19T17:58:11.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fd6e54640 1 -- 172.21.15.26:0/3418410244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6fd0164c00 con 0x7f6fd00afe70 2023-12-19T17:58:11.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fc67fc640 1 -- 172.21.15.26:0/3418410244 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v173) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6fc00ee080 con 0x7f6fd00afe70 2023-12-19T17:58:11.300 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T17:58:11.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fd6e54640 1 -- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6fb4065b80 msgr2=0x7f6fb4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fd6e54640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6fb4065b80 0x7f6fb4068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6fd015a230 tx=0x7f6fcc008040 comp rx=0 tx=0).stop 2023-12-19T17:58:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fd6e54640 1 -- 172.21.15.26:0/3418410244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd00afe70 msgr2=0x7f6fd00ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fd6e54640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd00afe70 0x7f6fd00ce060 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f6fc00af040 tx=0x7f6fc00192a0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fd6e54640 1 -- 172.21.15.26:0/3418410244 shutdown_connections 2023-12-19T17:58:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fd6e54640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6fb4065b80 0x7f6fb4068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fd6e54640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6fd015ab40 0x7f6fd00cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fd6e54640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fd00afe70 0x7f6fd00ce060 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fd6e54640 1 --2- 172.21.15.26:0/3418410244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6fd006f0a0 0x7f6fd0146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fd6e54640 1 -- 172.21.15.26:0/3418410244 >> 172.21.15.26:0/3418410244 conn(0x7f6fd00658e0 msgr2=0x7f6fd015e540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:11.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fd6e54640 1 -- 172.21.15.26:0/3418410244 shutdown_connections 2023-12-19T17:58:11.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.298+0000 7f6fd6e54640 1 -- 172.21.15.26:0/3418410244 wait complete. 2023-12-19T17:58:11.322 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T17:58:11.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.462+0000 7f96e99f0640 1 Processor -- start 2023-12-19T17:58:11.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.462+0000 7f96e99f0640 1 -- start start 2023-12-19T17:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e99f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96e415c7f0 0x7f96e406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e99f0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e406ced0 con 0x7f96e40b30d0 2023-12-19T17:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e99f0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e406d1b0 con 0x7f96e40adb80 2023-12-19T17:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e99f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e406d770 con 0x7f96e415c7f0 2023-12-19T17:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f96e40b30d0 0x7f96e40cb100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50174/0 (socket says 172.21.15.26:50174) 2023-12-19T17:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e37fe640 1 -- 172.21.15.26:0/4193167345 learned_addr learned my addr 172.21.15.26:0/4193167345 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:11.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e2ffd640 1 --2- 172.21.15.26:0/4193167345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96e415c7f0 0x7f96e406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323340869 0 0) 0x7f96e406ced0 con 0x7f96e40b30d0 2023-12-19T17:58:11.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d0003450 con 0x7f96e40b30d0 2023-12-19T17:58:11.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2697964614 0 0) 0x7f96d0003450 con 0x7f96e40b30d0 2023-12-19T17:58:11.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96e415c7f0 msgr2=0x7f96e406c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:11.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 --2- 172.21.15.26:0/4193167345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96e415c7f0 0x7f96e406c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 >> v1:172.21.15.160:6789/0 conn(0x7f96e40adb80 legacy=0x7f96e40cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e406d490 con 0x7f96e40b30d0 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f96dc001a00 con 0x7f96e40b30d0 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 >> v1:172.21.15.26:6789/0 conn(0x7f96e40b30d0 legacy=0x7f96e40cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 --2- 172.21.15.26:0/4193167345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96d0003aa0 0x7f96d0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 --2- 172.21.15.26:0/4193167345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d0008860 0x7f96d000ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 --2- 172.21.15.26:0/4193167345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d000b3e0 0x7f96d000d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e406d770 con 0x7f96d0008860 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d000e100 con 0x7f96d0003aa0 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d000e2c0 con 0x7f96d000b3e0 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e2ffd640 1 --2- 172.21.15.26:0/4193167345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96e415c7f0 0x7f96e406c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96dbfff640 1 --2- 172.21.15.26:0/4193167345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d0008860 0x7f96d000ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e37fe640 1 --2- 172.21.15.26:0/4193167345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d000b3e0 0x7f96d000d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96e2ffd640 1 --2- 172.21.15.26:0/4193167345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96d0003aa0 0x7f96d0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96dbfff640 1 -- 172.21.15.26:0/4193167345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d000b3e0 msgr2=0x7f96d000d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:11.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96dbfff640 1 --2- 172.21.15.26:0/4193167345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d000b3e0 0x7f96d000d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96dbfff640 1 -- 172.21.15.26:0/4193167345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96d0003aa0 msgr2=0x7f96d0003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:11.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96dbfff640 1 --2- 172.21.15.26:0/4193167345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96d0003aa0 0x7f96d0003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.466+0000 7f96dbfff640 1 -- 172.21.15.26:0/4193167345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d0003450 con 0x7f96d0008860 2023-12-19T17:58:11.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.470+0000 7f96dbfff640 1 --2- 172.21.15.26:0/4193167345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d0008860 0x7f96d000ac50 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f96cc001910 tx=0x7f96cc01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c436d118c4dc9ca server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:11.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.470+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0ab600 con 0x7f96d0008860 2023-12-19T17:58:11.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.470+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc0a7050 con 0x7f96d0008860 2023-12-19T17:58:11.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.470+0000 7f96e27fc640 1 -- 172.21.15.26:0/4193167345 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0ab9c0 con 0x7f96d0008860 2023-12-19T17:58:11.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.470+0000 7f96e99f0640 1 -- 172.21.15.26:0/4193167345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d0008860 msgr2=0x7f96d000ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.470+0000 7f96e99f0640 1 --2- 172.21.15.26:0/4193167345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96d0008860 0x7f96d000ac50 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f96cc001910 tx=0x7f96cc01c540 comp rx=0 tx=0).stop 2023-12-19T17:58:11.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.470+0000 7f96e37fe640 1 -- 172.21.15.26:0/4193167345 reap_dead start 2023-12-19T17:58:11.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.470+0000 7f96e99f0640 1 -- 172.21.15.26:0/4193167345 shutdown_connections 2023-12-19T17:58:11.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.470+0000 7f96e99f0640 1 -- 172.21.15.26:0/4193167345 >> 172.21.15.26:0/4193167345 conn(0x7f96e40658e0 msgr2=0x7f96e4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:11.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.470+0000 7f96e99f0640 1 -- 172.21.15.26:0/4193167345 shutdown_connections 2023-12-19T17:58:11.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.470+0000 7f96e99f0640 1 -- 172.21.15.26:0/4193167345 wait complete. 2023-12-19T17:58:11.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.470+0000 7f96e99f0640 1 Processor -- start 2023-12-19T17:58:11.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.470+0000 7f96e99f0640 1 -- start start 2023-12-19T17:58:11.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e99f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d0008860 0x7f96e414c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e99f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e414cf20 0x7f96e41e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e99f0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96e40ac850 0x7f96e40accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e99f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e40ad210 con 0x7f96e414cf20 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e99f0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e415aa70 con 0x7f96e40ac850 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e99f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e414ccb0 con 0x7f96d0008860 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e414cf20 0x7f96e41e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96dbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e414cf20 0x7f96e41e4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51172/0 (socket says 172.21.15.26:51172) 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96dbfff640 1 -- 172.21.15.26:0/3536329825 learned_addr learned my addr 172.21.15.26:0/3536329825 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e2ffd640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d0008860 0x7f96e414c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e37fe640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96e40ac850 0x7f96e40accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96dbfff640 1 -- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d0008860 msgr2=0x7f96e414c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96dbfff640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d0008860 0x7f96e414c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96dbfff640 1 -- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96e40ac850 msgr2=0x7f96e40accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96dbfff640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96e40ac850 0x7f96e40accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96dbfff640 1 -- 172.21.15.26:0/3536329825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e41f4ac0 con 0x7f96e414cf20 2023-12-19T17:58:11.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e37fe640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96e40ac850 0x7f96e40accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:58:11.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96dbfff640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e414cf20 0x7f96e41e4a30 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f96cc097e40 tx=0x7f96d0003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:11.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e0ff9640 1 -- 172.21.15.26:0/3536329825 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc005180 con 0x7f96e414cf20 2023-12-19T17:58:11.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e0ff9640 1 -- 172.21.15.26:0/3536329825 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc0052e0 con 0x7f96e414cf20 2023-12-19T17:58:11.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e0ff9640 1 -- 172.21.15.26:0/3536329825 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0ab9a0 con 0x7f96e414cf20 2023-12-19T17:58:11.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e99f0640 1 -- 172.21.15.26:0/3536329825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e41e5330 con 0x7f96e414cf20 2023-12-19T17:58:11.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e99f0640 1 -- 172.21.15.26:0/3536329825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e41e58f0 con 0x7f96e414cf20 2023-12-19T17:58:11.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.474+0000 7f96e99f0640 1 -- 172.21.15.26:0/3536329825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e4074270 con 0x7f96e414cf20 2023-12-19T17:58:11.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.478+0000 7f96e0ff9640 1 -- 172.21.15.26:0/3536329825 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f96cc0b2070 con 0x7f96e414cf20 2023-12-19T17:58:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.478+0000 7f96e0ff9640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f96c4065b80 0x7f96c4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.478+0000 7f96e0ff9640 1 -- 172.21.15.26:0/3536329825 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f96cc11cd90 con 0x7f96e414cf20 2023-12-19T17:58:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.478+0000 7f96e2ffd640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f96c4065b80 0x7f96c4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.478+0000 7f96e2ffd640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f96c4065b80 0x7f96c4068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f96e40670d0 tx=0x7f96d4008040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:11.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.482+0000 7f96e0ff9640 1 -- 172.21.15.26:0/3536329825 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f96cc0edf90 con 0x7f96e414cf20 2023-12-19T17:58:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e99f0640 1 -- 172.21.15.26:0/3536329825 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f96e4070fd0 con 0x7f96e414cf20 2023-12-19T17:58:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e0ff9640 1 -- 172.21.15.26:0/3536329825 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v173) v1 ==== 84+0+65 (secure 0 0 0) 0x7f96cc0edb20 con 0x7f96e414cf20 2023-12-19T17:58:11.644 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:11.645 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T17:58:11.645 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:11.645 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T17:58:11.645 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T17:58:11.645 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:11.645 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T17:58:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e99f0640 1 -- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f96c4065b80 msgr2=0x7f96c4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e99f0640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f96c4065b80 0x7f96c4068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f96e40670d0 tx=0x7f96d4008040 comp rx=0 tx=0).stop 2023-12-19T17:58:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e99f0640 1 -- 172.21.15.26:0/3536329825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e414cf20 msgr2=0x7f96e41e4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e99f0640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e414cf20 0x7f96e41e4a30 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f96cc097e40 tx=0x7f96d0003700 comp rx=0 tx=0).stop 2023-12-19T17:58:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e99f0640 1 -- 172.21.15.26:0/3536329825 shutdown_connections 2023-12-19T17:58:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e99f0640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f96e40ac850 0x7f96e40accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e99f0640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e414cf20 0x7f96e41e4a30 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e99f0640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f96c4065b80 0x7f96c4068040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e99f0640 1 --2- 172.21.15.26:0/3536329825 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f96d0008860 0x7f96e414c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e99f0640 1 -- 172.21.15.26:0/3536329825 >> 172.21.15.26:0/3536329825 conn(0x7f96e40658e0 msgr2=0x7f96e40cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e99f0640 1 -- 172.21.15.26:0/3536329825 shutdown_connections 2023-12-19T17:58:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.642+0000 7f96e99f0640 1 -- 172.21.15.26:0/3536329825 wait complete. 2023-12-19T17:58:11.666 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T17:58:11.666 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T17:58:11.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.810+0000 7f03daedf640 1 Processor -- start 2023-12-19T17:58:11.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.810+0000 7f03daedf640 1 -- start start 2023-12-19T17:58:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.814+0000 7f03daedf640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f03d4159830 0x7f03d415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.814+0000 7f03daedf640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03d40c9310 0x7f03d40cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.814+0000 7f03daedf640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d415e3d0 con 0x7f03d40cb110 2023-12-19T17:58:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.814+0000 7f03daedf640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d406f7c0 con 0x7f03d40c9310 2023-12-19T17:58:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.814+0000 7f03daedf640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d40ca2b0 con 0x7f03d4159830 2023-12-19T17:58:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.814+0000 7f03d3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03d40c9310 0x7f03d40cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.814+0000 7f03d947c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f03d40cb110 0x7f03d40701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50184/0 (socket says 172.21.15.26:50184) 2023-12-19T17:58:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.814+0000 7f03d947c640 1 -- 172.21.15.26:0/417194213 learned_addr learned my addr 172.21.15.26:0/417194213 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.814+0000 7f03d8c7b640 1 --2- 172.21.15.26:0/417194213 >> v2:172.21.15.160:3300/0 conn(0x7f03d4159830 0x7f03d415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.814+0000 7f03d3fff640 1 -- 172.21.15.26:0/417194213 >> v2:172.21.15.160:3300/0 conn(0x7f03d4159830 msgr2=0x7f03d415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:11.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.814+0000 7f03d3fff640 1 --2- 172.21.15.26:0/417194213 >> v2:172.21.15.160:3300/0 conn(0x7f03d4159830 0x7f03d415d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.814+0000 7f03d3fff640 1 -- 172.21.15.26:0/417194213 >> v1:172.21.15.26:6789/0 conn(0x7f03d40cb110 legacy=0x7f03d40701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.814+0000 7f03d3fff640 1 -- 172.21.15.26:0/417194213 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d415de10 con 0x7f03d40c9310 2023-12-19T17:58:11.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03d3fff640 1 --2- 172.21.15.26:0/417194213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03d40c9310 0x7f03d40cc430 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f03d406ce60 tx=0x7f03c001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44e2e5fa0546b586 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03d37fe640 1 -- 172.21.15.26:0/417194213 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00ab600 con 0x7f03d40c9310 2023-12-19T17:58:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03d37fe640 1 -- 172.21.15.26:0/417194213 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c00a7050 con 0x7f03d40c9310 2023-12-19T17:58:11.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03d37fe640 1 -- 172.21.15.26:0/417194213 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00ab980 con 0x7f03d40c9310 2023-12-19T17:58:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03daedf640 1 -- 172.21.15.26:0/417194213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03d40c9310 msgr2=0x7f03d40cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03daedf640 1 --2- 172.21.15.26:0/417194213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03d40c9310 0x7f03d40cc430 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f03d406ce60 tx=0x7f03c001c260 comp rx=0 tx=0).stop 2023-12-19T17:58:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03daedf640 1 -- 172.21.15.26:0/417194213 shutdown_connections 2023-12-19T17:58:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03daedf640 1 --2- 172.21.15.26:0/417194213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03d40c9310 0x7f03d40cc430 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03daedf640 1 --2- 172.21.15.26:0/417194213 >> v2:172.21.15.160:3300/0 conn(0x7f03d4159830 0x7f03d415d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03daedf640 1 -- 172.21.15.26:0/417194213 >> 172.21.15.26:0/417194213 conn(0x7f03d40b0270 msgr2=0x7f03d40b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03daedf640 1 -- 172.21.15.26:0/417194213 shutdown_connections 2023-12-19T17:58:11.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03daedf640 1 -- 172.21.15.26:0/417194213 wait complete. 2023-12-19T17:58:11.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03daedf640 1 Processor -- start 2023-12-19T17:58:11.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03daedf640 1 -- start start 2023-12-19T17:58:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03daedf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03d40c9310 0x7f03d4151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03daedf640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03d40cb110 0x7f03d41659a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.818+0000 7f03daedf640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03d4159830 0x7f03d40acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03daedf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41661f0 con 0x7f03d40c9310 2023-12-19T17:58:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03daedf640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41664d0 con 0x7f03d40cb110 2023-12-19T17:58:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03daedf640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d41667b0 con 0x7f03d4159830 2023-12-19T17:58:11.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d8c7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03d40c9310 0x7f03d4151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d8c7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03d40c9310 0x7f03d4151820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51200/0 (socket says 172.21.15.26:51200) 2023-12-19T17:58:11.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d8c7b640 1 -- 172.21.15.26:0/473644003 learned_addr learned my addr 172.21.15.26:0/473644003 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:11.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d3fff640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03d40cb110 0x7f03d41659a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d947c640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03d4159830 0x7f03d40acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d8c7b640 1 -- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03d4159830 msgr2=0x7f03d40acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d8c7b640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03d4159830 0x7f03d40acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d8c7b640 1 -- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03d40cb110 msgr2=0x7f03d41659a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:11.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d8c7b640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03d40cb110 0x7f03d41659a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d8c7b640 1 -- 172.21.15.26:0/473644003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c0009420 con 0x7f03d40c9310 2023-12-19T17:58:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d3fff640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03d40cb110 0x7f03d41659a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:58:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d947c640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03d4159830 0x7f03d40acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:58:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d8c7b640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03d40c9310 0x7f03d4151820 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f03c800d5f0 tx=0x7f03c800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d17fa640 1 -- 172.21.15.26:0/473644003 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c80125f0 con 0x7f03d40c9310 2023-12-19T17:58:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d17fa640 1 -- 172.21.15.26:0/473644003 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c8012750 con 0x7f03d40c9310 2023-12-19T17:58:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d17fa640 1 -- 172.21.15.26:0/473644003 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8012ae0 con 0x7f03d40c9310 2023-12-19T17:58:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03daedf640 1 -- 172.21.15.26:0/473644003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d4166a90 con 0x7f03d40c9310 2023-12-19T17:58:11.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03daedf640 1 -- 172.21.15.26:0/473644003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d4165ee0 con 0x7f03d40c9310 2023-12-19T17:58:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03daedf640 1 -- 172.21.15.26:0/473644003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d4074270 con 0x7f03d40c9310 2023-12-19T17:58:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d17fa640 1 -- 172.21.15.26:0/473644003 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f03c8012c40 con 0x7f03d40c9310 2023-12-19T17:58:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.822+0000 7f03d17fa640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f03b8065b80 0x7f03b8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.826+0000 7f03d17fa640 1 -- 172.21.15.26:0/473644003 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f03c8083800 con 0x7f03d40c9310 2023-12-19T17:58:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.826+0000 7f03d3fff640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f03b8065b80 0x7f03b8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.826+0000 7f03d3fff640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f03b8065b80 0x7f03b8068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f03c00ad000 tx=0x7f03c0008040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.830+0000 7f03d17fa640 1 -- 172.21.15.26:0/473644003 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f03c8051080 con 0x7f03d40c9310 2023-12-19T17:58:11.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:11.990+0000 7f03daedf640 1 -- 172.21.15.26:0/473644003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f03d415e0f0 con 0x7f03d40c9310 2023-12-19T17:58:12.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.054+0000 7f03d17fa640 1 -- 172.21.15.26:0/473644003 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v174) v1 ==== 138+0+0 (secure 0 0 0) 0x7f03c8007380 con 0x7f03d40c9310 2023-12-19T17:58:12.057 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T17:58:12.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.058+0000 7f03daedf640 1 -- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f03b8065b80 msgr2=0x7f03b8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:12.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.058+0000 7f03daedf640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f03b8065b80 0x7f03b8068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f03c00ad000 tx=0x7f03c0008040 comp rx=0 tx=0).stop 2023-12-19T17:58:12.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.058+0000 7f03daedf640 1 -- 172.21.15.26:0/473644003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03d40c9310 msgr2=0x7f03d4151820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:12.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.058+0000 7f03daedf640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03d40c9310 0x7f03d4151820 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f03c800d5f0 tx=0x7f03c800dab0 comp rx=0 tx=0).stop 2023-12-19T17:58:12.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.058+0000 7f03daedf640 1 -- 172.21.15.26:0/473644003 shutdown_connections 2023-12-19T17:58:12.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.058+0000 7f03daedf640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f03b8065b80 0x7f03b8068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:12.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.058+0000 7f03daedf640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03d4159830 0x7f03d40acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:12.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.058+0000 7f03daedf640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03d40cb110 0x7f03d41659a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:12.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.058+0000 7f03daedf640 1 --2- 172.21.15.26:0/473644003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03d40c9310 0x7f03d4151820 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:12.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.058+0000 7f03daedf640 1 -- 172.21.15.26:0/473644003 >> 172.21.15.26:0/473644003 conn(0x7f03d40b0270 msgr2=0x7f03d40ae1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:12.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.058+0000 7f03daedf640 1 -- 172.21.15.26:0/473644003 shutdown_connections 2023-12-19T17:58:12.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.058+0000 7f03daedf640 1 -- 172.21.15.26:0/473644003 wait complete. 2023-12-19T17:58:12.083 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T17:58:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.230+0000 7f8bd1666640 1 Processor -- start 2023-12-19T17:58:12.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.230+0000 7f8bd1666640 1 -- start start 2023-12-19T17:58:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.230+0000 7f8bd1666640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8bcc15d460 0x7f8bcc0cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.230+0000 7f8bd1666640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bcc0adaa0 con 0x7f8bcc15d080 2023-12-19T17:58:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.230+0000 7f8bd1666640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bcc06b5c0 con 0x7f8bcc06d350 2023-12-19T17:58:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.230+0000 7f8bd1666640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bcc0c92c0 con 0x7f8bcc15d460 2023-12-19T17:58:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.230+0000 7f8bcaffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8bcc15d080 0x7f8bcc0c9530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50192/0 (socket says 172.21.15.26:50192) 2023-12-19T17:58:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.230+0000 7f8bcaffd640 1 -- 172.21.15.26:0/3641744857 learned_addr learned my addr 172.21.15.26:0/3641744857 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.230+0000 7f8bcb7fe640 1 --2- 172.21.15.26:0/3641744857 >> v2:172.21.15.160:3300/0 conn(0x7f8bcc15d460 0x7f8bcc0cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.234+0000 7f8bca7fc640 1 -- 172.21.15.26:0/3641744857 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431551957 0 0) 0x7f8bcc0adaa0 con 0x7f8bcc15d080 2023-12-19T17:58:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.234+0000 7f8bca7fc640 1 -- 172.21.15.26:0/3641744857 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bb8003530 con 0x7f8bcc15d080 2023-12-19T17:58:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.234+0000 7f8bcb7fe640 1 -- 172.21.15.26:0/3641744857 >> v1:172.21.15.160:6789/0 conn(0x7f8bcc06d350 legacy=0x7f8bcc0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.234+0000 7f8bcb7fe640 1 -- 172.21.15.26:0/3641744857 >> v1:172.21.15.26:6789/0 conn(0x7f8bcc15d080 legacy=0x7f8bcc0c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.234+0000 7f8bcb7fe640 1 -- 172.21.15.26:0/3641744857 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc06b8a0 con 0x7f8bcc15d460 2023-12-19T17:58:12.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.234+0000 7f8bcb7fe640 1 --2- 172.21.15.26:0/3641744857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bcc15d460 0x7f8bcc0cd5e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009d90 tx=0x7f8bc401c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a070fe78330688b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.234+0000 7f8bca7fc640 1 -- 172.21.15.26:0/3641744857 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40a5910 con 0x7f8bcc15d460 2023-12-19T17:58:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.234+0000 7f8bca7fc640 1 -- 172.21.15.26:0/3641744857 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc40a5a70 con 0x7f8bcc15d460 2023-12-19T17:58:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.234+0000 7f8bca7fc640 1 -- 172.21.15.26:0/3641744857 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40a5de0 con 0x7f8bcc15d460 2023-12-19T17:58:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 -- 172.21.15.26:0/3641744857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bcc15d460 msgr2=0x7f8bcc0cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 --2- 172.21.15.26:0/3641744857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bcc15d460 0x7f8bcc0cd5e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009d90 tx=0x7f8bc401c830 comp rx=0 tx=0).stop 2023-12-19T17:58:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 -- 172.21.15.26:0/3641744857 shutdown_connections 2023-12-19T17:58:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 --2- 172.21.15.26:0/3641744857 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bcc15d460 0x7f8bcc0cd5e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 -- 172.21.15.26:0/3641744857 >> 172.21.15.26:0/3641744857 conn(0x7f8bcc0ce7f0 msgr2=0x7f8bcc0cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 -- 172.21.15.26:0/3641744857 shutdown_connections 2023-12-19T17:58:12.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 -- 172.21.15.26:0/3641744857 wait complete. 2023-12-19T17:58:12.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 Processor -- start 2023-12-19T17:58:12.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 -- start start 2023-12-19T17:58:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bcc06d350 0x7f8bcc0c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bcc15d080 0x7f8bcc15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bcc15d460 0x7f8bcc0c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc15aff0 con 0x7f8bcc15d460 2023-12-19T17:58:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc15b2d0 con 0x7f8bcc15d080 2023-12-19T17:58:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bd1666640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc15b5b0 con 0x7f8bcc06d350 2023-12-19T17:58:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bcb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bcc15d460 0x7f8bcc0c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bcb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bcc15d460 0x7f8bcc0c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51208/0 (socket says 172.21.15.26:51208) 2023-12-19T17:58:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bcb7fe640 1 -- 172.21.15.26:0/3934956029 learned_addr learned my addr 172.21.15.26:0/3934956029 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bcaffd640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bcc06d350 0x7f8bcc0c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bc3fff640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bcc15d080 0x7f8bcc15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bcb7fe640 1 -- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bcc06d350 msgr2=0x7f8bcc0c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bcb7fe640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bcc06d350 0x7f8bcc0c42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bcb7fe640 1 -- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bcc15d080 msgr2=0x7f8bcc15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bcb7fe640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bcc15d080 0x7f8bcc15a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.238+0000 7f8bcb7fe640 1 -- 172.21.15.26:0/3934956029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc4009a70 con 0x7f8bcc15d460 2023-12-19T17:58:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.242+0000 7f8bcb7fe640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bcc15d460 0x7f8bcc0c6ee0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4019440 tx=0x7f8bc40195d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:12.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.242+0000 7f8bc37fe640 1 -- 172.21.15.26:0/3934956029 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40ad850 con 0x7f8bcc15d460 2023-12-19T17:58:12.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.242+0000 7f8bc37fe640 1 -- 172.21.15.26:0/3934956029 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc40ad9b0 con 0x7f8bcc15d460 2023-12-19T17:58:12.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.242+0000 7f8bc37fe640 1 -- 172.21.15.26:0/3934956029 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40add60 con 0x7f8bcc15d460 2023-12-19T17:58:12.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.242+0000 7f8bd1666640 1 -- 172.21.15.26:0/3934956029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc0ae2d0 con 0x7f8bcc15d460 2023-12-19T17:58:12.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.242+0000 7f8bd1666640 1 -- 172.21.15.26:0/3934956029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc0c4810 con 0x7f8bcc15d460 2023-12-19T17:58:12.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.242+0000 7f8bd1666640 1 -- 172.21.15.26:0/3934956029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc04e850 con 0x7f8bcc15d460 2023-12-19T17:58:12.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.242+0000 7f8bc37fe640 1 -- 172.21.15.26:0/3934956029 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8bc40b4070 con 0x7f8bcc15d460 2023-12-19T17:58:12.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.242+0000 7f8bc37fe640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8bac065b80 0x7f8bac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:12.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.242+0000 7f8bc37fe640 1 -- 172.21.15.26:0/3934956029 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5959+0+0 (secure 0 0 0) 0x7f8bc411e100 con 0x7f8bcc15d460 2023-12-19T17:58:12.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.246+0000 7f8bcaffd640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8bac065b80 0x7f8bac068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:12.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.246+0000 7f8bcaffd640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8bac065b80 0x7f8bac068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc005890 tx=0x7f8bbc009040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:12.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.246+0000 7f8bc37fe640 1 -- 172.21.15.26:0/3934956029 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8bc40eba40 con 0x7f8bcc15d460 2023-12-19T17:58:12.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:12.406+0000 7f8bd1666640 1 -- 172.21.15.26:0/3934956029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8bcc0ae780 con 0x7f8bcc15d460 2023-12-19T17:58:13.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.066+0000 7f8bc37fe640 1 -- 172.21.15.26:0/3934956029 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v175) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8bc40ef060 con 0x7f8bcc15d460 2023-12-19T17:58:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.074+0000 7f8bd1666640 1 -- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8bac065b80 msgr2=0x7f8bac068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.074+0000 7f8bd1666640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8bac065b80 0x7f8bac068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc005890 tx=0x7f8bbc009040 comp rx=0 tx=0).stop 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.074+0000 7f8bd1666640 1 -- 172.21.15.26:0/3934956029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bcc15d460 msgr2=0x7f8bcc0c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.074+0000 7f8bd1666640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bcc15d460 0x7f8bcc0c6ee0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4019440 tx=0x7f8bc40195d0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.074+0000 7f8bd1666640 1 -- 172.21.15.26:0/3934956029 shutdown_connections 2023-12-19T17:58:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.074+0000 7f8bd1666640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bcc15d460 0x7f8bcc0c6ee0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.074+0000 7f8bd1666640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8bcc15d080 0x7f8bcc15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.074+0000 7f8bd1666640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8bac065b80 0x7f8bac068040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.074+0000 7f8bd1666640 1 --2- 172.21.15.26:0/3934956029 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8bcc06d350 0x7f8bcc0c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.074+0000 7f8bd1666640 1 -- 172.21.15.26:0/3934956029 >> 172.21.15.26:0/3934956029 conn(0x7f8bcc0ce7f0 msgr2=0x7f8bcc155d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:13.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.078+0000 7f8bd1666640 1 -- 172.21.15.26:0/3934956029 shutdown_connections 2023-12-19T17:58:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.078+0000 7f8bd1666640 1 -- 172.21.15.26:0/3934956029 wait complete. 2023-12-19T17:58:13.114 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T17:58:13.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.258+0000 7f49e9820640 1 Processor -- start 2023-12-19T17:58:13.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49e9820640 1 -- start start 2023-12-19T17:58:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49e9820640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49e415ada0 0x7f49e40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49e9820640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49e406ac80 0x7f49e40b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49e9820640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e4070d90 con 0x7f49e415a9c0 2023-12-19T17:58:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49e9820640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e40cc270 con 0x7f49e415ada0 2023-12-19T17:58:13.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49e9820640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e4071070 con 0x7f49e406ac80 2023-12-19T17:58:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49dbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49e415ada0 0x7f49e40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49dbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f49e415ada0 0x7f49e40706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51218/0 (socket says 172.21.15.26:51218) 2023-12-19T17:58:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49dbfff640 1 -- 172.21.15.26:0/4204737730 learned_addr learned my addr 172.21.15.26:0/4204737730 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49e37fe640 1 --2- 172.21.15.26:0/4204737730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49e406ac80 0x7f49e40b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49dbfff640 1 -- 172.21.15.26:0/4204737730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49e406ac80 msgr2=0x7f49e40b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49dbfff640 1 --2- 172.21.15.26:0/4204737730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49e406ac80 0x7f49e40b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49dbfff640 1 -- 172.21.15.26:0/4204737730 >> v1:172.21.15.160:6789/0 conn(0x7f49e415a9c0 legacy=0x7f49e40b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49dbfff640 1 -- 172.21.15.26:0/4204737730 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4071350 con 0x7f49e415ada0 2023-12-19T17:58:13.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.262+0000 7f49e37fe640 1 --2- 172.21.15.26:0/4204737730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49e406ac80 0x7f49e40b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:13.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49dbfff640 1 --2- 172.21.15.26:0/4204737730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e415ada0 0x7f49e40706d0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009870 tx=0x7f49cc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4730a01879b5d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:13.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49e27fc640 1 -- 172.21.15.26:0/4204737730 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0ad640 con 0x7f49e415ada0 2023-12-19T17:58:13.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49e27fc640 1 -- 172.21.15.26:0/4204737730 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc0ad7a0 con 0x7f49e415ada0 2023-12-19T17:58:13.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49e27fc640 1 -- 172.21.15.26:0/4204737730 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0adb30 con 0x7f49e415ada0 2023-12-19T17:58:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49e9820640 1 -- 172.21.15.26:0/4204737730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e415ada0 msgr2=0x7f49e40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49e9820640 1 --2- 172.21.15.26:0/4204737730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e415ada0 0x7f49e40706d0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009870 tx=0x7f49cc01c480 comp rx=0 tx=0).stop 2023-12-19T17:58:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49e9820640 1 -- 172.21.15.26:0/4204737730 shutdown_connections 2023-12-19T17:58:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49e9820640 1 --2- 172.21.15.26:0/4204737730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49e406ac80 0x7f49e40b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49e9820640 1 --2- 172.21.15.26:0/4204737730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e415ada0 0x7f49e40706d0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49e9820640 1 -- 172.21.15.26:0/4204737730 >> 172.21.15.26:0/4204737730 conn(0x7f49e40ce730 msgr2=0x7f49e40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49e9820640 1 -- 172.21.15.26:0/4204737730 shutdown_connections 2023-12-19T17:58:13.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49e9820640 1 -- 172.21.15.26:0/4204737730 wait complete. 2023-12-19T17:58:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49e9820640 1 Processor -- start 2023-12-19T17:58:13.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.266+0000 7f49e9820640 1 -- start start 2023-12-19T17:58:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e9820640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e406ac80 0x7f49e4150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e9820640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49e415a9c0 0x7f49e40c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e9820640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e415ada0 0x7f49e4159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e9820640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e4150ce0 con 0x7f49e406ac80 2023-12-19T17:58:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e9820640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e4150fc0 con 0x7f49e415ada0 2023-12-19T17:58:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e9820640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e41512a0 con 0x7f49e415a9c0 2023-12-19T17:58:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e406ac80 0x7f49e4150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e406ac80 0x7f49e4150700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51232/0 (socket says 172.21.15.26:51232) 2023-12-19T17:58:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e2ffd640 1 -- 172.21.15.26:0/2576639672 learned_addr learned my addr 172.21.15.26:0/2576639672 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:13.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e37fe640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e415ada0 0x7f49e4159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49dbfff640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49e415a9c0 0x7f49e40c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e2ffd640 1 -- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49e415a9c0 msgr2=0x7f49e40c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e2ffd640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49e415a9c0 0x7f49e40c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e2ffd640 1 -- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e415ada0 msgr2=0x7f49e4159560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e2ffd640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e415ada0 0x7f49e4159560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e2ffd640 1 -- 172.21.15.26:0/2576639672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0094d0 con 0x7f49e406ac80 2023-12-19T17:58:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e2ffd640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e406ac80 0x7f49e4150700 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f49d400c500 tx=0x7f49d400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e0ff9640 1 -- 172.21.15.26:0/2576639672 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4005220 con 0x7f49e406ac80 2023-12-19T17:58:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e0ff9640 1 -- 172.21.15.26:0/2576639672 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4005380 con 0x7f49e406ac80 2023-12-19T17:58:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e0ff9640 1 -- 172.21.15.26:0/2576639672 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d4007270 con 0x7f49e406ac80 2023-12-19T17:58:13.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e9820640 1 -- 172.21.15.26:0/2576639672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e4151580 con 0x7f49e406ac80 2023-12-19T17:58:13.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e9820640 1 -- 172.21.15.26:0/2576639672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e4151f00 con 0x7f49e406ac80 2023-12-19T17:58:13.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.270+0000 7f49e9820640 1 -- 172.21.15.26:0/2576639672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e4151aa0 con 0x7f49e406ac80 2023-12-19T17:58:13.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.274+0000 7f49e0ff9640 1 -- 172.21.15.26:0/2576639672 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f49d40073d0 con 0x7f49e406ac80 2023-12-19T17:58:13.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.274+0000 7f49e0ff9640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49c4065bd0 0x7f49c4068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:13.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.274+0000 7f49e0ff9640 1 -- 172.21.15.26:0/2576639672 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f49d4083890 con 0x7f49e406ac80 2023-12-19T17:58:13.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.274+0000 7f49dbfff640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49c4065bd0 0x7f49c4068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:13.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.274+0000 7f49dbfff640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49c4065bd0 0x7f49c4068090 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0af000 tx=0x7f49cc009040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:13.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.278+0000 7f49e0ff9640 1 -- 172.21.15.26:0/2576639672 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f49d40510e0 con 0x7f49e406ac80 2023-12-19T17:58:13.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.470+0000 7f49e9820640 1 -- 172.21.15.26:0/2576639672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f49e4151aa0 con 0x7f49e406ac80 2023-12-19T17:58:13.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.478+0000 7f49e0ff9640 1 -- 172.21.15.26:0/2576639672 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v77) v1 ==== 174+0+0 (secure 0 0 0) 0x7f49e4151aa0 con 0x7f49e406ac80 2023-12-19T17:58:13.481 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 20 and data pool 21 2023-12-19T17:58:13.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.482+0000 7f49e9820640 1 -- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49c4065bd0 msgr2=0x7f49c4068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.482+0000 7f49e9820640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49c4065bd0 0x7f49c4068090 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0af000 tx=0x7f49cc009040 comp rx=0 tx=0).stop 2023-12-19T17:58:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.482+0000 7f49e9820640 1 -- 172.21.15.26:0/2576639672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e406ac80 msgr2=0x7f49e4150700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.482+0000 7f49e9820640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e406ac80 0x7f49e4150700 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f49d400c500 tx=0x7f49d400c9c0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.486+0000 7f49e9820640 1 -- 172.21.15.26:0/2576639672 shutdown_connections 2023-12-19T17:58:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.486+0000 7f49e9820640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49e415ada0 0x7f49e4159560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.486+0000 7f49e9820640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f49c4065bd0 0x7f49c4068090 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.486+0000 7f49e9820640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49e415a9c0 0x7f49e40c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.486+0000 7f49e9820640 1 --2- 172.21.15.26:0/2576639672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49e406ac80 0x7f49e4150700 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.486+0000 7f49e9820640 1 -- 172.21.15.26:0/2576639672 >> 172.21.15.26:0/2576639672 conn(0x7f49e40ce730 msgr2=0x7f49e40afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.486+0000 7f49e9820640 1 -- 172.21.15.26:0/2576639672 shutdown_connections 2023-12-19T17:58:13.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.486+0000 7f49e9820640 1 -- 172.21.15.26:0/2576639672 wait complete. 2023-12-19T17:58:13.525 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:58:13.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152f5b3640 1 Processor -- start 2023-12-19T17:58:13.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152f5b3640 1 -- start start 2023-12-19T17:58:13.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152f5b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1528154220 0x7f152815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:13.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152f5b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f152806b3d0 0x7f15280ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:13.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152f5b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15280cec40 0x7f152815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:13.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152f5b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15280cf0d0 con 0x7f15280cec40 2023-12-19T17:58:13.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152f5b3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152806bc30 con 0x7f1528154220 2023-12-19T17:58:13.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152f5b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15280cc6e0 con 0x7f152806b3d0 2023-12-19T17:58:13.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152db50640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15280cec40 0x7f152815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:13.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152db50640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f15280cec40 0x7f152815d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51248/0 (socket says 172.21.15.26:51248) 2023-12-19T17:58:13.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152db50640 1 -- 172.21.15.26:0/1817714309 learned_addr learned my addr 172.21.15.26:0/1817714309 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:13.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152cb4e640 1 --2- 172.21.15.26:0/1817714309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f152806b3d0 0x7f15280ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:13.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152d34f640 1 --2- 172.21.15.26:0/1817714309 >> v2:172.21.15.160:3300/0 conn(0x7f1528154220 0x7f152815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:13.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152db50640 1 -- 172.21.15.26:0/1817714309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f152806b3d0 msgr2=0x7f15280ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:13.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152db50640 1 --2- 172.21.15.26:0/1817714309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f152806b3d0 0x7f15280ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152db50640 1 -- 172.21.15.26:0/1817714309 >> v2:172.21.15.160:3300/0 conn(0x7f1528154220 msgr2=0x7f152815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:13.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152db50640 1 --2- 172.21.15.26:0/1817714309 >> v2:172.21.15.160:3300/0 conn(0x7f1528154220 0x7f152815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152db50640 1 -- 172.21.15.26:0/1817714309 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152815afc0 con 0x7f15280cec40 2023-12-19T17:58:13.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.666+0000 7f152d34f640 1 --2- 172.21.15.26:0/1817714309 >> v2:172.21.15.160:3300/0 conn(0x7f1528154220 0x7f152815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.670+0000 7f152db50640 1 --2- 172.21.15.26:0/1817714309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15280cec40 0x7f152815d5f0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f15280667b0 tx=0x7f152001e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97468f3c55b3c3b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.670+0000 7f151ffff640 1 -- 172.21.15.26:0/1817714309 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152000b2d0 con 0x7f15280cec40 2023-12-19T17:58:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.670+0000 7f151ffff640 1 -- 172.21.15.26:0/1817714309 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152000b430 con 0x7f15280cec40 2023-12-19T17:58:13.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.670+0000 7f151ffff640 1 -- 172.21.15.26:0/1817714309 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15200a7990 con 0x7f15280cec40 2023-12-19T17:58:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.670+0000 7f152f5b3640 1 -- 172.21.15.26:0/1817714309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15280cec40 msgr2=0x7f152815d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.670+0000 7f152f5b3640 1 --2- 172.21.15.26:0/1817714309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15280cec40 0x7f152815d5f0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f15280667b0 tx=0x7f152001e8e0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.670+0000 7f152f5b3640 1 -- 172.21.15.26:0/1817714309 shutdown_connections 2023-12-19T17:58:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.670+0000 7f152f5b3640 1 --2- 172.21.15.26:0/1817714309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15280cec40 0x7f152815d5f0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.670+0000 7f152f5b3640 1 --2- 172.21.15.26:0/1817714309 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f152806b3d0 0x7f15280ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.670+0000 7f152f5b3640 1 --2- 172.21.15.26:0/1817714309 >> v2:172.21.15.160:3300/0 conn(0x7f1528154220 0x7f152815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.670+0000 7f152f5b3640 1 -- 172.21.15.26:0/1817714309 >> 172.21.15.26:0/1817714309 conn(0x7f15280658e0 msgr2=0x7f1528065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.670+0000 7f152f5b3640 1 -- 172.21.15.26:0/1817714309 shutdown_connections 2023-12-19T17:58:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.670+0000 7f152f5b3640 1 -- 172.21.15.26:0/1817714309 wait complete. 2023-12-19T17:58:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152f5b3640 1 Processor -- start 2023-12-19T17:58:13.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152f5b3640 1 -- start start 2023-12-19T17:58:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152f5b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152806b3d0 0x7f152814d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152f5b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15280cec40 0x7f15281398d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:13.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152f5b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1528154220 0x7f15281520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152f5b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152814ddd0 con 0x7f15280cec40 2023-12-19T17:58:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152f5b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152813c890 con 0x7f152806b3d0 2023-12-19T17:58:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152f5b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152813cb70 con 0x7f1528154220 2023-12-19T17:58:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152cb4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15280cec40 0x7f15281398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152cb4e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15280cec40 0x7f15281398d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51256/0 (socket says 172.21.15.26:51256) 2023-12-19T17:58:13.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152cb4e640 1 -- 172.21.15.26:0/1238744278 learned_addr learned my addr 172.21.15.26:0/1238744278 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152d34f640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152806b3d0 0x7f152814d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152db50640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1528154220 0x7f15281520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152cb4e640 1 -- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1528154220 msgr2=0x7f15281520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152cb4e640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1528154220 0x7f15281520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152cb4e640 1 -- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152806b3d0 msgr2=0x7f152814d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152cb4e640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152806b3d0 0x7f152814d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152cb4e640 1 -- 172.21.15.26:0/1238744278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152000bd40 con 0x7f15280cec40 2023-12-19T17:58:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152cb4e640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15280cec40 0x7f15281398d0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f151800e870 tx=0x7f151800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f151dffb640 1 -- 172.21.15.26:0/1238744278 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151800ee60 con 0x7f15280cec40 2023-12-19T17:58:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f151dffb640 1 -- 172.21.15.26:0/1238744278 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151800c940 con 0x7f15280cec40 2023-12-19T17:58:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f151dffb640 1 -- 172.21.15.26:0/1238744278 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151800ccf0 con 0x7f15280cec40 2023-12-19T17:58:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152f5b3640 1 -- 172.21.15.26:0/1238744278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152813ce50 con 0x7f15280cec40 2023-12-19T17:58:13.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.674+0000 7f152f5b3640 1 -- 172.21.15.26:0/1238744278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152814c5d0 con 0x7f15280cec40 2023-12-19T17:58:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.678+0000 7f152f5b3640 1 -- 172.21.15.26:0/1238744278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1528074270 con 0x7f15280cec40 2023-12-19T17:58:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.678+0000 7f151dffb640 1 -- 172.21.15.26:0/1238744278 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1518013470 con 0x7f15280cec40 2023-12-19T17:58:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.678+0000 7f151dffb640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f150c065b80 0x7f150c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.678+0000 7f151dffb640 1 -- 172.21.15.26:0/1238744278 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f1518083550 con 0x7f15280cec40 2023-12-19T17:58:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.678+0000 7f152d34f640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f150c065b80 0x7f150c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:13.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.678+0000 7f152d34f640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f150c065b80 0x7f150c068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f15280b3be0 tx=0x7f1524008040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:13.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.682+0000 7f151dffb640 1 -- 172.21.15.26:0/1238744278 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1518050c60 con 0x7f15280cec40 2023-12-19T17:58:13.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f152f5b3640 1 -- 172.21.15.26:0/1238744278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1528150ce0 con 0x7f15280cec40 2023-12-19T17:58:13.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f151dffb640 1 -- 172.21.15.26:0/1238744278 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+17229 (secure 0 0 0) 0x7f1518054280 con 0x7f15280cec40 2023-12-19T17:58:13.853 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:13.853 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":176,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:58:13.473518+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:58:12.042476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T17:58:12.457937+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:58:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f152f5b3640 1 -- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f150c065b80 msgr2=0x7f150c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f152f5b3640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f150c065b80 0x7f150c068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f15280b3be0 tx=0x7f1524008040 comp rx=0 tx=0).stop 2023-12-19T17:58:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f152f5b3640 1 -- 172.21.15.26:0/1238744278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15280cec40 msgr2=0x7f15281398d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f152f5b3640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15280cec40 0x7f15281398d0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f151800e870 tx=0x7f151800ed30 comp rx=0 tx=0).stop 2023-12-19T17:58:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f152f5b3640 1 -- 172.21.15.26:0/1238744278 shutdown_connections 2023-12-19T17:58:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f152f5b3640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f150c065b80 0x7f150c068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f152f5b3640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1528154220 0x7f15281520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f152f5b3640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15280cec40 0x7f15281398d0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f152f5b3640 1 --2- 172.21.15.26:0/1238744278 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f152806b3d0 0x7f152814d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f152f5b3640 1 -- 172.21.15.26:0/1238744278 >> 172.21.15.26:0/1238744278 conn(0x7f15280658e0 msgr2=0x7f15280ac850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f152f5b3640 1 -- 172.21.15.26:0/1238744278 shutdown_connections 2023-12-19T17:58:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:13.850+0000 7f152f5b3640 1 -- 172.21.15.26:0/1238744278 wait complete. 2023-12-19T17:58:13.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:58:14.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.014+0000 7f5cc09ce640 1 Processor -- start 2023-12-19T17:58:14.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.014+0000 7f5cc09ce640 1 -- start start 2023-12-19T17:58:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cc09ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5cbc154220 0x7f5cbc15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cc09ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cbc0cee10 0x7f5cbc15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cc09ce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cbc06bc30 con 0x7f5cbc06b3d0 2023-12-19T17:58:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cc09ce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cbc0cc6e0 con 0x7f5cbc154220 2023-12-19T17:58:14.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cc09ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc15afc0 con 0x7f5cbc0cee10 2023-12-19T17:58:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cba59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5cbc154220 0x7f5cbc15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cba59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5cbc154220 0x7f5cbc15aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51262/0 (socket says 172.21.15.26:51262) 2023-12-19T17:58:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cba59c640 1 -- 172.21.15.26:0/3555824095 learned_addr learned my addr 172.21.15.26:0/3555824095 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cbad9d640 1 --2- 172.21.15.26:0/3555824095 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cbc0cee10 0x7f5cbc15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cba59c640 1 -- 172.21.15.26:0/3555824095 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cbc0cee10 msgr2=0x7f5cbc15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cba59c640 1 --2- 172.21.15.26:0/3555824095 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cbc0cee10 0x7f5cbc15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cba59c640 1 -- 172.21.15.26:0/3555824095 >> v1:172.21.15.160:6789/0 conn(0x7f5cbc06b3d0 legacy=0x7f5cbc0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:14.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cba59c640 1 -- 172.21.15.26:0/3555824095 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc06b950 con 0x7f5cbc154220 2023-12-19T17:58:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.018+0000 7f5cba59c640 1 --2- 172.21.15.26:0/3555824095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc154220 0x7f5cbc15aa80 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009870 tx=0x7f5cac01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d8a62f97ac09f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cb9d9b640 1 -- 172.21.15.26:0/3555824095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0aa670 con 0x7f5cbc154220 2023-12-19T17:58:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cb9d9b640 1 -- 172.21.15.26:0/3555824095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac0aa7d0 con 0x7f5cbc154220 2023-12-19T17:58:14.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cb9d9b640 1 -- 172.21.15.26:0/3555824095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0aab60 con 0x7f5cbc154220 2023-12-19T17:58:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 -- 172.21.15.26:0/3555824095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc154220 msgr2=0x7f5cbc15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 --2- 172.21.15.26:0/3555824095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc154220 0x7f5cbc15aa80 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009870 tx=0x7f5cac01c510 comp rx=0 tx=0).stop 2023-12-19T17:58:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 -- 172.21.15.26:0/3555824095 shutdown_connections 2023-12-19T17:58:14.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 --2- 172.21.15.26:0/3555824095 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cbc0cee10 0x7f5cbc15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 --2- 172.21.15.26:0/3555824095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc154220 0x7f5cbc15aa80 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 -- 172.21.15.26:0/3555824095 >> 172.21.15.26:0/3555824095 conn(0x7f5cbc0658e0 msgr2=0x7f5cbc065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 -- 172.21.15.26:0/3555824095 shutdown_connections 2023-12-19T17:58:14.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 -- 172.21.15.26:0/3555824095 wait complete. 2023-12-19T17:58:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 Processor -- start 2023-12-19T17:58:14.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 -- start start 2023-12-19T17:58:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5cbc06b3d0 0x7f5cbc0b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc0cee10 0x7f5cbc0b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cbc154220 0x7f5cbc1499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc0b29e0 con 0x7f5cbc0cee10 2023-12-19T17:58:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc0b1dd0 con 0x7f5cbc06b3d0 2023-12-19T17:58:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cc09ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc0b20b0 con 0x7f5cbc154220 2023-12-19T17:58:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cb3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc0cee10 0x7f5cbc0b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cbad9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cbc154220 0x7f5cbc1499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.022+0000 7f5cb3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc0cee10 0x7f5cbc0b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51274/0 (socket says 172.21.15.26:51274) 2023-12-19T17:58:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cb3fff640 1 -- 172.21.15.26:0/1416649649 learned_addr learned my addr 172.21.15.26:0/1416649649 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cba59c640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5cbc06b3d0 0x7f5cbc0b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:14.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cb3fff640 1 -- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cbc154220 msgr2=0x7f5cbc1499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cb3fff640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cbc154220 0x7f5cbc1499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cb3fff640 1 -- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5cbc06b3d0 msgr2=0x7f5cbc0b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cb3fff640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5cbc06b3d0 0x7f5cbc0b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cb3fff640 1 -- 172.21.15.26:0/1416649649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac0094d0 con 0x7f5cbc0cee10 2023-12-19T17:58:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cba59c640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5cbc06b3d0 0x7f5cbc0b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:58:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cb3fff640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc0cee10 0x7f5cbc0b1580 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c3c0 tx=0x7f5ca400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cb2ffd640 1 -- 172.21.15.26:0/1416649649 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40052d0 con 0x7f5cbc0cee10 2023-12-19T17:58:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cb2ffd640 1 -- 172.21.15.26:0/1416649649 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca4005430 con 0x7f5cbc0cee10 2023-12-19T17:58:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cc09ce640 1 -- 172.21.15.26:0/1416649649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cbc0b2390 con 0x7f5cbc0cee10 2023-12-19T17:58:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cb2ffd640 1 -- 172.21.15.26:0/1416649649 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4003840 con 0x7f5cbc0cee10 2023-12-19T17:58:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cc09ce640 1 -- 172.21.15.26:0/1416649649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc0b33b0 con 0x7f5cbc0cee10 2023-12-19T17:58:14.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cc09ce640 1 -- 172.21.15.26:0/1416649649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cbc0b2b50 con 0x7f5cbc0cee10 2023-12-19T17:58:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cb2ffd640 1 -- 172.21.15.26:0/1416649649 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5ca40039a0 con 0x7f5cbc0cee10 2023-12-19T17:58:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cb2ffd640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5c9c065b80 0x7f5c9c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.026+0000 7f5cb2ffd640 1 -- 172.21.15.26:0/1416649649 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f5ca4082b20 con 0x7f5cbc0cee10 2023-12-19T17:58:14.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.030+0000 7f5cba59c640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5c9c065b80 0x7f5c9c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:14.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.030+0000 7f5cba59c640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5c9c065b80 0x7f5c9c068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0ae000 tx=0x7f5cac0b5040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:14.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.034+0000 7f5cb2ffd640 1 -- 172.21.15.26:0/1416649649 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5ca4050100 con 0x7f5cbc0cee10 2023-12-19T17:58:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.190+0000 7f5cc09ce640 1 -- 172.21.15.26:0/1416649649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cbc074270 con 0x7f5cbc0cee10 2023-12-19T17:58:14.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.190+0000 7f5cb2ffd640 1 -- 172.21.15.26:0/1416649649 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+17229 (secure 0 0 0) 0x7f5ca4017090 con 0x7f5cbc0cee10 2023-12-19T17:58:14.195 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:14.195 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":176,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:58:13.473518+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:58:12.042476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T17:58:12.457937+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:58:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.194+0000 7f5cc09ce640 1 -- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5c9c065b80 msgr2=0x7f5c9c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.194+0000 7f5cc09ce640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5c9c065b80 0x7f5c9c068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0ae000 tx=0x7f5cac0b5040 comp rx=0 tx=0).stop 2023-12-19T17:58:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.194+0000 7f5cc09ce640 1 -- 172.21.15.26:0/1416649649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc0cee10 msgr2=0x7f5cbc0b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.194+0000 7f5cc09ce640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc0cee10 0x7f5cbc0b1580 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c3c0 tx=0x7f5ca400c880 comp rx=0 tx=0).stop 2023-12-19T17:58:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.194+0000 7f5cc09ce640 1 -- 172.21.15.26:0/1416649649 shutdown_connections 2023-12-19T17:58:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.194+0000 7f5cc09ce640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f5c9c065b80 0x7f5c9c068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.194+0000 7f5cc09ce640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5cbc154220 0x7f5cbc1499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.194+0000 7f5cc09ce640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cbc0cee10 0x7f5cbc0b1580 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.194+0000 7f5cc09ce640 1 --2- 172.21.15.26:0/1416649649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5cbc06b3d0 0x7f5cbc0b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.194+0000 7f5cc09ce640 1 -- 172.21.15.26:0/1416649649 >> 172.21.15.26:0/1416649649 conn(0x7f5cbc0658e0 msgr2=0x7f5cbc0ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.194+0000 7f5cc09ce640 1 -- 172.21.15.26:0/1416649649 shutdown_connections 2023-12-19T17:58:14.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.194+0000 7f5cc09ce640 1 -- 172.21.15.26:0/1416649649 wait complete. 2023-12-19T17:58:14.217 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T17:58:14.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a9c66640 1 Processor -- start 2023-12-19T17:58:14.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a9c66640 1 -- start start 2023-12-19T17:58:14.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a9c66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0a4159820 0x7fe0a415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:14.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a9c66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe0a406f6c0 0x7fe0a40c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:14.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a9c66640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a406fb60 con 0x7fe0a4159c00 2023-12-19T17:58:14.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a9c66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a415e470 con 0x7fe0a406f6c0 2023-12-19T17:58:14.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a9c66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a406fe40 con 0x7fe0a4159820 2023-12-19T17:58:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0a4159820 0x7fe0a415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a2ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe0a406f6c0 0x7fe0a40c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a3fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe0a4159c00 0x7fe0a4070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:54324/0 (socket says 172.21.15.26:54324) 2023-12-19T17:58:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a3fff640 1 -- 172.21.15.26:0/3132137017 learned_addr learned my addr 172.21.15.26:0/3132137017 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a27fc640 1 -- 172.21.15.26:0/3132137017 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931382359 0 0) 0x7fe0a406fb60 con 0x7fe0a4159c00 2023-12-19T17:58:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a27fc640 1 -- 172.21.15.26:0/3132137017 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe08c003530 con 0x7fe0a4159c00 2023-12-19T17:58:14.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a37fe640 1 -- 172.21.15.26:0/3132137017 >> v2:172.21.15.160:3300/0 conn(0x7fe0a406f6c0 msgr2=0x7fe0a40c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:14.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a37fe640 1 --2- 172.21.15.26:0/3132137017 >> v2:172.21.15.160:3300/0 conn(0x7fe0a406f6c0 0x7fe0a40c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a37fe640 1 -- 172.21.15.26:0/3132137017 >> v1:172.21.15.160:6789/0 conn(0x7fe0a4159c00 legacy=0x7fe0a4070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:14.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a37fe640 1 -- 172.21.15.26:0/3132137017 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a4070120 con 0x7fe0a4159820 2023-12-19T17:58:14.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.362+0000 7fe0a2ffd640 1 --2- 172.21.15.26:0/3132137017 >> v2:172.21.15.160:3300/0 conn(0x7fe0a406f6c0 0x7fe0a40c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:58:14.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.366+0000 7fe0a37fe640 1 --2- 172.21.15.26:0/3132137017 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0a4159820 0x7fe0a415df30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe09400ba60 tx=0x7fe09401eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b007cb0edf788d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:14.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.366+0000 7fe0a27fc640 1 -- 172.21.15.26:0/3132137017 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940a7bf0 con 0x7fe0a4159820 2023-12-19T17:58:14.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.366+0000 7fe0a27fc640 1 -- 172.21.15.26:0/3132137017 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0940a7d50 con 0x7fe0a4159820 2023-12-19T17:58:14.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.366+0000 7fe0a27fc640 1 -- 172.21.15.26:0/3132137017 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940af660 con 0x7fe0a4159820 2023-12-19T17:58:14.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.366+0000 7fe0a9c66640 1 -- 172.21.15.26:0/3132137017 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0a4159820 msgr2=0x7fe0a415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:14.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.366+0000 7fe0a9c66640 1 --2- 172.21.15.26:0/3132137017 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0a4159820 0x7fe0a415df30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe09400ba60 tx=0x7fe09401eb20 comp rx=0 tx=0).stop 2023-12-19T17:58:14.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.366+0000 7fe0a9c66640 1 -- 172.21.15.26:0/3132137017 shutdown_connections 2023-12-19T17:58:14.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.366+0000 7fe0a9c66640 1 --2- 172.21.15.26:0/3132137017 >> v2:172.21.15.160:3300/0 conn(0x7fe0a406f6c0 0x7fe0a40c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.366+0000 7fe0a9c66640 1 --2- 172.21.15.26:0/3132137017 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0a4159820 0x7fe0a415df30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.366+0000 7fe0a9c66640 1 -- 172.21.15.26:0/3132137017 >> 172.21.15.26:0/3132137017 conn(0x7fe0a40ce730 msgr2=0x7fe0a40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:14.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.366+0000 7fe0a9c66640 1 -- 172.21.15.26:0/3132137017 shutdown_connections 2023-12-19T17:58:14.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.366+0000 7fe0a9c66640 1 -- 172.21.15.26:0/3132137017 wait complete. 2023-12-19T17:58:14.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a9c66640 1 Processor -- start 2023-12-19T17:58:14.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a9c66640 1 -- start start 2023-12-19T17:58:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a9c66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a406f6c0 0x7fe0a40c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:14.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a9c66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0a4159820 0x7fe0a4152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a9c66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0a4159c00 0x7fe0a40c0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a9c66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a40c2c10 con 0x7fe0a406f6c0 2023-12-19T17:58:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a9c66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a41556d0 con 0x7fe0a4159820 2023-12-19T17:58:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a9c66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a40c2270 con 0x7fe0a4159c00 2023-12-19T17:58:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a406f6c0 0x7fe0a40c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a406f6c0 0x7fe0a40c3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51286/0 (socket says 172.21.15.26:51286) 2023-12-19T17:58:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a37fe640 1 -- 172.21.15.26:0/1240422464 learned_addr learned my addr 172.21.15.26:0/1240422464 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a2ffd640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0a4159820 0x7fe0a4152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a3fff640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0a4159c00 0x7fe0a40c0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a37fe640 1 -- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0a4159c00 msgr2=0x7fe0a40c0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a37fe640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0a4159c00 0x7fe0a40c0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a37fe640 1 -- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0a4159820 msgr2=0x7fe0a4152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:14.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a37fe640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0a4159820 0x7fe0a4152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a37fe640 1 -- 172.21.15.26:0/1240422464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe098009420 con 0x7fe0a406f6c0 2023-12-19T17:58:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a3fff640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0a4159c00 0x7fe0a40c0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:14.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a37fe640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a406f6c0 0x7fe0a40c3030 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fe09401c750 tx=0x7fe09401b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe08bfff640 1 -- 172.21.15.26:0/1240422464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940b0c10 con 0x7fe0a406f6c0 2023-12-19T17:58:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe08bfff640 1 -- 172.21.15.26:0/1240422464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0940b0d70 con 0x7fe0a406f6c0 2023-12-19T17:58:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a9c66640 1 -- 172.21.15.26:0/1240422464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09400b6c0 con 0x7fe0a406f6c0 2023-12-19T17:58:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe08bfff640 1 -- 172.21.15.26:0/1240422464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940ba670 con 0x7fe0a406f6c0 2023-12-19T17:58:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a9c66640 1 -- 172.21.15.26:0/1240422464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a406b880 con 0x7fe0a406f6c0 2023-12-19T17:58:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.370+0000 7fe0a9c66640 1 -- 172.21.15.26:0/1240422464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a4074270 con 0x7fe0a406f6c0 2023-12-19T17:58:14.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.374+0000 7fe08bfff640 1 -- 172.21.15.26:0/1240422464 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe0940b6070 con 0x7fe0a406f6c0 2023-12-19T17:58:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.374+0000 7fe08bfff640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe090065b80 0x7fe090068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:14.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.374+0000 7fe08bfff640 1 -- 172.21.15.26:0/1240422464 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fe09411e5b0 con 0x7fe0a406f6c0 2023-12-19T17:58:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.374+0000 7fe0a2ffd640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe090065b80 0x7fe090068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:14.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.374+0000 7fe0a2ffd640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe090065b80 0x7fe090068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0a41f9140 tx=0x7fe098009190 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:14.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.378+0000 7fe08bfff640 1 -- 172.21.15.26:0/1240422464 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe0940ebca0 con 0x7fe0a406f6c0 2023-12-19T17:58:14.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:14.570+0000 7fe0a9c66640 1 -- 172.21.15.26:0/1240422464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe0a40b46b0 con 0x7fe0a406f6c0 2023-12-19T17:58:15.052 INFO:tasks.ceph.mds.a.smithi026.stderr:2023-12-19T17:58:15.050+0000 7f8eb2b9d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T17:58:15.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.482+0000 7fe08bfff640 1 -- 172.21.15.26:0/1240422464 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v79) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe0940ef2c0 con 0x7fe0a406f6c0 2023-12-19T17:58:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.486+0000 7fe0a9c66640 1 -- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe090065b80 msgr2=0x7fe090068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.486+0000 7fe0a9c66640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe090065b80 0x7fe090068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0a41f9140 tx=0x7fe098009190 comp rx=0 tx=0).stop 2023-12-19T17:58:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.486+0000 7fe0a9c66640 1 -- 172.21.15.26:0/1240422464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a406f6c0 msgr2=0x7fe0a40c3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.486+0000 7fe0a9c66640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a406f6c0 0x7fe0a40c3030 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fe09401c750 tx=0x7fe09401b8c0 comp rx=0 tx=0).stop 2023-12-19T17:58:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.486+0000 7fe0a9c66640 1 -- 172.21.15.26:0/1240422464 shutdown_connections 2023-12-19T17:58:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.486+0000 7fe0a9c66640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe090065b80 0x7fe090068040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.486+0000 7fe0a9c66640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe0a4159c00 0x7fe0a40c0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.486+0000 7fe0a9c66640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe0a4159820 0x7fe0a4152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.486+0000 7fe0a9c66640 1 --2- 172.21.15.26:0/1240422464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0a406f6c0 0x7fe0a40c3030 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.486+0000 7fe0a9c66640 1 -- 172.21.15.26:0/1240422464 >> 172.21.15.26:0/1240422464 conn(0x7fe0a40ce730 msgr2=0x7fe0a40af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.486+0000 7fe0a9c66640 1 -- 172.21.15.26:0/1240422464 shutdown_connections 2023-12-19T17:58:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.486+0000 7fe0a9c66640 1 -- 172.21.15.26:0/1240422464 wait complete. 2023-12-19T17:58:15.512 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T17:58:15.512 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T17:58:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.662+0000 7f6c1d28c640 1 Processor -- start 2023-12-19T17:58:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.662+0000 7f6c1d28c640 1 -- start start 2023-12-19T17:58:15.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.662+0000 7f6c1d28c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6c18159140 0x7f6c180ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:15.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.662+0000 7f6c1d28c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c180cd190 0x7f6c1806ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:15.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.662+0000 7f6c1d28c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1806f490 con 0x7f6c1806b1c0 2023-12-19T17:58:15.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.662+0000 7f6c1d28c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1806b650 con 0x7f6c18159140 2023-12-19T17:58:15.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.662+0000 7f6c1d28c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c18159520 con 0x7f6c180cd190 2023-12-19T17:58:15.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.662+0000 7f6c177fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6c1806b1c0 0x7f6c1815ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50204/0 (socket says 172.21.15.26:50204) 2023-12-19T17:58:15.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.662+0000 7f6c177fe640 1 -- 172.21.15.26:0/2171340692 learned_addr learned my addr 172.21.15.26:0/2171340692 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:15.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.662+0000 7f6c16ffd640 1 --2- 172.21.15.26:0/2171340692 >> v2:172.21.15.160:3300/0 conn(0x7f6c18159140 0x7f6c180ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:15.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.662+0000 7f6c0ffff640 1 --2- 172.21.15.26:0/2171340692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c180cd190 0x7f6c1806ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:15.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.662+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679802084 0 0) 0x7f6c1806f490 con 0x7f6c1806b1c0 2023-12-19T17:58:15.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.662+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c04005740 con 0x7f6c1806b1c0 2023-12-19T17:58:15.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1406610845 0 0) 0x7f6c04005740 con 0x7f6c1806b1c0 2023-12-19T17:58:15.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c180cd190 msgr2=0x7f6c1806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:15.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 --2- 172.21.15.26:0/2171340692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c180cd190 0x7f6c1806ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:15.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 >> v2:172.21.15.160:3300/0 conn(0x7f6c18159140 msgr2=0x7f6c180ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:15.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 --2- 172.21.15.26:0/2171340692 >> v2:172.21.15.160:3300/0 conn(0x7f6c18159140 0x7f6c180ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:15.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c180cd690 con 0x7f6c1806b1c0 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f6c10001a00 con 0x7f6c1806b1c0 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c0ffff640 1 --2- 172.21.15.26:0/2171340692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c180cd190 0x7f6c1806ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c16ffd640 1 --2- 172.21.15.26:0/2171340692 >> v2:172.21.15.160:3300/0 conn(0x7f6c18159140 0x7f6c180ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 >> v1:172.21.15.26:6789/0 conn(0x7f6c1806b1c0 legacy=0x7f6c1815ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 --2- 172.21.15.26:0/2171340692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c180cd190 0x7f6c04005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 --2- 172.21.15.26:0/2171340692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c040060d0 0x7f6c040069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 --2- 172.21.15.26:0/2171340692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c04006f20 0x7f6c040073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1806b650 con 0x7f6c04006f20 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c18159520 con 0x7f6c040060d0 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0400dba0 con 0x7f6c180cd190 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c177fe640 1 --2- 172.21.15.26:0/2171340692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c04006f20 0x7f6c040073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c16ffd640 1 --2- 172.21.15.26:0/2171340692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c180cd190 0x7f6c04005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c0ffff640 1 --2- 172.21.15.26:0/2171340692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c040060d0 0x7f6c040069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c177fe640 1 -- 172.21.15.26:0/2171340692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c180cd190 msgr2=0x7f6c04005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:15.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c177fe640 1 --2- 172.21.15.26:0/2171340692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c180cd190 0x7f6c04005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c177fe640 1 -- 172.21.15.26:0/2171340692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c040060d0 msgr2=0x7f6c040069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c177fe640 1 --2- 172.21.15.26:0/2171340692 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c040060d0 0x7f6c040069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c177fe640 1 -- 172.21.15.26:0/2171340692 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c00009420 con 0x7f6c04006f20 2023-12-19T17:58:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.666+0000 7f6c177fe640 1 --2- 172.21.15.26:0/2171340692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c04006f20 0x7f6c040073a0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000a830 tx=0x7f6c1001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4a01682a1eac85 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c100ad630 con 0x7f6c04006f20 2023-12-19T17:58:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c100a8070 con 0x7f6c04006f20 2023-12-19T17:58:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c167fc640 1 -- 172.21.15.26:0/2171340692 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c100ada30 con 0x7f6c04006f20 2023-12-19T17:58:15.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 -- 172.21.15.26:0/2171340692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c04006f20 msgr2=0x7f6c040073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:15.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 --2- 172.21.15.26:0/2171340692 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c04006f20 0x7f6c040073a0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6c1000a830 tx=0x7f6c1001d360 comp rx=0 tx=0).stop 2023-12-19T17:58:15.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c177fe640 1 -- 172.21.15.26:0/2171340692 reap_dead start 2023-12-19T17:58:15.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 -- 172.21.15.26:0/2171340692 shutdown_connections 2023-12-19T17:58:15.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 -- 172.21.15.26:0/2171340692 >> 172.21.15.26:0/2171340692 conn(0x7f6c180ce730 msgr2=0x7f6c180ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:15.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 -- 172.21.15.26:0/2171340692 shutdown_connections 2023-12-19T17:58:15.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 -- 172.21.15.26:0/2171340692 wait complete. 2023-12-19T17:58:15.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 Processor -- start 2023-12-19T17:58:15.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 -- start start 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04006f20 0x7f6c1814b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c180c3c90 0x7f6c180c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c1814c6c0 0x7f6c1814cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c180c2c80 con 0x7f6c180c3c90 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c180c2ed0 con 0x7f6c04006f20 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c1d28c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c180c31b0 con 0x7f6c1814c6c0 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c0ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c180c3c90 0x7f6c180c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c0ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c180c3c90 0x7f6c180c4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51304/0 (socket says 172.21.15.26:51304) 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c0ffff640 1 -- 172.21.15.26:0/1422999476 learned_addr learned my addr 172.21.15.26:0/1422999476 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c16ffd640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04006f20 0x7f6c1814b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.670+0000 7f6c177fe640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c1814c6c0 0x7f6c1814cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c0ffff640 1 -- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c1814c6c0 msgr2=0x7f6c1814cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c0ffff640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c1814c6c0 0x7f6c1814cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:15.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c0ffff640 1 -- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04006f20 msgr2=0x7f6c1814b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:15.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c0ffff640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04006f20 0x7f6c1814b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:15.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c0ffff640 1 -- 172.21.15.26:0/1422999476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c180c3490 con 0x7f6c180c3c90 2023-12-19T17:58:15.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c0ffff640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c180c3c90 0x7f6c180c4110 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f6c00011cd0 tx=0x7f6c0000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:15.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c0f7fe640 1 -- 172.21.15.26:0/1422999476 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0000fd20 con 0x7f6c180c3c90 2023-12-19T17:58:15.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c0f7fe640 1 -- 172.21.15.26:0/1422999476 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0000fe80 con 0x7f6c180c3c90 2023-12-19T17:58:15.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c1d28c640 1 -- 172.21.15.26:0/1422999476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c180c3770 con 0x7f6c180c3c90 2023-12-19T17:58:15.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c0f7fe640 1 -- 172.21.15.26:0/1422999476 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00004240 con 0x7f6c180c3c90 2023-12-19T17:58:15.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c1d28c640 1 -- 172.21.15.26:0/1422999476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c180c4f80 con 0x7f6c180c3c90 2023-12-19T17:58:15.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c1d28c640 1 -- 172.21.15.26:0/1422999476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c18074270 con 0x7f6c180c3c90 2023-12-19T17:58:15.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c0f7fe640 1 -- 172.21.15.26:0/1422999476 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6c000043a0 con 0x7f6c180c3c90 2023-12-19T17:58:15.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c0f7fe640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6bf8065bb0 0x7f6bf8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:15.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.674+0000 7f6c0f7fe640 1 -- 172.21.15.26:0/1422999476 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f6c00083ca0 con 0x7f6c180c3c90 2023-12-19T17:58:15.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.678+0000 7f6c16ffd640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6bf8065bb0 0x7f6bf8068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:15.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.678+0000 7f6c16ffd640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6bf8065bb0 0x7f6bf8068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6c08006e90 tx=0x7f6c08006de0 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:15.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.678+0000 7f6c0f7fe640 1 -- 172.21.15.26:0/1422999476 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6c00051310 con 0x7f6c180c3c90 2023-12-19T17:58:15.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:15.870+0000 7f6c1d28c640 1 -- 172.21.15.26:0/1422999476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6c1806b820 con 0x7f6c180c3c90 2023-12-19T17:58:16.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.490+0000 7f6c0f7fe640 1 -- 172.21.15.26:0/1422999476 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v80) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6c00022070 con 0x7f6c180c3c90 2023-12-19T17:58:16.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.490+0000 7f6c1d28c640 1 -- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6bf8065bb0 msgr2=0x7f6bf8068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:16.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.490+0000 7f6c1d28c640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6bf8065bb0 0x7f6bf8068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6c08006e90 tx=0x7f6c08006de0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.490+0000 7f6c1d28c640 1 -- 172.21.15.26:0/1422999476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c180c3c90 msgr2=0x7f6c180c4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:16.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.490+0000 7f6c1d28c640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c180c3c90 0x7f6c180c4110 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f6c00011cd0 tx=0x7f6c0000f610 comp rx=0 tx=0).stop 2023-12-19T17:58:16.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.490+0000 7f6c1d28c640 1 -- 172.21.15.26:0/1422999476 shutdown_connections 2023-12-19T17:58:16.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.490+0000 7f6c1d28c640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6bf8065bb0 0x7f6bf8068070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.490+0000 7f6c1d28c640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6c1814c6c0 0x7f6c1814cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.490+0000 7f6c1d28c640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c180c3c90 0x7f6c180c4110 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.490+0000 7f6c1d28c640 1 --2- 172.21.15.26:0/1422999476 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6c04006f20 0x7f6c1814b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.490+0000 7f6c1d28c640 1 -- 172.21.15.26:0/1422999476 >> 172.21.15.26:0/1422999476 conn(0x7f6c180ce730 msgr2=0x7f6c180ad820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:16.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.490+0000 7f6c1d28c640 1 -- 172.21.15.26:0/1422999476 shutdown_connections 2023-12-19T17:58:16.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.490+0000 7f6c1d28c640 1 -- 172.21.15.26:0/1422999476 wait complete. 2023-12-19T17:58:16.517 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T17:58:16.517 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:58:16.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.658+0000 7fee435ce640 1 Processor -- start 2023-12-19T17:58:16.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.658+0000 7fee435ce640 1 -- start start 2023-12-19T17:58:16.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.658+0000 7fee435ce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fee3c0cb1d0 0x7fee3c0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:16.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee435ce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3c06d4b0 con 0x7fee3c0b1d40 2023-12-19T17:58:16.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee435ce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3c0cbaf0 con 0x7fee3c06bee0 2023-12-19T17:58:16.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee435ce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3c0cc030 con 0x7fee3c0cb1d0 2023-12-19T17:58:16.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee41b6b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fee3c0b1d40 0x7fee3c0b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34416/0 (socket says 172.21.15.26:34416) 2023-12-19T17:58:16.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee41b6b640 1 -- 172.21.15.26:0/980501671 learned_addr learned my addr 172.21.15.26:0/980501671 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:16.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee4136a640 1 --2- 172.21.15.26:0/980501671 >> v2:172.21.15.160:3300/0 conn(0x7fee3c0cb1d0 0x7fee3c0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:16.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 478728453 0 0) 0x7fee3c06d4b0 con 0x7fee3c0b1d40 2023-12-19T17:58:16.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee28003530 con 0x7fee3c0b1d40 2023-12-19T17:58:16.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3577321700 0 0) 0x7fee28003530 con 0x7fee3c0b1d40 2023-12-19T17:58:16.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 >> v2:172.21.15.160:3300/0 conn(0x7fee3c0cb1d0 msgr2=0x7fee3c0cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:16.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 --2- 172.21.15.26:0/980501671 >> v2:172.21.15.160:3300/0 conn(0x7fee3c0cb1d0 0x7fee3c0cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 >> v1:172.21.15.160:6789/0 conn(0x7fee3c06bee0 legacy=0x7fee3c06c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:16.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c06cc10 con 0x7fee3c0b1d40 2023-12-19T17:58:16.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fee34001a00 con 0x7fee3c0b1d40 2023-12-19T17:58:16.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 >> v1:172.21.15.26:6789/0 conn(0x7fee3c0b1d40 legacy=0x7fee3c0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:16.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 --2- 172.21.15.26:0/980501671 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee28003980 0x7fee28003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:16.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 --2- 172.21.15.26:0/980501671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee280047d0 0x7fee28004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:16.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 --2- 172.21.15.26:0/980501671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28005190 0x7fee2800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:16.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0cc030 con 0x7fee28005190 2023-12-19T17:58:16.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2800c130 con 0x7fee280047d0 2023-12-19T17:58:16.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2800c2f0 con 0x7fee28003980 2023-12-19T17:58:16.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee41b6b640 1 --2- 172.21.15.26:0/980501671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28005190 0x7fee2800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee4136a640 1 --2- 172.21.15.26:0/980501671 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee28003980 0x7fee28003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee40b69640 1 --2- 172.21.15.26:0/980501671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee280047d0 0x7fee28004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee41b6b640 1 -- 172.21.15.26:0/980501671 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee28003980 msgr2=0x7fee28003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee41b6b640 1 --2- 172.21.15.26:0/980501671 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee28003980 0x7fee28003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee41b6b640 1 -- 172.21.15.26:0/980501671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee280047d0 msgr2=0x7fee28004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:16.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee41b6b640 1 --2- 172.21.15.26:0/980501671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee280047d0 0x7fee28004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee41b6b640 1 -- 172.21.15.26:0/980501671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee28003530 con 0x7fee28005190 2023-12-19T17:58:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.662+0000 7fee40b69640 1 --2- 172.21.15.26:0/980501671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee280047d0 0x7fee28004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee41b6b640 1 -- 172.21.15.26:0/980501671 reap_dead start 2023-12-19T17:58:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee41b6b640 1 --2- 172.21.15.26:0/980501671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28005190 0x7fee2800b990 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fee3400a830 tx=0x7fee3401faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05a5826b53bf0b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee340aa6d0 con 0x7fee28005190 2023-12-19T17:58:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee340aa870 con 0x7fee28005190 2023-12-19T17:58:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee33fff640 1 -- 172.21.15.26:0/980501671 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee340aac20 con 0x7fee28005190 2023-12-19T17:58:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee435ce640 1 -- 172.21.15.26:0/980501671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28005190 msgr2=0x7fee2800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee435ce640 1 --2- 172.21.15.26:0/980501671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28005190 0x7fee2800b990 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fee3400a830 tx=0x7fee3401faa0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee435ce640 1 -- 172.21.15.26:0/980501671 shutdown_connections 2023-12-19T17:58:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee435ce640 1 --2- 172.21.15.26:0/980501671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee28005190 0x7fee2800b990 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee435ce640 1 -- 172.21.15.26:0/980501671 >> 172.21.15.26:0/980501671 conn(0x7fee3c0658e0 msgr2=0x7fee3c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee435ce640 1 -- 172.21.15.26:0/980501671 shutdown_connections 2023-12-19T17:58:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee435ce640 1 -- 172.21.15.26:0/980501671 wait complete. 2023-12-19T17:58:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee435ce640 1 Processor -- start 2023-12-19T17:58:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.666+0000 7fee435ce640 1 -- start start 2023-12-19T17:58:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee435ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee28005190 0x7fee3c156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee435ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee3c0af2b0 0x7fee3c0af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee435ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee3c0cda40 0x7fee3c0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee435ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0afc70 con 0x7fee3c0cda40 2023-12-19T17:58:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee435ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0cf3d0 con 0x7fee28005190 2023-12-19T17:58:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee435ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3c0ce7c0 con 0x7fee3c0af2b0 2023-12-19T17:58:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee41b6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee3c0cda40 0x7fee3c0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee41b6b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee3c0cda40 0x7fee3c0cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38684/0 (socket says 172.21.15.26:38684) 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee41b6b640 1 -- 172.21.15.26:0/1170758860 learned_addr learned my addr 172.21.15.26:0/1170758860 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee40b69640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee3c0af2b0 0x7fee3c0af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee4136a640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee28005190 0x7fee3c156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee41b6b640 1 -- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee3c0af2b0 msgr2=0x7fee3c0af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee41b6b640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee3c0af2b0 0x7fee3c0af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee41b6b640 1 -- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee28005190 msgr2=0x7fee3c156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee41b6b640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee28005190 0x7fee3c156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee41b6b640 1 -- 172.21.15.26:0/1170758860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c0ceaa0 con 0x7fee3c0cda40 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee41b6b640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee3c0cda40 0x7fee3c0cdec0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fee34007040 tx=0x7fee28005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee31ffb640 1 -- 172.21.15.26:0/1170758860 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee340aa6d0 con 0x7fee3c0cda40 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee31ffb640 1 -- 172.21.15.26:0/1170758860 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee340017c0 con 0x7fee3c0cda40 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee31ffb640 1 -- 172.21.15.26:0/1170758860 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34004b70 con 0x7fee3c0cda40 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee435ce640 1 -- 172.21.15.26:0/1170758860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c0ced80 con 0x7fee3c0cda40 2023-12-19T17:58:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee435ce640 1 -- 172.21.15.26:0/1170758860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3c156da0 con 0x7fee3c0cda40 2023-12-19T17:58:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.670+0000 7fee435ce640 1 -- 172.21.15.26:0/1170758860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3c074270 con 0x7fee3c0cda40 2023-12-19T17:58:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.674+0000 7fee31ffb640 1 -- 172.21.15.26:0/1170758860 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fee34004cd0 con 0x7fee3c0cda40 2023-12-19T17:58:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.674+0000 7fee31ffb640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fee20065bb0 0x7fee20068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.674+0000 7fee31ffb640 1 -- 172.21.15.26:0/1170758860 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fee34120c10 con 0x7fee3c0cda40 2023-12-19T17:58:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.674+0000 7fee4136a640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fee20065bb0 0x7fee20068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.674+0000 7fee4136a640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fee20065bb0 0x7fee20068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fee38009b60 tx=0x7fee38009190 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.678+0000 7fee31ffb640 1 -- 172.21.15.26:0/1170758860 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fee340ee140 con 0x7fee3c0cda40 2023-12-19T17:58:16.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.866+0000 7fee435ce640 1 -- 172.21.15.26:0/1170758860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee3c0cf510 con 0x7fee3c0cda40 2023-12-19T17:58:16.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.866+0000 7fee31ffb640 1 -- 172.21.15.26:0/1170758860 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4713 (secure 0 0 0) 0x7fee340f1760 con 0x7fee3c0cda40 2023-12-19T17:58:16.871 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:16.871 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2924688599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2924688599},{"type":"v1","addr":"172.21.15.26:6835","nonce":2924688599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/330338341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":330338341},{"type":"v1","addr":"172.21.15.160:6836","nonce":330338341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9031,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/887954085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":887954085},{"type":"v1","addr":"172.21.15.160:6837","nonce":887954085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:58:13.473962+0000","modified":"2023-12-19T17:58:16.484662+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9513},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9513":{"gid":9513,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/3354708287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3354708287},{"type":"v1","addr":"172.21.15.26:6834","nonce":3354708287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-19T17:58:16.871 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 80 2023-12-19T17:58:16.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.870+0000 7fee435ce640 1 -- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fee20065bb0 msgr2=0x7fee20068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:16.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.870+0000 7fee435ce640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fee20065bb0 0x7fee20068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fee38009b60 tx=0x7fee38009190 comp rx=0 tx=0).stop 2023-12-19T17:58:16.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.870+0000 7fee435ce640 1 -- 172.21.15.26:0/1170758860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee3c0cda40 msgr2=0x7fee3c0cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:16.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.870+0000 7fee435ce640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee3c0cda40 0x7fee3c0cdec0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fee34007040 tx=0x7fee28005800 comp rx=0 tx=0).stop 2023-12-19T17:58:16.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.870+0000 7fee435ce640 1 -- 172.21.15.26:0/1170758860 shutdown_connections 2023-12-19T17:58:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.870+0000 7fee435ce640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee3c0cda40 0x7fee3c0cdec0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.870+0000 7fee435ce640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fee20065bb0 0x7fee20068070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.870+0000 7fee435ce640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee3c0af2b0 0x7fee3c0af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.870+0000 7fee435ce640 1 --2- 172.21.15.26:0/1170758860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee28005190 0x7fee3c156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.870+0000 7fee435ce640 1 -- 172.21.15.26:0/1170758860 >> 172.21.15.26:0/1170758860 conn(0x7fee3c0658e0 msgr2=0x7fee3c06ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.870+0000 7fee435ce640 1 -- 172.21.15.26:0/1170758860 shutdown_connections 2023-12-19T17:58:16.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:16.870+0000 7fee435ce640 1 -- 172.21.15.26:0/1170758860 wait complete. 2023-12-19T17:58:16.892 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T17:58:17.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53bdf98640 1 Processor -- start 2023-12-19T17:58:17.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53bdf98640 1 -- start start 2023-12-19T17:58:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53bdf98640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f53b80b85e0 0x7f53b80cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53bdf98640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f53b80cde60 0x7f53b80d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53bdf98640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b80d0d40 con 0x7f53b815cda0 2023-12-19T17:58:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53bdf98640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b80ce2d0 con 0x7f53b80cde60 2023-12-19T17:58:17.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53bdf98640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b8161150 con 0x7f53b80b85e0 2023-12-19T17:58:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53b7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f53b80cde60 0x7f53b80d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53b7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f53b80cde60 0x7f53b80d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38694/0 (socket says 172.21.15.26:38694) 2023-12-19T17:58:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53b7fff640 1 -- 172.21.15.26:0/2430809343 learned_addr learned my addr 172.21.15.26:0/2430809343 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53b6ffd640 1 --2- 172.21.15.26:0/2430809343 >> v2:172.21.15.160:3300/0 conn(0x7f53b80b85e0 0x7f53b80cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53b7fff640 1 -- 172.21.15.26:0/2430809343 >> v2:172.21.15.160:3300/0 conn(0x7f53b80b85e0 msgr2=0x7f53b80cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53b7fff640 1 --2- 172.21.15.26:0/2430809343 >> v2:172.21.15.160:3300/0 conn(0x7f53b80b85e0 0x7f53b80cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53b7fff640 1 -- 172.21.15.26:0/2430809343 >> v1:172.21.15.160:6789/0 conn(0x7f53b815cda0 legacy=0x7f53b8160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53b7fff640 1 -- 172.21.15.26:0/2430809343 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b8161430 con 0x7f53b80cde60 2023-12-19T17:58:17.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.038+0000 7f53b6ffd640 1 --2- 172.21.15.26:0/2430809343 >> v2:172.21.15.160:3300/0 conn(0x7f53b80b85e0 0x7f53b80cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:17.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.042+0000 7f53b7fff640 1 --2- 172.21.15.26:0/2430809343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b80cde60 0x7f53b80d1520 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f53b80b83a0 tx=0x7f53b001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c32d3be8931f2c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.042+0000 7f53b67fc640 1 -- 172.21.15.26:0/2430809343 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b00a5860 con 0x7f53b80cde60 2023-12-19T17:58:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.042+0000 7f53b67fc640 1 -- 172.21.15.26:0/2430809343 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b00a59c0 con 0x7f53b80cde60 2023-12-19T17:58:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.042+0000 7f53b67fc640 1 -- 172.21.15.26:0/2430809343 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b00a5d50 con 0x7f53b80cde60 2023-12-19T17:58:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.042+0000 7f53bdf98640 1 -- 172.21.15.26:0/2430809343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b80cde60 msgr2=0x7f53b80d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.042+0000 7f53bdf98640 1 --2- 172.21.15.26:0/2430809343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b80cde60 0x7f53b80d1520 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f53b80b83a0 tx=0x7f53b001c9a0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.042+0000 7f53bdf98640 1 -- 172.21.15.26:0/2430809343 shutdown_connections 2023-12-19T17:58:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.042+0000 7f53bdf98640 1 --2- 172.21.15.26:0/2430809343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b80cde60 0x7f53b80d1520 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.042+0000 7f53bdf98640 1 --2- 172.21.15.26:0/2430809343 >> v2:172.21.15.160:3300/0 conn(0x7f53b80b85e0 0x7f53b80cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.042+0000 7f53bdf98640 1 -- 172.21.15.26:0/2430809343 >> 172.21.15.26:0/2430809343 conn(0x7f53b80658e0 msgr2=0x7f53b8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.042+0000 7f53bdf98640 1 -- 172.21.15.26:0/2430809343 shutdown_connections 2023-12-19T17:58:17.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.042+0000 7f53bdf98640 1 -- 172.21.15.26:0/2430809343 wait complete. 2023-12-19T17:58:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53bdf98640 1 Processor -- start 2023-12-19T17:58:17.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53bdf98640 1 -- start start 2023-12-19T17:58:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53bdf98640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53b80b85e0 0x7f53b80cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53bdf98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b80cde60 0x7f53b80cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53bdf98640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53b815cda0 0x7f53b8151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53bdf98640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b814ca50 con 0x7f53b80cde60 2023-12-19T17:58:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53bdf98640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b814cbe0 con 0x7f53b80b85e0 2023-12-19T17:58:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53bdf98640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b814ef30 con 0x7f53b815cda0 2023-12-19T17:58:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53b6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b80cde60 0x7f53b80cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53b6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b80cde60 0x7f53b80cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38708/0 (socket says 172.21.15.26:38708) 2023-12-19T17:58:17.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53b6ffd640 1 -- 172.21.15.26:0/1123079801 learned_addr learned my addr 172.21.15.26:0/1123079801 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53b7fff640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53b815cda0 0x7f53b8151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53b77fe640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53b80b85e0 0x7f53b80cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53b6ffd640 1 -- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53b815cda0 msgr2=0x7f53b8151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53b6ffd640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53b815cda0 0x7f53b8151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53b6ffd640 1 -- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53b80b85e0 msgr2=0x7f53b80cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53b6ffd640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53b80b85e0 0x7f53b80cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53b6ffd640 1 -- 172.21.15.26:0/1123079801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b0009b30 con 0x7f53b80cde60 2023-12-19T17:58:17.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53b7fff640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53b815cda0 0x7f53b8151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:58:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53b77fe640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53b80b85e0 0x7f53b80cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:58:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53b6ffd640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b80cde60 0x7f53b80cbce0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00e760 tx=0x7f53ac00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f539ffff640 1 -- 172.21.15.26:0/1123079801 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac005ba0 con 0x7f53b80cde60 2023-12-19T17:58:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f539ffff640 1 -- 172.21.15.26:0/1123079801 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac005d00 con 0x7f53b80cde60 2023-12-19T17:58:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f539ffff640 1 -- 172.21.15.26:0/1123079801 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac004250 con 0x7f53b80cde60 2023-12-19T17:58:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53bdf98640 1 -- 172.21.15.26:0/1123079801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b81520b0 con 0x7f53b80cde60 2023-12-19T17:58:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.046+0000 7f53bdf98640 1 -- 172.21.15.26:0/1123079801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b8152670 con 0x7f53b80cde60 2023-12-19T17:58:17.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.050+0000 7f53bdf98640 1 -- 172.21.15.26:0/1123079801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b8074270 con 0x7f53b80cde60 2023-12-19T17:58:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.050+0000 7f539ffff640 1 -- 172.21.15.26:0/1123079801 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f53ac0043b0 con 0x7f53b80cde60 2023-12-19T17:58:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.050+0000 7f539ffff640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53a4065b80 0x7f53a4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.050+0000 7f539ffff640 1 -- 172.21.15.26:0/1123079801 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f53ac0835e0 con 0x7f53b80cde60 2023-12-19T17:58:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.050+0000 7f53b77fe640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53a4065b80 0x7f53a4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.050+0000 7f53b77fe640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53a4065b80 0x7f53a4068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53a8000f70 tx=0x7f53a8009040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:17.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.054+0000 7f539ffff640 1 -- 172.21.15.26:0/1123079801 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f53ac050b30 con 0x7f53b80cde60 2023-12-19T17:58:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.214+0000 7f53bdf98640 1 -- 172.21.15.26:0/1123079801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53b80d08f0 con 0x7f53b80cde60 2023-12-19T17:58:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.214+0000 7f539ffff640 1 -- 172.21.15.26:0/1123079801 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+17216 (secure 0 0 0) 0x7f53ac054150 con 0x7f53b80cde60 2023-12-19T17:58:17.218 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:17.218 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":177,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T17:58:14.476685+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:58:12.042476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T17:58:12.457937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T17:58:17.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.218+0000 7f53bdf98640 1 -- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53a4065b80 msgr2=0x7f53a4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.218+0000 7f53bdf98640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53a4065b80 0x7f53a4068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f53a8000f70 tx=0x7f53a8009040 comp rx=0 tx=0).stop 2023-12-19T17:58:17.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.218+0000 7f53bdf98640 1 -- 172.21.15.26:0/1123079801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b80cde60 msgr2=0x7f53b80cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.218+0000 7f53bdf98640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b80cde60 0x7f53b80cbce0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00e760 tx=0x7f53ac00ec20 comp rx=0 tx=0).stop 2023-12-19T17:58:17.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.218+0000 7f53bdf98640 1 -- 172.21.15.26:0/1123079801 shutdown_connections 2023-12-19T17:58:17.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.218+0000 7f53bdf98640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f53a4065b80 0x7f53a4068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.218+0000 7f53bdf98640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53b815cda0 0x7f53b8151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.218+0000 7f53bdf98640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53b80cde60 0x7f53b80cbce0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.218+0000 7f53bdf98640 1 --2- 172.21.15.26:0/1123079801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53b80b85e0 0x7f53b80cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.218+0000 7f53bdf98640 1 -- 172.21.15.26:0/1123079801 >> 172.21.15.26:0/1123079801 conn(0x7f53b80658e0 msgr2=0x7f53b80ccf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.218+0000 7f53bdf98640 1 -- 172.21.15.26:0/1123079801 shutdown_connections 2023-12-19T17:58:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.218+0000 7f53bdf98640 1 -- 172.21.15.26:0/1123079801 wait complete. 2023-12-19T17:58:17.240 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T17:58:17.240 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T17:58:17.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.378+0000 7efd785a5640 1 Processor -- start 2023-12-19T17:58:17.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.378+0000 7efd785a5640 1 -- start start 2023-12-19T17:58:17.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.382+0000 7efd785a5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd70155030 0x7efd7015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.382+0000 7efd785a5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7006c970 con 0x7efd7015f370 2023-12-19T17:58:17.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.382+0000 7efd785a5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7015b1f0 con 0x7efd700ce740 2023-12-19T17:58:17.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.382+0000 7efd785a5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7006c790 con 0x7efd70155030 2023-12-19T17:58:17.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.382+0000 7efd76341640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd70155030 0x7efd7015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.382+0000 7efd76341640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd70155030 0x7efd7015bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38710/0 (socket says 172.21.15.26:38710) 2023-12-19T17:58:17.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.382+0000 7efd76341640 1 -- 172.21.15.26:0/3193779525 learned_addr learned my addr 172.21.15.26:0/3193779525 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:17.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.382+0000 7efd76341640 1 -- 172.21.15.26:0/3193779525 >> v1:172.21.15.160:6789/0 conn(0x7efd700ce740 legacy=0x7efd700ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:17.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.382+0000 7efd76341640 1 -- 172.21.15.26:0/3193779525 >> v1:172.21.15.26:6789/0 conn(0x7efd7015f370 legacy=0x7efd7006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.382+0000 7efd76341640 1 -- 172.21.15.26:0/3193779525 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7015b4d0 con 0x7efd70155030 2023-12-19T17:58:17.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.382+0000 7efd7533f640 1 -- 172.21.15.26:0/3193779525 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593096576 0 0) 0x7efd7006c970 con 0x7efd7015f370 2023-12-19T17:58:17.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.382+0000 7efd76341640 1 --2- 172.21.15.26:0/3193779525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd70155030 0x7efd7015bcd0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7efd6c009870 tx=0x7efd6c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af41dde35d984f39 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:17.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd7533f640 1 -- 172.21.15.26:0/3193779525 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c0aa650 con 0x7efd70155030 2023-12-19T17:58:17.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd7533f640 1 -- 172.21.15.26:0/3193779525 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6c0aa7b0 con 0x7efd70155030 2023-12-19T17:58:17.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd7533f640 1 -- 172.21.15.26:0/3193779525 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c0aab40 con 0x7efd70155030 2023-12-19T17:58:17.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 -- 172.21.15.26:0/3193779525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd70155030 msgr2=0x7efd7015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 --2- 172.21.15.26:0/3193779525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd70155030 0x7efd7015bcd0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7efd6c009870 tx=0x7efd6c01c4d0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 -- 172.21.15.26:0/3193779525 shutdown_connections 2023-12-19T17:58:17.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 --2- 172.21.15.26:0/3193779525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd70155030 0x7efd7015bcd0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 -- 172.21.15.26:0/3193779525 >> 172.21.15.26:0/3193779525 conn(0x7efd700658e0 msgr2=0x7efd70065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:17.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 -- 172.21.15.26:0/3193779525 shutdown_connections 2023-12-19T17:58:17.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 -- 172.21.15.26:0/3193779525 wait complete. 2023-12-19T17:58:17.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 Processor -- start 2023-12-19T17:58:17.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 -- start start 2023-12-19T17:58:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd700ce740 0x7efd70135890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd70155030 0x7efd70153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd7015f370 0x7efd7014ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd70135dd0 con 0x7efd700ce740 2023-12-19T17:58:17.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd70154410 con 0x7efd7015f370 2023-12-19T17:58:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd785a5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7014a350 con 0x7efd70155030 2023-12-19T17:58:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd76341640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd700ce740 0x7efd70135890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd76341640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd700ce740 0x7efd70135890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38720/0 (socket says 172.21.15.26:38720) 2023-12-19T17:58:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.386+0000 7efd76341640 1 -- 172.21.15.26:0/4166679445 learned_addr learned my addr 172.21.15.26:0/4166679445 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.390+0000 7efd76341640 1 -- 172.21.15.26:0/4166679445 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd70155030 msgr2=0x7efd70153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.390+0000 7efd76341640 1 --2- 172.21.15.26:0/4166679445 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd70155030 0x7efd70153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.390+0000 7efd76341640 1 -- 172.21.15.26:0/4166679445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd7015f370 msgr2=0x7efd7014ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.390+0000 7efd76341640 1 --2- 172.21.15.26:0/4166679445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd7015f370 0x7efd7014ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.390+0000 7efd76341640 1 -- 172.21.15.26:0/4166679445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c0094d0 con 0x7efd700ce740 2023-12-19T17:58:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.390+0000 7efd76341640 1 --2- 172.21.15.26:0/4166679445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd700ce740 0x7efd70135890 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0ac040 tx=0x7efd6c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.390+0000 7efd66ffd640 1 -- 172.21.15.26:0/4166679445 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c0b3800 con 0x7efd700ce740 2023-12-19T17:58:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.390+0000 7efd66ffd640 1 -- 172.21.15.26:0/4166679445 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6c0b3960 con 0x7efd700ce740 2023-12-19T17:58:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.390+0000 7efd66ffd640 1 -- 172.21.15.26:0/4166679445 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c0b3d10 con 0x7efd700ce740 2023-12-19T17:58:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.390+0000 7efd785a5640 1 -- 172.21.15.26:0/4166679445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7014a630 con 0x7efd700ce740 2023-12-19T17:58:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.390+0000 7efd785a5640 1 -- 172.21.15.26:0/4166679445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd70154050 con 0x7efd700ce740 2023-12-19T17:58:17.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.390+0000 7efd785a5640 1 -- 172.21.15.26:0/4166679445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7014a1b0 con 0x7efd700ce740 2023-12-19T17:58:17.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.390+0000 7efd66ffd640 1 -- 172.21.15.26:0/4166679445 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7efd6c0b2380 con 0x7efd700ce740 2023-12-19T17:58:17.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.394+0000 7efd66ffd640 1 --2- 172.21.15.26:0/4166679445 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd54065b80 0x7efd54068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.394+0000 7efd66ffd640 1 -- 172.21.15.26:0/4166679445 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7efd6c11dff0 con 0x7efd700ce740 2023-12-19T17:58:17.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.394+0000 7efd75b40640 1 --2- 172.21.15.26:0/4166679445 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd54065b80 0x7efd54068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.394+0000 7efd75b40640 1 --2- 172.21.15.26:0/4166679445 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd54065b80 0x7efd54068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efd60007fd0 tx=0x7efd60007940 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:17.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.398+0000 7efd66ffd640 1 -- 172.21.15.26:0/4166679445 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efd7014a1b0 con 0x7efd700ce740 2023-12-19T17:58:17.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.558+0000 7efd785a5640 1 -- 172.21.15.26:0/4166679445 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd700cef80 con 0x7efd54065b80 2023-12-19T17:58:17.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.566+0000 7efd66ffd640 1 -- 172.21.15.26:0/4166679445 <== mgr.8821 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289968 (secure 0 0 0) 0x7efd600050d0 con 0x7efd54065b80 2023-12-19T17:58:17.574 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T17:58:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.570+0000 7efd785a5640 1 -- 172.21.15.26:0/4166679445 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd54065b80 msgr2=0x7efd54068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.570+0000 7efd785a5640 1 --2- 172.21.15.26:0/4166679445 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd54065b80 0x7efd54068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efd60007fd0 tx=0x7efd60007940 comp rx=0 tx=0).stop 2023-12-19T17:58:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.570+0000 7efd785a5640 1 -- 172.21.15.26:0/4166679445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd700ce740 msgr2=0x7efd70135890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.570+0000 7efd785a5640 1 --2- 172.21.15.26:0/4166679445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd700ce740 0x7efd70135890 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0ac040 tx=0x7efd6c01ad10 comp rx=0 tx=0).stop 2023-12-19T17:58:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.570+0000 7efd785a5640 1 -- 172.21.15.26:0/4166679445 shutdown_connections 2023-12-19T17:58:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.570+0000 7efd785a5640 1 --2- 172.21.15.26:0/4166679445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd7015f370 0x7efd7014ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.570+0000 7efd785a5640 1 --2- 172.21.15.26:0/4166679445 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd54065b80 0x7efd54068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.570+0000 7efd785a5640 1 --2- 172.21.15.26:0/4166679445 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd70155030 0x7efd70153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.570+0000 7efd785a5640 1 --2- 172.21.15.26:0/4166679445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd700ce740 0x7efd70135890 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.570+0000 7efd785a5640 1 -- 172.21.15.26:0/4166679445 >> 172.21.15.26:0/4166679445 conn(0x7efd700658e0 msgr2=0x7efd7014e760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.570+0000 7efd785a5640 1 -- 172.21.15.26:0/4166679445 shutdown_connections 2023-12-19T17:58:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.570+0000 7efd785a5640 1 -- 172.21.15.26:0/4166679445 wait complete. 2023-12-19T17:58:17.595 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-19T17:58:15.593305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1634,"num_read_kb":3381,"num_write":520,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":330,"ondisk_log_size":330,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":432400,"kb_used_data":18384,"kb_used_omap":0,"kb_used_meta":413952,"kb_avail":754542320,"statfs":{"total":773094113280,"available":772651335680,"internally_reserved":0,"allocated":18825216,"data_stored":10360420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":423886848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000715"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485017+0000","last_change":"2023-12-19T17:58:13.511775+0000","last_active":"2023-12-19T17:58:14.485017+0000","last_peered":"2023-12-19T17:58:14.485017+0000","last_clean":"2023-12-19T17:58:14.485017+0000","last_became_active":"2023-12-19T17:58:13.511470+0000","last_became_peered":"2023-12-19T17:58:13.511470+0000","last_unstale":"2023-12-19T17:58:14.485017+0000","last_undegraded":"2023-12-19T17:58:14.485017+0000","last_fullsized":"2023-12-19T17:58:14.485017+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:00.402828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914870+0000","last_change":"2023-12-19T17:58:13.500992+0000","last_active":"2023-12-19T17:58:14.914870+0000","last_peered":"2023-12-19T17:58:14.914870+0000","last_clean":"2023-12-19T17:58:14.914870+0000","last_became_active":"2023-12-19T17:58:13.500743+0000","last_became_peered":"2023-12-19T17:58:13.500743+0000","last_unstale":"2023-12-19T17:58:14.914870+0000","last_undegraded":"2023-12-19T17:58:14.914870+0000","last_fullsized":"2023-12-19T17:58:14.914870+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:59:51.315329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486866+0000","last_change":"2023-12-19T17:58:13.502221+0000","last_active":"2023-12-19T17:58:14.486866+0000","last_peered":"2023-12-19T17:58:14.486866+0000","last_clean":"2023-12-19T17:58:14.486866+0000","last_became_active":"2023-12-19T17:58:13.501953+0000","last_became_peered":"2023-12-19T17:58:13.501953+0000","last_unstale":"2023-12-19T17:58:14.486866+0000","last_undegraded":"2023-12-19T17:58:14.486866+0000","last_fullsized":"2023-12-19T17:58:14.486866+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:41:39.880180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915551+0000","last_change":"2023-12-19T17:58:13.499008+0000","last_active":"2023-12-19T17:58:14.915551+0000","last_peered":"2023-12-19T17:58:14.915551+0000","last_clean":"2023-12-19T17:58:14.915551+0000","last_became_active":"2023-12-19T17:58:13.498669+0000","last_became_peered":"2023-12-19T17:58:13.498669+0000","last_unstale":"2023-12-19T17:58:14.915551+0000","last_undegraded":"2023-12-19T17:58:14.915551+0000","last_fullsized":"2023-12-19T17:58:14.915551+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:36:08.870192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486004+0000","last_change":"2023-12-19T17:58:13.508329+0000","last_active":"2023-12-19T17:58:14.486004+0000","last_peered":"2023-12-19T17:58:14.486004+0000","last_clean":"2023-12-19T17:58:14.486004+0000","last_became_active":"2023-12-19T17:58:13.508047+0000","last_became_peered":"2023-12-19T17:58:13.508047+0000","last_unstale":"2023-12-19T17:58:14.486004+0000","last_undegraded":"2023-12-19T17:58:14.486004+0000","last_fullsized":"2023-12-19T17:58:14.486004+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:46.206669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484457+0000","last_change":"2023-12-19T17:58:13.499918+0000","last_active":"2023-12-19T17:58:14.484457+0000","last_peered":"2023-12-19T17:58:14.484457+0000","last_clean":"2023-12-19T17:58:14.484457+0000","last_became_active":"2023-12-19T17:58:13.499655+0000","last_became_peered":"2023-12-19T17:58:13.499655+0000","last_unstale":"2023-12-19T17:58:14.484457+0000","last_undegraded":"2023-12-19T17:58:14.484457+0000","last_fullsized":"2023-12-19T17:58:14.484457+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:49.808864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914785+0000","last_change":"2023-12-19T17:58:13.486291+0000","last_active":"2023-12-19T17:58:14.914785+0000","last_peered":"2023-12-19T17:58:14.914785+0000","last_clean":"2023-12-19T17:58:14.914785+0000","last_became_active":"2023-12-19T17:58:13.486025+0000","last_became_peered":"2023-12-19T17:58:13.486025+0000","last_unstale":"2023-12-19T17:58:14.914785+0000","last_undegraded":"2023-12-19T17:58:14.914785+0000","last_fullsized":"2023-12-19T17:58:14.914785+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:11:13.862544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484294+0000","last_change":"2023-12-19T17:58:13.491479+0000","last_active":"2023-12-19T17:58:14.484294+0000","last_peered":"2023-12-19T17:58:14.484294+0000","last_clean":"2023-12-19T17:58:14.484294+0000","last_became_active":"2023-12-19T17:58:13.491216+0000","last_became_peered":"2023-12-19T17:58:13.491216+0000","last_unstale":"2023-12-19T17:58:14.484294+0000","last_undegraded":"2023-12-19T17:58:14.484294+0000","last_fullsized":"2023-12-19T17:58:14.484294+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:16:31.307309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914936+0000","last_change":"2023-12-19T17:58:13.504084+0000","last_active":"2023-12-19T17:58:14.914936+0000","last_peered":"2023-12-19T17:58:14.914936+0000","last_clean":"2023-12-19T17:58:14.914936+0000","last_became_active":"2023-12-19T17:58:13.503836+0000","last_became_peered":"2023-12-19T17:58:13.503836+0000","last_unstale":"2023-12-19T17:58:14.914936+0000","last_undegraded":"2023-12-19T17:58:14.914936+0000","last_fullsized":"2023-12-19T17:58:14.914936+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:27:03.254793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914945+0000","last_change":"2023-12-19T17:58:13.507494+0000","last_active":"2023-12-19T17:58:14.914945+0000","last_peered":"2023-12-19T17:58:14.914945+0000","last_clean":"2023-12-19T17:58:14.914945+0000","last_became_active":"2023-12-19T17:58:13.507201+0000","last_became_peered":"2023-12-19T17:58:13.507201+0000","last_unstale":"2023-12-19T17:58:14.914945+0000","last_undegraded":"2023-12-19T17:58:14.914945+0000","last_fullsized":"2023-12-19T17:58:14.914945+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:17.870336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484601+0000","last_change":"2023-12-19T17:58:13.494898+0000","last_active":"2023-12-19T17:58:14.484601+0000","last_peered":"2023-12-19T17:58:14.484601+0000","last_clean":"2023-12-19T17:58:14.484601+0000","last_became_active":"2023-12-19T17:58:13.494614+0000","last_became_peered":"2023-12-19T17:58:13.494614+0000","last_unstale":"2023-12-19T17:58:14.484601+0000","last_undegraded":"2023-12-19T17:58:14.484601+0000","last_fullsized":"2023-12-19T17:58:14.484601+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:25.449341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914567+0000","last_change":"2023-12-19T17:58:13.510743+0000","last_active":"2023-12-19T17:58:14.914567+0000","last_peered":"2023-12-19T17:58:14.914567+0000","last_clean":"2023-12-19T17:58:14.914567+0000","last_became_active":"2023-12-19T17:58:13.510477+0000","last_became_peered":"2023-12-19T17:58:13.510477+0000","last_unstale":"2023-12-19T17:58:14.914567+0000","last_undegraded":"2023-12-19T17:58:14.914567+0000","last_fullsized":"2023-12-19T17:58:14.914567+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:14.799735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486237+0000","last_change":"2023-12-19T17:58:13.508677+0000","last_active":"2023-12-19T17:58:14.486237+0000","last_peered":"2023-12-19T17:58:14.486237+0000","last_clean":"2023-12-19T17:58:14.486237+0000","last_became_active":"2023-12-19T17:58:13.508394+0000","last_became_peered":"2023-12-19T17:58:13.508394+0000","last_unstale":"2023-12-19T17:58:14.486237+0000","last_undegraded":"2023-12-19T17:58:14.486237+0000","last_fullsized":"2023-12-19T17:58:14.486237+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:50:14.899624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914906+0000","last_change":"2023-12-19T17:58:13.499430+0000","last_active":"2023-12-19T17:58:14.914906+0000","last_peered":"2023-12-19T17:58:14.914906+0000","last_clean":"2023-12-19T17:58:14.914906+0000","last_became_active":"2023-12-19T17:58:13.499178+0000","last_became_peered":"2023-12-19T17:58:13.499178+0000","last_unstale":"2023-12-19T17:58:14.914906+0000","last_undegraded":"2023-12-19T17:58:14.914906+0000","last_fullsized":"2023-12-19T17:58:14.914906+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:34:16.126406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914871+0000","last_change":"2023-12-19T17:58:13.498555+0000","last_active":"2023-12-19T17:58:14.914871+0000","last_peered":"2023-12-19T17:58:14.914871+0000","last_clean":"2023-12-19T17:58:14.914871+0000","last_became_active":"2023-12-19T17:58:13.498269+0000","last_became_peered":"2023-12-19T17:58:13.498269+0000","last_unstale":"2023-12-19T17:58:14.914871+0000","last_undegraded":"2023-12-19T17:58:14.914871+0000","last_fullsized":"2023-12-19T17:58:14.914871+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:29.304411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914435+0000","last_change":"2023-12-19T17:58:13.504020+0000","last_active":"2023-12-19T17:58:14.914435+0000","last_peered":"2023-12-19T17:58:14.914435+0000","last_clean":"2023-12-19T17:58:14.914435+0000","last_became_active":"2023-12-19T17:58:13.503745+0000","last_became_peered":"2023-12-19T17:58:13.503745+0000","last_unstale":"2023-12-19T17:58:14.914435+0000","last_undegraded":"2023-12-19T17:58:14.914435+0000","last_fullsized":"2023-12-19T17:58:14.914435+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:50:14.899624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915285+0000","last_change":"2023-12-19T17:58:13.511251+0000","last_active":"2023-12-19T17:58:14.915285+0000","last_peered":"2023-12-19T17:58:14.915285+0000","last_clean":"2023-12-19T17:58:14.915285+0000","last_became_active":"2023-12-19T17:58:13.508979+0000","last_became_peered":"2023-12-19T17:58:13.508979+0000","last_unstale":"2023-12-19T17:58:14.915285+0000","last_undegraded":"2023-12-19T17:58:14.915285+0000","last_fullsized":"2023-12-19T17:58:14.915285+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:26:02.125211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486947+0000","last_change":"2023-12-19T17:58:13.502151+0000","last_active":"2023-12-19T17:58:14.486947+0000","last_peered":"2023-12-19T17:58:14.486947+0000","last_clean":"2023-12-19T17:58:14.486947+0000","last_became_active":"2023-12-19T17:58:13.501886+0000","last_became_peered":"2023-12-19T17:58:13.501886+0000","last_unstale":"2023-12-19T17:58:14.486947+0000","last_undegraded":"2023-12-19T17:58:14.486947+0000","last_fullsized":"2023-12-19T17:58:14.486947+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:34:38.046676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484716+0000","last_change":"2023-12-19T17:58:13.497314+0000","last_active":"2023-12-19T17:58:14.484716+0000","last_peered":"2023-12-19T17:58:14.484716+0000","last_clean":"2023-12-19T17:58:14.484716+0000","last_became_active":"2023-12-19T17:58:13.496934+0000","last_became_peered":"2023-12-19T17:58:13.496934+0000","last_unstale":"2023-12-19T17:58:14.484716+0000","last_undegraded":"2023-12-19T17:58:14.484716+0000","last_fullsized":"2023-12-19T17:58:14.484716+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:05:27.227183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914731+0000","last_change":"2023-12-19T17:58:13.494217+0000","last_active":"2023-12-19T17:58:14.914731+0000","last_peered":"2023-12-19T17:58:14.914731+0000","last_clean":"2023-12-19T17:58:14.914731+0000","last_became_active":"2023-12-19T17:58:13.493950+0000","last_became_peered":"2023-12-19T17:58:13.493950+0000","last_unstale":"2023-12-19T17:58:14.914731+0000","last_undegraded":"2023-12-19T17:58:14.914731+0000","last_fullsized":"2023-12-19T17:58:14.914731+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:35:13.994534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914893+0000","last_change":"2023-12-19T17:58:13.505180+0000","last_active":"2023-12-19T17:58:14.914893+0000","last_peered":"2023-12-19T17:58:14.914893+0000","last_clean":"2023-12-19T17:58:14.914893+0000","last_became_active":"2023-12-19T17:58:13.504854+0000","last_became_peered":"2023-12-19T17:58:13.504854+0000","last_unstale":"2023-12-19T17:58:14.914893+0000","last_undegraded":"2023-12-19T17:58:14.914893+0000","last_fullsized":"2023-12-19T17:58:14.914893+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:43:23.041550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914962+0000","last_change":"2023-12-19T17:58:13.493827+0000","last_active":"2023-12-19T17:58:14.914962+0000","last_peered":"2023-12-19T17:58:14.914962+0000","last_clean":"2023-12-19T17:58:14.914962+0000","last_became_active":"2023-12-19T17:58:13.493549+0000","last_became_peered":"2023-12-19T17:58:13.493549+0000","last_unstale":"2023-12-19T17:58:14.914962+0000","last_undegraded":"2023-12-19T17:58:14.914962+0000","last_fullsized":"2023-12-19T17:58:14.914962+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:34:38.046676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914741+0000","last_change":"2023-12-19T17:58:13.495208+0000","last_active":"2023-12-19T17:58:14.914741+0000","last_peered":"2023-12-19T17:58:14.914741+0000","last_clean":"2023-12-19T17:58:14.914741+0000","last_became_active":"2023-12-19T17:58:13.494771+0000","last_became_peered":"2023-12-19T17:58:13.494771+0000","last_unstale":"2023-12-19T17:58:14.914741+0000","last_undegraded":"2023-12-19T17:58:14.914741+0000","last_fullsized":"2023-12-19T17:58:14.914741+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:40:18.048253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914454+0000","last_change":"2023-12-19T17:58:13.504433+0000","last_active":"2023-12-19T17:58:14.914454+0000","last_peered":"2023-12-19T17:58:14.914454+0000","last_clean":"2023-12-19T17:58:14.914454+0000","last_became_active":"2023-12-19T17:58:13.504113+0000","last_became_peered":"2023-12-19T17:58:13.504113+0000","last_unstale":"2023-12-19T17:58:14.914454+0000","last_undegraded":"2023-12-19T17:58:14.914454+0000","last_fullsized":"2023-12-19T17:58:14.914454+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:40:29.577603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915277+0000","last_change":"2023-12-19T17:58:13.508323+0000","last_active":"2023-12-19T17:58:14.915277+0000","last_peered":"2023-12-19T17:58:14.915277+0000","last_clean":"2023-12-19T17:58:14.915277+0000","last_became_active":"2023-12-19T17:58:13.507584+0000","last_became_peered":"2023-12-19T17:58:13.507584+0000","last_unstale":"2023-12-19T17:58:14.915277+0000","last_undegraded":"2023-12-19T17:58:14.915277+0000","last_fullsized":"2023-12-19T17:58:14.915277+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:10:32.458962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484711+0000","last_change":"2023-12-19T17:58:13.497157+0000","last_active":"2023-12-19T17:58:14.484711+0000","last_peered":"2023-12-19T17:58:14.484711+0000","last_clean":"2023-12-19T17:58:14.484711+0000","last_became_active":"2023-12-19T17:58:13.496873+0000","last_became_peered":"2023-12-19T17:58:13.496873+0000","last_unstale":"2023-12-19T17:58:14.484711+0000","last_undegraded":"2023-12-19T17:58:14.484711+0000","last_fullsized":"2023-12-19T17:58:14.484711+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:14.799735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914642+0000","last_change":"2023-12-19T17:58:13.498578+0000","last_active":"2023-12-19T17:58:14.914642+0000","last_peered":"2023-12-19T17:58:14.914642+0000","last_clean":"2023-12-19T17:58:14.914642+0000","last_became_active":"2023-12-19T17:58:13.498322+0000","last_became_peered":"2023-12-19T17:58:13.498322+0000","last_unstale":"2023-12-19T17:58:14.914642+0000","last_undegraded":"2023-12-19T17:58:14.914642+0000","last_fullsized":"2023-12-19T17:58:14.914642+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:39.351617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915364+0000","last_change":"2023-12-19T17:58:13.501483+0000","last_active":"2023-12-19T17:58:14.915364+0000","last_peered":"2023-12-19T17:58:14.915364+0000","last_clean":"2023-12-19T17:58:14.915364+0000","last_became_active":"2023-12-19T17:58:13.499848+0000","last_became_peered":"2023-12-19T17:58:13.499848+0000","last_unstale":"2023-12-19T17:58:14.915364+0000","last_undegraded":"2023-12-19T17:58:14.915364+0000","last_fullsized":"2023-12-19T17:58:14.915364+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:29.059098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484486+0000","last_change":"2023-12-19T17:58:13.500787+0000","last_active":"2023-12-19T17:58:14.484486+0000","last_peered":"2023-12-19T17:58:14.484486+0000","last_clean":"2023-12-19T17:58:14.484486+0000","last_became_active":"2023-12-19T17:58:13.500302+0000","last_became_peered":"2023-12-19T17:58:13.500302+0000","last_unstale":"2023-12-19T17:58:14.484486+0000","last_undegraded":"2023-12-19T17:58:14.484486+0000","last_fullsized":"2023-12-19T17:58:14.484486+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:26:02.125211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915023+0000","last_change":"2023-12-19T17:58:13.496813+0000","last_active":"2023-12-19T17:58:14.915023+0000","last_peered":"2023-12-19T17:58:14.915023+0000","last_clean":"2023-12-19T17:58:14.915023+0000","last_became_active":"2023-12-19T17:58:13.495857+0000","last_became_peered":"2023-12-19T17:58:13.495857+0000","last_unstale":"2023-12-19T17:58:14.915023+0000","last_undegraded":"2023-12-19T17:58:14.915023+0000","last_fullsized":"2023-12-19T17:58:14.915023+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:41:39.880180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914087+0000","last_change":"2023-12-19T17:58:13.500389+0000","last_active":"2023-12-19T17:58:14.914087+0000","last_peered":"2023-12-19T17:58:14.914087+0000","last_clean":"2023-12-19T17:58:14.914087+0000","last_became_active":"2023-12-19T17:58:13.499316+0000","last_became_peered":"2023-12-19T17:58:13.499316+0000","last_unstale":"2023-12-19T17:58:14.914087+0000","last_undegraded":"2023-12-19T17:58:14.914087+0000","last_fullsized":"2023-12-19T17:58:14.914087+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:14.799735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914739+0000","last_change":"2023-12-19T17:58:13.502787+0000","last_active":"2023-12-19T17:58:14.914739+0000","last_peered":"2023-12-19T17:58:14.914739+0000","last_clean":"2023-12-19T17:58:14.914739+0000","last_became_active":"2023-12-19T17:58:13.502515+0000","last_became_peered":"2023-12-19T17:58:13.502515+0000","last_unstale":"2023-12-19T17:58:14.914739+0000","last_undegraded":"2023-12-19T17:58:14.914739+0000","last_fullsized":"2023-12-19T17:58:14.914739+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:19:06.534141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915064+0000","last_change":"2023-12-19T17:58:13.509487+0000","last_active":"2023-12-19T17:58:14.915064+0000","last_peered":"2023-12-19T17:58:14.915064+0000","last_clean":"2023-12-19T17:58:14.915064+0000","last_became_active":"2023-12-19T17:58:13.508677+0000","last_became_peered":"2023-12-19T17:58:13.508677+0000","last_unstale":"2023-12-19T17:58:14.915064+0000","last_undegraded":"2023-12-19T17:58:14.915064+0000","last_fullsized":"2023-12-19T17:58:14.915064+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:26.054027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"176'4","reported_seq":15,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915142+0000","last_change":"2023-12-19T17:58:13.092896+0000","last_active":"2023-12-19T17:58:14.915142+0000","last_peered":"2023-12-19T17:58:14.915142+0000","last_clean":"2023-12-19T17:58:14.915142+0000","last_became_active":"2023-12-19T17:58:13.092330+0000","last_became_peered":"2023-12-19T17:58:13.092330+0000","last_unstale":"2023-12-19T17:58:14.915142+0000","last_undegraded":"2023-12-19T17:58:14.915142+0000","last_fullsized":"2023-12-19T17:58:14.915142+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:26:35.026082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914791+0000","last_change":"2023-12-19T17:58:13.503555+0000","last_active":"2023-12-19T17:58:14.914791+0000","last_peered":"2023-12-19T17:58:14.914791+0000","last_clean":"2023-12-19T17:58:14.914791+0000","last_became_active":"2023-12-19T17:58:13.501132+0000","last_became_peered":"2023-12-19T17:58:13.501132+0000","last_unstale":"2023-12-19T17:58:14.914791+0000","last_undegraded":"2023-12-19T17:58:14.914791+0000","last_fullsized":"2023-12-19T17:58:14.914791+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:40:29.577603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"176'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914658+0000","last_change":"2023-12-19T17:58:13.092702+0000","last_active":"2023-12-19T17:58:14.914658+0000","last_peered":"2023-12-19T17:58:14.914658+0000","last_clean":"2023-12-19T17:58:14.914658+0000","last_became_active":"2023-12-19T17:58:13.092347+0000","last_became_peered":"2023-12-19T17:58:13.092347+0000","last_unstale":"2023-12-19T17:58:14.914658+0000","last_undegraded":"2023-12-19T17:58:14.914658+0000","last_fullsized":"2023-12-19T17:58:14.914658+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:07.798146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484519+0000","last_change":"2023-12-19T17:58:13.490769+0000","last_active":"2023-12-19T17:58:14.484519+0000","last_peered":"2023-12-19T17:58:14.484519+0000","last_clean":"2023-12-19T17:58:14.484519+0000","last_became_active":"2023-12-19T17:58:13.490505+0000","last_became_peered":"2023-12-19T17:58:13.490505+0000","last_unstale":"2023-12-19T17:58:14.484519+0000","last_undegraded":"2023-12-19T17:58:14.484519+0000","last_fullsized":"2023-12-19T17:58:14.484519+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:43:57.132764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"176'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915514+0000","last_change":"2023-12-19T17:58:13.082905+0000","last_active":"2023-12-19T17:58:14.915514+0000","last_peered":"2023-12-19T17:58:14.915514+0000","last_clean":"2023-12-19T17:58:14.915514+0000","last_became_active":"2023-12-19T17:58:13.081254+0000","last_became_peered":"2023-12-19T17:58:13.081254+0000","last_unstale":"2023-12-19T17:58:14.915514+0000","last_undegraded":"2023-12-19T17:58:14.915514+0000","last_fullsized":"2023-12-19T17:58:14.915514+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:43:56.664402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914918+0000","last_change":"2023-12-19T17:58:13.510286+0000","last_active":"2023-12-19T17:58:14.914918+0000","last_peered":"2023-12-19T17:58:14.914918+0000","last_clean":"2023-12-19T17:58:14.914918+0000","last_became_active":"2023-12-19T17:58:13.509123+0000","last_became_peered":"2023-12-19T17:58:13.509123+0000","last_unstale":"2023-12-19T17:58:14.914918+0000","last_undegraded":"2023-12-19T17:58:14.914918+0000","last_fullsized":"2023-12-19T17:58:14.914918+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:05:27.227183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"176'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915597+0000","last_change":"2023-12-19T17:58:13.089210+0000","last_active":"2023-12-19T17:58:14.915597+0000","last_peered":"2023-12-19T17:58:14.915597+0000","last_clean":"2023-12-19T17:58:14.915597+0000","last_became_active":"2023-12-19T17:58:13.088232+0000","last_became_peered":"2023-12-19T17:58:13.088232+0000","last_unstale":"2023-12-19T17:58:14.915597+0000","last_undegraded":"2023-12-19T17:58:14.915597+0000","last_fullsized":"2023-12-19T17:58:14.915597+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:17.813715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914244+0000","last_change":"2023-12-19T17:58:13.498269+0000","last_active":"2023-12-19T17:58:14.914244+0000","last_peered":"2023-12-19T17:58:14.914244+0000","last_clean":"2023-12-19T17:58:14.914244+0000","last_became_active":"2023-12-19T17:58:13.498005+0000","last_became_peered":"2023-12-19T17:58:13.498005+0000","last_unstale":"2023-12-19T17:58:14.914244+0000","last_undegraded":"2023-12-19T17:58:14.914244+0000","last_fullsized":"2023-12-19T17:58:14.914244+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:05:27.227183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"176'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914649+0000","last_change":"2023-12-19T17:58:13.093194+0000","last_active":"2023-12-19T17:58:14.914649+0000","last_peered":"2023-12-19T17:58:14.914649+0000","last_clean":"2023-12-19T17:58:14.914649+0000","last_became_active":"2023-12-19T17:58:13.092476+0000","last_became_peered":"2023-12-19T17:58:13.092476+0000","last_unstale":"2023-12-19T17:58:14.914649+0000","last_undegraded":"2023-12-19T17:58:14.914649+0000","last_fullsized":"2023-12-19T17:58:14.914649+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:19:22.849330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915188+0000","last_change":"2023-12-19T17:58:13.496911+0000","last_active":"2023-12-19T17:58:14.915188+0000","last_peered":"2023-12-19T17:58:14.915188+0000","last_clean":"2023-12-19T17:58:14.915188+0000","last_became_active":"2023-12-19T17:58:13.496635+0000","last_became_peered":"2023-12-19T17:58:13.496635+0000","last_unstale":"2023-12-19T17:58:14.915188+0000","last_undegraded":"2023-12-19T17:58:14.915188+0000","last_fullsized":"2023-12-19T17:58:14.915188+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:33:09.805442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"176'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485842+0000","last_change":"2023-12-19T17:58:13.089476+0000","last_active":"2023-12-19T17:58:14.485842+0000","last_peered":"2023-12-19T17:58:14.485842+0000","last_clean":"2023-12-19T17:58:14.485842+0000","last_became_active":"2023-12-19T17:58:13.089242+0000","last_became_peered":"2023-12-19T17:58:13.089242+0000","last_unstale":"2023-12-19T17:58:14.485842+0000","last_undegraded":"2023-12-19T17:58:14.485842+0000","last_fullsized":"2023-12-19T17:58:14.485842+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:55.995033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914060+0000","last_change":"2023-12-19T17:58:13.497322+0000","last_active":"2023-12-19T17:58:14.914060+0000","last_peered":"2023-12-19T17:58:14.914060+0000","last_clean":"2023-12-19T17:58:14.914060+0000","last_became_active":"2023-12-19T17:58:13.497048+0000","last_became_peered":"2023-12-19T17:58:13.497048+0000","last_unstale":"2023-12-19T17:58:14.914060+0000","last_undegraded":"2023-12-19T17:58:14.914060+0000","last_fullsized":"2023-12-19T17:58:14.914060+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:26.054027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914383+0000","last_change":"2023-12-19T17:58:13.089925+0000","last_active":"2023-12-19T17:58:14.914383+0000","last_peered":"2023-12-19T17:58:14.914383+0000","last_clean":"2023-12-19T17:58:14.914383+0000","last_became_active":"2023-12-19T17:58:13.088473+0000","last_became_peered":"2023-12-19T17:58:13.088473+0000","last_unstale":"2023-12-19T17:58:14.914383+0000","last_undegraded":"2023-12-19T17:58:14.914383+0000","last_fullsized":"2023-12-19T17:58:14.914383+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:00:04.341736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485736+0000","last_change":"2023-12-19T17:58:13.497977+0000","last_active":"2023-12-19T17:58:14.485736+0000","last_peered":"2023-12-19T17:58:14.485736+0000","last_clean":"2023-12-19T17:58:14.485736+0000","last_became_active":"2023-12-19T17:58:13.497627+0000","last_became_peered":"2023-12-19T17:58:13.497627+0000","last_unstale":"2023-12-19T17:58:14.485736+0000","last_undegraded":"2023-12-19T17:58:14.485736+0000","last_fullsized":"2023-12-19T17:58:14.485736+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:45:42.444705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914512+0000","last_change":"2023-12-19T17:58:13.089636+0000","last_active":"2023-12-19T17:58:14.914512+0000","last_peered":"2023-12-19T17:58:14.914512+0000","last_clean":"2023-12-19T17:58:14.914512+0000","last_became_active":"2023-12-19T17:58:13.089328+0000","last_became_peered":"2023-12-19T17:58:13.089328+0000","last_unstale":"2023-12-19T17:58:14.914512+0000","last_undegraded":"2023-12-19T17:58:14.914512+0000","last_fullsized":"2023-12-19T17:58:14.914512+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:02.104297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486094+0000","last_change":"2023-12-19T17:58:13.496146+0000","last_active":"2023-12-19T17:58:14.486094+0000","last_peered":"2023-12-19T17:58:14.486094+0000","last_clean":"2023-12-19T17:58:14.486094+0000","last_became_active":"2023-12-19T17:58:13.495862+0000","last_became_peered":"2023-12-19T17:58:13.495862+0000","last_unstale":"2023-12-19T17:58:14.486094+0000","last_undegraded":"2023-12-19T17:58:14.486094+0000","last_fullsized":"2023-12-19T17:58:14.486094+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:42:13.687152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914925+0000","last_change":"2023-12-19T17:58:13.091763+0000","last_active":"2023-12-19T17:58:14.914925+0000","last_peered":"2023-12-19T17:58:14.914925+0000","last_clean":"2023-12-19T17:58:14.914925+0000","last_became_active":"2023-12-19T17:58:13.091449+0000","last_became_peered":"2023-12-19T17:58:13.091449+0000","last_unstale":"2023-12-19T17:58:14.914925+0000","last_undegraded":"2023-12-19T17:58:14.914925+0000","last_fullsized":"2023-12-19T17:58:14.914925+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:19:32.748126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915182+0000","last_change":"2023-12-19T17:58:13.509781+0000","last_active":"2023-12-19T17:58:14.915182+0000","last_peered":"2023-12-19T17:58:14.915182+0000","last_clean":"2023-12-19T17:58:14.915182+0000","last_became_active":"2023-12-19T17:58:13.508650+0000","last_became_peered":"2023-12-19T17:58:13.508650+0000","last_unstale":"2023-12-19T17:58:14.915182+0000","last_undegraded":"2023-12-19T17:58:14.915182+0000","last_fullsized":"2023-12-19T17:58:14.915182+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:00.402828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915181+0000","last_change":"2023-12-19T17:58:13.092307+0000","last_active":"2023-12-19T17:58:14.915181+0000","last_peered":"2023-12-19T17:58:14.915181+0000","last_clean":"2023-12-19T17:58:14.915181+0000","last_became_active":"2023-12-19T17:58:13.091998+0000","last_became_peered":"2023-12-19T17:58:13.091998+0000","last_unstale":"2023-12-19T17:58:14.915181+0000","last_undegraded":"2023-12-19T17:58:14.915181+0000","last_fullsized":"2023-12-19T17:58:14.915181+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:47:09.951173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915077+0000","last_change":"2023-12-19T17:58:13.500738+0000","last_active":"2023-12-19T17:58:14.915077+0000","last_peered":"2023-12-19T17:58:14.915077+0000","last_clean":"2023-12-19T17:58:14.915077+0000","last_became_active":"2023-12-19T17:58:13.500508+0000","last_became_peered":"2023-12-19T17:58:13.500508+0000","last_unstale":"2023-12-19T17:58:14.915077+0000","last_undegraded":"2023-12-19T17:58:14.915077+0000","last_fullsized":"2023-12-19T17:58:14.915077+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:45:42.444705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"176'6","reported_seq":17,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915456+0000","last_change":"2023-12-19T17:58:13.089843+0000","last_active":"2023-12-19T17:58:14.915456+0000","last_peered":"2023-12-19T17:58:14.915456+0000","last_clean":"2023-12-19T17:58:14.915456+0000","last_became_active":"2023-12-19T17:58:13.089523+0000","last_became_peered":"2023-12-19T17:58:13.089523+0000","last_unstale":"2023-12-19T17:58:14.915456+0000","last_undegraded":"2023-12-19T17:58:14.915456+0000","last_fullsized":"2023-12-19T17:58:14.915456+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:05:43.382647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486477+0000","last_change":"2023-12-19T17:58:13.505531+0000","last_active":"2023-12-19T17:58:14.486477+0000","last_peered":"2023-12-19T17:58:14.486477+0000","last_clean":"2023-12-19T17:58:14.486477+0000","last_became_active":"2023-12-19T17:58:13.505202+0000","last_became_peered":"2023-12-19T17:58:13.505202+0000","last_unstale":"2023-12-19T17:58:14.486477+0000","last_undegraded":"2023-12-19T17:58:14.486477+0000","last_fullsized":"2023-12-19T17:58:14.486477+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:03:42.151305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"176'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486758+0000","last_change":"2023-12-19T17:58:13.086018+0000","last_active":"2023-12-19T17:58:14.486758+0000","last_peered":"2023-12-19T17:58:14.486758+0000","last_clean":"2023-12-19T17:58:14.486758+0000","last_became_active":"2023-12-19T17:58:13.085569+0000","last_became_peered":"2023-12-19T17:58:13.085569+0000","last_unstale":"2023-12-19T17:58:14.486758+0000","last_undegraded":"2023-12-19T17:58:14.486758+0000","last_fullsized":"2023-12-19T17:58:14.486758+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:18:46.972146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914275+0000","last_change":"2023-12-19T17:58:13.500672+0000","last_active":"2023-12-19T17:58:14.914275+0000","last_peered":"2023-12-19T17:58:14.914275+0000","last_clean":"2023-12-19T17:58:14.914275+0000","last_became_active":"2023-12-19T17:58:13.500425+0000","last_became_peered":"2023-12-19T17:58:13.500425+0000","last_unstale":"2023-12-19T17:58:14.914275+0000","last_undegraded":"2023-12-19T17:58:14.914275+0000","last_fullsized":"2023-12-19T17:58:14.914275+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:00.402828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486495+0000","last_change":"2023-12-19T17:58:13.087130+0000","last_active":"2023-12-19T17:58:14.486495+0000","last_peered":"2023-12-19T17:58:14.486495+0000","last_clean":"2023-12-19T17:58:14.486495+0000","last_became_active":"2023-12-19T17:58:13.086841+0000","last_became_peered":"2023-12-19T17:58:13.086841+0000","last_unstale":"2023-12-19T17:58:14.486495+0000","last_undegraded":"2023-12-19T17:58:14.486495+0000","last_fullsized":"2023-12-19T17:58:14.486495+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:02.104297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485936+0000","last_change":"2023-12-19T17:58:13.499626+0000","last_active":"2023-12-19T17:58:14.485936+0000","last_peered":"2023-12-19T17:58:14.485936+0000","last_clean":"2023-12-19T17:58:14.485936+0000","last_became_active":"2023-12-19T17:58:13.499337+0000","last_became_peered":"2023-12-19T17:58:13.499337+0000","last_unstale":"2023-12-19T17:58:14.485936+0000","last_undegraded":"2023-12-19T17:58:14.485936+0000","last_fullsized":"2023-12-19T17:58:14.485936+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:26:35.242597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484381+0000","last_change":"2023-12-19T17:58:13.085345+0000","last_active":"2023-12-19T17:58:14.484381+0000","last_peered":"2023-12-19T17:58:14.484381+0000","last_clean":"2023-12-19T17:58:14.484381+0000","last_became_active":"2023-12-19T17:58:13.084999+0000","last_became_peered":"2023-12-19T17:58:13.084999+0000","last_unstale":"2023-12-19T17:58:14.484381+0000","last_undegraded":"2023-12-19T17:58:14.484381+0000","last_fullsized":"2023-12-19T17:58:14.484381+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:26:35.026082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485655+0000","last_change":"2023-12-19T17:58:13.492863+0000","last_active":"2023-12-19T17:58:14.485655+0000","last_peered":"2023-12-19T17:58:14.485655+0000","last_clean":"2023-12-19T17:58:14.485655+0000","last_became_active":"2023-12-19T17:58:13.492604+0000","last_became_peered":"2023-12-19T17:58:13.492604+0000","last_unstale":"2023-12-19T17:58:14.485655+0000","last_undegraded":"2023-12-19T17:58:14.485655+0000","last_fullsized":"2023-12-19T17:58:14.485655+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:40.235025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"176'8","reported_seq":19,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914638+0000","last_change":"2023-12-19T17:58:13.087821+0000","last_active":"2023-12-19T17:58:14.914638+0000","last_peered":"2023-12-19T17:58:14.914638+0000","last_clean":"2023-12-19T17:58:14.914638+0000","last_became_active":"2023-12-19T17:58:13.087532+0000","last_became_peered":"2023-12-19T17:58:13.087532+0000","last_unstale":"2023-12-19T17:58:14.914638+0000","last_undegraded":"2023-12-19T17:58:14.914638+0000","last_fullsized":"2023-12-19T17:58:14.914638+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:57.517770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915135+0000","last_change":"2023-12-19T17:58:13.505204+0000","last_active":"2023-12-19T17:58:14.915135+0000","last_peered":"2023-12-19T17:58:14.915135+0000","last_clean":"2023-12-19T17:58:14.915135+0000","last_became_active":"2023-12-19T17:58:13.504894+0000","last_became_peered":"2023-12-19T17:58:13.504894+0000","last_unstale":"2023-12-19T17:58:14.915135+0000","last_undegraded":"2023-12-19T17:58:14.915135+0000","last_fullsized":"2023-12-19T17:58:14.915135+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:49.808864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"176'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486319+0000","last_change":"2023-12-19T17:58:13.083200+0000","last_active":"2023-12-19T17:58:14.486319+0000","last_peered":"2023-12-19T17:58:14.486319+0000","last_clean":"2023-12-19T17:58:14.486319+0000","last_became_active":"2023-12-19T17:58:13.082938+0000","last_became_peered":"2023-12-19T17:58:13.082938+0000","last_unstale":"2023-12-19T17:58:14.486319+0000","last_undegraded":"2023-12-19T17:58:14.486319+0000","last_fullsized":"2023-12-19T17:58:14.486319+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:39.370236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915315+0000","last_change":"2023-12-19T17:58:13.510086+0000","last_active":"2023-12-19T17:58:14.915315+0000","last_peered":"2023-12-19T17:58:14.915315+0000","last_clean":"2023-12-19T17:58:14.915315+0000","last_became_active":"2023-12-19T17:58:13.509510+0000","last_became_peered":"2023-12-19T17:58:13.509510+0000","last_unstale":"2023-12-19T17:58:14.915315+0000","last_undegraded":"2023-12-19T17:58:14.915315+0000","last_fullsized":"2023-12-19T17:58:14.915315+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:22.896718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"176'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914754+0000","last_change":"2023-12-19T17:58:13.091802+0000","last_active":"2023-12-19T17:58:14.914754+0000","last_peered":"2023-12-19T17:58:14.914754+0000","last_clean":"2023-12-19T17:58:14.914754+0000","last_became_active":"2023-12-19T17:58:13.091483+0000","last_became_peered":"2023-12-19T17:58:13.091483+0000","last_unstale":"2023-12-19T17:58:14.914754+0000","last_undegraded":"2023-12-19T17:58:14.914754+0000","last_fullsized":"2023-12-19T17:58:14.914754+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:03.465410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486848+0000","last_change":"2023-12-19T17:58:13.512182+0000","last_active":"2023-12-19T17:58:14.486848+0000","last_peered":"2023-12-19T17:58:14.486848+0000","last_clean":"2023-12-19T17:58:14.486848+0000","last_became_active":"2023-12-19T17:58:13.511913+0000","last_became_peered":"2023-12-19T17:58:13.511913+0000","last_unstale":"2023-12-19T17:58:14.486848+0000","last_undegraded":"2023-12-19T17:58:14.486848+0000","last_fullsized":"2023-12-19T17:58:14.486848+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:37.415349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"176'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915242+0000","last_change":"2023-12-19T17:58:13.093203+0000","last_active":"2023-12-19T17:58:14.915242+0000","last_peered":"2023-12-19T17:58:14.915242+0000","last_clean":"2023-12-19T17:58:14.915242+0000","last_became_active":"2023-12-19T17:58:13.092111+0000","last_became_peered":"2023-12-19T17:58:13.092111+0000","last_unstale":"2023-12-19T17:58:14.915242+0000","last_undegraded":"2023-12-19T17:58:14.915242+0000","last_fullsized":"2023-12-19T17:58:14.915242+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:47:31.206655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484614+0000","last_change":"2023-12-19T17:58:13.500853+0000","last_active":"2023-12-19T17:58:14.484614+0000","last_peered":"2023-12-19T17:58:14.484614+0000","last_clean":"2023-12-19T17:58:14.484614+0000","last_became_active":"2023-12-19T17:58:13.500256+0000","last_became_peered":"2023-12-19T17:58:13.500256+0000","last_unstale":"2023-12-19T17:58:14.484614+0000","last_undegraded":"2023-12-19T17:58:14.484614+0000","last_fullsized":"2023-12-19T17:58:14.484614+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:29.304411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914845+0000","last_change":"2023-12-19T17:58:13.095219+0000","last_active":"2023-12-19T17:58:14.914845+0000","last_peered":"2023-12-19T17:58:14.914845+0000","last_clean":"2023-12-19T17:58:14.914845+0000","last_became_active":"2023-12-19T17:58:13.094936+0000","last_became_peered":"2023-12-19T17:58:13.094936+0000","last_unstale":"2023-12-19T17:58:14.914845+0000","last_undegraded":"2023-12-19T17:58:14.914845+0000","last_fullsized":"2023-12-19T17:58:14.914845+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:39:15.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915077+0000","last_change":"2023-12-19T17:58:13.496062+0000","last_active":"2023-12-19T17:58:14.915077+0000","last_peered":"2023-12-19T17:58:14.915077+0000","last_clean":"2023-12-19T17:58:14.915077+0000","last_became_active":"2023-12-19T17:58:13.495067+0000","last_became_peered":"2023-12-19T17:58:13.495067+0000","last_unstale":"2023-12-19T17:58:14.915077+0000","last_undegraded":"2023-12-19T17:58:14.915077+0000","last_fullsized":"2023-12-19T17:58:14.915077+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:40.235025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"176'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915336+0000","last_change":"2023-12-19T17:58:13.083333+0000","last_active":"2023-12-19T17:58:14.915336+0000","last_peered":"2023-12-19T17:58:14.915336+0000","last_clean":"2023-12-19T17:58:14.915336+0000","last_became_active":"2023-12-19T17:58:13.083019+0000","last_became_peered":"2023-12-19T17:58:13.083019+0000","last_unstale":"2023-12-19T17:58:14.915336+0000","last_undegraded":"2023-12-19T17:58:14.915336+0000","last_fullsized":"2023-12-19T17:58:14.915336+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:46.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915172+0000","last_change":"2023-12-19T17:58:13.485884+0000","last_active":"2023-12-19T17:58:14.915172+0000","last_peered":"2023-12-19T17:58:14.915172+0000","last_clean":"2023-12-19T17:58:14.915172+0000","last_became_active":"2023-12-19T17:58:13.485541+0000","last_became_peered":"2023-12-19T17:58:13.485541+0000","last_unstale":"2023-12-19T17:58:14.915172+0000","last_undegraded":"2023-12-19T17:58:14.915172+0000","last_fullsized":"2023-12-19T17:58:14.915172+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:17:12.269946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914571+0000","last_change":"2023-12-19T17:58:13.092150+0000","last_active":"2023-12-19T17:58:14.914571+0000","last_peered":"2023-12-19T17:58:14.914571+0000","last_clean":"2023-12-19T17:58:14.914571+0000","last_became_active":"2023-12-19T17:58:13.091853+0000","last_became_peered":"2023-12-19T17:58:13.091853+0000","last_unstale":"2023-12-19T17:58:14.914571+0000","last_undegraded":"2023-12-19T17:58:14.914571+0000","last_fullsized":"2023-12-19T17:58:14.914571+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:08:47.461390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914481+0000","last_change":"2023-12-19T17:58:13.486929+0000","last_active":"2023-12-19T17:58:14.914481+0000","last_peered":"2023-12-19T17:58:14.914481+0000","last_clean":"2023-12-19T17:58:14.914481+0000","last_became_active":"2023-12-19T17:58:13.486664+0000","last_became_peered":"2023-12-19T17:58:13.486664+0000","last_unstale":"2023-12-19T17:58:14.914481+0000","last_undegraded":"2023-12-19T17:58:14.914481+0000","last_fullsized":"2023-12-19T17:58:14.914481+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:03:42.151305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914467+0000","last_change":"2023-12-19T17:58:13.092726+0000","last_active":"2023-12-19T17:58:14.914467+0000","last_peered":"2023-12-19T17:58:14.914467+0000","last_clean":"2023-12-19T17:58:14.914467+0000","last_became_active":"2023-12-19T17:58:13.092013+0000","last_became_peered":"2023-12-19T17:58:13.092013+0000","last_unstale":"2023-12-19T17:58:14.914467+0000","last_undegraded":"2023-12-19T17:58:14.914467+0000","last_fullsized":"2023-12-19T17:58:14.914467+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:17:13.408148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915157+0000","last_change":"2023-12-19T17:58:13.486204+0000","last_active":"2023-12-19T17:58:14.915157+0000","last_peered":"2023-12-19T17:58:14.915157+0000","last_clean":"2023-12-19T17:58:14.915157+0000","last_became_active":"2023-12-19T17:58:13.485921+0000","last_became_peered":"2023-12-19T17:58:13.485921+0000","last_unstale":"2023-12-19T17:58:14.915157+0000","last_undegraded":"2023-12-19T17:58:14.915157+0000","last_fullsized":"2023-12-19T17:58:14.915157+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:26:45.339697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485675+0000","last_change":"2023-12-19T17:58:13.087755+0000","last_active":"2023-12-19T17:58:14.485675+0000","last_peered":"2023-12-19T17:58:14.485675+0000","last_clean":"2023-12-19T17:58:14.485675+0000","last_became_active":"2023-12-19T17:58:13.087449+0000","last_became_peered":"2023-12-19T17:58:13.087449+0000","last_unstale":"2023-12-19T17:58:14.485675+0000","last_undegraded":"2023-12-19T17:58:14.485675+0000","last_fullsized":"2023-12-19T17:58:14.485675+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:15:04.127346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486078+0000","last_change":"2023-12-19T17:58:13.508091+0000","last_active":"2023-12-19T17:58:14.486078+0000","last_peered":"2023-12-19T17:58:14.486078+0000","last_clean":"2023-12-19T17:58:14.486078+0000","last_became_active":"2023-12-19T17:58:13.507813+0000","last_became_peered":"2023-12-19T17:58:13.507813+0000","last_unstale":"2023-12-19T17:58:14.486078+0000","last_undegraded":"2023-12-19T17:58:14.486078+0000","last_fullsized":"2023-12-19T17:58:14.486078+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:01:17.175289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"176'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915292+0000","last_change":"2023-12-19T17:58:13.092990+0000","last_active":"2023-12-19T17:58:14.915292+0000","last_peered":"2023-12-19T17:58:14.915292+0000","last_clean":"2023-12-19T17:58:14.915292+0000","last_became_active":"2023-12-19T17:58:13.092634+0000","last_became_peered":"2023-12-19T17:58:13.092634+0000","last_unstale":"2023-12-19T17:58:14.915292+0000","last_undegraded":"2023-12-19T17:58:14.915292+0000","last_fullsized":"2023-12-19T17:58:14.915292+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:48.130738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486898+0000","last_change":"2023-12-19T17:58:13.505468+0000","last_active":"2023-12-19T17:58:14.486898+0000","last_peered":"2023-12-19T17:58:14.486898+0000","last_clean":"2023-12-19T17:58:14.486898+0000","last_became_active":"2023-12-19T17:58:13.505181+0000","last_became_peered":"2023-12-19T17:58:13.505181+0000","last_unstale":"2023-12-19T17:58:14.486898+0000","last_undegraded":"2023-12-19T17:58:14.486898+0000","last_fullsized":"2023-12-19T17:58:14.486898+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:33.295886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"176'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484805+0000","last_change":"2023-12-19T17:58:13.090504+0000","last_active":"2023-12-19T17:58:14.484805+0000","last_peered":"2023-12-19T17:58:14.484805+0000","last_clean":"2023-12-19T17:58:14.484805+0000","last_became_active":"2023-12-19T17:58:13.090180+0000","last_became_peered":"2023-12-19T17:58:13.090180+0000","last_unstale":"2023-12-19T17:58:14.484805+0000","last_undegraded":"2023-12-19T17:58:14.484805+0000","last_fullsized":"2023-12-19T17:58:14.484805+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:58:26.950125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914548+0000","last_change":"2023-12-19T17:58:13.496201+0000","last_active":"2023-12-19T17:58:14.914548+0000","last_peered":"2023-12-19T17:58:14.914548+0000","last_clean":"2023-12-19T17:58:14.914548+0000","last_became_active":"2023-12-19T17:58:13.495921+0000","last_became_peered":"2023-12-19T17:58:13.495921+0000","last_unstale":"2023-12-19T17:58:14.914548+0000","last_undegraded":"2023-12-19T17:58:14.914548+0000","last_fullsized":"2023-12-19T17:58:14.914548+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:17.870336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484812+0000","last_change":"2023-12-19T17:58:13.086573+0000","last_active":"2023-12-19T17:58:14.484812+0000","last_peered":"2023-12-19T17:58:14.484812+0000","last_clean":"2023-12-19T17:58:14.484812+0000","last_became_active":"2023-12-19T17:58:13.086111+0000","last_became_peered":"2023-12-19T17:58:13.086111+0000","last_unstale":"2023-12-19T17:58:14.484812+0000","last_undegraded":"2023-12-19T17:58:14.484812+0000","last_fullsized":"2023-12-19T17:58:14.484812+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:08:47.461390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484726+0000","last_change":"2023-12-19T17:58:13.494408+0000","last_active":"2023-12-19T17:58:14.484726+0000","last_peered":"2023-12-19T17:58:14.484726+0000","last_clean":"2023-12-19T17:58:14.484726+0000","last_became_active":"2023-12-19T17:58:13.494170+0000","last_became_peered":"2023-12-19T17:58:13.494170+0000","last_unstale":"2023-12-19T17:58:14.484726+0000","last_undegraded":"2023-12-19T17:58:14.484726+0000","last_fullsized":"2023-12-19T17:58:14.484726+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:28:05.209260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484692+0000","last_change":"2023-12-19T17:58:13.086066+0000","last_active":"2023-12-19T17:58:14.484692+0000","last_peered":"2023-12-19T17:58:14.484692+0000","last_clean":"2023-12-19T17:58:14.484692+0000","last_became_active":"2023-12-19T17:58:13.085795+0000","last_became_peered":"2023-12-19T17:58:13.085795+0000","last_unstale":"2023-12-19T17:58:14.484692+0000","last_undegraded":"2023-12-19T17:58:14.484692+0000","last_fullsized":"2023-12-19T17:58:14.484692+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:44:07.815304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914405+0000","last_change":"2023-12-19T17:58:13.495612+0000","last_active":"2023-12-19T17:58:14.914405+0000","last_peered":"2023-12-19T17:58:14.914405+0000","last_clean":"2023-12-19T17:58:14.914405+0000","last_became_active":"2023-12-19T17:58:13.495342+0000","last_became_peered":"2023-12-19T17:58:13.495342+0000","last_unstale":"2023-12-19T17:58:14.914405+0000","last_undegraded":"2023-12-19T17:58:14.914405+0000","last_fullsized":"2023-12-19T17:58:14.914405+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:47.531250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915317+0000","last_change":"2023-12-19T17:58:13.076018+0000","last_active":"2023-12-19T17:58:14.915317+0000","last_peered":"2023-12-19T17:58:14.915317+0000","last_clean":"2023-12-19T17:58:14.915317+0000","last_became_active":"2023-12-19T17:58:13.075737+0000","last_became_peered":"2023-12-19T17:58:13.075737+0000","last_unstale":"2023-12-19T17:58:14.915317+0000","last_undegraded":"2023-12-19T17:58:14.915317+0000","last_fullsized":"2023-12-19T17:58:14.915317+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:56:15.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486197+0000","last_change":"2023-12-19T17:58:13.493436+0000","last_active":"2023-12-19T17:58:14.486197+0000","last_peered":"2023-12-19T17:58:14.486197+0000","last_clean":"2023-12-19T17:58:14.486197+0000","last_became_active":"2023-12-19T17:58:13.493124+0000","last_became_peered":"2023-12-19T17:58:13.493124+0000","last_unstale":"2023-12-19T17:58:14.486197+0000","last_undegraded":"2023-12-19T17:58:14.486197+0000","last_fullsized":"2023-12-19T17:58:14.486197+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:36:10.784681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915241+0000","last_change":"2023-12-19T17:58:13.092618+0000","last_active":"2023-12-19T17:58:14.915241+0000","last_peered":"2023-12-19T17:58:14.915241+0000","last_clean":"2023-12-19T17:58:14.915241+0000","last_became_active":"2023-12-19T17:58:13.092394+0000","last_became_peered":"2023-12-19T17:58:13.092394+0000","last_unstale":"2023-12-19T17:58:14.915241+0000","last_undegraded":"2023-12-19T17:58:14.915241+0000","last_fullsized":"2023-12-19T17:58:14.915241+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:58:26.950125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914601+0000","last_change":"2023-12-19T17:58:13.508248+0000","last_active":"2023-12-19T17:58:14.914601+0000","last_peered":"2023-12-19T17:58:14.914601+0000","last_clean":"2023-12-19T17:58:14.914601+0000","last_became_active":"2023-12-19T17:58:13.507965+0000","last_became_peered":"2023-12-19T17:58:13.507965+0000","last_unstale":"2023-12-19T17:58:14.914601+0000","last_undegraded":"2023-12-19T17:58:14.914601+0000","last_fullsized":"2023-12-19T17:58:14.914601+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:27:03.254793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486084+0000","last_change":"2023-12-19T17:58:13.086445+0000","last_active":"2023-12-19T17:58:14.486084+0000","last_peered":"2023-12-19T17:58:14.486084+0000","last_clean":"2023-12-19T17:58:14.486084+0000","last_became_active":"2023-12-19T17:58:13.086187+0000","last_became_peered":"2023-12-19T17:58:13.086187+0000","last_unstale":"2023-12-19T17:58:14.486084+0000","last_undegraded":"2023-12-19T17:58:14.486084+0000","last_fullsized":"2023-12-19T17:58:14.486084+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:17.813715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485752+0000","last_change":"2023-12-19T17:58:13.492514+0000","last_active":"2023-12-19T17:58:14.485752+0000","last_peered":"2023-12-19T17:58:14.485752+0000","last_clean":"2023-12-19T17:58:14.485752+0000","last_became_active":"2023-12-19T17:58:13.492259+0000","last_became_peered":"2023-12-19T17:58:13.492259+0000","last_unstale":"2023-12-19T17:58:14.485752+0000","last_undegraded":"2023-12-19T17:58:14.485752+0000","last_fullsized":"2023-12-19T17:58:14.485752+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:10:32.458962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914157+0000","last_change":"2023-12-19T17:58:13.094141+0000","last_active":"2023-12-19T17:58:14.914157+0000","last_peered":"2023-12-19T17:58:14.914157+0000","last_clean":"2023-12-19T17:58:14.914157+0000","last_became_active":"2023-12-19T17:58:13.093822+0000","last_became_peered":"2023-12-19T17:58:13.093822+0000","last_unstale":"2023-12-19T17:58:14.914157+0000","last_undegraded":"2023-12-19T17:58:14.914157+0000","last_fullsized":"2023-12-19T17:58:14.914157+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:16.122567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915380+0000","last_change":"2023-12-19T17:58:13.486835+0000","last_active":"2023-12-19T17:58:14.915380+0000","last_peered":"2023-12-19T17:58:14.915380+0000","last_clean":"2023-12-19T17:58:14.915380+0000","last_became_active":"2023-12-19T17:58:13.486591+0000","last_became_peered":"2023-12-19T17:58:13.486591+0000","last_unstale":"2023-12-19T17:58:14.915380+0000","last_undegraded":"2023-12-19T17:58:14.915380+0000","last_fullsized":"2023-12-19T17:58:14.915380+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:40:53.684360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"176'3","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486968+0000","last_change":"2023-12-19T17:58:13.087498+0000","last_active":"2023-12-19T17:58:14.486968+0000","last_peered":"2023-12-19T17:58:14.486968+0000","last_clean":"2023-12-19T17:58:14.486968+0000","last_became_active":"2023-12-19T17:58:13.087220+0000","last_became_peered":"2023-12-19T17:58:13.087220+0000","last_unstale":"2023-12-19T17:58:14.486968+0000","last_undegraded":"2023-12-19T17:58:14.486968+0000","last_fullsized":"2023-12-19T17:58:14.486968+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:34:55.421495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"165'291","reported_seq":950,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-19T17:58:13.487606+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T17:58:13.487606+0000","last_peered":"2023-12-19T17:58:13.487606+0000","last_clean":"2023-12-19T17:58:13.487606+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T17:58:13.487606+0000","last_undegraded":"2023-12-19T17:58:13.487606+0000","last_fullsized":"2023-12-19T17:58:13.487606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1634,"num_read_kb":3381,"num_write":475,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1634,"num_read_kb":3381,"num_write":475,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":291,"ondisk_log_size":291,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510349,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37880,"kb_used_data":2736,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94333960,"statfs":{"total":96636764160,"available":96597975040,"internally_reserved":0,"allocated":2801664,"data_stored":1739648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72699999999999998,"15min":0.73599999999999999},"min":{"1min":0.52900000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89000000000000001,"5min":1.012,"15min":1.0209999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.71399999999999997,"15min":0.72899999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78700000000000003,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.624}]},{"osd":1,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74199999999999999,"15min":0.74099999999999999},"min":{"1min":0.52300000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.91100000000000003,"5min":0.96899999999999997,"15min":0.997},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.747,"15min":0.75700000000000001},"min":{"1min":0.47599999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.05},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91400000000000003,"5min":0.92400000000000004,"15min":0.97999999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.50800000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94899999999999995,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.755,"15min":0.748},"min":{"1min":0.57999999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92900000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76100000000000001,"15min":0.77200000000000002},"min":{"1min":0.54200000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.99099999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.76300000000000001}]},{"osd":4,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.442,"5min":0.377,"15min":0.377},"max":{"1min":0.92900000000000005,"5min":0.93899999999999995,"15min":0.94299999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63200000000000001,"15min":0.61299999999999999},"min":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89000000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59799999999999998}]},{"osd":5,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65500000000000003,"15min":0.63},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.36799999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.67600000000000005,"15min":0.66300000000000003},"min":{"1min":0.45800000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.73599999999999999,"15min":0.71999999999999997},"min":{"1min":0.55900000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.70899999999999996,"15min":0.68799999999999994},"min":{"1min":0.52700000000000002,"5min":0.45900000000000002,"15min":0.44500000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.79100000000000004}]}]},{"osd":6,"up_from":15,"seq":64424510349,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63544,"kb_used_data":2160,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308296,"statfs":{"total":96636764160,"available":96571695104,"internally_reserved":0,"allocated":2211840,"data_stored":1148774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.48799999999999999,"5min":0.40300000000000002,"15min":0.36799999999999999},"max":{"1min":0.995,"5min":0.995,"15min":1.02},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.747,"15min":0.73799999999999999},"min":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92900000000000005,"5min":0.95699999999999996,"15min":1.329},"last":0.85199999999999998}]},{"osd":1,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.75,"15min":0.73799999999999999},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.42899999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.1419999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.77200000000000002,"15min":0.75700000000000001},"min":{"1min":0.65600000000000003,"5min":0.47799999999999998,"15min":0.45200000000000001},"max":{"1min":0.96499999999999997,"5min":1.026,"15min":1.026},"last":0.82599999999999996}]},{"osd":2,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78200000000000003,"15min":0.76300000000000001},"min":{"1min":0.61299999999999999,"5min":0.497,"15min":0.48799999999999999},"max":{"1min":0.97099999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76000000000000001,"15min":0.76900000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.497},"max":{"1min":0.97999999999999998,"5min":1.0029999999999999,"15min":1.853},"last":0.76200000000000001}]},{"osd":3,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80100000000000005,"15min":0.79400000000000004},"min":{"1min":0.63700000000000001,"5min":0.59899999999999998,"15min":0.45400000000000001},"max":{"1min":0.93799999999999994,"5min":1.0149999999999999,"15min":1.542},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75800000000000001,"15min":0.73699999999999999},"min":{"1min":0.57099999999999995,"5min":0.54700000000000004,"15min":0.48799999999999999},"max":{"1min":0.94699999999999995,"5min":0.98699999999999999,"15min":1.04},"last":0.67700000000000005}]},{"osd":4,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.36799999999999999},"max":{"1min":0.85399999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.499,"5min":0.41499999999999998,"15min":0.372},"max":{"1min":1.131,"5min":1.131,"15min":1.3919999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.46600000000000003,"5min":0.442,"15min":0.34100000000000003},"max":{"1min":0.86399999999999999,"5min":0.91000000000000003,"15min":1.0920000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.54700000000000004,"5min":0.44700000000000001,"15min":0.41399999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.65600000000000003}]},{"osd":7,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.57899999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1,"5min":1.0580000000000001,"15min":1.4259999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.71599999999999997},"min":{"1min":0.59499999999999997,"5min":0.503,"15min":0.42699999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0369999999999999},"last":0.747}]}]},{"osd":5,"up_from":15,"seq":64424510349,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45488,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94326352,"statfs":{"total":96636764160,"available":96590184448,"internally_reserved":0,"allocated":2203648,"data_stored":1148650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.40799999999999997},"max":{"1min":0.91500000000000004,"5min":1.0269999999999999,"15min":1.1120000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.49299999999999999,"5min":0.48899999999999999,"15min":0.374},"max":{"1min":0.879,"5min":1.0289999999999999,"15min":1.073},"last":0.86899999999999999}]},{"osd":1,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.75900000000000001,"15min":0.76400000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96299999999999997,"5min":1.0029999999999999,"15min":1.1659999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72999999999999998,"15min":0.73999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97499999999999998,"5min":1.0249999999999999,"15min":1.0469999999999999},"last":0.88800000000000001}]},{"osd":2,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93899999999999995,"5min":1.05,"15min":1.1040000000000001},"last":1.04},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92200000000000004,"5min":0.95899999999999996,"15min":1.0349999999999999},"last":0.90800000000000003}]},{"osd":3,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.131},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76100000000000001,"15min":0.77100000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90500000000000003,"5min":0.99299999999999999,"15min":1.125},"last":0.97899999999999998}]},{"osd":4,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.35699999999999998},"max":{"1min":0.88200000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.441,"5min":0.44,"15min":0.41399999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.77400000000000002}]},{"osd":6,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":1.048},"last":0.745},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.64600000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82899999999999996,"5min":0.85499999999999998,"15min":0.97699999999999998},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65700000000000003,"15min":0.66800000000000004},"min":{"1min":0.55100000000000005,"5min":0.46800000000000003,"15min":0.46300000000000002},"max":{"1min":0.89500000000000002,"5min":0.97399999999999998,"15min":1.016},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69399999999999995,"15min":0.70099999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.41299999999999998},"max":{"1min":0.90700000000000003,"5min":0.94999999999999996,"15min":1.0289999999999999},"last":0.80100000000000005}]}]},{"osd":4,"up_from":15,"seq":64424510349,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59424,"kb_used_data":2136,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312416,"statfs":{"total":96636764160,"available":96575913984,"internally_reserved":0,"allocated":2187264,"data_stored":1141596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.71199999999999997,"15min":0.73699999999999999},"min":{"1min":0.47299999999999998,"5min":0.44,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":1.036,"15min":1.056},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70799999999999996,"15min":0.73099999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.40400000000000003},"max":{"1min":0.80600000000000005,"5min":0.92500000000000004,"15min":1.0620000000000001},"last":0.58399999999999996}]},{"osd":1,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.74099999999999999,"15min":0.75600000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.995,"15min":1.0980000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.749,"15min":0.753},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92300000000000004,"5min":1.0489999999999999,"15min":1.0549999999999999},"last":0.71899999999999997}]},{"osd":2,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.73999999999999999,"15min":0.75900000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.995,"5min":1.026,"15min":1.042},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.55000000000000004,"5min":0.496,"15min":0.47299999999999998},"max":{"1min":0.95099999999999996,"5min":1.0149999999999999,"15min":1.018},"last":0.62}]},{"osd":3,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.72699999999999998,"15min":0.74099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90300000000000002,"5min":0.96799999999999997,"15min":1.0229999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.76700000000000002,"15min":0.77700000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92700000000000005,"5min":0.997,"15min":1.0049999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.625,"15min":0.62},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.379},"max":{"1min":0.78000000000000003,"5min":1.0029999999999999,"15min":1.0109999999999999},"last":0.73599999999999999}]},{"osd":6,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88800000000000001,"5min":1.03,"15min":1.079},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69799999999999995,"15min":0.67900000000000005},"min":{"1min":0.47299999999999998,"5min":0.46500000000000002,"15min":0.34599999999999997},"max":{"1min":0.873,"5min":0.96999999999999997,"15min":0.99099999999999999},"last":0.754}]},{"osd":7,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66800000000000004,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.44800000000000001,"15min":0.435},"max":{"1min":0.78700000000000003,"5min":1.075,"15min":1.075},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71699999999999997,"15min":0.70999999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.45500000000000002},"max":{"1min":0.93000000000000005,"5min":1.0629999999999999,"15min":1.1160000000000001},"last":0.98199999999999998}]}]},{"osd":3,"up_from":15,"seq":64424510349,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47152,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324688,"statfs":{"total":96636764160,"available":96588480512,"internally_reserved":0,"allocated":2203648,"data_stored":1148718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.47799999999999998,"15min":0.47299999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.29499999999999998},"max":{"1min":0.751,"5min":0.76700000000000002,"15min":1.01},"last":0.43099999999999999}]},{"osd":1,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.307},"max":{"1min":0.68799999999999994,"5min":0.72199999999999998,"15min":0.83899999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.39900000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72199999999999998,"5min":0.79100000000000004,"15min":0.92100000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.73199999999999998,"5min":0.93000000000000005,"15min":0.98699999999999999},"last":0.44900000000000001}]},{"osd":4,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.52700000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.82199999999999995,"15min":0.82999999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2210000000000001,"5min":1.236,"15min":1.371},"last":0.60899999999999999}]},{"osd":5,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.81899999999999995,"15min":0.82999999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.296},"last":0.745},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82399999999999995,"15min":0.81899999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.52800000000000002},"max":{"1min":1.1220000000000001,"5min":1.133,"15min":1.2290000000000001},"last":0.61899999999999999}]},{"osd":6,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85499999999999998,"15min":0.84999999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.359},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.85799999999999998,"15min":0.84899999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.59599999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.2829999999999999},"last":0.63300000000000001}]},{"osd":7,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.89500000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.4530000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89600000000000002,"15min":0.89900000000000002},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.59599999999999997},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.70399999999999996}]}]},{"osd":2,"up_from":15,"seq":64424510348,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63536,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308304,"statfs":{"total":96636764160,"available":96571703296,"internally_reserved":0,"allocated":2203648,"data_stored":1143230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44700000000000001,"15min":0.45800000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59999999999999998,"5min":0.71499999999999997,"15min":0.80800000000000005},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32000000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88600000000000001},"last":0.34899999999999998}]},{"osd":1,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.62,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.36499999999999999},"max":{"1min":0.80200000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.51900000000000002},"min":{"1min":0.375,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.67900000000000005,"5min":0.79600000000000004,"15min":0.93300000000000005},"last":0.43099999999999999}]},{"osd":3,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.60799999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82699999999999996,"5min":0.88,"15min":0.93300000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.38400000000000001,"15min":0.35599999999999998},"max":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.93300000000000005},"last":0.47099999999999997}]},{"osd":4,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.80100000000000005,"15min":0.80600000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.54000000000000004},"max":{"1min":1.1020000000000001,"5min":1.1639999999999999,"15min":1.3360000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79800000000000004,"15min":0.79000000000000004},"min":{"1min":0.53600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.1180000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.86599999999999999}]},{"osd":5,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80300000000000005,"15min":0.80500000000000005},"min":{"1min":0.60499999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.071,"5min":1.339,"15min":1.339},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.083,"5min":1.351,"15min":1.351},"last":0.81399999999999995}]},{"osd":6,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84199999999999997,"15min":0.84299999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.55800000000000005},"max":{"1min":1.0349999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.46600000000000003},"max":{"1min":1.1399999999999999,"5min":1.1619999999999999,"15min":1.171},"last":0.94399999999999995}]},{"osd":7,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.82699999999999996},"min":{"1min":0.69399999999999995,"5min":0.58699999999999997,"15min":0.53500000000000003},"max":{"1min":1.21,"5min":1.2629999999999999,"15min":1.3580000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.64900000000000002,"5min":0.63300000000000001,"15min":0.61899999999999999},"max":{"1min":1.1279999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.99199999999999999}]}]},{"osd":1,"up_from":15,"seq":64424510349,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55344,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316496,"statfs":{"total":96636764160,"available":96580091904,"internally_reserved":0,"allocated":2203648,"data_stored":1149210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.44600000000000001,"15min":0.45400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.28100000000000003}]},{"osd":2,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68300000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.57399999999999995,"15min":0.56799999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.32600000000000001}]},{"osd":3,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77300000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":0.81799999999999995,"15min":0.85199999999999998},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80700000000000005,"15min":0.80500000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.004,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.749},{"interface":"front","average":{"1min":0.752,"5min":0.79400000000000004,"15min":0.79900000000000004},"min":{"1min":0.51000000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.048,"5min":1.355,"15min":1.355},"last":0.67900000000000005}]},{"osd":5,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.81499999999999995,"15min":0.80600000000000005},"min":{"1min":0.60099999999999998,"5min":0.55600000000000005,"15min":0.54600000000000004},"max":{"1min":0.91400000000000003,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.81000000000000005,"15min":0.81200000000000006},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.92300000000000004,"5min":1.149,"15min":1.149},"last":0.73499999999999999}]},{"osd":6,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52200000000000002},"max":{"1min":0.91300000000000003,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.83099999999999996,"15min":0.82699999999999996},"min":{"1min":0.59499999999999997,"5min":0.56999999999999995,"15min":0.56499999999999995},"max":{"1min":0.95599999999999996,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.72799999999999998}]},{"osd":7,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.86899999999999999,"15min":0.85699999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.97099999999999997,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.82899999999999996,"15min":0.84699999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.59499999999999997},"max":{"1min":0.94999999999999996,"5min":1.258,"15min":1.258},"last":0.77000000000000002}]}]},{"osd":0,"up_from":14,"seq":60129543054,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60032,"kb_used_data":2744,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311808,"statfs":{"total":96636764160,"available":96575291392,"internally_reserved":0,"allocated":2809856,"data_stored":1740594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46000000000000002,"15min":0.45500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64100000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.29599999999999999},"max":{"1min":0.71499999999999997,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.35699999999999998,"15min":0.34300000000000003},"max":{"1min":0.60899999999999999,"5min":0.78500000000000003,"15min":0.83299999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65900000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.65000000000000002,"5min":0.84799999999999998,"15min":0.90400000000000003},"last":0.62},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.501,"5min":0.497,"15min":0.45000000000000001},"max":{"1min":0.71299999999999997,"5min":0.90000000000000002,"15min":0.93500000000000005},"last":0.68600000000000005}]},{"osd":4,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.44700000000000001},"max":{"1min":1.1080000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52400000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.52700000000000002}]},{"osd":5,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80000000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.50700000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.169},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81799999999999995,"15min":0.82499999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.351},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1619999999999999,"5min":1.173,"15min":1.2350000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82699999999999996,"15min":0.83099999999999996},"min":{"1min":0.63300000000000001,"5min":0.60299999999999998,"15min":0.505},"max":{"1min":1.079,"5min":1.1020000000000001,"15min":1.329},"last":0.625}]},{"osd":7,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84999999999999998,"15min":0.85599999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2909999999999999},"last":0.66000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T17:58:17.600 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T17:58:17.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.774+0000 7fc50196d640 1 Processor -- start 2023-12-19T17:58:17.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.774+0000 7fc50196d640 1 -- start start 2023-12-19T17:58:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.778+0000 7fc50196d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc4fc06f9d0 0x7fc4fc0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.778+0000 7fc50196d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4fc0c5b40 con 0x7fc4fc0c4070 2023-12-19T17:58:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.778+0000 7fc50196d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4fc0c5e20 con 0x7fc4fc15a7c0 2023-12-19T17:58:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.778+0000 7fc50196d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4fc0ce770 con 0x7fc4fc06f9d0 2023-12-19T17:58:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.778+0000 7fc4f3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc4fc06f9d0 0x7fc4fc0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.778+0000 7fc4f3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc4fc06f9d0 0x7fc4fc0ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38734/0 (socket says 172.21.15.26:38734) 2023-12-19T17:58:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.778+0000 7fc4f3fff640 1 -- 172.21.15.26:0/3603547579 learned_addr learned my addr 172.21.15.26:0/3603547579 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.778+0000 7fc4f3fff640 1 -- 172.21.15.26:0/3603547579 >> v1:172.21.15.160:6789/0 conn(0x7fc4fc15a7c0 legacy=0x7fc4fc15ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.778+0000 7fc4f3fff640 1 -- 172.21.15.26:0/3603547579 >> v1:172.21.15.26:6789/0 conn(0x7fc4fc0c4070 legacy=0x7fc4fc0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.778+0000 7fc4f3fff640 1 -- 172.21.15.26:0/3603547579 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc0cea50 con 0x7fc4fc06f9d0 2023-12-19T17:58:17.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc4f3fff640 1 --2- 172.21.15.26:0/3603547579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4fc06f9d0 0x7fc4fc0ceff0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009870 tx=0x7fc4e401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e0c126d3cf35db server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:17.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc4fa7fc640 1 -- 172.21.15.26:0/3603547579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40ab620 con 0x7fc4fc06f9d0 2023-12-19T17:58:17.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc4fa7fc640 1 -- 172.21.15.26:0/3603547579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e40ab780 con 0x7fc4fc06f9d0 2023-12-19T17:58:17.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc4fa7fc640 1 -- 172.21.15.26:0/3603547579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40abb10 con 0x7fc4fc06f9d0 2023-12-19T17:58:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 -- 172.21.15.26:0/3603547579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4fc06f9d0 msgr2=0x7fc4fc0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 --2- 172.21.15.26:0/3603547579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4fc06f9d0 0x7fc4fc0ceff0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009870 tx=0x7fc4e401c510 comp rx=0 tx=0).stop 2023-12-19T17:58:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 -- 172.21.15.26:0/3603547579 shutdown_connections 2023-12-19T17:58:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 --2- 172.21.15.26:0/3603547579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4fc06f9d0 0x7fc4fc0ceff0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 -- 172.21.15.26:0/3603547579 >> 172.21.15.26:0/3603547579 conn(0x7fc4fc06d5d0 msgr2=0x7fc4fc069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 -- 172.21.15.26:0/3603547579 shutdown_connections 2023-12-19T17:58:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 -- 172.21.15.26:0/3603547579 wait complete. 2023-12-19T17:58:17.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 Processor -- start 2023-12-19T17:58:17.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 -- start start 2023-12-19T17:58:17.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4fc06f9d0 0x7fc4fc14dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4fc0c4070 0x7fc4fc0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4fc15a7c0 0x7fc4fc14f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc15e090 con 0x7fc4fc0c4070 2023-12-19T17:58:17.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc0b4280 con 0x7fc4fc06f9d0 2023-12-19T17:58:17.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.782+0000 7fc50196d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc0b4560 con 0x7fc4fc15a7c0 2023-12-19T17:58:17.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4fc0c4070 0x7fc4fc0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4fc0c4070 0x7fc4fc0b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38742/0 (socket says 172.21.15.26:38742) 2023-12-19T17:58:17.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4f3fff640 1 -- 172.21.15.26:0/303540639 learned_addr learned my addr 172.21.15.26:0/303540639 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:17.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4faffd640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4fc06f9d0 0x7fc4fc14dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4fb7fe640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4fc15a7c0 0x7fc4fc14f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4f3fff640 1 -- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4fc15a7c0 msgr2=0x7fc4fc14f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4f3fff640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4fc15a7c0 0x7fc4fc14f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4f3fff640 1 -- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4fc06f9d0 msgr2=0x7fc4fc14dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4f3fff640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4fc06f9d0 0x7fc4fc14dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4f3fff640 1 -- 172.21.15.26:0/303540639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e40094d0 con 0x7fc4fc0c4070 2023-12-19T17:58:17.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4fb7fe640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4fc15a7c0 0x7fc4fc14f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4f3fff640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4fc0c4070 0x7fc4fc0b2200 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4019120 tx=0x7fc4e401ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4f8ff9640 1 -- 172.21.15.26:0/303540639 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40be3f0 con 0x7fc4fc0c4070 2023-12-19T17:58:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4f8ff9640 1 -- 172.21.15.26:0/303540639 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e40be550 con 0x7fc4fc0c4070 2023-12-19T17:58:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4f8ff9640 1 -- 172.21.15.26:0/303540639 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40be900 con 0x7fc4fc0c4070 2023-12-19T17:58:17.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc50196d640 1 -- 172.21.15.26:0/303540639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4fc0b4840 con 0x7fc4fc0c4070 2023-12-19T17:58:17.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc50196d640 1 -- 172.21.15.26:0/303540639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4fc0b2740 con 0x7fc4fc0c4070 2023-12-19T17:58:17.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc50196d640 1 -- 172.21.15.26:0/303540639 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4fc15e200 con 0x7fc4fc0c4070 2023-12-19T17:58:17.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.786+0000 7fc4f8ff9640 1 -- 172.21.15.26:0/303540639 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc4e40b43f0 con 0x7fc4fc0c4070 2023-12-19T17:58:17.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.790+0000 7fc4f8ff9640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc4dc065bb0 0x7fc4dc068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:17.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.790+0000 7fc4f8ff9640 1 -- 172.21.15.26:0/303540639 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fc4e411d9b0 con 0x7fc4fc0c4070 2023-12-19T17:58:17.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.790+0000 7fc4faffd640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc4dc065bb0 0x7fc4dc068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:17.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.790+0000 7fc4faffd640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc4dc065bb0 0x7fc4dc068070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc151ee0 tx=0x7fc4ec009040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.794+0000 7fc4f8ff9640 1 -- 172.21.15.26:0/303540639 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc4fc15e200 con 0x7fc4fc0c4070 2023-12-19T17:58:17.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.950+0000 7fc50196d640 1 -- 172.21.15.26:0/303540639 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc4fc16acd0 con 0x7fc4dc065bb0 2023-12-19T17:58:17.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.958+0000 7fc4f8ff9640 1 -- 172.21.15.26:0/303540639 <== mgr.8821 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289968 (secure 0 0 0) 0x7fc4ec001d70 con 0x7fc4dc065bb0 2023-12-19T17:58:17.962 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:17.964 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T17:58:17.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.958+0000 7fc50196d640 1 -- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc4dc065bb0 msgr2=0x7fc4dc068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.958+0000 7fc50196d640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc4dc065bb0 0x7fc4dc068070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc151ee0 tx=0x7fc4ec009040 comp rx=0 tx=0).stop 2023-12-19T17:58:17.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.958+0000 7fc50196d640 1 -- 172.21.15.26:0/303540639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4fc0c4070 msgr2=0x7fc4fc0b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:17.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.958+0000 7fc50196d640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4fc0c4070 0x7fc4fc0b2200 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4019120 tx=0x7fc4e401ad50 comp rx=0 tx=0).stop 2023-12-19T17:58:17.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.958+0000 7fc50196d640 1 -- 172.21.15.26:0/303540639 shutdown_connections 2023-12-19T17:58:17.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.958+0000 7fc50196d640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fc4dc065bb0 0x7fc4dc068070 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.958+0000 7fc50196d640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc4fc15a7c0 0x7fc4fc14f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.958+0000 7fc50196d640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4fc0c4070 0x7fc4fc0b2200 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.958+0000 7fc50196d640 1 --2- 172.21.15.26:0/303540639 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4fc06f9d0 0x7fc4fc14dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:17.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.958+0000 7fc50196d640 1 -- 172.21.15.26:0/303540639 >> 172.21.15.26:0/303540639 conn(0x7fc4fc06d5d0 msgr2=0x7fc4fc0708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:17.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.962+0000 7fc50196d640 1 -- 172.21.15.26:0/303540639 shutdown_connections 2023-12-19T17:58:17.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:17.962+0000 7fc50196d640 1 -- 172.21.15.26:0/303540639 wait complete. 2023-12-19T17:58:17.983 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-19T17:58:17.593775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1634,"num_read_kb":3381,"num_write":520,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":330,"ondisk_log_size":330,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":432400,"kb_used_data":18384,"kb_used_omap":0,"kb_used_meta":413952,"kb_avail":754542320,"statfs":{"total":773094113280,"available":772651335680,"internally_reserved":0,"allocated":18825216,"data_stored":10360420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":423886848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.537335"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485017+0000","last_change":"2023-12-19T17:58:13.511775+0000","last_active":"2023-12-19T17:58:14.485017+0000","last_peered":"2023-12-19T17:58:14.485017+0000","last_clean":"2023-12-19T17:58:14.485017+0000","last_became_active":"2023-12-19T17:58:13.511470+0000","last_became_peered":"2023-12-19T17:58:13.511470+0000","last_unstale":"2023-12-19T17:58:14.485017+0000","last_undegraded":"2023-12-19T17:58:14.485017+0000","last_fullsized":"2023-12-19T17:58:14.485017+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:00.402828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914870+0000","last_change":"2023-12-19T17:58:13.500992+0000","last_active":"2023-12-19T17:58:14.914870+0000","last_peered":"2023-12-19T17:58:14.914870+0000","last_clean":"2023-12-19T17:58:14.914870+0000","last_became_active":"2023-12-19T17:58:13.500743+0000","last_became_peered":"2023-12-19T17:58:13.500743+0000","last_unstale":"2023-12-19T17:58:14.914870+0000","last_undegraded":"2023-12-19T17:58:14.914870+0000","last_fullsized":"2023-12-19T17:58:14.914870+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:59:51.315329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486866+0000","last_change":"2023-12-19T17:58:13.502221+0000","last_active":"2023-12-19T17:58:14.486866+0000","last_peered":"2023-12-19T17:58:14.486866+0000","last_clean":"2023-12-19T17:58:14.486866+0000","last_became_active":"2023-12-19T17:58:13.501953+0000","last_became_peered":"2023-12-19T17:58:13.501953+0000","last_unstale":"2023-12-19T17:58:14.486866+0000","last_undegraded":"2023-12-19T17:58:14.486866+0000","last_fullsized":"2023-12-19T17:58:14.486866+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:41:39.880180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915551+0000","last_change":"2023-12-19T17:58:13.499008+0000","last_active":"2023-12-19T17:58:14.915551+0000","last_peered":"2023-12-19T17:58:14.915551+0000","last_clean":"2023-12-19T17:58:14.915551+0000","last_became_active":"2023-12-19T17:58:13.498669+0000","last_became_peered":"2023-12-19T17:58:13.498669+0000","last_unstale":"2023-12-19T17:58:14.915551+0000","last_undegraded":"2023-12-19T17:58:14.915551+0000","last_fullsized":"2023-12-19T17:58:14.915551+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:36:08.870192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486004+0000","last_change":"2023-12-19T17:58:13.508329+0000","last_active":"2023-12-19T17:58:14.486004+0000","last_peered":"2023-12-19T17:58:14.486004+0000","last_clean":"2023-12-19T17:58:14.486004+0000","last_became_active":"2023-12-19T17:58:13.508047+0000","last_became_peered":"2023-12-19T17:58:13.508047+0000","last_unstale":"2023-12-19T17:58:14.486004+0000","last_undegraded":"2023-12-19T17:58:14.486004+0000","last_fullsized":"2023-12-19T17:58:14.486004+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:46.206669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484457+0000","last_change":"2023-12-19T17:58:13.499918+0000","last_active":"2023-12-19T17:58:14.484457+0000","last_peered":"2023-12-19T17:58:14.484457+0000","last_clean":"2023-12-19T17:58:14.484457+0000","last_became_active":"2023-12-19T17:58:13.499655+0000","last_became_peered":"2023-12-19T17:58:13.499655+0000","last_unstale":"2023-12-19T17:58:14.484457+0000","last_undegraded":"2023-12-19T17:58:14.484457+0000","last_fullsized":"2023-12-19T17:58:14.484457+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:49.808864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914785+0000","last_change":"2023-12-19T17:58:13.486291+0000","last_active":"2023-12-19T17:58:14.914785+0000","last_peered":"2023-12-19T17:58:14.914785+0000","last_clean":"2023-12-19T17:58:14.914785+0000","last_became_active":"2023-12-19T17:58:13.486025+0000","last_became_peered":"2023-12-19T17:58:13.486025+0000","last_unstale":"2023-12-19T17:58:14.914785+0000","last_undegraded":"2023-12-19T17:58:14.914785+0000","last_fullsized":"2023-12-19T17:58:14.914785+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:11:13.862544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484294+0000","last_change":"2023-12-19T17:58:13.491479+0000","last_active":"2023-12-19T17:58:14.484294+0000","last_peered":"2023-12-19T17:58:14.484294+0000","last_clean":"2023-12-19T17:58:14.484294+0000","last_became_active":"2023-12-19T17:58:13.491216+0000","last_became_peered":"2023-12-19T17:58:13.491216+0000","last_unstale":"2023-12-19T17:58:14.484294+0000","last_undegraded":"2023-12-19T17:58:14.484294+0000","last_fullsized":"2023-12-19T17:58:14.484294+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:16:31.307309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914936+0000","last_change":"2023-12-19T17:58:13.504084+0000","last_active":"2023-12-19T17:58:14.914936+0000","last_peered":"2023-12-19T17:58:14.914936+0000","last_clean":"2023-12-19T17:58:14.914936+0000","last_became_active":"2023-12-19T17:58:13.503836+0000","last_became_peered":"2023-12-19T17:58:13.503836+0000","last_unstale":"2023-12-19T17:58:14.914936+0000","last_undegraded":"2023-12-19T17:58:14.914936+0000","last_fullsized":"2023-12-19T17:58:14.914936+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:27:03.254793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914945+0000","last_change":"2023-12-19T17:58:13.507494+0000","last_active":"2023-12-19T17:58:14.914945+0000","last_peered":"2023-12-19T17:58:14.914945+0000","last_clean":"2023-12-19T17:58:14.914945+0000","last_became_active":"2023-12-19T17:58:13.507201+0000","last_became_peered":"2023-12-19T17:58:13.507201+0000","last_unstale":"2023-12-19T17:58:14.914945+0000","last_undegraded":"2023-12-19T17:58:14.914945+0000","last_fullsized":"2023-12-19T17:58:14.914945+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:17.870336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484601+0000","last_change":"2023-12-19T17:58:13.494898+0000","last_active":"2023-12-19T17:58:14.484601+0000","last_peered":"2023-12-19T17:58:14.484601+0000","last_clean":"2023-12-19T17:58:14.484601+0000","last_became_active":"2023-12-19T17:58:13.494614+0000","last_became_peered":"2023-12-19T17:58:13.494614+0000","last_unstale":"2023-12-19T17:58:14.484601+0000","last_undegraded":"2023-12-19T17:58:14.484601+0000","last_fullsized":"2023-12-19T17:58:14.484601+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:25.449341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914567+0000","last_change":"2023-12-19T17:58:13.510743+0000","last_active":"2023-12-19T17:58:14.914567+0000","last_peered":"2023-12-19T17:58:14.914567+0000","last_clean":"2023-12-19T17:58:14.914567+0000","last_became_active":"2023-12-19T17:58:13.510477+0000","last_became_peered":"2023-12-19T17:58:13.510477+0000","last_unstale":"2023-12-19T17:58:14.914567+0000","last_undegraded":"2023-12-19T17:58:14.914567+0000","last_fullsized":"2023-12-19T17:58:14.914567+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:14.799735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486237+0000","last_change":"2023-12-19T17:58:13.508677+0000","last_active":"2023-12-19T17:58:14.486237+0000","last_peered":"2023-12-19T17:58:14.486237+0000","last_clean":"2023-12-19T17:58:14.486237+0000","last_became_active":"2023-12-19T17:58:13.508394+0000","last_became_peered":"2023-12-19T17:58:13.508394+0000","last_unstale":"2023-12-19T17:58:14.486237+0000","last_undegraded":"2023-12-19T17:58:14.486237+0000","last_fullsized":"2023-12-19T17:58:14.486237+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:50:14.899624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914906+0000","last_change":"2023-12-19T17:58:13.499430+0000","last_active":"2023-12-19T17:58:14.914906+0000","last_peered":"2023-12-19T17:58:14.914906+0000","last_clean":"2023-12-19T17:58:14.914906+0000","last_became_active":"2023-12-19T17:58:13.499178+0000","last_became_peered":"2023-12-19T17:58:13.499178+0000","last_unstale":"2023-12-19T17:58:14.914906+0000","last_undegraded":"2023-12-19T17:58:14.914906+0000","last_fullsized":"2023-12-19T17:58:14.914906+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:34:16.126406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914871+0000","last_change":"2023-12-19T17:58:13.498555+0000","last_active":"2023-12-19T17:58:14.914871+0000","last_peered":"2023-12-19T17:58:14.914871+0000","last_clean":"2023-12-19T17:58:14.914871+0000","last_became_active":"2023-12-19T17:58:13.498269+0000","last_became_peered":"2023-12-19T17:58:13.498269+0000","last_unstale":"2023-12-19T17:58:14.914871+0000","last_undegraded":"2023-12-19T17:58:14.914871+0000","last_fullsized":"2023-12-19T17:58:14.914871+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:29.304411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914435+0000","last_change":"2023-12-19T17:58:13.504020+0000","last_active":"2023-12-19T17:58:14.914435+0000","last_peered":"2023-12-19T17:58:14.914435+0000","last_clean":"2023-12-19T17:58:14.914435+0000","last_became_active":"2023-12-19T17:58:13.503745+0000","last_became_peered":"2023-12-19T17:58:13.503745+0000","last_unstale":"2023-12-19T17:58:14.914435+0000","last_undegraded":"2023-12-19T17:58:14.914435+0000","last_fullsized":"2023-12-19T17:58:14.914435+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:50:14.899624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915285+0000","last_change":"2023-12-19T17:58:13.511251+0000","last_active":"2023-12-19T17:58:14.915285+0000","last_peered":"2023-12-19T17:58:14.915285+0000","last_clean":"2023-12-19T17:58:14.915285+0000","last_became_active":"2023-12-19T17:58:13.508979+0000","last_became_peered":"2023-12-19T17:58:13.508979+0000","last_unstale":"2023-12-19T17:58:14.915285+0000","last_undegraded":"2023-12-19T17:58:14.915285+0000","last_fullsized":"2023-12-19T17:58:14.915285+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:26:02.125211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486947+0000","last_change":"2023-12-19T17:58:13.502151+0000","last_active":"2023-12-19T17:58:14.486947+0000","last_peered":"2023-12-19T17:58:14.486947+0000","last_clean":"2023-12-19T17:58:14.486947+0000","last_became_active":"2023-12-19T17:58:13.501886+0000","last_became_peered":"2023-12-19T17:58:13.501886+0000","last_unstale":"2023-12-19T17:58:14.486947+0000","last_undegraded":"2023-12-19T17:58:14.486947+0000","last_fullsized":"2023-12-19T17:58:14.486947+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:34:38.046676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484716+0000","last_change":"2023-12-19T17:58:13.497314+0000","last_active":"2023-12-19T17:58:14.484716+0000","last_peered":"2023-12-19T17:58:14.484716+0000","last_clean":"2023-12-19T17:58:14.484716+0000","last_became_active":"2023-12-19T17:58:13.496934+0000","last_became_peered":"2023-12-19T17:58:13.496934+0000","last_unstale":"2023-12-19T17:58:14.484716+0000","last_undegraded":"2023-12-19T17:58:14.484716+0000","last_fullsized":"2023-12-19T17:58:14.484716+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:05:27.227183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914731+0000","last_change":"2023-12-19T17:58:13.494217+0000","last_active":"2023-12-19T17:58:14.914731+0000","last_peered":"2023-12-19T17:58:14.914731+0000","last_clean":"2023-12-19T17:58:14.914731+0000","last_became_active":"2023-12-19T17:58:13.493950+0000","last_became_peered":"2023-12-19T17:58:13.493950+0000","last_unstale":"2023-12-19T17:58:14.914731+0000","last_undegraded":"2023-12-19T17:58:14.914731+0000","last_fullsized":"2023-12-19T17:58:14.914731+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:35:13.994534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914893+0000","last_change":"2023-12-19T17:58:13.505180+0000","last_active":"2023-12-19T17:58:14.914893+0000","last_peered":"2023-12-19T17:58:14.914893+0000","last_clean":"2023-12-19T17:58:14.914893+0000","last_became_active":"2023-12-19T17:58:13.504854+0000","last_became_peered":"2023-12-19T17:58:13.504854+0000","last_unstale":"2023-12-19T17:58:14.914893+0000","last_undegraded":"2023-12-19T17:58:14.914893+0000","last_fullsized":"2023-12-19T17:58:14.914893+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:43:23.041550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914962+0000","last_change":"2023-12-19T17:58:13.493827+0000","last_active":"2023-12-19T17:58:14.914962+0000","last_peered":"2023-12-19T17:58:14.914962+0000","last_clean":"2023-12-19T17:58:14.914962+0000","last_became_active":"2023-12-19T17:58:13.493549+0000","last_became_peered":"2023-12-19T17:58:13.493549+0000","last_unstale":"2023-12-19T17:58:14.914962+0000","last_undegraded":"2023-12-19T17:58:14.914962+0000","last_fullsized":"2023-12-19T17:58:14.914962+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:34:38.046676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914741+0000","last_change":"2023-12-19T17:58:13.495208+0000","last_active":"2023-12-19T17:58:14.914741+0000","last_peered":"2023-12-19T17:58:14.914741+0000","last_clean":"2023-12-19T17:58:14.914741+0000","last_became_active":"2023-12-19T17:58:13.494771+0000","last_became_peered":"2023-12-19T17:58:13.494771+0000","last_unstale":"2023-12-19T17:58:14.914741+0000","last_undegraded":"2023-12-19T17:58:14.914741+0000","last_fullsized":"2023-12-19T17:58:14.914741+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:40:18.048253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914454+0000","last_change":"2023-12-19T17:58:13.504433+0000","last_active":"2023-12-19T17:58:14.914454+0000","last_peered":"2023-12-19T17:58:14.914454+0000","last_clean":"2023-12-19T17:58:14.914454+0000","last_became_active":"2023-12-19T17:58:13.504113+0000","last_became_peered":"2023-12-19T17:58:13.504113+0000","last_unstale":"2023-12-19T17:58:14.914454+0000","last_undegraded":"2023-12-19T17:58:14.914454+0000","last_fullsized":"2023-12-19T17:58:14.914454+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:40:29.577603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915277+0000","last_change":"2023-12-19T17:58:13.508323+0000","last_active":"2023-12-19T17:58:14.915277+0000","last_peered":"2023-12-19T17:58:14.915277+0000","last_clean":"2023-12-19T17:58:14.915277+0000","last_became_active":"2023-12-19T17:58:13.507584+0000","last_became_peered":"2023-12-19T17:58:13.507584+0000","last_unstale":"2023-12-19T17:58:14.915277+0000","last_undegraded":"2023-12-19T17:58:14.915277+0000","last_fullsized":"2023-12-19T17:58:14.915277+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:10:32.458962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484711+0000","last_change":"2023-12-19T17:58:13.497157+0000","last_active":"2023-12-19T17:58:14.484711+0000","last_peered":"2023-12-19T17:58:14.484711+0000","last_clean":"2023-12-19T17:58:14.484711+0000","last_became_active":"2023-12-19T17:58:13.496873+0000","last_became_peered":"2023-12-19T17:58:13.496873+0000","last_unstale":"2023-12-19T17:58:14.484711+0000","last_undegraded":"2023-12-19T17:58:14.484711+0000","last_fullsized":"2023-12-19T17:58:14.484711+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:14.799735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914642+0000","last_change":"2023-12-19T17:58:13.498578+0000","last_active":"2023-12-19T17:58:14.914642+0000","last_peered":"2023-12-19T17:58:14.914642+0000","last_clean":"2023-12-19T17:58:14.914642+0000","last_became_active":"2023-12-19T17:58:13.498322+0000","last_became_peered":"2023-12-19T17:58:13.498322+0000","last_unstale":"2023-12-19T17:58:14.914642+0000","last_undegraded":"2023-12-19T17:58:14.914642+0000","last_fullsized":"2023-12-19T17:58:14.914642+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:39.351617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915364+0000","last_change":"2023-12-19T17:58:13.501483+0000","last_active":"2023-12-19T17:58:14.915364+0000","last_peered":"2023-12-19T17:58:14.915364+0000","last_clean":"2023-12-19T17:58:14.915364+0000","last_became_active":"2023-12-19T17:58:13.499848+0000","last_became_peered":"2023-12-19T17:58:13.499848+0000","last_unstale":"2023-12-19T17:58:14.915364+0000","last_undegraded":"2023-12-19T17:58:14.915364+0000","last_fullsized":"2023-12-19T17:58:14.915364+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:29.059098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484486+0000","last_change":"2023-12-19T17:58:13.500787+0000","last_active":"2023-12-19T17:58:14.484486+0000","last_peered":"2023-12-19T17:58:14.484486+0000","last_clean":"2023-12-19T17:58:14.484486+0000","last_became_active":"2023-12-19T17:58:13.500302+0000","last_became_peered":"2023-12-19T17:58:13.500302+0000","last_unstale":"2023-12-19T17:58:14.484486+0000","last_undegraded":"2023-12-19T17:58:14.484486+0000","last_fullsized":"2023-12-19T17:58:14.484486+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:26:02.125211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915023+0000","last_change":"2023-12-19T17:58:13.496813+0000","last_active":"2023-12-19T17:58:14.915023+0000","last_peered":"2023-12-19T17:58:14.915023+0000","last_clean":"2023-12-19T17:58:14.915023+0000","last_became_active":"2023-12-19T17:58:13.495857+0000","last_became_peered":"2023-12-19T17:58:13.495857+0000","last_unstale":"2023-12-19T17:58:14.915023+0000","last_undegraded":"2023-12-19T17:58:14.915023+0000","last_fullsized":"2023-12-19T17:58:14.915023+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:41:39.880180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914087+0000","last_change":"2023-12-19T17:58:13.500389+0000","last_active":"2023-12-19T17:58:14.914087+0000","last_peered":"2023-12-19T17:58:14.914087+0000","last_clean":"2023-12-19T17:58:14.914087+0000","last_became_active":"2023-12-19T17:58:13.499316+0000","last_became_peered":"2023-12-19T17:58:13.499316+0000","last_unstale":"2023-12-19T17:58:14.914087+0000","last_undegraded":"2023-12-19T17:58:14.914087+0000","last_fullsized":"2023-12-19T17:58:14.914087+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:14.799735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914739+0000","last_change":"2023-12-19T17:58:13.502787+0000","last_active":"2023-12-19T17:58:14.914739+0000","last_peered":"2023-12-19T17:58:14.914739+0000","last_clean":"2023-12-19T17:58:14.914739+0000","last_became_active":"2023-12-19T17:58:13.502515+0000","last_became_peered":"2023-12-19T17:58:13.502515+0000","last_unstale":"2023-12-19T17:58:14.914739+0000","last_undegraded":"2023-12-19T17:58:14.914739+0000","last_fullsized":"2023-12-19T17:58:14.914739+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:19:06.534141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915064+0000","last_change":"2023-12-19T17:58:13.509487+0000","last_active":"2023-12-19T17:58:14.915064+0000","last_peered":"2023-12-19T17:58:14.915064+0000","last_clean":"2023-12-19T17:58:14.915064+0000","last_became_active":"2023-12-19T17:58:13.508677+0000","last_became_peered":"2023-12-19T17:58:13.508677+0000","last_unstale":"2023-12-19T17:58:14.915064+0000","last_undegraded":"2023-12-19T17:58:14.915064+0000","last_fullsized":"2023-12-19T17:58:14.915064+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:26.054027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"176'4","reported_seq":15,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915142+0000","last_change":"2023-12-19T17:58:13.092896+0000","last_active":"2023-12-19T17:58:14.915142+0000","last_peered":"2023-12-19T17:58:14.915142+0000","last_clean":"2023-12-19T17:58:14.915142+0000","last_became_active":"2023-12-19T17:58:13.092330+0000","last_became_peered":"2023-12-19T17:58:13.092330+0000","last_unstale":"2023-12-19T17:58:14.915142+0000","last_undegraded":"2023-12-19T17:58:14.915142+0000","last_fullsized":"2023-12-19T17:58:14.915142+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:26:35.026082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914791+0000","last_change":"2023-12-19T17:58:13.503555+0000","last_active":"2023-12-19T17:58:14.914791+0000","last_peered":"2023-12-19T17:58:14.914791+0000","last_clean":"2023-12-19T17:58:14.914791+0000","last_became_active":"2023-12-19T17:58:13.501132+0000","last_became_peered":"2023-12-19T17:58:13.501132+0000","last_unstale":"2023-12-19T17:58:14.914791+0000","last_undegraded":"2023-12-19T17:58:14.914791+0000","last_fullsized":"2023-12-19T17:58:14.914791+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:40:29.577603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"176'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914658+0000","last_change":"2023-12-19T17:58:13.092702+0000","last_active":"2023-12-19T17:58:14.914658+0000","last_peered":"2023-12-19T17:58:14.914658+0000","last_clean":"2023-12-19T17:58:14.914658+0000","last_became_active":"2023-12-19T17:58:13.092347+0000","last_became_peered":"2023-12-19T17:58:13.092347+0000","last_unstale":"2023-12-19T17:58:14.914658+0000","last_undegraded":"2023-12-19T17:58:14.914658+0000","last_fullsized":"2023-12-19T17:58:14.914658+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:07.798146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484519+0000","last_change":"2023-12-19T17:58:13.490769+0000","last_active":"2023-12-19T17:58:14.484519+0000","last_peered":"2023-12-19T17:58:14.484519+0000","last_clean":"2023-12-19T17:58:14.484519+0000","last_became_active":"2023-12-19T17:58:13.490505+0000","last_became_peered":"2023-12-19T17:58:13.490505+0000","last_unstale":"2023-12-19T17:58:14.484519+0000","last_undegraded":"2023-12-19T17:58:14.484519+0000","last_fullsized":"2023-12-19T17:58:14.484519+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:43:57.132764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"176'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915514+0000","last_change":"2023-12-19T17:58:13.082905+0000","last_active":"2023-12-19T17:58:14.915514+0000","last_peered":"2023-12-19T17:58:14.915514+0000","last_clean":"2023-12-19T17:58:14.915514+0000","last_became_active":"2023-12-19T17:58:13.081254+0000","last_became_peered":"2023-12-19T17:58:13.081254+0000","last_unstale":"2023-12-19T17:58:14.915514+0000","last_undegraded":"2023-12-19T17:58:14.915514+0000","last_fullsized":"2023-12-19T17:58:14.915514+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:43:56.664402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914918+0000","last_change":"2023-12-19T17:58:13.510286+0000","last_active":"2023-12-19T17:58:14.914918+0000","last_peered":"2023-12-19T17:58:14.914918+0000","last_clean":"2023-12-19T17:58:14.914918+0000","last_became_active":"2023-12-19T17:58:13.509123+0000","last_became_peered":"2023-12-19T17:58:13.509123+0000","last_unstale":"2023-12-19T17:58:14.914918+0000","last_undegraded":"2023-12-19T17:58:14.914918+0000","last_fullsized":"2023-12-19T17:58:14.914918+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:05:27.227183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"176'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915597+0000","last_change":"2023-12-19T17:58:13.089210+0000","last_active":"2023-12-19T17:58:14.915597+0000","last_peered":"2023-12-19T17:58:14.915597+0000","last_clean":"2023-12-19T17:58:14.915597+0000","last_became_active":"2023-12-19T17:58:13.088232+0000","last_became_peered":"2023-12-19T17:58:13.088232+0000","last_unstale":"2023-12-19T17:58:14.915597+0000","last_undegraded":"2023-12-19T17:58:14.915597+0000","last_fullsized":"2023-12-19T17:58:14.915597+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:17.813715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914244+0000","last_change":"2023-12-19T17:58:13.498269+0000","last_active":"2023-12-19T17:58:14.914244+0000","last_peered":"2023-12-19T17:58:14.914244+0000","last_clean":"2023-12-19T17:58:14.914244+0000","last_became_active":"2023-12-19T17:58:13.498005+0000","last_became_peered":"2023-12-19T17:58:13.498005+0000","last_unstale":"2023-12-19T17:58:14.914244+0000","last_undegraded":"2023-12-19T17:58:14.914244+0000","last_fullsized":"2023-12-19T17:58:14.914244+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:05:27.227183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"176'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914649+0000","last_change":"2023-12-19T17:58:13.093194+0000","last_active":"2023-12-19T17:58:14.914649+0000","last_peered":"2023-12-19T17:58:14.914649+0000","last_clean":"2023-12-19T17:58:14.914649+0000","last_became_active":"2023-12-19T17:58:13.092476+0000","last_became_peered":"2023-12-19T17:58:13.092476+0000","last_unstale":"2023-12-19T17:58:14.914649+0000","last_undegraded":"2023-12-19T17:58:14.914649+0000","last_fullsized":"2023-12-19T17:58:14.914649+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:19:22.849330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915188+0000","last_change":"2023-12-19T17:58:13.496911+0000","last_active":"2023-12-19T17:58:14.915188+0000","last_peered":"2023-12-19T17:58:14.915188+0000","last_clean":"2023-12-19T17:58:14.915188+0000","last_became_active":"2023-12-19T17:58:13.496635+0000","last_became_peered":"2023-12-19T17:58:13.496635+0000","last_unstale":"2023-12-19T17:58:14.915188+0000","last_undegraded":"2023-12-19T17:58:14.915188+0000","last_fullsized":"2023-12-19T17:58:14.915188+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:33:09.805442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"176'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485842+0000","last_change":"2023-12-19T17:58:13.089476+0000","last_active":"2023-12-19T17:58:14.485842+0000","last_peered":"2023-12-19T17:58:14.485842+0000","last_clean":"2023-12-19T17:58:14.485842+0000","last_became_active":"2023-12-19T17:58:13.089242+0000","last_became_peered":"2023-12-19T17:58:13.089242+0000","last_unstale":"2023-12-19T17:58:14.485842+0000","last_undegraded":"2023-12-19T17:58:14.485842+0000","last_fullsized":"2023-12-19T17:58:14.485842+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:55.995033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914060+0000","last_change":"2023-12-19T17:58:13.497322+0000","last_active":"2023-12-19T17:58:14.914060+0000","last_peered":"2023-12-19T17:58:14.914060+0000","last_clean":"2023-12-19T17:58:14.914060+0000","last_became_active":"2023-12-19T17:58:13.497048+0000","last_became_peered":"2023-12-19T17:58:13.497048+0000","last_unstale":"2023-12-19T17:58:14.914060+0000","last_undegraded":"2023-12-19T17:58:14.914060+0000","last_fullsized":"2023-12-19T17:58:14.914060+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:26.054027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914383+0000","last_change":"2023-12-19T17:58:13.089925+0000","last_active":"2023-12-19T17:58:14.914383+0000","last_peered":"2023-12-19T17:58:14.914383+0000","last_clean":"2023-12-19T17:58:14.914383+0000","last_became_active":"2023-12-19T17:58:13.088473+0000","last_became_peered":"2023-12-19T17:58:13.088473+0000","last_unstale":"2023-12-19T17:58:14.914383+0000","last_undegraded":"2023-12-19T17:58:14.914383+0000","last_fullsized":"2023-12-19T17:58:14.914383+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:00:04.341736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485736+0000","last_change":"2023-12-19T17:58:13.497977+0000","last_active":"2023-12-19T17:58:14.485736+0000","last_peered":"2023-12-19T17:58:14.485736+0000","last_clean":"2023-12-19T17:58:14.485736+0000","last_became_active":"2023-12-19T17:58:13.497627+0000","last_became_peered":"2023-12-19T17:58:13.497627+0000","last_unstale":"2023-12-19T17:58:14.485736+0000","last_undegraded":"2023-12-19T17:58:14.485736+0000","last_fullsized":"2023-12-19T17:58:14.485736+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:45:42.444705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914512+0000","last_change":"2023-12-19T17:58:13.089636+0000","last_active":"2023-12-19T17:58:14.914512+0000","last_peered":"2023-12-19T17:58:14.914512+0000","last_clean":"2023-12-19T17:58:14.914512+0000","last_became_active":"2023-12-19T17:58:13.089328+0000","last_became_peered":"2023-12-19T17:58:13.089328+0000","last_unstale":"2023-12-19T17:58:14.914512+0000","last_undegraded":"2023-12-19T17:58:14.914512+0000","last_fullsized":"2023-12-19T17:58:14.914512+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:02.104297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486094+0000","last_change":"2023-12-19T17:58:13.496146+0000","last_active":"2023-12-19T17:58:14.486094+0000","last_peered":"2023-12-19T17:58:14.486094+0000","last_clean":"2023-12-19T17:58:14.486094+0000","last_became_active":"2023-12-19T17:58:13.495862+0000","last_became_peered":"2023-12-19T17:58:13.495862+0000","last_unstale":"2023-12-19T17:58:14.486094+0000","last_undegraded":"2023-12-19T17:58:14.486094+0000","last_fullsized":"2023-12-19T17:58:14.486094+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:42:13.687152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914925+0000","last_change":"2023-12-19T17:58:13.091763+0000","last_active":"2023-12-19T17:58:14.914925+0000","last_peered":"2023-12-19T17:58:14.914925+0000","last_clean":"2023-12-19T17:58:14.914925+0000","last_became_active":"2023-12-19T17:58:13.091449+0000","last_became_peered":"2023-12-19T17:58:13.091449+0000","last_unstale":"2023-12-19T17:58:14.914925+0000","last_undegraded":"2023-12-19T17:58:14.914925+0000","last_fullsized":"2023-12-19T17:58:14.914925+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:19:32.748126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915182+0000","last_change":"2023-12-19T17:58:13.509781+0000","last_active":"2023-12-19T17:58:14.915182+0000","last_peered":"2023-12-19T17:58:14.915182+0000","last_clean":"2023-12-19T17:58:14.915182+0000","last_became_active":"2023-12-19T17:58:13.508650+0000","last_became_peered":"2023-12-19T17:58:13.508650+0000","last_unstale":"2023-12-19T17:58:14.915182+0000","last_undegraded":"2023-12-19T17:58:14.915182+0000","last_fullsized":"2023-12-19T17:58:14.915182+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:00.402828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915181+0000","last_change":"2023-12-19T17:58:13.092307+0000","last_active":"2023-12-19T17:58:14.915181+0000","last_peered":"2023-12-19T17:58:14.915181+0000","last_clean":"2023-12-19T17:58:14.915181+0000","last_became_active":"2023-12-19T17:58:13.091998+0000","last_became_peered":"2023-12-19T17:58:13.091998+0000","last_unstale":"2023-12-19T17:58:14.915181+0000","last_undegraded":"2023-12-19T17:58:14.915181+0000","last_fullsized":"2023-12-19T17:58:14.915181+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:47:09.951173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915077+0000","last_change":"2023-12-19T17:58:13.500738+0000","last_active":"2023-12-19T17:58:14.915077+0000","last_peered":"2023-12-19T17:58:14.915077+0000","last_clean":"2023-12-19T17:58:14.915077+0000","last_became_active":"2023-12-19T17:58:13.500508+0000","last_became_peered":"2023-12-19T17:58:13.500508+0000","last_unstale":"2023-12-19T17:58:14.915077+0000","last_undegraded":"2023-12-19T17:58:14.915077+0000","last_fullsized":"2023-12-19T17:58:14.915077+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:45:42.444705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"176'6","reported_seq":17,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915456+0000","last_change":"2023-12-19T17:58:13.089843+0000","last_active":"2023-12-19T17:58:14.915456+0000","last_peered":"2023-12-19T17:58:14.915456+0000","last_clean":"2023-12-19T17:58:14.915456+0000","last_became_active":"2023-12-19T17:58:13.089523+0000","last_became_peered":"2023-12-19T17:58:13.089523+0000","last_unstale":"2023-12-19T17:58:14.915456+0000","last_undegraded":"2023-12-19T17:58:14.915456+0000","last_fullsized":"2023-12-19T17:58:14.915456+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:05:43.382647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486477+0000","last_change":"2023-12-19T17:58:13.505531+0000","last_active":"2023-12-19T17:58:14.486477+0000","last_peered":"2023-12-19T17:58:14.486477+0000","last_clean":"2023-12-19T17:58:14.486477+0000","last_became_active":"2023-12-19T17:58:13.505202+0000","last_became_peered":"2023-12-19T17:58:13.505202+0000","last_unstale":"2023-12-19T17:58:14.486477+0000","last_undegraded":"2023-12-19T17:58:14.486477+0000","last_fullsized":"2023-12-19T17:58:14.486477+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:03:42.151305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"176'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486758+0000","last_change":"2023-12-19T17:58:13.086018+0000","last_active":"2023-12-19T17:58:14.486758+0000","last_peered":"2023-12-19T17:58:14.486758+0000","last_clean":"2023-12-19T17:58:14.486758+0000","last_became_active":"2023-12-19T17:58:13.085569+0000","last_became_peered":"2023-12-19T17:58:13.085569+0000","last_unstale":"2023-12-19T17:58:14.486758+0000","last_undegraded":"2023-12-19T17:58:14.486758+0000","last_fullsized":"2023-12-19T17:58:14.486758+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:18:46.972146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914275+0000","last_change":"2023-12-19T17:58:13.500672+0000","last_active":"2023-12-19T17:58:14.914275+0000","last_peered":"2023-12-19T17:58:14.914275+0000","last_clean":"2023-12-19T17:58:14.914275+0000","last_became_active":"2023-12-19T17:58:13.500425+0000","last_became_peered":"2023-12-19T17:58:13.500425+0000","last_unstale":"2023-12-19T17:58:14.914275+0000","last_undegraded":"2023-12-19T17:58:14.914275+0000","last_fullsized":"2023-12-19T17:58:14.914275+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:00.402828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486495+0000","last_change":"2023-12-19T17:58:13.087130+0000","last_active":"2023-12-19T17:58:14.486495+0000","last_peered":"2023-12-19T17:58:14.486495+0000","last_clean":"2023-12-19T17:58:14.486495+0000","last_became_active":"2023-12-19T17:58:13.086841+0000","last_became_peered":"2023-12-19T17:58:13.086841+0000","last_unstale":"2023-12-19T17:58:14.486495+0000","last_undegraded":"2023-12-19T17:58:14.486495+0000","last_fullsized":"2023-12-19T17:58:14.486495+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:02.104297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485936+0000","last_change":"2023-12-19T17:58:13.499626+0000","last_active":"2023-12-19T17:58:14.485936+0000","last_peered":"2023-12-19T17:58:14.485936+0000","last_clean":"2023-12-19T17:58:14.485936+0000","last_became_active":"2023-12-19T17:58:13.499337+0000","last_became_peered":"2023-12-19T17:58:13.499337+0000","last_unstale":"2023-12-19T17:58:14.485936+0000","last_undegraded":"2023-12-19T17:58:14.485936+0000","last_fullsized":"2023-12-19T17:58:14.485936+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:26:35.242597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484381+0000","last_change":"2023-12-19T17:58:13.085345+0000","last_active":"2023-12-19T17:58:14.484381+0000","last_peered":"2023-12-19T17:58:14.484381+0000","last_clean":"2023-12-19T17:58:14.484381+0000","last_became_active":"2023-12-19T17:58:13.084999+0000","last_became_peered":"2023-12-19T17:58:13.084999+0000","last_unstale":"2023-12-19T17:58:14.484381+0000","last_undegraded":"2023-12-19T17:58:14.484381+0000","last_fullsized":"2023-12-19T17:58:14.484381+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:26:35.026082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485655+0000","last_change":"2023-12-19T17:58:13.492863+0000","last_active":"2023-12-19T17:58:14.485655+0000","last_peered":"2023-12-19T17:58:14.485655+0000","last_clean":"2023-12-19T17:58:14.485655+0000","last_became_active":"2023-12-19T17:58:13.492604+0000","last_became_peered":"2023-12-19T17:58:13.492604+0000","last_unstale":"2023-12-19T17:58:14.485655+0000","last_undegraded":"2023-12-19T17:58:14.485655+0000","last_fullsized":"2023-12-19T17:58:14.485655+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:40.235025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"176'8","reported_seq":19,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914638+0000","last_change":"2023-12-19T17:58:13.087821+0000","last_active":"2023-12-19T17:58:14.914638+0000","last_peered":"2023-12-19T17:58:14.914638+0000","last_clean":"2023-12-19T17:58:14.914638+0000","last_became_active":"2023-12-19T17:58:13.087532+0000","last_became_peered":"2023-12-19T17:58:13.087532+0000","last_unstale":"2023-12-19T17:58:14.914638+0000","last_undegraded":"2023-12-19T17:58:14.914638+0000","last_fullsized":"2023-12-19T17:58:14.914638+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:20:57.517770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915135+0000","last_change":"2023-12-19T17:58:13.505204+0000","last_active":"2023-12-19T17:58:14.915135+0000","last_peered":"2023-12-19T17:58:14.915135+0000","last_clean":"2023-12-19T17:58:14.915135+0000","last_became_active":"2023-12-19T17:58:13.504894+0000","last_became_peered":"2023-12-19T17:58:13.504894+0000","last_unstale":"2023-12-19T17:58:14.915135+0000","last_undegraded":"2023-12-19T17:58:14.915135+0000","last_fullsized":"2023-12-19T17:58:14.915135+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:49.808864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"176'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486319+0000","last_change":"2023-12-19T17:58:13.083200+0000","last_active":"2023-12-19T17:58:14.486319+0000","last_peered":"2023-12-19T17:58:14.486319+0000","last_clean":"2023-12-19T17:58:14.486319+0000","last_became_active":"2023-12-19T17:58:13.082938+0000","last_became_peered":"2023-12-19T17:58:13.082938+0000","last_unstale":"2023-12-19T17:58:14.486319+0000","last_undegraded":"2023-12-19T17:58:14.486319+0000","last_fullsized":"2023-12-19T17:58:14.486319+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:39.370236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915315+0000","last_change":"2023-12-19T17:58:13.510086+0000","last_active":"2023-12-19T17:58:14.915315+0000","last_peered":"2023-12-19T17:58:14.915315+0000","last_clean":"2023-12-19T17:58:14.915315+0000","last_became_active":"2023-12-19T17:58:13.509510+0000","last_became_peered":"2023-12-19T17:58:13.509510+0000","last_unstale":"2023-12-19T17:58:14.915315+0000","last_undegraded":"2023-12-19T17:58:14.915315+0000","last_fullsized":"2023-12-19T17:58:14.915315+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:22.896718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"176'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914754+0000","last_change":"2023-12-19T17:58:13.091802+0000","last_active":"2023-12-19T17:58:14.914754+0000","last_peered":"2023-12-19T17:58:14.914754+0000","last_clean":"2023-12-19T17:58:14.914754+0000","last_became_active":"2023-12-19T17:58:13.091483+0000","last_became_peered":"2023-12-19T17:58:13.091483+0000","last_unstale":"2023-12-19T17:58:14.914754+0000","last_undegraded":"2023-12-19T17:58:14.914754+0000","last_fullsized":"2023-12-19T17:58:14.914754+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:03.465410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486848+0000","last_change":"2023-12-19T17:58:13.512182+0000","last_active":"2023-12-19T17:58:14.486848+0000","last_peered":"2023-12-19T17:58:14.486848+0000","last_clean":"2023-12-19T17:58:14.486848+0000","last_became_active":"2023-12-19T17:58:13.511913+0000","last_became_peered":"2023-12-19T17:58:13.511913+0000","last_unstale":"2023-12-19T17:58:14.486848+0000","last_undegraded":"2023-12-19T17:58:14.486848+0000","last_fullsized":"2023-12-19T17:58:14.486848+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:37.415349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"176'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915242+0000","last_change":"2023-12-19T17:58:13.093203+0000","last_active":"2023-12-19T17:58:14.915242+0000","last_peered":"2023-12-19T17:58:14.915242+0000","last_clean":"2023-12-19T17:58:14.915242+0000","last_became_active":"2023-12-19T17:58:13.092111+0000","last_became_peered":"2023-12-19T17:58:13.092111+0000","last_unstale":"2023-12-19T17:58:14.915242+0000","last_undegraded":"2023-12-19T17:58:14.915242+0000","last_fullsized":"2023-12-19T17:58:14.915242+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:47:31.206655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484614+0000","last_change":"2023-12-19T17:58:13.500853+0000","last_active":"2023-12-19T17:58:14.484614+0000","last_peered":"2023-12-19T17:58:14.484614+0000","last_clean":"2023-12-19T17:58:14.484614+0000","last_became_active":"2023-12-19T17:58:13.500256+0000","last_became_peered":"2023-12-19T17:58:13.500256+0000","last_unstale":"2023-12-19T17:58:14.484614+0000","last_undegraded":"2023-12-19T17:58:14.484614+0000","last_fullsized":"2023-12-19T17:58:14.484614+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:29.304411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914845+0000","last_change":"2023-12-19T17:58:13.095219+0000","last_active":"2023-12-19T17:58:14.914845+0000","last_peered":"2023-12-19T17:58:14.914845+0000","last_clean":"2023-12-19T17:58:14.914845+0000","last_became_active":"2023-12-19T17:58:13.094936+0000","last_became_peered":"2023-12-19T17:58:13.094936+0000","last_unstale":"2023-12-19T17:58:14.914845+0000","last_undegraded":"2023-12-19T17:58:14.914845+0000","last_fullsized":"2023-12-19T17:58:14.914845+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:39:15.717854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915077+0000","last_change":"2023-12-19T17:58:13.496062+0000","last_active":"2023-12-19T17:58:14.915077+0000","last_peered":"2023-12-19T17:58:14.915077+0000","last_clean":"2023-12-19T17:58:14.915077+0000","last_became_active":"2023-12-19T17:58:13.495067+0000","last_became_peered":"2023-12-19T17:58:13.495067+0000","last_unstale":"2023-12-19T17:58:14.915077+0000","last_undegraded":"2023-12-19T17:58:14.915077+0000","last_fullsized":"2023-12-19T17:58:14.915077+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:40.235025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"176'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915336+0000","last_change":"2023-12-19T17:58:13.083333+0000","last_active":"2023-12-19T17:58:14.915336+0000","last_peered":"2023-12-19T17:58:14.915336+0000","last_clean":"2023-12-19T17:58:14.915336+0000","last_became_active":"2023-12-19T17:58:13.083019+0000","last_became_peered":"2023-12-19T17:58:13.083019+0000","last_unstale":"2023-12-19T17:58:14.915336+0000","last_undegraded":"2023-12-19T17:58:14.915336+0000","last_fullsized":"2023-12-19T17:58:14.915336+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:46.086030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915172+0000","last_change":"2023-12-19T17:58:13.485884+0000","last_active":"2023-12-19T17:58:14.915172+0000","last_peered":"2023-12-19T17:58:14.915172+0000","last_clean":"2023-12-19T17:58:14.915172+0000","last_became_active":"2023-12-19T17:58:13.485541+0000","last_became_peered":"2023-12-19T17:58:13.485541+0000","last_unstale":"2023-12-19T17:58:14.915172+0000","last_undegraded":"2023-12-19T17:58:14.915172+0000","last_fullsized":"2023-12-19T17:58:14.915172+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:17:12.269946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914571+0000","last_change":"2023-12-19T17:58:13.092150+0000","last_active":"2023-12-19T17:58:14.914571+0000","last_peered":"2023-12-19T17:58:14.914571+0000","last_clean":"2023-12-19T17:58:14.914571+0000","last_became_active":"2023-12-19T17:58:13.091853+0000","last_became_peered":"2023-12-19T17:58:13.091853+0000","last_unstale":"2023-12-19T17:58:14.914571+0000","last_undegraded":"2023-12-19T17:58:14.914571+0000","last_fullsized":"2023-12-19T17:58:14.914571+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:08:47.461390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914481+0000","last_change":"2023-12-19T17:58:13.486929+0000","last_active":"2023-12-19T17:58:14.914481+0000","last_peered":"2023-12-19T17:58:14.914481+0000","last_clean":"2023-12-19T17:58:14.914481+0000","last_became_active":"2023-12-19T17:58:13.486664+0000","last_became_peered":"2023-12-19T17:58:13.486664+0000","last_unstale":"2023-12-19T17:58:14.914481+0000","last_undegraded":"2023-12-19T17:58:14.914481+0000","last_fullsized":"2023-12-19T17:58:14.914481+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:03:42.151305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914467+0000","last_change":"2023-12-19T17:58:13.092726+0000","last_active":"2023-12-19T17:58:14.914467+0000","last_peered":"2023-12-19T17:58:14.914467+0000","last_clean":"2023-12-19T17:58:14.914467+0000","last_became_active":"2023-12-19T17:58:13.092013+0000","last_became_peered":"2023-12-19T17:58:13.092013+0000","last_unstale":"2023-12-19T17:58:14.914467+0000","last_undegraded":"2023-12-19T17:58:14.914467+0000","last_fullsized":"2023-12-19T17:58:14.914467+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:17:13.408148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915157+0000","last_change":"2023-12-19T17:58:13.486204+0000","last_active":"2023-12-19T17:58:14.915157+0000","last_peered":"2023-12-19T17:58:14.915157+0000","last_clean":"2023-12-19T17:58:14.915157+0000","last_became_active":"2023-12-19T17:58:13.485921+0000","last_became_peered":"2023-12-19T17:58:13.485921+0000","last_unstale":"2023-12-19T17:58:14.915157+0000","last_undegraded":"2023-12-19T17:58:14.915157+0000","last_fullsized":"2023-12-19T17:58:14.915157+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:26:45.339697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485675+0000","last_change":"2023-12-19T17:58:13.087755+0000","last_active":"2023-12-19T17:58:14.485675+0000","last_peered":"2023-12-19T17:58:14.485675+0000","last_clean":"2023-12-19T17:58:14.485675+0000","last_became_active":"2023-12-19T17:58:13.087449+0000","last_became_peered":"2023-12-19T17:58:13.087449+0000","last_unstale":"2023-12-19T17:58:14.485675+0000","last_undegraded":"2023-12-19T17:58:14.485675+0000","last_fullsized":"2023-12-19T17:58:14.485675+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:15:04.127346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486078+0000","last_change":"2023-12-19T17:58:13.508091+0000","last_active":"2023-12-19T17:58:14.486078+0000","last_peered":"2023-12-19T17:58:14.486078+0000","last_clean":"2023-12-19T17:58:14.486078+0000","last_became_active":"2023-12-19T17:58:13.507813+0000","last_became_peered":"2023-12-19T17:58:13.507813+0000","last_unstale":"2023-12-19T17:58:14.486078+0000","last_undegraded":"2023-12-19T17:58:14.486078+0000","last_fullsized":"2023-12-19T17:58:14.486078+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:01:17.175289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"176'2","reported_seq":13,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915292+0000","last_change":"2023-12-19T17:58:13.092990+0000","last_active":"2023-12-19T17:58:14.915292+0000","last_peered":"2023-12-19T17:58:14.915292+0000","last_clean":"2023-12-19T17:58:14.915292+0000","last_became_active":"2023-12-19T17:58:13.092634+0000","last_became_peered":"2023-12-19T17:58:13.092634+0000","last_unstale":"2023-12-19T17:58:14.915292+0000","last_undegraded":"2023-12-19T17:58:14.915292+0000","last_fullsized":"2023-12-19T17:58:14.915292+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:48.130738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486898+0000","last_change":"2023-12-19T17:58:13.505468+0000","last_active":"2023-12-19T17:58:14.486898+0000","last_peered":"2023-12-19T17:58:14.486898+0000","last_clean":"2023-12-19T17:58:14.486898+0000","last_became_active":"2023-12-19T17:58:13.505181+0000","last_became_peered":"2023-12-19T17:58:13.505181+0000","last_unstale":"2023-12-19T17:58:14.486898+0000","last_undegraded":"2023-12-19T17:58:14.486898+0000","last_fullsized":"2023-12-19T17:58:14.486898+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:33.295886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"176'1","reported_seq":12,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484805+0000","last_change":"2023-12-19T17:58:13.090504+0000","last_active":"2023-12-19T17:58:14.484805+0000","last_peered":"2023-12-19T17:58:14.484805+0000","last_clean":"2023-12-19T17:58:14.484805+0000","last_became_active":"2023-12-19T17:58:13.090180+0000","last_became_peered":"2023-12-19T17:58:13.090180+0000","last_unstale":"2023-12-19T17:58:14.484805+0000","last_undegraded":"2023-12-19T17:58:14.484805+0000","last_fullsized":"2023-12-19T17:58:14.484805+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:58:26.950125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914548+0000","last_change":"2023-12-19T17:58:13.496201+0000","last_active":"2023-12-19T17:58:14.914548+0000","last_peered":"2023-12-19T17:58:14.914548+0000","last_clean":"2023-12-19T17:58:14.914548+0000","last_became_active":"2023-12-19T17:58:13.495921+0000","last_became_peered":"2023-12-19T17:58:13.495921+0000","last_unstale":"2023-12-19T17:58:14.914548+0000","last_undegraded":"2023-12-19T17:58:14.914548+0000","last_fullsized":"2023-12-19T17:58:14.914548+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:17.870336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484812+0000","last_change":"2023-12-19T17:58:13.086573+0000","last_active":"2023-12-19T17:58:14.484812+0000","last_peered":"2023-12-19T17:58:14.484812+0000","last_clean":"2023-12-19T17:58:14.484812+0000","last_became_active":"2023-12-19T17:58:13.086111+0000","last_became_peered":"2023-12-19T17:58:13.086111+0000","last_unstale":"2023-12-19T17:58:14.484812+0000","last_undegraded":"2023-12-19T17:58:14.484812+0000","last_fullsized":"2023-12-19T17:58:14.484812+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:08:47.461390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484726+0000","last_change":"2023-12-19T17:58:13.494408+0000","last_active":"2023-12-19T17:58:14.484726+0000","last_peered":"2023-12-19T17:58:14.484726+0000","last_clean":"2023-12-19T17:58:14.484726+0000","last_became_active":"2023-12-19T17:58:13.494170+0000","last_became_peered":"2023-12-19T17:58:13.494170+0000","last_unstale":"2023-12-19T17:58:14.484726+0000","last_undegraded":"2023-12-19T17:58:14.484726+0000","last_fullsized":"2023-12-19T17:58:14.484726+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:28:05.209260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.484692+0000","last_change":"2023-12-19T17:58:13.086066+0000","last_active":"2023-12-19T17:58:14.484692+0000","last_peered":"2023-12-19T17:58:14.484692+0000","last_clean":"2023-12-19T17:58:14.484692+0000","last_became_active":"2023-12-19T17:58:13.085795+0000","last_became_peered":"2023-12-19T17:58:13.085795+0000","last_unstale":"2023-12-19T17:58:14.484692+0000","last_undegraded":"2023-12-19T17:58:14.484692+0000","last_fullsized":"2023-12-19T17:58:14.484692+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:44:07.815304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914405+0000","last_change":"2023-12-19T17:58:13.495612+0000","last_active":"2023-12-19T17:58:14.914405+0000","last_peered":"2023-12-19T17:58:14.914405+0000","last_clean":"2023-12-19T17:58:14.914405+0000","last_became_active":"2023-12-19T17:58:13.495342+0000","last_became_peered":"2023-12-19T17:58:13.495342+0000","last_unstale":"2023-12-19T17:58:14.914405+0000","last_undegraded":"2023-12-19T17:58:14.914405+0000","last_fullsized":"2023-12-19T17:58:14.914405+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:47.531250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915317+0000","last_change":"2023-12-19T17:58:13.076018+0000","last_active":"2023-12-19T17:58:14.915317+0000","last_peered":"2023-12-19T17:58:14.915317+0000","last_clean":"2023-12-19T17:58:14.915317+0000","last_became_active":"2023-12-19T17:58:13.075737+0000","last_became_peered":"2023-12-19T17:58:13.075737+0000","last_unstale":"2023-12-19T17:58:14.915317+0000","last_undegraded":"2023-12-19T17:58:14.915317+0000","last_fullsized":"2023-12-19T17:58:14.915317+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:56:15.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486197+0000","last_change":"2023-12-19T17:58:13.493436+0000","last_active":"2023-12-19T17:58:14.486197+0000","last_peered":"2023-12-19T17:58:14.486197+0000","last_clean":"2023-12-19T17:58:14.486197+0000","last_became_active":"2023-12-19T17:58:13.493124+0000","last_became_peered":"2023-12-19T17:58:13.493124+0000","last_unstale":"2023-12-19T17:58:14.486197+0000","last_undegraded":"2023-12-19T17:58:14.486197+0000","last_fullsized":"2023-12-19T17:58:14.486197+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:36:10.784681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915241+0000","last_change":"2023-12-19T17:58:13.092618+0000","last_active":"2023-12-19T17:58:14.915241+0000","last_peered":"2023-12-19T17:58:14.915241+0000","last_clean":"2023-12-19T17:58:14.915241+0000","last_became_active":"2023-12-19T17:58:13.092394+0000","last_became_peered":"2023-12-19T17:58:13.092394+0000","last_unstale":"2023-12-19T17:58:14.915241+0000","last_undegraded":"2023-12-19T17:58:14.915241+0000","last_fullsized":"2023-12-19T17:58:14.915241+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:58:26.950125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914601+0000","last_change":"2023-12-19T17:58:13.508248+0000","last_active":"2023-12-19T17:58:14.914601+0000","last_peered":"2023-12-19T17:58:14.914601+0000","last_clean":"2023-12-19T17:58:14.914601+0000","last_became_active":"2023-12-19T17:58:13.507965+0000","last_became_peered":"2023-12-19T17:58:13.507965+0000","last_unstale":"2023-12-19T17:58:14.914601+0000","last_undegraded":"2023-12-19T17:58:14.914601+0000","last_fullsized":"2023-12-19T17:58:14.914601+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:27:03.254793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486084+0000","last_change":"2023-12-19T17:58:13.086445+0000","last_active":"2023-12-19T17:58:14.486084+0000","last_peered":"2023-12-19T17:58:14.486084+0000","last_clean":"2023-12-19T17:58:14.486084+0000","last_became_active":"2023-12-19T17:58:13.086187+0000","last_became_peered":"2023-12-19T17:58:13.086187+0000","last_unstale":"2023-12-19T17:58:14.486084+0000","last_undegraded":"2023-12-19T17:58:14.486084+0000","last_fullsized":"2023-12-19T17:58:14.486084+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:17.813715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.485752+0000","last_change":"2023-12-19T17:58:13.492514+0000","last_active":"2023-12-19T17:58:14.485752+0000","last_peered":"2023-12-19T17:58:14.485752+0000","last_clean":"2023-12-19T17:58:14.485752+0000","last_became_active":"2023-12-19T17:58:13.492259+0000","last_became_peered":"2023-12-19T17:58:13.492259+0000","last_unstale":"2023-12-19T17:58:14.485752+0000","last_undegraded":"2023-12-19T17:58:14.485752+0000","last_fullsized":"2023-12-19T17:58:14.485752+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:10:32.458962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.914157+0000","last_change":"2023-12-19T17:58:13.094141+0000","last_active":"2023-12-19T17:58:14.914157+0000","last_peered":"2023-12-19T17:58:14.914157+0000","last_clean":"2023-12-19T17:58:14.914157+0000","last_became_active":"2023-12-19T17:58:13.093822+0000","last_became_peered":"2023-12-19T17:58:13.093822+0000","last_unstale":"2023-12-19T17:58:14.914157+0000","last_undegraded":"2023-12-19T17:58:14.914157+0000","last_fullsized":"2023-12-19T17:58:14.914157+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:16.122567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.915380+0000","last_change":"2023-12-19T17:58:13.486835+0000","last_active":"2023-12-19T17:58:14.915380+0000","last_peered":"2023-12-19T17:58:14.915380+0000","last_clean":"2023-12-19T17:58:14.915380+0000","last_became_active":"2023-12-19T17:58:13.486591+0000","last_became_peered":"2023-12-19T17:58:13.486591+0000","last_unstale":"2023-12-19T17:58:14.915380+0000","last_undegraded":"2023-12-19T17:58:14.915380+0000","last_fullsized":"2023-12-19T17:58:14.915380+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":175,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:13.055805+0000","last_clean_scrub_stamp":"2023-12-19T17:58:13.055805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:40:53.684360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"176'3","reported_seq":14,"reported_epoch":177,"state":"active+clean","last_fresh":"2023-12-19T17:58:14.486968+0000","last_change":"2023-12-19T17:58:13.087498+0000","last_active":"2023-12-19T17:58:14.486968+0000","last_peered":"2023-12-19T17:58:14.486968+0000","last_clean":"2023-12-19T17:58:14.486968+0000","last_became_active":"2023-12-19T17:58:13.087220+0000","last_became_peered":"2023-12-19T17:58:13.087220+0000","last_unstale":"2023-12-19T17:58:14.486968+0000","last_undegraded":"2023-12-19T17:58:14.486968+0000","last_fullsized":"2023-12-19T17:58:14.486968+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":174,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T17:58:12.047034+0000","last_clean_scrub_stamp":"2023-12-19T17:58:12.047034+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:34:55.421495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"165'291","reported_seq":950,"reported_epoch":176,"state":"active+clean","last_fresh":"2023-12-19T17:58:13.487606+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T17:58:13.487606+0000","last_peered":"2023-12-19T17:58:13.487606+0000","last_clean":"2023-12-19T17:58:13.487606+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T17:58:13.487606+0000","last_undegraded":"2023-12-19T17:58:13.487606+0000","last_fullsized":"2023-12-19T17:58:13.487606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1634,"num_read_kb":3381,"num_write":475,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1634,"num_read_kb":3381,"num_write":475,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":291,"ondisk_log_size":291,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510349,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37880,"kb_used_data":2736,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94333960,"statfs":{"total":96636764160,"available":96597975040,"internally_reserved":0,"allocated":2801664,"data_stored":1739648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.72699999999999998,"15min":0.73599999999999999},"min":{"1min":0.52900000000000003,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89000000000000001,"5min":1.012,"15min":1.0209999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.71399999999999997,"15min":0.72899999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78700000000000003,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.624}]},{"osd":1,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74199999999999999,"15min":0.74099999999999999},"min":{"1min":0.52300000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.91100000000000003,"5min":0.96899999999999997,"15min":0.997},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.747,"15min":0.75700000000000001},"min":{"1min":0.47599999999999998,"5min":0.443,"15min":0.443},"max":{"1min":0.99399999999999999,"5min":1.0169999999999999,"15min":1.05},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91400000000000003,"5min":0.92400000000000004,"15min":0.97999999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.50800000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94899999999999995,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.92600000000000005}]},{"osd":3,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.755,"15min":0.748},"min":{"1min":0.57999999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.92900000000000005,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76100000000000001,"15min":0.77200000000000002},"min":{"1min":0.54200000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.99099999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.76300000000000001}]},{"osd":4,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60399999999999998,"15min":0.58599999999999997},"min":{"1min":0.442,"5min":0.377,"15min":0.377},"max":{"1min":0.92900000000000005,"5min":0.93899999999999995,"15min":0.94299999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63200000000000001,"15min":0.61299999999999999},"min":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89000000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.59799999999999998}]},{"osd":5,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65500000000000003,"15min":0.63},"min":{"1min":0.44700000000000001,"5min":0.40400000000000003,"15min":0.36799999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.67600000000000005,"15min":0.66300000000000003},"min":{"1min":0.45800000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Dec 19 17:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.73599999999999999,"15min":0.71999999999999997},"min":{"1min":0.55900000000000005,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.70899999999999996,"15min":0.68799999999999994},"min":{"1min":0.52700000000000002,"5min":0.45900000000000002,"15min":0.44500000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.79100000000000004}]}]},{"osd":6,"up_from":15,"seq":64424510349,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63544,"kb_used_data":2160,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308296,"statfs":{"total":96636764160,"available":96571695104,"internally_reserved":0,"allocated":2211840,"data_stored":1148774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76000000000000001,"15min":0.73899999999999999},"min":{"1min":0.48799999999999999,"5min":0.40300000000000002,"15min":0.36799999999999999},"max":{"1min":0.995,"5min":0.995,"15min":1.02},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.747,"15min":0.73799999999999999},"min":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.92900000000000005,"5min":0.95699999999999996,"15min":1.329},"last":0.85199999999999998}]},{"osd":1,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.75,"15min":0.73799999999999999},"min":{"1min":0.52400000000000002,"5min":0.50600000000000001,"15min":0.42899999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.1419999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.77200000000000002,"15min":0.75700000000000001},"min":{"1min":0.65600000000000003,"5min":0.47799999999999998,"15min":0.45200000000000001},"max":{"1min":0.96499999999999997,"5min":1.026,"15min":1.026},"last":0.82599999999999996}]},{"osd":2,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.78200000000000003,"15min":0.76300000000000001},"min":{"1min":0.61299999999999999,"5min":0.497,"15min":0.48799999999999999},"max":{"1min":0.97099999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76000000000000001,"15min":0.76900000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.497},"max":{"1min":0.97999999999999998,"5min":1.0029999999999999,"15min":1.853},"last":0.76200000000000001}]},{"osd":3,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.80100000000000005,"15min":0.79400000000000004},"min":{"1min":0.63700000000000001,"5min":0.59899999999999998,"15min":0.45400000000000001},"max":{"1min":0.93799999999999994,"5min":1.0149999999999999,"15min":1.542},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.75800000000000001,"15min":0.73699999999999999},"min":{"1min":0.57099999999999995,"5min":0.54700000000000004,"15min":0.48799999999999999},"max":{"1min":0.94699999999999995,"5min":0.98699999999999999,"15min":1.04},"last":0.67700000000000005}]},{"osd":4,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.41699999999999998,"5min":0.41599999999999998,"15min":0.36799999999999999},"max":{"1min":0.85399999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.499,"5min":0.41499999999999998,"15min":0.372},"max":{"1min":1.131,"5min":1.131,"15min":1.3919999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60999999999999999,"15min":0.60399999999999998},"min":{"1min":0.46600000000000003,"5min":0.442,"15min":0.34100000000000003},"max":{"1min":0.86399999999999999,"5min":0.91000000000000003,"15min":1.0920000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.66900000000000004,"15min":0.66200000000000003},"min":{"1min":0.54700000000000004,"5min":0.44700000000000001,"15min":0.41399999999999998},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.65600000000000003}]},{"osd":7,"last update":"Tue Dec 19 17:57:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.57899999999999996,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1,"5min":1.0580000000000001,"15min":1.4259999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.71599999999999997},"min":{"1min":0.59499999999999997,"5min":0.503,"15min":0.42699999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0369999999999999},"last":0.747}]}]},{"osd":5,"up_from":15,"seq":64424510349,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45488,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":43328,"kb_avail":94326352,"statfs":{"total":96636764160,"available":96590184448,"internally_reserved":0,"allocated":2203648,"data_stored":1148650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":44367872},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.40799999999999997},"max":{"1min":0.91500000000000004,"5min":1.0269999999999999,"15min":1.1120000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.49299999999999999,"5min":0.48899999999999999,"15min":0.374},"max":{"1min":0.879,"5min":1.0289999999999999,"15min":1.073},"last":0.86899999999999999}]},{"osd":1,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.75900000000000001,"15min":0.76400000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.96299999999999997,"5min":1.0029999999999999,"15min":1.1659999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72999999999999998,"15min":0.73999999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97499999999999998,"5min":1.0249999999999999,"15min":1.0469999999999999},"last":0.88800000000000001}]},{"osd":2,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.48399999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93899999999999995,"5min":1.05,"15min":1.1040000000000001},"last":1.04},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92200000000000004,"5min":0.95899999999999996,"15min":1.0349999999999999},"last":0.90800000000000003}]},{"osd":3,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.131},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76100000000000001,"15min":0.77100000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90500000000000003,"5min":0.99299999999999999,"15min":1.125},"last":0.97899999999999998}]},{"osd":4,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.35699999999999998},"max":{"1min":0.88200000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.61499999999999999,"15min":0.61399999999999999},"min":{"1min":0.441,"5min":0.44,"15min":0.41399999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.77400000000000002}]},{"osd":6,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":1.048},"last":0.745},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.66000000000000003,"15min":0.64600000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82899999999999996,"5min":0.85499999999999998,"15min":0.97699999999999998},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Dec 19 17:57:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65700000000000003,"15min":0.66800000000000004},"min":{"1min":0.55100000000000005,"5min":0.46800000000000003,"15min":0.46300000000000002},"max":{"1min":0.89500000000000002,"5min":0.97399999999999998,"15min":1.016},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69399999999999995,"15min":0.70099999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.41299999999999998},"max":{"1min":0.90700000000000003,"5min":0.94999999999999996,"15min":1.0289999999999999},"last":0.80100000000000005}]}]},{"osd":4,"up_from":15,"seq":64424510349,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59424,"kb_used_data":2136,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312416,"statfs":{"total":96636764160,"available":96575913984,"internally_reserved":0,"allocated":2187264,"data_stored":1141596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.71199999999999997,"15min":0.73699999999999999},"min":{"1min":0.47299999999999998,"5min":0.44,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":1.036,"15min":1.056},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70799999999999996,"15min":0.73099999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.40400000000000003},"max":{"1min":0.80600000000000005,"5min":0.92500000000000004,"15min":1.0620000000000001},"last":0.58399999999999996}]},{"osd":1,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.74099999999999999,"15min":0.75600000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.995,"15min":1.0980000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.749,"15min":0.753},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92300000000000004,"5min":1.0489999999999999,"15min":1.0549999999999999},"last":0.71899999999999997}]},{"osd":2,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.73999999999999999,"15min":0.75900000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.995,"5min":1.026,"15min":1.042},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.55000000000000004,"5min":0.496,"15min":0.47299999999999998},"max":{"1min":0.95099999999999996,"5min":1.0149999999999999,"15min":1.018},"last":0.62}]},{"osd":3,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.72699999999999998,"15min":0.74099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.90300000000000002,"5min":0.96799999999999997,"15min":1.0229999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.76700000000000002,"15min":0.77700000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.92700000000000005,"5min":0.997,"15min":1.0049999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.36799999999999999},"max":{"1min":0.71499999999999997,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.625,"15min":0.62},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.379},"max":{"1min":0.78000000000000003,"5min":1.0029999999999999,"15min":1.0109999999999999},"last":0.73599999999999999}]},{"osd":6,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88800000000000001,"5min":1.03,"15min":1.079},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.69799999999999995,"15min":0.67900000000000005},"min":{"1min":0.47299999999999998,"5min":0.46500000000000002,"15min":0.34599999999999997},"max":{"1min":0.873,"5min":0.96999999999999997,"15min":0.99099999999999999},"last":0.754}]},{"osd":7,"last update":"Tue Dec 19 17:57:57 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.66800000000000004,"15min":0.67800000000000005},"min":{"1min":0.47099999999999997,"5min":0.44800000000000001,"15min":0.435},"max":{"1min":0.78700000000000003,"5min":1.075,"15min":1.075},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.71699999999999997,"15min":0.70999999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.45500000000000002},"max":{"1min":0.93000000000000005,"5min":1.0629999999999999,"15min":1.1160000000000001},"last":0.98199999999999998}]}]},{"osd":3,"up_from":15,"seq":64424510349,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47152,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324688,"statfs":{"total":96636764160,"available":96588480512,"internally_reserved":0,"allocated":2203648,"data_stored":1148718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.47799999999999998,"15min":0.47299999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.36199999999999999,"5min":0.313,"15min":0.29499999999999998},"max":{"1min":0.751,"5min":0.76700000000000002,"15min":1.01},"last":0.43099999999999999}]},{"osd":1,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.49099999999999999,"15min":0.48199999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.307},"max":{"1min":0.68799999999999994,"5min":0.72199999999999998,"15min":0.83899999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.39900000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57299999999999995,"15min":0.56999999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.72199999999999998,"5min":0.79100000000000004,"15min":0.92100000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.73199999999999998,"5min":0.93000000000000005,"15min":0.98699999999999999},"last":0.44900000000000001}]},{"osd":4,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.52700000000000002},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.82199999999999995,"15min":0.82999999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.2210000000000001,"5min":1.236,"15min":1.371},"last":0.60899999999999999}]},{"osd":5,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.81899999999999995,"15min":0.82999999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.296},"last":0.745},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.82399999999999995,"15min":0.81899999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.52800000000000002},"max":{"1min":1.1220000000000001,"5min":1.133,"15min":1.2290000000000001},"last":0.61899999999999999}]},{"osd":6,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.85499999999999998,"15min":0.84999999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.359},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.85799999999999998,"15min":0.84899999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.59599999999999997},"max":{"1min":1.232,"5min":1.232,"15min":1.2829999999999999},"last":0.63300000000000001}]},{"osd":7,"last update":"Tue Dec 19 17:57:58 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.89900000000000002,"15min":0.89500000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.4530000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.89600000000000002,"15min":0.89900000000000002},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.59599999999999997},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.70399999999999996}]}]},{"osd":2,"up_from":15,"seq":64424510348,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63536,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308304,"statfs":{"total":96636764160,"available":96571703296,"internally_reserved":0,"allocated":2203648,"data_stored":1143230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44700000000000001,"15min":0.45800000000000002},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59999999999999998,"5min":0.71499999999999997,"15min":0.80800000000000005},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.502,"15min":0.504},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32000000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88600000000000001},"last":0.34899999999999998}]},{"osd":1,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.62,"15min":0.61099999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.36499999999999999},"max":{"1min":0.80200000000000005,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.51900000000000002},"min":{"1min":0.375,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.67900000000000005,"5min":0.79600000000000004,"15min":0.93300000000000005},"last":0.43099999999999999}]},{"osd":3,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.60799999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82699999999999996,"5min":0.88,"15min":0.93300000000000005},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.40500000000000003,"5min":0.38400000000000001,"15min":0.35599999999999998},"max":{"1min":0.74299999999999999,"5min":0.79900000000000004,"15min":0.93300000000000005},"last":0.47099999999999997}]},{"osd":4,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.80100000000000005,"15min":0.80600000000000005},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.54000000000000004},"max":{"1min":1.1020000000000001,"5min":1.1639999999999999,"15min":1.3360000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79800000000000004,"15min":0.79000000000000004},"min":{"1min":0.53600000000000003,"5min":0.443,"15min":0.443},"max":{"1min":1.1180000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.86599999999999999}]},{"osd":5,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80300000000000005,"15min":0.80500000000000005},"min":{"1min":0.60499999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.071,"5min":1.339,"15min":1.339},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81799999999999995,"15min":0.81699999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.083,"5min":1.351,"15min":1.351},"last":0.81399999999999995}]},{"osd":6,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84199999999999997,"15min":0.84299999999999997},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.55800000000000005},"max":{"1min":1.0349999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.46600000000000003},"max":{"1min":1.1399999999999999,"5min":1.1619999999999999,"15min":1.171},"last":0.94399999999999995}]},{"osd":7,"last update":"Tue Dec 19 17:57:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.82699999999999996},"min":{"1min":0.69399999999999995,"5min":0.58699999999999997,"15min":0.53500000000000003},"max":{"1min":1.21,"5min":1.2629999999999999,"15min":1.3580000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.64900000000000002,"5min":0.63300000000000001,"15min":0.61899999999999999},"max":{"1min":1.1279999999999999,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.99199999999999999}]}]},{"osd":1,"up_from":15,"seq":64424510349,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55344,"kb_used_data":2152,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316496,"statfs":{"total":96636764160,"available":96580091904,"internally_reserved":0,"allocated":2203648,"data_stored":1149210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.44600000000000001,"15min":0.45400000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.28100000000000003}]},{"osd":2,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52600000000000002,"15min":0.51800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.68300000000000005,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.57399999999999995,"15min":0.56799999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.32600000000000001}]},{"osd":3,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77300000000000002,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78800000000000003,"5min":0.81799999999999995,"15min":0.85199999999999998},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80700000000000005,"15min":0.80500000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.004,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.749},{"interface":"front","average":{"1min":0.752,"5min":0.79400000000000004,"15min":0.79900000000000004},"min":{"1min":0.51000000000000001,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.048,"5min":1.355,"15min":1.355},"last":0.67900000000000005}]},{"osd":5,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.81499999999999995,"15min":0.80600000000000005},"min":{"1min":0.60099999999999998,"5min":0.55600000000000005,"15min":0.54600000000000004},"max":{"1min":0.91400000000000003,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.81000000000000005,"15min":0.81200000000000006},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.92300000000000004,"5min":1.149,"15min":1.149},"last":0.73499999999999999}]},{"osd":6,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.80600000000000005,"15min":0.81399999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52200000000000002},"max":{"1min":0.91300000000000003,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.83099999999999996,"15min":0.82699999999999996},"min":{"1min":0.59499999999999997,"5min":0.56999999999999995,"15min":0.56499999999999995},"max":{"1min":0.95599999999999996,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.72799999999999998}]},{"osd":7,"last update":"Tue Dec 19 17:57:50 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.86899999999999999,"15min":0.85699999999999998},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":0.97099999999999997,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.82899999999999996,"15min":0.84699999999999998},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.59499999999999997},"max":{"1min":0.94999999999999996,"5min":1.258,"15min":1.258},"last":0.77000000000000002}]}]},{"osd":0,"up_from":14,"seq":60129543054,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60032,"kb_used_data":2744,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311808,"statfs":{"total":96636764160,"available":96575291392,"internally_reserved":0,"allocated":2809856,"data_stored":1740594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.46000000000000002,"15min":0.45500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64100000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.51600000000000001,"15min":0.50700000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.29599999999999999},"max":{"1min":0.71499999999999997,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.35699999999999998,"15min":0.34300000000000003},"max":{"1min":0.60899999999999999,"5min":0.78500000000000003,"15min":0.83299999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.56000000000000005,"15min":0.55100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.65900000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53500000000000003,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.35899999999999999,"15min":0.33500000000000002},"max":{"1min":0.65000000000000002,"5min":0.84799999999999998,"15min":0.90400000000000003},"last":0.62},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64000000000000001,"15min":0.64400000000000002},"min":{"1min":0.501,"5min":0.497,"15min":0.45000000000000001},"max":{"1min":0.71299999999999997,"5min":0.90000000000000002,"15min":0.93500000000000005},"last":0.68600000000000005}]},{"osd":4,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80200000000000005,"15min":0.80300000000000005},"min":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.44700000000000001},"max":{"1min":1.1080000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52400000000000002},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.52700000000000002}]},{"osd":5,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80000000000000004,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.50700000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.169},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81799999999999995,"15min":0.82499999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.351},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1619999999999999,"5min":1.173,"15min":1.2350000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82699999999999996,"15min":0.83099999999999996},"min":{"1min":0.63300000000000001,"5min":0.60299999999999998,"15min":0.505},"max":{"1min":1.079,"5min":1.1020000000000001,"15min":1.329},"last":0.625}]},{"osd":7,"last update":"Tue Dec 19 17:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84999999999999998,"15min":0.85599999999999998},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2909999999999999},"last":0.66000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T17:58:17.988 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T17:58:17.988 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T17:58:18.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.174+0000 7efd53198640 1 Processor -- start 2023-12-19T17:58:18.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.174+0000 7efd53198640 1 -- start start 2023-12-19T17:58:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.174+0000 7efd53198640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd4c0adac0 0x7efd4c0c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.174+0000 7efd53198640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd4c15a480 0x7efd4c06fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.174+0000 7efd53198640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efd4c0701f0 0x7efd4c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.174+0000 7efd53198640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd4c070660 con 0x7efd4c15a480 2023-12-19T17:58:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.174+0000 7efd53198640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd4c06b3d0 con 0x7efd4c0701f0 2023-12-19T17:58:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.174+0000 7efd53198640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c0ca5e0 con 0x7efd4c0adac0 2023-12-19T17:58:18.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.174+0000 7efd4bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd4c15a480 0x7efd4c06fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.174+0000 7efd4bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd4c15a480 0x7efd4c06fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38758/0 (socket says 172.21.15.26:38758) 2023-12-19T17:58:18.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.174+0000 7efd4bfff640 1 -- 172.21.15.26:0/3535213599 learned_addr learned my addr 172.21.15.26:0/3535213599 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:18.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.174+0000 7efd50f34640 1 --2- 172.21.15.26:0/3535213599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd4c0adac0 0x7efd4c0c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.174+0000 7efd51735640 1 --2- 172.21.15.26:0/3535213599 >> v2:172.21.15.160:3300/0 conn(0x7efd4c0701f0 0x7efd4c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd4bfff640 1 -- 172.21.15.26:0/3535213599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd4c0adac0 msgr2=0x7efd4c0c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:18.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd4bfff640 1 --2- 172.21.15.26:0/3535213599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd4c0adac0 0x7efd4c0c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd4bfff640 1 -- 172.21.15.26:0/3535213599 >> v2:172.21.15.160:3300/0 conn(0x7efd4c0701f0 msgr2=0x7efd4c15cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:18.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd4bfff640 1 --2- 172.21.15.26:0/3535213599 >> v2:172.21.15.160:3300/0 conn(0x7efd4c0701f0 0x7efd4c15cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd4bfff640 1 -- 172.21.15.26:0/3535213599 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c0ca8c0 con 0x7efd4c15a480 2023-12-19T17:58:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd4bfff640 1 --2- 172.21.15.26:0/3535213599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd4c15a480 0x7efd4c06fcb0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0cb180 tx=0x7efd3801eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6a982678ad12ff server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd4b7fe640 1 -- 172.21.15.26:0/3535213599 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd380a7bb0 con 0x7efd4c15a480 2023-12-19T17:58:18.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd4b7fe640 1 -- 172.21.15.26:0/3535213599 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd380a7d10 con 0x7efd4c15a480 2023-12-19T17:58:18.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd4b7fe640 1 -- 172.21.15.26:0/3535213599 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd380af660 con 0x7efd4c15a480 2023-12-19T17:58:18.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd53198640 1 -- 172.21.15.26:0/3535213599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd4c15a480 msgr2=0x7efd4c06fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:18.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd53198640 1 --2- 172.21.15.26:0/3535213599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd4c15a480 0x7efd4c06fcb0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0cb180 tx=0x7efd3801eba0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd53198640 1 -- 172.21.15.26:0/3535213599 shutdown_connections 2023-12-19T17:58:18.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd53198640 1 --2- 172.21.15.26:0/3535213599 >> v2:172.21.15.160:3300/0 conn(0x7efd4c0701f0 0x7efd4c15cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd53198640 1 --2- 172.21.15.26:0/3535213599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd4c15a480 0x7efd4c06fcb0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd53198640 1 --2- 172.21.15.26:0/3535213599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd4c0adac0 0x7efd4c0c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.178+0000 7efd53198640 1 -- 172.21.15.26:0/3535213599 >> 172.21.15.26:0/3535213599 conn(0x7efd4c0660d0 msgr2=0x7efd4c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:18.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd53198640 1 -- 172.21.15.26:0/3535213599 shutdown_connections 2023-12-19T17:58:18.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd53198640 1 -- 172.21.15.26:0/3535213599 wait complete. 2023-12-19T17:58:18.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd53198640 1 Processor -- start 2023-12-19T17:58:18.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd53198640 1 -- start start 2023-12-19T17:58:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd53198640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd4c0701f0 0x7efd4c14bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd53198640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd4c0adac0 0x7efd4c15bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd53198640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c15a480 0x7efd4c0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd53198640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c149700 con 0x7efd4c0701f0 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd53198640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c15cd70 con 0x7efd4c15a480 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd53198640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c15c1f0 con 0x7efd4c0adac0 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd50f34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd4c0701f0 0x7efd4c14bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd50f34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd4c0701f0 0x7efd4c14bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38764/0 (socket says 172.21.15.26:38764) 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd50f34640 1 -- 172.21.15.26:0/502171933 learned_addr learned my addr 172.21.15.26:0/502171933 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd4bfff640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd4c0adac0 0x7efd4c15bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd51735640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c15a480 0x7efd4c0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd50f34640 1 -- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd4c0adac0 msgr2=0x7efd4c15bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd50f34640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd4c0adac0 0x7efd4c15bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd50f34640 1 -- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c15a480 msgr2=0x7efd4c0c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd50f34640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c15a480 0x7efd4c0c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd50f34640 1 -- 172.21.15.26:0/502171933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3800b770 con 0x7efd4c0701f0 2023-12-19T17:58:18.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd51735640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c15a480 0x7efd4c0c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd50f34640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd4c0701f0 0x7efd4c14bde0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7efd4000d800 tx=0x7efd4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd497fa640 1 -- 172.21.15.26:0/502171933 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4000b6c0 con 0x7efd4c0701f0 2023-12-19T17:58:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd497fa640 1 -- 172.21.15.26:0/502171933 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4000b820 con 0x7efd4c0701f0 2023-12-19T17:58:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd53198640 1 -- 172.21.15.26:0/502171933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4c15c4d0 con 0x7efd4c0701f0 2023-12-19T17:58:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.182+0000 7efd497fa640 1 -- 172.21.15.26:0/502171933 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4000bb90 con 0x7efd4c0701f0 2023-12-19T17:58:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.186+0000 7efd53198640 1 -- 172.21.15.26:0/502171933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4c15ca90 con 0x7efd4c0701f0 2023-12-19T17:58:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.186+0000 7efd53198640 1 -- 172.21.15.26:0/502171933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4c0ca290 con 0x7efd4c0701f0 2023-12-19T17:58:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.186+0000 7efd497fa640 1 -- 172.21.15.26:0/502171933 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7efd4000bcf0 con 0x7efd4c0701f0 2023-12-19T17:58:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.186+0000 7efd497fa640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd30065bb0 0x7efd30068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.186+0000 7efd497fa640 1 -- 172.21.15.26:0/502171933 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7efd40083ab0 con 0x7efd4c0701f0 2023-12-19T17:58:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.186+0000 7efd4bfff640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd30065bb0 0x7efd30068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.186+0000 7efd4bfff640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd30065bb0 0x7efd30068070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efd380b1000 tx=0x7efd38009040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.190+0000 7efd497fa640 1 -- 172.21.15.26:0/502171933 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efd40051070 con 0x7efd4c0701f0 2023-12-19T17:58:18.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.406+0000 7efd53198640 1 -- 172.21.15.26:0/502171933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7efd4c074270 con 0x7efd4c0701f0 2023-12-19T17:58:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.410+0000 7efd497fa640 1 -- 172.21.15.26:0/502171933 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v44) v1 ==== 190+0+0 (secure 0 0 0) 0x7efd40054690 con 0x7efd4c0701f0 2023-12-19T17:58:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2023-12-19T17:58:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.414+0000 7efd53198640 1 -- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd30065bb0 msgr2=0x7efd30068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.414+0000 7efd53198640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd30065bb0 0x7efd30068070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efd380b1000 tx=0x7efd38009040 comp rx=0 tx=0).stop 2023-12-19T17:58:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.414+0000 7efd53198640 1 -- 172.21.15.26:0/502171933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd4c0701f0 msgr2=0x7efd4c14bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.414+0000 7efd53198640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd4c0701f0 0x7efd4c14bde0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7efd4000d800 tx=0x7efd4000dcc0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.414+0000 7efd53198640 1 -- 172.21.15.26:0/502171933 shutdown_connections 2023-12-19T17:58:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.414+0000 7efd53198640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd4c15a480 0x7efd4c0c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.414+0000 7efd53198640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7efd30065bb0 0x7efd30068070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.414+0000 7efd53198640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd4c0adac0 0x7efd4c15bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.414+0000 7efd53198640 1 --2- 172.21.15.26:0/502171933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd4c0701f0 0x7efd4c14bde0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.414+0000 7efd53198640 1 -- 172.21.15.26:0/502171933 >> 172.21.15.26:0/502171933 conn(0x7efd4c0660d0 msgr2=0x7efd4c070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.414+0000 7efd53198640 1 -- 172.21.15.26:0/502171933 shutdown_connections 2023-12-19T17:58:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.414+0000 7efd53198640 1 -- 172.21.15.26:0/502171933 wait complete. 2023-12-19T17:58:18.436 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T17:58:18.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c98c2d640 1 Processor -- start 2023-12-19T17:58:18.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c98c2d640 1 -- start start 2023-12-19T17:58:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c98c2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c940cee50 0x7f1c9415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c98c2d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9406c970 con 0x7f1c94155030 2023-12-19T17:58:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c98c2d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9415b1f0 con 0x7f1c9415f370 2023-12-19T17:58:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c98c2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9406c790 con 0x7f1c940cee50 2023-12-19T17:58:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1c94155030 0x7f1c9406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34434/0 (socket says 172.21.15.26:34434) 2023-12-19T17:58:18.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9259c640 1 -- 172.21.15.26:0/355915311 learned_addr learned my addr 172.21.15.26:0/355915311 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c92d9d640 1 --2- 172.21.15.26:0/355915311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c940cee50 0x7f1c9415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3418225223 0 0) 0x7f1c9406c970 con 0x7f1c94155030 2023-12-19T17:58:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c7c003530 con 0x7f1c94155030 2023-12-19T17:58:18.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1542802214 0 0) 0x7f1c7c003530 con 0x7f1c94155030 2023-12-19T17:58:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c940cee50 msgr2=0x7f1c9415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 --2- 172.21.15.26:0/355915311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c940cee50 0x7f1c9415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 >> v1:172.21.15.160:6789/0 conn(0x7f1c9415f370 legacy=0x7f1c940ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9415b4d0 con 0x7f1c94155030 2023-12-19T17:58:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f1c84001680 con 0x7f1c94155030 2023-12-19T17:58:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 >> v1:172.21.15.26:6789/0 conn(0x7f1c94155030 legacy=0x7f1c9406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 --2- 172.21.15.26:0/355915311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c7c0039b0 0x7f1c7c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 --2- 172.21.15.26:0/355915311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c7c004360 0x7f1c7c004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 --2- 172.21.15.26:0/355915311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c940cee50 0x7f1c7c005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9406c790 con 0x7f1c7c004360 2023-12-19T17:58:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c00b9c0 con 0x7f1c7c0039b0 2023-12-19T17:58:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.578+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c00bb80 con 0x7f1c940cee50 2023-12-19T17:58:18.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.582+0000 7f1c91d9b640 1 --2- 172.21.15.26:0/355915311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c7c004360 0x7f1c7c004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.582+0000 7f1c9259c640 1 --2- 172.21.15.26:0/355915311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c7c0039b0 0x7f1c7c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.582+0000 7f1c92d9d640 1 --2- 172.21.15.26:0/355915311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c940cee50 0x7f1c7c005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.582+0000 7f1c91d9b640 1 -- 172.21.15.26:0/355915311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c940cee50 msgr2=0x7f1c7c005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:18.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.582+0000 7f1c91d9b640 1 --2- 172.21.15.26:0/355915311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c940cee50 0x7f1c7c005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.582+0000 7f1c91d9b640 1 -- 172.21.15.26:0/355915311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c7c0039b0 msgr2=0x7f1c7c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:18.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.582+0000 7f1c91d9b640 1 --2- 172.21.15.26:0/355915311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c7c0039b0 0x7f1c7c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.582+0000 7f1c91d9b640 1 -- 172.21.15.26:0/355915311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c003530 con 0x7f1c7c004360 2023-12-19T17:58:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.582+0000 7f1c91d9b640 1 --2- 172.21.15.26:0/355915311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c7c004360 0x7f1c7c004830 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f1c88001860 tx=0x7f1c8801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc98d90d3b43750c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.582+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c880a8870 con 0x7f1c7c004360 2023-12-19T17:58:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.582+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c880a89d0 con 0x7f1c7c004360 2023-12-19T17:58:18.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.582+0000 7f1c9159a640 1 -- 172.21.15.26:0/355915311 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c880a8d80 con 0x7f1c7c004360 2023-12-19T17:58:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 -- 172.21.15.26:0/355915311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c7c004360 msgr2=0x7f1c7c004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 --2- 172.21.15.26:0/355915311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c7c004360 0x7f1c7c004830 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f1c88001860 tx=0x7f1c8801ec50 comp rx=0 tx=0).stop 2023-12-19T17:58:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c92d9d640 1 -- 172.21.15.26:0/355915311 reap_dead start 2023-12-19T17:58:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 -- 172.21.15.26:0/355915311 shutdown_connections 2023-12-19T17:58:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 -- 172.21.15.26:0/355915311 >> 172.21.15.26:0/355915311 conn(0x7f1c940658e0 msgr2=0x7f1c94065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 -- 172.21.15.26:0/355915311 shutdown_connections 2023-12-19T17:58:18.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 -- 172.21.15.26:0/355915311 wait complete. 2023-12-19T17:58:18.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 Processor -- start 2023-12-19T17:58:18.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 -- start start 2023-12-19T17:58:18.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c7c004360 0x7f1c941eda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c940c3660 0x7f1c940c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c941560c0 0x7f1c94156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94143710 con 0x7f1c940c3660 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940c4020 con 0x7f1c7c004360 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c98c2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c94156a80 con 0x7f1c941560c0 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c91d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c940c3660 0x7f1c940c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c91d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c940c3660 0x7f1c940c3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38770/0 (socket says 172.21.15.26:38770) 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c91d9b640 1 -- 172.21.15.26:0/3144504134 learned_addr learned my addr 172.21.15.26:0/3144504134 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.586+0000 7f1c9259c640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c7c004360 0x7f1c941eda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c92d9d640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c941560c0 0x7f1c94156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c91d9b640 1 -- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c941560c0 msgr2=0x7f1c94156540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c91d9b640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c941560c0 0x7f1c94156540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c91d9b640 1 -- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c7c004360 msgr2=0x7f1c941eda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c91d9b640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c7c004360 0x7f1c941eda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c91d9b640 1 -- 172.21.15.26:0/3144504134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94156cd0 con 0x7f1c940c3660 2023-12-19T17:58:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c92d9d640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c941560c0 0x7f1c94156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:18.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c91d9b640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c940c3660 0x7f1c940c3ae0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f1c880b3040 tx=0x7f1c7c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:18.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c82ffd640 1 -- 172.21.15.26:0/3144504134 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88005310 con 0x7f1c940c3660 2023-12-19T17:58:18.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c82ffd640 1 -- 172.21.15.26:0/3144504134 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c88005470 con 0x7f1c940c3660 2023-12-19T17:58:18.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c82ffd640 1 -- 172.21.15.26:0/3144504134 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c88006a00 con 0x7f1c940c3660 2023-12-19T17:58:18.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c98c2d640 1 -- 172.21.15.26:0/3144504134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c94156fb0 con 0x7f1c940c3660 2023-12-19T17:58:18.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c98c2d640 1 -- 172.21.15.26:0/3144504134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c94157f10 con 0x7f1c940c3660 2023-12-19T17:58:18.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c98c2d640 1 -- 172.21.15.26:0/3144504134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c94066200 con 0x7f1c940c3660 2023-12-19T17:58:18.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c82ffd640 1 -- 172.21.15.26:0/3144504134 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1c880b7040 con 0x7f1c940c3660 2023-12-19T17:58:18.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c82ffd640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1c74065bb0 0x7f1c74068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:18.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.590+0000 7f1c82ffd640 1 -- 172.21.15.26:0/3144504134 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f1c8811fb60 con 0x7f1c940c3660 2023-12-19T17:58:18.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.594+0000 7f1c9259c640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1c74065bb0 0x7f1c74068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:18.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.594+0000 7f1c9259c640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1c74065bb0 0x7f1c74068070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1c84000f70 tx=0x7f1c84000ee0 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:18.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.594+0000 7f1c82ffd640 1 -- 172.21.15.26:0/3144504134 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1c880ed140 con 0x7f1c940c3660 2023-12-19T17:58:18.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.786+0000 7f1c98c2d640 1 -- 172.21.15.26:0/3144504134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c94074270 con 0x7f1c940c3660 2023-12-19T17:58:18.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.786+0000 7f1c82ffd640 1 -- 172.21.15.26:0/3144504134 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f1c880b5090 con 0x7f1c940c3660 2023-12-19T17:58:18.790 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T17:58:18.790 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2924688599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2924688599},{"type":"v1","addr":"172.21.15.26:6835","nonce":2924688599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/330338341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":330338341},{"type":"v1","addr":"172.21.15.160:6836","nonce":330338341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9031,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/887954085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":887954085},{"type":"v1","addr":"172.21.15.160:6837","nonce":887954085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:58:13.473962+0000","modified":"2023-12-19T17:58:16.484662+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9513},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9513":{"gid":9513,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/3354708287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3354708287},{"type":"v1","addr":"172.21.15.26:6834","nonce":3354708287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-19T17:58:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 80 2023-12-19T17:58:18.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.786+0000 7f1c98c2d640 1 -- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1c74065bb0 msgr2=0x7f1c74068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:18.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.786+0000 7f1c98c2d640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1c74065bb0 0x7f1c74068070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1c84000f70 tx=0x7f1c84000ee0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.786+0000 7f1c98c2d640 1 -- 172.21.15.26:0/3144504134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c940c3660 msgr2=0x7f1c940c3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.786+0000 7f1c98c2d640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c940c3660 0x7f1c940c3ae0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f1c880b3040 tx=0x7f1c7c005800 comp rx=0 tx=0).stop 2023-12-19T17:58:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.786+0000 7f1c98c2d640 1 -- 172.21.15.26:0/3144504134 shutdown_connections 2023-12-19T17:58:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.786+0000 7f1c98c2d640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f1c74065bb0 0x7f1c74068070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.786+0000 7f1c98c2d640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1c941560c0 0x7f1c94156540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.786+0000 7f1c98c2d640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c940c3660 0x7f1c940c3ae0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.786+0000 7f1c98c2d640 1 --2- 172.21.15.26:0/3144504134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c7c004360 0x7f1c941eda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.786+0000 7f1c98c2d640 1 -- 172.21.15.26:0/3144504134 >> 172.21.15.26:0/3144504134 conn(0x7f1c940658e0 msgr2=0x7f1c9415a910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.786+0000 7f1c98c2d640 1 -- 172.21.15.26:0/3144504134 shutdown_connections 2023-12-19T17:58:18.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:18.790+0000 7f1c98c2d640 1 -- 172.21.15.26:0/3144504134 wait complete. 2023-12-19T17:58:19.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T17:58:13.473962+0000', 'modified': '2023-12-19T17:58:16.484662+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9513}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9513': {'gid': 9513, 'name': 'a', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6834/3354708287', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 3354708287}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 3354708287}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T17:58:19.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T17:58:19.214 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-19T17:58:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.358+0000 7fe714bcc640 1 Processor -- start 2023-12-19T17:58:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.358+0000 7fe714bcc640 1 -- start start 2023-12-19T17:58:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.358+0000 7fe714bcc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe710154220 0x7fe71015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.358+0000 7fe714bcc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe71006b3d0 0x7fe7100ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.358+0000 7fe714bcc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71006bc30 con 0x7fe7100cec40 2023-12-19T17:58:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.358+0000 7fe714bcc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7100cf0d0 con 0x7fe71006b3d0 2023-12-19T17:58:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.358+0000 7fe714bcc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100cc6e0 con 0x7fe710154220 2023-12-19T17:58:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.358+0000 7fe70dd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe71006b3d0 0x7fe7100ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.358+0000 7fe70dd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe71006b3d0 0x7fe7100ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38778/0 (socket says 172.21.15.26:38778) 2023-12-19T17:58:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.358+0000 7fe70dd9b640 1 -- 172.21.15.26:0/2448967161 learned_addr learned my addr 172.21.15.26:0/2448967161 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.362+0000 7fe70e59c640 1 --2- 172.21.15.26:0/2448967161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe710154220 0x7fe71015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.362+0000 7fe70dd9b640 1 -- 172.21.15.26:0/2448967161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe710154220 msgr2=0x7fe71015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.362+0000 7fe70dd9b640 1 --2- 172.21.15.26:0/2448967161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe710154220 0x7fe71015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.362+0000 7fe70dd9b640 1 -- 172.21.15.26:0/2448967161 >> v1:172.21.15.160:6789/0 conn(0x7fe7100cec40 legacy=0x7fe71015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.362+0000 7fe70dd9b640 1 -- 172.21.15.26:0/2448967161 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71015afc0 con 0x7fe71006b3d0 2023-12-19T17:58:19.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.362+0000 7fe70dd9b640 1 --2- 172.21.15.26:0/2448967161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe71006b3d0 0x7fe7100ce700 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fe704009870 tx=0x7fe70401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2bf97da5650ac1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:19.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.362+0000 7fe70d59a640 1 -- 172.21.15.26:0/2448967161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040a7070 con 0x7fe71006b3d0 2023-12-19T17:58:19.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.362+0000 7fe70d59a640 1 -- 172.21.15.26:0/2448967161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7040ab430 con 0x7fe71006b3d0 2023-12-19T17:58:19.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.362+0000 7fe70d59a640 1 -- 172.21.15.26:0/2448967161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040ab7c0 con 0x7fe71006b3d0 2023-12-19T17:58:19.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.362+0000 7fe714bcc640 1 -- 172.21.15.26:0/2448967161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe71006b3d0 msgr2=0x7fe7100ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.362+0000 7fe714bcc640 1 --2- 172.21.15.26:0/2448967161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe71006b3d0 0x7fe7100ce700 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fe704009870 tx=0x7fe70401c510 comp rx=0 tx=0).stop 2023-12-19T17:58:19.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 -- 172.21.15.26:0/2448967161 shutdown_connections 2023-12-19T17:58:19.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 --2- 172.21.15.26:0/2448967161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe71006b3d0 0x7fe7100ce700 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 --2- 172.21.15.26:0/2448967161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe710154220 0x7fe71015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 -- 172.21.15.26:0/2448967161 >> 172.21.15.26:0/2448967161 conn(0x7fe7100658e0 msgr2=0x7fe710065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:19.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 -- 172.21.15.26:0/2448967161 shutdown_connections 2023-12-19T17:58:19.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 -- 172.21.15.26:0/2448967161 wait complete. 2023-12-19T17:58:19.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 Processor -- start 2023-12-19T17:58:19.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 -- start start 2023-12-19T17:58:19.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe71006b3d0 0x7fe71014ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7100cec40 0x7fe7100b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe710154220 0x7fe71014ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71014b3b0 con 0x7fe7100cec40 2023-12-19T17:58:19.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100b3800 con 0x7fe710154220 2023-12-19T17:58:19.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe714bcc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100b2220 con 0x7fe71006b3d0 2023-12-19T17:58:19.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe70dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7100cec40 0x7fe7100b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe70dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7100cec40 0x7fe7100b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38782/0 (socket says 172.21.15.26:38782) 2023-12-19T17:58:19.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe70dd9b640 1 -- 172.21.15.26:0/3147448345 learned_addr learned my addr 172.21.15.26:0/3147448345 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:19.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe70ed9d640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe710154220 0x7fe71014ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe70e59c640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe71006b3d0 0x7fe71014ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe70dd9b640 1 -- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe71006b3d0 msgr2=0x7fe71014ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe70dd9b640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe71006b3d0 0x7fe71014ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe70dd9b640 1 -- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe710154220 msgr2=0x7fe71014ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe70dd9b640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe710154220 0x7fe71014ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.366+0000 7fe70dd9b640 1 -- 172.21.15.26:0/3147448345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7040094d0 con 0x7fe7100cec40 2023-12-19T17:58:19.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.370+0000 7fe70dd9b640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7100cec40 0x7fe7100b32c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fe70401a140 tx=0x7fe7040192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:19.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.370+0000 7fe6feffd640 1 -- 172.21.15.26:0/3147448345 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040ba070 con 0x7fe7100cec40 2023-12-19T17:58:19.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.370+0000 7fe6feffd640 1 -- 172.21.15.26:0/3147448345 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7040b4420 con 0x7fe7100cec40 2023-12-19T17:58:19.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.370+0000 7fe6feffd640 1 -- 172.21.15.26:0/3147448345 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040b47d0 con 0x7fe7100cec40 2023-12-19T17:58:19.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.370+0000 7fe714bcc640 1 -- 172.21.15.26:0/3147448345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7100b2500 con 0x7fe7100cec40 2023-12-19T17:58:19.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.370+0000 7fe714bcc640 1 -- 172.21.15.26:0/3147448345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7100b2ac0 con 0x7fe7100cec40 2023-12-19T17:58:19.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.370+0000 7fe6feffd640 1 -- 172.21.15.26:0/3147448345 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe7040b4930 con 0x7fe7100cec40 2023-12-19T17:58:19.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.370+0000 7fe6feffd640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe6f0065b80 0x7fe6f0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.370+0000 7fe6feffd640 1 -- 172.21.15.26:0/3147448345 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fe70411cf70 con 0x7fe7100cec40 2023-12-19T17:58:19.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.370+0000 7fe70e59c640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe6f0065b80 0x7fe6f0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.370+0000 7fe717f1f000 1 Processor -- start 2023-12-19T17:58:19.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 -- start start 2023-12-19T17:58:19.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70e59c640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe6f0065b80 0x7fe6f0068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe700009b60 tx=0x7fe700009190 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 0x5599d249dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249e1a0 0x5599d244ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d244b1c0 0x5599d244b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d226ec90 con 0x5599d249d880 2023-12-19T17:58:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d22a0220 con 0x5599d244b1c0 2023-12-19T17:58:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599d229d7f0 con 0x5599d249e1a0 2023-12-19T17:58:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 0x5599d249dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 0x5599d249dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38786/0 (socket says 172.21.15.26:38786) 2023-12-19T17:58:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70e59c640 1 -- 172.21.15.26:0/4290844746 learned_addr learned my addr 172.21.15.26:0/4290844746 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70dd9b640 1 --2- 172.21.15.26:0/4290844746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249e1a0 0x5599d244ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70ed9d640 1 --2- 172.21.15.26:0/4290844746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d244b1c0 0x5599d244b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70e59c640 1 -- 172.21.15.26:0/4290844746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249e1a0 msgr2=0x5599d244ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70e59c640 1 --2- 172.21.15.26:0/4290844746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249e1a0 0x5599d244ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70e59c640 1 -- 172.21.15.26:0/4290844746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d244b1c0 msgr2=0x5599d244b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70e59c640 1 --2- 172.21.15.26:0/4290844746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d244b1c0 0x5599d244b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70e59c640 1 -- 172.21.15.26:0/4290844746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599d1f95890 con 0x5599d249d880 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70dd9b640 1 --2- 172.21.15.26:0/4290844746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249e1a0 0x5599d244ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70ed9d640 1 --2- 172.21.15.26:0/4290844746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d244b1c0 0x5599d244b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe70e59c640 1 --2- 172.21.15.26:0/4290844746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 0x5599d249dc60 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fe700011ec0 tx=0x7fe700014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b8532f2ddabacf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe6effff640 1 -- 172.21.15.26:0/4290844746 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70001c040 con 0x5599d249d880 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe6effff640 1 -- 172.21.15.26:0/4290844746 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe700003830 con 0x5599d249d880 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe6effff640 1 -- 172.21.15.26:0/4290844746 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700003ba0 con 0x5599d249d880 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 -- 172.21.15.26:0/4290844746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 msgr2=0x5599d249dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 --2- 172.21.15.26:0/4290844746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 0x5599d249dc60 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fe700011ec0 tx=0x7fe700014000 comp rx=0 tx=0).stop 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 -- 172.21.15.26:0/4290844746 shutdown_connections 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 --2- 172.21.15.26:0/4290844746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d244b1c0 0x5599d244b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 --2- 172.21.15.26:0/4290844746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249e1a0 0x5599d244ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 --2- 172.21.15.26:0/4290844746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 0x5599d249dc60 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 -- 172.21.15.26:0/4290844746 >> 172.21.15.26:0/4290844746 conn(0x5599d2451950 msgr2=0x5599d2449560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 -- 172.21.15.26:0/4290844746 shutdown_connections 2023-12-19T17:58:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 -- 172.21.15.26:0/4290844746 wait complete. 2023-12-19T17:58:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.374+0000 7fe717f1f000 1 -- start start 2023-12-19T17:58:19.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 1 Processor -- start 2023-12-19T17:58:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d244b1c0 0x5599d2450720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 0x5599d245f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d249e1a0 0x5599d245fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d22990e0 con 0x5599d249d880 2023-12-19T17:58:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d2238310 con 0x5599d249e1a0 2023-12-19T17:58:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599d2256770 con 0x5599d244b1c0 2023-12-19T17:58:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 10 client.0 _ll_register_callbacks cb 0x5599d2436b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T17:58:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 0x5599d245f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 0x5599d245f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38790/0 (socket says 172.21.15.26:38790) 2023-12-19T17:58:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70dd9b640 1 -- 172.21.15.26:0/2513372677 learned_addr learned my addr 172.21.15.26:0/2513372677 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70e59c640 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d244b1c0 0x5599d2450720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70ed9d640 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d249e1a0 0x5599d245fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70dd9b640 1 -- 172.21.15.26:0/2513372677 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d244b1c0 msgr2=0x5599d2450720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70dd9b640 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d244b1c0 0x5599d2450720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70dd9b640 1 -- 172.21.15.26:0/2513372677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d249e1a0 msgr2=0x5599d245fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70dd9b640 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d249e1a0 0x5599d245fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70dd9b640 1 -- 172.21.15.26:0/2513372677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe700011ba0 con 0x5599d249d880 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70ed9d640 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d249e1a0 0x5599d245fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70dd9b640 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 0x5599d245f890 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe7040ede00 tx=0x7fe7040ee1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213a32b462c56969 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe6ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe6ed7fa640 1 -- 172.21.15.26:0/2513372677 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040ed7f0 con 0x5599d249d880 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 1 -- 172.21.15.26:0/2513372677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599d2268c10 con 0x5599d249d880 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe6ed7fa640 1 -- 172.21.15.26:0/2513372677 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fe7040f5ba0 con 0x5599d249d880 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe6ed7fa640 1 -- 172.21.15.26:0/2513372677 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7040f5d60 con 0x5599d249d880 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 10 client.9648 fetch_fsmap learned FSMap version 80 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 1 -- 172.21.15.26:0/2513372677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x5599d225e260 con 0x5599d249d880 2023-12-19T17:58:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe6ed7fa640 1 -- 172.21.15.26:0/2513372677 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe704109020 con 0x5599d249d880 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe6ed7fa640 1 -- 172.21.15.26:0/2513372677 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 80) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe7040f2bc0 con 0x5599d249d880 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 10 client.9648 fetch_fsmap finished waiting for FSMap version 80 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 10 client.9648 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 20 client.9648 populate_metadata read hostname 'smithi026' 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.26:6832/3354708287,v1:172.21.15.26:6834/3354708287] conn(0x5599d24a50e0 0x5599d24a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 4 client.9648 mds_command: new command op to 9513 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe717f1f000 1 -- 172.21.15.26:0/2513372677 --> [v2:172.21.15.26:6832/3354708287,v1:172.21.15.26:6834/3354708287] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599d2297a30 con 0x5599d24a50e0 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70e59c640 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.26:6832/3354708287,v1:172.21.15.26:6834/3354708287] conn(0x5599d24a50e0 0x5599d24a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe70e59c640 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.26:6832/3354708287,v1:172.21.15.26:6834/3354708287] conn(0x5599d24a50e0 0x5599d24a54c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f372a6e60aa89c4 server_cookie=ea34572afbbff5cd in_seq=0 out_seq=0 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.378+0000 7fe6ed7fa640 10 client.9648 ms_handle_connect on v2:172.21.15.26:6832/3354708287 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe6ed7fa640 1 -- 172.21.15.26:0/2513372677 <== mds.0 v2:172.21.15.26:6832/3354708287 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5599d2297a30 con 0x5599d24a50e0 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe6ed7fa640 10 client.9648 handle_command_reply: tid=0 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe717f1f000 1 client.9648 shutdown 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe717f1f000 1 -- 172.21.15.26:0/2513372677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 msgr2=0x5599d245f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe717f1f000 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 0x5599d245f890 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe7040ede00 tx=0x7fe7040ee1c0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe717f1f000 1 -- 172.21.15.26:0/2513372677 shutdown_connections 2023-12-19T17:58:19.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe717f1f000 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.26:6832/3354708287,v1:172.21.15.26:6834/3354708287] conn(0x5599d24a50e0 0x5599d24a54c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe717f1f000 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d249e1a0 0x5599d245fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe717f1f000 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249d880 0x5599d245f890 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe717f1f000 1 --2- 172.21.15.26:0/2513372677 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d244b1c0 0x5599d2450720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe6ed7fa640 0 client.9648 ms_handle_reset on v2:172.21.15.26:6832/3354708287 2023-12-19T17:58:19.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe717f1f000 1 -- 172.21.15.26:0/2513372677 >> 172.21.15.26:0/2513372677 conn(0x5599d2451950 msgr2=0x5599d2449c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:19.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe717f1f000 1 -- 172.21.15.26:0/2513372677 shutdown_connections 2023-12-19T17:58:19.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe717f1f000 1 -- 172.21.15.26:0/2513372677 wait complete. 2023-12-19T17:58:19.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.382+0000 7fe717f1f000 20 client.9648 trim_cache size 0 max 16384 2023-12-19T17:58:19.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 Processor -- start 2023-12-19T17:58:19.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 -- start start 2023-12-19T17:58:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d244b1c0 0x5599d24f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d249d880 0x5599d24f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249e1a0 0x5599d24f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d1f56940 con 0x5599d244b1c0 2023-12-19T17:58:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d22990e0 con 0x5599d249d880 2023-12-19T17:58:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599d244dd70 con 0x5599d249e1a0 2023-12-19T17:58:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe70e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d244b1c0 0x5599d24f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe70e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d244b1c0 0x5599d24f5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38796/0 (socket says 172.21.15.26:38796) 2023-12-19T17:58:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe70e59c640 1 -- 172.21.15.26:0/4009666032 learned_addr learned my addr 172.21.15.26:0/4009666032 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe70e59c640 1 -- 172.21.15.26:0/4009666032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249e1a0 msgr2=0x5599d24f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe70e59c640 1 --2- 172.21.15.26:0/4009666032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249e1a0 0x5599d24f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe70e59c640 1 -- 172.21.15.26:0/4009666032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d249d880 msgr2=0x5599d24f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe70dd9b640 1 --2- 172.21.15.26:0/4009666032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d249d880 0x5599d24f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe70e59c640 1 --2- 172.21.15.26:0/4009666032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d249d880 0x5599d24f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe70e59c640 1 -- 172.21.15.26:0/4009666032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599d22443b0 con 0x5599d244b1c0 2023-12-19T17:58:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe70dd9b640 1 --2- 172.21.15.26:0/4009666032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d249d880 0x5599d24f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T17:58:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe70e59c640 1 --2- 172.21.15.26:0/4009666032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d244b1c0 0x5599d24f5b00 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fe700024040 tx=0x7fe700016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103d683e1e182e3b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe6fcff9640 1 -- 172.21.15.26:0/4009666032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700003b10 con 0x5599d244b1c0 2023-12-19T17:58:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe6fcff9640 1 -- 172.21.15.26:0/4009666032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe700033040 con 0x5599d244b1c0 2023-12-19T17:58:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe6fcff9640 1 -- 172.21.15.26:0/4009666032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700003e70 con 0x5599d244b1c0 2023-12-19T17:58:19.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 -- 172.21.15.26:0/4009666032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d244b1c0 msgr2=0x5599d24f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 --2- 172.21.15.26:0/4009666032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d244b1c0 0x5599d24f5b00 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fe700024040 tx=0x7fe700016000 comp rx=0 tx=0).stop 2023-12-19T17:58:19.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 -- 172.21.15.26:0/4009666032 shutdown_connections 2023-12-19T17:58:19.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 --2- 172.21.15.26:0/4009666032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249e1a0 0x5599d24f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 --2- 172.21.15.26:0/4009666032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d249d880 0x5599d24f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 --2- 172.21.15.26:0/4009666032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d244b1c0 0x5599d24f5b00 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 -- 172.21.15.26:0/4009666032 >> 172.21.15.26:0/4009666032 conn(0x5599d2451950 msgr2=0x5599d245f750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:19.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 -- 172.21.15.26:0/4009666032 shutdown_connections 2023-12-19T17:58:19.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 -- 172.21.15.26:0/4009666032 wait complete. 2023-12-19T17:58:19.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.398+0000 7fe717f1f000 1 -- start start 2023-12-19T17:58:19.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 1 Processor -- start 2023-12-19T17:58:19.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d244b1c0 0x5599d24dda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249d880 0x5599d24de360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249e1a0 0x5599d24de8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d2294a50 con 0x5599d249e1a0 2023-12-19T17:58:19.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599d2241860 con 0x5599d244b1c0 2023-12-19T17:58:19.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5599d223e570 con 0x5599d249d880 2023-12-19T17:58:19.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 10 client.0 _ll_register_callbacks cb 0x5599d2247640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T17:58:19.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe70ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249e1a0 0x5599d24de8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe70ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249e1a0 0x5599d24de8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38812/0 (socket says 172.21.15.26:38812) 2023-12-19T17:58:19.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe70ed9d640 1 -- 172.21.15.26:0/2107581529 learned_addr learned my addr 172.21.15.26:0/2107581529 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:19.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe70dd9b640 1 --2- 172.21.15.26:0/2107581529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249d880 0x5599d24de360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe70e59c640 1 --2- 172.21.15.26:0/2107581529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d244b1c0 0x5599d24dda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe70ed9d640 1 -- 172.21.15.26:0/2107581529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249d880 msgr2=0x5599d24de360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:19.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe70ed9d640 1 --2- 172.21.15.26:0/2107581529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249d880 0x5599d24de360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe70ed9d640 1 -- 172.21.15.26:0/2107581529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d244b1c0 msgr2=0x5599d24dda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:19.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe70ed9d640 1 --2- 172.21.15.26:0/2107581529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d244b1c0 0x5599d24dda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe70ed9d640 1 -- 172.21.15.26:0/2107581529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599d1f95890 con 0x5599d249e1a0 2023-12-19T17:58:19.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe70ed9d640 1 --2- 172.21.15.26:0/2107581529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249e1a0 0x5599d24de8a0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fe708001450 tx=0x7fe708001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db5473f79ddcfa2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:19.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe6ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T17:58:19.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe6ed7fa640 1 -- 172.21.15.26:0/2107581529 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708016660 con 0x5599d249e1a0 2023-12-19T17:58:19.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe6ed7fa640 1 -- 172.21.15.26:0/2107581529 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fe7080167c0 con 0x5599d249e1a0 2023-12-19T17:58:19.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 1 -- 172.21.15.26:0/2107581529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599d2450a60 con 0x5599d249e1a0 2023-12-19T17:58:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe6ed7fa640 1 -- 172.21.15.26:0/2107581529 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe708016980 con 0x5599d249e1a0 2023-12-19T17:58:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 10 client.9654 fetch_fsmap learned FSMap version 80 2023-12-19T17:58:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 1 -- 172.21.15.26:0/2107581529 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) v3 -- 0x5599d1f95cc0 con 0x5599d249e1a0 2023-12-19T17:58:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe6ed7fa640 1 -- 172.21.15.26:0/2107581529 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5599d2450a60 con 0x5599d249e1a0 2023-12-19T17:58:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe6ed7fa640 1 -- 172.21.15.26:0/2107581529 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 80) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe708016ae0 con 0x5599d249e1a0 2023-12-19T17:58:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 10 client.9654 fetch_fsmap finished waiting for FSMap version 80 2023-12-19T17:58:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 10 client.9654 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T17:58:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 20 client.9654 populate_metadata read hostname 'smithi026' 2023-12-19T17:58:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 1 --2- 172.21.15.26:0/2107581529 >> [v2:172.21.15.26:6832/3354708287,v1:172.21.15.26:6834/3354708287] conn(0x5599d24dede0 0x5599d24df1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 4 client.9654 mds_command: new command op to 9513 tid=0[{"prefix": "status"}] 2023-12-19T17:58:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe717f1f000 1 -- 172.21.15.26:0/2107581529 --> [v2:172.21.15.26:6832/3354708287,v1:172.21.15.26:6834/3354708287] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5599d225ee00 con 0x5599d24dede0 2023-12-19T17:58:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.402+0000 7fe70e59c640 1 --2- 172.21.15.26:0/2107581529 >> [v2:172.21.15.26:6832/3354708287,v1:172.21.15.26:6834/3354708287] conn(0x5599d24dede0 0x5599d24df1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe6ed7fa640 10 client.9654 ms_handle_connect on v2:172.21.15.26:6832/3354708287 2023-12-19T17:58:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe70e59c640 1 --2- 172.21.15.26:0/2107581529 >> [v2:172.21.15.26:6832/3354708287,v1:172.21.15.26:6834/3354708287] conn(0x5599d24dede0 0x5599d24df1c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8671ad4908f63b server_cookie=a5955295466ed51b in_seq=0 out_seq=0 2023-12-19T17:58:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe6ed7fa640 1 -- 172.21.15.26:0/2107581529 <== mds.0 v2:172.21.15.26:6832/3354708287 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5599d225ee00 con 0x5599d24dede0 2023-12-19T17:58:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe6ed7fa640 10 client.9654 handle_command_reply: tid=0 2023-12-19T17:58:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe717f1f000 1 client.9654 shutdown 2023-12-19T17:58:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe717f1f000 1 -- 172.21.15.26:0/2107581529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249e1a0 msgr2=0x5599d24de8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe717f1f000 1 --2- 172.21.15.26:0/2107581529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249e1a0 0x5599d24de8a0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fe708001450 tx=0x7fe708001c70 comp rx=0 tx=0).stop 2023-12-19T17:58:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe717f1f000 1 -- 172.21.15.26:0/2107581529 shutdown_connections 2023-12-19T17:58:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe717f1f000 1 --2- 172.21.15.26:0/2107581529 >> [v2:172.21.15.26:6832/3354708287,v1:172.21.15.26:6834/3354708287] conn(0x5599d24dede0 0x5599d24df1c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe717f1f000 1 --2- 172.21.15.26:0/2107581529 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5599d249e1a0 0x5599d24de8a0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe717f1f000 1 --2- 172.21.15.26:0/2107581529 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5599d249d880 0x5599d24de360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe717f1f000 1 --2- 172.21.15.26:0/2107581529 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5599d244b1c0 0x5599d24dda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe6ed7fa640 0 client.9654 ms_handle_reset on v2:172.21.15.26:6832/3354708287 2023-12-19T17:58:19.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe717f1f000 1 -- 172.21.15.26:0/2107581529 >> 172.21.15.26:0/2107581529 conn(0x5599d2451950 msgr2=0x5599d249a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:19.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe717f1f000 1 -- 172.21.15.26:0/2107581529 shutdown_connections 2023-12-19T17:58:19.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe717f1f000 1 -- 172.21.15.26:0/2107581529 wait complete. 2023-12-19T17:58:19.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe717f1f000 20 client.9654 trim_cache size 0 max 16384 2023-12-19T17:58:19.416 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T17:58:19.416 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T17:58:19.416 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T17:58:19.417 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 9513, 2023-12-19T17:58:19.417 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T17:58:19.417 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T17:58:19.417 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T17:58:19.417 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.9322692459999997, 2023-12-19T17:58:19.417 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 80, 2023-12-19T17:58:19.417 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 177, 2023-12-19T17:58:19.417 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 177, 2023-12-19T17:58:19.417 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 10.239672422 2023-12-19T17:58:19.417 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T17:58:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe714bcc640 1 -- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe6f0065b80 msgr2=0x7fe6f0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe714bcc640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe6f0065b80 0x7fe6f0068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe700009b60 tx=0x7fe700009190 comp rx=0 tx=0).stop 2023-12-19T17:58:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe714bcc640 1 -- 172.21.15.26:0/3147448345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7100cec40 msgr2=0x7fe7100b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe714bcc640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7100cec40 0x7fe7100b32c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fe70401a140 tx=0x7fe7040192b0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe714bcc640 1 -- 172.21.15.26:0/3147448345 shutdown_connections 2023-12-19T17:58:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe714bcc640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe710154220 0x7fe71014ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe714bcc640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7100cec40 0x7fe7100b32c0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe714bcc640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fe6f0065b80 0x7fe6f0068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe714bcc640 1 --2- 172.21.15.26:0/3147448345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe71006b3d0 0x7fe71014ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe714bcc640 1 -- 172.21.15.26:0/3147448345 >> 172.21.15.26:0/3147448345 conn(0x7fe7100658e0 msgr2=0x7fe710065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:19.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe714bcc640 1 -- 172.21.15.26:0/3147448345 shutdown_connections 2023-12-19T17:58:19.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.406+0000 7fe714bcc640 1 -- 172.21.15.26:0/3147448345 wait complete. 2023-12-19T17:58:19.429 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T17:58:19.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f629de20640 1 Processor -- start 2023-12-19T17:58:19.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f629de20640 1 -- start start 2023-12-19T17:58:19.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f629de20640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629815a7c0 0x7f629815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f629de20640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f629806f9d0 0x7f62980ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f629de20640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f62980c4070 0x7f62980cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f629de20640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629815e2f0 con 0x7f629806f9d0 2023-12-19T17:58:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f629de20640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62980c5b40 con 0x7f62980c4070 2023-12-19T17:58:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f629de20640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62980c5e20 con 0x7f629815a7c0 2023-12-19T17:58:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f6296ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f629806f9d0 0x7f62980ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f62977fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629815a7c0 0x7f629815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f6296ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f629806f9d0 0x7f62980ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38828/0 (socket says 172.21.15.26:38828) 2023-12-19T17:58:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f6296ffd640 1 -- 172.21.15.26:0/937475494 learned_addr learned my addr 172.21.15.26:0/937475494 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f6297fff640 1 --2- 172.21.15.26:0/937475494 >> v2:172.21.15.160:3300/0 conn(0x7f62980c4070 0x7f62980cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f6296ffd640 1 -- 172.21.15.26:0/937475494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629815a7c0 msgr2=0x7f629815ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:19.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f6296ffd640 1 --2- 172.21.15.26:0/937475494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629815a7c0 0x7f629815ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f6296ffd640 1 -- 172.21.15.26:0/937475494 >> v2:172.21.15.160:3300/0 conn(0x7f62980c4070 msgr2=0x7f62980cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:19.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f6296ffd640 1 --2- 172.21.15.26:0/937475494 >> v2:172.21.15.160:3300/0 conn(0x7f62980c4070 0x7f62980cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.570+0000 7f6296ffd640 1 -- 172.21.15.26:0/937475494 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62980ce770 con 0x7f629806f9d0 2023-12-19T17:58:19.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.574+0000 7f6296ffd640 1 --2- 172.21.15.26:0/937475494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629806f9d0 0x7f62980ceff0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f62980ce430 tx=0x7f628c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39390408b78c8399 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:19.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.574+0000 7f62967fc640 1 -- 172.21.15.26:0/937475494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c0ab600 con 0x7f629806f9d0 2023-12-19T17:58:19.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.574+0000 7f62967fc640 1 -- 172.21.15.26:0/937475494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628c0ab760 con 0x7f629806f9d0 2023-12-19T17:58:19.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.574+0000 7f62967fc640 1 -- 172.21.15.26:0/937475494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c0abaf0 con 0x7f629806f9d0 2023-12-19T17:58:19.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.574+0000 7f629de20640 1 -- 172.21.15.26:0/937475494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629806f9d0 msgr2=0x7f62980ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.574+0000 7f629de20640 1 --2- 172.21.15.26:0/937475494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629806f9d0 0x7f62980ceff0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f62980ce430 tx=0x7f628c01c440 comp rx=0 tx=0).stop 2023-12-19T17:58:19.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.574+0000 7f629de20640 1 -- 172.21.15.26:0/937475494 shutdown_connections 2023-12-19T17:58:19.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.574+0000 7f629de20640 1 --2- 172.21.15.26:0/937475494 >> v2:172.21.15.160:3300/0 conn(0x7f62980c4070 0x7f62980cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.574+0000 7f629de20640 1 --2- 172.21.15.26:0/937475494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629806f9d0 0x7f62980ceff0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.574+0000 7f629de20640 1 --2- 172.21.15.26:0/937475494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629815a7c0 0x7f629815ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.574+0000 7f629de20640 1 -- 172.21.15.26:0/937475494 >> 172.21.15.26:0/937475494 conn(0x7f629806d5d0 msgr2=0x7f6298069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:19.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.574+0000 7f629de20640 1 -- 172.21.15.26:0/937475494 shutdown_connections 2023-12-19T17:58:19.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.574+0000 7f629de20640 1 -- 172.21.15.26:0/937475494 wait complete. 2023-12-19T17:58:19.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f629de20640 1 Processor -- start 2023-12-19T17:58:19.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f629de20640 1 -- start start 2023-12-19T17:58:19.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f629de20640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629806f9d0 0x7f629814ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f629de20640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62980c4070 0x7f62980c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f629de20640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629815a7c0 0x7f6298158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f629de20640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62980c9040 con 0x7f629806f9d0 2023-12-19T17:58:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f629de20640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62980c7400 con 0x7f62980c4070 2023-12-19T17:58:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f629de20640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629814c150 con 0x7f629815a7c0 2023-12-19T17:58:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f62977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629806f9d0 0x7f629814ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f62977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629806f9d0 0x7f629814ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38834/0 (socket says 172.21.15.26:38834) 2023-12-19T17:58:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f62977fe640 1 -- 172.21.15.26:0/2491480477 learned_addr learned my addr 172.21.15.26:0/2491480477 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f6297fff640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629815a7c0 0x7f6298158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f6296ffd640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62980c4070 0x7f62980c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f62977fe640 1 -- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629815a7c0 msgr2=0x7f6298158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f62977fe640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629815a7c0 0x7f6298158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f62977fe640 1 -- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62980c4070 msgr2=0x7f62980c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f62977fe640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62980c4070 0x7f62980c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f62977fe640 1 -- 172.21.15.26:0/2491480477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0094d0 con 0x7f629806f9d0 2023-12-19T17:58:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f6296ffd640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62980c4070 0x7f62980c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f62977fe640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629806f9d0 0x7f629814ac50 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f628800e930 tx=0x7f628800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f627ffff640 1 -- 172.21.15.26:0/2491480477 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288009690 con 0x7f629806f9d0 2023-12-19T17:58:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f627ffff640 1 -- 172.21.15.26:0/2491480477 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628800ca00 con 0x7f629806f9d0 2023-12-19T17:58:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f627ffff640 1 -- 172.21.15.26:0/2491480477 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800cd90 con 0x7f629806f9d0 2023-12-19T17:58:19.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f629de20640 1 -- 172.21.15.26:0/2491480477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629814c430 con 0x7f629806f9d0 2023-12-19T17:58:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f629de20640 1 -- 172.21.15.26:0/2491480477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629814ca40 con 0x7f629806f9d0 2023-12-19T17:58:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.578+0000 7f629de20640 1 -- 172.21.15.26:0/2491480477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6298074270 con 0x7f629806f9d0 2023-12-19T17:58:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.582+0000 7f627ffff640 1 -- 172.21.15.26:0/2491480477 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6288013470 con 0x7f629806f9d0 2023-12-19T17:58:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.582+0000 7f627ffff640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6284065b80 0x7f6284068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.582+0000 7f627ffff640 1 -- 172.21.15.26:0/2491480477 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f6288083b40 con 0x7f629806f9d0 2023-12-19T17:58:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.582+0000 7f6296ffd640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6284065b80 0x7f6284068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.582+0000 7f6296ffd640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6284065b80 0x7f6284068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f628c0af000 tx=0x7f628c0a7000 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:19.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.586+0000 7f627ffff640 1 -- 172.21.15.26:0/2491480477 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6288051270 con 0x7f629806f9d0 2023-12-19T17:58:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.778+0000 7f629de20640 1 -- 172.21.15.26:0/2491480477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f62980b5270 con 0x7f629806f9d0 2023-12-19T17:58:19.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.778+0000 7f627ffff640 1 -- 172.21.15.26:0/2491480477 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6288021070 con 0x7f629806f9d0 2023-12-19T17:58:19.783 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T17:58:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.782+0000 7f629de20640 1 -- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6284065b80 msgr2=0x7f6284068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.782+0000 7f629de20640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6284065b80 0x7f6284068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f628c0af000 tx=0x7f628c0a7000 comp rx=0 tx=0).stop 2023-12-19T17:58:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.782+0000 7f629de20640 1 -- 172.21.15.26:0/2491480477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629806f9d0 msgr2=0x7f629814ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.782+0000 7f629de20640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629806f9d0 0x7f629814ac50 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f628800e930 tx=0x7f628800edf0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.782+0000 7f629de20640 1 -- 172.21.15.26:0/2491480477 shutdown_connections 2023-12-19T17:58:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.782+0000 7f629de20640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f6284065b80 0x7f6284068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.782+0000 7f629de20640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f629815a7c0 0x7f6298158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.782+0000 7f629de20640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62980c4070 0x7f62980c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.782+0000 7f629de20640 1 --2- 172.21.15.26:0/2491480477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f629806f9d0 0x7f629814ac50 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.782+0000 7f629de20640 1 -- 172.21.15.26:0/2491480477 >> 172.21.15.26:0/2491480477 conn(0x7f629806d5d0 msgr2=0x7f62981568d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.782+0000 7f629de20640 1 -- 172.21.15.26:0/2491480477 shutdown_connections 2023-12-19T17:58:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:19.782+0000 7f629de20640 1 -- 172.21.15.26:0/2491480477 wait complete. 2023-12-19T17:58:19.805 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-19T17:58:19.805 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T17:58:19.805 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T17:58:19.805 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T17:58:19.805 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T17:58:19.805 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-19T17:58:19.805 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-19T17:58:19.806 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T17:58:19.806 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T17:58:19.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:58:19.806 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T17:58:19.816 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T17:58:19.816 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T17:58:19.816 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T17:58:19.816 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 37841sec preferred_lft 37841sec 2023-12-19T17:58:19.817 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:58:19.818 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T17:58:19.819 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T17:58:19.819 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-19T17:58:19.819 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:58:19.819 INFO:teuthology.orchestra.run.smithi026.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T17:58:19.819 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T17:58:19.819 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-19T17:58:19.819 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T17:58:19.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:58:19.819 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T17:58:19.882 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T17:58:19.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:58:19.885 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T17:58:19.941 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T17:58:19.941 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T17:58:19.941 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T17:58:19.941 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T17:58:19.942 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T17:58:19.942 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T17:58:19.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:58:19.942 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T17:58:19.942 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T17:58:19.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T17:58:19.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T17:58:19.942 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T17:58:19.942 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T17:58:20.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:58:20.036 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T17:58:20.046 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T17:58:20.046 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T17:58:20.046 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T17:58:20.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:58:20.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T17:58:20.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T17:58:20.109 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T17:58:20.159 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T17:58:20.160 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T17:58:20.160 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T17:58:20.211 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T17:58:20.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:58:20.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T17:58:20.307 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T17:58:20.307 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T17:58:20.321 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T17:58:20.321 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T17:58:20.321 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T17:58:20.321 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T17:58:20.321 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T17:58:20.325 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T17:58:20.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:58:20.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T17:58:20.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:58:20.347 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T17:58:20.414 INFO:teuthology.orchestra.run.smithi026.stdout:client8099 2023-12-19T17:58:20.415 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T17:58:20.415 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T17:58:20.465 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-19T17:58:20.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41b31f0640 1 Processor -- start 2023-12-19T17:58:20.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41b31f0640 1 -- start start 2023-12-19T17:58:20.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41b31f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41ac154220 0x7f41ac15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41b31f0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41ac06bc30 con 0x7f41ac06b3d0 2023-12-19T17:58:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41b31f0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41ac0cc6e0 con 0x7f41ac0cee10 2023-12-19T17:58:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41b31f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac15afc0 con 0x7f41ac154220 2023-12-19T17:58:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41abfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f41ac06b3d0 0x7f41ac0ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34438/0 (socket says 172.21.15.26:34438) 2023-12-19T17:58:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41abfff640 1 -- 172.21.15.26:0/827593409 learned_addr learned my addr 172.21.15.26:0/827593409 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:20.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41b0f8c640 1 --2- 172.21.15.26:0/827593409 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41ac154220 0x7f41ac15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41ab7fe640 1 -- 172.21.15.26:0/827593409 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4030805925 0 0) 0x7f41ac06bc30 con 0x7f41ac06b3d0 2023-12-19T17:58:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41ab7fe640 1 -- 172.21.15.26:0/827593409 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f419c003530 con 0x7f41ac06b3d0 2023-12-19T17:58:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41b0f8c640 1 -- 172.21.15.26:0/827593409 >> v1:172.21.15.160:6789/0 conn(0x7f41ac0cee10 legacy=0x7f41ac15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41ab7fe640 1 -- 172.21.15.26:0/827593409 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 656276480 0 0) 0x7f41ac0cc6e0 con 0x7f41ac0cee10 2023-12-19T17:58:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41b0f8c640 1 -- 172.21.15.26:0/827593409 >> v1:172.21.15.26:6789/0 conn(0x7f41ac06b3d0 legacy=0x7f41ac0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:20.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.654+0000 7f41b0f8c640 1 -- 172.21.15.26:0/827593409 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac06b950 con 0x7f41ac154220 2023-12-19T17:58:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.658+0000 7f41b0f8c640 1 --2- 172.21.15.26:0/827593409 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41ac154220 0x7f41ac15aa80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f41a00097f0 tx=0x7f41a001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3b5e2da05aced3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.658+0000 7f41ab7fe640 1 -- 172.21.15.26:0/827593409 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a00aa5f0 con 0x7f41ac154220 2023-12-19T17:58:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.658+0000 7f41ab7fe640 1 -- 172.21.15.26:0/827593409 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a00aa750 con 0x7f41ac154220 2023-12-19T17:58:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.658+0000 7f41ab7fe640 1 -- 172.21.15.26:0/827593409 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a00aaae0 con 0x7f41ac154220 2023-12-19T17:58:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.658+0000 7f41b31f0640 1 -- 172.21.15.26:0/827593409 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41ac154220 msgr2=0x7f41ac15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:20.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.658+0000 7f41b31f0640 1 --2- 172.21.15.26:0/827593409 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41ac154220 0x7f41ac15aa80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f41a00097f0 tx=0x7f41a001c490 comp rx=0 tx=0).stop 2023-12-19T17:58:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.658+0000 7f41b31f0640 1 -- 172.21.15.26:0/827593409 shutdown_connections 2023-12-19T17:58:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.658+0000 7f41b31f0640 1 --2- 172.21.15.26:0/827593409 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41ac154220 0x7f41ac15aa80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.658+0000 7f41b31f0640 1 -- 172.21.15.26:0/827593409 >> 172.21.15.26:0/827593409 conn(0x7f41ac0658e0 msgr2=0x7f41ac065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.658+0000 7f41b31f0640 1 -- 172.21.15.26:0/827593409 shutdown_connections 2023-12-19T17:58:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.658+0000 7f41b31f0640 1 -- 172.21.15.26:0/827593409 wait complete. 2023-12-19T17:58:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41b31f0640 1 Processor -- start 2023-12-19T17:58:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41b31f0640 1 -- start start 2023-12-19T17:58:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41b31f0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41ac06b3d0 0x7f41ac15de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41b31f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41ac0cee10 0x7f41ac1500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41b31f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41ac154220 0x7f41ac151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41b31f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac15d0d0 con 0x7f41ac0cee10 2023-12-19T17:58:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41b31f0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac15d3b0 con 0x7f41ac06b3d0 2023-12-19T17:58:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41b31f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac15d690 con 0x7f41ac154220 2023-12-19T17:58:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41ac0cee10 0x7f41ac1500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41abfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41ac0cee10 0x7f41ac1500c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38846/0 (socket says 172.21.15.26:38846) 2023-12-19T17:58:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41abfff640 1 -- 172.21.15.26:0/1976327020 learned_addr learned my addr 172.21.15.26:0/1976327020 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41b0f8c640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41ac06b3d0 0x7f41ac15de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41b178d640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41ac154220 0x7f41ac151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41abfff640 1 -- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41ac154220 msgr2=0x7f41ac151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41abfff640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41ac154220 0x7f41ac151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41abfff640 1 -- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41ac06b3d0 msgr2=0x7f41ac15de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41abfff640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41ac06b3d0 0x7f41ac15de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41abfff640 1 -- 172.21.15.26:0/1976327020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a0009420 con 0x7f41ac0cee10 2023-12-19T17:58:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41abfff640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41ac0cee10 0x7f41ac1500c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f419800c3c0 tx=0x7f419800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41a97fa640 1 -- 172.21.15.26:0/1976327020 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198005cf0 con 0x7f41ac0cee10 2023-12-19T17:58:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41a97fa640 1 -- 172.21.15.26:0/1976327020 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4198005e50 con 0x7f41ac0cee10 2023-12-19T17:58:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41a97fa640 1 -- 172.21.15.26:0/1976327020 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41980072f0 con 0x7f41ac0cee10 2023-12-19T17:58:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41b31f0640 1 -- 172.21.15.26:0/1976327020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ac15d970 con 0x7f41ac0cee10 2023-12-19T17:58:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.662+0000 7f41b31f0640 1 -- 172.21.15.26:0/1976327020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ac150600 con 0x7f41ac0cee10 2023-12-19T17:58:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.666+0000 7f41b31f0640 1 -- 172.21.15.26:0/1976327020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ac074270 con 0x7f41ac0cee10 2023-12-19T17:58:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.666+0000 7f41a97fa640 1 -- 172.21.15.26:0/1976327020 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4198007450 con 0x7f41ac0cee10 2023-12-19T17:58:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.666+0000 7f41a97fa640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4190065b80 0x7f4190068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.666+0000 7f41a97fa640 1 -- 172.21.15.26:0/1976327020 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f4198082b40 con 0x7f41ac0cee10 2023-12-19T17:58:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.666+0000 7f41b0f8c640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4190065b80 0x7f4190068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.666+0000 7f41b0f8c640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4190065b80 0x7f4190068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f41a00ae000 tx=0x7f41a00a7000 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.670+0000 7f41a97fa640 1 -- 172.21.15.26:0/1976327020 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f41980501b0 con 0x7f41ac0cee10 2023-12-19T17:58:20.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.854+0000 7f41b31f0640 1 -- 172.21.15.26:0/1976327020 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f41ac1508b0 con 0x7f4190065b80 2023-12-19T17:58:20.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.854+0000 7f41a97fa640 1 -- 172.21.15.26:0/1976327020 <== mgr.8821 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f41ac1508b0 con 0x7f4190065b80 2023-12-19T17:58:20.859 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T17:58:20.859 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T17:58:20.859 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2023-12-19T17:58:20.859 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T17:58:20.859 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T17:58:20.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.858+0000 7f41b31f0640 1 -- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4190065b80 msgr2=0x7f4190068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:20.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.858+0000 7f41b31f0640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4190065b80 0x7f4190068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f41a00ae000 tx=0x7f41a00a7000 comp rx=0 tx=0).stop 2023-12-19T17:58:20.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.858+0000 7f41b31f0640 1 -- 172.21.15.26:0/1976327020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41ac0cee10 msgr2=0x7f41ac1500c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:20.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.858+0000 7f41b31f0640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41ac0cee10 0x7f41ac1500c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f419800c3c0 tx=0x7f419800c880 comp rx=0 tx=0).stop 2023-12-19T17:58:20.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.858+0000 7f41b31f0640 1 -- 172.21.15.26:0/1976327020 shutdown_connections 2023-12-19T17:58:20.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.858+0000 7f41b31f0640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f4190065b80 0x7f4190068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:20.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.858+0000 7f41b31f0640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41ac154220 0x7f41ac151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:20.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.858+0000 7f41b31f0640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41ac0cee10 0x7f41ac1500c0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:20.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.858+0000 7f41b31f0640 1 --2- 172.21.15.26:0/1976327020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41ac06b3d0 0x7f41ac15de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:20.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.858+0000 7f41b31f0640 1 -- 172.21.15.26:0/1976327020 >> 172.21.15.26:0/1976327020 conn(0x7f41ac0658e0 msgr2=0x7f41ac0ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:20.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.858+0000 7f41b31f0640 1 -- 172.21.15.26:0/1976327020 shutdown_connections 2023-12-19T17:58:20.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:20.858+0000 7f41b31f0640 1 -- 172.21.15.26:0/1976327020 wait complete. 2023-12-19T17:58:20.880 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-19T17:58:21.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.022+0000 7fae068f9640 1 Processor -- start 2023-12-19T17:58:21.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.022+0000 7fae068f9640 1 -- start start 2023-12-19T17:58:21.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.022+0000 7fae068f9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fae0006f6d0 0x7fae0015dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:21.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.022+0000 7fae068f9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0015e240 con 0x7fae000cb860 2023-12-19T17:58:21.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.022+0000 7fae068f9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae000c9520 con 0x7fae001599e0 2023-12-19T17:58:21.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.022+0000 7fae068f9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae000c9800 con 0x7fae0006f6d0 2023-12-19T17:58:21.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.026+0000 7fadff7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fae000cb860 0x7fae0015d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34448/0 (socket says 172.21.15.26:34448) 2023-12-19T17:58:21.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.026+0000 7fadff7fe640 1 -- 172.21.15.26:0/1203494256 learned_addr learned my addr 172.21.15.26:0/1203494256 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:21.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.026+0000 7fae04e96640 1 --2- 172.21.15.26:0/1203494256 >> v2:172.21.15.26:3300/0 conn(0x7fae0006f6d0 0x7fae0015dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:21.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.026+0000 7fadfeffd640 1 -- 172.21.15.26:0/1203494256 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328802095 0 0) 0x7fae0015e240 con 0x7fae000cb860 2023-12-19T17:58:21.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.026+0000 7fae04e96640 1 -- 172.21.15.26:0/1203494256 >> v1:172.21.15.160:6789/0 conn(0x7fae001599e0 legacy=0x7fae000cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:21.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.026+0000 7fae04e96640 1 -- 172.21.15.26:0/1203494256 >> v1:172.21.15.26:6789/0 conn(0x7fae000cb860 legacy=0x7fae0015d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:21.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.026+0000 7fae04e96640 1 -- 172.21.15.26:0/1203494256 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae000c9ae0 con 0x7fae0006f6d0 2023-12-19T17:58:21.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.026+0000 7fae04e96640 1 --2- 172.21.15.26:0/1203494256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0006f6d0 0x7fae0015dc40 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009d90 tx=0x7fadf801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97eba7d4010266a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:21.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.026+0000 7fadfeffd640 1 -- 172.21.15.26:0/1203494256 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf80a5a30 con 0x7fae0006f6d0 2023-12-19T17:58:21.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.026+0000 7fadfeffd640 1 -- 172.21.15.26:0/1203494256 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf80a5b90 con 0x7fae0006f6d0 2023-12-19T17:58:21.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.026+0000 7fadfeffd640 1 -- 172.21.15.26:0/1203494256 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf80ad440 con 0x7fae0006f6d0 2023-12-19T17:58:21.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 -- 172.21.15.26:0/1203494256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0006f6d0 msgr2=0x7fae0015dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:21.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 --2- 172.21.15.26:0/1203494256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0006f6d0 0x7fae0015dc40 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009d90 tx=0x7fadf801c950 comp rx=0 tx=0).stop 2023-12-19T17:58:21.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 -- 172.21.15.26:0/1203494256 shutdown_connections 2023-12-19T17:58:21.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 --2- 172.21.15.26:0/1203494256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae0006f6d0 0x7fae0015dc40 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 -- 172.21.15.26:0/1203494256 >> 172.21.15.26:0/1203494256 conn(0x7fae00066160 msgr2=0x7fae000b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:21.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 -- 172.21.15.26:0/1203494256 shutdown_connections 2023-12-19T17:58:21.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 -- 172.21.15.26:0/1203494256 wait complete. 2023-12-19T17:58:21.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 Processor -- start 2023-12-19T17:58:21.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 -- start start 2023-12-19T17:58:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae0006f6d0 0x7fae000c1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae000cb860 0x7fae0015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae001599e0 0x7fae000afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae000b0240 con 0x7fae000cb860 2023-12-19T17:58:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015a6c0 con 0x7fae0006f6d0 2023-12-19T17:58:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae068f9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015a9a0 con 0x7fae001599e0 2023-12-19T17:58:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fadff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae000cb860 0x7fae0015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:21.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fadff7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae000cb860 0x7fae0015a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38862/0 (socket says 172.21.15.26:38862) 2023-12-19T17:58:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fadff7fe640 1 -- 172.21.15.26:0/1044272533 learned_addr learned my addr 172.21.15.26:0/1044272533 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fae04e96640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae001599e0 0x7fae000afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:21.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.030+0000 7fadfffff640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae0006f6d0 0x7fae000c1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:21.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fadff7fe640 1 -- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae001599e0 msgr2=0x7fae000afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:21.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fadff7fe640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae001599e0 0x7fae000afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fadff7fe640 1 -- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae0006f6d0 msgr2=0x7fae000c1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fadff7fe640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae0006f6d0 0x7fae000c1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fadff7fe640 1 -- 172.21.15.26:0/1044272533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf8009a70 con 0x7fae000cb860 2023-12-19T17:58:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fadff7fe640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae000cb860 0x7fae0015a030 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fadf400c450 tx=0x7fadf400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fadfcff9640 1 -- 172.21.15.26:0/1044272533 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4005cf0 con 0x7fae000cb860 2023-12-19T17:58:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fadfcff9640 1 -- 172.21.15.26:0/1044272533 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf4005e70 con 0x7fae000cb860 2023-12-19T17:58:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fae068f9640 1 -- 172.21.15.26:0/1044272533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0015ac80 con 0x7fae000cb860 2023-12-19T17:58:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fadfcff9640 1 -- 172.21.15.26:0/1044272533 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4007290 con 0x7fae000cb860 2023-12-19T17:58:21.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fae068f9640 1 -- 172.21.15.26:0/1044272533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0015b240 con 0x7fae000cb860 2023-12-19T17:58:21.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fae068f9640 1 -- 172.21.15.26:0/1044272533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae00074270 con 0x7fae000cb860 2023-12-19T17:58:21.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fadfcff9640 1 -- 172.21.15.26:0/1044272533 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 89) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fadf40073f0 con 0x7fae000cb860 2023-12-19T17:58:21.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fadfcff9640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fadec065b80 0x7fadec068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:21.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.034+0000 7fadfcff9640 1 -- 172.21.15.26:0/1044272533 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fadf4082ff0 con 0x7fae000cb860 2023-12-19T17:58:21.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.038+0000 7fadfffff640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fadec065b80 0x7fadec068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:21.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.038+0000 7fadfffff640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fadec065b80 0x7fadec068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fadf0001770 tx=0x7fadf0009040 comp rx=0 tx=0).ready entity=mgr.8821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:21.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.042+0000 7fadfcff9640 1 -- 172.21.15.26:0/1044272533 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fadf40505e0 con 0x7fae000cb860 2023-12-19T17:58:21.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.238+0000 7fae068f9640 1 -- 172.21.15.26:0/1044272533 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fae000c2050 con 0x7fae000cb860 2023-12-19T17:58:21.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.726+0000 7fadfcff9640 1 -- 172.21.15.26:0/1044272533 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 90) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fadf4081230 con 0x7fae000cb860 2023-12-19T17:58:21.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.726+0000 7fadfcff9640 1 -- 172.21.15.26:0/1044272533 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v90) v1 ==== 92+0+0 (secure 0 0 0) 0x7fadf40806f0 con 0x7fae000cb860 2023-12-19T17:58:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.730+0000 7fae068f9640 1 -- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fadec065b80 msgr2=0x7fadec068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:21.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.730+0000 7fae068f9640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fadec065b80 0x7fadec068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fadf0001770 tx=0x7fadf0009040 comp rx=0 tx=0).stop 2023-12-19T17:58:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.730+0000 7fae068f9640 1 -- 172.21.15.26:0/1044272533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae000cb860 msgr2=0x7fae0015a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.730+0000 7fae068f9640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae000cb860 0x7fae0015a030 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fadf400c450 tx=0x7fadf400c910 comp rx=0 tx=0).stop 2023-12-19T17:58:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.730+0000 7fae068f9640 1 -- 172.21.15.26:0/1044272533 shutdown_connections 2023-12-19T17:58:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.730+0000 7fae068f9640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fadec065b80 0x7fadec068040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.730+0000 7fae068f9640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fae001599e0 0x7fae000afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.730+0000 7fae068f9640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fae000cb860 0x7fae0015a030 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.730+0000 7fae068f9640 1 --2- 172.21.15.26:0/1044272533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fae0006f6d0 0x7fae000c1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.730+0000 7fae068f9640 1 -- 172.21.15.26:0/1044272533 >> 172.21.15.26:0/1044272533 conn(0x7fae00066160 msgr2=0x7fae001504f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.730+0000 7fae068f9640 1 -- 172.21.15.26:0/1044272533 shutdown_connections 2023-12-19T17:58:21.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.730+0000 7fae068f9640 1 -- 172.21.15.26:0/1044272533 wait complete. 2023-12-19T17:58:21.763 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-19T17:58:21.908 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:21.905+0000 7f145de8d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:58:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.910+0000 7f25fe5a2640 1 Processor -- start 2023-12-19T17:58:21.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.910+0000 7f25fe5a2640 1 -- start start 2023-12-19T17:58:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.910+0000 7f25fe5a2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f25f815cc90 0x7f25f80cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.910+0000 7f25fe5a2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f25f815d070 0x7f25f80cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:21.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.910+0000 7f25fe5a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25f8160a20 0x7f25f80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:21.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.910+0000 7f25fe5a2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25f80cfe40 con 0x7f25f815cc90 2023-12-19T17:58:21.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.910+0000 7f25fe5a2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25f80cdb20 con 0x7f25f815d070 2023-12-19T17:58:21.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.910+0000 7f25fe5a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f80cde00 con 0x7f25f8160a20 2023-12-19T17:58:21.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.910+0000 7f25fd5a0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f25f815cc90 0x7f25f80cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:21.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25fcd9f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f25f815d070 0x7f25f80cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:21.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25fd5a0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f25f815cc90 0x7f25f80cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38876/0 (socket says 172.21.15.26:38876) 2023-12-19T17:58:21.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25fd5a0640 1 -- 172.21.15.26:0/309153550 learned_addr learned my addr 172.21.15.26:0/309153550 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:21.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25fdda1640 1 --2- 172.21.15.26:0/309153550 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25f8160a20 0x7f25f80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25fd5a0640 1 -- 172.21.15.26:0/309153550 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25f8160a20 msgr2=0x7f25f80cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25fd5a0640 1 --2- 172.21.15.26:0/309153550 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25f8160a20 0x7f25f80cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25fd5a0640 1 -- 172.21.15.26:0/309153550 >> v2:172.21.15.160:3300/0 conn(0x7f25f815d070 msgr2=0x7f25f80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T17:58:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25fd5a0640 1 --2- 172.21.15.26:0/309153550 >> v2:172.21.15.160:3300/0 conn(0x7f25f815d070 0x7f25f80cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25fd5a0640 1 -- 172.21.15.26:0/309153550 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f80ce0e0 con 0x7f25f815cc90 2023-12-19T17:58:21.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25fdda1640 1 --2- 172.21.15.26:0/309153550 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25f8160a20 0x7f25f80cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:21.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25fd5a0640 1 --2- 172.21.15.26:0/309153550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f815cc90 0x7f25f80cd2f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f25f8071890 tx=0x7f25f401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c27f8acd7d0309b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:21.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25effff640 1 -- 172.21.15.26:0/309153550 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f40aa5f0 con 0x7f25f815cc90 2023-12-19T17:58:21.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25effff640 1 -- 172.21.15.26:0/309153550 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f40aa750 con 0x7f25f815cc90 2023-12-19T17:58:21.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.914+0000 7f25effff640 1 -- 172.21.15.26:0/309153550 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f40aaae0 con 0x7f25f815cc90 2023-12-19T17:58:21.920 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f2e82a4d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T17:58:21.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 -- 172.21.15.26:0/309153550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f815cc90 msgr2=0x7f25f80cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 --2- 172.21.15.26:0/309153550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f815cc90 0x7f25f80cd2f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f25f8071890 tx=0x7f25f401c4d0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 -- 172.21.15.26:0/309153550 shutdown_connections 2023-12-19T17:58:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 --2- 172.21.15.26:0/309153550 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25f8160a20 0x7f25f80cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 --2- 172.21.15.26:0/309153550 >> v2:172.21.15.160:3300/0 conn(0x7f25f815d070 0x7f25f80cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 --2- 172.21.15.26:0/309153550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f815cc90 0x7f25f80cd2f0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 -- 172.21.15.26:0/309153550 >> 172.21.15.26:0/309153550 conn(0x7f25f806a320 msgr2=0x7f25f80658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 -- 172.21.15.26:0/309153550 shutdown_connections 2023-12-19T17:58:21.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 -- 172.21.15.26:0/309153550 wait complete. 2023-12-19T17:58:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 Processor -- start 2023-12-19T17:58:21.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 -- start start 2023-12-19T17:58:21.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f815cc90 0x7f25f814cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:21.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25f815d070 0x7f25f8153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:21.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25f8160a20 0x7f25f8154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:21.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f815b220 con 0x7f25f815cc90 2023-12-19T17:58:21.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f814d0d0 con 0x7f25f815d070 2023-12-19T17:58:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fe5a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f815a5a0 con 0x7f25f8160a20 2023-12-19T17:58:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fd5a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f815cc90 0x7f25f814cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fd5a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f815cc90 0x7f25f814cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38880/0 (socket says 172.21.15.26:38880) 2023-12-19T17:58:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fd5a0640 1 -- 172.21.15.26:0/4294716029 learned_addr learned my addr 172.21.15.26:0/4294716029 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.918+0000 7f25fcd9f640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25f815d070 0x7f25f8153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25fdda1640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25f8160a20 0x7f25f8154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25fd5a0640 1 -- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25f8160a20 msgr2=0x7f25f8154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25fd5a0640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25f8160a20 0x7f25f8154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25fd5a0640 1 -- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25f815d070 msgr2=0x7f25f8153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25fd5a0640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25f815d070 0x7f25f8153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25fd5a0640 1 -- 172.21.15.26:0/4294716029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f40094d0 con 0x7f25f815cc90 2023-12-19T17:58:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25fcd9f640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25f815d070 0x7f25f8153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T17:58:21.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25fd5a0640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f815cc90 0x7f25f814cb90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f25f401c300 tx=0x7f25f401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:21.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25edffb640 1 -- 172.21.15.26:0/4294716029 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f40bd5f0 con 0x7f25f815cc90 2023-12-19T17:58:21.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25edffb640 1 -- 172.21.15.26:0/4294716029 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25f40bd750 con 0x7f25f815cc90 2023-12-19T17:58:21.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25fe5a2640 1 -- 172.21.15.26:0/4294716029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f815a880 con 0x7f25f815cc90 2023-12-19T17:58:21.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25edffb640 1 -- 172.21.15.26:0/4294716029 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25f40bdb00 con 0x7f25f815cc90 2023-12-19T17:58:21.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25fe5a2640 1 -- 172.21.15.26:0/4294716029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f815ae40 con 0x7f25f815cc90 2023-12-19T17:58:21.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25fe5a2640 1 -- 172.21.15.26:0/4294716029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f815da30 con 0x7f25f815cc90 2023-12-19T17:58:21.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.922+0000 7f25edffb640 1 -- 172.21.15.26:0/4294716029 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 90) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f25f40b33f0 con 0x7f25f815cc90 2023-12-19T17:58:21.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.926+0000 7f25edffb640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f25dc065bd0 0x7f25dc068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:21.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.926+0000 7f25edffb640 1 -- 172.21.15.26:0/4294716029 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f25f40a7070 con 0x7f25f815cc90 2023-12-19T17:58:21.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.926+0000 7f25fcd9f640 1 -- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f25dc065bd0 msgr2=0x7f25dc068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:58:21.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.926+0000 7f25fcd9f640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f25dc065bd0 0x7f25dc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:58:21.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:21.930+0000 7f25edffb640 1 -- 172.21.15.26:0/4294716029 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f25f40ebc60 con 0x7f25f815cc90 2023-12-19T17:58:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.090+0000 7f25fe5a2640 1 -- 172.21.15.26:0/4294716029 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f25f8061930 con 0x7f25f815cc90 2023-12-19T17:58:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.094+0000 7f25edffb640 1 -- 172.21.15.26:0/4294716029 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v59)=0 v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7f25f40ef280 con 0x7f25f815cc90 2023-12-19T17:58:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.094+0000 7f25fe5a2640 1 -- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f25dc065bd0 msgr2=0x7f25dc068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:58:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.094+0000 7f25fe5a2640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f25dc065bd0 0x7f25dc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.094+0000 7f25fe5a2640 1 -- 172.21.15.26:0/4294716029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f815cc90 msgr2=0x7f25f814cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.094+0000 7f25fe5a2640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f815cc90 0x7f25f814cb90 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f25f401c300 tx=0x7f25f401ad10 comp rx=0 tx=0).stop 2023-12-19T17:58:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.094+0000 7f25fe5a2640 1 -- 172.21.15.26:0/4294716029 shutdown_connections 2023-12-19T17:58:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.094+0000 7f25fe5a2640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f25dc065bd0 0x7f25dc068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.094+0000 7f25fe5a2640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f25f8160a20 0x7f25f8154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.094+0000 7f25fe5a2640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25f815d070 0x7f25f8153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.094+0000 7f25fe5a2640 1 --2- 172.21.15.26:0/4294716029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25f815cc90 0x7f25f814cb90 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.094+0000 7f25fe5a2640 1 -- 172.21.15.26:0/4294716029 >> 172.21.15.26:0/4294716029 conn(0x7f25f806a320 msgr2=0x7f25f80cd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.094+0000 7f25fe5a2640 1 -- 172.21.15.26:0/4294716029 shutdown_connections 2023-12-19T17:58:22.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.094+0000 7f25fe5a2640 1 -- 172.21.15.26:0/4294716029 wait complete. 2023-12-19T17:58:22.119 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-19T17:58:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.262+0000 7f3f73752640 1 Processor -- start 2023-12-19T17:58:22.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.262+0000 7f3f73752640 1 -- start start 2023-12-19T17:58:22.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f73752640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f6c15ccf0 0x7f3f6c0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:22.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f73752640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6c15e200 con 0x7f3f6c0b4600 2023-12-19T17:58:22.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f73752640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f6c0d0aa0 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f73752640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c0d1380 con 0x7f3f6c15ccf0 2023-12-19T17:58:22.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f71cef640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3f6c0b4600 0x7f3f6c160c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34458/0 (socket says 172.21.15.26:34458) 2023-12-19T17:58:22.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f71cef640 1 -- 172.21.15.26:0/3713709850 learned_addr learned my addr 172.21.15.26:0/3713709850 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:22.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f714ee640 1 --2- 172.21.15.26:0/3713709850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f6c15ccf0 0x7f3f6c0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f63fff640 1 -- 172.21.15.26:0/3713709850 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661103833 0 0) 0x7f3f6c15e200 con 0x7f3f6c0b4600 2023-12-19T17:58:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f63fff640 1 -- 172.21.15.26:0/3713709850 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f58003530 con 0x7f3f6c0b4600 2023-12-19T17:58:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f63fff640 1 -- 172.21.15.26:0/3713709850 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330671514 0 0) 0x7f3f6c0d0aa0 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f63fff640 1 -- 172.21.15.26:0/3713709850 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f6c15e200 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f714ee640 1 -- 172.21.15.26:0/3713709850 >> v1:172.21.15.160:6789/0 conn(0x7f3f6c15d0d0 legacy=0x7f3f6c0b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f714ee640 1 -- 172.21.15.26:0/3713709850 >> v1:172.21.15.26:6789/0 conn(0x7f3f6c0b4600 legacy=0x7f3f6c160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.266+0000 7f3f714ee640 1 -- 172.21.15.26:0/3713709850 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c15d680 con 0x7f3f6c15ccf0 2023-12-19T17:58:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.270+0000 7f3f714ee640 1 --2- 172.21.15.26:0/3713709850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f6c15ccf0 0x7f3f6c0d0e40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3f680097f0 tx=0x7f3f6801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7943ad26af68870 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.270+0000 7f3f63fff640 1 -- 172.21.15.26:0/3713709850 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f680ac600 con 0x7f3f6c15ccf0 2023-12-19T17:58:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.270+0000 7f3f63fff640 1 -- 172.21.15.26:0/3713709850 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f680a8050 con 0x7f3f6c15ccf0 2023-12-19T17:58:22.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.270+0000 7f3f63fff640 1 -- 172.21.15.26:0/3713709850 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f680ac960 con 0x7f3f6c15ccf0 2023-12-19T17:58:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.270+0000 7f3f73752640 1 -- 172.21.15.26:0/3713709850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f6c15ccf0 msgr2=0x7f3f6c0d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.270+0000 7f3f73752640 1 --2- 172.21.15.26:0/3713709850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f6c15ccf0 0x7f3f6c0d0e40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3f680097f0 tx=0x7f3f6801c490 comp rx=0 tx=0).stop 2023-12-19T17:58:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.270+0000 7f3f73752640 1 -- 172.21.15.26:0/3713709850 shutdown_connections 2023-12-19T17:58:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.270+0000 7f3f73752640 1 --2- 172.21.15.26:0/3713709850 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f6c15ccf0 0x7f3f6c0d0e40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.270+0000 7f3f73752640 1 -- 172.21.15.26:0/3713709850 >> 172.21.15.26:0/3713709850 conn(0x7f3f6c065ee0 msgr2=0x7f3f6c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.270+0000 7f3f73752640 1 -- 172.21.15.26:0/3713709850 shutdown_connections 2023-12-19T17:58:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.270+0000 7f3f73752640 1 -- 172.21.15.26:0/3713709850 wait complete. 2023-12-19T17:58:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.270+0000 7f3f73752640 1 Processor -- start 2023-12-19T17:58:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.270+0000 7f3f73752640 1 -- start start 2023-12-19T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f73752640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f6c0b4600 0x7f3f6c0c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f73752640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f6c15ccf0 0x7f3f6c155e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f73752640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f6c15d0d0 0x7f3f6c152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f73752640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c155750 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f73752640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c153970 con 0x7f3f6c0b4600 2023-12-19T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f73752640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c153c50 con 0x7f3f6c15ccf0 2023-12-19T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f71cef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f6c15d0d0 0x7f3f6c152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f71cef640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f6c15d0d0 0x7f3f6c152c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38886/0 (socket says 172.21.15.26:38886) 2023-12-19T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f71cef640 1 -- 172.21.15.26:0/3036624954 learned_addr learned my addr 172.21.15.26:0/3036624954 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f714ee640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f6c0b4600 0x7f3f6c0c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f70ced640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f6c15ccf0 0x7f3f6c155e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f71cef640 1 -- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f6c15ccf0 msgr2=0x7f3f6c155e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f71cef640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f6c15ccf0 0x7f3f6c155e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f71cef640 1 -- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f6c0b4600 msgr2=0x7f3f6c0c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f71cef640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f6c0b4600 0x7f3f6c0c66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f71cef640 1 -- 172.21.15.26:0/3036624954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f68009420 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f714ee640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f6c0b4600 0x7f3f6c0c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:58:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f71cef640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f6c15d0d0 0x7f3f6c152c10 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400caa0 tx=0x7f3f6400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f61ffb640 1 -- 172.21.15.26:0/3036624954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6400ac00 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f61ffb640 1 -- 172.21.15.26:0/3036624954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6400ad80 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f73752640 1 -- 172.21.15.26:0/3036624954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6c153f30 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f61ffb640 1 -- 172.21.15.26:0/3036624954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64012640 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f73752640 1 -- 172.21.15.26:0/3036624954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6c156520 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.274+0000 7f3f73752640 1 -- 172.21.15.26:0/3036624954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6c1537d0 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.278+0000 7f3f61ffb640 1 -- 172.21.15.26:0/3036624954 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 90) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f3f64007070 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.278+0000 7f3f61ffb640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f50065bd0 0x7f3f50068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:22.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.278+0000 7f3f61ffb640 1 -- 172.21.15.26:0/3036624954 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f3f64083590 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.278+0000 7f3f714ee640 1 -- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f50065bd0 msgr2=0x7f3f50068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:58:22.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.278+0000 7f3f714ee640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f50065bd0 0x7f3f50068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:58:22.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.282+0000 7f3f61ffb640 1 -- 172.21.15.26:0/3036624954 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3f6c1537d0 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.325 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:22.321+0000 7f145de8d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:58:22.365 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:22.362+0000 7f2e82a4d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T17:58:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.442+0000 7f3f73752640 1 -- 172.21.15.26:0/3036624954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f3f6c1537d0 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.446+0000 7f3f61ffb640 1 -- 172.21.15.26:0/3036624954 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v60)=0 v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7f3f6c1537d0 con 0x7f3f6c15d0d0 2023-12-19T17:58:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.450+0000 7f3f73752640 1 -- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f50065bd0 msgr2=0x7f3f50068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:58:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.450+0000 7f3f73752640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f50065bd0 0x7f3f50068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.450+0000 7f3f73752640 1 -- 172.21.15.26:0/3036624954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f6c15d0d0 msgr2=0x7f3f6c152c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.450+0000 7f3f73752640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f6c15d0d0 0x7f3f6c152c10 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400caa0 tx=0x7f3f6400cf60 comp rx=0 tx=0).stop 2023-12-19T17:58:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.450+0000 7f3f73752640 1 -- 172.21.15.26:0/3036624954 shutdown_connections 2023-12-19T17:58:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.450+0000 7f3f73752640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f6c15d0d0 0x7f3f6c152c10 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.450+0000 7f3f73752640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f3f50065bd0 0x7f3f50068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.450+0000 7f3f73752640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f6c15ccf0 0x7f3f6c155e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.450+0000 7f3f73752640 1 --2- 172.21.15.26:0/3036624954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f6c0b4600 0x7f3f6c0c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.450+0000 7f3f73752640 1 -- 172.21.15.26:0/3036624954 >> 172.21.15.26:0/3036624954 conn(0x7f3f6c065ee0 msgr2=0x7f3f6c0d20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.450+0000 7f3f73752640 1 -- 172.21.15.26:0/3036624954 shutdown_connections 2023-12-19T17:58:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.450+0000 7f3f73752640 1 -- 172.21.15.26:0/3036624954 wait complete. 2023-12-19T17:58:22.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:58:22.472 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-19T17:58:22.476 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:22.473+0000 7f145de8d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:58:22.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T17:58:22.487 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-12-19T17:58:22.526 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:22.522+0000 7f2e82a4d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T17:58:22.537 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/test_restart 1M 2023-12-19T17:58:22.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.722+0000 7f8ca542d640 1 Processor -- start 2023-12-19T17:58:22.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.722+0000 7f8ca542d640 1 -- start start 2023-12-19T17:58:22.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.722+0000 7f8ca542d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ca00b4600 0x7f8ca00b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:22.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.722+0000 7f8ca542d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca00b4f50 con 0x7f8ca015ccf0 2023-12-19T17:58:22.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.722+0000 7f8ca542d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca015e200 con 0x7f8ca015d0d0 2023-12-19T17:58:22.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.722+0000 7f8ca542d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca00d0aa0 con 0x7f8ca00b4600 2023-12-19T17:58:22.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.726+0000 7f8c9effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8ca015ccf0 0x7f8ca00d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34462/0 (socket says 172.21.15.26:34462) 2023-12-19T17:58:22.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.726+0000 7f8c9effd640 1 -- 172.21.15.26:0/417406976 learned_addr learned my addr 172.21.15.26:0/417406976 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:22.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.726+0000 7f8c9f7fe640 1 --2- 172.21.15.26:0/417406976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ca00b4600 0x7f8ca00b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:22.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.726+0000 7f8c9f7fe640 1 -- 172.21.15.26:0/417406976 >> v1:172.21.15.160:6789/0 conn(0x7f8ca015d0d0 legacy=0x7f8ca00b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:22.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.726+0000 7f8c9f7fe640 1 -- 172.21.15.26:0/417406976 >> v1:172.21.15.26:6789/0 conn(0x7f8ca015ccf0 legacy=0x7f8ca00d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:22.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.726+0000 7f8c9f7fe640 1 -- 172.21.15.26:0/417406976 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca015d680 con 0x7f8ca00b4600 2023-12-19T17:58:22.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.726+0000 7f8c9f7fe640 1 --2- 172.21.15.26:0/417406976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ca00b4600 0x7f8ca00b4a10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009dc0 tx=0x7f8c9801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fada9c5a614de63 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:22.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.726+0000 7f8c9dffb640 1 -- 172.21.15.26:0/417406976 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c980a5a00 con 0x7f8ca00b4600 2023-12-19T17:58:22.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.726+0000 7f8c9dffb640 1 -- 172.21.15.26:0/417406976 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c980a5b60 con 0x7f8ca00b4600 2023-12-19T17:58:22.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.726+0000 7f8c9dffb640 1 -- 172.21.15.26:0/417406976 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c980ad440 con 0x7f8ca00b4600 2023-12-19T17:58:22.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 -- 172.21.15.26:0/417406976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ca00b4600 msgr2=0x7f8ca00b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:22.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 --2- 172.21.15.26:0/417406976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ca00b4600 0x7f8ca00b4a10 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009dc0 tx=0x7f8c9801c940 comp rx=0 tx=0).stop 2023-12-19T17:58:22.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 -- 172.21.15.26:0/417406976 shutdown_connections 2023-12-19T17:58:22.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 --2- 172.21.15.26:0/417406976 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ca00b4600 0x7f8ca00b4a10 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 -- 172.21.15.26:0/417406976 >> 172.21.15.26:0/417406976 conn(0x7f8ca0065ee0 msgr2=0x7f8ca00cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 -- 172.21.15.26:0/417406976 shutdown_connections 2023-12-19T17:58:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 -- 172.21.15.26:0/417406976 wait complete. 2023-12-19T17:58:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 Processor -- start 2023-12-19T17:58:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 -- start start 2023-12-19T17:58:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ca00b4600 0x7f8ca0158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ca015ccf0 0x7f8ca01561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ca015d0d0 0x7f8ca00d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca00caa60 con 0x7f8ca00b4600 2023-12-19T17:58:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca00cad40 con 0x7f8ca015ccf0 2023-12-19T17:58:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8ca542d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca00cb020 con 0x7f8ca015d0d0 2023-12-19T17:58:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8c9effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ca00b4600 0x7f8ca0158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8c9effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ca00b4600 0x7f8ca0158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38896/0 (socket says 172.21.15.26:38896) 2023-12-19T17:58:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.730+0000 7f8c9effd640 1 -- 172.21.15.26:0/3836450774 learned_addr learned my addr 172.21.15.26:0/3836450774 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8c9f7fe640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ca015d0d0 0x7f8ca00d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8c9e7fc640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ca015ccf0 0x7f8ca01561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8c9effd640 1 -- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ca015d0d0 msgr2=0x7f8ca00d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8c9effd640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ca015d0d0 0x7f8ca00d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8c9effd640 1 -- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ca015ccf0 msgr2=0x7f8ca01561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8c9effd640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ca015ccf0 0x7f8ca01561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8c9effd640 1 -- 172.21.15.26:0/3836450774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c98009a70 con 0x7f8ca00b4600 2023-12-19T17:58:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8c9effd640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ca00b4600 0x7f8ca0158750 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000c400 tx=0x7f8c9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:22.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8c877fe640 1 -- 172.21.15.26:0/3836450774 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90005cf0 con 0x7f8ca00b4600 2023-12-19T17:58:22.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8c877fe640 1 -- 172.21.15.26:0/3836450774 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c90005e50 con 0x7f8ca00b4600 2023-12-19T17:58:22.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8c877fe640 1 -- 172.21.15.26:0/3836450774 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900072d0 con 0x7f8ca00b4600 2023-12-19T17:58:22.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8ca542d640 1 -- 172.21.15.26:0/3836450774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca00cb300 con 0x7f8ca00b4600 2023-12-19T17:58:22.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8ca542d640 1 -- 172.21.15.26:0/3836450774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca0158c90 con 0x7f8ca00b4600 2023-12-19T17:58:22.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8ca542d640 1 -- 172.21.15.26:0/3836450774 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca00cb820 con 0x7f8ca00b4600 2023-12-19T17:58:22.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8c877fe640 1 -- 172.21.15.26:0/3836450774 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 90) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f8c90007430 con 0x7f8ca00b4600 2023-12-19T17:58:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.734+0000 7f8c877fe640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 0x7f8c8c0680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.738+0000 7f8c877fe640 1 -- 172.21.15.26:0/3836450774 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f8c90082a40 con 0x7f8ca00b4600 2023-12-19T17:58:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.738+0000 7f8c9e7fc640 1 -- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 msgr2=0x7f8c8c0680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:58:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.738+0000 7f8c9e7fc640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 0x7f8c8c0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T17:58:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.742+0000 7f8c877fe640 1 -- 172.21.15.26:0/3836450774 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8ca00cb820 con 0x7f8ca00b4600 2023-12-19T17:58:22.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.918+0000 7f8ca542d640 1 -- 172.21.15.26:0/3836450774 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ca015df20 con 0x7f8c8c065c00 2023-12-19T17:58:22.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.942+0000 7f8c9e7fc640 1 -- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 msgr2=0x7f8c8c0680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:58:22.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:22.942+0000 7f8c9e7fc640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 0x7f8c8c0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T17:58:22.956 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:22.953+0000 7f145de8d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:58:23.015 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:23.009+0000 7f145de8d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:58:23.039 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:23.038+0000 7f2e82a4d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T17:58:23.070 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:23.065+0000 7f145de8d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:58:23.101 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:23.098+0000 7f2e82a4d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T17:58:23.155 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:23.149+0000 7f145de8d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:58:23.159 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:23.154+0000 7f2e82a4d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T17:58:23.213 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:23.209+0000 7f145de8d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:58:23.249 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:23.246+0000 7f2e82a4d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T17:58:23.296 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:23.293+0000 7f145de8d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:58:23.308 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:23.306+0000 7f2e82a4d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T17:58:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:23.346+0000 7f8c9e7fc640 1 -- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 msgr2=0x7f8c8c0680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:58:23.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:23.346+0000 7f8c9e7fc640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 0x7f8c8c0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-19T17:58:23.352 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:23.349+0000 7f145de8d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:58:23.396 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:23.394+0000 7f2e82a4d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T17:58:23.455 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:23.454+0000 7f2e82a4d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T17:58:23.496 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:23.489+0000 7f145de8d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:58:23.554 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:23.549+0000 7f145de8d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:58:23.606 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:23.602+0000 7f2e82a4d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T17:58:23.667 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:23.666+0000 7f2e82a4d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T17:58:23.794 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:58:23.794 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:58:23.794 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:58:23.802 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:23.797+0000 7f145de8d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:58:23.919 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T17:58:23.919 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T17:58:23.919 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T17:58:23.927 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:23.926+0000 7f2e82a4d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T17:58:23.937 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:23.933+0000 7f145de8d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:58:24.061 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:24.057+0000 7f145de8d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:58:24.069 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:24.066+0000 7f2e82a4d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:58:24.131 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:24.125+0000 7f145de8d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:58:24.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:24.150+0000 7f8c9e7fc640 1 -- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 msgr2=0x7f8c8c0680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/38098 2023-12-19T17:58:24.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:24.150+0000 7f8c9e7fc640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 0x7f8c8c0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-19T17:58:24.201 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:24.198+0000 7f2e82a4d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T17:58:24.267 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:24.261+0000 7f145de8d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:58:24.275 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:24.270+0000 7f2e82a4d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T17:58:24.418 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:24.414+0000 7f2e82a4d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T17:58:24.804 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:24.801+0000 7f145de8d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:58:24.889 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:24.885+0000 7f145de8d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:58:24.994 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:24.990+0000 7f2e82a4d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T17:58:25.039 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.033+0000 7f145de8d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:58:25.084 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:25.082+0000 7f2e82a4d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T17:58:25.122 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.117+0000 7f145de8d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:58:25.241 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:25.238+0000 7f2e82a4d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T17:58:25.329 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:25.326+0000 7f2e82a4d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T17:58:25.438 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.433+0000 7f145de8d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:58:25.498 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.493+0000 7f145de8d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:58:25.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:25.506+0000 7f8c877fe640 1 -- 172.21.15.26:0/3836450774 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 91) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f8c90050520 con 0x7f8ca00b4600 2023-12-19T17:58:25.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:25.506+0000 7f8c877fe640 1 -- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 msgr2=0x7f8c8c0680c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T17:58:25.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:25.506+0000 7f8c877fe640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 0x7f8c8c0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:25.642 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.637+0000 7f13e3d08640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:58:25.642 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.637+0000 7f13e3d08640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:58:25.643 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.637+0000 7f13e3d08640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:58:25.643 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.637+0000 7f13e3d08640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:58:25.643 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.637+0000 7f13e3d08640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:58:25.643 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.637+0000 7f13e0501640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:58:25.643 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.637+0000 7f13e0501640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:58:25.643 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.637+0000 7f13e0501640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:58:25.643 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.637+0000 7f13e0501640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:58:25.643 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:58:25.637+0000 7f13e0501640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:58:25.678 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:25.674+0000 7f2e82a4d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T17:58:25.741 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T17:58:25.738+0000 7f2e82a4d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T17:58:26.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.510+0000 7f8c877fe640 1 -- 172.21.15.26:0/3836450774 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 92) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f8c90080c50 con 0x7f8ca00b4600 2023-12-19T17:58:26.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.510+0000 7f8c877fe640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 0x7f8c8c0680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:26.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.510+0000 7f8c877fe640 1 -- 172.21.15.26:0/3836450774 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c8c06bc00 con 0x7f8c8c065c00 2023-12-19T17:58:26.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.518+0000 7f8c9e7fc640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 0x7f8c8c0680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:26.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.522+0000 7f8c9e7fc640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 0x7f8c8c0680c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ca01f8b70 tx=0x7f8c94007e40 comp rx=0 tx=0).ready entity=mgr.9172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:26.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.586+0000 7f8c877fe640 1 -- 172.21.15.26:0/3836450774 <== mgr.9172 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f8c8c06bc00 con 0x7f8c8c065c00 2023-12-19T17:58:26.588 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-12-19T17:58:26.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.586+0000 7f8ca542d640 1 -- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 msgr2=0x7f8c8c0680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.586+0000 7f8ca542d640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 0x7f8c8c0680c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ca01f8b70 tx=0x7f8c94007e40 comp rx=0 tx=0).stop 2023-12-19T17:58:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.586+0000 7f8ca542d640 1 -- 172.21.15.26:0/3836450774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ca00b4600 msgr2=0x7f8ca0158750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.586+0000 7f8ca542d640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ca00b4600 0x7f8ca0158750 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000c400 tx=0x7f8c9000c8c0 comp rx=0 tx=0).stop 2023-12-19T17:58:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.586+0000 7f8ca542d640 1 -- 172.21.15.26:0/3836450774 shutdown_connections 2023-12-19T17:58:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.586+0000 7f8ca542d640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8c8c065c00 0x7f8c8c0680c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.586+0000 7f8ca542d640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ca015d0d0 0x7f8ca00d1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.586+0000 7f8ca542d640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ca015ccf0 0x7f8ca01561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.586+0000 7f8ca542d640 1 --2- 172.21.15.26:0/3836450774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ca00b4600 0x7f8ca0158750 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.586+0000 7f8ca542d640 1 -- 172.21.15.26:0/3836450774 >> 172.21.15.26:0/3836450774 conn(0x7f8ca0065ee0 msgr2=0x7f8ca00d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.586+0000 7f8ca542d640 1 -- 172.21.15.26:0/3836450774 shutdown_connections 2023-12-19T17:58:26.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.586+0000 7f8ca542d640 1 -- 172.21.15.26:0/3836450774 wait complete. 2023-12-19T17:58:26.612 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-12-19T17:58:26.612 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/test_restart 1M 2023-12-19T17:58:26.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f08bbe640 1 Processor -- start 2023-12-19T17:58:26.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f08bbe640 1 -- start start 2023-12-19T17:58:26.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f08bbe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8f04159c00 0x7f8f0415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:26.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f08bbe640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0406fb60 con 0x7f8f0406f6c0 2023-12-19T17:58:26.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f08bbe640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0406fe40 con 0x7f8f04159820 2023-12-19T17:58:26.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f08bbe640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0415e470 con 0x7f8f04159c00 2023-12-19T17:58:26.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8efbfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8f0406f6c0 0x7f8f04070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55310/0 (socket says 172.21.15.26:55310) 2023-12-19T17:58:26.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8efbfff640 1 -- 172.21.15.26:0/2317878154 learned_addr learned my addr 172.21.15.26:0/2317878154 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T17:58:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f02d9d640 1 --2- 172.21.15.26:0/2317878154 >> v2:172.21.15.160:3300/0 conn(0x7f8f04159c00 0x7f8f0415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f01d9b640 1 -- 172.21.15.26:0/2317878154 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030917852 0 0) 0x7f8f0406fb60 con 0x7f8f0406f6c0 2023-12-19T17:58:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f01d9b640 1 -- 172.21.15.26:0/2317878154 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ef0003530 con 0x7f8f0406f6c0 2023-12-19T17:58:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f01d9b640 1 -- 172.21.15.26:0/2317878154 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480227544 0 0) 0x7f8f0406fe40 con 0x7f8f04159820 2023-12-19T17:58:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f01d9b640 1 -- 172.21.15.26:0/2317878154 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f0406fb60 con 0x7f8f04159820 2023-12-19T17:58:26.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f02d9d640 1 -- 172.21.15.26:0/2317878154 >> v1:172.21.15.160:6789/0 conn(0x7f8f04159820 legacy=0x7f8f040c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:26.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f02d9d640 1 -- 172.21.15.26:0/2317878154 >> v1:172.21.15.26:6789/0 conn(0x7f8f0406f6c0 legacy=0x7f8f04070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:26.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f01d9b640 1 -- 172.21.15.26:0/2317878154 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2018811140 0 0) 0x7f8ef0003530 con 0x7f8f0406f6c0 2023-12-19T17:58:26.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.758+0000 7f8f02d9d640 1 -- 172.21.15.26:0/2317878154 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f04070120 con 0x7f8f04159c00 2023-12-19T17:58:26.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.762+0000 7f8f02d9d640 1 --2- 172.21.15.26:0/2317878154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f04159c00 0x7f8f0415df30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009d90 tx=0x7f8efc01c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50c77a13ed22df3d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:26.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.762+0000 7f8f01d9b640 1 -- 172.21.15.26:0/2317878154 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0a59e0 con 0x7f8f04159c00 2023-12-19T17:58:26.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.762+0000 7f8f01d9b640 1 -- 172.21.15.26:0/2317878154 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc0a5b40 con 0x7f8f04159c00 2023-12-19T17:58:26.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.762+0000 7f8f01d9b640 1 -- 172.21.15.26:0/2317878154 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0ad440 con 0x7f8f04159c00 2023-12-19T17:58:26.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.762+0000 7f8f08bbe640 1 -- 172.21.15.26:0/2317878154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f04159c00 msgr2=0x7f8f0415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:26.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.762+0000 7f8f08bbe640 1 --2- 172.21.15.26:0/2317878154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f04159c00 0x7f8f0415df30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009d90 tx=0x7f8efc01c920 comp rx=0 tx=0).stop 2023-12-19T17:58:26.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.762+0000 7f8f08bbe640 1 -- 172.21.15.26:0/2317878154 shutdown_connections 2023-12-19T17:58:26.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.762+0000 7f8f08bbe640 1 --2- 172.21.15.26:0/2317878154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f04159c00 0x7f8f0415df30 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:26.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.762+0000 7f8f08bbe640 1 -- 172.21.15.26:0/2317878154 >> 172.21.15.26:0/2317878154 conn(0x7f8f040ce730 msgr2=0x7f8f040ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:26.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.762+0000 7f8f08bbe640 1 -- 172.21.15.26:0/2317878154 shutdown_connections 2023-12-19T17:58:26.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.762+0000 7f8f08bbe640 1 -- 172.21.15.26:0/2317878154 wait complete. 2023-12-19T17:58:26.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f08bbe640 1 Processor -- start 2023-12-19T17:58:26.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f08bbe640 1 -- start start 2023-12-19T17:58:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f08bbe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f0406f6c0 0x7f8f0414f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f08bbe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f04159820 0x7f8f040ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f08bbe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f04159c00 0x7f8f040acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f08bbe640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f040c50e0 con 0x7f8f04159c00 2023-12-19T17:58:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f08bbe640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f04159fe0 con 0x7f8f04159820 2023-12-19T17:58:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f08bbe640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0415a230 con 0x7f8f0406f6c0 2023-12-19T17:58:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f02d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f04159c00 0x7f8f040acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f0259c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f0406f6c0 0x7f8f0414f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f02d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f04159c00 0x7f8f040acda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51846/0 (socket says 172.21.15.26:51846) 2023-12-19T17:58:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f0259c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f0406f6c0 0x7f8f0414f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:52436/0 (socket says 172.21.15.26:52436) 2023-12-19T17:58:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f02d9d640 1 -- 172.21.15.26:0/1682605704 learned_addr learned my addr 172.21.15.26:0/1682605704 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T17:58:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8efbfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f04159820 0x7f8f040ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:26.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f02d9d640 1 -- 172.21.15.26:0/1682605704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f0406f6c0 msgr2=0x7f8f0414f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:26.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f02d9d640 1 --2- 172.21.15.26:0/1682605704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f0406f6c0 0x7f8f0414f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:26.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f02d9d640 1 -- 172.21.15.26:0/1682605704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f04159820 msgr2=0x7f8f040ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:26.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f02d9d640 1 --2- 172.21.15.26:0/1682605704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f04159820 0x7f8f040ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:26.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f02d9d640 1 -- 172.21.15.26:0/1682605704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc009a70 con 0x7f8f04159c00 2023-12-19T17:58:26.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8efbfff640 1 --2- 172.21.15.26:0/1682605704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f04159820 0x7f8f040ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T17:58:26.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f02d9d640 1 --2- 172.21.15.26:0/1682605704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f04159c00 0x7f8f040acda0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8efc01ce00 tx=0x7f8efc0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:26.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8efaffd640 1 -- 172.21.15.26:0/1682605704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0adca0 con 0x7f8f04159c00 2023-12-19T17:58:26.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8efaffd640 1 -- 172.21.15.26:0/1682605704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc0ade00 con 0x7f8f04159c00 2023-12-19T17:58:26.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8efaffd640 1 -- 172.21.15.26:0/1682605704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc003ba0 con 0x7f8f04159c00 2023-12-19T17:58:26.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.766+0000 7f8f08bbe640 1 -- 172.21.15.26:0/1682605704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0406b880 con 0x7f8f04159c00 2023-12-19T17:58:26.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.770+0000 7f8f08bbe640 1 -- 172.21.15.26:0/1682605704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0415a7f0 con 0x7f8f04159c00 2023-12-19T17:58:26.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.770+0000 7f8f08bbe640 1 -- 172.21.15.26:0/1682605704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f04074270 con 0x7f8f04159c00 2023-12-19T17:58:26.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.770+0000 7f8efaffd640 1 -- 172.21.15.26:0/1682605704 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 92) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f8efc003d00 con 0x7f8f04159c00 2023-12-19T17:58:26.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.770+0000 7f8efaffd640 1 --2- 172.21.15.26:0/1682605704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8ee4065900 0x7f8ee4067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T17:58:26.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.770+0000 7f8efaffd640 1 -- 172.21.15.26:0/1682605704 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f8efc11e5c0 con 0x7f8f04159c00 2023-12-19T17:58:26.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.770+0000 7f8f0259c640 1 --2- 172.21.15.26:0/1682605704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8ee4065900 0x7f8ee4067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T17:58:26.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.774+0000 7f8f0259c640 1 --2- 172.21.15.26:0/1682605704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8ee4065900 0x7f8ee4067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40011f0 tx=0x7f8ef4009040 comp rx=0 tx=0).ready entity=mgr.9172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T17:58:26.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.774+0000 7f8efaffd640 1 -- 172.21.15.26:0/1682605704 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8efc0ebf30 con 0x7f8f04159c00 2023-12-19T17:58:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:26.954+0000 7f8f08bbe640 1 -- 172.21.15.26:0/1682605704 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f040c9590 con 0x7f8ee4065900 2023-12-19T17:58:27.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:27.002+0000 7f8efaffd640 1 -- 172.21.15.26:0/1682605704 <== mgr.9172 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f8ef40014e0 con 0x7f8ee4065900 2023-12-19T17:58:27.004 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-12-19T17:58:27.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:27.002+0000 7f8f08bbe640 1 -- 172.21.15.26:0/1682605704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8ee4065900 msgr2=0x7f8ee4067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:27.002+0000 7f8f08bbe640 1 --2- 172.21.15.26:0/1682605704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8ee4065900 0x7f8ee4067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40011f0 tx=0x7f8ef4009040 comp rx=0 tx=0).stop 2023-12-19T17:58:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:27.002+0000 7f8f08bbe640 1 -- 172.21.15.26:0/1682605704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f04159c00 msgr2=0x7f8f040acda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T17:58:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:27.002+0000 7f8f08bbe640 1 --2- 172.21.15.26:0/1682605704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f04159c00 0x7f8f040acda0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f8efc01ce00 tx=0x7f8efc0196c0 comp rx=0 tx=0).stop 2023-12-19T17:58:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:27.002+0000 7f8f08bbe640 1 -- 172.21.15.26:0/1682605704 shutdown_connections 2023-12-19T17:58:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:27.002+0000 7f8f08bbe640 1 --2- 172.21.15.26:0/1682605704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f04159c00 0x7f8f040acda0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:27.002+0000 7f8f08bbe640 1 --2- 172.21.15.26:0/1682605704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f04159820 0x7f8f040ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:27.002+0000 7f8f08bbe640 1 --2- 172.21.15.26:0/1682605704 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f8ee4065900 0x7f8ee4067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:27.002+0000 7f8f08bbe640 1 --2- 172.21.15.26:0/1682605704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8f0406f6c0 0x7f8f0414f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T17:58:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:27.002+0000 7f8f08bbe640 1 -- 172.21.15.26:0/1682605704 >> 172.21.15.26:0/1682605704 conn(0x7f8f040ce730 msgr2=0x7f8f040caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T17:58:27.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:27.002+0000 7f8f08bbe640 1 -- 172.21.15.26:0/1682605704 shutdown_connections 2023-12-19T17:58:27.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T17:58:27.002+0000 7f8f08bbe640 1 -- 172.21.15.26:0/1682605704 wait complete. 2023-12-19T17:58:27.026 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-12-19T17:58:38.288 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:58:38.293 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:59:00.018 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:59:00.017+0000 7f13d2ce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:59:00.018 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:59:00.017+0000 7f13d2ce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:59:00.018 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:59:00.017+0000 7f13d2ce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:59:00.018 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:59:00.017+0000 7f13d2ce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:59:00.019 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T17:59:00.017+0000 7f13d2ce6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T17:59:08.352 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:59:08.357 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:59:38.426 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T17:59:38.430 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:00:08.546 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:00:08.551 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:00:38.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:00:38.640 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:01:08.698 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:01:08.704 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:01:38.838 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:01:38.843 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:02:08.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:02:08.981 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:02:39.089 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:02:39.094 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:03:09.218 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:03:09.223 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:03:39.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:03:39.338 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:04:09.463 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:04:09.468 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:04:39.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:04:39.555 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:05:09.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:05:09.688 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:05:39.774 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:05:39.778 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:06:09.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:06:09.905 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:06:40.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:06:40.036 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:07:10.170 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:07:10.174 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:07:40.291 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:07:40.296 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:08:10.404 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:08:10.409 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:08:40.535 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:08:40.540 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:09:10.599 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:09:10.604 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:09:16.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:09:16.027 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-12-19T18:09:16.040 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_59_00_UTC 2023-12-19T18:09:16.040 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_00_00_UTC 2023-12-19T18:09:16.040 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_01_00_UTC 2023-12-19T18:09:16.040 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_02_00_UTC 2023-12-19T18:09:16.040 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_03_00_UTC 2023-12-19T18:09:16.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_04_00_UTC 2023-12-19T18:09:16.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_05_00_UTC 2023-12-19T18:09:16.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_06_00_UTC 2023-12-19T18:09:16.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_07_00_UTC 2023-12-19T18:09:16.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_08_00_UTC 2023-12-19T18:09:16.041 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_09_00_UTC 2023-12-19T18:09:16.042 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-19-17_59_00_UTC', 'scheduled-2023-12-19-18_00_00_UTC', 'scheduled-2023-12-19-18_01_00_UTC', 'scheduled-2023-12-19-18_02_00_UTC', 'scheduled-2023-12-19-18_03_00_UTC', 'scheduled-2023-12-19-18_04_00_UTC', 'scheduled-2023-12-19-18_05_00_UTC', 'scheduled-2023-12-19-18_06_00_UTC', 'scheduled-2023-12-19-18_07_00_UTC', 'scheduled-2023-12-19-18_08_00_UTC', 'scheduled-2023-12-19-18_09_00_UTC'] 2023-12-19T18:09:16.042 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-12-19T18:09:16.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd00d76d640 1 Processor -- start 2023-12-19T18:09:16.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd00d76d640 1 -- start start 2023-12-19T18:09:16.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd00d76d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd00815cfb0 0x7fd0080b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:09:16.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd00d76d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0080d2250 con 0x7fd0080b4600 2023-12-19T18:09:16.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd00d76d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0080d1640 con 0x7fd0080d26e0 2023-12-19T18:09:16.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd00d76d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0080d1920 con 0x7fd00815cfb0 2023-12-19T18:09:16.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd0077fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd0080b4600 0x7fd0080b5cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57202/0 (socket says 172.21.15.26:57202) 2023-12-19T18:09:16.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd0077fe640 1 -- 172.21.15.26:0/951074281 learned_addr learned my addr 172.21.15.26:0/951074281 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:09:16.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd006ffd640 1 --2- 172.21.15.26:0/951074281 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd00815cfb0 0x7fd0080b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:09:16.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd0067fc640 1 -- 172.21.15.26:0/951074281 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924719106 0 0) 0x7fd0080d2250 con 0x7fd0080b4600 2023-12-19T18:09:16.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd0067fc640 1 -- 172.21.15.26:0/951074281 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcff4003530 con 0x7fd0080b4600 2023-12-19T18:09:16.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd0067fc640 1 -- 172.21.15.26:0/951074281 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266107745 0 0) 0x7fd0080d1640 con 0x7fd0080d26e0 2023-12-19T18:09:16.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd0067fc640 1 -- 172.21.15.26:0/951074281 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0080d2250 con 0x7fd0080d26e0 2023-12-19T18:09:16.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd0067fc640 1 -- 172.21.15.26:0/951074281 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2565055617 0 0) 0x7fcff4003530 con 0x7fd0080b4600 2023-12-19T18:09:16.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd006ffd640 1 -- 172.21.15.26:0/951074281 >> v1:172.21.15.160:6789/0 conn(0x7fd0080d26e0 legacy=0x7fd0080d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:09:16.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd006ffd640 1 -- 172.21.15.26:0/951074281 >> v1:172.21.15.26:6789/0 conn(0x7fd0080b4600 legacy=0x7fd0080b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:09:16.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.236+0000 7fd006ffd640 1 -- 172.21.15.26:0/951074281 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0080d1c00 con 0x7fd00815cfb0 2023-12-19T18:09:16.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.240+0000 7fd006ffd640 1 --2- 172.21.15.26:0/951074281 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd00815cfb0 0x7fd0080b8730 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcff80097f0 tx=0x7fcff801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90303d80dd6d29bc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:09:16.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.240+0000 7fd0067fc640 1 -- 172.21.15.26:0/951074281 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000001a00 con 0x7fd00815cfb0 2023-12-19T18:09:16.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.240+0000 7fd0067fc640 1 -- 172.21.15.26:0/951074281 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff80a7050 con 0x7fd00815cfb0 2023-12-19T18:09:16.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.240+0000 7fd0067fc640 1 -- 172.21.15.26:0/951074281 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff80ab620 con 0x7fd00815cfb0 2023-12-19T18:09:16.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.240+0000 7fd00d76d640 1 -- 172.21.15.26:0/951074281 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd00815cfb0 msgr2=0x7fd0080b8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:09:16.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.240+0000 7fd00d76d640 1 --2- 172.21.15.26:0/951074281 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd00815cfb0 0x7fd0080b8730 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcff80097f0 tx=0x7fcff801c3b0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.240+0000 7fd00d76d640 1 -- 172.21.15.26:0/951074281 shutdown_connections 2023-12-19T18:09:16.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.240+0000 7fd00d76d640 1 --2- 172.21.15.26:0/951074281 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd00815cfb0 0x7fd0080b8730 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.240+0000 7fd00d76d640 1 -- 172.21.15.26:0/951074281 >> 172.21.15.26:0/951074281 conn(0x7fd0080cf600 msgr2=0x7fd0080cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:09:16.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fd00d76d640 1 -- 172.21.15.26:0/951074281 shutdown_connections 2023-12-19T18:09:16.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fd00d76d640 1 -- 172.21.15.26:0/951074281 wait complete. 2023-12-19T18:09:16.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fd00d76d640 1 Processor -- start 2023-12-19T18:09:16.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fd00d76d640 1 -- start start 2023-12-19T18:09:16.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fd00d76d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd0080b4600 0x7fd0080cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:09:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fd00d76d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d26e0 0x7fd0080b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:09:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fd00d76d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd00815cfb0 0x7fd0080b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:09:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fd00d76d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00814b870 con 0x7fd0080d26e0 2023-12-19T18:09:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fd00d76d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0080ccaa0 con 0x7fd00815cfb0 2023-12-19T18:09:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fd00d76d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0080c9cf0 con 0x7fd0080b4600 2023-12-19T18:09:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fcffffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d26e0 0x7fd0080b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:09:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fcffffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d26e0 0x7fd0080b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50402/0 (socket says 172.21.15.26:50402) 2023-12-19T18:09:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fcffffff640 1 -- 172.21.15.26:0/2602322885 learned_addr learned my addr 172.21.15.26:0/2602322885 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:09:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fd0077fe640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd00815cfb0 0x7fd0080b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:09:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fd006ffd640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd0080b4600 0x7fd0080cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:09:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fcffffff640 1 -- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd0080b4600 msgr2=0x7fd0080cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:09:16.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fcffffff640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd0080b4600 0x7fd0080cbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fcffffff640 1 -- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd00815cfb0 msgr2=0x7fd0080b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:09:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fcffffff640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd00815cfb0 0x7fd0080b3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fcffffff640 1 -- 172.21.15.26:0/2602322885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff8009420 con 0x7fd0080d26e0 2023-12-19T18:09:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fcffffff640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d26e0 0x7fd0080b2f40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fcff000c500 tx=0x7fcff000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:09:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fcfff7fe640 1 -- 172.21.15.26:0/2602322885 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff00052d0 con 0x7fd0080d26e0 2023-12-19T18:09:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fcfff7fe640 1 -- 172.21.15.26:0/2602322885 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff0005430 con 0x7fd0080d26e0 2023-12-19T18:09:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fd00d76d640 1 -- 172.21.15.26:0/2602322885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0080c9fd0 con 0x7fd0080d26e0 2023-12-19T18:09:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.244+0000 7fcfff7fe640 1 -- 172.21.15.26:0/2602322885 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0012870 con 0x7fd0080d26e0 2023-12-19T18:09:16.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.248+0000 7fd00d76d640 1 -- 172.21.15.26:0/2602322885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0080ca590 con 0x7fd0080d26e0 2023-12-19T18:09:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.248+0000 7fd00d76d640 1 -- 172.21.15.26:0/2602322885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd008074270 con 0x7fd0080d26e0 2023-12-19T18:09:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.248+0000 7fcfff7fe640 1 -- 172.21.15.26:0/2602322885 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 96) v1 ==== 85359+0+0 (secure 0 0 0) 0x7fcff00129d0 con 0x7fd0080d26e0 2023-12-19T18:09:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.248+0000 7fcfff7fe640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcfe8065bd0 0x7fcfe8068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:09:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.248+0000 7fcfff7fe640 1 -- 172.21.15.26:0/2602322885 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6814+0+0 (secure 0 0 0) 0x7fcff00835d0 con 0x7fd0080d26e0 2023-12-19T18:09:16.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.248+0000 7fd006ffd640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcfe8065bd0 0x7fcfe8068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:09:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.252+0000 7fd006ffd640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcfe8065bd0 0x7fcfe8068090 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcff80ad000 tx=0x7fcff8008040 comp rx=0 tx=0).ready entity=mgr.9172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:09:16.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.252+0000 7fcfff7fe640 1 -- 172.21.15.26:0/2602322885 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcff00509a0 con 0x7fd0080d26e0 2023-12-19T18:09:16.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.432+0000 7fd00d76d640 1 -- 172.21.15.26:0/2602322885 --> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0080ca780 con 0x7fcfe8065bd0 2023-12-19T18:09:16.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.440+0000 7fcfff7fe640 1 -- 172.21.15.26:0/2602322885 <== mgr.9172 v2:172.21.15.160:6832/38098 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fd0080ca780 con 0x7fcfe8065bd0 2023-12-19T18:09:16.443 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:09:16.443 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T17:58:26", "first": "2023-12-19T17:59:00", "last": "2023-12-19T18:09:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-12-19T18:09:16.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.444+0000 7fd00d76d640 1 -- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcfe8065bd0 msgr2=0x7fcfe8068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:09:16.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.444+0000 7fd00d76d640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcfe8065bd0 0x7fcfe8068090 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcff80ad000 tx=0x7fcff8008040 comp rx=0 tx=0).stop 2023-12-19T18:09:16.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.444+0000 7fd00d76d640 1 -- 172.21.15.26:0/2602322885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d26e0 msgr2=0x7fd0080b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:09:16.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.444+0000 7fd00d76d640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d26e0 0x7fd0080b2f40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fcff000c500 tx=0x7fcff000c9c0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.444+0000 7fd00d76d640 1 -- 172.21.15.26:0/2602322885 shutdown_connections 2023-12-19T18:09:16.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.444+0000 7fd00d76d640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd00815cfb0 0x7fd0080b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.444+0000 7fd00d76d640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0080d26e0 0x7fd0080b2f40 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.444+0000 7fd00d76d640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7fcfe8065bd0 0x7fcfe8068090 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.444+0000 7fd00d76d640 1 --2- 172.21.15.26:0/2602322885 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd0080b4600 0x7fd0080cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.444+0000 7fd00d76d640 1 -- 172.21.15.26:0/2602322885 >> 172.21.15.26:0/2602322885 conn(0x7fd0080cf600 msgr2=0x7fd008158300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:09:16.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.444+0000 7fd00d76d640 1 -- 172.21.15.26:0/2602322885 shutdown_connections 2023-12-19T18:09:16.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.444+0000 7fd00d76d640 1 -- 172.21.15.26:0/2602322885 wait complete. 2023-12-19T18:09:16.463 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T17:58:26", "first": "2023-12-19T17:59:00", "last": "2023-12-19T18:09:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-12-19T18:09:16.464 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-12-19T00:00:00', 'created': '2023-12-19T17:58:26', 'first': '2023-12-19T17:59:00', 'last': '2023-12-19T18:09:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-12-19T18:09:16.464 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-19T18:09:16.464 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-19T18:09:16.464 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-19T18:09:16.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67efd6d640 1 Processor -- start 2023-12-19T18:09:16.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67efd6d640 1 -- start start 2023-12-19T18:09:16.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67efd6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67e8155030 0x7f67e815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:09:16.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67efd6d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f67e815f370 0x7f67e806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:09:16.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67efd6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e806c970 con 0x7f67e80ce740 2023-12-19T18:09:16.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67efd6d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e815b1f0 con 0x7f67e815f370 2023-12-19T18:09:16.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67efd6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e815b4d0 con 0x7f67e8155030 2023-12-19T18:09:16.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67ee30a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f67e80ce740 0x7f67e80ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57212/0 (socket says 172.21.15.26:57212) 2023-12-19T18:09:16.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67ee30a640 1 -- 172.21.15.26:0/684306602 learned_addr learned my addr 172.21.15.26:0/684306602 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:09:16.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67ed308640 1 --2- 172.21.15.26:0/684306602 >> v2:172.21.15.160:3300/0 conn(0x7f67e815f370 0x7f67e806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:09:16.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67edb09640 1 --2- 172.21.15.26:0/684306602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67e8155030 0x7f67e815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:09:16.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67ecb07640 1 -- 172.21.15.26:0/684306602 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953227993 0 0) 0x7f67e806c970 con 0x7f67e80ce740 2023-12-19T18:09:16.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67ecb07640 1 -- 172.21.15.26:0/684306602 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d4003530 con 0x7f67e80ce740 2023-12-19T18:09:16.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67ecb07640 1 -- 172.21.15.26:0/684306602 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1369012214 0 0) 0x7f67d4003530 con 0x7f67e80ce740 2023-12-19T18:09:16.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67ed308640 1 -- 172.21.15.26:0/684306602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67e8155030 msgr2=0x7f67e815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:09:16.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67ed308640 1 --2- 172.21.15.26:0/684306602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67e8155030 0x7f67e815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67ed308640 1 -- 172.21.15.26:0/684306602 >> v1:172.21.15.26:6789/0 conn(0x7f67e80ce740 legacy=0x7f67e80ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:09:16.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67ed308640 1 -- 172.21.15.26:0/684306602 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e815b7b0 con 0x7f67e815f370 2023-12-19T18:09:16.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.608+0000 7f67edb09640 1 --2- 172.21.15.26:0/684306602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67e8155030 0x7f67e815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:09:16.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.612+0000 7f67ed308640 1 --2- 172.21.15.26:0/684306602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e815f370 0x7f67e806c080 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f67d80097c0 tx=0x7f67d801c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb3b4ba376db5501 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:09:16.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.612+0000 7f67ecb07640 1 -- 172.21.15.26:0/684306602 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0001a00 con 0x7f67e815f370 2023-12-19T18:09:16.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.612+0000 7f67ecb07640 1 -- 172.21.15.26:0/684306602 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d80a7050 con 0x7f67e815f370 2023-12-19T18:09:16.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.612+0000 7f67ecb07640 1 -- 172.21.15.26:0/684306602 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d80ab600 con 0x7f67e815f370 2023-12-19T18:09:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.612+0000 7f67efd6d640 1 -- 172.21.15.26:0/684306602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e815f370 msgr2=0x7f67e806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:09:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.612+0000 7f67efd6d640 1 --2- 172.21.15.26:0/684306602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e815f370 0x7f67e806c080 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f67d80097c0 tx=0x7f67d801c3a0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.612+0000 7f67efd6d640 1 -- 172.21.15.26:0/684306602 shutdown_connections 2023-12-19T18:09:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.612+0000 7f67efd6d640 1 --2- 172.21.15.26:0/684306602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e815f370 0x7f67e806c080 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.612+0000 7f67efd6d640 1 --2- 172.21.15.26:0/684306602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67e8155030 0x7f67e815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.612+0000 7f67efd6d640 1 -- 172.21.15.26:0/684306602 >> 172.21.15.26:0/684306602 conn(0x7f67e80658e0 msgr2=0x7f67e8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:09:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.612+0000 7f67efd6d640 1 -- 172.21.15.26:0/684306602 shutdown_connections 2023-12-19T18:09:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.612+0000 7f67efd6d640 1 -- 172.21.15.26:0/684306602 wait complete. 2023-12-19T18:09:16.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67efd6d640 1 Processor -- start 2023-12-19T18:09:16.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67efd6d640 1 -- start start 2023-12-19T18:09:16.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67efd6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e80ce740 0x7f67e80aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67efd6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e8155030 0x7f67e814e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67efd6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67e815f370 0x7f67e814a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67efd6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e814b380 con 0x7f67e80ce740 2023-12-19T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67efd6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e814b660 con 0x7f67e8155030 2023-12-19T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67efd6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e814b940 con 0x7f67e815f370 2023-12-19T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67edb09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e80ce740 0x7f67e80aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67edb09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e80ce740 0x7f67e80aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50406/0 (socket says 172.21.15.26:50406) 2023-12-19T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67edb09640 1 -- 172.21.15.26:0/3712554809 learned_addr learned my addr 172.21.15.26:0/3712554809 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67ed308640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e8155030 0x7f67e814e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67ee30a640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67e815f370 0x7f67e814a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67edb09640 1 -- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67e815f370 msgr2=0x7f67e814a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67edb09640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67e815f370 0x7f67e814a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67edb09640 1 -- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e8155030 msgr2=0x7f67e814e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67edb09640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e8155030 0x7f67e814e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67edb09640 1 -- 172.21.15.26:0/3712554809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e4009420 con 0x7f67e80ce740 2023-12-19T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67ee30a640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67e815f370 0x7f67e814a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67edb09640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e80ce740 0x7f67e80aff00 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f67e400ebe0 tx=0x7f67e400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67de7fc640 1 -- 172.21.15.26:0/3712554809 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e400eee0 con 0x7f67e80ce740 2023-12-19T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67de7fc640 1 -- 172.21.15.26:0/3712554809 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e400cc30 con 0x7f67e80ce740 2023-12-19T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67efd6d640 1 -- 172.21.15.26:0/3712554809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d8009420 con 0x7f67e80ce740 2023-12-19T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67de7fc640 1 -- 172.21.15.26:0/3712554809 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e4013470 con 0x7f67e80ce740 2023-12-19T18:09:16.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.616+0000 7f67efd6d640 1 -- 172.21.15.26:0/3712554809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e806c740 con 0x7f67e80ce740 2023-12-19T18:09:16.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.620+0000 7f67efd6d640 1 -- 172.21.15.26:0/3712554809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e8149f20 con 0x7f67e80ce740 2023-12-19T18:09:16.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.620+0000 7f67de7fc640 1 -- 172.21.15.26:0/3712554809 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 96) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f67e40135d0 con 0x7f67e80ce740 2023-12-19T18:09:16.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.620+0000 7f67de7fc640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f67cc065bd0 0x7f67cc068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:09:16.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.620+0000 7f67de7fc640 1 -- 172.21.15.26:0/3712554809 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f67e4083c00 con 0x7f67e80ce740 2023-12-19T18:09:16.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.620+0000 7f67ed308640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f67cc065bd0 0x7f67cc068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:09:16.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.624+0000 7f67ed308640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f67cc065bd0 0x7f67cc068090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f67d80ad000 tx=0x7f67d80a6000 comp rx=0 tx=0).ready entity=mgr.9172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:09:16.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.624+0000 7f67de7fc640 1 -- 172.21.15.26:0/3712554809 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f67e4050f40 con 0x7f67e80ce740 2023-12-19T18:09:16.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.816+0000 7f67efd6d640 1 -- 172.21.15.26:0/3712554809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f67e8074270 con 0x7f67e80ce740 2023-12-19T18:09:16.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.820+0000 7f67de7fc640 1 -- 172.21.15.26:0/3712554809 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v96) v1 ==== 74+0+163579 (secure 0 0 0) 0x7f67e4054560 con 0x7f67e80ce740 2023-12-19T18:09:16.824 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:09:16.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.824+0000 7f67efd6d640 1 -- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f67cc065bd0 msgr2=0x7f67cc068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:09:16.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.824+0000 7f67efd6d640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f67cc065bd0 0x7f67cc068090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f67d80ad000 tx=0x7f67d80a6000 comp rx=0 tx=0).stop 2023-12-19T18:09:16.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.824+0000 7f67efd6d640 1 -- 172.21.15.26:0/3712554809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e80ce740 msgr2=0x7f67e80aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:09:16.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.824+0000 7f67efd6d640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e80ce740 0x7f67e80aff00 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f67e400ebe0 tx=0x7f67e400c520 comp rx=0 tx=0).stop 2023-12-19T18:09:16.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.824+0000 7f67efd6d640 1 -- 172.21.15.26:0/3712554809 shutdown_connections 2023-12-19T18:09:16.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.824+0000 7f67efd6d640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:6832/38098,v1:172.21.15.160:6833/38098] conn(0x7f67cc065bd0 0x7f67cc068090 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.824+0000 7f67efd6d640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f67e815f370 0x7f67e814a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.824+0000 7f67efd6d640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67e8155030 0x7f67e814e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.824+0000 7f67efd6d640 1 --2- 172.21.15.26:0/3712554809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f67e80ce740 0x7f67e80aff00 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:09:16.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.824+0000 7f67efd6d640 1 -- 172.21.15.26:0/3712554809 >> 172.21.15.26:0/3712554809 conn(0x7f67e80658e0 msgr2=0x7f67e815a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:09:16.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.824+0000 7f67efd6d640 1 -- 172.21.15.26:0/3712554809 shutdown_connections 2023-12-19T18:09:16.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:09:16.824+0000 7f67efd6d640 1 -- 172.21.15.26:0/3712554809 wait complete. 2023-12-19T18:09:16.844 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":96,"active_gid":9172,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":38098},{"type":"v1","addr":"172.21.15.160:6833","nonce":38098}]},"active_addr":"172.21.15.160:6833/38098","active_change":"2023-12-19T17:58:25.502115+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":8120,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":178,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":3458963623}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":1302880583}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":4076846554}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":1333176784}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":3381018054}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":1063410185}]}]} 2023-12-19T18:09:16.850 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: x 2023-12-19T18:09:16.850 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-12-19T18:09:16.850 INFO:tasks.ceph.mgr.x:Stopping old one... 2023-12-19T18:09:16.851 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-12-19T18:09:16.851 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:09:16.914 INFO:tasks.ceph.mgr.x:Stopped 2023-12-19T18:09:16.914 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-12-19T18:09:16.918 INFO:tasks.ceph.mgr.x:Started 2023-12-19T18:09:17.093 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:17.092+0000 7efeab7db180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:09:17.507 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:17.504+0000 7efeab7db180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:09:17.658 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:17.656+0000 7efeab7db180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:09:18.134 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:18.132+0000 7efeab7db180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:09:18.190 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:18.188+0000 7efeab7db180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:09:18.246 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:18.244+0000 7efeab7db180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:09:18.331 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:18.328+0000 7efeab7db180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:09:18.386 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:18.384+0000 7efeab7db180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:09:18.469 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:18.468+0000 7efeab7db180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:09:18.524 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:18.524+0000 7efeab7db180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:09:18.667 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:18.664+0000 7efeab7db180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:09:18.724 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:18.724+0000 7efeab7db180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:09:18.963 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:09:18.963 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:09:18.963 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:09:18.972 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:18.972+0000 7efeab7db180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:09:19.106 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:19.104+0000 7efeab7db180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:09:19.228 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:19.228+0000 7efeab7db180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:09:19.297 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:19.296+0000 7efeab7db180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:09:19.431 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:19.428+0000 7efeab7db180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:09:19.965 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:19.964+0000 7efeab7db180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:09:20.050 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.048+0000 7efeab7db180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:09:20.196 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.196+0000 7efeab7db180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:09:20.279 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.276+0000 7efeab7db180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:09:20.592 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.592+0000 7efeab7db180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:09:20.652 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.652+0000 7efeab7db180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:09:20.817 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.816+0000 7efe30e57640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:09:20.817 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.816+0000 7efe30e57640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:09:20.817 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.816+0000 7efe30e57640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:09:20.817 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.816+0000 7efe30e57640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:09:20.818 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.816+0000 7efe30e57640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:09:20.818 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.816+0000 7efe2de51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:09:20.818 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.816+0000 7efe2de51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:09:20.818 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.816+0000 7efe2de51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:09:20.818 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.816+0000 7efe2de51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:09:20.818 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:09:20.816+0000 7efe2de51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:09:40.680 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:09:40.685 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:10:00.815 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:10:00.811+0000 7efe38e67640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:10:00.816 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:10:00.811+0000 7efe38e67640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:10:00.816 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:10:00.811+0000 7efe38e67640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:10:00.816 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:10:00.811+0000 7efe38e67640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:10:00.816 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:10:00.811+0000 7efe38e67640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:10:10.800 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:10:10.804 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:10:40.880 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:10:40.885 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:11:10.936 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:11:10.940 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:11:41.092 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:11:41.096 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:12:11.214 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:12:11.219 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:12:41.345 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:12:41.350 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:13:11.475 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:13:11.479 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:13:41.597 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:13:41.602 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:14:11.744 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:14:11.749 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:14:16.919 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/test_restart 1M 2023-12-19T18:14:17.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.076+0000 7f54eae54640 1 Processor -- start 2023-12-19T18:14:17.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.076+0000 7f54eae54640 1 -- start start 2023-12-19T18:14:17.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54eae54640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f54e415cc90 0x7f54e40cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54eae54640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f54e4160a20 0x7f54e40d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54eae54640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e40cdb20 con 0x7f54e415d070 2023-12-19T18:14:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54eae54640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e40cde00 con 0x7f54e415cc90 2023-12-19T18:14:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54eae54640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e40ce0e0 con 0x7f54e4160a20 2023-12-19T18:14:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54e9e52640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f54e415cc90 0x7f54e40cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54e9e52640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f54e415cc90 0x7f54e40cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45234/0 (socket says 172.21.15.26:45234) 2023-12-19T18:14:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54e9e52640 1 -- 172.21.15.26:0/991826034 learned_addr learned my addr 172.21.15.26:0/991826034 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54ea653640 1 --2- 172.21.15.26:0/991826034 >> v2:172.21.15.160:3300/0 conn(0x7f54e4160a20 0x7f54e40d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54e9e52640 1 -- 172.21.15.26:0/991826034 >> v2:172.21.15.160:3300/0 conn(0x7f54e4160a20 msgr2=0x7f54e40d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54e9e52640 1 --2- 172.21.15.26:0/991826034 >> v2:172.21.15.160:3300/0 conn(0x7f54e4160a20 0x7f54e40d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54e9e52640 1 -- 172.21.15.26:0/991826034 >> v1:172.21.15.160:6789/0 conn(0x7f54e415d070 legacy=0x7f54e40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54e9e52640 1 -- 172.21.15.26:0/991826034 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40ce3c0 con 0x7f54e415cc90 2023-12-19T18:14:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.080+0000 7f54ea653640 1 --2- 172.21.15.26:0/991826034 >> v2:172.21.15.160:3300/0 conn(0x7f54e4160a20 0x7f54e40d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:17.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54e9e52640 1 --2- 172.21.15.26:0/991826034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e415cc90 0x7f54e40cd2f0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f54e40b2320 tx=0x7f54e001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24e9524e10c21d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:17.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54e8e50640 1 -- 172.21.15.26:0/991826034 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e00ab670 con 0x7f54e415cc90 2023-12-19T18:14:17.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54e8e50640 1 -- 172.21.15.26:0/991826034 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e00ab7d0 con 0x7f54e415cc90 2023-12-19T18:14:17.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54e8e50640 1 -- 172.21.15.26:0/991826034 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e00abb60 con 0x7f54e415cc90 2023-12-19T18:14:17.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54eae54640 1 -- 172.21.15.26:0/991826034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e415cc90 msgr2=0x7f54e40cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54eae54640 1 --2- 172.21.15.26:0/991826034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e415cc90 0x7f54e40cd2f0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f54e40b2320 tx=0x7f54e001c490 comp rx=0 tx=0).stop 2023-12-19T18:14:17.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54eae54640 1 -- 172.21.15.26:0/991826034 shutdown_connections 2023-12-19T18:14:17.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54eae54640 1 --2- 172.21.15.26:0/991826034 >> v2:172.21.15.160:3300/0 conn(0x7f54e4160a20 0x7f54e40d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54eae54640 1 --2- 172.21.15.26:0/991826034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e415cc90 0x7f54e40cd2f0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54eae54640 1 -- 172.21.15.26:0/991826034 >> 172.21.15.26:0/991826034 conn(0x7f54e406a320 msgr2=0x7f54e40658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:17.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54eae54640 1 -- 172.21.15.26:0/991826034 shutdown_connections 2023-12-19T18:14:17.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54eae54640 1 -- 172.21.15.26:0/991826034 wait complete. 2023-12-19T18:14:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54eae54640 1 Processor -- start 2023-12-19T18:14:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.084+0000 7f54eae54640 1 -- start start 2023-12-19T18:14:17.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54eae54640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54e415cc90 0x7f54e4153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54eae54640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54e415d070 0x7f54e40cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54eae54640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e4160a20 0x7f54e415e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54eae54640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e4156a70 con 0x7f54e4160a20 2023-12-19T18:14:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54eae54640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e41562a0 con 0x7f54e415cc90 2023-12-19T18:14:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54eae54640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40cc140 con 0x7f54e415d070 2023-12-19T18:14:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54ea653640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e4160a20 0x7f54e415e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54e9e52640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54e415cc90 0x7f54e4153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54ea653640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e4160a20 0x7f54e415e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45248/0 (socket says 172.21.15.26:45248) 2023-12-19T18:14:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54ea653640 1 -- 172.21.15.26:0/2537346605 learned_addr learned my addr 172.21.15.26:0/2537346605 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54e9651640 1 --2- 172.21.15.26:0/2537346605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54e415d070 0x7f54e40cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54ea653640 1 -- 172.21.15.26:0/2537346605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54e415d070 msgr2=0x7f54e40cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54ea653640 1 --2- 172.21.15.26:0/2537346605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54e415d070 0x7f54e40cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54ea653640 1 -- 172.21.15.26:0/2537346605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54e415cc90 msgr2=0x7f54e4153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54ea653640 1 --2- 172.21.15.26:0/2537346605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54e415cc90 0x7f54e4153a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54ea653640 1 -- 172.21.15.26:0/2537346605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e00094d0 con 0x7f54e4160a20 2023-12-19T18:14:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54ea653640 1 --2- 172.21.15.26:0/2537346605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e4160a20 0x7f54e415e7b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00cd80 tx=0x7f54dc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54da7fc640 1 -- 172.21.15.26:0/2537346605 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00f680 con 0x7f54e4160a20 2023-12-19T18:14:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54da7fc640 1 -- 172.21.15.26:0/2537346605 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc00f800 con 0x7f54e4160a20 2023-12-19T18:14:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54da7fc640 1 -- 172.21.15.26:0/2537346605 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00fbb0 con 0x7f54e4160a20 2023-12-19T18:14:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54eae54640 1 -- 172.21.15.26:0/2537346605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e40cbb50 con 0x7f54e4160a20 2023-12-19T18:14:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54eae54640 1 -- 172.21.15.26:0/2537346605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e4156790 con 0x7f54e4160a20 2023-12-19T18:14:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54eae54640 1 -- 172.21.15.26:0/2537346605 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e40cc2b0 con 0x7f54e4160a20 2023-12-19T18:14:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.088+0000 7f54da7fc640 1 -- 172.21.15.26:0/2537346605 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 102) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f54dc00b040 con 0x7f54e4160a20 2023-12-19T18:14:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.092+0000 7f54da7fc640 1 --2- 172.21.15.26:0/2537346605 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f54c8065c20 0x7f54c80680e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.092+0000 7f54da7fc640 1 -- 172.21.15.26:0/2537346605 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 7072+0+0 (secure 0 0 0) 0x7f54dc0842a0 con 0x7f54e4160a20 2023-12-19T18:14:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.092+0000 7f54e9e52640 1 --2- 172.21.15.26:0/2537346605 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f54c8065c20 0x7f54c80680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.092+0000 7f54e9e52640 1 --2- 172.21.15.26:0/2537346605 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f54c8065c20 0x7f54c80680e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f54e00af000 tx=0x7f54e00b6040 comp rx=0 tx=0).ready entity=mgr.8204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:17.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.096+0000 7f54da7fc640 1 -- 172.21.15.26:0/2537346605 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f54dc0514f0 con 0x7f54e4160a20 2023-12-19T18:14:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.276+0000 7f54eae54640 1 -- 172.21.15.26:0/2537346605 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f54e4061930 con 0x7f54c8065c20 2023-12-19T18:14:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.340+0000 7f54da7fc640 1 -- 172.21.15.26:0/2537346605 <== mgr.8204 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f54e4061930 con 0x7f54c8065c20 2023-12-19T18:14:17.343 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-12-19T18:14:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.344+0000 7f54eae54640 1 -- 172.21.15.26:0/2537346605 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f54c8065c20 msgr2=0x7f54c80680e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.344+0000 7f54eae54640 1 --2- 172.21.15.26:0/2537346605 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f54c8065c20 0x7f54c80680e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f54e00af000 tx=0x7f54e00b6040 comp rx=0 tx=0).stop 2023-12-19T18:14:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.344+0000 7f54eae54640 1 -- 172.21.15.26:0/2537346605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e4160a20 msgr2=0x7f54e415e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.344+0000 7f54eae54640 1 --2- 172.21.15.26:0/2537346605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e4160a20 0x7f54e415e7b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00cd80 tx=0x7f54dc00a650 comp rx=0 tx=0).stop 2023-12-19T18:14:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.344+0000 7f54eae54640 1 -- 172.21.15.26:0/2537346605 shutdown_connections 2023-12-19T18:14:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.344+0000 7f54eae54640 1 --2- 172.21.15.26:0/2537346605 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f54c8065c20 0x7f54c80680e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.344+0000 7f54eae54640 1 --2- 172.21.15.26:0/2537346605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54e4160a20 0x7f54e415e7b0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.344+0000 7f54eae54640 1 --2- 172.21.15.26:0/2537346605 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f54e415d070 0x7f54e40cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.344+0000 7f54eae54640 1 --2- 172.21.15.26:0/2537346605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54e415cc90 0x7f54e4153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.344+0000 7f54eae54640 1 -- 172.21.15.26:0/2537346605 >> 172.21.15.26:0/2537346605 conn(0x7f54e406a320 msgr2=0x7f54e415f840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.344+0000 7f54eae54640 1 -- 172.21.15.26:0/2537346605 shutdown_connections 2023-12-19T18:14:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.344+0000 7f54eae54640 1 -- 172.21.15.26:0/2537346605 wait complete. 2023-12-19T18:14:17.363 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-12-19T18:14:17.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:17.364 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-12-19T18:14:17.374 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_59_00_UTC 2023-12-19T18:14:17.374 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_00_00_UTC 2023-12-19T18:14:17.374 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_01_00_UTC 2023-12-19T18:14:17.374 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_02_00_UTC 2023-12-19T18:14:17.374 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_03_00_UTC 2023-12-19T18:14:17.374 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_04_00_UTC 2023-12-19T18:14:17.374 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_05_00_UTC 2023-12-19T18:14:17.374 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_06_00_UTC 2023-12-19T18:14:17.374 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_07_00_UTC 2023-12-19T18:14:17.374 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_08_00_UTC 2023-12-19T18:14:17.375 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_09_00_UTC 2023-12-19T18:14:17.375 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_10_00_UTC 2023-12-19T18:14:17.375 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_11_00_UTC 2023-12-19T18:14:17.375 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_12_00_UTC 2023-12-19T18:14:17.375 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_13_00_UTC 2023-12-19T18:14:17.375 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_14_00_UTC 2023-12-19T18:14:17.376 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-19-17_59_00_UTC', 'scheduled-2023-12-19-18_00_00_UTC', 'scheduled-2023-12-19-18_01_00_UTC', 'scheduled-2023-12-19-18_02_00_UTC', 'scheduled-2023-12-19-18_03_00_UTC', 'scheduled-2023-12-19-18_04_00_UTC', 'scheduled-2023-12-19-18_05_00_UTC', 'scheduled-2023-12-19-18_06_00_UTC', 'scheduled-2023-12-19-18_07_00_UTC', 'scheduled-2023-12-19-18_08_00_UTC', 'scheduled-2023-12-19-18_09_00_UTC', 'scheduled-2023-12-19-18_10_00_UTC', 'scheduled-2023-12-19-18_11_00_UTC', 'scheduled-2023-12-19-18_12_00_UTC', 'scheduled-2023-12-19-18_13_00_UTC', 'scheduled-2023-12-19-18_14_00_UTC'] 2023-12-19T18:14:17.376 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-12-19T18:14:17.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bffb3640 1 Processor -- start 2023-12-19T18:14:17.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bffb3640 1 -- start start 2023-12-19T18:14:17.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bffb3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f15b815cbd0 0x7f15b80b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bffb3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15b80cc720 0x7f15b80ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bffb3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b80cd080 con 0x7f15b8160850 2023-12-19T18:14:17.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bffb3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b80ac900 con 0x7f15b815cbd0 2023-12-19T18:14:17.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bffb3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80acbe0 con 0x7f15b80cc720 2023-12-19T18:14:17.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15be550640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f15b8160850 0x7f15b8160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46746/0 (socket says 172.21.15.26:46746) 2023-12-19T18:14:17.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15be550640 1 -- 172.21.15.26:0/693664621 learned_addr learned my addr 172.21.15.26:0/693664621 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:17.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bdd4f640 1 --2- 172.21.15.26:0/693664621 >> v2:172.21.15.160:3300/0 conn(0x7f15b815cbd0 0x7f15b80b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bd54e640 1 --2- 172.21.15.26:0/693664621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15b80cc720 0x7f15b80ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bcd4d640 1 -- 172.21.15.26:0/693664621 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907347558 0 0) 0x7f15b80cd080 con 0x7f15b8160850 2023-12-19T18:14:17.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bcd4d640 1 -- 172.21.15.26:0/693664621 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15a4003530 con 0x7f15b8160850 2023-12-19T18:14:17.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bdd4f640 1 -- 172.21.15.26:0/693664621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15b80cc720 msgr2=0x7f15b80ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:17.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bdd4f640 1 --2- 172.21.15.26:0/693664621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15b80cc720 0x7f15b80ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bdd4f640 1 -- 172.21.15.26:0/693664621 >> v1:172.21.15.26:6789/0 conn(0x7f15b8160850 legacy=0x7f15b8160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.564+0000 7f15bdd4f640 1 -- 172.21.15.26:0/693664621 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b80acec0 con 0x7f15b815cbd0 2023-12-19T18:14:17.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.568+0000 7f15bdd4f640 1 --2- 172.21.15.26:0/693664621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b815cbd0 0x7f15b80b0910 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f15b40097c0 tx=0x7f15b401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6de2ffaa6c73dfbf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.568+0000 7f15bcd4d640 1 -- 172.21.15.26:0/693664621 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b40ab650 con 0x7f15b815cbd0 2023-12-19T18:14:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.568+0000 7f15bcd4d640 1 -- 172.21.15.26:0/693664621 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b40ab7b0 con 0x7f15b815cbd0 2023-12-19T18:14:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.568+0000 7f15bcd4d640 1 -- 172.21.15.26:0/693664621 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b40abb40 con 0x7f15b815cbd0 2023-12-19T18:14:17.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.568+0000 7f15bffb3640 1 -- 172.21.15.26:0/693664621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b815cbd0 msgr2=0x7f15b80b0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.568+0000 7f15bffb3640 1 --2- 172.21.15.26:0/693664621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b815cbd0 0x7f15b80b0910 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f15b40097c0 tx=0x7f15b401c450 comp rx=0 tx=0).stop 2023-12-19T18:14:17.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.568+0000 7f15bffb3640 1 -- 172.21.15.26:0/693664621 shutdown_connections 2023-12-19T18:14:17.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.568+0000 7f15bffb3640 1 --2- 172.21.15.26:0/693664621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15b80cc720 0x7f15b80ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.568+0000 7f15bffb3640 1 --2- 172.21.15.26:0/693664621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b815cbd0 0x7f15b80b0910 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.568+0000 7f15bffb3640 1 -- 172.21.15.26:0/693664621 >> 172.21.15.26:0/693664621 conn(0x7f15b80cebe0 msgr2=0x7f15b80ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:17.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.568+0000 7f15bffb3640 1 -- 172.21.15.26:0/693664621 shutdown_connections 2023-12-19T18:14:17.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.568+0000 7f15bffb3640 1 -- 172.21.15.26:0/693664621 wait complete. 2023-12-19T18:14:17.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bffb3640 1 Processor -- start 2023-12-19T18:14:17.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bffb3640 1 -- start start 2023-12-19T18:14:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bffb3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b80cc720 0x7f15b8150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bffb3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b815cbd0 0x7f15b80b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bffb3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15b8160850 0x7f15b814be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bffb3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80aea20 con 0x7f15b80cc720 2023-12-19T18:14:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bffb3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80aed00 con 0x7f15b815cbd0 2023-12-19T18:14:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bffb3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80aefe0 con 0x7f15b8160850 2023-12-19T18:14:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bdd4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b80cc720 0x7f15b8150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bdd4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b80cc720 0x7f15b8150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45258/0 (socket says 172.21.15.26:45258) 2023-12-19T18:14:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bdd4f640 1 -- 172.21.15.26:0/2532279748 learned_addr learned my addr 172.21.15.26:0/2532279748 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bd54e640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b815cbd0 0x7f15b80b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15be550640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15b8160850 0x7f15b814be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bdd4f640 1 -- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15b8160850 msgr2=0x7f15b814be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bdd4f640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15b8160850 0x7f15b814be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bdd4f640 1 -- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b815cbd0 msgr2=0x7f15b80b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bdd4f640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b815cbd0 0x7f15b80b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bdd4f640 1 -- 172.21.15.26:0/2532279748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b4009420 con 0x7f15b80cc720 2023-12-19T18:14:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bdd4f640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b80cc720 0x7f15b8150ee0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f15b40ad040 tx=0x7f15b401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15ae7fc640 1 -- 172.21.15.26:0/2532279748 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b40bf610 con 0x7f15b80cc720 2023-12-19T18:14:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15ae7fc640 1 -- 172.21.15.26:0/2532279748 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b40bf770 con 0x7f15b80cc720 2023-12-19T18:14:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bffb3640 1 -- 172.21.15.26:0/2532279748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b80af2c0 con 0x7f15b80cc720 2023-12-19T18:14:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15ae7fc640 1 -- 172.21.15.26:0/2532279748 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b40bfae0 con 0x7f15b80cc720 2023-12-19T18:14:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bffb3640 1 -- 172.21.15.26:0/2532279748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b80b0030 con 0x7f15b80cc720 2023-12-19T18:14:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.572+0000 7f15bffb3640 1 -- 172.21.15.26:0/2532279748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b8074270 con 0x7f15b80cc720 2023-12-19T18:14:17.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.576+0000 7f15ae7fc640 1 -- 172.21.15.26:0/2532279748 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 102) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f15b40bfc40 con 0x7f15b80cc720 2023-12-19T18:14:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.576+0000 7f15ae7fc640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f159c065bd0 0x7f159c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.576+0000 7f15ae7fc640 1 -- 172.21.15.26:0/2532279748 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 7072+0+0 (secure 0 0 0) 0x7f15b411e800 con 0x7f15b80cc720 2023-12-19T18:14:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.576+0000 7f15bd54e640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f159c065bd0 0x7f159c068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.576+0000 7f15bd54e640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f159c065bd0 0x7f159c068090 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f15b806ca00 tx=0x7f15a8008040 comp rx=0 tx=0).ready entity=mgr.8204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:17.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.580+0000 7f15ae7fc640 1 -- 172.21.15.26:0/2532279748 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f15b40ebaf0 con 0x7f15b80cc720 2023-12-19T18:14:17.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.756+0000 7f15bffb3640 1 -- 172.21.15.26:0/2532279748 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15b80c9880 con 0x7f159c065bd0 2023-12-19T18:14:17.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.768+0000 7f15ae7fc640 1 -- 172.21.15.26:0/2532279748 <== mgr.8204 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7f15b80c9880 con 0x7f159c065bd0 2023-12-19T18:14:17.767 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:17.767 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T17:58:26", "first": "2023-12-19T17:59:00", "last": "2023-12-19T18:14:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-12-19T18:14:17.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.768+0000 7f15bffb3640 1 -- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f159c065bd0 msgr2=0x7f159c068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.768+0000 7f15bffb3640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f159c065bd0 0x7f159c068090 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f15b806ca00 tx=0x7f15a8008040 comp rx=0 tx=0).stop 2023-12-19T18:14:17.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.768+0000 7f15bffb3640 1 -- 172.21.15.26:0/2532279748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b80cc720 msgr2=0x7f15b8150ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.768+0000 7f15bffb3640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b80cc720 0x7f15b8150ee0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f15b40ad040 tx=0x7f15b401ac90 comp rx=0 tx=0).stop 2023-12-19T18:14:17.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.768+0000 7f15bffb3640 1 -- 172.21.15.26:0/2532279748 shutdown_connections 2023-12-19T18:14:17.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.768+0000 7f15bffb3640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f159c065bd0 0x7f159c068090 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.768+0000 7f15bffb3640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f15b8160850 0x7f15b814be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.768+0000 7f15bffb3640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15b815cbd0 0x7f15b80b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.768+0000 7f15bffb3640 1 --2- 172.21.15.26:0/2532279748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f15b80cc720 0x7f15b8150ee0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.768+0000 7f15bffb3640 1 -- 172.21.15.26:0/2532279748 >> 172.21.15.26:0/2532279748 conn(0x7f15b80cebe0 msgr2=0x7f15b8071060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:17.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.768+0000 7f15bffb3640 1 -- 172.21.15.26:0/2532279748 shutdown_connections 2023-12-19T18:14:17.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.768+0000 7f15bffb3640 1 -- 172.21.15.26:0/2532279748 wait complete. 2023-12-19T18:14:17.788 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T17:58:26", "first": "2023-12-19T17:59:00", "last": "2023-12-19T18:14:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-12-19T18:14:17.788 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-12-19T00:00:00', 'created': '2023-12-19T17:58:26', 'first': '2023-12-19T17:59:00', 'last': '2023-12-19T18:14:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-12-19T18:14:17.788 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-12-19T18:14:17.788 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-12-19T18:14:17.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/test_restart 1M 2023-12-19T18:14:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.932+0000 7f71ad50a640 1 Processor -- start 2023-12-19T18:14:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.932+0000 7f71ad50a640 1 -- start start 2023-12-19T18:14:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f71ad50a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f71a815ada0 0x7f71a80706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f71ad50a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71a806ac80 0x7f71a80b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f71ad50a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a8070d90 con 0x7f71a815a9c0 2023-12-19T18:14:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f71ad50a640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a80cc270 con 0x7f71a815ada0 2023-12-19T18:14:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f71ad50a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a8071070 con 0x7f71a806ac80 2023-12-19T18:14:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f719ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f71a815ada0 0x7f71a80706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f71a6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f71a815a9c0 0x7f71a80b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46756/0 (socket says 172.21.15.26:46756) 2023-12-19T18:14:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f71a6ffd640 1 -- 172.21.15.26:0/3584791454 learned_addr learned my addr 172.21.15.26:0/3584791454 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f719ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f71a815ada0 0x7f71a80706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45270/0 (socket says 172.21.15.26:45270) 2023-12-19T18:14:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f71a77fe640 1 --2- 172.21.15.26:0/3584791454 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71a806ac80 0x7f71a80b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f719ffff640 1 -- 172.21.15.26:0/3584791454 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71a806ac80 msgr2=0x7f71a80b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f719ffff640 1 --2- 172.21.15.26:0/3584791454 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71a806ac80 0x7f71a80b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f719ffff640 1 -- 172.21.15.26:0/3584791454 >> v1:172.21.15.26:6789/0 conn(0x7f71a815a9c0 legacy=0x7f71a80b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.936+0000 7f719ffff640 1 -- 172.21.15.26:0/3584791454 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a8071350 con 0x7f71a815ada0 2023-12-19T18:14:17.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f719ffff640 1 --2- 172.21.15.26:0/3584791454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a815ada0 0x7f71a80706d0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f71900097c0 tx=0x7f719001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c961f2d1559aaa1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f71a67fc640 1 -- 172.21.15.26:0/3584791454 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71900ab600 con 0x7f71a815ada0 2023-12-19T18:14:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f71a67fc640 1 -- 172.21.15.26:0/3584791454 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71900a7050 con 0x7f71a815ada0 2023-12-19T18:14:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f71a67fc640 1 -- 172.21.15.26:0/3584791454 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71900ab980 con 0x7f71a815ada0 2023-12-19T18:14:17.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f71ad50a640 1 -- 172.21.15.26:0/3584791454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a815ada0 msgr2=0x7f71a80706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f71ad50a640 1 --2- 172.21.15.26:0/3584791454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a815ada0 0x7f71a80706d0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f71900097c0 tx=0x7f719001c3c0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f71ad50a640 1 -- 172.21.15.26:0/3584791454 shutdown_connections 2023-12-19T18:14:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f71ad50a640 1 --2- 172.21.15.26:0/3584791454 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71a806ac80 0x7f71a80b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f71ad50a640 1 --2- 172.21.15.26:0/3584791454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a815ada0 0x7f71a80706d0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f71ad50a640 1 -- 172.21.15.26:0/3584791454 >> 172.21.15.26:0/3584791454 conn(0x7f71a80ce730 msgr2=0x7f71a80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f71ad50a640 1 -- 172.21.15.26:0/3584791454 shutdown_connections 2023-12-19T18:14:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f71ad50a640 1 -- 172.21.15.26:0/3584791454 wait complete. 2023-12-19T18:14:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f71ad50a640 1 Processor -- start 2023-12-19T18:14:17.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.940+0000 7f71ad50a640 1 -- start start 2023-12-19T18:14:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71ad50a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a806ac80 0x7f71a8150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71ad50a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a815a9c0 0x7f71a8159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71ad50a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71a815ada0 0x7f71a8159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71ad50a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a80c6220 con 0x7f71a806ac80 2023-12-19T18:14:17.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71ad50a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a80c6470 con 0x7f71a815a9c0 2023-12-19T18:14:17.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71ad50a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a80c6750 con 0x7f71a815ada0 2023-12-19T18:14:17.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71a6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a806ac80 0x7f71a8150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71a6ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a806ac80 0x7f71a8150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45272/0 (socket says 172.21.15.26:45272) 2023-12-19T18:14:17.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71a6ffd640 1 -- 172.21.15.26:0/1243930646 learned_addr learned my addr 172.21.15.26:0/1243930646 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:17.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71a77fe640 1 --2- 172.21.15.26:0/1243930646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71a815ada0 0x7f71a8159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71a6ffd640 1 -- 172.21.15.26:0/1243930646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71a815ada0 msgr2=0x7f71a8159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71a6ffd640 1 --2- 172.21.15.26:0/1243930646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71a815ada0 0x7f71a8159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71a6ffd640 1 -- 172.21.15.26:0/1243930646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a815a9c0 msgr2=0x7f71a8159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:17.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71a6ffd640 1 --2- 172.21.15.26:0/1243930646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a815a9c0 0x7f71a8159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:17.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71a6ffd640 1 -- 172.21.15.26:0/1243930646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7190009420 con 0x7f71a806ac80 2023-12-19T18:14:17.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71a6ffd640 1 --2- 172.21.15.26:0/1243930646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a806ac80 0x7f71a8150d70 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f719800c500 tx=0x7f719800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:17.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f719f7fe640 1 -- 172.21.15.26:0/1243930646 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198005cf0 con 0x7f71a806ac80 2023-12-19T18:14:17.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f719f7fe640 1 -- 172.21.15.26:0/1243930646 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7198005e50 con 0x7f71a806ac80 2023-12-19T18:14:17.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71ad50a640 1 -- 172.21.15.26:0/1243930646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a80c6a30 con 0x7f71a806ac80 2023-12-19T18:14:17.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f719f7fe640 1 -- 172.21.15.26:0/1243930646 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71980072f0 con 0x7f71a806ac80 2023-12-19T18:14:17.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71ad50a640 1 -- 172.21.15.26:0/1243930646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a81512b0 con 0x7f71a806ac80 2023-12-19T18:14:17.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f71ad50a640 1 -- 172.21.15.26:0/1243930646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a80c6ee0 con 0x7f71a806ac80 2023-12-19T18:14:17.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.944+0000 7f719f7fe640 1 -- 172.21.15.26:0/1243930646 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 102) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f7198007450 con 0x7f71a806ac80 2023-12-19T18:14:17.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.948+0000 7f719f7fe640 1 --2- 172.21.15.26:0/1243930646 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7188065bd0 0x7f7188068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:17.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.948+0000 7f719f7fe640 1 -- 172.21.15.26:0/1243930646 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 7072+0+0 (secure 0 0 0) 0x7f7198082d10 con 0x7f71a806ac80 2023-12-19T18:14:17.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.948+0000 7f719ffff640 1 --2- 172.21.15.26:0/1243930646 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7188065bd0 0x7f7188068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:17.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.948+0000 7f719ffff640 1 --2- 172.21.15.26:0/1243930646 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7188065bd0 0x7f7188068090 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f71900ad000 tx=0x7f7190004740 comp rx=0 tx=0).ready entity=mgr.8204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:17.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:17.952+0000 7f719f7fe640 1 -- 172.21.15.26:0/1243930646 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f71a80c6ee0 con 0x7f71a806ac80 2023-12-19T18:14:18.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.128+0000 7f71ad50a640 1 -- 172.21.15.26:0/1243930646 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f71a8167ac0 con 0x7f7188065bd0 2023-12-19T18:14:18.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.204+0000 7f719f7fe640 1 -- 172.21.15.26:0/1243930646 <== mgr.8204 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f71900ab580 con 0x7f7188065bd0 2023-12-19T18:14:18.205 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-12-19T18:14:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.208+0000 7f71ad50a640 1 -- 172.21.15.26:0/1243930646 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7188065bd0 msgr2=0x7f7188068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.208+0000 7f71ad50a640 1 --2- 172.21.15.26:0/1243930646 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7188065bd0 0x7f7188068090 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f71900ad000 tx=0x7f7190004740 comp rx=0 tx=0).stop 2023-12-19T18:14:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.208+0000 7f71ad50a640 1 -- 172.21.15.26:0/1243930646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a806ac80 msgr2=0x7f71a8150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.208+0000 7f71ad50a640 1 --2- 172.21.15.26:0/1243930646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a806ac80 0x7f71a8150d70 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f719800c500 tx=0x7f719800c9c0 comp rx=0 tx=0).stop 2023-12-19T18:14:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.208+0000 7f71ad50a640 1 -- 172.21.15.26:0/1243930646 shutdown_connections 2023-12-19T18:14:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.208+0000 7f71ad50a640 1 --2- 172.21.15.26:0/1243930646 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7188065bd0 0x7f7188068090 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.208+0000 7f71ad50a640 1 --2- 172.21.15.26:0/1243930646 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71a815ada0 0x7f71a8159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.208+0000 7f71ad50a640 1 --2- 172.21.15.26:0/1243930646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a815a9c0 0x7f71a8159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.208+0000 7f71ad50a640 1 --2- 172.21.15.26:0/1243930646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71a806ac80 0x7f71a8150d70 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.208+0000 7f71ad50a640 1 -- 172.21.15.26:0/1243930646 >> 172.21.15.26:0/1243930646 conn(0x7f71a80ce730 msgr2=0x7f71a80c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.208+0000 7f71ad50a640 1 -- 172.21.15.26:0/1243930646 shutdown_connections 2023-12-19T18:14:18.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:18.208+0000 7f71ad50a640 1 -- 172.21.15.26:0/1243930646 wait complete. 2023-12-19T18:14:18.226 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-12-19T18:14:18.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.227 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-12-19T18:14:18.237 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-17_59_00_UTC 2023-12-19T18:14:18.237 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_00_00_UTC 2023-12-19T18:14:18.237 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_01_00_UTC 2023-12-19T18:14:18.237 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_02_00_UTC 2023-12-19T18:14:18.237 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_03_00_UTC 2023-12-19T18:14:18.237 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_04_00_UTC 2023-12-19T18:14:18.237 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_05_00_UTC 2023-12-19T18:14:18.238 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_06_00_UTC 2023-12-19T18:14:18.238 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_07_00_UTC 2023-12-19T18:14:18.238 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_08_00_UTC 2023-12-19T18:14:18.238 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_09_00_UTC 2023-12-19T18:14:18.238 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_10_00_UTC 2023-12-19T18:14:18.238 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_11_00_UTC 2023-12-19T18:14:18.238 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_12_00_UTC 2023-12-19T18:14:18.238 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_13_00_UTC 2023-12-19T18:14:18.238 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_14_00_UTC 2023-12-19T18:14:18.239 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-17_59_00_UTC 2023-12-19T18:14:18.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.240 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-17_59_00_UTC) 2023-12-19T18:14:18.299 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_00_00_UTC 2023-12-19T18:14:18.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.300 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_00_00_UTC) 2023-12-19T18:14:18.356 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_01_00_UTC 2023-12-19T18:14:18.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.357 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_01_00_UTC) 2023-12-19T18:14:18.412 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_02_00_UTC 2023-12-19T18:14:18.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.412 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_02_00_UTC) 2023-12-19T18:14:18.468 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_03_00_UTC 2023-12-19T18:14:18.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.468 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_03_00_UTC) 2023-12-19T18:14:18.524 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_04_00_UTC 2023-12-19T18:14:18.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.525 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_04_00_UTC) 2023-12-19T18:14:18.581 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_05_00_UTC 2023-12-19T18:14:18.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.581 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_05_00_UTC) 2023-12-19T18:14:18.637 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_06_00_UTC 2023-12-19T18:14:18.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.637 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_06_00_UTC) 2023-12-19T18:14:18.692 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_07_00_UTC 2023-12-19T18:14:18.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.693 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_07_00_UTC) 2023-12-19T18:14:18.748 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_08_00_UTC 2023-12-19T18:14:18.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.749 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_08_00_UTC) 2023-12-19T18:14:18.804 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_09_00_UTC 2023-12-19T18:14:18.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.805 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_09_00_UTC) 2023-12-19T18:14:18.861 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_10_00_UTC 2023-12-19T18:14:18.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.861 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_10_00_UTC) 2023-12-19T18:14:18.920 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_11_00_UTC 2023-12-19T18:14:18.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.921 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_11_00_UTC) 2023-12-19T18:14:18.980 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_12_00_UTC 2023-12-19T18:14:18.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:18.981 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_12_00_UTC) 2023-12-19T18:14:19.041 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_13_00_UTC 2023-12-19T18:14:19.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:19.041 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_13_00_UTC) 2023-12-19T18:14:19.096 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_14_00_UTC 2023-12-19T18:14:19.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:19.097 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-19-18_14_00_UTC) 2023-12-19T18:14:19.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:19.153 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-12-19T18:14:19.212 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-19T18:14:19.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa101404640 1 Processor -- start 2023-12-19T18:14:19.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa101404640 1 -- start start 2023-12-19T18:14:19.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa101404640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa0fc15cdf0 0x7fa0fc06ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa101404640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa0fc06f270 0x7fa0fc0cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa101404640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0fc0cfdf0 0x7fa0fc071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa101404640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0fc15d1d0 con 0x7fa0fc06f270 2023-12-19T18:14:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa101404640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0fc06f6d0 con 0x7fa0fc15cdf0 2023-12-19T18:14:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa101404640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0d0280 con 0x7fa0fc0cfdf0 2023-12-19T18:14:19.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa0fa7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa0fc06f270 0x7fa0fc0cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:19.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa0fa7fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa0fc06f270 0x7fa0fc0cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45286/0 (socket says 172.21.15.26:45286) 2023-12-19T18:14:19.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa0fa7fc640 1 -- 172.21.15.26:0/1133279080 learned_addr learned my addr 172.21.15.26:0/1133279080 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:19.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa0faffd640 1 --2- 172.21.15.26:0/1133279080 >> v2:172.21.15.160:3300/0 conn(0x7fa0fc15cdf0 0x7fa0fc06ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:19.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa0fb7fe640 1 --2- 172.21.15.26:0/1133279080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0fc0cfdf0 0x7fa0fc071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:19.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa0fa7fc640 1 -- 172.21.15.26:0/1133279080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0fc0cfdf0 msgr2=0x7fa0fc071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:19.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa0fa7fc640 1 --2- 172.21.15.26:0/1133279080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0fc0cfdf0 0x7fa0fc071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:19.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa0fa7fc640 1 -- 172.21.15.26:0/1133279080 >> v2:172.21.15.160:3300/0 conn(0x7fa0fc15cdf0 msgr2=0x7fa0fc06ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:19.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa0fa7fc640 1 --2- 172.21.15.26:0/1133279080 >> v2:172.21.15.160:3300/0 conn(0x7fa0fc15cdf0 0x7fa0fc06ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:19.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.400+0000 7fa0fa7fc640 1 -- 172.21.15.26:0/1133279080 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc161140 con 0x7fa0fc06f270 2023-12-19T18:14:19.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.404+0000 7fa0fa7fc640 1 --2- 172.21.15.26:0/1133279080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0fc06f270 0x7fa0fc0cf8b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0d17e0 tx=0x7fa0f001c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da592d22c00438a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:19.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.404+0000 7fa0f9ffb640 1 -- 172.21.15.26:0/1133279080 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00ab600 con 0x7fa0fc06f270 2023-12-19T18:14:19.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.404+0000 7fa0f9ffb640 1 -- 172.21.15.26:0/1133279080 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f00a7050 con 0x7fa0fc06f270 2023-12-19T18:14:19.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.404+0000 7fa0f9ffb640 1 -- 172.21.15.26:0/1133279080 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00ab980 con 0x7fa0fc06f270 2023-12-19T18:14:19.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.404+0000 7fa101404640 1 -- 172.21.15.26:0/1133279080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0fc06f270 msgr2=0x7fa0fc0cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:19.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.404+0000 7fa101404640 1 --2- 172.21.15.26:0/1133279080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0fc06f270 0x7fa0fc0cf8b0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0d17e0 tx=0x7fa0f001c210 comp rx=0 tx=0).stop 2023-12-19T18:14:19.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.404+0000 7fa101404640 1 -- 172.21.15.26:0/1133279080 shutdown_connections 2023-12-19T18:14:19.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.404+0000 7fa101404640 1 --2- 172.21.15.26:0/1133279080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0fc0cfdf0 0x7fa0fc071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:19.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.404+0000 7fa101404640 1 --2- 172.21.15.26:0/1133279080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0fc06f270 0x7fa0fc0cf8b0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:19.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.404+0000 7fa101404640 1 --2- 172.21.15.26:0/1133279080 >> v2:172.21.15.160:3300/0 conn(0x7fa0fc15cdf0 0x7fa0fc06ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:19.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.404+0000 7fa101404640 1 -- 172.21.15.26:0/1133279080 >> 172.21.15.26:0/1133279080 conn(0x7fa0fc0d0830 msgr2=0x7fa0fc0d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:19.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.404+0000 7fa101404640 1 -- 172.21.15.26:0/1133279080 shutdown_connections 2023-12-19T18:14:19.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.404+0000 7fa101404640 1 -- 172.21.15.26:0/1133279080 wait complete. 2023-12-19T18:14:19.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa101404640 1 Processor -- start 2023-12-19T18:14:19.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa101404640 1 -- start start 2023-12-19T18:14:19.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa101404640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0fc06f270 0x7fa0fc152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa101404640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0fc0cfdf0 0x7fa0fc0c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa101404640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0fc15cdf0 0x7fa0fc1547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa101404640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc1527d0 con 0x7fa0fc0cfdf0 2023-12-19T18:14:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa101404640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0c4fe0 con 0x7fa0fc15cdf0 2023-12-19T18:14:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa101404640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc0c52c0 con 0x7fa0fc06f270 2023-12-19T18:14:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0fa7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0fc0cfdf0 0x7fa0fc0c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0fa7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0fc0cfdf0 0x7fa0fc0c6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45296/0 (socket says 172.21.15.26:45296) 2023-12-19T18:14:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0fa7fc640 1 -- 172.21.15.26:0/1721887450 learned_addr learned my addr 172.21.15.26:0/1721887450 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0fb7fe640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0fc15cdf0 0x7fa0fc1547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:19.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0faffd640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0fc06f270 0x7fa0fc152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0fa7fc640 1 -- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0fc06f270 msgr2=0x7fa0fc152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0fa7fc640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0fc06f270 0x7fa0fc152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0fa7fc640 1 -- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0fc15cdf0 msgr2=0x7fa0fc1547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0fa7fc640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0fc15cdf0 0x7fa0fc1547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0fa7fc640 1 -- 172.21.15.26:0/1721887450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00094d0 con 0x7fa0fc0cfdf0 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0fa7fc640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0fc0cfdf0 0x7fa0fc0c6080 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fa0f001a8c0 tx=0x7fa0f0018fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0e37fe640 1 -- 172.21.15.26:0/1721887450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00b37d0 con 0x7fa0fc0cfdf0 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0e37fe640 1 -- 172.21.15.26:0/1721887450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f00b3930 con 0x7fa0fc0cfdf0 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa0e37fe640 1 -- 172.21.15.26:0/1721887450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00b3ca0 con 0x7fa0fc0cfdf0 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa101404640 1 -- 172.21.15.26:0/1721887450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc0c55a0 con 0x7fa0fc0cfdf0 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa101404640 1 -- 172.21.15.26:0/1721887450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc0c5b60 con 0x7fa0fc0cfdf0 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.408+0000 7fa101404640 1 -- 172.21.15.26:0/1721887450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc04e850 con 0x7fa0fc0cfdf0 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.412+0000 7fa0e37fe640 1 -- 172.21.15.26:0/1721887450 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 102) v1 ==== 85359+0+0 (secure 0 0 0) 0x7fa0f00b2070 con 0x7fa0fc0cfdf0 2023-12-19T18:14:19.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.412+0000 7fa0e37fe640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa0e8065bd0 0x7fa0e8068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.412+0000 7fa0e37fe640 1 -- 172.21.15.26:0/1721887450 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 7072+0+0 (secure 0 0 0) 0x7fa0f011dbe0 con 0x7fa0fc0cfdf0 2023-12-19T18:14:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.412+0000 7fa0faffd640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa0e8065bd0 0x7fa0e8068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:19.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.412+0000 7fa0faffd640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa0e8065bd0 0x7fa0e8068090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec003fd0 tx=0x7fa0ec0039c0 comp rx=0 tx=0).ready entity=mgr.8204 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:19.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.416+0000 7fa0e37fe640 1 -- 172.21.15.26:0/1721887450 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa0f00eadb0 con 0x7fa0fc0cfdf0 2023-12-19T18:14:19.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:19.612+0000 7fa101404640 1 -- 172.21.15.26:0/1721887450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fa0fc0b2d30 con 0x7fa0fc0cfdf0 2023-12-19T18:14:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.048+0000 7fa0e37fe640 1 -- 172.21.15.26:0/1721887450 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 103) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fa0f00eb180 con 0x7fa0fc0cfdf0 2023-12-19T18:14:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.052+0000 7fa0e37fe640 1 -- 172.21.15.26:0/1721887450 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v103) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa0f0115a60 con 0x7fa0fc0cfdf0 2023-12-19T18:14:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.056+0000 7fa101404640 1 -- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa0e8065bd0 msgr2=0x7fa0e8068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.056+0000 7fa101404640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa0e8065bd0 0x7fa0e8068090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec003fd0 tx=0x7fa0ec0039c0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.056+0000 7fa101404640 1 -- 172.21.15.26:0/1721887450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0fc0cfdf0 msgr2=0x7fa0fc0c6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.056+0000 7fa101404640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0fc0cfdf0 0x7fa0fc0c6080 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fa0f001a8c0 tx=0x7fa0f0018fb0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.056+0000 7fa101404640 1 -- 172.21.15.26:0/1721887450 shutdown_connections 2023-12-19T18:14:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.056+0000 7fa101404640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa0e8065bd0 0x7fa0e8068090 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.056+0000 7fa101404640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0fc15cdf0 0x7fa0fc1547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.056+0000 7fa101404640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0fc0cfdf0 0x7fa0fc0c6080 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.056+0000 7fa101404640 1 --2- 172.21.15.26:0/1721887450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa0fc06f270 0x7fa0fc152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.056+0000 7fa101404640 1 -- 172.21.15.26:0/1721887450 >> 172.21.15.26:0/1721887450 conn(0x7fa0fc0d0830 msgr2=0x7fa0fc0b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.056+0000 7fa101404640 1 -- 172.21.15.26:0/1721887450 shutdown_connections 2023-12-19T18:14:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.056+0000 7fa101404640 1 -- 172.21.15.26:0/1721887450 wait complete. 2023-12-19T18:14:20.087 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:14:20.087 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:14:20.174 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:14:20.255 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:20.256+0000 7fd96f9d4180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:14:20.274 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:14:20.274 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:14:20.274 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:14:20.284 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:14:20.284 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:14:20.298 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:20.295+0000 7fdf4b9db180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:14:20.336 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.588+0000 7ff568939640 1 Processor -- start 2023-12-19T18:14:20.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.588+0000 7ff568939640 1 -- start start 2023-12-19T18:14:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.588+0000 7ff568939640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff5640b4d10 0x7ff5641679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.588+0000 7ff568939640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5640d2960 con 0x7ff56415cf60 2023-12-19T18:14:20.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.588+0000 7ff568939640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5640d15f0 con 0x7ff5640b88e0 2023-12-19T18:14:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.588+0000 7ff568939640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5640d18c0 con 0x7ff5640b4d10 2023-12-19T18:14:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.592+0000 7ff56259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff56415cf60 0x7ff5640d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46766/0 (socket says 172.21.15.26:46766) 2023-12-19T18:14:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.592+0000 7ff56259c640 1 -- 172.21.15.26:0/2892109374 learned_addr learned my addr 172.21.15.26:0/2892109374 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.592+0000 7ff562d9d640 1 --2- 172.21.15.26:0/2892109374 >> v2:172.21.15.160:3300/0 conn(0x7ff5640b4d10 0x7ff5641679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.592+0000 7ff56159a640 1 -- 172.21.15.26:0/2892109374 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1322410002 0 0) 0x7ff5640d2960 con 0x7ff56415cf60 2023-12-19T18:14:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.592+0000 7ff56159a640 1 -- 172.21.15.26:0/2892109374 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff54c003530 con 0x7ff56415cf60 2023-12-19T18:14:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.592+0000 7ff56159a640 1 -- 172.21.15.26:0/2892109374 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864131603 0 0) 0x7ff5640d15f0 con 0x7ff5640b88e0 2023-12-19T18:14:20.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.592+0000 7ff56159a640 1 -- 172.21.15.26:0/2892109374 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5640d2960 con 0x7ff5640b88e0 2023-12-19T18:14:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.592+0000 7ff562d9d640 1 -- 172.21.15.26:0/2892109374 >> v1:172.21.15.160:6789/0 conn(0x7ff5640b88e0 legacy=0x7ff5640b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.592+0000 7ff562d9d640 1 -- 172.21.15.26:0/2892109374 >> v1:172.21.15.26:6789/0 conn(0x7ff56415cf60 legacy=0x7ff5640d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:20.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.592+0000 7ff562d9d640 1 -- 172.21.15.26:0/2892109374 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5640d1ba0 con 0x7ff5640b4d10 2023-12-19T18:14:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.592+0000 7ff562d9d640 1 --2- 172.21.15.26:0/2892109374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5640b4d10 0x7ff5641679a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009d90 tx=0x7ff55c01f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=138c38fbe7b1c28c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff56159a640 1 -- 172.21.15.26:0/2892109374 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0af600 con 0x7ff5640b4d10 2023-12-19T18:14:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff56159a640 1 -- 172.21.15.26:0/2892109374 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55c0ab050 con 0x7ff5640b4d10 2023-12-19T18:14:20.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff56159a640 1 -- 172.21.15.26:0/2892109374 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0af980 con 0x7ff5640b4d10 2023-12-19T18:14:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff568939640 1 -- 172.21.15.26:0/2892109374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5640b4d10 msgr2=0x7ff5641679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff568939640 1 --2- 172.21.15.26:0/2892109374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5640b4d10 0x7ff5641679a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009d90 tx=0x7ff55c01f130 comp rx=0 tx=0).stop 2023-12-19T18:14:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff568939640 1 -- 172.21.15.26:0/2892109374 shutdown_connections 2023-12-19T18:14:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff568939640 1 --2- 172.21.15.26:0/2892109374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff5640b4d10 0x7ff5641679a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff568939640 1 -- 172.21.15.26:0/2892109374 >> 172.21.15.26:0/2892109374 conn(0x7ff564065ee0 msgr2=0x7ff5640cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff568939640 1 -- 172.21.15.26:0/2892109374 shutdown_connections 2023-12-19T18:14:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff568939640 1 -- 172.21.15.26:0/2892109374 wait complete. 2023-12-19T18:14:20.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff568939640 1 Processor -- start 2023-12-19T18:14:20.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff568939640 1 -- start start 2023-12-19T18:14:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff568939640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5640b4d10 0x7ff5641f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.596+0000 7ff568939640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5640b88e0 0x7ff5641f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff568939640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff56415cf60 0x7ff5640ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff568939640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641f3460 con 0x7ff5640b88e0 2023-12-19T18:14:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff568939640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641f3c80 con 0x7ff56415cf60 2023-12-19T18:14:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff568939640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641f3f60 con 0x7ff5640b4d10 2023-12-19T18:14:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff56259c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5640b4d10 0x7ff5641f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff56259c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5640b4d10 0x7ff5641f36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:36228/0 (socket says 172.21.15.26:36228) 2023-12-19T18:14:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff56259c640 1 -- 172.21.15.26:0/3980007067 learned_addr learned my addr 172.21.15.26:0/3980007067 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff562d9d640 1 --2- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff56415cf60 0x7ff5640ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:20.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff561d9b640 1 --2- 172.21.15.26:0/3980007067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5640b88e0 0x7ff5641f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff56259c640 1 -- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff56415cf60 msgr2=0x7ff5640ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff56259c640 1 --2- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff56415cf60 0x7ff5640ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff56259c640 1 -- 172.21.15.26:0/3980007067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5640b88e0 msgr2=0x7ff5641f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff56259c640 1 --2- 172.21.15.26:0/3980007067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5640b88e0 0x7ff5641f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff56259c640 1 -- 172.21.15.26:0/3980007067 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c009a70 con 0x7ff5640b4d10 2023-12-19T18:14:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff56259c640 1 --2- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5640b4d10 0x7ff5641f36a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff55400c420 tx=0x7ff55400c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff552ffd640 1 -- 172.21.15.26:0/3980007067 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554005cf0 con 0x7ff5640b4d10 2023-12-19T18:14:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff552ffd640 1 -- 172.21.15.26:0/3980007067 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff554005e50 con 0x7ff5640b4d10 2023-12-19T18:14:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff552ffd640 1 -- 172.21.15.26:0/3980007067 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5540072d0 con 0x7ff5640b4d10 2023-12-19T18:14:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff568939640 1 -- 172.21.15.26:0/3980007067 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5641f4240 con 0x7ff5640b4d10 2023-12-19T18:14:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff568939640 1 -- 172.21.15.26:0/3980007067 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5641f4800 con 0x7ff5640b4d10 2023-12-19T18:14:20.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff568939640 1 -- 172.21.15.26:0/3980007067 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff564074270 con 0x7ff5640b4d10 2023-12-19T18:14:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.600+0000 7ff552ffd640 1 -- 172.21.15.26:0/3980007067 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 103) v1 ==== 85342+0+0 (secure 0 0 0) 0x7ff554007430 con 0x7ff5640b4d10 2023-12-19T18:14:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.604+0000 7ff552ffd640 1 --2- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff544065b80 0x7ff544068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.604+0000 7ff552ffd640 1 -- 172.21.15.26:0/3980007067 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 7160+0+0 (secure 0 0 0) 0x7ff554082db0 con 0x7ff5640b4d10 2023-12-19T18:14:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.604+0000 7ff561d9b640 1 -- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff544065b80 msgr2=0x7ff544068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:20.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.604+0000 7ff561d9b640 1 --2- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff544065b80 0x7ff544068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:20.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.608+0000 7ff552ffd640 1 -- 172.21.15.26:0/3980007067 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff554087070 con 0x7ff5640b4d10 2023-12-19T18:14:20.769 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:20.767+0000 7fdf4b9db180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:14:20.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.780+0000 7ff568939640 1 -- 172.21.15.26:0/3980007067 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5641f40a0 con 0x7ff5640b4d10 2023-12-19T18:14:20.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.780+0000 7ff552ffd640 1 -- 172.21.15.26:0/3980007067 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+18307 (secure 0 0 0) 0x7ff5641f40a0 con 0x7ff5640b4d10 2023-12-19T18:14:20.781 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:20.781 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":182,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:14:20.039999+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:58:12.042476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T17:58:12.457937+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":32,"snap_epoch":182,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.29","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3381018054":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/3458963623":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/1302880583":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/1063410185":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/819581921":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/4076846554":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/3832178910":"2023-12-20T17:58:25.501946+0000","172.21.15.160:6833/38098":"2023-12-20T17:58:25.501946+0000","172.21.15.160:6832/38098":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/1419214494":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/1333176784":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/4141924506":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/2994631753":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/2198643724":"2023-12-20T17:58:25.501946+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":21,"snaps":[{"begin":2,"length":15},{"begin":18,"length":15}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":2,"length":15},{"begin":18,"length":15}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.780+0000 7ff568939640 1 -- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff544065b80 msgr2=0x7ff544068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:14:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.780+0000 7ff568939640 1 --2- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff544065b80 0x7ff544068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.780+0000 7ff568939640 1 -- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5640b4d10 msgr2=0x7ff5641f36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.780+0000 7ff568939640 1 --2- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5640b4d10 0x7ff5641f36a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff55400c420 tx=0x7ff55400c8e0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.780+0000 7ff568939640 1 -- 172.21.15.26:0/3980007067 shutdown_connections 2023-12-19T18:14:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.780+0000 7ff568939640 1 --2- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff544065b80 0x7ff544068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.780+0000 7ff568939640 1 --2- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff56415cf60 0x7ff5640ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.780+0000 7ff568939640 1 --2- 172.21.15.26:0/3980007067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5640b88e0 0x7ff5641f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.780+0000 7ff568939640 1 --2- 172.21.15.26:0/3980007067 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff5640b4d10 0x7ff5641f36a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.780+0000 7ff568939640 1 -- 172.21.15.26:0/3980007067 >> 172.21.15.26:0/3980007067 conn(0x7ff564065ee0 msgr2=0x7ff5640b7670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.780+0000 7ff568939640 1 -- 172.21.15.26:0/3980007067 shutdown_connections 2023-12-19T18:14:20.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.784+0000 7ff568939640 1 -- 172.21.15.26:0/3980007067 wait complete. 2023-12-19T18:14:20.791 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:20.792+0000 7fd96f9d4180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:14:20.806 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T17:58:12.042476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '176', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T17:58:12.457937+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '182', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 182, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:14:20.807 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:14:20.917 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:20.915+0000 7fdf4b9db180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:14:20.951 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:20.952+0000 7fd96f9d4180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:14:20.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.956+0000 7ff75c88c640 1 Processor -- start 2023-12-19T18:14:20.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.956+0000 7ff75c88c640 1 -- start start 2023-12-19T18:14:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.956+0000 7ff75c88c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff75806ac80 0x7ff75806b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.956+0000 7ff75c88c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff75806f920 0x7ff75806fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.956+0000 7ff75c88c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75806f490 con 0x7ff758159140 2023-12-19T18:14:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.956+0000 7ff75c88c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75806b650 con 0x7ff75806ac80 2023-12-19T18:14:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.956+0000 7ff75c88c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff758159520 con 0x7ff75806f920 2023-12-19T18:14:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.956+0000 7ff755d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff75806ac80 0x7ff75806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.956+0000 7ff755d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff75806ac80 0x7ff75806b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45322/0 (socket says 172.21.15.26:45322) 2023-12-19T18:14:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.956+0000 7ff755d9b640 1 -- 172.21.15.26:0/2817995288 learned_addr learned my addr 172.21.15.26:0/2817995288 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.956+0000 7ff756d9d640 1 --2- 172.21.15.26:0/2817995288 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff75806f920 0x7ff75806fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.960+0000 7ff755d9b640 1 -- 172.21.15.26:0/2817995288 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff75806f920 msgr2=0x7ff75806fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:20.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.960+0000 7ff755d9b640 1 --2- 172.21.15.26:0/2817995288 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff75806f920 0x7ff75806fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.960+0000 7ff755d9b640 1 -- 172.21.15.26:0/2817995288 >> v1:172.21.15.26:6789/0 conn(0x7ff758159140 legacy=0x7ff7580ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.960+0000 7ff755d9b640 1 -- 172.21.15.26:0/2817995288 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7580cd690 con 0x7ff75806ac80 2023-12-19T18:14:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.960+0000 7ff75559a640 1 -- 172.21.15.26:0/2817995288 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67610609 0 0) 0x7ff75806f490 con 0x7ff758159140 2023-12-19T18:14:20.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.960+0000 7ff755d9b640 1 --2- 172.21.15.26:0/2817995288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75806ac80 0x7ff75806b110 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009870 tx=0x7ff74c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=497105c590a10d85 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:20.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.960+0000 7ff75559a640 1 -- 172.21.15.26:0/2817995288 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c0aa5f0 con 0x7ff75806ac80 2023-12-19T18:14:20.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.960+0000 7ff75559a640 1 -- 172.21.15.26:0/2817995288 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c0aa750 con 0x7ff75806ac80 2023-12-19T18:14:20.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.960+0000 7ff75559a640 1 -- 172.21.15.26:0/2817995288 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c0aaae0 con 0x7ff75806ac80 2023-12-19T18:14:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 -- 172.21.15.26:0/2817995288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75806ac80 msgr2=0x7ff75806b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 --2- 172.21.15.26:0/2817995288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75806ac80 0x7ff75806b110 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009870 tx=0x7ff74c01c510 comp rx=0 tx=0).stop 2023-12-19T18:14:20.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 -- 172.21.15.26:0/2817995288 shutdown_connections 2023-12-19T18:14:20.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 --2- 172.21.15.26:0/2817995288 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff75806f920 0x7ff75806fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 --2- 172.21.15.26:0/2817995288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff75806ac80 0x7ff75806b110 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 -- 172.21.15.26:0/2817995288 >> 172.21.15.26:0/2817995288 conn(0x7ff7580ce730 msgr2=0x7ff7580ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:20.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 -- 172.21.15.26:0/2817995288 shutdown_connections 2023-12-19T18:14:20.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 -- 172.21.15.26:0/2817995288 wait complete. 2023-12-19T18:14:20.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 Processor -- start 2023-12-19T18:14:20.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 -- start start 2023-12-19T18:14:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff75806ac80 0x7ff758148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff75806f920 0x7ff7580c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff758159140 0x7ff7580ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580b2270 con 0x7ff758159140 2023-12-19T18:14:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580b1660 con 0x7ff75806f920 2023-12-19T18:14:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75c88c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7580b24c0 con 0x7ff75806ac80 2023-12-19T18:14:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff756d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff758159140 0x7ff7580ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff756d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff758159140 0x7ff7580ae5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45334/0 (socket says 172.21.15.26:45334) 2023-12-19T18:14:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff756d9d640 1 -- 172.21.15.26:0/572324763 learned_addr learned my addr 172.21.15.26:0/572324763 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff75659c640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff75806ac80 0x7ff758148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.964+0000 7ff755d9b640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff75806f920 0x7ff7580c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff756d9d640 1 -- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff75806ac80 msgr2=0x7ff758148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff756d9d640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff75806ac80 0x7ff758148350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff756d9d640 1 -- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff75806f920 msgr2=0x7ff7580c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff756d9d640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff75806f920 0x7ff7580c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff756d9d640 1 -- 172.21.15.26:0/572324763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c0094d0 con 0x7ff758159140 2023-12-19T18:14:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff75659c640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff75806ac80 0x7ff758148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff756d9d640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff758159140 0x7ff7580ae5f0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7ff750009dd0 tx=0x7ff75000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff746ffd640 1 -- 172.21.15.26:0/572324763 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750014480 con 0x7ff758159140 2023-12-19T18:14:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff746ffd640 1 -- 172.21.15.26:0/572324763 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75000f040 con 0x7ff758159140 2023-12-19T18:14:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff746ffd640 1 -- 172.21.15.26:0/572324763 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750014840 con 0x7ff758159140 2023-12-19T18:14:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff75c88c640 1 -- 172.21.15.26:0/572324763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7580b2c40 con 0x7ff758159140 2023-12-19T18:14:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff75c88c640 1 -- 172.21.15.26:0/572324763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7580b3200 con 0x7ff758159140 2023-12-19T18:14:20.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff75c88c640 1 -- 172.21.15.26:0/572324763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7580b33f0 con 0x7ff758159140 2023-12-19T18:14:20.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff746ffd640 1 -- 172.21.15.26:0/572324763 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 103) v1 ==== 85342+0+0 (secure 0 0 0) 0x7ff7500149a0 con 0x7ff758159140 2023-12-19T18:14:20.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff746ffd640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff738065b80 0x7ff738068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:20.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff746ffd640 1 -- 172.21.15.26:0/572324763 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 7160+0+0 (secure 0 0 0) 0x7ff750084460 con 0x7ff758159140 2023-12-19T18:14:20.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff75659c640 1 -- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff738065b80 msgr2=0x7ff738068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:20.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.968+0000 7ff75659c640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff738065b80 0x7ff738068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:20.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:20.976+0000 7ff746ffd640 1 -- 172.21.15.26:0/572324763 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff750051650 con 0x7ff758159140 2023-12-19T18:14:21.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff75c88c640 1 -- 172.21.15.26:0/572324763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff758074270 con 0x7ff758159140 2023-12-19T18:14:21.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff746ffd640 1 -- 172.21.15.26:0/572324763 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff750054c70 con 0x7ff758159140 2023-12-19T18:14:21.140 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:14:21.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff75c88c640 1 -- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff738065b80 msgr2=0x7ff738068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:14:21.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff75c88c640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff738065b80 0x7ff738068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff75c88c640 1 -- 172.21.15.26:0/572324763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff758159140 msgr2=0x7ff7580ae5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:21.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff75c88c640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff758159140 0x7ff7580ae5f0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7ff750009dd0 tx=0x7ff75000c490 comp rx=0 tx=0).stop 2023-12-19T18:14:21.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff75c88c640 1 -- 172.21.15.26:0/572324763 shutdown_connections 2023-12-19T18:14:21.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff75c88c640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff738065b80 0x7ff738068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff75c88c640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff758159140 0x7ff7580ae5f0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff75c88c640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff75806f920 0x7ff7580c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff75c88c640 1 --2- 172.21.15.26:0/572324763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff75806ac80 0x7ff758148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff75c88c640 1 -- 172.21.15.26:0/572324763 >> 172.21.15.26:0/572324763 conn(0x7ff7580ce730 msgr2=0x7ff7580ca930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:21.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff75c88c640 1 -- 172.21.15.26:0/572324763 shutdown_connections 2023-12-19T18:14:21.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.140+0000 7ff75c88c640 1 -- 172.21.15.26:0/572324763 wait complete. 2023-12-19T18:14:21.163 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:14:21.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.316+0000 7ffbe0641640 1 Processor -- start 2023-12-19T18:14:21.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.316+0000 7ffbe0641640 1 -- start start 2023-12-19T18:14:21.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.316+0000 7ffbe0641640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbd815cf80 0x7ffbd80cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:21.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.316+0000 7ffbe0641640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffbd80ccbd0 0x7ffbd80cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:21.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.316+0000 7ffbe0641640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffbd80cd910 0x7ffbd806a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:21.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.316+0000 7ffbe0641640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd80cdda0 con 0x7ffbd80cd910 2023-12-19T18:14:21.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.316+0000 7ffbe0641640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd80ce750 con 0x7ffbd80ccbd0 2023-12-19T18:14:21.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.316+0000 7ffbe0641640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd80afaa0 con 0x7ffbd815cf80 2023-12-19T18:14:21.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.316+0000 7ffbdebde640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffbd80cd910 0x7ffbd806a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:21.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.316+0000 7ffbde3dd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbd815cf80 0x7ffbd80cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:21.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.316+0000 7ffbdebde640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffbd80cd910 0x7ffbd806a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45338/0 (socket says 172.21.15.26:45338) 2023-12-19T18:14:21.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.316+0000 7ffbdebde640 1 -- 172.21.15.26:0/3604725536 learned_addr learned my addr 172.21.15.26:0/3604725536 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:21.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.316+0000 7ffbddbdc640 1 --2- 172.21.15.26:0/3604725536 >> v2:172.21.15.160:3300/0 conn(0x7ffbd80ccbd0 0x7ffbd80cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:21.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.320+0000 7ffbdebde640 1 -- 172.21.15.26:0/3604725536 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbd815cf80 msgr2=0x7ffbd80cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:21.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.320+0000 7ffbdebde640 1 --2- 172.21.15.26:0/3604725536 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbd815cf80 0x7ffbd80cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.320+0000 7ffbdebde640 1 -- 172.21.15.26:0/3604725536 >> v2:172.21.15.160:3300/0 conn(0x7ffbd80ccbd0 msgr2=0x7ffbd80cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:21.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.320+0000 7ffbdebde640 1 --2- 172.21.15.26:0/3604725536 >> v2:172.21.15.160:3300/0 conn(0x7ffbd80ccbd0 0x7ffbd80cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.320+0000 7ffbdebde640 1 -- 172.21.15.26:0/3604725536 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd80afd80 con 0x7ffbd80cd910 2023-12-19T18:14:21.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.320+0000 7ffbdebde640 1 --2- 172.21.15.26:0/3604725536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbd80cd910 0x7ffbd806a800 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80d1ac0 tx=0x7ffbd001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eee5941e89649f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:21.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.320+0000 7ffbdd3db640 1 -- 172.21.15.26:0/3604725536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd00a5820 con 0x7ffbd80cd910 2023-12-19T18:14:21.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.320+0000 7ffbdd3db640 1 -- 172.21.15.26:0/3604725536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd00a5980 con 0x7ffbd80cd910 2023-12-19T18:14:21.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.320+0000 7ffbdd3db640 1 -- 172.21.15.26:0/3604725536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd00a5d10 con 0x7ffbd80cd910 2023-12-19T18:14:21.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 -- 172.21.15.26:0/3604725536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbd80cd910 msgr2=0x7ffbd806a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:21.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 --2- 172.21.15.26:0/3604725536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbd80cd910 0x7ffbd806a800 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7ffbd80d1ac0 tx=0x7ffbd001c9a0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 -- 172.21.15.26:0/3604725536 shutdown_connections 2023-12-19T18:14:21.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 --2- 172.21.15.26:0/3604725536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbd80cd910 0x7ffbd806a800 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 --2- 172.21.15.26:0/3604725536 >> v2:172.21.15.160:3300/0 conn(0x7ffbd80ccbd0 0x7ffbd80cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 --2- 172.21.15.26:0/3604725536 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbd815cf80 0x7ffbd80cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 -- 172.21.15.26:0/3604725536 >> 172.21.15.26:0/3604725536 conn(0x7ffbd8071490 msgr2=0x7ffbd80718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:21.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 -- 172.21.15.26:0/3604725536 shutdown_connections 2023-12-19T18:14:21.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 -- 172.21.15.26:0/3604725536 wait complete. 2023-12-19T18:14:21.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 Processor -- start 2023-12-19T18:14:21.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 -- start start 2023-12-19T18:14:21.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbd80ccbd0 0x7ffbd80b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:21.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbd80cd910 0x7ffbd81550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:21.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbd815cf80 0x7ffbd80b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:21.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8159f10 con 0x7ffbd80cd910 2023-12-19T18:14:21.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd80b4a60 con 0x7ffbd815cf80 2023-12-19T18:14:21.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbe0641640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd80b4d40 con 0x7ffbd80ccbd0 2023-12-19T18:14:21.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbddbdc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbd80cd910 0x7ffbd81550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:21.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbde3dd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbd80ccbd0 0x7ffbd80b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:21.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbddbdc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbd80cd910 0x7ffbd81550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45350/0 (socket says 172.21.15.26:45350) 2023-12-19T18:14:21.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbdebde640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbd815cf80 0x7ffbd80b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:21.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbde3dd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbd80ccbd0 0x7ffbd80b4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:36268/0 (socket says 172.21.15.26:36268) 2023-12-19T18:14:21.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.324+0000 7ffbddbdc640 1 -- 172.21.15.26:0/1690344265 learned_addr learned my addr 172.21.15.26:0/1690344265 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:21.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbddbdc640 1 -- 172.21.15.26:0/1690344265 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbd80ccbd0 msgr2=0x7ffbd80b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:21.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbddbdc640 1 --2- 172.21.15.26:0/1690344265 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbd80ccbd0 0x7ffbd80b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbddbdc640 1 -- 172.21.15.26:0/1690344265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbd815cf80 msgr2=0x7ffbd80b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:21.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbddbdc640 1 --2- 172.21.15.26:0/1690344265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbd815cf80 0x7ffbd80b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbddbdc640 1 -- 172.21.15.26:0/1690344265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd0009b30 con 0x7ffbd80cd910 2023-12-19T18:14:21.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbddbdc640 1 --2- 172.21.15.26:0/1690344265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbd80cd910 0x7ffbd81550b0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800e7f0 tx=0x7ffbc800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:21.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbceffd640 1 -- 172.21.15.26:0/1690344265 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8009550 con 0x7ffbd80cd910 2023-12-19T18:14:21.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbceffd640 1 -- 172.21.15.26:0/1690344265 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc80059a0 con 0x7ffbd80cd910 2023-12-19T18:14:21.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbe0641640 1 -- 172.21.15.26:0/1690344265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd80b5020 con 0x7ffbd80cd910 2023-12-19T18:14:21.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbceffd640 1 -- 172.21.15.26:0/1690344265 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8005d30 con 0x7ffbd80cd910 2023-12-19T18:14:21.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbe0641640 1 -- 172.21.15.26:0/1690344265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd80b36c0 con 0x7ffbd80cd910 2023-12-19T18:14:21.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbe0641640 1 -- 172.21.15.26:0/1690344265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd80b54d0 con 0x7ffbd80cd910 2023-12-19T18:14:21.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbceffd640 1 -- 172.21.15.26:0/1690344265 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 103) v1 ==== 85342+0+0 (secure 0 0 0) 0x7ffbc800c940 con 0x7ffbd80cd910 2023-12-19T18:14:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.328+0000 7ffbceffd640 1 --2- 172.21.15.26:0/1690344265 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffbbc065b80 0x7ffbbc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.332+0000 7ffbceffd640 1 -- 172.21.15.26:0/1690344265 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 7144+0+0 (secure 0 0 0) 0x7ffbc8083b40 con 0x7ffbd80cd910 2023-12-19T18:14:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.332+0000 7ffbde3dd640 1 -- 172.21.15.26:0/1690344265 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffbbc065b80 msgr2=0x7ffbbc068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.332+0000 7ffbde3dd640 1 --2- 172.21.15.26:0/1690344265 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffbbc065b80 0x7ffbbc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:21.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.340+0000 7ffbceffd640 1 -- 172.21.15.26:0/1690344265 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffbc8050ef0 con 0x7ffbd80cd910 2023-12-19T18:14:21.414 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:21.411+0000 7fdf4b9db180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:14:21.473 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:21.471+0000 7fdf4b9db180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:14:21.489 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:21.488+0000 7fd96f9d4180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:14:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.496+0000 7ffbe0641640 1 -- 172.21.15.26:0/1690344265 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffbd8074270 con 0x7ffbd80cd910 2023-12-19T18:14:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.496+0000 7ffbceffd640 1 -- 172.21.15.26:0/1690344265 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v183) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffbc8054510 con 0x7ffbd80cd910 2023-12-19T18:14:21.497 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:14:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.500+0000 7ffbe0641640 1 -- 172.21.15.26:0/1690344265 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffbbc065b80 msgr2=0x7ffbbc068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:14:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.500+0000 7ffbe0641640 1 --2- 172.21.15.26:0/1690344265 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffbbc065b80 0x7ffbbc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.500+0000 7ffbe0641640 1 -- 172.21.15.26:0/1690344265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbd80cd910 msgr2=0x7ffbd81550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.500+0000 7ffbe0641640 1 --2- 172.21.15.26:0/1690344265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbd80cd910 0x7ffbd81550b0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800e7f0 tx=0x7ffbc800ecb0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.500+0000 7ffbe0641640 1 -- 172.21.15.26:0/1690344265 shutdown_connections 2023-12-19T18:14:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.500+0000 7ffbe0641640 1 --2- 172.21.15.26:0/1690344265 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffbbc065b80 0x7ffbbc068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.500+0000 7ffbe0641640 1 --2- 172.21.15.26:0/1690344265 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbd815cf80 0x7ffbd80b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.500+0000 7ffbe0641640 1 --2- 172.21.15.26:0/1690344265 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffbd80cd910 0x7ffbd81550b0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.500+0000 7ffbe0641640 1 --2- 172.21.15.26:0/1690344265 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffbd80ccbd0 0x7ffbd80b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.500+0000 7ffbe0641640 1 -- 172.21.15.26:0/1690344265 >> 172.21.15.26:0/1690344265 conn(0x7ffbd8071490 msgr2=0x7ffbd80d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.500+0000 7ffbe0641640 1 -- 172.21.15.26:0/1690344265 shutdown_connections 2023-12-19T18:14:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.500+0000 7ffbe0641640 1 -- 172.21.15.26:0/1690344265 wait complete. 2023-12-19T18:14:21.519 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:14:21.529 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:21.527+0000 7fdf4b9db180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:14:21.556 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:21.556+0000 7fd96f9d4180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:14:21.618 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:21.616+0000 7fd96f9d4180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:14:21.619 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:21.615+0000 7fdf4b9db180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:14:21.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b80e5640 1 Processor -- start 2023-12-19T18:14:21.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b80e5640 1 -- start start 2023-12-19T18:14:21.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b80e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57b015ccf0 0x7f57b00d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:21.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b80e5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f57b015d0d0 0x7f57b00b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:21.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b80e5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b015e200 con 0x7f57b00b4600 2023-12-19T18:14:21.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b80e5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b00d0aa0 con 0x7f57b015d0d0 2023-12-19T18:14:21.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b80e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b00d1380 con 0x7f57b015ccf0 2023-12-19T18:14:21.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b6682640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f57b00b4600 0x7f57b0160c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46784/0 (socket says 172.21.15.26:46784) 2023-12-19T18:14:21.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b6682640 1 -- 172.21.15.26:0/3539735804 learned_addr learned my addr 172.21.15.26:0/3539735804 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:21.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b5e81640 1 --2- 172.21.15.26:0/3539735804 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57b015ccf0 0x7f57b00d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:21.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b5680640 1 --2- 172.21.15.26:0/3539735804 >> v2:172.21.15.160:3300/0 conn(0x7f57b015d0d0 0x7f57b00b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:21.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b4e7f640 1 -- 172.21.15.26:0/3539735804 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741306781 0 0) 0x7f57b015e200 con 0x7f57b00b4600 2023-12-19T18:14:21.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b5e81640 1 -- 172.21.15.26:0/3539735804 >> v2:172.21.15.160:3300/0 conn(0x7f57b015d0d0 msgr2=0x7f57b00b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:21.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b5e81640 1 --2- 172.21.15.26:0/3539735804 >> v2:172.21.15.160:3300/0 conn(0x7f57b015d0d0 0x7f57b00b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b5e81640 1 -- 172.21.15.26:0/3539735804 >> v1:172.21.15.26:6789/0 conn(0x7f57b00b4600 legacy=0x7f57b0160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:21.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.672+0000 7f57b5e81640 1 -- 172.21.15.26:0/3539735804 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b015d680 con 0x7f57b015ccf0 2023-12-19T18:14:21.675 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:21.675+0000 7fdf4b9db180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:14:21.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.676+0000 7f57b5e81640 1 --2- 172.21.15.26:0/3539735804 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57b015ccf0 0x7f57b00d0e40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0097c0 tx=0x7f57ac01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f62a7066e1e84d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:21.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.676+0000 7f57b4e7f640 1 -- 172.21.15.26:0/3539735804 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0ad660 con 0x7f57b015ccf0 2023-12-19T18:14:21.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.676+0000 7f57b4e7f640 1 -- 172.21.15.26:0/3539735804 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac0ad7c0 con 0x7f57b015ccf0 2023-12-19T18:14:21.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.676+0000 7f57b4e7f640 1 -- 172.21.15.26:0/3539735804 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0adb50 con 0x7f57b015ccf0 2023-12-19T18:14:21.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.676+0000 7f57b80e5640 1 -- 172.21.15.26:0/3539735804 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57b015ccf0 msgr2=0x7f57b00d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.676+0000 7f57b80e5640 1 --2- 172.21.15.26:0/3539735804 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57b015ccf0 0x7f57b00d0e40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0097c0 tx=0x7f57ac01c450 comp rx=0 tx=0).stop 2023-12-19T18:14:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.676+0000 7f57b80e5640 1 -- 172.21.15.26:0/3539735804 shutdown_connections 2023-12-19T18:14:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.676+0000 7f57b80e5640 1 --2- 172.21.15.26:0/3539735804 >> v2:172.21.15.160:3300/0 conn(0x7f57b015d0d0 0x7f57b00b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.676+0000 7f57b80e5640 1 --2- 172.21.15.26:0/3539735804 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57b015ccf0 0x7f57b00d0e40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.676+0000 7f57b80e5640 1 -- 172.21.15.26:0/3539735804 >> 172.21.15.26:0/3539735804 conn(0x7f57b0065ee0 msgr2=0x7f57b00cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.676+0000 7f57b80e5640 1 -- 172.21.15.26:0/3539735804 shutdown_connections 2023-12-19T18:14:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.676+0000 7f57b80e5640 1 -- 172.21.15.26:0/3539735804 wait complete. 2023-12-19T18:14:21.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b80e5640 1 Processor -- start 2023-12-19T18:14:21.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b80e5640 1 -- start start 2023-12-19T18:14:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b80e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57b00b4600 0x7f57b015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b80e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57b015ccf0 0x7f57b015e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b80e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57b015d0d0 0x7f57b01f8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b80e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b00b7b00 con 0x7f57b00b4600 2023-12-19T18:14:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b80e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b0157a20 con 0x7f57b015ccf0 2023-12-19T18:14:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b80e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b0157d00 con 0x7f57b015d0d0 2023-12-19T18:14:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b5e81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57b00b4600 0x7f57b015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b5e81640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57b00b4600 0x7f57b015e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45356/0 (socket says 172.21.15.26:45356) 2023-12-19T18:14:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b5e81640 1 -- 172.21.15.26:0/3006260257 learned_addr learned my addr 172.21.15.26:0/3006260257 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b5680640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57b015ccf0 0x7f57b015e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b6682640 1 --2- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57b015d0d0 0x7f57b01f8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:21.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b5e81640 1 -- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57b015d0d0 msgr2=0x7f57b01f8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:21.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b5e81640 1 --2- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57b015d0d0 0x7f57b01f8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b5e81640 1 -- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57b015ccf0 msgr2=0x7f57b015e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b5e81640 1 --2- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57b015ccf0 0x7f57b015e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b5e81640 1 -- 172.21.15.26:0/3006260257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac009420 con 0x7f57b00b4600 2023-12-19T18:14:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b5680640 1 --2- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57b015ccf0 0x7f57b015e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b5e81640 1 --2- 172.21.15.26:0/3006260257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57b00b4600 0x7f57b015e3f0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0b0040 tx=0x7f57ac01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57a67fc640 1 -- 172.21.15.26:0/3006260257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0b9650 con 0x7f57b00b4600 2023-12-19T18:14:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57a67fc640 1 -- 172.21.15.26:0/3006260257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ac0b97f0 con 0x7f57b00b4600 2023-12-19T18:14:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57a67fc640 1 -- 172.21.15.26:0/3006260257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0b9ba0 con 0x7f57b00b4600 2023-12-19T18:14:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b80e5640 1 -- 172.21.15.26:0/3006260257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b0157fe0 con 0x7f57b00b4600 2023-12-19T18:14:21.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.680+0000 7f57b80e5640 1 -- 172.21.15.26:0/3006260257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b00c9030 con 0x7f57b00b4600 2023-12-19T18:14:21.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.684+0000 7f57b80e5640 1 -- 172.21.15.26:0/3006260257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b0074270 con 0x7f57b00b4600 2023-12-19T18:14:21.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.684+0000 7f57a67fc640 1 -- 172.21.15.26:0/3006260257 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 103) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f57ac0b9d00 con 0x7f57b00b4600 2023-12-19T18:14:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.684+0000 7f57a67fc640 1 --2- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5794065b80 0x7f5794068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.684+0000 7f57a67fc640 1 -- 172.21.15.26:0/3006260257 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 7144+0+0 (secure 0 0 0) 0x7f57ac11e430 con 0x7f57b00b4600 2023-12-19T18:14:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.684+0000 7f57b5680640 1 -- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5794065b80 msgr2=0x7f5794068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.684+0000 7f57b5680640 1 --2- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5794065b80 0x7f5794068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:21.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.688+0000 7f57a67fc640 1 -- 172.21.15.26:0/3006260257 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f57ac0eb6d0 con 0x7f57b00b4600 2023-12-19T18:14:21.713 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:21.712+0000 7fd96f9d4180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:14:21.760 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:21.759+0000 7fdf4b9db180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:14:21.773 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:21.772+0000 7fd96f9d4180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:14:21.816 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:21.815+0000 7fdf4b9db180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:14:21.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.844+0000 7f57b80e5640 1 -- 172.21.15.26:0/3006260257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57b00d1960 con 0x7f57b00b4600 2023-12-19T18:14:21.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.848+0000 7f57a67fc640 1 -- 172.21.15.26:0/3006260257 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v183) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57ac0eecf0 con 0x7f57b00b4600 2023-12-19T18:14:21.847 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:14:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.848+0000 7f57b80e5640 1 -- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5794065b80 msgr2=0x7f5794068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:14:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.848+0000 7f57b80e5640 1 --2- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5794065b80 0x7f5794068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.848+0000 7f57b80e5640 1 -- 172.21.15.26:0/3006260257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57b00b4600 msgr2=0x7f57b015e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.848+0000 7f57b80e5640 1 --2- 172.21.15.26:0/3006260257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57b00b4600 0x7f57b015e3f0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0b0040 tx=0x7f57ac01ac90 comp rx=0 tx=0).stop 2023-12-19T18:14:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.848+0000 7f57b80e5640 1 -- 172.21.15.26:0/3006260257 shutdown_connections 2023-12-19T18:14:21.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.848+0000 7f57b80e5640 1 --2- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5794065b80 0x7f5794068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.848+0000 7f57b80e5640 1 --2- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57b015d0d0 0x7f57b01f8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.848+0000 7f57b80e5640 1 --2- 172.21.15.26:0/3006260257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57b015ccf0 0x7f57b015e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.848+0000 7f57b80e5640 1 --2- 172.21.15.26:0/3006260257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57b00b4600 0x7f57b015e3f0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.848+0000 7f57b80e5640 1 -- 172.21.15.26:0/3006260257 >> 172.21.15.26:0/3006260257 conn(0x7f57b0065ee0 msgr2=0x7f57b0160c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.848+0000 7f57b80e5640 1 -- 172.21.15.26:0/3006260257 shutdown_connections 2023-12-19T18:14:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:21.848+0000 7f57b80e5640 1 -- 172.21.15.26:0/3006260257 wait complete. 2023-12-19T18:14:21.863 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:21.864+0000 7fd96f9d4180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:14:21.869 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:14:21.923 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:21.920+0000 7fd96f9d4180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:14:21.958 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:21.955+0000 7fdf4b9db180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:14:22.017 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:22.015+0000 7fdf4b9db180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:14:22.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd1347bc640 1 Processor -- start 2023-12-19T18:14:22.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd1347bc640 1 -- start start 2023-12-19T18:14:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd1347bc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd12c154220 0x7fd12c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd1347bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12c0cee10 0x7fd12c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:22.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd1347bc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12c06bc30 con 0x7fd12c06b3d0 2023-12-19T18:14:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd1347bc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c0cc6e0 con 0x7fd12c154220 2023-12-19T18:14:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd1347bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c15afc0 con 0x7fd12c0cee10 2023-12-19T18:14:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd131d57640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd12c06b3d0 0x7fd12c0ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46796/0 (socket says 172.21.15.26:46796) 2023-12-19T18:14:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd131d57640 1 -- 172.21.15.26:0/2414465464 learned_addr learned my addr 172.21.15.26:0/2414465464 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd132558640 1 --2- 172.21.15.26:0/2414465464 >> v2:172.21.15.160:3300/0 conn(0x7fd12c154220 0x7fd12c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:22.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd132d59640 1 --2- 172.21.15.26:0/2414465464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12c0cee10 0x7fd12c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:22.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd132558640 1 -- 172.21.15.26:0/2414465464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12c0cee10 msgr2=0x7fd12c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:22.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd132558640 1 --2- 172.21.15.26:0/2414465464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12c0cee10 0x7fd12c15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd132558640 1 -- 172.21.15.26:0/2414465464 >> v1:172.21.15.26:6789/0 conn(0x7fd12c06b3d0 legacy=0x7fd12c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:22.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.056+0000 7fd132558640 1 -- 172.21.15.26:0/2414465464 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c06b950 con 0x7fd12c154220 2023-12-19T18:14:22.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.060+0000 7fd132558640 1 --2- 172.21.15.26:0/2414465464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd12c154220 0x7fd12c15aa80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd1280097c0 tx=0x7fd12801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1097b8826f868d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:22.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.060+0000 7fd131556640 1 -- 172.21.15.26:0/2414465464 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1280ab650 con 0x7fd12c154220 2023-12-19T18:14:22.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.060+0000 7fd131556640 1 -- 172.21.15.26:0/2414465464 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1280ab7b0 con 0x7fd12c154220 2023-12-19T18:14:22.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.060+0000 7fd131556640 1 -- 172.21.15.26:0/2414465464 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1280abb40 con 0x7fd12c154220 2023-12-19T18:14:22.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.060+0000 7fd1347bc640 1 -- 172.21.15.26:0/2414465464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd12c154220 msgr2=0x7fd12c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:22.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.060+0000 7fd1347bc640 1 --2- 172.21.15.26:0/2414465464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd12c154220 0x7fd12c15aa80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd1280097c0 tx=0x7fd12801c450 comp rx=0 tx=0).stop 2023-12-19T18:14:22.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.060+0000 7fd1347bc640 1 -- 172.21.15.26:0/2414465464 shutdown_connections 2023-12-19T18:14:22.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.060+0000 7fd1347bc640 1 --2- 172.21.15.26:0/2414465464 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12c0cee10 0x7fd12c15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.060+0000 7fd1347bc640 1 --2- 172.21.15.26:0/2414465464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd12c154220 0x7fd12c15aa80 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.060+0000 7fd1347bc640 1 -- 172.21.15.26:0/2414465464 >> 172.21.15.26:0/2414465464 conn(0x7fd12c0658e0 msgr2=0x7fd12c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:22.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.060+0000 7fd1347bc640 1 -- 172.21.15.26:0/2414465464 shutdown_connections 2023-12-19T18:14:22.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.060+0000 7fd1347bc640 1 -- 172.21.15.26:0/2414465464 wait complete. 2023-12-19T18:14:22.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd1347bc640 1 Processor -- start 2023-12-19T18:14:22.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd1347bc640 1 -- start start 2023-12-19T18:14:22.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd1347bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd12c06b3d0 0x7fd12c146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:22.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd1347bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12c0cee10 0x7fd12c14bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:22.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd1347bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12c154220 0x7fd12c14d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:22.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd1347bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c14d250 con 0x7fd12c0cee10 2023-12-19T18:14:22.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd1347bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c14c640 con 0x7fd12c06b3d0 2023-12-19T18:14:22.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd1347bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c14c920 con 0x7fd12c154220 2023-12-19T18:14:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd131d57640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12c0cee10 0x7fd12c14bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd132d59640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12c154220 0x7fd12c14d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd132558640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd12c06b3d0 0x7fd12c146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd132d59640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12c154220 0x7fd12c14d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:36306/0 (socket says 172.21.15.26:36306) 2023-12-19T18:14:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd132558640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd12c06b3d0 0x7fd12c146370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:40642/0 (socket says 172.21.15.26:40642) 2023-12-19T18:14:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd132d59640 1 -- 172.21.15.26:0/1690577591 learned_addr learned my addr 172.21.15.26:0/1690577591 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd131d57640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12c0cee10 0x7fd12c14bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45360/0 (socket says 172.21.15.26:45360) 2023-12-19T18:14:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd131d57640 1 -- 172.21.15.26:0/1690577591 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12c154220 msgr2=0x7fd12c14d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd131d57640 1 --2- 172.21.15.26:0/1690577591 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12c154220 0x7fd12c14d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd131d57640 1 -- 172.21.15.26:0/1690577591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd12c06b3d0 msgr2=0x7fd12c146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd131d57640 1 --2- 172.21.15.26:0/1690577591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd12c06b3d0 0x7fd12c146370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd131d57640 1 -- 172.21.15.26:0/1690577591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd128009420 con 0x7fd12c0cee10 2023-12-19T18:14:22.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd131d57640 1 --2- 172.21.15.26:0/1690577591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12c0cee10 0x7fd12c14bc80 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00c340 tx=0x7fd11c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:22.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd122ffd640 1 -- 172.21.15.26:0/1690577591 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c005cf0 con 0x7fd12c0cee10 2023-12-19T18:14:22.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd122ffd640 1 -- 172.21.15.26:0/1690577591 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c005e50 con 0x7fd12c0cee10 2023-12-19T18:14:22.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd122ffd640 1 -- 172.21.15.26:0/1690577591 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c0072d0 con 0x7fd12c0cee10 2023-12-19T18:14:22.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd1347bc640 1 -- 172.21.15.26:0/1690577591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c14cc00 con 0x7fd12c0cee10 2023-12-19T18:14:22.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.064+0000 7fd1347bc640 1 -- 172.21.15.26:0/1690577591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c1468b0 con 0x7fd12c0cee10 2023-12-19T18:14:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.068+0000 7fd1347bc640 1 -- 172.21.15.26:0/1690577591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c14d400 con 0x7fd12c0cee10 2023-12-19T18:14:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.068+0000 7fd122ffd640 1 -- 172.21.15.26:0/1690577591 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 103) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fd11c007430 con 0x7fd12c0cee10 2023-12-19T18:14:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.068+0000 7fd122ffd640 1 --2- 172.21.15.26:0/1690577591 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd110065b80 0x7fd110068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.068+0000 7fd122ffd640 1 -- 172.21.15.26:0/1690577591 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 7144+0+0 (secure 0 0 0) 0x7fd11c082d00 con 0x7fd12c0cee10 2023-12-19T18:14:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.068+0000 7fd132558640 1 -- 172.21.15.26:0/1690577591 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd110065b80 msgr2=0x7fd110068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:22.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.068+0000 7fd132558640 1 --2- 172.21.15.26:0/1690577591 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd110065b80 0x7fd110068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:22.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.072+0000 7fd122ffd640 1 -- 172.21.15.26:0/1690577591 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd12c14d400 con 0x7fd12c0cee10 2023-12-19T18:14:22.087 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:22.088+0000 7fd96f9d4180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:14:22.161 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:22.160+0000 7fd96f9d4180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:14:22.260 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:14:22.260 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:14:22.261 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:14:22.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.264+0000 7fd1347bc640 1 -- 172.21.15.26:0/1690577591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd12c074270 con 0x7fd12c0cee10 2023-12-19T18:14:22.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.264+0000 7fd122ffd640 1 -- 172.21.15.26:0/1690577591 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 14 entries v183) v1 ==== 81+0+821 (secure 0 0 0) 0x7fd11c017080 con 0x7fd12c0cee10 2023-12-19T18:14:22.266 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3381018054 2023-12-20T18:09:20.657217+0000 2023-12-19T18:14:22.267 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3458963623 2023-12-20T18:09:20.657217+0000 2023-12-19T18:14:22.267 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1302880583 2023-12-20T18:09:20.657217+0000 2023-12-19T18:14:22.267 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1063410185 2023-12-20T18:09:20.657217+0000 2023-12-19T18:14:22.267 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/819581921 2023-12-20T17:58:25.501946+0000 2023-12-19T18:14:22.267 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/4076846554 2023-12-20T18:09:20.657217+0000 2023-12-19T18:14:22.267 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3832178910 2023-12-20T17:58:25.501946+0000 2023-12-19T18:14:22.267 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6833/38098 2023-12-20T17:58:25.501946+0000 2023-12-19T18:14:22.267 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6832/38098 2023-12-20T17:58:25.501946+0000 2023-12-19T18:14:22.267 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1419214494 2023-12-20T17:58:25.501946+0000 2023-12-19T18:14:22.267 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1333176784 2023-12-20T18:09:20.657217+0000 2023-12-19T18:14:22.268 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/4141924506 2023-12-20T17:58:25.501946+0000 2023-12-19T18:14:22.268 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2994631753 2023-12-20T17:58:25.501946+0000 2023-12-19T18:14:22.268 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2198643724 2023-12-20T17:58:25.501946+0000 2023-12-19T18:14:22.268 INFO:teuthology.orchestra.run.smithi026.stderr:listed 14 entries 2023-12-19T18:14:22.268 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:22.267+0000 7fdf4b9db180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:14:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.268+0000 7fd1347bc640 1 -- 172.21.15.26:0/1690577591 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd110065b80 msgr2=0x7fd110068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:14:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.268+0000 7fd1347bc640 1 --2- 172.21.15.26:0/1690577591 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd110065b80 0x7fd110068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.268+0000 7fd1347bc640 1 -- 172.21.15.26:0/1690577591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12c0cee10 msgr2=0x7fd12c14bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.268+0000 7fd1347bc640 1 --2- 172.21.15.26:0/1690577591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12c0cee10 0x7fd12c14bc80 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00c340 tx=0x7fd11c00c800 comp rx=0 tx=0).stop 2023-12-19T18:14:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.268+0000 7fd1347bc640 1 -- 172.21.15.26:0/1690577591 shutdown_connections 2023-12-19T18:14:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.268+0000 7fd1347bc640 1 --2- 172.21.15.26:0/1690577591 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd110065b80 0x7fd110068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.268+0000 7fd1347bc640 1 --2- 172.21.15.26:0/1690577591 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12c154220 0x7fd12c14d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.268+0000 7fd1347bc640 1 --2- 172.21.15.26:0/1690577591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12c0cee10 0x7fd12c14bc80 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.268+0000 7fd1347bc640 1 --2- 172.21.15.26:0/1690577591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd12c06b3d0 0x7fd12c146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.268+0000 7fd1347bc640 1 -- 172.21.15.26:0/1690577591 >> 172.21.15.26:0/1690577591 conn(0x7fd12c0658e0 msgr2=0x7fd12c0ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.268+0000 7fd1347bc640 1 -- 172.21.15.26:0/1690577591 shutdown_connections 2023-12-19T18:14:22.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.268+0000 7fd1347bc640 1 -- 172.21.15.26:0/1690577591 wait complete. 2023-12-19T18:14:22.298 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T18:14:22.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:22.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-19T18:14:22.403 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:22.399+0000 7fdf4b9db180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:14:22.408 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T18:14:22.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:22.408 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:14:22.418 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:14:22.419 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:14:22.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:14:22.430 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:14:22.430 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:14:22.432 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:14:22.432 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:14:22.432 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:14:22.442 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:22.440+0000 7fd96f9d4180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:14:22.444 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:14:22.444 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:14:22.444 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:14:22.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:14:22.480 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:14:22.528 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:22.527+0000 7fdf4b9db180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:14:22.586 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:22.584+0000 7fd96f9d4180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:14:22.599 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:22.595+0000 7fdf4b9db180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:14:22.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5483ba6640 1 Processor -- start 2023-12-19T18:14:22.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5483ba6640 1 -- start start 2023-12-19T18:14:22.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5483ba6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f547c06bee0 0x7f547c06c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:22.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5483ba6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f547c0b1d40 0x7f547c0b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:22.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5483ba6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547c06d4b0 con 0x7f547c0cb1d0 2023-12-19T18:14:22.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5483ba6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547c0cc030 con 0x7f547c06bee0 2023-12-19T18:14:22.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5483ba6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547c06cc10 con 0x7f547c0b1d40 2023-12-19T18:14:22.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5481141640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f547c06bee0 0x7f547c06c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:22.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5481942640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f547c0cb1d0 0x7f547c0cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46800/0 (socket says 172.21.15.26:46800) 2023-12-19T18:14:22.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5481942640 1 -- 172.21.15.26:0/3278343 learned_addr learned my addr 172.21.15.26:0/3278343 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:22.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5482143640 1 --2- 172.21.15.26:0/3278343 >> v2:172.21.15.160:3300/0 conn(0x7f547c0b1d40 0x7f547c0b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:22.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5480940640 1 -- 172.21.15.26:0/3278343 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633247807 0 0) 0x7f547c06d4b0 con 0x7f547c0cb1d0 2023-12-19T18:14:22.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5480940640 1 -- 172.21.15.26:0/3278343 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5468003530 con 0x7f547c0cb1d0 2023-12-19T18:14:22.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5481141640 1 -- 172.21.15.26:0/3278343 >> v2:172.21.15.160:3300/0 conn(0x7f547c0b1d40 msgr2=0x7f547c0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:22.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5481141640 1 --2- 172.21.15.26:0/3278343 >> v2:172.21.15.160:3300/0 conn(0x7f547c0b1d40 0x7f547c0b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5481141640 1 -- 172.21.15.26:0/3278343 >> v1:172.21.15.26:6789/0 conn(0x7f547c0cb1d0 legacy=0x7f547c0cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:22.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.632+0000 7f5481141640 1 -- 172.21.15.26:0/3278343 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c06cef0 con 0x7f547c06bee0 2023-12-19T18:14:22.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.636+0000 7f5481141640 1 --2- 172.21.15.26:0/3278343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f547c06bee0 0x7f547c06c340 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f547c066890 tx=0x7f546c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29aada56791e0e8b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.636+0000 7f5480940640 1 -- 172.21.15.26:0/3278343 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c0ac600 con 0x7f547c06bee0 2023-12-19T18:14:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.636+0000 7f5480940640 1 -- 172.21.15.26:0/3278343 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c0a8050 con 0x7f547c06bee0 2023-12-19T18:14:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.636+0000 7f5480940640 1 -- 172.21.15.26:0/3278343 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c0ac980 con 0x7f547c06bee0 2023-12-19T18:14:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.636+0000 7f5483ba6640 1 -- 172.21.15.26:0/3278343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f547c06bee0 msgr2=0x7f547c06c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.636+0000 7f5483ba6640 1 --2- 172.21.15.26:0/3278343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f547c06bee0 0x7f547c06c340 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f547c066890 tx=0x7f546c01c3a0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.636+0000 7f5483ba6640 1 -- 172.21.15.26:0/3278343 shutdown_connections 2023-12-19T18:14:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.636+0000 7f5483ba6640 1 --2- 172.21.15.26:0/3278343 >> v2:172.21.15.160:3300/0 conn(0x7f547c0b1d40 0x7f547c0b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.636+0000 7f5483ba6640 1 --2- 172.21.15.26:0/3278343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f547c06bee0 0x7f547c06c340 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.636+0000 7f5483ba6640 1 -- 172.21.15.26:0/3278343 >> 172.21.15.26:0/3278343 conn(0x7f547c0658e0 msgr2=0x7f547c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:22.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.636+0000 7f5483ba6640 1 -- 172.21.15.26:0/3278343 shutdown_connections 2023-12-19T18:14:22.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.636+0000 7f5483ba6640 1 -- 172.21.15.26:0/3278343 wait complete. 2023-12-19T18:14:22.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5483ba6640 1 Processor -- start 2023-12-19T18:14:22.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5483ba6640 1 -- start start 2023-12-19T18:14:22.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5483ba6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f547c06bee0 0x7f547c1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:22.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5483ba6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f547c0b1d40 0x7f547c150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:22.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5483ba6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f547c0cb1d0 0x7f547c0b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5483ba6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c15aa80 con 0x7f547c0cb1d0 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5483ba6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c159e70 con 0x7f547c0b1d40 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5483ba6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547c15a150 con 0x7f547c06bee0 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5482143640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f547c0cb1d0 0x7f547c0b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5482143640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f547c0cb1d0 0x7f547c0b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45388/0 (socket says 172.21.15.26:45388) 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5482143640 1 -- 172.21.15.26:0/3475343511 learned_addr learned my addr 172.21.15.26:0/3475343511 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5481942640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f547c06bee0 0x7f547c1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5481141640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f547c0b1d40 0x7f547c150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5482143640 1 -- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f547c06bee0 msgr2=0x7f547c1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5482143640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f547c06bee0 0x7f547c1561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5482143640 1 -- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f547c0b1d40 msgr2=0x7f547c150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5482143640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f547c0b1d40 0x7f547c150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5482143640 1 -- 172.21.15.26:0/3475343511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c009420 con 0x7f547c0cb1d0 2023-12-19T18:14:22.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5481942640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f547c06bee0 0x7f547c1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:22.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5482143640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f547c0cb1d0 0x7f547c0b03d0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f547400efc0 tx=0x7f547400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:22.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f54727fc640 1 -- 172.21.15.26:0/3475343511 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474014480 con 0x7f547c0cb1d0 2023-12-19T18:14:22.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f54727fc640 1 -- 172.21.15.26:0/3475343511 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547400f040 con 0x7f547c0cb1d0 2023-12-19T18:14:22.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f54727fc640 1 -- 172.21.15.26:0/3475343511 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474014820 con 0x7f547c0cb1d0 2023-12-19T18:14:22.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5483ba6640 1 -- 172.21.15.26:0/3475343511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547c15a430 con 0x7f547c0cb1d0 2023-12-19T18:14:22.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.640+0000 7f5483ba6640 1 -- 172.21.15.26:0/3475343511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547c1509c0 con 0x7f547c0cb1d0 2023-12-19T18:14:22.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.644+0000 7f5483ba6640 1 -- 172.21.15.26:0/3475343511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547c15abf0 con 0x7f547c0cb1d0 2023-12-19T18:14:22.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.644+0000 7f54727fc640 1 -- 172.21.15.26:0/3475343511 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 103) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f5474014980 con 0x7f547c0cb1d0 2023-12-19T18:14:22.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.644+0000 7f54727fc640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5460065b80 0x7f5460068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:22.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.644+0000 7f54727fc640 1 -- 172.21.15.26:0/3475343511 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 7144+0+0 (secure 0 0 0) 0x7f54740844f0 con 0x7f547c0cb1d0 2023-12-19T18:14:22.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.644+0000 7f5481942640 1 -- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5460065b80 msgr2=0x7f5460068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:22.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.644+0000 7f5481942640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5460065b80 0x7f5460068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:22.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.648+0000 7f54727fc640 1 -- 172.21.15.26:0/3475343511 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f54740516e0 con 0x7f547c0cb1d0 2023-12-19T18:14:22.721 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:22.720+0000 7fd96f9d4180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:14:22.734 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:22.731+0000 7fdf4b9db180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:14:22.796 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:22.796+0000 7fd96f9d4180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:14:22.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.844+0000 7f5483ba6640 1 -- 172.21.15.26:0/3475343511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f547c074270 con 0x7f547c0cb1d0 2023-12-19T18:14:22.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5481942640 1 -- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5460065b80 msgr2=0x7f5460068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:22.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5481942640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5460065b80 0x7f5460068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T18:14:22.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f54727fc640 1 -- 172.21.15.26:0/3475343511 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f5474054d00 con 0x7f547c0cb1d0 2023-12-19T18:14:22.848 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:22.848 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2924688599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2924688599},{"type":"v1","addr":"172.21.15.26:6835","nonce":2924688599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/330338341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":330338341},{"type":"v1","addr":"172.21.15.160:6836","nonce":330338341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9031,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/887954085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":887954085},{"type":"v1","addr":"172.21.15.160:6837","nonce":887954085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:58:13.473962+0000","modified":"2023-12-19T17:58:16.484662+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9513},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9513":{"gid":9513,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/3354708287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3354708287},{"type":"v1","addr":"172.21.15.26:6834","nonce":3354708287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-19T18:14:22.848 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 80 2023-12-19T18:14:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5483ba6640 1 -- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5460065b80 msgr2=0x7f5460068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:14:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5483ba6640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5460065b80 0x7f5460068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5483ba6640 1 -- 172.21.15.26:0/3475343511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f547c0cb1d0 msgr2=0x7f547c0b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5483ba6640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f547c0cb1d0 0x7f547c0b03d0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f547400efc0 tx=0x7f547400c490 comp rx=0 tx=0).stop 2023-12-19T18:14:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5483ba6640 1 -- 172.21.15.26:0/3475343511 shutdown_connections 2023-12-19T18:14:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5483ba6640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5460065b80 0x7f5460068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5483ba6640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f547c0cb1d0 0x7f547c0b03d0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5483ba6640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f547c0b1d40 0x7f547c150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5483ba6640 1 --2- 172.21.15.26:0/3475343511 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f547c06bee0 0x7f547c1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5483ba6640 1 -- 172.21.15.26:0/3475343511 >> 172.21.15.26:0/3475343511 conn(0x7f547c0658e0 msgr2=0x7f547c0708b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5483ba6640 1 -- 172.21.15.26:0/3475343511 shutdown_connections 2023-12-19T18:14:22.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:22.848+0000 7f5483ba6640 1 -- 172.21.15.26:0/3475343511 wait complete. 2023-12-19T18:14:22.870 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:22.942 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:22.940+0000 7fd96f9d4180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:14:23.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.016+0000 7f4309afc640 1 Processor -- start 2023-12-19T18:14:23.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.016+0000 7f4309afc640 1 -- start start 2023-12-19T18:14:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.016+0000 7f4309afc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f43040b88e0 0x7f43040b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.016+0000 7f4309afc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43040d2960 con 0x7f430415cf60 2023-12-19T18:14:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.016+0000 7f4309afc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43040b4f40 con 0x7f43040b4b40 2023-12-19T18:14:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.016+0000 7f4309afc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43040d15f0 con 0x7f43040b88e0 2023-12-19T18:14:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.016+0000 7f43037fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f430415cf60 0x7f43040d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46812/0 (socket says 172.21.15.26:46812) 2023-12-19T18:14:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.020+0000 7f43037fe640 1 -- 172.21.15.26:0/2359538171 learned_addr learned my addr 172.21.15.26:0/2359538171 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:23.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.020+0000 7f4302ffd640 1 --2- 172.21.15.26:0/2359538171 >> v2:172.21.15.160:3300/0 conn(0x7f43040b88e0 0x7f43040b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.020+0000 7f43027fc640 1 -- 172.21.15.26:0/2359538171 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246738463 0 0) 0x7f43040d2960 con 0x7f430415cf60 2023-12-19T18:14:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.020+0000 7f43027fc640 1 -- 172.21.15.26:0/2359538171 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42ec003530 con 0x7f430415cf60 2023-12-19T18:14:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.020+0000 7f43027fc640 1 -- 172.21.15.26:0/2359538171 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757685262 0 0) 0x7f43040b4f40 con 0x7f43040b4b40 2023-12-19T18:14:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.020+0000 7f43027fc640 1 -- 172.21.15.26:0/2359538171 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43040d2960 con 0x7f43040b4b40 2023-12-19T18:14:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.020+0000 7f4302ffd640 1 -- 172.21.15.26:0/2359538171 >> v1:172.21.15.160:6789/0 conn(0x7f43040b4b40 legacy=0x7f43040b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.020+0000 7f4302ffd640 1 -- 172.21.15.26:0/2359538171 >> v1:172.21.15.26:6789/0 conn(0x7f430415cf60 legacy=0x7f43040d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.020+0000 7f43027fc640 1 -- 172.21.15.26:0/2359538171 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3550389277 0 0) 0x7f42ec003530 con 0x7f430415cf60 2023-12-19T18:14:23.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.020+0000 7f4302ffd640 1 -- 172.21.15.26:0/2359538171 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43040d18c0 con 0x7f43040b88e0 2023-12-19T18:14:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.020+0000 7f4302ffd640 1 --2- 172.21.15.26:0/2359538171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43040b88e0 0x7f43040b4600 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f42f80097c0 tx=0x7f42f801eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db046a3711a44a0b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.020+0000 7f43027fc640 1 -- 172.21.15.26:0/2359538171 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f4001680 con 0x7f43040b88e0 2023-12-19T18:14:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f43027fc640 1 -- 172.21.15.26:0/2359538171 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f80a8610 con 0x7f43040b88e0 2023-12-19T18:14:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f43027fc640 1 -- 172.21.15.26:0/2359538171 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f80a8980 con 0x7f43040b88e0 2023-12-19T18:14:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 -- 172.21.15.26:0/2359538171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43040b88e0 msgr2=0x7f43040b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 --2- 172.21.15.26:0/2359538171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43040b88e0 0x7f43040b4600 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f42f80097c0 tx=0x7f42f801eb90 comp rx=0 tx=0).stop 2023-12-19T18:14:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 -- 172.21.15.26:0/2359538171 shutdown_connections 2023-12-19T18:14:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 --2- 172.21.15.26:0/2359538171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43040b88e0 0x7f43040b4600 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 -- 172.21.15.26:0/2359538171 >> 172.21.15.26:0/2359538171 conn(0x7f4304065ee0 msgr2=0x7f43040cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 -- 172.21.15.26:0/2359538171 shutdown_connections 2023-12-19T18:14:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 -- 172.21.15.26:0/2359538171 wait complete. 2023-12-19T18:14:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 Processor -- start 2023-12-19T18:14:23.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 -- start start 2023-12-19T18:14:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43040b4b40 0x7f43040c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43040b88e0 0x7f4304152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f430415cf60 0x7f4304153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4304155a40 con 0x7f43040b88e0 2023-12-19T18:14:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4304154e90 con 0x7f43040b4b40 2023-12-19T18:14:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.024+0000 7f4309afc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4304155110 con 0x7f430415cf60 2023-12-19T18:14:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f4302ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43040b88e0 0x7f4304152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f43037fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43040b4b40 0x7f43040c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f4302ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43040b88e0 0x7f4304152d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45390/0 (socket says 172.21.15.26:45390) 2023-12-19T18:14:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f4302ffd640 1 -- 172.21.15.26:0/2998536735 learned_addr learned my addr 172.21.15.26:0/2998536735 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f4303fff640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f430415cf60 0x7f4304153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f4302ffd640 1 -- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f430415cf60 msgr2=0x7f4304153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f4302ffd640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f430415cf60 0x7f4304153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f4302ffd640 1 -- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43040b4b40 msgr2=0x7f43040c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f4302ffd640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43040b4b40 0x7f43040c9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f4302ffd640 1 -- 172.21.15.26:0/2998536735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f8009420 con 0x7f43040b88e0 2023-12-19T18:14:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f43037fe640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43040b4b40 0x7f43040c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:23.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f4302ffd640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43040b88e0 0x7f4304152d10 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f42f801b7a0 tx=0x7f42f801b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f42ebfff640 1 -- 172.21.15.26:0/2998536735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f80a9ca0 con 0x7f43040b88e0 2023-12-19T18:14:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f42ebfff640 1 -- 172.21.15.26:0/2998536735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f80a9e00 con 0x7f43040b88e0 2023-12-19T18:14:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f42ebfff640 1 -- 172.21.15.26:0/2998536735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f8003820 con 0x7f43040b88e0 2023-12-19T18:14:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f4309afc640 1 -- 172.21.15.26:0/2998536735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43041553f0 con 0x7f43040b88e0 2023-12-19T18:14:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f4309afc640 1 -- 172.21.15.26:0/2998536735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4304155ed0 con 0x7f43040b88e0 2023-12-19T18:14:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.028+0000 7f4309afc640 1 -- 172.21.15.26:0/2998536735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4304153a70 con 0x7f43040b88e0 2023-12-19T18:14:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.032+0000 7f42ebfff640 1 -- 172.21.15.26:0/2998536735 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 103) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f42f80b7040 con 0x7f43040b88e0 2023-12-19T18:14:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.032+0000 7f42ebfff640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42f0065b80 0x7f42f0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.032+0000 7f42ebfff640 1 -- 172.21.15.26:0/2998536735 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 7144+0+0 (secure 0 0 0) 0x7f42f8120d50 con 0x7f43040b88e0 2023-12-19T18:14:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.032+0000 7f43037fe640 1 -- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42f0065b80 msgr2=0x7f42f0068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:23.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.032+0000 7f43037fe640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42f0065b80 0x7f42f0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.036+0000 7f42ebfff640 1 -- 172.21.15.26:0/2998536735 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4304153a70 con 0x7f43040b88e0 2023-12-19T18:14:23.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.236+0000 7f43037fe640 1 -- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42f0065b80 msgr2=0x7f42f0068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:23.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.236+0000 7f43037fe640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42f0065b80 0x7f42f0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T18:14:23.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.244+0000 7f4309afc640 1 -- 172.21.15.26:0/2998536735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4304153a70 con 0x7f43040b88e0 2023-12-19T18:14:23.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.248+0000 7f42ebfff640 1 -- 172.21.15.26:0/2998536735 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+18281 (secure 0 0 0) 0x7f4304153a70 con 0x7f43040b88e0 2023-12-19T18:14:23.248 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:23.249 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":183,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:14:21.047959+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:58:12.042476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T17:58:12.457937+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":183,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.29","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/3381018054":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/3458963623":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/1302880583":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/1063410185":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/819581921":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/4076846554":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/3832178910":"2023-12-20T17:58:25.501946+0000","172.21.15.160:6833/38098":"2023-12-20T17:58:25.501946+0000","172.21.15.160:6832/38098":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/1419214494":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/1333176784":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/4141924506":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/2994631753":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/2198643724":"2023-12-20T17:58:25.501946+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":17,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:23.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.248+0000 7f4309afc640 1 -- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42f0065b80 msgr2=0x7f42f0068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:14:23.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.248+0000 7f4309afc640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42f0065b80 0x7f42f0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.248+0000 7f4309afc640 1 -- 172.21.15.26:0/2998536735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43040b88e0 msgr2=0x7f4304152d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.248+0000 7f4309afc640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43040b88e0 0x7f4304152d10 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f42f801b7a0 tx=0x7f42f801b930 comp rx=0 tx=0).stop 2023-12-19T18:14:23.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.248+0000 7f4309afc640 1 -- 172.21.15.26:0/2998536735 shutdown_connections 2023-12-19T18:14:23.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.248+0000 7f4309afc640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42f0065b80 0x7f42f0068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.248+0000 7f4309afc640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f430415cf60 0x7f4304153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.248+0000 7f4309afc640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43040b88e0 0x7f4304152d10 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.248+0000 7f4309afc640 1 --2- 172.21.15.26:0/2998536735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43040b4b40 0x7f43040c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.248+0000 7f4309afc640 1 -- 172.21.15.26:0/2998536735 >> 172.21.15.26:0/2998536735 conn(0x7f4304065ee0 msgr2=0x7f43040b5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:23.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.248+0000 7f4309afc640 1 -- 172.21.15.26:0/2998536735 shutdown_connections 2023-12-19T18:14:23.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.248+0000 7f4309afc640 1 -- 172.21.15.26:0/2998536735 wait complete. 2023-12-19T18:14:23.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T17:58:12.042476+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '176', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T17:58:12.457937+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 183, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:14:23.272 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:14:23.276 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:23.275+0000 7fdf4b9db180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:14:23.359 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:23.359+0000 7fdf4b9db180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:14:23.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.420+0000 7f24042f2640 1 Processor -- start 2023-12-19T18:14:23.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.420+0000 7f24042f2640 1 -- start start 2023-12-19T18:14:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.420+0000 7f24042f2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23fc06f6c0 0x7f23fc15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.420+0000 7f24042f2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23fc06fb60 con 0x7f23fc159820 2023-12-19T18:14:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.420+0000 7f24042f2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23fc06fe40 con 0x7f23fc159c00 2023-12-19T18:14:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.420+0000 7f24042f2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc15e470 con 0x7f23fc06f6c0 2023-12-19T18:14:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240208e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f23fc159820 0x7f23fc0c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46814/0 (socket says 172.21.15.26:46814) 2023-12-19T18:14:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240208e640 1 -- 172.21.15.26:0/2972206746 learned_addr learned my addr 172.21.15.26:0/2972206746 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:23.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240188d640 1 --2- 172.21.15.26:0/2972206746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23fc06f6c0 0x7f23fc15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240108c640 1 -- 172.21.15.26:0/2972206746 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696679524 0 0) 0x7f23fc06fb60 con 0x7f23fc159820 2023-12-19T18:14:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240108c640 1 -- 172.21.15.26:0/2972206746 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23e8003530 con 0x7f23fc159820 2023-12-19T18:14:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240108c640 1 -- 172.21.15.26:0/2972206746 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270396674 0 0) 0x7f23fc06fe40 con 0x7f23fc159c00 2023-12-19T18:14:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240108c640 1 -- 172.21.15.26:0/2972206746 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23fc06fb60 con 0x7f23fc159c00 2023-12-19T18:14:23.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240188d640 1 -- 172.21.15.26:0/2972206746 >> v1:172.21.15.160:6789/0 conn(0x7f23fc159c00 legacy=0x7f23fc070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240188d640 1 -- 172.21.15.26:0/2972206746 >> v1:172.21.15.26:6789/0 conn(0x7f23fc159820 legacy=0x7f23fc0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240188d640 1 -- 172.21.15.26:0/2972206746 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc070120 con 0x7f23fc06f6c0 2023-12-19T18:14:23.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240188d640 1 --2- 172.21.15.26:0/2972206746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23fc06f6c0 0x7f23fc15df30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0097f0 tx=0x7f23ec01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd94970c5641dec4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:23.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240108c640 1 -- 172.21.15.26:0/2972206746 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec0aa5f0 con 0x7f23fc06f6c0 2023-12-19T18:14:23.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240108c640 1 -- 172.21.15.26:0/2972206746 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ec0aa750 con 0x7f23fc06f6c0 2023-12-19T18:14:23.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.424+0000 7f240108c640 1 -- 172.21.15.26:0/2972206746 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec0aaae0 con 0x7f23fc06f6c0 2023-12-19T18:14:23.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 -- 172.21.15.26:0/2972206746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23fc06f6c0 msgr2=0x7f23fc15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 --2- 172.21.15.26:0/2972206746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23fc06f6c0 0x7f23fc15df30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0097f0 tx=0x7f23ec01c490 comp rx=0 tx=0).stop 2023-12-19T18:14:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 -- 172.21.15.26:0/2972206746 shutdown_connections 2023-12-19T18:14:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 --2- 172.21.15.26:0/2972206746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23fc06f6c0 0x7f23fc15df30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 -- 172.21.15.26:0/2972206746 >> 172.21.15.26:0/2972206746 conn(0x7f23fc0ce730 msgr2=0x7f23fc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 -- 172.21.15.26:0/2972206746 shutdown_connections 2023-12-19T18:14:23.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 -- 172.21.15.26:0/2972206746 wait complete. 2023-12-19T18:14:23.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 Processor -- start 2023-12-19T18:14:23.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 -- start start 2023-12-19T18:14:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23fc06f6c0 0x7f23fc0c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23fc159820 0x7f23fc153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23fc159c00 0x7f23fc0afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc0affa0 con 0x7f23fc159820 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc0c46c0 con 0x7f23fc06f6c0 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f24042f2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc0c49a0 con 0x7f23fc159c00 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f240188d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23fc159820 0x7f23fc153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f240188d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23fc159820 0x7f23fc153da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45402/0 (socket says 172.21.15.26:45402) 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.428+0000 7f240188d640 1 -- 172.21.15.26:0/3548381800 learned_addr learned my addr 172.21.15.26:0/3548381800 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f240208e640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23fc06f6c0 0x7f23fc0c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f240288f640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23fc159c00 0x7f23fc0afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f240188d640 1 -- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23fc159c00 msgr2=0x7f23fc0afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f240188d640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23fc159c00 0x7f23fc0afa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f240188d640 1 -- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23fc06f6c0 msgr2=0x7f23fc0c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f240188d640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23fc06f6c0 0x7f23fc0c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f240188d640 1 -- 172.21.15.26:0/3548381800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ec009420 con 0x7f23fc159820 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f240208e640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23fc06f6c0 0x7f23fc0c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f240188d640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23fc159820 0x7f23fc153da0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0ac040 tx=0x7f23ec01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f23f2ffd640 1 -- 172.21.15.26:0/3548381800 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec0be610 con 0x7f23fc159820 2023-12-19T18:14:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f23f2ffd640 1 -- 172.21.15.26:0/3548381800 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ec0be7b0 con 0x7f23fc159820 2023-12-19T18:14:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f24042f2640 1 -- 172.21.15.26:0/3548381800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23fc06b880 con 0x7f23fc159820 2023-12-19T18:14:23.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f23f2ffd640 1 -- 172.21.15.26:0/3548381800 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec0beb60 con 0x7f23fc159820 2023-12-19T18:14:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f24042f2640 1 -- 172.21.15.26:0/3548381800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23fc0c4f60 con 0x7f23fc159820 2023-12-19T18:14:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f24042f2640 1 -- 172.21.15.26:0/3548381800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23fc074270 con 0x7f23fc159820 2023-12-19T18:14:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.432+0000 7f23f2ffd640 1 -- 172.21.15.26:0/3548381800 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 103) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f23ec0b4400 con 0x7f23fc159820 2023-12-19T18:14:23.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.436+0000 7f23f2ffd640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f23e0065b80 0x7f23e0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.436+0000 7f23f2ffd640 1 -- 172.21.15.26:0/3548381800 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 7144+0+0 (secure 0 0 0) 0x7f23ec0a7070 con 0x7f23fc159820 2023-12-19T18:14:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.436+0000 7f240208e640 1 -- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f23e0065b80 msgr2=0x7f23e0068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:23.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.436+0000 7f240208e640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f23e0065b80 0x7f23e0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:23.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.440+0000 7f23f2ffd640 1 -- 172.21.15.26:0/3548381800 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f23ec0eb7c0 con 0x7f23fc159820 2023-12-19T18:14:23.509 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:23.507+0000 7fdf4b9db180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:14:23.575 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:23.576+0000 7fd96f9d4180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:14:23.592 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:23.591+0000 7fdf4b9db180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:14:23.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f24042f2640 1 -- 172.21.15.26:0/3548381800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23fc0b46b0 con 0x7f23fc159820 2023-12-19T18:14:23.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f23f2ffd640 1 -- 172.21.15.26:0/3548381800 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23ec0eede0 con 0x7f23fc159820 2023-12-19T18:14:23.600 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:14:23.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f24042f2640 1 -- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f23e0065b80 msgr2=0x7f23e0068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:14:23.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f24042f2640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f23e0065b80 0x7f23e0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f24042f2640 1 -- 172.21.15.26:0/3548381800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23fc159820 msgr2=0x7f23fc153da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f24042f2640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23fc159820 0x7f23fc153da0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f23ec0ac040 tx=0x7f23ec01acd0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f24042f2640 1 -- 172.21.15.26:0/3548381800 shutdown_connections 2023-12-19T18:14:23.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f24042f2640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f23e0065b80 0x7f23e0068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f24042f2640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23fc159c00 0x7f23fc0afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f24042f2640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23fc159820 0x7f23fc153da0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f24042f2640 1 --2- 172.21.15.26:0/3548381800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23fc06f6c0 0x7f23fc0c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f24042f2640 1 -- 172.21.15.26:0/3548381800 >> 172.21.15.26:0/3548381800 conn(0x7f23fc0ce730 msgr2=0x7f23fc0ae430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:23.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f24042f2640 1 -- 172.21.15.26:0/3548381800 shutdown_connections 2023-12-19T18:14:23.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.600+0000 7f24042f2640 1 -- 172.21.15.26:0/3548381800 wait complete. 2023-12-19T18:14:23.623 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:14:23.669 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:23.668+0000 7fd96f9d4180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:14:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d2cf96640 1 Processor -- start 2023-12-19T18:14:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d2cf96640 1 -- start start 2023-12-19T18:14:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d2cf96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d2815cd20 0x7f2d280b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d2cf96640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d280b5c00 con 0x7f2d280b4dd0 2023-12-19T18:14:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d2cf96640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2815d440 con 0x7f2d280b89c0 2023-12-19T18:14:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d2cf96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2815d6e0 con 0x7f2d2815cd20 2023-12-19T18:14:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d26d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2d280b4dd0 0x7f2d280b1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46824/0 (socket says 172.21.15.26:46824) 2023-12-19T18:14:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d26d9d640 1 -- 172.21.15.26:0/394683692 learned_addr learned my addr 172.21.15.26:0/394683692 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d2659c640 1 --2- 172.21.15.26:0/394683692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d2815cd20 0x7f2d280b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d2559a640 1 -- 172.21.15.26:0/394683692 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376060749 0 0) 0x7f2d280b5c00 con 0x7f2d280b4dd0 2023-12-19T18:14:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d2559a640 1 -- 172.21.15.26:0/394683692 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d10003530 con 0x7f2d280b4dd0 2023-12-19T18:14:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d2559a640 1 -- 172.21.15.26:0/394683692 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642586003 0 0) 0x7f2d2815d440 con 0x7f2d280b89c0 2023-12-19T18:14:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d2659c640 1 -- 172.21.15.26:0/394683692 >> v1:172.21.15.160:6789/0 conn(0x7f2d280b89c0 legacy=0x7f2d280b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d2659c640 1 -- 172.21.15.26:0/394683692 >> v1:172.21.15.26:6789/0 conn(0x7f2d280b4dd0 legacy=0x7f2d280b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.772+0000 7f2d2659c640 1 -- 172.21.15.26:0/394683692 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2815d9c0 con 0x7f2d2815cd20 2023-12-19T18:14:23.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.776+0000 7f2d2659c640 1 --2- 172.21.15.26:0/394683692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d2815cd20 0x7f2d280b8480 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2d180097f0 tx=0x7f2d1801ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a05d2e6109adbcb9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:23.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.776+0000 7f2d2559a640 1 -- 172.21.15.26:0/394683692 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d180a89b0 con 0x7f2d2815cd20 2023-12-19T18:14:23.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.776+0000 7f2d2559a640 1 -- 172.21.15.26:0/394683692 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d180a8b10 con 0x7f2d2815cd20 2023-12-19T18:14:23.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.776+0000 7f2d2559a640 1 -- 172.21.15.26:0/394683692 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d180a8ea0 con 0x7f2d2815cd20 2023-12-19T18:14:23.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.776+0000 7f2d2cf96640 1 -- 172.21.15.26:0/394683692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d2815cd20 msgr2=0x7f2d280b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.776+0000 7f2d2cf96640 1 --2- 172.21.15.26:0/394683692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d2815cd20 0x7f2d280b8480 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2d180097f0 tx=0x7f2d1801ecb0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.776+0000 7f2d2cf96640 1 -- 172.21.15.26:0/394683692 shutdown_connections 2023-12-19T18:14:23.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.776+0000 7f2d2cf96640 1 --2- 172.21.15.26:0/394683692 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d2815cd20 0x7f2d280b8480 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.776+0000 7f2d2cf96640 1 -- 172.21.15.26:0/394683692 >> 172.21.15.26:0/394683692 conn(0x7f2d2806a120 msgr2=0x7f2d2806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:23.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.776+0000 7f2d2cf96640 1 -- 172.21.15.26:0/394683692 shutdown_connections 2023-12-19T18:14:23.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.776+0000 7f2d2cf96640 1 -- 172.21.15.26:0/394683692 wait complete. 2023-12-19T18:14:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d2cf96640 1 Processor -- start 2023-12-19T18:14:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d2cf96640 1 -- start start 2023-12-19T18:14:23.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d2cf96640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d280b4dd0 0x7f2d280ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d2cf96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d280b89c0 0x7f2d28153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d2cf96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d2815cd20 0x7f2d280cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d2cf96640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d280c73f0 con 0x7f2d2815cd20 2023-12-19T18:14:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d2cf96640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d281f3160 con 0x7f2d280b4dd0 2023-12-19T18:14:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d2cf96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d281f35f0 con 0x7f2d280b89c0 2023-12-19T18:14:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d26d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d2815cd20 0x7f2d280cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d26d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d2815cd20 0x7f2d280cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45410/0 (socket says 172.21.15.26:45410) 2023-12-19T18:14:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d26d9d640 1 -- 172.21.15.26:0/3131755645 learned_addr learned my addr 172.21.15.26:0/3131755645 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d25d9b640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d280b89c0 0x7f2d28153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d2659c640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d280b4dd0 0x7f2d280ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d26d9d640 1 -- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d280b89c0 msgr2=0x7f2d28153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d26d9d640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d280b89c0 0x7f2d28153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d26d9d640 1 -- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d280b4dd0 msgr2=0x7f2d280ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d26d9d640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d280b4dd0 0x7f2d280ca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d26d9d640 1 -- 172.21.15.26:0/3131755645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d18009420 con 0x7f2d2815cd20 2023-12-19T18:14:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d2659c640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d280b4dd0 0x7f2d280ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d26d9d640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d2815cd20 0x7f2d280cc320 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000cb40 tx=0x7f2d2000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d16ffd640 1 -- 172.21.15.26:0/3131755645 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000aca0 con 0x7f2d2815cd20 2023-12-19T18:14:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d16ffd640 1 -- 172.21.15.26:0/3131755645 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2000ae00 con 0x7f2d2815cd20 2023-12-19T18:14:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d16ffd640 1 -- 172.21.15.26:0/3131755645 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20012620 con 0x7f2d2815cd20 2023-12-19T18:14:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d2cf96640 1 -- 172.21.15.26:0/3131755645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d281f2870 con 0x7f2d2815cd20 2023-12-19T18:14:23.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.780+0000 7f2d2cf96640 1 -- 172.21.15.26:0/3131755645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d280cac40 con 0x7f2d2815cd20 2023-12-19T18:14:23.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.784+0000 7f2d2cf96640 1 -- 172.21.15.26:0/3131755645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d28074270 con 0x7f2d2815cd20 2023-12-19T18:14:23.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.784+0000 7f2d16ffd640 1 -- 172.21.15.26:0/3131755645 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 103) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f2d20007070 con 0x7f2d2815cd20 2023-12-19T18:14:23.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.784+0000 7f2d16ffd640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2d08065b80 0x7f2d08068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:23.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.784+0000 7f2d16ffd640 1 -- 172.21.15.26:0/3131755645 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 7144+0+0 (secure 0 0 0) 0x7f2d20083690 con 0x7f2d2815cd20 2023-12-19T18:14:23.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.784+0000 7f2d2659c640 1 -- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2d08065b80 msgr2=0x7f2d08068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:23.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.784+0000 7f2d2659c640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2d08065b80 0x7f2d08068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:23.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.788+0000 7f2d16ffd640 1 -- 172.21.15.26:0/3131755645 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2d20050a40 con 0x7f2d2815cd20 2023-12-19T18:14:23.831 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:23.828+0000 7fd96f9d4180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:14:23.907 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:23.903+0000 7fdf4b9db180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:14:23.920 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:23.920+0000 7fd96f9d4180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:14:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.944+0000 7f2d2cf96640 1 -- 172.21.15.26:0/3131755645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d280cabe0 con 0x7f2d2815cd20 2023-12-19T18:14:23.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.944+0000 7f2d16ffd640 1 -- 172.21.15.26:0/3131755645 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v183) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d20054060 con 0x7f2d2815cd20 2023-12-19T18:14:23.947 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:14:23.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.948+0000 7f2d2cf96640 1 -- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2d08065b80 msgr2=0x7f2d08068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:14:23.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.948+0000 7f2d2cf96640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2d08065b80 0x7f2d08068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.948+0000 7f2d2cf96640 1 -- 172.21.15.26:0/3131755645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d2815cd20 msgr2=0x7f2d280cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:23.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.948+0000 7f2d2cf96640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d2815cd20 0x7f2d280cc320 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000cb40 tx=0x7f2d2000a060 comp rx=0 tx=0).stop 2023-12-19T18:14:23.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.948+0000 7f2d2cf96640 1 -- 172.21.15.26:0/3131755645 shutdown_connections 2023-12-19T18:14:23.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.948+0000 7f2d2cf96640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2d08065b80 0x7f2d08068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.948+0000 7f2d2cf96640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d2815cd20 0x7f2d280cc320 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.948+0000 7f2d2cf96640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2d280b89c0 0x7f2d28153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.948+0000 7f2d2cf96640 1 --2- 172.21.15.26:0/3131755645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d280b4dd0 0x7f2d280ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:23.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.948+0000 7f2d2cf96640 1 -- 172.21.15.26:0/3131755645 >> 172.21.15.26:0/3131755645 conn(0x7f2d2806a120 msgr2=0x7f2d280b66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:23.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.948+0000 7f2d2cf96640 1 -- 172.21.15.26:0/3131755645 shutdown_connections 2023-12-19T18:14:23.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:23.948+0000 7f2d2cf96640 1 -- 172.21.15.26:0/3131755645 wait complete. 2023-12-19T18:14:23.967 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:23.963+0000 7fdf4b9db180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:14:23.968 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:14:24.045 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:24.043+0000 7fded7152640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:24.045 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:24.043+0000 7fded7152640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:24.045 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:24.043+0000 7fded7152640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:24.045 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:24.043+0000 7fded7152640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:24.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:24.043+0000 7fded7152640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:24.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:24.043+0000 7fded414c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:24.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:24.043+0000 7fded414c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:24.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:24.043+0000 7fded414c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:24.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:24.043+0000 7fded414c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:24.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:24.043+0000 7fded414c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:24.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.164+0000 7f860da0b640 1 Processor -- start 2023-12-19T18:14:24.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.164+0000 7f860da0b640 1 -- start start 2023-12-19T18:14:24.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f860da0b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86080b8460 0x7f86080b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f860da0b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86080b4ef0 con 0x7f8608160330 2023-12-19T18:14:24.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f860da0b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86080d2370 con 0x7f86080b4a90 2023-12-19T18:14:24.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f860da0b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86080d2800 con 0x7f86080b8460 2023-12-19T18:14:24.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f8606ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8608160330 0x7f86080d0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46832/0 (socket says 172.21.15.26:46832) 2023-12-19T18:14:24.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f8606ffd640 1 -- 172.21.15.26:0/2117412612 learned_addr learned my addr 172.21.15.26:0/2117412612 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:24.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f85fffff640 1 --2- 172.21.15.26:0/2117412612 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86080b8460 0x7f86080b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f86067fc640 1 -- 172.21.15.26:0/2117412612 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738312097 0 0) 0x7f86080b4ef0 con 0x7f8608160330 2023-12-19T18:14:24.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f86067fc640 1 -- 172.21.15.26:0/2117412612 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85f4003530 con 0x7f8608160330 2023-12-19T18:14:24.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f86067fc640 1 -- 172.21.15.26:0/2117412612 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717897932 0 0) 0x7f86080d2370 con 0x7f86080b4a90 2023-12-19T18:14:24.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f86067fc640 1 -- 172.21.15.26:0/2117412612 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86080b4ef0 con 0x7f86080b4a90 2023-12-19T18:14:24.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f85fffff640 1 -- 172.21.15.26:0/2117412612 >> v1:172.21.15.160:6789/0 conn(0x7f86080b4a90 legacy=0x7f86080b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f86067fc640 1 -- 172.21.15.26:0/2117412612 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2099808182 0 0) 0x7f85f4003530 con 0x7f8608160330 2023-12-19T18:14:24.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f85fffff640 1 -- 172.21.15.26:0/2117412612 >> v1:172.21.15.26:6789/0 conn(0x7f8608160330 legacy=0x7f86080d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.168+0000 7f85fffff640 1 -- 172.21.15.26:0/2117412612 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86080d1760 con 0x7f86080b8460 2023-12-19T18:14:24.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.172+0000 7f85fffff640 1 --2- 172.21.15.26:0/2117412612 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86080b8460 0x7f86080b4550 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f85f00097f0 tx=0x7f85f001c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=891286779a2c5d30 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:24.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.172+0000 7f86067fc640 1 -- 172.21.15.26:0/2117412612 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8001680 con 0x7f86080b8460 2023-12-19T18:14:24.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.172+0000 7f86067fc640 1 -- 172.21.15.26:0/2117412612 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f00a7050 con 0x7f86080b8460 2023-12-19T18:14:24.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.172+0000 7f86067fc640 1 -- 172.21.15.26:0/2117412612 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f00ab620 con 0x7f86080b8460 2023-12-19T18:14:24.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 -- 172.21.15.26:0/2117412612 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86080b8460 msgr2=0x7f86080b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 --2- 172.21.15.26:0/2117412612 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86080b8460 0x7f86080b4550 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f85f00097f0 tx=0x7f85f001c3b0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 -- 172.21.15.26:0/2117412612 shutdown_connections 2023-12-19T18:14:24.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 --2- 172.21.15.26:0/2117412612 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86080b8460 0x7f86080b4550 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 -- 172.21.15.26:0/2117412612 >> 172.21.15.26:0/2117412612 conn(0x7f8608065ee0 msgr2=0x7f86080cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:24.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 -- 172.21.15.26:0/2117412612 shutdown_connections 2023-12-19T18:14:24.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 -- 172.21.15.26:0/2117412612 wait complete. 2023-12-19T18:14:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 Processor -- start 2023-12-19T18:14:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 -- start start 2023-12-19T18:14:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86080b4a90 0x7f86080c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86080b8460 0x7f86080d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8608160330 0x7f860814b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86080c7ca0 con 0x7f8608160330 2023-12-19T18:14:24.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86080ca330 con 0x7f86080b4a90 2023-12-19T18:14:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f860da0b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86080c76e0 con 0x7f86080b8460 2023-12-19T18:14:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f86077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8608160330 0x7f860814b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f86077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8608160330 0x7f860814b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45422/0 (socket says 172.21.15.26:45422) 2023-12-19T18:14:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f86077fe640 1 -- 172.21.15.26:0/3384449373 learned_addr learned my addr 172.21.15.26:0/3384449373 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f85fffff640 1 --2- 172.21.15.26:0/3384449373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86080b8460 0x7f86080d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f8606ffd640 1 --2- 172.21.15.26:0/3384449373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86080b4a90 0x7f86080c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f86077fe640 1 -- 172.21.15.26:0/3384449373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86080b8460 msgr2=0x7f86080d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f86077fe640 1 --2- 172.21.15.26:0/3384449373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86080b8460 0x7f86080d2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f86077fe640 1 -- 172.21.15.26:0/3384449373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86080b4a90 msgr2=0x7f86080c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f86077fe640 1 --2- 172.21.15.26:0/3384449373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86080b4a90 0x7f86080c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.176+0000 7f86077fe640 1 -- 172.21.15.26:0/3384449373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f0009420 con 0x7f8608160330 2023-12-19T18:14:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.180+0000 7f86077fe640 1 --2- 172.21.15.26:0/3384449373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8608160330 0x7f860814b5f0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f860000c730 tx=0x7f860000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:24.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.180+0000 7f8604ff9640 1 -- 172.21.15.26:0/3384449373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600003bd0 con 0x7f8608160330 2023-12-19T18:14:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.180+0000 7f8604ff9640 1 -- 172.21.15.26:0/3384449373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8600003d30 con 0x7f8608160330 2023-12-19T18:14:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.180+0000 7f860da0b640 1 -- 172.21.15.26:0/3384449373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86080c79c0 con 0x7f8608160330 2023-12-19T18:14:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.180+0000 7f8604ff9640 1 -- 172.21.15.26:0/3384449373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8600003ea0 con 0x7f8608160330 2023-12-19T18:14:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.180+0000 7f860da0b640 1 -- 172.21.15.26:0/3384449373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860814dd80 con 0x7f8608160330 2023-12-19T18:14:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.180+0000 7f860da0b640 1 -- 172.21.15.26:0/3384449373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f860804e850 con 0x7f8608160330 2023-12-19T18:14:24.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.180+0000 7f8604ff9640 1 -- 172.21.15.26:0/3384449373 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 104) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f860000aa90 con 0x7f8608160330 2023-12-19T18:14:24.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.180+0000 7f8604ff9640 1 -- 172.21.15.26:0/3384449373 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f86000830e0 con 0x7f8608160330 2023-12-19T18:14:24.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.188+0000 7f8604ff9640 1 -- 172.21.15.26:0/3384449373 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f86000507e0 con 0x7f8608160330 2023-12-19T18:14:24.274 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:24.272+0000 7fd96f9d4180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:14:24.337 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:24.336+0000 7fd96f9d4180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:14:24.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.360+0000 7f860da0b640 1 -- 172.21.15.26:0/3384449373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86080d2000 con 0x7f8608160330 2023-12-19T18:14:24.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.360+0000 7f8604ff9640 1 -- 172.21.15.26:0/3384449373 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v184) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8600053e00 con 0x7f8608160330 2023-12-19T18:14:24.361 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:14:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.360+0000 7f860da0b640 1 -- 172.21.15.26:0/3384449373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8608160330 msgr2=0x7f860814b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.360+0000 7f860da0b640 1 --2- 172.21.15.26:0/3384449373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8608160330 0x7f860814b5f0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f860000c730 tx=0x7f860000cbf0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.360+0000 7f860da0b640 1 -- 172.21.15.26:0/3384449373 shutdown_connections 2023-12-19T18:14:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.360+0000 7f860da0b640 1 --2- 172.21.15.26:0/3384449373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8608160330 0x7f860814b5f0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.360+0000 7f860da0b640 1 --2- 172.21.15.26:0/3384449373 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86080b8460 0x7f86080d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.360+0000 7f860da0b640 1 --2- 172.21.15.26:0/3384449373 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86080b4a90 0x7f86080c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.360+0000 7f860da0b640 1 -- 172.21.15.26:0/3384449373 >> 172.21.15.26:0/3384449373 conn(0x7f8608065ee0 msgr2=0x7f86080b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.360+0000 7f860da0b640 1 -- 172.21.15.26:0/3384449373 shutdown_connections 2023-12-19T18:14:24.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.360+0000 7f860da0b640 1 -- 172.21.15.26:0/3384449373 wait complete. 2023-12-19T18:14:24.383 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:14:24.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.524+0000 7f8916866640 1 Processor -- start 2023-12-19T18:14:24.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.524+0000 7f8916866640 1 -- start start 2023-12-19T18:14:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f8916866640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f89100cee10 0x7f8910164980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f8916866640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891006bc30 con 0x7f8910154220 2023-12-19T18:14:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f8916866640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89100cc6e0 con 0x7f891006b3d0 2023-12-19T18:14:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f8916866640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891006b950 con 0x7f89100cee10 2023-12-19T18:14:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f890ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8910154220 0x7f891015aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46846/0 (socket says 172.21.15.26:46846) 2023-12-19T18:14:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f890ffff640 1 -- 172.21.15.26:0/760407871 learned_addr learned my addr 172.21.15.26:0/760407871 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f8914e03640 1 --2- 172.21.15.26:0/760407871 >> v2:172.21.15.26:3300/0 conn(0x7f89100cee10 0x7f8910164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f890effd640 1 -- 172.21.15.26:0/760407871 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395408259 0 0) 0x7f891006bc30 con 0x7f8910154220 2023-12-19T18:14:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f890effd640 1 -- 172.21.15.26:0/760407871 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88f8003530 con 0x7f8910154220 2023-12-19T18:14:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f890effd640 1 -- 172.21.15.26:0/760407871 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998537754 0 0) 0x7f89100cc6e0 con 0x7f891006b3d0 2023-12-19T18:14:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f890effd640 1 -- 172.21.15.26:0/760407871 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f891006bc30 con 0x7f891006b3d0 2023-12-19T18:14:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f8914e03640 1 -- 172.21.15.26:0/760407871 >> v1:172.21.15.160:6789/0 conn(0x7f891006b3d0 legacy=0x7f89100ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f8914e03640 1 -- 172.21.15.26:0/760407871 >> v1:172.21.15.26:6789/0 conn(0x7f8910154220 legacy=0x7f891015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.528+0000 7f8914e03640 1 -- 172.21.15.26:0/760407871 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89100cc930 con 0x7f89100cee10 2023-12-19T18:14:24.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8914e03640 1 --2- 172.21.15.26:0/760407871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89100cee10 0x7f8910164980 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f8908009d90 tx=0x7f890801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead0157edb29f6b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:24.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f890effd640 1 -- 172.21.15.26:0/760407871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89080a59e0 con 0x7f89100cee10 2023-12-19T18:14:24.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f890effd640 1 -- 172.21.15.26:0/760407871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89080a5b40 con 0x7f89100cee10 2023-12-19T18:14:24.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f890effd640 1 -- 172.21.15.26:0/760407871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89080ad440 con 0x7f89100cee10 2023-12-19T18:14:24.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 -- 172.21.15.26:0/760407871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89100cee10 msgr2=0x7f8910164980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 --2- 172.21.15.26:0/760407871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89100cee10 0x7f8910164980 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f8908009d90 tx=0x7f890801c920 comp rx=0 tx=0).stop 2023-12-19T18:14:24.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 -- 172.21.15.26:0/760407871 shutdown_connections 2023-12-19T18:14:24.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 --2- 172.21.15.26:0/760407871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89100cee10 0x7f8910164980 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 -- 172.21.15.26:0/760407871 >> 172.21.15.26:0/760407871 conn(0x7f89100658e0 msgr2=0x7f8910065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:24.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 -- 172.21.15.26:0/760407871 shutdown_connections 2023-12-19T18:14:24.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 -- 172.21.15.26:0/760407871 wait complete. 2023-12-19T18:14:24.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 Processor -- start 2023-12-19T18:14:24.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 -- start start 2023-12-19T18:14:24.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f891006b3d0 0x7f89100b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f89100cee10 0x7f891014aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8910154220 0x7f891014b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891014c9d0 con 0x7f891006b3d0 2023-12-19T18:14:24.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891014ba70 con 0x7f8910154220 2023-12-19T18:14:24.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.532+0000 7f8916866640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8910162cd0 con 0x7f89100cee10 2023-12-19T18:14:24.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f891006b3d0 0x7f89100b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f891006b3d0 0x7f89100b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45444/0 (socket says 172.21.15.26:45444) 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890ffff640 1 -- 172.21.15.26:0/2309195536 learned_addr learned my addr 172.21.15.26:0/2309195536 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890f7fe640 1 --2- 172.21.15.26:0/2309195536 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f89100cee10 0x7f891014aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f8914e03640 1 --2- 172.21.15.26:0/2309195536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8910154220 0x7f891014b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890ffff640 1 -- 172.21.15.26:0/2309195536 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f89100cee10 msgr2=0x7f891014aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890ffff640 1 --2- 172.21.15.26:0/2309195536 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f89100cee10 0x7f891014aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890ffff640 1 -- 172.21.15.26:0/2309195536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8910154220 msgr2=0x7f891014b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890ffff640 1 --2- 172.21.15.26:0/2309195536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8910154220 0x7f891014b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890ffff640 1 -- 172.21.15.26:0/2309195536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8908009a70 con 0x7f891006b3d0 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890ffff640 1 --2- 172.21.15.26:0/2309195536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f891006b3d0 0x7f89100b16c0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f890000c450 tx=0x7f890000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890cff9640 1 -- 172.21.15.26:0/2309195536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8900005cf0 con 0x7f891006b3d0 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890cff9640 1 -- 172.21.15.26:0/2309195536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8900005e50 con 0x7f891006b3d0 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890cff9640 1 -- 172.21.15.26:0/2309195536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8900007270 con 0x7f891006b3d0 2023-12-19T18:14:24.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f8916866640 1 -- 172.21.15.26:0/2309195536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8910162fb0 con 0x7f891006b3d0 2023-12-19T18:14:24.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f8916866640 1 -- 172.21.15.26:0/2309195536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8910163570 con 0x7f891006b3d0 2023-12-19T18:14:24.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f8916866640 1 -- 172.21.15.26:0/2309195536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8910163790 con 0x7f891006b3d0 2023-12-19T18:14:24.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.536+0000 7f890cff9640 1 -- 172.21.15.26:0/2309195536 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 104) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f89000073d0 con 0x7f891006b3d0 2023-12-19T18:14:24.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.540+0000 7f890cff9640 1 -- 172.21.15.26:0/2309195536 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f8900082820 con 0x7f891006b3d0 2023-12-19T18:14:24.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.540+0000 7f890cff9640 1 -- 172.21.15.26:0/2309195536 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f890004ff20 con 0x7f891006b3d0 2023-12-19T18:14:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.732+0000 7f8916866640 1 -- 172.21.15.26:0/2309195536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8910074270 con 0x7f891006b3d0 2023-12-19T18:14:24.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.732+0000 7f890cff9640 1 -- 172.21.15.26:0/2309195536 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f8900017090 con 0x7f891006b3d0 2023-12-19T18:14:24.734 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:24.734 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2924688599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2924688599},{"type":"v1","addr":"172.21.15.26:6835","nonce":2924688599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/330338341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":330338341},{"type":"v1","addr":"172.21.15.160:6836","nonce":330338341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9031,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/887954085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":887954085},{"type":"v1","addr":"172.21.15.160:6837","nonce":887954085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:58:13.473962+0000","modified":"2023-12-19T17:58:16.484662+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9513},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9513":{"gid":9513,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/3354708287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3354708287},{"type":"v1","addr":"172.21.15.26:6834","nonce":3354708287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-19T18:14:24.734 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 80 2023-12-19T18:14:24.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.736+0000 7f8916866640 1 -- 172.21.15.26:0/2309195536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f891006b3d0 msgr2=0x7f89100b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.736+0000 7f8916866640 1 --2- 172.21.15.26:0/2309195536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f891006b3d0 0x7f89100b16c0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f890000c450 tx=0x7f890000c910 comp rx=0 tx=0).stop 2023-12-19T18:14:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.736+0000 7f8916866640 1 -- 172.21.15.26:0/2309195536 shutdown_connections 2023-12-19T18:14:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.736+0000 7f8916866640 1 --2- 172.21.15.26:0/2309195536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8910154220 0x7f891014b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.736+0000 7f8916866640 1 --2- 172.21.15.26:0/2309195536 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f89100cee10 0x7f891014aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.736+0000 7f8916866640 1 --2- 172.21.15.26:0/2309195536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f891006b3d0 0x7f89100b16c0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.736+0000 7f8916866640 1 -- 172.21.15.26:0/2309195536 >> 172.21.15.26:0/2309195536 conn(0x7f89100658e0 msgr2=0x7f8910065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.736+0000 7f8916866640 1 -- 172.21.15.26:0/2309195536 shutdown_connections 2023-12-19T18:14:24.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.736+0000 7f8916866640 1 -- 172.21.15.26:0/2309195536 wait complete. 2023-12-19T18:14:24.755 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.896+0000 7f748ca55640 1 Processor -- start 2023-12-19T18:14:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.896+0000 7f748ca55640 1 -- start start 2023-12-19T18:14:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f748ca55640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f74880cb1d0 0x7f74880cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f748ca55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74880b1d40 0x7f74880b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f748ca55640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748806d4b0 con 0x7f748806bee0 2023-12-19T18:14:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f748ca55640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74880cc030 con 0x7f74880cb1d0 2023-12-19T18:14:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f748ca55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74880cbaf0 con 0x7f74880b1d40 2023-12-19T18:14:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f748659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f74880cb1d0 0x7f74880cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f748659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f74880cb1d0 0x7f74880cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45454/0 (socket says 172.21.15.26:45454) 2023-12-19T18:14:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f748659c640 1 -- 172.21.15.26:0/1694621822 learned_addr learned my addr 172.21.15.26:0/1694621822 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f7486d9d640 1 --2- 172.21.15.26:0/1694621822 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74880b1d40 0x7f74880b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f748659c640 1 -- 172.21.15.26:0/1694621822 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74880b1d40 msgr2=0x7f74880b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f748659c640 1 --2- 172.21.15.26:0/1694621822 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74880b1d40 0x7f74880b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f748659c640 1 -- 172.21.15.26:0/1694621822 >> v1:172.21.15.26:6789/0 conn(0x7f748806bee0 legacy=0x7f748806c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f748659c640 1 -- 172.21.15.26:0/1694621822 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748806cc10 con 0x7f74880cb1d0 2023-12-19T18:14:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.900+0000 7f748659c640 1 --2- 172.21.15.26:0/1694621822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74880cb1d0 0x7f74880cb5b0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7478009870 tx=0x7f747801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8d281c8857f077 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f7485d9b640 1 -- 172.21.15.26:0/1694621822 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74780aa610 con 0x7f74880cb1d0 2023-12-19T18:14:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f7485d9b640 1 -- 172.21.15.26:0/1694621822 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74780a6040 con 0x7f74880cb1d0 2023-12-19T18:14:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f7485d9b640 1 -- 172.21.15.26:0/1694621822 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74780aa990 con 0x7f74880cb1d0 2023-12-19T18:14:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 -- 172.21.15.26:0/1694621822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74880cb1d0 msgr2=0x7f74880cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 --2- 172.21.15.26:0/1694621822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74880cb1d0 0x7f74880cb5b0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f7478009870 tx=0x7f747801c4b0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 -- 172.21.15.26:0/1694621822 shutdown_connections 2023-12-19T18:14:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 --2- 172.21.15.26:0/1694621822 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74880b1d40 0x7f74880b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 --2- 172.21.15.26:0/1694621822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f74880cb1d0 0x7f74880cb5b0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 -- 172.21.15.26:0/1694621822 >> 172.21.15.26:0/1694621822 conn(0x7f74880658e0 msgr2=0x7f7488065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 -- 172.21.15.26:0/1694621822 shutdown_connections 2023-12-19T18:14:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 -- 172.21.15.26:0/1694621822 wait complete. 2023-12-19T18:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 Processor -- start 2023-12-19T18:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 -- start start 2023-12-19T18:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f748806bee0 0x7f748814dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74880b1d40 0x7f7488162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74880cb1d0 0x7f7488163230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748814e140 con 0x7f748806bee0 2023-12-19T18:14:24.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7488158d60 con 0x7f74880cb1d0 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748ca55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748814f5f0 con 0x7f74880b1d40 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.904+0000 7f748659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f748806bee0 0x7f748814dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f748659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f748806bee0 0x7f748814dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45460/0 (socket says 172.21.15.26:45460) 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f748659c640 1 -- 172.21.15.26:0/2657821106 learned_addr learned my addr 172.21.15.26:0/2657821106 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f747ffff640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74880b1d40 0x7f7488162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f7486d9d640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74880cb1d0 0x7f7488163230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f748659c640 1 -- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74880b1d40 msgr2=0x7f7488162cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f748659c640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74880b1d40 0x7f7488162cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f748659c640 1 -- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74880cb1d0 msgr2=0x7f7488163230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f748659c640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74880cb1d0 0x7f7488163230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f748659c640 1 -- 172.21.15.26:0/2657821106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74780094d0 con 0x7f748806bee0 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f748659c640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f748806bee0 0x7f748814dc00 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f74780ac040 tx=0x7f747801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f747effd640 1 -- 172.21.15.26:0/2657821106 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74780bd5f0 con 0x7f748806bee0 2023-12-19T18:14:24.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f747effd640 1 -- 172.21.15.26:0/2657821106 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74780bd750 con 0x7f748806bee0 2023-12-19T18:14:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f747effd640 1 -- 172.21.15.26:0/2657821106 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74780bdb00 con 0x7f748806bee0 2023-12-19T18:14:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f748ca55640 1 -- 172.21.15.26:0/2657821106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74880ce600 con 0x7f748806bee0 2023-12-19T18:14:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f748ca55640 1 -- 172.21.15.26:0/2657821106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74880cebc0 con 0x7f748806bee0 2023-12-19T18:14:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f748ca55640 1 -- 172.21.15.26:0/2657821106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7488074270 con 0x7f748806bee0 2023-12-19T18:14:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f747effd640 1 -- 172.21.15.26:0/2657821106 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 104) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f74780b33f0 con 0x7f748806bee0 2023-12-19T18:14:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.908+0000 7f747effd640 1 -- 172.21.15.26:0/2657821106 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f747811def0 con 0x7f748806bee0 2023-12-19T18:14:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.912+0000 7f747effd640 1 -- 172.21.15.26:0/2657821106 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f74780eb710 con 0x7f748806bee0 2023-12-19T18:14:24.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.984+0000 7f747effd640 1 -- 172.21.15.26:0/2657821106 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 105) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f74780eb1c0 con 0x7f748806bee0 2023-12-19T18:14:24.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.984+0000 7f747effd640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f746801c9e0 0x7f746806dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:24.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.984+0000 7f747ffff640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f746801c9e0 0x7f746806dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:24.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:24.988+0000 7f747ffff640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f746801c9e0 0x7f746806dd90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7488157910 tx=0x7f7470009040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:25.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.076+0000 7f748ca55640 1 -- 172.21.15.26:0/2657821106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74880cf710 con 0x7f748806bee0 2023-12-19T18:14:25.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.080+0000 7f747effd640 1 -- 172.21.15.26:0/2657821106 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+18709 (secure 0 0 0) 0x7f74780c0c70 con 0x7f748806bee0 2023-12-19T18:14:25.080 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:25.080 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":184,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:14:23.971783+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:58:12.042476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T17:58:12.457937+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":183,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.29","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/986493927":"2023-12-20T18:14:23.971750+0000","172.21.15.160:0/1845619584":"2023-12-20T18:14:23.971750+0000","172.21.15.160:0/387998771":"2023-12-20T18:14:23.971750+0000","172.21.15.160:0/2094392597":"2023-12-20T18:14:23.971750+0000","172.21.15.160:6832/48394":"2023-12-20T18:14:23.971750+0000","172.21.15.160:0/3381018054":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/3458963623":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/1302880583":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/1063410185":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/819581921":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/3919731969":"2023-12-20T18:14:23.971750+0000","172.21.15.160:0/4076846554":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/3832178910":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/317596538":"2023-12-20T18:14:23.971750+0000","172.21.15.160:6833/38098":"2023-12-20T17:58:25.501946+0000","172.21.15.160:6832/38098":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/1419214494":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/1333176784":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/4141924506":"2023-12-20T17:58:25.501946+0000","172.21.15.160:6833/48394":"2023-12-20T18:14:23.971750+0000","172.21.15.160:0/2994631753":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/2198643724":"2023-12-20T17:58:25.501946+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:25.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.080+0000 7f748ca55640 1 -- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f746801c9e0 msgr2=0x7f746806dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.080+0000 7f748ca55640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f746801c9e0 0x7f746806dd90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7488157910 tx=0x7f7470009040 comp rx=0 tx=0).stop 2023-12-19T18:14:25.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.080+0000 7f748ca55640 1 -- 172.21.15.26:0/2657821106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f748806bee0 msgr2=0x7f748814dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.080+0000 7f748ca55640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f748806bee0 0x7f748814dc00 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f74780ac040 tx=0x7f747801acf0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.080+0000 7f748ca55640 1 -- 172.21.15.26:0/2657821106 shutdown_connections 2023-12-19T18:14:25.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.080+0000 7f748ca55640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f746801c9e0 0x7f746806dd90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.080+0000 7f748ca55640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f74880cb1d0 0x7f7488163230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.080+0000 7f748ca55640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f74880b1d40 0x7f7488162cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.080+0000 7f748ca55640 1 --2- 172.21.15.26:0/2657821106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f748806bee0 0x7f748814dc00 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.080+0000 7f748ca55640 1 -- 172.21.15.26:0/2657821106 >> 172.21.15.26:0/2657821106 conn(0x7f74880658e0 msgr2=0x7f7488151840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:25.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.080+0000 7f748ca55640 1 -- 172.21.15.26:0/2657821106 shutdown_connections 2023-12-19T18:14:25.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.080+0000 7f748ca55640 1 -- 172.21.15.26:0/2657821106 wait complete. 2023-12-19T18:14:25.101 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:14:25.102 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:14:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.244+0000 7f1a08066640 1 Processor -- start 2023-12-19T18:14:25.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.244+0000 7f1a08066640 1 -- start start 2023-12-19T18:14:25.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a08066640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1a00154220 0x7f1a0015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a08066640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a0006b3d0 0x7f1a000ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a08066640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a0006bc30 con 0x7f1a000cec40 2023-12-19T18:14:25.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a08066640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a000cf0d0 con 0x7f1a0006b3d0 2023-12-19T18:14:25.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a08066640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a000cc6e0 con 0x7f1a00154220 2023-12-19T18:14:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a05601640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a0006b3d0 0x7f1a000ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a06603640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1a000cec40 0x7f1a0015d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46860/0 (socket says 172.21.15.26:46860) 2023-12-19T18:14:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a05601640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a0006b3d0 0x7f1a000ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45476/0 (socket says 172.21.15.26:45476) 2023-12-19T18:14:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a06603640 1 -- 172.21.15.26:0/2191116721 learned_addr learned my addr 172.21.15.26:0/2191116721 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a05e02640 1 --2- 172.21.15.26:0/2191116721 >> v2:172.21.15.160:3300/0 conn(0x7f1a00154220 0x7f1a0015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a05601640 1 -- 172.21.15.26:0/2191116721 >> v2:172.21.15.160:3300/0 conn(0x7f1a00154220 msgr2=0x7f1a0015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a05601640 1 --2- 172.21.15.26:0/2191116721 >> v2:172.21.15.160:3300/0 conn(0x7f1a00154220 0x7f1a0015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a05601640 1 -- 172.21.15.26:0/2191116721 >> v1:172.21.15.26:6789/0 conn(0x7f1a000cec40 legacy=0x7f1a0015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.248+0000 7f1a05601640 1 -- 172.21.15.26:0/2191116721 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0015afc0 con 0x7f1a0006b3d0 2023-12-19T18:14:25.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a05601640 1 --2- 172.21.15.26:0/2191116721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a0006b3d0 0x7f1a000ce700 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f1a000667b0 tx=0x7f19f001eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45004c65192f1192 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:25.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a04e00640 1 -- 172.21.15.26:0/2191116721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f00a7af0 con 0x7f1a0006b3d0 2023-12-19T18:14:25.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a04e00640 1 -- 172.21.15.26:0/2191116721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f00a7c50 con 0x7f1a0006b3d0 2023-12-19T18:14:25.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a04e00640 1 -- 172.21.15.26:0/2191116721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f00af440 con 0x7f1a0006b3d0 2023-12-19T18:14:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a08066640 1 -- 172.21.15.26:0/2191116721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a0006b3d0 msgr2=0x7f1a000ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a08066640 1 --2- 172.21.15.26:0/2191116721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a0006b3d0 0x7f1a000ce700 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f1a000667b0 tx=0x7f19f001eb00 comp rx=0 tx=0).stop 2023-12-19T18:14:25.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a08066640 1 -- 172.21.15.26:0/2191116721 shutdown_connections 2023-12-19T18:14:25.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a08066640 1 --2- 172.21.15.26:0/2191116721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a0006b3d0 0x7f1a000ce700 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a08066640 1 --2- 172.21.15.26:0/2191116721 >> v2:172.21.15.160:3300/0 conn(0x7f1a00154220 0x7f1a0015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a08066640 1 -- 172.21.15.26:0/2191116721 >> 172.21.15.26:0/2191116721 conn(0x7f1a000658e0 msgr2=0x7f1a00065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:25.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a08066640 1 -- 172.21.15.26:0/2191116721 shutdown_connections 2023-12-19T18:14:25.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a08066640 1 -- 172.21.15.26:0/2191116721 wait complete. 2023-12-19T18:14:25.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a08066640 1 Processor -- start 2023-12-19T18:14:25.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.252+0000 7f1a08066640 1 -- start start 2023-12-19T18:14:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a08066640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a0006b3d0 0x7f1a00151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a08066640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a000cec40 0x7f1a001524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a08066640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a00154220 0x7f1a00070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a08066640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0006f760 con 0x7f1a00154220 2023-12-19T18:14:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a08066640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0006fa40 con 0x7f1a0006b3d0 2023-12-19T18:14:25.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a08066640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0006fd20 con 0x7f1a000cec40 2023-12-19T18:14:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a06603640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a00154220 0x7f1a00070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a06603640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a00154220 0x7f1a00070800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45482/0 (socket says 172.21.15.26:45482) 2023-12-19T18:14:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a06603640 1 -- 172.21.15.26:0/3589118981 learned_addr learned my addr 172.21.15.26:0/3589118981 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a05601640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a000cec40 0x7f1a001524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a05e02640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a0006b3d0 0x7f1a00151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a06603640 1 -- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a000cec40 msgr2=0x7f1a001524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a06603640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a000cec40 0x7f1a001524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a06603640 1 -- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a0006b3d0 msgr2=0x7f1a00151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a06603640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a0006b3d0 0x7f1a00151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a06603640 1 -- 172.21.15.26:0/3589118981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f000b6c0 con 0x7f1a00154220 2023-12-19T18:14:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a06603640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a00154220 0x7f1a00070800 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f19f800ca80 tx=0x7f19f800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:25.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f19f67fc640 1 -- 172.21.15.26:0/3589118981 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f800ac30 con 0x7f1a00154220 2023-12-19T18:14:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f19f67fc640 1 -- 172.21.15.26:0/3589118981 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f800ad90 con 0x7f1a00154220 2023-12-19T18:14:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f19f67fc640 1 -- 172.21.15.26:0/3589118981 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8012600 con 0x7f1a00154220 2023-12-19T18:14:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a08066640 1 -- 172.21.15.26:0/3589118981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a00070000 con 0x7f1a00154220 2023-12-19T18:14:25.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a08066640 1 -- 172.21.15.26:0/3589118981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0013c5f0 con 0x7f1a00154220 2023-12-19T18:14:25.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f1a08066640 1 -- 172.21.15.26:0/3589118981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a00074270 con 0x7f1a00154220 2023-12-19T18:14:25.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.256+0000 7f19f67fc640 1 -- 172.21.15.26:0/3589118981 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 105) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f19f8007070 con 0x7f1a00154220 2023-12-19T18:14:25.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.260+0000 7f19f67fc640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f19e40658b0 0x7f19e4067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.260+0000 7f19f67fc640 1 -- 172.21.15.26:0/3589118981 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f19f80833a0 con 0x7f1a00154220 2023-12-19T18:14:25.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.260+0000 7f1a05e02640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f19e40658b0 0x7f19e4067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.260+0000 7f1a05e02640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f19e40658b0 0x7f19e4067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f19fc007a60 tx=0x7f19fc008040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.264+0000 7f19f67fc640 1 -- 172.21.15.26:0/3589118981 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f19f80509b0 con 0x7f1a00154220 2023-12-19T18:14:25.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.452+0000 7f1a08066640 1 -- 172.21.15.26:0/3589118981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a000704b0 con 0x7f1a00154220 2023-12-19T18:14:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.456+0000 7f19f67fc640 1 -- 172.21.15.26:0/3589118981 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f19f8017090 con 0x7f1a00154220 2023-12-19T18:14:25.455 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:25.455 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2924688599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2924688599},{"type":"v1","addr":"172.21.15.26:6835","nonce":2924688599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/330338341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":330338341},{"type":"v1","addr":"172.21.15.160:6836","nonce":330338341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9031,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/887954085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":887954085},{"type":"v1","addr":"172.21.15.160:6837","nonce":887954085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:58:13.473962+0000","modified":"2023-12-19T17:58:16.484662+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9513},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9513":{"gid":9513,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/3354708287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3354708287},{"type":"v1","addr":"172.21.15.26:6834","nonce":3354708287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-19T18:14:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 80 2023-12-19T18:14:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.456+0000 7f1a08066640 1 -- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f19e40658b0 msgr2=0x7f19e4067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.456+0000 7f1a08066640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f19e40658b0 0x7f19e4067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f19fc007a60 tx=0x7f19fc008040 comp rx=0 tx=0).stop 2023-12-19T18:14:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.456+0000 7f1a08066640 1 -- 172.21.15.26:0/3589118981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a00154220 msgr2=0x7f1a00070800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.456+0000 7f1a08066640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a00154220 0x7f1a00070800 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f19f800ca80 tx=0x7f19f800cf40 comp rx=0 tx=0).stop 2023-12-19T18:14:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.456+0000 7f1a08066640 1 -- 172.21.15.26:0/3589118981 shutdown_connections 2023-12-19T18:14:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.456+0000 7f1a08066640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f19e40658b0 0x7f19e4067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.456+0000 7f1a08066640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a00154220 0x7f1a00070800 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.456+0000 7f1a08066640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1a000cec40 0x7f1a001524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.456+0000 7f1a08066640 1 --2- 172.21.15.26:0/3589118981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1a0006b3d0 0x7f1a00151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.456+0000 7f1a08066640 1 -- 172.21.15.26:0/3589118981 >> 172.21.15.26:0/3589118981 conn(0x7f1a000658e0 msgr2=0x7f1a0015c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.456+0000 7f1a08066640 1 -- 172.21.15.26:0/3589118981 shutdown_connections 2023-12-19T18:14:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.456+0000 7f1a08066640 1 -- 172.21.15.26:0/3589118981 wait complete. 2023-12-19T18:14:25.477 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:14:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f313f48c640 1 Processor -- start 2023-12-19T18:14:25.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f313f48c640 1 -- start start 2023-12-19T18:14:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f313f48c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f31380d3300 0x7f31380d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f313f48c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31380d29d0 con 0x7f31380ce040 2023-12-19T18:14:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f313f48c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313815d680 con 0x7f313815cd40 2023-12-19T18:14:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f313f48c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31380ce4b0 con 0x7f31380d3300 2023-12-19T18:14:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f313ca27640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f31380ce040 0x7f31380d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46876/0 (socket says 172.21.15.26:46876) 2023-12-19T18:14:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f313ca27640 1 -- 172.21.15.26:0/2923476393 learned_addr learned my addr 172.21.15.26:0/2923476393 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f313da29640 1 --2- 172.21.15.26:0/2923476393 >> v2:172.21.15.26:3300/0 conn(0x7f31380d3300 0x7f31380d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f312ffff640 1 -- 172.21.15.26:0/2923476393 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128424634 0 0) 0x7f3128001120 con 0x7f31380ce040 2023-12-19T18:14:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f312ffff640 1 -- 172.21.15.26:0/2923476393 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3124003530 con 0x7f31380ce040 2023-12-19T18:14:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f313da29640 1 -- 172.21.15.26:0/2923476393 >> v1:172.21.15.160:6789/0 conn(0x7f313815cd40 legacy=0x7f3138160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f313da29640 1 -- 172.21.15.26:0/2923476393 >> v1:172.21.15.26:6789/0 conn(0x7f31380ce040 legacy=0x7f31380d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.620+0000 7f313da29640 1 -- 172.21.15.26:0/2923476393 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3138161750 con 0x7f31380d3300 2023-12-19T18:14:25.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.624+0000 7f313da29640 1 --2- 172.21.15.26:0/2923476393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31380d3300 0x7f31380d0e30 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f3130009d90 tx=0x7f313001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c903e58399960131 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:25.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.624+0000 7f312ffff640 1 -- 172.21.15.26:0/2923476393 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31300a59e0 con 0x7f31380d3300 2023-12-19T18:14:25.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.624+0000 7f312ffff640 1 -- 172.21.15.26:0/2923476393 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31300a5b40 con 0x7f31380d3300 2023-12-19T18:14:25.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.624+0000 7f312ffff640 1 -- 172.21.15.26:0/2923476393 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31300ad440 con 0x7f31380d3300 2023-12-19T18:14:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.624+0000 7f313f48c640 1 -- 172.21.15.26:0/2923476393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31380d3300 msgr2=0x7f31380d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.624+0000 7f313f48c640 1 --2- 172.21.15.26:0/2923476393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31380d3300 0x7f31380d0e30 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f3130009d90 tx=0x7f313001c920 comp rx=0 tx=0).stop 2023-12-19T18:14:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.624+0000 7f313f48c640 1 -- 172.21.15.26:0/2923476393 shutdown_connections 2023-12-19T18:14:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.624+0000 7f313f48c640 1 --2- 172.21.15.26:0/2923476393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31380d3300 0x7f31380d0e30 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.624+0000 7f313f48c640 1 -- 172.21.15.26:0/2923476393 >> 172.21.15.26:0/2923476393 conn(0x7f313806a120 msgr2=0x7f313806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.624+0000 7f313f48c640 1 -- 172.21.15.26:0/2923476393 shutdown_connections 2023-12-19T18:14:25.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.624+0000 7f313f48c640 1 -- 172.21.15.26:0/2923476393 wait complete. 2023-12-19T18:14:25.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313f48c640 1 Processor -- start 2023-12-19T18:14:25.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313f48c640 1 -- start start 2023-12-19T18:14:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313f48c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31380ce040 0x7f3138158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313f48c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31380d3300 0x7f31380c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313f48c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f313815cd40 0x7f313815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313f48c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3138157340 con 0x7f31380d3300 2023-12-19T18:14:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313f48c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3138155b20 con 0x7f31380ce040 2023-12-19T18:14:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313f48c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31380c8cd0 con 0x7f313815cd40 2023-12-19T18:14:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313ca27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31380d3300 0x7f31380c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313ca27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31380d3300 0x7f31380c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:45500/0 (socket says 172.21.15.26:45500) 2023-12-19T18:14:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313ca27640 1 -- 172.21.15.26:0/1134427039 learned_addr learned my addr 172.21.15.26:0/1134427039 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:25.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313d228640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31380ce040 0x7f3138158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313da29640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f313815cd40 0x7f313815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313ca27640 1 -- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f313815cd40 msgr2=0x7f313815dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313ca27640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f313815cd40 0x7f313815dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313ca27640 1 -- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31380ce040 msgr2=0x7f3138158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313ca27640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31380ce040 0x7f3138158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313ca27640 1 -- 172.21.15.26:0/1134427039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3130009a70 con 0x7f31380d3300 2023-12-19T18:14:25.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313da29640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f313815cd40 0x7f313815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:25.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313ca27640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31380d3300 0x7f31380c9d30 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f312800c4d0 tx=0x7f312800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:25.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f312dffb640 1 -- 172.21.15.26:0/1134427039 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128005cf0 con 0x7f31380d3300 2023-12-19T18:14:25.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f312dffb640 1 -- 172.21.15.26:0/1134427039 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3128005e50 con 0x7f31380d3300 2023-12-19T18:14:25.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313f48c640 1 -- 172.21.15.26:0/1134427039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31380c8fb0 con 0x7f31380d3300 2023-12-19T18:14:25.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f312dffb640 1 -- 172.21.15.26:0/1134427039 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3128007290 con 0x7f31380d3300 2023-12-19T18:14:25.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.628+0000 7f313f48c640 1 -- 172.21.15.26:0/1134427039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3138156ed0 con 0x7f31380d3300 2023-12-19T18:14:25.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.632+0000 7f313f48c640 1 -- 172.21.15.26:0/1134427039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31381574f0 con 0x7f31380d3300 2023-12-19T18:14:25.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.632+0000 7f312dffb640 1 -- 172.21.15.26:0/1134427039 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 105) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f31280073f0 con 0x7f31380d3300 2023-12-19T18:14:25.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.632+0000 7f312dffb640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f311c0658b0 0x7f311c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.632+0000 7f312dffb640 1 -- 172.21.15.26:0/1134427039 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f31280828e0 con 0x7f31380d3300 2023-12-19T18:14:25.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.632+0000 7f313d228640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f311c0658b0 0x7f311c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.632+0000 7f313d228640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f311c0658b0 0x7f311c067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3134001740 tx=0x7f3134009040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:25.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.636+0000 7f312dffb640 1 -- 172.21.15.26:0/1134427039 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f31381574f0 con 0x7f31380d3300 2023-12-19T18:14:25.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f313f48c640 1 -- 172.21.15.26:0/1134427039 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3138074270 con 0x7f31380d3300 2023-12-19T18:14:25.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f312dffb640 1 -- 172.21.15.26:0/1134427039 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f3128017090 con 0x7f31380d3300 2023-12-19T18:14:25.828 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:25.828 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":80,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2924688599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2924688599},{"type":"v1","addr":"172.21.15.26:6835","nonce":2924688599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/330338341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":330338341},{"type":"v1","addr":"172.21.15.160:6836","nonce":330338341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9031,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/887954085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":887954085},{"type":"v1","addr":"172.21.15.160:6837","nonce":887954085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T17:58:13.473962+0000","modified":"2023-12-19T17:58:16.484662+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9513},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9513":{"gid":9513,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6834/3354708287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3354708287},{"type":"v1","addr":"172.21.15.26:6834","nonce":3354708287}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-19T18:14:25.828 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 80 2023-12-19T18:14:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f313f48c640 1 -- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f311c0658b0 msgr2=0x7f311c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f313f48c640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f311c0658b0 0x7f311c067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3134001740 tx=0x7f3134009040 comp rx=0 tx=0).stop 2023-12-19T18:14:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f313f48c640 1 -- 172.21.15.26:0/1134427039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31380d3300 msgr2=0x7f31380c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f313f48c640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31380d3300 0x7f31380c9d30 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f312800c4d0 tx=0x7f312800c990 comp rx=0 tx=0).stop 2023-12-19T18:14:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f313f48c640 1 -- 172.21.15.26:0/1134427039 shutdown_connections 2023-12-19T18:14:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f313f48c640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f311c0658b0 0x7f311c067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f313f48c640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f313815cd40 0x7f313815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f313f48c640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f31380d3300 0x7f31380c9d30 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f313f48c640 1 --2- 172.21.15.26:0/1134427039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31380ce040 0x7f3138158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f313f48c640 1 -- 172.21.15.26:0/1134427039 >> 172.21.15.26:0/1134427039 conn(0x7f313806a120 msgr2=0x7f31380b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:25.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f313f48c640 1 -- 172.21.15.26:0/1134427039 shutdown_connections 2023-12-19T18:14:25.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.828+0000 7f313f48c640 1 -- 172.21.15.26:0/1134427039 wait complete. 2023-12-19T18:14:25.849 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.992+0000 7f0818ba6640 1 Processor -- start 2023-12-19T18:14:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.992+0000 7f0818ba6640 1 -- start start 2023-12-19T18:14:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f0818ba6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f081415c7f0 0x7f081406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f0818ba6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f08140adb80 0x7f08140cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f0818ba6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08140ccc20 0x7f08140b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f0818ba6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081406d770 con 0x7f081415c7f0 2023-12-19T18:14:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f0818ba6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081406ced0 con 0x7f08140adb80 2023-12-19T18:14:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f0818ba6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081406d1b0 con 0x7f08140ccc20 2023-12-19T18:14:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f081259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f081415c7f0 0x7f081406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f081259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f081415c7f0 0x7f081406c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38892/0 (socket says 172.21.15.26:38892) 2023-12-19T18:14:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f081259c640 1 -- 172.21.15.26:0/1154582599 learned_addr learned my addr 172.21.15.26:0/1154582599 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f0812d9d640 1 --2- 172.21.15.26:0/1154582599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08140ccc20 0x7f08140b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f081259c640 1 -- 172.21.15.26:0/1154582599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08140ccc20 msgr2=0x7f08140b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f080bfff640 1 --2- 172.21.15.26:0/1154582599 >> v2:172.21.15.160:3300/0 conn(0x7f08140adb80 0x7f08140cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f081259c640 1 --2- 172.21.15.26:0/1154582599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08140ccc20 0x7f08140b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f081259c640 1 -- 172.21.15.26:0/1154582599 >> v2:172.21.15.160:3300/0 conn(0x7f08140adb80 msgr2=0x7f08140cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f081259c640 1 --2- 172.21.15.26:0/1154582599 >> v2:172.21.15.160:3300/0 conn(0x7f08140adb80 0x7f08140cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f081259c640 1 -- 172.21.15.26:0/1154582599 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081406d490 con 0x7f081415c7f0 2023-12-19T18:14:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f0812d9d640 1 --2- 172.21.15.26:0/1154582599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08140ccc20 0x7f08140b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:25.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f080bfff640 1 --2- 172.21.15.26:0/1154582599 >> v2:172.21.15.160:3300/0 conn(0x7f08140adb80 0x7f08140cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:25.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:25.996+0000 7f081259c640 1 --2- 172.21.15.26:0/1154582599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f081415c7f0 0x7f081406c7f0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f08140667b0 tx=0x7f080401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefa7383bd846f2f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0811d9b640 1 -- 172.21.15.26:0/1154582599 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08040ab600 con 0x7f081415c7f0 2023-12-19T18:14:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0811d9b640 1 -- 172.21.15.26:0/1154582599 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08040a7050 con 0x7f081415c7f0 2023-12-19T18:14:25.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0811d9b640 1 -- 172.21.15.26:0/1154582599 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08040ab980 con 0x7f081415c7f0 2023-12-19T18:14:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 -- 172.21.15.26:0/1154582599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f081415c7f0 msgr2=0x7f081406c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 --2- 172.21.15.26:0/1154582599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f081415c7f0 0x7f081406c7f0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f08140667b0 tx=0x7f080401c500 comp rx=0 tx=0).stop 2023-12-19T18:14:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 -- 172.21.15.26:0/1154582599 shutdown_connections 2023-12-19T18:14:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 --2- 172.21.15.26:0/1154582599 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08140ccc20 0x7f08140b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 --2- 172.21.15.26:0/1154582599 >> v2:172.21.15.160:3300/0 conn(0x7f08140adb80 0x7f08140cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 --2- 172.21.15.26:0/1154582599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f081415c7f0 0x7f081406c7f0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 -- 172.21.15.26:0/1154582599 >> 172.21.15.26:0/1154582599 conn(0x7f08140658e0 msgr2=0x7f0814065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 -- 172.21.15.26:0/1154582599 shutdown_connections 2023-12-19T18:14:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 -- 172.21.15.26:0/1154582599 wait complete. 2023-12-19T18:14:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 Processor -- start 2023-12-19T18:14:26.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 -- start start 2023-12-19T18:14:26.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08140adb80 0x7f08141486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08140ccc20 0x7f0814153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f081415c7f0 0x7f0814153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081414f190 con 0x7f08140ccc20 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081414b240 con 0x7f08140adb80 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.000+0000 7f0818ba6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081414b520 con 0x7f081415c7f0 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08140ccc20 0x7f0814153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08140ccc20 0x7f0814153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38894/0 (socket says 172.21.15.26:38894) 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080bfff640 1 -- 172.21.15.26:0/444450105 learned_addr learned my addr 172.21.15.26:0/444450105 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f081259c640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08140adb80 0x7f08141486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f0812d9d640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f081415c7f0 0x7f0814153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080bfff640 1 -- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f081415c7f0 msgr2=0x7f0814153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080bfff640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f081415c7f0 0x7f0814153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080bfff640 1 -- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08140adb80 msgr2=0x7f08141486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080bfff640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08140adb80 0x7f08141486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080bfff640 1 -- 172.21.15.26:0/444450105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08040094d0 con 0x7f08140ccc20 2023-12-19T18:14:26.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080bfff640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08140ccc20 0x7f0814153550 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00c7c0 tx=0x7f07fc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080affd640 1 -- 172.21.15.26:0/444450105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc007a30 con 0x7f08140ccc20 2023-12-19T18:14:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080affd640 1 -- 172.21.15.26:0/444450105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc007b90 con 0x7f08140ccc20 2023-12-19T18:14:26.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080affd640 1 -- 172.21.15.26:0/444450105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00ce70 con 0x7f08140ccc20 2023-12-19T18:14:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f0818ba6640 1 -- 172.21.15.26:0/444450105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081414ead0 con 0x7f08140ccc20 2023-12-19T18:14:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f0818ba6640 1 -- 172.21.15.26:0/444450105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0814148dc0 con 0x7f08140ccc20 2023-12-19T18:14:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f0818ba6640 1 -- 172.21.15.26:0/444450105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081414f2d0 con 0x7f08140ccc20 2023-12-19T18:14:26.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080affd640 1 -- 172.21.15.26:0/444450105 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 105) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f07fc0113f0 con 0x7f08140ccc20 2023-12-19T18:14:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.004+0000 7f080affd640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f07f40658e0 0x7f07f4067da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.008+0000 7f080affd640 1 -- 172.21.15.26:0/444450105 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f07fc083570 con 0x7f08140ccc20 2023-12-19T18:14:26.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.008+0000 7f081259c640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f07f40658e0 0x7f07f4067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.008+0000 7f081259c640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f07f40658e0 0x7f07f4067da0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f08040ad000 tx=0x7f0804008040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:26.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.012+0000 7f080affd640 1 -- 172.21.15.26:0/444450105 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f081414f2d0 con 0x7f08140ccc20 2023-12-19T18:14:26.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.072+0000 7f080affd640 1 -- 172.21.15.26:0/444450105 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 106) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f07fc050f90 con 0x7f08140ccc20 2023-12-19T18:14:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.176+0000 7f0818ba6640 1 -- 172.21.15.26:0/444450105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f081414f2d0 con 0x7f08140ccc20 2023-12-19T18:14:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.176+0000 7f080affd640 1 -- 172.21.15.26:0/444450105 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+18709 (secure 0 0 0) 0x7f081414f2d0 con 0x7f08140ccc20 2023-12-19T18:14:26.179 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:26.179 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":184,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:14:23.971783+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T17:58:12.042476+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T17:58:12.457937+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":183,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.29","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/986493927":"2023-12-20T18:14:23.971750+0000","172.21.15.160:0/1845619584":"2023-12-20T18:14:23.971750+0000","172.21.15.160:0/387998771":"2023-12-20T18:14:23.971750+0000","172.21.15.160:0/2094392597":"2023-12-20T18:14:23.971750+0000","172.21.15.160:6832/48394":"2023-12-20T18:14:23.971750+0000","172.21.15.160:0/3381018054":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/3458963623":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/1302880583":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/1063410185":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/819581921":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/3919731969":"2023-12-20T18:14:23.971750+0000","172.21.15.160:0/4076846554":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/3832178910":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/317596538":"2023-12-20T18:14:23.971750+0000","172.21.15.160:6833/38098":"2023-12-20T17:58:25.501946+0000","172.21.15.160:6832/38098":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/1419214494":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/1333176784":"2023-12-20T18:09:20.657217+0000","172.21.15.160:0/4141924506":"2023-12-20T17:58:25.501946+0000","172.21.15.160:6833/48394":"2023-12-20T18:14:23.971750+0000","172.21.15.160:0/2994631753":"2023-12-20T17:58:25.501946+0000","172.21.15.160:0/2198643724":"2023-12-20T17:58:25.501946+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.180+0000 7f0818ba6640 1 -- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f07f40658e0 msgr2=0x7f07f4067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.180+0000 7f0818ba6640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f07f40658e0 0x7f07f4067da0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f08040ad000 tx=0x7f0804008040 comp rx=0 tx=0).stop 2023-12-19T18:14:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.180+0000 7f0818ba6640 1 -- 172.21.15.26:0/444450105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08140ccc20 msgr2=0x7f0814153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.180+0000 7f0818ba6640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08140ccc20 0x7f0814153550 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00c7c0 tx=0x7f07fc00cc80 comp rx=0 tx=0).stop 2023-12-19T18:14:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.180+0000 7f0818ba6640 1 -- 172.21.15.26:0/444450105 shutdown_connections 2023-12-19T18:14:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.180+0000 7f0818ba6640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f07f40658e0 0x7f07f4067da0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.180+0000 7f0818ba6640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f081415c7f0 0x7f0814153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.180+0000 7f0818ba6640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08140ccc20 0x7f0814153550 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.180+0000 7f0818ba6640 1 --2- 172.21.15.26:0/444450105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08140adb80 0x7f08141486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.180+0000 7f0818ba6640 1 -- 172.21.15.26:0/444450105 >> 172.21.15.26:0/444450105 conn(0x7f08140658e0 msgr2=0x7f0814159540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.180+0000 7f0818ba6640 1 -- 172.21.15.26:0/444450105 shutdown_connections 2023-12-19T18:14:26.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.180+0000 7f0818ba6640 1 -- 172.21.15.26:0/444450105 wait complete. 2023-12-19T18:14:26.201 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:14:26.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.344+0000 7f77b8d6d640 1 Processor -- start 2023-12-19T18:14:26.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.344+0000 7f77b8d6d640 1 -- start start 2023-12-19T18:14:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.344+0000 7f77b8d6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77b406b3d0 0x7f77b40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.344+0000 7f77b8d6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b406bc30 con 0x7f77b4154220 2023-12-19T18:14:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.344+0000 7f77b8d6d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b40cc6e0 con 0x7f77b40cec40 2023-12-19T18:14:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.344+0000 7f77b8d6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40cf0d0 con 0x7f77b406b3d0 2023-12-19T18:14:26.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.344+0000 7f77b259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f77b4154220 0x7f77b415aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39196/0 (socket says 172.21.15.26:39196) 2023-12-19T18:14:26.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.344+0000 7f77b259c640 1 -- 172.21.15.26:0/2125669066 learned_addr learned my addr 172.21.15.26:0/2125669066 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:26.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.344+0000 7f77abfff640 1 --2- 172.21.15.26:0/2125669066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77b406b3d0 0x7f77b40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861656082 0 0) 0x7f77b406bc30 con 0x7f77b4154220 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77a0003530 con 0x7f77b4154220 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541040293 0 0) 0x7f77b40cc6e0 con 0x7f77b40cec40 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77b406bc30 con 0x7f77b40cec40 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4271650817 0 0) 0x7f77a0003530 con 0x7f77b4154220 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77b406b3d0 msgr2=0x7f77b40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 --2- 172.21.15.26:0/2125669066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77b406b3d0 0x7f77b40ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 >> v1:172.21.15.160:6789/0 conn(0x7f77b40cec40 legacy=0x7f77b415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b406b950 con 0x7f77b4154220 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f77a4001680 con 0x7f77b4154220 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 >> v1:172.21.15.26:6789/0 conn(0x7f77b4154220 legacy=0x7f77b415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 --2- 172.21.15.26:0/2125669066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77a0003b40 0x7f77a0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 --2- 172.21.15.26:0/2125669066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a00047d0 0x7f77a0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 --2- 172.21.15.26:0/2125669066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77a0005190 0x7f77a000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40cf0d0 con 0x7f77a00047d0 2023-12-19T18:14:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a000e190 con 0x7f77a0005190 2023-12-19T18:14:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a000e350 con 0x7f77a0003b40 2023-12-19T18:14:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77abfff640 1 --2- 172.21.15.26:0/2125669066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77b406b3d0 0x7f77b40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77abfff640 1 --2- 172.21.15.26:0/2125669066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a00047d0 0x7f77a0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b259c640 1 --2- 172.21.15.26:0/2125669066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77a0003b40 0x7f77a0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77b2d9d640 1 --2- 172.21.15.26:0/2125669066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77a0005190 0x7f77a000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77abfff640 1 -- 172.21.15.26:0/2125669066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77a0003b40 msgr2=0x7f77a0003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77abfff640 1 --2- 172.21.15.26:0/2125669066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77a0003b40 0x7f77a0003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77abfff640 1 -- 172.21.15.26:0/2125669066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77a0005190 msgr2=0x7f77a000dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77abfff640 1 --2- 172.21.15.26:0/2125669066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77a0005190 0x7f77a000dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.348+0000 7f77abfff640 1 -- 172.21.15.26:0/2125669066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a0003530 con 0x7f77a00047d0 2023-12-19T18:14:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77abfff640 1 --2- 172.21.15.26:0/2125669066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a00047d0 0x7f77a0004c50 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f77b4066670 tx=0x7f779c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6e0b5a7d9f57eb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c004030 con 0x7f77a00047d0 2023-12-19T18:14:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c004190 con 0x7f77a00047d0 2023-12-19T18:14:26.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b1d9b640 1 -- 172.21.15.26:0/2125669066 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c004520 con 0x7f77a00047d0 2023-12-19T18:14:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2125669066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a00047d0 msgr2=0x7f77a0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 --2- 172.21.15.26:0/2125669066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a00047d0 0x7f77a0004c50 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f77b4066670 tx=0x7f779c01ca90 comp rx=0 tx=0).stop 2023-12-19T18:14:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b2d9d640 1 -- 172.21.15.26:0/2125669066 reap_dead start 2023-12-19T18:14:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2125669066 shutdown_connections 2023-12-19T18:14:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2125669066 >> 172.21.15.26:0/2125669066 conn(0x7f77b40658e0 msgr2=0x7f77b4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2125669066 shutdown_connections 2023-12-19T18:14:26.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2125669066 wait complete. 2023-12-19T18:14:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 Processor -- start 2023-12-19T18:14:26.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 -- start start 2023-12-19T18:14:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a00047d0 0x7f77b40b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77b414a220 0x7f77b4156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77b40ac850 0x7f77b40accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40ad210 con 0x7f77a00047d0 2023-12-19T18:14:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b4157060 con 0x7f77b414a220 2023-12-19T18:14:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b8d6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b4157330 con 0x7f77b40ac850 2023-12-19T18:14:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.352+0000 7f77b259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a00047d0 0x7f77b40b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77b259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a00047d0 0x7f77b40b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38922/0 (socket says 172.21.15.26:38922) 2023-12-19T18:14:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77b259c640 1 -- 172.21.15.26:0/2112554204 learned_addr learned my addr 172.21.15.26:0/2112554204 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77b2d9d640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77b40ac850 0x7f77b40accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77abfff640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77b414a220 0x7f77b4156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77b259c640 1 -- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77b40ac850 msgr2=0x7f77b40accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77b259c640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77b40ac850 0x7f77b40accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77b259c640 1 -- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77b414a220 msgr2=0x7f77b4156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77b259c640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77b414a220 0x7f77b4156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77b259c640 1 -- 172.21.15.26:0/2112554204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b4157610 con 0x7f77a00047d0 2023-12-19T18:14:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77b259c640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a00047d0 0x7f77b40b2ec0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f77a4004f50 tx=0x7f77a40074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77aaffd640 1 -- 172.21.15.26:0/2112554204 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4007740 con 0x7f77a00047d0 2023-12-19T18:14:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77aaffd640 1 -- 172.21.15.26:0/2112554204 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a400f940 con 0x7f77a00047d0 2023-12-19T18:14:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77aaffd640 1 -- 172.21.15.26:0/2112554204 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a400fcd0 con 0x7f77a00047d0 2023-12-19T18:14:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2112554204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b41578f0 con 0x7f77a00047d0 2023-12-19T18:14:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2112554204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b40b0760 con 0x7f77a00047d0 2023-12-19T18:14:26.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2112554204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b4074270 con 0x7f77a00047d0 2023-12-19T18:14:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77aaffd640 1 -- 172.21.15.26:0/2112554204 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 106) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f77a401d460 con 0x7f77a00047d0 2023-12-19T18:14:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77aaffd640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7794065af0 0x7f7794067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.356+0000 7f77aaffd640 1 -- 172.21.15.26:0/2112554204 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 7444+0+0 (secure 0 0 0) 0x7f77a4083750 con 0x7f77a00047d0 2023-12-19T18:14:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.360+0000 7f77abfff640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7794065af0 0x7f7794067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.360+0000 7f77abfff640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7794065af0 0x7f7794067fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f779c0ad000 tx=0x7f779c0af040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:26.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.360+0000 7f77aaffd640 1 -- 172.21.15.26:0/2112554204 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f77a4050b90 con 0x7f77a00047d0 2023-12-19T18:14:26.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.552+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2112554204 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f77b4071540 con 0x7f77a00047d0 2023-12-19T18:14:26.555 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T18:14:26.556+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:14:26.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.564+0000 7f77aaffd640 1 -- 172.21.15.26:0/2112554204 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) v1 ==== 161+0+0 (secure 0 0 0) 0x7f77a40541b0 con 0x7f77a00047d0 2023-12-19T18:14:26.565 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:14:26.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.568+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7794065af0 msgr2=0x7f7794067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.568+0000 7f77b8d6d640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7794065af0 0x7f7794067fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f779c0ad000 tx=0x7f779c0af040 comp rx=0 tx=0).stop 2023-12-19T18:14:26.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.568+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2112554204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a00047d0 msgr2=0x7f77b40b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.568+0000 7f77b8d6d640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a00047d0 0x7f77b40b2ec0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f77a4004f50 tx=0x7f77a40074f0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.568+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2112554204 shutdown_connections 2023-12-19T18:14:26.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.568+0000 7f77b8d6d640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7794065af0 0x7f7794067fb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.568+0000 7f77b8d6d640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77b40ac850 0x7f77b40accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.568+0000 7f77b8d6d640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77b414a220 0x7f77b4156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.568+0000 7f77b8d6d640 1 --2- 172.21.15.26:0/2112554204 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77a00047d0 0x7f77b40b2ec0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.568+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2112554204 >> 172.21.15.26:0/2112554204 conn(0x7f77b40658e0 msgr2=0x7f77b40cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:26.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.568+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2112554204 shutdown_connections 2023-12-19T18:14:26.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.568+0000 7f77b8d6d640 1 -- 172.21.15.26:0/2112554204 wait complete. 2023-12-19T18:14:26.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:14:26.642 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T18:14:26.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.744+0000 7f4477709640 1 Processor -- start 2023-12-19T18:14:26.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.744+0000 7f4477709640 1 -- start start 2023-12-19T18:14:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.744+0000 7f4477709640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f44701599e0 0x7f447006f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.744+0000 7f4477709640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44700cb150 0x7f44700cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.744+0000 7f4477709640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44700cbaf0 con 0x7f447015d530 2023-12-19T18:14:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.744+0000 7f4477709640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447015e050 con 0x7f44701599e0 2023-12-19T18:14:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.744+0000 7f4477709640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447015e240 con 0x7f44700cb150 2023-12-19T18:14:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.744+0000 7f4474ca4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44700cb150 0x7f44700cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.744+0000 7f44754a5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f44701599e0 0x7f447006f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.744+0000 7f4474ca4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44700cb150 0x7f44700cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:56644/0 (socket says 172.21.15.26:56644) 2023-12-19T18:14:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.744+0000 7f4474ca4640 1 -- 172.21.15.26:0/2285083607 learned_addr learned my addr 172.21.15.26:0/2285083607 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.744+0000 7f4475ca6640 1 --1- 172.21.15.26:0/2285083607 >> v1:172.21.15.160:6789/0 conn(0x7f447015d530 0x7f447015d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:58270/0 (socket says 172.21.15.26:58270) 2023-12-19T18:14:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.748+0000 7f44754a5640 1 -- 172.21.15.26:0/2285083607 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44700cb150 msgr2=0x7f44700cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.748+0000 7f44754a5640 1 --2- 172.21.15.26:0/2285083607 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44700cb150 0x7f44700cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.748+0000 7f44754a5640 1 -- 172.21.15.26:0/2285083607 >> v1:172.21.15.160:6789/0 conn(0x7f447015d530 legacy=0x7f447015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.748+0000 7f4467fff640 1 -- 172.21.15.26:0/2285083607 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136705030 0 0) 0x7f44700cbaf0 con 0x7f447015d530 2023-12-19T18:14:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.748+0000 7f44754a5640 1 -- 172.21.15.26:0/2285083607 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44700c9520 con 0x7f44701599e0 2023-12-19T18:14:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.748+0000 7f4474ca4640 1 --2- 172.21.15.26:0/2285083607 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44700cb150 0x7f44700cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:26.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.748+0000 7f44754a5640 1 --2- 172.21.15.26:0/2285083607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44701599e0 0x7f447006f6d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f446c0097c0 tx=0x7f446c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fe4797ee23c3ca server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:26.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.748+0000 7f4467fff640 1 -- 172.21.15.26:0/2285083607 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0ab680 con 0x7f44701599e0 2023-12-19T18:14:26.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.748+0000 7f4467fff640 1 -- 172.21.15.26:0/2285083607 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c0ab7e0 con 0x7f44701599e0 2023-12-19T18:14:26.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.748+0000 7f4467fff640 1 -- 172.21.15.26:0/2285083607 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0abb70 con 0x7f44701599e0 2023-12-19T18:14:26.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 -- 172.21.15.26:0/2285083607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44701599e0 msgr2=0x7f447006f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 --2- 172.21.15.26:0/2285083607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44701599e0 0x7f447006f6d0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f446c0097c0 tx=0x7f446c01c480 comp rx=0 tx=0).stop 2023-12-19T18:14:26.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 -- 172.21.15.26:0/2285083607 shutdown_connections 2023-12-19T18:14:26.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 --2- 172.21.15.26:0/2285083607 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44700cb150 0x7f44700cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 --2- 172.21.15.26:0/2285083607 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44701599e0 0x7f447006f6d0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 -- 172.21.15.26:0/2285083607 >> 172.21.15.26:0/2285083607 conn(0x7f4470066160 msgr2=0x7f44700b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:26.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 -- 172.21.15.26:0/2285083607 shutdown_connections 2023-12-19T18:14:26.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 -- 172.21.15.26:0/2285083607 wait complete. 2023-12-19T18:14:26.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 Processor -- start 2023-12-19T18:14:26.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 -- start start 2023-12-19T18:14:26.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44700cb150 0x7f4470159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44701599e0 0x7f4470154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f447015d530 0x7f4470154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44701f9970 con 0x7f44700cb150 2023-12-19T18:14:26.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44701f5290 con 0x7f447015d530 2023-12-19T18:14:26.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4477709640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44701f8df0 con 0x7f44701599e0 2023-12-19T18:14:26.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f44754a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44700cb150 0x7f4470159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f44754a5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44700cb150 0x7f4470159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38956/0 (socket says 172.21.15.26:38956) 2023-12-19T18:14:26.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f44754a5640 1 -- 172.21.15.26:0/1189213026 learned_addr learned my addr 172.21.15.26:0/1189213026 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:26.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4474ca4640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44701599e0 0x7f4470154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4475ca6640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f447015d530 0x7f4470154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f44754a5640 1 -- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44701599e0 msgr2=0x7f4470154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f44754a5640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44701599e0 0x7f4470154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f44754a5640 1 -- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f447015d530 msgr2=0x7f4470154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:26.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f44754a5640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f447015d530 0x7f4470154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:26.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f44754a5640 1 -- 172.21.15.26:0/1189213026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446000b770 con 0x7f44700cb150 2023-12-19T18:14:26.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.752+0000 7f4475ca6640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f447015d530 0x7f4470154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:26.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.756+0000 7f44754a5640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44700cb150 0x7f4470159f40 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f446c0ad040 tx=0x7f446c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:26.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.756+0000 7f4465ffb640 1 -- 172.21.15.26:0/1189213026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0b5810 con 0x7f44700cb150 2023-12-19T18:14:26.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.756+0000 7f4465ffb640 1 -- 172.21.15.26:0/1189213026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c0b59b0 con 0x7f44700cb150 2023-12-19T18:14:26.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.756+0000 7f4477709640 1 -- 172.21.15.26:0/1189213026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446c009420 con 0x7f44700cb150 2023-12-19T18:14:26.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.756+0000 7f4465ffb640 1 -- 172.21.15.26:0/1189213026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c0b5d60 con 0x7f44700cb150 2023-12-19T18:14:26.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.756+0000 7f4477709640 1 -- 172.21.15.26:0/1189213026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44701f93b0 con 0x7f44700cb150 2023-12-19T18:14:26.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.756+0000 7f4477709640 1 -- 172.21.15.26:0/1189213026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4470074270 con 0x7f44700cb150 2023-12-19T18:14:26.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.756+0000 7f4465ffb640 1 -- 172.21.15.26:0/1189213026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 106) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f446c0c83f0 con 0x7f44700cb150 2023-12-19T18:14:26.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.756+0000 7f4465ffb640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4454065af0 0x7f4454067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:26.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.756+0000 7f4465ffb640 1 -- 172.21.15.26:0/1189213026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 7530+0+0 (secure 0 0 0) 0x7f446c11e8c0 con 0x7f44700cb150 2023-12-19T18:14:26.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.756+0000 7f4474ca4640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4454065af0 0x7f4454067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:26.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.760+0000 7f4474ca4640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4454065af0 0x7f4454067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f44700c1fa0 tx=0x7f446000b450 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:26.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.760+0000 7f4465ffb640 1 -- 172.21.15.26:0/1189213026 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f446c0ebaf0 con 0x7f44700cb150 2023-12-19T18:14:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:26.952+0000 7f4477709640 1 -- 172.21.15.26:0/1189213026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f44700c3390 con 0x7f44700cb150 2023-12-19T18:14:27.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.568+0000 7f4465ffb640 1 -- 172.21.15.26:0/1189213026 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) v1 ==== 104+0+0 (secure 0 0 0) 0x7f446c0ab640 con 0x7f44700cb150 2023-12-19T18:14:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.572+0000 7f4477709640 1 -- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4454065af0 msgr2=0x7f4454067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.572+0000 7f4477709640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4454065af0 0x7f4454067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f44700c1fa0 tx=0x7f446000b450 comp rx=0 tx=0).stop 2023-12-19T18:14:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.572+0000 7f4477709640 1 -- 172.21.15.26:0/1189213026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44700cb150 msgr2=0x7f4470159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.572+0000 7f4477709640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44700cb150 0x7f4470159f40 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f446c0ad040 tx=0x7f446c019220 comp rx=0 tx=0).stop 2023-12-19T18:14:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.572+0000 7f4477709640 1 -- 172.21.15.26:0/1189213026 shutdown_connections 2023-12-19T18:14:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.572+0000 7f4477709640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4454065af0 0x7f4454067fb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.572+0000 7f4477709640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f447015d530 0x7f4470154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.572+0000 7f4477709640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f44701599e0 0x7f4470154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.572+0000 7f4477709640 1 --2- 172.21.15.26:0/1189213026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44700cb150 0x7f4470159f40 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:27.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.572+0000 7f4477709640 1 -- 172.21.15.26:0/1189213026 >> 172.21.15.26:0/1189213026 conn(0x7f4470066160 msgr2=0x7f4470070320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.572+0000 7f4477709640 1 -- 172.21.15.26:0/1189213026 shutdown_connections 2023-12-19T18:14:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.572+0000 7f4477709640 1 -- 172.21.15.26:0/1189213026 wait complete. 2023-12-19T18:14:27.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:14:27.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9fc2d640 1 Processor -- start 2023-12-19T18:14:27.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9fc2d640 1 -- start start 2023-12-19T18:14:27.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9fc2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d980b3f90 0x7f5d9806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:27.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9fc2d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9806cbf0 con 0x7f5d980b1590 2023-12-19T18:14:27.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9fc2d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d980ca8b0 con 0x7f5d9815aad0 2023-12-19T18:14:27.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9fc2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9806b6f0 con 0x7f5d980b3f90 2023-12-19T18:14:27.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9d1c8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5d980b1590 0x7f5d980cad40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39202/0 (socket says 172.21.15.26:39202) 2023-12-19T18:14:27.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9d1c8640 1 -- 172.21.15.26:0/1617313442 learned_addr learned my addr 172.21.15.26:0/1617313442 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:27.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9e1ca640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d980b3f90 0x7f5d9806c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974350348 0 0) 0x7f5d88001120 con 0x7f5d980b1590 2023-12-19T18:14:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d84003530 con 0x7f5d980b1590 2023-12-19T18:14:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100861084 0 0) 0x7f5d980ca8b0 con 0x7f5d9815aad0 2023-12-19T18:14:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d88001120 con 0x7f5d9815aad0 2023-12-19T18:14:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2986712503 0 0) 0x7f5d84003530 con 0x7f5d980b1590 2023-12-19T18:14:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d980b3f90 msgr2=0x7f5d9806c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d980b3f90 0x7f5d9806c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 >> v1:172.21.15.160:6789/0 conn(0x7f5d9815aad0 legacy=0x7f5d98070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9806b9d0 con 0x7f5d980b1590 2023-12-19T18:14:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f5d88001810 con 0x7f5d980b1590 2023-12-19T18:14:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9e1ca640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d980b3f90 0x7f5d9806c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 >> v1:172.21.15.26:6789/0 conn(0x7f5d980b1590 legacy=0x7f5d980cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d84003b40 0x7f5d84003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d840047d0 0x7f5d84004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d84005190 0x7f5d8400fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9806b6f0 con 0x7f5d840047d0 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d84010390 con 0x7f5d84005190 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d84010550 con 0x7f5d84003b40 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9d1c8640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d840047d0 0x7f5d84004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.792+0000 7f5d9d9c9640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d84003b40 0x7f5d84003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9e1ca640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d84005190 0x7f5d8400fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9d1c8640 1 -- 172.21.15.26:0/1617313442 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d84003b40 msgr2=0x7f5d84003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9d1c8640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d84003b40 0x7f5d84003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9d1c8640 1 -- 172.21.15.26:0/1617313442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d84005190 msgr2=0x7f5d8400fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9d1c8640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d84005190 0x7f5d8400fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9d1c8640 1 -- 172.21.15.26:0/1617313442 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d84003530 con 0x7f5d840047d0 2023-12-19T18:14:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9e1ca640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d84005190 0x7f5d8400fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:27.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9d1c8640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d840047d0 0x7f5d84004c50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f5d980b19b0 tx=0x7f5d8801cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ad5f2f0fd7884 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:27.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d880058e0 con 0x7f5d840047d0 2023-12-19T18:14:27.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d88005a60 con 0x7f5d840047d0 2023-12-19T18:14:27.798 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9c9c7640 1 -- 172.21.15.26:0/1617313442 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88005e50 con 0x7f5d840047d0 2023-12-19T18:14:27.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1617313442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d840047d0 msgr2=0x7f5d84004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:27.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9fc2d640 1 --2- 172.21.15.26:0/1617313442 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d840047d0 0x7f5d84004c50 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f5d980b19b0 tx=0x7f5d8801cb80 comp rx=0 tx=0).stop 2023-12-19T18:14:27.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.796+0000 7f5d9e1ca640 1 -- 172.21.15.26:0/1617313442 reap_dead start 2023-12-19T18:14:27.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1617313442 shutdown_connections 2023-12-19T18:14:27.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1617313442 >> 172.21.15.26:0/1617313442 conn(0x7f5d980b0270 msgr2=0x7f5d980b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:27.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1617313442 shutdown_connections 2023-12-19T18:14:27.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1617313442 wait complete. 2023-12-19T18:14:27.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9fc2d640 1 Processor -- start 2023-12-19T18:14:27.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9fc2d640 1 -- start start 2023-12-19T18:14:27.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9fc2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d840047d0 0x7f5d980b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:27.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9fc2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d980cdd50 0x7f5d98155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:27.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9fc2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d9815d980 0x7f5d980cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:27.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9fc2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d980c4890 con 0x7f5d840047d0 2023-12-19T18:14:27.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9fc2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9815d4a0 con 0x7f5d980cdd50 2023-12-19T18:14:27.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9fc2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d98155570 con 0x7f5d9815d980 2023-12-19T18:14:27.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9d9c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d840047d0 0x7f5d980b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:27.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9d9c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d840047d0 0x7f5d980b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38982/0 (socket says 172.21.15.26:38982) 2023-12-19T18:14:27.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9d9c9640 1 -- 172.21.15.26:0/1930880410 learned_addr learned my addr 172.21.15.26:0/1930880410 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:27.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9d1c8640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d980cdd50 0x7f5d98155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:27.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9e1ca640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d9815d980 0x7f5d980cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:27.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9d9c9640 1 -- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d9815d980 msgr2=0x7f5d980cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:27.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9d9c9640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d9815d980 0x7f5d980cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:27.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9d9c9640 1 -- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d980cdd50 msgr2=0x7f5d98155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:27.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9d9c9640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d980cdd50 0x7f5d98155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:27.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9d9c9640 1 -- 172.21.15.26:0/1930880410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d980c4a80 con 0x7f5d840047d0 2023-12-19T18:14:27.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9e1ca640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d9815d980 0x7f5d980cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:27.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.800+0000 7f5d9d9c9640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d840047d0 0x7f5d980b2170 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f5d94007ca0 tx=0x7f5d94012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:27.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.804+0000 7f5d8e7fc640 1 -- 172.21.15.26:0/1930880410 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d94006ff0 con 0x7f5d840047d0 2023-12-19T18:14:27.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.804+0000 7f5d8e7fc640 1 -- 172.21.15.26:0/1930880410 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d94010b90 con 0x7f5d840047d0 2023-12-19T18:14:27.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.804+0000 7f5d8e7fc640 1 -- 172.21.15.26:0/1930880410 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d94012ee0 con 0x7f5d840047d0 2023-12-19T18:14:27.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.804+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1930880410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d980c4d60 con 0x7f5d840047d0 2023-12-19T18:14:27.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.804+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1930880410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9815b3a0 con 0x7f5d840047d0 2023-12-19T18:14:27.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.804+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1930880410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9815d660 con 0x7f5d840047d0 2023-12-19T18:14:27.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.804+0000 7f5d8e7fc640 1 -- 172.21.15.26:0/1930880410 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 106) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f5d94016480 con 0x7f5d840047d0 2023-12-19T18:14:27.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.804+0000 7f5d8e7fc640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5d7c065b20 0x7f5d7c067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:27.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.804+0000 7f5d8e7fc640 1 -- 172.21.15.26:0/1930880410 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 7530+0+0 (secure 0 0 0) 0x7f5d94087090 con 0x7f5d840047d0 2023-12-19T18:14:27.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.804+0000 7f5d9d1c8640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5d7c065b20 0x7f5d7c067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:27.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.808+0000 7f5d9d1c8640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5d7c065b20 0x7f5d7c067fe0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5d880aa000 tx=0x7f5d88001b90 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:27.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.808+0000 7f5d8e7fc640 1 -- 172.21.15.26:0/1930880410 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5d940542d0 con 0x7f5d840047d0 2023-12-19T18:14:27.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:27.968+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1930880410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5d98074270 con 0x7f5d840047d0 2023-12-19T18:14:28.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.576+0000 7f5d8e7fc640 1 -- 172.21.15.26:0/1930880410 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5d940546a0 con 0x7f5d840047d0 2023-12-19T18:14:28.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.580+0000 7f5d8e7fc640 1 -- 172.21.15.26:0/1930880410 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v186) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5d94082af0 con 0x7f5d840047d0 2023-12-19T18:14:28.580 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:14:28.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.580+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5d7c065b20 msgr2=0x7f5d7c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:28.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.580+0000 7f5d9fc2d640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5d7c065b20 0x7f5d7c067fe0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5d880aa000 tx=0x7f5d88001b90 comp rx=0 tx=0).stop 2023-12-19T18:14:28.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.580+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1930880410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d840047d0 msgr2=0x7f5d980b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:28.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.580+0000 7f5d9fc2d640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d840047d0 0x7f5d980b2170 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f5d94007ca0 tx=0x7f5d94012c90 comp rx=0 tx=0).stop 2023-12-19T18:14:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.584+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1930880410 shutdown_connections 2023-12-19T18:14:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.584+0000 7f5d9fc2d640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5d7c065b20 0x7f5d7c067fe0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.584+0000 7f5d9fc2d640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d9815d980 0x7f5d980cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.584+0000 7f5d9fc2d640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d980cdd50 0x7f5d98155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.584+0000 7f5d9fc2d640 1 --2- 172.21.15.26:0/1930880410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d840047d0 0x7f5d980b2170 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.584+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1930880410 >> 172.21.15.26:0/1930880410 conn(0x7f5d980b0270 msgr2=0x7f5d980b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.584+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1930880410 shutdown_connections 2023-12-19T18:14:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.584+0000 7f5d9fc2d640 1 -- 172.21.15.26:0/1930880410 wait complete. 2023-12-19T18:14:28.606 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:14:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8f76b6640 1 Processor -- start 2023-12-19T18:14:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8f76b6640 1 -- start start 2023-12-19T18:14:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8f76b6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb8f006c830 0x7fb8f00cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8f76b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f00cce50 con 0x7fb8f00c1a50 2023-12-19T18:14:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8f76b6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f00cc520 con 0x7fb8f00ada30 2023-12-19T18:14:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8f76b6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f0161800 con 0x7fb8f006c830 2023-12-19T18:14:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8f5c53640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb8f00c1a50 0x7fb8f00ca570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39214/0 (socket says 172.21.15.26:39214) 2023-12-19T18:14:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8f5c53640 1 -- 172.21.15.26:0/4180249439 learned_addr learned my addr 172.21.15.26:0/4180249439 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8f4c51640 1 --2- 172.21.15.26:0/4180249439 >> v2:172.21.15.160:3300/0 conn(0x7fb8f006c830 0x7fb8f00cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8e7fff640 1 -- 172.21.15.26:0/4180249439 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904472142 0 0) 0x7fb8f00cce50 con 0x7fb8f00c1a50 2023-12-19T18:14:28.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8e7fff640 1 -- 172.21.15.26:0/4180249439 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8dc003530 con 0x7fb8f00c1a50 2023-12-19T18:14:28.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8f4c51640 1 -- 172.21.15.26:0/4180249439 >> v1:172.21.15.160:6789/0 conn(0x7fb8f00ada30 legacy=0x7fb8f00c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:28.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8e7fff640 1 -- 172.21.15.26:0/4180249439 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 589007698 0 0) 0x7fb8f00cc520 con 0x7fb8f00ada30 2023-12-19T18:14:28.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8f4c51640 1 -- 172.21.15.26:0/4180249439 >> v1:172.21.15.26:6789/0 conn(0x7fb8f00c1a50 legacy=0x7fb8f00ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:28.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.764+0000 7fb8f4c51640 1 -- 172.21.15.26:0/4180249439 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f00cc800 con 0x7fb8f006c830 2023-12-19T18:14:28.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.768+0000 7fb8f4c51640 1 --2- 172.21.15.26:0/4180249439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8f006c830 0x7fb8f00cd2e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00097c0 tx=0x7fb8e001c230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c7c1281795765de server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:28.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.768+0000 7fb8e7fff640 1 -- 172.21.15.26:0/4180249439 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8001a00 con 0x7fb8f006c830 2023-12-19T18:14:28.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.768+0000 7fb8e7fff640 1 -- 172.21.15.26:0/4180249439 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e00a7050 con 0x7fb8f006c830 2023-12-19T18:14:28.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.768+0000 7fb8e7fff640 1 -- 172.21.15.26:0/4180249439 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00ab600 con 0x7fb8f006c830 2023-12-19T18:14:28.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.768+0000 7fb8f76b6640 1 -- 172.21.15.26:0/4180249439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8f006c830 msgr2=0x7fb8f00cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:28.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.768+0000 7fb8f76b6640 1 --2- 172.21.15.26:0/4180249439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8f006c830 0x7fb8f00cd2e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00097c0 tx=0x7fb8e001c230 comp rx=0 tx=0).stop 2023-12-19T18:14:28.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.768+0000 7fb8f76b6640 1 -- 172.21.15.26:0/4180249439 shutdown_connections 2023-12-19T18:14:28.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.768+0000 7fb8f76b6640 1 --2- 172.21.15.26:0/4180249439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8f006c830 0x7fb8f00cd2e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:28.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.768+0000 7fb8f76b6640 1 -- 172.21.15.26:0/4180249439 >> 172.21.15.26:0/4180249439 conn(0x7fb8f00b0270 msgr2=0x7fb8f00b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:28.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.768+0000 7fb8f76b6640 1 -- 172.21.15.26:0/4180249439 shutdown_connections 2023-12-19T18:14:28.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.768+0000 7fb8f76b6640 1 -- 172.21.15.26:0/4180249439 wait complete. 2023-12-19T18:14:28.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f76b6640 1 Processor -- start 2023-12-19T18:14:28.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f76b6640 1 -- start start 2023-12-19T18:14:28.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f76b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8f006c830 0x7fb8f0160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:28.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f76b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00ada30 0x7fb8f00cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:28.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f76b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8f00c1a50 0x7fb8f00cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:28.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f76b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00c4d40 con 0x7fb8f00ada30 2023-12-19T18:14:28.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f76b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00c32c0 con 0x7fb8f00c1a50 2023-12-19T18:14:28.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f76b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00c4730 con 0x7fb8f006c830 2023-12-19T18:14:28.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f4c51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00ada30 0x7fb8f00cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:28.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f4c51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00ada30 0x7fb8f00cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38996/0 (socket says 172.21.15.26:38996) 2023-12-19T18:14:28.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f4c51640 1 -- 172.21.15.26:0/309104575 learned_addr learned my addr 172.21.15.26:0/309104575 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:28.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f5452640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8f006c830 0x7fb8f0160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:28.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f5c53640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8f00c1a50 0x7fb8f00cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:28.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f4c51640 1 -- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8f006c830 msgr2=0x7fb8f0160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:28.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f4c51640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8f006c830 0x7fb8f0160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:28.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f4c51640 1 -- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8f00c1a50 msgr2=0x7fb8f00cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:28.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f4c51640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8f00c1a50 0x7fb8f00cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:28.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f4c51640 1 -- 172.21.15.26:0/309104575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e0009420 con 0x7fb8f00ada30 2023-12-19T18:14:28.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f4c51640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00ada30 0x7fb8f00cb520 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0000c10 tx=0x7fb8e0018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:28.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8e5ffb640 1 -- 172.21.15.26:0/309104575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e00b9070 con 0x7fb8f00ada30 2023-12-19T18:14:28.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8e5ffb640 1 -- 172.21.15.26:0/309104575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e00038b0 con 0x7fb8f00ada30 2023-12-19T18:14:28.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8e5ffb640 1 -- 172.21.15.26:0/309104575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0003c20 con 0x7fb8f00ada30 2023-12-19T18:14:28.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.772+0000 7fb8f76b6640 1 -- 172.21.15.26:0/309104575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f0153280 con 0x7fb8f00ada30 2023-12-19T18:14:28.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.776+0000 7fb8f76b6640 1 -- 172.21.15.26:0/309104575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f00c2fe0 con 0x7fb8f00ada30 2023-12-19T18:14:28.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.776+0000 7fb8f76b6640 1 -- 172.21.15.26:0/309104575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f00c4ef0 con 0x7fb8f00ada30 2023-12-19T18:14:28.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.776+0000 7fb8e5ffb640 1 -- 172.21.15.26:0/309104575 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb8e00b2040 con 0x7fb8f00ada30 2023-12-19T18:14:28.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.776+0000 7fb8e5ffb640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb8d4065bb0 0x7fb8d4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:28.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.776+0000 7fb8e5ffb640 1 -- 172.21.15.26:0/309104575 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fb8e011cf70 con 0x7fb8f00ada30 2023-12-19T18:14:28.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.776+0000 7fb8f5452640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb8d4065bb0 0x7fb8d4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:28.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.776+0000 7fb8f5452640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb8d4065bb0 0x7fb8d4068070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec000f70 tx=0x7fb8ec009040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:28.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.780+0000 7fb8e5ffb640 1 -- 172.21.15.26:0/309104575 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb8f00c4ef0 con 0x7fb8f00ada30 2023-12-19T18:14:28.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:28.940+0000 7fb8f76b6640 1 -- 172.21.15.26:0/309104575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb8f0074270 con 0x7fb8f00ada30 2023-12-19T18:14:29.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.012+0000 7fb8e5ffb640 1 -- 172.21.15.26:0/309104575 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v187) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb8e00b0020 con 0x7fb8f00ada30 2023-12-19T18:14:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T18:14:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.016+0000 7fb8f76b6640 1 -- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb8d4065bb0 msgr2=0x7fb8d4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.016+0000 7fb8f76b6640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb8d4065bb0 0x7fb8d4068070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec000f70 tx=0x7fb8ec009040 comp rx=0 tx=0).stop 2023-12-19T18:14:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.016+0000 7fb8f76b6640 1 -- 172.21.15.26:0/309104575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00ada30 msgr2=0x7fb8f00cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.016+0000 7fb8f76b6640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00ada30 0x7fb8f00cb520 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0000c10 tx=0x7fb8e0018fd0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.016+0000 7fb8f76b6640 1 -- 172.21.15.26:0/309104575 shutdown_connections 2023-12-19T18:14:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.016+0000 7fb8f76b6640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb8d4065bb0 0x7fb8d4068070 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.016+0000 7fb8f76b6640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb8f00c1a50 0x7fb8f00cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.016+0000 7fb8f76b6640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8f00ada30 0x7fb8f00cb520 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.016+0000 7fb8f76b6640 1 --2- 172.21.15.26:0/309104575 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb8f006c830 0x7fb8f0160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.016+0000 7fb8f76b6640 1 -- 172.21.15.26:0/309104575 >> 172.21.15.26:0/309104575 conn(0x7fb8f00b0270 msgr2=0x7fb8f0070240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.016+0000 7fb8f76b6640 1 -- 172.21.15.26:0/309104575 shutdown_connections 2023-12-19T18:14:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.016+0000 7fb8f76b6640 1 -- 172.21.15.26:0/309104575 wait complete. 2023-12-19T18:14:29.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-19T18:14:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f30a1020640 1 Processor -- start 2023-12-19T18:14:29.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f30a1020640 1 -- start start 2023-12-19T18:14:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f30a1020640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f309c1585f0 0x7f309c0cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f30a1020640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f309c0cb5d0 0x7f309c0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f30a1020640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309c15b1b0 con 0x7f309c0cb150 2023-12-19T18:14:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f30a1020640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c0b47a0 con 0x7f309c0cb5d0 2023-12-19T18:14:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f30a1020640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c15b490 con 0x7f309c1585f0 2023-12-19T18:14:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f309b59e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f309c0cb5d0 0x7f309c0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f309b59e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f309c0cb5d0 0x7f309c0cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:50694/0 (socket says 172.21.15.26:50694) 2023-12-19T18:14:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f309b59e640 1 -- 172.21.15.26:0/463018940 learned_addr learned my addr 172.21.15.26:0/463018940 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:29.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f309ad9d640 1 --2- 172.21.15.26:0/463018940 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f309c1585f0 0x7f309c0cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f309b59e640 1 -- 172.21.15.26:0/463018940 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f309c1585f0 msgr2=0x7f309c0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f309b59e640 1 --2- 172.21.15.26:0/463018940 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f309c1585f0 0x7f309c0cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f309b59e640 1 -- 172.21.15.26:0/463018940 >> v1:172.21.15.160:6789/0 conn(0x7f309c0cb150 legacy=0x7f309c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f309b59e640 1 -- 172.21.15.26:0/463018940 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c15b770 con 0x7f309c0cb5d0 2023-12-19T18:14:29.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.196+0000 7f309ad9d640 1 --2- 172.21.15.26:0/463018940 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f309c1585f0 0x7f309c0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.200+0000 7f309b59e640 1 --2- 172.21.15.26:0/463018940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c0cb5d0 0x7f309c0cca20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3094009e50 tx=0x7f309401c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb35ebf9f5979df server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.200+0000 7f309a59c640 1 -- 172.21.15.26:0/463018940 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30940a5830 con 0x7f309c0cb5d0 2023-12-19T18:14:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.200+0000 7f309a59c640 1 -- 172.21.15.26:0/463018940 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30940a5990 con 0x7f309c0cb5d0 2023-12-19T18:14:29.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.200+0000 7f309a59c640 1 -- 172.21.15.26:0/463018940 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30940a5d20 con 0x7f309c0cb5d0 2023-12-19T18:14:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.200+0000 7f30a1020640 1 -- 172.21.15.26:0/463018940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c0cb5d0 msgr2=0x7f309c0cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.200+0000 7f30a1020640 1 --2- 172.21.15.26:0/463018940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c0cb5d0 0x7f309c0cca20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3094009e50 tx=0x7f309401c950 comp rx=0 tx=0).stop 2023-12-19T18:14:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.200+0000 7f30a1020640 1 -- 172.21.15.26:0/463018940 shutdown_connections 2023-12-19T18:14:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.200+0000 7f30a1020640 1 --2- 172.21.15.26:0/463018940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c0cb5d0 0x7f309c0cca20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.200+0000 7f30a1020640 1 --2- 172.21.15.26:0/463018940 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f309c1585f0 0x7f309c0cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.200+0000 7f30a1020640 1 -- 172.21.15.26:0/463018940 >> 172.21.15.26:0/463018940 conn(0x7f309c066160 msgr2=0x7f309c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.200+0000 7f30a1020640 1 -- 172.21.15.26:0/463018940 shutdown_connections 2023-12-19T18:14:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.200+0000 7f30a1020640 1 -- 172.21.15.26:0/463018940 wait complete. 2023-12-19T18:14:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f30a1020640 1 Processor -- start 2023-12-19T18:14:29.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f30a1020640 1 -- start start 2023-12-19T18:14:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f30a1020640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f309c0cb150 0x7f309c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f30a1020640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f309c0cb5d0 0x7f309c154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f30a1020640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c1585f0 0x7f309c0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f30a1020640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0b37e0 con 0x7f309c0cb150 2023-12-19T18:14:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f30a1020640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0b2070 con 0x7f309c1585f0 2023-12-19T18:14:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f30a1020640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0b3a10 con 0x7f309c0cb5d0 2023-12-19T18:14:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f309ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f309c0cb150 0x7f309c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f309ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f309c0cb150 0x7f309c0b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39004/0 (socket says 172.21.15.26:39004) 2023-12-19T18:14:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f309ad9d640 1 -- 172.21.15.26:0/1209196824 learned_addr learned my addr 172.21.15.26:0/1209196824 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f309b59e640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c1585f0 0x7f309c0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f3093fff640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f309c0cb5d0 0x7f309c154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f309ad9d640 1 -- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f309c0cb5d0 msgr2=0x7f309c154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f309ad9d640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f309c0cb5d0 0x7f309c154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f309ad9d640 1 -- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c1585f0 msgr2=0x7f309c0b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f309ad9d640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c1585f0 0x7f309c0b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f309ad9d640 1 -- 172.21.15.26:0/1209196824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c009420 con 0x7f309c0cb150 2023-12-19T18:14:29.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f309ad9d640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f309c0cb150 0x7f309c0b04b0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f308c00e980 tx=0x7f308c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f30937fe640 1 -- 172.21.15.26:0/1209196824 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c00ca00 con 0x7f309c0cb150 2023-12-19T18:14:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f30937fe640 1 -- 172.21.15.26:0/1209196824 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c00cb60 con 0x7f309c0cb150 2023-12-19T18:14:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f30937fe640 1 -- 172.21.15.26:0/1209196824 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c013470 con 0x7f309c0cb150 2023-12-19T18:14:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f30a1020640 1 -- 172.21.15.26:0/1209196824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3094009b30 con 0x7f309c0cb150 2023-12-19T18:14:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.204+0000 7f30a1020640 1 -- 172.21.15.26:0/1209196824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c15c3e0 con 0x7f309c0cb150 2023-12-19T18:14:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.208+0000 7f30a1020640 1 -- 172.21.15.26:0/1209196824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c074270 con 0x7f309c0cb150 2023-12-19T18:14:29.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.208+0000 7f30937fe640 1 -- 172.21.15.26:0/1209196824 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f308c0135d0 con 0x7f309c0cb150 2023-12-19T18:14:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.208+0000 7f30937fe640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f307c065b80 0x7f307c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.208+0000 7f30937fe640 1 -- 172.21.15.26:0/1209196824 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6649+0+0 (secure 0 0 0) 0x7f308c0838c0 con 0x7f309c0cb150 2023-12-19T18:14:29.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.208+0000 7f3093fff640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f307c065b80 0x7f307c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.208+0000 7f3093fff640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f307c065b80 0x7f307c068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f309c1558f0 tx=0x7f3084009040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.212+0000 7f30937fe640 1 -- 172.21.15.26:0/1209196824 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f308c050e70 con 0x7f309c0cb150 2023-12-19T18:14:29.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.376+0000 7f30a1020640 1 -- 172.21.15.26:0/1209196824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f309c165800 con 0x7f309c0cb150 2023-12-19T18:14:29.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.380+0000 7f30937fe640 1 -- 172.21.15.26:0/1209196824 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v61) v1 ==== 120+0+0 (secure 0 0 0) 0x7f308c054490 con 0x7f309c0cb150 2023-12-19T18:14:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.384+0000 7f30a1020640 1 -- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f307c065b80 msgr2=0x7f307c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.384+0000 7f30a1020640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f307c065b80 0x7f307c068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f309c1558f0 tx=0x7f3084009040 comp rx=0 tx=0).stop 2023-12-19T18:14:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.384+0000 7f30a1020640 1 -- 172.21.15.26:0/1209196824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f309c0cb150 msgr2=0x7f309c0b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.384+0000 7f30a1020640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f309c0cb150 0x7f309c0b04b0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f308c00e980 tx=0x7f308c00ee40 comp rx=0 tx=0).stop 2023-12-19T18:14:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.384+0000 7f30a1020640 1 -- 172.21.15.26:0/1209196824 shutdown_connections 2023-12-19T18:14:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.384+0000 7f30a1020640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f307c065b80 0x7f307c068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.384+0000 7f30a1020640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f309c1585f0 0x7f309c0b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.384+0000 7f30a1020640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f309c0cb5d0 0x7f309c154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.384+0000 7f30a1020640 1 --2- 172.21.15.26:0/1209196824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f309c0cb150 0x7f309c0b04b0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.384+0000 7f30a1020640 1 -- 172.21.15.26:0/1209196824 >> 172.21.15.26:0/1209196824 conn(0x7f309c066160 msgr2=0x7f309c155ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.384+0000 7f30a1020640 1 -- 172.21.15.26:0/1209196824 shutdown_connections 2023-12-19T18:14:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.384+0000 7f30a1020640 1 -- 172.21.15.26:0/1209196824 wait complete. 2023-12-19T18:14:29.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:14:29.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.552+0000 7f7e59873640 1 Processor -- start 2023-12-19T18:14:29.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.552+0000 7f7e59873640 1 -- start start 2023-12-19T18:14:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.552+0000 7f7e59873640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7e5406ba20 0x7f7e54163fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.552+0000 7f7e59873640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e540cc310 con 0x7f7e54159090 2023-12-19T18:14:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.552+0000 7f7e59873640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e540c9eb0 con 0x7f7e540ca820 2023-12-19T18:14:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.552+0000 7f7e59873640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e540ca110 con 0x7f7e5406ba20 2023-12-19T18:14:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e52ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7e54159090 0x7f7e540c97a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39226/0 (socket says 172.21.15.26:39226) 2023-12-19T18:14:29.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e52ffd640 1 -- 172.21.15.26:0/4109542706 learned_addr learned my addr 172.21.15.26:0/4109542706 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:29.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e537fe640 1 --2- 172.21.15.26:0/4109542706 >> v2:172.21.15.160:3300/0 conn(0x7f7e5406ba20 0x7f7e54163fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056417381 0 0) 0x7f7e540cc310 con 0x7f7e54159090 2023-12-19T18:14:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e40003450 con 0x7f7e54159090 2023-12-19T18:14:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4255536354 0 0) 0x7f7e40003450 con 0x7f7e54159090 2023-12-19T18:14:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 >> v2:172.21.15.160:3300/0 conn(0x7f7e5406ba20 msgr2=0x7f7e54163fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 --2- 172.21.15.26:0/4109542706 >> v2:172.21.15.160:3300/0 conn(0x7f7e5406ba20 0x7f7e54163fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 >> v1:172.21.15.160:6789/0 conn(0x7f7e540ca820 legacy=0x7f7e5406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e540ca3f0 con 0x7f7e54159090 2023-12-19T18:14:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f7e44001680 con 0x7f7e54159090 2023-12-19T18:14:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 >> v1:172.21.15.26:6789/0 conn(0x7f7e54159090 legacy=0x7f7e540c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 --2- 172.21.15.26:0/4109542706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e40003a70 0x7f7e40003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 --2- 172.21.15.26:0/4109542706 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e40008860 0x7f7e4000ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 --2- 172.21.15.26:0/4109542706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e4000b490 0x7f7e4000d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540ca110 con 0x7f7e4000b490 2023-12-19T18:14:29.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4000e230 con 0x7f7e40003a70 2023-12-19T18:14:29.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4000e3f0 con 0x7f7e40008860 2023-12-19T18:14:29.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e537fe640 1 --2- 172.21.15.26:0/4109542706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e4000b490 0x7f7e4000d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e52ffd640 1 --2- 172.21.15.26:0/4109542706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e40003a70 0x7f7e40003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e4bfff640 1 --2- 172.21.15.26:0/4109542706 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e40008860 0x7f7e4000ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e537fe640 1 -- 172.21.15.26:0/4109542706 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e40008860 msgr2=0x7f7e4000ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:29.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e537fe640 1 --2- 172.21.15.26:0/4109542706 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e40008860 0x7f7e4000ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e537fe640 1 -- 172.21.15.26:0/4109542706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e40003a70 msgr2=0x7f7e40003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:29.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e537fe640 1 --2- 172.21.15.26:0/4109542706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e40003a70 0x7f7e40003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.556+0000 7f7e537fe640 1 -- 172.21.15.26:0/4109542706 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e40003450 con 0x7f7e4000b490 2023-12-19T18:14:29.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e537fe640 1 -- 172.21.15.26:0/4109542706 reap_dead start 2023-12-19T18:14:29.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e537fe640 1 --2- 172.21.15.26:0/4109542706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e4000b490 0x7f7e4000d950 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7e540ad7f0 tx=0x7f7e4c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d626d2b9c90259 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0a5c50 con 0x7f7e4000b490 2023-12-19T18:14:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c00f040 con 0x7f7e4000b490 2023-12-19T18:14:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e527fc640 1 -- 172.21.15.26:0/4109542706 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0039a0 con 0x7f7e4000b490 2023-12-19T18:14:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e59873640 1 -- 172.21.15.26:0/4109542706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e4000b490 msgr2=0x7f7e4000d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e59873640 1 --2- 172.21.15.26:0/4109542706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e4000b490 0x7f7e4000d950 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7e540ad7f0 tx=0x7f7e4c01cc40 comp rx=0 tx=0).stop 2023-12-19T18:14:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e59873640 1 -- 172.21.15.26:0/4109542706 shutdown_connections 2023-12-19T18:14:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e59873640 1 --2- 172.21.15.26:0/4109542706 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e4000b490 0x7f7e4000d950 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e59873640 1 -- 172.21.15.26:0/4109542706 >> 172.21.15.26:0/4109542706 conn(0x7f7e54066160 msgr2=0x7f7e540b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e59873640 1 -- 172.21.15.26:0/4109542706 shutdown_connections 2023-12-19T18:14:29.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e59873640 1 -- 172.21.15.26:0/4109542706 wait complete. 2023-12-19T18:14:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e59873640 1 Processor -- start 2023-12-19T18:14:29.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.560+0000 7f7e59873640 1 -- start start 2023-12-19T18:14:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e59873640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e4000b490 0x7f7e540c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e59873640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e540c8670 0x7f7e54070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e59873640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e54148980 0x7f7e54148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e59873640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e54149340 con 0x7f7e4000b490 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e59873640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540c7d00 con 0x7f7e54148980 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e59873640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e540c6c60 con 0x7f7e540c8670 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e52ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e4000b490 0x7f7e540c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e52ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e4000b490 0x7f7e540c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39028/0 (socket says 172.21.15.26:39028) 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e52ffd640 1 -- 172.21.15.26:0/1901222019 learned_addr learned my addr 172.21.15.26:0/1901222019 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e4bfff640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e540c8670 0x7f7e54070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e537fe640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e54148980 0x7f7e54148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e52ffd640 1 -- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e540c8670 msgr2=0x7f7e54070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e52ffd640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e540c8670 0x7f7e54070ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e52ffd640 1 -- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e54148980 msgr2=0x7f7e54148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e52ffd640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e54148980 0x7f7e54148e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e52ffd640 1 -- 172.21.15.26:0/1901222019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e540c6f40 con 0x7f7e4000b490 2023-12-19T18:14:29.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e52ffd640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e4000b490 0x7f7e540c6560 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7e44007290 tx=0x7f7e44007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:29.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e50ff9640 1 -- 172.21.15.26:0/1901222019 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4400f9c0 con 0x7f7e4000b490 2023-12-19T18:14:29.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e50ff9640 1 -- 172.21.15.26:0/1901222019 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4400fb40 con 0x7f7e4000b490 2023-12-19T18:14:29.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e50ff9640 1 -- 172.21.15.26:0/1901222019 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e440133f0 con 0x7f7e4000b490 2023-12-19T18:14:29.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e59873640 1 -- 172.21.15.26:0/1901222019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e540c7220 con 0x7f7e4000b490 2023-12-19T18:14:29.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e59873640 1 -- 172.21.15.26:0/1901222019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e540c77e0 con 0x7f7e4000b490 2023-12-19T18:14:29.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e59873640 1 -- 172.21.15.26:0/1901222019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e540cd780 con 0x7f7e4000b490 2023-12-19T18:14:29.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.564+0000 7f7e50ff9640 1 -- 172.21.15.26:0/1901222019 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7e44013550 con 0x7f7e4000b490 2023-12-19T18:14:29.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.568+0000 7f7e50ff9640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7e34065b80 0x7f7e34068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.568+0000 7f7e50ff9640 1 -- 172.21.15.26:0/1901222019 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6649+0+0 (secure 0 0 0) 0x7f7e44083720 con 0x7f7e4000b490 2023-12-19T18:14:29.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.568+0000 7f7e4bfff640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7e34065b80 0x7f7e34068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.568+0000 7f7e4bfff640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7e34065b80 0x7f7e34068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7e5406f750 tx=0x7f7e3c009040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:29.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.572+0000 7f7e50ff9640 1 -- 172.21.15.26:0/1901222019 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7e44050cd0 con 0x7f7e4000b490 2023-12-19T18:14:29.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.728+0000 7f7e59873640 1 -- 172.21.15.26:0/1901222019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7e54074270 con 0x7f7e4000b490 2023-12-19T18:14:29.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.732+0000 7f7e50ff9640 1 -- 172.21.15.26:0/1901222019 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7e440542f0 con 0x7f7e4000b490 2023-12-19T18:14:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.736+0000 7f7e59873640 1 -- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7e34065b80 msgr2=0x7f7e34068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.736+0000 7f7e59873640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7e34065b80 0x7f7e34068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7e5406f750 tx=0x7f7e3c009040 comp rx=0 tx=0).stop 2023-12-19T18:14:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.736+0000 7f7e59873640 1 -- 172.21.15.26:0/1901222019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e4000b490 msgr2=0x7f7e540c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.736+0000 7f7e59873640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e4000b490 0x7f7e540c6560 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7e44007290 tx=0x7f7e44007750 comp rx=0 tx=0).stop 2023-12-19T18:14:29.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.736+0000 7f7e59873640 1 -- 172.21.15.26:0/1901222019 shutdown_connections 2023-12-19T18:14:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.736+0000 7f7e59873640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7e34065b80 0x7f7e34068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.736+0000 7f7e59873640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e54148980 0x7f7e54148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.736+0000 7f7e59873640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e540c8670 0x7f7e54070ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.736+0000 7f7e59873640 1 --2- 172.21.15.26:0/1901222019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e4000b490 0x7f7e540c6560 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.736+0000 7f7e59873640 1 -- 172.21.15.26:0/1901222019 >> 172.21.15.26:0/1901222019 conn(0x7f7e54066160 msgr2=0x7f7e540cb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.736+0000 7f7e59873640 1 -- 172.21.15.26:0/1901222019 shutdown_connections 2023-12-19T18:14:29.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.736+0000 7f7e59873640 1 -- 172.21.15.26:0/1901222019 wait complete. 2023-12-19T18:14:29.756 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-19T18:14:29.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.900+0000 7f839e739640 1 Processor -- start 2023-12-19T18:14:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.900+0000 7f839e739640 1 -- start start 2023-12-19T18:14:29.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.900+0000 7f839e739640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83981585f0 0x7f83980cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.900+0000 7f839e739640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83980cd130 con 0x7f83980cb5d0 2023-12-19T18:14:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.900+0000 7f839e739640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839815b1b0 con 0x7f83980cb150 2023-12-19T18:14:29.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.900+0000 7f839e739640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83980b47a0 con 0x7f83981585f0 2023-12-19T18:14:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.900+0000 7f839ccd6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f83980cb5d0 0x7f83980cca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39242/0 (socket says 172.21.15.26:39242) 2023-12-19T18:14:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.900+0000 7f839ccd6640 1 -- 172.21.15.26:0/1746496328 learned_addr learned my addr 172.21.15.26:0/1746496328 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.900+0000 7f8397fff640 1 --2- 172.21.15.26:0/1746496328 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83981585f0 0x7f83980cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.900+0000 7f8396ffd640 1 -- 172.21.15.26:0/1746496328 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2213655892 0 0) 0x7f83980cd130 con 0x7f83980cb5d0 2023-12-19T18:14:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.900+0000 7f8396ffd640 1 -- 172.21.15.26:0/1746496328 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8380003530 con 0x7f83980cb5d0 2023-12-19T18:14:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.904+0000 7f8396ffd640 1 -- 172.21.15.26:0/1746496328 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130299295 0 0) 0x7f839815b1b0 con 0x7f83980cb150 2023-12-19T18:14:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.904+0000 7f8396ffd640 1 -- 172.21.15.26:0/1746496328 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83980cd130 con 0x7f83980cb150 2023-12-19T18:14:29.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.904+0000 7f8396ffd640 1 -- 172.21.15.26:0/1746496328 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3462881173 0 0) 0x7f8380003530 con 0x7f83980cb5d0 2023-12-19T18:14:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.904+0000 7f8397fff640 1 -- 172.21.15.26:0/1746496328 >> v1:172.21.15.160:6789/0 conn(0x7f83980cb150 legacy=0x7f83980cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.904+0000 7f8397fff640 1 -- 172.21.15.26:0/1746496328 >> v1:172.21.15.26:6789/0 conn(0x7f83980cb5d0 legacy=0x7f83980cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.904+0000 7f8397fff640 1 -- 172.21.15.26:0/1746496328 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839815b490 con 0x7f83981585f0 2023-12-19T18:14:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.904+0000 7f8397fff640 1 --2- 172.21.15.26:0/1746496328 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83981585f0 0x7f83980cdce0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f83880097f0 tx=0x7f838801c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c80b53869f075610 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.904+0000 7f8396ffd640 1 -- 172.21.15.26:0/1746496328 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8390001a00 con 0x7f83981585f0 2023-12-19T18:14:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.904+0000 7f8396ffd640 1 -- 172.21.15.26:0/1746496328 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83880a7050 con 0x7f83981585f0 2023-12-19T18:14:29.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.904+0000 7f8396ffd640 1 -- 172.21.15.26:0/1746496328 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83880ab620 con 0x7f83981585f0 2023-12-19T18:14:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 -- 172.21.15.26:0/1746496328 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83981585f0 msgr2=0x7f83980cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 --2- 172.21.15.26:0/1746496328 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83981585f0 0x7f83980cdce0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f83880097f0 tx=0x7f838801c320 comp rx=0 tx=0).stop 2023-12-19T18:14:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 -- 172.21.15.26:0/1746496328 shutdown_connections 2023-12-19T18:14:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 --2- 172.21.15.26:0/1746496328 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83981585f0 0x7f83980cdce0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 -- 172.21.15.26:0/1746496328 >> 172.21.15.26:0/1746496328 conn(0x7f8398066160 msgr2=0x7f83980b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:29.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 -- 172.21.15.26:0/1746496328 shutdown_connections 2023-12-19T18:14:29.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 -- 172.21.15.26:0/1746496328 wait complete. 2023-12-19T18:14:29.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 Processor -- start 2023-12-19T18:14:29.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 -- start start 2023-12-19T18:14:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83980cb150 0x7f83980ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83980cb5d0 0x7f839814b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83981585f0 0x7f8398150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839814b470 con 0x7f83980cb5d0 2023-12-19T18:14:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83980cae20 con 0x7f83980cb150 2023-12-19T18:14:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839e739640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8398159e60 con 0x7f83981585f0 2023-12-19T18:14:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f83977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83980cb5d0 0x7f839814b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f83977fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83980cb5d0 0x7f839814b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39042/0 (socket says 172.21.15.26:39042) 2023-12-19T18:14:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f83977fe640 1 -- 172.21.15.26:0/2308178154 learned_addr learned my addr 172.21.15.26:0/2308178154 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839ccd6640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83981585f0 0x7f8398150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f8397fff640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83980cb150 0x7f83980ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f83977fe640 1 -- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83981585f0 msgr2=0x7f8398150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f83977fe640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83981585f0 0x7f8398150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f83977fe640 1 -- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83980cb150 msgr2=0x7f83980ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f83977fe640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83980cb150 0x7f83980ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f83977fe640 1 -- 172.21.15.26:0/2308178154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8388009420 con 0x7f83980cb5d0 2023-12-19T18:14:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f839ccd6640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83981585f0 0x7f8398150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.908+0000 7f8397fff640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83980cb150 0x7f83980ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:29.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.912+0000 7f83977fe640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83980cb5d0 0x7f839814b890 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c490 tx=0x7f838c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:29.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.912+0000 7f8394ff9640 1 -- 172.21.15.26:0/2308178154 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c0052d0 con 0x7f83980cb5d0 2023-12-19T18:14:29.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.912+0000 7f8394ff9640 1 -- 172.21.15.26:0/2308178154 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c005450 con 0x7f83980cb5d0 2023-12-19T18:14:29.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.912+0000 7f8394ff9640 1 -- 172.21.15.26:0/2308178154 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c012870 con 0x7f83980cb5d0 2023-12-19T18:14:29.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.912+0000 7f839e739640 1 -- 172.21.15.26:0/2308178154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83980ca8c0 con 0x7f83980cb5d0 2023-12-19T18:14:29.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.912+0000 7f839e739640 1 -- 172.21.15.26:0/2308178154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83980cab40 con 0x7f83980cb5d0 2023-12-19T18:14:29.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.912+0000 7f839e739640 1 -- 172.21.15.26:0/2308178154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839806d210 con 0x7f83980cb5d0 2023-12-19T18:14:29.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.912+0000 7f8394ff9640 1 -- 172.21.15.26:0/2308178154 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f838c0129d0 con 0x7f83980cb5d0 2023-12-19T18:14:29.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.912+0000 7f8394ff9640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8384065bb0 0x7f8384068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:29.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.912+0000 7f8394ff9640 1 -- 172.21.15.26:0/2308178154 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6649+0+0 (secure 0 0 0) 0x7f838c083260 con 0x7f83980cb5d0 2023-12-19T18:14:29.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.916+0000 7f8397fff640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8384065bb0 0x7f8384068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:29.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.916+0000 7f8397fff640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8384065bb0 0x7f8384068070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f83880ad000 tx=0x7f83880a6000 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:29.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:29.916+0000 7f8394ff9640 1 -- 172.21.15.26:0/2308178154 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f838c0506d0 con 0x7f83980cb5d0 2023-12-19T18:14:30.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.076+0000 7f839e739640 1 -- 172.21.15.26:0/2308178154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f839814ac50 con 0x7f83980cb5d0 2023-12-19T18:14:30.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.088+0000 7f8394ff9640 1 -- 172.21.15.26:0/2308178154 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v63) v1 ==== 115+0+0 (secure 0 0 0) 0x7f839814ac50 con 0x7f83980cb5d0 2023-12-19T18:14:30.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.088+0000 7f839e739640 1 -- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8384065bb0 msgr2=0x7f8384068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:30.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.088+0000 7f839e739640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8384065bb0 0x7f8384068070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f83880ad000 tx=0x7f83880a6000 comp rx=0 tx=0).stop 2023-12-19T18:14:30.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.088+0000 7f839e739640 1 -- 172.21.15.26:0/2308178154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83980cb5d0 msgr2=0x7f839814b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:30.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.088+0000 7f839e739640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83980cb5d0 0x7f839814b890 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c490 tx=0x7f838c00c950 comp rx=0 tx=0).stop 2023-12-19T18:14:30.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.092+0000 7f839e739640 1 -- 172.21.15.26:0/2308178154 shutdown_connections 2023-12-19T18:14:30.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.092+0000 7f839e739640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8384065bb0 0x7f8384068070 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:30.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.092+0000 7f839e739640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83981585f0 0x7f8398150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:30.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.092+0000 7f839e739640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83980cb5d0 0x7f839814b890 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:30.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.092+0000 7f839e739640 1 --2- 172.21.15.26:0/2308178154 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83980cb150 0x7f83980ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:30.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.092+0000 7f839e739640 1 -- 172.21.15.26:0/2308178154 >> 172.21.15.26:0/2308178154 conn(0x7f8398066160 msgr2=0x7f83980cc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:30.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.092+0000 7f839e739640 1 -- 172.21.15.26:0/2308178154 shutdown_connections 2023-12-19T18:14:30.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.092+0000 7f839e739640 1 -- 172.21.15.26:0/2308178154 wait complete. 2023-12-19T18:14:30.112 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-12-19T18:14:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f0771739640 1 Processor -- start 2023-12-19T18:14:30.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f0771739640 1 -- start start 2023-12-19T18:14:30.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f0771739640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f076c0cda40 0x7f076c06afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:30.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f0771739640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f076c06b510 con 0x7f076c0b25d0 2023-12-19T18:14:30.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f0771739640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f076c06b790 con 0x7f076c0d1290 2023-12-19T18:14:30.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f0771739640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076c06abb0 con 0x7f076c0cda40 2023-12-19T18:14:30.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f0763fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f076c0b25d0 0x7f076c0cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39246/0 (socket says 172.21.15.26:39246) 2023-12-19T18:14:30.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f0763fff640 1 -- 172.21.15.26:0/3437442434 learned_addr learned my addr 172.21.15.26:0/3437442434 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:30.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f076b7fe640 1 --2- 172.21.15.26:0/3437442434 >> v2:172.21.15.160:3300/0 conn(0x7f076c0cda40 0x7f076c06afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f076a7fc640 1 -- 172.21.15.26:0/3437442434 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880785888 0 0) 0x7f076c06b510 con 0x7f076c0b25d0 2023-12-19T18:14:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f076a7fc640 1 -- 172.21.15.26:0/3437442434 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0758003530 con 0x7f076c0b25d0 2023-12-19T18:14:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f076a7fc640 1 -- 172.21.15.26:0/3437442434 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045449644 0 0) 0x7f076c06b790 con 0x7f076c0d1290 2023-12-19T18:14:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f076a7fc640 1 -- 172.21.15.26:0/3437442434 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f076c06b510 con 0x7f076c0d1290 2023-12-19T18:14:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f076b7fe640 1 -- 172.21.15.26:0/3437442434 >> v1:172.21.15.160:6789/0 conn(0x7f076c0d1290 legacy=0x7f076c0b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f076b7fe640 1 -- 172.21.15.26:0/3437442434 >> v1:172.21.15.26:6789/0 conn(0x7f076c0b25d0 legacy=0x7f076c0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:30.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.256+0000 7f076b7fe640 1 -- 172.21.15.26:0/3437442434 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c06bce0 con 0x7f076c0cda40 2023-12-19T18:14:30.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.260+0000 7f076b7fe640 1 --2- 172.21.15.26:0/3437442434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f076c0cda40 0x7f076c06afd0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0764009d90 tx=0x7f076401c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=337e6efb15573dac server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:30.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.260+0000 7f076a7fc640 1 -- 172.21.15.26:0/3437442434 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07640a5870 con 0x7f076c0cda40 2023-12-19T18:14:30.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.260+0000 7f076a7fc640 1 -- 172.21.15.26:0/3437442434 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07640a59d0 con 0x7f076c0cda40 2023-12-19T18:14:30.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.260+0000 7f076a7fc640 1 -- 172.21.15.26:0/3437442434 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07640a5d60 con 0x7f076c0cda40 2023-12-19T18:14:30.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.260+0000 7f0771739640 1 -- 172.21.15.26:0/3437442434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f076c0cda40 msgr2=0x7f076c06afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:30.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.260+0000 7f0771739640 1 --2- 172.21.15.26:0/3437442434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f076c0cda40 0x7f076c06afd0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0764009d90 tx=0x7f076401c8c0 comp rx=0 tx=0).stop 2023-12-19T18:14:30.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.260+0000 7f0771739640 1 -- 172.21.15.26:0/3437442434 shutdown_connections 2023-12-19T18:14:30.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.260+0000 7f0771739640 1 --2- 172.21.15.26:0/3437442434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f076c0cda40 0x7f076c06afd0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:30.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.260+0000 7f0771739640 1 -- 172.21.15.26:0/3437442434 >> 172.21.15.26:0/3437442434 conn(0x7f076c06d570 msgr2=0x7f076c06d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:30.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.260+0000 7f0771739640 1 -- 172.21.15.26:0/3437442434 shutdown_connections 2023-12-19T18:14:30.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.260+0000 7f0771739640 1 -- 172.21.15.26:0/3437442434 wait complete. 2023-12-19T18:14:30.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f0771739640 1 Processor -- start 2023-12-19T18:14:30.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f0771739640 1 -- start start 2023-12-19T18:14:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f0771739640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f076c0b25d0 0x7f076c157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f0771739640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f076c0cda40 0x7f076c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f0771739640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f076c0d1290 0x7f076c155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f0771739640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c1428c0 con 0x7f076c0b25d0 2023-12-19T18:14:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f0771739640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c158310 con 0x7f076c0d1290 2023-12-19T18:14:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f0771739640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076c141cb0 con 0x7f076c0cda40 2023-12-19T18:14:30.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f076affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f076c0b25d0 0x7f076c157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:30.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f076affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f076c0b25d0 0x7f076c157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39046/0 (socket says 172.21.15.26:39046) 2023-12-19T18:14:30.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f076affd640 1 -- 172.21.15.26:0/3393479398 learned_addr learned my addr 172.21.15.26:0/3393479398 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:30.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f0763fff640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f076c0cda40 0x7f076c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:30.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f076b7fe640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f076c0d1290 0x7f076c155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:30.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f076affd640 1 -- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f076c0cda40 msgr2=0x7f076c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:30.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f076affd640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f076c0cda40 0x7f076c15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:30.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f076affd640 1 -- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f076c0d1290 msgr2=0x7f076c155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:30.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f076affd640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f076c0d1290 0x7f076c155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:30.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f076affd640 1 -- 172.21.15.26:0/3393479398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0764009a70 con 0x7f076c0b25d0 2023-12-19T18:14:30.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f076b7fe640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f076c0d1290 0x7f076c155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:30.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f076affd640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f076c0b25d0 0x7f076c157850 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f075c00c430 tx=0x7f075c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:30.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f07637fe640 1 -- 172.21.15.26:0/3393479398 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c0052d0 con 0x7f076c0b25d0 2023-12-19T18:14:30.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f07637fe640 1 -- 172.21.15.26:0/3393479398 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c005450 con 0x7f076c0b25d0 2023-12-19T18:14:30.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f07637fe640 1 -- 172.21.15.26:0/3393479398 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c003840 con 0x7f076c0b25d0 2023-12-19T18:14:30.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f0771739640 1 -- 172.21.15.26:0/3393479398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076c141f90 con 0x7f076c0b25d0 2023-12-19T18:14:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.264+0000 7f0771739640 1 -- 172.21.15.26:0/3393479398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076c142550 con 0x7f076c0b25d0 2023-12-19T18:14:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.268+0000 7f0771739640 1 -- 172.21.15.26:0/3393479398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076c142710 con 0x7f076c0b25d0 2023-12-19T18:14:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.268+0000 7f07637fe640 1 -- 172.21.15.26:0/3393479398 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f075c0039a0 con 0x7f076c0b25d0 2023-12-19T18:14:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.268+0000 7f07637fe640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f074c065b80 0x7f074c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:30.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.268+0000 7f07637fe640 1 -- 172.21.15.26:0/3393479398 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6649+0+0 (secure 0 0 0) 0x7f075c082db0 con 0x7f076c0b25d0 2023-12-19T18:14:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.268+0000 7f0763fff640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f074c065b80 0x7f074c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:30.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.268+0000 7f0763fff640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f074c065b80 0x7f074c068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f076c0722f0 tx=0x7f0754009040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:30.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.272+0000 7f07637fe640 1 -- 172.21.15.26:0/3393479398 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f075c087070 con 0x7f076c0b25d0 2023-12-19T18:14:30.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:30.484+0000 7f0771739640 1 -- 172.21.15.26:0/3393479398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f076c074270 con 0x7f076c0b25d0 2023-12-19T18:14:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.080+0000 7f07637fe640 1 -- 172.21.15.26:0/3393479398 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2959) v1 ==== 165+0+0 (secure 0 0 0) 0x7f075c0170e0 con 0x7f076c0b25d0 2023-12-19T18:14:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.084+0000 7f0771739640 1 -- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f074c065b80 msgr2=0x7f074c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.084+0000 7f0771739640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f074c065b80 0x7f074c068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f076c0722f0 tx=0x7f0754009040 comp rx=0 tx=0).stop 2023-12-19T18:14:31.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.084+0000 7f0771739640 1 -- 172.21.15.26:0/3393479398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f076c0b25d0 msgr2=0x7f076c157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.084+0000 7f0771739640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f076c0b25d0 0x7f076c157850 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f075c00c430 tx=0x7f075c00c8f0 comp rx=0 tx=0).stop 2023-12-19T18:14:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.084+0000 7f0771739640 1 -- 172.21.15.26:0/3393479398 shutdown_connections 2023-12-19T18:14:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.084+0000 7f0771739640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f074c065b80 0x7f074c068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.084+0000 7f0771739640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f076c0d1290 0x7f076c155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.084+0000 7f0771739640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f076c0cda40 0x7f076c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.084+0000 7f0771739640 1 --2- 172.21.15.26:0/3393479398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f076c0b25d0 0x7f076c157850 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.084+0000 7f0771739640 1 -- 172.21.15.26:0/3393479398 >> 172.21.15.26:0/3393479398 conn(0x7f076c06d570 msgr2=0x7f076c15f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.084+0000 7f0771739640 1 -- 172.21.15.26:0/3393479398 shutdown_connections 2023-12-19T18:14:31.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.084+0000 7f0771739640 1 -- 172.21.15.26:0/3393479398 wait complete. 2023-12-19T18:14:31.108 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-12-19T18:14:31.109 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-12-19T18:14:31.109 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-19T18:14:31.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-12-19T18:14:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe415ff3640 1 Processor -- start 2023-12-19T18:14:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe415ff3640 1 -- start start 2023-12-19T18:14:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe415ff3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe4100b85e0 0x7fe4100cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe415ff3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4100d0d40 con 0x7fe4100cde60 2023-12-19T18:14:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe415ff3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe410161150 con 0x7fe41015cda0 2023-12-19T18:14:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe415ff3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4100ce2d0 con 0x7fe4100b85e0 2023-12-19T18:14:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe40effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe4100b85e0 0x7fe4100cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe40ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe4100cde60 0x7fe4100d1520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39250/0 (socket says 172.21.15.26:39250) 2023-12-19T18:14:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe40effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe4100b85e0 0x7fe4100cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39056/0 (socket says 172.21.15.26:39056) 2023-12-19T18:14:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe40ffff640 1 -- 172.21.15.26:0/2715010926 learned_addr learned my addr 172.21.15.26:0/2715010926 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe40effd640 1 -- 172.21.15.26:0/2715010926 >> v1:172.21.15.160:6789/0 conn(0x7fe41015cda0 legacy=0x7fe410160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe40effd640 1 -- 172.21.15.26:0/2715010926 >> v1:172.21.15.26:6789/0 conn(0x7fe4100cde60 legacy=0x7fe4100d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe40effd640 1 -- 172.21.15.26:0/2715010926 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe410161430 con 0x7fe4100b85e0 2023-12-19T18:14:31.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.256+0000 7fe40e7fc640 1 -- 172.21.15.26:0/2715010926 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048618022 0 0) 0x7fe4100d0d40 con 0x7fe4100cde60 2023-12-19T18:14:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.260+0000 7fe40effd640 1 --2- 172.21.15.26:0/2715010926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4100b85e0 0x7fe4100cd920 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fe4040097a0 tx=0x7fe40401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4234dfcf31d0e958 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.260+0000 7fe40e7fc640 1 -- 172.21.15.26:0/2715010926 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4040ab5f0 con 0x7fe4100b85e0 2023-12-19T18:14:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.260+0000 7fe40e7fc640 1 -- 172.21.15.26:0/2715010926 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4040ab750 con 0x7fe4100b85e0 2023-12-19T18:14:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.260+0000 7fe40e7fc640 1 -- 172.21.15.26:0/2715010926 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4040abae0 con 0x7fe4100b85e0 2023-12-19T18:14:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.260+0000 7fe415ff3640 1 -- 172.21.15.26:0/2715010926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4100b85e0 msgr2=0x7fe4100cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.260+0000 7fe415ff3640 1 --2- 172.21.15.26:0/2715010926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4100b85e0 0x7fe4100cd920 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fe4040097a0 tx=0x7fe40401c3f0 comp rx=0 tx=0).stop 2023-12-19T18:14:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.260+0000 7fe415ff3640 1 -- 172.21.15.26:0/2715010926 shutdown_connections 2023-12-19T18:14:31.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.260+0000 7fe415ff3640 1 --2- 172.21.15.26:0/2715010926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4100b85e0 0x7fe4100cd920 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:31.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.260+0000 7fe415ff3640 1 -- 172.21.15.26:0/2715010926 >> 172.21.15.26:0/2715010926 conn(0x7fe4100658e0 msgr2=0x7fe410065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:31.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.260+0000 7fe415ff3640 1 -- 172.21.15.26:0/2715010926 shutdown_connections 2023-12-19T18:14:31.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.260+0000 7fe415ff3640 1 -- 172.21.15.26:0/2715010926 wait complete. 2023-12-19T18:14:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe415ff3640 1 Processor -- start 2023-12-19T18:14:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe415ff3640 1 -- start start 2023-12-19T18:14:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe415ff3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4100b85e0 0x7fe4100c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe415ff3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4100cde60 0x7fe4100c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe415ff3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe41015cda0 0x7fe410148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe415ff3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100c8b70 con 0x7fe4100b85e0 2023-12-19T18:14:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe415ff3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100c8210 con 0x7fe41015cda0 2023-12-19T18:14:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe415ff3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100c84b0 con 0x7fe4100cde60 2023-12-19T18:14:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe40f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4100b85e0 0x7fe4100c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe40f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4100b85e0 0x7fe4100c6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39058/0 (socket says 172.21.15.26:39058) 2023-12-19T18:14:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe40f7fe640 1 -- 172.21.15.26:0/4181606563 learned_addr learned my addr 172.21.15.26:0/4181606563 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe40effd640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4100cde60 0x7fe4100c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe40ffff640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe41015cda0 0x7fe410148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe40f7fe640 1 -- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4100cde60 msgr2=0x7fe4100c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe40f7fe640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4100cde60 0x7fe4100c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe40f7fe640 1 -- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe41015cda0 msgr2=0x7fe410148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe40f7fe640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe41015cda0 0x7fe410148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe40f7fe640 1 -- 172.21.15.26:0/4181606563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4040093b0 con 0x7fe4100b85e0 2023-12-19T18:14:31.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe40ffff640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe41015cda0 0x7fe410148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:31.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe40f7fe640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4100b85e0 0x7fe4100c6780 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe40000b410 tx=0x7fe40000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:31.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe3f7fff640 1 -- 172.21.15.26:0/4181606563 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400004020 con 0x7fe4100b85e0 2023-12-19T18:14:31.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe3f7fff640 1 -- 172.21.15.26:0/4181606563 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe400004180 con 0x7fe4100b85e0 2023-12-19T18:14:31.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe3f7fff640 1 -- 172.21.15.26:0/4181606563 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400011630 con 0x7fe4100b85e0 2023-12-19T18:14:31.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe415ff3640 1 -- 172.21.15.26:0/4181606563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4100c8790 con 0x7fe4100b85e0 2023-12-19T18:14:31.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.264+0000 7fe415ff3640 1 -- 172.21.15.26:0/4181606563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4100c94d0 con 0x7fe4100b85e0 2023-12-19T18:14:31.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.268+0000 7fe415ff3640 1 -- 172.21.15.26:0/4181606563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe410074270 con 0x7fe4100b85e0 2023-12-19T18:14:31.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.268+0000 7fe3f7fff640 1 -- 172.21.15.26:0/4181606563 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe400011790 con 0x7fe4100b85e0 2023-12-19T18:14:31.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.268+0000 7fe3f7fff640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe3fc065b80 0x7fe3fc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:31.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.268+0000 7fe3f7fff640 1 -- 172.21.15.26:0/4181606563 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6649+0+0 (secure 0 0 0) 0x7fe4000833e0 con 0x7fe4100b85e0 2023-12-19T18:14:31.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.268+0000 7fe40effd640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe3fc065b80 0x7fe3fc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:31.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.272+0000 7fe40effd640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe3fc065b80 0x7fe3fc068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe4040af000 tx=0x7fe4040a7000 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:31.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.272+0000 7fe3f7fff640 1 -- 172.21.15.26:0/4181606563 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe4000508e0 con 0x7fe4100b85e0 2023-12-19T18:14:31.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:31.484+0000 7fe415ff3640 1 -- 172.21.15.26:0/4181606563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7fe410156e20 con 0x7fe4100b85e0 2023-12-19T18:14:32.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.088+0000 7fe3f7fff640 1 -- 172.21.15.26:0/4181606563 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v2960) v1 ==== 166+0+0 (secure 0 0 0) 0x7fe400053f00 con 0x7fe4100b85e0 2023-12-19T18:14:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.092+0000 7fe415ff3640 1 -- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe3fc065b80 msgr2=0x7fe3fc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.092+0000 7fe415ff3640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe3fc065b80 0x7fe3fc068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe4040af000 tx=0x7fe4040a7000 comp rx=0 tx=0).stop 2023-12-19T18:14:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.092+0000 7fe415ff3640 1 -- 172.21.15.26:0/4181606563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4100b85e0 msgr2=0x7fe4100c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.092+0000 7fe415ff3640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4100b85e0 0x7fe4100c6780 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe40000b410 tx=0x7fe40000b8d0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.092+0000 7fe415ff3640 1 -- 172.21.15.26:0/4181606563 shutdown_connections 2023-12-19T18:14:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.092+0000 7fe415ff3640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe3fc065b80 0x7fe3fc068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.092+0000 7fe415ff3640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe41015cda0 0x7fe410148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.092+0000 7fe415ff3640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe4100cde60 0x7fe4100c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.092+0000 7fe415ff3640 1 --2- 172.21.15.26:0/4181606563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe4100b85e0 0x7fe4100c6780 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.092+0000 7fe415ff3640 1 -- 172.21.15.26:0/4181606563 >> 172.21.15.26:0/4181606563 conn(0x7fe4100658e0 msgr2=0x7fe41015be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.092+0000 7fe415ff3640 1 -- 172.21.15.26:0/4181606563 shutdown_connections 2023-12-19T18:14:32.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.092+0000 7fe415ff3640 1 -- 172.21.15.26:0/4181606563 wait complete. 2023-12-19T18:14:32.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:14:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.264+0000 7faa71612640 1 Processor -- start 2023-12-19T18:14:32.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.264+0000 7faa71612640 1 -- start start 2023-12-19T18:14:32.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa71612640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c06ac80 0x7faa6c06b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa71612640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6c06b620 con 0x7faa6c159140 2023-12-19T18:14:32.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa71612640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6c06f490 con 0x7faa6c06f920 2023-12-19T18:14:32.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa71612640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c159520 con 0x7faa6c06ac80 2023-12-19T18:14:32.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faa6c159140 0x7faa6c0ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39264/0 (socket says 172.21.15.26:39264) 2023-12-19T18:14:32.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6affd640 1 -- 172.21.15.26:0/3343051418 learned_addr learned my addr 172.21.15.26:0/3343051418 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:32.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa63fff640 1 --2- 172.21.15.26:0/3343051418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c06ac80 0x7faa6c06b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111997369 0 0) 0x7faa6c06b620 con 0x7faa6c159140 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa58003530 con 0x7faa6c159140 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 83342082 0 0) 0x7faa58003530 con 0x7faa6c159140 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c06ac80 msgr2=0x7faa6c06b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 --2- 172.21.15.26:0/3343051418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c06ac80 0x7faa6c06b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 >> v1:172.21.15.160:6789/0 conn(0x7faa6c06f920 legacy=0x7faa6c06fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c0cd690 con 0x7faa6c159140 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7faa5c001680 con 0x7faa6c159140 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 >> v1:172.21.15.26:6789/0 conn(0x7faa6c159140 legacy=0x7faa6c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 --2- 172.21.15.26:0/3343051418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa580039b0 0x7faa58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 --2- 172.21.15.26:0/3343051418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c06ac80 0x7faa580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 --2- 172.21.15.26:0/3343051418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa58004d10 0x7faa58005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c159520 con 0x7faa58004d10 2023-12-19T18:14:32.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5800cb80 con 0x7faa580039b0 2023-12-19T18:14:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5800cd40 con 0x7faa6c06ac80 2023-12-19T18:14:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6b7fe640 1 --2- 172.21.15.26:0/3343051418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa58004d10 0x7faa58005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa63fff640 1 --2- 172.21.15.26:0/3343051418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c06ac80 0x7faa580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa6affd640 1 --2- 172.21.15.26:0/3343051418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa580039b0 0x7faa58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa63fff640 1 -- 172.21.15.26:0/3343051418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa580039b0 msgr2=0x7faa58003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa63fff640 1 --2- 172.21.15.26:0/3343051418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa580039b0 0x7faa58003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa63fff640 1 -- 172.21.15.26:0/3343051418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa58004d10 msgr2=0x7faa58005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa63fff640 1 --2- 172.21.15.26:0/3343051418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa58004d10 0x7faa58005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.268+0000 7faa63fff640 1 -- 172.21.15.26:0/3343051418 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa58003530 con 0x7faa6c06ac80 2023-12-19T18:14:32.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.272+0000 7faa63fff640 1 --2- 172.21.15.26:0/3343051418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c06ac80 0x7faa580047d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faa54009670 tx=0x7faa5401ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0890dcbc9dc4f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:32.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.272+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa540a7910 con 0x7faa6c06ac80 2023-12-19T18:14:32.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.272+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa540a7a70 con 0x7faa6c06ac80 2023-12-19T18:14:32.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.272+0000 7faa6a7fc640 1 -- 172.21.15.26:0/3343051418 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa540a7e20 con 0x7faa6c06ac80 2023-12-19T18:14:32.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.272+0000 7faa71612640 1 -- 172.21.15.26:0/3343051418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c06ac80 msgr2=0x7faa580047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:32.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.272+0000 7faa71612640 1 --2- 172.21.15.26:0/3343051418 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c06ac80 0x7faa580047d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faa54009670 tx=0x7faa5401ed10 comp rx=0 tx=0).stop 2023-12-19T18:14:32.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.272+0000 7faa6b7fe640 1 -- 172.21.15.26:0/3343051418 reap_dead start 2023-12-19T18:14:32.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.272+0000 7faa71612640 1 -- 172.21.15.26:0/3343051418 shutdown_connections 2023-12-19T18:14:32.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.272+0000 7faa71612640 1 -- 172.21.15.26:0/3343051418 >> 172.21.15.26:0/3343051418 conn(0x7faa6c0ce730 msgr2=0x7faa6c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:32.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.272+0000 7faa71612640 1 -- 172.21.15.26:0/3343051418 shutdown_connections 2023-12-19T18:14:32.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.272+0000 7faa71612640 1 -- 172.21.15.26:0/3343051418 wait complete. 2023-12-19T18:14:32.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa71612640 1 Processor -- start 2023-12-19T18:14:32.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa71612640 1 -- start start 2023-12-19T18:14:32.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa71612640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa6c06ac80 0x7faa6c0b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa71612640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa6c157fb0 0x7faa6c1f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa71612640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c0ac860 0x7faa6c0acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa71612640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c155cf0 con 0x7faa6c157fb0 2023-12-19T18:14:32.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa71612640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0ad220 con 0x7faa6c06ac80 2023-12-19T18:14:32.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa71612640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0b1710 con 0x7faa6c0ac860 2023-12-19T18:14:32.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa63fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa6c157fb0 0x7faa6c1f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa63fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa6c157fb0 0x7faa6c1f2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39072/0 (socket says 172.21.15.26:39072) 2023-12-19T18:14:32.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa63fff640 1 -- 172.21.15.26:0/217944068 learned_addr learned my addr 172.21.15.26:0/217944068 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:32.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa6affd640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa6c06ac80 0x7faa6c0b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa6b7fe640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c0ac860 0x7faa6c0acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa63fff640 1 -- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c0ac860 msgr2=0x7faa6c0acce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:32.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa63fff640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c0ac860 0x7faa6c0acce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa63fff640 1 -- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa6c06ac80 msgr2=0x7faa6c0b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:32.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa63fff640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa6c06ac80 0x7faa6c0b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa63fff640 1 -- 172.21.15.26:0/217944068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c0b40e0 con 0x7faa6c157fb0 2023-12-19T18:14:32.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa63fff640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa6c157fb0 0x7faa6c1f2e70 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7faa54025240 tx=0x7faa58005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:32.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa637fe640 1 -- 172.21.15.26:0/217944068 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54004230 con 0x7faa6c157fb0 2023-12-19T18:14:32.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa637fe640 1 -- 172.21.15.26:0/217944068 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa54004390 con 0x7faa6c157fb0 2023-12-19T18:14:32.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa71612640 1 -- 172.21.15.26:0/217944068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6c0b3e00 con 0x7faa6c157fb0 2023-12-19T18:14:32.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa637fe640 1 -- 172.21.15.26:0/217944068 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54003820 con 0x7faa6c157fb0 2023-12-19T18:14:32.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.276+0000 7faa71612640 1 -- 172.21.15.26:0/217944068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6c155910 con 0x7faa6c157fb0 2023-12-19T18:14:32.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.280+0000 7faa71612640 1 -- 172.21.15.26:0/217944068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6c155e30 con 0x7faa6c157fb0 2023-12-19T18:14:32.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.280+0000 7faa637fe640 1 -- 172.21.15.26:0/217944068 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faa54003980 con 0x7faa6c157fb0 2023-12-19T18:14:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.280+0000 7faa637fe640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faa4c065b80 0x7faa4c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.280+0000 7faa637fe640 1 -- 172.21.15.26:0/217944068 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6649+0+0 (secure 0 0 0) 0x7faa5411d940 con 0x7faa6c157fb0 2023-12-19T18:14:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.280+0000 7faa6affd640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faa4c065b80 0x7faa4c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.280+0000 7faa6affd640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faa4c065b80 0x7faa4c068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0017b0 tx=0x7faa5c001440 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:32.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.284+0000 7faa637fe640 1 -- 172.21.15.26:0/217944068 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faa6c155e30 con 0x7faa6c157fb0 2023-12-19T18:14:32.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.444+0000 7faa71612640 1 -- 172.21.15.26:0/217944068 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faa6c155e30 con 0x7faa6c157fb0 2023-12-19T18:14:32.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.448+0000 7faa637fe640 1 -- 172.21.15.26:0/217944068 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa6c155e30 con 0x7faa6c157fb0 2023-12-19T18:14:32.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.452+0000 7faa71612640 1 -- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faa4c065b80 msgr2=0x7faa4c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:32.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.452+0000 7faa71612640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faa4c065b80 0x7faa4c068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faa5c0017b0 tx=0x7faa5c001440 comp rx=0 tx=0).stop 2023-12-19T18:14:32.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.452+0000 7faa71612640 1 -- 172.21.15.26:0/217944068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa6c157fb0 msgr2=0x7faa6c1f2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:32.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.452+0000 7faa71612640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa6c157fb0 0x7faa6c1f2e70 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7faa54025240 tx=0x7faa58005800 comp rx=0 tx=0).stop 2023-12-19T18:14:32.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.452+0000 7faa71612640 1 -- 172.21.15.26:0/217944068 shutdown_connections 2023-12-19T18:14:32.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.452+0000 7faa71612640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faa4c065b80 0x7faa4c068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.452+0000 7faa71612640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa6c0ac860 0x7faa6c0acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.452+0000 7faa71612640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa6c157fb0 0x7faa6c1f2e70 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.452+0000 7faa71612640 1 --2- 172.21.15.26:0/217944068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa6c06ac80 0x7faa6c0b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.452+0000 7faa71612640 1 -- 172.21.15.26:0/217944068 >> 172.21.15.26:0/217944068 conn(0x7faa6c0ce730 msgr2=0x7faa6c0c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:32.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.452+0000 7faa71612640 1 -- 172.21.15.26:0/217944068 shutdown_connections 2023-12-19T18:14:32.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.452+0000 7faa71612640 1 -- 172.21.15.26:0/217944068 wait complete. 2023-12-19T18:14:32.471 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:14:32.472 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:14:32.562 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:14:32.639 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:14:32.639 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:14:32.639 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:14:32.649 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:14:32.649 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:14:32.649 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:14:32.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:14:32.659 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:14:32.659 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:14:32.702 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:14:32.702 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:14:32.702 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:14:32.716 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:14:32.716 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:14:32.716 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:14:32.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.884+0000 7f7f84be9640 1 Processor -- start 2023-12-19T18:14:32.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.884+0000 7f7f84be9640 1 -- start start 2023-12-19T18:14:32.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f84be9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f80154220 0x7f7f8015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f84be9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7f8006b3d0 0x7f7f800ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f84be9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7f800cec40 0x7f7f8015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f84be9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f800cf0d0 con 0x7f7f8006b3d0 2023-12-19T18:14:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f84be9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8006bc30 con 0x7f7f800cec40 2023-12-19T18:14:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f84be9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f800cc6e0 con 0x7f7f80154220 2023-12-19T18:14:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f7dd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7f8006b3d0 0x7f7f800ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f7dd9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7f8006b3d0 0x7f7f800ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39088/0 (socket says 172.21.15.26:39088) 2023-12-19T18:14:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f7dd9b640 1 -- 172.21.15.26:0/568672889 learned_addr learned my addr 172.21.15.26:0/568672889 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f7ed9d640 1 --2- 172.21.15.26:0/568672889 >> v2:172.21.15.160:3300/0 conn(0x7f7f800cec40 0x7f7f8015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f7e59c640 1 --2- 172.21.15.26:0/568672889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f80154220 0x7f7f8015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f7dd9b640 1 -- 172.21.15.26:0/568672889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f80154220 msgr2=0x7f7f8015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f7dd9b640 1 --2- 172.21.15.26:0/568672889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f80154220 0x7f7f8015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f7dd9b640 1 -- 172.21.15.26:0/568672889 >> v2:172.21.15.160:3300/0 conn(0x7f7f800cec40 msgr2=0x7f7f8015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f7dd9b640 1 --2- 172.21.15.26:0/568672889 >> v2:172.21.15.160:3300/0 conn(0x7f7f800cec40 0x7f7f8015d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f7dd9b640 1 -- 172.21.15.26:0/568672889 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8015afc0 con 0x7f7f8006b3d0 2023-12-19T18:14:32.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f7ed9d640 1 --2- 172.21.15.26:0/568672889 >> v2:172.21.15.160:3300/0 conn(0x7f7f800cec40 0x7f7f8015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:32.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.888+0000 7f7f7e59c640 1 --2- 172.21.15.26:0/568672889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f80154220 0x7f7f8015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:32.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f7dd9b640 1 --2- 172.21.15.26:0/568672889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f8006b3d0 0x7f7f800ce700 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f7f800ae310 tx=0x7f7f7401c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7508cf5656a6f2cc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:32.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f7d59a640 1 -- 172.21.15.26:0/568672889 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f740ab600 con 0x7f7f8006b3d0 2023-12-19T18:14:32.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f7d59a640 1 -- 172.21.15.26:0/568672889 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f740a7050 con 0x7f7f8006b3d0 2023-12-19T18:14:32.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f7d59a640 1 -- 172.21.15.26:0/568672889 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f740ab980 con 0x7f7f8006b3d0 2023-12-19T18:14:32.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f84be9640 1 -- 172.21.15.26:0/568672889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f8006b3d0 msgr2=0x7f7f800ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:32.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f84be9640 1 --2- 172.21.15.26:0/568672889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f8006b3d0 0x7f7f800ce700 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f7f800ae310 tx=0x7f7f7401c210 comp rx=0 tx=0).stop 2023-12-19T18:14:32.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f84be9640 1 -- 172.21.15.26:0/568672889 shutdown_connections 2023-12-19T18:14:32.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f84be9640 1 --2- 172.21.15.26:0/568672889 >> v2:172.21.15.160:3300/0 conn(0x7f7f800cec40 0x7f7f8015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f84be9640 1 --2- 172.21.15.26:0/568672889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f8006b3d0 0x7f7f800ce700 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f84be9640 1 --2- 172.21.15.26:0/568672889 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f80154220 0x7f7f8015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f84be9640 1 -- 172.21.15.26:0/568672889 >> 172.21.15.26:0/568672889 conn(0x7f7f800658e0 msgr2=0x7f7f80065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:32.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f84be9640 1 -- 172.21.15.26:0/568672889 shutdown_connections 2023-12-19T18:14:32.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f84be9640 1 -- 172.21.15.26:0/568672889 wait complete. 2023-12-19T18:14:32.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f84be9640 1 Processor -- start 2023-12-19T18:14:32.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.892+0000 7f7f84be9640 1 -- start start 2023-12-19T18:14:32.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f84be9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f8006b3d0 0x7f7f801432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f84be9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f800cec40 0x7f7f800ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f84be9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f80154220 0x7f7f800acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f84be9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8015cbe0 con 0x7f7f80154220 2023-12-19T18:14:32.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f84be9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f801437f0 con 0x7f7f800cec40 2023-12-19T18:14:32.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f84be9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8015bfd0 con 0x7f7f8006b3d0 2023-12-19T18:14:32.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f7ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f80154220 0x7f7f800acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f7ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f80154220 0x7f7f800acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39090/0 (socket says 172.21.15.26:39090) 2023-12-19T18:14:32.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f7ed9d640 1 -- 172.21.15.26:0/1585699260 learned_addr learned my addr 172.21.15.26:0/1585699260 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:32.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f7e59c640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f8006b3d0 0x7f7f801432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f7dd9b640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f800cec40 0x7f7f800ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f7ed9d640 1 -- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f8006b3d0 msgr2=0x7f7f801432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:32.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f7ed9d640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f8006b3d0 0x7f7f801432b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f7ed9d640 1 -- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f800cec40 msgr2=0x7f7f800ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:32.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f7ed9d640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f800cec40 0x7f7f800ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:32.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f7ed9d640 1 -- 172.21.15.26:0/1585699260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f740094d0 con 0x7f7f80154220 2023-12-19T18:14:32.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f7ed9d640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f80154220 0x7f7f800acd90 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800cef0 tx=0x7f7f7800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:32.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f6effd640 1 -- 172.21.15.26:0/1585699260 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7800d840 con 0x7f7f80154220 2023-12-19T18:14:32.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f6effd640 1 -- 172.21.15.26:0/1585699260 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7800d9a0 con 0x7f7f80154220 2023-12-19T18:14:32.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f84be9640 1 -- 172.21.15.26:0/1585699260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8015c2b0 con 0x7f7f80154220 2023-12-19T18:14:32.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f6effd640 1 -- 172.21.15.26:0/1585699260 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7800dd50 con 0x7f7f80154220 2023-12-19T18:14:32.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f84be9640 1 -- 172.21.15.26:0/1585699260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8015c870 con 0x7f7f80154220 2023-12-19T18:14:32.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f84be9640 1 -- 172.21.15.26:0/1585699260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f80074270 con 0x7f7f80154220 2023-12-19T18:14:32.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.896+0000 7f7f6effd640 1 -- 172.21.15.26:0/1585699260 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7f7801e3b0 con 0x7f7f80154220 2023-12-19T18:14:32.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.900+0000 7f7f6effd640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f60065bb0 0x7f7f60068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:32.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.900+0000 7f7f6effd640 1 -- 172.21.15.26:0/1585699260 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6649+0+0 (secure 0 0 0) 0x7f7f78083f60 con 0x7f7f80154220 2023-12-19T18:14:32.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.900+0000 7f7f7e59c640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f60065bb0 0x7f7f60068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:32.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.900+0000 7f7f7e59c640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f60065bb0 0x7f7f60068070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000ab60 tx=0x7f7f70008040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:32.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:32.904+0000 7f7f6effd640 1 -- 172.21.15.26:0/1585699260 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7f780515a0 con 0x7f7f80154220 2023-12-19T18:14:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f84be9640 1 -- 172.21.15.26:0/1585699260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f800cdf60 con 0x7f7f80154220 2023-12-19T18:14:33.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f6effd640 1 -- 172.21.15.26:0/1585699260 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+3530 (secure 0 0 0) 0x7f7f7801a090 con 0x7f7f80154220 2023-12-19T18:14:33.096 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:33.097 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":82,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7994,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/2924688599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":2924688599},{"type":"v1","addr":"172.21.15.26:6835","nonce":2924688599}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9028,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/330338341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":330338341},{"type":"v1","addr":"172.21.15.160:6836","nonce":330338341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":9031,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/887954085","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":887954085},{"type":"v1","addr":"172.21.15.160:6837","nonce":887954085}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":76},{"gid":10011,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3395774569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3395774569},{"type":"v1","addr":"172.21.15.26:6834","nonce":3395774569}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[]} 2023-12-19T18:14:33.097 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 82 2023-12-19T18:14:33.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f84be9640 1 -- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f60065bb0 msgr2=0x7f7f60068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:33.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f84be9640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f60065bb0 0x7f7f60068070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000ab60 tx=0x7f7f70008040 comp rx=0 tx=0).stop 2023-12-19T18:14:33.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f84be9640 1 -- 172.21.15.26:0/1585699260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f80154220 msgr2=0x7f7f800acd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:33.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f84be9640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f80154220 0x7f7f800acd90 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f7f7800cef0 tx=0x7f7f7800a710 comp rx=0 tx=0).stop 2023-12-19T18:14:33.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f84be9640 1 -- 172.21.15.26:0/1585699260 shutdown_connections 2023-12-19T18:14:33.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f84be9640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f60065bb0 0x7f7f60068070 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:33.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f84be9640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f80154220 0x7f7f800acd90 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:33.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f84be9640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f800cec40 0x7f7f800ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:33.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f84be9640 1 --2- 172.21.15.26:0/1585699260 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f8006b3d0 0x7f7f801432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:33.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f84be9640 1 -- 172.21.15.26:0/1585699260 >> 172.21.15.26:0/1585699260 conn(0x7f7f800658e0 msgr2=0x7f7f800b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:33.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f84be9640 1 -- 172.21.15.26:0/1585699260 shutdown_connections 2023-12-19T18:14:33.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.096+0000 7f7f84be9640 1 -- 172.21.15.26:0/1585699260 wait complete. 2023-12-19T18:14:33.118 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:14:33.118 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:14:33.119 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:14:33.119 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:14:33.119 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:14:33.119 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:14:33.119 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:14:33.119 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:14:33.120 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:14:33.120 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:14:33.120 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:14:33.120 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:14:33.120 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:14:33.120 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:14:33.120 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:14:33.120 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:14:33.130 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:14:33.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:14:33.132 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:14:33.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:14:33.135 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:14:33.135 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:14:33.136 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:14:33.137 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:14:33.181 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:14:33.182 DEBUG:teuthology.parallel:result is None 2023-12-19T18:14:33.182 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:14:33.183 DEBUG:teuthology.parallel:result is None 2023-12-19T18:14:33.184 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:14:33.184 DEBUG:teuthology.parallel:result is None 2023-12-19T18:14:33.185 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:14:33.185 DEBUG:teuthology.parallel:result is None 2023-12-19T18:14:33.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:14:33.269 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T18:14:33.273 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T18:14:33.274 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T18:14:33.278 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T18:14:33.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf6bcc4640 1 Processor -- start 2023-12-19T18:14:33.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf6bcc4640 1 -- start start 2023-12-19T18:14:33.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf6bcc4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf640cb1d0 0x7faf640cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:33.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf6bcc4640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf6406bee0 0x7faf6406c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:33.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf6bcc4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf6406d4b0 con 0x7faf64164c90 2023-12-19T18:14:33.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf6bcc4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf640cc030 con 0x7faf640cb1d0 2023-12-19T18:14:33.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf6bcc4640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf640cbaf0 con 0x7faf6406bee0 2023-12-19T18:14:33.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf69a60640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf640cb1d0 0x7faf640cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:33.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf69a60640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf640cb1d0 0x7faf640cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39122/0 (socket says 172.21.15.26:39122) 2023-12-19T18:14:33.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf69a60640 1 -- 172.21.15.26:0/590238060 learned_addr learned my addr 172.21.15.26:0/590238060 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:33.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf6925f640 1 --2- 172.21.15.26:0/590238060 >> v2:172.21.15.160:3300/0 conn(0x7faf6406bee0 0x7faf6406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:33.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf69a60640 1 -- 172.21.15.26:0/590238060 >> v2:172.21.15.160:3300/0 conn(0x7faf6406bee0 msgr2=0x7faf6406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:33.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf69a60640 1 --2- 172.21.15.26:0/590238060 >> v2:172.21.15.160:3300/0 conn(0x7faf6406bee0 0x7faf6406c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:33.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf69a60640 1 -- 172.21.15.26:0/590238060 >> v1:172.21.15.160:6789/0 conn(0x7faf64164c90 legacy=0x7faf64165100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:33.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.348+0000 7faf69a60640 1 -- 172.21.15.26:0/590238060 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6406cc10 con 0x7faf640cb1d0 2023-12-19T18:14:33.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.352+0000 7faf69a60640 1 --2- 172.21.15.26:0/590238060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf640cb1d0 0x7faf640cb5b0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7faf640667b0 tx=0x7faf6001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526649bf294f6954 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.352+0000 7faf68a5e640 1 -- 172.21.15.26:0/590238060 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf600ab640 con 0x7faf640cb1d0 2023-12-19T18:14:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.352+0000 7faf68a5e640 1 -- 172.21.15.26:0/590238060 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf600a7070 con 0x7faf640cb1d0 2023-12-19T18:14:33.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.352+0000 7faf68a5e640 1 -- 172.21.15.26:0/590238060 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf600ab9c0 con 0x7faf640cb1d0 2023-12-19T18:14:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.352+0000 7faf6bcc4640 1 -- 172.21.15.26:0/590238060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf640cb1d0 msgr2=0x7faf640cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.352+0000 7faf6bcc4640 1 --2- 172.21.15.26:0/590238060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf640cb1d0 0x7faf640cb5b0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7faf640667b0 tx=0x7faf6001c440 comp rx=0 tx=0).stop 2023-12-19T18:14:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.352+0000 7faf6bcc4640 1 -- 172.21.15.26:0/590238060 shutdown_connections 2023-12-19T18:14:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.352+0000 7faf6bcc4640 1 --2- 172.21.15.26:0/590238060 >> v2:172.21.15.160:3300/0 conn(0x7faf6406bee0 0x7faf6406c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.352+0000 7faf6bcc4640 1 --2- 172.21.15.26:0/590238060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf640cb1d0 0x7faf640cb5b0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.352+0000 7faf6bcc4640 1 -- 172.21.15.26:0/590238060 >> 172.21.15.26:0/590238060 conn(0x7faf640658e0 msgr2=0x7faf64065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.352+0000 7faf6bcc4640 1 -- 172.21.15.26:0/590238060 shutdown_connections 2023-12-19T18:14:33.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.352+0000 7faf6bcc4640 1 -- 172.21.15.26:0/590238060 wait complete. 2023-12-19T18:14:33.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6bcc4640 1 Processor -- start 2023-12-19T18:14:33.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6bcc4640 1 -- start start 2023-12-19T18:14:33.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6bcc4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf6406bee0 0x7faf6415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6bcc4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf640cb1d0 0x7faf640cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6bcc4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf64164c90 0x7faf640cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6bcc4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6415bcc0 con 0x7faf640cb1d0 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6bcc4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf64159cb0 con 0x7faf64164c90 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6bcc4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6415a270 con 0x7faf6406bee0 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6925f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf640cb1d0 0x7faf640cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6925f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf640cb1d0 0x7faf640cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39138/0 (socket says 172.21.15.26:39138) 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6925f640 1 -- 172.21.15.26:0/3645618613 learned_addr learned my addr 172.21.15.26:0/3645618613 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6a261640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf64164c90 0x7faf640cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf69a60640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf6406bee0 0x7faf6415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6925f640 1 -- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf6406bee0 msgr2=0x7faf6415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6925f640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf6406bee0 0x7faf6415a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6925f640 1 -- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf64164c90 msgr2=0x7faf640cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:33.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6925f640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf64164c90 0x7faf640cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6925f640 1 -- 172.21.15.26:0/3645618613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf600094d0 con 0x7faf640cb1d0 2023-12-19T18:14:33.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6925f640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf640cb1d0 0x7faf640cd010 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7faf5400e870 tx=0x7faf5400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:33.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf5a7fc640 1 -- 172.21.15.26:0/3645618613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54009510 con 0x7faf640cb1d0 2023-12-19T18:14:33.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf5a7fc640 1 -- 172.21.15.26:0/3645618613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5400eec0 con 0x7faf640cb1d0 2023-12-19T18:14:33.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6bcc4640 1 -- 172.21.15.26:0/3645618613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6415bed0 con 0x7faf640cb1d0 2023-12-19T18:14:33.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf5a7fc640 1 -- 172.21.15.26:0/3645618613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5400cb40 con 0x7faf640cb1d0 2023-12-19T18:14:33.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.356+0000 7faf6bcc4640 1 -- 172.21.15.26:0/3645618613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6415ac40 con 0x7faf640cb1d0 2023-12-19T18:14:33.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.360+0000 7faf6bcc4640 1 -- 172.21.15.26:0/3645618613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf64074270 con 0x7faf640cb1d0 2023-12-19T18:14:33.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.360+0000 7faf5a7fc640 1 -- 172.21.15.26:0/3645618613 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faf5400ccc0 con 0x7faf640cb1d0 2023-12-19T18:14:33.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.360+0000 7faf5a7fc640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf48065b80 0x7faf48068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.360+0000 7faf5a7fc640 1 -- 172.21.15.26:0/3645618613 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6621+0+0 (secure 0 0 0) 0x7faf54083b20 con 0x7faf640cb1d0 2023-12-19T18:14:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.360+0000 7faf69a60640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf48065b80 0x7faf48068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:33.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.360+0000 7faf69a60640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf48065b80 0x7faf48068040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf600af000 tx=0x7faf600b6040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:33.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.364+0000 7faf5a7fc640 1 -- 172.21.15.26:0/3645618613 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faf54051050 con 0x7faf640cb1d0 2023-12-19T18:14:33.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:33.536+0000 7faf6bcc4640 1 -- 172.21.15.26:0/3645618613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7faf640ccdf0 con 0x7faf640cb1d0 2023-12-19T18:14:34.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.332+0000 7faf5a7fc640 1 -- 172.21.15.26:0/3645618613 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v192) v1 ==== 97+0+0 (secure 0 0 0) 0x7faf54021080 con 0x7faf640cb1d0 2023-12-19T18:14:34.333 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2023-12-19T18:14:34.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.332+0000 7faf6bcc4640 1 -- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf48065b80 msgr2=0x7faf48068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:34.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.332+0000 7faf6bcc4640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf48065b80 0x7faf48068040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faf600af000 tx=0x7faf600b6040 comp rx=0 tx=0).stop 2023-12-19T18:14:34.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.332+0000 7faf6bcc4640 1 -- 172.21.15.26:0/3645618613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf640cb1d0 msgr2=0x7faf640cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:34.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.332+0000 7faf6bcc4640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf640cb1d0 0x7faf640cd010 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7faf5400e870 tx=0x7faf5400ec70 comp rx=0 tx=0).stop 2023-12-19T18:14:34.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.332+0000 7faf6bcc4640 1 -- 172.21.15.26:0/3645618613 shutdown_connections 2023-12-19T18:14:34.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.332+0000 7faf6bcc4640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf48065b80 0x7faf48068040 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.332+0000 7faf6bcc4640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf64164c90 0x7faf640cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.332+0000 7faf6bcc4640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf640cb1d0 0x7faf640cd010 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.332+0000 7faf6bcc4640 1 --2- 172.21.15.26:0/3645618613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf6406bee0 0x7faf6415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.332+0000 7faf6bcc4640 1 -- 172.21.15.26:0/3645618613 >> 172.21.15.26:0/3645618613 conn(0x7faf640658e0 msgr2=0x7faf64165100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:34.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.332+0000 7faf6bcc4640 1 -- 172.21.15.26:0/3645618613 shutdown_connections 2023-12-19T18:14:34.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.332+0000 7faf6bcc4640 1 -- 172.21.15.26:0/3645618613 wait complete. 2023-12-19T18:14:34.358 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:14:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.504+0000 7f9a53273640 1 Processor -- start 2023-12-19T18:14:34.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.504+0000 7f9a53273640 1 -- start start 2023-12-19T18:14:34.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a53273640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a4c0cb1d0 0x7f9a4c0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:34.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a53273640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9a4c06bee0 0x7f9a4c06c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:34.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a53273640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c06d4b0 con 0x7f9a4c164c90 2023-12-19T18:14:34.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a53273640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a4c0cc030 con 0x7f9a4c06bee0 2023-12-19T18:14:34.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a53273640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0cbaf0 con 0x7f9a4c0cb1d0 2023-12-19T18:14:34.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a5100f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a4c0cb1d0 0x7f9a4c0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:34.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a5080e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9a4c06bee0 0x7f9a4c06c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:34.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a51810640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f9a4c164c90 0x7f9a4c165120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:58388/0 (socket says 172.21.15.26:58388) 2023-12-19T18:14:34.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a51810640 1 -- 172.21.15.26:0/3542818733 learned_addr learned my addr 172.21.15.26:0/3542818733 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:34.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a5100f640 1 -- 172.21.15.26:0/3542818733 >> v2:172.21.15.160:3300/0 conn(0x7f9a4c06bee0 msgr2=0x7f9a4c06c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:34.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a5100f640 1 --2- 172.21.15.26:0/3542818733 >> v2:172.21.15.160:3300/0 conn(0x7f9a4c06bee0 0x7f9a4c06c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a5100f640 1 -- 172.21.15.26:0/3542818733 >> v1:172.21.15.160:6789/0 conn(0x7f9a4c164c90 legacy=0x7f9a4c165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:34.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a5100f640 1 -- 172.21.15.26:0/3542818733 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c06cc10 con 0x7f9a4c0cb1d0 2023-12-19T18:14:34.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a43fff640 1 -- 172.21.15.26:0/3542818733 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 787215402 0 0) 0x7f9a4c06d4b0 con 0x7f9a4c164c90 2023-12-19T18:14:34.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.508+0000 7f9a5080e640 1 --2- 172.21.15.26:0/3542818733 >> v2:172.21.15.160:3300/0 conn(0x7f9a4c06bee0 0x7f9a4c06c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.512+0000 7f9a5100f640 1 --2- 172.21.15.26:0/3542818733 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a4c0cb1d0 0x7f9a4c0cb5b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9a480097c0 tx=0x7f9a4801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1e7d522af179adc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.512+0000 7f9a43fff640 1 -- 172.21.15.26:0/3542818733 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a480ac620 con 0x7f9a4c0cb1d0 2023-12-19T18:14:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.512+0000 7f9a43fff640 1 -- 172.21.15.26:0/3542818733 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a480ac780 con 0x7f9a4c0cb1d0 2023-12-19T18:14:34.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.512+0000 7f9a43fff640 1 -- 172.21.15.26:0/3542818733 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a480acb10 con 0x7f9a4c0cb1d0 2023-12-19T18:14:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.512+0000 7f9a53273640 1 -- 172.21.15.26:0/3542818733 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a4c0cb1d0 msgr2=0x7f9a4c0cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.512+0000 7f9a53273640 1 --2- 172.21.15.26:0/3542818733 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a4c0cb1d0 0x7f9a4c0cb5b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9a480097c0 tx=0x7f9a4801c3c0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.512+0000 7f9a53273640 1 -- 172.21.15.26:0/3542818733 shutdown_connections 2023-12-19T18:14:34.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.512+0000 7f9a53273640 1 --2- 172.21.15.26:0/3542818733 >> v2:172.21.15.160:3300/0 conn(0x7f9a4c06bee0 0x7f9a4c06c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.512+0000 7f9a53273640 1 --2- 172.21.15.26:0/3542818733 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a4c0cb1d0 0x7f9a4c0cb5b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.512+0000 7f9a53273640 1 -- 172.21.15.26:0/3542818733 >> 172.21.15.26:0/3542818733 conn(0x7f9a4c0658e0 msgr2=0x7f9a4c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:34.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.512+0000 7f9a53273640 1 -- 172.21.15.26:0/3542818733 shutdown_connections 2023-12-19T18:14:34.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.512+0000 7f9a53273640 1 -- 172.21.15.26:0/3542818733 wait complete. 2023-12-19T18:14:34.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a53273640 1 Processor -- start 2023-12-19T18:14:34.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a53273640 1 -- start start 2023-12-19T18:14:34.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a53273640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a4c06bee0 0x7f9a4c148140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:34.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a53273640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a4c0cb1d0 0x7f9a4c14bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:34.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a53273640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c164c90 0x7f9a4c150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:34.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a53273640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c148680 con 0x7f9a4c164c90 2023-12-19T18:14:34.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a53273640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0ce020 con 0x7f9a4c06bee0 2023-12-19T18:14:34.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a53273640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c1489e0 con 0x7f9a4c0cb1d0 2023-12-19T18:14:34.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a51810640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c164c90 0x7f9a4c150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:34.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a51810640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c164c90 0x7f9a4c150f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39148/0 (socket says 172.21.15.26:39148) 2023-12-19T18:14:34.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a51810640 1 -- 172.21.15.26:0/4113302750 learned_addr learned my addr 172.21.15.26:0/4113302750 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:34.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a5100f640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a4c06bee0 0x7f9a4c148140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:34.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a5080e640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a4c0cb1d0 0x7f9a4c14bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:34.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a51810640 1 -- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a4c0cb1d0 msgr2=0x7f9a4c14bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:34.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a51810640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a4c0cb1d0 0x7f9a4c14bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a51810640 1 -- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a4c06bee0 msgr2=0x7f9a4c148140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:34.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a51810640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a4c06bee0 0x7f9a4c148140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a51810640 1 -- 172.21.15.26:0/4113302750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c009420 con 0x7f9a4c164c90 2023-12-19T18:14:34.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a5100f640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a4c06bee0 0x7f9a4c148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:34.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a51810640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c164c90 0x7f9a4c150f80 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f9a44001570 tx=0x7f9a4400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:34.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a41ffb640 1 -- 172.21.15.26:0/4113302750 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400a640 con 0x7f9a4c164c90 2023-12-19T18:14:34.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a41ffb640 1 -- 172.21.15.26:0/4113302750 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4400a7c0 con 0x7f9a4c164c90 2023-12-19T18:14:34.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a53273640 1 -- 172.21.15.26:0/4113302750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a48009420 con 0x7f9a4c164c90 2023-12-19T18:14:34.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a41ffb640 1 -- 172.21.15.26:0/4113302750 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400ab70 con 0x7f9a4c164c90 2023-12-19T18:14:34.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a53273640 1 -- 172.21.15.26:0/4113302750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c148fa0 con 0x7f9a4c164c90 2023-12-19T18:14:34.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.516+0000 7f9a53273640 1 -- 172.21.15.26:0/4113302750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4c074270 con 0x7f9a4c164c90 2023-12-19T18:14:34.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.520+0000 7f9a41ffb640 1 -- 172.21.15.26:0/4113302750 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9a4400acd0 con 0x7f9a4c164c90 2023-12-19T18:14:34.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.520+0000 7f9a41ffb640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9a30065b80 0x7f9a30068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:34.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.520+0000 7f9a41ffb640 1 -- 172.21.15.26:0/4113302750 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5589+0+0 (secure 0 0 0) 0x7f9a44082bb0 con 0x7f9a4c164c90 2023-12-19T18:14:34.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.520+0000 7f9a5100f640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9a30065b80 0x7f9a30068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:34.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.520+0000 7f9a5100f640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9a30065b80 0x7f9a30068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9a480ae000 tx=0x7f9a480a7000 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:34.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.524+0000 7f9a41ffb640 1 -- 172.21.15.26:0/4113302750 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9a44050430 con 0x7f9a4c164c90 2023-12-19T18:14:34.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.740+0000 7f9a53273640 1 -- 172.21.15.26:0/4113302750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9a4c0cf710 con 0x7f9a4c164c90 2023-12-19T18:14:34.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.740+0000 7f9a41ffb640 1 -- 172.21.15.26:0/4113302750 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v50) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9a44018090 con 0x7f9a4c164c90 2023-12-19T18:14:34.742 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:34.742 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:14:34.742 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2023-12-19T18:14:34.742 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.742 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2023-12-19T18:14:34.742 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg==", 2023-12-19T18:14:34.742 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.742 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg==", 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.743 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg==", 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw==", 2023-12-19T18:14:34.744 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig==", 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.745 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.746 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2023-12-19T18:14:34.746 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA==", 2023-12-19T18:14:34.746 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.746 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:14:34.746 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:14:34.746 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.746 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.746 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.746 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.746 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2023-12-19T18:14:34.747 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ==", 2023-12-19T18:14:34.747 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.747 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:14:34.747 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:14:34.747 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.747 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.747 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.747 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.747 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2023-12-19T18:14:34.747 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ==", 2023-12-19T18:14:34.747 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.747 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:14:34.747 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:14:34.748 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.748 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.748 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A==", 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ==", 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:14:34.749 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg==", 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.750 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg==", 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ==", 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T18:14:34.751 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ==", 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2023-12-19T18:14:34.752 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ==", 2023-12-19T18:14:34.753 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.753 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:14:34.753 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T18:14:34.753 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T18:14:34.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T18:14:34.753 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.753 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.753 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2023-12-19T18:14:34.753 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ==", 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlT5ynDxAA7Sjbdwrc1RyAXJboysB54Q==", 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:14:34.754 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:14:34.755 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.755 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.755 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2023-12-19T18:14:34.755 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA==", 2023-12-19T18:14:34.755 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.755 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T18:14:34.755 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T18:14:34.755 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.755 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.755 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:14:34.755 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:34.756 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2023-12-19T18:14:34.756 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg==", 2023-12-19T18:14:34.756 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:14:34.756 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T18:14:34.756 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T18:14:34.756 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:14:34.757 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.757 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:34.757 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-12-19T18:14:34.757 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:14:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.744+0000 7f9a53273640 1 -- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9a30065b80 msgr2=0x7f9a30068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.744+0000 7f9a53273640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9a30065b80 0x7f9a30068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9a480ae000 tx=0x7f9a480a7000 comp rx=0 tx=0).stop 2023-12-19T18:14:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.744+0000 7f9a53273640 1 -- 172.21.15.26:0/4113302750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c164c90 msgr2=0x7f9a4c150f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.744+0000 7f9a53273640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c164c90 0x7f9a4c150f80 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f9a44001570 tx=0x7f9a4400cca0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.744+0000 7f9a53273640 1 -- 172.21.15.26:0/4113302750 shutdown_connections 2023-12-19T18:14:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.744+0000 7f9a53273640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9a30065b80 0x7f9a30068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.744+0000 7f9a53273640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a4c164c90 0x7f9a4c150f80 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.744+0000 7f9a53273640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a4c0cb1d0 0x7f9a4c14bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.744+0000 7f9a53273640 1 --2- 172.21.15.26:0/4113302750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a4c06bee0 0x7f9a4c148140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.744+0000 7f9a53273640 1 -- 172.21.15.26:0/4113302750 >> 172.21.15.26:0/4113302750 conn(0x7f9a4c0658e0 msgr2=0x7f9a4c160b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:34.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.744+0000 7f9a53273640 1 -- 172.21.15.26:0/4113302750 shutdown_connections 2023-12-19T18:14:34.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.744+0000 7f9a53273640 1 -- 172.21.15.26:0/4113302750 wait complete. 2023-12-19T18:14:34.764 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:34.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23ff8f640 1 Processor -- start 2023-12-19T18:14:34.908 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23ff8f640 1 -- start start 2023-12-19T18:14:34.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23ff8f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc238155030 0x7fc23815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:34.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23ff8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc23815f370 0x7fc23806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:34.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23ff8f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc2380ce740 0x7fc2380cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:34.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23ff8f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2380cf130 con 0x7fc238155030 2023-12-19T18:14:34.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23ff8f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc23806c970 con 0x7fc2380ce740 2023-12-19T18:14:34.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23ff8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23815b1f0 con 0x7fc23815f370 2023-12-19T18:14:34.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23dd2b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc238155030 0x7fc23815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:34.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23dd2b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc238155030 0x7fc23815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39160/0 (socket says 172.21.15.26:39160) 2023-12-19T18:14:34.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23dd2b640 1 -- 172.21.15.26:0/1648529852 learned_addr learned my addr 172.21.15.26:0/1648529852 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:34.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23e52c640 1 --2- 172.21.15.26:0/1648529852 >> v2:172.21.15.160:3300/0 conn(0x7fc2380ce740 0x7fc2380cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:34.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23d52a640 1 --2- 172.21.15.26:0/1648529852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc23815f370 0x7fc23806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:34.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23dd2b640 1 -- 172.21.15.26:0/1648529852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc23815f370 msgr2=0x7fc23806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:34.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23dd2b640 1 --2- 172.21.15.26:0/1648529852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc23815f370 0x7fc23806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23dd2b640 1 -- 172.21.15.26:0/1648529852 >> v2:172.21.15.160:3300/0 conn(0x7fc2380ce740 msgr2=0x7fc2380cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:34.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23dd2b640 1 --2- 172.21.15.26:0/1648529852 >> v2:172.21.15.160:3300/0 conn(0x7fc2380ce740 0x7fc2380cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.908+0000 7fc23dd2b640 1 -- 172.21.15.26:0/1648529852 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23815b4d0 con 0x7fc238155030 2023-12-19T18:14:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.912+0000 7fc23dd2b640 1 --2- 172.21.15.26:0/1648529852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc238155030 0x7fc23815bcd0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fc23806f300 tx=0x7fc23401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1216d32a68a81dfe server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.912+0000 7fc23cd29640 1 -- 172.21.15.26:0/1648529852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2340aa610 con 0x7fc238155030 2023-12-19T18:14:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.912+0000 7fc23cd29640 1 -- 172.21.15.26:0/1648529852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2340a6040 con 0x7fc238155030 2023-12-19T18:14:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.912+0000 7fc23cd29640 1 -- 172.21.15.26:0/1648529852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2340aa990 con 0x7fc238155030 2023-12-19T18:14:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.912+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1648529852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc238155030 msgr2=0x7fc23815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:34.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.912+0000 7fc23ff8f640 1 --2- 172.21.15.26:0/1648529852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc238155030 0x7fc23815bcd0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fc23806f300 tx=0x7fc23401c4d0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.912+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1648529852 shutdown_connections 2023-12-19T18:14:34.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.912+0000 7fc23ff8f640 1 --2- 172.21.15.26:0/1648529852 >> v2:172.21.15.160:3300/0 conn(0x7fc2380ce740 0x7fc2380cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.912+0000 7fc23ff8f640 1 --2- 172.21.15.26:0/1648529852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc23815f370 0x7fc23806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.912+0000 7fc23ff8f640 1 --2- 172.21.15.26:0/1648529852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc238155030 0x7fc23815bcd0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.912+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1648529852 >> 172.21.15.26:0/1648529852 conn(0x7fc2380658e0 msgr2=0x7fc238065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:34.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.912+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1648529852 shutdown_connections 2023-12-19T18:14:34.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.912+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1648529852 wait complete. 2023-12-19T18:14:34.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23ff8f640 1 Processor -- start 2023-12-19T18:14:34.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23ff8f640 1 -- start start 2023-12-19T18:14:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23ff8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2380ce740 0x7fc238145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23ff8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc238155030 0x7fc23815cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23ff8f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23815f370 0x7fc23815d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23ff8f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc238146810 con 0x7fc238155030 2023-12-19T18:14:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23ff8f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc238153890 con 0x7fc23815f370 2023-12-19T18:14:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23ff8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc238153b70 con 0x7fc2380ce740 2023-12-19T18:14:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23d52a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc238155030 0x7fc23815cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23d52a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc238155030 0x7fc23815cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39174/0 (socket says 172.21.15.26:39174) 2023-12-19T18:14:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23d52a640 1 -- 172.21.15.26:0/1410750515 learned_addr learned my addr 172.21.15.26:0/1410750515 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:34.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23dd2b640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2380ce740 0x7fc238145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:34.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23e52c640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23815f370 0x7fc23815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:34.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23d52a640 1 -- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2380ce740 msgr2=0x7fc238145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:34.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23d52a640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2380ce740 0x7fc238145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23d52a640 1 -- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23815f370 msgr2=0x7fc23815d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:34.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23d52a640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23815f370 0x7fc23815d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:34.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23d52a640 1 -- 172.21.15.26:0/1410750515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2340094d0 con 0x7fc238155030 2023-12-19T18:14:34.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23e52c640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23815f370 0x7fc23815d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:34.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23d52a640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc238155030 0x7fc23815cc40 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fc22800d7d0 tx=0x7fc22800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:34.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc22e7fc640 1 -- 172.21.15.26:0/1410750515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc228018070 con 0x7fc238155030 2023-12-19T18:14:34.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc22e7fc640 1 -- 172.21.15.26:0/1410750515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc228004020 con 0x7fc238155030 2023-12-19T18:14:34.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc22e7fc640 1 -- 172.21.15.26:0/1410750515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2280041d0 con 0x7fc238155030 2023-12-19T18:14:34.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1410750515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23806c740 con 0x7fc238155030 2023-12-19T18:14:34.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1410750515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2380ae100 con 0x7fc238155030 2023-12-19T18:14:34.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.916+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1410750515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2380ae2c0 con 0x7fc238155030 2023-12-19T18:14:34.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.920+0000 7fc22e7fc640 1 -- 172.21.15.26:0/1410750515 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc228004330 con 0x7fc238155030 2023-12-19T18:14:34.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.920+0000 7fc22e7fc640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc21c065b80 0x7fc21c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.920+0000 7fc22e7fc640 1 -- 172.21.15.26:0/1410750515 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fc228083170 con 0x7fc238155030 2023-12-19T18:14:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.920+0000 7fc23dd2b640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc21c065b80 0x7fc21c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:34.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.920+0000 7fc23dd2b640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc21c065b80 0x7fc21c068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc2340ae000 tx=0x7fc2340a7000 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:34.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:34.924+0000 7fc22e7fc640 1 -- 172.21.15.26:0/1410750515 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc2280509f0 con 0x7fc238155030 2023-12-19T18:14:35.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.084+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1410750515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc238153d90 con 0x7fc238155030 2023-12-19T18:14:35.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.084+0000 7fc22e7fc640 1 -- 172.21.15.26:0/1410750515 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+13504 (secure 0 0 0) 0x7fc228054010 con 0x7fc238155030 2023-12-19T18:14:35.086 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:35.086 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":192,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:14:34.321882+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":179,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.088+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc21c065b80 msgr2=0x7fc21c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.088+0000 7fc23ff8f640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc21c065b80 0x7fc21c068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc2340ae000 tx=0x7fc2340a7000 comp rx=0 tx=0).stop 2023-12-19T18:14:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.088+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1410750515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc238155030 msgr2=0x7fc23815cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.088+0000 7fc23ff8f640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc238155030 0x7fc23815cc40 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fc22800d7d0 tx=0x7fc22800dc90 comp rx=0 tx=0).stop 2023-12-19T18:14:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.088+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1410750515 shutdown_connections 2023-12-19T18:14:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.088+0000 7fc23ff8f640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc21c065b80 0x7fc21c068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.088+0000 7fc23ff8f640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc23815f370 0x7fc23815d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.088+0000 7fc23ff8f640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc238155030 0x7fc23815cc40 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.088+0000 7fc23ff8f640 1 --2- 172.21.15.26:0/1410750515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc2380ce740 0x7fc238145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.088+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1410750515 >> 172.21.15.26:0/1410750515 conn(0x7fc2380658e0 msgr2=0x7fc2381504b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.088+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1410750515 shutdown_connections 2023-12-19T18:14:35.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.088+0000 7fc23ff8f640 1 -- 172.21.15.26:0/1410750515 wait complete. 2023-12-19T18:14:35.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:14:35.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:14:35.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb2a925640 1 Processor -- start 2023-12-19T18:14:35.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb2a925640 1 -- start start 2023-12-19T18:14:35.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb2a925640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb24159820 0x7feb2415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb2a925640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb24159c00 0x7feb24070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb2a925640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2406fb60 con 0x7feb2406f6c0 2023-12-19T18:14:35.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb2a925640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2415e470 con 0x7feb24159c00 2023-12-19T18:14:35.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb2a925640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2406fe40 con 0x7feb24159820 2023-12-19T18:14:35.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb237fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feb2406f6c0 0x7feb240c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39286/0 (socket says 172.21.15.26:39286) 2023-12-19T18:14:35.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb237fe640 1 -- 172.21.15.26:0/592438497 learned_addr learned my addr 172.21.15.26:0/592438497 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:35.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb28ec2640 1 --2- 172.21.15.26:0/592438497 >> v2:172.21.15.26:3300/0 conn(0x7feb24159c00 0x7feb24070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb23fff640 1 --2- 172.21.15.26:0/592438497 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb24159820 0x7feb2415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb28ec2640 1 -- 172.21.15.26:0/592438497 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb24159820 msgr2=0x7feb2415df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:35.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb28ec2640 1 --2- 172.21.15.26:0/592438497 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb24159820 0x7feb2415df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb28ec2640 1 -- 172.21.15.26:0/592438497 >> v1:172.21.15.26:6789/0 conn(0x7feb2406f6c0 legacy=0x7feb240c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb28ec2640 1 -- 172.21.15.26:0/592438497 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb24070120 con 0x7feb24159c00 2023-12-19T18:14:35.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.256+0000 7feb22ffd640 1 -- 172.21.15.26:0/592438497 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426410650 0 0) 0x7feb2406fb60 con 0x7feb2406f6c0 2023-12-19T18:14:35.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.260+0000 7feb28ec2640 1 --2- 172.21.15.26:0/592438497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24159c00 0x7feb24070c00 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009d90 tx=0x7feb1c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f256d5e67d0262 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:35.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.260+0000 7feb22ffd640 1 -- 172.21.15.26:0/592438497 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c0a58f0 con 0x7feb24159c00 2023-12-19T18:14:35.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.260+0000 7feb22ffd640 1 -- 172.21.15.26:0/592438497 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c0a5a50 con 0x7feb24159c00 2023-12-19T18:14:35.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.260+0000 7feb22ffd640 1 -- 172.21.15.26:0/592438497 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c0a5de0 con 0x7feb24159c00 2023-12-19T18:14:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.260+0000 7feb2a925640 1 -- 172.21.15.26:0/592438497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24159c00 msgr2=0x7feb24070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.260+0000 7feb2a925640 1 --2- 172.21.15.26:0/592438497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24159c00 0x7feb24070c00 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009d90 tx=0x7feb1c01c920 comp rx=0 tx=0).stop 2023-12-19T18:14:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.260+0000 7feb2a925640 1 -- 172.21.15.26:0/592438497 shutdown_connections 2023-12-19T18:14:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.260+0000 7feb2a925640 1 --2- 172.21.15.26:0/592438497 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24159c00 0x7feb24070c00 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.260+0000 7feb2a925640 1 --2- 172.21.15.26:0/592438497 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb24159820 0x7feb2415df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.260+0000 7feb2a925640 1 -- 172.21.15.26:0/592438497 >> 172.21.15.26:0/592438497 conn(0x7feb240ce730 msgr2=0x7feb240ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.260+0000 7feb2a925640 1 -- 172.21.15.26:0/592438497 shutdown_connections 2023-12-19T18:14:35.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.260+0000 7feb2a925640 1 -- 172.21.15.26:0/592438497 wait complete. 2023-12-19T18:14:35.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb2a925640 1 Processor -- start 2023-12-19T18:14:35.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb2a925640 1 -- start start 2023-12-19T18:14:35.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb2a925640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb2406f6c0 0x7feb24148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb2a925640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb24159820 0x7feb24155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb2a925640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24159c00 0x7feb24153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb2a925640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb241492e0 con 0x7feb24159c00 2023-12-19T18:14:35.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb2a925640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb24154370 con 0x7feb24159820 2023-12-19T18:14:35.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb2a925640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb24154650 con 0x7feb2406f6c0 2023-12-19T18:14:35.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb28ec2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24159c00 0x7feb24153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb28ec2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24159c00 0x7feb24153c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39192/0 (socket says 172.21.15.26:39192) 2023-12-19T18:14:35.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb28ec2640 1 -- 172.21.15.26:0/1253179201 learned_addr learned my addr 172.21.15.26:0/1253179201 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:35.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb23fff640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb2406f6c0 0x7feb24148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb237fe640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb24159820 0x7feb24155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb28ec2640 1 -- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb2406f6c0 msgr2=0x7feb24148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb28ec2640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb2406f6c0 0x7feb24148a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb28ec2640 1 -- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb24159820 msgr2=0x7feb24155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb28ec2640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb24159820 0x7feb24155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb28ec2640 1 -- 172.21.15.26:0/1253179201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1c009a70 con 0x7feb24159c00 2023-12-19T18:14:35.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb237fe640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb24159820 0x7feb24155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:35.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb28ec2640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24159c00 0x7feb24153c70 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7feb1c01afd0 tx=0x7feb1c0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb20ff9640 1 -- 172.21.15.26:0/1253179201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c0ad850 con 0x7feb24159c00 2023-12-19T18:14:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb20ff9640 1 -- 172.21.15.26:0/1253179201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c0a58e0 con 0x7feb24159c00 2023-12-19T18:14:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb20ff9640 1 -- 172.21.15.26:0/1253179201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c0adbd0 con 0x7feb24159c00 2023-12-19T18:14:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb2a925640 1 -- 172.21.15.26:0/1253179201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2406b880 con 0x7feb24159c00 2023-12-19T18:14:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.264+0000 7feb2a925640 1 -- 172.21.15.26:0/1253179201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb24154c10 con 0x7feb24159c00 2023-12-19T18:14:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.268+0000 7feb2a925640 1 -- 172.21.15.26:0/1253179201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2406b680 con 0x7feb24159c00 2023-12-19T18:14:35.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.268+0000 7feb20ff9640 1 -- 172.21.15.26:0/1253179201 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7feb1c0b4070 con 0x7feb24159c00 2023-12-19T18:14:35.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.268+0000 7feb20ff9640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7feb10065b80 0x7feb10068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.268+0000 7feb20ff9640 1 -- 172.21.15.26:0/1253179201 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5589+0+0 (secure 0 0 0) 0x7feb1c11dea0 con 0x7feb24159c00 2023-12-19T18:14:35.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.268+0000 7feb23fff640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7feb10065b80 0x7feb10068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.268+0000 7feb23fff640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7feb10065b80 0x7feb10068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feb24149620 tx=0x7feb14008040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:35.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.272+0000 7feb20ff9640 1 -- 172.21.15.26:0/1253179201 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feb1c0eb720 con 0x7feb24159c00 2023-12-19T18:14:35.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb2a925640 1 -- 172.21.15.26:0/1253179201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb24074270 con 0x7feb24159c00 2023-12-19T18:14:35.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb20ff9640 1 -- 172.21.15.26:0/1253179201 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v192) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb1c0eed40 con 0x7feb24159c00 2023-12-19T18:14:35.429 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:14:35.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb2a925640 1 -- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7feb10065b80 msgr2=0x7feb10068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb2a925640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7feb10065b80 0x7feb10068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feb24149620 tx=0x7feb14008040 comp rx=0 tx=0).stop 2023-12-19T18:14:35.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb2a925640 1 -- 172.21.15.26:0/1253179201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24159c00 msgr2=0x7feb24153c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb2a925640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24159c00 0x7feb24153c70 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7feb1c01afd0 tx=0x7feb1c0196c0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb2a925640 1 -- 172.21.15.26:0/1253179201 shutdown_connections 2023-12-19T18:14:35.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb2a925640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7feb10065b80 0x7feb10068040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb2a925640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb24159c00 0x7feb24153c70 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb2a925640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feb24159820 0x7feb24155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb2a925640 1 --2- 172.21.15.26:0/1253179201 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feb2406f6c0 0x7feb24148a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb2a925640 1 -- 172.21.15.26:0/1253179201 >> 172.21.15.26:0/1253179201 conn(0x7feb240ce730 msgr2=0x7feb240acf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:35.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb2a925640 1 -- 172.21.15.26:0/1253179201 shutdown_connections 2023-12-19T18:14:35.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.428+0000 7feb2a925640 1 -- 172.21.15.26:0/1253179201 wait complete. 2023-12-19T18:14:35.450 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:14:35.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.592+0000 7faf9ca8c640 1 Processor -- start 2023-12-19T18:14:35.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.592+0000 7faf9ca8c640 1 -- start start 2023-12-19T18:14:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.592+0000 7faf9ca8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf98154220 0x7faf9815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.592+0000 7faf9ca8c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf9806b3d0 0x7faf980ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.592+0000 7faf9ca8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9806bc30 con 0x7faf980cec40 2023-12-19T18:14:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.592+0000 7faf9ca8c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf980cf0d0 con 0x7faf9806b3d0 2023-12-19T18:14:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.592+0000 7faf9ca8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf980cc6e0 con 0x7faf98154220 2023-12-19T18:14:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.596+0000 7faf96d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faf980cec40 0x7faf9815d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39302/0 (socket says 172.21.15.26:39302) 2023-12-19T18:14:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.596+0000 7faf96d9d640 1 -- 172.21.15.26:0/2204835540 learned_addr learned my addr 172.21.15.26:0/2204835540 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.596+0000 7faf95d9b640 1 --2- 172.21.15.26:0/2204835540 >> v2:172.21.15.160:3300/0 conn(0x7faf9806b3d0 0x7faf980ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.596+0000 7faf9659c640 1 --2- 172.21.15.26:0/2204835540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf98154220 0x7faf9815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.596+0000 7faf9559a640 1 -- 172.21.15.26:0/2204835540 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 102293672 0 0) 0x7faf9806bc30 con 0x7faf980cec40 2023-12-19T18:14:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.596+0000 7faf9559a640 1 -- 172.21.15.26:0/2204835540 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf80003530 con 0x7faf980cec40 2023-12-19T18:14:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.596+0000 7faf9559a640 1 -- 172.21.15.26:0/2204835540 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2829547260 0 0) 0x7faf80003530 con 0x7faf980cec40 2023-12-19T18:14:35.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.596+0000 7faf95d9b640 1 -- 172.21.15.26:0/2204835540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf98154220 msgr2=0x7faf9815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.596+0000 7faf95d9b640 1 --2- 172.21.15.26:0/2204835540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf98154220 0x7faf9815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.596+0000 7faf95d9b640 1 -- 172.21.15.26:0/2204835540 >> v1:172.21.15.26:6789/0 conn(0x7faf980cec40 legacy=0x7faf9815d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.596+0000 7faf95d9b640 1 -- 172.21.15.26:0/2204835540 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9815afc0 con 0x7faf9806b3d0 2023-12-19T18:14:35.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.596+0000 7faf9659c640 1 --2- 172.21.15.26:0/2204835540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf98154220 0x7faf9815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:35.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf95d9b640 1 --2- 172.21.15.26:0/2204835540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9806b3d0 0x7faf980ce700 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0097c0 tx=0x7faf8c01e580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=560d7e59c792002 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9559a640 1 -- 172.21.15.26:0/2204835540 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf90001a00 con 0x7faf9806b3d0 2023-12-19T18:14:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9559a640 1 -- 172.21.15.26:0/2204835540 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c0ae440 con 0x7faf9806b3d0 2023-12-19T18:14:35.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9559a640 1 -- 172.21.15.26:0/2204835540 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c0ae7d0 con 0x7faf9806b3d0 2023-12-19T18:14:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 -- 172.21.15.26:0/2204835540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9806b3d0 msgr2=0x7faf980ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 --2- 172.21.15.26:0/2204835540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9806b3d0 0x7faf980ce700 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0097c0 tx=0x7faf8c01e580 comp rx=0 tx=0).stop 2023-12-19T18:14:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 -- 172.21.15.26:0/2204835540 shutdown_connections 2023-12-19T18:14:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 --2- 172.21.15.26:0/2204835540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf9806b3d0 0x7faf980ce700 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 --2- 172.21.15.26:0/2204835540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf98154220 0x7faf9815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 -- 172.21.15.26:0/2204835540 >> 172.21.15.26:0/2204835540 conn(0x7faf980658e0 msgr2=0x7faf98065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:35.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 -- 172.21.15.26:0/2204835540 shutdown_connections 2023-12-19T18:14:35.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 -- 172.21.15.26:0/2204835540 wait complete. 2023-12-19T18:14:35.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 Processor -- start 2023-12-19T18:14:35.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 -- start start 2023-12-19T18:14:35.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9806b3d0 0x7faf9814d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf980cec40 0x7faf980b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf98154220 0x7faf980b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf98153ac0 con 0x7faf9806b3d0 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf981537e0 con 0x7faf980cec40 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.600+0000 7faf9ca8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9806f890 con 0x7faf98154220 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf9659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9806b3d0 0x7faf9814d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf9659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9806b3d0 0x7faf9814d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39200/0 (socket says 172.21.15.26:39200) 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf9659c640 1 -- 172.21.15.26:0/166536131 learned_addr learned my addr 172.21.15.26:0/166536131 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf95d9b640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf980cec40 0x7faf980b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf96d9d640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf98154220 0x7faf980b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf9659c640 1 -- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf98154220 msgr2=0x7faf980b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf9659c640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf98154220 0x7faf980b19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf9659c640 1 -- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf980cec40 msgr2=0x7faf980b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf9659c640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf980cec40 0x7faf980b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf9659c640 1 -- 172.21.15.26:0/166536131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf88009420 con 0x7faf9806b3d0 2023-12-19T18:14:35.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf9659c640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9806b3d0 0x7faf9814d090 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7faf8800ebe0 tx=0x7faf8800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf86ffd640 1 -- 172.21.15.26:0/166536131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8800eee0 con 0x7faf9806b3d0 2023-12-19T18:14:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf86ffd640 1 -- 172.21.15.26:0/166536131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8800cc30 con 0x7faf9806b3d0 2023-12-19T18:14:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf86ffd640 1 -- 172.21.15.26:0/166536131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf88013470 con 0x7faf9806b3d0 2023-12-19T18:14:35.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf9ca8c640 1 -- 172.21.15.26:0/166536131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf8c009420 con 0x7faf9806b3d0 2023-12-19T18:14:35.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf9ca8c640 1 -- 172.21.15.26:0/166536131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9806fe50 con 0x7faf9806b3d0 2023-12-19T18:14:35.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf9ca8c640 1 -- 172.21.15.26:0/166536131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf98074270 con 0x7faf9806b3d0 2023-12-19T18:14:35.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.604+0000 7faf86ffd640 1 -- 172.21.15.26:0/166536131 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faf880135d0 con 0x7faf9806b3d0 2023-12-19T18:14:35.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.608+0000 7faf86ffd640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf78065b80 0x7faf78068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.608+0000 7faf86ffd640 1 -- 172.21.15.26:0/166536131 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5589+0+0 (secure 0 0 0) 0x7faf880835a0 con 0x7faf9806b3d0 2023-12-19T18:14:35.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.608+0000 7faf95d9b640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf78065b80 0x7faf78068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.608+0000 7faf95d9b640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf78065b80 0x7faf78068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0b0000 tx=0x7faf8c008040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.612+0000 7faf86ffd640 1 -- 172.21.15.26:0/166536131 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faf88050eb0 con 0x7faf9806b3d0 2023-12-19T18:14:35.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.768+0000 7faf9ca8c640 1 -- 172.21.15.26:0/166536131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7faf9815a300 con 0x7faf9806b3d0 2023-12-19T18:14:35.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.768+0000 7faf86ffd640 1 -- 172.21.15.26:0/166536131 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v192) v1 ==== 84+0+65 (secure 0 0 0) 0x7faf880544d0 con 0x7faf9806b3d0 2023-12-19T18:14:35.769 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:35.769 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T18:14:35.770 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:35.770 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T18:14:35.770 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T18:14:35.770 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:35.770 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T18:14:35.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.772+0000 7faf9ca8c640 1 -- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf78065b80 msgr2=0x7faf78068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.772+0000 7faf9ca8c640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf78065b80 0x7faf78068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0b0000 tx=0x7faf8c008040 comp rx=0 tx=0).stop 2023-12-19T18:14:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.772+0000 7faf9ca8c640 1 -- 172.21.15.26:0/166536131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9806b3d0 msgr2=0x7faf9814d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.772+0000 7faf9ca8c640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9806b3d0 0x7faf9814d090 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7faf8800ebe0 tx=0x7faf8800c520 comp rx=0 tx=0).stop 2023-12-19T18:14:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.772+0000 7faf9ca8c640 1 -- 172.21.15.26:0/166536131 shutdown_connections 2023-12-19T18:14:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.772+0000 7faf9ca8c640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf78065b80 0x7faf78068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.772+0000 7faf9ca8c640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf98154220 0x7faf980b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.772+0000 7faf9ca8c640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf980cec40 0x7faf980b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.772+0000 7faf9ca8c640 1 --2- 172.21.15.26:0/166536131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9806b3d0 0x7faf9814d090 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.772+0000 7faf9ca8c640 1 -- 172.21.15.26:0/166536131 >> 172.21.15.26:0/166536131 conn(0x7faf980658e0 msgr2=0x7faf980cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.772+0000 7faf9ca8c640 1 -- 172.21.15.26:0/166536131 shutdown_connections 2023-12-19T18:14:35.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.772+0000 7faf9ca8c640 1 -- 172.21.15.26:0/166536131 wait complete. 2023-12-19T18:14:35.791 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:14:35.791 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:14:35.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.936+0000 7fec9da04640 1 Processor -- start 2023-12-19T18:14:35.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.936+0000 7fec9da04640 1 -- start start 2023-12-19T18:14:35.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec9da04640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec9806bb60 0x7fec9815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec9da04640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9815bdb0 con 0x7fec980cb010 2023-12-19T18:14:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec9da04640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec980ca250 con 0x7fec9815b250 2023-12-19T18:14:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec9da04640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec980cadd0 con 0x7fec9806bb60 2023-12-19T18:14:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec96ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fec980cb010 0x7fec980cb3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39870/0 (socket says 172.21.15.26:39870) 2023-12-19T18:14:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec96ffd640 1 -- 172.21.15.26:0/1834041304 learned_addr learned my addr 172.21.15.26:0/1834041304 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec967fc640 1 --2- 172.21.15.26:0/1834041304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec9806bb60 0x7fec9815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec95ffb640 1 -- 172.21.15.26:0/1834041304 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415197062 0 0) 0x7fec9815bdb0 con 0x7fec980cb010 2023-12-19T18:14:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec95ffb640 1 -- 172.21.15.26:0/1834041304 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec80003530 con 0x7fec980cb010 2023-12-19T18:14:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec95ffb640 1 -- 172.21.15.26:0/1834041304 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695688864 0 0) 0x7fec980ca250 con 0x7fec9815b250 2023-12-19T18:14:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec95ffb640 1 -- 172.21.15.26:0/1834041304 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec9815bdb0 con 0x7fec9815b250 2023-12-19T18:14:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec95ffb640 1 -- 172.21.15.26:0/1834041304 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1861120582 0 0) 0x7fec80003530 con 0x7fec980cb010 2023-12-19T18:14:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec967fc640 1 -- 172.21.15.26:0/1834041304 >> v1:172.21.15.160:6789/0 conn(0x7fec9815b250 legacy=0x7fec9815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec967fc640 1 -- 172.21.15.26:0/1834041304 >> v1:172.21.15.26:6789/0 conn(0x7fec980cb010 legacy=0x7fec980cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.940+0000 7fec967fc640 1 -- 172.21.15.26:0/1834041304 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec980ca530 con 0x7fec9806bb60 2023-12-19T18:14:35.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.944+0000 7fec967fc640 1 --2- 172.21.15.26:0/1834041304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec9806bb60 0x7fec9815e3f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0097f0 tx=0x7fec8c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13eeb6dcea9448fb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:35.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.944+0000 7fec95ffb640 1 -- 172.21.15.26:0/1834041304 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88001680 con 0x7fec9806bb60 2023-12-19T18:14:35.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.944+0000 7fec95ffb640 1 -- 172.21.15.26:0/1834041304 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c0a7050 con 0x7fec9806bb60 2023-12-19T18:14:35.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.944+0000 7fec95ffb640 1 -- 172.21.15.26:0/1834041304 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0ab620 con 0x7fec9806bb60 2023-12-19T18:14:35.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.944+0000 7fec9da04640 1 -- 172.21.15.26:0/1834041304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec9806bb60 msgr2=0x7fec9815e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.944+0000 7fec9da04640 1 --2- 172.21.15.26:0/1834041304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec9806bb60 0x7fec9815e3f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0097f0 tx=0x7fec8c01c3e0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.944+0000 7fec9da04640 1 -- 172.21.15.26:0/1834041304 shutdown_connections 2023-12-19T18:14:35.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.944+0000 7fec9da04640 1 --2- 172.21.15.26:0/1834041304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec9806bb60 0x7fec9815e3f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.944+0000 7fec9da04640 1 -- 172.21.15.26:0/1834041304 >> 172.21.15.26:0/1834041304 conn(0x7fec98066020 msgr2=0x7fec980b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:35.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.944+0000 7fec9da04640 1 -- 172.21.15.26:0/1834041304 shutdown_connections 2023-12-19T18:14:35.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.944+0000 7fec9da04640 1 -- 172.21.15.26:0/1834041304 wait complete. 2023-12-19T18:14:35.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.944+0000 7fec9da04640 1 Processor -- start 2023-12-19T18:14:35.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.944+0000 7fec9da04640 1 -- start start 2023-12-19T18:14:35.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec9da04640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec9806bb60 0x7fec98152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec9da04640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec980cb010 0x7fec980b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec9da04640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec9815b250 0x7fec9815c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec9da04640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec98153790 con 0x7fec980cb010 2023-12-19T18:14:35.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec9da04640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec98153a70 con 0x7fec9806bb60 2023-12-19T18:14:35.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec9da04640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec98153d50 con 0x7fec9815b250 2023-12-19T18:14:35.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec967fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec980cb010 0x7fec980b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec967fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec980cb010 0x7fec980b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33842/0 (socket says 172.21.15.26:33842) 2023-12-19T18:14:35.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec967fc640 1 -- 172.21.15.26:0/3364272320 learned_addr learned my addr 172.21.15.26:0/3364272320 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:35.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec96ffd640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec9806bb60 0x7fec98152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec977fe640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec9815b250 0x7fec9815c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec967fc640 1 -- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec9815b250 msgr2=0x7fec9815c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec967fc640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec9815b250 0x7fec9815c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec967fc640 1 -- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec9806bb60 msgr2=0x7fec98152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:35.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec967fc640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec9806bb60 0x7fec98152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:35.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec967fc640 1 -- 172.21.15.26:0/3364272320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c009420 con 0x7fec980cb010 2023-12-19T18:14:35.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec977fe640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec9815b250 0x7fec9815c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:35.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec967fc640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec980cb010 0x7fec980b1300 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0ae040 tx=0x7fec8c01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:35.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec7ffff640 1 -- 172.21.15.26:0/3364272320 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c0abcc0 con 0x7fec980cb010 2023-12-19T18:14:35.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec7ffff640 1 -- 172.21.15.26:0/3364272320 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c0abe40 con 0x7fec980cb010 2023-12-19T18:14:35.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec7ffff640 1 -- 172.21.15.26:0/3364272320 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c004260 con 0x7fec980cb010 2023-12-19T18:14:35.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec9da04640 1 -- 172.21.15.26:0/3364272320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec980cd2b0 con 0x7fec980cb010 2023-12-19T18:14:35.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec9da04640 1 -- 172.21.15.26:0/3364272320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec980ccf60 con 0x7fec980cb010 2023-12-19T18:14:35.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec9da04640 1 -- 172.21.15.26:0/3364272320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec980cd4a0 con 0x7fec980cb010 2023-12-19T18:14:35.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.948+0000 7fec7ffff640 1 -- 172.21.15.26:0/3364272320 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fec8c0043c0 con 0x7fec980cb010 2023-12-19T18:14:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.952+0000 7fec7ffff640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fec84065b80 0x7fec84068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.952+0000 7fec7ffff640 1 -- 172.21.15.26:0/3364272320 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5589+0+0 (secure 0 0 0) 0x7fec8c11cb60 con 0x7fec980cb010 2023-12-19T18:14:35.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.952+0000 7fec96ffd640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fec84065b80 0x7fec84068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:35.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.952+0000 7fec96ffd640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fec84065b80 0x7fec84068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fec88000f70 tx=0x7fec880016c0 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:35.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:35.956+0000 7fec7ffff640 1 -- 172.21.15.26:0/3364272320 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fec8c0ea240 con 0x7fec980cb010 2023-12-19T18:14:36.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:36.116+0000 7fec9da04640 1 -- 172.21.15.26:0/3364272320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fec9804e850 con 0x7fec980cb010 2023-12-19T18:14:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.116+0000 7fec7ffff640 1 -- 172.21.15.26:0/3364272320 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v193) v1 ==== 138+0+0 (secure 0 0 0) 0x7fec8c0bc0b0 con 0x7fec980cb010 2023-12-19T18:14:37.115 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T18:14:37.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.116+0000 7fec9da04640 1 -- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fec84065b80 msgr2=0x7fec84068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:37.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.116+0000 7fec9da04640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fec84065b80 0x7fec84068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fec88000f70 tx=0x7fec880016c0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.116+0000 7fec9da04640 1 -- 172.21.15.26:0/3364272320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec980cb010 msgr2=0x7fec980b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:37.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.116+0000 7fec9da04640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec980cb010 0x7fec980b1300 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0ae040 tx=0x7fec8c01ac20 comp rx=0 tx=0).stop 2023-12-19T18:14:37.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.120+0000 7fec9da04640 1 -- 172.21.15.26:0/3364272320 shutdown_connections 2023-12-19T18:14:37.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.120+0000 7fec9da04640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fec84065b80 0x7fec84068040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.120+0000 7fec9da04640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec9815b250 0x7fec9815c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.120+0000 7fec9da04640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec980cb010 0x7fec980b1300 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.120+0000 7fec9da04640 1 --2- 172.21.15.26:0/3364272320 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec9806bb60 0x7fec98152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.120+0000 7fec9da04640 1 -- 172.21.15.26:0/3364272320 >> 172.21.15.26:0/3364272320 conn(0x7fec98066020 msgr2=0x7fec980cb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:37.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.120+0000 7fec9da04640 1 -- 172.21.15.26:0/3364272320 shutdown_connections 2023-12-19T18:14:37.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.120+0000 7fec9da04640 1 -- 172.21.15.26:0/3364272320 wait complete. 2023-12-19T18:14:37.146 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:14:37.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4598e66640 1 Processor -- start 2023-12-19T18:14:37.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4598e66640 1 -- start start 2023-12-19T18:14:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4598e66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f459415ada0 0x7f45940ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4598e66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4594070db0 0x7f45940b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4598e66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45940b0a60 0x7f45940ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4598e66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45940715b0 con 0x7f45940b0a60 2023-12-19T18:14:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4598e66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45940adfd0 con 0x7f459415ada0 2023-12-19T18:14:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4598e66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459406b500 con 0x7f4594070db0 2023-12-19T18:14:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4592d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45940b0a60 0x7f45940ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4592d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f45940b0a60 0x7f45940ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33848/0 (socket says 172.21.15.26:33848) 2023-12-19T18:14:37.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4592d9d640 1 -- 172.21.15.26:0/3075208961 learned_addr learned my addr 172.21.15.26:0/3075208961 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f458bfff640 1 --2- 172.21.15.26:0/3075208961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4594070db0 0x7f45940b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f459259c640 1 --2- 172.21.15.26:0/3075208961 >> v2:172.21.15.160:3300/0 conn(0x7f459415ada0 0x7f45940ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4592d9d640 1 -- 172.21.15.26:0/3075208961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4594070db0 msgr2=0x7f45940b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4592d9d640 1 --2- 172.21.15.26:0/3075208961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4594070db0 0x7f45940b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4592d9d640 1 -- 172.21.15.26:0/3075208961 >> v2:172.21.15.160:3300/0 conn(0x7f459415ada0 msgr2=0x7f45940ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4592d9d640 1 --2- 172.21.15.26:0/3075208961 >> v2:172.21.15.160:3300/0 conn(0x7f459415ada0 0x7f45940ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f4592d9d640 1 -- 172.21.15.26:0/3075208961 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4594070b60 con 0x7f45940b0a60 2023-12-19T18:14:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f459259c640 1 --2- 172.21.15.26:0/3075208961 >> v2:172.21.15.160:3300/0 conn(0x7f459415ada0 0x7f45940ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:37.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.300+0000 7f458bfff640 1 --2- 172.21.15.26:0/3075208961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4594070db0 0x7f45940b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:37.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.304+0000 7f4592d9d640 1 --2- 172.21.15.26:0/3075208961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45940b0a60 0x7f45940ae170 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f458c009e80 tx=0x7f458c01c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd996c1f90e4e21 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:37.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.304+0000 7f4591d9b640 1 -- 172.21.15.26:0/3075208961 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c0a6460 con 0x7f45940b0a60 2023-12-19T18:14:37.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.304+0000 7f4591d9b640 1 -- 172.21.15.26:0/3075208961 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458c0a65c0 con 0x7f45940b0a60 2023-12-19T18:14:37.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.304+0000 7f4591d9b640 1 -- 172.21.15.26:0/3075208961 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c0a6950 con 0x7f45940b0a60 2023-12-19T18:14:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.304+0000 7f4598e66640 1 -- 172.21.15.26:0/3075208961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45940b0a60 msgr2=0x7f45940ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.304+0000 7f4598e66640 1 --2- 172.21.15.26:0/3075208961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45940b0a60 0x7f45940ae170 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f458c009e80 tx=0x7f458c01c6c0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.304+0000 7f4598e66640 1 -- 172.21.15.26:0/3075208961 shutdown_connections 2023-12-19T18:14:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.304+0000 7f4598e66640 1 --2- 172.21.15.26:0/3075208961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45940b0a60 0x7f45940ae170 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.304+0000 7f4598e66640 1 --2- 172.21.15.26:0/3075208961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4594070db0 0x7f45940b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.304+0000 7f4598e66640 1 --2- 172.21.15.26:0/3075208961 >> v2:172.21.15.160:3300/0 conn(0x7f459415ada0 0x7f45940ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.304+0000 7f4598e66640 1 -- 172.21.15.26:0/3075208961 >> 172.21.15.26:0/3075208961 conn(0x7f45940660d0 msgr2=0x7f45940ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.304+0000 7f4598e66640 1 -- 172.21.15.26:0/3075208961 shutdown_connections 2023-12-19T18:14:37.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.304+0000 7f4598e66640 1 -- 172.21.15.26:0/3075208961 wait complete. 2023-12-19T18:14:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f4598e66640 1 Processor -- start 2023-12-19T18:14:37.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f4598e66640 1 -- start start 2023-12-19T18:14:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f4598e66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4594070db0 0x7f45940ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f4598e66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45940b0a60 0x7f4594152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f4598e66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f459415ada0 0x7f4594158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:37.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f4598e66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45940c2280 con 0x7f4594070db0 2023-12-19T18:14:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f4598e66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4594159750 con 0x7f459415ada0 2023-12-19T18:14:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f4598e66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4594159a30 con 0x7f45940b0a60 2023-12-19T18:14:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f459259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4594070db0 0x7f45940ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f459259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4594070db0 0x7f45940ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33858/0 (socket says 172.21.15.26:33858) 2023-12-19T18:14:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f459259c640 1 -- 172.21.15.26:0/1034657261 learned_addr learned my addr 172.21.15.26:0/1034657261 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f4592d9d640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f459415ada0 0x7f4594158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f458bfff640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45940b0a60 0x7f4594152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f459259c640 1 -- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45940b0a60 msgr2=0x7f4594152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f459259c640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45940b0a60 0x7f4594152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f459259c640 1 -- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f459415ada0 msgr2=0x7f4594158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f459259c640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f459415ada0 0x7f4594158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f459259c640 1 -- 172.21.15.26:0/1034657261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c009b30 con 0x7f4594070db0 2023-12-19T18:14:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f458bfff640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45940b0a60 0x7f4594152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f459259c640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4594070db0 0x7f45940ada70 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f458400c7f0 tx=0x7f458400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f458affd640 1 -- 172.21.15.26:0/1034657261 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4584007820 con 0x7f4594070db0 2023-12-19T18:14:37.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f458affd640 1 -- 172.21.15.26:0/1034657261 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4584007980 con 0x7f4594070db0 2023-12-19T18:14:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f458affd640 1 -- 172.21.15.26:0/1034657261 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4584007d30 con 0x7f4594070db0 2023-12-19T18:14:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.308+0000 7f4598e66640 1 -- 172.21.15.26:0/1034657261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45940c2460 con 0x7f4594070db0 2023-12-19T18:14:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.312+0000 7f4598e66640 1 -- 172.21.15.26:0/1034657261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4594159460 con 0x7f4594070db0 2023-12-19T18:14:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.312+0000 7f4598e66640 1 -- 172.21.15.26:0/1034657261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45940c2980 con 0x7f4594070db0 2023-12-19T18:14:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.312+0000 7f458affd640 1 -- 172.21.15.26:0/1034657261 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f45840113f0 con 0x7f4594070db0 2023-12-19T18:14:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.312+0000 7f458affd640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4574065b80 0x7f4574068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.312+0000 7f458affd640 1 -- 172.21.15.26:0/1034657261 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5959+0+0 (secure 0 0 0) 0x7f4584083830 con 0x7f4594070db0 2023-12-19T18:14:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.312+0000 7f458bfff640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4574065b80 0x7f4574068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:37.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.316+0000 7f458bfff640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4574065b80 0x7f4574068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45940c8a00 tx=0x7f457c008040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:37.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.316+0000 7f458affd640 1 -- 172.21.15.26:0/1034657261 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f45940c2980 con 0x7f4594070db0 2023-12-19T18:14:37.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:37.476+0000 7f4598e66640 1 -- 172.21.15.26:0/1034657261 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f45940c2980 con 0x7f4594070db0 2023-12-19T18:14:38.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.128+0000 7f458affd640 1 -- 172.21.15.26:0/1034657261 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v194) v1 ==== 170+0+0 (secure 0 0 0) 0x7f45940c2980 con 0x7f4594070db0 2023-12-19T18:14:38.128 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T18:14:38.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.136+0000 7f4598e66640 1 -- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4574065b80 msgr2=0x7f4574068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.136+0000 7f4598e66640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4574065b80 0x7f4574068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f45940c8a00 tx=0x7f457c008040 comp rx=0 tx=0).stop 2023-12-19T18:14:38.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.136+0000 7f4598e66640 1 -- 172.21.15.26:0/1034657261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4594070db0 msgr2=0x7f45940ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.136+0000 7f4598e66640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4594070db0 0x7f45940ada70 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f458400c7f0 tx=0x7f458400ccb0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.140+0000 7f4598e66640 1 -- 172.21.15.26:0/1034657261 shutdown_connections 2023-12-19T18:14:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.140+0000 7f4598e66640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4574065b80 0x7f4574068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.140+0000 7f4598e66640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f459415ada0 0x7f4594158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.140+0000 7f4598e66640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f45940b0a60 0x7f4594152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.140+0000 7f4598e66640 1 --2- 172.21.15.26:0/1034657261 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4594070db0 0x7f45940ada70 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.140+0000 7f4598e66640 1 -- 172.21.15.26:0/1034657261 >> 172.21.15.26:0/1034657261 conn(0x7f45940660d0 msgr2=0x7f459406bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:38.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.140+0000 7f4598e66640 1 -- 172.21.15.26:0/1034657261 shutdown_connections 2023-12-19T18:14:38.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.140+0000 7f4598e66640 1 -- 172.21.15.26:0/1034657261 wait complete. 2023-12-19T18:14:38.171 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:14:38.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.316+0000 7f6b6b2e5640 1 Processor -- start 2023-12-19T18:14:38.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.316+0000 7f6b6b2e5640 1 -- start start 2023-12-19T18:14:38.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b6b2e5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6b6415a9c0 0x7f6b640706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b6b2e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6415ada0 0x7f6b640b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b6b2e5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b640b46c0 con 0x7f6b6406ac80 2023-12-19T18:14:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b6b2e5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b640cc270 con 0x7f6b6415a9c0 2023-12-19T18:14:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b6b2e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b64070d90 con 0x7f6b6415ada0 2023-12-19T18:14:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b69882640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6b6406ac80 0x7f6b640b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39874/0 (socket says 172.21.15.26:39874) 2023-12-19T18:14:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b69882640 1 -- 172.21.15.26:0/2759448069 learned_addr learned my addr 172.21.15.26:0/2759448069 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b69081640 1 --2- 172.21.15.26:0/2759448069 >> v2:172.21.15.160:3300/0 conn(0x7f6b6415a9c0 0x7f6b640706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b68880640 1 --2- 172.21.15.26:0/2759448069 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6415ada0 0x7f6b640b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b5bfff640 1 -- 172.21.15.26:0/2759448069 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1556321666 0 0) 0x7f6b640b46c0 con 0x7f6b6406ac80 2023-12-19T18:14:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b5bfff640 1 -- 172.21.15.26:0/2759448069 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b50003530 con 0x7f6b6406ac80 2023-12-19T18:14:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b69081640 1 -- 172.21.15.26:0/2759448069 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6415ada0 msgr2=0x7f6b640b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b69081640 1 --2- 172.21.15.26:0/2759448069 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6415ada0 0x7f6b640b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b69081640 1 -- 172.21.15.26:0/2759448069 >> v1:172.21.15.26:6789/0 conn(0x7f6b6406ac80 legacy=0x7f6b640b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b69081640 1 -- 172.21.15.26:0/2759448069 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b64071070 con 0x7f6b6415a9c0 2023-12-19T18:14:38.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.320+0000 7f6b68880640 1 --2- 172.21.15.26:0/2759448069 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6415ada0 0x7f6b640b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:38.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b69081640 1 --2- 172.21.15.26:0/2759448069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b6415a9c0 0x7f6b640706d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6b600097c0 tx=0x7f6b6001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a74315c9ad8d2181 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:38.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b5bfff640 1 -- 172.21.15.26:0/2759448069 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600ac620 con 0x7f6b6415a9c0 2023-12-19T18:14:38.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b5bfff640 1 -- 172.21.15.26:0/2759448069 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b600ac780 con 0x7f6b6415a9c0 2023-12-19T18:14:38.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b5bfff640 1 -- 172.21.15.26:0/2759448069 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600acb10 con 0x7f6b6415a9c0 2023-12-19T18:14:38.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2759448069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b6415a9c0 msgr2=0x7f6b640706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b6b2e5640 1 --2- 172.21.15.26:0/2759448069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b6415a9c0 0x7f6b640706d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6b600097c0 tx=0x7f6b6001c3c0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2759448069 shutdown_connections 2023-12-19T18:14:38.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b6b2e5640 1 --2- 172.21.15.26:0/2759448069 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6415ada0 0x7f6b640b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b6b2e5640 1 --2- 172.21.15.26:0/2759448069 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b6415a9c0 0x7f6b640706d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2759448069 >> 172.21.15.26:0/2759448069 conn(0x7f6b640ce730 msgr2=0x7f6b640ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:38.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2759448069 shutdown_connections 2023-12-19T18:14:38.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2759448069 wait complete. 2023-12-19T18:14:38.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b6b2e5640 1 Processor -- start 2023-12-19T18:14:38.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.324+0000 7f6b6b2e5640 1 -- start start 2023-12-19T18:14:38.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b6b2e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6406ac80 0x7f6b640c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b6b2e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6415a9c0 0x7f6b640c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b6b2e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b6415ada0 0x7f6b640ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b6b2e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640b1770 con 0x7f6b6415a9c0 2023-12-19T18:14:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b6b2e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640c5d50 con 0x7f6b6415ada0 2023-12-19T18:14:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b6b2e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b640c6030 con 0x7f6b6406ac80 2023-12-19T18:14:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b68880640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6415a9c0 0x7f6b640c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b68880640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6415a9c0 0x7f6b640c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33866/0 (socket says 172.21.15.26:33866) 2023-12-19T18:14:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b68880640 1 -- 172.21.15.26:0/2126849160 learned_addr learned my addr 172.21.15.26:0/2126849160 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b69081640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6406ac80 0x7f6b640c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b69882640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b6415ada0 0x7f6b640ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b68880640 1 -- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6406ac80 msgr2=0x7f6b640c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b68880640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6406ac80 0x7f6b640c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b68880640 1 -- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b6415ada0 msgr2=0x7f6b640ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b68880640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b6415ada0 0x7f6b640ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b68880640 1 -- 172.21.15.26:0/2126849160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b54009420 con 0x7f6b6415a9c0 2023-12-19T18:14:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b69081640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6406ac80 0x7f6b640c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b69882640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b6415ada0 0x7f6b640ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b68880640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6415a9c0 0x7f6b640c7cc0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400e980 tx=0x7f6b5400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b59ffb640 1 -- 172.21.15.26:0/2126849160 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5400ca00 con 0x7f6b6415a9c0 2023-12-19T18:14:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b59ffb640 1 -- 172.21.15.26:0/2126849160 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5400cb60 con 0x7f6b6415a9c0 2023-12-19T18:14:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b59ffb640 1 -- 172.21.15.26:0/2126849160 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54013470 con 0x7f6b6415a9c0 2023-12-19T18:14:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2126849160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b60009420 con 0x7f6b6415a9c0 2023-12-19T18:14:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2126849160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b640b1390 con 0x7f6b6415a9c0 2023-12-19T18:14:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.328+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2126849160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b64074270 con 0x7f6b6415a9c0 2023-12-19T18:14:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.332+0000 7f6b59ffb640 1 -- 172.21.15.26:0/2126849160 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6b540135d0 con 0x7f6b6415a9c0 2023-12-19T18:14:38.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.332+0000 7f6b59ffb640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b48065b80 0x7f6b48068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.332+0000 7f6b59ffb640 1 -- 172.21.15.26:0/2126849160 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6341+0+0 (secure 0 0 0) 0x7f6b54083f20 con 0x7f6b6415a9c0 2023-12-19T18:14:38.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.332+0000 7f6b69081640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b48065b80 0x7f6b48068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.332+0000 7f6b69081640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b48065b80 0x7f6b48068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b600ae000 tx=0x7f6b600a7000 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:38.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.336+0000 7f6b59ffb640 1 -- 172.21.15.26:0/2126849160 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6b54051670 con 0x7f6b6415a9c0 2023-12-19T18:14:38.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.524+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2126849160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6b640c8cd0 con 0x7f6b6415a9c0 2023-12-19T18:14:38.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.532+0000 7f6b59ffb640 1 -- 172.21.15.26:0/2126849160 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v84) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6b54054c90 con 0x7f6b6415a9c0 2023-12-19T18:14:38.536 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 22 and data pool 23 2023-12-19T18:14:38.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.540+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b48065b80 msgr2=0x7f6b48068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.540+0000 7f6b6b2e5640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b48065b80 0x7f6b48068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b600ae000 tx=0x7f6b600a7000 comp rx=0 tx=0).stop 2023-12-19T18:14:38.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.540+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2126849160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6415a9c0 msgr2=0x7f6b640c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.540+0000 7f6b6b2e5640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6415a9c0 0x7f6b640c7cc0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400e980 tx=0x7f6b5400ee40 comp rx=0 tx=0).stop 2023-12-19T18:14:38.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.540+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2126849160 shutdown_connections 2023-12-19T18:14:38.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.540+0000 7f6b6b2e5640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b48065b80 0x7f6b48068040 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6b600ae000 tx=0x7f6b600a7000 comp rx=0 tx=0).stop 2023-12-19T18:14:38.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.540+0000 7f6b6b2e5640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b6415ada0 0x7f6b640ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.540+0000 7f6b6b2e5640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6415a9c0 0x7f6b640c7cc0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.540+0000 7f6b6b2e5640 1 --2- 172.21.15.26:0/2126849160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6406ac80 0x7f6b640c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.540+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2126849160 >> 172.21.15.26:0/2126849160 conn(0x7f6b640ce730 msgr2=0x7f6b64154170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:38.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.540+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2126849160 shutdown_connections 2023-12-19T18:14:38.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.544+0000 7f6b6b2e5640 1 -- 172.21.15.26:0/2126849160 wait complete. 2023-12-19T18:14:38.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:38.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.720+0000 7f66cd241640 1 Processor -- start 2023-12-19T18:14:38.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.720+0000 7f66cd241640 1 -- start start 2023-12-19T18:14:38.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.720+0000 7f66cd241640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f66c815c7f0 0x7f66c806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66cd241640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66c80adb80 0x7f66c80cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66cd241640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c806ced0 con 0x7f66c80ccc20 2023-12-19T18:14:38.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66cd241640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c806d770 con 0x7f66c80adb80 2023-12-19T18:14:38.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66cd241640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c806d1b0 con 0x7f66c815c7f0 2023-12-19T18:14:38.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66c659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66c80adb80 0x7f66c80cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66c659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f66c80adb80 0x7f66c80cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33868/0 (socket says 172.21.15.26:33868) 2023-12-19T18:14:38.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66c659c640 1 -- 172.21.15.26:0/516784757 learned_addr learned my addr 172.21.15.26:0/516784757 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:38.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66c6d9d640 1 --2- 172.21.15.26:0/516784757 >> v2:172.21.15.160:3300/0 conn(0x7f66c815c7f0 0x7f66c806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66c659c640 1 -- 172.21.15.26:0/516784757 >> v2:172.21.15.160:3300/0 conn(0x7f66c815c7f0 msgr2=0x7f66c806c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:38.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66c659c640 1 --2- 172.21.15.26:0/516784757 >> v2:172.21.15.160:3300/0 conn(0x7f66c815c7f0 0x7f66c806c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66c659c640 1 -- 172.21.15.26:0/516784757 >> v1:172.21.15.160:6789/0 conn(0x7f66c80ccc20 legacy=0x7f66c80b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:38.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66c659c640 1 -- 172.21.15.26:0/516784757 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c806d490 con 0x7f66c80adb80 2023-12-19T18:14:38.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66c6d9d640 1 --2- 172.21.15.26:0/516784757 >> v2:172.21.15.160:3300/0 conn(0x7f66c815c7f0 0x7f66c806c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:38.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66c659c640 1 --2- 172.21.15.26:0/516784757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c80adb80 0x7f66c80cc6e0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f66c80667b0 tx=0x7f66bc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f622f6ab0a792e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:38.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66c5d9b640 1 -- 172.21.15.26:0/516784757 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0ac410 con 0x7f66c80adb80 2023-12-19T18:14:38.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66c5d9b640 1 -- 172.21.15.26:0/516784757 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc0ac570 con 0x7f66c80adb80 2023-12-19T18:14:38.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.724+0000 7f66c5d9b640 1 -- 172.21.15.26:0/516784757 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0ac900 con 0x7f66c80adb80 2023-12-19T18:14:38.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 -- 172.21.15.26:0/516784757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c80adb80 msgr2=0x7f66c80cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 --2- 172.21.15.26:0/516784757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c80adb80 0x7f66c80cc6e0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f66c80667b0 tx=0x7f66bc01c4a0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 -- 172.21.15.26:0/516784757 shutdown_connections 2023-12-19T18:14:38.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 --2- 172.21.15.26:0/516784757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c80adb80 0x7f66c80cc6e0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 --2- 172.21.15.26:0/516784757 >> v2:172.21.15.160:3300/0 conn(0x7f66c815c7f0 0x7f66c806c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 -- 172.21.15.26:0/516784757 >> 172.21.15.26:0/516784757 conn(0x7f66c80658e0 msgr2=0x7f66c8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:38.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 -- 172.21.15.26:0/516784757 shutdown_connections 2023-12-19T18:14:38.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 -- 172.21.15.26:0/516784757 wait complete. 2023-12-19T18:14:38.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 Processor -- start 2023-12-19T18:14:38.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 -- start start 2023-12-19T18:14:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c80adb80 0x7f66c814e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66c80ccc20 0x7f66c8155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f66c815c7f0 0x7f66c80ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c8150320 con 0x7f66c80adb80 2023-12-19T18:14:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c81561f0 con 0x7f66c80ccc20 2023-12-19T18:14:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66cd241640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c8154760 con 0x7f66c815c7f0 2023-12-19T18:14:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66c6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c80adb80 0x7f66c814e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66c6d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c80adb80 0x7f66c814e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33878/0 (socket says 172.21.15.26:33878) 2023-12-19T18:14:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66c659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66c80ccc20 0x7f66c8155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66c6d9d640 1 -- 172.21.15.26:0/3283995778 learned_addr learned my addr 172.21.15.26:0/3283995778 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.728+0000 7f66c759e640 1 --2- 172.21.15.26:0/3283995778 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f66c815c7f0 0x7f66c80ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.732+0000 7f66c6d9d640 1 -- 172.21.15.26:0/3283995778 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f66c815c7f0 msgr2=0x7f66c80ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.732+0000 7f66c6d9d640 1 --2- 172.21.15.26:0/3283995778 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f66c815c7f0 0x7f66c80ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.732+0000 7f66c6d9d640 1 -- 172.21.15.26:0/3283995778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66c80ccc20 msgr2=0x7f66c8155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.732+0000 7f66c6d9d640 1 --2- 172.21.15.26:0/3283995778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66c80ccc20 0x7f66c8155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.732+0000 7f66c6d9d640 1 -- 172.21.15.26:0/3283995778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc0094d0 con 0x7f66c80adb80 2023-12-19T18:14:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.732+0000 7f66c6d9d640 1 --2- 172.21.15.26:0/3283995778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c80adb80 0x7f66c814e7d0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c7f0 tx=0x7f66b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.732+0000 7f66af7fe640 1 -- 172.21.15.26:0/3283995778 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b80077c0 con 0x7f66c80adb80 2023-12-19T18:14:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.732+0000 7f66af7fe640 1 -- 172.21.15.26:0/3283995778 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b8007920 con 0x7f66c80adb80 2023-12-19T18:14:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.732+0000 7f66af7fe640 1 -- 172.21.15.26:0/3283995778 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8007cd0 con 0x7f66c80adb80 2023-12-19T18:14:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.732+0000 7f66cd241640 1 -- 172.21.15.26:0/3283995778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c8154a40 con 0x7f66c80adb80 2023-12-19T18:14:38.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.732+0000 7f66cd241640 1 -- 172.21.15.26:0/3283995778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c8155000 con 0x7f66c80adb80 2023-12-19T18:14:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.732+0000 7f66cd241640 1 -- 172.21.15.26:0/3283995778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c8074270 con 0x7f66c80adb80 2023-12-19T18:14:38.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.732+0000 7f66af7fe640 1 -- 172.21.15.26:0/3283995778 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f66b80113f0 con 0x7f66c80adb80 2023-12-19T18:14:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.736+0000 7f66af7fe640 1 --2- 172.21.15.26:0/3283995778 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f66b4065b80 0x7f66b4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.736+0000 7f66af7fe640 1 -- 172.21.15.26:0/3283995778 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f66b8083920 con 0x7f66c80adb80 2023-12-19T18:14:38.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.736+0000 7f66c659c640 1 --2- 172.21.15.26:0/3283995778 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f66b4065b80 0x7f66b4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.736+0000 7f66c659c640 1 --2- 172.21.15.26:0/3283995778 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f66b4065b80 0x7f66b4068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0ae000 tx=0x7f66bc009040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:38.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.740+0000 7f66af7fe640 1 -- 172.21.15.26:0/3283995778 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f66b8050e80 con 0x7f66c80adb80 2023-12-19T18:14:38.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.896+0000 7f66cd241640 1 -- 172.21.15.26:0/3283995778 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66c8070fd0 con 0x7f66c80adb80 2023-12-19T18:14:38.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.896+0000 7f66af7fe640 1 -- 172.21.15.26:0/3283995778 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+17221 (secure 0 0 0) 0x7f66b80544a0 con 0x7f66c80adb80 2023-12-19T18:14:38.898 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:38.899 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":195,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:14:38.527518+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:36.168666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:37.527319+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:38.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.900+0000 7f66cd241640 1 -- 172.21.15.26:0/3283995778 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f66b4065b80 msgr2=0x7f66b4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.900+0000 7f66cd241640 1 --2- 172.21.15.26:0/3283995778 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f66b4065b80 0x7f66b4068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f66bc0ae000 tx=0x7f66bc009040 comp rx=0 tx=0).stop 2023-12-19T18:14:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.900+0000 7f66cd241640 1 -- 172.21.15.26:0/3283995778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c80adb80 msgr2=0x7f66c814e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.900+0000 7f66cd241640 1 --2- 172.21.15.26:0/3283995778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c80adb80 0x7f66c814e7d0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c7f0 tx=0x7f66b800ccb0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.900+0000 7f66cd241640 1 -- 172.21.15.26:0/3283995778 shutdown_connections 2023-12-19T18:14:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.900+0000 7f66cd241640 1 --2- 172.21.15.26:0/3283995778 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f66b4065b80 0x7f66b4068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.900+0000 7f66cd241640 1 --2- 172.21.15.26:0/3283995778 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f66c815c7f0 0x7f66c80ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.900+0000 7f66cd241640 1 --2- 172.21.15.26:0/3283995778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f66c80ccc20 0x7f66c8155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.900+0000 7f66cd241640 1 --2- 172.21.15.26:0/3283995778 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f66c80adb80 0x7f66c814e7d0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.900+0000 7f66cd241640 1 -- 172.21.15.26:0/3283995778 >> 172.21.15.26:0/3283995778 conn(0x7f66c80658e0 msgr2=0x7f66c80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.900+0000 7f66cd241640 1 -- 172.21.15.26:0/3283995778 shutdown_connections 2023-12-19T18:14:38.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:38.900+0000 7f66cd241640 1 -- 172.21.15.26:0/3283995778 wait complete. 2023-12-19T18:14:38.920 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.064+0000 7f9d61c8c640 1 Processor -- start 2023-12-19T18:14:39.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.064+0000 7f9d61c8c640 1 -- start start 2023-12-19T18:14:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.064+0000 7f9d61c8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d5c0ccc20 0x7f9d5c161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.064+0000 7f9d61c8c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9d5c0d0480 0x7f9d5c0cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.064+0000 7f9d61c8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d5c160890 con 0x7f9d5c0cd050 2023-12-19T18:14:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.064+0000 7f9d61c8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5c0cdaf0 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.064+0000 7f9d61c8c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5c160b40 con 0x7f9d5c0d0480 2023-12-19T18:14:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.064+0000 7f9d5b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d5c0ccc20 0x7f9d5c161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.064+0000 7f9d5affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9d5c0cd050 0x7f9d5c0cfd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39886/0 (socket says 172.21.15.26:39886) 2023-12-19T18:14:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.064+0000 7f9d5affd640 1 -- 172.21.15.26:0/56464808 learned_addr learned my addr 172.21.15.26:0/56464808 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d5bfff640 1 --2- 172.21.15.26:0/56464808 >> v2:172.21.15.160:3300/0 conn(0x7f9d5c0d0480 0x7f9d5c0cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d5b7fe640 1 -- 172.21.15.26:0/56464808 >> v2:172.21.15.160:3300/0 conn(0x7f9d5c0d0480 msgr2=0x7f9d5c0cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d5b7fe640 1 --2- 172.21.15.26:0/56464808 >> v2:172.21.15.160:3300/0 conn(0x7f9d5c0d0480 0x7f9d5c0cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d5b7fe640 1 -- 172.21.15.26:0/56464808 >> v1:172.21.15.26:6789/0 conn(0x7f9d5c0cd050 legacy=0x7f9d5c0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d5b7fe640 1 -- 172.21.15.26:0/56464808 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c160e20 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d5a7fc640 1 -- 172.21.15.26:0/56464808 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182487568 0 0) 0x7f9d50001120 con 0x7f9d5c0cd050 2023-12-19T18:14:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d5b7fe640 1 --2- 172.21.15.26:0/56464808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d5c0ccc20 0x7f9d5c161620 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c06e5e0 tx=0x7f9d4c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e70cb6064e9bf4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d5a7fc640 1 -- 172.21.15.26:0/56464808 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0a8050 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d5a7fc640 1 -- 172.21.15.26:0/56464808 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c0ac5f0 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d5a7fc640 1 -- 172.21.15.26:0/56464808 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0ac980 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d61c8c640 1 -- 172.21.15.26:0/56464808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d5c0ccc20 msgr2=0x7f9d5c161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d61c8c640 1 --2- 172.21.15.26:0/56464808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d5c0ccc20 0x7f9d5c161620 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c06e5e0 tx=0x7f9d4c01c3d0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d61c8c640 1 -- 172.21.15.26:0/56464808 shutdown_connections 2023-12-19T18:14:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d61c8c640 1 --2- 172.21.15.26:0/56464808 >> v2:172.21.15.160:3300/0 conn(0x7f9d5c0d0480 0x7f9d5c0cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d61c8c640 1 --2- 172.21.15.26:0/56464808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d5c0ccc20 0x7f9d5c161620 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.068+0000 7f9d61c8c640 1 -- 172.21.15.26:0/56464808 >> 172.21.15.26:0/56464808 conn(0x7f9d5c06d510 msgr2=0x7f9d5c06d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d61c8c640 1 -- 172.21.15.26:0/56464808 shutdown_connections 2023-12-19T18:14:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d61c8c640 1 -- 172.21.15.26:0/56464808 wait complete. 2023-12-19T18:14:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d61c8c640 1 Processor -- start 2023-12-19T18:14:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d61c8c640 1 -- start start 2023-12-19T18:14:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d61c8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d5c0ccc20 0x7f9d5c0c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d61c8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9d5c0cd050 0x7f9d5c0c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d61c8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d5c0d0480 0x7f9d5c0c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d61c8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c1591d0 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d61c8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c1594b0 con 0x7f9d5c0d0480 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d61c8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c159790 con 0x7f9d5c0cd050 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d5b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d5c0ccc20 0x7f9d5c0c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d5b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d5c0ccc20 0x7f9d5c0c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33898/0 (socket says 172.21.15.26:33898) 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d5b7fe640 1 -- 172.21.15.26:0/3476027645 learned_addr learned my addr 172.21.15.26:0/3476027645 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d5bfff640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d5c0d0480 0x7f9d5c0c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d5affd640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9d5c0cd050 0x7f9d5c0c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d5b7fe640 1 -- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9d5c0cd050 msgr2=0x7f9d5c0c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d5b7fe640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9d5c0cd050 0x7f9d5c0c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d5b7fe640 1 -- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d5c0d0480 msgr2=0x7f9d5c0c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d5b7fe640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d5c0d0480 0x7f9d5c0c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d5b7fe640 1 -- 172.21.15.26:0/3476027645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c009420 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d5bfff640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d5c0d0480 0x7f9d5c0c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.072+0000 7f9d5b7fe640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d5c0ccc20 0x7f9d5c0c4580 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c01aa80 tx=0x7f9d4c019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.076+0000 7f9d43fff640 1 -- 172.21.15.26:0/3476027645 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0a8050 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.076+0000 7f9d43fff640 1 -- 172.21.15.26:0/3476027645 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c0b5640 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.076+0000 7f9d43fff640 1 -- 172.21.15.26:0/3476027645 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0b59f0 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.076+0000 7f9d61c8c640 1 -- 172.21.15.26:0/3476027645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5c159a70 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.076+0000 7f9d61c8c640 1 -- 172.21.15.26:0/3476027645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c0c6bd0 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.076+0000 7f9d61c8c640 1 -- 172.21.15.26:0/3476027645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5c074270 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.076+0000 7f9d43fff640 1 -- 172.21.15.26:0/3476027645 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9d4c0b3040 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.076+0000 7f9d43fff640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9d48065b80 0x7f9d48068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.076+0000 7f9d43fff640 1 -- 172.21.15.26:0/3476027645 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f9d4c11dd70 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.076+0000 7f9d5affd640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9d48065b80 0x7f9d48068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.080+0000 7f9d5affd640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9d48065b80 0x7f9d48068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9d50007fd0 tx=0x7f9d500079c0 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.080+0000 7f9d43fff640 1 -- 172.21.15.26:0/3476027645 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9d4c0eb210 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.103 INFO:tasks.ceph.mds.c.smithi026.stderr:2023-12-19T18:14:39.104+0000 7f517c464640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:14:39.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.244+0000 7f9d61c8c640 1 -- 172.21.15.26:0/3476027645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d5c159f20 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.244+0000 7f9d43fff640 1 -- 172.21.15.26:0/3476027645 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+17221 (secure 0 0 0) 0x7f9d4c0b9090 con 0x7f9d5c0ccc20 2023-12-19T18:14:39.245 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:39.245 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":195,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:14:38.527518+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:36.168666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:37.527319+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.244+0000 7f9d61c8c640 1 -- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9d48065b80 msgr2=0x7f9d48068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.244+0000 7f9d61c8c640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9d48065b80 0x7f9d48068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9d50007fd0 tx=0x7f9d500079c0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.248+0000 7f9d61c8c640 1 -- 172.21.15.26:0/3476027645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d5c0ccc20 msgr2=0x7f9d5c0c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.248+0000 7f9d61c8c640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d5c0ccc20 0x7f9d5c0c4580 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c01aa80 tx=0x7f9d4c019170 comp rx=0 tx=0).stop 2023-12-19T18:14:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.248+0000 7f9d61c8c640 1 -- 172.21.15.26:0/3476027645 shutdown_connections 2023-12-19T18:14:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.248+0000 7f9d61c8c640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9d48065b80 0x7f9d48068040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.248+0000 7f9d61c8c640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d5c0d0480 0x7f9d5c0c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.248+0000 7f9d61c8c640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9d5c0cd050 0x7f9d5c0c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.248+0000 7f9d61c8c640 1 --2- 172.21.15.26:0/3476027645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d5c0ccc20 0x7f9d5c0c4580 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.248+0000 7f9d61c8c640 1 -- 172.21.15.26:0/3476027645 >> 172.21.15.26:0/3476027645 conn(0x7f9d5c06d510 msgr2=0x7f9d5c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:39.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.248+0000 7f9d61c8c640 1 -- 172.21.15.26:0/3476027645 shutdown_connections 2023-12-19T18:14:39.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.248+0000 7f9d61c8c640 1 -- 172.21.15.26:0/3476027645 wait complete. 2023-12-19T18:14:39.267 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:14:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.412+0000 7eff1242d640 1 Processor -- start 2023-12-19T18:14:39.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.412+0000 7eff1242d640 1 -- start start 2023-12-19T18:14:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.412+0000 7eff1242d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7eff0c0706d0 0x7eff0c0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.412+0000 7eff1242d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c070ae0 con 0x7eff0c15a9c0 2023-12-19T18:14:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.412+0000 7eff1242d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c070d90 con 0x7eff0c15ada0 2023-12-19T18:14:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.412+0000 7eff1242d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0c0cc270 con 0x7eff0c0706d0 2023-12-19T18:14:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.412+0000 7eff0bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7eff0c15a9c0 0x7eff0c0b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39896/0 (socket says 172.21.15.26:39896) 2023-12-19T18:14:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.412+0000 7eff0bfff640 1 -- 172.21.15.26:0/814773859 learned_addr learned my addr 172.21.15.26:0/814773859 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.412+0000 7eff109ca640 1 --2- 172.21.15.26:0/814773859 >> v2:172.21.15.26:3300/0 conn(0x7eff0c0706d0 0x7eff0c0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.416+0000 7eff109ca640 1 -- 172.21.15.26:0/814773859 >> v1:172.21.15.160:6789/0 conn(0x7eff0c15ada0 legacy=0x7eff0c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.416+0000 7eff109ca640 1 -- 172.21.15.26:0/814773859 >> v1:172.21.15.26:6789/0 conn(0x7eff0c15a9c0 legacy=0x7eff0c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.416+0000 7eff0affd640 1 -- 172.21.15.26:0/814773859 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280593785 0 0) 0x7eff0c070d90 con 0x7eff0c15ada0 2023-12-19T18:14:39.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.416+0000 7eff109ca640 1 -- 172.21.15.26:0/814773859 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c071070 con 0x7eff0c0706d0 2023-12-19T18:14:39.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.416+0000 7eff109ca640 1 --2- 172.21.15.26:0/814773859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff0c0706d0 0x7eff0c0b01b0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7eff04009d90 tx=0x7eff0401c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72151712c4e921c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.416+0000 7eff0affd640 1 -- 172.21.15.26:0/814773859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff040a6680 con 0x7eff0c0706d0 2023-12-19T18:14:39.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.416+0000 7eff0affd640 1 -- 172.21.15.26:0/814773859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff040a67e0 con 0x7eff0c0706d0 2023-12-19T18:14:39.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.416+0000 7eff0affd640 1 -- 172.21.15.26:0/814773859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff040a6b70 con 0x7eff0c0706d0 2023-12-19T18:14:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.416+0000 7eff1242d640 1 -- 172.21.15.26:0/814773859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff0c0706d0 msgr2=0x7eff0c0b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.416+0000 7eff1242d640 1 --2- 172.21.15.26:0/814773859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff0c0706d0 0x7eff0c0b01b0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7eff04009d90 tx=0x7eff0401c720 comp rx=0 tx=0).stop 2023-12-19T18:14:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.416+0000 7eff1242d640 1 -- 172.21.15.26:0/814773859 shutdown_connections 2023-12-19T18:14:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.416+0000 7eff1242d640 1 --2- 172.21.15.26:0/814773859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff0c0706d0 0x7eff0c0b01b0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.416+0000 7eff1242d640 1 -- 172.21.15.26:0/814773859 >> 172.21.15.26:0/814773859 conn(0x7eff0c0ce730 msgr2=0x7eff0c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff1242d640 1 -- 172.21.15.26:0/814773859 shutdown_connections 2023-12-19T18:14:39.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff1242d640 1 -- 172.21.15.26:0/814773859 wait complete. 2023-12-19T18:14:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff1242d640 1 Processor -- start 2023-12-19T18:14:39.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff1242d640 1 -- start start 2023-12-19T18:14:39.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff1242d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff0c0706d0 0x7eff0c152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff1242d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff0c15a9c0 0x7eff0c0c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff1242d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff0c15ada0 0x7eff0c0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff1242d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0b2450 con 0x7eff0c15a9c0 2023-12-19T18:14:39.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff1242d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c152ea0 con 0x7eff0c0706d0 2023-12-19T18:14:39.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff1242d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0c5770 con 0x7eff0c15ada0 2023-12-19T18:14:39.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff0b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff0c15a9c0 0x7eff0c0c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff0b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff0c15a9c0 0x7eff0c0c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33928/0 (socket says 172.21.15.26:33928) 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff0b7fe640 1 -- 172.21.15.26:0/3676821949 learned_addr learned my addr 172.21.15.26:0/3676821949 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff0bfff640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff0c0706d0 0x7eff0c152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff109ca640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff0c15ada0 0x7eff0c0c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff0b7fe640 1 -- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff0c15ada0 msgr2=0x7eff0c0c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff0b7fe640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff0c15ada0 0x7eff0c0c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff0b7fe640 1 -- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff0c0706d0 msgr2=0x7eff0c152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff0b7fe640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff0c0706d0 0x7eff0c152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff0b7fe640 1 -- 172.21.15.26:0/3676821949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff04009a70 con 0x7eff0c15a9c0 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff109ca640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff0c15ada0 0x7eff0c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff0bfff640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff0c0706d0 0x7eff0c152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.420+0000 7eff0b7fe640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff0c15a9c0 0x7eff0c0c7290 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7eff0000c4a0 tx=0x7eff0000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.424+0000 7eff08ff9640 1 -- 172.21.15.26:0/3676821949 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00005480 con 0x7eff0c15a9c0 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.424+0000 7eff08ff9640 1 -- 172.21.15.26:0/3676821949 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff00003600 con 0x7eff0c15a9c0 2023-12-19T18:14:39.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.424+0000 7eff08ff9640 1 -- 172.21.15.26:0/3676821949 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff000039b0 con 0x7eff0c15a9c0 2023-12-19T18:14:39.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.424+0000 7eff1242d640 1 -- 172.21.15.26:0/3676821949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0c0c5a50 con 0x7eff0c15a9c0 2023-12-19T18:14:39.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.424+0000 7eff1242d640 1 -- 172.21.15.26:0/3676821949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0c0b20e0 con 0x7eff0c15a9c0 2023-12-19T18:14:39.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.424+0000 7eff1242d640 1 -- 172.21.15.26:0/3676821949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0c0b25c0 con 0x7eff0c15a9c0 2023-12-19T18:14:39.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.424+0000 7eff08ff9640 1 -- 172.21.15.26:0/3676821949 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7eff00003b10 con 0x7eff0c15a9c0 2023-12-19T18:14:39.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.424+0000 7eff08ff9640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7efef8065b80 0x7efef8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.424+0000 7eff08ff9640 1 -- 172.21.15.26:0/3676821949 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6441+0+0 (secure 0 0 0) 0x7eff00082c30 con 0x7eff0c15a9c0 2023-12-19T18:14:39.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.424+0000 7eff0bfff640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7efef8065b80 0x7efef8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.424+0000 7eff0bfff640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7efef8065b80 0x7efef8068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efefc000f70 tx=0x7efefc007d30 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.428+0000 7eff08ff9640 1 -- 172.21.15.26:0/3676821949 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7eff00050160 con 0x7eff0c15a9c0 2023-12-19T18:14:39.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:39.620+0000 7eff1242d640 1 -- 172.21.15.26:0/3676821949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7eff0c06b360 con 0x7eff0c15a9c0 2023-12-19T18:14:40.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.540+0000 7eff08ff9640 1 -- 172.21.15.26:0/3676821949 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v86) v1 ==== 118+0+0 (secure 0 0 0) 0x7eff00017070 con 0x7eff0c15a9c0 2023-12-19T18:14:40.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.544+0000 7eff1242d640 1 -- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7efef8065b80 msgr2=0x7efef8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:40.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.544+0000 7eff1242d640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7efef8065b80 0x7efef8068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efefc000f70 tx=0x7efefc007d30 comp rx=0 tx=0).stop 2023-12-19T18:14:40.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.544+0000 7eff1242d640 1 -- 172.21.15.26:0/3676821949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff0c15a9c0 msgr2=0x7eff0c0c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:40.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.544+0000 7eff1242d640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff0c15a9c0 0x7eff0c0c7290 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7eff0000c4a0 tx=0x7eff0000c960 comp rx=0 tx=0).stop 2023-12-19T18:14:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.544+0000 7eff1242d640 1 -- 172.21.15.26:0/3676821949 shutdown_connections 2023-12-19T18:14:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.544+0000 7eff1242d640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7efef8065b80 0x7efef8068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.544+0000 7eff1242d640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7eff0c15ada0 0x7eff0c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.544+0000 7eff1242d640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff0c15a9c0 0x7eff0c0c7290 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.544+0000 7eff1242d640 1 --2- 172.21.15.26:0/3676821949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7eff0c0706d0 0x7eff0c152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.544+0000 7eff1242d640 1 -- 172.21.15.26:0/3676821949 >> 172.21.15.26:0/3676821949 conn(0x7eff0c0ce730 msgr2=0x7eff0c06c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.544+0000 7eff1242d640 1 -- 172.21.15.26:0/3676821949 shutdown_connections 2023-12-19T18:14:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.544+0000 7eff1242d640 1 -- 172.21.15.26:0/3676821949 wait complete. 2023-12-19T18:14:40.567 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:14:40.567 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:14:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.716+0000 7f99eb59e640 1 Processor -- start 2023-12-19T18:14:40.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.716+0000 7f99eb59e640 1 -- start start 2023-12-19T18:14:40.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99eb59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99ec15ce50 0x7f99ec0b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:40.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99eb59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f99ec06b850 0x7f99ec15d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:40.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99eb59e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99ec0722c0 con 0x7f99ec0b2830 2023-12-19T18:14:40.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99eb59e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ec161460 con 0x7f99ec06b850 2023-12-19T18:14:40.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99eb59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec0b2c70 con 0x7f99ec15ce50 2023-12-19T18:14:40.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99e3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f99ec0b2830 0x7f99ec06b140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39902/0 (socket says 172.21.15.26:39902) 2023-12-19T18:14:40.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99ead9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f99ec06b850 0x7f99ec15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:40.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99e3fff640 1 -- 172.21.15.26:0/674920787 learned_addr learned my addr 172.21.15.26:0/674920787 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:40.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99ea59c640 1 --2- 172.21.15.26:0/674920787 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99ec15ce50 0x7f99ec0b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:40.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99ead9d640 1 -- 172.21.15.26:0/674920787 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99ec15ce50 msgr2=0x7f99ec0b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:40.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99ead9d640 1 --2- 172.21.15.26:0/674920787 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99ec15ce50 0x7f99ec0b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99ead9d640 1 -- 172.21.15.26:0/674920787 >> v1:172.21.15.26:6789/0 conn(0x7f99ec0b2830 legacy=0x7f99ec06b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:40.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99ead9d640 1 -- 172.21.15.26:0/674920787 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec0b2e60 con 0x7f99ec06b850 2023-12-19T18:14:40.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.720+0000 7f99e9d9b640 1 -- 172.21.15.26:0/674920787 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3974799950 0 0) 0x7f99ec0722c0 con 0x7f99ec0b2830 2023-12-19T18:14:40.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99ead9d640 1 --2- 172.21.15.26:0/674920787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ec06b850 0x7f99ec15d230 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009d90 tx=0x7f99e401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48e0a70406a10f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:40.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99e9d9b640 1 -- 172.21.15.26:0/674920787 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e40a5780 con 0x7f99ec06b850 2023-12-19T18:14:40.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99e9d9b640 1 -- 172.21.15.26:0/674920787 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e40a58e0 con 0x7f99ec06b850 2023-12-19T18:14:40.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99e9d9b640 1 -- 172.21.15.26:0/674920787 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e40a5c70 con 0x7f99ec06b850 2023-12-19T18:14:40.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99eb59e640 1 -- 172.21.15.26:0/674920787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ec06b850 msgr2=0x7f99ec15d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:40.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99eb59e640 1 --2- 172.21.15.26:0/674920787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ec06b850 0x7f99ec15d230 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f99e4009d90 tx=0x7f99e401c8c0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99eb59e640 1 -- 172.21.15.26:0/674920787 shutdown_connections 2023-12-19T18:14:40.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99eb59e640 1 --2- 172.21.15.26:0/674920787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ec06b850 0x7f99ec15d230 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99eb59e640 1 --2- 172.21.15.26:0/674920787 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99ec15ce50 0x7f99ec0b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99eb59e640 1 -- 172.21.15.26:0/674920787 >> 172.21.15.26:0/674920787 conn(0x7f99ec0cd2f0 msgr2=0x7f99ec0cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:40.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99eb59e640 1 -- 172.21.15.26:0/674920787 shutdown_connections 2023-12-19T18:14:40.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99eb59e640 1 -- 172.21.15.26:0/674920787 wait complete. 2023-12-19T18:14:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99eb59e640 1 Processor -- start 2023-12-19T18:14:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.724+0000 7f99eb59e640 1 -- start start 2023-12-19T18:14:40.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99eb59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99ec06b850 0x7f99ec0c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99eb59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec0b2830 0x7f99ec0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99eb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ec15ce50 0x7f99ec15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99eb59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec0c8df0 con 0x7f99ec15ce50 2023-12-19T18:14:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99eb59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec0c72c0 con 0x7f99ec0b2830 2023-12-19T18:14:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99eb59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ec0c6830 con 0x7f99ec06b850 2023-12-19T18:14:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99ead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ec15ce50 0x7f99ec15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99ead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ec15ce50 0x7f99ec15b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33936/0 (socket says 172.21.15.26:33936) 2023-12-19T18:14:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99ead9d640 1 -- 172.21.15.26:0/3114805694 learned_addr learned my addr 172.21.15.26:0/3114805694 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99ea59c640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99ec06b850 0x7f99ec0c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99e3fff640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec0b2830 0x7f99ec0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99ead9d640 1 -- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99ec06b850 msgr2=0x7f99ec0c62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99ead9d640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99ec06b850 0x7f99ec0c62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99ead9d640 1 -- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec0b2830 msgr2=0x7f99ec0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99ead9d640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec0b2830 0x7f99ec0c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99ead9d640 1 -- 172.21.15.26:0/3114805694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e4009a70 con 0x7f99ec15ce50 2023-12-19T18:14:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99ea59c640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99ec06b850 0x7f99ec0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99ead9d640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ec15ce50 0x7f99ec15b1a0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f99e40b0040 tx=0x7f99e4019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99e2ffd640 1 -- 172.21.15.26:0/3114805694 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e40b9860 con 0x7f99ec15ce50 2023-12-19T18:14:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99e2ffd640 1 -- 172.21.15.26:0/3114805694 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e40b99c0 con 0x7f99ec15ce50 2023-12-19T18:14:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99e2ffd640 1 -- 172.21.15.26:0/3114805694 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e40b9d70 con 0x7f99ec15ce50 2023-12-19T18:14:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99eb59e640 1 -- 172.21.15.26:0/3114805694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ec158530 con 0x7f99ec15ce50 2023-12-19T18:14:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99eb59e640 1 -- 172.21.15.26:0/3114805694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ec0c9040 con 0x7f99ec15ce50 2023-12-19T18:14:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99eb59e640 1 -- 172.21.15.26:0/3114805694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ec06be80 con 0x7f99ec15ce50 2023-12-19T18:14:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.728+0000 7f99e2ffd640 1 -- 172.21.15.26:0/3114805694 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f99e40ad440 con 0x7f99ec15ce50 2023-12-19T18:14:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.732+0000 7f99e2ffd640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f99cc065b80 0x7f99cc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.732+0000 7f99e2ffd640 1 -- 172.21.15.26:0/3114805694 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f99e411e920 con 0x7f99ec15ce50 2023-12-19T18:14:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.732+0000 7f99ea59c640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f99cc065b80 0x7f99cc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.732+0000 7f99ea59c640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f99cc065b80 0x7f99cc068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00b9f0 tx=0x7f99dc008040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.736+0000 7f99e2ffd640 1 -- 172.21.15.26:0/3114805694 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f99e40ebfa0 con 0x7f99ec15ce50 2023-12-19T18:14:40.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:40.924+0000 7f99eb59e640 1 -- 172.21.15.26:0/3114805694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f99ec0b2e60 con 0x7f99ec15ce50 2023-12-19T18:14:41.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.540+0000 7f99e2ffd640 1 -- 172.21.15.26:0/3114805694 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v87) v1 ==== 122+0+0 (secure 0 0 0) 0x7f99ec0b2e60 con 0x7f99ec15ce50 2023-12-19T18:14:41.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.544+0000 7f99eb59e640 1 -- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f99cc065b80 msgr2=0x7f99cc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:41.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.544+0000 7f99eb59e640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f99cc065b80 0x7f99cc068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f99dc00b9f0 tx=0x7f99dc008040 comp rx=0 tx=0).stop 2023-12-19T18:14:41.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.544+0000 7f99eb59e640 1 -- 172.21.15.26:0/3114805694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ec15ce50 msgr2=0x7f99ec15b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:41.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.544+0000 7f99eb59e640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ec15ce50 0x7f99ec15b1a0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f99e40b0040 tx=0x7f99e4019660 comp rx=0 tx=0).stop 2023-12-19T18:14:41.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.544+0000 7f99eb59e640 1 -- 172.21.15.26:0/3114805694 shutdown_connections 2023-12-19T18:14:41.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.544+0000 7f99eb59e640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f99cc065b80 0x7f99cc068040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.544+0000 7f99eb59e640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99ec15ce50 0x7f99ec15b1a0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.544+0000 7f99eb59e640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99ec0b2830 0x7f99ec0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.544+0000 7f99eb59e640 1 --2- 172.21.15.26:0/3114805694 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99ec06b850 0x7f99ec0c62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.544+0000 7f99eb59e640 1 -- 172.21.15.26:0/3114805694 >> 172.21.15.26:0/3114805694 conn(0x7f99ec0cd2f0 msgr2=0x7f99ec15fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:41.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.544+0000 7f99eb59e640 1 -- 172.21.15.26:0/3114805694 shutdown_connections 2023-12-19T18:14:41.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.544+0000 7f99eb59e640 1 -- 172.21.15.26:0/3114805694 wait complete. 2023-12-19T18:14:41.566 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:14:41.567 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:14:41.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f9946518640 1 Processor -- start 2023-12-19T18:14:41.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f9946518640 1 -- start start 2023-12-19T18:14:41.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f9946518640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f994015cf60 0x7f99400d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f9946518640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f99400d1420 0x7f99400b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f9946518640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99400d2960 con 0x7f99400b88e0 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f9946518640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99400d18c0 con 0x7f99400d1420 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f9946518640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99400d1ba0 con 0x7f994015cf60 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f993f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f99400d1420 0x7f99400b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f9944ab5640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f99400b88e0 0x7f9940069d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39910/0 (socket says 172.21.15.26:39910) 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f9944ab5640 1 -- 172.21.15.26:0/1257311120 learned_addr learned my addr 172.21.15.26:0/1257311120 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f993ffff640 1 --2- 172.21.15.26:0/1257311120 >> v2:172.21.15.160:3300/0 conn(0x7f994015cf60 0x7f99400d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f993effd640 1 -- 172.21.15.26:0/1257311120 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680913400 0 0) 0x7f99400d2960 con 0x7f99400b88e0 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f993effd640 1 -- 172.21.15.26:0/1257311120 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9928003530 con 0x7f99400b88e0 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f993f7fe640 1 -- 172.21.15.26:0/1257311120 >> v2:172.21.15.160:3300/0 conn(0x7f994015cf60 msgr2=0x7f99400d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f993f7fe640 1 --2- 172.21.15.26:0/1257311120 >> v2:172.21.15.160:3300/0 conn(0x7f994015cf60 0x7f99400d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f993f7fe640 1 -- 172.21.15.26:0/1257311120 >> v1:172.21.15.26:6789/0 conn(0x7f99400b88e0 legacy=0x7f9940069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f993f7fe640 1 -- 172.21.15.26:0/1257311120 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99400d1e80 con 0x7f99400d1420 2023-12-19T18:14:41.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.712+0000 7f993ffff640 1 --2- 172.21.15.26:0/1257311120 >> v2:172.21.15.160:3300/0 conn(0x7f994015cf60 0x7f99400d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:41.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.716+0000 7f993f7fe640 1 --2- 172.21.15.26:0/1257311120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400d1420 0x7f99400b4600 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f994006ae20 tx=0x7f993401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2961bfb0fd9b2b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:41.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.716+0000 7f993effd640 1 -- 172.21.15.26:0/1257311120 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340a8950 con 0x7f99400d1420 2023-12-19T18:14:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.716+0000 7f993effd640 1 -- 172.21.15.26:0/1257311120 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99340a8ab0 con 0x7f99400d1420 2023-12-19T18:14:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.716+0000 7f993effd640 1 -- 172.21.15.26:0/1257311120 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340a8e40 con 0x7f99400d1420 2023-12-19T18:14:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.716+0000 7f9946518640 1 -- 172.21.15.26:0/1257311120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400d1420 msgr2=0x7f99400b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.716+0000 7f9946518640 1 --2- 172.21.15.26:0/1257311120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400d1420 0x7f99400b4600 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f994006ae20 tx=0x7f993401ec50 comp rx=0 tx=0).stop 2023-12-19T18:14:41.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.716+0000 7f9946518640 1 -- 172.21.15.26:0/1257311120 shutdown_connections 2023-12-19T18:14:41.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.716+0000 7f9946518640 1 --2- 172.21.15.26:0/1257311120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400d1420 0x7f99400b4600 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.716+0000 7f9946518640 1 --2- 172.21.15.26:0/1257311120 >> v2:172.21.15.160:3300/0 conn(0x7f994015cf60 0x7f99400d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.716+0000 7f9946518640 1 -- 172.21.15.26:0/1257311120 >> 172.21.15.26:0/1257311120 conn(0x7f9940065ee0 msgr2=0x7f99400cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:41.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.716+0000 7f9946518640 1 -- 172.21.15.26:0/1257311120 shutdown_connections 2023-12-19T18:14:41.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.716+0000 7f9946518640 1 -- 172.21.15.26:0/1257311120 wait complete. 2023-12-19T18:14:41.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f9946518640 1 Processor -- start 2023-12-19T18:14:41.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f9946518640 1 -- start start 2023-12-19T18:14:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f9946518640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99400b88e0 0x7f994015ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f9946518640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400d1420 0x7f99400cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f9946518640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f994015cf60 0x7f9940159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f9946518640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994015df60 con 0x7f99400d1420 2023-12-19T18:14:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f9946518640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99400ca140 con 0x7f99400b88e0 2023-12-19T18:14:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f9946518640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99400c9c50 con 0x7f994015cf60 2023-12-19T18:14:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400d1420 0x7f99400cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400d1420 0x7f99400cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33956/0 (socket says 172.21.15.26:33956) 2023-12-19T18:14:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993f7fe640 1 -- 172.21.15.26:0/3040380383 learned_addr learned my addr 172.21.15.26:0/3040380383 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993f7fe640 1 -- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f994015cf60 msgr2=0x7f9940159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:41.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993f7fe640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f994015cf60 0x7f9940159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993f7fe640 1 -- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99400b88e0 msgr2=0x7f994015ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993ffff640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99400b88e0 0x7f994015ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993f7fe640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99400b88e0 0x7f994015ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993f7fe640 1 -- 172.21.15.26:0/3040380383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9934009420 con 0x7f99400d1420 2023-12-19T18:14:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993ffff640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99400b88e0 0x7f994015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:41.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993f7fe640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400d1420 0x7f99400cc660 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f99340b3040 tx=0x7f993401b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993cff9640 1 -- 172.21.15.26:0/3040380383 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340b0dd0 con 0x7f99400d1420 2023-12-19T18:14:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993cff9640 1 -- 172.21.15.26:0/3040380383 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99340b9430 con 0x7f99400d1420 2023-12-19T18:14:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f993cff9640 1 -- 172.21.15.26:0/3040380383 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340b97e0 con 0x7f99400d1420 2023-12-19T18:14:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f9946518640 1 -- 172.21.15.26:0/3040380383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99400c9ec0 con 0x7f99400d1420 2023-12-19T18:14:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.720+0000 7f9946518640 1 -- 172.21.15.26:0/3040380383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994015e420 con 0x7f99400d1420 2023-12-19T18:14:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.724+0000 7f9946518640 1 -- 172.21.15.26:0/3040380383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9940074270 con 0x7f99400d1420 2023-12-19T18:14:41.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.724+0000 7f993cff9640 1 -- 172.21.15.26:0/3040380383 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f99340b7040 con 0x7f99400d1420 2023-12-19T18:14:41.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.724+0000 7f993cff9640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f992c065b80 0x7f992c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:41.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.724+0000 7f993cff9640 1 -- 172.21.15.26:0/3040380383 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f9934121210 con 0x7f99400d1420 2023-12-19T18:14:41.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.724+0000 7f993ffff640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f992c065b80 0x7f992c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:41.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.724+0000 7f993ffff640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f992c065b80 0x7f992c068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99400b15c0 tx=0x7f9930008040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:41.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.728+0000 7f993cff9640 1 -- 172.21.15.26:0/3040380383 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f99340ee740 con 0x7f99400d1420 2023-12-19T18:14:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.916+0000 7f9946518640 1 -- 172.21.15.26:0/3040380383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f994015c690 con 0x7f99400d1420 2023-12-19T18:14:41.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.916+0000 7f993cff9640 1 -- 172.21.15.26:0/3040380383 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4723 (secure 0 0 0) 0x7f99340f1d60 con 0x7f99400d1420 2023-12-19T18:14:41.918 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:41.919 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3922467901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3922467901},{"type":"v1","addr":"172.21.15.160:6835","nonce":3922467901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":9562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3692512090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3692512090},{"type":"v1","addr":"172.21.15.160:6837","nonce":3692512090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":10083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3961033710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3961033710},{"type":"v1","addr":"172.21.15.26:6833","nonce":3961033710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:38.527840+0000","modified":"2023-12-19T18:14:41.539799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10086":{"gid":10086,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1537720341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1537720341},{"type":"v1","addr":"172.21.15.26:6835","nonce":1537720341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:14:41.919 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 87 2023-12-19T18:14:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.920+0000 7f9946518640 1 -- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f992c065b80 msgr2=0x7f992c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.920+0000 7f9946518640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f992c065b80 0x7f992c068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99400b15c0 tx=0x7f9930008040 comp rx=0 tx=0).stop 2023-12-19T18:14:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.920+0000 7f9946518640 1 -- 172.21.15.26:0/3040380383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400d1420 msgr2=0x7f99400cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.920+0000 7f9946518640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400d1420 0x7f99400cc660 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f99340b3040 tx=0x7f993401b9f0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.920+0000 7f9946518640 1 -- 172.21.15.26:0/3040380383 shutdown_connections 2023-12-19T18:14:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.920+0000 7f9946518640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f992c065b80 0x7f992c068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.920+0000 7f9946518640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f994015cf60 0x7f9940159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.920+0000 7f9946518640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400d1420 0x7f99400cc660 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.920+0000 7f9946518640 1 --2- 172.21.15.26:0/3040380383 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f99400b88e0 0x7f994015ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.920+0000 7f9946518640 1 -- 172.21.15.26:0/3040380383 >> 172.21.15.26:0/3040380383 conn(0x7f9940065ee0 msgr2=0x7f99400b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.920+0000 7f9946518640 1 -- 172.21.15.26:0/3040380383 shutdown_connections 2023-12-19T18:14:41.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:41.920+0000 7f9946518640 1 -- 172.21.15.26:0/3040380383 wait complete. 2023-12-19T18:14:41.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:41.965 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:14:41.990 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:14:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.080+0000 7f21c2ff0640 1 Processor -- start 2023-12-19T18:14:42.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.080+0000 7f21c2ff0640 1 -- start start 2023-12-19T18:14:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21c2ff0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f21bc154220 0x7f21bc15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21c2ff0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21bc06b3d0 0x7f21bc0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21c2ff0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21bc06bc30 con 0x7f21bc0cec40 2023-12-19T18:14:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21c2ff0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21bc0cf0d0 con 0x7f21bc06b3d0 2023-12-19T18:14:42.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21c2ff0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21bc0cc6e0 con 0x7f21bc154220 2023-12-19T18:14:42.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21bbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21bc06b3d0 0x7f21bc0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21c158d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f21bc0cec40 0x7f21bc15d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39924/0 (socket says 172.21.15.26:39924) 2023-12-19T18:14:42.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21c158d640 1 -- 172.21.15.26:0/2855406688 learned_addr learned my addr 172.21.15.26:0/2855406688 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:42.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21c0d8c640 1 --2- 172.21.15.26:0/2855406688 >> v2:172.21.15.160:3300/0 conn(0x7f21bc154220 0x7f21bc15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21bb7fe640 1 -- 172.21.15.26:0/2855406688 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393079462 0 0) 0x7f21bc06bc30 con 0x7f21bc0cec40 2023-12-19T18:14:42.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21bb7fe640 1 -- 172.21.15.26:0/2855406688 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21ac003530 con 0x7f21bc0cec40 2023-12-19T18:14:42.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21bbfff640 1 -- 172.21.15.26:0/2855406688 >> v2:172.21.15.160:3300/0 conn(0x7f21bc154220 msgr2=0x7f21bc15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:42.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21bbfff640 1 --2- 172.21.15.26:0/2855406688 >> v2:172.21.15.160:3300/0 conn(0x7f21bc154220 0x7f21bc15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21bbfff640 1 -- 172.21.15.26:0/2855406688 >> v1:172.21.15.26:6789/0 conn(0x7f21bc0cec40 legacy=0x7f21bc15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.084+0000 7f21bbfff640 1 -- 172.21.15.26:0/2855406688 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc15afc0 con 0x7f21bc06b3d0 2023-12-19T18:14:42.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21bbfff640 1 --2- 172.21.15.26:0/2855406688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21bc06b3d0 0x7f21bc0ce700 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0667b0 tx=0x7f21a801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116ac5c0ac0efea9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:42.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21bb7fe640 1 -- 172.21.15.26:0/2855406688 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8004230 con 0x7f21bc06b3d0 2023-12-19T18:14:42.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21bb7fe640 1 -- 172.21.15.26:0/2855406688 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a80a7050 con 0x7f21bc06b3d0 2023-12-19T18:14:42.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21bb7fe640 1 -- 172.21.15.26:0/2855406688 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8003620 con 0x7f21bc06b3d0 2023-12-19T18:14:42.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21c2ff0640 1 -- 172.21.15.26:0/2855406688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21bc06b3d0 msgr2=0x7f21bc0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21c2ff0640 1 --2- 172.21.15.26:0/2855406688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21bc06b3d0 0x7f21bc0ce700 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0667b0 tx=0x7f21a801c430 comp rx=0 tx=0).stop 2023-12-19T18:14:42.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21c2ff0640 1 -- 172.21.15.26:0/2855406688 shutdown_connections 2023-12-19T18:14:42.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21c2ff0640 1 --2- 172.21.15.26:0/2855406688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21bc06b3d0 0x7f21bc0ce700 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21c2ff0640 1 --2- 172.21.15.26:0/2855406688 >> v2:172.21.15.160:3300/0 conn(0x7f21bc154220 0x7f21bc15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21c2ff0640 1 -- 172.21.15.26:0/2855406688 >> 172.21.15.26:0/2855406688 conn(0x7f21bc0658e0 msgr2=0x7f21bc065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:42.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21c2ff0640 1 -- 172.21.15.26:0/2855406688 shutdown_connections 2023-12-19T18:14:42.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21c2ff0640 1 -- 172.21.15.26:0/2855406688 wait complete. 2023-12-19T18:14:42.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21c2ff0640 1 Processor -- start 2023-12-19T18:14:42.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.088+0000 7f21c2ff0640 1 -- start start 2023-12-19T18:14:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c2ff0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21bc06b3d0 0x7f21bc14f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c2ff0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f21bc0cec40 0x7f21bc151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c2ff0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21bc154220 0x7f21bc15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c2ff0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc1f7050 con 0x7f21bc154220 2023-12-19T18:14:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c2ff0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc153830 con 0x7f21bc06b3d0 2023-12-19T18:14:42.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c2ff0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc1f6740 con 0x7f21bc0cec40 2023-12-19T18:14:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c158d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21bc154220 0x7f21bc15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c158d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21bc154220 0x7f21bc15b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33986/0 (socket says 172.21.15.26:33986) 2023-12-19T18:14:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c158d640 1 -- 172.21.15.26:0/1944800298 learned_addr learned my addr 172.21.15.26:0/1944800298 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c0d8c640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21bc06b3d0 0x7f21bc14f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21bbfff640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f21bc0cec40 0x7f21bc151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c158d640 1 -- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f21bc0cec40 msgr2=0x7f21bc151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c158d640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f21bc0cec40 0x7f21bc151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c158d640 1 -- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21bc06b3d0 msgr2=0x7f21bc14f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c158d640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21bc06b3d0 0x7f21bc14f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c158d640 1 -- 172.21.15.26:0/1944800298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a8009420 con 0x7f21bc154220 2023-12-19T18:14:42.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c158d640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21bc154220 0x7f21bc15b180 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f21b400caa0 tx=0x7f21b400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21b97fa640 1 -- 172.21.15.26:0/1944800298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b400ac00 con 0x7f21bc154220 2023-12-19T18:14:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21b97fa640 1 -- 172.21.15.26:0/1944800298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b400ad60 con 0x7f21bc154220 2023-12-19T18:14:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c2ff0640 1 -- 172.21.15.26:0/1944800298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc1523c0 con 0x7f21bc154220 2023-12-19T18:14:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21b97fa640 1 -- 172.21.15.26:0/1944800298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b4012620 con 0x7f21bc154220 2023-12-19T18:14:42.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c2ff0640 1 -- 172.21.15.26:0/1944800298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc1f6990 con 0x7f21bc154220 2023-12-19T18:14:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.092+0000 7f21c2ff0640 1 -- 172.21.15.26:0/1944800298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc074270 con 0x7f21bc154220 2023-12-19T18:14:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.096+0000 7f21b97fa640 1 -- 172.21.15.26:0/1944800298 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f21b4007070 con 0x7f21bc154220 2023-12-19T18:14:42.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.096+0000 7f21b97fa640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f21a0065b80 0x7f21a0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.096+0000 7f21b97fa640 1 -- 172.21.15.26:0/1944800298 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f21b4083520 con 0x7f21bc154220 2023-12-19T18:14:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.096+0000 7f21c0d8c640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f21a0065b80 0x7f21a0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.096+0000 7f21c0d8c640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f21a0065b80 0x7f21a0068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0ad200 tx=0x7f21b0009190 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:42.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.100+0000 7f21b97fa640 1 -- 172.21.15.26:0/1944800298 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f21b4050b90 con 0x7f21bc154220 2023-12-19T18:14:42.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.256+0000 7f21c2ff0640 1 -- 172.21.15.26:0/1944800298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21bc06c660 con 0x7f21bc154220 2023-12-19T18:14:42.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.260+0000 7f21b97fa640 1 -- 172.21.15.26:0/1944800298 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+17208 (secure 0 0 0) 0x7f21b40541b0 con 0x7f21bc154220 2023-12-19T18:14:42.260 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:42.260 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":196,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:14:39.530261+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:36.168666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:37.527319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.260+0000 7f21c2ff0640 1 -- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f21a0065b80 msgr2=0x7f21a0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.260+0000 7f21c2ff0640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f21a0065b80 0x7f21a0068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0ad200 tx=0x7f21b0009190 comp rx=0 tx=0).stop 2023-12-19T18:14:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.260+0000 7f21c2ff0640 1 -- 172.21.15.26:0/1944800298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21bc154220 msgr2=0x7f21bc15b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.260+0000 7f21c2ff0640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21bc154220 0x7f21bc15b180 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f21b400caa0 tx=0x7f21b400cf60 comp rx=0 tx=0).stop 2023-12-19T18:14:42.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.260+0000 7f21c2ff0640 1 -- 172.21.15.26:0/1944800298 shutdown_connections 2023-12-19T18:14:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.260+0000 7f21c2ff0640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f21a0065b80 0x7f21a0068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.260+0000 7f21c2ff0640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21bc154220 0x7f21bc15b180 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.260+0000 7f21c2ff0640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f21bc0cec40 0x7f21bc151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.260+0000 7f21c2ff0640 1 --2- 172.21.15.26:0/1944800298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f21bc06b3d0 0x7f21bc14f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.260+0000 7f21c2ff0640 1 -- 172.21.15.26:0/1944800298 >> 172.21.15.26:0/1944800298 conn(0x7f21bc0658e0 msgr2=0x7f21bc065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.260+0000 7f21c2ff0640 1 -- 172.21.15.26:0/1944800298 shutdown_connections 2023-12-19T18:14:42.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.260+0000 7f21c2ff0640 1 -- 172.21.15.26:0/1944800298 wait complete. 2023-12-19T18:14:42.282 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:14:42.283 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:14:42.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f2531d98640 1 Processor -- start 2023-12-19T18:14:42.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f2531d98640 1 -- start start 2023-12-19T18:14:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f2531d98640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f252c154220 0x7f252c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f2531d98640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f252c0cee10 0x7f252c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f2531d98640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252c06bc30 con 0x7f252c06b3d0 2023-12-19T18:14:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f2531d98640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252c0cc6e0 con 0x7f252c0cee10 2023-12-19T18:14:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f2531d98640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252c15afc0 con 0x7f252c154220 2023-12-19T18:14:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f252bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f252c0cee10 0x7f252c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f252bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f252c0cee10 0x7f252c15b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33992/0 (socket says 172.21.15.26:33992) 2023-12-19T18:14:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f252bfff640 1 -- 172.21.15.26:0/27979803 learned_addr learned my addr 172.21.15.26:0/27979803 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f252b7fe640 1 --2- 172.21.15.26:0/27979803 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f252c154220 0x7f252c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f252bfff640 1 -- 172.21.15.26:0/27979803 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f252c154220 msgr2=0x7f252c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f252bfff640 1 --2- 172.21.15.26:0/27979803 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f252c154220 0x7f252c15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f252bfff640 1 -- 172.21.15.26:0/27979803 >> v1:172.21.15.160:6789/0 conn(0x7f252c06b3d0 legacy=0x7f252c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.468+0000 7f252bfff640 1 -- 172.21.15.26:0/27979803 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c06b950 con 0x7f252c0cee10 2023-12-19T18:14:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.472+0000 7f252bfff640 1 --2- 172.21.15.26:0/27979803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f252c0cee10 0x7f252c15b1c0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f2524009e50 tx=0x7f252401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac70f3b1a594a9a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.472+0000 7f252a7fc640 1 -- 172.21.15.26:0/27979803 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25240a58a0 con 0x7f252c0cee10 2023-12-19T18:14:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.472+0000 7f252a7fc640 1 -- 172.21.15.26:0/27979803 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25240a5a00 con 0x7f252c0cee10 2023-12-19T18:14:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.472+0000 7f252a7fc640 1 -- 172.21.15.26:0/27979803 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25240a5d90 con 0x7f252c0cee10 2023-12-19T18:14:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.472+0000 7f2531d98640 1 -- 172.21.15.26:0/27979803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f252c0cee10 msgr2=0x7f252c15b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.472+0000 7f2531d98640 1 --2- 172.21.15.26:0/27979803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f252c0cee10 0x7f252c15b1c0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f2524009e50 tx=0x7f252401c9e0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.472+0000 7f2531d98640 1 -- 172.21.15.26:0/27979803 shutdown_connections 2023-12-19T18:14:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.472+0000 7f2531d98640 1 --2- 172.21.15.26:0/27979803 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f252c0cee10 0x7f252c15b1c0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.472+0000 7f2531d98640 1 --2- 172.21.15.26:0/27979803 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f252c154220 0x7f252c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.472+0000 7f2531d98640 1 -- 172.21.15.26:0/27979803 >> 172.21.15.26:0/27979803 conn(0x7f252c0658e0 msgr2=0x7f252c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.472+0000 7f2531d98640 1 -- 172.21.15.26:0/27979803 shutdown_connections 2023-12-19T18:14:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.472+0000 7f2531d98640 1 -- 172.21.15.26:0/27979803 wait complete. 2023-12-19T18:14:42.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f2531d98640 1 Processor -- start 2023-12-19T18:14:42.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f2531d98640 1 -- start start 2023-12-19T18:14:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f2531d98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f252c06b3d0 0x7f252c14e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f2531d98640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f252c0cee10 0x7f252c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f2531d98640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f252c154220 0x7f252c1580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f2531d98640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c159340 con 0x7f252c06b3d0 2023-12-19T18:14:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f2531d98640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c152240 con 0x7f252c154220 2023-12-19T18:14:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f2531d98640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252c14ed20 con 0x7f252c0cee10 2023-12-19T18:14:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f252b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f252c06b3d0 0x7f252c14e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f252b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f252c06b3d0 0x7f252c14e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34006/0 (socket says 172.21.15.26:34006) 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f252b7fe640 1 -- 172.21.15.26:0/3279500933 learned_addr learned my addr 172.21.15.26:0/3279500933 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f252affd640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f252c0cee10 0x7f252c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f252bfff640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f252c154220 0x7f252c1580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f252b7fe640 1 -- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f252c0cee10 msgr2=0x7f252c158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f252b7fe640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f252c0cee10 0x7f252c158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f252b7fe640 1 -- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f252c154220 msgr2=0x7f252c1580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f252b7fe640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f252c154220 0x7f252c1580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f252b7fe640 1 -- 172.21.15.26:0/3279500933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524009b30 con 0x7f252c06b3d0 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f252b7fe640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f252c06b3d0 0x7f252c14e710 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f251c00e930 tx=0x7f251c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f2513fff640 1 -- 172.21.15.26:0/3279500933 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c009690 con 0x7f252c06b3d0 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f2513fff640 1 -- 172.21.15.26:0/3279500933 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c00ca00 con 0x7f252c06b3d0 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f2513fff640 1 -- 172.21.15.26:0/3279500933 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c00cd70 con 0x7f252c06b3d0 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f2531d98640 1 -- 172.21.15.26:0/3279500933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252c14f000 con 0x7f252c06b3d0 2023-12-19T18:14:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.476+0000 7f2531d98640 1 -- 172.21.15.26:0/3279500933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252c14f2e0 con 0x7f252c06b3d0 2023-12-19T18:14:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.480+0000 7f2531d98640 1 -- 172.21.15.26:0/3279500933 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252c151d30 con 0x7f252c06b3d0 2023-12-19T18:14:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.480+0000 7f2513fff640 1 -- 172.21.15.26:0/3279500933 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f251c013470 con 0x7f252c06b3d0 2023-12-19T18:14:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.480+0000 7f2513fff640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2518065b80 0x7f2518068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.480+0000 7f2513fff640 1 -- 172.21.15.26:0/3279500933 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f251c083820 con 0x7f252c06b3d0 2023-12-19T18:14:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.480+0000 7f252affd640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2518065b80 0x7f2518068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.484+0000 7f252affd640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2518065b80 0x7f2518068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f252c14ec50 tx=0x7f2520009040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.484+0000 7f2513fff640 1 -- 172.21.15.26:0/3279500933 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f252c151d30 con 0x7f252c06b3d0 2023-12-19T18:14:42.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.640+0000 7f2531d98640 1 -- 172.21.15.26:0/3279500933 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f252c0ccef0 con 0x7f2518065b80 2023-12-19T18:14:42.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.652+0000 7f2513fff640 1 -- 172.21.15.26:0/3279500933 <== mgr.8276 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289459 (secure 0 0 0) 0x7f2520001450 con 0x7f2518065b80 2023-12-19T18:14:42.654 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:42.656 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:14:42.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.652+0000 7f2531d98640 1 -- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2518065b80 msgr2=0x7f2518068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.652+0000 7f2531d98640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2518065b80 0x7f2518068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f252c14ec50 tx=0x7f2520009040 comp rx=0 tx=0).stop 2023-12-19T18:14:42.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.652+0000 7f2531d98640 1 -- 172.21.15.26:0/3279500933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f252c06b3d0 msgr2=0x7f252c14e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.652+0000 7f2531d98640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f252c06b3d0 0x7f252c14e710 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f251c00e930 tx=0x7f251c00edf0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.652+0000 7f2531d98640 1 -- 172.21.15.26:0/3279500933 shutdown_connections 2023-12-19T18:14:42.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.652+0000 7f2531d98640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2518065b80 0x7f2518068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.652+0000 7f2531d98640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f252c154220 0x7f252c1580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.652+0000 7f2531d98640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f252c0cee10 0x7f252c158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.652+0000 7f2531d98640 1 --2- 172.21.15.26:0/3279500933 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f252c06b3d0 0x7f252c14e710 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.652+0000 7f2531d98640 1 -- 172.21.15.26:0/3279500933 >> 172.21.15.26:0/3279500933 conn(0x7f252c0658e0 msgr2=0x7f252c0af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:42.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.656+0000 7f2531d98640 1 -- 172.21.15.26:0/3279500933 shutdown_connections 2023-12-19T18:14:42.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.656+0000 7f2531d98640 1 -- 172.21.15.26:0/3279500933 wait complete. 2023-12-19T18:14:42.674 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-19T18:14:42.001534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1922,"num_read_kb":3999,"num_write":579,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":362,"ondisk_log_size":362,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":431808,"kb_used_data":20224,"kb_used_omap":0,"kb_used_meta":411520,"kb_avail":754542912,"statfs":{"total":773094113280,"available":772651941888,"internally_reserved":0,"allocated":20709376,"data_stored":11431892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":421396480},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.885791"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.563789+0000","last_change":"2023-12-19T18:14:38.552280+0000","last_active":"2023-12-19T18:14:38.563789+0000","last_peered":"2023-12-19T18:14:38.563789+0000","last_clean":"2023-12-19T18:14:38.563789+0000","last_became_active":"2023-12-19T18:14:38.552022+0000","last_became_peered":"2023-12-19T18:14:38.552022+0000","last_unstale":"2023-12-19T18:14:38.563789+0000","last_undegraded":"2023-12-19T18:14:38.563789+0000","last_fullsized":"2023-12-19T18:14:38.563789+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:13:42.870515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536987+0000","last_change":"2023-12-19T18:14:38.557909+0000","last_active":"2023-12-19T18:14:39.536987+0000","last_peered":"2023-12-19T18:14:39.536987+0000","last_clean":"2023-12-19T18:14:39.536987+0000","last_became_active":"2023-12-19T18:14:38.557650+0000","last_became_peered":"2023-12-19T18:14:38.557650+0000","last_unstale":"2023-12-19T18:14:39.536987+0000","last_undegraded":"2023-12-19T18:14:39.536987+0000","last_fullsized":"2023-12-19T18:14:39.536987+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:32:04.959408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537641+0000","last_change":"2023-12-19T18:14:38.543248+0000","last_active":"2023-12-19T18:14:39.537641+0000","last_peered":"2023-12-19T18:14:39.537641+0000","last_clean":"2023-12-19T18:14:39.537641+0000","last_became_active":"2023-12-19T18:14:38.542894+0000","last_became_peered":"2023-12-19T18:14:38.542894+0000","last_unstale":"2023-12-19T18:14:39.537641+0000","last_undegraded":"2023-12-19T18:14:39.537641+0000","last_fullsized":"2023-12-19T18:14:39.537641+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:06.859307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.548920+0000","last_change":"2023-12-19T18:14:38.547197+0000","last_active":"2023-12-19T18:14:38.548920+0000","last_peered":"2023-12-19T18:14:38.548920+0000","last_clean":"2023-12-19T18:14:38.548920+0000","last_became_active":"2023-12-19T18:14:38.546903+0000","last_became_peered":"2023-12-19T18:14:38.546903+0000","last_unstale":"2023-12-19T18:14:38.548920+0000","last_undegraded":"2023-12-19T18:14:38.548920+0000","last_fullsized":"2023-12-19T18:14:38.548920+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:03.792973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536484+0000","last_change":"2023-12-19T18:14:38.551345+0000","last_active":"2023-12-19T18:14:39.536484+0000","last_peered":"2023-12-19T18:14:39.536484+0000","last_clean":"2023-12-19T18:14:39.536484+0000","last_became_active":"2023-12-19T18:14:38.551084+0000","last_became_peered":"2023-12-19T18:14:38.551084+0000","last_unstale":"2023-12-19T18:14:39.536484+0000","last_undegraded":"2023-12-19T18:14:39.536484+0000","last_fullsized":"2023-12-19T18:14:39.536484+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:53.439000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660018+0000","last_change":"2023-12-19T18:14:38.567542+0000","last_active":"2023-12-19T18:14:39.660018+0000","last_peered":"2023-12-19T18:14:39.660018+0000","last_clean":"2023-12-19T18:14:39.660018+0000","last_became_active":"2023-12-19T18:14:38.567279+0000","last_became_peered":"2023-12-19T18:14:38.567279+0000","last_unstale":"2023-12-19T18:14:39.660018+0000","last_undegraded":"2023-12-19T18:14:39.660018+0000","last_fullsized":"2023-12-19T18:14:39.660018+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:04.274917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535127+0000","last_change":"2023-12-19T18:14:38.566538+0000","last_active":"2023-12-19T18:14:39.535127+0000","last_peered":"2023-12-19T18:14:39.535127+0000","last_clean":"2023-12-19T18:14:39.535127+0000","last_became_active":"2023-12-19T18:14:38.566286+0000","last_became_peered":"2023-12-19T18:14:38.566286+0000","last_unstale":"2023-12-19T18:14:39.535127+0000","last_undegraded":"2023-12-19T18:14:39.535127+0000","last_fullsized":"2023-12-19T18:14:39.535127+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:20:08.417607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536785+0000","last_change":"2023-12-19T18:14:38.549470+0000","last_active":"2023-12-19T18:14:39.536785+0000","last_peered":"2023-12-19T18:14:39.536785+0000","last_clean":"2023-12-19T18:14:39.536785+0000","last_became_active":"2023-12-19T18:14:38.549187+0000","last_became_peered":"2023-12-19T18:14:38.549187+0000","last_unstale":"2023-12-19T18:14:39.536785+0000","last_undegraded":"2023-12-19T18:14:39.536785+0000","last_fullsized":"2023-12-19T18:14:39.536785+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:04:33.703685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537162+0000","last_change":"2023-12-19T18:14:38.556753+0000","last_active":"2023-12-19T18:14:39.537162+0000","last_peered":"2023-12-19T18:14:39.537162+0000","last_clean":"2023-12-19T18:14:39.537162+0000","last_became_active":"2023-12-19T18:14:38.554373+0000","last_became_peered":"2023-12-19T18:14:38.554373+0000","last_unstale":"2023-12-19T18:14:39.537162+0000","last_undegraded":"2023-12-19T18:14:39.537162+0000","last_fullsized":"2023-12-19T18:14:39.537162+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:45.976373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537033+0000","last_change":"2023-12-19T18:14:38.560035+0000","last_active":"2023-12-19T18:14:39.537033+0000","last_peered":"2023-12-19T18:14:39.537033+0000","last_clean":"2023-12-19T18:14:39.537033+0000","last_became_active":"2023-12-19T18:14:38.559779+0000","last_became_peered":"2023-12-19T18:14:38.559779+0000","last_unstale":"2023-12-19T18:14:39.537033+0000","last_undegraded":"2023-12-19T18:14:39.537033+0000","last_fullsized":"2023-12-19T18:14:39.537033+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:59:30.883370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536693+0000","last_change":"2023-12-19T18:14:38.553655+0000","last_active":"2023-12-19T18:14:39.536693+0000","last_peered":"2023-12-19T18:14:39.536693+0000","last_clean":"2023-12-19T18:14:39.536693+0000","last_became_active":"2023-12-19T18:14:38.553368+0000","last_became_peered":"2023-12-19T18:14:38.553368+0000","last_unstale":"2023-12-19T18:14:39.536693+0000","last_undegraded":"2023-12-19T18:14:39.536693+0000","last_fullsized":"2023-12-19T18:14:39.536693+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:57:45.606591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536572+0000","last_change":"2023-12-19T18:14:38.553042+0000","last_active":"2023-12-19T18:14:39.536572+0000","last_peered":"2023-12-19T18:14:39.536572+0000","last_clean":"2023-12-19T18:14:39.536572+0000","last_became_active":"2023-12-19T18:14:38.552731+0000","last_became_peered":"2023-12-19T18:14:38.552731+0000","last_unstale":"2023-12-19T18:14:39.536572+0000","last_undegraded":"2023-12-19T18:14:39.536572+0000","last_fullsized":"2023-12-19T18:14:39.536572+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:07.574774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.538011+0000","last_change":"2023-12-19T18:14:38.557664+0000","last_active":"2023-12-19T18:14:39.538011+0000","last_peered":"2023-12-19T18:14:39.538011+0000","last_clean":"2023-12-19T18:14:39.538011+0000","last_became_active":"2023-12-19T18:14:38.556636+0000","last_became_peered":"2023-12-19T18:14:38.556636+0000","last_unstale":"2023-12-19T18:14:39.538011+0000","last_undegraded":"2023-12-19T18:14:39.538011+0000","last_fullsized":"2023-12-19T18:14:39.538011+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:21:32.182507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537469+0000","last_change":"2023-12-19T18:14:38.549523+0000","last_active":"2023-12-19T18:14:39.537469+0000","last_peered":"2023-12-19T18:14:39.537469+0000","last_clean":"2023-12-19T18:14:39.537469+0000","last_became_active":"2023-12-19T18:14:38.549216+0000","last_became_peered":"2023-12-19T18:14:38.549216+0000","last_unstale":"2023-12-19T18:14:39.537469+0000","last_undegraded":"2023-12-19T18:14:39.537469+0000","last_fullsized":"2023-12-19T18:14:39.537469+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:09:58.131056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.561224+0000","last_change":"2023-12-19T18:14:38.556426+0000","last_active":"2023-12-19T18:14:38.561224+0000","last_peered":"2023-12-19T18:14:38.561224+0000","last_clean":"2023-12-19T18:14:38.561224+0000","last_became_active":"2023-12-19T18:14:38.556173+0000","last_became_peered":"2023-12-19T18:14:38.556173+0000","last_unstale":"2023-12-19T18:14:38.561224+0000","last_undegraded":"2023-12-19T18:14:38.561224+0000","last_fullsized":"2023-12-19T18:14:38.561224+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:33:03.044215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.534937+0000","last_change":"2023-12-19T18:14:38.550324+0000","last_active":"2023-12-19T18:14:39.534937+0000","last_peered":"2023-12-19T18:14:39.534937+0000","last_clean":"2023-12-19T18:14:39.534937+0000","last_became_active":"2023-12-19T18:14:38.550070+0000","last_became_peered":"2023-12-19T18:14:38.550070+0000","last_unstale":"2023-12-19T18:14:39.534937+0000","last_undegraded":"2023-12-19T18:14:39.534937+0000","last_fullsized":"2023-12-19T18:14:39.534937+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:27:05.940012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660220+0000","last_change":"2023-12-19T18:14:38.557675+0000","last_active":"2023-12-19T18:14:39.660220+0000","last_peered":"2023-12-19T18:14:39.660220+0000","last_clean":"2023-12-19T18:14:39.660220+0000","last_became_active":"2023-12-19T18:14:38.557360+0000","last_became_peered":"2023-12-19T18:14:38.557360+0000","last_unstale":"2023-12-19T18:14:39.660220+0000","last_undegraded":"2023-12-19T18:14:39.660220+0000","last_fullsized":"2023-12-19T18:14:39.660220+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:11:27.767680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.560874+0000","last_change":"2023-12-19T18:14:38.556251+0000","last_active":"2023-12-19T18:14:38.560874+0000","last_peered":"2023-12-19T18:14:38.560874+0000","last_clean":"2023-12-19T18:14:38.560874+0000","last_became_active":"2023-12-19T18:14:38.555987+0000","last_became_peered":"2023-12-19T18:14:38.555987+0000","last_unstale":"2023-12-19T18:14:38.560874+0000","last_undegraded":"2023-12-19T18:14:38.560874+0000","last_fullsized":"2023-12-19T18:14:38.560874+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:10.230253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660208+0000","last_change":"2023-12-19T18:14:38.551632+0000","last_active":"2023-12-19T18:14:39.660208+0000","last_peered":"2023-12-19T18:14:39.660208+0000","last_clean":"2023-12-19T18:14:39.660208+0000","last_became_active":"2023-12-19T18:14:38.551239+0000","last_became_peered":"2023-12-19T18:14:38.551239+0000","last_unstale":"2023-12-19T18:14:39.660208+0000","last_undegraded":"2023-12-19T18:14:39.660208+0000","last_fullsized":"2023-12-19T18:14:39.660208+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:22.433655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537521+0000","last_change":"2023-12-19T18:14:38.545883+0000","last_active":"2023-12-19T18:14:39.537521+0000","last_peered":"2023-12-19T18:14:39.537521+0000","last_clean":"2023-12-19T18:14:39.537521+0000","last_became_active":"2023-12-19T18:14:38.544972+0000","last_became_peered":"2023-12-19T18:14:38.544972+0000","last_unstale":"2023-12-19T18:14:39.537521+0000","last_undegraded":"2023-12-19T18:14:39.537521+0000","last_fullsized":"2023-12-19T18:14:39.537521+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:14.639095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.659996+0000","last_change":"2023-12-19T18:14:38.553843+0000","last_active":"2023-12-19T18:14:39.659996+0000","last_peered":"2023-12-19T18:14:39.659996+0000","last_clean":"2023-12-19T18:14:39.659996+0000","last_became_active":"2023-12-19T18:14:38.553610+0000","last_became_peered":"2023-12-19T18:14:38.553610+0000","last_unstale":"2023-12-19T18:14:39.659996+0000","last_undegraded":"2023-12-19T18:14:39.659996+0000","last_fullsized":"2023-12-19T18:14:39.659996+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:31.488536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536586+0000","last_change":"2023-12-19T18:14:38.564751+0000","last_active":"2023-12-19T18:14:39.536586+0000","last_peered":"2023-12-19T18:14:39.536586+0000","last_clean":"2023-12-19T18:14:39.536586+0000","last_became_active":"2023-12-19T18:14:38.564381+0000","last_became_peered":"2023-12-19T18:14:38.564381+0000","last_unstale":"2023-12-19T18:14:39.536586+0000","last_undegraded":"2023-12-19T18:14:39.536586+0000","last_fullsized":"2023-12-19T18:14:39.536586+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:09.082505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.659911+0000","last_change":"2023-12-19T18:14:38.553142+0000","last_active":"2023-12-19T18:14:39.659911+0000","last_peered":"2023-12-19T18:14:39.659911+0000","last_clean":"2023-12-19T18:14:39.659911+0000","last_became_active":"2023-12-19T18:14:38.552850+0000","last_became_peered":"2023-12-19T18:14:38.552850+0000","last_unstale":"2023-12-19T18:14:39.659911+0000","last_undegraded":"2023-12-19T18:14:39.659911+0000","last_fullsized":"2023-12-19T18:14:39.659911+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:01:07.428157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536917+0000","last_change":"2023-12-19T18:14:38.553525+0000","last_active":"2023-12-19T18:14:39.536917+0000","last_peered":"2023-12-19T18:14:39.536917+0000","last_clean":"2023-12-19T18:14:39.536917+0000","last_became_active":"2023-12-19T18:14:38.553231+0000","last_became_peered":"2023-12-19T18:14:38.553231+0000","last_unstale":"2023-12-19T18:14:39.536917+0000","last_undegraded":"2023-12-19T18:14:39.536917+0000","last_fullsized":"2023-12-19T18:14:39.536917+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:50.277877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537959+0000","last_change":"2023-12-19T18:14:38.541116+0000","last_active":"2023-12-19T18:14:39.537959+0000","last_peered":"2023-12-19T18:14:39.537959+0000","last_clean":"2023-12-19T18:14:39.537959+0000","last_became_active":"2023-12-19T18:14:38.540825+0000","last_became_peered":"2023-12-19T18:14:38.540825+0000","last_unstale":"2023-12-19T18:14:39.537959+0000","last_undegraded":"2023-12-19T18:14:39.537959+0000","last_fullsized":"2023-12-19T18:14:39.537959+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:14:53.761212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535222+0000","last_change":"2023-12-19T18:14:38.567880+0000","last_active":"2023-12-19T18:14:39.535222+0000","last_peered":"2023-12-19T18:14:39.535222+0000","last_clean":"2023-12-19T18:14:39.535222+0000","last_became_active":"2023-12-19T18:14:38.567393+0000","last_became_peered":"2023-12-19T18:14:38.567393+0000","last_unstale":"2023-12-19T18:14:39.535222+0000","last_undegraded":"2023-12-19T18:14:39.535222+0000","last_fullsized":"2023-12-19T18:14:39.535222+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:41.971560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537750+0000","last_change":"2023-12-19T18:14:38.544254+0000","last_active":"2023-12-19T18:14:39.537750+0000","last_peered":"2023-12-19T18:14:39.537750+0000","last_clean":"2023-12-19T18:14:39.537750+0000","last_became_active":"2023-12-19T18:14:38.543980+0000","last_became_peered":"2023-12-19T18:14:38.543980+0000","last_unstale":"2023-12-19T18:14:39.537750+0000","last_undegraded":"2023-12-19T18:14:39.537750+0000","last_fullsized":"2023-12-19T18:14:39.537750+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:17:53.367376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.543078+0000","last_change":"2023-12-19T18:14:38.539808+0000","last_active":"2023-12-19T18:14:38.543078+0000","last_peered":"2023-12-19T18:14:38.543078+0000","last_clean":"2023-12-19T18:14:38.543078+0000","last_became_active":"2023-12-19T18:14:38.539528+0000","last_became_peered":"2023-12-19T18:14:38.539528+0000","last_unstale":"2023-12-19T18:14:38.543078+0000","last_undegraded":"2023-12-19T18:14:38.543078+0000","last_fullsized":"2023-12-19T18:14:38.543078+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:42:55.210047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.538041+0000","last_change":"2023-12-19T18:14:38.543688+0000","last_active":"2023-12-19T18:14:39.538041+0000","last_peered":"2023-12-19T18:14:39.538041+0000","last_clean":"2023-12-19T18:14:39.538041+0000","last_became_active":"2023-12-19T18:14:38.543404+0000","last_became_peered":"2023-12-19T18:14:38.543404+0000","last_unstale":"2023-12-19T18:14:39.538041+0000","last_undegraded":"2023-12-19T18:14:39.538041+0000","last_fullsized":"2023-12-19T18:14:39.538041+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:30:37.743915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.548538+0000","last_change":"2023-12-19T18:14:38.545547+0000","last_active":"2023-12-19T18:14:38.548538+0000","last_peered":"2023-12-19T18:14:38.548538+0000","last_clean":"2023-12-19T18:14:38.548538+0000","last_became_active":"2023-12-19T18:14:38.545264+0000","last_became_peered":"2023-12-19T18:14:38.545264+0000","last_unstale":"2023-12-19T18:14:38.548538+0000","last_undegraded":"2023-12-19T18:14:38.548538+0000","last_fullsized":"2023-12-19T18:14:38.548538+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:03:43.536300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.565736+0000","last_change":"2023-12-19T18:14:38.556112+0000","last_active":"2023-12-19T18:14:38.565736+0000","last_peered":"2023-12-19T18:14:38.565736+0000","last_clean":"2023-12-19T18:14:38.565736+0000","last_became_active":"2023-12-19T18:14:38.555810+0000","last_became_peered":"2023-12-19T18:14:38.555810+0000","last_unstale":"2023-12-19T18:14:38.565736+0000","last_undegraded":"2023-12-19T18:14:38.565736+0000","last_fullsized":"2023-12-19T18:14:38.565736+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:46:44.414862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.561144+0000","last_change":"2023-12-19T18:14:38.553204+0000","last_active":"2023-12-19T18:14:38.561144+0000","last_peered":"2023-12-19T18:14:38.561144+0000","last_clean":"2023-12-19T18:14:38.561144+0000","last_became_active":"2023-12-19T18:14:38.546596+0000","last_became_peered":"2023-12-19T18:14:38.546596+0000","last_unstale":"2023-12-19T18:14:38.561144+0000","last_undegraded":"2023-12-19T18:14:38.561144+0000","last_fullsized":"2023-12-19T18:14:38.561144+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:44:54.211011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535237+0000","last_change":"2023-12-19T18:14:38.563385+0000","last_active":"2023-12-19T18:14:39.535237+0000","last_peered":"2023-12-19T18:14:39.535237+0000","last_clean":"2023-12-19T18:14:39.535237+0000","last_became_active":"2023-12-19T18:14:38.562599+0000","last_became_peered":"2023-12-19T18:14:38.562599+0000","last_unstale":"2023-12-19T18:14:39.535237+0000","last_undegraded":"2023-12-19T18:14:39.535237+0000","last_fullsized":"2023-12-19T18:14:39.535237+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:02:32.672622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"195'4","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.566582+0000","last_change":"2023-12-19T18:14:38.152178+0000","last_active":"2023-12-19T18:14:38.566582+0000","last_peered":"2023-12-19T18:14:38.566582+0000","last_clean":"2023-12-19T18:14:38.566582+0000","last_became_active":"2023-12-19T18:14:38.151843+0000","last_became_peered":"2023-12-19T18:14:38.151843+0000","last_unstale":"2023-12-19T18:14:38.566582+0000","last_undegraded":"2023-12-19T18:14:38.566582+0000","last_fullsized":"2023-12-19T18:14:38.566582+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:53.356036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.546619+0000","last_change":"2023-12-19T18:14:38.543456+0000","last_active":"2023-12-19T18:14:38.546619+0000","last_peered":"2023-12-19T18:14:38.546619+0000","last_clean":"2023-12-19T18:14:38.546619+0000","last_became_active":"2023-12-19T18:14:38.543164+0000","last_became_peered":"2023-12-19T18:14:38.543164+0000","last_unstale":"2023-12-19T18:14:38.546619+0000","last_undegraded":"2023-12-19T18:14:38.546619+0000","last_fullsized":"2023-12-19T18:14:38.546619+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:35:33.925360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"195'1","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.565592+0000","last_change":"2023-12-19T18:14:38.153100+0000","last_active":"2023-12-19T18:14:38.565592+0000","last_peered":"2023-12-19T18:14:38.565592+0000","last_clean":"2023-12-19T18:14:38.565592+0000","last_became_active":"2023-12-19T18:14:38.152624+0000","last_became_peered":"2023-12-19T18:14:38.152624+0000","last_unstale":"2023-12-19T18:14:38.565592+0000","last_undegraded":"2023-12-19T18:14:38.565592+0000","last_fullsized":"2023-12-19T18:14:38.565592+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:59:29.874589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660094+0000","last_change":"2023-12-19T18:14:38.546775+0000","last_active":"2023-12-19T18:14:39.660094+0000","last_peered":"2023-12-19T18:14:39.660094+0000","last_clean":"2023-12-19T18:14:39.660094+0000","last_became_active":"2023-12-19T18:14:38.546500+0000","last_became_peered":"2023-12-19T18:14:38.546500+0000","last_unstale":"2023-12-19T18:14:39.660094+0000","last_undegraded":"2023-12-19T18:14:39.660094+0000","last_fullsized":"2023-12-19T18:14:39.660094+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:03:18.457399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"195'2","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.560672+0000","last_change":"2023-12-19T18:14:38.154428+0000","last_active":"2023-12-19T18:14:38.560672+0000","last_peered":"2023-12-19T18:14:38.560672+0000","last_clean":"2023-12-19T18:14:38.560672+0000","last_became_active":"2023-12-19T18:14:38.153988+0000","last_became_peered":"2023-12-19T18:14:38.153988+0000","last_unstale":"2023-12-19T18:14:38.560672+0000","last_undegraded":"2023-12-19T18:14:38.560672+0000","last_fullsized":"2023-12-19T18:14:38.560672+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:13:41.861734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.538078+0000","last_change":"2023-12-19T18:14:38.546776+0000","last_active":"2023-12-19T18:14:39.538078+0000","last_peered":"2023-12-19T18:14:39.538078+0000","last_clean":"2023-12-19T18:14:39.538078+0000","last_became_active":"2023-12-19T18:14:38.546503+0000","last_became_peered":"2023-12-19T18:14:38.546503+0000","last_unstale":"2023-12-19T18:14:39.538078+0000","last_undegraded":"2023-12-19T18:14:39.538078+0000","last_fullsized":"2023-12-19T18:14:39.538078+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:14.806434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"195'2","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.563449+0000","last_change":"2023-12-19T18:14:38.145182+0000","last_active":"2023-12-19T18:14:38.563449+0000","last_peered":"2023-12-19T18:14:38.563449+0000","last_clean":"2023-12-19T18:14:38.563449+0000","last_became_active":"2023-12-19T18:14:38.144875+0000","last_became_peered":"2023-12-19T18:14:38.144875+0000","last_unstale":"2023-12-19T18:14:38.563449+0000","last_undegraded":"2023-12-19T18:14:38.563449+0000","last_fullsized":"2023-12-19T18:14:38.563449+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:31:54.174071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535425+0000","last_change":"2023-12-19T18:14:38.564692+0000","last_active":"2023-12-19T18:14:39.535425+0000","last_peered":"2023-12-19T18:14:39.535425+0000","last_clean":"2023-12-19T18:14:39.535425+0000","last_became_active":"2023-12-19T18:14:38.564428+0000","last_became_peered":"2023-12-19T18:14:38.564428+0000","last_unstale":"2023-12-19T18:14:39.535425+0000","last_undegraded":"2023-12-19T18:14:39.535425+0000","last_fullsized":"2023-12-19T18:14:39.535425+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:24.598857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"195'1","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.538089+0000","last_change":"2023-12-19T18:14:38.150995+0000","last_active":"2023-12-19T18:14:39.538089+0000","last_peered":"2023-12-19T18:14:39.538089+0000","last_clean":"2023-12-19T18:14:39.538089+0000","last_became_active":"2023-12-19T18:14:38.150325+0000","last_became_peered":"2023-12-19T18:14:38.150325+0000","last_unstale":"2023-12-19T18:14:39.538089+0000","last_undegraded":"2023-12-19T18:14:39.538089+0000","last_fullsized":"2023-12-19T18:14:39.538089+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:13.415745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.553802+0000","last_change":"2023-12-19T18:14:38.549299+0000","last_active":"2023-12-19T18:14:38.553802+0000","last_peered":"2023-12-19T18:14:38.553802+0000","last_clean":"2023-12-19T18:14:38.553802+0000","last_became_active":"2023-12-19T18:14:38.549007+0000","last_became_peered":"2023-12-19T18:14:38.549007+0000","last_unstale":"2023-12-19T18:14:38.553802+0000","last_undegraded":"2023-12-19T18:14:38.553802+0000","last_fullsized":"2023-12-19T18:14:38.553802+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:05:57.229774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"195'1","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.562825+0000","last_change":"2023-12-19T18:14:38.146423+0000","last_active":"2023-12-19T18:14:38.562825+0000","last_peered":"2023-12-19T18:14:38.562825+0000","last_clean":"2023-12-19T18:14:38.562825+0000","last_became_active":"2023-12-19T18:14:38.146116+0000","last_became_peered":"2023-12-19T18:14:38.146116+0000","last_unstale":"2023-12-19T18:14:38.562825+0000","last_undegraded":"2023-12-19T18:14:38.562825+0000","last_fullsized":"2023-12-19T18:14:38.562825+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:53.356036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.553889+0000","last_change":"2023-12-19T18:14:38.549715+0000","last_active":"2023-12-19T18:14:38.553889+0000","last_peered":"2023-12-19T18:14:38.553889+0000","last_clean":"2023-12-19T18:14:38.553889+0000","last_became_active":"2023-12-19T18:14:38.549426+0000","last_became_peered":"2023-12-19T18:14:38.549426+0000","last_unstale":"2023-12-19T18:14:38.553889+0000","last_undegraded":"2023-12-19T18:14:38.553889+0000","last_fullsized":"2023-12-19T18:14:38.553889+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:04.030577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536612+0000","last_change":"2023-12-19T18:14:38.153816+0000","last_active":"2023-12-19T18:14:39.536612+0000","last_peered":"2023-12-19T18:14:39.536612+0000","last_clean":"2023-12-19T18:14:39.536612+0000","last_became_active":"2023-12-19T18:14:38.153423+0000","last_became_peered":"2023-12-19T18:14:38.153423+0000","last_unstale":"2023-12-19T18:14:39.536612+0000","last_undegraded":"2023-12-19T18:14:39.536612+0000","last_fullsized":"2023-12-19T18:14:39.536612+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:58:42.525810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660034+0000","last_change":"2023-12-19T18:14:38.564879+0000","last_active":"2023-12-19T18:14:39.660034+0000","last_peered":"2023-12-19T18:14:39.660034+0000","last_clean":"2023-12-19T18:14:39.660034+0000","last_became_active":"2023-12-19T18:14:38.564612+0000","last_became_peered":"2023-12-19T18:14:38.564612+0000","last_unstale":"2023-12-19T18:14:39.660034+0000","last_undegraded":"2023-12-19T18:14:39.660034+0000","last_fullsized":"2023-12-19T18:14:39.660034+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:24:29.807045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535031+0000","last_change":"2023-12-19T18:14:38.143601+0000","last_active":"2023-12-19T18:14:39.535031+0000","last_peered":"2023-12-19T18:14:39.535031+0000","last_clean":"2023-12-19T18:14:39.535031+0000","last_became_active":"2023-12-19T18:14:38.143321+0000","last_became_peered":"2023-12-19T18:14:38.143321+0000","last_unstale":"2023-12-19T18:14:39.535031+0000","last_undegraded":"2023-12-19T18:14:39.535031+0000","last_fullsized":"2023-12-19T18:14:39.535031+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:37:51.634247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.566385+0000","last_change":"2023-12-19T18:14:38.561096+0000","last_active":"2023-12-19T18:14:38.566385+0000","last_peered":"2023-12-19T18:14:38.566385+0000","last_clean":"2023-12-19T18:14:38.566385+0000","last_became_active":"2023-12-19T18:14:38.560801+0000","last_became_peered":"2023-12-19T18:14:38.560801+0000","last_unstale":"2023-12-19T18:14:38.566385+0000","last_undegraded":"2023-12-19T18:14:38.566385+0000","last_fullsized":"2023-12-19T18:14:38.566385+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:37:34.132929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660257+0000","last_change":"2023-12-19T18:14:38.153381+0000","last_active":"2023-12-19T18:14:39.660257+0000","last_peered":"2023-12-19T18:14:39.660257+0000","last_clean":"2023-12-19T18:14:39.660257+0000","last_became_active":"2023-12-19T18:14:38.152008+0000","last_became_peered":"2023-12-19T18:14:38.152008+0000","last_unstale":"2023-12-19T18:14:39.660257+0000","last_undegraded":"2023-12-19T18:14:39.660257+0000","last_fullsized":"2023-12-19T18:14:39.660257+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:54.330119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.565968+0000","last_change":"2023-12-19T18:14:38.559625+0000","last_active":"2023-12-19T18:14:38.565968+0000","last_peered":"2023-12-19T18:14:38.565968+0000","last_clean":"2023-12-19T18:14:38.565968+0000","last_became_active":"2023-12-19T18:14:38.559300+0000","last_became_peered":"2023-12-19T18:14:38.559300+0000","last_unstale":"2023-12-19T18:14:38.565968+0000","last_undegraded":"2023-12-19T18:14:38.565968+0000","last_fullsized":"2023-12-19T18:14:38.565968+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:05:57.229774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"195'3","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660209+0000","last_change":"2023-12-19T18:14:38.145023+0000","last_active":"2023-12-19T18:14:39.660209+0000","last_peered":"2023-12-19T18:14:39.660209+0000","last_clean":"2023-12-19T18:14:39.660209+0000","last_became_active":"2023-12-19T18:14:38.144728+0000","last_became_peered":"2023-12-19T18:14:38.144728+0000","last_unstale":"2023-12-19T18:14:39.660209+0000","last_undegraded":"2023-12-19T18:14:39.660209+0000","last_fullsized":"2023-12-19T18:14:39.660209+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:50.524202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.550640+0000","last_change":"2023-12-19T18:14:38.548265+0000","last_active":"2023-12-19T18:14:38.550640+0000","last_peered":"2023-12-19T18:14:38.550640+0000","last_clean":"2023-12-19T18:14:38.550640+0000","last_became_active":"2023-12-19T18:14:38.547907+0000","last_became_peered":"2023-12-19T18:14:38.547907+0000","last_unstale":"2023-12-19T18:14:38.550640+0000","last_undegraded":"2023-12-19T18:14:38.550640+0000","last_fullsized":"2023-12-19T18:14:38.550640+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:25.892977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"195'6","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.570244+0000","last_change":"2023-12-19T18:14:38.144796+0000","last_active":"2023-12-19T18:14:38.570244+0000","last_peered":"2023-12-19T18:14:38.570244+0000","last_clean":"2023-12-19T18:14:38.570244+0000","last_became_active":"2023-12-19T18:14:38.144531+0000","last_became_peered":"2023-12-19T18:14:38.144531+0000","last_unstale":"2023-12-19T18:14:38.570244+0000","last_undegraded":"2023-12-19T18:14:38.570244+0000","last_fullsized":"2023-12-19T18:14:38.570244+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:06.565993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.562342+0000","last_change":"2023-12-19T18:14:38.557099+0000","last_active":"2023-12-19T18:14:38.562342+0000","last_peered":"2023-12-19T18:14:38.562342+0000","last_clean":"2023-12-19T18:14:38.562342+0000","last_became_active":"2023-12-19T18:14:38.556812+0000","last_became_peered":"2023-12-19T18:14:38.556812+0000","last_unstale":"2023-12-19T18:14:38.562342+0000","last_undegraded":"2023-12-19T18:14:38.562342+0000","last_fullsized":"2023-12-19T18:14:38.562342+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:31:33.954931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"195'2","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.566486+0000","last_change":"2023-12-19T18:14:38.150017+0000","last_active":"2023-12-19T18:14:38.566486+0000","last_peered":"2023-12-19T18:14:38.566486+0000","last_clean":"2023-12-19T18:14:38.566486+0000","last_became_active":"2023-12-19T18:14:38.149695+0000","last_became_peered":"2023-12-19T18:14:38.149695+0000","last_unstale":"2023-12-19T18:14:38.566486+0000","last_undegraded":"2023-12-19T18:14:38.566486+0000","last_fullsized":"2023-12-19T18:14:38.566486+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:14:59.693406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535187+0000","last_change":"2023-12-19T18:14:38.557555+0000","last_active":"2023-12-19T18:14:39.535187+0000","last_peered":"2023-12-19T18:14:39.535187+0000","last_clean":"2023-12-19T18:14:39.535187+0000","last_became_active":"2023-12-19T18:14:38.557292+0000","last_became_peered":"2023-12-19T18:14:38.557292+0000","last_unstale":"2023-12-19T18:14:39.535187+0000","last_undegraded":"2023-12-19T18:14:39.535187+0000","last_fullsized":"2023-12-19T18:14:39.535187+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:46.428487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.537606+0000","last_change":"2023-12-19T18:14:38.145593+0000","last_active":"2023-12-19T18:14:38.537606+0000","last_peered":"2023-12-19T18:14:38.537606+0000","last_clean":"2023-12-19T18:14:38.537606+0000","last_became_active":"2023-12-19T18:14:38.145328+0000","last_became_peered":"2023-12-19T18:14:38.145328+0000","last_unstale":"2023-12-19T18:14:38.537606+0000","last_undegraded":"2023-12-19T18:14:38.537606+0000","last_fullsized":"2023-12-19T18:14:38.537606+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:56:15.017376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.541978+0000","last_change":"2023-12-19T18:14:38.539859+0000","last_active":"2023-12-19T18:14:38.541978+0000","last_peered":"2023-12-19T18:14:38.541978+0000","last_clean":"2023-12-19T18:14:38.541978+0000","last_became_active":"2023-12-19T18:14:38.539539+0000","last_became_peered":"2023-12-19T18:14:38.539539+0000","last_unstale":"2023-12-19T18:14:38.541978+0000","last_undegraded":"2023-12-19T18:14:38.541978+0000","last_fullsized":"2023-12-19T18:14:38.541978+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:02:09.451847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.532910+0000","last_change":"2023-12-19T18:14:38.149760+0000","last_active":"2023-12-19T18:14:38.532910+0000","last_peered":"2023-12-19T18:14:38.532910+0000","last_clean":"2023-12-19T18:14:38.532910+0000","last_became_active":"2023-12-19T18:14:38.149441+0000","last_became_peered":"2023-12-19T18:14:38.149441+0000","last_unstale":"2023-12-19T18:14:38.532910+0000","last_undegraded":"2023-12-19T18:14:38.532910+0000","last_fullsized":"2023-12-19T18:14:38.532910+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:33:02.035434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537234+0000","last_change":"2023-12-19T18:14:38.561761+0000","last_active":"2023-12-19T18:14:39.537234+0000","last_peered":"2023-12-19T18:14:39.537234+0000","last_clean":"2023-12-19T18:14:39.537234+0000","last_became_active":"2023-12-19T18:14:38.561487+0000","last_became_peered":"2023-12-19T18:14:38.561487+0000","last_unstale":"2023-12-19T18:14:39.537234+0000","last_undegraded":"2023-12-19T18:14:39.537234+0000","last_fullsized":"2023-12-19T18:14:39.537234+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:32.366343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"195'8","reported_seq":18,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.567881+0000","last_change":"2023-12-19T18:14:38.139818+0000","last_active":"2023-12-19T18:14:38.567881+0000","last_peered":"2023-12-19T18:14:38.567881+0000","last_clean":"2023-12-19T18:14:38.567881+0000","last_became_active":"2023-12-19T18:14:38.139512+0000","last_became_peered":"2023-12-19T18:14:38.139512+0000","last_unstale":"2023-12-19T18:14:38.567881+0000","last_undegraded":"2023-12-19T18:14:38.567881+0000","last_fullsized":"2023-12-19T18:14:38.567881+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:32:03.950627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.561026+0000","last_change":"2023-12-19T18:14:38.552771+0000","last_active":"2023-12-19T18:14:38.561026+0000","last_peered":"2023-12-19T18:14:38.561026+0000","last_clean":"2023-12-19T18:14:38.561026+0000","last_became_active":"2023-12-19T18:14:38.552489+0000","last_became_peered":"2023-12-19T18:14:38.552489+0000","last_unstale":"2023-12-19T18:14:38.561026+0000","last_undegraded":"2023-12-19T18:14:38.561026+0000","last_fullsized":"2023-12-19T18:14:38.561026+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:30:38.872598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"195'1","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537962+0000","last_change":"2023-12-19T18:14:38.154621+0000","last_active":"2023-12-19T18:14:39.537962+0000","last_peered":"2023-12-19T18:14:39.537962+0000","last_clean":"2023-12-19T18:14:39.537962+0000","last_became_active":"2023-12-19T18:14:38.153849+0000","last_became_peered":"2023-12-19T18:14:38.153849+0000","last_unstale":"2023-12-19T18:14:39.537962+0000","last_undegraded":"2023-12-19T18:14:39.537962+0000","last_fullsized":"2023-12-19T18:14:39.537962+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:21.424875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.546629+0000","last_change":"2023-12-19T18:14:38.544944+0000","last_active":"2023-12-19T18:14:38.546629+0000","last_peered":"2023-12-19T18:14:38.546629+0000","last_clean":"2023-12-19T18:14:38.546629+0000","last_became_active":"2023-12-19T18:14:38.544640+0000","last_became_peered":"2023-12-19T18:14:38.544640+0000","last_unstale":"2023-12-19T18:14:38.546629+0000","last_undegraded":"2023-12-19T18:14:38.546629+0000","last_fullsized":"2023-12-19T18:14:38.546629+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:42:55.210047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"195'2","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537863+0000","last_change":"2023-12-19T18:14:38.151213+0000","last_active":"2023-12-19T18:14:39.537863+0000","last_peered":"2023-12-19T18:14:39.537863+0000","last_clean":"2023-12-19T18:14:39.537863+0000","last_became_active":"2023-12-19T18:14:38.150866+0000","last_became_peered":"2023-12-19T18:14:38.150866+0000","last_unstale":"2023-12-19T18:14:39.537863+0000","last_undegraded":"2023-12-19T18:14:39.537863+0000","last_fullsized":"2023-12-19T18:14:39.537863+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:56:58.937364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.547316+0000","last_change":"2023-12-19T18:14:38.544659+0000","last_active":"2023-12-19T18:14:38.547316+0000","last_peered":"2023-12-19T18:14:38.547316+0000","last_clean":"2023-12-19T18:14:38.547316+0000","last_became_active":"2023-12-19T18:14:38.544382+0000","last_became_peered":"2023-12-19T18:14:38.544382+0000","last_unstale":"2023-12-19T18:14:38.547316+0000","last_undegraded":"2023-12-19T18:14:38.547316+0000","last_fullsized":"2023-12-19T18:14:38.547316+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:45:15.039695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"195'1","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.568028+0000","last_change":"2023-12-19T18:14:38.140731+0000","last_active":"2023-12-19T18:14:38.568028+0000","last_peered":"2023-12-19T18:14:38.568028+0000","last_clean":"2023-12-19T18:14:38.568028+0000","last_became_active":"2023-12-19T18:14:38.140467+0000","last_became_peered":"2023-12-19T18:14:38.140467+0000","last_unstale":"2023-12-19T18:14:38.568028+0000","last_undegraded":"2023-12-19T18:14:38.568028+0000","last_fullsized":"2023-12-19T18:14:38.568028+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:40.296023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535208+0000","last_change":"2023-12-19T18:14:38.567457+0000","last_active":"2023-12-19T18:14:39.535208+0000","last_peered":"2023-12-19T18:14:39.535208+0000","last_clean":"2023-12-19T18:14:39.535208+0000","last_became_active":"2023-12-19T18:14:38.567136+0000","last_became_peered":"2023-12-19T18:14:38.567136+0000","last_unstale":"2023-12-19T18:14:39.535208+0000","last_undegraded":"2023-12-19T18:14:39.535208+0000","last_fullsized":"2023-12-19T18:14:39.535208+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:58:45.881349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.541641+0000","last_change":"2023-12-19T18:14:38.147705+0000","last_active":"2023-12-19T18:14:38.541641+0000","last_peered":"2023-12-19T18:14:38.541641+0000","last_clean":"2023-12-19T18:14:38.541641+0000","last_became_active":"2023-12-19T18:14:38.147433+0000","last_became_peered":"2023-12-19T18:14:38.147433+0000","last_unstale":"2023-12-19T18:14:38.541641+0000","last_undegraded":"2023-12-19T18:14:38.541641+0000","last_fullsized":"2023-12-19T18:14:38.541641+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:00:43.905340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537431+0000","last_change":"2023-12-19T18:14:38.545796+0000","last_active":"2023-12-19T18:14:39.537431+0000","last_peered":"2023-12-19T18:14:39.537431+0000","last_clean":"2023-12-19T18:14:39.537431+0000","last_became_active":"2023-12-19T18:14:38.544913+0000","last_became_peered":"2023-12-19T18:14:38.544913+0000","last_unstale":"2023-12-19T18:14:39.537431+0000","last_undegraded":"2023-12-19T18:14:39.537431+0000","last_fullsized":"2023-12-19T18:14:39.537431+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:06:43.178068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"195'2","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.565370+0000","last_change":"2023-12-19T18:14:38.154453+0000","last_active":"2023-12-19T18:14:38.565370+0000","last_peered":"2023-12-19T18:14:38.565370+0000","last_clean":"2023-12-19T18:14:38.565370+0000","last_became_active":"2023-12-19T18:14:38.153854+0000","last_became_peered":"2023-12-19T18:14:38.153854+0000","last_unstale":"2023-12-19T18:14:38.565370+0000","last_undegraded":"2023-12-19T18:14:38.565370+0000","last_fullsized":"2023-12-19T18:14:38.565370+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:58.287223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535283+0000","last_change":"2023-12-19T18:14:38.559775+0000","last_active":"2023-12-19T18:14:39.535283+0000","last_peered":"2023-12-19T18:14:39.535283+0000","last_clean":"2023-12-19T18:14:39.535283+0000","last_became_active":"2023-12-19T18:14:38.559512+0000","last_became_peered":"2023-12-19T18:14:38.559512+0000","last_unstale":"2023-12-19T18:14:39.535283+0000","last_undegraded":"2023-12-19T18:14:39.535283+0000","last_fullsized":"2023-12-19T18:14:39.535283+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:10:18.702122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.542035+0000","last_change":"2023-12-19T18:14:38.147302+0000","last_active":"2023-12-19T18:14:38.542035+0000","last_peered":"2023-12-19T18:14:38.542035+0000","last_clean":"2023-12-19T18:14:38.542035+0000","last_became_active":"2023-12-19T18:14:38.147003+0000","last_became_peered":"2023-12-19T18:14:38.147003+0000","last_unstale":"2023-12-19T18:14:38.542035+0000","last_undegraded":"2023-12-19T18:14:38.542035+0000","last_fullsized":"2023-12-19T18:14:38.542035+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:14:59.693406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.560918+0000","last_change":"2023-12-19T18:14:38.551584+0000","last_active":"2023-12-19T18:14:38.560918+0000","last_peered":"2023-12-19T18:14:38.560918+0000","last_clean":"2023-12-19T18:14:38.560918+0000","last_became_active":"2023-12-19T18:14:38.551312+0000","last_became_peered":"2023-12-19T18:14:38.551312+0000","last_unstale":"2023-12-19T18:14:38.560918+0000","last_undegraded":"2023-12-19T18:14:38.560918+0000","last_fullsized":"2023-12-19T18:14:38.560918+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:36:34.011690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.532943+0000","last_change":"2023-12-19T18:14:38.147014+0000","last_active":"2023-12-19T18:14:38.532943+0000","last_peered":"2023-12-19T18:14:38.532943+0000","last_clean":"2023-12-19T18:14:38.532943+0000","last_became_active":"2023-12-19T18:14:38.146758+0000","last_became_peered":"2023-12-19T18:14:38.146758+0000","last_unstale":"2023-12-19T18:14:38.532943+0000","last_undegraded":"2023-12-19T18:14:38.532943+0000","last_fullsized":"2023-12-19T18:14:38.532943+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:55:05.518903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660053+0000","last_change":"2023-12-19T18:14:38.553169+0000","last_active":"2023-12-19T18:14:39.660053+0000","last_peered":"2023-12-19T18:14:39.660053+0000","last_clean":"2023-12-19T18:14:39.660053+0000","last_became_active":"2023-12-19T18:14:38.552629+0000","last_became_peered":"2023-12-19T18:14:38.552629+0000","last_unstale":"2023-12-19T18:14:39.660053+0000","last_undegraded":"2023-12-19T18:14:39.660053+0000","last_fullsized":"2023-12-19T18:14:39.660053+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:56:59.946145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.541655+0000","last_change":"2023-12-19T18:14:38.139912+0000","last_active":"2023-12-19T18:14:38.541655+0000","last_peered":"2023-12-19T18:14:38.541655+0000","last_clean":"2023-12-19T18:14:38.541655+0000","last_became_active":"2023-12-19T18:14:38.139631+0000","last_became_peered":"2023-12-19T18:14:38.139631+0000","last_unstale":"2023-12-19T18:14:38.541655+0000","last_undegraded":"2023-12-19T18:14:38.541655+0000","last_fullsized":"2023-12-19T18:14:38.541655+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:35:14.383673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.566049+0000","last_change":"2023-12-19T18:14:38.560175+0000","last_active":"2023-12-19T18:14:38.566049+0000","last_peered":"2023-12-19T18:14:38.566049+0000","last_clean":"2023-12-19T18:14:38.566049+0000","last_became_active":"2023-12-19T18:14:38.559764+0000","last_became_peered":"2023-12-19T18:14:38.559764+0000","last_unstale":"2023-12-19T18:14:38.566049+0000","last_undegraded":"2023-12-19T18:14:38.566049+0000","last_fullsized":"2023-12-19T18:14:38.566049+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:03.792973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"195'2","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.566331+0000","last_change":"2023-12-19T18:14:38.150122+0000","last_active":"2023-12-19T18:14:38.566331+0000","last_peered":"2023-12-19T18:14:38.566331+0000","last_clean":"2023-12-19T18:14:38.566331+0000","last_became_active":"2023-12-19T18:14:38.149061+0000","last_became_peered":"2023-12-19T18:14:38.149061+0000","last_unstale":"2023-12-19T18:14:38.566331+0000","last_undegraded":"2023-12-19T18:14:38.566331+0000","last_fullsized":"2023-12-19T18:14:38.566331+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:58.287223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535228+0000","last_change":"2023-12-19T18:14:38.555610+0000","last_active":"2023-12-19T18:14:39.535228+0000","last_peered":"2023-12-19T18:14:39.535228+0000","last_clean":"2023-12-19T18:14:39.535228+0000","last_became_active":"2023-12-19T18:14:38.554505+0000","last_became_peered":"2023-12-19T18:14:38.554505+0000","last_unstale":"2023-12-19T18:14:39.535228+0000","last_undegraded":"2023-12-19T18:14:39.535228+0000","last_fullsized":"2023-12-19T18:14:39.535228+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:18:27.989688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537748+0000","last_change":"2023-12-19T18:14:38.150362+0000","last_active":"2023-12-19T18:14:39.537748+0000","last_peered":"2023-12-19T18:14:39.537748+0000","last_clean":"2023-12-19T18:14:39.537748+0000","last_became_active":"2023-12-19T18:14:38.149541+0000","last_became_peered":"2023-12-19T18:14:38.149541+0000","last_unstale":"2023-12-19T18:14:39.537748+0000","last_undegraded":"2023-12-19T18:14:39.537748+0000","last_fullsized":"2023-12-19T18:14:39.537748+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:01:06.419377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660063+0000","last_change":"2023-12-19T18:14:38.547371+0000","last_active":"2023-12-19T18:14:39.660063+0000","last_peered":"2023-12-19T18:14:39.660063+0000","last_clean":"2023-12-19T18:14:39.660063+0000","last_became_active":"2023-12-19T18:14:38.546149+0000","last_became_peered":"2023-12-19T18:14:38.546149+0000","last_unstale":"2023-12-19T18:14:39.660063+0000","last_undegraded":"2023-12-19T18:14:39.660063+0000","last_fullsized":"2023-12-19T18:14:39.660063+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:14.424526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536866+0000","last_change":"2023-12-19T18:14:38.151549+0000","last_active":"2023-12-19T18:14:39.536866+0000","last_peered":"2023-12-19T18:14:39.536866+0000","last_clean":"2023-12-19T18:14:39.536866+0000","last_became_active":"2023-12-19T18:14:38.150642+0000","last_became_peered":"2023-12-19T18:14:38.150642+0000","last_unstale":"2023-12-19T18:14:39.536866+0000","last_undegraded":"2023-12-19T18:14:39.536866+0000","last_fullsized":"2023-12-19T18:14:39.536866+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:50.636065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.561175+0000","last_change":"2023-12-19T18:14:38.556310+0000","last_active":"2023-12-19T18:14:38.561175+0000","last_peered":"2023-12-19T18:14:38.561175+0000","last_clean":"2023-12-19T18:14:38.561175+0000","last_became_active":"2023-12-19T18:14:38.556035+0000","last_became_peered":"2023-12-19T18:14:38.556035+0000","last_unstale":"2023-12-19T18:14:38.561175+0000","last_undegraded":"2023-12-19T18:14:38.561175+0000","last_fullsized":"2023-12-19T18:14:38.561175+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:04.030577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.538807+0000","last_change":"2023-12-19T18:14:38.155386+0000","last_active":"2023-12-19T18:14:38.538807+0000","last_peered":"2023-12-19T18:14:38.538807+0000","last_clean":"2023-12-19T18:14:38.538807+0000","last_became_active":"2023-12-19T18:14:38.155090+0000","last_became_peered":"2023-12-19T18:14:38.155090+0000","last_unstale":"2023-12-19T18:14:38.538807+0000","last_undegraded":"2023-12-19T18:14:38.538807+0000","last_fullsized":"2023-12-19T18:14:38.538807+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:33:02.035434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.560943+0000","last_change":"2023-12-19T18:14:38.556518+0000","last_active":"2023-12-19T18:14:38.560943+0000","last_peered":"2023-12-19T18:14:38.560943+0000","last_clean":"2023-12-19T18:14:38.560943+0000","last_became_active":"2023-12-19T18:14:38.555850+0000","last_became_peered":"2023-12-19T18:14:38.555850+0000","last_unstale":"2023-12-19T18:14:38.560943+0000","last_undegraded":"2023-12-19T18:14:38.560943+0000","last_fullsized":"2023-12-19T18:14:38.560943+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:56.677526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.536608+0000","last_change":"2023-12-19T18:14:38.147774+0000","last_active":"2023-12-19T18:14:38.536608+0000","last_peered":"2023-12-19T18:14:38.536608+0000","last_clean":"2023-12-19T18:14:38.536608+0000","last_became_active":"2023-12-19T18:14:38.147452+0000","last_became_peered":"2023-12-19T18:14:38.147452+0000","last_unstale":"2023-12-19T18:14:38.536608+0000","last_undegraded":"2023-12-19T18:14:38.536608+0000","last_fullsized":"2023-12-19T18:14:38.536608+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:44.967592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.567780+0000","last_change":"2023-12-19T18:14:38.563043+0000","last_active":"2023-12-19T18:14:38.567780+0000","last_peered":"2023-12-19T18:14:38.567780+0000","last_clean":"2023-12-19T18:14:38.567780+0000","last_became_active":"2023-12-19T18:14:38.562735+0000","last_became_peered":"2023-12-19T18:14:38.562735+0000","last_unstale":"2023-12-19T18:14:38.567780+0000","last_undegraded":"2023-12-19T18:14:38.567780+0000","last_fullsized":"2023-12-19T18:14:38.567780+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:26:03.035425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.533773+0000","last_change":"2023-12-19T18:14:38.144637+0000","last_active":"2023-12-19T18:14:38.533773+0000","last_peered":"2023-12-19T18:14:38.533773+0000","last_clean":"2023-12-19T18:14:38.533773+0000","last_became_active":"2023-12-19T18:14:38.143545+0000","last_became_peered":"2023-12-19T18:14:38.143545+0000","last_unstale":"2023-12-19T18:14:38.533773+0000","last_undegraded":"2023-12-19T18:14:38.533773+0000","last_fullsized":"2023-12-19T18:14:38.533773+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:56:15.017376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535003+0000","last_change":"2023-12-19T18:14:38.565919+0000","last_active":"2023-12-19T18:14:39.535003+0000","last_peered":"2023-12-19T18:14:39.535003+0000","last_clean":"2023-12-19T18:14:39.535003+0000","last_became_active":"2023-12-19T18:14:38.564564+0000","last_became_peered":"2023-12-19T18:14:38.564564+0000","last_unstale":"2023-12-19T18:14:39.535003+0000","last_undegraded":"2023-12-19T18:14:39.535003+0000","last_fullsized":"2023-12-19T18:14:39.535003+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:25:15.851908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536831+0000","last_change":"2023-12-19T18:14:38.152239+0000","last_active":"2023-12-19T18:14:39.536831+0000","last_peered":"2023-12-19T18:14:39.536831+0000","last_clean":"2023-12-19T18:14:39.536831+0000","last_became_active":"2023-12-19T18:14:38.151919+0000","last_became_peered":"2023-12-19T18:14:38.151919+0000","last_unstale":"2023-12-19T18:14:39.536831+0000","last_undegraded":"2023-12-19T18:14:39.536831+0000","last_fullsized":"2023-12-19T18:14:39.536831+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:29:19.462927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537233+0000","last_change":"2023-12-19T18:14:38.546681+0000","last_active":"2023-12-19T18:14:39.537233+0000","last_peered":"2023-12-19T18:14:39.537233+0000","last_clean":"2023-12-19T18:14:39.537233+0000","last_became_active":"2023-12-19T18:14:38.546405+0000","last_became_peered":"2023-12-19T18:14:38.546405+0000","last_unstale":"2023-12-19T18:14:39.537233+0000","last_undegraded":"2023-12-19T18:14:39.537233+0000","last_fullsized":"2023-12-19T18:14:39.537233+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:17:01.221162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.536096+0000","last_change":"2023-12-19T18:14:38.149007+0000","last_active":"2023-12-19T18:14:38.536096+0000","last_peered":"2023-12-19T18:14:38.536096+0000","last_clean":"2023-12-19T18:14:38.536096+0000","last_became_active":"2023-12-19T18:14:38.148727+0000","last_became_peered":"2023-12-19T18:14:38.148727+0000","last_unstale":"2023-12-19T18:14:38.536096+0000","last_undegraded":"2023-12-19T18:14:38.536096+0000","last_fullsized":"2023-12-19T18:14:38.536096+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:52.430219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.563644+0000","last_change":"2023-12-19T18:14:38.558001+0000","last_active":"2023-12-19T18:14:38.563644+0000","last_peered":"2023-12-19T18:14:38.563644+0000","last_clean":"2023-12-19T18:14:38.563644+0000","last_became_active":"2023-12-19T18:14:38.557768+0000","last_became_peered":"2023-12-19T18:14:38.557768+0000","last_unstale":"2023-12-19T18:14:38.563644+0000","last_undegraded":"2023-12-19T18:14:38.563644+0000","last_fullsized":"2023-12-19T18:14:38.563644+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:45:15.039695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.538195+0000","last_change":"2023-12-19T18:14:38.148333+0000","last_active":"2023-12-19T18:14:38.538195+0000","last_peered":"2023-12-19T18:14:38.538195+0000","last_clean":"2023-12-19T18:14:38.538195+0000","last_became_active":"2023-12-19T18:14:38.148068+0000","last_became_peered":"2023-12-19T18:14:38.148068+0000","last_unstale":"2023-12-19T18:14:38.538195+0000","last_undegraded":"2023-12-19T18:14:38.538195+0000","last_fullsized":"2023-12-19T18:14:38.538195+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:08.073725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"184'324","reported_seq":1090,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.132191+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:14:38.132191+0000","last_peered":"2023-12-19T18:14:38.132191+0000","last_clean":"2023-12-19T18:14:38.132191+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:14:38.132191+0000","last_undegraded":"2023-12-19T18:14:38.132191+0000","last_fullsized":"2023-12-19T18:14:38.132191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1922,"num_read_kb":3999,"num_write":535,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1922,"num_read_kb":3999,"num_write":535,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":324,"ondisk_log_size":324,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60264,"kb_used_data":2976,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311576,"statfs":{"total":96636764160,"available":96575053824,"internally_reserved":0,"allocated":3047424,"data_stored":1871720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.74299999999999999,"15min":0.72599999999999998},"min":{"1min":0.52500000000000002,"5min":0.45600000000000002,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.754,"15min":0.73899999999999999},"min":{"1min":0.61099999999999999,"5min":0.52100000000000002,"15min":0.45500000000000002},"max":{"1min":1.022,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.80000000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.748},"min":{"1min":0.57499999999999996,"5min":0.50800000000000001,"15min":0.48499999999999999},"max":{"1min":0.91900000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.55800000000000005,"5min":0.48099999999999998,"15min":0.42099999999999999},"max":{"1min":0.92000000000000004,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.82299999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75700000000000001,"15min":0.74099999999999999},"min":{"1min":0.64000000000000001,"5min":0.55400000000000005,"15min":0.38900000000000001},"max":{"1min":0.97899999999999998,"5min":1.036,"15min":1.0669999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77800000000000002,"15min":0.78400000000000003},"min":{"1min":0.59999999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0109999999999999,"5min":1.159,"15min":1.236},"last":0.67300000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.61599999999999999,"5min":0.55700000000000005,"15min":0.48999999999999999},"max":{"1min":1.002,"5min":1.079,"15min":1.079},"last":0.746},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.78300000000000003,"15min":0.78400000000000003},"min":{"1min":0.69199999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96099999999999997,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.73099999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.46300000000000002,"5min":0.40400000000000003,"15min":0.40300000000000002},"max":{"1min":0.84799999999999998,"5min":1.046,"15min":1.302},"last":0.629},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.626},"min":{"1min":0.44900000000000001,"5min":0.44600000000000001,"15min":0.42699999999999999},"max":{"1min":0.84699999999999998,"5min":0.95099999999999996,"15min":1.0209999999999999},"last":0.58899999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61799999999999999,"15min":0.63500000000000001},"min":{"1min":0.47399999999999998,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.88300000000000001,"5min":0.96199999999999997,"15min":1.1299999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65400000000000003,"15min":0.66500000000000004},"min":{"1min":0.51700000000000002,"5min":0.46899999999999997,"15min":0.433},"max":{"1min":0.86499999999999999,"5min":0.97299999999999998,"15min":1.0800000000000001},"last":0.76900000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.72099999999999997},"min":{"1min":0.56299999999999994,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":0.99099999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.505,"5min":0.47199999999999998,"15min":0.42799999999999999},"max":{"1min":0.89700000000000002,"5min":1.0600000000000001,"15min":1.093},"last":0.68999999999999995}]}]},{"osd":6,"up_from":15,"seq":64424510545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63772,"kb_used_data":2388,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308068,"statfs":{"total":96636764160,"available":96571461632,"internally_reserved":0,"allocated":2445312,"data_stored":1286100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72899999999999998,"15min":0.747},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.43099999999999999},"max":{"1min":0.95699999999999996,"5min":0.98199999999999998,"15min":1.042},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72199999999999998,"15min":0.73899999999999999},"min":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.46000000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.65300000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71199999999999997,"15min":0.73499999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.54000000000000004,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":0.99199999999999999,"15min":1.1259999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.621,"5min":0.502,"15min":0.502},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0609999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74099999999999999,"15min":0.75700000000000001},"min":{"1min":0.57499999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.95399999999999996,"5min":1.0129999999999999,"15min":1.111},"last":0.61499999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76700000000000002,"15min":0.78600000000000003},"min":{"1min":0.58099999999999996,"5min":0.495,"15min":0.495},"max":{"1min":0.89400000000000002,"5min":1.048,"15min":1.1519999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.63800000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.436,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59699999999999998,"15min":0.60899999999999999},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.375},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.503}]},{"osd":5,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59899999999999998,"15min":0.60599999999999998},"min":{"1min":0.44,"5min":0.39700000000000002,"15min":0.374},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64800000000000002,"15min":0.66700000000000004},"min":{"1min":0.50900000000000001,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.66700000000000004}]},{"osd":7,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.52900000000000003,"5min":0.502,"15min":0.46000000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.51700000000000002,"5min":0.5,"15min":0.44700000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.72499999999999998}]}]},{"osd":5,"up_from":15,"seq":64424510545,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29332,"kb_used_data":2380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342508,"statfs":{"total":96636764160,"available":96606728192,"internally_reserved":0,"allocated":2437120,"data_stored":1279650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.74399999999999999,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.34399999999999997},"max":{"1min":1.002,"5min":1.0109999999999999,"15min":1.0149999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75700000000000001,"15min":0.74399999999999999},"min":{"1min":0.48599999999999999,"5min":0.48399999999999999,"15min":0.40699999999999997},"max":{"1min":0.96099999999999997,"5min":0.98299999999999998,"15min":1.0229999999999999},"last":0.56200000000000006}]},{"osd":1,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78300000000000003,"15min":0.76100000000000001},"min":{"1min":0.52100000000000002,"5min":0.497,"15min":0.49299999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.47499999999999998},"max":{"1min":0.88900000000000001,"5min":1.004,"15min":1.004},"last":0.46300000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.80100000000000005,"15min":0.78200000000000003},"min":{"1min":0.68799999999999994,"5min":0.55600000000000005,"15min":0.47999999999999998},"max":{"1min":1.0289999999999999,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.5},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76800000000000002,"15min":0.73799999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.44400000000000001},"max":{"1min":0.93999999999999995,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.47499999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.76800000000000002},"min":{"1min":0.56399999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0129999999999999,"5min":1.05,"15min":1.05},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.78700000000000003,"15min":0.77900000000000003},"min":{"1min":0.61499999999999999,"5min":0.56100000000000005,"15min":0.50800000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.0329999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.623,"15min":0.58299999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.35399999999999998},"max":{"1min":0.90200000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65200000000000002,"15min":0.629},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.41599999999999998},"max":{"1min":0.98499999999999999,"5min":1.208,"15min":1.208},"last":0.67100000000000004}]},{"osd":6,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65600000000000003,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.435},"max":{"1min":0.90000000000000002,"5min":0.98099999999999998,"15min":0.98999999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68999999999999995,"15min":0.66100000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.40699999999999997},"max":{"1min":1,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72599999999999998,"15min":0.68999999999999995},"min":{"1min":0.439,"5min":0.436,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.169,"15min":1.169},"last":0.623},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.71399999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.018,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.59599999999999997}]}]},{"osd":4,"up_from":15,"seq":64424510545,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63772,"kb_used_data":2388,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308068,"statfs":{"total":96636764160,"available":96571461632,"internally_reserved":0,"allocated":2445312,"data_stored":1280680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.751,"15min":0.74199999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.35899999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.109},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.76100000000000001,"15min":0.755},"min":{"1min":0.621,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":1.4630000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.77000000000000002,"15min":0.76100000000000001},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96799999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.76500000000000001},"min":{"1min":0.51000000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.997,"15min":1.1020000000000001},"last":0.49099999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.79100000000000004,"15min":0.77200000000000002},"min":{"1min":0.63800000000000001,"5min":0.53800000000000003,"15min":0.47699999999999998},"max":{"1min":1.0289999999999999,"5min":1.0309999999999999,"15min":1.071},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.79100000000000004,"15min":0.77800000000000002},"min":{"1min":0.68300000000000005,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":1.0009999999999999,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.67600000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.76500000000000001,"15min":0.76200000000000001},"min":{"1min":0.60699999999999998,"5min":0.56000000000000005,"15min":0.501},"max":{"1min":0.95699999999999996,"5min":0.98499999999999999,"15min":1.7809999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80400000000000005,"15min":0.79500000000000004},"min":{"1min":0.65000000000000002,"5min":0.61399999999999999,"15min":0.55000000000000004},"max":{"1min":0.98399999999999999,"5min":1.0249999999999999,"15min":1.5680000000000001},"last":0.65400000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65400000000000003,"15min":0.624},"min":{"1min":0.51800000000000002,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.85099999999999998,"5min":0.95699999999999996,"15min":1.1020000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":1.042,"5min":1.042,"15min":3.508},"last":0.627}]},{"osd":6,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67700000000000005,"15min":0.65600000000000003},"min":{"1min":0.50700000000000001,"5min":0.437,"15min":0.40200000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.4099999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.72599999999999998,"15min":0.70999999999999996},"min":{"1min":0.58999999999999997,"5min":0.48499999999999999,"15min":0.45600000000000002},"max":{"1min":0.92600000000000005,"5min":0.96599999999999997,"15min":1.0760000000000001},"last":0.73099999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.72399999999999998,"15min":0.71499999999999997},"min":{"1min":0.53900000000000003,"5min":0.50800000000000001,"15min":0.41899999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.831},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76300000000000001,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.50600000000000001,"15min":0.46999999999999997},"max":{"1min":1.0229999999999999,"5min":1.036,"15min":1.0549999999999999},"last":0.746}]}]},{"osd":3,"up_from":15,"seq":64424510545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47376,"kb_used_data":2376,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324464,"statfs":{"total":96636764160,"available":96588251136,"internally_reserved":0,"allocated":2433024,"data_stored":1279080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.77100000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.32800000000000001},"max":{"1min":0.75700000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.38700000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49299999999999999,"15min":0.5},"min":{"1min":0.36099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.82199999999999995},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.42799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.89600000000000002},"last":0.67600000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.40799999999999997,"5min":0.40400000000000003,"15min":0.378},"max":{"1min":0.752,"5min":0.79600000000000004,"15min":0.89400000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59699999999999998,"15min":0.60799999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.40100000000000002},"max":{"1min":0.748,"5min":0.83899999999999997,"15min":0.91300000000000003},"last":0.45100000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82399999999999995,"15min":0.83599999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.024,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82699999999999996,"15min":0.83099999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.56599999999999995},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.69499999999999995}]},{"osd":5,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.84299999999999997,"15min":0.84999999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.57399999999999995},"max":{"1min":1.1120000000000001,"5min":1.2749999999999999,"15min":1.397},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.83399999999999996,"15min":0.83599999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55100000000000005},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.262},"last":0.81399999999999995}]},{"osd":6,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.84599999999999997,"15min":0.85299999999999998},"min":{"1min":0.63400000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1859999999999999,"5min":1.234,"15min":1.274},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.59799999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.165,"5min":1.2490000000000001,"15min":1.349},"last":0.68600000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91800000000000004,"15min":0.91200000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2330000000000001,"5min":1.3540000000000001,"15min":1.486},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90800000000000003,"15min":0.90500000000000003},"min":{"1min":0.72099999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3999999999999999},"last":0.79400000000000004}]}]},{"osd":2,"up_from":15,"seq":64424510544,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47376,"kb_used_data":2376,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324464,"statfs":{"total":96636764160,"available":96588251136,"internally_reserved":0,"allocated":2433024,"data_stored":1279136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.45600000000000002,"15min":0.45500000000000002},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.27100000000000002},"max":{"1min":0.64200000000000002,"5min":0.752,"15min":0.752},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.495}]},{"osd":1,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.622,"15min":0.629},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.38500000000000001},"max":{"1min":0.83799999999999997,"5min":0.89100000000000001,"15min":0.96399999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51700000000000002,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70199999999999996,"5min":0.746,"15min":0.86299999999999999},"last":0.30599999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60799999999999998,"15min":0.622},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":0.86199999999999999,"15min":0.95999999999999996},"last":0.435},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57899999999999996,"15min":0.58599999999999997},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77600000000000002,"5min":0.79700000000000004,"15min":0.875},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81799999999999995,"15min":0.80900000000000005},"min":{"1min":0.64200000000000002,"5min":0.56699999999999995,"15min":0.56499999999999995},"max":{"1min":1.0880000000000001,"5min":1.306,"15min":1.306},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78800000000000003,"15min":0.80100000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2070000000000001,"5min":1.272,"15min":1.466},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.81000000000000005},"min":{"1min":0.55700000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.1080000000000001,"5min":1.2050000000000001,"15min":1.3049999999999999},"last":0.75},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.83199999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.55000000000000004},"max":{"1min":1.0700000000000001,"5min":1.077,"15min":1.429},"last":0.86699999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.86599999999999999,"15min":0.85099999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.59699999999999998},"max":{"1min":1.044,"5min":1.365,"15min":1.4119999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81200000000000006,"15min":0.81399999999999995},"min":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.53500000000000003},"max":{"1min":1.117,"5min":1.2,"15min":1.2470000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.56000000000000005},"max":{"1min":1.159,"5min":1.393,"15min":1.45},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.876,"5min":0.871,"15min":0.872},"min":{"1min":0.69599999999999995,"5min":0.68400000000000005,"15min":0.61799999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.73699999999999999}]}]},{"osd":1,"up_from":15,"seq":64424510545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59664,"kb_used_data":2376,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312176,"statfs":{"total":96636764160,"available":96575668224,"internally_reserved":0,"allocated":2433024,"data_stored":1279080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46899999999999997,"15min":0.46400000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61199999999999999,"5min":0.77800000000000002,"15min":0.89400000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.38500000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82199999999999995,"5min":0.83999999999999997,"15min":0.89200000000000002},"last":0.40999999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69799999999999995,"5min":0.871,"15min":0.871},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.46800000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71899999999999997,"5min":0.86199999999999999,"15min":0.877},"last":0.44700000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.498,"5min":0.45800000000000002,"15min":0.442},"max":{"1min":0.79100000000000004,"5min":0.94499999999999995,"15min":0.99399999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77900000000000003,"5min":0.82699999999999996,"15min":0.86599999999999999},"last":0.61799999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80000000000000004,"15min":0.81699999999999995},"min":{"1min":0.51400000000000001,"5min":0.48599999999999999,"15min":0.47799999999999998},"max":{"1min":1.1319999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.95499999999999996}]},{"osd":5,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80500000000000005,"15min":0.80700000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.2629999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.875,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.57799999999999996},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.92900000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.80600000000000005,"15min":0.81899999999999995},"min":{"1min":0.54600000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2689999999999999},"last":0.875},{"interface":"front","average":{"1min":0.878,"5min":0.85199999999999998,"15min":0.85299999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.82699999999999996}]},{"osd":7,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.85199999999999998,"15min":0.85999999999999999},"min":{"1min":0.61699999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.1679999999999999,"5min":1.236,"15min":1.236},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.80500000000000005}]}]},{"osd":0,"up_from":14,"seq":60129543250,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60252,"kb_used_data":2964,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311588,"statfs":{"total":96636764160,"available":96575066112,"internally_reserved":0,"allocated":3035136,"data_stored":1876446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46100000000000002,"15min":0.46000000000000002},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64900000000000002,"5min":1.216,"15min":1.216},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52600000000000002,"15min":0.53100000000000003},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.73599999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.58999999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.52800000000000002,"15min":0.53500000000000003},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77000000000000002,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.38500000000000001},"max":{"1min":0.68500000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.64900000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69399999999999995,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81699999999999995,"5min":1.032,"15min":1.032},"last":0.68600000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81000000000000005,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.55200000000000005,"15min":0.496},"max":{"1min":1.0229999999999999,"5min":1.1359999999999999,"15min":1.381},"last":0.998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.622,"5min":0.58699999999999997,"15min":0.47799999999999998},"max":{"1min":1.137,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.88}]},{"osd":5,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.82099999999999995,"15min":0.82599999999999996},"min":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1180000000000001,"5min":1.216,"15min":1.3420000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.80900000000000005,"15min":0.82699999999999996},"min":{"1min":0.59899999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.087,"5min":1.087,"15min":1.288},"last":1.1020000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84299999999999997,"15min":0.84899999999999998},"min":{"1min":0.628,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1140000000000001,"5min":1.349,"15min":1.4019999999999999},"last":1.054},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83299999999999996,"15min":0.84299999999999997},"min":{"1min":0.59499999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.98099999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84499999999999997,"15min":0.85599999999999998},"min":{"1min":0.66400000000000003,"5min":0.64600000000000002,"15min":0.56599999999999995},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.415},"last":1.075},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85899999999999999,"15min":0.86399999999999999},"min":{"1min":0.65700000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0980000000000001,"5min":1.395,"15min":1.395},"last":0.94699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:14:42.680 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:14:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.864+0000 7f679888c640 1 Processor -- start 2023-12-19T18:14:42.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.864+0000 7f679888c640 1 -- start start 2023-12-19T18:14:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.864+0000 7f679888c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6794154220 0x7f679415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.864+0000 7f679888c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f679406b3d0 0x7f67940ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.864+0000 7f679888c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679406bc30 con 0x7f67940cec40 2023-12-19T18:14:42.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.864+0000 7f679888c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67940cf0d0 con 0x7f6794154220 2023-12-19T18:14:42.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.864+0000 7f679888c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67940cc6e0 con 0x7f679406b3d0 2023-12-19T18:14:42.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.864+0000 7f679259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6794154220 0x7f679415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.864+0000 7f6792d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f67940cec40 0x7f679415d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39926/0 (socket says 172.21.15.26:39926) 2023-12-19T18:14:42.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.864+0000 7f6792d9d640 1 -- 172.21.15.26:0/3094559012 learned_addr learned my addr 172.21.15.26:0/3094559012 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.868+0000 7f6791d9b640 1 --2- 172.21.15.26:0/3094559012 >> v2:172.21.15.160:3300/0 conn(0x7f679406b3d0 0x7f67940ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.868+0000 7f679259c640 1 -- 172.21.15.26:0/3094559012 >> v2:172.21.15.160:3300/0 conn(0x7f679406b3d0 msgr2=0x7f67940ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.868+0000 7f679259c640 1 --2- 172.21.15.26:0/3094559012 >> v2:172.21.15.160:3300/0 conn(0x7f679406b3d0 0x7f67940ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.868+0000 7f679259c640 1 -- 172.21.15.26:0/3094559012 >> v1:172.21.15.26:6789/0 conn(0x7f67940cec40 legacy=0x7f679415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.868+0000 7f679259c640 1 -- 172.21.15.26:0/3094559012 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679415afc0 con 0x7f6794154220 2023-12-19T18:14:42.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.868+0000 7f679159a640 1 -- 172.21.15.26:0/3094559012 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098445153 0 0) 0x7f679406bc30 con 0x7f67940cec40 2023-12-19T18:14:42.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.868+0000 7f679259c640 1 --2- 172.21.15.26:0/3094559012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6794154220 0x7f679415aa80 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f67940cf880 tx=0x7f678401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115924812172e3bc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.868+0000 7f679159a640 1 -- 172.21.15.26:0/3094559012 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784004230 con 0x7f6794154220 2023-12-19T18:14:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.868+0000 7f679159a640 1 -- 172.21.15.26:0/3094559012 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67840a7050 con 0x7f6794154220 2023-12-19T18:14:42.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.868+0000 7f679159a640 1 -- 172.21.15.26:0/3094559012 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784003620 con 0x7f6794154220 2023-12-19T18:14:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.868+0000 7f679888c640 1 -- 172.21.15.26:0/3094559012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6794154220 msgr2=0x7f679415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.868+0000 7f679888c640 1 --2- 172.21.15.26:0/3094559012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6794154220 0x7f679415aa80 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f67940cf880 tx=0x7f678401c430 comp rx=0 tx=0).stop 2023-12-19T18:14:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 -- 172.21.15.26:0/3094559012 shutdown_connections 2023-12-19T18:14:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 --2- 172.21.15.26:0/3094559012 >> v2:172.21.15.160:3300/0 conn(0x7f679406b3d0 0x7f67940ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 --2- 172.21.15.26:0/3094559012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6794154220 0x7f679415aa80 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 -- 172.21.15.26:0/3094559012 >> 172.21.15.26:0/3094559012 conn(0x7f67940658e0 msgr2=0x7f6794065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 -- 172.21.15.26:0/3094559012 shutdown_connections 2023-12-19T18:14:42.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 -- 172.21.15.26:0/3094559012 wait complete. 2023-12-19T18:14:42.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 Processor -- start 2023-12-19T18:14:42.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 -- start start 2023-12-19T18:14:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f679406b3d0 0x7f67940b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67940cec40 0x7f6794070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6794154220 0x7f67941f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6794156590 con 0x7f6794154220 2023-12-19T18:14:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6794155980 con 0x7f67940cec40 2023-12-19T18:14:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679888c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6794155c60 con 0x7f679406b3d0 2023-12-19T18:14:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f6792d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6794154220 0x7f67941f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f6792d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6794154220 0x7f67941f87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34032/0 (socket says 172.21.15.26:34032) 2023-12-19T18:14:42.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f6792d9d640 1 -- 172.21.15.26:0/4606677 learned_addr learned my addr 172.21.15.26:0/4606677 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f6791d9b640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67940cec40 0x7f6794070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f679259c640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f679406b3d0 0x7f67940b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f6792d9d640 1 -- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f679406b3d0 msgr2=0x7f67940b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f6792d9d640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f679406b3d0 0x7f67940b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f6792d9d640 1 -- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67940cec40 msgr2=0x7f6794070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f6792d9d640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67940cec40 0x7f6794070620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.872+0000 7f6792d9d640 1 -- 172.21.15.26:0/4606677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6784009420 con 0x7f6794154220 2023-12-19T18:14:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.876+0000 7f6792d9d640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6794154220 0x7f67941f87b0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c700 tx=0x7f678c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.876+0000 7f6782ffd640 1 -- 172.21.15.26:0/4606677 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c00ce10 con 0x7f6794154220 2023-12-19T18:14:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.876+0000 7f6782ffd640 1 -- 172.21.15.26:0/4606677 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678c00a640 con 0x7f6794154220 2023-12-19T18:14:42.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.876+0000 7f6782ffd640 1 -- 172.21.15.26:0/4606677 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678c00aa70 con 0x7f6794154220 2023-12-19T18:14:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.876+0000 7f679888c640 1 -- 172.21.15.26:0/4606677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6794155f40 con 0x7f6794154220 2023-12-19T18:14:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.876+0000 7f679888c640 1 -- 172.21.15.26:0/4606677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679415cca0 con 0x7f6794154220 2023-12-19T18:14:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.876+0000 7f679888c640 1 -- 172.21.15.26:0/4606677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6794156740 con 0x7f6794154220 2023-12-19T18:14:42.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.876+0000 7f6782ffd640 1 -- 172.21.15.26:0/4606677 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f678c00abd0 con 0x7f6794154220 2023-12-19T18:14:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.876+0000 7f6782ffd640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6774065b80 0x7f6774068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.876+0000 7f6782ffd640 1 -- 172.21.15.26:0/4606677 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f678c082a20 con 0x7f6794154220 2023-12-19T18:14:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.876+0000 7f679259c640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6774065b80 0x7f6774068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:42.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.880+0000 7f679259c640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6774065b80 0x7f6774068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f67840ad000 tx=0x7f67840a6000 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:42.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:42.880+0000 7f6782ffd640 1 -- 172.21.15.26:0/4606677 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6794156740 con 0x7f6794154220 2023-12-19T18:14:43.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.036+0000 7f679888c640 1 -- 172.21.15.26:0/4606677 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f67940ccef0 con 0x7f6774065b80 2023-12-19T18:14:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.044+0000 7f6782ffd640 1 -- 172.21.15.26:0/4606677 <== mgr.8276 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289459 (secure 0 0 0) 0x7f67840ba070 con 0x7f6774065b80 2023-12-19T18:14:43.046 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:14:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.048+0000 7f679888c640 1 -- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6774065b80 msgr2=0x7f6774068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.048+0000 7f679888c640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6774065b80 0x7f6774068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f67840ad000 tx=0x7f67840a6000 comp rx=0 tx=0).stop 2023-12-19T18:14:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.048+0000 7f679888c640 1 -- 172.21.15.26:0/4606677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6794154220 msgr2=0x7f67941f87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.048+0000 7f679888c640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6794154220 0x7f67941f87b0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f678c00c700 tx=0x7f678c00cbc0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.048+0000 7f679888c640 1 -- 172.21.15.26:0/4606677 shutdown_connections 2023-12-19T18:14:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.048+0000 7f679888c640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6774065b80 0x7f6774068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.048+0000 7f679888c640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6794154220 0x7f67941f87b0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.048+0000 7f679888c640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67940cec40 0x7f6794070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.048+0000 7f679888c640 1 --2- 172.21.15.26:0/4606677 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f679406b3d0 0x7f67940b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.048+0000 7f679888c640 1 -- 172.21.15.26:0/4606677 >> 172.21.15.26:0/4606677 conn(0x7f67940658e0 msgr2=0x7f67940cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.048+0000 7f679888c640 1 -- 172.21.15.26:0/4606677 shutdown_connections 2023-12-19T18:14:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.048+0000 7f679888c640 1 -- 172.21.15.26:0/4606677 wait complete. 2023-12-19T18:14:43.067 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-19T18:14:42.001534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1922,"num_read_kb":3999,"num_write":579,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":362,"ondisk_log_size":362,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":431808,"kb_used_data":20224,"kb_used_omap":0,"kb_used_meta":411520,"kb_avail":754542912,"statfs":{"total":773094113280,"available":772651941888,"internally_reserved":0,"allocated":20709376,"data_stored":11431892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":421396480},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.885791"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.563789+0000","last_change":"2023-12-19T18:14:38.552280+0000","last_active":"2023-12-19T18:14:38.563789+0000","last_peered":"2023-12-19T18:14:38.563789+0000","last_clean":"2023-12-19T18:14:38.563789+0000","last_became_active":"2023-12-19T18:14:38.552022+0000","last_became_peered":"2023-12-19T18:14:38.552022+0000","last_unstale":"2023-12-19T18:14:38.563789+0000","last_undegraded":"2023-12-19T18:14:38.563789+0000","last_fullsized":"2023-12-19T18:14:38.563789+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:13:42.870515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536987+0000","last_change":"2023-12-19T18:14:38.557909+0000","last_active":"2023-12-19T18:14:39.536987+0000","last_peered":"2023-12-19T18:14:39.536987+0000","last_clean":"2023-12-19T18:14:39.536987+0000","last_became_active":"2023-12-19T18:14:38.557650+0000","last_became_peered":"2023-12-19T18:14:38.557650+0000","last_unstale":"2023-12-19T18:14:39.536987+0000","last_undegraded":"2023-12-19T18:14:39.536987+0000","last_fullsized":"2023-12-19T18:14:39.536987+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:32:04.959408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537641+0000","last_change":"2023-12-19T18:14:38.543248+0000","last_active":"2023-12-19T18:14:39.537641+0000","last_peered":"2023-12-19T18:14:39.537641+0000","last_clean":"2023-12-19T18:14:39.537641+0000","last_became_active":"2023-12-19T18:14:38.542894+0000","last_became_peered":"2023-12-19T18:14:38.542894+0000","last_unstale":"2023-12-19T18:14:39.537641+0000","last_undegraded":"2023-12-19T18:14:39.537641+0000","last_fullsized":"2023-12-19T18:14:39.537641+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:06.859307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.548920+0000","last_change":"2023-12-19T18:14:38.547197+0000","last_active":"2023-12-19T18:14:38.548920+0000","last_peered":"2023-12-19T18:14:38.548920+0000","last_clean":"2023-12-19T18:14:38.548920+0000","last_became_active":"2023-12-19T18:14:38.546903+0000","last_became_peered":"2023-12-19T18:14:38.546903+0000","last_unstale":"2023-12-19T18:14:38.548920+0000","last_undegraded":"2023-12-19T18:14:38.548920+0000","last_fullsized":"2023-12-19T18:14:38.548920+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:03.792973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536484+0000","last_change":"2023-12-19T18:14:38.551345+0000","last_active":"2023-12-19T18:14:39.536484+0000","last_peered":"2023-12-19T18:14:39.536484+0000","last_clean":"2023-12-19T18:14:39.536484+0000","last_became_active":"2023-12-19T18:14:38.551084+0000","last_became_peered":"2023-12-19T18:14:38.551084+0000","last_unstale":"2023-12-19T18:14:39.536484+0000","last_undegraded":"2023-12-19T18:14:39.536484+0000","last_fullsized":"2023-12-19T18:14:39.536484+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:53.439000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660018+0000","last_change":"2023-12-19T18:14:38.567542+0000","last_active":"2023-12-19T18:14:39.660018+0000","last_peered":"2023-12-19T18:14:39.660018+0000","last_clean":"2023-12-19T18:14:39.660018+0000","last_became_active":"2023-12-19T18:14:38.567279+0000","last_became_peered":"2023-12-19T18:14:38.567279+0000","last_unstale":"2023-12-19T18:14:39.660018+0000","last_undegraded":"2023-12-19T18:14:39.660018+0000","last_fullsized":"2023-12-19T18:14:39.660018+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:04.274917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535127+0000","last_change":"2023-12-19T18:14:38.566538+0000","last_active":"2023-12-19T18:14:39.535127+0000","last_peered":"2023-12-19T18:14:39.535127+0000","last_clean":"2023-12-19T18:14:39.535127+0000","last_became_active":"2023-12-19T18:14:38.566286+0000","last_became_peered":"2023-12-19T18:14:38.566286+0000","last_unstale":"2023-12-19T18:14:39.535127+0000","last_undegraded":"2023-12-19T18:14:39.535127+0000","last_fullsized":"2023-12-19T18:14:39.535127+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:20:08.417607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536785+0000","last_change":"2023-12-19T18:14:38.549470+0000","last_active":"2023-12-19T18:14:39.536785+0000","last_peered":"2023-12-19T18:14:39.536785+0000","last_clean":"2023-12-19T18:14:39.536785+0000","last_became_active":"2023-12-19T18:14:38.549187+0000","last_became_peered":"2023-12-19T18:14:38.549187+0000","last_unstale":"2023-12-19T18:14:39.536785+0000","last_undegraded":"2023-12-19T18:14:39.536785+0000","last_fullsized":"2023-12-19T18:14:39.536785+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:04:33.703685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537162+0000","last_change":"2023-12-19T18:14:38.556753+0000","last_active":"2023-12-19T18:14:39.537162+0000","last_peered":"2023-12-19T18:14:39.537162+0000","last_clean":"2023-12-19T18:14:39.537162+0000","last_became_active":"2023-12-19T18:14:38.554373+0000","last_became_peered":"2023-12-19T18:14:38.554373+0000","last_unstale":"2023-12-19T18:14:39.537162+0000","last_undegraded":"2023-12-19T18:14:39.537162+0000","last_fullsized":"2023-12-19T18:14:39.537162+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:45.976373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537033+0000","last_change":"2023-12-19T18:14:38.560035+0000","last_active":"2023-12-19T18:14:39.537033+0000","last_peered":"2023-12-19T18:14:39.537033+0000","last_clean":"2023-12-19T18:14:39.537033+0000","last_became_active":"2023-12-19T18:14:38.559779+0000","last_became_peered":"2023-12-19T18:14:38.559779+0000","last_unstale":"2023-12-19T18:14:39.537033+0000","last_undegraded":"2023-12-19T18:14:39.537033+0000","last_fullsized":"2023-12-19T18:14:39.537033+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:59:30.883370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536693+0000","last_change":"2023-12-19T18:14:38.553655+0000","last_active":"2023-12-19T18:14:39.536693+0000","last_peered":"2023-12-19T18:14:39.536693+0000","last_clean":"2023-12-19T18:14:39.536693+0000","last_became_active":"2023-12-19T18:14:38.553368+0000","last_became_peered":"2023-12-19T18:14:38.553368+0000","last_unstale":"2023-12-19T18:14:39.536693+0000","last_undegraded":"2023-12-19T18:14:39.536693+0000","last_fullsized":"2023-12-19T18:14:39.536693+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:57:45.606591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536572+0000","last_change":"2023-12-19T18:14:38.553042+0000","last_active":"2023-12-19T18:14:39.536572+0000","last_peered":"2023-12-19T18:14:39.536572+0000","last_clean":"2023-12-19T18:14:39.536572+0000","last_became_active":"2023-12-19T18:14:38.552731+0000","last_became_peered":"2023-12-19T18:14:38.552731+0000","last_unstale":"2023-12-19T18:14:39.536572+0000","last_undegraded":"2023-12-19T18:14:39.536572+0000","last_fullsized":"2023-12-19T18:14:39.536572+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:07.574774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.538011+0000","last_change":"2023-12-19T18:14:38.557664+0000","last_active":"2023-12-19T18:14:39.538011+0000","last_peered":"2023-12-19T18:14:39.538011+0000","last_clean":"2023-12-19T18:14:39.538011+0000","last_became_active":"2023-12-19T18:14:38.556636+0000","last_became_peered":"2023-12-19T18:14:38.556636+0000","last_unstale":"2023-12-19T18:14:39.538011+0000","last_undegraded":"2023-12-19T18:14:39.538011+0000","last_fullsized":"2023-12-19T18:14:39.538011+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:21:32.182507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537469+0000","last_change":"2023-12-19T18:14:38.549523+0000","last_active":"2023-12-19T18:14:39.537469+0000","last_peered":"2023-12-19T18:14:39.537469+0000","last_clean":"2023-12-19T18:14:39.537469+0000","last_became_active":"2023-12-19T18:14:38.549216+0000","last_became_peered":"2023-12-19T18:14:38.549216+0000","last_unstale":"2023-12-19T18:14:39.537469+0000","last_undegraded":"2023-12-19T18:14:39.537469+0000","last_fullsized":"2023-12-19T18:14:39.537469+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:09:58.131056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.561224+0000","last_change":"2023-12-19T18:14:38.556426+0000","last_active":"2023-12-19T18:14:38.561224+0000","last_peered":"2023-12-19T18:14:38.561224+0000","last_clean":"2023-12-19T18:14:38.561224+0000","last_became_active":"2023-12-19T18:14:38.556173+0000","last_became_peered":"2023-12-19T18:14:38.556173+0000","last_unstale":"2023-12-19T18:14:38.561224+0000","last_undegraded":"2023-12-19T18:14:38.561224+0000","last_fullsized":"2023-12-19T18:14:38.561224+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:33:03.044215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.534937+0000","last_change":"2023-12-19T18:14:38.550324+0000","last_active":"2023-12-19T18:14:39.534937+0000","last_peered":"2023-12-19T18:14:39.534937+0000","last_clean":"2023-12-19T18:14:39.534937+0000","last_became_active":"2023-12-19T18:14:38.550070+0000","last_became_peered":"2023-12-19T18:14:38.550070+0000","last_unstale":"2023-12-19T18:14:39.534937+0000","last_undegraded":"2023-12-19T18:14:39.534937+0000","last_fullsized":"2023-12-19T18:14:39.534937+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:27:05.940012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660220+0000","last_change":"2023-12-19T18:14:38.557675+0000","last_active":"2023-12-19T18:14:39.660220+0000","last_peered":"2023-12-19T18:14:39.660220+0000","last_clean":"2023-12-19T18:14:39.660220+0000","last_became_active":"2023-12-19T18:14:38.557360+0000","last_became_peered":"2023-12-19T18:14:38.557360+0000","last_unstale":"2023-12-19T18:14:39.660220+0000","last_undegraded":"2023-12-19T18:14:39.660220+0000","last_fullsized":"2023-12-19T18:14:39.660220+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:11:27.767680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.560874+0000","last_change":"2023-12-19T18:14:38.556251+0000","last_active":"2023-12-19T18:14:38.560874+0000","last_peered":"2023-12-19T18:14:38.560874+0000","last_clean":"2023-12-19T18:14:38.560874+0000","last_became_active":"2023-12-19T18:14:38.555987+0000","last_became_peered":"2023-12-19T18:14:38.555987+0000","last_unstale":"2023-12-19T18:14:38.560874+0000","last_undegraded":"2023-12-19T18:14:38.560874+0000","last_fullsized":"2023-12-19T18:14:38.560874+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:10.230253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660208+0000","last_change":"2023-12-19T18:14:38.551632+0000","last_active":"2023-12-19T18:14:39.660208+0000","last_peered":"2023-12-19T18:14:39.660208+0000","last_clean":"2023-12-19T18:14:39.660208+0000","last_became_active":"2023-12-19T18:14:38.551239+0000","last_became_peered":"2023-12-19T18:14:38.551239+0000","last_unstale":"2023-12-19T18:14:39.660208+0000","last_undegraded":"2023-12-19T18:14:39.660208+0000","last_fullsized":"2023-12-19T18:14:39.660208+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:22.433655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537521+0000","last_change":"2023-12-19T18:14:38.545883+0000","last_active":"2023-12-19T18:14:39.537521+0000","last_peered":"2023-12-19T18:14:39.537521+0000","last_clean":"2023-12-19T18:14:39.537521+0000","last_became_active":"2023-12-19T18:14:38.544972+0000","last_became_peered":"2023-12-19T18:14:38.544972+0000","last_unstale":"2023-12-19T18:14:39.537521+0000","last_undegraded":"2023-12-19T18:14:39.537521+0000","last_fullsized":"2023-12-19T18:14:39.537521+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:14.639095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.659996+0000","last_change":"2023-12-19T18:14:38.553843+0000","last_active":"2023-12-19T18:14:39.659996+0000","last_peered":"2023-12-19T18:14:39.659996+0000","last_clean":"2023-12-19T18:14:39.659996+0000","last_became_active":"2023-12-19T18:14:38.553610+0000","last_became_peered":"2023-12-19T18:14:38.553610+0000","last_unstale":"2023-12-19T18:14:39.659996+0000","last_undegraded":"2023-12-19T18:14:39.659996+0000","last_fullsized":"2023-12-19T18:14:39.659996+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:31.488536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536586+0000","last_change":"2023-12-19T18:14:38.564751+0000","last_active":"2023-12-19T18:14:39.536586+0000","last_peered":"2023-12-19T18:14:39.536586+0000","last_clean":"2023-12-19T18:14:39.536586+0000","last_became_active":"2023-12-19T18:14:38.564381+0000","last_became_peered":"2023-12-19T18:14:38.564381+0000","last_unstale":"2023-12-19T18:14:39.536586+0000","last_undegraded":"2023-12-19T18:14:39.536586+0000","last_fullsized":"2023-12-19T18:14:39.536586+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:09.082505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.659911+0000","last_change":"2023-12-19T18:14:38.553142+0000","last_active":"2023-12-19T18:14:39.659911+0000","last_peered":"2023-12-19T18:14:39.659911+0000","last_clean":"2023-12-19T18:14:39.659911+0000","last_became_active":"2023-12-19T18:14:38.552850+0000","last_became_peered":"2023-12-19T18:14:38.552850+0000","last_unstale":"2023-12-19T18:14:39.659911+0000","last_undegraded":"2023-12-19T18:14:39.659911+0000","last_fullsized":"2023-12-19T18:14:39.659911+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:01:07.428157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536917+0000","last_change":"2023-12-19T18:14:38.553525+0000","last_active":"2023-12-19T18:14:39.536917+0000","last_peered":"2023-12-19T18:14:39.536917+0000","last_clean":"2023-12-19T18:14:39.536917+0000","last_became_active":"2023-12-19T18:14:38.553231+0000","last_became_peered":"2023-12-19T18:14:38.553231+0000","last_unstale":"2023-12-19T18:14:39.536917+0000","last_undegraded":"2023-12-19T18:14:39.536917+0000","last_fullsized":"2023-12-19T18:14:39.536917+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:40:50.277877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537959+0000","last_change":"2023-12-19T18:14:38.541116+0000","last_active":"2023-12-19T18:14:39.537959+0000","last_peered":"2023-12-19T18:14:39.537959+0000","last_clean":"2023-12-19T18:14:39.537959+0000","last_became_active":"2023-12-19T18:14:38.540825+0000","last_became_peered":"2023-12-19T18:14:38.540825+0000","last_unstale":"2023-12-19T18:14:39.537959+0000","last_undegraded":"2023-12-19T18:14:39.537959+0000","last_fullsized":"2023-12-19T18:14:39.537959+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:14:53.761212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535222+0000","last_change":"2023-12-19T18:14:38.567880+0000","last_active":"2023-12-19T18:14:39.535222+0000","last_peered":"2023-12-19T18:14:39.535222+0000","last_clean":"2023-12-19T18:14:39.535222+0000","last_became_active":"2023-12-19T18:14:38.567393+0000","last_became_peered":"2023-12-19T18:14:38.567393+0000","last_unstale":"2023-12-19T18:14:39.535222+0000","last_undegraded":"2023-12-19T18:14:39.535222+0000","last_fullsized":"2023-12-19T18:14:39.535222+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:41.971560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537750+0000","last_change":"2023-12-19T18:14:38.544254+0000","last_active":"2023-12-19T18:14:39.537750+0000","last_peered":"2023-12-19T18:14:39.537750+0000","last_clean":"2023-12-19T18:14:39.537750+0000","last_became_active":"2023-12-19T18:14:38.543980+0000","last_became_peered":"2023-12-19T18:14:38.543980+0000","last_unstale":"2023-12-19T18:14:39.537750+0000","last_undegraded":"2023-12-19T18:14:39.537750+0000","last_fullsized":"2023-12-19T18:14:39.537750+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:17:53.367376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.543078+0000","last_change":"2023-12-19T18:14:38.539808+0000","last_active":"2023-12-19T18:14:38.543078+0000","last_peered":"2023-12-19T18:14:38.543078+0000","last_clean":"2023-12-19T18:14:38.543078+0000","last_became_active":"2023-12-19T18:14:38.539528+0000","last_became_peered":"2023-12-19T18:14:38.539528+0000","last_unstale":"2023-12-19T18:14:38.543078+0000","last_undegraded":"2023-12-19T18:14:38.543078+0000","last_fullsized":"2023-12-19T18:14:38.543078+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:42:55.210047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.538041+0000","last_change":"2023-12-19T18:14:38.543688+0000","last_active":"2023-12-19T18:14:39.538041+0000","last_peered":"2023-12-19T18:14:39.538041+0000","last_clean":"2023-12-19T18:14:39.538041+0000","last_became_active":"2023-12-19T18:14:38.543404+0000","last_became_peered":"2023-12-19T18:14:38.543404+0000","last_unstale":"2023-12-19T18:14:39.538041+0000","last_undegraded":"2023-12-19T18:14:39.538041+0000","last_fullsized":"2023-12-19T18:14:39.538041+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:30:37.743915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.548538+0000","last_change":"2023-12-19T18:14:38.545547+0000","last_active":"2023-12-19T18:14:38.548538+0000","last_peered":"2023-12-19T18:14:38.548538+0000","last_clean":"2023-12-19T18:14:38.548538+0000","last_became_active":"2023-12-19T18:14:38.545264+0000","last_became_peered":"2023-12-19T18:14:38.545264+0000","last_unstale":"2023-12-19T18:14:38.548538+0000","last_undegraded":"2023-12-19T18:14:38.548538+0000","last_fullsized":"2023-12-19T18:14:38.548538+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:03:43.536300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.565736+0000","last_change":"2023-12-19T18:14:38.556112+0000","last_active":"2023-12-19T18:14:38.565736+0000","last_peered":"2023-12-19T18:14:38.565736+0000","last_clean":"2023-12-19T18:14:38.565736+0000","last_became_active":"2023-12-19T18:14:38.555810+0000","last_became_peered":"2023-12-19T18:14:38.555810+0000","last_unstale":"2023-12-19T18:14:38.565736+0000","last_undegraded":"2023-12-19T18:14:38.565736+0000","last_fullsized":"2023-12-19T18:14:38.565736+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:46:44.414862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.561144+0000","last_change":"2023-12-19T18:14:38.553204+0000","last_active":"2023-12-19T18:14:38.561144+0000","last_peered":"2023-12-19T18:14:38.561144+0000","last_clean":"2023-12-19T18:14:38.561144+0000","last_became_active":"2023-12-19T18:14:38.546596+0000","last_became_peered":"2023-12-19T18:14:38.546596+0000","last_unstale":"2023-12-19T18:14:38.561144+0000","last_undegraded":"2023-12-19T18:14:38.561144+0000","last_fullsized":"2023-12-19T18:14:38.561144+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:44:54.211011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535237+0000","last_change":"2023-12-19T18:14:38.563385+0000","last_active":"2023-12-19T18:14:39.535237+0000","last_peered":"2023-12-19T18:14:39.535237+0000","last_clean":"2023-12-19T18:14:39.535237+0000","last_became_active":"2023-12-19T18:14:38.562599+0000","last_became_peered":"2023-12-19T18:14:38.562599+0000","last_unstale":"2023-12-19T18:14:39.535237+0000","last_undegraded":"2023-12-19T18:14:39.535237+0000","last_fullsized":"2023-12-19T18:14:39.535237+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:02:32.672622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"195'4","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.566582+0000","last_change":"2023-12-19T18:14:38.152178+0000","last_active":"2023-12-19T18:14:38.566582+0000","last_peered":"2023-12-19T18:14:38.566582+0000","last_clean":"2023-12-19T18:14:38.566582+0000","last_became_active":"2023-12-19T18:14:38.151843+0000","last_became_peered":"2023-12-19T18:14:38.151843+0000","last_unstale":"2023-12-19T18:14:38.566582+0000","last_undegraded":"2023-12-19T18:14:38.566582+0000","last_fullsized":"2023-12-19T18:14:38.566582+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:53.356036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.546619+0000","last_change":"2023-12-19T18:14:38.543456+0000","last_active":"2023-12-19T18:14:38.546619+0000","last_peered":"2023-12-19T18:14:38.546619+0000","last_clean":"2023-12-19T18:14:38.546619+0000","last_became_active":"2023-12-19T18:14:38.543164+0000","last_became_peered":"2023-12-19T18:14:38.543164+0000","last_unstale":"2023-12-19T18:14:38.546619+0000","last_undegraded":"2023-12-19T18:14:38.546619+0000","last_fullsized":"2023-12-19T18:14:38.546619+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:35:33.925360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"195'1","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.565592+0000","last_change":"2023-12-19T18:14:38.153100+0000","last_active":"2023-12-19T18:14:38.565592+0000","last_peered":"2023-12-19T18:14:38.565592+0000","last_clean":"2023-12-19T18:14:38.565592+0000","last_became_active":"2023-12-19T18:14:38.152624+0000","last_became_peered":"2023-12-19T18:14:38.152624+0000","last_unstale":"2023-12-19T18:14:38.565592+0000","last_undegraded":"2023-12-19T18:14:38.565592+0000","last_fullsized":"2023-12-19T18:14:38.565592+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:59:29.874589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660094+0000","last_change":"2023-12-19T18:14:38.546775+0000","last_active":"2023-12-19T18:14:39.660094+0000","last_peered":"2023-12-19T18:14:39.660094+0000","last_clean":"2023-12-19T18:14:39.660094+0000","last_became_active":"2023-12-19T18:14:38.546500+0000","last_became_peered":"2023-12-19T18:14:38.546500+0000","last_unstale":"2023-12-19T18:14:39.660094+0000","last_undegraded":"2023-12-19T18:14:39.660094+0000","last_fullsized":"2023-12-19T18:14:39.660094+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:03:18.457399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"195'2","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.560672+0000","last_change":"2023-12-19T18:14:38.154428+0000","last_active":"2023-12-19T18:14:38.560672+0000","last_peered":"2023-12-19T18:14:38.560672+0000","last_clean":"2023-12-19T18:14:38.560672+0000","last_became_active":"2023-12-19T18:14:38.153988+0000","last_became_peered":"2023-12-19T18:14:38.153988+0000","last_unstale":"2023-12-19T18:14:38.560672+0000","last_undegraded":"2023-12-19T18:14:38.560672+0000","last_fullsized":"2023-12-19T18:14:38.560672+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:13:41.861734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.538078+0000","last_change":"2023-12-19T18:14:38.546776+0000","last_active":"2023-12-19T18:14:39.538078+0000","last_peered":"2023-12-19T18:14:39.538078+0000","last_clean":"2023-12-19T18:14:39.538078+0000","last_became_active":"2023-12-19T18:14:38.546503+0000","last_became_peered":"2023-12-19T18:14:38.546503+0000","last_unstale":"2023-12-19T18:14:39.538078+0000","last_undegraded":"2023-12-19T18:14:39.538078+0000","last_fullsized":"2023-12-19T18:14:39.538078+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:14.806434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"195'2","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.563449+0000","last_change":"2023-12-19T18:14:38.145182+0000","last_active":"2023-12-19T18:14:38.563449+0000","last_peered":"2023-12-19T18:14:38.563449+0000","last_clean":"2023-12-19T18:14:38.563449+0000","last_became_active":"2023-12-19T18:14:38.144875+0000","last_became_peered":"2023-12-19T18:14:38.144875+0000","last_unstale":"2023-12-19T18:14:38.563449+0000","last_undegraded":"2023-12-19T18:14:38.563449+0000","last_fullsized":"2023-12-19T18:14:38.563449+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:31:54.174071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535425+0000","last_change":"2023-12-19T18:14:38.564692+0000","last_active":"2023-12-19T18:14:39.535425+0000","last_peered":"2023-12-19T18:14:39.535425+0000","last_clean":"2023-12-19T18:14:39.535425+0000","last_became_active":"2023-12-19T18:14:38.564428+0000","last_became_peered":"2023-12-19T18:14:38.564428+0000","last_unstale":"2023-12-19T18:14:39.535425+0000","last_undegraded":"2023-12-19T18:14:39.535425+0000","last_fullsized":"2023-12-19T18:14:39.535425+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:24.598857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"195'1","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.538089+0000","last_change":"2023-12-19T18:14:38.150995+0000","last_active":"2023-12-19T18:14:39.538089+0000","last_peered":"2023-12-19T18:14:39.538089+0000","last_clean":"2023-12-19T18:14:39.538089+0000","last_became_active":"2023-12-19T18:14:38.150325+0000","last_became_peered":"2023-12-19T18:14:38.150325+0000","last_unstale":"2023-12-19T18:14:39.538089+0000","last_undegraded":"2023-12-19T18:14:39.538089+0000","last_fullsized":"2023-12-19T18:14:39.538089+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:13.415745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.553802+0000","last_change":"2023-12-19T18:14:38.549299+0000","last_active":"2023-12-19T18:14:38.553802+0000","last_peered":"2023-12-19T18:14:38.553802+0000","last_clean":"2023-12-19T18:14:38.553802+0000","last_became_active":"2023-12-19T18:14:38.549007+0000","last_became_peered":"2023-12-19T18:14:38.549007+0000","last_unstale":"2023-12-19T18:14:38.553802+0000","last_undegraded":"2023-12-19T18:14:38.553802+0000","last_fullsized":"2023-12-19T18:14:38.553802+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:05:57.229774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"195'1","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.562825+0000","last_change":"2023-12-19T18:14:38.146423+0000","last_active":"2023-12-19T18:14:38.562825+0000","last_peered":"2023-12-19T18:14:38.562825+0000","last_clean":"2023-12-19T18:14:38.562825+0000","last_became_active":"2023-12-19T18:14:38.146116+0000","last_became_peered":"2023-12-19T18:14:38.146116+0000","last_unstale":"2023-12-19T18:14:38.562825+0000","last_undegraded":"2023-12-19T18:14:38.562825+0000","last_fullsized":"2023-12-19T18:14:38.562825+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:53.356036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.553889+0000","last_change":"2023-12-19T18:14:38.549715+0000","last_active":"2023-12-19T18:14:38.553889+0000","last_peered":"2023-12-19T18:14:38.553889+0000","last_clean":"2023-12-19T18:14:38.553889+0000","last_became_active":"2023-12-19T18:14:38.549426+0000","last_became_peered":"2023-12-19T18:14:38.549426+0000","last_unstale":"2023-12-19T18:14:38.553889+0000","last_undegraded":"2023-12-19T18:14:38.553889+0000","last_fullsized":"2023-12-19T18:14:38.553889+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:04.030577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536612+0000","last_change":"2023-12-19T18:14:38.153816+0000","last_active":"2023-12-19T18:14:39.536612+0000","last_peered":"2023-12-19T18:14:39.536612+0000","last_clean":"2023-12-19T18:14:39.536612+0000","last_became_active":"2023-12-19T18:14:38.153423+0000","last_became_peered":"2023-12-19T18:14:38.153423+0000","last_unstale":"2023-12-19T18:14:39.536612+0000","last_undegraded":"2023-12-19T18:14:39.536612+0000","last_fullsized":"2023-12-19T18:14:39.536612+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:58:42.525810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660034+0000","last_change":"2023-12-19T18:14:38.564879+0000","last_active":"2023-12-19T18:14:39.660034+0000","last_peered":"2023-12-19T18:14:39.660034+0000","last_clean":"2023-12-19T18:14:39.660034+0000","last_became_active":"2023-12-19T18:14:38.564612+0000","last_became_peered":"2023-12-19T18:14:38.564612+0000","last_unstale":"2023-12-19T18:14:39.660034+0000","last_undegraded":"2023-12-19T18:14:39.660034+0000","last_fullsized":"2023-12-19T18:14:39.660034+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:24:29.807045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535031+0000","last_change":"2023-12-19T18:14:38.143601+0000","last_active":"2023-12-19T18:14:39.535031+0000","last_peered":"2023-12-19T18:14:39.535031+0000","last_clean":"2023-12-19T18:14:39.535031+0000","last_became_active":"2023-12-19T18:14:38.143321+0000","last_became_peered":"2023-12-19T18:14:38.143321+0000","last_unstale":"2023-12-19T18:14:39.535031+0000","last_undegraded":"2023-12-19T18:14:39.535031+0000","last_fullsized":"2023-12-19T18:14:39.535031+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:37:51.634247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.566385+0000","last_change":"2023-12-19T18:14:38.561096+0000","last_active":"2023-12-19T18:14:38.566385+0000","last_peered":"2023-12-19T18:14:38.566385+0000","last_clean":"2023-12-19T18:14:38.566385+0000","last_became_active":"2023-12-19T18:14:38.560801+0000","last_became_peered":"2023-12-19T18:14:38.560801+0000","last_unstale":"2023-12-19T18:14:38.566385+0000","last_undegraded":"2023-12-19T18:14:38.566385+0000","last_fullsized":"2023-12-19T18:14:38.566385+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:37:34.132929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660257+0000","last_change":"2023-12-19T18:14:38.153381+0000","last_active":"2023-12-19T18:14:39.660257+0000","last_peered":"2023-12-19T18:14:39.660257+0000","last_clean":"2023-12-19T18:14:39.660257+0000","last_became_active":"2023-12-19T18:14:38.152008+0000","last_became_peered":"2023-12-19T18:14:38.152008+0000","last_unstale":"2023-12-19T18:14:39.660257+0000","last_undegraded":"2023-12-19T18:14:39.660257+0000","last_fullsized":"2023-12-19T18:14:39.660257+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:54.330119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.565968+0000","last_change":"2023-12-19T18:14:38.559625+0000","last_active":"2023-12-19T18:14:38.565968+0000","last_peered":"2023-12-19T18:14:38.565968+0000","last_clean":"2023-12-19T18:14:38.565968+0000","last_became_active":"2023-12-19T18:14:38.559300+0000","last_became_peered":"2023-12-19T18:14:38.559300+0000","last_unstale":"2023-12-19T18:14:38.565968+0000","last_undegraded":"2023-12-19T18:14:38.565968+0000","last_fullsized":"2023-12-19T18:14:38.565968+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:05:57.229774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"195'3","reported_seq":14,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660209+0000","last_change":"2023-12-19T18:14:38.145023+0000","last_active":"2023-12-19T18:14:39.660209+0000","last_peered":"2023-12-19T18:14:39.660209+0000","last_clean":"2023-12-19T18:14:39.660209+0000","last_became_active":"2023-12-19T18:14:38.144728+0000","last_became_peered":"2023-12-19T18:14:38.144728+0000","last_unstale":"2023-12-19T18:14:39.660209+0000","last_undegraded":"2023-12-19T18:14:39.660209+0000","last_fullsized":"2023-12-19T18:14:39.660209+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:50.524202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.550640+0000","last_change":"2023-12-19T18:14:38.548265+0000","last_active":"2023-12-19T18:14:38.550640+0000","last_peered":"2023-12-19T18:14:38.550640+0000","last_clean":"2023-12-19T18:14:38.550640+0000","last_became_active":"2023-12-19T18:14:38.547907+0000","last_became_peered":"2023-12-19T18:14:38.547907+0000","last_unstale":"2023-12-19T18:14:38.550640+0000","last_undegraded":"2023-12-19T18:14:38.550640+0000","last_fullsized":"2023-12-19T18:14:38.550640+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:25.892977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"195'6","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.570244+0000","last_change":"2023-12-19T18:14:38.144796+0000","last_active":"2023-12-19T18:14:38.570244+0000","last_peered":"2023-12-19T18:14:38.570244+0000","last_clean":"2023-12-19T18:14:38.570244+0000","last_became_active":"2023-12-19T18:14:38.144531+0000","last_became_peered":"2023-12-19T18:14:38.144531+0000","last_unstale":"2023-12-19T18:14:38.570244+0000","last_undegraded":"2023-12-19T18:14:38.570244+0000","last_fullsized":"2023-12-19T18:14:38.570244+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:06.565993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.562342+0000","last_change":"2023-12-19T18:14:38.557099+0000","last_active":"2023-12-19T18:14:38.562342+0000","last_peered":"2023-12-19T18:14:38.562342+0000","last_clean":"2023-12-19T18:14:38.562342+0000","last_became_active":"2023-12-19T18:14:38.556812+0000","last_became_peered":"2023-12-19T18:14:38.556812+0000","last_unstale":"2023-12-19T18:14:38.562342+0000","last_undegraded":"2023-12-19T18:14:38.562342+0000","last_fullsized":"2023-12-19T18:14:38.562342+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:31:33.954931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"195'2","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.566486+0000","last_change":"2023-12-19T18:14:38.150017+0000","last_active":"2023-12-19T18:14:38.566486+0000","last_peered":"2023-12-19T18:14:38.566486+0000","last_clean":"2023-12-19T18:14:38.566486+0000","last_became_active":"2023-12-19T18:14:38.149695+0000","last_became_peered":"2023-12-19T18:14:38.149695+0000","last_unstale":"2023-12-19T18:14:38.566486+0000","last_undegraded":"2023-12-19T18:14:38.566486+0000","last_fullsized":"2023-12-19T18:14:38.566486+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:14:59.693406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535187+0000","last_change":"2023-12-19T18:14:38.557555+0000","last_active":"2023-12-19T18:14:39.535187+0000","last_peered":"2023-12-19T18:14:39.535187+0000","last_clean":"2023-12-19T18:14:39.535187+0000","last_became_active":"2023-12-19T18:14:38.557292+0000","last_became_peered":"2023-12-19T18:14:38.557292+0000","last_unstale":"2023-12-19T18:14:39.535187+0000","last_undegraded":"2023-12-19T18:14:39.535187+0000","last_fullsized":"2023-12-19T18:14:39.535187+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:46.428487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.537606+0000","last_change":"2023-12-19T18:14:38.145593+0000","last_active":"2023-12-19T18:14:38.537606+0000","last_peered":"2023-12-19T18:14:38.537606+0000","last_clean":"2023-12-19T18:14:38.537606+0000","last_became_active":"2023-12-19T18:14:38.145328+0000","last_became_peered":"2023-12-19T18:14:38.145328+0000","last_unstale":"2023-12-19T18:14:38.537606+0000","last_undegraded":"2023-12-19T18:14:38.537606+0000","last_fullsized":"2023-12-19T18:14:38.537606+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:56:15.017376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.541978+0000","last_change":"2023-12-19T18:14:38.539859+0000","last_active":"2023-12-19T18:14:38.541978+0000","last_peered":"2023-12-19T18:14:38.541978+0000","last_clean":"2023-12-19T18:14:38.541978+0000","last_became_active":"2023-12-19T18:14:38.539539+0000","last_became_peered":"2023-12-19T18:14:38.539539+0000","last_unstale":"2023-12-19T18:14:38.541978+0000","last_undegraded":"2023-12-19T18:14:38.541978+0000","last_fullsized":"2023-12-19T18:14:38.541978+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:02:09.451847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.532910+0000","last_change":"2023-12-19T18:14:38.149760+0000","last_active":"2023-12-19T18:14:38.532910+0000","last_peered":"2023-12-19T18:14:38.532910+0000","last_clean":"2023-12-19T18:14:38.532910+0000","last_became_active":"2023-12-19T18:14:38.149441+0000","last_became_peered":"2023-12-19T18:14:38.149441+0000","last_unstale":"2023-12-19T18:14:38.532910+0000","last_undegraded":"2023-12-19T18:14:38.532910+0000","last_fullsized":"2023-12-19T18:14:38.532910+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:33:02.035434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537234+0000","last_change":"2023-12-19T18:14:38.561761+0000","last_active":"2023-12-19T18:14:39.537234+0000","last_peered":"2023-12-19T18:14:39.537234+0000","last_clean":"2023-12-19T18:14:39.537234+0000","last_became_active":"2023-12-19T18:14:38.561487+0000","last_became_peered":"2023-12-19T18:14:38.561487+0000","last_unstale":"2023-12-19T18:14:39.537234+0000","last_undegraded":"2023-12-19T18:14:39.537234+0000","last_fullsized":"2023-12-19T18:14:39.537234+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:32.366343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"195'8","reported_seq":18,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.567881+0000","last_change":"2023-12-19T18:14:38.139818+0000","last_active":"2023-12-19T18:14:38.567881+0000","last_peered":"2023-12-19T18:14:38.567881+0000","last_clean":"2023-12-19T18:14:38.567881+0000","last_became_active":"2023-12-19T18:14:38.139512+0000","last_became_peered":"2023-12-19T18:14:38.139512+0000","last_unstale":"2023-12-19T18:14:38.567881+0000","last_undegraded":"2023-12-19T18:14:38.567881+0000","last_fullsized":"2023-12-19T18:14:38.567881+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:32:03.950627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.561026+0000","last_change":"2023-12-19T18:14:38.552771+0000","last_active":"2023-12-19T18:14:38.561026+0000","last_peered":"2023-12-19T18:14:38.561026+0000","last_clean":"2023-12-19T18:14:38.561026+0000","last_became_active":"2023-12-19T18:14:38.552489+0000","last_became_peered":"2023-12-19T18:14:38.552489+0000","last_unstale":"2023-12-19T18:14:38.561026+0000","last_undegraded":"2023-12-19T18:14:38.561026+0000","last_fullsized":"2023-12-19T18:14:38.561026+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:30:38.872598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"195'1","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537962+0000","last_change":"2023-12-19T18:14:38.154621+0000","last_active":"2023-12-19T18:14:39.537962+0000","last_peered":"2023-12-19T18:14:39.537962+0000","last_clean":"2023-12-19T18:14:39.537962+0000","last_became_active":"2023-12-19T18:14:38.153849+0000","last_became_peered":"2023-12-19T18:14:38.153849+0000","last_unstale":"2023-12-19T18:14:39.537962+0000","last_undegraded":"2023-12-19T18:14:39.537962+0000","last_fullsized":"2023-12-19T18:14:39.537962+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:21.424875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.546629+0000","last_change":"2023-12-19T18:14:38.544944+0000","last_active":"2023-12-19T18:14:38.546629+0000","last_peered":"2023-12-19T18:14:38.546629+0000","last_clean":"2023-12-19T18:14:38.546629+0000","last_became_active":"2023-12-19T18:14:38.544640+0000","last_became_peered":"2023-12-19T18:14:38.544640+0000","last_unstale":"2023-12-19T18:14:38.546629+0000","last_undegraded":"2023-12-19T18:14:38.546629+0000","last_fullsized":"2023-12-19T18:14:38.546629+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:42:55.210047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"195'2","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537863+0000","last_change":"2023-12-19T18:14:38.151213+0000","last_active":"2023-12-19T18:14:39.537863+0000","last_peered":"2023-12-19T18:14:39.537863+0000","last_clean":"2023-12-19T18:14:39.537863+0000","last_became_active":"2023-12-19T18:14:38.150866+0000","last_became_peered":"2023-12-19T18:14:38.150866+0000","last_unstale":"2023-12-19T18:14:39.537863+0000","last_undegraded":"2023-12-19T18:14:39.537863+0000","last_fullsized":"2023-12-19T18:14:39.537863+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:56:58.937364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.547316+0000","last_change":"2023-12-19T18:14:38.544659+0000","last_active":"2023-12-19T18:14:38.547316+0000","last_peered":"2023-12-19T18:14:38.547316+0000","last_clean":"2023-12-19T18:14:38.547316+0000","last_became_active":"2023-12-19T18:14:38.544382+0000","last_became_peered":"2023-12-19T18:14:38.544382+0000","last_unstale":"2023-12-19T18:14:38.547316+0000","last_undegraded":"2023-12-19T18:14:38.547316+0000","last_fullsized":"2023-12-19T18:14:38.547316+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:45:15.039695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"195'1","reported_seq":11,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.568028+0000","last_change":"2023-12-19T18:14:38.140731+0000","last_active":"2023-12-19T18:14:38.568028+0000","last_peered":"2023-12-19T18:14:38.568028+0000","last_clean":"2023-12-19T18:14:38.568028+0000","last_became_active":"2023-12-19T18:14:38.140467+0000","last_became_peered":"2023-12-19T18:14:38.140467+0000","last_unstale":"2023-12-19T18:14:38.568028+0000","last_undegraded":"2023-12-19T18:14:38.568028+0000","last_fullsized":"2023-12-19T18:14:38.568028+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:40.296023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535208+0000","last_change":"2023-12-19T18:14:38.567457+0000","last_active":"2023-12-19T18:14:39.535208+0000","last_peered":"2023-12-19T18:14:39.535208+0000","last_clean":"2023-12-19T18:14:39.535208+0000","last_became_active":"2023-12-19T18:14:38.567136+0000","last_became_peered":"2023-12-19T18:14:38.567136+0000","last_unstale":"2023-12-19T18:14:39.535208+0000","last_undegraded":"2023-12-19T18:14:39.535208+0000","last_fullsized":"2023-12-19T18:14:39.535208+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:58:45.881349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.541641+0000","last_change":"2023-12-19T18:14:38.147705+0000","last_active":"2023-12-19T18:14:38.541641+0000","last_peered":"2023-12-19T18:14:38.541641+0000","last_clean":"2023-12-19T18:14:38.541641+0000","last_became_active":"2023-12-19T18:14:38.147433+0000","last_became_peered":"2023-12-19T18:14:38.147433+0000","last_unstale":"2023-12-19T18:14:38.541641+0000","last_undegraded":"2023-12-19T18:14:38.541641+0000","last_fullsized":"2023-12-19T18:14:38.541641+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:00:43.905340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537431+0000","last_change":"2023-12-19T18:14:38.545796+0000","last_active":"2023-12-19T18:14:39.537431+0000","last_peered":"2023-12-19T18:14:39.537431+0000","last_clean":"2023-12-19T18:14:39.537431+0000","last_became_active":"2023-12-19T18:14:38.544913+0000","last_became_peered":"2023-12-19T18:14:38.544913+0000","last_unstale":"2023-12-19T18:14:39.537431+0000","last_undegraded":"2023-12-19T18:14:39.537431+0000","last_fullsized":"2023-12-19T18:14:39.537431+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:06:43.178068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"195'2","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.565370+0000","last_change":"2023-12-19T18:14:38.154453+0000","last_active":"2023-12-19T18:14:38.565370+0000","last_peered":"2023-12-19T18:14:38.565370+0000","last_clean":"2023-12-19T18:14:38.565370+0000","last_became_active":"2023-12-19T18:14:38.153854+0000","last_became_peered":"2023-12-19T18:14:38.153854+0000","last_unstale":"2023-12-19T18:14:38.565370+0000","last_undegraded":"2023-12-19T18:14:38.565370+0000","last_fullsized":"2023-12-19T18:14:38.565370+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:58.287223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535283+0000","last_change":"2023-12-19T18:14:38.559775+0000","last_active":"2023-12-19T18:14:39.535283+0000","last_peered":"2023-12-19T18:14:39.535283+0000","last_clean":"2023-12-19T18:14:39.535283+0000","last_became_active":"2023-12-19T18:14:38.559512+0000","last_became_peered":"2023-12-19T18:14:38.559512+0000","last_unstale":"2023-12-19T18:14:39.535283+0000","last_undegraded":"2023-12-19T18:14:39.535283+0000","last_fullsized":"2023-12-19T18:14:39.535283+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:10:18.702122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.542035+0000","last_change":"2023-12-19T18:14:38.147302+0000","last_active":"2023-12-19T18:14:38.542035+0000","last_peered":"2023-12-19T18:14:38.542035+0000","last_clean":"2023-12-19T18:14:38.542035+0000","last_became_active":"2023-12-19T18:14:38.147003+0000","last_became_peered":"2023-12-19T18:14:38.147003+0000","last_unstale":"2023-12-19T18:14:38.542035+0000","last_undegraded":"2023-12-19T18:14:38.542035+0000","last_fullsized":"2023-12-19T18:14:38.542035+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:14:59.693406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.560918+0000","last_change":"2023-12-19T18:14:38.551584+0000","last_active":"2023-12-19T18:14:38.560918+0000","last_peered":"2023-12-19T18:14:38.560918+0000","last_clean":"2023-12-19T18:14:38.560918+0000","last_became_active":"2023-12-19T18:14:38.551312+0000","last_became_peered":"2023-12-19T18:14:38.551312+0000","last_unstale":"2023-12-19T18:14:38.560918+0000","last_undegraded":"2023-12-19T18:14:38.560918+0000","last_fullsized":"2023-12-19T18:14:38.560918+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:36:34.011690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.532943+0000","last_change":"2023-12-19T18:14:38.147014+0000","last_active":"2023-12-19T18:14:38.532943+0000","last_peered":"2023-12-19T18:14:38.532943+0000","last_clean":"2023-12-19T18:14:38.532943+0000","last_became_active":"2023-12-19T18:14:38.146758+0000","last_became_peered":"2023-12-19T18:14:38.146758+0000","last_unstale":"2023-12-19T18:14:38.532943+0000","last_undegraded":"2023-12-19T18:14:38.532943+0000","last_fullsized":"2023-12-19T18:14:38.532943+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:55:05.518903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660053+0000","last_change":"2023-12-19T18:14:38.553169+0000","last_active":"2023-12-19T18:14:39.660053+0000","last_peered":"2023-12-19T18:14:39.660053+0000","last_clean":"2023-12-19T18:14:39.660053+0000","last_became_active":"2023-12-19T18:14:38.552629+0000","last_became_peered":"2023-12-19T18:14:38.552629+0000","last_unstale":"2023-12-19T18:14:39.660053+0000","last_undegraded":"2023-12-19T18:14:39.660053+0000","last_fullsized":"2023-12-19T18:14:39.660053+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:56:59.946145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.541655+0000","last_change":"2023-12-19T18:14:38.139912+0000","last_active":"2023-12-19T18:14:38.541655+0000","last_peered":"2023-12-19T18:14:38.541655+0000","last_clean":"2023-12-19T18:14:38.541655+0000","last_became_active":"2023-12-19T18:14:38.139631+0000","last_became_peered":"2023-12-19T18:14:38.139631+0000","last_unstale":"2023-12-19T18:14:38.541655+0000","last_undegraded":"2023-12-19T18:14:38.541655+0000","last_fullsized":"2023-12-19T18:14:38.541655+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:35:14.383673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.566049+0000","last_change":"2023-12-19T18:14:38.560175+0000","last_active":"2023-12-19T18:14:38.566049+0000","last_peered":"2023-12-19T18:14:38.566049+0000","last_clean":"2023-12-19T18:14:38.566049+0000","last_became_active":"2023-12-19T18:14:38.559764+0000","last_became_peered":"2023-12-19T18:14:38.559764+0000","last_unstale":"2023-12-19T18:14:38.566049+0000","last_undegraded":"2023-12-19T18:14:38.566049+0000","last_fullsized":"2023-12-19T18:14:38.566049+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:03.792973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"195'2","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.566331+0000","last_change":"2023-12-19T18:14:38.150122+0000","last_active":"2023-12-19T18:14:38.566331+0000","last_peered":"2023-12-19T18:14:38.566331+0000","last_clean":"2023-12-19T18:14:38.566331+0000","last_became_active":"2023-12-19T18:14:38.149061+0000","last_became_peered":"2023-12-19T18:14:38.149061+0000","last_unstale":"2023-12-19T18:14:38.566331+0000","last_undegraded":"2023-12-19T18:14:38.566331+0000","last_fullsized":"2023-12-19T18:14:38.566331+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:58.287223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535228+0000","last_change":"2023-12-19T18:14:38.555610+0000","last_active":"2023-12-19T18:14:39.535228+0000","last_peered":"2023-12-19T18:14:39.535228+0000","last_clean":"2023-12-19T18:14:39.535228+0000","last_became_active":"2023-12-19T18:14:38.554505+0000","last_became_peered":"2023-12-19T18:14:38.554505+0000","last_unstale":"2023-12-19T18:14:39.535228+0000","last_undegraded":"2023-12-19T18:14:39.535228+0000","last_fullsized":"2023-12-19T18:14:39.535228+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:18:27.989688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537748+0000","last_change":"2023-12-19T18:14:38.150362+0000","last_active":"2023-12-19T18:14:39.537748+0000","last_peered":"2023-12-19T18:14:39.537748+0000","last_clean":"2023-12-19T18:14:39.537748+0000","last_became_active":"2023-12-19T18:14:38.149541+0000","last_became_peered":"2023-12-19T18:14:38.149541+0000","last_unstale":"2023-12-19T18:14:39.537748+0000","last_undegraded":"2023-12-19T18:14:39.537748+0000","last_fullsized":"2023-12-19T18:14:39.537748+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:01:06.419377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.660063+0000","last_change":"2023-12-19T18:14:38.547371+0000","last_active":"2023-12-19T18:14:39.660063+0000","last_peered":"2023-12-19T18:14:39.660063+0000","last_clean":"2023-12-19T18:14:39.660063+0000","last_became_active":"2023-12-19T18:14:38.546149+0000","last_became_peered":"2023-12-19T18:14:38.546149+0000","last_unstale":"2023-12-19T18:14:39.660063+0000","last_undegraded":"2023-12-19T18:14:39.660063+0000","last_fullsized":"2023-12-19T18:14:39.660063+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:14.424526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536866+0000","last_change":"2023-12-19T18:14:38.151549+0000","last_active":"2023-12-19T18:14:39.536866+0000","last_peered":"2023-12-19T18:14:39.536866+0000","last_clean":"2023-12-19T18:14:39.536866+0000","last_became_active":"2023-12-19T18:14:38.150642+0000","last_became_peered":"2023-12-19T18:14:38.150642+0000","last_unstale":"2023-12-19T18:14:39.536866+0000","last_undegraded":"2023-12-19T18:14:39.536866+0000","last_fullsized":"2023-12-19T18:14:39.536866+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:50.636065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.561175+0000","last_change":"2023-12-19T18:14:38.556310+0000","last_active":"2023-12-19T18:14:38.561175+0000","last_peered":"2023-12-19T18:14:38.561175+0000","last_clean":"2023-12-19T18:14:38.561175+0000","last_became_active":"2023-12-19T18:14:38.556035+0000","last_became_peered":"2023-12-19T18:14:38.556035+0000","last_unstale":"2023-12-19T18:14:38.561175+0000","last_undegraded":"2023-12-19T18:14:38.561175+0000","last_fullsized":"2023-12-19T18:14:38.561175+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:04.030577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.538807+0000","last_change":"2023-12-19T18:14:38.155386+0000","last_active":"2023-12-19T18:14:38.538807+0000","last_peered":"2023-12-19T18:14:38.538807+0000","last_clean":"2023-12-19T18:14:38.538807+0000","last_became_active":"2023-12-19T18:14:38.155090+0000","last_became_peered":"2023-12-19T18:14:38.155090+0000","last_unstale":"2023-12-19T18:14:38.538807+0000","last_undegraded":"2023-12-19T18:14:38.538807+0000","last_fullsized":"2023-12-19T18:14:38.538807+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:33:02.035434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.560943+0000","last_change":"2023-12-19T18:14:38.556518+0000","last_active":"2023-12-19T18:14:38.560943+0000","last_peered":"2023-12-19T18:14:38.560943+0000","last_clean":"2023-12-19T18:14:38.560943+0000","last_became_active":"2023-12-19T18:14:38.555850+0000","last_became_peered":"2023-12-19T18:14:38.555850+0000","last_unstale":"2023-12-19T18:14:38.560943+0000","last_undegraded":"2023-12-19T18:14:38.560943+0000","last_fullsized":"2023-12-19T18:14:38.560943+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:56.677526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.536608+0000","last_change":"2023-12-19T18:14:38.147774+0000","last_active":"2023-12-19T18:14:38.536608+0000","last_peered":"2023-12-19T18:14:38.536608+0000","last_clean":"2023-12-19T18:14:38.536608+0000","last_became_active":"2023-12-19T18:14:38.147452+0000","last_became_peered":"2023-12-19T18:14:38.147452+0000","last_unstale":"2023-12-19T18:14:38.536608+0000","last_undegraded":"2023-12-19T18:14:38.536608+0000","last_fullsized":"2023-12-19T18:14:38.536608+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:44.967592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.567780+0000","last_change":"2023-12-19T18:14:38.563043+0000","last_active":"2023-12-19T18:14:38.567780+0000","last_peered":"2023-12-19T18:14:38.567780+0000","last_clean":"2023-12-19T18:14:38.567780+0000","last_became_active":"2023-12-19T18:14:38.562735+0000","last_became_peered":"2023-12-19T18:14:38.562735+0000","last_unstale":"2023-12-19T18:14:38.567780+0000","last_undegraded":"2023-12-19T18:14:38.567780+0000","last_fullsized":"2023-12-19T18:14:38.567780+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:26:03.035425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.533773+0000","last_change":"2023-12-19T18:14:38.144637+0000","last_active":"2023-12-19T18:14:38.533773+0000","last_peered":"2023-12-19T18:14:38.533773+0000","last_clean":"2023-12-19T18:14:38.533773+0000","last_became_active":"2023-12-19T18:14:38.143545+0000","last_became_peered":"2023-12-19T18:14:38.143545+0000","last_unstale":"2023-12-19T18:14:38.533773+0000","last_undegraded":"2023-12-19T18:14:38.533773+0000","last_fullsized":"2023-12-19T18:14:38.533773+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:56:15.017376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.535003+0000","last_change":"2023-12-19T18:14:38.565919+0000","last_active":"2023-12-19T18:14:39.535003+0000","last_peered":"2023-12-19T18:14:39.535003+0000","last_clean":"2023-12-19T18:14:39.535003+0000","last_became_active":"2023-12-19T18:14:38.564564+0000","last_became_peered":"2023-12-19T18:14:38.564564+0000","last_unstale":"2023-12-19T18:14:39.535003+0000","last_undegraded":"2023-12-19T18:14:39.535003+0000","last_fullsized":"2023-12-19T18:14:39.535003+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:25:15.851908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.536831+0000","last_change":"2023-12-19T18:14:38.152239+0000","last_active":"2023-12-19T18:14:39.536831+0000","last_peered":"2023-12-19T18:14:39.536831+0000","last_clean":"2023-12-19T18:14:39.536831+0000","last_became_active":"2023-12-19T18:14:38.151919+0000","last_became_peered":"2023-12-19T18:14:38.151919+0000","last_unstale":"2023-12-19T18:14:39.536831+0000","last_undegraded":"2023-12-19T18:14:39.536831+0000","last_fullsized":"2023-12-19T18:14:39.536831+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:29:19.462927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-12-19T18:14:39.537233+0000","last_change":"2023-12-19T18:14:38.546681+0000","last_active":"2023-12-19T18:14:39.537233+0000","last_peered":"2023-12-19T18:14:39.537233+0000","last_clean":"2023-12-19T18:14:39.537233+0000","last_became_active":"2023-12-19T18:14:38.546405+0000","last_became_peered":"2023-12-19T18:14:38.546405+0000","last_unstale":"2023-12-19T18:14:39.537233+0000","last_undegraded":"2023-12-19T18:14:39.537233+0000","last_fullsized":"2023-12-19T18:14:39.537233+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:17:01.221162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.536096+0000","last_change":"2023-12-19T18:14:38.149007+0000","last_active":"2023-12-19T18:14:38.536096+0000","last_peered":"2023-12-19T18:14:38.536096+0000","last_clean":"2023-12-19T18:14:38.536096+0000","last_became_active":"2023-12-19T18:14:38.148727+0000","last_became_peered":"2023-12-19T18:14:38.148727+0000","last_unstale":"2023-12-19T18:14:38.536096+0000","last_undegraded":"2023-12-19T18:14:38.536096+0000","last_fullsized":"2023-12-19T18:14:38.536096+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:52.430219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.563644+0000","last_change":"2023-12-19T18:14:38.558001+0000","last_active":"2023-12-19T18:14:38.563644+0000","last_peered":"2023-12-19T18:14:38.563644+0000","last_clean":"2023-12-19T18:14:38.563644+0000","last_became_active":"2023-12-19T18:14:38.557768+0000","last_became_peered":"2023-12-19T18:14:38.557768+0000","last_unstale":"2023-12-19T18:14:38.563644+0000","last_undegraded":"2023-12-19T18:14:38.563644+0000","last_fullsized":"2023-12-19T18:14:38.563644+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:38.115030+0000","last_clean_scrub_stamp":"2023-12-19T18:14:38.115030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:45:15.039695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.538195+0000","last_change":"2023-12-19T18:14:38.148333+0000","last_active":"2023-12-19T18:14:38.538195+0000","last_peered":"2023-12-19T18:14:38.538195+0000","last_clean":"2023-12-19T18:14:38.538195+0000","last_became_active":"2023-12-19T18:14:38.148068+0000","last_became_peered":"2023-12-19T18:14:38.148068+0000","last_unstale":"2023-12-19T18:14:38.538195+0000","last_undegraded":"2023-12-19T18:14:38.538195+0000","last_fullsized":"2023-12-19T18:14:38.538195+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:37.106249+0000","last_clean_scrub_stamp":"2023-12-19T18:14:37.106249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:08.073725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"184'324","reported_seq":1090,"reported_epoch":194,"state":"active+clean","last_fresh":"2023-12-19T18:14:38.132191+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:14:38.132191+0000","last_peered":"2023-12-19T18:14:38.132191+0000","last_clean":"2023-12-19T18:14:38.132191+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:14:38.132191+0000","last_undegraded":"2023-12-19T18:14:38.132191+0000","last_fullsized":"2023-12-19T18:14:38.132191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1922,"num_read_kb":3999,"num_write":535,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1922,"num_read_kb":3999,"num_write":535,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":324,"ondisk_log_size":324,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60264,"kb_used_data":2976,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311576,"statfs":{"total":96636764160,"available":96575053824,"internally_reserved":0,"allocated":3047424,"data_stored":1871720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.74299999999999999,"15min":0.72599999999999998},"min":{"1min":0.52500000000000002,"5min":0.45600000000000002,"15min":0.39900000000000002},"max":{"1min":0.90600000000000003,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.754,"15min":0.73899999999999999},"min":{"1min":0.61099999999999999,"5min":0.52100000000000002,"15min":0.45500000000000002},"max":{"1min":1.022,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.80000000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76400000000000001,"15min":0.748},"min":{"1min":0.57499999999999996,"5min":0.50800000000000001,"15min":0.48499999999999999},"max":{"1min":0.91900000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.75900000000000001,"15min":0.755},"min":{"1min":0.55800000000000005,"5min":0.48099999999999998,"15min":0.42099999999999999},"max":{"1min":0.92000000000000004,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.82299999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75700000000000001,"15min":0.74099999999999999},"min":{"1min":0.64000000000000001,"5min":0.55400000000000005,"15min":0.38900000000000001},"max":{"1min":0.97899999999999998,"5min":1.036,"15min":1.0669999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77800000000000002,"15min":0.78400000000000003},"min":{"1min":0.59999999999999998,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0109999999999999,"5min":1.159,"15min":1.236},"last":0.67300000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.61599999999999999,"5min":0.55700000000000005,"15min":0.48999999999999999},"max":{"1min":1.002,"5min":1.079,"15min":1.079},"last":0.746},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.78300000000000003,"15min":0.78400000000000003},"min":{"1min":0.69199999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.96099999999999997,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.73099999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.59799999999999998},"min":{"1min":0.46300000000000002,"5min":0.40400000000000003,"15min":0.40300000000000002},"max":{"1min":0.84799999999999998,"5min":1.046,"15min":1.302},"last":0.629},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.626},"min":{"1min":0.44900000000000001,"5min":0.44600000000000001,"15min":0.42699999999999999},"max":{"1min":0.84699999999999998,"5min":0.95099999999999996,"15min":1.0209999999999999},"last":0.58899999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61799999999999999,"15min":0.63500000000000001},"min":{"1min":0.47399999999999998,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.88300000000000001,"5min":0.96199999999999997,"15min":1.1299999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65400000000000003,"15min":0.66500000000000004},"min":{"1min":0.51700000000000002,"5min":0.46899999999999997,"15min":0.433},"max":{"1min":0.86499999999999999,"5min":0.97299999999999998,"15min":1.0800000000000001},"last":0.76900000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71599999999999997,"15min":0.72099999999999997},"min":{"1min":0.56299999999999994,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":0.99099999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69299999999999995,"15min":0.70499999999999996},"min":{"1min":0.505,"5min":0.47199999999999998,"15min":0.42799999999999999},"max":{"1min":0.89700000000000002,"5min":1.0600000000000001,"15min":1.093},"last":0.68999999999999995}]}]},{"osd":6,"up_from":15,"seq":64424510545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63772,"kb_used_data":2388,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308068,"statfs":{"total":96636764160,"available":96571461632,"internally_reserved":0,"allocated":2445312,"data_stored":1286100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72899999999999998,"15min":0.747},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.43099999999999999},"max":{"1min":0.95699999999999996,"5min":0.98199999999999998,"15min":1.042},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72199999999999998,"15min":0.73899999999999999},"min":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.46000000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.65300000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71199999999999997,"15min":0.73499999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.754,"15min":0.75800000000000001},"min":{"1min":0.54000000000000004,"5min":0.434,"15min":0.434},"max":{"1min":0.92900000000000005,"5min":0.99199999999999999,"15min":1.1259999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.621,"5min":0.502,"15min":0.502},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0609999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74099999999999999,"15min":0.75700000000000001},"min":{"1min":0.57499999999999996,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.95399999999999996,"5min":1.0129999999999999,"15min":1.111},"last":0.61499999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76700000000000002,"15min":0.78600000000000003},"min":{"1min":0.58099999999999996,"5min":0.495,"15min":0.495},"max":{"1min":0.89400000000000002,"5min":1.048,"15min":1.1519999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.73399999999999999},"min":{"1min":0.48099999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.63800000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58499999999999996,"15min":0.59299999999999997},"min":{"1min":0.436,"5min":0.38700000000000001,"15min":0.371},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59699999999999998,"15min":0.60899999999999999},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.375},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.503}]},{"osd":5,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59899999999999998,"15min":0.60599999999999998},"min":{"1min":0.44,"5min":0.39700000000000002,"15min":0.374},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64800000000000002,"15min":0.66700000000000004},"min":{"1min":0.50900000000000001,"5min":0.433,"15min":0.41599999999999998},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.66700000000000004}]},{"osd":7,"last update":"Tue Dec 19 18:14:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.52900000000000003,"5min":0.502,"15min":0.46000000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72699999999999998,"15min":0.72599999999999998},"min":{"1min":0.51700000000000002,"5min":0.5,"15min":0.44700000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.72499999999999998}]}]},{"osd":5,"up_from":15,"seq":64424510545,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29332,"kb_used_data":2380,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342508,"statfs":{"total":96636764160,"available":96606728192,"internally_reserved":0,"allocated":2437120,"data_stored":1279650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.74399999999999999,"15min":0.73199999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.34399999999999997},"max":{"1min":1.002,"5min":1.0109999999999999,"15min":1.0149999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.75700000000000001,"15min":0.74399999999999999},"min":{"1min":0.48599999999999999,"5min":0.48399999999999999,"15min":0.40699999999999997},"max":{"1min":0.96099999999999997,"5min":0.98299999999999998,"15min":1.0229999999999999},"last":0.56200000000000006}]},{"osd":1,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78300000000000003,"15min":0.76100000000000001},"min":{"1min":0.52100000000000002,"5min":0.497,"15min":0.49299999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.745,"15min":0.74099999999999999},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.47499999999999998},"max":{"1min":0.88900000000000001,"5min":1.004,"15min":1.004},"last":0.46300000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.80100000000000005,"15min":0.78200000000000003},"min":{"1min":0.68799999999999994,"5min":0.55600000000000005,"15min":0.47999999999999998},"max":{"1min":1.0289999999999999,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.5},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76800000000000002,"15min":0.73799999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.44400000000000001},"max":{"1min":0.93999999999999995,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.47499999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.76800000000000002},"min":{"1min":0.56399999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0129999999999999,"5min":1.05,"15min":1.05},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.78700000000000003,"15min":0.77900000000000003},"min":{"1min":0.61499999999999999,"5min":0.56100000000000005,"15min":0.50800000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.0329999999999999},"last":0.54400000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.623,"15min":0.58299999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.35399999999999998},"max":{"1min":0.90200000000000002,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65200000000000002,"15min":0.629},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.41599999999999998},"max":{"1min":0.98499999999999999,"5min":1.208,"15min":1.208},"last":0.67100000000000004}]},{"osd":6,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65600000000000003,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.435},"max":{"1min":0.90000000000000002,"5min":0.98099999999999998,"15min":0.98999999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68999999999999995,"15min":0.66100000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.40699999999999997},"max":{"1min":1,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:14:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72599999999999998,"15min":0.68999999999999995},"min":{"1min":0.439,"5min":0.436,"15min":0.436},"max":{"1min":0.95499999999999996,"5min":1.169,"15min":1.169},"last":0.623},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.71399999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.018,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.59599999999999997}]}]},{"osd":4,"up_from":15,"seq":64424510545,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63772,"kb_used_data":2388,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308068,"statfs":{"total":96636764160,"available":96571461632,"internally_reserved":0,"allocated":2445312,"data_stored":1280680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.751,"15min":0.74199999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.35899999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.109},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.76100000000000001,"15min":0.755},"min":{"1min":0.621,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.997,"5min":0.997,"15min":1.4630000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.77000000000000002,"15min":0.76100000000000001},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.96799999999999997,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.76500000000000001},"min":{"1min":0.51000000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.997,"15min":1.1020000000000001},"last":0.49099999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.79100000000000004,"15min":0.77200000000000002},"min":{"1min":0.63800000000000001,"5min":0.53800000000000003,"15min":0.47699999999999998},"max":{"1min":1.0289999999999999,"5min":1.0309999999999999,"15min":1.071},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.79100000000000004,"15min":0.77800000000000002},"min":{"1min":0.68300000000000005,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":1.0009999999999999,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.67600000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.76500000000000001,"15min":0.76200000000000001},"min":{"1min":0.60699999999999998,"5min":0.56000000000000005,"15min":0.501},"max":{"1min":0.95699999999999996,"5min":0.98499999999999999,"15min":1.7809999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80400000000000005,"15min":0.79500000000000004},"min":{"1min":0.65000000000000002,"5min":0.61399999999999999,"15min":0.55000000000000004},"max":{"1min":0.98399999999999999,"5min":1.0249999999999999,"15min":1.5680000000000001},"last":0.65400000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.65400000000000003,"15min":0.624},"min":{"1min":0.51800000000000002,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.85099999999999998,"5min":0.95699999999999996,"15min":1.1020000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":1.042,"5min":1.042,"15min":3.508},"last":0.627}]},{"osd":6,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.67700000000000005,"15min":0.65600000000000003},"min":{"1min":0.50700000000000001,"5min":0.437,"15min":0.40200000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.4099999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.72599999999999998,"15min":0.70999999999999996},"min":{"1min":0.58999999999999997,"5min":0.48499999999999999,"15min":0.45600000000000002},"max":{"1min":0.92600000000000005,"5min":0.96599999999999997,"15min":1.0760000000000001},"last":0.73099999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:14:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.72399999999999998,"15min":0.71499999999999997},"min":{"1min":0.53900000000000003,"5min":0.50800000000000001,"15min":0.41899999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.831},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76300000000000001,"15min":0.752},"min":{"1min":0.57299999999999995,"5min":0.50600000000000001,"15min":0.46999999999999997},"max":{"1min":1.0229999999999999,"5min":1.036,"15min":1.0549999999999999},"last":0.746}]}]},{"osd":3,"up_from":15,"seq":64424510545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47376,"kb_used_data":2376,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324464,"statfs":{"total":96636764160,"available":96588251136,"internally_reserved":0,"allocated":2433024,"data_stored":1279080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49199999999999999,"15min":0.48499999999999999},"min":{"1min":0.34100000000000003,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.77100000000000002,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.52900000000000003,"15min":0.53500000000000003},"min":{"1min":0.35399999999999998,"5min":0.33700000000000002,"15min":0.32800000000000001},"max":{"1min":0.75700000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.38700000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49299999999999999,"15min":0.5},"min":{"1min":0.36099999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.82199999999999995},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.42799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.89600000000000002},"last":0.67600000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.40799999999999997,"5min":0.40400000000000003,"15min":0.378},"max":{"1min":0.752,"5min":0.79600000000000004,"15min":0.89400000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59699999999999998,"15min":0.60799999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.40100000000000002},"max":{"1min":0.748,"5min":0.83899999999999997,"15min":0.91300000000000003},"last":0.45100000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.82399999999999995,"15min":0.83599999999999997},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.024,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82699999999999996,"15min":0.83099999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.56599999999999995},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.69499999999999995}]},{"osd":5,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.84299999999999997,"15min":0.84999999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.57399999999999995},"max":{"1min":1.1120000000000001,"5min":1.2749999999999999,"15min":1.397},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.83399999999999996,"15min":0.83599999999999997},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55100000000000005},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.262},"last":0.81399999999999995}]},{"osd":6,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.84599999999999997,"15min":0.85299999999999998},"min":{"1min":0.63400000000000001,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1859999999999999,"5min":1.234,"15min":1.274},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85399999999999998,"15min":0.85499999999999998},"min":{"1min":0.59799999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.165,"5min":1.2490000000000001,"15min":1.349},"last":0.68600000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:14:33 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91800000000000004,"15min":0.91200000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.2330000000000001,"5min":1.3540000000000001,"15min":1.486},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90800000000000003,"15min":0.90500000000000003},"min":{"1min":0.72099999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3999999999999999},"last":0.79400000000000004}]}]},{"osd":2,"up_from":15,"seq":64424510544,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47376,"kb_used_data":2376,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324464,"statfs":{"total":96636764160,"available":96588251136,"internally_reserved":0,"allocated":2433024,"data_stored":1279136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.45600000000000002,"15min":0.45500000000000002},"min":{"1min":0.317,"5min":0.29999999999999999,"15min":0.27100000000000002},"max":{"1min":0.64200000000000002,"5min":0.752,"15min":0.752},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.50900000000000001,"15min":0.51500000000000001},"min":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.495}]},{"osd":1,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.622,"15min":0.629},"min":{"1min":0.47299999999999998,"5min":0.40100000000000002,"15min":0.38500000000000001},"max":{"1min":0.83799999999999997,"5min":0.89100000000000001,"15min":0.96399999999999997},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51700000000000002,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.70199999999999996,"5min":0.746,"15min":0.86299999999999999},"last":0.30599999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60799999999999998,"15min":0.622},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":0.86199999999999999,"15min":0.95999999999999996},"last":0.435},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57899999999999996,"15min":0.58599999999999997},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77600000000000002,"5min":0.79700000000000004,"15min":0.875},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81799999999999995,"15min":0.80900000000000005},"min":{"1min":0.64200000000000002,"5min":0.56699999999999995,"15min":0.56499999999999995},"max":{"1min":1.0880000000000001,"5min":1.306,"15min":1.306},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78800000000000003,"15min":0.80100000000000005},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.2070000000000001,"5min":1.272,"15min":1.466},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.81000000000000005},"min":{"1min":0.55700000000000005,"5min":0.441,"15min":0.441},"max":{"1min":1.1080000000000001,"5min":1.2050000000000001,"15min":1.3049999999999999},"last":0.75},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.83199999999999996},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.55000000000000004},"max":{"1min":1.0700000000000001,"5min":1.077,"15min":1.429},"last":0.86699999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.86599999999999999,"15min":0.85099999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.59699999999999998},"max":{"1min":1.044,"5min":1.365,"15min":1.4119999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81200000000000006,"15min":0.81399999999999995},"min":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.53500000000000003},"max":{"1min":1.117,"5min":1.2,"15min":1.2470000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.56000000000000005},"max":{"1min":1.159,"5min":1.393,"15min":1.45},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.876,"5min":0.871,"15min":0.872},"min":{"1min":0.69599999999999995,"5min":0.68400000000000005,"15min":0.61799999999999999},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.73699999999999999}]}]},{"osd":1,"up_from":15,"seq":64424510545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59664,"kb_used_data":2376,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312176,"statfs":{"total":96636764160,"available":96575668224,"internally_reserved":0,"allocated":2433024,"data_stored":1279080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46899999999999997,"15min":0.46400000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61199999999999999,"5min":0.77800000000000002,"15min":0.89400000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.38500000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82199999999999995,"5min":0.83999999999999997,"15min":0.89200000000000002},"last":0.40999999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69799999999999995,"5min":0.871,"15min":0.871},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.46800000000000003,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.71899999999999997,"5min":0.86199999999999999,"15min":0.877},"last":0.44700000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.64600000000000002,"15min":0.63500000000000001},"min":{"1min":0.498,"5min":0.45800000000000002,"15min":0.442},"max":{"1min":0.79100000000000004,"5min":0.94499999999999995,"15min":0.99399999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56000000000000005,"15min":0.55400000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77900000000000003,"5min":0.82699999999999996,"15min":0.86599999999999999},"last":0.61799999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80700000000000005,"15min":0.80300000000000005},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80000000000000004,"15min":0.81699999999999995},"min":{"1min":0.51400000000000001,"5min":0.48599999999999999,"15min":0.47799999999999998},"max":{"1min":1.1319999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.95499999999999996}]},{"osd":5,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.80500000000000005,"15min":0.80700000000000005},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.236,"5min":1.236,"15min":1.2629999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.875,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.57799999999999996},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.92900000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.80600000000000005,"15min":0.81899999999999995},"min":{"1min":0.54600000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2689999999999999},"last":0.875},{"interface":"front","average":{"1min":0.878,"5min":0.85199999999999998,"15min":0.85299999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.82699999999999996}]},{"osd":7,"last update":"Tue Dec 19 18:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.85199999999999998,"15min":0.85999999999999999},"min":{"1min":0.61699999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.1679999999999999,"5min":1.236,"15min":1.236},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.80500000000000005}]}]},{"osd":0,"up_from":14,"seq":60129543250,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60252,"kb_used_data":2964,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311588,"statfs":{"total":96636764160,"available":96575066112,"internally_reserved":0,"allocated":3035136,"data_stored":1876446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46100000000000002,"15min":0.46000000000000002},"min":{"1min":0.34599999999999997,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.64900000000000002,"5min":1.216,"15min":1.216},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52600000000000002,"15min":0.53100000000000003},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.32900000000000001},"max":{"1min":0.73599999999999999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.58999999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.52800000000000002,"15min":0.53500000000000003},"min":{"1min":0.35999999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77000000000000002,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.38500000000000001},"max":{"1min":0.68500000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.64900000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.55200000000000005,"15min":0.55400000000000005},"min":{"1min":0.36299999999999999,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69399999999999995,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.81699999999999995,"5min":1.032,"15min":1.032},"last":0.68600000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81000000000000005,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.55200000000000005,"15min":0.496},"max":{"1min":1.0229999999999999,"5min":1.1359999999999999,"15min":1.381},"last":0.998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.622,"5min":0.58699999999999997,"15min":0.47799999999999998},"max":{"1min":1.137,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.88}]},{"osd":5,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.82099999999999995,"15min":0.82599999999999996},"min":{"1min":0.59499999999999997,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1180000000000001,"5min":1.216,"15min":1.3420000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.80900000000000005,"15min":0.82699999999999996},"min":{"1min":0.59899999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.087,"5min":1.087,"15min":1.288},"last":1.1020000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.84299999999999997,"15min":0.84899999999999998},"min":{"1min":0.628,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.1140000000000001,"5min":1.349,"15min":1.4019999999999999},"last":1.054},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83299999999999996,"15min":0.84299999999999997},"min":{"1min":0.59499999999999997,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.98099999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:14:14 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84499999999999997,"15min":0.85599999999999998},"min":{"1min":0.66400000000000003,"5min":0.64600000000000002,"15min":0.56599999999999995},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.415},"last":1.075},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85899999999999999,"15min":0.86399999999999999},"min":{"1min":0.65700000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0980000000000001,"5min":1.395,"15min":1.395},"last":0.94699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:14:43.072 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:14:43.072 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:14:43.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.260+0000 7f00ab8e5640 1 Processor -- start 2023-12-19T18:14:43.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.260+0000 7f00ab8e5640 1 -- start start 2023-12-19T18:14:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00ab8e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00a40b0c30 0x7f00a40ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00ab8e5640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a406b500 con 0x7f00a415ada0 2023-12-19T18:14:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00ab8e5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a4070b60 con 0x7f00a4070db0 2023-12-19T18:14:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00ab8e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40715b0 con 0x7f00a40b0c30 2023-12-19T18:14:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00a9681640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f00a415ada0 0x7f00a40b0520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39928/0 (socket says 172.21.15.26:39928) 2023-12-19T18:14:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00a9681640 1 -- 172.21.15.26:0/1819927821 learned_addr learned my addr 172.21.15.26:0/1819927821 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:43.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00a8e80640 1 --2- 172.21.15.26:0/1819927821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00a40b0c30 0x7f00a40ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:43.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2854676208 0 0) 0x7f00a406b500 con 0x7f00a415ada0 2023-12-19T18:14:43.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0090003530 con 0x7f00a415ada0 2023-12-19T18:14:43.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099556207 0 0) 0x7f00a4070b60 con 0x7f00a4070db0 2023-12-19T18:14:43.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00a406b500 con 0x7f00a4070db0 2023-12-19T18:14:43.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3480036343 0 0) 0x7f0090003530 con 0x7f00a415ada0 2023-12-19T18:14:43.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00a40b0c30 msgr2=0x7f00a40ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:43.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 --2- 172.21.15.26:0/1819927821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00a40b0c30 0x7f00a40ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 >> v1:172.21.15.160:6789/0 conn(0x7f00a4070db0 legacy=0x7f00a406b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a40702c0 con 0x7f00a415ada0 2023-12-19T18:14:43.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f00a0001680 con 0x7f00a415ada0 2023-12-19T18:14:43.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 >> v1:172.21.15.26:6789/0 conn(0x7f00a415ada0 legacy=0x7f00a40b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 --2- 172.21.15.26:0/1819927821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00a40b0c30 0x7f0090003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:43.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 --2- 172.21.15.26:0/1819927821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00900040a0 0x7f00900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:43.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 --2- 172.21.15.26:0/1819927821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0090004d10 0x7f0090005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:43.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40715b0 con 0x7f0090004d10 2023-12-19T18:14:43.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009000ba70 con 0x7f00900040a0 2023-12-19T18:14:43.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009000bc30 con 0x7f00a40b0c30 2023-12-19T18:14:43.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00a9e82640 1 --2- 172.21.15.26:0/1819927821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0090004d10 0x7f0090005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:43.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00a9681640 1 --2- 172.21.15.26:0/1819927821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00a40b0c30 0x7f0090003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:43.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00a8e80640 1 --2- 172.21.15.26:0/1819927821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00900040a0 0x7f00900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:43.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00a9e82640 1 -- 172.21.15.26:0/1819927821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00a40b0c30 msgr2=0x7f0090003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:43.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00a9e82640 1 --2- 172.21.15.26:0/1819927821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00a40b0c30 0x7f0090003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00a9e82640 1 -- 172.21.15.26:0/1819927821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00900040a0 msgr2=0x7f00900047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:43.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00a9e82640 1 --2- 172.21.15.26:0/1819927821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00900040a0 0x7f00900047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.264+0000 7f00a9e82640 1 -- 172.21.15.26:0/1819927821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0090003530 con 0x7f0090004d10 2023-12-19T18:14:43.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.268+0000 7f00a9e82640 1 --2- 172.21.15.26:0/1819927821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0090004d10 0x7f0090005190 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f009c001fd0 tx=0x7f009c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b377a7b31b69d805 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:43.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.268+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c003b80 con 0x7f0090004d10 2023-12-19T18:14:43.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.268+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c003d00 con 0x7f0090004d10 2023-12-19T18:14:43.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.268+0000 7f009bfff640 1 -- 172.21.15.26:0/1819927821 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c005310 con 0x7f0090004d10 2023-12-19T18:14:43.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.268+0000 7f00ab8e5640 1 -- 172.21.15.26:0/1819927821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0090004d10 msgr2=0x7f0090005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.268+0000 7f00ab8e5640 1 --2- 172.21.15.26:0/1819927821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0090004d10 0x7f0090005190 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f009c001fd0 tx=0x7f009c01c8e0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.268+0000 7f00a9e82640 1 -- 172.21.15.26:0/1819927821 reap_dead start 2023-12-19T18:14:43.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.268+0000 7f00ab8e5640 1 -- 172.21.15.26:0/1819927821 shutdown_connections 2023-12-19T18:14:43.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.268+0000 7f00ab8e5640 1 -- 172.21.15.26:0/1819927821 >> 172.21.15.26:0/1819927821 conn(0x7f00a40660d0 msgr2=0x7f00a40ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:43.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.268+0000 7f00ab8e5640 1 -- 172.21.15.26:0/1819927821 shutdown_connections 2023-12-19T18:14:43.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.268+0000 7f00ab8e5640 1 -- 172.21.15.26:0/1819927821 wait complete. 2023-12-19T18:14:43.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.268+0000 7f00ab8e5640 1 Processor -- start 2023-12-19T18:14:43.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00ab8e5640 1 -- start start 2023-12-19T18:14:43.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00ab8e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00a4157220 0x7f00a41f0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:43.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00ab8e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a40c7510 0x7f00a40c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:43.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00ab8e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00a40c3a20 0x7f00a40cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:43.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00ab8e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40c2fe0 con 0x7f00a4157220 2023-12-19T18:14:43.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00ab8e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a4157600 con 0x7f00a40c7510 2023-12-19T18:14:43.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00ab8e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a41f0b60 con 0x7f00a40c3a20 2023-12-19T18:14:43.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00a9681640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00a4157220 0x7f00a41f0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:43.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00a9681640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00a4157220 0x7f00a41f0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34054/0 (socket says 172.21.15.26:34054) 2023-12-19T18:14:43.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00a9681640 1 -- 172.21.15.26:0/5060636 learned_addr learned my addr 172.21.15.26:0/5060636 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:43.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00a9e82640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00a40c3a20 0x7f00a40cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:43.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00a8e80640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a40c7510 0x7f00a40c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00a9681640 1 -- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00a40c3a20 msgr2=0x7f00a40cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00a9681640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00a40c3a20 0x7f00a40cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00a9681640 1 -- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a40c7510 msgr2=0x7f00a40c34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00a9681640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a40c7510 0x7f00a40c34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00a9681640 1 -- 172.21.15.26:0/5060636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a41f0e40 con 0x7f00a4157220 2023-12-19T18:14:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00a9681640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00a4157220 0x7f00a41f0220 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f00a0012510 tx=0x7f00a0012910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:43.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f0099ffb640 1 -- 172.21.15.26:0/5060636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0014810 con 0x7f00a4157220 2023-12-19T18:14:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f0099ffb640 1 -- 172.21.15.26:0/5060636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a0014990 con 0x7f00a4157220 2023-12-19T18:14:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00ab8e5640 1 -- 172.21.15.26:0/5060636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a41f1120 con 0x7f00a4157220 2023-12-19T18:14:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f0099ffb640 1 -- 172.21.15.26:0/5060636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0014d40 con 0x7f00a4157220 2023-12-19T18:14:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00ab8e5640 1 -- 172.21.15.26:0/5060636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a40c5ba0 con 0x7f00a4157220 2023-12-19T18:14:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.272+0000 7f00ab8e5640 1 -- 172.21.15.26:0/5060636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a40c5dc0 con 0x7f00a4157220 2023-12-19T18:14:43.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.276+0000 7f0099ffb640 1 -- 172.21.15.26:0/5060636 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f00a00153e0 con 0x7f00a4157220 2023-12-19T18:14:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.276+0000 7f0099ffb640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0088065bb0 0x7f0088068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.276+0000 7f0099ffb640 1 -- 172.21.15.26:0/5060636 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f00a0086190 con 0x7f00a4157220 2023-12-19T18:14:43.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.276+0000 7f00a8e80640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0088065bb0 0x7f0088068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:43.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.276+0000 7f00a8e80640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0088065bb0 0x7f0088068070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f00a406cee0 tx=0x7f0094009190 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:43.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.280+0000 7f0099ffb640 1 -- 172.21.15.26:0/5060636 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f00a0057290 con 0x7f00a4157220 2023-12-19T18:14:43.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.492+0000 7f00ab8e5640 1 -- 172.21.15.26:0/5060636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f00a4074270 con 0x7f00a4157220 2023-12-19T18:14:43.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.496+0000 7f0099ffb640 1 -- 172.21.15.26:0/5060636 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v51) v1 ==== 190+0+0 (secure 0 0 0) 0x7f00a0056e20 con 0x7f00a4157220 2023-12-19T18:14:43.498 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2023-12-19T18:14:43.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.500+0000 7f00ab8e5640 1 -- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0088065bb0 msgr2=0x7f0088068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.500+0000 7f00ab8e5640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0088065bb0 0x7f0088068070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f00a406cee0 tx=0x7f0094009190 comp rx=0 tx=0).stop 2023-12-19T18:14:43.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.500+0000 7f00ab8e5640 1 -- 172.21.15.26:0/5060636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00a4157220 msgr2=0x7f00a41f0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.500+0000 7f00ab8e5640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00a4157220 0x7f00a41f0220 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f00a0012510 tx=0x7f00a0012910 comp rx=0 tx=0).stop 2023-12-19T18:14:43.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.500+0000 7f00ab8e5640 1 -- 172.21.15.26:0/5060636 shutdown_connections 2023-12-19T18:14:43.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.500+0000 7f00ab8e5640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0088065bb0 0x7f0088068070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.500+0000 7f00ab8e5640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f00a40c3a20 0x7f00a40cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.500+0000 7f00ab8e5640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00a40c7510 0x7f00a40c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.500+0000 7f00ab8e5640 1 --2- 172.21.15.26:0/5060636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00a4157220 0x7f00a41f0220 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.500+0000 7f00ab8e5640 1 -- 172.21.15.26:0/5060636 >> 172.21.15.26:0/5060636 conn(0x7f00a40660d0 msgr2=0x7f00a4159ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.500+0000 7f00ab8e5640 1 -- 172.21.15.26:0/5060636 shutdown_connections 2023-12-19T18:14:43.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.500+0000 7f00ab8e5640 1 -- 172.21.15.26:0/5060636 wait complete. 2023-12-19T18:14:43.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:14:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.664+0000 7fbe34d51640 1 Processor -- start 2023-12-19T18:14:43.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.664+0000 7fbe34d51640 1 -- start start 2023-12-19T18:14:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.664+0000 7fbe34d51640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbe300b4b30 0x7fbe30167750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.664+0000 7fbe34d51640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3015cfd0 con 0x7fbe3015cbf0 2023-12-19T18:14:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.664+0000 7fbe34d51640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe300b6fc0 con 0x7fbe300b6ae0 2023-12-19T18:14:43.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.664+0000 7fbe34d51640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe300b72a0 con 0x7fbe300b4b30 2023-12-19T18:14:43.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.664+0000 7fbe2f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbe3015cbf0 0x7fbe300b82c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39940/0 (socket says 172.21.15.26:39940) 2023-12-19T18:14:43.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.664+0000 7fbe2f7fe640 1 -- 172.21.15.26:0/1578135505 learned_addr learned my addr 172.21.15.26:0/1578135505 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:43.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.664+0000 7fbe2ffff640 1 --2- 172.21.15.26:0/1578135505 >> v2:172.21.15.160:3300/0 conn(0x7fbe300b4b30 0x7fbe30167750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:43.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.668+0000 7fbe2e7fc640 1 -- 172.21.15.26:0/1578135505 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 866575169 0 0) 0x7fbe3015cfd0 con 0x7fbe3015cbf0 2023-12-19T18:14:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.668+0000 7fbe2e7fc640 1 -- 172.21.15.26:0/1578135505 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe18003530 con 0x7fbe3015cbf0 2023-12-19T18:14:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.668+0000 7fbe2e7fc640 1 -- 172.21.15.26:0/1578135505 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436201752 0 0) 0x7fbe300b6fc0 con 0x7fbe300b6ae0 2023-12-19T18:14:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.668+0000 7fbe2e7fc640 1 -- 172.21.15.26:0/1578135505 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe3015cfd0 con 0x7fbe300b6ae0 2023-12-19T18:14:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.668+0000 7fbe2ffff640 1 -- 172.21.15.26:0/1578135505 >> v1:172.21.15.160:6789/0 conn(0x7fbe300b6ae0 legacy=0x7fbe300b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.668+0000 7fbe2ffff640 1 -- 172.21.15.26:0/1578135505 >> v1:172.21.15.26:6789/0 conn(0x7fbe3015cbf0 legacy=0x7fbe300b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.668+0000 7fbe2ffff640 1 -- 172.21.15.26:0/1578135505 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe300b7580 con 0x7fbe300b4b30 2023-12-19T18:14:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.668+0000 7fbe2ffff640 1 --2- 172.21.15.26:0/1578135505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe300b4b30 0x7fbe30167750 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009d90 tx=0x7fbe2801c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1400b0ea22ef468 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.668+0000 7fbe2e7fc640 1 -- 172.21.15.26:0/1578135505 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280a59a0 con 0x7fbe300b4b30 2023-12-19T18:14:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.668+0000 7fbe2e7fc640 1 -- 172.21.15.26:0/1578135505 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe280a5b00 con 0x7fbe300b4b30 2023-12-19T18:14:43.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.668+0000 7fbe2e7fc640 1 -- 172.21.15.26:0/1578135505 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280a5e70 con 0x7fbe300b4b30 2023-12-19T18:14:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 -- 172.21.15.26:0/1578135505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe300b4b30 msgr2=0x7fbe30167750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 --2- 172.21.15.26:0/1578135505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe300b4b30 0x7fbe30167750 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009d90 tx=0x7fbe2801c8c0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 -- 172.21.15.26:0/1578135505 shutdown_connections 2023-12-19T18:14:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 --2- 172.21.15.26:0/1578135505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe300b4b30 0x7fbe30167750 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 -- 172.21.15.26:0/1578135505 >> 172.21.15.26:0/1578135505 conn(0x7fbe3006a120 msgr2=0x7fbe3006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:43.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 -- 172.21.15.26:0/1578135505 shutdown_connections 2023-12-19T18:14:43.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 -- 172.21.15.26:0/1578135505 wait complete. 2023-12-19T18:14:43.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 Processor -- start 2023-12-19T18:14:43.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 -- start start 2023-12-19T18:14:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe300b4b30 0x7fbe300cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe300b6ae0 0x7fbe3014b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3015cbf0 0x7fbe300d2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3014b940 con 0x7fbe3015cbf0 2023-12-19T18:14:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe301554f0 con 0x7fbe300b6ae0 2023-12-19T18:14:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe34d51640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3014bdf0 con 0x7fbe300b4b30 2023-12-19T18:14:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe2ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3015cbf0 0x7fbe300d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe2ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3015cbf0 0x7fbe300d2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34060/0 (socket says 172.21.15.26:34060) 2023-12-19T18:14:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe2ffff640 1 -- 172.21.15.26:0/406549879 learned_addr learned my addr 172.21.15.26:0/406549879 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe2f7fe640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe300b4b30 0x7fbe300cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:43.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe2effd640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe300b6ae0 0x7fbe3014b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:43.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe2ffff640 1 -- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe300b4b30 msgr2=0x7fbe300cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe2ffff640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe300b4b30 0x7fbe300cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe2ffff640 1 -- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe300b6ae0 msgr2=0x7fbe3014b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe2ffff640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe300b6ae0 0x7fbe3014b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.672+0000 7fbe2ffff640 1 -- 172.21.15.26:0/406549879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe28009a70 con 0x7fbe3015cbf0 2023-12-19T18:14:43.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.676+0000 7fbe2ffff640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3015cbf0 0x7fbe300d2070 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fbe2801cda0 tx=0x7fbe28019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:43.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.676+0000 7fbe17fff640 1 -- 172.21.15.26:0/406549879 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280aed10 con 0x7fbe3015cbf0 2023-12-19T18:14:43.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.676+0000 7fbe17fff640 1 -- 172.21.15.26:0/406549879 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe280aee70 con 0x7fbe3015cbf0 2023-12-19T18:14:43.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.676+0000 7fbe17fff640 1 -- 172.21.15.26:0/406549879 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280b7610 con 0x7fbe3015cbf0 2023-12-19T18:14:43.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.676+0000 7fbe34d51640 1 -- 172.21.15.26:0/406549879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe300c64d0 con 0x7fbe3015cbf0 2023-12-19T18:14:43.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.676+0000 7fbe34d51640 1 -- 172.21.15.26:0/406549879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe30154730 con 0x7fbe3015cbf0 2023-12-19T18:14:43.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.676+0000 7fbe34d51640 1 -- 172.21.15.26:0/406549879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe30067400 con 0x7fbe3015cbf0 2023-12-19T18:14:43.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.676+0000 7fbe17fff640 1 -- 172.21.15.26:0/406549879 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbe280b4050 con 0x7fbe3015cbf0 2023-12-19T18:14:43.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.676+0000 7fbe17fff640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbe1c065b80 0x7fbe1c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:43.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.676+0000 7fbe17fff640 1 -- 172.21.15.26:0/406549879 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fbe2811e5b0 con 0x7fbe3015cbf0 2023-12-19T18:14:43.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.680+0000 7fbe2f7fe640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbe1c065b80 0x7fbe1c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:43.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.680+0000 7fbe2f7fe640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbe1c065b80 0x7fbe1c068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe20007fd0 tx=0x7fbe20007d80 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:43.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.684+0000 7fbe17fff640 1 -- 172.21.15.26:0/406549879 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbe280ebd50 con 0x7fbe3015cbf0 2023-12-19T18:14:43.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe34d51640 1 -- 172.21.15.26:0/406549879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe3006a6a0 con 0x7fbe3015cbf0 2023-12-19T18:14:43.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe17fff640 1 -- 172.21.15.26:0/406549879 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4723 (secure 0 0 0) 0x7fbe28009450 con 0x7fbe3015cbf0 2023-12-19T18:14:43.872 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:43.872 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3922467901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3922467901},{"type":"v1","addr":"172.21.15.160:6835","nonce":3922467901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":9562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3692512090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3692512090},{"type":"v1","addr":"172.21.15.160:6837","nonce":3692512090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":10083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3961033710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3961033710},{"type":"v1","addr":"172.21.15.26:6833","nonce":3961033710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:38.527840+0000","modified":"2023-12-19T18:14:41.539799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10086":{"gid":10086,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1537720341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1537720341},{"type":"v1","addr":"172.21.15.26:6835","nonce":1537720341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:14:43.873 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 87 2023-12-19T18:14:43.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe34d51640 1 -- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbe1c065b80 msgr2=0x7fbe1c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe34d51640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbe1c065b80 0x7fbe1c068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe20007fd0 tx=0x7fbe20007d80 comp rx=0 tx=0).stop 2023-12-19T18:14:43.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe34d51640 1 -- 172.21.15.26:0/406549879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3015cbf0 msgr2=0x7fbe300d2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:43.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe34d51640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3015cbf0 0x7fbe300d2070 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fbe2801cda0 tx=0x7fbe28019660 comp rx=0 tx=0).stop 2023-12-19T18:14:43.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe34d51640 1 -- 172.21.15.26:0/406549879 shutdown_connections 2023-12-19T18:14:43.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe34d51640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbe1c065b80 0x7fbe1c068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe34d51640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe3015cbf0 0x7fbe300d2070 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe34d51640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbe300b6ae0 0x7fbe3014b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe34d51640 1 --2- 172.21.15.26:0/406549879 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbe300b4b30 0x7fbe300cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:43.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe34d51640 1 -- 172.21.15.26:0/406549879 >> 172.21.15.26:0/406549879 conn(0x7fbe3006a120 msgr2=0x7fbe30153ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:43.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe34d51640 1 -- 172.21.15.26:0/406549879 shutdown_connections 2023-12-19T18:14:43.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:43.872+0000 7fbe34d51640 1 -- 172.21.15.26:0/406549879 wait complete. 2023-12-19T18:14:44.296 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:14:38.527840+0000', 'modified': '2023-12-19T18:14:41.539799+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10086}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10086': {'gid': 10086, 'name': 'c', 'rank': 0, 'incarnation': 84, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/1537720341', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 1537720341}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1537720341}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:14:44.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:14:44.297 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-19T18:14:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.444+0000 7f1ce262d640 1 Processor -- start 2023-12-19T18:14:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.444+0000 7f1ce262d640 1 -- start start 2023-12-19T18:14:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.444+0000 7f1ce262d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cdc1573f0 0x7f1cdc15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.444+0000 7f1ce262d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc15e110 con 0x7f1cdc0cea80 2023-12-19T18:14:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.444+0000 7f1ce262d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc15e3f0 con 0x7f1cdc1577d0 2023-12-19T18:14:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.444+0000 7f1ce262d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc15a300 con 0x7f1cdc1573f0 2023-12-19T18:14:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.444+0000 7f1cdbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cdc1573f0 0x7f1cdc15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.444+0000 7f1ce0bca640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1cdc0cea80 0x7f1cdc06f6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39950/0 (socket says 172.21.15.26:39950) 2023-12-19T18:14:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.444+0000 7f1ce0bca640 1 -- 172.21.15.26:0/3148485510 learned_addr learned my addr 172.21.15.26:0/3148485510 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.444+0000 7f1cdbfff640 1 -- 172.21.15.26:0/3148485510 >> v1:172.21.15.160:6789/0 conn(0x7f1cdc1577d0 legacy=0x7f1cdc15d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.444+0000 7f1cdbfff640 1 -- 172.21.15.26:0/3148485510 >> v1:172.21.15.26:6789/0 conn(0x7f1cdc0cea80 legacy=0x7f1cdc06f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.444+0000 7f1cdbfff640 1 -- 172.21.15.26:0/3148485510 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc15a5e0 con 0x7f1cdc1573f0 2023-12-19T18:14:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.444+0000 7f1cdaffd640 1 -- 172.21.15.26:0/3148485510 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827721257 0 0) 0x7f1cdc15e110 con 0x7f1cdc0cea80 2023-12-19T18:14:44.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.448+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/3148485510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc1573f0 0x7f1cdc15ab00 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0097c0 tx=0x7f1ccc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e66f16ed77523c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.448+0000 7f1cdaffd640 1 -- 172.21.15.26:0/3148485510 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0ab610 con 0x7f1cdc1573f0 2023-12-19T18:14:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.448+0000 7f1cdaffd640 1 -- 172.21.15.26:0/3148485510 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc0ab770 con 0x7f1cdc1573f0 2023-12-19T18:14:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.448+0000 7f1cdaffd640 1 -- 172.21.15.26:0/3148485510 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0abb00 con 0x7f1cdc1573f0 2023-12-19T18:14:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.448+0000 7f1ce262d640 1 -- 172.21.15.26:0/3148485510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc1573f0 msgr2=0x7f1cdc15ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.448+0000 7f1ce262d640 1 --2- 172.21.15.26:0/3148485510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc1573f0 0x7f1cdc15ab00 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0097c0 tx=0x7f1ccc01c440 comp rx=0 tx=0).stop 2023-12-19T18:14:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.448+0000 7f1ce262d640 1 -- 172.21.15.26:0/3148485510 shutdown_connections 2023-12-19T18:14:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.448+0000 7f1ce262d640 1 --2- 172.21.15.26:0/3148485510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc1573f0 0x7f1cdc15ab00 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.448+0000 7f1ce262d640 1 -- 172.21.15.26:0/3148485510 >> 172.21.15.26:0/3148485510 conn(0x7f1cdc0b1aa0 msgr2=0x7f1cdc0b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.448+0000 7f1ce262d640 1 -- 172.21.15.26:0/3148485510 shutdown_connections 2023-12-19T18:14:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.448+0000 7f1ce262d640 1 -- 172.21.15.26:0/3148485510 wait complete. 2023-12-19T18:14:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1ce262d640 1 Processor -- start 2023-12-19T18:14:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1ce262d640 1 -- start start 2023-12-19T18:14:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1ce262d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0cea80 0x7f1cdc144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1ce262d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc1573f0 0x7f1cdc148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1ce262d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cdc1577d0 0x7f1cdc14c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1ce262d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc144890 con 0x7f1cdc0cea80 2023-12-19T18:14:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1ce262d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc14d0e0 con 0x7f1cdc1573f0 2023-12-19T18:14:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1ce262d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc14d3c0 con 0x7f1cdc1577d0 2023-12-19T18:14:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cdbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0cea80 0x7f1cdc144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cdbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0cea80 0x7f1cdc144350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34072/0 (socket says 172.21.15.26:34072) 2023-12-19T18:14:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cdbfff640 1 -- 172.21.15.26:0/2405736512 learned_addr learned my addr 172.21.15.26:0/2405736512 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1ce0bca640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cdc1577d0 0x7f1cdc14c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cdb7fe640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc1573f0 0x7f1cdc148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cdbfff640 1 -- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cdc1577d0 msgr2=0x7f1cdc14c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cdc1577d0 0x7f1cdc14c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cdbfff640 1 -- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc1573f0 msgr2=0x7f1cdc148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc1573f0 0x7f1cdc148090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cdbfff640 1 -- 172.21.15.26:0/2405736512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc009420 con 0x7f1cdc0cea80 2023-12-19T18:14:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cdb7fe640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc1573f0 0x7f1cdc148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1ce0bca640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cdc1577d0 0x7f1cdc14c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0cea80 0x7f1cdc144350 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0ad040 tx=0x7f1ccc01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cd8ff9640 1 -- 172.21.15.26:0/2405736512 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0b5810 con 0x7f1cdc0cea80 2023-12-19T18:14:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cd8ff9640 1 -- 172.21.15.26:0/2405736512 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc0b59b0 con 0x7f1cdc0cea80 2023-12-19T18:14:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1ce262d640 1 -- 172.21.15.26:0/2405736512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc0b29c0 con 0x7f1cdc0cea80 2023-12-19T18:14:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1cd8ff9640 1 -- 172.21.15.26:0/2405736512 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0b5d60 con 0x7f1cdc0cea80 2023-12-19T18:14:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.452+0000 7f1ce262d640 1 -- 172.21.15.26:0/2405736512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc14d6a0 con 0x7f1cdc0cea80 2023-12-19T18:14:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1cd8ff9640 1 -- 172.21.15.26:0/2405736512 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1ccc0b4390 con 0x7f1cdc0cea80 2023-12-19T18:14:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1cd8ff9640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1cc8065b80 0x7f1cc8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1cd8ff9640 1 -- 172.21.15.26:0/2405736512 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f1ccc11e300 con 0x7f1cdc0cea80 2023-12-19T18:14:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1cdb7fe640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1cc8065b80 0x7f1cc8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1ce599c000 1 Processor -- start 2023-12-19T18:14:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1ce599c000 1 -- start start 2023-12-19T18:14:44.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1cdb7fe640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1cc8065b80 0x7f1cc8068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc150fb0 tx=0x7f1cd0009040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:44.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1ce599c000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728070 0x5611ee728450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1ce599c000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 0x5611ee6ddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1ce599c000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee6de1c0 0x5611ee6de5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1ce599c000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ee501c90 con 0x5611ee728070 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1ce599c000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ee533220 con 0x5611ee728990 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1ce599c000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611ee5307f0 con 0x5611ee6de1c0 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1cdbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728070 0x5611ee728450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1cdbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728070 0x5611ee728450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34084/0 (socket says 172.21.15.26:34084) 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.456+0000 7f1cdbfff640 1 -- 172.21.15.26:0/3995725791 learned_addr learned my addr 172.21.15.26:0/3995725791 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1cdb7fe640 1 --2- 172.21.15.26:0/3995725791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 0x5611ee6ddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1cdbfff640 1 -- 172.21.15.26:0/3995725791 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee6de1c0 msgr2=0x5611ee6de5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce0bca640 1 --2- 172.21.15.26:0/3995725791 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee6de1c0 0x5611ee6de5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/3995725791 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee6de1c0 0x5611ee6de5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1cdbfff640 1 -- 172.21.15.26:0/3995725791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 msgr2=0x5611ee6ddc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:44.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/3995725791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 0x5611ee6ddc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1cdbfff640 1 -- 172.21.15.26:0/3995725791 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611ee228890 con 0x5611ee728070 2023-12-19T18:14:44.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1cdb7fe640 1 --2- 172.21.15.26:0/3995725791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 0x5611ee6ddc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:44.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce0bca640 1 --2- 172.21.15.26:0/3995725791 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee6de1c0 0x5611ee6de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:44.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/3995725791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728070 0x5611ee728450 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0ef060 tx=0x7f1ccc0efb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774551acf2d6e598 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:44.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1cc1ffb640 1 -- 172.21.15.26:0/3995725791 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0f3190 con 0x5611ee728070 2023-12-19T18:14:44.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1cc1ffb640 1 -- 172.21.15.26:0/3995725791 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc0f7340 con 0x5611ee728070 2023-12-19T18:14:44.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1cc1ffb640 1 -- 172.21.15.26:0/3995725791 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc11c4a0 con 0x5611ee728070 2023-12-19T18:14:44.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 -- 172.21.15.26:0/3995725791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728070 msgr2=0x5611ee728450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 --2- 172.21.15.26:0/3995725791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728070 0x5611ee728450 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0ef060 tx=0x7f1ccc0efb20 comp rx=0 tx=0).stop 2023-12-19T18:14:44.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 -- 172.21.15.26:0/3995725791 shutdown_connections 2023-12-19T18:14:44.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 --2- 172.21.15.26:0/3995725791 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee6de1c0 0x5611ee6de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 --2- 172.21.15.26:0/3995725791 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 0x5611ee6ddc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 --2- 172.21.15.26:0/3995725791 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728070 0x5611ee728450 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 -- 172.21.15.26:0/3995725791 >> 172.21.15.26:0/3995725791 conn(0x5611ee6e4950 msgr2=0x5611ee6dc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:44.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 -- 172.21.15.26:0/3995725791 shutdown_connections 2023-12-19T18:14:44.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 -- 172.21.15.26:0/3995725791 wait complete. 2023-12-19T18:14:44.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 -- start start 2023-12-19T18:14:44.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 Processor -- start 2023-12-19T18:14:44.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 0x5611ee6e37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 0x5611ee6e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 0x5611ee715500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ee52c0e0 con 0x5611ee6de1c0 2023-12-19T18:14:44.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ee4cb310 con 0x5611ee728990 2023-12-19T18:14:44.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611ee4e9770 con 0x5611ee728070 2023-12-19T18:14:44.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1ce599c000 10 client.0 _ll_register_callbacks cb 0x5611ee6c9b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:14:44.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1cdbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 0x5611ee6e37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.460+0000 7f1cdbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 0x5611ee6e37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34090/0 (socket says 172.21.15.26:34090) 2023-12-19T18:14:44.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cdbfff640 1 -- 172.21.15.26:0/2958113992 learned_addr learned my addr 172.21.15.26:0/2958113992 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:44.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cdbfff640 1 -- 172.21.15.26:0/2958113992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 msgr2=0x5611ee6e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:44.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cdb7fe640 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 0x5611ee6e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce0bca640 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 0x5611ee715500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 0x5611ee6e3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cdbfff640 1 -- 172.21.15.26:0/2958113992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 msgr2=0x5611ee715500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:44.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 0x5611ee715500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cdbfff640 1 -- 172.21.15.26:0/2958113992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc14d6a0 con 0x5611ee6de1c0 2023-12-19T18:14:44.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cdb7fe640 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 0x5611ee6e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:44.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce0bca640 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 0x5611ee715500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:44.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 0x5611ee6e37d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc01c9b0 tx=0x7f1ccc0191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf1d11171e5211f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cbaffd640 1 -- 172.21.15.26:0/2958113992 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0f7340 con 0x5611ee6de1c0 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 -- 172.21.15.26:0/2958113992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611ee4fbc10 con 0x5611ee6de1c0 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cbaffd640 1 -- 172.21.15.26:0/2958113992 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f1ccc0f5020 con 0x5611ee6de1c0 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cbaffd640 1 -- 172.21.15.26:0/2958113992 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ccc0f51e0 con 0x5611ee6de1c0 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 10 client.10233 fetch_fsmap learned FSMap version 87 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 -- 172.21.15.26:0/2958113992 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x5611ee4f1260 con 0x5611ee6de1c0 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cbaffd640 1 -- 172.21.15.26:0/2958113992 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5611ee4fbc10 con 0x5611ee6de1c0 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cbaffd640 1 -- 172.21.15.26:0/2958113992 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 87) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1ccc0c8020 con 0x5611ee6de1c0 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 10 client.10233 fetch_fsmap finished waiting for FSMap version 87 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 10 client.10233 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 20 client.10233 populate_metadata read hostname 'smithi026' 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.26:6834/1537720341,v1:172.21.15.26:6835/1537720341] conn(0x5611ee737310 0x5611ee739700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 4 client.10233 mds_command: new command op to 10086 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:14:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 -- 172.21.15.26:0/2958113992 --> [v2:172.21.15.26:6834/1537720341,v1:172.21.15.26:6835/1537720341] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611ee52aa30 con 0x5611ee737310 2023-12-19T18:14:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cdb7fe640 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.26:6834/1537720341,v1:172.21.15.26:6835/1537720341] conn(0x5611ee737310 0x5611ee739700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cdb7fe640 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.26:6834/1537720341,v1:172.21.15.26:6835/1537720341] conn(0x5611ee737310 0x5611ee739700 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3544da39a1c749c server_cookie=f7ee867bb9c8244c in_seq=0 out_seq=0 2023-12-19T18:14:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cbaffd640 10 client.10233 ms_handle_connect on v2:172.21.15.26:6834/1537720341 2023-12-19T18:14:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cbaffd640 1 -- 172.21.15.26:0/2958113992 <== mds.0 v2:172.21.15.26:6834/1537720341 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5611ee52aa30 con 0x5611ee737310 2023-12-19T18:14:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cbaffd640 10 client.10233 handle_command_reply: tid=0 2023-12-19T18:14:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 client.10233 shutdown 2023-12-19T18:14:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 -- 172.21.15.26:0/2958113992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 msgr2=0x5611ee6e37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 0x5611ee6e37d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc01c9b0 tx=0x7f1ccc0191e0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 -- 172.21.15.26:0/2958113992 shutdown_connections 2023-12-19T18:14:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.26:6834/1537720341,v1:172.21.15.26:6835/1537720341] conn(0x5611ee737310 0x5611ee739700 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 0x5611ee715500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 0x5611ee6e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 --2- 172.21.15.26:0/2958113992 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 0x5611ee6e37d0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1cbaffd640 0 client.10233 ms_handle_reset on v2:172.21.15.26:6834/1537720341 2023-12-19T18:14:44.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 -- 172.21.15.26:0/2958113992 >> 172.21.15.26:0/2958113992 conn(0x5611ee6e4950 msgr2=0x5611ee6dcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:44.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 -- 172.21.15.26:0/2958113992 shutdown_connections 2023-12-19T18:14:44.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.464+0000 7f1ce599c000 1 -- 172.21.15.26:0/2958113992 wait complete. 2023-12-19T18:14:44.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.468+0000 7f1ce599c000 20 client.10233 trim_cache size 0 max 16384 2023-12-19T18:14:44.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.480+0000 7f1ce599c000 1 Processor -- start 2023-12-19T18:14:44.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.480+0000 7f1ce599c000 1 -- start start 2023-12-19T18:14:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 0x5611ee7141b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 0x5611ee7146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 0x5611ee739700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ee1e9940 con 0x5611ee6de1c0 2023-12-19T18:14:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ee4e9770 con 0x5611ee728990 2023-12-19T18:14:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611ee527a50 con 0x5611ee728070 2023-12-19T18:14:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1cdbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 0x5611ee7141b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1cdbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 0x5611ee7141b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34096/0 (socket says 172.21.15.26:34096) 2023-12-19T18:14:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1cdbfff640 1 -- 172.21.15.26:0/3452475182 learned_addr learned my addr 172.21.15.26:0/3452475182 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1cdbfff640 1 -- 172.21.15.26:0/3452475182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 msgr2=0x5611ee7146f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/3452475182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 0x5611ee7146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1cdbfff640 1 -- 172.21.15.26:0/3452475182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 msgr2=0x5611ee739700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:44.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/3452475182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 0x5611ee739700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1cdbfff640 1 -- 172.21.15.26:0/3452475182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611ee4d73b0 con 0x5611ee6de1c0 2023-12-19T18:14:44.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/3452475182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 0x5611ee7141b0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0ef060 tx=0x7f1ccc104000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271ec2c20c8e3d81 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:44.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1cc27fc640 1 -- 172.21.15.26:0/3452475182 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0f7340 con 0x5611ee6de1c0 2023-12-19T18:14:44.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1cc27fc640 1 -- 172.21.15.26:0/3452475182 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc0f3190 con 0x5611ee6de1c0 2023-12-19T18:14:44.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1cc27fc640 1 -- 172.21.15.26:0/3452475182 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0f6eb0 con 0x5611ee6de1c0 2023-12-19T18:14:44.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 -- 172.21.15.26:0/3452475182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 msgr2=0x5611ee7141b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 --2- 172.21.15.26:0/3452475182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 0x5611ee7141b0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0ef060 tx=0x7f1ccc104000 comp rx=0 tx=0).stop 2023-12-19T18:14:44.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 -- 172.21.15.26:0/3452475182 shutdown_connections 2023-12-19T18:14:44.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 --2- 172.21.15.26:0/3452475182 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee728990 0x5611ee739700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 --2- 172.21.15.26:0/3452475182 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 0x5611ee7146f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 --2- 172.21.15.26:0/3452475182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee6de1c0 0x5611ee7141b0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 -- 172.21.15.26:0/3452475182 >> 172.21.15.26:0/3452475182 conn(0x5611ee6e4950 msgr2=0x5611ee6e2390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:44.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 -- 172.21.15.26:0/3452475182 shutdown_connections 2023-12-19T18:14:44.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 -- 172.21.15.26:0/3452475182 wait complete. 2023-12-19T18:14:44.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 -- start start 2023-12-19T18:14:44.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 Processor -- start 2023-12-19T18:14:44.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee6de1c0 0x5611ee7839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 0x5611ee783f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728990 0x5611ee7101b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ee73df10 con 0x5611ee728990 2023-12-19T18:14:44.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611ee726550 con 0x5611ee6de1c0 2023-12-19T18:14:44.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5611ee5211f0 con 0x5611ee728070 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.484+0000 7f1ce599c000 10 client.0 _ll_register_callbacks cb 0x5611ee4da640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce0bca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728990 0x5611ee7101b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce0bca640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728990 0x5611ee7101b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34110/0 (socket says 172.21.15.26:34110) 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce0bca640 1 -- 172.21.15.26:0/377486453 learned_addr learned my addr 172.21.15.26:0/377486453 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cdb7fe640 1 --2- 172.21.15.26:0/377486453 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 0x5611ee783f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/377486453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee6de1c0 0x5611ee7839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce0bca640 1 -- 172.21.15.26:0/377486453 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 msgr2=0x5611ee783f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce0bca640 1 --2- 172.21.15.26:0/377486453 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 0x5611ee783f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce0bca640 1 -- 172.21.15.26:0/377486453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee6de1c0 msgr2=0x5611ee7839f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce0bca640 1 --2- 172.21.15.26:0/377486453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee6de1c0 0x5611ee7839f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce0bca640 1 -- 172.21.15.26:0/377486453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5611ee4f1260 con 0x5611ee728990 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce0bca640 1 --2- 172.21.15.26:0/377486453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728990 0x5611ee7101b0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400f510 tx=0x7f1cd400f910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2604e6237f688ab server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T18:14:44.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cbaffd640 1 -- 172.21.15.26:0/377486453 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4018070 con 0x5611ee728990 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cbaffd640 1 -- 172.21.15.26:0/377486453 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f1cd400c070 con 0x5611ee728990 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 -- 172.21.15.26:0/377486453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611ee525160 con 0x5611ee728990 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cbaffd640 1 -- 172.21.15.26:0/377486453 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cd4004d80 con 0x5611ee728990 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 10 client.10239 fetch_fsmap learned FSMap version 87 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 -- 172.21.15.26:0/377486453 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x5611ee228cc0 con 0x5611ee728990 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cbaffd640 1 -- 172.21.15.26:0/377486453 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5611ee525160 con 0x5611ee728990 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cbaffd640 1 -- 172.21.15.26:0/377486453 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 87) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1cd40070d0 con 0x5611ee728990 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 10 client.10239 fetch_fsmap finished waiting for FSMap version 87 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 10 client.10239 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 20 client.10239 populate_metadata read hostname 'smithi026' 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 --2- 172.21.15.26:0/377486453 >> [v2:172.21.15.26:6834/1537720341,v1:172.21.15.26:6835/1537720341] conn(0x5611ee78a8c0 0x5611ee78ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 4 client.10239 mds_command: new command op to 10086 tid=0[{"prefix": "status"}] 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 -- 172.21.15.26:0/377486453 --> [v2:172.21.15.26:6834/1537720341,v1:172.21.15.26:6835/1537720341] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5611ee524e80 con 0x5611ee78a8c0 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/377486453 >> [v2:172.21.15.26:6834/1537720341,v1:172.21.15.26:6835/1537720341] conn(0x5611ee78a8c0 0x5611ee78ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cdbfff640 1 --2- 172.21.15.26:0/377486453 >> [v2:172.21.15.26:6834/1537720341,v1:172.21.15.26:6835/1537720341] conn(0x5611ee78a8c0 0x5611ee78ccb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f0509402b312007 server_cookie=9d783074858c0c3e in_seq=0 out_seq=0 2023-12-19T18:14:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cbaffd640 10 client.10239 ms_handle_connect on v2:172.21.15.26:6834/1537720341 2023-12-19T18:14:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cbaffd640 1 -- 172.21.15.26:0/377486453 <== mds.0 v2:172.21.15.26:6834/1537720341 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x5611ee524e80 con 0x5611ee78a8c0 2023-12-19T18:14:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cbaffd640 10 client.10239 handle_command_reply: tid=0 2023-12-19T18:14:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 client.10239 shutdown 2023-12-19T18:14:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 -- 172.21.15.26:0/377486453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728990 msgr2=0x5611ee7101b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 --2- 172.21.15.26:0/377486453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728990 0x5611ee7101b0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400f510 tx=0x7f1cd400f910 comp rx=0 tx=0).stop 2023-12-19T18:14:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 -- 172.21.15.26:0/377486453 shutdown_connections 2023-12-19T18:14:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 --2- 172.21.15.26:0/377486453 >> [v2:172.21.15.26:6834/1537720341,v1:172.21.15.26:6835/1537720341] conn(0x5611ee78a8c0 0x5611ee78ccb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 --2- 172.21.15.26:0/377486453 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5611ee728990 0x5611ee7101b0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 --2- 172.21.15.26:0/377486453 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x5611ee728070 0x5611ee783f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 --2- 172.21.15.26:0/377486453 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5611ee6de1c0 0x5611ee7839f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1cbaffd640 0 client.10239 ms_handle_reset on v2:172.21.15.26:6834/1537720341 2023-12-19T18:14:44.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 -- 172.21.15.26:0/377486453 >> 172.21.15.26:0/377486453 conn(0x5611ee6e4950 msgr2=0x5611ee6e2390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:44.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 -- 172.21.15.26:0/377486453 shutdown_connections 2023-12-19T18:14:44.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 1 -- 172.21.15.26:0/377486453 wait complete. 2023-12-19T18:14:44.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.488+0000 7f1ce599c000 20 client.10239 trim_cache size 0 max 16384 2023-12-19T18:14:44.494 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:14:44.494 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T18:14:44.494 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T18:14:44.494 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 10086, 2023-12-19T18:14:44.494 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T18:14:44.494 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T18:14:44.494 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T18:14:44.495 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.9605443459999998, 2023-12-19T18:14:44.495 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 87, 2023-12-19T18:14:44.495 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 196, 2023-12-19T18:14:44.495 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 196, 2023-12-19T18:14:44.495 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 11.212925323 2023-12-19T18:14:44.495 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:14:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.492+0000 7f1ce262d640 1 -- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1cc8065b80 msgr2=0x7f1cc8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.492+0000 7f1ce262d640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1cc8065b80 0x7f1cc8068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc150fb0 tx=0x7f1cd0009040 comp rx=0 tx=0).stop 2023-12-19T18:14:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.492+0000 7f1ce262d640 1 -- 172.21.15.26:0/2405736512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0cea80 msgr2=0x7f1cdc144350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.492+0000 7f1ce262d640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0cea80 0x7f1cdc144350 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0ad040 tx=0x7f1ccc01ac80 comp rx=0 tx=0).stop 2023-12-19T18:14:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.492+0000 7f1ce262d640 1 -- 172.21.15.26:0/2405736512 shutdown_connections 2023-12-19T18:14:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.492+0000 7f1ce262d640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1cc8065b80 0x7f1cc8068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.492+0000 7f1ce262d640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cdc1577d0 0x7f1cdc14c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.492+0000 7f1ce262d640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cdc1573f0 0x7f1cdc148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.492+0000 7f1ce262d640 1 --2- 172.21.15.26:0/2405736512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cdc0cea80 0x7f1cdc144350 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.492+0000 7f1ce262d640 1 -- 172.21.15.26:0/2405736512 >> 172.21.15.26:0/2405736512 conn(0x7f1cdc0b1aa0 msgr2=0x7f1cdc0cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.492+0000 7f1ce262d640 1 -- 172.21.15.26:0/2405736512 shutdown_connections 2023-12-19T18:14:44.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.492+0000 7f1ce262d640 1 -- 172.21.15.26:0/2405736512 wait complete. 2023-12-19T18:14:44.510 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:14:44.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.652+0000 7fb44a52c640 1 Processor -- start 2023-12-19T18:14:44.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.652+0000 7fb44a52c640 1 -- start start 2023-12-19T18:14:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb44a52c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb44406b3d0 0x7fb4440ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb44a52c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb4440cec40 0x7fb44415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb44a52c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44406bc30 con 0x7fb444154220 2023-12-19T18:14:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb44a52c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4440cf0d0 con 0x7fb4440cec40 2023-12-19T18:14:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb44a52c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4440cc6e0 con 0x7fb44406b3d0 2023-12-19T18:14:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb448ac9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb4440cec40 0x7fb44415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb448ac9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb4440cec40 0x7fb44415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34114/0 (socket says 172.21.15.26:34114) 2023-12-19T18:14:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb448ac9640 1 -- 172.21.15.26:0/1382614478 learned_addr learned my addr 172.21.15.26:0/1382614478 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:44.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb4437fe640 1 --2- 172.21.15.26:0/1382614478 >> v2:172.21.15.160:3300/0 conn(0x7fb44406b3d0 0x7fb4440ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb448ac9640 1 -- 172.21.15.26:0/1382614478 >> v2:172.21.15.160:3300/0 conn(0x7fb44406b3d0 msgr2=0x7fb4440ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb448ac9640 1 --2- 172.21.15.26:0/1382614478 >> v2:172.21.15.160:3300/0 conn(0x7fb44406b3d0 0x7fb4440ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb448ac9640 1 -- 172.21.15.26:0/1382614478 >> v1:172.21.15.160:6789/0 conn(0x7fb444154220 legacy=0x7fb44415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb442ffd640 1 -- 172.21.15.26:0/1382614478 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502761108 0 0) 0x7fb44406bc30 con 0x7fb444154220 2023-12-19T18:14:44.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.656+0000 7fb448ac9640 1 -- 172.21.15.26:0/1382614478 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44406b950 con 0x7fb4440cec40 2023-12-19T18:14:44.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb448ac9640 1 --2- 172.21.15.26:0/1382614478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4440cec40 0x7fb44415b1c0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fb4440cf880 tx=0x7fb43c01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1724db0a8ce7950 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb442ffd640 1 -- 172.21.15.26:0/1382614478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c00b2d0 con 0x7fb4440cec40 2023-12-19T18:14:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb442ffd640 1 -- 172.21.15.26:0/1382614478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c00b430 con 0x7fb4440cec40 2023-12-19T18:14:44.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb442ffd640 1 -- 172.21.15.26:0/1382614478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0a7c90 con 0x7fb4440cec40 2023-12-19T18:14:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb44a52c640 1 -- 172.21.15.26:0/1382614478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4440cec40 msgr2=0x7fb44415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb44a52c640 1 --2- 172.21.15.26:0/1382614478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4440cec40 0x7fb44415b1c0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fb4440cf880 tx=0x7fb43c01eba0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb44a52c640 1 -- 172.21.15.26:0/1382614478 shutdown_connections 2023-12-19T18:14:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb44a52c640 1 --2- 172.21.15.26:0/1382614478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4440cec40 0x7fb44415b1c0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb44a52c640 1 --2- 172.21.15.26:0/1382614478 >> v2:172.21.15.160:3300/0 conn(0x7fb44406b3d0 0x7fb4440ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb44a52c640 1 -- 172.21.15.26:0/1382614478 >> 172.21.15.26:0/1382614478 conn(0x7fb4440658e0 msgr2=0x7fb444065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb44a52c640 1 -- 172.21.15.26:0/1382614478 shutdown_connections 2023-12-19T18:14:44.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb44a52c640 1 -- 172.21.15.26:0/1382614478 wait complete. 2023-12-19T18:14:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb44a52c640 1 Processor -- start 2023-12-19T18:14:44.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.660+0000 7fb44a52c640 1 -- start start 2023-12-19T18:14:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb44a52c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb44406b3d0 0x7fb444154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb44a52c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4440cec40 0x7fb4441f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb44a52c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444154220 0x7fb444150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb44a52c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4440b2900 con 0x7fb44406b3d0 2023-12-19T18:14:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb44a52c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb444152c70 con 0x7fb444154220 2023-12-19T18:14:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb44a52c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb444152f50 con 0x7fb4440cec40 2023-12-19T18:14:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb443fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb44406b3d0 0x7fb444154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb443fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb44406b3d0 0x7fb444154690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34126/0 (socket says 172.21.15.26:34126) 2023-12-19T18:14:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb443fff640 1 -- 172.21.15.26:0/1164436868 learned_addr learned my addr 172.21.15.26:0/1164436868 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb4437fe640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4440cec40 0x7fb4441f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb448ac9640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444154220 0x7fb444150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb443fff640 1 -- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4440cec40 msgr2=0x7fb4441f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb443fff640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4440cec40 0x7fb4441f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb443fff640 1 -- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444154220 msgr2=0x7fb444150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb443fff640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444154220 0x7fb444150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb443fff640 1 -- 172.21.15.26:0/1164436868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c00bd40 con 0x7fb44406b3d0 2023-12-19T18:14:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb4437fe640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4440cec40 0x7fb4441f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb448ac9640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444154220 0x7fb444150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb443fff640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb44406b3d0 0x7fb444154690 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb43400c4d0 tx=0x7fb43400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb440ff9640 1 -- 172.21.15.26:0/1164436868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb434005480 con 0x7fb44406b3d0 2023-12-19T18:14:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb440ff9640 1 -- 172.21.15.26:0/1164436868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb434003600 con 0x7fb44406b3d0 2023-12-19T18:14:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb44a52c640 1 -- 172.21.15.26:0/1164436868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4440b2240 con 0x7fb44406b3d0 2023-12-19T18:14:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb440ff9640 1 -- 172.21.15.26:0/1164436868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4340039b0 con 0x7fb44406b3d0 2023-12-19T18:14:44.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb44a52c640 1 -- 172.21.15.26:0/1164436868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4441f53b0 con 0x7fb44406b3d0 2023-12-19T18:14:44.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb44a52c640 1 -- 172.21.15.26:0/1164436868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb444074270 con 0x7fb44406b3d0 2023-12-19T18:14:44.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.664+0000 7fb440ff9640 1 -- 172.21.15.26:0/1164436868 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb434003b10 con 0x7fb44406b3d0 2023-12-19T18:14:44.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.668+0000 7fb440ff9640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb430065b80 0x7fb430068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:44.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.668+0000 7fb440ff9640 1 -- 172.21.15.26:0/1164436868 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fb434082c10 con 0x7fb44406b3d0 2023-12-19T18:14:44.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.668+0000 7fb4437fe640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb430065b80 0x7fb430068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:44.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.668+0000 7fb4437fe640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb430065b80 0x7fb430068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb444150330 tx=0x7fb438008040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:44.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.672+0000 7fb440ff9640 1 -- 172.21.15.26:0/1164436868 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb434050290 con 0x7fb44406b3d0 2023-12-19T18:14:44.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb44a52c640 1 -- 172.21.15.26:0/1164436868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb4440cdf60 con 0x7fb44406b3d0 2023-12-19T18:14:44.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb440ff9640 1 -- 172.21.15.26:0/1164436868 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v87) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb434017070 con 0x7fb44406b3d0 2023-12-19T18:14:44.864 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:14:44.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb44a52c640 1 -- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb430065b80 msgr2=0x7fb430068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb44a52c640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb430065b80 0x7fb430068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb444150330 tx=0x7fb438008040 comp rx=0 tx=0).stop 2023-12-19T18:14:44.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb44a52c640 1 -- 172.21.15.26:0/1164436868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb44406b3d0 msgr2=0x7fb444154690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:44.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb44a52c640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb44406b3d0 0x7fb444154690 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb43400c4d0 tx=0x7fb43400c990 comp rx=0 tx=0).stop 2023-12-19T18:14:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb44a52c640 1 -- 172.21.15.26:0/1164436868 shutdown_connections 2023-12-19T18:14:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb44a52c640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb430065b80 0x7fb430068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb44a52c640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444154220 0x7fb444150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb44a52c640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4440cec40 0x7fb4441f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb44a52c640 1 --2- 172.21.15.26:0/1164436868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb44406b3d0 0x7fb444154690 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb44a52c640 1 -- 172.21.15.26:0/1164436868 >> 172.21.15.26:0/1164436868 conn(0x7fb4440658e0 msgr2=0x7fb4440ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb44a52c640 1 -- 172.21.15.26:0/1164436868 shutdown_connections 2023-12-19T18:14:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:44.864+0000 7fb44a52c640 1 -- 172.21.15.26:0/1164436868 wait complete. 2023-12-19T18:14:44.885 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-19T18:14:44.885 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:14:44.885 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T18:14:44.885 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:14:44.885 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:14:44.885 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-19T18:14:44.885 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-19T18:14:44.885 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:14:44.885 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:14:44.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:44.886 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T18:14:44.896 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:14:44.896 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:14:44.896 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:14:44.896 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:14:44.896 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T18:14:44.897 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:14:44.897 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T18:14:44.897 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:14:44.897 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T18:14:44.897 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T18:14:44.897 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:14:44.897 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T18:14:44.897 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:14:44.897 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:14:44.897 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T18:14:44.897 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T18:14:44.897 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 36856sec preferred_lft 36856sec 2023-12-19T18:14:44.897 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T18:14:44.898 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:14:44.898 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T18:14:44.898 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:14:44.898 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T18:14:44.898 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T18:14:44.898 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T18:14:44.898 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T18:14:44.898 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:14:44.898 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:14:44.898 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T18:14:44.898 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:14:44.898 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:14:44.898 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-19T18:14:44.899 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:14:44.899 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:14:44.899 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:14:44.899 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-19T18:14:44.899 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:14:44.899 INFO:teuthology.orchestra.run.smithi026.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:14:44.899 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:14:44.899 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-19T18:14:44.899 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:14:44.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:44.899 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:14:44.958 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T18:14:44.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:44.960 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T18:14:45.012 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T18:14:45.013 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:14:45.013 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T18:14:45.013 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T18:14:45.013 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T18:14:45.013 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:14:45.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:45.013 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T18:14:45.013 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T18:14:45.014 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:14:45.014 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:14:45.014 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:14:45.014 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T18:14:45.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:45.108 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T18:14:45.118 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:14:45.118 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:14:45.119 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:14:45.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:45.119 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T18:14:45.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:14:45.184 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:14:45.239 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:14:45.240 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:14:45.240 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:14:45.290 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T18:14:45.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:45.291 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T18:14:45.386 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T18:14:45.387 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T18:14:45.399 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T18:14:45.399 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T18:14:45.400 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T18:14:45.400 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T18:14:45.400 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T18:14:45.403 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T18:14:45.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:45.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:14:45.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:45.425 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T18:14:45.484 INFO:teuthology.orchestra.run.smithi026.stdout:client8534 2023-12-19T18:14:45.484 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:14:45.485 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:14:45.540 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-19T18:14:45.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.724+0000 7fcf77ae5640 1 Processor -- start 2023-12-19T18:14:45.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.724+0000 7fcf77ae5640 1 -- start start 2023-12-19T18:14:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf77ae5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcf70154220 0x7fcf7015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf77ae5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcf7006b3d0 0x7fcf700ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf77ae5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf700cec40 0x7fcf7015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:45.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf77ae5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf700cf0d0 con 0x7fcf70154220 2023-12-19T18:14:45.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf77ae5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7006bc30 con 0x7fcf7006b3d0 2023-12-19T18:14:45.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf77ae5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf700cc6e0 con 0x7fcf700cec40 2023-12-19T18:14:45.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf75881640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcf70154220 0x7fcf7015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:45.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf75881640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcf70154220 0x7fcf7015aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34128/0 (socket says 172.21.15.26:34128) 2023-12-19T18:14:45.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf75881640 1 -- 172.21.15.26:0/2678910466 learned_addr learned my addr 172.21.15.26:0/2678910466 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:45.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf75080640 1 --2- 172.21.15.26:0/2678910466 >> v2:172.21.15.160:3300/0 conn(0x7fcf7006b3d0 0x7fcf700ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:45.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf76082640 1 --2- 172.21.15.26:0/2678910466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf700cec40 0x7fcf7015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:45.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf75881640 1 -- 172.21.15.26:0/2678910466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf700cec40 msgr2=0x7fcf7015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:45.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf75881640 1 --2- 172.21.15.26:0/2678910466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf700cec40 0x7fcf7015d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:45.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf75881640 1 -- 172.21.15.26:0/2678910466 >> v2:172.21.15.160:3300/0 conn(0x7fcf7006b3d0 msgr2=0x7fcf700ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:45.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf75881640 1 --2- 172.21.15.26:0/2678910466 >> v2:172.21.15.160:3300/0 conn(0x7fcf7006b3d0 0x7fcf700ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:45.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf75881640 1 -- 172.21.15.26:0/2678910466 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7015afc0 con 0x7fcf70154220 2023-12-19T18:14:45.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.728+0000 7fcf75881640 1 --2- 172.21.15.26:0/2678910466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf70154220 0x7fcf7015aa80 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fcf700cf880 tx=0x7fcf6c01e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ae3a4162306a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:45.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf7487f640 1 -- 172.21.15.26:0/2678910466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0ae640 con 0x7fcf70154220 2023-12-19T18:14:45.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf7487f640 1 -- 172.21.15.26:0/2678910466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c0ae7a0 con 0x7fcf70154220 2023-12-19T18:14:45.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf7487f640 1 -- 172.21.15.26:0/2678910466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0aeb30 con 0x7fcf70154220 2023-12-19T18:14:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 -- 172.21.15.26:0/2678910466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf70154220 msgr2=0x7fcf7015aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 --2- 172.21.15.26:0/2678910466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf70154220 0x7fcf7015aa80 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fcf700cf880 tx=0x7fcf6c01e420 comp rx=0 tx=0).stop 2023-12-19T18:14:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 -- 172.21.15.26:0/2678910466 shutdown_connections 2023-12-19T18:14:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 --2- 172.21.15.26:0/2678910466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf700cec40 0x7fcf7015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 --2- 172.21.15.26:0/2678910466 >> v2:172.21.15.160:3300/0 conn(0x7fcf7006b3d0 0x7fcf700ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 --2- 172.21.15.26:0/2678910466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf70154220 0x7fcf7015aa80 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 -- 172.21.15.26:0/2678910466 >> 172.21.15.26:0/2678910466 conn(0x7fcf700658e0 msgr2=0x7fcf70065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 -- 172.21.15.26:0/2678910466 shutdown_connections 2023-12-19T18:14:45.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 -- 172.21.15.26:0/2678910466 wait complete. 2023-12-19T18:14:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 Processor -- start 2023-12-19T18:14:45.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 -- start start 2023-12-19T18:14:45.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf7006b3d0 0x7fcf7014bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:45.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf700cec40 0x7fcf7014d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:45.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf70154220 0x7fcf700b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7014e9e0 con 0x7fcf7006b3d0 2023-12-19T18:14:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7014dd60 con 0x7fcf70154220 2023-12-19T18:14:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf77ae5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7014e040 con 0x7fcf700cec40 2023-12-19T18:14:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf75881640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf7006b3d0 0x7fcf7014bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.732+0000 7fcf75881640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf7006b3d0 0x7fcf7014bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34132/0 (socket says 172.21.15.26:34132) 2023-12-19T18:14:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf75881640 1 -- 172.21.15.26:0/386187157 learned_addr learned my addr 172.21.15.26:0/386187157 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf75080640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf700cec40 0x7fcf7014d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf76082640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf70154220 0x7fcf700b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf75881640 1 -- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf700cec40 msgr2=0x7fcf7014d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:45.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf75881640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf700cec40 0x7fcf7014d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf75881640 1 -- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf70154220 msgr2=0x7fcf700b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf75881640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf70154220 0x7fcf700b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf75881640 1 -- 172.21.15.26:0/386187157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c0094d0 con 0x7fcf7006b3d0 2023-12-19T18:14:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf75080640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf700cec40 0x7fcf7014d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf76082640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf70154220 0x7fcf700b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf75881640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf7006b3d0 0x7fcf7014bba0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0b1040 tx=0x7fcf6c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf667fc640 1 -- 172.21.15.26:0/386187157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0b87f0 con 0x7fcf7006b3d0 2023-12-19T18:14:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf667fc640 1 -- 172.21.15.26:0/386187157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c0b8970 con 0x7fcf7006b3d0 2023-12-19T18:14:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf667fc640 1 -- 172.21.15.26:0/386187157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0b8d20 con 0x7fcf7006b3d0 2023-12-19T18:14:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf77ae5640 1 -- 172.21.15.26:0/386187157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7014e320 con 0x7fcf7006b3d0 2023-12-19T18:14:45.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf77ae5640 1 -- 172.21.15.26:0/386187157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7014c0e0 con 0x7fcf7006b3d0 2023-12-19T18:14:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf77ae5640 1 -- 172.21.15.26:0/386187157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf70074270 con 0x7fcf7006b3d0 2023-12-19T18:14:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf667fc640 1 -- 172.21.15.26:0/386187157 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fcf6c0b9480 con 0x7fcf7006b3d0 2023-12-19T18:14:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.736+0000 7fcf667fc640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcf54065b80 0x7fcf54068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.740+0000 7fcf667fc640 1 -- 172.21.15.26:0/386187157 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fcf6c11e410 con 0x7fcf7006b3d0 2023-12-19T18:14:45.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.740+0000 7fcf75080640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcf54065b80 0x7fcf54068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:45.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.740+0000 7fcf75080640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcf54065b80 0x7fcf54068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcf700afcd0 tx=0x7fcf60008040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:45.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.740+0000 7fcf667fc640 1 -- 172.21.15.26:0/386187157 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcf6c0eba70 con 0x7fcf7006b3d0 2023-12-19T18:14:45.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.920+0000 7fcf77ae5640 1 -- 172.21.15.26:0/386187157 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf700ccef0 con 0x7fcf54065b80 2023-12-19T18:14:45.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.920+0000 7fcf667fc640 1 -- 172.21.15.26:0/386187157 <== mgr.8276 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fcf6000b6e0 con 0x7fcf54065b80 2023-12-19T18:14:45.923 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T18:14:45.923 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:14:45.923 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2023-12-19T18:14:45.923 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:14:45.923 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T18:14:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.924+0000 7fcf77ae5640 1 -- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcf54065b80 msgr2=0x7fcf54068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.924+0000 7fcf77ae5640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcf54065b80 0x7fcf54068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcf700afcd0 tx=0x7fcf60008040 comp rx=0 tx=0).stop 2023-12-19T18:14:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.924+0000 7fcf77ae5640 1 -- 172.21.15.26:0/386187157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf7006b3d0 msgr2=0x7fcf7014bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.924+0000 7fcf77ae5640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf7006b3d0 0x7fcf7014bba0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0b1040 tx=0x7fcf6c01cc60 comp rx=0 tx=0).stop 2023-12-19T18:14:45.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.924+0000 7fcf77ae5640 1 -- 172.21.15.26:0/386187157 shutdown_connections 2023-12-19T18:14:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.924+0000 7fcf77ae5640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcf54065b80 0x7fcf54068040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.924+0000 7fcf77ae5640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf70154220 0x7fcf700b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.924+0000 7fcf77ae5640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf700cec40 0x7fcf7014d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.924+0000 7fcf77ae5640 1 --2- 172.21.15.26:0/386187157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf7006b3d0 0x7fcf7014bba0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.924+0000 7fcf77ae5640 1 -- 172.21.15.26:0/386187157 >> 172.21.15.26:0/386187157 conn(0x7fcf700658e0 msgr2=0x7fcf700cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.924+0000 7fcf77ae5640 1 -- 172.21.15.26:0/386187157 shutdown_connections 2023-12-19T18:14:45.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:45.924+0000 7fcf77ae5640 1 -- 172.21.15.26:0/386187157 wait complete. 2023-12-19T18:14:45.945 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-19T18:14:46.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.088+0000 7f25a11db640 1 Processor -- start 2023-12-19T18:14:46.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.088+0000 7f25a11db640 1 -- start start 2023-12-19T18:14:46.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.092+0000 7f25a11db640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f259c1599e0 0x7f259c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:46.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.092+0000 7f25a11db640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c15e240 con 0x7f259c0cb150 2023-12-19T18:14:46.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.092+0000 7f25a11db640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c0c9520 con 0x7f259c15dc40 2023-12-19T18:14:46.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.092+0000 7f25a11db640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259c15e050 con 0x7f259c1599e0 2023-12-19T18:14:46.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.092+0000 7f259ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f259c1599e0 0x7f259c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:46.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.092+0000 7f2593fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f259c0cb150 0x7f259c15d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53250/0 (socket says 172.21.15.26:53250) 2023-12-19T18:14:46.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.092+0000 7f2593fff640 1 -- 172.21.15.26:0/3598401580 learned_addr learned my addr 172.21.15.26:0/3598401580 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:46.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.092+0000 7f259ad9d640 1 -- 172.21.15.26:0/3598401580 >> v1:172.21.15.160:6789/0 conn(0x7f259c15dc40 legacy=0x7f259c0ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:46.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.092+0000 7f259ad9d640 1 -- 172.21.15.26:0/3598401580 >> v1:172.21.15.26:6789/0 conn(0x7f259c0cb150 legacy=0x7f259c15d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:46.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.092+0000 7f259ad9d640 1 -- 172.21.15.26:0/3598401580 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c0c9800 con 0x7f259c1599e0 2023-12-19T18:14:46.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.096+0000 7f259ad9d640 1 --2- 172.21.15.26:0/3598401580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c1599e0 0x7f259c06f6d0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f258c0097c0 tx=0x7f258c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d51e22e406a4134 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:46.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.096+0000 7f259a59c640 1 -- 172.21.15.26:0/3598401580 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c0caf20 con 0x7f259c1599e0 2023-12-19T18:14:46.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.096+0000 7f259a59c640 1 -- 172.21.15.26:0/3598401580 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c0a7070 con 0x7f259c1599e0 2023-12-19T18:14:46.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.096+0000 7f259a59c640 1 -- 172.21.15.26:0/3598401580 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0ab860 con 0x7f259c1599e0 2023-12-19T18:14:46.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.096+0000 7f25a11db640 1 -- 172.21.15.26:0/3598401580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c1599e0 msgr2=0x7f259c06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:46.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.096+0000 7f25a11db640 1 --2- 172.21.15.26:0/3598401580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c1599e0 0x7f259c06f6d0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f258c0097c0 tx=0x7f258c01c480 comp rx=0 tx=0).stop 2023-12-19T18:14:46.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.096+0000 7f25a11db640 1 -- 172.21.15.26:0/3598401580 shutdown_connections 2023-12-19T18:14:46.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.096+0000 7f25a11db640 1 --2- 172.21.15.26:0/3598401580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c1599e0 0x7f259c06f6d0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.096+0000 7f25a11db640 1 -- 172.21.15.26:0/3598401580 >> 172.21.15.26:0/3598401580 conn(0x7f259c066160 msgr2=0x7f259c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:46.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.096+0000 7f25a11db640 1 -- 172.21.15.26:0/3598401580 shutdown_connections 2023-12-19T18:14:46.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.096+0000 7f25a11db640 1 -- 172.21.15.26:0/3598401580 wait complete. 2023-12-19T18:14:46.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.096+0000 7f25a11db640 1 Processor -- start 2023-12-19T18:14:46.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.096+0000 7f25a11db640 1 -- start start 2023-12-19T18:14:46.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f25a11db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c0cb150 0x7f259c14f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:46.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f25a11db640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f259c1599e0 0x7f259c0c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:46.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f25a11db640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f259c15dc40 0x7f259c1519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:46.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f25a11db640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0c3060 con 0x7f259c0cb150 2023-12-19T18:14:46.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f25a11db640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c1574e0 con 0x7f259c1599e0 2023-12-19T18:14:46.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f25a11db640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0c4d70 con 0x7f259c15dc40 2023-12-19T18:14:46.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f259ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c0cb150 0x7f259c14f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:46.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f259ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c0cb150 0x7f259c14f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37166/0 (socket says 172.21.15.26:37166) 2023-12-19T18:14:46.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f259ad9d640 1 -- 172.21.15.26:0/2476760499 learned_addr learned my addr 172.21.15.26:0/2476760499 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f2593fff640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f259c1599e0 0x7f259c0c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f259b59e640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f259c15dc40 0x7f259c1519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f259ad9d640 1 -- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f259c15dc40 msgr2=0x7f259c1519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f259ad9d640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f259c15dc40 0x7f259c1519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f259ad9d640 1 -- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f259c1599e0 msgr2=0x7f259c0c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f259ad9d640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f259c1599e0 0x7f259c0c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f259ad9d640 1 -- 172.21.15.26:0/2476760499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c009420 con 0x7f259c0cb150 2023-12-19T18:14:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f259b59e640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f259c15dc40 0x7f259c1519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f2593fff640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f259c1599e0 0x7f259c0c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f259ad9d640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c0cb150 0x7f259c14f790 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f258c0ad040 tx=0x7f258c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f25937fe640 1 -- 172.21.15.26:0/2476760499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0bf410 con 0x7f259c0cb150 2023-12-19T18:14:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f25937fe640 1 -- 172.21.15.26:0/2476760499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c0bf590 con 0x7f259c0cb150 2023-12-19T18:14:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f25937fe640 1 -- 172.21.15.26:0/2476760499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0bf940 con 0x7f259c0cb150 2023-12-19T18:14:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f25a11db640 1 -- 172.21.15.26:0/2476760499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c0cbaf0 con 0x7f259c0cb150 2023-12-19T18:14:46.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f25a11db640 1 -- 172.21.15.26:0/2476760499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c157100 con 0x7f259c0cb150 2023-12-19T18:14:46.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f25a11db640 1 -- 172.21.15.26:0/2476760499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259c074270 con 0x7f259c0cb150 2023-12-19T18:14:46.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.100+0000 7f25937fe640 1 -- 172.21.15.26:0/2476760499 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 107) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f258c0b5400 con 0x7f259c0cb150 2023-12-19T18:14:46.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.104+0000 7f25937fe640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f257c065bd0 0x7f257c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:46.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.104+0000 7f25937fe640 1 -- 172.21.15.26:0/2476760499 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f258c11e1e0 con 0x7f259c0cb150 2023-12-19T18:14:46.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.104+0000 7f2593fff640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f257c065bd0 0x7f257c068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:46.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.104+0000 7f2593fff640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f257c065bd0 0x7f257c068090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f259c0c82d0 tx=0x7f2584009040 comp rx=0 tx=0).ready entity=mgr.8276 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:46.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.108+0000 7f25937fe640 1 -- 172.21.15.26:0/2476760499 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f258c0eb680 con 0x7f259c0cb150 2023-12-19T18:14:46.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.304+0000 7f25a11db640 1 -- 172.21.15.26:0/2476760499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f259c0c2050 con 0x7f259c0cb150 2023-12-19T18:14:46.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.568+0000 7f25937fe640 1 -- 172.21.15.26:0/2476760499 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 108) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f258c0eba50 con 0x7f259c0cb150 2023-12-19T18:14:46.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.568+0000 7f25937fe640 1 -- 172.21.15.26:0/2476760499 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v108) v1 ==== 92+0+0 (secure 0 0 0) 0x7f258c0eee80 con 0x7f259c0cb150 2023-12-19T18:14:46.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.572+0000 7f25a11db640 1 -- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f257c065bd0 msgr2=0x7f257c068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:46.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.572+0000 7f25a11db640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f257c065bd0 0x7f257c068090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f259c0c82d0 tx=0x7f2584009040 comp rx=0 tx=0).stop 2023-12-19T18:14:46.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.572+0000 7f25a11db640 1 -- 172.21.15.26:0/2476760499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c0cb150 msgr2=0x7f259c14f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:46.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.572+0000 7f25a11db640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c0cb150 0x7f259c14f790 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f258c0ad040 tx=0x7f258c01acc0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.572+0000 7f25a11db640 1 -- 172.21.15.26:0/2476760499 shutdown_connections 2023-12-19T18:14:46.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.572+0000 7f25a11db640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f257c065bd0 0x7f257c068090 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.572+0000 7f25a11db640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f259c15dc40 0x7f259c1519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.572+0000 7f25a11db640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f259c1599e0 0x7f259c0c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.572+0000 7f25a11db640 1 --2- 172.21.15.26:0/2476760499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f259c0cb150 0x7f259c14f790 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.572+0000 7f25a11db640 1 -- 172.21.15.26:0/2476760499 >> 172.21.15.26:0/2476760499 conn(0x7f259c066160 msgr2=0x7f259c0ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:46.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.572+0000 7f25a11db640 1 -- 172.21.15.26:0/2476760499 shutdown_connections 2023-12-19T18:14:46.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.572+0000 7f25a11db640 1 -- 172.21.15.26:0/2476760499 wait complete. 2023-12-19T18:14:46.601 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-19T18:14:46.750 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:46.747+0000 7f6c44528180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:14:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52ef0f2640 1 Processor -- start 2023-12-19T18:14:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52ef0f2640 1 -- start start 2023-12-19T18:14:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52ef0f2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f52e806db00 0x7f52e806bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52ef0f2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52e80ccc90 0x7f52e80cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52ef0f2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e80ca360 con 0x7f52e80ad7f0 2023-12-19T18:14:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52ef0f2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e80cee80 con 0x7f52e80ccc90 2023-12-19T18:14:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52ef0f2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e80cd650 con 0x7f52e806db00 2023-12-19T18:14:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52e7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52e80ccc90 0x7f52e80cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52ed68f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f52e80ad7f0 0x7f52e80adc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53270/0 (socket says 172.21.15.26:53270) 2023-12-19T18:14:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52ed68f640 1 -- 172.21.15.26:0/3342798771 learned_addr learned my addr 172.21.15.26:0/3342798771 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52ece8e640 1 --2- 172.21.15.26:0/3342798771 >> v2:172.21.15.160:3300/0 conn(0x7f52e806db00 0x7f52e806bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52e7fff640 1 -- 172.21.15.26:0/3342798771 >> v2:172.21.15.160:3300/0 conn(0x7f52e806db00 msgr2=0x7f52e806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52e7fff640 1 --2- 172.21.15.26:0/3342798771 >> v2:172.21.15.160:3300/0 conn(0x7f52e806db00 0x7f52e806bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52e7fff640 1 -- 172.21.15.26:0/3342798771 >> v1:172.21.15.26:6789/0 conn(0x7f52e80ad7f0 legacy=0x7f52e80adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.751+0000 7f52e7fff640 1 -- 172.21.15.26:0/3342798771 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e80c9750 con 0x7f52e80ccc90 2023-12-19T18:14:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52e7fff640 1 --2- 172.21.15.26:0/3342798771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e80ccc90 0x7f52e80cd110 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f52e80ccb40 tx=0x7f52d401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efcffde0c124a43 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52e77fe640 1 -- 172.21.15.26:0/3342798771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40ac630 con 0x7f52e80ccc90 2023-12-19T18:14:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52e77fe640 1 -- 172.21.15.26:0/3342798771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d40ac790 con 0x7f52e80ccc90 2023-12-19T18:14:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52e77fe640 1 -- 172.21.15.26:0/3342798771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40acb20 con 0x7f52e80ccc90 2023-12-19T18:14:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52ef0f2640 1 -- 172.21.15.26:0/3342798771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e80ccc90 msgr2=0x7f52e80cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52ef0f2640 1 --2- 172.21.15.26:0/3342798771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e80ccc90 0x7f52e80cd110 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f52e80ccb40 tx=0x7f52d401c3d0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52ef0f2640 1 -- 172.21.15.26:0/3342798771 shutdown_connections 2023-12-19T18:14:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52ef0f2640 1 --2- 172.21.15.26:0/3342798771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e80ccc90 0x7f52e80cd110 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52ef0f2640 1 --2- 172.21.15.26:0/3342798771 >> v2:172.21.15.160:3300/0 conn(0x7f52e806db00 0x7f52e806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52ef0f2640 1 -- 172.21.15.26:0/3342798771 >> 172.21.15.26:0/3342798771 conn(0x7f52e80b0b60 msgr2=0x7f52e80b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52ef0f2640 1 -- 172.21.15.26:0/3342798771 shutdown_connections 2023-12-19T18:14:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52ef0f2640 1 -- 172.21.15.26:0/3342798771 wait complete. 2023-12-19T18:14:46.758 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f90dba49180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:14:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52ef0f2640 1 Processor -- start 2023-12-19T18:14:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.755+0000 7f52ef0f2640 1 -- start start 2023-12-19T18:14:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ef0f2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52e806db00 0x7f52e815d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ef0f2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52e80ad7f0 0x7f52e8152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ef0f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e80ccc90 0x7f52e8153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ef0f2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8071910 con 0x7f52e80ccc90 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ef0f2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e806fa10 con 0x7f52e806db00 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ef0f2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8153890 con 0x7f52e80ad7f0 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ed68f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e80ccc90 0x7f52e8153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ed68f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e80ccc90 0x7f52e8153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37194/0 (socket says 172.21.15.26:37194) 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ece8e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52e806db00 0x7f52e815d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ed68f640 1 -- 172.21.15.26:0/2277484293 learned_addr learned my addr 172.21.15.26:0/2277484293 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52e7fff640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52e80ad7f0 0x7f52e8152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ed68f640 1 -- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52e80ad7f0 msgr2=0x7f52e8152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ed68f640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52e80ad7f0 0x7f52e8152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ed68f640 1 -- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52e806db00 msgr2=0x7f52e815d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ed68f640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52e806db00 0x7f52e815d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ed68f640 1 -- 172.21.15.26:0/2277484293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d4009420 con 0x7f52e80ccc90 2023-12-19T18:14:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ece8e640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52e806db00 0x7f52e815d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52e7fff640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52e80ad7f0 0x7f52e8152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ed68f640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e80ccc90 0x7f52e8153350 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f52e000cbc0 tx=0x7f52e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52e57fa640 1 -- 172.21.15.26:0/2277484293 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e000ad40 con 0x7f52e80ccc90 2023-12-19T18:14:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52e57fa640 1 -- 172.21.15.26:0/2277484293 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e0012400 con 0x7f52e80ccc90 2023-12-19T18:14:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52e57fa640 1 -- 172.21.15.26:0/2277484293 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e000aea0 con 0x7f52e80ccc90 2023-12-19T18:14:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ef0f2640 1 -- 172.21.15.26:0/2277484293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e8155120 con 0x7f52e80ccc90 2023-12-19T18:14:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ef0f2640 1 -- 172.21.15.26:0/2277484293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e8071e00 con 0x7f52e80ccc90 2023-12-19T18:14:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.759+0000 7f52ef0f2640 1 -- 172.21.15.26:0/2277484293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e806fb80 con 0x7f52e80ccc90 2023-12-19T18:14:46.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.763+0000 7f52e57fa640 1 -- 172.21.15.26:0/2277484293 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 108) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f52e0007070 con 0x7f52e80ccc90 2023-12-19T18:14:46.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.763+0000 7f52e57fa640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52cc065bd0 0x7f52cc068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:46.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.763+0000 7f52e57fa640 1 -- 172.21.15.26:0/2277484293 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f52e00834e0 con 0x7f52e80ccc90 2023-12-19T18:14:46.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.763+0000 7f52ece8e640 1 -- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52cc065bd0 msgr2=0x7f52cc068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:46.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.763+0000 7f52ece8e640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52cc065bd0 0x7f52cc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:46.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.771+0000 7f52e57fa640 1 -- 172.21.15.26:0/2277484293 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f52e0050b50 con 0x7f52e80ccc90 2023-12-19T18:14:46.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.931+0000 7f52ef0f2640 1 -- 172.21.15.26:0/2277484293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f52e8074270 con 0x7f52e80ccc90 2023-12-19T18:14:46.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.935+0000 7f52e57fa640 1 -- 172.21.15.26:0/2277484293 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f52e0054170 con 0x7f52e80ccc90 2023-12-19T18:14:46.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.939+0000 7f52ef0f2640 1 -- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52cc065bd0 msgr2=0x7f52cc068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:14:46.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.939+0000 7f52ef0f2640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52cc065bd0 0x7f52cc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.939+0000 7f52ef0f2640 1 -- 172.21.15.26:0/2277484293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e80ccc90 msgr2=0x7f52e8153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.939+0000 7f52ef0f2640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e80ccc90 0x7f52e8153350 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f52e000cbc0 tx=0x7f52e000a430 comp rx=0 tx=0).stop 2023-12-19T18:14:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.939+0000 7f52ef0f2640 1 -- 172.21.15.26:0/2277484293 shutdown_connections 2023-12-19T18:14:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.939+0000 7f52ef0f2640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52cc065bd0 0x7f52cc068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.939+0000 7f52ef0f2640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e80ccc90 0x7f52e8153350 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.939+0000 7f52ef0f2640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52e80ad7f0 0x7f52e8152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.939+0000 7f52ef0f2640 1 --2- 172.21.15.26:0/2277484293 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52e806db00 0x7f52e815d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.939+0000 7f52ef0f2640 1 -- 172.21.15.26:0/2277484293 >> 172.21.15.26:0/2277484293 conn(0x7f52e80b0b60 msgr2=0x7f52e80adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.939+0000 7f52ef0f2640 1 -- 172.21.15.26:0/2277484293 shutdown_connections 2023-12-19T18:14:46.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:46.939+0000 7f52ef0f2640 1 -- 172.21.15.26:0/2277484293 wait complete. 2023-12-19T18:14:46.961 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-19T18:14:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.107+0000 7fbc57273640 1 Processor -- start 2023-12-19T18:14:47.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.107+0000 7fbc57273640 1 -- start start 2023-12-19T18:14:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc57273640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbc500b8600 0x7fbc500b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:47.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc57273640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbc500b4b30 0x7fbc500d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc57273640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc500b5bd0 con 0x7fbc5015cde0 2023-12-19T18:14:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc57273640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5015e380 con 0x7fbc500b8600 2023-12-19T18:14:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc57273640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc500b4fc0 con 0x7fbc500b4b30 2023-12-19T18:14:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc5480e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbc500b8600 0x7fbc500b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc5480e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbc500b8600 0x7fbc500b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:58620/0 (socket says 172.21.15.26:58620) 2023-12-19T18:14:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc5500f640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fbc5015cde0 0x7fbc5015d1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:41992/0 (socket says 172.21.15.26:41992) 2023-12-19T18:14:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc5480e640 1 -- 172.21.15.26:0/3885164691 learned_addr learned my addr 172.21.15.26:0/3885164691 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:47.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc55810640 1 --2- 172.21.15.26:0/3885164691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbc500b4b30 0x7fbc500d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc5480e640 1 -- 172.21.15.26:0/3885164691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbc500b4b30 msgr2=0x7fbc500d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc5480e640 1 --2- 172.21.15.26:0/3885164691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbc500b4b30 0x7fbc500d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc5480e640 1 -- 172.21.15.26:0/3885164691 >> v1:172.21.15.160:6789/0 conn(0x7fbc5015cde0 legacy=0x7fbc5015d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc5480e640 1 -- 172.21.15.26:0/3885164691 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc500d0ee0 con 0x7fbc500b8600 2023-12-19T18:14:47.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.111+0000 7fbc47fff640 1 -- 172.21.15.26:0/3885164691 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885985038 0 0) 0x7fbc500b5bd0 con 0x7fbc5015cde0 2023-12-19T18:14:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc5480e640 1 --2- 172.21.15.26:0/3885164691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc500b8600 0x7fbc500b45f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009870 tx=0x7fbc4001ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5bd44a5a2d9f412 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc47fff640 1 -- 172.21.15.26:0/3885164691 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc400a8a30 con 0x7fbc500b8600 2023-12-19T18:14:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc47fff640 1 -- 172.21.15.26:0/3885164691 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc400a8b90 con 0x7fbc500b8600 2023-12-19T18:14:47.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc47fff640 1 -- 172.21.15.26:0/3885164691 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc400b0440 con 0x7fbc500b8600 2023-12-19T18:14:47.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc57273640 1 -- 172.21.15.26:0/3885164691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc500b8600 msgr2=0x7fbc500b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:47.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc57273640 1 --2- 172.21.15.26:0/3885164691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc500b8600 0x7fbc500b45f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009870 tx=0x7fbc4001ed30 comp rx=0 tx=0).stop 2023-12-19T18:14:47.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc57273640 1 -- 172.21.15.26:0/3885164691 shutdown_connections 2023-12-19T18:14:47.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc57273640 1 --2- 172.21.15.26:0/3885164691 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbc500b4b30 0x7fbc500d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc57273640 1 --2- 172.21.15.26:0/3885164691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc500b8600 0x7fbc500b45f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc57273640 1 -- 172.21.15.26:0/3885164691 >> 172.21.15.26:0/3885164691 conn(0x7fbc50065ee0 msgr2=0x7fbc500d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:47.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc57273640 1 -- 172.21.15.26:0/3885164691 shutdown_connections 2023-12-19T18:14:47.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc57273640 1 -- 172.21.15.26:0/3885164691 wait complete. 2023-12-19T18:14:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc57273640 1 Processor -- start 2023-12-19T18:14:47.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.115+0000 7fbc57273640 1 -- start start 2023-12-19T18:14:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc57273640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc500b4b30 0x7fbc500cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc57273640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbc500b8600 0x7fbc501f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:47.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc57273640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc5015cde0 0x7fbc50150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc57273640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc500abda0 con 0x7fbc500b4b30 2023-12-19T18:14:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc57273640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc501f5b60 con 0x7fbc5015cde0 2023-12-19T18:14:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc57273640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc501f2ab0 con 0x7fbc500b8600 2023-12-19T18:14:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc5500f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc500b4b30 0x7fbc500cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc5500f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc500b4b30 0x7fbc500cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37204/0 (socket says 172.21.15.26:37204) 2023-12-19T18:14:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc5500f640 1 -- 172.21.15.26:0/3939740118 learned_addr learned my addr 172.21.15.26:0/3939740118 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc5480e640 1 --2- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbc500b8600 0x7fbc501f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc55810640 1 --2- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc5015cde0 0x7fbc50150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc5500f640 1 -- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbc500b8600 msgr2=0x7fbc501f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc5500f640 1 --2- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbc500b8600 0x7fbc501f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc5500f640 1 -- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc5015cde0 msgr2=0x7fbc50150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc5500f640 1 --2- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc5015cde0 0x7fbc50150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc5500f640 1 -- 172.21.15.26:0/3939740118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc400094d0 con 0x7fbc500b4b30 2023-12-19T18:14:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc5500f640 1 --2- 172.21.15.26:0/3939740118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc500b4b30 0x7fbc500cbf60 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00c410 tx=0x7fbc4c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc45ffb640 1 -- 172.21.15.26:0/3939740118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0074e0 con 0x7fbc500b4b30 2023-12-19T18:14:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc45ffb640 1 -- 172.21.15.26:0/3939740118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c007640 con 0x7fbc500b4b30 2023-12-19T18:14:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc57273640 1 -- 172.21.15.26:0/3939740118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc501f2d30 con 0x7fbc500b4b30 2023-12-19T18:14:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc45ffb640 1 -- 172.21.15.26:0/3939740118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c004a30 con 0x7fbc500b4b30 2023-12-19T18:14:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc57273640 1 -- 172.21.15.26:0/3939740118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc501f30c0 con 0x7fbc500b4b30 2023-12-19T18:14:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc57273640 1 -- 172.21.15.26:0/3939740118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc500ccef0 con 0x7fbc500b4b30 2023-12-19T18:14:47.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.119+0000 7fbc45ffb640 1 -- 172.21.15.26:0/3939740118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 108) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fbc4c004b90 con 0x7fbc500b4b30 2023-12-19T18:14:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.123+0000 7fbc45ffb640 1 --2- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbc34065bd0 0x7fbc34068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.123+0000 7fbc45ffb640 1 -- 172.21.15.26:0/3939740118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fbc4c0832e0 con 0x7fbc500b4b30 2023-12-19T18:14:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.123+0000 7fbc5480e640 1 -- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbc34065bd0 msgr2=0x7fbc34068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:47.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.123+0000 7fbc5480e640 1 --2- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbc34065bd0 0x7fbc34068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:47.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.127+0000 7fbc45ffb640 1 -- 172.21.15.26:0/3939740118 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbc4c050960 con 0x7fbc500b4b30 2023-12-19T18:14:47.172 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:47.171+0000 7f6c44528180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:14:47.206 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:47.203+0000 7f90dba49180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:14:47.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.287+0000 7fbc57273640 1 -- 172.21.15.26:0/3939740118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fbc50074270 con 0x7fbc500b4b30 2023-12-19T18:14:47.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.291+0000 7fbc45ffb640 1 -- 172.21.15.26:0/3939740118 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) v1 ==== 133+0+0 (secure 0 0 0) 0x7fbc4c053f80 con 0x7fbc500b4b30 2023-12-19T18:14:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.295+0000 7fbc57273640 1 -- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbc34065bd0 msgr2=0x7fbc34068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:14:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.295+0000 7fbc57273640 1 --2- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbc34065bd0 0x7fbc34068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.295+0000 7fbc57273640 1 -- 172.21.15.26:0/3939740118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc500b4b30 msgr2=0x7fbc500cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.295+0000 7fbc57273640 1 --2- 172.21.15.26:0/3939740118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc500b4b30 0x7fbc500cbf60 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00c410 tx=0x7fbc4c00c8d0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.295+0000 7fbc57273640 1 -- 172.21.15.26:0/3939740118 shutdown_connections 2023-12-19T18:14:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.295+0000 7fbc57273640 1 --2- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbc34065bd0 0x7fbc34068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.295+0000 7fbc57273640 1 --2- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbc5015cde0 0x7fbc50150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.295+0000 7fbc57273640 1 --2- 172.21.15.26:0/3939740118 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbc500b8600 0x7fbc501f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.295+0000 7fbc57273640 1 --2- 172.21.15.26:0/3939740118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbc500b4b30 0x7fbc500cbf60 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.295+0000 7fbc57273640 1 -- 172.21.15.26:0/3939740118 >> 172.21.15.26:0/3939740118 conn(0x7fbc50065ee0 msgr2=0x7fbc5015f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.295+0000 7fbc57273640 1 -- 172.21.15.26:0/3939740118 shutdown_connections 2023-12-19T18:14:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.295+0000 7fbc57273640 1 -- 172.21.15.26:0/3939740118 wait complete. 2023-12-19T18:14:47.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:47.317 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-19T18:14:47.324 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:47.323+0000 7f6c44528180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:14:47.333 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1M 2023-12-19T18:14:47.366 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:47.363+0000 7f90dba49180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:14:47.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.511+0000 7ff1addf0640 1 Processor -- start 2023-12-19T18:14:47.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.511+0000 7ff1addf0640 1 -- start start 2023-12-19T18:14:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.511+0000 7ff1addf0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1a8159820 0x7ff1a815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:47.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.511+0000 7ff1addf0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff1a806f6c0 0x7ff1a80c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.511+0000 7ff1addf0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1a8159c00 0x7ff1a8070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.511+0000 7ff1addf0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a815e470 con 0x7ff1a8159820 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.511+0000 7ff1addf0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a806fb60 con 0x7ff1a806f6c0 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.511+0000 7ff1addf0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a806fe40 con 0x7ff1a8159c00 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.515+0000 7ff1a77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1a8159820 0x7ff1a815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.515+0000 7ff1a77fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1a8159820 0x7ff1a815df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37208/0 (socket says 172.21.15.26:37208) 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.515+0000 7ff1a77fe640 1 -- 172.21.15.26:0/4105794597 learned_addr learned my addr 172.21.15.26:0/4105794597 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.515+0000 7ff1a7fff640 1 --2- 172.21.15.26:0/4105794597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1a8159c00 0x7ff1a8070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.515+0000 7ff1a6ffd640 1 --2- 172.21.15.26:0/4105794597 >> v2:172.21.15.160:3300/0 conn(0x7ff1a806f6c0 0x7ff1a80c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.515+0000 7ff1a77fe640 1 -- 172.21.15.26:0/4105794597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1a8159c00 msgr2=0x7ff1a8070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.515+0000 7ff1a77fe640 1 --2- 172.21.15.26:0/4105794597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1a8159c00 0x7ff1a8070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.515+0000 7ff1a77fe640 1 -- 172.21.15.26:0/4105794597 >> v2:172.21.15.160:3300/0 conn(0x7ff1a806f6c0 msgr2=0x7ff1a80c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.515+0000 7ff1a77fe640 1 --2- 172.21.15.26:0/4105794597 >> v2:172.21.15.160:3300/0 conn(0x7ff1a806f6c0 0x7ff1a80c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.515+0000 7ff1a77fe640 1 -- 172.21.15.26:0/4105794597 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a8070120 con 0x7ff1a8159820 2023-12-19T18:14:47.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.515+0000 7ff1a77fe640 1 --2- 172.21.15.26:0/4105794597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a8159820 0x7ff1a815df30 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80caf30 tx=0x7ff19801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bab855a44d446e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1a67fc640 1 -- 172.21.15.26:0/4105794597 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1980aa5f0 con 0x7ff1a8159820 2023-12-19T18:14:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1a67fc640 1 -- 172.21.15.26:0/4105794597 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1980aa750 con 0x7ff1a8159820 2023-12-19T18:14:47.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1a67fc640 1 -- 172.21.15.26:0/4105794597 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1980aaae0 con 0x7ff1a8159820 2023-12-19T18:14:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 -- 172.21.15.26:0/4105794597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a8159820 msgr2=0x7ff1a815df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 --2- 172.21.15.26:0/4105794597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a8159820 0x7ff1a815df30 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80caf30 tx=0x7ff19801c4d0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 -- 172.21.15.26:0/4105794597 shutdown_connections 2023-12-19T18:14:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 --2- 172.21.15.26:0/4105794597 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1a8159c00 0x7ff1a8070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 --2- 172.21.15.26:0/4105794597 >> v2:172.21.15.160:3300/0 conn(0x7ff1a806f6c0 0x7ff1a80c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 --2- 172.21.15.26:0/4105794597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a8159820 0x7ff1a815df30 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 -- 172.21.15.26:0/4105794597 >> 172.21.15.26:0/4105794597 conn(0x7ff1a80ce730 msgr2=0x7ff1a80ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 -- 172.21.15.26:0/4105794597 shutdown_connections 2023-12-19T18:14:47.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 -- 172.21.15.26:0/4105794597 wait complete. 2023-12-19T18:14:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 Processor -- start 2023-12-19T18:14:47.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 -- start start 2023-12-19T18:14:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a806f6c0 0x7ff1a80cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8159820 0x7ff1a81551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1a8159c00 0x7ff1a80c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a815a020 con 0x7ff1a806f6c0 2023-12-19T18:14:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a80ce0e0 con 0x7ff1a8159820 2023-12-19T18:14:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.519+0000 7ff1addf0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a815a2c0 con 0x7ff1a8159c00 2023-12-19T18:14:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a806f6c0 0x7ff1a80cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1a6ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8159820 0x7ff1a81551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:47.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a806f6c0 0x7ff1a80cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37216/0 (socket says 172.21.15.26:37216) 2023-12-19T18:14:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1a77fe640 1 -- 172.21.15.26:0/2717743075 learned_addr learned my addr 172.21.15.26:0/2717743075 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:47.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1a7fff640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1a8159c00 0x7ff1a80c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1a77fe640 1 -- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1a8159c00 msgr2=0x7ff1a80c21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1a77fe640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1a8159c00 0x7ff1a80c21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1a77fe640 1 -- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8159820 msgr2=0x7ff1a81551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1a77fe640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8159820 0x7ff1a81551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1a77fe640 1 -- 172.21.15.26:0/2717743075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1980094d0 con 0x7ff1a806f6c0 2023-12-19T18:14:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1a77fe640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a806f6c0 0x7ff1a80cda50 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7ff19801ab80 tx=0x7ff19801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff18ffff640 1 -- 172.21.15.26:0/2717743075 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1980bd5f0 con 0x7ff1a806f6c0 2023-12-19T18:14:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff18ffff640 1 -- 172.21.15.26:0/2717743075 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1980bd750 con 0x7ff1a806f6c0 2023-12-19T18:14:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1addf0640 1 -- 172.21.15.26:0/2717743075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a806b880 con 0x7ff1a806f6c0 2023-12-19T18:14:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff18ffff640 1 -- 172.21.15.26:0/2717743075 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1980bdae0 con 0x7ff1a806f6c0 2023-12-19T18:14:47.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1addf0640 1 -- 172.21.15.26:0/2717743075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a815a880 con 0x7ff1a806f6c0 2023-12-19T18:14:47.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.523+0000 7ff1addf0640 1 -- 172.21.15.26:0/2717743075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a815aa70 con 0x7ff1a806f6c0 2023-12-19T18:14:47.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.527+0000 7ff18ffff640 1 -- 172.21.15.26:0/2717743075 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 108) v1 ==== 85347+0+0 (secure 0 0 0) 0x7ff1980b33f0 con 0x7ff1a806f6c0 2023-12-19T18:14:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.527+0000 7ff18ffff640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 0x7ff1940680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:47.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.527+0000 7ff18ffff640 1 -- 172.21.15.26:0/2717743075 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6441+0+0 (secure 0 0 0) 0x7ff1980a7070 con 0x7ff1a806f6c0 2023-12-19T18:14:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.527+0000 7ff1a6ffd640 1 -- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 msgr2=0x7ff1940680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:47.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.527+0000 7ff1a6ffd640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 0x7ff1940680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:47.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.531+0000 7ff18ffff640 1 -- 172.21.15.26:0/2717743075 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff1980eb920 con 0x7ff1a806f6c0 2023-12-19T18:14:47.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.711+0000 7ff1addf0640 1 -- 172.21.15.26:0/2717743075 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1a806b700 con 0x7ff194065c00 2023-12-19T18:14:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.731+0000 7ff1a6ffd640 1 -- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 msgr2=0x7ff1940680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:47.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:47.731+0000 7ff1a6ffd640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 0x7ff1940680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T18:14:47.806 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:47.803+0000 7f6c44528180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:14:47.865 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:47.863+0000 7f6c44528180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:14:47.880 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:47.879+0000 7f90dba49180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:14:47.921 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:47.919+0000 7f6c44528180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:14:47.941 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:47.939+0000 7f90dba49180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:14:47.999 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:47.999+0000 7f90dba49180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:14:48.007 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:48.003+0000 7f6c44528180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:14:48.066 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:48.063+0000 7f6c44528180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:14:48.089 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:48.087+0000 7f90dba49180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:14:48.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:48.135+0000 7ff1a6ffd640 1 -- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 msgr2=0x7ff1940680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:48.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:48.135+0000 7ff1a6ffd640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 0x7ff1940680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-19T18:14:48.149 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:48.147+0000 7f90dba49180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:14:48.151 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:48.147+0000 7f6c44528180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:14:48.208 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:48.207+0000 7f6c44528180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:14:48.237 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:48.235+0000 7f90dba49180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:14:48.296 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:48.295+0000 7f90dba49180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:14:48.353 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:48.351+0000 7f6c44528180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:14:48.411 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:48.407+0000 7f6c44528180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:14:48.446 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:48.443+0000 7f90dba49180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:14:48.507 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:48.507+0000 7f90dba49180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:14:48.653 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:14:48.654 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:14:48.654 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:14:48.661 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:48.659+0000 7f6c44528180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:14:48.757 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:14:48.757 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:14:48.757 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:14:48.765 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:48.763+0000 7f90dba49180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:14:48.797 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:48.795+0000 7f6c44528180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:14:48.911 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:48.907+0000 7f90dba49180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:14:49.062 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:48.919+0000 7f6c44528180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:14:49.062 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:48.991+0000 7f6c44528180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:14:49.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:48.939+0000 7ff1a6ffd640 1 -- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 msgr2=0x7ff1940680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:14:49.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:48.939+0000 7ff1a6ffd640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 0x7ff1940680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-19T18:14:49.063 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:49.039+0000 7f90dba49180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:14:49.113 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:49.111+0000 7f90dba49180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:14:49.131 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:49.127+0000 7f6c44528180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:14:49.255 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:49.255+0000 7f90dba49180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:14:49.671 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:49.667+0000 7f6c44528180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:14:49.756 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:49.755+0000 7f6c44528180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:14:49.824 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:49.823+0000 7f90dba49180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:14:49.906 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:49.903+0000 7f6c44528180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:14:49.912 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:49.911+0000 7f90dba49180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:14:49.989 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:49.987+0000 7f6c44528180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:14:50.068 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:50.067+0000 7f90dba49180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:14:50.155 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:50.155+0000 7f90dba49180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:14:50.309 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:50.307+0000 7f6c44528180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:14:50.370 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:50.367+0000 7f6c44528180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:14:50.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:50.379+0000 7ff18ffff640 1 -- 172.21.15.26:0/2717743075 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 109) v1 ==== 84956+0+0 (secure 0 0 0) 0x7ff1980ebcf0 con 0x7ff1a806f6c0 2023-12-19T18:14:50.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:50.379+0000 7ff18ffff640 1 -- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 msgr2=0x7ff1940680c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:14:50.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:50.379+0000 7ff18ffff640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 0x7ff1940680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.498 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:50.495+0000 7f90dba49180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:14:50.499 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:50.495+0000 7f6bc7489640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:50.499 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:50.495+0000 7f6bc7489640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:50.499 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:50.495+0000 7f6bc7489640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:50.499 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:50.495+0000 7f6bc7489640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:50.499 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:50.495+0000 7f6bc7489640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:50.500 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:50.499+0000 7f6bca48f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:50.500 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:50.499+0000 7f6bca48f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:50.500 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:50.499+0000 7f6bca48f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:50.500 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:50.499+0000 7f6bca48f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:50.500 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:14:50.499+0000 7f6bca48f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:14:50.562 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:14:50.559+0000 7f90dba49180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:14:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.387+0000 7ff18ffff640 1 -- 172.21.15.26:0/2717743075 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 110) v1 ==== 85083+0+0 (secure 0 0 0) 0x7ff19811c5e0 con 0x7ff1a806f6c0 2023-12-19T18:14:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.387+0000 7ff18ffff640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 0x7ff1940680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.387+0000 7ff18ffff640 1 -- 172.21.15.26:0/2717743075 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff19406bc00 con 0x7ff194065c00 2023-12-19T18:14:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.391+0000 7ff1a6ffd640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 0x7ff1940680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.395+0000 7ff1a6ffd640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 0x7ff1940680c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff1a806b540 tx=0x7ff19c009190 comp rx=0 tx=0).ready entity=mgr.9700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:51.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.459+0000 7ff18ffff640 1 -- 172.21.15.26:0/2717743075 <== mgr.9700 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff19406bc00 con 0x7ff194065c00 2023-12-19T18:14:51.461 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path snap_test_dir1 2023-12-19T18:14:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.463+0000 7ff1addf0640 1 -- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 msgr2=0x7ff1940680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.463+0000 7ff1addf0640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 0x7ff1940680c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff1a806b540 tx=0x7ff19c009190 comp rx=0 tx=0).stop 2023-12-19T18:14:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.463+0000 7ff1addf0640 1 -- 172.21.15.26:0/2717743075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a806f6c0 msgr2=0x7ff1a80cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.463+0000 7ff1addf0640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a806f6c0 0x7ff1a80cda50 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7ff19801ab80 tx=0x7ff19801ad10 comp rx=0 tx=0).stop 2023-12-19T18:14:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.463+0000 7ff1addf0640 1 -- 172.21.15.26:0/2717743075 shutdown_connections 2023-12-19T18:14:51.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.463+0000 7ff1addf0640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff194065c00 0x7ff1940680c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.463+0000 7ff1addf0640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1a8159c00 0x7ff1a80c21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.463+0000 7ff1addf0640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8159820 0x7ff1a81551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.463+0000 7ff1addf0640 1 --2- 172.21.15.26:0/2717743075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1a806f6c0 0x7ff1a80cda50 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.463+0000 7ff1addf0640 1 -- 172.21.15.26:0/2717743075 >> 172.21.15.26:0/2717743075 conn(0x7ff1a80ce730 msgr2=0x7ff1a80af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.463+0000 7ff1addf0640 1 -- 172.21.15.26:0/2717743075 shutdown_connections 2023-12-19T18:14:51.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.463+0000 7ff1addf0640 1 -- 172.21.15.26:0/2717743075 wait complete. 2023-12-19T18:14:51.487 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-19T18:14:51.488 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1M 2023-12-19T18:14:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.639+0000 7f3fe1ec4640 1 Processor -- start 2023-12-19T18:14:51.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.639+0000 7f3fe1ec4640 1 -- start start 2023-12-19T18:14:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.643+0000 7f3fe1ec4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3fdc0b1590 0x7f3fdc06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.643+0000 7f3fe1ec4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fdc06cbf0 con 0x7f3fdc15aad0 2023-12-19T18:14:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.643+0000 7f3fe1ec4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fdc0ca8b0 con 0x7f3fdc0b3f90 2023-12-19T18:14:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.643+0000 7f3fe1ec4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fdc06b6f0 con 0x7f3fdc0b1590 2023-12-19T18:14:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.643+0000 7f3fdaffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3fdc0b1590 0x7f3fdc06c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.643+0000 7f3fdb7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3fdc15aad0 0x7f3fdc070870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53276/0 (socket says 172.21.15.26:53276) 2023-12-19T18:14:51.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.643+0000 7f3fdb7fe640 1 -- 172.21.15.26:0/140484891 learned_addr learned my addr 172.21.15.26:0/140484891 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.643+0000 7f3fdaffd640 1 -- 172.21.15.26:0/140484891 >> v1:172.21.15.160:6789/0 conn(0x7f3fdc0b3f90 legacy=0x7f3fdc0cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.643+0000 7f3fdaffd640 1 -- 172.21.15.26:0/140484891 >> v1:172.21.15.26:6789/0 conn(0x7f3fdc15aad0 legacy=0x7f3fdc070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.643+0000 7f3fdaffd640 1 -- 172.21.15.26:0/140484891 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc06b9d0 con 0x7f3fdc0b1590 2023-12-19T18:14:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.647+0000 7f3fdaffd640 1 --2- 172.21.15.26:0/140484891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fdc0b1590 0x7f3fdc06c6b0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00097c0 tx=0x7f3fd001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f087676ad1c7b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.647+0000 7f3fda7fc640 1 -- 172.21.15.26:0/140484891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd00ad660 con 0x7f3fdc0b1590 2023-12-19T18:14:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.647+0000 7f3fda7fc640 1 -- 172.21.15.26:0/140484891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd00ad7c0 con 0x7f3fdc0b1590 2023-12-19T18:14:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.647+0000 7f3fda7fc640 1 -- 172.21.15.26:0/140484891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd00adb50 con 0x7f3fdc0b1590 2023-12-19T18:14:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.647+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/140484891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fdc0b1590 msgr2=0x7f3fdc06c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.647+0000 7f3fe1ec4640 1 --2- 172.21.15.26:0/140484891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fdc0b1590 0x7f3fdc06c6b0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00097c0 tx=0x7f3fd001c450 comp rx=0 tx=0).stop 2023-12-19T18:14:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.647+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/140484891 shutdown_connections 2023-12-19T18:14:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.647+0000 7f3fe1ec4640 1 --2- 172.21.15.26:0/140484891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fdc0b1590 0x7f3fdc06c6b0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.647+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/140484891 >> 172.21.15.26:0/140484891 conn(0x7f3fdc0b0270 msgr2=0x7f3fdc0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.647+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/140484891 shutdown_connections 2023-12-19T18:14:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.647+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/140484891 wait complete. 2023-12-19T18:14:51.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.647+0000 7f3fe1ec4640 1 Processor -- start 2023-12-19T18:14:51.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.647+0000 7f3fe1ec4640 1 -- start start 2023-12-19T18:14:51.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fe1ec4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3fdc0b1590 0x7f3fdc0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fe1ec4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fdc0b3f90 0x7f3fdc159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fe1ec4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fdc15aad0 0x7f3fdc0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fe1ec4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0c5bf0 con 0x7f3fdc15aad0 2023-12-19T18:14:51.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fe1ec4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0c90d0 con 0x7f3fdc0b3f90 2023-12-19T18:14:51.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fe1ec4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0c93b0 con 0x7f3fdc0b1590 2023-12-19T18:14:51.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fdbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fdc15aad0 0x7f3fdc0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fdbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fdc15aad0 0x7f3fdc0b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37240/0 (socket says 172.21.15.26:37240) 2023-12-19T18:14:51.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fdbfff640 1 -- 172.21.15.26:0/1656860735 learned_addr learned my addr 172.21.15.26:0/1656860735 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:51.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fdaffd640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fdc0b3f90 0x7f3fdc159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fdb7fe640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3fdc0b1590 0x7f3fdc0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fdbfff640 1 -- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3fdc0b1590 msgr2=0x7f3fdc0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fdbfff640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3fdc0b1590 0x7f3fdc0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fdbfff640 1 -- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fdc0b3f90 msgr2=0x7f3fdc159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fdbfff640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fdc0b3f90 0x7f3fdc159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fdbfff640 1 -- 172.21.15.26:0/1656860735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd0009420 con 0x7f3fdc15aad0 2023-12-19T18:14:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fdbfff640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fdc15aad0 0x7f3fdc0b29e0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400c770 tx=0x7f3fd400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fc3fff640 1 -- 172.21.15.26:0/1656860735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4003b80 con 0x7f3fdc15aad0 2023-12-19T18:14:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fc3fff640 1 -- 172.21.15.26:0/1656860735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd4003ce0 con 0x7f3fdc15aad0 2023-12-19T18:14:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fc3fff640 1 -- 172.21.15.26:0/1656860735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400aa90 con 0x7f3fdc15aad0 2023-12-19T18:14:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/1656860735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc0c9690 con 0x7f3fdc15aad0 2023-12-19T18:14:51.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/1656860735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc0c1f60 con 0x7f3fdc15aad0 2023-12-19T18:14:51.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/1656860735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fdc0c5d30 con 0x7f3fdc15aad0 2023-12-19T18:14:51.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.651+0000 7f3fc3fff640 1 -- 172.21.15.26:0/1656860735 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 110) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f3fd400abf0 con 0x7f3fdc15aad0 2023-12-19T18:14:51.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.655+0000 7f3fc3fff640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3fc8065900 0x7f3fc8067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.655+0000 7f3fc3fff640 1 -- 172.21.15.26:0/1656860735 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f3fd4082f30 con 0x7f3fdc15aad0 2023-12-19T18:14:51.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.655+0000 7f3fdb7fe640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3fc8065900 0x7f3fc8067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.655+0000 7f3fdb7fe640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3fc8065900 0x7f3fc8067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc005890 tx=0x7f3fcc009040 comp rx=0 tx=0).ready entity=mgr.9700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:51.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.659+0000 7f3fc3fff640 1 -- 172.21.15.26:0/1656860735 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3fdc0c5d30 con 0x7f3fdc15aad0 2023-12-19T18:14:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.831+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/1656860735 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fdc168710 con 0x7f3fc8065900 2023-12-19T18:14:51.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.887+0000 7f3fc3fff640 1 -- 172.21.15.26:0/1656860735 <== mgr.9700 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f3fcc001490 con 0x7f3fc8065900 2023-12-19T18:14:51.890 INFO:teuthology.orchestra.run.smithi026.stdout:Retention added to path snap_test_dir1 2023-12-19T18:14:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.887+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3fc8065900 msgr2=0x7f3fc8067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.887+0000 7f3fe1ec4640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3fc8065900 0x7f3fc8067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc005890 tx=0x7f3fcc009040 comp rx=0 tx=0).stop 2023-12-19T18:14:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.887+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/1656860735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fdc15aad0 msgr2=0x7f3fdc0b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.887+0000 7f3fe1ec4640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fdc15aad0 0x7f3fdc0b29e0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400c770 tx=0x7f3fd400cc30 comp rx=0 tx=0).stop 2023-12-19T18:14:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.887+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/1656860735 shutdown_connections 2023-12-19T18:14:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.887+0000 7f3fe1ec4640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3fc8065900 0x7f3fc8067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.887+0000 7f3fe1ec4640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fdc15aad0 0x7f3fdc0b29e0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.887+0000 7f3fe1ec4640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3fdc0b3f90 0x7f3fdc159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.887+0000 7f3fe1ec4640 1 --2- 172.21.15.26:0/1656860735 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3fdc0b1590 0x7f3fdc0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.887+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/1656860735 >> 172.21.15.26:0/1656860735 conn(0x7f3fdc0b0270 msgr2=0x7f3fdc0aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.891+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/1656860735 shutdown_connections 2023-12-19T18:14:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:51.891+0000 7f3fe1ec4640 1 -- 172.21.15.26:0/1656860735 wait complete. 2023-12-19T18:14:51.911 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-12-19T18:14:51.911 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-19-18_15_00 in ~9s... 2023-12-19T18:14:51.911 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-12-19T18:14:51.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-12-19T18:14:52.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f461d066640 1 Processor -- start 2023-12-19T18:14:52.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f461d066640 1 -- start start 2023-12-19T18:14:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f461d066640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46180cb600 0x7f461815eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f461d066640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46180ce3f0 con 0x7f46180cb150 2023-12-19T18:14:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f461d066640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46180cba50 con 0x7f46181585f0 2023-12-19T18:14:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f461d066640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46180b47a0 con 0x7f46180cb600 2023-12-19T18:14:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f460ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f46180cb150 0x7f461815e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53288/0 (socket says 172.21.15.26:53288) 2023-12-19T18:14:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f460ffff640 1 -- 172.21.15.26:0/1349065979 learned_addr learned my addr 172.21.15.26:0/1349065979 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:14:52.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f461759e640 1 --2- 172.21.15.26:0/1349065979 >> v2:172.21.15.26:3300/0 conn(0x7f46180cb600 0x7f461815eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f461659c640 1 -- 172.21.15.26:0/1349065979 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242608455 0 0) 0x7f46180ce3f0 con 0x7f46180cb150 2023-12-19T18:14:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f461659c640 1 -- 172.21.15.26:0/1349065979 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4604003530 con 0x7f46180cb150 2023-12-19T18:14:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f461759e640 1 -- 172.21.15.26:0/1349065979 >> v1:172.21.15.160:6789/0 conn(0x7f46181585f0 legacy=0x7f46180cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f461759e640 1 -- 172.21.15.26:0/1349065979 >> v1:172.21.15.26:6789/0 conn(0x7f46180cb150 legacy=0x7f461815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.055+0000 7f461759e640 1 -- 172.21.15.26:0/1349065979 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461815b1b0 con 0x7f46180cb600 2023-12-19T18:14:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.059+0000 7f461759e640 1 --2- 172.21.15.26:0/1349065979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46180cb600 0x7f461815eda0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4610009d90 tx=0x7f461001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe18f3e038387a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.059+0000 7f461659c640 1 -- 172.21.15.26:0/1349065979 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100a5950 con 0x7f46180cb600 2023-12-19T18:14:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.059+0000 7f461659c640 1 -- 172.21.15.26:0/1349065979 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46100a5ab0 con 0x7f46180cb600 2023-12-19T18:14:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.059+0000 7f461659c640 1 -- 172.21.15.26:0/1349065979 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100a5e40 con 0x7f46180cb600 2023-12-19T18:14:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.059+0000 7f461d066640 1 -- 172.21.15.26:0/1349065979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46180cb600 msgr2=0x7f461815eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.059+0000 7f461d066640 1 --2- 172.21.15.26:0/1349065979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46180cb600 0x7f461815eda0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4610009d90 tx=0x7f461001c890 comp rx=0 tx=0).stop 2023-12-19T18:14:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.059+0000 7f461d066640 1 -- 172.21.15.26:0/1349065979 shutdown_connections 2023-12-19T18:14:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.059+0000 7f461d066640 1 --2- 172.21.15.26:0/1349065979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46180cb600 0x7f461815eda0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.059+0000 7f461d066640 1 -- 172.21.15.26:0/1349065979 >> 172.21.15.26:0/1349065979 conn(0x7f4618066160 msgr2=0x7f46180b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.059+0000 7f461d066640 1 -- 172.21.15.26:0/1349065979 shutdown_connections 2023-12-19T18:14:52.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.059+0000 7f461d066640 1 -- 172.21.15.26:0/1349065979 wait complete. 2023-12-19T18:14:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461d066640 1 Processor -- start 2023-12-19T18:14:52.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461d066640 1 -- start start 2023-12-19T18:14:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461d066640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46180cb150 0x7f46181534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461d066640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46180cb600 0x7f46181539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461d066640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46181585f0 0x7f4618154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461d066640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180c7b70 con 0x7f46181585f0 2023-12-19T18:14:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461d066640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4618155460 con 0x7f46180cb600 2023-12-19T18:14:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461d066640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4618155740 con 0x7f46180cb150 2023-12-19T18:14:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46181585f0 0x7f4618154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461759e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46181585f0 0x7f4618154880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37250/0 (socket says 172.21.15.26:37250) 2023-12-19T18:14:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461759e640 1 -- 172.21.15.26:0/362065117 learned_addr learned my addr 172.21.15.26:0/362065117 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:14:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f4616d9d640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46180cb150 0x7f46181534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f460ffff640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46180cb600 0x7f46181539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461759e640 1 -- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46180cb150 msgr2=0x7f46181534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461759e640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46180cb150 0x7f46181534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461759e640 1 -- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46180cb600 msgr2=0x7f46181539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461759e640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46180cb600 0x7f46181539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461759e640 1 -- 172.21.15.26:0/362065117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610009a70 con 0x7f46181585f0 2023-12-19T18:14:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f461759e640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46181585f0 0x7f4618154880 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f461001a4c0 tx=0x7f4610019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f460f7fe640 1 -- 172.21.15.26:0/362065117 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100add10 con 0x7f46181585f0 2023-12-19T18:14:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.063+0000 7f460f7fe640 1 -- 172.21.15.26:0/362065117 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46100ade70 con 0x7f46181585f0 2023-12-19T18:14:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.067+0000 7f461d066640 1 -- 172.21.15.26:0/362065117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4618155a20 con 0x7f46181585f0 2023-12-19T18:14:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.067+0000 7f460f7fe640 1 -- 172.21.15.26:0/362065117 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100b9650 con 0x7f46181585f0 2023-12-19T18:14:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.067+0000 7f461d066640 1 -- 172.21.15.26:0/362065117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4618155fe0 con 0x7f46181585f0 2023-12-19T18:14:52.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.067+0000 7f461d066640 1 -- 172.21.15.26:0/362065117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46180b1320 con 0x7f46181585f0 2023-12-19T18:14:52.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.067+0000 7f460f7fe640 1 -- 172.21.15.26:0/362065117 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 110) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f46100b97b0 con 0x7f46181585f0 2023-12-19T18:14:52.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.067+0000 7f460f7fe640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f45f8065900 0x7f45f8067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:52.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.067+0000 7f460f7fe640 1 -- 172.21.15.26:0/362065117 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f461011e810 con 0x7f46181585f0 2023-12-19T18:14:52.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.067+0000 7f4616d9d640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f45f8065900 0x7f45f8067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:52.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.071+0000 7f4616d9d640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f45f8065900 0x7f45f8067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f46181f6ed0 tx=0x7f4608009040 comp rx=0 tx=0).ready entity=mgr.9700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:52.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.071+0000 7f460f7fe640 1 -- 172.21.15.26:0/362065117 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f46180b1320 con 0x7f46181585f0 2023-12-19T18:14:52.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.247+0000 7f461d066640 1 -- 172.21.15.26:0/362065117 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46181589d0 con 0x7f45f8065900 2023-12-19T18:14:52.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.255+0000 7f460f7fe640 1 -- 172.21.15.26:0/362065117 <== mgr.9700 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f4608001510 con 0x7f45f8065900 2023-12-19T18:14:52.256 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:14:52.257 INFO:teuthology.orchestra.run.smithi026.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-19T18:14:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.255+0000 7f461d066640 1 -- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f45f8065900 msgr2=0x7f45f8067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.255+0000 7f461d066640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f45f8065900 0x7f45f8067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f46181f6ed0 tx=0x7f4608009040 comp rx=0 tx=0).stop 2023-12-19T18:14:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.255+0000 7f461d066640 1 -- 172.21.15.26:0/362065117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46181585f0 msgr2=0x7f4618154880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.255+0000 7f461d066640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46181585f0 0x7f4618154880 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f461001a4c0 tx=0x7f4610019630 comp rx=0 tx=0).stop 2023-12-19T18:14:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.255+0000 7f461d066640 1 -- 172.21.15.26:0/362065117 shutdown_connections 2023-12-19T18:14:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.255+0000 7f461d066640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f45f8065900 0x7f45f8067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.255+0000 7f461d066640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46181585f0 0x7f4618154880 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.255+0000 7f461d066640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46180cb600 0x7f46181539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.255+0000 7f461d066640 1 --2- 172.21.15.26:0/362065117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f46180cb150 0x7f46181534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.255+0000 7f461d066640 1 -- 172.21.15.26:0/362065117 >> 172.21.15.26:0/362065117 conn(0x7f4618066160 msgr2=0x7f461815e540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.255+0000 7f461d066640 1 -- 172.21.15.26:0/362065117 shutdown_connections 2023-12-19T18:14:52.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:14:52.255+0000 7f461d066640 1 -- 172.21.15.26:0/362065117 wait complete. 2023-12-19T18:14:52.278 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-19T18:14:52.278 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-12-19T18:14:52.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:52.278 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:14:52.289 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T18:14:52.289 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:14:52.289 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:14:53.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:53.290 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:14:53.303 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T18:14:53.303 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:14:53.303 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:14:54.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:54.304 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:14:54.317 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T18:14:54.317 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:14:54.317 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:14:55.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:55.318 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:14:55.330 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T18:14:55.330 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:14:55.330 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:14:56.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:56.332 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:14:56.344 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T18:14:56.344 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:14:56.345 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:14:57.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:57.346 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:14:57.358 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T18:14:57.359 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:14:57.359 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:14:58.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:58.359 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:14:58.372 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T18:14:58.372 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:14:58.372 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:14:59.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:14:59.373 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:14:59.386 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T18:14:59.386 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:14:59.386 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:00.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:00.387 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:00.401 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-19T18:15:00.401 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:00.401 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:00.901 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:15:00.899+0000 7f6bb936d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:15:00.901 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:15:00.899+0000 7f6bb936d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:15:00.901 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:15:00.899+0000 7f6bb936d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:15:00.902 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:15:00.899+0000 7f6bb936d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:15:00.902 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:15:00.899+0000 7f6bb936d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:15:01.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:01.402 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:01.413 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:01.414 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:01.414 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:01.414 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:01.414 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-19-18_15_00_UTC'] 2023-12-19T18:15:01.414 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703009691.9112766, now = 1703009701.4146702, timo = 9 2023-12-19T18:15:02.415 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-12-19-18_15_00 in ~69s... 2023-12-19T18:15:02.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:02.416 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:02.427 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:02.428 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:02.428 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:02.428 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:03.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:03.429 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:03.441 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:03.442 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:03.442 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:03.442 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:04.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:04.443 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:04.453 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:04.454 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:04.454 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:04.454 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:05.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:05.456 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:05.469 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:05.469 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:05.469 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:05.469 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:06.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:06.470 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:06.481 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:06.482 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:06.482 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:06.482 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:07.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:07.484 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:07.495 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:07.496 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:07.496 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:07.496 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:08.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:08.498 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:08.509 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:08.510 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:08.510 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:08.510 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:09.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:09.512 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:09.523 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:09.524 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:09.524 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:09.524 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:10.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:10.525 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:10.536 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:10.537 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:10.537 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:10.538 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:11.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:11.539 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:11.550 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:11.551 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:11.551 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:11.552 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:12.113 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:15:12.117 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:15:12.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:12.553 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:12.564 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:12.565 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:12.565 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:12.565 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:13.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:13.566 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:13.576 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:13.577 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:13.578 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:13.578 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:14.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:14.579 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:14.590 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:14.591 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:14.591 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:14.592 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:15.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:15.592 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:15.604 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:15.605 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:15.605 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:15.605 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:16.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:16.606 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:16.619 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:16.619 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:16.619 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:16.619 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:17.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:17.621 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:17.632 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:17.633 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:17.633 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:17.633 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:18.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:18.635 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:18.646 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:18.648 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:18.648 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:18.648 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:19.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:19.683 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:19.695 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:19.696 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:19.696 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:19.696 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:20.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:20.697 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:20.709 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:20.710 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:20.710 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:20.710 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:21.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:21.711 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:21.723 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:21.724 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:21.724 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:21.724 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:22.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:22.725 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:22.736 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:22.737 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:22.738 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:22.738 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:23.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:23.739 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:23.751 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:23.752 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:23.752 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:23.752 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:24.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:24.753 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:24.765 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:24.766 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:24.766 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:24.766 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:25.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:25.767 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:25.778 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:25.779 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:25.779 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:25.779 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:26.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:26.781 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:26.792 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:26.793 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:26.793 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:26.793 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:27.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:27.795 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:27.806 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:27.808 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:27.808 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:27.808 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:28.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:28.809 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:28.820 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:28.821 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:28.821 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:28.821 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:29.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:29.823 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:29.834 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:29.835 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:29.835 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:29.835 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:30.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:30.837 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:30.848 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:30.849 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:30.850 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:30.850 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:31.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:31.850 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:31.862 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:31.863 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:31.863 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:31.863 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:32.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:32.865 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:32.876 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:32.877 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:32.877 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:32.877 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:33.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:33.879 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:33.890 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:33.891 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:33.891 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:33.891 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:34.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:34.893 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:34.904 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:34.905 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:34.905 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:34.905 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:35.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:35.907 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:35.918 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:35.919 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:35.919 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:35.919 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:36.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:36.920 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:36.932 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:36.933 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:36.933 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:36.933 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:37.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:37.934 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:37.945 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:37.946 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:37.946 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:37.946 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:38.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:38.947 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:38.959 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:38.960 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:38.960 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:38.960 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:39.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:39.961 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:39.972 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:39.973 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:39.973 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:39.973 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:40.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:40.974 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:40.985 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:40.986 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:40.987 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:40.987 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:41.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:41.988 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:41.999 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:42.000 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:42.000 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:42.000 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:42.201 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:15:42.205 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:15:43.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:43.001 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:43.012 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:43.014 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:43.014 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:43.014 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:44.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:44.015 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:44.026 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:44.027 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:44.027 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:44.027 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:45.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:45.029 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:45.040 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:45.041 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:45.041 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:45.041 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:46.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:46.043 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:46.053 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:46.054 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:46.054 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:46.054 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:47.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:47.056 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:47.067 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:47.069 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:47.069 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:47.069 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:48.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:48.070 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:48.081 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:48.082 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:48.082 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:48.082 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:49.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:49.083 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:49.094 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:49.095 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:49.095 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:49.095 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:50.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:50.096 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:50.106 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:50.107 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:50.107 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:50.107 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:51.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:51.108 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:51.119 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:51.120 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:51.120 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:51.120 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:52.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:52.122 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:52.133 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:52.134 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:52.134 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:52.134 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:53.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:53.135 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:53.146 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:53.147 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:53.147 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:53.147 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:54.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:54.148 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:54.159 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:54.160 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:54.160 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:54.160 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:55.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:55.161 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:55.172 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:55.173 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:55.173 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:55.173 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:56.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:56.175 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:56.185 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:56.187 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:56.187 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:56.187 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:57.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:57.188 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:57.199 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:57.200 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:57.200 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:57.200 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:58.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:58.201 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:58.212 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:58.214 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:58.214 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:58.214 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:15:59.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:59.215 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:15:59.226 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:15:59.227 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:15:59.227 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:15:59.227 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:16:00.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:00.228 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:16:00.239 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_15_00_UTC 2023-12-19T18:16:00.240 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:16:00.240 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-19T18:16:00.240 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-19T18:16:01.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:01.242 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:16:01.253 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_16_00_UTC 2023-12-19T18:16:01.254 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-19-18_16_00_UTC'} 2023-12-19T18:16:01.254 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-19-18_16_00_UTC'} 2023-12-19T18:16:01.254 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-12-19-18_15_00_UTC'} 2023-12-19T18:16:01.254 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-12-19-18_15_00_UTC'] 2023-12-19T18:16:01.254 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703009691.9112766, now = 1703009761.254807, timo = 69 2023-12-19T18:16:02.255 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-19T18:16:02.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f625dbce640 1 Processor -- start 2023-12-19T18:16:02.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f625dbce640 1 -- start start 2023-12-19T18:16:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f625dbce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62581599e0 0x7f625806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f625dbce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62580cb150 0x7f62580cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f625dbce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62580cbaf0 con 0x7f625815d530 2023-12-19T18:16:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f625dbce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625815e050 con 0x7f62581599e0 2023-12-19T18:16:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f625dbce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625815e240 con 0x7f62580cb150 2023-12-19T18:16:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f62577fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f62581599e0 0x7f625806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f6257fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f625815d530 0x7f625815d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43068/0 (socket says 172.21.15.26:43068) 2023-12-19T18:16:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f6257fff640 1 -- 172.21.15.26:0/967537263 learned_addr learned my addr 172.21.15.26:0/967537263 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f6256ffd640 1 --2- 172.21.15.26:0/967537263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62580cb150 0x7f62580cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f62577fe640 1 -- 172.21.15.26:0/967537263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62580cb150 msgr2=0x7f62580cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f62577fe640 1 --2- 172.21.15.26:0/967537263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62580cb150 0x7f62580cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f62567fc640 1 -- 172.21.15.26:0/967537263 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888269613 0 0) 0x7f62580cbaf0 con 0x7f625815d530 2023-12-19T18:16:02.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f62577fe640 1 -- 172.21.15.26:0/967537263 >> v1:172.21.15.26:6789/0 conn(0x7f625815d530 legacy=0x7f625815d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.410+0000 7f62577fe640 1 -- 172.21.15.26:0/967537263 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62580c9520 con 0x7f62581599e0 2023-12-19T18:16:02.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.414+0000 7f62577fe640 1 --2- 172.21.15.26:0/967537263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62581599e0 0x7f625806f6d0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f62480097c0 tx=0x7f624801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dee3db2f071c547 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:02.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.414+0000 7f62567fc640 1 -- 172.21.15.26:0/967537263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62480ab680 con 0x7f62581599e0 2023-12-19T18:16:02.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.414+0000 7f62567fc640 1 -- 172.21.15.26:0/967537263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62480ab7e0 con 0x7f62581599e0 2023-12-19T18:16:02.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.414+0000 7f62567fc640 1 -- 172.21.15.26:0/967537263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62480abb70 con 0x7f62581599e0 2023-12-19T18:16:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.414+0000 7f625dbce640 1 -- 172.21.15.26:0/967537263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62581599e0 msgr2=0x7f625806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.414+0000 7f625dbce640 1 --2- 172.21.15.26:0/967537263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62581599e0 0x7f625806f6d0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f62480097c0 tx=0x7f624801c480 comp rx=0 tx=0).stop 2023-12-19T18:16:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.414+0000 7f625dbce640 1 -- 172.21.15.26:0/967537263 shutdown_connections 2023-12-19T18:16:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.414+0000 7f625dbce640 1 --2- 172.21.15.26:0/967537263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62580cb150 0x7f62580cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.414+0000 7f625dbce640 1 --2- 172.21.15.26:0/967537263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62581599e0 0x7f625806f6d0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.414+0000 7f625dbce640 1 -- 172.21.15.26:0/967537263 >> 172.21.15.26:0/967537263 conn(0x7f6258066160 msgr2=0x7f62580b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.414+0000 7f625dbce640 1 -- 172.21.15.26:0/967537263 shutdown_connections 2023-12-19T18:16:02.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.414+0000 7f625dbce640 1 -- 172.21.15.26:0/967537263 wait complete. 2023-12-19T18:16:02.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f625dbce640 1 Processor -- start 2023-12-19T18:16:02.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f625dbce640 1 -- start start 2023-12-19T18:16:02.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f625dbce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62580cb150 0x7f62580c1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f625dbce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62581599e0 0x7f625814ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f625dbce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f625815d530 0x7f62580c2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f625dbce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625814f2a0 con 0x7f62581599e0 2023-12-19T18:16:02.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f625dbce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62580c2960 con 0x7f62580cb150 2023-12-19T18:16:02.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f625dbce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62580afe90 con 0x7f625815d530 2023-12-19T18:16:02.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f6256ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62581599e0 0x7f625814ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f6256ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62581599e0 0x7f625814ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43956/0 (socket says 172.21.15.26:43956) 2023-12-19T18:16:02.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f6256ffd640 1 -- 172.21.15.26:0/3316504391 learned_addr learned my addr 172.21.15.26:0/3316504391 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:02.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f62577fe640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62580cb150 0x7f62580c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f6257fff640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f625815d530 0x7f62580c2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f6256ffd640 1 -- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f625815d530 msgr2=0x7f62580c2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f6256ffd640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f625815d530 0x7f62580c2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f6256ffd640 1 -- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62580cb150 msgr2=0x7f62580c1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f6256ffd640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62580cb150 0x7f62580c1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f6256ffd640 1 -- 172.21.15.26:0/3316504391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6248009420 con 0x7f62581599e0 2023-12-19T18:16:02.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f62577fe640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62580cb150 0x7f62580c1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:02.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f6256ffd640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62581599e0 0x7f625814ed60 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f624c00e930 tx=0x7f624c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:02.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f623ffff640 1 -- 172.21.15.26:0/3316504391 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c009690 con 0x7f62581599e0 2023-12-19T18:16:02.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f623ffff640 1 -- 172.21.15.26:0/3316504391 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c00ca00 con 0x7f62581599e0 2023-12-19T18:16:02.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f623ffff640 1 -- 172.21.15.26:0/3316504391 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c00cd70 con 0x7f62581599e0 2023-12-19T18:16:02.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f625dbce640 1 -- 172.21.15.26:0/3316504391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62580b0170 con 0x7f62581599e0 2023-12-19T18:16:02.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.418+0000 7f625dbce640 1 -- 172.21.15.26:0/3316504391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62580c25f0 con 0x7f62581599e0 2023-12-19T18:16:02.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.422+0000 7f625dbce640 1 -- 172.21.15.26:0/3316504391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6258074270 con 0x7f62581599e0 2023-12-19T18:16:02.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.422+0000 7f623ffff640 1 -- 172.21.15.26:0/3316504391 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 113) v1 ==== 85461+0+0 (secure 0 0 0) 0x7f624c013470 con 0x7f62581599e0 2023-12-19T18:16:02.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.422+0000 7f623ffff640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6244065cf0 0x7f62440681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.422+0000 7f623ffff640 1 -- 172.21.15.26:0/3316504391 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f624c083d90 con 0x7f62581599e0 2023-12-19T18:16:02.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.422+0000 7f62577fe640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6244065cf0 0x7f62440681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.422+0000 7f62577fe640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6244065cf0 0x7f62440681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f62480af000 tx=0x7f6248004740 comp rx=0 tx=0).ready entity=mgr.9700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:02.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.426+0000 7f623ffff640 1 -- 172.21.15.26:0/3316504391 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f624c050fd0 con 0x7f62581599e0 2023-12-19T18:16:02.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.602+0000 7f625dbce640 1 -- 172.21.15.26:0/3316504391 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6258159dc0 con 0x7f6244065cf0 2023-12-19T18:16:02.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.674+0000 7f623ffff640 1 -- 172.21.15.26:0/3316504391 <== mgr.9700 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f62480ab950 con 0x7f6244065cf0 2023-12-19T18:16:02.675 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path snap_test_dir1 2023-12-19T18:16:02.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.674+0000 7f625dbce640 1 -- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6244065cf0 msgr2=0x7f62440681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.674+0000 7f625dbce640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6244065cf0 0x7f62440681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f62480af000 tx=0x7f6248004740 comp rx=0 tx=0).stop 2023-12-19T18:16:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.674+0000 7f625dbce640 1 -- 172.21.15.26:0/3316504391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62581599e0 msgr2=0x7f625814ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.674+0000 7f625dbce640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62581599e0 0x7f625814ed60 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f624c00e930 tx=0x7f624c00edf0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.674+0000 7f625dbce640 1 -- 172.21.15.26:0/3316504391 shutdown_connections 2023-12-19T18:16:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.674+0000 7f625dbce640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6244065cf0 0x7f62440681b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.674+0000 7f625dbce640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f625815d530 0x7f62580c2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.674+0000 7f625dbce640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62581599e0 0x7f625814ed60 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.674+0000 7f625dbce640 1 --2- 172.21.15.26:0/3316504391 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62580cb150 0x7f62580c1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.674+0000 7f625dbce640 1 -- 172.21.15.26:0/3316504391 >> 172.21.15.26:0/3316504391 conn(0x7f6258066160 msgr2=0x7f62580cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.674+0000 7f625dbce640 1 -- 172.21.15.26:0/3316504391 shutdown_connections 2023-12-19T18:16:02.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:02.674+0000 7f625dbce640 1 -- 172.21.15.26:0/3316504391 wait complete. 2023-12-19T18:16:02.697 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-19T18:16:02.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:02.697 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:16:02.707 INFO:teuthology.orchestra.run.smithi026.stdout:scheduled-2023-12-19-18_16_00_UTC 2023-12-19T18:16:02.708 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-19-18_16_00_UTC 2023-12-19T18:16:02.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:02.708 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-19-18_16_00_UTC) 2023-12-19T18:16:02.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:02.769 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-19T18:16:02.823 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-19T18:16:03.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f460b640 1 Processor -- start 2023-12-19T18:16:03.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f460b640 1 -- start start 2023-12-19T18:16:03.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f460b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36ec159090 0x7f36ec0c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f460b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f36ec0ca820 0x7f36ec06b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f460b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ec0cc310 con 0x7f36ec06b850 2023-12-19T18:16:03.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f460b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ec06bd00 con 0x7f36ec0ca820 2023-12-19T18:16:03.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f460b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0ca110 con 0x7f36ec159090 2023-12-19T18:16:03.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f2ba8640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f36ec06b850 0x7f36ec15cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:49358/0 (socket says 172.21.15.26:49358) 2023-12-19T18:16:03.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f2ba8640 1 -- 172.21.15.26:0/975527222 learned_addr learned my addr 172.21.15.26:0/975527222 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:03.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f23a7640 1 --2- 172.21.15.26:0/975527222 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36ec159090 0x7f36ec0c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f1ba6640 1 --2- 172.21.15.26:0/975527222 >> v2:172.21.15.160:3300/0 conn(0x7f36ec0ca820 0x7f36ec06b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f13a5640 1 -- 172.21.15.26:0/975527222 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111656666 0 0) 0x7f36ec0cc310 con 0x7f36ec06b850 2023-12-19T18:16:03.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f13a5640 1 -- 172.21.15.26:0/975527222 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d8003530 con 0x7f36ec06b850 2023-12-19T18:16:03.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f1ba6640 1 -- 172.21.15.26:0/975527222 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36ec159090 msgr2=0x7f36ec0c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:03.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f1ba6640 1 --2- 172.21.15.26:0/975527222 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36ec159090 0x7f36ec0c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f1ba6640 1 -- 172.21.15.26:0/975527222 >> v1:172.21.15.160:6789/0 conn(0x7f36ec06b850 legacy=0x7f36ec15cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f1ba6640 1 -- 172.21.15.26:0/975527222 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec0ca3f0 con 0x7f36ec0ca820 2023-12-19T18:16:03.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.010+0000 7f36f23a7640 1 --2- 172.21.15.26:0/975527222 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36ec159090 0x7f36ec0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.014+0000 7f36f1ba6640 1 --2- 172.21.15.26:0/975527222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36ec0ca820 0x7f36ec06b310 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0097c0 tx=0x7f36dc01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8783c7cc9b57bd29 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.014+0000 7f36f13a5640 1 -- 172.21.15.26:0/975527222 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0ab600 con 0x7f36ec0ca820 2023-12-19T18:16:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.014+0000 7f36f13a5640 1 -- 172.21.15.26:0/975527222 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36dc0a7050 con 0x7f36ec0ca820 2023-12-19T18:16:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.014+0000 7f36f13a5640 1 -- 172.21.15.26:0/975527222 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc0ab980 con 0x7f36ec0ca820 2023-12-19T18:16:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.014+0000 7f36f460b640 1 -- 172.21.15.26:0/975527222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36ec0ca820 msgr2=0x7f36ec06b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.014+0000 7f36f460b640 1 --2- 172.21.15.26:0/975527222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36ec0ca820 0x7f36ec06b310 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0097c0 tx=0x7f36dc01c450 comp rx=0 tx=0).stop 2023-12-19T18:16:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.014+0000 7f36f460b640 1 -- 172.21.15.26:0/975527222 shutdown_connections 2023-12-19T18:16:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.014+0000 7f36f460b640 1 --2- 172.21.15.26:0/975527222 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36ec0ca820 0x7f36ec06b310 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.014+0000 7f36f460b640 1 --2- 172.21.15.26:0/975527222 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36ec159090 0x7f36ec0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.014+0000 7f36f460b640 1 -- 172.21.15.26:0/975527222 >> 172.21.15.26:0/975527222 conn(0x7f36ec066160 msgr2=0x7f36ec0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.014+0000 7f36f460b640 1 -- 172.21.15.26:0/975527222 shutdown_connections 2023-12-19T18:16:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.014+0000 7f36f460b640 1 -- 172.21.15.26:0/975527222 wait complete. 2023-12-19T18:16:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f460b640 1 Processor -- start 2023-12-19T18:16:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f460b640 1 -- start start 2023-12-19T18:16:03.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f460b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ec06b850 0x7f36ec150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f460b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36ec0ca820 0x7f36ec0c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f460b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36ec159090 0x7f36ec0c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f460b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec151410 con 0x7f36ec06b850 2023-12-19T18:16:03.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f460b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec150a40 con 0x7f36ec159090 2023-12-19T18:16:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f460b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ec0c9670 con 0x7f36ec0ca820 2023-12-19T18:16:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f23a7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ec06b850 0x7f36ec150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f23a7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ec06b850 0x7f36ec150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:43968/0 (socket says 172.21.15.26:43968) 2023-12-19T18:16:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f23a7640 1 -- 172.21.15.26:0/42463136 learned_addr learned my addr 172.21.15.26:0/42463136 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f2ba8640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36ec159090 0x7f36ec0c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f1ba6640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36ec0ca820 0x7f36ec0c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f23a7640 1 -- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36ec0ca820 msgr2=0x7f36ec0c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f23a7640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36ec0ca820 0x7f36ec0c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f23a7640 1 -- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36ec159090 msgr2=0x7f36ec0c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f23a7640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36ec159090 0x7f36ec0c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f23a7640 1 -- 172.21.15.26:0/42463136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e8009420 con 0x7f36ec06b850 2023-12-19T18:16:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f1ba6640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36ec0ca820 0x7f36ec0c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f23a7640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ec06b850 0x7f36ec150ed0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f36e800e9b0 tx=0x7f36e800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36e2ffd640 1 -- 172.21.15.26:0/42463136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e800ca00 con 0x7f36ec06b850 2023-12-19T18:16:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36e2ffd640 1 -- 172.21.15.26:0/42463136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e800cb60 con 0x7f36ec06b850 2023-12-19T18:16:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f460b640 1 -- 172.21.15.26:0/42463136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36dc009420 con 0x7f36ec06b850 2023-12-19T18:16:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36e2ffd640 1 -- 172.21.15.26:0/42463136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8013470 con 0x7f36ec06b850 2023-12-19T18:16:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f460b640 1 -- 172.21.15.26:0/42463136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ec150110 con 0x7f36ec06b850 2023-12-19T18:16:03.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.018+0000 7f36f460b640 1 -- 172.21.15.26:0/42463136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ec074270 con 0x7f36ec06b850 2023-12-19T18:16:03.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.022+0000 7f36e2ffd640 1 -- 172.21.15.26:0/42463136 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 113) v1 ==== 85461+0+0 (secure 0 0 0) 0x7f36e80135d0 con 0x7f36ec06b850 2023-12-19T18:16:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.022+0000 7f36e2ffd640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f36d0065cf0 0x7f36d00681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.022+0000 7f36e2ffd640 1 -- 172.21.15.26:0/42463136 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f36e8083db0 con 0x7f36ec06b850 2023-12-19T18:16:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.022+0000 7f36f1ba6640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f36d0065cf0 0x7f36d00681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.022+0000 7f36f1ba6640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f36d0065cf0 0x7f36d00681b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0ad000 tx=0x7f36dc008040 comp rx=0 tx=0).ready entity=mgr.9700 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:03.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.026+0000 7f36e2ffd640 1 -- 172.21.15.26:0/42463136 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f36e8088070 con 0x7f36ec06b850 2023-12-19T18:16:03.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.226+0000 7f36f460b640 1 -- 172.21.15.26:0/42463136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f36ec0cbe80 con 0x7f36ec06b850 2023-12-19T18:16:03.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.490+0000 7f36e2ffd640 1 -- 172.21.15.26:0/42463136 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 114) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f36e8051300 con 0x7f36ec06b850 2023-12-19T18:16:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.494+0000 7f36e2ffd640 1 -- 172.21.15.26:0/42463136 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v114) v1 ==== 93+0+0 (secure 0 0 0) 0x7f36e807fb20 con 0x7f36ec06b850 2023-12-19T18:16:03.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.494+0000 7f36f460b640 1 -- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f36d0065cf0 msgr2=0x7f36d00681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.494+0000 7f36f460b640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f36d0065cf0 0x7f36d00681b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0ad000 tx=0x7f36dc008040 comp rx=0 tx=0).stop 2023-12-19T18:16:03.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.494+0000 7f36f460b640 1 -- 172.21.15.26:0/42463136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ec06b850 msgr2=0x7f36ec150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.494+0000 7f36f460b640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ec06b850 0x7f36ec150ed0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f36e800e9b0 tx=0x7f36e800ee70 comp rx=0 tx=0).stop 2023-12-19T18:16:03.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.494+0000 7f36f460b640 1 -- 172.21.15.26:0/42463136 shutdown_connections 2023-12-19T18:16:03.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.494+0000 7f36f460b640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f36d0065cf0 0x7f36d00681b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.494+0000 7f36f460b640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f36ec159090 0x7f36ec0c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.494+0000 7f36f460b640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f36ec0ca820 0x7f36ec0c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.494+0000 7f36f460b640 1 --2- 172.21.15.26:0/42463136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36ec06b850 0x7f36ec150ed0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.494+0000 7f36f460b640 1 -- 172.21.15.26:0/42463136 >> 172.21.15.26:0/42463136 conn(0x7f36ec066160 msgr2=0x7f36ec0cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:03.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.494+0000 7f36f460b640 1 -- 172.21.15.26:0/42463136 shutdown_connections 2023-12-19T18:16:03.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.494+0000 7f36f460b640 1 -- 172.21.15.26:0/42463136 wait complete. 2023-12-19T18:16:03.519 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:16:03.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:16:03.621 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:16:03.675 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:03.674+0000 7f5baf047180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:16:03.678 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:03.674+0000 7f183264b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:16:03.726 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:16:03.726 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:16:03.726 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:03.736 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:16:03.736 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:03.789 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:03.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f846b939640 1 Processor -- start 2023-12-19T18:16:03.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f846b939640 1 -- start start 2023-12-19T18:16:03.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f846b939640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8464155030 0x7f846415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f846b939640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f846415f370 0x7f846406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f846b939640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f84640ce740 0x7f84640cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f846b939640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84640cf130 con 0x7f84640ce740 2023-12-19T18:16:03.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f846b939640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846406c970 con 0x7f846415f370 2023-12-19T18:16:03.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f846b939640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846415b1f0 con 0x7f8464155030 2023-12-19T18:16:03.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f8469ed6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f84640ce740 0x7f84640cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f8469ed6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f84640ce740 0x7f84640cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44000/0 (socket says 172.21.15.26:44000) 2023-12-19T18:16:03.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f8469ed6640 1 -- 172.21.15.26:0/123396945 learned_addr learned my addr 172.21.15.26:0/123396945 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:03.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f84696d5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8464155030 0x7f846415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f8468ed4640 1 --2- 172.21.15.26:0/123396945 >> v2:172.21.15.160:3300/0 conn(0x7f846415f370 0x7f846406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f8469ed6640 1 -- 172.21.15.26:0/123396945 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8464155030 msgr2=0x7f846415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f8469ed6640 1 --2- 172.21.15.26:0/123396945 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8464155030 0x7f846415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f8469ed6640 1 -- 172.21.15.26:0/123396945 >> v2:172.21.15.160:3300/0 conn(0x7f846415f370 msgr2=0x7f846406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f8469ed6640 1 --2- 172.21.15.26:0/123396945 >> v2:172.21.15.160:3300/0 conn(0x7f846415f370 0x7f846406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f8469ed6640 1 -- 172.21.15.26:0/123396945 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846415b4d0 con 0x7f84640ce740 2023-12-19T18:16:03.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.974+0000 7f8468ed4640 1 --2- 172.21.15.26:0/123396945 >> v2:172.21.15.160:3300/0 conn(0x7f846415f370 0x7f846406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:03.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.978+0000 7f8469ed6640 1 --2- 172.21.15.26:0/123396945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84640ce740 0x7f84640cebf0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f84640b3ac0 tx=0x7f845c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bebc796994b097a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:03.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.978+0000 7f845bfff640 1 -- 172.21.15.26:0/123396945 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c0a5840 con 0x7f84640ce740 2023-12-19T18:16:03.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.978+0000 7f845bfff640 1 -- 172.21.15.26:0/123396945 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c0a59a0 con 0x7f84640ce740 2023-12-19T18:16:03.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.978+0000 7f845bfff640 1 -- 172.21.15.26:0/123396945 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c0a5d30 con 0x7f84640ce740 2023-12-19T18:16:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.978+0000 7f846b939640 1 -- 172.21.15.26:0/123396945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84640ce740 msgr2=0x7f84640cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.978+0000 7f846b939640 1 --2- 172.21.15.26:0/123396945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84640ce740 0x7f84640cebf0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f84640b3ac0 tx=0x7f845c01c9a0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.978+0000 7f846b939640 1 -- 172.21.15.26:0/123396945 shutdown_connections 2023-12-19T18:16:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.978+0000 7f846b939640 1 --2- 172.21.15.26:0/123396945 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f84640ce740 0x7f84640cebf0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.978+0000 7f846b939640 1 --2- 172.21.15.26:0/123396945 >> v2:172.21.15.160:3300/0 conn(0x7f846415f370 0x7f846406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.978+0000 7f846b939640 1 --2- 172.21.15.26:0/123396945 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8464155030 0x7f846415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.978+0000 7f846b939640 1 -- 172.21.15.26:0/123396945 >> 172.21.15.26:0/123396945 conn(0x7f84640658e0 msgr2=0x7f8464065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:03.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.978+0000 7f846b939640 1 -- 172.21.15.26:0/123396945 shutdown_connections 2023-12-19T18:16:03.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.978+0000 7f846b939640 1 -- 172.21.15.26:0/123396945 wait complete. 2023-12-19T18:16:03.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f846b939640 1 Processor -- start 2023-12-19T18:16:03.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f846b939640 1 -- start start 2023-12-19T18:16:03.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f846b939640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84640ce740 0x7f84641535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f846b939640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8464155030 0x7f846406b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f846b939640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846415f370 0x7f846406be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f846b939640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84640adca0 con 0x7f846415f370 2023-12-19T18:16:03.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f846b939640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8464154000 con 0x7f84640ce740 2023-12-19T18:16:03.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f846b939640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84640adef0 con 0x7f8464155030 2023-12-19T18:16:03.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f8469ed6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846415f370 0x7f846406be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f8469ed6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846415f370 0x7f846406be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44014/0 (socket says 172.21.15.26:44014) 2023-12-19T18:16:03.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f8469ed6640 1 -- 172.21.15.26:0/1875606103 learned_addr learned my addr 172.21.15.26:0/1875606103 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:03.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f8468ed4640 1 --2- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8464155030 0x7f846406b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f84696d5640 1 --2- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84640ce740 0x7f84641535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f8469ed6640 1 -- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8464155030 msgr2=0x7f846406b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f8469ed6640 1 --2- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8464155030 0x7f846406b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f8469ed6640 1 -- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84640ce740 msgr2=0x7f84641535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f8469ed6640 1 --2- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84640ce740 0x7f84641535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f8469ed6640 1 -- 172.21.15.26:0/1875606103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8454009420 con 0x7f846415f370 2023-12-19T18:16:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f8469ed6640 1 --2- 172.21.15.26:0/1875606103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846415f370 0x7f846406be60 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f845c01b050 tx=0x7f845c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f8459ffb640 1 -- 172.21.15.26:0/1875606103 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c0a5840 con 0x7f846415f370 2023-12-19T18:16:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f8459ffb640 1 -- 172.21.15.26:0/1875606103 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c0b8670 con 0x7f846415f370 2023-12-19T18:16:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f846b939640 1 -- 172.21.15.26:0/1875606103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845c009b30 con 0x7f846415f370 2023-12-19T18:16:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f8459ffb640 1 -- 172.21.15.26:0/1875606103 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c0b8a00 con 0x7f846415f370 2023-12-19T18:16:03.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.982+0000 7f846b939640 1 -- 172.21.15.26:0/1875606103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846406c740 con 0x7f846415f370 2023-12-19T18:16:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.986+0000 7f846b939640 1 -- 172.21.15.26:0/1875606103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8464074270 con 0x7f846415f370 2023-12-19T18:16:03.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.986+0000 7f8459ffb640 1 -- 172.21.15.26:0/1875606103 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 114) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f845c0b8b60 con 0x7f846415f370 2023-12-19T18:16:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.986+0000 7f8459ffb640 1 --2- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8448065ca0 0x7f8448068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.986+0000 7f8459ffb640 1 -- 172.21.15.26:0/1875606103 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6873+0+0 (secure 0 0 0) 0x7f845c11ed90 con 0x7f846415f370 2023-12-19T18:16:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.986+0000 7f84696d5640 1 -- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8448065ca0 msgr2=0x7f8448068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.986+0000 7f84696d5640 1 --2- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8448065ca0 0x7f8448068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:16:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:03.990+0000 7f8459ffb640 1 -- 172.21.15.26:0/1875606103 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f845c0ec1f0 con 0x7f846415f370 2023-12-19T18:16:04.102 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:04.098+0000 7f183264b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:16:04.128 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:04.126+0000 7f5baf047180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:16:04.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.150+0000 7f846b939640 1 -- 172.21.15.26:0/1875606103 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84640ccc30 con 0x7f846415f370 2023-12-19T18:16:04.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.154+0000 7f8459ffb640 1 -- 172.21.15.26:0/1875606103 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+17865 (secure 0 0 0) 0x7f845c0bc090 con 0x7f846415f370 2023-12-19T18:16:04.156 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:04.156 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":198,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:03.479803+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:36.168666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:37.527319+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":4,"snap_epoch":198,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/2345576187":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/1296996055":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/4184305047":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/2077682555":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/1610506837":"2023-12-20T18:14:50.374461+0000","172.21.15.160:6833/48394":"2023-12-20T18:14:50.374461+0000","172.21.15.160:6832/48394":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/3136437506":"2023-12-20T18:14:50.374461+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":2,"length":1},{"begin":4,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.154+0000 7f846b939640 1 -- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8448065ca0 msgr2=0x7f8448068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:16:04.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.154+0000 7f846b939640 1 --2- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8448065ca0 0x7f8448068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.154+0000 7f846b939640 1 -- 172.21.15.26:0/1875606103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846415f370 msgr2=0x7f846406be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.154+0000 7f846b939640 1 --2- 172.21.15.26:0/1875606103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846415f370 0x7f846406be60 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f845c01b050 tx=0x7f845c019740 comp rx=0 tx=0).stop 2023-12-19T18:16:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.154+0000 7f846b939640 1 -- 172.21.15.26:0/1875606103 shutdown_connections 2023-12-19T18:16:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.154+0000 7f846b939640 1 --2- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8448065ca0 0x7f8448068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.154+0000 7f846b939640 1 --2- 172.21.15.26:0/1875606103 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f846415f370 0x7f846406be60 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.154+0000 7f846b939640 1 --2- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8464155030 0x7f846406b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.154+0000 7f846b939640 1 --2- 172.21.15.26:0/1875606103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f84640ce740 0x7f84641535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.154+0000 7f846b939640 1 -- 172.21.15.26:0/1875606103 >> 172.21.15.26:0/1875606103 conn(0x7f84640658e0 msgr2=0x7f846415c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.154+0000 7f846b939640 1 -- 172.21.15.26:0/1875606103 shutdown_connections 2023-12-19T18:16:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.154+0000 7f846b939640 1 -- 172.21.15.26:0/1875606103 wait complete. 2023-12-19T18:16:04.181 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:36.168666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '195', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:37.527319+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '198', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 4, 'snap_epoch': 198, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:16:04.181 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:16:04.250 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:04.246+0000 7f183264b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:16:04.289 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:04.286+0000 7f5baf047180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:16:04.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.326+0000 7f4de8fb3640 1 Processor -- start 2023-12-19T18:16:04.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.326+0000 7f4de8fb3640 1 -- start start 2023-12-19T18:16:04.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.326+0000 7f4de8fb3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4de415cd20 0x7f4de40b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.326+0000 7f4de8fb3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de40b5c00 con 0x7f4de40b4dd0 2023-12-19T18:16:04.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.326+0000 7f4de8fb3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de415d440 con 0x7f4de40b89c0 2023-12-19T18:16:04.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.326+0000 7f4de8fb3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de415d6e0 con 0x7f4de415cd20 2023-12-19T18:16:04.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.326+0000 7f4de2d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4de40b4dd0 0x7f4de40b1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43082/0 (socket says 172.21.15.26:43082) 2023-12-19T18:16:04.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.326+0000 7f4de2d9d640 1 -- 172.21.15.26:0/54576907 learned_addr learned my addr 172.21.15.26:0/54576907 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:04.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.330+0000 7f4de259c640 1 --2- 172.21.15.26:0/54576907 >> v2:172.21.15.160:3300/0 conn(0x7f4de415cd20 0x7f4de40b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.330+0000 7f4de1d9b640 1 -- 172.21.15.26:0/54576907 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186791814 0 0) 0x7f4de40b5c00 con 0x7f4de40b4dd0 2023-12-19T18:16:04.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.330+0000 7f4de1d9b640 1 -- 172.21.15.26:0/54576907 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dd0003530 con 0x7f4de40b4dd0 2023-12-19T18:16:04.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.330+0000 7f4de259c640 1 -- 172.21.15.26:0/54576907 >> v1:172.21.15.160:6789/0 conn(0x7f4de40b89c0 legacy=0x7f4de40b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.330+0000 7f4de259c640 1 -- 172.21.15.26:0/54576907 >> v1:172.21.15.26:6789/0 conn(0x7f4de40b4dd0 legacy=0x7f4de40b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.330+0000 7f4de259c640 1 -- 172.21.15.26:0/54576907 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de415d9c0 con 0x7f4de415cd20 2023-12-19T18:16:04.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.330+0000 7f4de259c640 1 --2- 172.21.15.26:0/54576907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4de415cd20 0x7f4de40b8480 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40097c0 tx=0x7f4dd401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9238bd474d79825 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:04.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.330+0000 7f4de1d9b640 1 -- 172.21.15.26:0/54576907 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd40a8970 con 0x7f4de415cd20 2023-12-19T18:16:04.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.330+0000 7f4de1d9b640 1 -- 172.21.15.26:0/54576907 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd40a8ad0 con 0x7f4de415cd20 2023-12-19T18:16:04.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.330+0000 7f4de1d9b640 1 -- 172.21.15.26:0/54576907 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd40a8e60 con 0x7f4de415cd20 2023-12-19T18:16:04.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 -- 172.21.15.26:0/54576907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4de415cd20 msgr2=0x7f4de40b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 --2- 172.21.15.26:0/54576907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4de415cd20 0x7f4de40b8480 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4dd40097c0 tx=0x7f4dd401ec70 comp rx=0 tx=0).stop 2023-12-19T18:16:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 -- 172.21.15.26:0/54576907 shutdown_connections 2023-12-19T18:16:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 --2- 172.21.15.26:0/54576907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4de415cd20 0x7f4de40b8480 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 -- 172.21.15.26:0/54576907 >> 172.21.15.26:0/54576907 conn(0x7f4de406a120 msgr2=0x7f4de406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 -- 172.21.15.26:0/54576907 shutdown_connections 2023-12-19T18:16:04.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 -- 172.21.15.26:0/54576907 wait complete. 2023-12-19T18:16:04.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 Processor -- start 2023-12-19T18:16:04.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 -- start start 2023-12-19T18:16:04.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4de40b4dd0 0x7f4de4151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4de40b89c0 0x7f4de40cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de415cd20 0x7f4de40d2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de41f5e30 con 0x7f4de415cd20 2023-12-19T18:16:04.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de41f6970 con 0x7f4de40b89c0 2023-12-19T18:16:04.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de8fb3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de41516d0 con 0x7f4de40b4dd0 2023-12-19T18:16:04.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de415cd20 0x7f4de40d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de2d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de415cd20 0x7f4de40d2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44030/0 (socket says 172.21.15.26:44030) 2023-12-19T18:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de2d9d640 1 -- 172.21.15.26:0/3084052045 learned_addr learned my addr 172.21.15.26:0/3084052045 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4ddbfff640 1 --2- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4de40b89c0 0x7f4de40cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.334+0000 7f4de259c640 1 --2- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4de40b4dd0 0x7f4de4151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4de2d9d640 1 -- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4de40b4dd0 msgr2=0x7f4de4151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4de2d9d640 1 --2- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4de40b4dd0 0x7f4de4151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4de2d9d640 1 -- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4de40b89c0 msgr2=0x7f4de40cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4de2d9d640 1 --2- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4de40b89c0 0x7f4de40cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4de2d9d640 1 -- 172.21.15.26:0/3084052045 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd4009420 con 0x7f4de415cd20 2023-12-19T18:16:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4de2d9d640 1 --2- 172.21.15.26:0/3084052045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de415cd20 0x7f4de40d2620 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc002240 tx=0x7f4ddc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4ddaffd640 1 -- 172.21.15.26:0/3084052045 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc00ac30 con 0x7f4de415cd20 2023-12-19T18:16:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4ddaffd640 1 -- 172.21.15.26:0/3084052045 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ddc00ad90 con 0x7f4de415cd20 2023-12-19T18:16:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4ddaffd640 1 -- 172.21.15.26:0/3084052045 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc012600 con 0x7f4de415cd20 2023-12-19T18:16:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4de8fb3640 1 -- 172.21.15.26:0/3084052045 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de41f6040 con 0x7f4de415cd20 2023-12-19T18:16:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4de8fb3640 1 -- 172.21.15.26:0/3084052045 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de41f6600 con 0x7f4de415cd20 2023-12-19T18:16:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4de8fb3640 1 -- 172.21.15.26:0/3084052045 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de41f6b20 con 0x7f4de415cd20 2023-12-19T18:16:04.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4ddaffd640 1 -- 172.21.15.26:0/3084052045 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 114) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f4ddc012760 con 0x7f4de415cd20 2023-12-19T18:16:04.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4ddaffd640 1 --2- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4dc4065ca0 0x7f4dc4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4ddaffd640 1 -- 172.21.15.26:0/3084052045 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6873+0+0 (secure 0 0 0) 0x7f4ddc083950 con 0x7f4de415cd20 2023-12-19T18:16:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4de259c640 1 -- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4dc4065ca0 msgr2=0x7f4dc4068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:04.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.338+0000 7f4de259c640 1 --2- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4dc4065ca0 0x7f4dc4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:16:04.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.342+0000 7f4ddaffd640 1 -- 172.21.15.26:0/3084052045 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4ddc050b30 con 0x7f4de415cd20 2023-12-19T18:16:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.510+0000 7f4de8fb3640 1 -- 172.21.15.26:0/3084052045 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4de4074270 con 0x7f4de415cd20 2023-12-19T18:16:04.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.510+0000 7f4ddaffd640 1 -- 172.21.15.26:0/3084052045 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v199) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ddc001b90 con 0x7f4de415cd20 2023-12-19T18:16:04.513 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:16:04.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.514+0000 7f4de8fb3640 1 -- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4dc4065ca0 msgr2=0x7f4dc4068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:16:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.514+0000 7f4de8fb3640 1 --2- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4dc4065ca0 0x7f4dc4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.514+0000 7f4de8fb3640 1 -- 172.21.15.26:0/3084052045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de415cd20 msgr2=0x7f4de40d2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.514+0000 7f4de8fb3640 1 --2- 172.21.15.26:0/3084052045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de415cd20 0x7f4de40d2620 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc002240 tx=0x7f4ddc00cf70 comp rx=0 tx=0).stop 2023-12-19T18:16:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.514+0000 7f4de8fb3640 1 -- 172.21.15.26:0/3084052045 shutdown_connections 2023-12-19T18:16:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.514+0000 7f4de8fb3640 1 --2- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4dc4065ca0 0x7f4dc4068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.514+0000 7f4de8fb3640 1 --2- 172.21.15.26:0/3084052045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4de415cd20 0x7f4de40d2620 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.514+0000 7f4de8fb3640 1 --2- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4de40b89c0 0x7f4de40cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.514+0000 7f4de8fb3640 1 --2- 172.21.15.26:0/3084052045 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4de40b4dd0 0x7f4de4151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.514+0000 7f4de8fb3640 1 -- 172.21.15.26:0/3084052045 >> 172.21.15.26:0/3084052045 conn(0x7f4de406a120 msgr2=0x7f4de415afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.514+0000 7f4de8fb3640 1 -- 172.21.15.26:0/3084052045 shutdown_connections 2023-12-19T18:16:04.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.514+0000 7f4de8fb3640 1 -- 172.21.15.26:0/3084052045 wait complete. 2023-12-19T18:16:04.536 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:16:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.682+0000 7f11953b3640 1 Processor -- start 2023-12-19T18:16:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.682+0000 7f11953b3640 1 -- start start 2023-12-19T18:16:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f11953b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f11900b8460 0x7f11900b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f11953b3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11900d2370 con 0x7f1190160330 2023-12-19T18:16:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f11953b3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11900d2800 con 0x7f11900b4a90 2023-12-19T18:16:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f11953b3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11900b4f20 con 0x7f11900b8460 2023-12-19T18:16:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118effd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1190160330 0x7f11900d0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43084/0 (socket says 172.21.15.26:43084) 2023-12-19T18:16:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118effd640 1 -- 172.21.15.26:0/1868542070 learned_addr learned my addr 172.21.15.26:0/1868542070 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f1187fff640 1 --2- 172.21.15.26:0/1868542070 >> v2:172.21.15.160:3300/0 conn(0x7f11900b8460 0x7f11900b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625524813 0 0) 0x7f11900d2370 con 0x7f1190160330 2023-12-19T18:16:04.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f117c003530 con 0x7f1190160330 2023-12-19T18:16:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653180919 0 0) 0x7f11900d2800 con 0x7f11900b4a90 2023-12-19T18:16:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11900d2370 con 0x7f11900b4a90 2023-12-19T18:16:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2561684947 0 0) 0x7f117c003530 con 0x7f1190160330 2023-12-19T18:16:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 >> v2:172.21.15.160:3300/0 conn(0x7f11900b8460 msgr2=0x7f11900b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 --2- 172.21.15.26:0/1868542070 >> v2:172.21.15.160:3300/0 conn(0x7f11900b8460 0x7f11900b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 >> v1:172.21.15.160:6789/0 conn(0x7f11900b4a90 legacy=0x7f11900b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11900d1760 con 0x7f1190160330 2023-12-19T18:16:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f1180001680 con 0x7f1190160330 2023-12-19T18:16:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 >> v1:172.21.15.26:6789/0 conn(0x7f1190160330 legacy=0x7f11900d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 --2- 172.21.15.26:0/1868542070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c003b10 0x7f117c003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 --2- 172.21.15.26:0/1868542070 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f117c0047d0 0x7f117c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 --2- 172.21.15.26:0/1868542070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f117c005190 0x7f117c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11900b4f20 con 0x7f117c003b10 2023-12-19T18:16:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c00c010 con 0x7f117c005190 2023-12-19T18:16:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c00c1d0 con 0x7f117c0047d0 2023-12-19T18:16:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118effd640 1 --2- 172.21.15.26:0/1868542070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c003b10 0x7f117c003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118f7fe640 1 --2- 172.21.15.26:0/1868542070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f117c005190 0x7f117c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f1187fff640 1 --2- 172.21.15.26:0/1868542070 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f117c0047d0 0x7f117c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118effd640 1 -- 172.21.15.26:0/1868542070 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f117c0047d0 msgr2=0x7f117c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118effd640 1 --2- 172.21.15.26:0/1868542070 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f117c0047d0 0x7f117c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118effd640 1 -- 172.21.15.26:0/1868542070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f117c005190 msgr2=0x7f117c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118effd640 1 --2- 172.21.15.26:0/1868542070 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f117c005190 0x7f117c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118effd640 1 -- 172.21.15.26:0/1868542070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c003530 con 0x7f117c003b10 2023-12-19T18:16:04.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.686+0000 7f118f7fe640 1 -- 172.21.15.26:0/1868542070 reap_dead start 2023-12-19T18:16:04.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.690+0000 7f118effd640 1 --2- 172.21.15.26:0/1868542070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c003b10 0x7f117c003f10 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f118000c160 tx=0x7f118001cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d892a039a38f77 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.690+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11800a5920 con 0x7f117c003b10 2023-12-19T18:16:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.690+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11800a5a80 con 0x7f117c003b10 2023-12-19T18:16:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.690+0000 7f118e7fc640 1 -- 172.21.15.26:0/1868542070 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11800a5e90 con 0x7f117c003b10 2023-12-19T18:16:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.690+0000 7f11953b3640 1 -- 172.21.15.26:0/1868542070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c003b10 msgr2=0x7f117c003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.690+0000 7f11953b3640 1 --2- 172.21.15.26:0/1868542070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c003b10 0x7f117c003f10 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f118000c160 tx=0x7f118001cb90 comp rx=0 tx=0).stop 2023-12-19T18:16:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.690+0000 7f11953b3640 1 -- 172.21.15.26:0/1868542070 shutdown_connections 2023-12-19T18:16:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.690+0000 7f11953b3640 1 --2- 172.21.15.26:0/1868542070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f117c003b10 0x7f117c003f10 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.690+0000 7f11953b3640 1 -- 172.21.15.26:0/1868542070 >> 172.21.15.26:0/1868542070 conn(0x7f1190065ee0 msgr2=0x7f11900cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.690+0000 7f11953b3640 1 -- 172.21.15.26:0/1868542070 shutdown_connections 2023-12-19T18:16:04.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.690+0000 7f11953b3640 1 -- 172.21.15.26:0/1868542070 wait complete. 2023-12-19T18:16:04.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f11953b3640 1 Processor -- start 2023-12-19T18:16:04.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f11953b3640 1 -- start start 2023-12-19T18:16:04.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f11953b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f117c003b10 0x7f119015e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f11953b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1190150af0 0x7f11901519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f11953b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1190151f30 0x7f11900b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f11953b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119015d920 con 0x7f1190150af0 2023-12-19T18:16:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f11953b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11901f51a0 con 0x7f117c003b10 2023-12-19T18:16:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f11953b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119015db70 con 0x7f1190151f30 2023-12-19T18:16:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f1187fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1190150af0 0x7f11901519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f1187fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1190150af0 0x7f11901519f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44034/0 (socket says 172.21.15.26:44034) 2023-12-19T18:16:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f1187fff640 1 -- 172.21.15.26:0/652174462 learned_addr learned my addr 172.21.15.26:0/652174462 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f118effd640 1 --2- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f117c003b10 0x7f119015e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f118f7fe640 1 --2- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1190151f30 0x7f11900b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f1187fff640 1 -- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1190151f30 msgr2=0x7f11900b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f1187fff640 1 --2- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1190151f30 0x7f11900b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f1187fff640 1 -- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f117c003b10 msgr2=0x7f119015e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f1187fff640 1 --2- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f117c003b10 0x7f119015e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f1187fff640 1 -- 172.21.15.26:0/652174462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119015de50 con 0x7f1190150af0 2023-12-19T18:16:04.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f1187fff640 1 --2- 172.21.15.26:0/652174462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1190150af0 0x7f11901519f0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f1178011aa0 tx=0x7f1178011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f11877fe640 1 -- 172.21.15.26:0/652174462 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117800fac0 con 0x7f1190150af0 2023-12-19T18:16:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f11877fe640 1 -- 172.21.15.26:0/652174462 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117800fc20 con 0x7f1190150af0 2023-12-19T18:16:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f11953b3640 1 -- 172.21.15.26:0/652174462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119015e130 con 0x7f1190150af0 2023-12-19T18:16:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f11877fe640 1 -- 172.21.15.26:0/652174462 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178014440 con 0x7f1190150af0 2023-12-19T18:16:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.694+0000 7f11953b3640 1 -- 172.21.15.26:0/652174462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119015ee70 con 0x7f1190150af0 2023-12-19T18:16:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.698+0000 7f11953b3640 1 -- 172.21.15.26:0/652174462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11900d0e90 con 0x7f1190150af0 2023-12-19T18:16:04.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.698+0000 7f11877fe640 1 -- 172.21.15.26:0/652174462 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 114) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f11780145a0 con 0x7f1190150af0 2023-12-19T18:16:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.698+0000 7f11877fe640 1 --2- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1170065ca0 0x7f1170068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.698+0000 7f11877fe640 1 -- 172.21.15.26:0/652174462 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f11780842a0 con 0x7f1190150af0 2023-12-19T18:16:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.698+0000 7f118effd640 1 -- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1170065ca0 msgr2=0x7f1170068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.698+0000 7f118effd640 1 --2- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1170065ca0 0x7f1170068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:16:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.702+0000 7f11877fe640 1 -- 172.21.15.26:0/652174462 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1178051470 con 0x7f1190150af0 2023-12-19T18:16:04.736 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:04.734+0000 7f183264b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:16:04.793 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:04.790+0000 7f183264b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:16:04.820 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:04.818+0000 7f5baf047180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:16:04.848 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:04.846+0000 7f183264b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:16:04.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.858+0000 7f11953b3640 1 -- 172.21.15.26:0/652174462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11900d1fd0 con 0x7f1190150af0 2023-12-19T18:16:04.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.858+0000 7f11877fe640 1 -- 172.21.15.26:0/652174462 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v199) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1178054a90 con 0x7f1190150af0 2023-12-19T18:16:04.862 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:16:04.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.862+0000 7f11953b3640 1 -- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1170065ca0 msgr2=0x7f1170068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:16:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.862+0000 7f11953b3640 1 --2- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1170065ca0 0x7f1170068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.862+0000 7f11953b3640 1 -- 172.21.15.26:0/652174462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1190150af0 msgr2=0x7f11901519f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.862+0000 7f11953b3640 1 --2- 172.21.15.26:0/652174462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1190150af0 0x7f11901519f0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f1178011aa0 tx=0x7f1178011ea0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.862+0000 7f11953b3640 1 -- 172.21.15.26:0/652174462 shutdown_connections 2023-12-19T18:16:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.862+0000 7f11953b3640 1 --2- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1170065ca0 0x7f1170068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.862+0000 7f11953b3640 1 --2- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1190151f30 0x7f11900b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.862+0000 7f11953b3640 1 --2- 172.21.15.26:0/652174462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1190150af0 0x7f11901519f0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.862+0000 7f11953b3640 1 --2- 172.21.15.26:0/652174462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f117c003b10 0x7f119015e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.862+0000 7f11953b3640 1 -- 172.21.15.26:0/652174462 >> 172.21.15.26:0/652174462 conn(0x7f1190065ee0 msgr2=0x7f1190152c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.862+0000 7f11953b3640 1 -- 172.21.15.26:0/652174462 shutdown_connections 2023-12-19T18:16:04.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:04.862+0000 7f11953b3640 1 -- 172.21.15.26:0/652174462 wait complete. 2023-12-19T18:16:04.883 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:04.882+0000 7f5baf047180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:16:04.885 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:16:04.932 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:04.930+0000 7f183264b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:16:04.944 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:04.942+0000 7f5baf047180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:16:04.993 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:04.990+0000 7f183264b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:16:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.034+0000 7fdc2fb39640 1 Processor -- start 2023-12-19T18:16:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.034+0000 7fdc2fb39640 1 -- start start 2023-12-19T18:16:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.034+0000 7fdc2fb39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdc2815ccf0 0x7fdc280d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.034+0000 7fdc2fb39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc280b4600 0x7fdc280b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.034+0000 7fdc2fb39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc280b4f50 con 0x7fdc2815d0d0 2023-12-19T18:16:05.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.034+0000 7fdc2fb39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2815e200 con 0x7fdc2815ccf0 2023-12-19T18:16:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.034+0000 7fdc2fb39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280d0aa0 con 0x7fdc280b4600 2023-12-19T18:16:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.034+0000 7fdc2d8d5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdc2815ccf0 0x7fdc280d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.038+0000 7fdc2d8d5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdc2815ccf0 0x7fdc280d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44040/0 (socket says 172.21.15.26:44040) 2023-12-19T18:16:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.038+0000 7fdc2d8d5640 1 -- 172.21.15.26:0/3074419868 learned_addr learned my addr 172.21.15.26:0/3074419868 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.038+0000 7fdc2e0d6640 1 --2- 172.21.15.26:0/3074419868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc280b4600 0x7fdc280b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.038+0000 7fdc2d8d5640 1 -- 172.21.15.26:0/3074419868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc280b4600 msgr2=0x7fdc280b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.038+0000 7fdc2d8d5640 1 --2- 172.21.15.26:0/3074419868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc280b4600 0x7fdc280b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.038+0000 7fdc2d8d5640 1 -- 172.21.15.26:0/3074419868 >> v1:172.21.15.160:6789/0 conn(0x7fdc2815d0d0 legacy=0x7fdc280b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.038+0000 7fdc2d8d5640 1 -- 172.21.15.26:0/3074419868 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc280d1380 con 0x7fdc2815ccf0 2023-12-19T18:16:05.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.038+0000 7fdc2c8d3640 1 -- 172.21.15.26:0/3074419868 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881942328 0 0) 0x7fdc280b4f50 con 0x7fdc2815d0d0 2023-12-19T18:16:05.041 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:05.038+0000 7f5baf047180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:16:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.042+0000 7fdc2d8d5640 1 --2- 172.21.15.26:0/3074419868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2815ccf0 0x7fdc280d0e40 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fdc280d1570 tx=0x7fdc2401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7db6cd02e1f93ad server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.042+0000 7fdc2c8d3640 1 -- 172.21.15.26:0/3074419868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc240aa650 con 0x7fdc2815ccf0 2023-12-19T18:16:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.042+0000 7fdc2c8d3640 1 -- 172.21.15.26:0/3074419868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc240aa7b0 con 0x7fdc2815ccf0 2023-12-19T18:16:05.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.042+0000 7fdc2c8d3640 1 -- 172.21.15.26:0/3074419868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc240aab40 con 0x7fdc2815ccf0 2023-12-19T18:16:05.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.042+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3074419868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2815ccf0 msgr2=0x7fdc280d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.042+0000 7fdc2fb39640 1 --2- 172.21.15.26:0/3074419868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2815ccf0 0x7fdc280d0e40 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fdc280d1570 tx=0x7fdc2401c4f0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.042+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3074419868 shutdown_connections 2023-12-19T18:16:05.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.042+0000 7fdc2fb39640 1 --2- 172.21.15.26:0/3074419868 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc280b4600 0x7fdc280b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.042+0000 7fdc2fb39640 1 --2- 172.21.15.26:0/3074419868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2815ccf0 0x7fdc280d0e40 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.042+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3074419868 >> 172.21.15.26:0/3074419868 conn(0x7fdc28065ee0 msgr2=0x7fdc280cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:05.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.042+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3074419868 shutdown_connections 2023-12-19T18:16:05.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.042+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3074419868 wait complete. 2023-12-19T18:16:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.046+0000 7fdc2fb39640 1 Processor -- start 2023-12-19T18:16:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.046+0000 7fdc2fb39640 1 -- start start 2023-12-19T18:16:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.046+0000 7fdc2fb39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc280b4600 0x7fdc280c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.046+0000 7fdc2fb39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2815ccf0 0x7fdc281575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.046+0000 7fdc2fb39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc2815d0d0 0x7fdc28166bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.046+0000 7fdc2fb39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc28158860 con 0x7fdc2815ccf0 2023-12-19T18:16:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.046+0000 7fdc2fb39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc280c9350 con 0x7fdc280b4600 2023-12-19T18:16:05.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.046+0000 7fdc2fb39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc28158ab0 con 0x7fdc2815d0d0 2023-12-19T18:16:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.046+0000 7fdc2d0d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2815ccf0 0x7fdc281575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.046+0000 7fdc2d0d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2815ccf0 0x7fdc281575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44044/0 (socket says 172.21.15.26:44044) 2023-12-19T18:16:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.046+0000 7fdc2d8d5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc280b4600 0x7fdc280c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.046+0000 7fdc2d0d4640 1 -- 172.21.15.26:0/3031690468 learned_addr learned my addr 172.21.15.26:0/3031690468 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.046+0000 7fdc2e0d6640 1 --2- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc2815d0d0 0x7fdc28166bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc2d0d4640 1 -- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc2815d0d0 msgr2=0x7fdc28166bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc2d0d4640 1 --2- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc2815d0d0 0x7fdc28166bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc2d0d4640 1 -- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc280b4600 msgr2=0x7fdc280c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc2d0d4640 1 --2- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc280b4600 0x7fdc280c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc2d0d4640 1 -- 172.21.15.26:0/3031690468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc240094d0 con 0x7fdc2815ccf0 2023-12-19T18:16:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc2d8d5640 1 --2- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc280b4600 0x7fdc280c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:05.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc2d0d4640 1 --2- 172.21.15.26:0/3031690468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2815ccf0 0x7fdc281575a0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fdc18001ec0 tx=0x7fdc1800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc1e7fc640 1 -- 172.21.15.26:0/3031690468 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180054e0 con 0x7fdc2815ccf0 2023-12-19T18:16:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc1e7fc640 1 -- 172.21.15.26:0/3031690468 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc18003600 con 0x7fdc2815ccf0 2023-12-19T18:16:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc1e7fc640 1 -- 172.21.15.26:0/3031690468 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180039b0 con 0x7fdc2815ccf0 2023-12-19T18:16:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3031690468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc28158d90 con 0x7fdc2815ccf0 2023-12-19T18:16:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3031690468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc280c8fb0 con 0x7fdc2815ccf0 2023-12-19T18:16:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3031690468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc28074270 con 0x7fdc2815ccf0 2023-12-19T18:16:05.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc1e7fc640 1 -- 172.21.15.26:0/3031690468 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 114) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fdc18003b10 con 0x7fdc2815ccf0 2023-12-19T18:16:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc1e7fc640 1 --2- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdc0c065ca0 0x7fdc0c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.050+0000 7fdc1e7fc640 1 -- 172.21.15.26:0/3031690468 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6857+0+0 (secure 0 0 0) 0x7fdc18082fe0 con 0x7fdc2815ccf0 2023-12-19T18:16:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.054+0000 7fdc2d8d5640 1 -- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdc0c065ca0 msgr2=0x7fdc0c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:05.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.054+0000 7fdc2d8d5640 1 --2- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdc0c065ca0 0x7fdc0c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:16:05.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.054+0000 7fdc1e7fc640 1 -- 172.21.15.26:0/3031690468 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdc18050210 con 0x7fdc2815ccf0 2023-12-19T18:16:05.078 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:05.074+0000 7f183264b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:16:05.107 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:05.106+0000 7f5baf047180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:16:05.135 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:05.130+0000 7f183264b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:16:05.199 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:05.198+0000 7f5baf047180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:16:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.218+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3031690468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc280b25e0 con 0x7fdc2815ccf0 2023-12-19T18:16:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.218+0000 7fdc1e7fc640 1 -- 172.21.15.26:0/3031690468 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v199) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc18017070 con 0x7fdc2815ccf0 2023-12-19T18:16:05.221 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:16:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.218+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdc0c065ca0 msgr2=0x7fdc0c068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:16:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.218+0000 7fdc2fb39640 1 --2- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdc0c065ca0 0x7fdc0c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.218+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3031690468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2815ccf0 msgr2=0x7fdc281575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.218+0000 7fdc2fb39640 1 --2- 172.21.15.26:0/3031690468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2815ccf0 0x7fdc281575a0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fdc18001ec0 tx=0x7fdc1800c8e0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.222+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3031690468 shutdown_connections 2023-12-19T18:16:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.222+0000 7fdc2fb39640 1 --2- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdc0c065ca0 0x7fdc0c068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.222+0000 7fdc2fb39640 1 --2- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc2815d0d0 0x7fdc28166bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.222+0000 7fdc2fb39640 1 --2- 172.21.15.26:0/3031690468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc2815ccf0 0x7fdc281575a0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.222+0000 7fdc2fb39640 1 --2- 172.21.15.26:0/3031690468 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc280b4600 0x7fdc280c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.222+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3031690468 >> 172.21.15.26:0/3031690468 conn(0x7fdc28065ee0 msgr2=0x7fdc280b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.222+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3031690468 shutdown_connections 2023-12-19T18:16:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.222+0000 7fdc2fb39640 1 -- 172.21.15.26:0/3031690468 wait complete. 2023-12-19T18:16:05.243 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:16:05.261 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:05.258+0000 7f5baf047180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:16:05.277 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:05.274+0000 7f183264b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:16:05.336 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:05.334+0000 7f183264b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:16:05.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.386+0000 7ff99aec4640 1 Processor -- start 2023-12-19T18:16:05.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.386+0000 7ff99aec4640 1 -- start start 2023-12-19T18:16:05.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.386+0000 7ff99aec4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff994155030 0x7ff99415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.386+0000 7ff99aec4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff99415f370 0x7ff99406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.386+0000 7ff99aec4640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99406c970 con 0x7ff9940ce740 2023-12-19T18:16:05.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.386+0000 7ff99aec4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99415b1f0 con 0x7ff99415f370 2023-12-19T18:16:05.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.386+0000 7ff99aec4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99415b4d0 con 0x7ff994155030 2023-12-19T18:16:05.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff999461640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff9940ce740 0x7ff9940ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43094/0 (socket says 172.21.15.26:43094) 2023-12-19T18:16:05.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff999461640 1 -- 172.21.15.26:0/636929102 learned_addr learned my addr 172.21.15.26:0/636929102 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:05.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff993fff640 1 --2- 172.21.15.26:0/636929102 >> v2:172.21.15.26:3300/0 conn(0x7ff99415f370 0x7ff99406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff998c60640 1 --2- 172.21.15.26:0/636929102 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff994155030 0x7ff99415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff9937fe640 1 -- 172.21.15.26:0/636929102 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765508417 0 0) 0x7ff99406c970 con 0x7ff9940ce740 2023-12-19T18:16:05.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff9937fe640 1 -- 172.21.15.26:0/636929102 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff984003530 con 0x7ff9940ce740 2023-12-19T18:16:05.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff993fff640 1 -- 172.21.15.26:0/636929102 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff994155030 msgr2=0x7ff99415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:05.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff993fff640 1 --2- 172.21.15.26:0/636929102 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff994155030 0x7ff99415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff993fff640 1 -- 172.21.15.26:0/636929102 >> v1:172.21.15.26:6789/0 conn(0x7ff9940ce740 legacy=0x7ff9940ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff993fff640 1 -- 172.21.15.26:0/636929102 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99415b7b0 con 0x7ff99415f370 2023-12-19T18:16:05.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff993fff640 1 --2- 172.21.15.26:0/636929102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415f370 0x7ff99406c080 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7ff9800097c0 tx=0x7ff98001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75876c6ec5a3e5e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:05.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff9937fe640 1 -- 172.21.15.26:0/636929102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9800a6040 con 0x7ff99415f370 2023-12-19T18:16:05.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff9937fe640 1 -- 172.21.15.26:0/636929102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9800aa600 con 0x7ff99415f370 2023-12-19T18:16:05.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.390+0000 7ff9937fe640 1 -- 172.21.15.26:0/636929102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9800aa990 con 0x7ff99415f370 2023-12-19T18:16:05.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 -- 172.21.15.26:0/636929102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415f370 msgr2=0x7ff99406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 --2- 172.21.15.26:0/636929102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415f370 0x7ff99406c080 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7ff9800097c0 tx=0x7ff98001c480 comp rx=0 tx=0).stop 2023-12-19T18:16:05.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 -- 172.21.15.26:0/636929102 shutdown_connections 2023-12-19T18:16:05.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 --2- 172.21.15.26:0/636929102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff99415f370 0x7ff99406c080 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 --2- 172.21.15.26:0/636929102 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff994155030 0x7ff99415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 -- 172.21.15.26:0/636929102 >> 172.21.15.26:0/636929102 conn(0x7ff9940658e0 msgr2=0x7ff994065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:05.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 -- 172.21.15.26:0/636929102 shutdown_connections 2023-12-19T18:16:05.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 -- 172.21.15.26:0/636929102 wait complete. 2023-12-19T18:16:05.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 Processor -- start 2023-12-19T18:16:05.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 -- start start 2023-12-19T18:16:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9940ce740 0x7ff994147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff994155030 0x7ff994159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff99415f370 0x7ff9940ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99414c7a0 con 0x7ff9940ce740 2023-12-19T18:16:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99414caf0 con 0x7ff994155030 2023-12-19T18:16:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff99aec4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99414cdd0 con 0x7ff99415f370 2023-12-19T18:16:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.394+0000 7ff998c60640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9940ce740 0x7ff994147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff998c60640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9940ce740 0x7ff994147ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44056/0 (socket says 172.21.15.26:44056) 2023-12-19T18:16:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff998c60640 1 -- 172.21.15.26:0/1714703952 learned_addr learned my addr 172.21.15.26:0/1714703952 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff993fff640 1 --2- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff994155030 0x7ff994159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff999461640 1 --2- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff99415f370 0x7ff9940ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff998c60640 1 -- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff99415f370 msgr2=0x7ff9940ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff998c60640 1 --2- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff99415f370 0x7ff9940ae570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff998c60640 1 -- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff994155030 msgr2=0x7ff994159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff998c60640 1 --2- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff994155030 0x7ff994159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff998c60640 1 -- 172.21.15.26:0/1714703952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff980009420 con 0x7ff9940ce740 2023-12-19T18:16:05.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff998c60640 1 --2- 172.21.15.26:0/1714703952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9940ce740 0x7ff994147ab0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7ff98800e930 tx=0x7ff98800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:05.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff9917fa640 1 -- 172.21.15.26:0/1714703952 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff988009690 con 0x7ff9940ce740 2023-12-19T18:16:05.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff9917fa640 1 -- 172.21.15.26:0/1714703952 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98800ca00 con 0x7ff9940ce740 2023-12-19T18:16:05.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff9917fa640 1 -- 172.21.15.26:0/1714703952 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98800cd70 con 0x7ff9940ce740 2023-12-19T18:16:05.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff99aec4640 1 -- 172.21.15.26:0/1714703952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99406c740 con 0x7ff9940ce740 2023-12-19T18:16:05.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff99aec4640 1 -- 172.21.15.26:0/1714703952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff994150100 con 0x7ff9940ce740 2023-12-19T18:16:05.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.398+0000 7ff99aec4640 1 -- 172.21.15.26:0/1714703952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99414ff60 con 0x7ff9940ce740 2023-12-19T18:16:05.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.402+0000 7ff9917fa640 1 -- 172.21.15.26:0/1714703952 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 114) v1 ==== 85444+0+0 (secure 0 0 0) 0x7ff988013470 con 0x7ff9940ce740 2023-12-19T18:16:05.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.402+0000 7ff9917fa640 1 --2- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff978065ca0 0x7ff978068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.402+0000 7ff9917fa640 1 -- 172.21.15.26:0/1714703952 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6857+0+0 (secure 0 0 0) 0x7ff988083c20 con 0x7ff9940ce740 2023-12-19T18:16:05.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.402+0000 7ff993fff640 1 -- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff978065ca0 msgr2=0x7ff978068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:05.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.402+0000 7ff993fff640 1 --2- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff978065ca0 0x7ff978068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:16:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.406+0000 7ff9917fa640 1 -- 172.21.15.26:0/1714703952 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff99414ff60 con 0x7ff9940ce740 2023-12-19T18:16:05.419 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:05.418+0000 7f5baf047180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:16:05.482 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:05.478+0000 7f5baf047180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:16:05.575 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:16:05.576 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:16:05.576 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:16:05.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.578+0000 7ff99aec4640 1 -- 172.21.15.26:0/1714703952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff99414ff60 con 0x7ff9940ce740 2023-12-19T18:16:05.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.578+0000 7ff9917fa640 1 -- 172.21.15.26:0/1714703952 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v199) v1 ==== 80+0+468 (secure 0 0 0) 0x7ff99414ff60 con 0x7ff9940ce740 2023-12-19T18:16:05.581 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2345576187 2023-12-20T18:14:50.374461+0000 2023-12-19T18:16:05.581 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1296996055 2023-12-20T18:14:50.374461+0000 2023-12-19T18:16:05.581 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/4184305047 2023-12-20T18:14:50.374461+0000 2023-12-19T18:16:05.582 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2077682555 2023-12-20T18:14:50.374461+0000 2023-12-19T18:16:05.582 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1610506837 2023-12-20T18:14:50.374461+0000 2023-12-19T18:16:05.582 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6833/48394 2023-12-20T18:14:50.374461+0000 2023-12-19T18:16:05.582 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6832/48394 2023-12-20T18:14:50.374461+0000 2023-12-19T18:16:05.582 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3136437506 2023-12-20T18:14:50.374461+0000 2023-12-19T18:16:05.582 INFO:teuthology.orchestra.run.smithi026.stderr:listed 8 entries 2023-12-19T18:16:05.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.582+0000 7ff99aec4640 1 -- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff978065ca0 msgr2=0x7ff978068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:16:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.582+0000 7ff99aec4640 1 --2- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff978065ca0 0x7ff978068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.582+0000 7ff99aec4640 1 -- 172.21.15.26:0/1714703952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9940ce740 msgr2=0x7ff994147ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.582+0000 7ff99aec4640 1 --2- 172.21.15.26:0/1714703952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9940ce740 0x7ff994147ab0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7ff98800e930 tx=0x7ff98800edf0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.582+0000 7ff99aec4640 1 -- 172.21.15.26:0/1714703952 shutdown_connections 2023-12-19T18:16:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.582+0000 7ff99aec4640 1 --2- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff978065ca0 0x7ff978068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.582+0000 7ff99aec4640 1 --2- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff99415f370 0x7ff9940ae570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.582+0000 7ff99aec4640 1 --2- 172.21.15.26:0/1714703952 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff994155030 0x7ff994159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.582+0000 7ff99aec4640 1 --2- 172.21.15.26:0/1714703952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9940ce740 0x7ff994147ab0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.582+0000 7ff99aec4640 1 -- 172.21.15.26:0/1714703952 >> 172.21.15.26:0/1714703952 conn(0x7ff9940658e0 msgr2=0x7ff99415d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.582+0000 7ff99aec4640 1 -- 172.21.15.26:0/1714703952 shutdown_connections 2023-12-19T18:16:05.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.582+0000 7ff99aec4640 1 -- 172.21.15.26:0/1714703952 wait complete. 2023-12-19T18:16:05.585 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:05.582+0000 7f183264b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:16:05.604 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T18:16:05.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:05.604 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-19T18:16:05.716 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T18:16:05.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:05.716 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:16:05.719 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:05.714+0000 7f183264b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:16:05.726 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:16:05.726 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:05.735 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:16:05.735 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:16:05.735 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:05.737 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:16:05.737 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:16:05.737 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:16:05.746 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:05.742+0000 7f5baf047180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:16:05.748 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:16:05.748 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:16:05.748 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:05.788 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:16:05.788 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:05.845 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:05.842+0000 7f183264b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:16:05.892 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:05.890+0000 7f5baf047180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:16:05.913 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:05.910+0000 7f183264b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:16:05.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f937df09640 1 Processor -- start 2023-12-19T18:16:05.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f937df09640 1 -- start start 2023-12-19T18:16:05.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f937df09640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9378154220 0x7f937815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f937df09640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f937806b3d0 0x7f93780ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f937df09640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f937806bc30 con 0x7f93780cec40 2023-12-19T18:16:05.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f937df09640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93780cf0d0 con 0x7f9378154220 2023-12-19T18:16:05.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f937df09640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93780cc6e0 con 0x7f937806b3d0 2023-12-19T18:16:05.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f93777fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9378154220 0x7f937815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f9377fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f93780cec40 0x7f937815d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32968/0 (socket says 172.21.15.26:32968) 2023-12-19T18:16:05.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f9377fff640 1 -- 172.21.15.26:0/1573337159 learned_addr learned my addr 172.21.15.26:0/1573337159 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:05.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f936ffff640 1 --2- 172.21.15.26:0/1573337159 >> v2:172.21.15.160:3300/0 conn(0x7f937806b3d0 0x7f93780ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f9376ffd640 1 -- 172.21.15.26:0/1573337159 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497696030 0 0) 0x7f937806bc30 con 0x7f93780cec40 2023-12-19T18:16:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f9376ffd640 1 -- 172.21.15.26:0/1573337159 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9364003530 con 0x7f93780cec40 2023-12-19T18:16:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f93777fe640 1 -- 172.21.15.26:0/1573337159 >> v2:172.21.15.160:3300/0 conn(0x7f937806b3d0 msgr2=0x7f93780ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f93777fe640 1 --2- 172.21.15.26:0/1573337159 >> v2:172.21.15.160:3300/0 conn(0x7f937806b3d0 0x7f93780ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f93777fe640 1 -- 172.21.15.26:0/1573337159 >> v1:172.21.15.26:6789/0 conn(0x7f93780cec40 legacy=0x7f937815d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.934+0000 7f93777fe640 1 -- 172.21.15.26:0/1573337159 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937815afc0 con 0x7f9378154220 2023-12-19T18:16:05.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.938+0000 7f93777fe640 1 --2- 172.21.15.26:0/1573337159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9378154220 0x7f937815aa80 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f93780cf880 tx=0x7f936801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6598157430f23ffc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.938+0000 7f9376ffd640 1 -- 172.21.15.26:0/1573337159 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368004230 con 0x7f9378154220 2023-12-19T18:16:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.938+0000 7f9376ffd640 1 -- 172.21.15.26:0/1573337159 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93680a7050 con 0x7f9378154220 2023-12-19T18:16:05.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.938+0000 7f9376ffd640 1 -- 172.21.15.26:0/1573337159 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368003620 con 0x7f9378154220 2023-12-19T18:16:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.938+0000 7f937df09640 1 -- 172.21.15.26:0/1573337159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9378154220 msgr2=0x7f937815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.938+0000 7f937df09640 1 --2- 172.21.15.26:0/1573337159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9378154220 0x7f937815aa80 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f93780cf880 tx=0x7f936801c430 comp rx=0 tx=0).stop 2023-12-19T18:16:05.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.938+0000 7f937df09640 1 -- 172.21.15.26:0/1573337159 shutdown_connections 2023-12-19T18:16:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.938+0000 7f937df09640 1 --2- 172.21.15.26:0/1573337159 >> v2:172.21.15.160:3300/0 conn(0x7f937806b3d0 0x7f93780ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.938+0000 7f937df09640 1 --2- 172.21.15.26:0/1573337159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9378154220 0x7f937815aa80 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.938+0000 7f937df09640 1 -- 172.21.15.26:0/1573337159 >> 172.21.15.26:0/1573337159 conn(0x7f93780658e0 msgr2=0x7f9378065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.938+0000 7f937df09640 1 -- 172.21.15.26:0/1573337159 shutdown_connections 2023-12-19T18:16:05.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.938+0000 7f937df09640 1 -- 172.21.15.26:0/1573337159 wait complete. 2023-12-19T18:16:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f937df09640 1 Processor -- start 2023-12-19T18:16:05.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f937df09640 1 -- start start 2023-12-19T18:16:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f937df09640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f937806b3d0 0x7f9378147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f937df09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93780cec40 0x7f9378159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f937df09640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9378154220 0x7f9378157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f937df09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9378145bc0 con 0x7f93780cec40 2023-12-19T18:16:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f937df09640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93781599b0 con 0x7f9378154220 2023-12-19T18:16:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f937df09640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9378145870 con 0x7f937806b3d0 2023-12-19T18:16:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f936ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93780cec40 0x7f9378159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f936ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93780cec40 0x7f9378159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48230/0 (socket says 172.21.15.26:48230) 2023-12-19T18:16:05.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f93777fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f937806b3d0 0x7f9378147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f936ffff640 1 -- 172.21.15.26:0/3465036568 learned_addr learned my addr 172.21.15.26:0/3465036568 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f9377fff640 1 --2- 172.21.15.26:0/3465036568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9378154220 0x7f9378157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f936ffff640 1 -- 172.21.15.26:0/3465036568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f937806b3d0 msgr2=0x7f9378147870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f936ffff640 1 --2- 172.21.15.26:0/3465036568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f937806b3d0 0x7f9378147870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f936ffff640 1 -- 172.21.15.26:0/3465036568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9378154220 msgr2=0x7f9378157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f936ffff640 1 --2- 172.21.15.26:0/3465036568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9378154220 0x7f9378157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f936ffff640 1 -- 172.21.15.26:0/3465036568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9368009420 con 0x7f93780cec40 2023-12-19T18:16:05.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f936ffff640 1 --2- 172.21.15.26:0/3465036568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93780cec40 0x7f9378159470 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f936000e8a0 tx=0x7f936000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f9374ff9640 1 -- 172.21.15.26:0/3465036568 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360009510 con 0x7f93780cec40 2023-12-19T18:16:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f9374ff9640 1 -- 172.21.15.26:0/3465036568 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936000c8e0 con 0x7f93780cec40 2023-12-19T18:16:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f9374ff9640 1 -- 172.21.15.26:0/3465036568 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936000cc50 con 0x7f93780cec40 2023-12-19T18:16:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f937df09640 1 -- 172.21.15.26:0/3465036568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9378158160 con 0x7f93780cec40 2023-12-19T18:16:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.942+0000 7f937df09640 1 -- 172.21.15.26:0/3465036568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9378148130 con 0x7f93780cec40 2023-12-19T18:16:05.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.946+0000 7f937df09640 1 -- 172.21.15.26:0/3465036568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9378074270 con 0x7f93780cec40 2023-12-19T18:16:05.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.946+0000 7f9374ff9640 1 -- 172.21.15.26:0/3465036568 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 114) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f9360013470 con 0x7f93780cec40 2023-12-19T18:16:05.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.946+0000 7f9374ff9640 1 --2- 172.21.15.26:0/3465036568 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9358065cd0 0x7f9358068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.946+0000 7f9374ff9640 1 -- 172.21.15.26:0/3465036568 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f9360083a10 con 0x7f93780cec40 2023-12-19T18:16:05.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.946+0000 7f93777fe640 1 -- 172.21.15.26:0/3465036568 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9358065cd0 msgr2=0x7f9358068190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:05.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.946+0000 7f93777fe640 1 --2- 172.21.15.26:0/3465036568 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9358065cd0 0x7f9358068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:16:05.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:05.950+0000 7f9374ff9640 1 -- 172.21.15.26:0/3465036568 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9360050c90 con 0x7f93780cec40 2023-12-19T18:16:06.027 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:06.026+0000 7f5baf047180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:16:06.048 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:06.046+0000 7f183264b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:16:06.103 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:06.102+0000 7f5baf047180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:16:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f937df09640 1 -- 172.21.15.26:0/3465036568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93780cdf60 con 0x7f93780cec40 2023-12-19T18:16:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f9374ff9640 1 -- 172.21.15.26:0/3465036568 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4723 (secure 0 0 0) 0x7f93600542b0 con 0x7f93780cec40 2023-12-19T18:16:06.148 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:06.148 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3922467901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3922467901},{"type":"v1","addr":"172.21.15.160:6835","nonce":3922467901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":9562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3692512090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3692512090},{"type":"v1","addr":"172.21.15.160:6837","nonce":3692512090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":10083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3961033710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3961033710},{"type":"v1","addr":"172.21.15.26:6833","nonce":3961033710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:38.527840+0000","modified":"2023-12-19T18:14:41.539799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10086":{"gid":10086,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1537720341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1537720341},{"type":"v1","addr":"172.21.15.26:6835","nonce":1537720341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:16:06.148 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 87 2023-12-19T18:16:06.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f937df09640 1 -- 172.21.15.26:0/3465036568 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9358065cd0 msgr2=0x7f9358068190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:16:06.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f937df09640 1 --2- 172.21.15.26:0/3465036568 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9358065cd0 0x7f9358068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f937df09640 1 -- 172.21.15.26:0/3465036568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93780cec40 msgr2=0x7f9378159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f937df09640 1 --2- 172.21.15.26:0/3465036568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93780cec40 0x7f9378159470 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f936000e8a0 tx=0x7f936000eca0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f937df09640 1 -- 172.21.15.26:0/3465036568 shutdown_connections 2023-12-19T18:16:06.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f937df09640 1 --2- 172.21.15.26:0/3465036568 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9358065cd0 0x7f9358068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f937df09640 1 --2- 172.21.15.26:0/3465036568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9378154220 0x7f9378157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f937df09640 1 --2- 172.21.15.26:0/3465036568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93780cec40 0x7f9378159470 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f937df09640 1 --2- 172.21.15.26:0/3465036568 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f937806b3d0 0x7f9378147870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f937df09640 1 -- 172.21.15.26:0/3465036568 >> 172.21.15.26:0/3465036568 conn(0x7f93780658e0 msgr2=0x7f9378067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:06.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f937df09640 1 -- 172.21.15.26:0/3465036568 shutdown_connections 2023-12-19T18:16:06.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.146+0000 7f937df09640 1 -- 172.21.15.26:0/3465036568 wait complete. 2023-12-19T18:16:06.171 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:06.252 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:06.250+0000 7f5baf047180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:16:06.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42f58d1640 1 Processor -- start 2023-12-19T18:16:06.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42f58d1640 1 -- start start 2023-12-19T18:16:06.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42f58d1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f42f00cfbe0 0x7f42f015de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42f58d1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f00cd240 con 0x7f42f015ccb0 2023-12-19T18:16:06.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42f58d1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f00cd520 con 0x7f42f00b8340 2023-12-19T18:16:06.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42f58d1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f00b8ac0 con 0x7f42f00cfbe0 2023-12-19T18:16:06.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42eeffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f42f015ccb0 0x7f42f00cf4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32984/0 (socket says 172.21.15.26:32984) 2023-12-19T18:16:06.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42eeffd640 1 -- 172.21.15.26:0/2199730308 learned_addr learned my addr 172.21.15.26:0/2199730308 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:06.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42ee7fc640 1 --2- 172.21.15.26:0/2199730308 >> v2:172.21.15.160:3300/0 conn(0x7f42f00cfbe0 0x7f42f015de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42edffb640 1 -- 172.21.15.26:0/2199730308 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618776575 0 0) 0x7f42f00cd240 con 0x7f42f015ccb0 2023-12-19T18:16:06.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42edffb640 1 -- 172.21.15.26:0/2199730308 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d8003530 con 0x7f42f015ccb0 2023-12-19T18:16:06.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42edffb640 1 -- 172.21.15.26:0/2199730308 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491872 0 0) 0x7f42f00cd520 con 0x7f42f00b8340 2023-12-19T18:16:06.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42edffb640 1 -- 172.21.15.26:0/2199730308 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42f00cd240 con 0x7f42f00b8340 2023-12-19T18:16:06.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42ee7fc640 1 -- 172.21.15.26:0/2199730308 >> v1:172.21.15.160:6789/0 conn(0x7f42f00b8340 legacy=0x7f42f00d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42ee7fc640 1 -- 172.21.15.26:0/2199730308 >> v1:172.21.15.26:6789/0 conn(0x7f42f015ccb0 legacy=0x7f42f00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.314+0000 7f42ee7fc640 1 -- 172.21.15.26:0/2199730308 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f00cd800 con 0x7f42f00cfbe0 2023-12-19T18:16:06.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.318+0000 7f42ee7fc640 1 --2- 172.21.15.26:0/2199730308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f00cfbe0 0x7f42f015de70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f42e40097c0 tx=0x7f42e401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4104947c234fa922 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:06.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.318+0000 7f42edffb640 1 -- 172.21.15.26:0/2199730308 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e40ac640 con 0x7f42f00cfbe0 2023-12-19T18:16:06.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.318+0000 7f42edffb640 1 -- 172.21.15.26:0/2199730308 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e40ac7a0 con 0x7f42f00cfbe0 2023-12-19T18:16:06.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.318+0000 7f42edffb640 1 -- 172.21.15.26:0/2199730308 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e40acb30 con 0x7f42f00cfbe0 2023-12-19T18:16:06.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 -- 172.21.15.26:0/2199730308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f00cfbe0 msgr2=0x7f42f015de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 --2- 172.21.15.26:0/2199730308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f00cfbe0 0x7f42f015de70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f42e40097c0 tx=0x7f42e401c480 comp rx=0 tx=0).stop 2023-12-19T18:16:06.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 -- 172.21.15.26:0/2199730308 shutdown_connections 2023-12-19T18:16:06.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 --2- 172.21.15.26:0/2199730308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f00cfbe0 0x7f42f015de70 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 -- 172.21.15.26:0/2199730308 >> 172.21.15.26:0/2199730308 conn(0x7f42f00658e0 msgr2=0x7f42f0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:06.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 -- 172.21.15.26:0/2199730308 shutdown_connections 2023-12-19T18:16:06.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 -- 172.21.15.26:0/2199730308 wait complete. 2023-12-19T18:16:06.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 Processor -- start 2023-12-19T18:16:06.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 -- start start 2023-12-19T18:16:06.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42f00b8340 0x7f42f00b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42f00cfbe0 0x7f42f0157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f015ccb0 0x7f42f014f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f00cdbe0 con 0x7f42f00cfbe0 2023-12-19T18:16:06.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f014e800 con 0x7f42f015ccb0 2023-12-19T18:16:06.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42f58d1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f00b6fa0 con 0x7f42f00b8340 2023-12-19T18:16:06.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42ee7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42f00cfbe0 0x7f42f0157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42ee7fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42f00cfbe0 0x7f42f0157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48240/0 (socket says 172.21.15.26:48240) 2023-12-19T18:16:06.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42ee7fc640 1 -- 172.21.15.26:0/1732721102 learned_addr learned my addr 172.21.15.26:0/1732721102 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:06.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42ef7fe640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f015ccb0 0x7f42f014f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.322+0000 7f42eeffd640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42f00b8340 0x7f42f00b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42ee7fc640 1 -- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42f00b8340 msgr2=0x7f42f00b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42ee7fc640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42f00b8340 0x7f42f00b7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42ee7fc640 1 -- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f015ccb0 msgr2=0x7f42f014f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42ee7fc640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f015ccb0 0x7f42f014f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42ee7fc640 1 -- 172.21.15.26:0/1732721102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e4009420 con 0x7f42f00cfbe0 2023-12-19T18:16:06.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42eeffd640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42f00b8340 0x7f42f00b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:06.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42ef7fe640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f015ccb0 0x7f42f014f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:06.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42ee7fc640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42f00cfbe0 0x7f42f0157610 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f42e4019090 tx=0x7f42e4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:06.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42d7fff640 1 -- 172.21.15.26:0/1732721102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e40bb070 con 0x7f42f00cfbe0 2023-12-19T18:16:06.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42d7fff640 1 -- 172.21.15.26:0/1732721102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e40b5410 con 0x7f42f00cfbe0 2023-12-19T18:16:06.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42f58d1640 1 -- 172.21.15.26:0/1732721102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f0157b50 con 0x7f42f00cfbe0 2023-12-19T18:16:06.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42d7fff640 1 -- 172.21.15.26:0/1732721102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e40b57c0 con 0x7f42f00cfbe0 2023-12-19T18:16:06.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42f58d1640 1 -- 172.21.15.26:0/1732721102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f00b6670 con 0x7f42f00cfbe0 2023-12-19T18:16:06.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42f58d1640 1 -- 172.21.15.26:0/1732721102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f004e850 con 0x7f42f00cfbe0 2023-12-19T18:16:06.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42d7fff640 1 -- 172.21.15.26:0/1732721102 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 114) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f42e40b3040 con 0x7f42f00cfbe0 2023-12-19T18:16:06.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42d7fff640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42dc065cd0 0x7f42dc068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42d7fff640 1 -- 172.21.15.26:0/1732721102 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f42e411d300 con 0x7f42f00cfbe0 2023-12-19T18:16:06.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42eeffd640 1 -- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42dc065cd0 msgr2=0x7f42dc068190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:06.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.326+0000 7f42eeffd640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42dc065cd0 0x7f42dc068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:16:06.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.330+0000 7f42d7fff640 1 -- 172.21.15.26:0/1732721102 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f42e40ea4f0 con 0x7f42f00cfbe0 2023-12-19T18:16:06.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.490+0000 7f42f58d1640 1 -- 172.21.15.26:0/1732721102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42f0153e30 con 0x7f42f00cfbe0 2023-12-19T18:16:06.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.494+0000 7f42d7fff640 1 -- 172.21.15.26:0/1732721102 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+17842 (secure 0 0 0) 0x7f42e40edb10 con 0x7f42f00cfbe0 2023-12-19T18:16:06.496 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:06.496 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":199,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:04.482106+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:36.168666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:37.527319+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":199,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/2345576187":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/1296996055":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/4184305047":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/2077682555":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/1610506837":"2023-12-20T18:14:50.374461+0000","172.21.15.160:6833/48394":"2023-12-20T18:14:50.374461+0000","172.21.15.160:6832/48394":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/3136437506":"2023-12-20T18:14:50.374461+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.494+0000 7f42f58d1640 1 -- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42dc065cd0 msgr2=0x7f42dc068190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:16:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.494+0000 7f42f58d1640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42dc065cd0 0x7f42dc068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.494+0000 7f42f58d1640 1 -- 172.21.15.26:0/1732721102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42f00cfbe0 msgr2=0x7f42f0157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.494+0000 7f42f58d1640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42f00cfbe0 0x7f42f0157610 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f42e4019090 tx=0x7f42e4019220 comp rx=0 tx=0).stop 2023-12-19T18:16:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.494+0000 7f42f58d1640 1 -- 172.21.15.26:0/1732721102 shutdown_connections 2023-12-19T18:16:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.494+0000 7f42f58d1640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42dc065cd0 0x7f42dc068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.494+0000 7f42f58d1640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f015ccb0 0x7f42f014f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.494+0000 7f42f58d1640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42f00cfbe0 0x7f42f0157610 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.494+0000 7f42f58d1640 1 --2- 172.21.15.26:0/1732721102 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42f00b8340 0x7f42f00b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.494+0000 7f42f58d1640 1 -- 172.21.15.26:0/1732721102 >> 172.21.15.26:0/1732721102 conn(0x7f42f00658e0 msgr2=0x7f42f00cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:06.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.494+0000 7f42f58d1640 1 -- 172.21.15.26:0/1732721102 shutdown_connections 2023-12-19T18:16:06.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.494+0000 7f42f58d1640 1 -- 172.21.15.26:0/1732721102 wait complete. 2023-12-19T18:16:06.519 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:36.168666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '195', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:37.527319+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '199', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 199, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:16:06.519 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:16:06.589 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:06.586+0000 7f183264b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:16:06.672 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:06.670+0000 7f183264b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:16:06.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.670+0000 7f08bb960640 1 Processor -- start 2023-12-19T18:16:06.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.670+0000 7f08bb960640 1 -- start start 2023-12-19T18:16:06.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08bb960640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08b40b8600 0x7f08b40b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08bb960640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f08b40b4b30 0x7f08b40d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08bb960640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b40b5bd0 con 0x7f08b415cde0 2023-12-19T18:16:06.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08bb960640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b415e380 con 0x7f08b40b4b30 2023-12-19T18:16:06.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08bb960640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b40b4fc0 con 0x7f08b40b8600 2023-12-19T18:16:06.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08b9efd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f08b40b4b30 0x7f08b40d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08b8efb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08b40b8600 0x7f08b40b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08b96fc640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f08b415cde0 0x7f08b415d1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:49408/0 (socket says 172.21.15.26:49408) 2023-12-19T18:16:06.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08b8efb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08b40b8600 0x7f08b40b45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:41742/0 (socket says 172.21.15.26:41742) 2023-12-19T18:16:06.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08b96fc640 1 -- 172.21.15.26:0/2322043819 learned_addr learned my addr 172.21.15.26:0/2322043819 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:06.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08b9efd640 1 --2- 172.21.15.26:0/2322043819 >> v2:172.21.15.160:3300/0 conn(0x7f08b40b4b30 0x7f08b40d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:42132/0 (socket says 172.21.15.26:42132) 2023-12-19T18:16:06.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08b8efb640 1 -- 172.21.15.26:0/2322043819 >> v2:172.21.15.160:3300/0 conn(0x7f08b40b4b30 msgr2=0x7f08b40d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:06.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08b8efb640 1 --2- 172.21.15.26:0/2322043819 >> v2:172.21.15.160:3300/0 conn(0x7f08b40b4b30 0x7f08b40d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08b8efb640 1 -- 172.21.15.26:0/2322043819 >> v1:172.21.15.160:6789/0 conn(0x7f08b415cde0 legacy=0x7f08b415d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08b8efb640 1 -- 172.21.15.26:0/2322043819 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b40d0ee0 con 0x7f08b40b8600 2023-12-19T18:16:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08b9efd640 1 --2- 172.21.15.26:0/2322043819 >> v2:172.21.15.160:3300/0 conn(0x7f08b40b4b30 0x7f08b40d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.674+0000 7f08b8efb640 1 --2- 172.21.15.26:0/2322043819 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08b40b8600 0x7f08b40b45f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f08a40097c0 tx=0x7f08a401ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=745c4fda5f8da443 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08abfff640 1 -- 172.21.15.26:0/2322043819 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40a8970 con 0x7f08b40b8600 2023-12-19T18:16:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08abfff640 1 -- 172.21.15.26:0/2322043819 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a40a8ad0 con 0x7f08b40b8600 2023-12-19T18:16:06.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08abfff640 1 -- 172.21.15.26:0/2322043819 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40a8e60 con 0x7f08b40b8600 2023-12-19T18:16:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 -- 172.21.15.26:0/2322043819 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08b40b8600 msgr2=0x7f08b40b45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 --2- 172.21.15.26:0/2322043819 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08b40b8600 0x7f08b40b45f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f08a40097c0 tx=0x7f08a401ec70 comp rx=0 tx=0).stop 2023-12-19T18:16:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 -- 172.21.15.26:0/2322043819 shutdown_connections 2023-12-19T18:16:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 --2- 172.21.15.26:0/2322043819 >> v2:172.21.15.160:3300/0 conn(0x7f08b40b4b30 0x7f08b40d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 --2- 172.21.15.26:0/2322043819 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08b40b8600 0x7f08b40b45f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 -- 172.21.15.26:0/2322043819 >> 172.21.15.26:0/2322043819 conn(0x7f08b4065ee0 msgr2=0x7f08b40d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 -- 172.21.15.26:0/2322043819 shutdown_connections 2023-12-19T18:16:06.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 -- 172.21.15.26:0/2322043819 wait complete. 2023-12-19T18:16:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 Processor -- start 2023-12-19T18:16:06.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 -- start start 2023-12-19T18:16:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08b40b4b30 0x7f08b414b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08b40b8600 0x7f08b414e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b415cde0 0x7f08b40b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b41f6a40 con 0x7f08b415cde0 2023-12-19T18:16:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b41f5b90 con 0x7f08b40b4b30 2023-12-19T18:16:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.678+0000 7f08bb960640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b414ea00 con 0x7f08b40b8600 2023-12-19T18:16:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08b9efd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b415cde0 0x7f08b40b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08b9efd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b415cde0 0x7f08b40b65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48256/0 (socket says 172.21.15.26:48256) 2023-12-19T18:16:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08b96fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08b40b4b30 0x7f08b414b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08b9efd640 1 -- 172.21.15.26:0/1443969262 learned_addr learned my addr 172.21.15.26:0/1443969262 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:06.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08b8efb640 1 --2- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08b40b8600 0x7f08b414e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08b9efd640 1 -- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08b40b8600 msgr2=0x7f08b414e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08b9efd640 1 --2- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08b40b8600 0x7f08b414e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08b9efd640 1 -- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08b40b4b30 msgr2=0x7f08b414b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08b9efd640 1 --2- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08b40b4b30 0x7f08b414b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08b9efd640 1 -- 172.21.15.26:0/1443969262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac009a70 con 0x7f08b415cde0 2023-12-19T18:16:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08b8efb640 1 --2- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08b40b8600 0x7f08b414e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08b9efd640 1 --2- 172.21.15.26:0/1443969262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b415cde0 0x7f08b40b65b0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00efc0 tx=0x7f08ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08a9ffb640 1 -- 172.21.15.26:0/1443969262 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac009cd0 con 0x7f08b415cde0 2023-12-19T18:16:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08a9ffb640 1 -- 172.21.15.26:0/1443969262 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac00f040 con 0x7f08b415cde0 2023-12-19T18:16:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08bb960640 1 -- 172.21.15.26:0/1443969262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a4009420 con 0x7f08b415cde0 2023-12-19T18:16:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08a9ffb640 1 -- 172.21.15.26:0/1443969262 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0039a0 con 0x7f08b415cde0 2023-12-19T18:16:06.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08bb960640 1 -- 172.21.15.26:0/1443969262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b41f6110 con 0x7f08b415cde0 2023-12-19T18:16:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08bb960640 1 -- 172.21.15.26:0/1443969262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b414ba70 con 0x7f08b415cde0 2023-12-19T18:16:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.682+0000 7f08a9ffb640 1 -- 172.21.15.26:0/1443969262 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 114) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f08ac014480 con 0x7f08b415cde0 2023-12-19T18:16:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.686+0000 7f08a9ffb640 1 --2- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0898065cd0 0x7f0898068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.686+0000 7f08a9ffb640 1 -- 172.21.15.26:0/1443969262 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f08ac0846f0 con 0x7f08b415cde0 2023-12-19T18:16:06.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.686+0000 7f08b96fc640 1 -- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0898065cd0 msgr2=0x7f0898068190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:06.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.686+0000 7f08b96fc640 1 --2- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0898065cd0 0x7f0898068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:16:06.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.690+0000 7f08a9ffb640 1 -- 172.21.15.26:0/1443969262 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f08ac0517d0 con 0x7f08b415cde0 2023-12-19T18:16:06.820 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:06.818+0000 7f183264b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:16:06.843 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:06.838+0000 7f5baf047180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:16:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.846+0000 7f08bb960640 1 -- 172.21.15.26:0/1443969262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08b4074270 con 0x7f08b415cde0 2023-12-19T18:16:06.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.846+0000 7f08a9ffb640 1 -- 172.21.15.26:0/1443969262 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v199) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08ac054df0 con 0x7f08b415cde0 2023-12-19T18:16:06.850 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:16:06.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.850+0000 7f08bb960640 1 -- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0898065cd0 msgr2=0x7f0898068190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:16:06.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.850+0000 7f08bb960640 1 --2- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0898065cd0 0x7f0898068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.850+0000 7f08bb960640 1 -- 172.21.15.26:0/1443969262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b415cde0 msgr2=0x7f08b40b65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.850+0000 7f08bb960640 1 --2- 172.21.15.26:0/1443969262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b415cde0 0x7f08b40b65b0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00efc0 tx=0x7f08ac00c490 comp rx=0 tx=0).stop 2023-12-19T18:16:06.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.850+0000 7f08bb960640 1 -- 172.21.15.26:0/1443969262 shutdown_connections 2023-12-19T18:16:06.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.850+0000 7f08bb960640 1 --2- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0898065cd0 0x7f0898068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.850+0000 7f08bb960640 1 --2- 172.21.15.26:0/1443969262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08b415cde0 0x7f08b40b65b0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.850+0000 7f08bb960640 1 --2- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08b40b8600 0x7f08b414e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.850+0000 7f08bb960640 1 --2- 172.21.15.26:0/1443969262 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08b40b4b30 0x7f08b414b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.850+0000 7f08bb960640 1 -- 172.21.15.26:0/1443969262 >> 172.21.15.26:0/1443969262 conn(0x7f08b4065ee0 msgr2=0x7f08b40d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:06.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.850+0000 7f08bb960640 1 -- 172.21.15.26:0/1443969262 shutdown_connections 2023-12-19T18:16:06.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:06.850+0000 7f08bb960640 1 -- 172.21.15.26:0/1443969262 wait complete. 2023-12-19T18:16:06.872 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:16:06.902 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:06.898+0000 7f183264b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:16:06.936 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:06.934+0000 7f5baf047180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:16:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.018+0000 7f2b0b1b3640 1 Processor -- start 2023-12-19T18:16:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.018+0000 7f2b0b1b3640 1 -- start start 2023-12-19T18:16:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.018+0000 7f2b0b1b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b0415ccf0 0x7f2b040d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.018+0000 7f2b0b1b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2b0415d0d0 0x7f2b040b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.018+0000 7f2b0b1b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b040b4600 0x7f2b040b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.018+0000 7f2b0b1b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b040b4f70 con 0x7f2b040b4600 2023-12-19T18:16:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.018+0000 7f2b0b1b3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b0415e200 con 0x7f2b0415d0d0 2023-12-19T18:16:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.018+0000 7f2b0b1b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b040d0aa0 con 0x7f2b0415ccf0 2023-12-19T18:16:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.018+0000 7f2b09750640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b040b4600 0x7f2b040b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.018+0000 7f2b09750640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b040b4600 0x7f2b040b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48270/0 (socket says 172.21.15.26:48270) 2023-12-19T18:16:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.018+0000 7f2b09750640 1 -- 172.21.15.26:0/2405023290 learned_addr learned my addr 172.21.15.26:0/2405023290 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.018+0000 7f2b08f4f640 1 --2- 172.21.15.26:0/2405023290 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b0415ccf0 0x7f2b040d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.018+0000 7f2b03fff640 1 --2- 172.21.15.26:0/2405023290 >> v2:172.21.15.160:3300/0 conn(0x7f2b0415d0d0 0x7f2b040b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.022+0000 7f2b09750640 1 -- 172.21.15.26:0/2405023290 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b0415ccf0 msgr2=0x7f2b040d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.022+0000 7f2b09750640 1 --2- 172.21.15.26:0/2405023290 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b0415ccf0 0x7f2b040d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.022+0000 7f2b09750640 1 -- 172.21.15.26:0/2405023290 >> v2:172.21.15.160:3300/0 conn(0x7f2b0415d0d0 msgr2=0x7f2b040b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.022+0000 7f2b09750640 1 --2- 172.21.15.26:0/2405023290 >> v2:172.21.15.160:3300/0 conn(0x7f2b0415d0d0 0x7f2b040b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.022+0000 7f2b09750640 1 -- 172.21.15.26:0/2405023290 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b040d1380 con 0x7f2b040b4600 2023-12-19T18:16:07.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.022+0000 7f2b09750640 1 --2- 172.21.15.26:0/2405023290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b040b4600 0x7f2b040b4a30 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f2b040d1570 tx=0x7f2afc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af10709884615658 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.022+0000 7f2b037fe640 1 -- 172.21.15.26:0/2405023290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc0a5730 con 0x7f2b040b4600 2023-12-19T18:16:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.022+0000 7f2b037fe640 1 -- 172.21.15.26:0/2405023290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2afc0a5890 con 0x7f2b040b4600 2023-12-19T18:16:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.022+0000 7f2b037fe640 1 -- 172.21.15.26:0/2405023290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc0a5c20 con 0x7f2b040b4600 2023-12-19T18:16:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.022+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/2405023290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b040b4600 msgr2=0x7f2b040b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.022+0000 7f2b0b1b3640 1 --2- 172.21.15.26:0/2405023290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b040b4600 0x7f2b040b4a30 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f2b040d1570 tx=0x7f2afc01c9a0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/2405023290 shutdown_connections 2023-12-19T18:16:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 --2- 172.21.15.26:0/2405023290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b040b4600 0x7f2b040b4a30 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 --2- 172.21.15.26:0/2405023290 >> v2:172.21.15.160:3300/0 conn(0x7f2b0415d0d0 0x7f2b040b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 --2- 172.21.15.26:0/2405023290 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b0415ccf0 0x7f2b040d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/2405023290 >> 172.21.15.26:0/2405023290 conn(0x7f2b04065ee0 msgr2=0x7f2b040cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:07.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/2405023290 shutdown_connections 2023-12-19T18:16:07.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/2405023290 wait complete. 2023-12-19T18:16:07.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 Processor -- start 2023-12-19T18:16:07.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 -- start start 2023-12-19T18:16:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b040b4600 0x7f2b04157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b0415ccf0 0x7f2b04156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b0415d0d0 0x7f2b040d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b04157a40 con 0x7f2b040b4600 2023-12-19T18:16:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b04159de0 con 0x7f2b0415d0d0 2023-12-19T18:16:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b0b1b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b041571a0 con 0x7f2b0415ccf0 2023-12-19T18:16:07.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b08f4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b040b4600 0x7f2b04157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b08f4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b040b4600 0x7f2b04157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48286/0 (socket says 172.21.15.26:48286) 2023-12-19T18:16:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b08f4f640 1 -- 172.21.15.26:0/3420788983 learned_addr learned my addr 172.21.15.26:0/3420788983 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b09750640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b0415d0d0 0x7f2b040d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b03fff640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b0415ccf0 0x7f2b04156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b08f4f640 1 -- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b0415ccf0 msgr2=0x7f2b04156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b08f4f640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b0415ccf0 0x7f2b04156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b08f4f640 1 -- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b0415d0d0 msgr2=0x7f2b040d1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b08f4f640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b0415d0d0 0x7f2b040d1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b08f4f640 1 -- 172.21.15.26:0/3420788983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc009b30 con 0x7f2b040b4600 2023-12-19T18:16:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.026+0000 7f2b09750640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b0415d0d0 0x7f2b040d1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.030+0000 7f2b08f4f640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b040b4600 0x7f2b04157cd0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f2af800e870 tx=0x7f2af800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.030+0000 7f2b017fa640 1 -- 172.21.15.26:0/3420788983 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af800ee60 con 0x7f2b040b4600 2023-12-19T18:16:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.030+0000 7f2b017fa640 1 -- 172.21.15.26:0/3420788983 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af800c940 con 0x7f2b040b4600 2023-12-19T18:16:07.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.030+0000 7f2b017fa640 1 -- 172.21.15.26:0/3420788983 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af800ccf0 con 0x7f2b040b4600 2023-12-19T18:16:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.030+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/3420788983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b04157480 con 0x7f2b040b4600 2023-12-19T18:16:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.030+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/3420788983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b04155060 con 0x7f2b040b4600 2023-12-19T18:16:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.030+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/3420788983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b04154ec0 con 0x7f2b040b4600 2023-12-19T18:16:07.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.030+0000 7f2b017fa640 1 -- 172.21.15.26:0/3420788983 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 114) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f2af8013470 con 0x7f2b040b4600 2023-12-19T18:16:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.030+0000 7f2b017fa640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2ae8065ca0 0x7f2ae8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.030+0000 7f2b017fa640 1 -- 172.21.15.26:0/3420788983 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6857+0+0 (secure 0 0 0) 0x7f2af8083aa0 con 0x7f2b040b4600 2023-12-19T18:16:07.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.030+0000 7f2b03fff640 1 -- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2ae8065ca0 msgr2=0x7f2ae8068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:07.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.030+0000 7f2b03fff640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2ae8065ca0 0x7f2ae8068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:16:07.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.034+0000 7f2b017fa640 1 -- 172.21.15.26:0/3420788983 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2b04154ec0 con 0x7f2b040b4600 2023-12-19T18:16:07.098 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:07.094+0000 7f5baf047180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:16:07.186 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:07.182+0000 7f5baf047180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:16:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.194+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/3420788983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2b04154ec0 con 0x7f2b040b4600 2023-12-19T18:16:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.194+0000 7f2b017fa640 1 -- 172.21.15.26:0/3420788983 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v199) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2b04154ec0 con 0x7f2b040b4600 2023-12-19T18:16:07.198 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:16:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.198+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2ae8065ca0 msgr2=0x7f2ae8068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:16:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.198+0000 7f2b0b1b3640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2ae8065ca0 0x7f2ae8068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.198+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/3420788983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b040b4600 msgr2=0x7f2b04157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.198+0000 7f2b0b1b3640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b040b4600 0x7f2b04157cd0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f2af800e870 tx=0x7f2af800ed30 comp rx=0 tx=0).stop 2023-12-19T18:16:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.198+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/3420788983 shutdown_connections 2023-12-19T18:16:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.198+0000 7f2b0b1b3640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2ae8065ca0 0x7f2ae8068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.198+0000 7f2b0b1b3640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b0415d0d0 0x7f2b040d1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.198+0000 7f2b0b1b3640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b0415ccf0 0x7f2b04156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.198+0000 7f2b0b1b3640 1 --2- 172.21.15.26:0/3420788983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b040b4600 0x7f2b04157cd0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.198+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/3420788983 >> 172.21.15.26:0/3420788983 conn(0x7f2b04065ee0 msgr2=0x7f2b0415eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.198+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/3420788983 shutdown_connections 2023-12-19T18:16:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.198+0000 7f2b0b1b3640 1 -- 172.21.15.26:0/3420788983 wait complete. 2023-12-19T18:16:07.214 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:07.210+0000 7f183264b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:16:07.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:16:07.273 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:07.270+0000 7f183264b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:16:07.348 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:07.346+0000 7f17ba19d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:07.348 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:07.346+0000 7f17ba19d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:07.348 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:07.346+0000 7f17ba19d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:07.348 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:07.346+0000 7f17ba19d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:07.349 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:07.346+0000 7f17ba19d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:07.349 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:07.346+0000 7f17bc9a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:07.349 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:07.346+0000 7f17bc9a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:07.349 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:07.346+0000 7f17bc9a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:07.349 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:07.346+0000 7f17bc9a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:07.349 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:07.346+0000 7f17bc9a2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:07.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f36f39640 1 Processor -- start 2023-12-19T18:16:07.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f36f39640 1 -- start start 2023-12-19T18:16:07.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f36f39640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7f30159820 0x7f7f3015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f36f39640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3006fb60 con 0x7f7f3006f6c0 2023-12-19T18:16:07.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f36f39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3006fe40 con 0x7f7f30159c00 2023-12-19T18:16:07.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f36f39640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f3015e470 con 0x7f7f30159820 2023-12-19T18:16:07.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f2ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7f3006f6c0 0x7f7f300c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33000/0 (socket says 172.21.15.26:33000) 2023-12-19T18:16:07.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f2ffff640 1 -- 172.21.15.26:0/2632402872 learned_addr learned my addr 172.21.15.26:0/2632402872 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:07.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f34cd5640 1 --2- 172.21.15.26:0/2632402872 >> v2:172.21.15.160:3300/0 conn(0x7f7f30159820 0x7f7f3015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f2f7fe640 1 -- 172.21.15.26:0/2632402872 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683466054 0 0) 0x7f7f3006fb60 con 0x7f7f3006f6c0 2023-12-19T18:16:07.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f2f7fe640 1 -- 172.21.15.26:0/2632402872 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f20003530 con 0x7f7f3006f6c0 2023-12-19T18:16:07.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f2f7fe640 1 -- 172.21.15.26:0/2632402872 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828195595 0 0) 0x7f7f3006fe40 con 0x7f7f30159c00 2023-12-19T18:16:07.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f34cd5640 1 -- 172.21.15.26:0/2632402872 >> v1:172.21.15.160:6789/0 conn(0x7f7f30159c00 legacy=0x7f7f30070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f34cd5640 1 -- 172.21.15.26:0/2632402872 >> v1:172.21.15.26:6789/0 conn(0x7f7f3006f6c0 legacy=0x7f7f300c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.374+0000 7f7f34cd5640 1 -- 172.21.15.26:0/2632402872 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f30070120 con 0x7f7f30159820 2023-12-19T18:16:07.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.378+0000 7f7f34cd5640 1 --2- 172.21.15.26:0/2632402872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f30159820 0x7f7f3015df30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7f240097c0 tx=0x7f7f2401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7959c8ad85a68f5f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.378+0000 7f7f2f7fe640 1 -- 172.21.15.26:0/2632402872 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f240ab650 con 0x7f7f30159820 2023-12-19T18:16:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.378+0000 7f7f2f7fe640 1 -- 172.21.15.26:0/2632402872 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f240ab7b0 con 0x7f7f30159820 2023-12-19T18:16:07.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.378+0000 7f7f2f7fe640 1 -- 172.21.15.26:0/2632402872 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f240abb40 con 0x7f7f30159820 2023-12-19T18:16:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.378+0000 7f7f36f39640 1 -- 172.21.15.26:0/2632402872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f30159820 msgr2=0x7f7f3015df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.378+0000 7f7f36f39640 1 --2- 172.21.15.26:0/2632402872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f30159820 0x7f7f3015df30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f7f240097c0 tx=0x7f7f2401c450 comp rx=0 tx=0).stop 2023-12-19T18:16:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.378+0000 7f7f36f39640 1 -- 172.21.15.26:0/2632402872 shutdown_connections 2023-12-19T18:16:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.378+0000 7f7f36f39640 1 --2- 172.21.15.26:0/2632402872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f30159820 0x7f7f3015df30 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.378+0000 7f7f36f39640 1 -- 172.21.15.26:0/2632402872 >> 172.21.15.26:0/2632402872 conn(0x7f7f300ce730 msgr2=0x7f7f300ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.378+0000 7f7f36f39640 1 -- 172.21.15.26:0/2632402872 shutdown_connections 2023-12-19T18:16:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.378+0000 7f7f36f39640 1 -- 172.21.15.26:0/2632402872 wait complete. 2023-12-19T18:16:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f36f39640 1 Processor -- start 2023-12-19T18:16:07.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f36f39640 1 -- start start 2023-12-19T18:16:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f36f39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f3006f6c0 0x7f7f300c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f36f39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f30159820 0x7f7f30149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f36f39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f30159c00 0x7f7f300cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f36f39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f301f50c0 con 0x7f7f3006f6c0 2023-12-19T18:16:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f36f39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f301f5360 con 0x7f7f30159820 2023-12-19T18:16:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f36f39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f301f5640 con 0x7f7f30159c00 2023-12-19T18:16:07.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f34cd5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f3006f6c0 0x7f7f300c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f34cd5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f3006f6c0 0x7f7f300c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48292/0 (socket says 172.21.15.26:48292) 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f34cd5640 1 -- 172.21.15.26:0/2099970240 learned_addr learned my addr 172.21.15.26:0/2099970240 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f2ffff640 1 --2- 172.21.15.26:0/2099970240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f30159820 0x7f7f30149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f354d6640 1 --2- 172.21.15.26:0/2099970240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f30159c00 0x7f7f300cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f34cd5640 1 -- 172.21.15.26:0/2099970240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f30159c00 msgr2=0x7f7f300cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f34cd5640 1 --2- 172.21.15.26:0/2099970240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f30159c00 0x7f7f300cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f34cd5640 1 -- 172.21.15.26:0/2099970240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f30159820 msgr2=0x7f7f30149290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f34cd5640 1 --2- 172.21.15.26:0/2099970240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f30159820 0x7f7f30149290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f34cd5640 1 -- 172.21.15.26:0/2099970240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f24009420 con 0x7f7f3006f6c0 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f2ffff640 1 --2- 172.21.15.26:0/2099970240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f30159820 0x7f7f30149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f34cd5640 1 --2- 172.21.15.26:0/2099970240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f3006f6c0 0x7f7f300c49a0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f7f240ad040 tx=0x7f7f2401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f2d7fa640 1 -- 172.21.15.26:0/2099970240 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f24004220 con 0x7f7f3006f6c0 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f2d7fa640 1 -- 172.21.15.26:0/2099970240 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f24004380 con 0x7f7f3006f6c0 2023-12-19T18:16:07.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f2d7fa640 1 -- 172.21.15.26:0/2099970240 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f24003840 con 0x7f7f3006f6c0 2023-12-19T18:16:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f36f39640 1 -- 172.21.15.26:0/2099970240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3006b880 con 0x7f7f3006f6c0 2023-12-19T18:16:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.382+0000 7f7f36f39640 1 -- 172.21.15.26:0/2099970240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f301f5c00 con 0x7f7f3006f6c0 2023-12-19T18:16:07.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.386+0000 7f7f36f39640 1 -- 172.21.15.26:0/2099970240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f30074270 con 0x7f7f3006f6c0 2023-12-19T18:16:07.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.386+0000 7f7f2d7fa640 1 -- 172.21.15.26:0/2099970240 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 115) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f7f240039a0 con 0x7f7f3006f6c0 2023-12-19T18:16:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.386+0000 7f7f2d7fa640 1 -- 172.21.15.26:0/2099970240 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 7157+0+0 (secure 0 0 0) 0x7f7f2411d610 con 0x7f7f3006f6c0 2023-12-19T18:16:07.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.390+0000 7f7f2d7fa640 1 -- 172.21.15.26:0/2099970240 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7f240eae30 con 0x7f7f3006f6c0 2023-12-19T18:16:07.544 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:07.542+0000 7f5baf047180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:16:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.554+0000 7f7f36f39640 1 -- 172.21.15.26:0/2099970240 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f300b46b0 con 0x7f7f3006f6c0 2023-12-19T18:16:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.554+0000 7f7f2d7fa640 1 -- 172.21.15.26:0/2099970240 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v200) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f240044f0 con 0x7f7f3006f6c0 2023-12-19T18:16:07.555 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:16:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.554+0000 7f7f36f39640 1 -- 172.21.15.26:0/2099970240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f3006f6c0 msgr2=0x7f7f300c49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.554+0000 7f7f36f39640 1 --2- 172.21.15.26:0/2099970240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f3006f6c0 0x7f7f300c49a0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f7f240ad040 tx=0x7f7f2401ac90 comp rx=0 tx=0).stop 2023-12-19T18:16:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.554+0000 7f7f36f39640 1 -- 172.21.15.26:0/2099970240 shutdown_connections 2023-12-19T18:16:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.554+0000 7f7f36f39640 1 --2- 172.21.15.26:0/2099970240 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f30159c00 0x7f7f300cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.554+0000 7f7f36f39640 1 --2- 172.21.15.26:0/2099970240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f30159820 0x7f7f30149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.554+0000 7f7f36f39640 1 --2- 172.21.15.26:0/2099970240 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f3006f6c0 0x7f7f300c49a0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.554+0000 7f7f36f39640 1 -- 172.21.15.26:0/2099970240 >> 172.21.15.26:0/2099970240 conn(0x7f7f300ce730 msgr2=0x7f7f300ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:07.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.554+0000 7f7f36f39640 1 -- 172.21.15.26:0/2099970240 shutdown_connections 2023-12-19T18:16:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.554+0000 7f7f36f39640 1 -- 172.21.15.26:0/2099970240 wait complete. 2023-12-19T18:16:07.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:07.609 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:07.606+0000 7f5baf047180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:16:07.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5e24d8640 1 Processor -- start 2023-12-19T18:16:07.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5e24d8640 1 -- start start 2023-12-19T18:16:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5e24d8640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc5dc154220 0x7fc5dc15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5e24d8640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5dc06b3d0 0x7fc5dc0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5e24d8640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5dc06bc30 con 0x7fc5dc0cec40 2023-12-19T18:16:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5e24d8640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5dc0cf0d0 con 0x7fc5dc154220 2023-12-19T18:16:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5e24d8640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc0cc6e0 con 0x7fc5dc06b3d0 2023-12-19T18:16:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5e0a75640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc5dc0cec40 0x7fc5dc15d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33010/0 (socket says 172.21.15.26:33010) 2023-12-19T18:16:07.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5e0a75640 1 -- 172.21.15.26:0/2703600796 learned_addr learned my addr 172.21.15.26:0/2703600796 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5dbfff640 1 --2- 172.21.15.26:0/2703600796 >> v2:172.21.15.160:3300/0 conn(0x7fc5dc154220 0x7fc5dc15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5db7fe640 1 --2- 172.21.15.26:0/2703600796 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5dc06b3d0 0x7fc5dc0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5daffd640 1 -- 172.21.15.26:0/2703600796 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014706597 0 0) 0x7fc5dc06bc30 con 0x7fc5dc0cec40 2023-12-19T18:16:07.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5daffd640 1 -- 172.21.15.26:0/2703600796 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5c4003530 con 0x7fc5dc0cec40 2023-12-19T18:16:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5dbfff640 1 -- 172.21.15.26:0/2703600796 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5dc06b3d0 msgr2=0x7fc5dc0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5dbfff640 1 --2- 172.21.15.26:0/2703600796 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5dc06b3d0 0x7fc5dc0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5dbfff640 1 -- 172.21.15.26:0/2703600796 >> v1:172.21.15.26:6789/0 conn(0x7fc5dc0cec40 legacy=0x7fc5dc15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5dbfff640 1 -- 172.21.15.26:0/2703600796 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc15afc0 con 0x7fc5dc154220 2023-12-19T18:16:07.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.722+0000 7fc5db7fe640 1 --2- 172.21.15.26:0/2703600796 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5dc06b3d0 0x7fc5dc0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:07.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.726+0000 7fc5dbfff640 1 --2- 172.21.15.26:0/2703600796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5dc154220 0x7fc5dc15aa80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0097c0 tx=0x7fc5cc01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dceb21bd2e3f2c60 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.726+0000 7fc5daffd640 1 -- 172.21.15.26:0/2703600796 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc0ab600 con 0x7fc5dc154220 2023-12-19T18:16:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.726+0000 7fc5daffd640 1 -- 172.21.15.26:0/2703600796 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5cc0a7050 con 0x7fc5dc154220 2023-12-19T18:16:07.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.726+0000 7fc5daffd640 1 -- 172.21.15.26:0/2703600796 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc0ab980 con 0x7fc5dc154220 2023-12-19T18:16:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.726+0000 7fc5e24d8640 1 -- 172.21.15.26:0/2703600796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5dc154220 msgr2=0x7fc5dc15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.726+0000 7fc5e24d8640 1 --2- 172.21.15.26:0/2703600796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5dc154220 0x7fc5dc15aa80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0097c0 tx=0x7fc5cc01c450 comp rx=0 tx=0).stop 2023-12-19T18:16:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.726+0000 7fc5e24d8640 1 -- 172.21.15.26:0/2703600796 shutdown_connections 2023-12-19T18:16:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.726+0000 7fc5e24d8640 1 --2- 172.21.15.26:0/2703600796 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5dc06b3d0 0x7fc5dc0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.726+0000 7fc5e24d8640 1 --2- 172.21.15.26:0/2703600796 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5dc154220 0x7fc5dc15aa80 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.726+0000 7fc5e24d8640 1 -- 172.21.15.26:0/2703600796 >> 172.21.15.26:0/2703600796 conn(0x7fc5dc0658e0 msgr2=0x7fc5dc065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.726+0000 7fc5e24d8640 1 -- 172.21.15.26:0/2703600796 shutdown_connections 2023-12-19T18:16:07.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.726+0000 7fc5e24d8640 1 -- 172.21.15.26:0/2703600796 wait complete. 2023-12-19T18:16:07.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e24d8640 1 Processor -- start 2023-12-19T18:16:07.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e24d8640 1 -- start start 2023-12-19T18:16:07.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e24d8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5dc06b3d0 0x7fc5dc14dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e24d8640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5dc0cec40 0x7fc5dc159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e24d8640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5dc154220 0x7fc5dc0b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e24d8640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc14e690 con 0x7fc5dc154220 2023-12-19T18:16:07.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e24d8640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc154600 con 0x7fc5dc06b3d0 2023-12-19T18:16:07.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e24d8640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc1548e0 con 0x7fc5dc0cec40 2023-12-19T18:16:07.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e0a75640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5dc154220 0x7fc5dc0b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e0a75640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5dc154220 0x7fc5dc0b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48294/0 (socket says 172.21.15.26:48294) 2023-12-19T18:16:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e0a75640 1 -- 172.21.15.26:0/1625762882 learned_addr learned my addr 172.21.15.26:0/1625762882 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5db7fe640 1 --2- 172.21.15.26:0/1625762882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5dc0cec40 0x7fc5dc159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5dbfff640 1 --2- 172.21.15.26:0/1625762882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5dc06b3d0 0x7fc5dc14dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e0a75640 1 -- 172.21.15.26:0/1625762882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5dc0cec40 msgr2=0x7fc5dc159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e0a75640 1 --2- 172.21.15.26:0/1625762882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5dc0cec40 0x7fc5dc159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e0a75640 1 -- 172.21.15.26:0/1625762882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5dc06b3d0 msgr2=0x7fc5dc14dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e0a75640 1 --2- 172.21.15.26:0/1625762882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5dc06b3d0 0x7fc5dc14dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e0a75640 1 -- 172.21.15.26:0/1625762882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d0009420 con 0x7fc5dc154220 2023-12-19T18:16:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e0a75640 1 --2- 172.21.15.26:0/1625762882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5dc154220 0x7fc5dc0b0630 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400caa0 tx=0x7fc5d400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5d8ff9640 1 -- 172.21.15.26:0/1625762882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d400ac00 con 0x7fc5dc154220 2023-12-19T18:16:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5d8ff9640 1 -- 172.21.15.26:0/1625762882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d400ad60 con 0x7fc5dc154220 2023-12-19T18:16:07.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e24d8640 1 -- 172.21.15.26:0/1625762882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc009420 con 0x7fc5dc154220 2023-12-19T18:16:07.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5d8ff9640 1 -- 172.21.15.26:0/1625762882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4012620 con 0x7fc5dc154220 2023-12-19T18:16:07.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e24d8640 1 -- 172.21.15.26:0/1625762882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5dc154ea0 con 0x7fc5dc154220 2023-12-19T18:16:07.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.730+0000 7fc5e24d8640 1 -- 172.21.15.26:0/1625762882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5dc0ccbf0 con 0x7fc5dc154220 2023-12-19T18:16:07.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.734+0000 7fc5d8ff9640 1 -- 172.21.15.26:0/1625762882 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 115) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fc5d4007070 con 0x7fc5dc154220 2023-12-19T18:16:07.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.734+0000 7fc5d8ff9640 1 -- 172.21.15.26:0/1625762882 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 7157+0+0 (secure 0 0 0) 0x7fc5d40831c0 con 0x7fc5dc154220 2023-12-19T18:16:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.738+0000 7fc5d8ff9640 1 -- 172.21.15.26:0/1625762882 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc5dc0ccbf0 con 0x7fc5dc154220 2023-12-19T18:16:07.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.930+0000 7fc5e24d8640 1 -- 172.21.15.26:0/1625762882 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5dc0ccbf0 con 0x7fc5dc154220 2023-12-19T18:16:07.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.930+0000 7fc5d8ff9640 1 -- 172.21.15.26:0/1625762882 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4723 (secure 0 0 0) 0x7fc5dc0ccbf0 con 0x7fc5dc154220 2023-12-19T18:16:07.933 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:07.934 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3922467901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3922467901},{"type":"v1","addr":"172.21.15.160:6835","nonce":3922467901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":9562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3692512090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3692512090},{"type":"v1","addr":"172.21.15.160:6837","nonce":3692512090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":10083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3961033710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3961033710},{"type":"v1","addr":"172.21.15.26:6833","nonce":3961033710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:38.527840+0000","modified":"2023-12-19T18:14:41.539799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10086":{"gid":10086,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1537720341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1537720341},{"type":"v1","addr":"172.21.15.26:6835","nonce":1537720341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:16:07.934 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 87 2023-12-19T18:16:07.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.934+0000 7fc5e24d8640 1 -- 172.21.15.26:0/1625762882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5dc154220 msgr2=0x7fc5dc0b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.934+0000 7fc5e24d8640 1 --2- 172.21.15.26:0/1625762882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5dc154220 0x7fc5dc0b0630 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400caa0 tx=0x7fc5d400cf60 comp rx=0 tx=0).stop 2023-12-19T18:16:07.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.934+0000 7fc5e24d8640 1 -- 172.21.15.26:0/1625762882 shutdown_connections 2023-12-19T18:16:07.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.934+0000 7fc5e24d8640 1 --2- 172.21.15.26:0/1625762882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5dc154220 0x7fc5dc0b0630 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.934+0000 7fc5e24d8640 1 --2- 172.21.15.26:0/1625762882 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc5dc0cec40 0x7fc5dc159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.934+0000 7fc5e24d8640 1 --2- 172.21.15.26:0/1625762882 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5dc06b3d0 0x7fc5dc14dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.934+0000 7fc5e24d8640 1 -- 172.21.15.26:0/1625762882 >> 172.21.15.26:0/1625762882 conn(0x7fc5dc0658e0 msgr2=0x7fc5dc067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:07.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.934+0000 7fc5e24d8640 1 -- 172.21.15.26:0/1625762882 shutdown_connections 2023-12-19T18:16:07.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:07.934+0000 7fc5e24d8640 1 -- 172.21.15.26:0/1625762882 wait complete. 2023-12-19T18:16:07.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:08.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.142+0000 7f86426fc640 1 Processor -- start 2023-12-19T18:16:08.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.142+0000 7f86426fc640 1 -- start start 2023-12-19T18:16:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.146+0000 7f86426fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f863c0ce030 0x7f863c167c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.146+0000 7f86426fc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863c0d0d40 con 0x7f863c15cda0 2023-12-19T18:16:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.146+0000 7f86426fc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863c161150 con 0x7f863c0b85e0 2023-12-19T18:16:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.146+0000 7f86426fc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863c161430 con 0x7f863c0ce030 2023-12-19T18:16:08.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.146+0000 7f863bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f863c15cda0 0x7f863c160920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33016/0 (socket says 172.21.15.26:33016) 2023-12-19T18:16:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.146+0000 7f863bfff640 1 -- 172.21.15.26:0/3128187505 learned_addr learned my addr 172.21.15.26:0/3128187505 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.146+0000 7f8640c99640 1 --2- 172.21.15.26:0/3128187505 >> v2:172.21.15.26:3300/0 conn(0x7f863c0ce030 0x7f863c167c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.146+0000 7f8640c99640 1 -- 172.21.15.26:0/3128187505 >> v1:172.21.15.160:6789/0 conn(0x7f863c0b85e0 legacy=0x7f863c0cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.146+0000 7f8640c99640 1 -- 172.21.15.26:0/3128187505 >> v1:172.21.15.26:6789/0 conn(0x7f863c15cda0 legacy=0x7f863c160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.146+0000 7f8640c99640 1 -- 172.21.15.26:0/3128187505 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c161710 con 0x7f863c0ce030 2023-12-19T18:16:08.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.146+0000 7f8640c99640 1 --2- 172.21.15.26:0/3128187505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863c0ce030 0x7f863c167c50 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f8634009d90 tx=0x7f863401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4ece0838374c37 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f863affd640 1 -- 172.21.15.26:0/3128187505 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340a5970 con 0x7f863c0ce030 2023-12-19T18:16:08.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f863affd640 1 -- 172.21.15.26:0/3128187505 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86340a5ad0 con 0x7f863c0ce030 2023-12-19T18:16:08.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f863affd640 1 -- 172.21.15.26:0/3128187505 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340a5e60 con 0x7f863c0ce030 2023-12-19T18:16:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 -- 172.21.15.26:0/3128187505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863c0ce030 msgr2=0x7f863c167c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 --2- 172.21.15.26:0/3128187505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863c0ce030 0x7f863c167c50 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f8634009d90 tx=0x7f863401c920 comp rx=0 tx=0).stop 2023-12-19T18:16:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 -- 172.21.15.26:0/3128187505 shutdown_connections 2023-12-19T18:16:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 --2- 172.21.15.26:0/3128187505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863c0ce030 0x7f863c167c50 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 -- 172.21.15.26:0/3128187505 >> 172.21.15.26:0/3128187505 conn(0x7f863c0658e0 msgr2=0x7f863c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 -- 172.21.15.26:0/3128187505 shutdown_connections 2023-12-19T18:16:08.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 -- 172.21.15.26:0/3128187505 wait complete. 2023-12-19T18:16:08.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 Processor -- start 2023-12-19T18:16:08.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 -- start start 2023-12-19T18:16:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f863c0b85e0 0x7f863c149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f863c0ce030 0x7f863c0c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863c15cda0 0x7f863c15ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c14df90 con 0x7f863c15cda0 2023-12-19T18:16:08.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c14cae0 con 0x7f863c0ce030 2023-12-19T18:16:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f86426fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863c14e1e0 con 0x7f863c0b85e0 2023-12-19T18:16:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f8640c99640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863c15cda0 0x7f863c15ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f8640c99640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863c15cda0 0x7f863c15ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48306/0 (socket says 172.21.15.26:48306) 2023-12-19T18:16:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.150+0000 7f8640c99640 1 -- 172.21.15.26:0/3247675954 learned_addr learned my addr 172.21.15.26:0/3247675954 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f863b7fe640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f863c0ce030 0x7f863c0c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f863bfff640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f863c0b85e0 0x7f863c149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f8640c99640 1 -- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f863c0b85e0 msgr2=0x7f863c149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f8640c99640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f863c0b85e0 0x7f863c149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f8640c99640 1 -- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f863c0ce030 msgr2=0x7f863c0c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f8640c99640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f863c0ce030 0x7f863c0c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f8640c99640 1 -- 172.21.15.26:0/3247675954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8634009a70 con 0x7f863c15cda0 2023-12-19T18:16:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f863b7fe640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f863c0ce030 0x7f863c0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:08.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f8640c99640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863c15cda0 0x7f863c15ec70 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f86340b0040 tx=0x7f86340196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f8638ff9640 1 -- 172.21.15.26:0/3247675954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340ade40 con 0x7f863c15cda0 2023-12-19T18:16:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f8638ff9640 1 -- 172.21.15.26:0/3247675954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86340b9450 con 0x7f863c15cda0 2023-12-19T18:16:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f8638ff9640 1 -- 172.21.15.26:0/3247675954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340b9800 con 0x7f863c15cda0 2023-12-19T18:16:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f86426fc640 1 -- 172.21.15.26:0/3247675954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863c14e4c0 con 0x7f863c15cda0 2023-12-19T18:16:08.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f86426fc640 1 -- 172.21.15.26:0/3247675954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863c1477c0 con 0x7f863c15cda0 2023-12-19T18:16:08.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f86426fc640 1 -- 172.21.15.26:0/3247675954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863c074270 con 0x7f863c15cda0 2023-12-19T18:16:08.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f8638ff9640 1 -- 172.21.15.26:0/3247675954 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 115) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f86340b9960 con 0x7f863c15cda0 2023-12-19T18:16:08.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.154+0000 7f8638ff9640 1 -- 172.21.15.26:0/3247675954 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 7157+0+0 (secure 0 0 0) 0x7f863411eab0 con 0x7f863c15cda0 2023-12-19T18:16:08.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.158+0000 7f8638ff9640 1 -- 172.21.15.26:0/3247675954 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f86340ec3d0 con 0x7f863c15cda0 2023-12-19T18:16:08.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.286+0000 7f8638ff9640 1 -- 172.21.15.26:0/3247675954 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 116) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f86340ebe80 con 0x7f863c15cda0 2023-12-19T18:16:08.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.286+0000 7f8638ff9640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f862806bb10 0x7f862806df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.286+0000 7f863bfff640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f862806bb10 0x7f862806df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.290+0000 7f863bfff640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f862806bb10 0x7f862806df00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f862c005890 tx=0x7f862c009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.322+0000 7f86426fc640 1 -- 172.21.15.26:0/3247675954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f863c14e970 con 0x7f863c15cda0 2023-12-19T18:16:08.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.322+0000 7f8638ff9640 1 -- 172.21.15.26:0/3247675954 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+18279 (secure 0 0 0) 0x7f86340c6690 con 0x7f863c15cda0 2023-12-19T18:16:08.326 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:08.326 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":200,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:07.277746+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:36.168666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:37.527319+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":199,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/2530870731":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/3220458370":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/1056342855":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/3344208914":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/4114252288":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/2345576187":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/1441498583":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/1296996055":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/4184305047":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/2077682555":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/2269135450":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/1610506837":"2023-12-20T18:14:50.374461+0000","172.21.15.160:6833/48394":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/3503481093":"2023-12-20T18:16:07.277686+0000","172.21.15.160:6832/48394":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/3136437506":"2023-12-20T18:14:50.374461+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:08.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.326+0000 7f86426fc640 1 -- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f862806bb10 msgr2=0x7f862806df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.326+0000 7f86426fc640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f862806bb10 0x7f862806df00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f862c005890 tx=0x7f862c009040 comp rx=0 tx=0).stop 2023-12-19T18:16:08.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.326+0000 7f86426fc640 1 -- 172.21.15.26:0/3247675954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863c15cda0 msgr2=0x7f863c15ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.326+0000 7f86426fc640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863c15cda0 0x7f863c15ec70 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f86340b0040 tx=0x7f86340196c0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.326+0000 7f86426fc640 1 -- 172.21.15.26:0/3247675954 shutdown_connections 2023-12-19T18:16:08.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.326+0000 7f86426fc640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f862806bb10 0x7f862806df00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.326+0000 7f86426fc640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f863c15cda0 0x7f863c15ec70 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.326+0000 7f86426fc640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f863c0ce030 0x7f863c0c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.326+0000 7f86426fc640 1 --2- 172.21.15.26:0/3247675954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f863c0b85e0 0x7f863c149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.326+0000 7f86426fc640 1 -- 172.21.15.26:0/3247675954 >> 172.21.15.26:0/3247675954 conn(0x7f863c0658e0 msgr2=0x7f863c15b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:08.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.326+0000 7f86426fc640 1 -- 172.21.15.26:0/3247675954 shutdown_connections 2023-12-19T18:16:08.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.326+0000 7f86426fc640 1 -- 172.21.15.26:0/3247675954 wait complete. 2023-12-19T18:16:08.348 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:16:08.348 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:08.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53c39b3640 1 Processor -- start 2023-12-19T18:16:08.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53c39b3640 1 -- start start 2023-12-19T18:16:08.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53c39b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53bc0ce040 0x7f53bc0d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53c39b3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53bc0d29d0 con 0x7f53bc0d3130 2023-12-19T18:16:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53c39b3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53bc0d3570 con 0x7f53bc15cd40 2023-12-19T18:16:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53c39b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc15d680 con 0x7f53bc0ce040 2023-12-19T18:16:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53c1f50640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f53bc0d3130 0x7f53bc0d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33032/0 (socket says 172.21.15.26:33032) 2023-12-19T18:16:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53c1f50640 1 -- 172.21.15.26:0/50409552 learned_addr learned my addr 172.21.15.26:0/50409552 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53c0f4e640 1 --2- 172.21.15.26:0/50409552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53bc0ce040 0x7f53bc0d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53b3fff640 1 -- 172.21.15.26:0/50409552 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326065099 0 0) 0x7f53b40014a0 con 0x7f53bc0d3130 2023-12-19T18:16:08.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53b3fff640 1 -- 172.21.15.26:0/50409552 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53a8003530 con 0x7f53bc0d3130 2023-12-19T18:16:08.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53c0f4e640 1 -- 172.21.15.26:0/50409552 >> v1:172.21.15.160:6789/0 conn(0x7f53bc15cd40 legacy=0x7f53bc160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53b3fff640 1 -- 172.21.15.26:0/50409552 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838245989 0 0) 0x7f53bc0d3570 con 0x7f53bc15cd40 2023-12-19T18:16:08.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53c0f4e640 1 -- 172.21.15.26:0/50409552 >> v1:172.21.15.26:6789/0 conn(0x7f53bc0d3130 legacy=0x7f53bc0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.490+0000 7f53c0f4e640 1 -- 172.21.15.26:0/50409552 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc161750 con 0x7f53bc0ce040 2023-12-19T18:16:08.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.494+0000 7f53c0f4e640 1 --2- 172.21.15.26:0/50409552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53bc0ce040 0x7f53bc0d2bf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0097f0 tx=0x7f53ac01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=949048eec709c605 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.494+0000 7f53b3fff640 1 -- 172.21.15.26:0/50409552 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0a6040 con 0x7f53bc0ce040 2023-12-19T18:16:08.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.494+0000 7f53b3fff640 1 -- 172.21.15.26:0/50409552 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ac0aa600 con 0x7f53bc0ce040 2023-12-19T18:16:08.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.494+0000 7f53b3fff640 1 -- 172.21.15.26:0/50409552 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0aa990 con 0x7f53bc0ce040 2023-12-19T18:16:08.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.494+0000 7f53c39b3640 1 -- 172.21.15.26:0/50409552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53bc0ce040 msgr2=0x7f53bc0d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.494+0000 7f53c39b3640 1 --2- 172.21.15.26:0/50409552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53bc0ce040 0x7f53bc0d2bf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0097f0 tx=0x7f53ac01c4a0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.494+0000 7f53c39b3640 1 -- 172.21.15.26:0/50409552 shutdown_connections 2023-12-19T18:16:08.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.494+0000 7f53c39b3640 1 --2- 172.21.15.26:0/50409552 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53bc0ce040 0x7f53bc0d2bf0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.494+0000 7f53c39b3640 1 -- 172.21.15.26:0/50409552 >> 172.21.15.26:0/50409552 conn(0x7f53bc06a120 msgr2=0x7f53bc06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:08.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.494+0000 7f53c39b3640 1 -- 172.21.15.26:0/50409552 shutdown_connections 2023-12-19T18:16:08.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.494+0000 7f53c39b3640 1 -- 172.21.15.26:0/50409552 wait complete. 2023-12-19T18:16:08.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c39b3640 1 Processor -- start 2023-12-19T18:16:08.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c39b3640 1 -- start start 2023-12-19T18:16:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c39b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53bc0ce040 0x7f53bc153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c39b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53bc0d3130 0x7f53bc15c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c39b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53bc15cd40 0x7f53bc155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c39b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc15b210 con 0x7f53bc0ce040 2023-12-19T18:16:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c39b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc15b460 con 0x7f53bc0d3130 2023-12-19T18:16:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c39b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc15b740 con 0x7f53bc15cd40 2023-12-19T18:16:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c174f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53bc0ce040 0x7f53bc153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c174f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53bc0ce040 0x7f53bc153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48308/0 (socket says 172.21.15.26:48308) 2023-12-19T18:16:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c174f640 1 -- 172.21.15.26:0/1701153019 learned_addr learned my addr 172.21.15.26:0/1701153019 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c0f4e640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53bc0d3130 0x7f53bc15c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c1f50640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53bc15cd40 0x7f53bc155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c174f640 1 -- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53bc15cd40 msgr2=0x7f53bc155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c174f640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53bc15cd40 0x7f53bc155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c174f640 1 -- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53bc0d3130 msgr2=0x7f53bc15c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c174f640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53bc0d3130 0x7f53bc15c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c174f640 1 -- 172.21.15.26:0/1701153019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac009420 con 0x7f53bc0ce040 2023-12-19T18:16:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.498+0000 7f53c174f640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53bc0ce040 0x7f53bc153dc0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f53b800c3c0 tx=0x7f53b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.502+0000 7f53b1ffb640 1 -- 172.21.15.26:0/1701153019 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b80054e0 con 0x7f53bc0ce040 2023-12-19T18:16:08.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.502+0000 7f53b1ffb640 1 -- 172.21.15.26:0/1701153019 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b8003650 con 0x7f53bc0ce040 2023-12-19T18:16:08.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.502+0000 7f53b1ffb640 1 -- 172.21.15.26:0/1701153019 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8003a00 con 0x7f53bc0ce040 2023-12-19T18:16:08.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.502+0000 7f53c39b3640 1 -- 172.21.15.26:0/1701153019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53bc15ba20 con 0x7f53bc0ce040 2023-12-19T18:16:08.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.502+0000 7f53c39b3640 1 -- 172.21.15.26:0/1701153019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc14c5f0 con 0x7f53bc0ce040 2023-12-19T18:16:08.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.502+0000 7f53c39b3640 1 -- 172.21.15.26:0/1701153019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53bc074270 con 0x7f53bc0ce040 2023-12-19T18:16:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.506+0000 7f53b1ffb640 1 -- 172.21.15.26:0/1701153019 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 116) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f53b8003b60 con 0x7f53bc0ce040 2023-12-19T18:16:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.506+0000 7f53b1ffb640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f53a00658b0 0x7f53a0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.506+0000 7f53b1ffb640 1 -- 172.21.15.26:0/1701153019 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 7157+0+0 (secure 0 0 0) 0x7f53b8082820 con 0x7f53bc0ce040 2023-12-19T18:16:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.506+0000 7f53c0f4e640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f53a00658b0 0x7f53a0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.506+0000 7f53c0f4e640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f53a00658b0 0x7f53a0067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0ae000 tx=0x7f53ac0b5040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.510+0000 7f53b1ffb640 1 -- 172.21.15.26:0/1701153019 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f53b8050020 con 0x7f53bc0ce040 2023-12-19T18:16:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.702+0000 7f53c39b3640 1 -- 172.21.15.26:0/1701153019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53bc0d2c90 con 0x7f53bc0ce040 2023-12-19T18:16:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.702+0000 7f53b1ffb640 1 -- 172.21.15.26:0/1701153019 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4723 (secure 0 0 0) 0x7f53b8053640 con 0x7f53bc0ce040 2023-12-19T18:16:08.707 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:08.707 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3922467901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3922467901},{"type":"v1","addr":"172.21.15.160:6835","nonce":3922467901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":9562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3692512090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3692512090},{"type":"v1","addr":"172.21.15.160:6837","nonce":3692512090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":10083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3961033710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3961033710},{"type":"v1","addr":"172.21.15.26:6833","nonce":3961033710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:38.527840+0000","modified":"2023-12-19T18:14:41.539799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10086":{"gid":10086,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1537720341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1537720341},{"type":"v1","addr":"172.21.15.26:6835","nonce":1537720341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:16:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 87 2023-12-19T18:16:08.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.706+0000 7f53c39b3640 1 -- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f53a00658b0 msgr2=0x7f53a0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.706+0000 7f53c39b3640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f53a00658b0 0x7f53a0067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0ae000 tx=0x7f53ac0b5040 comp rx=0 tx=0).stop 2023-12-19T18:16:08.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.706+0000 7f53c39b3640 1 -- 172.21.15.26:0/1701153019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53bc0ce040 msgr2=0x7f53bc153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.706+0000 7f53c39b3640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53bc0ce040 0x7f53bc153dc0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f53b800c3c0 tx=0x7f53b800c880 comp rx=0 tx=0).stop 2023-12-19T18:16:08.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.706+0000 7f53c39b3640 1 -- 172.21.15.26:0/1701153019 shutdown_connections 2023-12-19T18:16:08.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.706+0000 7f53c39b3640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f53a00658b0 0x7f53a0067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.706+0000 7f53c39b3640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53bc15cd40 0x7f53bc155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.706+0000 7f53c39b3640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53bc0d3130 0x7f53bc15c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.706+0000 7f53c39b3640 1 --2- 172.21.15.26:0/1701153019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53bc0ce040 0x7f53bc153dc0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.706+0000 7f53c39b3640 1 -- 172.21.15.26:0/1701153019 >> 172.21.15.26:0/1701153019 conn(0x7f53bc06a120 msgr2=0x7f53bc15fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:08.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.706+0000 7f53c39b3640 1 -- 172.21.15.26:0/1701153019 shutdown_connections 2023-12-19T18:16:08.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.706+0000 7f53c39b3640 1 -- 172.21.15.26:0/1701153019 wait complete. 2023-12-19T18:16:08.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:08.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7653ba6640 1 Processor -- start 2023-12-19T18:16:08.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7653ba6640 1 -- start start 2023-12-19T18:16:08.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7653ba6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f764c15cf60 0x7f764c0d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7653ba6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764c0d2960 con 0x7f764c0d1420 2023-12-19T18:16:08.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7653ba6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764c0d18c0 con 0x7f764c0b4d10 2023-12-19T18:16:08.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7653ba6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764c0d1ba0 con 0x7f764c15cf60 2023-12-19T18:16:08.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7651141640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f764c0d1420 0x7f764c0b4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33040/0 (socket says 172.21.15.26:33040) 2023-12-19T18:16:08.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7651141640 1 -- 172.21.15.26:0/2317326349 learned_addr learned my addr 172.21.15.26:0/2317326349 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:08.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7651942640 1 --2- 172.21.15.26:0/2317326349 >> v2:172.21.15.160:3300/0 conn(0x7f764c15cf60 0x7f764c0d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7650940640 1 -- 172.21.15.26:0/2317326349 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89713470 0 0) 0x7f764c0d2960 con 0x7f764c0d1420 2023-12-19T18:16:08.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7650940640 1 -- 172.21.15.26:0/2317326349 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7638003530 con 0x7f764c0d1420 2023-12-19T18:16:08.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7650940640 1 -- 172.21.15.26:0/2317326349 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280387573 0 0) 0x7f764c0d18c0 con 0x7f764c0b4d10 2023-12-19T18:16:08.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7650940640 1 -- 172.21.15.26:0/2317326349 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f764c0d2960 con 0x7f764c0b4d10 2023-12-19T18:16:08.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7651942640 1 -- 172.21.15.26:0/2317326349 >> v1:172.21.15.160:6789/0 conn(0x7f764c0b4d10 legacy=0x7f764c0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7651942640 1 -- 172.21.15.26:0/2317326349 >> v1:172.21.15.26:6789/0 conn(0x7f764c0d1420 legacy=0x7f764c0b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.870+0000 7f7651942640 1 -- 172.21.15.26:0/2317326349 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c0d1e80 con 0x7f764c15cf60 2023-12-19T18:16:08.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.874+0000 7f7651942640 1 --2- 172.21.15.26:0/2317326349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f764c15cf60 0x7f764c0d0ee0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f76480097c0 tx=0x7f764801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3943993624f22d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.874+0000 7f7650940640 1 -- 172.21.15.26:0/2317326349 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480a8970 con 0x7f764c15cf60 2023-12-19T18:16:08.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.874+0000 7f7650940640 1 -- 172.21.15.26:0/2317326349 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76480a8ad0 con 0x7f764c15cf60 2023-12-19T18:16:08.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.874+0000 7f7650940640 1 -- 172.21.15.26:0/2317326349 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480a8e40 con 0x7f764c15cf60 2023-12-19T18:16:08.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.874+0000 7f7653ba6640 1 -- 172.21.15.26:0/2317326349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f764c15cf60 msgr2=0x7f764c0d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.874+0000 7f7653ba6640 1 --2- 172.21.15.26:0/2317326349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f764c15cf60 0x7f764c0d0ee0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f76480097c0 tx=0x7f764801ec70 comp rx=0 tx=0).stop 2023-12-19T18:16:08.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.874+0000 7f7653ba6640 1 -- 172.21.15.26:0/2317326349 shutdown_connections 2023-12-19T18:16:08.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.874+0000 7f7653ba6640 1 --2- 172.21.15.26:0/2317326349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f764c15cf60 0x7f764c0d0ee0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.874+0000 7f7653ba6640 1 -- 172.21.15.26:0/2317326349 >> 172.21.15.26:0/2317326349 conn(0x7f764c065ee0 msgr2=0x7f764c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:08.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.874+0000 7f7653ba6640 1 -- 172.21.15.26:0/2317326349 shutdown_connections 2023-12-19T18:16:08.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.874+0000 7f7653ba6640 1 -- 172.21.15.26:0/2317326349 wait complete. 2023-12-19T18:16:08.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7653ba6640 1 Processor -- start 2023-12-19T18:16:08.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7653ba6640 1 -- start start 2023-12-19T18:16:08.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7653ba6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f764c0b4d10 0x7f764c15d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7653ba6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f764c0d1420 0x7f764c15d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7653ba6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c15cf60 0x7f764c0cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7653ba6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c15e2f0 con 0x7f764c15cf60 2023-12-19T18:16:08.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7653ba6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0c79b0 con 0x7f764c0d1420 2023-12-19T18:16:08.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7653ba6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0c7c20 con 0x7f764c0b4d10 2023-12-19T18:16:08.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7652143640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c15cf60 0x7f764c0cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7652143640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c15cf60 0x7f764c0cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48312/0 (socket says 172.21.15.26:48312) 2023-12-19T18:16:08.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7652143640 1 -- 172.21.15.26:0/1161480704 learned_addr learned my addr 172.21.15.26:0/1161480704 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:08.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7651141640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f764c0d1420 0x7f764c15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7652143640 1 -- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f764c0b4d10 msgr2=0x7f764c15d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7652143640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f764c0b4d10 0x7f764c15d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7652143640 1 -- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f764c0d1420 msgr2=0x7f764c15d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7652143640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f764c0d1420 0x7f764c15d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7652143640 1 -- 172.21.15.26:0/1161480704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7648009420 con 0x7f764c15cf60 2023-12-19T18:16:08.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7651141640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f764c0d1420 0x7f764c15d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:08.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7652143640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c15cf60 0x7f764c0cb550 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f764400c7c0 tx=0x7f764400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f76427fc640 1 -- 172.21.15.26:0/1161480704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644003b80 con 0x7f764c15cf60 2023-12-19T18:16:08.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f76427fc640 1 -- 172.21.15.26:0/1161480704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7644003ce0 con 0x7f764c15cf60 2023-12-19T18:16:08.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f76427fc640 1 -- 172.21.15.26:0/1161480704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400aa50 con 0x7f764c15cf60 2023-12-19T18:16:08.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7653ba6640 1 -- 172.21.15.26:0/1161480704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c0c7f00 con 0x7f764c15cf60 2023-12-19T18:16:08.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.878+0000 7f7653ba6640 1 -- 172.21.15.26:0/1161480704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c0cc290 con 0x7f764c15cf60 2023-12-19T18:16:08.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.882+0000 7f7653ba6640 1 -- 172.21.15.26:0/1161480704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c0c8420 con 0x7f764c15cf60 2023-12-19T18:16:08.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.882+0000 7f76427fc640 1 -- 172.21.15.26:0/1161480704 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 116) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f764400abb0 con 0x7f764c15cf60 2023-12-19T18:16:08.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.882+0000 7f76427fc640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f76300658b0 0x7f7630067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.882+0000 7f76427fc640 1 -- 172.21.15.26:0/1161480704 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 7157+0+0 (secure 0 0 0) 0x7f7644083300 con 0x7f764c15cf60 2023-12-19T18:16:08.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.882+0000 7f7651942640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f76300658b0 0x7f7630067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.882+0000 7f7651942640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f76300658b0 0x7f7630067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f76480b2000 tx=0x7f7648004740 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:08.886+0000 7f76427fc640 1 -- 172.21.15.26:0/1161480704 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f764c0c8420 con 0x7f764c15cf60 2023-12-19T18:16:09.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.078+0000 7f7653ba6640 1 -- 172.21.15.26:0/1161480704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f764c0c8420 con 0x7f764c15cf60 2023-12-19T18:16:09.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.078+0000 7f76427fc640 1 -- 172.21.15.26:0/1161480704 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4723 (secure 0 0 0) 0x7f764c0c8420 con 0x7f764c15cf60 2023-12-19T18:16:09.081 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:09.081 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3922467901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3922467901},{"type":"v1","addr":"172.21.15.160:6835","nonce":3922467901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":9562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3692512090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3692512090},{"type":"v1","addr":"172.21.15.160:6837","nonce":3692512090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":10083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3961033710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3961033710},{"type":"v1","addr":"172.21.15.26:6833","nonce":3961033710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:38.527840+0000","modified":"2023-12-19T18:14:41.539799+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10086},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10086":{"gid":10086,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1537720341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1537720341},{"type":"v1","addr":"172.21.15.26:6835","nonce":1537720341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:16:09.082 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 87 2023-12-19T18:16:09.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.082+0000 7f7653ba6640 1 -- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f76300658b0 msgr2=0x7f7630067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.082+0000 7f7653ba6640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f76300658b0 0x7f7630067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f76480b2000 tx=0x7f7648004740 comp rx=0 tx=0).stop 2023-12-19T18:16:09.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.082+0000 7f7653ba6640 1 -- 172.21.15.26:0/1161480704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c15cf60 msgr2=0x7f764c0cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.082+0000 7f7653ba6640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c15cf60 0x7f764c0cb550 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f764400c7c0 tx=0x7f764400cc80 comp rx=0 tx=0).stop 2023-12-19T18:16:09.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.082+0000 7f7653ba6640 1 -- 172.21.15.26:0/1161480704 shutdown_connections 2023-12-19T18:16:09.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.082+0000 7f7653ba6640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f76300658b0 0x7f7630067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.082+0000 7f7653ba6640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f764c15cf60 0x7f764c0cb550 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.082+0000 7f7653ba6640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f764c0d1420 0x7f764c15d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.082+0000 7f7653ba6640 1 --2- 172.21.15.26:0/1161480704 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f764c0b4d10 0x7f764c15d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.082+0000 7f7653ba6640 1 -- 172.21.15.26:0/1161480704 >> 172.21.15.26:0/1161480704 conn(0x7f764c065ee0 msgr2=0x7f764c0b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:09.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.082+0000 7f7653ba6640 1 -- 172.21.15.26:0/1161480704 shutdown_connections 2023-12-19T18:16:09.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.082+0000 7f7653ba6640 1 -- 172.21.15.26:0/1161480704 wait complete. 2023-12-19T18:16:09.102 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:09.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.242+0000 7f2b3e3ce640 1 Processor -- start 2023-12-19T18:16:09.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.242+0000 7f2b3e3ce640 1 -- start start 2023-12-19T18:16:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.242+0000 7f2b3e3ce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2b38154220 0x7f2b3815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.242+0000 7f2b3e3ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b380cee10 0x7f2b3815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.242+0000 7f2b3e3ce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b3806bc30 con 0x7f2b3806b3d0 2023-12-19T18:16:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.242+0000 7f2b3e3ce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b380cc6e0 con 0x7f2b38154220 2023-12-19T18:16:09.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.242+0000 7f2b3e3ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3815afc0 con 0x7f2b380cee10 2023-12-19T18:16:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.246+0000 7f2b37fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2b38154220 0x7f2b3815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.246+0000 7f2b37fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2b38154220 0x7f2b3815aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:42226/0 (socket says 172.21.15.26:42226) 2023-12-19T18:16:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.246+0000 7f2b3c96b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b380cee10 0x7f2b3815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.246+0000 7f2b37fff640 1 -- 172.21.15.26:0/1547710323 learned_addr learned my addr 172.21.15.26:0/1547710323 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.246+0000 7f2b377fe640 1 --1- 172.21.15.26:0/1547710323 >> v1:172.21.15.160:6789/0 conn(0x7f2b3806b3d0 0x7f2b380ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:49450/0 (socket says 172.21.15.26:49450) 2023-12-19T18:16:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.246+0000 7f2b37fff640 1 -- 172.21.15.26:0/1547710323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b380cee10 msgr2=0x7f2b3815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.246+0000 7f2b37fff640 1 --2- 172.21.15.26:0/1547710323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b380cee10 0x7f2b3815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.246+0000 7f2b37fff640 1 -- 172.21.15.26:0/1547710323 >> v1:172.21.15.160:6789/0 conn(0x7f2b3806b3d0 legacy=0x7f2b380ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.246+0000 7f2b37fff640 1 -- 172.21.15.26:0/1547710323 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3806b950 con 0x7f2b38154220 2023-12-19T18:16:09.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.246+0000 7f2b3c96b640 1 --2- 172.21.15.26:0/1547710323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b380cee10 0x7f2b3815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b37fff640 1 --2- 172.21.15.26:0/1547710323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b38154220 0x7f2b3815aa80 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009870 tx=0x7f2b2801c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=557f5bd8e4f23 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b36ffd640 1 -- 172.21.15.26:0/1547710323 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b280ab600 con 0x7f2b38154220 2023-12-19T18:16:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b36ffd640 1 -- 172.21.15.26:0/1547710323 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b280a7050 con 0x7f2b38154220 2023-12-19T18:16:09.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b36ffd640 1 -- 172.21.15.26:0/1547710323 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b280ab980 con 0x7f2b38154220 2023-12-19T18:16:09.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1547710323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b38154220 msgr2=0x7f2b3815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b3e3ce640 1 --2- 172.21.15.26:0/1547710323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b38154220 0x7f2b3815aa80 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009870 tx=0x7f2b2801c3d0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1547710323 shutdown_connections 2023-12-19T18:16:09.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b3e3ce640 1 --2- 172.21.15.26:0/1547710323 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b380cee10 0x7f2b3815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b3e3ce640 1 --2- 172.21.15.26:0/1547710323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b38154220 0x7f2b3815aa80 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1547710323 >> 172.21.15.26:0/1547710323 conn(0x7f2b380658e0 msgr2=0x7f2b38065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:09.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1547710323 shutdown_connections 2023-12-19T18:16:09.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1547710323 wait complete. 2023-12-19T18:16:09.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b3e3ce640 1 Processor -- start 2023-12-19T18:16:09.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.250+0000 7f2b3e3ce640 1 -- start start 2023-12-19T18:16:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b3e3ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b3806b3d0 0x7f2b38142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b3e3ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b380cee10 0x7f2b381e3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b3e3ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b38154220 0x7f2b380b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b3e3ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b38142a90 con 0x7f2b380cee10 2023-12-19T18:16:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b3e3ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b381547c0 con 0x7f2b3806b3d0 2023-12-19T18:16:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b3e3ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b38154aa0 con 0x7f2b38154220 2023-12-19T18:16:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b380cee10 0x7f2b381e3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b380cee10 0x7f2b381e3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48324/0 (socket says 172.21.15.26:48324) 2023-12-19T18:16:09.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b377fe640 1 -- 172.21.15.26:0/1543913448 learned_addr learned my addr 172.21.15.26:0/1543913448 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:09.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b37fff640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b3806b3d0 0x7f2b38142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b3c96b640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b38154220 0x7f2b380b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b377fe640 1 -- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b38154220 msgr2=0x7f2b380b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b377fe640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b38154220 0x7f2b380b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b377fe640 1 -- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b3806b3d0 msgr2=0x7f2b38142550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b377fe640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b3806b3d0 0x7f2b38142550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b377fe640 1 -- 172.21.15.26:0/1543913448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b30009a70 con 0x7f2b380cee10 2023-12-19T18:16:09.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b377fe640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b380cee10 0x7f2b381e3430 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009e30 tx=0x7f2b2c00c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b34ff9640 1 -- 172.21.15.26:0/1543913448 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c007830 con 0x7f2b380cee10 2023-12-19T18:16:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b34ff9640 1 -- 172.21.15.26:0/1543913448 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c007990 con 0x7f2b380cee10 2023-12-19T18:16:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b34ff9640 1 -- 172.21.15.26:0/1543913448 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c007d60 con 0x7f2b380cee10 2023-12-19T18:16:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1543913448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b280094d0 con 0x7f2b380cee10 2023-12-19T18:16:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1543913448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b38155060 con 0x7f2b380cee10 2023-12-19T18:16:09.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1543913448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b38074270 con 0x7f2b380cee10 2023-12-19T18:16:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.254+0000 7f2b34ff9640 1 -- 172.21.15.26:0/1543913448 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 116) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f2b2c009070 con 0x7f2b380cee10 2023-12-19T18:16:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.258+0000 7f2b34ff9640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2b240658b0 0x7f2b24067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.258+0000 7f2b34ff9640 1 -- 172.21.15.26:0/1543913448 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 7157+0+0 (secure 0 0 0) 0x7f2b2c085350 con 0x7f2b380cee10 2023-12-19T18:16:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.258+0000 7f2b37fff640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2b240658b0 0x7f2b24067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.258+0000 7f2b37fff640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2b240658b0 0x7f2b24067d70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2b280ad000 tx=0x7f2b280a6000 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.262+0000 7f2b34ff9640 1 -- 172.21.15.26:0/1543913448 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2b2c0529d0 con 0x7f2b380cee10 2023-12-19T18:16:09.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1543913448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b381579b0 con 0x7f2b380cee10 2023-12-19T18:16:09.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b34ff9640 1 -- 172.21.15.26:0/1543913448 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v200) v1 ==== 74+0+18279 (secure 0 0 0) 0x7f2b2c019020 con 0x7f2b380cee10 2023-12-19T18:16:09.428 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:09.428 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":200,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:07.277746+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:36.168666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:37.527319+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":199,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/2530870731":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/3220458370":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/1056342855":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/3344208914":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/4114252288":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/2345576187":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/1441498583":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/1296996055":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/4184305047":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/2077682555":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/2269135450":"2023-12-20T18:16:07.277686+0000","172.21.15.160:0/1610506837":"2023-12-20T18:14:50.374461+0000","172.21.15.160:6833/48394":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/3503481093":"2023-12-20T18:16:07.277686+0000","172.21.15.160:6832/48394":"2023-12-20T18:14:50.374461+0000","172.21.15.160:0/3136437506":"2023-12-20T18:14:50.374461+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2b240658b0 msgr2=0x7f2b24067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b3e3ce640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2b240658b0 0x7f2b24067d70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2b280ad000 tx=0x7f2b280a6000 comp rx=0 tx=0).stop 2023-12-19T18:16:09.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1543913448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b380cee10 msgr2=0x7f2b381e3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b3e3ce640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b380cee10 0x7f2b381e3430 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009e30 tx=0x7f2b2c00c430 comp rx=0 tx=0).stop 2023-12-19T18:16:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1543913448 shutdown_connections 2023-12-19T18:16:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b3e3ce640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2b240658b0 0x7f2b24067d70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b3e3ce640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b38154220 0x7f2b380b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b3e3ce640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b380cee10 0x7f2b381e3430 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b3e3ce640 1 --2- 172.21.15.26:0/1543913448 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b3806b3d0 0x7f2b38142550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1543913448 >> 172.21.15.26:0/1543913448 conn(0x7f2b380658e0 msgr2=0x7f2b380af280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1543913448 shutdown_connections 2023-12-19T18:16:09.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.426+0000 7f2b3e3ce640 1 -- 172.21.15.26:0/1543913448 wait complete. 2023-12-19T18:16:09.450 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:16:09.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.590+0000 7f9efc36d640 1 Processor -- start 2023-12-19T18:16:09.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.590+0000 7f9efc36d640 1 -- start start 2023-12-19T18:16:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9efc36d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9ef415cd40 0x7f9ef40ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9efc36d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4160900 0x7f9ef4160d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9efc36d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef40d29d0 con 0x7f9ef40d2bf0 2023-12-19T18:16:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9efc36d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef4161280 con 0x7f9ef415cd40 2023-12-19T18:16:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9efc36d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef415d680 con 0x7f9ef4160900 2023-12-19T18:16:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9efa90a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9ef40d2bf0 0x7f9ef40d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33056/0 (socket says 172.21.15.26:33056) 2023-12-19T18:16:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9efa90a640 1 -- 172.21.15.26:0/117807368 learned_addr learned my addr 172.21.15.26:0/117807368 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9908640 1 --2- 172.21.15.26:0/117807368 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4160900 0x7f9ef4160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9efa109640 1 --2- 172.21.15.26:0/117807368 >> v2:172.21.15.160:3300/0 conn(0x7f9ef415cd40 0x7f9ef40ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010700714 0 0) 0x7f9eec0014a0 con 0x7f9ef40d2bf0 2023-12-19T18:16:09.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ee0003530 con 0x7f9ef40d2bf0 2023-12-19T18:16:09.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2958312856 0 0) 0x7f9ee0003530 con 0x7f9ef40d2bf0 2023-12-19T18:16:09.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4160900 msgr2=0x7f9ef4160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:09.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 --2- 172.21.15.26:0/117807368 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4160900 0x7f9ef4160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 >> v2:172.21.15.160:3300/0 conn(0x7f9ef415cd40 msgr2=0x7f9ef40ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:09.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 --2- 172.21.15.26:0/117807368 >> v2:172.21.15.160:3300/0 conn(0x7f9ef415cd40 0x7f9ef40ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef4161750 con 0x7f9ef40d2bf0 2023-12-19T18:16:09.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f9eec001b90 con 0x7f9ef40d2bf0 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9efa109640 1 --2- 172.21.15.26:0/117807368 >> v2:172.21.15.160:3300/0 conn(0x7f9ef415cd40 0x7f9ef40ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9908640 1 --2- 172.21.15.26:0/117807368 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4160900 0x7f9ef4160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 >> v1:172.21.15.26:6789/0 conn(0x7f9ef40d2bf0 legacy=0x7f9ef40d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 --2- 172.21.15.26:0/117807368 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4160900 0x7f9ee0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 --2- 172.21.15.26:0/117807368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0003ec0 0x7f9ee00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 --2- 172.21.15.26:0/117807368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ee0004d10 0x7f9ee0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef415d680 con 0x7f9ee0003ec0 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee000b990 con 0x7f9ee0004d10 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee000bad0 con 0x7f9ef4160900 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9908640 1 --2- 172.21.15.26:0/117807368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0003ec0 0x7f9ee00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9efa109640 1 --2- 172.21.15.26:0/117807368 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4160900 0x7f9ee0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9efa90a640 1 --2- 172.21.15.26:0/117807368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ee0004d10 0x7f9ee0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9908640 1 -- 172.21.15.26:0/117807368 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4160900 msgr2=0x7f9ee0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:09.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9908640 1 --2- 172.21.15.26:0/117807368 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4160900 0x7f9ee0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9908640 1 -- 172.21.15.26:0/117807368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ee0004d10 msgr2=0x7f9ee0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9908640 1 --2- 172.21.15.26:0/117807368 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ee0004d10 0x7f9ee0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.594+0000 7f9ef9908640 1 -- 172.21.15.26:0/117807368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee4009420 con 0x7f9ee0003ec0 2023-12-19T18:16:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.598+0000 7f9ef9908640 1 --2- 172.21.15.26:0/117807368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0003ec0 0x7f9ee00047d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009cb0 tx=0x7f9ee401cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0b9b6d4be6c43b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.598+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee40a57a0 con 0x7f9ee0003ec0 2023-12-19T18:16:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.598+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee40a5900 con 0x7f9ee0003ec0 2023-12-19T18:16:09.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.598+0000 7f9ef9107640 1 -- 172.21.15.26:0/117807368 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee40a5cf0 con 0x7f9ee0003ec0 2023-12-19T18:16:09.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.598+0000 7f9efc36d640 1 -- 172.21.15.26:0/117807368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0003ec0 msgr2=0x7f9ee00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.598+0000 7f9efc36d640 1 --2- 172.21.15.26:0/117807368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0003ec0 0x7f9ee00047d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4009cb0 tx=0x7f9ee401cbf0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.598+0000 7f9efa90a640 1 -- 172.21.15.26:0/117807368 reap_dead start 2023-12-19T18:16:09.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.598+0000 7f9efc36d640 1 -- 172.21.15.26:0/117807368 shutdown_connections 2023-12-19T18:16:09.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.598+0000 7f9efc36d640 1 -- 172.21.15.26:0/117807368 >> 172.21.15.26:0/117807368 conn(0x7f9ef406a120 msgr2=0x7f9ef406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:09.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.598+0000 7f9efc36d640 1 -- 172.21.15.26:0/117807368 shutdown_connections 2023-12-19T18:16:09.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.598+0000 7f9efc36d640 1 -- 172.21.15.26:0/117807368 wait complete. 2023-12-19T18:16:09.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efc36d640 1 Processor -- start 2023-12-19T18:16:09.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efc36d640 1 -- start start 2023-12-19T18:16:09.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efc36d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0003ec0 0x7f9ef41f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efc36d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4154c40 0x7f9ef4154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efc36d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ef41f1d60 0x7f9ef40b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efc36d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef41550c0 con 0x7f9ee0003ec0 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efc36d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef4155300 con 0x7f9ef41f1d60 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efc36d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef41555e0 con 0x7f9ef4154c40 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efa109640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0003ec0 0x7f9ef41f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efa109640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0003ec0 0x7f9ef41f6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48338/0 (socket says 172.21.15.26:48338) 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efa109640 1 -- 172.21.15.26:0/1500093846 learned_addr learned my addr 172.21.15.26:0/1500093846 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9ef9908640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4154c40 0x7f9ef4154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efa90a640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ef41f1d60 0x7f9ef40b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efa109640 1 -- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4154c40 msgr2=0x7f9ef4154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efa109640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4154c40 0x7f9ef4154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efa109640 1 -- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ef41f1d60 msgr2=0x7f9ef40b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efa109640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ef41f1d60 0x7f9ef40b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efa109640 1 -- 172.21.15.26:0/1500093846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef41f5a60 con 0x7f9ee0003ec0 2023-12-19T18:16:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efa90a640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ef41f1d60 0x7f9ef40b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efa109640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0003ec0 0x7f9ef41f6540 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0011cd0 tx=0x7f9ef000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9eeaffd640 1 -- 172.21.15.26:0/1500093846 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef000fd20 con 0x7f9ee0003ec0 2023-12-19T18:16:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9eeaffd640 1 -- 172.21.15.26:0/1500093846 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef000fea0 con 0x7f9ee0003ec0 2023-12-19T18:16:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efc36d640 1 -- 172.21.15.26:0/1500093846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef41f5d40 con 0x7f9ee0003ec0 2023-12-19T18:16:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9eeaffd640 1 -- 172.21.15.26:0/1500093846 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0004260 con 0x7f9ee0003ec0 2023-12-19T18:16:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efc36d640 1 -- 172.21.15.26:0/1500093846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef41f21e0 con 0x7f9ee0003ec0 2023-12-19T18:16:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.602+0000 7f9efc36d640 1 -- 172.21.15.26:0/1500093846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef4074270 con 0x7f9ee0003ec0 2023-12-19T18:16:09.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.606+0000 7f9eeaffd640 1 -- 172.21.15.26:0/1500093846 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 116) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f9ef00043c0 con 0x7f9ee0003ec0 2023-12-19T18:16:09.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.606+0000 7f9eeaffd640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9ed80658b0 0x7f9ed8067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.606+0000 7f9eeaffd640 1 -- 172.21.15.26:0/1500093846 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 7157+0+0 (secure 0 0 0) 0x7f9ef0083fb0 con 0x7f9ee0003ec0 2023-12-19T18:16:09.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.606+0000 7f9ef9908640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9ed80658b0 0x7f9ed8067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.606+0000 7f9ef9908640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9ed80658b0 0x7f9ed8067d70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40af000 tx=0x7f9ee40046d0 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.610+0000 7f9eeaffd640 1 -- 172.21.15.26:0/1500093846 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9ef0051630 con 0x7f9ee0003ec0 2023-12-19T18:16:09.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.802+0000 7f9efc36d640 1 -- 172.21.15.26:0/1500093846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9ef415c8a0 con 0x7f9ee0003ec0 2023-12-19T18:16:09.804 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T18:16:09.802+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:16:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.810+0000 7f9eeaffd640 1 -- 172.21.15.26:0/1500093846 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9ef0059e60 con 0x7f9ee0003ec0 2023-12-19T18:16:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:16:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.814+0000 7f9efc36d640 1 -- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9ed80658b0 msgr2=0x7f9ed8067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.814+0000 7f9efc36d640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9ed80658b0 0x7f9ed8067d70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40af000 tx=0x7f9ee40046d0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.814+0000 7f9efc36d640 1 -- 172.21.15.26:0/1500093846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0003ec0 msgr2=0x7f9ef41f6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.814+0000 7f9efc36d640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0003ec0 0x7f9ef41f6540 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0011cd0 tx=0x7f9ef000f610 comp rx=0 tx=0).stop 2023-12-19T18:16:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.814+0000 7f9efc36d640 1 -- 172.21.15.26:0/1500093846 shutdown_connections 2023-12-19T18:16:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.814+0000 7f9efc36d640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9ed80658b0 0x7f9ed8067d70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.814+0000 7f9efc36d640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ef41f1d60 0x7f9ef40b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.814+0000 7f9efc36d640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9ef4154c40 0x7f9ef4154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.814+0000 7f9efc36d640 1 --2- 172.21.15.26:0/1500093846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ee0003ec0 0x7f9ef41f6540 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.814+0000 7f9efc36d640 1 -- 172.21.15.26:0/1500093846 >> 172.21.15.26:0/1500093846 conn(0x7f9ef406a120 msgr2=0x7f9ef40d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.814+0000 7f9efc36d640 1 -- 172.21.15.26:0/1500093846 shutdown_connections 2023-12-19T18:16:09.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.814+0000 7f9efc36d640 1 -- 172.21.15.26:0/1500093846 wait complete. 2023-12-19T18:16:09.837 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:16:09.882 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T18:16:09.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f533928c640 1 Processor -- start 2023-12-19T18:16:09.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f533928c640 1 -- start start 2023-12-19T18:16:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f533928c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53340b8940 0x7f53340d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f533928c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53340d21d0 con 0x7f53340d1120 2023-12-19T18:16:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f533928c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53340d15c0 con 0x7f53340b4600 2023-12-19T18:16:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f533928c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53340d18a0 con 0x7f53340b8940 2023-12-19T18:16:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f53337fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f53340d1120 0x7f53340b5c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33078/0 (socket says 172.21.15.26:33078) 2023-12-19T18:16:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f53337fe640 1 -- 172.21.15.26:0/2190010924 learned_addr learned my addr 172.21.15.26:0/2190010924 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:09.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f532bfff640 1 --2- 172.21.15.26:0/2190010924 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53340b8940 0x7f53340d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f53327fc640 1 -- 172.21.15.26:0/2190010924 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800472254 0 0) 0x7f53340d21d0 con 0x7f53340d1120 2023-12-19T18:16:09.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f53327fc640 1 -- 172.21.15.26:0/2190010924 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5320003530 con 0x7f53340d1120 2023-12-19T18:16:09.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f53327fc640 1 -- 172.21.15.26:0/2190010924 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378338563 0 0) 0x7f53340d15c0 con 0x7f53340b4600 2023-12-19T18:16:09.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f53327fc640 1 -- 172.21.15.26:0/2190010924 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53340d21d0 con 0x7f53340b4600 2023-12-19T18:16:09.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f532bfff640 1 -- 172.21.15.26:0/2190010924 >> v1:172.21.15.160:6789/0 conn(0x7f53340b4600 legacy=0x7f53340b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f532bfff640 1 -- 172.21.15.26:0/2190010924 >> v1:172.21.15.26:6789/0 conn(0x7f53340d1120 legacy=0x7f53340b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.982+0000 7f532bfff640 1 -- 172.21.15.26:0/2190010924 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53340d1b80 con 0x7f53340b8940 2023-12-19T18:16:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.986+0000 7f532bfff640 1 --2- 172.21.15.26:0/2190010924 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53340b8940 0x7f53340d0be0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f531c0097f0 tx=0x7f531c01ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=347c6198d1e63a5d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.986+0000 7f53327fc640 1 -- 172.21.15.26:0/2190010924 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0a89b0 con 0x7f53340b8940 2023-12-19T18:16:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.986+0000 7f53327fc640 1 -- 172.21.15.26:0/2190010924 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c0a8b10 con 0x7f53340b8940 2023-12-19T18:16:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.986+0000 7f53327fc640 1 -- 172.21.15.26:0/2190010924 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0a8ea0 con 0x7f53340b8940 2023-12-19T18:16:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 -- 172.21.15.26:0/2190010924 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53340b8940 msgr2=0x7f53340d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 --2- 172.21.15.26:0/2190010924 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53340b8940 0x7f53340d0be0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f531c0097f0 tx=0x7f531c01ecb0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 -- 172.21.15.26:0/2190010924 shutdown_connections 2023-12-19T18:16:09.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 --2- 172.21.15.26:0/2190010924 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53340b8940 0x7f53340d0be0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 -- 172.21.15.26:0/2190010924 >> 172.21.15.26:0/2190010924 conn(0x7f5334065ee0 msgr2=0x7f53340cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:09.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 -- 172.21.15.26:0/2190010924 shutdown_connections 2023-12-19T18:16:09.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 -- 172.21.15.26:0/2190010924 wait complete. 2023-12-19T18:16:09.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 Processor -- start 2023-12-19T18:16:09.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 -- start start 2023-12-19T18:16:09.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53340b4600 0x7f5334151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53340b8940 0x7f5334154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53340d1120 0x7f5334155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533415fb20 con 0x7f53340b8940 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5334155a70 con 0x7f53340b4600 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f533928c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5334152090 con 0x7f53340d1120 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f532bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53340b8940 0x7f5334154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f532bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53340b8940 0x7f5334154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48360/0 (socket says 172.21.15.26:48360) 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f532bfff640 1 -- 172.21.15.26:0/1753243466 learned_addr learned my addr 172.21.15.26:0/1753243466 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f5332ffd640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53340b4600 0x7f5334151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f53337fe640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53340d1120 0x7f5334155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f532bfff640 1 -- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53340d1120 msgr2=0x7f5334155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f532bfff640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53340d1120 0x7f5334155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f532bfff640 1 -- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53340b4600 msgr2=0x7f5334151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f532bfff640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53340b4600 0x7f5334151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.990+0000 7f532bfff640 1 -- 172.21.15.26:0/1753243466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531c009420 con 0x7f53340b8940 2023-12-19T18:16:09.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.994+0000 7f532bfff640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53340b8940 0x7f5334154c20 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f531c0097f0 tx=0x7f531c01ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.994+0000 7f532b7fe640 1 -- 172.21.15.26:0/1753243466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0b1cf0 con 0x7f53340b8940 2023-12-19T18:16:09.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.994+0000 7f532b7fe640 1 -- 172.21.15.26:0/1753243466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531c0b1e70 con 0x7f53340b8940 2023-12-19T18:16:09.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.994+0000 7f532b7fe640 1 -- 172.21.15.26:0/1753243466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531c0b9630 con 0x7f53340b8940 2023-12-19T18:16:09.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.994+0000 7f533928c640 1 -- 172.21.15.26:0/1753243466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5334155790 con 0x7f53340b8940 2023-12-19T18:16:09.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.994+0000 7f533928c640 1 -- 172.21.15.26:0/1753243466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533415f7b0 con 0x7f53340b8940 2023-12-19T18:16:09.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.994+0000 7f533928c640 1 -- 172.21.15.26:0/1753243466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5334074270 con 0x7f53340b8940 2023-12-19T18:16:09.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.994+0000 7f532b7fe640 1 -- 172.21.15.26:0/1753243466 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 116) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f531c0b7040 con 0x7f53340b8940 2023-12-19T18:16:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.994+0000 7f532b7fe640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f53140658b0 0x7f5314067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.994+0000 7f532b7fe640 1 -- 172.21.15.26:0/1753243466 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 7243+0+0 (secure 0 0 0) 0x7f531c120e10 con 0x7f53340b8940 2023-12-19T18:16:09.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.998+0000 7f5332ffd640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f53140658b0 0x7f5314067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.998+0000 7f5332ffd640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f53140658b0 0x7f5314067d70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f53240011f0 tx=0x7f5324009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:10.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:09.998+0000 7f532b7fe640 1 -- 172.21.15.26:0/1753243466 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f531c124030 con 0x7f53340b8940 2023-12-19T18:16:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.070+0000 7f532b7fe640 1 -- 172.21.15.26:0/1753243466 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 117) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f531c0ee8a0 con 0x7f53340b8940 2023-12-19T18:16:10.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.194+0000 7f533928c640 1 -- 172.21.15.26:0/1753243466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f53340b3f20 con 0x7f53340b8940 2023-12-19T18:16:10.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.814+0000 7f532b7fe640 1 -- 172.21.15.26:0/1753243466 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v89) v1 ==== 104+0+0 (secure 0 0 0) 0x7f531c0bb2b0 con 0x7f53340b8940 2023-12-19T18:16:10.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.818+0000 7f533928c640 1 -- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f53140658b0 msgr2=0x7f5314067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.818+0000 7f533928c640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f53140658b0 0x7f5314067d70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f53240011f0 tx=0x7f5324009040 comp rx=0 tx=0).stop 2023-12-19T18:16:10.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.818+0000 7f533928c640 1 -- 172.21.15.26:0/1753243466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53340b8940 msgr2=0x7f5334154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.818+0000 7f533928c640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53340b8940 0x7f5334154c20 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f531c0097f0 tx=0x7f531c01ba50 comp rx=0 tx=0).stop 2023-12-19T18:16:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.818+0000 7f533928c640 1 -- 172.21.15.26:0/1753243466 shutdown_connections 2023-12-19T18:16:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.818+0000 7f533928c640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f53140658b0 0x7f5314067d70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.818+0000 7f533928c640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f53340d1120 0x7f5334155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.818+0000 7f533928c640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f53340b8940 0x7f5334154c20 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.818+0000 7f533928c640 1 --2- 172.21.15.26:0/1753243466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53340b4600 0x7f5334151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.818+0000 7f533928c640 1 -- 172.21.15.26:0/1753243466 >> 172.21.15.26:0/1753243466 conn(0x7f5334065ee0 msgr2=0x7f53340cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.818+0000 7f533928c640 1 -- 172.21.15.26:0/1753243466 shutdown_connections 2023-12-19T18:16:10.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:10.818+0000 7f533928c640 1 -- 172.21.15.26:0/1753243466 wait complete. 2023-12-19T18:16:10.843 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:16:11.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.034+0000 7fac31041640 1 Processor -- start 2023-12-19T18:16:11.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.034+0000 7fac31041640 1 -- start start 2023-12-19T18:16:11.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.034+0000 7fac31041640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fac2c070ff0 0x7fac2c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.034+0000 7fac31041640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac2c0c94a0 0x7fac2c06f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.034+0000 7fac31041640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2c06c4c0 con 0x7fac2c1597d0 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.034+0000 7fac31041640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c0713d0 con 0x7fac2c070ff0 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.034+0000 7fac31041640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c159530 con 0x7fac2c0c94a0 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.038+0000 7fac2a59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fac2c1597d0 0x7fac2c0c8d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33088/0 (socket says 172.21.15.26:33088) 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.038+0000 7fac2a59c640 1 -- 172.21.15.26:0/2459001171 learned_addr learned my addr 172.21.15.26:0/2459001171 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.038+0000 7fac2ad9d640 1 --2- 172.21.15.26:0/2459001171 >> v2:172.21.15.160:3300/0 conn(0x7fac2c070ff0 0x7fac2c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.038+0000 7fac2b59e640 1 --2- 172.21.15.26:0/2459001171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac2c0c94a0 0x7fac2c06f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.038+0000 7fac29d9b640 1 -- 172.21.15.26:0/2459001171 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871876479 0 0) 0x7fac2c06c4c0 con 0x7fac2c1597d0 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.038+0000 7fac29d9b640 1 -- 172.21.15.26:0/2459001171 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac14003530 con 0x7fac2c1597d0 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.038+0000 7fac2b59e640 1 -- 172.21.15.26:0/2459001171 >> v2:172.21.15.160:3300/0 conn(0x7fac2c070ff0 msgr2=0x7fac2c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.038+0000 7fac2b59e640 1 --2- 172.21.15.26:0/2459001171 >> v2:172.21.15.160:3300/0 conn(0x7fac2c070ff0 0x7fac2c06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.038+0000 7fac2b59e640 1 -- 172.21.15.26:0/2459001171 >> v1:172.21.15.26:6789/0 conn(0x7fac2c1597d0 legacy=0x7fac2c0c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.038+0000 7fac2b59e640 1 -- 172.21.15.26:0/2459001171 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c06b420 con 0x7fac2c0c94a0 2023-12-19T18:16:11.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.038+0000 7fac2b59e640 1 --2- 172.21.15.26:0/2459001171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac2c0c94a0 0x7fac2c06f290 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fac24009d90 tx=0x7fac2401c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ebd3208536fabb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:11.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac29d9b640 1 -- 172.21.15.26:0/2459001171 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac240a5960 con 0x7fac2c0c94a0 2023-12-19T18:16:11.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac29d9b640 1 -- 172.21.15.26:0/2459001171 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac240a5ac0 con 0x7fac2c0c94a0 2023-12-19T18:16:11.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac29d9b640 1 -- 172.21.15.26:0/2459001171 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac240a5e50 con 0x7fac2c0c94a0 2023-12-19T18:16:11.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 -- 172.21.15.26:0/2459001171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac2c0c94a0 msgr2=0x7fac2c06f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 --2- 172.21.15.26:0/2459001171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac2c0c94a0 0x7fac2c06f290 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fac24009d90 tx=0x7fac2401c920 comp rx=0 tx=0).stop 2023-12-19T18:16:11.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 -- 172.21.15.26:0/2459001171 shutdown_connections 2023-12-19T18:16:11.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 --2- 172.21.15.26:0/2459001171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac2c0c94a0 0x7fac2c06f290 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 --2- 172.21.15.26:0/2459001171 >> v2:172.21.15.160:3300/0 conn(0x7fac2c070ff0 0x7fac2c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 -- 172.21.15.26:0/2459001171 >> 172.21.15.26:0/2459001171 conn(0x7fac2c0b0270 msgr2=0x7fac2c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:11.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 -- 172.21.15.26:0/2459001171 shutdown_connections 2023-12-19T18:16:11.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 -- 172.21.15.26:0/2459001171 wait complete. 2023-12-19T18:16:11.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 Processor -- start 2023-12-19T18:16:11.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 -- start start 2023-12-19T18:16:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2c070ff0 0x7fac2c157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac2c0c94a0 0x7fac2c1f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac2c1597d0 0x7fac2c0ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c0c33b0 con 0x7fac2c070ff0 2023-12-19T18:16:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c157d50 con 0x7fac2c0c94a0 2023-12-19T18:16:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac31041640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c1f9800 con 0x7fac2c1597d0 2023-12-19T18:16:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac2ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2c070ff0 0x7fac2c157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac2ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2c070ff0 0x7fac2c157810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48370/0 (socket says 172.21.15.26:48370) 2023-12-19T18:16:11.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.042+0000 7fac2ad9d640 1 -- 172.21.15.26:0/3723401 learned_addr learned my addr 172.21.15.26:0/3723401 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:11.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac2a59c640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac2c0c94a0 0x7fac2c1f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac2b59e640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac2c1597d0 0x7fac2c0ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac2ad9d640 1 -- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac2c1597d0 msgr2=0x7fac2c0ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac2ad9d640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac2c1597d0 0x7fac2c0ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac2ad9d640 1 -- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac2c0c94a0 msgr2=0x7fac2c1f92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac2ad9d640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac2c0c94a0 0x7fac2c1f92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac2ad9d640 1 -- 172.21.15.26:0/3723401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac24009a70 con 0x7fac2c070ff0 2023-12-19T18:16:11.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac2ad9d640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2c070ff0 0x7fac2c157810 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00e940 tx=0x7fac1c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:11.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac137fe640 1 -- 172.21.15.26:0/3723401 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c00cc40 con 0x7fac2c070ff0 2023-12-19T18:16:11.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac137fe640 1 -- 172.21.15.26:0/3723401 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c00cda0 con 0x7fac2c070ff0 2023-12-19T18:16:11.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac137fe640 1 -- 172.21.15.26:0/3723401 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0115f0 con 0x7fac2c070ff0 2023-12-19T18:16:11.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac31041640 1 -- 172.21.15.26:0/3723401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2c0c85f0 con 0x7fac2c070ff0 2023-12-19T18:16:11.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac31041640 1 -- 172.21.15.26:0/3723401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2c0c2a10 con 0x7fac2c070ff0 2023-12-19T18:16:11.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac31041640 1 -- 172.21.15.26:0/3723401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2c0c3620 con 0x7fac2c070ff0 2023-12-19T18:16:11.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.046+0000 7fac137fe640 1 -- 172.21.15.26:0/3723401 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 117) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fac1c011750 con 0x7fac2c070ff0 2023-12-19T18:16:11.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.050+0000 7fac137fe640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fac18065af0 0x7fac18067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.050+0000 7fac137fe640 1 -- 172.21.15.26:0/3723401 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 7243+0+0 (secure 0 0 0) 0x7fac1c0839a0 con 0x7fac2c070ff0 2023-12-19T18:16:11.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.050+0000 7fac2a59c640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fac18065af0 0x7fac18067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.050+0000 7fac2a59c640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fac18065af0 0x7fac18067fb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac2c1f8840 tx=0x7fac20009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:11.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.054+0000 7fac137fe640 1 -- 172.21.15.26:0/3723401 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fac1c050cf0 con 0x7fac2c070ff0 2023-12-19T18:16:11.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.214+0000 7fac31041640 1 -- 172.21.15.26:0/3723401 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fac2c074270 con 0x7fac2c070ff0 2023-12-19T18:16:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.826+0000 7fac137fe640 1 -- 172.21.15.26:0/3723401 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fac1c0510c0 con 0x7fac2c070ff0 2023-12-19T18:16:11.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.826+0000 7fac137fe640 1 -- 172.21.15.26:0/3723401 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v202) v1 ==== 181+0+0 (secure 0 0 0) 0x7fac1c07fab0 con 0x7fac2c070ff0 2023-12-19T18:16:11.831 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:16:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.830+0000 7fac31041640 1 -- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fac18065af0 msgr2=0x7fac18067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.830+0000 7fac31041640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fac18065af0 0x7fac18067fb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fac2c1f8840 tx=0x7fac20009040 comp rx=0 tx=0).stop 2023-12-19T18:16:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.830+0000 7fac31041640 1 -- 172.21.15.26:0/3723401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2c070ff0 msgr2=0x7fac2c157810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.830+0000 7fac31041640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2c070ff0 0x7fac2c157810 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00e940 tx=0x7fac1c00ee00 comp rx=0 tx=0).stop 2023-12-19T18:16:11.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.830+0000 7fac31041640 1 -- 172.21.15.26:0/3723401 shutdown_connections 2023-12-19T18:16:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.830+0000 7fac31041640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fac18065af0 0x7fac18067fb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.830+0000 7fac31041640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fac2c1597d0 0x7fac2c0ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.830+0000 7fac31041640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fac2c0c94a0 0x7fac2c1f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.830+0000 7fac31041640 1 --2- 172.21.15.26:0/3723401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2c070ff0 0x7fac2c157810 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.830+0000 7fac31041640 1 -- 172.21.15.26:0/3723401 >> 172.21.15.26:0/3723401 conn(0x7fac2c0b0270 msgr2=0x7fac2c0ad5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.830+0000 7fac31041640 1 -- 172.21.15.26:0/3723401 shutdown_connections 2023-12-19T18:16:11.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:11.830+0000 7fac31041640 1 -- 172.21.15.26:0/3723401 wait complete. 2023-12-19T18:16:11.856 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:16:12.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.002+0000 7f8c1f6b6640 1 Processor -- start 2023-12-19T18:16:12.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.002+0000 7f8c1f6b6640 1 -- start start 2023-12-19T18:16:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c1f6b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c180ce440 0x7f8c180ac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c1f6b6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1806d2b0 con 0x7f8c181573a0 2023-12-19T18:16:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c1f6b6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c180cc8e0 con 0x7f8c180cb680 2023-12-19T18:16:12.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c1f6b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180ccb50 con 0x7f8c180ce440 2023-12-19T18:16:12.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c1d452640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8c181573a0 0x7f8c1815dc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33090/0 (socket says 172.21.15.26:33090) 2023-12-19T18:16:12.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c1d452640 1 -- 172.21.15.26:0/308041579 learned_addr learned my addr 172.21.15.26:0/308041579 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:12.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c1cc51640 1 --2- 172.21.15.26:0/308041579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c180ce440 0x7f8c180ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c0ffff640 1 -- 172.21.15.26:0/308041579 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011619645 0 0) 0x7f8c1806d2b0 con 0x7f8c181573a0 2023-12-19T18:16:12.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c0ffff640 1 -- 172.21.15.26:0/308041579 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c04003530 con 0x7f8c181573a0 2023-12-19T18:16:12.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c0ffff640 1 -- 172.21.15.26:0/308041579 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280753823 0 0) 0x7f8c180cc8e0 con 0x7f8c180cb680 2023-12-19T18:16:12.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c0ffff640 1 -- 172.21.15.26:0/308041579 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c1806d2b0 con 0x7f8c180cb680 2023-12-19T18:16:12.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c1cc51640 1 -- 172.21.15.26:0/308041579 >> v1:172.21.15.160:6789/0 conn(0x7f8c180cb680 legacy=0x7f8c1806bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c1cc51640 1 -- 172.21.15.26:0/308041579 >> v1:172.21.15.26:6789/0 conn(0x7f8c181573a0 legacy=0x7f8c1815dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.006+0000 7f8c1cc51640 1 -- 172.21.15.26:0/308041579 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c180cce30 con 0x7f8c180ce440 2023-12-19T18:16:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1cc51640 1 --2- 172.21.15.26:0/308041579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c180ce440 0x7f8c180ac850 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097f0 tx=0x7f8c0801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b50196342275a0f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c0ffff640 1 -- 172.21.15.26:0/308041579 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c080aa640 con 0x7f8c180ce440 2023-12-19T18:16:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c0ffff640 1 -- 172.21.15.26:0/308041579 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c080aa7a0 con 0x7f8c180ce440 2023-12-19T18:16:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c0ffff640 1 -- 172.21.15.26:0/308041579 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c080aab10 con 0x7f8c180ce440 2023-12-19T18:16:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/308041579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c180ce440 msgr2=0x7f8c180ac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 --2- 172.21.15.26:0/308041579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c180ce440 0x7f8c180ac850 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097f0 tx=0x7f8c0801c4c0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/308041579 shutdown_connections 2023-12-19T18:16:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 --2- 172.21.15.26:0/308041579 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c180ce440 0x7f8c180ac850 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/308041579 >> 172.21.15.26:0/308041579 conn(0x7f8c180b0980 msgr2=0x7f8c180b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/308041579 shutdown_connections 2023-12-19T18:16:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/308041579 wait complete. 2023-12-19T18:16:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 Processor -- start 2023-12-19T18:16:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 -- start start 2023-12-19T18:16:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c180cb680 0x7f8c18153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c180ce440 0x7f8c18147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c181573a0 0x7f8c18148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18152300 con 0x7f8c180ce440 2023-12-19T18:16:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1814a0a0 con 0x7f8c180cb680 2023-12-19T18:16:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1f6b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18148fc0 con 0x7f8c181573a0 2023-12-19T18:16:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.010+0000 7f8c1cc51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c180ce440 0x7f8c18147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1cc51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c180ce440 0x7f8c18147cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48382/0 (socket says 172.21.15.26:48382) 2023-12-19T18:16:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1cc51640 1 -- 172.21.15.26:0/3852921671 learned_addr learned my addr 172.21.15.26:0/3852921671 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1d452640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c180cb680 0x7f8c18153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1dc53640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c181573a0 0x7f8c18148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1cc51640 1 -- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c181573a0 msgr2=0x7f8c18148210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1cc51640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c181573a0 0x7f8c18148210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1cc51640 1 -- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c180cb680 msgr2=0x7f8c18153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1cc51640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c180cb680 0x7f8c18153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1cc51640 1 -- 172.21.15.26:0/3852921671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c08009420 con 0x7f8c180ce440 2023-12-19T18:16:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1dc53640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c181573a0 0x7f8c18148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1cc51640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c180ce440 0x7f8c18147cd0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f8c080190d0 tx=0x7f8c08019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c0dffb640 1 -- 172.21.15.26:0/3852921671 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08004300 con 0x7f8c180ce440 2023-12-19T18:16:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c0dffb640 1 -- 172.21.15.26:0/3852921671 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c08004460 con 0x7f8c180ce440 2023-12-19T18:16:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c0dffb640 1 -- 172.21.15.26:0/3852921671 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c080aab10 con 0x7f8c180ce440 2023-12-19T18:16:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/3852921671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c180cc020 con 0x7f8c180ce440 2023-12-19T18:16:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/3852921671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c181524e0 con 0x7f8c180ce440 2023-12-19T18:16:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/3852921671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c18074270 con 0x7f8c180ce440 2023-12-19T18:16:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c0dffb640 1 -- 172.21.15.26:0/3852921671 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8c080a8050 con 0x7f8c180ce440 2023-12-19T18:16:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.014+0000 7f8c0dffb640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8bfc065b80 0x7f8bfc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.018+0000 7f8c0dffb640 1 -- 172.21.15.26:0/3852921671 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6805+0+0 (secure 0 0 0) 0x7f8c0811d490 con 0x7f8c180ce440 2023-12-19T18:16:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.018+0000 7f8c1d452640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8bfc065b80 0x7f8bfc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.018+0000 7f8c1d452640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8bfc065b80 0x7f8bfc068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8c14007fd0 tx=0x7f8c14007d80 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:12.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.022+0000 7f8c0dffb640 1 -- 172.21.15.26:0/3852921671 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8c080ea980 con 0x7f8c180ce440 2023-12-19T18:16:12.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.182+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/3852921671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c181516d0 con 0x7f8c180ce440 2023-12-19T18:16:12.265 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:16:12.269 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:16:12.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.826+0000 7f8c0dffb640 1 -- 172.21.15.26:0/3852921671 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v203) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8c080edfa0 con 0x7f8c180ce440 2023-12-19T18:16:12.830 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T18:16:12.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.830+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8bfc065b80 msgr2=0x7f8bfc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.830+0000 7f8c1f6b6640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8bfc065b80 0x7f8bfc068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8c14007fd0 tx=0x7f8c14007d80 comp rx=0 tx=0).stop 2023-12-19T18:16:12.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.830+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/3852921671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c180ce440 msgr2=0x7f8c18147cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.830+0000 7f8c1f6b6640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c180ce440 0x7f8c18147cd0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f8c080190d0 tx=0x7f8c08019260 comp rx=0 tx=0).stop 2023-12-19T18:16:12.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.830+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/3852921671 shutdown_connections 2023-12-19T18:16:12.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.830+0000 7f8c1f6b6640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8bfc065b80 0x7f8bfc068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.830+0000 7f8c1f6b6640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c181573a0 0x7f8c18148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.830+0000 7f8c1f6b6640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c180ce440 0x7f8c18147cd0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.830+0000 7f8c1f6b6640 1 --2- 172.21.15.26:0/3852921671 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c180cb680 0x7f8c18153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.830+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/3852921671 >> 172.21.15.26:0/3852921671 conn(0x7f8c180b0980 msgr2=0x7f8c180aceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.830+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/3852921671 shutdown_connections 2023-12-19T18:16:12.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:12.830+0000 7f8c1f6b6640 1 -- 172.21.15.26:0/3852921671 wait complete. 2023-12-19T18:16:12.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-19T18:16:13.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50f02d640 1 Processor -- start 2023-12-19T18:16:13.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50f02d640 1 -- start start 2023-12-19T18:16:13.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50f02d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc50815cd10 0x7fc5080cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50f02d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc50806d3e0 0x7fc50806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50f02d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc50806c130 0x7fc508160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50f02d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50806c5c0 con 0x7fc50806c130 2023-12-19T18:16:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50f02d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50815d0f0 con 0x7fc50815cd10 2023-12-19T18:16:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50f02d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5080c98b0 con 0x7fc50806d3e0 2023-12-19T18:16:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50d5ca640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc50806c130 0x7fc508160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50d5ca640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc50806c130 0x7fc508160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48394/0 (socket says 172.21.15.26:48394) 2023-12-19T18:16:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50d5ca640 1 -- 172.21.15.26:0/3360879542 learned_addr learned my addr 172.21.15.26:0/3360879542 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50cdc9640 1 --2- 172.21.15.26:0/3360879542 >> v2:172.21.15.160:3300/0 conn(0x7fc50815cd10 0x7fc5080cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc507fff640 1 --2- 172.21.15.26:0/3360879542 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc50806d3e0 0x7fc50806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50d5ca640 1 -- 172.21.15.26:0/3360879542 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc50806d3e0 msgr2=0x7fc50806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50d5ca640 1 --2- 172.21.15.26:0/3360879542 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc50806d3e0 0x7fc50806bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50d5ca640 1 -- 172.21.15.26:0/3360879542 >> v2:172.21.15.160:3300/0 conn(0x7fc50815cd10 msgr2=0x7fc5080cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50d5ca640 1 --2- 172.21.15.26:0/3360879542 >> v2:172.21.15.160:3300/0 conn(0x7fc50815cd10 0x7fc5080cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50d5ca640 1 -- 172.21.15.26:0/3360879542 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5080cd0e0 con 0x7fc50806c130 2023-12-19T18:16:13.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc50cdc9640 1 --2- 172.21.15.26:0/3360879542 >> v2:172.21.15.160:3300/0 conn(0x7fc50815cd10 0x7fc5080cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:13.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.006+0000 7fc507fff640 1 --2- 172.21.15.26:0/3360879542 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc50806d3e0 0x7fc50806bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:13.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.010+0000 7fc50d5ca640 1 --2- 172.21.15.26:0/3360879542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc50806c130 0x7fc508160790 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fc5080cff10 tx=0x7fc50001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4db6a9c4efe5191 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.010+0000 7fc5077fe640 1 -- 172.21.15.26:0/3360879542 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5000af400 con 0x7fc50806c130 2023-12-19T18:16:13.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.010+0000 7fc5077fe640 1 -- 172.21.15.26:0/3360879542 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5000ab050 con 0x7fc50806c130 2023-12-19T18:16:13.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.010+0000 7fc5077fe640 1 -- 172.21.15.26:0/3360879542 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5000af780 con 0x7fc50806c130 2023-12-19T18:16:13.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.010+0000 7fc50f02d640 1 -- 172.21.15.26:0/3360879542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc50806c130 msgr2=0x7fc508160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.010+0000 7fc50f02d640 1 --2- 172.21.15.26:0/3360879542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc50806c130 0x7fc508160790 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fc5080cff10 tx=0x7fc50001f1c0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.010+0000 7fc50f02d640 1 -- 172.21.15.26:0/3360879542 shutdown_connections 2023-12-19T18:16:13.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.010+0000 7fc50f02d640 1 --2- 172.21.15.26:0/3360879542 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc50806c130 0x7fc508160790 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.010+0000 7fc50f02d640 1 --2- 172.21.15.26:0/3360879542 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc50806d3e0 0x7fc50806bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.010+0000 7fc50f02d640 1 --2- 172.21.15.26:0/3360879542 >> v2:172.21.15.160:3300/0 conn(0x7fc50815cd10 0x7fc5080cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.010+0000 7fc50f02d640 1 -- 172.21.15.26:0/3360879542 >> 172.21.15.26:0/3360879542 conn(0x7fc5080cf260 msgr2=0x7fc5080cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:13.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.010+0000 7fc50f02d640 1 -- 172.21.15.26:0/3360879542 shutdown_connections 2023-12-19T18:16:13.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.010+0000 7fc50f02d640 1 -- 172.21.15.26:0/3360879542 wait complete. 2023-12-19T18:16:13.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc50f02d640 1 Processor -- start 2023-12-19T18:16:13.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc50f02d640 1 -- start start 2023-12-19T18:16:13.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc50f02d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc50806c130 0x7fc5080adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc50f02d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc50806d3e0 0x7fc5080acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc50f02d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc50815cd10 0x7fc5081e8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc50f02d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50815a010 con 0x7fc50806d3e0 2023-12-19T18:16:13.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc50f02d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5080aff10 con 0x7fc50806c130 2023-12-19T18:16:13.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc50f02d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5080b01f0 con 0x7fc50815cd10 2023-12-19T18:16:13.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc507fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc50806d3e0 0x7fc5080acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc507fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc50806d3e0 0x7fc5080acf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48402/0 (socket says 172.21.15.26:48402) 2023-12-19T18:16:13.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc507fff640 1 -- 172.21.15.26:0/4133454504 learned_addr learned my addr 172.21.15.26:0/4133454504 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:13.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc50d5ca640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc50815cd10 0x7fc5081e8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc50cdc9640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc50806c130 0x7fc5080adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc507fff640 1 -- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc50815cd10 msgr2=0x7fc5081e8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc507fff640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc50815cd10 0x7fc5081e8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc507fff640 1 -- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc50806c130 msgr2=0x7fc5080adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc507fff640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc50806c130 0x7fc5080adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc507fff640 1 -- 172.21.15.26:0/4133454504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc500009b30 con 0x7fc50806d3e0 2023-12-19T18:16:13.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc507fff640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc50806d3e0 0x7fc5080acf70 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400e810 tx=0x7fc4f400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc5057fa640 1 -- 172.21.15.26:0/4133454504 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4004030 con 0x7fc50806d3e0 2023-12-19T18:16:13.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc5057fa640 1 -- 172.21.15.26:0/4133454504 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f4004190 con 0x7fc50806d3e0 2023-12-19T18:16:13.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc50f02d640 1 -- 172.21.15.26:0/4133454504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50815a260 con 0x7fc50806d3e0 2023-12-19T18:16:13.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc5057fa640 1 -- 172.21.15.26:0/4133454504 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4004520 con 0x7fc50806d3e0 2023-12-19T18:16:13.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.014+0000 7fc50f02d640 1 -- 172.21.15.26:0/4133454504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc508155780 con 0x7fc50806d3e0 2023-12-19T18:16:13.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.018+0000 7fc50f02d640 1 -- 172.21.15.26:0/4133454504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc508074270 con 0x7fc50806d3e0 2023-12-19T18:16:13.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.018+0000 7fc5057fa640 1 -- 172.21.15.26:0/4133454504 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc4f400c6d0 con 0x7fc50806d3e0 2023-12-19T18:16:13.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.018+0000 7fc5057fa640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc4ec065b80 0x7fc4ec068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.018+0000 7fc5057fa640 1 -- 172.21.15.26:0/4133454504 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6391+0+0 (secure 0 0 0) 0x7fc4f40832a0 con 0x7fc50806d3e0 2023-12-19T18:16:13.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.018+0000 7fc50cdc9640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc4ec065b80 0x7fc4ec068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.018+0000 7fc50cdc9640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc4ec065b80 0x7fc4ec068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00ba10 tx=0x7fc4fc008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.022+0000 7fc5057fa640 1 -- 172.21.15.26:0/4133454504 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc4f40506f0 con 0x7fc50806d3e0 2023-12-19T18:16:13.181 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.178+0000 7fc50f02d640 1 -- 172.21.15.26:0/4133454504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fc5080b1b40 con 0x7fc50806d3e0 2023-12-19T18:16:13.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.182+0000 7fc5057fa640 1 -- 172.21.15.26:0/4133454504 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v67) v1 ==== 120+0+0 (secure 0 0 0) 0x7fc4f4053d10 con 0x7fc50806d3e0 2023-12-19T18:16:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.186+0000 7fc50f02d640 1 -- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc4ec065b80 msgr2=0x7fc4ec068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.186+0000 7fc50f02d640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc4ec065b80 0x7fc4ec068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00ba10 tx=0x7fc4fc008040 comp rx=0 tx=0).stop 2023-12-19T18:16:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.186+0000 7fc50f02d640 1 -- 172.21.15.26:0/4133454504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc50806d3e0 msgr2=0x7fc5080acf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.186+0000 7fc50f02d640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc50806d3e0 0x7fc5080acf70 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400e810 tx=0x7fc4f400ecd0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.186+0000 7fc50f02d640 1 -- 172.21.15.26:0/4133454504 shutdown_connections 2023-12-19T18:16:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.186+0000 7fc50f02d640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc4ec065b80 0x7fc4ec068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.186+0000 7fc50f02d640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc50815cd10 0x7fc5081e8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.186+0000 7fc50f02d640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc50806d3e0 0x7fc5080acf70 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.186+0000 7fc50f02d640 1 --2- 172.21.15.26:0/4133454504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc50806c130 0x7fc5080adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.186+0000 7fc50f02d640 1 -- 172.21.15.26:0/4133454504 >> 172.21.15.26:0/4133454504 conn(0x7fc5080cf260 msgr2=0x7fc5080cabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.186+0000 7fc50f02d640 1 -- 172.21.15.26:0/4133454504 shutdown_connections 2023-12-19T18:16:13.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.186+0000 7fc50f02d640 1 -- 172.21.15.26:0/4133454504 wait complete. 2023-12-19T18:16:13.208 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:16:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4d02939640 1 Processor -- start 2023-12-19T18:16:13.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4d02939640 1 -- start start 2023-12-19T18:16:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4d02939640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4cfc0cfbd0 0x7f4cfc0d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4d02939640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cfc0b71f0 con 0x7f4cfc15cc70 2023-12-19T18:16:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4d02939640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cfc0d1370 con 0x7f4cfc0b88b0 2023-12-19T18:16:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4d02939640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cfc0d0580 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4cfbfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4cfc15cc70 0x7f4cfc0d0c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33102/0 (socket says 172.21.15.26:33102) 2023-12-19T18:16:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4cfbfff640 1 -- 172.21.15.26:0/3527607157 learned_addr learned my addr 172.21.15.26:0/3527607157 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4d00ed6640 1 --2- 172.21.15.26:0/3527607157 >> v2:172.21.15.160:3300/0 conn(0x7f4cfc0cfbd0 0x7f4cfc0d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4cfaffd640 1 -- 172.21.15.26:0/3527607157 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906432911 0 0) 0x7f4cfc0b71f0 con 0x7f4cfc15cc70 2023-12-19T18:16:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4cfaffd640 1 -- 172.21.15.26:0/3527607157 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ce4003530 con 0x7f4cfc15cc70 2023-12-19T18:16:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4d00ed6640 1 -- 172.21.15.26:0/3527607157 >> v1:172.21.15.160:6789/0 conn(0x7f4cfc0b88b0 legacy=0x7f4cfc0b5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4d00ed6640 1 -- 172.21.15.26:0/3527607157 >> v1:172.21.15.26:6789/0 conn(0x7f4cfc15cc70 legacy=0x7f4cfc0d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.354+0000 7f4d00ed6640 1 -- 172.21.15.26:0/3527607157 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc0b6310 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.358+0000 7f4d00ed6640 1 --2- 172.21.15.26:0/3527607157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cfc0cfbd0 0x7f4cfc0d0040 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009d90 tx=0x7f4cf401f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33c01fcf45d118dd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.358+0000 7f4cfaffd640 1 -- 172.21.15.26:0/3527607157 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf40af600 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.358+0000 7f4cfaffd640 1 -- 172.21.15.26:0/3527607157 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf40ab050 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.358+0000 7f4cfaffd640 1 -- 172.21.15.26:0/3527607157 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf40af960 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.358+0000 7f4d02939640 1 -- 172.21.15.26:0/3527607157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cfc0cfbd0 msgr2=0x7f4cfc0d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.358+0000 7f4d02939640 1 --2- 172.21.15.26:0/3527607157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cfc0cfbd0 0x7f4cfc0d0040 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4009d90 tx=0x7f4cf401f130 comp rx=0 tx=0).stop 2023-12-19T18:16:13.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.358+0000 7f4d02939640 1 -- 172.21.15.26:0/3527607157 shutdown_connections 2023-12-19T18:16:13.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.358+0000 7f4d02939640 1 --2- 172.21.15.26:0/3527607157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cfc0cfbd0 0x7f4cfc0d0040 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.358+0000 7f4d02939640 1 -- 172.21.15.26:0/3527607157 >> 172.21.15.26:0/3527607157 conn(0x7f4cfc06a120 msgr2=0x7f4cfc06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:13.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.358+0000 7f4d02939640 1 -- 172.21.15.26:0/3527607157 shutdown_connections 2023-12-19T18:16:13.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.358+0000 7f4d02939640 1 -- 172.21.15.26:0/3527607157 wait complete. 2023-12-19T18:16:13.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.358+0000 7f4d02939640 1 Processor -- start 2023-12-19T18:16:13.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4d02939640 1 -- start start 2023-12-19T18:16:13.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4d02939640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4cfc0b88b0 0x7f4cfc1f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4d02939640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0cfbd0 0x7f4cfc1f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4d02939640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cfc15cc70 0x7f4cfc15f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4d02939640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc0c7ae0 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4d02939640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc1f67a0 con 0x7f4cfc15cc70 2023-12-19T18:16:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4d02939640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc1f3700 con 0x7f4cfc0b88b0 2023-12-19T18:16:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4cfb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0cfbd0 0x7f4cfc1f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4cfb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0cfbd0 0x7f4cfc1f5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48408/0 (socket says 172.21.15.26:48408) 2023-12-19T18:16:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4cfb7fe640 1 -- 172.21.15.26:0/240485076 learned_addr learned my addr 172.21.15.26:0/240485076 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4d00ed6640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cfc15cc70 0x7f4cfc15f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4cfbfff640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4cfc0b88b0 0x7f4cfc1f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4cfb7fe640 1 -- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4cfc0b88b0 msgr2=0x7f4cfc1f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4cfb7fe640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4cfc0b88b0 0x7f4cfc1f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4cfb7fe640 1 -- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cfc15cc70 msgr2=0x7f4cfc15f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4cfb7fe640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cfc15cc70 0x7f4cfc15f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4cfb7fe640 1 -- 172.21.15.26:0/240485076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf4009a70 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4cfb7fe640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0cfbd0 0x7f4cfc1f5010 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000c450 tx=0x7f4cf000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4cf8ff9640 1 -- 172.21.15.26:0/240485076 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00052d0 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4cf8ff9640 1 -- 172.21.15.26:0/240485076 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf0005430 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4cf8ff9640 1 -- 172.21.15.26:0/240485076 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0007260 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4d02939640 1 -- 172.21.15.26:0/240485076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cfc15d4c0 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4d02939640 1 -- 172.21.15.26:0/240485076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cfc0c7fd0 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.362+0000 7f4d02939640 1 -- 172.21.15.26:0/240485076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cfc074270 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.366+0000 7f4cf8ff9640 1 -- 172.21.15.26:0/240485076 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4cf00073c0 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.366+0000 7f4cf8ff9640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4ce8065b80 0x7f4ce8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.366+0000 7f4cf8ff9640 1 -- 172.21.15.26:0/240485076 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f4cf0082d00 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.366+0000 7f4cfbfff640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4ce8065b80 0x7f4ce8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.366+0000 7f4cfbfff640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4ce8065b80 0x7f4ce8068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0011f0 tx=0x7f4cec007d40 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.370+0000 7f4cf8ff9640 1 -- 172.21.15.26:0/240485076 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4cf00503a0 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.526+0000 7f4d02939640 1 -- 172.21.15.26:0/240485076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4cfc0b82d0 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.530+0000 7f4cf8ff9640 1 -- 172.21.15.26:0/240485076 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v68) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4cf00539c0 con 0x7f4cfc0cfbd0 2023-12-19T18:16:13.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.534+0000 7f4d02939640 1 -- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4ce8065b80 msgr2=0x7f4ce8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.534+0000 7f4d02939640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4ce8065b80 0x7f4ce8068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4cec0011f0 tx=0x7f4cec007d40 comp rx=0 tx=0).stop 2023-12-19T18:16:13.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.534+0000 7f4d02939640 1 -- 172.21.15.26:0/240485076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0cfbd0 msgr2=0x7f4cfc1f5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.534+0000 7f4d02939640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0cfbd0 0x7f4cfc1f5010 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f4cf000c450 tx=0x7f4cf000c910 comp rx=0 tx=0).stop 2023-12-19T18:16:13.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.534+0000 7f4d02939640 1 -- 172.21.15.26:0/240485076 shutdown_connections 2023-12-19T18:16:13.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.534+0000 7f4d02939640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4ce8065b80 0x7f4ce8068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.534+0000 7f4d02939640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cfc15cc70 0x7f4cfc15f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.534+0000 7f4d02939640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4cfc0cfbd0 0x7f4cfc1f5010 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.534+0000 7f4d02939640 1 --2- 172.21.15.26:0/240485076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4cfc0b88b0 0x7f4cfc1f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.534+0000 7f4d02939640 1 -- 172.21.15.26:0/240485076 >> 172.21.15.26:0/240485076 conn(0x7f4cfc06a120 msgr2=0x7f4cfc0cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:13.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.534+0000 7f4d02939640 1 -- 172.21.15.26:0/240485076 shutdown_connections 2023-12-19T18:16:13.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.534+0000 7f4d02939640 1 -- 172.21.15.26:0/240485076 wait complete. 2023-12-19T18:16:13.556 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-19T18:16:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.698+0000 7fd083717640 1 Processor -- start 2023-12-19T18:16:13.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.698+0000 7fd083717640 1 -- start start 2023-12-19T18:16:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd083717640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd07c06d1e0 0x7fd07c0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd083717640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07c071550 con 0x7fd07c071100 2023-12-19T18:16:13.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd083717640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07c1615b0 con 0x7fd07c0adc60 2023-12-19T18:16:13.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd083717640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c0c8d90 con 0x7fd07c06d1e0 2023-12-19T18:16:13.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd081cb4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd07c071100 0x7fd07c06f250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33104/0 (socket says 172.21.15.26:33104) 2023-12-19T18:16:13.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd081cb4640 1 -- 172.21.15.26:0/1506056355 learned_addr learned my addr 172.21.15.26:0/1506056355 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:13.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd080cb2640 1 --2- 172.21.15.26:0/1506056355 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd07c06d1e0 0x7fd07c0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd073fff640 1 -- 172.21.15.26:0/1506056355 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685172117 0 0) 0x7fd07c071550 con 0x7fd07c071100 2023-12-19T18:16:13.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd073fff640 1 -- 172.21.15.26:0/1506056355 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd068003530 con 0x7fd07c071100 2023-12-19T18:16:13.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd073fff640 1 -- 172.21.15.26:0/1506056355 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559265258 0 0) 0x7fd07c1615b0 con 0x7fd07c0adc60 2023-12-19T18:16:13.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd073fff640 1 -- 172.21.15.26:0/1506056355 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd07c071550 con 0x7fd07c0adc60 2023-12-19T18:16:13.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd080cb2640 1 -- 172.21.15.26:0/1506056355 >> v1:172.21.15.160:6789/0 conn(0x7fd07c0adc60 legacy=0x7fd07c06cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd080cb2640 1 -- 172.21.15.26:0/1506056355 >> v1:172.21.15.26:6789/0 conn(0x7fd07c071100 legacy=0x7fd07c06f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.702+0000 7fd080cb2640 1 -- 172.21.15.26:0/1506056355 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c0c9070 con 0x7fd07c06d1e0 2023-12-19T18:16:13.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.706+0000 7fd080cb2640 1 --2- 172.21.15.26:0/1506056355 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd07c06d1e0 0x7fd07c0ca640 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0097f0 tx=0x7fd06c01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=840d60b3045bdfbd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.706+0000 7fd073fff640 1 -- 172.21.15.26:0/1506056355 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0aa5f0 con 0x7fd07c06d1e0 2023-12-19T18:16:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.706+0000 7fd073fff640 1 -- 172.21.15.26:0/1506056355 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c0aa750 con 0x7fd07c06d1e0 2023-12-19T18:16:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.706+0000 7fd073fff640 1 -- 172.21.15.26:0/1506056355 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0aaac0 con 0x7fd07c06d1e0 2023-12-19T18:16:13.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.706+0000 7fd083717640 1 -- 172.21.15.26:0/1506056355 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd07c06d1e0 msgr2=0x7fd07c0ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.706+0000 7fd083717640 1 --2- 172.21.15.26:0/1506056355 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd07c06d1e0 0x7fd07c0ca640 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0097f0 tx=0x7fd06c01c490 comp rx=0 tx=0).stop 2023-12-19T18:16:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.706+0000 7fd083717640 1 -- 172.21.15.26:0/1506056355 shutdown_connections 2023-12-19T18:16:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.706+0000 7fd083717640 1 --2- 172.21.15.26:0/1506056355 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd07c06d1e0 0x7fd07c0ca640 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.706+0000 7fd083717640 1 -- 172.21.15.26:0/1506056355 >> 172.21.15.26:0/1506056355 conn(0x7fd07c066160 msgr2=0x7fd07c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.706+0000 7fd083717640 1 -- 172.21.15.26:0/1506056355 shutdown_connections 2023-12-19T18:16:13.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.706+0000 7fd083717640 1 -- 172.21.15.26:0/1506056355 wait complete. 2023-12-19T18:16:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.706+0000 7fd083717640 1 Processor -- start 2023-12-19T18:16:13.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.706+0000 7fd083717640 1 -- start start 2023-12-19T18:16:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd083717640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c06d1e0 0x7fd07c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd083717640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd07c071100 0x7fd07c0c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd083717640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd07c0adc60 0x7fd07c14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd083717640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c154630 con 0x7fd07c06d1e0 2023-12-19T18:16:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd083717640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c153430 con 0x7fd07c0adc60 2023-12-19T18:16:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd083717640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c154350 con 0x7fd07c071100 2023-12-19T18:16:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd0814b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c06d1e0 0x7fd07c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd0814b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c06d1e0 0x7fd07c152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48420/0 (socket says 172.21.15.26:48420) 2023-12-19T18:16:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd0814b3640 1 -- 172.21.15.26:0/3978088259 learned_addr learned my addr 172.21.15.26:0/3978088259 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd081cb4640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd07c0adc60 0x7fd07c14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd080cb2640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd07c071100 0x7fd07c0c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd0814b3640 1 -- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd07c071100 msgr2=0x7fd07c0c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd0814b3640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd07c071100 0x7fd07c0c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd0814b3640 1 -- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd07c0adc60 msgr2=0x7fd07c14f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd0814b3640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd07c0adc60 0x7fd07c14f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd0814b3640 1 -- 172.21.15.26:0/3978088259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c009420 con 0x7fd07c06d1e0 2023-12-19T18:16:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd0814b3640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c06d1e0 0x7fd07c152660 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fd07800c460 tx=0x7fd07800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd071ffb640 1 -- 172.21.15.26:0/3978088259 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0780052d0 con 0x7fd07c06d1e0 2023-12-19T18:16:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd071ffb640 1 -- 172.21.15.26:0/3978088259 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd078005450 con 0x7fd07c06d1e0 2023-12-19T18:16:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd083717640 1 -- 172.21.15.26:0/3978088259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07c15bac0 con 0x7fd07c06d1e0 2023-12-19T18:16:13.713 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd071ffb640 1 -- 172.21.15.26:0/3978088259 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078012870 con 0x7fd07c06d1e0 2023-12-19T18:16:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd083717640 1 -- 172.21.15.26:0/3978088259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07c0c79f0 con 0x7fd07c06d1e0 2023-12-19T18:16:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd083717640 1 -- 172.21.15.26:0/3978088259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07c15b920 con 0x7fd07c06d1e0 2023-12-19T18:16:13.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.710+0000 7fd071ffb640 1 -- 172.21.15.26:0/3978088259 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd0780129d0 con 0x7fd07c06d1e0 2023-12-19T18:16:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.714+0000 7fd071ffb640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd060065b80 0x7fd060068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.714+0000 7fd071ffb640 1 -- 172.21.15.26:0/3978088259 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6391+0+0 (secure 0 0 0) 0x7fd078083160 con 0x7fd07c06d1e0 2023-12-19T18:16:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.714+0000 7fd080cb2640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd060065b80 0x7fd060068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.714+0000 7fd080cb2640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd060065b80 0x7fd060068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0ae000 tx=0x7fd06c0a7000 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.718+0000 7fd071ffb640 1 -- 172.21.15.26:0/3978088259 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd07c15b920 con 0x7fd07c06d1e0 2023-12-19T18:16:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.894+0000 7fd083717640 1 -- 172.21.15.26:0/3978088259 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fd07c15b920 con 0x7fd07c06d1e0 2023-12-19T18:16:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.898+0000 7fd071ffb640 1 -- 172.21.15.26:0/3978088259 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v69) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd07c15b920 con 0x7fd07c06d1e0 2023-12-19T18:16:13.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.902+0000 7fd083717640 1 -- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd060065b80 msgr2=0x7fd060068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.902+0000 7fd083717640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd060065b80 0x7fd060068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd06c0ae000 tx=0x7fd06c0a7000 comp rx=0 tx=0).stop 2023-12-19T18:16:13.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.902+0000 7fd083717640 1 -- 172.21.15.26:0/3978088259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c06d1e0 msgr2=0x7fd07c152660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.902+0000 7fd083717640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c06d1e0 0x7fd07c152660 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fd07800c460 tx=0x7fd07800c920 comp rx=0 tx=0).stop 2023-12-19T18:16:13.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.902+0000 7fd083717640 1 -- 172.21.15.26:0/3978088259 shutdown_connections 2023-12-19T18:16:13.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.902+0000 7fd083717640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd060065b80 0x7fd060068040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.902+0000 7fd083717640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd07c0adc60 0x7fd07c14f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.902+0000 7fd083717640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd07c071100 0x7fd07c0c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.902+0000 7fd083717640 1 --2- 172.21.15.26:0/3978088259 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c06d1e0 0x7fd07c152660 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.902+0000 7fd083717640 1 -- 172.21.15.26:0/3978088259 >> 172.21.15.26:0/3978088259 conn(0x7fd07c066160 msgr2=0x7fd07c0b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:13.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.902+0000 7fd083717640 1 -- 172.21.15.26:0/3978088259 shutdown_connections 2023-12-19T18:16:13.905 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:13.902+0000 7fd083717640 1 -- 172.21.15.26:0/3978088259 wait complete. 2023-12-19T18:16:13.923 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-12-19T18:16:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1db9f0640 1 Processor -- start 2023-12-19T18:16:14.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1db9f0640 1 -- start start 2023-12-19T18:16:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1db9f0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff1d406b3d0 0x7ff1d40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1db9f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1d40cec40 0x7ff1d415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1db9f0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d406bc30 con 0x7ff1d4154220 2023-12-19T18:16:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1db9f0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d40cf0d0 con 0x7ff1d406b3d0 2023-12-19T18:16:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1db9f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d40cc6e0 con 0x7ff1d40cec40 2023-12-19T18:16:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1d978c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff1d4154220 0x7ff1d415aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33118/0 (socket says 172.21.15.26:33118) 2023-12-19T18:16:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1d978c640 1 -- 172.21.15.26:0/3959103394 learned_addr learned my addr 172.21.15.26:0/3959103394 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1d8f8b640 1 --2- 172.21.15.26:0/3959103394 >> v2:172.21.15.160:3300/0 conn(0x7ff1d406b3d0 0x7ff1d40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:14.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1d9f8d640 1 --2- 172.21.15.26:0/3959103394 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1d40cec40 0x7ff1d415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1d9f8d640 1 -- 172.21.15.26:0/3959103394 >> v2:172.21.15.160:3300/0 conn(0x7ff1d406b3d0 msgr2=0x7ff1d40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1d9f8d640 1 --2- 172.21.15.26:0/3959103394 >> v2:172.21.15.160:3300/0 conn(0x7ff1d406b3d0 0x7ff1d40ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1d9f8d640 1 -- 172.21.15.26:0/3959103394 >> v1:172.21.15.26:6789/0 conn(0x7ff1d4154220 legacy=0x7ff1d415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:14.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.066+0000 7ff1d9f8d640 1 -- 172.21.15.26:0/3959103394 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d406b950 con 0x7ff1d40cec40 2023-12-19T18:16:14.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.070+0000 7ff1d9f8d640 1 --2- 172.21.15.26:0/3959103394 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1d40cec40 0x7ff1d415b1c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009d90 tx=0x7ff1cc01c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c4a376727d06647 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:14.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.070+0000 7ff1cbfff640 1 -- 172.21.15.26:0/3959103394 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0a59e0 con 0x7ff1d40cec40 2023-12-19T18:16:14.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.070+0000 7ff1cbfff640 1 -- 172.21.15.26:0/3959103394 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc0a5b40 con 0x7ff1d40cec40 2023-12-19T18:16:14.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.070+0000 7ff1cbfff640 1 -- 172.21.15.26:0/3959103394 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0ad440 con 0x7ff1d40cec40 2023-12-19T18:16:14.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.070+0000 7ff1db9f0640 1 -- 172.21.15.26:0/3959103394 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1d40cec40 msgr2=0x7ff1d415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:14.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.070+0000 7ff1db9f0640 1 --2- 172.21.15.26:0/3959103394 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1d40cec40 0x7ff1d415b1c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc009d90 tx=0x7ff1cc01c920 comp rx=0 tx=0).stop 2023-12-19T18:16:14.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.070+0000 7ff1db9f0640 1 -- 172.21.15.26:0/3959103394 shutdown_connections 2023-12-19T18:16:14.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.070+0000 7ff1db9f0640 1 --2- 172.21.15.26:0/3959103394 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1d40cec40 0x7ff1d415b1c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.070+0000 7ff1db9f0640 1 --2- 172.21.15.26:0/3959103394 >> v2:172.21.15.160:3300/0 conn(0x7ff1d406b3d0 0x7ff1d40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.070+0000 7ff1db9f0640 1 -- 172.21.15.26:0/3959103394 >> 172.21.15.26:0/3959103394 conn(0x7ff1d40658e0 msgr2=0x7ff1d4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:14.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.070+0000 7ff1db9f0640 1 -- 172.21.15.26:0/3959103394 shutdown_connections 2023-12-19T18:16:14.074 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.070+0000 7ff1db9f0640 1 -- 172.21.15.26:0/3959103394 wait complete. 2023-12-19T18:16:14.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1db9f0640 1 Processor -- start 2023-12-19T18:16:14.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1db9f0640 1 -- start start 2023-12-19T18:16:14.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1db9f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d406b3d0 0x7ff1d414b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:14.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1db9f0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1d40cec40 0x7ff1d40b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:14.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1db9f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1d4154220 0x7ff1d40b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:14.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1db9f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d40b3a10 con 0x7ff1d406b3d0 2023-12-19T18:16:14.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1db9f0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d40b2e00 con 0x7ff1d40cec40 2023-12-19T18:16:14.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1db9f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d40b30e0 con 0x7ff1d4154220 2023-12-19T18:16:14.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1d978c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d406b3d0 0x7ff1d414b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:14.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1d978c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d406b3d0 0x7ff1d414b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48432/0 (socket says 172.21.15.26:48432) 2023-12-19T18:16:14.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1d978c640 1 -- 172.21.15.26:0/2126187767 learned_addr learned my addr 172.21.15.26:0/2126187767 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:14.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1d8f8b640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1d40cec40 0x7ff1d40b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:14.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1d9f8d640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1d4154220 0x7ff1d40b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:14.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1d978c640 1 -- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1d4154220 msgr2=0x7ff1d40b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:14.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1d978c640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1d4154220 0x7ff1d40b27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1d978c640 1 -- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1d40cec40 msgr2=0x7ff1d40b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:14.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1d978c640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1d40cec40 0x7ff1d40b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1d978c640 1 -- 172.21.15.26:0/2126187767 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc009a70 con 0x7ff1d406b3d0 2023-12-19T18:16:14.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1d978c640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d406b3d0 0x7ff1d414b980 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000c500 tx=0x7ff1d000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:14.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1c9ffb640 1 -- 172.21.15.26:0/2126187767 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0005cf0 con 0x7ff1d406b3d0 2023-12-19T18:16:14.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1c9ffb640 1 -- 172.21.15.26:0/2126187767 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d0005e50 con 0x7ff1d406b3d0 2023-12-19T18:16:14.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1db9f0640 1 -- 172.21.15.26:0/2126187767 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d40b33c0 con 0x7ff1d406b3d0 2023-12-19T18:16:14.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1c9ffb640 1 -- 172.21.15.26:0/2126187767 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00072f0 con 0x7ff1d406b3d0 2023-12-19T18:16:14.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.074+0000 7ff1db9f0640 1 -- 172.21.15.26:0/2126187767 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d414bec0 con 0x7ff1d406b3d0 2023-12-19T18:16:14.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.078+0000 7ff1db9f0640 1 -- 172.21.15.26:0/2126187767 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d4074270 con 0x7ff1d406b3d0 2023-12-19T18:16:14.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.078+0000 7ff1c9ffb640 1 -- 172.21.15.26:0/2126187767 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff1d0007450 con 0x7ff1d406b3d0 2023-12-19T18:16:14.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.078+0000 7ff1c9ffb640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff1b8065b80 0x7ff1b8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:14.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.078+0000 7ff1c9ffb640 1 -- 172.21.15.26:0/2126187767 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6391+0+0 (secure 0 0 0) 0x7ff1d0082ab0 con 0x7ff1d406b3d0 2023-12-19T18:16:14.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.078+0000 7ff1d8f8b640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff1b8065b80 0x7ff1b8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:14.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.078+0000 7ff1d8f8b640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff1b8065b80 0x7ff1b8068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4164950 tx=0x7ff1c4008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:14.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.082+0000 7ff1c9ffb640 1 -- 172.21.15.26:0/2126187767 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff1d0050150 con 0x7ff1d406b3d0 2023-12-19T18:16:14.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.298+0000 7ff1db9f0640 1 -- 172.21.15.26:0/2126187767 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7ff1d40ce790 con 0x7ff1d406b3d0 2023-12-19T18:16:14.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.834+0000 7ff1c9ffb640 1 -- 172.21.15.26:0/2126187767 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3026) v1 ==== 163+0+0 (secure 0 0 0) 0x7ff1d0017020 con 0x7ff1d406b3d0 2023-12-19T18:16:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.838+0000 7ff1db9f0640 1 -- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff1b8065b80 msgr2=0x7ff1b8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.838+0000 7ff1db9f0640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff1b8065b80 0x7ff1b8068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4164950 tx=0x7ff1c4008040 comp rx=0 tx=0).stop 2023-12-19T18:16:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.838+0000 7ff1db9f0640 1 -- 172.21.15.26:0/2126187767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d406b3d0 msgr2=0x7ff1d414b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.838+0000 7ff1db9f0640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d406b3d0 0x7ff1d414b980 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000c500 tx=0x7ff1d000c9c0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.838+0000 7ff1db9f0640 1 -- 172.21.15.26:0/2126187767 shutdown_connections 2023-12-19T18:16:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.838+0000 7ff1db9f0640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff1b8065b80 0x7ff1b8068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.838+0000 7ff1db9f0640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1d4154220 0x7ff1d40b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.838+0000 7ff1db9f0640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1d40cec40 0x7ff1d40b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.838+0000 7ff1db9f0640 1 --2- 172.21.15.26:0/2126187767 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1d406b3d0 0x7ff1d414b980 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.838+0000 7ff1db9f0640 1 -- 172.21.15.26:0/2126187767 >> 172.21.15.26:0/2126187767 conn(0x7ff1d40658e0 msgr2=0x7ff1d414f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.838+0000 7ff1db9f0640 1 -- 172.21.15.26:0/2126187767 shutdown_connections 2023-12-19T18:16:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:14.838+0000 7ff1db9f0640 1 -- 172.21.15.26:0/2126187767 wait complete. 2023-12-19T18:16:14.862 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-12-19T18:16:14.863 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command succes with --fs argument for non-default fs 2023-12-19T18:16:14.863 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-12-19T18:16:14.863 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-12-19T18:16:15.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f64737b3640 1 Processor -- start 2023-12-19T18:16:15.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f64737b3640 1 -- start start 2023-12-19T18:16:15.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f64737b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f646c0b88e0 0x7f646c0b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:15.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f64737b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f646c0b4b40 0x7f646c0b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:15.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f64737b3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646c0b4f90 con 0x7f646c15cf60 2023-12-19T18:16:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f64737b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646c0d2960 con 0x7f646c0b88e0 2023-12-19T18:16:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f64737b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0d15f0 con 0x7f646c0b4b40 2023-12-19T18:16:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f647154f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f646c15cf60 0x7f646c0d0ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33122/0 (socket says 172.21.15.26:33122) 2023-12-19T18:16:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f647154f640 1 -- 172.21.15.26:0/3480211543 learned_addr learned my addr 172.21.15.26:0/3480211543 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f6470d4e640 1 --2- 172.21.15.26:0/3480211543 >> v2:172.21.15.26:3300/0 conn(0x7f646c0b88e0 0x7f646c0b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f6471d50640 1 --2- 172.21.15.26:0/3480211543 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f646c0b4b40 0x7f646c0b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f6470d4e640 1 -- 172.21.15.26:0/3480211543 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f646c0b4b40 msgr2=0x7f646c0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f6470d4e640 1 --2- 172.21.15.26:0/3480211543 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f646c0b4b40 0x7f646c0b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f6470d4e640 1 -- 172.21.15.26:0/3480211543 >> v1:172.21.15.26:6789/0 conn(0x7f646c15cf60 legacy=0x7f646c0d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:15.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.006+0000 7f6470d4e640 1 -- 172.21.15.26:0/3480211543 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f646c0d18c0 con 0x7f646c0b88e0 2023-12-19T18:16:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.010+0000 7f6470d4e640 1 --2- 172.21.15.26:0/3480211543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f646c0b88e0 0x7f646c0b4600 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f645c0097c0 tx=0x7f645c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481f54c6e171a71a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.010+0000 7f6463fff640 1 -- 172.21.15.26:0/3480211543 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c0a8970 con 0x7f646c0b88e0 2023-12-19T18:16:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.010+0000 7f6463fff640 1 -- 172.21.15.26:0/3480211543 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c0a8ad0 con 0x7f646c0b88e0 2023-12-19T18:16:15.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.010+0000 7f6463fff640 1 -- 172.21.15.26:0/3480211543 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c0a8e60 con 0x7f646c0b88e0 2023-12-19T18:16:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.010+0000 7f64737b3640 1 -- 172.21.15.26:0/3480211543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f646c0b88e0 msgr2=0x7f646c0b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.010+0000 7f64737b3640 1 --2- 172.21.15.26:0/3480211543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f646c0b88e0 0x7f646c0b4600 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f645c0097c0 tx=0x7f645c01ec70 comp rx=0 tx=0).stop 2023-12-19T18:16:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.010+0000 7f64737b3640 1 -- 172.21.15.26:0/3480211543 shutdown_connections 2023-12-19T18:16:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.010+0000 7f64737b3640 1 --2- 172.21.15.26:0/3480211543 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f646c0b4b40 0x7f646c0b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.010+0000 7f64737b3640 1 --2- 172.21.15.26:0/3480211543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f646c0b88e0 0x7f646c0b4600 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.010+0000 7f64737b3640 1 -- 172.21.15.26:0/3480211543 >> 172.21.15.26:0/3480211543 conn(0x7f646c065ee0 msgr2=0x7f646c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.010+0000 7f64737b3640 1 -- 172.21.15.26:0/3480211543 shutdown_connections 2023-12-19T18:16:15.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.010+0000 7f64737b3640 1 -- 172.21.15.26:0/3480211543 wait complete. 2023-12-19T18:16:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f64737b3640 1 Processor -- start 2023-12-19T18:16:15.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f64737b3640 1 -- start start 2023-12-19T18:16:15.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f64737b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f646c0b4b40 0x7f646c15e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:15.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f64737b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f646c0b88e0 0x7f646c0c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:15.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f64737b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f646c15cf60 0x7f646c151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f64737b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c15e5a0 con 0x7f646c0b4b40 2023-12-19T18:16:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f64737b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0c85d0 con 0x7f646c0b88e0 2023-12-19T18:16:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f64737b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646c0c90b0 con 0x7f646c15cf60 2023-12-19T18:16:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f647154f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f646c0b4b40 0x7f646c15e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f647154f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f646c0b4b40 0x7f646c15e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48448/0 (socket says 172.21.15.26:48448) 2023-12-19T18:16:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f647154f640 1 -- 172.21.15.26:0/1978495811 learned_addr learned my addr 172.21.15.26:0/1978495811 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f6470d4e640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f646c0b88e0 0x7f646c0c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f6471d50640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f646c15cf60 0x7f646c151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f647154f640 1 -- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f646c15cf60 msgr2=0x7f646c151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f647154f640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f646c15cf60 0x7f646c151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f647154f640 1 -- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f646c0b88e0 msgr2=0x7f646c0c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f647154f640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f646c0b88e0 0x7f646c0c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f647154f640 1 -- 172.21.15.26:0/1978495811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c009420 con 0x7f646c0b4b40 2023-12-19T18:16:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f6470d4e640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f646c0b88e0 0x7f646c0c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f6471d50640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f646c15cf60 0x7f646c151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f647154f640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f646c0b4b40 0x7f646c15e060 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f6468007fd0 tx=0x7f646800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:15.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f6461ffb640 1 -- 172.21.15.26:0/1978495811 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6468005cf0 con 0x7f646c0b4b40 2023-12-19T18:16:15.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f6461ffb640 1 -- 172.21.15.26:0/1978495811 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6468005e70 con 0x7f646c0b4b40 2023-12-19T18:16:15.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f6461ffb640 1 -- 172.21.15.26:0/1978495811 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64680072f0 con 0x7f646c0b4b40 2023-12-19T18:16:15.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f64737b3640 1 -- 172.21.15.26:0/1978495811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646c0c8810 con 0x7f646c0b4b40 2023-12-19T18:16:15.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.014+0000 7f64737b3640 1 -- 172.21.15.26:0/1978495811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f646c0c8dd0 con 0x7f646c0b4b40 2023-12-19T18:16:15.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.018+0000 7f64737b3640 1 -- 172.21.15.26:0/1978495811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646c074270 con 0x7f646c0b4b40 2023-12-19T18:16:15.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.018+0000 7f6461ffb640 1 -- 172.21.15.26:0/1978495811 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6468007450 con 0x7f646c0b4b40 2023-12-19T18:16:15.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.018+0000 7f6461ffb640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6450065bb0 0x7f6450068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:15.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.018+0000 7f6461ffb640 1 -- 172.21.15.26:0/1978495811 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f6468082930 con 0x7f646c0b4b40 2023-12-19T18:16:15.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.018+0000 7f6470d4e640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6450065bb0 0x7f6450068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:15.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.018+0000 7f6470d4e640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6450065bb0 0x7f6450068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f645c0b2000 tx=0x7f645c008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:15.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.022+0000 7f6461ffb640 1 -- 172.21.15.26:0/1978495811 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f646804ff20 con 0x7f646c0b4b40 2023-12-19T18:16:15.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.234+0000 7f64737b3640 1 -- 172.21.15.26:0/1978495811 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7f646c0b4f90 con 0x7f646c0b4b40 2023-12-19T18:16:15.845 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.842+0000 7f6461ffb640 1 -- 172.21.15.26:0/1978495811 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3027) v1 ==== 189+0+0 (secure 0 0 0) 0x7f646c0b4f90 con 0x7f646c0b4b40 2023-12-19T18:16:15.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.846+0000 7f64737b3640 1 -- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6450065bb0 msgr2=0x7f6450068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.846+0000 7f64737b3640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6450065bb0 0x7f6450068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f645c0b2000 tx=0x7f645c008040 comp rx=0 tx=0).stop 2023-12-19T18:16:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.846+0000 7f64737b3640 1 -- 172.21.15.26:0/1978495811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f646c0b4b40 msgr2=0x7f646c15e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.846+0000 7f64737b3640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f646c0b4b40 0x7f646c15e060 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f6468007fd0 tx=0x7f646800c890 comp rx=0 tx=0).stop 2023-12-19T18:16:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.846+0000 7f64737b3640 1 -- 172.21.15.26:0/1978495811 shutdown_connections 2023-12-19T18:16:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.846+0000 7f64737b3640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6450065bb0 0x7f6450068070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.846+0000 7f64737b3640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f646c15cf60 0x7f646c151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.846+0000 7f64737b3640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f646c0b88e0 0x7f646c0c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.846+0000 7f64737b3640 1 --2- 172.21.15.26:0/1978495811 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f646c0b4b40 0x7f646c15e060 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.846+0000 7f64737b3640 1 -- 172.21.15.26:0/1978495811 >> 172.21.15.26:0/1978495811 conn(0x7f646c065ee0 msgr2=0x7f646c0b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.846+0000 7f64737b3640 1 -- 172.21.15.26:0/1978495811 shutdown_connections 2023-12-19T18:16:15.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:15.846+0000 7f64737b3640 1 -- 172.21.15.26:0/1978495811 wait complete. 2023-12-19T18:16:15.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:16:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.018+0000 7f3573b39640 1 Processor -- start 2023-12-19T18:16:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.018+0000 7f3573b39640 1 -- start start 2023-12-19T18:16:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.018+0000 7f3573b39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f356c06c3f0 0x7f356c06c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.018+0000 7f3573b39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f356c0c8df0 0x7f356c0c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.018+0000 7f3573b39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356c0b01b0 con 0x7f356c161230 2023-12-19T18:16:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.018+0000 7f3573b39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356c0ca4e0 con 0x7f356c06c3f0 2023-12-19T18:16:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.018+0000 7f3573b39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0b0400 con 0x7f356c0c8df0 2023-12-19T18:16:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.022+0000 7f35710d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f356c06c3f0 0x7f356c06c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.022+0000 7f35710d4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f356c06c3f0 0x7f356c06c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44362/0 (socket says 172.21.15.26:44362) 2023-12-19T18:16:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.022+0000 7f35710d4640 1 -- 172.21.15.26:0/32549163 learned_addr learned my addr 172.21.15.26:0/32549163 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.022+0000 7f35720d6640 1 --2- 172.21.15.26:0/32549163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f356c0c8df0 0x7f356c0c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.022+0000 7f35710d4640 1 -- 172.21.15.26:0/32549163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f356c0c8df0 msgr2=0x7f356c0c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.022+0000 7f35710d4640 1 --2- 172.21.15.26:0/32549163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f356c0c8df0 0x7f356c0c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.022+0000 7f35710d4640 1 -- 172.21.15.26:0/32549163 >> v1:172.21.15.160:6789/0 conn(0x7f356c161230 legacy=0x7f356c0ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.022+0000 7f35710d4640 1 -- 172.21.15.26:0/32549163 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c0b06e0 con 0x7f356c06c3f0 2023-12-19T18:16:16.026 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.022+0000 7f35710d4640 1 --2- 172.21.15.26:0/32549163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c06c3f0 0x7f356c06c850 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f355c009870 tx=0x7f355c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba3fb7987168daf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:16.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.022+0000 7f35708d3640 1 -- 172.21.15.26:0/32549163 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c0a6040 con 0x7f356c06c3f0 2023-12-19T18:16:16.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.022+0000 7f35708d3640 1 -- 172.21.15.26:0/32549163 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c0aa600 con 0x7f356c06c3f0 2023-12-19T18:16:16.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.022+0000 7f35708d3640 1 -- 172.21.15.26:0/32549163 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c0aa990 con 0x7f356c06c3f0 2023-12-19T18:16:16.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 -- 172.21.15.26:0/32549163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c06c3f0 msgr2=0x7f356c06c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 --2- 172.21.15.26:0/32549163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c06c3f0 0x7f356c06c850 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f355c009870 tx=0x7f355c01c480 comp rx=0 tx=0).stop 2023-12-19T18:16:16.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 -- 172.21.15.26:0/32549163 shutdown_connections 2023-12-19T18:16:16.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 --2- 172.21.15.26:0/32549163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f356c0c8df0 0x7f356c0c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 --2- 172.21.15.26:0/32549163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c06c3f0 0x7f356c06c850 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 -- 172.21.15.26:0/32549163 >> 172.21.15.26:0/32549163 conn(0x7f356c0ce730 msgr2=0x7f356c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:16.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 -- 172.21.15.26:0/32549163 shutdown_connections 2023-12-19T18:16:16.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 -- 172.21.15.26:0/32549163 wait complete. 2023-12-19T18:16:16.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 Processor -- start 2023-12-19T18:16:16.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 -- start start 2023-12-19T18:16:16.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c06c3f0 0x7f356c0af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f356c0c8df0 0x7f356c0c4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f356c161230 0x7f356c153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0c4350 con 0x7f356c06c3f0 2023-12-19T18:16:16.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0c3760 con 0x7f356c161230 2023-12-19T18:16:16.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f3573b39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356c1539e0 con 0x7f356c0c8df0 2023-12-19T18:16:16.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f35718d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c06c3f0 0x7f356c0af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f35718d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c06c3f0 0x7f356c0af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44378/0 (socket says 172.21.15.26:44378) 2023-12-19T18:16:16.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f35718d5640 1 -- 172.21.15.26:0/4236823030 learned_addr learned my addr 172.21.15.26:0/4236823030 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:16.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f35710d4640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f356c0c8df0 0x7f356c0c4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f35720d6640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f356c161230 0x7f356c153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f35718d5640 1 -- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f356c0c8df0 msgr2=0x7f356c0c4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f35718d5640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f356c0c8df0 0x7f356c0c4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f35718d5640 1 -- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f356c161230 msgr2=0x7f356c153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f35718d5640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f356c161230 0x7f356c153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f35718d5640 1 -- 172.21.15.26:0/4236823030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c0094d0 con 0x7f356c06c3f0 2023-12-19T18:16:16.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.026+0000 7f35710d4640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f356c0c8df0 0x7f356c0c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:16.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.030+0000 7f35718d5640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c06c3f0 0x7f356c0af7f0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f356800c530 tx=0x7f356800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:16.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.030+0000 7f35627fc640 1 -- 172.21.15.26:0/4236823030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568005220 con 0x7f356c06c3f0 2023-12-19T18:16:16.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.030+0000 7f35627fc640 1 -- 172.21.15.26:0/4236823030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3568005380 con 0x7f356c06c3f0 2023-12-19T18:16:16.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.030+0000 7f35627fc640 1 -- 172.21.15.26:0/4236823030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568007270 con 0x7f356c06c3f0 2023-12-19T18:16:16.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.030+0000 7f3573b39640 1 -- 172.21.15.26:0/4236823030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356c0c39b0 con 0x7f356c06c3f0 2023-12-19T18:16:16.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.030+0000 7f3573b39640 1 -- 172.21.15.26:0/4236823030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356c0c3f70 con 0x7f356c06c3f0 2023-12-19T18:16:16.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.030+0000 7f3573b39640 1 -- 172.21.15.26:0/4236823030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356c074270 con 0x7f356c06c3f0 2023-12-19T18:16:16.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.030+0000 7f35627fc640 1 -- 172.21.15.26:0/4236823030 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f35680073d0 con 0x7f356c06c3f0 2023-12-19T18:16:16.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.030+0000 7f35627fc640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3550065b80 0x7f3550068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.030+0000 7f35627fc640 1 -- 172.21.15.26:0/4236823030 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f3568082ee0 con 0x7f356c06c3f0 2023-12-19T18:16:16.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.034+0000 7f35710d4640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3550065b80 0x7f3550068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.034+0000 7f35710d4640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3550065b80 0x7f3550068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f355c0ae000 tx=0x7f355c0a7000 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:16.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.034+0000 7f35627fc640 1 -- 172.21.15.26:0/4236823030 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f35680504d0 con 0x7f356c06c3f0 2023-12-19T18:16:16.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.190+0000 7f3573b39640 1 -- 172.21.15.26:0/4236823030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f356c0b46b0 con 0x7f356c06c3f0 2023-12-19T18:16:16.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.198+0000 7f35627fc640 1 -- 172.21.15.26:0/4236823030 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v70)=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3568053af0 con 0x7f356c06c3f0 2023-12-19T18:16:16.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.198+0000 7f3573b39640 1 -- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3550065b80 msgr2=0x7f3550068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.198+0000 7f3573b39640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3550065b80 0x7f3550068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f355c0ae000 tx=0x7f355c0a7000 comp rx=0 tx=0).stop 2023-12-19T18:16:16.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.198+0000 7f3573b39640 1 -- 172.21.15.26:0/4236823030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c06c3f0 msgr2=0x7f356c0af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.198+0000 7f3573b39640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c06c3f0 0x7f356c0af7f0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f356800c530 tx=0x7f356800c9f0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.198+0000 7f3573b39640 1 -- 172.21.15.26:0/4236823030 shutdown_connections 2023-12-19T18:16:16.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.198+0000 7f3573b39640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3550065b80 0x7f3550068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.198+0000 7f3573b39640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f356c161230 0x7f356c153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.198+0000 7f3573b39640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f356c0c8df0 0x7f356c0c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.198+0000 7f3573b39640 1 --2- 172.21.15.26:0/4236823030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f356c06c3f0 0x7f356c0af7f0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.198+0000 7f3573b39640 1 -- 172.21.15.26:0/4236823030 >> 172.21.15.26:0/4236823030 conn(0x7f356c0ce730 msgr2=0x7f356c0cb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.198+0000 7f3573b39640 1 -- 172.21.15.26:0/4236823030 shutdown_connections 2023-12-19T18:16:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.198+0000 7f3573b39640 1 -- 172.21.15.26:0/4236823030 wait complete. 2023-12-19T18:16:16.222 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:16:16.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:16:16.321 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:16:16.418 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:16:16.418 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:16:16.419 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:16.428 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:16:16.428 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:16:16.428 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:16.438 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:16:16.439 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:16:16.439 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:16.483 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:16:16.484 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:16:16.484 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:16.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:16:16.493 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:16:16.493 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.666+0000 7f60972e5640 1 Processor -- start 2023-12-19T18:16:16.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.666+0000 7f60972e5640 1 -- start start 2023-12-19T18:16:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.670+0000 7f60972e5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f60900cb1d0 0x7f60900cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.670+0000 7f60972e5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f60900b1d40 0x7f60900b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.670+0000 7f60972e5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609006d4b0 con 0x7f609006bee0 2023-12-19T18:16:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.670+0000 7f60972e5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60900cc030 con 0x7f60900cb1d0 2023-12-19T18:16:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.670+0000 7f60972e5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60900cbaf0 con 0x7f60900b1d40 2023-12-19T18:16:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.670+0000 7f6095081640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f60900cb1d0 0x7f60900cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.670+0000 7f6095081640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f60900cb1d0 0x7f60900cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44388/0 (socket says 172.21.15.26:44388) 2023-12-19T18:16:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.670+0000 7f6095081640 1 -- 172.21.15.26:0/4024344226 learned_addr learned my addr 172.21.15.26:0/4024344226 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.670+0000 7f6095882640 1 --2- 172.21.15.26:0/4024344226 >> v2:172.21.15.160:3300/0 conn(0x7f60900b1d40 0x7f60900b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.670+0000 7f6095081640 1 -- 172.21.15.26:0/4024344226 >> v2:172.21.15.160:3300/0 conn(0x7f60900b1d40 msgr2=0x7f60900b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.670+0000 7f6095081640 1 --2- 172.21.15.26:0/4024344226 >> v2:172.21.15.160:3300/0 conn(0x7f60900b1d40 0x7f60900b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.670+0000 7f6095081640 1 -- 172.21.15.26:0/4024344226 >> v1:172.21.15.160:6789/0 conn(0x7f609006bee0 legacy=0x7f609006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:16.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.670+0000 7f6095081640 1 -- 172.21.15.26:0/4024344226 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609006cc10 con 0x7f60900cb1d0 2023-12-19T18:16:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f6095081640 1 --2- 172.21.15.26:0/4024344226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60900cb1d0 0x7f60900cb5b0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f60900cca50 tx=0x7f608c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12cae8f9441ad80 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f6087fff640 1 -- 172.21.15.26:0/4024344226 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c0a7070 con 0x7f60900cb1d0 2023-12-19T18:16:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f6087fff640 1 -- 172.21.15.26:0/4024344226 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608c0ab430 con 0x7f60900cb1d0 2023-12-19T18:16:16.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f6087fff640 1 -- 172.21.15.26:0/4024344226 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c0ab7c0 con 0x7f60900cb1d0 2023-12-19T18:16:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 -- 172.21.15.26:0/4024344226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60900cb1d0 msgr2=0x7f60900cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 --2- 172.21.15.26:0/4024344226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60900cb1d0 0x7f60900cb5b0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f60900cca50 tx=0x7f608c01c430 comp rx=0 tx=0).stop 2023-12-19T18:16:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 -- 172.21.15.26:0/4024344226 shutdown_connections 2023-12-19T18:16:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 --2- 172.21.15.26:0/4024344226 >> v2:172.21.15.160:3300/0 conn(0x7f60900b1d40 0x7f60900b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 --2- 172.21.15.26:0/4024344226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60900cb1d0 0x7f60900cb5b0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 -- 172.21.15.26:0/4024344226 >> 172.21.15.26:0/4024344226 conn(0x7f60900658e0 msgr2=0x7f6090065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 -- 172.21.15.26:0/4024344226 shutdown_connections 2023-12-19T18:16:16.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 -- 172.21.15.26:0/4024344226 wait complete. 2023-12-19T18:16:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 Processor -- start 2023-12-19T18:16:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 -- start start 2023-12-19T18:16:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f609006bee0 0x7f60901586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60900b1d40 0x7f609014db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f60900cb1d0 0x7f60900703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609014f350 con 0x7f60900b1d40 2023-12-19T18:16:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609014f7e0 con 0x7f609006bee0 2023-12-19T18:16:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.674+0000 7f60972e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609014fc90 con 0x7f60900cb1d0 2023-12-19T18:16:16.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6094880640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60900b1d40 0x7f609014db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6094880640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60900b1d40 0x7f609014db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44394/0 (socket says 172.21.15.26:44394) 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6094880640 1 -- 172.21.15.26:0/903589095 learned_addr learned my addr 172.21.15.26:0/903589095 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6095081640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f609006bee0 0x7f60901586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6095882640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f60900cb1d0 0x7f60900703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6094880640 1 -- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f60900cb1d0 msgr2=0x7f60900703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6094880640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f60900cb1d0 0x7f60900703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6094880640 1 -- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f609006bee0 msgr2=0x7f60901586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6094880640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f609006bee0 0x7f60901586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6094880640 1 -- 172.21.15.26:0/903589095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c0094d0 con 0x7f60900b1d40 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6094880640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60900b1d40 0x7f609014db50 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f608000b440 tx=0x7f608000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6085ffb640 1 -- 172.21.15.26:0/903589095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080011890 con 0x7f60900b1d40 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6085ffb640 1 -- 172.21.15.26:0/903589095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60800119f0 con 0x7f60900b1d40 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f60972e5640 1 -- 172.21.15.26:0/903589095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609015a6b0 con 0x7f60900b1d40 2023-12-19T18:16:16.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6085ffb640 1 -- 172.21.15.26:0/903589095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080011da0 con 0x7f60900b1d40 2023-12-19T18:16:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f60972e5640 1 -- 172.21.15.26:0/903589095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609014efe0 con 0x7f60900b1d40 2023-12-19T18:16:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f60972e5640 1 -- 172.21.15.26:0/903589095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609014ee40 con 0x7f60900b1d40 2023-12-19T18:16:16.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6085ffb640 1 -- 172.21.15.26:0/903589095 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f608001a400 con 0x7f60900b1d40 2023-12-19T18:16:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.678+0000 7f6085ffb640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6074065b80 0x7f6074068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.682+0000 7f6085ffb640 1 -- 172.21.15.26:0/903589095 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f6080083540 con 0x7f60900b1d40 2023-12-19T18:16:16.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.682+0000 7f6095081640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6074065b80 0x7f6074068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.682+0000 7f6095081640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6074065b80 0x7f6074068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f608c0af000 tx=0x7f608c008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:16.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.682+0000 7f6085ffb640 1 -- 172.21.15.26:0/903589095 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f609014ee40 con 0x7f60900b1d40 2023-12-19T18:16:16.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.874+0000 7f60972e5640 1 -- 172.21.15.26:0/903589095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f609014f500 con 0x7f60900b1d40 2023-12-19T18:16:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.874+0000 7f6085ffb640 1 -- 172.21.15.26:0/903589095 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+3534 (secure 0 0 0) 0x7f6080015de0 con 0x7f60900b1d40 2023-12-19T18:16:16.878 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:16.878 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":89,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/3922467901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3922467901},{"type":"v1","addr":"172.21.15.160:6835","nonce":3922467901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":9562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3692512090","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":3692512090},{"type":"v1","addr":"172.21.15.160:6837","nonce":3692512090}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":10083,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3961033710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3961033710},{"type":"v1","addr":"172.21.15.26:6833","nonce":3961033710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":83},{"gid":10524,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/213893620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":213893620},{"type":"v1","addr":"172.21.15.26:6835","nonce":213893620}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89}],"filesystems":[]} 2023-12-19T18:16:16.878 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 89 2023-12-19T18:16:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.878+0000 7f60972e5640 1 -- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6074065b80 msgr2=0x7f6074068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.878+0000 7f60972e5640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6074065b80 0x7f6074068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f608c0af000 tx=0x7f608c008040 comp rx=0 tx=0).stop 2023-12-19T18:16:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.878+0000 7f60972e5640 1 -- 172.21.15.26:0/903589095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60900b1d40 msgr2=0x7f609014db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.878+0000 7f60972e5640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60900b1d40 0x7f609014db50 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f608000b440 tx=0x7f608000b900 comp rx=0 tx=0).stop 2023-12-19T18:16:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.878+0000 7f60972e5640 1 -- 172.21.15.26:0/903589095 shutdown_connections 2023-12-19T18:16:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.878+0000 7f60972e5640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6074065b80 0x7f6074068040 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.878+0000 7f60972e5640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f60900cb1d0 0x7f60900703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.878+0000 7f60972e5640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60900b1d40 0x7f609014db50 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.878+0000 7f60972e5640 1 --2- 172.21.15.26:0/903589095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f609006bee0 0x7f60901586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.878+0000 7f60972e5640 1 -- 172.21.15.26:0/903589095 >> 172.21.15.26:0/903589095 conn(0x7f60900658e0 msgr2=0x7f6090163f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.878+0000 7f60972e5640 1 -- 172.21.15.26:0/903589095 shutdown_connections 2023-12-19T18:16:16.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:16.878+0000 7f60972e5640 1 -- 172.21.15.26:0/903589095 wait complete. 2023-12-19T18:16:16.900 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:16:16.900 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:16:16.900 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:16:16.901 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:16:16.901 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:16:16.901 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:16:16.901 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:16:16.901 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:16:16.901 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:16:16.901 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:16:16.902 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:16:16.902 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:16:16.902 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:16:16.902 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:16:16.902 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:16:16.902 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:16:16.916 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:16:16.916 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:16:16.920 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:16:16.920 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:16:16.921 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:16:16.921 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:16:16.962 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:16:16.962 DEBUG:teuthology.parallel:result is None 2023-12-19T18:16:16.966 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:16:16.967 DEBUG:teuthology.parallel:result is None 2023-12-19T18:16:16.968 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:16:16.969 DEBUG:teuthology.parallel:result is None 2023-12-19T18:16:17.048 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T18:16:17.063 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T18:16:17.063 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T18:16:18.985 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-12-19T18:16:22.916 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:16:22.916 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:16:22.921 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:16:22.921 DEBUG:teuthology.parallel:result is None 2023-12-19T18:16:22.921 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:16:23.017 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T18:16:23.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.078+0000 7f52e558e640 1 Processor -- start 2023-12-19T18:16:23.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.078+0000 7f52e558e640 1 -- start start 2023-12-19T18:16:23.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.078+0000 7f52e558e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52e00cc0a0 0x7f52e00cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.078+0000 7f52e558e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e00adde0 con 0x7f52e0159d20 2023-12-19T18:16:23.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.078+0000 7f52e558e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e00b28d0 con 0x7f52e00cdee0 2023-12-19T18:16:23.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.078+0000 7f52e558e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e00c9c20 con 0x7f52e00cc0a0 2023-12-19T18:16:23.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.078+0000 7f52dffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52e00cc0a0 0x7f52e00cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.078+0000 7f52df7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f52e0159d20 0x7f52e00cc9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34746/0 (socket says 172.21.15.26:34746) 2023-12-19T18:16:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.078+0000 7f52df7fe640 1 -- 172.21.15.26:0/3140614741 learned_addr learned my addr 172.21.15.26:0/3140614741 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.078+0000 7f52dffff640 1 -- 172.21.15.26:0/3140614741 >> v1:172.21.15.160:6789/0 conn(0x7f52e00cdee0 legacy=0x7f52e00b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.078+0000 7f52dffff640 1 -- 172.21.15.26:0/3140614741 >> v1:172.21.15.26:6789/0 conn(0x7f52e0159d20 legacy=0x7f52e00cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.078+0000 7f52dffff640 1 -- 172.21.15.26:0/3140614741 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e00adb00 con 0x7f52e00cc0a0 2023-12-19T18:16:23.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.078+0000 7f52deffd640 1 -- 172.21.15.26:0/3140614741 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347969601 0 0) 0x7f52e00adde0 con 0x7f52e0159d20 2023-12-19T18:16:23.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.082+0000 7f52dffff640 1 --2- 172.21.15.26:0/3140614741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e00cc0a0 0x7f52e00cc480 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f52d00097c0 tx=0x7f52d001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efb94db51e71409 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.082+0000 7f52deffd640 1 -- 172.21.15.26:0/3140614741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00ad680 con 0x7f52e00cc0a0 2023-12-19T18:16:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.082+0000 7f52deffd640 1 -- 172.21.15.26:0/3140614741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d00ad7e0 con 0x7f52e00cc0a0 2023-12-19T18:16:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.082+0000 7f52deffd640 1 -- 172.21.15.26:0/3140614741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d00adb70 con 0x7f52e00cc0a0 2023-12-19T18:16:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.082+0000 7f52e558e640 1 -- 172.21.15.26:0/3140614741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e00cc0a0 msgr2=0x7f52e00cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.082+0000 7f52e558e640 1 --2- 172.21.15.26:0/3140614741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e00cc0a0 0x7f52e00cc480 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f52d00097c0 tx=0x7f52d001c380 comp rx=0 tx=0).stop 2023-12-19T18:16:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.082+0000 7f52e558e640 1 -- 172.21.15.26:0/3140614741 shutdown_connections 2023-12-19T18:16:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.082+0000 7f52e558e640 1 --2- 172.21.15.26:0/3140614741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e00cc0a0 0x7f52e00cc480 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.082+0000 7f52e558e640 1 -- 172.21.15.26:0/3140614741 >> 172.21.15.26:0/3140614741 conn(0x7f52e006b0f0 msgr2=0x7f52e006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.082+0000 7f52e558e640 1 -- 172.21.15.26:0/3140614741 shutdown_connections 2023-12-19T18:16:23.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.082+0000 7f52e558e640 1 -- 172.21.15.26:0/3140614741 wait complete. 2023-12-19T18:16:23.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52e558e640 1 Processor -- start 2023-12-19T18:16:23.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52e558e640 1 -- start start 2023-12-19T18:16:23.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52e558e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52e00cc0a0 0x7f52e0150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52e558e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e00cdee0 0x7f52e01507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52e558e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52e0159d20 0x7f52e00c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52e558e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e0150d10 con 0x7f52e00cdee0 2023-12-19T18:16:23.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52e558e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e01f87b0 con 0x7f52e0159d20 2023-12-19T18:16:23.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52e558e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e01f84b0 con 0x7f52e00cc0a0 2023-12-19T18:16:23.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52df7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e00cdee0 0x7f52e01507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52df7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e00cdee0 0x7f52e01507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44448/0 (socket says 172.21.15.26:44448) 2023-12-19T18:16:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52df7fe640 1 -- 172.21.15.26:0/3667306387 learned_addr learned my addr 172.21.15.26:0/3667306387 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52dffff640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52e00cc0a0 0x7f52e0150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52e4d8d640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52e0159d20 0x7f52e00c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52df7fe640 1 -- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52e00cc0a0 msgr2=0x7f52e0150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52df7fe640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52e00cc0a0 0x7f52e0150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52df7fe640 1 -- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52e0159d20 msgr2=0x7f52e00c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52df7fe640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52e0159d20 0x7f52e00c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52df7fe640 1 -- 172.21.15.26:0/3667306387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d0009420 con 0x7f52e00cdee0 2023-12-19T18:16:23.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52df7fe640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e00cdee0 0x7f52e01507d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f52d400c4b0 tx=0x7f52d400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52dcff9640 1 -- 172.21.15.26:0/3667306387 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40074e0 con 0x7f52e00cdee0 2023-12-19T18:16:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52dcff9640 1 -- 172.21.15.26:0/3667306387 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d4007640 con 0x7f52e00cdee0 2023-12-19T18:16:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52dcff9640 1 -- 172.21.15.26:0/3667306387 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40049f0 con 0x7f52e00cdee0 2023-12-19T18:16:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.086+0000 7f52e558e640 1 -- 172.21.15.26:0/3667306387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e00c5740 con 0x7f52e00cdee0 2023-12-19T18:16:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.090+0000 7f52e558e640 1 -- 172.21.15.26:0/3667306387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e00c59e0 con 0x7f52e00cdee0 2023-12-19T18:16:23.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.090+0000 7f52e558e640 1 -- 172.21.15.26:0/3667306387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e0067400 con 0x7f52e00cdee0 2023-12-19T18:16:23.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.090+0000 7f52dcff9640 1 -- 172.21.15.26:0/3667306387 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f52d4004b50 con 0x7f52e00cdee0 2023-12-19T18:16:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.090+0000 7f52dcff9640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52cc065bb0 0x7f52cc068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.090+0000 7f52dcff9640 1 -- 172.21.15.26:0/3667306387 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f52d40827f0 con 0x7f52e00cdee0 2023-12-19T18:16:23.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.090+0000 7f52dffff640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52cc065bb0 0x7f52cc068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.094+0000 7f52dffff640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52cc065bb0 0x7f52cc068070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f52d00af000 tx=0x7f52d00a7000 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:23.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.094+0000 7f52dcff9640 1 -- 172.21.15.26:0/3667306387 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f52d404fd50 con 0x7f52e00cdee0 2023-12-19T18:16:23.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:23.266+0000 7f52e558e640 1 -- 172.21.15.26:0/3667306387 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f52e01f88f0 con 0x7f52e00cdee0 2023-12-19T18:16:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.038+0000 7f52dcff9640 1 -- 172.21.15.26:0/3667306387 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v208) v1 ==== 97+0+0 (secure 0 0 0) 0x7f52d4018080 con 0x7f52e00cdee0 2023-12-19T18:16:24.042 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2023-12-19T18:16:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.042+0000 7f52e558e640 1 -- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52cc065bb0 msgr2=0x7f52cc068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.042+0000 7f52e558e640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52cc065bb0 0x7f52cc068070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f52d00af000 tx=0x7f52d00a7000 comp rx=0 tx=0).stop 2023-12-19T18:16:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.042+0000 7f52e558e640 1 -- 172.21.15.26:0/3667306387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e00cdee0 msgr2=0x7f52e01507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.042+0000 7f52e558e640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e00cdee0 0x7f52e01507d0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f52d400c4b0 tx=0x7f52d400c970 comp rx=0 tx=0).stop 2023-12-19T18:16:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.042+0000 7f52e558e640 1 -- 172.21.15.26:0/3667306387 shutdown_connections 2023-12-19T18:16:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.042+0000 7f52e558e640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52cc065bb0 0x7f52cc068070 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.042+0000 7f52e558e640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52e0159d20 0x7f52e00c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.042+0000 7f52e558e640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52e00cdee0 0x7f52e01507d0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.042+0000 7f52e558e640 1 --2- 172.21.15.26:0/3667306387 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52e00cc0a0 0x7f52e0150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.042+0000 7f52e558e640 1 -- 172.21.15.26:0/3667306387 >> 172.21.15.26:0/3667306387 conn(0x7f52e006b0f0 msgr2=0x7f52e00718e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.042+0000 7f52e558e640 1 -- 172.21.15.26:0/3667306387 shutdown_connections 2023-12-19T18:16:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.042+0000 7f52e558e640 1 -- 172.21.15.26:0/3667306387 wait complete. 2023-12-19T18:16:24.067 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:16:24.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.210+0000 7f483e62d640 1 Processor -- start 2023-12-19T18:16:24.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.210+0000 7f483e62d640 1 -- start start 2023-12-19T18:16:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f483e62d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f483806b3d0 0x7f48380ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f483e62d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48380cec40 0x7f483815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f483e62d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483806bc30 con 0x7f4838154220 2023-12-19T18:16:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f483e62d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48380cf0d0 con 0x7f483806b3d0 2023-12-19T18:16:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f483e62d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48380cc6e0 con 0x7f48380cec40 2023-12-19T18:16:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f4837fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4838154220 0x7f483815aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34748/0 (socket says 172.21.15.26:34748) 2023-12-19T18:16:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f4837fff640 1 -- 172.21.15.26:0/1526757610 learned_addr learned my addr 172.21.15.26:0/1526757610 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f483cbca640 1 --2- 172.21.15.26:0/1526757610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48380cec40 0x7f483815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f48377fe640 1 --2- 172.21.15.26:0/1526757610 >> v2:172.21.15.160:3300/0 conn(0x7f483806b3d0 0x7f48380ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f4836ffd640 1 -- 172.21.15.26:0/1526757610 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789195304 0 0) 0x7f483806bc30 con 0x7f4838154220 2023-12-19T18:16:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f4836ffd640 1 -- 172.21.15.26:0/1526757610 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4820003530 con 0x7f4838154220 2023-12-19T18:16:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f483cbca640 1 -- 172.21.15.26:0/1526757610 >> v2:172.21.15.160:3300/0 conn(0x7f483806b3d0 msgr2=0x7f48380ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f483cbca640 1 --2- 172.21.15.26:0/1526757610 >> v2:172.21.15.160:3300/0 conn(0x7f483806b3d0 0x7f48380ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f483cbca640 1 -- 172.21.15.26:0/1526757610 >> v1:172.21.15.26:6789/0 conn(0x7f4838154220 legacy=0x7f483815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f483cbca640 1 -- 172.21.15.26:0/1526757610 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483806b950 con 0x7f48380cec40 2023-12-19T18:16:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.214+0000 7f48377fe640 1 --2- 172.21.15.26:0/1526757610 >> v2:172.21.15.160:3300/0 conn(0x7f483806b3d0 0x7f48380ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f483cbca640 1 --2- 172.21.15.26:0/1526757610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48380cec40 0x7f483815b1c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4830009d90 tx=0x7f483001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62dae08214291939 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f4836ffd640 1 -- 172.21.15.26:0/1526757610 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300a59e0 con 0x7f48380cec40 2023-12-19T18:16:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f4836ffd640 1 -- 172.21.15.26:0/1526757610 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48300a5b40 con 0x7f48380cec40 2023-12-19T18:16:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f4836ffd640 1 -- 172.21.15.26:0/1526757610 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300ad440 con 0x7f48380cec40 2023-12-19T18:16:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f483e62d640 1 -- 172.21.15.26:0/1526757610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48380cec40 msgr2=0x7f483815b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f483e62d640 1 --2- 172.21.15.26:0/1526757610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48380cec40 0x7f483815b1c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4830009d90 tx=0x7f483001c920 comp rx=0 tx=0).stop 2023-12-19T18:16:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f483e62d640 1 -- 172.21.15.26:0/1526757610 shutdown_connections 2023-12-19T18:16:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f483e62d640 1 --2- 172.21.15.26:0/1526757610 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48380cec40 0x7f483815b1c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f483e62d640 1 --2- 172.21.15.26:0/1526757610 >> v2:172.21.15.160:3300/0 conn(0x7f483806b3d0 0x7f48380ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f483e62d640 1 -- 172.21.15.26:0/1526757610 >> 172.21.15.26:0/1526757610 conn(0x7f48380658e0 msgr2=0x7f4838065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f483e62d640 1 -- 172.21.15.26:0/1526757610 shutdown_connections 2023-12-19T18:16:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f483e62d640 1 -- 172.21.15.26:0/1526757610 wait complete. 2023-12-19T18:16:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f483e62d640 1 Processor -- start 2023-12-19T18:16:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.218+0000 7f483e62d640 1 -- start start 2023-12-19T18:16:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f483e62d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f483806b3d0 0x7f4838143e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f483e62d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48380cec40 0x7f48381432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f483e62d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4838154220 0x7f4838157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f483e62d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4838166ca0 con 0x7f48380cec40 2023-12-19T18:16:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f483e62d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4838166990 con 0x7f483806b3d0 2023-12-19T18:16:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f483e62d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4838144640 con 0x7f4838154220 2023-12-19T18:16:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f48377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48380cec40 0x7f48381432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f48377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48380cec40 0x7f48381432b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44462/0 (socket says 172.21.15.26:44462) 2023-12-19T18:16:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f48377fe640 1 -- 172.21.15.26:0/3234347329 learned_addr learned my addr 172.21.15.26:0/3234347329 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f4837fff640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f483806b3d0 0x7f4838143e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f483cbca640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4838154220 0x7f4838157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f48377fe640 1 -- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4838154220 msgr2=0x7f4838157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f48377fe640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4838154220 0x7f4838157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f48377fe640 1 -- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f483806b3d0 msgr2=0x7f4838143e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f48377fe640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f483806b3d0 0x7f4838143e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f48377fe640 1 -- 172.21.15.26:0/3234347329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c009420 con 0x7f48380cec40 2023-12-19T18:16:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f48377fe640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48380cec40 0x7f48381432b0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f482c00e980 tx=0x7f482c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f4834ff9640 1 -- 172.21.15.26:0/3234347329 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c00ca00 con 0x7f48380cec40 2023-12-19T18:16:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f4834ff9640 1 -- 172.21.15.26:0/3234347329 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c00cb60 con 0x7f48380cec40 2023-12-19T18:16:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f4834ff9640 1 -- 172.21.15.26:0/3234347329 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c013470 con 0x7f48380cec40 2023-12-19T18:16:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f483e62d640 1 -- 172.21.15.26:0/3234347329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4830009a70 con 0x7f48380cec40 2023-12-19T18:16:24.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f483e62d640 1 -- 172.21.15.26:0/3234347329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48380afd40 con 0x7f48380cec40 2023-12-19T18:16:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f483e62d640 1 -- 172.21.15.26:0/3234347329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4838074270 con 0x7f48380cec40 2023-12-19T18:16:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.222+0000 7f4834ff9640 1 -- 172.21.15.26:0/3234347329 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f482c0135d0 con 0x7f48380cec40 2023-12-19T18:16:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.226+0000 7f4834ff9640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4824065bb0 0x7f4824068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.226+0000 7f4834ff9640 1 -- 172.21.15.26:0/3234347329 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f482c0834f0 con 0x7f48380cec40 2023-12-19T18:16:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.226+0000 7f4837fff640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4824065bb0 0x7f4824068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.226+0000 7f4837fff640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4824065bb0 0x7f4824068070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f48280011f0 tx=0x7f4828007d40 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:24.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.230+0000 7f4834ff9640 1 -- 172.21.15.26:0/3234347329 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f482c050e00 con 0x7f48380cec40 2023-12-19T18:16:24.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.438+0000 7f483e62d640 1 -- 172.21.15.26:0/3234347329 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f48380affb0 con 0x7f48380cec40 2023-12-19T18:16:24.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.438+0000 7f4834ff9640 1 -- 172.21.15.26:0/3234347329 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v51) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f482c054420 con 0x7f48380cec40 2023-12-19T18:16:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:24.443 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:16:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2023-12-19T18:16:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2023-12-19T18:16:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg==", 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg==", 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:16:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg==", 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw==", 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig==", 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA==", 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2023-12-19T18:16:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ==", 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ==", 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A==", 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ==", 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg==", 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2023-12-19T18:16:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg==", 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ==", 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ==", 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ==", 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ==", 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlT5ynDxAA7Sjbdwrc1RyAXJboysB54Q==", 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA==", 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T18:16:24.455 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T18:16:24.455 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.455 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.455 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:24.455 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:24.455 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2023-12-19T18:16:24.455 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg==", 2023-12-19T18:16:24.455 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:16:24.455 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T18:16:24.455 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T18:16:24.455 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:16:24.456 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.456 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:24.456 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-12-19T18:16:24.456 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:16:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.442+0000 7f483e62d640 1 -- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4824065bb0 msgr2=0x7f4824068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.442+0000 7f483e62d640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4824065bb0 0x7f4824068070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f48280011f0 tx=0x7f4828007d40 comp rx=0 tx=0).stop 2023-12-19T18:16:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.442+0000 7f483e62d640 1 -- 172.21.15.26:0/3234347329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48380cec40 msgr2=0x7f48381432b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.442+0000 7f483e62d640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48380cec40 0x7f48381432b0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f482c00e980 tx=0x7f482c00ee40 comp rx=0 tx=0).stop 2023-12-19T18:16:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.442+0000 7f483e62d640 1 -- 172.21.15.26:0/3234347329 shutdown_connections 2023-12-19T18:16:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.442+0000 7f483e62d640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4824065bb0 0x7f4824068070 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.442+0000 7f483e62d640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4838154220 0x7f4838157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.442+0000 7f483e62d640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48380cec40 0x7f48381432b0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.442+0000 7f483e62d640 1 --2- 172.21.15.26:0/3234347329 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f483806b3d0 0x7f4838143e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.442+0000 7f483e62d640 1 -- 172.21.15.26:0/3234347329 >> 172.21.15.26:0/3234347329 conn(0x7f48380658e0 msgr2=0x7f48380cbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.442+0000 7f483e62d640 1 -- 172.21.15.26:0/3234347329 shutdown_connections 2023-12-19T18:16:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.442+0000 7f483e62d640 1 -- 172.21.15.26:0/3234347329 wait complete. 2023-12-19T18:16:24.465 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.606+0000 7f71f4db3640 1 Processor -- start 2023-12-19T18:16:24.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.606+0000 7f71f4db3640 1 -- start start 2023-12-19T18:16:24.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.606+0000 7f71f4db3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f71f015c7f0 0x7f71f006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.606+0000 7f71f4db3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f006ced0 con 0x7f71f00adb80 2023-12-19T18:16:24.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.606+0000 7f71f4db3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f006d1b0 con 0x7f71f00b30d0 2023-12-19T18:16:24.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.606+0000 7f71f4db3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f006d770 con 0x7f71f015c7f0 2023-12-19T18:16:24.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71e7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f71f00adb80 0x7f71f00cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34758/0 (socket says 172.21.15.26:34758) 2023-12-19T18:16:24.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71e7fff640 1 -- 172.21.15.26:0/1420682621 learned_addr learned my addr 172.21.15.26:0/1420682621 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:24.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71ee59c640 1 --2- 172.21.15.26:0/1420682621 >> v2:172.21.15.160:3300/0 conn(0x7f71f015c7f0 0x7f71f006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322917094 0 0) 0x7f71f006ced0 con 0x7f71f00adb80 2023-12-19T18:16:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71dc003450 con 0x7f71f00adb80 2023-12-19T18:16:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3083760834 0 0) 0x7f71dc003450 con 0x7f71f00adb80 2023-12-19T18:16:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 >> v2:172.21.15.160:3300/0 conn(0x7f71f015c7f0 msgr2=0x7f71f006c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 --2- 172.21.15.26:0/1420682621 >> v2:172.21.15.160:3300/0 conn(0x7f71f015c7f0 0x7f71f006c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 >> v1:172.21.15.160:6789/0 conn(0x7f71f00b30d0 legacy=0x7f71f00b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f006d490 con 0x7f71f00adb80 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f71d8001680 con 0x7f71f00adb80 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 >> v1:172.21.15.26:6789/0 conn(0x7f71f00adb80 legacy=0x7f71f00cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 --2- 172.21.15.26:0/1420682621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71dc003a70 0x7f71dc003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 --2- 172.21.15.26:0/1420682621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71dc008860 0x7f71dc00ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 --2- 172.21.15.26:0/1420682621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc00b490 0x7f71dc00d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f006d770 con 0x7f71dc008860 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc00e230 con 0x7f71dc00b490 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc00e3f0 con 0x7f71dc003a70 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71ee59c640 1 --2- 172.21.15.26:0/1420682621 >> v2:172.21.15.160:3300/0 conn(0x7f71f015c7f0 0x7f71f006c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71e7fff640 1 --2- 172.21.15.26:0/1420682621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71dc008860 0x7f71dc00ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71eed9d640 1 --2- 172.21.15.26:0/1420682621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc00b490 0x7f71dc00d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71ee59c640 1 --2- 172.21.15.26:0/1420682621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71dc003a70 0x7f71dc003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71e7fff640 1 -- 172.21.15.26:0/1420682621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71dc003a70 msgr2=0x7f71dc003e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71e7fff640 1 --2- 172.21.15.26:0/1420682621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71dc003a70 0x7f71dc003e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71e7fff640 1 -- 172.21.15.26:0/1420682621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc00b490 msgr2=0x7f71dc00d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71e7fff640 1 --2- 172.21.15.26:0/1420682621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc00b490 0x7f71dc00d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71e7fff640 1 -- 172.21.15.26:0/1420682621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc003450 con 0x7f71dc008860 2023-12-19T18:16:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71eed9d640 1 -- 172.21.15.26:0/1420682621 reap_dead start 2023-12-19T18:16:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.610+0000 7f71ee59c640 1 --2- 172.21.15.26:0/1420682621 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71dc003a70 0x7f71dc003e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.614+0000 7f71e7fff640 1 --2- 172.21.15.26:0/1420682621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71dc008860 0x7f71dc00ac90 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f71d800c240 tx=0x7f71d801f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acca57dcfd763b50 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:24.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.614+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8005ae0 con 0x7f71dc008860 2023-12-19T18:16:24.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.614+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d80ab070 con 0x7f71dc008860 2023-12-19T18:16:24.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.614+0000 7f71edd9b640 1 -- 172.21.15.26:0/1420682621 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8007070 con 0x7f71dc008860 2023-12-19T18:16:24.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.614+0000 7f71f4db3640 1 -- 172.21.15.26:0/1420682621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71dc008860 msgr2=0x7f71dc00ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.614+0000 7f71f4db3640 1 --2- 172.21.15.26:0/1420682621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71dc008860 0x7f71dc00ac90 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f71d800c240 tx=0x7f71d801f3b0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.614+0000 7f71f4db3640 1 -- 172.21.15.26:0/1420682621 shutdown_connections 2023-12-19T18:16:24.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.614+0000 7f71f4db3640 1 --2- 172.21.15.26:0/1420682621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71dc008860 0x7f71dc00ac90 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.614+0000 7f71f4db3640 1 -- 172.21.15.26:0/1420682621 >> 172.21.15.26:0/1420682621 conn(0x7f71f00658e0 msgr2=0x7f71f0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:24.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.614+0000 7f71f4db3640 1 -- 172.21.15.26:0/1420682621 shutdown_connections 2023-12-19T18:16:24.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.614+0000 7f71f4db3640 1 -- 172.21.15.26:0/1420682621 wait complete. 2023-12-19T18:16:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.614+0000 7f71f4db3640 1 Processor -- start 2023-12-19T18:16:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.614+0000 7f71f4db3640 1 -- start start 2023-12-19T18:16:24.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71f4db3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71dc008860 0x7f71f0154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71f4db3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71f0146120 0x7f71f00700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71f4db3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f0070620 0x7f71f0153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71f4db3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0153d30 con 0x7f71f0070620 2023-12-19T18:16:24.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71f4db3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0153fd0 con 0x7f71f0146120 2023-12-19T18:16:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71f4db3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f01542b0 con 0x7f71dc008860 2023-12-19T18:16:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71eed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f0070620 0x7f71f0153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71ee59c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71dc008860 0x7f71f0154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71eed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f0070620 0x7f71f0153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44468/0 (socket says 172.21.15.26:44468) 2023-12-19T18:16:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71ee59c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71dc008860 0x7f71f0154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:46524/0 (socket says 172.21.15.26:46524) 2023-12-19T18:16:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71eed9d640 1 -- 172.21.15.26:0/511404472 learned_addr learned my addr 172.21.15.26:0/511404472 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71e7fff640 1 --2- 172.21.15.26:0/511404472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71f0146120 0x7f71f00700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71eed9d640 1 -- 172.21.15.26:0/511404472 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71dc008860 msgr2=0x7f71f0154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71eed9d640 1 --2- 172.21.15.26:0/511404472 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71dc008860 0x7f71f0154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71eed9d640 1 -- 172.21.15.26:0/511404472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71f0146120 msgr2=0x7f71f00700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71eed9d640 1 --2- 172.21.15.26:0/511404472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71f0146120 0x7f71f00700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71eed9d640 1 -- 172.21.15.26:0/511404472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f0154590 con 0x7f71f0070620 2023-12-19T18:16:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71eed9d640 1 --2- 172.21.15.26:0/511404472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f0070620 0x7f71f0153040 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f71f00cd8e0 tx=0x7f71e800fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71e6ffd640 1 -- 172.21.15.26:0/511404472 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e800fec0 con 0x7f71f0070620 2023-12-19T18:16:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71e6ffd640 1 -- 172.21.15.26:0/511404472 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e8006d70 con 0x7f71f0070620 2023-12-19T18:16:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71e6ffd640 1 -- 172.21.15.26:0/511404472 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8003ba0 con 0x7f71f0070620 2023-12-19T18:16:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71f4db3640 1 -- 172.21.15.26:0/511404472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f01468e0 con 0x7f71f0070620 2023-12-19T18:16:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71f4db3640 1 -- 172.21.15.26:0/511404472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f0154ff0 con 0x7f71f0070620 2023-12-19T18:16:24.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.618+0000 7f71f4db3640 1 -- 172.21.15.26:0/511404472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f0074270 con 0x7f71f0070620 2023-12-19T18:16:24.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.622+0000 7f71e6ffd640 1 -- 172.21.15.26:0/511404472 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f71e8003d00 con 0x7f71f0070620 2023-12-19T18:16:24.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.622+0000 7f71e6ffd640 1 --2- 172.21.15.26:0/511404472 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71d0065b80 0x7f71d0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.622+0000 7f71e6ffd640 1 -- 172.21.15.26:0/511404472 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f71e8082730 con 0x7f71f0070620 2023-12-19T18:16:24.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.622+0000 7f71ee59c640 1 --2- 172.21.15.26:0/511404472 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71d0065b80 0x7f71d0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.622+0000 7f71ee59c640 1 --2- 172.21.15.26:0/511404472 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71d0065b80 0x7f71d0068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f71f00693e0 tx=0x7f71e0008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:24.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.626+0000 7f71e6ffd640 1 -- 172.21.15.26:0/511404472 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f71e8050150 con 0x7f71f0070620 2023-12-19T18:16:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71f4db3640 1 -- 172.21.15.26:0/511404472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71f0070fd0 con 0x7f71f0070620 2023-12-19T18:16:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71e6ffd640 1 -- 172.21.15.26:0/511404472 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v208) v1 ==== 74+0+13549 (secure 0 0 0) 0x7f71e8053770 con 0x7f71f0070620 2023-12-19T18:16:24.785 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:24.785 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":208,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:24.034781+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":194,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71f4db3640 1 -- 172.21.15.26:0/511404472 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71d0065b80 msgr2=0x7f71d0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71f4db3640 1 --2- 172.21.15.26:0/511404472 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71d0065b80 0x7f71d0068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f71f00693e0 tx=0x7f71e0008040 comp rx=0 tx=0).stop 2023-12-19T18:16:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71f4db3640 1 -- 172.21.15.26:0/511404472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f0070620 msgr2=0x7f71f0153040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71f4db3640 1 --2- 172.21.15.26:0/511404472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f0070620 0x7f71f0153040 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f71f00cd8e0 tx=0x7f71e800fbb0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71f4db3640 1 -- 172.21.15.26:0/511404472 shutdown_connections 2023-12-19T18:16:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71f4db3640 1 --2- 172.21.15.26:0/511404472 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71d0065b80 0x7f71d0068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71f4db3640 1 --2- 172.21.15.26:0/511404472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71f0070620 0x7f71f0153040 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71f4db3640 1 --2- 172.21.15.26:0/511404472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71f0146120 0x7f71f00700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71f4db3640 1 --2- 172.21.15.26:0/511404472 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71dc008860 0x7f71f0154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71f4db3640 1 -- 172.21.15.26:0/511404472 >> 172.21.15.26:0/511404472 conn(0x7f71f00658e0 msgr2=0x7f71f00b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71f4db3640 1 -- 172.21.15.26:0/511404472 shutdown_connections 2023-12-19T18:16:24.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.782+0000 7f71f4db3640 1 -- 172.21.15.26:0/511404472 wait complete. 2023-12-19T18:16:24.807 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:16:24.807 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:16:24.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.950+0000 7fee468d7640 1 Processor -- start 2023-12-19T18:16:24.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.950+0000 7fee468d7640 1 -- start start 2023-12-19T18:16:24.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.950+0000 7fee468d7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee4015a770 0x7fee400cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.950+0000 7fee468d7640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fee400cfae0 0x7fee40160a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.950+0000 7fee468d7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee40160f80 0x7fee400b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.950+0000 7fee468d7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee400cff30 con 0x7fee4015a770 2023-12-19T18:16:24.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.950+0000 7fee468d7640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee400cd490 con 0x7fee400cfae0 2023-12-19T18:16:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.950+0000 7fee468d7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee400cd770 con 0x7fee40160f80 2023-12-19T18:16:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.950+0000 7fee458d5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee4015a770 0x7fee400cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.950+0000 7fee458d5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee4015a770 0x7fee400cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44472/0 (socket says 172.21.15.26:44472) 2023-12-19T18:16:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.950+0000 7fee458d5640 1 -- 172.21.15.26:0/356818156 learned_addr learned my addr 172.21.15.26:0/356818156 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.950+0000 7fee450d4640 1 --2- 172.21.15.26:0/356818156 >> v2:172.21.15.160:3300/0 conn(0x7fee400cfae0 0x7fee40160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.950+0000 7fee460d6640 1 --2- 172.21.15.26:0/356818156 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee40160f80 0x7fee400b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.954+0000 7fee458d5640 1 -- 172.21.15.26:0/356818156 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee40160f80 msgr2=0x7fee400b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.954+0000 7fee458d5640 1 --2- 172.21.15.26:0/356818156 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee40160f80 0x7fee400b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.954+0000 7fee458d5640 1 -- 172.21.15.26:0/356818156 >> v2:172.21.15.160:3300/0 conn(0x7fee400cfae0 msgr2=0x7fee40160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.954+0000 7fee458d5640 1 --2- 172.21.15.26:0/356818156 >> v2:172.21.15.160:3300/0 conn(0x7fee400cfae0 0x7fee40160a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.954+0000 7fee458d5640 1 -- 172.21.15.26:0/356818156 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee400cda50 con 0x7fee4015a770 2023-12-19T18:16:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.954+0000 7fee450d4640 1 --2- 172.21.15.26:0/356818156 >> v2:172.21.15.160:3300/0 conn(0x7fee400cfae0 0x7fee40160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:24.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.954+0000 7fee458d5640 1 --2- 172.21.15.26:0/356818156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015a770 0x7fee400cf5a0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fee400b8510 tx=0x7fee3c01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b14facb1068ca4c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:24.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.954+0000 7fee448d3640 1 -- 172.21.15.26:0/356818156 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0ab400 con 0x7fee4015a770 2023-12-19T18:16:24.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.954+0000 7fee448d3640 1 -- 172.21.15.26:0/356818156 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c0a7050 con 0x7fee4015a770 2023-12-19T18:16:24.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.954+0000 7fee448d3640 1 -- 172.21.15.26:0/356818156 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0ab780 con 0x7fee4015a770 2023-12-19T18:16:24.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 -- 172.21.15.26:0/356818156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015a770 msgr2=0x7fee400cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 --2- 172.21.15.26:0/356818156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015a770 0x7fee400cf5a0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fee400b8510 tx=0x7fee3c01c210 comp rx=0 tx=0).stop 2023-12-19T18:16:24.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 -- 172.21.15.26:0/356818156 shutdown_connections 2023-12-19T18:16:24.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 --2- 172.21.15.26:0/356818156 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee40160f80 0x7fee400b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 --2- 172.21.15.26:0/356818156 >> v2:172.21.15.160:3300/0 conn(0x7fee400cfae0 0x7fee40160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 --2- 172.21.15.26:0/356818156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015a770 0x7fee400cf5a0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 -- 172.21.15.26:0/356818156 >> 172.21.15.26:0/356818156 conn(0x7fee4006a150 msgr2=0x7fee4006a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:24.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 -- 172.21.15.26:0/356818156 shutdown_connections 2023-12-19T18:16:24.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 -- 172.21.15.26:0/356818156 wait complete. 2023-12-19T18:16:24.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 Processor -- start 2023-12-19T18:16:24.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 -- start start 2023-12-19T18:16:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee400cfae0 0x7fee400c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015a770 0x7fee4015f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee40160f80 0x7fee4015f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee400c9150 con 0x7fee4015a770 2023-12-19T18:16:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee40156b80 con 0x7fee40160f80 2023-12-19T18:16:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee468d7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee40156e60 con 0x7fee400cfae0 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee450d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015a770 0x7fee4015f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee450d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015a770 0x7fee4015f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44478/0 (socket says 172.21.15.26:44478) 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee450d4640 1 -- 172.21.15.26:0/2091246009 learned_addr learned my addr 172.21.15.26:0/2091246009 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee460d6640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee40160f80 0x7fee4015f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee458d5640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee400cfae0 0x7fee400c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee450d4640 1 -- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee400cfae0 msgr2=0x7fee400c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee450d4640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee400cfae0 0x7fee400c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee450d4640 1 -- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee40160f80 msgr2=0x7fee4015f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee450d4640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee40160f80 0x7fee4015f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee450d4640 1 -- 172.21.15.26:0/2091246009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee30009420 con 0x7fee4015a770 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.958+0000 7fee450d4640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015a770 0x7fee4015f490 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fee3000e9b0 tx=0x7fee3000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.962+0000 7fee367fc640 1 -- 172.21.15.26:0/2091246009 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30009700 con 0x7fee4015a770 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.962+0000 7fee367fc640 1 -- 172.21.15.26:0/2091246009 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3000ca00 con 0x7fee4015a770 2023-12-19T18:16:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.962+0000 7fee367fc640 1 -- 172.21.15.26:0/2091246009 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3000cd90 con 0x7fee4015a770 2023-12-19T18:16:24.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.962+0000 7fee468d7640 1 -- 172.21.15.26:0/2091246009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3c0094d0 con 0x7fee4015a770 2023-12-19T18:16:24.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.962+0000 7fee468d7640 1 -- 172.21.15.26:0/2091246009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee40157420 con 0x7fee4015a770 2023-12-19T18:16:24.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.962+0000 7fee468d7640 1 -- 172.21.15.26:0/2091246009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee40061930 con 0x7fee4015a770 2023-12-19T18:16:24.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.962+0000 7fee367fc640 1 -- 172.21.15.26:0/2091246009 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fee30013470 con 0x7fee4015a770 2023-12-19T18:16:24.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.962+0000 7fee367fc640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee24065b80 0x7fee24068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.962+0000 7fee367fc640 1 -- 172.21.15.26:0/2091246009 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5617+0+0 (secure 0 0 0) 0x7fee30083560 con 0x7fee4015a770 2023-12-19T18:16:24.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.962+0000 7fee458d5640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee24065b80 0x7fee24068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.966+0000 7fee458d5640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee24065b80 0x7fee24068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0ad000 tx=0x7fee3c008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:24.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:24.966+0000 7fee367fc640 1 -- 172.21.15.26:0/2091246009 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fee30050e50 con 0x7fee4015a770 2023-12-19T18:16:25.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee468d7640 1 -- 172.21.15.26:0/2091246009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee4015be20 con 0x7fee4015a770 2023-12-19T18:16:25.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee367fc640 1 -- 172.21.15.26:0/2091246009 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v208) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee30081e60 con 0x7fee4015a770 2023-12-19T18:16:25.129 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:16:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee468d7640 1 -- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee24065b80 msgr2=0x7fee24068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee468d7640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee24065b80 0x7fee24068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0ad000 tx=0x7fee3c008040 comp rx=0 tx=0).stop 2023-12-19T18:16:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee468d7640 1 -- 172.21.15.26:0/2091246009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015a770 msgr2=0x7fee4015f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee468d7640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015a770 0x7fee4015f490 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fee3000e9b0 tx=0x7fee3000ee70 comp rx=0 tx=0).stop 2023-12-19T18:16:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee468d7640 1 -- 172.21.15.26:0/2091246009 shutdown_connections 2023-12-19T18:16:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee468d7640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee24065b80 0x7fee24068040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee468d7640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee40160f80 0x7fee4015f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee468d7640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015a770 0x7fee4015f490 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee468d7640 1 --2- 172.21.15.26:0/2091246009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee400cfae0 0x7fee400c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee468d7640 1 -- 172.21.15.26:0/2091246009 >> 172.21.15.26:0/2091246009 conn(0x7fee4006a150 msgr2=0x7fee400d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:25.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee468d7640 1 -- 172.21.15.26:0/2091246009 shutdown_connections 2023-12-19T18:16:25.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.126+0000 7fee468d7640 1 -- 172.21.15.26:0/2091246009 wait complete. 2023-12-19T18:16:25.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:16:25.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.290+0000 7ff123073640 1 Processor -- start 2023-12-19T18:16:25.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.290+0000 7ff123073640 1 -- start start 2023-12-19T18:16:25.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.290+0000 7ff123073640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff11c154220 0x7ff11c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.290+0000 7ff123073640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff11c0cee10 0x7ff11c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.290+0000 7ff123073640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11c06bc30 con 0x7ff11c06b3d0 2023-12-19T18:16:25.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.290+0000 7ff123073640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11c0cc6e0 con 0x7ff11c0cee10 2023-12-19T18:16:25.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.290+0000 7ff123073640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11c15afc0 con 0x7ff11c154220 2023-12-19T18:16:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.294+0000 7ff121610640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff11c0cee10 0x7ff11c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.294+0000 7ff113fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff11c06b3d0 0x7ff11c0ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34768/0 (socket says 172.21.15.26:34768) 2023-12-19T18:16:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.294+0000 7ff113fff640 1 -- 172.21.15.26:0/2285770532 learned_addr learned my addr 172.21.15.26:0/2285770532 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.294+0000 7ff120e0f640 1 --2- 172.21.15.26:0/2285770532 >> v2:172.21.15.160:3300/0 conn(0x7ff11c154220 0x7ff11c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.294+0000 7ff121610640 1 -- 172.21.15.26:0/2285770532 >> v2:172.21.15.160:3300/0 conn(0x7ff11c154220 msgr2=0x7ff11c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.294+0000 7ff121610640 1 --2- 172.21.15.26:0/2285770532 >> v2:172.21.15.160:3300/0 conn(0x7ff11c154220 0x7ff11c15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.294+0000 7ff121610640 1 -- 172.21.15.26:0/2285770532 >> v1:172.21.15.26:6789/0 conn(0x7ff11c06b3d0 legacy=0x7ff11c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.294+0000 7ff121610640 1 -- 172.21.15.26:0/2285770532 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c06b950 con 0x7ff11c0cee10 2023-12-19T18:16:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.294+0000 7ff121610640 1 --2- 172.21.15.26:0/2285770532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff11c0cee10 0x7ff11c15b1c0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0667b0 tx=0x7ff11401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da6dec3d63c06e8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.294+0000 7ff1137fe640 1 -- 172.21.15.26:0/2285770532 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1140a59a0 con 0x7ff11c0cee10 2023-12-19T18:16:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.294+0000 7ff1137fe640 1 -- 172.21.15.26:0/2285770532 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1140a5b00 con 0x7ff11c0cee10 2023-12-19T18:16:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.294+0000 7ff1137fe640 1 -- 172.21.15.26:0/2285770532 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1140a5e90 con 0x7ff11c0cee10 2023-12-19T18:16:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 -- 172.21.15.26:0/2285770532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff11c0cee10 msgr2=0x7ff11c15b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 --2- 172.21.15.26:0/2285770532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff11c0cee10 0x7ff11c15b1c0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0667b0 tx=0x7ff11401c8e0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 -- 172.21.15.26:0/2285770532 shutdown_connections 2023-12-19T18:16:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 --2- 172.21.15.26:0/2285770532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff11c0cee10 0x7ff11c15b1c0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 --2- 172.21.15.26:0/2285770532 >> v2:172.21.15.160:3300/0 conn(0x7ff11c154220 0x7ff11c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 -- 172.21.15.26:0/2285770532 >> 172.21.15.26:0/2285770532 conn(0x7ff11c0658e0 msgr2=0x7ff11c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 -- 172.21.15.26:0/2285770532 shutdown_connections 2023-12-19T18:16:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 -- 172.21.15.26:0/2285770532 wait complete. 2023-12-19T18:16:25.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 Processor -- start 2023-12-19T18:16:25.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 -- start start 2023-12-19T18:16:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff11c06b3d0 0x7ff11c14ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff11c0cee10 0x7ff11c149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff11c154220 0x7ff11c152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0aebe0 con 0x7ff11c154220 2023-12-19T18:16:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0af4c0 con 0x7ff11c06b3d0 2023-12-19T18:16:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff123073640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0af7a0 con 0x7ff11c0cee10 2023-12-19T18:16:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff121610640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff11c154220 0x7ff11c152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff121610640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff11c154220 0x7ff11c152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44490/0 (socket says 172.21.15.26:44490) 2023-12-19T18:16:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff121610640 1 -- 172.21.15.26:0/3102450808 learned_addr learned my addr 172.21.15.26:0/3102450808 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff120e0f640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff11c06b3d0 0x7ff11c14ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff121610640 1 -- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff11c0cee10 msgr2=0x7ff11c149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.298+0000 7ff113fff640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff11c0cee10 0x7ff11c149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff121610640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff11c0cee10 0x7ff11c149ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff121610640 1 -- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff11c06b3d0 msgr2=0x7ff11c14ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff121610640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff11c06b3d0 0x7ff11c14ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff121610640 1 -- 172.21.15.26:0/3102450808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff114009a70 con 0x7ff11c154220 2023-12-19T18:16:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff120e0f640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff11c06b3d0 0x7ff11c14ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff113fff640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff11c0cee10 0x7ff11c149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff121610640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff11c154220 0x7ff11c152bc0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7ff1140194f0 tx=0x7ff114019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff1117fa640 1 -- 172.21.15.26:0/3102450808 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1140ade00 con 0x7ff11c154220 2023-12-19T18:16:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff1117fa640 1 -- 172.21.15.26:0/3102450808 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff114009450 con 0x7ff11c154220 2023-12-19T18:16:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff1117fa640 1 -- 172.21.15.26:0/3102450808 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1140b9690 con 0x7ff11c154220 2023-12-19T18:16:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff123073640 1 -- 172.21.15.26:0/3102450808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11c0afa80 con 0x7ff11c154220 2023-12-19T18:16:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff123073640 1 -- 172.21.15.26:0/3102450808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11c0af0e0 con 0x7ff11c154220 2023-12-19T18:16:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff123073640 1 -- 172.21.15.26:0/3102450808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11c0af270 con 0x7ff11c154220 2023-12-19T18:16:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff1117fa640 1 -- 172.21.15.26:0/3102450808 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff1140b97f0 con 0x7ff11c154220 2023-12-19T18:16:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff1117fa640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff108065b80 0x7ff108068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff1117fa640 1 -- 172.21.15.26:0/3102450808 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5617+0+0 (secure 0 0 0) 0x7ff11411eb00 con 0x7ff11c154220 2023-12-19T18:16:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.302+0000 7ff120e0f640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff108065b80 0x7ff108068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.306+0000 7ff120e0f640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff108065b80 0x7ff108068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff11c15d260 tx=0x7ff118008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:25.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.306+0000 7ff1117fa640 1 -- 172.21.15.26:0/3102450808 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff114120e60 con 0x7ff11c154220 2023-12-19T18:16:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.466+0000 7ff123073640 1 -- 172.21.15.26:0/3102450808 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff11c074270 con 0x7ff11c154220 2023-12-19T18:16:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.466+0000 7ff1117fa640 1 -- 172.21.15.26:0/3102450808 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v208) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff1140bd060 con 0x7ff11c154220 2023-12-19T18:16:25.470 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:25.470 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T18:16:25.470 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:25.470 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T18:16:25.470 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T18:16:25.470 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:25.470 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T18:16:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.470+0000 7ff123073640 1 -- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff108065b80 msgr2=0x7ff108068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.470+0000 7ff123073640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff108065b80 0x7ff108068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff11c15d260 tx=0x7ff118008040 comp rx=0 tx=0).stop 2023-12-19T18:16:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.470+0000 7ff123073640 1 -- 172.21.15.26:0/3102450808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff11c154220 msgr2=0x7ff11c152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.470+0000 7ff123073640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff11c154220 0x7ff11c152bc0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7ff1140194f0 tx=0x7ff114019680 comp rx=0 tx=0).stop 2023-12-19T18:16:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.470+0000 7ff123073640 1 -- 172.21.15.26:0/3102450808 shutdown_connections 2023-12-19T18:16:25.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.470+0000 7ff123073640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff108065b80 0x7ff108068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.470+0000 7ff123073640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff11c154220 0x7ff11c152bc0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.470+0000 7ff123073640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff11c0cee10 0x7ff11c149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.470+0000 7ff123073640 1 --2- 172.21.15.26:0/3102450808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff11c06b3d0 0x7ff11c14ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.470+0000 7ff123073640 1 -- 172.21.15.26:0/3102450808 >> 172.21.15.26:0/3102450808 conn(0x7ff11c0658e0 msgr2=0x7ff11c065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:25.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.470+0000 7ff123073640 1 -- 172.21.15.26:0/3102450808 shutdown_connections 2023-12-19T18:16:25.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.470+0000 7ff123073640 1 -- 172.21.15.26:0/3102450808 wait complete. 2023-12-19T18:16:25.492 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:16:25.492 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:16:25.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5f4c4640 1 Processor -- start 2023-12-19T18:16:25.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5f4c4640 1 -- start start 2023-12-19T18:16:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5f4c4640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0f5815aad0 0x7f0f5806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5f4c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f580b3f90 0x7f0f580cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5f4c4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5806cbf0 con 0x7f0f580b1590 2023-12-19T18:16:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5f4c4640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f580ca8b0 con 0x7f0f5815aad0 2023-12-19T18:16:25.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5f4c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5806b6f0 con 0x7f0f580b3f90 2023-12-19T18:16:25.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5d260640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0f5815aad0 0x7f0f5806c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5da61640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f580b3f90 0x7f0f580cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5ca5f640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f0f580b1590 0x7f0f58070870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:57656/0 (socket says 172.21.15.26:57656) 2023-12-19T18:16:25.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5ca5f640 1 -- 172.21.15.26:0/1805528311 learned_addr learned my addr 172.21.15.26:0/1805528311 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:25.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5da61640 1 -- 172.21.15.26:0/1805528311 >> v2:172.21.15.160:3300/0 conn(0x7f0f5815aad0 msgr2=0x7f0f5806c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:25.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5da61640 1 --2- 172.21.15.26:0/1805528311 >> v2:172.21.15.160:3300/0 conn(0x7f0f5815aad0 0x7f0f5806c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f4ffff640 1 -- 172.21.15.26:0/1805528311 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854557425 0 0) 0x7f0f48001120 con 0x7f0f580b1590 2023-12-19T18:16:25.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5da61640 1 -- 172.21.15.26:0/1805528311 >> v1:172.21.15.160:6789/0 conn(0x7f0f580b1590 legacy=0x7f0f58070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5da61640 1 -- 172.21.15.26:0/1805528311 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5806b9d0 con 0x7f0f580b3f90 2023-12-19T18:16:25.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.638+0000 7f0f5d260640 1 --2- 172.21.15.26:0/1805528311 >> v2:172.21.15.160:3300/0 conn(0x7f0f5815aad0 0x7f0f5806c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.642+0000 7f0f5da61640 1 --2- 172.21.15.26:0/1805528311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f580b3f90 0x7f0f580cad40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009d90 tx=0x7f0f5001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55016fb54e9588c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.642+0000 7f0f4ffff640 1 -- 172.21.15.26:0/1805528311 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f500a58f0 con 0x7f0f580b3f90 2023-12-19T18:16:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.642+0000 7f0f4ffff640 1 -- 172.21.15.26:0/1805528311 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f500a5a50 con 0x7f0f580b3f90 2023-12-19T18:16:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.642+0000 7f0f4ffff640 1 -- 172.21.15.26:0/1805528311 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f500a5de0 con 0x7f0f580b3f90 2023-12-19T18:16:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.642+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/1805528311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f580b3f90 msgr2=0x7f0f580cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.642+0000 7f0f5f4c4640 1 --2- 172.21.15.26:0/1805528311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f580b3f90 0x7f0f580cad40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009d90 tx=0x7f0f5001c920 comp rx=0 tx=0).stop 2023-12-19T18:16:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.642+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/1805528311 shutdown_connections 2023-12-19T18:16:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.642+0000 7f0f5f4c4640 1 --2- 172.21.15.26:0/1805528311 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f580b3f90 0x7f0f580cad40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.642+0000 7f0f5f4c4640 1 --2- 172.21.15.26:0/1805528311 >> v2:172.21.15.160:3300/0 conn(0x7f0f5815aad0 0x7f0f5806c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.642+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/1805528311 >> 172.21.15.26:0/1805528311 conn(0x7f0f580b0270 msgr2=0x7f0f580b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.642+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/1805528311 shutdown_connections 2023-12-19T18:16:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.642+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/1805528311 wait complete. 2023-12-19T18:16:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5f4c4640 1 Processor -- start 2023-12-19T18:16:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5f4c4640 1 -- start start 2023-12-19T18:16:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5f4c4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f580b1590 0x7f0f580c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5f4c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f580b3f90 0x7f0f580a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5f4c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f5815aad0 0x7f0f58159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5f4c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f580c7a90 con 0x7f0f580b3f90 2023-12-19T18:16:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5f4c4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f580c7d70 con 0x7f0f580b1590 2023-12-19T18:16:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5f4c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f580c8050 con 0x7f0f5815aad0 2023-12-19T18:16:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5ca5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f580b3f90 0x7f0f580a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5ca5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f580b3f90 0x7f0f580a8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44506/0 (socket says 172.21.15.26:44506) 2023-12-19T18:16:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5ca5f640 1 -- 172.21.15.26:0/3552364254 learned_addr learned my addr 172.21.15.26:0/3552364254 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5d260640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f580b1590 0x7f0f580c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5da61640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f5815aad0 0x7f0f58159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5ca5f640 1 -- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f5815aad0 msgr2=0x7f0f58159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5ca5f640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f5815aad0 0x7f0f58159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5ca5f640 1 -- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f580b1590 msgr2=0x7f0f580c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5ca5f640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f580b1590 0x7f0f580c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5ca5f640 1 -- 172.21.15.26:0/3552364254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f54009420 con 0x7f0f580b3f90 2023-12-19T18:16:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5ca5f640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f580b3f90 0x7f0f580a8e60 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c780 tx=0x7f0f4800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f4dffb640 1 -- 172.21.15.26:0/3552364254 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480072d0 con 0x7f0f580b3f90 2023-12-19T18:16:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f4dffb640 1 -- 172.21.15.26:0/3552364254 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f48007430 con 0x7f0f580b3f90 2023-12-19T18:16:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f4dffb640 1 -- 172.21.15.26:0/3552364254 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480047f0 con 0x7f0f580b3f90 2023-12-19T18:16:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/3552364254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f50009a70 con 0x7f0f580b3f90 2023-12-19T18:16:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.646+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/3552364254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f580c8610 con 0x7f0f580b3f90 2023-12-19T18:16:25.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.650+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/3552364254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f580c8800 con 0x7f0f580b3f90 2023-12-19T18:16:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.650+0000 7f0f4dffb640 1 -- 172.21.15.26:0/3552364254 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0f48004950 con 0x7f0f580b3f90 2023-12-19T18:16:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.650+0000 7f0f4dffb640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0f3c065bb0 0x7f0f3c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.650+0000 7f0f4dffb640 1 -- 172.21.15.26:0/3552364254 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f0f48082250 con 0x7f0f580b3f90 2023-12-19T18:16:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.650+0000 7f0f5d260640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0f3c065bb0 0x7f0f3c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.650+0000 7f0f5d260640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0f3c065bb0 0x7f0f3c068070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0f580aa540 tx=0x7f0f54009190 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.654+0000 7f0f4dffb640 1 -- 172.21.15.26:0/3552364254 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0f4804fb60 con 0x7f0f580b3f90 2023-12-19T18:16:25.698 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-12-19T18:16:25.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:25.810+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/3552364254 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0f5806cbf0 con 0x7f0f580b3f90 2023-12-19T18:16:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.382+0000 7f0f4dffb640 1 -- 172.21.15.26:0/3552364254 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v209) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0f5806cbf0 con 0x7f0f580b3f90 2023-12-19T18:16:26.387 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T18:16:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.386+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0f3c065bb0 msgr2=0x7f0f3c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.386+0000 7f0f5f4c4640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0f3c065bb0 0x7f0f3c068070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0f580aa540 tx=0x7f0f54009190 comp rx=0 tx=0).stop 2023-12-19T18:16:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.386+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/3552364254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f580b3f90 msgr2=0x7f0f580a8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.386+0000 7f0f5f4c4640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f580b3f90 0x7f0f580a8e60 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c780 tx=0x7f0f4800cc40 comp rx=0 tx=0).stop 2023-12-19T18:16:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.386+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/3552364254 shutdown_connections 2023-12-19T18:16:26.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.386+0000 7f0f5f4c4640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0f3c065bb0 0x7f0f3c068070 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.386+0000 7f0f5f4c4640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f5815aad0 0x7f0f58159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.386+0000 7f0f5f4c4640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f580b3f90 0x7f0f580a8e60 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.386+0000 7f0f5f4c4640 1 --2- 172.21.15.26:0/3552364254 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f580b1590 0x7f0f580c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.386+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/3552364254 >> 172.21.15.26:0/3552364254 conn(0x7f0f580b0270 msgr2=0x7f0f580af190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:26.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.386+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/3552364254 shutdown_connections 2023-12-19T18:16:26.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.386+0000 7f0f5f4c4640 1 -- 172.21.15.26:0/3552364254 wait complete. 2023-12-19T18:16:26.417 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:16:26.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.566+0000 7f8331c66640 1 Processor -- start 2023-12-19T18:16:26.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.566+0000 7f8331c66640 1 -- start start 2023-12-19T18:16:26.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.566+0000 7f8331c66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c15ada0 0x7f832c0ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.566+0000 7f8331c66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f832c070db0 0x7f832c0b0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.566+0000 7f8331c66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f832c0b0a60 0x7f832c0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.566+0000 7f8331c66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c0715b0 con 0x7f832c070db0 2023-12-19T18:16:26.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.566+0000 7f8331c66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832c0adfd0 con 0x7f832c0b0a60 2023-12-19T18:16:26.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.566+0000 7f8331c66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c06b500 con 0x7f832c15ada0 2023-12-19T18:16:26.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.566+0000 7f8323fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f832c070db0 0x7f832c0b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.566+0000 7f8323fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f832c070db0 0x7f832c0b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40342/0 (socket says 172.21.15.26:40342) 2023-12-19T18:16:26.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.566+0000 7f8323fff640 1 -- 172.21.15.26:0/3396393650 learned_addr learned my addr 172.21.15.26:0/3396393650 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:26.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.566+0000 7f832bfff640 1 --2- 172.21.15.26:0/3396393650 >> v2:172.21.15.160:3300/0 conn(0x7f832c0b0a60 0x7f832c0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.566+0000 7f832b7fe640 1 --2- 172.21.15.26:0/3396393650 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c15ada0 0x7f832c0ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8323fff640 1 -- 172.21.15.26:0/3396393650 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c15ada0 msgr2=0x7f832c0ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:26.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8323fff640 1 --2- 172.21.15.26:0/3396393650 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c15ada0 0x7f832c0ada90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8323fff640 1 -- 172.21.15.26:0/3396393650 >> v2:172.21.15.160:3300/0 conn(0x7f832c0b0a60 msgr2=0x7f832c0ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:26.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8323fff640 1 --2- 172.21.15.26:0/3396393650 >> v2:172.21.15.160:3300/0 conn(0x7f832c0b0a60 0x7f832c0ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8323fff640 1 -- 172.21.15.26:0/3396393650 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c070b60 con 0x7f832c070db0 2023-12-19T18:16:26.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8323fff640 1 --2- 172.21.15.26:0/3396393650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c070db0 0x7f832c0b0520 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f83140098a0 tx=0x7f831401c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4ee6f45aff32ff server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:26.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f832affd640 1 -- 172.21.15.26:0/3396393650 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83140ab600 con 0x7f832c070db0 2023-12-19T18:16:26.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f832affd640 1 -- 172.21.15.26:0/3396393650 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83140a7050 con 0x7f832c070db0 2023-12-19T18:16:26.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f832affd640 1 -- 172.21.15.26:0/3396393650 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83140ab980 con 0x7f832c070db0 2023-12-19T18:16:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8331c66640 1 -- 172.21.15.26:0/3396393650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c070db0 msgr2=0x7f832c0b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8331c66640 1 --2- 172.21.15.26:0/3396393650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c070db0 0x7f832c0b0520 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f83140098a0 tx=0x7f831401c1f0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8331c66640 1 -- 172.21.15.26:0/3396393650 shutdown_connections 2023-12-19T18:16:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8331c66640 1 --2- 172.21.15.26:0/3396393650 >> v2:172.21.15.160:3300/0 conn(0x7f832c0b0a60 0x7f832c0ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8331c66640 1 --2- 172.21.15.26:0/3396393650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c070db0 0x7f832c0b0520 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8331c66640 1 --2- 172.21.15.26:0/3396393650 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c15ada0 0x7f832c0ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8331c66640 1 -- 172.21.15.26:0/3396393650 >> 172.21.15.26:0/3396393650 conn(0x7f832c0660d0 msgr2=0x7f832c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8331c66640 1 -- 172.21.15.26:0/3396393650 shutdown_connections 2023-12-19T18:16:26.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.570+0000 7f8331c66640 1 -- 172.21.15.26:0/3396393650 wait complete. 2023-12-19T18:16:26.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8331c66640 1 Processor -- start 2023-12-19T18:16:26.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8331c66640 1 -- start start 2023-12-19T18:16:26.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8331c66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f832c070db0 0x7f832c15c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8331c66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c0b0a60 0x7f832c14f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8331c66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c15ada0 0x7f832c0c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8331c66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c15c5b0 con 0x7f832c15ada0 2023-12-19T18:16:26.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8331c66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c15c880 con 0x7f832c070db0 2023-12-19T18:16:26.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8331c66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c15cb60 con 0x7f832c0b0a60 2023-12-19T18:16:26.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f832bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c15ada0 0x7f832c0c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f832bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c15ada0 0x7f832c0c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40350/0 (socket says 172.21.15.26:40350) 2023-12-19T18:16:26.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f832bfff640 1 -- 172.21.15.26:0/1885771001 learned_addr learned my addr 172.21.15.26:0/1885771001 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:26.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8323fff640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c0b0a60 0x7f832c14f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f832b7fe640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f832c070db0 0x7f832c15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f832bfff640 1 -- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c0b0a60 msgr2=0x7f832c14f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f832bfff640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c0b0a60 0x7f832c14f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f832bfff640 1 -- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f832c070db0 msgr2=0x7f832c15c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f832bfff640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f832c070db0 0x7f832c15c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f832bfff640 1 -- 172.21.15.26:0/1885771001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83140094d0 con 0x7f832c15ada0 2023-12-19T18:16:26.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f832b7fe640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f832c070db0 0x7f832c15c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:26.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f832bfff640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c15ada0 0x7f832c0c4560 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f8324007e60 tx=0x7f832400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:26.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8328ff9640 1 -- 172.21.15.26:0/1885771001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324014620 con 0x7f832c15ada0 2023-12-19T18:16:26.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8328ff9640 1 -- 172.21.15.26:0/1885771001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832400ee90 con 0x7f832c15ada0 2023-12-19T18:16:26.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8331c66640 1 -- 172.21.15.26:0/1885771001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c0c2400 con 0x7f832c15ada0 2023-12-19T18:16:26.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8328ff9640 1 -- 172.21.15.26:0/1885771001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83240149e0 con 0x7f832c15ada0 2023-12-19T18:16:26.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.574+0000 7f8331c66640 1 -- 172.21.15.26:0/1885771001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c14fe00 con 0x7f832c15ada0 2023-12-19T18:16:26.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.578+0000 7f8331c66640 1 -- 172.21.15.26:0/1885771001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c0c2920 con 0x7f832c15ada0 2023-12-19T18:16:26.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.578+0000 7f8328ff9640 1 -- 172.21.15.26:0/1885771001 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8324014b40 con 0x7f832c15ada0 2023-12-19T18:16:26.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.578+0000 7f8328ff9640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f830c065bb0 0x7f830c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.578+0000 7f8328ff9640 1 -- 172.21.15.26:0/1885771001 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5987+0+0 (secure 0 0 0) 0x7f8324083cb0 con 0x7f832c15ada0 2023-12-19T18:16:26.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.578+0000 7f832b7fe640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f830c065bb0 0x7f830c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.578+0000 7f832b7fe640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f830c065bb0 0x7f830c068070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f831c00b9f0 tx=0x7f831c008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:26.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.582+0000 7f8328ff9640 1 -- 172.21.15.26:0/1885771001 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f832c0c2920 con 0x7f832c15ada0 2023-12-19T18:16:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:26.742+0000 7f8331c66640 1 -- 172.21.15.26:0/1885771001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f832c0c2920 con 0x7f832c15ada0 2023-12-19T18:16:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.390+0000 7f8328ff9640 1 -- 172.21.15.26:0/1885771001 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v210) v1 ==== 170+0+0 (secure 0 0 0) 0x7f832c0c2920 con 0x7f832c15ada0 2023-12-19T18:16:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T18:16:27.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.398+0000 7f8331c66640 1 -- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f830c065bb0 msgr2=0x7f830c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:27.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.398+0000 7f8331c66640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f830c065bb0 0x7f830c068070 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f831c00b9f0 tx=0x7f831c008040 comp rx=0 tx=0).stop 2023-12-19T18:16:27.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.398+0000 7f8331c66640 1 -- 172.21.15.26:0/1885771001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c15ada0 msgr2=0x7f832c0c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:27.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.398+0000 7f8331c66640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c15ada0 0x7f832c0c4560 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f8324007e60 tx=0x7f832400e4b0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.402+0000 7f8331c66640 1 -- 172.21.15.26:0/1885771001 shutdown_connections 2023-12-19T18:16:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.402+0000 7f8331c66640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f830c065bb0 0x7f830c068070 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.402+0000 7f8331c66640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f832c15ada0 0x7f832c0c4560 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.402+0000 7f8331c66640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f832c0b0a60 0x7f832c14f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.402+0000 7f8331c66640 1 --2- 172.21.15.26:0/1885771001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f832c070db0 0x7f832c15c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.402+0000 7f8331c66640 1 -- 172.21.15.26:0/1885771001 >> 172.21.15.26:0/1885771001 conn(0x7f832c0660d0 msgr2=0x7f832c1549b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:27.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.402+0000 7f8331c66640 1 -- 172.21.15.26:0/1885771001 shutdown_connections 2023-12-19T18:16:27.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.402+0000 7f8331c66640 1 -- 172.21.15.26:0/1885771001 wait complete. 2023-12-19T18:16:27.440 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:16:27.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.586+0000 7f340ff39640 1 Processor -- start 2023-12-19T18:16:27.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.586+0000 7f340ff39640 1 -- start start 2023-12-19T18:16:27.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340ff39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3408156e50 0x7f340815a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:27.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340ff39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3408070c10 0x7f34080ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:27.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340ff39640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34080cafc0 con 0x7f34080c9c30 2023-12-19T18:16:27.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340ff39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34080ca440 con 0x7f3408070c10 2023-12-19T18:16:27.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340ff39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34080ca720 con 0x7f3408156e50 2023-12-19T18:16:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340e4d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3408070c10 0x7f34080ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340e4d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3408070c10 0x7f34080ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40364/0 (socket says 172.21.15.26:40364) 2023-12-19T18:16:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340e4d6640 1 -- 172.21.15.26:0/1121259617 learned_addr learned my addr 172.21.15.26:0/1121259617 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340d4d4640 1 --1- 172.21.15.26:0/1121259617 >> v1:172.21.15.26:6789/0 conn(0x7f34080c9c30 0x7f3408070500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47046/0 (socket says 172.21.15.26:47046) 2023-12-19T18:16:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340dcd5640 1 --2- 172.21.15.26:0/1121259617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3408156e50 0x7f340815a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340e4d6640 1 -- 172.21.15.26:0/1121259617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3408156e50 msgr2=0x7f340815a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340e4d6640 1 --2- 172.21.15.26:0/1121259617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3408156e50 0x7f340815a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340e4d6640 1 -- 172.21.15.26:0/1121259617 >> v1:172.21.15.26:6789/0 conn(0x7f34080c9c30 legacy=0x7f3408070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340e4d6640 1 -- 172.21.15.26:0/1121259617 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34080caa00 con 0x7f3408070c10 2023-12-19T18:16:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340ccd3640 1 -- 172.21.15.26:0/1121259617 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438415754 0 0) 0x7f34080cafc0 con 0x7f34080c9c30 2023-12-19T18:16:27.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.590+0000 7f340e4d6640 1 --2- 172.21.15.26:0/1121259617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3408070c10 0x7f34080ccef0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f3400009e50 tx=0x7f340001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f6aa5e5e7dac13 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ccd3640 1 -- 172.21.15.26:0/1121259617 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34080ca240 con 0x7f3408070c10 2023-12-19T18:16:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ccd3640 1 -- 172.21.15.26:0/1121259617 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34000a58d0 con 0x7f3408070c10 2023-12-19T18:16:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ccd3640 1 -- 172.21.15.26:0/1121259617 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34000a5c60 con 0x7f3408070c10 2023-12-19T18:16:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 -- 172.21.15.26:0/1121259617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3408070c10 msgr2=0x7f34080ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 --2- 172.21.15.26:0/1121259617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3408070c10 0x7f34080ccef0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f3400009e50 tx=0x7f340001c9e0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 -- 172.21.15.26:0/1121259617 shutdown_connections 2023-12-19T18:16:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 --2- 172.21.15.26:0/1121259617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3408070c10 0x7f34080ccef0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 --2- 172.21.15.26:0/1121259617 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3408156e50 0x7f340815a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 -- 172.21.15.26:0/1121259617 >> 172.21.15.26:0/1121259617 conn(0x7f34080b0b60 msgr2=0x7f34080b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 -- 172.21.15.26:0/1121259617 shutdown_connections 2023-12-19T18:16:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 -- 172.21.15.26:0/1121259617 wait complete. 2023-12-19T18:16:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 Processor -- start 2023-12-19T18:16:27.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 -- start start 2023-12-19T18:16:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3408070c10 0x7f3408151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34080c9c30 0x7f34080af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3408156e50 0x7f34080afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3408152a20 con 0x7f3408070c10 2023-12-19T18:16:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3408155f60 con 0x7f3408156e50 2023-12-19T18:16:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.594+0000 7f340ff39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340806e230 con 0x7f34080c9c30 2023-12-19T18:16:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340dcd5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3408070c10 0x7f3408151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340dcd5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3408070c10 0x7f3408151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40374/0 (socket says 172.21.15.26:40374) 2023-12-19T18:16:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340dcd5640 1 -- 172.21.15.26:0/740227666 learned_addr learned my addr 172.21.15.26:0/740227666 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340d4d4640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34080c9c30 0x7f34080af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340e4d6640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3408156e50 0x7f34080afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340dcd5640 1 -- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34080c9c30 msgr2=0x7f34080af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340dcd5640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34080c9c30 0x7f34080af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340dcd5640 1 -- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3408156e50 msgr2=0x7f34080afef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340dcd5640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3408156e50 0x7f34080afef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340dcd5640 1 -- 172.21.15.26:0/740227666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3400009b30 con 0x7f3408070c10 2023-12-19T18:16:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340e4d6640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3408156e50 0x7f34080afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340d4d4640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34080c9c30 0x7f34080af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340dcd5640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3408070c10 0x7f3408151e30 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f340400e930 tx=0x7f340400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f33fe7fc640 1 -- 172.21.15.26:0/740227666 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404009690 con 0x7f3408070c10 2023-12-19T18:16:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f33fe7fc640 1 -- 172.21.15.26:0/740227666 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340400c9a0 con 0x7f3408070c10 2023-12-19T18:16:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340ff39640 1 -- 172.21.15.26:0/740227666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34081551a0 con 0x7f3408070c10 2023-12-19T18:16:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f33fe7fc640 1 -- 172.21.15.26:0/740227666 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340400cd10 con 0x7f3408070c10 2023-12-19T18:16:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340ff39640 1 -- 172.21.15.26:0/740227666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3408155760 con 0x7f3408070c10 2023-12-19T18:16:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f340ff39640 1 -- 172.21.15.26:0/740227666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3408074270 con 0x7f3408070c10 2023-12-19T18:16:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.598+0000 7f33fe7fc640 1 -- 172.21.15.26:0/740227666 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3404013470 con 0x7f3408070c10 2023-12-19T18:16:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.602+0000 7f33fe7fc640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f33ec065b80 0x7f33ec068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.602+0000 7f33fe7fc640 1 -- 172.21.15.26:0/740227666 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6369+0+0 (secure 0 0 0) 0x7f3404083fc0 con 0x7f3408070c10 2023-12-19T18:16:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.602+0000 7f340d4d4640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f33ec065b80 0x7f33ec068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.602+0000 7f340d4d4640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f33ec065b80 0x7f33ec068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f33f8007fd0 tx=0x7f33f8009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.602+0000 7f33fe7fc640 1 -- 172.21.15.26:0/740227666 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3404051670 con 0x7f3408070c10 2023-12-19T18:16:27.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.794+0000 7f340ff39640 1 -- 172.21.15.26:0/740227666 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f34080702e0 con 0x7f3408070c10 2023-12-19T18:16:27.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.802+0000 7f33fe7fc640 1 -- 172.21.15.26:0/740227666 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v92) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3404054c90 con 0x7f3408070c10 2023-12-19T18:16:27.804 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 24 and data pool 25 2023-12-19T18:16:27.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.806+0000 7f340ff39640 1 -- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f33ec065b80 msgr2=0x7f33ec068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:27.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.806+0000 7f340ff39640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f33ec065b80 0x7f33ec068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f33f8007fd0 tx=0x7f33f8009040 comp rx=0 tx=0).stop 2023-12-19T18:16:27.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.806+0000 7f340ff39640 1 -- 172.21.15.26:0/740227666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3408070c10 msgr2=0x7f3408151e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:27.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.806+0000 7f340ff39640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3408070c10 0x7f3408151e30 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f340400e930 tx=0x7f340400edf0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.806+0000 7f340ff39640 1 -- 172.21.15.26:0/740227666 shutdown_connections 2023-12-19T18:16:27.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.806+0000 7f340ff39640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f33ec065b80 0x7f33ec068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.806+0000 7f340ff39640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3408156e50 0x7f34080afef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.806+0000 7f340ff39640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34080c9c30 0x7f34080af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.806+0000 7f340ff39640 1 --2- 172.21.15.26:0/740227666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3408070c10 0x7f3408151e30 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.806+0000 7f340ff39640 1 -- 172.21.15.26:0/740227666 >> 172.21.15.26:0/740227666 conn(0x7f34080b0b60 msgr2=0x7f340806fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:27.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.806+0000 7f340ff39640 1 -- 172.21.15.26:0/740227666 shutdown_connections 2023-12-19T18:16:27.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.806+0000 7f340ff39640 1 -- 172.21.15.26:0/740227666 wait complete. 2023-12-19T18:16:27.854 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79ffa1f640 1 Processor -- start 2023-12-19T18:16:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79ffa1f640 1 -- start start 2023-12-19T18:16:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79ffa1f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f79f80b3080 0x7f79f80d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79ffa1f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f80cfb20 con 0x7f79f80b2bd0 2023-12-19T18:16:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79ffa1f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f80cfe00 con 0x7f79f815af80 2023-12-19T18:16:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79ffa1f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f80d00e0 con 0x7f79f80b3080 2023-12-19T18:16:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79fcfba640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f79f80b2bd0 0x7f79f80cc6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47060/0 (socket says 172.21.15.26:47060) 2023-12-19T18:16:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79fcfba640 1 -- 172.21.15.26:0/1428513805 learned_addr learned my addr 172.21.15.26:0/1428513805 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79fdfbc640 1 --2- 172.21.15.26:0/1428513805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f79f80b3080 0x7f79f80d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79effff640 1 -- 172.21.15.26:0/1428513805 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1087163155 0 0) 0x7f79f80cfb20 con 0x7f79f80b2bd0 2023-12-19T18:16:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79effff640 1 -- 172.21.15.26:0/1428513805 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79e4003530 con 0x7f79f80b2bd0 2023-12-19T18:16:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79effff640 1 -- 172.21.15.26:0/1428513805 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131804710 0 0) 0x7f79f80cfe00 con 0x7f79f815af80 2023-12-19T18:16:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79effff640 1 -- 172.21.15.26:0/1428513805 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f80cfb20 con 0x7f79f815af80 2023-12-19T18:16:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79effff640 1 -- 172.21.15.26:0/1428513805 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1216283832 0 0) 0x7f79e4003530 con 0x7f79f80b2bd0 2023-12-19T18:16:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79fdfbc640 1 -- 172.21.15.26:0/1428513805 >> v1:172.21.15.160:6789/0 conn(0x7f79f815af80 legacy=0x7f79f80b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79fdfbc640 1 -- 172.21.15.26:0/1428513805 >> v1:172.21.15.26:6789/0 conn(0x7f79f80b2bd0 legacy=0x7f79f80cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.994+0000 7f79fdfbc640 1 -- 172.21.15.26:0/1428513805 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f80d03c0 con 0x7f79f80b3080 2023-12-19T18:16:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.998+0000 7f79fdfbc640 1 --2- 172.21.15.26:0/1428513805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f79f80b3080 0x7f79f80d0bc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009dc0 tx=0x7f79f001c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=666d2b9c78e1620c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.998+0000 7f79effff640 1 -- 172.21.15.26:0/1428513805 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e80016a0 con 0x7f79f80b3080 2023-12-19T18:16:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.998+0000 7f79effff640 1 -- 172.21.15.26:0/1428513805 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f00a55b0 con 0x7f79f80b3080 2023-12-19T18:16:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.998+0000 7f79effff640 1 -- 172.21.15.26:0/1428513805 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f00a5920 con 0x7f79f80b3080 2023-12-19T18:16:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.998+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1428513805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f79f80b3080 msgr2=0x7f79f80d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.998+0000 7f79ffa1f640 1 --2- 172.21.15.26:0/1428513805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f79f80b3080 0x7f79f80d0bc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f79f0009dc0 tx=0x7f79f001c860 comp rx=0 tx=0).stop 2023-12-19T18:16:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.998+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1428513805 shutdown_connections 2023-12-19T18:16:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.998+0000 7f79ffa1f640 1 --2- 172.21.15.26:0/1428513805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f79f80b3080 0x7f79f80d0bc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:27.998+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1428513805 >> 172.21.15.26:0/1428513805 conn(0x7f79f806d570 msgr2=0x7f79f806d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1428513805 shutdown_connections 2023-12-19T18:16:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1428513805 wait complete. 2023-12-19T18:16:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79ffa1f640 1 Processor -- start 2023-12-19T18:16:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79ffa1f640 1 -- start start 2023-12-19T18:16:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79ffa1f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f79f80b2bd0 0x7f79f8148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79ffa1f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f79f80b3080 0x7f79f80ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79ffa1f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79f815af80 0x7f79f80b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79ffa1f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f8150f00 con 0x7f79f815af80 2023-12-19T18:16:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79ffa1f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f80ad790 con 0x7f79f80b3080 2023-12-19T18:16:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79ffa1f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f8148e50 con 0x7f79f80b2bd0 2023-12-19T18:16:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79fdfbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79f815af80 0x7f79f80b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79fdfbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79f815af80 0x7f79f80b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40382/0 (socket says 172.21.15.26:40382) 2023-12-19T18:16:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79fdfbc640 1 -- 172.21.15.26:0/1187554241 learned_addr learned my addr 172.21.15.26:0/1187554241 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79fd7bb640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f79f80b2bd0 0x7f79f8148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79fcfba640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f79f80b3080 0x7f79f80ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79fdfbc640 1 -- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f79f80b2bd0 msgr2=0x7f79f8148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79fdfbc640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f79f80b2bd0 0x7f79f8148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79fdfbc640 1 -- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f79f80b3080 msgr2=0x7f79f80ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79fdfbc640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f79f80b3080 0x7f79f80ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79fdfbc640 1 -- 172.21.15.26:0/1187554241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f0009a70 con 0x7f79f815af80 2023-12-19T18:16:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79fdfbc640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79f815af80 0x7f79f80b4c40 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f79f001af10 tx=0x7f79f0019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.002+0000 7f79edffb640 1 -- 172.21.15.26:0/1187554241 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0009450 con 0x7f79f815af80 2023-12-19T18:16:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.006+0000 7f79edffb640 1 -- 172.21.15.26:0/1187554241 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f00ad650 con 0x7f79f815af80 2023-12-19T18:16:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.006+0000 7f79edffb640 1 -- 172.21.15.26:0/1187554241 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f00ad9e0 con 0x7f79f815af80 2023-12-19T18:16:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.006+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1187554241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f80ad1d0 con 0x7f79f815af80 2023-12-19T18:16:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.006+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1187554241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f8150b20 con 0x7f79f815af80 2023-12-19T18:16:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.006+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1187554241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f8151040 con 0x7f79f815af80 2023-12-19T18:16:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.006+0000 7f79edffb640 1 -- 172.21.15.26:0/1187554241 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f79f00b4070 con 0x7f79f815af80 2023-12-19T18:16:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.006+0000 7f79edffb640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f79dc065b80 0x7f79dc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.006+0000 7f79edffb640 1 -- 172.21.15.26:0/1187554241 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f79f011da50 con 0x7f79f815af80 2023-12-19T18:16:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.006+0000 7f79fd7bb640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f79dc065b80 0x7f79dc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.010+0000 7f79fd7bb640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f79dc065b80 0x7f79dc068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79f40011f0 tx=0x7f79f4009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.010+0000 7f79edffb640 1 -- 172.21.15.26:0/1187554241 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f79f8151040 con 0x7f79f815af80 2023-12-19T18:16:28.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.170+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1187554241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79f8151040 con 0x7f79f815af80 2023-12-19T18:16:28.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.170+0000 7f79edffb640 1 -- 172.21.15.26:0/1187554241 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+17274 (secure 0 0 0) 0x7f79f8151040 con 0x7f79f815af80 2023-12-19T18:16:28.174 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:28.175 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":211,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:27.798283+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:25.863942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:26.798049+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.174+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f79dc065b80 msgr2=0x7f79dc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.174+0000 7f79ffa1f640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f79dc065b80 0x7f79dc068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79f40011f0 tx=0x7f79f4009040 comp rx=0 tx=0).stop 2023-12-19T18:16:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.174+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1187554241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79f815af80 msgr2=0x7f79f80b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.174+0000 7f79ffa1f640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79f815af80 0x7f79f80b4c40 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f79f001af10 tx=0x7f79f0019600 comp rx=0 tx=0).stop 2023-12-19T18:16:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.174+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1187554241 shutdown_connections 2023-12-19T18:16:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.174+0000 7f79ffa1f640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f79dc065b80 0x7f79dc068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.174+0000 7f79ffa1f640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79f815af80 0x7f79f80b4c40 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.174+0000 7f79ffa1f640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f79f80b3080 0x7f79f80ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.174+0000 7f79ffa1f640 1 --2- 172.21.15.26:0/1187554241 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f79f80b2bd0 0x7f79f8148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.174+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1187554241 >> 172.21.15.26:0/1187554241 conn(0x7f79f806d570 msgr2=0x7f79f80b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.174+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1187554241 shutdown_connections 2023-12-19T18:16:28.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.174+0000 7f79ffa1f640 1 -- 172.21.15.26:0/1187554241 wait complete. 2023-12-19T18:16:28.196 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2cc3bc640 1 Processor -- start 2023-12-19T18:16:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2cc3bc640 1 -- start start 2023-12-19T18:16:28.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2cc3bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2c40cbc50 0x7fb2c40b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2cc3bc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c40cc0d0 con 0x7fb2c406cb30 2023-12-19T18:16:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2cc3bc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c415ddc0 con 0x7fb2c40b3310 2023-12-19T18:16:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2cc3bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c415e0a0 con 0x7fb2c40cbc50 2023-12-19T18:16:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2ca158640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb2c406cb30 0x7fb2c415d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47064/0 (socket says 172.21.15.26:47064) 2023-12-19T18:16:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2ca158640 1 -- 172.21.15.26:0/2979991913 learned_addr learned my addr 172.21.15.26:0/2979991913 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2c9957640 1 --2- 172.21.15.26:0/2979991913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2c40cbc50 0x7fb2c40b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2c9156640 1 -- 172.21.15.26:0/2979991913 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238214301 0 0) 0x7fb2c0001120 con 0x7fb2c406cb30 2023-12-19T18:16:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2c9156640 1 -- 172.21.15.26:0/2979991913 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2b0003530 con 0x7fb2c406cb30 2023-12-19T18:16:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2c9156640 1 -- 172.21.15.26:0/2979991913 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343002889 0 0) 0x7fb2c415ddc0 con 0x7fb2c40b3310 2023-12-19T18:16:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2c9957640 1 -- 172.21.15.26:0/2979991913 >> v1:172.21.15.160:6789/0 conn(0x7fb2c40b3310 legacy=0x7fb2c40ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2c9957640 1 -- 172.21.15.26:0/2979991913 >> v1:172.21.15.26:6789/0 conn(0x7fb2c406cb30 legacy=0x7fb2c415d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.338+0000 7fb2c9957640 1 -- 172.21.15.26:0/2979991913 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c40cb450 con 0x7fb2c40cbc50 2023-12-19T18:16:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.342+0000 7fb2c9957640 1 --2- 172.21.15.26:0/2979991913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2c40cbc50 0x7fb2c40b2dd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40097f0 tx=0x7fb2b401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80c74d6eb63e3d14 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.342+0000 7fb2c9156640 1 -- 172.21.15.26:0/2979991913 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40aa610 con 0x7fb2c40cbc50 2023-12-19T18:16:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.342+0000 7fb2c9156640 1 -- 172.21.15.26:0/2979991913 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b40a6040 con 0x7fb2c40cbc50 2023-12-19T18:16:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.342+0000 7fb2c9156640 1 -- 172.21.15.26:0/2979991913 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40aa990 con 0x7fb2c40cbc50 2023-12-19T18:16:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.342+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/2979991913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2c40cbc50 msgr2=0x7fb2c40b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.342+0000 7fb2cc3bc640 1 --2- 172.21.15.26:0/2979991913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2c40cbc50 0x7fb2c40b2dd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40097f0 tx=0x7fb2b401c490 comp rx=0 tx=0).stop 2023-12-19T18:16:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.342+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/2979991913 shutdown_connections 2023-12-19T18:16:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.342+0000 7fb2cc3bc640 1 --2- 172.21.15.26:0/2979991913 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2c40cbc50 0x7fb2c40b2dd0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.342+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/2979991913 >> 172.21.15.26:0/2979991913 conn(0x7fb2c40658e0 msgr2=0x7fb2c4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.342+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/2979991913 shutdown_connections 2023-12-19T18:16:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.342+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/2979991913 wait complete. 2023-12-19T18:16:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2cc3bc640 1 Processor -- start 2023-12-19T18:16:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2cc3bc640 1 -- start start 2023-12-19T18:16:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2cc3bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2c406cb30 0x7fb2c415bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2cc3bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2c40b3310 0x7fb2c41595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2cc3bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40cbc50 0x7fb2c4155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2cc3bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c415b760 con 0x7fb2c40cbc50 2023-12-19T18:16:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2cc3bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c415ab50 con 0x7fb2c406cb30 2023-12-19T18:16:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2cc3bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c415ae30 con 0x7fb2c40b3310 2023-12-19T18:16:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2ca959640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40cbc50 0x7fb2c4155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2ca959640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40cbc50 0x7fb2c4155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40386/0 (socket says 172.21.15.26:40386) 2023-12-19T18:16:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2ca959640 1 -- 172.21.15.26:0/3159535291 learned_addr learned my addr 172.21.15.26:0/3159535291 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2ca158640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2c406cb30 0x7fb2c415bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2c9957640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2c40b3310 0x7fb2c41595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2ca959640 1 -- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2c40b3310 msgr2=0x7fb2c41595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2ca959640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2c40b3310 0x7fb2c41595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2ca959640 1 -- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2c406cb30 msgr2=0x7fb2c415bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2ca959640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2c406cb30 0x7fb2c415bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2ca959640 1 -- 172.21.15.26:0/3159535291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b4009420 con 0x7fb2c40cbc50 2023-12-19T18:16:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2c9957640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2c40b3310 0x7fb2c41595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2ca959640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40cbc50 0x7fb2c4155180 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc001570 tx=0x7fb2bc00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2baffd640 1 -- 172.21.15.26:0/3159535291 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc003db0 con 0x7fb2c40cbc50 2023-12-19T18:16:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2baffd640 1 -- 172.21.15.26:0/3159535291 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc005070 con 0x7fb2c40cbc50 2023-12-19T18:16:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/3159535291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c415b110 con 0x7fb2c40cbc50 2023-12-19T18:16:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2baffd640 1 -- 172.21.15.26:0/3159535291 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc005420 con 0x7fb2c40cbc50 2023-12-19T18:16:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.346+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/3159535291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c4159b30 con 0x7fb2c40cbc50 2023-12-19T18:16:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.350+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/3159535291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c415b8d0 con 0x7fb2c40cbc50 2023-12-19T18:16:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.350+0000 7fb2baffd640 1 -- 172.21.15.26:0/3159535291 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb2bc005580 con 0x7fb2c40cbc50 2023-12-19T18:16:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.350+0000 7fb2baffd640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb2a8065b80 0x7fb2a8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.350+0000 7fb2baffd640 1 -- 172.21.15.26:0/3159535291 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fb2bc082e10 con 0x7fb2c40cbc50 2023-12-19T18:16:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.350+0000 7fb2ca158640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb2a8065b80 0x7fb2a8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.354+0000 7fb2ca158640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb2a8065b80 0x7fb2a8068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005890 tx=0x7fb2c0009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.354+0000 7fb2baffd640 1 -- 172.21.15.26:0/3159535291 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb2bc050470 con 0x7fb2c40cbc50 2023-12-19T18:16:28.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.514+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/3159535291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2c4074270 con 0x7fb2c40cbc50 2023-12-19T18:16:28.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.518+0000 7fb2baffd640 1 -- 172.21.15.26:0/3159535291 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+17274 (secure 0 0 0) 0x7fb2bc017020 con 0x7fb2c40cbc50 2023-12-19T18:16:28.520 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:28.520 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":211,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:27.798283+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:25.863942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:26.798049+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.518+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb2a8065b80 msgr2=0x7fb2a8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.518+0000 7fb2cc3bc640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb2a8065b80 0x7fb2a8068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0005890 tx=0x7fb2c0009040 comp rx=0 tx=0).stop 2023-12-19T18:16:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.518+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/3159535291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40cbc50 msgr2=0x7fb2c4155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.518+0000 7fb2cc3bc640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40cbc50 0x7fb2c4155180 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc001570 tx=0x7fb2bc00cd20 comp rx=0 tx=0).stop 2023-12-19T18:16:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.518+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/3159535291 shutdown_connections 2023-12-19T18:16:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.518+0000 7fb2cc3bc640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb2a8065b80 0x7fb2a8068040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.518+0000 7fb2cc3bc640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2c40cbc50 0x7fb2c4155180 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.518+0000 7fb2cc3bc640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2c40b3310 0x7fb2c41595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.518+0000 7fb2cc3bc640 1 --2- 172.21.15.26:0/3159535291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2c406cb30 0x7fb2c415bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.518+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/3159535291 >> 172.21.15.26:0/3159535291 conn(0x7fb2c40658e0 msgr2=0x7fb2c40cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:28.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.518+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/3159535291 shutdown_connections 2023-12-19T18:16:28.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.518+0000 7fb2cc3bc640 1 -- 172.21.15.26:0/3159535291 wait complete. 2023-12-19T18:16:28.542 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:16:28.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f8609f39640 1 Processor -- start 2023-12-19T18:16:28.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f8609f39640 1 -- start start 2023-12-19T18:16:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f8609f39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86040ccf10 0x7f86040ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f8609f39640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f860415d250 0x7f860415d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f8609f39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86040b01b0 0x7f86040b0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f8609f39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8604157730 con 0x7f86040ccf10 2023-12-19T18:16:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f8609f39640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86040b0b80 con 0x7f860415d250 2023-12-19T18:16:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f8609f39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86040ac860 con 0x7f86040b01b0 2023-12-19T18:16:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f86037fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86040ccf10 0x7f86040ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f86037fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f86040ccf10 0x7f86040ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40402/0 (socket says 172.21.15.26:40402) 2023-12-19T18:16:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f86037fe640 1 -- 172.21.15.26:0/3730658869 learned_addr learned my addr 172.21.15.26:0/3730658869 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:28.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f8603fff640 1 --2- 172.21.15.26:0/3730658869 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86040b01b0 0x7f86040b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f8602ffd640 1 --2- 172.21.15.26:0/3730658869 >> v2:172.21.15.160:3300/0 conn(0x7f860415d250 0x7f860415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f86037fe640 1 -- 172.21.15.26:0/3730658869 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86040b01b0 msgr2=0x7f86040b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:28.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f86037fe640 1 --2- 172.21.15.26:0/3730658869 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86040b01b0 0x7f86040b0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f86037fe640 1 -- 172.21.15.26:0/3730658869 >> v2:172.21.15.160:3300/0 conn(0x7f860415d250 msgr2=0x7f860415d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:28.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f86037fe640 1 --2- 172.21.15.26:0/3730658869 >> v2:172.21.15.160:3300/0 conn(0x7f860415d250 0x7f860415d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.686+0000 7f86037fe640 1 -- 172.21.15.26:0/3730658869 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86040ad7f0 con 0x7f86040ccf10 2023-12-19T18:16:28.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.690+0000 7f86037fe640 1 --2- 172.21.15.26:0/3730658869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86040ccf10 0x7f86040ad2b0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f860406efe0 tx=0x7f85f401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326caf86b13d4766 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:28.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.690+0000 7f86027fc640 1 -- 172.21.15.26:0/3730658869 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f40a89b0 con 0x7f86040ccf10 2023-12-19T18:16:28.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.690+0000 7f86027fc640 1 -- 172.21.15.26:0/3730658869 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f40a8b10 con 0x7f86040ccf10 2023-12-19T18:16:28.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.690+0000 7f86027fc640 1 -- 172.21.15.26:0/3730658869 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f40a8ea0 con 0x7f86040ccf10 2023-12-19T18:16:28.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.690+0000 7f8609f39640 1 -- 172.21.15.26:0/3730658869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86040ccf10 msgr2=0x7f86040ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.690+0000 7f8609f39640 1 --2- 172.21.15.26:0/3730658869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86040ccf10 0x7f86040ad2b0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f860406efe0 tx=0x7f85f401ecf0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.690+0000 7f8609f39640 1 -- 172.21.15.26:0/3730658869 shutdown_connections 2023-12-19T18:16:28.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.690+0000 7f8609f39640 1 --2- 172.21.15.26:0/3730658869 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86040b01b0 0x7f86040b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.690+0000 7f8609f39640 1 --2- 172.21.15.26:0/3730658869 >> v2:172.21.15.160:3300/0 conn(0x7f860415d250 0x7f860415d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.690+0000 7f8609f39640 1 --2- 172.21.15.26:0/3730658869 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86040ccf10 0x7f86040ad2b0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.690+0000 7f8609f39640 1 -- 172.21.15.26:0/3730658869 >> 172.21.15.26:0/3730658869 conn(0x7f86040ce730 msgr2=0x7f86040ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:28.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.690+0000 7f8609f39640 1 -- 172.21.15.26:0/3730658869 shutdown_connections 2023-12-19T18:16:28.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.690+0000 7f8609f39640 1 -- 172.21.15.26:0/3730658869 wait complete. 2023-12-19T18:16:28.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8609f39640 1 Processor -- start 2023-12-19T18:16:28.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8609f39640 1 -- start start 2023-12-19T18:16:28.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8609f39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86040b01b0 0x7f860414d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8609f39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86040ccf10 0x7f860414cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8609f39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f860415d250 0x7f86040c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8609f39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86040c73d0 con 0x7f860415d250 2023-12-19T18:16:28.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8609f39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860414d170 con 0x7f86040ccf10 2023-12-19T18:16:28.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8609f39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86040c7080 con 0x7f86040b01b0 2023-12-19T18:16:28.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8603fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f860415d250 0x7f86040c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8603fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f860415d250 0x7f86040c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40414/0 (socket says 172.21.15.26:40414) 2023-12-19T18:16:28.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8603fff640 1 -- 172.21.15.26:0/904661863 learned_addr learned my addr 172.21.15.26:0/904661863 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:28.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f86037fe640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86040b01b0 0x7f860414d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8602ffd640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86040ccf10 0x7f860414cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8603fff640 1 -- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86040b01b0 msgr2=0x7f860414d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8603fff640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86040b01b0 0x7f860414d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8603fff640 1 -- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86040ccf10 msgr2=0x7f860414cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8603fff640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86040ccf10 0x7f860414cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8603fff640 1 -- 172.21.15.26:0/904661863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f40094d0 con 0x7f860415d250 2023-12-19T18:16:28.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8603fff640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f860415d250 0x7f86040c5770 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f85fc007e60 tx=0x7f85fc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:28.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f85ebfff640 1 -- 172.21.15.26:0/904661863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc014620 con 0x7f860415d250 2023-12-19T18:16:28.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f85ebfff640 1 -- 172.21.15.26:0/904661863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85fc00ee90 con 0x7f860415d250 2023-12-19T18:16:28.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f85ebfff640 1 -- 172.21.15.26:0/904661863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc0149e0 con 0x7f860415d250 2023-12-19T18:16:28.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8609f39640 1 -- 172.21.15.26:0/904661863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86040c6250 con 0x7f860415d250 2023-12-19T18:16:28.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.694+0000 7f8609f39640 1 -- 172.21.15.26:0/904661863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86040c6810 con 0x7f860415d250 2023-12-19T18:16:28.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.698+0000 7f8609f39640 1 -- 172.21.15.26:0/904661863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86040c7510 con 0x7f860415d250 2023-12-19T18:16:28.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.698+0000 7f85ebfff640 1 -- 172.21.15.26:0/904661863 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f85fc014b40 con 0x7f860415d250 2023-12-19T18:16:28.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.698+0000 7f85ebfff640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f85f0065bb0 0x7f85f0068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.698+0000 7f85ebfff640 1 -- 172.21.15.26:0/904661863 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f85fc083ce0 con 0x7f860415d250 2023-12-19T18:16:28.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.698+0000 7f86037fe640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f85f0065bb0 0x7f85f0068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.698+0000 7f86037fe640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f85f0065bb0 0x7f85f0068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f85f40b2000 tx=0x7f85f4009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:28.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.702+0000 7f85ebfff640 1 -- 172.21.15.26:0/904661863 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f86040c7510 con 0x7f860415d250 2023-12-19T18:16:28.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:28.894+0000 7f8609f39640 1 -- 172.21.15.26:0/904661863 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f86040c7510 con 0x7f860415d250 2023-12-19T18:16:29.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.810+0000 7f85ebfff640 1 -- 172.21.15.26:0/904661863 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v95) v1 ==== 118+0+0 (secure 0 0 0) 0x7f86040c7510 con 0x7f860415d250 2023-12-19T18:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.814+0000 7f8609f39640 1 -- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f85f0065bb0 msgr2=0x7f85f0068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.814+0000 7f8609f39640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f85f0065bb0 0x7f85f0068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f85f40b2000 tx=0x7f85f4009040 comp rx=0 tx=0).stop 2023-12-19T18:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.814+0000 7f8609f39640 1 -- 172.21.15.26:0/904661863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f860415d250 msgr2=0x7f86040c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.814+0000 7f8609f39640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f860415d250 0x7f86040c5770 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f85fc007e60 tx=0x7f85fc00e4b0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.814+0000 7f8609f39640 1 -- 172.21.15.26:0/904661863 shutdown_connections 2023-12-19T18:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.814+0000 7f8609f39640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f85f0065bb0 0x7f85f0068070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.814+0000 7f8609f39640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f860415d250 0x7f86040c5770 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.814+0000 7f8609f39640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f86040ccf10 0x7f860414cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.814+0000 7f8609f39640 1 --2- 172.21.15.26:0/904661863 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f86040b01b0 0x7f860414d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.814+0000 7f8609f39640 1 -- 172.21.15.26:0/904661863 >> 172.21.15.26:0/904661863 conn(0x7f86040ce730 msgr2=0x7f86040ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.814+0000 7f8609f39640 1 -- 172.21.15.26:0/904661863 shutdown_connections 2023-12-19T18:16:29.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.814+0000 7f8609f39640 1 -- 172.21.15.26:0/904661863 wait complete. 2023-12-19T18:16:29.842 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:16:29.842 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:16:29.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e08d09640 1 Processor -- start 2023-12-19T18:16:29.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e08d09640 1 -- start start 2023-12-19T18:16:29.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e08d09640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6e0415cd20 0x7f6e040b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:29.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e08d09640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040b4dd0 0x7f6e040b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:29.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e08d09640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e040b5c00 con 0x7f6e040b89c0 2023-12-19T18:16:29.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e08d09640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e0415d440 con 0x7f6e0415cd20 2023-12-19T18:16:29.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e08d09640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0415d6e0 con 0x7f6e040b4dd0 2023-12-19T18:16:29.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6dfbfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6e040b89c0 0x7f6e040b46c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47072/0 (socket says 172.21.15.26:47072) 2023-12-19T18:16:29.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6dfbfff640 1 -- 172.21.15.26:0/3421083684 learned_addr learned my addr 172.21.15.26:0/3421083684 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e02d9d640 1 --2- 172.21.15.26:0/3421083684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040b4dd0 0x7f6e040b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e0259c640 1 --2- 172.21.15.26:0/3421083684 >> v2:172.21.15.160:3300/0 conn(0x7f6e0415cd20 0x7f6e040b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796390815 0 0) 0x7f6e040b5c00 con 0x7f6e040b89c0 2023-12-19T18:16:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6df0003530 con 0x7f6e040b89c0 2023-12-19T18:16:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 155291053 0 0) 0x7f6df0003530 con 0x7f6e040b89c0 2023-12-19T18:16:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040b4dd0 msgr2=0x7f6e040b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e01d9b640 1 --2- 172.21.15.26:0/3421083684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040b4dd0 0x7f6e040b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 >> v2:172.21.15.160:3300/0 conn(0x7f6e0415cd20 msgr2=0x7f6e040b8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e01d9b640 1 --2- 172.21.15.26:0/3421083684 >> v2:172.21.15.160:3300/0 conn(0x7f6e0415cd20 0x7f6e040b8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0415d9c0 con 0x7f6e040b89c0 2023-12-19T18:16:29.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f6dec001680 con 0x7f6e040b89c0 2023-12-19T18:16:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.990+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 >> v1:172.21.15.26:6789/0 conn(0x7f6e040b89c0 legacy=0x7f6e040b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e01d9b640 1 --2- 172.21.15.26:0/3421083684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0003980 0x7f6df0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e01d9b640 1 --2- 172.21.15.26:0/3421083684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6df00047d0 0x7f6df0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e01d9b640 1 --2- 172.21.15.26:0/3421083684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040b4dd0 0x7f6df0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0415d440 con 0x7f6df0003980 2023-12-19T18:16:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0415d6e0 con 0x7f6df00047d0 2023-12-19T18:16:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df000b9c0 con 0x7f6e040b4dd0 2023-12-19T18:16:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e0259c640 1 --2- 172.21.15.26:0/3421083684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0003980 0x7f6df0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6dfbfff640 1 --2- 172.21.15.26:0/3421083684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6df00047d0 0x7f6df0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e02d9d640 1 --2- 172.21.15.26:0/3421083684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040b4dd0 0x7f6df0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e0259c640 1 -- 172.21.15.26:0/3421083684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040b4dd0 msgr2=0x7f6df0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e0259c640 1 --2- 172.21.15.26:0/3421083684 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040b4dd0 0x7f6df0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e0259c640 1 -- 172.21.15.26:0/3421083684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6df00047d0 msgr2=0x7f6df0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:29.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e0259c640 1 --2- 172.21.15.26:0/3421083684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6df00047d0 0x7f6df0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e0259c640 1 -- 172.21.15.26:0/3421083684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df0003530 con 0x7f6df0003980 2023-12-19T18:16:29.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e0259c640 1 --2- 172.21.15.26:0/3421083684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0003980 0x7f6df0003da0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009ae0 tx=0x7f6df401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b0945720f3be0c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:29.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df40a55a0 con 0x7f6df0003980 2023-12-19T18:16:29.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df40a5700 con 0x7f6df0003980 2023-12-19T18:16:29.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.994+0000 7f6e01d9b640 1 -- 172.21.15.26:0/3421083684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df40a5ab0 con 0x7f6df0003980 2023-12-19T18:16:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 -- 172.21.15.26:0/3421083684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0003980 msgr2=0x7f6df0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 --2- 172.21.15.26:0/3421083684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6df0003980 0x7f6df0003da0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009ae0 tx=0x7f6df401ca90 comp rx=0 tx=0).stop 2023-12-19T18:16:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e02d9d640 1 -- 172.21.15.26:0/3421083684 reap_dead start 2023-12-19T18:16:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 -- 172.21.15.26:0/3421083684 shutdown_connections 2023-12-19T18:16:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 -- 172.21.15.26:0/3421083684 >> 172.21.15.26:0/3421083684 conn(0x7f6e0406a120 msgr2=0x7f6e0406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 -- 172.21.15.26:0/3421083684 shutdown_connections 2023-12-19T18:16:30.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 -- 172.21.15.26:0/3421083684 wait complete. 2023-12-19T18:16:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 Processor -- start 2023-12-19T18:16:30.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 -- start start 2023-12-19T18:16:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6df0003980 0x7f6e040c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040c6580 0x7f6e040cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e040b5cb0 0x7f6e040b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040b6670 con 0x7f6e040b5cb0 2023-12-19T18:16:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040c6a00 con 0x7f6df0003980 2023-12-19T18:16:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e08d09640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040c83c0 con 0x7f6e040c6580 2023-12-19T18:16:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e02d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e040b5cb0 0x7f6e040b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e02d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e040b5cb0 0x7f6e040b6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40422/0 (socket says 172.21.15.26:40422) 2023-12-19T18:16:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:29.998+0000 7f6e02d9d640 1 -- 172.21.15.26:0/4232500030 learned_addr learned my addr 172.21.15.26:0/4232500030 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:30.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6dfbfff640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040c6580 0x7f6e040cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6e0259c640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6df0003980 0x7f6e040c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6e02d9d640 1 -- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040c6580 msgr2=0x7f6e040cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6e02d9d640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040c6580 0x7f6e040cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6e02d9d640 1 -- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6df0003980 msgr2=0x7f6e040c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6e02d9d640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6df0003980 0x7f6e040c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6e02d9d640 1 -- 172.21.15.26:0/4232500030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e040ccca0 con 0x7f6e040b5cb0 2023-12-19T18:16:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6dfbfff640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040c6580 0x7f6e040cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6e02d9d640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e040b5cb0 0x7f6e040b6130 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f6e04161950 tx=0x7f6dfc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6dfaffd640 1 -- 172.21.15.26:0/4232500030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc00e040 con 0x7f6e040b5cb0 2023-12-19T18:16:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6dfaffd640 1 -- 172.21.15.26:0/4232500030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc00f040 con 0x7f6e040b5cb0 2023-12-19T18:16:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6e08d09640 1 -- 172.21.15.26:0/4232500030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e040c7ce0 con 0x7f6e040b5cb0 2023-12-19T18:16:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6dfaffd640 1 -- 172.21.15.26:0/4232500030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc003c00 con 0x7f6e040b5cb0 2023-12-19T18:16:30.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6e08d09640 1 -- 172.21.15.26:0/4232500030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e040c6f20 con 0x7f6e040b5cb0 2023-12-19T18:16:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6e08d09640 1 -- 172.21.15.26:0/4232500030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e04074270 con 0x7f6e040b5cb0 2023-12-19T18:16:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6dfaffd640 1 -- 172.21.15.26:0/4232500030 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6dfc004030 con 0x7f6e040b5cb0 2023-12-19T18:16:30.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6dfaffd640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6de4065b80 0x7f6de4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.002+0000 7f6dfaffd640 1 -- 172.21.15.26:0/4232500030 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f6dfc083ef0 con 0x7f6e040b5cb0 2023-12-19T18:16:30.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.006+0000 7f6e0259c640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6de4065b80 0x7f6de4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.006+0000 7f6e0259c640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6de4065b80 0x7f6de4068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6df40af000 tx=0x7f6df40046d0 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:30.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.010+0000 7f6dfaffd640 1 -- 172.21.15.26:0/4232500030 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6dfc0512f0 con 0x7f6e040b5cb0 2023-12-19T18:16:30.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:30.198+0000 7f6e08d09640 1 -- 172.21.15.26:0/4232500030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6e0415aab0 con 0x7f6e040b5cb0 2023-12-19T18:16:31.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.078+0000 7f6dfaffd640 1 -- 172.21.15.26:0/4232500030 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v96) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6dfc054910 con 0x7f6e040b5cb0 2023-12-19T18:16:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.078+0000 7f6e08d09640 1 -- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6de4065b80 msgr2=0x7f6de4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.078+0000 7f6e08d09640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6de4065b80 0x7f6de4068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6df40af000 tx=0x7f6df40046d0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.078+0000 7f6e08d09640 1 -- 172.21.15.26:0/4232500030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e040b5cb0 msgr2=0x7f6e040b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.078+0000 7f6e08d09640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e040b5cb0 0x7f6e040b6130 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f6e04161950 tx=0x7f6dfc00f580 comp rx=0 tx=0).stop 2023-12-19T18:16:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.078+0000 7f6e08d09640 1 -- 172.21.15.26:0/4232500030 shutdown_connections 2023-12-19T18:16:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.078+0000 7f6e08d09640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6de4065b80 0x7f6de4068040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.078+0000 7f6e08d09640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e040b5cb0 0x7f6e040b6130 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.078+0000 7f6e08d09640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e040c6580 0x7f6e040cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.078+0000 7f6e08d09640 1 --2- 172.21.15.26:0/4232500030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6df0003980 0x7f6e040c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.078+0000 7f6e08d09640 1 -- 172.21.15.26:0/4232500030 >> 172.21.15.26:0/4232500030 conn(0x7f6e0406a120 msgr2=0x7f6e040d2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.078+0000 7f6e08d09640 1 -- 172.21.15.26:0/4232500030 shutdown_connections 2023-12-19T18:16:31.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.078+0000 7f6e08d09640 1 -- 172.21.15.26:0/4232500030 wait complete. 2023-12-19T18:16:31.106 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:16:31.106 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:31.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.250+0000 7f1dccd6d640 1 Processor -- start 2023-12-19T18:16:31.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.250+0000 7f1dccd6d640 1 -- start start 2023-12-19T18:16:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dccd6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1dc806b3d0 0x7f1dc80ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dccd6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dc80cec40 0x7f1dc815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dccd6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dc806bc30 con 0x7f1dc8154220 2023-12-19T18:16:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dccd6d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dc80cf0d0 con 0x7f1dc806b3d0 2023-12-19T18:16:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dccd6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80cc6e0 con 0x7f1dc80cec40 2023-12-19T18:16:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dbffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1dc806b3d0 0x7f1dc80ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dbffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1dc806b3d0 0x7f1dc80ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40432/0 (socket says 172.21.15.26:40432) 2023-12-19T18:16:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dbffff640 1 -- 172.21.15.26:0/112750367 learned_addr learned my addr 172.21.15.26:0/112750367 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:31.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dc6d9d640 1 --2- 172.21.15.26:0/112750367 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dc80cec40 0x7f1dc815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dbffff640 1 -- 172.21.15.26:0/112750367 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dc80cec40 msgr2=0x7f1dc815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dbffff640 1 --2- 172.21.15.26:0/112750367 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dc80cec40 0x7f1dc815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dbffff640 1 -- 172.21.15.26:0/112750367 >> v1:172.21.15.26:6789/0 conn(0x7f1dc8154220 legacy=0x7f1dc815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dbffff640 1 -- 172.21.15.26:0/112750367 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc806b950 con 0x7f1dc806b3d0 2023-12-19T18:16:31.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.254+0000 7f1dc5d9b640 1 -- 172.21.15.26:0/112750367 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158607356 0 0) 0x7f1dc806bc30 con 0x7f1dc8154220 2023-12-19T18:16:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dbffff640 1 --2- 172.21.15.26:0/112750367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dc806b3d0 0x7f1dc80ce700 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f1db000bb10 tx=0x7f1db001ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b23d5328eaeb9c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dc5d9b640 1 -- 172.21.15.26:0/112750367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db00a7cb0 con 0x7f1dc806b3d0 2023-12-19T18:16:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dc5d9b640 1 -- 172.21.15.26:0/112750367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db00a7e10 con 0x7f1dc806b3d0 2023-12-19T18:16:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dc5d9b640 1 -- 172.21.15.26:0/112750367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db00af660 con 0x7f1dc806b3d0 2023-12-19T18:16:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dccd6d640 1 -- 172.21.15.26:0/112750367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dc806b3d0 msgr2=0x7f1dc80ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dccd6d640 1 --2- 172.21.15.26:0/112750367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dc806b3d0 0x7f1dc80ce700 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f1db000bb10 tx=0x7f1db001ebe0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dccd6d640 1 -- 172.21.15.26:0/112750367 shutdown_connections 2023-12-19T18:16:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dccd6d640 1 --2- 172.21.15.26:0/112750367 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dc80cec40 0x7f1dc815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dccd6d640 1 --2- 172.21.15.26:0/112750367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dc806b3d0 0x7f1dc80ce700 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dccd6d640 1 -- 172.21.15.26:0/112750367 >> 172.21.15.26:0/112750367 conn(0x7f1dc80658e0 msgr2=0x7f1dc8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dccd6d640 1 -- 172.21.15.26:0/112750367 shutdown_connections 2023-12-19T18:16:31.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dccd6d640 1 -- 172.21.15.26:0/112750367 wait complete. 2023-12-19T18:16:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dccd6d640 1 Processor -- start 2023-12-19T18:16:31.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.258+0000 7f1dccd6d640 1 -- start start 2023-12-19T18:16:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dccd6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dc806b3d0 0x7f1dc80b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dccd6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dc80cec40 0x7f1dc81f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dccd6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dc8154220 0x7f1dc80acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dccd6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80b19a0 con 0x7f1dc806b3d0 2023-12-19T18:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dccd6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc813ecf0 con 0x7f1dc8154220 2023-12-19T18:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dccd6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc80712a0 con 0x7f1dc80cec40 2023-12-19T18:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dc659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dc806b3d0 0x7f1dc80b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dc659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dc806b3d0 0x7f1dc80b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40436/0 (socket says 172.21.15.26:40436) 2023-12-19T18:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dc659c640 1 -- 172.21.15.26:0/3878717817 learned_addr learned my addr 172.21.15.26:0/3878717817 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dc6d9d640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dc8154220 0x7f1dc80acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dbffff640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dc80cec40 0x7f1dc81f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dc659c640 1 -- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dc80cec40 msgr2=0x7f1dc81f92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dc659c640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dc80cec40 0x7f1dc81f92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dc659c640 1 -- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dc8154220 msgr2=0x7f1dc80acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dc659c640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dc8154220 0x7f1dc80acab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dc659c640 1 -- 172.21.15.26:0/3878717817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db000b770 con 0x7f1dc806b3d0 2023-12-19T18:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dc6d9d640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dc8154220 0x7f1dc80acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dc659c640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dc806b3d0 0x7f1dc80b1460 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c450 tx=0x7f1db800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dbeffd640 1 -- 172.21.15.26:0/3878717817 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8005480 con 0x7f1dc806b3d0 2023-12-19T18:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dbeffd640 1 -- 172.21.15.26:0/3878717817 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1db8003600 con 0x7f1dc806b3d0 2023-12-19T18:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dbeffd640 1 -- 172.21.15.26:0/3878717817 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db80039b0 con 0x7f1dc806b3d0 2023-12-19T18:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dccd6d640 1 -- 172.21.15.26:0/3878717817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc814d4c0 con 0x7f1dc806b3d0 2023-12-19T18:16:31.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dccd6d640 1 -- 172.21.15.26:0/3878717817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc814d1e0 con 0x7f1dc806b3d0 2023-12-19T18:16:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dccd6d640 1 -- 172.21.15.26:0/3878717817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc8074270 con 0x7f1dc806b3d0 2023-12-19T18:16:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.262+0000 7f1dbeffd640 1 -- 172.21.15.26:0/3878717817 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1db8003b10 con 0x7f1dc806b3d0 2023-12-19T18:16:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.266+0000 7f1dbeffd640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1da8065b80 0x7f1da8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.266+0000 7f1dbeffd640 1 -- 172.21.15.26:0/3878717817 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f1db8082b70 con 0x7f1dc806b3d0 2023-12-19T18:16:31.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.266+0000 7f1dbffff640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1da8065b80 0x7f1da8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.266+0000 7f1dbffff640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1da8065b80 0x7f1da8068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1db00b1000 tx=0x7f1db0008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.266+0000 7f1dbeffd640 1 -- 172.21.15.26:0/3878717817 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1db8050120 con 0x7f1dc806b3d0 2023-12-19T18:16:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dccd6d640 1 -- 172.21.15.26:0/3878717817 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dc80cdf60 con 0x7f1dc806b3d0 2023-12-19T18:16:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dbeffd640 1 -- 172.21.15.26:0/3878717817 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4725 (secure 0 0 0) 0x7f1db8017070 con 0x7f1dc806b3d0 2023-12-19T18:16:31.461 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:31.461 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3349262044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3349262044},{"type":"v1","addr":"172.21.15.160:6836","nonce":3349262044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3865658274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3865658274},{"type":"v1","addr":"172.21.15.26:6833","nonce":3865658274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10605,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1294752100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1294752100},{"type":"v1","addr":"172.21.15.160:6837","nonce":1294752100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:27.798614+0000","modified":"2023-12-19T18:16:31.076294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3981286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3981286992},{"type":"v1","addr":"172.21.15.26:6835","nonce":3981286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-12-19T18:16:31.461 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 96 2023-12-19T18:16:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dccd6d640 1 -- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1da8065b80 msgr2=0x7f1da8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dccd6d640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1da8065b80 0x7f1da8068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1db00b1000 tx=0x7f1db0008040 comp rx=0 tx=0).stop 2023-12-19T18:16:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dccd6d640 1 -- 172.21.15.26:0/3878717817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dc806b3d0 msgr2=0x7f1dc80b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dccd6d640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dc806b3d0 0x7f1dc80b1460 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c450 tx=0x7f1db800c910 comp rx=0 tx=0).stop 2023-12-19T18:16:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dccd6d640 1 -- 172.21.15.26:0/3878717817 shutdown_connections 2023-12-19T18:16:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dccd6d640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1da8065b80 0x7f1da8068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dccd6d640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1dc8154220 0x7f1dc80acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dccd6d640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1dc80cec40 0x7f1dc81f92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dccd6d640 1 --2- 172.21.15.26:0/3878717817 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dc806b3d0 0x7f1dc80b1460 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dccd6d640 1 -- 172.21.15.26:0/3878717817 >> 172.21.15.26:0/3878717817 conn(0x7f1dc80658e0 msgr2=0x7f1dc814f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dccd6d640 1 -- 172.21.15.26:0/3878717817 shutdown_connections 2023-12-19T18:16:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.458+0000 7f1dccd6d640 1 -- 172.21.15.26:0/3878717817 wait complete. 2023-12-19T18:16:31.482 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.622+0000 7fba97e20640 1 Processor -- start 2023-12-19T18:16:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.622+0000 7fba97e20640 1 -- start start 2023-12-19T18:16:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.626+0000 7fba97e20640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba900cee10 0x7fba90164980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.626+0000 7fba97e20640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9006bc30 con 0x7fba90154220 2023-12-19T18:16:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.626+0000 7fba97e20640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba900cc6e0 con 0x7fba9006b3d0 2023-12-19T18:16:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.626+0000 7fba97e20640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9006b950 con 0x7fba900cee10 2023-12-19T18:16:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.626+0000 7fba95bbc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fba90154220 0x7fba9015aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47080/0 (socket says 172.21.15.26:47080) 2023-12-19T18:16:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.626+0000 7fba95bbc640 1 -- 172.21.15.26:0/2687168233 learned_addr learned my addr 172.21.15.26:0/2687168233 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.626+0000 7fba963bd640 1 --2- 172.21.15.26:0/2687168233 >> v2:172.21.15.26:3300/0 conn(0x7fba900cee10 0x7fba90164980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.626+0000 7fba94bba640 1 -- 172.21.15.26:0/2687168233 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226991928 0 0) 0x7fba9006bc30 con 0x7fba90154220 2023-12-19T18:16:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.626+0000 7fba94bba640 1 -- 172.21.15.26:0/2687168233 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba7c003530 con 0x7fba90154220 2023-12-19T18:16:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.626+0000 7fba963bd640 1 -- 172.21.15.26:0/2687168233 >> v1:172.21.15.160:6789/0 conn(0x7fba9006b3d0 legacy=0x7fba900ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.626+0000 7fba963bd640 1 -- 172.21.15.26:0/2687168233 >> v1:172.21.15.26:6789/0 conn(0x7fba90154220 legacy=0x7fba9015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.626+0000 7fba963bd640 1 -- 172.21.15.26:0/2687168233 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba900cc930 con 0x7fba900cee10 2023-12-19T18:16:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.626+0000 7fba963bd640 1 --2- 172.21.15.26:0/2687168233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba900cee10 0x7fba90164980 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fba88009d90 tx=0x7fba8801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941b4bd8c7b65b31 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba94bba640 1 -- 172.21.15.26:0/2687168233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba880a59e0 con 0x7fba900cee10 2023-12-19T18:16:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba94bba640 1 -- 172.21.15.26:0/2687168233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba880a5b40 con 0x7fba900cee10 2023-12-19T18:16:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba94bba640 1 -- 172.21.15.26:0/2687168233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba880ad440 con 0x7fba900cee10 2023-12-19T18:16:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 -- 172.21.15.26:0/2687168233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba900cee10 msgr2=0x7fba90164980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 --2- 172.21.15.26:0/2687168233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba900cee10 0x7fba90164980 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fba88009d90 tx=0x7fba8801c920 comp rx=0 tx=0).stop 2023-12-19T18:16:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 -- 172.21.15.26:0/2687168233 shutdown_connections 2023-12-19T18:16:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 --2- 172.21.15.26:0/2687168233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba900cee10 0x7fba90164980 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 -- 172.21.15.26:0/2687168233 >> 172.21.15.26:0/2687168233 conn(0x7fba900658e0 msgr2=0x7fba90065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 -- 172.21.15.26:0/2687168233 shutdown_connections 2023-12-19T18:16:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 -- 172.21.15.26:0/2687168233 wait complete. 2023-12-19T18:16:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 Processor -- start 2023-12-19T18:16:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 -- start start 2023-12-19T18:16:31.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba9006b3d0 0x7fba90070e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fba900cee10 0x7fba90158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba90154220 0x7fba901592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba90154690 con 0x7fba9006b3d0 2023-12-19T18:16:31.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba900ae120 con 0x7fba90154220 2023-12-19T18:16:31.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba97e20640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba900ad510 con 0x7fba900cee10 2023-12-19T18:16:31.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba95bbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba9006b3d0 0x7fba90070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba95bbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba9006b3d0 0x7fba90070e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40462/0 (socket says 172.21.15.26:40462) 2023-12-19T18:16:31.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.630+0000 7fba95bbc640 1 -- 172.21.15.26:0/2622951891 learned_addr learned my addr 172.21.15.26:0/2622951891 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:31.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba95bbc640 1 -- 172.21.15.26:0/2622951891 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fba900cee10 msgr2=0x7fba90158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba95bbc640 1 --2- 172.21.15.26:0/2622951891 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fba900cee10 0x7fba90158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba95bbc640 1 -- 172.21.15.26:0/2622951891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba90154220 msgr2=0x7fba901592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba95bbc640 1 --2- 172.21.15.26:0/2622951891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba90154220 0x7fba901592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba95bbc640 1 -- 172.21.15.26:0/2622951891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba88009a70 con 0x7fba9006b3d0 2023-12-19T18:16:31.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba95bbc640 1 --2- 172.21.15.26:0/2622951891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba9006b3d0 0x7fba90070e40 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00c420 tx=0x7fba8c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba867fc640 1 -- 172.21.15.26:0/2622951891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c005cf0 con 0x7fba9006b3d0 2023-12-19T18:16:31.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba867fc640 1 -- 172.21.15.26:0/2622951891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c005e50 con 0x7fba9006b3d0 2023-12-19T18:16:31.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba97e20640 1 -- 172.21.15.26:0/2622951891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba900ad7f0 con 0x7fba9006b3d0 2023-12-19T18:16:31.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba867fc640 1 -- 172.21.15.26:0/2622951891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0072f0 con 0x7fba9006b3d0 2023-12-19T18:16:31.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba97e20640 1 -- 172.21.15.26:0/2622951891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba900addb0 con 0x7fba9006b3d0 2023-12-19T18:16:31.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba97e20640 1 -- 172.21.15.26:0/2622951891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba90074270 con 0x7fba9006b3d0 2023-12-19T18:16:31.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba867fc640 1 -- 172.21.15.26:0/2622951891 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fba8c007450 con 0x7fba9006b3d0 2023-12-19T18:16:31.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.634+0000 7fba867fc640 1 --2- 172.21.15.26:0/2622951891 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fba74065b80 0x7fba74068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.638+0000 7fba867fc640 1 -- 172.21.15.26:0/2622951891 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fba8c082ca0 con 0x7fba9006b3d0 2023-12-19T18:16:31.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.638+0000 7fba953bb640 1 --2- 172.21.15.26:0/2622951891 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fba74065b80 0x7fba74068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.638+0000 7fba953bb640 1 --2- 172.21.15.26:0/2622951891 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fba74065b80 0x7fba74068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fba80000f70 tx=0x7fba800079e0 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.642+0000 7fba867fc640 1 -- 172.21.15.26:0/2622951891 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fba8c053c40 con 0x7fba9006b3d0 2023-12-19T18:16:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba97e20640 1 -- 172.21.15.26:0/2622951891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba900cdf60 con 0x7fba9006b3d0 2023-12-19T18:16:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba867fc640 1 -- 172.21.15.26:0/2622951891 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v212) v1 ==== 74+0+17261 (secure 0 0 0) 0x7fba8c07ae60 con 0x7fba9006b3d0 2023-12-19T18:16:31.805 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:31.805 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":212,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:28.801422+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:25.863942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:26.798049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba97e20640 1 -- 172.21.15.26:0/2622951891 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fba74065b80 msgr2=0x7fba74068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba97e20640 1 --2- 172.21.15.26:0/2622951891 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fba74065b80 0x7fba74068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fba80000f70 tx=0x7fba800079e0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba97e20640 1 -- 172.21.15.26:0/2622951891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba9006b3d0 msgr2=0x7fba90070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba97e20640 1 --2- 172.21.15.26:0/2622951891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba9006b3d0 0x7fba90070e40 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fba8c00c420 tx=0x7fba8c00c8e0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba97e20640 1 -- 172.21.15.26:0/2622951891 shutdown_connections 2023-12-19T18:16:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba97e20640 1 --2- 172.21.15.26:0/2622951891 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fba74065b80 0x7fba74068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba97e20640 1 --2- 172.21.15.26:0/2622951891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba90154220 0x7fba901592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba97e20640 1 --2- 172.21.15.26:0/2622951891 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fba900cee10 0x7fba90158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba97e20640 1 --2- 172.21.15.26:0/2622951891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba9006b3d0 0x7fba90070e40 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba97e20640 1 -- 172.21.15.26:0/2622951891 >> 172.21.15.26:0/2622951891 conn(0x7fba900658e0 msgr2=0x7fba900ac850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba97e20640 1 -- 172.21.15.26:0/2622951891 shutdown_connections 2023-12-19T18:16:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.802+0000 7fba97e20640 1 -- 172.21.15.26:0/2622951891 wait complete. 2023-12-19T18:16:31.826 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:16:31.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:16:31.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.966+0000 7f8ba9e66640 1 Processor -- start 2023-12-19T18:16:31.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.966+0000 7f8ba9e66640 1 -- start start 2023-12-19T18:16:31.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.970+0000 7f8ba9e66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ba406b3d0 0x7f8ba40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.970+0000 7f8ba9e66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba406bc30 con 0x7f8ba4154220 2023-12-19T18:16:31.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.970+0000 7f8ba9e66640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba40cc6e0 con 0x7f8ba40cec40 2023-12-19T18:16:31.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.970+0000 7f8ba9e66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba40cf0d0 con 0x7f8ba406b3d0 2023-12-19T18:16:31.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.970+0000 7f8ba2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ba406b3d0 0x7f8ba40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.970+0000 7f8ba37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8ba4154220 0x7f8ba415aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47088/0 (socket says 172.21.15.26:47088) 2023-12-19T18:16:31.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.970+0000 7f8ba2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ba406b3d0 0x7f8ba40ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40478/0 (socket says 172.21.15.26:40478) 2023-12-19T18:16:31.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.970+0000 7f8ba37fe640 1 -- 172.21.15.26:0/2675692605 learned_addr learned my addr 172.21.15.26:0/2675692605 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:31.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.970+0000 7f8ba2ffd640 1 -- 172.21.15.26:0/2675692605 >> v1:172.21.15.160:6789/0 conn(0x7f8ba40cec40 legacy=0x7f8ba415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.970+0000 7f8ba2ffd640 1 -- 172.21.15.26:0/2675692605 >> v1:172.21.15.26:6789/0 conn(0x7f8ba4154220 legacy=0x7f8ba415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.970+0000 7f8ba2ffd640 1 -- 172.21.15.26:0/2675692605 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba406b950 con 0x7f8ba406b3d0 2023-12-19T18:16:31.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.970+0000 7f8ba2ffd640 1 --2- 172.21.15.26:0/2675692605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba406b3d0 0x7f8ba40ce700 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f8b980097c0 tx=0x7f8b9801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6876eb1b0cb1c28 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba27fc640 1 -- 172.21.15.26:0/2675692605 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b980ab650 con 0x7f8ba406b3d0 2023-12-19T18:16:31.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba27fc640 1 -- 172.21.15.26:0/2675692605 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b980ab7b0 con 0x7f8ba406b3d0 2023-12-19T18:16:31.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba27fc640 1 -- 172.21.15.26:0/2675692605 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b980abb40 con 0x7f8ba406b3d0 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 -- 172.21.15.26:0/2675692605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba406b3d0 msgr2=0x7f8ba40ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 --2- 172.21.15.26:0/2675692605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba406b3d0 0x7f8ba40ce700 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f8b980097c0 tx=0x7f8b9801c450 comp rx=0 tx=0).stop 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 -- 172.21.15.26:0/2675692605 shutdown_connections 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 --2- 172.21.15.26:0/2675692605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba406b3d0 0x7f8ba40ce700 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 -- 172.21.15.26:0/2675692605 >> 172.21.15.26:0/2675692605 conn(0x7f8ba40658e0 msgr2=0x7f8ba4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 -- 172.21.15.26:0/2675692605 shutdown_connections 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 -- 172.21.15.26:0/2675692605 wait complete. 2023-12-19T18:16:31.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 Processor -- start 2023-12-19T18:16:31.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 -- start start 2023-12-19T18:16:31.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ba406b3d0 0x7f8ba4149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba40cec40 0x7f8ba414aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba4154220 0x7f8ba4158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba414a5d0 con 0x7f8ba4154220 2023-12-19T18:16:31.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba414d360 con 0x7f8ba406b3d0 2023-12-19T18:16:31.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.974+0000 7f8ba9e66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba414b1c0 con 0x7f8ba40cec40 2023-12-19T18:16:31.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba4154220 0x7f8ba4158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba37fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ba406b3d0 0x7f8ba4149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba4154220 0x7f8ba4158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40482/0 (socket says 172.21.15.26:40482) 2023-12-19T18:16:31.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba3fff640 1 -- 172.21.15.26:0/826098585 learned_addr learned my addr 172.21.15.26:0/826098585 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:31.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba2ffd640 1 --2- 172.21.15.26:0/826098585 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba40cec40 0x7f8ba414aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba3fff640 1 -- 172.21.15.26:0/826098585 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba40cec40 msgr2=0x7f8ba414aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba3fff640 1 --2- 172.21.15.26:0/826098585 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba40cec40 0x7f8ba414aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba3fff640 1 -- 172.21.15.26:0/826098585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ba406b3d0 msgr2=0x7f8ba4149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba3fff640 1 --2- 172.21.15.26:0/826098585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ba406b3d0 0x7f8ba4149cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba3fff640 1 -- 172.21.15.26:0/826098585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b98009420 con 0x7f8ba4154220 2023-12-19T18:16:31.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba3fff640 1 --2- 172.21.15.26:0/826098585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba4154220 0x7f8ba4158ed0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00c7a0 tx=0x7f8b9c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8b8bfff640 1 -- 172.21.15.26:0/826098585 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c003b80 con 0x7f8ba4154220 2023-12-19T18:16:31.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8b8bfff640 1 -- 172.21.15.26:0/826098585 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c003ce0 con 0x7f8ba4154220 2023-12-19T18:16:31.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8b8bfff640 1 -- 172.21.15.26:0/826098585 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c00aa90 con 0x7f8ba4154220 2023-12-19T18:16:31.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba9e66640 1 -- 172.21.15.26:0/826098585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba414d590 con 0x7f8ba4154220 2023-12-19T18:16:31.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba9e66640 1 -- 172.21.15.26:0/826098585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba414db50 con 0x7f8ba4154220 2023-12-19T18:16:31.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8ba9e66640 1 -- 172.21.15.26:0/826098585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba4074270 con 0x7f8ba4154220 2023-12-19T18:16:31.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8b8bfff640 1 -- 172.21.15.26:0/826098585 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8b9c00abf0 con 0x7f8ba4154220 2023-12-19T18:16:31.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.978+0000 7f8b8bfff640 1 --2- 172.21.15.26:0/826098585 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8b90065b80 0x7f8b90068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.982+0000 7f8b8bfff640 1 -- 172.21.15.26:0/826098585 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f8b9c0833b0 con 0x7f8ba4154220 2023-12-19T18:16:31.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.982+0000 7f8ba37fe640 1 --2- 172.21.15.26:0/826098585 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8b90065b80 0x7f8b90068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.982+0000 7f8ba37fe640 1 --2- 172.21.15.26:0/826098585 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8b90065b80 0x7f8b90068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b94005890 tx=0x7f8b94009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:31.982+0000 7f8b8bfff640 1 -- 172.21.15.26:0/826098585 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8b9c050970 con 0x7f8ba4154220 2023-12-19T18:16:32.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.138+0000 7f8ba9e66640 1 -- 172.21.15.26:0/826098585 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ba40ccef0 con 0x7f8b90065b80 2023-12-19T18:16:32.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.150+0000 7f8b8bfff640 1 -- 172.21.15.26:0/826098585 <== mgr.9805 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289476 (secure 0 0 0) 0x7f8b94001530 con 0x7f8b90065b80 2023-12-19T18:16:32.156 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:32.158 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:16:32.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.154+0000 7f8ba9e66640 1 -- 172.21.15.26:0/826098585 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8b90065b80 msgr2=0x7f8b90068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.154+0000 7f8ba9e66640 1 --2- 172.21.15.26:0/826098585 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8b90065b80 0x7f8b90068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b94005890 tx=0x7f8b94009040 comp rx=0 tx=0).stop 2023-12-19T18:16:32.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.154+0000 7f8ba9e66640 1 -- 172.21.15.26:0/826098585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba4154220 msgr2=0x7f8ba4158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.154+0000 7f8ba9e66640 1 --2- 172.21.15.26:0/826098585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba4154220 0x7f8ba4158ed0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00c7a0 tx=0x7f8b9c00cc60 comp rx=0 tx=0).stop 2023-12-19T18:16:32.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.154+0000 7f8ba9e66640 1 -- 172.21.15.26:0/826098585 shutdown_connections 2023-12-19T18:16:32.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.154+0000 7f8ba9e66640 1 --2- 172.21.15.26:0/826098585 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8b90065b80 0x7f8b90068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.154+0000 7f8ba9e66640 1 --2- 172.21.15.26:0/826098585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ba4154220 0x7f8ba4158ed0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.154+0000 7f8ba9e66640 1 --2- 172.21.15.26:0/826098585 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8ba40cec40 0x7f8ba414aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.154+0000 7f8ba9e66640 1 --2- 172.21.15.26:0/826098585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8ba406b3d0 0x7f8ba4149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.154+0000 7f8ba9e66640 1 -- 172.21.15.26:0/826098585 >> 172.21.15.26:0/826098585 conn(0x7f8ba40658e0 msgr2=0x7f8ba415d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:32.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.154+0000 7f8ba9e66640 1 -- 172.21.15.26:0/826098585 shutdown_connections 2023-12-19T18:16:32.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.154+0000 7f8ba9e66640 1 -- 172.21.15.26:0/826098585 wait complete. 2023-12-19T18:16:32.176 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-12-19T18:16:31.304517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":619,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":384,"ondisk_log_size":384,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422684,"kb_used_data":21724,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754552036,"statfs":{"total":773094113280,"available":772661284864,"internally_reserved":0,"allocated":22245376,"data_stored":12290640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.917534"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.836370+0000","last_change":"2023-12-19T18:16:27.834221+0000","last_active":"2023-12-19T18:16:27.836370+0000","last_peered":"2023-12-19T18:16:27.836370+0000","last_clean":"2023-12-19T18:16:27.836370+0000","last_became_active":"2023-12-19T18:16:27.825886+0000","last_became_peered":"2023-12-19T18:16:27.825886+0000","last_unstale":"2023-12-19T18:16:27.836370+0000","last_undegraded":"2023-12-19T18:16:27.836370+0000","last_fullsized":"2023-12-19T18:16:27.836370+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:12.119996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834634+0000","last_change":"2023-12-19T18:16:27.831511+0000","last_active":"2023-12-19T18:16:27.834634+0000","last_peered":"2023-12-19T18:16:27.834634+0000","last_clean":"2023-12-19T18:16:27.834634+0000","last_became_active":"2023-12-19T18:16:27.831255+0000","last_became_peered":"2023-12-19T18:16:27.831255+0000","last_unstale":"2023-12-19T18:16:27.834634+0000","last_undegraded":"2023-12-19T18:16:27.834634+0000","last_fullsized":"2023-12-19T18:16:27.834634+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:42.382254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.820569+0000","last_change":"2023-12-19T18:16:27.815730+0000","last_active":"2023-12-19T18:16:27.820569+0000","last_peered":"2023-12-19T18:16:27.820569+0000","last_clean":"2023-12-19T18:16:27.820569+0000","last_became_active":"2023-12-19T18:16:27.815341+0000","last_became_peered":"2023-12-19T18:16:27.815341+0000","last_unstale":"2023-12-19T18:16:27.820569+0000","last_undegraded":"2023-12-19T18:16:27.820569+0000","last_fullsized":"2023-12-19T18:16:27.820569+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:39.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834527+0000","last_change":"2023-12-19T18:16:27.830449+0000","last_active":"2023-12-19T18:16:27.834527+0000","last_peered":"2023-12-19T18:16:27.834527+0000","last_clean":"2023-12-19T18:16:27.834527+0000","last_became_active":"2023-12-19T18:16:27.830189+0000","last_became_peered":"2023-12-19T18:16:27.830189+0000","last_unstale":"2023-12-19T18:16:27.834527+0000","last_undegraded":"2023-12-19T18:16:27.834527+0000","last_fullsized":"2023-12-19T18:16:27.834527+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:06.397912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837876+0000","last_change":"2023-12-19T18:16:27.835673+0000","last_active":"2023-12-19T18:16:27.837876+0000","last_peered":"2023-12-19T18:16:27.837876+0000","last_clean":"2023-12-19T18:16:27.837876+0000","last_became_active":"2023-12-19T18:16:27.835392+0000","last_became_peered":"2023-12-19T18:16:27.835392+0000","last_unstale":"2023-12-19T18:16:27.837876+0000","last_undegraded":"2023-12-19T18:16:27.837876+0000","last_fullsized":"2023-12-19T18:16:27.837876+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:23:06.990327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.829676+0000","last_change":"2023-12-19T18:16:27.822167+0000","last_active":"2023-12-19T18:16:27.829676+0000","last_peered":"2023-12-19T18:16:27.829676+0000","last_clean":"2023-12-19T18:16:27.829676+0000","last_became_active":"2023-12-19T18:16:27.821752+0000","last_became_peered":"2023-12-19T18:16:27.821752+0000","last_unstale":"2023-12-19T18:16:27.829676+0000","last_undegraded":"2023-12-19T18:16:27.829676+0000","last_fullsized":"2023-12-19T18:16:27.829676+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:17.866181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.828067+0000","last_change":"2023-12-19T18:16:27.823852+0000","last_active":"2023-12-19T18:16:27.828067+0000","last_peered":"2023-12-19T18:16:27.828067+0000","last_clean":"2023-12-19T18:16:27.828067+0000","last_became_active":"2023-12-19T18:16:27.823564+0000","last_became_peered":"2023-12-19T18:16:27.823564+0000","last_unstale":"2023-12-19T18:16:27.828067+0000","last_undegraded":"2023-12-19T18:16:27.828067+0000","last_fullsized":"2023-12-19T18:16:27.828067+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:59.000440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.836791+0000","last_change":"2023-12-19T18:16:27.825730+0000","last_active":"2023-12-19T18:16:27.836791+0000","last_peered":"2023-12-19T18:16:27.836791+0000","last_clean":"2023-12-19T18:16:27.836791+0000","last_became_active":"2023-12-19T18:16:27.825136+0000","last_became_peered":"2023-12-19T18:16:27.825136+0000","last_unstale":"2023-12-19T18:16:27.836791+0000","last_undegraded":"2023-12-19T18:16:27.836791+0000","last_fullsized":"2023-12-19T18:16:27.836791+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:48.330735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.835843+0000","last_change":"2023-12-19T18:16:27.832580+0000","last_active":"2023-12-19T18:16:27.835843+0000","last_peered":"2023-12-19T18:16:27.835843+0000","last_clean":"2023-12-19T18:16:27.835843+0000","last_became_active":"2023-12-19T18:16:27.832291+0000","last_became_peered":"2023-12-19T18:16:27.832291+0000","last_unstale":"2023-12-19T18:16:27.835843+0000","last_undegraded":"2023-12-19T18:16:27.835843+0000","last_fullsized":"2023-12-19T18:16:27.835843+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:29.305405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.827726+0000","last_change":"2023-12-19T18:16:27.819945+0000","last_active":"2023-12-19T18:16:27.827726+0000","last_peered":"2023-12-19T18:16:27.827726+0000","last_clean":"2023-12-19T18:16:27.827726+0000","last_became_active":"2023-12-19T18:16:27.819634+0000","last_became_peered":"2023-12-19T18:16:27.819634+0000","last_unstale":"2023-12-19T18:16:27.827726+0000","last_undegraded":"2023-12-19T18:16:27.827726+0000","last_fullsized":"2023-12-19T18:16:27.827726+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:22.401852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.824952+0000","last_change":"2023-12-19T18:16:27.819797+0000","last_active":"2023-12-19T18:16:27.824952+0000","last_peered":"2023-12-19T18:16:27.824952+0000","last_clean":"2023-12-19T18:16:27.824952+0000","last_became_active":"2023-12-19T18:16:27.819507+0000","last_became_peered":"2023-12-19T18:16:27.819507+0000","last_unstale":"2023-12-19T18:16:27.824952+0000","last_undegraded":"2023-12-19T18:16:27.824952+0000","last_fullsized":"2023-12-19T18:16:27.824952+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:08.287716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834510+0000","last_change":"2023-12-19T18:16:27.829943+0000","last_active":"2023-12-19T18:16:27.834510+0000","last_peered":"2023-12-19T18:16:27.834510+0000","last_clean":"2023-12-19T18:16:27.834510+0000","last_became_active":"2023-12-19T18:16:27.829592+0000","last_became_peered":"2023-12-19T18:16:27.829592+0000","last_unstale":"2023-12-19T18:16:27.834510+0000","last_undegraded":"2023-12-19T18:16:27.834510+0000","last_fullsized":"2023-12-19T18:16:27.834510+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:46.245564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834280+0000","last_change":"2023-12-19T18:16:27.829736+0000","last_active":"2023-12-19T18:16:27.834280+0000","last_peered":"2023-12-19T18:16:27.834280+0000","last_clean":"2023-12-19T18:16:27.834280+0000","last_became_active":"2023-12-19T18:16:27.829448+0000","last_became_peered":"2023-12-19T18:16:27.829448+0000","last_unstale":"2023-12-19T18:16:27.834280+0000","last_undegraded":"2023-12-19T18:16:27.834280+0000","last_fullsized":"2023-12-19T18:16:27.834280+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:47.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.827082+0000","last_change":"2023-12-19T18:16:27.823186+0000","last_active":"2023-12-19T18:16:27.827082+0000","last_peered":"2023-12-19T18:16:27.827082+0000","last_clean":"2023-12-19T18:16:27.827082+0000","last_became_active":"2023-12-19T18:16:27.820405+0000","last_became_peered":"2023-12-19T18:16:27.820405+0000","last_unstale":"2023-12-19T18:16:27.827082+0000","last_undegraded":"2023-12-19T18:16:27.827082+0000","last_fullsized":"2023-12-19T18:16:27.827082+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:41.515700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834653+0000","last_change":"2023-12-19T18:16:27.830751+0000","last_active":"2023-12-19T18:16:27.834653+0000","last_peered":"2023-12-19T18:16:27.834653+0000","last_clean":"2023-12-19T18:16:27.834653+0000","last_became_active":"2023-12-19T18:16:27.830490+0000","last_became_peered":"2023-12-19T18:16:27.830490+0000","last_unstale":"2023-12-19T18:16:27.834653+0000","last_undegraded":"2023-12-19T18:16:27.834653+0000","last_fullsized":"2023-12-19T18:16:27.834653+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:31.093343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837674+0000","last_change":"2023-12-19T18:16:27.826828+0000","last_active":"2023-12-19T18:16:27.837674+0000","last_peered":"2023-12-19T18:16:27.837674+0000","last_clean":"2023-12-19T18:16:27.837674+0000","last_became_active":"2023-12-19T18:16:27.825994+0000","last_became_peered":"2023-12-19T18:16:27.825994+0000","last_unstale":"2023-12-19T18:16:27.837674+0000","last_undegraded":"2023-12-19T18:16:27.837674+0000","last_fullsized":"2023-12-19T18:16:27.837674+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:17.748218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837797+0000","last_change":"2023-12-19T18:16:27.834880+0000","last_active":"2023-12-19T18:16:27.837797+0000","last_peered":"2023-12-19T18:16:27.837797+0000","last_clean":"2023-12-19T18:16:27.837797+0000","last_became_active":"2023-12-19T18:16:27.834494+0000","last_became_peered":"2023-12-19T18:16:27.834494+0000","last_unstale":"2023-12-19T18:16:27.837797+0000","last_undegraded":"2023-12-19T18:16:27.837797+0000","last_fullsized":"2023-12-19T18:16:27.837797+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:06.992701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.822991+0000","last_change":"2023-12-19T18:16:27.820781+0000","last_active":"2023-12-19T18:16:27.822991+0000","last_peered":"2023-12-19T18:16:27.822991+0000","last_clean":"2023-12-19T18:16:27.822991+0000","last_became_active":"2023-12-19T18:16:27.820483+0000","last_became_peered":"2023-12-19T18:16:27.820483+0000","last_unstale":"2023-12-19T18:16:27.822991+0000","last_undegraded":"2023-12-19T18:16:27.822991+0000","last_fullsized":"2023-12-19T18:16:27.822991+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:23.445435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.827776+0000","last_change":"2023-12-19T18:16:27.820194+0000","last_active":"2023-12-19T18:16:27.827776+0000","last_peered":"2023-12-19T18:16:27.827776+0000","last_clean":"2023-12-19T18:16:27.827776+0000","last_became_active":"2023-12-19T18:16:27.819900+0000","last_became_peered":"2023-12-19T18:16:27.819900+0000","last_unstale":"2023-12-19T18:16:27.827776+0000","last_undegraded":"2023-12-19T18:16:27.827776+0000","last_fullsized":"2023-12-19T18:16:27.827776+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:00.858729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837984+0000","last_change":"2023-12-19T18:16:27.833054+0000","last_active":"2023-12-19T18:16:27.837984+0000","last_peered":"2023-12-19T18:16:27.837984+0000","last_clean":"2023-12-19T18:16:27.837984+0000","last_became_active":"2023-12-19T18:16:27.832738+0000","last_became_peered":"2023-12-19T18:16:27.832738+0000","last_unstale":"2023-12-19T18:16:27.837984+0000","last_undegraded":"2023-12-19T18:16:27.837984+0000","last_fullsized":"2023-12-19T18:16:27.837984+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:57.892685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834012+0000","last_change":"2023-12-19T18:16:27.831551+0000","last_active":"2023-12-19T18:16:27.834012+0000","last_peered":"2023-12-19T18:16:27.834012+0000","last_clean":"2023-12-19T18:16:27.834012+0000","last_became_active":"2023-12-19T18:16:27.831291+0000","last_became_peered":"2023-12-19T18:16:27.831291+0000","last_unstale":"2023-12-19T18:16:27.834012+0000","last_undegraded":"2023-12-19T18:16:27.834012+0000","last_fullsized":"2023-12-19T18:16:27.834012+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:09.914084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837462+0000","last_change":"2023-12-19T18:16:27.830084+0000","last_active":"2023-12-19T18:16:27.837462+0000","last_peered":"2023-12-19T18:16:27.837462+0000","last_clean":"2023-12-19T18:16:27.837462+0000","last_became_active":"2023-12-19T18:16:27.829787+0000","last_became_peered":"2023-12-19T18:16:27.829787+0000","last_unstale":"2023-12-19T18:16:27.837462+0000","last_undegraded":"2023-12-19T18:16:27.837462+0000","last_fullsized":"2023-12-19T18:16:27.837462+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:34:16.356894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.831781+0000","last_change":"2023-12-19T18:16:27.826145+0000","last_active":"2023-12-19T18:16:27.831781+0000","last_peered":"2023-12-19T18:16:27.831781+0000","last_clean":"2023-12-19T18:16:27.831781+0000","last_became_active":"2023-12-19T18:16:27.825889+0000","last_became_peered":"2023-12-19T18:16:27.825889+0000","last_unstale":"2023-12-19T18:16:27.831781+0000","last_undegraded":"2023-12-19T18:16:27.831781+0000","last_fullsized":"2023-12-19T18:16:27.831781+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:59:41.673539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837653+0000","last_change":"2023-12-19T18:16:27.826642+0000","last_active":"2023-12-19T18:16:27.837653+0000","last_peered":"2023-12-19T18:16:27.837653+0000","last_clean":"2023-12-19T18:16:27.837653+0000","last_became_active":"2023-12-19T18:16:27.825460+0000","last_became_peered":"2023-12-19T18:16:27.825460+0000","last_unstale":"2023-12-19T18:16:27.837653+0000","last_undegraded":"2023-12-19T18:16:27.837653+0000","last_fullsized":"2023-12-19T18:16:27.837653+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:05.671602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.836036+0000","last_change":"2023-12-19T18:16:27.834875+0000","last_active":"2023-12-19T18:16:27.836036+0000","last_peered":"2023-12-19T18:16:27.836036+0000","last_clean":"2023-12-19T18:16:27.836036+0000","last_became_active":"2023-12-19T18:16:27.834561+0000","last_became_peered":"2023-12-19T18:16:27.834561+0000","last_unstale":"2023-12-19T18:16:27.836036+0000","last_undegraded":"2023-12-19T18:16:27.836036+0000","last_fullsized":"2023-12-19T18:16:27.836036+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:30.301379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.833793+0000","last_change":"2023-12-19T18:16:27.830119+0000","last_active":"2023-12-19T18:16:27.833793+0000","last_peered":"2023-12-19T18:16:27.833793+0000","last_clean":"2023-12-19T18:16:27.833793+0000","last_became_active":"2023-12-19T18:16:27.829850+0000","last_became_peered":"2023-12-19T18:16:27.829850+0000","last_unstale":"2023-12-19T18:16:27.833793+0000","last_undegraded":"2023-12-19T18:16:27.833793+0000","last_fullsized":"2023-12-19T18:16:27.833793+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:51.190615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.828863+0000","last_change":"2023-12-19T18:16:27.820913+0000","last_active":"2023-12-19T18:16:27.828863+0000","last_peered":"2023-12-19T18:16:27.828863+0000","last_clean":"2023-12-19T18:16:27.828863+0000","last_became_active":"2023-12-19T18:16:27.820634+0000","last_became_peered":"2023-12-19T18:16:27.820634+0000","last_unstale":"2023-12-19T18:16:27.828863+0000","last_undegraded":"2023-12-19T18:16:27.828863+0000","last_fullsized":"2023-12-19T18:16:27.828863+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:53:09.066279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834337+0000","last_change":"2023-12-19T18:16:27.830103+0000","last_active":"2023-12-19T18:16:27.834337+0000","last_peered":"2023-12-19T18:16:27.834337+0000","last_clean":"2023-12-19T18:16:27.834337+0000","last_became_active":"2023-12-19T18:16:27.829444+0000","last_became_peered":"2023-12-19T18:16:27.829444+0000","last_unstale":"2023-12-19T18:16:27.834337+0000","last_undegraded":"2023-12-19T18:16:27.834337+0000","last_fullsized":"2023-12-19T18:16:27.834337+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:50:29.947932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834019+0000","last_change":"2023-12-19T18:16:27.826500+0000","last_active":"2023-12-19T18:16:27.834019+0000","last_peered":"2023-12-19T18:16:27.834019+0000","last_clean":"2023-12-19T18:16:27.834019+0000","last_became_active":"2023-12-19T18:16:27.826216+0000","last_became_peered":"2023-12-19T18:16:27.826216+0000","last_unstale":"2023-12-19T18:16:27.834019+0000","last_undegraded":"2023-12-19T18:16:27.834019+0000","last_fullsized":"2023-12-19T18:16:27.834019+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:33.032362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.830779+0000","last_change":"2023-12-19T18:16:27.825014+0000","last_active":"2023-12-19T18:16:27.830779+0000","last_peered":"2023-12-19T18:16:27.830779+0000","last_clean":"2023-12-19T18:16:27.830779+0000","last_became_active":"2023-12-19T18:16:27.822140+0000","last_became_peered":"2023-12-19T18:16:27.822140+0000","last_unstale":"2023-12-19T18:16:27.830779+0000","last_undegraded":"2023-12-19T18:16:27.830779+0000","last_fullsized":"2023-12-19T18:16:27.830779+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:36.972809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834899+0000","last_change":"2023-12-19T18:16:27.833518+0000","last_active":"2023-12-19T18:16:27.834899+0000","last_peered":"2023-12-19T18:16:27.834899+0000","last_clean":"2023-12-19T18:16:27.834899+0000","last_became_active":"2023-12-19T18:16:27.833251+0000","last_became_peered":"2023-12-19T18:16:27.833251+0000","last_unstale":"2023-12-19T18:16:27.834899+0000","last_undegraded":"2023-12-19T18:16:27.834899+0000","last_fullsized":"2023-12-19T18:16:27.834899+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:49.276917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.836070+0000","last_change":"2023-12-19T18:16:27.831030+0000","last_active":"2023-12-19T18:16:27.836070+0000","last_peered":"2023-12-19T18:16:27.836070+0000","last_clean":"2023-12-19T18:16:27.836070+0000","last_became_active":"2023-12-19T18:16:27.830764+0000","last_became_peered":"2023-12-19T18:16:27.830764+0000","last_unstale":"2023-12-19T18:16:27.836070+0000","last_undegraded":"2023-12-19T18:16:27.836070+0000","last_fullsized":"2023-12-19T18:16:27.836070+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:11.685663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.839371+0000","last_change":"2023-12-19T18:16:27.830378+0000","last_active":"2023-12-19T18:16:27.839371+0000","last_peered":"2023-12-19T18:16:27.839371+0000","last_clean":"2023-12-19T18:16:27.839371+0000","last_became_active":"2023-12-19T18:16:27.830122+0000","last_became_peered":"2023-12-19T18:16:27.830122+0000","last_unstale":"2023-12-19T18:16:27.839371+0000","last_undegraded":"2023-12-19T18:16:27.839371+0000","last_fullsized":"2023-12-19T18:16:27.839371+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:21.365516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"211'4","reported_seq":14,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.835298+0000","last_change":"2023-12-19T18:16:27.414364+0000","last_active":"2023-12-19T18:16:27.835298+0000","last_peered":"2023-12-19T18:16:27.835298+0000","last_clean":"2023-12-19T18:16:27.835298+0000","last_became_active":"2023-12-19T18:16:27.413648+0000","last_became_peered":"2023-12-19T18:16:27.413648+0000","last_unstale":"2023-12-19T18:16:27.835298+0000","last_undegraded":"2023-12-19T18:16:27.835298+0000","last_fullsized":"2023-12-19T18:16:27.835298+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:27:32.703908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837160+0000","last_change":"2023-12-19T18:16:27.830766+0000","last_active":"2023-12-19T18:16:27.837160+0000","last_peered":"2023-12-19T18:16:27.837160+0000","last_clean":"2023-12-19T18:16:27.837160+0000","last_became_active":"2023-12-19T18:16:27.830220+0000","last_became_peered":"2023-12-19T18:16:27.830220+0000","last_unstale":"2023-12-19T18:16:27.837160+0000","last_undegraded":"2023-12-19T18:16:27.837160+0000","last_fullsized":"2023-12-19T18:16:27.837160+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:03:17.772391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"211'1","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.831972+0000","last_change":"2023-12-19T18:16:27.407669+0000","last_active":"2023-12-19T18:16:27.831972+0000","last_peered":"2023-12-19T18:16:27.831972+0000","last_clean":"2023-12-19T18:16:27.831972+0000","last_became_active":"2023-12-19T18:16:27.407150+0000","last_became_peered":"2023-12-19T18:16:27.407150+0000","last_unstale":"2023-12-19T18:16:27.831972+0000","last_undegraded":"2023-12-19T18:16:27.831972+0000","last_fullsized":"2023-12-19T18:16:27.831972+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:59.675090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.833517+0000","last_change":"2023-12-19T18:16:27.829031+0000","last_active":"2023-12-19T18:16:27.833517+0000","last_peered":"2023-12-19T18:16:27.833517+0000","last_clean":"2023-12-19T18:16:27.833517+0000","last_became_active":"2023-12-19T18:16:27.826634+0000","last_became_peered":"2023-12-19T18:16:27.826634+0000","last_unstale":"2023-12-19T18:16:27.833517+0000","last_undegraded":"2023-12-19T18:16:27.833517+0000","last_fullsized":"2023-12-19T18:16:27.833517+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:23.429228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"211'2","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834535+0000","last_change":"2023-12-19T18:16:27.415013+0000","last_active":"2023-12-19T18:16:27.834535+0000","last_peered":"2023-12-19T18:16:27.834535+0000","last_clean":"2023-12-19T18:16:27.834535+0000","last_became_active":"2023-12-19T18:16:27.414582+0000","last_became_peered":"2023-12-19T18:16:27.414582+0000","last_unstale":"2023-12-19T18:16:27.834535+0000","last_undegraded":"2023-12-19T18:16:27.834535+0000","last_fullsized":"2023-12-19T18:16:27.834535+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:34:01.457288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.814928+0000","last_change":"2023-12-19T18:16:27.814063+0000","last_active":"2023-12-19T18:16:27.814928+0000","last_peered":"2023-12-19T18:16:27.814928+0000","last_clean":"2023-12-19T18:16:27.814928+0000","last_became_active":"2023-12-19T18:16:27.813564+0000","last_became_peered":"2023-12-19T18:16:27.813564+0000","last_unstale":"2023-12-19T18:16:27.814928+0000","last_undegraded":"2023-12-19T18:16:27.814928+0000","last_fullsized":"2023-12-19T18:16:27.814928+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:36.733148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"211'2","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834457+0000","last_change":"2023-12-19T18:16:27.409441+0000","last_active":"2023-12-19T18:16:27.834457+0000","last_peered":"2023-12-19T18:16:27.834457+0000","last_clean":"2023-12-19T18:16:27.834457+0000","last_became_active":"2023-12-19T18:16:27.408438+0000","last_became_peered":"2023-12-19T18:16:27.408438+0000","last_unstale":"2023-12-19T18:16:27.834457+0000","last_undegraded":"2023-12-19T18:16:27.834457+0000","last_fullsized":"2023-12-19T18:16:27.834457+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:40.980300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837446+0000","last_change":"2023-12-19T18:16:27.828398+0000","last_active":"2023-12-19T18:16:27.837446+0000","last_peered":"2023-12-19T18:16:27.837446+0000","last_clean":"2023-12-19T18:16:27.837446+0000","last_became_active":"2023-12-19T18:16:27.828082+0000","last_became_peered":"2023-12-19T18:16:27.828082+0000","last_unstale":"2023-12-19T18:16:27.837446+0000","last_undegraded":"2023-12-19T18:16:27.837446+0000","last_fullsized":"2023-12-19T18:16:27.837446+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:42:30.998150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"211'1","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837387+0000","last_change":"2023-12-19T18:16:27.415036+0000","last_active":"2023-12-19T18:16:27.837387+0000","last_peered":"2023-12-19T18:16:27.837387+0000","last_clean":"2023-12-19T18:16:27.837387+0000","last_became_active":"2023-12-19T18:16:27.414747+0000","last_became_peered":"2023-12-19T18:16:27.414747+0000","last_unstale":"2023-12-19T18:16:27.837387+0000","last_undegraded":"2023-12-19T18:16:27.837387+0000","last_fullsized":"2023-12-19T18:16:27.837387+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:03.064138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.836984+0000","last_change":"2023-12-19T18:16:27.827386+0000","last_active":"2023-12-19T18:16:27.836984+0000","last_peered":"2023-12-19T18:16:27.836984+0000","last_clean":"2023-12-19T18:16:27.836984+0000","last_became_active":"2023-12-19T18:16:27.827086+0000","last_became_peered":"2023-12-19T18:16:27.827086+0000","last_unstale":"2023-12-19T18:16:27.836984+0000","last_undegraded":"2023-12-19T18:16:27.836984+0000","last_fullsized":"2023-12-19T18:16:27.836984+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:37.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"211'1","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.831347+0000","last_change":"2023-12-19T18:16:27.414372+0000","last_active":"2023-12-19T18:16:27.831347+0000","last_peered":"2023-12-19T18:16:27.831347+0000","last_clean":"2023-12-19T18:16:27.831347+0000","last_became_active":"2023-12-19T18:16:27.414088+0000","last_became_peered":"2023-12-19T18:16:27.414088+0000","last_unstale":"2023-12-19T18:16:27.831347+0000","last_undegraded":"2023-12-19T18:16:27.831347+0000","last_fullsized":"2023-12-19T18:16:27.831347+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:18.945766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.817641+0000","last_change":"2023-12-19T18:16:27.816735+0000","last_active":"2023-12-19T18:16:27.817641+0000","last_peered":"2023-12-19T18:16:27.817641+0000","last_clean":"2023-12-19T18:16:27.817641+0000","last_became_active":"2023-12-19T18:16:27.816436+0000","last_became_peered":"2023-12-19T18:16:27.816436+0000","last_unstale":"2023-12-19T18:16:27.817641+0000","last_undegraded":"2023-12-19T18:16:27.817641+0000","last_fullsized":"2023-12-19T18:16:27.817641+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:37.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.811238+0000","last_change":"2023-12-19T18:16:27.413444+0000","last_active":"2023-12-19T18:16:27.811238+0000","last_peered":"2023-12-19T18:16:27.811238+0000","last_clean":"2023-12-19T18:16:27.811238+0000","last_became_active":"2023-12-19T18:16:27.413066+0000","last_became_peered":"2023-12-19T18:16:27.413066+0000","last_unstale":"2023-12-19T18:16:27.811238+0000","last_undegraded":"2023-12-19T18:16:27.811238+0000","last_fullsized":"2023-12-19T18:16:27.811238+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:57.997657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834600+0000","last_change":"2023-12-19T18:16:27.830538+0000","last_active":"2023-12-19T18:16:27.834600+0000","last_peered":"2023-12-19T18:16:27.834600+0000","last_clean":"2023-12-19T18:16:27.834600+0000","last_became_active":"2023-12-19T18:16:27.828974+0000","last_became_peered":"2023-12-19T18:16:27.828974+0000","last_unstale":"2023-12-19T18:16:27.834600+0000","last_undegraded":"2023-12-19T18:16:27.834600+0000","last_fullsized":"2023-12-19T18:16:27.834600+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:36.961741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.809625+0000","last_change":"2023-12-19T18:16:27.405556+0000","last_active":"2023-12-19T18:16:27.809625+0000","last_peered":"2023-12-19T18:16:27.809625+0000","last_clean":"2023-12-19T18:16:27.809625+0000","last_became_active":"2023-12-19T18:16:27.404804+0000","last_became_peered":"2023-12-19T18:16:27.404804+0000","last_unstale":"2023-12-19T18:16:27.809625+0000","last_undegraded":"2023-12-19T18:16:27.809625+0000","last_fullsized":"2023-12-19T18:16:27.809625+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:05.395130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.839394+0000","last_change":"2023-12-19T18:16:27.835564+0000","last_active":"2023-12-19T18:16:27.839394+0000","last_peered":"2023-12-19T18:16:27.839394+0000","last_clean":"2023-12-19T18:16:27.839394+0000","last_became_active":"2023-12-19T18:16:27.835319+0000","last_became_peered":"2023-12-19T18:16:27.835319+0000","last_unstale":"2023-12-19T18:16:27.839394+0000","last_undegraded":"2023-12-19T18:16:27.839394+0000","last_fullsized":"2023-12-19T18:16:27.839394+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:10:35.037185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.820369+0000","last_change":"2023-12-19T18:16:27.410603+0000","last_active":"2023-12-19T18:16:27.820369+0000","last_peered":"2023-12-19T18:16:27.820369+0000","last_clean":"2023-12-19T18:16:27.820369+0000","last_became_active":"2023-12-19T18:16:27.409260+0000","last_became_peered":"2023-12-19T18:16:27.409260+0000","last_unstale":"2023-12-19T18:16:27.820369+0000","last_undegraded":"2023-12-19T18:16:27.820369+0000","last_fullsized":"2023-12-19T18:16:27.820369+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:06.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.812914+0000","last_change":"2023-12-19T18:16:27.811752+0000","last_active":"2023-12-19T18:16:27.812914+0000","last_peered":"2023-12-19T18:16:27.812914+0000","last_clean":"2023-12-19T18:16:27.812914+0000","last_became_active":"2023-12-19T18:16:27.811273+0000","last_became_peered":"2023-12-19T18:16:27.811273+0000","last_unstale":"2023-12-19T18:16:27.812914+0000","last_undegraded":"2023-12-19T18:16:27.812914+0000","last_fullsized":"2023-12-19T18:16:27.812914+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:23.609660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"211'3","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.832083+0000","last_change":"2023-12-19T18:16:27.409442+0000","last_active":"2023-12-19T18:16:27.832083+0000","last_peered":"2023-12-19T18:16:27.832083+0000","last_clean":"2023-12-19T18:16:27.832083+0000","last_became_active":"2023-12-19T18:16:27.409094+0000","last_became_peered":"2023-12-19T18:16:27.409094+0000","last_unstale":"2023-12-19T18:16:27.832083+0000","last_undegraded":"2023-12-19T18:16:27.832083+0000","last_fullsized":"2023-12-19T18:16:27.832083+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:18.713070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837195+0000","last_change":"2023-12-19T18:16:27.829311+0000","last_active":"2023-12-19T18:16:27.837195+0000","last_peered":"2023-12-19T18:16:27.837195+0000","last_clean":"2023-12-19T18:16:27.837195+0000","last_became_active":"2023-12-19T18:16:27.828953+0000","last_became_peered":"2023-12-19T18:16:27.828953+0000","last_unstale":"2023-12-19T18:16:27.837195+0000","last_undegraded":"2023-12-19T18:16:27.837195+0000","last_fullsized":"2023-12-19T18:16:27.837195+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:23.417710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"211'6","reported_seq":16,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.839649+0000","last_change":"2023-12-19T18:16:27.414573+0000","last_active":"2023-12-19T18:16:27.839649+0000","last_peered":"2023-12-19T18:16:27.839649+0000","last_clean":"2023-12-19T18:16:27.839649+0000","last_became_active":"2023-12-19T18:16:27.414268+0000","last_became_peered":"2023-12-19T18:16:27.414268+0000","last_unstale":"2023-12-19T18:16:27.839649+0000","last_undegraded":"2023-12-19T18:16:27.839649+0000","last_fullsized":"2023-12-19T18:16:27.839649+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:11.117214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.829693+0000","last_change":"2023-12-19T18:16:27.824833+0000","last_active":"2023-12-19T18:16:27.829693+0000","last_peered":"2023-12-19T18:16:27.829693+0000","last_clean":"2023-12-19T18:16:27.829693+0000","last_became_active":"2023-12-19T18:16:27.824487+0000","last_became_peered":"2023-12-19T18:16:27.824487+0000","last_unstale":"2023-12-19T18:16:27.829693+0000","last_undegraded":"2023-12-19T18:16:27.829693+0000","last_fullsized":"2023-12-19T18:16:27.829693+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"211'2","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.838393+0000","last_change":"2023-12-19T18:16:27.415000+0000","last_active":"2023-12-19T18:16:27.838393+0000","last_peered":"2023-12-19T18:16:27.838393+0000","last_clean":"2023-12-19T18:16:27.838393+0000","last_became_active":"2023-12-19T18:16:27.414679+0000","last_became_peered":"2023-12-19T18:16:27.414679+0000","last_unstale":"2023-12-19T18:16:27.838393+0000","last_undegraded":"2023-12-19T18:16:27.838393+0000","last_fullsized":"2023-12-19T18:16:27.838393+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:50.187832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.831349+0000","last_change":"2023-12-19T18:16:27.824516+0000","last_active":"2023-12-19T18:16:27.831349+0000","last_peered":"2023-12-19T18:16:27.831349+0000","last_clean":"2023-12-19T18:16:27.831349+0000","last_became_active":"2023-12-19T18:16:27.823143+0000","last_became_peered":"2023-12-19T18:16:27.823143+0000","last_unstale":"2023-12-19T18:16:27.831349+0000","last_undegraded":"2023-12-19T18:16:27.831349+0000","last_fullsized":"2023-12-19T18:16:27.831349+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:52.341930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.804347+0000","last_change":"2023-12-19T18:16:27.417401+0000","last_active":"2023-12-19T18:16:27.804347+0000","last_peered":"2023-12-19T18:16:27.804347+0000","last_clean":"2023-12-19T18:16:27.804347+0000","last_became_active":"2023-12-19T18:16:27.416774+0000","last_became_peered":"2023-12-19T18:16:27.416774+0000","last_unstale":"2023-12-19T18:16:27.804347+0000","last_undegraded":"2023-12-19T18:16:27.804347+0000","last_fullsized":"2023-12-19T18:16:27.804347+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:03.620596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.827050+0000","last_change":"2023-12-19T18:16:27.821814+0000","last_active":"2023-12-19T18:16:27.827050+0000","last_peered":"2023-12-19T18:16:27.827050+0000","last_clean":"2023-12-19T18:16:27.827050+0000","last_became_active":"2023-12-19T18:16:27.821519+0000","last_became_peered":"2023-12-19T18:16:27.821519+0000","last_unstale":"2023-12-19T18:16:27.827050+0000","last_undegraded":"2023-12-19T18:16:27.827050+0000","last_fullsized":"2023-12-19T18:16:27.827050+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:40.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.813482+0000","last_change":"2023-12-19T18:16:27.403403+0000","last_active":"2023-12-19T18:16:27.813482+0000","last_peered":"2023-12-19T18:16:27.813482+0000","last_clean":"2023-12-19T18:16:27.813482+0000","last_became_active":"2023-12-19T18:16:27.403121+0000","last_became_peered":"2023-12-19T18:16:27.403121+0000","last_unstale":"2023-12-19T18:16:27.813482+0000","last_undegraded":"2023-12-19T18:16:27.813482+0000","last_fullsized":"2023-12-19T18:16:27.813482+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:36:58.560143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.824494+0000","last_change":"2023-12-19T18:16:27.814277+0000","last_active":"2023-12-19T18:16:27.824494+0000","last_peered":"2023-12-19T18:16:27.824494+0000","last_clean":"2023-12-19T18:16:27.824494+0000","last_became_active":"2023-12-19T18:16:27.814023+0000","last_became_peered":"2023-12-19T18:16:27.814023+0000","last_unstale":"2023-12-19T18:16:27.824494+0000","last_undegraded":"2023-12-19T18:16:27.824494+0000","last_fullsized":"2023-12-19T18:16:27.824494+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:32.135133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"211'8","reported_seq":18,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.839966+0000","last_change":"2023-12-19T18:16:27.415118+0000","last_active":"2023-12-19T18:16:27.839966+0000","last_peered":"2023-12-19T18:16:27.839966+0000","last_clean":"2023-12-19T18:16:27.839966+0000","last_became_active":"2023-12-19T18:16:27.414687+0000","last_became_peered":"2023-12-19T18:16:27.414687+0000","last_unstale":"2023-12-19T18:16:27.839966+0000","last_undegraded":"2023-12-19T18:16:27.839966+0000","last_fullsized":"2023-12-19T18:16:27.839966+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:16.723922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.825345+0000","last_change":"2023-12-19T18:16:27.814804+0000","last_active":"2023-12-19T18:16:27.825345+0000","last_peered":"2023-12-19T18:16:27.825345+0000","last_clean":"2023-12-19T18:16:27.825345+0000","last_became_active":"2023-12-19T18:16:27.814534+0000","last_became_peered":"2023-12-19T18:16:27.814534+0000","last_unstale":"2023-12-19T18:16:27.825345+0000","last_undegraded":"2023-12-19T18:16:27.825345+0000","last_fullsized":"2023-12-19T18:16:27.825345+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:45.690711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"211'1","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834739+0000","last_change":"2023-12-19T18:16:27.414971+0000","last_active":"2023-12-19T18:16:27.834739+0000","last_peered":"2023-12-19T18:16:27.834739+0000","last_clean":"2023-12-19T18:16:27.834739+0000","last_became_active":"2023-12-19T18:16:27.414662+0000","last_became_peered":"2023-12-19T18:16:27.414662+0000","last_unstale":"2023-12-19T18:16:27.834739+0000","last_undegraded":"2023-12-19T18:16:27.834739+0000","last_fullsized":"2023-12-19T18:16:27.834739+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:47.204637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.827264+0000","last_change":"2023-12-19T18:16:27.823846+0000","last_active":"2023-12-19T18:16:27.827264+0000","last_peered":"2023-12-19T18:16:27.827264+0000","last_clean":"2023-12-19T18:16:27.827264+0000","last_became_active":"2023-12-19T18:16:27.823514+0000","last_became_peered":"2023-12-19T18:16:27.823514+0000","last_unstale":"2023-12-19T18:16:27.827264+0000","last_undegraded":"2023-12-19T18:16:27.827264+0000","last_fullsized":"2023-12-19T18:16:27.827264+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:20.691537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"211'2","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.830784+0000","last_change":"2023-12-19T18:16:27.409426+0000","last_active":"2023-12-19T18:16:27.830784+0000","last_peered":"2023-12-19T18:16:27.830784+0000","last_clean":"2023-12-19T18:16:27.830784+0000","last_became_active":"2023-12-19T18:16:27.407151+0000","last_became_peered":"2023-12-19T18:16:27.407151+0000","last_unstale":"2023-12-19T18:16:27.830784+0000","last_undegraded":"2023-12-19T18:16:27.830784+0000","last_fullsized":"2023-12-19T18:16:27.830784+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:30:56.976667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.820630+0000","last_change":"2023-12-19T18:16:27.816436+0000","last_active":"2023-12-19T18:16:27.820630+0000","last_peered":"2023-12-19T18:16:27.820630+0000","last_clean":"2023-12-19T18:16:27.820630+0000","last_became_active":"2023-12-19T18:16:27.816092+0000","last_became_peered":"2023-12-19T18:16:27.816092+0000","last_unstale":"2023-12-19T18:16:27.820630+0000","last_undegraded":"2023-12-19T18:16:27.820630+0000","last_fullsized":"2023-12-19T18:16:27.820630+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:07:26.713649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"211'1","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.836314+0000","last_change":"2023-12-19T18:16:27.416654+0000","last_active":"2023-12-19T18:16:27.836314+0000","last_peered":"2023-12-19T18:16:27.836314+0000","last_clean":"2023-12-19T18:16:27.836314+0000","last_became_active":"2023-12-19T18:16:27.416388+0000","last_became_peered":"2023-12-19T18:16:27.416388+0000","last_unstale":"2023-12-19T18:16:27.836314+0000","last_undegraded":"2023-12-19T18:16:27.836314+0000","last_fullsized":"2023-12-19T18:16:27.836314+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:51.339148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.835698+0000","last_change":"2023-12-19T18:16:27.832376+0000","last_active":"2023-12-19T18:16:27.835698+0000","last_peered":"2023-12-19T18:16:27.835698+0000","last_clean":"2023-12-19T18:16:27.835698+0000","last_became_active":"2023-12-19T18:16:27.832060+0000","last_became_peered":"2023-12-19T18:16:27.832060+0000","last_unstale":"2023-12-19T18:16:27.835698+0000","last_undegraded":"2023-12-19T18:16:27.835698+0000","last_fullsized":"2023-12-19T18:16:27.835698+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:17.432889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.809631+0000","last_change":"2023-12-19T18:16:27.408051+0000","last_active":"2023-12-19T18:16:27.809631+0000","last_peered":"2023-12-19T18:16:27.809631+0000","last_clean":"2023-12-19T18:16:27.809631+0000","last_became_active":"2023-12-19T18:16:27.407796+0000","last_became_peered":"2023-12-19T18:16:27.407796+0000","last_unstale":"2023-12-19T18:16:27.809631+0000","last_undegraded":"2023-12-19T18:16:27.809631+0000","last_fullsized":"2023-12-19T18:16:27.809631+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:41:09.697612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837427+0000","last_change":"2023-12-19T18:16:27.827898+0000","last_active":"2023-12-19T18:16:27.837427+0000","last_peered":"2023-12-19T18:16:27.837427+0000","last_clean":"2023-12-19T18:16:27.837427+0000","last_became_active":"2023-12-19T18:16:27.827613+0000","last_became_peered":"2023-12-19T18:16:27.827613+0000","last_unstale":"2023-12-19T18:16:27.837427+0000","last_undegraded":"2023-12-19T18:16:27.837427+0000","last_fullsized":"2023-12-19T18:16:27.837427+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:29.033205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"211'2","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.832328+0000","last_change":"2023-12-19T18:16:27.415902+0000","last_active":"2023-12-19T18:16:27.832328+0000","last_peered":"2023-12-19T18:16:27.832328+0000","last_clean":"2023-12-19T18:16:27.832328+0000","last_became_active":"2023-12-19T18:16:27.415062+0000","last_became_peered":"2023-12-19T18:16:27.415062+0000","last_unstale":"2023-12-19T18:16:27.832328+0000","last_undegraded":"2023-12-19T18:16:27.832328+0000","last_fullsized":"2023-12-19T18:16:27.832328+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:22.442653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.832685+0000","last_change":"2023-12-19T18:16:27.828160+0000","last_active":"2023-12-19T18:16:27.832685+0000","last_peered":"2023-12-19T18:16:27.832685+0000","last_clean":"2023-12-19T18:16:27.832685+0000","last_became_active":"2023-12-19T18:16:27.827882+0000","last_became_peered":"2023-12-19T18:16:27.827882+0000","last_unstale":"2023-12-19T18:16:27.832685+0000","last_undegraded":"2023-12-19T18:16:27.832685+0000","last_fullsized":"2023-12-19T18:16:27.832685+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:17.726704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.815898+0000","last_change":"2023-12-19T18:16:27.404853+0000","last_active":"2023-12-19T18:16:27.815898+0000","last_peered":"2023-12-19T18:16:27.815898+0000","last_clean":"2023-12-19T18:16:27.815898+0000","last_became_active":"2023-12-19T18:16:27.404347+0000","last_became_peered":"2023-12-19T18:16:27.404347+0000","last_unstale":"2023-12-19T18:16:27.815898+0000","last_undegraded":"2023-12-19T18:16:27.815898+0000","last_fullsized":"2023-12-19T18:16:27.815898+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:16.360996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834048+0000","last_change":"2023-12-19T18:16:27.824198+0000","last_active":"2023-12-19T18:16:27.834048+0000","last_peered":"2023-12-19T18:16:27.834048+0000","last_clean":"2023-12-19T18:16:27.834048+0000","last_became_active":"2023-12-19T18:16:27.823738+0000","last_became_peered":"2023-12-19T18:16:27.823738+0000","last_unstale":"2023-12-19T18:16:27.834048+0000","last_undegraded":"2023-12-19T18:16:27.834048+0000","last_fullsized":"2023-12-19T18:16:27.834048+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:53.509897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.811125+0000","last_change":"2023-12-19T18:16:27.417198+0000","last_active":"2023-12-19T18:16:27.811125+0000","last_peered":"2023-12-19T18:16:27.811125+0000","last_clean":"2023-12-19T18:16:27.811125+0000","last_became_active":"2023-12-19T18:16:27.416902+0000","last_became_peered":"2023-12-19T18:16:27.416902+0000","last_unstale":"2023-12-19T18:16:27.811125+0000","last_undegraded":"2023-12-19T18:16:27.811125+0000","last_fullsized":"2023-12-19T18:16:27.811125+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:54.434372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.832288+0000","last_change":"2023-12-19T18:16:27.825355+0000","last_active":"2023-12-19T18:16:27.832288+0000","last_peered":"2023-12-19T18:16:27.832288+0000","last_clean":"2023-12-19T18:16:27.832288+0000","last_became_active":"2023-12-19T18:16:27.825067+0000","last_became_peered":"2023-12-19T18:16:27.825067+0000","last_unstale":"2023-12-19T18:16:27.832288+0000","last_undegraded":"2023-12-19T18:16:27.832288+0000","last_fullsized":"2023-12-19T18:16:27.832288+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:02.928407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.807242+0000","last_change":"2023-12-19T18:16:27.403537+0000","last_active":"2023-12-19T18:16:27.807242+0000","last_peered":"2023-12-19T18:16:27.807242+0000","last_clean":"2023-12-19T18:16:27.807242+0000","last_became_active":"2023-12-19T18:16:27.403250+0000","last_became_peered":"2023-12-19T18:16:27.403250+0000","last_unstale":"2023-12-19T18:16:27.807242+0000","last_undegraded":"2023-12-19T18:16:27.807242+0000","last_fullsized":"2023-12-19T18:16:27.807242+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:56.889902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.833161+0000","last_change":"2023-12-19T18:16:27.826090+0000","last_active":"2023-12-19T18:16:27.833161+0000","last_peered":"2023-12-19T18:16:27.833161+0000","last_clean":"2023-12-19T18:16:27.833161+0000","last_became_active":"2023-12-19T18:16:27.825791+0000","last_became_peered":"2023-12-19T18:16:27.825791+0000","last_unstale":"2023-12-19T18:16:27.833161+0000","last_undegraded":"2023-12-19T18:16:27.833161+0000","last_fullsized":"2023-12-19T18:16:27.833161+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:39.382953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"211'2","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.838073+0000","last_change":"2023-12-19T18:16:27.409874+0000","last_active":"2023-12-19T18:16:27.838073+0000","last_peered":"2023-12-19T18:16:27.838073+0000","last_clean":"2023-12-19T18:16:27.838073+0000","last_became_active":"2023-12-19T18:16:27.409598+0000","last_became_peered":"2023-12-19T18:16:27.409598+0000","last_unstale":"2023-12-19T18:16:27.838073+0000","last_undegraded":"2023-12-19T18:16:27.838073+0000","last_fullsized":"2023-12-19T18:16:27.838073+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:13.721057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.833968+0000","last_change":"2023-12-19T18:16:27.824257+0000","last_active":"2023-12-19T18:16:27.833968+0000","last_peered":"2023-12-19T18:16:27.833968+0000","last_clean":"2023-12-19T18:16:27.833968+0000","last_became_active":"2023-12-19T18:16:27.823989+0000","last_became_peered":"2023-12-19T18:16:27.823989+0000","last_unstale":"2023-12-19T18:16:27.833968+0000","last_undegraded":"2023-12-19T18:16:27.833968+0000","last_fullsized":"2023-12-19T18:16:27.833968+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:07:07.505127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.805262+0000","last_change":"2023-12-19T18:16:27.414152+0000","last_active":"2023-12-19T18:16:27.805262+0000","last_peered":"2023-12-19T18:16:27.805262+0000","last_clean":"2023-12-19T18:16:27.805262+0000","last_became_active":"2023-12-19T18:16:27.413877+0000","last_became_peered":"2023-12-19T18:16:27.413877+0000","last_unstale":"2023-12-19T18:16:27.805262+0000","last_undegraded":"2023-12-19T18:16:27.805262+0000","last_fullsized":"2023-12-19T18:16:27.805262+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:44:52.553685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.835195+0000","last_change":"2023-12-19T18:16:27.825535+0000","last_active":"2023-12-19T18:16:27.835195+0000","last_peered":"2023-12-19T18:16:27.835195+0000","last_clean":"2023-12-19T18:16:27.835195+0000","last_became_active":"2023-12-19T18:16:27.824814+0000","last_became_peered":"2023-12-19T18:16:27.824814+0000","last_unstale":"2023-12-19T18:16:27.835195+0000","last_undegraded":"2023-12-19T18:16:27.835195+0000","last_fullsized":"2023-12-19T18:16:27.835195+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:29.357409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.804201+0000","last_change":"2023-12-19T18:16:27.420249+0000","last_active":"2023-12-19T18:16:27.804201+0000","last_peered":"2023-12-19T18:16:27.804201+0000","last_clean":"2023-12-19T18:16:27.804201+0000","last_became_active":"2023-12-19T18:16:27.419360+0000","last_became_peered":"2023-12-19T18:16:27.419360+0000","last_unstale":"2023-12-19T18:16:27.804201+0000","last_undegraded":"2023-12-19T18:16:27.804201+0000","last_fullsized":"2023-12-19T18:16:27.804201+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:07.969904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834501+0000","last_change":"2023-12-19T18:16:27.825200+0000","last_active":"2023-12-19T18:16:27.834501+0000","last_peered":"2023-12-19T18:16:27.834501+0000","last_clean":"2023-12-19T18:16:27.834501+0000","last_became_active":"2023-12-19T18:16:27.824919+0000","last_became_peered":"2023-12-19T18:16:27.824919+0000","last_unstale":"2023-12-19T18:16:27.834501+0000","last_undegraded":"2023-12-19T18:16:27.834501+0000","last_fullsized":"2023-12-19T18:16:27.834501+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.817261+0000","last_change":"2023-12-19T18:16:27.409169+0000","last_active":"2023-12-19T18:16:27.817261+0000","last_peered":"2023-12-19T18:16:27.817261+0000","last_clean":"2023-12-19T18:16:27.817261+0000","last_became_active":"2023-12-19T18:16:27.408882+0000","last_became_peered":"2023-12-19T18:16:27.408882+0000","last_unstale":"2023-12-19T18:16:27.817261+0000","last_undegraded":"2023-12-19T18:16:27.817261+0000","last_fullsized":"2023-12-19T18:16:27.817261+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:01.478246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.832772+0000","last_change":"2023-12-19T18:16:27.829925+0000","last_active":"2023-12-19T18:16:27.832772+0000","last_peered":"2023-12-19T18:16:27.832772+0000","last_clean":"2023-12-19T18:16:27.832772+0000","last_became_active":"2023-12-19T18:16:27.828958+0000","last_became_peered":"2023-12-19T18:16:27.828958+0000","last_unstale":"2023-12-19T18:16:27.832772+0000","last_undegraded":"2023-12-19T18:16:27.832772+0000","last_fullsized":"2023-12-19T18:16:27.832772+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:46.995476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.807783+0000","last_change":"2023-12-19T18:16:27.410513+0000","last_active":"2023-12-19T18:16:27.807783+0000","last_peered":"2023-12-19T18:16:27.807783+0000","last_clean":"2023-12-19T18:16:27.807783+0000","last_became_active":"2023-12-19T18:16:27.410166+0000","last_became_peered":"2023-12-19T18:16:27.410166+0000","last_unstale":"2023-12-19T18:16:27.807783+0000","last_undegraded":"2023-12-19T18:16:27.807783+0000","last_fullsized":"2023-12-19T18:16:27.807783+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:10.682880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.828526+0000","last_change":"2023-12-19T18:16:27.822423+0000","last_active":"2023-12-19T18:16:27.828526+0000","last_peered":"2023-12-19T18:16:27.828526+0000","last_clean":"2023-12-19T18:16:27.828526+0000","last_became_active":"2023-12-19T18:16:27.822106+0000","last_became_peered":"2023-12-19T18:16:27.822106+0000","last_unstale":"2023-12-19T18:16:27.828526+0000","last_undegraded":"2023-12-19T18:16:27.828526+0000","last_fullsized":"2023-12-19T18:16:27.828526+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:41.021760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.807347+0000","last_change":"2023-12-19T18:16:27.405381+0000","last_active":"2023-12-19T18:16:27.807347+0000","last_peered":"2023-12-19T18:16:27.807347+0000","last_clean":"2023-12-19T18:16:27.807347+0000","last_became_active":"2023-12-19T18:16:27.405113+0000","last_became_peered":"2023-12-19T18:16:27.405113+0000","last_unstale":"2023-12-19T18:16:27.807347+0000","last_undegraded":"2023-12-19T18:16:27.807347+0000","last_fullsized":"2023-12-19T18:16:27.807347+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:08.911302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.833551+0000","last_change":"2023-12-19T18:16:27.824001+0000","last_active":"2023-12-19T18:16:27.833551+0000","last_peered":"2023-12-19T18:16:27.833551+0000","last_clean":"2023-12-19T18:16:27.833551+0000","last_became_active":"2023-12-19T18:16:27.823010+0000","last_became_peered":"2023-12-19T18:16:27.823010+0000","last_unstale":"2023-12-19T18:16:27.833551+0000","last_undegraded":"2023-12-19T18:16:27.833551+0000","last_fullsized":"2023-12-19T18:16:27.833551+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:30:56.775994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.825851+0000","last_change":"2023-12-19T18:16:27.407456+0000","last_active":"2023-12-19T18:16:27.825851+0000","last_peered":"2023-12-19T18:16:27.825851+0000","last_clean":"2023-12-19T18:16:27.825851+0000","last_became_active":"2023-12-19T18:16:27.407208+0000","last_became_peered":"2023-12-19T18:16:27.407208+0000","last_unstale":"2023-12-19T18:16:27.825851+0000","last_undegraded":"2023-12-19T18:16:27.825851+0000","last_fullsized":"2023-12-19T18:16:27.825851+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:07.654229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837641+0000","last_change":"2023-12-19T18:16:27.832432+0000","last_active":"2023-12-19T18:16:27.837641+0000","last_peered":"2023-12-19T18:16:27.837641+0000","last_clean":"2023-12-19T18:16:27.837641+0000","last_became_active":"2023-12-19T18:16:27.832116+0000","last_became_peered":"2023-12-19T18:16:27.832116+0000","last_unstale":"2023-12-19T18:16:27.837641+0000","last_undegraded":"2023-12-19T18:16:27.837641+0000","last_fullsized":"2023-12-19T18:16:27.837641+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:35.165338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.811087+0000","last_change":"2023-12-19T18:16:27.412171+0000","last_active":"2023-12-19T18:16:27.811087+0000","last_peered":"2023-12-19T18:16:27.811087+0000","last_clean":"2023-12-19T18:16:27.811087+0000","last_became_active":"2023-12-19T18:16:27.411872+0000","last_became_peered":"2023-12-19T18:16:27.411872+0000","last_unstale":"2023-12-19T18:16:27.811087+0000","last_undegraded":"2023-12-19T18:16:27.811087+0000","last_fullsized":"2023-12-19T18:16:27.811087+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:48.264905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834193+0000","last_change":"2023-12-19T18:16:27.821899+0000","last_active":"2023-12-19T18:16:27.834193+0000","last_peered":"2023-12-19T18:16:27.834193+0000","last_clean":"2023-12-19T18:16:27.834193+0000","last_became_active":"2023-12-19T18:16:27.821634+0000","last_became_peered":"2023-12-19T18:16:27.821634+0000","last_unstale":"2023-12-19T18:16:27.834193+0000","last_undegraded":"2023-12-19T18:16:27.834193+0000","last_fullsized":"2023-12-19T18:16:27.834193+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:34.391068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.811113+0000","last_change":"2023-12-19T18:16:27.411230+0000","last_active":"2023-12-19T18:16:27.811113+0000","last_peered":"2023-12-19T18:16:27.811113+0000","last_clean":"2023-12-19T18:16:27.811113+0000","last_became_active":"2023-12-19T18:16:27.410670+0000","last_became_peered":"2023-12-19T18:16:27.410670+0000","last_unstale":"2023-12-19T18:16:27.811113+0000","last_undegraded":"2023-12-19T18:16:27.811113+0000","last_fullsized":"2023-12-19T18:16:27.811113+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:22.426446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"200'346","reported_seq":1183,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.400271+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:16:27.400271+0000","last_peered":"2023-12-19T18:16:27.400271+0000","last_clean":"2023-12-19T18:16:27.400271+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:16:27.400271+0000","last_undegraded":"2023-12-19T18:16:27.400271+0000","last_fullsized":"2023-12-19T18:16:27.400271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510566,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64544,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307296,"statfs":{"total":96636764160,"available":96570671104,"internally_reserved":0,"allocated":3235840,"data_stored":1982617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96399999999999997,"5min":1.048,"15min":1.1180000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92400000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92400000000000004,"5min":1.0389999999999999,"15min":1.0580000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72399999999999998,"15min":0.748},"min":{"1min":0.499,"5min":0.48099999999999998,"15min":0.435},"max":{"1min":0.89500000000000002,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.69899999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.38900000000000001},"max":{"1min":0.93400000000000005,"5min":1.026,"15min":1.0669999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.0820000000000001,"15min":1.236},"last":0.75900000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.75600000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88300000000000001,"5min":1.002,"15min":1.079},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98899999999999999,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.71799999999999997}]},{"osd":4,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81599999999999995,"5min":1.0009999999999999,"15min":1.302},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.871,"15min":1.0209999999999999},"last":0.53600000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.52700000000000002,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.88100000000000001,"5min":1.097,"15min":1.1299999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64900000000000002,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":1.026,"15min":1.0800000000000001},"last":0.55800000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.56999999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.93999999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85699999999999998,"5min":1.01,"15min":1.093},"last":0.67200000000000004}]}]},{"osd":6,"up_from":15,"seq":64424510566,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47572,"kb_used_data":2572,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324268,"statfs":{"total":96636764160,"available":96588050432,"internally_reserved":0,"allocated":2633728,"data_stored":1392145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72799999999999998,"15min":0.74199999999999999},"min":{"1min":0.55400000000000005,"5min":0.46800000000000003,"15min":0.43099999999999999},"max":{"1min":0.92400000000000004,"5min":0.96499999999999997,"15min":1.042},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71799999999999997,"15min":0.73499999999999999},"min":{"1min":0.57499999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92000000000000004,"5min":1.224,"15min":1.224},"last":0.54300000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70099999999999996,"15min":0.72799999999999998},"min":{"1min":0.49099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98099999999999998,"5min":1.103,"15min":1.103},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.64100000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":0.94599999999999995,"5min":0.99199999999999999,"15min":1.1259999999999999},"last":0.57199999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.502},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0609999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.749},"min":{"1min":0.60899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83299999999999996,"5min":1.0089999999999999,"15min":1.111},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.755,"15min":0.78200000000000003},"min":{"1min":0.61799999999999999,"5min":0.495,"15min":0.495},"max":{"1min":0.91200000000000003,"5min":0.97999999999999998,"15min":1.1519999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.51800000000000002,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.98699999999999999,"5min":1.268,"15min":1.268},"last":0.55900000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92200000000000004,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":0.81299999999999994,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.434}]},{"osd":5,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.93799999999999994,"5min":1.194,"15min":1.194},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":1.141,"15min":1.141},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.95899999999999996,"5min":1.159,"15min":1.159},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94199999999999995,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.67500000000000004}]}]},{"osd":5,"up_from":15,"seq":64424510566,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47560,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324280,"statfs":{"total":96636764160,"available":96588062720,"internally_reserved":0,"allocated":2621440,"data_stored":1384610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.72299999999999998,"15min":0.72899999999999998},"min":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.34399999999999997},"max":{"1min":0.95699999999999996,"5min":1.0109999999999999,"15min":1.0149999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.74099999999999999,"15min":0.74299999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.86899999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.53300000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.94199999999999995,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73199999999999998,"15min":0.73799999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.83299999999999996}]},{"osd":2,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77800000000000002,"15min":0.77900000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":0.97599999999999998,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.004,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.94499999999999995}]},{"osd":3,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76900000000000002,"15min":0.76600000000000001},"min":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95799999999999996,"5min":1.05,"15min":1.05},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.0329999999999999},"last":0.97099999999999997}]},{"osd":4,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59999999999999998,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.94499999999999995,"15min":1.0800000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64500000000000002,"15min":0.628},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.41599999999999998},"max":{"1min":0.85499999999999998,"5min":1.208,"15min":1.208},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.64800000000000002,"15min":0.63400000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.98999999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.67500000000000004,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.40699999999999997},"max":{"1min":0.90400000000000003,"5min":1.008,"15min":1.0449999999999999},"last":0.71399999999999997}]},{"osd":7,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.70799999999999996,"15min":0.68600000000000005},"min":{"1min":0.48699999999999999,"5min":0.439,"15min":0.436},"max":{"1min":0.78400000000000003,"5min":1.169,"15min":1.169},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71699999999999997,"15min":0.71099999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81999999999999995,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.748}]}]},{"osd":4,"up_from":15,"seq":64424510566,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47548,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324292,"statfs":{"total":96636764160,"available":96588075008,"internally_reserved":0,"allocated":2609152,"data_stored":1377575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.747,"15min":0.74099999999999999},"min":{"1min":0.51200000000000001,"5min":0.45800000000000002,"15min":0.35899999999999999},"max":{"1min":1.034,"5min":1.036,"15min":1.109},"last":0.626},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90100000000000002,"5min":0.997,"15min":1.01},"last":0.73099999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77100000000000002,"15min":0.76000000000000001},"min":{"1min":0.52700000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0049999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.755},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.49099999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.96999999999999997,"5min":0.997,"15min":1.1020000000000001},"last":0.747}]},{"osd":2,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78600000000000003,"15min":0.77400000000000002},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.47699999999999998},"max":{"1min":1.012,"5min":1.0289999999999999,"15min":1.071},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.77600000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.47799999999999998},"max":{"1min":0.97099999999999997,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.68899999999999995}]},{"osd":3,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76400000000000001,"15min":0.75900000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.501},"max":{"1min":0.96799999999999997,"5min":0.98499999999999999,"15min":1.0409999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80400000000000005,"15min":0.79400000000000004},"min":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.55000000000000004},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.056},"last":0.79900000000000004}]},{"osd":5,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.86199999999999999,"5min":0.95699999999999996,"15min":1.1020000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66400000000000003,"15min":0.64500000000000002},"min":{"1min":0.53100000000000003,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.93300000000000005,"5min":1.042,"15min":1.042},"last":0.64500000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67700000000000005,"15min":0.65800000000000003},"min":{"1min":0.49299999999999999,"5min":0.437,"15min":0.40200000000000002},"max":{"1min":0.91000000000000003,"5min":1.024,"15min":1.0669999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72699999999999998,"15min":0.71399999999999997},"min":{"1min":0.48899999999999999,"5min":0.48499999999999999,"15min":0.45600000000000002},"max":{"1min":0.995,"5min":0.995,"15min":1.0760000000000001},"last":0.59999999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72599999999999998,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.41899999999999998},"max":{"1min":0.96299999999999997,"5min":1.087,"15min":1.087},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.46999999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70999999999999996}]}]},{"osd":3,"up_from":15,"seq":64424510566,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47560,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324280,"statfs":{"total":96636764160,"available":96588062720,"internally_reserved":0,"allocated":2621440,"data_stored":1384610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.31900000000000001,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.375},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74399999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.502,"15min":0.501},"min":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39600000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.49099999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.442,"15min":0.40100000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.57499999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83399999999999996,"15min":0.83699999999999997},"min":{"1min":0.65400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0609999999999999,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83499999999999996,"15min":0.83199999999999996},"min":{"1min":0.68899999999999995,"5min":0.57499999999999996,"15min":0.56599999999999995},"max":{"1min":1.085,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.63100000000000001,"5min":0.60999999999999999,"15min":0.57399999999999995},"max":{"1min":1.087,"5min":1.1739999999999999,"15min":1.397},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.84299999999999997,"15min":0.83999999999999997},"min":{"1min":0.60899999999999999,"5min":0.55900000000000005,"15min":0.55100000000000005},"max":{"1min":1.1919999999999999,"5min":1.2230000000000001,"15min":1.262},"last":0.752}]},{"osd":6,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.85399999999999998,"15min":0.85799999999999998},"min":{"1min":0.67100000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.175,"5min":1.234,"15min":1.274},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.85299999999999998,"15min":0.85599999999999998},"min":{"1min":0.65500000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.083,"5min":1.165,"15min":1.349},"last":0.68300000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.92100000000000004,"15min":0.91500000000000004},"min":{"1min":0.71799999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.157,"5min":1.3540000000000001,"15min":1.486},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.92000000000000004,"15min":0.90700000000000003},"min":{"1min":0.73199999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.80800000000000005}]}]},{"osd":2,"up_from":15,"seq":64424510565,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47584,"kb_used_data":2584,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324256,"statfs":{"total":96636764160,"available":96588038144,"internally_reserved":0,"allocated":2646016,"data_stored":1392839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.29299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.34100000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.41499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.432}]},{"osd":3,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.86199999999999999,"15min":0.95999999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.36899999999999999},"max":{"1min":0.67200000000000004,"5min":0.79700000000000004,"15min":0.875},"last":0.48599999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.624,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.048,"5min":1.306,"15min":1.306},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.05,"5min":1.272,"15min":1.466},"last":0.79600000000000004}]},{"osd":5,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.64300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.97699999999999998,"5min":1.1459999999999999,"15min":1.3049999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.81999999999999995,"15min":0.82599999999999996},"min":{"1min":0.61899999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.429},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.71199999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.216,"5min":1.365,"15min":1.4119999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80600000000000005,"15min":0.80800000000000005},"min":{"1min":0.625,"5min":0.55200000000000005,"15min":0.53500000000000003},"max":{"1min":0.94499999999999995,"5min":1.2,"15min":1.2470000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81499999999999995,"15min":0.82799999999999996},"min":{"1min":0.64400000000000002,"5min":0.59899999999999998,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.393,"15min":1.45},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85999999999999999,"15min":0.86099999999999999},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1850000000000001,"5min":1.276,"15min":1.276},"last":0.871}]}]},{"osd":1,"up_from":15,"seq":64424510566,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59872,"kb_used_data":2584,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311968,"statfs":{"total":96636764160,"available":96575455232,"internally_reserved":0,"allocated":2646016,"data_stored":1393709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.33900000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63700000000000001,"5min":0.71999999999999997,"15min":0.89400000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.53300000000000003},"min":{"1min":0.32200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.439}]},{"osd":2,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.32600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.45000000000000001,"15min":0.442},"max":{"1min":0.82699999999999996,"5min":0.94199999999999995,"15min":0.94499999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.42099999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.65100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93400000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79100000000000004,"15min":0.81499999999999995},"min":{"1min":0.502,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0409999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.59499999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.78400000000000003,"15min":0.80500000000000005},"min":{"1min":0.49099999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97499999999999998,"5min":1.236,"15min":1.2609999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98399999999999999,"5min":1.353,"15min":1.353},"last":0.56399999999999995}]},{"osd":6,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.60999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.008,"5min":1.2270000000000001,"15min":1.2689999999999999},"last":0.621},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.84299999999999997,"15min":0.84999999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.093,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.64100000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.123,"5min":1.399,"15min":1.399},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.84299999999999997,"15min":0.85799999999999998},"min":{"1min":0.54100000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.105,"5min":1.365,"15min":1.365},"last":0.54100000000000004}]}]},{"osd":0,"up_from":14,"seq":60129543271,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60444,"kb_used_data":3156,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311396,"statfs":{"total":96636764160,"available":96574869504,"internally_reserved":0,"allocated":3231744,"data_stored":1982535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.438,"15min":0.45700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":1.216},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.42699999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64700000000000002,"5min":0.84999999999999998,"15min":1.1699999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72299999999999998,"5min":0.78900000000000003,"15min":0.96799999999999997},"last":0.39100000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.623,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.82699999999999996,"5min":0.871,"15min":1.032},"last":0.51200000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.63,"5min":0.51800000000000002,"15min":0.496},"max":{"1min":0.93100000000000005,"5min":1.1200000000000001,"15min":1.381},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.59199999999999997,"5min":0.56799999999999995,"15min":0.47799999999999998},"max":{"1min":1.02,"5min":1.137,"15min":1.2849999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80600000000000005,"15min":0.81899999999999995},"min":{"1min":0.57299999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.008,"5min":1.1930000000000001,"15min":1.3420000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.69599999999999995,"5min":0.59599999999999997,"15min":0.59499999999999997},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.288},"last":0.84799999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.51300000000000001},"max":{"1min":0.98899999999999999,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.83399999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0229999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.84999999999999998},"min":{"1min":0.65300000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.044,"5min":1.1899999999999999,"15min":1.415},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.83199999999999996,"15min":0.85599999999999998},"min":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0780000000000001,"5min":1.1379999999999999,"15min":1.395},"last":0.74099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:16:32.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:16:32.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b3d4f0640 1 Processor -- start 2023-12-19T18:16:32.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b3d4f0640 1 -- start start 2023-12-19T18:16:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b3d4f0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8b380affe0 0x7f8b38071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b3d4f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380b3960 0x7f8b380b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b3d4f0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3815e7c0 con 0x7f8b380b2030 2023-12-19T18:16:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b3d4f0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b380c97a0 con 0x7f8b380affe0 2023-12-19T18:16:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b3d4f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b38072360 con 0x7f8b380b3960 2023-12-19T18:16:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b37fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8b380b2030 0x7f8b380c9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47090/0 (socket says 172.21.15.26:47090) 2023-12-19T18:16:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b37fff640 1 -- 172.21.15.26:0/4243329522 learned_addr learned my addr 172.21.15.26:0/4243329522 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:32.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b3ccef640 1 --2- 172.21.15.26:0/4243329522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380b3960 0x7f8b380b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b377fe640 1 --2- 172.21.15.26:0/4243329522 >> v2:172.21.15.160:3300/0 conn(0x7f8b380affe0 0x7f8b38071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818350291 0 0) 0x7f8b3815e7c0 con 0x7f8b380b2030 2023-12-19T18:16:32.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b20003530 con 0x7f8b380b2030 2023-12-19T18:16:32.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.358+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2442589883 0 0) 0x7f8b20003530 con 0x7f8b380b2030 2023-12-19T18:16:32.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380b3960 msgr2=0x7f8b380b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:32.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 --2- 172.21.15.26:0/4243329522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380b3960 0x7f8b380b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 >> v2:172.21.15.160:3300/0 conn(0x7f8b380affe0 msgr2=0x7f8b38071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 --2- 172.21.15.26:0/4243329522 >> v2:172.21.15.160:3300/0 conn(0x7f8b380affe0 0x7f8b38071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b380b04c0 con 0x7f8b380b2030 2023-12-19T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f8b28001680 con 0x7f8b380b2030 2023-12-19T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 >> v1:172.21.15.26:6789/0 conn(0x7f8b380b2030 legacy=0x7f8b380c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 --2- 172.21.15.26:0/4243329522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b20003a20 0x7f8b20003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 --2- 172.21.15.26:0/4243329522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380b3960 0x7f8b200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 --2- 172.21.15.26:0/4243329522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b20004d10 0x7f8b20005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380c97a0 con 0x7f8b20003a20 2023-12-19T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b38072360 con 0x7f8b20004d10 2023-12-19T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2000fdd0 con 0x7f8b380b3960 2023-12-19T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b37fff640 1 --2- 172.21.15.26:0/4243329522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b20003a20 0x7f8b20003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b377fe640 1 --2- 172.21.15.26:0/4243329522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380b3960 0x7f8b200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b3ccef640 1 --2- 172.21.15.26:0/4243329522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b20004d10 0x7f8b20005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b37fff640 1 -- 172.21.15.26:0/4243329522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380b3960 msgr2=0x7f8b200047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b37fff640 1 --2- 172.21.15.26:0/4243329522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b380b3960 0x7f8b200047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b37fff640 1 -- 172.21.15.26:0/4243329522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b20004d10 msgr2=0x7f8b20005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b37fff640 1 --2- 172.21.15.26:0/4243329522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b20004d10 0x7f8b20005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b37fff640 1 -- 172.21.15.26:0/4243329522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b20003530 con 0x7f8b20003a20 2023-12-19T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b37fff640 1 --2- 172.21.15.26:0/4243329522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b20003a20 0x7f8b20003e40 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800c240 tx=0x7f8b2801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ade6050afc7924 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:32.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b280a58f0 con 0x7f8b20003a20 2023-12-19T18:16:32.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b280a5a50 con 0x7f8b20003a20 2023-12-19T18:16:32.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.362+0000 7f8b36ffd640 1 -- 172.21.15.26:0/4243329522 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b280a5e40 con 0x7f8b20003a20 2023-12-19T18:16:32.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/4243329522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b20003a20 msgr2=0x7f8b20003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 --2- 172.21.15.26:0/4243329522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b20003a20 0x7f8b20003e40 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800c240 tx=0x7f8b2801cc10 comp rx=0 tx=0).stop 2023-12-19T18:16:32.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3ccef640 1 -- 172.21.15.26:0/4243329522 reap_dead start 2023-12-19T18:16:32.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/4243329522 shutdown_connections 2023-12-19T18:16:32.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/4243329522 >> 172.21.15.26:0/4243329522 conn(0x7f8b3806b0f0 msgr2=0x7f8b3806b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:32.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/4243329522 shutdown_connections 2023-12-19T18:16:32.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/4243329522 wait complete. 2023-12-19T18:16:32.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 Processor -- start 2023-12-19T18:16:32.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 -- start start 2023-12-19T18:16:32.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b20003a20 0x7f8b3815c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b38145c70 0x7f8b381460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b3815a0b0 0x7f8b3815a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3815b100 con 0x7f8b20003a20 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b38146630 con 0x7f8b3815a0b0 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3d4f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b38143eb0 con 0x7f8b38145c70 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b20003a20 0x7f8b3815c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b20003a20 0x7f8b3815c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40488/0 (socket says 172.21.15.26:40488) 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b37fff640 1 -- 172.21.15.26:0/2721159904 learned_addr learned my addr 172.21.15.26:0/2721159904 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b377fe640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b38145c70 0x7f8b381460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.366+0000 7f8b3ccef640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b3815a0b0 0x7f8b3815a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b37fff640 1 -- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b38145c70 msgr2=0x7f8b381460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b37fff640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b38145c70 0x7f8b381460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b37fff640 1 -- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b3815a0b0 msgr2=0x7f8b3815a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b37fff640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b3815a0b0 0x7f8b3815a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b37fff640 1 -- 172.21.15.26:0/2721159904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3815ae20 con 0x7f8b20003a20 2023-12-19T18:16:32.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b377fe640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b38145c70 0x7f8b381460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:32.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b3ccef640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b3815a0b0 0x7f8b3815a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:32.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b37fff640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b20003a20 0x7f8b3815c840 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f8b2801ca40 tx=0x7f8b20005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:32.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b34ff9640 1 -- 172.21.15.26:0/2721159904 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b280b4070 con 0x7f8b20003a20 2023-12-19T18:16:32.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b34ff9640 1 -- 172.21.15.26:0/2721159904 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b28000ee0 con 0x7f8b20003a20 2023-12-19T18:16:32.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b34ff9640 1 -- 172.21.15.26:0/2721159904 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b280a58f0 con 0x7f8b20003a20 2023-12-19T18:16:32.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/2721159904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b38143860 con 0x7f8b20003a20 2023-12-19T18:16:32.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/2721159904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3815aa70 con 0x7f8b20003a20 2023-12-19T18:16:32.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/2721159904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b381436c0 con 0x7f8b20003a20 2023-12-19T18:16:32.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b34ff9640 1 -- 172.21.15.26:0/2721159904 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8b28001440 con 0x7f8b20003a20 2023-12-19T18:16:32.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b34ff9640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8b24065bb0 0x7f8b24068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.370+0000 7f8b34ff9640 1 -- 172.21.15.26:0/2721159904 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f8b2811d1f0 con 0x7f8b20003a20 2023-12-19T18:16:32.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.374+0000 7f8b377fe640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8b24065bb0 0x7f8b24068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.374+0000 7f8b377fe640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8b24065bb0 0x7f8b24068070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b380c94e0 tx=0x7f8b2c008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:32.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.374+0000 7f8b34ff9640 1 -- 172.21.15.26:0/2721159904 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8b381436c0 con 0x7f8b20003a20 2023-12-19T18:16:32.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.530+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/2721159904 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b380ca920 con 0x7f8b24065bb0 2023-12-19T18:16:32.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.538+0000 7f8b34ff9640 1 -- 172.21.15.26:0/2721159904 <== mgr.9805 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289476 (secure 0 0 0) 0x7f8b2c013960 con 0x7f8b24065bb0 2023-12-19T18:16:32.543 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:32.544 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:16:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.542+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8b24065bb0 msgr2=0x7f8b24068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.542+0000 7f8b3d4f0640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8b24065bb0 0x7f8b24068070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8b380c94e0 tx=0x7f8b2c008040 comp rx=0 tx=0).stop 2023-12-19T18:16:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.542+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/2721159904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b20003a20 msgr2=0x7f8b3815c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.542+0000 7f8b3d4f0640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b20003a20 0x7f8b3815c840 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f8b2801ca40 tx=0x7f8b20005800 comp rx=0 tx=0).stop 2023-12-19T18:16:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.542+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/2721159904 shutdown_connections 2023-12-19T18:16:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.542+0000 7f8b3d4f0640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8b24065bb0 0x7f8b24068070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.542+0000 7f8b3d4f0640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b3815a0b0 0x7f8b3815a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.542+0000 7f8b3d4f0640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8b38145c70 0x7f8b381460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.542+0000 7f8b3d4f0640 1 --2- 172.21.15.26:0/2721159904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b20003a20 0x7f8b3815c840 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.542+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/2721159904 >> 172.21.15.26:0/2721159904 conn(0x7f8b3806b0f0 msgr2=0x7f8b3806f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.542+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/2721159904 shutdown_connections 2023-12-19T18:16:32.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.542+0000 7f8b3d4f0640 1 -- 172.21.15.26:0/2721159904 wait complete. 2023-12-19T18:16:32.563 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-12-19T18:16:31.304517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":619,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":384,"ondisk_log_size":384,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":422684,"kb_used_data":21724,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754552036,"statfs":{"total":773094113280,"available":772661284864,"internally_reserved":0,"allocated":22245376,"data_stored":12290640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.917534"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.836370+0000","last_change":"2023-12-19T18:16:27.834221+0000","last_active":"2023-12-19T18:16:27.836370+0000","last_peered":"2023-12-19T18:16:27.836370+0000","last_clean":"2023-12-19T18:16:27.836370+0000","last_became_active":"2023-12-19T18:16:27.825886+0000","last_became_peered":"2023-12-19T18:16:27.825886+0000","last_unstale":"2023-12-19T18:16:27.836370+0000","last_undegraded":"2023-12-19T18:16:27.836370+0000","last_fullsized":"2023-12-19T18:16:27.836370+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:12.119996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834634+0000","last_change":"2023-12-19T18:16:27.831511+0000","last_active":"2023-12-19T18:16:27.834634+0000","last_peered":"2023-12-19T18:16:27.834634+0000","last_clean":"2023-12-19T18:16:27.834634+0000","last_became_active":"2023-12-19T18:16:27.831255+0000","last_became_peered":"2023-12-19T18:16:27.831255+0000","last_unstale":"2023-12-19T18:16:27.834634+0000","last_undegraded":"2023-12-19T18:16:27.834634+0000","last_fullsized":"2023-12-19T18:16:27.834634+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:42.382254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.820569+0000","last_change":"2023-12-19T18:16:27.815730+0000","last_active":"2023-12-19T18:16:27.820569+0000","last_peered":"2023-12-19T18:16:27.820569+0000","last_clean":"2023-12-19T18:16:27.820569+0000","last_became_active":"2023-12-19T18:16:27.815341+0000","last_became_peered":"2023-12-19T18:16:27.815341+0000","last_unstale":"2023-12-19T18:16:27.820569+0000","last_undegraded":"2023-12-19T18:16:27.820569+0000","last_fullsized":"2023-12-19T18:16:27.820569+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:39.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834527+0000","last_change":"2023-12-19T18:16:27.830449+0000","last_active":"2023-12-19T18:16:27.834527+0000","last_peered":"2023-12-19T18:16:27.834527+0000","last_clean":"2023-12-19T18:16:27.834527+0000","last_became_active":"2023-12-19T18:16:27.830189+0000","last_became_peered":"2023-12-19T18:16:27.830189+0000","last_unstale":"2023-12-19T18:16:27.834527+0000","last_undegraded":"2023-12-19T18:16:27.834527+0000","last_fullsized":"2023-12-19T18:16:27.834527+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:06.397912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837876+0000","last_change":"2023-12-19T18:16:27.835673+0000","last_active":"2023-12-19T18:16:27.837876+0000","last_peered":"2023-12-19T18:16:27.837876+0000","last_clean":"2023-12-19T18:16:27.837876+0000","last_became_active":"2023-12-19T18:16:27.835392+0000","last_became_peered":"2023-12-19T18:16:27.835392+0000","last_unstale":"2023-12-19T18:16:27.837876+0000","last_undegraded":"2023-12-19T18:16:27.837876+0000","last_fullsized":"2023-12-19T18:16:27.837876+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:23:06.990327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.829676+0000","last_change":"2023-12-19T18:16:27.822167+0000","last_active":"2023-12-19T18:16:27.829676+0000","last_peered":"2023-12-19T18:16:27.829676+0000","last_clean":"2023-12-19T18:16:27.829676+0000","last_became_active":"2023-12-19T18:16:27.821752+0000","last_became_peered":"2023-12-19T18:16:27.821752+0000","last_unstale":"2023-12-19T18:16:27.829676+0000","last_undegraded":"2023-12-19T18:16:27.829676+0000","last_fullsized":"2023-12-19T18:16:27.829676+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:17.866181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.828067+0000","last_change":"2023-12-19T18:16:27.823852+0000","last_active":"2023-12-19T18:16:27.828067+0000","last_peered":"2023-12-19T18:16:27.828067+0000","last_clean":"2023-12-19T18:16:27.828067+0000","last_became_active":"2023-12-19T18:16:27.823564+0000","last_became_peered":"2023-12-19T18:16:27.823564+0000","last_unstale":"2023-12-19T18:16:27.828067+0000","last_undegraded":"2023-12-19T18:16:27.828067+0000","last_fullsized":"2023-12-19T18:16:27.828067+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:59.000440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.836791+0000","last_change":"2023-12-19T18:16:27.825730+0000","last_active":"2023-12-19T18:16:27.836791+0000","last_peered":"2023-12-19T18:16:27.836791+0000","last_clean":"2023-12-19T18:16:27.836791+0000","last_became_active":"2023-12-19T18:16:27.825136+0000","last_became_peered":"2023-12-19T18:16:27.825136+0000","last_unstale":"2023-12-19T18:16:27.836791+0000","last_undegraded":"2023-12-19T18:16:27.836791+0000","last_fullsized":"2023-12-19T18:16:27.836791+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:48.330735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.835843+0000","last_change":"2023-12-19T18:16:27.832580+0000","last_active":"2023-12-19T18:16:27.835843+0000","last_peered":"2023-12-19T18:16:27.835843+0000","last_clean":"2023-12-19T18:16:27.835843+0000","last_became_active":"2023-12-19T18:16:27.832291+0000","last_became_peered":"2023-12-19T18:16:27.832291+0000","last_unstale":"2023-12-19T18:16:27.835843+0000","last_undegraded":"2023-12-19T18:16:27.835843+0000","last_fullsized":"2023-12-19T18:16:27.835843+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:29.305405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.827726+0000","last_change":"2023-12-19T18:16:27.819945+0000","last_active":"2023-12-19T18:16:27.827726+0000","last_peered":"2023-12-19T18:16:27.827726+0000","last_clean":"2023-12-19T18:16:27.827726+0000","last_became_active":"2023-12-19T18:16:27.819634+0000","last_became_peered":"2023-12-19T18:16:27.819634+0000","last_unstale":"2023-12-19T18:16:27.827726+0000","last_undegraded":"2023-12-19T18:16:27.827726+0000","last_fullsized":"2023-12-19T18:16:27.827726+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:22.401852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.824952+0000","last_change":"2023-12-19T18:16:27.819797+0000","last_active":"2023-12-19T18:16:27.824952+0000","last_peered":"2023-12-19T18:16:27.824952+0000","last_clean":"2023-12-19T18:16:27.824952+0000","last_became_active":"2023-12-19T18:16:27.819507+0000","last_became_peered":"2023-12-19T18:16:27.819507+0000","last_unstale":"2023-12-19T18:16:27.824952+0000","last_undegraded":"2023-12-19T18:16:27.824952+0000","last_fullsized":"2023-12-19T18:16:27.824952+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:08.287716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834510+0000","last_change":"2023-12-19T18:16:27.829943+0000","last_active":"2023-12-19T18:16:27.834510+0000","last_peered":"2023-12-19T18:16:27.834510+0000","last_clean":"2023-12-19T18:16:27.834510+0000","last_became_active":"2023-12-19T18:16:27.829592+0000","last_became_peered":"2023-12-19T18:16:27.829592+0000","last_unstale":"2023-12-19T18:16:27.834510+0000","last_undegraded":"2023-12-19T18:16:27.834510+0000","last_fullsized":"2023-12-19T18:16:27.834510+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:46.245564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834280+0000","last_change":"2023-12-19T18:16:27.829736+0000","last_active":"2023-12-19T18:16:27.834280+0000","last_peered":"2023-12-19T18:16:27.834280+0000","last_clean":"2023-12-19T18:16:27.834280+0000","last_became_active":"2023-12-19T18:16:27.829448+0000","last_became_peered":"2023-12-19T18:16:27.829448+0000","last_unstale":"2023-12-19T18:16:27.834280+0000","last_undegraded":"2023-12-19T18:16:27.834280+0000","last_fullsized":"2023-12-19T18:16:27.834280+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:47.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.827082+0000","last_change":"2023-12-19T18:16:27.823186+0000","last_active":"2023-12-19T18:16:27.827082+0000","last_peered":"2023-12-19T18:16:27.827082+0000","last_clean":"2023-12-19T18:16:27.827082+0000","last_became_active":"2023-12-19T18:16:27.820405+0000","last_became_peered":"2023-12-19T18:16:27.820405+0000","last_unstale":"2023-12-19T18:16:27.827082+0000","last_undegraded":"2023-12-19T18:16:27.827082+0000","last_fullsized":"2023-12-19T18:16:27.827082+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:41.515700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834653+0000","last_change":"2023-12-19T18:16:27.830751+0000","last_active":"2023-12-19T18:16:27.834653+0000","last_peered":"2023-12-19T18:16:27.834653+0000","last_clean":"2023-12-19T18:16:27.834653+0000","last_became_active":"2023-12-19T18:16:27.830490+0000","last_became_peered":"2023-12-19T18:16:27.830490+0000","last_unstale":"2023-12-19T18:16:27.834653+0000","last_undegraded":"2023-12-19T18:16:27.834653+0000","last_fullsized":"2023-12-19T18:16:27.834653+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:31.093343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837674+0000","last_change":"2023-12-19T18:16:27.826828+0000","last_active":"2023-12-19T18:16:27.837674+0000","last_peered":"2023-12-19T18:16:27.837674+0000","last_clean":"2023-12-19T18:16:27.837674+0000","last_became_active":"2023-12-19T18:16:27.825994+0000","last_became_peered":"2023-12-19T18:16:27.825994+0000","last_unstale":"2023-12-19T18:16:27.837674+0000","last_undegraded":"2023-12-19T18:16:27.837674+0000","last_fullsized":"2023-12-19T18:16:27.837674+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:17.748218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837797+0000","last_change":"2023-12-19T18:16:27.834880+0000","last_active":"2023-12-19T18:16:27.837797+0000","last_peered":"2023-12-19T18:16:27.837797+0000","last_clean":"2023-12-19T18:16:27.837797+0000","last_became_active":"2023-12-19T18:16:27.834494+0000","last_became_peered":"2023-12-19T18:16:27.834494+0000","last_unstale":"2023-12-19T18:16:27.837797+0000","last_undegraded":"2023-12-19T18:16:27.837797+0000","last_fullsized":"2023-12-19T18:16:27.837797+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:06.992701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.822991+0000","last_change":"2023-12-19T18:16:27.820781+0000","last_active":"2023-12-19T18:16:27.822991+0000","last_peered":"2023-12-19T18:16:27.822991+0000","last_clean":"2023-12-19T18:16:27.822991+0000","last_became_active":"2023-12-19T18:16:27.820483+0000","last_became_peered":"2023-12-19T18:16:27.820483+0000","last_unstale":"2023-12-19T18:16:27.822991+0000","last_undegraded":"2023-12-19T18:16:27.822991+0000","last_fullsized":"2023-12-19T18:16:27.822991+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:23.445435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.827776+0000","last_change":"2023-12-19T18:16:27.820194+0000","last_active":"2023-12-19T18:16:27.827776+0000","last_peered":"2023-12-19T18:16:27.827776+0000","last_clean":"2023-12-19T18:16:27.827776+0000","last_became_active":"2023-12-19T18:16:27.819900+0000","last_became_peered":"2023-12-19T18:16:27.819900+0000","last_unstale":"2023-12-19T18:16:27.827776+0000","last_undegraded":"2023-12-19T18:16:27.827776+0000","last_fullsized":"2023-12-19T18:16:27.827776+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:00.858729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837984+0000","last_change":"2023-12-19T18:16:27.833054+0000","last_active":"2023-12-19T18:16:27.837984+0000","last_peered":"2023-12-19T18:16:27.837984+0000","last_clean":"2023-12-19T18:16:27.837984+0000","last_became_active":"2023-12-19T18:16:27.832738+0000","last_became_peered":"2023-12-19T18:16:27.832738+0000","last_unstale":"2023-12-19T18:16:27.837984+0000","last_undegraded":"2023-12-19T18:16:27.837984+0000","last_fullsized":"2023-12-19T18:16:27.837984+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:57.892685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834012+0000","last_change":"2023-12-19T18:16:27.831551+0000","last_active":"2023-12-19T18:16:27.834012+0000","last_peered":"2023-12-19T18:16:27.834012+0000","last_clean":"2023-12-19T18:16:27.834012+0000","last_became_active":"2023-12-19T18:16:27.831291+0000","last_became_peered":"2023-12-19T18:16:27.831291+0000","last_unstale":"2023-12-19T18:16:27.834012+0000","last_undegraded":"2023-12-19T18:16:27.834012+0000","last_fullsized":"2023-12-19T18:16:27.834012+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:09.914084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837462+0000","last_change":"2023-12-19T18:16:27.830084+0000","last_active":"2023-12-19T18:16:27.837462+0000","last_peered":"2023-12-19T18:16:27.837462+0000","last_clean":"2023-12-19T18:16:27.837462+0000","last_became_active":"2023-12-19T18:16:27.829787+0000","last_became_peered":"2023-12-19T18:16:27.829787+0000","last_unstale":"2023-12-19T18:16:27.837462+0000","last_undegraded":"2023-12-19T18:16:27.837462+0000","last_fullsized":"2023-12-19T18:16:27.837462+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:34:16.356894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.831781+0000","last_change":"2023-12-19T18:16:27.826145+0000","last_active":"2023-12-19T18:16:27.831781+0000","last_peered":"2023-12-19T18:16:27.831781+0000","last_clean":"2023-12-19T18:16:27.831781+0000","last_became_active":"2023-12-19T18:16:27.825889+0000","last_became_peered":"2023-12-19T18:16:27.825889+0000","last_unstale":"2023-12-19T18:16:27.831781+0000","last_undegraded":"2023-12-19T18:16:27.831781+0000","last_fullsized":"2023-12-19T18:16:27.831781+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:59:41.673539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837653+0000","last_change":"2023-12-19T18:16:27.826642+0000","last_active":"2023-12-19T18:16:27.837653+0000","last_peered":"2023-12-19T18:16:27.837653+0000","last_clean":"2023-12-19T18:16:27.837653+0000","last_became_active":"2023-12-19T18:16:27.825460+0000","last_became_peered":"2023-12-19T18:16:27.825460+0000","last_unstale":"2023-12-19T18:16:27.837653+0000","last_undegraded":"2023-12-19T18:16:27.837653+0000","last_fullsized":"2023-12-19T18:16:27.837653+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:05.671602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.836036+0000","last_change":"2023-12-19T18:16:27.834875+0000","last_active":"2023-12-19T18:16:27.836036+0000","last_peered":"2023-12-19T18:16:27.836036+0000","last_clean":"2023-12-19T18:16:27.836036+0000","last_became_active":"2023-12-19T18:16:27.834561+0000","last_became_peered":"2023-12-19T18:16:27.834561+0000","last_unstale":"2023-12-19T18:16:27.836036+0000","last_undegraded":"2023-12-19T18:16:27.836036+0000","last_fullsized":"2023-12-19T18:16:27.836036+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:30.301379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.833793+0000","last_change":"2023-12-19T18:16:27.830119+0000","last_active":"2023-12-19T18:16:27.833793+0000","last_peered":"2023-12-19T18:16:27.833793+0000","last_clean":"2023-12-19T18:16:27.833793+0000","last_became_active":"2023-12-19T18:16:27.829850+0000","last_became_peered":"2023-12-19T18:16:27.829850+0000","last_unstale":"2023-12-19T18:16:27.833793+0000","last_undegraded":"2023-12-19T18:16:27.833793+0000","last_fullsized":"2023-12-19T18:16:27.833793+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:51.190615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.828863+0000","last_change":"2023-12-19T18:16:27.820913+0000","last_active":"2023-12-19T18:16:27.828863+0000","last_peered":"2023-12-19T18:16:27.828863+0000","last_clean":"2023-12-19T18:16:27.828863+0000","last_became_active":"2023-12-19T18:16:27.820634+0000","last_became_peered":"2023-12-19T18:16:27.820634+0000","last_unstale":"2023-12-19T18:16:27.828863+0000","last_undegraded":"2023-12-19T18:16:27.828863+0000","last_fullsized":"2023-12-19T18:16:27.828863+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:53:09.066279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834337+0000","last_change":"2023-12-19T18:16:27.830103+0000","last_active":"2023-12-19T18:16:27.834337+0000","last_peered":"2023-12-19T18:16:27.834337+0000","last_clean":"2023-12-19T18:16:27.834337+0000","last_became_active":"2023-12-19T18:16:27.829444+0000","last_became_peered":"2023-12-19T18:16:27.829444+0000","last_unstale":"2023-12-19T18:16:27.834337+0000","last_undegraded":"2023-12-19T18:16:27.834337+0000","last_fullsized":"2023-12-19T18:16:27.834337+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:50:29.947932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834019+0000","last_change":"2023-12-19T18:16:27.826500+0000","last_active":"2023-12-19T18:16:27.834019+0000","last_peered":"2023-12-19T18:16:27.834019+0000","last_clean":"2023-12-19T18:16:27.834019+0000","last_became_active":"2023-12-19T18:16:27.826216+0000","last_became_peered":"2023-12-19T18:16:27.826216+0000","last_unstale":"2023-12-19T18:16:27.834019+0000","last_undegraded":"2023-12-19T18:16:27.834019+0000","last_fullsized":"2023-12-19T18:16:27.834019+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:33.032362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.830779+0000","last_change":"2023-12-19T18:16:27.825014+0000","last_active":"2023-12-19T18:16:27.830779+0000","last_peered":"2023-12-19T18:16:27.830779+0000","last_clean":"2023-12-19T18:16:27.830779+0000","last_became_active":"2023-12-19T18:16:27.822140+0000","last_became_peered":"2023-12-19T18:16:27.822140+0000","last_unstale":"2023-12-19T18:16:27.830779+0000","last_undegraded":"2023-12-19T18:16:27.830779+0000","last_fullsized":"2023-12-19T18:16:27.830779+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:36.972809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834899+0000","last_change":"2023-12-19T18:16:27.833518+0000","last_active":"2023-12-19T18:16:27.834899+0000","last_peered":"2023-12-19T18:16:27.834899+0000","last_clean":"2023-12-19T18:16:27.834899+0000","last_became_active":"2023-12-19T18:16:27.833251+0000","last_became_peered":"2023-12-19T18:16:27.833251+0000","last_unstale":"2023-12-19T18:16:27.834899+0000","last_undegraded":"2023-12-19T18:16:27.834899+0000","last_fullsized":"2023-12-19T18:16:27.834899+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:49.276917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.836070+0000","last_change":"2023-12-19T18:16:27.831030+0000","last_active":"2023-12-19T18:16:27.836070+0000","last_peered":"2023-12-19T18:16:27.836070+0000","last_clean":"2023-12-19T18:16:27.836070+0000","last_became_active":"2023-12-19T18:16:27.830764+0000","last_became_peered":"2023-12-19T18:16:27.830764+0000","last_unstale":"2023-12-19T18:16:27.836070+0000","last_undegraded":"2023-12-19T18:16:27.836070+0000","last_fullsized":"2023-12-19T18:16:27.836070+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:11.685663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.839371+0000","last_change":"2023-12-19T18:16:27.830378+0000","last_active":"2023-12-19T18:16:27.839371+0000","last_peered":"2023-12-19T18:16:27.839371+0000","last_clean":"2023-12-19T18:16:27.839371+0000","last_became_active":"2023-12-19T18:16:27.830122+0000","last_became_peered":"2023-12-19T18:16:27.830122+0000","last_unstale":"2023-12-19T18:16:27.839371+0000","last_undegraded":"2023-12-19T18:16:27.839371+0000","last_fullsized":"2023-12-19T18:16:27.839371+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:21.365516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"211'4","reported_seq":14,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.835298+0000","last_change":"2023-12-19T18:16:27.414364+0000","last_active":"2023-12-19T18:16:27.835298+0000","last_peered":"2023-12-19T18:16:27.835298+0000","last_clean":"2023-12-19T18:16:27.835298+0000","last_became_active":"2023-12-19T18:16:27.413648+0000","last_became_peered":"2023-12-19T18:16:27.413648+0000","last_unstale":"2023-12-19T18:16:27.835298+0000","last_undegraded":"2023-12-19T18:16:27.835298+0000","last_fullsized":"2023-12-19T18:16:27.835298+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:27:32.703908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837160+0000","last_change":"2023-12-19T18:16:27.830766+0000","last_active":"2023-12-19T18:16:27.837160+0000","last_peered":"2023-12-19T18:16:27.837160+0000","last_clean":"2023-12-19T18:16:27.837160+0000","last_became_active":"2023-12-19T18:16:27.830220+0000","last_became_peered":"2023-12-19T18:16:27.830220+0000","last_unstale":"2023-12-19T18:16:27.837160+0000","last_undegraded":"2023-12-19T18:16:27.837160+0000","last_fullsized":"2023-12-19T18:16:27.837160+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:03:17.772391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"211'1","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.831972+0000","last_change":"2023-12-19T18:16:27.407669+0000","last_active":"2023-12-19T18:16:27.831972+0000","last_peered":"2023-12-19T18:16:27.831972+0000","last_clean":"2023-12-19T18:16:27.831972+0000","last_became_active":"2023-12-19T18:16:27.407150+0000","last_became_peered":"2023-12-19T18:16:27.407150+0000","last_unstale":"2023-12-19T18:16:27.831972+0000","last_undegraded":"2023-12-19T18:16:27.831972+0000","last_fullsized":"2023-12-19T18:16:27.831972+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:59.675090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.833517+0000","last_change":"2023-12-19T18:16:27.829031+0000","last_active":"2023-12-19T18:16:27.833517+0000","last_peered":"2023-12-19T18:16:27.833517+0000","last_clean":"2023-12-19T18:16:27.833517+0000","last_became_active":"2023-12-19T18:16:27.826634+0000","last_became_peered":"2023-12-19T18:16:27.826634+0000","last_unstale":"2023-12-19T18:16:27.833517+0000","last_undegraded":"2023-12-19T18:16:27.833517+0000","last_fullsized":"2023-12-19T18:16:27.833517+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:23.429228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"211'2","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834535+0000","last_change":"2023-12-19T18:16:27.415013+0000","last_active":"2023-12-19T18:16:27.834535+0000","last_peered":"2023-12-19T18:16:27.834535+0000","last_clean":"2023-12-19T18:16:27.834535+0000","last_became_active":"2023-12-19T18:16:27.414582+0000","last_became_peered":"2023-12-19T18:16:27.414582+0000","last_unstale":"2023-12-19T18:16:27.834535+0000","last_undegraded":"2023-12-19T18:16:27.834535+0000","last_fullsized":"2023-12-19T18:16:27.834535+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:34:01.457288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.814928+0000","last_change":"2023-12-19T18:16:27.814063+0000","last_active":"2023-12-19T18:16:27.814928+0000","last_peered":"2023-12-19T18:16:27.814928+0000","last_clean":"2023-12-19T18:16:27.814928+0000","last_became_active":"2023-12-19T18:16:27.813564+0000","last_became_peered":"2023-12-19T18:16:27.813564+0000","last_unstale":"2023-12-19T18:16:27.814928+0000","last_undegraded":"2023-12-19T18:16:27.814928+0000","last_fullsized":"2023-12-19T18:16:27.814928+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:36.733148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"211'2","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834457+0000","last_change":"2023-12-19T18:16:27.409441+0000","last_active":"2023-12-19T18:16:27.834457+0000","last_peered":"2023-12-19T18:16:27.834457+0000","last_clean":"2023-12-19T18:16:27.834457+0000","last_became_active":"2023-12-19T18:16:27.408438+0000","last_became_peered":"2023-12-19T18:16:27.408438+0000","last_unstale":"2023-12-19T18:16:27.834457+0000","last_undegraded":"2023-12-19T18:16:27.834457+0000","last_fullsized":"2023-12-19T18:16:27.834457+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:40.980300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837446+0000","last_change":"2023-12-19T18:16:27.828398+0000","last_active":"2023-12-19T18:16:27.837446+0000","last_peered":"2023-12-19T18:16:27.837446+0000","last_clean":"2023-12-19T18:16:27.837446+0000","last_became_active":"2023-12-19T18:16:27.828082+0000","last_became_peered":"2023-12-19T18:16:27.828082+0000","last_unstale":"2023-12-19T18:16:27.837446+0000","last_undegraded":"2023-12-19T18:16:27.837446+0000","last_fullsized":"2023-12-19T18:16:27.837446+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:42:30.998150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"211'1","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837387+0000","last_change":"2023-12-19T18:16:27.415036+0000","last_active":"2023-12-19T18:16:27.837387+0000","last_peered":"2023-12-19T18:16:27.837387+0000","last_clean":"2023-12-19T18:16:27.837387+0000","last_became_active":"2023-12-19T18:16:27.414747+0000","last_became_peered":"2023-12-19T18:16:27.414747+0000","last_unstale":"2023-12-19T18:16:27.837387+0000","last_undegraded":"2023-12-19T18:16:27.837387+0000","last_fullsized":"2023-12-19T18:16:27.837387+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:03.064138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.836984+0000","last_change":"2023-12-19T18:16:27.827386+0000","last_active":"2023-12-19T18:16:27.836984+0000","last_peered":"2023-12-19T18:16:27.836984+0000","last_clean":"2023-12-19T18:16:27.836984+0000","last_became_active":"2023-12-19T18:16:27.827086+0000","last_became_peered":"2023-12-19T18:16:27.827086+0000","last_unstale":"2023-12-19T18:16:27.836984+0000","last_undegraded":"2023-12-19T18:16:27.836984+0000","last_fullsized":"2023-12-19T18:16:27.836984+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:37.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"211'1","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.831347+0000","last_change":"2023-12-19T18:16:27.414372+0000","last_active":"2023-12-19T18:16:27.831347+0000","last_peered":"2023-12-19T18:16:27.831347+0000","last_clean":"2023-12-19T18:16:27.831347+0000","last_became_active":"2023-12-19T18:16:27.414088+0000","last_became_peered":"2023-12-19T18:16:27.414088+0000","last_unstale":"2023-12-19T18:16:27.831347+0000","last_undegraded":"2023-12-19T18:16:27.831347+0000","last_fullsized":"2023-12-19T18:16:27.831347+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:18.945766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.817641+0000","last_change":"2023-12-19T18:16:27.816735+0000","last_active":"2023-12-19T18:16:27.817641+0000","last_peered":"2023-12-19T18:16:27.817641+0000","last_clean":"2023-12-19T18:16:27.817641+0000","last_became_active":"2023-12-19T18:16:27.816436+0000","last_became_peered":"2023-12-19T18:16:27.816436+0000","last_unstale":"2023-12-19T18:16:27.817641+0000","last_undegraded":"2023-12-19T18:16:27.817641+0000","last_fullsized":"2023-12-19T18:16:27.817641+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:37.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.811238+0000","last_change":"2023-12-19T18:16:27.413444+0000","last_active":"2023-12-19T18:16:27.811238+0000","last_peered":"2023-12-19T18:16:27.811238+0000","last_clean":"2023-12-19T18:16:27.811238+0000","last_became_active":"2023-12-19T18:16:27.413066+0000","last_became_peered":"2023-12-19T18:16:27.413066+0000","last_unstale":"2023-12-19T18:16:27.811238+0000","last_undegraded":"2023-12-19T18:16:27.811238+0000","last_fullsized":"2023-12-19T18:16:27.811238+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:57.997657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834600+0000","last_change":"2023-12-19T18:16:27.830538+0000","last_active":"2023-12-19T18:16:27.834600+0000","last_peered":"2023-12-19T18:16:27.834600+0000","last_clean":"2023-12-19T18:16:27.834600+0000","last_became_active":"2023-12-19T18:16:27.828974+0000","last_became_peered":"2023-12-19T18:16:27.828974+0000","last_unstale":"2023-12-19T18:16:27.834600+0000","last_undegraded":"2023-12-19T18:16:27.834600+0000","last_fullsized":"2023-12-19T18:16:27.834600+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:36.961741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.809625+0000","last_change":"2023-12-19T18:16:27.405556+0000","last_active":"2023-12-19T18:16:27.809625+0000","last_peered":"2023-12-19T18:16:27.809625+0000","last_clean":"2023-12-19T18:16:27.809625+0000","last_became_active":"2023-12-19T18:16:27.404804+0000","last_became_peered":"2023-12-19T18:16:27.404804+0000","last_unstale":"2023-12-19T18:16:27.809625+0000","last_undegraded":"2023-12-19T18:16:27.809625+0000","last_fullsized":"2023-12-19T18:16:27.809625+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:05.395130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.839394+0000","last_change":"2023-12-19T18:16:27.835564+0000","last_active":"2023-12-19T18:16:27.839394+0000","last_peered":"2023-12-19T18:16:27.839394+0000","last_clean":"2023-12-19T18:16:27.839394+0000","last_became_active":"2023-12-19T18:16:27.835319+0000","last_became_peered":"2023-12-19T18:16:27.835319+0000","last_unstale":"2023-12-19T18:16:27.839394+0000","last_undegraded":"2023-12-19T18:16:27.839394+0000","last_fullsized":"2023-12-19T18:16:27.839394+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:10:35.037185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.820369+0000","last_change":"2023-12-19T18:16:27.410603+0000","last_active":"2023-12-19T18:16:27.820369+0000","last_peered":"2023-12-19T18:16:27.820369+0000","last_clean":"2023-12-19T18:16:27.820369+0000","last_became_active":"2023-12-19T18:16:27.409260+0000","last_became_peered":"2023-12-19T18:16:27.409260+0000","last_unstale":"2023-12-19T18:16:27.820369+0000","last_undegraded":"2023-12-19T18:16:27.820369+0000","last_fullsized":"2023-12-19T18:16:27.820369+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:06.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.812914+0000","last_change":"2023-12-19T18:16:27.811752+0000","last_active":"2023-12-19T18:16:27.812914+0000","last_peered":"2023-12-19T18:16:27.812914+0000","last_clean":"2023-12-19T18:16:27.812914+0000","last_became_active":"2023-12-19T18:16:27.811273+0000","last_became_peered":"2023-12-19T18:16:27.811273+0000","last_unstale":"2023-12-19T18:16:27.812914+0000","last_undegraded":"2023-12-19T18:16:27.812914+0000","last_fullsized":"2023-12-19T18:16:27.812914+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:23.609660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"211'3","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.832083+0000","last_change":"2023-12-19T18:16:27.409442+0000","last_active":"2023-12-19T18:16:27.832083+0000","last_peered":"2023-12-19T18:16:27.832083+0000","last_clean":"2023-12-19T18:16:27.832083+0000","last_became_active":"2023-12-19T18:16:27.409094+0000","last_became_peered":"2023-12-19T18:16:27.409094+0000","last_unstale":"2023-12-19T18:16:27.832083+0000","last_undegraded":"2023-12-19T18:16:27.832083+0000","last_fullsized":"2023-12-19T18:16:27.832083+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:18.713070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837195+0000","last_change":"2023-12-19T18:16:27.829311+0000","last_active":"2023-12-19T18:16:27.837195+0000","last_peered":"2023-12-19T18:16:27.837195+0000","last_clean":"2023-12-19T18:16:27.837195+0000","last_became_active":"2023-12-19T18:16:27.828953+0000","last_became_peered":"2023-12-19T18:16:27.828953+0000","last_unstale":"2023-12-19T18:16:27.837195+0000","last_undegraded":"2023-12-19T18:16:27.837195+0000","last_fullsized":"2023-12-19T18:16:27.837195+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:23.417710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"211'6","reported_seq":16,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.839649+0000","last_change":"2023-12-19T18:16:27.414573+0000","last_active":"2023-12-19T18:16:27.839649+0000","last_peered":"2023-12-19T18:16:27.839649+0000","last_clean":"2023-12-19T18:16:27.839649+0000","last_became_active":"2023-12-19T18:16:27.414268+0000","last_became_peered":"2023-12-19T18:16:27.414268+0000","last_unstale":"2023-12-19T18:16:27.839649+0000","last_undegraded":"2023-12-19T18:16:27.839649+0000","last_fullsized":"2023-12-19T18:16:27.839649+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:11.117214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.829693+0000","last_change":"2023-12-19T18:16:27.824833+0000","last_active":"2023-12-19T18:16:27.829693+0000","last_peered":"2023-12-19T18:16:27.829693+0000","last_clean":"2023-12-19T18:16:27.829693+0000","last_became_active":"2023-12-19T18:16:27.824487+0000","last_became_peered":"2023-12-19T18:16:27.824487+0000","last_unstale":"2023-12-19T18:16:27.829693+0000","last_undegraded":"2023-12-19T18:16:27.829693+0000","last_fullsized":"2023-12-19T18:16:27.829693+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"211'2","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.838393+0000","last_change":"2023-12-19T18:16:27.415000+0000","last_active":"2023-12-19T18:16:27.838393+0000","last_peered":"2023-12-19T18:16:27.838393+0000","last_clean":"2023-12-19T18:16:27.838393+0000","last_became_active":"2023-12-19T18:16:27.414679+0000","last_became_peered":"2023-12-19T18:16:27.414679+0000","last_unstale":"2023-12-19T18:16:27.838393+0000","last_undegraded":"2023-12-19T18:16:27.838393+0000","last_fullsized":"2023-12-19T18:16:27.838393+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:50.187832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.831349+0000","last_change":"2023-12-19T18:16:27.824516+0000","last_active":"2023-12-19T18:16:27.831349+0000","last_peered":"2023-12-19T18:16:27.831349+0000","last_clean":"2023-12-19T18:16:27.831349+0000","last_became_active":"2023-12-19T18:16:27.823143+0000","last_became_peered":"2023-12-19T18:16:27.823143+0000","last_unstale":"2023-12-19T18:16:27.831349+0000","last_undegraded":"2023-12-19T18:16:27.831349+0000","last_fullsized":"2023-12-19T18:16:27.831349+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:52.341930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.804347+0000","last_change":"2023-12-19T18:16:27.417401+0000","last_active":"2023-12-19T18:16:27.804347+0000","last_peered":"2023-12-19T18:16:27.804347+0000","last_clean":"2023-12-19T18:16:27.804347+0000","last_became_active":"2023-12-19T18:16:27.416774+0000","last_became_peered":"2023-12-19T18:16:27.416774+0000","last_unstale":"2023-12-19T18:16:27.804347+0000","last_undegraded":"2023-12-19T18:16:27.804347+0000","last_fullsized":"2023-12-19T18:16:27.804347+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:03.620596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.827050+0000","last_change":"2023-12-19T18:16:27.821814+0000","last_active":"2023-12-19T18:16:27.827050+0000","last_peered":"2023-12-19T18:16:27.827050+0000","last_clean":"2023-12-19T18:16:27.827050+0000","last_became_active":"2023-12-19T18:16:27.821519+0000","last_became_peered":"2023-12-19T18:16:27.821519+0000","last_unstale":"2023-12-19T18:16:27.827050+0000","last_undegraded":"2023-12-19T18:16:27.827050+0000","last_fullsized":"2023-12-19T18:16:27.827050+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:40.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.813482+0000","last_change":"2023-12-19T18:16:27.403403+0000","last_active":"2023-12-19T18:16:27.813482+0000","last_peered":"2023-12-19T18:16:27.813482+0000","last_clean":"2023-12-19T18:16:27.813482+0000","last_became_active":"2023-12-19T18:16:27.403121+0000","last_became_peered":"2023-12-19T18:16:27.403121+0000","last_unstale":"2023-12-19T18:16:27.813482+0000","last_undegraded":"2023-12-19T18:16:27.813482+0000","last_fullsized":"2023-12-19T18:16:27.813482+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:36:58.560143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.824494+0000","last_change":"2023-12-19T18:16:27.814277+0000","last_active":"2023-12-19T18:16:27.824494+0000","last_peered":"2023-12-19T18:16:27.824494+0000","last_clean":"2023-12-19T18:16:27.824494+0000","last_became_active":"2023-12-19T18:16:27.814023+0000","last_became_peered":"2023-12-19T18:16:27.814023+0000","last_unstale":"2023-12-19T18:16:27.824494+0000","last_undegraded":"2023-12-19T18:16:27.824494+0000","last_fullsized":"2023-12-19T18:16:27.824494+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:32.135133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"211'8","reported_seq":18,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.839966+0000","last_change":"2023-12-19T18:16:27.415118+0000","last_active":"2023-12-19T18:16:27.839966+0000","last_peered":"2023-12-19T18:16:27.839966+0000","last_clean":"2023-12-19T18:16:27.839966+0000","last_became_active":"2023-12-19T18:16:27.414687+0000","last_became_peered":"2023-12-19T18:16:27.414687+0000","last_unstale":"2023-12-19T18:16:27.839966+0000","last_undegraded":"2023-12-19T18:16:27.839966+0000","last_fullsized":"2023-12-19T18:16:27.839966+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:16.723922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.825345+0000","last_change":"2023-12-19T18:16:27.814804+0000","last_active":"2023-12-19T18:16:27.825345+0000","last_peered":"2023-12-19T18:16:27.825345+0000","last_clean":"2023-12-19T18:16:27.825345+0000","last_became_active":"2023-12-19T18:16:27.814534+0000","last_became_peered":"2023-12-19T18:16:27.814534+0000","last_unstale":"2023-12-19T18:16:27.825345+0000","last_undegraded":"2023-12-19T18:16:27.825345+0000","last_fullsized":"2023-12-19T18:16:27.825345+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:45.690711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"211'1","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834739+0000","last_change":"2023-12-19T18:16:27.414971+0000","last_active":"2023-12-19T18:16:27.834739+0000","last_peered":"2023-12-19T18:16:27.834739+0000","last_clean":"2023-12-19T18:16:27.834739+0000","last_became_active":"2023-12-19T18:16:27.414662+0000","last_became_peered":"2023-12-19T18:16:27.414662+0000","last_unstale":"2023-12-19T18:16:27.834739+0000","last_undegraded":"2023-12-19T18:16:27.834739+0000","last_fullsized":"2023-12-19T18:16:27.834739+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:47.204637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.827264+0000","last_change":"2023-12-19T18:16:27.823846+0000","last_active":"2023-12-19T18:16:27.827264+0000","last_peered":"2023-12-19T18:16:27.827264+0000","last_clean":"2023-12-19T18:16:27.827264+0000","last_became_active":"2023-12-19T18:16:27.823514+0000","last_became_peered":"2023-12-19T18:16:27.823514+0000","last_unstale":"2023-12-19T18:16:27.827264+0000","last_undegraded":"2023-12-19T18:16:27.827264+0000","last_fullsized":"2023-12-19T18:16:27.827264+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:20.691537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"211'2","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.830784+0000","last_change":"2023-12-19T18:16:27.409426+0000","last_active":"2023-12-19T18:16:27.830784+0000","last_peered":"2023-12-19T18:16:27.830784+0000","last_clean":"2023-12-19T18:16:27.830784+0000","last_became_active":"2023-12-19T18:16:27.407151+0000","last_became_peered":"2023-12-19T18:16:27.407151+0000","last_unstale":"2023-12-19T18:16:27.830784+0000","last_undegraded":"2023-12-19T18:16:27.830784+0000","last_fullsized":"2023-12-19T18:16:27.830784+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:30:56.976667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.820630+0000","last_change":"2023-12-19T18:16:27.816436+0000","last_active":"2023-12-19T18:16:27.820630+0000","last_peered":"2023-12-19T18:16:27.820630+0000","last_clean":"2023-12-19T18:16:27.820630+0000","last_became_active":"2023-12-19T18:16:27.816092+0000","last_became_peered":"2023-12-19T18:16:27.816092+0000","last_unstale":"2023-12-19T18:16:27.820630+0000","last_undegraded":"2023-12-19T18:16:27.820630+0000","last_fullsized":"2023-12-19T18:16:27.820630+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:07:26.713649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"211'1","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.836314+0000","last_change":"2023-12-19T18:16:27.416654+0000","last_active":"2023-12-19T18:16:27.836314+0000","last_peered":"2023-12-19T18:16:27.836314+0000","last_clean":"2023-12-19T18:16:27.836314+0000","last_became_active":"2023-12-19T18:16:27.416388+0000","last_became_peered":"2023-12-19T18:16:27.416388+0000","last_unstale":"2023-12-19T18:16:27.836314+0000","last_undegraded":"2023-12-19T18:16:27.836314+0000","last_fullsized":"2023-12-19T18:16:27.836314+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:51.339148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.835698+0000","last_change":"2023-12-19T18:16:27.832376+0000","last_active":"2023-12-19T18:16:27.835698+0000","last_peered":"2023-12-19T18:16:27.835698+0000","last_clean":"2023-12-19T18:16:27.835698+0000","last_became_active":"2023-12-19T18:16:27.832060+0000","last_became_peered":"2023-12-19T18:16:27.832060+0000","last_unstale":"2023-12-19T18:16:27.835698+0000","last_undegraded":"2023-12-19T18:16:27.835698+0000","last_fullsized":"2023-12-19T18:16:27.835698+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:17.432889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.809631+0000","last_change":"2023-12-19T18:16:27.408051+0000","last_active":"2023-12-19T18:16:27.809631+0000","last_peered":"2023-12-19T18:16:27.809631+0000","last_clean":"2023-12-19T18:16:27.809631+0000","last_became_active":"2023-12-19T18:16:27.407796+0000","last_became_peered":"2023-12-19T18:16:27.407796+0000","last_unstale":"2023-12-19T18:16:27.809631+0000","last_undegraded":"2023-12-19T18:16:27.809631+0000","last_fullsized":"2023-12-19T18:16:27.809631+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:41:09.697612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837427+0000","last_change":"2023-12-19T18:16:27.827898+0000","last_active":"2023-12-19T18:16:27.837427+0000","last_peered":"2023-12-19T18:16:27.837427+0000","last_clean":"2023-12-19T18:16:27.837427+0000","last_became_active":"2023-12-19T18:16:27.827613+0000","last_became_peered":"2023-12-19T18:16:27.827613+0000","last_unstale":"2023-12-19T18:16:27.837427+0000","last_undegraded":"2023-12-19T18:16:27.837427+0000","last_fullsized":"2023-12-19T18:16:27.837427+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:29.033205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"211'2","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.832328+0000","last_change":"2023-12-19T18:16:27.415902+0000","last_active":"2023-12-19T18:16:27.832328+0000","last_peered":"2023-12-19T18:16:27.832328+0000","last_clean":"2023-12-19T18:16:27.832328+0000","last_became_active":"2023-12-19T18:16:27.415062+0000","last_became_peered":"2023-12-19T18:16:27.415062+0000","last_unstale":"2023-12-19T18:16:27.832328+0000","last_undegraded":"2023-12-19T18:16:27.832328+0000","last_fullsized":"2023-12-19T18:16:27.832328+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:22.442653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.832685+0000","last_change":"2023-12-19T18:16:27.828160+0000","last_active":"2023-12-19T18:16:27.832685+0000","last_peered":"2023-12-19T18:16:27.832685+0000","last_clean":"2023-12-19T18:16:27.832685+0000","last_became_active":"2023-12-19T18:16:27.827882+0000","last_became_peered":"2023-12-19T18:16:27.827882+0000","last_unstale":"2023-12-19T18:16:27.832685+0000","last_undegraded":"2023-12-19T18:16:27.832685+0000","last_fullsized":"2023-12-19T18:16:27.832685+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:17.726704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.815898+0000","last_change":"2023-12-19T18:16:27.404853+0000","last_active":"2023-12-19T18:16:27.815898+0000","last_peered":"2023-12-19T18:16:27.815898+0000","last_clean":"2023-12-19T18:16:27.815898+0000","last_became_active":"2023-12-19T18:16:27.404347+0000","last_became_peered":"2023-12-19T18:16:27.404347+0000","last_unstale":"2023-12-19T18:16:27.815898+0000","last_undegraded":"2023-12-19T18:16:27.815898+0000","last_fullsized":"2023-12-19T18:16:27.815898+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:16.360996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834048+0000","last_change":"2023-12-19T18:16:27.824198+0000","last_active":"2023-12-19T18:16:27.834048+0000","last_peered":"2023-12-19T18:16:27.834048+0000","last_clean":"2023-12-19T18:16:27.834048+0000","last_became_active":"2023-12-19T18:16:27.823738+0000","last_became_peered":"2023-12-19T18:16:27.823738+0000","last_unstale":"2023-12-19T18:16:27.834048+0000","last_undegraded":"2023-12-19T18:16:27.834048+0000","last_fullsized":"2023-12-19T18:16:27.834048+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:53.509897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.811125+0000","last_change":"2023-12-19T18:16:27.417198+0000","last_active":"2023-12-19T18:16:27.811125+0000","last_peered":"2023-12-19T18:16:27.811125+0000","last_clean":"2023-12-19T18:16:27.811125+0000","last_became_active":"2023-12-19T18:16:27.416902+0000","last_became_peered":"2023-12-19T18:16:27.416902+0000","last_unstale":"2023-12-19T18:16:27.811125+0000","last_undegraded":"2023-12-19T18:16:27.811125+0000","last_fullsized":"2023-12-19T18:16:27.811125+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:54.434372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.832288+0000","last_change":"2023-12-19T18:16:27.825355+0000","last_active":"2023-12-19T18:16:27.832288+0000","last_peered":"2023-12-19T18:16:27.832288+0000","last_clean":"2023-12-19T18:16:27.832288+0000","last_became_active":"2023-12-19T18:16:27.825067+0000","last_became_peered":"2023-12-19T18:16:27.825067+0000","last_unstale":"2023-12-19T18:16:27.832288+0000","last_undegraded":"2023-12-19T18:16:27.832288+0000","last_fullsized":"2023-12-19T18:16:27.832288+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:02.928407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.807242+0000","last_change":"2023-12-19T18:16:27.403537+0000","last_active":"2023-12-19T18:16:27.807242+0000","last_peered":"2023-12-19T18:16:27.807242+0000","last_clean":"2023-12-19T18:16:27.807242+0000","last_became_active":"2023-12-19T18:16:27.403250+0000","last_became_peered":"2023-12-19T18:16:27.403250+0000","last_unstale":"2023-12-19T18:16:27.807242+0000","last_undegraded":"2023-12-19T18:16:27.807242+0000","last_fullsized":"2023-12-19T18:16:27.807242+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:56.889902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.833161+0000","last_change":"2023-12-19T18:16:27.826090+0000","last_active":"2023-12-19T18:16:27.833161+0000","last_peered":"2023-12-19T18:16:27.833161+0000","last_clean":"2023-12-19T18:16:27.833161+0000","last_became_active":"2023-12-19T18:16:27.825791+0000","last_became_peered":"2023-12-19T18:16:27.825791+0000","last_unstale":"2023-12-19T18:16:27.833161+0000","last_undegraded":"2023-12-19T18:16:27.833161+0000","last_fullsized":"2023-12-19T18:16:27.833161+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:39.382953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"211'2","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.838073+0000","last_change":"2023-12-19T18:16:27.409874+0000","last_active":"2023-12-19T18:16:27.838073+0000","last_peered":"2023-12-19T18:16:27.838073+0000","last_clean":"2023-12-19T18:16:27.838073+0000","last_became_active":"2023-12-19T18:16:27.409598+0000","last_became_peered":"2023-12-19T18:16:27.409598+0000","last_unstale":"2023-12-19T18:16:27.838073+0000","last_undegraded":"2023-12-19T18:16:27.838073+0000","last_fullsized":"2023-12-19T18:16:27.838073+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:13.721057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.833968+0000","last_change":"2023-12-19T18:16:27.824257+0000","last_active":"2023-12-19T18:16:27.833968+0000","last_peered":"2023-12-19T18:16:27.833968+0000","last_clean":"2023-12-19T18:16:27.833968+0000","last_became_active":"2023-12-19T18:16:27.823989+0000","last_became_peered":"2023-12-19T18:16:27.823989+0000","last_unstale":"2023-12-19T18:16:27.833968+0000","last_undegraded":"2023-12-19T18:16:27.833968+0000","last_fullsized":"2023-12-19T18:16:27.833968+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:07:07.505127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.805262+0000","last_change":"2023-12-19T18:16:27.414152+0000","last_active":"2023-12-19T18:16:27.805262+0000","last_peered":"2023-12-19T18:16:27.805262+0000","last_clean":"2023-12-19T18:16:27.805262+0000","last_became_active":"2023-12-19T18:16:27.413877+0000","last_became_peered":"2023-12-19T18:16:27.413877+0000","last_unstale":"2023-12-19T18:16:27.805262+0000","last_undegraded":"2023-12-19T18:16:27.805262+0000","last_fullsized":"2023-12-19T18:16:27.805262+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:44:52.553685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.835195+0000","last_change":"2023-12-19T18:16:27.825535+0000","last_active":"2023-12-19T18:16:27.835195+0000","last_peered":"2023-12-19T18:16:27.835195+0000","last_clean":"2023-12-19T18:16:27.835195+0000","last_became_active":"2023-12-19T18:16:27.824814+0000","last_became_peered":"2023-12-19T18:16:27.824814+0000","last_unstale":"2023-12-19T18:16:27.835195+0000","last_undegraded":"2023-12-19T18:16:27.835195+0000","last_fullsized":"2023-12-19T18:16:27.835195+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:29.357409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.804201+0000","last_change":"2023-12-19T18:16:27.420249+0000","last_active":"2023-12-19T18:16:27.804201+0000","last_peered":"2023-12-19T18:16:27.804201+0000","last_clean":"2023-12-19T18:16:27.804201+0000","last_became_active":"2023-12-19T18:16:27.419360+0000","last_became_peered":"2023-12-19T18:16:27.419360+0000","last_unstale":"2023-12-19T18:16:27.804201+0000","last_undegraded":"2023-12-19T18:16:27.804201+0000","last_fullsized":"2023-12-19T18:16:27.804201+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:07.969904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834501+0000","last_change":"2023-12-19T18:16:27.825200+0000","last_active":"2023-12-19T18:16:27.834501+0000","last_peered":"2023-12-19T18:16:27.834501+0000","last_clean":"2023-12-19T18:16:27.834501+0000","last_became_active":"2023-12-19T18:16:27.824919+0000","last_became_peered":"2023-12-19T18:16:27.824919+0000","last_unstale":"2023-12-19T18:16:27.834501+0000","last_undegraded":"2023-12-19T18:16:27.834501+0000","last_fullsized":"2023-12-19T18:16:27.834501+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.817261+0000","last_change":"2023-12-19T18:16:27.409169+0000","last_active":"2023-12-19T18:16:27.817261+0000","last_peered":"2023-12-19T18:16:27.817261+0000","last_clean":"2023-12-19T18:16:27.817261+0000","last_became_active":"2023-12-19T18:16:27.408882+0000","last_became_peered":"2023-12-19T18:16:27.408882+0000","last_unstale":"2023-12-19T18:16:27.817261+0000","last_undegraded":"2023-12-19T18:16:27.817261+0000","last_fullsized":"2023-12-19T18:16:27.817261+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:01.478246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.832772+0000","last_change":"2023-12-19T18:16:27.829925+0000","last_active":"2023-12-19T18:16:27.832772+0000","last_peered":"2023-12-19T18:16:27.832772+0000","last_clean":"2023-12-19T18:16:27.832772+0000","last_became_active":"2023-12-19T18:16:27.828958+0000","last_became_peered":"2023-12-19T18:16:27.828958+0000","last_unstale":"2023-12-19T18:16:27.832772+0000","last_undegraded":"2023-12-19T18:16:27.832772+0000","last_fullsized":"2023-12-19T18:16:27.832772+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:46.995476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.807783+0000","last_change":"2023-12-19T18:16:27.410513+0000","last_active":"2023-12-19T18:16:27.807783+0000","last_peered":"2023-12-19T18:16:27.807783+0000","last_clean":"2023-12-19T18:16:27.807783+0000","last_became_active":"2023-12-19T18:16:27.410166+0000","last_became_peered":"2023-12-19T18:16:27.410166+0000","last_unstale":"2023-12-19T18:16:27.807783+0000","last_undegraded":"2023-12-19T18:16:27.807783+0000","last_fullsized":"2023-12-19T18:16:27.807783+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:10.682880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.828526+0000","last_change":"2023-12-19T18:16:27.822423+0000","last_active":"2023-12-19T18:16:27.828526+0000","last_peered":"2023-12-19T18:16:27.828526+0000","last_clean":"2023-12-19T18:16:27.828526+0000","last_became_active":"2023-12-19T18:16:27.822106+0000","last_became_peered":"2023-12-19T18:16:27.822106+0000","last_unstale":"2023-12-19T18:16:27.828526+0000","last_undegraded":"2023-12-19T18:16:27.828526+0000","last_fullsized":"2023-12-19T18:16:27.828526+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:41.021760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.807347+0000","last_change":"2023-12-19T18:16:27.405381+0000","last_active":"2023-12-19T18:16:27.807347+0000","last_peered":"2023-12-19T18:16:27.807347+0000","last_clean":"2023-12-19T18:16:27.807347+0000","last_became_active":"2023-12-19T18:16:27.405113+0000","last_became_peered":"2023-12-19T18:16:27.405113+0000","last_unstale":"2023-12-19T18:16:27.807347+0000","last_undegraded":"2023-12-19T18:16:27.807347+0000","last_fullsized":"2023-12-19T18:16:27.807347+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:08.911302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.833551+0000","last_change":"2023-12-19T18:16:27.824001+0000","last_active":"2023-12-19T18:16:27.833551+0000","last_peered":"2023-12-19T18:16:27.833551+0000","last_clean":"2023-12-19T18:16:27.833551+0000","last_became_active":"2023-12-19T18:16:27.823010+0000","last_became_peered":"2023-12-19T18:16:27.823010+0000","last_unstale":"2023-12-19T18:16:27.833551+0000","last_undegraded":"2023-12-19T18:16:27.833551+0000","last_fullsized":"2023-12-19T18:16:27.833551+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:30:56.775994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.825851+0000","last_change":"2023-12-19T18:16:27.407456+0000","last_active":"2023-12-19T18:16:27.825851+0000","last_peered":"2023-12-19T18:16:27.825851+0000","last_clean":"2023-12-19T18:16:27.825851+0000","last_became_active":"2023-12-19T18:16:27.407208+0000","last_became_peered":"2023-12-19T18:16:27.407208+0000","last_unstale":"2023-12-19T18:16:27.825851+0000","last_undegraded":"2023-12-19T18:16:27.825851+0000","last_fullsized":"2023-12-19T18:16:27.825851+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:07.654229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.837641+0000","last_change":"2023-12-19T18:16:27.832432+0000","last_active":"2023-12-19T18:16:27.837641+0000","last_peered":"2023-12-19T18:16:27.837641+0000","last_clean":"2023-12-19T18:16:27.837641+0000","last_became_active":"2023-12-19T18:16:27.832116+0000","last_became_peered":"2023-12-19T18:16:27.832116+0000","last_unstale":"2023-12-19T18:16:27.837641+0000","last_undegraded":"2023-12-19T18:16:27.837641+0000","last_fullsized":"2023-12-19T18:16:27.837641+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:35.165338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.811087+0000","last_change":"2023-12-19T18:16:27.412171+0000","last_active":"2023-12-19T18:16:27.811087+0000","last_peered":"2023-12-19T18:16:27.811087+0000","last_clean":"2023-12-19T18:16:27.811087+0000","last_became_active":"2023-12-19T18:16:27.411872+0000","last_became_peered":"2023-12-19T18:16:27.411872+0000","last_unstale":"2023-12-19T18:16:27.811087+0000","last_undegraded":"2023-12-19T18:16:27.811087+0000","last_fullsized":"2023-12-19T18:16:27.811087+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:48.264905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.834193+0000","last_change":"2023-12-19T18:16:27.821899+0000","last_active":"2023-12-19T18:16:27.834193+0000","last_peered":"2023-12-19T18:16:27.834193+0000","last_clean":"2023-12-19T18:16:27.834193+0000","last_became_active":"2023-12-19T18:16:27.821634+0000","last_became_peered":"2023-12-19T18:16:27.821634+0000","last_unstale":"2023-12-19T18:16:27.834193+0000","last_undegraded":"2023-12-19T18:16:27.834193+0000","last_fullsized":"2023-12-19T18:16:27.834193+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:34.391068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.811113+0000","last_change":"2023-12-19T18:16:27.411230+0000","last_active":"2023-12-19T18:16:27.811113+0000","last_peered":"2023-12-19T18:16:27.811113+0000","last_clean":"2023-12-19T18:16:27.811113+0000","last_became_active":"2023-12-19T18:16:27.410670+0000","last_became_peered":"2023-12-19T18:16:27.410670+0000","last_unstale":"2023-12-19T18:16:27.811113+0000","last_undegraded":"2023-12-19T18:16:27.811113+0000","last_fullsized":"2023-12-19T18:16:27.811113+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:22.426446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"200'346","reported_seq":1183,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.400271+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:16:27.400271+0000","last_peered":"2023-12-19T18:16:27.400271+0000","last_clean":"2023-12-19T18:16:27.400271+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:16:27.400271+0000","last_undegraded":"2023-12-19T18:16:27.400271+0000","last_fullsized":"2023-12-19T18:16:27.400271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510566,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64544,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307296,"statfs":{"total":96636764160,"available":96570671104,"internally_reserved":0,"allocated":3235840,"data_stored":1982617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96399999999999997,"5min":1.048,"15min":1.1180000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92400000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92400000000000004,"5min":1.0389999999999999,"15min":1.0580000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72399999999999998,"15min":0.748},"min":{"1min":0.499,"5min":0.48099999999999998,"15min":0.435},"max":{"1min":0.89500000000000002,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.69899999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.38900000000000001},"max":{"1min":0.93400000000000005,"5min":1.026,"15min":1.0669999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.0820000000000001,"15min":1.236},"last":0.75900000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.75600000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88300000000000001,"5min":1.002,"15min":1.079},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98899999999999999,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.71799999999999997}]},{"osd":4,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81599999999999995,"5min":1.0009999999999999,"15min":1.302},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.871,"15min":1.0209999999999999},"last":0.53600000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.52700000000000002,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.88100000000000001,"5min":1.097,"15min":1.1299999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64900000000000002,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":1.026,"15min":1.0800000000000001},"last":0.55800000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.56999999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.93999999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85699999999999998,"5min":1.01,"15min":1.093},"last":0.67200000000000004}]}]},{"osd":6,"up_from":15,"seq":64424510566,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47572,"kb_used_data":2572,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324268,"statfs":{"total":96636764160,"available":96588050432,"internally_reserved":0,"allocated":2633728,"data_stored":1392145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72799999999999998,"15min":0.74199999999999999},"min":{"1min":0.55400000000000005,"5min":0.46800000000000003,"15min":0.43099999999999999},"max":{"1min":0.92400000000000004,"5min":0.96499999999999997,"15min":1.042},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71799999999999997,"15min":0.73499999999999999},"min":{"1min":0.57499999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92000000000000004,"5min":1.224,"15min":1.224},"last":0.54300000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70099999999999996,"15min":0.72799999999999998},"min":{"1min":0.49099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98099999999999998,"5min":1.103,"15min":1.103},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.64100000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":0.94599999999999995,"5min":0.99199999999999999,"15min":1.1259999999999999},"last":0.57199999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.502},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0609999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.749},"min":{"1min":0.60899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83299999999999996,"5min":1.0089999999999999,"15min":1.111},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.755,"15min":0.78200000000000003},"min":{"1min":0.61799999999999999,"5min":0.495,"15min":0.495},"max":{"1min":0.91200000000000003,"5min":0.97999999999999998,"15min":1.1519999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.51800000000000002,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.98699999999999999,"5min":1.268,"15min":1.268},"last":0.55900000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92200000000000004,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":0.81299999999999994,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.434}]},{"osd":5,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.93799999999999994,"5min":1.194,"15min":1.194},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":1.141,"15min":1.141},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.95899999999999996,"5min":1.159,"15min":1.159},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94199999999999995,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.67500000000000004}]}]},{"osd":5,"up_from":15,"seq":64424510566,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47560,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324280,"statfs":{"total":96636764160,"available":96588062720,"internally_reserved":0,"allocated":2621440,"data_stored":1384610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.72299999999999998,"15min":0.72899999999999998},"min":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.34399999999999997},"max":{"1min":0.95699999999999996,"5min":1.0109999999999999,"15min":1.0149999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.74099999999999999,"15min":0.74299999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.86899999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.76100000000000001,"15min":0.75900000000000001},"min":{"1min":0.53300000000000003,"5min":0.497,"15min":0.497},"max":{"1min":0.94199999999999995,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73199999999999998,"15min":0.73799999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.83299999999999996}]},{"osd":2,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.77800000000000002,"15min":0.77900000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":0.97599999999999998,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.004,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.94499999999999995}]},{"osd":3,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76900000000000002,"15min":0.76600000000000001},"min":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95799999999999996,"5min":1.05,"15min":1.05},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.77300000000000002,"15min":0.77400000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.0329999999999999},"last":0.97099999999999997}]},{"osd":4,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59999999999999998,"15min":0.57999999999999996},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.35399999999999998},"max":{"1min":0.80500000000000005,"5min":0.94499999999999995,"15min":1.0800000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.64500000000000002,"15min":0.628},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.41599999999999998},"max":{"1min":0.85499999999999998,"5min":1.208,"15min":1.208},"last":0.56699999999999995}]},{"osd":6,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.64800000000000002,"15min":0.63400000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.98999999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.67500000000000004,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.40699999999999997},"max":{"1min":0.90400000000000003,"5min":1.008,"15min":1.0449999999999999},"last":0.71399999999999997}]},{"osd":7,"last update":"Tue Dec 19 18:15:27 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.70799999999999996,"15min":0.68600000000000005},"min":{"1min":0.48699999999999999,"5min":0.439,"15min":0.436},"max":{"1min":0.78400000000000003,"5min":1.169,"15min":1.169},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71699999999999997,"15min":0.71099999999999997},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.81999999999999995,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.748}]}]},{"osd":4,"up_from":15,"seq":64424510566,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47548,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324292,"statfs":{"total":96636764160,"available":96588075008,"internally_reserved":0,"allocated":2609152,"data_stored":1377575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.747,"15min":0.74099999999999999},"min":{"1min":0.51200000000000001,"5min":0.45800000000000002,"15min":0.35899999999999999},"max":{"1min":1.034,"5min":1.036,"15min":1.109},"last":0.626},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90100000000000002,"5min":0.997,"15min":1.01},"last":0.73099999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77100000000000002,"15min":0.76000000000000001},"min":{"1min":0.52700000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0049999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.755},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.49099999999999999,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.96999999999999997,"5min":0.997,"15min":1.1020000000000001},"last":0.747}]},{"osd":2,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78600000000000003,"15min":0.77400000000000002},"min":{"1min":0.56100000000000005,"5min":0.54800000000000004,"15min":0.47699999999999998},"max":{"1min":1.012,"5min":1.0289999999999999,"15min":1.071},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.79000000000000004,"15min":0.77600000000000002},"min":{"1min":0.503,"5min":0.503,"15min":0.47799999999999998},"max":{"1min":0.97099999999999997,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.68899999999999995}]},{"osd":3,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76400000000000001,"15min":0.75900000000000001},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.501},"max":{"1min":0.96799999999999997,"5min":0.98499999999999999,"15min":1.0409999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80400000000000005,"15min":0.79400000000000004},"min":{"1min":0.63800000000000001,"5min":0.61399999999999999,"15min":0.55000000000000004},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.056},"last":0.79900000000000004}]},{"osd":5,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.64300000000000002,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.86199999999999999,"5min":0.95699999999999996,"15min":1.1020000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.66400000000000003,"15min":0.64500000000000002},"min":{"1min":0.53100000000000003,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.93300000000000005,"5min":1.042,"15min":1.042},"last":0.64500000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67700000000000005,"15min":0.65800000000000003},"min":{"1min":0.49299999999999999,"5min":0.437,"15min":0.40200000000000002},"max":{"1min":0.91000000000000003,"5min":1.024,"15min":1.0669999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72699999999999998,"15min":0.71399999999999997},"min":{"1min":0.48899999999999999,"5min":0.48499999999999999,"15min":0.45600000000000002},"max":{"1min":0.995,"5min":0.995,"15min":1.0760000000000001},"last":0.59999999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:15:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72599999999999998,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.41899999999999998},"max":{"1min":0.96299999999999997,"5min":1.087,"15min":1.087},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.46999999999999997},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70999999999999996}]}]},{"osd":3,"up_from":15,"seq":64424510566,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47560,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324280,"statfs":{"total":96636764160,"available":96588062720,"internally_reserved":0,"allocated":2621440,"data_stored":1384610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.5,"15min":0.48699999999999999},"min":{"1min":0.31900000000000001,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.375},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.53500000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74399999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.45500000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.502,"15min":0.501},"min":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.39600000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.49099999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58299999999999996,"15min":0.57799999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.442,"15min":0.40100000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.57499999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.83399999999999996,"15min":0.83699999999999997},"min":{"1min":0.65400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0609999999999999,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83499999999999996,"15min":0.83199999999999996},"min":{"1min":0.68899999999999995,"5min":0.57499999999999996,"15min":0.56599999999999995},"max":{"1min":1.085,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.67200000000000004}]},{"osd":5,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.63100000000000001,"5min":0.60999999999999999,"15min":0.57399999999999995},"max":{"1min":1.087,"5min":1.1739999999999999,"15min":1.397},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.84299999999999997,"15min":0.83999999999999997},"min":{"1min":0.60899999999999999,"5min":0.55900000000000005,"15min":0.55100000000000005},"max":{"1min":1.1919999999999999,"5min":1.2230000000000001,"15min":1.262},"last":0.752}]},{"osd":6,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.85399999999999998,"15min":0.85799999999999998},"min":{"1min":0.67100000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.175,"5min":1.234,"15min":1.274},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.85299999999999998,"15min":0.85599999999999998},"min":{"1min":0.65500000000000003,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.083,"5min":1.165,"15min":1.349},"last":0.68300000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:15:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.92100000000000004,"15min":0.91500000000000004},"min":{"1min":0.71799999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.157,"5min":1.3540000000000001,"15min":1.486},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.92000000000000004,"15min":0.90700000000000003},"min":{"1min":0.73199999999999998,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.80800000000000005}]}]},{"osd":2,"up_from":15,"seq":64424510565,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47584,"kb_used_data":2584,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324256,"statfs":{"total":96636764160,"available":96588038144,"internally_reserved":0,"allocated":2646016,"data_stored":1392839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.29299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.34100000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.41499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.432}]},{"osd":3,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.86199999999999999,"15min":0.95999999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.36899999999999999},"max":{"1min":0.67200000000000004,"5min":0.79700000000000004,"15min":0.875},"last":0.48599999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.624,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.048,"5min":1.306,"15min":1.306},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.05,"5min":1.272,"15min":1.466},"last":0.79600000000000004}]},{"osd":5,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.64300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.97699999999999998,"5min":1.1459999999999999,"15min":1.3049999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.81999999999999995,"15min":0.82599999999999996},"min":{"1min":0.61899999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.429},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.71199999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.216,"5min":1.365,"15min":1.4119999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80600000000000005,"15min":0.80800000000000005},"min":{"1min":0.625,"5min":0.55200000000000005,"15min":0.53500000000000003},"max":{"1min":0.94499999999999995,"5min":1.2,"15min":1.2470000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81499999999999995,"15min":0.82799999999999996},"min":{"1min":0.64400000000000002,"5min":0.59899999999999998,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.393,"15min":1.45},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85999999999999999,"15min":0.86099999999999999},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1850000000000001,"5min":1.276,"15min":1.276},"last":0.871}]}]},{"osd":1,"up_from":15,"seq":64424510566,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59872,"kb_used_data":2584,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311968,"statfs":{"total":96636764160,"available":96575455232,"internally_reserved":0,"allocated":2646016,"data_stored":1393709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.33900000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63700000000000001,"5min":0.71999999999999997,"15min":0.89400000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.53300000000000003},"min":{"1min":0.32200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.439}]},{"osd":2,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.32600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.45000000000000001,"15min":0.442},"max":{"1min":0.82699999999999996,"5min":0.94199999999999995,"15min":0.94499999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.42099999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.65100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93400000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79100000000000004,"15min":0.81499999999999995},"min":{"1min":0.502,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0409999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.59499999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.78400000000000003,"15min":0.80500000000000005},"min":{"1min":0.49099999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97499999999999998,"5min":1.236,"15min":1.2609999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98399999999999999,"5min":1.353,"15min":1.353},"last":0.56399999999999995}]},{"osd":6,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.60999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.008,"5min":1.2270000000000001,"15min":1.2689999999999999},"last":0.621},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.84299999999999997,"15min":0.84999999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.093,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.64100000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.123,"5min":1.399,"15min":1.399},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.84299999999999997,"15min":0.85799999999999998},"min":{"1min":0.54100000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.105,"5min":1.365,"15min":1.365},"last":0.54100000000000004}]}]},{"osd":0,"up_from":14,"seq":60129543271,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60444,"kb_used_data":3156,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311396,"statfs":{"total":96636764160,"available":96574869504,"internally_reserved":0,"allocated":3231744,"data_stored":1982535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.438,"15min":0.45700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":1.216},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.42699999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64700000000000002,"5min":0.84999999999999998,"15min":1.1699999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72299999999999998,"5min":0.78900000000000003,"15min":0.96799999999999997},"last":0.39100000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.623,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.82699999999999996,"5min":0.871,"15min":1.032},"last":0.51200000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.63,"5min":0.51800000000000002,"15min":0.496},"max":{"1min":0.93100000000000005,"5min":1.1200000000000001,"15min":1.381},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.59199999999999997,"5min":0.56799999999999995,"15min":0.47799999999999998},"max":{"1min":1.02,"5min":1.137,"15min":1.2849999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80600000000000005,"15min":0.81899999999999995},"min":{"1min":0.57299999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.008,"5min":1.1930000000000001,"15min":1.3420000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.69599999999999995,"5min":0.59599999999999997,"15min":0.59499999999999997},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.288},"last":0.84799999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.51300000000000001},"max":{"1min":0.98899999999999999,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.83399999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0229999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.84999999999999998},"min":{"1min":0.65300000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.044,"5min":1.1899999999999999,"15min":1.415},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.83199999999999996,"15min":0.85599999999999998},"min":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0780000000000001,"5min":1.1379999999999999,"15min":1.395},"last":0.74099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:16:32.568 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:16:32.568 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:16:32.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.754+0000 7f5b9f96d640 1 Processor -- start 2023-12-19T18:16:32.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.754+0000 7f5b9f96d640 1 -- start start 2023-12-19T18:16:32.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.754+0000 7f5b9f96d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5b9806c740 0x7f5b9806f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.754+0000 7f5b9f96d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5b9806cb20 0x7f5b980ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.754+0000 7f5b9f96d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b9806f790 con 0x7f5b980cad40 2023-12-19T18:16:32.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.754+0000 7f5b9f96d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9815d5a0 con 0x7f5b9806c740 2023-12-19T18:16:32.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.754+0000 7f5b9f96d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9815d880 con 0x7f5b9806cb20 2023-12-19T18:16:32.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.754+0000 7f5b9d709640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5b9806c740 0x7f5b9806f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.754+0000 7f5b9df0a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5b980cad40 0x7f5b980c9080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47094/0 (socket says 172.21.15.26:47094) 2023-12-19T18:16:32.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.754+0000 7f5b9df0a640 1 -- 172.21.15.26:0/3131084561 learned_addr learned my addr 172.21.15.26:0/3131084561 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:32.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.758+0000 7f5b9cf08640 1 --2- 172.21.15.26:0/3131084561 >> v2:172.21.15.160:3300/0 conn(0x7f5b9806cb20 0x7f5b980ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.758+0000 7f5b8ffff640 1 -- 172.21.15.26:0/3131084561 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243776100 0 0) 0x7f5b9806f790 con 0x7f5b980cad40 2023-12-19T18:16:32.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.758+0000 7f5b9d709640 1 -- 172.21.15.26:0/3131084561 >> v2:172.21.15.160:3300/0 conn(0x7f5b9806cb20 msgr2=0x7f5b980ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:32.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.758+0000 7f5b9d709640 1 --2- 172.21.15.26:0/3131084561 >> v2:172.21.15.160:3300/0 conn(0x7f5b9806cb20 0x7f5b980ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.758+0000 7f5b9d709640 1 -- 172.21.15.26:0/3131084561 >> v1:172.21.15.26:6789/0 conn(0x7f5b980cad40 legacy=0x7f5b980c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.758+0000 7f5b9d709640 1 -- 172.21.15.26:0/3131084561 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9815db60 con 0x7f5b9806c740 2023-12-19T18:16:32.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.758+0000 7f5b9cf08640 1 --2- 172.21.15.26:0/3131084561 >> v2:172.21.15.160:3300/0 conn(0x7f5b9806cb20 0x7f5b980ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:32.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.758+0000 7f5b9d709640 1 --2- 172.21.15.26:0/3131084561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9806c740 0x7f5b9806f250 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f5b9806cf90 tx=0x7f5b9401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0f379e7bdd82e8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:32.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.758+0000 7f5b8ffff640 1 -- 172.21.15.26:0/3131084561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940a7070 con 0x7f5b9806c740 2023-12-19T18:16:32.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.758+0000 7f5b8ffff640 1 -- 172.21.15.26:0/3131084561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b940ab430 con 0x7f5b9806c740 2023-12-19T18:16:32.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.758+0000 7f5b8ffff640 1 -- 172.21.15.26:0/3131084561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940ab7c0 con 0x7f5b9806c740 2023-12-19T18:16:32.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.758+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3131084561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9806c740 msgr2=0x7f5b9806f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.758+0000 7f5b9f96d640 1 --2- 172.21.15.26:0/3131084561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9806c740 0x7f5b9806f250 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f5b9806cf90 tx=0x7f5b9401c430 comp rx=0 tx=0).stop 2023-12-19T18:16:32.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3131084561 shutdown_connections 2023-12-19T18:16:32.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 --2- 172.21.15.26:0/3131084561 >> v2:172.21.15.160:3300/0 conn(0x7f5b9806cb20 0x7f5b980ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 --2- 172.21.15.26:0/3131084561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9806c740 0x7f5b9806f250 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3131084561 >> 172.21.15.26:0/3131084561 conn(0x7f5b980660d0 msgr2=0x7f5b980ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:32.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3131084561 shutdown_connections 2023-12-19T18:16:32.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3131084561 wait complete. 2023-12-19T18:16:32.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 Processor -- start 2023-12-19T18:16:32.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 -- start start 2023-12-19T18:16:32.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5b9806c740 0x7f5b981508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9806cb20 0x7f5b981547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b980cad40 0x7f5b9814f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98150e30 con 0x7f5b9806cb20 2023-12-19T18:16:32.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98154d00 con 0x7f5b980cad40 2023-12-19T18:16:32.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9f96d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b981510b0 con 0x7f5b9806c740 2023-12-19T18:16:32.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9cf08640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9806cb20 0x7f5b981547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9cf08640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9806cb20 0x7f5b981547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40502/0 (socket says 172.21.15.26:40502) 2023-12-19T18:16:32.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9cf08640 1 -- 172.21.15.26:0/3841986066 learned_addr learned my addr 172.21.15.26:0/3841986066 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:32.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9d709640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5b9806c740 0x7f5b981508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9df0a640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b980cad40 0x7f5b9814f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9cf08640 1 -- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5b9806c740 msgr2=0x7f5b981508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9cf08640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5b9806c740 0x7f5b981508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9cf08640 1 -- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b980cad40 msgr2=0x7f5b9814f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9cf08640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b980cad40 0x7f5b9814f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9cf08640 1 -- 172.21.15.26:0/3841986066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94009420 con 0x7f5b9806cb20 2023-12-19T18:16:32.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.762+0000 7f5b9df0a640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b980cad40 0x7f5b9814f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:32.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.766+0000 7f5b9cf08640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9806cb20 0x7f5b981547c0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800e810 tx=0x7f5b8800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:32.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.766+0000 7f5b8dffb640 1 -- 172.21.15.26:0/3841986066 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8800c8d0 con 0x7f5b9806cb20 2023-12-19T18:16:32.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.766+0000 7f5b8dffb640 1 -- 172.21.15.26:0/3841986066 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8800ca30 con 0x7f5b9806cb20 2023-12-19T18:16:32.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.766+0000 7f5b8dffb640 1 -- 172.21.15.26:0/3841986066 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8800cdc0 con 0x7f5b9806cb20 2023-12-19T18:16:32.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.766+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3841986066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b98151390 con 0x7f5b9806cb20 2023-12-19T18:16:32.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.766+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3841986066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b98151950 con 0x7f5b9806cb20 2023-12-19T18:16:32.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.766+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3841986066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b98074270 con 0x7f5b9806cb20 2023-12-19T18:16:32.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.766+0000 7f5b8dffb640 1 -- 172.21.15.26:0/3841986066 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5b88013410 con 0x7f5b9806cb20 2023-12-19T18:16:32.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.766+0000 7f5b8dffb640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5b7c065b80 0x7f5b7c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.766+0000 7f5b8dffb640 1 -- 172.21.15.26:0/3841986066 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f5b88083500 con 0x7f5b9806cb20 2023-12-19T18:16:32.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.766+0000 7f5b9d709640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5b7c065b80 0x7f5b7c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.770+0000 7f5b9d709640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5b7c065b80 0x7f5b7c068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5b940af000 tx=0x7f5b940a7000 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:32.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.770+0000 7f5b8dffb640 1 -- 172.21.15.26:0/3841986066 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5b88050a50 con 0x7f5b9806cb20 2023-12-19T18:16:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.982+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3841986066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5b9815d310 con 0x7f5b9806cb20 2023-12-19T18:16:32.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.986+0000 7f5b8dffb640 1 -- 172.21.15.26:0/3841986066 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v52) v1 ==== 190+0+0 (secure 0 0 0) 0x7f5b88009480 con 0x7f5b9806cb20 2023-12-19T18:16:32.990 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2023-12-19T18:16:32.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.990+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5b7c065b80 msgr2=0x7f5b7c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.990+0000 7f5b9f96d640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5b7c065b80 0x7f5b7c068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5b940af000 tx=0x7f5b940a7000 comp rx=0 tx=0).stop 2023-12-19T18:16:32.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.990+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3841986066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9806cb20 msgr2=0x7f5b981547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.990+0000 7f5b9f96d640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9806cb20 0x7f5b981547c0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800e810 tx=0x7f5b8800ecd0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.990+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3841986066 shutdown_connections 2023-12-19T18:16:32.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.990+0000 7f5b9f96d640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5b7c065b80 0x7f5b7c068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.990+0000 7f5b9f96d640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b980cad40 0x7f5b9814f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.990+0000 7f5b9f96d640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9806cb20 0x7f5b981547c0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.990+0000 7f5b9f96d640 1 --2- 172.21.15.26:0/3841986066 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5b9806c740 0x7f5b981508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.990+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3841986066 >> 172.21.15.26:0/3841986066 conn(0x7f5b980660d0 msgr2=0x7f5b98158eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.990+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3841986066 shutdown_connections 2023-12-19T18:16:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:32.990+0000 7f5b9f96d640 1 -- 172.21.15.26:0/3841986066 wait complete. 2023-12-19T18:16:33.013 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.154+0000 7faf6a5bc640 1 Processor -- start 2023-12-19T18:16:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.154+0000 7faf6a5bc640 1 -- start start 2023-12-19T18:16:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.154+0000 7faf6a5bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf64154220 0x7faf6415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.154+0000 7faf6a5bc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf6406b3d0 0x7faf640ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.154+0000 7faf6a5bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf640cec40 0x7faf6415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.154+0000 7faf6a5bc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf640cf0d0 con 0x7faf64154220 2023-12-19T18:16:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.154+0000 7faf6a5bc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf6406bc30 con 0x7faf6406b3d0 2023-12-19T18:16:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.154+0000 7faf6a5bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf640cc6e0 con 0x7faf640cec40 2023-12-19T18:16:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf63fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf64154220 0x7faf6415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf63fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf64154220 0x7faf6415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40512/0 (socket says 172.21.15.26:40512) 2023-12-19T18:16:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf63fff640 1 -- 172.21.15.26:0/2980885998 learned_addr learned my addr 172.21.15.26:0/2980885998 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf68b59640 1 --2- 172.21.15.26:0/2980885998 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf640cec40 0x7faf6415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf637fe640 1 --2- 172.21.15.26:0/2980885998 >> v2:172.21.15.160:3300/0 conn(0x7faf6406b3d0 0x7faf640ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf63fff640 1 -- 172.21.15.26:0/2980885998 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf640cec40 msgr2=0x7faf6415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf63fff640 1 --2- 172.21.15.26:0/2980885998 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf640cec40 0x7faf6415d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf63fff640 1 -- 172.21.15.26:0/2980885998 >> v2:172.21.15.160:3300/0 conn(0x7faf6406b3d0 msgr2=0x7faf640ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf63fff640 1 --2- 172.21.15.26:0/2980885998 >> v2:172.21.15.160:3300/0 conn(0x7faf6406b3d0 0x7faf640ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf63fff640 1 -- 172.21.15.26:0/2980885998 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6415afc0 con 0x7faf64154220 2023-12-19T18:16:33.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf63fff640 1 --2- 172.21.15.26:0/2980885998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf64154220 0x7faf6415aa80 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7faf640cf880 tx=0x7faf5401c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c996ba3666df1686 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf62ffd640 1 -- 172.21.15.26:0/2980885998 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf540ab600 con 0x7faf64154220 2023-12-19T18:16:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf62ffd640 1 -- 172.21.15.26:0/2980885998 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf540a7050 con 0x7faf64154220 2023-12-19T18:16:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.158+0000 7faf62ffd640 1 -- 172.21.15.26:0/2980885998 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf540ab980 con 0x7faf64154220 2023-12-19T18:16:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 -- 172.21.15.26:0/2980885998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf64154220 msgr2=0x7faf6415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 --2- 172.21.15.26:0/2980885998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf64154220 0x7faf6415aa80 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7faf640cf880 tx=0x7faf5401c210 comp rx=0 tx=0).stop 2023-12-19T18:16:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 -- 172.21.15.26:0/2980885998 shutdown_connections 2023-12-19T18:16:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 --2- 172.21.15.26:0/2980885998 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf640cec40 0x7faf6415d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 --2- 172.21.15.26:0/2980885998 >> v2:172.21.15.160:3300/0 conn(0x7faf6406b3d0 0x7faf640ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 --2- 172.21.15.26:0/2980885998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf64154220 0x7faf6415aa80 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 -- 172.21.15.26:0/2980885998 >> 172.21.15.26:0/2980885998 conn(0x7faf640658e0 msgr2=0x7faf64065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 -- 172.21.15.26:0/2980885998 shutdown_connections 2023-12-19T18:16:33.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 -- 172.21.15.26:0/2980885998 wait complete. 2023-12-19T18:16:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 Processor -- start 2023-12-19T18:16:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 -- start start 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6406b3d0 0x7faf641509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf640cec40 0x7faf641f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf64154220 0x7faf6406f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf64150860 con 0x7faf6406b3d0 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf641502a0 con 0x7faf640cec40 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf6a5bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf64150580 con 0x7faf64154220 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf63fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6406b3d0 0x7faf641509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf63fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6406b3d0 0x7faf641509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40518/0 (socket says 172.21.15.26:40518) 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf63fff640 1 -- 172.21.15.26:0/3604845954 learned_addr learned my addr 172.21.15.26:0/3604845954 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf637fe640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf640cec40 0x7faf641f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.162+0000 7faf68b59640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf64154220 0x7faf6406f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf63fff640 1 -- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf64154220 msgr2=0x7faf6406f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf63fff640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf64154220 0x7faf6406f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf63fff640 1 -- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf640cec40 msgr2=0x7faf641f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf63fff640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf640cec40 0x7faf641f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf63fff640 1 -- 172.21.15.26:0/3604845954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf540094d0 con 0x7faf6406b3d0 2023-12-19T18:16:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf63fff640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6406b3d0 0x7faf641509f0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7faf540ae040 tx=0x7faf5401aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf60ff9640 1 -- 172.21.15.26:0/3604845954 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf540b6620 con 0x7faf6406b3d0 2023-12-19T18:16:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf60ff9640 1 -- 172.21.15.26:0/3604845954 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf540abe90 con 0x7faf6406b3d0 2023-12-19T18:16:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf60ff9640 1 -- 172.21.15.26:0/3604845954 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf540b69e0 con 0x7faf6406b3d0 2023-12-19T18:16:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf6a5bc640 1 -- 172.21.15.26:0/3604845954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf64070040 con 0x7faf6406b3d0 2023-12-19T18:16:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf6a5bc640 1 -- 172.21.15.26:0/3604845954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf64070840 con 0x7faf6406b3d0 2023-12-19T18:16:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf6a5bc640 1 -- 172.21.15.26:0/3604845954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf64074270 con 0x7faf6406b3d0 2023-12-19T18:16:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf60ff9640 1 -- 172.21.15.26:0/3604845954 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faf540b2040 con 0x7faf6406b3d0 2023-12-19T18:16:33.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf60ff9640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf50065b80 0x7faf50068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.166+0000 7faf60ff9640 1 -- 172.21.15.26:0/3604845954 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7faf5411d880 con 0x7faf6406b3d0 2023-12-19T18:16:33.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.170+0000 7faf637fe640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf50065b80 0x7faf50068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.170+0000 7faf637fe640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf50065b80 0x7faf50068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf64067dd0 tx=0x7faf58008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.170+0000 7faf60ff9640 1 -- 172.21.15.26:0/3604845954 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faf540ead30 con 0x7faf6406b3d0 2023-12-19T18:16:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf6a5bc640 1 -- 172.21.15.26:0/3604845954 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf640cdf60 con 0x7faf6406b3d0 2023-12-19T18:16:33.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf60ff9640 1 -- 172.21.15.26:0/3604845954 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4725 (secure 0 0 0) 0x7faf540ee350 con 0x7faf6406b3d0 2023-12-19T18:16:33.364 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:33.365 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3349262044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3349262044},{"type":"v1","addr":"172.21.15.160:6836","nonce":3349262044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3865658274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3865658274},{"type":"v1","addr":"172.21.15.26:6833","nonce":3865658274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10605,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1294752100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1294752100},{"type":"v1","addr":"172.21.15.160:6837","nonce":1294752100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:27.798614+0000","modified":"2023-12-19T18:16:31.076294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3981286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3981286992},{"type":"v1","addr":"172.21.15.26:6835","nonce":3981286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-12-19T18:16:33.365 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 96 2023-12-19T18:16:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf6a5bc640 1 -- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf50065b80 msgr2=0x7faf50068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf6a5bc640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf50065b80 0x7faf50068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faf64067dd0 tx=0x7faf58008040 comp rx=0 tx=0).stop 2023-12-19T18:16:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf6a5bc640 1 -- 172.21.15.26:0/3604845954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6406b3d0 msgr2=0x7faf641509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf6a5bc640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6406b3d0 0x7faf641509f0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7faf540ae040 tx=0x7faf5401aa50 comp rx=0 tx=0).stop 2023-12-19T18:16:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf6a5bc640 1 -- 172.21.15.26:0/3604845954 shutdown_connections 2023-12-19T18:16:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf6a5bc640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf50065b80 0x7faf50068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf6a5bc640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf64154220 0x7faf6406f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf6a5bc640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf640cec40 0x7faf641f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf6a5bc640 1 --2- 172.21.15.26:0/3604845954 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf6406b3d0 0x7faf641509f0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf6a5bc640 1 -- 172.21.15.26:0/3604845954 >> 172.21.15.26:0/3604845954 conn(0x7faf640658e0 msgr2=0x7faf64065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf6a5bc640 1 -- 172.21.15.26:0/3604845954 shutdown_connections 2023-12-19T18:16:33.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.362+0000 7faf6a5bc640 1 -- 172.21.15.26:0/3604845954 wait complete. 2023-12-19T18:16:33.374 INFO:tasks.ceph.mds.a.smithi026.stderr:2023-12-19T18:16:33.370+0000 7f7dc369e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:16:33.788 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:16:27.798614+0000', 'modified': '2023-12-19T18:16:31.076294+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10611}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10611': {'gid': 10611, 'name': 'a', 'rank': 0, 'incarnation': 92, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/3981286992', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 3981286992}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 3981286992}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:16:33.788 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:16:33.789 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-19T18:16:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a18db3640 1 Processor -- start 2023-12-19T18:16:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a18db3640 1 -- start start 2023-12-19T18:16:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a18db3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a14154220 0x7f6a1415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a18db3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a1406b3d0 0x7f6a140ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a18db3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1406bc30 con 0x7f6a140cec40 2023-12-19T18:16:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a18db3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a140cf0d0 con 0x7f6a14154220 2023-12-19T18:16:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a18db3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140cc6e0 con 0x7f6a1406b3d0 2023-12-19T18:16:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a1259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a14154220 0x7f6a1415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a1259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a14154220 0x7f6a1415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40534/0 (socket says 172.21.15.26:40534) 2023-12-19T18:16:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a1259c640 1 -- 172.21.15.26:0/1001188371 learned_addr learned my addr 172.21.15.26:0/1001188371 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a0bfff640 1 --2- 172.21.15.26:0/1001188371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a1406b3d0 0x7f6a140ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a1259c640 1 -- 172.21.15.26:0/1001188371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a1406b3d0 msgr2=0x7f6a140ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a1259c640 1 --2- 172.21.15.26:0/1001188371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a1406b3d0 0x7f6a140ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a1259c640 1 -- 172.21.15.26:0/1001188371 >> v1:172.21.15.160:6789/0 conn(0x7f6a140cec40 legacy=0x7f6a1415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.934+0000 7f6a1259c640 1 -- 172.21.15.26:0/1001188371 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1415afc0 con 0x7f6a14154220 2023-12-19T18:16:33.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.938+0000 7f6a1259c640 1 --2- 172.21.15.26:0/1001188371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a14154220 0x7f6a1415aa80 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400bb10 tx=0x7f6a0401ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f858231263706bb9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.938+0000 7f6a11d9b640 1 -- 172.21.15.26:0/1001188371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a040a7bd0 con 0x7f6a14154220 2023-12-19T18:16:33.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.938+0000 7f6a11d9b640 1 -- 172.21.15.26:0/1001188371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a040a7d30 con 0x7f6a14154220 2023-12-19T18:16:33.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.938+0000 7f6a11d9b640 1 -- 172.21.15.26:0/1001188371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a040a7ea0 con 0x7f6a14154220 2023-12-19T18:16:33.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.938+0000 7f6a18db3640 1 -- 172.21.15.26:0/1001188371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a14154220 msgr2=0x7f6a1415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.941 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.938+0000 7f6a18db3640 1 --2- 172.21.15.26:0/1001188371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a14154220 0x7f6a1415aa80 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400bb10 tx=0x7f6a0401ebe0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.938+0000 7f6a18db3640 1 -- 172.21.15.26:0/1001188371 shutdown_connections 2023-12-19T18:16:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.938+0000 7f6a18db3640 1 --2- 172.21.15.26:0/1001188371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a1406b3d0 0x7f6a140ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.938+0000 7f6a18db3640 1 --2- 172.21.15.26:0/1001188371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a14154220 0x7f6a1415aa80 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.938+0000 7f6a18db3640 1 -- 172.21.15.26:0/1001188371 >> 172.21.15.26:0/1001188371 conn(0x7f6a140658e0 msgr2=0x7f6a14065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.938+0000 7f6a18db3640 1 -- 172.21.15.26:0/1001188371 shutdown_connections 2023-12-19T18:16:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.938+0000 7f6a18db3640 1 -- 172.21.15.26:0/1001188371 wait complete. 2023-12-19T18:16:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a18db3640 1 Processor -- start 2023-12-19T18:16:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a18db3640 1 -- start start 2023-12-19T18:16:33.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a18db3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1406b3d0 0x7f6a14151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a18db3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a140cec40 0x7f6a140b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a18db3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a14154220 0x7f6a14143e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a18db3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14151000 con 0x7f6a1406b3d0 2023-12-19T18:16:33.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a18db3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a141512e0 con 0x7f6a14154220 2023-12-19T18:16:33.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a18db3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a141515c0 con 0x7f6a140cec40 2023-12-19T18:16:33.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a1259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1406b3d0 0x7f6a14151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a1259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1406b3d0 0x7f6a14151dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40536/0 (socket says 172.21.15.26:40536) 2023-12-19T18:16:33.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a1259c640 1 -- 172.21.15.26:0/196113028 learned_addr learned my addr 172.21.15.26:0/196113028 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a0bfff640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a140cec40 0x7f6a140b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a12d9d640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a14154220 0x7f6a14143e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a1259c640 1 -- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a140cec40 msgr2=0x7f6a140b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a1259c640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a140cec40 0x7f6a140b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a1259c640 1 -- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a14154220 msgr2=0x7f6a14143e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a1259c640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a14154220 0x7f6a14143e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a1259c640 1 -- 172.21.15.26:0/196113028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0400b770 con 0x7f6a1406b3d0 2023-12-19T18:16:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a1259c640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1406b3d0 0x7f6a14151dc0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400b2c0 tx=0x7f6a0401b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a0affd640 1 -- 172.21.15.26:0/196113028 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a040a7ea0 con 0x7f6a1406b3d0 2023-12-19T18:16:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a0affd640 1 -- 172.21.15.26:0/196113028 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a040afe30 con 0x7f6a1406b3d0 2023-12-19T18:16:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a0affd640 1 -- 172.21.15.26:0/196113028 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a040ba680 con 0x7f6a1406b3d0 2023-12-19T18:16:33.947 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a18db3640 1 -- 172.21.15.26:0/196113028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a141518a0 con 0x7f6a1406b3d0 2023-12-19T18:16:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.942+0000 7f6a18db3640 1 -- 172.21.15.26:0/196113028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a140b3660 con 0x7f6a1406b3d0 2023-12-19T18:16:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a0affd640 1 -- 172.21.15.26:0/196113028 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6a040b93e0 con 0x7f6a1406b3d0 2023-12-19T18:16:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a0affd640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f69f4065b80 0x7f69f4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.948 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a0affd640 1 -- 172.21.15.26:0/196113028 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f6a0411dc20 con 0x7f6a1406b3d0 2023-12-19T18:16:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a0bfff640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f69f4065b80 0x7f69f4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a1c124000 1 Processor -- start 2023-12-19T18:16:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a1c124000 1 -- start start 2023-12-19T18:16:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a0bfff640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f69f4065b80 0x7f69f4068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a140ad200 tx=0x7f69fc009190 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a1c124000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178c3e070 0x558178c3e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a1c124000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e990 0x558178bf3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a1c124000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 0x558178bf45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a1c124000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558178a17c90 con 0x558178bf41c0 2023-12-19T18:16:33.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a1c124000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558178a49220 con 0x558178c3e070 2023-12-19T18:16:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a1c124000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x558178a467f0 con 0x558178c3e990 2023-12-19T18:16:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 0x558178bf45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 0x558178bf45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40542/0 (socket says 172.21.15.26:40542) 2023-12-19T18:16:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a12d9d640 1 -- 172.21.15.26:0/1299938722 learned_addr learned my addr 172.21.15.26:0/1299938722 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a0bfff640 1 --2- 172.21.15.26:0/1299938722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e990 0x558178bf3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a1259c640 1 --2- 172.21.15.26:0/1299938722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178c3e070 0x558178c3e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a12d9d640 1 -- 172.21.15.26:0/1299938722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e990 msgr2=0x558178bf3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a12d9d640 1 --2- 172.21.15.26:0/1299938722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e990 0x558178bf3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a12d9d640 1 -- 172.21.15.26:0/1299938722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178c3e070 msgr2=0x558178c3e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a12d9d640 1 --2- 172.21.15.26:0/1299938722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178c3e070 0x558178c3e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a12d9d640 1 -- 172.21.15.26:0/1299938722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55817873e890 con 0x558178bf41c0 2023-12-19T18:16:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.946+0000 7f6a0bfff640 1 --2- 172.21.15.26:0/1299938722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e990 0x558178bf3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a12d9d640 1 --2- 172.21.15.26:0/1299938722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 0x558178bf45a0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c001ce0 tx=0x7f6a0c001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b0bb6208fd2088 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f69f3fff640 1 -- 172.21.15.26:0/1299938722 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c018070 con 0x558178bf41c0 2023-12-19T18:16:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f69f3fff640 1 -- 172.21.15.26:0/1299938722 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c003b90 con 0x558178bf41c0 2023-12-19T18:16:33.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f69f3fff640 1 -- 172.21.15.26:0/1299938722 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c003d00 con 0x558178bf41c0 2023-12-19T18:16:33.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 -- 172.21.15.26:0/1299938722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 msgr2=0x558178bf45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 --2- 172.21.15.26:0/1299938722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 0x558178bf45a0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c001ce0 tx=0x7f6a0c001610 comp rx=0 tx=0).stop 2023-12-19T18:16:33.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 -- 172.21.15.26:0/1299938722 shutdown_connections 2023-12-19T18:16:33.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 --2- 172.21.15.26:0/1299938722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 0x558178bf45a0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 --2- 172.21.15.26:0/1299938722 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e990 0x558178bf3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 --2- 172.21.15.26:0/1299938722 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178c3e070 0x558178c3e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 -- 172.21.15.26:0/1299938722 >> 172.21.15.26:0/1299938722 conn(0x558178bfa950 msgr2=0x558178bf2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 -- 172.21.15.26:0/1299938722 shutdown_connections 2023-12-19T18:16:33.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 -- 172.21.15.26:0/1299938722 wait complete. 2023-12-19T18:16:33.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 -- start start 2023-12-19T18:16:33.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 Processor -- start 2023-12-19T18:16:33.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 0x558178bf9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 0x558178c00080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178c3e990 0x558178c005c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558178a420e0 con 0x558178bf41c0 2023-12-19T18:16:33.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581789e1310 con 0x558178c3e990 2023-12-19T18:16:33.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581789ff770 con 0x558178c3e070 2023-12-19T18:16:33.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1c124000 10 client.0 _ll_register_callbacks cb 0x558178bdfb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:16:33.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 0x558178bf9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 0x558178bf9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40546/0 (socket says 172.21.15.26:40546) 2023-12-19T18:16:33.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a1259c640 1 -- 172.21.15.26:0/2871137957 learned_addr learned my addr 172.21.15.26:0/2871137957 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:33.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a0bfff640 1 --2- 172.21.15.26:0/2871137957 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 0x558178c00080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.950+0000 7f6a12d9d640 1 --2- 172.21.15.26:0/2871137957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178c3e990 0x558178c005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1259c640 1 -- 172.21.15.26:0/2871137957 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 msgr2=0x558178c00080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1259c640 1 --2- 172.21.15.26:0/2871137957 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 0x558178c00080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1259c640 1 -- 172.21.15.26:0/2871137957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178c3e990 msgr2=0x558178c005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1259c640 1 --2- 172.21.15.26:0/2871137957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178c3e990 0x558178c005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1259c640 1 -- 172.21.15.26:0/2871137957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1406bc30 con 0x558178bf41c0 2023-12-19T18:16:33.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1259c640 1 --2- 172.21.15.26:0/2871137957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 0x558178bf9740 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f6a0401c810 tx=0x7f6a040eeaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91f4c2007032c54 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f69f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T18:16:33.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f69f17fa640 1 -- 172.21.15.26:0/2871137957 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a040f6dd0 con 0x558178bf41c0 2023-12-19T18:16:33.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f69f17fa640 1 -- 172.21.15.26:0/2871137957 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f6a04106180 con 0x558178bf41c0 2023-12-19T18:16:33.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 -- 172.21.15.26:0/2871137957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558178a11c10 con 0x558178bf41c0 2023-12-19T18:16:33.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f69f17fa640 1 -- 172.21.15.26:0/2871137957 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a04106340 con 0x558178bf41c0 2023-12-19T18:16:33.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 10 client.10749 fetch_fsmap learned FSMap version 96 2023-12-19T18:16:33.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 -- 172.21.15.26:0/2871137957 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x558178a07260 con 0x558178bf41c0 2023-12-19T18:16:33.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f69f17fa640 1 -- 172.21.15.26:0/2871137957 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0410b020 con 0x558178bf41c0 2023-12-19T18:16:33.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f69f17fa640 1 -- 172.21.15.26:0/2871137957 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 96) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6a040ee390 con 0x558178bf41c0 2023-12-19T18:16:33.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 10 client.10749 fetch_fsmap finished waiting for FSMap version 96 2023-12-19T18:16:33.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 10 client.10749 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T18:16:33.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 20 client.10749 populate_metadata read hostname 'smithi026' 2023-12-19T18:16:33.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 --2- 172.21.15.26:0/2871137957 >> [v2:172.21.15.26:6834/3981286992,v1:172.21.15.26:6835/3981286992] conn(0x558178c458d0 0x558178c45cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 4 client.10749 mds_command: new command op to 10611 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:16:33.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 -- 172.21.15.26:0/2871137957 --> [v2:172.21.15.26:6834/3981286992,v1:172.21.15.26:6835/3981286992] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558178a40a30 con 0x558178c458d0 2023-12-19T18:16:33.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a0bfff640 1 --2- 172.21.15.26:0/2871137957 >> [v2:172.21.15.26:6834/3981286992,v1:172.21.15.26:6835/3981286992] conn(0x558178c458d0 0x558178c45cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a0bfff640 1 --2- 172.21.15.26:0/2871137957 >> [v2:172.21.15.26:6834/3981286992,v1:172.21.15.26:6835/3981286992] conn(0x558178c458d0 0x558178c45cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e9e902e7dbf720 server_cookie=dfd7da942942a286 in_seq=0 out_seq=0 2023-12-19T18:16:33.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f69f17fa640 10 client.10749 ms_handle_connect on v2:172.21.15.26:6834/3981286992 2023-12-19T18:16:33.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f69f17fa640 1 -- 172.21.15.26:0/2871137957 <== mds.0 v2:172.21.15.26:6834/3981286992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558178a40a30 con 0x558178c458d0 2023-12-19T18:16:33.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f69f17fa640 10 client.10749 handle_command_reply: tid=0 2023-12-19T18:16:33.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 client.10749 shutdown 2023-12-19T18:16:33.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 -- 172.21.15.26:0/2871137957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 msgr2=0x558178bf9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 --2- 172.21.15.26:0/2871137957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 0x558178bf9740 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f6a0401c810 tx=0x7f6a040eeaa0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 -- 172.21.15.26:0/2871137957 shutdown_connections 2023-12-19T18:16:33.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 --2- 172.21.15.26:0/2871137957 >> [v2:172.21.15.26:6834/3981286992,v1:172.21.15.26:6835/3981286992] conn(0x558178c458d0 0x558178c45cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 --2- 172.21.15.26:0/2871137957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178c3e990 0x558178c005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 --2- 172.21.15.26:0/2871137957 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 0x558178c00080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 --2- 172.21.15.26:0/2871137957 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178bf41c0 0x558178bf9740 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f69f17fa640 0 client.10749 ms_handle_reset on v2:172.21.15.26:6834/3981286992 2023-12-19T18:16:33.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 -- 172.21.15.26:0/2871137957 >> 172.21.15.26:0/2871137957 conn(0x558178bfa950 msgr2=0x558178bf2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 -- 172.21.15.26:0/2871137957 shutdown_connections 2023-12-19T18:16:33.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 1 -- 172.21.15.26:0/2871137957 wait complete. 2023-12-19T18:16:33.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.954+0000 7f6a1c124000 20 client.10749 trim_cache size 0 max 16384 2023-12-19T18:16:33.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a1c124000 1 Processor -- start 2023-12-19T18:16:33.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a1c124000 1 -- start start 2023-12-19T18:16:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a1c124000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178bf41c0 0x558178c960e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a1c124000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 0x558178c96620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a1c124000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 0x558178c96b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a1c124000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581786ff940 con 0x558178c3e990 2023-12-19T18:16:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a1c124000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558178a3f4e0 con 0x558178bf41c0 2023-12-19T18:16:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a1c124000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x558178a40f70 con 0x558178c3e070 2023-12-19T18:16:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 0x558178c96b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 0x558178c96b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40550/0 (socket says 172.21.15.26:40550) 2023-12-19T18:16:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a12d9d640 1 -- 172.21.15.26:0/3289392032 learned_addr learned my addr 172.21.15.26:0/3289392032 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a1259c640 1 --2- 172.21.15.26:0/3289392032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178bf41c0 0x558178c960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a0bfff640 1 --2- 172.21.15.26:0/3289392032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 0x558178c96620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a12d9d640 1 -- 172.21.15.26:0/3289392032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 msgr2=0x558178c96620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a12d9d640 1 --2- 172.21.15.26:0/3289392032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 0x558178c96620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a12d9d640 1 -- 172.21.15.26:0/3289392032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178bf41c0 msgr2=0x558178c960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a12d9d640 1 --2- 172.21.15.26:0/3289392032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178bf41c0 0x558178c960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.970+0000 7f6a12d9d640 1 -- 172.21.15.26:0/3289392032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581789ed3b0 con 0x558178c3e990 2023-12-19T18:16:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a12d9d640 1 --2- 172.21.15.26:0/3289392032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 0x558178c96b60 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c001650 tx=0x7f6a0c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49971ad8186b9c85 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a08ff9640 1 -- 172.21.15.26:0/3289392032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c018070 con 0x558178c3e990 2023-12-19T18:16:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a08ff9640 1 -- 172.21.15.26:0/3289392032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c019040 con 0x558178c3e990 2023-12-19T18:16:33.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a08ff9640 1 -- 172.21.15.26:0/3289392032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c006b60 con 0x558178c3e990 2023-12-19T18:16:33.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 -- 172.21.15.26:0/3289392032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 msgr2=0x558178c96b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 --2- 172.21.15.26:0/3289392032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 0x558178c96b60 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c001650 tx=0x7f6a0c00a000 comp rx=0 tx=0).stop 2023-12-19T18:16:33.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 -- 172.21.15.26:0/3289392032 shutdown_connections 2023-12-19T18:16:33.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 --2- 172.21.15.26:0/3289392032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 0x558178c96b60 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 --2- 172.21.15.26:0/3289392032 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 0x558178c96620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 --2- 172.21.15.26:0/3289392032 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178bf41c0 0x558178c960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 -- 172.21.15.26:0/3289392032 >> 172.21.15.26:0/3289392032 conn(0x558178bfa950 msgr2=0x558178bfff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 -- 172.21.15.26:0/3289392032 shutdown_connections 2023-12-19T18:16:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 -- 172.21.15.26:0/3289392032 wait complete. 2023-12-19T18:16:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 -- start start 2023-12-19T18:16:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 Processor -- start 2023-12-19T18:16:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178bf41c0 0x558178c7f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 0x558178c7f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 0x558178c7fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x558178a0fa80 con 0x558178c3e990 2023-12-19T18:16:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558178a371f0 con 0x558178bf41c0 2023-12-19T18:16:33.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581789e7570 con 0x558178c3e070 2023-12-19T18:16:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1c124000 10 client.0 _ll_register_callbacks cb 0x5581789f0640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:16:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 0x558178c7fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a12d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 0x558178c7fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40566/0 (socket says 172.21.15.26:40566) 2023-12-19T18:16:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a12d9d640 1 -- 172.21.15.26:0/1259843914 learned_addr learned my addr 172.21.15.26:0/1259843914 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a1259c640 1 --2- 172.21.15.26:0/1259843914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178bf41c0 0x558178c7f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a0bfff640 1 --2- 172.21.15.26:0/1259843914 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 0x558178c7f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a12d9d640 1 -- 172.21.15.26:0/1259843914 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 msgr2=0x558178c7f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a12d9d640 1 --2- 172.21.15.26:0/1259843914 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 0x558178c7f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a12d9d640 1 -- 172.21.15.26:0/1259843914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178bf41c0 msgr2=0x558178c7f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a12d9d640 1 --2- 172.21.15.26:0/1259843914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178bf41c0 0x558178c7f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.974+0000 7f6a12d9d640 1 -- 172.21.15.26:0/1259843914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55817873e890 con 0x558178c3e990 2023-12-19T18:16:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a12d9d640 1 --2- 172.21.15.26:0/1259843914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 0x558178c7fbd0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009010 tx=0x7f6a0c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74239d09cba94244 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f69f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T18:16:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f69f17fa640 1 -- 172.21.15.26:0/1259843914 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c018070 con 0x558178c3e990 2023-12-19T18:16:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f69f17fa640 1 -- 172.21.15.26:0/1259843914 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f6a0c00fce0 con 0x558178c3e990 2023-12-19T18:16:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 -- 172.21.15.26:0/1259843914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558178bfa3c0 con 0x558178c3e990 2023-12-19T18:16:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f69f17fa640 1 -- 172.21.15.26:0/1259843914 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a0c00fea0 con 0x558178c3e990 2023-12-19T18:16:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 10 client.10755 fetch_fsmap learned FSMap version 96 2023-12-19T18:16:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 -- 172.21.15.26:0/1259843914 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x55817873ecc0 con 0x558178c3e990 2023-12-19T18:16:33.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f69f17fa640 1 -- 172.21.15.26:0/1259843914 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558178bfa3c0 con 0x558178c3e990 2023-12-19T18:16:33.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f69f17fa640 1 -- 172.21.15.26:0/1259843914 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 96) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6a0c012070 con 0x558178c3e990 2023-12-19T18:16:33.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 10 client.10755 fetch_fsmap finished waiting for FSMap version 96 2023-12-19T18:16:33.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 10 client.10755 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T18:16:33.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 20 client.10755 populate_metadata read hostname 'smithi026' 2023-12-19T18:16:33.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 --2- 172.21.15.26:0/1259843914 >> [v2:172.21.15.26:6834/3981286992,v1:172.21.15.26:6835/3981286992] conn(0x558178c80110 0x558178ca3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 4 client.10755 mds_command: new command op to 10611 tid=0[{"prefix": "status"}] 2023-12-19T18:16:33.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 -- 172.21.15.26:0/1259843914 --> [v2:172.21.15.26:6834/3981286992,v1:172.21.15.26:6835/3981286992] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558178a07e00 con 0x558178c80110 2023-12-19T18:16:33.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1259c640 1 --2- 172.21.15.26:0/1259843914 >> [v2:172.21.15.26:6834/3981286992,v1:172.21.15.26:6835/3981286992] conn(0x558178c80110 0x558178ca3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1259c640 1 --2- 172.21.15.26:0/1259843914 >> [v2:172.21.15.26:6834/3981286992,v1:172.21.15.26:6835/3981286992] conn(0x558178c80110 0x558178ca3380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8aae6a3356fd4e9 server_cookie=41b506efa0e1d9ba in_seq=0 out_seq=0 2023-12-19T18:16:33.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f69f17fa640 10 client.10755 ms_handle_connect on v2:172.21.15.26:6834/3981286992 2023-12-19T18:16:33.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f69f17fa640 1 -- 172.21.15.26:0/1259843914 <== mds.0 v2:172.21.15.26:6834/3981286992 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x558178a07e00 con 0x558178c80110 2023-12-19T18:16:33.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f69f17fa640 10 client.10755 handle_command_reply: tid=0 2023-12-19T18:16:33.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 client.10755 shutdown 2023-12-19T18:16:33.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 -- 172.21.15.26:0/1259843914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 msgr2=0x558178c7fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 --2- 172.21.15.26:0/1259843914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 0x558178c7fbd0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009010 tx=0x7f6a0c001260 comp rx=0 tx=0).stop 2023-12-19T18:16:33.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 -- 172.21.15.26:0/1259843914 shutdown_connections 2023-12-19T18:16:33.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 --2- 172.21.15.26:0/1259843914 >> [v2:172.21.15.26:6834/3981286992,v1:172.21.15.26:6835/3981286992] conn(0x558178c80110 0x558178ca3380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 --2- 172.21.15.26:0/1259843914 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x558178c3e990 0x558178c7fbd0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 --2- 172.21.15.26:0/1259843914 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x558178c3e070 0x558178c7f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 --2- 172.21.15.26:0/1259843914 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558178bf41c0 0x558178c7f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f69f17fa640 0 client.10755 ms_handle_reset on v2:172.21.15.26:6834/3981286992 2023-12-19T18:16:33.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 -- 172.21.15.26:0/1259843914 >> 172.21.15.26:0/1259843914 conn(0x558178bfa950 msgr2=0x558178c3b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 -- 172.21.15.26:0/1259843914 shutdown_connections 2023-12-19T18:16:33.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 1 -- 172.21.15.26:0/1259843914 wait complete. 2023-12-19T18:16:33.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a1c124000 20 client.10755 trim_cache size 0 max 16384 2023-12-19T18:16:33.987 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:16:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T18:16:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T18:16:33.987 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 10611, 2023-12-19T18:16:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T18:16:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T18:16:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T18:16:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 6.1811621069999996, 2023-12-19T18:16:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 96, 2023-12-19T18:16:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 212, 2023-12-19T18:16:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 212, 2023-12-19T18:16:33.988 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 10.960182353 2023-12-19T18:16:33.988 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:16:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a18db3640 1 -- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f69f4065b80 msgr2=0x7f69f4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a18db3640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f69f4065b80 0x7f69f4068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a140ad200 tx=0x7f69fc009190 comp rx=0 tx=0).stop 2023-12-19T18:16:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a18db3640 1 -- 172.21.15.26:0/196113028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1406b3d0 msgr2=0x7f6a14151dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a18db3640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1406b3d0 0x7f6a14151dc0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400b2c0 tx=0x7f6a0401b980 comp rx=0 tx=0).stop 2023-12-19T18:16:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a18db3640 1 -- 172.21.15.26:0/196113028 shutdown_connections 2023-12-19T18:16:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a18db3640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f69f4065b80 0x7f69f4068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a18db3640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a14154220 0x7f6a14143e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a18db3640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6a140cec40 0x7f6a140b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a18db3640 1 --2- 172.21.15.26:0/196113028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a1406b3d0 0x7f6a14151dc0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a18db3640 1 -- 172.21.15.26:0/196113028 >> 172.21.15.26:0/196113028 conn(0x7f6a140658e0 msgr2=0x7f6a140adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a18db3640 1 -- 172.21.15.26:0/196113028 shutdown_connections 2023-12-19T18:16:33.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:33.978+0000 7f6a18db3640 1 -- 172.21.15.26:0/196113028 wait complete. 2023-12-19T18:16:34.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:16:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f5119b51640 1 Processor -- start 2023-12-19T18:16:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f5119b51640 1 -- start start 2023-12-19T18:16:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f5119b51640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51140b2030 0x7f5114071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f5119b51640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511415e7c0 con 0x7f51140b3960 2023-12-19T18:16:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f5119b51640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51140b04c0 con 0x7f51140affe0 2023-12-19T18:16:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f5119b51640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51140c97a0 con 0x7f51140b2030 2023-12-19T18:16:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f5118b4f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51140b2030 0x7f5114071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f5118b4f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51140b2030 0x7f5114071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40574/0 (socket says 172.21.15.26:40574) 2023-12-19T18:16:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f5118b4f640 1 -- 172.21.15.26:0/550141543 learned_addr learned my addr 172.21.15.26:0/550141543 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f5119350640 1 --1- 172.21.15.26:0/550141543 >> v1:172.21.15.26:6789/0 conn(0x7f51140b3960 0x7f51140b3e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47106/0 (socket says 172.21.15.26:47106) 2023-12-19T18:16:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f5118b4f640 1 -- 172.21.15.26:0/550141543 >> v1:172.21.15.160:6789/0 conn(0x7f51140affe0 legacy=0x7f51140c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f5118b4f640 1 -- 172.21.15.26:0/550141543 >> v1:172.21.15.26:6789/0 conn(0x7f51140b3960 legacy=0x7f51140b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f5118b4f640 1 -- 172.21.15.26:0/550141543 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51140b07a0 con 0x7f51140b2030 2023-12-19T18:16:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.146+0000 7f51137fe640 1 -- 172.21.15.26:0/550141543 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984284280 0 0) 0x7f511415e7c0 con 0x7f51140b3960 2023-12-19T18:16:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.150+0000 7f5118b4f640 1 --2- 172.21.15.26:0/550141543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51140b2030 0x7f5114071e20 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f5108009870 tx=0x7f510801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa94484e4ff983a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.150+0000 7f51137fe640 1 -- 172.21.15.26:0/550141543 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51080a7070 con 0x7f51140b2030 2023-12-19T18:16:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.150+0000 7f51137fe640 1 -- 172.21.15.26:0/550141543 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51080ab430 con 0x7f51140b2030 2023-12-19T18:16:34.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.150+0000 7f51137fe640 1 -- 172.21.15.26:0/550141543 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51080ab7c0 con 0x7f51140b2030 2023-12-19T18:16:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.150+0000 7f5119b51640 1 -- 172.21.15.26:0/550141543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51140b2030 msgr2=0x7f5114071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.150+0000 7f5119b51640 1 --2- 172.21.15.26:0/550141543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51140b2030 0x7f5114071e20 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f5108009870 tx=0x7f510801c540 comp rx=0 tx=0).stop 2023-12-19T18:16:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.150+0000 7f5119b51640 1 -- 172.21.15.26:0/550141543 shutdown_connections 2023-12-19T18:16:34.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.150+0000 7f5119b51640 1 --2- 172.21.15.26:0/550141543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51140b2030 0x7f5114071e20 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.150+0000 7f5119b51640 1 -- 172.21.15.26:0/550141543 >> 172.21.15.26:0/550141543 conn(0x7f511406b0f0 msgr2=0x7f511406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.150+0000 7f5119b51640 1 -- 172.21.15.26:0/550141543 shutdown_connections 2023-12-19T18:16:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.150+0000 7f5119b51640 1 -- 172.21.15.26:0/550141543 wait complete. 2023-12-19T18:16:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.150+0000 7f5119b51640 1 Processor -- start 2023-12-19T18:16:34.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5119b51640 1 -- start start 2023-12-19T18:16:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5119b51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51140affe0 0x7f51140c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5119b51640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51140b2030 0x7f5114159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5119b51640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51140b3960 0x7f51140adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5119b51640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51140c4930 con 0x7f51140affe0 2023-12-19T18:16:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5119b51640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51141520b0 con 0x7f51140b3960 2023-12-19T18:16:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5119b51640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5114152350 con 0x7f51140b2030 2023-12-19T18:16:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5118b4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51140affe0 0x7f51140c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5118b4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51140affe0 0x7f51140c43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40580/0 (socket says 172.21.15.26:40580) 2023-12-19T18:16:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5118b4f640 1 -- 172.21.15.26:0/2305025450 learned_addr learned my addr 172.21.15.26:0/2305025450 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5113fff640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51140b2030 0x7f5114159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:34.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5119350640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51140b3960 0x7f51140adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5118b4f640 1 -- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51140b2030 msgr2=0x7f5114159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5118b4f640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51140b2030 0x7f5114159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5118b4f640 1 -- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51140b3960 msgr2=0x7f51140adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5118b4f640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51140b3960 0x7f51140adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5118b4f640 1 -- 172.21.15.26:0/2305025450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080094d0 con 0x7f51140affe0 2023-12-19T18:16:34.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5118b4f640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51140affe0 0x7f51140c43f0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f51080ad040 tx=0x7f510801ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f51117fa640 1 -- 172.21.15.26:0/2305025450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51080b4800 con 0x7f51140affe0 2023-12-19T18:16:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f51117fa640 1 -- 172.21.15.26:0/2305025450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51080b4980 con 0x7f51140affe0 2023-12-19T18:16:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5119b51640 1 -- 172.21.15.26:0/2305025450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5114152630 con 0x7f51140affe0 2023-12-19T18:16:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f51117fa640 1 -- 172.21.15.26:0/2305025450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51080b4d30 con 0x7f51140affe0 2023-12-19T18:16:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5119b51640 1 -- 172.21.15.26:0/2305025450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5114152bf0 con 0x7f51140affe0 2023-12-19T18:16:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.154+0000 7f5119b51640 1 -- 172.21.15.26:0/2305025450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5114067400 con 0x7f51140affe0 2023-12-19T18:16:34.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.158+0000 7f51117fa640 1 -- 172.21.15.26:0/2305025450 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f51080be3f0 con 0x7f51140affe0 2023-12-19T18:16:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.158+0000 7f51117fa640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f50f8065b80 0x7f50f8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.158+0000 7f51117fa640 1 -- 172.21.15.26:0/2305025450 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f510811d580 con 0x7f51140affe0 2023-12-19T18:16:34.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.158+0000 7f5113fff640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f50f8065b80 0x7f50f8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:34.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.158+0000 7f5113fff640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f50f8065b80 0x7f50f8068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f51140aeca0 tx=0x7f5100009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:34.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.162+0000 7f51117fa640 1 -- 172.21.15.26:0/2305025450 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f51080ea980 con 0x7f51140affe0 2023-12-19T18:16:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f5119b51640 1 -- 172.21.15.26:0/2305025450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f51140b2a80 con 0x7f51140affe0 2023-12-19T18:16:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f51117fa640 1 -- 172.21.15.26:0/2305025450 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) v1 ==== 53+0+73 (secure 0 0 0) 0x7f51080edfa0 con 0x7f51140affe0 2023-12-19T18:16:34.356 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:16:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f5119b51640 1 -- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f50f8065b80 msgr2=0x7f50f8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f5119b51640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f50f8065b80 0x7f50f8068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f51140aeca0 tx=0x7f5100009040 comp rx=0 tx=0).stop 2023-12-19T18:16:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f5119b51640 1 -- 172.21.15.26:0/2305025450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51140affe0 msgr2=0x7f51140c43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f5119b51640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51140affe0 0x7f51140c43f0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f51080ad040 tx=0x7f510801ad80 comp rx=0 tx=0).stop 2023-12-19T18:16:34.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f5119b51640 1 -- 172.21.15.26:0/2305025450 shutdown_connections 2023-12-19T18:16:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f5119b51640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f50f8065b80 0x7f50f8068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f5119b51640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51140b3960 0x7f51140adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f5119b51640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51140b2030 0x7f5114159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f5119b51640 1 --2- 172.21.15.26:0/2305025450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51140affe0 0x7f51140c43f0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f5119b51640 1 -- 172.21.15.26:0/2305025450 >> 172.21.15.26:0/2305025450 conn(0x7f511406b0f0 msgr2=0x7f51140aee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f5119b51640 1 -- 172.21.15.26:0/2305025450 shutdown_connections 2023-12-19T18:16:34.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:34.354+0000 7f5119b51640 1 -- 172.21.15.26:0/2305025450 wait complete. 2023-12-19T18:16:34.377 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-19T18:16:34.377 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:16:34.377 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T18:16:34.378 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:16:34.378 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:16:34.378 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-19T18:16:34.378 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-19T18:16:34.378 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:16:34.378 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:16:34.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:34.378 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T18:16:34.388 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:16:34.388 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:16:34.389 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:16:34.389 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:34.389 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T18:16:34.389 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:34.389 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T18:16:34.389 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:34.389 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T18:16:34.389 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T18:16:34.389 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:34.389 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T18:16:34.390 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:16:34.390 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:34.390 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T18:16:34.390 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T18:16:34.390 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 36746sec preferred_lft 36746sec 2023-12-19T18:16:34.390 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T18:16:34.390 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:34.390 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T18:16:34.390 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:34.390 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:34.391 INFO:teuthology.orchestra.run.smithi026.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:16:34.392 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:16:34.392 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-19T18:16:34.392 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:34.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:34.392 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:16:34.450 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T18:16:34.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:34.453 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T18:16:34.505 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T18:16:34.505 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:16:34.505 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T18:16:34.505 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T18:16:34.505 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T18:16:34.505 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:16:34.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:34.506 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T18:16:34.506 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T18:16:34.506 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:16:34.506 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:16:34.506 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:16:34.506 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T18:16:34.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:34.599 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T18:16:34.609 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:16:34.609 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:16:34.610 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:16:34.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:34.610 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T18:16:34.676 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:16:34.676 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:16:34.727 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:16:34.727 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:16:34.727 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:16:34.779 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T18:16:34.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:34.779 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T18:16:34.875 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T18:16:34.876 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T18:16:34.891 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T18:16:34.891 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T18:16:34.891 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T18:16:34.891 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T18:16:34.891 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T18:16:34.897 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T18:16:34.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:34.898 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:16:34.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:34.919 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T18:16:34.979 INFO:teuthology.orchestra.run.smithi026.stdout:client10141 2023-12-19T18:16:34.980 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:16:34.980 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:35.032 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-12-19T18:16:35.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.222+0000 7f8789d7b640 1 Processor -- start 2023-12-19T18:16:35.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.222+0000 7f8789d7b640 1 -- start start 2023-12-19T18:16:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f8789d7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8784070ff0 0x7f878406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f8789d7b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f87841597d0 0x7f87840c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f8789d7b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f878406c4c0 con 0x7f87840c92d0 2023-12-19T18:16:35.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f8789d7b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87840c9780 con 0x7f87841597d0 2023-12-19T18:16:35.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f8789d7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87840713d0 con 0x7f8784070ff0 2023-12-19T18:16:35.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f87837fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8784070ff0 0x7f878406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f8782ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f87841597d0 0x7f87840c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f87837fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8784070ff0 0x7f878406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:48004/0 (socket says 172.21.15.26:48004) 2023-12-19T18:16:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f87837fe640 1 -- 172.21.15.26:0/4117004611 learned_addr learned my addr 172.21.15.26:0/4117004611 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f8783fff640 1 --1- 172.21.15.26:0/4117004611 >> v1:172.21.15.160:6789/0 conn(0x7f87840c92d0 0x7f878415cce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:33742/0 (socket says 172.21.15.26:33742) 2023-12-19T18:16:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f87837fe640 1 -- 172.21.15.26:0/4117004611 >> v2:172.21.15.160:3300/0 conn(0x7f87841597d0 msgr2=0x7f87840c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f87837fe640 1 --2- 172.21.15.26:0/4117004611 >> v2:172.21.15.160:3300/0 conn(0x7f87841597d0 0x7f87840c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f87837fe640 1 -- 172.21.15.26:0/4117004611 >> v1:172.21.15.160:6789/0 conn(0x7f87840c92d0 legacy=0x7f878415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:35.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.226+0000 7f87837fe640 1 -- 172.21.15.26:0/4117004611 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8784159530 con 0x7f8784070ff0 2023-12-19T18:16:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f87837fe640 1 --2- 172.21.15.26:0/4117004611 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8784070ff0 0x7f878406ab40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f877400aa10 tx=0x7f877401d8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=645b819dd5c8bf36 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f87827fc640 1 -- 172.21.15.26:0/4117004611 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87740a6910 con 0x7f8784070ff0 2023-12-19T18:16:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f87827fc640 1 -- 172.21.15.26:0/4117004611 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87740a6a70 con 0x7f8784070ff0 2023-12-19T18:16:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f87827fc640 1 -- 172.21.15.26:0/4117004611 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87740a6e00 con 0x7f8784070ff0 2023-12-19T18:16:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f8789d7b640 1 -- 172.21.15.26:0/4117004611 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8784070ff0 msgr2=0x7f878406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f8789d7b640 1 --2- 172.21.15.26:0/4117004611 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8784070ff0 0x7f878406ab40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f877400aa10 tx=0x7f877401d8e0 comp rx=0 tx=0).stop 2023-12-19T18:16:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f8789d7b640 1 -- 172.21.15.26:0/4117004611 shutdown_connections 2023-12-19T18:16:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f8789d7b640 1 --2- 172.21.15.26:0/4117004611 >> v2:172.21.15.160:3300/0 conn(0x7f87841597d0 0x7f87840c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f8789d7b640 1 --2- 172.21.15.26:0/4117004611 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8784070ff0 0x7f878406ab40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f8789d7b640 1 -- 172.21.15.26:0/4117004611 >> 172.21.15.26:0/4117004611 conn(0x7f87840b0270 msgr2=0x7f87840b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f8789d7b640 1 -- 172.21.15.26:0/4117004611 shutdown_connections 2023-12-19T18:16:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f8789d7b640 1 -- 172.21.15.26:0/4117004611 wait complete. 2023-12-19T18:16:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f8789d7b640 1 Processor -- start 2023-12-19T18:16:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.230+0000 7f8789d7b640 1 -- start start 2023-12-19T18:16:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8789d7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8784070ff0 0x7f87840ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8789d7b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87840c92d0 0x7f878415c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8789d7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87841597d0 0x7f878414ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8789d7b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87840ada50 con 0x7f87841597d0 2023-12-19T18:16:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8789d7b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87840adee0 con 0x7f87840c92d0 2023-12-19T18:16:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8789d7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878415ca90 con 0x7f8784070ff0 2023-12-19T18:16:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8783fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87841597d0 0x7f878414ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8783fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87841597d0 0x7f878414ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40586/0 (socket says 172.21.15.26:40586) 2023-12-19T18:16:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8783fff640 1 -- 172.21.15.26:0/513275662 learned_addr learned my addr 172.21.15.26:0/513275662 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8782ffd640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87840c92d0 0x7f878415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:35.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f87837fe640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8784070ff0 0x7f87840ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8783fff640 1 -- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8784070ff0 msgr2=0x7f87840ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:35.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8783fff640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8784070ff0 0x7f87840ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8783fff640 1 -- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87840c92d0 msgr2=0x7f878415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8783fff640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87840c92d0 0x7f878415c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8783fff640 1 -- 172.21.15.26:0/513275662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877400a670 con 0x7f87841597d0 2023-12-19T18:16:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8783fff640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87841597d0 0x7f878414ecf0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f877c00c7d0 tx=0x7f877c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f876bfff640 1 -- 172.21.15.26:0/513275662 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c00a640 con 0x7f87841597d0 2023-12-19T18:16:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f876bfff640 1 -- 172.21.15.26:0/513275662 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877c00a7a0 con 0x7f87841597d0 2023-12-19T18:16:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8789d7b640 1 -- 172.21.15.26:0/513275662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878414f230 con 0x7f87841597d0 2023-12-19T18:16:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f876bfff640 1 -- 172.21.15.26:0/513275662 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c00ab10 con 0x7f87841597d0 2023-12-19T18:16:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8789d7b640 1 -- 172.21.15.26:0/513275662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87840ad120 con 0x7f87841597d0 2023-12-19T18:16:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.234+0000 7f8789d7b640 1 -- 172.21.15.26:0/513275662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87840adbc0 con 0x7f87841597d0 2023-12-19T18:16:35.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.238+0000 7f876bfff640 1 -- 172.21.15.26:0/513275662 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f877c00ac70 con 0x7f87841597d0 2023-12-19T18:16:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.238+0000 7f876bfff640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8770065b80 0x7f8770068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:35.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.238+0000 7f876bfff640 1 -- 172.21.15.26:0/513275662 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f877c082e10 con 0x7f87841597d0 2023-12-19T18:16:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.238+0000 7f87837fe640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8770065b80 0x7f8770068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:35.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.238+0000 7f87837fe640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8770065b80 0x7f8770068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f87740b0000 tx=0x7f8774008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:35.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.242+0000 7f876bfff640 1 -- 172.21.15.26:0/513275662 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f877c086030 con 0x7f87841597d0 2023-12-19T18:16:35.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:35.438+0000 7f8789d7b640 1 -- 172.21.15.26:0/513275662 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8784074270 con 0x7f87841597d0 2023-12-19T18:16:36.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.106+0000 7f876bfff640 1 -- 172.21.15.26:0/513275662 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v97) v1 ==== 136+0+0 (secure 0 0 0) 0x7f877c0503d0 con 0x7f87841597d0 2023-12-19T18:16:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.110+0000 7f8789d7b640 1 -- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8770065b80 msgr2=0x7f8770068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.110+0000 7f8789d7b640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8770065b80 0x7f8770068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f87740b0000 tx=0x7f8774008040 comp rx=0 tx=0).stop 2023-12-19T18:16:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.110+0000 7f8789d7b640 1 -- 172.21.15.26:0/513275662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87841597d0 msgr2=0x7f878414ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.110+0000 7f8789d7b640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87841597d0 0x7f878414ecf0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f877c00c7d0 tx=0x7f877c00cc90 comp rx=0 tx=0).stop 2023-12-19T18:16:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.110+0000 7f8789d7b640 1 -- 172.21.15.26:0/513275662 shutdown_connections 2023-12-19T18:16:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.110+0000 7f8789d7b640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8770065b80 0x7f8770068040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.110+0000 7f8789d7b640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87841597d0 0x7f878414ecf0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.110+0000 7f8789d7b640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87840c92d0 0x7f878415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.110+0000 7f8789d7b640 1 --2- 172.21.15.26:0/513275662 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8784070ff0 0x7f87840ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.110+0000 7f8789d7b640 1 -- 172.21.15.26:0/513275662 >> 172.21.15.26:0/513275662 conn(0x7f87840b0270 msgr2=0x7f878406f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.110+0000 7f8789d7b640 1 -- 172.21.15.26:0/513275662 shutdown_connections 2023-12-19T18:16:36.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.110+0000 7f8789d7b640 1 -- 172.21.15.26:0/513275662 wait complete. 2023-12-19T18:16:36.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.278+0000 7f20074fc640 1 Processor -- start 2023-12-19T18:16:36.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.278+0000 7f20074fc640 1 -- start start 2023-12-19T18:16:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f20074fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2000154220 0x7f200015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f20074fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20000cee10 0x7f200015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f20074fc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200006bc30 con 0x7f200006b3d0 2023-12-19T18:16:36.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f20074fc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20000cc6e0 con 0x7f20000cee10 2023-12-19T18:16:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f20074fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200015afc0 con 0x7f2000154220 2023-12-19T18:16:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f2005a99640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20000cee10 0x7f200015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f2005a99640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f20000cee10 0x7f200015b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39282/0 (socket says 172.21.15.26:39282) 2023-12-19T18:16:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f2005a99640 1 -- 172.21.15.26:0/1657554452 learned_addr learned my addr 172.21.15.26:0/1657554452 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f2005298640 1 --2- 172.21.15.26:0/1657554452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2000154220 0x7f200015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f2005a99640 1 -- 172.21.15.26:0/1657554452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2000154220 msgr2=0x7f200015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f2005a99640 1 --2- 172.21.15.26:0/1657554452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2000154220 0x7f200015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f2005a99640 1 -- 172.21.15.26:0/1657554452 >> v1:172.21.15.26:6789/0 conn(0x7f200006b3d0 legacy=0x7f20000ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f2005a99640 1 -- 172.21.15.26:0/1657554452 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200006b950 con 0x7f20000cee10 2023-12-19T18:16:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f1ff7fff640 1 -- 172.21.15.26:0/1657554452 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126580418 0 0) 0x7f200006bc30 con 0x7f200006b3d0 2023-12-19T18:16:36.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.282+0000 7f2005298640 1 --2- 172.21.15.26:0/1657554452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2000154220 0x7f200015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:36.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f2005a99640 1 --2- 172.21.15.26:0/1657554452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20000cee10 0x7f200015b1c0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009e50 tx=0x7f1ff801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8988726993ff679d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f1ff7fff640 1 -- 172.21.15.26:0/1657554452 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff80a58a0 con 0x7f20000cee10 2023-12-19T18:16:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f1ff7fff640 1 -- 172.21.15.26:0/1657554452 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff80a5a00 con 0x7f20000cee10 2023-12-19T18:16:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f1ff7fff640 1 -- 172.21.15.26:0/1657554452 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff80a5d90 con 0x7f20000cee10 2023-12-19T18:16:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 -- 172.21.15.26:0/1657554452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20000cee10 msgr2=0x7f200015b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 --2- 172.21.15.26:0/1657554452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20000cee10 0x7f200015b1c0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009e50 tx=0x7f1ff801c9e0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 -- 172.21.15.26:0/1657554452 shutdown_connections 2023-12-19T18:16:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 --2- 172.21.15.26:0/1657554452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20000cee10 0x7f200015b1c0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 --2- 172.21.15.26:0/1657554452 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2000154220 0x7f200015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 -- 172.21.15.26:0/1657554452 >> 172.21.15.26:0/1657554452 conn(0x7f20000658e0 msgr2=0x7f2000065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:36.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 -- 172.21.15.26:0/1657554452 shutdown_connections 2023-12-19T18:16:36.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 -- 172.21.15.26:0/1657554452 wait complete. 2023-12-19T18:16:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 Processor -- start 2023-12-19T18:16:36.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 -- start start 2023-12-19T18:16:36.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200006b3d0 0x7f200014ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20000cee10 0x7f20001f8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2000154220 0x7f20001f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200015ca10 con 0x7f200006b3d0 2023-12-19T18:16:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200015b510 con 0x7f20000cee10 2023-12-19T18:16:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.286+0000 7f20074fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20001f9680 con 0x7f2000154220 2023-12-19T18:16:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f2005298640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200006b3d0 0x7f200014ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f2005298640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200006b3d0 0x7f200014ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39288/0 (socket says 172.21.15.26:39288) 2023-12-19T18:16:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f2005298640 1 -- 172.21.15.26:0/532707148 learned_addr learned my addr 172.21.15.26:0/532707148 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f2005298640 1 -- 172.21.15.26:0/532707148 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2000154220 msgr2=0x7f20001f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f2005298640 1 --2- 172.21.15.26:0/532707148 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2000154220 0x7f20001f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f2005298640 1 -- 172.21.15.26:0/532707148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20000cee10 msgr2=0x7f20001f8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f2005298640 1 --2- 172.21.15.26:0/532707148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20000cee10 0x7f20001f8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f2005298640 1 -- 172.21.15.26:0/532707148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff8009b30 con 0x7f200006b3d0 2023-12-19T18:16:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f2005298640 1 --2- 172.21.15.26:0/532707148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200006b3d0 0x7f200014ffb0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00d830 tx=0x7f1ffc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f1ff5ffb640 1 -- 172.21.15.26:0/532707148 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0096c0 con 0x7f200006b3d0 2023-12-19T18:16:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f1ff5ffb640 1 -- 172.21.15.26:0/532707148 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ffc009820 con 0x7f200006b3d0 2023-12-19T18:16:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f1ff5ffb640 1 -- 172.21.15.26:0/532707148 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc009bd0 con 0x7f200006b3d0 2023-12-19T18:16:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f20074fc640 1 -- 172.21.15.26:0/532707148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20001f9960 con 0x7f200006b3d0 2023-12-19T18:16:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f20074fc640 1 -- 172.21.15.26:0/532707148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200015c630 con 0x7f200006b3d0 2023-12-19T18:16:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f20074fc640 1 -- 172.21.15.26:0/532707148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2000074270 con 0x7f200006b3d0 2023-12-19T18:16:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.290+0000 7f1ff5ffb640 1 -- 172.21.15.26:0/532707148 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1ffc014410 con 0x7f200006b3d0 2023-12-19T18:16:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.294+0000 7f1ff5ffb640 1 --2- 172.21.15.26:0/532707148 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1fe4065b80 0x7f1fe4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.294+0000 7f1ff5ffb640 1 -- 172.21.15.26:0/532707148 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f1ffc083f80 con 0x7f200006b3d0 2023-12-19T18:16:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.294+0000 7f2004a97640 1 --2- 172.21.15.26:0/532707148 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1fe4065b80 0x7f1fe4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.294+0000 7f2004a97640 1 --2- 172.21.15.26:0/532707148 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1fe4065b80 0x7f1fe4068040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0007fd0 tx=0x7f1ff0007920 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.298+0000 7f1ff5ffb640 1 -- 172.21.15.26:0/532707148 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1ffc051520 con 0x7f200006b3d0 2023-12-19T18:16:36.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.454+0000 7f20074fc640 1 -- 172.21.15.26:0/532707148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20000715c0 con 0x7f200006b3d0 2023-12-19T18:16:36.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.458+0000 7f1ff5ffb640 1 -- 172.21.15.26:0/532707148 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v212) v1 ==== 74+0+17261 (secure 0 0 0) 0x7f1ffc054b40 con 0x7f200006b3d0 2023-12-19T18:16:36.459 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:36.460 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":212,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:28.801422+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:25.863942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:26.798049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:36.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.458+0000 7f20074fc640 1 -- 172.21.15.26:0/532707148 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1fe4065b80 msgr2=0x7f1fe4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.458+0000 7f20074fc640 1 --2- 172.21.15.26:0/532707148 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1fe4065b80 0x7f1fe4068040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0007fd0 tx=0x7f1ff0007920 comp rx=0 tx=0).stop 2023-12-19T18:16:36.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.458+0000 7f20074fc640 1 -- 172.21.15.26:0/532707148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200006b3d0 msgr2=0x7f200014ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.458+0000 7f20074fc640 1 --2- 172.21.15.26:0/532707148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200006b3d0 0x7f200014ffb0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc00d830 tx=0x7f1ffc00dcf0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.458+0000 7f20074fc640 1 -- 172.21.15.26:0/532707148 shutdown_connections 2023-12-19T18:16:36.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.458+0000 7f20074fc640 1 --2- 172.21.15.26:0/532707148 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1fe4065b80 0x7f1fe4068040 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.458+0000 7f20074fc640 1 --2- 172.21.15.26:0/532707148 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2000154220 0x7f20001f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.458+0000 7f20074fc640 1 --2- 172.21.15.26:0/532707148 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20000cee10 0x7f20001f8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.458+0000 7f20074fc640 1 --2- 172.21.15.26:0/532707148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f200006b3d0 0x7f200014ffb0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.458+0000 7f20074fc640 1 -- 172.21.15.26:0/532707148 >> 172.21.15.26:0/532707148 conn(0x7f20000658e0 msgr2=0x7f20000ac850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:36.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.458+0000 7f20074fc640 1 -- 172.21.15.26:0/532707148 shutdown_connections 2023-12-19T18:16:36.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.458+0000 7f20074fc640 1 -- 172.21.15.26:0/532707148 wait complete. 2023-12-19T18:16:36.481 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:16:25.863942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '211', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:16:26.798049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:16:36.481 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:16:36.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d64ff0640 1 Processor -- start 2023-12-19T18:16:36.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d64ff0640 1 -- start start 2023-12-19T18:16:36.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d64ff0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5d6015d0d0 0x7f5d600b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d64ff0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d600b4600 0x7f5d600b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d64ff0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d6015e200 con 0x7f5d6015ccf0 2023-12-19T18:16:36.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d64ff0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d600b4f20 con 0x7f5d600b4600 2023-12-19T18:16:36.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d64ff0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d600d0aa0 con 0x7f5d6015d0d0 2023-12-19T18:16:36.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d5e59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5d6015ccf0 0x7f5d600d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41350/0 (socket says 172.21.15.26:41350) 2023-12-19T18:16:36.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d5e59c640 1 -- 172.21.15.26:0/3806492794 learned_addr learned my addr 172.21.15.26:0/3806492794 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:36.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d5ed9d640 1 --2- 172.21.15.26:0/3806492794 >> v2:172.21.15.26:3300/0 conn(0x7f5d600b4600 0x7f5d600b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d57fff640 1 --2- 172.21.15.26:0/3806492794 >> v2:172.21.15.160:3300/0 conn(0x7f5d6015d0d0 0x7f5d600b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d5dd9b640 1 -- 172.21.15.26:0/3806492794 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511877259 0 0) 0x7f5d6015e200 con 0x7f5d6015ccf0 2023-12-19T18:16:36.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d5dd9b640 1 -- 172.21.15.26:0/3806492794 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d4c003530 con 0x7f5d6015ccf0 2023-12-19T18:16:36.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d5ed9d640 1 -- 172.21.15.26:0/3806492794 >> v2:172.21.15.160:3300/0 conn(0x7f5d6015d0d0 msgr2=0x7f5d600b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:36.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d5ed9d640 1 --2- 172.21.15.26:0/3806492794 >> v2:172.21.15.160:3300/0 conn(0x7f5d6015d0d0 0x7f5d600b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d5ed9d640 1 -- 172.21.15.26:0/3806492794 >> v1:172.21.15.26:6789/0 conn(0x7f5d6015ccf0 legacy=0x7f5d600d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.626+0000 7f5d5ed9d640 1 -- 172.21.15.26:0/3806492794 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6015d680 con 0x7f5d600b4600 2023-12-19T18:16:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.630+0000 7f5d5ed9d640 1 --2- 172.21.15.26:0/3806492794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d600b4600 0x7f5d600b49e0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f5d600d0400 tx=0x7f5d5801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94545099059ffaeb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.630+0000 7f5d5dd9b640 1 -- 172.21.15.26:0/3806492794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d580a59a0 con 0x7f5d600b4600 2023-12-19T18:16:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.630+0000 7f5d5dd9b640 1 -- 172.21.15.26:0/3806492794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d580a5b00 con 0x7f5d600b4600 2023-12-19T18:16:36.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.630+0000 7f5d5dd9b640 1 -- 172.21.15.26:0/3806492794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d580a5e90 con 0x7f5d600b4600 2023-12-19T18:16:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.630+0000 7f5d64ff0640 1 -- 172.21.15.26:0/3806492794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d600b4600 msgr2=0x7f5d600b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.630+0000 7f5d64ff0640 1 --2- 172.21.15.26:0/3806492794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d600b4600 0x7f5d600b49e0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f5d600d0400 tx=0x7f5d5801c8e0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.630+0000 7f5d64ff0640 1 -- 172.21.15.26:0/3806492794 shutdown_connections 2023-12-19T18:16:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.630+0000 7f5d64ff0640 1 --2- 172.21.15.26:0/3806492794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d600b4600 0x7f5d600b49e0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.630+0000 7f5d64ff0640 1 --2- 172.21.15.26:0/3806492794 >> v2:172.21.15.160:3300/0 conn(0x7f5d6015d0d0 0x7f5d600b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.630+0000 7f5d64ff0640 1 -- 172.21.15.26:0/3806492794 >> 172.21.15.26:0/3806492794 conn(0x7f5d60065ee0 msgr2=0x7f5d600cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.630+0000 7f5d64ff0640 1 -- 172.21.15.26:0/3806492794 shutdown_connections 2023-12-19T18:16:36.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.630+0000 7f5d64ff0640 1 -- 172.21.15.26:0/3806492794 wait complete. 2023-12-19T18:16:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d64ff0640 1 Processor -- start 2023-12-19T18:16:36.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d64ff0640 1 -- start start 2023-12-19T18:16:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d64ff0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d600b4600 0x7f5d600b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d64ff0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d6015ccf0 0x7f5d6014ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d64ff0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d6015d0d0 0x7f5d600cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d64ff0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600b7ae0 con 0x7f5d6015ccf0 2023-12-19T18:16:36.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d64ff0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600b7800 con 0x7f5d600b4600 2023-12-19T18:16:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d64ff0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6014bf60 con 0x7f5d6015d0d0 2023-12-19T18:16:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d57fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d6015ccf0 0x7f5d6014ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d57fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d6015ccf0 0x7f5d6014ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39306/0 (socket says 172.21.15.26:39306) 2023-12-19T18:16:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d57fff640 1 -- 172.21.15.26:0/1142958667 learned_addr learned my addr 172.21.15.26:0/1142958667 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d5e59c640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d600b4600 0x7f5d600b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d5ed9d640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d6015d0d0 0x7f5d600cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d57fff640 1 -- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d6015d0d0 msgr2=0x7f5d600cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d57fff640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d6015d0d0 0x7f5d600cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d57fff640 1 -- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d600b4600 msgr2=0x7f5d600b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d57fff640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d600b4600 0x7f5d600b6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d57fff640 1 -- 172.21.15.26:0/1142958667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d58009a70 con 0x7f5d6015ccf0 2023-12-19T18:16:36.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d57fff640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d6015ccf0 0x7f5d6014ca40 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800e870 tx=0x7f5d4800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d56ffd640 1 -- 172.21.15.26:0/1142958667 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48009510 con 0x7f5d6015ccf0 2023-12-19T18:16:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d56ffd640 1 -- 172.21.15.26:0/1142958667 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4800eec0 con 0x7f5d6015ccf0 2023-12-19T18:16:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d56ffd640 1 -- 172.21.15.26:0/1142958667 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4800cb40 con 0x7f5d6015ccf0 2023-12-19T18:16:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d64ff0640 1 -- 172.21.15.26:0/1142958667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6014c240 con 0x7f5d6015ccf0 2023-12-19T18:16:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.634+0000 7f5d64ff0640 1 -- 172.21.15.26:0/1142958667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d600b7460 con 0x7f5d6015ccf0 2023-12-19T18:16:36.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.638+0000 7f5d64ff0640 1 -- 172.21.15.26:0/1142958667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d60074270 con 0x7f5d6015ccf0 2023-12-19T18:16:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.638+0000 7f5d56ffd640 1 -- 172.21.15.26:0/1142958667 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5d4800ccc0 con 0x7f5d6015ccf0 2023-12-19T18:16:36.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.638+0000 7f5d56ffd640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5d40065b80 0x7f5d40068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.638+0000 7f5d56ffd640 1 -- 172.21.15.26:0/1142958667 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f5d48083640 con 0x7f5d6015ccf0 2023-12-19T18:16:36.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.638+0000 7f5d5e59c640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5d40065b80 0x7f5d40068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.638+0000 7f5d5e59c640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5d40065b80 0x7f5d40068040 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d500011f0 tx=0x7f5d50009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:36.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.642+0000 7f5d56ffd640 1 -- 172.21.15.26:0/1142958667 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5d48050c80 con 0x7f5d6015ccf0 2023-12-19T18:16:36.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.798+0000 7f5d64ff0640 1 -- 172.21.15.26:0/1142958667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d600ce450 con 0x7f5d6015ccf0 2023-12-19T18:16:36.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.798+0000 7f5d56ffd640 1 -- 172.21.15.26:0/1142958667 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v212) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d480542a0 con 0x7f5d6015ccf0 2023-12-19T18:16:36.802 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:16:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.802+0000 7f5d64ff0640 1 -- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5d40065b80 msgr2=0x7f5d40068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.802+0000 7f5d64ff0640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5d40065b80 0x7f5d40068040 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5d500011f0 tx=0x7f5d50009040 comp rx=0 tx=0).stop 2023-12-19T18:16:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.802+0000 7f5d64ff0640 1 -- 172.21.15.26:0/1142958667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d6015ccf0 msgr2=0x7f5d6014ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.802+0000 7f5d64ff0640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d6015ccf0 0x7f5d6014ca40 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800e870 tx=0x7f5d4800ec70 comp rx=0 tx=0).stop 2023-12-19T18:16:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.802+0000 7f5d64ff0640 1 -- 172.21.15.26:0/1142958667 shutdown_connections 2023-12-19T18:16:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.802+0000 7f5d64ff0640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5d40065b80 0x7f5d40068040 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.802+0000 7f5d64ff0640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5d6015d0d0 0x7f5d600cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.802+0000 7f5d64ff0640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d6015ccf0 0x7f5d6014ca40 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.802+0000 7f5d64ff0640 1 --2- 172.21.15.26:0/1142958667 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5d600b4600 0x7f5d600b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.802+0000 7f5d64ff0640 1 -- 172.21.15.26:0/1142958667 >> 172.21.15.26:0/1142958667 conn(0x7f5d60065ee0 msgr2=0x7f5d6015e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:36.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.802+0000 7f5d64ff0640 1 -- 172.21.15.26:0/1142958667 shutdown_connections 2023-12-19T18:16:36.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.802+0000 7f5d64ff0640 1 -- 172.21.15.26:0/1142958667 wait complete. 2023-12-19T18:16:36.823 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:16:36.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.966+0000 7fbbb5525640 1 Processor -- start 2023-12-19T18:16:36.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.966+0000 7fbbb5525640 1 -- start start 2023-12-19T18:16:36.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbbb5525640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbbb015cdf0 0x7fbbb006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbbb5525640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbb006f270 0x7fbbb00cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbbb5525640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb006f6d0 con 0x7fbbb00cfdf0 2023-12-19T18:16:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbbb5525640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb015d1d0 con 0x7fbbb006f270 2023-12-19T18:16:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbbb5525640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb0161140 con 0x7fbbb015cdf0 2023-12-19T18:16:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbba7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbb006f270 0x7fbbb00cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbba7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbbb006f270 0x7fbbb00cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39322/0 (socket says 172.21.15.26:39322) 2023-12-19T18:16:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbba7fff640 1 -- 172.21.15.26:0/3745586747 learned_addr learned my addr 172.21.15.26:0/3745586747 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbba7fff640 1 -- 172.21.15.26:0/3745586747 >> v2:172.21.15.160:3300/0 conn(0x7fbbb015cdf0 msgr2=0x7fbbb006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbba7fff640 1 --2- 172.21.15.26:0/3745586747 >> v2:172.21.15.160:3300/0 conn(0x7fbbb015cdf0 0x7fbbb006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbba7fff640 1 -- 172.21.15.26:0/3745586747 >> v1:172.21.15.160:6789/0 conn(0x7fbbb00cfdf0 legacy=0x7fbbb0071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:36.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbba7fff640 1 -- 172.21.15.26:0/3745586747 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb00cbe90 con 0x7fbbb006f270 2023-12-19T18:16:36.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbba7fff640 1 --2- 172.21.15.26:0/3745586747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb006f270 0x7fbbb00cf8b0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00d17e0 tx=0x7fbb9801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4ecc0cf532f9ae server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbbae7fc640 1 -- 172.21.15.26:0/3745586747 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb980ac410 con 0x7fbbb006f270 2023-12-19T18:16:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbbae7fc640 1 -- 172.21.15.26:0/3745586747 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb980ac570 con 0x7fbbb006f270 2023-12-19T18:16:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.970+0000 7fbbae7fc640 1 -- 172.21.15.26:0/3745586747 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb980ac900 con 0x7fbbb006f270 2023-12-19T18:16:36.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 -- 172.21.15.26:0/3745586747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb006f270 msgr2=0x7fbbb00cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 --2- 172.21.15.26:0/3745586747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb006f270 0x7fbbb00cf8b0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00d17e0 tx=0x7fbb9801c490 comp rx=0 tx=0).stop 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 -- 172.21.15.26:0/3745586747 shutdown_connections 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 --2- 172.21.15.26:0/3745586747 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb006f270 0x7fbbb00cf8b0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 --2- 172.21.15.26:0/3745586747 >> v2:172.21.15.160:3300/0 conn(0x7fbbb015cdf0 0x7fbbb006ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 -- 172.21.15.26:0/3745586747 >> 172.21.15.26:0/3745586747 conn(0x7fbbb00d0830 msgr2=0x7fbbb00d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 -- 172.21.15.26:0/3745586747 shutdown_connections 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 -- 172.21.15.26:0/3745586747 wait complete. 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 Processor -- start 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 -- start start 2023-12-19T18:16:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbb006f270 0x7fbbb0157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbb00cfdf0 0x7fbbb015f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb015cdf0 0x7fbbb015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00aebd0 con 0x7fbbb015cdf0 2023-12-19T18:16:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01582c0 con 0x7fbbb00cfdf0 2023-12-19T18:16:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbb5525640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00af480 con 0x7fbbb006f270 2023-12-19T18:16:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbaf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb015cdf0 0x7fbbb015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbaf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb015cdf0 0x7fbbb015dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39336/0 (socket says 172.21.15.26:39336) 2023-12-19T18:16:36.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbaf7fe640 1 -- 172.21.15.26:0/997984746 learned_addr learned my addr 172.21.15.26:0/997984746 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:36.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbba7fff640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbb00cfdf0 0x7fbbb015f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbaeffd640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbb006f270 0x7fbbb0157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbaf7fe640 1 -- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbb006f270 msgr2=0x7fbbb0157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbaf7fe640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbb006f270 0x7fbbb0157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbaf7fe640 1 -- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbb00cfdf0 msgr2=0x7fbbb015f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:36.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbaf7fe640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbb00cfdf0 0x7fbbb015f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:36.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbaf7fe640 1 -- 172.21.15.26:0/997984746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb980094d0 con 0x7fbbb015cdf0 2023-12-19T18:16:36.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbba7fff640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbb00cfdf0 0x7fbbb015f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:36.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.974+0000 7fbbaeffd640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbb006f270 0x7fbbb0157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:36.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.978+0000 7fbbaf7fe640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb015cdf0 0x7fbbb015dcf0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fbba800c780 tx=0x7fbba800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:36.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.978+0000 7fbba77fe640 1 -- 172.21.15.26:0/997984746 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8003d90 con 0x7fbbb015cdf0 2023-12-19T18:16:36.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.978+0000 7fbba77fe640 1 -- 172.21.15.26:0/997984746 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba800ce30 con 0x7fbbb015cdf0 2023-12-19T18:16:36.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.978+0000 7fbba77fe640 1 -- 172.21.15.26:0/997984746 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8005290 con 0x7fbbb015cdf0 2023-12-19T18:16:36.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.978+0000 7fbbb5525640 1 -- 172.21.15.26:0/997984746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb00d0280 con 0x7fbbb015cdf0 2023-12-19T18:16:36.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.978+0000 7fbbb5525640 1 -- 172.21.15.26:0/997984746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb015eb90 con 0x7fbbb015cdf0 2023-12-19T18:16:36.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.978+0000 7fbbb5525640 1 -- 172.21.15.26:0/997984746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb004e850 con 0x7fbbb015cdf0 2023-12-19T18:16:36.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.978+0000 7fbba77fe640 1 -- 172.21.15.26:0/997984746 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbba80053f0 con 0x7fbbb015cdf0 2023-12-19T18:16:36.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.978+0000 7fbba77fe640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbb90065b80 0x7fbb90068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:36.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.978+0000 7fbba77fe640 1 -- 172.21.15.26:0/997984746 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fbba8082a50 con 0x7fbbb015cdf0 2023-12-19T18:16:36.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.978+0000 7fbbaeffd640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbb90065b80 0x7fbb90068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:36.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.982+0000 7fbbaeffd640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbb90065b80 0x7fbb90068040 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbba000a870 tx=0x7fbba0008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:36.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:36.982+0000 7fbba77fe640 1 -- 172.21.15.26:0/997984746 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbba804fff0 con 0x7fbbb015cdf0 2023-12-19T18:16:37.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.138+0000 7fbbb5525640 1 -- 172.21.15.26:0/997984746 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbbb006e650 con 0x7fbbb015cdf0 2023-12-19T18:16:37.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.142+0000 7fbba77fe640 1 -- 172.21.15.26:0/997984746 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v212) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbba8017020 con 0x7fbbb015cdf0 2023-12-19T18:16:37.144 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:16:37.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.142+0000 7fbbb5525640 1 -- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbb90065b80 msgr2=0x7fbb90068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.142+0000 7fbbb5525640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbb90065b80 0x7fbb90068040 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbba000a870 tx=0x7fbba0008040 comp rx=0 tx=0).stop 2023-12-19T18:16:37.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.142+0000 7fbbb5525640 1 -- 172.21.15.26:0/997984746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb015cdf0 msgr2=0x7fbbb015dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.142+0000 7fbbb5525640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb015cdf0 0x7fbbb015dcf0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fbba800c780 tx=0x7fbba800cc40 comp rx=0 tx=0).stop 2023-12-19T18:16:37.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.142+0000 7fbbb5525640 1 -- 172.21.15.26:0/997984746 shutdown_connections 2023-12-19T18:16:37.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.142+0000 7fbbb5525640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbb90065b80 0x7fbb90068040 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.142+0000 7fbbb5525640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbbb015cdf0 0x7fbbb015dcf0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.142+0000 7fbbb5525640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbbb00cfdf0 0x7fbbb015f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.142+0000 7fbbb5525640 1 --2- 172.21.15.26:0/997984746 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbbb006f270 0x7fbbb0157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.142+0000 7fbbb5525640 1 -- 172.21.15.26:0/997984746 >> 172.21.15.26:0/997984746 conn(0x7fbbb00d0830 msgr2=0x7fbbb00b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.142+0000 7fbbb5525640 1 -- 172.21.15.26:0/997984746 shutdown_connections 2023-12-19T18:16:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.142+0000 7fbbb5525640 1 -- 172.21.15.26:0/997984746 wait complete. 2023-12-19T18:16:37.166 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:16:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6e95a640 1 Processor -- start 2023-12-19T18:16:37.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6e95a640 1 -- start start 2023-12-19T18:16:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6e95a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3a6815d070 0x7f3a680cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6e95a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a68160a20 0x7f3a680cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6e95a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a680cdb20 con 0x7f3a6815cc90 2023-12-19T18:16:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6e95a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a680cfe40 con 0x7f3a6815d070 2023-12-19T18:16:37.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6e95a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680cde00 con 0x7f3a68160a20 2023-12-19T18:16:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6d958640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3a6815cc90 0x7f3a680cd2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41360/0 (socket says 172.21.15.26:41360) 2023-12-19T18:16:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6d958640 1 -- 172.21.15.26:0/1571236449 learned_addr learned my addr 172.21.15.26:0/1571236449 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6e159640 1 --2- 172.21.15.26:0/1571236449 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a68160a20 0x7f3a680cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6d157640 1 --2- 172.21.15.26:0/1571236449 >> v2:172.21.15.160:3300/0 conn(0x7f3a6815d070 0x7f3a680cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6c956640 1 -- 172.21.15.26:0/1571236449 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818176455 0 0) 0x7f3a680cdb20 con 0x7f3a6815cc90 2023-12-19T18:16:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6c956640 1 -- 172.21.15.26:0/1571236449 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a54003530 con 0x7f3a6815cc90 2023-12-19T18:16:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6c956640 1 -- 172.21.15.26:0/1571236449 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 474344421 0 0) 0x7f3a54003530 con 0x7f3a6815cc90 2023-12-19T18:16:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6e159640 1 -- 172.21.15.26:0/1571236449 >> v2:172.21.15.160:3300/0 conn(0x7f3a6815d070 msgr2=0x7f3a680cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6e159640 1 --2- 172.21.15.26:0/1571236449 >> v2:172.21.15.160:3300/0 conn(0x7f3a6815d070 0x7f3a680cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6e159640 1 -- 172.21.15.26:0/1571236449 >> v1:172.21.15.26:6789/0 conn(0x7f3a6815cc90 legacy=0x7f3a680cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.310+0000 7f3a6e159640 1 -- 172.21.15.26:0/1571236449 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a680ce0e0 con 0x7f3a68160a20 2023-12-19T18:16:37.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.314+0000 7f3a6e159640 1 --2- 172.21.15.26:0/1571236449 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a68160a20 0x7f3a680cf900 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009d90 tx=0x7f3a6001c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c39214ac48d0d39 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:37.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.314+0000 7f3a6c956640 1 -- 172.21.15.26:0/1571236449 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64001680 con 0x7f3a68160a20 2023-12-19T18:16:37.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.314+0000 7f3a6c956640 1 -- 172.21.15.26:0/1571236449 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a600a5640 con 0x7f3a68160a20 2023-12-19T18:16:37.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.314+0000 7f3a6c956640 1 -- 172.21.15.26:0/1571236449 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a600a59d0 con 0x7f3a68160a20 2023-12-19T18:16:37.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.314+0000 7f3a6e95a640 1 -- 172.21.15.26:0/1571236449 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a68160a20 msgr2=0x7f3a680cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.314+0000 7f3a6e95a640 1 --2- 172.21.15.26:0/1571236449 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a68160a20 0x7f3a680cf900 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009d90 tx=0x7f3a6001c840 comp rx=0 tx=0).stop 2023-12-19T18:16:37.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.314+0000 7f3a6e95a640 1 -- 172.21.15.26:0/1571236449 shutdown_connections 2023-12-19T18:16:37.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.314+0000 7f3a6e95a640 1 --2- 172.21.15.26:0/1571236449 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a68160a20 0x7f3a680cf900 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.314+0000 7f3a6e95a640 1 --2- 172.21.15.26:0/1571236449 >> v2:172.21.15.160:3300/0 conn(0x7f3a6815d070 0x7f3a680cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.314+0000 7f3a6e95a640 1 -- 172.21.15.26:0/1571236449 >> 172.21.15.26:0/1571236449 conn(0x7f3a6806a320 msgr2=0x7f3a680658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:37.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.314+0000 7f3a6e95a640 1 -- 172.21.15.26:0/1571236449 shutdown_connections 2023-12-19T18:16:37.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.314+0000 7f3a6e95a640 1 -- 172.21.15.26:0/1571236449 wait complete. 2023-12-19T18:16:37.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6e95a640 1 Processor -- start 2023-12-19T18:16:37.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6e95a640 1 -- start start 2023-12-19T18:16:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6e95a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a6815cc90 0x7f3a68156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6e95a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6815d070 0x7f3a680cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6e95a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a68160a20 0x7f3a680cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6e95a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680caac0 con 0x7f3a6815d070 2023-12-19T18:16:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6e95a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680b54c0 con 0x7f3a6815cc90 2023-12-19T18:16:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6e95a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a68156740 con 0x7f3a68160a20 2023-12-19T18:16:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6d157640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6815d070 0x7f3a680cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6d157640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6815d070 0x7f3a680cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39350/0 (socket says 172.21.15.26:39350) 2023-12-19T18:16:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6d157640 1 -- 172.21.15.26:0/3822551493 learned_addr learned my addr 172.21.15.26:0/3822551493 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6d958640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a6815cc90 0x7f3a68156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6e159640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a68160a20 0x7f3a680cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6d157640 1 -- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a68160a20 msgr2=0x7f3a680cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6d157640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a68160a20 0x7f3a680cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6d157640 1 -- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a6815cc90 msgr2=0x7f3a68156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6d157640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a6815cc90 0x7f3a68156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6d157640 1 -- 172.21.15.26:0/3822551493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a60009a70 con 0x7f3a6815d070 2023-12-19T18:16:37.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6d157640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6815d070 0x7f3a680cb2c0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800e930 tx=0x7f3a5800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:37.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a5e7fc640 1 -- 172.21.15.26:0/3822551493 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58009690 con 0x7f3a6815d070 2023-12-19T18:16:37.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a5e7fc640 1 -- 172.21.15.26:0/3822551493 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5800ca00 con 0x7f3a6815d070 2023-12-19T18:16:37.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6e95a640 1 -- 172.21.15.26:0/3822551493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a680c9b60 con 0x7f3a6815d070 2023-12-19T18:16:37.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a5e7fc640 1 -- 172.21.15.26:0/3822551493 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5800cd70 con 0x7f3a6815d070 2023-12-19T18:16:37.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.318+0000 7f3a6e95a640 1 -- 172.21.15.26:0/3822551493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a680ca120 con 0x7f3a6815d070 2023-12-19T18:16:37.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.322+0000 7f3a6e95a640 1 -- 172.21.15.26:0/3822551493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a68061930 con 0x7f3a6815d070 2023-12-19T18:16:37.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.322+0000 7f3a5e7fc640 1 -- 172.21.15.26:0/3822551493 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3a58013470 con 0x7f3a6815d070 2023-12-19T18:16:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.322+0000 7f3a5e7fc640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3a4c065b80 0x7f3a4c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.322+0000 7f3a5e7fc640 1 -- 172.21.15.26:0/3822551493 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f3a58083860 con 0x7f3a6815d070 2023-12-19T18:16:37.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.322+0000 7f3a6d958640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3a4c065b80 0x7f3a4c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.322+0000 7f3a6d958640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3a4c065b80 0x7f3a4c068040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3a64005890 tx=0x7f3a64001660 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:37.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.326+0000 7f3a5e7fc640 1 -- 172.21.15.26:0/3822551493 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3a58050ef0 con 0x7f3a6815d070 2023-12-19T18:16:37.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a6e95a640 1 -- 172.21.15.26:0/3822551493 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3a68160fc0 con 0x7f3a6815d070 2023-12-19T18:16:37.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a5e7fc640 1 -- 172.21.15.26:0/3822551493 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v212) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a58054510 con 0x7f3a6815d070 2023-12-19T18:16:37.488 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:16:37.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a6e95a640 1 -- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3a4c065b80 msgr2=0x7f3a4c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a6e95a640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3a4c065b80 0x7f3a4c068040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3a64005890 tx=0x7f3a64001660 comp rx=0 tx=0).stop 2023-12-19T18:16:37.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a6e95a640 1 -- 172.21.15.26:0/3822551493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6815d070 msgr2=0x7f3a680cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a6e95a640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6815d070 0x7f3a680cb2c0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800e930 tx=0x7f3a5800edf0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a6e95a640 1 -- 172.21.15.26:0/3822551493 shutdown_connections 2023-12-19T18:16:37.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a6e95a640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3a4c065b80 0x7f3a4c068040 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a6e95a640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3a68160a20 0x7f3a680cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a6e95a640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a6815d070 0x7f3a680cb2c0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a6e95a640 1 --2- 172.21.15.26:0/3822551493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a6815cc90 0x7f3a68156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a6e95a640 1 -- 172.21.15.26:0/3822551493 >> 172.21.15.26:0/3822551493 conn(0x7f3a6806a320 msgr2=0x7f3a68158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:37.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a6e95a640 1 -- 172.21.15.26:0/3822551493 shutdown_connections 2023-12-19T18:16:37.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.486+0000 7f3a6e95a640 1 -- 172.21.15.26:0/3822551493 wait complete. 2023-12-19T18:16:37.510 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:16:37.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.650+0000 7fd2a9ff3640 1 Processor -- start 2023-12-19T18:16:37.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.650+0000 7fd2a9ff3640 1 -- start start 2023-12-19T18:16:37.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.650+0000 7fd2a9ff3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd2a4154220 0x7fd2a415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.650+0000 7fd2a9ff3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2a40cee10 0x7fd2a415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.650+0000 7fd2a9ff3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a406bc30 con 0x7fd2a406b3d0 2023-12-19T18:16:37.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.650+0000 7fd2a9ff3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a40cc6e0 con 0x7fd2a4154220 2023-12-19T18:16:37.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.650+0000 7fd2a9ff3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a415afc0 con 0x7fd2a40cee10 2023-12-19T18:16:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.650+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2a40cee10 0x7fd2a415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.654+0000 7fd2a2ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd2a406b3d0 0x7fd2a40ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:59838/0 (socket says 172.21.15.26:59838) 2023-12-19T18:16:37.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.654+0000 7fd2a2ffd640 1 -- 172.21.15.26:0/2452371405 learned_addr learned my addr 172.21.15.26:0/2452371405 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.654+0000 7fd2a37fe640 1 --2- 172.21.15.26:0/2452371405 >> v2:172.21.15.160:3300/0 conn(0x7fd2a4154220 0x7fd2a415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.654+0000 7fd2a3fff640 1 -- 172.21.15.26:0/2452371405 >> v2:172.21.15.160:3300/0 conn(0x7fd2a4154220 msgr2=0x7fd2a415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.654+0000 7fd2a3fff640 1 --2- 172.21.15.26:0/2452371405 >> v2:172.21.15.160:3300/0 conn(0x7fd2a4154220 0x7fd2a415aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.654+0000 7fd2a3fff640 1 -- 172.21.15.26:0/2452371405 >> v1:172.21.15.160:6789/0 conn(0x7fd2a406b3d0 legacy=0x7fd2a40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.654+0000 7fd2a3fff640 1 -- 172.21.15.26:0/2452371405 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a406b950 con 0x7fd2a40cee10 2023-12-19T18:16:37.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.654+0000 7fd2a27fc640 1 -- 172.21.15.26:0/2452371405 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206588564 0 0) 0x7fd2a406bc30 con 0x7fd2a406b3d0 2023-12-19T18:16:37.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.654+0000 7fd2a3fff640 1 --2- 172.21.15.26:0/2452371405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2a40cee10 0x7fd2a415b1c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009d90 tx=0x7fd29c01c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea2938bfc7f8282e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:37.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.654+0000 7fd2a27fc640 1 -- 172.21.15.26:0/2452371405 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0a59e0 con 0x7fd2a40cee10 2023-12-19T18:16:37.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.654+0000 7fd2a27fc640 1 -- 172.21.15.26:0/2452371405 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c0a5b40 con 0x7fd2a40cee10 2023-12-19T18:16:37.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.654+0000 7fd2a27fc640 1 -- 172.21.15.26:0/2452371405 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0ad440 con 0x7fd2a40cee10 2023-12-19T18:16:37.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/2452371405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2a40cee10 msgr2=0x7fd2a415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 --2- 172.21.15.26:0/2452371405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2a40cee10 0x7fd2a415b1c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009d90 tx=0x7fd29c01c920 comp rx=0 tx=0).stop 2023-12-19T18:16:37.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/2452371405 shutdown_connections 2023-12-19T18:16:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 --2- 172.21.15.26:0/2452371405 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2a40cee10 0x7fd2a415b1c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 --2- 172.21.15.26:0/2452371405 >> v2:172.21.15.160:3300/0 conn(0x7fd2a4154220 0x7fd2a415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/2452371405 >> 172.21.15.26:0/2452371405 conn(0x7fd2a40658e0 msgr2=0x7fd2a4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/2452371405 shutdown_connections 2023-12-19T18:16:37.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/2452371405 wait complete. 2023-12-19T18:16:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 Processor -- start 2023-12-19T18:16:37.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 -- start start 2023-12-19T18:16:37.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2a406b3d0 0x7fd2a40b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a40cee10 0x7fd2a40acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2a4154220 0x7fd2a414cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a414d0f0 con 0x7fd2a40cee10 2023-12-19T18:16:37.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a40b2d80 con 0x7fd2a406b3d0 2023-12-19T18:16:37.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a9ff3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a40b3060 con 0x7fd2a4154220 2023-12-19T18:16:37.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a40cee10 0x7fd2a40acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a40cee10 0x7fd2a40acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39366/0 (socket says 172.21.15.26:39366) 2023-12-19T18:16:37.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a2ffd640 1 -- 172.21.15.26:0/82542432 learned_addr learned my addr 172.21.15.26:0/82542432 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:37.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a3fff640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2a4154220 0x7fd2a414cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.658+0000 7fd2a37fe640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2a406b3d0 0x7fd2a40b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd2a2ffd640 1 -- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2a4154220 msgr2=0x7fd2a414cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd2a2ffd640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2a4154220 0x7fd2a414cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd2a2ffd640 1 -- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2a406b3d0 msgr2=0x7fd2a40b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd2a2ffd640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2a406b3d0 0x7fd2a40b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd2a2ffd640 1 -- 172.21.15.26:0/82542432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c009a70 con 0x7fd2a40cee10 2023-12-19T18:16:37.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd2a2ffd640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a40cee10 0x7fd2a40acdf0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fd2980011f0 tx=0x7fd29800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:37.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd28bfff640 1 -- 172.21.15.26:0/82542432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2980074e0 con 0x7fd2a40cee10 2023-12-19T18:16:37.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd28bfff640 1 -- 172.21.15.26:0/82542432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd298007640 con 0x7fd2a40cee10 2023-12-19T18:16:37.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd28bfff640 1 -- 172.21.15.26:0/82542432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298004a10 con 0x7fd2a40cee10 2023-12-19T18:16:37.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/82542432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a40b3340 con 0x7fd2a40cee10 2023-12-19T18:16:37.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/82542432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a414d530 con 0x7fd2a40cee10 2023-12-19T18:16:37.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/82542432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a4074270 con 0x7fd2a40cee10 2023-12-19T18:16:37.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd28bfff640 1 -- 172.21.15.26:0/82542432 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd298004b70 con 0x7fd2a40cee10 2023-12-19T18:16:37.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd28bfff640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd290065b80 0x7fd290068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.662+0000 7fd28bfff640 1 -- 172.21.15.26:0/82542432 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fd298082800 con 0x7fd2a40cee10 2023-12-19T18:16:37.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.666+0000 7fd2a37fe640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd290065b80 0x7fd290068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.666+0000 7fd2a37fe640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd290065b80 0x7fd290068040 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4070d00 tx=0x7fd294009190 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:37.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.666+0000 7fd28bfff640 1 -- 172.21.15.26:0/82542432 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd29804fdd0 con 0x7fd2a40cee10 2023-12-19T18:16:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/82542432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd2a415d200 con 0x7fd2a40cee10 2023-12-19T18:16:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd28bfff640 1 -- 172.21.15.26:0/82542432 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v212) v1 ==== 84+0+209 (secure 0 0 0) 0x7fd2980533f0 con 0x7fd2a40cee10 2023-12-19T18:16:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:37.829 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T18:16:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T18:16:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T18:16:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 24, 2023-12-19T18:16:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": "cephfs_metadata" 2023-12-19T18:16:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 25, 2023-12-19T18:16:37.829 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": "cephfs_data" 2023-12-19T18:16:37.830 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:37.830 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T18:16:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd290065b80 msgr2=0x7fd290068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd2a9ff3640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd290065b80 0x7fd290068040 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4070d00 tx=0x7fd294009190 comp rx=0 tx=0).stop 2023-12-19T18:16:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/82542432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a40cee10 msgr2=0x7fd2a40acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd2a9ff3640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a40cee10 0x7fd2a40acdf0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fd2980011f0 tx=0x7fd29800c9b0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/82542432 shutdown_connections 2023-12-19T18:16:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd2a9ff3640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd290065b80 0x7fd290068040 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd2a9ff3640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd2a4154220 0x7fd2a414cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd2a9ff3640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2a40cee10 0x7fd2a40acdf0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd2a9ff3640 1 --2- 172.21.15.26:0/82542432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd2a406b3d0 0x7fd2a40b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/82542432 >> 172.21.15.26:0/82542432 conn(0x7fd2a40658e0 msgr2=0x7fd2a40ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/82542432 shutdown_connections 2023-12-19T18:16:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.826+0000 7fd2a9ff3640 1 -- 172.21.15.26:0/82542432 wait complete. 2023-12-19T18:16:37.850 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-12-19T18:16:37.850 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-12-19T18:16:37.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.994+0000 7feee00e5640 1 Processor -- start 2023-12-19T18:16:37.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.994+0000 7feee00e5640 1 -- start start 2023-12-19T18:16:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.994+0000 7feee00e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feed815aad0 0x7feed806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.994+0000 7feee00e5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feed80b3f90 0x7feed80cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.994+0000 7feee00e5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed806cbf0 con 0x7feed80b1590 2023-12-19T18:16:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.994+0000 7feee00e5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed80ca8b0 con 0x7feed80b3f90 2023-12-19T18:16:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.994+0000 7feee00e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed806b6f0 con 0x7feed815aad0 2023-12-19T18:16:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.998+0000 7feede682640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feed80b3f90 0x7feed80cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.998+0000 7feede682640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feed80b3f90 0x7feed80cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39368/0 (socket says 172.21.15.26:39368) 2023-12-19T18:16:37.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.998+0000 7feede682640 1 -- 172.21.15.26:0/3414457494 learned_addr learned my addr 172.21.15.26:0/3414457494 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.998+0000 7feedde81640 1 --2- 172.21.15.26:0/3414457494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feed815aad0 0x7feed806c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.998+0000 7feede682640 1 -- 172.21.15.26:0/3414457494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feed815aad0 msgr2=0x7feed806c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.998+0000 7feede682640 1 --2- 172.21.15.26:0/3414457494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feed815aad0 0x7feed806c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.998+0000 7feede682640 1 -- 172.21.15.26:0/3414457494 >> v1:172.21.15.160:6789/0 conn(0x7feed80b1590 legacy=0x7feed8070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.998+0000 7feede682640 1 -- 172.21.15.26:0/3414457494 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed806b9d0 con 0x7feed80b3f90 2023-12-19T18:16:38.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.998+0000 7feedde81640 1 --2- 172.21.15.26:0/3414457494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feed815aad0 0x7feed806c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:38.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.998+0000 7feede682640 1 --2- 172.21.15.26:0/3414457494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed80b3f90 0x7feed80cad40 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7feed0009e50 tx=0x7feed001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b868f9ad0756d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:38.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.998+0000 7feedce7f640 1 -- 172.21.15.26:0/3414457494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed00a5730 con 0x7feed80b3f90 2023-12-19T18:16:38.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.998+0000 7feedce7f640 1 -- 172.21.15.26:0/3414457494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed00a5890 con 0x7feed80b3f90 2023-12-19T18:16:38.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:37.998+0000 7feedce7f640 1 -- 172.21.15.26:0/3414457494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed00a5c20 con 0x7feed80b3f90 2023-12-19T18:16:38.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 -- 172.21.15.26:0/3414457494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed80b3f90 msgr2=0x7feed80cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:38.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 --2- 172.21.15.26:0/3414457494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed80b3f90 0x7feed80cad40 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7feed0009e50 tx=0x7feed001c9a0 comp rx=0 tx=0).stop 2023-12-19T18:16:38.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 -- 172.21.15.26:0/3414457494 shutdown_connections 2023-12-19T18:16:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 --2- 172.21.15.26:0/3414457494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed80b3f90 0x7feed80cad40 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 --2- 172.21.15.26:0/3414457494 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feed815aad0 0x7feed806c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 -- 172.21.15.26:0/3414457494 >> 172.21.15.26:0/3414457494 conn(0x7feed80b0270 msgr2=0x7feed80b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 -- 172.21.15.26:0/3414457494 shutdown_connections 2023-12-19T18:16:38.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 -- 172.21.15.26:0/3414457494 wait complete. 2023-12-19T18:16:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 Processor -- start 2023-12-19T18:16:38.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 -- start start 2023-12-19T18:16:38.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feed80b1590 0x7feed8159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed80b3f90 0x7feed80ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feed815aad0 0x7feed80acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed80c61b0 con 0x7feed80b3f90 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed80c7d40 con 0x7feed815aad0 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feee00e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed80c5810 con 0x7feed80b1590 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feedd680640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed80b3f90 0x7feed80ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feedd680640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed80b3f90 0x7feed80ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39376/0 (socket says 172.21.15.26:39376) 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feedd680640 1 -- 172.21.15.26:0/3357498759 learned_addr learned my addr 172.21.15.26:0/3357498759 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feedde81640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feed80b1590 0x7feed8159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feede682640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feed815aad0 0x7feed80acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feedd680640 1 -- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feed80b1590 msgr2=0x7feed8159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feedd680640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feed80b1590 0x7feed8159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feedd680640 1 -- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feed815aad0 msgr2=0x7feed80acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feedd680640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feed815aad0 0x7feed80acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:38.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.002+0000 7feedd680640 1 -- 172.21.15.26:0/3357498759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed0009b30 con 0x7feed80b3f90 2023-12-19T18:16:38.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.006+0000 7feedd680640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed80b3f90 0x7feed80ac860 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7feec800c410 tx=0x7feec800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:38.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.006+0000 7feece7fc640 1 -- 172.21.15.26:0/3357498759 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8011790 con 0x7feed80b3f90 2023-12-19T18:16:38.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.006+0000 7feece7fc640 1 -- 172.21.15.26:0/3357498759 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec80118f0 con 0x7feed80b3f90 2023-12-19T18:16:38.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.006+0000 7feee00e5640 1 -- 172.21.15.26:0/3357498759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed80c5af0 con 0x7feed80b3f90 2023-12-19T18:16:38.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.006+0000 7feece7fc640 1 -- 172.21.15.26:0/3357498759 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8011ce0 con 0x7feed80b3f90 2023-12-19T18:16:38.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.006+0000 7feee00e5640 1 -- 172.21.15.26:0/3357498759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed815a210 con 0x7feed80b3f90 2023-12-19T18:16:38.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.006+0000 7feee00e5640 1 -- 172.21.15.26:0/3357498759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed80c62f0 con 0x7feed80b3f90 2023-12-19T18:16:38.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.006+0000 7feece7fc640 1 -- 172.21.15.26:0/3357498759 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7feec801c460 con 0x7feed80b3f90 2023-12-19T18:16:38.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.006+0000 7feece7fc640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7feebc065b80 0x7feebc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:38.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.006+0000 7feece7fc640 1 -- 172.21.15.26:0/3357498759 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6469+0+0 (secure 0 0 0) 0x7feec8083190 con 0x7feed80b3f90 2023-12-19T18:16:38.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.006+0000 7feedde81640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7feebc065b80 0x7feebc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.010+0000 7feedde81640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7feebc065b80 0x7feebc068040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feed4006fd0 tx=0x7feed4008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:38.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.010+0000 7feece7fc640 1 -- 172.21.15.26:0/3357498759 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feed80c62f0 con 0x7feed80b3f90 2023-12-19T18:16:38.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:38.170+0000 7feee00e5640 1 -- 172.21.15.26:0/3357498759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7feed80c62f0 con 0x7feed80b3f90 2023-12-19T18:16:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.122+0000 7feece7fc640 1 -- 172.21.15.26:0/3357498759 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v213) v1 ==== 144+0+0 (secure 0 0 0) 0x7feed80c62f0 con 0x7feed80b3f90 2023-12-19T18:16:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'backup_fs_metadata' created 2023-12-19T18:16:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.126+0000 7feee00e5640 1 -- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7feebc065b80 msgr2=0x7feebc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:39.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.126+0000 7feee00e5640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7feebc065b80 0x7feebc068040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feed4006fd0 tx=0x7feed4008040 comp rx=0 tx=0).stop 2023-12-19T18:16:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.126+0000 7feee00e5640 1 -- 172.21.15.26:0/3357498759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed80b3f90 msgr2=0x7feed80ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.126+0000 7feee00e5640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed80b3f90 0x7feed80ac860 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7feec800c410 tx=0x7feec800c8d0 comp rx=0 tx=0).stop 2023-12-19T18:16:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.126+0000 7feee00e5640 1 -- 172.21.15.26:0/3357498759 shutdown_connections 2023-12-19T18:16:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.126+0000 7feee00e5640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7feebc065b80 0x7feebc068040 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.126+0000 7feee00e5640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feed815aad0 0x7feed80acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.126+0000 7feee00e5640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed80b3f90 0x7feed80ac860 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:39.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.126+0000 7feee00e5640 1 --2- 172.21.15.26:0/3357498759 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7feed80b1590 0x7feed8159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.126+0000 7feee00e5640 1 -- 172.21.15.26:0/3357498759 >> 172.21.15.26:0/3357498759 conn(0x7feed80b0270 msgr2=0x7feed80b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.130+0000 7feee00e5640 1 -- 172.21.15.26:0/3357498759 shutdown_connections 2023-12-19T18:16:39.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.130+0000 7feee00e5640 1 -- 172.21.15.26:0/3357498759 wait complete. 2023-12-19T18:16:39.159 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:16:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.310+0000 7f7957e66640 1 Processor -- start 2023-12-19T18:16:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.310+0000 7f7957e66640 1 -- start start 2023-12-19T18:16:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.310+0000 7f7957e66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f79500b0c30 0x7f79500ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.310+0000 7f7957e66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f795006b500 con 0x7f7950070db0 2023-12-19T18:16:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.310+0000 7f7957e66640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7950070b60 con 0x7f795015ada0 2023-12-19T18:16:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.310+0000 7f7957e66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79500715b0 con 0x7f79500b0c30 2023-12-19T18:16:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.310+0000 7f7955401640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f79500b0c30 0x7f79500ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.310+0000 7f7956403640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7950070db0 0x7f795006b990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41368/0 (socket says 172.21.15.26:41368) 2023-12-19T18:16:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.310+0000 7f7956403640 1 -- 172.21.15.26:0/2819591579 learned_addr learned my addr 172.21.15.26:0/2819591579 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.310+0000 7f7955401640 1 -- 172.21.15.26:0/2819591579 >> v1:172.21.15.160:6789/0 conn(0x7f795015ada0 legacy=0x7f79500b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.310+0000 7f7955401640 1 -- 172.21.15.26:0/2819591579 >> v1:172.21.15.26:6789/0 conn(0x7f7950070db0 legacy=0x7f795006b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.310+0000 7f7955401640 1 -- 172.21.15.26:0/2819591579 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79500702c0 con 0x7f79500b0c30 2023-12-19T18:16:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.310+0000 7f7954c00640 1 -- 172.21.15.26:0/2819591579 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235113590 0 0) 0x7f795006b500 con 0x7f7950070db0 2023-12-19T18:16:39.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.314+0000 7f7955401640 1 --2- 172.21.15.26:0/2819591579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79500b0c30 0x7f79500ada90 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f79400097f0 tx=0x7f794001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293302970312b235 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:39.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.314+0000 7f7954c00640 1 -- 172.21.15.26:0/2819591579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79400ab600 con 0x7f79500b0c30 2023-12-19T18:16:39.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.314+0000 7f7954c00640 1 -- 172.21.15.26:0/2819591579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79400a7050 con 0x7f79500b0c30 2023-12-19T18:16:39.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.314+0000 7f7954c00640 1 -- 172.21.15.26:0/2819591579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79400ab980 con 0x7f79500b0c30 2023-12-19T18:16:39.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.314+0000 7f7957e66640 1 -- 172.21.15.26:0/2819591579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79500b0c30 msgr2=0x7f79500ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:39.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.314+0000 7f7957e66640 1 --2- 172.21.15.26:0/2819591579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79500b0c30 0x7f79500ada90 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f79400097f0 tx=0x7f794001c480 comp rx=0 tx=0).stop 2023-12-19T18:16:39.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.314+0000 7f7957e66640 1 -- 172.21.15.26:0/2819591579 shutdown_connections 2023-12-19T18:16:39.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.314+0000 7f7957e66640 1 --2- 172.21.15.26:0/2819591579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79500b0c30 0x7f79500ada90 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:39.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.314+0000 7f7957e66640 1 -- 172.21.15.26:0/2819591579 >> 172.21.15.26:0/2819591579 conn(0x7f79500660d0 msgr2=0x7f79500ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:39.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.314+0000 7f7957e66640 1 -- 172.21.15.26:0/2819591579 shutdown_connections 2023-12-19T18:16:39.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.314+0000 7f7957e66640 1 -- 172.21.15.26:0/2819591579 wait complete. 2023-12-19T18:16:39.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7957e66640 1 Processor -- start 2023-12-19T18:16:39.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7957e66640 1 -- start start 2023-12-19T18:16:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7957e66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7950070db0 0x7f79500c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7957e66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f79500b0c30 0x7f795015ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7957e66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f795015ada0 0x7f795015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7957e66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795015dbb0 con 0x7f7950070db0 2023-12-19T18:16:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7957e66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795015d5f0 con 0x7f79500b0c30 2023-12-19T18:16:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7957e66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795015d8d0 con 0x7f795015ada0 2023-12-19T18:16:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7955c02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7950070db0 0x7f79500c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7955c02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7950070db0 0x7f79500c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39394/0 (socket says 172.21.15.26:39394) 2023-12-19T18:16:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7955c02640 1 -- 172.21.15.26:0/1334795750 learned_addr learned my addr 172.21.15.26:0/1334795750 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7955401640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f79500b0c30 0x7f795015ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7956403640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f795015ada0 0x7f795015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7955c02640 1 -- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f795015ada0 msgr2=0x7f795015cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7955c02640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f795015ada0 0x7f795015cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7955c02640 1 -- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f79500b0c30 msgr2=0x7f795015ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7955c02640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f79500b0c30 0x7f795015ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7955c02640 1 -- 172.21.15.26:0/1334795750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7940009420 con 0x7f7950070db0 2023-12-19T18:16:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7956403640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f795015ada0 0x7f795015cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7955c02640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7950070db0 0x7f79500c46e0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c500 tx=0x7f794c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f79467fc640 1 -- 172.21.15.26:0/1334795750 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c005220 con 0x7f7950070db0 2023-12-19T18:16:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f79467fc640 1 -- 172.21.15.26:0/1334795750 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c005380 con 0x7f7950070db0 2023-12-19T18:16:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f79467fc640 1 -- 172.21.15.26:0/1334795750 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c007270 con 0x7f7950070db0 2023-12-19T18:16:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7957e66640 1 -- 172.21.15.26:0/1334795750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7950156c90 con 0x7f7950070db0 2023-12-19T18:16:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.318+0000 7f7957e66640 1 -- 172.21.15.26:0/1334795750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79501410e0 con 0x7f7950070db0 2023-12-19T18:16:39.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.322+0000 7f7957e66640 1 -- 172.21.15.26:0/1334795750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7950156af0 con 0x7f7950070db0 2023-12-19T18:16:39.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.322+0000 7f79467fc640 1 -- 172.21.15.26:0/1334795750 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f794c0073d0 con 0x7f7950070db0 2023-12-19T18:16:39.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.322+0000 7f79467fc640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7934065b80 0x7f7934068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:39.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.322+0000 7f79467fc640 1 -- 172.21.15.26:0/1334795750 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6842+0+0 (secure 0 0 0) 0x7f794c083140 con 0x7f7950070db0 2023-12-19T18:16:39.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.322+0000 7f7955401640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7934065b80 0x7f7934068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:39.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.322+0000 7f7955401640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7934065b80 0x7f7934068040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79400ad000 tx=0x7f79400a6000 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.326+0000 7f79467fc640 1 -- 172.21.15.26:0/1334795750 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7950156af0 con 0x7f7950070db0 2023-12-19T18:16:39.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:39.486+0000 7f7957e66640 1 -- 172.21.15.26:0/1334795750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f79501571b0 con 0x7f7950070db0 2023-12-19T18:16:40.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.134+0000 7f79467fc640 1 -- 172.21.15.26:0/1334795750 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v214) v1 ==== 176+0+0 (secure 0 0 0) 0x7f79501571b0 con 0x7f7950070db0 2023-12-19T18:16:40.136 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'backup_fs_data' created 2023-12-19T18:16:40.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.146+0000 7f7957e66640 1 -- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7934065b80 msgr2=0x7f7934068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.146+0000 7f7957e66640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7934065b80 0x7f7934068040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79400ad000 tx=0x7f79400a6000 comp rx=0 tx=0).stop 2023-12-19T18:16:40.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.146+0000 7f7957e66640 1 -- 172.21.15.26:0/1334795750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7950070db0 msgr2=0x7f79500c46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.146+0000 7f7957e66640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7950070db0 0x7f79500c46e0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f794c00c500 tx=0x7f794c00c9c0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.146+0000 7f7957e66640 1 -- 172.21.15.26:0/1334795750 shutdown_connections 2023-12-19T18:16:40.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.146+0000 7f7957e66640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7934065b80 0x7f7934068040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.146+0000 7f7957e66640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f795015ada0 0x7f795015cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.146+0000 7f7957e66640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f79500b0c30 0x7f795015ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.146+0000 7f7957e66640 1 --2- 172.21.15.26:0/1334795750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7950070db0 0x7f79500c46e0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.146+0000 7f7957e66640 1 -- 172.21.15.26:0/1334795750 >> 172.21.15.26:0/1334795750 conn(0x7f79500660d0 msgr2=0x7f7950159f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:40.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.150+0000 7f7957e66640 1 -- 172.21.15.26:0/1334795750 shutdown_connections 2023-12-19T18:16:40.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.150+0000 7f7957e66640 1 -- 172.21.15.26:0/1334795750 wait complete. 2023-12-19T18:16:40.182 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-12-19T18:16:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f435a9f3640 1 Processor -- start 2023-12-19T18:16:40.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f435a9f3640 1 -- start start 2023-12-19T18:16:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f435a9f3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4354159820 0x7f435415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f435a9f3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435406fb60 con 0x7f4354159c00 2023-12-19T18:16:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f435a9f3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f435406fe40 con 0x7f435406f6c0 2023-12-19T18:16:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f435a9f3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f435415e470 con 0x7f4354159820 2023-12-19T18:16:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f4353fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4354159820 0x7f435415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f4358f90640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4354159c00 0x7f4354070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41372/0 (socket says 172.21.15.26:41372) 2023-12-19T18:16:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f4358f90640 1 -- 172.21.15.26:0/828005019 learned_addr learned my addr 172.21.15.26:0/828005019 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:40.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f4353fff640 1 --2- 172.21.15.26:0/828005019 >> v2:172.21.15.26:3300/0 conn(0x7f4354159820 0x7f435415df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39406/0 (socket says 172.21.15.26:39406) 2023-12-19T18:16:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f4353fff640 1 -- 172.21.15.26:0/828005019 >> v1:172.21.15.160:6789/0 conn(0x7f435406f6c0 legacy=0x7f43540c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f4353fff640 1 -- 172.21.15.26:0/828005019 >> v1:172.21.15.26:6789/0 conn(0x7f4354159c00 legacy=0x7f4354070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f4353fff640 1 -- 172.21.15.26:0/828005019 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4354070120 con 0x7f4354159820 2023-12-19T18:16:40.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.326+0000 7f4352ffd640 1 -- 172.21.15.26:0/828005019 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912198571 0 0) 0x7f435406fb60 con 0x7f4354159c00 2023-12-19T18:16:40.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.330+0000 7f4353fff640 1 --2- 172.21.15.26:0/828005019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4354159820 0x7f435415df30 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f434400ba60 tx=0x7f434401eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ee1f42f0d75be server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.330+0000 7f4352ffd640 1 -- 172.21.15.26:0/828005019 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43440a7bf0 con 0x7f4354159820 2023-12-19T18:16:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.330+0000 7f4352ffd640 1 -- 172.21.15.26:0/828005019 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43440a7d50 con 0x7f4354159820 2023-12-19T18:16:40.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.330+0000 7f4352ffd640 1 -- 172.21.15.26:0/828005019 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43440af660 con 0x7f4354159820 2023-12-19T18:16:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.330+0000 7f435a9f3640 1 -- 172.21.15.26:0/828005019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4354159820 msgr2=0x7f435415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.330+0000 7f435a9f3640 1 --2- 172.21.15.26:0/828005019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4354159820 0x7f435415df30 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f434400ba60 tx=0x7f434401eb20 comp rx=0 tx=0).stop 2023-12-19T18:16:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.330+0000 7f435a9f3640 1 -- 172.21.15.26:0/828005019 shutdown_connections 2023-12-19T18:16:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.330+0000 7f435a9f3640 1 --2- 172.21.15.26:0/828005019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4354159820 0x7f435415df30 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.330+0000 7f435a9f3640 1 -- 172.21.15.26:0/828005019 >> 172.21.15.26:0/828005019 conn(0x7f43540ce730 msgr2=0x7f43540ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.330+0000 7f435a9f3640 1 -- 172.21.15.26:0/828005019 shutdown_connections 2023-12-19T18:16:40.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.330+0000 7f435a9f3640 1 -- 172.21.15.26:0/828005019 wait complete. 2023-12-19T18:16:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f435a9f3640 1 Processor -- start 2023-12-19T18:16:40.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f435a9f3640 1 -- start start 2023-12-19T18:16:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f435a9f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f435406f6c0 0x7f4354147700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f435a9f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4354159820 0x7f43540c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f435a9f3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4354159c00 0x7f4354150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f435a9f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4354159fe0 con 0x7f435406f6c0 2023-12-19T18:16:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f435a9f3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43540c3050 con 0x7f4354159c00 2023-12-19T18:16:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f435a9f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43540c3330 con 0x7f4354159820 2023-12-19T18:16:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4353fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f435406f6c0 0x7f4354147700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4353fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f435406f6c0 0x7f4354147700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39412/0 (socket says 172.21.15.26:39412) 2023-12-19T18:16:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4353fff640 1 -- 172.21.15.26:0/1250104142 learned_addr learned my addr 172.21.15.26:0/1250104142 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4358f90640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4354159c00 0x7f4354150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:40.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f43537fe640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4354159820 0x7f43540c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4353fff640 1 -- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4354159820 msgr2=0x7f43540c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4353fff640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4354159820 0x7f43540c2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4353fff640 1 -- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4354159c00 msgr2=0x7f4354150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4353fff640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4354159c00 0x7f4354150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4353fff640 1 -- 172.21.15.26:0/1250104142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434400b6c0 con 0x7f435406f6c0 2023-12-19T18:16:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4358f90640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4354159c00 0x7f4354150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f43537fe640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4354159820 0x7f43540c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4353fff640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f435406f6c0 0x7f4354147700 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f434401b730 tx=0x7f434401b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4350ff9640 1 -- 172.21.15.26:0/1250104142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43440a7b90 con 0x7f435406f6c0 2023-12-19T18:16:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4350ff9640 1 -- 172.21.15.26:0/1250104142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43440b8440 con 0x7f435406f6c0 2023-12-19T18:16:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f4350ff9640 1 -- 172.21.15.26:0/1250104142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43440b87f0 con 0x7f435406f6c0 2023-12-19T18:16:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.334+0000 7f435a9f3640 1 -- 172.21.15.26:0/1250104142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435406b880 con 0x7f435406f6c0 2023-12-19T18:16:40.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.338+0000 7f435a9f3640 1 -- 172.21.15.26:0/1250104142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43540ca430 con 0x7f435406f6c0 2023-12-19T18:16:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.338+0000 7f435a9f3640 1 -- 172.21.15.26:0/1250104142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4354074270 con 0x7f435406f6c0 2023-12-19T18:16:40.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.338+0000 7f4350ff9640 1 -- 172.21.15.26:0/1250104142 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f43440b6070 con 0x7f435406f6c0 2023-12-19T18:16:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.338+0000 7f4350ff9640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4340065b80 0x7f4340068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.342+0000 7f4350ff9640 1 -- 172.21.15.26:0/1250104142 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 7227+0+0 (secure 0 0 0) 0x7f434411ebc0 con 0x7f435406f6c0 2023-12-19T18:16:40.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.342+0000 7f43537fe640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4340065b80 0x7f4340068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:40.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.342+0000 7f43537fe640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4340065b80 0x7f4340068040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4354157c70 tx=0x7f4348009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:40.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.342+0000 7f4350ff9640 1 -- 172.21.15.26:0/1250104142 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f43440ebef0 con 0x7f435406f6c0 2023-12-19T18:16:40.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.530+0000 7f435a9f3640 1 -- 172.21.15.26:0/1250104142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7f43540b46b0 con 0x7f435406f6c0 2023-12-19T18:16:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.542+0000 7f4350ff9640 1 -- 172.21.15.26:0/1250104142 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 26 and data pool 27 v98) v1 ==== 183+0+0 (secure 0 0 0) 0x7f43440bc020 con 0x7f435406f6c0 2023-12-19T18:16:40.544 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 26 and data pool 27 2023-12-19T18:16:40.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.546+0000 7f435a9f3640 1 -- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4340065b80 msgr2=0x7f4340068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.546+0000 7f435a9f3640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4340065b80 0x7f4340068040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4354157c70 tx=0x7f4348009040 comp rx=0 tx=0).stop 2023-12-19T18:16:40.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.546+0000 7f435a9f3640 1 -- 172.21.15.26:0/1250104142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f435406f6c0 msgr2=0x7f4354147700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.546+0000 7f435a9f3640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f435406f6c0 0x7f4354147700 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f434401b730 tx=0x7f434401b8c0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.550+0000 7f435a9f3640 1 -- 172.21.15.26:0/1250104142 shutdown_connections 2023-12-19T18:16:40.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.550+0000 7f435a9f3640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4340065b80 0x7f4340068040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.550+0000 7f435a9f3640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4354159c00 0x7f4354150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.550+0000 7f435a9f3640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4354159820 0x7f43540c2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.550+0000 7f435a9f3640 1 --2- 172.21.15.26:0/1250104142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f435406f6c0 0x7f4354147700 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.550+0000 7f435a9f3640 1 -- 172.21.15.26:0/1250104142 >> 172.21.15.26:0/1250104142 conn(0x7f43540ce730 msgr2=0x7f4354158e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:40.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.550+0000 7f435a9f3640 1 -- 172.21.15.26:0/1250104142 shutdown_connections 2023-12-19T18:16:40.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.550+0000 7f435a9f3640 1 -- 172.21.15.26:0/1250104142 wait complete. 2023-12-19T18:16:40.593 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.734+0000 7f23a5620640 1 Processor -- start 2023-12-19T18:16:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.734+0000 7f23a5620640 1 -- start start 2023-12-19T18:16:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.734+0000 7f23a5620640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f23a0154220 0x7f23a015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.734+0000 7f23a5620640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f23a006b3d0 0x7f23a00ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.734+0000 7f23a5620640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a006bc30 con 0x7f23a00cec40 2023-12-19T18:16:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.734+0000 7f23a5620640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a00cf0d0 con 0x7f23a006b3d0 2023-12-19T18:16:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.734+0000 7f23a5620640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a00cc6e0 con 0x7f23a0154220 2023-12-19T18:16:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.734+0000 7f23967fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f23a006b3d0 0x7f23a00ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.734+0000 7f239f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f23a00cec40 0x7f23a015d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41382/0 (socket says 172.21.15.26:41382) 2023-12-19T18:16:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.734+0000 7f239f7fe640 1 -- 172.21.15.26:0/418758684 learned_addr learned my addr 172.21.15.26:0/418758684 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f239effd640 1 --2- 172.21.15.26:0/418758684 >> v2:172.21.15.160:3300/0 conn(0x7f23a0154220 0x7f23a015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f239e7fc640 1 -- 172.21.15.26:0/418758684 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200262880 0 0) 0x7f23a006bc30 con 0x7f23a00cec40 2023-12-19T18:16:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f239e7fc640 1 -- 172.21.15.26:0/418758684 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f238c003530 con 0x7f23a00cec40 2023-12-19T18:16:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f23967fc640 1 -- 172.21.15.26:0/418758684 >> v2:172.21.15.160:3300/0 conn(0x7f23a0154220 msgr2=0x7f23a015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f23967fc640 1 --2- 172.21.15.26:0/418758684 >> v2:172.21.15.160:3300/0 conn(0x7f23a0154220 0x7f23a015aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f23967fc640 1 -- 172.21.15.26:0/418758684 >> v1:172.21.15.26:6789/0 conn(0x7f23a00cec40 legacy=0x7f23a015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f23967fc640 1 -- 172.21.15.26:0/418758684 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a015afc0 con 0x7f23a006b3d0 2023-12-19T18:16:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f239effd640 1 --2- 172.21.15.26:0/418758684 >> v2:172.21.15.160:3300/0 conn(0x7f23a0154220 0x7f23a015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f23967fc640 1 --2- 172.21.15.26:0/418758684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a006b3d0 0x7f23a00ce700 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f23a00667b0 tx=0x7f238801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e120f190f0dc7e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f239e7fc640 1 -- 172.21.15.26:0/418758684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388004230 con 0x7f23a006b3d0 2023-12-19T18:16:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f239e7fc640 1 -- 172.21.15.26:0/418758684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23880a7050 con 0x7f23a006b3d0 2023-12-19T18:16:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f239e7fc640 1 -- 172.21.15.26:0/418758684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388003620 con 0x7f23a006b3d0 2023-12-19T18:16:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f23a5620640 1 -- 172.21.15.26:0/418758684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a006b3d0 msgr2=0x7f23a00ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f23a5620640 1 --2- 172.21.15.26:0/418758684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a006b3d0 0x7f23a00ce700 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f23a00667b0 tx=0x7f238801c430 comp rx=0 tx=0).stop 2023-12-19T18:16:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.738+0000 7f23a5620640 1 -- 172.21.15.26:0/418758684 shutdown_connections 2023-12-19T18:16:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23a5620640 1 --2- 172.21.15.26:0/418758684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a006b3d0 0x7f23a00ce700 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23a5620640 1 --2- 172.21.15.26:0/418758684 >> v2:172.21.15.160:3300/0 conn(0x7f23a0154220 0x7f23a015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23a5620640 1 -- 172.21.15.26:0/418758684 >> 172.21.15.26:0/418758684 conn(0x7f23a00658e0 msgr2=0x7f23a0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23a5620640 1 -- 172.21.15.26:0/418758684 shutdown_connections 2023-12-19T18:16:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23a5620640 1 -- 172.21.15.26:0/418758684 wait complete. 2023-12-19T18:16:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23a5620640 1 Processor -- start 2023-12-19T18:16:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23a5620640 1 -- start start 2023-12-19T18:16:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23a5620640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23a006b3d0 0x7f23a014c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23a5620640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23a00cec40 0x7f23a0070290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23a5620640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a0154220 0x7f23a00b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23a5620640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a014cb30 con 0x7f23a0154220 2023-12-19T18:16:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23a5620640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a014d5f0 con 0x7f23a006b3d0 2023-12-19T18:16:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23a5620640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a0070d30 con 0x7f23a00cec40 2023-12-19T18:16:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f239f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a0154220 0x7f23a00b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f239f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a0154220 0x7f23a00b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39432/0 (socket says 172.21.15.26:39432) 2023-12-19T18:16:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f239f7fe640 1 -- 172.21.15.26:0/1219258528 learned_addr learned my addr 172.21.15.26:0/1219258528 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f239effd640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23a006b3d0 0x7f23a014c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:40.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f23967fc640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23a00cec40 0x7f23a0070290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f239f7fe640 1 -- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23a00cec40 msgr2=0x7f23a0070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f239f7fe640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23a00cec40 0x7f23a0070290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f239f7fe640 1 -- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23a006b3d0 msgr2=0x7f23a014c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f239f7fe640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23a006b3d0 0x7f23a014c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.742+0000 7f239f7fe640 1 -- 172.21.15.26:0/1219258528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2388009420 con 0x7f23a0154220 2023-12-19T18:16:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.746+0000 7f239f7fe640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a0154220 0x7f23a00b1e80 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f239800c3c0 tx=0x7f239800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.746+0000 7f2397fff640 1 -- 172.21.15.26:0/1219258528 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398005cf0 con 0x7f23a0154220 2023-12-19T18:16:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.746+0000 7f2397fff640 1 -- 172.21.15.26:0/1219258528 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2398005e50 con 0x7f23a0154220 2023-12-19T18:16:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.746+0000 7f2397fff640 1 -- 172.21.15.26:0/1219258528 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2398007310 con 0x7f23a0154220 2023-12-19T18:16:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.746+0000 7f23a5620640 1 -- 172.21.15.26:0/1219258528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a014d800 con 0x7f23a0154220 2023-12-19T18:16:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.746+0000 7f23a5620640 1 -- 172.21.15.26:0/1219258528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a014ddc0 con 0x7f23a0154220 2023-12-19T18:16:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.746+0000 7f23a5620640 1 -- 172.21.15.26:0/1219258528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a014dfb0 con 0x7f23a0154220 2023-12-19T18:16:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.746+0000 7f2397fff640 1 -- 172.21.15.26:0/1219258528 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2398007470 con 0x7f23a0154220 2023-12-19T18:16:40.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.746+0000 7f2397fff640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2380065bb0 0x7f2380068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:40.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.746+0000 7f2397fff640 1 -- 172.21.15.26:0/1219258528 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f2398082ac0 con 0x7f23a0154220 2023-12-19T18:16:40.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.750+0000 7f239effd640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2380065bb0 0x7f2380068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:40.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.750+0000 7f239effd640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2380065bb0 0x7f2380068070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23a0157410 tx=0x7f2390009890 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:40.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.754+0000 7f2397fff640 1 -- 172.21.15.26:0/1219258528 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f239804fdc0 con 0x7f23a0154220 2023-12-19T18:16:40.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.910+0000 7f23a5620640 1 -- 172.21.15.26:0/1219258528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23a0074270 con 0x7f23a0154220 2023-12-19T18:16:40.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.914+0000 7f2397fff640 1 -- 172.21.15.26:0/1219258528 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+21002 (secure 0 0 0) 0x7f2398017090 con 0x7f23a0154220 2023-12-19T18:16:40.916 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:40.916 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:40.537231+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:25.863942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:26.798049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:16:38.226152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:16:39.540217+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.914+0000 7f23a5620640 1 -- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2380065bb0 msgr2=0x7f2380068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.914+0000 7f23a5620640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2380065bb0 0x7f2380068070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23a0157410 tx=0x7f2390009890 comp rx=0 tx=0).stop 2023-12-19T18:16:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.914+0000 7f23a5620640 1 -- 172.21.15.26:0/1219258528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a0154220 msgr2=0x7f23a00b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.914+0000 7f23a5620640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a0154220 0x7f23a00b1e80 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f239800c3c0 tx=0x7f239800c880 comp rx=0 tx=0).stop 2023-12-19T18:16:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.914+0000 7f23a5620640 1 -- 172.21.15.26:0/1219258528 shutdown_connections 2023-12-19T18:16:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.914+0000 7f23a5620640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2380065bb0 0x7f2380068070 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.914+0000 7f23a5620640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23a0154220 0x7f23a00b1e80 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.914+0000 7f23a5620640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23a00cec40 0x7f23a0070290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.914+0000 7f23a5620640 1 --2- 172.21.15.26:0/1219258528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23a006b3d0 0x7f23a014c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.914+0000 7f23a5620640 1 -- 172.21.15.26:0/1219258528 >> 172.21.15.26:0/1219258528 conn(0x7f23a00658e0 msgr2=0x7f23a01531d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.914+0000 7f23a5620640 1 -- 172.21.15.26:0/1219258528 shutdown_connections 2023-12-19T18:16:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:40.914+0000 7f23a5620640 1 -- 172.21.15.26:0/1219258528 wait complete. 2023-12-19T18:16:40.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.078+0000 7fd615325640 1 Processor -- start 2023-12-19T18:16:41.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.078+0000 7fd615325640 1 -- start start 2023-12-19T18:16:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd615325640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd610155030 0x7fd61015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd615325640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd61015f370 0x7fd61006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd615325640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6100ce740 0x7fd6100cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd615325640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6100cf0f0 con 0x7fd610155030 2023-12-19T18:16:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd615325640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61006c970 con 0x7fd61015f370 2023-12-19T18:16:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd615325640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61015b1f0 con 0x7fd6100ce740 2023-12-19T18:16:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd60effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd610155030 0x7fd61015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd60effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd610155030 0x7fd61015bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39436/0 (socket says 172.21.15.26:39436) 2023-12-19T18:16:41.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd60effd640 1 -- 172.21.15.26:0/3991548088 learned_addr learned my addr 172.21.15.26:0/3991548088 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd607fff640 1 --2- 172.21.15.26:0/3991548088 >> v2:172.21.15.160:3300/0 conn(0x7fd61015f370 0x7fd61006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd60f7fe640 1 --2- 172.21.15.26:0/3991548088 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6100ce740 0x7fd6100cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd60effd640 1 -- 172.21.15.26:0/3991548088 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6100ce740 msgr2=0x7fd6100cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd60effd640 1 --2- 172.21.15.26:0/3991548088 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6100ce740 0x7fd6100cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd60effd640 1 -- 172.21.15.26:0/3991548088 >> v2:172.21.15.160:3300/0 conn(0x7fd61015f370 msgr2=0x7fd61006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:41.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd60effd640 1 --2- 172.21.15.26:0/3991548088 >> v2:172.21.15.160:3300/0 conn(0x7fd61015f370 0x7fd61006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.082+0000 7fd60effd640 1 -- 172.21.15.26:0/3991548088 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61015b4d0 con 0x7fd610155030 2023-12-19T18:16:41.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd60effd640 1 --2- 172.21.15.26:0/3991548088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd610155030 0x7fd61015bcd0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd6100b1780 tx=0x7fd60001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c674ca00a12401b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:41.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd60e7fc640 1 -- 172.21.15.26:0/3991548088 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6000aa610 con 0x7fd610155030 2023-12-19T18:16:41.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd60e7fc640 1 -- 172.21.15.26:0/3991548088 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6000a6040 con 0x7fd610155030 2023-12-19T18:16:41.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd60e7fc640 1 -- 172.21.15.26:0/3991548088 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6000aa990 con 0x7fd610155030 2023-12-19T18:16:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd615325640 1 -- 172.21.15.26:0/3991548088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd610155030 msgr2=0x7fd61015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd615325640 1 --2- 172.21.15.26:0/3991548088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd610155030 0x7fd61015bcd0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd6100b1780 tx=0x7fd60001c4d0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd615325640 1 -- 172.21.15.26:0/3991548088 shutdown_connections 2023-12-19T18:16:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd615325640 1 --2- 172.21.15.26:0/3991548088 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd6100ce740 0x7fd6100cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd615325640 1 --2- 172.21.15.26:0/3991548088 >> v2:172.21.15.160:3300/0 conn(0x7fd61015f370 0x7fd61006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd615325640 1 --2- 172.21.15.26:0/3991548088 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd610155030 0x7fd61015bcd0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd615325640 1 -- 172.21.15.26:0/3991548088 >> 172.21.15.26:0/3991548088 conn(0x7fd6100658e0 msgr2=0x7fd610065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd615325640 1 -- 172.21.15.26:0/3991548088 shutdown_connections 2023-12-19T18:16:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd615325640 1 -- 172.21.15.26:0/3991548088 wait complete. 2023-12-19T18:16:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd615325640 1 Processor -- start 2023-12-19T18:16:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.086+0000 7fd615325640 1 -- start start 2023-12-19T18:16:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd615325640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6100ce740 0x7fd61015dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd615325640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd610155030 0x7fd6100b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd615325640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd61015f370 0x7fd61013ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd615325640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6101443d0 con 0x7fd6100ce740 2023-12-19T18:16:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd615325640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61015c3e0 con 0x7fd61015f370 2023-12-19T18:16:41.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd615325640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd610143e10 con 0x7fd610155030 2023-12-19T18:16:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd60effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6100ce740 0x7fd61015dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd60effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6100ce740 0x7fd61015dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39446/0 (socket says 172.21.15.26:39446) 2023-12-19T18:16:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd60effd640 1 -- 172.21.15.26:0/2669932938 learned_addr learned my addr 172.21.15.26:0/2669932938 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd607fff640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd610155030 0x7fd6100b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd60f7fe640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd61015f370 0x7fd61013ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd60effd640 1 -- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd610155030 msgr2=0x7fd6100b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd60effd640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd610155030 0x7fd6100b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd60effd640 1 -- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd61015f370 msgr2=0x7fd61013ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd60effd640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd61015f370 0x7fd61013ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd60effd640 1 -- 172.21.15.26:0/2669932938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6000094d0 con 0x7fd6100ce740 2023-12-19T18:16:41.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd60effd640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6100ce740 0x7fd61015dd60 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fd6000ac040 tx=0x7fd60001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:41.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd6077fe640 1 -- 172.21.15.26:0/2669932938 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6000bd5f0 con 0x7fd6100ce740 2023-12-19T18:16:41.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd6077fe640 1 -- 172.21.15.26:0/2669932938 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6000bd770 con 0x7fd6100ce740 2023-12-19T18:16:41.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd6077fe640 1 -- 172.21.15.26:0/2669932938 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6000bdb20 con 0x7fd6100ce740 2023-12-19T18:16:41.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd615325640 1 -- 172.21.15.26:0/2669932938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd61006c740 con 0x7fd6100ce740 2023-12-19T18:16:41.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd615325640 1 -- 172.21.15.26:0/2669932938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd61015c000 con 0x7fd6100ce740 2023-12-19T18:16:41.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.090+0000 7fd615325640 1 -- 172.21.15.26:0/2669932938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd61015be60 con 0x7fd6100ce740 2023-12-19T18:16:41.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.094+0000 7fd6077fe640 1 -- 172.21.15.26:0/2669932938 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd6000b33f0 con 0x7fd6100ce740 2023-12-19T18:16:41.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.094+0000 7fd6077fe640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd5f0065b80 0x7fd5f0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.094+0000 7fd6077fe640 1 -- 172.21.15.26:0/2669932938 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fd60011e700 con 0x7fd6100ce740 2023-12-19T18:16:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.094+0000 7fd607fff640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd5f0065b80 0x7fd5f0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.094+0000 7fd607fff640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd5f0065b80 0x7fd5f0068040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd610071410 tx=0x7fd5f8008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:41.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.098+0000 7fd6077fe640 1 -- 172.21.15.26:0/2669932938 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd61015be60 con 0x7fd6100ce740 2023-12-19T18:16:41.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.258+0000 7fd615325640 1 -- 172.21.15.26:0/2669932938 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd61015be60 con 0x7fd6100ce740 2023-12-19T18:16:41.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.258+0000 7fd6077fe640 1 -- 172.21.15.26:0/2669932938 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+21002 (secure 0 0 0) 0x7fd61015be60 con 0x7fd6100ce740 2023-12-19T18:16:41.261 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:41.261 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":215,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:40.537231+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:25.863942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:26.798049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:16:38.226152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:16:39.540217+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:41.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.258+0000 7fd615325640 1 -- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd5f0065b80 msgr2=0x7fd5f0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.258+0000 7fd615325640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd5f0065b80 0x7fd5f0068040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd610071410 tx=0x7fd5f8008040 comp rx=0 tx=0).stop 2023-12-19T18:16:41.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.258+0000 7fd615325640 1 -- 172.21.15.26:0/2669932938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6100ce740 msgr2=0x7fd61015dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.258+0000 7fd615325640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6100ce740 0x7fd61015dd60 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fd6000ac040 tx=0x7fd60001ad10 comp rx=0 tx=0).stop 2023-12-19T18:16:41.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.258+0000 7fd615325640 1 -- 172.21.15.26:0/2669932938 shutdown_connections 2023-12-19T18:16:41.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.258+0000 7fd615325640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd5f0065b80 0x7fd5f0068040 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.258+0000 7fd615325640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd61015f370 0x7fd61013ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.258+0000 7fd615325640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd610155030 0x7fd6100b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.258+0000 7fd615325640 1 --2- 172.21.15.26:0/2669932938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd6100ce740 0x7fd61015dd60 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.258+0000 7fd615325640 1 -- 172.21.15.26:0/2669932938 >> 172.21.15.26:0/2669932938 conn(0x7fd6100658e0 msgr2=0x7fd6101543e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:41.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.258+0000 7fd615325640 1 -- 172.21.15.26:0/2669932938 shutdown_connections 2023-12-19T18:16:41.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.262+0000 7fd615325640 1 -- 172.21.15.26:0/2669932938 wait complete. 2023-12-19T18:16:41.283 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-12-19T18:16:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.438+0000 7f02c3f39640 1 Processor -- start 2023-12-19T18:16:41.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.438+0000 7f02c3f39640 1 -- start start 2023-12-19T18:16:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c3f39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f02bc161230 0x7f02bc0ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c3f39640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f02bc06c3f0 0x7f02bc06c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c3f39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02bc0b01b0 con 0x7f02bc164e50 2023-12-19T18:16:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c3f39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02bc0c9e00 con 0x7f02bc161230 2023-12-19T18:16:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c3f39640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02bc0adfb0 con 0x7f02bc06c3f0 2023-12-19T18:16:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c1cd5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f02bc161230 0x7f02bc0ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c1cd5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f02bc161230 0x7f02bc0ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39460/0 (socket says 172.21.15.26:39460) 2023-12-19T18:16:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c1cd5640 1 -- 172.21.15.26:0/2978163161 learned_addr learned my addr 172.21.15.26:0/2978163161 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c14d4640 1 --2- 172.21.15.26:0/2978163161 >> v2:172.21.15.160:3300/0 conn(0x7f02bc06c3f0 0x7f02bc06c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c1cd5640 1 -- 172.21.15.26:0/2978163161 >> v2:172.21.15.160:3300/0 conn(0x7f02bc06c3f0 msgr2=0x7f02bc06c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:41.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c1cd5640 1 --2- 172.21.15.26:0/2978163161 >> v2:172.21.15.160:3300/0 conn(0x7f02bc06c3f0 0x7f02bc06c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c1cd5640 1 -- 172.21.15.26:0/2978163161 >> v1:172.21.15.160:6789/0 conn(0x7f02bc164e50 legacy=0x7f02bc1652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c1cd5640 1 -- 172.21.15.26:0/2978163161 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc0ca4e0 con 0x7f02bc161230 2023-12-19T18:16:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c14d4640 1 --2- 172.21.15.26:0/2978163161 >> v2:172.21.15.160:3300/0 conn(0x7f02bc06c3f0 0x7f02bc06c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:41.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c1cd5640 1 --2- 172.21.15.26:0/2978163161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc161230 0x7f02bc0ada70 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0ca0f0 tx=0x7f02b801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6d6602bed686d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.442+0000 7f02c0cd3640 1 -- 172.21.15.26:0/2978163161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b80ab600 con 0x7f02bc161230 2023-12-19T18:16:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c0cd3640 1 -- 172.21.15.26:0/2978163161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b80ab760 con 0x7f02bc161230 2023-12-19T18:16:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c0cd3640 1 -- 172.21.15.26:0/2978163161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b80abaf0 con 0x7f02bc161230 2023-12-19T18:16:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 -- 172.21.15.26:0/2978163161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc161230 msgr2=0x7f02bc0ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 --2- 172.21.15.26:0/2978163161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc161230 0x7f02bc0ada70 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0ca0f0 tx=0x7f02b801c400 comp rx=0 tx=0).stop 2023-12-19T18:16:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 -- 172.21.15.26:0/2978163161 shutdown_connections 2023-12-19T18:16:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 --2- 172.21.15.26:0/2978163161 >> v2:172.21.15.160:3300/0 conn(0x7f02bc06c3f0 0x7f02bc06c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 --2- 172.21.15.26:0/2978163161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc161230 0x7f02bc0ada70 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 -- 172.21.15.26:0/2978163161 >> 172.21.15.26:0/2978163161 conn(0x7f02bc0ce730 msgr2=0x7f02bc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 -- 172.21.15.26:0/2978163161 shutdown_connections 2023-12-19T18:16:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 -- 172.21.15.26:0/2978163161 wait complete. 2023-12-19T18:16:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 Processor -- start 2023-12-19T18:16:41.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 -- start start 2023-12-19T18:16:41.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02bc06c3f0 0x7f02bc14e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc161230 0x7f02bc0c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02bc164e50 0x7f02bc1597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0c48a0 con 0x7f02bc161230 2023-12-19T18:16:41.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc14ea50 con 0x7f02bc164e50 2023-12-19T18:16:41.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c3f39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02bc0c3c20 con 0x7f02bc06c3f0 2023-12-19T18:16:41.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c14d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc161230 0x7f02bc0c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c14d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc161230 0x7f02bc0c3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39476/0 (socket says 172.21.15.26:39476) 2023-12-19T18:16:41.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.446+0000 7f02c14d4640 1 -- 172.21.15.26:0/511764905 learned_addr learned my addr 172.21.15.26:0/511764905 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:41.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02c1cd5640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02bc06c3f0 0x7f02bc14e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02c24d6640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02bc164e50 0x7f02bc1597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02c14d4640 1 -- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02bc06c3f0 msgr2=0x7f02bc14e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02c14d4640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02bc06c3f0 0x7f02bc14e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02c14d4640 1 -- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02bc164e50 msgr2=0x7f02bc1597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02c14d4640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02bc164e50 0x7f02bc1597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02c14d4640 1 -- 172.21.15.26:0/511764905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b80094d0 con 0x7f02bc161230 2023-12-19T18:16:41.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02c24d6640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02bc164e50 0x7f02bc1597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:41.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02c1cd5640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02bc06c3f0 0x7f02bc14e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:41.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02c14d4640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc161230 0x7f02bc0c3310 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00c7f0 tx=0x7f02ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:41.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02b27fc640 1 -- 172.21.15.26:0/511764905 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac007820 con 0x7f02bc161230 2023-12-19T18:16:41.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02b27fc640 1 -- 172.21.15.26:0/511764905 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ac007980 con 0x7f02bc161230 2023-12-19T18:16:41.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02b27fc640 1 -- 172.21.15.26:0/511764905 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac007d30 con 0x7f02bc161230 2023-12-19T18:16:41.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02c3f39640 1 -- 172.21.15.26:0/511764905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02bc0c3f00 con 0x7f02bc161230 2023-12-19T18:16:41.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02c3f39640 1 -- 172.21.15.26:0/511764905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02bc0c44c0 con 0x7f02bc161230 2023-12-19T18:16:41.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02c3f39640 1 -- 172.21.15.26:0/511764905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02bc074270 con 0x7f02bc161230 2023-12-19T18:16:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02b27fc640 1 -- 172.21.15.26:0/511764905 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f02ac0113f0 con 0x7f02bc161230 2023-12-19T18:16:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02b27fc640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f02a0065b80 0x7f02a0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.450+0000 7f02b27fc640 1 -- 172.21.15.26:0/511764905 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f02ac083d70 con 0x7f02bc161230 2023-12-19T18:16:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.454+0000 7f02c1cd5640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f02a0065b80 0x7f02a0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.454+0000 7f02c1cd5640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f02a0065b80 0x7f02a0068040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f02b80af000 tx=0x7f02b80a7000 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:41.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.454+0000 7f02b27fc640 1 -- 172.21.15.26:0/511764905 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f02ac050ea0 con 0x7f02bc161230 2023-12-19T18:16:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:41.650+0000 7f02c3f39640 1 -- 172.21.15.26:0/511764905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f02bc0b46b0 con 0x7f02bc161230 2023-12-19T18:16:42.395 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:16:42.399 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:16:42.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.546+0000 7f02b27fc640 1 -- 172.21.15.26:0/511764905 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v101) v1 ==== 121+0+0 (secure 0 0 0) 0x7f02ac0544c0 con 0x7f02bc161230 2023-12-19T18:16:42.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.550+0000 7f02c3f39640 1 -- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f02a0065b80 msgr2=0x7f02a0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:42.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.550+0000 7f02c3f39640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f02a0065b80 0x7f02a0068040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f02b80af000 tx=0x7f02b80a7000 comp rx=0 tx=0).stop 2023-12-19T18:16:42.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.550+0000 7f02c3f39640 1 -- 172.21.15.26:0/511764905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc161230 msgr2=0x7f02bc0c3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:42.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.550+0000 7f02c3f39640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc161230 0x7f02bc0c3310 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00c7f0 tx=0x7f02ac00ccb0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.550+0000 7f02c3f39640 1 -- 172.21.15.26:0/511764905 shutdown_connections 2023-12-19T18:16:42.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.550+0000 7f02c3f39640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f02a0065b80 0x7f02a0068040 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.550+0000 7f02c3f39640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02bc164e50 0x7f02bc1597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.550+0000 7f02c3f39640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02bc161230 0x7f02bc0c3310 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.550+0000 7f02c3f39640 1 --2- 172.21.15.26:0/511764905 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f02bc06c3f0 0x7f02bc14e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.550+0000 7f02c3f39640 1 -- 172.21.15.26:0/511764905 >> 172.21.15.26:0/511764905 conn(0x7f02bc0ce730 msgr2=0x7f02bc0c9680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:42.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.550+0000 7f02c3f39640 1 -- 172.21.15.26:0/511764905 shutdown_connections 2023-12-19T18:16:42.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.550+0000 7f02c3f39640 1 -- 172.21.15.26:0/511764905 wait complete. 2023-12-19T18:16:42.576 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:16:42.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-12-19T18:16:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.722+0000 7fe5b116d640 1 Processor -- start 2023-12-19T18:16:42.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.722+0000 7fe5b116d640 1 -- start start 2023-12-19T18:16:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5b116d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5ac159820 0x7fe5ac15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5b116d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe5ac06f6c0 0x7fe5ac0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5b116d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5ac159c00 0x7fe5ac070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5b116d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac15e470 con 0x7fe5ac159820 2023-12-19T18:16:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5b116d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac06fb60 con 0x7fe5ac06f6c0 2023-12-19T18:16:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5b116d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac06fe40 con 0x7fe5ac159c00 2023-12-19T18:16:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5aad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5ac159820 0x7fe5ac15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5aad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5ac159820 0x7fe5ac15df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39484/0 (socket says 172.21.15.26:39484) 2023-12-19T18:16:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5aad9d640 1 -- 172.21.15.26:0/3526113072 learned_addr learned my addr 172.21.15.26:0/3526113072 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5aa59c640 1 --2- 172.21.15.26:0/3526113072 >> v2:172.21.15.160:3300/0 conn(0x7fe5ac06f6c0 0x7fe5ac0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:42.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5ab59e640 1 --2- 172.21.15.26:0/3526113072 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5ac159c00 0x7fe5ac070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:42.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5aad9d640 1 -- 172.21.15.26:0/3526113072 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5ac159c00 msgr2=0x7fe5ac070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:42.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5aad9d640 1 --2- 172.21.15.26:0/3526113072 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5ac159c00 0x7fe5ac070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5aad9d640 1 -- 172.21.15.26:0/3526113072 >> v2:172.21.15.160:3300/0 conn(0x7fe5ac06f6c0 msgr2=0x7fe5ac0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:42.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5aad9d640 1 --2- 172.21.15.26:0/3526113072 >> v2:172.21.15.160:3300/0 conn(0x7fe5ac06f6c0 0x7fe5ac0c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5aad9d640 1 -- 172.21.15.26:0/3526113072 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac070120 con 0x7fe5ac159820 2023-12-19T18:16:42.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5aa59c640 1 --2- 172.21.15.26:0/3526113072 >> v2:172.21.15.160:3300/0 conn(0x7fe5ac06f6c0 0x7fe5ac0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:42.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5ab59e640 1 --2- 172.21.15.26:0/3526113072 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5ac159c00 0x7fe5ac070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:42.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.726+0000 7fe5aad9d640 1 --2- 172.21.15.26:0/3526113072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5ac159820 0x7fe5ac15df30 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0caf30 tx=0x7fe59c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f026a447313f99fe server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5a9d9b640 1 -- 172.21.15.26:0/3526113072 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0aa5f0 con 0x7fe5ac159820 2023-12-19T18:16:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5a9d9b640 1 -- 172.21.15.26:0/3526113072 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59c0aa750 con 0x7fe5ac159820 2023-12-19T18:16:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5a9d9b640 1 -- 172.21.15.26:0/3526113072 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0aaae0 con 0x7fe5ac159820 2023-12-19T18:16:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 -- 172.21.15.26:0/3526113072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5ac159820 msgr2=0x7fe5ac15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 --2- 172.21.15.26:0/3526113072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5ac159820 0x7fe5ac15df30 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0caf30 tx=0x7fe59c01c4d0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 -- 172.21.15.26:0/3526113072 shutdown_connections 2023-12-19T18:16:42.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 --2- 172.21.15.26:0/3526113072 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5ac159c00 0x7fe5ac070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 --2- 172.21.15.26:0/3526113072 >> v2:172.21.15.160:3300/0 conn(0x7fe5ac06f6c0 0x7fe5ac0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 --2- 172.21.15.26:0/3526113072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5ac159820 0x7fe5ac15df30 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 -- 172.21.15.26:0/3526113072 >> 172.21.15.26:0/3526113072 conn(0x7fe5ac0ce730 msgr2=0x7fe5ac0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 -- 172.21.15.26:0/3526113072 shutdown_connections 2023-12-19T18:16:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 -- 172.21.15.26:0/3526113072 wait complete. 2023-12-19T18:16:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 Processor -- start 2023-12-19T18:16:42.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 -- start start 2023-12-19T18:16:42.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5ac06f6c0 0x7fe5ac1f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:42.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5ac159820 0x7fe5ac0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5ac159c00 0x7fe5ac15c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac15ab90 con 0x7fe5ac06f6c0 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac15ce90 con 0x7fe5ac159c00 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.730+0000 7fe5b116d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac0b4120 con 0x7fe5ac159820 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5ac06f6c0 0x7fe5ac1f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5ac06f6c0 0x7fe5ac1f88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39496/0 (socket says 172.21.15.26:39496) 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5aad9d640 1 -- 172.21.15.26:0/3956217023 learned_addr learned my addr 172.21.15.26:0/3956217023 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5aa59c640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5ac159820 0x7fe5ac0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5ab59e640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5ac159c00 0x7fe5ac15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5aad9d640 1 -- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5ac159820 msgr2=0x7fe5ac0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5aad9d640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5ac159820 0x7fe5ac0b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5aad9d640 1 -- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5ac159c00 msgr2=0x7fe5ac15c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5aad9d640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5ac159c00 0x7fe5ac15c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5aad9d640 1 -- 172.21.15.26:0/3956217023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c0094d0 con 0x7fe5ac06f6c0 2023-12-19T18:16:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5aad9d640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5ac06f6c0 0x7fe5ac1f88a0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fe59c01ab80 tx=0x7fe59c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5937fe640 1 -- 172.21.15.26:0/3956217023 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0b3800 con 0x7fe5ac06f6c0 2023-12-19T18:16:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5937fe640 1 -- 172.21.15.26:0/3956217023 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59c0b3960 con 0x7fe5ac06f6c0 2023-12-19T18:16:42.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5937fe640 1 -- 172.21.15.26:0/3956217023 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0b3cf0 con 0x7fe5ac06f6c0 2023-12-19T18:16:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5b116d640 1 -- 172.21.15.26:0/3956217023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ac06b880 con 0x7fe5ac06f6c0 2023-12-19T18:16:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5b116d640 1 -- 172.21.15.26:0/3956217023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac0b3b60 con 0x7fe5ac06f6c0 2023-12-19T18:16:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5b116d640 1 -- 172.21.15.26:0/3956217023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ac074270 con 0x7fe5ac06f6c0 2023-12-19T18:16:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5937fe640 1 -- 172.21.15.26:0/3956217023 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe59c0b2380 con 0x7fe5ac06f6c0 2023-12-19T18:16:42.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.734+0000 7fe5937fe640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe598065bb0 0x7fe598068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:42.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.738+0000 7fe5937fe640 1 -- 172.21.15.26:0/3956217023 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fe59c0a7070 con 0x7fe5ac06f6c0 2023-12-19T18:16:42.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.738+0000 7fe5aa59c640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe598065bb0 0x7fe598068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:42.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.738+0000 7fe5aa59c640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe598065bb0 0x7fe598068070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac158700 tx=0x7fe5a0008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:42.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.742+0000 7fe5937fe640 1 -- 172.21.15.26:0/3956217023 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe59c0eb7f0 con 0x7fe5ac06f6c0 2023-12-19T18:16:42.862 INFO:tasks.ceph.mds.d.smithi160.stderr:2023-12-19T18:16:42.857+0000 7f2d85203640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:16:42.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:42.930+0000 7fe5b116d640 1 -- 172.21.15.26:0/3956217023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe5ac0b46b0 con 0x7fe5ac06f6c0 2023-12-19T18:16:43.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.550+0000 7fe5937fe640 1 -- 172.21.15.26:0/3956217023 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v102) v1 ==== 125+0+0 (secure 0 0 0) 0x7fe59c0eee10 con 0x7fe5ac06f6c0 2023-12-19T18:16:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.554+0000 7fe5b116d640 1 -- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe598065bb0 msgr2=0x7fe598068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.554+0000 7fe5b116d640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe598065bb0 0x7fe598068070 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac158700 tx=0x7fe5a0008040 comp rx=0 tx=0).stop 2023-12-19T18:16:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.554+0000 7fe5b116d640 1 -- 172.21.15.26:0/3956217023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5ac06f6c0 msgr2=0x7fe5ac1f88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.554+0000 7fe5b116d640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5ac06f6c0 0x7fe5ac1f88a0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fe59c01ab80 tx=0x7fe59c01ad10 comp rx=0 tx=0).stop 2023-12-19T18:16:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.554+0000 7fe5b116d640 1 -- 172.21.15.26:0/3956217023 shutdown_connections 2023-12-19T18:16:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.554+0000 7fe5b116d640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe598065bb0 0x7fe598068070 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.554+0000 7fe5b116d640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5ac159c00 0x7fe5ac15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.554+0000 7fe5b116d640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe5ac159820 0x7fe5ac0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.554+0000 7fe5b116d640 1 --2- 172.21.15.26:0/3956217023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5ac06f6c0 0x7fe5ac1f88a0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.554+0000 7fe5b116d640 1 -- 172.21.15.26:0/3956217023 >> 172.21.15.26:0/3956217023 conn(0x7fe5ac0ce730 msgr2=0x7fe5ac0c97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.554+0000 7fe5b116d640 1 -- 172.21.15.26:0/3956217023 shutdown_connections 2023-12-19T18:16:43.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.554+0000 7fe5b116d640 1 -- 172.21.15.26:0/3956217023 wait complete. 2023-12-19T18:16:43.576 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-12-19T18:16:43.576 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:43.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d6841f640 1 Processor -- start 2023-12-19T18:16:43.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d6841f640 1 -- start start 2023-12-19T18:16:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d6841f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7d600c9620 0x7f7d600c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d6841f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d600cefa0 0x7f7d600cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d6841f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d600cf930 con 0x7f7d60159260 2023-12-19T18:16:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d6841f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d600cba80 con 0x7f7d600cefa0 2023-12-19T18:16:43.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d6841f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d60159640 con 0x7f7d600c9620 2023-12-19T18:16:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d669bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d600cefa0 0x7f7d600cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d669bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d600cefa0 0x7f7d600cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39500/0 (socket says 172.21.15.26:39500) 2023-12-19T18:16:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d669bc640 1 -- 172.21.15.26:0/253204773 learned_addr learned my addr 172.21.15.26:0/253204773 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d669bc640 1 -- 172.21.15.26:0/253204773 >> v2:172.21.15.160:3300/0 conn(0x7f7d600c9620 msgr2=0x7f7d600c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d659ba640 1 --2- 172.21.15.26:0/253204773 >> v2:172.21.15.160:3300/0 conn(0x7f7d600c9620 0x7f7d600c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d669bc640 1 --2- 172.21.15.26:0/253204773 >> v2:172.21.15.160:3300/0 conn(0x7f7d600c9620 0x7f7d600c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d669bc640 1 -- 172.21.15.26:0/253204773 >> v1:172.21.15.26:6789/0 conn(0x7f7d60159260 legacy=0x7f7d600c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d669bc640 1 -- 172.21.15.26:0/253204773 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d600c4be0 con 0x7f7d600cefa0 2023-12-19T18:16:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d659ba640 1 --2- 172.21.15.26:0/253204773 >> v2:172.21.15.160:3300/0 conn(0x7f7d600c9620 0x7f7d600c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:43.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.718+0000 7f7d651b9640 1 -- 172.21.15.26:0/253204773 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065684059 0 0) 0x7f7d5c001190 con 0x7f7d60159260 2023-12-19T18:16:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.722+0000 7f7d669bc640 1 --2- 172.21.15.26:0/253204773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d600cefa0 0x7f7d600cf3f0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f7d6015cd50 tx=0x7f7d5801ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d1b64526e55143 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.722+0000 7f7d651b9640 1 -- 172.21.15.26:0/253204773 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d580a7c00 con 0x7f7d600cefa0 2023-12-19T18:16:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.722+0000 7f7d651b9640 1 -- 172.21.15.26:0/253204773 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d580ab070 con 0x7f7d600cefa0 2023-12-19T18:16:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.722+0000 7f7d651b9640 1 -- 172.21.15.26:0/253204773 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d580b1400 con 0x7f7d600cefa0 2023-12-19T18:16:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.722+0000 7f7d6841f640 1 -- 172.21.15.26:0/253204773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d600cefa0 msgr2=0x7f7d600cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:43.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.722+0000 7f7d6841f640 1 --2- 172.21.15.26:0/253204773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d600cefa0 0x7f7d600cf3f0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f7d6015cd50 tx=0x7f7d5801ebb0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.722+0000 7f7d6841f640 1 -- 172.21.15.26:0/253204773 shutdown_connections 2023-12-19T18:16:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.722+0000 7f7d6841f640 1 --2- 172.21.15.26:0/253204773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d600cefa0 0x7f7d600cf3f0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.722+0000 7f7d6841f640 1 --2- 172.21.15.26:0/253204773 >> v2:172.21.15.160:3300/0 conn(0x7f7d600c9620 0x7f7d600c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.722+0000 7f7d6841f640 1 -- 172.21.15.26:0/253204773 >> 172.21.15.26:0/253204773 conn(0x7f7d6006d5d0 msgr2=0x7f7d60069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.722+0000 7f7d6841f640 1 -- 172.21.15.26:0/253204773 shutdown_connections 2023-12-19T18:16:43.726 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.722+0000 7f7d6841f640 1 -- 172.21.15.26:0/253204773 wait complete. 2023-12-19T18:16:43.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d6841f640 1 Processor -- start 2023-12-19T18:16:43.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d6841f640 1 -- start start 2023-12-19T18:16:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d6841f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d600c9620 0x7f7d600af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d6841f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d600cefa0 0x7f7d6014b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d6841f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d60159260 0x7f7d6014beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d6841f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6014d5f0 con 0x7f7d60159260 2023-12-19T18:16:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d6841f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6014af60 con 0x7f7d600c9620 2023-12-19T18:16:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d6841f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6014c550 con 0x7f7d600cefa0 2023-12-19T18:16:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d669bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d60159260 0x7f7d6014beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d669bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d60159260 0x7f7d6014beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39508/0 (socket says 172.21.15.26:39508) 2023-12-19T18:16:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d669bc640 1 -- 172.21.15.26:0/657041188 learned_addr learned my addr 172.21.15.26:0/657041188 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d661bb640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d600c9620 0x7f7d600af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d659ba640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d600cefa0 0x7f7d6014b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d669bc640 1 -- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d600cefa0 msgr2=0x7f7d6014b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d669bc640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d600cefa0 0x7f7d6014b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d669bc640 1 -- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d600c9620 msgr2=0x7f7d600af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d669bc640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d600c9620 0x7f7d600af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d669bc640 1 -- 172.21.15.26:0/657041188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5800bd40 con 0x7f7d60159260 2023-12-19T18:16:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d659ba640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d600cefa0 0x7f7d6014b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d669bc640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d60159260 0x7f7d6014beb0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f7d580251a0 tx=0x7f7d5801d3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d56ffd640 1 -- 172.21.15.26:0/657041188 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d580039a0 con 0x7f7d60159260 2023-12-19T18:16:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d56ffd640 1 -- 172.21.15.26:0/657041188 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d58003b20 con 0x7f7d60159260 2023-12-19T18:16:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d6841f640 1 -- 172.21.15.26:0/657041188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6014c830 con 0x7f7d60159260 2023-12-19T18:16:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d56ffd640 1 -- 172.21.15.26:0/657041188 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58004030 con 0x7f7d60159260 2023-12-19T18:16:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d6841f640 1 -- 172.21.15.26:0/657041188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6014cdf0 con 0x7f7d60159260 2023-12-19T18:16:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.726+0000 7f7d6841f640 1 -- 172.21.15.26:0/657041188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d60074270 con 0x7f7d60159260 2023-12-19T18:16:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.730+0000 7f7d56ffd640 1 -- 172.21.15.26:0/657041188 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7d58004190 con 0x7f7d60159260 2023-12-19T18:16:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.730+0000 7f7d56ffd640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7d44065b80 0x7f7d44068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.730+0000 7f7d56ffd640 1 -- 172.21.15.26:0/657041188 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f7d5811e260 con 0x7f7d60159260 2023-12-19T18:16:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.730+0000 7f7d661bb640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7d44065b80 0x7f7d44068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.730+0000 7f7d661bb640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7d44065b80 0x7f7d44068040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d60163f80 tx=0x7f7d5c009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:43.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.734+0000 7f7d56ffd640 1 -- 172.21.15.26:0/657041188 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7d580eb410 con 0x7f7d60159260 2023-12-19T18:16:43.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.926+0000 7f7d6841f640 1 -- 172.21.15.26:0/657041188 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d600b1690 con 0x7f7d60159260 2023-12-19T18:16:43.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.930+0000 7f7d56ffd640 1 -- 172.21.15.26:0/657041188 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5917 (secure 0 0 0) 0x7f7d580eea30 con 0x7f7d60159260 2023-12-19T18:16:43.931 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:43.931 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3349262044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3349262044},{"type":"v1","addr":"172.21.15.160:6836","nonce":3349262044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3865658274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3865658274},{"type":"v1","addr":"172.21.15.26:6833","nonce":3865658274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:27.798614+0000","modified":"2023-12-19T18:16:31.076294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3981286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3981286992},{"type":"v1","addr":"172.21.15.26:6835","nonce":3981286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:40.537606+0000","modified":"2023-12-19T18:16:43.550176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10605":{"gid":10605,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.160:6837/1294752100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1294752100},{"type":"v1","addr":"172.21.15.160:6837","nonce":1294752100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-19T18:16:43.932 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 102 2023-12-19T18:16:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.930+0000 7f7d6841f640 1 -- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7d44065b80 msgr2=0x7f7d44068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.930+0000 7f7d6841f640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7d44065b80 0x7f7d44068040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d60163f80 tx=0x7f7d5c009040 comp rx=0 tx=0).stop 2023-12-19T18:16:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.930+0000 7f7d6841f640 1 -- 172.21.15.26:0/657041188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d60159260 msgr2=0x7f7d6014beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.930+0000 7f7d6841f640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d60159260 0x7f7d6014beb0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f7d580251a0 tx=0x7f7d5801d3f0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.930+0000 7f7d6841f640 1 -- 172.21.15.26:0/657041188 shutdown_connections 2023-12-19T18:16:43.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.930+0000 7f7d6841f640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7d44065b80 0x7f7d44068040 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.930+0000 7f7d6841f640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d60159260 0x7f7d6014beb0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.930+0000 7f7d6841f640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d600cefa0 0x7f7d6014b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.930+0000 7f7d6841f640 1 --2- 172.21.15.26:0/657041188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d600c9620 0x7f7d600af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.930+0000 7f7d6841f640 1 -- 172.21.15.26:0/657041188 >> 172.21.15.26:0/657041188 conn(0x7f7d6006d5d0 msgr2=0x7f7d6006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:43.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.930+0000 7f7d6841f640 1 -- 172.21.15.26:0/657041188 shutdown_connections 2023-12-19T18:16:43.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:43.930+0000 7f7d6841f640 1 -- 172.21.15.26:0/657041188 wait complete. 2023-12-19T18:16:43.953 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:44.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.094+0000 7fda11655640 1 Processor -- start 2023-12-19T18:16:44.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.094+0000 7fda11655640 1 -- start start 2023-12-19T18:16:44.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda11655640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda0c06c7f0 0x7fda0c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda11655640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0c06cc50 con 0x7fda0c0ccc20 2023-12-19T18:16:44.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda11655640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0c06ced0 con 0x7fda0c15c7f0 2023-12-19T18:16:44.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda11655640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c06d770 con 0x7fda0c06c7f0 2023-12-19T18:16:44.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fda0c0ccc20 0x7fda0c0b30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41400/0 (socket says 172.21.15.26:41400) 2023-12-19T18:16:44.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0b7fe640 1 -- 172.21.15.26:0/2453534888 learned_addr learned my addr 172.21.15.26:0/2453534888 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda027fc640 1 --2- 172.21.15.26:0/2453534888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda0c06c7f0 0x7fda0c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501814453 0 0) 0x7fda0c06cc50 con 0x7fda0c0ccc20 2023-12-19T18:16:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9f8003450 con 0x7fda0c0ccc20 2023-12-19T18:16:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883523411 0 0) 0x7fda0c06ced0 con 0x7fda0c15c7f0 2023-12-19T18:16:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda0c06cc50 con 0x7fda0c15c7f0 2023-12-19T18:16:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1571526140 0 0) 0x7fd9f8003450 con 0x7fda0c0ccc20 2023-12-19T18:16:44.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda0c06c7f0 msgr2=0x7fda0c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 --2- 172.21.15.26:0/2453534888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda0c06c7f0 0x7fda0c0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 >> v1:172.21.15.160:6789/0 conn(0x7fda0c15c7f0 legacy=0x7fda0c0adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c06d1b0 con 0x7fda0c0ccc20 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fda04001680 con 0x7fda0c0ccc20 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 >> v1:172.21.15.26:6789/0 conn(0x7fda0c0ccc20 legacy=0x7fda0c0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 --2- 172.21.15.26:0/2453534888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8003c30 0x7fd9f8008860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 --2- 172.21.15.26:0/2453534888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda0c06c7f0 0x7fd9f800adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 --2- 172.21.15.26:0/2453534888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9f800b2f0 0x7fd9f800d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c06d770 con 0x7fd9f8003c30 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f800dc20 con 0x7fd9f800b2f0 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f800dde0 con 0x7fda0c06c7f0 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0affd640 1 --2- 172.21.15.26:0/2453534888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8003c30 0x7fd9f8008860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda027fc640 1 --2- 172.21.15.26:0/2453534888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda0c06c7f0 0x7fd9f800adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0b7fe640 1 --2- 172.21.15.26:0/2453534888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9f800b2f0 0x7fd9f800d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0affd640 1 -- 172.21.15.26:0/2453534888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda0c06c7f0 msgr2=0x7fd9f800adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:44.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0affd640 1 --2- 172.21.15.26:0/2453534888 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda0c06c7f0 0x7fd9f800adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0affd640 1 -- 172.21.15.26:0/2453534888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9f800b2f0 msgr2=0x7fd9f800d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:44.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0affd640 1 --2- 172.21.15.26:0/2453534888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd9f800b2f0 0x7fd9f800d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.098+0000 7fda0affd640 1 -- 172.21.15.26:0/2453534888 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f8003450 con 0x7fd9f8003c30 2023-12-19T18:16:44.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.102+0000 7fda0affd640 1 --2- 172.21.15.26:0/2453534888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8003c30 0x7fd9f8008860 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0018d0 tx=0x7fd9fc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e173cb94eca98fd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:44.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.102+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc0052d0 con 0x7fd9f8003c30 2023-12-19T18:16:44.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.102+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc0a7050 con 0x7fd9f8003c30 2023-12-19T18:16:44.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.102+0000 7fda0a7fc640 1 -- 172.21.15.26:0/2453534888 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc003600 con 0x7fd9f8003c30 2023-12-19T18:16:44.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.102+0000 7fda11655640 1 -- 172.21.15.26:0/2453534888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8003c30 msgr2=0x7fd9f8008860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.102+0000 7fda11655640 1 --2- 172.21.15.26:0/2453534888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8003c30 0x7fd9f8008860 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0018d0 tx=0x7fd9fc01c560 comp rx=0 tx=0).stop 2023-12-19T18:16:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.102+0000 7fda0b7fe640 1 -- 172.21.15.26:0/2453534888 reap_dead start 2023-12-19T18:16:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.102+0000 7fda11655640 1 -- 172.21.15.26:0/2453534888 shutdown_connections 2023-12-19T18:16:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.102+0000 7fda11655640 1 -- 172.21.15.26:0/2453534888 >> 172.21.15.26:0/2453534888 conn(0x7fda0c0658e0 msgr2=0x7fda0c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.102+0000 7fda11655640 1 -- 172.21.15.26:0/2453534888 shutdown_connections 2023-12-19T18:16:44.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.102+0000 7fda11655640 1 -- 172.21.15.26:0/2453534888 wait complete. 2023-12-19T18:16:44.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda11655640 1 Processor -- start 2023-12-19T18:16:44.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda11655640 1 -- start start 2023-12-19T18:16:44.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda11655640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8003c30 0x7fda0c0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda11655640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda0c1e6a40 0x7fda0c141190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda11655640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda0c0b0640 0x7fda0c0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda11655640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c15dcd0 con 0x7fd9f8003c30 2023-12-19T18:16:44.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda11655640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c0b1000 con 0x7fda0c0b0640 2023-12-19T18:16:44.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda11655640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c15f750 con 0x7fda0c1e6a40 2023-12-19T18:16:44.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda0affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8003c30 0x7fda0c0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda0affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8003c30 0x7fda0c0c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39532/0 (socket says 172.21.15.26:39532) 2023-12-19T18:16:44.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda0affd640 1 -- 172.21.15.26:0/2539673132 learned_addr learned my addr 172.21.15.26:0/2539673132 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:44.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda027fc640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda0c1e6a40 0x7fda0c141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda0b7fe640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda0c0b0640 0x7fda0c0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda0affd640 1 -- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda0c1e6a40 msgr2=0x7fda0c141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda0affd640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda0c1e6a40 0x7fda0c141190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda0affd640 1 -- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda0c0b0640 msgr2=0x7fda0c0b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda0affd640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda0c0b0640 0x7fda0c0b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda0affd640 1 -- 172.21.15.26:0/2539673132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c15fa30 con 0x7fd9f8003c30 2023-12-19T18:16:44.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda0affd640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8003c30 0x7fda0c0c9cb0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0ad040 tx=0x7fd9f8003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:44.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda03fff640 1 -- 172.21.15.26:0/2539673132 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc0069e0 con 0x7fd9f8003c30 2023-12-19T18:16:44.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda03fff640 1 -- 172.21.15.26:0/2539673132 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc006b40 con 0x7fd9f8003c30 2023-12-19T18:16:44.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda03fff640 1 -- 172.21.15.26:0/2539673132 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc007030 con 0x7fd9f8003c30 2023-12-19T18:16:44.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda11655640 1 -- 172.21.15.26:0/2539673132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0c15d680 con 0x7fd9f8003c30 2023-12-19T18:16:44.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda11655640 1 -- 172.21.15.26:0/2539673132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0c1e6ec0 con 0x7fd9f8003c30 2023-12-19T18:16:44.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.106+0000 7fda11655640 1 -- 172.21.15.26:0/2539673132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0c074270 con 0x7fd9f8003c30 2023-12-19T18:16:44.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.110+0000 7fda03fff640 1 -- 172.21.15.26:0/2539673132 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd9fc007190 con 0x7fd9f8003c30 2023-12-19T18:16:44.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.110+0000 7fda03fff640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd9ec065bb0 0x7fd9ec068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.110+0000 7fda03fff640 1 -- 172.21.15.26:0/2539673132 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fd9fc11d080 con 0x7fd9f8003c30 2023-12-19T18:16:44.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.110+0000 7fda027fc640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd9ec065bb0 0x7fd9ec068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.110+0000 7fda027fc640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd9ec065bb0 0x7fd9ec068070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0ca540 tx=0x7fd9f400b040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:44.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.114+0000 7fda03fff640 1 -- 172.21.15.26:0/2539673132 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd9fc0ea1d0 con 0x7fd9f8003c30 2023-12-19T18:16:44.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.274+0000 7fda11655640 1 -- 172.21.15.26:0/2539673132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda0c070fd0 con 0x7fd9f8003c30 2023-12-19T18:16:44.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.274+0000 7fda03fff640 1 -- 172.21.15.26:0/2539673132 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v216) v1 ==== 74+0+20989 (secure 0 0 0) 0x7fd9fc0bf020 con 0x7fd9f8003c30 2023-12-19T18:16:44.278 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:44.278 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":216,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:41.541069+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:25.863942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:26.798049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:16:38.226152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:16:39.540217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:44.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.278+0000 7fda11655640 1 -- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd9ec065bb0 msgr2=0x7fd9ec068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.278+0000 7fda11655640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd9ec065bb0 0x7fd9ec068070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0ca540 tx=0x7fd9f400b040 comp rx=0 tx=0).stop 2023-12-19T18:16:44.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.278+0000 7fda11655640 1 -- 172.21.15.26:0/2539673132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8003c30 msgr2=0x7fda0c0c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.278+0000 7fda11655640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8003c30 0x7fda0c0c9cb0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0ad040 tx=0x7fd9f8003700 comp rx=0 tx=0).stop 2023-12-19T18:16:44.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.278+0000 7fda11655640 1 -- 172.21.15.26:0/2539673132 shutdown_connections 2023-12-19T18:16:44.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.278+0000 7fda11655640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd9ec065bb0 0x7fd9ec068070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.278+0000 7fda11655640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda0c0b0640 0x7fda0c0b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.278+0000 7fda11655640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fda0c1e6a40 0x7fda0c141190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.278+0000 7fda11655640 1 --2- 172.21.15.26:0/2539673132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9f8003c30 0x7fda0c0c9cb0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.278+0000 7fda11655640 1 -- 172.21.15.26:0/2539673132 >> 172.21.15.26:0/2539673132 conn(0x7fda0c0658e0 msgr2=0x7fda0c0b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:44.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.278+0000 7fda11655640 1 -- 172.21.15.26:0/2539673132 shutdown_connections 2023-12-19T18:16:44.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.278+0000 7fda11655640 1 -- 172.21.15.26:0/2539673132 wait complete. 2023-12-19T18:16:44.300 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:16:44.300 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:16:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43c697640 1 Processor -- start 2023-12-19T18:16:44.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43c697640 1 -- start start 2023-12-19T18:16:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43c697640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff43415cda0 0x7ff434160920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43c697640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff4340ce030 0x7ff4340d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43c697640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4340d0d40 con 0x7ff4340b85e0 2023-12-19T18:16:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43c697640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff434161150 con 0x7ff4340ce030 2023-12-19T18:16:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43c697640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff434161430 con 0x7ff43415cda0 2023-12-19T18:16:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43a433640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff43415cda0 0x7ff434160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43a433640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff43415cda0 0x7ff434160920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:56256/0 (socket says 172.21.15.26:56256) 2023-12-19T18:16:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43ac34640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff4340ce030 0x7ff4340d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.446 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43a433640 1 -- 172.21.15.26:0/2174937921 learned_addr learned my addr 172.21.15.26:0/2174937921 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43a433640 1 -- 172.21.15.26:0/2174937921 >> v2:172.21.15.160:3300/0 conn(0x7ff4340ce030 msgr2=0x7ff4340d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43a433640 1 --2- 172.21.15.26:0/2174937921 >> v2:172.21.15.160:3300/0 conn(0x7ff4340ce030 0x7ff4340d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43a433640 1 -- 172.21.15.26:0/2174937921 >> v1:172.21.15.160:6789/0 conn(0x7ff4340b85e0 legacy=0x7ff4340cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.442+0000 7ff43a433640 1 -- 172.21.15.26:0/2174937921 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff434161710 con 0x7ff43415cda0 2023-12-19T18:16:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.446+0000 7ff43a433640 1 --2- 172.21.15.26:0/2174937921 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff43415cda0 0x7ff434160920 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff430009870 tx=0x7ff43001c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=949dc2212ff3a62c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:44.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.446+0000 7ff439431640 1 -- 172.21.15.26:0/2174937921 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4300aa650 con 0x7ff43415cda0 2023-12-19T18:16:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.446+0000 7ff439431640 1 -- 172.21.15.26:0/2174937921 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4300aa7b0 con 0x7ff43415cda0 2023-12-19T18:16:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.446+0000 7ff439431640 1 -- 172.21.15.26:0/2174937921 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4300aab40 con 0x7ff43415cda0 2023-12-19T18:16:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.446+0000 7ff43c697640 1 -- 172.21.15.26:0/2174937921 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff43415cda0 msgr2=0x7ff434160920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.446+0000 7ff43c697640 1 --2- 172.21.15.26:0/2174937921 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff43415cda0 0x7ff434160920 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff430009870 tx=0x7ff43001c510 comp rx=0 tx=0).stop 2023-12-19T18:16:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.446+0000 7ff43c697640 1 -- 172.21.15.26:0/2174937921 shutdown_connections 2023-12-19T18:16:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.446+0000 7ff43c697640 1 --2- 172.21.15.26:0/2174937921 >> v2:172.21.15.160:3300/0 conn(0x7ff4340ce030 0x7ff4340d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.446+0000 7ff43c697640 1 --2- 172.21.15.26:0/2174937921 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff43415cda0 0x7ff434160920 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.446+0000 7ff43c697640 1 -- 172.21.15.26:0/2174937921 >> 172.21.15.26:0/2174937921 conn(0x7ff4340658e0 msgr2=0x7ff434065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.446+0000 7ff43c697640 1 -- 172.21.15.26:0/2174937921 shutdown_connections 2023-12-19T18:16:44.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.446+0000 7ff43c697640 1 -- 172.21.15.26:0/2174937921 wait complete. 2023-12-19T18:16:44.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43c697640 1 Processor -- start 2023-12-19T18:16:44.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43c697640 1 -- start start 2023-12-19T18:16:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43c697640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4340b85e0 0x7ff4340c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43c697640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4340ce030 0x7ff4340b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43c697640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff43415cda0 0x7ff4340cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43c697640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff434152210 con 0x7ff43415cda0 2023-12-19T18:16:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43c697640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340cc550 con 0x7ff4340ce030 2023-12-19T18:16:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43c697640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340cc830 con 0x7ff4340b85e0 2023-12-19T18:16:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43ac34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff43415cda0 0x7ff4340cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43ac34640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff43415cda0 0x7ff4340cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39548/0 (socket says 172.21.15.26:39548) 2023-12-19T18:16:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43ac34640 1 -- 172.21.15.26:0/2233791200 learned_addr learned my addr 172.21.15.26:0/2233791200 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff439c32640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4340ce030 0x7ff4340b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43a433640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4340b85e0 0x7ff4340c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43ac34640 1 -- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4340b85e0 msgr2=0x7ff4340c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43ac34640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4340b85e0 0x7ff4340c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43ac34640 1 -- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4340ce030 msgr2=0x7ff4340b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43ac34640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4340ce030 0x7ff4340b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43ac34640 1 -- 172.21.15.26:0/2233791200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4300094d0 con 0x7ff43415cda0 2023-12-19T18:16:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43ac34640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff43415cda0 0x7ff4340cd110 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00efc0 tx=0x7ff42c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff42affd640 1 -- 172.21.15.26:0/2233791200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c009ca0 con 0x7ff43415cda0 2023-12-19T18:16:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff42affd640 1 -- 172.21.15.26:0/2233791200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42c00f040 con 0x7ff43415cda0 2023-12-19T18:16:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff42affd640 1 -- 172.21.15.26:0/2233791200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c0146c0 con 0x7ff43415cda0 2023-12-19T18:16:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43c697640 1 -- 172.21.15.26:0/2233791200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4340ccb10 con 0x7ff43415cda0 2023-12-19T18:16:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.450+0000 7ff43c697640 1 -- 172.21.15.26:0/2233791200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4340cc200 con 0x7ff43415cda0 2023-12-19T18:16:44.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.454+0000 7ff43c697640 1 -- 172.21.15.26:0/2233791200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4340ca0c0 con 0x7ff43415cda0 2023-12-19T18:16:44.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.454+0000 7ff42affd640 1 -- 172.21.15.26:0/2233791200 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff42c014820 con 0x7ff43415cda0 2023-12-19T18:16:44.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.454+0000 7ff42affd640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff418065b80 0x7ff418068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.454+0000 7ff42affd640 1 -- 172.21.15.26:0/2233791200 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7ff42c0848c0 con 0x7ff43415cda0 2023-12-19T18:16:44.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.454+0000 7ff43a433640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff418065b80 0x7ff418068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.454+0000 7ff43a433640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff418065b80 0x7ff418068040 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff4300ae000 tx=0x7ff4300a7000 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:44.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.458+0000 7ff42affd640 1 -- 172.21.15.26:0/2233791200 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff4340ca0c0 con 0x7ff43415cda0 2023-12-19T18:16:44.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.610+0000 7ff43c697640 1 -- 172.21.15.26:0/2233791200 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff43415d180 con 0x7ff418065b80 2023-12-19T18:16:44.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.630+0000 7ff42affd640 1 -- 172.21.15.26:0/2233791200 <== mgr.9805 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526710 (secure 0 0 0) 0x7ff4300aa630 con 0x7ff418065b80 2023-12-19T18:16:44.636 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:44.640 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:16:44.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.634+0000 7ff43c697640 1 -- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff418065b80 msgr2=0x7ff418068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.634+0000 7ff43c697640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff418065b80 0x7ff418068040 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff4300ae000 tx=0x7ff4300a7000 comp rx=0 tx=0).stop 2023-12-19T18:16:44.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.634+0000 7ff43c697640 1 -- 172.21.15.26:0/2233791200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff43415cda0 msgr2=0x7ff4340cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.634+0000 7ff43c697640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff43415cda0 0x7ff4340cd110 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00efc0 tx=0x7ff42c00c490 comp rx=0 tx=0).stop 2023-12-19T18:16:44.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.634+0000 7ff43c697640 1 -- 172.21.15.26:0/2233791200 shutdown_connections 2023-12-19T18:16:44.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.634+0000 7ff43c697640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff418065b80 0x7ff418068040 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.634+0000 7ff43c697640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff43415cda0 0x7ff4340cd110 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.634+0000 7ff43c697640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4340ce030 0x7ff4340b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.634+0000 7ff43c697640 1 --2- 172.21.15.26:0/2233791200 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff4340b85e0 0x7ff4340c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.634+0000 7ff43c697640 1 -- 172.21.15.26:0/2233791200 >> 172.21.15.26:0/2233791200 conn(0x7ff4340658e0 msgr2=0x7ff43414f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:44.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.634+0000 7ff43c697640 1 -- 172.21.15.26:0/2233791200 shutdown_connections 2023-12-19T18:16:44.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.634+0000 7ff43c697640 1 -- 172.21.15.26:0/2233791200 wait complete. 2023-12-19T18:16:44.656 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-12-19T18:16:43.307249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594769,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":657,"num_write_kb":2443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":417,"ondisk_log_size":417,"up":364,"acting":364,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423232,"kb_used_data":22272,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754551488,"statfs":{"total":773094113280,"available":772660723712,"internally_reserved":0,"allocated":22806528,"data_stored":12598456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.182086"},"pg_stats":[{"pgid":"27.1a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357761+0000","last_change":"2023-12-19T18:16:40.564006+0000","last_active":"2023-12-19T18:16:42.357761+0000","last_peered":"2023-12-19T18:16:42.357761+0000","last_clean":"2023-12-19T18:16:42.357761+0000","last_became_active":"2023-12-19T18:16:40.563756+0000","last_became_peered":"2023-12-19T18:16:40.563756+0000","last_unstale":"2023-12-19T18:16:42.357761+0000","last_undegraded":"2023-12-19T18:16:42.357761+0000","last_fullsized":"2023-12-19T18:16:42.357761+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:17.484857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"200'346","reported_seq":1184,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.811441+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:16:27.811441+0000","last_peered":"2023-12-19T18:16:27.811441+0000","last_clean":"2023-12-19T18:16:27.811441+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:16:27.811441+0000","last_undegraded":"2023-12-19T18:16:27.811441+0000","last_fullsized":"2023-12-19T18:16:27.811441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305339+0000","last_change":"2023-12-19T18:16:27.411230+0000","last_active":"2023-12-19T18:16:28.305339+0000","last_peered":"2023-12-19T18:16:28.305339+0000","last_clean":"2023-12-19T18:16:28.305339+0000","last_became_active":"2023-12-19T18:16:27.410670+0000","last_became_peered":"2023-12-19T18:16:27.410670+0000","last_unstale":"2023-12-19T18:16:28.305339+0000","last_undegraded":"2023-12-19T18:16:28.305339+0000","last_fullsized":"2023-12-19T18:16:28.305339+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:22.426446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554070+0000","last_change":"2023-12-19T18:16:27.821899+0000","last_active":"2023-12-19T18:16:40.554070+0000","last_peered":"2023-12-19T18:16:40.554070+0000","last_clean":"2023-12-19T18:16:40.554070+0000","last_became_active":"2023-12-19T18:16:27.821634+0000","last_became_peered":"2023-12-19T18:16:27.821634+0000","last_unstale":"2023-12-19T18:16:40.554070+0000","last_undegraded":"2023-12-19T18:16:40.554070+0000","last_fullsized":"2023-12-19T18:16:40.554070+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:34.391068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857383+0000","last_change":"2023-12-19T18:16:40.150476+0000","last_active":"2023-12-19T18:16:41.857383+0000","last_peered":"2023-12-19T18:16:41.857383+0000","last_clean":"2023-12-19T18:16:41.857383+0000","last_became_active":"2023-12-19T18:16:40.149843+0000","last_became_peered":"2023-12-19T18:16:40.149843+0000","last_unstale":"2023-12-19T18:16:41.857383+0000","last_undegraded":"2023-12-19T18:16:41.857383+0000","last_fullsized":"2023-12-19T18:16:41.857383+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:35.179628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857177+0000","last_change":"2023-12-19T18:16:40.569150+0000","last_active":"2023-12-19T18:16:41.857177+0000","last_peered":"2023-12-19T18:16:41.857177+0000","last_clean":"2023-12-19T18:16:41.857177+0000","last_became_active":"2023-12-19T18:16:40.568754+0000","last_became_peered":"2023-12-19T18:16:40.568754+0000","last_unstale":"2023-12-19T18:16:41.857177+0000","last_undegraded":"2023-12-19T18:16:41.857177+0000","last_fullsized":"2023-12-19T18:16:41.857177+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:32.689972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548913+0000","last_change":"2023-12-19T18:16:27.412171+0000","last_active":"2023-12-19T18:16:40.548913+0000","last_peered":"2023-12-19T18:16:40.548913+0000","last_clean":"2023-12-19T18:16:40.548913+0000","last_became_active":"2023-12-19T18:16:27.411872+0000","last_became_peered":"2023-12-19T18:16:27.411872+0000","last_unstale":"2023-12-19T18:16:40.548913+0000","last_undegraded":"2023-12-19T18:16:40.548913+0000","last_fullsized":"2023-12-19T18:16:40.548913+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:48.264905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548191+0000","last_change":"2023-12-19T18:16:27.832432+0000","last_active":"2023-12-19T18:16:40.548191+0000","last_peered":"2023-12-19T18:16:40.548191+0000","last_clean":"2023-12-19T18:16:40.548191+0000","last_became_active":"2023-12-19T18:16:27.832116+0000","last_became_peered":"2023-12-19T18:16:27.832116+0000","last_unstale":"2023-12-19T18:16:40.548191+0000","last_undegraded":"2023-12-19T18:16:40.548191+0000","last_fullsized":"2023-12-19T18:16:40.548191+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:35.165338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359499+0000","last_change":"2023-12-19T18:16:40.157009+0000","last_active":"2023-12-19T18:16:42.359499+0000","last_peered":"2023-12-19T18:16:42.359499+0000","last_clean":"2023-12-19T18:16:42.359499+0000","last_became_active":"2023-12-19T18:16:40.156734+0000","last_became_peered":"2023-12-19T18:16:40.156734+0000","last_unstale":"2023-12-19T18:16:42.359499+0000","last_undegraded":"2023-12-19T18:16:42.359499+0000","last_fullsized":"2023-12-19T18:16:42.359499+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:18.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357983+0000","last_change":"2023-12-19T18:16:40.560903+0000","last_active":"2023-12-19T18:16:42.357983+0000","last_peered":"2023-12-19T18:16:42.357983+0000","last_clean":"2023-12-19T18:16:42.357983+0000","last_became_active":"2023-12-19T18:16:40.560636+0000","last_became_peered":"2023-12-19T18:16:40.560636+0000","last_unstale":"2023-12-19T18:16:42.357983+0000","last_undegraded":"2023-12-19T18:16:42.357983+0000","last_fullsized":"2023-12-19T18:16:42.357983+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:25:38.247442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561435+0000","last_change":"2023-12-19T18:16:27.407456+0000","last_active":"2023-12-19T18:16:40.561435+0000","last_peered":"2023-12-19T18:16:40.561435+0000","last_clean":"2023-12-19T18:16:40.561435+0000","last_became_active":"2023-12-19T18:16:27.407208+0000","last_became_peered":"2023-12-19T18:16:27.407208+0000","last_unstale":"2023-12-19T18:16:40.561435+0000","last_undegraded":"2023-12-19T18:16:40.561435+0000","last_fullsized":"2023-12-19T18:16:40.561435+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:07.654229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560528+0000","last_change":"2023-12-19T18:16:27.824001+0000","last_active":"2023-12-19T18:16:40.560528+0000","last_peered":"2023-12-19T18:16:40.560528+0000","last_clean":"2023-12-19T18:16:40.560528+0000","last_became_active":"2023-12-19T18:16:27.823010+0000","last_became_peered":"2023-12-19T18:16:27.823010+0000","last_unstale":"2023-12-19T18:16:40.560528+0000","last_undegraded":"2023-12-19T18:16:40.560528+0000","last_fullsized":"2023-12-19T18:16:40.560528+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:30:56.775994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:39.117449+0000","last_change":"2023-12-19T18:16:39.117449+0000","last_active":"2023-12-19T18:16:39.117449+0000","last_peered":"2023-12-19T18:16:39.117449+0000","last_clean":"2023-12-19T18:16:39.117449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:39.117449+0000","last_undegraded":"2023-12-19T18:16:39.117449+0000","last_fullsized":"2023-12-19T18:16:39.117449+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552525+0000","last_change":"2023-12-19T18:16:40.570024+0000","last_active":"2023-12-19T18:16:41.552525+0000","last_peered":"2023-12-19T18:16:41.552525+0000","last_clean":"2023-12-19T18:16:41.552525+0000","last_became_active":"2023-12-19T18:16:40.569586+0000","last_became_peered":"2023-12-19T18:16:40.569586+0000","last_unstale":"2023-12-19T18:16:41.552525+0000","last_undegraded":"2023-12-19T18:16:41.552525+0000","last_fullsized":"2023-12-19T18:16:41.552525+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:03.932038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547644+0000","last_change":"2023-12-19T18:16:27.405381+0000","last_active":"2023-12-19T18:16:40.547644+0000","last_peered":"2023-12-19T18:16:40.547644+0000","last_clean":"2023-12-19T18:16:40.547644+0000","last_became_active":"2023-12-19T18:16:27.405113+0000","last_became_peered":"2023-12-19T18:16:27.405113+0000","last_unstale":"2023-12-19T18:16:40.547644+0000","last_undegraded":"2023-12-19T18:16:40.547644+0000","last_fullsized":"2023-12-19T18:16:40.547644+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:08.911302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305362+0000","last_change":"2023-12-19T18:16:27.822423+0000","last_active":"2023-12-19T18:16:28.305362+0000","last_peered":"2023-12-19T18:16:28.305362+0000","last_clean":"2023-12-19T18:16:28.305362+0000","last_became_active":"2023-12-19T18:16:27.822106+0000","last_became_peered":"2023-12-19T18:16:27.822106+0000","last_unstale":"2023-12-19T18:16:28.305362+0000","last_undegraded":"2023-12-19T18:16:28.305362+0000","last_fullsized":"2023-12-19T18:16:28.305362+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:41.021760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.856687+0000","last_change":"2023-12-19T18:16:40.154697+0000","last_active":"2023-12-19T18:16:41.856687+0000","last_peered":"2023-12-19T18:16:41.856687+0000","last_clean":"2023-12-19T18:16:41.856687+0000","last_became_active":"2023-12-19T18:16:40.154394+0000","last_became_peered":"2023-12-19T18:16:40.154394+0000","last_unstale":"2023-12-19T18:16:41.856687+0000","last_undegraded":"2023-12-19T18:16:41.856687+0000","last_fullsized":"2023-12-19T18:16:41.856687+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:52.755953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357512+0000","last_change":"2023-12-19T18:16:40.572030+0000","last_active":"2023-12-19T18:16:42.357512+0000","last_peered":"2023-12-19T18:16:42.357512+0000","last_clean":"2023-12-19T18:16:42.357512+0000","last_became_active":"2023-12-19T18:16:40.570330+0000","last_became_peered":"2023-12-19T18:16:40.570330+0000","last_unstale":"2023-12-19T18:16:42.357512+0000","last_undegraded":"2023-12-19T18:16:42.357512+0000","last_fullsized":"2023-12-19T18:16:42.357512+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:55.123677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547774+0000","last_change":"2023-12-19T18:16:27.410513+0000","last_active":"2023-12-19T18:16:40.547774+0000","last_peered":"2023-12-19T18:16:40.547774+0000","last_clean":"2023-12-19T18:16:40.547774+0000","last_became_active":"2023-12-19T18:16:27.410166+0000","last_became_peered":"2023-12-19T18:16:27.410166+0000","last_unstale":"2023-12-19T18:16:40.547774+0000","last_undegraded":"2023-12-19T18:16:40.547774+0000","last_fullsized":"2023-12-19T18:16:40.547774+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:10.682880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305351+0000","last_change":"2023-12-19T18:16:27.829925+0000","last_active":"2023-12-19T18:16:28.305351+0000","last_peered":"2023-12-19T18:16:28.305351+0000","last_clean":"2023-12-19T18:16:28.305351+0000","last_became_active":"2023-12-19T18:16:27.828958+0000","last_became_peered":"2023-12-19T18:16:27.828958+0000","last_unstale":"2023-12-19T18:16:28.305351+0000","last_undegraded":"2023-12-19T18:16:28.305351+0000","last_fullsized":"2023-12-19T18:16:28.305351+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:46.995476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554304+0000","last_change":"2023-12-19T18:16:40.156673+0000","last_active":"2023-12-19T18:16:41.554304+0000","last_peered":"2023-12-19T18:16:41.554304+0000","last_clean":"2023-12-19T18:16:41.554304+0000","last_became_active":"2023-12-19T18:16:40.156391+0000","last_became_peered":"2023-12-19T18:16:40.156391+0000","last_unstale":"2023-12-19T18:16:41.554304+0000","last_undegraded":"2023-12-19T18:16:41.554304+0000","last_fullsized":"2023-12-19T18:16:41.554304+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:46.681719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360169+0000","last_change":"2023-12-19T18:16:40.571314+0000","last_active":"2023-12-19T18:16:42.360169+0000","last_peered":"2023-12-19T18:16:42.360169+0000","last_clean":"2023-12-19T18:16:42.360169+0000","last_became_active":"2023-12-19T18:16:40.570538+0000","last_became_peered":"2023-12-19T18:16:40.570538+0000","last_unstale":"2023-12-19T18:16:42.360169+0000","last_undegraded":"2023-12-19T18:16:42.360169+0000","last_fullsized":"2023-12-19T18:16:42.360169+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:02:14.652060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543489+0000","last_change":"2023-12-19T18:16:27.409169+0000","last_active":"2023-12-19T18:16:40.543489+0000","last_peered":"2023-12-19T18:16:40.543489+0000","last_clean":"2023-12-19T18:16:40.543489+0000","last_became_active":"2023-12-19T18:16:27.408882+0000","last_became_peered":"2023-12-19T18:16:27.408882+0000","last_unstale":"2023-12-19T18:16:40.543489+0000","last_undegraded":"2023-12-19T18:16:40.543489+0000","last_fullsized":"2023-12-19T18:16:40.543489+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:01.478246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544313+0000","last_change":"2023-12-19T18:16:27.825200+0000","last_active":"2023-12-19T18:16:40.544313+0000","last_peered":"2023-12-19T18:16:40.544313+0000","last_clean":"2023-12-19T18:16:40.544313+0000","last_became_active":"2023-12-19T18:16:27.824919+0000","last_became_peered":"2023-12-19T18:16:27.824919+0000","last_unstale":"2023-12-19T18:16:40.544313+0000","last_undegraded":"2023-12-19T18:16:40.544313+0000","last_fullsized":"2023-12-19T18:16:40.544313+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:39.117449+0000","last_change":"2023-12-19T18:16:39.117449+0000","last_active":"2023-12-19T18:16:39.117449+0000","last_peered":"2023-12-19T18:16:39.117449+0000","last_clean":"2023-12-19T18:16:39.117449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:39.117449+0000","last_undegraded":"2023-12-19T18:16:39.117449+0000","last_fullsized":"2023-12-19T18:16:39.117449+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857196+0000","last_change":"2023-12-19T18:16:40.568314+0000","last_active":"2023-12-19T18:16:41.857196+0000","last_peered":"2023-12-19T18:16:41.857196+0000","last_clean":"2023-12-19T18:16:41.857196+0000","last_became_active":"2023-12-19T18:16:40.567857+0000","last_became_peered":"2023-12-19T18:16:40.567857+0000","last_unstale":"2023-12-19T18:16:41.857196+0000","last_undegraded":"2023-12-19T18:16:41.857196+0000","last_fullsized":"2023-12-19T18:16:41.857196+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:49.703164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555637+0000","last_change":"2023-12-19T18:16:27.420249+0000","last_active":"2023-12-19T18:16:40.555637+0000","last_peered":"2023-12-19T18:16:40.555637+0000","last_clean":"2023-12-19T18:16:40.555637+0000","last_became_active":"2023-12-19T18:16:27.419360+0000","last_became_peered":"2023-12-19T18:16:27.419360+0000","last_unstale":"2023-12-19T18:16:40.555637+0000","last_undegraded":"2023-12-19T18:16:40.555637+0000","last_fullsized":"2023-12-19T18:16:40.555637+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:07.969904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543383+0000","last_change":"2023-12-19T18:16:27.825535+0000","last_active":"2023-12-19T18:16:40.543383+0000","last_peered":"2023-12-19T18:16:40.543383+0000","last_clean":"2023-12-19T18:16:40.543383+0000","last_became_active":"2023-12-19T18:16:27.824814+0000","last_became_peered":"2023-12-19T18:16:27.824814+0000","last_unstale":"2023-12-19T18:16:40.543383+0000","last_undegraded":"2023-12-19T18:16:40.543383+0000","last_fullsized":"2023-12-19T18:16:40.543383+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:29.357409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358846+0000","last_change":"2023-12-19T18:16:40.160113+0000","last_active":"2023-12-19T18:16:42.358846+0000","last_peered":"2023-12-19T18:16:42.358846+0000","last_clean":"2023-12-19T18:16:42.358846+0000","last_became_active":"2023-12-19T18:16:40.159826+0000","last_became_peered":"2023-12-19T18:16:40.159826+0000","last_unstale":"2023-12-19T18:16:42.358846+0000","last_undegraded":"2023-12-19T18:16:42.358846+0000","last_fullsized":"2023-12-19T18:16:42.358846+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:45.058000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554995+0000","last_change":"2023-12-19T18:16:40.566980+0000","last_active":"2023-12-19T18:16:41.554995+0000","last_peered":"2023-12-19T18:16:41.554995+0000","last_clean":"2023-12-19T18:16:41.554995+0000","last_became_active":"2023-12-19T18:16:40.566716+0000","last_became_peered":"2023-12-19T18:16:40.566716+0000","last_unstale":"2023-12-19T18:16:41.554995+0000","last_undegraded":"2023-12-19T18:16:41.554995+0000","last_fullsized":"2023-12-19T18:16:41.554995+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:30.489642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.6","version":"212'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558804+0000","last_change":"2023-12-19T18:16:27.414152+0000","last_active":"2023-12-19T18:16:40.558804+0000","last_peered":"2023-12-19T18:16:40.558804+0000","last_clean":"2023-12-19T18:16:40.558804+0000","last_became_active":"2023-12-19T18:16:27.413877+0000","last_became_peered":"2023-12-19T18:16:27.413877+0000","last_unstale":"2023-12-19T18:16:40.558804+0000","last_undegraded":"2023-12-19T18:16:40.558804+0000","last_fullsized":"2023-12-19T18:16:40.558804+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:44:52.553685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558691+0000","last_change":"2023-12-19T18:16:27.824257+0000","last_active":"2023-12-19T18:16:40.558691+0000","last_peered":"2023-12-19T18:16:40.558691+0000","last_clean":"2023-12-19T18:16:40.558691+0000","last_became_active":"2023-12-19T18:16:27.823989+0000","last_became_peered":"2023-12-19T18:16:27.823989+0000","last_unstale":"2023-12-19T18:16:40.558691+0000","last_undegraded":"2023-12-19T18:16:40.558691+0000","last_fullsized":"2023-12-19T18:16:40.558691+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:07:07.505127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357977+0000","last_change":"2023-12-19T18:16:40.160316+0000","last_active":"2023-12-19T18:16:42.357977+0000","last_peered":"2023-12-19T18:16:42.357977+0000","last_clean":"2023-12-19T18:16:42.357977+0000","last_became_active":"2023-12-19T18:16:40.159312+0000","last_became_peered":"2023-12-19T18:16:40.159312+0000","last_unstale":"2023-12-19T18:16:42.357977+0000","last_undegraded":"2023-12-19T18:16:42.357977+0000","last_fullsized":"2023-12-19T18:16:42.357977+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:04.832569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553051+0000","last_change":"2023-12-19T18:16:40.563986+0000","last_active":"2023-12-19T18:16:41.553051+0000","last_peered":"2023-12-19T18:16:41.553051+0000","last_clean":"2023-12-19T18:16:41.553051+0000","last_became_active":"2023-12-19T18:16:40.563693+0000","last_became_peered":"2023-12-19T18:16:40.563693+0000","last_unstale":"2023-12-19T18:16:41.553051+0000","last_undegraded":"2023-12-19T18:16:41.553051+0000","last_fullsized":"2023-12-19T18:16:41.553051+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:27.465263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"211'2","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305451+0000","last_change":"2023-12-19T18:16:27.409874+0000","last_active":"2023-12-19T18:16:28.305451+0000","last_peered":"2023-12-19T18:16:28.305451+0000","last_clean":"2023-12-19T18:16:28.305451+0000","last_became_active":"2023-12-19T18:16:27.409598+0000","last_became_peered":"2023-12-19T18:16:27.409598+0000","last_unstale":"2023-12-19T18:16:28.305451+0000","last_undegraded":"2023-12-19T18:16:28.305451+0000","last_fullsized":"2023-12-19T18:16:28.305451+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:13.721057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549450+0000","last_change":"2023-12-19T18:16:27.826090+0000","last_active":"2023-12-19T18:16:40.549450+0000","last_peered":"2023-12-19T18:16:40.549450+0000","last_clean":"2023-12-19T18:16:40.549450+0000","last_became_active":"2023-12-19T18:16:27.825791+0000","last_became_peered":"2023-12-19T18:16:27.825791+0000","last_unstale":"2023-12-19T18:16:40.549450+0000","last_undegraded":"2023-12-19T18:16:40.549450+0000","last_fullsized":"2023-12-19T18:16:40.549450+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:39.382953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359959+0000","last_change":"2023-12-19T18:16:40.160037+0000","last_active":"2023-12-19T18:16:42.359959+0000","last_peered":"2023-12-19T18:16:42.359959+0000","last_clean":"2023-12-19T18:16:42.359959+0000","last_became_active":"2023-12-19T18:16:40.159737+0000","last_became_peered":"2023-12-19T18:16:40.159737+0000","last_unstale":"2023-12-19T18:16:42.359959+0000","last_undegraded":"2023-12-19T18:16:42.359959+0000","last_fullsized":"2023-12-19T18:16:42.359959+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:42.336188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358621+0000","last_change":"2023-12-19T18:16:40.567065+0000","last_active":"2023-12-19T18:16:42.358621+0000","last_peered":"2023-12-19T18:16:42.358621+0000","last_clean":"2023-12-19T18:16:42.358621+0000","last_became_active":"2023-12-19T18:16:40.566804+0000","last_became_peered":"2023-12-19T18:16:40.566804+0000","last_unstale":"2023-12-19T18:16:42.358621+0000","last_undegraded":"2023-12-19T18:16:42.358621+0000","last_fullsized":"2023-12-19T18:16:42.358621+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:21.029140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546688+0000","last_change":"2023-12-19T18:16:27.403537+0000","last_active":"2023-12-19T18:16:40.546688+0000","last_peered":"2023-12-19T18:16:40.546688+0000","last_clean":"2023-12-19T18:16:40.546688+0000","last_became_active":"2023-12-19T18:16:27.403250+0000","last_became_peered":"2023-12-19T18:16:27.403250+0000","last_unstale":"2023-12-19T18:16:40.546688+0000","last_undegraded":"2023-12-19T18:16:40.546688+0000","last_fullsized":"2023-12-19T18:16:40.546688+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:56.889902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548113+0000","last_change":"2023-12-19T18:16:27.825355+0000","last_active":"2023-12-19T18:16:40.548113+0000","last_peered":"2023-12-19T18:16:40.548113+0000","last_clean":"2023-12-19T18:16:40.548113+0000","last_became_active":"2023-12-19T18:16:27.825067+0000","last_became_peered":"2023-12-19T18:16:27.825067+0000","last_unstale":"2023-12-19T18:16:40.548113+0000","last_undegraded":"2023-12-19T18:16:40.548113+0000","last_fullsized":"2023-12-19T18:16:40.548113+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:02.928407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358143+0000","last_change":"2023-12-19T18:16:40.152882+0000","last_active":"2023-12-19T18:16:42.358143+0000","last_peered":"2023-12-19T18:16:42.358143+0000","last_clean":"2023-12-19T18:16:42.358143+0000","last_became_active":"2023-12-19T18:16:40.152561+0000","last_became_peered":"2023-12-19T18:16:40.152561+0000","last_unstale":"2023-12-19T18:16:42.358143+0000","last_undegraded":"2023-12-19T18:16:42.358143+0000","last_fullsized":"2023-12-19T18:16:42.358143+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:11:26.922031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550797+0000","last_change":"2023-12-19T18:16:40.560282+0000","last_active":"2023-12-19T18:16:41.550797+0000","last_peered":"2023-12-19T18:16:41.550797+0000","last_clean":"2023-12-19T18:16:41.550797+0000","last_became_active":"2023-12-19T18:16:40.560017+0000","last_became_peered":"2023-12-19T18:16:40.560017+0000","last_unstale":"2023-12-19T18:16:41.550797+0000","last_undegraded":"2023-12-19T18:16:41.550797+0000","last_fullsized":"2023-12-19T18:16:41.550797+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:51:01.095692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553689+0000","last_change":"2023-12-19T18:16:27.417198+0000","last_active":"2023-12-19T18:16:40.553689+0000","last_peered":"2023-12-19T18:16:40.553689+0000","last_clean":"2023-12-19T18:16:40.553689+0000","last_became_active":"2023-12-19T18:16:27.416902+0000","last_became_peered":"2023-12-19T18:16:27.416902+0000","last_unstale":"2023-12-19T18:16:40.553689+0000","last_undegraded":"2023-12-19T18:16:40.553689+0000","last_fullsized":"2023-12-19T18:16:40.553689+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:54.434372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560449+0000","last_change":"2023-12-19T18:16:27.824198+0000","last_active":"2023-12-19T18:16:40.560449+0000","last_peered":"2023-12-19T18:16:40.560449+0000","last_clean":"2023-12-19T18:16:40.560449+0000","last_became_active":"2023-12-19T18:16:27.823738+0000","last_became_peered":"2023-12-19T18:16:27.823738+0000","last_unstale":"2023-12-19T18:16:40.560449+0000","last_undegraded":"2023-12-19T18:16:40.560449+0000","last_fullsized":"2023-12-19T18:16:40.560449+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:53.509897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550952+0000","last_change":"2023-12-19T18:16:40.160436+0000","last_active":"2023-12-19T18:16:41.550952+0000","last_peered":"2023-12-19T18:16:41.550952+0000","last_clean":"2023-12-19T18:16:41.550952+0000","last_became_active":"2023-12-19T18:16:40.159692+0000","last_became_peered":"2023-12-19T18:16:40.159692+0000","last_unstale":"2023-12-19T18:16:41.550952+0000","last_undegraded":"2023-12-19T18:16:41.550952+0000","last_fullsized":"2023-12-19T18:16:41.550952+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:47:01.985064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554294+0000","last_change":"2023-12-19T18:16:40.572114+0000","last_active":"2023-12-19T18:16:41.554294+0000","last_peered":"2023-12-19T18:16:41.554294+0000","last_clean":"2023-12-19T18:16:41.554294+0000","last_became_active":"2023-12-19T18:16:40.571811+0000","last_became_peered":"2023-12-19T18:16:40.571811+0000","last_unstale":"2023-12-19T18:16:41.554294+0000","last_undegraded":"2023-12-19T18:16:41.554294+0000","last_fullsized":"2023-12-19T18:16:41.554294+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:01.072159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550771+0000","last_change":"2023-12-19T18:16:27.404853+0000","last_active":"2023-12-19T18:16:40.550771+0000","last_peered":"2023-12-19T18:16:40.550771+0000","last_clean":"2023-12-19T18:16:40.550771+0000","last_became_active":"2023-12-19T18:16:27.404347+0000","last_became_peered":"2023-12-19T18:16:27.404347+0000","last_unstale":"2023-12-19T18:16:40.550771+0000","last_undegraded":"2023-12-19T18:16:40.550771+0000","last_fullsized":"2023-12-19T18:16:40.550771+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:16.360996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558229+0000","last_change":"2023-12-19T18:16:27.828160+0000","last_active":"2023-12-19T18:16:40.558229+0000","last_peered":"2023-12-19T18:16:40.558229+0000","last_clean":"2023-12-19T18:16:40.558229+0000","last_became_active":"2023-12-19T18:16:27.827882+0000","last_became_peered":"2023-12-19T18:16:27.827882+0000","last_unstale":"2023-12-19T18:16:40.558229+0000","last_undegraded":"2023-12-19T18:16:40.558229+0000","last_fullsized":"2023-12-19T18:16:40.558229+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:17.726704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358933+0000","last_change":"2023-12-19T18:16:40.154706+0000","last_active":"2023-12-19T18:16:42.358933+0000","last_peered":"2023-12-19T18:16:42.358933+0000","last_clean":"2023-12-19T18:16:42.358933+0000","last_became_active":"2023-12-19T18:16:40.154154+0000","last_became_peered":"2023-12-19T18:16:40.154154+0000","last_unstale":"2023-12-19T18:16:42.358933+0000","last_undegraded":"2023-12-19T18:16:42.358933+0000","last_fullsized":"2023-12-19T18:16:42.358933+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:55.343366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552500+0000","last_change":"2023-12-19T18:16:40.555828+0000","last_active":"2023-12-19T18:16:41.552500+0000","last_peered":"2023-12-19T18:16:41.552500+0000","last_clean":"2023-12-19T18:16:41.552500+0000","last_became_active":"2023-12-19T18:16:40.555500+0000","last_became_peered":"2023-12-19T18:16:40.555500+0000","last_unstale":"2023-12-19T18:16:41.552500+0000","last_undegraded":"2023-12-19T18:16:41.552500+0000","last_fullsized":"2023-12-19T18:16:41.552500+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:50.483528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.b","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551100+0000","last_change":"2023-12-19T18:16:27.415902+0000","last_active":"2023-12-19T18:16:40.551100+0000","last_peered":"2023-12-19T18:16:40.551100+0000","last_clean":"2023-12-19T18:16:40.551100+0000","last_became_active":"2023-12-19T18:16:27.415062+0000","last_became_peered":"2023-12-19T18:16:27.415062+0000","last_unstale":"2023-12-19T18:16:40.551100+0000","last_undegraded":"2023-12-19T18:16:40.551100+0000","last_fullsized":"2023-12-19T18:16:40.551100+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:22.442653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556868+0000","last_change":"2023-12-19T18:16:27.827898+0000","last_active":"2023-12-19T18:16:40.556868+0000","last_peered":"2023-12-19T18:16:40.556868+0000","last_clean":"2023-12-19T18:16:40.556868+0000","last_became_active":"2023-12-19T18:16:27.827613+0000","last_became_peered":"2023-12-19T18:16:27.827613+0000","last_unstale":"2023-12-19T18:16:40.556868+0000","last_undegraded":"2023-12-19T18:16:40.556868+0000","last_fullsized":"2023-12-19T18:16:40.556868+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:29.033205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357614+0000","last_change":"2023-12-19T18:16:40.157634+0000","last_active":"2023-12-19T18:16:42.357614+0000","last_peered":"2023-12-19T18:16:42.357614+0000","last_clean":"2023-12-19T18:16:42.357614+0000","last_became_active":"2023-12-19T18:16:40.157363+0000","last_became_peered":"2023-12-19T18:16:40.157363+0000","last_unstale":"2023-12-19T18:16:42.357614+0000","last_undegraded":"2023-12-19T18:16:42.357614+0000","last_fullsized":"2023-12-19T18:16:42.357614+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:23.564600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553681+0000","last_change":"2023-12-19T18:16:40.556441+0000","last_active":"2023-12-19T18:16:41.553681+0000","last_peered":"2023-12-19T18:16:41.553681+0000","last_clean":"2023-12-19T18:16:41.553681+0000","last_became_active":"2023-12-19T18:16:40.556187+0000","last_became_peered":"2023-12-19T18:16:40.556187+0000","last_unstale":"2023-12-19T18:16:41.553681+0000","last_undegraded":"2023-12-19T18:16:41.553681+0000","last_fullsized":"2023-12-19T18:16:41.553681+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:16:05.083354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562653+0000","last_change":"2023-12-19T18:16:27.408051+0000","last_active":"2023-12-19T18:16:40.562653+0000","last_peered":"2023-12-19T18:16:40.562653+0000","last_clean":"2023-12-19T18:16:40.562653+0000","last_became_active":"2023-12-19T18:16:27.407796+0000","last_became_peered":"2023-12-19T18:16:27.407796+0000","last_unstale":"2023-12-19T18:16:40.562653+0000","last_undegraded":"2023-12-19T18:16:40.562653+0000","last_fullsized":"2023-12-19T18:16:40.562653+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:41:09.697612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560009+0000","last_change":"2023-12-19T18:16:27.832376+0000","last_active":"2023-12-19T18:16:40.560009+0000","last_peered":"2023-12-19T18:16:40.560009+0000","last_clean":"2023-12-19T18:16:40.560009+0000","last_became_active":"2023-12-19T18:16:27.832060+0000","last_became_peered":"2023-12-19T18:16:27.832060+0000","last_unstale":"2023-12-19T18:16:40.560009+0000","last_undegraded":"2023-12-19T18:16:40.560009+0000","last_fullsized":"2023-12-19T18:16:40.560009+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:17.432889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.e","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553537+0000","last_change":"2023-12-19T18:16:40.152827+0000","last_active":"2023-12-19T18:16:41.553537+0000","last_peered":"2023-12-19T18:16:41.553537+0000","last_clean":"2023-12-19T18:16:41.553537+0000","last_became_active":"2023-12-19T18:16:40.152559+0000","last_became_peered":"2023-12-19T18:16:40.152559+0000","last_unstale":"2023-12-19T18:16:41.553537+0000","last_undegraded":"2023-12-19T18:16:41.553537+0000","last_fullsized":"2023-12-19T18:16:41.553537+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:59.363074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553423+0000","last_change":"2023-12-19T18:16:40.572243+0000","last_active":"2023-12-19T18:16:41.553423+0000","last_peered":"2023-12-19T18:16:41.553423+0000","last_clean":"2023-12-19T18:16:41.553423+0000","last_became_active":"2023-12-19T18:16:40.571938+0000","last_became_peered":"2023-12-19T18:16:40.571938+0000","last_unstale":"2023-12-19T18:16:41.553423+0000","last_undegraded":"2023-12-19T18:16:41.553423+0000","last_fullsized":"2023-12-19T18:16:41.553423+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:50.735904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"211'1","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305390+0000","last_change":"2023-12-19T18:16:27.416654+0000","last_active":"2023-12-19T18:16:28.305390+0000","last_peered":"2023-12-19T18:16:28.305390+0000","last_clean":"2023-12-19T18:16:28.305390+0000","last_became_active":"2023-12-19T18:16:27.416388+0000","last_became_peered":"2023-12-19T18:16:27.416388+0000","last_unstale":"2023-12-19T18:16:28.305390+0000","last_undegraded":"2023-12-19T18:16:28.305390+0000","last_fullsized":"2023-12-19T18:16:28.305390+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:51.339148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547637+0000","last_change":"2023-12-19T18:16:27.816436+0000","last_active":"2023-12-19T18:16:40.547637+0000","last_peered":"2023-12-19T18:16:40.547637+0000","last_clean":"2023-12-19T18:16:40.547637+0000","last_became_active":"2023-12-19T18:16:27.816092+0000","last_became_peered":"2023-12-19T18:16:27.816092+0000","last_unstale":"2023-12-19T18:16:40.547637+0000","last_undegraded":"2023-12-19T18:16:40.547637+0000","last_fullsized":"2023-12-19T18:16:40.547637+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:07:26.713649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360338+0000","last_change":"2023-12-19T18:16:40.154797+0000","last_active":"2023-12-19T18:16:42.360338+0000","last_peered":"2023-12-19T18:16:42.360338+0000","last_clean":"2023-12-19T18:16:42.360338+0000","last_became_active":"2023-12-19T18:16:40.154490+0000","last_became_peered":"2023-12-19T18:16:40.154490+0000","last_unstale":"2023-12-19T18:16:42.360338+0000","last_undegraded":"2023-12-19T18:16:42.360338+0000","last_fullsized":"2023-12-19T18:16:42.360338+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:10:02.821689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551216+0000","last_change":"2023-12-19T18:16:40.573560+0000","last_active":"2023-12-19T18:16:41.551216+0000","last_peered":"2023-12-19T18:16:41.551216+0000","last_clean":"2023-12-19T18:16:41.551216+0000","last_became_active":"2023-12-19T18:16:40.573277+0000","last_became_peered":"2023-12-19T18:16:40.573277+0000","last_unstale":"2023-12-19T18:16:41.551216+0000","last_undegraded":"2023-12-19T18:16:41.551216+0000","last_fullsized":"2023-12-19T18:16:41.551216+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:23:53.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544525+0000","last_change":"2023-12-19T18:16:27.409426+0000","last_active":"2023-12-19T18:16:40.544525+0000","last_peered":"2023-12-19T18:16:40.544525+0000","last_clean":"2023-12-19T18:16:40.544525+0000","last_became_active":"2023-12-19T18:16:27.407151+0000","last_became_peered":"2023-12-19T18:16:27.407151+0000","last_unstale":"2023-12-19T18:16:40.544525+0000","last_undegraded":"2023-12-19T18:16:40.544525+0000","last_fullsized":"2023-12-19T18:16:40.544525+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:30:56.976667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550067+0000","last_change":"2023-12-19T18:16:27.823846+0000","last_active":"2023-12-19T18:16:40.550067+0000","last_peered":"2023-12-19T18:16:40.550067+0000","last_clean":"2023-12-19T18:16:40.550067+0000","last_became_active":"2023-12-19T18:16:27.823514+0000","last_became_peered":"2023-12-19T18:16:27.823514+0000","last_unstale":"2023-12-19T18:16:40.550067+0000","last_undegraded":"2023-12-19T18:16:40.550067+0000","last_fullsized":"2023-12-19T18:16:40.550067+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:20.691537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358069+0000","last_change":"2023-12-19T18:16:40.162626+0000","last_active":"2023-12-19T18:16:42.358069+0000","last_peered":"2023-12-19T18:16:42.358069+0000","last_clean":"2023-12-19T18:16:42.358069+0000","last_became_active":"2023-12-19T18:16:40.161257+0000","last_became_peered":"2023-12-19T18:16:40.161257+0000","last_unstale":"2023-12-19T18:16:42.358069+0000","last_undegraded":"2023-12-19T18:16:42.358069+0000","last_fullsized":"2023-12-19T18:16:42.358069+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:21:05.448555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357894+0000","last_change":"2023-12-19T18:16:40.558784+0000","last_active":"2023-12-19T18:16:42.357894+0000","last_peered":"2023-12-19T18:16:42.357894+0000","last_clean":"2023-12-19T18:16:42.357894+0000","last_became_active":"2023-12-19T18:16:40.556768+0000","last_became_peered":"2023-12-19T18:16:40.556768+0000","last_unstale":"2023-12-19T18:16:42.357894+0000","last_undegraded":"2023-12-19T18:16:42.357894+0000","last_fullsized":"2023-12-19T18:16:42.357894+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:14:19.106971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553891+0000","last_change":"2023-12-19T18:16:27.414971+0000","last_active":"2023-12-19T18:16:40.553891+0000","last_peered":"2023-12-19T18:16:40.553891+0000","last_clean":"2023-12-19T18:16:40.553891+0000","last_became_active":"2023-12-19T18:16:27.414662+0000","last_became_peered":"2023-12-19T18:16:27.414662+0000","last_unstale":"2023-12-19T18:16:40.553891+0000","last_undegraded":"2023-12-19T18:16:40.553891+0000","last_fullsized":"2023-12-19T18:16:40.553891+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:47.204637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558717+0000","last_change":"2023-12-19T18:16:27.814804+0000","last_active":"2023-12-19T18:16:40.558717+0000","last_peered":"2023-12-19T18:16:40.558717+0000","last_clean":"2023-12-19T18:16:40.558717+0000","last_became_active":"2023-12-19T18:16:27.814534+0000","last_became_peered":"2023-12-19T18:16:27.814534+0000","last_unstale":"2023-12-19T18:16:40.558717+0000","last_undegraded":"2023-12-19T18:16:40.558717+0000","last_fullsized":"2023-12-19T18:16:40.558717+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:45.690711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551146+0000","last_change":"2023-12-19T18:16:40.158484+0000","last_active":"2023-12-19T18:16:41.551146+0000","last_peered":"2023-12-19T18:16:41.551146+0000","last_clean":"2023-12-19T18:16:41.551146+0000","last_became_active":"2023-12-19T18:16:40.158205+0000","last_became_peered":"2023-12-19T18:16:40.158205+0000","last_unstale":"2023-12-19T18:16:41.551146+0000","last_undegraded":"2023-12-19T18:16:41.551146+0000","last_fullsized":"2023-12-19T18:16:41.551146+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:23.673873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550951+0000","last_change":"2023-12-19T18:16:40.555429+0000","last_active":"2023-12-19T18:16:41.550951+0000","last_peered":"2023-12-19T18:16:41.550951+0000","last_clean":"2023-12-19T18:16:41.550951+0000","last_became_active":"2023-12-19T18:16:40.555158+0000","last_became_peered":"2023-12-19T18:16:40.555158+0000","last_unstale":"2023-12-19T18:16:41.550951+0000","last_undegraded":"2023-12-19T18:16:41.550951+0000","last_fullsized":"2023-12-19T18:16:41.550951+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:53.628064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.10","version":"211'8","reported_seq":19,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305320+0000","last_change":"2023-12-19T18:16:27.415118+0000","last_active":"2023-12-19T18:16:28.305320+0000","last_peered":"2023-12-19T18:16:28.305320+0000","last_clean":"2023-12-19T18:16:28.305320+0000","last_became_active":"2023-12-19T18:16:27.414687+0000","last_became_peered":"2023-12-19T18:16:27.414687+0000","last_unstale":"2023-12-19T18:16:28.305320+0000","last_undegraded":"2023-12-19T18:16:28.305320+0000","last_fullsized":"2023-12-19T18:16:28.305320+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:16.723922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555219+0000","last_change":"2023-12-19T18:16:27.814277+0000","last_active":"2023-12-19T18:16:40.555219+0000","last_peered":"2023-12-19T18:16:40.555219+0000","last_clean":"2023-12-19T18:16:40.555219+0000","last_became_active":"2023-12-19T18:16:27.814023+0000","last_became_peered":"2023-12-19T18:16:27.814023+0000","last_unstale":"2023-12-19T18:16:40.555219+0000","last_undegraded":"2023-12-19T18:16:40.555219+0000","last_fullsized":"2023-12-19T18:16:40.555219+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:32.135133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554395+0000","last_change":"2023-12-19T18:16:40.154784+0000","last_active":"2023-12-19T18:16:41.554395+0000","last_peered":"2023-12-19T18:16:41.554395+0000","last_clean":"2023-12-19T18:16:41.554395+0000","last_became_active":"2023-12-19T18:16:40.154523+0000","last_became_peered":"2023-12-19T18:16:40.154523+0000","last_unstale":"2023-12-19T18:16:41.554395+0000","last_undegraded":"2023-12-19T18:16:41.554395+0000","last_fullsized":"2023-12-19T18:16:41.554395+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:14.215221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359308+0000","last_change":"2023-12-19T18:16:40.558066+0000","last_active":"2023-12-19T18:16:42.359308+0000","last_peered":"2023-12-19T18:16:42.359308+0000","last_clean":"2023-12-19T18:16:42.359308+0000","last_became_active":"2023-12-19T18:16:40.557771+0000","last_became_peered":"2023-12-19T18:16:40.557771+0000","last_unstale":"2023-12-19T18:16:42.359308+0000","last_undegraded":"2023-12-19T18:16:42.359308+0000","last_fullsized":"2023-12-19T18:16:42.359308+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:28.573589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543014+0000","last_change":"2023-12-19T18:16:27.403403+0000","last_active":"2023-12-19T18:16:40.543014+0000","last_peered":"2023-12-19T18:16:40.543014+0000","last_clean":"2023-12-19T18:16:40.543014+0000","last_became_active":"2023-12-19T18:16:27.403121+0000","last_became_peered":"2023-12-19T18:16:27.403121+0000","last_unstale":"2023-12-19T18:16:40.543014+0000","last_undegraded":"2023-12-19T18:16:40.543014+0000","last_fullsized":"2023-12-19T18:16:40.543014+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:36:58.560143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546582+0000","last_change":"2023-12-19T18:16:27.821814+0000","last_active":"2023-12-19T18:16:40.546582+0000","last_peered":"2023-12-19T18:16:40.546582+0000","last_clean":"2023-12-19T18:16:40.546582+0000","last_became_active":"2023-12-19T18:16:27.821519+0000","last_became_peered":"2023-12-19T18:16:27.821519+0000","last_unstale":"2023-12-19T18:16:40.546582+0000","last_undegraded":"2023-12-19T18:16:40.546582+0000","last_fullsized":"2023-12-19T18:16:40.546582+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:40.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.13","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554534+0000","last_change":"2023-12-19T18:16:40.160940+0000","last_active":"2023-12-19T18:16:41.554534+0000","last_peered":"2023-12-19T18:16:41.554534+0000","last_clean":"2023-12-19T18:16:41.554534+0000","last_became_active":"2023-12-19T18:16:40.160594+0000","last_became_peered":"2023-12-19T18:16:40.160594+0000","last_unstale":"2023-12-19T18:16:41.554534+0000","last_undegraded":"2023-12-19T18:16:41.554534+0000","last_fullsized":"2023-12-19T18:16:41.554534+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:01.011110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357451+0000","last_change":"2023-12-19T18:16:40.564547+0000","last_active":"2023-12-19T18:16:42.357451+0000","last_peered":"2023-12-19T18:16:42.357451+0000","last_clean":"2023-12-19T18:16:42.357451+0000","last_became_active":"2023-12-19T18:16:40.564300+0000","last_became_peered":"2023-12-19T18:16:40.564300+0000","last_unstale":"2023-12-19T18:16:42.357451+0000","last_undegraded":"2023-12-19T18:16:42.357451+0000","last_fullsized":"2023-12-19T18:16:42.357451+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:42.276098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556939+0000","last_change":"2023-12-19T18:16:27.417401+0000","last_active":"2023-12-19T18:16:40.556939+0000","last_peered":"2023-12-19T18:16:40.556939+0000","last_clean":"2023-12-19T18:16:40.556939+0000","last_became_active":"2023-12-19T18:16:27.416774+0000","last_became_peered":"2023-12-19T18:16:27.416774+0000","last_unstale":"2023-12-19T18:16:40.556939+0000","last_undegraded":"2023-12-19T18:16:40.556939+0000","last_fullsized":"2023-12-19T18:16:40.556939+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:03.620596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558146+0000","last_change":"2023-12-19T18:16:27.824516+0000","last_active":"2023-12-19T18:16:40.558146+0000","last_peered":"2023-12-19T18:16:40.558146+0000","last_clean":"2023-12-19T18:16:40.558146+0000","last_became_active":"2023-12-19T18:16:27.823143+0000","last_became_peered":"2023-12-19T18:16:27.823143+0000","last_unstale":"2023-12-19T18:16:40.558146+0000","last_undegraded":"2023-12-19T18:16:40.558146+0000","last_fullsized":"2023-12-19T18:16:40.558146+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:52.341930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"215'8","reported_seq":19,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357609+0000","last_change":"2023-12-19T18:16:40.161467+0000","last_active":"2023-12-19T18:16:42.357609+0000","last_peered":"2023-12-19T18:16:42.357609+0000","last_clean":"2023-12-19T18:16:42.357609+0000","last_became_active":"2023-12-19T18:16:40.161143+0000","last_became_peered":"2023-12-19T18:16:40.161143+0000","last_unstale":"2023-12-19T18:16:42.357609+0000","last_undegraded":"2023-12-19T18:16:42.357609+0000","last_fullsized":"2023-12-19T18:16:42.357609+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:21:09.532919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357722+0000","last_change":"2023-12-19T18:16:40.571041+0000","last_active":"2023-12-19T18:16:42.357722+0000","last_peered":"2023-12-19T18:16:42.357722+0000","last_clean":"2023-12-19T18:16:42.357722+0000","last_became_active":"2023-12-19T18:16:40.570705+0000","last_became_peered":"2023-12-19T18:16:40.570705+0000","last_unstale":"2023-12-19T18:16:42.357722+0000","last_undegraded":"2023-12-19T18:16:42.357722+0000","last_fullsized":"2023-12-19T18:16:42.357722+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:43.042802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.13","version":"211'2","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305372+0000","last_change":"2023-12-19T18:16:27.415000+0000","last_active":"2023-12-19T18:16:28.305372+0000","last_peered":"2023-12-19T18:16:28.305372+0000","last_clean":"2023-12-19T18:16:28.305372+0000","last_became_active":"2023-12-19T18:16:27.414679+0000","last_became_peered":"2023-12-19T18:16:27.414679+0000","last_unstale":"2023-12-19T18:16:28.305372+0000","last_undegraded":"2023-12-19T18:16:28.305372+0000","last_fullsized":"2023-12-19T18:16:28.305372+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:50.187832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554789+0000","last_change":"2023-12-19T18:16:27.824833+0000","last_active":"2023-12-19T18:16:40.554789+0000","last_peered":"2023-12-19T18:16:40.554789+0000","last_clean":"2023-12-19T18:16:40.554789+0000","last_became_active":"2023-12-19T18:16:27.824487+0000","last_became_peered":"2023-12-19T18:16:27.824487+0000","last_unstale":"2023-12-19T18:16:40.554789+0000","last_undegraded":"2023-12-19T18:16:40.554789+0000","last_fullsized":"2023-12-19T18:16:40.554789+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550312+0000","last_change":"2023-12-19T18:16:40.150872+0000","last_active":"2023-12-19T18:16:41.550312+0000","last_peered":"2023-12-19T18:16:41.550312+0000","last_clean":"2023-12-19T18:16:41.550312+0000","last_became_active":"2023-12-19T18:16:40.150586+0000","last_became_peered":"2023-12-19T18:16:40.150586+0000","last_unstale":"2023-12-19T18:16:41.550312+0000","last_undegraded":"2023-12-19T18:16:41.550312+0000","last_fullsized":"2023-12-19T18:16:41.550312+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:55.343366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358620+0000","last_change":"2023-12-19T18:16:40.571454+0000","last_active":"2023-12-19T18:16:42.358620+0000","last_peered":"2023-12-19T18:16:42.358620+0000","last_clean":"2023-12-19T18:16:42.358620+0000","last_became_active":"2023-12-19T18:16:40.570299+0000","last_became_peered":"2023-12-19T18:16:40.570299+0000","last_unstale":"2023-12-19T18:16:42.358620+0000","last_undegraded":"2023-12-19T18:16:42.358620+0000","last_fullsized":"2023-12-19T18:16:42.358620+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:31:09.517418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"212'8","reported_seq":23,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553799+0000","last_change":"2023-12-19T18:16:27.414573+0000","last_active":"2023-12-19T18:16:40.553799+0000","last_peered":"2023-12-19T18:16:40.553799+0000","last_clean":"2023-12-19T18:16:40.553799+0000","last_became_active":"2023-12-19T18:16:27.414268+0000","last_became_peered":"2023-12-19T18:16:27.414268+0000","last_unstale":"2023-12-19T18:16:40.553799+0000","last_undegraded":"2023-12-19T18:16:40.553799+0000","last_fullsized":"2023-12-19T18:16:40.553799+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:11.117214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555559+0000","last_change":"2023-12-19T18:16:27.829311+0000","last_active":"2023-12-19T18:16:40.555559+0000","last_peered":"2023-12-19T18:16:40.555559+0000","last_clean":"2023-12-19T18:16:40.555559+0000","last_became_active":"2023-12-19T18:16:27.828953+0000","last_became_peered":"2023-12-19T18:16:27.828953+0000","last_unstale":"2023-12-19T18:16:40.555559+0000","last_undegraded":"2023-12-19T18:16:40.555559+0000","last_fullsized":"2023-12-19T18:16:40.555559+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:23.417710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554767+0000","last_change":"2023-12-19T18:16:40.160698+0000","last_active":"2023-12-19T18:16:41.554767+0000","last_peered":"2023-12-19T18:16:41.554767+0000","last_clean":"2023-12-19T18:16:41.554767+0000","last_became_active":"2023-12-19T18:16:40.160406+0000","last_became_peered":"2023-12-19T18:16:40.160406+0000","last_unstale":"2023-12-19T18:16:41.554767+0000","last_undegraded":"2023-12-19T18:16:41.554767+0000","last_fullsized":"2023-12-19T18:16:41.554767+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:10.441625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"211'3","reported_seq":18,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555258+0000","last_change":"2023-12-19T18:16:27.409442+0000","last_active":"2023-12-19T18:16:40.555258+0000","last_peered":"2023-12-19T18:16:40.555258+0000","last_clean":"2023-12-19T18:16:40.555258+0000","last_became_active":"2023-12-19T18:16:27.409094+0000","last_became_peered":"2023-12-19T18:16:27.409094+0000","last_unstale":"2023-12-19T18:16:40.555258+0000","last_undegraded":"2023-12-19T18:16:40.555258+0000","last_fullsized":"2023-12-19T18:16:40.555258+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:18.713070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562498+0000","last_change":"2023-12-19T18:16:27.811752+0000","last_active":"2023-12-19T18:16:40.562498+0000","last_peered":"2023-12-19T18:16:40.562498+0000","last_clean":"2023-12-19T18:16:40.562498+0000","last_became_active":"2023-12-19T18:16:27.811273+0000","last_became_peered":"2023-12-19T18:16:27.811273+0000","last_unstale":"2023-12-19T18:16:40.562498+0000","last_undegraded":"2023-12-19T18:16:40.562498+0000","last_fullsized":"2023-12-19T18:16:40.562498+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:23.609660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360336+0000","last_change":"2023-12-19T18:16:40.162801+0000","last_active":"2023-12-19T18:16:42.360336+0000","last_peered":"2023-12-19T18:16:42.360336+0000","last_clean":"2023-12-19T18:16:42.360336+0000","last_became_active":"2023-12-19T18:16:40.162367+0000","last_became_peered":"2023-12-19T18:16:40.162367+0000","last_unstale":"2023-12-19T18:16:42.360336+0000","last_undegraded":"2023-12-19T18:16:42.360336+0000","last_fullsized":"2023-12-19T18:16:42.360336+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:08:09.383590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359637+0000","last_change":"2023-12-19T18:16:40.570887+0000","last_active":"2023-12-19T18:16:42.359637+0000","last_peered":"2023-12-19T18:16:42.359637+0000","last_clean":"2023-12-19T18:16:42.359637+0000","last_became_active":"2023-12-19T18:16:40.570475+0000","last_became_peered":"2023-12-19T18:16:40.570475+0000","last_unstale":"2023-12-19T18:16:42.359637+0000","last_undegraded":"2023-12-19T18:16:42.359637+0000","last_fullsized":"2023-12-19T18:16:42.359637+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:53:00.119855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549599+0000","last_change":"2023-12-19T18:16:27.410603+0000","last_active":"2023-12-19T18:16:40.549599+0000","last_peered":"2023-12-19T18:16:40.549599+0000","last_clean":"2023-12-19T18:16:40.549599+0000","last_became_active":"2023-12-19T18:16:27.409260+0000","last_became_peered":"2023-12-19T18:16:27.409260+0000","last_unstale":"2023-12-19T18:16:40.549599+0000","last_undegraded":"2023-12-19T18:16:40.549599+0000","last_fullsized":"2023-12-19T18:16:40.549599+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:06.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543878+0000","last_change":"2023-12-19T18:16:27.835564+0000","last_active":"2023-12-19T18:16:40.543878+0000","last_peered":"2023-12-19T18:16:40.543878+0000","last_clean":"2023-12-19T18:16:40.543878+0000","last_became_active":"2023-12-19T18:16:27.835319+0000","last_became_peered":"2023-12-19T18:16:27.835319+0000","last_unstale":"2023-12-19T18:16:40.543878+0000","last_undegraded":"2023-12-19T18:16:40.543878+0000","last_fullsized":"2023-12-19T18:16:40.543878+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:10:35.037185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:39.117449+0000","last_change":"2023-12-19T18:16:39.117449+0000","last_active":"2023-12-19T18:16:39.117449+0000","last_peered":"2023-12-19T18:16:39.117449+0000","last_clean":"2023-12-19T18:16:39.117449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:39.117449+0000","last_undegraded":"2023-12-19T18:16:39.117449+0000","last_fullsized":"2023-12-19T18:16:39.117449+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565329+0000","last_change":"2023-12-19T18:16:27.405556+0000","last_active":"2023-12-19T18:16:40.565329+0000","last_peered":"2023-12-19T18:16:40.565329+0000","last_clean":"2023-12-19T18:16:40.565329+0000","last_became_active":"2023-12-19T18:16:27.404804+0000","last_became_peered":"2023-12-19T18:16:27.404804+0000","last_unstale":"2023-12-19T18:16:40.565329+0000","last_undegraded":"2023-12-19T18:16:40.565329+0000","last_fullsized":"2023-12-19T18:16:40.565329+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:05.395130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551274+0000","last_change":"2023-12-19T18:16:27.830538+0000","last_active":"2023-12-19T18:16:40.551274+0000","last_peered":"2023-12-19T18:16:40.551274+0000","last_clean":"2023-12-19T18:16:40.551274+0000","last_became_active":"2023-12-19T18:16:27.828974+0000","last_became_peered":"2023-12-19T18:16:27.828974+0000","last_unstale":"2023-12-19T18:16:40.551274+0000","last_undegraded":"2023-12-19T18:16:40.551274+0000","last_fullsized":"2023-12-19T18:16:40.551274+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:36.961741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.15","version":"215'3","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551392+0000","last_change":"2023-12-19T18:16:40.151095+0000","last_active":"2023-12-19T18:16:41.551392+0000","last_peered":"2023-12-19T18:16:41.551392+0000","last_clean":"2023-12-19T18:16:41.551392+0000","last_became_active":"2023-12-19T18:16:40.150609+0000","last_became_peered":"2023-12-19T18:16:40.150609+0000","last_unstale":"2023-12-19T18:16:41.551392+0000","last_undegraded":"2023-12-19T18:16:41.551392+0000","last_fullsized":"2023-12-19T18:16:41.551392+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:15.058778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305298+0000","last_change":"2023-12-19T18:16:27.413444+0000","last_active":"2023-12-19T18:16:28.305298+0000","last_peered":"2023-12-19T18:16:28.305298+0000","last_clean":"2023-12-19T18:16:28.305298+0000","last_became_active":"2023-12-19T18:16:27.413066+0000","last_became_peered":"2023-12-19T18:16:27.413066+0000","last_unstale":"2023-12-19T18:16:28.305298+0000","last_undegraded":"2023-12-19T18:16:28.305298+0000","last_fullsized":"2023-12-19T18:16:28.305298+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:57.997657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558113+0000","last_change":"2023-12-19T18:16:27.816735+0000","last_active":"2023-12-19T18:16:40.558113+0000","last_peered":"2023-12-19T18:16:40.558113+0000","last_clean":"2023-12-19T18:16:40.558113+0000","last_became_active":"2023-12-19T18:16:27.816436+0000","last_became_peered":"2023-12-19T18:16:27.816436+0000","last_unstale":"2023-12-19T18:16:40.558113+0000","last_undegraded":"2023-12-19T18:16:40.558113+0000","last_fullsized":"2023-12-19T18:16:40.558113+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:37.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1a","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359740+0000","last_change":"2023-12-19T18:16:40.161453+0000","last_active":"2023-12-19T18:16:42.359740+0000","last_peered":"2023-12-19T18:16:42.359740+0000","last_clean":"2023-12-19T18:16:42.359740+0000","last_became_active":"2023-12-19T18:16:40.161191+0000","last_became_peered":"2023-12-19T18:16:40.161191+0000","last_unstale":"2023-12-19T18:16:42.359740+0000","last_undegraded":"2023-12-19T18:16:42.359740+0000","last_fullsized":"2023-12-19T18:16:42.359740+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:13.660363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544353+0000","last_change":"2023-12-19T18:16:27.414372+0000","last_active":"2023-12-19T18:16:40.544353+0000","last_peered":"2023-12-19T18:16:40.544353+0000","last_clean":"2023-12-19T18:16:40.544353+0000","last_became_active":"2023-12-19T18:16:27.414088+0000","last_became_peered":"2023-12-19T18:16:27.414088+0000","last_unstale":"2023-12-19T18:16:40.544353+0000","last_undegraded":"2023-12-19T18:16:40.544353+0000","last_fullsized":"2023-12-19T18:16:40.544353+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:18.945766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543503+0000","last_change":"2023-12-19T18:16:27.827386+0000","last_active":"2023-12-19T18:16:40.543503+0000","last_peered":"2023-12-19T18:16:40.543503+0000","last_clean":"2023-12-19T18:16:40.543503+0000","last_became_active":"2023-12-19T18:16:27.827086+0000","last_became_peered":"2023-12-19T18:16:27.827086+0000","last_unstale":"2023-12-19T18:16:40.543503+0000","last_undegraded":"2023-12-19T18:16:40.543503+0000","last_fullsized":"2023-12-19T18:16:40.543503+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:37.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554117+0000","last_change":"2023-12-19T18:16:40.156738+0000","last_active":"2023-12-19T18:16:41.554117+0000","last_peered":"2023-12-19T18:16:41.554117+0000","last_clean":"2023-12-19T18:16:41.554117+0000","last_became_active":"2023-12-19T18:16:40.156488+0000","last_became_peered":"2023-12-19T18:16:40.156488+0000","last_unstale":"2023-12-19T18:16:41.554117+0000","last_undegraded":"2023-12-19T18:16:41.554117+0000","last_fullsized":"2023-12-19T18:16:41.554117+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:48.695934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358850+0000","last_change":"2023-12-19T18:16:40.575709+0000","last_active":"2023-12-19T18:16:42.358850+0000","last_peered":"2023-12-19T18:16:42.358850+0000","last_clean":"2023-12-19T18:16:42.358850+0000","last_became_active":"2023-12-19T18:16:40.575453+0000","last_became_peered":"2023-12-19T18:16:40.575453+0000","last_unstale":"2023-12-19T18:16:42.358850+0000","last_undegraded":"2023-12-19T18:16:42.358850+0000","last_fullsized":"2023-12-19T18:16:42.358850+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:44.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1b","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551641+0000","last_change":"2023-12-19T18:16:27.415036+0000","last_active":"2023-12-19T18:16:40.551641+0000","last_peered":"2023-12-19T18:16:40.551641+0000","last_clean":"2023-12-19T18:16:40.551641+0000","last_became_active":"2023-12-19T18:16:27.414747+0000","last_became_peered":"2023-12-19T18:16:27.414747+0000","last_unstale":"2023-12-19T18:16:40.551641+0000","last_undegraded":"2023-12-19T18:16:40.551641+0000","last_fullsized":"2023-12-19T18:16:40.551641+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:03.064138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556794+0000","last_change":"2023-12-19T18:16:27.828398+0000","last_active":"2023-12-19T18:16:40.556794+0000","last_peered":"2023-12-19T18:16:40.556794+0000","last_clean":"2023-12-19T18:16:40.556794+0000","last_became_active":"2023-12-19T18:16:27.828082+0000","last_became_peered":"2023-12-19T18:16:27.828082+0000","last_unstale":"2023-12-19T18:16:40.556794+0000","last_undegraded":"2023-12-19T18:16:40.556794+0000","last_fullsized":"2023-12-19T18:16:40.556794+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:42:30.998150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358258+0000","last_change":"2023-12-19T18:16:40.159943+0000","last_active":"2023-12-19T18:16:42.358258+0000","last_peered":"2023-12-19T18:16:42.358258+0000","last_clean":"2023-12-19T18:16:42.358258+0000","last_became_active":"2023-12-19T18:16:40.159652+0000","last_became_peered":"2023-12-19T18:16:40.159652+0000","last_unstale":"2023-12-19T18:16:42.358258+0000","last_undegraded":"2023-12-19T18:16:42.358258+0000","last_fullsized":"2023-12-19T18:16:42.358258+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:02:40.928028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.555138+0000","last_change":"2023-12-19T18:16:40.573382+0000","last_active":"2023-12-19T18:16:41.555138+0000","last_peered":"2023-12-19T18:16:41.555138+0000","last_clean":"2023-12-19T18:16:41.555138+0000","last_became_active":"2023-12-19T18:16:40.573092+0000","last_became_peered":"2023-12-19T18:16:40.573092+0000","last_unstale":"2023-12-19T18:16:41.555138+0000","last_undegraded":"2023-12-19T18:16:41.555138+0000","last_fullsized":"2023-12-19T18:16:41.555138+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:10:47.778609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553461+0000","last_change":"2023-12-19T18:16:40.163131+0000","last_active":"2023-12-19T18:16:41.553461+0000","last_peered":"2023-12-19T18:16:41.553461+0000","last_clean":"2023-12-19T18:16:41.553461+0000","last_became_active":"2023-12-19T18:16:40.162621+0000","last_became_peered":"2023-12-19T18:16:40.162621+0000","last_unstale":"2023-12-19T18:16:41.553461+0000","last_undegraded":"2023-12-19T18:16:41.553461+0000","last_fullsized":"2023-12-19T18:16:41.553461+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:57.979757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559871+0000","last_change":"2023-12-19T18:16:27.409441+0000","last_active":"2023-12-19T18:16:40.559871+0000","last_peered":"2023-12-19T18:16:40.559871+0000","last_clean":"2023-12-19T18:16:40.559871+0000","last_became_active":"2023-12-19T18:16:27.408438+0000","last_became_peered":"2023-12-19T18:16:27.408438+0000","last_unstale":"2023-12-19T18:16:40.559871+0000","last_undegraded":"2023-12-19T18:16:40.559871+0000","last_fullsized":"2023-12-19T18:16:40.559871+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:40.980300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546499+0000","last_change":"2023-12-19T18:16:27.814063+0000","last_active":"2023-12-19T18:16:40.546499+0000","last_peered":"2023-12-19T18:16:40.546499+0000","last_clean":"2023-12-19T18:16:40.546499+0000","last_became_active":"2023-12-19T18:16:27.813564+0000","last_became_peered":"2023-12-19T18:16:27.813564+0000","last_unstale":"2023-12-19T18:16:40.546499+0000","last_undegraded":"2023-12-19T18:16:40.546499+0000","last_fullsized":"2023-12-19T18:16:40.546499+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:36.733148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553348+0000","last_change":"2023-12-19T18:16:40.564402+0000","last_active":"2023-12-19T18:16:41.553348+0000","last_peered":"2023-12-19T18:16:41.553348+0000","last_clean":"2023-12-19T18:16:41.553348+0000","last_became_active":"2023-12-19T18:16:40.561338+0000","last_became_peered":"2023-12-19T18:16:40.561338+0000","last_unstale":"2023-12-19T18:16:41.553348+0000","last_undegraded":"2023-12-19T18:16:41.553348+0000","last_fullsized":"2023-12-19T18:16:41.553348+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:37:11.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"215'4","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857590+0000","last_change":"2023-12-19T18:16:40.152799+0000","last_active":"2023-12-19T18:16:41.857590+0000","last_peered":"2023-12-19T18:16:41.857590+0000","last_clean":"2023-12-19T18:16:41.857590+0000","last_became_active":"2023-12-19T18:16:40.152482+0000","last_became_peered":"2023-12-19T18:16:40.152482+0000","last_unstale":"2023-12-19T18:16:41.857590+0000","last_undegraded":"2023-12-19T18:16:41.857590+0000","last_fullsized":"2023-12-19T18:16:41.857590+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:23.854189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1d","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548666+0000","last_change":"2023-12-19T18:16:27.415013+0000","last_active":"2023-12-19T18:16:40.548666+0000","last_peered":"2023-12-19T18:16:40.548666+0000","last_clean":"2023-12-19T18:16:40.548666+0000","last_became_active":"2023-12-19T18:16:27.414582+0000","last_became_peered":"2023-12-19T18:16:27.414582+0000","last_unstale":"2023-12-19T18:16:40.548666+0000","last_undegraded":"2023-12-19T18:16:40.548666+0000","last_fullsized":"2023-12-19T18:16:40.548666+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:34:01.457288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562417+0000","last_change":"2023-12-19T18:16:27.829031+0000","last_active":"2023-12-19T18:16:40.562417+0000","last_peered":"2023-12-19T18:16:40.562417+0000","last_clean":"2023-12-19T18:16:40.562417+0000","last_became_active":"2023-12-19T18:16:27.826634+0000","last_became_peered":"2023-12-19T18:16:27.826634+0000","last_unstale":"2023-12-19T18:16:40.562417+0000","last_undegraded":"2023-12-19T18:16:40.562417+0000","last_fullsized":"2023-12-19T18:16:40.562417+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:23.429228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551066+0000","last_change":"2023-12-19T18:16:40.570487+0000","last_active":"2023-12-19T18:16:41.551066+0000","last_peered":"2023-12-19T18:16:41.551066+0000","last_clean":"2023-12-19T18:16:41.551066+0000","last_became_active":"2023-12-19T18:16:40.569839+0000","last_became_peered":"2023-12-19T18:16:40.569839+0000","last_unstale":"2023-12-19T18:16:41.551066+0000","last_undegraded":"2023-12-19T18:16:41.551066+0000","last_fullsized":"2023-12-19T18:16:41.551066+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:59.414695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548366+0000","last_change":"2023-12-19T18:16:27.407669+0000","last_active":"2023-12-19T18:16:40.548366+0000","last_peered":"2023-12-19T18:16:40.548366+0000","last_clean":"2023-12-19T18:16:40.548366+0000","last_became_active":"2023-12-19T18:16:27.407150+0000","last_became_peered":"2023-12-19T18:16:27.407150+0000","last_unstale":"2023-12-19T18:16:40.548366+0000","last_undegraded":"2023-12-19T18:16:40.548366+0000","last_fullsized":"2023-12-19T18:16:40.548366+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:59.675090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558634+0000","last_change":"2023-12-19T18:16:27.830766+0000","last_active":"2023-12-19T18:16:40.558634+0000","last_peered":"2023-12-19T18:16:40.558634+0000","last_clean":"2023-12-19T18:16:40.558634+0000","last_became_active":"2023-12-19T18:16:27.830220+0000","last_became_peered":"2023-12-19T18:16:27.830220+0000","last_unstale":"2023-12-19T18:16:40.558634+0000","last_undegraded":"2023-12-19T18:16:40.558634+0000","last_fullsized":"2023-12-19T18:16:40.558634+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:03:17.772391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:39.117449+0000","last_change":"2023-12-19T18:16:39.117449+0000","last_active":"2023-12-19T18:16:39.117449+0000","last_peered":"2023-12-19T18:16:39.117449+0000","last_clean":"2023-12-19T18:16:39.117449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:39.117449+0000","last_undegraded":"2023-12-19T18:16:39.117449+0000","last_fullsized":"2023-12-19T18:16:39.117449+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359497+0000","last_change":"2023-12-19T18:16:40.565564+0000","last_active":"2023-12-19T18:16:42.359497+0000","last_peered":"2023-12-19T18:16:42.359497+0000","last_clean":"2023-12-19T18:16:42.359497+0000","last_became_active":"2023-12-19T18:16:40.565300+0000","last_became_peered":"2023-12-19T18:16:40.565300+0000","last_unstale":"2023-12-19T18:16:42.359497+0000","last_undegraded":"2023-12-19T18:16:42.359497+0000","last_fullsized":"2023-12-19T18:16:42.359497+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:14.849896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1f","version":"211'4","reported_seq":19,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553798+0000","last_change":"2023-12-19T18:16:27.414364+0000","last_active":"2023-12-19T18:16:40.553798+0000","last_peered":"2023-12-19T18:16:40.553798+0000","last_clean":"2023-12-19T18:16:40.553798+0000","last_became_active":"2023-12-19T18:16:27.413648+0000","last_became_peered":"2023-12-19T18:16:27.413648+0000","last_unstale":"2023-12-19T18:16:40.553798+0000","last_undegraded":"2023-12-19T18:16:40.553798+0000","last_fullsized":"2023-12-19T18:16:40.553798+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:27:32.703908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544529+0000","last_change":"2023-12-19T18:16:27.830378+0000","last_active":"2023-12-19T18:16:40.544529+0000","last_peered":"2023-12-19T18:16:40.544529+0000","last_clean":"2023-12-19T18:16:40.544529+0000","last_became_active":"2023-12-19T18:16:27.830122+0000","last_became_peered":"2023-12-19T18:16:27.830122+0000","last_unstale":"2023-12-19T18:16:40.544529+0000","last_undegraded":"2023-12-19T18:16:40.544529+0000","last_fullsized":"2023-12-19T18:16:40.544529+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:21.365516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1d","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358451+0000","last_change":"2023-12-19T18:16:40.162566+0000","last_active":"2023-12-19T18:16:42.358451+0000","last_peered":"2023-12-19T18:16:42.358451+0000","last_clean":"2023-12-19T18:16:42.358451+0000","last_became_active":"2023-12-19T18:16:40.162255+0000","last_became_peered":"2023-12-19T18:16:40.162255+0000","last_unstale":"2023-12-19T18:16:42.358451+0000","last_undegraded":"2023-12-19T18:16:42.358451+0000","last_fullsized":"2023-12-19T18:16:42.358451+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:20.530183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552668+0000","last_change":"2023-12-19T18:16:40.570029+0000","last_active":"2023-12-19T18:16:41.552668+0000","last_peered":"2023-12-19T18:16:41.552668+0000","last_clean":"2023-12-19T18:16:41.552668+0000","last_became_active":"2023-12-19T18:16:40.569731+0000","last_became_peered":"2023-12-19T18:16:40.569731+0000","last_unstale":"2023-12-19T18:16:41.552668+0000","last_undegraded":"2023-12-19T18:16:41.552668+0000","last_fullsized":"2023-12-19T18:16:41.552668+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:25:46.635350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550877+0000","last_change":"2023-12-19T18:16:27.831030+0000","last_active":"2023-12-19T18:16:40.550877+0000","last_peered":"2023-12-19T18:16:40.550877+0000","last_clean":"2023-12-19T18:16:40.550877+0000","last_became_active":"2023-12-19T18:16:27.830764+0000","last_became_peered":"2023-12-19T18:16:27.830764+0000","last_unstale":"2023-12-19T18:16:40.550877+0000","last_undegraded":"2023-12-19T18:16:40.550877+0000","last_fullsized":"2023-12-19T18:16:40.550877+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:11.685663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857085+0000","last_change":"2023-12-19T18:16:40.569158+0000","last_active":"2023-12-19T18:16:41.857085+0000","last_peered":"2023-12-19T18:16:41.857085+0000","last_clean":"2023-12-19T18:16:41.857085+0000","last_became_active":"2023-12-19T18:16:40.568894+0000","last_became_peered":"2023-12-19T18:16:40.568894+0000","last_unstale":"2023-12-19T18:16:41.857085+0000","last_undegraded":"2023-12-19T18:16:41.857085+0000","last_fullsized":"2023-12-19T18:16:41.857085+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:02.018340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553603+0000","last_change":"2023-12-19T18:16:27.833518+0000","last_active":"2023-12-19T18:16:40.553603+0000","last_peered":"2023-12-19T18:16:40.553603+0000","last_clean":"2023-12-19T18:16:40.553603+0000","last_became_active":"2023-12-19T18:16:27.833251+0000","last_became_peered":"2023-12-19T18:16:27.833251+0000","last_unstale":"2023-12-19T18:16:40.553603+0000","last_undegraded":"2023-12-19T18:16:40.553603+0000","last_fullsized":"2023-12-19T18:16:40.553603+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:49.276917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357285+0000","last_change":"2023-12-19T18:16:40.563610+0000","last_active":"2023-12-19T18:16:42.357285+0000","last_peered":"2023-12-19T18:16:42.357285+0000","last_clean":"2023-12-19T18:16:42.357285+0000","last_became_active":"2023-12-19T18:16:40.563300+0000","last_became_peered":"2023-12-19T18:16:40.563300+0000","last_unstale":"2023-12-19T18:16:42.357285+0000","last_undegraded":"2023-12-19T18:16:42.357285+0000","last_fullsized":"2023-12-19T18:16:42.357285+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:28:48.518493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547548+0000","last_change":"2023-12-19T18:16:27.825014+0000","last_active":"2023-12-19T18:16:40.547548+0000","last_peered":"2023-12-19T18:16:40.547548+0000","last_clean":"2023-12-19T18:16:40.547548+0000","last_became_active":"2023-12-19T18:16:27.822140+0000","last_became_peered":"2023-12-19T18:16:27.822140+0000","last_unstale":"2023-12-19T18:16:40.547548+0000","last_undegraded":"2023-12-19T18:16:40.547548+0000","last_fullsized":"2023-12-19T18:16:40.547548+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:36.972809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550313+0000","last_change":"2023-12-19T18:16:40.576594+0000","last_active":"2023-12-19T18:16:41.550313+0000","last_peered":"2023-12-19T18:16:41.550313+0000","last_clean":"2023-12-19T18:16:41.550313+0000","last_became_active":"2023-12-19T18:16:40.576304+0000","last_became_peered":"2023-12-19T18:16:40.576304+0000","last_unstale":"2023-12-19T18:16:41.550313+0000","last_undegraded":"2023-12-19T18:16:41.550313+0000","last_fullsized":"2023-12-19T18:16:41.550313+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:58.373923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561436+0000","last_change":"2023-12-19T18:16:27.826500+0000","last_active":"2023-12-19T18:16:40.561436+0000","last_peered":"2023-12-19T18:16:40.561436+0000","last_clean":"2023-12-19T18:16:40.561436+0000","last_became_active":"2023-12-19T18:16:27.826216+0000","last_became_peered":"2023-12-19T18:16:27.826216+0000","last_unstale":"2023-12-19T18:16:40.561436+0000","last_undegraded":"2023-12-19T18:16:40.561436+0000","last_fullsized":"2023-12-19T18:16:40.561436+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:33.032362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359854+0000","last_change":"2023-12-19T18:16:40.565942+0000","last_active":"2023-12-19T18:16:42.359854+0000","last_peered":"2023-12-19T18:16:42.359854+0000","last_clean":"2023-12-19T18:16:42.359854+0000","last_became_active":"2023-12-19T18:16:40.565557+0000","last_became_peered":"2023-12-19T18:16:40.565557+0000","last_unstale":"2023-12-19T18:16:42.359854+0000","last_undegraded":"2023-12-19T18:16:42.359854+0000","last_fullsized":"2023-12-19T18:16:42.359854+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:23.839152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548841+0000","last_change":"2023-12-19T18:16:27.830103+0000","last_active":"2023-12-19T18:16:40.548841+0000","last_peered":"2023-12-19T18:16:40.548841+0000","last_clean":"2023-12-19T18:16:40.548841+0000","last_became_active":"2023-12-19T18:16:27.829444+0000","last_became_peered":"2023-12-19T18:16:27.829444+0000","last_unstale":"2023-12-19T18:16:40.548841+0000","last_undegraded":"2023-12-19T18:16:40.548841+0000","last_fullsized":"2023-12-19T18:16:40.548841+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:50:29.947932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553462+0000","last_change":"2023-12-19T18:16:40.563980+0000","last_active":"2023-12-19T18:16:41.553462+0000","last_peered":"2023-12-19T18:16:41.553462+0000","last_clean":"2023-12-19T18:16:41.553462+0000","last_became_active":"2023-12-19T18:16:40.563683+0000","last_became_peered":"2023-12-19T18:16:40.563683+0000","last_unstale":"2023-12-19T18:16:41.553462+0000","last_undegraded":"2023-12-19T18:16:41.553462+0000","last_fullsized":"2023-12-19T18:16:41.553462+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:36.170652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548463+0000","last_change":"2023-12-19T18:16:27.820913+0000","last_active":"2023-12-19T18:16:40.548463+0000","last_peered":"2023-12-19T18:16:40.548463+0000","last_clean":"2023-12-19T18:16:40.548463+0000","last_became_active":"2023-12-19T18:16:27.820634+0000","last_became_peered":"2023-12-19T18:16:27.820634+0000","last_unstale":"2023-12-19T18:16:40.548463+0000","last_undegraded":"2023-12-19T18:16:40.548463+0000","last_fullsized":"2023-12-19T18:16:40.548463+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:53:09.066279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357879+0000","last_change":"2023-12-19T18:16:40.562872+0000","last_active":"2023-12-19T18:16:42.357879+0000","last_peered":"2023-12-19T18:16:42.357879+0000","last_clean":"2023-12-19T18:16:42.357879+0000","last_became_active":"2023-12-19T18:16:40.562615+0000","last_became_peered":"2023-12-19T18:16:40.562615+0000","last_unstale":"2023-12-19T18:16:42.357879+0000","last_undegraded":"2023-12-19T18:16:42.357879+0000","last_fullsized":"2023-12-19T18:16:42.357879+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:54.454071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565322+0000","last_change":"2023-12-19T18:16:27.830119+0000","last_active":"2023-12-19T18:16:40.565322+0000","last_peered":"2023-12-19T18:16:40.565322+0000","last_clean":"2023-12-19T18:16:40.565322+0000","last_became_active":"2023-12-19T18:16:27.829850+0000","last_became_peered":"2023-12-19T18:16:27.829850+0000","last_unstale":"2023-12-19T18:16:40.565322+0000","last_undegraded":"2023-12-19T18:16:40.565322+0000","last_fullsized":"2023-12-19T18:16:40.565322+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:51.190615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857238+0000","last_change":"2023-12-19T18:16:40.570092+0000","last_active":"2023-12-19T18:16:41.857238+0000","last_peered":"2023-12-19T18:16:41.857238+0000","last_clean":"2023-12-19T18:16:41.857238+0000","last_became_active":"2023-12-19T18:16:40.568697+0000","last_became_peered":"2023-12-19T18:16:40.568697+0000","last_unstale":"2023-12-19T18:16:41.857238+0000","last_undegraded":"2023-12-19T18:16:41.857238+0000","last_fullsized":"2023-12-19T18:16:41.857238+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:11.448855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.564229+0000","last_change":"2023-12-19T18:16:27.834875+0000","last_active":"2023-12-19T18:16:40.564229+0000","last_peered":"2023-12-19T18:16:40.564229+0000","last_clean":"2023-12-19T18:16:40.564229+0000","last_became_active":"2023-12-19T18:16:27.834561+0000","last_became_peered":"2023-12-19T18:16:27.834561+0000","last_unstale":"2023-12-19T18:16:40.564229+0000","last_undegraded":"2023-12-19T18:16:40.564229+0000","last_fullsized":"2023-12-19T18:16:40.564229+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:30.301379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554348+0000","last_change":"2023-12-19T18:16:40.561527+0000","last_active":"2023-12-19T18:16:41.554348+0000","last_peered":"2023-12-19T18:16:41.554348+0000","last_clean":"2023-12-19T18:16:41.554348+0000","last_became_active":"2023-12-19T18:16:40.561251+0000","last_became_peered":"2023-12-19T18:16:40.561251+0000","last_unstale":"2023-12-19T18:16:41.554348+0000","last_undegraded":"2023-12-19T18:16:41.554348+0000","last_fullsized":"2023-12-19T18:16:41.554348+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:50.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543417+0000","last_change":"2023-12-19T18:16:27.826642+0000","last_active":"2023-12-19T18:16:40.543417+0000","last_peered":"2023-12-19T18:16:40.543417+0000","last_clean":"2023-12-19T18:16:40.543417+0000","last_became_active":"2023-12-19T18:16:27.825460+0000","last_became_peered":"2023-12-19T18:16:27.825460+0000","last_unstale":"2023-12-19T18:16:40.543417+0000","last_undegraded":"2023-12-19T18:16:40.543417+0000","last_fullsized":"2023-12-19T18:16:40.543417+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:05.671602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551313+0000","last_change":"2023-12-19T18:16:27.826145+0000","last_active":"2023-12-19T18:16:40.551313+0000","last_peered":"2023-12-19T18:16:40.551313+0000","last_clean":"2023-12-19T18:16:40.551313+0000","last_became_active":"2023-12-19T18:16:27.825889+0000","last_became_peered":"2023-12-19T18:16:27.825889+0000","last_unstale":"2023-12-19T18:16:40.551313+0000","last_undegraded":"2023-12-19T18:16:40.551313+0000","last_fullsized":"2023-12-19T18:16:40.551313+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:59:41.673539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555865+0000","last_change":"2023-12-19T18:16:27.830084+0000","last_active":"2023-12-19T18:16:40.555865+0000","last_peered":"2023-12-19T18:16:40.555865+0000","last_clean":"2023-12-19T18:16:40.555865+0000","last_became_active":"2023-12-19T18:16:27.829787+0000","last_became_peered":"2023-12-19T18:16:27.829787+0000","last_unstale":"2023-12-19T18:16:40.555865+0000","last_undegraded":"2023-12-19T18:16:40.555865+0000","last_fullsized":"2023-12-19T18:16:40.555865+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:34:16.356894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554193+0000","last_change":"2023-12-19T18:16:40.563996+0000","last_active":"2023-12-19T18:16:41.554193+0000","last_peered":"2023-12-19T18:16:41.554193+0000","last_clean":"2023-12-19T18:16:41.554193+0000","last_became_active":"2023-12-19T18:16:40.563686+0000","last_became_peered":"2023-12-19T18:16:40.563686+0000","last_unstale":"2023-12-19T18:16:41.554193+0000","last_undegraded":"2023-12-19T18:16:41.554193+0000","last_fullsized":"2023-12-19T18:16:41.554193+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:34.106940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551542+0000","last_change":"2023-12-19T18:16:27.831551+0000","last_active":"2023-12-19T18:16:40.551542+0000","last_peered":"2023-12-19T18:16:40.551542+0000","last_clean":"2023-12-19T18:16:40.551542+0000","last_became_active":"2023-12-19T18:16:27.831291+0000","last_became_peered":"2023-12-19T18:16:27.831291+0000","last_unstale":"2023-12-19T18:16:40.551542+0000","last_undegraded":"2023-12-19T18:16:40.551542+0000","last_fullsized":"2023-12-19T18:16:40.551542+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:09.914084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551186+0000","last_change":"2023-12-19T18:16:40.563764+0000","last_active":"2023-12-19T18:16:41.551186+0000","last_peered":"2023-12-19T18:16:41.551186+0000","last_clean":"2023-12-19T18:16:41.551186+0000","last_became_active":"2023-12-19T18:16:40.563464+0000","last_became_peered":"2023-12-19T18:16:40.563464+0000","last_unstale":"2023-12-19T18:16:41.551186+0000","last_undegraded":"2023-12-19T18:16:41.551186+0000","last_fullsized":"2023-12-19T18:16:41.551186+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:52:34.311679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548663+0000","last_change":"2023-12-19T18:16:27.833054+0000","last_active":"2023-12-19T18:16:40.548663+0000","last_peered":"2023-12-19T18:16:40.548663+0000","last_clean":"2023-12-19T18:16:40.548663+0000","last_became_active":"2023-12-19T18:16:27.832738+0000","last_became_peered":"2023-12-19T18:16:27.832738+0000","last_unstale":"2023-12-19T18:16:40.548663+0000","last_undegraded":"2023-12-19T18:16:40.548663+0000","last_fullsized":"2023-12-19T18:16:40.548663+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:57.892685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360068+0000","last_change":"2023-12-19T18:16:40.577515+0000","last_active":"2023-12-19T18:16:42.360068+0000","last_peered":"2023-12-19T18:16:42.360068+0000","last_clean":"2023-12-19T18:16:42.360068+0000","last_became_active":"2023-12-19T18:16:40.577096+0000","last_became_peered":"2023-12-19T18:16:40.577096+0000","last_unstale":"2023-12-19T18:16:42.360068+0000","last_undegraded":"2023-12-19T18:16:42.360068+0000","last_fullsized":"2023-12-19T18:16:42.360068+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:36.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555479+0000","last_change":"2023-12-19T18:16:27.820194+0000","last_active":"2023-12-19T18:16:40.555479+0000","last_peered":"2023-12-19T18:16:40.555479+0000","last_clean":"2023-12-19T18:16:40.555479+0000","last_became_active":"2023-12-19T18:16:27.819900+0000","last_became_peered":"2023-12-19T18:16:27.819900+0000","last_unstale":"2023-12-19T18:16:40.555479+0000","last_undegraded":"2023-12-19T18:16:40.555479+0000","last_fullsized":"2023-12-19T18:16:40.555479+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:00.858729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360137+0000","last_change":"2023-12-19T18:16:40.561878+0000","last_active":"2023-12-19T18:16:42.360137+0000","last_peered":"2023-12-19T18:16:42.360137+0000","last_clean":"2023-12-19T18:16:42.360137+0000","last_became_active":"2023-12-19T18:16:40.561626+0000","last_became_peered":"2023-12-19T18:16:40.561626+0000","last_unstale":"2023-12-19T18:16:42.360137+0000","last_undegraded":"2023-12-19T18:16:42.360137+0000","last_fullsized":"2023-12-19T18:16:42.360137+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:38:57.938991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305416+0000","last_change":"2023-12-19T18:16:27.820781+0000","last_active":"2023-12-19T18:16:28.305416+0000","last_peered":"2023-12-19T18:16:28.305416+0000","last_clean":"2023-12-19T18:16:28.305416+0000","last_became_active":"2023-12-19T18:16:27.820483+0000","last_became_peered":"2023-12-19T18:16:27.820483+0000","last_unstale":"2023-12-19T18:16:28.305416+0000","last_undegraded":"2023-12-19T18:16:28.305416+0000","last_fullsized":"2023-12-19T18:16:28.305416+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:23.445435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551324+0000","last_change":"2023-12-19T18:16:40.560485+0000","last_active":"2023-12-19T18:16:41.551324+0000","last_peered":"2023-12-19T18:16:41.551324+0000","last_clean":"2023-12-19T18:16:41.551324+0000","last_became_active":"2023-12-19T18:16:40.560232+0000","last_became_peered":"2023-12-19T18:16:40.560232+0000","last_unstale":"2023-12-19T18:16:41.551324+0000","last_undegraded":"2023-12-19T18:16:41.551324+0000","last_fullsized":"2023-12-19T18:16:41.551324+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:30.252976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553708+0000","last_change":"2023-12-19T18:16:27.834880+0000","last_active":"2023-12-19T18:16:40.553708+0000","last_peered":"2023-12-19T18:16:40.553708+0000","last_clean":"2023-12-19T18:16:40.553708+0000","last_became_active":"2023-12-19T18:16:27.834494+0000","last_became_peered":"2023-12-19T18:16:27.834494+0000","last_unstale":"2023-12-19T18:16:40.553708+0000","last_undegraded":"2023-12-19T18:16:40.553708+0000","last_fullsized":"2023-12-19T18:16:40.553708+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:06.992701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357587+0000","last_change":"2023-12-19T18:16:40.567126+0000","last_active":"2023-12-19T18:16:42.357587+0000","last_peered":"2023-12-19T18:16:42.357587+0000","last_clean":"2023-12-19T18:16:42.357587+0000","last_became_active":"2023-12-19T18:16:40.566878+0000","last_became_peered":"2023-12-19T18:16:40.566878+0000","last_unstale":"2023-12-19T18:16:42.357587+0000","last_undegraded":"2023-12-19T18:16:42.357587+0000","last_fullsized":"2023-12-19T18:16:42.357587+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:59.062853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543300+0000","last_change":"2023-12-19T18:16:27.826828+0000","last_active":"2023-12-19T18:16:40.543300+0000","last_peered":"2023-12-19T18:16:40.543300+0000","last_clean":"2023-12-19T18:16:40.543300+0000","last_became_active":"2023-12-19T18:16:27.825994+0000","last_became_peered":"2023-12-19T18:16:27.825994+0000","last_unstale":"2023-12-19T18:16:40.543300+0000","last_undegraded":"2023-12-19T18:16:40.543300+0000","last_fullsized":"2023-12-19T18:16:40.543300+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:17.748218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857358+0000","last_change":"2023-12-19T18:16:40.569008+0000","last_active":"2023-12-19T18:16:41.857358+0000","last_peered":"2023-12-19T18:16:41.857358+0000","last_clean":"2023-12-19T18:16:41.857358+0000","last_became_active":"2023-12-19T18:16:40.568420+0000","last_became_peered":"2023-12-19T18:16:40.568420+0000","last_unstale":"2023-12-19T18:16:41.857358+0000","last_undegraded":"2023-12-19T18:16:41.857358+0000","last_fullsized":"2023-12-19T18:16:41.857358+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:31:10.720873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561285+0000","last_change":"2023-12-19T18:16:27.830751+0000","last_active":"2023-12-19T18:16:40.561285+0000","last_peered":"2023-12-19T18:16:40.561285+0000","last_clean":"2023-12-19T18:16:40.561285+0000","last_became_active":"2023-12-19T18:16:27.830490+0000","last_became_peered":"2023-12-19T18:16:27.830490+0000","last_unstale":"2023-12-19T18:16:40.561285+0000","last_undegraded":"2023-12-19T18:16:40.561285+0000","last_fullsized":"2023-12-19T18:16:40.561285+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:31.093343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550245+0000","last_change":"2023-12-19T18:16:40.571696+0000","last_active":"2023-12-19T18:16:41.550245+0000","last_peered":"2023-12-19T18:16:41.550245+0000","last_clean":"2023-12-19T18:16:41.550245+0000","last_became_active":"2023-12-19T18:16:40.571407+0000","last_became_peered":"2023-12-19T18:16:40.571407+0000","last_unstale":"2023-12-19T18:16:41.550245+0000","last_undegraded":"2023-12-19T18:16:41.550245+0000","last_fullsized":"2023-12-19T18:16:41.550245+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:55.222921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548271+0000","last_change":"2023-12-19T18:16:27.823186+0000","last_active":"2023-12-19T18:16:40.548271+0000","last_peered":"2023-12-19T18:16:40.548271+0000","last_clean":"2023-12-19T18:16:40.548271+0000","last_became_active":"2023-12-19T18:16:27.820405+0000","last_became_peered":"2023-12-19T18:16:27.820405+0000","last_unstale":"2023-12-19T18:16:40.548271+0000","last_undegraded":"2023-12-19T18:16:40.548271+0000","last_fullsized":"2023-12-19T18:16:40.548271+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:41.515700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357953+0000","last_change":"2023-12-19T18:16:40.569297+0000","last_active":"2023-12-19T18:16:42.357953+0000","last_peered":"2023-12-19T18:16:42.357953+0000","last_clean":"2023-12-19T18:16:42.357953+0000","last_became_active":"2023-12-19T18:16:40.569050+0000","last_became_peered":"2023-12-19T18:16:40.569050+0000","last_unstale":"2023-12-19T18:16:42.357953+0000","last_undegraded":"2023-12-19T18:16:42.357953+0000","last_fullsized":"2023-12-19T18:16:42.357953+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:28:56.831758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561350+0000","last_change":"2023-12-19T18:16:27.829736+0000","last_active":"2023-12-19T18:16:40.561350+0000","last_peered":"2023-12-19T18:16:40.561350+0000","last_clean":"2023-12-19T18:16:40.561350+0000","last_became_active":"2023-12-19T18:16:27.829448+0000","last_became_peered":"2023-12-19T18:16:27.829448+0000","last_unstale":"2023-12-19T18:16:40.561350+0000","last_undegraded":"2023-12-19T18:16:40.561350+0000","last_fullsized":"2023-12-19T18:16:40.561350+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:47.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551154+0000","last_change":"2023-12-19T18:16:40.559877+0000","last_active":"2023-12-19T18:16:41.551154+0000","last_peered":"2023-12-19T18:16:41.551154+0000","last_clean":"2023-12-19T18:16:41.551154+0000","last_became_active":"2023-12-19T18:16:40.558100+0000","last_became_peered":"2023-12-19T18:16:40.558100+0000","last_unstale":"2023-12-19T18:16:41.551154+0000","last_undegraded":"2023-12-19T18:16:41.551154+0000","last_fullsized":"2023-12-19T18:16:41.551154+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:44.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559776+0000","last_change":"2023-12-19T18:16:27.829943+0000","last_active":"2023-12-19T18:16:40.559776+0000","last_peered":"2023-12-19T18:16:40.559776+0000","last_clean":"2023-12-19T18:16:40.559776+0000","last_became_active":"2023-12-19T18:16:27.829592+0000","last_became_peered":"2023-12-19T18:16:27.829592+0000","last_unstale":"2023-12-19T18:16:40.559776+0000","last_undegraded":"2023-12-19T18:16:40.559776+0000","last_fullsized":"2023-12-19T18:16:40.559776+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:46.245564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553575+0000","last_change":"2023-12-19T18:16:40.575384+0000","last_active":"2023-12-19T18:16:41.553575+0000","last_peered":"2023-12-19T18:16:41.553575+0000","last_clean":"2023-12-19T18:16:41.553575+0000","last_became_active":"2023-12-19T18:16:40.575124+0000","last_became_peered":"2023-12-19T18:16:40.575124+0000","last_unstale":"2023-12-19T18:16:41.553575+0000","last_undegraded":"2023-12-19T18:16:41.553575+0000","last_fullsized":"2023-12-19T18:16:41.553575+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:53.796369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565426+0000","last_change":"2023-12-19T18:16:27.819797+0000","last_active":"2023-12-19T18:16:40.565426+0000","last_peered":"2023-12-19T18:16:40.565426+0000","last_clean":"2023-12-19T18:16:40.565426+0000","last_became_active":"2023-12-19T18:16:27.819507+0000","last_became_peered":"2023-12-19T18:16:27.819507+0000","last_unstale":"2023-12-19T18:16:40.565426+0000","last_undegraded":"2023-12-19T18:16:40.565426+0000","last_fullsized":"2023-12-19T18:16:40.565426+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:08.287716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857264+0000","last_change":"2023-12-19T18:16:40.570173+0000","last_active":"2023-12-19T18:16:41.857264+0000","last_peered":"2023-12-19T18:16:41.857264+0000","last_clean":"2023-12-19T18:16:41.857264+0000","last_became_active":"2023-12-19T18:16:40.569912+0000","last_became_peered":"2023-12-19T18:16:40.569912+0000","last_unstale":"2023-12-19T18:16:41.857264+0000","last_undegraded":"2023-12-19T18:16:41.857264+0000","last_fullsized":"2023-12-19T18:16:41.857264+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:47.688949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558633+0000","last_change":"2023-12-19T18:16:27.819945+0000","last_active":"2023-12-19T18:16:40.558633+0000","last_peered":"2023-12-19T18:16:40.558633+0000","last_clean":"2023-12-19T18:16:40.558633+0000","last_became_active":"2023-12-19T18:16:27.819634+0000","last_became_peered":"2023-12-19T18:16:27.819634+0000","last_unstale":"2023-12-19T18:16:40.558633+0000","last_undegraded":"2023-12-19T18:16:40.558633+0000","last_fullsized":"2023-12-19T18:16:40.558633+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:22.401852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358420+0000","last_change":"2023-12-19T18:16:40.566307+0000","last_active":"2023-12-19T18:16:42.358420+0000","last_peered":"2023-12-19T18:16:42.358420+0000","last_clean":"2023-12-19T18:16:42.358420+0000","last_became_active":"2023-12-19T18:16:40.566059+0000","last_became_peered":"2023-12-19T18:16:40.566059+0000","last_unstale":"2023-12-19T18:16:42.358420+0000","last_undegraded":"2023-12-19T18:16:42.358420+0000","last_fullsized":"2023-12-19T18:16:42.358420+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:45.773786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549949+0000","last_change":"2023-12-19T18:16:27.832580+0000","last_active":"2023-12-19T18:16:40.549949+0000","last_peered":"2023-12-19T18:16:40.549949+0000","last_clean":"2023-12-19T18:16:40.549949+0000","last_became_active":"2023-12-19T18:16:27.832291+0000","last_became_peered":"2023-12-19T18:16:27.832291+0000","last_unstale":"2023-12-19T18:16:40.549949+0000","last_undegraded":"2023-12-19T18:16:40.549949+0000","last_fullsized":"2023-12-19T18:16:40.549949+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:29.305405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359160+0000","last_change":"2023-12-19T18:16:40.567943+0000","last_active":"2023-12-19T18:16:42.359160+0000","last_peered":"2023-12-19T18:16:42.359160+0000","last_clean":"2023-12-19T18:16:42.359160+0000","last_became_active":"2023-12-19T18:16:40.567701+0000","last_became_peered":"2023-12-19T18:16:40.567701+0000","last_unstale":"2023-12-19T18:16:42.359160+0000","last_undegraded":"2023-12-19T18:16:42.359160+0000","last_fullsized":"2023-12-19T18:16:42.359160+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:50:31.894613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543104+0000","last_change":"2023-12-19T18:16:27.825730+0000","last_active":"2023-12-19T18:16:40.543104+0000","last_peered":"2023-12-19T18:16:40.543104+0000","last_clean":"2023-12-19T18:16:40.543104+0000","last_became_active":"2023-12-19T18:16:27.825136+0000","last_became_peered":"2023-12-19T18:16:27.825136+0000","last_unstale":"2023-12-19T18:16:40.543104+0000","last_undegraded":"2023-12-19T18:16:40.543104+0000","last_fullsized":"2023-12-19T18:16:40.543104+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:48.330735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359437+0000","last_change":"2023-12-19T18:16:40.558188+0000","last_active":"2023-12-19T18:16:42.359437+0000","last_peered":"2023-12-19T18:16:42.359437+0000","last_clean":"2023-12-19T18:16:42.359437+0000","last_became_active":"2023-12-19T18:16:40.557804+0000","last_became_peered":"2023-12-19T18:16:40.557804+0000","last_unstale":"2023-12-19T18:16:42.359437+0000","last_undegraded":"2023-12-19T18:16:42.359437+0000","last_fullsized":"2023-12-19T18:16:42.359437+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:35:20.500418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558018+0000","last_change":"2023-12-19T18:16:27.823852+0000","last_active":"2023-12-19T18:16:40.558018+0000","last_peered":"2023-12-19T18:16:40.558018+0000","last_clean":"2023-12-19T18:16:40.558018+0000","last_became_active":"2023-12-19T18:16:27.823564+0000","last_became_peered":"2023-12-19T18:16:27.823564+0000","last_unstale":"2023-12-19T18:16:40.558018+0000","last_undegraded":"2023-12-19T18:16:40.558018+0000","last_fullsized":"2023-12-19T18:16:40.558018+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:59.000440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358756+0000","last_change":"2023-12-19T18:16:40.561015+0000","last_active":"2023-12-19T18:16:42.358756+0000","last_peered":"2023-12-19T18:16:42.358756+0000","last_clean":"2023-12-19T18:16:42.358756+0000","last_became_active":"2023-12-19T18:16:40.560771+0000","last_became_peered":"2023-12-19T18:16:40.560771+0000","last_unstale":"2023-12-19T18:16:42.358756+0000","last_undegraded":"2023-12-19T18:16:42.358756+0000","last_fullsized":"2023-12-19T18:16:42.358756+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:04.628464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554625+0000","last_change":"2023-12-19T18:16:27.822167+0000","last_active":"2023-12-19T18:16:40.554625+0000","last_peered":"2023-12-19T18:16:40.554625+0000","last_clean":"2023-12-19T18:16:40.554625+0000","last_became_active":"2023-12-19T18:16:27.821752+0000","last_became_peered":"2023-12-19T18:16:27.821752+0000","last_unstale":"2023-12-19T18:16:40.554625+0000","last_undegraded":"2023-12-19T18:16:40.554625+0000","last_fullsized":"2023-12-19T18:16:40.554625+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:17.866181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357777+0000","last_change":"2023-12-19T18:16:40.565973+0000","last_active":"2023-12-19T18:16:42.357777+0000","last_peered":"2023-12-19T18:16:42.357777+0000","last_clean":"2023-12-19T18:16:42.357777+0000","last_became_active":"2023-12-19T18:16:40.565418+0000","last_became_peered":"2023-12-19T18:16:40.565418+0000","last_unstale":"2023-12-19T18:16:42.357777+0000","last_undegraded":"2023-12-19T18:16:42.357777+0000","last_fullsized":"2023-12-19T18:16:42.357777+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:54.473566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551716+0000","last_change":"2023-12-19T18:16:27.835673+0000","last_active":"2023-12-19T18:16:40.551716+0000","last_peered":"2023-12-19T18:16:40.551716+0000","last_clean":"2023-12-19T18:16:40.551716+0000","last_became_active":"2023-12-19T18:16:27.835392+0000","last_became_peered":"2023-12-19T18:16:27.835392+0000","last_unstale":"2023-12-19T18:16:40.551716+0000","last_undegraded":"2023-12-19T18:16:40.551716+0000","last_fullsized":"2023-12-19T18:16:40.551716+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:23:06.990327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553304+0000","last_change":"2023-12-19T18:16:40.564946+0000","last_active":"2023-12-19T18:16:41.553304+0000","last_peered":"2023-12-19T18:16:41.553304+0000","last_clean":"2023-12-19T18:16:41.553304+0000","last_became_active":"2023-12-19T18:16:40.564679+0000","last_became_peered":"2023-12-19T18:16:40.564679+0000","last_unstale":"2023-12-19T18:16:41.553304+0000","last_undegraded":"2023-12-19T18:16:41.553304+0000","last_fullsized":"2023-12-19T18:16:41.553304+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:02:28.048063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559694+0000","last_change":"2023-12-19T18:16:27.830449+0000","last_active":"2023-12-19T18:16:40.559694+0000","last_peered":"2023-12-19T18:16:40.559694+0000","last_clean":"2023-12-19T18:16:40.559694+0000","last_became_active":"2023-12-19T18:16:27.830189+0000","last_became_peered":"2023-12-19T18:16:27.830189+0000","last_unstale":"2023-12-19T18:16:40.559694+0000","last_undegraded":"2023-12-19T18:16:40.559694+0000","last_fullsized":"2023-12-19T18:16:40.559694+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:06.397912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360333+0000","last_change":"2023-12-19T18:16:40.576744+0000","last_active":"2023-12-19T18:16:42.360333+0000","last_peered":"2023-12-19T18:16:42.360333+0000","last_clean":"2023-12-19T18:16:42.360333+0000","last_became_active":"2023-12-19T18:16:40.576425+0000","last_became_peered":"2023-12-19T18:16:40.576425+0000","last_unstale":"2023-12-19T18:16:42.360333+0000","last_undegraded":"2023-12-19T18:16:42.360333+0000","last_fullsized":"2023-12-19T18:16:42.360333+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:26.259551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548052+0000","last_change":"2023-12-19T18:16:27.815730+0000","last_active":"2023-12-19T18:16:40.548052+0000","last_peered":"2023-12-19T18:16:40.548052+0000","last_clean":"2023-12-19T18:16:40.548052+0000","last_became_active":"2023-12-19T18:16:27.815341+0000","last_became_peered":"2023-12-19T18:16:27.815341+0000","last_unstale":"2023-12-19T18:16:40.548052+0000","last_undegraded":"2023-12-19T18:16:40.548052+0000","last_fullsized":"2023-12-19T18:16:40.548052+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:39.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562588+0000","last_change":"2023-12-19T18:16:27.831511+0000","last_active":"2023-12-19T18:16:40.562588+0000","last_peered":"2023-12-19T18:16:40.562588+0000","last_clean":"2023-12-19T18:16:40.562588+0000","last_became_active":"2023-12-19T18:16:27.831255+0000","last_became_peered":"2023-12-19T18:16:27.831255+0000","last_unstale":"2023-12-19T18:16:40.562588+0000","last_undegraded":"2023-12-19T18:16:40.562588+0000","last_fullsized":"2023-12-19T18:16:40.562588+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:42.382254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358249+0000","last_change":"2023-12-19T18:16:40.576919+0000","last_active":"2023-12-19T18:16:42.358249+0000","last_peered":"2023-12-19T18:16:42.358249+0000","last_clean":"2023-12-19T18:16:42.358249+0000","last_became_active":"2023-12-19T18:16:40.576351+0000","last_became_peered":"2023-12-19T18:16:40.576351+0000","last_unstale":"2023-12-19T18:16:42.358249+0000","last_undegraded":"2023-12-19T18:16:42.358249+0000","last_fullsized":"2023-12-19T18:16:42.358249+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:36.351083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305437+0000","last_change":"2023-12-19T18:16:27.834221+0000","last_active":"2023-12-19T18:16:28.305437+0000","last_peered":"2023-12-19T18:16:28.305437+0000","last_clean":"2023-12-19T18:16:28.305437+0000","last_became_active":"2023-12-19T18:16:27.825886+0000","last_became_peered":"2023-12-19T18:16:27.825886+0000","last_unstale":"2023-12-19T18:16:28.305437+0000","last_undegraded":"2023-12-19T18:16:28.305437+0000","last_fullsized":"2023-12-19T18:16:28.305437+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:12.119996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":56,"acting":56,"num_store_stats":7},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510568,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64556,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307284,"statfs":{"total":96636764160,"available":96570658816,"internally_reserved":0,"allocated":3248128,"data_stored":1989677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96399999999999997,"5min":1.048,"15min":1.1180000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92400000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.70899999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92400000000000004,"5min":1.0389999999999999,"15min":1.0580000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72399999999999998,"15min":0.748},"min":{"1min":0.499,"5min":0.48099999999999998,"15min":0.435},"max":{"1min":0.89500000000000002,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.89000000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.38900000000000001},"max":{"1min":0.93400000000000005,"5min":1.026,"15min":1.0669999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.0820000000000001,"15min":1.236},"last":0.83499999999999996}]},{"osd":3,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.75600000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88300000000000001,"5min":1.002,"15min":1.079},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98899999999999999,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.65900000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81599999999999995,"5min":1.0009999999999999,"15min":1.302},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.871,"15min":1.0209999999999999},"last":0.92200000000000004}]},{"osd":5,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.52700000000000002,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.88100000000000001,"5min":1.097,"15min":1.1299999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64900000000000002,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":1.026,"15min":1.0800000000000001},"last":0.93100000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.56999999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.93999999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85699999999999998,"5min":1.01,"15min":1.093},"last":0.875}]}]},{"osd":6,"up_from":15,"seq":64424510569,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47636,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324204,"statfs":{"total":96636764160,"available":96587984896,"internally_reserved":0,"allocated":2699264,"data_stored":1431450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72799999999999998,"15min":0.74199999999999999},"min":{"1min":0.55400000000000005,"5min":0.46800000000000003,"15min":0.43099999999999999},"max":{"1min":0.92400000000000004,"5min":0.96499999999999997,"15min":1.042},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71799999999999997,"15min":0.73499999999999999},"min":{"1min":0.57499999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92000000000000004,"5min":1.224,"15min":1.224},"last":0.69299999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70099999999999996,"15min":0.72799999999999998},"min":{"1min":0.49099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98099999999999998,"5min":1.103,"15min":1.103},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.64100000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":0.94599999999999995,"5min":0.99199999999999999,"15min":1.1259999999999999},"last":0.76000000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.502},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0609999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.749},"min":{"1min":0.60899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83299999999999996,"5min":1.0089999999999999,"15min":1.111},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.755,"15min":0.78200000000000003},"min":{"1min":0.61799999999999999,"5min":0.495,"15min":0.495},"max":{"1min":0.91200000000000003,"5min":0.97999999999999998,"15min":1.1519999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.51800000000000002,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.98699999999999999,"5min":1.268,"15min":1.268},"last":0.72299999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92200000000000004,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":0.81299999999999994,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78600000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.93799999999999994,"5min":1.194,"15min":1.194},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":1.141,"15min":1.141},"last":0.95199999999999996}]},{"osd":7,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.95899999999999996,"5min":1.159,"15min":1.159},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94199999999999995,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.99199999999999999}]}]},{"osd":5,"up_from":15,"seq":64424510569,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47636,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324204,"statfs":{"total":96636764160,"available":96587984896,"internally_reserved":0,"allocated":2699264,"data_stored":1431450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72499999999999998,"15min":0.72999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34399999999999997},"max":{"1min":0.96899999999999997,"5min":1.0109999999999999,"15min":1.0149999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.40699999999999997},"max":{"1min":0.89200000000000002,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"min":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.497},"max":{"1min":0.95599999999999996,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.505,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78800000000000003,"15min":0.78100000000000003},"min":{"1min":0.61699999999999999,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":1.022,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94499999999999995,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.48499999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94999999999999996,"5min":1.05,"15min":1.05},"last":0.753},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.77400000000000002},"min":{"1min":0.56200000000000006,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98799999999999999,"5min":1.022,"15min":1.0329999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.79000000000000004,"5min":0.94499999999999995,"15min":1.0800000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41599999999999998},"max":{"1min":0.71099999999999997,"5min":1.208,"15min":1.208},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64800000000000002,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.94899999999999995,"15min":0.98999999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.65500000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.83499999999999996,"5min":1.008,"15min":1.0449999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.46500000000000002,"5min":0.439,"15min":0.436},"max":{"1min":0.77300000000000002,"5min":1.169,"15min":1.169},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71499999999999997,"15min":0.70999999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85699999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74199999999999999}]}]},{"osd":4,"up_from":15,"seq":64424510569,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47668,"kb_used_data":2668,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324172,"statfs":{"total":96636764160,"available":96587952128,"internally_reserved":0,"allocated":2732032,"data_stored":1435923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.51600000000000001,"5min":0.45800000000000002,"15min":0.45000000000000001},"max":{"1min":1.016,"5min":1.036,"15min":1.109},"last":0.62},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.502,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.91800000000000004,"5min":0.997,"15min":1.01},"last":0.72999999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.59199999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94699999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.997,"15min":1.1020000000000001},"last":0.68200000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79100000000000004,"15min":0.77700000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.47699999999999998},"max":{"1min":0.998,"5min":1.0289999999999999,"15min":1.071},"last":0.503},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.71699999999999997}]},{"osd":3,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76700000000000002,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93899999999999995,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.55500000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.66200000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65300000000000002,"15min":0.63100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.75},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67200000000000004,"15min":0.64600000000000002},"min":{"1min":0.495,"5min":0.42899999999999999,"15min":0.39600000000000002},"max":{"1min":0.88900000000000001,"5min":1.042,"15min":1.042},"last":0.52600000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.66100000000000003},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":1.024,"15min":1.0669999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0760000000000001},"last":0.78300000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.41899999999999998},"max":{"1min":0.92400000000000004,"5min":1.087,"15min":1.087},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77600000000000002,"15min":0.76200000000000001},"min":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.46999999999999997},"max":{"1min":0.98599999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70099999999999996}]}]},{"osd":3,"up_from":15,"seq":64424510569,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47656,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324184,"statfs":{"total":96636764160,"available":96587964416,"internally_reserved":0,"allocated":2719744,"data_stored":1431244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.63200000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74099999999999999,"5min":0.78300000000000003,"15min":0.80000000000000004},"last":0.504}]},{"osd":1,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.498},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71499999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.44}]},{"osd":2,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.77000000000000002,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.46000000000000002}]},{"osd":4,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81599999999999995,"15min":0.82799999999999996},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.012,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.81999999999999995,"15min":0.82299999999999995},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.56599999999999995},"max":{"1min":0.86499999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.65800000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82799999999999996,"15min":0.84199999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99099999999999999,"5min":1.1739999999999999,"15min":1.397},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.59399999999999997,"5min":0.55900000000000005,"15min":0.55100000000000005},"max":{"1min":0.91000000000000003,"5min":1.2230000000000001,"15min":1.262},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.83899999999999997,"15min":0.84799999999999998},"min":{"1min":0.60199999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.96099999999999997,"5min":1.234,"15min":1.274},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83799999999999997,"15min":0.84499999999999997},"min":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94599999999999995,"5min":1.165,"15min":1.302},"last":0.64500000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0640000000000001,"5min":1.3540000000000001,"15min":1.486},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89500000000000002},"min":{"1min":0.68899999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.93000000000000005,"5min":1.421,"15min":1.421},"last":0.626}]}]},{"osd":2,"up_from":15,"seq":64424510568,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47632,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324208,"statfs":{"total":96636764160,"available":96587988992,"internally_reserved":0,"allocated":2695168,"data_stored":1423635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.29299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.41499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.56000000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.86199999999999999,"15min":0.95999999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.36899999999999999},"max":{"1min":0.67200000000000004,"5min":0.79700000000000004,"15min":0.875},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.624,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.048,"5min":1.306,"15min":1.306},"last":0.753},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.05,"5min":1.272,"15min":1.466},"last":0.80700000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.64300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.97699999999999998,"5min":1.1459999999999999,"15min":1.3049999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.81999999999999995,"15min":0.82599999999999996},"min":{"1min":0.61899999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.429},"last":0.78400000000000003}]},{"osd":6,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.71199999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.216,"5min":1.365,"15min":1.4119999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80600000000000005,"15min":0.80800000000000005},"min":{"1min":0.625,"5min":0.55200000000000005,"15min":0.53500000000000003},"max":{"1min":0.94499999999999995,"5min":1.2,"15min":1.2470000000000001},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81499999999999995,"15min":0.82799999999999996},"min":{"1min":0.64400000000000002,"5min":0.59899999999999998,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.393,"15min":1.45},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85999999999999999,"15min":0.86099999999999999},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1850000000000001,"5min":1.276,"15min":1.276},"last":0.88600000000000001}]}]},{"osd":1,"up_from":15,"seq":64424510569,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59940,"kb_used_data":2652,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311900,"statfs":{"total":96636764160,"available":96575385600,"internally_reserved":0,"allocated":2715648,"data_stored":1433729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.33900000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63700000000000001,"5min":0.71999999999999997,"15min":0.89400000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.53300000000000003},"min":{"1min":0.32200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.32600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.505},{"interface":"front","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.45000000000000001,"15min":0.442},"max":{"1min":0.82699999999999996,"5min":0.94199999999999995,"15min":0.94499999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.65100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93400000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79100000000000004,"15min":0.81499999999999995},"min":{"1min":0.502,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0409999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.86599999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.78400000000000003,"15min":0.80500000000000005},"min":{"1min":0.49099999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97499999999999998,"5min":1.236,"15min":1.2609999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98399999999999999,"5min":1.353,"15min":1.353},"last":1.05}]},{"osd":6,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.60999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.008,"5min":1.2270000000000001,"15min":1.2689999999999999},"last":0.878},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.84299999999999997,"15min":0.84999999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.093,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":1.034}]},{"osd":7,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.64100000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.123,"5min":1.399,"15min":1.399},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.84299999999999997,"15min":0.85799999999999998},"min":{"1min":0.54100000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.105,"5min":1.365,"15min":1.365},"last":0.90100000000000002}]}]},{"osd":0,"up_from":14,"seq":60129543274,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60508,"kb_used_data":3220,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311332,"statfs":{"total":96636764160,"available":96574803968,"internally_reserved":0,"allocated":3297280,"data_stored":2021348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.438,"15min":0.45700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":1.216},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.317}]},{"osd":2,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64700000000000002,"5min":0.84999999999999998,"15min":1.1699999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72299999999999998,"5min":0.78900000000000003,"15min":0.96799999999999997},"last":0.442}]},{"osd":3,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.623,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.82699999999999996,"5min":0.871,"15min":1.032},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.63,"5min":0.51800000000000002,"15min":0.496},"max":{"1min":0.93100000000000005,"5min":1.1200000000000001,"15min":1.381},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.59199999999999997,"5min":0.56799999999999995,"15min":0.47799999999999998},"max":{"1min":1.02,"5min":1.137,"15min":1.2849999999999999},"last":0.752}]},{"osd":5,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80600000000000005,"15min":0.81899999999999995},"min":{"1min":0.57299999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.008,"5min":1.1930000000000001,"15min":1.3420000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.69599999999999995,"5min":0.59599999999999997,"15min":0.59499999999999997},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.288},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.51300000000000001},"max":{"1min":0.98899999999999999,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.83399999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0229999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.84999999999999998},"min":{"1min":0.65300000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.044,"5min":1.1899999999999999,"15min":1.415},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.83199999999999996,"15min":0.85599999999999998},"min":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0780000000000001,"5min":1.1379999999999999,"15min":1.395},"last":0.85599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:16:44.717 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:16:44.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.858+0000 7f94911ce640 1 Processor -- start 2023-12-19T18:16:44.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.858+0000 7f94911ce640 1 -- start start 2023-12-19T18:16:44.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f94911ce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f948c15c7f0 0x7f948c06c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f94911ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c0adb80 0x7f948c0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f94911ce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948c06ced0 con 0x7f948c0ccc20 2023-12-19T18:16:44.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f94911ce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c06d770 con 0x7f948c15c7f0 2023-12-19T18:16:44.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f94911ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c06d1b0 con 0x7f948c0adb80 2023-12-19T18:16:44.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948b59e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f948c0ccc20 0x7f948c0b30d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41412/0 (socket says 172.21.15.26:41412) 2023-12-19T18:16:44.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948b59e640 1 -- 172.21.15.26:0/585450459 learned_addr learned my addr 172.21.15.26:0/585450459 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f9483fff640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c0adb80 0x7f948c0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948ad9d640 1 --2- 172.21.15.26:0/585450459 >> v2:172.21.15.160:3300/0 conn(0x7f948c15c7f0 0x7f948c06c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823534921 0 0) 0x7f948c06ced0 con 0x7f948c0ccc20 2023-12-19T18:16:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9478003530 con 0x7f948c0ccc20 2023-12-19T18:16:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3627091181 0 0) 0x7f9478003530 con 0x7f948c0ccc20 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c0adb80 msgr2=0x7f948c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c0adb80 0x7f948c0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 >> v2:172.21.15.160:3300/0 conn(0x7f948c15c7f0 msgr2=0x7f948c06c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 --2- 172.21.15.26:0/585450459 >> v2:172.21.15.160:3300/0 conn(0x7f948c15c7f0 0x7f948c06c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c06d490 con 0x7f948c0ccc20 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f9484001a00 con 0x7f948c0ccc20 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 >> v1:172.21.15.26:6789/0 conn(0x7f948c0ccc20 legacy=0x7f948c0b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9478003980 0x7f9478003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94780047d0 0x7f9478004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9478005190 0x7f947800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c06d770 con 0x7f9478003980 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c06d1b0 con 0x7f9478005190 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947800bed0 con 0x7f94780047d0 2023-12-19T18:16:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948ad9d640 1 --2- 172.21.15.26:0/585450459 >> v2:172.21.15.160:3300/0 conn(0x7f948c15c7f0 0x7f948c06c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f9483fff640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c0adb80 0x7f948c0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948ad9d640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9478003980 0x7f9478003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948b59e640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9478005190 0x7f947800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f9483fff640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94780047d0 0x7f9478004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948ad9d640 1 -- 172.21.15.26:0/585450459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94780047d0 msgr2=0x7f9478004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948ad9d640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94780047d0 0x7f9478004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948ad9d640 1 -- 172.21.15.26:0/585450459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9478005190 msgr2=0x7f947800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948ad9d640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9478005190 0x7f947800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f948ad9d640 1 -- 172.21.15.26:0/585450459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9478003530 con 0x7f9478003980 2023-12-19T18:16:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.862+0000 7f9483fff640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f94780047d0 0x7f9478004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:44.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.866+0000 7f948ad9d640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9478003980 0x7f9478003da0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f947c009cc0 tx=0x7f947c01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de77357cced7e1d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:44.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.866+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c004020 con 0x7f9478003980 2023-12-19T18:16:44.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.866+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c004180 con 0x7f9478003980 2023-12-19T18:16:44.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.866+0000 7f948a59c640 1 -- 172.21.15.26:0/585450459 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0a57e0 con 0x7f9478003980 2023-12-19T18:16:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.866+0000 7f94911ce640 1 -- 172.21.15.26:0/585450459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9478003980 msgr2=0x7f9478003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.866+0000 7f94911ce640 1 --2- 172.21.15.26:0/585450459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9478003980 0x7f9478003da0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f947c009cc0 tx=0x7f947c01cc30 comp rx=0 tx=0).stop 2023-12-19T18:16:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.866+0000 7f948b59e640 1 -- 172.21.15.26:0/585450459 reap_dead start 2023-12-19T18:16:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.866+0000 7f94911ce640 1 -- 172.21.15.26:0/585450459 shutdown_connections 2023-12-19T18:16:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.866+0000 7f94911ce640 1 -- 172.21.15.26:0/585450459 >> 172.21.15.26:0/585450459 conn(0x7f948c0658e0 msgr2=0x7f948c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.866+0000 7f94911ce640 1 -- 172.21.15.26:0/585450459 shutdown_connections 2023-12-19T18:16:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.866+0000 7f94911ce640 1 -- 172.21.15.26:0/585450459 wait complete. 2023-12-19T18:16:44.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.866+0000 7f94911ce640 1 Processor -- start 2023-12-19T18:16:44.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f94911ce640 1 -- start start 2023-12-19T18:16:44.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f94911ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9478003980 0x7f948c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f94911ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c1f23e0 0x7f948c1542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f94911ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c15ec90 0x7f948c15f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f94911ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c15f650 con 0x7f9478003980 2023-12-19T18:16:44.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f94911ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c154830 con 0x7f948c1f23e0 2023-12-19T18:16:44.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f94911ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c1f2860 con 0x7f948c15ec90 2023-12-19T18:16:44.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f948ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9478003980 0x7f948c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f948ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9478003980 0x7f948c15cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39572/0 (socket says 172.21.15.26:39572) 2023-12-19T18:16:44.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f948ad9d640 1 -- 172.21.15.26:0/265108111 learned_addr learned my addr 172.21.15.26:0/265108111 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f948b59e640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c15ec90 0x7f948c15f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f9483fff640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c1f23e0 0x7f948c1542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f948ad9d640 1 -- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c15ec90 msgr2=0x7f948c15f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f948ad9d640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c15ec90 0x7f948c15f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f948ad9d640 1 -- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c1f23e0 msgr2=0x7f948c1542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f948ad9d640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c1f23e0 0x7f948c1542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f948ad9d640 1 -- 172.21.15.26:0/265108111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c15d2e0 con 0x7f9478003980 2023-12-19T18:16:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f948ad9d640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9478003980 0x7f948c15cbd0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f947c0af040 tx=0x7f9478005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f94837fe640 1 -- 172.21.15.26:0/265108111 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0bc070 con 0x7f9478003980 2023-12-19T18:16:44.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f94837fe640 1 -- 172.21.15.26:0/265108111 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c00f080 con 0x7f9478003980 2023-12-19T18:16:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f94837fe640 1 -- 172.21.15.26:0/265108111 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c004510 con 0x7f9478003980 2023-12-19T18:16:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f94911ce640 1 -- 172.21.15.26:0/265108111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948c15d5c0 con 0x7f9478003980 2023-12-19T18:16:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f94911ce640 1 -- 172.21.15.26:0/265108111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c15db80 con 0x7f9478003980 2023-12-19T18:16:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.870+0000 7f94911ce640 1 -- 172.21.15.26:0/265108111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948c074270 con 0x7f9478003980 2023-12-19T18:16:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.874+0000 7f94837fe640 1 -- 172.21.15.26:0/265108111 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f947c0a57e0 con 0x7f9478003980 2023-12-19T18:16:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.874+0000 7f94837fe640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f946c065bb0 0x7f946c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.874+0000 7f94837fe640 1 -- 172.21.15.26:0/265108111 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f947c11d340 con 0x7f9478003980 2023-12-19T18:16:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.874+0000 7f9483fff640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f946c065bb0 0x7f946c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.874+0000 7f9483fff640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f946c065bb0 0x7f946c068070 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f948c14da10 tx=0x7f9474008040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:44.878+0000 7f94837fe640 1 -- 172.21.15.26:0/265108111 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f947c0ea590 con 0x7f9478003980 2023-12-19T18:16:45.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.030+0000 7f94911ce640 1 -- 172.21.15.26:0/265108111 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f948c070fd0 con 0x7f946c065bb0 2023-12-19T18:16:45.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.042+0000 7f94837fe640 1 -- 172.21.15.26:0/265108111 <== mgr.9805 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+526710 (secure 0 0 0) 0x7f948c070fd0 con 0x7f946c065bb0 2023-12-19T18:16:45.051 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:45.055 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:16:45.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.046+0000 7f94911ce640 1 -- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f946c065bb0 msgr2=0x7f946c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.046+0000 7f94911ce640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f946c065bb0 0x7f946c068070 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f948c14da10 tx=0x7f9474008040 comp rx=0 tx=0).stop 2023-12-19T18:16:45.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.046+0000 7f94911ce640 1 -- 172.21.15.26:0/265108111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9478003980 msgr2=0x7f948c15cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.050+0000 7f94911ce640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9478003980 0x7f948c15cbd0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f947c0af040 tx=0x7f9478005800 comp rx=0 tx=0).stop 2023-12-19T18:16:45.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.050+0000 7f94911ce640 1 -- 172.21.15.26:0/265108111 shutdown_connections 2023-12-19T18:16:45.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.050+0000 7f94911ce640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f946c065bb0 0x7f946c068070 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.050+0000 7f94911ce640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f948c15ec90 0x7f948c15f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.050+0000 7f94911ce640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f948c1f23e0 0x7f948c1542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.050+0000 7f94911ce640 1 --2- 172.21.15.26:0/265108111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9478003980 0x7f948c15cbd0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.050+0000 7f94911ce640 1 -- 172.21.15.26:0/265108111 >> 172.21.15.26:0/265108111 conn(0x7f948c0658e0 msgr2=0x7f948c0b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:45.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.050+0000 7f94911ce640 1 -- 172.21.15.26:0/265108111 shutdown_connections 2023-12-19T18:16:45.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.050+0000 7f94911ce640 1 -- 172.21.15.26:0/265108111 wait complete. 2023-12-19T18:16:45.072 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-12-19T18:16:43.307249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594769,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":657,"num_write_kb":2443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":417,"ondisk_log_size":417,"up":364,"acting":364,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423232,"kb_used_data":22272,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754551488,"statfs":{"total":773094113280,"available":772660723712,"internally_reserved":0,"allocated":22806528,"data_stored":12598456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.182086"},"pg_stats":[{"pgid":"27.1a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357761+0000","last_change":"2023-12-19T18:16:40.564006+0000","last_active":"2023-12-19T18:16:42.357761+0000","last_peered":"2023-12-19T18:16:42.357761+0000","last_clean":"2023-12-19T18:16:42.357761+0000","last_became_active":"2023-12-19T18:16:40.563756+0000","last_became_peered":"2023-12-19T18:16:40.563756+0000","last_unstale":"2023-12-19T18:16:42.357761+0000","last_undegraded":"2023-12-19T18:16:42.357761+0000","last_fullsized":"2023-12-19T18:16:42.357761+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:17.484857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"200'346","reported_seq":1184,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.811441+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:16:27.811441+0000","last_peered":"2023-12-19T18:16:27.811441+0000","last_clean":"2023-12-19T18:16:27.811441+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:16:27.811441+0000","last_undegraded":"2023-12-19T18:16:27.811441+0000","last_fullsized":"2023-12-19T18:16:27.811441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305339+0000","last_change":"2023-12-19T18:16:27.411230+0000","last_active":"2023-12-19T18:16:28.305339+0000","last_peered":"2023-12-19T18:16:28.305339+0000","last_clean":"2023-12-19T18:16:28.305339+0000","last_became_active":"2023-12-19T18:16:27.410670+0000","last_became_peered":"2023-12-19T18:16:27.410670+0000","last_unstale":"2023-12-19T18:16:28.305339+0000","last_undegraded":"2023-12-19T18:16:28.305339+0000","last_fullsized":"2023-12-19T18:16:28.305339+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:22.426446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554070+0000","last_change":"2023-12-19T18:16:27.821899+0000","last_active":"2023-12-19T18:16:40.554070+0000","last_peered":"2023-12-19T18:16:40.554070+0000","last_clean":"2023-12-19T18:16:40.554070+0000","last_became_active":"2023-12-19T18:16:27.821634+0000","last_became_peered":"2023-12-19T18:16:27.821634+0000","last_unstale":"2023-12-19T18:16:40.554070+0000","last_undegraded":"2023-12-19T18:16:40.554070+0000","last_fullsized":"2023-12-19T18:16:40.554070+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:34.391068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857383+0000","last_change":"2023-12-19T18:16:40.150476+0000","last_active":"2023-12-19T18:16:41.857383+0000","last_peered":"2023-12-19T18:16:41.857383+0000","last_clean":"2023-12-19T18:16:41.857383+0000","last_became_active":"2023-12-19T18:16:40.149843+0000","last_became_peered":"2023-12-19T18:16:40.149843+0000","last_unstale":"2023-12-19T18:16:41.857383+0000","last_undegraded":"2023-12-19T18:16:41.857383+0000","last_fullsized":"2023-12-19T18:16:41.857383+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:35.179628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857177+0000","last_change":"2023-12-19T18:16:40.569150+0000","last_active":"2023-12-19T18:16:41.857177+0000","last_peered":"2023-12-19T18:16:41.857177+0000","last_clean":"2023-12-19T18:16:41.857177+0000","last_became_active":"2023-12-19T18:16:40.568754+0000","last_became_peered":"2023-12-19T18:16:40.568754+0000","last_unstale":"2023-12-19T18:16:41.857177+0000","last_undegraded":"2023-12-19T18:16:41.857177+0000","last_fullsized":"2023-12-19T18:16:41.857177+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:32.689972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548913+0000","last_change":"2023-12-19T18:16:27.412171+0000","last_active":"2023-12-19T18:16:40.548913+0000","last_peered":"2023-12-19T18:16:40.548913+0000","last_clean":"2023-12-19T18:16:40.548913+0000","last_became_active":"2023-12-19T18:16:27.411872+0000","last_became_peered":"2023-12-19T18:16:27.411872+0000","last_unstale":"2023-12-19T18:16:40.548913+0000","last_undegraded":"2023-12-19T18:16:40.548913+0000","last_fullsized":"2023-12-19T18:16:40.548913+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:48.264905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548191+0000","last_change":"2023-12-19T18:16:27.832432+0000","last_active":"2023-12-19T18:16:40.548191+0000","last_peered":"2023-12-19T18:16:40.548191+0000","last_clean":"2023-12-19T18:16:40.548191+0000","last_became_active":"2023-12-19T18:16:27.832116+0000","last_became_peered":"2023-12-19T18:16:27.832116+0000","last_unstale":"2023-12-19T18:16:40.548191+0000","last_undegraded":"2023-12-19T18:16:40.548191+0000","last_fullsized":"2023-12-19T18:16:40.548191+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:35.165338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359499+0000","last_change":"2023-12-19T18:16:40.157009+0000","last_active":"2023-12-19T18:16:42.359499+0000","last_peered":"2023-12-19T18:16:42.359499+0000","last_clean":"2023-12-19T18:16:42.359499+0000","last_became_active":"2023-12-19T18:16:40.156734+0000","last_became_peered":"2023-12-19T18:16:40.156734+0000","last_unstale":"2023-12-19T18:16:42.359499+0000","last_undegraded":"2023-12-19T18:16:42.359499+0000","last_fullsized":"2023-12-19T18:16:42.359499+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:18.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357983+0000","last_change":"2023-12-19T18:16:40.560903+0000","last_active":"2023-12-19T18:16:42.357983+0000","last_peered":"2023-12-19T18:16:42.357983+0000","last_clean":"2023-12-19T18:16:42.357983+0000","last_became_active":"2023-12-19T18:16:40.560636+0000","last_became_peered":"2023-12-19T18:16:40.560636+0000","last_unstale":"2023-12-19T18:16:42.357983+0000","last_undegraded":"2023-12-19T18:16:42.357983+0000","last_fullsized":"2023-12-19T18:16:42.357983+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:25:38.247442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561435+0000","last_change":"2023-12-19T18:16:27.407456+0000","last_active":"2023-12-19T18:16:40.561435+0000","last_peered":"2023-12-19T18:16:40.561435+0000","last_clean":"2023-12-19T18:16:40.561435+0000","last_became_active":"2023-12-19T18:16:27.407208+0000","last_became_peered":"2023-12-19T18:16:27.407208+0000","last_unstale":"2023-12-19T18:16:40.561435+0000","last_undegraded":"2023-12-19T18:16:40.561435+0000","last_fullsized":"2023-12-19T18:16:40.561435+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:07.654229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560528+0000","last_change":"2023-12-19T18:16:27.824001+0000","last_active":"2023-12-19T18:16:40.560528+0000","last_peered":"2023-12-19T18:16:40.560528+0000","last_clean":"2023-12-19T18:16:40.560528+0000","last_became_active":"2023-12-19T18:16:27.823010+0000","last_became_peered":"2023-12-19T18:16:27.823010+0000","last_unstale":"2023-12-19T18:16:40.560528+0000","last_undegraded":"2023-12-19T18:16:40.560528+0000","last_fullsized":"2023-12-19T18:16:40.560528+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:30:56.775994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:39.117449+0000","last_change":"2023-12-19T18:16:39.117449+0000","last_active":"2023-12-19T18:16:39.117449+0000","last_peered":"2023-12-19T18:16:39.117449+0000","last_clean":"2023-12-19T18:16:39.117449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:39.117449+0000","last_undegraded":"2023-12-19T18:16:39.117449+0000","last_fullsized":"2023-12-19T18:16:39.117449+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552525+0000","last_change":"2023-12-19T18:16:40.570024+0000","last_active":"2023-12-19T18:16:41.552525+0000","last_peered":"2023-12-19T18:16:41.552525+0000","last_clean":"2023-12-19T18:16:41.552525+0000","last_became_active":"2023-12-19T18:16:40.569586+0000","last_became_peered":"2023-12-19T18:16:40.569586+0000","last_unstale":"2023-12-19T18:16:41.552525+0000","last_undegraded":"2023-12-19T18:16:41.552525+0000","last_fullsized":"2023-12-19T18:16:41.552525+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:03.932038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547644+0000","last_change":"2023-12-19T18:16:27.405381+0000","last_active":"2023-12-19T18:16:40.547644+0000","last_peered":"2023-12-19T18:16:40.547644+0000","last_clean":"2023-12-19T18:16:40.547644+0000","last_became_active":"2023-12-19T18:16:27.405113+0000","last_became_peered":"2023-12-19T18:16:27.405113+0000","last_unstale":"2023-12-19T18:16:40.547644+0000","last_undegraded":"2023-12-19T18:16:40.547644+0000","last_fullsized":"2023-12-19T18:16:40.547644+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:08.911302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305362+0000","last_change":"2023-12-19T18:16:27.822423+0000","last_active":"2023-12-19T18:16:28.305362+0000","last_peered":"2023-12-19T18:16:28.305362+0000","last_clean":"2023-12-19T18:16:28.305362+0000","last_became_active":"2023-12-19T18:16:27.822106+0000","last_became_peered":"2023-12-19T18:16:27.822106+0000","last_unstale":"2023-12-19T18:16:28.305362+0000","last_undegraded":"2023-12-19T18:16:28.305362+0000","last_fullsized":"2023-12-19T18:16:28.305362+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:41.021760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.856687+0000","last_change":"2023-12-19T18:16:40.154697+0000","last_active":"2023-12-19T18:16:41.856687+0000","last_peered":"2023-12-19T18:16:41.856687+0000","last_clean":"2023-12-19T18:16:41.856687+0000","last_became_active":"2023-12-19T18:16:40.154394+0000","last_became_peered":"2023-12-19T18:16:40.154394+0000","last_unstale":"2023-12-19T18:16:41.856687+0000","last_undegraded":"2023-12-19T18:16:41.856687+0000","last_fullsized":"2023-12-19T18:16:41.856687+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:52.755953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357512+0000","last_change":"2023-12-19T18:16:40.572030+0000","last_active":"2023-12-19T18:16:42.357512+0000","last_peered":"2023-12-19T18:16:42.357512+0000","last_clean":"2023-12-19T18:16:42.357512+0000","last_became_active":"2023-12-19T18:16:40.570330+0000","last_became_peered":"2023-12-19T18:16:40.570330+0000","last_unstale":"2023-12-19T18:16:42.357512+0000","last_undegraded":"2023-12-19T18:16:42.357512+0000","last_fullsized":"2023-12-19T18:16:42.357512+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:55.123677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547774+0000","last_change":"2023-12-19T18:16:27.410513+0000","last_active":"2023-12-19T18:16:40.547774+0000","last_peered":"2023-12-19T18:16:40.547774+0000","last_clean":"2023-12-19T18:16:40.547774+0000","last_became_active":"2023-12-19T18:16:27.410166+0000","last_became_peered":"2023-12-19T18:16:27.410166+0000","last_unstale":"2023-12-19T18:16:40.547774+0000","last_undegraded":"2023-12-19T18:16:40.547774+0000","last_fullsized":"2023-12-19T18:16:40.547774+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:10.682880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305351+0000","last_change":"2023-12-19T18:16:27.829925+0000","last_active":"2023-12-19T18:16:28.305351+0000","last_peered":"2023-12-19T18:16:28.305351+0000","last_clean":"2023-12-19T18:16:28.305351+0000","last_became_active":"2023-12-19T18:16:27.828958+0000","last_became_peered":"2023-12-19T18:16:27.828958+0000","last_unstale":"2023-12-19T18:16:28.305351+0000","last_undegraded":"2023-12-19T18:16:28.305351+0000","last_fullsized":"2023-12-19T18:16:28.305351+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:46.995476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554304+0000","last_change":"2023-12-19T18:16:40.156673+0000","last_active":"2023-12-19T18:16:41.554304+0000","last_peered":"2023-12-19T18:16:41.554304+0000","last_clean":"2023-12-19T18:16:41.554304+0000","last_became_active":"2023-12-19T18:16:40.156391+0000","last_became_peered":"2023-12-19T18:16:40.156391+0000","last_unstale":"2023-12-19T18:16:41.554304+0000","last_undegraded":"2023-12-19T18:16:41.554304+0000","last_fullsized":"2023-12-19T18:16:41.554304+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:46.681719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360169+0000","last_change":"2023-12-19T18:16:40.571314+0000","last_active":"2023-12-19T18:16:42.360169+0000","last_peered":"2023-12-19T18:16:42.360169+0000","last_clean":"2023-12-19T18:16:42.360169+0000","last_became_active":"2023-12-19T18:16:40.570538+0000","last_became_peered":"2023-12-19T18:16:40.570538+0000","last_unstale":"2023-12-19T18:16:42.360169+0000","last_undegraded":"2023-12-19T18:16:42.360169+0000","last_fullsized":"2023-12-19T18:16:42.360169+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:02:14.652060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543489+0000","last_change":"2023-12-19T18:16:27.409169+0000","last_active":"2023-12-19T18:16:40.543489+0000","last_peered":"2023-12-19T18:16:40.543489+0000","last_clean":"2023-12-19T18:16:40.543489+0000","last_became_active":"2023-12-19T18:16:27.408882+0000","last_became_peered":"2023-12-19T18:16:27.408882+0000","last_unstale":"2023-12-19T18:16:40.543489+0000","last_undegraded":"2023-12-19T18:16:40.543489+0000","last_fullsized":"2023-12-19T18:16:40.543489+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:01.478246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544313+0000","last_change":"2023-12-19T18:16:27.825200+0000","last_active":"2023-12-19T18:16:40.544313+0000","last_peered":"2023-12-19T18:16:40.544313+0000","last_clean":"2023-12-19T18:16:40.544313+0000","last_became_active":"2023-12-19T18:16:27.824919+0000","last_became_peered":"2023-12-19T18:16:27.824919+0000","last_unstale":"2023-12-19T18:16:40.544313+0000","last_undegraded":"2023-12-19T18:16:40.544313+0000","last_fullsized":"2023-12-19T18:16:40.544313+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:39.117449+0000","last_change":"2023-12-19T18:16:39.117449+0000","last_active":"2023-12-19T18:16:39.117449+0000","last_peered":"2023-12-19T18:16:39.117449+0000","last_clean":"2023-12-19T18:16:39.117449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:39.117449+0000","last_undegraded":"2023-12-19T18:16:39.117449+0000","last_fullsized":"2023-12-19T18:16:39.117449+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857196+0000","last_change":"2023-12-19T18:16:40.568314+0000","last_active":"2023-12-19T18:16:41.857196+0000","last_peered":"2023-12-19T18:16:41.857196+0000","last_clean":"2023-12-19T18:16:41.857196+0000","last_became_active":"2023-12-19T18:16:40.567857+0000","last_became_peered":"2023-12-19T18:16:40.567857+0000","last_unstale":"2023-12-19T18:16:41.857196+0000","last_undegraded":"2023-12-19T18:16:41.857196+0000","last_fullsized":"2023-12-19T18:16:41.857196+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:49.703164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555637+0000","last_change":"2023-12-19T18:16:27.420249+0000","last_active":"2023-12-19T18:16:40.555637+0000","last_peered":"2023-12-19T18:16:40.555637+0000","last_clean":"2023-12-19T18:16:40.555637+0000","last_became_active":"2023-12-19T18:16:27.419360+0000","last_became_peered":"2023-12-19T18:16:27.419360+0000","last_unstale":"2023-12-19T18:16:40.555637+0000","last_undegraded":"2023-12-19T18:16:40.555637+0000","last_fullsized":"2023-12-19T18:16:40.555637+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:07.969904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543383+0000","last_change":"2023-12-19T18:16:27.825535+0000","last_active":"2023-12-19T18:16:40.543383+0000","last_peered":"2023-12-19T18:16:40.543383+0000","last_clean":"2023-12-19T18:16:40.543383+0000","last_became_active":"2023-12-19T18:16:27.824814+0000","last_became_peered":"2023-12-19T18:16:27.824814+0000","last_unstale":"2023-12-19T18:16:40.543383+0000","last_undegraded":"2023-12-19T18:16:40.543383+0000","last_fullsized":"2023-12-19T18:16:40.543383+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:29.357409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358846+0000","last_change":"2023-12-19T18:16:40.160113+0000","last_active":"2023-12-19T18:16:42.358846+0000","last_peered":"2023-12-19T18:16:42.358846+0000","last_clean":"2023-12-19T18:16:42.358846+0000","last_became_active":"2023-12-19T18:16:40.159826+0000","last_became_peered":"2023-12-19T18:16:40.159826+0000","last_unstale":"2023-12-19T18:16:42.358846+0000","last_undegraded":"2023-12-19T18:16:42.358846+0000","last_fullsized":"2023-12-19T18:16:42.358846+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:45.058000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554995+0000","last_change":"2023-12-19T18:16:40.566980+0000","last_active":"2023-12-19T18:16:41.554995+0000","last_peered":"2023-12-19T18:16:41.554995+0000","last_clean":"2023-12-19T18:16:41.554995+0000","last_became_active":"2023-12-19T18:16:40.566716+0000","last_became_peered":"2023-12-19T18:16:40.566716+0000","last_unstale":"2023-12-19T18:16:41.554995+0000","last_undegraded":"2023-12-19T18:16:41.554995+0000","last_fullsized":"2023-12-19T18:16:41.554995+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:30.489642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.6","version":"212'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558804+0000","last_change":"2023-12-19T18:16:27.414152+0000","last_active":"2023-12-19T18:16:40.558804+0000","last_peered":"2023-12-19T18:16:40.558804+0000","last_clean":"2023-12-19T18:16:40.558804+0000","last_became_active":"2023-12-19T18:16:27.413877+0000","last_became_peered":"2023-12-19T18:16:27.413877+0000","last_unstale":"2023-12-19T18:16:40.558804+0000","last_undegraded":"2023-12-19T18:16:40.558804+0000","last_fullsized":"2023-12-19T18:16:40.558804+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:44:52.553685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558691+0000","last_change":"2023-12-19T18:16:27.824257+0000","last_active":"2023-12-19T18:16:40.558691+0000","last_peered":"2023-12-19T18:16:40.558691+0000","last_clean":"2023-12-19T18:16:40.558691+0000","last_became_active":"2023-12-19T18:16:27.823989+0000","last_became_peered":"2023-12-19T18:16:27.823989+0000","last_unstale":"2023-12-19T18:16:40.558691+0000","last_undegraded":"2023-12-19T18:16:40.558691+0000","last_fullsized":"2023-12-19T18:16:40.558691+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:07:07.505127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357977+0000","last_change":"2023-12-19T18:16:40.160316+0000","last_active":"2023-12-19T18:16:42.357977+0000","last_peered":"2023-12-19T18:16:42.357977+0000","last_clean":"2023-12-19T18:16:42.357977+0000","last_became_active":"2023-12-19T18:16:40.159312+0000","last_became_peered":"2023-12-19T18:16:40.159312+0000","last_unstale":"2023-12-19T18:16:42.357977+0000","last_undegraded":"2023-12-19T18:16:42.357977+0000","last_fullsized":"2023-12-19T18:16:42.357977+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:04.832569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553051+0000","last_change":"2023-12-19T18:16:40.563986+0000","last_active":"2023-12-19T18:16:41.553051+0000","last_peered":"2023-12-19T18:16:41.553051+0000","last_clean":"2023-12-19T18:16:41.553051+0000","last_became_active":"2023-12-19T18:16:40.563693+0000","last_became_peered":"2023-12-19T18:16:40.563693+0000","last_unstale":"2023-12-19T18:16:41.553051+0000","last_undegraded":"2023-12-19T18:16:41.553051+0000","last_fullsized":"2023-12-19T18:16:41.553051+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:27.465263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"211'2","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305451+0000","last_change":"2023-12-19T18:16:27.409874+0000","last_active":"2023-12-19T18:16:28.305451+0000","last_peered":"2023-12-19T18:16:28.305451+0000","last_clean":"2023-12-19T18:16:28.305451+0000","last_became_active":"2023-12-19T18:16:27.409598+0000","last_became_peered":"2023-12-19T18:16:27.409598+0000","last_unstale":"2023-12-19T18:16:28.305451+0000","last_undegraded":"2023-12-19T18:16:28.305451+0000","last_fullsized":"2023-12-19T18:16:28.305451+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:13.721057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549450+0000","last_change":"2023-12-19T18:16:27.826090+0000","last_active":"2023-12-19T18:16:40.549450+0000","last_peered":"2023-12-19T18:16:40.549450+0000","last_clean":"2023-12-19T18:16:40.549450+0000","last_became_active":"2023-12-19T18:16:27.825791+0000","last_became_peered":"2023-12-19T18:16:27.825791+0000","last_unstale":"2023-12-19T18:16:40.549450+0000","last_undegraded":"2023-12-19T18:16:40.549450+0000","last_fullsized":"2023-12-19T18:16:40.549450+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:39.382953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359959+0000","last_change":"2023-12-19T18:16:40.160037+0000","last_active":"2023-12-19T18:16:42.359959+0000","last_peered":"2023-12-19T18:16:42.359959+0000","last_clean":"2023-12-19T18:16:42.359959+0000","last_became_active":"2023-12-19T18:16:40.159737+0000","last_became_peered":"2023-12-19T18:16:40.159737+0000","last_unstale":"2023-12-19T18:16:42.359959+0000","last_undegraded":"2023-12-19T18:16:42.359959+0000","last_fullsized":"2023-12-19T18:16:42.359959+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:42.336188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358621+0000","last_change":"2023-12-19T18:16:40.567065+0000","last_active":"2023-12-19T18:16:42.358621+0000","last_peered":"2023-12-19T18:16:42.358621+0000","last_clean":"2023-12-19T18:16:42.358621+0000","last_became_active":"2023-12-19T18:16:40.566804+0000","last_became_peered":"2023-12-19T18:16:40.566804+0000","last_unstale":"2023-12-19T18:16:42.358621+0000","last_undegraded":"2023-12-19T18:16:42.358621+0000","last_fullsized":"2023-12-19T18:16:42.358621+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:21.029140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546688+0000","last_change":"2023-12-19T18:16:27.403537+0000","last_active":"2023-12-19T18:16:40.546688+0000","last_peered":"2023-12-19T18:16:40.546688+0000","last_clean":"2023-12-19T18:16:40.546688+0000","last_became_active":"2023-12-19T18:16:27.403250+0000","last_became_peered":"2023-12-19T18:16:27.403250+0000","last_unstale":"2023-12-19T18:16:40.546688+0000","last_undegraded":"2023-12-19T18:16:40.546688+0000","last_fullsized":"2023-12-19T18:16:40.546688+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:56.889902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548113+0000","last_change":"2023-12-19T18:16:27.825355+0000","last_active":"2023-12-19T18:16:40.548113+0000","last_peered":"2023-12-19T18:16:40.548113+0000","last_clean":"2023-12-19T18:16:40.548113+0000","last_became_active":"2023-12-19T18:16:27.825067+0000","last_became_peered":"2023-12-19T18:16:27.825067+0000","last_unstale":"2023-12-19T18:16:40.548113+0000","last_undegraded":"2023-12-19T18:16:40.548113+0000","last_fullsized":"2023-12-19T18:16:40.548113+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:02.928407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358143+0000","last_change":"2023-12-19T18:16:40.152882+0000","last_active":"2023-12-19T18:16:42.358143+0000","last_peered":"2023-12-19T18:16:42.358143+0000","last_clean":"2023-12-19T18:16:42.358143+0000","last_became_active":"2023-12-19T18:16:40.152561+0000","last_became_peered":"2023-12-19T18:16:40.152561+0000","last_unstale":"2023-12-19T18:16:42.358143+0000","last_undegraded":"2023-12-19T18:16:42.358143+0000","last_fullsized":"2023-12-19T18:16:42.358143+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:11:26.922031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550797+0000","last_change":"2023-12-19T18:16:40.560282+0000","last_active":"2023-12-19T18:16:41.550797+0000","last_peered":"2023-12-19T18:16:41.550797+0000","last_clean":"2023-12-19T18:16:41.550797+0000","last_became_active":"2023-12-19T18:16:40.560017+0000","last_became_peered":"2023-12-19T18:16:40.560017+0000","last_unstale":"2023-12-19T18:16:41.550797+0000","last_undegraded":"2023-12-19T18:16:41.550797+0000","last_fullsized":"2023-12-19T18:16:41.550797+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:51:01.095692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553689+0000","last_change":"2023-12-19T18:16:27.417198+0000","last_active":"2023-12-19T18:16:40.553689+0000","last_peered":"2023-12-19T18:16:40.553689+0000","last_clean":"2023-12-19T18:16:40.553689+0000","last_became_active":"2023-12-19T18:16:27.416902+0000","last_became_peered":"2023-12-19T18:16:27.416902+0000","last_unstale":"2023-12-19T18:16:40.553689+0000","last_undegraded":"2023-12-19T18:16:40.553689+0000","last_fullsized":"2023-12-19T18:16:40.553689+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:54.434372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560449+0000","last_change":"2023-12-19T18:16:27.824198+0000","last_active":"2023-12-19T18:16:40.560449+0000","last_peered":"2023-12-19T18:16:40.560449+0000","last_clean":"2023-12-19T18:16:40.560449+0000","last_became_active":"2023-12-19T18:16:27.823738+0000","last_became_peered":"2023-12-19T18:16:27.823738+0000","last_unstale":"2023-12-19T18:16:40.560449+0000","last_undegraded":"2023-12-19T18:16:40.560449+0000","last_fullsized":"2023-12-19T18:16:40.560449+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:53.509897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550952+0000","last_change":"2023-12-19T18:16:40.160436+0000","last_active":"2023-12-19T18:16:41.550952+0000","last_peered":"2023-12-19T18:16:41.550952+0000","last_clean":"2023-12-19T18:16:41.550952+0000","last_became_active":"2023-12-19T18:16:40.159692+0000","last_became_peered":"2023-12-19T18:16:40.159692+0000","last_unstale":"2023-12-19T18:16:41.550952+0000","last_undegraded":"2023-12-19T18:16:41.550952+0000","last_fullsized":"2023-12-19T18:16:41.550952+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:47:01.985064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554294+0000","last_change":"2023-12-19T18:16:40.572114+0000","last_active":"2023-12-19T18:16:41.554294+0000","last_peered":"2023-12-19T18:16:41.554294+0000","last_clean":"2023-12-19T18:16:41.554294+0000","last_became_active":"2023-12-19T18:16:40.571811+0000","last_became_peered":"2023-12-19T18:16:40.571811+0000","last_unstale":"2023-12-19T18:16:41.554294+0000","last_undegraded":"2023-12-19T18:16:41.554294+0000","last_fullsized":"2023-12-19T18:16:41.554294+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:01.072159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550771+0000","last_change":"2023-12-19T18:16:27.404853+0000","last_active":"2023-12-19T18:16:40.550771+0000","last_peered":"2023-12-19T18:16:40.550771+0000","last_clean":"2023-12-19T18:16:40.550771+0000","last_became_active":"2023-12-19T18:16:27.404347+0000","last_became_peered":"2023-12-19T18:16:27.404347+0000","last_unstale":"2023-12-19T18:16:40.550771+0000","last_undegraded":"2023-12-19T18:16:40.550771+0000","last_fullsized":"2023-12-19T18:16:40.550771+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:16.360996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558229+0000","last_change":"2023-12-19T18:16:27.828160+0000","last_active":"2023-12-19T18:16:40.558229+0000","last_peered":"2023-12-19T18:16:40.558229+0000","last_clean":"2023-12-19T18:16:40.558229+0000","last_became_active":"2023-12-19T18:16:27.827882+0000","last_became_peered":"2023-12-19T18:16:27.827882+0000","last_unstale":"2023-12-19T18:16:40.558229+0000","last_undegraded":"2023-12-19T18:16:40.558229+0000","last_fullsized":"2023-12-19T18:16:40.558229+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:17.726704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358933+0000","last_change":"2023-12-19T18:16:40.154706+0000","last_active":"2023-12-19T18:16:42.358933+0000","last_peered":"2023-12-19T18:16:42.358933+0000","last_clean":"2023-12-19T18:16:42.358933+0000","last_became_active":"2023-12-19T18:16:40.154154+0000","last_became_peered":"2023-12-19T18:16:40.154154+0000","last_unstale":"2023-12-19T18:16:42.358933+0000","last_undegraded":"2023-12-19T18:16:42.358933+0000","last_fullsized":"2023-12-19T18:16:42.358933+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:55.343366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552500+0000","last_change":"2023-12-19T18:16:40.555828+0000","last_active":"2023-12-19T18:16:41.552500+0000","last_peered":"2023-12-19T18:16:41.552500+0000","last_clean":"2023-12-19T18:16:41.552500+0000","last_became_active":"2023-12-19T18:16:40.555500+0000","last_became_peered":"2023-12-19T18:16:40.555500+0000","last_unstale":"2023-12-19T18:16:41.552500+0000","last_undegraded":"2023-12-19T18:16:41.552500+0000","last_fullsized":"2023-12-19T18:16:41.552500+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:50.483528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.b","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551100+0000","last_change":"2023-12-19T18:16:27.415902+0000","last_active":"2023-12-19T18:16:40.551100+0000","last_peered":"2023-12-19T18:16:40.551100+0000","last_clean":"2023-12-19T18:16:40.551100+0000","last_became_active":"2023-12-19T18:16:27.415062+0000","last_became_peered":"2023-12-19T18:16:27.415062+0000","last_unstale":"2023-12-19T18:16:40.551100+0000","last_undegraded":"2023-12-19T18:16:40.551100+0000","last_fullsized":"2023-12-19T18:16:40.551100+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:22.442653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556868+0000","last_change":"2023-12-19T18:16:27.827898+0000","last_active":"2023-12-19T18:16:40.556868+0000","last_peered":"2023-12-19T18:16:40.556868+0000","last_clean":"2023-12-19T18:16:40.556868+0000","last_became_active":"2023-12-19T18:16:27.827613+0000","last_became_peered":"2023-12-19T18:16:27.827613+0000","last_unstale":"2023-12-19T18:16:40.556868+0000","last_undegraded":"2023-12-19T18:16:40.556868+0000","last_fullsized":"2023-12-19T18:16:40.556868+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:29.033205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357614+0000","last_change":"2023-12-19T18:16:40.157634+0000","last_active":"2023-12-19T18:16:42.357614+0000","last_peered":"2023-12-19T18:16:42.357614+0000","last_clean":"2023-12-19T18:16:42.357614+0000","last_became_active":"2023-12-19T18:16:40.157363+0000","last_became_peered":"2023-12-19T18:16:40.157363+0000","last_unstale":"2023-12-19T18:16:42.357614+0000","last_undegraded":"2023-12-19T18:16:42.357614+0000","last_fullsized":"2023-12-19T18:16:42.357614+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:23.564600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553681+0000","last_change":"2023-12-19T18:16:40.556441+0000","last_active":"2023-12-19T18:16:41.553681+0000","last_peered":"2023-12-19T18:16:41.553681+0000","last_clean":"2023-12-19T18:16:41.553681+0000","last_became_active":"2023-12-19T18:16:40.556187+0000","last_became_peered":"2023-12-19T18:16:40.556187+0000","last_unstale":"2023-12-19T18:16:41.553681+0000","last_undegraded":"2023-12-19T18:16:41.553681+0000","last_fullsized":"2023-12-19T18:16:41.553681+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:16:05.083354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562653+0000","last_change":"2023-12-19T18:16:27.408051+0000","last_active":"2023-12-19T18:16:40.562653+0000","last_peered":"2023-12-19T18:16:40.562653+0000","last_clean":"2023-12-19T18:16:40.562653+0000","last_became_active":"2023-12-19T18:16:27.407796+0000","last_became_peered":"2023-12-19T18:16:27.407796+0000","last_unstale":"2023-12-19T18:16:40.562653+0000","last_undegraded":"2023-12-19T18:16:40.562653+0000","last_fullsized":"2023-12-19T18:16:40.562653+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:41:09.697612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560009+0000","last_change":"2023-12-19T18:16:27.832376+0000","last_active":"2023-12-19T18:16:40.560009+0000","last_peered":"2023-12-19T18:16:40.560009+0000","last_clean":"2023-12-19T18:16:40.560009+0000","last_became_active":"2023-12-19T18:16:27.832060+0000","last_became_peered":"2023-12-19T18:16:27.832060+0000","last_unstale":"2023-12-19T18:16:40.560009+0000","last_undegraded":"2023-12-19T18:16:40.560009+0000","last_fullsized":"2023-12-19T18:16:40.560009+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:17.432889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.e","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553537+0000","last_change":"2023-12-19T18:16:40.152827+0000","last_active":"2023-12-19T18:16:41.553537+0000","last_peered":"2023-12-19T18:16:41.553537+0000","last_clean":"2023-12-19T18:16:41.553537+0000","last_became_active":"2023-12-19T18:16:40.152559+0000","last_became_peered":"2023-12-19T18:16:40.152559+0000","last_unstale":"2023-12-19T18:16:41.553537+0000","last_undegraded":"2023-12-19T18:16:41.553537+0000","last_fullsized":"2023-12-19T18:16:41.553537+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:59.363074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553423+0000","last_change":"2023-12-19T18:16:40.572243+0000","last_active":"2023-12-19T18:16:41.553423+0000","last_peered":"2023-12-19T18:16:41.553423+0000","last_clean":"2023-12-19T18:16:41.553423+0000","last_became_active":"2023-12-19T18:16:40.571938+0000","last_became_peered":"2023-12-19T18:16:40.571938+0000","last_unstale":"2023-12-19T18:16:41.553423+0000","last_undegraded":"2023-12-19T18:16:41.553423+0000","last_fullsized":"2023-12-19T18:16:41.553423+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:50.735904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"211'1","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305390+0000","last_change":"2023-12-19T18:16:27.416654+0000","last_active":"2023-12-19T18:16:28.305390+0000","last_peered":"2023-12-19T18:16:28.305390+0000","last_clean":"2023-12-19T18:16:28.305390+0000","last_became_active":"2023-12-19T18:16:27.416388+0000","last_became_peered":"2023-12-19T18:16:27.416388+0000","last_unstale":"2023-12-19T18:16:28.305390+0000","last_undegraded":"2023-12-19T18:16:28.305390+0000","last_fullsized":"2023-12-19T18:16:28.305390+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:51.339148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547637+0000","last_change":"2023-12-19T18:16:27.816436+0000","last_active":"2023-12-19T18:16:40.547637+0000","last_peered":"2023-12-19T18:16:40.547637+0000","last_clean":"2023-12-19T18:16:40.547637+0000","last_became_active":"2023-12-19T18:16:27.816092+0000","last_became_peered":"2023-12-19T18:16:27.816092+0000","last_unstale":"2023-12-19T18:16:40.547637+0000","last_undegraded":"2023-12-19T18:16:40.547637+0000","last_fullsized":"2023-12-19T18:16:40.547637+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:07:26.713649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360338+0000","last_change":"2023-12-19T18:16:40.154797+0000","last_active":"2023-12-19T18:16:42.360338+0000","last_peered":"2023-12-19T18:16:42.360338+0000","last_clean":"2023-12-19T18:16:42.360338+0000","last_became_active":"2023-12-19T18:16:40.154490+0000","last_became_peered":"2023-12-19T18:16:40.154490+0000","last_unstale":"2023-12-19T18:16:42.360338+0000","last_undegraded":"2023-12-19T18:16:42.360338+0000","last_fullsized":"2023-12-19T18:16:42.360338+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:10:02.821689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551216+0000","last_change":"2023-12-19T18:16:40.573560+0000","last_active":"2023-12-19T18:16:41.551216+0000","last_peered":"2023-12-19T18:16:41.551216+0000","last_clean":"2023-12-19T18:16:41.551216+0000","last_became_active":"2023-12-19T18:16:40.573277+0000","last_became_peered":"2023-12-19T18:16:40.573277+0000","last_unstale":"2023-12-19T18:16:41.551216+0000","last_undegraded":"2023-12-19T18:16:41.551216+0000","last_fullsized":"2023-12-19T18:16:41.551216+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:23:53.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544525+0000","last_change":"2023-12-19T18:16:27.409426+0000","last_active":"2023-12-19T18:16:40.544525+0000","last_peered":"2023-12-19T18:16:40.544525+0000","last_clean":"2023-12-19T18:16:40.544525+0000","last_became_active":"2023-12-19T18:16:27.407151+0000","last_became_peered":"2023-12-19T18:16:27.407151+0000","last_unstale":"2023-12-19T18:16:40.544525+0000","last_undegraded":"2023-12-19T18:16:40.544525+0000","last_fullsized":"2023-12-19T18:16:40.544525+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:30:56.976667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550067+0000","last_change":"2023-12-19T18:16:27.823846+0000","last_active":"2023-12-19T18:16:40.550067+0000","last_peered":"2023-12-19T18:16:40.550067+0000","last_clean":"2023-12-19T18:16:40.550067+0000","last_became_active":"2023-12-19T18:16:27.823514+0000","last_became_peered":"2023-12-19T18:16:27.823514+0000","last_unstale":"2023-12-19T18:16:40.550067+0000","last_undegraded":"2023-12-19T18:16:40.550067+0000","last_fullsized":"2023-12-19T18:16:40.550067+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:20.691537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358069+0000","last_change":"2023-12-19T18:16:40.162626+0000","last_active":"2023-12-19T18:16:42.358069+0000","last_peered":"2023-12-19T18:16:42.358069+0000","last_clean":"2023-12-19T18:16:42.358069+0000","last_became_active":"2023-12-19T18:16:40.161257+0000","last_became_peered":"2023-12-19T18:16:40.161257+0000","last_unstale":"2023-12-19T18:16:42.358069+0000","last_undegraded":"2023-12-19T18:16:42.358069+0000","last_fullsized":"2023-12-19T18:16:42.358069+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:21:05.448555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357894+0000","last_change":"2023-12-19T18:16:40.558784+0000","last_active":"2023-12-19T18:16:42.357894+0000","last_peered":"2023-12-19T18:16:42.357894+0000","last_clean":"2023-12-19T18:16:42.357894+0000","last_became_active":"2023-12-19T18:16:40.556768+0000","last_became_peered":"2023-12-19T18:16:40.556768+0000","last_unstale":"2023-12-19T18:16:42.357894+0000","last_undegraded":"2023-12-19T18:16:42.357894+0000","last_fullsized":"2023-12-19T18:16:42.357894+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:14:19.106971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553891+0000","last_change":"2023-12-19T18:16:27.414971+0000","last_active":"2023-12-19T18:16:40.553891+0000","last_peered":"2023-12-19T18:16:40.553891+0000","last_clean":"2023-12-19T18:16:40.553891+0000","last_became_active":"2023-12-19T18:16:27.414662+0000","last_became_peered":"2023-12-19T18:16:27.414662+0000","last_unstale":"2023-12-19T18:16:40.553891+0000","last_undegraded":"2023-12-19T18:16:40.553891+0000","last_fullsized":"2023-12-19T18:16:40.553891+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:47.204637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558717+0000","last_change":"2023-12-19T18:16:27.814804+0000","last_active":"2023-12-19T18:16:40.558717+0000","last_peered":"2023-12-19T18:16:40.558717+0000","last_clean":"2023-12-19T18:16:40.558717+0000","last_became_active":"2023-12-19T18:16:27.814534+0000","last_became_peered":"2023-12-19T18:16:27.814534+0000","last_unstale":"2023-12-19T18:16:40.558717+0000","last_undegraded":"2023-12-19T18:16:40.558717+0000","last_fullsized":"2023-12-19T18:16:40.558717+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:45.690711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551146+0000","last_change":"2023-12-19T18:16:40.158484+0000","last_active":"2023-12-19T18:16:41.551146+0000","last_peered":"2023-12-19T18:16:41.551146+0000","last_clean":"2023-12-19T18:16:41.551146+0000","last_became_active":"2023-12-19T18:16:40.158205+0000","last_became_peered":"2023-12-19T18:16:40.158205+0000","last_unstale":"2023-12-19T18:16:41.551146+0000","last_undegraded":"2023-12-19T18:16:41.551146+0000","last_fullsized":"2023-12-19T18:16:41.551146+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:23.673873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550951+0000","last_change":"2023-12-19T18:16:40.555429+0000","last_active":"2023-12-19T18:16:41.550951+0000","last_peered":"2023-12-19T18:16:41.550951+0000","last_clean":"2023-12-19T18:16:41.550951+0000","last_became_active":"2023-12-19T18:16:40.555158+0000","last_became_peered":"2023-12-19T18:16:40.555158+0000","last_unstale":"2023-12-19T18:16:41.550951+0000","last_undegraded":"2023-12-19T18:16:41.550951+0000","last_fullsized":"2023-12-19T18:16:41.550951+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:53.628064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.10","version":"211'8","reported_seq":19,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305320+0000","last_change":"2023-12-19T18:16:27.415118+0000","last_active":"2023-12-19T18:16:28.305320+0000","last_peered":"2023-12-19T18:16:28.305320+0000","last_clean":"2023-12-19T18:16:28.305320+0000","last_became_active":"2023-12-19T18:16:27.414687+0000","last_became_peered":"2023-12-19T18:16:27.414687+0000","last_unstale":"2023-12-19T18:16:28.305320+0000","last_undegraded":"2023-12-19T18:16:28.305320+0000","last_fullsized":"2023-12-19T18:16:28.305320+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:16.723922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555219+0000","last_change":"2023-12-19T18:16:27.814277+0000","last_active":"2023-12-19T18:16:40.555219+0000","last_peered":"2023-12-19T18:16:40.555219+0000","last_clean":"2023-12-19T18:16:40.555219+0000","last_became_active":"2023-12-19T18:16:27.814023+0000","last_became_peered":"2023-12-19T18:16:27.814023+0000","last_unstale":"2023-12-19T18:16:40.555219+0000","last_undegraded":"2023-12-19T18:16:40.555219+0000","last_fullsized":"2023-12-19T18:16:40.555219+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:32.135133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554395+0000","last_change":"2023-12-19T18:16:40.154784+0000","last_active":"2023-12-19T18:16:41.554395+0000","last_peered":"2023-12-19T18:16:41.554395+0000","last_clean":"2023-12-19T18:16:41.554395+0000","last_became_active":"2023-12-19T18:16:40.154523+0000","last_became_peered":"2023-12-19T18:16:40.154523+0000","last_unstale":"2023-12-19T18:16:41.554395+0000","last_undegraded":"2023-12-19T18:16:41.554395+0000","last_fullsized":"2023-12-19T18:16:41.554395+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:14.215221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359308+0000","last_change":"2023-12-19T18:16:40.558066+0000","last_active":"2023-12-19T18:16:42.359308+0000","last_peered":"2023-12-19T18:16:42.359308+0000","last_clean":"2023-12-19T18:16:42.359308+0000","last_became_active":"2023-12-19T18:16:40.557771+0000","last_became_peered":"2023-12-19T18:16:40.557771+0000","last_unstale":"2023-12-19T18:16:42.359308+0000","last_undegraded":"2023-12-19T18:16:42.359308+0000","last_fullsized":"2023-12-19T18:16:42.359308+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:28.573589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543014+0000","last_change":"2023-12-19T18:16:27.403403+0000","last_active":"2023-12-19T18:16:40.543014+0000","last_peered":"2023-12-19T18:16:40.543014+0000","last_clean":"2023-12-19T18:16:40.543014+0000","last_became_active":"2023-12-19T18:16:27.403121+0000","last_became_peered":"2023-12-19T18:16:27.403121+0000","last_unstale":"2023-12-19T18:16:40.543014+0000","last_undegraded":"2023-12-19T18:16:40.543014+0000","last_fullsized":"2023-12-19T18:16:40.543014+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:36:58.560143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546582+0000","last_change":"2023-12-19T18:16:27.821814+0000","last_active":"2023-12-19T18:16:40.546582+0000","last_peered":"2023-12-19T18:16:40.546582+0000","last_clean":"2023-12-19T18:16:40.546582+0000","last_became_active":"2023-12-19T18:16:27.821519+0000","last_became_peered":"2023-12-19T18:16:27.821519+0000","last_unstale":"2023-12-19T18:16:40.546582+0000","last_undegraded":"2023-12-19T18:16:40.546582+0000","last_fullsized":"2023-12-19T18:16:40.546582+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:40.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.13","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554534+0000","last_change":"2023-12-19T18:16:40.160940+0000","last_active":"2023-12-19T18:16:41.554534+0000","last_peered":"2023-12-19T18:16:41.554534+0000","last_clean":"2023-12-19T18:16:41.554534+0000","last_became_active":"2023-12-19T18:16:40.160594+0000","last_became_peered":"2023-12-19T18:16:40.160594+0000","last_unstale":"2023-12-19T18:16:41.554534+0000","last_undegraded":"2023-12-19T18:16:41.554534+0000","last_fullsized":"2023-12-19T18:16:41.554534+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:01.011110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357451+0000","last_change":"2023-12-19T18:16:40.564547+0000","last_active":"2023-12-19T18:16:42.357451+0000","last_peered":"2023-12-19T18:16:42.357451+0000","last_clean":"2023-12-19T18:16:42.357451+0000","last_became_active":"2023-12-19T18:16:40.564300+0000","last_became_peered":"2023-12-19T18:16:40.564300+0000","last_unstale":"2023-12-19T18:16:42.357451+0000","last_undegraded":"2023-12-19T18:16:42.357451+0000","last_fullsized":"2023-12-19T18:16:42.357451+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:42.276098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556939+0000","last_change":"2023-12-19T18:16:27.417401+0000","last_active":"2023-12-19T18:16:40.556939+0000","last_peered":"2023-12-19T18:16:40.556939+0000","last_clean":"2023-12-19T18:16:40.556939+0000","last_became_active":"2023-12-19T18:16:27.416774+0000","last_became_peered":"2023-12-19T18:16:27.416774+0000","last_unstale":"2023-12-19T18:16:40.556939+0000","last_undegraded":"2023-12-19T18:16:40.556939+0000","last_fullsized":"2023-12-19T18:16:40.556939+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:03.620596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558146+0000","last_change":"2023-12-19T18:16:27.824516+0000","last_active":"2023-12-19T18:16:40.558146+0000","last_peered":"2023-12-19T18:16:40.558146+0000","last_clean":"2023-12-19T18:16:40.558146+0000","last_became_active":"2023-12-19T18:16:27.823143+0000","last_became_peered":"2023-12-19T18:16:27.823143+0000","last_unstale":"2023-12-19T18:16:40.558146+0000","last_undegraded":"2023-12-19T18:16:40.558146+0000","last_fullsized":"2023-12-19T18:16:40.558146+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:52.341930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"215'8","reported_seq":19,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357609+0000","last_change":"2023-12-19T18:16:40.161467+0000","last_active":"2023-12-19T18:16:42.357609+0000","last_peered":"2023-12-19T18:16:42.357609+0000","last_clean":"2023-12-19T18:16:42.357609+0000","last_became_active":"2023-12-19T18:16:40.161143+0000","last_became_peered":"2023-12-19T18:16:40.161143+0000","last_unstale":"2023-12-19T18:16:42.357609+0000","last_undegraded":"2023-12-19T18:16:42.357609+0000","last_fullsized":"2023-12-19T18:16:42.357609+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:21:09.532919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357722+0000","last_change":"2023-12-19T18:16:40.571041+0000","last_active":"2023-12-19T18:16:42.357722+0000","last_peered":"2023-12-19T18:16:42.357722+0000","last_clean":"2023-12-19T18:16:42.357722+0000","last_became_active":"2023-12-19T18:16:40.570705+0000","last_became_peered":"2023-12-19T18:16:40.570705+0000","last_unstale":"2023-12-19T18:16:42.357722+0000","last_undegraded":"2023-12-19T18:16:42.357722+0000","last_fullsized":"2023-12-19T18:16:42.357722+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:43.042802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.13","version":"211'2","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305372+0000","last_change":"2023-12-19T18:16:27.415000+0000","last_active":"2023-12-19T18:16:28.305372+0000","last_peered":"2023-12-19T18:16:28.305372+0000","last_clean":"2023-12-19T18:16:28.305372+0000","last_became_active":"2023-12-19T18:16:27.414679+0000","last_became_peered":"2023-12-19T18:16:27.414679+0000","last_unstale":"2023-12-19T18:16:28.305372+0000","last_undegraded":"2023-12-19T18:16:28.305372+0000","last_fullsized":"2023-12-19T18:16:28.305372+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:50.187832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554789+0000","last_change":"2023-12-19T18:16:27.824833+0000","last_active":"2023-12-19T18:16:40.554789+0000","last_peered":"2023-12-19T18:16:40.554789+0000","last_clean":"2023-12-19T18:16:40.554789+0000","last_became_active":"2023-12-19T18:16:27.824487+0000","last_became_peered":"2023-12-19T18:16:27.824487+0000","last_unstale":"2023-12-19T18:16:40.554789+0000","last_undegraded":"2023-12-19T18:16:40.554789+0000","last_fullsized":"2023-12-19T18:16:40.554789+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550312+0000","last_change":"2023-12-19T18:16:40.150872+0000","last_active":"2023-12-19T18:16:41.550312+0000","last_peered":"2023-12-19T18:16:41.550312+0000","last_clean":"2023-12-19T18:16:41.550312+0000","last_became_active":"2023-12-19T18:16:40.150586+0000","last_became_peered":"2023-12-19T18:16:40.150586+0000","last_unstale":"2023-12-19T18:16:41.550312+0000","last_undegraded":"2023-12-19T18:16:41.550312+0000","last_fullsized":"2023-12-19T18:16:41.550312+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:55.343366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358620+0000","last_change":"2023-12-19T18:16:40.571454+0000","last_active":"2023-12-19T18:16:42.358620+0000","last_peered":"2023-12-19T18:16:42.358620+0000","last_clean":"2023-12-19T18:16:42.358620+0000","last_became_active":"2023-12-19T18:16:40.570299+0000","last_became_peered":"2023-12-19T18:16:40.570299+0000","last_unstale":"2023-12-19T18:16:42.358620+0000","last_undegraded":"2023-12-19T18:16:42.358620+0000","last_fullsized":"2023-12-19T18:16:42.358620+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:31:09.517418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"212'8","reported_seq":23,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553799+0000","last_change":"2023-12-19T18:16:27.414573+0000","last_active":"2023-12-19T18:16:40.553799+0000","last_peered":"2023-12-19T18:16:40.553799+0000","last_clean":"2023-12-19T18:16:40.553799+0000","last_became_active":"2023-12-19T18:16:27.414268+0000","last_became_peered":"2023-12-19T18:16:27.414268+0000","last_unstale":"2023-12-19T18:16:40.553799+0000","last_undegraded":"2023-12-19T18:16:40.553799+0000","last_fullsized":"2023-12-19T18:16:40.553799+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:11.117214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555559+0000","last_change":"2023-12-19T18:16:27.829311+0000","last_active":"2023-12-19T18:16:40.555559+0000","last_peered":"2023-12-19T18:16:40.555559+0000","last_clean":"2023-12-19T18:16:40.555559+0000","last_became_active":"2023-12-19T18:16:27.828953+0000","last_became_peered":"2023-12-19T18:16:27.828953+0000","last_unstale":"2023-12-19T18:16:40.555559+0000","last_undegraded":"2023-12-19T18:16:40.555559+0000","last_fullsized":"2023-12-19T18:16:40.555559+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:23.417710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554767+0000","last_change":"2023-12-19T18:16:40.160698+0000","last_active":"2023-12-19T18:16:41.554767+0000","last_peered":"2023-12-19T18:16:41.554767+0000","last_clean":"2023-12-19T18:16:41.554767+0000","last_became_active":"2023-12-19T18:16:40.160406+0000","last_became_peered":"2023-12-19T18:16:40.160406+0000","last_unstale":"2023-12-19T18:16:41.554767+0000","last_undegraded":"2023-12-19T18:16:41.554767+0000","last_fullsized":"2023-12-19T18:16:41.554767+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:10.441625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.15","version":"211'3","reported_seq":18,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555258+0000","last_change":"2023-12-19T18:16:27.409442+0000","last_active":"2023-12-19T18:16:40.555258+0000","last_peered":"2023-12-19T18:16:40.555258+0000","last_clean":"2023-12-19T18:16:40.555258+0000","last_became_active":"2023-12-19T18:16:27.409094+0000","last_became_peered":"2023-12-19T18:16:27.409094+0000","last_unstale":"2023-12-19T18:16:40.555258+0000","last_undegraded":"2023-12-19T18:16:40.555258+0000","last_fullsized":"2023-12-19T18:16:40.555258+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:18.713070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562498+0000","last_change":"2023-12-19T18:16:27.811752+0000","last_active":"2023-12-19T18:16:40.562498+0000","last_peered":"2023-12-19T18:16:40.562498+0000","last_clean":"2023-12-19T18:16:40.562498+0000","last_became_active":"2023-12-19T18:16:27.811273+0000","last_became_peered":"2023-12-19T18:16:27.811273+0000","last_unstale":"2023-12-19T18:16:40.562498+0000","last_undegraded":"2023-12-19T18:16:40.562498+0000","last_fullsized":"2023-12-19T18:16:40.562498+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:23.609660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360336+0000","last_change":"2023-12-19T18:16:40.162801+0000","last_active":"2023-12-19T18:16:42.360336+0000","last_peered":"2023-12-19T18:16:42.360336+0000","last_clean":"2023-12-19T18:16:42.360336+0000","last_became_active":"2023-12-19T18:16:40.162367+0000","last_became_peered":"2023-12-19T18:16:40.162367+0000","last_unstale":"2023-12-19T18:16:42.360336+0000","last_undegraded":"2023-12-19T18:16:42.360336+0000","last_fullsized":"2023-12-19T18:16:42.360336+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:08:09.383590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359637+0000","last_change":"2023-12-19T18:16:40.570887+0000","last_active":"2023-12-19T18:16:42.359637+0000","last_peered":"2023-12-19T18:16:42.359637+0000","last_clean":"2023-12-19T18:16:42.359637+0000","last_became_active":"2023-12-19T18:16:40.570475+0000","last_became_peered":"2023-12-19T18:16:40.570475+0000","last_unstale":"2023-12-19T18:16:42.359637+0000","last_undegraded":"2023-12-19T18:16:42.359637+0000","last_fullsized":"2023-12-19T18:16:42.359637+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:53:00.119855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549599+0000","last_change":"2023-12-19T18:16:27.410603+0000","last_active":"2023-12-19T18:16:40.549599+0000","last_peered":"2023-12-19T18:16:40.549599+0000","last_clean":"2023-12-19T18:16:40.549599+0000","last_became_active":"2023-12-19T18:16:27.409260+0000","last_became_peered":"2023-12-19T18:16:27.409260+0000","last_unstale":"2023-12-19T18:16:40.549599+0000","last_undegraded":"2023-12-19T18:16:40.549599+0000","last_fullsized":"2023-12-19T18:16:40.549599+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:06.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543878+0000","last_change":"2023-12-19T18:16:27.835564+0000","last_active":"2023-12-19T18:16:40.543878+0000","last_peered":"2023-12-19T18:16:40.543878+0000","last_clean":"2023-12-19T18:16:40.543878+0000","last_became_active":"2023-12-19T18:16:27.835319+0000","last_became_peered":"2023-12-19T18:16:27.835319+0000","last_unstale":"2023-12-19T18:16:40.543878+0000","last_undegraded":"2023-12-19T18:16:40.543878+0000","last_fullsized":"2023-12-19T18:16:40.543878+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:10:35.037185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:39.117449+0000","last_change":"2023-12-19T18:16:39.117449+0000","last_active":"2023-12-19T18:16:39.117449+0000","last_peered":"2023-12-19T18:16:39.117449+0000","last_clean":"2023-12-19T18:16:39.117449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:39.117449+0000","last_undegraded":"2023-12-19T18:16:39.117449+0000","last_fullsized":"2023-12-19T18:16:39.117449+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565329+0000","last_change":"2023-12-19T18:16:27.405556+0000","last_active":"2023-12-19T18:16:40.565329+0000","last_peered":"2023-12-19T18:16:40.565329+0000","last_clean":"2023-12-19T18:16:40.565329+0000","last_became_active":"2023-12-19T18:16:27.404804+0000","last_became_peered":"2023-12-19T18:16:27.404804+0000","last_unstale":"2023-12-19T18:16:40.565329+0000","last_undegraded":"2023-12-19T18:16:40.565329+0000","last_fullsized":"2023-12-19T18:16:40.565329+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:05.395130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551274+0000","last_change":"2023-12-19T18:16:27.830538+0000","last_active":"2023-12-19T18:16:40.551274+0000","last_peered":"2023-12-19T18:16:40.551274+0000","last_clean":"2023-12-19T18:16:40.551274+0000","last_became_active":"2023-12-19T18:16:27.828974+0000","last_became_peered":"2023-12-19T18:16:27.828974+0000","last_unstale":"2023-12-19T18:16:40.551274+0000","last_undegraded":"2023-12-19T18:16:40.551274+0000","last_fullsized":"2023-12-19T18:16:40.551274+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:36.961741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.15","version":"215'3","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551392+0000","last_change":"2023-12-19T18:16:40.151095+0000","last_active":"2023-12-19T18:16:41.551392+0000","last_peered":"2023-12-19T18:16:41.551392+0000","last_clean":"2023-12-19T18:16:41.551392+0000","last_became_active":"2023-12-19T18:16:40.150609+0000","last_became_peered":"2023-12-19T18:16:40.150609+0000","last_unstale":"2023-12-19T18:16:41.551392+0000","last_undegraded":"2023-12-19T18:16:41.551392+0000","last_fullsized":"2023-12-19T18:16:41.551392+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:15.058778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305298+0000","last_change":"2023-12-19T18:16:27.413444+0000","last_active":"2023-12-19T18:16:28.305298+0000","last_peered":"2023-12-19T18:16:28.305298+0000","last_clean":"2023-12-19T18:16:28.305298+0000","last_became_active":"2023-12-19T18:16:27.413066+0000","last_became_peered":"2023-12-19T18:16:27.413066+0000","last_unstale":"2023-12-19T18:16:28.305298+0000","last_undegraded":"2023-12-19T18:16:28.305298+0000","last_fullsized":"2023-12-19T18:16:28.305298+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:57.997657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558113+0000","last_change":"2023-12-19T18:16:27.816735+0000","last_active":"2023-12-19T18:16:40.558113+0000","last_peered":"2023-12-19T18:16:40.558113+0000","last_clean":"2023-12-19T18:16:40.558113+0000","last_became_active":"2023-12-19T18:16:27.816436+0000","last_became_peered":"2023-12-19T18:16:27.816436+0000","last_unstale":"2023-12-19T18:16:40.558113+0000","last_undegraded":"2023-12-19T18:16:40.558113+0000","last_fullsized":"2023-12-19T18:16:40.558113+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:37.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1a","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359740+0000","last_change":"2023-12-19T18:16:40.161453+0000","last_active":"2023-12-19T18:16:42.359740+0000","last_peered":"2023-12-19T18:16:42.359740+0000","last_clean":"2023-12-19T18:16:42.359740+0000","last_became_active":"2023-12-19T18:16:40.161191+0000","last_became_peered":"2023-12-19T18:16:40.161191+0000","last_unstale":"2023-12-19T18:16:42.359740+0000","last_undegraded":"2023-12-19T18:16:42.359740+0000","last_fullsized":"2023-12-19T18:16:42.359740+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:13.660363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"24.1a","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544353+0000","last_change":"2023-12-19T18:16:27.414372+0000","last_active":"2023-12-19T18:16:40.544353+0000","last_peered":"2023-12-19T18:16:40.544353+0000","last_clean":"2023-12-19T18:16:40.544353+0000","last_became_active":"2023-12-19T18:16:27.414088+0000","last_became_peered":"2023-12-19T18:16:27.414088+0000","last_unstale":"2023-12-19T18:16:40.544353+0000","last_undegraded":"2023-12-19T18:16:40.544353+0000","last_fullsized":"2023-12-19T18:16:40.544353+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:18.945766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543503+0000","last_change":"2023-12-19T18:16:27.827386+0000","last_active":"2023-12-19T18:16:40.543503+0000","last_peered":"2023-12-19T18:16:40.543503+0000","last_clean":"2023-12-19T18:16:40.543503+0000","last_became_active":"2023-12-19T18:16:27.827086+0000","last_became_peered":"2023-12-19T18:16:27.827086+0000","last_unstale":"2023-12-19T18:16:40.543503+0000","last_undegraded":"2023-12-19T18:16:40.543503+0000","last_fullsized":"2023-12-19T18:16:40.543503+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:37.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554117+0000","last_change":"2023-12-19T18:16:40.156738+0000","last_active":"2023-12-19T18:16:41.554117+0000","last_peered":"2023-12-19T18:16:41.554117+0000","last_clean":"2023-12-19T18:16:41.554117+0000","last_became_active":"2023-12-19T18:16:40.156488+0000","last_became_peered":"2023-12-19T18:16:40.156488+0000","last_unstale":"2023-12-19T18:16:41.554117+0000","last_undegraded":"2023-12-19T18:16:41.554117+0000","last_fullsized":"2023-12-19T18:16:41.554117+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:48.695934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358850+0000","last_change":"2023-12-19T18:16:40.575709+0000","last_active":"2023-12-19T18:16:42.358850+0000","last_peered":"2023-12-19T18:16:42.358850+0000","last_clean":"2023-12-19T18:16:42.358850+0000","last_became_active":"2023-12-19T18:16:40.575453+0000","last_became_peered":"2023-12-19T18:16:40.575453+0000","last_unstale":"2023-12-19T18:16:42.358850+0000","last_undegraded":"2023-12-19T18:16:42.358850+0000","last_fullsized":"2023-12-19T18:16:42.358850+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:44.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1b","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551641+0000","last_change":"2023-12-19T18:16:27.415036+0000","last_active":"2023-12-19T18:16:40.551641+0000","last_peered":"2023-12-19T18:16:40.551641+0000","last_clean":"2023-12-19T18:16:40.551641+0000","last_became_active":"2023-12-19T18:16:27.414747+0000","last_became_peered":"2023-12-19T18:16:27.414747+0000","last_unstale":"2023-12-19T18:16:40.551641+0000","last_undegraded":"2023-12-19T18:16:40.551641+0000","last_fullsized":"2023-12-19T18:16:40.551641+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:03.064138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556794+0000","last_change":"2023-12-19T18:16:27.828398+0000","last_active":"2023-12-19T18:16:40.556794+0000","last_peered":"2023-12-19T18:16:40.556794+0000","last_clean":"2023-12-19T18:16:40.556794+0000","last_became_active":"2023-12-19T18:16:27.828082+0000","last_became_peered":"2023-12-19T18:16:27.828082+0000","last_unstale":"2023-12-19T18:16:40.556794+0000","last_undegraded":"2023-12-19T18:16:40.556794+0000","last_fullsized":"2023-12-19T18:16:40.556794+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:42:30.998150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358258+0000","last_change":"2023-12-19T18:16:40.159943+0000","last_active":"2023-12-19T18:16:42.358258+0000","last_peered":"2023-12-19T18:16:42.358258+0000","last_clean":"2023-12-19T18:16:42.358258+0000","last_became_active":"2023-12-19T18:16:40.159652+0000","last_became_peered":"2023-12-19T18:16:40.159652+0000","last_unstale":"2023-12-19T18:16:42.358258+0000","last_undegraded":"2023-12-19T18:16:42.358258+0000","last_fullsized":"2023-12-19T18:16:42.358258+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:02:40.928028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.555138+0000","last_change":"2023-12-19T18:16:40.573382+0000","last_active":"2023-12-19T18:16:41.555138+0000","last_peered":"2023-12-19T18:16:41.555138+0000","last_clean":"2023-12-19T18:16:41.555138+0000","last_became_active":"2023-12-19T18:16:40.573092+0000","last_became_peered":"2023-12-19T18:16:40.573092+0000","last_unstale":"2023-12-19T18:16:41.555138+0000","last_undegraded":"2023-12-19T18:16:41.555138+0000","last_fullsized":"2023-12-19T18:16:41.555138+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:10:47.778609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553461+0000","last_change":"2023-12-19T18:16:40.163131+0000","last_active":"2023-12-19T18:16:41.553461+0000","last_peered":"2023-12-19T18:16:41.553461+0000","last_clean":"2023-12-19T18:16:41.553461+0000","last_became_active":"2023-12-19T18:16:40.162621+0000","last_became_peered":"2023-12-19T18:16:40.162621+0000","last_unstale":"2023-12-19T18:16:41.553461+0000","last_undegraded":"2023-12-19T18:16:41.553461+0000","last_fullsized":"2023-12-19T18:16:41.553461+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:57.979757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559871+0000","last_change":"2023-12-19T18:16:27.409441+0000","last_active":"2023-12-19T18:16:40.559871+0000","last_peered":"2023-12-19T18:16:40.559871+0000","last_clean":"2023-12-19T18:16:40.559871+0000","last_became_active":"2023-12-19T18:16:27.408438+0000","last_became_peered":"2023-12-19T18:16:27.408438+0000","last_unstale":"2023-12-19T18:16:40.559871+0000","last_undegraded":"2023-12-19T18:16:40.559871+0000","last_fullsized":"2023-12-19T18:16:40.559871+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:40.980300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546499+0000","last_change":"2023-12-19T18:16:27.814063+0000","last_active":"2023-12-19T18:16:40.546499+0000","last_peered":"2023-12-19T18:16:40.546499+0000","last_clean":"2023-12-19T18:16:40.546499+0000","last_became_active":"2023-12-19T18:16:27.813564+0000","last_became_peered":"2023-12-19T18:16:27.813564+0000","last_unstale":"2023-12-19T18:16:40.546499+0000","last_undegraded":"2023-12-19T18:16:40.546499+0000","last_fullsized":"2023-12-19T18:16:40.546499+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:36.733148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553348+0000","last_change":"2023-12-19T18:16:40.564402+0000","last_active":"2023-12-19T18:16:41.553348+0000","last_peered":"2023-12-19T18:16:41.553348+0000","last_clean":"2023-12-19T18:16:41.553348+0000","last_became_active":"2023-12-19T18:16:40.561338+0000","last_became_peered":"2023-12-19T18:16:40.561338+0000","last_unstale":"2023-12-19T18:16:41.553348+0000","last_undegraded":"2023-12-19T18:16:41.553348+0000","last_fullsized":"2023-12-19T18:16:41.553348+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:37:11.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"215'4","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857590+0000","last_change":"2023-12-19T18:16:40.152799+0000","last_active":"2023-12-19T18:16:41.857590+0000","last_peered":"2023-12-19T18:16:41.857590+0000","last_clean":"2023-12-19T18:16:41.857590+0000","last_became_active":"2023-12-19T18:16:40.152482+0000","last_became_peered":"2023-12-19T18:16:40.152482+0000","last_unstale":"2023-12-19T18:16:41.857590+0000","last_undegraded":"2023-12-19T18:16:41.857590+0000","last_fullsized":"2023-12-19T18:16:41.857590+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:23.854189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1d","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548666+0000","last_change":"2023-12-19T18:16:27.415013+0000","last_active":"2023-12-19T18:16:40.548666+0000","last_peered":"2023-12-19T18:16:40.548666+0000","last_clean":"2023-12-19T18:16:40.548666+0000","last_became_active":"2023-12-19T18:16:27.414582+0000","last_became_peered":"2023-12-19T18:16:27.414582+0000","last_unstale":"2023-12-19T18:16:40.548666+0000","last_undegraded":"2023-12-19T18:16:40.548666+0000","last_fullsized":"2023-12-19T18:16:40.548666+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:34:01.457288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562417+0000","last_change":"2023-12-19T18:16:27.829031+0000","last_active":"2023-12-19T18:16:40.562417+0000","last_peered":"2023-12-19T18:16:40.562417+0000","last_clean":"2023-12-19T18:16:40.562417+0000","last_became_active":"2023-12-19T18:16:27.826634+0000","last_became_peered":"2023-12-19T18:16:27.826634+0000","last_unstale":"2023-12-19T18:16:40.562417+0000","last_undegraded":"2023-12-19T18:16:40.562417+0000","last_fullsized":"2023-12-19T18:16:40.562417+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:23.429228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551066+0000","last_change":"2023-12-19T18:16:40.570487+0000","last_active":"2023-12-19T18:16:41.551066+0000","last_peered":"2023-12-19T18:16:41.551066+0000","last_clean":"2023-12-19T18:16:41.551066+0000","last_became_active":"2023-12-19T18:16:40.569839+0000","last_became_peered":"2023-12-19T18:16:40.569839+0000","last_unstale":"2023-12-19T18:16:41.551066+0000","last_undegraded":"2023-12-19T18:16:41.551066+0000","last_fullsized":"2023-12-19T18:16:41.551066+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:59.414695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548366+0000","last_change":"2023-12-19T18:16:27.407669+0000","last_active":"2023-12-19T18:16:40.548366+0000","last_peered":"2023-12-19T18:16:40.548366+0000","last_clean":"2023-12-19T18:16:40.548366+0000","last_became_active":"2023-12-19T18:16:27.407150+0000","last_became_peered":"2023-12-19T18:16:27.407150+0000","last_unstale":"2023-12-19T18:16:40.548366+0000","last_undegraded":"2023-12-19T18:16:40.548366+0000","last_fullsized":"2023-12-19T18:16:40.548366+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:59.675090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558634+0000","last_change":"2023-12-19T18:16:27.830766+0000","last_active":"2023-12-19T18:16:40.558634+0000","last_peered":"2023-12-19T18:16:40.558634+0000","last_clean":"2023-12-19T18:16:40.558634+0000","last_became_active":"2023-12-19T18:16:27.830220+0000","last_became_peered":"2023-12-19T18:16:27.830220+0000","last_unstale":"2023-12-19T18:16:40.558634+0000","last_undegraded":"2023-12-19T18:16:40.558634+0000","last_fullsized":"2023-12-19T18:16:40.558634+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:03:17.772391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:39.117449+0000","last_change":"2023-12-19T18:16:39.117449+0000","last_active":"2023-12-19T18:16:39.117449+0000","last_peered":"2023-12-19T18:16:39.117449+0000","last_clean":"2023-12-19T18:16:39.117449+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:39.117449+0000","last_undegraded":"2023-12-19T18:16:39.117449+0000","last_fullsized":"2023-12-19T18:16:39.117449+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359497+0000","last_change":"2023-12-19T18:16:40.565564+0000","last_active":"2023-12-19T18:16:42.359497+0000","last_peered":"2023-12-19T18:16:42.359497+0000","last_clean":"2023-12-19T18:16:42.359497+0000","last_became_active":"2023-12-19T18:16:40.565300+0000","last_became_peered":"2023-12-19T18:16:40.565300+0000","last_unstale":"2023-12-19T18:16:42.359497+0000","last_undegraded":"2023-12-19T18:16:42.359497+0000","last_fullsized":"2023-12-19T18:16:42.359497+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:14.849896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1f","version":"211'4","reported_seq":19,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553798+0000","last_change":"2023-12-19T18:16:27.414364+0000","last_active":"2023-12-19T18:16:40.553798+0000","last_peered":"2023-12-19T18:16:40.553798+0000","last_clean":"2023-12-19T18:16:40.553798+0000","last_became_active":"2023-12-19T18:16:27.413648+0000","last_became_peered":"2023-12-19T18:16:27.413648+0000","last_unstale":"2023-12-19T18:16:40.553798+0000","last_undegraded":"2023-12-19T18:16:40.553798+0000","last_fullsized":"2023-12-19T18:16:40.553798+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:27:32.703908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544529+0000","last_change":"2023-12-19T18:16:27.830378+0000","last_active":"2023-12-19T18:16:40.544529+0000","last_peered":"2023-12-19T18:16:40.544529+0000","last_clean":"2023-12-19T18:16:40.544529+0000","last_became_active":"2023-12-19T18:16:27.830122+0000","last_became_peered":"2023-12-19T18:16:27.830122+0000","last_unstale":"2023-12-19T18:16:40.544529+0000","last_undegraded":"2023-12-19T18:16:40.544529+0000","last_fullsized":"2023-12-19T18:16:40.544529+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:21.365516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1d","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358451+0000","last_change":"2023-12-19T18:16:40.162566+0000","last_active":"2023-12-19T18:16:42.358451+0000","last_peered":"2023-12-19T18:16:42.358451+0000","last_clean":"2023-12-19T18:16:42.358451+0000","last_became_active":"2023-12-19T18:16:40.162255+0000","last_became_peered":"2023-12-19T18:16:40.162255+0000","last_unstale":"2023-12-19T18:16:42.358451+0000","last_undegraded":"2023-12-19T18:16:42.358451+0000","last_fullsized":"2023-12-19T18:16:42.358451+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:20.530183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552668+0000","last_change":"2023-12-19T18:16:40.570029+0000","last_active":"2023-12-19T18:16:41.552668+0000","last_peered":"2023-12-19T18:16:41.552668+0000","last_clean":"2023-12-19T18:16:41.552668+0000","last_became_active":"2023-12-19T18:16:40.569731+0000","last_became_peered":"2023-12-19T18:16:40.569731+0000","last_unstale":"2023-12-19T18:16:41.552668+0000","last_undegraded":"2023-12-19T18:16:41.552668+0000","last_fullsized":"2023-12-19T18:16:41.552668+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:25:46.635350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550877+0000","last_change":"2023-12-19T18:16:27.831030+0000","last_active":"2023-12-19T18:16:40.550877+0000","last_peered":"2023-12-19T18:16:40.550877+0000","last_clean":"2023-12-19T18:16:40.550877+0000","last_became_active":"2023-12-19T18:16:27.830764+0000","last_became_peered":"2023-12-19T18:16:27.830764+0000","last_unstale":"2023-12-19T18:16:40.550877+0000","last_undegraded":"2023-12-19T18:16:40.550877+0000","last_fullsized":"2023-12-19T18:16:40.550877+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:11.685663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857085+0000","last_change":"2023-12-19T18:16:40.569158+0000","last_active":"2023-12-19T18:16:41.857085+0000","last_peered":"2023-12-19T18:16:41.857085+0000","last_clean":"2023-12-19T18:16:41.857085+0000","last_became_active":"2023-12-19T18:16:40.568894+0000","last_became_peered":"2023-12-19T18:16:40.568894+0000","last_unstale":"2023-12-19T18:16:41.857085+0000","last_undegraded":"2023-12-19T18:16:41.857085+0000","last_fullsized":"2023-12-19T18:16:41.857085+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:02.018340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553603+0000","last_change":"2023-12-19T18:16:27.833518+0000","last_active":"2023-12-19T18:16:40.553603+0000","last_peered":"2023-12-19T18:16:40.553603+0000","last_clean":"2023-12-19T18:16:40.553603+0000","last_became_active":"2023-12-19T18:16:27.833251+0000","last_became_peered":"2023-12-19T18:16:27.833251+0000","last_unstale":"2023-12-19T18:16:40.553603+0000","last_undegraded":"2023-12-19T18:16:40.553603+0000","last_fullsized":"2023-12-19T18:16:40.553603+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:49.276917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357285+0000","last_change":"2023-12-19T18:16:40.563610+0000","last_active":"2023-12-19T18:16:42.357285+0000","last_peered":"2023-12-19T18:16:42.357285+0000","last_clean":"2023-12-19T18:16:42.357285+0000","last_became_active":"2023-12-19T18:16:40.563300+0000","last_became_peered":"2023-12-19T18:16:40.563300+0000","last_unstale":"2023-12-19T18:16:42.357285+0000","last_undegraded":"2023-12-19T18:16:42.357285+0000","last_fullsized":"2023-12-19T18:16:42.357285+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:28:48.518493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547548+0000","last_change":"2023-12-19T18:16:27.825014+0000","last_active":"2023-12-19T18:16:40.547548+0000","last_peered":"2023-12-19T18:16:40.547548+0000","last_clean":"2023-12-19T18:16:40.547548+0000","last_became_active":"2023-12-19T18:16:27.822140+0000","last_became_peered":"2023-12-19T18:16:27.822140+0000","last_unstale":"2023-12-19T18:16:40.547548+0000","last_undegraded":"2023-12-19T18:16:40.547548+0000","last_fullsized":"2023-12-19T18:16:40.547548+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:36.972809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550313+0000","last_change":"2023-12-19T18:16:40.576594+0000","last_active":"2023-12-19T18:16:41.550313+0000","last_peered":"2023-12-19T18:16:41.550313+0000","last_clean":"2023-12-19T18:16:41.550313+0000","last_became_active":"2023-12-19T18:16:40.576304+0000","last_became_peered":"2023-12-19T18:16:40.576304+0000","last_unstale":"2023-12-19T18:16:41.550313+0000","last_undegraded":"2023-12-19T18:16:41.550313+0000","last_fullsized":"2023-12-19T18:16:41.550313+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:58.373923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561436+0000","last_change":"2023-12-19T18:16:27.826500+0000","last_active":"2023-12-19T18:16:40.561436+0000","last_peered":"2023-12-19T18:16:40.561436+0000","last_clean":"2023-12-19T18:16:40.561436+0000","last_became_active":"2023-12-19T18:16:27.826216+0000","last_became_peered":"2023-12-19T18:16:27.826216+0000","last_unstale":"2023-12-19T18:16:40.561436+0000","last_undegraded":"2023-12-19T18:16:40.561436+0000","last_fullsized":"2023-12-19T18:16:40.561436+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:33.032362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359854+0000","last_change":"2023-12-19T18:16:40.565942+0000","last_active":"2023-12-19T18:16:42.359854+0000","last_peered":"2023-12-19T18:16:42.359854+0000","last_clean":"2023-12-19T18:16:42.359854+0000","last_became_active":"2023-12-19T18:16:40.565557+0000","last_became_peered":"2023-12-19T18:16:40.565557+0000","last_unstale":"2023-12-19T18:16:42.359854+0000","last_undegraded":"2023-12-19T18:16:42.359854+0000","last_fullsized":"2023-12-19T18:16:42.359854+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:23.839152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548841+0000","last_change":"2023-12-19T18:16:27.830103+0000","last_active":"2023-12-19T18:16:40.548841+0000","last_peered":"2023-12-19T18:16:40.548841+0000","last_clean":"2023-12-19T18:16:40.548841+0000","last_became_active":"2023-12-19T18:16:27.829444+0000","last_became_peered":"2023-12-19T18:16:27.829444+0000","last_unstale":"2023-12-19T18:16:40.548841+0000","last_undegraded":"2023-12-19T18:16:40.548841+0000","last_fullsized":"2023-12-19T18:16:40.548841+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:50:29.947932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553462+0000","last_change":"2023-12-19T18:16:40.563980+0000","last_active":"2023-12-19T18:16:41.553462+0000","last_peered":"2023-12-19T18:16:41.553462+0000","last_clean":"2023-12-19T18:16:41.553462+0000","last_became_active":"2023-12-19T18:16:40.563683+0000","last_became_peered":"2023-12-19T18:16:40.563683+0000","last_unstale":"2023-12-19T18:16:41.553462+0000","last_undegraded":"2023-12-19T18:16:41.553462+0000","last_fullsized":"2023-12-19T18:16:41.553462+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:36.170652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548463+0000","last_change":"2023-12-19T18:16:27.820913+0000","last_active":"2023-12-19T18:16:40.548463+0000","last_peered":"2023-12-19T18:16:40.548463+0000","last_clean":"2023-12-19T18:16:40.548463+0000","last_became_active":"2023-12-19T18:16:27.820634+0000","last_became_peered":"2023-12-19T18:16:27.820634+0000","last_unstale":"2023-12-19T18:16:40.548463+0000","last_undegraded":"2023-12-19T18:16:40.548463+0000","last_fullsized":"2023-12-19T18:16:40.548463+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:53:09.066279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357879+0000","last_change":"2023-12-19T18:16:40.562872+0000","last_active":"2023-12-19T18:16:42.357879+0000","last_peered":"2023-12-19T18:16:42.357879+0000","last_clean":"2023-12-19T18:16:42.357879+0000","last_became_active":"2023-12-19T18:16:40.562615+0000","last_became_peered":"2023-12-19T18:16:40.562615+0000","last_unstale":"2023-12-19T18:16:42.357879+0000","last_undegraded":"2023-12-19T18:16:42.357879+0000","last_fullsized":"2023-12-19T18:16:42.357879+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:54.454071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565322+0000","last_change":"2023-12-19T18:16:27.830119+0000","last_active":"2023-12-19T18:16:40.565322+0000","last_peered":"2023-12-19T18:16:40.565322+0000","last_clean":"2023-12-19T18:16:40.565322+0000","last_became_active":"2023-12-19T18:16:27.829850+0000","last_became_peered":"2023-12-19T18:16:27.829850+0000","last_unstale":"2023-12-19T18:16:40.565322+0000","last_undegraded":"2023-12-19T18:16:40.565322+0000","last_fullsized":"2023-12-19T18:16:40.565322+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:51.190615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857238+0000","last_change":"2023-12-19T18:16:40.570092+0000","last_active":"2023-12-19T18:16:41.857238+0000","last_peered":"2023-12-19T18:16:41.857238+0000","last_clean":"2023-12-19T18:16:41.857238+0000","last_became_active":"2023-12-19T18:16:40.568697+0000","last_became_peered":"2023-12-19T18:16:40.568697+0000","last_unstale":"2023-12-19T18:16:41.857238+0000","last_undegraded":"2023-12-19T18:16:41.857238+0000","last_fullsized":"2023-12-19T18:16:41.857238+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:11.448855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.564229+0000","last_change":"2023-12-19T18:16:27.834875+0000","last_active":"2023-12-19T18:16:40.564229+0000","last_peered":"2023-12-19T18:16:40.564229+0000","last_clean":"2023-12-19T18:16:40.564229+0000","last_became_active":"2023-12-19T18:16:27.834561+0000","last_became_peered":"2023-12-19T18:16:27.834561+0000","last_unstale":"2023-12-19T18:16:40.564229+0000","last_undegraded":"2023-12-19T18:16:40.564229+0000","last_fullsized":"2023-12-19T18:16:40.564229+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:30.301379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554348+0000","last_change":"2023-12-19T18:16:40.561527+0000","last_active":"2023-12-19T18:16:41.554348+0000","last_peered":"2023-12-19T18:16:41.554348+0000","last_clean":"2023-12-19T18:16:41.554348+0000","last_became_active":"2023-12-19T18:16:40.561251+0000","last_became_peered":"2023-12-19T18:16:40.561251+0000","last_unstale":"2023-12-19T18:16:41.554348+0000","last_undegraded":"2023-12-19T18:16:41.554348+0000","last_fullsized":"2023-12-19T18:16:41.554348+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:50.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543417+0000","last_change":"2023-12-19T18:16:27.826642+0000","last_active":"2023-12-19T18:16:40.543417+0000","last_peered":"2023-12-19T18:16:40.543417+0000","last_clean":"2023-12-19T18:16:40.543417+0000","last_became_active":"2023-12-19T18:16:27.825460+0000","last_became_peered":"2023-12-19T18:16:27.825460+0000","last_unstale":"2023-12-19T18:16:40.543417+0000","last_undegraded":"2023-12-19T18:16:40.543417+0000","last_fullsized":"2023-12-19T18:16:40.543417+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:05.671602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551313+0000","last_change":"2023-12-19T18:16:27.826145+0000","last_active":"2023-12-19T18:16:40.551313+0000","last_peered":"2023-12-19T18:16:40.551313+0000","last_clean":"2023-12-19T18:16:40.551313+0000","last_became_active":"2023-12-19T18:16:27.825889+0000","last_became_peered":"2023-12-19T18:16:27.825889+0000","last_unstale":"2023-12-19T18:16:40.551313+0000","last_undegraded":"2023-12-19T18:16:40.551313+0000","last_fullsized":"2023-12-19T18:16:40.551313+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:59:41.673539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555865+0000","last_change":"2023-12-19T18:16:27.830084+0000","last_active":"2023-12-19T18:16:40.555865+0000","last_peered":"2023-12-19T18:16:40.555865+0000","last_clean":"2023-12-19T18:16:40.555865+0000","last_became_active":"2023-12-19T18:16:27.829787+0000","last_became_peered":"2023-12-19T18:16:27.829787+0000","last_unstale":"2023-12-19T18:16:40.555865+0000","last_undegraded":"2023-12-19T18:16:40.555865+0000","last_fullsized":"2023-12-19T18:16:40.555865+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:34:16.356894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554193+0000","last_change":"2023-12-19T18:16:40.563996+0000","last_active":"2023-12-19T18:16:41.554193+0000","last_peered":"2023-12-19T18:16:41.554193+0000","last_clean":"2023-12-19T18:16:41.554193+0000","last_became_active":"2023-12-19T18:16:40.563686+0000","last_became_peered":"2023-12-19T18:16:40.563686+0000","last_unstale":"2023-12-19T18:16:41.554193+0000","last_undegraded":"2023-12-19T18:16:41.554193+0000","last_fullsized":"2023-12-19T18:16:41.554193+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:34.106940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551542+0000","last_change":"2023-12-19T18:16:27.831551+0000","last_active":"2023-12-19T18:16:40.551542+0000","last_peered":"2023-12-19T18:16:40.551542+0000","last_clean":"2023-12-19T18:16:40.551542+0000","last_became_active":"2023-12-19T18:16:27.831291+0000","last_became_peered":"2023-12-19T18:16:27.831291+0000","last_unstale":"2023-12-19T18:16:40.551542+0000","last_undegraded":"2023-12-19T18:16:40.551542+0000","last_fullsized":"2023-12-19T18:16:40.551542+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:09.914084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551186+0000","last_change":"2023-12-19T18:16:40.563764+0000","last_active":"2023-12-19T18:16:41.551186+0000","last_peered":"2023-12-19T18:16:41.551186+0000","last_clean":"2023-12-19T18:16:41.551186+0000","last_became_active":"2023-12-19T18:16:40.563464+0000","last_became_peered":"2023-12-19T18:16:40.563464+0000","last_unstale":"2023-12-19T18:16:41.551186+0000","last_undegraded":"2023-12-19T18:16:41.551186+0000","last_fullsized":"2023-12-19T18:16:41.551186+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:52:34.311679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548663+0000","last_change":"2023-12-19T18:16:27.833054+0000","last_active":"2023-12-19T18:16:40.548663+0000","last_peered":"2023-12-19T18:16:40.548663+0000","last_clean":"2023-12-19T18:16:40.548663+0000","last_became_active":"2023-12-19T18:16:27.832738+0000","last_became_peered":"2023-12-19T18:16:27.832738+0000","last_unstale":"2023-12-19T18:16:40.548663+0000","last_undegraded":"2023-12-19T18:16:40.548663+0000","last_fullsized":"2023-12-19T18:16:40.548663+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:57.892685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360068+0000","last_change":"2023-12-19T18:16:40.577515+0000","last_active":"2023-12-19T18:16:42.360068+0000","last_peered":"2023-12-19T18:16:42.360068+0000","last_clean":"2023-12-19T18:16:42.360068+0000","last_became_active":"2023-12-19T18:16:40.577096+0000","last_became_peered":"2023-12-19T18:16:40.577096+0000","last_unstale":"2023-12-19T18:16:42.360068+0000","last_undegraded":"2023-12-19T18:16:42.360068+0000","last_fullsized":"2023-12-19T18:16:42.360068+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:36.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555479+0000","last_change":"2023-12-19T18:16:27.820194+0000","last_active":"2023-12-19T18:16:40.555479+0000","last_peered":"2023-12-19T18:16:40.555479+0000","last_clean":"2023-12-19T18:16:40.555479+0000","last_became_active":"2023-12-19T18:16:27.819900+0000","last_became_peered":"2023-12-19T18:16:27.819900+0000","last_unstale":"2023-12-19T18:16:40.555479+0000","last_undegraded":"2023-12-19T18:16:40.555479+0000","last_fullsized":"2023-12-19T18:16:40.555479+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:00.858729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360137+0000","last_change":"2023-12-19T18:16:40.561878+0000","last_active":"2023-12-19T18:16:42.360137+0000","last_peered":"2023-12-19T18:16:42.360137+0000","last_clean":"2023-12-19T18:16:42.360137+0000","last_became_active":"2023-12-19T18:16:40.561626+0000","last_became_peered":"2023-12-19T18:16:40.561626+0000","last_unstale":"2023-12-19T18:16:42.360137+0000","last_undegraded":"2023-12-19T18:16:42.360137+0000","last_fullsized":"2023-12-19T18:16:42.360137+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:38:57.938991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305416+0000","last_change":"2023-12-19T18:16:27.820781+0000","last_active":"2023-12-19T18:16:28.305416+0000","last_peered":"2023-12-19T18:16:28.305416+0000","last_clean":"2023-12-19T18:16:28.305416+0000","last_became_active":"2023-12-19T18:16:27.820483+0000","last_became_peered":"2023-12-19T18:16:27.820483+0000","last_unstale":"2023-12-19T18:16:28.305416+0000","last_undegraded":"2023-12-19T18:16:28.305416+0000","last_fullsized":"2023-12-19T18:16:28.305416+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:23.445435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551324+0000","last_change":"2023-12-19T18:16:40.560485+0000","last_active":"2023-12-19T18:16:41.551324+0000","last_peered":"2023-12-19T18:16:41.551324+0000","last_clean":"2023-12-19T18:16:41.551324+0000","last_became_active":"2023-12-19T18:16:40.560232+0000","last_became_peered":"2023-12-19T18:16:40.560232+0000","last_unstale":"2023-12-19T18:16:41.551324+0000","last_undegraded":"2023-12-19T18:16:41.551324+0000","last_fullsized":"2023-12-19T18:16:41.551324+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:30.252976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553708+0000","last_change":"2023-12-19T18:16:27.834880+0000","last_active":"2023-12-19T18:16:40.553708+0000","last_peered":"2023-12-19T18:16:40.553708+0000","last_clean":"2023-12-19T18:16:40.553708+0000","last_became_active":"2023-12-19T18:16:27.834494+0000","last_became_peered":"2023-12-19T18:16:27.834494+0000","last_unstale":"2023-12-19T18:16:40.553708+0000","last_undegraded":"2023-12-19T18:16:40.553708+0000","last_fullsized":"2023-12-19T18:16:40.553708+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:06.992701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357587+0000","last_change":"2023-12-19T18:16:40.567126+0000","last_active":"2023-12-19T18:16:42.357587+0000","last_peered":"2023-12-19T18:16:42.357587+0000","last_clean":"2023-12-19T18:16:42.357587+0000","last_became_active":"2023-12-19T18:16:40.566878+0000","last_became_peered":"2023-12-19T18:16:40.566878+0000","last_unstale":"2023-12-19T18:16:42.357587+0000","last_undegraded":"2023-12-19T18:16:42.357587+0000","last_fullsized":"2023-12-19T18:16:42.357587+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:59.062853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543300+0000","last_change":"2023-12-19T18:16:27.826828+0000","last_active":"2023-12-19T18:16:40.543300+0000","last_peered":"2023-12-19T18:16:40.543300+0000","last_clean":"2023-12-19T18:16:40.543300+0000","last_became_active":"2023-12-19T18:16:27.825994+0000","last_became_peered":"2023-12-19T18:16:27.825994+0000","last_unstale":"2023-12-19T18:16:40.543300+0000","last_undegraded":"2023-12-19T18:16:40.543300+0000","last_fullsized":"2023-12-19T18:16:40.543300+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:17.748218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857358+0000","last_change":"2023-12-19T18:16:40.569008+0000","last_active":"2023-12-19T18:16:41.857358+0000","last_peered":"2023-12-19T18:16:41.857358+0000","last_clean":"2023-12-19T18:16:41.857358+0000","last_became_active":"2023-12-19T18:16:40.568420+0000","last_became_peered":"2023-12-19T18:16:40.568420+0000","last_unstale":"2023-12-19T18:16:41.857358+0000","last_undegraded":"2023-12-19T18:16:41.857358+0000","last_fullsized":"2023-12-19T18:16:41.857358+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:31:10.720873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561285+0000","last_change":"2023-12-19T18:16:27.830751+0000","last_active":"2023-12-19T18:16:40.561285+0000","last_peered":"2023-12-19T18:16:40.561285+0000","last_clean":"2023-12-19T18:16:40.561285+0000","last_became_active":"2023-12-19T18:16:27.830490+0000","last_became_peered":"2023-12-19T18:16:27.830490+0000","last_unstale":"2023-12-19T18:16:40.561285+0000","last_undegraded":"2023-12-19T18:16:40.561285+0000","last_fullsized":"2023-12-19T18:16:40.561285+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:31.093343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550245+0000","last_change":"2023-12-19T18:16:40.571696+0000","last_active":"2023-12-19T18:16:41.550245+0000","last_peered":"2023-12-19T18:16:41.550245+0000","last_clean":"2023-12-19T18:16:41.550245+0000","last_became_active":"2023-12-19T18:16:40.571407+0000","last_became_peered":"2023-12-19T18:16:40.571407+0000","last_unstale":"2023-12-19T18:16:41.550245+0000","last_undegraded":"2023-12-19T18:16:41.550245+0000","last_fullsized":"2023-12-19T18:16:41.550245+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:55.222921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548271+0000","last_change":"2023-12-19T18:16:27.823186+0000","last_active":"2023-12-19T18:16:40.548271+0000","last_peered":"2023-12-19T18:16:40.548271+0000","last_clean":"2023-12-19T18:16:40.548271+0000","last_became_active":"2023-12-19T18:16:27.820405+0000","last_became_peered":"2023-12-19T18:16:27.820405+0000","last_unstale":"2023-12-19T18:16:40.548271+0000","last_undegraded":"2023-12-19T18:16:40.548271+0000","last_fullsized":"2023-12-19T18:16:40.548271+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:41.515700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357953+0000","last_change":"2023-12-19T18:16:40.569297+0000","last_active":"2023-12-19T18:16:42.357953+0000","last_peered":"2023-12-19T18:16:42.357953+0000","last_clean":"2023-12-19T18:16:42.357953+0000","last_became_active":"2023-12-19T18:16:40.569050+0000","last_became_peered":"2023-12-19T18:16:40.569050+0000","last_unstale":"2023-12-19T18:16:42.357953+0000","last_undegraded":"2023-12-19T18:16:42.357953+0000","last_fullsized":"2023-12-19T18:16:42.357953+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:28:56.831758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561350+0000","last_change":"2023-12-19T18:16:27.829736+0000","last_active":"2023-12-19T18:16:40.561350+0000","last_peered":"2023-12-19T18:16:40.561350+0000","last_clean":"2023-12-19T18:16:40.561350+0000","last_became_active":"2023-12-19T18:16:27.829448+0000","last_became_peered":"2023-12-19T18:16:27.829448+0000","last_unstale":"2023-12-19T18:16:40.561350+0000","last_undegraded":"2023-12-19T18:16:40.561350+0000","last_fullsized":"2023-12-19T18:16:40.561350+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:47.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551154+0000","last_change":"2023-12-19T18:16:40.559877+0000","last_active":"2023-12-19T18:16:41.551154+0000","last_peered":"2023-12-19T18:16:41.551154+0000","last_clean":"2023-12-19T18:16:41.551154+0000","last_became_active":"2023-12-19T18:16:40.558100+0000","last_became_peered":"2023-12-19T18:16:40.558100+0000","last_unstale":"2023-12-19T18:16:41.551154+0000","last_undegraded":"2023-12-19T18:16:41.551154+0000","last_fullsized":"2023-12-19T18:16:41.551154+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:44.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559776+0000","last_change":"2023-12-19T18:16:27.829943+0000","last_active":"2023-12-19T18:16:40.559776+0000","last_peered":"2023-12-19T18:16:40.559776+0000","last_clean":"2023-12-19T18:16:40.559776+0000","last_became_active":"2023-12-19T18:16:27.829592+0000","last_became_peered":"2023-12-19T18:16:27.829592+0000","last_unstale":"2023-12-19T18:16:40.559776+0000","last_undegraded":"2023-12-19T18:16:40.559776+0000","last_fullsized":"2023-12-19T18:16:40.559776+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:46.245564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553575+0000","last_change":"2023-12-19T18:16:40.575384+0000","last_active":"2023-12-19T18:16:41.553575+0000","last_peered":"2023-12-19T18:16:41.553575+0000","last_clean":"2023-12-19T18:16:41.553575+0000","last_became_active":"2023-12-19T18:16:40.575124+0000","last_became_peered":"2023-12-19T18:16:40.575124+0000","last_unstale":"2023-12-19T18:16:41.553575+0000","last_undegraded":"2023-12-19T18:16:41.553575+0000","last_fullsized":"2023-12-19T18:16:41.553575+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:53.796369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565426+0000","last_change":"2023-12-19T18:16:27.819797+0000","last_active":"2023-12-19T18:16:40.565426+0000","last_peered":"2023-12-19T18:16:40.565426+0000","last_clean":"2023-12-19T18:16:40.565426+0000","last_became_active":"2023-12-19T18:16:27.819507+0000","last_became_peered":"2023-12-19T18:16:27.819507+0000","last_unstale":"2023-12-19T18:16:40.565426+0000","last_undegraded":"2023-12-19T18:16:40.565426+0000","last_fullsized":"2023-12-19T18:16:40.565426+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:08.287716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857264+0000","last_change":"2023-12-19T18:16:40.570173+0000","last_active":"2023-12-19T18:16:41.857264+0000","last_peered":"2023-12-19T18:16:41.857264+0000","last_clean":"2023-12-19T18:16:41.857264+0000","last_became_active":"2023-12-19T18:16:40.569912+0000","last_became_peered":"2023-12-19T18:16:40.569912+0000","last_unstale":"2023-12-19T18:16:41.857264+0000","last_undegraded":"2023-12-19T18:16:41.857264+0000","last_fullsized":"2023-12-19T18:16:41.857264+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:47.688949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558633+0000","last_change":"2023-12-19T18:16:27.819945+0000","last_active":"2023-12-19T18:16:40.558633+0000","last_peered":"2023-12-19T18:16:40.558633+0000","last_clean":"2023-12-19T18:16:40.558633+0000","last_became_active":"2023-12-19T18:16:27.819634+0000","last_became_peered":"2023-12-19T18:16:27.819634+0000","last_unstale":"2023-12-19T18:16:40.558633+0000","last_undegraded":"2023-12-19T18:16:40.558633+0000","last_fullsized":"2023-12-19T18:16:40.558633+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:22.401852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358420+0000","last_change":"2023-12-19T18:16:40.566307+0000","last_active":"2023-12-19T18:16:42.358420+0000","last_peered":"2023-12-19T18:16:42.358420+0000","last_clean":"2023-12-19T18:16:42.358420+0000","last_became_active":"2023-12-19T18:16:40.566059+0000","last_became_peered":"2023-12-19T18:16:40.566059+0000","last_unstale":"2023-12-19T18:16:42.358420+0000","last_undegraded":"2023-12-19T18:16:42.358420+0000","last_fullsized":"2023-12-19T18:16:42.358420+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:45.773786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549949+0000","last_change":"2023-12-19T18:16:27.832580+0000","last_active":"2023-12-19T18:16:40.549949+0000","last_peered":"2023-12-19T18:16:40.549949+0000","last_clean":"2023-12-19T18:16:40.549949+0000","last_became_active":"2023-12-19T18:16:27.832291+0000","last_became_peered":"2023-12-19T18:16:27.832291+0000","last_unstale":"2023-12-19T18:16:40.549949+0000","last_undegraded":"2023-12-19T18:16:40.549949+0000","last_fullsized":"2023-12-19T18:16:40.549949+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:29.305405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359160+0000","last_change":"2023-12-19T18:16:40.567943+0000","last_active":"2023-12-19T18:16:42.359160+0000","last_peered":"2023-12-19T18:16:42.359160+0000","last_clean":"2023-12-19T18:16:42.359160+0000","last_became_active":"2023-12-19T18:16:40.567701+0000","last_became_peered":"2023-12-19T18:16:40.567701+0000","last_unstale":"2023-12-19T18:16:42.359160+0000","last_undegraded":"2023-12-19T18:16:42.359160+0000","last_fullsized":"2023-12-19T18:16:42.359160+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:50:31.894613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543104+0000","last_change":"2023-12-19T18:16:27.825730+0000","last_active":"2023-12-19T18:16:40.543104+0000","last_peered":"2023-12-19T18:16:40.543104+0000","last_clean":"2023-12-19T18:16:40.543104+0000","last_became_active":"2023-12-19T18:16:27.825136+0000","last_became_peered":"2023-12-19T18:16:27.825136+0000","last_unstale":"2023-12-19T18:16:40.543104+0000","last_undegraded":"2023-12-19T18:16:40.543104+0000","last_fullsized":"2023-12-19T18:16:40.543104+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:48.330735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359437+0000","last_change":"2023-12-19T18:16:40.558188+0000","last_active":"2023-12-19T18:16:42.359437+0000","last_peered":"2023-12-19T18:16:42.359437+0000","last_clean":"2023-12-19T18:16:42.359437+0000","last_became_active":"2023-12-19T18:16:40.557804+0000","last_became_peered":"2023-12-19T18:16:40.557804+0000","last_unstale":"2023-12-19T18:16:42.359437+0000","last_undegraded":"2023-12-19T18:16:42.359437+0000","last_fullsized":"2023-12-19T18:16:42.359437+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:35:20.500418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558018+0000","last_change":"2023-12-19T18:16:27.823852+0000","last_active":"2023-12-19T18:16:40.558018+0000","last_peered":"2023-12-19T18:16:40.558018+0000","last_clean":"2023-12-19T18:16:40.558018+0000","last_became_active":"2023-12-19T18:16:27.823564+0000","last_became_peered":"2023-12-19T18:16:27.823564+0000","last_unstale":"2023-12-19T18:16:40.558018+0000","last_undegraded":"2023-12-19T18:16:40.558018+0000","last_fullsized":"2023-12-19T18:16:40.558018+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:59.000440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358756+0000","last_change":"2023-12-19T18:16:40.561015+0000","last_active":"2023-12-19T18:16:42.358756+0000","last_peered":"2023-12-19T18:16:42.358756+0000","last_clean":"2023-12-19T18:16:42.358756+0000","last_became_active":"2023-12-19T18:16:40.560771+0000","last_became_peered":"2023-12-19T18:16:40.560771+0000","last_unstale":"2023-12-19T18:16:42.358756+0000","last_undegraded":"2023-12-19T18:16:42.358756+0000","last_fullsized":"2023-12-19T18:16:42.358756+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:04.628464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554625+0000","last_change":"2023-12-19T18:16:27.822167+0000","last_active":"2023-12-19T18:16:40.554625+0000","last_peered":"2023-12-19T18:16:40.554625+0000","last_clean":"2023-12-19T18:16:40.554625+0000","last_became_active":"2023-12-19T18:16:27.821752+0000","last_became_peered":"2023-12-19T18:16:27.821752+0000","last_unstale":"2023-12-19T18:16:40.554625+0000","last_undegraded":"2023-12-19T18:16:40.554625+0000","last_fullsized":"2023-12-19T18:16:40.554625+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:17.866181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357777+0000","last_change":"2023-12-19T18:16:40.565973+0000","last_active":"2023-12-19T18:16:42.357777+0000","last_peered":"2023-12-19T18:16:42.357777+0000","last_clean":"2023-12-19T18:16:42.357777+0000","last_became_active":"2023-12-19T18:16:40.565418+0000","last_became_peered":"2023-12-19T18:16:40.565418+0000","last_unstale":"2023-12-19T18:16:42.357777+0000","last_undegraded":"2023-12-19T18:16:42.357777+0000","last_fullsized":"2023-12-19T18:16:42.357777+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:54.473566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551716+0000","last_change":"2023-12-19T18:16:27.835673+0000","last_active":"2023-12-19T18:16:40.551716+0000","last_peered":"2023-12-19T18:16:40.551716+0000","last_clean":"2023-12-19T18:16:40.551716+0000","last_became_active":"2023-12-19T18:16:27.835392+0000","last_became_peered":"2023-12-19T18:16:27.835392+0000","last_unstale":"2023-12-19T18:16:40.551716+0000","last_undegraded":"2023-12-19T18:16:40.551716+0000","last_fullsized":"2023-12-19T18:16:40.551716+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:23:06.990327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553304+0000","last_change":"2023-12-19T18:16:40.564946+0000","last_active":"2023-12-19T18:16:41.553304+0000","last_peered":"2023-12-19T18:16:41.553304+0000","last_clean":"2023-12-19T18:16:41.553304+0000","last_became_active":"2023-12-19T18:16:40.564679+0000","last_became_peered":"2023-12-19T18:16:40.564679+0000","last_unstale":"2023-12-19T18:16:41.553304+0000","last_undegraded":"2023-12-19T18:16:41.553304+0000","last_fullsized":"2023-12-19T18:16:41.553304+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:02:28.048063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559694+0000","last_change":"2023-12-19T18:16:27.830449+0000","last_active":"2023-12-19T18:16:40.559694+0000","last_peered":"2023-12-19T18:16:40.559694+0000","last_clean":"2023-12-19T18:16:40.559694+0000","last_became_active":"2023-12-19T18:16:27.830189+0000","last_became_peered":"2023-12-19T18:16:27.830189+0000","last_unstale":"2023-12-19T18:16:40.559694+0000","last_undegraded":"2023-12-19T18:16:40.559694+0000","last_fullsized":"2023-12-19T18:16:40.559694+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:06.397912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360333+0000","last_change":"2023-12-19T18:16:40.576744+0000","last_active":"2023-12-19T18:16:42.360333+0000","last_peered":"2023-12-19T18:16:42.360333+0000","last_clean":"2023-12-19T18:16:42.360333+0000","last_became_active":"2023-12-19T18:16:40.576425+0000","last_became_peered":"2023-12-19T18:16:40.576425+0000","last_unstale":"2023-12-19T18:16:42.360333+0000","last_undegraded":"2023-12-19T18:16:42.360333+0000","last_fullsized":"2023-12-19T18:16:42.360333+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:26.259551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548052+0000","last_change":"2023-12-19T18:16:27.815730+0000","last_active":"2023-12-19T18:16:40.548052+0000","last_peered":"2023-12-19T18:16:40.548052+0000","last_clean":"2023-12-19T18:16:40.548052+0000","last_became_active":"2023-12-19T18:16:27.815341+0000","last_became_peered":"2023-12-19T18:16:27.815341+0000","last_unstale":"2023-12-19T18:16:40.548052+0000","last_undegraded":"2023-12-19T18:16:40.548052+0000","last_fullsized":"2023-12-19T18:16:40.548052+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:39.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:16:40.124680+0000","last_change":"2023-12-19T18:16:40.124680+0000","last_active":"2023-12-19T18:16:40.124680+0000","last_peered":"2023-12-19T18:16:40.124680+0000","last_clean":"2023-12-19T18:16:40.124680+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:16:40.124680+0000","last_undegraded":"2023-12-19T18:16:40.124680+0000","last_fullsized":"2023-12-19T18:16:40.124680+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562588+0000","last_change":"2023-12-19T18:16:27.831511+0000","last_active":"2023-12-19T18:16:40.562588+0000","last_peered":"2023-12-19T18:16:40.562588+0000","last_clean":"2023-12-19T18:16:40.562588+0000","last_became_active":"2023-12-19T18:16:27.831255+0000","last_became_peered":"2023-12-19T18:16:27.831255+0000","last_unstale":"2023-12-19T18:16:40.562588+0000","last_undegraded":"2023-12-19T18:16:40.562588+0000","last_fullsized":"2023-12-19T18:16:40.562588+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:42.382254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358249+0000","last_change":"2023-12-19T18:16:40.576919+0000","last_active":"2023-12-19T18:16:42.358249+0000","last_peered":"2023-12-19T18:16:42.358249+0000","last_clean":"2023-12-19T18:16:42.358249+0000","last_became_active":"2023-12-19T18:16:40.576351+0000","last_became_peered":"2023-12-19T18:16:40.576351+0000","last_unstale":"2023-12-19T18:16:42.358249+0000","last_undegraded":"2023-12-19T18:16:42.358249+0000","last_fullsized":"2023-12-19T18:16:42.358249+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:36.351083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-12-19T18:16:28.305437+0000","last_change":"2023-12-19T18:16:27.834221+0000","last_active":"2023-12-19T18:16:28.305437+0000","last_peered":"2023-12-19T18:16:28.305437+0000","last_clean":"2023-12-19T18:16:28.305437+0000","last_became_active":"2023-12-19T18:16:27.825886+0000","last_became_peered":"2023-12-19T18:16:27.825886+0000","last_unstale":"2023-12-19T18:16:28.305437+0000","last_undegraded":"2023-12-19T18:16:28.305437+0000","last_fullsized":"2023-12-19T18:16:28.305437+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:12.119996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":56,"acting":56,"num_store_stats":7},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":114,"acting":114,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510568,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64556,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307284,"statfs":{"total":96636764160,"available":96570658816,"internally_reserved":0,"allocated":3248128,"data_stored":1989677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96399999999999997,"5min":1.048,"15min":1.1180000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92400000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.70899999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92400000000000004,"5min":1.0389999999999999,"15min":1.0580000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72399999999999998,"15min":0.748},"min":{"1min":0.499,"5min":0.48099999999999998,"15min":0.435},"max":{"1min":0.89500000000000002,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.89000000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.38900000000000001},"max":{"1min":0.93400000000000005,"5min":1.026,"15min":1.0669999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.0820000000000001,"15min":1.236},"last":0.83499999999999996}]},{"osd":3,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.75600000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88300000000000001,"5min":1.002,"15min":1.079},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98899999999999999,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.65900000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81599999999999995,"5min":1.0009999999999999,"15min":1.302},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.871,"15min":1.0209999999999999},"last":0.92200000000000004}]},{"osd":5,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.52700000000000002,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.88100000000000001,"5min":1.097,"15min":1.1299999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64900000000000002,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":1.026,"15min":1.0800000000000001},"last":0.93100000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.56999999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.93999999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85699999999999998,"5min":1.01,"15min":1.093},"last":0.875}]}]},{"osd":6,"up_from":15,"seq":64424510569,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47636,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324204,"statfs":{"total":96636764160,"available":96587984896,"internally_reserved":0,"allocated":2699264,"data_stored":1431450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72799999999999998,"15min":0.74199999999999999},"min":{"1min":0.55400000000000005,"5min":0.46800000000000003,"15min":0.43099999999999999},"max":{"1min":0.92400000000000004,"5min":0.96499999999999997,"15min":1.042},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71799999999999997,"15min":0.73499999999999999},"min":{"1min":0.57499999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92000000000000004,"5min":1.224,"15min":1.224},"last":0.69299999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70099999999999996,"15min":0.72799999999999998},"min":{"1min":0.49099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98099999999999998,"5min":1.103,"15min":1.103},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.64100000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":0.94599999999999995,"5min":0.99199999999999999,"15min":1.1259999999999999},"last":0.76000000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.502},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0609999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.749},"min":{"1min":0.60899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83299999999999996,"5min":1.0089999999999999,"15min":1.111},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.755,"15min":0.78200000000000003},"min":{"1min":0.61799999999999999,"5min":0.495,"15min":0.495},"max":{"1min":0.91200000000000003,"5min":0.97999999999999998,"15min":1.1519999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.51800000000000002,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.98699999999999999,"5min":1.268,"15min":1.268},"last":0.72299999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92200000000000004,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":0.81299999999999994,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78600000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.93799999999999994,"5min":1.194,"15min":1.194},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":1.141,"15min":1.141},"last":0.95199999999999996}]},{"osd":7,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.95899999999999996,"5min":1.159,"15min":1.159},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94199999999999995,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.99199999999999999}]}]},{"osd":5,"up_from":15,"seq":64424510569,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47636,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324204,"statfs":{"total":96636764160,"available":96587984896,"internally_reserved":0,"allocated":2699264,"data_stored":1431450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72499999999999998,"15min":0.72999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34399999999999997},"max":{"1min":0.96899999999999997,"5min":1.0109999999999999,"15min":1.0149999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.40699999999999997},"max":{"1min":0.89200000000000002,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"min":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.497},"max":{"1min":0.95599999999999996,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.505,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78800000000000003,"15min":0.78100000000000003},"min":{"1min":0.61699999999999999,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":1.022,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94499999999999995,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.48499999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94999999999999996,"5min":1.05,"15min":1.05},"last":0.753},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.77400000000000002},"min":{"1min":0.56200000000000006,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98799999999999999,"5min":1.022,"15min":1.0329999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.79000000000000004,"5min":0.94499999999999995,"15min":1.0800000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41599999999999998},"max":{"1min":0.71099999999999997,"5min":1.208,"15min":1.208},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64800000000000002,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.94899999999999995,"15min":0.98999999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.65500000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.83499999999999996,"5min":1.008,"15min":1.0449999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.46500000000000002,"5min":0.439,"15min":0.436},"max":{"1min":0.77300000000000002,"5min":1.169,"15min":1.169},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71499999999999997,"15min":0.70999999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85699999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74199999999999999}]}]},{"osd":4,"up_from":15,"seq":64424510569,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47668,"kb_used_data":2668,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324172,"statfs":{"total":96636764160,"available":96587952128,"internally_reserved":0,"allocated":2732032,"data_stored":1435923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.51600000000000001,"5min":0.45800000000000002,"15min":0.45000000000000001},"max":{"1min":1.016,"5min":1.036,"15min":1.109},"last":0.62},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.502,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.91800000000000004,"5min":0.997,"15min":1.01},"last":0.72999999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.59199999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94699999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.997,"15min":1.1020000000000001},"last":0.68200000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79100000000000004,"15min":0.77700000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.47699999999999998},"max":{"1min":0.998,"5min":1.0289999999999999,"15min":1.071},"last":0.503},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.71699999999999997}]},{"osd":3,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76700000000000002,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93899999999999995,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.55500000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.66200000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65300000000000002,"15min":0.63100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.75},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67200000000000004,"15min":0.64600000000000002},"min":{"1min":0.495,"5min":0.42899999999999999,"15min":0.39600000000000002},"max":{"1min":0.88900000000000001,"5min":1.042,"15min":1.042},"last":0.52600000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.66100000000000003},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":1.024,"15min":1.0669999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0760000000000001},"last":0.78300000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.41899999999999998},"max":{"1min":0.92400000000000004,"5min":1.087,"15min":1.087},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77600000000000002,"15min":0.76200000000000001},"min":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.46999999999999997},"max":{"1min":0.98599999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70099999999999996}]}]},{"osd":3,"up_from":15,"seq":64424510569,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47656,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324184,"statfs":{"total":96636764160,"available":96587964416,"internally_reserved":0,"allocated":2719744,"data_stored":1431244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.63200000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74099999999999999,"5min":0.78300000000000003,"15min":0.80000000000000004},"last":0.504}]},{"osd":1,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.498},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71499999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.44}]},{"osd":2,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.77000000000000002,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.46000000000000002}]},{"osd":4,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81599999999999995,"15min":0.82799999999999996},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.012,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.81999999999999995,"15min":0.82299999999999995},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.56599999999999995},"max":{"1min":0.86499999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.65800000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82799999999999996,"15min":0.84199999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99099999999999999,"5min":1.1739999999999999,"15min":1.397},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.59399999999999997,"5min":0.55900000000000005,"15min":0.55100000000000005},"max":{"1min":0.91000000000000003,"5min":1.2230000000000001,"15min":1.262},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.83899999999999997,"15min":0.84799999999999998},"min":{"1min":0.60199999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.96099999999999997,"5min":1.234,"15min":1.274},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83799999999999997,"15min":0.84499999999999997},"min":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94599999999999995,"5min":1.165,"15min":1.302},"last":0.64500000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0640000000000001,"5min":1.3540000000000001,"15min":1.486},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89500000000000002},"min":{"1min":0.68899999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.93000000000000005,"5min":1.421,"15min":1.421},"last":0.626}]}]},{"osd":2,"up_from":15,"seq":64424510568,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47632,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324208,"statfs":{"total":96636764160,"available":96587988992,"internally_reserved":0,"allocated":2695168,"data_stored":1423635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.29299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.41499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.56000000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.86199999999999999,"15min":0.95999999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.36899999999999999},"max":{"1min":0.67200000000000004,"5min":0.79700000000000004,"15min":0.875},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.624,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.048,"5min":1.306,"15min":1.306},"last":0.753},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.05,"5min":1.272,"15min":1.466},"last":0.80700000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.64300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.97699999999999998,"5min":1.1459999999999999,"15min":1.3049999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.81999999999999995,"15min":0.82599999999999996},"min":{"1min":0.61899999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.429},"last":0.78400000000000003}]},{"osd":6,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.71199999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.216,"5min":1.365,"15min":1.4119999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80600000000000005,"15min":0.80800000000000005},"min":{"1min":0.625,"5min":0.55200000000000005,"15min":0.53500000000000003},"max":{"1min":0.94499999999999995,"5min":1.2,"15min":1.2470000000000001},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81499999999999995,"15min":0.82799999999999996},"min":{"1min":0.64400000000000002,"5min":0.59899999999999998,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.393,"15min":1.45},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85999999999999999,"15min":0.86099999999999999},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1850000000000001,"5min":1.276,"15min":1.276},"last":0.88600000000000001}]}]},{"osd":1,"up_from":15,"seq":64424510569,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59940,"kb_used_data":2652,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311900,"statfs":{"total":96636764160,"available":96575385600,"internally_reserved":0,"allocated":2715648,"data_stored":1433729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.33900000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63700000000000001,"5min":0.71999999999999997,"15min":0.89400000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.53300000000000003},"min":{"1min":0.32200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.32600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.505},{"interface":"front","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.45000000000000001,"15min":0.442},"max":{"1min":0.82699999999999996,"5min":0.94199999999999995,"15min":0.94499999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.65100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93400000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79100000000000004,"15min":0.81499999999999995},"min":{"1min":0.502,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0409999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.86599999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.78400000000000003,"15min":0.80500000000000005},"min":{"1min":0.49099999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97499999999999998,"5min":1.236,"15min":1.2609999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98399999999999999,"5min":1.353,"15min":1.353},"last":1.05}]},{"osd":6,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.60999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.008,"5min":1.2270000000000001,"15min":1.2689999999999999},"last":0.878},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.84299999999999997,"15min":0.84999999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.093,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":1.034}]},{"osd":7,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.64100000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.123,"5min":1.399,"15min":1.399},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.84299999999999997,"15min":0.85799999999999998},"min":{"1min":0.54100000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.105,"5min":1.365,"15min":1.365},"last":0.90100000000000002}]}]},{"osd":0,"up_from":14,"seq":60129543274,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60508,"kb_used_data":3220,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311332,"statfs":{"total":96636764160,"available":96574803968,"internally_reserved":0,"allocated":3297280,"data_stored":2021348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.438,"15min":0.45700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":1.216},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.317}]},{"osd":2,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64700000000000002,"5min":0.84999999999999998,"15min":1.1699999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72299999999999998,"5min":0.78900000000000003,"15min":0.96799999999999997},"last":0.442}]},{"osd":3,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.623,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.82699999999999996,"5min":0.871,"15min":1.032},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.63,"5min":0.51800000000000002,"15min":0.496},"max":{"1min":0.93100000000000005,"5min":1.1200000000000001,"15min":1.381},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.59199999999999997,"5min":0.56799999999999995,"15min":0.47799999999999998},"max":{"1min":1.02,"5min":1.137,"15min":1.2849999999999999},"last":0.752}]},{"osd":5,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80600000000000005,"15min":0.81899999999999995},"min":{"1min":0.57299999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.008,"5min":1.1930000000000001,"15min":1.3420000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.69599999999999995,"5min":0.59599999999999997,"15min":0.59499999999999997},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.288},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.51300000000000001},"max":{"1min":0.98899999999999999,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.83399999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0229999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.84999999999999998},"min":{"1min":0.65300000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.044,"5min":1.1899999999999999,"15min":1.415},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.83199999999999996,"15min":0.85599999999999998},"min":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0780000000000001,"5min":1.1379999999999999,"15min":1.395},"last":0.85599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:16:45.080 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:16:45.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.258+0000 7f32e29a2640 1 Processor -- start 2023-12-19T18:16:45.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.258+0000 7f32e29a2640 1 -- start start 2023-12-19T18:16:45.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.258+0000 7f32e29a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32dc0ada30 0x7f32dc0c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.258+0000 7f32e29a2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f32dc0cb570 0x7f32dc0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.258+0000 7f32e29a2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32dc0ade10 con 0x7f32dc164f00 2023-12-19T18:16:45.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.258+0000 7f32e29a2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32dc0b16e0 con 0x7f32dc0cb570 2023-12-19T18:16:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.258+0000 7f32e29a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc0cc9d0 con 0x7f32dc0ada30 2023-12-19T18:16:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e21a1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f32dc0cb570 0x7f32dc0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e21a1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f32dc0cb570 0x7f32dc0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39582/0 (socket says 172.21.15.26:39582) 2023-12-19T18:16:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e21a1640 1 -- 172.21.15.26:0/226490256 learned_addr learned my addr 172.21.15.26:0/226490256 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e119f640 1 --1- 172.21.15.26:0/226490256 >> v1:172.21.15.26:6789/0 conn(0x7f32dc164f00 0x7f32dc0cae60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41414/0 (socket says 172.21.15.26:41414) 2023-12-19T18:16:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e19a0640 1 --2- 172.21.15.26:0/226490256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32dc0ada30 0x7f32dc0c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e21a1640 1 -- 172.21.15.26:0/226490256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32dc0ada30 msgr2=0x7f32dc0c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e21a1640 1 --2- 172.21.15.26:0/226490256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32dc0ada30 0x7f32dc0c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e21a1640 1 -- 172.21.15.26:0/226490256 >> v1:172.21.15.26:6789/0 conn(0x7f32dc164f00 legacy=0x7f32dc0cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e21a1640 1 -- 172.21.15.26:0/226490256 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc161830 con 0x7f32dc0cb570 2023-12-19T18:16:45.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e099e640 1 -- 172.21.15.26:0/226490256 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393860772 0 0) 0x7f32dc0ade10 con 0x7f32dc164f00 2023-12-19T18:16:45.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e21a1640 1 --2- 172.21.15.26:0/226490256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0cb570 0x7f32dc0cce60 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f32d4009e50 tx=0x7f32d401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b063c8300ccd76 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:45.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e099e640 1 -- 172.21.15.26:0/226490256 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d40a57d0 con 0x7f32dc0cb570 2023-12-19T18:16:45.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e099e640 1 -- 172.21.15.26:0/226490256 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d40a5930 con 0x7f32dc0cb570 2023-12-19T18:16:45.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.262+0000 7f32e099e640 1 -- 172.21.15.26:0/226490256 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d40a5cc0 con 0x7f32dc0cb570 2023-12-19T18:16:45.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 -- 172.21.15.26:0/226490256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0cb570 msgr2=0x7f32dc0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 --2- 172.21.15.26:0/226490256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0cb570 0x7f32dc0cce60 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f32d4009e50 tx=0x7f32d401ca10 comp rx=0 tx=0).stop 2023-12-19T18:16:45.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 -- 172.21.15.26:0/226490256 shutdown_connections 2023-12-19T18:16:45.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 --2- 172.21.15.26:0/226490256 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0cb570 0x7f32dc0cce60 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 --2- 172.21.15.26:0/226490256 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32dc0ada30 0x7f32dc0c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 -- 172.21.15.26:0/226490256 >> 172.21.15.26:0/226490256 conn(0x7f32dc06b0f0 msgr2=0x7f32dc06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:45.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 -- 172.21.15.26:0/226490256 shutdown_connections 2023-12-19T18:16:45.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 -- 172.21.15.26:0/226490256 wait complete. 2023-12-19T18:16:45.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 Processor -- start 2023-12-19T18:16:45.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 -- start start 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0ada30 0x7f32dc158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32dc0cb570 0x7f32dc15c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32dc164f00 0x7f32dc0c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc15df90 con 0x7f32dc0ada30 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc158b60 con 0x7f32dc0cb570 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e29a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32dc15d310 con 0x7f32dc164f00 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e19a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0ada30 0x7f32dc158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e19a0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0ada30 0x7f32dc158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39590/0 (socket says 172.21.15.26:39590) 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e19a0640 1 -- 172.21.15.26:0/221857047 learned_addr learned my addr 172.21.15.26:0/221857047 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e21a1640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32dc164f00 0x7f32dc0c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e119f640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32dc0cb570 0x7f32dc15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e19a0640 1 -- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32dc164f00 msgr2=0x7f32dc0c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e19a0640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32dc164f00 0x7f32dc0c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e19a0640 1 -- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32dc0cb570 msgr2=0x7f32dc15c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e19a0640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32dc0cb570 0x7f32dc15c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e19a0640 1 -- 172.21.15.26:0/221857047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d4009b30 con 0x7f32dc0ada30 2023-12-19T18:16:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.266+0000 7f32e119f640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32dc0cb570 0x7f32dc15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.270+0000 7f32e19a0640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0ada30 0x7f32dc158620 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f32d800e960 tx=0x7f32d800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:45.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.270+0000 7f32d27fc640 1 -- 172.21.15.26:0/221857047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d8009690 con 0x7f32dc0ada30 2023-12-19T18:16:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.270+0000 7f32d27fc640 1 -- 172.21.15.26:0/221857047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d800ca00 con 0x7f32dc0ada30 2023-12-19T18:16:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.270+0000 7f32d27fc640 1 -- 172.21.15.26:0/221857047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d800cd70 con 0x7f32dc0ada30 2023-12-19T18:16:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.270+0000 7f32e29a2640 1 -- 172.21.15.26:0/221857047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32dc15d5f0 con 0x7f32dc0ada30 2023-12-19T18:16:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.270+0000 7f32e29a2640 1 -- 172.21.15.26:0/221857047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32dc15dbb0 con 0x7f32dc0ada30 2023-12-19T18:16:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.270+0000 7f32e29a2640 1 -- 172.21.15.26:0/221857047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32dc15e200 con 0x7f32dc0ada30 2023-12-19T18:16:45.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.270+0000 7f32d27fc640 1 -- 172.21.15.26:0/221857047 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f32d8013470 con 0x7f32dc0ada30 2023-12-19T18:16:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.270+0000 7f32d27fc640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 0x7f32c0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.270+0000 7f32d27fc640 1 -- 172.21.15.26:0/221857047 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f32d8083b90 con 0x7f32dc0ada30 2023-12-19T18:16:45.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.270+0000 7f32e119f640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 0x7f32c0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.274+0000 7f32e119f640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 0x7f32c0068040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0c5310 tx=0x7f32cc007cd0 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:45.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.274+0000 7f32d27fc640 1 -- 172.21.15.26:0/221857047 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f32d8050e90 con 0x7f32dc0ada30 2023-12-19T18:16:45.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.430+0000 7f32e29a2640 1 -- 172.21.15.26:0/221857047 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32dc1659a0 con 0x7f32c0065b80 2023-12-19T18:16:45.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.442+0000 7f32d27fc640 1 -- 172.21.15.26:0/221857047 <== mgr.9805 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528500 (secure 0 0 0) 0x7f32cc0073a0 con 0x7f32c0065b80 2023-12-19T18:16:45.451 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:45.455 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:16:45.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.446+0000 7f32e29a2640 1 -- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 msgr2=0x7f32c0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.446+0000 7f32e29a2640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 0x7f32c0068040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0c5310 tx=0x7f32cc007cd0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.446+0000 7f32e29a2640 1 -- 172.21.15.26:0/221857047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0ada30 msgr2=0x7f32dc158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.446+0000 7f32e29a2640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0ada30 0x7f32dc158620 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f32d800e960 tx=0x7f32d800ee20 comp rx=0 tx=0).stop 2023-12-19T18:16:45.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.446+0000 7f32e29a2640 1 -- 172.21.15.26:0/221857047 shutdown_connections 2023-12-19T18:16:45.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.446+0000 7f32e29a2640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 0x7f32c0068040 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.446+0000 7f32e29a2640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32dc164f00 0x7f32dc0c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.446+0000 7f32e29a2640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32dc0cb570 0x7f32dc15c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.446+0000 7f32e29a2640 1 --2- 172.21.15.26:0/221857047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32dc0ada30 0x7f32dc158620 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.446+0000 7f32e29a2640 1 -- 172.21.15.26:0/221857047 >> 172.21.15.26:0/221857047 conn(0x7f32dc06b0f0 msgr2=0x7f32dc0ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:45.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.446+0000 7f32e29a2640 1 -- 172.21.15.26:0/221857047 shutdown_connections 2023-12-19T18:16:45.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.446+0000 7f32e29a2640 1 -- 172.21.15.26:0/221857047 wait complete. 2023-12-19T18:16:45.469 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-12-19T18:16:45.307976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596333,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":667,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":426,"ondisk_log_size":426,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":344,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423292,"kb_used_data":22332,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754551428,"statfs":{"total":773094113280,"available":772660662272,"internally_reserved":0,"allocated":22867968,"data_stored":12631683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001750"},"pg_stats":[{"pgid":"27.1a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357761+0000","last_change":"2023-12-19T18:16:40.564006+0000","last_active":"2023-12-19T18:16:42.357761+0000","last_peered":"2023-12-19T18:16:42.357761+0000","last_clean":"2023-12-19T18:16:42.357761+0000","last_became_active":"2023-12-19T18:16:40.563756+0000","last_became_peered":"2023-12-19T18:16:40.563756+0000","last_unstale":"2023-12-19T18:16:42.357761+0000","last_undegraded":"2023-12-19T18:16:42.357761+0000","last_fullsized":"2023-12-19T18:16:42.357761+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:17.484857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"200'346","reported_seq":1188,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560456+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:16:40.560456+0000","last_peered":"2023-12-19T18:16:40.560456+0000","last_clean":"2023-12-19T18:16:40.560456+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:16:40.560456+0000","last_undegraded":"2023-12-19T18:16:40.560456+0000","last_fullsized":"2023-12-19T18:16:40.560456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560313+0000","last_change":"2023-12-19T18:16:27.411230+0000","last_active":"2023-12-19T18:16:40.560313+0000","last_peered":"2023-12-19T18:16:40.560313+0000","last_clean":"2023-12-19T18:16:40.560313+0000","last_became_active":"2023-12-19T18:16:27.410670+0000","last_became_peered":"2023-12-19T18:16:27.410670+0000","last_unstale":"2023-12-19T18:16:40.560313+0000","last_undegraded":"2023-12-19T18:16:40.560313+0000","last_fullsized":"2023-12-19T18:16:40.560313+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:22.426446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554070+0000","last_change":"2023-12-19T18:16:27.821899+0000","last_active":"2023-12-19T18:16:40.554070+0000","last_peered":"2023-12-19T18:16:40.554070+0000","last_clean":"2023-12-19T18:16:40.554070+0000","last_became_active":"2023-12-19T18:16:27.821634+0000","last_became_peered":"2023-12-19T18:16:27.821634+0000","last_unstale":"2023-12-19T18:16:40.554070+0000","last_undegraded":"2023-12-19T18:16:40.554070+0000","last_fullsized":"2023-12-19T18:16:40.554070+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:34.391068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857383+0000","last_change":"2023-12-19T18:16:40.150476+0000","last_active":"2023-12-19T18:16:41.857383+0000","last_peered":"2023-12-19T18:16:41.857383+0000","last_clean":"2023-12-19T18:16:41.857383+0000","last_became_active":"2023-12-19T18:16:40.149843+0000","last_became_peered":"2023-12-19T18:16:40.149843+0000","last_unstale":"2023-12-19T18:16:41.857383+0000","last_undegraded":"2023-12-19T18:16:41.857383+0000","last_fullsized":"2023-12-19T18:16:41.857383+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:35.179628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857177+0000","last_change":"2023-12-19T18:16:40.569150+0000","last_active":"2023-12-19T18:16:41.857177+0000","last_peered":"2023-12-19T18:16:41.857177+0000","last_clean":"2023-12-19T18:16:41.857177+0000","last_became_active":"2023-12-19T18:16:40.568754+0000","last_became_peered":"2023-12-19T18:16:40.568754+0000","last_unstale":"2023-12-19T18:16:41.857177+0000","last_undegraded":"2023-12-19T18:16:41.857177+0000","last_fullsized":"2023-12-19T18:16:41.857177+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:32.689972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548913+0000","last_change":"2023-12-19T18:16:27.412171+0000","last_active":"2023-12-19T18:16:40.548913+0000","last_peered":"2023-12-19T18:16:40.548913+0000","last_clean":"2023-12-19T18:16:40.548913+0000","last_became_active":"2023-12-19T18:16:27.411872+0000","last_became_peered":"2023-12-19T18:16:27.411872+0000","last_unstale":"2023-12-19T18:16:40.548913+0000","last_undegraded":"2023-12-19T18:16:40.548913+0000","last_fullsized":"2023-12-19T18:16:40.548913+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:48.264905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548191+0000","last_change":"2023-12-19T18:16:27.832432+0000","last_active":"2023-12-19T18:16:40.548191+0000","last_peered":"2023-12-19T18:16:40.548191+0000","last_clean":"2023-12-19T18:16:40.548191+0000","last_became_active":"2023-12-19T18:16:27.832116+0000","last_became_peered":"2023-12-19T18:16:27.832116+0000","last_unstale":"2023-12-19T18:16:40.548191+0000","last_undegraded":"2023-12-19T18:16:40.548191+0000","last_fullsized":"2023-12-19T18:16:40.548191+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:35.165338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359499+0000","last_change":"2023-12-19T18:16:40.157009+0000","last_active":"2023-12-19T18:16:42.359499+0000","last_peered":"2023-12-19T18:16:42.359499+0000","last_clean":"2023-12-19T18:16:42.359499+0000","last_became_active":"2023-12-19T18:16:40.156734+0000","last_became_peered":"2023-12-19T18:16:40.156734+0000","last_unstale":"2023-12-19T18:16:42.359499+0000","last_undegraded":"2023-12-19T18:16:42.359499+0000","last_fullsized":"2023-12-19T18:16:42.359499+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:18.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357983+0000","last_change":"2023-12-19T18:16:40.560903+0000","last_active":"2023-12-19T18:16:42.357983+0000","last_peered":"2023-12-19T18:16:42.357983+0000","last_clean":"2023-12-19T18:16:42.357983+0000","last_became_active":"2023-12-19T18:16:40.560636+0000","last_became_peered":"2023-12-19T18:16:40.560636+0000","last_unstale":"2023-12-19T18:16:42.357983+0000","last_undegraded":"2023-12-19T18:16:42.357983+0000","last_fullsized":"2023-12-19T18:16:42.357983+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:25:38.247442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561435+0000","last_change":"2023-12-19T18:16:27.407456+0000","last_active":"2023-12-19T18:16:40.561435+0000","last_peered":"2023-12-19T18:16:40.561435+0000","last_clean":"2023-12-19T18:16:40.561435+0000","last_became_active":"2023-12-19T18:16:27.407208+0000","last_became_peered":"2023-12-19T18:16:27.407208+0000","last_unstale":"2023-12-19T18:16:40.561435+0000","last_undegraded":"2023-12-19T18:16:40.561435+0000","last_fullsized":"2023-12-19T18:16:40.561435+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:07.654229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560528+0000","last_change":"2023-12-19T18:16:27.824001+0000","last_active":"2023-12-19T18:16:40.560528+0000","last_peered":"2023-12-19T18:16:40.560528+0000","last_clean":"2023-12-19T18:16:40.560528+0000","last_became_active":"2023-12-19T18:16:27.823010+0000","last_became_peered":"2023-12-19T18:16:27.823010+0000","last_unstale":"2023-12-19T18:16:40.560528+0000","last_undegraded":"2023-12-19T18:16:40.560528+0000","last_fullsized":"2023-12-19T18:16:40.560528+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:30:56.775994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074880+0000","last_change":"2023-12-19T18:16:40.164940+0000","last_active":"2023-12-19T18:16:42.074880+0000","last_peered":"2023-12-19T18:16:42.074880+0000","last_clean":"2023-12-19T18:16:42.074880+0000","last_became_active":"2023-12-19T18:16:40.163710+0000","last_became_peered":"2023-12-19T18:16:40.163710+0000","last_unstale":"2023-12-19T18:16:42.074880+0000","last_undegraded":"2023-12-19T18:16:42.074880+0000","last_fullsized":"2023-12-19T18:16:42.074880+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:29.482411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552525+0000","last_change":"2023-12-19T18:16:40.570024+0000","last_active":"2023-12-19T18:16:41.552525+0000","last_peered":"2023-12-19T18:16:41.552525+0000","last_clean":"2023-12-19T18:16:41.552525+0000","last_became_active":"2023-12-19T18:16:40.569586+0000","last_became_peered":"2023-12-19T18:16:40.569586+0000","last_unstale":"2023-12-19T18:16:41.552525+0000","last_undegraded":"2023-12-19T18:16:41.552525+0000","last_fullsized":"2023-12-19T18:16:41.552525+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:03.932038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547644+0000","last_change":"2023-12-19T18:16:27.405381+0000","last_active":"2023-12-19T18:16:40.547644+0000","last_peered":"2023-12-19T18:16:40.547644+0000","last_clean":"2023-12-19T18:16:40.547644+0000","last_became_active":"2023-12-19T18:16:27.405113+0000","last_became_peered":"2023-12-19T18:16:27.405113+0000","last_unstale":"2023-12-19T18:16:40.547644+0000","last_undegraded":"2023-12-19T18:16:40.547644+0000","last_fullsized":"2023-12-19T18:16:40.547644+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:08.911302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560761+0000","last_change":"2023-12-19T18:16:27.822423+0000","last_active":"2023-12-19T18:16:40.560761+0000","last_peered":"2023-12-19T18:16:40.560761+0000","last_clean":"2023-12-19T18:16:40.560761+0000","last_became_active":"2023-12-19T18:16:27.822106+0000","last_became_peered":"2023-12-19T18:16:27.822106+0000","last_unstale":"2023-12-19T18:16:40.560761+0000","last_undegraded":"2023-12-19T18:16:40.560761+0000","last_fullsized":"2023-12-19T18:16:40.560761+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:41.021760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.856687+0000","last_change":"2023-12-19T18:16:40.154697+0000","last_active":"2023-12-19T18:16:41.856687+0000","last_peered":"2023-12-19T18:16:41.856687+0000","last_clean":"2023-12-19T18:16:41.856687+0000","last_became_active":"2023-12-19T18:16:40.154394+0000","last_became_peered":"2023-12-19T18:16:40.154394+0000","last_unstale":"2023-12-19T18:16:41.856687+0000","last_undegraded":"2023-12-19T18:16:41.856687+0000","last_fullsized":"2023-12-19T18:16:41.856687+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:52.755953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357512+0000","last_change":"2023-12-19T18:16:40.572030+0000","last_active":"2023-12-19T18:16:42.357512+0000","last_peered":"2023-12-19T18:16:42.357512+0000","last_clean":"2023-12-19T18:16:42.357512+0000","last_became_active":"2023-12-19T18:16:40.570330+0000","last_became_peered":"2023-12-19T18:16:40.570330+0000","last_unstale":"2023-12-19T18:16:42.357512+0000","last_undegraded":"2023-12-19T18:16:42.357512+0000","last_fullsized":"2023-12-19T18:16:42.357512+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:55.123677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547774+0000","last_change":"2023-12-19T18:16:27.410513+0000","last_active":"2023-12-19T18:16:40.547774+0000","last_peered":"2023-12-19T18:16:40.547774+0000","last_clean":"2023-12-19T18:16:40.547774+0000","last_became_active":"2023-12-19T18:16:27.410166+0000","last_became_peered":"2023-12-19T18:16:27.410166+0000","last_unstale":"2023-12-19T18:16:40.547774+0000","last_undegraded":"2023-12-19T18:16:40.547774+0000","last_fullsized":"2023-12-19T18:16:40.547774+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:10.682880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560587+0000","last_change":"2023-12-19T18:16:27.829925+0000","last_active":"2023-12-19T18:16:40.560587+0000","last_peered":"2023-12-19T18:16:40.560587+0000","last_clean":"2023-12-19T18:16:40.560587+0000","last_became_active":"2023-12-19T18:16:27.828958+0000","last_became_peered":"2023-12-19T18:16:27.828958+0000","last_unstale":"2023-12-19T18:16:40.560587+0000","last_undegraded":"2023-12-19T18:16:40.560587+0000","last_fullsized":"2023-12-19T18:16:40.560587+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:46.995476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554304+0000","last_change":"2023-12-19T18:16:40.156673+0000","last_active":"2023-12-19T18:16:41.554304+0000","last_peered":"2023-12-19T18:16:41.554304+0000","last_clean":"2023-12-19T18:16:41.554304+0000","last_became_active":"2023-12-19T18:16:40.156391+0000","last_became_peered":"2023-12-19T18:16:40.156391+0000","last_unstale":"2023-12-19T18:16:41.554304+0000","last_undegraded":"2023-12-19T18:16:41.554304+0000","last_fullsized":"2023-12-19T18:16:41.554304+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:46.681719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360169+0000","last_change":"2023-12-19T18:16:40.571314+0000","last_active":"2023-12-19T18:16:42.360169+0000","last_peered":"2023-12-19T18:16:42.360169+0000","last_clean":"2023-12-19T18:16:42.360169+0000","last_became_active":"2023-12-19T18:16:40.570538+0000","last_became_peered":"2023-12-19T18:16:40.570538+0000","last_unstale":"2023-12-19T18:16:42.360169+0000","last_undegraded":"2023-12-19T18:16:42.360169+0000","last_fullsized":"2023-12-19T18:16:42.360169+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:02:14.652060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543489+0000","last_change":"2023-12-19T18:16:27.409169+0000","last_active":"2023-12-19T18:16:40.543489+0000","last_peered":"2023-12-19T18:16:40.543489+0000","last_clean":"2023-12-19T18:16:40.543489+0000","last_became_active":"2023-12-19T18:16:27.408882+0000","last_became_peered":"2023-12-19T18:16:27.408882+0000","last_unstale":"2023-12-19T18:16:40.543489+0000","last_undegraded":"2023-12-19T18:16:40.543489+0000","last_fullsized":"2023-12-19T18:16:40.543489+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:01.478246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544313+0000","last_change":"2023-12-19T18:16:27.825200+0000","last_active":"2023-12-19T18:16:40.544313+0000","last_peered":"2023-12-19T18:16:40.544313+0000","last_clean":"2023-12-19T18:16:40.544313+0000","last_became_active":"2023-12-19T18:16:27.824919+0000","last_became_peered":"2023-12-19T18:16:27.824919+0000","last_unstale":"2023-12-19T18:16:40.544313+0000","last_undegraded":"2023-12-19T18:16:40.544313+0000","last_fullsized":"2023-12-19T18:16:40.544313+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.6","version":"216'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.077099+0000","last_change":"2023-12-19T18:16:40.164339+0000","last_active":"2023-12-19T18:16:42.077099+0000","last_peered":"2023-12-19T18:16:42.077099+0000","last_clean":"2023-12-19T18:16:42.077099+0000","last_became_active":"2023-12-19T18:16:40.163870+0000","last_became_peered":"2023-12-19T18:16:40.163870+0000","last_unstale":"2023-12-19T18:16:42.077099+0000","last_undegraded":"2023-12-19T18:16:42.077099+0000","last_fullsized":"2023-12-19T18:16:42.077099+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:14:22.971741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857196+0000","last_change":"2023-12-19T18:16:40.568314+0000","last_active":"2023-12-19T18:16:41.857196+0000","last_peered":"2023-12-19T18:16:41.857196+0000","last_clean":"2023-12-19T18:16:41.857196+0000","last_became_active":"2023-12-19T18:16:40.567857+0000","last_became_peered":"2023-12-19T18:16:40.567857+0000","last_unstale":"2023-12-19T18:16:41.857196+0000","last_undegraded":"2023-12-19T18:16:41.857196+0000","last_fullsized":"2023-12-19T18:16:41.857196+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:49.703164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555637+0000","last_change":"2023-12-19T18:16:27.420249+0000","last_active":"2023-12-19T18:16:40.555637+0000","last_peered":"2023-12-19T18:16:40.555637+0000","last_clean":"2023-12-19T18:16:40.555637+0000","last_became_active":"2023-12-19T18:16:27.419360+0000","last_became_peered":"2023-12-19T18:16:27.419360+0000","last_unstale":"2023-12-19T18:16:40.555637+0000","last_undegraded":"2023-12-19T18:16:40.555637+0000","last_fullsized":"2023-12-19T18:16:40.555637+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:07.969904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543383+0000","last_change":"2023-12-19T18:16:27.825535+0000","last_active":"2023-12-19T18:16:40.543383+0000","last_peered":"2023-12-19T18:16:40.543383+0000","last_clean":"2023-12-19T18:16:40.543383+0000","last_became_active":"2023-12-19T18:16:27.824814+0000","last_became_peered":"2023-12-19T18:16:27.824814+0000","last_unstale":"2023-12-19T18:16:40.543383+0000","last_undegraded":"2023-12-19T18:16:40.543383+0000","last_fullsized":"2023-12-19T18:16:40.543383+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:29.357409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358846+0000","last_change":"2023-12-19T18:16:40.160113+0000","last_active":"2023-12-19T18:16:42.358846+0000","last_peered":"2023-12-19T18:16:42.358846+0000","last_clean":"2023-12-19T18:16:42.358846+0000","last_became_active":"2023-12-19T18:16:40.159826+0000","last_became_peered":"2023-12-19T18:16:40.159826+0000","last_unstale":"2023-12-19T18:16:42.358846+0000","last_undegraded":"2023-12-19T18:16:42.358846+0000","last_fullsized":"2023-12-19T18:16:42.358846+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:45.058000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554995+0000","last_change":"2023-12-19T18:16:40.566980+0000","last_active":"2023-12-19T18:16:41.554995+0000","last_peered":"2023-12-19T18:16:41.554995+0000","last_clean":"2023-12-19T18:16:41.554995+0000","last_became_active":"2023-12-19T18:16:40.566716+0000","last_became_peered":"2023-12-19T18:16:40.566716+0000","last_unstale":"2023-12-19T18:16:41.554995+0000","last_undegraded":"2023-12-19T18:16:41.554995+0000","last_fullsized":"2023-12-19T18:16:41.554995+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:30.489642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.6","version":"212'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558804+0000","last_change":"2023-12-19T18:16:27.414152+0000","last_active":"2023-12-19T18:16:40.558804+0000","last_peered":"2023-12-19T18:16:40.558804+0000","last_clean":"2023-12-19T18:16:40.558804+0000","last_became_active":"2023-12-19T18:16:27.413877+0000","last_became_peered":"2023-12-19T18:16:27.413877+0000","last_unstale":"2023-12-19T18:16:40.558804+0000","last_undegraded":"2023-12-19T18:16:40.558804+0000","last_fullsized":"2023-12-19T18:16:40.558804+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:44:52.553685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558691+0000","last_change":"2023-12-19T18:16:27.824257+0000","last_active":"2023-12-19T18:16:40.558691+0000","last_peered":"2023-12-19T18:16:40.558691+0000","last_clean":"2023-12-19T18:16:40.558691+0000","last_became_active":"2023-12-19T18:16:27.823989+0000","last_became_peered":"2023-12-19T18:16:27.823989+0000","last_unstale":"2023-12-19T18:16:40.558691+0000","last_undegraded":"2023-12-19T18:16:40.558691+0000","last_fullsized":"2023-12-19T18:16:40.558691+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:07:07.505127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357977+0000","last_change":"2023-12-19T18:16:40.160316+0000","last_active":"2023-12-19T18:16:42.357977+0000","last_peered":"2023-12-19T18:16:42.357977+0000","last_clean":"2023-12-19T18:16:42.357977+0000","last_became_active":"2023-12-19T18:16:40.159312+0000","last_became_peered":"2023-12-19T18:16:40.159312+0000","last_unstale":"2023-12-19T18:16:42.357977+0000","last_undegraded":"2023-12-19T18:16:42.357977+0000","last_fullsized":"2023-12-19T18:16:42.357977+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:04.832569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553051+0000","last_change":"2023-12-19T18:16:40.563986+0000","last_active":"2023-12-19T18:16:41.553051+0000","last_peered":"2023-12-19T18:16:41.553051+0000","last_clean":"2023-12-19T18:16:41.553051+0000","last_became_active":"2023-12-19T18:16:40.563693+0000","last_became_peered":"2023-12-19T18:16:40.563693+0000","last_unstale":"2023-12-19T18:16:41.553051+0000","last_undegraded":"2023-12-19T18:16:41.553051+0000","last_fullsized":"2023-12-19T18:16:41.553051+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:27.465263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562303+0000","last_change":"2023-12-19T18:16:27.409874+0000","last_active":"2023-12-19T18:16:40.562303+0000","last_peered":"2023-12-19T18:16:40.562303+0000","last_clean":"2023-12-19T18:16:40.562303+0000","last_became_active":"2023-12-19T18:16:27.409598+0000","last_became_peered":"2023-12-19T18:16:27.409598+0000","last_unstale":"2023-12-19T18:16:40.562303+0000","last_undegraded":"2023-12-19T18:16:40.562303+0000","last_fullsized":"2023-12-19T18:16:40.562303+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:13.721057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549450+0000","last_change":"2023-12-19T18:16:27.826090+0000","last_active":"2023-12-19T18:16:40.549450+0000","last_peered":"2023-12-19T18:16:40.549450+0000","last_clean":"2023-12-19T18:16:40.549450+0000","last_became_active":"2023-12-19T18:16:27.825791+0000","last_became_peered":"2023-12-19T18:16:27.825791+0000","last_unstale":"2023-12-19T18:16:40.549450+0000","last_undegraded":"2023-12-19T18:16:40.549450+0000","last_fullsized":"2023-12-19T18:16:40.549450+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:39.382953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359959+0000","last_change":"2023-12-19T18:16:40.160037+0000","last_active":"2023-12-19T18:16:42.359959+0000","last_peered":"2023-12-19T18:16:42.359959+0000","last_clean":"2023-12-19T18:16:42.359959+0000","last_became_active":"2023-12-19T18:16:40.159737+0000","last_became_peered":"2023-12-19T18:16:40.159737+0000","last_unstale":"2023-12-19T18:16:42.359959+0000","last_undegraded":"2023-12-19T18:16:42.359959+0000","last_fullsized":"2023-12-19T18:16:42.359959+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:42.336188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358621+0000","last_change":"2023-12-19T18:16:40.567065+0000","last_active":"2023-12-19T18:16:42.358621+0000","last_peered":"2023-12-19T18:16:42.358621+0000","last_clean":"2023-12-19T18:16:42.358621+0000","last_became_active":"2023-12-19T18:16:40.566804+0000","last_became_peered":"2023-12-19T18:16:40.566804+0000","last_unstale":"2023-12-19T18:16:42.358621+0000","last_undegraded":"2023-12-19T18:16:42.358621+0000","last_fullsized":"2023-12-19T18:16:42.358621+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:21.029140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546688+0000","last_change":"2023-12-19T18:16:27.403537+0000","last_active":"2023-12-19T18:16:40.546688+0000","last_peered":"2023-12-19T18:16:40.546688+0000","last_clean":"2023-12-19T18:16:40.546688+0000","last_became_active":"2023-12-19T18:16:27.403250+0000","last_became_peered":"2023-12-19T18:16:27.403250+0000","last_unstale":"2023-12-19T18:16:40.546688+0000","last_undegraded":"2023-12-19T18:16:40.546688+0000","last_fullsized":"2023-12-19T18:16:40.546688+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:56.889902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548113+0000","last_change":"2023-12-19T18:16:27.825355+0000","last_active":"2023-12-19T18:16:40.548113+0000","last_peered":"2023-12-19T18:16:40.548113+0000","last_clean":"2023-12-19T18:16:40.548113+0000","last_became_active":"2023-12-19T18:16:27.825067+0000","last_became_peered":"2023-12-19T18:16:27.825067+0000","last_unstale":"2023-12-19T18:16:40.548113+0000","last_undegraded":"2023-12-19T18:16:40.548113+0000","last_fullsized":"2023-12-19T18:16:40.548113+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:02.928407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358143+0000","last_change":"2023-12-19T18:16:40.152882+0000","last_active":"2023-12-19T18:16:42.358143+0000","last_peered":"2023-12-19T18:16:42.358143+0000","last_clean":"2023-12-19T18:16:42.358143+0000","last_became_active":"2023-12-19T18:16:40.152561+0000","last_became_peered":"2023-12-19T18:16:40.152561+0000","last_unstale":"2023-12-19T18:16:42.358143+0000","last_undegraded":"2023-12-19T18:16:42.358143+0000","last_fullsized":"2023-12-19T18:16:42.358143+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:11:26.922031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550797+0000","last_change":"2023-12-19T18:16:40.560282+0000","last_active":"2023-12-19T18:16:41.550797+0000","last_peered":"2023-12-19T18:16:41.550797+0000","last_clean":"2023-12-19T18:16:41.550797+0000","last_became_active":"2023-12-19T18:16:40.560017+0000","last_became_peered":"2023-12-19T18:16:40.560017+0000","last_unstale":"2023-12-19T18:16:41.550797+0000","last_undegraded":"2023-12-19T18:16:41.550797+0000","last_fullsized":"2023-12-19T18:16:41.550797+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:51:01.095692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553689+0000","last_change":"2023-12-19T18:16:27.417198+0000","last_active":"2023-12-19T18:16:40.553689+0000","last_peered":"2023-12-19T18:16:40.553689+0000","last_clean":"2023-12-19T18:16:40.553689+0000","last_became_active":"2023-12-19T18:16:27.416902+0000","last_became_peered":"2023-12-19T18:16:27.416902+0000","last_unstale":"2023-12-19T18:16:40.553689+0000","last_undegraded":"2023-12-19T18:16:40.553689+0000","last_fullsized":"2023-12-19T18:16:40.553689+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:54.434372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560449+0000","last_change":"2023-12-19T18:16:27.824198+0000","last_active":"2023-12-19T18:16:40.560449+0000","last_peered":"2023-12-19T18:16:40.560449+0000","last_clean":"2023-12-19T18:16:40.560449+0000","last_became_active":"2023-12-19T18:16:27.823738+0000","last_became_peered":"2023-12-19T18:16:27.823738+0000","last_unstale":"2023-12-19T18:16:40.560449+0000","last_undegraded":"2023-12-19T18:16:40.560449+0000","last_fullsized":"2023-12-19T18:16:40.560449+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:53.509897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550952+0000","last_change":"2023-12-19T18:16:40.160436+0000","last_active":"2023-12-19T18:16:41.550952+0000","last_peered":"2023-12-19T18:16:41.550952+0000","last_clean":"2023-12-19T18:16:41.550952+0000","last_became_active":"2023-12-19T18:16:40.159692+0000","last_became_peered":"2023-12-19T18:16:40.159692+0000","last_unstale":"2023-12-19T18:16:41.550952+0000","last_undegraded":"2023-12-19T18:16:41.550952+0000","last_fullsized":"2023-12-19T18:16:41.550952+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:47:01.985064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554294+0000","last_change":"2023-12-19T18:16:40.572114+0000","last_active":"2023-12-19T18:16:41.554294+0000","last_peered":"2023-12-19T18:16:41.554294+0000","last_clean":"2023-12-19T18:16:41.554294+0000","last_became_active":"2023-12-19T18:16:40.571811+0000","last_became_peered":"2023-12-19T18:16:40.571811+0000","last_unstale":"2023-12-19T18:16:41.554294+0000","last_undegraded":"2023-12-19T18:16:41.554294+0000","last_fullsized":"2023-12-19T18:16:41.554294+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:01.072159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550771+0000","last_change":"2023-12-19T18:16:27.404853+0000","last_active":"2023-12-19T18:16:40.550771+0000","last_peered":"2023-12-19T18:16:40.550771+0000","last_clean":"2023-12-19T18:16:40.550771+0000","last_became_active":"2023-12-19T18:16:27.404347+0000","last_became_peered":"2023-12-19T18:16:27.404347+0000","last_unstale":"2023-12-19T18:16:40.550771+0000","last_undegraded":"2023-12-19T18:16:40.550771+0000","last_fullsized":"2023-12-19T18:16:40.550771+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:16.360996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558229+0000","last_change":"2023-12-19T18:16:27.828160+0000","last_active":"2023-12-19T18:16:40.558229+0000","last_peered":"2023-12-19T18:16:40.558229+0000","last_clean":"2023-12-19T18:16:40.558229+0000","last_became_active":"2023-12-19T18:16:27.827882+0000","last_became_peered":"2023-12-19T18:16:27.827882+0000","last_unstale":"2023-12-19T18:16:40.558229+0000","last_undegraded":"2023-12-19T18:16:40.558229+0000","last_fullsized":"2023-12-19T18:16:40.558229+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:17.726704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358933+0000","last_change":"2023-12-19T18:16:40.154706+0000","last_active":"2023-12-19T18:16:42.358933+0000","last_peered":"2023-12-19T18:16:42.358933+0000","last_clean":"2023-12-19T18:16:42.358933+0000","last_became_active":"2023-12-19T18:16:40.154154+0000","last_became_peered":"2023-12-19T18:16:40.154154+0000","last_unstale":"2023-12-19T18:16:42.358933+0000","last_undegraded":"2023-12-19T18:16:42.358933+0000","last_fullsized":"2023-12-19T18:16:42.358933+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:55.343366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552500+0000","last_change":"2023-12-19T18:16:40.555828+0000","last_active":"2023-12-19T18:16:41.552500+0000","last_peered":"2023-12-19T18:16:41.552500+0000","last_clean":"2023-12-19T18:16:41.552500+0000","last_became_active":"2023-12-19T18:16:40.555500+0000","last_became_peered":"2023-12-19T18:16:40.555500+0000","last_unstale":"2023-12-19T18:16:41.552500+0000","last_undegraded":"2023-12-19T18:16:41.552500+0000","last_fullsized":"2023-12-19T18:16:41.552500+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:50.483528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.b","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551100+0000","last_change":"2023-12-19T18:16:27.415902+0000","last_active":"2023-12-19T18:16:40.551100+0000","last_peered":"2023-12-19T18:16:40.551100+0000","last_clean":"2023-12-19T18:16:40.551100+0000","last_became_active":"2023-12-19T18:16:27.415062+0000","last_became_peered":"2023-12-19T18:16:27.415062+0000","last_unstale":"2023-12-19T18:16:40.551100+0000","last_undegraded":"2023-12-19T18:16:40.551100+0000","last_fullsized":"2023-12-19T18:16:40.551100+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:22.442653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556868+0000","last_change":"2023-12-19T18:16:27.827898+0000","last_active":"2023-12-19T18:16:40.556868+0000","last_peered":"2023-12-19T18:16:40.556868+0000","last_clean":"2023-12-19T18:16:40.556868+0000","last_became_active":"2023-12-19T18:16:27.827613+0000","last_became_peered":"2023-12-19T18:16:27.827613+0000","last_unstale":"2023-12-19T18:16:40.556868+0000","last_undegraded":"2023-12-19T18:16:40.556868+0000","last_fullsized":"2023-12-19T18:16:40.556868+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:29.033205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357614+0000","last_change":"2023-12-19T18:16:40.157634+0000","last_active":"2023-12-19T18:16:42.357614+0000","last_peered":"2023-12-19T18:16:42.357614+0000","last_clean":"2023-12-19T18:16:42.357614+0000","last_became_active":"2023-12-19T18:16:40.157363+0000","last_became_peered":"2023-12-19T18:16:40.157363+0000","last_unstale":"2023-12-19T18:16:42.357614+0000","last_undegraded":"2023-12-19T18:16:42.357614+0000","last_fullsized":"2023-12-19T18:16:42.357614+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:23.564600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553681+0000","last_change":"2023-12-19T18:16:40.556441+0000","last_active":"2023-12-19T18:16:41.553681+0000","last_peered":"2023-12-19T18:16:41.553681+0000","last_clean":"2023-12-19T18:16:41.553681+0000","last_became_active":"2023-12-19T18:16:40.556187+0000","last_became_peered":"2023-12-19T18:16:40.556187+0000","last_unstale":"2023-12-19T18:16:41.553681+0000","last_undegraded":"2023-12-19T18:16:41.553681+0000","last_fullsized":"2023-12-19T18:16:41.553681+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:16:05.083354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562653+0000","last_change":"2023-12-19T18:16:27.408051+0000","last_active":"2023-12-19T18:16:40.562653+0000","last_peered":"2023-12-19T18:16:40.562653+0000","last_clean":"2023-12-19T18:16:40.562653+0000","last_became_active":"2023-12-19T18:16:27.407796+0000","last_became_peered":"2023-12-19T18:16:27.407796+0000","last_unstale":"2023-12-19T18:16:40.562653+0000","last_undegraded":"2023-12-19T18:16:40.562653+0000","last_fullsized":"2023-12-19T18:16:40.562653+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:41:09.697612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560009+0000","last_change":"2023-12-19T18:16:27.832376+0000","last_active":"2023-12-19T18:16:40.560009+0000","last_peered":"2023-12-19T18:16:40.560009+0000","last_clean":"2023-12-19T18:16:40.560009+0000","last_became_active":"2023-12-19T18:16:27.832060+0000","last_became_peered":"2023-12-19T18:16:27.832060+0000","last_unstale":"2023-12-19T18:16:40.560009+0000","last_undegraded":"2023-12-19T18:16:40.560009+0000","last_fullsized":"2023-12-19T18:16:40.560009+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:17.432889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.e","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553537+0000","last_change":"2023-12-19T18:16:40.152827+0000","last_active":"2023-12-19T18:16:41.553537+0000","last_peered":"2023-12-19T18:16:41.553537+0000","last_clean":"2023-12-19T18:16:41.553537+0000","last_became_active":"2023-12-19T18:16:40.152559+0000","last_became_peered":"2023-12-19T18:16:40.152559+0000","last_unstale":"2023-12-19T18:16:41.553537+0000","last_undegraded":"2023-12-19T18:16:41.553537+0000","last_fullsized":"2023-12-19T18:16:41.553537+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:59.363074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553423+0000","last_change":"2023-12-19T18:16:40.572243+0000","last_active":"2023-12-19T18:16:41.553423+0000","last_peered":"2023-12-19T18:16:41.553423+0000","last_clean":"2023-12-19T18:16:41.553423+0000","last_became_active":"2023-12-19T18:16:40.571938+0000","last_became_peered":"2023-12-19T18:16:40.571938+0000","last_unstale":"2023-12-19T18:16:41.553423+0000","last_undegraded":"2023-12-19T18:16:41.553423+0000","last_fullsized":"2023-12-19T18:16:41.553423+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:50.735904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562479+0000","last_change":"2023-12-19T18:16:27.416654+0000","last_active":"2023-12-19T18:16:40.562479+0000","last_peered":"2023-12-19T18:16:40.562479+0000","last_clean":"2023-12-19T18:16:40.562479+0000","last_became_active":"2023-12-19T18:16:27.416388+0000","last_became_peered":"2023-12-19T18:16:27.416388+0000","last_unstale":"2023-12-19T18:16:40.562479+0000","last_undegraded":"2023-12-19T18:16:40.562479+0000","last_fullsized":"2023-12-19T18:16:40.562479+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:51.339148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547637+0000","last_change":"2023-12-19T18:16:27.816436+0000","last_active":"2023-12-19T18:16:40.547637+0000","last_peered":"2023-12-19T18:16:40.547637+0000","last_clean":"2023-12-19T18:16:40.547637+0000","last_became_active":"2023-12-19T18:16:27.816092+0000","last_became_peered":"2023-12-19T18:16:27.816092+0000","last_unstale":"2023-12-19T18:16:40.547637+0000","last_undegraded":"2023-12-19T18:16:40.547637+0000","last_fullsized":"2023-12-19T18:16:40.547637+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:07:26.713649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360338+0000","last_change":"2023-12-19T18:16:40.154797+0000","last_active":"2023-12-19T18:16:42.360338+0000","last_peered":"2023-12-19T18:16:42.360338+0000","last_clean":"2023-12-19T18:16:42.360338+0000","last_became_active":"2023-12-19T18:16:40.154490+0000","last_became_peered":"2023-12-19T18:16:40.154490+0000","last_unstale":"2023-12-19T18:16:42.360338+0000","last_undegraded":"2023-12-19T18:16:42.360338+0000","last_fullsized":"2023-12-19T18:16:42.360338+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:10:02.821689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551216+0000","last_change":"2023-12-19T18:16:40.573560+0000","last_active":"2023-12-19T18:16:41.551216+0000","last_peered":"2023-12-19T18:16:41.551216+0000","last_clean":"2023-12-19T18:16:41.551216+0000","last_became_active":"2023-12-19T18:16:40.573277+0000","last_became_peered":"2023-12-19T18:16:40.573277+0000","last_unstale":"2023-12-19T18:16:41.551216+0000","last_undegraded":"2023-12-19T18:16:41.551216+0000","last_fullsized":"2023-12-19T18:16:41.551216+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:23:53.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544525+0000","last_change":"2023-12-19T18:16:27.409426+0000","last_active":"2023-12-19T18:16:40.544525+0000","last_peered":"2023-12-19T18:16:40.544525+0000","last_clean":"2023-12-19T18:16:40.544525+0000","last_became_active":"2023-12-19T18:16:27.407151+0000","last_became_peered":"2023-12-19T18:16:27.407151+0000","last_unstale":"2023-12-19T18:16:40.544525+0000","last_undegraded":"2023-12-19T18:16:40.544525+0000","last_fullsized":"2023-12-19T18:16:40.544525+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:30:56.976667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550067+0000","last_change":"2023-12-19T18:16:27.823846+0000","last_active":"2023-12-19T18:16:40.550067+0000","last_peered":"2023-12-19T18:16:40.550067+0000","last_clean":"2023-12-19T18:16:40.550067+0000","last_became_active":"2023-12-19T18:16:27.823514+0000","last_became_peered":"2023-12-19T18:16:27.823514+0000","last_unstale":"2023-12-19T18:16:40.550067+0000","last_undegraded":"2023-12-19T18:16:40.550067+0000","last_fullsized":"2023-12-19T18:16:40.550067+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:20.691537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358069+0000","last_change":"2023-12-19T18:16:40.162626+0000","last_active":"2023-12-19T18:16:42.358069+0000","last_peered":"2023-12-19T18:16:42.358069+0000","last_clean":"2023-12-19T18:16:42.358069+0000","last_became_active":"2023-12-19T18:16:40.161257+0000","last_became_peered":"2023-12-19T18:16:40.161257+0000","last_unstale":"2023-12-19T18:16:42.358069+0000","last_undegraded":"2023-12-19T18:16:42.358069+0000","last_fullsized":"2023-12-19T18:16:42.358069+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:21:05.448555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357894+0000","last_change":"2023-12-19T18:16:40.558784+0000","last_active":"2023-12-19T18:16:42.357894+0000","last_peered":"2023-12-19T18:16:42.357894+0000","last_clean":"2023-12-19T18:16:42.357894+0000","last_became_active":"2023-12-19T18:16:40.556768+0000","last_became_peered":"2023-12-19T18:16:40.556768+0000","last_unstale":"2023-12-19T18:16:42.357894+0000","last_undegraded":"2023-12-19T18:16:42.357894+0000","last_fullsized":"2023-12-19T18:16:42.357894+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:14:19.106971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553891+0000","last_change":"2023-12-19T18:16:27.414971+0000","last_active":"2023-12-19T18:16:40.553891+0000","last_peered":"2023-12-19T18:16:40.553891+0000","last_clean":"2023-12-19T18:16:40.553891+0000","last_became_active":"2023-12-19T18:16:27.414662+0000","last_became_peered":"2023-12-19T18:16:27.414662+0000","last_unstale":"2023-12-19T18:16:40.553891+0000","last_undegraded":"2023-12-19T18:16:40.553891+0000","last_fullsized":"2023-12-19T18:16:40.553891+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:47.204637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558717+0000","last_change":"2023-12-19T18:16:27.814804+0000","last_active":"2023-12-19T18:16:40.558717+0000","last_peered":"2023-12-19T18:16:40.558717+0000","last_clean":"2023-12-19T18:16:40.558717+0000","last_became_active":"2023-12-19T18:16:27.814534+0000","last_became_peered":"2023-12-19T18:16:27.814534+0000","last_unstale":"2023-12-19T18:16:40.558717+0000","last_undegraded":"2023-12-19T18:16:40.558717+0000","last_fullsized":"2023-12-19T18:16:40.558717+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:45.690711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551146+0000","last_change":"2023-12-19T18:16:40.158484+0000","last_active":"2023-12-19T18:16:41.551146+0000","last_peered":"2023-12-19T18:16:41.551146+0000","last_clean":"2023-12-19T18:16:41.551146+0000","last_became_active":"2023-12-19T18:16:40.158205+0000","last_became_peered":"2023-12-19T18:16:40.158205+0000","last_unstale":"2023-12-19T18:16:41.551146+0000","last_undegraded":"2023-12-19T18:16:41.551146+0000","last_fullsized":"2023-12-19T18:16:41.551146+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:23.673873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550951+0000","last_change":"2023-12-19T18:16:40.555429+0000","last_active":"2023-12-19T18:16:41.550951+0000","last_peered":"2023-12-19T18:16:41.550951+0000","last_clean":"2023-12-19T18:16:41.550951+0000","last_became_active":"2023-12-19T18:16:40.555158+0000","last_became_peered":"2023-12-19T18:16:40.555158+0000","last_unstale":"2023-12-19T18:16:41.550951+0000","last_undegraded":"2023-12-19T18:16:41.550951+0000","last_fullsized":"2023-12-19T18:16:41.550951+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:53.628064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.10","version":"211'8","reported_seq":23,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560457+0000","last_change":"2023-12-19T18:16:27.415118+0000","last_active":"2023-12-19T18:16:40.560457+0000","last_peered":"2023-12-19T18:16:40.560457+0000","last_clean":"2023-12-19T18:16:40.560457+0000","last_became_active":"2023-12-19T18:16:27.414687+0000","last_became_peered":"2023-12-19T18:16:27.414687+0000","last_unstale":"2023-12-19T18:16:40.560457+0000","last_undegraded":"2023-12-19T18:16:40.560457+0000","last_fullsized":"2023-12-19T18:16:40.560457+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:16.723922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555219+0000","last_change":"2023-12-19T18:16:27.814277+0000","last_active":"2023-12-19T18:16:40.555219+0000","last_peered":"2023-12-19T18:16:40.555219+0000","last_clean":"2023-12-19T18:16:40.555219+0000","last_became_active":"2023-12-19T18:16:27.814023+0000","last_became_peered":"2023-12-19T18:16:27.814023+0000","last_unstale":"2023-12-19T18:16:40.555219+0000","last_undegraded":"2023-12-19T18:16:40.555219+0000","last_fullsized":"2023-12-19T18:16:40.555219+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:32.135133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554395+0000","last_change":"2023-12-19T18:16:40.154784+0000","last_active":"2023-12-19T18:16:41.554395+0000","last_peered":"2023-12-19T18:16:41.554395+0000","last_clean":"2023-12-19T18:16:41.554395+0000","last_became_active":"2023-12-19T18:16:40.154523+0000","last_became_peered":"2023-12-19T18:16:40.154523+0000","last_unstale":"2023-12-19T18:16:41.554395+0000","last_undegraded":"2023-12-19T18:16:41.554395+0000","last_fullsized":"2023-12-19T18:16:41.554395+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:14.215221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359308+0000","last_change":"2023-12-19T18:16:40.558066+0000","last_active":"2023-12-19T18:16:42.359308+0000","last_peered":"2023-12-19T18:16:42.359308+0000","last_clean":"2023-12-19T18:16:42.359308+0000","last_became_active":"2023-12-19T18:16:40.557771+0000","last_became_peered":"2023-12-19T18:16:40.557771+0000","last_unstale":"2023-12-19T18:16:42.359308+0000","last_undegraded":"2023-12-19T18:16:42.359308+0000","last_fullsized":"2023-12-19T18:16:42.359308+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:28.573589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543014+0000","last_change":"2023-12-19T18:16:27.403403+0000","last_active":"2023-12-19T18:16:40.543014+0000","last_peered":"2023-12-19T18:16:40.543014+0000","last_clean":"2023-12-19T18:16:40.543014+0000","last_became_active":"2023-12-19T18:16:27.403121+0000","last_became_peered":"2023-12-19T18:16:27.403121+0000","last_unstale":"2023-12-19T18:16:40.543014+0000","last_undegraded":"2023-12-19T18:16:40.543014+0000","last_fullsized":"2023-12-19T18:16:40.543014+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:36:58.560143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546582+0000","last_change":"2023-12-19T18:16:27.821814+0000","last_active":"2023-12-19T18:16:40.546582+0000","last_peered":"2023-12-19T18:16:40.546582+0000","last_clean":"2023-12-19T18:16:40.546582+0000","last_became_active":"2023-12-19T18:16:27.821519+0000","last_became_peered":"2023-12-19T18:16:27.821519+0000","last_unstale":"2023-12-19T18:16:40.546582+0000","last_undegraded":"2023-12-19T18:16:40.546582+0000","last_fullsized":"2023-12-19T18:16:40.546582+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:40.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.13","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554534+0000","last_change":"2023-12-19T18:16:40.160940+0000","last_active":"2023-12-19T18:16:41.554534+0000","last_peered":"2023-12-19T18:16:41.554534+0000","last_clean":"2023-12-19T18:16:41.554534+0000","last_became_active":"2023-12-19T18:16:40.160594+0000","last_became_peered":"2023-12-19T18:16:40.160594+0000","last_unstale":"2023-12-19T18:16:41.554534+0000","last_undegraded":"2023-12-19T18:16:41.554534+0000","last_fullsized":"2023-12-19T18:16:41.554534+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:01.011110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357451+0000","last_change":"2023-12-19T18:16:40.564547+0000","last_active":"2023-12-19T18:16:42.357451+0000","last_peered":"2023-12-19T18:16:42.357451+0000","last_clean":"2023-12-19T18:16:42.357451+0000","last_became_active":"2023-12-19T18:16:40.564300+0000","last_became_peered":"2023-12-19T18:16:40.564300+0000","last_unstale":"2023-12-19T18:16:42.357451+0000","last_undegraded":"2023-12-19T18:16:42.357451+0000","last_fullsized":"2023-12-19T18:16:42.357451+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:42.276098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556939+0000","last_change":"2023-12-19T18:16:27.417401+0000","last_active":"2023-12-19T18:16:40.556939+0000","last_peered":"2023-12-19T18:16:40.556939+0000","last_clean":"2023-12-19T18:16:40.556939+0000","last_became_active":"2023-12-19T18:16:27.416774+0000","last_became_peered":"2023-12-19T18:16:27.416774+0000","last_unstale":"2023-12-19T18:16:40.556939+0000","last_undegraded":"2023-12-19T18:16:40.556939+0000","last_fullsized":"2023-12-19T18:16:40.556939+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:03.620596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558146+0000","last_change":"2023-12-19T18:16:27.824516+0000","last_active":"2023-12-19T18:16:40.558146+0000","last_peered":"2023-12-19T18:16:40.558146+0000","last_clean":"2023-12-19T18:16:40.558146+0000","last_became_active":"2023-12-19T18:16:27.823143+0000","last_became_peered":"2023-12-19T18:16:27.823143+0000","last_unstale":"2023-12-19T18:16:40.558146+0000","last_undegraded":"2023-12-19T18:16:40.558146+0000","last_fullsized":"2023-12-19T18:16:40.558146+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:52.341930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"215'8","reported_seq":19,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357609+0000","last_change":"2023-12-19T18:16:40.161467+0000","last_active":"2023-12-19T18:16:42.357609+0000","last_peered":"2023-12-19T18:16:42.357609+0000","last_clean":"2023-12-19T18:16:42.357609+0000","last_became_active":"2023-12-19T18:16:40.161143+0000","last_became_peered":"2023-12-19T18:16:40.161143+0000","last_unstale":"2023-12-19T18:16:42.357609+0000","last_undegraded":"2023-12-19T18:16:42.357609+0000","last_fullsized":"2023-12-19T18:16:42.357609+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:21:09.532919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357722+0000","last_change":"2023-12-19T18:16:40.571041+0000","last_active":"2023-12-19T18:16:42.357722+0000","last_peered":"2023-12-19T18:16:42.357722+0000","last_clean":"2023-12-19T18:16:42.357722+0000","last_became_active":"2023-12-19T18:16:40.570705+0000","last_became_peered":"2023-12-19T18:16:40.570705+0000","last_unstale":"2023-12-19T18:16:42.357722+0000","last_undegraded":"2023-12-19T18:16:42.357722+0000","last_fullsized":"2023-12-19T18:16:42.357722+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:43.042802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.13","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560853+0000","last_change":"2023-12-19T18:16:27.415000+0000","last_active":"2023-12-19T18:16:40.560853+0000","last_peered":"2023-12-19T18:16:40.560853+0000","last_clean":"2023-12-19T18:16:40.560853+0000","last_became_active":"2023-12-19T18:16:27.414679+0000","last_became_peered":"2023-12-19T18:16:27.414679+0000","last_unstale":"2023-12-19T18:16:40.560853+0000","last_undegraded":"2023-12-19T18:16:40.560853+0000","last_fullsized":"2023-12-19T18:16:40.560853+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:50.187832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554789+0000","last_change":"2023-12-19T18:16:27.824833+0000","last_active":"2023-12-19T18:16:40.554789+0000","last_peered":"2023-12-19T18:16:40.554789+0000","last_clean":"2023-12-19T18:16:40.554789+0000","last_became_active":"2023-12-19T18:16:27.824487+0000","last_became_peered":"2023-12-19T18:16:27.824487+0000","last_unstale":"2023-12-19T18:16:40.554789+0000","last_undegraded":"2023-12-19T18:16:40.554789+0000","last_fullsized":"2023-12-19T18:16:40.554789+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550312+0000","last_change":"2023-12-19T18:16:40.150872+0000","last_active":"2023-12-19T18:16:41.550312+0000","last_peered":"2023-12-19T18:16:41.550312+0000","last_clean":"2023-12-19T18:16:41.550312+0000","last_became_active":"2023-12-19T18:16:40.150586+0000","last_became_peered":"2023-12-19T18:16:40.150586+0000","last_unstale":"2023-12-19T18:16:41.550312+0000","last_undegraded":"2023-12-19T18:16:41.550312+0000","last_fullsized":"2023-12-19T18:16:41.550312+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:55.343366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358620+0000","last_change":"2023-12-19T18:16:40.571454+0000","last_active":"2023-12-19T18:16:42.358620+0000","last_peered":"2023-12-19T18:16:42.358620+0000","last_clean":"2023-12-19T18:16:42.358620+0000","last_became_active":"2023-12-19T18:16:40.570299+0000","last_became_peered":"2023-12-19T18:16:40.570299+0000","last_unstale":"2023-12-19T18:16:42.358620+0000","last_undegraded":"2023-12-19T18:16:42.358620+0000","last_fullsized":"2023-12-19T18:16:42.358620+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:31:09.517418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"212'8","reported_seq":23,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553799+0000","last_change":"2023-12-19T18:16:27.414573+0000","last_active":"2023-12-19T18:16:40.553799+0000","last_peered":"2023-12-19T18:16:40.553799+0000","last_clean":"2023-12-19T18:16:40.553799+0000","last_became_active":"2023-12-19T18:16:27.414268+0000","last_became_peered":"2023-12-19T18:16:27.414268+0000","last_unstale":"2023-12-19T18:16:40.553799+0000","last_undegraded":"2023-12-19T18:16:40.553799+0000","last_fullsized":"2023-12-19T18:16:40.553799+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:11.117214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555559+0000","last_change":"2023-12-19T18:16:27.829311+0000","last_active":"2023-12-19T18:16:40.555559+0000","last_peered":"2023-12-19T18:16:40.555559+0000","last_clean":"2023-12-19T18:16:40.555559+0000","last_became_active":"2023-12-19T18:16:27.828953+0000","last_became_peered":"2023-12-19T18:16:27.828953+0000","last_unstale":"2023-12-19T18:16:40.555559+0000","last_undegraded":"2023-12-19T18:16:40.555559+0000","last_fullsized":"2023-12-19T18:16:40.555559+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:23.417710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554767+0000","last_change":"2023-12-19T18:16:40.160698+0000","last_active":"2023-12-19T18:16:41.554767+0000","last_peered":"2023-12-19T18:16:41.554767+0000","last_clean":"2023-12-19T18:16:41.554767+0000","last_became_active":"2023-12-19T18:16:40.160406+0000","last_became_peered":"2023-12-19T18:16:40.160406+0000","last_unstale":"2023-12-19T18:16:41.554767+0000","last_undegraded":"2023-12-19T18:16:41.554767+0000","last_fullsized":"2023-12-19T18:16:41.554767+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:10.441625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.075170+0000","last_change":"2023-12-19T18:16:40.579076+0000","last_active":"2023-12-19T18:16:42.075170+0000","last_peered":"2023-12-19T18:16:42.075170+0000","last_clean":"2023-12-19T18:16:42.075170+0000","last_became_active":"2023-12-19T18:16:40.578641+0000","last_became_peered":"2023-12-19T18:16:40.578641+0000","last_unstale":"2023-12-19T18:16:42.075170+0000","last_undegraded":"2023-12-19T18:16:42.075170+0000","last_fullsized":"2023-12-19T18:16:42.075170+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:33:08.245627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.15","version":"211'3","reported_seq":18,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555258+0000","last_change":"2023-12-19T18:16:27.409442+0000","last_active":"2023-12-19T18:16:40.555258+0000","last_peered":"2023-12-19T18:16:40.555258+0000","last_clean":"2023-12-19T18:16:40.555258+0000","last_became_active":"2023-12-19T18:16:27.409094+0000","last_became_peered":"2023-12-19T18:16:27.409094+0000","last_unstale":"2023-12-19T18:16:40.555258+0000","last_undegraded":"2023-12-19T18:16:40.555258+0000","last_fullsized":"2023-12-19T18:16:40.555258+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:18.713070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562498+0000","last_change":"2023-12-19T18:16:27.811752+0000","last_active":"2023-12-19T18:16:40.562498+0000","last_peered":"2023-12-19T18:16:40.562498+0000","last_clean":"2023-12-19T18:16:40.562498+0000","last_became_active":"2023-12-19T18:16:27.811273+0000","last_became_peered":"2023-12-19T18:16:27.811273+0000","last_unstale":"2023-12-19T18:16:40.562498+0000","last_undegraded":"2023-12-19T18:16:40.562498+0000","last_fullsized":"2023-12-19T18:16:40.562498+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:23.609660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360336+0000","last_change":"2023-12-19T18:16:40.162801+0000","last_active":"2023-12-19T18:16:42.360336+0000","last_peered":"2023-12-19T18:16:42.360336+0000","last_clean":"2023-12-19T18:16:42.360336+0000","last_became_active":"2023-12-19T18:16:40.162367+0000","last_became_peered":"2023-12-19T18:16:40.162367+0000","last_unstale":"2023-12-19T18:16:42.360336+0000","last_undegraded":"2023-12-19T18:16:42.360336+0000","last_fullsized":"2023-12-19T18:16:42.360336+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:08:09.383590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359637+0000","last_change":"2023-12-19T18:16:40.570887+0000","last_active":"2023-12-19T18:16:42.359637+0000","last_peered":"2023-12-19T18:16:42.359637+0000","last_clean":"2023-12-19T18:16:42.359637+0000","last_became_active":"2023-12-19T18:16:40.570475+0000","last_became_peered":"2023-12-19T18:16:40.570475+0000","last_unstale":"2023-12-19T18:16:42.359637+0000","last_undegraded":"2023-12-19T18:16:42.359637+0000","last_fullsized":"2023-12-19T18:16:42.359637+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:53:00.119855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549599+0000","last_change":"2023-12-19T18:16:27.410603+0000","last_active":"2023-12-19T18:16:40.549599+0000","last_peered":"2023-12-19T18:16:40.549599+0000","last_clean":"2023-12-19T18:16:40.549599+0000","last_became_active":"2023-12-19T18:16:27.409260+0000","last_became_peered":"2023-12-19T18:16:27.409260+0000","last_unstale":"2023-12-19T18:16:40.549599+0000","last_undegraded":"2023-12-19T18:16:40.549599+0000","last_fullsized":"2023-12-19T18:16:40.549599+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:06.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543878+0000","last_change":"2023-12-19T18:16:27.835564+0000","last_active":"2023-12-19T18:16:40.543878+0000","last_peered":"2023-12-19T18:16:40.543878+0000","last_clean":"2023-12-19T18:16:40.543878+0000","last_became_active":"2023-12-19T18:16:27.835319+0000","last_became_peered":"2023-12-19T18:16:27.835319+0000","last_unstale":"2023-12-19T18:16:40.543878+0000","last_undegraded":"2023-12-19T18:16:40.543878+0000","last_fullsized":"2023-12-19T18:16:40.543878+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:10:35.037185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"215'6","reported_seq":17,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074989+0000","last_change":"2023-12-19T18:16:40.163664+0000","last_active":"2023-12-19T18:16:42.074989+0000","last_peered":"2023-12-19T18:16:42.074989+0000","last_clean":"2023-12-19T18:16:42.074989+0000","last_became_active":"2023-12-19T18:16:40.163301+0000","last_became_peered":"2023-12-19T18:16:40.163301+0000","last_unstale":"2023-12-19T18:16:42.074989+0000","last_undegraded":"2023-12-19T18:16:42.074989+0000","last_fullsized":"2023-12-19T18:16:42.074989+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:33.099709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074835+0000","last_change":"2023-12-19T18:16:40.581101+0000","last_active":"2023-12-19T18:16:42.074835+0000","last_peered":"2023-12-19T18:16:42.074835+0000","last_clean":"2023-12-19T18:16:42.074835+0000","last_became_active":"2023-12-19T18:16:40.579272+0000","last_became_peered":"2023-12-19T18:16:40.579272+0000","last_unstale":"2023-12-19T18:16:42.074835+0000","last_undegraded":"2023-12-19T18:16:42.074835+0000","last_fullsized":"2023-12-19T18:16:42.074835+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:12:36.065977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565329+0000","last_change":"2023-12-19T18:16:27.405556+0000","last_active":"2023-12-19T18:16:40.565329+0000","last_peered":"2023-12-19T18:16:40.565329+0000","last_clean":"2023-12-19T18:16:40.565329+0000","last_became_active":"2023-12-19T18:16:27.404804+0000","last_became_peered":"2023-12-19T18:16:27.404804+0000","last_unstale":"2023-12-19T18:16:40.565329+0000","last_undegraded":"2023-12-19T18:16:40.565329+0000","last_fullsized":"2023-12-19T18:16:40.565329+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:05.395130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551274+0000","last_change":"2023-12-19T18:16:27.830538+0000","last_active":"2023-12-19T18:16:40.551274+0000","last_peered":"2023-12-19T18:16:40.551274+0000","last_clean":"2023-12-19T18:16:40.551274+0000","last_became_active":"2023-12-19T18:16:27.828974+0000","last_became_peered":"2023-12-19T18:16:27.828974+0000","last_unstale":"2023-12-19T18:16:40.551274+0000","last_undegraded":"2023-12-19T18:16:40.551274+0000","last_fullsized":"2023-12-19T18:16:40.551274+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:36.961741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.15","version":"215'3","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551392+0000","last_change":"2023-12-19T18:16:40.151095+0000","last_active":"2023-12-19T18:16:41.551392+0000","last_peered":"2023-12-19T18:16:41.551392+0000","last_clean":"2023-12-19T18:16:41.551392+0000","last_became_active":"2023-12-19T18:16:40.150609+0000","last_became_peered":"2023-12-19T18:16:40.150609+0000","last_unstale":"2023-12-19T18:16:41.551392+0000","last_undegraded":"2023-12-19T18:16:41.551392+0000","last_fullsized":"2023-12-19T18:16:41.551392+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:15.058778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074922+0000","last_change":"2023-12-19T18:16:40.580029+0000","last_active":"2023-12-19T18:16:42.074922+0000","last_peered":"2023-12-19T18:16:42.074922+0000","last_clean":"2023-12-19T18:16:42.074922+0000","last_became_active":"2023-12-19T18:16:40.579698+0000","last_became_peered":"2023-12-19T18:16:40.579698+0000","last_unstale":"2023-12-19T18:16:42.074922+0000","last_undegraded":"2023-12-19T18:16:42.074922+0000","last_fullsized":"2023-12-19T18:16:42.074922+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:26:54.889491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560368+0000","last_change":"2023-12-19T18:16:27.413444+0000","last_active":"2023-12-19T18:16:40.560368+0000","last_peered":"2023-12-19T18:16:40.560368+0000","last_clean":"2023-12-19T18:16:40.560368+0000","last_became_active":"2023-12-19T18:16:27.413066+0000","last_became_peered":"2023-12-19T18:16:27.413066+0000","last_unstale":"2023-12-19T18:16:40.560368+0000","last_undegraded":"2023-12-19T18:16:40.560368+0000","last_fullsized":"2023-12-19T18:16:40.560368+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:57.997657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558113+0000","last_change":"2023-12-19T18:16:27.816735+0000","last_active":"2023-12-19T18:16:40.558113+0000","last_peered":"2023-12-19T18:16:40.558113+0000","last_clean":"2023-12-19T18:16:40.558113+0000","last_became_active":"2023-12-19T18:16:27.816436+0000","last_became_peered":"2023-12-19T18:16:27.816436+0000","last_unstale":"2023-12-19T18:16:40.558113+0000","last_undegraded":"2023-12-19T18:16:40.558113+0000","last_fullsized":"2023-12-19T18:16:40.558113+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:37.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1a","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359740+0000","last_change":"2023-12-19T18:16:40.161453+0000","last_active":"2023-12-19T18:16:42.359740+0000","last_peered":"2023-12-19T18:16:42.359740+0000","last_clean":"2023-12-19T18:16:42.359740+0000","last_became_active":"2023-12-19T18:16:40.161191+0000","last_became_peered":"2023-12-19T18:16:40.161191+0000","last_unstale":"2023-12-19T18:16:42.359740+0000","last_undegraded":"2023-12-19T18:16:42.359740+0000","last_fullsized":"2023-12-19T18:16:42.359740+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:13.660363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074624+0000","last_change":"2023-12-19T18:16:40.578142+0000","last_active":"2023-12-19T18:16:42.074624+0000","last_peered":"2023-12-19T18:16:42.074624+0000","last_clean":"2023-12-19T18:16:42.074624+0000","last_became_active":"2023-12-19T18:16:40.577655+0000","last_became_peered":"2023-12-19T18:16:40.577655+0000","last_unstale":"2023-12-19T18:16:42.074624+0000","last_undegraded":"2023-12-19T18:16:42.074624+0000","last_fullsized":"2023-12-19T18:16:42.074624+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:41.805677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544353+0000","last_change":"2023-12-19T18:16:27.414372+0000","last_active":"2023-12-19T18:16:40.544353+0000","last_peered":"2023-12-19T18:16:40.544353+0000","last_clean":"2023-12-19T18:16:40.544353+0000","last_became_active":"2023-12-19T18:16:27.414088+0000","last_became_peered":"2023-12-19T18:16:27.414088+0000","last_unstale":"2023-12-19T18:16:40.544353+0000","last_undegraded":"2023-12-19T18:16:40.544353+0000","last_fullsized":"2023-12-19T18:16:40.544353+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:18.945766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543503+0000","last_change":"2023-12-19T18:16:27.827386+0000","last_active":"2023-12-19T18:16:40.543503+0000","last_peered":"2023-12-19T18:16:40.543503+0000","last_clean":"2023-12-19T18:16:40.543503+0000","last_became_active":"2023-12-19T18:16:27.827086+0000","last_became_peered":"2023-12-19T18:16:27.827086+0000","last_unstale":"2023-12-19T18:16:40.543503+0000","last_undegraded":"2023-12-19T18:16:40.543503+0000","last_fullsized":"2023-12-19T18:16:40.543503+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:37.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554117+0000","last_change":"2023-12-19T18:16:40.156738+0000","last_active":"2023-12-19T18:16:41.554117+0000","last_peered":"2023-12-19T18:16:41.554117+0000","last_clean":"2023-12-19T18:16:41.554117+0000","last_became_active":"2023-12-19T18:16:40.156488+0000","last_became_peered":"2023-12-19T18:16:40.156488+0000","last_unstale":"2023-12-19T18:16:41.554117+0000","last_undegraded":"2023-12-19T18:16:41.554117+0000","last_fullsized":"2023-12-19T18:16:41.554117+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:48.695934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358850+0000","last_change":"2023-12-19T18:16:40.575709+0000","last_active":"2023-12-19T18:16:42.358850+0000","last_peered":"2023-12-19T18:16:42.358850+0000","last_clean":"2023-12-19T18:16:42.358850+0000","last_became_active":"2023-12-19T18:16:40.575453+0000","last_became_peered":"2023-12-19T18:16:40.575453+0000","last_unstale":"2023-12-19T18:16:42.358850+0000","last_undegraded":"2023-12-19T18:16:42.358850+0000","last_fullsized":"2023-12-19T18:16:42.358850+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:44.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1b","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551641+0000","last_change":"2023-12-19T18:16:27.415036+0000","last_active":"2023-12-19T18:16:40.551641+0000","last_peered":"2023-12-19T18:16:40.551641+0000","last_clean":"2023-12-19T18:16:40.551641+0000","last_became_active":"2023-12-19T18:16:27.414747+0000","last_became_peered":"2023-12-19T18:16:27.414747+0000","last_unstale":"2023-12-19T18:16:40.551641+0000","last_undegraded":"2023-12-19T18:16:40.551641+0000","last_fullsized":"2023-12-19T18:16:40.551641+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:03.064138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556794+0000","last_change":"2023-12-19T18:16:27.828398+0000","last_active":"2023-12-19T18:16:40.556794+0000","last_peered":"2023-12-19T18:16:40.556794+0000","last_clean":"2023-12-19T18:16:40.556794+0000","last_became_active":"2023-12-19T18:16:27.828082+0000","last_became_peered":"2023-12-19T18:16:27.828082+0000","last_unstale":"2023-12-19T18:16:40.556794+0000","last_undegraded":"2023-12-19T18:16:40.556794+0000","last_fullsized":"2023-12-19T18:16:40.556794+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:42:30.998150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358258+0000","last_change":"2023-12-19T18:16:40.159943+0000","last_active":"2023-12-19T18:16:42.358258+0000","last_peered":"2023-12-19T18:16:42.358258+0000","last_clean":"2023-12-19T18:16:42.358258+0000","last_became_active":"2023-12-19T18:16:40.159652+0000","last_became_peered":"2023-12-19T18:16:40.159652+0000","last_unstale":"2023-12-19T18:16:42.358258+0000","last_undegraded":"2023-12-19T18:16:42.358258+0000","last_fullsized":"2023-12-19T18:16:42.358258+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:02:40.928028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.555138+0000","last_change":"2023-12-19T18:16:40.573382+0000","last_active":"2023-12-19T18:16:41.555138+0000","last_peered":"2023-12-19T18:16:41.555138+0000","last_clean":"2023-12-19T18:16:41.555138+0000","last_became_active":"2023-12-19T18:16:40.573092+0000","last_became_peered":"2023-12-19T18:16:40.573092+0000","last_unstale":"2023-12-19T18:16:41.555138+0000","last_undegraded":"2023-12-19T18:16:41.555138+0000","last_fullsized":"2023-12-19T18:16:41.555138+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:10:47.778609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553461+0000","last_change":"2023-12-19T18:16:40.163131+0000","last_active":"2023-12-19T18:16:41.553461+0000","last_peered":"2023-12-19T18:16:41.553461+0000","last_clean":"2023-12-19T18:16:41.553461+0000","last_became_active":"2023-12-19T18:16:40.162621+0000","last_became_peered":"2023-12-19T18:16:40.162621+0000","last_unstale":"2023-12-19T18:16:41.553461+0000","last_undegraded":"2023-12-19T18:16:41.553461+0000","last_fullsized":"2023-12-19T18:16:41.553461+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:57.979757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559871+0000","last_change":"2023-12-19T18:16:27.409441+0000","last_active":"2023-12-19T18:16:40.559871+0000","last_peered":"2023-12-19T18:16:40.559871+0000","last_clean":"2023-12-19T18:16:40.559871+0000","last_became_active":"2023-12-19T18:16:27.408438+0000","last_became_peered":"2023-12-19T18:16:27.408438+0000","last_unstale":"2023-12-19T18:16:40.559871+0000","last_undegraded":"2023-12-19T18:16:40.559871+0000","last_fullsized":"2023-12-19T18:16:40.559871+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:40.980300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546499+0000","last_change":"2023-12-19T18:16:27.814063+0000","last_active":"2023-12-19T18:16:40.546499+0000","last_peered":"2023-12-19T18:16:40.546499+0000","last_clean":"2023-12-19T18:16:40.546499+0000","last_became_active":"2023-12-19T18:16:27.813564+0000","last_became_peered":"2023-12-19T18:16:27.813564+0000","last_unstale":"2023-12-19T18:16:40.546499+0000","last_undegraded":"2023-12-19T18:16:40.546499+0000","last_fullsized":"2023-12-19T18:16:40.546499+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:36.733148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553348+0000","last_change":"2023-12-19T18:16:40.564402+0000","last_active":"2023-12-19T18:16:41.553348+0000","last_peered":"2023-12-19T18:16:41.553348+0000","last_clean":"2023-12-19T18:16:41.553348+0000","last_became_active":"2023-12-19T18:16:40.561338+0000","last_became_peered":"2023-12-19T18:16:40.561338+0000","last_unstale":"2023-12-19T18:16:41.553348+0000","last_undegraded":"2023-12-19T18:16:41.553348+0000","last_fullsized":"2023-12-19T18:16:41.553348+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:37:11.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"215'4","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857590+0000","last_change":"2023-12-19T18:16:40.152799+0000","last_active":"2023-12-19T18:16:41.857590+0000","last_peered":"2023-12-19T18:16:41.857590+0000","last_clean":"2023-12-19T18:16:41.857590+0000","last_became_active":"2023-12-19T18:16:40.152482+0000","last_became_peered":"2023-12-19T18:16:40.152482+0000","last_unstale":"2023-12-19T18:16:41.857590+0000","last_undegraded":"2023-12-19T18:16:41.857590+0000","last_fullsized":"2023-12-19T18:16:41.857590+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:23.854189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1d","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548666+0000","last_change":"2023-12-19T18:16:27.415013+0000","last_active":"2023-12-19T18:16:40.548666+0000","last_peered":"2023-12-19T18:16:40.548666+0000","last_clean":"2023-12-19T18:16:40.548666+0000","last_became_active":"2023-12-19T18:16:27.414582+0000","last_became_peered":"2023-12-19T18:16:27.414582+0000","last_unstale":"2023-12-19T18:16:40.548666+0000","last_undegraded":"2023-12-19T18:16:40.548666+0000","last_fullsized":"2023-12-19T18:16:40.548666+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:34:01.457288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562417+0000","last_change":"2023-12-19T18:16:27.829031+0000","last_active":"2023-12-19T18:16:40.562417+0000","last_peered":"2023-12-19T18:16:40.562417+0000","last_clean":"2023-12-19T18:16:40.562417+0000","last_became_active":"2023-12-19T18:16:27.826634+0000","last_became_peered":"2023-12-19T18:16:27.826634+0000","last_unstale":"2023-12-19T18:16:40.562417+0000","last_undegraded":"2023-12-19T18:16:40.562417+0000","last_fullsized":"2023-12-19T18:16:40.562417+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:23.429228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551066+0000","last_change":"2023-12-19T18:16:40.570487+0000","last_active":"2023-12-19T18:16:41.551066+0000","last_peered":"2023-12-19T18:16:41.551066+0000","last_clean":"2023-12-19T18:16:41.551066+0000","last_became_active":"2023-12-19T18:16:40.569839+0000","last_became_peered":"2023-12-19T18:16:40.569839+0000","last_unstale":"2023-12-19T18:16:41.551066+0000","last_undegraded":"2023-12-19T18:16:41.551066+0000","last_fullsized":"2023-12-19T18:16:41.551066+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:59.414695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548366+0000","last_change":"2023-12-19T18:16:27.407669+0000","last_active":"2023-12-19T18:16:40.548366+0000","last_peered":"2023-12-19T18:16:40.548366+0000","last_clean":"2023-12-19T18:16:40.548366+0000","last_became_active":"2023-12-19T18:16:27.407150+0000","last_became_peered":"2023-12-19T18:16:27.407150+0000","last_unstale":"2023-12-19T18:16:40.548366+0000","last_undegraded":"2023-12-19T18:16:40.548366+0000","last_fullsized":"2023-12-19T18:16:40.548366+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:59.675090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558634+0000","last_change":"2023-12-19T18:16:27.830766+0000","last_active":"2023-12-19T18:16:40.558634+0000","last_peered":"2023-12-19T18:16:40.558634+0000","last_clean":"2023-12-19T18:16:40.558634+0000","last_became_active":"2023-12-19T18:16:27.830220+0000","last_became_peered":"2023-12-19T18:16:27.830220+0000","last_unstale":"2023-12-19T18:16:40.558634+0000","last_undegraded":"2023-12-19T18:16:40.558634+0000","last_fullsized":"2023-12-19T18:16:40.558634+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:03:17.772391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1c","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.075108+0000","last_change":"2023-12-19T18:16:40.164718+0000","last_active":"2023-12-19T18:16:42.075108+0000","last_peered":"2023-12-19T18:16:42.075108+0000","last_clean":"2023-12-19T18:16:42.075108+0000","last_became_active":"2023-12-19T18:16:40.164413+0000","last_became_peered":"2023-12-19T18:16:40.164413+0000","last_unstale":"2023-12-19T18:16:42.075108+0000","last_undegraded":"2023-12-19T18:16:42.075108+0000","last_fullsized":"2023-12-19T18:16:42.075108+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:49.453705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359497+0000","last_change":"2023-12-19T18:16:40.565564+0000","last_active":"2023-12-19T18:16:42.359497+0000","last_peered":"2023-12-19T18:16:42.359497+0000","last_clean":"2023-12-19T18:16:42.359497+0000","last_became_active":"2023-12-19T18:16:40.565300+0000","last_became_peered":"2023-12-19T18:16:40.565300+0000","last_unstale":"2023-12-19T18:16:42.359497+0000","last_undegraded":"2023-12-19T18:16:42.359497+0000","last_fullsized":"2023-12-19T18:16:42.359497+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:14.849896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1f","version":"211'4","reported_seq":19,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553798+0000","last_change":"2023-12-19T18:16:27.414364+0000","last_active":"2023-12-19T18:16:40.553798+0000","last_peered":"2023-12-19T18:16:40.553798+0000","last_clean":"2023-12-19T18:16:40.553798+0000","last_became_active":"2023-12-19T18:16:27.413648+0000","last_became_peered":"2023-12-19T18:16:27.413648+0000","last_unstale":"2023-12-19T18:16:40.553798+0000","last_undegraded":"2023-12-19T18:16:40.553798+0000","last_fullsized":"2023-12-19T18:16:40.553798+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:27:32.703908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544529+0000","last_change":"2023-12-19T18:16:27.830378+0000","last_active":"2023-12-19T18:16:40.544529+0000","last_peered":"2023-12-19T18:16:40.544529+0000","last_clean":"2023-12-19T18:16:40.544529+0000","last_became_active":"2023-12-19T18:16:27.830122+0000","last_became_peered":"2023-12-19T18:16:27.830122+0000","last_unstale":"2023-12-19T18:16:40.544529+0000","last_undegraded":"2023-12-19T18:16:40.544529+0000","last_fullsized":"2023-12-19T18:16:40.544529+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:21.365516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1d","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358451+0000","last_change":"2023-12-19T18:16:40.162566+0000","last_active":"2023-12-19T18:16:42.358451+0000","last_peered":"2023-12-19T18:16:42.358451+0000","last_clean":"2023-12-19T18:16:42.358451+0000","last_became_active":"2023-12-19T18:16:40.162255+0000","last_became_peered":"2023-12-19T18:16:40.162255+0000","last_unstale":"2023-12-19T18:16:42.358451+0000","last_undegraded":"2023-12-19T18:16:42.358451+0000","last_fullsized":"2023-12-19T18:16:42.358451+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:20.530183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552668+0000","last_change":"2023-12-19T18:16:40.570029+0000","last_active":"2023-12-19T18:16:41.552668+0000","last_peered":"2023-12-19T18:16:41.552668+0000","last_clean":"2023-12-19T18:16:41.552668+0000","last_became_active":"2023-12-19T18:16:40.569731+0000","last_became_peered":"2023-12-19T18:16:40.569731+0000","last_unstale":"2023-12-19T18:16:41.552668+0000","last_undegraded":"2023-12-19T18:16:41.552668+0000","last_fullsized":"2023-12-19T18:16:41.552668+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:25:46.635350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550877+0000","last_change":"2023-12-19T18:16:27.831030+0000","last_active":"2023-12-19T18:16:40.550877+0000","last_peered":"2023-12-19T18:16:40.550877+0000","last_clean":"2023-12-19T18:16:40.550877+0000","last_became_active":"2023-12-19T18:16:27.830764+0000","last_became_peered":"2023-12-19T18:16:27.830764+0000","last_unstale":"2023-12-19T18:16:40.550877+0000","last_undegraded":"2023-12-19T18:16:40.550877+0000","last_fullsized":"2023-12-19T18:16:40.550877+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:11.685663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857085+0000","last_change":"2023-12-19T18:16:40.569158+0000","last_active":"2023-12-19T18:16:41.857085+0000","last_peered":"2023-12-19T18:16:41.857085+0000","last_clean":"2023-12-19T18:16:41.857085+0000","last_became_active":"2023-12-19T18:16:40.568894+0000","last_became_peered":"2023-12-19T18:16:40.568894+0000","last_unstale":"2023-12-19T18:16:41.857085+0000","last_undegraded":"2023-12-19T18:16:41.857085+0000","last_fullsized":"2023-12-19T18:16:41.857085+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:02.018340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553603+0000","last_change":"2023-12-19T18:16:27.833518+0000","last_active":"2023-12-19T18:16:40.553603+0000","last_peered":"2023-12-19T18:16:40.553603+0000","last_clean":"2023-12-19T18:16:40.553603+0000","last_became_active":"2023-12-19T18:16:27.833251+0000","last_became_peered":"2023-12-19T18:16:27.833251+0000","last_unstale":"2023-12-19T18:16:40.553603+0000","last_undegraded":"2023-12-19T18:16:40.553603+0000","last_fullsized":"2023-12-19T18:16:40.553603+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:49.276917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357285+0000","last_change":"2023-12-19T18:16:40.563610+0000","last_active":"2023-12-19T18:16:42.357285+0000","last_peered":"2023-12-19T18:16:42.357285+0000","last_clean":"2023-12-19T18:16:42.357285+0000","last_became_active":"2023-12-19T18:16:40.563300+0000","last_became_peered":"2023-12-19T18:16:40.563300+0000","last_unstale":"2023-12-19T18:16:42.357285+0000","last_undegraded":"2023-12-19T18:16:42.357285+0000","last_fullsized":"2023-12-19T18:16:42.357285+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:28:48.518493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547548+0000","last_change":"2023-12-19T18:16:27.825014+0000","last_active":"2023-12-19T18:16:40.547548+0000","last_peered":"2023-12-19T18:16:40.547548+0000","last_clean":"2023-12-19T18:16:40.547548+0000","last_became_active":"2023-12-19T18:16:27.822140+0000","last_became_peered":"2023-12-19T18:16:27.822140+0000","last_unstale":"2023-12-19T18:16:40.547548+0000","last_undegraded":"2023-12-19T18:16:40.547548+0000","last_fullsized":"2023-12-19T18:16:40.547548+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:36.972809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550313+0000","last_change":"2023-12-19T18:16:40.576594+0000","last_active":"2023-12-19T18:16:41.550313+0000","last_peered":"2023-12-19T18:16:41.550313+0000","last_clean":"2023-12-19T18:16:41.550313+0000","last_became_active":"2023-12-19T18:16:40.576304+0000","last_became_peered":"2023-12-19T18:16:40.576304+0000","last_unstale":"2023-12-19T18:16:41.550313+0000","last_undegraded":"2023-12-19T18:16:41.550313+0000","last_fullsized":"2023-12-19T18:16:41.550313+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:58.373923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561436+0000","last_change":"2023-12-19T18:16:27.826500+0000","last_active":"2023-12-19T18:16:40.561436+0000","last_peered":"2023-12-19T18:16:40.561436+0000","last_clean":"2023-12-19T18:16:40.561436+0000","last_became_active":"2023-12-19T18:16:27.826216+0000","last_became_peered":"2023-12-19T18:16:27.826216+0000","last_unstale":"2023-12-19T18:16:40.561436+0000","last_undegraded":"2023-12-19T18:16:40.561436+0000","last_fullsized":"2023-12-19T18:16:40.561436+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:33.032362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359854+0000","last_change":"2023-12-19T18:16:40.565942+0000","last_active":"2023-12-19T18:16:42.359854+0000","last_peered":"2023-12-19T18:16:42.359854+0000","last_clean":"2023-12-19T18:16:42.359854+0000","last_became_active":"2023-12-19T18:16:40.565557+0000","last_became_peered":"2023-12-19T18:16:40.565557+0000","last_unstale":"2023-12-19T18:16:42.359854+0000","last_undegraded":"2023-12-19T18:16:42.359854+0000","last_fullsized":"2023-12-19T18:16:42.359854+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:23.839152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548841+0000","last_change":"2023-12-19T18:16:27.830103+0000","last_active":"2023-12-19T18:16:40.548841+0000","last_peered":"2023-12-19T18:16:40.548841+0000","last_clean":"2023-12-19T18:16:40.548841+0000","last_became_active":"2023-12-19T18:16:27.829444+0000","last_became_peered":"2023-12-19T18:16:27.829444+0000","last_unstale":"2023-12-19T18:16:40.548841+0000","last_undegraded":"2023-12-19T18:16:40.548841+0000","last_fullsized":"2023-12-19T18:16:40.548841+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:50:29.947932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553462+0000","last_change":"2023-12-19T18:16:40.563980+0000","last_active":"2023-12-19T18:16:41.553462+0000","last_peered":"2023-12-19T18:16:41.553462+0000","last_clean":"2023-12-19T18:16:41.553462+0000","last_became_active":"2023-12-19T18:16:40.563683+0000","last_became_peered":"2023-12-19T18:16:40.563683+0000","last_unstale":"2023-12-19T18:16:41.553462+0000","last_undegraded":"2023-12-19T18:16:41.553462+0000","last_fullsized":"2023-12-19T18:16:41.553462+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:36.170652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548463+0000","last_change":"2023-12-19T18:16:27.820913+0000","last_active":"2023-12-19T18:16:40.548463+0000","last_peered":"2023-12-19T18:16:40.548463+0000","last_clean":"2023-12-19T18:16:40.548463+0000","last_became_active":"2023-12-19T18:16:27.820634+0000","last_became_peered":"2023-12-19T18:16:27.820634+0000","last_unstale":"2023-12-19T18:16:40.548463+0000","last_undegraded":"2023-12-19T18:16:40.548463+0000","last_fullsized":"2023-12-19T18:16:40.548463+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:53:09.066279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357879+0000","last_change":"2023-12-19T18:16:40.562872+0000","last_active":"2023-12-19T18:16:42.357879+0000","last_peered":"2023-12-19T18:16:42.357879+0000","last_clean":"2023-12-19T18:16:42.357879+0000","last_became_active":"2023-12-19T18:16:40.562615+0000","last_became_peered":"2023-12-19T18:16:40.562615+0000","last_unstale":"2023-12-19T18:16:42.357879+0000","last_undegraded":"2023-12-19T18:16:42.357879+0000","last_fullsized":"2023-12-19T18:16:42.357879+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:54.454071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565322+0000","last_change":"2023-12-19T18:16:27.830119+0000","last_active":"2023-12-19T18:16:40.565322+0000","last_peered":"2023-12-19T18:16:40.565322+0000","last_clean":"2023-12-19T18:16:40.565322+0000","last_became_active":"2023-12-19T18:16:27.829850+0000","last_became_peered":"2023-12-19T18:16:27.829850+0000","last_unstale":"2023-12-19T18:16:40.565322+0000","last_undegraded":"2023-12-19T18:16:40.565322+0000","last_fullsized":"2023-12-19T18:16:40.565322+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:51.190615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857238+0000","last_change":"2023-12-19T18:16:40.570092+0000","last_active":"2023-12-19T18:16:41.857238+0000","last_peered":"2023-12-19T18:16:41.857238+0000","last_clean":"2023-12-19T18:16:41.857238+0000","last_became_active":"2023-12-19T18:16:40.568697+0000","last_became_peered":"2023-12-19T18:16:40.568697+0000","last_unstale":"2023-12-19T18:16:41.857238+0000","last_undegraded":"2023-12-19T18:16:41.857238+0000","last_fullsized":"2023-12-19T18:16:41.857238+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:11.448855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.564229+0000","last_change":"2023-12-19T18:16:27.834875+0000","last_active":"2023-12-19T18:16:40.564229+0000","last_peered":"2023-12-19T18:16:40.564229+0000","last_clean":"2023-12-19T18:16:40.564229+0000","last_became_active":"2023-12-19T18:16:27.834561+0000","last_became_peered":"2023-12-19T18:16:27.834561+0000","last_unstale":"2023-12-19T18:16:40.564229+0000","last_undegraded":"2023-12-19T18:16:40.564229+0000","last_fullsized":"2023-12-19T18:16:40.564229+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:30.301379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554348+0000","last_change":"2023-12-19T18:16:40.561527+0000","last_active":"2023-12-19T18:16:41.554348+0000","last_peered":"2023-12-19T18:16:41.554348+0000","last_clean":"2023-12-19T18:16:41.554348+0000","last_became_active":"2023-12-19T18:16:40.561251+0000","last_became_peered":"2023-12-19T18:16:40.561251+0000","last_unstale":"2023-12-19T18:16:41.554348+0000","last_undegraded":"2023-12-19T18:16:41.554348+0000","last_fullsized":"2023-12-19T18:16:41.554348+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:50.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543417+0000","last_change":"2023-12-19T18:16:27.826642+0000","last_active":"2023-12-19T18:16:40.543417+0000","last_peered":"2023-12-19T18:16:40.543417+0000","last_clean":"2023-12-19T18:16:40.543417+0000","last_became_active":"2023-12-19T18:16:27.825460+0000","last_became_peered":"2023-12-19T18:16:27.825460+0000","last_unstale":"2023-12-19T18:16:40.543417+0000","last_undegraded":"2023-12-19T18:16:40.543417+0000","last_fullsized":"2023-12-19T18:16:40.543417+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:05.671602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074707+0000","last_change":"2023-12-19T18:16:40.579582+0000","last_active":"2023-12-19T18:16:42.074707+0000","last_peered":"2023-12-19T18:16:42.074707+0000","last_clean":"2023-12-19T18:16:42.074707+0000","last_became_active":"2023-12-19T18:16:40.579319+0000","last_became_peered":"2023-12-19T18:16:40.579319+0000","last_unstale":"2023-12-19T18:16:42.074707+0000","last_undegraded":"2023-12-19T18:16:42.074707+0000","last_fullsized":"2023-12-19T18:16:42.074707+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:53:38.529823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551313+0000","last_change":"2023-12-19T18:16:27.826145+0000","last_active":"2023-12-19T18:16:40.551313+0000","last_peered":"2023-12-19T18:16:40.551313+0000","last_clean":"2023-12-19T18:16:40.551313+0000","last_became_active":"2023-12-19T18:16:27.825889+0000","last_became_peered":"2023-12-19T18:16:27.825889+0000","last_unstale":"2023-12-19T18:16:40.551313+0000","last_undegraded":"2023-12-19T18:16:40.551313+0000","last_fullsized":"2023-12-19T18:16:40.551313+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:59:41.673539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074764+0000","last_change":"2023-12-19T18:16:40.579381+0000","last_active":"2023-12-19T18:16:42.074764+0000","last_peered":"2023-12-19T18:16:42.074764+0000","last_clean":"2023-12-19T18:16:42.074764+0000","last_became_active":"2023-12-19T18:16:40.578655+0000","last_became_peered":"2023-12-19T18:16:40.578655+0000","last_unstale":"2023-12-19T18:16:42.074764+0000","last_undegraded":"2023-12-19T18:16:42.074764+0000","last_fullsized":"2023-12-19T18:16:42.074764+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:33:53.935941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555865+0000","last_change":"2023-12-19T18:16:27.830084+0000","last_active":"2023-12-19T18:16:40.555865+0000","last_peered":"2023-12-19T18:16:40.555865+0000","last_clean":"2023-12-19T18:16:40.555865+0000","last_became_active":"2023-12-19T18:16:27.829787+0000","last_became_peered":"2023-12-19T18:16:27.829787+0000","last_unstale":"2023-12-19T18:16:40.555865+0000","last_undegraded":"2023-12-19T18:16:40.555865+0000","last_fullsized":"2023-12-19T18:16:40.555865+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:34:16.356894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554193+0000","last_change":"2023-12-19T18:16:40.563996+0000","last_active":"2023-12-19T18:16:41.554193+0000","last_peered":"2023-12-19T18:16:41.554193+0000","last_clean":"2023-12-19T18:16:41.554193+0000","last_became_active":"2023-12-19T18:16:40.563686+0000","last_became_peered":"2023-12-19T18:16:40.563686+0000","last_unstale":"2023-12-19T18:16:41.554193+0000","last_undegraded":"2023-12-19T18:16:41.554193+0000","last_fullsized":"2023-12-19T18:16:41.554193+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:34.106940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551542+0000","last_change":"2023-12-19T18:16:27.831551+0000","last_active":"2023-12-19T18:16:40.551542+0000","last_peered":"2023-12-19T18:16:40.551542+0000","last_clean":"2023-12-19T18:16:40.551542+0000","last_became_active":"2023-12-19T18:16:27.831291+0000","last_became_peered":"2023-12-19T18:16:27.831291+0000","last_unstale":"2023-12-19T18:16:40.551542+0000","last_undegraded":"2023-12-19T18:16:40.551542+0000","last_fullsized":"2023-12-19T18:16:40.551542+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:09.914084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551186+0000","last_change":"2023-12-19T18:16:40.563764+0000","last_active":"2023-12-19T18:16:41.551186+0000","last_peered":"2023-12-19T18:16:41.551186+0000","last_clean":"2023-12-19T18:16:41.551186+0000","last_became_active":"2023-12-19T18:16:40.563464+0000","last_became_peered":"2023-12-19T18:16:40.563464+0000","last_unstale":"2023-12-19T18:16:41.551186+0000","last_undegraded":"2023-12-19T18:16:41.551186+0000","last_fullsized":"2023-12-19T18:16:41.551186+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:52:34.311679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548663+0000","last_change":"2023-12-19T18:16:27.833054+0000","last_active":"2023-12-19T18:16:40.548663+0000","last_peered":"2023-12-19T18:16:40.548663+0000","last_clean":"2023-12-19T18:16:40.548663+0000","last_became_active":"2023-12-19T18:16:27.832738+0000","last_became_peered":"2023-12-19T18:16:27.832738+0000","last_unstale":"2023-12-19T18:16:40.548663+0000","last_undegraded":"2023-12-19T18:16:40.548663+0000","last_fullsized":"2023-12-19T18:16:40.548663+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:57.892685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360068+0000","last_change":"2023-12-19T18:16:40.577515+0000","last_active":"2023-12-19T18:16:42.360068+0000","last_peered":"2023-12-19T18:16:42.360068+0000","last_clean":"2023-12-19T18:16:42.360068+0000","last_became_active":"2023-12-19T18:16:40.577096+0000","last_became_peered":"2023-12-19T18:16:40.577096+0000","last_unstale":"2023-12-19T18:16:42.360068+0000","last_undegraded":"2023-12-19T18:16:42.360068+0000","last_fullsized":"2023-12-19T18:16:42.360068+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:36.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555479+0000","last_change":"2023-12-19T18:16:27.820194+0000","last_active":"2023-12-19T18:16:40.555479+0000","last_peered":"2023-12-19T18:16:40.555479+0000","last_clean":"2023-12-19T18:16:40.555479+0000","last_became_active":"2023-12-19T18:16:27.819900+0000","last_became_peered":"2023-12-19T18:16:27.819900+0000","last_unstale":"2023-12-19T18:16:40.555479+0000","last_undegraded":"2023-12-19T18:16:40.555479+0000","last_fullsized":"2023-12-19T18:16:40.555479+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:00.858729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360137+0000","last_change":"2023-12-19T18:16:40.561878+0000","last_active":"2023-12-19T18:16:42.360137+0000","last_peered":"2023-12-19T18:16:42.360137+0000","last_clean":"2023-12-19T18:16:42.360137+0000","last_became_active":"2023-12-19T18:16:40.561626+0000","last_became_peered":"2023-12-19T18:16:40.561626+0000","last_unstale":"2023-12-19T18:16:42.360137+0000","last_undegraded":"2023-12-19T18:16:42.360137+0000","last_fullsized":"2023-12-19T18:16:42.360137+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:38:57.938991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562034+0000","last_change":"2023-12-19T18:16:27.820781+0000","last_active":"2023-12-19T18:16:40.562034+0000","last_peered":"2023-12-19T18:16:40.562034+0000","last_clean":"2023-12-19T18:16:40.562034+0000","last_became_active":"2023-12-19T18:16:27.820483+0000","last_became_peered":"2023-12-19T18:16:27.820483+0000","last_unstale":"2023-12-19T18:16:40.562034+0000","last_undegraded":"2023-12-19T18:16:40.562034+0000","last_fullsized":"2023-12-19T18:16:40.562034+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:23.445435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551324+0000","last_change":"2023-12-19T18:16:40.560485+0000","last_active":"2023-12-19T18:16:41.551324+0000","last_peered":"2023-12-19T18:16:41.551324+0000","last_clean":"2023-12-19T18:16:41.551324+0000","last_became_active":"2023-12-19T18:16:40.560232+0000","last_became_peered":"2023-12-19T18:16:40.560232+0000","last_unstale":"2023-12-19T18:16:41.551324+0000","last_undegraded":"2023-12-19T18:16:41.551324+0000","last_fullsized":"2023-12-19T18:16:41.551324+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:30.252976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553708+0000","last_change":"2023-12-19T18:16:27.834880+0000","last_active":"2023-12-19T18:16:40.553708+0000","last_peered":"2023-12-19T18:16:40.553708+0000","last_clean":"2023-12-19T18:16:40.553708+0000","last_became_active":"2023-12-19T18:16:27.834494+0000","last_became_peered":"2023-12-19T18:16:27.834494+0000","last_unstale":"2023-12-19T18:16:40.553708+0000","last_undegraded":"2023-12-19T18:16:40.553708+0000","last_fullsized":"2023-12-19T18:16:40.553708+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:06.992701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357587+0000","last_change":"2023-12-19T18:16:40.567126+0000","last_active":"2023-12-19T18:16:42.357587+0000","last_peered":"2023-12-19T18:16:42.357587+0000","last_clean":"2023-12-19T18:16:42.357587+0000","last_became_active":"2023-12-19T18:16:40.566878+0000","last_became_peered":"2023-12-19T18:16:40.566878+0000","last_unstale":"2023-12-19T18:16:42.357587+0000","last_undegraded":"2023-12-19T18:16:42.357587+0000","last_fullsized":"2023-12-19T18:16:42.357587+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:59.062853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543300+0000","last_change":"2023-12-19T18:16:27.826828+0000","last_active":"2023-12-19T18:16:40.543300+0000","last_peered":"2023-12-19T18:16:40.543300+0000","last_clean":"2023-12-19T18:16:40.543300+0000","last_became_active":"2023-12-19T18:16:27.825994+0000","last_became_peered":"2023-12-19T18:16:27.825994+0000","last_unstale":"2023-12-19T18:16:40.543300+0000","last_undegraded":"2023-12-19T18:16:40.543300+0000","last_fullsized":"2023-12-19T18:16:40.543300+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:17.748218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857358+0000","last_change":"2023-12-19T18:16:40.569008+0000","last_active":"2023-12-19T18:16:41.857358+0000","last_peered":"2023-12-19T18:16:41.857358+0000","last_clean":"2023-12-19T18:16:41.857358+0000","last_became_active":"2023-12-19T18:16:40.568420+0000","last_became_peered":"2023-12-19T18:16:40.568420+0000","last_unstale":"2023-12-19T18:16:41.857358+0000","last_undegraded":"2023-12-19T18:16:41.857358+0000","last_fullsized":"2023-12-19T18:16:41.857358+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:31:10.720873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561285+0000","last_change":"2023-12-19T18:16:27.830751+0000","last_active":"2023-12-19T18:16:40.561285+0000","last_peered":"2023-12-19T18:16:40.561285+0000","last_clean":"2023-12-19T18:16:40.561285+0000","last_became_active":"2023-12-19T18:16:27.830490+0000","last_became_peered":"2023-12-19T18:16:27.830490+0000","last_unstale":"2023-12-19T18:16:40.561285+0000","last_undegraded":"2023-12-19T18:16:40.561285+0000","last_fullsized":"2023-12-19T18:16:40.561285+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:31.093343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550245+0000","last_change":"2023-12-19T18:16:40.571696+0000","last_active":"2023-12-19T18:16:41.550245+0000","last_peered":"2023-12-19T18:16:41.550245+0000","last_clean":"2023-12-19T18:16:41.550245+0000","last_became_active":"2023-12-19T18:16:40.571407+0000","last_became_peered":"2023-12-19T18:16:40.571407+0000","last_unstale":"2023-12-19T18:16:41.550245+0000","last_undegraded":"2023-12-19T18:16:41.550245+0000","last_fullsized":"2023-12-19T18:16:41.550245+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:55.222921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548271+0000","last_change":"2023-12-19T18:16:27.823186+0000","last_active":"2023-12-19T18:16:40.548271+0000","last_peered":"2023-12-19T18:16:40.548271+0000","last_clean":"2023-12-19T18:16:40.548271+0000","last_became_active":"2023-12-19T18:16:27.820405+0000","last_became_peered":"2023-12-19T18:16:27.820405+0000","last_unstale":"2023-12-19T18:16:40.548271+0000","last_undegraded":"2023-12-19T18:16:40.548271+0000","last_fullsized":"2023-12-19T18:16:40.548271+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:41.515700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357953+0000","last_change":"2023-12-19T18:16:40.569297+0000","last_active":"2023-12-19T18:16:42.357953+0000","last_peered":"2023-12-19T18:16:42.357953+0000","last_clean":"2023-12-19T18:16:42.357953+0000","last_became_active":"2023-12-19T18:16:40.569050+0000","last_became_peered":"2023-12-19T18:16:40.569050+0000","last_unstale":"2023-12-19T18:16:42.357953+0000","last_undegraded":"2023-12-19T18:16:42.357953+0000","last_fullsized":"2023-12-19T18:16:42.357953+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:28:56.831758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561350+0000","last_change":"2023-12-19T18:16:27.829736+0000","last_active":"2023-12-19T18:16:40.561350+0000","last_peered":"2023-12-19T18:16:40.561350+0000","last_clean":"2023-12-19T18:16:40.561350+0000","last_became_active":"2023-12-19T18:16:27.829448+0000","last_became_peered":"2023-12-19T18:16:27.829448+0000","last_unstale":"2023-12-19T18:16:40.561350+0000","last_undegraded":"2023-12-19T18:16:40.561350+0000","last_fullsized":"2023-12-19T18:16:40.561350+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:47.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551154+0000","last_change":"2023-12-19T18:16:40.559877+0000","last_active":"2023-12-19T18:16:41.551154+0000","last_peered":"2023-12-19T18:16:41.551154+0000","last_clean":"2023-12-19T18:16:41.551154+0000","last_became_active":"2023-12-19T18:16:40.558100+0000","last_became_peered":"2023-12-19T18:16:40.558100+0000","last_unstale":"2023-12-19T18:16:41.551154+0000","last_undegraded":"2023-12-19T18:16:41.551154+0000","last_fullsized":"2023-12-19T18:16:41.551154+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:44.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559776+0000","last_change":"2023-12-19T18:16:27.829943+0000","last_active":"2023-12-19T18:16:40.559776+0000","last_peered":"2023-12-19T18:16:40.559776+0000","last_clean":"2023-12-19T18:16:40.559776+0000","last_became_active":"2023-12-19T18:16:27.829592+0000","last_became_peered":"2023-12-19T18:16:27.829592+0000","last_unstale":"2023-12-19T18:16:40.559776+0000","last_undegraded":"2023-12-19T18:16:40.559776+0000","last_fullsized":"2023-12-19T18:16:40.559776+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:46.245564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553575+0000","last_change":"2023-12-19T18:16:40.575384+0000","last_active":"2023-12-19T18:16:41.553575+0000","last_peered":"2023-12-19T18:16:41.553575+0000","last_clean":"2023-12-19T18:16:41.553575+0000","last_became_active":"2023-12-19T18:16:40.575124+0000","last_became_peered":"2023-12-19T18:16:40.575124+0000","last_unstale":"2023-12-19T18:16:41.553575+0000","last_undegraded":"2023-12-19T18:16:41.553575+0000","last_fullsized":"2023-12-19T18:16:41.553575+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:53.796369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565426+0000","last_change":"2023-12-19T18:16:27.819797+0000","last_active":"2023-12-19T18:16:40.565426+0000","last_peered":"2023-12-19T18:16:40.565426+0000","last_clean":"2023-12-19T18:16:40.565426+0000","last_became_active":"2023-12-19T18:16:27.819507+0000","last_became_peered":"2023-12-19T18:16:27.819507+0000","last_unstale":"2023-12-19T18:16:40.565426+0000","last_undegraded":"2023-12-19T18:16:40.565426+0000","last_fullsized":"2023-12-19T18:16:40.565426+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:08.287716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857264+0000","last_change":"2023-12-19T18:16:40.570173+0000","last_active":"2023-12-19T18:16:41.857264+0000","last_peered":"2023-12-19T18:16:41.857264+0000","last_clean":"2023-12-19T18:16:41.857264+0000","last_became_active":"2023-12-19T18:16:40.569912+0000","last_became_peered":"2023-12-19T18:16:40.569912+0000","last_unstale":"2023-12-19T18:16:41.857264+0000","last_undegraded":"2023-12-19T18:16:41.857264+0000","last_fullsized":"2023-12-19T18:16:41.857264+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:47.688949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558633+0000","last_change":"2023-12-19T18:16:27.819945+0000","last_active":"2023-12-19T18:16:40.558633+0000","last_peered":"2023-12-19T18:16:40.558633+0000","last_clean":"2023-12-19T18:16:40.558633+0000","last_became_active":"2023-12-19T18:16:27.819634+0000","last_became_peered":"2023-12-19T18:16:27.819634+0000","last_unstale":"2023-12-19T18:16:40.558633+0000","last_undegraded":"2023-12-19T18:16:40.558633+0000","last_fullsized":"2023-12-19T18:16:40.558633+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:22.401852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358420+0000","last_change":"2023-12-19T18:16:40.566307+0000","last_active":"2023-12-19T18:16:42.358420+0000","last_peered":"2023-12-19T18:16:42.358420+0000","last_clean":"2023-12-19T18:16:42.358420+0000","last_became_active":"2023-12-19T18:16:40.566059+0000","last_became_peered":"2023-12-19T18:16:40.566059+0000","last_unstale":"2023-12-19T18:16:42.358420+0000","last_undegraded":"2023-12-19T18:16:42.358420+0000","last_fullsized":"2023-12-19T18:16:42.358420+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:45.773786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549949+0000","last_change":"2023-12-19T18:16:27.832580+0000","last_active":"2023-12-19T18:16:40.549949+0000","last_peered":"2023-12-19T18:16:40.549949+0000","last_clean":"2023-12-19T18:16:40.549949+0000","last_became_active":"2023-12-19T18:16:27.832291+0000","last_became_peered":"2023-12-19T18:16:27.832291+0000","last_unstale":"2023-12-19T18:16:40.549949+0000","last_undegraded":"2023-12-19T18:16:40.549949+0000","last_fullsized":"2023-12-19T18:16:40.549949+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:29.305405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359160+0000","last_change":"2023-12-19T18:16:40.567943+0000","last_active":"2023-12-19T18:16:42.359160+0000","last_peered":"2023-12-19T18:16:42.359160+0000","last_clean":"2023-12-19T18:16:42.359160+0000","last_became_active":"2023-12-19T18:16:40.567701+0000","last_became_peered":"2023-12-19T18:16:40.567701+0000","last_unstale":"2023-12-19T18:16:42.359160+0000","last_undegraded":"2023-12-19T18:16:42.359160+0000","last_fullsized":"2023-12-19T18:16:42.359160+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:50:31.894613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543104+0000","last_change":"2023-12-19T18:16:27.825730+0000","last_active":"2023-12-19T18:16:40.543104+0000","last_peered":"2023-12-19T18:16:40.543104+0000","last_clean":"2023-12-19T18:16:40.543104+0000","last_became_active":"2023-12-19T18:16:27.825136+0000","last_became_peered":"2023-12-19T18:16:27.825136+0000","last_unstale":"2023-12-19T18:16:40.543104+0000","last_undegraded":"2023-12-19T18:16:40.543104+0000","last_fullsized":"2023-12-19T18:16:40.543104+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:48.330735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359437+0000","last_change":"2023-12-19T18:16:40.558188+0000","last_active":"2023-12-19T18:16:42.359437+0000","last_peered":"2023-12-19T18:16:42.359437+0000","last_clean":"2023-12-19T18:16:42.359437+0000","last_became_active":"2023-12-19T18:16:40.557804+0000","last_became_peered":"2023-12-19T18:16:40.557804+0000","last_unstale":"2023-12-19T18:16:42.359437+0000","last_undegraded":"2023-12-19T18:16:42.359437+0000","last_fullsized":"2023-12-19T18:16:42.359437+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:35:20.500418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558018+0000","last_change":"2023-12-19T18:16:27.823852+0000","last_active":"2023-12-19T18:16:40.558018+0000","last_peered":"2023-12-19T18:16:40.558018+0000","last_clean":"2023-12-19T18:16:40.558018+0000","last_became_active":"2023-12-19T18:16:27.823564+0000","last_became_peered":"2023-12-19T18:16:27.823564+0000","last_unstale":"2023-12-19T18:16:40.558018+0000","last_undegraded":"2023-12-19T18:16:40.558018+0000","last_fullsized":"2023-12-19T18:16:40.558018+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:59.000440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358756+0000","last_change":"2023-12-19T18:16:40.561015+0000","last_active":"2023-12-19T18:16:42.358756+0000","last_peered":"2023-12-19T18:16:42.358756+0000","last_clean":"2023-12-19T18:16:42.358756+0000","last_became_active":"2023-12-19T18:16:40.560771+0000","last_became_peered":"2023-12-19T18:16:40.560771+0000","last_unstale":"2023-12-19T18:16:42.358756+0000","last_undegraded":"2023-12-19T18:16:42.358756+0000","last_fullsized":"2023-12-19T18:16:42.358756+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:04.628464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554625+0000","last_change":"2023-12-19T18:16:27.822167+0000","last_active":"2023-12-19T18:16:40.554625+0000","last_peered":"2023-12-19T18:16:40.554625+0000","last_clean":"2023-12-19T18:16:40.554625+0000","last_became_active":"2023-12-19T18:16:27.821752+0000","last_became_peered":"2023-12-19T18:16:27.821752+0000","last_unstale":"2023-12-19T18:16:40.554625+0000","last_undegraded":"2023-12-19T18:16:40.554625+0000","last_fullsized":"2023-12-19T18:16:40.554625+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:17.866181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357777+0000","last_change":"2023-12-19T18:16:40.565973+0000","last_active":"2023-12-19T18:16:42.357777+0000","last_peered":"2023-12-19T18:16:42.357777+0000","last_clean":"2023-12-19T18:16:42.357777+0000","last_became_active":"2023-12-19T18:16:40.565418+0000","last_became_peered":"2023-12-19T18:16:40.565418+0000","last_unstale":"2023-12-19T18:16:42.357777+0000","last_undegraded":"2023-12-19T18:16:42.357777+0000","last_fullsized":"2023-12-19T18:16:42.357777+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:54.473566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551716+0000","last_change":"2023-12-19T18:16:27.835673+0000","last_active":"2023-12-19T18:16:40.551716+0000","last_peered":"2023-12-19T18:16:40.551716+0000","last_clean":"2023-12-19T18:16:40.551716+0000","last_became_active":"2023-12-19T18:16:27.835392+0000","last_became_peered":"2023-12-19T18:16:27.835392+0000","last_unstale":"2023-12-19T18:16:40.551716+0000","last_undegraded":"2023-12-19T18:16:40.551716+0000","last_fullsized":"2023-12-19T18:16:40.551716+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:23:06.990327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553304+0000","last_change":"2023-12-19T18:16:40.564946+0000","last_active":"2023-12-19T18:16:41.553304+0000","last_peered":"2023-12-19T18:16:41.553304+0000","last_clean":"2023-12-19T18:16:41.553304+0000","last_became_active":"2023-12-19T18:16:40.564679+0000","last_became_peered":"2023-12-19T18:16:40.564679+0000","last_unstale":"2023-12-19T18:16:41.553304+0000","last_undegraded":"2023-12-19T18:16:41.553304+0000","last_fullsized":"2023-12-19T18:16:41.553304+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:02:28.048063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559694+0000","last_change":"2023-12-19T18:16:27.830449+0000","last_active":"2023-12-19T18:16:40.559694+0000","last_peered":"2023-12-19T18:16:40.559694+0000","last_clean":"2023-12-19T18:16:40.559694+0000","last_became_active":"2023-12-19T18:16:27.830189+0000","last_became_peered":"2023-12-19T18:16:27.830189+0000","last_unstale":"2023-12-19T18:16:40.559694+0000","last_undegraded":"2023-12-19T18:16:40.559694+0000","last_fullsized":"2023-12-19T18:16:40.559694+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:06.397912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360333+0000","last_change":"2023-12-19T18:16:40.576744+0000","last_active":"2023-12-19T18:16:42.360333+0000","last_peered":"2023-12-19T18:16:42.360333+0000","last_clean":"2023-12-19T18:16:42.360333+0000","last_became_active":"2023-12-19T18:16:40.576425+0000","last_became_peered":"2023-12-19T18:16:40.576425+0000","last_unstale":"2023-12-19T18:16:42.360333+0000","last_undegraded":"2023-12-19T18:16:42.360333+0000","last_fullsized":"2023-12-19T18:16:42.360333+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:26.259551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548052+0000","last_change":"2023-12-19T18:16:27.815730+0000","last_active":"2023-12-19T18:16:40.548052+0000","last_peered":"2023-12-19T18:16:40.548052+0000","last_clean":"2023-12-19T18:16:40.548052+0000","last_became_active":"2023-12-19T18:16:27.815341+0000","last_became_peered":"2023-12-19T18:16:27.815341+0000","last_unstale":"2023-12-19T18:16:40.548052+0000","last_undegraded":"2023-12-19T18:16:40.548052+0000","last_fullsized":"2023-12-19T18:16:40.548052+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:39.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074872+0000","last_change":"2023-12-19T18:16:40.578780+0000","last_active":"2023-12-19T18:16:42.074872+0000","last_peered":"2023-12-19T18:16:42.074872+0000","last_clean":"2023-12-19T18:16:42.074872+0000","last_became_active":"2023-12-19T18:16:40.578426+0000","last_became_peered":"2023-12-19T18:16:40.578426+0000","last_unstale":"2023-12-19T18:16:42.074872+0000","last_undegraded":"2023-12-19T18:16:42.074872+0000","last_fullsized":"2023-12-19T18:16:42.074872+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:53:39.733238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562588+0000","last_change":"2023-12-19T18:16:27.831511+0000","last_active":"2023-12-19T18:16:40.562588+0000","last_peered":"2023-12-19T18:16:40.562588+0000","last_clean":"2023-12-19T18:16:40.562588+0000","last_became_active":"2023-12-19T18:16:27.831255+0000","last_became_peered":"2023-12-19T18:16:27.831255+0000","last_unstale":"2023-12-19T18:16:40.562588+0000","last_undegraded":"2023-12-19T18:16:40.562588+0000","last_fullsized":"2023-12-19T18:16:40.562588+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:42.382254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358249+0000","last_change":"2023-12-19T18:16:40.576919+0000","last_active":"2023-12-19T18:16:42.358249+0000","last_peered":"2023-12-19T18:16:42.358249+0000","last_clean":"2023-12-19T18:16:42.358249+0000","last_became_active":"2023-12-19T18:16:40.576351+0000","last_became_peered":"2023-12-19T18:16:40.576351+0000","last_unstale":"2023-12-19T18:16:42.358249+0000","last_undegraded":"2023-12-19T18:16:42.358249+0000","last_fullsized":"2023-12-19T18:16:42.358249+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:36.351083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562201+0000","last_change":"2023-12-19T18:16:27.834221+0000","last_active":"2023-12-19T18:16:40.562201+0000","last_peered":"2023-12-19T18:16:40.562201+0000","last_clean":"2023-12-19T18:16:40.562201+0000","last_became_active":"2023-12-19T18:16:27.825886+0000","last_became_peered":"2023-12-19T18:16:27.825886+0000","last_unstale":"2023-12-19T18:16:40.562201+0000","last_undegraded":"2023-12-19T18:16:40.562201+0000","last_fullsized":"2023-12-19T18:16:40.562201+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:12.119996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510569,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64616,"kb_used_data":3232,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307224,"statfs":{"total":96636764160,"available":96570597376,"internally_reserved":0,"allocated":3309568,"data_stored":2022904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96399999999999997,"5min":1.048,"15min":1.1180000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92400000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63400000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92400000000000004,"5min":1.0389999999999999,"15min":1.0580000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72399999999999998,"15min":0.748},"min":{"1min":0.499,"5min":0.48099999999999998,"15min":0.435},"max":{"1min":0.89500000000000002,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.67700000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.38900000000000001},"max":{"1min":0.93400000000000005,"5min":1.026,"15min":1.0669999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.0820000000000001,"15min":1.236},"last":0.57899999999999996}]},{"osd":3,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.75600000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88300000000000001,"5min":1.002,"15min":1.079},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98899999999999999,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.66000000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81599999999999995,"5min":1.0009999999999999,"15min":1.302},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.871,"15min":1.0209999999999999},"last":0.82699999999999996}]},{"osd":5,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.52700000000000002,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.88100000000000001,"5min":1.097,"15min":1.1299999999999999},"last":0.622},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64900000000000002,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":1.026,"15min":1.0800000000000001},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.56999999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.93999999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85699999999999998,"5min":1.01,"15min":1.093},"last":0.79700000000000004}]}]},{"osd":6,"up_from":15,"seq":64424510569,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47636,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324204,"statfs":{"total":96636764160,"available":96587984896,"internally_reserved":0,"allocated":2699264,"data_stored":1431450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72799999999999998,"15min":0.74199999999999999},"min":{"1min":0.55400000000000005,"5min":0.46800000000000003,"15min":0.43099999999999999},"max":{"1min":0.92400000000000004,"5min":0.96499999999999997,"15min":1.042},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71799999999999997,"15min":0.73499999999999999},"min":{"1min":0.57499999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92000000000000004,"5min":1.224,"15min":1.224},"last":0.69299999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70099999999999996,"15min":0.72799999999999998},"min":{"1min":0.49099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98099999999999998,"5min":1.103,"15min":1.103},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.64100000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":0.94599999999999995,"5min":0.99199999999999999,"15min":1.1259999999999999},"last":0.76000000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.502},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0609999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.749},"min":{"1min":0.60899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83299999999999996,"5min":1.0089999999999999,"15min":1.111},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.755,"15min":0.78200000000000003},"min":{"1min":0.61799999999999999,"5min":0.495,"15min":0.495},"max":{"1min":0.91200000000000003,"5min":0.97999999999999998,"15min":1.1519999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.51800000000000002,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.98699999999999999,"5min":1.268,"15min":1.268},"last":0.72299999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92200000000000004,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":0.81299999999999994,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78600000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.93799999999999994,"5min":1.194,"15min":1.194},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":1.141,"15min":1.141},"last":0.95199999999999996}]},{"osd":7,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.95899999999999996,"5min":1.159,"15min":1.159},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94199999999999995,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.99199999999999999}]}]},{"osd":5,"up_from":15,"seq":64424510569,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47636,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324204,"statfs":{"total":96636764160,"available":96587984896,"internally_reserved":0,"allocated":2699264,"data_stored":1431450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72499999999999998,"15min":0.72999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34399999999999997},"max":{"1min":0.96899999999999997,"5min":1.0109999999999999,"15min":1.0149999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.40699999999999997},"max":{"1min":0.89200000000000002,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"min":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.497},"max":{"1min":0.95599999999999996,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.505,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78800000000000003,"15min":0.78100000000000003},"min":{"1min":0.61699999999999999,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":1.022,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94499999999999995,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.48499999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94999999999999996,"5min":1.05,"15min":1.05},"last":0.753},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.77400000000000002},"min":{"1min":0.56200000000000006,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98799999999999999,"5min":1.022,"15min":1.0329999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.79000000000000004,"5min":0.94499999999999995,"15min":1.0800000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41599999999999998},"max":{"1min":0.71099999999999997,"5min":1.208,"15min":1.208},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64800000000000002,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.94899999999999995,"15min":0.98999999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.65500000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.83499999999999996,"5min":1.008,"15min":1.0449999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.46500000000000002,"5min":0.439,"15min":0.436},"max":{"1min":0.77300000000000002,"5min":1.169,"15min":1.169},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71499999999999997,"15min":0.70999999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85699999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74199999999999999}]}]},{"osd":4,"up_from":15,"seq":64424510569,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47668,"kb_used_data":2668,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324172,"statfs":{"total":96636764160,"available":96587952128,"internally_reserved":0,"allocated":2732032,"data_stored":1435923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.51600000000000001,"5min":0.45800000000000002,"15min":0.45000000000000001},"max":{"1min":1.016,"5min":1.036,"15min":1.109},"last":0.62},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.502,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.91800000000000004,"5min":0.997,"15min":1.01},"last":0.72999999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.59199999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94699999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.997,"15min":1.1020000000000001},"last":0.68200000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79100000000000004,"15min":0.77700000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.47699999999999998},"max":{"1min":0.998,"5min":1.0289999999999999,"15min":1.071},"last":0.503},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.71699999999999997}]},{"osd":3,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76700000000000002,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93899999999999995,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.55500000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.66200000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65300000000000002,"15min":0.63100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.75},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67200000000000004,"15min":0.64600000000000002},"min":{"1min":0.495,"5min":0.42899999999999999,"15min":0.39600000000000002},"max":{"1min":0.88900000000000001,"5min":1.042,"15min":1.042},"last":0.52600000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.66100000000000003},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":1.024,"15min":1.0669999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0760000000000001},"last":0.78300000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.41899999999999998},"max":{"1min":0.92400000000000004,"5min":1.087,"15min":1.087},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77600000000000002,"15min":0.76200000000000001},"min":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.46999999999999997},"max":{"1min":0.98599999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70099999999999996}]}]},{"osd":3,"up_from":15,"seq":64424510569,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47656,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324184,"statfs":{"total":96636764160,"available":96587964416,"internally_reserved":0,"allocated":2719744,"data_stored":1431244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.63200000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74099999999999999,"5min":0.78300000000000003,"15min":0.80000000000000004},"last":0.504}]},{"osd":1,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.498},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71499999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.44}]},{"osd":2,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.77000000000000002,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.46000000000000002}]},{"osd":4,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81599999999999995,"15min":0.82799999999999996},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.012,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.81999999999999995,"15min":0.82299999999999995},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.56599999999999995},"max":{"1min":0.86499999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.65800000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82799999999999996,"15min":0.84199999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99099999999999999,"5min":1.1739999999999999,"15min":1.397},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.59399999999999997,"5min":0.55900000000000005,"15min":0.55100000000000005},"max":{"1min":0.91000000000000003,"5min":1.2230000000000001,"15min":1.262},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.83899999999999997,"15min":0.84799999999999998},"min":{"1min":0.60199999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.96099999999999997,"5min":1.234,"15min":1.274},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83799999999999997,"15min":0.84499999999999997},"min":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94599999999999995,"5min":1.165,"15min":1.302},"last":0.64500000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0640000000000001,"5min":1.3540000000000001,"15min":1.486},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89500000000000002},"min":{"1min":0.68899999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.93000000000000005,"5min":1.421,"15min":1.421},"last":0.626}]}]},{"osd":2,"up_from":15,"seq":64424510568,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47632,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324208,"statfs":{"total":96636764160,"available":96587988992,"internally_reserved":0,"allocated":2695168,"data_stored":1423635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.29299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.41499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.56000000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.86199999999999999,"15min":0.95999999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.36899999999999999},"max":{"1min":0.67200000000000004,"5min":0.79700000000000004,"15min":0.875},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.624,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.048,"5min":1.306,"15min":1.306},"last":0.753},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.05,"5min":1.272,"15min":1.466},"last":0.80700000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.64300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.97699999999999998,"5min":1.1459999999999999,"15min":1.3049999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.81999999999999995,"15min":0.82599999999999996},"min":{"1min":0.61899999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.429},"last":0.78400000000000003}]},{"osd":6,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.71199999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.216,"5min":1.365,"15min":1.4119999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80600000000000005,"15min":0.80800000000000005},"min":{"1min":0.625,"5min":0.55200000000000005,"15min":0.53500000000000003},"max":{"1min":0.94499999999999995,"5min":1.2,"15min":1.2470000000000001},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81499999999999995,"15min":0.82799999999999996},"min":{"1min":0.64400000000000002,"5min":0.59899999999999998,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.393,"15min":1.45},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85999999999999999,"15min":0.86099999999999999},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1850000000000001,"5min":1.276,"15min":1.276},"last":0.88600000000000001}]}]},{"osd":1,"up_from":15,"seq":64424510569,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59940,"kb_used_data":2652,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311900,"statfs":{"total":96636764160,"available":96575385600,"internally_reserved":0,"allocated":2715648,"data_stored":1433729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.33900000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63700000000000001,"5min":0.71999999999999997,"15min":0.89400000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.53300000000000003},"min":{"1min":0.32200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.32600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.505},{"interface":"front","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.45000000000000001,"15min":0.442},"max":{"1min":0.82699999999999996,"5min":0.94199999999999995,"15min":0.94499999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.65100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93400000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79100000000000004,"15min":0.81499999999999995},"min":{"1min":0.502,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0409999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.86599999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.78400000000000003,"15min":0.80500000000000005},"min":{"1min":0.49099999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97499999999999998,"5min":1.236,"15min":1.2609999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98399999999999999,"5min":1.353,"15min":1.353},"last":1.05}]},{"osd":6,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.60999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.008,"5min":1.2270000000000001,"15min":1.2689999999999999},"last":0.878},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.84299999999999997,"15min":0.84999999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.093,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":1.034}]},{"osd":7,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.64100000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.123,"5min":1.399,"15min":1.399},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.84299999999999997,"15min":0.85799999999999998},"min":{"1min":0.54100000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.105,"5min":1.365,"15min":1.365},"last":0.90100000000000002}]}]},{"osd":0,"up_from":14,"seq":60129543274,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60508,"kb_used_data":3220,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311332,"statfs":{"total":96636764160,"available":96574803968,"internally_reserved":0,"allocated":3297280,"data_stored":2021348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.438,"15min":0.45700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":1.216},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.317}]},{"osd":2,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64700000000000002,"5min":0.84999999999999998,"15min":1.1699999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72299999999999998,"5min":0.78900000000000003,"15min":0.96799999999999997},"last":0.442}]},{"osd":3,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.623,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.82699999999999996,"5min":0.871,"15min":1.032},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.63,"5min":0.51800000000000002,"15min":0.496},"max":{"1min":0.93100000000000005,"5min":1.1200000000000001,"15min":1.381},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.59199999999999997,"5min":0.56799999999999995,"15min":0.47799999999999998},"max":{"1min":1.02,"5min":1.137,"15min":1.2849999999999999},"last":0.752}]},{"osd":5,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80600000000000005,"15min":0.81899999999999995},"min":{"1min":0.57299999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.008,"5min":1.1930000000000001,"15min":1.3420000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.69599999999999995,"5min":0.59599999999999997,"15min":0.59499999999999997},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.288},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.51300000000000001},"max":{"1min":0.98899999999999999,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.83399999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0229999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.84999999999999998},"min":{"1min":0.65300000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.044,"5min":1.1899999999999999,"15min":1.415},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.83199999999999996,"15min":0.85599999999999998},"min":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0780000000000001,"5min":1.1379999999999999,"15min":1.395},"last":0.85599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:16:45.478 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-19T18:16:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0e142d640 1 Processor -- start 2023-12-19T18:16:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0e142d640 1 -- start start 2023-12-19T18:16:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0e142d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff0dc0cd840 0x7ff0dc0cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0e142d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0dc06b7c0 con 0x7ff0dc0afd70 2023-12-19T18:16:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0e142d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0dc0b3240 con 0x7ff0dc0cc770 2023-12-19T18:16:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0e142d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0dc161730 con 0x7ff0dc0cd840 2023-12-19T18:16:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0db7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff0dc0afd70 0x7ff0dc0d0350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41420/0 (socket says 172.21.15.26:41420) 2023-12-19T18:16:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0db7fe640 1 -- 172.21.15.26:0/1117878244 learned_addr learned my addr 172.21.15.26:0/1117878244 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:45.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0d3fff640 1 --2- 172.21.15.26:0/1117878244 >> v2:172.21.15.160:3300/0 conn(0x7ff0dc0cd840 0x7ff0dc0cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0da7fc640 1 -- 172.21.15.26:0/1117878244 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326677577 0 0) 0x7ff0dc06b7c0 con 0x7ff0dc0afd70 2023-12-19T18:16:45.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0da7fc640 1 -- 172.21.15.26:0/1117878244 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0c8003530 con 0x7ff0dc0afd70 2023-12-19T18:16:45.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0da7fc640 1 -- 172.21.15.26:0/1117878244 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767980059 0 0) 0x7ff0dc0b3240 con 0x7ff0dc0cc770 2023-12-19T18:16:45.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0da7fc640 1 -- 172.21.15.26:0/1117878244 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0dc06b7c0 con 0x7ff0dc0cc770 2023-12-19T18:16:45.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0d3fff640 1 -- 172.21.15.26:0/1117878244 >> v1:172.21.15.160:6789/0 conn(0x7ff0dc0cc770 legacy=0x7ff0dc0cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0d3fff640 1 -- 172.21.15.26:0/1117878244 >> v1:172.21.15.26:6789/0 conn(0x7ff0dc0afd70 legacy=0x7ff0dc0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.654+0000 7ff0d3fff640 1 -- 172.21.15.26:0/1117878244 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc161a10 con 0x7ff0dc0cd840 2023-12-19T18:16:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.658+0000 7ff0d3fff640 1 --2- 172.21.15.26:0/1117878244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0dc0cd840 0x7ff0dc0cfe10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40097c0 tx=0x7ff0c401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a05d48c01757e00 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.658+0000 7ff0da7fc640 1 -- 172.21.15.26:0/1117878244 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40a6040 con 0x7ff0dc0cd840 2023-12-19T18:16:45.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.658+0000 7ff0da7fc640 1 -- 172.21.15.26:0/1117878244 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c40aa600 con 0x7ff0dc0cd840 2023-12-19T18:16:45.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.658+0000 7ff0da7fc640 1 -- 172.21.15.26:0/1117878244 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40aa990 con 0x7ff0dc0cd840 2023-12-19T18:16:45.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.658+0000 7ff0e142d640 1 -- 172.21.15.26:0/1117878244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0dc0cd840 msgr2=0x7ff0dc0cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.658+0000 7ff0e142d640 1 --2- 172.21.15.26:0/1117878244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0dc0cd840 0x7ff0dc0cfe10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40097c0 tx=0x7ff0c401c480 comp rx=0 tx=0).stop 2023-12-19T18:16:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.658+0000 7ff0e142d640 1 -- 172.21.15.26:0/1117878244 shutdown_connections 2023-12-19T18:16:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.658+0000 7ff0e142d640 1 --2- 172.21.15.26:0/1117878244 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0dc0cd840 0x7ff0dc0cfe10 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.658+0000 7ff0e142d640 1 -- 172.21.15.26:0/1117878244 >> 172.21.15.26:0/1117878244 conn(0x7ff0dc06d570 msgr2=0x7ff0dc06d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.658+0000 7ff0e142d640 1 -- 172.21.15.26:0/1117878244 shutdown_connections 2023-12-19T18:16:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.658+0000 7ff0e142d640 1 -- 172.21.15.26:0/1117878244 wait complete. 2023-12-19T18:16:45.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0e142d640 1 Processor -- start 2023-12-19T18:16:45.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0e142d640 1 -- start start 2023-12-19T18:16:45.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0e142d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0dc0afd70 0x7ff0dc0af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0e142d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0dc0cc770 0x7ff0dc0b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0e142d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0dc0cd840 0x7ff0dc155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0e142d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0ad230 con 0x7ff0dc0afd70 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0e142d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0b6330 con 0x7ff0dc0cc770 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0e142d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0b6610 con 0x7ff0dc0cd840 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0daffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0dc0afd70 0x7ff0dc0af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0daffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0dc0afd70 0x7ff0dc0af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39598/0 (socket says 172.21.15.26:39598) 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0d3fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0dc0cc770 0x7ff0dc0b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0daffd640 1 -- 172.21.15.26:0/426932717 learned_addr learned my addr 172.21.15.26:0/426932717 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0db7fe640 1 --2- 172.21.15.26:0/426932717 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0dc0cd840 0x7ff0dc155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0daffd640 1 -- 172.21.15.26:0/426932717 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0dc0cd840 msgr2=0x7ff0dc155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0daffd640 1 --2- 172.21.15.26:0/426932717 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0dc0cd840 0x7ff0dc155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0daffd640 1 -- 172.21.15.26:0/426932717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0dc0cc770 msgr2=0x7ff0dc0b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0daffd640 1 --2- 172.21.15.26:0/426932717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0dc0cc770 0x7ff0dc0b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0daffd640 1 -- 172.21.15.26:0/426932717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4009420 con 0x7ff0dc0afd70 2023-12-19T18:16:45.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0daffd640 1 --2- 172.21.15.26:0/426932717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0dc0afd70 0x7ff0dc0af2b0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00c430 tx=0x7ff0cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:45.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0d37fe640 1 -- 172.21.15.26:0/426932717 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0052d0 con 0x7ff0dc0afd70 2023-12-19T18:16:45.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0d37fe640 1 -- 172.21.15.26:0/426932717 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc005450 con 0x7ff0dc0afd70 2023-12-19T18:16:45.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0e142d640 1 -- 172.21.15.26:0/426932717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0dc0acbe0 con 0x7ff0dc0afd70 2023-12-19T18:16:45.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0d37fe640 1 -- 172.21.15.26:0/426932717 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc003840 con 0x7ff0dc0afd70 2023-12-19T18:16:45.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.662+0000 7ff0e142d640 1 -- 172.21.15.26:0/426932717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0dc0b3b30 con 0x7ff0dc0afd70 2023-12-19T18:16:45.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.666+0000 7ff0e142d640 1 -- 172.21.15.26:0/426932717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0dc074270 con 0x7ff0dc0afd70 2023-12-19T18:16:45.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.666+0000 7ff0d37fe640 1 -- 172.21.15.26:0/426932717 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff0cc0039a0 con 0x7ff0dc0afd70 2023-12-19T18:16:45.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.666+0000 7ff0d37fe640 1 --2- 172.21.15.26:0/426932717 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff0bc065bb0 0x7ff0bc068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.666+0000 7ff0d37fe640 1 -- 172.21.15.26:0/426932717 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7ff0cc082ff0 con 0x7ff0dc0afd70 2023-12-19T18:16:45.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.666+0000 7ff0d3fff640 1 --2- 172.21.15.26:0/426932717 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff0bc065bb0 0x7ff0bc068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.670+0000 7ff0d3fff640 1 --2- 172.21.15.26:0/426932717 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff0bc065bb0 0x7ff0bc068070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40ae000 tx=0x7ff0c40b5040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:45.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.670+0000 7ff0d37fe640 1 -- 172.21.15.26:0/426932717 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff0cc0b40a0 con 0x7ff0dc0afd70 2023-12-19T18:16:45.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0e142d640 1 -- 172.21.15.26:0/426932717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff0dc0b27c0 con 0x7ff0dc0afd70 2023-12-19T18:16:45.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0d37fe640 1 -- 172.21.15.26:0/426932717 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1125 (secure 0 0 0) 0x7ff0cc0501f0 con 0x7ff0dc0afd70 2023-12-19T18:16:45.888 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:45.889 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":5683,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":216,"num_osds":8,"num_up_osds":8,"osd_up_since":1703004134,"num_in_osds":8,"osd_in_since":1703004125,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":193}],"num_pgs":193,"num_pools":5,"num_objects":46,"data_bytes":596333,"bytes_used":433451008,"bytes_avail":772660662272,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":102,"id":12,"up":1,"in":1,"max":1,"id":13,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":13,"rank":0,"name":"d","status":"up:active","gid":10605},{"filesystem_id":12,"rank":0,"name":"a","status":"up:active","gid":10611}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2023-12-19T18:15:52.413021+0000","services":{}},"progress_events":{}} 2023-12-19T18:16:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0e142d640 1 -- 172.21.15.26:0/426932717 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff0bc065bb0 msgr2=0x7ff0bc068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0e142d640 1 --2- 172.21.15.26:0/426932717 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff0bc065bb0 0x7ff0bc068070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40ae000 tx=0x7ff0c40b5040 comp rx=0 tx=0).stop 2023-12-19T18:16:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0e142d640 1 -- 172.21.15.26:0/426932717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0dc0afd70 msgr2=0x7ff0dc0af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0e142d640 1 --2- 172.21.15.26:0/426932717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0dc0afd70 0x7ff0dc0af2b0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00c430 tx=0x7ff0cc00c8f0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0e142d640 1 -- 172.21.15.26:0/426932717 shutdown_connections 2023-12-19T18:16:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0e142d640 1 --2- 172.21.15.26:0/426932717 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff0bc065bb0 0x7ff0bc068070 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0e142d640 1 --2- 172.21.15.26:0/426932717 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff0dc0cd840 0x7ff0dc155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0e142d640 1 --2- 172.21.15.26:0/426932717 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff0dc0cc770 0x7ff0dc0b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0e142d640 1 --2- 172.21.15.26:0/426932717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0dc0afd70 0x7ff0dc0af2b0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0e142d640 1 -- 172.21.15.26:0/426932717 >> 172.21.15.26:0/426932717 conn(0x7ff0dc06d570 msgr2=0x7ff0dc0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:45.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0e142d640 1 -- 172.21.15.26:0/426932717 shutdown_connections 2023-12-19T18:16:45.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:45.886+0000 7ff0e142d640 1 -- 172.21.15.26:0/426932717 wait complete. 2023-12-19T18:16:45.910 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-19T18:16:45.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:16:46.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.050+0000 7f48cc28c640 1 Processor -- start 2023-12-19T18:16:46.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.050+0000 7f48cc28c640 1 -- start start 2023-12-19T18:16:46.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.050+0000 7f48cc28c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f48c415c7f0 0x7f48c406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:46.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.050+0000 7f48cc28c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48c40b30d0 0x7f48c40b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:46.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.050+0000 7f48cc28c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c40b3a60 con 0x7f48c40adb80 2023-12-19T18:16:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.050+0000 7f48cc28c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c406d770 con 0x7f48c415c7f0 2023-12-19T18:16:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.050+0000 7f48cc28c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c406ced0 con 0x7f48c40b30d0 2023-12-19T18:16:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.050+0000 7f48c9827640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f48c40adb80 0x7f48c40cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44464/0 (socket says 172.21.15.26:44464) 2023-12-19T18:16:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.050+0000 7f48c9827640 1 -- 172.21.15.26:0/445260163 learned_addr learned my addr 172.21.15.26:0/445260163 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.050+0000 7f48ca028640 1 --2- 172.21.15.26:0/445260163 >> v2:172.21.15.160:3300/0 conn(0x7f48c415c7f0 0x7f48c406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.050+0000 7f48ca829640 1 --2- 172.21.15.26:0/445260163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48c40b30d0 0x7f48c40b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.050+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823898155 0 0) 0x7f48c40b3a60 con 0x7f48c40adb80 2023-12-19T18:16:46.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.050+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48b0003530 con 0x7f48c40adb80 2023-12-19T18:16:46.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1882129805 0 0) 0x7f48b0003530 con 0x7f48c40adb80 2023-12-19T18:16:46.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48c40b30d0 msgr2=0x7f48c40b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:46.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 --2- 172.21.15.26:0/445260163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48c40b30d0 0x7f48c40b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 >> v2:172.21.15.160:3300/0 conn(0x7f48c415c7f0 msgr2=0x7f48c406c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:46.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 --2- 172.21.15.26:0/445260163 >> v2:172.21.15.160:3300/0 conn(0x7f48c415c7f0 0x7f48c406c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c406d1b0 con 0x7f48c40adb80 2023-12-19T18:16:46.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f48b4001680 con 0x7f48c40adb80 2023-12-19T18:16:46.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 >> v1:172.21.15.26:6789/0 conn(0x7f48c40adb80 legacy=0x7f48c40cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:46.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 --2- 172.21.15.26:0/445260163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48b0003980 0x7f48b0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:46.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 --2- 172.21.15.26:0/445260163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b00047d0 0x7f48b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:46.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 --2- 172.21.15.26:0/445260163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48c40b30d0 0x7f48b0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c406d770 con 0x7f48b00047d0 2023-12-19T18:16:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c406ced0 con 0x7f48b0003980 2023-12-19T18:16:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b000dba0 con 0x7f48c40b30d0 2023-12-19T18:16:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48ca028640 1 --2- 172.21.15.26:0/445260163 >> v2:172.21.15.160:3300/0 conn(0x7f48c415c7f0 0x7f48c406c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9827640 1 --2- 172.21.15.26:0/445260163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b00047d0 0x7f48b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48ca829640 1 --2- 172.21.15.26:0/445260163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48c40b30d0 0x7f48b0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48ca028640 1 --2- 172.21.15.26:0/445260163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48b0003980 0x7f48b0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9827640 1 -- 172.21.15.26:0/445260163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48c40b30d0 msgr2=0x7f48b0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9827640 1 --2- 172.21.15.26:0/445260163 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48c40b30d0 0x7f48b0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9827640 1 -- 172.21.15.26:0/445260163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48b0003980 msgr2=0x7f48b0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9827640 1 --2- 172.21.15.26:0/445260163 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48b0003980 0x7f48b0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9827640 1 -- 172.21.15.26:0/445260163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b0003530 con 0x7f48b00047d0 2023-12-19T18:16:46.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.054+0000 7f48c9827640 1 --2- 172.21.15.26:0/445260163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b00047d0 0x7f48b0004c50 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f48b400c240 tx=0x7f48b401f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a625ad7b5165cf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:46.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b40a8d00 con 0x7f48b00047d0 2023-12-19T18:16:46.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b40a8e60 con 0x7f48b00047d0 2023-12-19T18:16:46.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48c9026640 1 -- 172.21.15.26:0/445260163 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b4005b00 con 0x7f48b00047d0 2023-12-19T18:16:46.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 -- 172.21.15.26:0/445260163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b00047d0 msgr2=0x7f48b0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:46.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 --2- 172.21.15.26:0/445260163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48b00047d0 0x7f48b0004c50 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f48b400c240 tx=0x7f48b401f3d0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48ca829640 1 -- 172.21.15.26:0/445260163 reap_dead start 2023-12-19T18:16:46.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 -- 172.21.15.26:0/445260163 shutdown_connections 2023-12-19T18:16:46.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 -- 172.21.15.26:0/445260163 >> 172.21.15.26:0/445260163 conn(0x7f48c40658e0 msgr2=0x7f48c4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:46.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 -- 172.21.15.26:0/445260163 shutdown_connections 2023-12-19T18:16:46.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 -- 172.21.15.26:0/445260163 wait complete. 2023-12-19T18:16:46.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 Processor -- start 2023-12-19T18:16:46.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 -- start start 2023-12-19T18:16:46.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48b00047d0 0x7f48c41e4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:46.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c41e58e0 0x7f48c41e6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:46.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48c4149330 0x7f48c406f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:46.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c41e5450 con 0x7f48c41e58e0 2023-12-19T18:16:46.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c41e6cc0 con 0x7f48b00047d0 2023-12-19T18:16:46.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48cc28c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c41e5d40 con 0x7f48c4149330 2023-12-19T18:16:46.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48c9827640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c41e58e0 0x7f48c41e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:46.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48ca028640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48b00047d0 0x7f48c41e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:46.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48ca028640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48b00047d0 0x7f48c41e4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:41282/0 (socket says 172.21.15.26:41282) 2023-12-19T18:16:46.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48ca028640 1 -- 172.21.15.26:0/1944906565 learned_addr learned my addr 172.21.15.26:0/1944906565 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:46.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.058+0000 7f48c9827640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c41e58e0 0x7f48c41e6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32794/0 (socket says 172.21.15.26:32794) 2023-12-19T18:16:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48ca829640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48c4149330 0x7f48c406f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48ca028640 1 -- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48c4149330 msgr2=0x7f48c406f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48ca028640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48c4149330 0x7f48c406f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48ca028640 1 -- 172.21.15.26:0/1944906565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c41e58e0 msgr2=0x7f48c41e6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48ca028640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c41e58e0 0x7f48c41e6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48ca028640 1 -- 172.21.15.26:0/1944906565 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c41e4840 con 0x7f48b00047d0 2023-12-19T18:16:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48c9827640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c41e58e0 0x7f48c41e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48ca028640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48b00047d0 0x7f48c41e4160 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f48c0011900 tx=0x7f48c0011dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48baffd640 1 -- 172.21.15.26:0/1944906565 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c000fbd0 con 0x7f48b00047d0 2023-12-19T18:16:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48baffd640 1 -- 172.21.15.26:0/1944906565 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c000fd30 con 0x7f48b00047d0 2023-12-19T18:16:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48cc28c640 1 -- 172.21.15.26:0/1944906565 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c41e4b20 con 0x7f48b00047d0 2023-12-19T18:16:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48baffd640 1 -- 172.21.15.26:0/1944906565 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c0015660 con 0x7f48b00047d0 2023-12-19T18:16:46.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48cc28c640 1 -- 172.21.15.26:0/1944906565 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c41e50e0 con 0x7f48b00047d0 2023-12-19T18:16:46.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48cc28c640 1 -- 172.21.15.26:0/1944906565 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c4074270 con 0x7f48b00047d0 2023-12-19T18:16:46.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.062+0000 7f48baffd640 1 -- 172.21.15.26:0/1944906565 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f48c00157c0 con 0x7f48b00047d0 2023-12-19T18:16:46.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.066+0000 7f48baffd640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f48a8065b80 0x7f48a8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:46.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.066+0000 7f48baffd640 1 -- 172.21.15.26:0/1944906565 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f48c0084440 con 0x7f48b00047d0 2023-12-19T18:16:46.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.066+0000 7f48c9827640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f48a8065b80 0x7f48a8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:46.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.066+0000 7f48c9827640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f48a8065b80 0x7f48a8068040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48b40ad000 tx=0x7f48b4001a80 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:46.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.070+0000 7f48baffd640 1 -- 172.21.15.26:0/1944906565 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f48c0051610 con 0x7f48b00047d0 2023-12-19T18:16:46.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.222+0000 7f48cc28c640 1 -- 172.21.15.26:0/1944906565 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48c40b2bb0 con 0x7f48a8065b80 2023-12-19T18:16:46.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.234+0000 7f48baffd640 1 -- 172.21.15.26:0/1944906565 <== mgr.9805 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528500 (secure 0 0 0) 0x7f48b40047f0 con 0x7f48a8065b80 2023-12-19T18:16:46.244 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:46.248 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:16:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.242+0000 7f48cc28c640 1 -- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f48a8065b80 msgr2=0x7f48a8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.242+0000 7f48cc28c640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f48a8065b80 0x7f48a8068040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48b40ad000 tx=0x7f48b4001a80 comp rx=0 tx=0).stop 2023-12-19T18:16:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.242+0000 7f48cc28c640 1 -- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48b00047d0 msgr2=0x7f48c41e4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.242+0000 7f48cc28c640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48b00047d0 0x7f48c41e4160 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f48c0011900 tx=0x7f48c0011dc0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.242+0000 7f48cc28c640 1 -- 172.21.15.26:0/1944906565 shutdown_connections 2023-12-19T18:16:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.242+0000 7f48cc28c640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f48a8065b80 0x7f48a8068040 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.242+0000 7f48cc28c640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f48c4149330 0x7f48c406f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.242+0000 7f48cc28c640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48c41e58e0 0x7f48c41e6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.242+0000 7f48cc28c640 1 --2- 172.21.15.26:0/1944906565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48b00047d0 0x7f48c41e4160 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.242+0000 7f48cc28c640 1 -- 172.21.15.26:0/1944906565 >> 172.21.15.26:0/1944906565 conn(0x7f48c40658e0 msgr2=0x7f48c40af740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.242+0000 7f48cc28c640 1 -- 172.21.15.26:0/1944906565 shutdown_connections 2023-12-19T18:16:46.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:46.242+0000 7f48cc28c640 1 -- 172.21.15.26:0/1944906565 wait complete. 2023-12-19T18:16:46.263 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-12-19T18:16:45.307976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596333,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":667,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":426,"ondisk_log_size":426,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":344,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423292,"kb_used_data":22332,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754551428,"statfs":{"total":773094113280,"available":772660662272,"internally_reserved":0,"allocated":22867968,"data_stored":12631683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001750"},"pg_stats":[{"pgid":"27.1a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357761+0000","last_change":"2023-12-19T18:16:40.564006+0000","last_active":"2023-12-19T18:16:42.357761+0000","last_peered":"2023-12-19T18:16:42.357761+0000","last_clean":"2023-12-19T18:16:42.357761+0000","last_became_active":"2023-12-19T18:16:40.563756+0000","last_became_peered":"2023-12-19T18:16:40.563756+0000","last_unstale":"2023-12-19T18:16:42.357761+0000","last_undegraded":"2023-12-19T18:16:42.357761+0000","last_fullsized":"2023-12-19T18:16:42.357761+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:17.484857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"200'346","reported_seq":1188,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560456+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:16:40.560456+0000","last_peered":"2023-12-19T18:16:40.560456+0000","last_clean":"2023-12-19T18:16:40.560456+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:16:40.560456+0000","last_undegraded":"2023-12-19T18:16:40.560456+0000","last_fullsized":"2023-12-19T18:16:40.560456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560313+0000","last_change":"2023-12-19T18:16:27.411230+0000","last_active":"2023-12-19T18:16:40.560313+0000","last_peered":"2023-12-19T18:16:40.560313+0000","last_clean":"2023-12-19T18:16:40.560313+0000","last_became_active":"2023-12-19T18:16:27.410670+0000","last_became_peered":"2023-12-19T18:16:27.410670+0000","last_unstale":"2023-12-19T18:16:40.560313+0000","last_undegraded":"2023-12-19T18:16:40.560313+0000","last_fullsized":"2023-12-19T18:16:40.560313+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:22.426446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554070+0000","last_change":"2023-12-19T18:16:27.821899+0000","last_active":"2023-12-19T18:16:40.554070+0000","last_peered":"2023-12-19T18:16:40.554070+0000","last_clean":"2023-12-19T18:16:40.554070+0000","last_became_active":"2023-12-19T18:16:27.821634+0000","last_became_peered":"2023-12-19T18:16:27.821634+0000","last_unstale":"2023-12-19T18:16:40.554070+0000","last_undegraded":"2023-12-19T18:16:40.554070+0000","last_fullsized":"2023-12-19T18:16:40.554070+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:34.391068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857383+0000","last_change":"2023-12-19T18:16:40.150476+0000","last_active":"2023-12-19T18:16:41.857383+0000","last_peered":"2023-12-19T18:16:41.857383+0000","last_clean":"2023-12-19T18:16:41.857383+0000","last_became_active":"2023-12-19T18:16:40.149843+0000","last_became_peered":"2023-12-19T18:16:40.149843+0000","last_unstale":"2023-12-19T18:16:41.857383+0000","last_undegraded":"2023-12-19T18:16:41.857383+0000","last_fullsized":"2023-12-19T18:16:41.857383+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:35.179628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857177+0000","last_change":"2023-12-19T18:16:40.569150+0000","last_active":"2023-12-19T18:16:41.857177+0000","last_peered":"2023-12-19T18:16:41.857177+0000","last_clean":"2023-12-19T18:16:41.857177+0000","last_became_active":"2023-12-19T18:16:40.568754+0000","last_became_peered":"2023-12-19T18:16:40.568754+0000","last_unstale":"2023-12-19T18:16:41.857177+0000","last_undegraded":"2023-12-19T18:16:41.857177+0000","last_fullsized":"2023-12-19T18:16:41.857177+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:32.689972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548913+0000","last_change":"2023-12-19T18:16:27.412171+0000","last_active":"2023-12-19T18:16:40.548913+0000","last_peered":"2023-12-19T18:16:40.548913+0000","last_clean":"2023-12-19T18:16:40.548913+0000","last_became_active":"2023-12-19T18:16:27.411872+0000","last_became_peered":"2023-12-19T18:16:27.411872+0000","last_unstale":"2023-12-19T18:16:40.548913+0000","last_undegraded":"2023-12-19T18:16:40.548913+0000","last_fullsized":"2023-12-19T18:16:40.548913+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:48.264905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548191+0000","last_change":"2023-12-19T18:16:27.832432+0000","last_active":"2023-12-19T18:16:40.548191+0000","last_peered":"2023-12-19T18:16:40.548191+0000","last_clean":"2023-12-19T18:16:40.548191+0000","last_became_active":"2023-12-19T18:16:27.832116+0000","last_became_peered":"2023-12-19T18:16:27.832116+0000","last_unstale":"2023-12-19T18:16:40.548191+0000","last_undegraded":"2023-12-19T18:16:40.548191+0000","last_fullsized":"2023-12-19T18:16:40.548191+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:35.165338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359499+0000","last_change":"2023-12-19T18:16:40.157009+0000","last_active":"2023-12-19T18:16:42.359499+0000","last_peered":"2023-12-19T18:16:42.359499+0000","last_clean":"2023-12-19T18:16:42.359499+0000","last_became_active":"2023-12-19T18:16:40.156734+0000","last_became_peered":"2023-12-19T18:16:40.156734+0000","last_unstale":"2023-12-19T18:16:42.359499+0000","last_undegraded":"2023-12-19T18:16:42.359499+0000","last_fullsized":"2023-12-19T18:16:42.359499+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:18.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357983+0000","last_change":"2023-12-19T18:16:40.560903+0000","last_active":"2023-12-19T18:16:42.357983+0000","last_peered":"2023-12-19T18:16:42.357983+0000","last_clean":"2023-12-19T18:16:42.357983+0000","last_became_active":"2023-12-19T18:16:40.560636+0000","last_became_peered":"2023-12-19T18:16:40.560636+0000","last_unstale":"2023-12-19T18:16:42.357983+0000","last_undegraded":"2023-12-19T18:16:42.357983+0000","last_fullsized":"2023-12-19T18:16:42.357983+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:25:38.247442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561435+0000","last_change":"2023-12-19T18:16:27.407456+0000","last_active":"2023-12-19T18:16:40.561435+0000","last_peered":"2023-12-19T18:16:40.561435+0000","last_clean":"2023-12-19T18:16:40.561435+0000","last_became_active":"2023-12-19T18:16:27.407208+0000","last_became_peered":"2023-12-19T18:16:27.407208+0000","last_unstale":"2023-12-19T18:16:40.561435+0000","last_undegraded":"2023-12-19T18:16:40.561435+0000","last_fullsized":"2023-12-19T18:16:40.561435+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:07.654229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560528+0000","last_change":"2023-12-19T18:16:27.824001+0000","last_active":"2023-12-19T18:16:40.560528+0000","last_peered":"2023-12-19T18:16:40.560528+0000","last_clean":"2023-12-19T18:16:40.560528+0000","last_became_active":"2023-12-19T18:16:27.823010+0000","last_became_peered":"2023-12-19T18:16:27.823010+0000","last_unstale":"2023-12-19T18:16:40.560528+0000","last_undegraded":"2023-12-19T18:16:40.560528+0000","last_fullsized":"2023-12-19T18:16:40.560528+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:30:56.775994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074880+0000","last_change":"2023-12-19T18:16:40.164940+0000","last_active":"2023-12-19T18:16:42.074880+0000","last_peered":"2023-12-19T18:16:42.074880+0000","last_clean":"2023-12-19T18:16:42.074880+0000","last_became_active":"2023-12-19T18:16:40.163710+0000","last_became_peered":"2023-12-19T18:16:40.163710+0000","last_unstale":"2023-12-19T18:16:42.074880+0000","last_undegraded":"2023-12-19T18:16:42.074880+0000","last_fullsized":"2023-12-19T18:16:42.074880+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:29.482411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552525+0000","last_change":"2023-12-19T18:16:40.570024+0000","last_active":"2023-12-19T18:16:41.552525+0000","last_peered":"2023-12-19T18:16:41.552525+0000","last_clean":"2023-12-19T18:16:41.552525+0000","last_became_active":"2023-12-19T18:16:40.569586+0000","last_became_peered":"2023-12-19T18:16:40.569586+0000","last_unstale":"2023-12-19T18:16:41.552525+0000","last_undegraded":"2023-12-19T18:16:41.552525+0000","last_fullsized":"2023-12-19T18:16:41.552525+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:03.932038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547644+0000","last_change":"2023-12-19T18:16:27.405381+0000","last_active":"2023-12-19T18:16:40.547644+0000","last_peered":"2023-12-19T18:16:40.547644+0000","last_clean":"2023-12-19T18:16:40.547644+0000","last_became_active":"2023-12-19T18:16:27.405113+0000","last_became_peered":"2023-12-19T18:16:27.405113+0000","last_unstale":"2023-12-19T18:16:40.547644+0000","last_undegraded":"2023-12-19T18:16:40.547644+0000","last_fullsized":"2023-12-19T18:16:40.547644+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:08.911302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560761+0000","last_change":"2023-12-19T18:16:27.822423+0000","last_active":"2023-12-19T18:16:40.560761+0000","last_peered":"2023-12-19T18:16:40.560761+0000","last_clean":"2023-12-19T18:16:40.560761+0000","last_became_active":"2023-12-19T18:16:27.822106+0000","last_became_peered":"2023-12-19T18:16:27.822106+0000","last_unstale":"2023-12-19T18:16:40.560761+0000","last_undegraded":"2023-12-19T18:16:40.560761+0000","last_fullsized":"2023-12-19T18:16:40.560761+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:41.021760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.856687+0000","last_change":"2023-12-19T18:16:40.154697+0000","last_active":"2023-12-19T18:16:41.856687+0000","last_peered":"2023-12-19T18:16:41.856687+0000","last_clean":"2023-12-19T18:16:41.856687+0000","last_became_active":"2023-12-19T18:16:40.154394+0000","last_became_peered":"2023-12-19T18:16:40.154394+0000","last_unstale":"2023-12-19T18:16:41.856687+0000","last_undegraded":"2023-12-19T18:16:41.856687+0000","last_fullsized":"2023-12-19T18:16:41.856687+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:52.755953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357512+0000","last_change":"2023-12-19T18:16:40.572030+0000","last_active":"2023-12-19T18:16:42.357512+0000","last_peered":"2023-12-19T18:16:42.357512+0000","last_clean":"2023-12-19T18:16:42.357512+0000","last_became_active":"2023-12-19T18:16:40.570330+0000","last_became_peered":"2023-12-19T18:16:40.570330+0000","last_unstale":"2023-12-19T18:16:42.357512+0000","last_undegraded":"2023-12-19T18:16:42.357512+0000","last_fullsized":"2023-12-19T18:16:42.357512+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:55.123677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547774+0000","last_change":"2023-12-19T18:16:27.410513+0000","last_active":"2023-12-19T18:16:40.547774+0000","last_peered":"2023-12-19T18:16:40.547774+0000","last_clean":"2023-12-19T18:16:40.547774+0000","last_became_active":"2023-12-19T18:16:27.410166+0000","last_became_peered":"2023-12-19T18:16:27.410166+0000","last_unstale":"2023-12-19T18:16:40.547774+0000","last_undegraded":"2023-12-19T18:16:40.547774+0000","last_fullsized":"2023-12-19T18:16:40.547774+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:10.682880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560587+0000","last_change":"2023-12-19T18:16:27.829925+0000","last_active":"2023-12-19T18:16:40.560587+0000","last_peered":"2023-12-19T18:16:40.560587+0000","last_clean":"2023-12-19T18:16:40.560587+0000","last_became_active":"2023-12-19T18:16:27.828958+0000","last_became_peered":"2023-12-19T18:16:27.828958+0000","last_unstale":"2023-12-19T18:16:40.560587+0000","last_undegraded":"2023-12-19T18:16:40.560587+0000","last_fullsized":"2023-12-19T18:16:40.560587+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:46.995476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554304+0000","last_change":"2023-12-19T18:16:40.156673+0000","last_active":"2023-12-19T18:16:41.554304+0000","last_peered":"2023-12-19T18:16:41.554304+0000","last_clean":"2023-12-19T18:16:41.554304+0000","last_became_active":"2023-12-19T18:16:40.156391+0000","last_became_peered":"2023-12-19T18:16:40.156391+0000","last_unstale":"2023-12-19T18:16:41.554304+0000","last_undegraded":"2023-12-19T18:16:41.554304+0000","last_fullsized":"2023-12-19T18:16:41.554304+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:46.681719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360169+0000","last_change":"2023-12-19T18:16:40.571314+0000","last_active":"2023-12-19T18:16:42.360169+0000","last_peered":"2023-12-19T18:16:42.360169+0000","last_clean":"2023-12-19T18:16:42.360169+0000","last_became_active":"2023-12-19T18:16:40.570538+0000","last_became_peered":"2023-12-19T18:16:40.570538+0000","last_unstale":"2023-12-19T18:16:42.360169+0000","last_undegraded":"2023-12-19T18:16:42.360169+0000","last_fullsized":"2023-12-19T18:16:42.360169+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:02:14.652060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543489+0000","last_change":"2023-12-19T18:16:27.409169+0000","last_active":"2023-12-19T18:16:40.543489+0000","last_peered":"2023-12-19T18:16:40.543489+0000","last_clean":"2023-12-19T18:16:40.543489+0000","last_became_active":"2023-12-19T18:16:27.408882+0000","last_became_peered":"2023-12-19T18:16:27.408882+0000","last_unstale":"2023-12-19T18:16:40.543489+0000","last_undegraded":"2023-12-19T18:16:40.543489+0000","last_fullsized":"2023-12-19T18:16:40.543489+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:01.478246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544313+0000","last_change":"2023-12-19T18:16:27.825200+0000","last_active":"2023-12-19T18:16:40.544313+0000","last_peered":"2023-12-19T18:16:40.544313+0000","last_clean":"2023-12-19T18:16:40.544313+0000","last_became_active":"2023-12-19T18:16:27.824919+0000","last_became_peered":"2023-12-19T18:16:27.824919+0000","last_unstale":"2023-12-19T18:16:40.544313+0000","last_undegraded":"2023-12-19T18:16:40.544313+0000","last_fullsized":"2023-12-19T18:16:40.544313+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.6","version":"216'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.077099+0000","last_change":"2023-12-19T18:16:40.164339+0000","last_active":"2023-12-19T18:16:42.077099+0000","last_peered":"2023-12-19T18:16:42.077099+0000","last_clean":"2023-12-19T18:16:42.077099+0000","last_became_active":"2023-12-19T18:16:40.163870+0000","last_became_peered":"2023-12-19T18:16:40.163870+0000","last_unstale":"2023-12-19T18:16:42.077099+0000","last_undegraded":"2023-12-19T18:16:42.077099+0000","last_fullsized":"2023-12-19T18:16:42.077099+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:14:22.971741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857196+0000","last_change":"2023-12-19T18:16:40.568314+0000","last_active":"2023-12-19T18:16:41.857196+0000","last_peered":"2023-12-19T18:16:41.857196+0000","last_clean":"2023-12-19T18:16:41.857196+0000","last_became_active":"2023-12-19T18:16:40.567857+0000","last_became_peered":"2023-12-19T18:16:40.567857+0000","last_unstale":"2023-12-19T18:16:41.857196+0000","last_undegraded":"2023-12-19T18:16:41.857196+0000","last_fullsized":"2023-12-19T18:16:41.857196+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:49.703164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555637+0000","last_change":"2023-12-19T18:16:27.420249+0000","last_active":"2023-12-19T18:16:40.555637+0000","last_peered":"2023-12-19T18:16:40.555637+0000","last_clean":"2023-12-19T18:16:40.555637+0000","last_became_active":"2023-12-19T18:16:27.419360+0000","last_became_peered":"2023-12-19T18:16:27.419360+0000","last_unstale":"2023-12-19T18:16:40.555637+0000","last_undegraded":"2023-12-19T18:16:40.555637+0000","last_fullsized":"2023-12-19T18:16:40.555637+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:07.969904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543383+0000","last_change":"2023-12-19T18:16:27.825535+0000","last_active":"2023-12-19T18:16:40.543383+0000","last_peered":"2023-12-19T18:16:40.543383+0000","last_clean":"2023-12-19T18:16:40.543383+0000","last_became_active":"2023-12-19T18:16:27.824814+0000","last_became_peered":"2023-12-19T18:16:27.824814+0000","last_unstale":"2023-12-19T18:16:40.543383+0000","last_undegraded":"2023-12-19T18:16:40.543383+0000","last_fullsized":"2023-12-19T18:16:40.543383+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:29.357409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358846+0000","last_change":"2023-12-19T18:16:40.160113+0000","last_active":"2023-12-19T18:16:42.358846+0000","last_peered":"2023-12-19T18:16:42.358846+0000","last_clean":"2023-12-19T18:16:42.358846+0000","last_became_active":"2023-12-19T18:16:40.159826+0000","last_became_peered":"2023-12-19T18:16:40.159826+0000","last_unstale":"2023-12-19T18:16:42.358846+0000","last_undegraded":"2023-12-19T18:16:42.358846+0000","last_fullsized":"2023-12-19T18:16:42.358846+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:45.058000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554995+0000","last_change":"2023-12-19T18:16:40.566980+0000","last_active":"2023-12-19T18:16:41.554995+0000","last_peered":"2023-12-19T18:16:41.554995+0000","last_clean":"2023-12-19T18:16:41.554995+0000","last_became_active":"2023-12-19T18:16:40.566716+0000","last_became_peered":"2023-12-19T18:16:40.566716+0000","last_unstale":"2023-12-19T18:16:41.554995+0000","last_undegraded":"2023-12-19T18:16:41.554995+0000","last_fullsized":"2023-12-19T18:16:41.554995+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:30.489642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.6","version":"212'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558804+0000","last_change":"2023-12-19T18:16:27.414152+0000","last_active":"2023-12-19T18:16:40.558804+0000","last_peered":"2023-12-19T18:16:40.558804+0000","last_clean":"2023-12-19T18:16:40.558804+0000","last_became_active":"2023-12-19T18:16:27.413877+0000","last_became_peered":"2023-12-19T18:16:27.413877+0000","last_unstale":"2023-12-19T18:16:40.558804+0000","last_undegraded":"2023-12-19T18:16:40.558804+0000","last_fullsized":"2023-12-19T18:16:40.558804+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:44:52.553685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558691+0000","last_change":"2023-12-19T18:16:27.824257+0000","last_active":"2023-12-19T18:16:40.558691+0000","last_peered":"2023-12-19T18:16:40.558691+0000","last_clean":"2023-12-19T18:16:40.558691+0000","last_became_active":"2023-12-19T18:16:27.823989+0000","last_became_peered":"2023-12-19T18:16:27.823989+0000","last_unstale":"2023-12-19T18:16:40.558691+0000","last_undegraded":"2023-12-19T18:16:40.558691+0000","last_fullsized":"2023-12-19T18:16:40.558691+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:07:07.505127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357977+0000","last_change":"2023-12-19T18:16:40.160316+0000","last_active":"2023-12-19T18:16:42.357977+0000","last_peered":"2023-12-19T18:16:42.357977+0000","last_clean":"2023-12-19T18:16:42.357977+0000","last_became_active":"2023-12-19T18:16:40.159312+0000","last_became_peered":"2023-12-19T18:16:40.159312+0000","last_unstale":"2023-12-19T18:16:42.357977+0000","last_undegraded":"2023-12-19T18:16:42.357977+0000","last_fullsized":"2023-12-19T18:16:42.357977+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:04.832569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553051+0000","last_change":"2023-12-19T18:16:40.563986+0000","last_active":"2023-12-19T18:16:41.553051+0000","last_peered":"2023-12-19T18:16:41.553051+0000","last_clean":"2023-12-19T18:16:41.553051+0000","last_became_active":"2023-12-19T18:16:40.563693+0000","last_became_peered":"2023-12-19T18:16:40.563693+0000","last_unstale":"2023-12-19T18:16:41.553051+0000","last_undegraded":"2023-12-19T18:16:41.553051+0000","last_fullsized":"2023-12-19T18:16:41.553051+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:27.465263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562303+0000","last_change":"2023-12-19T18:16:27.409874+0000","last_active":"2023-12-19T18:16:40.562303+0000","last_peered":"2023-12-19T18:16:40.562303+0000","last_clean":"2023-12-19T18:16:40.562303+0000","last_became_active":"2023-12-19T18:16:27.409598+0000","last_became_peered":"2023-12-19T18:16:27.409598+0000","last_unstale":"2023-12-19T18:16:40.562303+0000","last_undegraded":"2023-12-19T18:16:40.562303+0000","last_fullsized":"2023-12-19T18:16:40.562303+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:13.721057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549450+0000","last_change":"2023-12-19T18:16:27.826090+0000","last_active":"2023-12-19T18:16:40.549450+0000","last_peered":"2023-12-19T18:16:40.549450+0000","last_clean":"2023-12-19T18:16:40.549450+0000","last_became_active":"2023-12-19T18:16:27.825791+0000","last_became_peered":"2023-12-19T18:16:27.825791+0000","last_unstale":"2023-12-19T18:16:40.549450+0000","last_undegraded":"2023-12-19T18:16:40.549450+0000","last_fullsized":"2023-12-19T18:16:40.549450+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:39.382953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359959+0000","last_change":"2023-12-19T18:16:40.160037+0000","last_active":"2023-12-19T18:16:42.359959+0000","last_peered":"2023-12-19T18:16:42.359959+0000","last_clean":"2023-12-19T18:16:42.359959+0000","last_became_active":"2023-12-19T18:16:40.159737+0000","last_became_peered":"2023-12-19T18:16:40.159737+0000","last_unstale":"2023-12-19T18:16:42.359959+0000","last_undegraded":"2023-12-19T18:16:42.359959+0000","last_fullsized":"2023-12-19T18:16:42.359959+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:42.336188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358621+0000","last_change":"2023-12-19T18:16:40.567065+0000","last_active":"2023-12-19T18:16:42.358621+0000","last_peered":"2023-12-19T18:16:42.358621+0000","last_clean":"2023-12-19T18:16:42.358621+0000","last_became_active":"2023-12-19T18:16:40.566804+0000","last_became_peered":"2023-12-19T18:16:40.566804+0000","last_unstale":"2023-12-19T18:16:42.358621+0000","last_undegraded":"2023-12-19T18:16:42.358621+0000","last_fullsized":"2023-12-19T18:16:42.358621+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:21.029140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546688+0000","last_change":"2023-12-19T18:16:27.403537+0000","last_active":"2023-12-19T18:16:40.546688+0000","last_peered":"2023-12-19T18:16:40.546688+0000","last_clean":"2023-12-19T18:16:40.546688+0000","last_became_active":"2023-12-19T18:16:27.403250+0000","last_became_peered":"2023-12-19T18:16:27.403250+0000","last_unstale":"2023-12-19T18:16:40.546688+0000","last_undegraded":"2023-12-19T18:16:40.546688+0000","last_fullsized":"2023-12-19T18:16:40.546688+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:56.889902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548113+0000","last_change":"2023-12-19T18:16:27.825355+0000","last_active":"2023-12-19T18:16:40.548113+0000","last_peered":"2023-12-19T18:16:40.548113+0000","last_clean":"2023-12-19T18:16:40.548113+0000","last_became_active":"2023-12-19T18:16:27.825067+0000","last_became_peered":"2023-12-19T18:16:27.825067+0000","last_unstale":"2023-12-19T18:16:40.548113+0000","last_undegraded":"2023-12-19T18:16:40.548113+0000","last_fullsized":"2023-12-19T18:16:40.548113+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:02.928407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358143+0000","last_change":"2023-12-19T18:16:40.152882+0000","last_active":"2023-12-19T18:16:42.358143+0000","last_peered":"2023-12-19T18:16:42.358143+0000","last_clean":"2023-12-19T18:16:42.358143+0000","last_became_active":"2023-12-19T18:16:40.152561+0000","last_became_peered":"2023-12-19T18:16:40.152561+0000","last_unstale":"2023-12-19T18:16:42.358143+0000","last_undegraded":"2023-12-19T18:16:42.358143+0000","last_fullsized":"2023-12-19T18:16:42.358143+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:11:26.922031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550797+0000","last_change":"2023-12-19T18:16:40.560282+0000","last_active":"2023-12-19T18:16:41.550797+0000","last_peered":"2023-12-19T18:16:41.550797+0000","last_clean":"2023-12-19T18:16:41.550797+0000","last_became_active":"2023-12-19T18:16:40.560017+0000","last_became_peered":"2023-12-19T18:16:40.560017+0000","last_unstale":"2023-12-19T18:16:41.550797+0000","last_undegraded":"2023-12-19T18:16:41.550797+0000","last_fullsized":"2023-12-19T18:16:41.550797+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:51:01.095692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553689+0000","last_change":"2023-12-19T18:16:27.417198+0000","last_active":"2023-12-19T18:16:40.553689+0000","last_peered":"2023-12-19T18:16:40.553689+0000","last_clean":"2023-12-19T18:16:40.553689+0000","last_became_active":"2023-12-19T18:16:27.416902+0000","last_became_peered":"2023-12-19T18:16:27.416902+0000","last_unstale":"2023-12-19T18:16:40.553689+0000","last_undegraded":"2023-12-19T18:16:40.553689+0000","last_fullsized":"2023-12-19T18:16:40.553689+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:54.434372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560449+0000","last_change":"2023-12-19T18:16:27.824198+0000","last_active":"2023-12-19T18:16:40.560449+0000","last_peered":"2023-12-19T18:16:40.560449+0000","last_clean":"2023-12-19T18:16:40.560449+0000","last_became_active":"2023-12-19T18:16:27.823738+0000","last_became_peered":"2023-12-19T18:16:27.823738+0000","last_unstale":"2023-12-19T18:16:40.560449+0000","last_undegraded":"2023-12-19T18:16:40.560449+0000","last_fullsized":"2023-12-19T18:16:40.560449+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:53.509897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550952+0000","last_change":"2023-12-19T18:16:40.160436+0000","last_active":"2023-12-19T18:16:41.550952+0000","last_peered":"2023-12-19T18:16:41.550952+0000","last_clean":"2023-12-19T18:16:41.550952+0000","last_became_active":"2023-12-19T18:16:40.159692+0000","last_became_peered":"2023-12-19T18:16:40.159692+0000","last_unstale":"2023-12-19T18:16:41.550952+0000","last_undegraded":"2023-12-19T18:16:41.550952+0000","last_fullsized":"2023-12-19T18:16:41.550952+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:47:01.985064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554294+0000","last_change":"2023-12-19T18:16:40.572114+0000","last_active":"2023-12-19T18:16:41.554294+0000","last_peered":"2023-12-19T18:16:41.554294+0000","last_clean":"2023-12-19T18:16:41.554294+0000","last_became_active":"2023-12-19T18:16:40.571811+0000","last_became_peered":"2023-12-19T18:16:40.571811+0000","last_unstale":"2023-12-19T18:16:41.554294+0000","last_undegraded":"2023-12-19T18:16:41.554294+0000","last_fullsized":"2023-12-19T18:16:41.554294+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:01.072159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550771+0000","last_change":"2023-12-19T18:16:27.404853+0000","last_active":"2023-12-19T18:16:40.550771+0000","last_peered":"2023-12-19T18:16:40.550771+0000","last_clean":"2023-12-19T18:16:40.550771+0000","last_became_active":"2023-12-19T18:16:27.404347+0000","last_became_peered":"2023-12-19T18:16:27.404347+0000","last_unstale":"2023-12-19T18:16:40.550771+0000","last_undegraded":"2023-12-19T18:16:40.550771+0000","last_fullsized":"2023-12-19T18:16:40.550771+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:16.360996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558229+0000","last_change":"2023-12-19T18:16:27.828160+0000","last_active":"2023-12-19T18:16:40.558229+0000","last_peered":"2023-12-19T18:16:40.558229+0000","last_clean":"2023-12-19T18:16:40.558229+0000","last_became_active":"2023-12-19T18:16:27.827882+0000","last_became_peered":"2023-12-19T18:16:27.827882+0000","last_unstale":"2023-12-19T18:16:40.558229+0000","last_undegraded":"2023-12-19T18:16:40.558229+0000","last_fullsized":"2023-12-19T18:16:40.558229+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:17.726704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358933+0000","last_change":"2023-12-19T18:16:40.154706+0000","last_active":"2023-12-19T18:16:42.358933+0000","last_peered":"2023-12-19T18:16:42.358933+0000","last_clean":"2023-12-19T18:16:42.358933+0000","last_became_active":"2023-12-19T18:16:40.154154+0000","last_became_peered":"2023-12-19T18:16:40.154154+0000","last_unstale":"2023-12-19T18:16:42.358933+0000","last_undegraded":"2023-12-19T18:16:42.358933+0000","last_fullsized":"2023-12-19T18:16:42.358933+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:55.343366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552500+0000","last_change":"2023-12-19T18:16:40.555828+0000","last_active":"2023-12-19T18:16:41.552500+0000","last_peered":"2023-12-19T18:16:41.552500+0000","last_clean":"2023-12-19T18:16:41.552500+0000","last_became_active":"2023-12-19T18:16:40.555500+0000","last_became_peered":"2023-12-19T18:16:40.555500+0000","last_unstale":"2023-12-19T18:16:41.552500+0000","last_undegraded":"2023-12-19T18:16:41.552500+0000","last_fullsized":"2023-12-19T18:16:41.552500+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:50.483528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.b","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551100+0000","last_change":"2023-12-19T18:16:27.415902+0000","last_active":"2023-12-19T18:16:40.551100+0000","last_peered":"2023-12-19T18:16:40.551100+0000","last_clean":"2023-12-19T18:16:40.551100+0000","last_became_active":"2023-12-19T18:16:27.415062+0000","last_became_peered":"2023-12-19T18:16:27.415062+0000","last_unstale":"2023-12-19T18:16:40.551100+0000","last_undegraded":"2023-12-19T18:16:40.551100+0000","last_fullsized":"2023-12-19T18:16:40.551100+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:22.442653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556868+0000","last_change":"2023-12-19T18:16:27.827898+0000","last_active":"2023-12-19T18:16:40.556868+0000","last_peered":"2023-12-19T18:16:40.556868+0000","last_clean":"2023-12-19T18:16:40.556868+0000","last_became_active":"2023-12-19T18:16:27.827613+0000","last_became_peered":"2023-12-19T18:16:27.827613+0000","last_unstale":"2023-12-19T18:16:40.556868+0000","last_undegraded":"2023-12-19T18:16:40.556868+0000","last_fullsized":"2023-12-19T18:16:40.556868+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:29.033205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357614+0000","last_change":"2023-12-19T18:16:40.157634+0000","last_active":"2023-12-19T18:16:42.357614+0000","last_peered":"2023-12-19T18:16:42.357614+0000","last_clean":"2023-12-19T18:16:42.357614+0000","last_became_active":"2023-12-19T18:16:40.157363+0000","last_became_peered":"2023-12-19T18:16:40.157363+0000","last_unstale":"2023-12-19T18:16:42.357614+0000","last_undegraded":"2023-12-19T18:16:42.357614+0000","last_fullsized":"2023-12-19T18:16:42.357614+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:23.564600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553681+0000","last_change":"2023-12-19T18:16:40.556441+0000","last_active":"2023-12-19T18:16:41.553681+0000","last_peered":"2023-12-19T18:16:41.553681+0000","last_clean":"2023-12-19T18:16:41.553681+0000","last_became_active":"2023-12-19T18:16:40.556187+0000","last_became_peered":"2023-12-19T18:16:40.556187+0000","last_unstale":"2023-12-19T18:16:41.553681+0000","last_undegraded":"2023-12-19T18:16:41.553681+0000","last_fullsized":"2023-12-19T18:16:41.553681+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:16:05.083354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562653+0000","last_change":"2023-12-19T18:16:27.408051+0000","last_active":"2023-12-19T18:16:40.562653+0000","last_peered":"2023-12-19T18:16:40.562653+0000","last_clean":"2023-12-19T18:16:40.562653+0000","last_became_active":"2023-12-19T18:16:27.407796+0000","last_became_peered":"2023-12-19T18:16:27.407796+0000","last_unstale":"2023-12-19T18:16:40.562653+0000","last_undegraded":"2023-12-19T18:16:40.562653+0000","last_fullsized":"2023-12-19T18:16:40.562653+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:41:09.697612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560009+0000","last_change":"2023-12-19T18:16:27.832376+0000","last_active":"2023-12-19T18:16:40.560009+0000","last_peered":"2023-12-19T18:16:40.560009+0000","last_clean":"2023-12-19T18:16:40.560009+0000","last_became_active":"2023-12-19T18:16:27.832060+0000","last_became_peered":"2023-12-19T18:16:27.832060+0000","last_unstale":"2023-12-19T18:16:40.560009+0000","last_undegraded":"2023-12-19T18:16:40.560009+0000","last_fullsized":"2023-12-19T18:16:40.560009+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:17.432889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.e","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553537+0000","last_change":"2023-12-19T18:16:40.152827+0000","last_active":"2023-12-19T18:16:41.553537+0000","last_peered":"2023-12-19T18:16:41.553537+0000","last_clean":"2023-12-19T18:16:41.553537+0000","last_became_active":"2023-12-19T18:16:40.152559+0000","last_became_peered":"2023-12-19T18:16:40.152559+0000","last_unstale":"2023-12-19T18:16:41.553537+0000","last_undegraded":"2023-12-19T18:16:41.553537+0000","last_fullsized":"2023-12-19T18:16:41.553537+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:59.363074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553423+0000","last_change":"2023-12-19T18:16:40.572243+0000","last_active":"2023-12-19T18:16:41.553423+0000","last_peered":"2023-12-19T18:16:41.553423+0000","last_clean":"2023-12-19T18:16:41.553423+0000","last_became_active":"2023-12-19T18:16:40.571938+0000","last_became_peered":"2023-12-19T18:16:40.571938+0000","last_unstale":"2023-12-19T18:16:41.553423+0000","last_undegraded":"2023-12-19T18:16:41.553423+0000","last_fullsized":"2023-12-19T18:16:41.553423+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:50.735904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562479+0000","last_change":"2023-12-19T18:16:27.416654+0000","last_active":"2023-12-19T18:16:40.562479+0000","last_peered":"2023-12-19T18:16:40.562479+0000","last_clean":"2023-12-19T18:16:40.562479+0000","last_became_active":"2023-12-19T18:16:27.416388+0000","last_became_peered":"2023-12-19T18:16:27.416388+0000","last_unstale":"2023-12-19T18:16:40.562479+0000","last_undegraded":"2023-12-19T18:16:40.562479+0000","last_fullsized":"2023-12-19T18:16:40.562479+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:51.339148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547637+0000","last_change":"2023-12-19T18:16:27.816436+0000","last_active":"2023-12-19T18:16:40.547637+0000","last_peered":"2023-12-19T18:16:40.547637+0000","last_clean":"2023-12-19T18:16:40.547637+0000","last_became_active":"2023-12-19T18:16:27.816092+0000","last_became_peered":"2023-12-19T18:16:27.816092+0000","last_unstale":"2023-12-19T18:16:40.547637+0000","last_undegraded":"2023-12-19T18:16:40.547637+0000","last_fullsized":"2023-12-19T18:16:40.547637+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:07:26.713649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360338+0000","last_change":"2023-12-19T18:16:40.154797+0000","last_active":"2023-12-19T18:16:42.360338+0000","last_peered":"2023-12-19T18:16:42.360338+0000","last_clean":"2023-12-19T18:16:42.360338+0000","last_became_active":"2023-12-19T18:16:40.154490+0000","last_became_peered":"2023-12-19T18:16:40.154490+0000","last_unstale":"2023-12-19T18:16:42.360338+0000","last_undegraded":"2023-12-19T18:16:42.360338+0000","last_fullsized":"2023-12-19T18:16:42.360338+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:10:02.821689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551216+0000","last_change":"2023-12-19T18:16:40.573560+0000","last_active":"2023-12-19T18:16:41.551216+0000","last_peered":"2023-12-19T18:16:41.551216+0000","last_clean":"2023-12-19T18:16:41.551216+0000","last_became_active":"2023-12-19T18:16:40.573277+0000","last_became_peered":"2023-12-19T18:16:40.573277+0000","last_unstale":"2023-12-19T18:16:41.551216+0000","last_undegraded":"2023-12-19T18:16:41.551216+0000","last_fullsized":"2023-12-19T18:16:41.551216+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:23:53.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544525+0000","last_change":"2023-12-19T18:16:27.409426+0000","last_active":"2023-12-19T18:16:40.544525+0000","last_peered":"2023-12-19T18:16:40.544525+0000","last_clean":"2023-12-19T18:16:40.544525+0000","last_became_active":"2023-12-19T18:16:27.407151+0000","last_became_peered":"2023-12-19T18:16:27.407151+0000","last_unstale":"2023-12-19T18:16:40.544525+0000","last_undegraded":"2023-12-19T18:16:40.544525+0000","last_fullsized":"2023-12-19T18:16:40.544525+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:30:56.976667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550067+0000","last_change":"2023-12-19T18:16:27.823846+0000","last_active":"2023-12-19T18:16:40.550067+0000","last_peered":"2023-12-19T18:16:40.550067+0000","last_clean":"2023-12-19T18:16:40.550067+0000","last_became_active":"2023-12-19T18:16:27.823514+0000","last_became_peered":"2023-12-19T18:16:27.823514+0000","last_unstale":"2023-12-19T18:16:40.550067+0000","last_undegraded":"2023-12-19T18:16:40.550067+0000","last_fullsized":"2023-12-19T18:16:40.550067+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:20.691537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358069+0000","last_change":"2023-12-19T18:16:40.162626+0000","last_active":"2023-12-19T18:16:42.358069+0000","last_peered":"2023-12-19T18:16:42.358069+0000","last_clean":"2023-12-19T18:16:42.358069+0000","last_became_active":"2023-12-19T18:16:40.161257+0000","last_became_peered":"2023-12-19T18:16:40.161257+0000","last_unstale":"2023-12-19T18:16:42.358069+0000","last_undegraded":"2023-12-19T18:16:42.358069+0000","last_fullsized":"2023-12-19T18:16:42.358069+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:21:05.448555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357894+0000","last_change":"2023-12-19T18:16:40.558784+0000","last_active":"2023-12-19T18:16:42.357894+0000","last_peered":"2023-12-19T18:16:42.357894+0000","last_clean":"2023-12-19T18:16:42.357894+0000","last_became_active":"2023-12-19T18:16:40.556768+0000","last_became_peered":"2023-12-19T18:16:40.556768+0000","last_unstale":"2023-12-19T18:16:42.357894+0000","last_undegraded":"2023-12-19T18:16:42.357894+0000","last_fullsized":"2023-12-19T18:16:42.357894+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:14:19.106971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553891+0000","last_change":"2023-12-19T18:16:27.414971+0000","last_active":"2023-12-19T18:16:40.553891+0000","last_peered":"2023-12-19T18:16:40.553891+0000","last_clean":"2023-12-19T18:16:40.553891+0000","last_became_active":"2023-12-19T18:16:27.414662+0000","last_became_peered":"2023-12-19T18:16:27.414662+0000","last_unstale":"2023-12-19T18:16:40.553891+0000","last_undegraded":"2023-12-19T18:16:40.553891+0000","last_fullsized":"2023-12-19T18:16:40.553891+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:47.204637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558717+0000","last_change":"2023-12-19T18:16:27.814804+0000","last_active":"2023-12-19T18:16:40.558717+0000","last_peered":"2023-12-19T18:16:40.558717+0000","last_clean":"2023-12-19T18:16:40.558717+0000","last_became_active":"2023-12-19T18:16:27.814534+0000","last_became_peered":"2023-12-19T18:16:27.814534+0000","last_unstale":"2023-12-19T18:16:40.558717+0000","last_undegraded":"2023-12-19T18:16:40.558717+0000","last_fullsized":"2023-12-19T18:16:40.558717+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:45.690711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551146+0000","last_change":"2023-12-19T18:16:40.158484+0000","last_active":"2023-12-19T18:16:41.551146+0000","last_peered":"2023-12-19T18:16:41.551146+0000","last_clean":"2023-12-19T18:16:41.551146+0000","last_became_active":"2023-12-19T18:16:40.158205+0000","last_became_peered":"2023-12-19T18:16:40.158205+0000","last_unstale":"2023-12-19T18:16:41.551146+0000","last_undegraded":"2023-12-19T18:16:41.551146+0000","last_fullsized":"2023-12-19T18:16:41.551146+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:23.673873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550951+0000","last_change":"2023-12-19T18:16:40.555429+0000","last_active":"2023-12-19T18:16:41.550951+0000","last_peered":"2023-12-19T18:16:41.550951+0000","last_clean":"2023-12-19T18:16:41.550951+0000","last_became_active":"2023-12-19T18:16:40.555158+0000","last_became_peered":"2023-12-19T18:16:40.555158+0000","last_unstale":"2023-12-19T18:16:41.550951+0000","last_undegraded":"2023-12-19T18:16:41.550951+0000","last_fullsized":"2023-12-19T18:16:41.550951+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:53.628064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.10","version":"211'8","reported_seq":23,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560457+0000","last_change":"2023-12-19T18:16:27.415118+0000","last_active":"2023-12-19T18:16:40.560457+0000","last_peered":"2023-12-19T18:16:40.560457+0000","last_clean":"2023-12-19T18:16:40.560457+0000","last_became_active":"2023-12-19T18:16:27.414687+0000","last_became_peered":"2023-12-19T18:16:27.414687+0000","last_unstale":"2023-12-19T18:16:40.560457+0000","last_undegraded":"2023-12-19T18:16:40.560457+0000","last_fullsized":"2023-12-19T18:16:40.560457+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:16.723922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555219+0000","last_change":"2023-12-19T18:16:27.814277+0000","last_active":"2023-12-19T18:16:40.555219+0000","last_peered":"2023-12-19T18:16:40.555219+0000","last_clean":"2023-12-19T18:16:40.555219+0000","last_became_active":"2023-12-19T18:16:27.814023+0000","last_became_peered":"2023-12-19T18:16:27.814023+0000","last_unstale":"2023-12-19T18:16:40.555219+0000","last_undegraded":"2023-12-19T18:16:40.555219+0000","last_fullsized":"2023-12-19T18:16:40.555219+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:32.135133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554395+0000","last_change":"2023-12-19T18:16:40.154784+0000","last_active":"2023-12-19T18:16:41.554395+0000","last_peered":"2023-12-19T18:16:41.554395+0000","last_clean":"2023-12-19T18:16:41.554395+0000","last_became_active":"2023-12-19T18:16:40.154523+0000","last_became_peered":"2023-12-19T18:16:40.154523+0000","last_unstale":"2023-12-19T18:16:41.554395+0000","last_undegraded":"2023-12-19T18:16:41.554395+0000","last_fullsized":"2023-12-19T18:16:41.554395+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:14.215221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359308+0000","last_change":"2023-12-19T18:16:40.558066+0000","last_active":"2023-12-19T18:16:42.359308+0000","last_peered":"2023-12-19T18:16:42.359308+0000","last_clean":"2023-12-19T18:16:42.359308+0000","last_became_active":"2023-12-19T18:16:40.557771+0000","last_became_peered":"2023-12-19T18:16:40.557771+0000","last_unstale":"2023-12-19T18:16:42.359308+0000","last_undegraded":"2023-12-19T18:16:42.359308+0000","last_fullsized":"2023-12-19T18:16:42.359308+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:28.573589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543014+0000","last_change":"2023-12-19T18:16:27.403403+0000","last_active":"2023-12-19T18:16:40.543014+0000","last_peered":"2023-12-19T18:16:40.543014+0000","last_clean":"2023-12-19T18:16:40.543014+0000","last_became_active":"2023-12-19T18:16:27.403121+0000","last_became_peered":"2023-12-19T18:16:27.403121+0000","last_unstale":"2023-12-19T18:16:40.543014+0000","last_undegraded":"2023-12-19T18:16:40.543014+0000","last_fullsized":"2023-12-19T18:16:40.543014+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:36:58.560143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546582+0000","last_change":"2023-12-19T18:16:27.821814+0000","last_active":"2023-12-19T18:16:40.546582+0000","last_peered":"2023-12-19T18:16:40.546582+0000","last_clean":"2023-12-19T18:16:40.546582+0000","last_became_active":"2023-12-19T18:16:27.821519+0000","last_became_peered":"2023-12-19T18:16:27.821519+0000","last_unstale":"2023-12-19T18:16:40.546582+0000","last_undegraded":"2023-12-19T18:16:40.546582+0000","last_fullsized":"2023-12-19T18:16:40.546582+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:40.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.13","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554534+0000","last_change":"2023-12-19T18:16:40.160940+0000","last_active":"2023-12-19T18:16:41.554534+0000","last_peered":"2023-12-19T18:16:41.554534+0000","last_clean":"2023-12-19T18:16:41.554534+0000","last_became_active":"2023-12-19T18:16:40.160594+0000","last_became_peered":"2023-12-19T18:16:40.160594+0000","last_unstale":"2023-12-19T18:16:41.554534+0000","last_undegraded":"2023-12-19T18:16:41.554534+0000","last_fullsized":"2023-12-19T18:16:41.554534+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:01.011110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357451+0000","last_change":"2023-12-19T18:16:40.564547+0000","last_active":"2023-12-19T18:16:42.357451+0000","last_peered":"2023-12-19T18:16:42.357451+0000","last_clean":"2023-12-19T18:16:42.357451+0000","last_became_active":"2023-12-19T18:16:40.564300+0000","last_became_peered":"2023-12-19T18:16:40.564300+0000","last_unstale":"2023-12-19T18:16:42.357451+0000","last_undegraded":"2023-12-19T18:16:42.357451+0000","last_fullsized":"2023-12-19T18:16:42.357451+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:42.276098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556939+0000","last_change":"2023-12-19T18:16:27.417401+0000","last_active":"2023-12-19T18:16:40.556939+0000","last_peered":"2023-12-19T18:16:40.556939+0000","last_clean":"2023-12-19T18:16:40.556939+0000","last_became_active":"2023-12-19T18:16:27.416774+0000","last_became_peered":"2023-12-19T18:16:27.416774+0000","last_unstale":"2023-12-19T18:16:40.556939+0000","last_undegraded":"2023-12-19T18:16:40.556939+0000","last_fullsized":"2023-12-19T18:16:40.556939+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:03.620596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558146+0000","last_change":"2023-12-19T18:16:27.824516+0000","last_active":"2023-12-19T18:16:40.558146+0000","last_peered":"2023-12-19T18:16:40.558146+0000","last_clean":"2023-12-19T18:16:40.558146+0000","last_became_active":"2023-12-19T18:16:27.823143+0000","last_became_peered":"2023-12-19T18:16:27.823143+0000","last_unstale":"2023-12-19T18:16:40.558146+0000","last_undegraded":"2023-12-19T18:16:40.558146+0000","last_fullsized":"2023-12-19T18:16:40.558146+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:52.341930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"215'8","reported_seq":19,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357609+0000","last_change":"2023-12-19T18:16:40.161467+0000","last_active":"2023-12-19T18:16:42.357609+0000","last_peered":"2023-12-19T18:16:42.357609+0000","last_clean":"2023-12-19T18:16:42.357609+0000","last_became_active":"2023-12-19T18:16:40.161143+0000","last_became_peered":"2023-12-19T18:16:40.161143+0000","last_unstale":"2023-12-19T18:16:42.357609+0000","last_undegraded":"2023-12-19T18:16:42.357609+0000","last_fullsized":"2023-12-19T18:16:42.357609+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:21:09.532919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357722+0000","last_change":"2023-12-19T18:16:40.571041+0000","last_active":"2023-12-19T18:16:42.357722+0000","last_peered":"2023-12-19T18:16:42.357722+0000","last_clean":"2023-12-19T18:16:42.357722+0000","last_became_active":"2023-12-19T18:16:40.570705+0000","last_became_peered":"2023-12-19T18:16:40.570705+0000","last_unstale":"2023-12-19T18:16:42.357722+0000","last_undegraded":"2023-12-19T18:16:42.357722+0000","last_fullsized":"2023-12-19T18:16:42.357722+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:43.042802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.13","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560853+0000","last_change":"2023-12-19T18:16:27.415000+0000","last_active":"2023-12-19T18:16:40.560853+0000","last_peered":"2023-12-19T18:16:40.560853+0000","last_clean":"2023-12-19T18:16:40.560853+0000","last_became_active":"2023-12-19T18:16:27.414679+0000","last_became_peered":"2023-12-19T18:16:27.414679+0000","last_unstale":"2023-12-19T18:16:40.560853+0000","last_undegraded":"2023-12-19T18:16:40.560853+0000","last_fullsized":"2023-12-19T18:16:40.560853+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:50.187832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554789+0000","last_change":"2023-12-19T18:16:27.824833+0000","last_active":"2023-12-19T18:16:40.554789+0000","last_peered":"2023-12-19T18:16:40.554789+0000","last_clean":"2023-12-19T18:16:40.554789+0000","last_became_active":"2023-12-19T18:16:27.824487+0000","last_became_peered":"2023-12-19T18:16:27.824487+0000","last_unstale":"2023-12-19T18:16:40.554789+0000","last_undegraded":"2023-12-19T18:16:40.554789+0000","last_fullsized":"2023-12-19T18:16:40.554789+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550312+0000","last_change":"2023-12-19T18:16:40.150872+0000","last_active":"2023-12-19T18:16:41.550312+0000","last_peered":"2023-12-19T18:16:41.550312+0000","last_clean":"2023-12-19T18:16:41.550312+0000","last_became_active":"2023-12-19T18:16:40.150586+0000","last_became_peered":"2023-12-19T18:16:40.150586+0000","last_unstale":"2023-12-19T18:16:41.550312+0000","last_undegraded":"2023-12-19T18:16:41.550312+0000","last_fullsized":"2023-12-19T18:16:41.550312+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:55.343366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358620+0000","last_change":"2023-12-19T18:16:40.571454+0000","last_active":"2023-12-19T18:16:42.358620+0000","last_peered":"2023-12-19T18:16:42.358620+0000","last_clean":"2023-12-19T18:16:42.358620+0000","last_became_active":"2023-12-19T18:16:40.570299+0000","last_became_peered":"2023-12-19T18:16:40.570299+0000","last_unstale":"2023-12-19T18:16:42.358620+0000","last_undegraded":"2023-12-19T18:16:42.358620+0000","last_fullsized":"2023-12-19T18:16:42.358620+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:31:09.517418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"212'8","reported_seq":23,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553799+0000","last_change":"2023-12-19T18:16:27.414573+0000","last_active":"2023-12-19T18:16:40.553799+0000","last_peered":"2023-12-19T18:16:40.553799+0000","last_clean":"2023-12-19T18:16:40.553799+0000","last_became_active":"2023-12-19T18:16:27.414268+0000","last_became_peered":"2023-12-19T18:16:27.414268+0000","last_unstale":"2023-12-19T18:16:40.553799+0000","last_undegraded":"2023-12-19T18:16:40.553799+0000","last_fullsized":"2023-12-19T18:16:40.553799+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:11.117214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555559+0000","last_change":"2023-12-19T18:16:27.829311+0000","last_active":"2023-12-19T18:16:40.555559+0000","last_peered":"2023-12-19T18:16:40.555559+0000","last_clean":"2023-12-19T18:16:40.555559+0000","last_became_active":"2023-12-19T18:16:27.828953+0000","last_became_peered":"2023-12-19T18:16:27.828953+0000","last_unstale":"2023-12-19T18:16:40.555559+0000","last_undegraded":"2023-12-19T18:16:40.555559+0000","last_fullsized":"2023-12-19T18:16:40.555559+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:23.417710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554767+0000","last_change":"2023-12-19T18:16:40.160698+0000","last_active":"2023-12-19T18:16:41.554767+0000","last_peered":"2023-12-19T18:16:41.554767+0000","last_clean":"2023-12-19T18:16:41.554767+0000","last_became_active":"2023-12-19T18:16:40.160406+0000","last_became_peered":"2023-12-19T18:16:40.160406+0000","last_unstale":"2023-12-19T18:16:41.554767+0000","last_undegraded":"2023-12-19T18:16:41.554767+0000","last_fullsized":"2023-12-19T18:16:41.554767+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:10.441625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.075170+0000","last_change":"2023-12-19T18:16:40.579076+0000","last_active":"2023-12-19T18:16:42.075170+0000","last_peered":"2023-12-19T18:16:42.075170+0000","last_clean":"2023-12-19T18:16:42.075170+0000","last_became_active":"2023-12-19T18:16:40.578641+0000","last_became_peered":"2023-12-19T18:16:40.578641+0000","last_unstale":"2023-12-19T18:16:42.075170+0000","last_undegraded":"2023-12-19T18:16:42.075170+0000","last_fullsized":"2023-12-19T18:16:42.075170+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:33:08.245627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.15","version":"211'3","reported_seq":18,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555258+0000","last_change":"2023-12-19T18:16:27.409442+0000","last_active":"2023-12-19T18:16:40.555258+0000","last_peered":"2023-12-19T18:16:40.555258+0000","last_clean":"2023-12-19T18:16:40.555258+0000","last_became_active":"2023-12-19T18:16:27.409094+0000","last_became_peered":"2023-12-19T18:16:27.409094+0000","last_unstale":"2023-12-19T18:16:40.555258+0000","last_undegraded":"2023-12-19T18:16:40.555258+0000","last_fullsized":"2023-12-19T18:16:40.555258+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:18.713070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562498+0000","last_change":"2023-12-19T18:16:27.811752+0000","last_active":"2023-12-19T18:16:40.562498+0000","last_peered":"2023-12-19T18:16:40.562498+0000","last_clean":"2023-12-19T18:16:40.562498+0000","last_became_active":"2023-12-19T18:16:27.811273+0000","last_became_peered":"2023-12-19T18:16:27.811273+0000","last_unstale":"2023-12-19T18:16:40.562498+0000","last_undegraded":"2023-12-19T18:16:40.562498+0000","last_fullsized":"2023-12-19T18:16:40.562498+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:23.609660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360336+0000","last_change":"2023-12-19T18:16:40.162801+0000","last_active":"2023-12-19T18:16:42.360336+0000","last_peered":"2023-12-19T18:16:42.360336+0000","last_clean":"2023-12-19T18:16:42.360336+0000","last_became_active":"2023-12-19T18:16:40.162367+0000","last_became_peered":"2023-12-19T18:16:40.162367+0000","last_unstale":"2023-12-19T18:16:42.360336+0000","last_undegraded":"2023-12-19T18:16:42.360336+0000","last_fullsized":"2023-12-19T18:16:42.360336+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:08:09.383590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359637+0000","last_change":"2023-12-19T18:16:40.570887+0000","last_active":"2023-12-19T18:16:42.359637+0000","last_peered":"2023-12-19T18:16:42.359637+0000","last_clean":"2023-12-19T18:16:42.359637+0000","last_became_active":"2023-12-19T18:16:40.570475+0000","last_became_peered":"2023-12-19T18:16:40.570475+0000","last_unstale":"2023-12-19T18:16:42.359637+0000","last_undegraded":"2023-12-19T18:16:42.359637+0000","last_fullsized":"2023-12-19T18:16:42.359637+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:53:00.119855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549599+0000","last_change":"2023-12-19T18:16:27.410603+0000","last_active":"2023-12-19T18:16:40.549599+0000","last_peered":"2023-12-19T18:16:40.549599+0000","last_clean":"2023-12-19T18:16:40.549599+0000","last_became_active":"2023-12-19T18:16:27.409260+0000","last_became_peered":"2023-12-19T18:16:27.409260+0000","last_unstale":"2023-12-19T18:16:40.549599+0000","last_undegraded":"2023-12-19T18:16:40.549599+0000","last_fullsized":"2023-12-19T18:16:40.549599+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:06.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543878+0000","last_change":"2023-12-19T18:16:27.835564+0000","last_active":"2023-12-19T18:16:40.543878+0000","last_peered":"2023-12-19T18:16:40.543878+0000","last_clean":"2023-12-19T18:16:40.543878+0000","last_became_active":"2023-12-19T18:16:27.835319+0000","last_became_peered":"2023-12-19T18:16:27.835319+0000","last_unstale":"2023-12-19T18:16:40.543878+0000","last_undegraded":"2023-12-19T18:16:40.543878+0000","last_fullsized":"2023-12-19T18:16:40.543878+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:10:35.037185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"215'6","reported_seq":17,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074989+0000","last_change":"2023-12-19T18:16:40.163664+0000","last_active":"2023-12-19T18:16:42.074989+0000","last_peered":"2023-12-19T18:16:42.074989+0000","last_clean":"2023-12-19T18:16:42.074989+0000","last_became_active":"2023-12-19T18:16:40.163301+0000","last_became_peered":"2023-12-19T18:16:40.163301+0000","last_unstale":"2023-12-19T18:16:42.074989+0000","last_undegraded":"2023-12-19T18:16:42.074989+0000","last_fullsized":"2023-12-19T18:16:42.074989+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:33.099709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074835+0000","last_change":"2023-12-19T18:16:40.581101+0000","last_active":"2023-12-19T18:16:42.074835+0000","last_peered":"2023-12-19T18:16:42.074835+0000","last_clean":"2023-12-19T18:16:42.074835+0000","last_became_active":"2023-12-19T18:16:40.579272+0000","last_became_peered":"2023-12-19T18:16:40.579272+0000","last_unstale":"2023-12-19T18:16:42.074835+0000","last_undegraded":"2023-12-19T18:16:42.074835+0000","last_fullsized":"2023-12-19T18:16:42.074835+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:12:36.065977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565329+0000","last_change":"2023-12-19T18:16:27.405556+0000","last_active":"2023-12-19T18:16:40.565329+0000","last_peered":"2023-12-19T18:16:40.565329+0000","last_clean":"2023-12-19T18:16:40.565329+0000","last_became_active":"2023-12-19T18:16:27.404804+0000","last_became_peered":"2023-12-19T18:16:27.404804+0000","last_unstale":"2023-12-19T18:16:40.565329+0000","last_undegraded":"2023-12-19T18:16:40.565329+0000","last_fullsized":"2023-12-19T18:16:40.565329+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:05.395130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551274+0000","last_change":"2023-12-19T18:16:27.830538+0000","last_active":"2023-12-19T18:16:40.551274+0000","last_peered":"2023-12-19T18:16:40.551274+0000","last_clean":"2023-12-19T18:16:40.551274+0000","last_became_active":"2023-12-19T18:16:27.828974+0000","last_became_peered":"2023-12-19T18:16:27.828974+0000","last_unstale":"2023-12-19T18:16:40.551274+0000","last_undegraded":"2023-12-19T18:16:40.551274+0000","last_fullsized":"2023-12-19T18:16:40.551274+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:36.961741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.15","version":"215'3","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551392+0000","last_change":"2023-12-19T18:16:40.151095+0000","last_active":"2023-12-19T18:16:41.551392+0000","last_peered":"2023-12-19T18:16:41.551392+0000","last_clean":"2023-12-19T18:16:41.551392+0000","last_became_active":"2023-12-19T18:16:40.150609+0000","last_became_peered":"2023-12-19T18:16:40.150609+0000","last_unstale":"2023-12-19T18:16:41.551392+0000","last_undegraded":"2023-12-19T18:16:41.551392+0000","last_fullsized":"2023-12-19T18:16:41.551392+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:15.058778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074922+0000","last_change":"2023-12-19T18:16:40.580029+0000","last_active":"2023-12-19T18:16:42.074922+0000","last_peered":"2023-12-19T18:16:42.074922+0000","last_clean":"2023-12-19T18:16:42.074922+0000","last_became_active":"2023-12-19T18:16:40.579698+0000","last_became_peered":"2023-12-19T18:16:40.579698+0000","last_unstale":"2023-12-19T18:16:42.074922+0000","last_undegraded":"2023-12-19T18:16:42.074922+0000","last_fullsized":"2023-12-19T18:16:42.074922+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:26:54.889491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560368+0000","last_change":"2023-12-19T18:16:27.413444+0000","last_active":"2023-12-19T18:16:40.560368+0000","last_peered":"2023-12-19T18:16:40.560368+0000","last_clean":"2023-12-19T18:16:40.560368+0000","last_became_active":"2023-12-19T18:16:27.413066+0000","last_became_peered":"2023-12-19T18:16:27.413066+0000","last_unstale":"2023-12-19T18:16:40.560368+0000","last_undegraded":"2023-12-19T18:16:40.560368+0000","last_fullsized":"2023-12-19T18:16:40.560368+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:57.997657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558113+0000","last_change":"2023-12-19T18:16:27.816735+0000","last_active":"2023-12-19T18:16:40.558113+0000","last_peered":"2023-12-19T18:16:40.558113+0000","last_clean":"2023-12-19T18:16:40.558113+0000","last_became_active":"2023-12-19T18:16:27.816436+0000","last_became_peered":"2023-12-19T18:16:27.816436+0000","last_unstale":"2023-12-19T18:16:40.558113+0000","last_undegraded":"2023-12-19T18:16:40.558113+0000","last_fullsized":"2023-12-19T18:16:40.558113+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:37.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1a","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359740+0000","last_change":"2023-12-19T18:16:40.161453+0000","last_active":"2023-12-19T18:16:42.359740+0000","last_peered":"2023-12-19T18:16:42.359740+0000","last_clean":"2023-12-19T18:16:42.359740+0000","last_became_active":"2023-12-19T18:16:40.161191+0000","last_became_peered":"2023-12-19T18:16:40.161191+0000","last_unstale":"2023-12-19T18:16:42.359740+0000","last_undegraded":"2023-12-19T18:16:42.359740+0000","last_fullsized":"2023-12-19T18:16:42.359740+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:13.660363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074624+0000","last_change":"2023-12-19T18:16:40.578142+0000","last_active":"2023-12-19T18:16:42.074624+0000","last_peered":"2023-12-19T18:16:42.074624+0000","last_clean":"2023-12-19T18:16:42.074624+0000","last_became_active":"2023-12-19T18:16:40.577655+0000","last_became_peered":"2023-12-19T18:16:40.577655+0000","last_unstale":"2023-12-19T18:16:42.074624+0000","last_undegraded":"2023-12-19T18:16:42.074624+0000","last_fullsized":"2023-12-19T18:16:42.074624+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:41.805677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544353+0000","last_change":"2023-12-19T18:16:27.414372+0000","last_active":"2023-12-19T18:16:40.544353+0000","last_peered":"2023-12-19T18:16:40.544353+0000","last_clean":"2023-12-19T18:16:40.544353+0000","last_became_active":"2023-12-19T18:16:27.414088+0000","last_became_peered":"2023-12-19T18:16:27.414088+0000","last_unstale":"2023-12-19T18:16:40.544353+0000","last_undegraded":"2023-12-19T18:16:40.544353+0000","last_fullsized":"2023-12-19T18:16:40.544353+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:18.945766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543503+0000","last_change":"2023-12-19T18:16:27.827386+0000","last_active":"2023-12-19T18:16:40.543503+0000","last_peered":"2023-12-19T18:16:40.543503+0000","last_clean":"2023-12-19T18:16:40.543503+0000","last_became_active":"2023-12-19T18:16:27.827086+0000","last_became_peered":"2023-12-19T18:16:27.827086+0000","last_unstale":"2023-12-19T18:16:40.543503+0000","last_undegraded":"2023-12-19T18:16:40.543503+0000","last_fullsized":"2023-12-19T18:16:40.543503+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:37.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554117+0000","last_change":"2023-12-19T18:16:40.156738+0000","last_active":"2023-12-19T18:16:41.554117+0000","last_peered":"2023-12-19T18:16:41.554117+0000","last_clean":"2023-12-19T18:16:41.554117+0000","last_became_active":"2023-12-19T18:16:40.156488+0000","last_became_peered":"2023-12-19T18:16:40.156488+0000","last_unstale":"2023-12-19T18:16:41.554117+0000","last_undegraded":"2023-12-19T18:16:41.554117+0000","last_fullsized":"2023-12-19T18:16:41.554117+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:48.695934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358850+0000","last_change":"2023-12-19T18:16:40.575709+0000","last_active":"2023-12-19T18:16:42.358850+0000","last_peered":"2023-12-19T18:16:42.358850+0000","last_clean":"2023-12-19T18:16:42.358850+0000","last_became_active":"2023-12-19T18:16:40.575453+0000","last_became_peered":"2023-12-19T18:16:40.575453+0000","last_unstale":"2023-12-19T18:16:42.358850+0000","last_undegraded":"2023-12-19T18:16:42.358850+0000","last_fullsized":"2023-12-19T18:16:42.358850+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:44.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1b","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551641+0000","last_change":"2023-12-19T18:16:27.415036+0000","last_active":"2023-12-19T18:16:40.551641+0000","last_peered":"2023-12-19T18:16:40.551641+0000","last_clean":"2023-12-19T18:16:40.551641+0000","last_became_active":"2023-12-19T18:16:27.414747+0000","last_became_peered":"2023-12-19T18:16:27.414747+0000","last_unstale":"2023-12-19T18:16:40.551641+0000","last_undegraded":"2023-12-19T18:16:40.551641+0000","last_fullsized":"2023-12-19T18:16:40.551641+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:03.064138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556794+0000","last_change":"2023-12-19T18:16:27.828398+0000","last_active":"2023-12-19T18:16:40.556794+0000","last_peered":"2023-12-19T18:16:40.556794+0000","last_clean":"2023-12-19T18:16:40.556794+0000","last_became_active":"2023-12-19T18:16:27.828082+0000","last_became_peered":"2023-12-19T18:16:27.828082+0000","last_unstale":"2023-12-19T18:16:40.556794+0000","last_undegraded":"2023-12-19T18:16:40.556794+0000","last_fullsized":"2023-12-19T18:16:40.556794+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:42:30.998150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358258+0000","last_change":"2023-12-19T18:16:40.159943+0000","last_active":"2023-12-19T18:16:42.358258+0000","last_peered":"2023-12-19T18:16:42.358258+0000","last_clean":"2023-12-19T18:16:42.358258+0000","last_became_active":"2023-12-19T18:16:40.159652+0000","last_became_peered":"2023-12-19T18:16:40.159652+0000","last_unstale":"2023-12-19T18:16:42.358258+0000","last_undegraded":"2023-12-19T18:16:42.358258+0000","last_fullsized":"2023-12-19T18:16:42.358258+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:02:40.928028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.555138+0000","last_change":"2023-12-19T18:16:40.573382+0000","last_active":"2023-12-19T18:16:41.555138+0000","last_peered":"2023-12-19T18:16:41.555138+0000","last_clean":"2023-12-19T18:16:41.555138+0000","last_became_active":"2023-12-19T18:16:40.573092+0000","last_became_peered":"2023-12-19T18:16:40.573092+0000","last_unstale":"2023-12-19T18:16:41.555138+0000","last_undegraded":"2023-12-19T18:16:41.555138+0000","last_fullsized":"2023-12-19T18:16:41.555138+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:10:47.778609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553461+0000","last_change":"2023-12-19T18:16:40.163131+0000","last_active":"2023-12-19T18:16:41.553461+0000","last_peered":"2023-12-19T18:16:41.553461+0000","last_clean":"2023-12-19T18:16:41.553461+0000","last_became_active":"2023-12-19T18:16:40.162621+0000","last_became_peered":"2023-12-19T18:16:40.162621+0000","last_unstale":"2023-12-19T18:16:41.553461+0000","last_undegraded":"2023-12-19T18:16:41.553461+0000","last_fullsized":"2023-12-19T18:16:41.553461+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:57.979757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559871+0000","last_change":"2023-12-19T18:16:27.409441+0000","last_active":"2023-12-19T18:16:40.559871+0000","last_peered":"2023-12-19T18:16:40.559871+0000","last_clean":"2023-12-19T18:16:40.559871+0000","last_became_active":"2023-12-19T18:16:27.408438+0000","last_became_peered":"2023-12-19T18:16:27.408438+0000","last_unstale":"2023-12-19T18:16:40.559871+0000","last_undegraded":"2023-12-19T18:16:40.559871+0000","last_fullsized":"2023-12-19T18:16:40.559871+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:40.980300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546499+0000","last_change":"2023-12-19T18:16:27.814063+0000","last_active":"2023-12-19T18:16:40.546499+0000","last_peered":"2023-12-19T18:16:40.546499+0000","last_clean":"2023-12-19T18:16:40.546499+0000","last_became_active":"2023-12-19T18:16:27.813564+0000","last_became_peered":"2023-12-19T18:16:27.813564+0000","last_unstale":"2023-12-19T18:16:40.546499+0000","last_undegraded":"2023-12-19T18:16:40.546499+0000","last_fullsized":"2023-12-19T18:16:40.546499+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:36.733148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553348+0000","last_change":"2023-12-19T18:16:40.564402+0000","last_active":"2023-12-19T18:16:41.553348+0000","last_peered":"2023-12-19T18:16:41.553348+0000","last_clean":"2023-12-19T18:16:41.553348+0000","last_became_active":"2023-12-19T18:16:40.561338+0000","last_became_peered":"2023-12-19T18:16:40.561338+0000","last_unstale":"2023-12-19T18:16:41.553348+0000","last_undegraded":"2023-12-19T18:16:41.553348+0000","last_fullsized":"2023-12-19T18:16:41.553348+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:37:11.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"215'4","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857590+0000","last_change":"2023-12-19T18:16:40.152799+0000","last_active":"2023-12-19T18:16:41.857590+0000","last_peered":"2023-12-19T18:16:41.857590+0000","last_clean":"2023-12-19T18:16:41.857590+0000","last_became_active":"2023-12-19T18:16:40.152482+0000","last_became_peered":"2023-12-19T18:16:40.152482+0000","last_unstale":"2023-12-19T18:16:41.857590+0000","last_undegraded":"2023-12-19T18:16:41.857590+0000","last_fullsized":"2023-12-19T18:16:41.857590+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:23.854189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1d","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548666+0000","last_change":"2023-12-19T18:16:27.415013+0000","last_active":"2023-12-19T18:16:40.548666+0000","last_peered":"2023-12-19T18:16:40.548666+0000","last_clean":"2023-12-19T18:16:40.548666+0000","last_became_active":"2023-12-19T18:16:27.414582+0000","last_became_peered":"2023-12-19T18:16:27.414582+0000","last_unstale":"2023-12-19T18:16:40.548666+0000","last_undegraded":"2023-12-19T18:16:40.548666+0000","last_fullsized":"2023-12-19T18:16:40.548666+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:34:01.457288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562417+0000","last_change":"2023-12-19T18:16:27.829031+0000","last_active":"2023-12-19T18:16:40.562417+0000","last_peered":"2023-12-19T18:16:40.562417+0000","last_clean":"2023-12-19T18:16:40.562417+0000","last_became_active":"2023-12-19T18:16:27.826634+0000","last_became_peered":"2023-12-19T18:16:27.826634+0000","last_unstale":"2023-12-19T18:16:40.562417+0000","last_undegraded":"2023-12-19T18:16:40.562417+0000","last_fullsized":"2023-12-19T18:16:40.562417+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:23.429228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551066+0000","last_change":"2023-12-19T18:16:40.570487+0000","last_active":"2023-12-19T18:16:41.551066+0000","last_peered":"2023-12-19T18:16:41.551066+0000","last_clean":"2023-12-19T18:16:41.551066+0000","last_became_active":"2023-12-19T18:16:40.569839+0000","last_became_peered":"2023-12-19T18:16:40.569839+0000","last_unstale":"2023-12-19T18:16:41.551066+0000","last_undegraded":"2023-12-19T18:16:41.551066+0000","last_fullsized":"2023-12-19T18:16:41.551066+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:59.414695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548366+0000","last_change":"2023-12-19T18:16:27.407669+0000","last_active":"2023-12-19T18:16:40.548366+0000","last_peered":"2023-12-19T18:16:40.548366+0000","last_clean":"2023-12-19T18:16:40.548366+0000","last_became_active":"2023-12-19T18:16:27.407150+0000","last_became_peered":"2023-12-19T18:16:27.407150+0000","last_unstale":"2023-12-19T18:16:40.548366+0000","last_undegraded":"2023-12-19T18:16:40.548366+0000","last_fullsized":"2023-12-19T18:16:40.548366+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:59.675090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558634+0000","last_change":"2023-12-19T18:16:27.830766+0000","last_active":"2023-12-19T18:16:40.558634+0000","last_peered":"2023-12-19T18:16:40.558634+0000","last_clean":"2023-12-19T18:16:40.558634+0000","last_became_active":"2023-12-19T18:16:27.830220+0000","last_became_peered":"2023-12-19T18:16:27.830220+0000","last_unstale":"2023-12-19T18:16:40.558634+0000","last_undegraded":"2023-12-19T18:16:40.558634+0000","last_fullsized":"2023-12-19T18:16:40.558634+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:03:17.772391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1c","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.075108+0000","last_change":"2023-12-19T18:16:40.164718+0000","last_active":"2023-12-19T18:16:42.075108+0000","last_peered":"2023-12-19T18:16:42.075108+0000","last_clean":"2023-12-19T18:16:42.075108+0000","last_became_active":"2023-12-19T18:16:40.164413+0000","last_became_peered":"2023-12-19T18:16:40.164413+0000","last_unstale":"2023-12-19T18:16:42.075108+0000","last_undegraded":"2023-12-19T18:16:42.075108+0000","last_fullsized":"2023-12-19T18:16:42.075108+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:49.453705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359497+0000","last_change":"2023-12-19T18:16:40.565564+0000","last_active":"2023-12-19T18:16:42.359497+0000","last_peered":"2023-12-19T18:16:42.359497+0000","last_clean":"2023-12-19T18:16:42.359497+0000","last_became_active":"2023-12-19T18:16:40.565300+0000","last_became_peered":"2023-12-19T18:16:40.565300+0000","last_unstale":"2023-12-19T18:16:42.359497+0000","last_undegraded":"2023-12-19T18:16:42.359497+0000","last_fullsized":"2023-12-19T18:16:42.359497+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:14.849896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1f","version":"211'4","reported_seq":19,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553798+0000","last_change":"2023-12-19T18:16:27.414364+0000","last_active":"2023-12-19T18:16:40.553798+0000","last_peered":"2023-12-19T18:16:40.553798+0000","last_clean":"2023-12-19T18:16:40.553798+0000","last_became_active":"2023-12-19T18:16:27.413648+0000","last_became_peered":"2023-12-19T18:16:27.413648+0000","last_unstale":"2023-12-19T18:16:40.553798+0000","last_undegraded":"2023-12-19T18:16:40.553798+0000","last_fullsized":"2023-12-19T18:16:40.553798+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:27:32.703908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544529+0000","last_change":"2023-12-19T18:16:27.830378+0000","last_active":"2023-12-19T18:16:40.544529+0000","last_peered":"2023-12-19T18:16:40.544529+0000","last_clean":"2023-12-19T18:16:40.544529+0000","last_became_active":"2023-12-19T18:16:27.830122+0000","last_became_peered":"2023-12-19T18:16:27.830122+0000","last_unstale":"2023-12-19T18:16:40.544529+0000","last_undegraded":"2023-12-19T18:16:40.544529+0000","last_fullsized":"2023-12-19T18:16:40.544529+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:21.365516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1d","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358451+0000","last_change":"2023-12-19T18:16:40.162566+0000","last_active":"2023-12-19T18:16:42.358451+0000","last_peered":"2023-12-19T18:16:42.358451+0000","last_clean":"2023-12-19T18:16:42.358451+0000","last_became_active":"2023-12-19T18:16:40.162255+0000","last_became_peered":"2023-12-19T18:16:40.162255+0000","last_unstale":"2023-12-19T18:16:42.358451+0000","last_undegraded":"2023-12-19T18:16:42.358451+0000","last_fullsized":"2023-12-19T18:16:42.358451+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:20.530183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552668+0000","last_change":"2023-12-19T18:16:40.570029+0000","last_active":"2023-12-19T18:16:41.552668+0000","last_peered":"2023-12-19T18:16:41.552668+0000","last_clean":"2023-12-19T18:16:41.552668+0000","last_became_active":"2023-12-19T18:16:40.569731+0000","last_became_peered":"2023-12-19T18:16:40.569731+0000","last_unstale":"2023-12-19T18:16:41.552668+0000","last_undegraded":"2023-12-19T18:16:41.552668+0000","last_fullsized":"2023-12-19T18:16:41.552668+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:25:46.635350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550877+0000","last_change":"2023-12-19T18:16:27.831030+0000","last_active":"2023-12-19T18:16:40.550877+0000","last_peered":"2023-12-19T18:16:40.550877+0000","last_clean":"2023-12-19T18:16:40.550877+0000","last_became_active":"2023-12-19T18:16:27.830764+0000","last_became_peered":"2023-12-19T18:16:27.830764+0000","last_unstale":"2023-12-19T18:16:40.550877+0000","last_undegraded":"2023-12-19T18:16:40.550877+0000","last_fullsized":"2023-12-19T18:16:40.550877+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:11.685663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857085+0000","last_change":"2023-12-19T18:16:40.569158+0000","last_active":"2023-12-19T18:16:41.857085+0000","last_peered":"2023-12-19T18:16:41.857085+0000","last_clean":"2023-12-19T18:16:41.857085+0000","last_became_active":"2023-12-19T18:16:40.568894+0000","last_became_peered":"2023-12-19T18:16:40.568894+0000","last_unstale":"2023-12-19T18:16:41.857085+0000","last_undegraded":"2023-12-19T18:16:41.857085+0000","last_fullsized":"2023-12-19T18:16:41.857085+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:02.018340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553603+0000","last_change":"2023-12-19T18:16:27.833518+0000","last_active":"2023-12-19T18:16:40.553603+0000","last_peered":"2023-12-19T18:16:40.553603+0000","last_clean":"2023-12-19T18:16:40.553603+0000","last_became_active":"2023-12-19T18:16:27.833251+0000","last_became_peered":"2023-12-19T18:16:27.833251+0000","last_unstale":"2023-12-19T18:16:40.553603+0000","last_undegraded":"2023-12-19T18:16:40.553603+0000","last_fullsized":"2023-12-19T18:16:40.553603+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:49.276917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357285+0000","last_change":"2023-12-19T18:16:40.563610+0000","last_active":"2023-12-19T18:16:42.357285+0000","last_peered":"2023-12-19T18:16:42.357285+0000","last_clean":"2023-12-19T18:16:42.357285+0000","last_became_active":"2023-12-19T18:16:40.563300+0000","last_became_peered":"2023-12-19T18:16:40.563300+0000","last_unstale":"2023-12-19T18:16:42.357285+0000","last_undegraded":"2023-12-19T18:16:42.357285+0000","last_fullsized":"2023-12-19T18:16:42.357285+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:28:48.518493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547548+0000","last_change":"2023-12-19T18:16:27.825014+0000","last_active":"2023-12-19T18:16:40.547548+0000","last_peered":"2023-12-19T18:16:40.547548+0000","last_clean":"2023-12-19T18:16:40.547548+0000","last_became_active":"2023-12-19T18:16:27.822140+0000","last_became_peered":"2023-12-19T18:16:27.822140+0000","last_unstale":"2023-12-19T18:16:40.547548+0000","last_undegraded":"2023-12-19T18:16:40.547548+0000","last_fullsized":"2023-12-19T18:16:40.547548+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:36.972809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550313+0000","last_change":"2023-12-19T18:16:40.576594+0000","last_active":"2023-12-19T18:16:41.550313+0000","last_peered":"2023-12-19T18:16:41.550313+0000","last_clean":"2023-12-19T18:16:41.550313+0000","last_became_active":"2023-12-19T18:16:40.576304+0000","last_became_peered":"2023-12-19T18:16:40.576304+0000","last_unstale":"2023-12-19T18:16:41.550313+0000","last_undegraded":"2023-12-19T18:16:41.550313+0000","last_fullsized":"2023-12-19T18:16:41.550313+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:58.373923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561436+0000","last_change":"2023-12-19T18:16:27.826500+0000","last_active":"2023-12-19T18:16:40.561436+0000","last_peered":"2023-12-19T18:16:40.561436+0000","last_clean":"2023-12-19T18:16:40.561436+0000","last_became_active":"2023-12-19T18:16:27.826216+0000","last_became_peered":"2023-12-19T18:16:27.826216+0000","last_unstale":"2023-12-19T18:16:40.561436+0000","last_undegraded":"2023-12-19T18:16:40.561436+0000","last_fullsized":"2023-12-19T18:16:40.561436+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:33.032362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359854+0000","last_change":"2023-12-19T18:16:40.565942+0000","last_active":"2023-12-19T18:16:42.359854+0000","last_peered":"2023-12-19T18:16:42.359854+0000","last_clean":"2023-12-19T18:16:42.359854+0000","last_became_active":"2023-12-19T18:16:40.565557+0000","last_became_peered":"2023-12-19T18:16:40.565557+0000","last_unstale":"2023-12-19T18:16:42.359854+0000","last_undegraded":"2023-12-19T18:16:42.359854+0000","last_fullsized":"2023-12-19T18:16:42.359854+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:23.839152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548841+0000","last_change":"2023-12-19T18:16:27.830103+0000","last_active":"2023-12-19T18:16:40.548841+0000","last_peered":"2023-12-19T18:16:40.548841+0000","last_clean":"2023-12-19T18:16:40.548841+0000","last_became_active":"2023-12-19T18:16:27.829444+0000","last_became_peered":"2023-12-19T18:16:27.829444+0000","last_unstale":"2023-12-19T18:16:40.548841+0000","last_undegraded":"2023-12-19T18:16:40.548841+0000","last_fullsized":"2023-12-19T18:16:40.548841+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:50:29.947932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553462+0000","last_change":"2023-12-19T18:16:40.563980+0000","last_active":"2023-12-19T18:16:41.553462+0000","last_peered":"2023-12-19T18:16:41.553462+0000","last_clean":"2023-12-19T18:16:41.553462+0000","last_became_active":"2023-12-19T18:16:40.563683+0000","last_became_peered":"2023-12-19T18:16:40.563683+0000","last_unstale":"2023-12-19T18:16:41.553462+0000","last_undegraded":"2023-12-19T18:16:41.553462+0000","last_fullsized":"2023-12-19T18:16:41.553462+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:36.170652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548463+0000","last_change":"2023-12-19T18:16:27.820913+0000","last_active":"2023-12-19T18:16:40.548463+0000","last_peered":"2023-12-19T18:16:40.548463+0000","last_clean":"2023-12-19T18:16:40.548463+0000","last_became_active":"2023-12-19T18:16:27.820634+0000","last_became_peered":"2023-12-19T18:16:27.820634+0000","last_unstale":"2023-12-19T18:16:40.548463+0000","last_undegraded":"2023-12-19T18:16:40.548463+0000","last_fullsized":"2023-12-19T18:16:40.548463+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:53:09.066279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357879+0000","last_change":"2023-12-19T18:16:40.562872+0000","last_active":"2023-12-19T18:16:42.357879+0000","last_peered":"2023-12-19T18:16:42.357879+0000","last_clean":"2023-12-19T18:16:42.357879+0000","last_became_active":"2023-12-19T18:16:40.562615+0000","last_became_peered":"2023-12-19T18:16:40.562615+0000","last_unstale":"2023-12-19T18:16:42.357879+0000","last_undegraded":"2023-12-19T18:16:42.357879+0000","last_fullsized":"2023-12-19T18:16:42.357879+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:54.454071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565322+0000","last_change":"2023-12-19T18:16:27.830119+0000","last_active":"2023-12-19T18:16:40.565322+0000","last_peered":"2023-12-19T18:16:40.565322+0000","last_clean":"2023-12-19T18:16:40.565322+0000","last_became_active":"2023-12-19T18:16:27.829850+0000","last_became_peered":"2023-12-19T18:16:27.829850+0000","last_unstale":"2023-12-19T18:16:40.565322+0000","last_undegraded":"2023-12-19T18:16:40.565322+0000","last_fullsized":"2023-12-19T18:16:40.565322+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:51.190615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857238+0000","last_change":"2023-12-19T18:16:40.570092+0000","last_active":"2023-12-19T18:16:41.857238+0000","last_peered":"2023-12-19T18:16:41.857238+0000","last_clean":"2023-12-19T18:16:41.857238+0000","last_became_active":"2023-12-19T18:16:40.568697+0000","last_became_peered":"2023-12-19T18:16:40.568697+0000","last_unstale":"2023-12-19T18:16:41.857238+0000","last_undegraded":"2023-12-19T18:16:41.857238+0000","last_fullsized":"2023-12-19T18:16:41.857238+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:11.448855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.564229+0000","last_change":"2023-12-19T18:16:27.834875+0000","last_active":"2023-12-19T18:16:40.564229+0000","last_peered":"2023-12-19T18:16:40.564229+0000","last_clean":"2023-12-19T18:16:40.564229+0000","last_became_active":"2023-12-19T18:16:27.834561+0000","last_became_peered":"2023-12-19T18:16:27.834561+0000","last_unstale":"2023-12-19T18:16:40.564229+0000","last_undegraded":"2023-12-19T18:16:40.564229+0000","last_fullsized":"2023-12-19T18:16:40.564229+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:30.301379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554348+0000","last_change":"2023-12-19T18:16:40.561527+0000","last_active":"2023-12-19T18:16:41.554348+0000","last_peered":"2023-12-19T18:16:41.554348+0000","last_clean":"2023-12-19T18:16:41.554348+0000","last_became_active":"2023-12-19T18:16:40.561251+0000","last_became_peered":"2023-12-19T18:16:40.561251+0000","last_unstale":"2023-12-19T18:16:41.554348+0000","last_undegraded":"2023-12-19T18:16:41.554348+0000","last_fullsized":"2023-12-19T18:16:41.554348+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:50.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543417+0000","last_change":"2023-12-19T18:16:27.826642+0000","last_active":"2023-12-19T18:16:40.543417+0000","last_peered":"2023-12-19T18:16:40.543417+0000","last_clean":"2023-12-19T18:16:40.543417+0000","last_became_active":"2023-12-19T18:16:27.825460+0000","last_became_peered":"2023-12-19T18:16:27.825460+0000","last_unstale":"2023-12-19T18:16:40.543417+0000","last_undegraded":"2023-12-19T18:16:40.543417+0000","last_fullsized":"2023-12-19T18:16:40.543417+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:05.671602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074707+0000","last_change":"2023-12-19T18:16:40.579582+0000","last_active":"2023-12-19T18:16:42.074707+0000","last_peered":"2023-12-19T18:16:42.074707+0000","last_clean":"2023-12-19T18:16:42.074707+0000","last_became_active":"2023-12-19T18:16:40.579319+0000","last_became_peered":"2023-12-19T18:16:40.579319+0000","last_unstale":"2023-12-19T18:16:42.074707+0000","last_undegraded":"2023-12-19T18:16:42.074707+0000","last_fullsized":"2023-12-19T18:16:42.074707+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:53:38.529823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551313+0000","last_change":"2023-12-19T18:16:27.826145+0000","last_active":"2023-12-19T18:16:40.551313+0000","last_peered":"2023-12-19T18:16:40.551313+0000","last_clean":"2023-12-19T18:16:40.551313+0000","last_became_active":"2023-12-19T18:16:27.825889+0000","last_became_peered":"2023-12-19T18:16:27.825889+0000","last_unstale":"2023-12-19T18:16:40.551313+0000","last_undegraded":"2023-12-19T18:16:40.551313+0000","last_fullsized":"2023-12-19T18:16:40.551313+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:59:41.673539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074764+0000","last_change":"2023-12-19T18:16:40.579381+0000","last_active":"2023-12-19T18:16:42.074764+0000","last_peered":"2023-12-19T18:16:42.074764+0000","last_clean":"2023-12-19T18:16:42.074764+0000","last_became_active":"2023-12-19T18:16:40.578655+0000","last_became_peered":"2023-12-19T18:16:40.578655+0000","last_unstale":"2023-12-19T18:16:42.074764+0000","last_undegraded":"2023-12-19T18:16:42.074764+0000","last_fullsized":"2023-12-19T18:16:42.074764+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:33:53.935941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555865+0000","last_change":"2023-12-19T18:16:27.830084+0000","last_active":"2023-12-19T18:16:40.555865+0000","last_peered":"2023-12-19T18:16:40.555865+0000","last_clean":"2023-12-19T18:16:40.555865+0000","last_became_active":"2023-12-19T18:16:27.829787+0000","last_became_peered":"2023-12-19T18:16:27.829787+0000","last_unstale":"2023-12-19T18:16:40.555865+0000","last_undegraded":"2023-12-19T18:16:40.555865+0000","last_fullsized":"2023-12-19T18:16:40.555865+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:34:16.356894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554193+0000","last_change":"2023-12-19T18:16:40.563996+0000","last_active":"2023-12-19T18:16:41.554193+0000","last_peered":"2023-12-19T18:16:41.554193+0000","last_clean":"2023-12-19T18:16:41.554193+0000","last_became_active":"2023-12-19T18:16:40.563686+0000","last_became_peered":"2023-12-19T18:16:40.563686+0000","last_unstale":"2023-12-19T18:16:41.554193+0000","last_undegraded":"2023-12-19T18:16:41.554193+0000","last_fullsized":"2023-12-19T18:16:41.554193+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:34.106940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551542+0000","last_change":"2023-12-19T18:16:27.831551+0000","last_active":"2023-12-19T18:16:40.551542+0000","last_peered":"2023-12-19T18:16:40.551542+0000","last_clean":"2023-12-19T18:16:40.551542+0000","last_became_active":"2023-12-19T18:16:27.831291+0000","last_became_peered":"2023-12-19T18:16:27.831291+0000","last_unstale":"2023-12-19T18:16:40.551542+0000","last_undegraded":"2023-12-19T18:16:40.551542+0000","last_fullsized":"2023-12-19T18:16:40.551542+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:09.914084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551186+0000","last_change":"2023-12-19T18:16:40.563764+0000","last_active":"2023-12-19T18:16:41.551186+0000","last_peered":"2023-12-19T18:16:41.551186+0000","last_clean":"2023-12-19T18:16:41.551186+0000","last_became_active":"2023-12-19T18:16:40.563464+0000","last_became_peered":"2023-12-19T18:16:40.563464+0000","last_unstale":"2023-12-19T18:16:41.551186+0000","last_undegraded":"2023-12-19T18:16:41.551186+0000","last_fullsized":"2023-12-19T18:16:41.551186+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:52:34.311679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548663+0000","last_change":"2023-12-19T18:16:27.833054+0000","last_active":"2023-12-19T18:16:40.548663+0000","last_peered":"2023-12-19T18:16:40.548663+0000","last_clean":"2023-12-19T18:16:40.548663+0000","last_became_active":"2023-12-19T18:16:27.832738+0000","last_became_peered":"2023-12-19T18:16:27.832738+0000","last_unstale":"2023-12-19T18:16:40.548663+0000","last_undegraded":"2023-12-19T18:16:40.548663+0000","last_fullsized":"2023-12-19T18:16:40.548663+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:57.892685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360068+0000","last_change":"2023-12-19T18:16:40.577515+0000","last_active":"2023-12-19T18:16:42.360068+0000","last_peered":"2023-12-19T18:16:42.360068+0000","last_clean":"2023-12-19T18:16:42.360068+0000","last_became_active":"2023-12-19T18:16:40.577096+0000","last_became_peered":"2023-12-19T18:16:40.577096+0000","last_unstale":"2023-12-19T18:16:42.360068+0000","last_undegraded":"2023-12-19T18:16:42.360068+0000","last_fullsized":"2023-12-19T18:16:42.360068+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:36.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555479+0000","last_change":"2023-12-19T18:16:27.820194+0000","last_active":"2023-12-19T18:16:40.555479+0000","last_peered":"2023-12-19T18:16:40.555479+0000","last_clean":"2023-12-19T18:16:40.555479+0000","last_became_active":"2023-12-19T18:16:27.819900+0000","last_became_peered":"2023-12-19T18:16:27.819900+0000","last_unstale":"2023-12-19T18:16:40.555479+0000","last_undegraded":"2023-12-19T18:16:40.555479+0000","last_fullsized":"2023-12-19T18:16:40.555479+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:00.858729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360137+0000","last_change":"2023-12-19T18:16:40.561878+0000","last_active":"2023-12-19T18:16:42.360137+0000","last_peered":"2023-12-19T18:16:42.360137+0000","last_clean":"2023-12-19T18:16:42.360137+0000","last_became_active":"2023-12-19T18:16:40.561626+0000","last_became_peered":"2023-12-19T18:16:40.561626+0000","last_unstale":"2023-12-19T18:16:42.360137+0000","last_undegraded":"2023-12-19T18:16:42.360137+0000","last_fullsized":"2023-12-19T18:16:42.360137+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:38:57.938991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562034+0000","last_change":"2023-12-19T18:16:27.820781+0000","last_active":"2023-12-19T18:16:40.562034+0000","last_peered":"2023-12-19T18:16:40.562034+0000","last_clean":"2023-12-19T18:16:40.562034+0000","last_became_active":"2023-12-19T18:16:27.820483+0000","last_became_peered":"2023-12-19T18:16:27.820483+0000","last_unstale":"2023-12-19T18:16:40.562034+0000","last_undegraded":"2023-12-19T18:16:40.562034+0000","last_fullsized":"2023-12-19T18:16:40.562034+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:23.445435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551324+0000","last_change":"2023-12-19T18:16:40.560485+0000","last_active":"2023-12-19T18:16:41.551324+0000","last_peered":"2023-12-19T18:16:41.551324+0000","last_clean":"2023-12-19T18:16:41.551324+0000","last_became_active":"2023-12-19T18:16:40.560232+0000","last_became_peered":"2023-12-19T18:16:40.560232+0000","last_unstale":"2023-12-19T18:16:41.551324+0000","last_undegraded":"2023-12-19T18:16:41.551324+0000","last_fullsized":"2023-12-19T18:16:41.551324+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:30.252976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553708+0000","last_change":"2023-12-19T18:16:27.834880+0000","last_active":"2023-12-19T18:16:40.553708+0000","last_peered":"2023-12-19T18:16:40.553708+0000","last_clean":"2023-12-19T18:16:40.553708+0000","last_became_active":"2023-12-19T18:16:27.834494+0000","last_became_peered":"2023-12-19T18:16:27.834494+0000","last_unstale":"2023-12-19T18:16:40.553708+0000","last_undegraded":"2023-12-19T18:16:40.553708+0000","last_fullsized":"2023-12-19T18:16:40.553708+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:06.992701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357587+0000","last_change":"2023-12-19T18:16:40.567126+0000","last_active":"2023-12-19T18:16:42.357587+0000","last_peered":"2023-12-19T18:16:42.357587+0000","last_clean":"2023-12-19T18:16:42.357587+0000","last_became_active":"2023-12-19T18:16:40.566878+0000","last_became_peered":"2023-12-19T18:16:40.566878+0000","last_unstale":"2023-12-19T18:16:42.357587+0000","last_undegraded":"2023-12-19T18:16:42.357587+0000","last_fullsized":"2023-12-19T18:16:42.357587+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:59.062853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543300+0000","last_change":"2023-12-19T18:16:27.826828+0000","last_active":"2023-12-19T18:16:40.543300+0000","last_peered":"2023-12-19T18:16:40.543300+0000","last_clean":"2023-12-19T18:16:40.543300+0000","last_became_active":"2023-12-19T18:16:27.825994+0000","last_became_peered":"2023-12-19T18:16:27.825994+0000","last_unstale":"2023-12-19T18:16:40.543300+0000","last_undegraded":"2023-12-19T18:16:40.543300+0000","last_fullsized":"2023-12-19T18:16:40.543300+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:17.748218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857358+0000","last_change":"2023-12-19T18:16:40.569008+0000","last_active":"2023-12-19T18:16:41.857358+0000","last_peered":"2023-12-19T18:16:41.857358+0000","last_clean":"2023-12-19T18:16:41.857358+0000","last_became_active":"2023-12-19T18:16:40.568420+0000","last_became_peered":"2023-12-19T18:16:40.568420+0000","last_unstale":"2023-12-19T18:16:41.857358+0000","last_undegraded":"2023-12-19T18:16:41.857358+0000","last_fullsized":"2023-12-19T18:16:41.857358+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:31:10.720873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561285+0000","last_change":"2023-12-19T18:16:27.830751+0000","last_active":"2023-12-19T18:16:40.561285+0000","last_peered":"2023-12-19T18:16:40.561285+0000","last_clean":"2023-12-19T18:16:40.561285+0000","last_became_active":"2023-12-19T18:16:27.830490+0000","last_became_peered":"2023-12-19T18:16:27.830490+0000","last_unstale":"2023-12-19T18:16:40.561285+0000","last_undegraded":"2023-12-19T18:16:40.561285+0000","last_fullsized":"2023-12-19T18:16:40.561285+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:31.093343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550245+0000","last_change":"2023-12-19T18:16:40.571696+0000","last_active":"2023-12-19T18:16:41.550245+0000","last_peered":"2023-12-19T18:16:41.550245+0000","last_clean":"2023-12-19T18:16:41.550245+0000","last_became_active":"2023-12-19T18:16:40.571407+0000","last_became_peered":"2023-12-19T18:16:40.571407+0000","last_unstale":"2023-12-19T18:16:41.550245+0000","last_undegraded":"2023-12-19T18:16:41.550245+0000","last_fullsized":"2023-12-19T18:16:41.550245+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:55.222921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548271+0000","last_change":"2023-12-19T18:16:27.823186+0000","last_active":"2023-12-19T18:16:40.548271+0000","last_peered":"2023-12-19T18:16:40.548271+0000","last_clean":"2023-12-19T18:16:40.548271+0000","last_became_active":"2023-12-19T18:16:27.820405+0000","last_became_peered":"2023-12-19T18:16:27.820405+0000","last_unstale":"2023-12-19T18:16:40.548271+0000","last_undegraded":"2023-12-19T18:16:40.548271+0000","last_fullsized":"2023-12-19T18:16:40.548271+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:41.515700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357953+0000","last_change":"2023-12-19T18:16:40.569297+0000","last_active":"2023-12-19T18:16:42.357953+0000","last_peered":"2023-12-19T18:16:42.357953+0000","last_clean":"2023-12-19T18:16:42.357953+0000","last_became_active":"2023-12-19T18:16:40.569050+0000","last_became_peered":"2023-12-19T18:16:40.569050+0000","last_unstale":"2023-12-19T18:16:42.357953+0000","last_undegraded":"2023-12-19T18:16:42.357953+0000","last_fullsized":"2023-12-19T18:16:42.357953+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:28:56.831758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561350+0000","last_change":"2023-12-19T18:16:27.829736+0000","last_active":"2023-12-19T18:16:40.561350+0000","last_peered":"2023-12-19T18:16:40.561350+0000","last_clean":"2023-12-19T18:16:40.561350+0000","last_became_active":"2023-12-19T18:16:27.829448+0000","last_became_peered":"2023-12-19T18:16:27.829448+0000","last_unstale":"2023-12-19T18:16:40.561350+0000","last_undegraded":"2023-12-19T18:16:40.561350+0000","last_fullsized":"2023-12-19T18:16:40.561350+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:47.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551154+0000","last_change":"2023-12-19T18:16:40.559877+0000","last_active":"2023-12-19T18:16:41.551154+0000","last_peered":"2023-12-19T18:16:41.551154+0000","last_clean":"2023-12-19T18:16:41.551154+0000","last_became_active":"2023-12-19T18:16:40.558100+0000","last_became_peered":"2023-12-19T18:16:40.558100+0000","last_unstale":"2023-12-19T18:16:41.551154+0000","last_undegraded":"2023-12-19T18:16:41.551154+0000","last_fullsized":"2023-12-19T18:16:41.551154+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:44.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559776+0000","last_change":"2023-12-19T18:16:27.829943+0000","last_active":"2023-12-19T18:16:40.559776+0000","last_peered":"2023-12-19T18:16:40.559776+0000","last_clean":"2023-12-19T18:16:40.559776+0000","last_became_active":"2023-12-19T18:16:27.829592+0000","last_became_peered":"2023-12-19T18:16:27.829592+0000","last_unstale":"2023-12-19T18:16:40.559776+0000","last_undegraded":"2023-12-19T18:16:40.559776+0000","last_fullsized":"2023-12-19T18:16:40.559776+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:46.245564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553575+0000","last_change":"2023-12-19T18:16:40.575384+0000","last_active":"2023-12-19T18:16:41.553575+0000","last_peered":"2023-12-19T18:16:41.553575+0000","last_clean":"2023-12-19T18:16:41.553575+0000","last_became_active":"2023-12-19T18:16:40.575124+0000","last_became_peered":"2023-12-19T18:16:40.575124+0000","last_unstale":"2023-12-19T18:16:41.553575+0000","last_undegraded":"2023-12-19T18:16:41.553575+0000","last_fullsized":"2023-12-19T18:16:41.553575+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:53.796369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565426+0000","last_change":"2023-12-19T18:16:27.819797+0000","last_active":"2023-12-19T18:16:40.565426+0000","last_peered":"2023-12-19T18:16:40.565426+0000","last_clean":"2023-12-19T18:16:40.565426+0000","last_became_active":"2023-12-19T18:16:27.819507+0000","last_became_peered":"2023-12-19T18:16:27.819507+0000","last_unstale":"2023-12-19T18:16:40.565426+0000","last_undegraded":"2023-12-19T18:16:40.565426+0000","last_fullsized":"2023-12-19T18:16:40.565426+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:08.287716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857264+0000","last_change":"2023-12-19T18:16:40.570173+0000","last_active":"2023-12-19T18:16:41.857264+0000","last_peered":"2023-12-19T18:16:41.857264+0000","last_clean":"2023-12-19T18:16:41.857264+0000","last_became_active":"2023-12-19T18:16:40.569912+0000","last_became_peered":"2023-12-19T18:16:40.569912+0000","last_unstale":"2023-12-19T18:16:41.857264+0000","last_undegraded":"2023-12-19T18:16:41.857264+0000","last_fullsized":"2023-12-19T18:16:41.857264+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:47.688949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558633+0000","last_change":"2023-12-19T18:16:27.819945+0000","last_active":"2023-12-19T18:16:40.558633+0000","last_peered":"2023-12-19T18:16:40.558633+0000","last_clean":"2023-12-19T18:16:40.558633+0000","last_became_active":"2023-12-19T18:16:27.819634+0000","last_became_peered":"2023-12-19T18:16:27.819634+0000","last_unstale":"2023-12-19T18:16:40.558633+0000","last_undegraded":"2023-12-19T18:16:40.558633+0000","last_fullsized":"2023-12-19T18:16:40.558633+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:22.401852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358420+0000","last_change":"2023-12-19T18:16:40.566307+0000","last_active":"2023-12-19T18:16:42.358420+0000","last_peered":"2023-12-19T18:16:42.358420+0000","last_clean":"2023-12-19T18:16:42.358420+0000","last_became_active":"2023-12-19T18:16:40.566059+0000","last_became_peered":"2023-12-19T18:16:40.566059+0000","last_unstale":"2023-12-19T18:16:42.358420+0000","last_undegraded":"2023-12-19T18:16:42.358420+0000","last_fullsized":"2023-12-19T18:16:42.358420+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:45.773786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549949+0000","last_change":"2023-12-19T18:16:27.832580+0000","last_active":"2023-12-19T18:16:40.549949+0000","last_peered":"2023-12-19T18:16:40.549949+0000","last_clean":"2023-12-19T18:16:40.549949+0000","last_became_active":"2023-12-19T18:16:27.832291+0000","last_became_peered":"2023-12-19T18:16:27.832291+0000","last_unstale":"2023-12-19T18:16:40.549949+0000","last_undegraded":"2023-12-19T18:16:40.549949+0000","last_fullsized":"2023-12-19T18:16:40.549949+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:29.305405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359160+0000","last_change":"2023-12-19T18:16:40.567943+0000","last_active":"2023-12-19T18:16:42.359160+0000","last_peered":"2023-12-19T18:16:42.359160+0000","last_clean":"2023-12-19T18:16:42.359160+0000","last_became_active":"2023-12-19T18:16:40.567701+0000","last_became_peered":"2023-12-19T18:16:40.567701+0000","last_unstale":"2023-12-19T18:16:42.359160+0000","last_undegraded":"2023-12-19T18:16:42.359160+0000","last_fullsized":"2023-12-19T18:16:42.359160+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:50:31.894613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543104+0000","last_change":"2023-12-19T18:16:27.825730+0000","last_active":"2023-12-19T18:16:40.543104+0000","last_peered":"2023-12-19T18:16:40.543104+0000","last_clean":"2023-12-19T18:16:40.543104+0000","last_became_active":"2023-12-19T18:16:27.825136+0000","last_became_peered":"2023-12-19T18:16:27.825136+0000","last_unstale":"2023-12-19T18:16:40.543104+0000","last_undegraded":"2023-12-19T18:16:40.543104+0000","last_fullsized":"2023-12-19T18:16:40.543104+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:48.330735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359437+0000","last_change":"2023-12-19T18:16:40.558188+0000","last_active":"2023-12-19T18:16:42.359437+0000","last_peered":"2023-12-19T18:16:42.359437+0000","last_clean":"2023-12-19T18:16:42.359437+0000","last_became_active":"2023-12-19T18:16:40.557804+0000","last_became_peered":"2023-12-19T18:16:40.557804+0000","last_unstale":"2023-12-19T18:16:42.359437+0000","last_undegraded":"2023-12-19T18:16:42.359437+0000","last_fullsized":"2023-12-19T18:16:42.359437+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:35:20.500418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558018+0000","last_change":"2023-12-19T18:16:27.823852+0000","last_active":"2023-12-19T18:16:40.558018+0000","last_peered":"2023-12-19T18:16:40.558018+0000","last_clean":"2023-12-19T18:16:40.558018+0000","last_became_active":"2023-12-19T18:16:27.823564+0000","last_became_peered":"2023-12-19T18:16:27.823564+0000","last_unstale":"2023-12-19T18:16:40.558018+0000","last_undegraded":"2023-12-19T18:16:40.558018+0000","last_fullsized":"2023-12-19T18:16:40.558018+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:59.000440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358756+0000","last_change":"2023-12-19T18:16:40.561015+0000","last_active":"2023-12-19T18:16:42.358756+0000","last_peered":"2023-12-19T18:16:42.358756+0000","last_clean":"2023-12-19T18:16:42.358756+0000","last_became_active":"2023-12-19T18:16:40.560771+0000","last_became_peered":"2023-12-19T18:16:40.560771+0000","last_unstale":"2023-12-19T18:16:42.358756+0000","last_undegraded":"2023-12-19T18:16:42.358756+0000","last_fullsized":"2023-12-19T18:16:42.358756+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:04.628464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554625+0000","last_change":"2023-12-19T18:16:27.822167+0000","last_active":"2023-12-19T18:16:40.554625+0000","last_peered":"2023-12-19T18:16:40.554625+0000","last_clean":"2023-12-19T18:16:40.554625+0000","last_became_active":"2023-12-19T18:16:27.821752+0000","last_became_peered":"2023-12-19T18:16:27.821752+0000","last_unstale":"2023-12-19T18:16:40.554625+0000","last_undegraded":"2023-12-19T18:16:40.554625+0000","last_fullsized":"2023-12-19T18:16:40.554625+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:17.866181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357777+0000","last_change":"2023-12-19T18:16:40.565973+0000","last_active":"2023-12-19T18:16:42.357777+0000","last_peered":"2023-12-19T18:16:42.357777+0000","last_clean":"2023-12-19T18:16:42.357777+0000","last_became_active":"2023-12-19T18:16:40.565418+0000","last_became_peered":"2023-12-19T18:16:40.565418+0000","last_unstale":"2023-12-19T18:16:42.357777+0000","last_undegraded":"2023-12-19T18:16:42.357777+0000","last_fullsized":"2023-12-19T18:16:42.357777+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:54.473566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551716+0000","last_change":"2023-12-19T18:16:27.835673+0000","last_active":"2023-12-19T18:16:40.551716+0000","last_peered":"2023-12-19T18:16:40.551716+0000","last_clean":"2023-12-19T18:16:40.551716+0000","last_became_active":"2023-12-19T18:16:27.835392+0000","last_became_peered":"2023-12-19T18:16:27.835392+0000","last_unstale":"2023-12-19T18:16:40.551716+0000","last_undegraded":"2023-12-19T18:16:40.551716+0000","last_fullsized":"2023-12-19T18:16:40.551716+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:23:06.990327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553304+0000","last_change":"2023-12-19T18:16:40.564946+0000","last_active":"2023-12-19T18:16:41.553304+0000","last_peered":"2023-12-19T18:16:41.553304+0000","last_clean":"2023-12-19T18:16:41.553304+0000","last_became_active":"2023-12-19T18:16:40.564679+0000","last_became_peered":"2023-12-19T18:16:40.564679+0000","last_unstale":"2023-12-19T18:16:41.553304+0000","last_undegraded":"2023-12-19T18:16:41.553304+0000","last_fullsized":"2023-12-19T18:16:41.553304+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:02:28.048063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559694+0000","last_change":"2023-12-19T18:16:27.830449+0000","last_active":"2023-12-19T18:16:40.559694+0000","last_peered":"2023-12-19T18:16:40.559694+0000","last_clean":"2023-12-19T18:16:40.559694+0000","last_became_active":"2023-12-19T18:16:27.830189+0000","last_became_peered":"2023-12-19T18:16:27.830189+0000","last_unstale":"2023-12-19T18:16:40.559694+0000","last_undegraded":"2023-12-19T18:16:40.559694+0000","last_fullsized":"2023-12-19T18:16:40.559694+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:06.397912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360333+0000","last_change":"2023-12-19T18:16:40.576744+0000","last_active":"2023-12-19T18:16:42.360333+0000","last_peered":"2023-12-19T18:16:42.360333+0000","last_clean":"2023-12-19T18:16:42.360333+0000","last_became_active":"2023-12-19T18:16:40.576425+0000","last_became_peered":"2023-12-19T18:16:40.576425+0000","last_unstale":"2023-12-19T18:16:42.360333+0000","last_undegraded":"2023-12-19T18:16:42.360333+0000","last_fullsized":"2023-12-19T18:16:42.360333+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:26.259551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548052+0000","last_change":"2023-12-19T18:16:27.815730+0000","last_active":"2023-12-19T18:16:40.548052+0000","last_peered":"2023-12-19T18:16:40.548052+0000","last_clean":"2023-12-19T18:16:40.548052+0000","last_became_active":"2023-12-19T18:16:27.815341+0000","last_became_peered":"2023-12-19T18:16:27.815341+0000","last_unstale":"2023-12-19T18:16:40.548052+0000","last_undegraded":"2023-12-19T18:16:40.548052+0000","last_fullsized":"2023-12-19T18:16:40.548052+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:39.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074872+0000","last_change":"2023-12-19T18:16:40.578780+0000","last_active":"2023-12-19T18:16:42.074872+0000","last_peered":"2023-12-19T18:16:42.074872+0000","last_clean":"2023-12-19T18:16:42.074872+0000","last_became_active":"2023-12-19T18:16:40.578426+0000","last_became_peered":"2023-12-19T18:16:40.578426+0000","last_unstale":"2023-12-19T18:16:42.074872+0000","last_undegraded":"2023-12-19T18:16:42.074872+0000","last_fullsized":"2023-12-19T18:16:42.074872+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:53:39.733238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562588+0000","last_change":"2023-12-19T18:16:27.831511+0000","last_active":"2023-12-19T18:16:40.562588+0000","last_peered":"2023-12-19T18:16:40.562588+0000","last_clean":"2023-12-19T18:16:40.562588+0000","last_became_active":"2023-12-19T18:16:27.831255+0000","last_became_peered":"2023-12-19T18:16:27.831255+0000","last_unstale":"2023-12-19T18:16:40.562588+0000","last_undegraded":"2023-12-19T18:16:40.562588+0000","last_fullsized":"2023-12-19T18:16:40.562588+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:42.382254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358249+0000","last_change":"2023-12-19T18:16:40.576919+0000","last_active":"2023-12-19T18:16:42.358249+0000","last_peered":"2023-12-19T18:16:42.358249+0000","last_clean":"2023-12-19T18:16:42.358249+0000","last_became_active":"2023-12-19T18:16:40.576351+0000","last_became_peered":"2023-12-19T18:16:40.576351+0000","last_unstale":"2023-12-19T18:16:42.358249+0000","last_undegraded":"2023-12-19T18:16:42.358249+0000","last_fullsized":"2023-12-19T18:16:42.358249+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:36.351083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562201+0000","last_change":"2023-12-19T18:16:27.834221+0000","last_active":"2023-12-19T18:16:40.562201+0000","last_peered":"2023-12-19T18:16:40.562201+0000","last_clean":"2023-12-19T18:16:40.562201+0000","last_became_active":"2023-12-19T18:16:27.825886+0000","last_became_peered":"2023-12-19T18:16:27.825886+0000","last_unstale":"2023-12-19T18:16:40.562201+0000","last_undegraded":"2023-12-19T18:16:40.562201+0000","last_fullsized":"2023-12-19T18:16:40.562201+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:12.119996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510569,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64616,"kb_used_data":3232,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307224,"statfs":{"total":96636764160,"available":96570597376,"internally_reserved":0,"allocated":3309568,"data_stored":2022904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96399999999999997,"5min":1.048,"15min":1.1180000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92400000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63400000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92400000000000004,"5min":1.0389999999999999,"15min":1.0580000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72399999999999998,"15min":0.748},"min":{"1min":0.499,"5min":0.48099999999999998,"15min":0.435},"max":{"1min":0.89500000000000002,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.67700000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.38900000000000001},"max":{"1min":0.93400000000000005,"5min":1.026,"15min":1.0669999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.0820000000000001,"15min":1.236},"last":0.57899999999999996}]},{"osd":3,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.75600000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88300000000000001,"5min":1.002,"15min":1.079},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98899999999999999,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.66000000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81599999999999995,"5min":1.0009999999999999,"15min":1.302},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.871,"15min":1.0209999999999999},"last":0.82699999999999996}]},{"osd":5,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.52700000000000002,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.88100000000000001,"5min":1.097,"15min":1.1299999999999999},"last":0.622},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64900000000000002,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":1.026,"15min":1.0800000000000001},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.56999999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.93999999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85699999999999998,"5min":1.01,"15min":1.093},"last":0.79700000000000004}]}]},{"osd":6,"up_from":15,"seq":64424510569,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47636,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324204,"statfs":{"total":96636764160,"available":96587984896,"internally_reserved":0,"allocated":2699264,"data_stored":1431450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72799999999999998,"15min":0.74199999999999999},"min":{"1min":0.55400000000000005,"5min":0.46800000000000003,"15min":0.43099999999999999},"max":{"1min":0.92400000000000004,"5min":0.96499999999999997,"15min":1.042},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71799999999999997,"15min":0.73499999999999999},"min":{"1min":0.57499999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92000000000000004,"5min":1.224,"15min":1.224},"last":0.69299999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70099999999999996,"15min":0.72799999999999998},"min":{"1min":0.49099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98099999999999998,"5min":1.103,"15min":1.103},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.64100000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":0.94599999999999995,"5min":0.99199999999999999,"15min":1.1259999999999999},"last":0.76000000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.502},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0609999999999999},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.749},"min":{"1min":0.60899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83299999999999996,"5min":1.0089999999999999,"15min":1.111},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.755,"15min":0.78200000000000003},"min":{"1min":0.61799999999999999,"5min":0.495,"15min":0.495},"max":{"1min":0.91200000000000003,"5min":0.97999999999999998,"15min":1.1519999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.51800000000000002,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.98699999999999999,"5min":1.268,"15min":1.268},"last":0.72299999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92200000000000004,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":0.81299999999999994,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.78600000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.93799999999999994,"5min":1.194,"15min":1.194},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":1.141,"15min":1.141},"last":0.95199999999999996}]},{"osd":7,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.95899999999999996,"5min":1.159,"15min":1.159},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94199999999999995,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.99199999999999999}]}]},{"osd":5,"up_from":15,"seq":64424510569,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47636,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324204,"statfs":{"total":96636764160,"available":96587984896,"internally_reserved":0,"allocated":2699264,"data_stored":1431450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72499999999999998,"15min":0.72999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34399999999999997},"max":{"1min":0.96899999999999997,"5min":1.0109999999999999,"15min":1.0149999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.40699999999999997},"max":{"1min":0.89200000000000002,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"min":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.497},"max":{"1min":0.95599999999999996,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.505,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78800000000000003,"15min":0.78100000000000003},"min":{"1min":0.61699999999999999,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":1.022,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94499999999999995,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.48499999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94999999999999996,"5min":1.05,"15min":1.05},"last":0.753},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.77400000000000002},"min":{"1min":0.56200000000000006,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98799999999999999,"5min":1.022,"15min":1.0329999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.79000000000000004,"5min":0.94499999999999995,"15min":1.0800000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41599999999999998},"max":{"1min":0.71099999999999997,"5min":1.208,"15min":1.208},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64800000000000002,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.94899999999999995,"15min":0.98999999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.65500000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.83499999999999996,"5min":1.008,"15min":1.0449999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.46500000000000002,"5min":0.439,"15min":0.436},"max":{"1min":0.77300000000000002,"5min":1.169,"15min":1.169},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71499999999999997,"15min":0.70999999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85699999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74199999999999999}]}]},{"osd":4,"up_from":15,"seq":64424510569,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47668,"kb_used_data":2668,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324172,"statfs":{"total":96636764160,"available":96587952128,"internally_reserved":0,"allocated":2732032,"data_stored":1435923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.51600000000000001,"5min":0.45800000000000002,"15min":0.45000000000000001},"max":{"1min":1.016,"5min":1.036,"15min":1.109},"last":0.62},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.502,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.91800000000000004,"5min":0.997,"15min":1.01},"last":0.72999999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.59199999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94699999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.997,"15min":1.1020000000000001},"last":0.68200000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79100000000000004,"15min":0.77700000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.47699999999999998},"max":{"1min":0.998,"5min":1.0289999999999999,"15min":1.071},"last":0.503},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.71699999999999997}]},{"osd":3,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76700000000000002,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93899999999999995,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.55500000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.66200000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65300000000000002,"15min":0.63100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.75},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67200000000000004,"15min":0.64600000000000002},"min":{"1min":0.495,"5min":0.42899999999999999,"15min":0.39600000000000002},"max":{"1min":0.88900000000000001,"5min":1.042,"15min":1.042},"last":0.52600000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.66100000000000003},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":1.024,"15min":1.0669999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0760000000000001},"last":0.78300000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.41899999999999998},"max":{"1min":0.92400000000000004,"5min":1.087,"15min":1.087},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77600000000000002,"15min":0.76200000000000001},"min":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.46999999999999997},"max":{"1min":0.98599999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.70099999999999996}]}]},{"osd":3,"up_from":15,"seq":64424510569,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47656,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324184,"statfs":{"total":96636764160,"available":96587964416,"internally_reserved":0,"allocated":2719744,"data_stored":1431244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.63200000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74099999999999999,"5min":0.78300000000000003,"15min":0.80000000000000004},"last":0.504}]},{"osd":1,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.498},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71499999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.44}]},{"osd":2,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.77000000000000002,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.46000000000000002}]},{"osd":4,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81599999999999995,"15min":0.82799999999999996},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.012,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.81999999999999995,"15min":0.82299999999999995},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.56599999999999995},"max":{"1min":0.86499999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.65800000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82799999999999996,"15min":0.84199999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99099999999999999,"5min":1.1739999999999999,"15min":1.397},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.59399999999999997,"5min":0.55900000000000005,"15min":0.55100000000000005},"max":{"1min":0.91000000000000003,"5min":1.2230000000000001,"15min":1.262},"last":0.60899999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.83899999999999997,"15min":0.84799999999999998},"min":{"1min":0.60199999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.96099999999999997,"5min":1.234,"15min":1.274},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83799999999999997,"15min":0.84499999999999997},"min":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94599999999999995,"5min":1.165,"15min":1.302},"last":0.64500000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0640000000000001,"5min":1.3540000000000001,"15min":1.486},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89500000000000002},"min":{"1min":0.68899999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.93000000000000005,"5min":1.421,"15min":1.421},"last":0.626}]}]},{"osd":2,"up_from":15,"seq":64424510568,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47632,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324208,"statfs":{"total":96636764160,"available":96587988992,"internally_reserved":0,"allocated":2695168,"data_stored":1423635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.29299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.41499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.56000000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.86199999999999999,"15min":0.95999999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.36899999999999999},"max":{"1min":0.67200000000000004,"5min":0.79700000000000004,"15min":0.875},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.624,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.048,"5min":1.306,"15min":1.306},"last":0.753},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.05,"5min":1.272,"15min":1.466},"last":0.80700000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.64300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.97699999999999998,"5min":1.1459999999999999,"15min":1.3049999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.81999999999999995,"15min":0.82599999999999996},"min":{"1min":0.61899999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.429},"last":0.78400000000000003}]},{"osd":6,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.71199999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.216,"5min":1.365,"15min":1.4119999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80600000000000005,"15min":0.80800000000000005},"min":{"1min":0.625,"5min":0.55200000000000005,"15min":0.53500000000000003},"max":{"1min":0.94499999999999995,"5min":1.2,"15min":1.2470000000000001},"last":0.72299999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81499999999999995,"15min":0.82799999999999996},"min":{"1min":0.64400000000000002,"5min":0.59899999999999998,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.393,"15min":1.45},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85999999999999999,"15min":0.86099999999999999},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1850000000000001,"5min":1.276,"15min":1.276},"last":0.88600000000000001}]}]},{"osd":1,"up_from":15,"seq":64424510569,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59940,"kb_used_data":2652,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311900,"statfs":{"total":96636764160,"available":96575385600,"internally_reserved":0,"allocated":2715648,"data_stored":1433729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.33900000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63700000000000001,"5min":0.71999999999999997,"15min":0.89400000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.53300000000000003},"min":{"1min":0.32200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.32600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.505},{"interface":"front","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.63400000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.45000000000000001,"15min":0.442},"max":{"1min":0.82699999999999996,"5min":0.94199999999999995,"15min":0.94499999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.65100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93400000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79100000000000004,"15min":0.81499999999999995},"min":{"1min":0.502,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0409999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.86599999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.78400000000000003,"15min":0.80500000000000005},"min":{"1min":0.49099999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97499999999999998,"5min":1.236,"15min":1.2609999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98399999999999999,"5min":1.353,"15min":1.353},"last":1.05}]},{"osd":6,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.60999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.008,"5min":1.2270000000000001,"15min":1.2689999999999999},"last":0.878},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.84299999999999997,"15min":0.84999999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.093,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":1.034}]},{"osd":7,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.64100000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.123,"5min":1.399,"15min":1.399},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.84299999999999997,"15min":0.85799999999999998},"min":{"1min":0.54100000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.105,"5min":1.365,"15min":1.365},"last":0.90100000000000002}]}]},{"osd":0,"up_from":14,"seq":60129543274,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60508,"kb_used_data":3220,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311332,"statfs":{"total":96636764160,"available":96574803968,"internally_reserved":0,"allocated":3297280,"data_stored":2021348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.438,"15min":0.45700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":1.216},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.317}]},{"osd":2,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64700000000000002,"5min":0.84999999999999998,"15min":1.1699999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72299999999999998,"5min":0.78900000000000003,"15min":0.96799999999999997},"last":0.442}]},{"osd":3,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.623,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.82699999999999996,"5min":0.871,"15min":1.032},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.63,"5min":0.51800000000000002,"15min":0.496},"max":{"1min":0.93100000000000005,"5min":1.1200000000000001,"15min":1.381},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.59199999999999997,"5min":0.56799999999999995,"15min":0.47799999999999998},"max":{"1min":1.02,"5min":1.137,"15min":1.2849999999999999},"last":0.752}]},{"osd":5,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80600000000000005,"15min":0.81899999999999995},"min":{"1min":0.57299999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.008,"5min":1.1930000000000001,"15min":1.3420000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.69599999999999995,"5min":0.59599999999999997,"15min":0.59499999999999997},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.288},"last":0.67200000000000004}]},{"osd":6,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.51300000000000001},"max":{"1min":0.98899999999999999,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.83399999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0229999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.78600000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.84999999999999998},"min":{"1min":0.65300000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.044,"5min":1.1899999999999999,"15min":1.415},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.83199999999999996,"15min":0.85599999999999998},"min":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0780000000000001,"5min":1.1379999999999999,"15min":1.395},"last":0.85599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:16:49.272 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:16:49.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.422+0000 7f22b2afc640 1 Processor -- start 2023-12-19T18:16:49.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.422+0000 7f22b2afc640 1 -- start start 2023-12-19T18:16:49.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b2afc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22ac155030 0x7f22ac15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b2afc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ac15f370 0x7f22ac06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b2afc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f22ac0ce740 0x7f22ac0cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b2afc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac0cf130 con 0x7f22ac155030 2023-12-19T18:16:49.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b2afc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac06c970 con 0x7f22ac0ce740 2023-12-19T18:16:49.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b2afc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac15b1f0 con 0x7f22ac15f370 2023-12-19T18:16:49.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b0898640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22ac155030 0x7f22ac15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b0898640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22ac155030 0x7f22ac15bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32804/0 (socket says 172.21.15.26:32804) 2023-12-19T18:16:49.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b0898640 1 -- 172.21.15.26:0/3654811467 learned_addr learned my addr 172.21.15.26:0/3654811467 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:49.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22abfff640 1 --2- 172.21.15.26:0/3654811467 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ac15f370 0x7f22ac06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b1099640 1 --2- 172.21.15.26:0/3654811467 >> v2:172.21.15.160:3300/0 conn(0x7f22ac0ce740 0x7f22ac0cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b0898640 1 -- 172.21.15.26:0/3654811467 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ac15f370 msgr2=0x7f22ac06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:49.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b0898640 1 --2- 172.21.15.26:0/3654811467 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ac15f370 0x7f22ac06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b0898640 1 -- 172.21.15.26:0/3654811467 >> v2:172.21.15.160:3300/0 conn(0x7f22ac0ce740 msgr2=0x7f22ac0cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:49.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b0898640 1 --2- 172.21.15.26:0/3654811467 >> v2:172.21.15.160:3300/0 conn(0x7f22ac0ce740 0x7f22ac0cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.426+0000 7f22b0898640 1 -- 172.21.15.26:0/3654811467 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac15b4d0 con 0x7f22ac155030 2023-12-19T18:16:49.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22b0898640 1 --2- 172.21.15.26:0/3654811467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac155030 0x7f22ac15bcd0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f22ac06f300 tx=0x7f22a001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b774e31fd99f18 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:49.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22ab7fe640 1 -- 172.21.15.26:0/3654811467 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a00aa610 con 0x7f22ac155030 2023-12-19T18:16:49.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22ab7fe640 1 -- 172.21.15.26:0/3654811467 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a00a6040 con 0x7f22ac155030 2023-12-19T18:16:49.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22ab7fe640 1 -- 172.21.15.26:0/3654811467 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a00aa990 con 0x7f22ac155030 2023-12-19T18:16:49.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22b2afc640 1 -- 172.21.15.26:0/3654811467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac155030 msgr2=0x7f22ac15bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:49.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22b2afc640 1 --2- 172.21.15.26:0/3654811467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac155030 0x7f22ac15bcd0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f22ac06f300 tx=0x7f22a001c4d0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22b2afc640 1 -- 172.21.15.26:0/3654811467 shutdown_connections 2023-12-19T18:16:49.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22b2afc640 1 --2- 172.21.15.26:0/3654811467 >> v2:172.21.15.160:3300/0 conn(0x7f22ac0ce740 0x7f22ac0cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22b2afc640 1 --2- 172.21.15.26:0/3654811467 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ac15f370 0x7f22ac06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22b2afc640 1 --2- 172.21.15.26:0/3654811467 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac155030 0x7f22ac15bcd0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22b2afc640 1 -- 172.21.15.26:0/3654811467 >> 172.21.15.26:0/3654811467 conn(0x7f22ac0658e0 msgr2=0x7f22ac065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:49.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22b2afc640 1 -- 172.21.15.26:0/3654811467 shutdown_connections 2023-12-19T18:16:49.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22b2afc640 1 -- 172.21.15.26:0/3654811467 wait complete. 2023-12-19T18:16:49.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22b2afc640 1 Processor -- start 2023-12-19T18:16:49.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.430+0000 7f22b2afc640 1 -- start start 2023-12-19T18:16:49.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b2afc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0ce740 0x7f22ac142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b2afc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ac155030 0x7f22ac06b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b2afc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ac15f370 0x7f22ac06be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b2afc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac1f3ed0 con 0x7f22ac0ce740 2023-12-19T18:16:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b2afc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac1f41b0 con 0x7f22ac15f370 2023-12-19T18:16:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b2afc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac1f4490 con 0x7f22ac155030 2023-12-19T18:16:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b0898640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0ce740 0x7f22ac142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b0898640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0ce740 0x7f22ac142e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32812/0 (socket says 172.21.15.26:32812) 2023-12-19T18:16:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b0898640 1 -- 172.21.15.26:0/2184792275 learned_addr learned my addr 172.21.15.26:0/2184792275 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b1099640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ac15f370 0x7f22ac06be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22abfff640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ac155030 0x7f22ac06b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b0898640 1 -- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ac155030 msgr2=0x7f22ac06b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b0898640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ac155030 0x7f22ac06b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b0898640 1 -- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ac15f370 msgr2=0x7f22ac06be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b0898640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ac15f370 0x7f22ac06be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b0898640 1 -- 172.21.15.26:0/2184792275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a00094d0 con 0x7f22ac0ce740 2023-12-19T18:16:49.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b0898640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0ce740 0x7f22ac142e40 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f22a00ac040 tx=0x7f22a001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:49.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22a97fa640 1 -- 172.21.15.26:0/2184792275 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a00bd3f0 con 0x7f22ac0ce740 2023-12-19T18:16:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22a97fa640 1 -- 172.21.15.26:0/2184792275 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a00bd570 con 0x7f22ac0ce740 2023-12-19T18:16:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b2afc640 1 -- 172.21.15.26:0/2184792275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac06c740 con 0x7f22ac0ce740 2023-12-19T18:16:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22a97fa640 1 -- 172.21.15.26:0/2184792275 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a00bd920 con 0x7f22ac0ce740 2023-12-19T18:16:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b2afc640 1 -- 172.21.15.26:0/2184792275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac1f4a50 con 0x7f22ac0ce740 2023-12-19T18:16:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.434+0000 7f22b2afc640 1 -- 172.21.15.26:0/2184792275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac074270 con 0x7f22ac0ce740 2023-12-19T18:16:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.438+0000 7f22a97fa640 1 -- 172.21.15.26:0/2184792275 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f22a00b33f0 con 0x7f22ac0ce740 2023-12-19T18:16:49.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.438+0000 7f22a97fa640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2290065b80 0x7f2290068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.438+0000 7f22a97fa640 1 -- 172.21.15.26:0/2184792275 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f22a011d790 con 0x7f22ac0ce740 2023-12-19T18:16:49.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.438+0000 7f22abfff640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2290065b80 0x7f2290068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.438+0000 7f22abfff640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2290065b80 0x7f2290068040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22ac15af10 tx=0x7f2298009190 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:49.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.442+0000 7f22a97fa640 1 -- 172.21.15.26:0/2184792275 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f22a00ea8e0 con 0x7f22ac0ce740 2023-12-19T18:16:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.598+0000 7f22b2afc640 1 -- 172.21.15.26:0/2184792275 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22ac06c5c0 con 0x7f2290065b80 2023-12-19T18:16:49.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.614+0000 7f22a97fa640 1 -- 172.21.15.26:0/2184792275 <== mgr.9805 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528425 (secure 0 0 0) 0x7f2298011910 con 0x7f2290065b80 2023-12-19T18:16:49.620 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:49.624 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:16:49.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.618+0000 7f22b2afc640 1 -- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2290065b80 msgr2=0x7f2290068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:49.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.618+0000 7f22b2afc640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2290065b80 0x7f2290068040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22ac15af10 tx=0x7f2298009190 comp rx=0 tx=0).stop 2023-12-19T18:16:49.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.618+0000 7f22b2afc640 1 -- 172.21.15.26:0/2184792275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0ce740 msgr2=0x7f22ac142e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:49.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.618+0000 7f22b2afc640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0ce740 0x7f22ac142e40 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f22a00ac040 tx=0x7f22a001ad10 comp rx=0 tx=0).stop 2023-12-19T18:16:49.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.618+0000 7f22b2afc640 1 -- 172.21.15.26:0/2184792275 shutdown_connections 2023-12-19T18:16:49.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.618+0000 7f22b2afc640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2290065b80 0x7f2290068040 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.618+0000 7f22b2afc640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22ac15f370 0x7f22ac06be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.618+0000 7f22b2afc640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f22ac155030 0x7f22ac06b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.618+0000 7f22b2afc640 1 --2- 172.21.15.26:0/2184792275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0ce740 0x7f22ac142e40 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.618+0000 7f22b2afc640 1 -- 172.21.15.26:0/2184792275 >> 172.21.15.26:0/2184792275 conn(0x7f22ac0658e0 msgr2=0x7f22ac06fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:49.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.618+0000 7f22b2afc640 1 -- 172.21.15.26:0/2184792275 shutdown_connections 2023-12-19T18:16:49.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.618+0000 7f22b2afc640 1 -- 172.21.15.26:0/2184792275 wait complete. 2023-12-19T18:16:49.640 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2023-12-19T18:16:49.309160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596333,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":667,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":426,"ondisk_log_size":426,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":423304,"kb_used_data":22344,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754551416,"statfs":{"total":773094113280,"available":772660649984,"internally_reserved":0,"allocated":22880256,"data_stored":12639610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.763440"},"pg_stats":[{"pgid":"27.1a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357761+0000","last_change":"2023-12-19T18:16:40.564006+0000","last_active":"2023-12-19T18:16:42.357761+0000","last_peered":"2023-12-19T18:16:42.357761+0000","last_clean":"2023-12-19T18:16:42.357761+0000","last_became_active":"2023-12-19T18:16:40.563756+0000","last_became_peered":"2023-12-19T18:16:40.563756+0000","last_unstale":"2023-12-19T18:16:42.357761+0000","last_undegraded":"2023-12-19T18:16:42.357761+0000","last_fullsized":"2023-12-19T18:16:42.357761+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:17.484857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"200'346","reported_seq":1188,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560456+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:16:40.560456+0000","last_peered":"2023-12-19T18:16:40.560456+0000","last_clean":"2023-12-19T18:16:40.560456+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:16:40.560456+0000","last_undegraded":"2023-12-19T18:16:40.560456+0000","last_fullsized":"2023-12-19T18:16:40.560456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560313+0000","last_change":"2023-12-19T18:16:27.411230+0000","last_active":"2023-12-19T18:16:40.560313+0000","last_peered":"2023-12-19T18:16:40.560313+0000","last_clean":"2023-12-19T18:16:40.560313+0000","last_became_active":"2023-12-19T18:16:27.410670+0000","last_became_peered":"2023-12-19T18:16:27.410670+0000","last_unstale":"2023-12-19T18:16:40.560313+0000","last_undegraded":"2023-12-19T18:16:40.560313+0000","last_fullsized":"2023-12-19T18:16:40.560313+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:22.426446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554070+0000","last_change":"2023-12-19T18:16:27.821899+0000","last_active":"2023-12-19T18:16:40.554070+0000","last_peered":"2023-12-19T18:16:40.554070+0000","last_clean":"2023-12-19T18:16:40.554070+0000","last_became_active":"2023-12-19T18:16:27.821634+0000","last_became_peered":"2023-12-19T18:16:27.821634+0000","last_unstale":"2023-12-19T18:16:40.554070+0000","last_undegraded":"2023-12-19T18:16:40.554070+0000","last_fullsized":"2023-12-19T18:16:40.554070+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:34.391068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857383+0000","last_change":"2023-12-19T18:16:40.150476+0000","last_active":"2023-12-19T18:16:41.857383+0000","last_peered":"2023-12-19T18:16:41.857383+0000","last_clean":"2023-12-19T18:16:41.857383+0000","last_became_active":"2023-12-19T18:16:40.149843+0000","last_became_peered":"2023-12-19T18:16:40.149843+0000","last_unstale":"2023-12-19T18:16:41.857383+0000","last_undegraded":"2023-12-19T18:16:41.857383+0000","last_fullsized":"2023-12-19T18:16:41.857383+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:35.179628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857177+0000","last_change":"2023-12-19T18:16:40.569150+0000","last_active":"2023-12-19T18:16:41.857177+0000","last_peered":"2023-12-19T18:16:41.857177+0000","last_clean":"2023-12-19T18:16:41.857177+0000","last_became_active":"2023-12-19T18:16:40.568754+0000","last_became_peered":"2023-12-19T18:16:40.568754+0000","last_unstale":"2023-12-19T18:16:41.857177+0000","last_undegraded":"2023-12-19T18:16:41.857177+0000","last_fullsized":"2023-12-19T18:16:41.857177+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:32.689972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548913+0000","last_change":"2023-12-19T18:16:27.412171+0000","last_active":"2023-12-19T18:16:40.548913+0000","last_peered":"2023-12-19T18:16:40.548913+0000","last_clean":"2023-12-19T18:16:40.548913+0000","last_became_active":"2023-12-19T18:16:27.411872+0000","last_became_peered":"2023-12-19T18:16:27.411872+0000","last_unstale":"2023-12-19T18:16:40.548913+0000","last_undegraded":"2023-12-19T18:16:40.548913+0000","last_fullsized":"2023-12-19T18:16:40.548913+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:48.264905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548191+0000","last_change":"2023-12-19T18:16:27.832432+0000","last_active":"2023-12-19T18:16:40.548191+0000","last_peered":"2023-12-19T18:16:40.548191+0000","last_clean":"2023-12-19T18:16:40.548191+0000","last_became_active":"2023-12-19T18:16:27.832116+0000","last_became_peered":"2023-12-19T18:16:27.832116+0000","last_unstale":"2023-12-19T18:16:40.548191+0000","last_undegraded":"2023-12-19T18:16:40.548191+0000","last_fullsized":"2023-12-19T18:16:40.548191+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:35.165338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359499+0000","last_change":"2023-12-19T18:16:40.157009+0000","last_active":"2023-12-19T18:16:42.359499+0000","last_peered":"2023-12-19T18:16:42.359499+0000","last_clean":"2023-12-19T18:16:42.359499+0000","last_became_active":"2023-12-19T18:16:40.156734+0000","last_became_peered":"2023-12-19T18:16:40.156734+0000","last_unstale":"2023-12-19T18:16:42.359499+0000","last_undegraded":"2023-12-19T18:16:42.359499+0000","last_fullsized":"2023-12-19T18:16:42.359499+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:18.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357983+0000","last_change":"2023-12-19T18:16:40.560903+0000","last_active":"2023-12-19T18:16:42.357983+0000","last_peered":"2023-12-19T18:16:42.357983+0000","last_clean":"2023-12-19T18:16:42.357983+0000","last_became_active":"2023-12-19T18:16:40.560636+0000","last_became_peered":"2023-12-19T18:16:40.560636+0000","last_unstale":"2023-12-19T18:16:42.357983+0000","last_undegraded":"2023-12-19T18:16:42.357983+0000","last_fullsized":"2023-12-19T18:16:42.357983+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:25:38.247442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561435+0000","last_change":"2023-12-19T18:16:27.407456+0000","last_active":"2023-12-19T18:16:40.561435+0000","last_peered":"2023-12-19T18:16:40.561435+0000","last_clean":"2023-12-19T18:16:40.561435+0000","last_became_active":"2023-12-19T18:16:27.407208+0000","last_became_peered":"2023-12-19T18:16:27.407208+0000","last_unstale":"2023-12-19T18:16:40.561435+0000","last_undegraded":"2023-12-19T18:16:40.561435+0000","last_fullsized":"2023-12-19T18:16:40.561435+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:07.654229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560528+0000","last_change":"2023-12-19T18:16:27.824001+0000","last_active":"2023-12-19T18:16:40.560528+0000","last_peered":"2023-12-19T18:16:40.560528+0000","last_clean":"2023-12-19T18:16:40.560528+0000","last_became_active":"2023-12-19T18:16:27.823010+0000","last_became_peered":"2023-12-19T18:16:27.823010+0000","last_unstale":"2023-12-19T18:16:40.560528+0000","last_undegraded":"2023-12-19T18:16:40.560528+0000","last_fullsized":"2023-12-19T18:16:40.560528+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:30:56.775994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074880+0000","last_change":"2023-12-19T18:16:40.164940+0000","last_active":"2023-12-19T18:16:42.074880+0000","last_peered":"2023-12-19T18:16:42.074880+0000","last_clean":"2023-12-19T18:16:42.074880+0000","last_became_active":"2023-12-19T18:16:40.163710+0000","last_became_peered":"2023-12-19T18:16:40.163710+0000","last_unstale":"2023-12-19T18:16:42.074880+0000","last_undegraded":"2023-12-19T18:16:42.074880+0000","last_fullsized":"2023-12-19T18:16:42.074880+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:29.482411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552525+0000","last_change":"2023-12-19T18:16:40.570024+0000","last_active":"2023-12-19T18:16:41.552525+0000","last_peered":"2023-12-19T18:16:41.552525+0000","last_clean":"2023-12-19T18:16:41.552525+0000","last_became_active":"2023-12-19T18:16:40.569586+0000","last_became_peered":"2023-12-19T18:16:40.569586+0000","last_unstale":"2023-12-19T18:16:41.552525+0000","last_undegraded":"2023-12-19T18:16:41.552525+0000","last_fullsized":"2023-12-19T18:16:41.552525+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:03.932038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547644+0000","last_change":"2023-12-19T18:16:27.405381+0000","last_active":"2023-12-19T18:16:40.547644+0000","last_peered":"2023-12-19T18:16:40.547644+0000","last_clean":"2023-12-19T18:16:40.547644+0000","last_became_active":"2023-12-19T18:16:27.405113+0000","last_became_peered":"2023-12-19T18:16:27.405113+0000","last_unstale":"2023-12-19T18:16:40.547644+0000","last_undegraded":"2023-12-19T18:16:40.547644+0000","last_fullsized":"2023-12-19T18:16:40.547644+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:08.911302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560761+0000","last_change":"2023-12-19T18:16:27.822423+0000","last_active":"2023-12-19T18:16:40.560761+0000","last_peered":"2023-12-19T18:16:40.560761+0000","last_clean":"2023-12-19T18:16:40.560761+0000","last_became_active":"2023-12-19T18:16:27.822106+0000","last_became_peered":"2023-12-19T18:16:27.822106+0000","last_unstale":"2023-12-19T18:16:40.560761+0000","last_undegraded":"2023-12-19T18:16:40.560761+0000","last_fullsized":"2023-12-19T18:16:40.560761+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:41.021760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.856687+0000","last_change":"2023-12-19T18:16:40.154697+0000","last_active":"2023-12-19T18:16:41.856687+0000","last_peered":"2023-12-19T18:16:41.856687+0000","last_clean":"2023-12-19T18:16:41.856687+0000","last_became_active":"2023-12-19T18:16:40.154394+0000","last_became_peered":"2023-12-19T18:16:40.154394+0000","last_unstale":"2023-12-19T18:16:41.856687+0000","last_undegraded":"2023-12-19T18:16:41.856687+0000","last_fullsized":"2023-12-19T18:16:41.856687+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:52.755953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357512+0000","last_change":"2023-12-19T18:16:40.572030+0000","last_active":"2023-12-19T18:16:42.357512+0000","last_peered":"2023-12-19T18:16:42.357512+0000","last_clean":"2023-12-19T18:16:42.357512+0000","last_became_active":"2023-12-19T18:16:40.570330+0000","last_became_peered":"2023-12-19T18:16:40.570330+0000","last_unstale":"2023-12-19T18:16:42.357512+0000","last_undegraded":"2023-12-19T18:16:42.357512+0000","last_fullsized":"2023-12-19T18:16:42.357512+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:55.123677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547774+0000","last_change":"2023-12-19T18:16:27.410513+0000","last_active":"2023-12-19T18:16:40.547774+0000","last_peered":"2023-12-19T18:16:40.547774+0000","last_clean":"2023-12-19T18:16:40.547774+0000","last_became_active":"2023-12-19T18:16:27.410166+0000","last_became_peered":"2023-12-19T18:16:27.410166+0000","last_unstale":"2023-12-19T18:16:40.547774+0000","last_undegraded":"2023-12-19T18:16:40.547774+0000","last_fullsized":"2023-12-19T18:16:40.547774+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:10.682880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560587+0000","last_change":"2023-12-19T18:16:27.829925+0000","last_active":"2023-12-19T18:16:40.560587+0000","last_peered":"2023-12-19T18:16:40.560587+0000","last_clean":"2023-12-19T18:16:40.560587+0000","last_became_active":"2023-12-19T18:16:27.828958+0000","last_became_peered":"2023-12-19T18:16:27.828958+0000","last_unstale":"2023-12-19T18:16:40.560587+0000","last_undegraded":"2023-12-19T18:16:40.560587+0000","last_fullsized":"2023-12-19T18:16:40.560587+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:46.995476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554304+0000","last_change":"2023-12-19T18:16:40.156673+0000","last_active":"2023-12-19T18:16:41.554304+0000","last_peered":"2023-12-19T18:16:41.554304+0000","last_clean":"2023-12-19T18:16:41.554304+0000","last_became_active":"2023-12-19T18:16:40.156391+0000","last_became_peered":"2023-12-19T18:16:40.156391+0000","last_unstale":"2023-12-19T18:16:41.554304+0000","last_undegraded":"2023-12-19T18:16:41.554304+0000","last_fullsized":"2023-12-19T18:16:41.554304+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:46.681719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360169+0000","last_change":"2023-12-19T18:16:40.571314+0000","last_active":"2023-12-19T18:16:42.360169+0000","last_peered":"2023-12-19T18:16:42.360169+0000","last_clean":"2023-12-19T18:16:42.360169+0000","last_became_active":"2023-12-19T18:16:40.570538+0000","last_became_peered":"2023-12-19T18:16:40.570538+0000","last_unstale":"2023-12-19T18:16:42.360169+0000","last_undegraded":"2023-12-19T18:16:42.360169+0000","last_fullsized":"2023-12-19T18:16:42.360169+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:02:14.652060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543489+0000","last_change":"2023-12-19T18:16:27.409169+0000","last_active":"2023-12-19T18:16:40.543489+0000","last_peered":"2023-12-19T18:16:40.543489+0000","last_clean":"2023-12-19T18:16:40.543489+0000","last_became_active":"2023-12-19T18:16:27.408882+0000","last_became_peered":"2023-12-19T18:16:27.408882+0000","last_unstale":"2023-12-19T18:16:40.543489+0000","last_undegraded":"2023-12-19T18:16:40.543489+0000","last_fullsized":"2023-12-19T18:16:40.543489+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:01.478246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544313+0000","last_change":"2023-12-19T18:16:27.825200+0000","last_active":"2023-12-19T18:16:40.544313+0000","last_peered":"2023-12-19T18:16:40.544313+0000","last_clean":"2023-12-19T18:16:40.544313+0000","last_became_active":"2023-12-19T18:16:27.824919+0000","last_became_peered":"2023-12-19T18:16:27.824919+0000","last_unstale":"2023-12-19T18:16:40.544313+0000","last_undegraded":"2023-12-19T18:16:40.544313+0000","last_fullsized":"2023-12-19T18:16:40.544313+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.6","version":"216'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.077099+0000","last_change":"2023-12-19T18:16:40.164339+0000","last_active":"2023-12-19T18:16:42.077099+0000","last_peered":"2023-12-19T18:16:42.077099+0000","last_clean":"2023-12-19T18:16:42.077099+0000","last_became_active":"2023-12-19T18:16:40.163870+0000","last_became_peered":"2023-12-19T18:16:40.163870+0000","last_unstale":"2023-12-19T18:16:42.077099+0000","last_undegraded":"2023-12-19T18:16:42.077099+0000","last_fullsized":"2023-12-19T18:16:42.077099+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:14:22.971741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857196+0000","last_change":"2023-12-19T18:16:40.568314+0000","last_active":"2023-12-19T18:16:41.857196+0000","last_peered":"2023-12-19T18:16:41.857196+0000","last_clean":"2023-12-19T18:16:41.857196+0000","last_became_active":"2023-12-19T18:16:40.567857+0000","last_became_peered":"2023-12-19T18:16:40.567857+0000","last_unstale":"2023-12-19T18:16:41.857196+0000","last_undegraded":"2023-12-19T18:16:41.857196+0000","last_fullsized":"2023-12-19T18:16:41.857196+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:49.703164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555637+0000","last_change":"2023-12-19T18:16:27.420249+0000","last_active":"2023-12-19T18:16:40.555637+0000","last_peered":"2023-12-19T18:16:40.555637+0000","last_clean":"2023-12-19T18:16:40.555637+0000","last_became_active":"2023-12-19T18:16:27.419360+0000","last_became_peered":"2023-12-19T18:16:27.419360+0000","last_unstale":"2023-12-19T18:16:40.555637+0000","last_undegraded":"2023-12-19T18:16:40.555637+0000","last_fullsized":"2023-12-19T18:16:40.555637+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:07.969904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543383+0000","last_change":"2023-12-19T18:16:27.825535+0000","last_active":"2023-12-19T18:16:40.543383+0000","last_peered":"2023-12-19T18:16:40.543383+0000","last_clean":"2023-12-19T18:16:40.543383+0000","last_became_active":"2023-12-19T18:16:27.824814+0000","last_became_peered":"2023-12-19T18:16:27.824814+0000","last_unstale":"2023-12-19T18:16:40.543383+0000","last_undegraded":"2023-12-19T18:16:40.543383+0000","last_fullsized":"2023-12-19T18:16:40.543383+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:29.357409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358846+0000","last_change":"2023-12-19T18:16:40.160113+0000","last_active":"2023-12-19T18:16:42.358846+0000","last_peered":"2023-12-19T18:16:42.358846+0000","last_clean":"2023-12-19T18:16:42.358846+0000","last_became_active":"2023-12-19T18:16:40.159826+0000","last_became_peered":"2023-12-19T18:16:40.159826+0000","last_unstale":"2023-12-19T18:16:42.358846+0000","last_undegraded":"2023-12-19T18:16:42.358846+0000","last_fullsized":"2023-12-19T18:16:42.358846+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:45.058000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554995+0000","last_change":"2023-12-19T18:16:40.566980+0000","last_active":"2023-12-19T18:16:41.554995+0000","last_peered":"2023-12-19T18:16:41.554995+0000","last_clean":"2023-12-19T18:16:41.554995+0000","last_became_active":"2023-12-19T18:16:40.566716+0000","last_became_peered":"2023-12-19T18:16:40.566716+0000","last_unstale":"2023-12-19T18:16:41.554995+0000","last_undegraded":"2023-12-19T18:16:41.554995+0000","last_fullsized":"2023-12-19T18:16:41.554995+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:30.489642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.6","version":"212'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558804+0000","last_change":"2023-12-19T18:16:27.414152+0000","last_active":"2023-12-19T18:16:40.558804+0000","last_peered":"2023-12-19T18:16:40.558804+0000","last_clean":"2023-12-19T18:16:40.558804+0000","last_became_active":"2023-12-19T18:16:27.413877+0000","last_became_peered":"2023-12-19T18:16:27.413877+0000","last_unstale":"2023-12-19T18:16:40.558804+0000","last_undegraded":"2023-12-19T18:16:40.558804+0000","last_fullsized":"2023-12-19T18:16:40.558804+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:44:52.553685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558691+0000","last_change":"2023-12-19T18:16:27.824257+0000","last_active":"2023-12-19T18:16:40.558691+0000","last_peered":"2023-12-19T18:16:40.558691+0000","last_clean":"2023-12-19T18:16:40.558691+0000","last_became_active":"2023-12-19T18:16:27.823989+0000","last_became_peered":"2023-12-19T18:16:27.823989+0000","last_unstale":"2023-12-19T18:16:40.558691+0000","last_undegraded":"2023-12-19T18:16:40.558691+0000","last_fullsized":"2023-12-19T18:16:40.558691+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:07:07.505127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357977+0000","last_change":"2023-12-19T18:16:40.160316+0000","last_active":"2023-12-19T18:16:42.357977+0000","last_peered":"2023-12-19T18:16:42.357977+0000","last_clean":"2023-12-19T18:16:42.357977+0000","last_became_active":"2023-12-19T18:16:40.159312+0000","last_became_peered":"2023-12-19T18:16:40.159312+0000","last_unstale":"2023-12-19T18:16:42.357977+0000","last_undegraded":"2023-12-19T18:16:42.357977+0000","last_fullsized":"2023-12-19T18:16:42.357977+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:04.832569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553051+0000","last_change":"2023-12-19T18:16:40.563986+0000","last_active":"2023-12-19T18:16:41.553051+0000","last_peered":"2023-12-19T18:16:41.553051+0000","last_clean":"2023-12-19T18:16:41.553051+0000","last_became_active":"2023-12-19T18:16:40.563693+0000","last_became_peered":"2023-12-19T18:16:40.563693+0000","last_unstale":"2023-12-19T18:16:41.553051+0000","last_undegraded":"2023-12-19T18:16:41.553051+0000","last_fullsized":"2023-12-19T18:16:41.553051+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:27.465263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562303+0000","last_change":"2023-12-19T18:16:27.409874+0000","last_active":"2023-12-19T18:16:40.562303+0000","last_peered":"2023-12-19T18:16:40.562303+0000","last_clean":"2023-12-19T18:16:40.562303+0000","last_became_active":"2023-12-19T18:16:27.409598+0000","last_became_peered":"2023-12-19T18:16:27.409598+0000","last_unstale":"2023-12-19T18:16:40.562303+0000","last_undegraded":"2023-12-19T18:16:40.562303+0000","last_fullsized":"2023-12-19T18:16:40.562303+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:13.721057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549450+0000","last_change":"2023-12-19T18:16:27.826090+0000","last_active":"2023-12-19T18:16:40.549450+0000","last_peered":"2023-12-19T18:16:40.549450+0000","last_clean":"2023-12-19T18:16:40.549450+0000","last_became_active":"2023-12-19T18:16:27.825791+0000","last_became_peered":"2023-12-19T18:16:27.825791+0000","last_unstale":"2023-12-19T18:16:40.549450+0000","last_undegraded":"2023-12-19T18:16:40.549450+0000","last_fullsized":"2023-12-19T18:16:40.549450+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:39.382953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359959+0000","last_change":"2023-12-19T18:16:40.160037+0000","last_active":"2023-12-19T18:16:42.359959+0000","last_peered":"2023-12-19T18:16:42.359959+0000","last_clean":"2023-12-19T18:16:42.359959+0000","last_became_active":"2023-12-19T18:16:40.159737+0000","last_became_peered":"2023-12-19T18:16:40.159737+0000","last_unstale":"2023-12-19T18:16:42.359959+0000","last_undegraded":"2023-12-19T18:16:42.359959+0000","last_fullsized":"2023-12-19T18:16:42.359959+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:42.336188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358621+0000","last_change":"2023-12-19T18:16:40.567065+0000","last_active":"2023-12-19T18:16:42.358621+0000","last_peered":"2023-12-19T18:16:42.358621+0000","last_clean":"2023-12-19T18:16:42.358621+0000","last_became_active":"2023-12-19T18:16:40.566804+0000","last_became_peered":"2023-12-19T18:16:40.566804+0000","last_unstale":"2023-12-19T18:16:42.358621+0000","last_undegraded":"2023-12-19T18:16:42.358621+0000","last_fullsized":"2023-12-19T18:16:42.358621+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:21.029140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546688+0000","last_change":"2023-12-19T18:16:27.403537+0000","last_active":"2023-12-19T18:16:40.546688+0000","last_peered":"2023-12-19T18:16:40.546688+0000","last_clean":"2023-12-19T18:16:40.546688+0000","last_became_active":"2023-12-19T18:16:27.403250+0000","last_became_peered":"2023-12-19T18:16:27.403250+0000","last_unstale":"2023-12-19T18:16:40.546688+0000","last_undegraded":"2023-12-19T18:16:40.546688+0000","last_fullsized":"2023-12-19T18:16:40.546688+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:56.889902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548113+0000","last_change":"2023-12-19T18:16:27.825355+0000","last_active":"2023-12-19T18:16:40.548113+0000","last_peered":"2023-12-19T18:16:40.548113+0000","last_clean":"2023-12-19T18:16:40.548113+0000","last_became_active":"2023-12-19T18:16:27.825067+0000","last_became_peered":"2023-12-19T18:16:27.825067+0000","last_unstale":"2023-12-19T18:16:40.548113+0000","last_undegraded":"2023-12-19T18:16:40.548113+0000","last_fullsized":"2023-12-19T18:16:40.548113+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:02.928407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358143+0000","last_change":"2023-12-19T18:16:40.152882+0000","last_active":"2023-12-19T18:16:42.358143+0000","last_peered":"2023-12-19T18:16:42.358143+0000","last_clean":"2023-12-19T18:16:42.358143+0000","last_became_active":"2023-12-19T18:16:40.152561+0000","last_became_peered":"2023-12-19T18:16:40.152561+0000","last_unstale":"2023-12-19T18:16:42.358143+0000","last_undegraded":"2023-12-19T18:16:42.358143+0000","last_fullsized":"2023-12-19T18:16:42.358143+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:11:26.922031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550797+0000","last_change":"2023-12-19T18:16:40.560282+0000","last_active":"2023-12-19T18:16:41.550797+0000","last_peered":"2023-12-19T18:16:41.550797+0000","last_clean":"2023-12-19T18:16:41.550797+0000","last_became_active":"2023-12-19T18:16:40.560017+0000","last_became_peered":"2023-12-19T18:16:40.560017+0000","last_unstale":"2023-12-19T18:16:41.550797+0000","last_undegraded":"2023-12-19T18:16:41.550797+0000","last_fullsized":"2023-12-19T18:16:41.550797+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:51:01.095692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553689+0000","last_change":"2023-12-19T18:16:27.417198+0000","last_active":"2023-12-19T18:16:40.553689+0000","last_peered":"2023-12-19T18:16:40.553689+0000","last_clean":"2023-12-19T18:16:40.553689+0000","last_became_active":"2023-12-19T18:16:27.416902+0000","last_became_peered":"2023-12-19T18:16:27.416902+0000","last_unstale":"2023-12-19T18:16:40.553689+0000","last_undegraded":"2023-12-19T18:16:40.553689+0000","last_fullsized":"2023-12-19T18:16:40.553689+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:54.434372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560449+0000","last_change":"2023-12-19T18:16:27.824198+0000","last_active":"2023-12-19T18:16:40.560449+0000","last_peered":"2023-12-19T18:16:40.560449+0000","last_clean":"2023-12-19T18:16:40.560449+0000","last_became_active":"2023-12-19T18:16:27.823738+0000","last_became_peered":"2023-12-19T18:16:27.823738+0000","last_unstale":"2023-12-19T18:16:40.560449+0000","last_undegraded":"2023-12-19T18:16:40.560449+0000","last_fullsized":"2023-12-19T18:16:40.560449+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:53.509897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550952+0000","last_change":"2023-12-19T18:16:40.160436+0000","last_active":"2023-12-19T18:16:41.550952+0000","last_peered":"2023-12-19T18:16:41.550952+0000","last_clean":"2023-12-19T18:16:41.550952+0000","last_became_active":"2023-12-19T18:16:40.159692+0000","last_became_peered":"2023-12-19T18:16:40.159692+0000","last_unstale":"2023-12-19T18:16:41.550952+0000","last_undegraded":"2023-12-19T18:16:41.550952+0000","last_fullsized":"2023-12-19T18:16:41.550952+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:47:01.985064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554294+0000","last_change":"2023-12-19T18:16:40.572114+0000","last_active":"2023-12-19T18:16:41.554294+0000","last_peered":"2023-12-19T18:16:41.554294+0000","last_clean":"2023-12-19T18:16:41.554294+0000","last_became_active":"2023-12-19T18:16:40.571811+0000","last_became_peered":"2023-12-19T18:16:40.571811+0000","last_unstale":"2023-12-19T18:16:41.554294+0000","last_undegraded":"2023-12-19T18:16:41.554294+0000","last_fullsized":"2023-12-19T18:16:41.554294+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:01.072159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550771+0000","last_change":"2023-12-19T18:16:27.404853+0000","last_active":"2023-12-19T18:16:40.550771+0000","last_peered":"2023-12-19T18:16:40.550771+0000","last_clean":"2023-12-19T18:16:40.550771+0000","last_became_active":"2023-12-19T18:16:27.404347+0000","last_became_peered":"2023-12-19T18:16:27.404347+0000","last_unstale":"2023-12-19T18:16:40.550771+0000","last_undegraded":"2023-12-19T18:16:40.550771+0000","last_fullsized":"2023-12-19T18:16:40.550771+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:16.360996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558229+0000","last_change":"2023-12-19T18:16:27.828160+0000","last_active":"2023-12-19T18:16:40.558229+0000","last_peered":"2023-12-19T18:16:40.558229+0000","last_clean":"2023-12-19T18:16:40.558229+0000","last_became_active":"2023-12-19T18:16:27.827882+0000","last_became_peered":"2023-12-19T18:16:27.827882+0000","last_unstale":"2023-12-19T18:16:40.558229+0000","last_undegraded":"2023-12-19T18:16:40.558229+0000","last_fullsized":"2023-12-19T18:16:40.558229+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:17.726704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358933+0000","last_change":"2023-12-19T18:16:40.154706+0000","last_active":"2023-12-19T18:16:42.358933+0000","last_peered":"2023-12-19T18:16:42.358933+0000","last_clean":"2023-12-19T18:16:42.358933+0000","last_became_active":"2023-12-19T18:16:40.154154+0000","last_became_peered":"2023-12-19T18:16:40.154154+0000","last_unstale":"2023-12-19T18:16:42.358933+0000","last_undegraded":"2023-12-19T18:16:42.358933+0000","last_fullsized":"2023-12-19T18:16:42.358933+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:55.343366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552500+0000","last_change":"2023-12-19T18:16:40.555828+0000","last_active":"2023-12-19T18:16:41.552500+0000","last_peered":"2023-12-19T18:16:41.552500+0000","last_clean":"2023-12-19T18:16:41.552500+0000","last_became_active":"2023-12-19T18:16:40.555500+0000","last_became_peered":"2023-12-19T18:16:40.555500+0000","last_unstale":"2023-12-19T18:16:41.552500+0000","last_undegraded":"2023-12-19T18:16:41.552500+0000","last_fullsized":"2023-12-19T18:16:41.552500+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:50.483528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.b","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551100+0000","last_change":"2023-12-19T18:16:27.415902+0000","last_active":"2023-12-19T18:16:40.551100+0000","last_peered":"2023-12-19T18:16:40.551100+0000","last_clean":"2023-12-19T18:16:40.551100+0000","last_became_active":"2023-12-19T18:16:27.415062+0000","last_became_peered":"2023-12-19T18:16:27.415062+0000","last_unstale":"2023-12-19T18:16:40.551100+0000","last_undegraded":"2023-12-19T18:16:40.551100+0000","last_fullsized":"2023-12-19T18:16:40.551100+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:22.442653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556868+0000","last_change":"2023-12-19T18:16:27.827898+0000","last_active":"2023-12-19T18:16:40.556868+0000","last_peered":"2023-12-19T18:16:40.556868+0000","last_clean":"2023-12-19T18:16:40.556868+0000","last_became_active":"2023-12-19T18:16:27.827613+0000","last_became_peered":"2023-12-19T18:16:27.827613+0000","last_unstale":"2023-12-19T18:16:40.556868+0000","last_undegraded":"2023-12-19T18:16:40.556868+0000","last_fullsized":"2023-12-19T18:16:40.556868+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:29.033205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357614+0000","last_change":"2023-12-19T18:16:40.157634+0000","last_active":"2023-12-19T18:16:42.357614+0000","last_peered":"2023-12-19T18:16:42.357614+0000","last_clean":"2023-12-19T18:16:42.357614+0000","last_became_active":"2023-12-19T18:16:40.157363+0000","last_became_peered":"2023-12-19T18:16:40.157363+0000","last_unstale":"2023-12-19T18:16:42.357614+0000","last_undegraded":"2023-12-19T18:16:42.357614+0000","last_fullsized":"2023-12-19T18:16:42.357614+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:23.564600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553681+0000","last_change":"2023-12-19T18:16:40.556441+0000","last_active":"2023-12-19T18:16:41.553681+0000","last_peered":"2023-12-19T18:16:41.553681+0000","last_clean":"2023-12-19T18:16:41.553681+0000","last_became_active":"2023-12-19T18:16:40.556187+0000","last_became_peered":"2023-12-19T18:16:40.556187+0000","last_unstale":"2023-12-19T18:16:41.553681+0000","last_undegraded":"2023-12-19T18:16:41.553681+0000","last_fullsized":"2023-12-19T18:16:41.553681+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:16:05.083354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562653+0000","last_change":"2023-12-19T18:16:27.408051+0000","last_active":"2023-12-19T18:16:40.562653+0000","last_peered":"2023-12-19T18:16:40.562653+0000","last_clean":"2023-12-19T18:16:40.562653+0000","last_became_active":"2023-12-19T18:16:27.407796+0000","last_became_peered":"2023-12-19T18:16:27.407796+0000","last_unstale":"2023-12-19T18:16:40.562653+0000","last_undegraded":"2023-12-19T18:16:40.562653+0000","last_fullsized":"2023-12-19T18:16:40.562653+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:41:09.697612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560009+0000","last_change":"2023-12-19T18:16:27.832376+0000","last_active":"2023-12-19T18:16:40.560009+0000","last_peered":"2023-12-19T18:16:40.560009+0000","last_clean":"2023-12-19T18:16:40.560009+0000","last_became_active":"2023-12-19T18:16:27.832060+0000","last_became_peered":"2023-12-19T18:16:27.832060+0000","last_unstale":"2023-12-19T18:16:40.560009+0000","last_undegraded":"2023-12-19T18:16:40.560009+0000","last_fullsized":"2023-12-19T18:16:40.560009+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:58:17.432889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.e","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553537+0000","last_change":"2023-12-19T18:16:40.152827+0000","last_active":"2023-12-19T18:16:41.553537+0000","last_peered":"2023-12-19T18:16:41.553537+0000","last_clean":"2023-12-19T18:16:41.553537+0000","last_became_active":"2023-12-19T18:16:40.152559+0000","last_became_peered":"2023-12-19T18:16:40.152559+0000","last_unstale":"2023-12-19T18:16:41.553537+0000","last_undegraded":"2023-12-19T18:16:41.553537+0000","last_fullsized":"2023-12-19T18:16:41.553537+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:59.363074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553423+0000","last_change":"2023-12-19T18:16:40.572243+0000","last_active":"2023-12-19T18:16:41.553423+0000","last_peered":"2023-12-19T18:16:41.553423+0000","last_clean":"2023-12-19T18:16:41.553423+0000","last_became_active":"2023-12-19T18:16:40.571938+0000","last_became_peered":"2023-12-19T18:16:40.571938+0000","last_unstale":"2023-12-19T18:16:41.553423+0000","last_undegraded":"2023-12-19T18:16:41.553423+0000","last_fullsized":"2023-12-19T18:16:41.553423+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:50.735904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562479+0000","last_change":"2023-12-19T18:16:27.416654+0000","last_active":"2023-12-19T18:16:40.562479+0000","last_peered":"2023-12-19T18:16:40.562479+0000","last_clean":"2023-12-19T18:16:40.562479+0000","last_became_active":"2023-12-19T18:16:27.416388+0000","last_became_peered":"2023-12-19T18:16:27.416388+0000","last_unstale":"2023-12-19T18:16:40.562479+0000","last_undegraded":"2023-12-19T18:16:40.562479+0000","last_fullsized":"2023-12-19T18:16:40.562479+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:51.339148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547637+0000","last_change":"2023-12-19T18:16:27.816436+0000","last_active":"2023-12-19T18:16:40.547637+0000","last_peered":"2023-12-19T18:16:40.547637+0000","last_clean":"2023-12-19T18:16:40.547637+0000","last_became_active":"2023-12-19T18:16:27.816092+0000","last_became_peered":"2023-12-19T18:16:27.816092+0000","last_unstale":"2023-12-19T18:16:40.547637+0000","last_undegraded":"2023-12-19T18:16:40.547637+0000","last_fullsized":"2023-12-19T18:16:40.547637+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:07:26.713649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360338+0000","last_change":"2023-12-19T18:16:40.154797+0000","last_active":"2023-12-19T18:16:42.360338+0000","last_peered":"2023-12-19T18:16:42.360338+0000","last_clean":"2023-12-19T18:16:42.360338+0000","last_became_active":"2023-12-19T18:16:40.154490+0000","last_became_peered":"2023-12-19T18:16:40.154490+0000","last_unstale":"2023-12-19T18:16:42.360338+0000","last_undegraded":"2023-12-19T18:16:42.360338+0000","last_fullsized":"2023-12-19T18:16:42.360338+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:10:02.821689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551216+0000","last_change":"2023-12-19T18:16:40.573560+0000","last_active":"2023-12-19T18:16:41.551216+0000","last_peered":"2023-12-19T18:16:41.551216+0000","last_clean":"2023-12-19T18:16:41.551216+0000","last_became_active":"2023-12-19T18:16:40.573277+0000","last_became_peered":"2023-12-19T18:16:40.573277+0000","last_unstale":"2023-12-19T18:16:41.551216+0000","last_undegraded":"2023-12-19T18:16:41.551216+0000","last_fullsized":"2023-12-19T18:16:41.551216+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:23:53.236602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544525+0000","last_change":"2023-12-19T18:16:27.409426+0000","last_active":"2023-12-19T18:16:40.544525+0000","last_peered":"2023-12-19T18:16:40.544525+0000","last_clean":"2023-12-19T18:16:40.544525+0000","last_became_active":"2023-12-19T18:16:27.407151+0000","last_became_peered":"2023-12-19T18:16:27.407151+0000","last_unstale":"2023-12-19T18:16:40.544525+0000","last_undegraded":"2023-12-19T18:16:40.544525+0000","last_fullsized":"2023-12-19T18:16:40.544525+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:30:56.976667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550067+0000","last_change":"2023-12-19T18:16:27.823846+0000","last_active":"2023-12-19T18:16:40.550067+0000","last_peered":"2023-12-19T18:16:40.550067+0000","last_clean":"2023-12-19T18:16:40.550067+0000","last_became_active":"2023-12-19T18:16:27.823514+0000","last_became_peered":"2023-12-19T18:16:27.823514+0000","last_unstale":"2023-12-19T18:16:40.550067+0000","last_undegraded":"2023-12-19T18:16:40.550067+0000","last_fullsized":"2023-12-19T18:16:40.550067+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:20.691537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358069+0000","last_change":"2023-12-19T18:16:40.162626+0000","last_active":"2023-12-19T18:16:42.358069+0000","last_peered":"2023-12-19T18:16:42.358069+0000","last_clean":"2023-12-19T18:16:42.358069+0000","last_became_active":"2023-12-19T18:16:40.161257+0000","last_became_peered":"2023-12-19T18:16:40.161257+0000","last_unstale":"2023-12-19T18:16:42.358069+0000","last_undegraded":"2023-12-19T18:16:42.358069+0000","last_fullsized":"2023-12-19T18:16:42.358069+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:21:05.448555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357894+0000","last_change":"2023-12-19T18:16:40.558784+0000","last_active":"2023-12-19T18:16:42.357894+0000","last_peered":"2023-12-19T18:16:42.357894+0000","last_clean":"2023-12-19T18:16:42.357894+0000","last_became_active":"2023-12-19T18:16:40.556768+0000","last_became_peered":"2023-12-19T18:16:40.556768+0000","last_unstale":"2023-12-19T18:16:42.357894+0000","last_undegraded":"2023-12-19T18:16:42.357894+0000","last_fullsized":"2023-12-19T18:16:42.357894+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:14:19.106971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553891+0000","last_change":"2023-12-19T18:16:27.414971+0000","last_active":"2023-12-19T18:16:40.553891+0000","last_peered":"2023-12-19T18:16:40.553891+0000","last_clean":"2023-12-19T18:16:40.553891+0000","last_became_active":"2023-12-19T18:16:27.414662+0000","last_became_peered":"2023-12-19T18:16:27.414662+0000","last_unstale":"2023-12-19T18:16:40.553891+0000","last_undegraded":"2023-12-19T18:16:40.553891+0000","last_fullsized":"2023-12-19T18:16:40.553891+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:47.204637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558717+0000","last_change":"2023-12-19T18:16:27.814804+0000","last_active":"2023-12-19T18:16:40.558717+0000","last_peered":"2023-12-19T18:16:40.558717+0000","last_clean":"2023-12-19T18:16:40.558717+0000","last_became_active":"2023-12-19T18:16:27.814534+0000","last_became_peered":"2023-12-19T18:16:27.814534+0000","last_unstale":"2023-12-19T18:16:40.558717+0000","last_undegraded":"2023-12-19T18:16:40.558717+0000","last_fullsized":"2023-12-19T18:16:40.558717+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:45.690711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551146+0000","last_change":"2023-12-19T18:16:40.158484+0000","last_active":"2023-12-19T18:16:41.551146+0000","last_peered":"2023-12-19T18:16:41.551146+0000","last_clean":"2023-12-19T18:16:41.551146+0000","last_became_active":"2023-12-19T18:16:40.158205+0000","last_became_peered":"2023-12-19T18:16:40.158205+0000","last_unstale":"2023-12-19T18:16:41.551146+0000","last_undegraded":"2023-12-19T18:16:41.551146+0000","last_fullsized":"2023-12-19T18:16:41.551146+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:23.673873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550951+0000","last_change":"2023-12-19T18:16:40.555429+0000","last_active":"2023-12-19T18:16:41.550951+0000","last_peered":"2023-12-19T18:16:41.550951+0000","last_clean":"2023-12-19T18:16:41.550951+0000","last_became_active":"2023-12-19T18:16:40.555158+0000","last_became_peered":"2023-12-19T18:16:40.555158+0000","last_unstale":"2023-12-19T18:16:41.550951+0000","last_undegraded":"2023-12-19T18:16:41.550951+0000","last_fullsized":"2023-12-19T18:16:41.550951+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:53.628064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.10","version":"211'8","reported_seq":23,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560457+0000","last_change":"2023-12-19T18:16:27.415118+0000","last_active":"2023-12-19T18:16:40.560457+0000","last_peered":"2023-12-19T18:16:40.560457+0000","last_clean":"2023-12-19T18:16:40.560457+0000","last_became_active":"2023-12-19T18:16:27.414687+0000","last_became_peered":"2023-12-19T18:16:27.414687+0000","last_unstale":"2023-12-19T18:16:40.560457+0000","last_undegraded":"2023-12-19T18:16:40.560457+0000","last_fullsized":"2023-12-19T18:16:40.560457+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:16.723922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555219+0000","last_change":"2023-12-19T18:16:27.814277+0000","last_active":"2023-12-19T18:16:40.555219+0000","last_peered":"2023-12-19T18:16:40.555219+0000","last_clean":"2023-12-19T18:16:40.555219+0000","last_became_active":"2023-12-19T18:16:27.814023+0000","last_became_peered":"2023-12-19T18:16:27.814023+0000","last_unstale":"2023-12-19T18:16:40.555219+0000","last_undegraded":"2023-12-19T18:16:40.555219+0000","last_fullsized":"2023-12-19T18:16:40.555219+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:32.135133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554395+0000","last_change":"2023-12-19T18:16:40.154784+0000","last_active":"2023-12-19T18:16:41.554395+0000","last_peered":"2023-12-19T18:16:41.554395+0000","last_clean":"2023-12-19T18:16:41.554395+0000","last_became_active":"2023-12-19T18:16:40.154523+0000","last_became_peered":"2023-12-19T18:16:40.154523+0000","last_unstale":"2023-12-19T18:16:41.554395+0000","last_undegraded":"2023-12-19T18:16:41.554395+0000","last_fullsized":"2023-12-19T18:16:41.554395+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:14.215221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359308+0000","last_change":"2023-12-19T18:16:40.558066+0000","last_active":"2023-12-19T18:16:42.359308+0000","last_peered":"2023-12-19T18:16:42.359308+0000","last_clean":"2023-12-19T18:16:42.359308+0000","last_became_active":"2023-12-19T18:16:40.557771+0000","last_became_peered":"2023-12-19T18:16:40.557771+0000","last_unstale":"2023-12-19T18:16:42.359308+0000","last_undegraded":"2023-12-19T18:16:42.359308+0000","last_fullsized":"2023-12-19T18:16:42.359308+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:28.573589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543014+0000","last_change":"2023-12-19T18:16:27.403403+0000","last_active":"2023-12-19T18:16:40.543014+0000","last_peered":"2023-12-19T18:16:40.543014+0000","last_clean":"2023-12-19T18:16:40.543014+0000","last_became_active":"2023-12-19T18:16:27.403121+0000","last_became_peered":"2023-12-19T18:16:27.403121+0000","last_unstale":"2023-12-19T18:16:40.543014+0000","last_undegraded":"2023-12-19T18:16:40.543014+0000","last_fullsized":"2023-12-19T18:16:40.543014+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:36:58.560143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546582+0000","last_change":"2023-12-19T18:16:27.821814+0000","last_active":"2023-12-19T18:16:40.546582+0000","last_peered":"2023-12-19T18:16:40.546582+0000","last_clean":"2023-12-19T18:16:40.546582+0000","last_became_active":"2023-12-19T18:16:27.821519+0000","last_became_peered":"2023-12-19T18:16:27.821519+0000","last_unstale":"2023-12-19T18:16:40.546582+0000","last_undegraded":"2023-12-19T18:16:40.546582+0000","last_fullsized":"2023-12-19T18:16:40.546582+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:40.251745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.13","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554534+0000","last_change":"2023-12-19T18:16:40.160940+0000","last_active":"2023-12-19T18:16:41.554534+0000","last_peered":"2023-12-19T18:16:41.554534+0000","last_clean":"2023-12-19T18:16:41.554534+0000","last_became_active":"2023-12-19T18:16:40.160594+0000","last_became_peered":"2023-12-19T18:16:40.160594+0000","last_unstale":"2023-12-19T18:16:41.554534+0000","last_undegraded":"2023-12-19T18:16:41.554534+0000","last_fullsized":"2023-12-19T18:16:41.554534+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:01.011110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357451+0000","last_change":"2023-12-19T18:16:40.564547+0000","last_active":"2023-12-19T18:16:42.357451+0000","last_peered":"2023-12-19T18:16:42.357451+0000","last_clean":"2023-12-19T18:16:42.357451+0000","last_became_active":"2023-12-19T18:16:40.564300+0000","last_became_peered":"2023-12-19T18:16:40.564300+0000","last_unstale":"2023-12-19T18:16:42.357451+0000","last_undegraded":"2023-12-19T18:16:42.357451+0000","last_fullsized":"2023-12-19T18:16:42.357451+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:42.276098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556939+0000","last_change":"2023-12-19T18:16:27.417401+0000","last_active":"2023-12-19T18:16:40.556939+0000","last_peered":"2023-12-19T18:16:40.556939+0000","last_clean":"2023-12-19T18:16:40.556939+0000","last_became_active":"2023-12-19T18:16:27.416774+0000","last_became_peered":"2023-12-19T18:16:27.416774+0000","last_unstale":"2023-12-19T18:16:40.556939+0000","last_undegraded":"2023-12-19T18:16:40.556939+0000","last_fullsized":"2023-12-19T18:16:40.556939+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:03.620596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558146+0000","last_change":"2023-12-19T18:16:27.824516+0000","last_active":"2023-12-19T18:16:40.558146+0000","last_peered":"2023-12-19T18:16:40.558146+0000","last_clean":"2023-12-19T18:16:40.558146+0000","last_became_active":"2023-12-19T18:16:27.823143+0000","last_became_peered":"2023-12-19T18:16:27.823143+0000","last_unstale":"2023-12-19T18:16:40.558146+0000","last_undegraded":"2023-12-19T18:16:40.558146+0000","last_fullsized":"2023-12-19T18:16:40.558146+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:15:52.341930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"215'8","reported_seq":19,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357609+0000","last_change":"2023-12-19T18:16:40.161467+0000","last_active":"2023-12-19T18:16:42.357609+0000","last_peered":"2023-12-19T18:16:42.357609+0000","last_clean":"2023-12-19T18:16:42.357609+0000","last_became_active":"2023-12-19T18:16:40.161143+0000","last_became_peered":"2023-12-19T18:16:40.161143+0000","last_unstale":"2023-12-19T18:16:42.357609+0000","last_undegraded":"2023-12-19T18:16:42.357609+0000","last_fullsized":"2023-12-19T18:16:42.357609+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:21:09.532919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357722+0000","last_change":"2023-12-19T18:16:40.571041+0000","last_active":"2023-12-19T18:16:42.357722+0000","last_peered":"2023-12-19T18:16:42.357722+0000","last_clean":"2023-12-19T18:16:42.357722+0000","last_became_active":"2023-12-19T18:16:40.570705+0000","last_became_peered":"2023-12-19T18:16:40.570705+0000","last_unstale":"2023-12-19T18:16:42.357722+0000","last_undegraded":"2023-12-19T18:16:42.357722+0000","last_fullsized":"2023-12-19T18:16:42.357722+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:43.042802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.13","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560853+0000","last_change":"2023-12-19T18:16:27.415000+0000","last_active":"2023-12-19T18:16:40.560853+0000","last_peered":"2023-12-19T18:16:40.560853+0000","last_clean":"2023-12-19T18:16:40.560853+0000","last_became_active":"2023-12-19T18:16:27.414679+0000","last_became_peered":"2023-12-19T18:16:27.414679+0000","last_unstale":"2023-12-19T18:16:40.560853+0000","last_undegraded":"2023-12-19T18:16:40.560853+0000","last_fullsized":"2023-12-19T18:16:40.560853+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:50.187832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554789+0000","last_change":"2023-12-19T18:16:27.824833+0000","last_active":"2023-12-19T18:16:40.554789+0000","last_peered":"2023-12-19T18:16:40.554789+0000","last_clean":"2023-12-19T18:16:40.554789+0000","last_became_active":"2023-12-19T18:16:27.824487+0000","last_became_peered":"2023-12-19T18:16:27.824487+0000","last_unstale":"2023-12-19T18:16:40.554789+0000","last_undegraded":"2023-12-19T18:16:40.554789+0000","last_fullsized":"2023-12-19T18:16:40.554789+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:34.024953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550312+0000","last_change":"2023-12-19T18:16:40.150872+0000","last_active":"2023-12-19T18:16:41.550312+0000","last_peered":"2023-12-19T18:16:41.550312+0000","last_clean":"2023-12-19T18:16:41.550312+0000","last_became_active":"2023-12-19T18:16:40.150586+0000","last_became_peered":"2023-12-19T18:16:40.150586+0000","last_unstale":"2023-12-19T18:16:41.550312+0000","last_undegraded":"2023-12-19T18:16:41.550312+0000","last_fullsized":"2023-12-19T18:16:41.550312+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:55.343366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358620+0000","last_change":"2023-12-19T18:16:40.571454+0000","last_active":"2023-12-19T18:16:42.358620+0000","last_peered":"2023-12-19T18:16:42.358620+0000","last_clean":"2023-12-19T18:16:42.358620+0000","last_became_active":"2023-12-19T18:16:40.570299+0000","last_became_peered":"2023-12-19T18:16:40.570299+0000","last_unstale":"2023-12-19T18:16:42.358620+0000","last_undegraded":"2023-12-19T18:16:42.358620+0000","last_fullsized":"2023-12-19T18:16:42.358620+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:31:09.517418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"212'8","reported_seq":23,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553799+0000","last_change":"2023-12-19T18:16:27.414573+0000","last_active":"2023-12-19T18:16:40.553799+0000","last_peered":"2023-12-19T18:16:40.553799+0000","last_clean":"2023-12-19T18:16:40.553799+0000","last_became_active":"2023-12-19T18:16:27.414268+0000","last_became_peered":"2023-12-19T18:16:27.414268+0000","last_unstale":"2023-12-19T18:16:40.553799+0000","last_undegraded":"2023-12-19T18:16:40.553799+0000","last_fullsized":"2023-12-19T18:16:40.553799+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:11.117214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555559+0000","last_change":"2023-12-19T18:16:27.829311+0000","last_active":"2023-12-19T18:16:40.555559+0000","last_peered":"2023-12-19T18:16:40.555559+0000","last_clean":"2023-12-19T18:16:40.555559+0000","last_became_active":"2023-12-19T18:16:27.828953+0000","last_became_peered":"2023-12-19T18:16:27.828953+0000","last_unstale":"2023-12-19T18:16:40.555559+0000","last_undegraded":"2023-12-19T18:16:40.555559+0000","last_fullsized":"2023-12-19T18:16:40.555559+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:23.417710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554767+0000","last_change":"2023-12-19T18:16:40.160698+0000","last_active":"2023-12-19T18:16:41.554767+0000","last_peered":"2023-12-19T18:16:41.554767+0000","last_clean":"2023-12-19T18:16:41.554767+0000","last_became_active":"2023-12-19T18:16:40.160406+0000","last_became_peered":"2023-12-19T18:16:40.160406+0000","last_unstale":"2023-12-19T18:16:41.554767+0000","last_undegraded":"2023-12-19T18:16:41.554767+0000","last_fullsized":"2023-12-19T18:16:41.554767+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:10.441625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.075170+0000","last_change":"2023-12-19T18:16:40.579076+0000","last_active":"2023-12-19T18:16:42.075170+0000","last_peered":"2023-12-19T18:16:42.075170+0000","last_clean":"2023-12-19T18:16:42.075170+0000","last_became_active":"2023-12-19T18:16:40.578641+0000","last_became_peered":"2023-12-19T18:16:40.578641+0000","last_unstale":"2023-12-19T18:16:42.075170+0000","last_undegraded":"2023-12-19T18:16:42.075170+0000","last_fullsized":"2023-12-19T18:16:42.075170+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:33:08.245627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.15","version":"211'3","reported_seq":18,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555258+0000","last_change":"2023-12-19T18:16:27.409442+0000","last_active":"2023-12-19T18:16:40.555258+0000","last_peered":"2023-12-19T18:16:40.555258+0000","last_clean":"2023-12-19T18:16:40.555258+0000","last_became_active":"2023-12-19T18:16:27.409094+0000","last_became_peered":"2023-12-19T18:16:27.409094+0000","last_unstale":"2023-12-19T18:16:40.555258+0000","last_undegraded":"2023-12-19T18:16:40.555258+0000","last_fullsized":"2023-12-19T18:16:40.555258+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:18.713070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562498+0000","last_change":"2023-12-19T18:16:27.811752+0000","last_active":"2023-12-19T18:16:40.562498+0000","last_peered":"2023-12-19T18:16:40.562498+0000","last_clean":"2023-12-19T18:16:40.562498+0000","last_became_active":"2023-12-19T18:16:27.811273+0000","last_became_peered":"2023-12-19T18:16:27.811273+0000","last_unstale":"2023-12-19T18:16:40.562498+0000","last_undegraded":"2023-12-19T18:16:40.562498+0000","last_fullsized":"2023-12-19T18:16:40.562498+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:23.609660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360336+0000","last_change":"2023-12-19T18:16:40.162801+0000","last_active":"2023-12-19T18:16:42.360336+0000","last_peered":"2023-12-19T18:16:42.360336+0000","last_clean":"2023-12-19T18:16:42.360336+0000","last_became_active":"2023-12-19T18:16:40.162367+0000","last_became_peered":"2023-12-19T18:16:40.162367+0000","last_unstale":"2023-12-19T18:16:42.360336+0000","last_undegraded":"2023-12-19T18:16:42.360336+0000","last_fullsized":"2023-12-19T18:16:42.360336+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:08:09.383590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359637+0000","last_change":"2023-12-19T18:16:40.570887+0000","last_active":"2023-12-19T18:16:42.359637+0000","last_peered":"2023-12-19T18:16:42.359637+0000","last_clean":"2023-12-19T18:16:42.359637+0000","last_became_active":"2023-12-19T18:16:40.570475+0000","last_became_peered":"2023-12-19T18:16:40.570475+0000","last_unstale":"2023-12-19T18:16:42.359637+0000","last_undegraded":"2023-12-19T18:16:42.359637+0000","last_fullsized":"2023-12-19T18:16:42.359637+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:53:00.119855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549599+0000","last_change":"2023-12-19T18:16:27.410603+0000","last_active":"2023-12-19T18:16:40.549599+0000","last_peered":"2023-12-19T18:16:40.549599+0000","last_clean":"2023-12-19T18:16:40.549599+0000","last_became_active":"2023-12-19T18:16:27.409260+0000","last_became_peered":"2023-12-19T18:16:27.409260+0000","last_unstale":"2023-12-19T18:16:40.549599+0000","last_undegraded":"2023-12-19T18:16:40.549599+0000","last_fullsized":"2023-12-19T18:16:40.549599+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:06.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543878+0000","last_change":"2023-12-19T18:16:27.835564+0000","last_active":"2023-12-19T18:16:40.543878+0000","last_peered":"2023-12-19T18:16:40.543878+0000","last_clean":"2023-12-19T18:16:40.543878+0000","last_became_active":"2023-12-19T18:16:27.835319+0000","last_became_peered":"2023-12-19T18:16:27.835319+0000","last_unstale":"2023-12-19T18:16:40.543878+0000","last_undegraded":"2023-12-19T18:16:40.543878+0000","last_fullsized":"2023-12-19T18:16:40.543878+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:10:35.037185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"215'6","reported_seq":17,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074989+0000","last_change":"2023-12-19T18:16:40.163664+0000","last_active":"2023-12-19T18:16:42.074989+0000","last_peered":"2023-12-19T18:16:42.074989+0000","last_clean":"2023-12-19T18:16:42.074989+0000","last_became_active":"2023-12-19T18:16:40.163301+0000","last_became_peered":"2023-12-19T18:16:40.163301+0000","last_unstale":"2023-12-19T18:16:42.074989+0000","last_undegraded":"2023-12-19T18:16:42.074989+0000","last_fullsized":"2023-12-19T18:16:42.074989+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:33.099709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074835+0000","last_change":"2023-12-19T18:16:40.581101+0000","last_active":"2023-12-19T18:16:42.074835+0000","last_peered":"2023-12-19T18:16:42.074835+0000","last_clean":"2023-12-19T18:16:42.074835+0000","last_became_active":"2023-12-19T18:16:40.579272+0000","last_became_peered":"2023-12-19T18:16:40.579272+0000","last_unstale":"2023-12-19T18:16:42.074835+0000","last_undegraded":"2023-12-19T18:16:42.074835+0000","last_fullsized":"2023-12-19T18:16:42.074835+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:12:36.065977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565329+0000","last_change":"2023-12-19T18:16:27.405556+0000","last_active":"2023-12-19T18:16:40.565329+0000","last_peered":"2023-12-19T18:16:40.565329+0000","last_clean":"2023-12-19T18:16:40.565329+0000","last_became_active":"2023-12-19T18:16:27.404804+0000","last_became_peered":"2023-12-19T18:16:27.404804+0000","last_unstale":"2023-12-19T18:16:40.565329+0000","last_undegraded":"2023-12-19T18:16:40.565329+0000","last_fullsized":"2023-12-19T18:16:40.565329+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:05.395130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551274+0000","last_change":"2023-12-19T18:16:27.830538+0000","last_active":"2023-12-19T18:16:40.551274+0000","last_peered":"2023-12-19T18:16:40.551274+0000","last_clean":"2023-12-19T18:16:40.551274+0000","last_became_active":"2023-12-19T18:16:27.828974+0000","last_became_peered":"2023-12-19T18:16:27.828974+0000","last_unstale":"2023-12-19T18:16:40.551274+0000","last_undegraded":"2023-12-19T18:16:40.551274+0000","last_fullsized":"2023-12-19T18:16:40.551274+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:36.961741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.15","version":"215'3","reported_seq":14,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551392+0000","last_change":"2023-12-19T18:16:40.151095+0000","last_active":"2023-12-19T18:16:41.551392+0000","last_peered":"2023-12-19T18:16:41.551392+0000","last_clean":"2023-12-19T18:16:41.551392+0000","last_became_active":"2023-12-19T18:16:40.150609+0000","last_became_peered":"2023-12-19T18:16:40.150609+0000","last_unstale":"2023-12-19T18:16:41.551392+0000","last_undegraded":"2023-12-19T18:16:41.551392+0000","last_fullsized":"2023-12-19T18:16:41.551392+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:15.058778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074922+0000","last_change":"2023-12-19T18:16:40.580029+0000","last_active":"2023-12-19T18:16:42.074922+0000","last_peered":"2023-12-19T18:16:42.074922+0000","last_clean":"2023-12-19T18:16:42.074922+0000","last_became_active":"2023-12-19T18:16:40.579698+0000","last_became_peered":"2023-12-19T18:16:40.579698+0000","last_unstale":"2023-12-19T18:16:42.074922+0000","last_undegraded":"2023-12-19T18:16:42.074922+0000","last_fullsized":"2023-12-19T18:16:42.074922+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:26:54.889491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.560368+0000","last_change":"2023-12-19T18:16:27.413444+0000","last_active":"2023-12-19T18:16:40.560368+0000","last_peered":"2023-12-19T18:16:40.560368+0000","last_clean":"2023-12-19T18:16:40.560368+0000","last_became_active":"2023-12-19T18:16:27.413066+0000","last_became_peered":"2023-12-19T18:16:27.413066+0000","last_unstale":"2023-12-19T18:16:40.560368+0000","last_undegraded":"2023-12-19T18:16:40.560368+0000","last_fullsized":"2023-12-19T18:16:40.560368+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:57.997657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558113+0000","last_change":"2023-12-19T18:16:27.816735+0000","last_active":"2023-12-19T18:16:40.558113+0000","last_peered":"2023-12-19T18:16:40.558113+0000","last_clean":"2023-12-19T18:16:40.558113+0000","last_became_active":"2023-12-19T18:16:27.816436+0000","last_became_peered":"2023-12-19T18:16:27.816436+0000","last_unstale":"2023-12-19T18:16:40.558113+0000","last_undegraded":"2023-12-19T18:16:40.558113+0000","last_fullsized":"2023-12-19T18:16:40.558113+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:56:37.742104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1a","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359740+0000","last_change":"2023-12-19T18:16:40.161453+0000","last_active":"2023-12-19T18:16:42.359740+0000","last_peered":"2023-12-19T18:16:42.359740+0000","last_clean":"2023-12-19T18:16:42.359740+0000","last_became_active":"2023-12-19T18:16:40.161191+0000","last_became_peered":"2023-12-19T18:16:40.161191+0000","last_unstale":"2023-12-19T18:16:42.359740+0000","last_undegraded":"2023-12-19T18:16:42.359740+0000","last_fullsized":"2023-12-19T18:16:42.359740+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:13.660363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074624+0000","last_change":"2023-12-19T18:16:40.578142+0000","last_active":"2023-12-19T18:16:42.074624+0000","last_peered":"2023-12-19T18:16:42.074624+0000","last_clean":"2023-12-19T18:16:42.074624+0000","last_became_active":"2023-12-19T18:16:40.577655+0000","last_became_peered":"2023-12-19T18:16:40.577655+0000","last_unstale":"2023-12-19T18:16:42.074624+0000","last_undegraded":"2023-12-19T18:16:42.074624+0000","last_fullsized":"2023-12-19T18:16:42.074624+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:41.805677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544353+0000","last_change":"2023-12-19T18:16:27.414372+0000","last_active":"2023-12-19T18:16:40.544353+0000","last_peered":"2023-12-19T18:16:40.544353+0000","last_clean":"2023-12-19T18:16:40.544353+0000","last_became_active":"2023-12-19T18:16:27.414088+0000","last_became_peered":"2023-12-19T18:16:27.414088+0000","last_unstale":"2023-12-19T18:16:40.544353+0000","last_undegraded":"2023-12-19T18:16:40.544353+0000","last_fullsized":"2023-12-19T18:16:40.544353+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:18.945766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543503+0000","last_change":"2023-12-19T18:16:27.827386+0000","last_active":"2023-12-19T18:16:40.543503+0000","last_peered":"2023-12-19T18:16:40.543503+0000","last_clean":"2023-12-19T18:16:40.543503+0000","last_became_active":"2023-12-19T18:16:27.827086+0000","last_became_peered":"2023-12-19T18:16:27.827086+0000","last_unstale":"2023-12-19T18:16:40.543503+0000","last_undegraded":"2023-12-19T18:16:40.543503+0000","last_fullsized":"2023-12-19T18:16:40.543503+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:37.719512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554117+0000","last_change":"2023-12-19T18:16:40.156738+0000","last_active":"2023-12-19T18:16:41.554117+0000","last_peered":"2023-12-19T18:16:41.554117+0000","last_clean":"2023-12-19T18:16:41.554117+0000","last_became_active":"2023-12-19T18:16:40.156488+0000","last_became_peered":"2023-12-19T18:16:40.156488+0000","last_unstale":"2023-12-19T18:16:41.554117+0000","last_undegraded":"2023-12-19T18:16:41.554117+0000","last_fullsized":"2023-12-19T18:16:41.554117+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:28:48.695934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358850+0000","last_change":"2023-12-19T18:16:40.575709+0000","last_active":"2023-12-19T18:16:42.358850+0000","last_peered":"2023-12-19T18:16:42.358850+0000","last_clean":"2023-12-19T18:16:42.358850+0000","last_became_active":"2023-12-19T18:16:40.575453+0000","last_became_peered":"2023-12-19T18:16:40.575453+0000","last_unstale":"2023-12-19T18:16:42.358850+0000","last_undegraded":"2023-12-19T18:16:42.358850+0000","last_fullsized":"2023-12-19T18:16:42.358850+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:44.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1b","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551641+0000","last_change":"2023-12-19T18:16:27.415036+0000","last_active":"2023-12-19T18:16:40.551641+0000","last_peered":"2023-12-19T18:16:40.551641+0000","last_clean":"2023-12-19T18:16:40.551641+0000","last_became_active":"2023-12-19T18:16:27.414747+0000","last_became_peered":"2023-12-19T18:16:27.414747+0000","last_unstale":"2023-12-19T18:16:40.551641+0000","last_undegraded":"2023-12-19T18:16:40.551641+0000","last_fullsized":"2023-12-19T18:16:40.551641+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:03.064138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.556794+0000","last_change":"2023-12-19T18:16:27.828398+0000","last_active":"2023-12-19T18:16:40.556794+0000","last_peered":"2023-12-19T18:16:40.556794+0000","last_clean":"2023-12-19T18:16:40.556794+0000","last_became_active":"2023-12-19T18:16:27.828082+0000","last_became_peered":"2023-12-19T18:16:27.828082+0000","last_unstale":"2023-12-19T18:16:40.556794+0000","last_undegraded":"2023-12-19T18:16:40.556794+0000","last_fullsized":"2023-12-19T18:16:40.556794+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:42:30.998150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358258+0000","last_change":"2023-12-19T18:16:40.159943+0000","last_active":"2023-12-19T18:16:42.358258+0000","last_peered":"2023-12-19T18:16:42.358258+0000","last_clean":"2023-12-19T18:16:42.358258+0000","last_became_active":"2023-12-19T18:16:40.159652+0000","last_became_peered":"2023-12-19T18:16:40.159652+0000","last_unstale":"2023-12-19T18:16:42.358258+0000","last_undegraded":"2023-12-19T18:16:42.358258+0000","last_fullsized":"2023-12-19T18:16:42.358258+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:02:40.928028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.555138+0000","last_change":"2023-12-19T18:16:40.573382+0000","last_active":"2023-12-19T18:16:41.555138+0000","last_peered":"2023-12-19T18:16:41.555138+0000","last_clean":"2023-12-19T18:16:41.555138+0000","last_became_active":"2023-12-19T18:16:40.573092+0000","last_became_peered":"2023-12-19T18:16:40.573092+0000","last_unstale":"2023-12-19T18:16:41.555138+0000","last_undegraded":"2023-12-19T18:16:41.555138+0000","last_fullsized":"2023-12-19T18:16:41.555138+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:10:47.778609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"215'1","reported_seq":12,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553461+0000","last_change":"2023-12-19T18:16:40.163131+0000","last_active":"2023-12-19T18:16:41.553461+0000","last_peered":"2023-12-19T18:16:41.553461+0000","last_clean":"2023-12-19T18:16:41.553461+0000","last_became_active":"2023-12-19T18:16:40.162621+0000","last_became_peered":"2023-12-19T18:16:40.162621+0000","last_unstale":"2023-12-19T18:16:41.553461+0000","last_undegraded":"2023-12-19T18:16:41.553461+0000","last_fullsized":"2023-12-19T18:16:41.553461+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:57.979757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559871+0000","last_change":"2023-12-19T18:16:27.409441+0000","last_active":"2023-12-19T18:16:40.559871+0000","last_peered":"2023-12-19T18:16:40.559871+0000","last_clean":"2023-12-19T18:16:40.559871+0000","last_became_active":"2023-12-19T18:16:27.408438+0000","last_became_peered":"2023-12-19T18:16:27.408438+0000","last_unstale":"2023-12-19T18:16:40.559871+0000","last_undegraded":"2023-12-19T18:16:40.559871+0000","last_fullsized":"2023-12-19T18:16:40.559871+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:40.980300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.546499+0000","last_change":"2023-12-19T18:16:27.814063+0000","last_active":"2023-12-19T18:16:40.546499+0000","last_peered":"2023-12-19T18:16:40.546499+0000","last_clean":"2023-12-19T18:16:40.546499+0000","last_became_active":"2023-12-19T18:16:27.813564+0000","last_became_peered":"2023-12-19T18:16:27.813564+0000","last_unstale":"2023-12-19T18:16:40.546499+0000","last_undegraded":"2023-12-19T18:16:40.546499+0000","last_fullsized":"2023-12-19T18:16:40.546499+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:28:36.733148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553348+0000","last_change":"2023-12-19T18:16:40.564402+0000","last_active":"2023-12-19T18:16:41.553348+0000","last_peered":"2023-12-19T18:16:41.553348+0000","last_clean":"2023-12-19T18:16:41.553348+0000","last_became_active":"2023-12-19T18:16:40.561338+0000","last_became_peered":"2023-12-19T18:16:40.561338+0000","last_unstale":"2023-12-19T18:16:41.553348+0000","last_undegraded":"2023-12-19T18:16:41.553348+0000","last_fullsized":"2023-12-19T18:16:41.553348+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:37:11.741863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"215'4","reported_seq":15,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857590+0000","last_change":"2023-12-19T18:16:40.152799+0000","last_active":"2023-12-19T18:16:41.857590+0000","last_peered":"2023-12-19T18:16:41.857590+0000","last_clean":"2023-12-19T18:16:41.857590+0000","last_became_active":"2023-12-19T18:16:40.152482+0000","last_became_peered":"2023-12-19T18:16:40.152482+0000","last_unstale":"2023-12-19T18:16:41.857590+0000","last_undegraded":"2023-12-19T18:16:41.857590+0000","last_fullsized":"2023-12-19T18:16:41.857590+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:23.854189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1d","version":"211'2","reported_seq":17,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548666+0000","last_change":"2023-12-19T18:16:27.415013+0000","last_active":"2023-12-19T18:16:40.548666+0000","last_peered":"2023-12-19T18:16:40.548666+0000","last_clean":"2023-12-19T18:16:40.548666+0000","last_became_active":"2023-12-19T18:16:27.414582+0000","last_became_peered":"2023-12-19T18:16:27.414582+0000","last_unstale":"2023-12-19T18:16:40.548666+0000","last_undegraded":"2023-12-19T18:16:40.548666+0000","last_fullsized":"2023-12-19T18:16:40.548666+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:34:01.457288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562417+0000","last_change":"2023-12-19T18:16:27.829031+0000","last_active":"2023-12-19T18:16:40.562417+0000","last_peered":"2023-12-19T18:16:40.562417+0000","last_clean":"2023-12-19T18:16:40.562417+0000","last_became_active":"2023-12-19T18:16:27.826634+0000","last_became_peered":"2023-12-19T18:16:27.826634+0000","last_unstale":"2023-12-19T18:16:40.562417+0000","last_undegraded":"2023-12-19T18:16:40.562417+0000","last_fullsized":"2023-12-19T18:16:40.562417+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:23.429228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551066+0000","last_change":"2023-12-19T18:16:40.570487+0000","last_active":"2023-12-19T18:16:41.551066+0000","last_peered":"2023-12-19T18:16:41.551066+0000","last_clean":"2023-12-19T18:16:41.551066+0000","last_became_active":"2023-12-19T18:16:40.569839+0000","last_became_peered":"2023-12-19T18:16:40.569839+0000","last_unstale":"2023-12-19T18:16:41.551066+0000","last_undegraded":"2023-12-19T18:16:41.551066+0000","last_fullsized":"2023-12-19T18:16:41.551066+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:59.414695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"211'1","reported_seq":16,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548366+0000","last_change":"2023-12-19T18:16:27.407669+0000","last_active":"2023-12-19T18:16:40.548366+0000","last_peered":"2023-12-19T18:16:40.548366+0000","last_clean":"2023-12-19T18:16:40.548366+0000","last_became_active":"2023-12-19T18:16:27.407150+0000","last_became_peered":"2023-12-19T18:16:27.407150+0000","last_unstale":"2023-12-19T18:16:40.548366+0000","last_undegraded":"2023-12-19T18:16:40.548366+0000","last_fullsized":"2023-12-19T18:16:40.548366+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:59.675090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558634+0000","last_change":"2023-12-19T18:16:27.830766+0000","last_active":"2023-12-19T18:16:40.558634+0000","last_peered":"2023-12-19T18:16:40.558634+0000","last_clean":"2023-12-19T18:16:40.558634+0000","last_became_active":"2023-12-19T18:16:27.830220+0000","last_became_peered":"2023-12-19T18:16:27.830220+0000","last_unstale":"2023-12-19T18:16:40.558634+0000","last_undegraded":"2023-12-19T18:16:40.558634+0000","last_fullsized":"2023-12-19T18:16:40.558634+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:03:17.772391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1c","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.075108+0000","last_change":"2023-12-19T18:16:40.164718+0000","last_active":"2023-12-19T18:16:42.075108+0000","last_peered":"2023-12-19T18:16:42.075108+0000","last_clean":"2023-12-19T18:16:42.075108+0000","last_became_active":"2023-12-19T18:16:40.164413+0000","last_became_peered":"2023-12-19T18:16:40.164413+0000","last_unstale":"2023-12-19T18:16:42.075108+0000","last_undegraded":"2023-12-19T18:16:42.075108+0000","last_fullsized":"2023-12-19T18:16:42.075108+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:49.453705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359497+0000","last_change":"2023-12-19T18:16:40.565564+0000","last_active":"2023-12-19T18:16:42.359497+0000","last_peered":"2023-12-19T18:16:42.359497+0000","last_clean":"2023-12-19T18:16:42.359497+0000","last_became_active":"2023-12-19T18:16:40.565300+0000","last_became_peered":"2023-12-19T18:16:40.565300+0000","last_unstale":"2023-12-19T18:16:42.359497+0000","last_undegraded":"2023-12-19T18:16:42.359497+0000","last_fullsized":"2023-12-19T18:16:42.359497+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:59:14.849896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1f","version":"211'4","reported_seq":19,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553798+0000","last_change":"2023-12-19T18:16:27.414364+0000","last_active":"2023-12-19T18:16:40.553798+0000","last_peered":"2023-12-19T18:16:40.553798+0000","last_clean":"2023-12-19T18:16:40.553798+0000","last_became_active":"2023-12-19T18:16:27.413648+0000","last_became_peered":"2023-12-19T18:16:27.413648+0000","last_unstale":"2023-12-19T18:16:40.553798+0000","last_undegraded":"2023-12-19T18:16:40.553798+0000","last_fullsized":"2023-12-19T18:16:40.553798+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:26.380474+0000","last_clean_scrub_stamp":"2023-12-19T18:16:26.380474+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:27:32.703908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.544529+0000","last_change":"2023-12-19T18:16:27.830378+0000","last_active":"2023-12-19T18:16:40.544529+0000","last_peered":"2023-12-19T18:16:40.544529+0000","last_clean":"2023-12-19T18:16:40.544529+0000","last_became_active":"2023-12-19T18:16:27.830122+0000","last_became_peered":"2023-12-19T18:16:27.830122+0000","last_unstale":"2023-12-19T18:16:40.544529+0000","last_undegraded":"2023-12-19T18:16:40.544529+0000","last_fullsized":"2023-12-19T18:16:40.544529+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:21.365516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1d","version":"215'2","reported_seq":13,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358451+0000","last_change":"2023-12-19T18:16:40.162566+0000","last_active":"2023-12-19T18:16:42.358451+0000","last_peered":"2023-12-19T18:16:42.358451+0000","last_clean":"2023-12-19T18:16:42.358451+0000","last_became_active":"2023-12-19T18:16:40.162255+0000","last_became_peered":"2023-12-19T18:16:40.162255+0000","last_unstale":"2023-12-19T18:16:42.358451+0000","last_undegraded":"2023-12-19T18:16:42.358451+0000","last_fullsized":"2023-12-19T18:16:42.358451+0000","mapping_epoch":213,"log_start":"0'0","ondisk_log_start":"0'0","created":213,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:39.117449+0000","last_clean_scrub_stamp":"2023-12-19T18:16:39.117449+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:20.530183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.552668+0000","last_change":"2023-12-19T18:16:40.570029+0000","last_active":"2023-12-19T18:16:41.552668+0000","last_peered":"2023-12-19T18:16:41.552668+0000","last_clean":"2023-12-19T18:16:41.552668+0000","last_became_active":"2023-12-19T18:16:40.569731+0000","last_became_peered":"2023-12-19T18:16:40.569731+0000","last_unstale":"2023-12-19T18:16:41.552668+0000","last_undegraded":"2023-12-19T18:16:41.552668+0000","last_fullsized":"2023-12-19T18:16:41.552668+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:25:46.635350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.550877+0000","last_change":"2023-12-19T18:16:27.831030+0000","last_active":"2023-12-19T18:16:40.550877+0000","last_peered":"2023-12-19T18:16:40.550877+0000","last_clean":"2023-12-19T18:16:40.550877+0000","last_became_active":"2023-12-19T18:16:27.830764+0000","last_became_peered":"2023-12-19T18:16:27.830764+0000","last_unstale":"2023-12-19T18:16:40.550877+0000","last_undegraded":"2023-12-19T18:16:40.550877+0000","last_fullsized":"2023-12-19T18:16:40.550877+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:11.685663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857085+0000","last_change":"2023-12-19T18:16:40.569158+0000","last_active":"2023-12-19T18:16:41.857085+0000","last_peered":"2023-12-19T18:16:41.857085+0000","last_clean":"2023-12-19T18:16:41.857085+0000","last_became_active":"2023-12-19T18:16:40.568894+0000","last_became_peered":"2023-12-19T18:16:40.568894+0000","last_unstale":"2023-12-19T18:16:41.857085+0000","last_undegraded":"2023-12-19T18:16:41.857085+0000","last_fullsized":"2023-12-19T18:16:41.857085+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:02.018340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553603+0000","last_change":"2023-12-19T18:16:27.833518+0000","last_active":"2023-12-19T18:16:40.553603+0000","last_peered":"2023-12-19T18:16:40.553603+0000","last_clean":"2023-12-19T18:16:40.553603+0000","last_became_active":"2023-12-19T18:16:27.833251+0000","last_became_peered":"2023-12-19T18:16:27.833251+0000","last_unstale":"2023-12-19T18:16:40.553603+0000","last_undegraded":"2023-12-19T18:16:40.553603+0000","last_fullsized":"2023-12-19T18:16:40.553603+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:49.276917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357285+0000","last_change":"2023-12-19T18:16:40.563610+0000","last_active":"2023-12-19T18:16:42.357285+0000","last_peered":"2023-12-19T18:16:42.357285+0000","last_clean":"2023-12-19T18:16:42.357285+0000","last_became_active":"2023-12-19T18:16:40.563300+0000","last_became_peered":"2023-12-19T18:16:40.563300+0000","last_unstale":"2023-12-19T18:16:42.357285+0000","last_undegraded":"2023-12-19T18:16:42.357285+0000","last_fullsized":"2023-12-19T18:16:42.357285+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:28:48.518493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.547548+0000","last_change":"2023-12-19T18:16:27.825014+0000","last_active":"2023-12-19T18:16:40.547548+0000","last_peered":"2023-12-19T18:16:40.547548+0000","last_clean":"2023-12-19T18:16:40.547548+0000","last_became_active":"2023-12-19T18:16:27.822140+0000","last_became_peered":"2023-12-19T18:16:27.822140+0000","last_unstale":"2023-12-19T18:16:40.547548+0000","last_undegraded":"2023-12-19T18:16:40.547548+0000","last_fullsized":"2023-12-19T18:16:40.547548+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:36.972809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550313+0000","last_change":"2023-12-19T18:16:40.576594+0000","last_active":"2023-12-19T18:16:41.550313+0000","last_peered":"2023-12-19T18:16:41.550313+0000","last_clean":"2023-12-19T18:16:41.550313+0000","last_became_active":"2023-12-19T18:16:40.576304+0000","last_became_peered":"2023-12-19T18:16:40.576304+0000","last_unstale":"2023-12-19T18:16:41.550313+0000","last_undegraded":"2023-12-19T18:16:41.550313+0000","last_fullsized":"2023-12-19T18:16:41.550313+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:58.373923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561436+0000","last_change":"2023-12-19T18:16:27.826500+0000","last_active":"2023-12-19T18:16:40.561436+0000","last_peered":"2023-12-19T18:16:40.561436+0000","last_clean":"2023-12-19T18:16:40.561436+0000","last_became_active":"2023-12-19T18:16:27.826216+0000","last_became_peered":"2023-12-19T18:16:27.826216+0000","last_unstale":"2023-12-19T18:16:40.561436+0000","last_undegraded":"2023-12-19T18:16:40.561436+0000","last_fullsized":"2023-12-19T18:16:40.561436+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:33.032362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359854+0000","last_change":"2023-12-19T18:16:40.565942+0000","last_active":"2023-12-19T18:16:42.359854+0000","last_peered":"2023-12-19T18:16:42.359854+0000","last_clean":"2023-12-19T18:16:42.359854+0000","last_became_active":"2023-12-19T18:16:40.565557+0000","last_became_peered":"2023-12-19T18:16:40.565557+0000","last_unstale":"2023-12-19T18:16:42.359854+0000","last_undegraded":"2023-12-19T18:16:42.359854+0000","last_fullsized":"2023-12-19T18:16:42.359854+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:23.839152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548841+0000","last_change":"2023-12-19T18:16:27.830103+0000","last_active":"2023-12-19T18:16:40.548841+0000","last_peered":"2023-12-19T18:16:40.548841+0000","last_clean":"2023-12-19T18:16:40.548841+0000","last_became_active":"2023-12-19T18:16:27.829444+0000","last_became_peered":"2023-12-19T18:16:27.829444+0000","last_unstale":"2023-12-19T18:16:40.548841+0000","last_undegraded":"2023-12-19T18:16:40.548841+0000","last_fullsized":"2023-12-19T18:16:40.548841+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:50:29.947932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553462+0000","last_change":"2023-12-19T18:16:40.563980+0000","last_active":"2023-12-19T18:16:41.553462+0000","last_peered":"2023-12-19T18:16:41.553462+0000","last_clean":"2023-12-19T18:16:41.553462+0000","last_became_active":"2023-12-19T18:16:40.563683+0000","last_became_peered":"2023-12-19T18:16:40.563683+0000","last_unstale":"2023-12-19T18:16:41.553462+0000","last_undegraded":"2023-12-19T18:16:41.553462+0000","last_fullsized":"2023-12-19T18:16:41.553462+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:36.170652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548463+0000","last_change":"2023-12-19T18:16:27.820913+0000","last_active":"2023-12-19T18:16:40.548463+0000","last_peered":"2023-12-19T18:16:40.548463+0000","last_clean":"2023-12-19T18:16:40.548463+0000","last_became_active":"2023-12-19T18:16:27.820634+0000","last_became_peered":"2023-12-19T18:16:27.820634+0000","last_unstale":"2023-12-19T18:16:40.548463+0000","last_undegraded":"2023-12-19T18:16:40.548463+0000","last_fullsized":"2023-12-19T18:16:40.548463+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:53:09.066279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357879+0000","last_change":"2023-12-19T18:16:40.562872+0000","last_active":"2023-12-19T18:16:42.357879+0000","last_peered":"2023-12-19T18:16:42.357879+0000","last_clean":"2023-12-19T18:16:42.357879+0000","last_became_active":"2023-12-19T18:16:40.562615+0000","last_became_peered":"2023-12-19T18:16:40.562615+0000","last_unstale":"2023-12-19T18:16:42.357879+0000","last_undegraded":"2023-12-19T18:16:42.357879+0000","last_fullsized":"2023-12-19T18:16:42.357879+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:54.454071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565322+0000","last_change":"2023-12-19T18:16:27.830119+0000","last_active":"2023-12-19T18:16:40.565322+0000","last_peered":"2023-12-19T18:16:40.565322+0000","last_clean":"2023-12-19T18:16:40.565322+0000","last_became_active":"2023-12-19T18:16:27.829850+0000","last_became_peered":"2023-12-19T18:16:27.829850+0000","last_unstale":"2023-12-19T18:16:40.565322+0000","last_undegraded":"2023-12-19T18:16:40.565322+0000","last_fullsized":"2023-12-19T18:16:40.565322+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:51.190615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857238+0000","last_change":"2023-12-19T18:16:40.570092+0000","last_active":"2023-12-19T18:16:41.857238+0000","last_peered":"2023-12-19T18:16:41.857238+0000","last_clean":"2023-12-19T18:16:41.857238+0000","last_became_active":"2023-12-19T18:16:40.568697+0000","last_became_peered":"2023-12-19T18:16:40.568697+0000","last_unstale":"2023-12-19T18:16:41.857238+0000","last_undegraded":"2023-12-19T18:16:41.857238+0000","last_fullsized":"2023-12-19T18:16:41.857238+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:11.448855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.564229+0000","last_change":"2023-12-19T18:16:27.834875+0000","last_active":"2023-12-19T18:16:40.564229+0000","last_peered":"2023-12-19T18:16:40.564229+0000","last_clean":"2023-12-19T18:16:40.564229+0000","last_became_active":"2023-12-19T18:16:27.834561+0000","last_became_peered":"2023-12-19T18:16:27.834561+0000","last_unstale":"2023-12-19T18:16:40.564229+0000","last_undegraded":"2023-12-19T18:16:40.564229+0000","last_fullsized":"2023-12-19T18:16:40.564229+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:30.301379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554348+0000","last_change":"2023-12-19T18:16:40.561527+0000","last_active":"2023-12-19T18:16:41.554348+0000","last_peered":"2023-12-19T18:16:41.554348+0000","last_clean":"2023-12-19T18:16:41.554348+0000","last_became_active":"2023-12-19T18:16:40.561251+0000","last_became_peered":"2023-12-19T18:16:40.561251+0000","last_unstale":"2023-12-19T18:16:41.554348+0000","last_undegraded":"2023-12-19T18:16:41.554348+0000","last_fullsized":"2023-12-19T18:16:41.554348+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:50.460935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543417+0000","last_change":"2023-12-19T18:16:27.826642+0000","last_active":"2023-12-19T18:16:40.543417+0000","last_peered":"2023-12-19T18:16:40.543417+0000","last_clean":"2023-12-19T18:16:40.543417+0000","last_became_active":"2023-12-19T18:16:27.825460+0000","last_became_peered":"2023-12-19T18:16:27.825460+0000","last_unstale":"2023-12-19T18:16:40.543417+0000","last_undegraded":"2023-12-19T18:16:40.543417+0000","last_fullsized":"2023-12-19T18:16:40.543417+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:05.671602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074707+0000","last_change":"2023-12-19T18:16:40.579582+0000","last_active":"2023-12-19T18:16:42.074707+0000","last_peered":"2023-12-19T18:16:42.074707+0000","last_clean":"2023-12-19T18:16:42.074707+0000","last_became_active":"2023-12-19T18:16:40.579319+0000","last_became_peered":"2023-12-19T18:16:40.579319+0000","last_unstale":"2023-12-19T18:16:42.074707+0000","last_undegraded":"2023-12-19T18:16:42.074707+0000","last_fullsized":"2023-12-19T18:16:42.074707+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:53:38.529823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551313+0000","last_change":"2023-12-19T18:16:27.826145+0000","last_active":"2023-12-19T18:16:40.551313+0000","last_peered":"2023-12-19T18:16:40.551313+0000","last_clean":"2023-12-19T18:16:40.551313+0000","last_became_active":"2023-12-19T18:16:27.825889+0000","last_became_peered":"2023-12-19T18:16:27.825889+0000","last_unstale":"2023-12-19T18:16:40.551313+0000","last_undegraded":"2023-12-19T18:16:40.551313+0000","last_fullsized":"2023-12-19T18:16:40.551313+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:59:41.673539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074764+0000","last_change":"2023-12-19T18:16:40.579381+0000","last_active":"2023-12-19T18:16:42.074764+0000","last_peered":"2023-12-19T18:16:42.074764+0000","last_clean":"2023-12-19T18:16:42.074764+0000","last_became_active":"2023-12-19T18:16:40.578655+0000","last_became_peered":"2023-12-19T18:16:40.578655+0000","last_unstale":"2023-12-19T18:16:42.074764+0000","last_undegraded":"2023-12-19T18:16:42.074764+0000","last_fullsized":"2023-12-19T18:16:42.074764+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:33:53.935941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555865+0000","last_change":"2023-12-19T18:16:27.830084+0000","last_active":"2023-12-19T18:16:40.555865+0000","last_peered":"2023-12-19T18:16:40.555865+0000","last_clean":"2023-12-19T18:16:40.555865+0000","last_became_active":"2023-12-19T18:16:27.829787+0000","last_became_peered":"2023-12-19T18:16:27.829787+0000","last_unstale":"2023-12-19T18:16:40.555865+0000","last_undegraded":"2023-12-19T18:16:40.555865+0000","last_fullsized":"2023-12-19T18:16:40.555865+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:34:16.356894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.554193+0000","last_change":"2023-12-19T18:16:40.563996+0000","last_active":"2023-12-19T18:16:41.554193+0000","last_peered":"2023-12-19T18:16:41.554193+0000","last_clean":"2023-12-19T18:16:41.554193+0000","last_became_active":"2023-12-19T18:16:40.563686+0000","last_became_peered":"2023-12-19T18:16:40.563686+0000","last_unstale":"2023-12-19T18:16:41.554193+0000","last_undegraded":"2023-12-19T18:16:41.554193+0000","last_fullsized":"2023-12-19T18:16:41.554193+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:34.106940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551542+0000","last_change":"2023-12-19T18:16:27.831551+0000","last_active":"2023-12-19T18:16:40.551542+0000","last_peered":"2023-12-19T18:16:40.551542+0000","last_clean":"2023-12-19T18:16:40.551542+0000","last_became_active":"2023-12-19T18:16:27.831291+0000","last_became_peered":"2023-12-19T18:16:27.831291+0000","last_unstale":"2023-12-19T18:16:40.551542+0000","last_undegraded":"2023-12-19T18:16:40.551542+0000","last_fullsized":"2023-12-19T18:16:40.551542+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:09.914084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551186+0000","last_change":"2023-12-19T18:16:40.563764+0000","last_active":"2023-12-19T18:16:41.551186+0000","last_peered":"2023-12-19T18:16:41.551186+0000","last_clean":"2023-12-19T18:16:41.551186+0000","last_became_active":"2023-12-19T18:16:40.563464+0000","last_became_peered":"2023-12-19T18:16:40.563464+0000","last_unstale":"2023-12-19T18:16:41.551186+0000","last_undegraded":"2023-12-19T18:16:41.551186+0000","last_fullsized":"2023-12-19T18:16:41.551186+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:52:34.311679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548663+0000","last_change":"2023-12-19T18:16:27.833054+0000","last_active":"2023-12-19T18:16:40.548663+0000","last_peered":"2023-12-19T18:16:40.548663+0000","last_clean":"2023-12-19T18:16:40.548663+0000","last_became_active":"2023-12-19T18:16:27.832738+0000","last_became_peered":"2023-12-19T18:16:27.832738+0000","last_unstale":"2023-12-19T18:16:40.548663+0000","last_undegraded":"2023-12-19T18:16:40.548663+0000","last_fullsized":"2023-12-19T18:16:40.548663+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:57.892685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360068+0000","last_change":"2023-12-19T18:16:40.577515+0000","last_active":"2023-12-19T18:16:42.360068+0000","last_peered":"2023-12-19T18:16:42.360068+0000","last_clean":"2023-12-19T18:16:42.360068+0000","last_became_active":"2023-12-19T18:16:40.577096+0000","last_became_peered":"2023-12-19T18:16:40.577096+0000","last_unstale":"2023-12-19T18:16:42.360068+0000","last_undegraded":"2023-12-19T18:16:42.360068+0000","last_fullsized":"2023-12-19T18:16:42.360068+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:36.711429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.555479+0000","last_change":"2023-12-19T18:16:27.820194+0000","last_active":"2023-12-19T18:16:40.555479+0000","last_peered":"2023-12-19T18:16:40.555479+0000","last_clean":"2023-12-19T18:16:40.555479+0000","last_became_active":"2023-12-19T18:16:27.819900+0000","last_became_peered":"2023-12-19T18:16:27.819900+0000","last_unstale":"2023-12-19T18:16:40.555479+0000","last_undegraded":"2023-12-19T18:16:40.555479+0000","last_fullsized":"2023-12-19T18:16:40.555479+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:00.858729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360137+0000","last_change":"2023-12-19T18:16:40.561878+0000","last_active":"2023-12-19T18:16:42.360137+0000","last_peered":"2023-12-19T18:16:42.360137+0000","last_clean":"2023-12-19T18:16:42.360137+0000","last_became_active":"2023-12-19T18:16:40.561626+0000","last_became_peered":"2023-12-19T18:16:40.561626+0000","last_unstale":"2023-12-19T18:16:42.360137+0000","last_undegraded":"2023-12-19T18:16:42.360137+0000","last_fullsized":"2023-12-19T18:16:42.360137+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:38:57.938991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562034+0000","last_change":"2023-12-19T18:16:27.820781+0000","last_active":"2023-12-19T18:16:40.562034+0000","last_peered":"2023-12-19T18:16:40.562034+0000","last_clean":"2023-12-19T18:16:40.562034+0000","last_became_active":"2023-12-19T18:16:27.820483+0000","last_became_peered":"2023-12-19T18:16:27.820483+0000","last_unstale":"2023-12-19T18:16:40.562034+0000","last_undegraded":"2023-12-19T18:16:40.562034+0000","last_fullsized":"2023-12-19T18:16:40.562034+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:23.445435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551324+0000","last_change":"2023-12-19T18:16:40.560485+0000","last_active":"2023-12-19T18:16:41.551324+0000","last_peered":"2023-12-19T18:16:41.551324+0000","last_clean":"2023-12-19T18:16:41.551324+0000","last_became_active":"2023-12-19T18:16:40.560232+0000","last_became_peered":"2023-12-19T18:16:40.560232+0000","last_unstale":"2023-12-19T18:16:41.551324+0000","last_undegraded":"2023-12-19T18:16:41.551324+0000","last_fullsized":"2023-12-19T18:16:41.551324+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:30.252976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.553708+0000","last_change":"2023-12-19T18:16:27.834880+0000","last_active":"2023-12-19T18:16:40.553708+0000","last_peered":"2023-12-19T18:16:40.553708+0000","last_clean":"2023-12-19T18:16:40.553708+0000","last_became_active":"2023-12-19T18:16:27.834494+0000","last_became_peered":"2023-12-19T18:16:27.834494+0000","last_unstale":"2023-12-19T18:16:40.553708+0000","last_undegraded":"2023-12-19T18:16:40.553708+0000","last_fullsized":"2023-12-19T18:16:40.553708+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:06.992701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357587+0000","last_change":"2023-12-19T18:16:40.567126+0000","last_active":"2023-12-19T18:16:42.357587+0000","last_peered":"2023-12-19T18:16:42.357587+0000","last_clean":"2023-12-19T18:16:42.357587+0000","last_became_active":"2023-12-19T18:16:40.566878+0000","last_became_peered":"2023-12-19T18:16:40.566878+0000","last_unstale":"2023-12-19T18:16:42.357587+0000","last_undegraded":"2023-12-19T18:16:42.357587+0000","last_fullsized":"2023-12-19T18:16:42.357587+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:59.062853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543300+0000","last_change":"2023-12-19T18:16:27.826828+0000","last_active":"2023-12-19T18:16:40.543300+0000","last_peered":"2023-12-19T18:16:40.543300+0000","last_clean":"2023-12-19T18:16:40.543300+0000","last_became_active":"2023-12-19T18:16:27.825994+0000","last_became_peered":"2023-12-19T18:16:27.825994+0000","last_unstale":"2023-12-19T18:16:40.543300+0000","last_undegraded":"2023-12-19T18:16:40.543300+0000","last_fullsized":"2023-12-19T18:16:40.543300+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:23:17.748218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857358+0000","last_change":"2023-12-19T18:16:40.569008+0000","last_active":"2023-12-19T18:16:41.857358+0000","last_peered":"2023-12-19T18:16:41.857358+0000","last_clean":"2023-12-19T18:16:41.857358+0000","last_became_active":"2023-12-19T18:16:40.568420+0000","last_became_peered":"2023-12-19T18:16:40.568420+0000","last_unstale":"2023-12-19T18:16:41.857358+0000","last_undegraded":"2023-12-19T18:16:41.857358+0000","last_fullsized":"2023-12-19T18:16:41.857358+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:31:10.720873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561285+0000","last_change":"2023-12-19T18:16:27.830751+0000","last_active":"2023-12-19T18:16:40.561285+0000","last_peered":"2023-12-19T18:16:40.561285+0000","last_clean":"2023-12-19T18:16:40.561285+0000","last_became_active":"2023-12-19T18:16:27.830490+0000","last_became_peered":"2023-12-19T18:16:27.830490+0000","last_unstale":"2023-12-19T18:16:40.561285+0000","last_undegraded":"2023-12-19T18:16:40.561285+0000","last_fullsized":"2023-12-19T18:16:40.561285+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:31.093343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.550245+0000","last_change":"2023-12-19T18:16:40.571696+0000","last_active":"2023-12-19T18:16:41.550245+0000","last_peered":"2023-12-19T18:16:41.550245+0000","last_clean":"2023-12-19T18:16:41.550245+0000","last_became_active":"2023-12-19T18:16:40.571407+0000","last_became_peered":"2023-12-19T18:16:40.571407+0000","last_unstale":"2023-12-19T18:16:41.550245+0000","last_undegraded":"2023-12-19T18:16:41.550245+0000","last_fullsized":"2023-12-19T18:16:41.550245+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:55.222921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548271+0000","last_change":"2023-12-19T18:16:27.823186+0000","last_active":"2023-12-19T18:16:40.548271+0000","last_peered":"2023-12-19T18:16:40.548271+0000","last_clean":"2023-12-19T18:16:40.548271+0000","last_became_active":"2023-12-19T18:16:27.820405+0000","last_became_peered":"2023-12-19T18:16:27.820405+0000","last_unstale":"2023-12-19T18:16:40.548271+0000","last_undegraded":"2023-12-19T18:16:40.548271+0000","last_fullsized":"2023-12-19T18:16:40.548271+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:41.515700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357953+0000","last_change":"2023-12-19T18:16:40.569297+0000","last_active":"2023-12-19T18:16:42.357953+0000","last_peered":"2023-12-19T18:16:42.357953+0000","last_clean":"2023-12-19T18:16:42.357953+0000","last_became_active":"2023-12-19T18:16:40.569050+0000","last_became_peered":"2023-12-19T18:16:40.569050+0000","last_unstale":"2023-12-19T18:16:42.357953+0000","last_undegraded":"2023-12-19T18:16:42.357953+0000","last_fullsized":"2023-12-19T18:16:42.357953+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:28:56.831758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.561350+0000","last_change":"2023-12-19T18:16:27.829736+0000","last_active":"2023-12-19T18:16:40.561350+0000","last_peered":"2023-12-19T18:16:40.561350+0000","last_clean":"2023-12-19T18:16:40.561350+0000","last_became_active":"2023-12-19T18:16:27.829448+0000","last_became_peered":"2023-12-19T18:16:27.829448+0000","last_unstale":"2023-12-19T18:16:40.561350+0000","last_undegraded":"2023-12-19T18:16:40.561350+0000","last_fullsized":"2023-12-19T18:16:40.561350+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:47.628881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.551154+0000","last_change":"2023-12-19T18:16:40.559877+0000","last_active":"2023-12-19T18:16:41.551154+0000","last_peered":"2023-12-19T18:16:41.551154+0000","last_clean":"2023-12-19T18:16:41.551154+0000","last_became_active":"2023-12-19T18:16:40.558100+0000","last_became_peered":"2023-12-19T18:16:40.558100+0000","last_unstale":"2023-12-19T18:16:41.551154+0000","last_undegraded":"2023-12-19T18:16:41.551154+0000","last_fullsized":"2023-12-19T18:16:41.551154+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:44.649904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559776+0000","last_change":"2023-12-19T18:16:27.829943+0000","last_active":"2023-12-19T18:16:40.559776+0000","last_peered":"2023-12-19T18:16:40.559776+0000","last_clean":"2023-12-19T18:16:40.559776+0000","last_became_active":"2023-12-19T18:16:27.829592+0000","last_became_peered":"2023-12-19T18:16:27.829592+0000","last_unstale":"2023-12-19T18:16:40.559776+0000","last_undegraded":"2023-12-19T18:16:40.559776+0000","last_fullsized":"2023-12-19T18:16:40.559776+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:46.245564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553575+0000","last_change":"2023-12-19T18:16:40.575384+0000","last_active":"2023-12-19T18:16:41.553575+0000","last_peered":"2023-12-19T18:16:41.553575+0000","last_clean":"2023-12-19T18:16:41.553575+0000","last_became_active":"2023-12-19T18:16:40.575124+0000","last_became_peered":"2023-12-19T18:16:40.575124+0000","last_unstale":"2023-12-19T18:16:41.553575+0000","last_undegraded":"2023-12-19T18:16:41.553575+0000","last_fullsized":"2023-12-19T18:16:41.553575+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:53.796369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.565426+0000","last_change":"2023-12-19T18:16:27.819797+0000","last_active":"2023-12-19T18:16:40.565426+0000","last_peered":"2023-12-19T18:16:40.565426+0000","last_clean":"2023-12-19T18:16:40.565426+0000","last_became_active":"2023-12-19T18:16:27.819507+0000","last_became_peered":"2023-12-19T18:16:27.819507+0000","last_unstale":"2023-12-19T18:16:40.565426+0000","last_undegraded":"2023-12-19T18:16:40.565426+0000","last_fullsized":"2023-12-19T18:16:40.565426+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:52:08.287716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.857264+0000","last_change":"2023-12-19T18:16:40.570173+0000","last_active":"2023-12-19T18:16:41.857264+0000","last_peered":"2023-12-19T18:16:41.857264+0000","last_clean":"2023-12-19T18:16:41.857264+0000","last_became_active":"2023-12-19T18:16:40.569912+0000","last_became_peered":"2023-12-19T18:16:40.569912+0000","last_unstale":"2023-12-19T18:16:41.857264+0000","last_undegraded":"2023-12-19T18:16:41.857264+0000","last_fullsized":"2023-12-19T18:16:41.857264+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:28:47.688949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558633+0000","last_change":"2023-12-19T18:16:27.819945+0000","last_active":"2023-12-19T18:16:40.558633+0000","last_peered":"2023-12-19T18:16:40.558633+0000","last_clean":"2023-12-19T18:16:40.558633+0000","last_became_active":"2023-12-19T18:16:27.819634+0000","last_became_peered":"2023-12-19T18:16:27.819634+0000","last_unstale":"2023-12-19T18:16:40.558633+0000","last_undegraded":"2023-12-19T18:16:40.558633+0000","last_fullsized":"2023-12-19T18:16:40.558633+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:22.401852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358420+0000","last_change":"2023-12-19T18:16:40.566307+0000","last_active":"2023-12-19T18:16:42.358420+0000","last_peered":"2023-12-19T18:16:42.358420+0000","last_clean":"2023-12-19T18:16:42.358420+0000","last_became_active":"2023-12-19T18:16:40.566059+0000","last_became_peered":"2023-12-19T18:16:40.566059+0000","last_unstale":"2023-12-19T18:16:42.358420+0000","last_undegraded":"2023-12-19T18:16:42.358420+0000","last_fullsized":"2023-12-19T18:16:42.358420+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:45.773786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.549949+0000","last_change":"2023-12-19T18:16:27.832580+0000","last_active":"2023-12-19T18:16:40.549949+0000","last_peered":"2023-12-19T18:16:40.549949+0000","last_clean":"2023-12-19T18:16:40.549949+0000","last_became_active":"2023-12-19T18:16:27.832291+0000","last_became_peered":"2023-12-19T18:16:27.832291+0000","last_unstale":"2023-12-19T18:16:40.549949+0000","last_undegraded":"2023-12-19T18:16:40.549949+0000","last_fullsized":"2023-12-19T18:16:40.549949+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:29.305405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359160+0000","last_change":"2023-12-19T18:16:40.567943+0000","last_active":"2023-12-19T18:16:42.359160+0000","last_peered":"2023-12-19T18:16:42.359160+0000","last_clean":"2023-12-19T18:16:42.359160+0000","last_became_active":"2023-12-19T18:16:40.567701+0000","last_became_peered":"2023-12-19T18:16:40.567701+0000","last_unstale":"2023-12-19T18:16:42.359160+0000","last_undegraded":"2023-12-19T18:16:42.359160+0000","last_fullsized":"2023-12-19T18:16:42.359160+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:50:31.894613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.543104+0000","last_change":"2023-12-19T18:16:27.825730+0000","last_active":"2023-12-19T18:16:40.543104+0000","last_peered":"2023-12-19T18:16:40.543104+0000","last_clean":"2023-12-19T18:16:40.543104+0000","last_became_active":"2023-12-19T18:16:27.825136+0000","last_became_peered":"2023-12-19T18:16:27.825136+0000","last_unstale":"2023-12-19T18:16:40.543104+0000","last_undegraded":"2023-12-19T18:16:40.543104+0000","last_fullsized":"2023-12-19T18:16:40.543104+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:48.330735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.359437+0000","last_change":"2023-12-19T18:16:40.558188+0000","last_active":"2023-12-19T18:16:42.359437+0000","last_peered":"2023-12-19T18:16:42.359437+0000","last_clean":"2023-12-19T18:16:42.359437+0000","last_became_active":"2023-12-19T18:16:40.557804+0000","last_became_peered":"2023-12-19T18:16:40.557804+0000","last_unstale":"2023-12-19T18:16:42.359437+0000","last_undegraded":"2023-12-19T18:16:42.359437+0000","last_fullsized":"2023-12-19T18:16:42.359437+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:35:20.500418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.558018+0000","last_change":"2023-12-19T18:16:27.823852+0000","last_active":"2023-12-19T18:16:40.558018+0000","last_peered":"2023-12-19T18:16:40.558018+0000","last_clean":"2023-12-19T18:16:40.558018+0000","last_became_active":"2023-12-19T18:16:27.823564+0000","last_became_peered":"2023-12-19T18:16:27.823564+0000","last_unstale":"2023-12-19T18:16:40.558018+0000","last_undegraded":"2023-12-19T18:16:40.558018+0000","last_fullsized":"2023-12-19T18:16:40.558018+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:59.000440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358756+0000","last_change":"2023-12-19T18:16:40.561015+0000","last_active":"2023-12-19T18:16:42.358756+0000","last_peered":"2023-12-19T18:16:42.358756+0000","last_clean":"2023-12-19T18:16:42.358756+0000","last_became_active":"2023-12-19T18:16:40.560771+0000","last_became_peered":"2023-12-19T18:16:40.560771+0000","last_unstale":"2023-12-19T18:16:42.358756+0000","last_undegraded":"2023-12-19T18:16:42.358756+0000","last_fullsized":"2023-12-19T18:16:42.358756+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:04.628464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.554625+0000","last_change":"2023-12-19T18:16:27.822167+0000","last_active":"2023-12-19T18:16:40.554625+0000","last_peered":"2023-12-19T18:16:40.554625+0000","last_clean":"2023-12-19T18:16:40.554625+0000","last_became_active":"2023-12-19T18:16:27.821752+0000","last_became_peered":"2023-12-19T18:16:27.821752+0000","last_unstale":"2023-12-19T18:16:40.554625+0000","last_undegraded":"2023-12-19T18:16:40.554625+0000","last_fullsized":"2023-12-19T18:16:40.554625+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:04:17.866181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.357777+0000","last_change":"2023-12-19T18:16:40.565973+0000","last_active":"2023-12-19T18:16:42.357777+0000","last_peered":"2023-12-19T18:16:42.357777+0000","last_clean":"2023-12-19T18:16:42.357777+0000","last_became_active":"2023-12-19T18:16:40.565418+0000","last_became_peered":"2023-12-19T18:16:40.565418+0000","last_unstale":"2023-12-19T18:16:42.357777+0000","last_undegraded":"2023-12-19T18:16:42.357777+0000","last_fullsized":"2023-12-19T18:16:42.357777+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:54.473566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.551716+0000","last_change":"2023-12-19T18:16:27.835673+0000","last_active":"2023-12-19T18:16:40.551716+0000","last_peered":"2023-12-19T18:16:40.551716+0000","last_clean":"2023-12-19T18:16:40.551716+0000","last_became_active":"2023-12-19T18:16:27.835392+0000","last_became_peered":"2023-12-19T18:16:27.835392+0000","last_unstale":"2023-12-19T18:16:40.551716+0000","last_undegraded":"2023-12-19T18:16:40.551716+0000","last_fullsized":"2023-12-19T18:16:40.551716+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:23:06.990327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:41.553304+0000","last_change":"2023-12-19T18:16:40.564946+0000","last_active":"2023-12-19T18:16:41.553304+0000","last_peered":"2023-12-19T18:16:41.553304+0000","last_clean":"2023-12-19T18:16:41.553304+0000","last_became_active":"2023-12-19T18:16:40.564679+0000","last_became_peered":"2023-12-19T18:16:40.564679+0000","last_unstale":"2023-12-19T18:16:41.553304+0000","last_undegraded":"2023-12-19T18:16:41.553304+0000","last_fullsized":"2023-12-19T18:16:41.553304+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:02:28.048063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.559694+0000","last_change":"2023-12-19T18:16:27.830449+0000","last_active":"2023-12-19T18:16:40.559694+0000","last_peered":"2023-12-19T18:16:40.559694+0000","last_clean":"2023-12-19T18:16:40.559694+0000","last_became_active":"2023-12-19T18:16:27.830189+0000","last_became_peered":"2023-12-19T18:16:27.830189+0000","last_unstale":"2023-12-19T18:16:40.559694+0000","last_undegraded":"2023-12-19T18:16:40.559694+0000","last_fullsized":"2023-12-19T18:16:40.559694+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:11:06.397912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.360333+0000","last_change":"2023-12-19T18:16:40.576744+0000","last_active":"2023-12-19T18:16:42.360333+0000","last_peered":"2023-12-19T18:16:42.360333+0000","last_clean":"2023-12-19T18:16:42.360333+0000","last_became_active":"2023-12-19T18:16:40.576425+0000","last_became_peered":"2023-12-19T18:16:40.576425+0000","last_unstale":"2023-12-19T18:16:42.360333+0000","last_undegraded":"2023-12-19T18:16:42.360333+0000","last_fullsized":"2023-12-19T18:16:42.360333+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:26.259551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.548052+0000","last_change":"2023-12-19T18:16:27.815730+0000","last_active":"2023-12-19T18:16:40.548052+0000","last_peered":"2023-12-19T18:16:40.548052+0000","last_clean":"2023-12-19T18:16:40.548052+0000","last_became_active":"2023-12-19T18:16:27.815341+0000","last_became_peered":"2023-12-19T18:16:27.815341+0000","last_unstale":"2023-12-19T18:16:40.548052+0000","last_undegraded":"2023-12-19T18:16:40.548052+0000","last_fullsized":"2023-12-19T18:16:40.548052+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:39.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.074872+0000","last_change":"2023-12-19T18:16:40.578780+0000","last_active":"2023-12-19T18:16:42.074872+0000","last_peered":"2023-12-19T18:16:42.074872+0000","last_clean":"2023-12-19T18:16:42.074872+0000","last_became_active":"2023-12-19T18:16:40.578426+0000","last_became_peered":"2023-12-19T18:16:40.578426+0000","last_unstale":"2023-12-19T18:16:42.074872+0000","last_undegraded":"2023-12-19T18:16:42.074872+0000","last_fullsized":"2023-12-19T18:16:42.074872+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:53:39.733238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562588+0000","last_change":"2023-12-19T18:16:27.831511+0000","last_active":"2023-12-19T18:16:40.562588+0000","last_peered":"2023-12-19T18:16:40.562588+0000","last_clean":"2023-12-19T18:16:40.562588+0000","last_became_active":"2023-12-19T18:16:27.831255+0000","last_became_peered":"2023-12-19T18:16:27.831255+0000","last_unstale":"2023-12-19T18:16:40.562588+0000","last_undegraded":"2023-12-19T18:16:40.562588+0000","last_fullsized":"2023-12-19T18:16:40.562588+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:42.382254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":10,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-12-19T18:16:42.358249+0000","last_change":"2023-12-19T18:16:40.576919+0000","last_active":"2023-12-19T18:16:42.358249+0000","last_peered":"2023-12-19T18:16:42.358249+0000","last_clean":"2023-12-19T18:16:42.358249+0000","last_became_active":"2023-12-19T18:16:40.576351+0000","last_became_peered":"2023-12-19T18:16:40.576351+0000","last_unstale":"2023-12-19T18:16:42.358249+0000","last_undegraded":"2023-12-19T18:16:42.358249+0000","last_fullsized":"2023-12-19T18:16:42.358249+0000","mapping_epoch":214,"log_start":"0'0","ondisk_log_start":"0'0","created":214,"last_epoch_clean":215,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:40.124680+0000","last_clean_scrub_stamp":"2023-12-19T18:16:40.124680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:08:36.351083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":14,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-12-19T18:16:40.562201+0000","last_change":"2023-12-19T18:16:27.834221+0000","last_active":"2023-12-19T18:16:40.562201+0000","last_peered":"2023-12-19T18:16:40.562201+0000","last_clean":"2023-12-19T18:16:40.562201+0000","last_became_active":"2023-12-19T18:16:27.825886+0000","last_became_peered":"2023-12-19T18:16:27.825886+0000","last_unstale":"2023-12-19T18:16:40.562201+0000","last_undegraded":"2023-12-19T18:16:40.562201+0000","last_fullsized":"2023-12-19T18:16:40.562201+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:27.383256+0000","last_clean_scrub_stamp":"2023-12-19T18:16:27.383256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:12.119996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2114,"num_read_kb":4411,"num_write":575,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":346,"ondisk_log_size":346,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510570,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64616,"kb_used_data":3232,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94307224,"statfs":{"total":96636764160,"available":96570597376,"internally_reserved":0,"allocated":3309568,"data_stored":2022904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96399999999999997,"5min":1.048,"15min":1.1180000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.92400000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73899999999999999,"15min":0.74299999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.92400000000000004,"5min":1.0389999999999999,"15min":1.0580000000000001},"last":0.62},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72399999999999998,"15min":0.748},"min":{"1min":0.499,"5min":0.48099999999999998,"15min":0.435},"max":{"1min":0.89500000000000002,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.69999999999999996}]},{"osd":2,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.38900000000000001},"max":{"1min":0.93400000000000005,"5min":1.026,"15min":1.0669999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.52500000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93899999999999995,"5min":1.0820000000000001,"15min":1.236},"last":0.67400000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.75600000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88300000000000001,"5min":1.002,"15min":1.079},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98899999999999999,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.79300000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81599999999999995,"5min":1.0009999999999999,"15min":1.302},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61599999999999999,"15min":0.625},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.871,"15min":1.0209999999999999},"last":0.90200000000000002}]},{"osd":5,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.52700000000000002,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.88100000000000001,"5min":1.097,"15min":1.1299999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64900000000000002,"15min":0.66500000000000004},"min":{"1min":0.48999999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.89800000000000002,"5min":1.026,"15min":1.0800000000000001},"last":0.81200000000000006}]},{"osd":6,"last update":"Tue Dec 19 18:16:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70999999999999996,"15min":0.71599999999999997},"min":{"1min":0.56999999999999995,"5min":0.443,"15min":0.443},"max":{"1min":0.93999999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69199999999999995,"15min":0.70199999999999996},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.85699999999999998,"5min":1.01,"15min":1.093},"last":0.755}]}]},{"osd":6,"up_from":15,"seq":64424510570,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47636,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324204,"statfs":{"total":96636764160,"available":96587984896,"internally_reserved":0,"allocated":2699264,"data_stored":1431450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72799999999999998,"15min":0.74199999999999999},"min":{"1min":0.55400000000000005,"5min":0.46800000000000003,"15min":0.43099999999999999},"max":{"1min":0.92400000000000004,"5min":0.96499999999999997,"15min":1.042},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.71799999999999997,"15min":0.73499999999999999},"min":{"1min":0.57499999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92000000000000004,"5min":1.224,"15min":1.224},"last":0.81999999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.70099999999999996,"15min":0.72799999999999998},"min":{"1min":0.49099999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98099999999999998,"5min":1.103,"15min":1.103},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.64100000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":0.94599999999999995,"5min":0.99199999999999999,"15min":1.1259999999999999},"last":0.93600000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75700000000000001,"15min":0.76500000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.502},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0609999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.749},"min":{"1min":0.60899999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83299999999999996,"5min":1.0089999999999999,"15min":1.111},"last":0.98999999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.755,"15min":0.78200000000000003},"min":{"1min":0.61799999999999999,"5min":0.495,"15min":0.495},"max":{"1min":0.91200000000000003,"5min":0.97999999999999998,"15min":1.1519999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.51800000000000002,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.98699999999999999,"5min":1.268,"15min":1.268},"last":1.036}]},{"osd":4,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59799999999999998,"15min":0.59399999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.92200000000000004,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60499999999999998,"15min":0.60999999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":0.81299999999999994,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.65900000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.93799999999999994,"5min":1.194,"15min":1.194},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":1.141,"15min":1.141},"last":0.755}]},{"osd":7,"last update":"Tue Dec 19 18:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.95899999999999996,"5min":1.159,"15min":1.159},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73099999999999998,"15min":0.72799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.94199999999999995,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.77100000000000002}]}]},{"osd":5,"up_from":15,"seq":64424510570,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47636,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324204,"statfs":{"total":96636764160,"available":96587984896,"internally_reserved":0,"allocated":2699264,"data_stored":1431450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72499999999999998,"15min":0.72999999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34399999999999997},"max":{"1min":0.96899999999999997,"5min":1.0109999999999999,"15min":1.0149999999999999},"last":0.433},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.40699999999999997},"max":{"1min":0.89200000000000002,"5min":0.98199999999999998,"15min":1.0229999999999999},"last":0.55300000000000005}]},{"osd":1,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76800000000000002,"15min":0.76200000000000001},"min":{"1min":0.54900000000000004,"5min":0.52100000000000002,"15min":0.497},"max":{"1min":0.95599999999999996,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.505,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78800000000000003,"15min":0.78100000000000003},"min":{"1min":0.61699999999999999,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":1.022,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.73699999999999999},"min":{"1min":0.49399999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94499999999999995,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.54200000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76800000000000002,"15min":0.76300000000000001},"min":{"1min":0.48499999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.94999999999999996,"5min":1.05,"15min":1.05},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77900000000000003,"15min":0.77400000000000002},"min":{"1min":0.56200000000000006,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.98799999999999999,"5min":1.022,"15min":1.0329999999999999},"last":0.62}]},{"osd":4,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.58199999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.79000000000000004,"5min":0.94499999999999995,"15min":1.0800000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.63,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41599999999999998},"max":{"1min":0.71099999999999997,"5min":1.208,"15min":1.208},"last":0.5}]},{"osd":6,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64800000000000002,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.94899999999999995,"15min":0.98999999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.66000000000000003,"15min":0.65500000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.83499999999999996,"5min":1.008,"15min":1.0449999999999999},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Dec 19 18:16:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.69099999999999995,"15min":0.68300000000000005},"min":{"1min":0.46500000000000002,"5min":0.439,"15min":0.436},"max":{"1min":0.77300000000000002,"5min":1.169,"15min":1.169},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71499999999999997,"15min":0.70999999999999996},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.85699999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.56599999999999995}]}]},{"osd":4,"up_from":15,"seq":64424510570,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47668,"kb_used_data":2668,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324172,"statfs":{"total":96636764160,"available":96587952128,"internally_reserved":0,"allocated":2732032,"data_stored":1435923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.51600000000000001,"5min":0.45800000000000002,"15min":0.45000000000000001},"max":{"1min":1.016,"5min":1.036,"15min":1.109},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.502,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.91800000000000004,"5min":0.997,"15min":1.01},"last":0.60399999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.59199999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94699999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.997,"15min":1.1020000000000001},"last":0.69199999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79100000000000004,"15min":0.77700000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.47699999999999998},"max":{"1min":0.998,"5min":1.0289999999999999,"15min":1.071},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.73499999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76700000000000002,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93899999999999995,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.55500000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.85999999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65300000000000002,"15min":0.63100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67200000000000004,"15min":0.64600000000000002},"min":{"1min":0.495,"5min":0.42899999999999999,"15min":0.39600000000000002},"max":{"1min":0.88900000000000001,"5min":1.042,"15min":1.042},"last":0.89800000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.66100000000000003},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":1.024,"15min":1.0669999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0760000000000001},"last":0.88600000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.41899999999999998},"max":{"1min":0.92400000000000004,"5min":1.087,"15min":1.087},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77600000000000002,"15min":0.76200000000000001},"min":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.46999999999999997},"max":{"1min":0.98599999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.98499999999999999}]}]},{"osd":3,"up_from":15,"seq":64424510570,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47656,"kb_used_data":2656,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324184,"statfs":{"total":96636764160,"available":96587964416,"internally_reserved":0,"allocated":2719744,"data_stored":1431244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.63200000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.502,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74099999999999999,"5min":0.78300000000000003,"15min":0.80000000000000004},"last":0.56299999999999994}]},{"osd":1,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.498},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71499999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.629}]},{"osd":2,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.77000000000000002,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.51800000000000002}]},{"osd":4,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81599999999999995,"15min":0.82799999999999996},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.012,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.81999999999999995,"15min":0.82299999999999995},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.56599999999999995},"max":{"1min":0.86499999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82799999999999996,"15min":0.84199999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99099999999999999,"5min":1.1739999999999999,"15min":1.397},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.59399999999999997,"5min":0.55900000000000005,"15min":0.55100000000000005},"max":{"1min":0.91000000000000003,"5min":1.2230000000000001,"15min":1.262},"last":0.81499999999999995}]},{"osd":6,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.83899999999999997,"15min":0.84799999999999998},"min":{"1min":0.60199999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.96099999999999997,"5min":1.234,"15min":1.274},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83799999999999997,"15min":0.84499999999999997},"min":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94599999999999995,"5min":1.165,"15min":1.302},"last":0.79600000000000004}]},{"osd":7,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0640000000000001,"5min":1.3540000000000001,"15min":1.486},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89500000000000002},"min":{"1min":0.68899999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.93000000000000005,"5min":1.421,"15min":1.421},"last":0.83899999999999997}]}]},{"osd":2,"up_from":15,"seq":64424510569,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47644,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324196,"statfs":{"total":96636764160,"available":96587976704,"internally_reserved":0,"allocated":2707456,"data_stored":1431562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45100000000000001},"min":{"1min":0.29299999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.502,"15min":0.50900000000000001},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71099999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.54300000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.61299999999999999,"15min":0.622},"min":{"1min":0.45400000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.52400000000000002,"15min":0.52800000000000002},"min":{"1min":0.41499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.73899999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59999999999999998,"15min":0.61099999999999999},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.75,"5min":0.86199999999999999,"15min":0.95999999999999996},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.38200000000000001,"15min":0.36899999999999999},"max":{"1min":0.67200000000000004,"5min":0.79700000000000004,"15min":0.875},"last":0.67200000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80700000000000005,"15min":0.80400000000000005},"min":{"1min":0.624,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.048,"5min":1.306,"15min":1.306},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.05,"5min":1.272,"15min":1.466},"last":1.048}]},{"osd":5,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.64300000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.97699999999999998,"5min":1.1459999999999999,"15min":1.3049999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.81999999999999995,"15min":0.82599999999999996},"min":{"1min":0.61899999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.429},"last":0.93500000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.84299999999999997,"15min":0.84499999999999997},"min":{"1min":0.71199999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.216,"5min":1.365,"15min":1.4119999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80600000000000005,"15min":0.80800000000000005},"min":{"1min":0.625,"5min":0.55200000000000005,"15min":0.53500000000000003},"max":{"1min":0.94499999999999995,"5min":1.2,"15min":1.2470000000000001},"last":0.91700000000000004}]},{"osd":7,"last update":"Tue Dec 19 18:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.81499999999999995,"15min":0.82799999999999996},"min":{"1min":0.64400000000000002,"5min":0.59899999999999998,"15min":0.56000000000000005},"max":{"1min":1.0149999999999999,"5min":1.393,"15min":1.45},"last":1.006},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.85999999999999999,"15min":0.86099999999999999},"min":{"1min":0.69699999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.1850000000000001,"5min":1.276,"15min":1.276},"last":1.0649999999999999}]}]},{"osd":1,"up_from":15,"seq":64424510570,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59940,"kb_used_data":2652,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311900,"statfs":{"total":96636764160,"available":96575385600,"internally_reserved":0,"allocated":2715648,"data_stored":1433729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.33900000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.63700000000000001,"5min":0.71999999999999997,"15min":0.89400000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.53300000000000003},"min":{"1min":0.32200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77700000000000002,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.443}]},{"osd":2,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.52700000000000002,"15min":0.53300000000000003},"min":{"1min":0.32600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.57899999999999996},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.53600000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.45000000000000001,"15min":0.442},"max":{"1min":0.82699999999999996,"5min":0.94199999999999995,"15min":0.94499999999999995},"last":0.495},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.39100000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.40699999999999997}]},{"osd":4,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.65100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.93400000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.79100000000000004,"15min":0.81499999999999995},"min":{"1min":0.502,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.0409999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.68200000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.78400000000000003,"15min":0.80500000000000005},"min":{"1min":0.49099999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97499999999999998,"5min":1.236,"15min":1.2609999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.56399999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.98399999999999999,"5min":1.353,"15min":1.353},"last":0.66600000000000004}]},{"osd":6,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80200000000000005,"15min":0.81399999999999995},"min":{"1min":0.60999999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.008,"5min":1.2270000000000001,"15min":1.2689999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.84299999999999997,"15min":0.84999999999999998},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.093,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.57199999999999995}]},{"osd":7,"last update":"Tue Dec 19 18:16:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.84999999999999998,"15min":0.85799999999999998},"min":{"1min":0.64100000000000001,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.123,"5min":1.399,"15min":1.399},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.84299999999999997,"15min":0.85799999999999998},"min":{"1min":0.54100000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.105,"5min":1.365,"15min":1.365},"last":0.72199999999999998}]}]},{"osd":0,"up_from":14,"seq":60129543275,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60508,"kb_used_data":3220,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311332,"statfs":{"total":96636764160,"available":96574803968,"internally_reserved":0,"allocated":3297280,"data_stored":2021348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.438,"15min":0.45700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":1.216},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50600000000000001,"15min":0.52600000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.33000000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.503,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.64700000000000002,"5min":0.84999999999999998,"15min":1.1699999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72299999999999998,"5min":0.78900000000000003,"15min":0.96799999999999997},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.52800000000000002,"15min":0.55500000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.79000000000000004,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.623,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.433,"15min":0.433},"max":{"1min":0.82699999999999996,"5min":0.871,"15min":1.032},"last":0.55300000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.78700000000000003,"15min":0.81100000000000005},"min":{"1min":0.63,"5min":0.51800000000000002,"15min":0.496},"max":{"1min":0.93100000000000005,"5min":1.1200000000000001,"15min":1.381},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.80100000000000005,"15min":0.81599999999999995},"min":{"1min":0.59199999999999997,"5min":0.56799999999999995,"15min":0.47799999999999998},"max":{"1min":1.02,"5min":1.137,"15min":1.2849999999999999},"last":0.58299999999999996}]},{"osd":5,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80600000000000005,"15min":0.81899999999999995},"min":{"1min":0.57299999999999995,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.008,"5min":1.1930000000000001,"15min":1.3420000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.69599999999999995,"5min":0.59599999999999997,"15min":0.59499999999999997},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.288},"last":0.79100000000000004}]},{"osd":6,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.51300000000000001},"max":{"1min":0.98899999999999999,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81499999999999995,"15min":0.83399999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.0229999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:16:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83599999999999997,"15min":0.84999999999999998},"min":{"1min":0.65300000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.044,"5min":1.1899999999999999,"15min":1.415},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.83199999999999996,"15min":0.85599999999999998},"min":{"1min":0.55400000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0780000000000001,"5min":1.1379999999999999,"15min":1.395},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:16:49.649 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:16:49.649 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.822+0000 7f6e4ea55640 1 Processor -- start 2023-12-19T18:16:49.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.822+0000 7f6e4ea55640 1 -- start start 2023-12-19T18:16:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.822+0000 7f6e4ea55640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e48154220 0x7f6e4815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.822+0000 7f6e4ea55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e4806b3d0 0x7f6e480ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.822+0000 7f6e4ea55640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6e480cec40 0x7f6e4815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.822+0000 7f6e4ea55640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e480cf0d0 con 0x7f6e48154220 2023-12-19T18:16:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.822+0000 7f6e4ea55640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4806bc30 con 0x7f6e480cec40 2023-12-19T18:16:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.822+0000 7f6e4ea55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480cc6e0 con 0x7f6e4806b3d0 2023-12-19T18:16:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.822+0000 7f6e47fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e48154220 0x7f6e4815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.822+0000 7f6e47fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e48154220 0x7f6e4815aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32826/0 (socket says 172.21.15.26:32826) 2023-12-19T18:16:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.822+0000 7f6e47fff640 1 -- 172.21.15.26:0/528965340 learned_addr learned my addr 172.21.15.26:0/528965340 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.822+0000 7f6e4cff2640 1 --2- 172.21.15.26:0/528965340 >> v2:172.21.15.160:3300/0 conn(0x7f6e480cec40 0x7f6e4815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.826+0000 7f6e477fe640 1 --2- 172.21.15.26:0/528965340 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e4806b3d0 0x7f6e480ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.826+0000 7f6e47fff640 1 -- 172.21.15.26:0/528965340 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e4806b3d0 msgr2=0x7f6e480ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.826+0000 7f6e47fff640 1 --2- 172.21.15.26:0/528965340 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e4806b3d0 0x7f6e480ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.826+0000 7f6e47fff640 1 -- 172.21.15.26:0/528965340 >> v2:172.21.15.160:3300/0 conn(0x7f6e480cec40 msgr2=0x7f6e4815d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.826+0000 7f6e47fff640 1 --2- 172.21.15.26:0/528965340 >> v2:172.21.15.160:3300/0 conn(0x7f6e480cec40 0x7f6e4815d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.826+0000 7f6e47fff640 1 -- 172.21.15.26:0/528965340 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4815afc0 con 0x7f6e48154220 2023-12-19T18:16:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.826+0000 7f6e47fff640 1 --2- 172.21.15.26:0/528965340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e48154220 0x7f6e4815aa80 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f6e480ae310 tx=0x7f6e3801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145ce1ab6a54549d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.826+0000 7f6e46ffd640 1 -- 172.21.15.26:0/528965340 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e380ab600 con 0x7f6e48154220 2023-12-19T18:16:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.826+0000 7f6e46ffd640 1 -- 172.21.15.26:0/528965340 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e380a7050 con 0x7f6e48154220 2023-12-19T18:16:49.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.826+0000 7f6e46ffd640 1 -- 172.21.15.26:0/528965340 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e380ab980 con 0x7f6e48154220 2023-12-19T18:16:49.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 -- 172.21.15.26:0/528965340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e48154220 msgr2=0x7f6e4815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:49.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 --2- 172.21.15.26:0/528965340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e48154220 0x7f6e4815aa80 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f6e480ae310 tx=0x7f6e3801c210 comp rx=0 tx=0).stop 2023-12-19T18:16:49.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 -- 172.21.15.26:0/528965340 shutdown_connections 2023-12-19T18:16:49.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 --2- 172.21.15.26:0/528965340 >> v2:172.21.15.160:3300/0 conn(0x7f6e480cec40 0x7f6e4815d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 --2- 172.21.15.26:0/528965340 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e4806b3d0 0x7f6e480ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 --2- 172.21.15.26:0/528965340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e48154220 0x7f6e4815aa80 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 -- 172.21.15.26:0/528965340 >> 172.21.15.26:0/528965340 conn(0x7f6e480658e0 msgr2=0x7f6e48065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:49.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 -- 172.21.15.26:0/528965340 shutdown_connections 2023-12-19T18:16:49.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 -- 172.21.15.26:0/528965340 wait complete. 2023-12-19T18:16:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 Processor -- start 2023-12-19T18:16:49.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 -- start start 2023-12-19T18:16:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e4806b3d0 0x7f6e48156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480cec40 0x7f6e481fd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e48154220 0x7f6e481f9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e48156ad0 con 0x7f6e480cec40 2023-12-19T18:16:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e481fa330 con 0x7f6e4806b3d0 2023-12-19T18:16:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4ea55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e481fa610 con 0x7f6e48154220 2023-12-19T18:16:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480cec40 0x7f6e481fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e477fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480cec40 0x7f6e481fd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32832/0 (socket says 172.21.15.26:32832) 2023-12-19T18:16:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e477fe640 1 -- 172.21.15.26:0/2872981181 learned_addr learned my addr 172.21.15.26:0/2872981181 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e47fff640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e4806b3d0 0x7f6e48156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e4cff2640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e48154220 0x7f6e481f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e477fe640 1 -- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e48154220 msgr2=0x7f6e481f9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e477fe640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e48154220 0x7f6e481f9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e477fe640 1 -- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e4806b3d0 msgr2=0x7f6e48156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:49.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e477fe640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e4806b3d0 0x7f6e48156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.830+0000 7f6e477fe640 1 -- 172.21.15.26:0/2872981181 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e380094d0 con 0x7f6e480cec40 2023-12-19T18:16:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.834+0000 7f6e477fe640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480cec40 0x7f6e481fd070 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00e870 tx=0x7f6e3c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.834+0000 7f6e44ff9640 1 -- 172.21.15.26:0/2872981181 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c00ee60 con 0x7f6e480cec40 2023-12-19T18:16:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.834+0000 7f6e44ff9640 1 -- 172.21.15.26:0/2872981181 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c00c940 con 0x7f6e480cec40 2023-12-19T18:16:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.834+0000 7f6e44ff9640 1 -- 172.21.15.26:0/2872981181 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c00ccf0 con 0x7f6e480cec40 2023-12-19T18:16:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.834+0000 7f6e4ea55640 1 -- 172.21.15.26:0/2872981181 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e481566f0 con 0x7f6e480cec40 2023-12-19T18:16:49.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.834+0000 7f6e4ea55640 1 -- 172.21.15.26:0/2872981181 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e48157430 con 0x7f6e480cec40 2023-12-19T18:16:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.834+0000 7f6e4ea55640 1 -- 172.21.15.26:0/2872981181 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e481fa830 con 0x7f6e480cec40 2023-12-19T18:16:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.834+0000 7f6e44ff9640 1 -- 172.21.15.26:0/2872981181 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6e3c013470 con 0x7f6e480cec40 2023-12-19T18:16:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.834+0000 7f6e44ff9640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6e34065b80 0x7f6e34068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.834+0000 7f6e44ff9640 1 -- 172.21.15.26:0/2872981181 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f6e3c083900 con 0x7f6e480cec40 2023-12-19T18:16:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.834+0000 7f6e47fff640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6e34065b80 0x7f6e34068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:49.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.838+0000 7f6e47fff640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6e34065b80 0x7f6e34068040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6e380ad000 tx=0x7f6e380a6000 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:49.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:49.842+0000 7f6e44ff9640 1 -- 172.21.15.26:0/2872981181 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6e3c050c00 con 0x7f6e480cec40 2023-12-19T18:16:50.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.030+0000 7f6e4ea55640 1 -- 172.21.15.26:0/2872981181 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e48156c10 con 0x7f6e480cec40 2023-12-19T18:16:50.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.030+0000 7f6e44ff9640 1 -- 172.21.15.26:0/2872981181 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5917 (secure 0 0 0) 0x7f6e48156c10 con 0x7f6e480cec40 2023-12-19T18:16:50.033 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:50.034 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3349262044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3349262044},{"type":"v1","addr":"172.21.15.160:6836","nonce":3349262044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3865658274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3865658274},{"type":"v1","addr":"172.21.15.26:6833","nonce":3865658274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:27.798614+0000","modified":"2023-12-19T18:16:31.076294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3981286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3981286992},{"type":"v1","addr":"172.21.15.26:6835","nonce":3981286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:40.537606+0000","modified":"2023-12-19T18:16:43.550176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10605":{"gid":10605,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.160:6837/1294752100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1294752100},{"type":"v1","addr":"172.21.15.160:6837","nonce":1294752100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-19T18:16:50.034 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 102 2023-12-19T18:16:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.034+0000 7f6e4ea55640 1 -- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6e34065b80 msgr2=0x7f6e34068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.034+0000 7f6e4ea55640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6e34065b80 0x7f6e34068040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6e380ad000 tx=0x7f6e380a6000 comp rx=0 tx=0).stop 2023-12-19T18:16:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.034+0000 7f6e4ea55640 1 -- 172.21.15.26:0/2872981181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480cec40 msgr2=0x7f6e481fd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.034+0000 7f6e4ea55640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480cec40 0x7f6e481fd070 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00e870 tx=0x7f6e3c00ed30 comp rx=0 tx=0).stop 2023-12-19T18:16:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.034+0000 7f6e4ea55640 1 -- 172.21.15.26:0/2872981181 shutdown_connections 2023-12-19T18:16:50.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.034+0000 7f6e4ea55640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6e34065b80 0x7f6e34068040 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.034+0000 7f6e4ea55640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6e48154220 0x7f6e481f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.034+0000 7f6e4ea55640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e480cec40 0x7f6e481fd070 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.034+0000 7f6e4ea55640 1 --2- 172.21.15.26:0/2872981181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e4806b3d0 0x7f6e48156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.034+0000 7f6e4ea55640 1 -- 172.21.15.26:0/2872981181 >> 172.21.15.26:0/2872981181 conn(0x7f6e480658e0 msgr2=0x7f6e480caa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:50.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.034+0000 7f6e4ea55640 1 -- 172.21.15.26:0/2872981181 shutdown_connections 2023-12-19T18:16:50.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.034+0000 7f6e4ea55640 1 -- 172.21.15.26:0/2872981181 wait complete. 2023-12-19T18:16:50.458 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:16:40.537606+0000', 'modified': '2023-12-19T18:16:43.550176+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10605}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10605': {'gid': 10605, 'name': 'd', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.160:6837/1294752100', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6835', 'nonce': 1294752100}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 1294752100}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:16:50.458 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:16:50.458 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-19T18:16:50.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.602+0000 7f7022e66640 1 Processor -- start 2023-12-19T18:16:50.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.602+0000 7f7022e66640 1 -- start start 2023-12-19T18:16:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.602+0000 7f7022e66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f701c06b3d0 0x7f701c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.602+0000 7f7022e66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701c06bc30 con 0x7f701c154220 2023-12-19T18:16:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.602+0000 7f7022e66640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f701c0cc6e0 con 0x7f701c0cec40 2023-12-19T18:16:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.602+0000 7f7022e66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f701c0cf0d0 con 0x7f701c06b3d0 2023-12-19T18:16:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.602+0000 7f701bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f701c06b3d0 0x7f701c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.602+0000 7f701bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f701c06b3d0 0x7f701c0ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32844/0 (socket says 172.21.15.26:32844) 2023-12-19T18:16:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.602+0000 7f701bfff640 1 -- 172.21.15.26:0/295538327 learned_addr learned my addr 172.21.15.26:0/295538327 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.602+0000 7f7020c02640 1 --1- 172.21.15.26:0/295538327 >> v1:172.21.15.26:6789/0 conn(0x7f701c154220 0x7f701c15aa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44472/0 (socket says 172.21.15.26:44472) 2023-12-19T18:16:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.606+0000 7f701bfff640 1 -- 172.21.15.26:0/295538327 >> v1:172.21.15.160:6789/0 conn(0x7f701c0cec40 legacy=0x7f701c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.606+0000 7f701bfff640 1 -- 172.21.15.26:0/295538327 >> v1:172.21.15.26:6789/0 conn(0x7f701c154220 legacy=0x7f701c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.606+0000 7f701bfff640 1 -- 172.21.15.26:0/295538327 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c06b950 con 0x7f701c06b3d0 2023-12-19T18:16:50.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.606+0000 7f701b7fe640 1 -- 172.21.15.26:0/295538327 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170353516 0 0) 0x7f701c06bc30 con 0x7f701c154220 2023-12-19T18:16:50.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.606+0000 7f701bfff640 1 --2- 172.21.15.26:0/295538327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f701c06b3d0 0x7f701c0ce700 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f700800bb10 tx=0x7f700801ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1867f59042042b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.606+0000 7f701b7fe640 1 -- 172.21.15.26:0/295538327 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080a7cb0 con 0x7f701c06b3d0 2023-12-19T18:16:50.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.606+0000 7f701b7fe640 1 -- 172.21.15.26:0/295538327 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70080a7e10 con 0x7f701c06b3d0 2023-12-19T18:16:50.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.606+0000 7f701b7fe640 1 -- 172.21.15.26:0/295538327 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080af660 con 0x7f701c06b3d0 2023-12-19T18:16:50.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.606+0000 7f7022e66640 1 -- 172.21.15.26:0/295538327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f701c06b3d0 msgr2=0x7f701c0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.606+0000 7f7022e66640 1 --2- 172.21.15.26:0/295538327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f701c06b3d0 0x7f701c0ce700 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f700800bb10 tx=0x7f700801ebe0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.606+0000 7f7022e66640 1 -- 172.21.15.26:0/295538327 shutdown_connections 2023-12-19T18:16:50.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.606+0000 7f7022e66640 1 --2- 172.21.15.26:0/295538327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f701c06b3d0 0x7f701c0ce700 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.606+0000 7f7022e66640 1 -- 172.21.15.26:0/295538327 >> 172.21.15.26:0/295538327 conn(0x7f701c0658e0 msgr2=0x7f701c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:50.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7022e66640 1 -- 172.21.15.26:0/295538327 shutdown_connections 2023-12-19T18:16:50.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7022e66640 1 -- 172.21.15.26:0/295538327 wait complete. 2023-12-19T18:16:50.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7022e66640 1 Processor -- start 2023-12-19T18:16:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7022e66640 1 -- start start 2023-12-19T18:16:50.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7022e66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f701c06b3d0 0x7f701c14f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7022e66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f701c0cec40 0x7f701c0710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7022e66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f701c154220 0x7f701c0b2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7022e66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c14f540 con 0x7f701c06b3d0 2023-12-19T18:16:50.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7022e66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c154600 con 0x7f701c0cec40 2023-12-19T18:16:50.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7022e66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701c1548e0 con 0x7f701c154220 2023-12-19T18:16:50.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7020c02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f701c06b3d0 0x7f701c14f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7020c02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f701c06b3d0 0x7f701c14f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32848/0 (socket says 172.21.15.26:32848) 2023-12-19T18:16:50.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7020c02640 1 -- 172.21.15.26:0/606890583 learned_addr learned my addr 172.21.15.26:0/606890583 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:50.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f701bfff640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f701c0cec40 0x7f701c0710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7021403640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f701c154220 0x7f701c0b2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7020c02640 1 -- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f701c154220 msgr2=0x7f701c0b2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7020c02640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f701c154220 0x7f701c0b2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7020c02640 1 -- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f701c0cec40 msgr2=0x7f701c0710c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7020c02640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f701c0cec40 0x7f701c0710c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7020c02640 1 -- 172.21.15.26:0/606890583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700800b770 con 0x7f701c06b3d0 2023-12-19T18:16:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.610+0000 7f7020c02640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f701c06b3d0 0x7f701c14f000 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f701000c4a0 tx=0x7f701000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.614+0000 7f70197fa640 1 -- 172.21.15.26:0/606890583 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70100074e0 con 0x7f701c06b3d0 2023-12-19T18:16:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.614+0000 7f70197fa640 1 -- 172.21.15.26:0/606890583 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7010007640 con 0x7f701c06b3d0 2023-12-19T18:16:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.614+0000 7f70197fa640 1 -- 172.21.15.26:0/606890583 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010004a30 con 0x7f701c06b3d0 2023-12-19T18:16:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.614+0000 7f7022e66640 1 -- 172.21.15.26:0/606890583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701c154bc0 con 0x7f701c06b3d0 2023-12-19T18:16:50.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.614+0000 7f7022e66640 1 -- 172.21.15.26:0/606890583 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c155180 con 0x7f701c06b3d0 2023-12-19T18:16:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.614+0000 7f70197fa640 1 -- 172.21.15.26:0/606890583 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7010004b90 con 0x7f701c06b3d0 2023-12-19T18:16:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.614+0000 7f70197fa640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7000065b80 0x7f7000068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70197fa640 1 -- 172.21.15.26:0/606890583 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f7010082c80 con 0x7f701c06b3d0 2023-12-19T18:16:50.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f701bfff640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7000065b80 0x7f7000068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 Processor -- start 2023-12-19T18:16:50.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 -- start start 2023-12-19T18:16:50.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f701bfff640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7000065b80 0x7f7000068040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f70080b1000 tx=0x7f70080a9070 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb66519070 0x55fb66519450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 0x55fb664cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb664cf1c0 0x55fb664cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb662f2c90 con 0x55fb66519990 2023-12-19T18:16:50.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb66324220 con 0x55fb66519070 2023-12-19T18:16:50.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb663217f0 con 0x55fb664cf1c0 2023-12-19T18:16:50.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f701bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 0x55fb664cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f701bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 0x55fb664cec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32858/0 (socket says 172.21.15.26:32858) 2023-12-19T18:16:50.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f701bfff640 1 -- 172.21.15.26:0/1517139652 learned_addr learned my addr 172.21.15.26:0/1517139652 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:50.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f7020c02640 1 --2- 172.21.15.26:0/1517139652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb66519070 0x55fb66519450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f7021403640 1 --2- 172.21.15.26:0/1517139652 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb664cf1c0 0x55fb664cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f701bfff640 1 -- 172.21.15.26:0/1517139652 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb664cf1c0 msgr2=0x55fb664cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:50.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f701bfff640 1 --2- 172.21.15.26:0/1517139652 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb664cf1c0 0x55fb664cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f701bfff640 1 -- 172.21.15.26:0/1517139652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb66519070 msgr2=0x55fb66519450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:50.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f701bfff640 1 --2- 172.21.15.26:0/1517139652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb66519070 0x55fb66519450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f701bfff640 1 -- 172.21.15.26:0/1517139652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb66019890 con 0x55fb66519990 2023-12-19T18:16:50.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f701bfff640 1 --2- 172.21.15.26:0/1517139652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 0x55fb664cec80 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f70080b2040 tx=0x7f7008004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f0d00a5012924e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70067fc640 1 -- 172.21.15.26:0/1517139652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080b9ce0 con 0x55fb66519990 2023-12-19T18:16:50.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70067fc640 1 -- 172.21.15.26:0/1517139652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70080b9e40 con 0x55fb66519990 2023-12-19T18:16:50.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70067fc640 1 -- 172.21.15.26:0/1517139652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080ce680 con 0x55fb66519990 2023-12-19T18:16:50.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 -- 172.21.15.26:0/1517139652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 msgr2=0x55fb664cec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 --2- 172.21.15.26:0/1517139652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 0x55fb664cec80 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f70080b2040 tx=0x7f7008004740 comp rx=0 tx=0).stop 2023-12-19T18:16:50.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 -- 172.21.15.26:0/1517139652 shutdown_connections 2023-12-19T18:16:50.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 --2- 172.21.15.26:0/1517139652 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb664cf1c0 0x55fb664cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 --2- 172.21.15.26:0/1517139652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 0x55fb664cec80 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 --2- 172.21.15.26:0/1517139652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb66519070 0x55fb66519450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 -- 172.21.15.26:0/1517139652 >> 172.21.15.26:0/1517139652 conn(0x55fb664d5950 msgr2=0x55fb664cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 -- 172.21.15.26:0/1517139652 shutdown_connections 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.618+0000 7f70261c6000 1 -- 172.21.15.26:0/1517139652 wait complete. 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 1 -- start start 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 1 Processor -- start 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb664cf1c0 0x55fb664d4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 0x55fb664db080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb66519990 0x55fb664db5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6631d0e0 con 0x55fb66519070 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb662bc310 con 0x55fb664cf1c0 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb662da770 con 0x55fb66519990 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 10 client.0 _ll_register_callbacks cb 0x55fb664bab60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f701bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 0x55fb664db080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f701bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 0x55fb664db080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32868/0 (socket says 172.21.15.26:32868) 2023-12-19T18:16:50.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f701bfff640 1 -- 172.21.15.26:0/1688553469 learned_addr learned my addr 172.21.15.26:0/1688553469 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f7020c02640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb664cf1c0 0x55fb664d4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f7021403640 1 --2- 172.21.15.26:0/1688553469 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb66519990 0x55fb664db5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f701bfff640 1 -- 172.21.15.26:0/1688553469 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb66519990 msgr2=0x55fb664db5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f701bfff640 1 --2- 172.21.15.26:0/1688553469 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb66519990 0x55fb664db5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f701bfff640 1 -- 172.21.15.26:0/1688553469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb664cf1c0 msgr2=0x55fb664d4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f701bfff640 1 --2- 172.21.15.26:0/1688553469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb664cf1c0 0x55fb664d4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f701bfff640 1 -- 172.21.15.26:0/1688553469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c06b950 con 0x55fb66519070 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f7021403640 1 --2- 172.21.15.26:0/1688553469 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb66519990 0x55fb664db5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f701bfff640 1 --2- 172.21.15.26:0/1688553469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 0x55fb664db080 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f70080b2040 tx=0x7f700801d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8d56d63e4bf201 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f6ffb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f6ffb7fe640 1 -- 172.21.15.26:0/1688553469 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080d9670 con 0x55fb66519070 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 1 -- 172.21.15.26:0/1688553469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb662ecc10 con 0x55fb66519070 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f6ffb7fe640 1 -- 172.21.15.26:0/1688553469 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f70080d5070 con 0x55fb66519070 2023-12-19T18:16:50.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f6ffb7fe640 1 -- 172.21.15.26:0/1688553469 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70080d97d0 con 0x55fb66519070 2023-12-19T18:16:50.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 10 client.10923 fetch_fsmap learned FSMap version 102 2023-12-19T18:16:50.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 1 -- 172.21.15.26:0/1688553469 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x55fb662e2260 con 0x55fb66519070 2023-12-19T18:16:50.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f6ffb7fe640 1 -- 172.21.15.26:0/1688553469 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb662ecc10 con 0x55fb66519070 2023-12-19T18:16:50.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f6ffb7fe640 1 -- 172.21.15.26:0/1688553469 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 102) v1 ==== 3681+0+0 (secure 0 0 0) 0x7f70080c1070 con 0x55fb66519070 2023-12-19T18:16:50.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 10 client.10923 fetch_fsmap finished waiting for FSMap version 102 2023-12-19T18:16:50.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 10 client.10923 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-19T18:16:50.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 20 client.10923 populate_metadata read hostname 'smithi026' 2023-12-19T18:16:50.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 1 --2- 172.21.15.26:0/1688553469 >> [v2:172.21.15.160:6835/1294752100,v1:172.21.15.160:6837/1294752100] conn(0x55fb665208d0 0x55fb66520cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 4 client.10923 mds_command: new command op to 10605 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:16:50.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.622+0000 7f70261c6000 1 -- 172.21.15.26:0/1688553469 --> [v2:172.21.15.160:6835/1294752100,v1:172.21.15.160:6837/1294752100] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb6631ba30 con 0x55fb665208d0 2023-12-19T18:16:50.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f7020c02640 1 --2- 172.21.15.26:0/1688553469 >> [v2:172.21.15.160:6835/1294752100,v1:172.21.15.160:6837/1294752100] conn(0x55fb665208d0 0x55fb66520cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f7020c02640 1 --2- 172.21.15.26:0/1688553469 >> [v2:172.21.15.160:6835/1294752100,v1:172.21.15.160:6837/1294752100] conn(0x55fb665208d0 0x55fb66520cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f5ec3a7cf844add server_cookie=e24d93d71d177b6a in_seq=0 out_seq=0 2023-12-19T18:16:50.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f6ffb7fe640 10 client.10923 ms_handle_connect on v2:172.21.15.160:6835/1294752100 2023-12-19T18:16:50.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f6ffb7fe640 1 -- 172.21.15.26:0/1688553469 <== mds.0 v2:172.21.15.160:6835/1294752100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fb6631ba30 con 0x55fb665208d0 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f6ffb7fe640 10 client.10923 handle_command_reply: tid=0 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f70261c6000 1 client.10923 shutdown 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f70261c6000 1 -- 172.21.15.26:0/1688553469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 msgr2=0x55fb664db080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f70261c6000 1 --2- 172.21.15.26:0/1688553469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 0x55fb664db080 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f70080b2040 tx=0x7f700801d420 comp rx=0 tx=0).stop 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f70261c6000 1 -- 172.21.15.26:0/1688553469 shutdown_connections 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f70261c6000 1 --2- 172.21.15.26:0/1688553469 >> [v2:172.21.15.160:6835/1294752100,v1:172.21.15.160:6837/1294752100] conn(0x55fb665208d0 0x55fb66520cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f70261c6000 1 --2- 172.21.15.26:0/1688553469 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb66519990 0x55fb664db5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f70261c6000 1 --2- 172.21.15.26:0/1688553469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 0x55fb664db080 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f70261c6000 1 --2- 172.21.15.26:0/1688553469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb664cf1c0 0x55fb664d4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f6ffb7fe640 0 client.10923 ms_handle_reset on v2:172.21.15.160:6835/1294752100 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f70261c6000 1 -- 172.21.15.26:0/1688553469 >> 172.21.15.26:0/1688553469 conn(0x55fb664d5950 msgr2=0x55fb664cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f70261c6000 1 -- 172.21.15.26:0/1688553469 shutdown_connections 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f70261c6000 1 -- 172.21.15.26:0/1688553469 wait complete. 2023-12-19T18:16:50.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.626+0000 7f70261c6000 20 client.10923 trim_cache size 0 max 16384 2023-12-19T18:16:50.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f70261c6000 1 Processor -- start 2023-12-19T18:16:50.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f70261c6000 1 -- start start 2023-12-19T18:16:50.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f70261c6000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb664cf1c0 0x55fb66575540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f70261c6000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 0x55fb66575a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f70261c6000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb66519990 0x55fb66575fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f70261c6000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb65fda940 con 0x55fb66519070 2023-12-19T18:16:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f70261c6000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb662c5860 con 0x55fb66519990 2023-12-19T18:16:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f70261c6000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb66360ce0 con 0x55fb664cf1c0 2023-12-19T18:16:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f701bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 0x55fb66575a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f701bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 0x55fb66575a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32880/0 (socket says 172.21.15.26:32880) 2023-12-19T18:16:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f701bfff640 1 -- 172.21.15.26:0/4102459045 learned_addr learned my addr 172.21.15.26:0/4102459045 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f701bfff640 1 -- 172.21.15.26:0/4102459045 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb664cf1c0 msgr2=0x55fb66575540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f701bfff640 1 --2- 172.21.15.26:0/4102459045 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb664cf1c0 0x55fb66575540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f701bfff640 1 -- 172.21.15.26:0/4102459045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb66519990 msgr2=0x55fb66575fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f701bfff640 1 --2- 172.21.15.26:0/4102459045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb66519990 0x55fb66575fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f701bfff640 1 -- 172.21.15.26:0/4102459045 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb662c83b0 con 0x55fb66519070 2023-12-19T18:16:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f701bfff640 1 --2- 172.21.15.26:0/4102459045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 0x55fb66575a80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f700801c810 tx=0x7f70080d2050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6adaea04eabd18 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f7006ffd640 1 -- 172.21.15.26:0/4102459045 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080ce460 con 0x55fb66519070 2023-12-19T18:16:50.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f7006ffd640 1 -- 172.21.15.26:0/4102459045 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70080d9470 con 0x55fb66519070 2023-12-19T18:16:50.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.642+0000 7f7006ffd640 1 -- 172.21.15.26:0/4102459045 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080d9820 con 0x55fb66519070 2023-12-19T18:16:50.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 -- 172.21.15.26:0/4102459045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 msgr2=0x55fb66575a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 --2- 172.21.15.26:0/4102459045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 0x55fb66575a80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f700801c810 tx=0x7f70080d2050 comp rx=0 tx=0).stop 2023-12-19T18:16:50.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 -- 172.21.15.26:0/4102459045 shutdown_connections 2023-12-19T18:16:50.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 --2- 172.21.15.26:0/4102459045 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb66519990 0x55fb66575fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 --2- 172.21.15.26:0/4102459045 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519070 0x55fb66575a80 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 --2- 172.21.15.26:0/4102459045 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb664cf1c0 0x55fb66575540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 -- 172.21.15.26:0/4102459045 >> 172.21.15.26:0/4102459045 conn(0x55fb664d5950 msgr2=0x55fb664daf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 -- 172.21.15.26:0/4102459045 shutdown_connections 2023-12-19T18:16:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 -- 172.21.15.26:0/4102459045 wait complete. 2023-12-19T18:16:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 -- start start 2023-12-19T18:16:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 Processor -- start 2023-12-19T18:16:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb664cf1c0 0x55fb665344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb66519070 0x55fb66534a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 0x55fb66534f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb6631fc40 con 0x55fb66519990 2023-12-19T18:16:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb664c69d0 con 0x55fb664cf1c0 2023-12-19T18:16:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb663254b0 con 0x55fb66519070 2023-12-19T18:16:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 10 client.0 _ll_register_callbacks cb 0x55fb662cb640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:16:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f7021403640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 0x55fb66534f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f7021403640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 0x55fb66534f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32884/0 (socket says 172.21.15.26:32884) 2023-12-19T18:16:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f7021403640 1 -- 172.21.15.26:0/1002299408 learned_addr learned my addr 172.21.15.26:0/1002299408 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f7020c02640 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb664cf1c0 0x55fb665344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f701bfff640 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb66519070 0x55fb66534a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f7021403640 1 -- 172.21.15.26:0/1002299408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb66519070 msgr2=0x55fb66534a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f7021403640 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb66519070 0x55fb66534a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f7021403640 1 -- 172.21.15.26:0/1002299408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb664cf1c0 msgr2=0x55fb665344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f7021403640 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb664cf1c0 0x55fb665344e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f7021403640 1 -- 172.21.15.26:0/1002299408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb662e2260 con 0x55fb66519990 2023-12-19T18:16:50.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f701bfff640 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb66519070 0x55fb66534a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:50.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f7021403640 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 0x55fb66534f60 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f7014005950 tx=0x7f7014001920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e13b08845074176 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f6ffb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T18:16:50.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f6ffb7fe640 1 -- 172.21.15.26:0/1002299408 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7014016480 con 0x55fb66519990 2023-12-19T18:16:50.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f6ffb7fe640 1 -- 172.21.15.26:0/1002299408 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f70140165e0 con 0x55fb66519990 2023-12-19T18:16:50.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 -- 172.21.15.26:0/1002299408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb662e2e00 con 0x55fb66519990 2023-12-19T18:16:50.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f6ffb7fe640 1 -- 172.21.15.26:0/1002299408 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70140167a0 con 0x55fb66519990 2023-12-19T18:16:50.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 10 client.10929 fetch_fsmap learned FSMap version 102 2023-12-19T18:16:50.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f70261c6000 1 -- 172.21.15.26:0/1002299408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x55fb66019cc0 con 0x55fb66519990 2023-12-19T18:16:50.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.646+0000 7f6ffb7fe640 1 -- 172.21.15.26:0/1002299408 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb662e2e00 con 0x55fb66519990 2023-12-19T18:16:50.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f6ffb7fe640 1 -- 172.21.15.26:0/1002299408 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 102) v1 ==== 3681+0+0 (secure 0 0 0) 0x7f7014016900 con 0x55fb66519990 2023-12-19T18:16:50.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 10 client.10929 fetch_fsmap finished waiting for FSMap version 102 2023-12-19T18:16:50.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 10 client.10929 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-19T18:16:50.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 20 client.10929 populate_metadata read hostname 'smithi026' 2023-12-19T18:16:50.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.160:6835/1294752100,v1:172.21.15.160:6837/1294752100] conn(0x55fb665354a0 0x55fb66535880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 4 client.10929 mds_command: new command op to 10605 tid=0[{"prefix": "status"}] 2023-12-19T18:16:50.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 1 -- 172.21.15.26:0/1002299408 --> [v2:172.21.15.160:6835/1294752100,v1:172.21.15.160:6837/1294752100] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fb66316160 con 0x55fb665354a0 2023-12-19T18:16:50.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7020c02640 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.160:6835/1294752100,v1:172.21.15.160:6837/1294752100] conn(0x55fb665354a0 0x55fb66535880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7020c02640 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.160:6835/1294752100,v1:172.21.15.160:6837/1294752100] conn(0x55fb665354a0 0x55fb66535880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=947ce591d596760f server_cookie=80b8e74c53d10f8c in_seq=0 out_seq=0 2023-12-19T18:16:50.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f6ffb7fe640 10 client.10929 ms_handle_connect on v2:172.21.15.160:6835/1294752100 2023-12-19T18:16:50.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f6ffb7fe640 1 -- 172.21.15.26:0/1002299408 <== mds.0 v2:172.21.15.160:6835/1294752100 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55fb66316160 con 0x55fb665354a0 2023-12-19T18:16:50.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f6ffb7fe640 10 client.10929 handle_command_reply: tid=0 2023-12-19T18:16:50.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 1 client.10929 shutdown 2023-12-19T18:16:50.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 1 -- 172.21.15.26:0/1002299408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 msgr2=0x55fb66534f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 0x55fb66534f60 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f7014005950 tx=0x7f7014001920 comp rx=0 tx=0).stop 2023-12-19T18:16:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 1 -- 172.21.15.26:0/1002299408 shutdown_connections 2023-12-19T18:16:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.160:6835/1294752100,v1:172.21.15.160:6837/1294752100] conn(0x55fb665354a0 0x55fb66535880 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55fb66519990 0x55fb66534f60 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55fb66519070 0x55fb66534a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 1 --2- 172.21.15.26:0/1002299408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55fb664cf1c0 0x55fb665344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f6ffb7fe640 0 client.10929 ms_handle_reset on v2:172.21.15.160:6835/1294752100 2023-12-19T18:16:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 1 -- 172.21.15.26:0/1002299408 >> 172.21.15.26:0/1002299408 conn(0x55fb664d5950 msgr2=0x55fb66513ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 1 -- 172.21.15.26:0/1002299408 shutdown_connections 2023-12-19T18:16:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 1 -- 172.21.15.26:0/1002299408 wait complete. 2023-12-19T18:16:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f70261c6000 20 client.10929 trim_cache size 0 max 16384 2023-12-19T18:16:50.659 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:16:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T18:16:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T18:16:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 10605, 2023-12-19T18:16:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T18:16:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T18:16:50.659 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "backup_fs", 2023-12-19T18:16:50.660 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 10.112477252, 2023-12-19T18:16:50.660 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 102, 2023-12-19T18:16:50.660 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 216, 2023-12-19T18:16:50.660 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 216, 2023-12-19T18:16:50.660 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 33.585432220000001 2023-12-19T18:16:50.660 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:16:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7022e66640 1 -- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7000065b80 msgr2=0x7f7000068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7022e66640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7000065b80 0x7f7000068040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f70080b1000 tx=0x7f70080a9070 comp rx=0 tx=0).stop 2023-12-19T18:16:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7022e66640 1 -- 172.21.15.26:0/606890583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f701c06b3d0 msgr2=0x7f701c14f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7022e66640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f701c06b3d0 0x7f701c14f000 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f701000c4a0 tx=0x7f701000c960 comp rx=0 tx=0).stop 2023-12-19T18:16:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7022e66640 1 -- 172.21.15.26:0/606890583 shutdown_connections 2023-12-19T18:16:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7022e66640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7000065b80 0x7f7000068040 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7022e66640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f701c154220 0x7f701c0b2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7022e66640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f701c0cec40 0x7f701c0710c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7022e66640 1 --2- 172.21.15.26:0/606890583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f701c06b3d0 0x7f701c14f000 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7022e66640 1 -- 172.21.15.26:0/606890583 >> 172.21.15.26:0/606890583 conn(0x7f701c0658e0 msgr2=0x7f701c15a850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7022e66640 1 -- 172.21.15.26:0/606890583 shutdown_connections 2023-12-19T18:16:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.650+0000 7f7022e66640 1 -- 172.21.15.26:0/606890583 wait complete. 2023-12-19T18:16:50.673 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-19T18:16:50.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.814+0000 7fd124b2c640 1 Processor -- start 2023-12-19T18:16:50.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.814+0000 7fd124b2c640 1 -- start start 2023-12-19T18:16:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.814+0000 7fd124b2c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd12006b3d0 0x7fd1200ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.814+0000 7fd124b2c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd1200cec40 0x7fd12015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.814+0000 7fd124b2c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12006bc30 con 0x7fd120154220 2023-12-19T18:16:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.814+0000 7fd124b2c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1200cf0d0 con 0x7fd12006b3d0 2023-12-19T18:16:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.814+0000 7fd124b2c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1200cc6e0 con 0x7fd1200cec40 2023-12-19T18:16:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.818+0000 7fd117fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd12006b3d0 0x7fd1200ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.818+0000 7fd117fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd12006b3d0 0x7fd1200ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32898/0 (socket says 172.21.15.26:32898) 2023-12-19T18:16:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.818+0000 7fd117fff640 1 -- 172.21.15.26:0/2637758317 learned_addr learned my addr 172.21.15.26:0/2637758317 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.818+0000 7fd11ed9d640 1 --2- 172.21.15.26:0/2637758317 >> v2:172.21.15.160:3300/0 conn(0x7fd1200cec40 0x7fd12015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.818+0000 7fd117fff640 1 -- 172.21.15.26:0/2637758317 >> v2:172.21.15.160:3300/0 conn(0x7fd1200cec40 msgr2=0x7fd12015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.818+0000 7fd117fff640 1 --2- 172.21.15.26:0/2637758317 >> v2:172.21.15.160:3300/0 conn(0x7fd1200cec40 0x7fd12015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.818+0000 7fd117fff640 1 -- 172.21.15.26:0/2637758317 >> v1:172.21.15.160:6789/0 conn(0x7fd120154220 legacy=0x7fd12015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.818+0000 7fd117fff640 1 -- 172.21.15.26:0/2637758317 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12006b950 con 0x7fd12006b3d0 2023-12-19T18:16:50.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.818+0000 7fd117fff640 1 --2- 172.21.15.26:0/2637758317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12006b3d0 0x7fd1200ce700 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fd120157bd0 tx=0x7fd10801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c42aab6c8cdfb6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.818+0000 7fd11dd9b640 1 -- 172.21.15.26:0/2637758317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1080aa430 con 0x7fd12006b3d0 2023-12-19T18:16:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.818+0000 7fd11dd9b640 1 -- 172.21.15.26:0/2637758317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1080aa590 con 0x7fd12006b3d0 2023-12-19T18:16:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.818+0000 7fd11dd9b640 1 -- 172.21.15.26:0/2637758317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1080aa920 con 0x7fd12006b3d0 2023-12-19T18:16:50.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 -- 172.21.15.26:0/2637758317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12006b3d0 msgr2=0x7fd1200ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 --2- 172.21.15.26:0/2637758317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12006b3d0 0x7fd1200ce700 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fd120157bd0 tx=0x7fd10801c4d0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 -- 172.21.15.26:0/2637758317 shutdown_connections 2023-12-19T18:16:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 --2- 172.21.15.26:0/2637758317 >> v2:172.21.15.160:3300/0 conn(0x7fd1200cec40 0x7fd12015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 --2- 172.21.15.26:0/2637758317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd12006b3d0 0x7fd1200ce700 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 -- 172.21.15.26:0/2637758317 >> 172.21.15.26:0/2637758317 conn(0x7fd1200658e0 msgr2=0x7fd120065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 -- 172.21.15.26:0/2637758317 shutdown_connections 2023-12-19T18:16:50.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 -- 172.21.15.26:0/2637758317 wait complete. 2023-12-19T18:16:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 Processor -- start 2023-12-19T18:16:50.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 -- start start 2023-12-19T18:16:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12006b3d0 0x7fd120152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1200cec40 0x7fd120133970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd120154220 0x7fd120138e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12014e550 con 0x7fd120154220 2023-12-19T18:16:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd120152c60 con 0x7fd1200cec40 2023-12-19T18:16:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd124b2c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd120152e50 con 0x7fd12006b3d0 2023-12-19T18:16:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd11ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd120154220 0x7fd120138e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd11ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd120154220 0x7fd120138e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32904/0 (socket says 172.21.15.26:32904) 2023-12-19T18:16:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd11ed9d640 1 -- 172.21.15.26:0/4175934456 learned_addr learned my addr 172.21.15.26:0/4175934456 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd117fff640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1200cec40 0x7fd120133970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.822+0000 7fd11e59c640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12006b3d0 0x7fd120152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd11ed9d640 1 -- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12006b3d0 msgr2=0x7fd120152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd11ed9d640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12006b3d0 0x7fd120152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd11ed9d640 1 -- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1200cec40 msgr2=0x7fd120133970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd11ed9d640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1200cec40 0x7fd120133970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd11ed9d640 1 -- 172.21.15.26:0/4175934456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1080094d0 con 0x7fd120154220 2023-12-19T18:16:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd11e59c640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12006b3d0 0x7fd120152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd11ed9d640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd120154220 0x7fd120138e80 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fd11800efc0 tx=0x7fd11800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd116ffd640 1 -- 172.21.15.26:0/4175934456 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118014480 con 0x7fd120154220 2023-12-19T18:16:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd116ffd640 1 -- 172.21.15.26:0/4175934456 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11800f040 con 0x7fd120154220 2023-12-19T18:16:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd124b2c640 1 -- 172.21.15.26:0/4175934456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12014dbb0 con 0x7fd120154220 2023-12-19T18:16:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd116ffd640 1 -- 172.21.15.26:0/4175934456 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118014820 con 0x7fd120154220 2023-12-19T18:16:50.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd124b2c640 1 -- 172.21.15.26:0/4175934456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12014e170 con 0x7fd120154220 2023-12-19T18:16:50.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd124b2c640 1 -- 172.21.15.26:0/4175934456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12014e690 con 0x7fd120154220 2023-12-19T18:16:50.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd116ffd640 1 -- 172.21.15.26:0/4175934456 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd118014980 con 0x7fd120154220 2023-12-19T18:16:50.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd116ffd640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd100065b80 0x7fd100068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.826+0000 7fd116ffd640 1 -- 172.21.15.26:0/4175934456 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fd1180845e0 con 0x7fd120154220 2023-12-19T18:16:50.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.830+0000 7fd11e59c640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd100065b80 0x7fd100068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.830+0000 7fd11e59c640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd100065b80 0x7fd100068040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd110000f70 tx=0x7fd110009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:50.830+0000 7fd116ffd640 1 -- 172.21.15.26:0/4175934456 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd12014e690 con 0x7fd120154220 2023-12-19T18:16:51.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.010+0000 7fd124b2c640 1 -- 172.21.15.26:0/4175934456 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1200ccef0 con 0x7fd100065b80 2023-12-19T18:16:51.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.014+0000 7fd116ffd640 1 -- 172.21.15.26:0/4175934456 <== mgr.9805 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7fd1100013f0 con 0x7fd100065b80 2023-12-19T18:16:51.015 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T18:16:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2023-12-19T18:16:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:16:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:16:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "backup_fs" 2023-12-19T18:16:51.015 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:16:51.015 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T18:16:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.014+0000 7fd124b2c640 1 -- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd100065b80 msgr2=0x7fd100068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.014+0000 7fd124b2c640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd100065b80 0x7fd100068040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd110000f70 tx=0x7fd110009040 comp rx=0 tx=0).stop 2023-12-19T18:16:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.014+0000 7fd124b2c640 1 -- 172.21.15.26:0/4175934456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd120154220 msgr2=0x7fd120138e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.014+0000 7fd124b2c640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd120154220 0x7fd120138e80 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fd11800efc0 tx=0x7fd11800c490 comp rx=0 tx=0).stop 2023-12-19T18:16:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.014+0000 7fd124b2c640 1 -- 172.21.15.26:0/4175934456 shutdown_connections 2023-12-19T18:16:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.014+0000 7fd124b2c640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd100065b80 0x7fd100068040 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.014+0000 7fd124b2c640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd120154220 0x7fd120138e80 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.014+0000 7fd124b2c640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd1200cec40 0x7fd120133970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.014+0000 7fd124b2c640 1 --2- 172.21.15.26:0/4175934456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd12006b3d0 0x7fd120152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.014+0000 7fd124b2c640 1 -- 172.21.15.26:0/4175934456 >> 172.21.15.26:0/4175934456 conn(0x7fd1200658e0 msgr2=0x7fd12015c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.014+0000 7fd124b2c640 1 -- 172.21.15.26:0/4175934456 shutdown_connections 2023-12-19T18:16:51.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.014+0000 7fd124b2c640 1 -- 172.21.15.26:0/4175934456 wait complete. 2023-12-19T18:16:51.036 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-19T18:16:51.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.178+0000 7f7f2bf39640 1 Processor -- start 2023-12-19T18:16:51.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.178+0000 7f7f2bf39640 1 -- start start 2023-12-19T18:16:51.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f2bf39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f240ca220 0x7f7f240cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f2bf39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7f2406dbe0 0x7f7f2406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f2bf39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f24159eb0 con 0x7f7f240c9e40 2023-12-19T18:16:51.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f2bf39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f2415a190 con 0x7f7f2406dbe0 2023-12-19T18:16:51.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f2bf39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2415a470 con 0x7f7f240ca220 2023-12-19T18:16:51.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f2a4d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7f2406dbe0 0x7f7f2406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f2a4d6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7f2406dbe0 0x7f7f2406bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32914/0 (socket says 172.21.15.26:32914) 2023-12-19T18:16:51.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f2a4d6640 1 -- 172.21.15.26:0/655474642 learned_addr learned my addr 172.21.15.26:0/655474642 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:51.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f294d4640 1 --2- 172.21.15.26:0/655474642 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f240ca220 0x7f7f240cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f2a4d6640 1 -- 172.21.15.26:0/655474642 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f240ca220 msgr2=0x7f7f240cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:51.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f2a4d6640 1 --2- 172.21.15.26:0/655474642 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f240ca220 0x7f7f240cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f2a4d6640 1 -- 172.21.15.26:0/655474642 >> v1:172.21.15.160:6789/0 conn(0x7f7f240c9e40 legacy=0x7f7f241596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f2a4d6640 1 -- 172.21.15.26:0/655474642 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2406d3e0 con 0x7f7f2406dbe0 2023-12-19T18:16:51.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.182+0000 7f7f294d4640 1 --2- 172.21.15.26:0/655474642 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f240ca220 0x7f7f240cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2a4d6640 1 --2- 172.21.15.26:0/655474642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f2406dbe0 0x7f7f2406bbf0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009e50 tx=0x7f7f1c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd817fd6abf0d9c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f28cd3640 1 -- 172.21.15.26:0/655474642 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0a5850 con 0x7f7f2406dbe0 2023-12-19T18:16:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f28cd3640 1 -- 172.21.15.26:0/655474642 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c0a59b0 con 0x7f7f2406dbe0 2023-12-19T18:16:51.187 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f28cd3640 1 -- 172.21.15.26:0/655474642 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0a5d40 con 0x7f7f2406dbe0 2023-12-19T18:16:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 -- 172.21.15.26:0/655474642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f2406dbe0 msgr2=0x7f7f2406bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 --2- 172.21.15.26:0/655474642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f2406dbe0 0x7f7f2406bbf0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009e50 tx=0x7f7f1c01c9e0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 -- 172.21.15.26:0/655474642 shutdown_connections 2023-12-19T18:16:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 --2- 172.21.15.26:0/655474642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f2406dbe0 0x7f7f2406bbf0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 --2- 172.21.15.26:0/655474642 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f240ca220 0x7f7f240cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 -- 172.21.15.26:0/655474642 >> 172.21.15.26:0/655474642 conn(0x7f7f240cf260 msgr2=0x7f7f240cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 -- 172.21.15.26:0/655474642 shutdown_connections 2023-12-19T18:16:51.189 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 -- 172.21.15.26:0/655474642 wait complete. 2023-12-19T18:16:51.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 Processor -- start 2023-12-19T18:16:51.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 -- start start 2023-12-19T18:16:51.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f2406dbe0 0x7f7f240c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f240c9e40 0x7f7f240c3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f240ca220 0x7f7f24153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240c57f0 con 0x7f7f240c9e40 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240c54a0 con 0x7f7f2406dbe0 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.186+0000 7f7f2bf39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240c3dc0 con 0x7f7f240ca220 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f294d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f240c9e40 0x7f7f240c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f294d4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f240c9e40 0x7f7f240c3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32928/0 (socket says 172.21.15.26:32928) 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f294d4640 1 -- 172.21.15.26:0/3179584799 learned_addr learned my addr 172.21.15.26:0/3179584799 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f29cd5640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f2406dbe0 0x7f7f240c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f2a4d6640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f240ca220 0x7f7f24153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f294d4640 1 -- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f240ca220 msgr2=0x7f7f24153450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f294d4640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f240ca220 0x7f7f24153450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f294d4640 1 -- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f2406dbe0 msgr2=0x7f7f240c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f294d4640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f2406dbe0 0x7f7f240c4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.192 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f294d4640 1 -- 172.21.15.26:0/3179584799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c009b30 con 0x7f7f240c9e40 2023-12-19T18:16:51.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f294d4640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f240c9e40 0x7f7f240c3880 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400d7d0 tx=0x7f7f1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:51.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f1a7fc640 1 -- 172.21.15.26:0/3179584799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400b6c0 con 0x7f7f240c9e40 2023-12-19T18:16:51.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f1a7fc640 1 -- 172.21.15.26:0/3179584799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1400b820 con 0x7f7f240c9e40 2023-12-19T18:16:51.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f1a7fc640 1 -- 172.21.15.26:0/3179584799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400bbd0 con 0x7f7f240c9e40 2023-12-19T18:16:51.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f2bf39640 1 -- 172.21.15.26:0/3179584799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f240c4890 con 0x7f7f240c9e40 2023-12-19T18:16:51.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f2bf39640 1 -- 172.21.15.26:0/3179584799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f240c4e50 con 0x7f7f240c9e40 2023-12-19T18:16:51.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f2bf39640 1 -- 172.21.15.26:0/3179584799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f240736d0 con 0x7f7f240c9e40 2023-12-19T18:16:51.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.190+0000 7f7f1a7fc640 1 -- 172.21.15.26:0/3179584799 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 118) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7f1400bd30 con 0x7f7f240c9e40 2023-12-19T18:16:51.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.194+0000 7f7f1a7fc640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f08065b80 0x7f7f08068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.194+0000 7f7f1a7fc640 1 -- 172.21.15.26:0/3179584799 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f7f14083f70 con 0x7f7f240c9e40 2023-12-19T18:16:51.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.194+0000 7f7f29cd5640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f08065b80 0x7f7f08068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.194+0000 7f7f29cd5640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f08065b80 0x7f7f08068040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7f20000f70 tx=0x7f7f20009040 comp rx=0 tx=0).ready entity=mgr.9805 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:51.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.198+0000 7f7f1a7fc640 1 -- 172.21.15.26:0/3179584799 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7f14051090 con 0x7f7f240c9e40 2023-12-19T18:16:51.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:51.390+0000 7f7f2bf39640 1 -- 172.21.15.26:0/3179584799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f7f240c5610 con 0x7f7f240c9e40 2023-12-19T18:16:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.098+0000 7f7f1a7fc640 1 -- 172.21.15.26:0/3179584799 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 119) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f7f14051460 con 0x7f7f240c9e40 2023-12-19T18:16:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.102+0000 7f7f1a7fc640 1 -- 172.21.15.26:0/3179584799 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v119) v1 ==== 92+0+0 (secure 0 0 0) 0x7f7f1407e710 con 0x7f7f240c9e40 2023-12-19T18:16:52.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.106+0000 7f7f2bf39640 1 -- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f08065b80 msgr2=0x7f7f08068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.106+0000 7f7f2bf39640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f08065b80 0x7f7f08068040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7f20000f70 tx=0x7f7f20009040 comp rx=0 tx=0).stop 2023-12-19T18:16:52.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.106+0000 7f7f2bf39640 1 -- 172.21.15.26:0/3179584799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f240c9e40 msgr2=0x7f7f240c3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.106+0000 7f7f2bf39640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f240c9e40 0x7f7f240c3880 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400d7d0 tx=0x7f7f1400dc90 comp rx=0 tx=0).stop 2023-12-19T18:16:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.106+0000 7f7f2bf39640 1 -- 172.21.15.26:0/3179584799 shutdown_connections 2023-12-19T18:16:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.106+0000 7f7f2bf39640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f08065b80 0x7f7f08068040 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.106+0000 7f7f2bf39640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7f240ca220 0x7f7f24153450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.106+0000 7f7f2bf39640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f240c9e40 0x7f7f240c3880 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.106+0000 7f7f2bf39640 1 --2- 172.21.15.26:0/3179584799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f2406dbe0 0x7f7f240c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.106+0000 7f7f2bf39640 1 -- 172.21.15.26:0/3179584799 >> 172.21.15.26:0/3179584799 conn(0x7f7f240cf260 msgr2=0x7f7f2415be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.106+0000 7f7f2bf39640 1 -- 172.21.15.26:0/3179584799 shutdown_connections 2023-12-19T18:16:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.106+0000 7f7f2bf39640 1 -- 172.21.15.26:0/3179584799 wait complete. 2023-12-19T18:16:52.131 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-19T18:16:52.282 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:52.277+0000 7f329914f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:16:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.282+0000 7f77c6d39640 1 Processor -- start 2023-12-19T18:16:52.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.282+0000 7f77c6d39640 1 -- start start 2023-12-19T18:16:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c6d39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77c015cde0 0x7f77c00b8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c6d39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77c015d650 0x7f77c00d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c6d39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c00b5bd0 con 0x7f77c015d1c0 2023-12-19T18:16:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c6d39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c015e380 con 0x7f77c015cde0 2023-12-19T18:16:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c6d39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c00b8b40 con 0x7f77c015d650 2023-12-19T18:16:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c4ad5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77c015cde0 0x7f77c00b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c4ad5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77c015cde0 0x7f77c00b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32934/0 (socket says 172.21.15.26:32934) 2023-12-19T18:16:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c4ad5640 1 -- 172.21.15.26:0/4201445740 learned_addr learned my addr 172.21.15.26:0/4201445740 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c52d6640 1 --2- 172.21.15.26:0/4201445740 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77c015d650 0x7f77c00d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c4ad5640 1 -- 172.21.15.26:0/4201445740 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77c015d650 msgr2=0x7f77c00d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c4ad5640 1 --2- 172.21.15.26:0/4201445740 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77c015d650 0x7f77c00d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c4ad5640 1 -- 172.21.15.26:0/4201445740 >> v1:172.21.15.160:6789/0 conn(0x7f77c015d1c0 legacy=0x7f77c00b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c4ad5640 1 -- 172.21.15.26:0/4201445740 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c00d0ee0 con 0x7f77c015cde0 2023-12-19T18:16:52.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f77c52d6640 1 --2- 172.21.15.26:0/4201445740 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77c015d650 0x7f77c00d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:52.289 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:52.286+0000 7f6948e2c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:16:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77c4ad5640 1 --2- 172.21.15.26:0/4201445740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77c015cde0 0x7f77c00b8600 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f77b40098a0 tx=0x7f77b401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173faceb4de4d813 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:52.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77bf7fe640 1 -- 172.21.15.26:0/4201445740 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40a8a10 con 0x7f77c015cde0 2023-12-19T18:16:52.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77bf7fe640 1 -- 172.21.15.26:0/4201445740 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b40a8b70 con 0x7f77c015cde0 2023-12-19T18:16:52.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77bf7fe640 1 -- 172.21.15.26:0/4201445740 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40b0440 con 0x7f77c015cde0 2023-12-19T18:16:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77c6d39640 1 -- 172.21.15.26:0/4201445740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77c015cde0 msgr2=0x7f77c00b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77c6d39640 1 --2- 172.21.15.26:0/4201445740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77c015cde0 0x7f77c00b8600 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f77b40098a0 tx=0x7f77b401ed10 comp rx=0 tx=0).stop 2023-12-19T18:16:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77c6d39640 1 -- 172.21.15.26:0/4201445740 shutdown_connections 2023-12-19T18:16:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77c6d39640 1 --2- 172.21.15.26:0/4201445740 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77c015d650 0x7f77c00d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77c6d39640 1 --2- 172.21.15.26:0/4201445740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77c015cde0 0x7f77c00b8600 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77c6d39640 1 -- 172.21.15.26:0/4201445740 >> 172.21.15.26:0/4201445740 conn(0x7f77c0065ee0 msgr2=0x7f77c00d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77c6d39640 1 -- 172.21.15.26:0/4201445740 shutdown_connections 2023-12-19T18:16:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77c6d39640 1 -- 172.21.15.26:0/4201445740 wait complete. 2023-12-19T18:16:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77c6d39640 1 Processor -- start 2023-12-19T18:16:52.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.290+0000 7f77c6d39640 1 -- start start 2023-12-19T18:16:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c6d39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77c015cde0 0x7f77c015b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c6d39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77c015d1c0 0x7f77c00cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c6d39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77c00cc7b0 0x7f77c01f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c6d39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c015b650 con 0x7f77c015cde0 2023-12-19T18:16:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c6d39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c00cb570 con 0x7f77c00cc7b0 2023-12-19T18:16:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c6d39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c00cb850 con 0x7f77c015d1c0 2023-12-19T18:16:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c4ad5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77c015cde0 0x7f77c015b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c4ad5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77c015cde0 0x7f77c015b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32948/0 (socket says 172.21.15.26:32948) 2023-12-19T18:16:52.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c4ad5640 1 -- 172.21.15.26:0/3845786390 learned_addr learned my addr 172.21.15.26:0/3845786390 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77bffff640 1 --2- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77c015d1c0 0x7f77c00cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c52d6640 1 --2- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77c00cc7b0 0x7f77c01f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c4ad5640 1 -- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77c015d1c0 msgr2=0x7f77c00cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c4ad5640 1 --2- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77c015d1c0 0x7f77c00cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c4ad5640 1 -- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77c00cc7b0 msgr2=0x7f77c01f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c4ad5640 1 --2- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77c00cc7b0 0x7f77c01f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c4ad5640 1 -- 172.21.15.26:0/3845786390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b40094d0 con 0x7f77c015cde0 2023-12-19T18:16:52.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c4ad5640 1 --2- 172.21.15.26:0/3845786390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77c015cde0 0x7f77c015b110 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f77b40b3040 tx=0x7f77b401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77bd7fa640 1 -- 172.21.15.26:0/3845786390 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40be070 con 0x7f77c015cde0 2023-12-19T18:16:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77bd7fa640 1 -- 172.21.15.26:0/3845786390 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b40ba400 con 0x7f77c015cde0 2023-12-19T18:16:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c6d39640 1 -- 172.21.15.26:0/3845786390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77c00cbb30 con 0x7f77c015cde0 2023-12-19T18:16:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77bd7fa640 1 -- 172.21.15.26:0/3845786390 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40ba7d0 con 0x7f77c015cde0 2023-12-19T18:16:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c6d39640 1 -- 172.21.15.26:0/3845786390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c00a9cc0 con 0x7f77c015cde0 2023-12-19T18:16:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77c6d39640 1 -- 172.21.15.26:0/3845786390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c00cbcc0 con 0x7f77c015cde0 2023-12-19T18:16:52.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.294+0000 7f77bd7fa640 1 -- 172.21.15.26:0/3845786390 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 119) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f77b40c3410 con 0x7f77c015cde0 2023-12-19T18:16:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.298+0000 7f77bd7fa640 1 --2- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f77a4065bd0 0x7f77a4068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.298+0000 7f77bd7fa640 1 -- 172.21.15.26:0/3845786390 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f77b41214c0 con 0x7f77c015cde0 2023-12-19T18:16:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.298+0000 7f77bffff640 1 -- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f77a4065bd0 msgr2=0x7f77a4068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:52.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.298+0000 7f77bffff640 1 --2- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f77a4065bd0 0x7f77a4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:16:52.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.302+0000 7f77bd7fa640 1 -- 172.21.15.26:0/3845786390 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f77b40ee7b0 con 0x7f77c015cde0 2023-12-19T18:16:52.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.462+0000 7f77c6d39640 1 -- 172.21.15.26:0/3845786390 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f77c00a9eb0 con 0x7f77c015cde0 2023-12-19T18:16:52.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.470+0000 7f77bd7fa640 1 -- 172.21.15.26:0/3845786390 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v71)=0 v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7f77b40f1dd0 con 0x7f77c015cde0 2023-12-19T18:16:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.470+0000 7f77c6d39640 1 -- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f77a4065bd0 msgr2=0x7f77a4068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:16:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.470+0000 7f77c6d39640 1 --2- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f77a4065bd0 0x7f77a4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.470+0000 7f77c6d39640 1 -- 172.21.15.26:0/3845786390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77c015cde0 msgr2=0x7f77c015b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.470+0000 7f77c6d39640 1 --2- 172.21.15.26:0/3845786390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77c015cde0 0x7f77c015b110 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f77b40b3040 tx=0x7f77b401bab0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.470+0000 7f77c6d39640 1 -- 172.21.15.26:0/3845786390 shutdown_connections 2023-12-19T18:16:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.470+0000 7f77c6d39640 1 --2- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f77a4065bd0 0x7f77a4068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.470+0000 7f77c6d39640 1 --2- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f77c00cc7b0 0x7f77c01f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.470+0000 7f77c6d39640 1 --2- 172.21.15.26:0/3845786390 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f77c015d1c0 0x7f77c00cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.470+0000 7f77c6d39640 1 --2- 172.21.15.26:0/3845786390 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77c015cde0 0x7f77c015b110 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.470+0000 7f77c6d39640 1 -- 172.21.15.26:0/3845786390 >> 172.21.15.26:0/3845786390 conn(0x7f77c0065ee0 msgr2=0x7f77c00b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.470+0000 7f77c6d39640 1 -- 172.21.15.26:0/3845786390 shutdown_connections 2023-12-19T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.470+0000 7f77c6d39640 1 -- 172.21.15.26:0/3845786390 wait complete. 2023-12-19T18:16:52.494 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-19T18:16:52.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.638+0000 7f1990bbc640 1 Processor -- start 2023-12-19T18:16:52.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.638+0000 7f1990bbc640 1 -- start start 2023-12-19T18:16:52.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f1990bbc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f198c15ada0 0x7f198c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f1990bbc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f198c06ac80 0x7f198c06b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f1990bbc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198c070d90 con 0x7f198c15a9c0 2023-12-19T18:16:52.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f1990bbc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198c0cc270 con 0x7f198c06ac80 2023-12-19T18:16:52.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f1990bbc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198c071070 con 0x7f198c15ada0 2023-12-19T18:16:52.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f198ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f198c06ac80 0x7f198c06b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f198a59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f198c15a9c0 0x7f198c0b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44476/0 (socket says 172.21.15.26:44476) 2023-12-19T18:16:52.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f198ad9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f198c06ac80 0x7f198c06b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32960/0 (socket says 172.21.15.26:32960) 2023-12-19T18:16:52.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f198a59c640 1 -- 172.21.15.26:0/1514376113 learned_addr learned my addr 172.21.15.26:0/1514376113 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:52.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f1989d9b640 1 --2- 172.21.15.26:0/1514376113 >> v2:172.21.15.160:3300/0 conn(0x7f198c15ada0 0x7f198c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f198ad9d640 1 -- 172.21.15.26:0/1514376113 >> v2:172.21.15.160:3300/0 conn(0x7f198c15ada0 msgr2=0x7f198c0706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:52.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f198ad9d640 1 --2- 172.21.15.26:0/1514376113 >> v2:172.21.15.160:3300/0 conn(0x7f198c15ada0 0x7f198c0706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f198ad9d640 1 -- 172.21.15.26:0/1514376113 >> v1:172.21.15.26:6789/0 conn(0x7f198c15a9c0 legacy=0x7f198c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f198ad9d640 1 -- 172.21.15.26:0/1514376113 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c071350 con 0x7f198c06ac80 2023-12-19T18:16:52.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f198959a640 1 -- 172.21.15.26:0/1514376113 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928530006 0 0) 0x7f198c070d90 con 0x7f198c15a9c0 2023-12-19T18:16:52.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.642+0000 7f198ad9d640 1 --2- 172.21.15.26:0/1514376113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198c06ac80 0x7f198c06b090 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f198c0cc3b0 tx=0x7f198401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d51774069933c57 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:52.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.646+0000 7f198959a640 1 -- 172.21.15.26:0/1514376113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840a5930 con 0x7f198c06ac80 2023-12-19T18:16:52.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.646+0000 7f198959a640 1 -- 172.21.15.26:0/1514376113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19840a5a90 con 0x7f198c06ac80 2023-12-19T18:16:52.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.646+0000 7f198959a640 1 -- 172.21.15.26:0/1514376113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840a5e20 con 0x7f198c06ac80 2023-12-19T18:16:52.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.646+0000 7f1990bbc640 1 -- 172.21.15.26:0/1514376113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198c06ac80 msgr2=0x7f198c06b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.646+0000 7f1990bbc640 1 --2- 172.21.15.26:0/1514376113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198c06ac80 0x7f198c06b090 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f198c0cc3b0 tx=0x7f198401c850 comp rx=0 tx=0).stop 2023-12-19T18:16:52.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.646+0000 7f1990bbc640 1 -- 172.21.15.26:0/1514376113 shutdown_connections 2023-12-19T18:16:52.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.646+0000 7f1990bbc640 1 --2- 172.21.15.26:0/1514376113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198c06ac80 0x7f198c06b090 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.646+0000 7f1990bbc640 1 --2- 172.21.15.26:0/1514376113 >> v2:172.21.15.160:3300/0 conn(0x7f198c15ada0 0x7f198c0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.646+0000 7f1990bbc640 1 -- 172.21.15.26:0/1514376113 >> 172.21.15.26:0/1514376113 conn(0x7f198c0ce730 msgr2=0x7f198c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:52.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.646+0000 7f1990bbc640 1 -- 172.21.15.26:0/1514376113 shutdown_connections 2023-12-19T18:16:52.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.646+0000 7f1990bbc640 1 -- 172.21.15.26:0/1514376113 wait complete. 2023-12-19T18:16:52.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.646+0000 7f1990bbc640 1 Processor -- start 2023-12-19T18:16:52.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.646+0000 7f1990bbc640 1 -- start start 2023-12-19T18:16:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f1990bbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f198c06ac80 0x7f198c14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f1990bbc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f198c15a9c0 0x7f198c0c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f1990bbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198c15ada0 0x7f198c15b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f1990bbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c1504a0 con 0x7f198c15ada0 2023-12-19T18:16:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f1990bbc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c150ea0 con 0x7f198c15a9c0 2023-12-19T18:16:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f1990bbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c0b1f50 con 0x7f198c06ac80 2023-12-19T18:16:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f198ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198c15ada0 0x7f198c15b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f198a59c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f198c06ac80 0x7f198c14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f1989d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f198c15a9c0 0x7f198c0c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f198ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198c15ada0 0x7f198c15b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32972/0 (socket says 172.21.15.26:32972) 2023-12-19T18:16:52.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f198a59c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f198c06ac80 0x7f198c14ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:40916/0 (socket says 172.21.15.26:40916) 2023-12-19T18:16:52.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f198ad9d640 1 -- 172.21.15.26:0/2717746630 learned_addr learned my addr 172.21.15.26:0/2717746630 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:52.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f198ad9d640 1 -- 172.21.15.26:0/2717746630 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f198c06ac80 msgr2=0x7f198c14ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f198ad9d640 1 --2- 172.21.15.26:0/2717746630 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f198c06ac80 0x7f198c14ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f198ad9d640 1 -- 172.21.15.26:0/2717746630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f198c15a9c0 msgr2=0x7f198c0c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f198ad9d640 1 --2- 172.21.15.26:0/2717746630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f198c15a9c0 0x7f198c0c6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f198ad9d640 1 -- 172.21.15.26:0/2717746630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1984009a70 con 0x7f198c15ada0 2023-12-19T18:16:52.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f1989d9b640 1 --2- 172.21.15.26:0/2717746630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f198c15a9c0 0x7f198c0c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:52.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f198ad9d640 1 --2- 172.21.15.26:0/2717746630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198c15ada0 0x7f198c15b230 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f198401a480 tx=0x7f19840195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:52.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f197affd640 1 -- 172.21.15.26:0/2717746630 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840a5930 con 0x7f198c15ada0 2023-12-19T18:16:52.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f197affd640 1 -- 172.21.15.26:0/2717746630 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1984009450 con 0x7f198c15ada0 2023-12-19T18:16:52.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f1990bbc640 1 -- 172.21.15.26:0/2717746630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198c0c6d60 con 0x7f198c15ada0 2023-12-19T18:16:52.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f197affd640 1 -- 172.21.15.26:0/2717746630 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840b9690 con 0x7f198c15ada0 2023-12-19T18:16:52.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f1990bbc640 1 -- 172.21.15.26:0/2717746630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c0b2480 con 0x7f198c15ada0 2023-12-19T18:16:52.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f1990bbc640 1 -- 172.21.15.26:0/2717746630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f198c074270 con 0x7f198c15ada0 2023-12-19T18:16:52.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.650+0000 7f197affd640 1 -- 172.21.15.26:0/2717746630 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 119) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f19840b97f0 con 0x7f198c15ada0 2023-12-19T18:16:52.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.654+0000 7f197affd640 1 --2- 172.21.15.26:0/2717746630 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f196c065bd0 0x7f196c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.654+0000 7f197affd640 1 -- 172.21.15.26:0/2717746630 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f198411f2c0 con 0x7f198c15ada0 2023-12-19T18:16:52.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.654+0000 7f198a59c640 1 -- 172.21.15.26:0/2717746630 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f196c065bd0 msgr2=0x7f196c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:52.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.654+0000 7f198a59c640 1 --2- 172.21.15.26:0/2717746630 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f196c065bd0 0x7f196c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:16:52.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.658+0000 7f197affd640 1 -- 172.21.15.26:0/2717746630 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f19840ec650 con 0x7f198c15ada0 2023-12-19T18:16:52.698 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:52.693+0000 7f329914f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:16:52.738 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:52.734+0000 7f6948e2c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:16:52.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.818+0000 7f1990bbc640 1 -- 172.21.15.26:0/2717746630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f198c06b390 con 0x7f198c15ada0 2023-12-19T18:16:52.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.822+0000 7f197affd640 1 -- 172.21.15.26:0/2717746630 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v72)=0 v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7f19840efc70 con 0x7f198c15ada0 2023-12-19T18:16:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.826+0000 7f1990bbc640 1 -- 172.21.15.26:0/2717746630 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f196c065bd0 msgr2=0x7f196c068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:16:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.826+0000 7f1990bbc640 1 --2- 172.21.15.26:0/2717746630 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f196c065bd0 0x7f196c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.826+0000 7f1990bbc640 1 -- 172.21.15.26:0/2717746630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198c15ada0 msgr2=0x7f198c15b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.826+0000 7f1990bbc640 1 --2- 172.21.15.26:0/2717746630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198c15ada0 0x7f198c15b230 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f198401a480 tx=0x7f19840195f0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.826+0000 7f1990bbc640 1 -- 172.21.15.26:0/2717746630 shutdown_connections 2023-12-19T18:16:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.826+0000 7f1990bbc640 1 --2- 172.21.15.26:0/2717746630 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f196c065bd0 0x7f196c068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.826+0000 7f1990bbc640 1 --2- 172.21.15.26:0/2717746630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f198c15ada0 0x7f198c15b230 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.826+0000 7f1990bbc640 1 --2- 172.21.15.26:0/2717746630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f198c15a9c0 0x7f198c0c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.826+0000 7f1990bbc640 1 --2- 172.21.15.26:0/2717746630 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f198c06ac80 0x7f198c14ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.826+0000 7f1990bbc640 1 -- 172.21.15.26:0/2717746630 >> 172.21.15.26:0/2717746630 conn(0x7f198c0ce730 msgr2=0x7f198c0ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.826+0000 7f1990bbc640 1 -- 172.21.15.26:0/2717746630 shutdown_connections 2023-12-19T18:16:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:52.826+0000 7f1990bbc640 1 -- 172.21.15.26:0/2717746630 wait complete. 2023-12-19T18:16:52.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:52.849 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-12-19T18:16:52.849 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:52.845+0000 7f329914f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:16:52.863 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-19T18:16:52.900 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:52.898+0000 7f6948e2c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:16:53.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.046+0000 7f2f74ba2640 1 Processor -- start 2023-12-19T18:16:53.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.046+0000 7f2f74ba2640 1 -- start start 2023-12-19T18:16:53.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f74ba2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f7015cc90 0x7f2f700cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f74ba2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2f7015d070 0x7f2f700cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f74ba2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2f70160a20 0x7f2f700cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f74ba2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f700cfe40 con 0x7f2f70160a20 2023-12-19T18:16:53.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f74ba2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f700cdb20 con 0x7f2f7015d070 2023-12-19T18:16:53.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f74ba2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f700cde00 con 0x7f2f7015cc90 2023-12-19T18:16:53.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f6ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2f70160a20 0x7f2f700cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f6ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2f70160a20 0x7f2f700cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32974/0 (socket says 172.21.15.26:32974) 2023-12-19T18:16:53.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f6ffff640 1 -- 172.21.15.26:0/2415587735 learned_addr learned my addr 172.21.15.26:0/2415587735 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:53.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f6effd640 1 --2- 172.21.15.26:0/2415587735 >> v2:172.21.15.160:3300/0 conn(0x7f2f7015d070 0x7f2f700cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f6f7fe640 1 --2- 172.21.15.26:0/2415587735 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f7015cc90 0x7f2f700cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f6ffff640 1 -- 172.21.15.26:0/2415587735 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f7015cc90 msgr2=0x7f2f700cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:53.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f6ffff640 1 --2- 172.21.15.26:0/2415587735 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f7015cc90 0x7f2f700cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f6ffff640 1 -- 172.21.15.26:0/2415587735 >> v2:172.21.15.160:3300/0 conn(0x7f2f7015d070 msgr2=0x7f2f700cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:53.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f6ffff640 1 --2- 172.21.15.26:0/2415587735 >> v2:172.21.15.160:3300/0 conn(0x7f2f7015d070 0x7f2f700cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.050+0000 7f2f6ffff640 1 -- 172.21.15.26:0/2415587735 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f700ce0e0 con 0x7f2f70160a20 2023-12-19T18:16:53.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.054+0000 7f2f6ffff640 1 --2- 172.21.15.26:0/2415587735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f70160a20 0x7f2f700cf900 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2f70071940 tx=0x7f2f6801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cdd457612bf610 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:53.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.054+0000 7f2f6e7fc640 1 -- 172.21.15.26:0/2415587735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680a5820 con 0x7f2f70160a20 2023-12-19T18:16:53.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.054+0000 7f2f6e7fc640 1 -- 172.21.15.26:0/2415587735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f680a5980 con 0x7f2f70160a20 2023-12-19T18:16:53.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.054+0000 7f2f6e7fc640 1 -- 172.21.15.26:0/2415587735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680a5d10 con 0x7f2f70160a20 2023-12-19T18:16:53.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.054+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2415587735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f70160a20 msgr2=0x7f2f700cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.054+0000 7f2f74ba2640 1 --2- 172.21.15.26:0/2415587735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f70160a20 0x7f2f700cf900 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2f70071940 tx=0x7f2f6801c9a0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.054+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2415587735 shutdown_connections 2023-12-19T18:16:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.054+0000 7f2f74ba2640 1 --2- 172.21.15.26:0/2415587735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f70160a20 0x7f2f700cf900 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.054+0000 7f2f74ba2640 1 --2- 172.21.15.26:0/2415587735 >> v2:172.21.15.160:3300/0 conn(0x7f2f7015d070 0x7f2f700cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.054+0000 7f2f74ba2640 1 --2- 172.21.15.26:0/2415587735 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f7015cc90 0x7f2f700cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.054+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2415587735 >> 172.21.15.26:0/2415587735 conn(0x7f2f7006a320 msgr2=0x7f2f700658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.054+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2415587735 shutdown_connections 2023-12-19T18:16:53.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.054+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2415587735 wait complete. 2023-12-19T18:16:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f74ba2640 1 Processor -- start 2023-12-19T18:16:53.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f74ba2640 1 -- start start 2023-12-19T18:16:53.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f74ba2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f7015cc90 0x7f2f7015c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f74ba2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f7015d070 0x7f2f700c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f74ba2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f70160a20 0x7f2f7015f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f74ba2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7015bec0 con 0x7f2f70160a20 2023-12-19T18:16:53.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f74ba2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7015ecc0 con 0x7f2f7015d070 2023-12-19T18:16:53.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f74ba2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f700c5ee0 con 0x7f2f7015cc90 2023-12-19T18:16:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f70160a20 0x7f2f7015f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f6f7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f7015cc90 0x7f2f7015c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f6ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f70160a20 0x7f2f7015f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32990/0 (socket says 172.21.15.26:32990) 2023-12-19T18:16:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f6ffff640 1 -- 172.21.15.26:0/2757014455 learned_addr learned my addr 172.21.15.26:0/2757014455 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f6effd640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f7015d070 0x7f2f700c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f6ffff640 1 -- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f7015cc90 msgr2=0x7f2f7015c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f6ffff640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f7015cc90 0x7f2f7015c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f6ffff640 1 -- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f7015d070 msgr2=0x7f2f700c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f6ffff640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f7015d070 0x7f2f700c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f6ffff640 1 -- 172.21.15.26:0/2757014455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f68009b30 con 0x7f2f70160a20 2023-12-19T18:16:53.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f6f7fe640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f7015cc90 0x7f2f7015c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.058+0000 7f2f6ffff640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f70160a20 0x7f2f7015f430 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f2f6801d240 tx=0x7f2f68019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.062+0000 7f2f57fff640 1 -- 172.21.15.26:0/2757014455 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680be070 con 0x7f2f70160a20 2023-12-19T18:16:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.062+0000 7f2f57fff640 1 -- 172.21.15.26:0/2757014455 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f680b8440 con 0x7f2f70160a20 2023-12-19T18:16:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.062+0000 7f2f57fff640 1 -- 172.21.15.26:0/2757014455 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680b87f0 con 0x7f2f70160a20 2023-12-19T18:16:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.062+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2757014455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7015de10 con 0x7f2f70160a20 2023-12-19T18:16:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.062+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2757014455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7015e390 con 0x7f2f70160a20 2023-12-19T18:16:53.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.062+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2757014455 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7004e850 con 0x7f2f70160a20 2023-12-19T18:16:53.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.062+0000 7f2f57fff640 1 -- 172.21.15.26:0/2757014455 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 119) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f2f680b8950 con 0x7f2f70160a20 2023-12-19T18:16:53.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.062+0000 7f2f57fff640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 0x7f2f5c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.062+0000 7f2f57fff640 1 -- 172.21.15.26:0/2757014455 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f2f6811e350 con 0x7f2f70160a20 2023-12-19T18:16:53.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.066+0000 7f2f6f7fe640 1 -- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 msgr2=0x7f2f5c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:53.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.066+0000 7f2f6f7fe640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 0x7f2f5c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:16:53.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.070+0000 7f2f57fff640 1 -- 172.21.15.26:0/2757014455 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2f680eb5c0 con 0x7f2f70160a20 2023-12-19T18:16:53.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.250+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2757014455 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f700cd7e0 con 0x7f2f5c065bd0 2023-12-19T18:16:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.270+0000 7f2f6f7fe640 1 -- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 msgr2=0x7f2f5c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.270+0000 7f2f6f7fe640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 0x7f2f5c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T18:16:53.329 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:53.325+0000 7f329914f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:16:53.387 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:53.381+0000 7f329914f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:16:53.415 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:53.414+0000 7f6948e2c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:16:53.442 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:53.437+0000 7f329914f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:16:53.477 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:53.474+0000 7f6948e2c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:16:53.528 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:53.525+0000 7f329914f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:16:53.536 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:53.534+0000 7f6948e2c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:16:53.587 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:53.581+0000 7f329914f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:16:53.626 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:53.622+0000 7f6948e2c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:16:53.671 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:53.665+0000 7f329914f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:16:53.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.674+0000 7f2f6f7fe640 1 -- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 msgr2=0x7f2f5c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:53.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:53.674+0000 7f2f6f7fe640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 0x7f2f5c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-19T18:16:53.686 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:53.682+0000 7f6948e2c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:16:53.728 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:53.725+0000 7f329914f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:16:53.775 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:53.774+0000 7f6948e2c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:16:53.834 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:53.830+0000 7f6948e2c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:16:53.870 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:53.865+0000 7f329914f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:16:53.928 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:53.925+0000 7f329914f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:16:53.984 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:53.982+0000 7f6948e2c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:16:54.046 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:54.042+0000 7f6948e2c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:16:54.168 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:16:54.169 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:16:54.169 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:16:54.176 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:54.173+0000 7f329914f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:16:54.296 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:16:54.296 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:16:54.296 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:16:54.304 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:54.302+0000 7f6948e2c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:16:54.310 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:54.309+0000 7f329914f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:16:54.434 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:54.433+0000 7f329914f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:16:54.445 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:54.442+0000 7f6948e2c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:16:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:54.478+0000 7f2f6f7fe640 1 -- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 msgr2=0x7f2f5c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:16:54.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:54.478+0000 7f2f6f7fe640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 0x7f2f5c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-19T18:16:54.503 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:54.505+0000 7f329914f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:16:54.577 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:54.574+0000 7f6948e2c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:16:54.638 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:54.637+0000 7f329914f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:16:54.649 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:54.646+0000 7f6948e2c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:16:54.791 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:54.790+0000 7f6948e2c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:16:55.174 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:55.173+0000 7f329914f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:16:55.258 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:55.257+0000 7f329914f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:16:55.360 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:55.358+0000 7f6948e2c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:16:55.405 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:55.405+0000 7f329914f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:16:55.448 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:55.446+0000 7f6948e2c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:16:55.487 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:55.485+0000 7f329914f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:16:55.603 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:55.602+0000 7f6948e2c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:16:55.691 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:55.686+0000 7f6948e2c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:16:55.801 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:55.801+0000 7f329914f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:16:55.860 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:55.861+0000 7f329914f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:16:55.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:55.870+0000 7f2f57fff640 1 -- 172.21.15.26:0/2757014455 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 120) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f2f6811c210 con 0x7f2f70160a20 2023-12-19T18:16:55.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:55.870+0000 7f2f57fff640 1 -- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 msgr2=0x7f2f5c068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:16:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:55.870+0000 7f2f57fff640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 0x7f2f5c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.043 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:56.038+0000 7f6948e2c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:16:56.045 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.045+0000 7f321b8b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.045+0000 7f321b8b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.045+0000 7f321b8b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.045+0000 7f321b8b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.045+0000 7f321b8b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.045+0000 7f321e8b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.045+0000 7f321e8b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.045+0000 7f321e8b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.045+0000 7f321e8b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.046 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.045+0000 7f321e8b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.058 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.057+0000 7f321b8b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.058 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.057+0000 7f321b8b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.058 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.057+0000 7f321b8b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.058 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.057+0000 7f321b8b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.058 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.057+0000 7f321b8b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.058 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.057+0000 7f321e8b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.059 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.057+0000 7f321e8b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.059 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.057+0000 7f321e8b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.059 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.057+0000 7f321e8b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.059 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:16:56.057+0000 7f321e8b6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:16:56.109 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:16:56.106+0000 7f6948e2c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:16:56.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.877+0000 7f2f57fff640 1 -- 172.21.15.26:0/2757014455 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 121) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f2f680eb990 con 0x7f2f70160a20 2023-12-19T18:16:56.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.877+0000 7f2f57fff640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 0x7f2f5c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.877+0000 7f2f57fff640 1 -- 172.21.15.26:0/2757014455 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f700cd7e0 con 0x7f2f5c065bd0 2023-12-19T18:16:56.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.877+0000 7f2f6f7fe640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 0x7f2f5c068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.881+0000 7f2f6f7fe640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 0x7f2f5c068090 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009a60 tx=0x7f2f60009190 comp rx=0 tx=0).ready entity=mgr.10300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:56.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.941+0000 7f2f57fff640 1 -- 172.21.15.26:0/2757014455 <== mgr.10300 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+53 (secure 0 0 0) 0x7f2f700cd7e0 con 0x7f2f5c065bd0 2023-12-19T18:16:56.947 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path mandatory_fs_argument_test_dir 2023-12-19T18:16:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.945+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 msgr2=0x7f2f5c068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.945+0000 7f2f74ba2640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 0x7f2f5c068090 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009a60 tx=0x7f2f60009190 comp rx=0 tx=0).stop 2023-12-19T18:16:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.945+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2757014455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f70160a20 msgr2=0x7f2f7015f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.945+0000 7f2f74ba2640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f70160a20 0x7f2f7015f430 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f2f6801d240 tx=0x7f2f68019740 comp rx=0 tx=0).stop 2023-12-19T18:16:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.945+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2757014455 shutdown_connections 2023-12-19T18:16:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.945+0000 7f2f74ba2640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f5c065bd0 0x7f2f5c068090 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.945+0000 7f2f74ba2640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f70160a20 0x7f2f7015f430 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.945+0000 7f2f74ba2640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f7015d070 0x7f2f700c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.945+0000 7f2f74ba2640 1 --2- 172.21.15.26:0/2757014455 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f7015cc90 0x7f2f7015c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.945+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2757014455 >> 172.21.15.26:0/2757014455 conn(0x7f2f7006a320 msgr2=0x7f2f700ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.945+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2757014455 shutdown_connections 2023-12-19T18:16:56.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:56.945+0000 7f2f74ba2640 1 -- 172.21.15.26:0/2757014455 wait complete. 2023-12-19T18:16:56.970 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path mandatory_fs_argument_test_dir 2023-12-19T18:16:56.970 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-19T18:16:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3cd1db640 1 Processor -- start 2023-12-19T18:16:57.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3cd1db640 1 -- start start 2023-12-19T18:16:57.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3cd1db640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3c80cea80 0x7fe3c80cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3cd1db640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe3c815c1c0 0x7fe3c815c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3cd1db640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c815bbf0 con 0x7fe3c8158500 2023-12-19T18:16:57.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3cd1db640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c80cf440 con 0x7fe3c815c1c0 2023-12-19T18:16:57.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3cd1db640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c815b2c0 con 0x7fe3c80cea80 2023-12-19T18:16:57.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3c759e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe3c815c1c0 0x7fe3c815c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3c759e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe3c815c1c0 0x7fe3c815c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:50208/0 (socket says 172.21.15.26:50208) 2023-12-19T18:16:57.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3bffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3c80cea80 0x7fe3c80cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3c759e640 1 -- 172.21.15.26:0/1480130232 learned_addr learned my addr 172.21.15.26:0/1480130232 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:57.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3c759e640 1 -- 172.21.15.26:0/1480130232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3c80cea80 msgr2=0x7fe3c80cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:57.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3c759e640 1 --2- 172.21.15.26:0/1480130232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3c80cea80 0x7fe3c80cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3c759e640 1 -- 172.21.15.26:0/1480130232 >> v1:172.21.15.160:6789/0 conn(0x7fe3c8158500 legacy=0x7fe3c806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.117+0000 7fe3c759e640 1 -- 172.21.15.26:0/1480130232 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c815b5a0 con 0x7fe3c815c1c0 2023-12-19T18:16:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.121+0000 7fe3c759e640 1 --2- 172.21.15.26:0/1480130232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3c815c1c0 0x7fe3c815c650 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009e50 tx=0x7fe3c001c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f491f7fd641c34c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.121+0000 7fe3c659c640 1 -- 172.21.15.26:0/1480130232 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c00a56d0 con 0x7fe3c815c1c0 2023-12-19T18:16:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.121+0000 7fe3c659c640 1 -- 172.21.15.26:0/1480130232 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c00a5830 con 0x7fe3c815c1c0 2023-12-19T18:16:57.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.121+0000 7fe3c659c640 1 -- 172.21.15.26:0/1480130232 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c00a5bc0 con 0x7fe3c815c1c0 2023-12-19T18:16:57.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.121+0000 7fe3cd1db640 1 -- 172.21.15.26:0/1480130232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3c815c1c0 msgr2=0x7fe3c815c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.121+0000 7fe3cd1db640 1 --2- 172.21.15.26:0/1480130232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3c815c1c0 0x7fe3c815c650 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009e50 tx=0x7fe3c001c830 comp rx=0 tx=0).stop 2023-12-19T18:16:57.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.121+0000 7fe3cd1db640 1 -- 172.21.15.26:0/1480130232 shutdown_connections 2023-12-19T18:16:57.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.121+0000 7fe3cd1db640 1 --2- 172.21.15.26:0/1480130232 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3c815c1c0 0x7fe3c815c650 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.121+0000 7fe3cd1db640 1 --2- 172.21.15.26:0/1480130232 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3c80cea80 0x7fe3c80cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.121+0000 7fe3cd1db640 1 -- 172.21.15.26:0/1480130232 >> 172.21.15.26:0/1480130232 conn(0x7fe3c80b1aa0 msgr2=0x7fe3c80b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:57.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.121+0000 7fe3cd1db640 1 -- 172.21.15.26:0/1480130232 shutdown_connections 2023-12-19T18:16:57.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.121+0000 7fe3cd1db640 1 -- 172.21.15.26:0/1480130232 wait complete. 2023-12-19T18:16:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3cd1db640 1 Processor -- start 2023-12-19T18:16:57.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3cd1db640 1 -- start start 2023-12-19T18:16:57.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3cd1db640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3c80cea80 0x7fe3c814c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3cd1db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3c8158500 0x7fe3c80c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3cd1db640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3c815c1c0 0x7fe3c80c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3cd1db640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c8152ce0 con 0x7fe3c8158500 2023-12-19T18:16:57.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3cd1db640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c8152fc0 con 0x7fe3c815c1c0 2023-12-19T18:16:57.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3cd1db640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80cc5e0 con 0x7fe3c80cea80 2023-12-19T18:16:57.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3c8158500 0x7fe3c80c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3bffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3c8158500 0x7fe3c80c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38086/0 (socket says 172.21.15.26:38086) 2023-12-19T18:16:57.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3bffff640 1 -- 172.21.15.26:0/2807485588 learned_addr learned my addr 172.21.15.26:0/2807485588 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3c759e640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3c815c1c0 0x7fe3c80c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3c6d9d640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3c80cea80 0x7fe3c814c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3bffff640 1 -- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3c80cea80 msgr2=0x7fe3c814c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3bffff640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3c80cea80 0x7fe3c814c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3bffff640 1 -- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3c815c1c0 msgr2=0x7fe3c80c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3bffff640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3c815c1c0 0x7fe3c80c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3bffff640 1 -- 172.21.15.26:0/2807485588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c0009b30 con 0x7fe3c8158500 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3c759e640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3c815c1c0 0x7fe3c80c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3bffff640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3c8158500 0x7fe3c80c7bc0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000e870 tx=0x7fe3b000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3bf7fe640 1 -- 172.21.15.26:0/2807485588 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0009510 con 0x7fe3c8158500 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3bf7fe640 1 -- 172.21.15.26:0/2807485588 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b000eec0 con 0x7fe3c8158500 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3cd1db640 1 -- 172.21.15.26:0/2807485588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c80cc8c0 con 0x7fe3c8158500 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3bf7fe640 1 -- 172.21.15.26:0/2807485588 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b000cb40 con 0x7fe3c8158500 2023-12-19T18:16:57.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3cd1db640 1 -- 172.21.15.26:0/2807485588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c81520f0 con 0x7fe3c8158500 2023-12-19T18:16:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.125+0000 7fe3cd1db640 1 -- 172.21.15.26:0/2807485588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c80cc420 con 0x7fe3c8158500 2023-12-19T18:16:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.129+0000 7fe3bf7fe640 1 -- 172.21.15.26:0/2807485588 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 121) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fe3b000cca0 con 0x7fe3c8158500 2023-12-19T18:16:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.129+0000 7fe3bf7fe640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe3a8065900 0x7fe3a8067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.129+0000 7fe3bf7fe640 1 -- 172.21.15.26:0/2807485588 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7fe3b00836f0 con 0x7fe3c8158500 2023-12-19T18:16:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.129+0000 7fe3c6d9d640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe3a8065900 0x7fe3a8067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.129+0000 7fe3c6d9d640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe3a8065900 0x7fe3a8067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8007fd0 tx=0x7fe3b8009040 comp rx=0 tx=0).ready entity=mgr.10300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.133+0000 7fe3bf7fe640 1 -- 172.21.15.26:0/2807485588 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe3b0050c20 con 0x7fe3c8158500 2023-12-19T18:16:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.325+0000 7fe3cd1db640 1 -- 172.21.15.26:0/2807485588 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3c806c790 con 0x7fe3a8065900 2023-12-19T18:16:57.378 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.373+0000 7fe3bf7fe640 1 -- 172.21.15.26:0/2807485588 <== mgr.10300 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fe3b80014f0 con 0x7fe3a8065900 2023-12-19T18:16:57.378 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule activated for path mandatory_fs_argument_test_dir 2023-12-19T18:16:57.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.377+0000 7fe3cd1db640 1 -- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe3a8065900 msgr2=0x7fe3a8067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.377+0000 7fe3cd1db640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe3a8065900 0x7fe3a8067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8007fd0 tx=0x7fe3b8009040 comp rx=0 tx=0).stop 2023-12-19T18:16:57.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.377+0000 7fe3cd1db640 1 -- 172.21.15.26:0/2807485588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3c8158500 msgr2=0x7fe3c80c7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.377+0000 7fe3cd1db640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3c8158500 0x7fe3c80c7bc0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000e870 tx=0x7fe3b000ec70 comp rx=0 tx=0).stop 2023-12-19T18:16:57.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.377+0000 7fe3cd1db640 1 -- 172.21.15.26:0/2807485588 shutdown_connections 2023-12-19T18:16:57.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.377+0000 7fe3cd1db640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe3a8065900 0x7fe3a8067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.377+0000 7fe3cd1db640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3c815c1c0 0x7fe3c80c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.377+0000 7fe3cd1db640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe3c8158500 0x7fe3c80c7bc0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.377+0000 7fe3cd1db640 1 --2- 172.21.15.26:0/2807485588 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe3c80cea80 0x7fe3c814c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.377+0000 7fe3cd1db640 1 -- 172.21.15.26:0/2807485588 >> 172.21.15.26:0/2807485588 conn(0x7fe3c80b1aa0 msgr2=0x7fe3c8165010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:57.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.377+0000 7fe3cd1db640 1 -- 172.21.15.26:0/2807485588 shutdown_connections 2023-12-19T18:16:57.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.377+0000 7fe3cd1db640 1 -- 172.21.15.26:0/2807485588 wait complete. 2023-12-19T18:16:57.399 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path mandatory_fs_argument_test_dir 2023-12-19T18:16:57.400 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-19T18:16:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c42b6640 1 Processor -- start 2023-12-19T18:16:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c42b6640 1 -- start start 2023-12-19T18:16:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c42b6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2bc15cde0 0x7fb2bc0b8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c42b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2bc15d650 0x7fb2bc0d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c42b6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc0b5bd0 con 0x7fb2bc15d1c0 2023-12-19T18:16:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c42b6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc15e380 con 0x7fb2bc15cde0 2023-12-19T18:16:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c42b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0b8b40 con 0x7fb2bc15d650 2023-12-19T18:16:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c2052640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2bc15cde0 0x7fb2bc0b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c2052640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2bc15cde0 0x7fb2bc0b8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38102/0 (socket says 172.21.15.26:38102) 2023-12-19T18:16:57.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c2052640 1 -- 172.21.15.26:0/2945252723 learned_addr learned my addr 172.21.15.26:0/2945252723 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:57.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c2853640 1 --2- 172.21.15.26:0/2945252723 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2bc15d650 0x7fb2bc0d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c2052640 1 -- 172.21.15.26:0/2945252723 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2bc15d650 msgr2=0x7fb2bc0d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:57.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c2052640 1 --2- 172.21.15.26:0/2945252723 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2bc15d650 0x7fb2bc0d1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c2052640 1 -- 172.21.15.26:0/2945252723 >> v1:172.21.15.160:6789/0 conn(0x7fb2bc15d1c0 legacy=0x7fb2bc0b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:57.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.545+0000 7fb2c2052640 1 -- 172.21.15.26:0/2945252723 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc0d0ee0 con 0x7fb2bc15cde0 2023-12-19T18:16:57.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.549+0000 7fb2c2052640 1 --2- 172.21.15.26:0/2945252723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2bc15cde0 0x7fb2bc0b8600 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80098a0 tx=0x7fb2b801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da57cc6d8b00909b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.549+0000 7fb2c1050640 1 -- 172.21.15.26:0/2945252723 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80a8a10 con 0x7fb2bc15cde0 2023-12-19T18:16:57.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.549+0000 7fb2c1050640 1 -- 172.21.15.26:0/2945252723 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b80a8b70 con 0x7fb2bc15cde0 2023-12-19T18:16:57.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.549+0000 7fb2c1050640 1 -- 172.21.15.26:0/2945252723 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80b0440 con 0x7fb2bc15cde0 2023-12-19T18:16:57.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.549+0000 7fb2c42b6640 1 -- 172.21.15.26:0/2945252723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2bc15cde0 msgr2=0x7fb2bc0b8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.549+0000 7fb2c42b6640 1 --2- 172.21.15.26:0/2945252723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2bc15cde0 0x7fb2bc0b8600 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80098a0 tx=0x7fb2b801ed10 comp rx=0 tx=0).stop 2023-12-19T18:16:57.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.549+0000 7fb2c42b6640 1 -- 172.21.15.26:0/2945252723 shutdown_connections 2023-12-19T18:16:57.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.549+0000 7fb2c42b6640 1 --2- 172.21.15.26:0/2945252723 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2bc15d650 0x7fb2bc0d1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.549+0000 7fb2c42b6640 1 --2- 172.21.15.26:0/2945252723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2bc15cde0 0x7fb2bc0b8600 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.549+0000 7fb2c42b6640 1 -- 172.21.15.26:0/2945252723 >> 172.21.15.26:0/2945252723 conn(0x7fb2bc065ee0 msgr2=0x7fb2bc0d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:57.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.549+0000 7fb2c42b6640 1 -- 172.21.15.26:0/2945252723 shutdown_connections 2023-12-19T18:16:57.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.549+0000 7fb2c42b6640 1 -- 172.21.15.26:0/2945252723 wait complete. 2023-12-19T18:16:57.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c42b6640 1 Processor -- start 2023-12-19T18:16:57.556 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c42b6640 1 -- start start 2023-12-19T18:16:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c42b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2bc15cde0 0x7fb2bc0cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c42b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2bc15d1c0 0x7fb2bc069f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c42b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2bc151180 0x7fb2bc151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c42b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc151b50 con 0x7fb2bc151180 2023-12-19T18:16:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c42b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0ab0c0 con 0x7fb2bc15d1c0 2023-12-19T18:16:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c42b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0aa9d0 con 0x7fb2bc15cde0 2023-12-19T18:16:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c2052640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2bc15cde0 0x7fb2bc0cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c2853640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2bc151180 0x7fb2bc151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c2052640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2bc15cde0 0x7fb2bc0cc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:54322/0 (socket says 172.21.15.26:54322) 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c2052640 1 -- 172.21.15.26:0/3095378612 learned_addr learned my addr 172.21.15.26:0/3095378612 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c1851640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2bc15d1c0 0x7fb2bc069f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c2052640 1 -- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2bc15d1c0 msgr2=0x7fb2bc069f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c2052640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2bc15d1c0 0x7fb2bc069f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c2052640 1 -- 172.21.15.26:0/3095378612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2bc151180 msgr2=0x7fb2bc151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c2052640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2bc151180 0x7fb2bc151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c2052640 1 -- 172.21.15.26:0/3095378612 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b80094d0 con 0x7fb2bc15cde0 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c2853640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2bc151180 0x7fb2bc151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c2052640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2bc15cde0 0x7fb2bc0cc8b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80b3040 tx=0x7fb2b801bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2b2ffd640 1 -- 172.21.15.26:0/3095378612 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80be070 con 0x7fb2bc15cde0 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2b2ffd640 1 -- 172.21.15.26:0/3095378612 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b80ba400 con 0x7fb2bc15cde0 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2b2ffd640 1 -- 172.21.15.26:0/3095378612 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b80ba7d0 con 0x7fb2bc15cde0 2023-12-19T18:16:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c42b6640 1 -- 172.21.15.26:0/3095378612 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc0a9930 con 0x7fb2bc15cde0 2023-12-19T18:16:57.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.553+0000 7fb2c42b6640 1 -- 172.21.15.26:0/3095378612 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc0a9ef0 con 0x7fb2bc15cde0 2023-12-19T18:16:57.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.557+0000 7fb2c42b6640 1 -- 172.21.15.26:0/3095378612 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc074270 con 0x7fb2bc15cde0 2023-12-19T18:16:57.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.557+0000 7fb2b2ffd640 1 -- 172.21.15.26:0/3095378612 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 121) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fb2b80c3410 con 0x7fb2bc15cde0 2023-12-19T18:16:57.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.557+0000 7fb2b2ffd640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb2a0065900 0x7fb2a0067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.557+0000 7fb2b2ffd640 1 -- 172.21.15.26:0/3095378612 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7fb2b81211b0 con 0x7fb2bc15cde0 2023-12-19T18:16:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.557+0000 7fb2c1851640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb2a0065900 0x7fb2a0067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.557+0000 7fb2c1851640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb2a0065900 0x7fb2a0067dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0b2d90 tx=0x7fb2ac009040 comp rx=0 tx=0).ready entity=mgr.10300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.561+0000 7fb2b2ffd640 1 -- 172.21.15.26:0/3095378612 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb2b80ee6e0 con 0x7fb2bc15cde0 2023-12-19T18:16:57.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.737+0000 7fb2c42b6640 1 -- 172.21.15.26:0/3095378612 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2bc15ddc0 con 0x7fb2a0065900 2023-12-19T18:16:57.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.793+0000 7fb2b2ffd640 1 -- 172.21.15.26:0/3095378612 <== mgr.10300 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7fb2ac0014f0 con 0x7fb2a0065900 2023-12-19T18:16:57.798 INFO:teuthology.orchestra.run.smithi026.stdout:Retention added to path mandatory_fs_argument_test_dir 2023-12-19T18:16:57.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.797+0000 7fb2c42b6640 1 -- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb2a0065900 msgr2=0x7fb2a0067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.797+0000 7fb2c42b6640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb2a0065900 0x7fb2a0067dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0b2d90 tx=0x7fb2ac009040 comp rx=0 tx=0).stop 2023-12-19T18:16:57.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.797+0000 7fb2c42b6640 1 -- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2bc15cde0 msgr2=0x7fb2bc0cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.797+0000 7fb2c42b6640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2bc15cde0 0x7fb2bc0cc8b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80b3040 tx=0x7fb2b801bab0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.797+0000 7fb2c42b6640 1 -- 172.21.15.26:0/3095378612 shutdown_connections 2023-12-19T18:16:57.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.797+0000 7fb2c42b6640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb2a0065900 0x7fb2a0067dc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.797+0000 7fb2c42b6640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2bc151180 0x7fb2bc151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.797+0000 7fb2c42b6640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2bc15d1c0 0x7fb2bc069f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.797+0000 7fb2c42b6640 1 --2- 172.21.15.26:0/3095378612 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2bc15cde0 0x7fb2bc0cc8b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.797+0000 7fb2c42b6640 1 -- 172.21.15.26:0/3095378612 >> 172.21.15.26:0/3095378612 conn(0x7fb2bc065ee0 msgr2=0x7fb2bc0b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:57.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.797+0000 7fb2c42b6640 1 -- 172.21.15.26:0/3095378612 shutdown_connections 2023-12-19T18:16:57.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.797+0000 7fb2c42b6640 1 -- 172.21.15.26:0/3095378612 wait complete. 2023-12-19T18:16:57.819 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path mandatory_fs_argument_test_dir 2023-12-19T18:16:57.819 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-12-19T18:16:57.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.969+0000 7fa1265a2640 1 Processor -- start 2023-12-19T18:16:57.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.969+0000 7fa1265a2640 1 -- start start 2023-12-19T18:16:57.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa1265a2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa1200c1b00 0x7fa1200cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa1265a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1200cb340 0x7fa1200c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa1265a2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1200ccd00 con 0x7fa1200caf60 2023-12-19T18:16:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa1265a2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1200c9c00 con 0x7fa1200c1b00 2023-12-19T18:16:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa1265a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200ccfe0 con 0x7fa1200cb340 2023-12-19T18:16:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa1255a0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa1200caf60 0x7fa1200c13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36460/0 (socket says 172.21.15.26:36460) 2023-12-19T18:16:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa1255a0640 1 -- 172.21.15.26:0/1322445647 learned_addr learned my addr 172.21.15.26:0/1322445647 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:16:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa124d9f640 1 --2- 172.21.15.26:0/1322445647 >> v2:172.21.15.26:3300/0 conn(0x7fa1200c1b00 0x7fa1200cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa125da1640 1 --2- 172.21.15.26:0/1322445647 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1200cb340 0x7fa1200c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa117fff640 1 -- 172.21.15.26:0/1322445647 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959902531 0 0) 0x7fa1200ccd00 con 0x7fa1200caf60 2023-12-19T18:16:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa117fff640 1 -- 172.21.15.26:0/1322445647 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa10c003530 con 0x7fa1200caf60 2023-12-19T18:16:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa124d9f640 1 -- 172.21.15.26:0/1322445647 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1200cb340 msgr2=0x7fa1200c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa124d9f640 1 --2- 172.21.15.26:0/1322445647 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1200cb340 0x7fa1200c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa124d9f640 1 -- 172.21.15.26:0/1322445647 >> v1:172.21.15.26:6789/0 conn(0x7fa1200caf60 legacy=0x7fa1200c13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.973+0000 7fa124d9f640 1 -- 172.21.15.26:0/1322445647 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1200cd2c0 con 0x7fa1200c1b00 2023-12-19T18:16:57.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa124d9f640 1 --2- 172.21.15.26:0/1322445647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1200c1b00 0x7fa1200cc600 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fa1100097c0 tx=0x7fa11001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a1d79170192612 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa117fff640 1 -- 172.21.15.26:0/1322445647 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1100a8970 con 0x7fa1200c1b00 2023-12-19T18:16:57.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa117fff640 1 -- 172.21.15.26:0/1322445647 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1100a8ad0 con 0x7fa1200c1b00 2023-12-19T18:16:57.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa117fff640 1 -- 172.21.15.26:0/1322445647 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1100a8e60 con 0x7fa1200c1b00 2023-12-19T18:16:57.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 -- 172.21.15.26:0/1322445647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1200c1b00 msgr2=0x7fa1200cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 --2- 172.21.15.26:0/1322445647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1200c1b00 0x7fa1200cc600 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fa1100097c0 tx=0x7fa11001ec70 comp rx=0 tx=0).stop 2023-12-19T18:16:57.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 -- 172.21.15.26:0/1322445647 shutdown_connections 2023-12-19T18:16:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 --2- 172.21.15.26:0/1322445647 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1200cb340 0x7fa1200c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 --2- 172.21.15.26:0/1322445647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1200c1b00 0x7fa1200cc600 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 -- 172.21.15.26:0/1322445647 >> 172.21.15.26:0/1322445647 conn(0x7fa120065fc0 msgr2=0x7fa12006b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 -- 172.21.15.26:0/1322445647 shutdown_connections 2023-12-19T18:16:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 -- 172.21.15.26:0/1322445647 wait complete. 2023-12-19T18:16:57.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 Processor -- start 2023-12-19T18:16:57.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 -- start start 2023-12-19T18:16:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1200c1b00 0x7fa1200c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1200caf60 0x7fa12006ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1200cb340 0x7fa1200c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa120071590 con 0x7fa1200caf60 2023-12-19T18:16:57.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa120070980 con 0x7fa1200c1b00 2023-12-19T18:16:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.977+0000 7fa1265a2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa120070c60 con 0x7fa1200cb340 2023-12-19T18:16:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa124d9f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1200caf60 0x7fa12006ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa124d9f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1200caf60 0x7fa12006ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38146/0 (socket says 172.21.15.26:38146) 2023-12-19T18:16:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa124d9f640 1 -- 172.21.15.26:0/2122637631 learned_addr learned my addr 172.21.15.26:0/2122637631 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa1255a0640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1200c1b00 0x7fa1200c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa125da1640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1200cb340 0x7fa1200c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa124d9f640 1 -- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1200cb340 msgr2=0x7fa1200c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa124d9f640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1200cb340 0x7fa1200c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa124d9f640 1 -- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1200c1b00 msgr2=0x7fa1200c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa124d9f640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1200c1b00 0x7fa1200c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa124d9f640 1 -- 172.21.15.26:0/2122637631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa110009420 con 0x7fa1200caf60 2023-12-19T18:16:57.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa125da1640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1200cb340 0x7fa1200c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:57.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa124d9f640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1200caf60 0x7fa12006ff00 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fa1100b3040 tx=0x7fa11001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa115ffb640 1 -- 172.21.15.26:0/2122637631 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1100b0d60 con 0x7fa1200caf60 2023-12-19T18:16:57.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa115ffb640 1 -- 172.21.15.26:0/2122637631 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1100b9430 con 0x7fa1200caf60 2023-12-19T18:16:57.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa115ffb640 1 -- 172.21.15.26:0/2122637631 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1100b97e0 con 0x7fa1200caf60 2023-12-19T18:16:57.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa1265a2640 1 -- 172.21.15.26:0/2122637631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa120070f40 con 0x7fa1200caf60 2023-12-19T18:16:57.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa1265a2640 1 -- 172.21.15.26:0/2122637631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1201f9950 con 0x7fa1200caf60 2023-12-19T18:16:57.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa1265a2640 1 -- 172.21.15.26:0/2122637631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa120071700 con 0x7fa1200caf60 2023-12-19T18:16:57.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.981+0000 7fa115ffb640 1 -- 172.21.15.26:0/2122637631 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 122) v1 ==== 85450+0+0 (secure 0 0 0) 0x7fa1100b7040 con 0x7fa1200caf60 2023-12-19T18:16:57.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.985+0000 7fa115ffb640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa104065cf0 0x7fa1040681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.985+0000 7fa115ffb640 1 -- 172.21.15.26:0/2122637631 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7fa110121970 con 0x7fa1200caf60 2023-12-19T18:16:57.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.985+0000 7fa1255a0640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa104065cf0 0x7fa1040681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.985+0000 7fa1255a0640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa104065cf0 0x7fa1040681b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa11c007fd0 tx=0x7fa11c009040 comp rx=0 tx=0).ready entity=mgr.10300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:57.985+0000 7fa115ffb640 1 -- 172.21.15.26:0/2122637631 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa1100ee780 con 0x7fa1200caf60 2023-12-19T18:16:58.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.165+0000 7fa1265a2640 1 -- 172.21.15.26:0/2122637631 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1200cd880 con 0x7fa104065cf0 2023-12-19T18:16:58.174 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.169+0000 7fa115ffb640 1 -- 172.21.15.26:0/2122637631 <== mgr.10300 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+88 (secure 0 0 0) 0x7fa11c001460 con 0x7fa104065cf0 2023-12-19T18:16:58.174 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:58.174 INFO:teuthology.orchestra.run.smithi026.stdout:{"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-19T18:16:58.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.173+0000 7fa1265a2640 1 -- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa104065cf0 msgr2=0x7fa1040681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.173+0000 7fa1265a2640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa104065cf0 0x7fa1040681b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa11c007fd0 tx=0x7fa11c009040 comp rx=0 tx=0).stop 2023-12-19T18:16:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.173+0000 7fa1265a2640 1 -- 172.21.15.26:0/2122637631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1200caf60 msgr2=0x7fa12006ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.173+0000 7fa1265a2640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1200caf60 0x7fa12006ff00 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fa1100b3040 tx=0x7fa11001ba10 comp rx=0 tx=0).stop 2023-12-19T18:16:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.173+0000 7fa1265a2640 1 -- 172.21.15.26:0/2122637631 shutdown_connections 2023-12-19T18:16:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.173+0000 7fa1265a2640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa104065cf0 0x7fa1040681b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.173+0000 7fa1265a2640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa1200cb340 0x7fa1200c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.173+0000 7fa1265a2640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1200caf60 0x7fa12006ff00 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.173+0000 7fa1265a2640 1 --2- 172.21.15.26:0/2122637631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa1200c1b00 0x7fa1200c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.173+0000 7fa1265a2640 1 -- 172.21.15.26:0/2122637631 >> 172.21.15.26:0/2122637631 conn(0x7fa120065fc0 msgr2=0x7fa1201634d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.173+0000 7fa1265a2640 1 -- 172.21.15.26:0/2122637631 shutdown_connections 2023-12-19T18:16:58.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.173+0000 7fa1265a2640 1 -- 172.21.15.26:0/2122637631 wait complete. 2023-12-19T18:16:58.196 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-19T18:16:58.196 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-12-19T18:16:58.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.337+0000 7fc6a3c41640 1 Processor -- start 2023-12-19T18:16:58.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a3c41640 1 -- start start 2023-12-19T18:16:58.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a3c41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc69c06b9e0 0x7fc69c06bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a3c41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc69c06dd60 0x7fc69c1597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a3c41640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69c06d030 con 0x7fc69c159cf0 2023-12-19T18:16:58.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a3c41640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69c06d8d0 con 0x7fc69c06dd60 2023-12-19T18:16:58.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a3c41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c159570 con 0x7fc69c06b9e0 2023-12-19T18:16:58.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a11dc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc69c06dd60 0x7fc69c1597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a11dc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc69c06dd60 0x7fc69c1597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38152/0 (socket says 172.21.15.26:38152) 2023-12-19T18:16:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a11dc640 1 -- 172.21.15.26:0/2040113392 learned_addr learned my addr 172.21.15.26:0/2040113392 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a19dd640 1 --2- 172.21.15.26:0/2040113392 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc69c06b9e0 0x7fc69c06bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a11dc640 1 -- 172.21.15.26:0/2040113392 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc69c06b9e0 msgr2=0x7fc69c06bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a11dc640 1 --2- 172.21.15.26:0/2040113392 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc69c06b9e0 0x7fc69c06bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a11dc640 1 -- 172.21.15.26:0/2040113392 >> v1:172.21.15.160:6789/0 conn(0x7fc69c159cf0 legacy=0x7fc69c15ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.341+0000 7fc6a11dc640 1 -- 172.21.15.26:0/2040113392 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c06c300 con 0x7fc69c06dd60 2023-12-19T18:16:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.345+0000 7fc6a11dc640 1 --2- 172.21.15.26:0/2040113392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69c06dd60 0x7fc69c1597b0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00aa10 tx=0x7fc68c01d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c118a8b06f76296e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.345+0000 7fc6a09db640 1 -- 172.21.15.26:0/2040113392 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0a6910 con 0x7fc69c06dd60 2023-12-19T18:16:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.345+0000 7fc6a09db640 1 -- 172.21.15.26:0/2040113392 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c0a6a70 con 0x7fc69c06dd60 2023-12-19T18:16:58.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.345+0000 7fc6a09db640 1 -- 172.21.15.26:0/2040113392 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0a6e00 con 0x7fc69c06dd60 2023-12-19T18:16:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.345+0000 7fc6a3c41640 1 -- 172.21.15.26:0/2040113392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69c06dd60 msgr2=0x7fc69c1597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.345+0000 7fc6a3c41640 1 --2- 172.21.15.26:0/2040113392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69c06dd60 0x7fc69c1597b0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00aa10 tx=0x7fc68c01d8e0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.345+0000 7fc6a3c41640 1 -- 172.21.15.26:0/2040113392 shutdown_connections 2023-12-19T18:16:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.345+0000 7fc6a3c41640 1 --2- 172.21.15.26:0/2040113392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69c06dd60 0x7fc69c1597b0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.345+0000 7fc6a3c41640 1 --2- 172.21.15.26:0/2040113392 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc69c06b9e0 0x7fc69c06bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.345+0000 7fc6a3c41640 1 -- 172.21.15.26:0/2040113392 >> 172.21.15.26:0/2040113392 conn(0x7fc69c0cf020 msgr2=0x7fc69c0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:58.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.345+0000 7fc6a3c41640 1 -- 172.21.15.26:0/2040113392 shutdown_connections 2023-12-19T18:16:58.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.345+0000 7fc6a3c41640 1 -- 172.21.15.26:0/2040113392 wait complete. 2023-12-19T18:16:58.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.345+0000 7fc6a3c41640 1 Processor -- start 2023-12-19T18:16:58.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a3c41640 1 -- start start 2023-12-19T18:16:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a3c41640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc69c06b9e0 0x7fc69c158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a3c41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc69c06dd60 0x7fc69c0c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a3c41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69c159cf0 0x7fc69c1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a3c41640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c152690 con 0x7fc69c159cf0 2023-12-19T18:16:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a3c41640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c151c20 con 0x7fc69c06b9e0 2023-12-19T18:16:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a3c41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c1516f0 con 0x7fc69c06dd60 2023-12-19T18:16:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a21de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69c159cf0 0x7fc69c1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a19dd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc69c06b9e0 0x7fc69c158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a21de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69c159cf0 0x7fc69c1511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38154/0 (socket says 172.21.15.26:38154) 2023-12-19T18:16:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a21de640 1 -- 172.21.15.26:0/3312455504 learned_addr learned my addr 172.21.15.26:0/3312455504 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:58.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a11dc640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc69c06dd60 0x7fc69c0c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a21de640 1 -- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc69c06dd60 msgr2=0x7fc69c0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a21de640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc69c06dd60 0x7fc69c0c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a21de640 1 -- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc69c06b9e0 msgr2=0x7fc69c158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a21de640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc69c06b9e0 0x7fc69c158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a21de640 1 -- 172.21.15.26:0/3312455504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c00a670 con 0x7fc69c159cf0 2023-12-19T18:16:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a19dd640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc69c06b9e0 0x7fc69c158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a21de640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69c159cf0 0x7fc69c1511b0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fc69400c730 tx=0x7fc69400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6927fc640 1 -- 172.21.15.26:0/3312455504 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc694003b80 con 0x7fc69c159cf0 2023-12-19T18:16:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6927fc640 1 -- 172.21.15.26:0/3312455504 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc694003ce0 con 0x7fc69c159cf0 2023-12-19T18:16:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6927fc640 1 -- 172.21.15.26:0/3312455504 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69400aa30 con 0x7fc69c159cf0 2023-12-19T18:16:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a3c41640 1 -- 172.21.15.26:0/3312455504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c0c8f30 con 0x7fc69c159cf0 2023-12-19T18:16:58.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a3c41640 1 -- 172.21.15.26:0/3312455504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c1522b0 con 0x7fc69c159cf0 2023-12-19T18:16:58.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.349+0000 7fc6a3c41640 1 -- 172.21.15.26:0/3312455504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69c152110 con 0x7fc69c159cf0 2023-12-19T18:16:58.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.353+0000 7fc6927fc640 1 -- 172.21.15.26:0/3312455504 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 122) v1 ==== 85450+0+0 (secure 0 0 0) 0x7fc69400ab90 con 0x7fc69c159cf0 2023-12-19T18:16:58.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.353+0000 7fc6927fc640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc680065cf0 0x7fc6800681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.353+0000 7fc6927fc640 1 -- 172.21.15.26:0/3312455504 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7fc694083a70 con 0x7fc69c159cf0 2023-12-19T18:16:58.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.353+0000 7fc6a19dd640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc680065cf0 0x7fc6800681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.353+0000 7fc6a19dd640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc680065cf0 0x7fc6800681b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc698009f30 tx=0x7fc698006f90 comp rx=0 tx=0).ready entity=mgr.10300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.357+0000 7fc6927fc640 1 -- 172.21.15.26:0/3312455504 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc69c152110 con 0x7fc69c159cf0 2023-12-19T18:16:58.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.533+0000 7fc6a3c41640 1 -- 172.21.15.26:0/3312455504 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc69c0ce9b0 con 0x7fc680065cf0 2023-12-19T18:16:58.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.541+0000 7fc6927fc640 1 -- 172.21.15.26:0/3312455504 <== mgr.10300 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+338 (secure 0 0 0) 0x7fc698009c60 con 0x7fc680065cf0 2023-12-19T18:16:58.544 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:16:58.545 INFO:teuthology.orchestra.run.smithi026.stdout:[{"fs": "backup_fs", "subvol": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T18:16:56", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-19T18:16:58.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.541+0000 7fc6a3c41640 1 -- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc680065cf0 msgr2=0x7fc6800681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.541+0000 7fc6a3c41640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc680065cf0 0x7fc6800681b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc698009f30 tx=0x7fc698006f90 comp rx=0 tx=0).stop 2023-12-19T18:16:58.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.541+0000 7fc6a3c41640 1 -- 172.21.15.26:0/3312455504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69c159cf0 msgr2=0x7fc69c1511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.541+0000 7fc6a3c41640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69c159cf0 0x7fc69c1511b0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fc69400c730 tx=0x7fc69400cbf0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.541+0000 7fc6a3c41640 1 -- 172.21.15.26:0/3312455504 shutdown_connections 2023-12-19T18:16:58.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.541+0000 7fc6a3c41640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc680065cf0 0x7fc6800681b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.541+0000 7fc6a3c41640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69c159cf0 0x7fc69c1511b0 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.541+0000 7fc6a3c41640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc69c06dd60 0x7fc69c0c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.541+0000 7fc6a3c41640 1 --2- 172.21.15.26:0/3312455504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc69c06b9e0 0x7fc69c158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.541+0000 7fc6a3c41640 1 -- 172.21.15.26:0/3312455504 >> 172.21.15.26:0/3312455504 conn(0x7fc69c0cf020 msgr2=0x7fc69c0cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:58.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.541+0000 7fc6a3c41640 1 -- 172.21.15.26:0/3312455504 shutdown_connections 2023-12-19T18:16:58.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.541+0000 7fc6a3c41640 1 -- 172.21.15.26:0/3312455504 wait complete. 2023-12-19T18:16:58.566 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "backup_fs", "subvol": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-12-19T00:00:00", "created": "2023-12-19T18:16:56", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-19T18:16:58.566 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-19T18:16:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.713+0000 7f078268c640 1 Processor -- start 2023-12-19T18:16:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.713+0000 7f078268c640 1 -- start start 2023-12-19T18:16:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.713+0000 7f078268c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f077c15a9c0 0x7f077c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.713+0000 7f078268c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f077c06ac80 0x7f077c06b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.713+0000 7f078268c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c070d90 con 0x7f077c15ada0 2023-12-19T18:16:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.713+0000 7f078268c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077c0cc270 con 0x7f077c15a9c0 2023-12-19T18:16:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.713+0000 7f078268c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077c071070 con 0x7f077c06ac80 2023-12-19T18:16:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.713+0000 7f077bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f077c15a9c0 0x7f077c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.713+0000 7f077bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f077c15a9c0 0x7f077c0706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38164/0 (socket says 172.21.15.26:38164) 2023-12-19T18:16:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.713+0000 7f077bfff640 1 -- 172.21.15.26:0/3283167850 learned_addr learned my addr 172.21.15.26:0/3283167850 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.713+0000 7f0780c29640 1 --2- 172.21.15.26:0/3283167850 >> v2:172.21.15.160:3300/0 conn(0x7f077c06ac80 0x7f077c06b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f077bfff640 1 -- 172.21.15.26:0/3283167850 >> v2:172.21.15.160:3300/0 conn(0x7f077c06ac80 msgr2=0x7f077c06b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f077bfff640 1 --2- 172.21.15.26:0/3283167850 >> v2:172.21.15.160:3300/0 conn(0x7f077c06ac80 0x7f077c06b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f077bfff640 1 -- 172.21.15.26:0/3283167850 >> v1:172.21.15.160:6789/0 conn(0x7f077c15ada0 legacy=0x7f077c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f077bfff640 1 -- 172.21.15.26:0/3283167850 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c071350 con 0x7f077c15a9c0 2023-12-19T18:16:58.721 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f077bfff640 1 --2- 172.21.15.26:0/3283167850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f077c15a9c0 0x7f077c0706d0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f077c0cc3b0 tx=0x7f076c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b66fd5b9fa9fef server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f077affd640 1 -- 172.21.15.26:0/3283167850 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c0ab660 con 0x7f077c15a9c0 2023-12-19T18:16:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f077affd640 1 -- 172.21.15.26:0/3283167850 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076c0ab7c0 con 0x7f077c15a9c0 2023-12-19T18:16:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f077affd640 1 -- 172.21.15.26:0/3283167850 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076c0abb50 con 0x7f077c15a9c0 2023-12-19T18:16:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f078268c640 1 -- 172.21.15.26:0/3283167850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f077c15a9c0 msgr2=0x7f077c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f078268c640 1 --2- 172.21.15.26:0/3283167850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f077c15a9c0 0x7f077c0706d0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f077c0cc3b0 tx=0x7f076c01c440 comp rx=0 tx=0).stop 2023-12-19T18:16:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f078268c640 1 -- 172.21.15.26:0/3283167850 shutdown_connections 2023-12-19T18:16:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f078268c640 1 --2- 172.21.15.26:0/3283167850 >> v2:172.21.15.160:3300/0 conn(0x7f077c06ac80 0x7f077c06b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f078268c640 1 --2- 172.21.15.26:0/3283167850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f077c15a9c0 0x7f077c0706d0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f078268c640 1 -- 172.21.15.26:0/3283167850 >> 172.21.15.26:0/3283167850 conn(0x7f077c0ce730 msgr2=0x7f077c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.717+0000 7f078268c640 1 -- 172.21.15.26:0/3283167850 shutdown_connections 2023-12-19T18:16:58.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f078268c640 1 -- 172.21.15.26:0/3283167850 wait complete. 2023-12-19T18:16:58.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f078268c640 1 Processor -- start 2023-12-19T18:16:58.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f078268c640 1 -- start start 2023-12-19T18:16:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f078268c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f077c06ac80 0x7f077c0c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f078268c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f077c15a9c0 0x7f077c150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f078268c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f077c15ada0 0x7f077c15d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f078268c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0c6940 con 0x7f077c15ada0 2023-12-19T18:16:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f078268c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0c5cc0 con 0x7f077c15a9c0 2023-12-19T18:16:58.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f078268c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0c5fa0 con 0x7f077c06ac80 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f0780c29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f077c15ada0 0x7f077c15d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f0780c29640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f077c15ada0 0x7f077c15d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38170/0 (socket says 172.21.15.26:38170) 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f0780c29640 1 -- 172.21.15.26:0/64294300 learned_addr learned my addr 172.21.15.26:0/64294300 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f077bfff640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f077c06ac80 0x7f077c0c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f077b7fe640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f077c15a9c0 0x7f077c150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f0780c29640 1 -- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f077c06ac80 msgr2=0x7f077c0c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f0780c29640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f077c06ac80 0x7f077c0c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f0780c29640 1 -- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f077c15a9c0 msgr2=0x7f077c150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f0780c29640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f077c15a9c0 0x7f077c150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f0780c29640 1 -- 172.21.15.26:0/64294300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c0094d0 con 0x7f077c15ada0 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.721+0000 7f0780c29640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f077c15ada0 0x7f077c15d450 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f0774007e40 tx=0x7f077400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.725+0000 7f0778ff9640 1 -- 172.21.15.26:0/64294300 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774014620 con 0x7f077c15ada0 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.725+0000 7f0778ff9640 1 -- 172.21.15.26:0/64294300 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077400ee90 con 0x7f077c15ada0 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.725+0000 7f0778ff9640 1 -- 172.21.15.26:0/64294300 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07740149e0 con 0x7f077c15ada0 2023-12-19T18:16:58.728 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.725+0000 7f078268c640 1 -- 172.21.15.26:0/64294300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c0c6280 con 0x7f077c15ada0 2023-12-19T18:16:58.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.725+0000 7f078268c640 1 -- 172.21.15.26:0/64294300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c1514b0 con 0x7f077c15ada0 2023-12-19T18:16:58.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.725+0000 7f078268c640 1 -- 172.21.15.26:0/64294300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077c0c6a80 con 0x7f077c15ada0 2023-12-19T18:16:58.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.725+0000 7f0778ff9640 1 -- 172.21.15.26:0/64294300 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 122) v1 ==== 85450+0+0 (secure 0 0 0) 0x7f0774014b40 con 0x7f077c15ada0 2023-12-19T18:16:58.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.725+0000 7f0778ff9640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0768065cf0 0x7f07680681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.725+0000 7f0778ff9640 1 -- 172.21.15.26:0/64294300 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7f07740845a0 con 0x7f077c15ada0 2023-12-19T18:16:58.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.725+0000 7f077bfff640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0768065cf0 0x7f07680681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.729+0000 7f077bfff640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0768065cf0 0x7f07680681b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f076c0af000 tx=0x7f076c0b6040 comp rx=0 tx=0).ready entity=mgr.10300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.729+0000 7f0778ff9640 1 -- 172.21.15.26:0/64294300 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f077c0c6a80 con 0x7f077c15ada0 2023-12-19T18:16:58.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.909+0000 7f078268c640 1 -- 172.21.15.26:0/64294300 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f077c0c6110 con 0x7f0768065cf0 2023-12-19T18:16:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.969+0000 7f0778ff9640 1 -- 172.21.15.26:0/64294300 <== mgr.10300 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f077c0c6110 con 0x7f0768065cf0 2023-12-19T18:16:58.974 INFO:teuthology.orchestra.run.smithi026.stdout:Retention removed from path mandatory_fs_argument_test_dir 2023-12-19T18:16:58.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.973+0000 7f078268c640 1 -- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0768065cf0 msgr2=0x7f07680681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.973+0000 7f078268c640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0768065cf0 0x7f07680681b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f076c0af000 tx=0x7f076c0b6040 comp rx=0 tx=0).stop 2023-12-19T18:16:58.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.973+0000 7f078268c640 1 -- 172.21.15.26:0/64294300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f077c15ada0 msgr2=0x7f077c15d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.973+0000 7f078268c640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f077c15ada0 0x7f077c15d450 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f0774007e40 tx=0x7f077400e4b0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.973+0000 7f078268c640 1 -- 172.21.15.26:0/64294300 shutdown_connections 2023-12-19T18:16:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.973+0000 7f078268c640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0768065cf0 0x7f07680681b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.973+0000 7f078268c640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f077c15ada0 0x7f077c15d450 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.973+0000 7f078268c640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f077c15a9c0 0x7f077c150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.973+0000 7f078268c640 1 --2- 172.21.15.26:0/64294300 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f077c06ac80 0x7f077c0c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.973+0000 7f078268c640 1 -- 172.21.15.26:0/64294300 >> 172.21.15.26:0/64294300 conn(0x7f077c0ce730 msgr2=0x7f077c0ae640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.973+0000 7f078268c640 1 -- 172.21.15.26:0/64294300 shutdown_connections 2023-12-19T18:16:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:58.973+0000 7f078268c640 1 -- 172.21.15.26:0/64294300 wait complete. 2023-12-19T18:16:58.997 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention removed from path mandatory_fs_argument_test_dir 2023-12-19T18:16:58.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-19T18:16:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf63055640 1 Processor -- start 2023-12-19T18:16:59.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf63055640 1 -- start start 2023-12-19T18:16:59.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf63055640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf5c15aad0 0x7fbf5c06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf63055640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbf5c0b1590 0x7fbf5c070870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf63055640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5c06cbf0 con 0x7fbf5c0b3f90 2023-12-19T18:16:59.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf63055640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5c0ca8b0 con 0x7fbf5c0b1590 2023-12-19T18:16:59.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf63055640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c06b6f0 con 0x7fbf5c15aad0 2023-12-19T18:16:59.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf5bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbf5c0b1590 0x7fbf5c070870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf5bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbf5c0b1590 0x7fbf5c070870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38176/0 (socket says 172.21.15.26:38176) 2023-12-19T18:16:59.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf5bfff640 1 -- 172.21.15.26:0/2662978406 learned_addr learned my addr 172.21.15.26:0/2662978406 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:59.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf60df1640 1 --2- 172.21.15.26:0/2662978406 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf5c15aad0 0x7fbf5c06c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf5bfff640 1 -- 172.21.15.26:0/2662978406 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf5c15aad0 msgr2=0x7fbf5c06c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:59.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf5bfff640 1 --2- 172.21.15.26:0/2662978406 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf5c15aad0 0x7fbf5c06c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf5bfff640 1 -- 172.21.15.26:0/2662978406 >> v1:172.21.15.26:6789/0 conn(0x7fbf5c0b3f90 legacy=0x7fbf5c0c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf5bfff640 1 -- 172.21.15.26:0/2662978406 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c06b9d0 con 0x7fbf5c0b1590 2023-12-19T18:16:59.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.145+0000 7fbf60df1640 1 --2- 172.21.15.26:0/2662978406 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf5c15aad0 0x7fbf5c06c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:59.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf5bfff640 1 --2- 172.21.15.26:0/2662978406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0b1590 0x7fbf5c070870 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009870 tx=0x7fbf4801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c742c048c1b40b56 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:59.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf5b7fe640 1 -- 172.21.15.26:0/2662978406 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf480aa670 con 0x7fbf5c0b1590 2023-12-19T18:16:59.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf5b7fe640 1 -- 172.21.15.26:0/2662978406 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf480aa7d0 con 0x7fbf5c0b1590 2023-12-19T18:16:59.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf5b7fe640 1 -- 172.21.15.26:0/2662978406 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf480aab60 con 0x7fbf5c0b1590 2023-12-19T18:16:59.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf63055640 1 -- 172.21.15.26:0/2662978406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0b1590 msgr2=0x7fbf5c070870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf63055640 1 --2- 172.21.15.26:0/2662978406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0b1590 0x7fbf5c070870 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009870 tx=0x7fbf4801c510 comp rx=0 tx=0).stop 2023-12-19T18:16:59.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf63055640 1 -- 172.21.15.26:0/2662978406 shutdown_connections 2023-12-19T18:16:59.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf63055640 1 --2- 172.21.15.26:0/2662978406 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0b1590 0x7fbf5c070870 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf63055640 1 --2- 172.21.15.26:0/2662978406 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf5c15aad0 0x7fbf5c06c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf63055640 1 -- 172.21.15.26:0/2662978406 >> 172.21.15.26:0/2662978406 conn(0x7fbf5c0b0270 msgr2=0x7fbf5c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:59.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf63055640 1 -- 172.21.15.26:0/2662978406 shutdown_connections 2023-12-19T18:16:59.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf63055640 1 -- 172.21.15.26:0/2662978406 wait complete. 2023-12-19T18:16:59.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf63055640 1 Processor -- start 2023-12-19T18:16:59.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.149+0000 7fbf63055640 1 -- start start 2023-12-19T18:16:59.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf63055640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf5c0b1590 0x7fbf5c1f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf63055640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0b3f90 0x7fbf5c1520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf63055640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf5c15aad0 0x7fbf5c1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf63055640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c1536a0 con 0x7fbf5c0b3f90 2023-12-19T18:16:59.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf63055640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0c58c0 con 0x7fbf5c15aad0 2023-12-19T18:16:59.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf63055640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c152b20 con 0x7fbf5c0b1590 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0b3f90 0x7fbf5c1520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf5bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0b3f90 0x7fbf5c1520d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38186/0 (socket says 172.21.15.26:38186) 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf5bfff640 1 -- 172.21.15.26:0/979850687 learned_addr learned my addr 172.21.15.26:0/979850687 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf60df1640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf5c0b1590 0x7fbf5c1f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf615f2640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf5c15aad0 0x7fbf5c1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf5bfff640 1 -- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf5c0b1590 msgr2=0x7fbf5c1f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf5bfff640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf5c0b1590 0x7fbf5c1f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf5bfff640 1 -- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf5c15aad0 msgr2=0x7fbf5c1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf5bfff640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf5c15aad0 0x7fbf5c1586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf5bfff640 1 -- 172.21.15.26:0/979850687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf480094d0 con 0x7fbf5c0b3f90 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf5bfff640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0b3f90 0x7fbf5c1520d0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fbf4801abc0 tx=0x7fbf4801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf597fa640 1 -- 172.21.15.26:0/979850687 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf480b3800 con 0x7fbf5c0b3f90 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf597fa640 1 -- 172.21.15.26:0/979850687 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf480b3980 con 0x7fbf5c0b3f90 2023-12-19T18:16:59.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf63055640 1 -- 172.21.15.26:0/979850687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5c152e00 con 0x7fbf5c0b3f90 2023-12-19T18:16:59.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf597fa640 1 -- 172.21.15.26:0/979850687 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf480b3d30 con 0x7fbf5c0b3f90 2023-12-19T18:16:59.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf63055640 1 -- 172.21.15.26:0/979850687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c1533c0 con 0x7fbf5c0b3f90 2023-12-19T18:16:59.161 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.153+0000 7fbf63055640 1 -- 172.21.15.26:0/979850687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5c074270 con 0x7fbf5c0b3f90 2023-12-19T18:16:59.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.157+0000 7fbf597fa640 1 -- 172.21.15.26:0/979850687 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 122) v1 ==== 85450+0+0 (secure 0 0 0) 0x7fbf480b2380 con 0x7fbf5c0b3f90 2023-12-19T18:16:59.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.157+0000 7fbf597fa640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbf40065cf0 0x7fbf400681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.157+0000 7fbf597fa640 1 -- 172.21.15.26:0/979850687 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7fbf4811e410 con 0x7fbf5c0b3f90 2023-12-19T18:16:59.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.157+0000 7fbf60df1640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbf40065cf0 0x7fbf400681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.157+0000 7fbf60df1640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbf40065cf0 0x7fbf400681b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbf50006e90 tx=0x7fbf50008040 comp rx=0 tx=0).ready entity=mgr.10300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:59.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.161+0000 7fbf597fa640 1 -- 172.21.15.26:0/979850687 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbf480eb1c0 con 0x7fbf5c0b3f90 2023-12-19T18:16:59.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.337+0000 7fbf63055640 1 -- 172.21.15.26:0/979850687 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf5c0711c0 con 0x7fbf40065cf0 2023-12-19T18:16:59.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.393+0000 7fbf597fa640 1 -- 172.21.15.26:0/979850687 <== mgr.10300 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+61 (secure 0 0 0) 0x7fbf500108d0 con 0x7fbf40065cf0 2023-12-19T18:16:59.396 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule deactivated for path mandatory_fs_argument_test_dir 2023-12-19T18:16:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.393+0000 7fbf63055640 1 -- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbf40065cf0 msgr2=0x7fbf400681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.393+0000 7fbf63055640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbf40065cf0 0x7fbf400681b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbf50006e90 tx=0x7fbf50008040 comp rx=0 tx=0).stop 2023-12-19T18:16:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.393+0000 7fbf63055640 1 -- 172.21.15.26:0/979850687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0b3f90 msgr2=0x7fbf5c1520d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.393+0000 7fbf63055640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0b3f90 0x7fbf5c1520d0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fbf4801abc0 tx=0x7fbf4801ad50 comp rx=0 tx=0).stop 2023-12-19T18:16:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.393+0000 7fbf63055640 1 -- 172.21.15.26:0/979850687 shutdown_connections 2023-12-19T18:16:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.393+0000 7fbf63055640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbf40065cf0 0x7fbf400681b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.393+0000 7fbf63055640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbf5c15aad0 0x7fbf5c1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.393+0000 7fbf63055640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf5c0b3f90 0x7fbf5c1520d0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.393+0000 7fbf63055640 1 --2- 172.21.15.26:0/979850687 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbf5c0b1590 0x7fbf5c1f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.393+0000 7fbf63055640 1 -- 172.21.15.26:0/979850687 >> 172.21.15.26:0/979850687 conn(0x7fbf5c0b0270 msgr2=0x7fbf5c0b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.393+0000 7fbf63055640 1 -- 172.21.15.26:0/979850687 shutdown_connections 2023-12-19T18:16:59.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.393+0000 7fbf63055640 1 -- 172.21.15.26:0/979850687 wait complete. 2023-12-19T18:16:59.418 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path mandatory_fs_argument_test_dir 2023-12-19T18:16:59.418 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-19T18:16:59.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.561+0000 7fc85c1b3640 1 Processor -- start 2023-12-19T18:16:59.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.561+0000 7fc85c1b3640 1 -- start start 2023-12-19T18:16:59.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.565+0000 7fc85c1b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc854159830 0x7fc85415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.565+0000 7fc85c1b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc8540cb110 0x7fc8540cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.565+0000 7fc85c1b3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85415e3d0 con 0x7fc8540c9310 2023-12-19T18:16:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.565+0000 7fc85c1b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85406f7c0 con 0x7fc8540cb110 2023-12-19T18:16:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.565+0000 7fc85c1b3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8540ca2b0 con 0x7fc854159830 2023-12-19T18:16:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.565+0000 7fc85a750640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc8540cb110 0x7fc8540cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.565+0000 7fc85a750640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc8540cb110 0x7fc8540cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38200/0 (socket says 172.21.15.26:38200) 2023-12-19T18:16:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.565+0000 7fc85a750640 1 -- 172.21.15.26:0/1521842863 learned_addr learned my addr 172.21.15.26:0/1521842863 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.565+0000 7fc859f4f640 1 --2- 172.21.15.26:0/1521842863 >> v2:172.21.15.160:3300/0 conn(0x7fc854159830 0x7fc85415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.565+0000 7fc85a750640 1 -- 172.21.15.26:0/1521842863 >> v2:172.21.15.160:3300/0 conn(0x7fc854159830 msgr2=0x7fc85415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.565+0000 7fc85a750640 1 --2- 172.21.15.26:0/1521842863 >> v2:172.21.15.160:3300/0 conn(0x7fc854159830 0x7fc85415d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.565+0000 7fc85a750640 1 -- 172.21.15.26:0/1521842863 >> v1:172.21.15.160:6789/0 conn(0x7fc8540c9310 legacy=0x7fc8540cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.565+0000 7fc85a750640 1 -- 172.21.15.26:0/1521842863 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8540cba80 con 0x7fc8540cb110 2023-12-19T18:16:59.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc85a750640 1 --2- 172.21.15.26:0/1521842863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8540cb110 0x7fc8540cb540 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fc85406ce60 tx=0x7fc84c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d2c7451ee73937 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:59.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc858f4d640 1 -- 172.21.15.26:0/1521842863 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0a56f0 con 0x7fc8540cb110 2023-12-19T18:16:59.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc858f4d640 1 -- 172.21.15.26:0/1521842863 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c0a5850 con 0x7fc8540cb110 2023-12-19T18:16:59.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc858f4d640 1 -- 172.21.15.26:0/1521842863 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0a5be0 con 0x7fc8540cb110 2023-12-19T18:16:59.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1521842863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8540cb110 msgr2=0x7fc8540cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc85c1b3640 1 --2- 172.21.15.26:0/1521842863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8540cb110 0x7fc8540cb540 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fc85406ce60 tx=0x7fc84c01c940 comp rx=0 tx=0).stop 2023-12-19T18:16:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1521842863 shutdown_connections 2023-12-19T18:16:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc85c1b3640 1 --2- 172.21.15.26:0/1521842863 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8540cb110 0x7fc8540cb540 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc85c1b3640 1 --2- 172.21.15.26:0/1521842863 >> v2:172.21.15.160:3300/0 conn(0x7fc854159830 0x7fc85415d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1521842863 >> 172.21.15.26:0/1521842863 conn(0x7fc8540b0270 msgr2=0x7fc8540b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1521842863 shutdown_connections 2023-12-19T18:16:59.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1521842863 wait complete. 2023-12-19T18:16:59.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc85c1b3640 1 Processor -- start 2023-12-19T18:16:59.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.569+0000 7fc85c1b3640 1 -- start start 2023-12-19T18:16:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc85c1b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8540c9310 0x7fc85414fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc85c1b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8540cb110 0x7fc8541464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc85c1b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc854159830 0x7fc8540ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc85c1b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85414ebe0 con 0x7fc8540c9310 2023-12-19T18:16:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc85c1b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85414eec0 con 0x7fc854159830 2023-12-19T18:16:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc85c1b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85414f1a0 con 0x7fc8540cb110 2023-12-19T18:16:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc859f4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8540c9310 0x7fc85414fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc859f4f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8540c9310 0x7fc85414fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38214/0 (socket says 172.21.15.26:38214) 2023-12-19T18:16:59.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc859f4f640 1 -- 172.21.15.26:0/1107389171 learned_addr learned my addr 172.21.15.26:0/1107389171 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:16:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc85a750640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc854159830 0x7fc8540ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc85974e640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8540cb110 0x7fc8541464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc859f4f640 1 -- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8540cb110 msgr2=0x7fc8541464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc859f4f640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8540cb110 0x7fc8541464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc859f4f640 1 -- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc854159830 msgr2=0x7fc8540ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc859f4f640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc854159830 0x7fc8540ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc859f4f640 1 -- 172.21.15.26:0/1107389171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c009b30 con 0x7fc8540c9310 2023-12-19T18:16:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc859f4f640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8540c9310 0x7fc85414fc80 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fc85000e870 tx=0x7fc85000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc84a7fc640 1 -- 172.21.15.26:0/1107389171 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850009510 con 0x7fc8540c9310 2023-12-19T18:16:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc84a7fc640 1 -- 172.21.15.26:0/1107389171 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85000eec0 con 0x7fc8540c9310 2023-12-19T18:16:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1107389171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85414f480 con 0x7fc8540c9310 2023-12-19T18:16:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc84a7fc640 1 -- 172.21.15.26:0/1107389171 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85000cb40 con 0x7fc8540c9310 2023-12-19T18:16:59.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1107389171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8541469f0 con 0x7fc8540c9310 2023-12-19T18:16:59.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1107389171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8540cf640 con 0x7fc8540c9310 2023-12-19T18:16:59.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.573+0000 7fc84a7fc640 1 -- 172.21.15.26:0/1107389171 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 122) v1 ==== 85450+0+0 (secure 0 0 0) 0x7fc85000cca0 con 0x7fc8540c9310 2023-12-19T18:16:59.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.577+0000 7fc84a7fc640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc838065cf0 0x7fc8380681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.577+0000 7fc84a7fc640 1 -- 172.21.15.26:0/1107389171 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7fc850083f40 con 0x7fc8540c9310 2023-12-19T18:16:59.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.577+0000 7fc85974e640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc838065cf0 0x7fc8380681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.577+0000 7fc85974e640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc838065cf0 0x7fc8380681b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc8540ad3b0 tx=0x7fc844009040 comp rx=0 tx=0).ready entity=mgr.10300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:59.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.581+0000 7fc84a7fc640 1 -- 172.21.15.26:0/1107389171 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc850050e00 con 0x7fc8540c9310 2023-12-19T18:16:59.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.757+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1107389171 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc85414f030 con 0x7fc838065cf0 2023-12-19T18:16:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.813+0000 7fc84a7fc640 1 -- 172.21.15.26:0/1107389171 <== mgr.10300 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7fc85414f030 con 0x7fc838065cf0 2023-12-19T18:16:59.817 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path mandatory_fs_argument_test_dir 2023-12-19T18:16:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.817+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc838065cf0 msgr2=0x7fc8380681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.817+0000 7fc85c1b3640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc838065cf0 0x7fc8380681b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc8540ad3b0 tx=0x7fc844009040 comp rx=0 tx=0).stop 2023-12-19T18:16:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.817+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1107389171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8540c9310 msgr2=0x7fc85414fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.817+0000 7fc85c1b3640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8540c9310 0x7fc85414fc80 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fc85000e870 tx=0x7fc85000ec70 comp rx=0 tx=0).stop 2023-12-19T18:16:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.817+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1107389171 shutdown_connections 2023-12-19T18:16:59.819 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.817+0000 7fc85c1b3640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc838065cf0 0x7fc8380681b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.817+0000 7fc85c1b3640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc854159830 0x7fc8540ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.817+0000 7fc85c1b3640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8540cb110 0x7fc8541464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.817+0000 7fc85c1b3640 1 --2- 172.21.15.26:0/1107389171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8540c9310 0x7fc85414fc80 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.817+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1107389171 >> 172.21.15.26:0/1107389171 conn(0x7fc8540b0270 msgr2=0x7fc8540ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:59.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.817+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1107389171 shutdown_connections 2023-12-19T18:16:59.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:16:59.817+0000 7fc85c1b3640 1 -- 172.21.15.26:0/1107389171 wait complete. 2023-12-19T18:16:59.841 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path mandatory_fs_argument_test_dir 2023-12-19T18:16:59.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:59.841 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-12-19T18:16:59.852 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-19T18:17:00.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb0a158f640 1 Processor -- start 2023-12-19T18:17:00.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb0a158f640 1 -- start start 2023-12-19T18:17:00.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb0a158f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb09c0cb860 0x7fb09c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb0a158f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09c15e240 con 0x7fb09c1599e0 2023-12-19T18:17:00.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb0a158f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09c0c9520 con 0x7fb09c15d530 2023-12-19T18:17:00.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb0a158f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c0c9800 con 0x7fb09c0cb860 2023-12-19T18:17:00.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb09affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb09c1599e0 0x7fb09c0cb150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36488/0 (socket says 172.21.15.26:36488) 2023-12-19T18:17:00.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb09affd640 1 -- 172.21.15.26:0/4201387980 learned_addr learned my addr 172.21.15.26:0/4201387980 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:00.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb09a7fc640 1 --2- 172.21.15.26:0/4201387980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb09c0cb860 0x7fb09c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb099ffb640 1 -- 172.21.15.26:0/4201387980 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080233659 0 0) 0x7fb09c15e240 con 0x7fb09c1599e0 2023-12-19T18:17:00.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb099ffb640 1 -- 172.21.15.26:0/4201387980 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb084003530 con 0x7fb09c1599e0 2023-12-19T18:17:00.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb099ffb640 1 -- 172.21.15.26:0/4201387980 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554641700 0 0) 0x7fb09c0c9520 con 0x7fb09c15d530 2023-12-19T18:17:00.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb099ffb640 1 -- 172.21.15.26:0/4201387980 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb09c15e240 con 0x7fb09c15d530 2023-12-19T18:17:00.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb09a7fc640 1 -- 172.21.15.26:0/4201387980 >> v1:172.21.15.160:6789/0 conn(0x7fb09c15d530 legacy=0x7fb09c15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb09a7fc640 1 -- 172.21.15.26:0/4201387980 >> v1:172.21.15.26:6789/0 conn(0x7fb09c1599e0 legacy=0x7fb09c0cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb099ffb640 1 -- 172.21.15.26:0/4201387980 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3184987409 0 0) 0x7fb084003530 con 0x7fb09c1599e0 2023-12-19T18:17:00.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.037+0000 7fb09a7fc640 1 -- 172.21.15.26:0/4201387980 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09c0c9ae0 con 0x7fb09c0cb860 2023-12-19T18:17:00.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.041+0000 7fb09a7fc640 1 --2- 172.21.15.26:0/4201387980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb09c0cb860 0x7fb09c06f6d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb0900097f0 tx=0x7fb09001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1468132ca7e5a6f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.041+0000 7fb099ffb640 1 -- 172.21.15.26:0/4201387980 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c001680 con 0x7fb09c0cb860 2023-12-19T18:17:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.041+0000 7fb099ffb640 1 -- 172.21.15.26:0/4201387980 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0900a7050 con 0x7fb09c0cb860 2023-12-19T18:17:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.041+0000 7fb099ffb640 1 -- 172.21.15.26:0/4201387980 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0900ab620 con 0x7fb09c0cb860 2023-12-19T18:17:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.041+0000 7fb0a158f640 1 -- 172.21.15.26:0/4201387980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb09c0cb860 msgr2=0x7fb09c06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.041+0000 7fb0a158f640 1 --2- 172.21.15.26:0/4201387980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb09c0cb860 0x7fb09c06f6d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb0900097f0 tx=0x7fb09001c3e0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.041+0000 7fb0a158f640 1 -- 172.21.15.26:0/4201387980 shutdown_connections 2023-12-19T18:17:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.041+0000 7fb0a158f640 1 --2- 172.21.15.26:0/4201387980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb09c0cb860 0x7fb09c06f6d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.041+0000 7fb0a158f640 1 -- 172.21.15.26:0/4201387980 >> 172.21.15.26:0/4201387980 conn(0x7fb09c066160 msgr2=0x7fb09c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.041+0000 7fb0a158f640 1 -- 172.21.15.26:0/4201387980 shutdown_connections 2023-12-19T18:17:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.041+0000 7fb0a158f640 1 -- 172.21.15.26:0/4201387980 wait complete. 2023-12-19T18:17:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb0a158f640 1 Processor -- start 2023-12-19T18:17:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb0a158f640 1 -- start start 2023-12-19T18:17:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb0a158f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb09c0cb860 0x7fb09c0acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb0a158f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb09c1599e0 0x7fb09c152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb0a158f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb09c15d530 0x7fb09c147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb0a158f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c154200 con 0x7fb09c15d530 2023-12-19T18:17:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb0a158f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c0afd40 con 0x7fb09c1599e0 2023-12-19T18:17:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb0a158f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c0b0020 con 0x7fb09c0cb860 2023-12-19T18:17:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb09b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb09c15d530 0x7fb09c147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb09a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb09c1599e0 0x7fb09c152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb09b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb09c15d530 0x7fb09c147d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38220/0 (socket says 172.21.15.26:38220) 2023-12-19T18:17:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb09b7fe640 1 -- 172.21.15.26:0/4209411513 learned_addr learned my addr 172.21.15.26:0/4209411513 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb09affd640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb09c0cb860 0x7fb09c0acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb09b7fe640 1 -- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb09c0cb860 msgr2=0x7fb09c0acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb09b7fe640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb09c0cb860 0x7fb09c0acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb09b7fe640 1 -- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb09c1599e0 msgr2=0x7fb09c152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb09b7fe640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb09c1599e0 0x7fb09c152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb09b7fe640 1 -- 172.21.15.26:0/4209411513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090009420 con 0x7fb09c15d530 2023-12-19T18:17:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb09a7fc640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb09c1599e0 0x7fb09c152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb09b7fe640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb09c15d530 0x7fb09c147d00 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fb09400c730 tx=0x7fb09400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb0837fe640 1 -- 172.21.15.26:0/4209411513 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb094003b80 con 0x7fb09c15d530 2023-12-19T18:17:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb0837fe640 1 -- 172.21.15.26:0/4209411513 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb094003ce0 con 0x7fb09c15d530 2023-12-19T18:17:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb0a158f640 1 -- 172.21.15.26:0/4209411513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09c153eb0 con 0x7fb09c15d530 2023-12-19T18:17:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb0837fe640 1 -- 172.21.15.26:0/4209411513 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09400aa90 con 0x7fb09c15d530 2023-12-19T18:17:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.045+0000 7fb0a158f640 1 -- 172.21.15.26:0/4209411513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09c148a40 con 0x7fb09c15d530 2023-12-19T18:17:00.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.049+0000 7fb0a158f640 1 -- 172.21.15.26:0/4209411513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09c0afba0 con 0x7fb09c15d530 2023-12-19T18:17:00.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.049+0000 7fb0837fe640 1 -- 172.21.15.26:0/4209411513 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 122) v1 ==== 85450+0+0 (secure 0 0 0) 0x7fb09400abf0 con 0x7fb09c15d530 2023-12-19T18:17:00.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.049+0000 7fb0837fe640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb088065cf0 0x7fb0880681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.049+0000 7fb0837fe640 1 -- 172.21.15.26:0/4209411513 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7fb094083b80 con 0x7fb09c15d530 2023-12-19T18:17:00.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.049+0000 7fb09affd640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb088065cf0 0x7fb0880681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.049+0000 7fb09affd640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb088065cf0 0x7fb0880681b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb08c001ec0 tx=0x7fb08c001a60 comp rx=0 tx=0).ready entity=mgr.10300 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:00.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.053+0000 7fb0837fe640 1 -- 172.21.15.26:0/4209411513 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb09c0afba0 con 0x7fb09c15d530 2023-12-19T18:17:00.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.089+0000 7fb0837fe640 1 -- 172.21.15.26:0/4209411513 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 123) v1 ==== 85506+0+0 (secure 0 0 0) 0x7fb094050e10 con 0x7fb09c15d530 2023-12-19T18:17:00.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:00.253+0000 7fb0a158f640 1 -- 172.21.15.26:0/4209411513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb09c0afba0 con 0x7fb09c15d530 2023-12-19T18:17:01.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.101+0000 7fb0837fe640 1 -- 172.21.15.26:0/4209411513 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 124) v1 ==== 85489+0+0 (secure 0 0 0) 0x7fb09405a260 con 0x7fb09c15d530 2023-12-19T18:17:01.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.101+0000 7fb0837fe640 1 -- 172.21.15.26:0/4209411513 <== mon.0 v2:172.21.15.26:3300/0 9 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v124) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb094051a50 con 0x7fb09c15d530 2023-12-19T18:17:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.105+0000 7fb0a158f640 1 -- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb088065cf0 msgr2=0x7fb0880681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.105+0000 7fb0a158f640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb088065cf0 0x7fb0880681b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb08c001ec0 tx=0x7fb08c001a60 comp rx=0 tx=0).stop 2023-12-19T18:17:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.105+0000 7fb0a158f640 1 -- 172.21.15.26:0/4209411513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb09c15d530 msgr2=0x7fb09c147d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.105+0000 7fb0a158f640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb09c15d530 0x7fb09c147d00 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fb09400c730 tx=0x7fb09400cbf0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.105+0000 7fb0a158f640 1 -- 172.21.15.26:0/4209411513 shutdown_connections 2023-12-19T18:17:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.105+0000 7fb0a158f640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb088065cf0 0x7fb0880681b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.105+0000 7fb0a158f640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb09c15d530 0x7fb09c147d00 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.105+0000 7fb0a158f640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb09c1599e0 0x7fb09c152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.105+0000 7fb0a158f640 1 --2- 172.21.15.26:0/4209411513 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb09c0cb860 0x7fb09c0acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.105+0000 7fb0a158f640 1 -- 172.21.15.26:0/4209411513 >> 172.21.15.26:0/4209411513 conn(0x7fb09c066160 msgr2=0x7fb09c0ca000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.105+0000 7fb0a158f640 1 -- 172.21.15.26:0/4209411513 shutdown_connections 2023-12-19T18:17:01.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.105+0000 7fb0a158f640 1 -- 172.21.15.26:0/4209411513 wait complete. 2023-12-19T18:17:01.134 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:17:01.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:17:01.199 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:17:01.259 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:17:01.259 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:17:01.259 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:01.269 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:17:01.269 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:01.282 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:01.281+0000 7f8fe7b1a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:17:01.289 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:01.285+0000 7f9c5762c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:17:01.298 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:01.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.477+0000 7f064ab39640 1 Processor -- start 2023-12-19T18:17:01.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.477+0000 7f064ab39640 1 -- start start 2023-12-19T18:17:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.481+0000 7f064ab39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f064415cf60 0x7f06440d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.481+0000 7f064ab39640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06440d2960 con 0x7f06440b4d10 2023-12-19T18:17:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.481+0000 7f064ab39640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06440d18c0 con 0x7f06440d1420 2023-12-19T18:17:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.481+0000 7f064ab39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06440d1ba0 con 0x7f064415cf60 2023-12-19T18:17:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.481+0000 7f06488d5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f064415cf60 0x7f06440d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.481+0000 7f06490d6640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f06440b4d10 0x7f06440b5cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36500/0 (socket says 172.21.15.26:36500) 2023-12-19T18:17:01.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.481+0000 7f06490d6640 1 -- 172.21.15.26:0/1973446892 learned_addr learned my addr 172.21.15.26:0/1973446892 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.481+0000 7f06437fe640 1 -- 172.21.15.26:0/1973446892 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2095691818 0 0) 0x7f06440d2960 con 0x7f06440b4d10 2023-12-19T18:17:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.481+0000 7f06488d5640 1 -- 172.21.15.26:0/1973446892 >> v1:172.21.15.160:6789/0 conn(0x7f06440d1420 legacy=0x7f06440b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.481+0000 7f06488d5640 1 -- 172.21.15.26:0/1973446892 >> v1:172.21.15.26:6789/0 conn(0x7f06440b4d10 legacy=0x7f06440b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.481+0000 7f06488d5640 1 -- 172.21.15.26:0/1973446892 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06440d1e80 con 0x7f064415cf60 2023-12-19T18:17:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.485+0000 7f06488d5640 1 --2- 172.21.15.26:0/1973446892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f064415cf60 0x7f06440d0ee0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f06380097c0 tx=0x7f063801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b0b02ad739496f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.485+0000 7f06437fe640 1 -- 172.21.15.26:0/1973446892 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06380a8970 con 0x7f064415cf60 2023-12-19T18:17:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.485+0000 7f06437fe640 1 -- 172.21.15.26:0/1973446892 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06380a8ad0 con 0x7f064415cf60 2023-12-19T18:17:01.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.485+0000 7f06437fe640 1 -- 172.21.15.26:0/1973446892 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06380a8e60 con 0x7f064415cf60 2023-12-19T18:17:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.485+0000 7f064ab39640 1 -- 172.21.15.26:0/1973446892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f064415cf60 msgr2=0x7f06440d0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.485+0000 7f064ab39640 1 --2- 172.21.15.26:0/1973446892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f064415cf60 0x7f06440d0ee0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f06380097c0 tx=0x7f063801ec70 comp rx=0 tx=0).stop 2023-12-19T18:17:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.485+0000 7f064ab39640 1 -- 172.21.15.26:0/1973446892 shutdown_connections 2023-12-19T18:17:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.485+0000 7f064ab39640 1 --2- 172.21.15.26:0/1973446892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f064415cf60 0x7f06440d0ee0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.485+0000 7f064ab39640 1 -- 172.21.15.26:0/1973446892 >> 172.21.15.26:0/1973446892 conn(0x7f0644065ee0 msgr2=0x7f06440cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.485+0000 7f064ab39640 1 -- 172.21.15.26:0/1973446892 shutdown_connections 2023-12-19T18:17:01.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.485+0000 7f064ab39640 1 -- 172.21.15.26:0/1973446892 wait complete. 2023-12-19T18:17:01.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.485+0000 7f064ab39640 1 Processor -- start 2023-12-19T18:17:01.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.485+0000 7f064ab39640 1 -- start start 2023-12-19T18:17:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f064ab39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06440b4d10 0x7f06441f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f064ab39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06440d1420 0x7f06441562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f064ab39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f064415cf60 0x7f06440b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f064ab39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0644156800 con 0x7f064415cf60 2023-12-19T18:17:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f064ab39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06440b3cc0 con 0x7f06440b4d10 2023-12-19T18:17:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f064ab39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06440b3f90 con 0x7f06440d1420 2023-12-19T18:17:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f06490d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f064415cf60 0x7f06440b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f06490d6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f064415cf60 0x7f06440b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38250/0 (socket says 172.21.15.26:38250) 2023-12-19T18:17:01.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f06488d5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06440b4d10 0x7f06441f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:01.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f06490d6640 1 -- 172.21.15.26:0/2083127571 learned_addr learned my addr 172.21.15.26:0/2083127571 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:01.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f0643fff640 1 --2- 172.21.15.26:0/2083127571 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06440d1420 0x7f06441562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:01.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f06490d6640 1 -- 172.21.15.26:0/2083127571 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06440d1420 msgr2=0x7f06441562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f06490d6640 1 --2- 172.21.15.26:0/2083127571 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06440d1420 0x7f06441562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f06490d6640 1 -- 172.21.15.26:0/2083127571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06440b4d10 msgr2=0x7f06441f3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f06490d6640 1 --2- 172.21.15.26:0/2083127571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06440b4d10 0x7f06441f3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f06490d6640 1 -- 172.21.15.26:0/2083127571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638009420 con 0x7f064415cf60 2023-12-19T18:17:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f06490d6640 1 --2- 172.21.15.26:0/2083127571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f064415cf60 0x7f06440b3780 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c7a0 tx=0x7f063c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f06417fa640 1 -- 172.21.15.26:0/2083127571 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c00a640 con 0x7f064415cf60 2023-12-19T18:17:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f06417fa640 1 -- 172.21.15.26:0/2083127571 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063c00a7a0 con 0x7f064415cf60 2023-12-19T18:17:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f06417fa640 1 -- 172.21.15.26:0/2083127571 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c00abd0 con 0x7f064415cf60 2023-12-19T18:17:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f064ab39640 1 -- 172.21.15.26:0/2083127571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06440b4270 con 0x7f064415cf60 2023-12-19T18:17:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f064ab39640 1 -- 172.21.15.26:0/2083127571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06441f2d30 con 0x7f064415cf60 2023-12-19T18:17:01.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.489+0000 7f064ab39640 1 -- 172.21.15.26:0/2083127571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0644074270 con 0x7f064415cf60 2023-12-19T18:17:01.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.493+0000 7f06417fa640 1 -- 172.21.15.26:0/2083127571 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 124) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f063c00ad30 con 0x7f064415cf60 2023-12-19T18:17:01.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.493+0000 7f06417fa640 1 --2- 172.21.15.26:0/2083127571 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0628065d30 0x7f06280681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:01.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.493+0000 7f06417fa640 1 -- 172.21.15.26:0/2083127571 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7f063c0832a0 con 0x7f064415cf60 2023-12-19T18:17:01.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.493+0000 7f06488d5640 1 -- 172.21.15.26:0/2083127571 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0628065d30 msgr2=0x7f06280681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:01.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.493+0000 7f06488d5640 1 --2- 172.21.15.26:0/2083127571 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0628065d30 0x7f06280681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:01.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.497+0000 7f06417fa640 1 -- 172.21.15.26:0/2083127571 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f063c0501d0 con 0x7f064415cf60 2023-12-19T18:17:01.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.665+0000 7f064ab39640 1 -- 172.21.15.26:0/2083127571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06440d36b0 con 0x7f064415cf60 2023-12-19T18:17:01.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.669+0000 7f06417fa640 1 -- 172.21.15.26:0/2083127571 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v217) v1 ==== 74+0+21487 (secure 0 0 0) 0x7f063c018090 con 0x7f064415cf60 2023-12-19T18:17:01.673 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:01.673 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":217,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:55.864913+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:25.863942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:26.798049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:16:38.226152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:16:39.540217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1134319862":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/3401801452":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/765362777":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/1696607504":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6833/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6832/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/2355423136":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/3498637577":"2023-12-20T18:16:55.864876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:01.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.669+0000 7f064ab39640 1 -- 172.21.15.26:0/2083127571 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0628065d30 msgr2=0x7f06280681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:01.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.669+0000 7f064ab39640 1 --2- 172.21.15.26:0/2083127571 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0628065d30 0x7f06280681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.669+0000 7f064ab39640 1 -- 172.21.15.26:0/2083127571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f064415cf60 msgr2=0x7f06440b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.669+0000 7f064ab39640 1 --2- 172.21.15.26:0/2083127571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f064415cf60 0x7f06440b3780 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c7a0 tx=0x7f063c00cc60 comp rx=0 tx=0).stop 2023-12-19T18:17:01.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.669+0000 7f064ab39640 1 -- 172.21.15.26:0/2083127571 shutdown_connections 2023-12-19T18:17:01.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.669+0000 7f064ab39640 1 --2- 172.21.15.26:0/2083127571 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0628065d30 0x7f06280681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.669+0000 7f064ab39640 1 --2- 172.21.15.26:0/2083127571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f064415cf60 0x7f06440b3780 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.669+0000 7f064ab39640 1 --2- 172.21.15.26:0/2083127571 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f06440d1420 0x7f06441562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.669+0000 7f064ab39640 1 --2- 172.21.15.26:0/2083127571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06440b4d10 0x7f06441f3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.669+0000 7f064ab39640 1 -- 172.21.15.26:0/2083127571 >> 172.21.15.26:0/2083127571 conn(0x7f0644065ee0 msgr2=0x7f064415bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.669+0000 7f064ab39640 1 -- 172.21.15.26:0/2083127571 shutdown_connections 2023-12-19T18:17:01.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.669+0000 7f064ab39640 1 -- 172.21.15.26:0/2083127571 wait complete. 2023-12-19T18:17:01.696 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:16:25.863942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '211', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:16:26.798049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-19T18:16:38.226152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '215', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-19T18:16:39.540217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '216', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:17:01.697 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:17:01.698 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:01.697+0000 7f8fe7b1a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:17:01.743 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:01.741+0000 7f9c5762c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:17:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.841+0000 7f2f32a2d640 1 Processor -- start 2023-12-19T18:17:01.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.841+0000 7f2f32a2d640 1 -- start start 2023-12-19T18:17:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.845+0000 7f2f32a2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f2c15ada0 0x7f2f2c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.845+0000 7f2f32a2d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2c070d90 con 0x7f2f2c15a9c0 2023-12-19T18:17:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.845+0000 7f2f32a2d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2c071070 con 0x7f2f2c06ac80 2023-12-19T18:17:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.845+0000 7f2f32a2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0cc270 con 0x7f2f2c15ada0 2023-12-19T18:17:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.845+0000 7f2f2bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2f2c15a9c0 0x7f2f2c0b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36516/0 (socket says 172.21.15.26:36516) 2023-12-19T18:17:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.845+0000 7f2f2bfff640 1 -- 172.21.15.26:0/560150503 learned_addr learned my addr 172.21.15.26:0/560150503 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.845+0000 7f2f2b7fe640 1 --2- 172.21.15.26:0/560150503 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f2c15ada0 0x7f2f2c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.845+0000 7f2f2b7fe640 1 -- 172.21.15.26:0/560150503 >> v1:172.21.15.160:6789/0 conn(0x7f2f2c06ac80 legacy=0x7f2f2c0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.845+0000 7f2f2affd640 1 -- 172.21.15.26:0/560150503 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924297893 0 0) 0x7f2f2c070d90 con 0x7f2f2c15a9c0 2023-12-19T18:17:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.845+0000 7f2f2b7fe640 1 -- 172.21.15.26:0/560150503 >> v1:172.21.15.26:6789/0 conn(0x7f2f2c15a9c0 legacy=0x7f2f2c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.845+0000 7f2f2b7fe640 1 -- 172.21.15.26:0/560150503 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c071350 con 0x7f2f2c15ada0 2023-12-19T18:17:01.849 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:01.849+0000 7f8fe7b1a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:17:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.849+0000 7f2f2b7fe640 1 --2- 172.21.15.26:0/560150503 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f2c15ada0 0x7f2f2c0706d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2f200097f0 tx=0x7f2f2001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bd8f27e10cd3249 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.849+0000 7f2f2affd640 1 -- 172.21.15.26:0/560150503 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f200ab620 con 0x7f2f2c15ada0 2023-12-19T18:17:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.849+0000 7f2f2affd640 1 -- 172.21.15.26:0/560150503 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f200ab780 con 0x7f2f2c15ada0 2023-12-19T18:17:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.849+0000 7f2f2affd640 1 -- 172.21.15.26:0/560150503 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f200abb10 con 0x7f2f2c15ada0 2023-12-19T18:17:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.849+0000 7f2f32a2d640 1 -- 172.21.15.26:0/560150503 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f2c15ada0 msgr2=0x7f2f2c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.849+0000 7f2f32a2d640 1 --2- 172.21.15.26:0/560150503 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f2c15ada0 0x7f2f2c0706d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2f200097f0 tx=0x7f2f2001c400 comp rx=0 tx=0).stop 2023-12-19T18:17:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.849+0000 7f2f32a2d640 1 -- 172.21.15.26:0/560150503 shutdown_connections 2023-12-19T18:17:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.849+0000 7f2f32a2d640 1 --2- 172.21.15.26:0/560150503 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f2c15ada0 0x7f2f2c0706d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.849+0000 7f2f32a2d640 1 -- 172.21.15.26:0/560150503 >> 172.21.15.26:0/560150503 conn(0x7f2f2c0ce730 msgr2=0x7f2f2c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.849+0000 7f2f32a2d640 1 -- 172.21.15.26:0/560150503 shutdown_connections 2023-12-19T18:17:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.849+0000 7f2f32a2d640 1 -- 172.21.15.26:0/560150503 wait complete. 2023-12-19T18:17:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.849+0000 7f2f32a2d640 1 Processor -- start 2023-12-19T18:17:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.849+0000 7f2f32a2d640 1 -- start start 2023-12-19T18:17:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f32a2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f2c06ac80 0x7f2f2c148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f32a2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f2c15a9c0 0x7f2f2c158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f32a2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f2c15ada0 0x7f2f2c0af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f32a2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0c5770 con 0x7f2f2c15a9c0 2023-12-19T18:17:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f32a2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0c62f0 con 0x7f2f2c15ada0 2023-12-19T18:17:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f32a2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0c59c0 con 0x7f2f2c06ac80 2023-12-19T18:17:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f2bfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f2c06ac80 0x7f2f2c148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f2b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f2c15a9c0 0x7f2f2c158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f2bfff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f2c06ac80 0x7f2f2c148f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:54462/0 (socket says 172.21.15.26:54462) 2023-12-19T18:17:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f2bfff640 1 -- 172.21.15.26:0/3786857623 learned_addr learned my addr 172.21.15.26:0/3786857623 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f30fca640 1 --2- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f2c15ada0 0x7f2f2c0af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f2bfff640 1 -- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f2c15ada0 msgr2=0x7f2f2c0af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f2bfff640 1 --2- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f2c15ada0 0x7f2f2c0af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f2bfff640 1 -- 172.21.15.26:0/3786857623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f2c15a9c0 msgr2=0x7f2f2c158af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f2bfff640 1 --2- 172.21.15.26:0/3786857623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f2c15a9c0 0x7f2f2c158af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f2bfff640 1 -- 172.21.15.26:0/3786857623 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f20009420 con 0x7f2f2c06ac80 2023-12-19T18:17:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f2bfff640 1 --2- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f2c06ac80 0x7f2f2c148f10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00c4d0 tx=0x7f2f1c00c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f28ff9640 1 -- 172.21.15.26:0/3786857623 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c0074e0 con 0x7f2f2c06ac80 2023-12-19T18:17:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f28ff9640 1 -- 172.21.15.26:0/3786857623 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c007640 con 0x7f2f2c06ac80 2023-12-19T18:17:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f32a2d640 1 -- 172.21.15.26:0/3786857623 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2c0c5ca0 con 0x7f2f2c06ac80 2023-12-19T18:17:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f28ff9640 1 -- 172.21.15.26:0/3786857623 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c004a30 con 0x7f2f2c06ac80 2023-12-19T18:17:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f32a2d640 1 -- 172.21.15.26:0/3786857623 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c149450 con 0x7f2f2c06ac80 2023-12-19T18:17:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.853+0000 7f2f32a2d640 1 -- 172.21.15.26:0/3786857623 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c074270 con 0x7f2f2c06ac80 2023-12-19T18:17:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.857+0000 7f2f28ff9640 1 -- 172.21.15.26:0/3786857623 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 124) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f2f1c004b90 con 0x7f2f2c06ac80 2023-12-19T18:17:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.857+0000 7f2f28ff9640 1 --2- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f18065d60 0x7f2f18068220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:01.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.857+0000 7f2f28ff9640 1 -- 172.21.15.26:0/3786857623 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7f2f1c0831a0 con 0x7f2f2c06ac80 2023-12-19T18:17:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.857+0000 7f2f2b7fe640 1 -- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f18065d60 msgr2=0x7f2f18068220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:01.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.857+0000 7f2f2b7fe640 1 --2- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f18065d60 0x7f2f18068220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:01.861+0000 7f2f28ff9640 1 -- 172.21.15.26:0/3786857623 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2f1c050160 con 0x7f2f2c06ac80 2023-12-19T18:17:01.916 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:01.913+0000 7f9c5762c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:17:02.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f32a2d640 1 -- 172.21.15.26:0/3786857623 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f2c0c5b00 con 0x7f2f2c06ac80 2023-12-19T18:17:02.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f28ff9640 1 -- 172.21.15.26:0/3786857623 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v217) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f2c0c5b00 con 0x7f2f2c06ac80 2023-12-19T18:17:02.036 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:17:02.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f32a2d640 1 -- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f18065d60 msgr2=0x7f2f18068220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:02.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f32a2d640 1 --2- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f18065d60 0x7f2f18068220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f32a2d640 1 -- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f2c06ac80 msgr2=0x7f2f2c148f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f32a2d640 1 --2- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f2c06ac80 0x7f2f2c148f10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00c4d0 tx=0x7f2f1c00c990 comp rx=0 tx=0).stop 2023-12-19T18:17:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f32a2d640 1 -- 172.21.15.26:0/3786857623 shutdown_connections 2023-12-19T18:17:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f32a2d640 1 --2- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f18065d60 0x7f2f18068220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f32a2d640 1 --2- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2f2c15ada0 0x7f2f2c0af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f32a2d640 1 --2- 172.21.15.26:0/3786857623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f2c15a9c0 0x7f2f2c158af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f32a2d640 1 --2- 172.21.15.26:0/3786857623 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2f2c06ac80 0x7f2f2c148f10 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f32a2d640 1 -- 172.21.15.26:0/3786857623 >> 172.21.15.26:0/3786857623 conn(0x7f2f2c0ce730 msgr2=0x7f2f2c152630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f32a2d640 1 -- 172.21.15.26:0/3786857623 shutdown_connections 2023-12-19T18:17:02.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.033+0000 7f2f32a2d640 1 -- 172.21.15.26:0/3786857623 wait complete. 2023-12-19T18:17:02.059 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:17:02.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8747dbe640 1 Processor -- start 2023-12-19T18:17:02.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8747dbe640 1 -- start start 2023-12-19T18:17:02.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8747dbe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f874006b110 0x7f8740070030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8747dbe640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f874006f490 con 0x7f874006ac80 2023-12-19T18:17:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8747dbe640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8740159520 con 0x7f8740159140 2023-12-19T18:17:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8747dbe640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87400cd690 con 0x7f874006b110 2023-12-19T18:17:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8745359640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f874006ac80 0x7f874006f920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36532/0 (socket says 172.21.15.26:36532) 2023-12-19T18:17:02.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8745359640 1 -- 172.21.15.26:0/3704990380 learned_addr learned my addr 172.21.15.26:0/3704990380 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f874635b640 1 --2- 172.21.15.26:0/3704990380 >> v2:172.21.15.160:3300/0 conn(0x7f874006b110 0x7f8740070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652692593 0 0) 0x7f874006f490 con 0x7f874006ac80 2023-12-19T18:17:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f872c003450 con 0x7f874006ac80 2023-12-19T18:17:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683381650 0 0) 0x7f8740159520 con 0x7f8740159140 2023-12-19T18:17:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f874006f490 con 0x7f8740159140 2023-12-19T18:17:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2013102776 0 0) 0x7f872c003450 con 0x7f874006ac80 2023-12-19T18:17:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 >> v2:172.21.15.160:3300/0 conn(0x7f874006b110 msgr2=0x7f8740070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 --2- 172.21.15.26:0/3704990380 >> v2:172.21.15.160:3300/0 conn(0x7f874006b110 0x7f8740070030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 >> v1:172.21.15.160:6789/0 conn(0x7f8740159140 legacy=0x7f87400ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400cd970 con 0x7f874006ac80 2023-12-19T18:17:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f8730001680 con 0x7f874006ac80 2023-12-19T18:17:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 >> v1:172.21.15.26:6789/0 conn(0x7f874006ac80 legacy=0x7f874006f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 --2- 172.21.15.26:0/3704990380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c003c00 0x7f872c004000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 --2- 172.21.15.26:0/3704990380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f872c008970 0x7f872c00ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 --2- 172.21.15.26:0/3704990380 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f872c00b470 0x7f872c00d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87400cd690 con 0x7f872c008970 2023-12-19T18:17:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c00e210 con 0x7f872c003c00 2023-12-19T18:17:02.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c00e3d0 con 0x7f872c00b470 2023-12-19T18:17:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8745359640 1 --2- 172.21.15.26:0/3704990380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f872c008970 0x7f872c00ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8745b5a640 1 --2- 172.21.15.26:0/3704990380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c003c00 0x7f872c004000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f874635b640 1 --2- 172.21.15.26:0/3704990380 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f872c00b470 0x7f872c00d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8745359640 1 -- 172.21.15.26:0/3704990380 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f872c00b470 msgr2=0x7f872c00d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:02.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8745359640 1 --2- 172.21.15.26:0/3704990380 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f872c00b470 0x7f872c00d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8745359640 1 -- 172.21.15.26:0/3704990380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c003c00 msgr2=0x7f872c004000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8745359640 1 --2- 172.21.15.26:0/3704990380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c003c00 0x7f872c004000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f8745359640 1 -- 172.21.15.26:0/3704990380 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c003450 con 0x7f872c008970 2023-12-19T18:17:02.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.205+0000 7f874635b640 1 -- 172.21.15.26:0/3704990380 reap_dead start 2023-12-19T18:17:02.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.209+0000 7f8745359640 1 --2- 172.21.15.26:0/3704990380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f872c008970 0x7f872c00ae00 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f873000c160 tx=0x7f873001cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb64c034597dcdcc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.209+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87300058e0 con 0x7f872c008970 2023-12-19T18:17:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.209+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8730005a60 con 0x7f872c008970 2023-12-19T18:17:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.209+0000 7f8744b58640 1 -- 172.21.15.26:0/3704990380 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730005e10 con 0x7f872c008970 2023-12-19T18:17:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.209+0000 7f8747dbe640 1 -- 172.21.15.26:0/3704990380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f872c008970 msgr2=0x7f872c00ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.209+0000 7f8747dbe640 1 --2- 172.21.15.26:0/3704990380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f872c008970 0x7f872c00ae00 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f873000c160 tx=0x7f873001cb60 comp rx=0 tx=0).stop 2023-12-19T18:17:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.209+0000 7f8747dbe640 1 -- 172.21.15.26:0/3704990380 shutdown_connections 2023-12-19T18:17:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.209+0000 7f8747dbe640 1 --2- 172.21.15.26:0/3704990380 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f872c008970 0x7f872c00ae00 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.209+0000 7f8747dbe640 1 -- 172.21.15.26:0/3704990380 >> 172.21.15.26:0/3704990380 conn(0x7f87400ce730 msgr2=0x7f87400ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.209+0000 7f8747dbe640 1 -- 172.21.15.26:0/3704990380 shutdown_connections 2023-12-19T18:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.209+0000 7f8747dbe640 1 -- 172.21.15.26:0/3704990380 wait complete. 2023-12-19T18:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.213+0000 7f8747dbe640 1 Processor -- start 2023-12-19T18:17:02.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.213+0000 7f8747dbe640 1 -- start start 2023-12-19T18:17:02.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f8747dbe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c008970 0x7f8740152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f8747dbe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87400b35e0 0x7f87401f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f8747dbe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87401e7520 0x7f87400c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f8747dbe640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87401f5ff0 con 0x7f87401e7520 2023-12-19T18:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f8747dbe640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87400b3a80 con 0x7f872c008970 2023-12-19T18:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f8747dbe640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87400b3d60 con 0x7f87400b35e0 2023-12-19T18:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f874635b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87401e7520 0x7f87400c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f874635b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87401e7520 0x7f87400c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38280/0 (socket says 172.21.15.26:38280) 2023-12-19T18:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f874635b640 1 -- 172.21.15.26:0/1702871528 learned_addr learned my addr 172.21.15.26:0/1702871528 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f8745359640 1 --2- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87400b35e0 0x7f87401f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f8745b5a640 1 --2- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c008970 0x7f8740152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f874635b640 1 -- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87400b35e0 msgr2=0x7f87401f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f874635b640 1 --2- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87400b35e0 0x7f87401f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f874635b640 1 -- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c008970 msgr2=0x7f8740152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f874635b640 1 --2- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c008970 0x7f8740152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f874635b640 1 -- 172.21.15.26:0/1702871528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400b4040 con 0x7f87401e7520 2023-12-19T18:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f874635b640 1 --2- 172.21.15.26:0/1702871528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87401e7520 0x7f87400c95e0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f87400cf470 tx=0x7f873800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f87367fc640 1 -- 172.21.15.26:0/1702871528 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873800e040 con 0x7f87401e7520 2023-12-19T18:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f87367fc640 1 -- 172.21.15.26:0/1702871528 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873800f040 con 0x7f87401e7520 2023-12-19T18:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f87367fc640 1 -- 172.21.15.26:0/1702871528 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738015430 con 0x7f87401e7520 2023-12-19T18:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f8747dbe640 1 -- 172.21.15.26:0/1702871528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87400b4320 con 0x7f87401e7520 2023-12-19T18:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f8747dbe640 1 -- 172.21.15.26:0/1702871528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87401f3110 con 0x7f87401e7520 2023-12-19T18:17:02.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.217+0000 7f8747dbe640 1 -- 172.21.15.26:0/1702871528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87401f6160 con 0x7f87401e7520 2023-12-19T18:17:02.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.221+0000 7f87367fc640 1 -- 172.21.15.26:0/1702871528 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 124) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f8738015590 con 0x7f87401e7520 2023-12-19T18:17:02.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.221+0000 7f87367fc640 1 --2- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8724065d30 0x7f87240681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.221+0000 7f8745b5a640 1 -- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8724065d30 msgr2=0x7f87240681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:02.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.221+0000 7f87367fc640 1 -- 172.21.15.26:0/1702871528 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7f8738084480 con 0x7f87401e7520 2023-12-19T18:17:02.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.221+0000 7f8745b5a640 1 --2- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8724065d30 0x7f87240681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:02.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.225+0000 7f87367fc640 1 -- 172.21.15.26:0/1702871528 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f87401f6160 con 0x7f87401e7520 2023-12-19T18:17:02.334 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:02.333+0000 7f8fe7b1a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:17:02.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.385+0000 7f8747dbe640 1 -- 172.21.15.26:0/1702871528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87401f6160 con 0x7f87401e7520 2023-12-19T18:17:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.385+0000 7f87367fc640 1 -- 172.21.15.26:0/1702871528 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v217) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87401f6160 con 0x7f87401e7520 2023-12-19T18:17:02.389 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:17:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.385+0000 7f8747dbe640 1 -- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8724065d30 msgr2=0x7f87240681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.385+0000 7f8747dbe640 1 --2- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8724065d30 0x7f87240681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.385+0000 7f8747dbe640 1 -- 172.21.15.26:0/1702871528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87401e7520 msgr2=0x7f87400c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.385+0000 7f8747dbe640 1 --2- 172.21.15.26:0/1702871528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87401e7520 0x7f87400c95e0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f87400cf470 tx=0x7f873800f1b0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.385+0000 7f8747dbe640 1 -- 172.21.15.26:0/1702871528 shutdown_connections 2023-12-19T18:17:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.385+0000 7f8747dbe640 1 --2- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8724065d30 0x7f87240681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.385+0000 7f8747dbe640 1 --2- 172.21.15.26:0/1702871528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87401e7520 0x7f87400c95e0 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.385+0000 7f8747dbe640 1 --2- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f87400b35e0 0x7f87401f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.385+0000 7f8747dbe640 1 --2- 172.21.15.26:0/1702871528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f872c008970 0x7f8740152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.385+0000 7f8747dbe640 1 -- 172.21.15.26:0/1702871528 >> 172.21.15.26:0/1702871528 conn(0x7f87400ce730 msgr2=0x7f8740154a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.389+0000 7f8747dbe640 1 -- 172.21.15.26:0/1702871528 shutdown_connections 2023-12-19T18:17:02.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.389+0000 7f8747dbe640 1 -- 172.21.15.26:0/1702871528 wait complete. 2023-12-19T18:17:02.393 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:02.393+0000 7f8fe7b1a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:17:02.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:17:02.440 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:02.437+0000 7f9c5762c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:17:02.448 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:02.449+0000 7f8fe7b1a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:17:02.503 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:02.497+0000 7f9c5762c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:17:02.534 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:02.533+0000 7f8fe7b1a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:17:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65c5620640 1 Processor -- start 2023-12-19T18:17:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65c5620640 1 -- start start 2023-12-19T18:17:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65c5620640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65c015a9c0 0x7f65c00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65c5620640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f65c015ada0 0x7f65c00b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65c5620640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c00b46c0 con 0x7f65c006ac80 2023-12-19T18:17:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65c5620640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c00cc270 con 0x7f65c015ada0 2023-12-19T18:17:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65c5620640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c0070d90 con 0x7f65c015a9c0 2023-12-19T18:17:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65bf7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f65c006ac80 0x7f65c00b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36538/0 (socket says 172.21.15.26:36538) 2023-12-19T18:17:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65bf7fe640 1 -- 172.21.15.26:0/4198677734 learned_addr learned my addr 172.21.15.26:0/4198677734 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65beffd640 1 --2- 172.21.15.26:0/4198677734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65c015a9c0 0x7f65c00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65b7fff640 1 --2- 172.21.15.26:0/4198677734 >> v2:172.21.15.160:3300/0 conn(0x7f65c015ada0 0x7f65c00b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65be7fc640 1 -- 172.21.15.26:0/4198677734 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556464719 0 0) 0x7f65c00b46c0 con 0x7f65c006ac80 2023-12-19T18:17:02.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65be7fc640 1 -- 172.21.15.26:0/4198677734 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65ac003530 con 0x7f65c006ac80 2023-12-19T18:17:02.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65beffd640 1 -- 172.21.15.26:0/4198677734 >> v2:172.21.15.160:3300/0 conn(0x7f65c015ada0 msgr2=0x7f65c00b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:02.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65beffd640 1 --2- 172.21.15.26:0/4198677734 >> v2:172.21.15.160:3300/0 conn(0x7f65c015ada0 0x7f65c00b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65beffd640 1 -- 172.21.15.26:0/4198677734 >> v1:172.21.15.26:6789/0 conn(0x7f65c006ac80 legacy=0x7f65c00b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.557+0000 7f65beffd640 1 -- 172.21.15.26:0/4198677734 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c0071070 con 0x7f65c015a9c0 2023-12-19T18:17:02.563 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:02.561+0000 7f9c5762c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:17:02.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.561+0000 7f65beffd640 1 --2- 172.21.15.26:0/4198677734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65c015a9c0 0x7f65c00706d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f65b00097c0 tx=0x7f65b001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e9afaf6a73a580a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.561+0000 7f65be7fc640 1 -- 172.21.15.26:0/4198677734 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b00ac620 con 0x7f65c015a9c0 2023-12-19T18:17:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.561+0000 7f65be7fc640 1 -- 172.21.15.26:0/4198677734 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b00ac780 con 0x7f65c015a9c0 2023-12-19T18:17:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.561+0000 7f65be7fc640 1 -- 172.21.15.26:0/4198677734 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b00acb10 con 0x7f65c015a9c0 2023-12-19T18:17:02.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.561+0000 7f65c5620640 1 -- 172.21.15.26:0/4198677734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65c015a9c0 msgr2=0x7f65c00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.561+0000 7f65c5620640 1 --2- 172.21.15.26:0/4198677734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65c015a9c0 0x7f65c00706d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f65b00097c0 tx=0x7f65b001c3c0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.565+0000 7f65c5620640 1 -- 172.21.15.26:0/4198677734 shutdown_connections 2023-12-19T18:17:02.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.565+0000 7f65c5620640 1 --2- 172.21.15.26:0/4198677734 >> v2:172.21.15.160:3300/0 conn(0x7f65c015ada0 0x7f65c00b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.565+0000 7f65c5620640 1 --2- 172.21.15.26:0/4198677734 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65c015a9c0 0x7f65c00706d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.565+0000 7f65c5620640 1 -- 172.21.15.26:0/4198677734 >> 172.21.15.26:0/4198677734 conn(0x7f65c00ce730 msgr2=0x7f65c00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:02.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.565+0000 7f65c5620640 1 -- 172.21.15.26:0/4198677734 shutdown_connections 2023-12-19T18:17:02.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.565+0000 7f65c5620640 1 -- 172.21.15.26:0/4198677734 wait complete. 2023-12-19T18:17:02.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.565+0000 7f65c5620640 1 Processor -- start 2023-12-19T18:17:02.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.565+0000 7f65c5620640 1 -- start start 2023-12-19T18:17:02.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65c5620640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65c006ac80 0x7f65c00c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65c5620640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65c015a9c0 0x7f65c00b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65c5620640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65c015ada0 0x7f65c00ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65c5620640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c00c9fc0 con 0x7f65c015ada0 2023-12-19T18:17:02.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65c5620640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c01511f0 con 0x7f65c015a9c0 2023-12-19T18:17:02.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65c5620640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c0150430 con 0x7f65c006ac80 2023-12-19T18:17:02.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65bf7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65c015ada0 0x7f65c00ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65beffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65c006ac80 0x7f65c00c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65beffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65c006ac80 0x7f65c00c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:54496/0 (socket says 172.21.15.26:54496) 2023-12-19T18:17:02.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65beffd640 1 -- 172.21.15.26:0/3608896677 learned_addr learned my addr 172.21.15.26:0/3608896677 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:02.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65b7fff640 1 --2- 172.21.15.26:0/3608896677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65c015a9c0 0x7f65c00b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65bf7fe640 1 -- 172.21.15.26:0/3608896677 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65c006ac80 msgr2=0x7f65c00c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65bf7fe640 1 --2- 172.21.15.26:0/3608896677 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65c006ac80 0x7f65c00c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65bf7fe640 1 -- 172.21.15.26:0/3608896677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65c015a9c0 msgr2=0x7f65c00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65bf7fe640 1 --2- 172.21.15.26:0/3608896677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65c015a9c0 0x7f65c00b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65bf7fe640 1 -- 172.21.15.26:0/3608896677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b0009420 con 0x7f65c015ada0 2023-12-19T18:17:02.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65bf7fe640 1 --2- 172.21.15.26:0/3608896677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65c015ada0 0x7f65c00ac860 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f65b800ca80 tx=0x7f65b800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:02.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65b77fe640 1 -- 172.21.15.26:0/3608896677 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b800ac30 con 0x7f65c015ada0 2023-12-19T18:17:02.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65b77fe640 1 -- 172.21.15.26:0/3608896677 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b800ad90 con 0x7f65c015ada0 2023-12-19T18:17:02.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65b77fe640 1 -- 172.21.15.26:0/3608896677 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8012600 con 0x7f65c015ada0 2023-12-19T18:17:02.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65c5620640 1 -- 172.21.15.26:0/3608896677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c0150710 con 0x7f65c015ada0 2023-12-19T18:17:02.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65c5620640 1 -- 172.21.15.26:0/3608896677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c0150cd0 con 0x7f65c015ada0 2023-12-19T18:17:02.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.569+0000 7f65c5620640 1 -- 172.21.15.26:0/3608896677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c0150ec0 con 0x7f65c015ada0 2023-12-19T18:17:02.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.573+0000 7f65b77fe640 1 -- 172.21.15.26:0/3608896677 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 124) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f65b8007070 con 0x7f65c015ada0 2023-12-19T18:17:02.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.573+0000 7f65b77fe640 1 --2- 172.21.15.26:0/3608896677 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f65a0065d30 0x7f65a00681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.573+0000 7f65b77fe640 1 -- 172.21.15.26:0/3608896677 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7f65b8083b60 con 0x7f65c015ada0 2023-12-19T18:17:02.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.573+0000 7f65beffd640 1 -- 172.21.15.26:0/3608896677 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f65a0065d30 msgr2=0x7f65a00681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:02.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.573+0000 7f65beffd640 1 --2- 172.21.15.26:0/3608896677 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f65a0065d30 0x7f65a00681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:02.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.577+0000 7f65b77fe640 1 -- 172.21.15.26:0/3608896677 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f65b8050b20 con 0x7f65c015ada0 2023-12-19T18:17:02.593 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:02.593+0000 7f8fe7b1a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:17:02.658 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:02.653+0000 7f9c5762c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:17:02.678 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:02.677+0000 7f8fe7b1a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:17:02.718 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:02.713+0000 7f9c5762c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:17:02.734 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:02.733+0000 7f8fe7b1a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:17:02.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.733+0000 7f65c5620640 1 -- 172.21.15.26:0/3608896677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65c0074270 con 0x7f65c015ada0 2023-12-19T18:17:02.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.733+0000 7f65b77fe640 1 -- 172.21.15.26:0/3608896677 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v217) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65b80170a0 con 0x7f65c015ada0 2023-12-19T18:17:02.738 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:17:02.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.737+0000 7f65c5620640 1 -- 172.21.15.26:0/3608896677 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f65a0065d30 msgr2=0x7f65a00681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:02.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.737+0000 7f65c5620640 1 --2- 172.21.15.26:0/3608896677 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f65a0065d30 0x7f65a00681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.737+0000 7f65c5620640 1 -- 172.21.15.26:0/3608896677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65c015ada0 msgr2=0x7f65c00ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.737+0000 7f65c5620640 1 --2- 172.21.15.26:0/3608896677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65c015ada0 0x7f65c00ac860 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f65b800ca80 tx=0x7f65b800cf40 comp rx=0 tx=0).stop 2023-12-19T18:17:02.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.737+0000 7f65c5620640 1 -- 172.21.15.26:0/3608896677 shutdown_connections 2023-12-19T18:17:02.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.737+0000 7f65c5620640 1 --2- 172.21.15.26:0/3608896677 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f65a0065d30 0x7f65a00681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.737+0000 7f65c5620640 1 --2- 172.21.15.26:0/3608896677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65c015ada0 0x7f65c00ac860 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.737+0000 7f65c5620640 1 --2- 172.21.15.26:0/3608896677 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65c015a9c0 0x7f65c00b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.737+0000 7f65c5620640 1 --2- 172.21.15.26:0/3608896677 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f65c006ac80 0x7f65c00c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.737+0000 7f65c5620640 1 -- 172.21.15.26:0/3608896677 >> 172.21.15.26:0/3608896677 conn(0x7f65c00ce730 msgr2=0x7f65c006c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:02.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.737+0000 7f65c5620640 1 -- 172.21.15.26:0/3608896677 shutdown_connections 2023-12-19T18:17:02.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.737+0000 7f65c5620640 1 -- 172.21.15.26:0/3608896677 wait complete. 2023-12-19T18:17:02.761 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-19T18:17:02.808 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:02.805+0000 7f9c5762c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:17:02.868 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:02.865+0000 7f9c5762c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:17:02.877 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:02.877+0000 7f8fe7b1a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:17:02.935 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:02.933+0000 7f8fe7b1a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:17:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.945+0000 7f0981db3640 1 Processor -- start 2023-12-19T18:17:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.945+0000 7f0981db3640 1 -- start start 2023-12-19T18:17:02.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f0981db3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f097c15a9c0 0x7f097c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f0981db3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f097c15ada0 0x7f097c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f0981db3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097c0b46c0 con 0x7f097c06ac80 2023-12-19T18:17:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f0981db3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f097c0cc270 con 0x7f097c15ada0 2023-12-19T18:17:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f0981db3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f097c070d90 con 0x7f097c15a9c0 2023-12-19T18:17:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f097affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f097c15ada0 0x7f097c0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f097bfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f097c06ac80 0x7f097c0b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36554/0 (socket says 172.21.15.26:36554) 2023-12-19T18:17:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f097bfff640 1 -- 172.21.15.26:0/4294450418 learned_addr learned my addr 172.21.15.26:0/4294450418 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:02.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f097b7fe640 1 --2- 172.21.15.26:0/4294450418 >> v2:172.21.15.160:3300/0 conn(0x7f097c15a9c0 0x7f097c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f097affd640 1 -- 172.21.15.26:0/4294450418 >> v2:172.21.15.160:3300/0 conn(0x7f097c15a9c0 msgr2=0x7f097c0706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:02.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f097affd640 1 --2- 172.21.15.26:0/4294450418 >> v2:172.21.15.160:3300/0 conn(0x7f097c15a9c0 0x7f097c0706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f097affd640 1 -- 172.21.15.26:0/4294450418 >> v1:172.21.15.26:6789/0 conn(0x7f097c06ac80 legacy=0x7f097c0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f097affd640 1 -- 172.21.15.26:0/4294450418 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097c071070 con 0x7f097c15ada0 2023-12-19T18:17:02.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.949+0000 7f097a7fc640 1 -- 172.21.15.26:0/4294450418 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433310492 0 0) 0x7f097c0b46c0 con 0x7f097c06ac80 2023-12-19T18:17:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f097affd640 1 --2- 172.21.15.26:0/4294450418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f097c15ada0 0x7f097c0b4180 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f097c0cf590 tx=0x7f097001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26a81dcf341199e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f097a7fc640 1 -- 172.21.15.26:0/4294450418 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09700ac600 con 0x7f097c15ada0 2023-12-19T18:17:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f097a7fc640 1 -- 172.21.15.26:0/4294450418 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09700a8050 con 0x7f097c15ada0 2023-12-19T18:17:02.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f097a7fc640 1 -- 172.21.15.26:0/4294450418 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09700ac980 con 0x7f097c15ada0 2023-12-19T18:17:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f0981db3640 1 -- 172.21.15.26:0/4294450418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f097c15ada0 msgr2=0x7f097c0b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f0981db3640 1 --2- 172.21.15.26:0/4294450418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f097c15ada0 0x7f097c0b4180 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f097c0cf590 tx=0x7f097001c3a0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f0981db3640 1 -- 172.21.15.26:0/4294450418 shutdown_connections 2023-12-19T18:17:02.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f0981db3640 1 --2- 172.21.15.26:0/4294450418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f097c15ada0 0x7f097c0b4180 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f0981db3640 1 --2- 172.21.15.26:0/4294450418 >> v2:172.21.15.160:3300/0 conn(0x7f097c15a9c0 0x7f097c0706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f0981db3640 1 -- 172.21.15.26:0/4294450418 >> 172.21.15.26:0/4294450418 conn(0x7f097c0ce730 msgr2=0x7f097c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:02.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f0981db3640 1 -- 172.21.15.26:0/4294450418 shutdown_connections 2023-12-19T18:17:02.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f0981db3640 1 -- 172.21.15.26:0/4294450418 wait complete. 2023-12-19T18:17:02.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f0981db3640 1 Processor -- start 2023-12-19T18:17:02.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.953+0000 7f0981db3640 1 -- start start 2023-12-19T18:17:02.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f0981db3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f097c06ac80 0x7f097c0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f0981db3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f097c15a9c0 0x7f097c14c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f0981db3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f097c15ada0 0x7f097c0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f0981db3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c14ca60 con 0x7f097c15a9c0 2023-12-19T18:17:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f0981db3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097c14cf40 con 0x7f097c15ada0 2023-12-19T18:17:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f0981db3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097c1f91c0 con 0x7f097c06ac80 2023-12-19T18:17:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f097b7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f097c06ac80 0x7f097c0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f097affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f097c15a9c0 0x7f097c14c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f097b7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f097c06ac80 0x7f097c0b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:54500/0 (socket says 172.21.15.26:54500) 2023-12-19T18:17:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f097b7fe640 1 -- 172.21.15.26:0/939818753 learned_addr learned my addr 172.21.15.26:0/939818753 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:02.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f097bfff640 1 --2- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f097c15ada0 0x7f097c0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f097b7fe640 1 -- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f097c15ada0 msgr2=0x7f097c0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f097b7fe640 1 --2- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f097c15ada0 0x7f097c0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f097b7fe640 1 -- 172.21.15.26:0/939818753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f097c15a9c0 msgr2=0x7f097c14c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f097b7fe640 1 --2- 172.21.15.26:0/939818753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f097c15a9c0 0x7f097c14c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f097b7fe640 1 -- 172.21.15.26:0/939818753 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0970009420 con 0x7f097c06ac80 2023-12-19T18:17:02.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f097b7fe640 1 --2- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f097c06ac80 0x7f097c0b1350 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f096c00e930 tx=0x7f096c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:02.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f0963fff640 1 -- 172.21.15.26:0/939818753 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c009690 con 0x7f097c06ac80 2023-12-19T18:17:02.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f0963fff640 1 -- 172.21.15.26:0/939818753 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096c00ca00 con 0x7f097c06ac80 2023-12-19T18:17:02.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f0963fff640 1 -- 172.21.15.26:0/939818753 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c00cd70 con 0x7f097c06ac80 2023-12-19T18:17:02.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f0981db3640 1 -- 172.21.15.26:0/939818753 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097c1f8ee0 con 0x7f097c06ac80 2023-12-19T18:17:02.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f0981db3640 1 -- 172.21.15.26:0/939818753 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097c0c7130 con 0x7f097c06ac80 2023-12-19T18:17:02.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f0981db3640 1 -- 172.21.15.26:0/939818753 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097c0c6cb0 con 0x7f097c06ac80 2023-12-19T18:17:02.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.957+0000 7f0963fff640 1 -- 172.21.15.26:0/939818753 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 124) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f096c013470 con 0x7f097c06ac80 2023-12-19T18:17:02.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.961+0000 7f0963fff640 1 --2- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0968065d30 0x7f09680681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.961+0000 7f0963fff640 1 -- 172.21.15.26:0/939818753 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7f096c083fb0 con 0x7f097c06ac80 2023-12-19T18:17:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.961+0000 7f097affd640 1 -- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0968065d30 msgr2=0x7f09680681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:02.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.961+0000 7f097affd640 1 --2- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0968065d30 0x7f09680681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:02.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:02.965+0000 7f0963fff640 1 -- 172.21.15.26:0/939818753 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f097c0c6cb0 con 0x7f097c06ac80 2023-12-19T18:17:03.024 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:03.021+0000 7f9c5762c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:17:03.087 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:03.085+0000 7f9c5762c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:17:03.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.121+0000 7f0981db3640 1 -- 172.21.15.26:0/939818753 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f097c0c6cb0 con 0x7f097c06ac80 2023-12-19T18:17:03.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.125+0000 7f0963fff640 1 -- 172.21.15.26:0/939818753 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v217) v1 ==== 107+0+11 (secure 0 0 0) 0x7f097c0c6cb0 con 0x7f097c06ac80 2023-12-19T18:17:03.127 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:17:03.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.125+0000 7f0981db3640 1 -- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0968065d30 msgr2=0x7f09680681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:03.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.125+0000 7f0981db3640 1 --2- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0968065d30 0x7f09680681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.125+0000 7f0981db3640 1 -- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f097c06ac80 msgr2=0x7f097c0b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.125+0000 7f0981db3640 1 --2- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f097c06ac80 0x7f097c0b1350 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f096c00e930 tx=0x7f096c00edf0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.125+0000 7f0981db3640 1 -- 172.21.15.26:0/939818753 shutdown_connections 2023-12-19T18:17:03.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.125+0000 7f0981db3640 1 --2- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0968065d30 0x7f09680681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.125+0000 7f0981db3640 1 --2- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f097c15ada0 0x7f097c0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.125+0000 7f0981db3640 1 --2- 172.21.15.26:0/939818753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f097c15a9c0 0x7f097c14c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.125+0000 7f0981db3640 1 --2- 172.21.15.26:0/939818753 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f097c06ac80 0x7f097c0b1350 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.125+0000 7f0981db3640 1 -- 172.21.15.26:0/939818753 >> 172.21.15.26:0/939818753 conn(0x7f097c0ce730 msgr2=0x7f097c159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:03.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.125+0000 7f0981db3640 1 -- 172.21.15.26:0/939818753 shutdown_connections 2023-12-19T18:17:03.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.125+0000 7f0981db3640 1 -- 172.21.15.26:0/939818753 wait complete. 2023-12-19T18:17:03.160 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-19T18:17:03.177 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:17:03.177 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:17:03.178 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:17:03.185 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:03.185+0000 7f8fe7b1a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:17:03.321 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:03.321+0000 7f8fe7b1a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:17:03.356 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:17:03.356 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:17:03.356 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:17:03.365 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:03.361+0000 7f9c5762c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:17:03.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.381+0000 7f5965c62640 1 Processor -- start 2023-12-19T18:17:03.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.381+0000 7f5965c62640 1 -- start start 2023-12-19T18:17:03.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f5965c62640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f596015cc90 0x7f59600cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f5965c62640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f596015d070 0x7f59600cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f5965c62640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5960160a20 0x7f59600cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f5965c62640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59600cfe40 con 0x7f596015d070 2023-12-19T18:17:03.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f5965c62640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59600cdb20 con 0x7f596015cc90 2023-12-19T18:17:03.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f5965c62640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59600cde00 con 0x7f5960160a20 2023-12-19T18:17:03.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f595ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f596015d070 0x7f59600cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f595ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f596015d070 0x7f59600cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38326/0 (socket says 172.21.15.26:38326) 2023-12-19T18:17:03.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f595ffff640 1 -- 172.21.15.26:0/601754356 learned_addr learned my addr 172.21.15.26:0/601754356 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:03.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f5965461640 1 --2- 172.21.15.26:0/601754356 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5960160a20 0x7f59600cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f5964c60640 1 --2- 172.21.15.26:0/601754356 >> v2:172.21.15.160:3300/0 conn(0x7f596015cc90 0x7f59600cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f595ffff640 1 -- 172.21.15.26:0/601754356 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5960160a20 msgr2=0x7f59600cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:03.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f595ffff640 1 --2- 172.21.15.26:0/601754356 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5960160a20 0x7f59600cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f595ffff640 1 -- 172.21.15.26:0/601754356 >> v2:172.21.15.160:3300/0 conn(0x7f596015cc90 msgr2=0x7f59600cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:03.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f595ffff640 1 --2- 172.21.15.26:0/601754356 >> v2:172.21.15.160:3300/0 conn(0x7f596015cc90 0x7f59600cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.385+0000 7f595ffff640 1 -- 172.21.15.26:0/601754356 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59600ce0e0 con 0x7f596015d070 2023-12-19T18:17:03.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f595ffff640 1 --2- 172.21.15.26:0/601754356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596015d070 0x7f59600cf3c0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f59600718e0 tx=0x7f594c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e032e5c8da4fbd90 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f595f7fe640 1 -- 172.21.15.26:0/601754356 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c0aa5f0 con 0x7f596015d070 2023-12-19T18:17:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f595f7fe640 1 -- 172.21.15.26:0/601754356 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c0aa750 con 0x7f596015d070 2023-12-19T18:17:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f595f7fe640 1 -- 172.21.15.26:0/601754356 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c0aaae0 con 0x7f596015d070 2023-12-19T18:17:03.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f5965c62640 1 -- 172.21.15.26:0/601754356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596015d070 msgr2=0x7f59600cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f5965c62640 1 --2- 172.21.15.26:0/601754356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596015d070 0x7f59600cf3c0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f59600718e0 tx=0x7f594c01c4d0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f5965c62640 1 -- 172.21.15.26:0/601754356 shutdown_connections 2023-12-19T18:17:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f5965c62640 1 --2- 172.21.15.26:0/601754356 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5960160a20 0x7f59600cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f5965c62640 1 --2- 172.21.15.26:0/601754356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596015d070 0x7f59600cf3c0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f5965c62640 1 --2- 172.21.15.26:0/601754356 >> v2:172.21.15.160:3300/0 conn(0x7f596015cc90 0x7f59600cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f5965c62640 1 -- 172.21.15.26:0/601754356 >> 172.21.15.26:0/601754356 conn(0x7f596006a320 msgr2=0x7f59600658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f5965c62640 1 -- 172.21.15.26:0/601754356 shutdown_connections 2023-12-19T18:17:03.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f5965c62640 1 -- 172.21.15.26:0/601754356 wait complete. 2023-12-19T18:17:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f5965c62640 1 Processor -- start 2023-12-19T18:17:03.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.389+0000 7f5965c62640 1 -- start start 2023-12-19T18:17:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965c62640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f596015cc90 0x7f59601561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965c62640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596015d070 0x7f59600d2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965c62640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5960160a20 0x7f59600d3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965c62640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59600c90a0 con 0x7f596015d070 2023-12-19T18:17:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965c62640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59600c9c90 con 0x7f596015cc90 2023-12-19T18:17:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965c62640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59600c9f70 con 0x7f5960160a20 2023-12-19T18:17:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f595ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596015d070 0x7f59600d2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5964c60640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f596015cc90 0x7f59601561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f595ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596015d070 0x7f59600d2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38334/0 (socket says 172.21.15.26:38334) 2023-12-19T18:17:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f595ffff640 1 -- 172.21.15.26:0/3776151304 learned_addr learned my addr 172.21.15.26:0/3776151304 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:03.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965461640 1 --2- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5960160a20 0x7f59600d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965461640 1 -- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f596015cc90 msgr2=0x7f59601561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965461640 1 --2- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f596015cc90 0x7f59601561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965461640 1 -- 172.21.15.26:0/3776151304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596015d070 msgr2=0x7f59600d2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965461640 1 --2- 172.21.15.26:0/3776151304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596015d070 0x7f59600d2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965461640 1 -- 172.21.15.26:0/3776151304 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c0094d0 con 0x7f5960160a20 2023-12-19T18:17:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f595ffff640 1 --2- 172.21.15.26:0/3776151304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596015d070 0x7f59600d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965461640 1 --2- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5960160a20 0x7f59600d3090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f595800eba0 tx=0x7f595800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f595d7fa640 1 -- 172.21.15.26:0/3776151304 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595800cd20 con 0x7f5960160a20 2023-12-19T18:17:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f595d7fa640 1 -- 172.21.15.26:0/3776151304 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595800f040 con 0x7f5960160a20 2023-12-19T18:17:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965c62640 1 -- 172.21.15.26:0/3776151304 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59600ca250 con 0x7f5960160a20 2023-12-19T18:17:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f595d7fa640 1 -- 172.21.15.26:0/3776151304 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59580146c0 con 0x7f5960160a20 2023-12-19T18:17:03.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965c62640 1 -- 172.21.15.26:0/3776151304 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5960156710 con 0x7f5960160a20 2023-12-19T18:17:03.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.393+0000 7f5965c62640 1 -- 172.21.15.26:0/3776151304 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59600ca770 con 0x7f5960160a20 2023-12-19T18:17:03.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.397+0000 7f595d7fa640 1 -- 172.21.15.26:0/3776151304 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 124) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f5958014820 con 0x7f5960160a20 2023-12-19T18:17:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.397+0000 7f595d7fa640 1 --2- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5944065d30 0x7f59440681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.397+0000 7f595d7fa640 1 -- 172.21.15.26:0/3776151304 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7f5958084a30 con 0x7f5960160a20 2023-12-19T18:17:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.397+0000 7f5964c60640 1 -- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5944065d30 msgr2=0x7f59440681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:03.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.397+0000 7f5964c60640 1 --2- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5944065d30 0x7f59440681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:03.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.401+0000 7f595d7fa640 1 -- 172.21.15.26:0/3776151304 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f59600ca770 con 0x7f5960160a20 2023-12-19T18:17:03.446 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:03.445+0000 7f8fe7b1a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:17:03.513 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:03.509+0000 7f9c5762c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:17:03.517 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:03.517+0000 7f8fe7b1a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:17:03.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.557+0000 7f5965c62640 1 -- 172.21.15.26:0/3776151304 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f59600ca770 con 0x7f5960160a20 2023-12-19T18:17:03.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.561+0000 7f595d7fa640 1 -- 172.21.15.26:0/3776151304 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v217) v1 ==== 103+0+11 (secure 0 0 0) 0x7f59600ca770 con 0x7f5960160a20 2023-12-19T18:17:03.564 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:17:03.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.561+0000 7f5965c62640 1 -- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5944065d30 msgr2=0x7f59440681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:03.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.561+0000 7f5965c62640 1 --2- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5944065d30 0x7f59440681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.561+0000 7f5965c62640 1 -- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5960160a20 msgr2=0x7f59600d3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.561+0000 7f5965c62640 1 --2- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5960160a20 0x7f59600d3090 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f595800eba0 tx=0x7f595800c490 comp rx=0 tx=0).stop 2023-12-19T18:17:03.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.561+0000 7f5965c62640 1 -- 172.21.15.26:0/3776151304 shutdown_connections 2023-12-19T18:17:03.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.561+0000 7f5965c62640 1 --2- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5944065d30 0x7f59440681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.561+0000 7f5965c62640 1 --2- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5960160a20 0x7f59600d3090 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.561+0000 7f5965c62640 1 --2- 172.21.15.26:0/3776151304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f596015d070 0x7f59600d2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.561+0000 7f5965c62640 1 --2- 172.21.15.26:0/3776151304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f596015cc90 0x7f59601561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.561+0000 7f5965c62640 1 -- 172.21.15.26:0/3776151304 >> 172.21.15.26:0/3776151304 conn(0x7f596006a320 msgr2=0x7f5960067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:03.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.561+0000 7f5965c62640 1 -- 172.21.15.26:0/3776151304 shutdown_connections 2023-12-19T18:17:03.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.561+0000 7f5965c62640 1 -- 172.21.15.26:0/3776151304 wait complete. 2023-12-19T18:17:03.589 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:17:03.647 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:03.645+0000 7f9c5762c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:17:03.653 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:03.653+0000 7f8fe7b1a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:17:03.722 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:03.717+0000 7f9c5762c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:17:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.729+0000 7fc36a41f640 1 Processor -- start 2023-12-19T18:17:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.729+0000 7fc36a41f640 1 -- start start 2023-12-19T18:17:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc36a41f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc36415a7c0 0x7fc36415ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc36a41f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc36406f9d0 0x7fc3640ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc36a41f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36415e2f0 con 0x7fc3640c4070 2023-12-19T18:17:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc36a41f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3640c5b40 con 0x7fc36406f9d0 2023-12-19T18:17:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc36a41f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3640c5e20 con 0x7fc36415a7c0 2023-12-19T18:17:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc3637fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc36406f9d0 0x7fc3640ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc3637fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc36406f9d0 0x7fc3640ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38338/0 (socket says 172.21.15.26:38338) 2023-12-19T18:17:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc3637fe640 1 -- 172.21.15.26:0/3599566247 learned_addr learned my addr 172.21.15.26:0/3599566247 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:03.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc363fff640 1 --2- 172.21.15.26:0/3599566247 >> v2:172.21.15.160:3300/0 conn(0x7fc36415a7c0 0x7fc36415ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc362ffd640 1 -- 172.21.15.26:0/3599566247 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662356511 0 0) 0x7fc36415e2f0 con 0x7fc3640c4070 2023-12-19T18:17:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc362ffd640 1 -- 172.21.15.26:0/3599566247 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc34c003530 con 0x7fc3640c4070 2023-12-19T18:17:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc3637fe640 1 -- 172.21.15.26:0/3599566247 >> v2:172.21.15.160:3300/0 conn(0x7fc36415a7c0 msgr2=0x7fc36415ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc3637fe640 1 --2- 172.21.15.26:0/3599566247 >> v2:172.21.15.160:3300/0 conn(0x7fc36415a7c0 0x7fc36415ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc3637fe640 1 -- 172.21.15.26:0/3599566247 >> v1:172.21.15.160:6789/0 conn(0x7fc3640c4070 legacy=0x7fc3640cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.733+0000 7fc3637fe640 1 -- 172.21.15.26:0/3599566247 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3640ce770 con 0x7fc36406f9d0 2023-12-19T18:17:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.737+0000 7fc3637fe640 1 --2- 172.21.15.26:0/3599566247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc36406f9d0 0x7fc3640ceff0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fc36406a910 tx=0x7fc35801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702dca37dbbba9b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.737+0000 7fc362ffd640 1 -- 172.21.15.26:0/3599566247 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3580ac640 con 0x7fc36406f9d0 2023-12-19T18:17:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.737+0000 7fc362ffd640 1 -- 172.21.15.26:0/3599566247 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3580ac7a0 con 0x7fc36406f9d0 2023-12-19T18:17:03.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.737+0000 7fc362ffd640 1 -- 172.21.15.26:0/3599566247 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3580acb10 con 0x7fc36406f9d0 2023-12-19T18:17:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.737+0000 7fc36a41f640 1 -- 172.21.15.26:0/3599566247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc36406f9d0 msgr2=0x7fc3640ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.737+0000 7fc36a41f640 1 --2- 172.21.15.26:0/3599566247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc36406f9d0 0x7fc3640ceff0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fc36406a910 tx=0x7fc35801c4d0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.737+0000 7fc36a41f640 1 -- 172.21.15.26:0/3599566247 shutdown_connections 2023-12-19T18:17:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.737+0000 7fc36a41f640 1 --2- 172.21.15.26:0/3599566247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc36406f9d0 0x7fc3640ceff0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.737+0000 7fc36a41f640 1 --2- 172.21.15.26:0/3599566247 >> v2:172.21.15.160:3300/0 conn(0x7fc36415a7c0 0x7fc36415ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.737+0000 7fc36a41f640 1 -- 172.21.15.26:0/3599566247 >> 172.21.15.26:0/3599566247 conn(0x7fc36406d5d0 msgr2=0x7fc364069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.737+0000 7fc36a41f640 1 -- 172.21.15.26:0/3599566247 shutdown_connections 2023-12-19T18:17:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.737+0000 7fc36a41f640 1 -- 172.21.15.26:0/3599566247 wait complete. 2023-12-19T18:17:03.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.737+0000 7fc36a41f640 1 Processor -- start 2023-12-19T18:17:03.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc36a41f640 1 -- start start 2023-12-19T18:17:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc36a41f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc36406f9d0 0x7fc36415e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc36a41f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc3640c4070 0x7fc364151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc36a41f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc36415a7c0 0x7fc364152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc36a41f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc364158ff0 con 0x7fc36415a7c0 2023-12-19T18:17:03.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc36a41f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3641592d0 con 0x7fc36406f9d0 2023-12-19T18:17:03.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc36a41f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3641595b0 con 0x7fc3640c4070 2023-12-19T18:17:03.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc3689bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc36415a7c0 0x7fc364152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc3689bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc36415a7c0 0x7fc364152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38348/0 (socket says 172.21.15.26:38348) 2023-12-19T18:17:03.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc3689bc640 1 -- 172.21.15.26:0/1582945426 learned_addr learned my addr 172.21.15.26:0/1582945426 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:03.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc363fff640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc36406f9d0 0x7fc36415e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc3637fe640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc3640c4070 0x7fc364151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc3689bc640 1 -- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc3640c4070 msgr2=0x7fc364151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc3689bc640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc3640c4070 0x7fc364151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc3689bc640 1 -- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc36406f9d0 msgr2=0x7fc36415e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc3689bc640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc36406f9d0 0x7fc36415e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc3689bc640 1 -- 172.21.15.26:0/1582945426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3580094d0 con 0x7fc36415a7c0 2023-12-19T18:17:03.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc3637fe640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc3640c4070 0x7fc364151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc3689bc640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc36415a7c0 0x7fc364152140 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00c790 tx=0x7fc35c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc360ff9640 1 -- 172.21.15.26:0/1582945426 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00cea0 con 0x7fc36415a7c0 2023-12-19T18:17:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc360ff9640 1 -- 172.21.15.26:0/1582945426 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c003b80 con 0x7fc36415a7c0 2023-12-19T18:17:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc36a41f640 1 -- 172.21.15.26:0/1582945426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc364159890 con 0x7fc36415a7c0 2023-12-19T18:17:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc360ff9640 1 -- 172.21.15.26:0/1582945426 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00a850 con 0x7fc36415a7c0 2023-12-19T18:17:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc36a41f640 1 -- 172.21.15.26:0/1582945426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36415ee70 con 0x7fc36415a7c0 2023-12-19T18:17:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.741+0000 7fc36a41f640 1 -- 172.21.15.26:0/1582945426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36415f060 con 0x7fc36415a7c0 2023-12-19T18:17:03.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.745+0000 7fc360ff9640 1 -- 172.21.15.26:0/1582945426 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 124) v1 ==== 85489+0+0 (secure 0 0 0) 0x7fc35c00a9b0 con 0x7fc36415a7c0 2023-12-19T18:17:03.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.745+0000 7fc360ff9640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc350065d30 0x7fc3500681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.745+0000 7fc360ff9640 1 -- 172.21.15.26:0/1582945426 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7fc35c083b10 con 0x7fc36415a7c0 2023-12-19T18:17:03.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.745+0000 7fc363fff640 1 -- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc350065d30 msgr2=0x7fc3500681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:03.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.745+0000 7fc363fff640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc350065d30 0x7fc3500681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:03.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.749+0000 7fc360ff9640 1 -- 172.21.15.26:0/1582945426 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc36415f060 con 0x7fc36415a7c0 2023-12-19T18:17:03.870 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:03.865+0000 7f9c5762c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:17:03.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.921+0000 7fc36a41f640 1 -- 172.21.15.26:0/1582945426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc36415f060 con 0x7fc36415a7c0 2023-12-19T18:17:03.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.921+0000 7fc360ff9640 1 -- 172.21.15.26:0/1582945426 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v217) v1 ==== 80+0+467 (secure 0 0 0) 0x7fc36415f060 con 0x7fc36415a7c0 2023-12-19T18:17:03.926 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1134319862 2023-12-20T18:16:55.864876+0000 2023-12-19T18:17:03.926 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3401801452 2023-12-20T18:16:55.864876+0000 2023-12-19T18:17:03.926 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/765362777 2023-12-20T18:16:55.864876+0000 2023-12-19T18:17:03.926 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1696607504 2023-12-20T18:16:55.864876+0000 2023-12-19T18:17:03.926 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6833/48394 2023-12-20T18:16:55.864876+0000 2023-12-19T18:17:03.926 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6832/48394 2023-12-20T18:16:55.864876+0000 2023-12-19T18:17:03.926 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2355423136 2023-12-20T18:16:55.864876+0000 2023-12-19T18:17:03.926 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3498637577 2023-12-20T18:16:55.864876+0000 2023-12-19T18:17:03.927 INFO:teuthology.orchestra.run.smithi026.stderr:listed 8 entries 2023-12-19T18:17:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.925+0000 7fc36a41f640 1 -- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc350065d30 msgr2=0x7fc3500681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.925+0000 7fc36a41f640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc350065d30 0x7fc3500681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.925+0000 7fc36a41f640 1 -- 172.21.15.26:0/1582945426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc36415a7c0 msgr2=0x7fc364152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.925+0000 7fc36a41f640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc36415a7c0 0x7fc364152140 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00c790 tx=0x7fc35c00cc50 comp rx=0 tx=0).stop 2023-12-19T18:17:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.925+0000 7fc36a41f640 1 -- 172.21.15.26:0/1582945426 shutdown_connections 2023-12-19T18:17:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.925+0000 7fc36a41f640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc350065d30 0x7fc3500681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.925+0000 7fc36a41f640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc36415a7c0 0x7fc364152140 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.925+0000 7fc36a41f640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc3640c4070 0x7fc364151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.925+0000 7fc36a41f640 1 --2- 172.21.15.26:0/1582945426 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc36406f9d0 0x7fc36415e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.925+0000 7fc36a41f640 1 -- 172.21.15.26:0/1582945426 >> 172.21.15.26:0/1582945426 conn(0x7fc36406d5d0 msgr2=0x7fc36415da80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.925+0000 7fc36a41f640 1 -- 172.21.15.26:0/1582945426 shutdown_connections 2023-12-19T18:17:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:03.925+0000 7fc36a41f640 1 -- 172.21.15.26:0/1582945426 wait complete. 2023-12-19T18:17:03.948 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T18:17:03.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:03.948 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-19T18:17:03.979 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T18:17:03.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:03.979 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:17:04.032 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:17:04.032 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:04.042 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:17:04.042 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:17:04.042 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:04.085 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:17:04.085 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:17:04.086 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:04.095 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:17:04.096 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:04.201 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.201+0000 7f8fe7b1a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:17:04.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.273+0000 7f80b79f3640 1 Processor -- start 2023-12-19T18:17:04.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.273+0000 7f80b79f3640 1 -- start start 2023-12-19T18:17:04.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.273+0000 7f80b79f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80b015cda0 0x7f80b0160920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.273+0000 7f80b79f3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80b00ce030 0x7f80b00d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.273+0000 7f80b79f3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b00d0d40 con 0x7f80b00b85e0 2023-12-19T18:17:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.273+0000 7f80b79f3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b0161150 con 0x7f80b00ce030 2023-12-19T18:17:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.273+0000 7f80b79f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b0161430 con 0x7f80b015cda0 2023-12-19T18:17:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.273+0000 7f80b5f90640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80b00ce030 0x7f80b00d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.273+0000 7f80b5f90640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80b00ce030 0x7f80b00d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38362/0 (socket says 172.21.15.26:38362) 2023-12-19T18:17:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.273+0000 7f80b5f90640 1 -- 172.21.15.26:0/2669453545 learned_addr learned my addr 172.21.15.26:0/2669453545 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:04.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.273+0000 7f80b578f640 1 --2- 172.21.15.26:0/2669453545 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80b015cda0 0x7f80b0160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.277+0000 7f80b5f90640 1 -- 172.21.15.26:0/2669453545 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80b015cda0 msgr2=0x7f80b0160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:04.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.277+0000 7f80b5f90640 1 --2- 172.21.15.26:0/2669453545 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80b015cda0 0x7f80b0160920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.277+0000 7f80b5f90640 1 -- 172.21.15.26:0/2669453545 >> v1:172.21.15.160:6789/0 conn(0x7f80b00b85e0 legacy=0x7f80b00cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.277+0000 7f80b5f90640 1 -- 172.21.15.26:0/2669453545 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b0161710 con 0x7f80b00ce030 2023-12-19T18:17:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.277+0000 7f80b5f90640 1 --2- 172.21.15.26:0/2669453545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00ce030 0x7f80b00d1520 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009e50 tx=0x7f80a801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b741fd9fbde72e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.277+0000 7f80a7fff640 1 -- 172.21.15.26:0/2669453545 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a80a5890 con 0x7f80b00ce030 2023-12-19T18:17:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.277+0000 7f80a7fff640 1 -- 172.21.15.26:0/2669453545 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a80a59f0 con 0x7f80b00ce030 2023-12-19T18:17:04.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.277+0000 7f80a7fff640 1 -- 172.21.15.26:0/2669453545 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a80a5d80 con 0x7f80b00ce030 2023-12-19T18:17:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 -- 172.21.15.26:0/2669453545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00ce030 msgr2=0x7f80b00d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 --2- 172.21.15.26:0/2669453545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00ce030 0x7f80b00d1520 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009e50 tx=0x7f80a801c9e0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 -- 172.21.15.26:0/2669453545 shutdown_connections 2023-12-19T18:17:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 --2- 172.21.15.26:0/2669453545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00ce030 0x7f80b00d1520 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 --2- 172.21.15.26:0/2669453545 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80b015cda0 0x7f80b0160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 -- 172.21.15.26:0/2669453545 >> 172.21.15.26:0/2669453545 conn(0x7f80b00658e0 msgr2=0x7f80b0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 -- 172.21.15.26:0/2669453545 shutdown_connections 2023-12-19T18:17:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 -- 172.21.15.26:0/2669453545 wait complete. 2023-12-19T18:17:04.284 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.285+0000 7f8fe7b1a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:17:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 Processor -- start 2023-12-19T18:17:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 -- start start 2023-12-19T18:17:04.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00b85e0 0x7f80b0158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80b00ce030 0x7f80b014c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80b015cda0 0x7f80b014f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b014db00 con 0x7f80b00b85e0 2023-12-19T18:17:04.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b014ce80 con 0x7f80b00ce030 2023-12-19T18:17:04.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b79f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b014d160 con 0x7f80b015cda0 2023-12-19T18:17:04.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b578f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00b85e0 0x7f80b0158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b578f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00b85e0 0x7f80b0158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38366/0 (socket says 172.21.15.26:38366) 2023-12-19T18:17:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b578f640 1 -- 172.21.15.26:0/4233699674 learned_addr learned my addr 172.21.15.26:0/4233699674 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b5f90640 1 --2- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80b015cda0 0x7f80b014f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.281+0000 7f80b4f8e640 1 --2- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80b00ce030 0x7f80b014c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.285+0000 7f80b578f640 1 -- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80b015cda0 msgr2=0x7f80b014f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.285+0000 7f80b578f640 1 --2- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80b015cda0 0x7f80b014f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.285+0000 7f80b578f640 1 -- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80b00ce030 msgr2=0x7f80b014c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.285+0000 7f80b578f640 1 --2- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80b00ce030 0x7f80b014c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.285+0000 7f80b578f640 1 -- 172.21.15.26:0/4233699674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a8009b30 con 0x7f80b00b85e0 2023-12-19T18:17:04.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.285+0000 7f80b578f640 1 --2- 172.21.15.26:0/4233699674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00b85e0 0x7f80b0158360 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00e870 tx=0x7f80ac00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.285+0000 7f80a5ffb640 1 -- 172.21.15.26:0/4233699674 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac009510 con 0x7f80b00b85e0 2023-12-19T18:17:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.285+0000 7f80a5ffb640 1 -- 172.21.15.26:0/4233699674 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac00eec0 con 0x7f80b00b85e0 2023-12-19T18:17:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.285+0000 7f80a5ffb640 1 -- 172.21.15.26:0/4233699674 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac00cb40 con 0x7f80b00b85e0 2023-12-19T18:17:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.285+0000 7f80b79f3640 1 -- 172.21.15.26:0/4233699674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b014d440 con 0x7f80b00b85e0 2023-12-19T18:17:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.285+0000 7f80b79f3640 1 -- 172.21.15.26:0/4233699674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b0150190 con 0x7f80b00b85e0 2023-12-19T18:17:04.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.285+0000 7f80b79f3640 1 -- 172.21.15.26:0/4233699674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b0074270 con 0x7f80b00b85e0 2023-12-19T18:17:04.290 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.285+0000 7f80a5ffb640 1 -- 172.21.15.26:0/4233699674 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 124) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f80ac00cca0 con 0x7f80b00b85e0 2023-12-19T18:17:04.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.289+0000 7f80a5ffb640 1 --2- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8094065d30 0x7f80940681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.289+0000 7f80a5ffb640 1 -- 172.21.15.26:0/4233699674 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7f80ac083e80 con 0x7f80b00b85e0 2023-12-19T18:17:04.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.289+0000 7f80b4f8e640 1 -- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8094065d30 msgr2=0x7f80940681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:04.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.289+0000 7f80b4f8e640 1 --2- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8094065d30 0x7f80940681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:04.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.293+0000 7f80a5ffb640 1 -- 172.21.15.26:0/4233699674 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f80ac050d20 con 0x7f80b00b85e0 2023-12-19T18:17:04.433 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.433+0000 7f8fe7b1a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:17:04.462 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:04.457+0000 7f9c5762c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:17:04.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80b79f3640 1 -- 172.21.15.26:0/4233699674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80b00d08f0 con 0x7f80b00b85e0 2023-12-19T18:17:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80a5ffb640 1 -- 172.21.15.26:0/4233699674 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5917 (secure 0 0 0) 0x7f80ac017090 con 0x7f80b00b85e0 2023-12-19T18:17:04.489 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:04.489 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3349262044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3349262044},{"type":"v1","addr":"172.21.15.160:6836","nonce":3349262044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3865658274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3865658274},{"type":"v1","addr":"172.21.15.26:6833","nonce":3865658274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:27.798614+0000","modified":"2023-12-19T18:16:31.076294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3981286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3981286992},{"type":"v1","addr":"172.21.15.26:6835","nonce":3981286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:40.537606+0000","modified":"2023-12-19T18:16:43.550176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10605":{"gid":10605,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.160:6837/1294752100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1294752100},{"type":"v1","addr":"172.21.15.160:6837","nonce":1294752100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-19T18:17:04.489 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 102 2023-12-19T18:17:04.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80b79f3640 1 -- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8094065d30 msgr2=0x7f80940681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:04.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80b79f3640 1 --2- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8094065d30 0x7f80940681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80b79f3640 1 -- 172.21.15.26:0/4233699674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00b85e0 msgr2=0x7f80b0158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80b79f3640 1 --2- 172.21.15.26:0/4233699674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00b85e0 0x7f80b0158360 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00e870 tx=0x7f80ac00ec70 comp rx=0 tx=0).stop 2023-12-19T18:17:04.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80b79f3640 1 -- 172.21.15.26:0/4233699674 shutdown_connections 2023-12-19T18:17:04.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80b79f3640 1 --2- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8094065d30 0x7f80940681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80b79f3640 1 --2- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80b015cda0 0x7f80b014f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80b79f3640 1 --2- 172.21.15.26:0/4233699674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80b00ce030 0x7f80b014c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80b79f3640 1 --2- 172.21.15.26:0/4233699674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80b00b85e0 0x7f80b0158360 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80b79f3640 1 -- 172.21.15.26:0/4233699674 >> 172.21.15.26:0/4233699674 conn(0x7f80b00658e0 msgr2=0x7f80b015bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:04.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80b79f3640 1 -- 172.21.15.26:0/4233699674 shutdown_connections 2023-12-19T18:17:04.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.485+0000 7f80b79f3640 1 -- 172.21.15.26:0/4233699674 wait complete. 2023-12-19T18:17:04.511 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:04.517 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.517+0000 7f8fe7b1a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:17:04.552 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:04.549+0000 7f9c5762c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:17:04.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.653+0000 7f5989fc0640 1 Processor -- start 2023-12-19T18:17:04.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f5989fc0640 1 -- start start 2023-12-19T18:17:04.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f5989fc0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5984155030 0x7f598415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f5989fc0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598406c970 con 0x7f59840ce740 2023-12-19T18:17:04.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f5989fc0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598415b1f0 con 0x7f598415f370 2023-12-19T18:17:04.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f5989fc0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598406c790 con 0x7f5984155030 2023-12-19T18:17:04.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f5983fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f59840ce740 0x7f59840ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36558/0 (socket says 172.21.15.26:36558) 2023-12-19T18:17:04.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f5983fff640 1 -- 172.21.15.26:0/1281818326 learned_addr learned my addr 172.21.15.26:0/1281818326 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:04.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f59837fe640 1 --2- 172.21.15.26:0/1281818326 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5984155030 0x7f598415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f59827fc640 1 -- 172.21.15.26:0/1281818326 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 689950243 0 0) 0x7f598406c970 con 0x7f59840ce740 2023-12-19T18:17:04.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f59827fc640 1 -- 172.21.15.26:0/1281818326 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f596c003530 con 0x7f59840ce740 2023-12-19T18:17:04.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f59827fc640 1 -- 172.21.15.26:0/1281818326 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175174227 0 0) 0x7f598415b1f0 con 0x7f598415f370 2023-12-19T18:17:04.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f59827fc640 1 -- 172.21.15.26:0/1281818326 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f598406c970 con 0x7f598415f370 2023-12-19T18:17:04.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f59837fe640 1 -- 172.21.15.26:0/1281818326 >> v1:172.21.15.160:6789/0 conn(0x7f598415f370 legacy=0x7f598406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f59837fe640 1 -- 172.21.15.26:0/1281818326 >> v1:172.21.15.26:6789/0 conn(0x7f59840ce740 legacy=0x7f59840ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.657+0000 7f59837fe640 1 -- 172.21.15.26:0/1281818326 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598415b4d0 con 0x7f5984155030 2023-12-19T18:17:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.661+0000 7f59837fe640 1 --2- 172.21.15.26:0/1281818326 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5984155030 0x7f598415bcd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f59740097f0 tx=0x7f597401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41511c1ee2d705e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:04.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.661+0000 7f59827fc640 1 -- 172.21.15.26:0/1281818326 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740aa610 con 0x7f5984155030 2023-12-19T18:17:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.661+0000 7f59827fc640 1 -- 172.21.15.26:0/1281818326 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59740a6040 con 0x7f5984155030 2023-12-19T18:17:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.661+0000 7f59827fc640 1 -- 172.21.15.26:0/1281818326 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740aa970 con 0x7f5984155030 2023-12-19T18:17:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.661+0000 7f5989fc0640 1 -- 172.21.15.26:0/1281818326 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5984155030 msgr2=0x7f598415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.661+0000 7f5989fc0640 1 --2- 172.21.15.26:0/1281818326 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5984155030 0x7f598415bcd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f59740097f0 tx=0x7f597401c490 comp rx=0 tx=0).stop 2023-12-19T18:17:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.661+0000 7f5989fc0640 1 -- 172.21.15.26:0/1281818326 shutdown_connections 2023-12-19T18:17:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.661+0000 7f5989fc0640 1 --2- 172.21.15.26:0/1281818326 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5984155030 0x7f598415bcd0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.661+0000 7f5989fc0640 1 -- 172.21.15.26:0/1281818326 >> 172.21.15.26:0/1281818326 conn(0x7f59840658e0 msgr2=0x7f5984065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.661+0000 7f5989fc0640 1 -- 172.21.15.26:0/1281818326 shutdown_connections 2023-12-19T18:17:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.661+0000 7f5989fc0640 1 -- 172.21.15.26:0/1281818326 wait complete. 2023-12-19T18:17:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f5989fc0640 1 Processor -- start 2023-12-19T18:17:04.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f5989fc0640 1 -- start start 2023-12-19T18:17:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f5989fc0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59840ce740 0x7f59841494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f5989fc0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5984155030 0x7f59841530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f5989fc0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f598415f370 0x7f598406b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f5989fc0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598414c230 con 0x7f59840ce740 2023-12-19T18:17:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f5989fc0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59840ae0c0 con 0x7f598415f370 2023-12-19T18:17:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f5989fc0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5984153900 con 0x7f5984155030 2023-12-19T18:17:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f59837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59840ce740 0x7f59841494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f59837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59840ce740 0x7f59841494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38382/0 (socket says 172.21.15.26:38382) 2023-12-19T18:17:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f59837fe640 1 -- 172.21.15.26:0/3557682296 learned_addr learned my addr 172.21.15.26:0/3557682296 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f5982ffd640 1 --2- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5984155030 0x7f59841530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f5983fff640 1 --2- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f598415f370 0x7f598406b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f59837fe640 1 -- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5984155030 msgr2=0x7f59841530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f59837fe640 1 --2- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5984155030 0x7f59841530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f59837fe640 1 -- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f598415f370 msgr2=0x7f598406b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f59837fe640 1 --2- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f598415f370 0x7f598406b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f59837fe640 1 -- 172.21.15.26:0/3557682296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5974009420 con 0x7f59840ce740 2023-12-19T18:17:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f59837fe640 1 --2- 172.21.15.26:0/3557682296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59840ce740 0x7f59841494c0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f5974000c10 tx=0x7f597401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f596bfff640 1 -- 172.21.15.26:0/3557682296 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740be410 con 0x7f59840ce740 2023-12-19T18:17:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f596bfff640 1 -- 172.21.15.26:0/3557682296 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59740be570 con 0x7f59840ce740 2023-12-19T18:17:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f596bfff640 1 -- 172.21.15.26:0/3557682296 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740be8e0 con 0x7f59840ce740 2023-12-19T18:17:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f5989fc0640 1 -- 172.21.15.26:0/3557682296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598414a810 con 0x7f59840ce740 2023-12-19T18:17:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f5989fc0640 1 -- 172.21.15.26:0/3557682296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598414add0 con 0x7f59840ce740 2023-12-19T18:17:04.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.665+0000 7f5989fc0640 1 -- 172.21.15.26:0/3557682296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5984074270 con 0x7f59840ce740 2023-12-19T18:17:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.669+0000 7f596bfff640 1 -- 172.21.15.26:0/3557682296 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 124) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f59740bea40 con 0x7f59840ce740 2023-12-19T18:17:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.669+0000 7f596bfff640 1 --2- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5970065d30 0x7f59700681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.669+0000 7f596bfff640 1 -- 172.21.15.26:0/3557682296 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7677+0+0 (secure 0 0 0) 0x7f597411e340 con 0x7f59840ce740 2023-12-19T18:17:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.669+0000 7f5982ffd640 1 -- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5970065d30 msgr2=0x7f59700681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.669+0000 7f5982ffd640 1 --2- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5970065d30 0x7f59700681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.673+0000 7f596bfff640 1 -- 172.21.15.26:0/3557682296 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f59740eb230 con 0x7f59840ce740 2023-12-19T18:17:04.719 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:04.713+0000 7f9c5762c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:17:04.807 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:04.805+0000 7f9c5762c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:17:04.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.833+0000 7f5989fc0640 1 -- 172.21.15.26:0/3557682296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59840ccc30 con 0x7f59840ce740 2023-12-19T18:17:04.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.837+0000 7f596bfff640 1 -- 172.21.15.26:0/3557682296 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v217) v1 ==== 74+0+21487 (secure 0 0 0) 0x7f59740ee850 con 0x7f59840ce740 2023-12-19T18:17:04.840 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.841+0000 7f8fe7b1a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:17:04.841 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:04.841 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":217,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:16:55.864913+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:25.863942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:26.798049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:16:38.226152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:16:39.540217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1134319862":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/3401801452":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/765362777":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/1696607504":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6833/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6832/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/2355423136":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/3498637577":"2023-12-20T18:16:55.864876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:04.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.841+0000 7f5989fc0640 1 -- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5970065d30 msgr2=0x7f59700681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:04.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.841+0000 7f5989fc0640 1 --2- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5970065d30 0x7f59700681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.841+0000 7f5989fc0640 1 -- 172.21.15.26:0/3557682296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59840ce740 msgr2=0x7f59841494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.841+0000 7f5989fc0640 1 --2- 172.21.15.26:0/3557682296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59840ce740 0x7f59841494c0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f5974000c10 tx=0x7f597401acd0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.841+0000 7f5989fc0640 1 -- 172.21.15.26:0/3557682296 shutdown_connections 2023-12-19T18:17:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.841+0000 7f5989fc0640 1 --2- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5970065d30 0x7f59700681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.841+0000 7f5989fc0640 1 --2- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f598415f370 0x7f598406b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.841+0000 7f5989fc0640 1 --2- 172.21.15.26:0/3557682296 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5984155030 0x7f59841530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.841+0000 7f5989fc0640 1 --2- 172.21.15.26:0/3557682296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59840ce740 0x7f59841494c0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.841+0000 7f5989fc0640 1 -- 172.21.15.26:0/3557682296 >> 172.21.15.26:0/3557682296 conn(0x7f59840658e0 msgr2=0x7f598414dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.841+0000 7f5989fc0640 1 -- 172.21.15.26:0/3557682296 shutdown_connections 2023-12-19T18:17:04.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:04.841+0000 7f5989fc0640 1 -- 172.21.15.26:0/3557682296 wait complete. 2023-12-19T18:17:04.863 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:16:25.863942+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '211', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:16:26.798049+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-19T18:16:38.226152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '215', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-19T18:16:39.540217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '216', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:17:04.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:17:04.901 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.901+0000 7f8fe7b1a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:17:04.980 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.981+0000 7f8f70189640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.980 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.981+0000 7f8f70189640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.980 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.981+0000 7f8f70189640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.980 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.981+0000 7f8f70189640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.981 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.981+0000 7f8f70189640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.981 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.981+0000 7f8f7298e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.981 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.981+0000 7f8f7298e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.981 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.981+0000 7f8f7298e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.981 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.981+0000 7f8f7298e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.981 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.981+0000 7f8f7298e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.992 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.993+0000 7f8f7298e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.992 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.993+0000 7f8f7298e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.992 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.993+0000 7f8f7298e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.992 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.993+0000 7f8f7298e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.993 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.993+0000 7f8f7298e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.993 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.993+0000 7f8f70189640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.993 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.993+0000 7f8f70189640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.993 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.993+0000 7f8f70189640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.993 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.993+0000 7f8f70189640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:04.993 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:04.993+0000 7f8f70189640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.061+0000 7fe482518640 1 Processor -- start 2023-12-19T18:17:05.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.061+0000 7fe482518640 1 -- start start 2023-12-19T18:17:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.061+0000 7fe482518640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe47c15ccf0 0x7fe47c0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.061+0000 7fe482518640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe47c15d0d0 0x7fe47c0b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.061+0000 7fe482518640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe47c0b4600 0x7fe47c0b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.061+0000 7fe482518640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47c0b4f90 con 0x7fe47c0b4600 2023-12-19T18:17:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.061+0000 7fe482518640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47c15e200 con 0x7fe47c15ccf0 2023-12-19T18:17:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.061+0000 7fe482518640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0d0aa0 con 0x7fe47c15d0d0 2023-12-19T18:17:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.061+0000 7fe480ab5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe47c0b4600 0x7fe47c0b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.061+0000 7fe480ab5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe47c0b4600 0x7fe47c0b4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38394/0 (socket says 172.21.15.26:38394) 2023-12-19T18:17:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.061+0000 7fe480ab5640 1 -- 172.21.15.26:0/909491883 learned_addr learned my addr 172.21.15.26:0/909491883 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.061+0000 7fe47b7fe640 1 --2- 172.21.15.26:0/909491883 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe47c15d0d0 0x7fe47c0b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.061+0000 7fe47bfff640 1 --2- 172.21.15.26:0/909491883 >> v2:172.21.15.160:3300/0 conn(0x7fe47c15ccf0 0x7fe47c0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.065+0000 7fe480ab5640 1 -- 172.21.15.26:0/909491883 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe47c15d0d0 msgr2=0x7fe47c0b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.065+0000 7fe480ab5640 1 --2- 172.21.15.26:0/909491883 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe47c15d0d0 0x7fe47c0b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.065+0000 7fe480ab5640 1 -- 172.21.15.26:0/909491883 >> v2:172.21.15.160:3300/0 conn(0x7fe47c15ccf0 msgr2=0x7fe47c0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.065+0000 7fe480ab5640 1 --2- 172.21.15.26:0/909491883 >> v2:172.21.15.160:3300/0 conn(0x7fe47c15ccf0 0x7fe47c0d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.065+0000 7fe480ab5640 1 -- 172.21.15.26:0/909491883 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c0d1380 con 0x7fe47c0b4600 2023-12-19T18:17:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.065+0000 7fe480ab5640 1 --2- 172.21.15.26:0/909491883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0b4600 0x7fe47c0b4a50 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0d3c20 tx=0x7fe47401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a5ee96eb120ac2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.065+0000 7fe47affd640 1 -- 172.21.15.26:0/909491883 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4740a5820 con 0x7fe47c0b4600 2023-12-19T18:17:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.065+0000 7fe47affd640 1 -- 172.21.15.26:0/909491883 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4740a5980 con 0x7fe47c0b4600 2023-12-19T18:17:05.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.065+0000 7fe47affd640 1 -- 172.21.15.26:0/909491883 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4740a5d10 con 0x7fe47c0b4600 2023-12-19T18:17:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 -- 172.21.15.26:0/909491883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0b4600 msgr2=0x7fe47c0b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 --2- 172.21.15.26:0/909491883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0b4600 0x7fe47c0b4a50 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fe47c0d3c20 tx=0x7fe47401c9a0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 -- 172.21.15.26:0/909491883 shutdown_connections 2023-12-19T18:17:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 --2- 172.21.15.26:0/909491883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0b4600 0x7fe47c0b4a50 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 --2- 172.21.15.26:0/909491883 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe47c15d0d0 0x7fe47c0b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 --2- 172.21.15.26:0/909491883 >> v2:172.21.15.160:3300/0 conn(0x7fe47c15ccf0 0x7fe47c0d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 -- 172.21.15.26:0/909491883 >> 172.21.15.26:0/909491883 conn(0x7fe47c065ee0 msgr2=0x7fe47c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 -- 172.21.15.26:0/909491883 shutdown_connections 2023-12-19T18:17:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 -- 172.21.15.26:0/909491883 wait complete. 2023-12-19T18:17:05.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 Processor -- start 2023-12-19T18:17:05.073 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 -- start start 2023-12-19T18:17:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0b4600 0x7fe47c154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe47c15ccf0 0x7fe47c0b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe47c15d0d0 0x7fe47c0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0b66d0 con 0x7fe47c0b4600 2023-12-19T18:17:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0b69b0 con 0x7fe47c15d0d0 2023-12-19T18:17:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe482518640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0b6c90 con 0x7fe47c15ccf0 2023-12-19T18:17:05.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe47bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0b4600 0x7fe47c154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe47bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0b4600 0x7fe47c154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38408/0 (socket says 172.21.15.26:38408) 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe47bfff640 1 -- 172.21.15.26:0/3815588441 learned_addr learned my addr 172.21.15.26:0/3815588441 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe480ab5640 1 --2- 172.21.15.26:0/3815588441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe47c15d0d0 0x7fe47c0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe47b7fe640 1 --2- 172.21.15.26:0/3815588441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe47c15ccf0 0x7fe47c0b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe47bfff640 1 -- 172.21.15.26:0/3815588441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe47c15ccf0 msgr2=0x7fe47c0b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe47bfff640 1 --2- 172.21.15.26:0/3815588441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe47c15ccf0 0x7fe47c0b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe47bfff640 1 -- 172.21.15.26:0/3815588441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe47c15d0d0 msgr2=0x7fe47c0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe47bfff640 1 --2- 172.21.15.26:0/3815588441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe47c15d0d0 0x7fe47c0b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.069+0000 7fe47bfff640 1 -- 172.21.15.26:0/3815588441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe474009b30 con 0x7fe47c0b4600 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.073+0000 7fe47bfff640 1 --2- 172.21.15.26:0/3815588441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0b4600 0x7fe47c154220 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00e870 tx=0x7fe46c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.073+0000 7fe478ff9640 1 -- 172.21.15.26:0/3815588441 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c00ee60 con 0x7fe47c0b4600 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.073+0000 7fe478ff9640 1 -- 172.21.15.26:0/3815588441 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c00c940 con 0x7fe47c0b4600 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.073+0000 7fe478ff9640 1 -- 172.21.15.26:0/3815588441 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c00ccf0 con 0x7fe47c0b4600 2023-12-19T18:17:05.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.073+0000 7fe482518640 1 -- 172.21.15.26:0/3815588441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47c0b6f70 con 0x7fe47c0b4600 2023-12-19T18:17:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.073+0000 7fe482518640 1 -- 172.21.15.26:0/3815588441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47c0c8f50 con 0x7fe47c0b4600 2023-12-19T18:17:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.073+0000 7fe482518640 1 -- 172.21.15.26:0/3815588441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47c074270 con 0x7fe47c0b4600 2023-12-19T18:17:05.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.073+0000 7fe478ff9640 1 -- 172.21.15.26:0/3815588441 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 125) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fe46c013470 con 0x7fe47c0b4600 2023-12-19T18:17:05.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.073+0000 7fe478ff9640 1 -- 172.21.15.26:0/3815588441 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 8064+0+0 (secure 0 0 0) 0x7fe46c0834f0 con 0x7fe47c0b4600 2023-12-19T18:17:05.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.077+0000 7fe478ff9640 1 -- 172.21.15.26:0/3815588441 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe46c050a80 con 0x7fe47c0b4600 2023-12-19T18:17:05.161 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:05.157+0000 7f9c5762c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:17:05.228 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:05.225+0000 7f9c5762c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:17:05.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.237+0000 7fe482518640 1 -- 172.21.15.26:0/3815588441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe47c070970 con 0x7fe47c0b4600 2023-12-19T18:17:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.237+0000 7fe478ff9640 1 -- 172.21.15.26:0/3815588441 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v218) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe46c0210a0 con 0x7fe47c0b4600 2023-12-19T18:17:05.243 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:17:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.241+0000 7fe482518640 1 -- 172.21.15.26:0/3815588441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0b4600 msgr2=0x7fe47c154220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.241+0000 7fe482518640 1 --2- 172.21.15.26:0/3815588441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0b4600 0x7fe47c154220 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00e870 tx=0x7fe46c00ed30 comp rx=0 tx=0).stop 2023-12-19T18:17:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.241+0000 7fe482518640 1 -- 172.21.15.26:0/3815588441 shutdown_connections 2023-12-19T18:17:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.241+0000 7fe482518640 1 --2- 172.21.15.26:0/3815588441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe47c15d0d0 0x7fe47c0b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.241+0000 7fe482518640 1 --2- 172.21.15.26:0/3815588441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe47c15ccf0 0x7fe47c0b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.241+0000 7fe482518640 1 --2- 172.21.15.26:0/3815588441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe47c0b4600 0x7fe47c154220 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.241+0000 7fe482518640 1 -- 172.21.15.26:0/3815588441 >> 172.21.15.26:0/3815588441 conn(0x7fe47c065ee0 msgr2=0x7fe47c0cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.241+0000 7fe482518640 1 -- 172.21.15.26:0/3815588441 shutdown_connections 2023-12-19T18:17:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.241+0000 7fe482518640 1 -- 172.21.15.26:0/3815588441 wait complete. 2023-12-19T18:17:05.264 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:17:05.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a57066640 1 Processor -- start 2023-12-19T18:17:05.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a57066640 1 -- start start 2023-12-19T18:17:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a57066640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9a5015a9c0 0x7f9a500706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a57066640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a5015ada0 0x7f9a500b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a57066640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a500b46c0 con 0x7f9a5006ac80 2023-12-19T18:17:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a57066640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a500cc270 con 0x7f9a5015a9c0 2023-12-19T18:17:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a57066640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a50070d90 con 0x7f9a5015ada0 2023-12-19T18:17:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a55603640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9a5006ac80 0x7f9a500b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36566/0 (socket says 172.21.15.26:36566) 2023-12-19T18:17:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a55603640 1 -- 172.21.15.26:0/967732235 learned_addr learned my addr 172.21.15.26:0/967732235 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a54e02640 1 --2- 172.21.15.26:0/967732235 >> v2:172.21.15.160:3300/0 conn(0x7f9a5015a9c0 0x7f9a500706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a4ffff640 1 --2- 172.21.15.26:0/967732235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a5015ada0 0x7f9a500b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a4f7fe640 1 -- 172.21.15.26:0/967732235 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181118079 0 0) 0x7f9a500b46c0 con 0x7f9a5006ac80 2023-12-19T18:17:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a4f7fe640 1 -- 172.21.15.26:0/967732235 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a40003530 con 0x7f9a5006ac80 2023-12-19T18:17:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a54e02640 1 -- 172.21.15.26:0/967732235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a5015ada0 msgr2=0x7f9a500b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a54e02640 1 --2- 172.21.15.26:0/967732235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a5015ada0 0x7f9a500b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a54e02640 1 -- 172.21.15.26:0/967732235 >> v1:172.21.15.26:6789/0 conn(0x7f9a5006ac80 legacy=0x7f9a500b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a4f7fe640 1 -- 172.21.15.26:0/967732235 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4206268042 0 0) 0x7f9a40003530 con 0x7f9a5006ac80 2023-12-19T18:17:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a54e02640 1 -- 172.21.15.26:0/967732235 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a50071070 con 0x7f9a5015a9c0 2023-12-19T18:17:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.409+0000 7f9a4ffff640 1 --2- 172.21.15.26:0/967732235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a5015ada0 0x7f9a500b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:05.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.413+0000 7f9a54e02640 1 --2- 172.21.15.26:0/967732235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5015a9c0 0x7f9a500706d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9a440097c0 tx=0x7f9a4401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd3739bd349a5ef server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:05.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.413+0000 7f9a4f7fe640 1 -- 172.21.15.26:0/967732235 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a440ac620 con 0x7f9a5015a9c0 2023-12-19T18:17:05.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.413+0000 7f9a4f7fe640 1 -- 172.21.15.26:0/967732235 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a440ac780 con 0x7f9a5015a9c0 2023-12-19T18:17:05.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.413+0000 7f9a4f7fe640 1 -- 172.21.15.26:0/967732235 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a440acb10 con 0x7f9a5015a9c0 2023-12-19T18:17:05.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.413+0000 7f9a57066640 1 -- 172.21.15.26:0/967732235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5015a9c0 msgr2=0x7f9a500706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.413+0000 7f9a57066640 1 --2- 172.21.15.26:0/967732235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5015a9c0 0x7f9a500706d0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9a440097c0 tx=0x7f9a4401c3c0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.413+0000 7f9a57066640 1 -- 172.21.15.26:0/967732235 shutdown_connections 2023-12-19T18:17:05.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.413+0000 7f9a57066640 1 --2- 172.21.15.26:0/967732235 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a5015ada0 0x7f9a500b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.413+0000 7f9a57066640 1 --2- 172.21.15.26:0/967732235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5015a9c0 0x7f9a500706d0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.413+0000 7f9a57066640 1 -- 172.21.15.26:0/967732235 >> 172.21.15.26:0/967732235 conn(0x7f9a500ce730 msgr2=0x7f9a500ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:05.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.413+0000 7f9a57066640 1 -- 172.21.15.26:0/967732235 shutdown_connections 2023-12-19T18:17:05.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.413+0000 7f9a57066640 1 -- 172.21.15.26:0/967732235 wait complete. 2023-12-19T18:17:05.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a57066640 1 Processor -- start 2023-12-19T18:17:05.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a57066640 1 -- start start 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a57066640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a5006ac80 0x7f9a50142870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a57066640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a5015a9c0 0x7f9a50159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a57066640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5015ada0 0x7f9a50159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a57066640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a50146190 con 0x7f9a5015a9c0 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a57066640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a501451c0 con 0x7f9a5015ada0 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a57066640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a501453d0 con 0x7f9a5006ac80 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a5015a9c0 0x7f9a50159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a4ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a5015a9c0 0x7f9a50159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38412/0 (socket says 172.21.15.26:38412) 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a4ffff640 1 -- 172.21.15.26:0/1914728161 learned_addr learned my addr 172.21.15.26:0/1914728161 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a54e02640 1 --2- 172.21.15.26:0/1914728161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a5006ac80 0x7f9a50142870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a55603640 1 --2- 172.21.15.26:0/1914728161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5015ada0 0x7f9a50159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a4ffff640 1 -- 172.21.15.26:0/1914728161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a5006ac80 msgr2=0x7f9a50142870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a4ffff640 1 --2- 172.21.15.26:0/1914728161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a5006ac80 0x7f9a50142870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a4ffff640 1 -- 172.21.15.26:0/1914728161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5015ada0 msgr2=0x7f9a50159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a4ffff640 1 --2- 172.21.15.26:0/1914728161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5015ada0 0x7f9a50159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a4ffff640 1 -- 172.21.15.26:0/1914728161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c009420 con 0x7f9a5015a9c0 2023-12-19T18:17:05.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a4ffff640 1 --2- 172.21.15.26:0/1914728161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a5015a9c0 0x7f9a50159640 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00e980 tx=0x7f9a3c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:05.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a4d7fa640 1 -- 172.21.15.26:0/1914728161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c00ca00 con 0x7f9a5015a9c0 2023-12-19T18:17:05.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a4d7fa640 1 -- 172.21.15.26:0/1914728161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c00cb60 con 0x7f9a5015a9c0 2023-12-19T18:17:05.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a57066640 1 -- 172.21.15.26:0/1914728161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a44009420 con 0x7f9a5015a9c0 2023-12-19T18:17:05.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a4d7fa640 1 -- 172.21.15.26:0/1914728161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c013470 con 0x7f9a5015a9c0 2023-12-19T18:17:05.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.417+0000 7f9a57066640 1 -- 172.21.15.26:0/1914728161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a50145990 con 0x7f9a5015a9c0 2023-12-19T18:17:05.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.421+0000 7f9a57066640 1 -- 172.21.15.26:0/1914728161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a50074270 con 0x7f9a5015a9c0 2023-12-19T18:17:05.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.421+0000 7f9a4d7fa640 1 -- 172.21.15.26:0/1914728161 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 125) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f9a3c0135d0 con 0x7f9a5015a9c0 2023-12-19T18:17:05.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.421+0000 7f9a4d7fa640 1 -- 172.21.15.26:0/1914728161 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 8064+0+0 (secure 0 0 0) 0x7f9a3c083890 con 0x7f9a5015a9c0 2023-12-19T18:17:05.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.425+0000 7f9a4d7fa640 1 -- 172.21.15.26:0/1914728161 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9a3c050d50 con 0x7f9a5015a9c0 2023-12-19T18:17:05.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.581+0000 7f9a57066640 1 -- 172.21.15.26:0/1914728161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a500680d0 con 0x7f9a5015a9c0 2023-12-19T18:17:05.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.581+0000 7f9a4d7fa640 1 -- 172.21.15.26:0/1914728161 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v218) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9a3c021020 con 0x7f9a5015a9c0 2023-12-19T18:17:05.586 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:17:05.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.585+0000 7f9a57066640 1 -- 172.21.15.26:0/1914728161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a5015a9c0 msgr2=0x7f9a50159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.585+0000 7f9a57066640 1 --2- 172.21.15.26:0/1914728161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a5015a9c0 0x7f9a50159640 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00e980 tx=0x7f9a3c00ee40 comp rx=0 tx=0).stop 2023-12-19T18:17:05.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.585+0000 7f9a57066640 1 -- 172.21.15.26:0/1914728161 shutdown_connections 2023-12-19T18:17:05.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.585+0000 7f9a57066640 1 --2- 172.21.15.26:0/1914728161 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a5015ada0 0x7f9a50159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.585+0000 7f9a57066640 1 --2- 172.21.15.26:0/1914728161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a5015a9c0 0x7f9a50159640 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.585+0000 7f9a57066640 1 --2- 172.21.15.26:0/1914728161 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a5006ac80 0x7f9a50142870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.585+0000 7f9a57066640 1 -- 172.21.15.26:0/1914728161 >> 172.21.15.26:0/1914728161 conn(0x7f9a500ce730 msgr2=0x7f9a50157a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:05.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.585+0000 7f9a57066640 1 -- 172.21.15.26:0/1914728161 shutdown_connections 2023-12-19T18:17:05.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.585+0000 7f9a57066640 1 -- 172.21.15.26:0/1914728161 wait complete. 2023-12-19T18:17:05.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:17:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.749+0000 7fb84f191640 1 Processor -- start 2023-12-19T18:17:05.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.749+0000 7fb84f191640 1 -- start start 2023-12-19T18:17:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84f191640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb848157ba0 0x7fb84806b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84f191640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb84806b8f0 0x7fb8480ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84f191640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb848158050 con 0x7fb848153f20 2023-12-19T18:17:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84f191640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8480b0600 con 0x7fb84806b8f0 2023-12-19T18:17:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84f191640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84806bd80 con 0x7fb848157ba0 2023-12-19T18:17:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84e990640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb84806b8f0 0x7fb8480ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84e990640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb84806b8f0 0x7fb8480ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38416/0 (socket says 172.21.15.26:38416) 2023-12-19T18:17:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84d98e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb848157ba0 0x7fb84806b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84e990640 1 -- 172.21.15.26:0/1650288175 learned_addr learned my addr 172.21.15.26:0/1650288175 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84e990640 1 -- 172.21.15.26:0/1650288175 >> v2:172.21.15.160:3300/0 conn(0x7fb848157ba0 msgr2=0x7fb84806b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84e990640 1 --2- 172.21.15.26:0/1650288175 >> v2:172.21.15.160:3300/0 conn(0x7fb848157ba0 0x7fb84806b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84e990640 1 -- 172.21.15.26:0/1650288175 >> v1:172.21.15.26:6789/0 conn(0x7fb848153f20 legacy=0x7fb8480b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84e990640 1 -- 172.21.15.26:0/1650288175 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84815a2b0 con 0x7fb84806b8f0 2023-12-19T18:17:05.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.753+0000 7fb84d18d640 1 -- 172.21.15.26:0/1650288175 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240308244 0 0) 0x7fb848158050 con 0x7fb848153f20 2023-12-19T18:17:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84e990640 1 --2- 172.21.15.26:0/1650288175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb84806b8f0 0x7fb8480ae5a0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fb8480cc410 tx=0x7fb84001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4364adba8001f323 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84d18d640 1 -- 172.21.15.26:0/1650288175 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8400a57d0 con 0x7fb84806b8f0 2023-12-19T18:17:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84d18d640 1 -- 172.21.15.26:0/1650288175 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8400a5930 con 0x7fb84806b8f0 2023-12-19T18:17:05.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84d18d640 1 -- 172.21.15.26:0/1650288175 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8400a5cc0 con 0x7fb84806b8f0 2023-12-19T18:17:05.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 -- 172.21.15.26:0/1650288175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb84806b8f0 msgr2=0x7fb8480ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 --2- 172.21.15.26:0/1650288175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb84806b8f0 0x7fb8480ae5a0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fb8480cc410 tx=0x7fb84001c910 comp rx=0 tx=0).stop 2023-12-19T18:17:05.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 -- 172.21.15.26:0/1650288175 shutdown_connections 2023-12-19T18:17:05.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 --2- 172.21.15.26:0/1650288175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb84806b8f0 0x7fb8480ae5a0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 --2- 172.21.15.26:0/1650288175 >> v2:172.21.15.160:3300/0 conn(0x7fb848157ba0 0x7fb84806b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 -- 172.21.15.26:0/1650288175 >> 172.21.15.26:0/1650288175 conn(0x7fb8480658e0 msgr2=0x7fb848065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:05.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 -- 172.21.15.26:0/1650288175 shutdown_connections 2023-12-19T18:17:05.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 -- 172.21.15.26:0/1650288175 wait complete. 2023-12-19T18:17:05.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 Processor -- start 2023-12-19T18:17:05.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 -- start start 2023-12-19T18:17:05.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb84806b8f0 0x7fb8481401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb848153f20 0x7fb8480adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb848157ba0 0x7fb8480a8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8480ad720 con 0x7fb848153f20 2023-12-19T18:17:05.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8481f3fc0 con 0x7fb84806b8f0 2023-12-19T18:17:05.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.757+0000 7fb84f191640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8480acd80 con 0x7fb848157ba0 2023-12-19T18:17:05.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84d98e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb848153f20 0x7fb8480adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84d98e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb848153f20 0x7fb8480adb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38428/0 (socket says 172.21.15.26:38428) 2023-12-19T18:17:05.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84d98e640 1 -- 172.21.15.26:0/875103392 learned_addr learned my addr 172.21.15.26:0/875103392 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:05.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84e990640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb848157ba0 0x7fb8480a8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84d98e640 1 -- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb848157ba0 msgr2=0x7fb8480a8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84d98e640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb848157ba0 0x7fb8480a8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84d98e640 1 -- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb84806b8f0 msgr2=0x7fb8481401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84e18f640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb84806b8f0 0x7fb8481401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84d98e640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb84806b8f0 0x7fb8481401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84d98e640 1 -- 172.21.15.26:0/875103392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb840009b30 con 0x7fb848153f20 2023-12-19T18:17:05.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84e990640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb848157ba0 0x7fb8480a8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:05.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84e18f640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb84806b8f0 0x7fb8481401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:05.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84d98e640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb848153f20 0x7fb8480adb40 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fb83800e930 tx=0x7fb83800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:05.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb83effd640 1 -- 172.21.15.26:0/875103392 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb838009690 con 0x7fb848153f20 2023-12-19T18:17:05.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb83effd640 1 -- 172.21.15.26:0/875103392 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83800ca00 con 0x7fb848153f20 2023-12-19T18:17:05.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb83effd640 1 -- 172.21.15.26:0/875103392 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83800cd70 con 0x7fb848153f20 2023-12-19T18:17:05.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84f191640 1 -- 172.21.15.26:0/875103392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8480ad060 con 0x7fb848153f20 2023-12-19T18:17:05.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84f191640 1 -- 172.21.15.26:0/875103392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8480ae080 con 0x7fb848153f20 2023-12-19T18:17:05.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb84f191640 1 -- 172.21.15.26:0/875103392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb848061930 con 0x7fb848153f20 2023-12-19T18:17:05.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.761+0000 7fb83effd640 1 -- 172.21.15.26:0/875103392 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 125) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fb838013470 con 0x7fb848153f20 2023-12-19T18:17:05.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.765+0000 7fb83effd640 1 -- 172.21.15.26:0/875103392 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 8064+0+0 (secure 0 0 0) 0x7fb838083830 con 0x7fb848153f20 2023-12-19T18:17:05.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.765+0000 7fb83effd640 1 -- 172.21.15.26:0/875103392 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb838050d00 con 0x7fb848153f20 2023-12-19T18:17:05.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.913+0000 7fb83effd640 1 -- 172.21.15.26:0/875103392 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 126) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fb8380507b0 con 0x7fb848153f20 2023-12-19T18:17:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.913+0000 7fb83effd640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb82c01c9b0 0x7fb82c070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:05.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.917+0000 7fb84e18f640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb82c01c9b0 0x7fb82c070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:05.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.917+0000 7fb84e18f640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb82c01c9b0 0x7fb82c070220 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb844000f70 tx=0x7fb844009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb84f191640 1 -- 172.21.15.26:0/875103392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb848159f00 con 0x7fb848153f20 2023-12-19T18:17:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb83effd640 1 -- 172.21.15.26:0/875103392 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v218) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb848159f00 con 0x7fb848153f20 2023-12-19T18:17:05.931 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:17:05.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb84f191640 1 -- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb82c01c9b0 msgr2=0x7fb82c070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb84f191640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb82c01c9b0 0x7fb82c070220 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb844000f70 tx=0x7fb844009040 comp rx=0 tx=0).stop 2023-12-19T18:17:05.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb84f191640 1 -- 172.21.15.26:0/875103392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb848153f20 msgr2=0x7fb8480adb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb84f191640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb848153f20 0x7fb8480adb40 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fb83800e930 tx=0x7fb83800edf0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb84f191640 1 -- 172.21.15.26:0/875103392 shutdown_connections 2023-12-19T18:17:05.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb84f191640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb82c01c9b0 0x7fb82c070220 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb84f191640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb848157ba0 0x7fb8480a8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb84f191640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb848153f20 0x7fb8480adb40 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb84f191640 1 --2- 172.21.15.26:0/875103392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb84806b8f0 0x7fb8481401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb84f191640 1 -- 172.21.15.26:0/875103392 >> 172.21.15.26:0/875103392 conn(0x7fb8480658e0 msgr2=0x7fb8480ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:05.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb84f191640 1 -- 172.21.15.26:0/875103392 shutdown_connections 2023-12-19T18:17:05.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:05.929+0000 7fb84f191640 1 -- 172.21.15.26:0/875103392 wait complete. 2023-12-19T18:17:05.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-19T18:17:06.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfd0803640 1 Processor -- start 2023-12-19T18:17:06.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfd0803640 1 -- start start 2023-12-19T18:17:06.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfd0803640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdfcc1602e0 0x7fdfcc06b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfd0803640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfcc06bd40 con 0x7fdfcc0b2af0 2023-12-19T18:17:06.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfd0803640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfcc06ef20 con 0x7fdfcc071530 2023-12-19T18:17:06.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfd0803640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfcc06f1c0 con 0x7fdfcc1602e0 2023-12-19T18:17:06.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfcb59e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdfcc0b2af0 0x7fdfcc160ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58162/0 (socket says 172.21.15.26:58162) 2023-12-19T18:17:06.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfcb59e640 1 -- 172.21.15.26:0/2356609680 learned_addr learned my addr 172.21.15.26:0/2356609680 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:06.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006474597 0 0) 0x7fdfcc06bd40 con 0x7fdfcc0b2af0 2023-12-19T18:17:06.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfb4003530 con 0x7fdfcc0b2af0 2023-12-19T18:17:06.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfcad9d640 1 --2- 172.21.15.26:0/2356609680 >> v2:172.21.15.160:3300/0 conn(0x7fdfcc1602e0 0x7fdfcc06b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4016958820 0 0) 0x7fdfb4003530 con 0x7fdfcc0b2af0 2023-12-19T18:17:06.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 >> v2:172.21.15.160:3300/0 conn(0x7fdfcc1602e0 msgr2=0x7fdfcc06b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:06.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 --2- 172.21.15.26:0/2356609680 >> v2:172.21.15.160:3300/0 conn(0x7fdfcc1602e0 0x7fdfcc06b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 >> v1:172.21.15.160:6789/0 conn(0x7fdfcc071530 legacy=0x7fdfcc0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc06f4a0 con 0x7fdfcc0b2af0 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fdfc4001a00 con 0x7fdfcc0b2af0 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 >> v1:172.21.15.26:6789/0 conn(0x7fdfcc0b2af0 legacy=0x7fdfcc160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 --2- 172.21.15.26:0/2356609680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfb4003980 0x7fdfb4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 --2- 172.21.15.26:0/2356609680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb40047d0 0x7fdfb4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 --2- 172.21.15.26:0/2356609680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfb4005190 0x7fdfb400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc06f1c0 con 0x7fdfb4003980 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb400c0e0 con 0x7fdfb40047d0 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb400c2a0 con 0x7fdfb4005190 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfcad9d640 1 --2- 172.21.15.26:0/2356609680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfb4003980 0x7fdfb4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfca59c640 1 --2- 172.21.15.26:0/2356609680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb40047d0 0x7fdfb4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfcb59e640 1 --2- 172.21.15.26:0/2356609680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfb4005190 0x7fdfb400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfcad9d640 1 -- 172.21.15.26:0/2356609680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfb4005190 msgr2=0x7fdfb400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfcad9d640 1 --2- 172.21.15.26:0/2356609680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfb4005190 0x7fdfb400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfcad9d640 1 -- 172.21.15.26:0/2356609680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb40047d0 msgr2=0x7fdfb4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:06.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfcad9d640 1 --2- 172.21.15.26:0/2356609680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb40047d0 0x7fdfb4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfcad9d640 1 -- 172.21.15.26:0/2356609680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb4003530 con 0x7fdfb4003980 2023-12-19T18:17:06.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.097+0000 7fdfcb59e640 1 -- 172.21.15.26:0/2356609680 reap_dead start 2023-12-19T18:17:06.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.101+0000 7fdfcad9d640 1 --2- 172.21.15.26:0/2356609680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfb4003980 0x7fdfb4003da0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00ba30 tx=0x7fdfbc01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eecf79a9334f9b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:06.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.101+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0ae630 con 0x7fdfb4003980 2023-12-19T18:17:06.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.101+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc0ae790 con 0x7fdfb4003980 2023-12-19T18:17:06.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.101+0000 7fdfc9d9b640 1 -- 172.21.15.26:0/2356609680 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc0aeb20 con 0x7fdfb4003980 2023-12-19T18:17:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.101+0000 7fdfd0803640 1 -- 172.21.15.26:0/2356609680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfb4003980 msgr2=0x7fdfb4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.101+0000 7fdfd0803640 1 --2- 172.21.15.26:0/2356609680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfb4003980 0x7fdfb4003da0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc00ba30 tx=0x7fdfbc01c6a0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.101+0000 7fdfd0803640 1 -- 172.21.15.26:0/2356609680 shutdown_connections 2023-12-19T18:17:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.101+0000 7fdfd0803640 1 --2- 172.21.15.26:0/2356609680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfb4003980 0x7fdfb4003da0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.101+0000 7fdfd0803640 1 -- 172.21.15.26:0/2356609680 >> 172.21.15.26:0/2356609680 conn(0x7fdfcc0cd3e0 msgr2=0x7fdfcc0cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.101+0000 7fdfd0803640 1 -- 172.21.15.26:0/2356609680 shutdown_connections 2023-12-19T18:17:06.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfd0803640 1 -- 172.21.15.26:0/2356609680 wait complete. 2023-12-19T18:17:06.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfd0803640 1 Processor -- start 2023-12-19T18:17:06.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfd0803640 1 -- start start 2023-12-19T18:17:06.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfd0803640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb4003980 0x7fdfcc0c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfd0803640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfcc1568e0 0x7fdfcc155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfd0803640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc15d4a0 0x7fdfcc15d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfd0803640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc0c8550 con 0x7fdfcc15d4a0 2023-12-19T18:17:06.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfd0803640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc0af360 con 0x7fdfb4003980 2023-12-19T18:17:06.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfd0803640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc0ca230 con 0x7fdfcc1568e0 2023-12-19T18:17:06.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfcb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc15d4a0 0x7fdfcc15d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfcad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb4003980 0x7fdfcc0c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfca59c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfcc1568e0 0x7fdfcc155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfcb59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc15d4a0 0x7fdfcc15d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48884/0 (socket says 172.21.15.26:48884) 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfcad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb4003980 0x7fdfcc0c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:57876/0 (socket says 172.21.15.26:57876) 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfcb59e640 1 -- 172.21.15.26:0/547816413 learned_addr learned my addr 172.21.15.26:0/547816413 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfcb59e640 1 -- 172.21.15.26:0/547816413 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfcc1568e0 msgr2=0x7fdfcc155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfcb59e640 1 --2- 172.21.15.26:0/547816413 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfcc1568e0 0x7fdfcc155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfcb59e640 1 -- 172.21.15.26:0/547816413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb4003980 msgr2=0x7fdfcc0c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfcb59e640 1 --2- 172.21.15.26:0/547816413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb4003980 0x7fdfcc0c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.105+0000 7fdfcb59e640 1 -- 172.21.15.26:0/547816413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc0ca510 con 0x7fdfcc15d4a0 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.109+0000 7fdfcb59e640 1 --2- 172.21.15.26:0/547816413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc15d4a0 0x7fdfcc15d920 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0b0270 tx=0x7fdfc400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.109+0000 7fdfb37fe640 1 -- 172.21.15.26:0/547816413 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400eb10 con 0x7fdfcc15d4a0 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.109+0000 7fdfb37fe640 1 -- 172.21.15.26:0/547816413 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc400ec70 con 0x7fdfcc15d4a0 2023-12-19T18:17:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.109+0000 7fdfd0803640 1 -- 172.21.15.26:0/547816413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfcc0ca7f0 con 0x7fdfcc15d4a0 2023-12-19T18:17:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.109+0000 7fdfb37fe640 1 -- 172.21.15.26:0/547816413 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4013420 con 0x7fdfcc15d4a0 2023-12-19T18:17:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.109+0000 7fdfd0803640 1 -- 172.21.15.26:0/547816413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfcc0c8970 con 0x7fdfcc15d4a0 2023-12-19T18:17:06.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.109+0000 7fdfd0803640 1 -- 172.21.15.26:0/547816413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfcc061930 con 0x7fdfcc15d4a0 2023-12-19T18:17:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.109+0000 7fdfb37fe640 1 -- 172.21.15.26:0/547816413 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 126) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fdfc4013580 con 0x7fdfcc15d4a0 2023-12-19T18:17:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.109+0000 7fdfb37fe640 1 --2- 172.21.15.26:0/547816413 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdfb80658e0 0x7fdfb8067da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.109+0000 7fdfb37fe640 1 -- 172.21.15.26:0/547816413 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 8064+0+0 (secure 0 0 0) 0x7fdfc40837e0 con 0x7fdfcc15d4a0 2023-12-19T18:17:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.109+0000 7fdfcad9d640 1 --2- 172.21.15.26:0/547816413 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdfb80658e0 0x7fdfb8067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.113+0000 7fdfcad9d640 1 --2- 172.21.15.26:0/547816413 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdfb80658e0 0x7fdfb8067da0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0b0000 tx=0x7fdfbc0046d0 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:06.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.113+0000 7fdfb37fe640 1 -- 172.21.15.26:0/547816413 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdfc40546f0 con 0x7fdfcc15d4a0 2023-12-19T18:17:06.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.269+0000 7fdfd0803640 1 -- 172.21.15.26:0/547816413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfcc0b68e0 con 0x7fdfcc15d4a0 2023-12-19T18:17:06.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.269+0000 7fdfb37fe640 1 -- 172.21.15.26:0/547816413 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v218) v1 ==== 107+0+11 (secure 0 0 0) 0x7fdfc4054280 con 0x7fdfcc15d4a0 2023-12-19T18:17:06.275 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:17:06.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.273+0000 7fdfd0803640 1 -- 172.21.15.26:0/547816413 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdfb80658e0 msgr2=0x7fdfb8067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.273+0000 7fdfd0803640 1 --2- 172.21.15.26:0/547816413 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdfb80658e0 0x7fdfb8067da0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0b0000 tx=0x7fdfbc0046d0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.273+0000 7fdfd0803640 1 -- 172.21.15.26:0/547816413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc15d4a0 msgr2=0x7fdfcc15d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.273+0000 7fdfd0803640 1 --2- 172.21.15.26:0/547816413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc15d4a0 0x7fdfcc15d920 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0b0270 tx=0x7fdfc400e530 comp rx=0 tx=0).stop 2023-12-19T18:17:06.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.273+0000 7fdfd0803640 1 -- 172.21.15.26:0/547816413 shutdown_connections 2023-12-19T18:17:06.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.273+0000 7fdfd0803640 1 --2- 172.21.15.26:0/547816413 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdfb80658e0 0x7fdfb8067da0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.273+0000 7fdfd0803640 1 --2- 172.21.15.26:0/547816413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfcc15d4a0 0x7fdfcc15d920 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.273+0000 7fdfd0803640 1 --2- 172.21.15.26:0/547816413 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfcc1568e0 0x7fdfcc155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.273+0000 7fdfd0803640 1 --2- 172.21.15.26:0/547816413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfb4003980 0x7fdfcc0c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.273+0000 7fdfd0803640 1 -- 172.21.15.26:0/547816413 >> 172.21.15.26:0/547816413 conn(0x7fdfcc0cd3e0 msgr2=0x7fdfcc1606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:06.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.273+0000 7fdfd0803640 1 -- 172.21.15.26:0/547816413 shutdown_connections 2023-12-19T18:17:06.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.273+0000 7fdfd0803640 1 -- 172.21.15.26:0/547816413 wait complete. 2023-12-19T18:17:06.296 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-19T18:17:06.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.437+0000 7fc82e734640 1 Processor -- start 2023-12-19T18:17:06.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.437+0000 7fc82e734640 1 -- start start 2023-12-19T18:17:06.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.441+0000 7fc82e734640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc82815ab40 0x7fc82806b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.441+0000 7fc82e734640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc82806f0a0 0x7fc82806b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.441+0000 7fc82e734640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82815af20 con 0x7fc8280afe70 2023-12-19T18:17:06.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.441+0000 7fc82e734640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8280aea90 con 0x7fc82806f0a0 2023-12-19T18:17:06.444 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.441+0000 7fc82e734640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280aed70 con 0x7fc82815ab40 2023-12-19T18:17:06.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.441+0000 7fc82cf31640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc82806f0a0 0x7fc82806b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.441+0000 7fc82cf31640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc82806f0a0 0x7fc82806b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48886/0 (socket says 172.21.15.26:48886) 2023-12-19T18:17:06.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.441+0000 7fc82cf31640 1 -- 172.21.15.26:0/2154325583 learned_addr learned my addr 172.21.15.26:0/2154325583 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:06.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.441+0000 7fc82d732640 1 --2- 172.21.15.26:0/2154325583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc82815ab40 0x7fc82806b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.441+0000 7fc82cf31640 1 -- 172.21.15.26:0/2154325583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc82815ab40 msgr2=0x7fc82806b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:06.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.441+0000 7fc82cf31640 1 --2- 172.21.15.26:0/2154325583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc82815ab40 0x7fc82806b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.441+0000 7fc82cf31640 1 -- 172.21.15.26:0/2154325583 >> v1:172.21.15.160:6789/0 conn(0x7fc8280afe70 legacy=0x7fc82815e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.441+0000 7fc82cf31640 1 -- 172.21.15.26:0/2154325583 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8280af050 con 0x7fc82806f0a0 2023-12-19T18:17:06.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82cf31640 1 --2- 172.21.15.26:0/2154325583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc82806f0a0 0x7fc82806b8f0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fc818009870 tx=0x7fc81801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b82d5b947341c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:06.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc81ffff640 1 -- 172.21.15.26:0/2154325583 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8180a7070 con 0x7fc82806f0a0 2023-12-19T18:17:06.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc81ffff640 1 -- 172.21.15.26:0/2154325583 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8180ab430 con 0x7fc82806f0a0 2023-12-19T18:17:06.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc81ffff640 1 -- 172.21.15.26:0/2154325583 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8180ab7c0 con 0x7fc82806f0a0 2023-12-19T18:17:06.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 -- 172.21.15.26:0/2154325583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc82806f0a0 msgr2=0x7fc82806b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 --2- 172.21.15.26:0/2154325583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc82806f0a0 0x7fc82806b8f0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fc818009870 tx=0x7fc81801c540 comp rx=0 tx=0).stop 2023-12-19T18:17:06.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 -- 172.21.15.26:0/2154325583 shutdown_connections 2023-12-19T18:17:06.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 --2- 172.21.15.26:0/2154325583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc82806f0a0 0x7fc82806b8f0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 --2- 172.21.15.26:0/2154325583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc82815ab40 0x7fc82806b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 -- 172.21.15.26:0/2154325583 >> 172.21.15.26:0/2154325583 conn(0x7fc8280658e0 msgr2=0x7fc828065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:06.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 -- 172.21.15.26:0/2154325583 shutdown_connections 2023-12-19T18:17:06.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 -- 172.21.15.26:0/2154325583 wait complete. 2023-12-19T18:17:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 Processor -- start 2023-12-19T18:17:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 -- start start 2023-12-19T18:17:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc82806f0a0 0x7fc82815b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8280afe70 0x7fc8280b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc82815ab40 0x7fc82814e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280b1760 con 0x7fc82815ab40 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.445+0000 7fc82e734640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82814ee10 con 0x7fc82806f0a0 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82e734640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82814cd70 con 0x7fc8280afe70 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82df33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc82815ab40 0x7fc82814e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82d732640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc82806f0a0 0x7fc82815b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82cf31640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8280afe70 0x7fc8280b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82d732640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc82806f0a0 0x7fc82815b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:57880/0 (socket says 172.21.15.26:57880) 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82cf31640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8280afe70 0x7fc8280b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:44858/0 (socket says 172.21.15.26:44858) 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82d732640 1 -- 172.21.15.26:0/1502518239 learned_addr learned my addr 172.21.15.26:0/1502518239 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82d732640 1 -- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8280afe70 msgr2=0x7fc8280b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82d732640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8280afe70 0x7fc8280b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82d732640 1 -- 172.21.15.26:0/1502518239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc82815ab40 msgr2=0x7fc82814e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82d732640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc82815ab40 0x7fc82814e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82d732640 1 -- 172.21.15.26:0/1502518239 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8180094d0 con 0x7fc82806f0a0 2023-12-19T18:17:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82cf31640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8280afe70 0x7fc8280b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82df33640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc82815ab40 0x7fc82814e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82d732640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc82806f0a0 0x7fc82815b100 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc82400e930 tx=0x7fc82400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc81dffb640 1 -- 172.21.15.26:0/1502518239 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824009690 con 0x7fc82806f0a0 2023-12-19T18:17:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc81dffb640 1 -- 172.21.15.26:0/1502518239 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82400ca00 con 0x7fc82806f0a0 2023-12-19T18:17:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82e734640 1 -- 172.21.15.26:0/1502518239 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82814d050 con 0x7fc82806f0a0 2023-12-19T18:17:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc81dffb640 1 -- 172.21.15.26:0/1502518239 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82400cd70 con 0x7fc82806f0a0 2023-12-19T18:17:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82e734640 1 -- 172.21.15.26:0/1502518239 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82814d330 con 0x7fc82806f0a0 2023-12-19T18:17:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc82e734640 1 -- 172.21.15.26:0/1502518239 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc828061930 con 0x7fc82806f0a0 2023-12-19T18:17:06.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.449+0000 7fc81dffb640 1 -- 172.21.15.26:0/1502518239 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 126) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fc8240059a0 con 0x7fc82806f0a0 2023-12-19T18:17:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.453+0000 7fc81dffb640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc80c0658b0 0x7fc80c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.453+0000 7fc81dffb640 1 -- 172.21.15.26:0/1502518239 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 8064+0+0 (secure 0 0 0) 0x7fc824083970 con 0x7fc82806f0a0 2023-12-19T18:17:06.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.453+0000 7fc82cf31640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc80c0658b0 0x7fc80c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.453+0000 7fc82cf31640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc80c0658b0 0x7fc80c067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc8180af000 tx=0x7fc818008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:06.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.453+0000 7fc81dffb640 1 -- 172.21.15.26:0/1502518239 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc824050e80 con 0x7fc82806f0a0 2023-12-19T18:17:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.609+0000 7fc82e734640 1 -- 172.21.15.26:0/1502518239 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fc82806fa30 con 0x7fc82806f0a0 2023-12-19T18:17:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.613+0000 7fc81dffb640 1 -- 172.21.15.26:0/1502518239 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v218) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc824021070 con 0x7fc82806f0a0 2023-12-19T18:17:06.615 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:17:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.613+0000 7fc82e734640 1 -- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc80c0658b0 msgr2=0x7fc80c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.613+0000 7fc82e734640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc80c0658b0 0x7fc80c067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc8180af000 tx=0x7fc818008040 comp rx=0 tx=0).stop 2023-12-19T18:17:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.613+0000 7fc82e734640 1 -- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc82806f0a0 msgr2=0x7fc82815b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.613+0000 7fc82e734640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc82806f0a0 0x7fc82815b100 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fc82400e930 tx=0x7fc82400edf0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.613+0000 7fc82e734640 1 -- 172.21.15.26:0/1502518239 shutdown_connections 2023-12-19T18:17:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.613+0000 7fc82e734640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc80c0658b0 0x7fc80c067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.613+0000 7fc82e734640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc82815ab40 0x7fc82814e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.613+0000 7fc82e734640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc8280afe70 0x7fc8280b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.613+0000 7fc82e734640 1 --2- 172.21.15.26:0/1502518239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc82806f0a0 0x7fc82815b100 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.613+0000 7fc82e734640 1 -- 172.21.15.26:0/1502518239 >> 172.21.15.26:0/1502518239 conn(0x7fc8280658e0 msgr2=0x7fc82806b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:06.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.613+0000 7fc82e734640 1 -- 172.21.15.26:0/1502518239 shutdown_connections 2023-12-19T18:17:06.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.613+0000 7fc82e734640 1 -- 172.21.15.26:0/1502518239 wait complete. 2023-12-19T18:17:06.636 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:06.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f54a1751640 1 Processor -- start 2023-12-19T18:17:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f54a1751640 1 -- start start 2023-12-19T18:17:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f54a1751640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f549c159d20 0x7f549c0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f54a1751640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c0adde0 con 0x7f549c0cdee0 2023-12-19T18:17:06.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f54a1751640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c0b28d0 con 0x7f549c0cc0a0 2023-12-19T18:17:06.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f54a1751640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c0c9c20 con 0x7f549c159d20 2023-12-19T18:17:06.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f54a0f50640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f549c0cdee0 0x7f549c0b2db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58166/0 (socket says 172.21.15.26:58166) 2023-12-19T18:17:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f54a0f50640 1 -- 172.21.15.26:0/3337713235 learned_addr learned my addr 172.21.15.26:0/3337713235 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f549b7fe640 1 --2- 172.21.15.26:0/3337713235 >> v2:172.21.15.160:3300/0 conn(0x7f549c159d20 0x7f549c0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f549affd640 1 -- 172.21.15.26:0/3337713235 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488697372 0 0) 0x7f549c0adde0 con 0x7f549c0cdee0 2023-12-19T18:17:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f549affd640 1 -- 172.21.15.26:0/3337713235 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5484003530 con 0x7f549c0cdee0 2023-12-19T18:17:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f549affd640 1 -- 172.21.15.26:0/3337713235 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1955011593 0 0) 0x7f5484003530 con 0x7f549c0cdee0 2023-12-19T18:17:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f549b7fe640 1 -- 172.21.15.26:0/3337713235 >> v1:172.21.15.160:6789/0 conn(0x7f549c0cc0a0 legacy=0x7f549c0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f549b7fe640 1 -- 172.21.15.26:0/3337713235 >> v1:172.21.15.26:6789/0 conn(0x7f549c0cdee0 legacy=0x7f549c0b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.777+0000 7f549b7fe640 1 -- 172.21.15.26:0/3337713235 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c0adb00 con 0x7f549c159d20 2023-12-19T18:17:06.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.781+0000 7f549b7fe640 1 --2- 172.21.15.26:0/3337713235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f549c159d20 0x7f549c0ccb90 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f549000ba60 tx=0x7f549001e9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e45182b735772a3b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:06.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.781+0000 7f549affd640 1 -- 172.21.15.26:0/3337713235 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5494001a00 con 0x7f549c159d20 2023-12-19T18:17:06.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.781+0000 7f549affd640 1 -- 172.21.15.26:0/3337713235 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54900a7800 con 0x7f549c159d20 2023-12-19T18:17:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.781+0000 7f549affd640 1 -- 172.21.15.26:0/3337713235 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54900a7b90 con 0x7f549c159d20 2023-12-19T18:17:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.781+0000 7f54a1751640 1 -- 172.21.15.26:0/3337713235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f549c159d20 msgr2=0x7f549c0ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.781+0000 7f54a1751640 1 --2- 172.21.15.26:0/3337713235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f549c159d20 0x7f549c0ccb90 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f549000ba60 tx=0x7f549001e9b0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.781+0000 7f54a1751640 1 -- 172.21.15.26:0/3337713235 shutdown_connections 2023-12-19T18:17:06.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.781+0000 7f54a1751640 1 --2- 172.21.15.26:0/3337713235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f549c159d20 0x7f549c0ccb90 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.781+0000 7f54a1751640 1 -- 172.21.15.26:0/3337713235 >> 172.21.15.26:0/3337713235 conn(0x7f549c06b0f0 msgr2=0x7f549c06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:06.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.781+0000 7f54a1751640 1 -- 172.21.15.26:0/3337713235 shutdown_connections 2023-12-19T18:17:06.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.781+0000 7f54a1751640 1 -- 172.21.15.26:0/3337713235 wait complete. 2023-12-19T18:17:06.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f54a1751640 1 Processor -- start 2023-12-19T18:17:06.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f54a1751640 1 -- start start 2023-12-19T18:17:06.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f54a1751640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549c0cc0a0 0x7f549c0c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f54a1751640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f549c0cdee0 0x7f549c14e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f54a1751640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f549c159d20 0x7f549c0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f54a1751640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0c8d20 con 0x7f549c0cc0a0 2023-12-19T18:17:06.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f54a1751640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c15b800 con 0x7f549c159d20 2023-12-19T18:17:06.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f54a1751640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c15bae0 con 0x7f549c0cdee0 2023-12-19T18:17:06.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f549bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549c0cc0a0 0x7f549c0c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f549bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549c0cc0a0 0x7f549c0c87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48906/0 (socket says 172.21.15.26:48906) 2023-12-19T18:17:06.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f549bfff640 1 -- 172.21.15.26:0/1751468365 learned_addr learned my addr 172.21.15.26:0/1751468365 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:06.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f54a0f50640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f549c159d20 0x7f549c0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f549b7fe640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f549c0cdee0 0x7f549c14e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f549bfff640 1 -- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f549c0cdee0 msgr2=0x7f549c14e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f549bfff640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f549c0cdee0 0x7f549c14e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f549bfff640 1 -- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f549c159d20 msgr2=0x7f549c0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f549bfff640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f549c159d20 0x7f549c0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f549bfff640 1 -- 172.21.15.26:0/1751468365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549000b6c0 con 0x7f549c0cc0a0 2023-12-19T18:17:06.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f549bfff640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549c0cc0a0 0x7f549c0c87e0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f548c00c500 tx=0x7f548c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:06.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f5498ff9640 1 -- 172.21.15.26:0/1751468365 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c005220 con 0x7f549c0cc0a0 2023-12-19T18:17:06.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f5498ff9640 1 -- 172.21.15.26:0/1751468365 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c005380 con 0x7f549c0cc0a0 2023-12-19T18:17:06.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f54a1751640 1 -- 172.21.15.26:0/1751468365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c15bdc0 con 0x7f549c0cc0a0 2023-12-19T18:17:06.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f5498ff9640 1 -- 172.21.15.26:0/1751468365 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c007270 con 0x7f549c0cc0a0 2023-12-19T18:17:06.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f54a1751640 1 -- 172.21.15.26:0/1751468365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c15c380 con 0x7f549c0cc0a0 2023-12-19T18:17:06.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.785+0000 7f54a1751640 1 -- 172.21.15.26:0/1751468365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c067400 con 0x7f549c0cc0a0 2023-12-19T18:17:06.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.789+0000 7f5498ff9640 1 -- 172.21.15.26:0/1751468365 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 126) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f548c0073d0 con 0x7f549c0cc0a0 2023-12-19T18:17:06.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.789+0000 7f5498ff9640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f54880658e0 0x7f5488067da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.789+0000 7f5498ff9640 1 -- 172.21.15.26:0/1751468365 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 8064+0+0 (secure 0 0 0) 0x7f548c082ee0 con 0x7f549c0cc0a0 2023-12-19T18:17:06.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.789+0000 7f549b7fe640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f54880658e0 0x7f5488067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.789+0000 7f549b7fe640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f54880658e0 0x7f5488067da0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f54900b1000 tx=0x7f5490008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:06.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.793+0000 7f5498ff9640 1 -- 172.21.15.26:0/1751468365 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f548c0502e0 con 0x7f549c0cc0a0 2023-12-19T18:17:06.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.985+0000 7f54a1751640 1 -- 172.21.15.26:0/1751468365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f549c0ad5a0 con 0x7f549c0cc0a0 2023-12-19T18:17:06.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.985+0000 7f5498ff9640 1 -- 172.21.15.26:0/1751468365 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5917 (secure 0 0 0) 0x7f548c053900 con 0x7f549c0cc0a0 2023-12-19T18:17:06.989 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:06.989 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3349262044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3349262044},{"type":"v1","addr":"172.21.15.160:6836","nonce":3349262044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3865658274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3865658274},{"type":"v1","addr":"172.21.15.26:6833","nonce":3865658274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:27.798614+0000","modified":"2023-12-19T18:16:31.076294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3981286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3981286992},{"type":"v1","addr":"172.21.15.26:6835","nonce":3981286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:40.537606+0000","modified":"2023-12-19T18:16:43.550176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10605":{"gid":10605,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.160:6837/1294752100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1294752100},{"type":"v1","addr":"172.21.15.160:6837","nonce":1294752100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-19T18:17:06.989 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 102 2023-12-19T18:17:06.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.989+0000 7f54a1751640 1 -- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f54880658e0 msgr2=0x7f5488067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.989+0000 7f54a1751640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f54880658e0 0x7f5488067da0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f54900b1000 tx=0x7f5490008040 comp rx=0 tx=0).stop 2023-12-19T18:17:06.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.989+0000 7f54a1751640 1 -- 172.21.15.26:0/1751468365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549c0cc0a0 msgr2=0x7f549c0c87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.989+0000 7f54a1751640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549c0cc0a0 0x7f549c0c87e0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f548c00c500 tx=0x7f548c00c9c0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.989+0000 7f54a1751640 1 -- 172.21.15.26:0/1751468365 shutdown_connections 2023-12-19T18:17:06.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.989+0000 7f54a1751640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f54880658e0 0x7f5488067da0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.989+0000 7f54a1751640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f549c159d20 0x7f549c0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.989+0000 7f54a1751640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f549c0cdee0 0x7f549c14e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.989+0000 7f54a1751640 1 --2- 172.21.15.26:0/1751468365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f549c0cc0a0 0x7f549c0c87e0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.989+0000 7f54a1751640 1 -- 172.21.15.26:0/1751468365 >> 172.21.15.26:0/1751468365 conn(0x7f549c06b0f0 msgr2=0x7f549c0716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:06.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.989+0000 7f54a1751640 1 -- 172.21.15.26:0/1751468365 shutdown_connections 2023-12-19T18:17:06.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:06.989+0000 7f54a1751640 1 -- 172.21.15.26:0/1751468365 wait complete. 2023-12-19T18:17:07.011 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:07.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2daf48c640 1 Processor -- start 2023-12-19T18:17:07.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2daf48c640 1 -- start start 2023-12-19T18:17:07.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2daf48c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2da8154220 0x7f2da815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2daf48c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2da806b3d0 0x7f2da80ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2daf48c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2da80cec40 0x7f2da815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2daf48c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da80cf0d0 con 0x7f2da80cec40 2023-12-19T18:17:07.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2daf48c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da806bc30 con 0x7f2da806b3d0 2023-12-19T18:17:07.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2daf48c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80cc6e0 con 0x7f2da8154220 2023-12-19T18:17:07.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2dada29640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2da80cec40 0x7f2da815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2dada29640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2da80cec40 0x7f2da815d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48912/0 (socket says 172.21.15.26:48912) 2023-12-19T18:17:07.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2dada29640 1 -- 172.21.15.26:0/1388559450 learned_addr learned my addr 172.21.15.26:0/1388559450 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:07.175 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2daca27640 1 --2- 172.21.15.26:0/1388559450 >> v2:172.21.15.160:3300/0 conn(0x7f2da806b3d0 0x7f2da80ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2dad228640 1 --2- 172.21.15.26:0/1388559450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2da8154220 0x7f2da815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2dada29640 1 -- 172.21.15.26:0/1388559450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2da8154220 msgr2=0x7f2da815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2dada29640 1 --2- 172.21.15.26:0/1388559450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2da8154220 0x7f2da815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2dada29640 1 -- 172.21.15.26:0/1388559450 >> v2:172.21.15.160:3300/0 conn(0x7f2da806b3d0 msgr2=0x7f2da80ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2dada29640 1 --2- 172.21.15.26:0/1388559450 >> v2:172.21.15.160:3300/0 conn(0x7f2da806b3d0 0x7f2da80ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2dada29640 1 -- 172.21.15.26:0/1388559450 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da815afc0 con 0x7f2da80cec40 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.153+0000 7f2dad228640 1 --2- 172.21.15.26:0/1388559450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2da8154220 0x7f2da815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.157+0000 7f2dada29640 1 --2- 172.21.15.26:0/1388559450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2da80cec40 0x7f2da815d5f0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f2da80cc430 tx=0x7f2da001c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08cf755eb53e745 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.157+0000 7f2d9ffff640 1 -- 172.21.15.26:0/1388559450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da00a5560 con 0x7f2da80cec40 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.157+0000 7f2d9ffff640 1 -- 172.21.15.26:0/1388559450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da00a56c0 con 0x7f2da80cec40 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.157+0000 7f2d9ffff640 1 -- 172.21.15.26:0/1388559450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da00a5a50 con 0x7f2da80cec40 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.157+0000 7f2daf48c640 1 -- 172.21.15.26:0/1388559450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2da80cec40 msgr2=0x7f2da815d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.157+0000 7f2daf48c640 1 --2- 172.21.15.26:0/1388559450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2da80cec40 0x7f2da815d5f0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f2da80cc430 tx=0x7f2da001c6e0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.157+0000 7f2daf48c640 1 -- 172.21.15.26:0/1388559450 shutdown_connections 2023-12-19T18:17:07.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.157+0000 7f2daf48c640 1 --2- 172.21.15.26:0/1388559450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2da80cec40 0x7f2da815d5f0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.157+0000 7f2daf48c640 1 --2- 172.21.15.26:0/1388559450 >> v2:172.21.15.160:3300/0 conn(0x7f2da806b3d0 0x7f2da80ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.157+0000 7f2daf48c640 1 --2- 172.21.15.26:0/1388559450 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2da8154220 0x7f2da815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.157+0000 7f2daf48c640 1 -- 172.21.15.26:0/1388559450 >> 172.21.15.26:0/1388559450 conn(0x7f2da80658e0 msgr2=0x7f2da8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.157+0000 7f2daf48c640 1 -- 172.21.15.26:0/1388559450 shutdown_connections 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.157+0000 7f2daf48c640 1 -- 172.21.15.26:0/1388559450 wait complete. 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daf48c640 1 Processor -- start 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daf48c640 1 -- start start 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daf48c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2da806b3d0 0x7f2da81527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daf48c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2da80cec40 0x7f2da80cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daf48c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2da8154220 0x7f2da81f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daf48c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80b1b00 con 0x7f2da80cec40 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daf48c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80b1820 con 0x7f2da806b3d0 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daf48c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da80b1f80 con 0x7f2da8154220 2023-12-19T18:17:07.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daca27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2da80cec40 0x7f2da80cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daca27640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2da80cec40 0x7f2da80cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48916/0 (socket says 172.21.15.26:48916) 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daca27640 1 -- 172.21.15.26:0/2823878461 learned_addr learned my addr 172.21.15.26:0/2823878461 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2dad228640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2da806b3d0 0x7f2da81527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2dada29640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2da8154220 0x7f2da81f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daca27640 1 -- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2da8154220 msgr2=0x7f2da81f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daca27640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2da8154220 0x7f2da81f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daca27640 1 -- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2da806b3d0 msgr2=0x7f2da81527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daca27640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2da806b3d0 0x7f2da81527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daca27640 1 -- 172.21.15.26:0/2823878461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da0009b30 con 0x7f2da80cec40 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2dada29640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2da8154220 0x7f2da81f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daca27640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2da80cec40 0x7f2da80cb450 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800e870 tx=0x7f2d9800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2d9dffb640 1 -- 172.21.15.26:0/2823878461 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9800ee60 con 0x7f2da80cec40 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2d9dffb640 1 -- 172.21.15.26:0/2823878461 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9800c940 con 0x7f2da80cec40 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daf48c640 1 -- 172.21.15.26:0/2823878461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da80b2260 con 0x7f2da80cec40 2023-12-19T18:17:07.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2d9dffb640 1 -- 172.21.15.26:0/2823878461 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9800ccf0 con 0x7f2da80cec40 2023-12-19T18:17:07.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.161+0000 7f2daf48c640 1 -- 172.21.15.26:0/2823878461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da80cb990 con 0x7f2da80cec40 2023-12-19T18:17:07.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.165+0000 7f2daf48c640 1 -- 172.21.15.26:0/2823878461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da80b1de0 con 0x7f2da80cec40 2023-12-19T18:17:07.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.165+0000 7f2d9dffb640 1 -- 172.21.15.26:0/2823878461 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 127) v1 ==== 85376+0+0 (secure 0 0 0) 0x7f2d98013470 con 0x7f2da80cec40 2023-12-19T18:17:07.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.165+0000 7f2d9dffb640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2d8c065c10 0x7f2d8c0680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.165+0000 7f2d9dffb640 1 -- 172.21.15.26:0/2823878461 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 8064+0+0 (secure 0 0 0) 0x7f2d98083d80 con 0x7f2da80cec40 2023-12-19T18:17:07.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.165+0000 7f2dad228640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2d8c065c10 0x7f2d8c0680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.165+0000 7f2dad228640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2d8c065c10 0x7f2d8c0680d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2da815d270 tx=0x7f2da4008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:07.179 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.169+0000 7f2d9dffb640 1 -- 172.21.15.26:0/2823878461 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2da80b1de0 con 0x7f2da80cec40 2023-12-19T18:17:07.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.329+0000 7f2daf48c640 1 -- 172.21.15.26:0/2823878461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2da80b1de0 con 0x7f2da80cec40 2023-12-19T18:17:07.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.329+0000 7f2d9dffb640 1 -- 172.21.15.26:0/2823878461 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v218) v1 ==== 74+0+22054 (secure 0 0 0) 0x7f2da80b1de0 con 0x7f2da80cec40 2023-12-19T18:17:07.334 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:07.334 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":218,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:04.905457+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:25.863942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:26.798049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:16:38.226152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:16:39.540217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1665633474":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3561084215":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1351823889":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3136172808":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3161716183":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/2592629251":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1089029505":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1544351773":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1134319862":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/4291270549":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3401801452":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/765362777":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/1696607504":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6833/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6832/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/2355423136":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/3498637577":"2023-12-20T18:16:55.864876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.333+0000 7f2daf48c640 1 -- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2d8c065c10 msgr2=0x7f2d8c0680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.333+0000 7f2daf48c640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2d8c065c10 0x7f2d8c0680d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2da815d270 tx=0x7f2da4008040 comp rx=0 tx=0).stop 2023-12-19T18:17:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.333+0000 7f2daf48c640 1 -- 172.21.15.26:0/2823878461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2da80cec40 msgr2=0x7f2da80cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.333+0000 7f2daf48c640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2da80cec40 0x7f2da80cb450 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800e870 tx=0x7f2d9800ed30 comp rx=0 tx=0).stop 2023-12-19T18:17:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.333+0000 7f2daf48c640 1 -- 172.21.15.26:0/2823878461 shutdown_connections 2023-12-19T18:17:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.333+0000 7f2daf48c640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2d8c065c10 0x7f2d8c0680d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.333+0000 7f2daf48c640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2da8154220 0x7f2da81f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.333+0000 7f2daf48c640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2da80cec40 0x7f2da80cb450 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.333+0000 7f2daf48c640 1 --2- 172.21.15.26:0/2823878461 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2da806b3d0 0x7f2da81527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.333+0000 7f2daf48c640 1 -- 172.21.15.26:0/2823878461 >> 172.21.15.26:0/2823878461 conn(0x7f2da80658e0 msgr2=0x7f2da80ae360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.333+0000 7f2daf48c640 1 -- 172.21.15.26:0/2823878461 shutdown_connections 2023-12-19T18:17:07.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.333+0000 7f2daf48c640 1 -- 172.21.15.26:0/2823878461 wait complete. 2023-12-19T18:17:07.356 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:17:07.356 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:07.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e9fc0640 1 Processor -- start 2023-12-19T18:17:07.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e9fc0640 1 -- start start 2023-12-19T18:17:07.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e9fc0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6e4155030 0x7ff6e415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e9fc0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff6e40ce740 0x7ff6e40cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e9fc0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e406c970 con 0x7ff6e415f370 2023-12-19T18:17:07.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e9fc0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e406c790 con 0x7ff6e40ce740 2023-12-19T18:17:07.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e9fc0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40cf130 con 0x7ff6e4155030 2023-12-19T18:17:07.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e37fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6e4155030 0x7ff6e415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e2ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7ff6e415f370 0x7ff6e406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:57496/0 (socket says 172.21.15.26:57496) 2023-12-19T18:17:07.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e2ffd640 1 -- 172.21.15.26:0/1555124375 learned_addr learned my addr 172.21.15.26:0/1555124375 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:07.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e3fff640 1 --2- 172.21.15.26:0/1555124375 >> v2:172.21.15.160:3300/0 conn(0x7ff6e40ce740 0x7ff6e40cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e27fc640 1 -- 172.21.15.26:0/1555124375 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402105264 0 0) 0x7ff6e406c970 con 0x7ff6e415f370 2023-12-19T18:17:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e27fc640 1 -- 172.21.15.26:0/1555124375 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6cc003530 con 0x7ff6e415f370 2023-12-19T18:17:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e37fe640 1 -- 172.21.15.26:0/1555124375 >> v2:172.21.15.160:3300/0 conn(0x7ff6e40ce740 msgr2=0x7ff6e40cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e37fe640 1 --2- 172.21.15.26:0/1555124375 >> v2:172.21.15.160:3300/0 conn(0x7ff6e40ce740 0x7ff6e40cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e37fe640 1 -- 172.21.15.26:0/1555124375 >> v1:172.21.15.160:6789/0 conn(0x7ff6e415f370 legacy=0x7ff6e406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.497+0000 7ff6e37fe640 1 -- 172.21.15.26:0/1555124375 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e415b1f0 con 0x7ff6e4155030 2023-12-19T18:17:07.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.501+0000 7ff6e37fe640 1 --2- 172.21.15.26:0/1555124375 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6e4155030 0x7ff6e415bcd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40097c0 tx=0x7ff6d401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d48c46dd38d33f0a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:07.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.501+0000 7ff6e27fc640 1 -- 172.21.15.26:0/1555124375 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40ab650 con 0x7ff6e4155030 2023-12-19T18:17:07.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.501+0000 7ff6e27fc640 1 -- 172.21.15.26:0/1555124375 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d40ab7b0 con 0x7ff6e4155030 2023-12-19T18:17:07.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.501+0000 7ff6e27fc640 1 -- 172.21.15.26:0/1555124375 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40abb40 con 0x7ff6e4155030 2023-12-19T18:17:07.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.501+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1555124375 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6e4155030 msgr2=0x7ff6e415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.501+0000 7ff6e9fc0640 1 --2- 172.21.15.26:0/1555124375 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6e4155030 0x7ff6e415bcd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40097c0 tx=0x7ff6d401c450 comp rx=0 tx=0).stop 2023-12-19T18:17:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.501+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1555124375 shutdown_connections 2023-12-19T18:17:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.501+0000 7ff6e9fc0640 1 --2- 172.21.15.26:0/1555124375 >> v2:172.21.15.160:3300/0 conn(0x7ff6e40ce740 0x7ff6e40cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.501+0000 7ff6e9fc0640 1 --2- 172.21.15.26:0/1555124375 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6e4155030 0x7ff6e415bcd0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.501+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1555124375 >> 172.21.15.26:0/1555124375 conn(0x7ff6e40658e0 msgr2=0x7ff6e4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.501+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1555124375 shutdown_connections 2023-12-19T18:17:07.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.501+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1555124375 wait complete. 2023-12-19T18:17:07.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e9fc0640 1 Processor -- start 2023-12-19T18:17:07.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e9fc0640 1 -- start start 2023-12-19T18:17:07.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e9fc0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e40ce740 0x7ff6e414c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e9fc0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6e4155030 0x7ff6e40b06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e9fc0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6e415f370 0x7ff6e40b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e9fc0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4070f60 con 0x7ff6e40ce740 2023-12-19T18:17:07.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e9fc0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40afc30 con 0x7ff6e415f370 2023-12-19T18:17:07.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e9fc0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4070c80 con 0x7ff6e4155030 2023-12-19T18:17:07.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e40ce740 0x7ff6e414c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e40ce740 0x7ff6e414c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48922/0 (socket says 172.21.15.26:48922) 2023-12-19T18:17:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e37fe640 1 -- 172.21.15.26:0/1818382134 learned_addr learned my addr 172.21.15.26:0/1818382134 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e2ffd640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6e4155030 0x7ff6e40b06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e37fe640 1 -- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6e4155030 msgr2=0x7ff6e40b06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e3fff640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6e415f370 0x7ff6e40b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e37fe640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6e4155030 0x7ff6e40b06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e37fe640 1 -- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6e415f370 msgr2=0x7ff6e40b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e37fe640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6e415f370 0x7ff6e40b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e37fe640 1 -- 172.21.15.26:0/1818382134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4009420 con 0x7ff6e40ce740 2023-12-19T18:17:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e2ffd640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6e4155030 0x7ff6e40b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e3fff640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6e415f370 0x7ff6e40b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:07.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e37fe640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e40ce740 0x7ff6e414c610 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7ff6d401a080 tx=0x7ff6d401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6cbfff640 1 -- 172.21.15.26:0/1818382134 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40bf610 con 0x7ff6e40ce740 2023-12-19T18:17:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6cbfff640 1 -- 172.21.15.26:0/1818382134 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d40bf770 con 0x7ff6e40ce740 2023-12-19T18:17:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6cbfff640 1 -- 172.21.15.26:0/1818382134 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40bfb20 con 0x7ff6e40ce740 2023-12-19T18:17:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1818382134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e40afed0 con 0x7ff6e40ce740 2023-12-19T18:17:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.505+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1818382134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e414cb50 con 0x7ff6e40ce740 2023-12-19T18:17:07.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.509+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1818382134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e40afa90 con 0x7ff6e40ce740 2023-12-19T18:17:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.509+0000 7ff6cbfff640 1 -- 172.21.15.26:0/1818382134 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 127) v1 ==== 85376+0+0 (secure 0 0 0) 0x7ff6d40bfc80 con 0x7ff6e40ce740 2023-12-19T18:17:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.509+0000 7ff6cbfff640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff6d0065c10 0x7ff6d00680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.509+0000 7ff6cbfff640 1 -- 172.21.15.26:0/1818382134 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 8064+0+0 (secure 0 0 0) 0x7ff6d411e830 con 0x7ff6e40ce740 2023-12-19T18:17:07.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.509+0000 7ff6e2ffd640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff6d0065c10 0x7ff6d00680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.509+0000 7ff6e2ffd640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff6d0065c10 0x7ff6d00680d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40b3920 tx=0x7ff6d8009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:07.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.513+0000 7ff6cbfff640 1 -- 172.21.15.26:0/1818382134 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff6e40afa90 con 0x7ff6e40ce740 2023-12-19T18:17:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.705+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1818382134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6e40afa90 con 0x7ff6e40ce740 2023-12-19T18:17:07.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.705+0000 7ff6cbfff640 1 -- 172.21.15.26:0/1818382134 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5917 (secure 0 0 0) 0x7ff6e40afa90 con 0x7ff6e40ce740 2023-12-19T18:17:07.709 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:07.709 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3349262044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3349262044},{"type":"v1","addr":"172.21.15.160:6836","nonce":3349262044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3865658274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3865658274},{"type":"v1","addr":"172.21.15.26:6833","nonce":3865658274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:27.798614+0000","modified":"2023-12-19T18:16:31.076294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3981286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3981286992},{"type":"v1","addr":"172.21.15.26:6835","nonce":3981286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:40.537606+0000","modified":"2023-12-19T18:16:43.550176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10605":{"gid":10605,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.160:6837/1294752100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1294752100},{"type":"v1","addr":"172.21.15.160:6837","nonce":1294752100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-19T18:17:07.710 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 102 2023-12-19T18:17:07.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.705+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff6d0065c10 msgr2=0x7ff6d00680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.705+0000 7ff6e9fc0640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff6d0065c10 0x7ff6d00680d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40b3920 tx=0x7ff6d8009040 comp rx=0 tx=0).stop 2023-12-19T18:17:07.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.705+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1818382134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e40ce740 msgr2=0x7ff6e414c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.705+0000 7ff6e9fc0640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e40ce740 0x7ff6e414c610 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7ff6d401a080 tx=0x7ff6d401ac90 comp rx=0 tx=0).stop 2023-12-19T18:17:07.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.709+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1818382134 shutdown_connections 2023-12-19T18:17:07.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.709+0000 7ff6e9fc0640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff6d0065c10 0x7ff6d00680d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.709+0000 7ff6e9fc0640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6e415f370 0x7ff6e40b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.709+0000 7ff6e9fc0640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6e4155030 0x7ff6e40b06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.709+0000 7ff6e9fc0640 1 --2- 172.21.15.26:0/1818382134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e40ce740 0x7ff6e414c610 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.709+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1818382134 >> 172.21.15.26:0/1818382134 conn(0x7ff6e40658e0 msgr2=0x7ff6e40b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:07.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.709+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1818382134 shutdown_connections 2023-12-19T18:17:07.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.709+0000 7ff6e9fc0640 1 -- 172.21.15.26:0/1818382134 wait complete. 2023-12-19T18:17:07.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:07.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41f9466640 1 Processor -- start 2023-12-19T18:17:07.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41f9466640 1 -- start start 2023-12-19T18:17:07.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41f9466640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f41f415cd40 0x7f41f40ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41f9466640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41f4160900 0x7f41f4160d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41f9466640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f40d29d0 con 0x7f41f40d2bf0 2023-12-19T18:17:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41f9466640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f4161280 con 0x7f41f415cd40 2023-12-19T18:17:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41f9466640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f415d680 con 0x7f41f4160900 2023-12-19T18:17:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41f37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f41f40d2bf0 0x7f41f40d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58182/0 (socket says 172.21.15.26:58182) 2023-12-19T18:17:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41f37fe640 1 -- 172.21.15.26:0/4172770078 learned_addr learned my addr 172.21.15.26:0/4172770078 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41ebfff640 1 --2- 172.21.15.26:0/4172770078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41f4160900 0x7f41f4160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41f2ffd640 1 --2- 172.21.15.26:0/4172770078 >> v2:172.21.15.160:3300/0 conn(0x7f41f415cd40 0x7f41f40ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41f27fc640 1 -- 172.21.15.26:0/4172770078 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603774595 0 0) 0x7f41ec0014a0 con 0x7f41f40d2bf0 2023-12-19T18:17:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41f27fc640 1 -- 172.21.15.26:0/4172770078 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41e0003530 con 0x7f41f40d2bf0 2023-12-19T18:17:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41ebfff640 1 -- 172.21.15.26:0/4172770078 >> v2:172.21.15.160:3300/0 conn(0x7f41f415cd40 msgr2=0x7f41f40ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41ebfff640 1 --2- 172.21.15.26:0/4172770078 >> v2:172.21.15.160:3300/0 conn(0x7f41f415cd40 0x7f41f40ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41ebfff640 1 -- 172.21.15.26:0/4172770078 >> v1:172.21.15.26:6789/0 conn(0x7f41f40d2bf0 legacy=0x7f41f40d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.873+0000 7f41ebfff640 1 -- 172.21.15.26:0/4172770078 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f4161750 con 0x7f41f4160900 2023-12-19T18:17:07.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.877+0000 7f41ebfff640 1 --2- 172.21.15.26:0/4172770078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41f4160900 0x7f41f4160d40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097c0 tx=0x7f41dc01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec83383583b564b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.877+0000 7f41f27fc640 1 -- 172.21.15.26:0/4172770078 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0ab600 con 0x7f41f4160900 2023-12-19T18:17:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.877+0000 7f41f27fc640 1 -- 172.21.15.26:0/4172770078 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc0a7050 con 0x7f41f4160900 2023-12-19T18:17:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.877+0000 7f41f27fc640 1 -- 172.21.15.26:0/4172770078 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0ab960 con 0x7f41f4160900 2023-12-19T18:17:07.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.877+0000 7f41f9466640 1 -- 172.21.15.26:0/4172770078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41f4160900 msgr2=0x7f41f4160d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.877+0000 7f41f9466640 1 --2- 172.21.15.26:0/4172770078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41f4160900 0x7f41f4160d40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097c0 tx=0x7f41dc01c480 comp rx=0 tx=0).stop 2023-12-19T18:17:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.877+0000 7f41f9466640 1 -- 172.21.15.26:0/4172770078 shutdown_connections 2023-12-19T18:17:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.877+0000 7f41f9466640 1 --2- 172.21.15.26:0/4172770078 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41f4160900 0x7f41f4160d40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.877+0000 7f41f9466640 1 --2- 172.21.15.26:0/4172770078 >> v2:172.21.15.160:3300/0 conn(0x7f41f415cd40 0x7f41f40ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.877+0000 7f41f9466640 1 -- 172.21.15.26:0/4172770078 >> 172.21.15.26:0/4172770078 conn(0x7f41f406a120 msgr2=0x7f41f406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.877+0000 7f41f9466640 1 -- 172.21.15.26:0/4172770078 shutdown_connections 2023-12-19T18:17:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.877+0000 7f41f9466640 1 -- 172.21.15.26:0/4172770078 wait complete. 2023-12-19T18:17:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f9466640 1 Processor -- start 2023-12-19T18:17:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f9466640 1 -- start start 2023-12-19T18:17:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f9466640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41f40d2bf0 0x7f41f41f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f9466640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41f415cd40 0x7f41f4140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f9466640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f4160900 0x7f41f40cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f9466640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f4152a10 con 0x7f41f4160900 2023-12-19T18:17:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f9466640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f41411e0 con 0x7f41f40d2bf0 2023-12-19T18:17:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f9466640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f41414c0 con 0x7f41f415cd40 2023-12-19T18:17:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f4160900 0x7f41f40cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f37fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f4160900 0x7f41f40cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48924/0 (socket says 172.21.15.26:48924) 2023-12-19T18:17:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f37fe640 1 -- 172.21.15.26:0/4174164203 learned_addr learned my addr 172.21.15.26:0/4174164203 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f2ffd640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41f40d2bf0 0x7f41f41f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41ebfff640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41f415cd40 0x7f41f4140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f37fe640 1 -- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41f415cd40 msgr2=0x7f41f4140990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f37fe640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41f415cd40 0x7f41f4140990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f37fe640 1 -- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41f40d2bf0 msgr2=0x7f41f41f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f37fe640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41f40d2bf0 0x7f41f41f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f37fe640 1 -- 172.21.15.26:0/4174164203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc009420 con 0x7f41f4160900 2023-12-19T18:17:07.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41f37fe640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f4160900 0x7f41f40cc310 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00cb40 tx=0x7f41ec00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:07.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41eb7fe640 1 -- 172.21.15.26:0/4174164203 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec00aca0 con 0x7f41f4160900 2023-12-19T18:17:07.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.881+0000 7f41eb7fe640 1 -- 172.21.15.26:0/4174164203 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec00ae00 con 0x7f41f4160900 2023-12-19T18:17:07.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.885+0000 7f41eb7fe640 1 -- 172.21.15.26:0/4174164203 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec012620 con 0x7f41f4160900 2023-12-19T18:17:07.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.885+0000 7f41f9466640 1 -- 172.21.15.26:0/4174164203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f41417a0 con 0x7f41f4160900 2023-12-19T18:17:07.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.885+0000 7f41f9466640 1 -- 172.21.15.26:0/4174164203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f41f7920 con 0x7f41f4160900 2023-12-19T18:17:07.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.885+0000 7f41f9466640 1 -- 172.21.15.26:0/4174164203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f4152ef0 con 0x7f41f4160900 2023-12-19T18:17:07.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.885+0000 7f41eb7fe640 1 -- 172.21.15.26:0/4174164203 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 127) v1 ==== 85376+0+0 (secure 0 0 0) 0x7f41ec007070 con 0x7f41f4160900 2023-12-19T18:17:07.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.885+0000 7f41eb7fe640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f41d4065c10 0x7f41d40680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.885+0000 7f41eb7fe640 1 -- 172.21.15.26:0/4174164203 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 8064+0+0 (secure 0 0 0) 0x7f41ec083af0 con 0x7f41f4160900 2023-12-19T18:17:07.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.885+0000 7f41f2ffd640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f41d4065c10 0x7f41d40680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.889+0000 7f41f2ffd640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f41d4065c10 0x7f41d40680d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009b40 tx=0x7f41e4006f90 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:07.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:07.889+0000 7f41eb7fe640 1 -- 172.21.15.26:0/4174164203 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f41f4152ef0 con 0x7f41f4160900 2023-12-19T18:17:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.081+0000 7f41f9466640 1 -- 172.21.15.26:0/4174164203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41f4152ef0 con 0x7f41f4160900 2023-12-19T18:17:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.081+0000 7f41eb7fe640 1 -- 172.21.15.26:0/4174164203 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5917 (secure 0 0 0) 0x7f41f4152ef0 con 0x7f41f4160900 2023-12-19T18:17:08.087 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:08.087 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3349262044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3349262044},{"type":"v1","addr":"172.21.15.160:6836","nonce":3349262044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3865658274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3865658274},{"type":"v1","addr":"172.21.15.26:6833","nonce":3865658274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:27.798614+0000","modified":"2023-12-19T18:16:31.076294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10611},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10611":{"gid":10611,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/3981286992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3981286992},{"type":"v1","addr":"172.21.15.26:6835","nonce":3981286992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:40.537606+0000","modified":"2023-12-19T18:16:43.550176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10605":{"gid":10605,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.160:6837/1294752100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1294752100},{"type":"v1","addr":"172.21.15.160:6837","nonce":1294752100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-19T18:17:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 102 2023-12-19T18:17:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.085+0000 7f41f9466640 1 -- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f41d4065c10 msgr2=0x7f41d40680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.085+0000 7f41f9466640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f41d4065c10 0x7f41d40680d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f41e4009b40 tx=0x7f41e4006f90 comp rx=0 tx=0).stop 2023-12-19T18:17:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.085+0000 7f41f9466640 1 -- 172.21.15.26:0/4174164203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f4160900 msgr2=0x7f41f40cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.085+0000 7f41f9466640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f4160900 0x7f41f40cc310 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00cb40 tx=0x7f41ec00a060 comp rx=0 tx=0).stop 2023-12-19T18:17:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.085+0000 7f41f9466640 1 -- 172.21.15.26:0/4174164203 shutdown_connections 2023-12-19T18:17:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.085+0000 7f41f9466640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f41d4065c10 0x7f41d40680d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.085+0000 7f41f9466640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f4160900 0x7f41f40cc310 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.085+0000 7f41f9466640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f41f415cd40 0x7f41f4140990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.085+0000 7f41f9466640 1 --2- 172.21.15.26:0/4174164203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41f40d2bf0 0x7f41f41f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.085+0000 7f41f9466640 1 -- 172.21.15.26:0/4174164203 >> 172.21.15.26:0/4174164203 conn(0x7f41f406a120 msgr2=0x7f41f406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.085+0000 7f41f9466640 1 -- 172.21.15.26:0/4174164203 shutdown_connections 2023-12-19T18:17:08.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.085+0000 7f41f9466640 1 -- 172.21.15.26:0/4174164203 wait complete. 2023-12-19T18:17:08.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd87536640 1 Processor -- start 2023-12-19T18:17:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd87536640 1 -- start start 2023-12-19T18:17:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd87536640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd8015cca0 0x7efd800b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd87536640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efd800b8760 0x7efd800cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd87536640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd800b6fb0 con 0x7efd800b6b20 2023-12-19T18:17:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd87536640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd800b7c30 con 0x7efd8015cca0 2023-12-19T18:17:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd87536640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd800b7290 con 0x7efd800b8760 2023-12-19T18:17:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd86534640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd8015cca0 0x7efd800b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd86d35640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efd800b6b20 0x7efd800b4420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58184/0 (socket says 172.21.15.26:58184) 2023-12-19T18:17:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd86d35640 1 -- 172.21.15.26:0/1994430260 learned_addr learned my addr 172.21.15.26:0/1994430260 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd85d33640 1 --2- 172.21.15.26:0/1994430260 >> v2:172.21.15.160:3300/0 conn(0x7efd800b8760 0x7efd800cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd86534640 1 -- 172.21.15.26:0/1994430260 >> v2:172.21.15.160:3300/0 conn(0x7efd800b8760 msgr2=0x7efd800cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd86534640 1 --2- 172.21.15.26:0/1994430260 >> v2:172.21.15.160:3300/0 conn(0x7efd800b8760 0x7efd800cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd86534640 1 -- 172.21.15.26:0/1994430260 >> v1:172.21.15.26:6789/0 conn(0x7efd800b6b20 legacy=0x7efd800b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd85532640 1 -- 172.21.15.26:0/1994430260 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453443747 0 0) 0x7efd800b6fb0 con 0x7efd800b6b20 2023-12-19T18:17:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.253+0000 7efd86534640 1 -- 172.21.15.26:0/1994430260 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd800b7570 con 0x7efd8015cca0 2023-12-19T18:17:08.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.257+0000 7efd86534640 1 --2- 172.21.15.26:0/1994430260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8015cca0 0x7efd800b65e0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7efd800b8050 tx=0x7efd7c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3eaa53b54649f90 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:08.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.257+0000 7efd85532640 1 -- 172.21.15.26:0/1994430260 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0ab650 con 0x7efd8015cca0 2023-12-19T18:17:08.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.257+0000 7efd85532640 1 -- 172.21.15.26:0/1994430260 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7c0ab7b0 con 0x7efd8015cca0 2023-12-19T18:17:08.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.257+0000 7efd85532640 1 -- 172.21.15.26:0/1994430260 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0abb40 con 0x7efd8015cca0 2023-12-19T18:17:08.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.257+0000 7efd87536640 1 -- 172.21.15.26:0/1994430260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8015cca0 msgr2=0x7efd800b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.257+0000 7efd87536640 1 --2- 172.21.15.26:0/1994430260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8015cca0 0x7efd800b65e0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7efd800b8050 tx=0x7efd7c01c430 comp rx=0 tx=0).stop 2023-12-19T18:17:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.257+0000 7efd87536640 1 -- 172.21.15.26:0/1994430260 shutdown_connections 2023-12-19T18:17:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.257+0000 7efd87536640 1 --2- 172.21.15.26:0/1994430260 >> v2:172.21.15.160:3300/0 conn(0x7efd800b8760 0x7efd800cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.257+0000 7efd87536640 1 --2- 172.21.15.26:0/1994430260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8015cca0 0x7efd800b65e0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.257+0000 7efd87536640 1 -- 172.21.15.26:0/1994430260 >> 172.21.15.26:0/1994430260 conn(0x7efd8006a120 msgr2=0x7efd8006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.257+0000 7efd87536640 1 -- 172.21.15.26:0/1994430260 shutdown_connections 2023-12-19T18:17:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.257+0000 7efd87536640 1 -- 172.21.15.26:0/1994430260 wait complete. 2023-12-19T18:17:08.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.257+0000 7efd87536640 1 Processor -- start 2023-12-19T18:17:08.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd87536640 1 -- start start 2023-12-19T18:17:08.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd87536640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd800b8760 0x7efd80166fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd87536640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8015cca0 0x7efd8015e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd87536640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd80154f10 0x7efd801553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd87536640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd801603f0 con 0x7efd8015cca0 2023-12-19T18:17:08.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd87536640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd80155900 con 0x7efd800b8760 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd87536640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8015b510 con 0x7efd80154f10 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd85d33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8015cca0 0x7efd8015e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd85d33640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8015cca0 0x7efd8015e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48938/0 (socket says 172.21.15.26:48938) 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd85d33640 1 -- 172.21.15.26:0/3536413092 learned_addr learned my addr 172.21.15.26:0/3536413092 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd86534640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd800b8760 0x7efd80166fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd86d35640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd80154f10 0x7efd801553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd85d33640 1 -- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd80154f10 msgr2=0x7efd801553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd85d33640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd80154f10 0x7efd801553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd85d33640 1 -- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd800b8760 msgr2=0x7efd80166fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd85d33640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd800b8760 0x7efd80166fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd85d33640 1 -- 172.21.15.26:0/3536413092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c009420 con 0x7efd8015cca0 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd86d35640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd80154f10 0x7efd801553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd85d33640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8015cca0 0x7efd8015e330 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7efd7000e930 tx=0x7efd7000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd76ffd640 1 -- 172.21.15.26:0/3536413092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70009690 con 0x7efd8015cca0 2023-12-19T18:17:08.266 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd76ffd640 1 -- 172.21.15.26:0/3536413092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7000ca00 con 0x7efd8015cca0 2023-12-19T18:17:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd76ffd640 1 -- 172.21.15.26:0/3536413092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7000cd70 con 0x7efd8015cca0 2023-12-19T18:17:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd87536640 1 -- 172.21.15.26:0/3536413092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd801674f0 con 0x7efd8015cca0 2023-12-19T18:17:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd87536640 1 -- 172.21.15.26:0/3536413092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8015abe0 con 0x7efd8015cca0 2023-12-19T18:17:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.261+0000 7efd87536640 1 -- 172.21.15.26:0/3536413092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd80067400 con 0x7efd8015cca0 2023-12-19T18:17:08.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.265+0000 7efd76ffd640 1 -- 172.21.15.26:0/3536413092 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 127) v1 ==== 85376+0+0 (secure 0 0 0) 0x7efd70013470 con 0x7efd8015cca0 2023-12-19T18:17:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.265+0000 7efd76ffd640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7efd64065c10 0x7efd640680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.265+0000 7efd76ffd640 1 -- 172.21.15.26:0/3536413092 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 8064+0+0 (secure 0 0 0) 0x7efd70084430 con 0x7efd8015cca0 2023-12-19T18:17:08.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.265+0000 7efd86534640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7efd64065c10 0x7efd640680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.265+0000 7efd86534640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7efd64065c10 0x7efd640680d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0af000 tx=0x7efd7c0a7000 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:08.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.269+0000 7efd76ffd640 1 -- 172.21.15.26:0/3536413092 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efd700512a0 con 0x7efd8015cca0 2023-12-19T18:17:08.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.429+0000 7efd87536640 1 -- 172.21.15.26:0/3536413092 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd8015b650 con 0x7efd8015cca0 2023-12-19T18:17:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.429+0000 7efd76ffd640 1 -- 172.21.15.26:0/3536413092 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v218) v1 ==== 74+0+22054 (secure 0 0 0) 0x7efd70021090 con 0x7efd8015cca0 2023-12-19T18:17:08.434 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:08.434 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":218,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:04.905457+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:25.863942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:26.798049+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:16:38.226152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:16:39.540217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1665633474":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3561084215":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1351823889":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3136172808":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3161716183":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/2592629251":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1089029505":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1544351773":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1134319862":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/4291270549":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3401801452":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/765362777":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/1696607504":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6833/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6832/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/2355423136":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/3498637577":"2023-12-20T18:16:55.864876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.433+0000 7efd87536640 1 -- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7efd64065c10 msgr2=0x7efd640680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.433+0000 7efd87536640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7efd64065c10 0x7efd640680d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0af000 tx=0x7efd7c0a7000 comp rx=0 tx=0).stop 2023-12-19T18:17:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.433+0000 7efd87536640 1 -- 172.21.15.26:0/3536413092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8015cca0 msgr2=0x7efd8015e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.433+0000 7efd87536640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8015cca0 0x7efd8015e330 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7efd7000e930 tx=0x7efd7000edf0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.433+0000 7efd87536640 1 -- 172.21.15.26:0/3536413092 shutdown_connections 2023-12-19T18:17:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.433+0000 7efd87536640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7efd64065c10 0x7efd640680d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.433+0000 7efd87536640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efd80154f10 0x7efd801553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.433+0000 7efd87536640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8015cca0 0x7efd8015e330 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.433+0000 7efd87536640 1 --2- 172.21.15.26:0/3536413092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd800b8760 0x7efd80166fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.433+0000 7efd87536640 1 -- 172.21.15.26:0/3536413092 >> 172.21.15.26:0/3536413092 conn(0x7efd8006a120 msgr2=0x7efd800d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.433+0000 7efd87536640 1 -- 172.21.15.26:0/3536413092 shutdown_connections 2023-12-19T18:17:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.433+0000 7efd87536640 1 -- 172.21.15.26:0/3536413092 wait complete. 2023-12-19T18:17:08.456 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:17:08.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66ff98640 1 Processor -- start 2023-12-19T18:17:08.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66ff98640 1 -- start start 2023-12-19T18:17:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66ff98640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6680b2d50 0x7fc66816c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66ff98640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66806d4b0 con 0x7fc6680cb1d0 2023-12-19T18:17:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66ff98640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66806cc10 con 0x7fc66806bee0 2023-12-19T18:17:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66ff98640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680cc030 con 0x7fc6680b2d50 2023-12-19T18:17:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66dd34640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc6680cb1d0 0x7fc6680cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58190/0 (socket says 172.21.15.26:58190) 2023-12-19T18:17:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66dd34640 1 -- 172.21.15.26:0/3460144308 learned_addr learned my addr 172.21.15.26:0/3460144308 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:08.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66e535640 1 --2- 172.21.15.26:0/3460144308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6680b2d50 0x7fc66816c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66cd32640 1 -- 172.21.15.26:0/3460144308 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099614019 0 0) 0x7fc66806d4b0 con 0x7fc6680cb1d0 2023-12-19T18:17:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66cd32640 1 -- 172.21.15.26:0/3460144308 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc654003530 con 0x7fc6680cb1d0 2023-12-19T18:17:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66cd32640 1 -- 172.21.15.26:0/3460144308 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352224373 0 0) 0x7fc66806cc10 con 0x7fc66806bee0 2023-12-19T18:17:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66cd32640 1 -- 172.21.15.26:0/3460144308 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc66806d4b0 con 0x7fc66806bee0 2023-12-19T18:17:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66e535640 1 -- 172.21.15.26:0/3460144308 >> v1:172.21.15.160:6789/0 conn(0x7fc66806bee0 legacy=0x7fc66806c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66e535640 1 -- 172.21.15.26:0/3460144308 >> v1:172.21.15.26:6789/0 conn(0x7fc6680cb1d0 legacy=0x7fc6680cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.597+0000 7fc66e535640 1 -- 172.21.15.26:0/3460144308 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66806cef0 con 0x7fc6680b2d50 2023-12-19T18:17:08.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.601+0000 7fc66e535640 1 --2- 172.21.15.26:0/3460144308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6680b2d50 0x7fc66816c020 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc660009dc0 tx=0x7fc66001c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8495f4320f507fff server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.601+0000 7fc66cd32640 1 -- 172.21.15.26:0/3460144308 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6600a59a0 con 0x7fc6680b2d50 2023-12-19T18:17:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.601+0000 7fc66cd32640 1 -- 172.21.15.26:0/3460144308 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6600a5b00 con 0x7fc6680b2d50 2023-12-19T18:17:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.601+0000 7fc66cd32640 1 -- 172.21.15.26:0/3460144308 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6600a5e90 con 0x7fc6680b2d50 2023-12-19T18:17:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.601+0000 7fc66ff98640 1 -- 172.21.15.26:0/3460144308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6680b2d50 msgr2=0x7fc66816c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.601+0000 7fc66ff98640 1 --2- 172.21.15.26:0/3460144308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6680b2d50 0x7fc66816c020 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc660009dc0 tx=0x7fc66001c8e0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.601+0000 7fc66ff98640 1 -- 172.21.15.26:0/3460144308 shutdown_connections 2023-12-19T18:17:08.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.601+0000 7fc66ff98640 1 --2- 172.21.15.26:0/3460144308 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6680b2d50 0x7fc66816c020 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.601+0000 7fc66ff98640 1 -- 172.21.15.26:0/3460144308 >> 172.21.15.26:0/3460144308 conn(0x7fc6680658e0 msgr2=0x7fc668065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:08.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.601+0000 7fc66ff98640 1 -- 172.21.15.26:0/3460144308 shutdown_connections 2023-12-19T18:17:08.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.601+0000 7fc66ff98640 1 -- 172.21.15.26:0/3460144308 wait complete. 2023-12-19T18:17:08.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66ff98640 1 Processor -- start 2023-12-19T18:17:08.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66ff98640 1 -- start start 2023-12-19T18:17:08.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66ff98640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc66806bee0 0x7fc6680ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66ff98640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6680b2d50 0x7fc66815ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66ff98640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6680cb1d0 0x7fc668163e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66ff98640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66814f340 con 0x7fc6680cb1d0 2023-12-19T18:17:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66ff98640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6681543a0 con 0x7fc6680b2d50 2023-12-19T18:17:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66ff98640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6680cf1f0 con 0x7fc66806bee0 2023-12-19T18:17:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66e535640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6680cb1d0 0x7fc668163e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66e535640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6680cb1d0 0x7fc668163e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48944/0 (socket says 172.21.15.26:48944) 2023-12-19T18:17:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66e535640 1 -- 172.21.15.26:0/1332269703 learned_addr learned my addr 172.21.15.26:0/1332269703 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66d533640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6680b2d50 0x7fc66815ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66dd34640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc66806bee0 0x7fc6680ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66e535640 1 -- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc66806bee0 msgr2=0x7fc6680ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66e535640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc66806bee0 0x7fc6680ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66e535640 1 -- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6680b2d50 msgr2=0x7fc66815ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66e535640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6680b2d50 0x7fc66815ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66e535640 1 -- 172.21.15.26:0/1332269703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc660009a70 con 0x7fc6680cb1d0 2023-12-19T18:17:08.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66d533640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6680b2d50 0x7fc66815ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:08.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66e535640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6680cb1d0 0x7fc668163e30 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fc66001a510 tx=0x7fc660019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:08.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc65e7fc640 1 -- 172.21.15.26:0/1332269703 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6600adc80 con 0x7fc6680cb1d0 2023-12-19T18:17:08.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc65e7fc640 1 -- 172.21.15.26:0/1332269703 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6600adde0 con 0x7fc6680cb1d0 2023-12-19T18:17:08.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66ff98640 1 -- 172.21.15.26:0/1332269703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6681545d0 con 0x7fc6680cb1d0 2023-12-19T18:17:08.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc65e7fc640 1 -- 172.21.15.26:0/1332269703 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6600b9690 con 0x7fc6680cb1d0 2023-12-19T18:17:08.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66ff98640 1 -- 172.21.15.26:0/1332269703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc668154b90 con 0x7fc6680cb1d0 2023-12-19T18:17:08.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.605+0000 7fc66ff98640 1 -- 172.21.15.26:0/1332269703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc668074270 con 0x7fc6680cb1d0 2023-12-19T18:17:08.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.609+0000 7fc65e7fc640 1 -- 172.21.15.26:0/1332269703 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 127) v1 ==== 85376+0+0 (secure 0 0 0) 0x7fc6600b97f0 con 0x7fc6680cb1d0 2023-12-19T18:17:08.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.609+0000 7fc65e7fc640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc64c065c10 0x7fc64c0680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.609+0000 7fc65e7fc640 1 -- 172.21.15.26:0/1332269703 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 8064+0+0 (secure 0 0 0) 0x7fc66011f310 con 0x7fc6680cb1d0 2023-12-19T18:17:08.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.609+0000 7fc66dd34640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc64c065c10 0x7fc64c0680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.609+0000 7fc66dd34640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc64c065c10 0x7fc64c0680d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc664000f70 tx=0x7fc664009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:08.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.613+0000 7fc65e7fc640 1 -- 172.21.15.26:0/1332269703 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc6600ec1b0 con 0x7fc6680cb1d0 2023-12-19T18:17:08.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.801+0000 7fc66ff98640 1 -- 172.21.15.26:0/1332269703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc6680cc6e0 con 0x7fc6680cb1d0 2023-12-19T18:17:08.807 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T18:17:08.805+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:17:08.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.817+0000 7fc65e7fc640 1 -- 172.21.15.26:0/1332269703 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v103) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc660009450 con 0x7fc6680cb1d0 2023-12-19T18:17:08.822 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:17:08.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.821+0000 7fc66ff98640 1 -- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc64c065c10 msgr2=0x7fc64c0680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.821+0000 7fc66ff98640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc64c065c10 0x7fc64c0680d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc664000f70 tx=0x7fc664009040 comp rx=0 tx=0).stop 2023-12-19T18:17:08.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.821+0000 7fc66ff98640 1 -- 172.21.15.26:0/1332269703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6680cb1d0 msgr2=0x7fc668163e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.821+0000 7fc66ff98640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6680cb1d0 0x7fc668163e30 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fc66001a510 tx=0x7fc660019680 comp rx=0 tx=0).stop 2023-12-19T18:17:08.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.821+0000 7fc66ff98640 1 -- 172.21.15.26:0/1332269703 shutdown_connections 2023-12-19T18:17:08.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.821+0000 7fc66ff98640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc64c065c10 0x7fc64c0680d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.821+0000 7fc66ff98640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6680cb1d0 0x7fc668163e30 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.821+0000 7fc66ff98640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6680b2d50 0x7fc66815ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.821+0000 7fc66ff98640 1 --2- 172.21.15.26:0/1332269703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc66806bee0 0x7fc6680ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.821+0000 7fc66ff98640 1 -- 172.21.15.26:0/1332269703 >> 172.21.15.26:0/1332269703 conn(0x7fc6680658e0 msgr2=0x7fc668164c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:08.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.821+0000 7fc66ff98640 1 -- 172.21.15.26:0/1332269703 shutdown_connections 2023-12-19T18:17:08.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.821+0000 7fc66ff98640 1 -- 172.21.15.26:0/1332269703 wait complete. 2023-12-19T18:17:08.830 INFO:tasks.ceph.mds.a.smithi026.stderr: -4462> 2023-12-19T18:16:33.370+0000 7f7dc369e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:17:08.845 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:17:08.881 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T18:17:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.985+0000 7ff9c90fc640 1 Processor -- start 2023-12-19T18:17:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9c90fc640 1 -- start start 2023-12-19T18:17:08.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9c90fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9c41585f0 0x7ff9c40cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9c90fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9c40cb150 0x7ff9c40cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9c90fc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c415b1b0 con 0x7ff9c40cc310 2023-12-19T18:17:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9c90fc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9c40cbb10 con 0x7ff9c41585f0 2023-12-19T18:17:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9c90fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40b47a0 con 0x7ff9c40cb150 2023-12-19T18:17:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9c2d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9c41585f0 0x7ff9c40cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9c2d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9c41585f0 0x7ff9c40cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48968/0 (socket says 172.21.15.26:48968) 2023-12-19T18:17:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9c2d9d640 1 -- 172.21.15.26:0/404599377 learned_addr learned my addr 172.21.15.26:0/404599377 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9bbfff640 1 --2- 172.21.15.26:0/404599377 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9c40cb150 0x7ff9c40cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9c2d9d640 1 -- 172.21.15.26:0/404599377 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9c40cb150 msgr2=0x7ff9c40cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9c2d9d640 1 --2- 172.21.15.26:0/404599377 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9c40cb150 0x7ff9c40cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9c2d9d640 1 -- 172.21.15.26:0/404599377 >> v1:172.21.15.160:6789/0 conn(0x7ff9c40cc310 legacy=0x7ff9c40cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9c2d9d640 1 -- 172.21.15.26:0/404599377 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c415b490 con 0x7ff9c41585f0 2023-12-19T18:17:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.989+0000 7ff9bbfff640 1 --2- 172.21.15.26:0/404599377 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9c40cb150 0x7ff9c40cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:08.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c2d9d640 1 --2- 172.21.15.26:0/404599377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9c41585f0 0x7ff9c40cdce0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009870 tx=0x7ff9b401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66d51b5ec6d2d0c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c259c640 1 -- 172.21.15.26:0/404599377 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40ab680 con 0x7ff9c41585f0 2023-12-19T18:17:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c259c640 1 -- 172.21.15.26:0/404599377 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b40ab7e0 con 0x7ff9c41585f0 2023-12-19T18:17:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c259c640 1 -- 172.21.15.26:0/404599377 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40abb70 con 0x7ff9c41585f0 2023-12-19T18:17:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c90fc640 1 -- 172.21.15.26:0/404599377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9c41585f0 msgr2=0x7ff9c40cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c90fc640 1 --2- 172.21.15.26:0/404599377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9c41585f0 0x7ff9c40cdce0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009870 tx=0x7ff9b401c480 comp rx=0 tx=0).stop 2023-12-19T18:17:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c90fc640 1 -- 172.21.15.26:0/404599377 shutdown_connections 2023-12-19T18:17:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c90fc640 1 --2- 172.21.15.26:0/404599377 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9c40cb150 0x7ff9c40cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c90fc640 1 --2- 172.21.15.26:0/404599377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9c41585f0 0x7ff9c40cdce0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c90fc640 1 -- 172.21.15.26:0/404599377 >> 172.21.15.26:0/404599377 conn(0x7ff9c4066160 msgr2=0x7ff9c40b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c90fc640 1 -- 172.21.15.26:0/404599377 shutdown_connections 2023-12-19T18:17:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c90fc640 1 -- 172.21.15.26:0/404599377 wait complete. 2023-12-19T18:17:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c90fc640 1 Processor -- start 2023-12-19T18:17:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.993+0000 7ff9c90fc640 1 -- start start 2023-12-19T18:17:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9c90fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9c40cb150 0x7ff9c414b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9c90fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9c40cc310 0x7ff9c4139bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9c90fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9c41585f0 0x7ff9c41511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9c90fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c4154540 con 0x7ff9c40cc310 2023-12-19T18:17:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9c90fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c413a100 con 0x7ff9c41585f0 2023-12-19T18:17:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9c90fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c41534a0 con 0x7ff9c40cb150 2023-12-19T18:17:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9bbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9c40cc310 0x7ff9c4139bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9c2d9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9c40cb150 0x7ff9c414b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9bbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9c40cc310 0x7ff9c4139bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48974/0 (socket says 172.21.15.26:48974) 2023-12-19T18:17:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9bbfff640 1 -- 172.21.15.26:0/133018351 learned_addr learned my addr 172.21.15.26:0/133018351 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:09.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9c359e640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9c41585f0 0x7ff9c41511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9bbfff640 1 -- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9c40cb150 msgr2=0x7ff9c414b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9bbfff640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9c40cb150 0x7ff9c414b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9bbfff640 1 -- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9c41585f0 msgr2=0x7ff9c41511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9bbfff640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9c41585f0 0x7ff9c41511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:09.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9bbfff640 1 -- 172.21.15.26:0/133018351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b40094d0 con 0x7ff9c40cc310 2023-12-19T18:17:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9c2d9d640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9c40cb150 0x7ff9c414b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9bbfff640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9c40cc310 0x7ff9c4139bc0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00d800 tx=0x7ff9ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9bb7fe640 1 -- 172.21.15.26:0/133018351 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac0096c0 con 0x7ff9c40cc310 2023-12-19T18:17:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9bb7fe640 1 -- 172.21.15.26:0/133018351 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac009820 con 0x7ff9c40cc310 2023-12-19T18:17:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9bb7fe640 1 -- 172.21.15.26:0/133018351 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac009bd0 con 0x7ff9c40cc310 2023-12-19T18:17:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9c90fc640 1 -- 172.21.15.26:0/133018351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c415c3e0 con 0x7ff9c40cc310 2023-12-19T18:17:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9c90fc640 1 -- 172.21.15.26:0/133018351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c4153a60 con 0x7ff9c40cc310 2023-12-19T18:17:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9c90fc640 1 -- 172.21.15.26:0/133018351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c4074270 con 0x7ff9c40cc310 2023-12-19T18:17:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:08.997+0000 7ff9bb7fe640 1 -- 172.21.15.26:0/133018351 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 127) v1 ==== 85376+0+0 (secure 0 0 0) 0x7ff9ac009d30 con 0x7ff9c40cc310 2023-12-19T18:17:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.001+0000 7ff9bb7fe640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff9a4065c10 0x7ff9a40680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.001+0000 7ff9bb7fe640 1 -- 172.21.15.26:0/133018351 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 8150+0+0 (secure 0 0 0) 0x7ff9ac0843e0 con 0x7ff9c40cc310 2023-12-19T18:17:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.001+0000 7ff9c2d9d640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff9a4065c10 0x7ff9a40680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.001+0000 7ff9c2d9d640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff9a4065c10 0x7ff9a40680d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40af000 tx=0x7ff9b40b6040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:09.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.005+0000 7ff9bb7fe640 1 -- 172.21.15.26:0/133018351 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff9ac051240 con 0x7ff9c40cc310 2023-12-19T18:17:09.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.197+0000 7ff9c90fc640 1 -- 172.21.15.26:0/133018351 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff9c40c2050 con 0x7ff9c40cc310 2023-12-19T18:17:09.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.833+0000 7ff9bb7fe640 1 -- 172.21.15.26:0/133018351 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ff9ac051610 con 0x7ff9c40cc310 2023-12-19T18:17:09.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.833+0000 7ff9bb7fe640 1 -- 172.21.15.26:0/133018351 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v104) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff9ac051240 con 0x7ff9c40cc310 2023-12-19T18:17:09.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.837+0000 7ff9c90fc640 1 -- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff9a4065c10 msgr2=0x7ff9a40680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:09.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.837+0000 7ff9c90fc640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff9a4065c10 0x7ff9a40680d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40af000 tx=0x7ff9b40b6040 comp rx=0 tx=0).stop 2023-12-19T18:17:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.837+0000 7ff9c90fc640 1 -- 172.21.15.26:0/133018351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9c40cc310 msgr2=0x7ff9c4139bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.837+0000 7ff9c90fc640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9c40cc310 0x7ff9c4139bc0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00d800 tx=0x7ff9ac00dcc0 comp rx=0 tx=0).stop 2023-12-19T18:17:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.837+0000 7ff9c90fc640 1 -- 172.21.15.26:0/133018351 shutdown_connections 2023-12-19T18:17:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.837+0000 7ff9c90fc640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff9a4065c10 0x7ff9a40680d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.837+0000 7ff9c90fc640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff9c41585f0 0x7ff9c41511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.837+0000 7ff9c90fc640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9c40cc310 0x7ff9c4139bc0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.837+0000 7ff9c90fc640 1 --2- 172.21.15.26:0/133018351 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff9c40cb150 0x7ff9c414b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.837+0000 7ff9c90fc640 1 -- 172.21.15.26:0/133018351 >> 172.21.15.26:0/133018351 conn(0x7ff9c4066160 msgr2=0x7ff9c40ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.837+0000 7ff9c90fc640 1 -- 172.21.15.26:0/133018351 shutdown_connections 2023-12-19T18:17:09.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:09.837+0000 7ff9c90fc640 1 -- 172.21.15.26:0/133018351 wait complete. 2023-12-19T18:17:09.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:17:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.053+0000 7f1ed5109640 1 Processor -- start 2023-12-19T18:17:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.053+0000 7f1ed5109640 1 -- start start 2023-12-19T18:17:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.053+0000 7f1ed5109640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1ed0159830 0x7f1ed015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.053+0000 7f1ed5109640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ed00cb110 0x7f1ed00cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.053+0000 7f1ed5109640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed015e3d0 con 0x7f1ed00c9310 2023-12-19T18:17:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.053+0000 7f1ed5109640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed006f7c0 con 0x7f1ed00cb110 2023-12-19T18:17:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.053+0000 7f1ed5109640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed00ca2b0 con 0x7f1ed0159830 2023-12-19T18:17:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.057+0000 7f1ecf59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ed00cb110 0x7f1ed00cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.057+0000 7f1ecf59e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ed00cb110 0x7f1ed00cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48976/0 (socket says 172.21.15.26:48976) 2023-12-19T18:17:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.057+0000 7f1ecf59e640 1 -- 172.21.15.26:0/1975548659 learned_addr learned my addr 172.21.15.26:0/1975548659 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.057+0000 7f1eced9d640 1 --2- 172.21.15.26:0/1975548659 >> v2:172.21.15.160:3300/0 conn(0x7f1ed0159830 0x7f1ed015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.057+0000 7f1ecf59e640 1 -- 172.21.15.26:0/1975548659 >> v2:172.21.15.160:3300/0 conn(0x7f1ed0159830 msgr2=0x7f1ed015d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.057+0000 7f1ecf59e640 1 --2- 172.21.15.26:0/1975548659 >> v2:172.21.15.160:3300/0 conn(0x7f1ed0159830 0x7f1ed015d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.057+0000 7f1ecf59e640 1 -- 172.21.15.26:0/1975548659 >> v1:172.21.15.160:6789/0 conn(0x7f1ed00c9310 legacy=0x7f1ed00cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.057+0000 7f1ecf59e640 1 -- 172.21.15.26:0/1975548659 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed00cba80 con 0x7f1ed00cb110 2023-12-19T18:17:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.057+0000 7f1ecf59e640 1 --2- 172.21.15.26:0/1975548659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00cb110 0x7f1ed00cb540 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f1ed006ce60 tx=0x7f1ec801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99308a7e9ee96e64 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.057+0000 7f1ece59c640 1 -- 172.21.15.26:0/1975548659 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec80a56f0 con 0x7f1ed00cb110 2023-12-19T18:17:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.057+0000 7f1ece59c640 1 -- 172.21.15.26:0/1975548659 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec80a5850 con 0x7f1ed00cb110 2023-12-19T18:17:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.057+0000 7f1ece59c640 1 -- 172.21.15.26:0/1975548659 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec80a5be0 con 0x7f1ed00cb110 2023-12-19T18:17:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 -- 172.21.15.26:0/1975548659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00cb110 msgr2=0x7f1ed00cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 --2- 172.21.15.26:0/1975548659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00cb110 0x7f1ed00cb540 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f1ed006ce60 tx=0x7f1ec801c940 comp rx=0 tx=0).stop 2023-12-19T18:17:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 -- 172.21.15.26:0/1975548659 shutdown_connections 2023-12-19T18:17:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 --2- 172.21.15.26:0/1975548659 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00cb110 0x7f1ed00cb540 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 --2- 172.21.15.26:0/1975548659 >> v2:172.21.15.160:3300/0 conn(0x7f1ed0159830 0x7f1ed015d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 -- 172.21.15.26:0/1975548659 >> 172.21.15.26:0/1975548659 conn(0x7f1ed00b0270 msgr2=0x7f1ed00b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 -- 172.21.15.26:0/1975548659 shutdown_connections 2023-12-19T18:17:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 -- 172.21.15.26:0/1975548659 wait complete. 2023-12-19T18:17:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 Processor -- start 2023-12-19T18:17:10.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 -- start start 2023-12-19T18:17:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00c9310 0x7f1ed00c2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00cb110 0x7f1ed00b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed0159830 0x7f1ed01521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed00c4b40 con 0x7f1ed00c9310 2023-12-19T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed00b4350 con 0x7f1ed00cb110 2023-12-19T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ed5109640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed00c2930 con 0x7f1ed0159830 2023-12-19T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1eced9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00c9310 0x7f1ed00c2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1eced9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00c9310 0x7f1ed00c2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48984/0 (socket says 172.21.15.26:48984) 2023-12-19T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1eced9d640 1 -- 172.21.15.26:0/1040348851 learned_addr learned my addr 172.21.15.26:0/1040348851 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1eced9d640 1 -- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed0159830 msgr2=0x7f1ed01521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1eced9d640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed0159830 0x7f1ed01521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.061+0000 7f1ec7fff640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00cb110 0x7f1ed00b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1eced9d640 1 -- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00cb110 msgr2=0x7f1ed00b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1eced9d640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00cb110 0x7f1ed00b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1eced9d640 1 -- 172.21.15.26:0/1040348851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec8009b30 con 0x7f1ed00c9310 2023-12-19T18:17:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1ec7fff640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00cb110 0x7f1ed00b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1eced9d640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00c9310 0x7f1ed00c2230 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000d620 tx=0x7f1ec000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1ec77fe640 1 -- 172.21.15.26:0/1040348851 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000b620 con 0x7f1ed00c9310 2023-12-19T18:17:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1ec77fe640 1 -- 172.21.15.26:0/1040348851 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec000b780 con 0x7f1ed00c9310 2023-12-19T18:17:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1ec77fe640 1 -- 172.21.15.26:0/1040348851 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000bb10 con 0x7f1ed00c9310 2023-12-19T18:17:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1ed5109640 1 -- 172.21.15.26:0/1040348851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed00c47e0 con 0x7f1ed00c9310 2023-12-19T18:17:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1ed5109640 1 -- 172.21.15.26:0/1040348851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed00b1970 con 0x7f1ed00c9310 2023-12-19T18:17:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1ed5109640 1 -- 172.21.15.26:0/1040348851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed0074270 con 0x7f1ed00c9310 2023-12-19T18:17:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1ec77fe640 1 -- 172.21.15.26:0/1040348851 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f1ec000bc70 con 0x7f1ed00c9310 2023-12-19T18:17:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1ec77fe640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1eb0065ca0 0x7f1eb0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1ec77fe640 1 -- 172.21.15.26:0/1040348851 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 8150+0+0 (secure 0 0 0) 0x7f1ec0084440 con 0x7f1ed00c9310 2023-12-19T18:17:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.065+0000 7f1ec7fff640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1eb0065ca0 0x7f1eb0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.069+0000 7f1ec7fff640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1eb0065ca0 0x7f1eb0068160 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0158dc0 tx=0x7f1eb8009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.073+0000 7f1ec77fe640 1 -- 172.21.15.26:0/1040348851 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1ec0051110 con 0x7f1ed00c9310 2023-12-19T18:17:10.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:10.229+0000 7f1ed5109640 1 -- 172.21.15.26:0/1040348851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1ed006fa30 con 0x7f1ed00c9310 2023-12-19T18:17:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.133+0000 7f1ec77fe640 1 -- 172.21.15.26:0/1040348851 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v220) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1ec0054730 con 0x7f1ed00c9310 2023-12-19T18:17:11.138 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:17:11.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.137+0000 7f1ed5109640 1 -- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1eb0065ca0 msgr2=0x7f1eb0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.137+0000 7f1ed5109640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1eb0065ca0 0x7f1eb0068160 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0158dc0 tx=0x7f1eb8009040 comp rx=0 tx=0).stop 2023-12-19T18:17:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.137+0000 7f1ed5109640 1 -- 172.21.15.26:0/1040348851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00c9310 msgr2=0x7f1ed00c2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.137+0000 7f1ed5109640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00c9310 0x7f1ed00c2230 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000d620 tx=0x7f1ec000dae0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.137+0000 7f1ed5109640 1 -- 172.21.15.26:0/1040348851 shutdown_connections 2023-12-19T18:17:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.137+0000 7f1ed5109640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1eb0065ca0 0x7f1eb0068160 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.137+0000 7f1ed5109640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed0159830 0x7f1ed01521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.137+0000 7f1ed5109640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed00cb110 0x7f1ed00b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.137+0000 7f1ed5109640 1 --2- 172.21.15.26:0/1040348851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed00c9310 0x7f1ed00c2230 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.137+0000 7f1ed5109640 1 -- 172.21.15.26:0/1040348851 >> 172.21.15.26:0/1040348851 conn(0x7f1ed00b0270 msgr2=0x7f1ed00b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.137+0000 7f1ed5109640 1 -- 172.21.15.26:0/1040348851 shutdown_connections 2023-12-19T18:17:11.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.137+0000 7f1ed5109640 1 -- 172.21.15.26:0/1040348851 wait complete. 2023-12-19T18:17:11.171 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:17:11.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0af6d640 1 Processor -- start 2023-12-19T18:17:11.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0af6d640 1 -- start start 2023-12-19T18:17:11.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0af6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f04159830 0x7f1f0415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0af6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f040cb110 0x7f1f040cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0af6d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0415e3d0 con 0x7f1f040c9310 2023-12-19T18:17:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0af6d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0406f7c0 con 0x7f1f040cb110 2023-12-19T18:17:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0af6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f040ca2b0 con 0x7f1f04159830 2023-12-19T18:17:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0950a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f040cb110 0x7f1f040cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0950a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f040cb110 0x7f1f040cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48988/0 (socket says 172.21.15.26:48988) 2023-12-19T18:17:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0950a640 1 -- 172.21.15.26:0/349727578 learned_addr learned my addr 172.21.15.26:0/349727578 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f08d09640 1 --2- 172.21.15.26:0/349727578 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f04159830 0x7f1f0415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0950a640 1 -- 172.21.15.26:0/349727578 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f04159830 msgr2=0x7f1f0415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0950a640 1 --2- 172.21.15.26:0/349727578 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f04159830 0x7f1f0415d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0950a640 1 -- 172.21.15.26:0/349727578 >> v1:172.21.15.160:6789/0 conn(0x7f1f040c9310 legacy=0x7f1f040cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.321+0000 7f1f0950a640 1 -- 172.21.15.26:0/349727578 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f040cba80 con 0x7f1f040cb110 2023-12-19T18:17:11.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.325+0000 7f1f0950a640 1 --2- 172.21.15.26:0/349727578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f040cb110 0x7f1f040cb540 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009e50 tx=0x7f1efc01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd2075bc47d0271 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:11.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.325+0000 7f1f037fe640 1 -- 172.21.15.26:0/349727578 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0a5730 con 0x7f1f040cb110 2023-12-19T18:17:11.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.325+0000 7f1f037fe640 1 -- 172.21.15.26:0/349727578 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc0a5890 con 0x7f1f040cb110 2023-12-19T18:17:11.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.325+0000 7f1f037fe640 1 -- 172.21.15.26:0/349727578 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0a5c20 con 0x7f1f040cb110 2023-12-19T18:17:11.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.325+0000 7f1f0af6d640 1 -- 172.21.15.26:0/349727578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f040cb110 msgr2=0x7f1f040cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.325+0000 7f1f0af6d640 1 --2- 172.21.15.26:0/349727578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f040cb110 0x7f1f040cb540 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009e50 tx=0x7f1efc01c980 comp rx=0 tx=0).stop 2023-12-19T18:17:11.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.325+0000 7f1f0af6d640 1 -- 172.21.15.26:0/349727578 shutdown_connections 2023-12-19T18:17:11.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.325+0000 7f1f0af6d640 1 --2- 172.21.15.26:0/349727578 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f040cb110 0x7f1f040cb540 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.325+0000 7f1f0af6d640 1 --2- 172.21.15.26:0/349727578 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f04159830 0x7f1f0415d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.325+0000 7f1f0af6d640 1 -- 172.21.15.26:0/349727578 >> 172.21.15.26:0/349727578 conn(0x7f1f040b0270 msgr2=0x7f1f040b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:11.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.325+0000 7f1f0af6d640 1 -- 172.21.15.26:0/349727578 shutdown_connections 2023-12-19T18:17:11.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.325+0000 7f1f0af6d640 1 -- 172.21.15.26:0/349727578 wait complete. 2023-12-19T18:17:11.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f0af6d640 1 Processor -- start 2023-12-19T18:17:11.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f0af6d640 1 -- start start 2023-12-19T18:17:11.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f0af6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f040c9310 0x7f1f040c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f0af6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f040cb110 0x7f1f040c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f0af6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f04159830 0x7f1f0406b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f0af6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f040b1b70 con 0x7f1f040cb110 2023-12-19T18:17:11.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f0af6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f040c5170 con 0x7f1f04159830 2023-12-19T18:17:11.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f0af6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0414f750 con 0x7f1f040c9310 2023-12-19T18:17:11.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f03fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f040cb110 0x7f1f040c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f03fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f040cb110 0x7f1f040c3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48990/0 (socket says 172.21.15.26:48990) 2023-12-19T18:17:11.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f03fff640 1 -- 172.21.15.26:0/3547503073 learned_addr learned my addr 172.21.15.26:0/3547503073 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:11.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f0950a640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f04159830 0x7f1f0406b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f08d09640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f040c9310 0x7f1f040c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f03fff640 1 -- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f040c9310 msgr2=0x7f1f040c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f03fff640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f040c9310 0x7f1f040c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f03fff640 1 -- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f04159830 msgr2=0x7f1f0406b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f03fff640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f04159830 0x7f1f0406b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f03fff640 1 -- 172.21.15.26:0/3547503073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc009b30 con 0x7f1f040cb110 2023-12-19T18:17:11.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f0950a640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f04159830 0x7f1f0406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:11.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f03fff640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f040cb110 0x7f1f040c3da0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000c4d0 tx=0x7f1ef000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:11.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f017fa640 1 -- 172.21.15.26:0/3547503073 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00052d0 con 0x7f1f040cb110 2023-12-19T18:17:11.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f017fa640 1 -- 172.21.15.26:0/3547503073 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef0005430 con 0x7f1f040cb110 2023-12-19T18:17:11.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f017fa640 1 -- 172.21.15.26:0/3547503073 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0012830 con 0x7f1f040cb110 2023-12-19T18:17:11.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f0af6d640 1 -- 172.21.15.26:0/3547503073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f040b1360 con 0x7f1f040cb110 2023-12-19T18:17:11.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f0af6d640 1 -- 172.21.15.26:0/3547503073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f040b15b0 con 0x7f1f040cb110 2023-12-19T18:17:11.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.329+0000 7f1f0af6d640 1 -- 172.21.15.26:0/3547503073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f04074270 con 0x7f1f040cb110 2023-12-19T18:17:11.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.333+0000 7f1f017fa640 1 -- 172.21.15.26:0/3547503073 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f1ef0012990 con 0x7f1f040cb110 2023-12-19T18:17:11.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.333+0000 7f1f017fa640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ee8065ca0 0x7f1ee8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.333+0000 7f1f017fa640 1 -- 172.21.15.26:0/3547503073 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 7712+0+0 (secure 0 0 0) 0x7f1ef0083920 con 0x7f1f040cb110 2023-12-19T18:17:11.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.333+0000 7f1f08d09640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ee8065ca0 0x7f1ee8068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.333+0000 7f1f08d09640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ee8065ca0 0x7f1ee8068160 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f04159e70 tx=0x7f1ef8008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:11.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.337+0000 7f1f017fa640 1 -- 172.21.15.26:0/3547503073 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1ef0050730 con 0x7f1f040cb110 2023-12-19T18:17:11.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:11.493+0000 7f1f0af6d640 1 -- 172.21.15.26:0/3547503073 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1f040b46c0 con 0x7f1f040cb110 2023-12-19T18:17:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.141+0000 7f1f017fa640 1 -- 172.21.15.26:0/3547503073 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v221) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1ef0053d50 con 0x7f1f040cb110 2023-12-19T18:17:12.148 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T18:17:12.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.145+0000 7f1f0af6d640 1 -- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ee8065ca0 msgr2=0x7f1ee8068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.145+0000 7f1f0af6d640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ee8065ca0 0x7f1ee8068160 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1f04159e70 tx=0x7f1ef8008040 comp rx=0 tx=0).stop 2023-12-19T18:17:12.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.145+0000 7f1f0af6d640 1 -- 172.21.15.26:0/3547503073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f040cb110 msgr2=0x7f1f040c3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.145+0000 7f1f0af6d640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f040cb110 0x7f1f040c3da0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f1ef000c4d0 tx=0x7f1ef000c990 comp rx=0 tx=0).stop 2023-12-19T18:17:12.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.149+0000 7f1f0af6d640 1 -- 172.21.15.26:0/3547503073 shutdown_connections 2023-12-19T18:17:12.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.149+0000 7f1f0af6d640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ee8065ca0 0x7f1ee8068160 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.149+0000 7f1f0af6d640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f04159830 0x7f1f0406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.149+0000 7f1f0af6d640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f040cb110 0x7f1f040c3da0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.149+0000 7f1f0af6d640 1 --2- 172.21.15.26:0/3547503073 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f040c9310 0x7f1f040c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.149+0000 7f1f0af6d640 1 -- 172.21.15.26:0/3547503073 >> 172.21.15.26:0/3547503073 conn(0x7f1f040b0270 msgr2=0x7f1f040af790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:12.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.149+0000 7f1f0af6d640 1 -- 172.21.15.26:0/3547503073 shutdown_connections 2023-12-19T18:17:12.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.149+0000 7f1f0af6d640 1 -- 172.21.15.26:0/3547503073 wait complete. 2023-12-19T18:17:12.181 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:12.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9f59e640 1 Processor -- start 2023-12-19T18:17:12.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9f59e640 1 -- start start 2023-12-19T18:17:12.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9f59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aa01606a0 0x7f2aa0160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9f59e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa00cf8c0 con 0x7f2aa00b1d90 2023-12-19T18:17:12.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9f59e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa0160fc0 con 0x7f2aa00ce370 2023-12-19T18:17:12.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9f59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00cf5e0 con 0x7f2aa01606a0 2023-12-19T18:17:12.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2aa00b1d90 0x7f2aa006a2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58210/0 (socket says 172.21.15.26:58210) 2023-12-19T18:17:12.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9ed9d640 1 -- 172.21.15.26:0/3846190443 learned_addr learned my addr 172.21.15.26:0/3846190443 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:12.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9e59c640 1 --2- 172.21.15.26:0/3846190443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aa01606a0 0x7f2aa0160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9d59a640 1 -- 172.21.15.26:0/3846190443 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1640235922 0 0) 0x7f2a980014a0 con 0x7f2aa00b1d90 2023-12-19T18:17:12.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9d59a640 1 -- 172.21.15.26:0/3846190443 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a88003530 con 0x7f2aa00b1d90 2023-12-19T18:17:12.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9d59a640 1 -- 172.21.15.26:0/3846190443 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648898577 0 0) 0x7f2aa0160fc0 con 0x7f2aa00ce370 2023-12-19T18:17:12.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9d59a640 1 -- 172.21.15.26:0/3846190443 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a980014a0 con 0x7f2aa00ce370 2023-12-19T18:17:12.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9e59c640 1 -- 172.21.15.26:0/3846190443 >> v1:172.21.15.160:6789/0 conn(0x7f2aa00ce370 legacy=0x7f2aa015c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9e59c640 1 -- 172.21.15.26:0/3846190443 >> v1:172.21.15.26:6789/0 conn(0x7f2aa00b1d90 legacy=0x7f2aa006a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.337+0000 7f2a9e59c640 1 -- 172.21.15.26:0/3846190443 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa00b02a0 con 0x7f2aa01606a0 2023-12-19T18:17:12.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.341+0000 7f2a9e59c640 1 --2- 172.21.15.26:0/3846190443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aa01606a0 0x7f2aa0160a80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2a900097f0 tx=0x7f2a9001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d394b1a93fe88104 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:12.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.341+0000 7f2a9d59a640 1 -- 172.21.15.26:0/3846190443 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a900aa5f0 con 0x7f2aa01606a0 2023-12-19T18:17:12.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.341+0000 7f2a9d59a640 1 -- 172.21.15.26:0/3846190443 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a900aa750 con 0x7f2aa01606a0 2023-12-19T18:17:12.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.341+0000 7f2a9d59a640 1 -- 172.21.15.26:0/3846190443 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a900aaac0 con 0x7f2aa01606a0 2023-12-19T18:17:12.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.341+0000 7f2a9f59e640 1 -- 172.21.15.26:0/3846190443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aa01606a0 msgr2=0x7f2aa0160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.341+0000 7f2a9f59e640 1 --2- 172.21.15.26:0/3846190443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aa01606a0 0x7f2aa0160a80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2a900097f0 tx=0x7f2a9001c490 comp rx=0 tx=0).stop 2023-12-19T18:17:12.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.341+0000 7f2a9f59e640 1 -- 172.21.15.26:0/3846190443 shutdown_connections 2023-12-19T18:17:12.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.341+0000 7f2a9f59e640 1 --2- 172.21.15.26:0/3846190443 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aa01606a0 0x7f2aa0160a80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.341+0000 7f2a9f59e640 1 -- 172.21.15.26:0/3846190443 >> 172.21.15.26:0/3846190443 conn(0x7f2aa00d07a0 msgr2=0x7f2aa00d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:12.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.341+0000 7f2a9f59e640 1 -- 172.21.15.26:0/3846190443 shutdown_connections 2023-12-19T18:17:12.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.341+0000 7f2a9f59e640 1 -- 172.21.15.26:0/3846190443 wait complete. 2023-12-19T18:17:12.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9f59e640 1 Processor -- start 2023-12-19T18:17:12.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9f59e640 1 -- start start 2023-12-19T18:17:12.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9f59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aa00b1d90 0x7f2aa0155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9f59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa00ce370 0x7f2aa015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9f59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aa01606a0 0x7f2aa00b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9f59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00b3db0 con 0x7f2aa00ce370 2023-12-19T18:17:12.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9f59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa015b6f0 con 0x7f2aa01606a0 2023-12-19T18:17:12.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9f59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00b36f0 con 0x7f2aa00b1d90 2023-12-19T18:17:12.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa00ce370 0x7f2aa015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa00ce370 0x7f2aa015abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48992/0 (socket says 172.21.15.26:48992) 2023-12-19T18:17:12.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9dd9b640 1 -- 172.21.15.26:0/118407852 learned_addr learned my addr 172.21.15.26:0/118407852 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:12.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9e59c640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aa00b1d90 0x7f2aa0155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9ed9d640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aa01606a0 0x7f2aa00b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9dd9b640 1 -- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aa00b1d90 msgr2=0x7f2aa0155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9dd9b640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aa00b1d90 0x7f2aa0155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9dd9b640 1 -- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aa01606a0 msgr2=0x7f2aa00b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9dd9b640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aa01606a0 0x7f2aa00b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9dd9b640 1 -- 172.21.15.26:0/118407852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a90009420 con 0x7f2aa00ce370 2023-12-19T18:17:12.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9dd9b640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa00ce370 0x7f2aa015abf0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400c500 tx=0x7f2a9400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:12.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a8effd640 1 -- 172.21.15.26:0/118407852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940052d0 con 0x7f2aa00ce370 2023-12-19T18:17:12.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a8effd640 1 -- 172.21.15.26:0/118407852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a94005450 con 0x7f2aa00ce370 2023-12-19T18:17:12.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9f59e640 1 -- 172.21.15.26:0/118407852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa00b39d0 con 0x7f2aa00ce370 2023-12-19T18:17:12.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a8effd640 1 -- 172.21.15.26:0/118407852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94012870 con 0x7f2aa00ce370 2023-12-19T18:17:12.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9f59e640 1 -- 172.21.15.26:0/118407852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa01563e0 con 0x7f2aa00ce370 2023-12-19T18:17:12.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.345+0000 7f2a9f59e640 1 -- 172.21.15.26:0/118407852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa00b3ef0 con 0x7f2aa00ce370 2023-12-19T18:17:12.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.349+0000 7f2a8effd640 1 -- 172.21.15.26:0/118407852 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f2a940129d0 con 0x7f2aa00ce370 2023-12-19T18:17:12.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.349+0000 7f2a8effd640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2a80065ca0 0x7f2a80068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.349+0000 7f2a8effd640 1 -- 172.21.15.26:0/118407852 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f2a940836f0 con 0x7f2aa00ce370 2023-12-19T18:17:12.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.349+0000 7f2a9e59c640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2a80065ca0 0x7f2a80068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.349+0000 7f2a9e59c640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2a80065ca0 0x7f2a80068160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2a900ae000 tx=0x7f2a900a7000 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:12.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.353+0000 7f2a8effd640 1 -- 172.21.15.26:0/118407852 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2aa00b3ef0 con 0x7f2aa00ce370 2023-12-19T18:17:12.488 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:17:12.492 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:17:12.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.517+0000 7f2a9f59e640 1 -- 172.21.15.26:0/118407852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2aa00b3ef0 con 0x7f2aa00ce370 2023-12-19T18:17:12.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.517+0000 7f2a8effd640 1 -- 172.21.15.26:0/118407852 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v221) v1 ==== 74+0+18472 (secure 0 0 0) 0x7f2aa00b3ef0 con 0x7f2aa00ce370 2023-12-19T18:17:12.522 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:12.522 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":221,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:12.136783+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:16:38.226152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:16:39.540217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1665633474":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3561084215":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1351823889":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3136172808":"2023-12-20T18:17:04.905418+0000","172.21.15.26:6835/3981286992":"2023-12-20T18:17:08.808978+0000","172.21.15.26:6834/3981286992":"2023-12-20T18:17:08.808978+0000","172.21.15.160:0/3161716183":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/2592629251":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1089029505":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1544351773":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1134319862":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/4291270549":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3401801452":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/765362777":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/1696607504":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6833/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6832/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/2355423136":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/3498637577":"2023-12-20T18:16:55.864876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.521+0000 7f2a9f59e640 1 -- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2a80065ca0 msgr2=0x7f2a80068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.521+0000 7f2a9f59e640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2a80065ca0 0x7f2a80068160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2a900ae000 tx=0x7f2a900a7000 comp rx=0 tx=0).stop 2023-12-19T18:17:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.521+0000 7f2a9f59e640 1 -- 172.21.15.26:0/118407852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa00ce370 msgr2=0x7f2aa015abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.521+0000 7f2a9f59e640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa00ce370 0x7f2aa015abf0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400c500 tx=0x7f2a9400c9c0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.521+0000 7f2a9f59e640 1 -- 172.21.15.26:0/118407852 shutdown_connections 2023-12-19T18:17:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.521+0000 7f2a9f59e640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2a80065ca0 0x7f2a80068160 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.521+0000 7f2a9f59e640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2aa01606a0 0x7f2aa00b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.521+0000 7f2a9f59e640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2aa00ce370 0x7f2aa015abf0 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.521+0000 7f2a9f59e640 1 --2- 172.21.15.26:0/118407852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2aa00b1d90 0x7f2aa0155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.521+0000 7f2a9f59e640 1 -- 172.21.15.26:0/118407852 >> 172.21.15.26:0/118407852 conn(0x7f2aa00d07a0 msgr2=0x7f2aa006d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.521+0000 7f2a9f59e640 1 -- 172.21.15.26:0/118407852 shutdown_connections 2023-12-19T18:17:12.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.521+0000 7f2a9f59e640 1 -- 172.21.15.26:0/118407852 wait complete. 2023-12-19T18:17:12.543 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-19T18:16:38.226152+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '215', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-19T18:16:39.540217+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '216', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:17:12.543 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:17:12.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18d71e9640 1 Processor -- start 2023-12-19T18:17:12.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18d71e9640 1 -- start start 2023-12-19T18:17:12.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18d71e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18d006f6c0 0x7f18d015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18d71e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d006fb60 con 0x7f18d0159c00 2023-12-19T18:17:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18d71e9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d006fe40 con 0x7f18d0159820 2023-12-19T18:17:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18d71e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d015e470 con 0x7f18d006f6c0 2023-12-19T18:17:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18d5786640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f18d0159c00 0x7f18d0070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58222/0 (socket says 172.21.15.26:58222) 2023-12-19T18:17:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18d5786640 1 -- 172.21.15.26:0/3228867509 learned_addr learned my addr 172.21.15.26:0/3228867509 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18cffff640 1 --2- 172.21.15.26:0/3228867509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18d006f6c0 0x7f18d015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18cf7fe640 1 -- 172.21.15.26:0/3228867509 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971956282 0 0) 0x7f18d006fb60 con 0x7f18d0159c00 2023-12-19T18:17:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18cf7fe640 1 -- 172.21.15.26:0/3228867509 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18c0003530 con 0x7f18d0159c00 2023-12-19T18:17:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18cf7fe640 1 -- 172.21.15.26:0/3228867509 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973860883 0 0) 0x7f18d006fe40 con 0x7f18d0159820 2023-12-19T18:17:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18cf7fe640 1 -- 172.21.15.26:0/3228867509 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18d006fb60 con 0x7f18d0159820 2023-12-19T18:17:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18cffff640 1 -- 172.21.15.26:0/3228867509 >> v1:172.21.15.160:6789/0 conn(0x7f18d0159820 legacy=0x7f18d00c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18cffff640 1 -- 172.21.15.26:0/3228867509 >> v1:172.21.15.26:6789/0 conn(0x7f18d0159c00 legacy=0x7f18d0070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.729+0000 7f18cffff640 1 -- 172.21.15.26:0/3228867509 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d0070120 con 0x7f18d006f6c0 2023-12-19T18:17:12.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.733+0000 7f18cffff640 1 --2- 172.21.15.26:0/3228867509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18d006f6c0 0x7f18d015df30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0097f0 tx=0x7f18bc01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ae6b110534c041c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.733+0000 7f18cf7fe640 1 -- 172.21.15.26:0/3228867509 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0aa5f0 con 0x7f18d006f6c0 2023-12-19T18:17:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.733+0000 7f18cf7fe640 1 -- 172.21.15.26:0/3228867509 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18bc0aa750 con 0x7f18d006f6c0 2023-12-19T18:17:12.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.733+0000 7f18cf7fe640 1 -- 172.21.15.26:0/3228867509 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0aaae0 con 0x7f18d006f6c0 2023-12-19T18:17:12.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.733+0000 7f18d71e9640 1 -- 172.21.15.26:0/3228867509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18d006f6c0 msgr2=0x7f18d015df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.733+0000 7f18d71e9640 1 --2- 172.21.15.26:0/3228867509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18d006f6c0 0x7f18d015df30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0097f0 tx=0x7f18bc01c490 comp rx=0 tx=0).stop 2023-12-19T18:17:12.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.733+0000 7f18d71e9640 1 -- 172.21.15.26:0/3228867509 shutdown_connections 2023-12-19T18:17:12.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.733+0000 7f18d71e9640 1 --2- 172.21.15.26:0/3228867509 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18d006f6c0 0x7f18d015df30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.733+0000 7f18d71e9640 1 -- 172.21.15.26:0/3228867509 >> 172.21.15.26:0/3228867509 conn(0x7f18d00ce730 msgr2=0x7f18d00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:12.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.733+0000 7f18d71e9640 1 -- 172.21.15.26:0/3228867509 shutdown_connections 2023-12-19T18:17:12.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.733+0000 7f18d71e9640 1 -- 172.21.15.26:0/3228867509 wait complete. 2023-12-19T18:17:12.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d71e9640 1 Processor -- start 2023-12-19T18:17:12.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d71e9640 1 -- start start 2023-12-19T18:17:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d71e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d006f6c0 0x7f18d00c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d71e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18d0159820 0x7f18d015c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d71e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18d0159c00 0x7f18d00cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d71e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00c5930 con 0x7f18d0159c00 2023-12-19T18:17:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d71e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00c4970 con 0x7f18d006f6c0 2023-12-19T18:17:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d71e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00c40d0 con 0x7f18d0159820 2023-12-19T18:17:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d5786640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18d0159c00 0x7f18d00cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d5786640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18d0159c00 0x7f18d00cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49008/0 (socket says 172.21.15.26:49008) 2023-12-19T18:17:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18cffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18d0159820 0x7f18d015c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d5786640 1 -- 172.21.15.26:0/373194213 learned_addr learned my addr 172.21.15.26:0/373194213 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d4f85640 1 --2- 172.21.15.26:0/373194213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d006f6c0 0x7f18d00c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d5786640 1 -- 172.21.15.26:0/373194213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18d0159820 msgr2=0x7f18d015c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d5786640 1 --2- 172.21.15.26:0/373194213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18d0159820 0x7f18d015c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d5786640 1 -- 172.21.15.26:0/373194213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d006f6c0 msgr2=0x7f18d00c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d5786640 1 --2- 172.21.15.26:0/373194213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d006f6c0 0x7f18d00c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d5786640 1 -- 172.21.15.26:0/373194213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc009420 con 0x7f18d0159c00 2023-12-19T18:17:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d5786640 1 --2- 172.21.15.26:0/373194213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18d0159c00 0x7f18d00cd650 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f18c800caa0 tx=0x7f18c800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18cd7fa640 1 -- 172.21.15.26:0/373194213 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c800ac00 con 0x7f18d0159c00 2023-12-19T18:17:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18cd7fa640 1 -- 172.21.15.26:0/373194213 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c800ad80 con 0x7f18d0159c00 2023-12-19T18:17:12.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18cd7fa640 1 -- 172.21.15.26:0/373194213 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8012640 con 0x7f18d0159c00 2023-12-19T18:17:12.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.737+0000 7f18d71e9640 1 -- 172.21.15.26:0/373194213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d006b880 con 0x7f18d0159c00 2023-12-19T18:17:12.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.741+0000 7f18d71e9640 1 -- 172.21.15.26:0/373194213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d00c4690 con 0x7f18d0159c00 2023-12-19T18:17:12.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.741+0000 7f18d71e9640 1 -- 172.21.15.26:0/373194213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d00c5aa0 con 0x7f18d0159c00 2023-12-19T18:17:12.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.741+0000 7f18cd7fa640 1 -- 172.21.15.26:0/373194213 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f18c8007070 con 0x7f18d0159c00 2023-12-19T18:17:12.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.741+0000 7f18cd7fa640 1 --2- 172.21.15.26:0/373194213 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f18b4065ca0 0x7f18b4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.741+0000 7f18cd7fa640 1 -- 172.21.15.26:0/373194213 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f18c8083ac0 con 0x7f18d0159c00 2023-12-19T18:17:12.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.741+0000 7f18d4f85640 1 --2- 172.21.15.26:0/373194213 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f18b4065ca0 0x7f18b4068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.741+0000 7f18d4f85640 1 --2- 172.21.15.26:0/373194213 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f18b4065ca0 0x7f18b4068160 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f18c40011f0 tx=0x7f18c4009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:12.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.745+0000 7f18cd7fa640 1 -- 172.21.15.26:0/373194213 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f18d00c5aa0 con 0x7f18d0159c00 2023-12-19T18:17:12.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.901+0000 7f18d71e9640 1 -- 172.21.15.26:0/373194213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18d00c5a70 con 0x7f18d0159c00 2023-12-19T18:17:12.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.901+0000 7f18cd7fa640 1 -- 172.21.15.26:0/373194213 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v221) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18c80170a0 con 0x7f18d0159c00 2023-12-19T18:17:12.907 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:17:12.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.905+0000 7f18d71e9640 1 -- 172.21.15.26:0/373194213 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f18b4065ca0 msgr2=0x7f18b4068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.905+0000 7f18d71e9640 1 --2- 172.21.15.26:0/373194213 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f18b4065ca0 0x7f18b4068160 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f18c40011f0 tx=0x7f18c4009040 comp rx=0 tx=0).stop 2023-12-19T18:17:12.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.905+0000 7f18d71e9640 1 -- 172.21.15.26:0/373194213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18d0159c00 msgr2=0x7f18d00cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.905+0000 7f18d71e9640 1 --2- 172.21.15.26:0/373194213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18d0159c00 0x7f18d00cd650 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f18c800caa0 tx=0x7f18c800cf60 comp rx=0 tx=0).stop 2023-12-19T18:17:12.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.905+0000 7f18d71e9640 1 -- 172.21.15.26:0/373194213 shutdown_connections 2023-12-19T18:17:12.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.905+0000 7f18d71e9640 1 --2- 172.21.15.26:0/373194213 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f18b4065ca0 0x7f18b4068160 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.905+0000 7f18d71e9640 1 --2- 172.21.15.26:0/373194213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18d0159c00 0x7f18d00cd650 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.905+0000 7f18d71e9640 1 --2- 172.21.15.26:0/373194213 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18d0159820 0x7f18d015c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.905+0000 7f18d71e9640 1 --2- 172.21.15.26:0/373194213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18d006f6c0 0x7f18d00c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.905+0000 7f18d71e9640 1 -- 172.21.15.26:0/373194213 >> 172.21.15.26:0/373194213 conn(0x7f18d00ce730 msgr2=0x7f18d00ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:12.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.905+0000 7f18d71e9640 1 -- 172.21.15.26:0/373194213 shutdown_connections 2023-12-19T18:17:12.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:12.905+0000 7f18d71e9640 1 -- 172.21.15.26:0/373194213 wait complete. 2023-12-19T18:17:12.928 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-19T18:17:13.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.069+0000 7ff13d5e9640 1 Processor -- start 2023-12-19T18:17:13.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff13d5e9640 1 -- start start 2023-12-19T18:17:13.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff13d5e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff138159820 0x7ff13815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff13d5e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13806fb60 con 0x7ff13806f6c0 2023-12-19T18:17:13.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff13d5e9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13806fe40 con 0x7ff138159c00 2023-12-19T18:17:13.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff13d5e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13815e470 con 0x7ff138159820 2023-12-19T18:17:13.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff12ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff13806f6c0 0x7ff1380c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58230/0 (socket says 172.21.15.26:58230) 2023-12-19T18:17:13.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff12ffff640 1 -- 172.21.15.26:0/1794612378 learned_addr learned my addr 172.21.15.26:0/1794612378 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:13.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff136ffd640 1 --2- 172.21.15.26:0/1794612378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff138159820 0x7ff13815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff1367fc640 1 -- 172.21.15.26:0/1794612378 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990031674 0 0) 0x7ff13806fb60 con 0x7ff13806f6c0 2023-12-19T18:17:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff1367fc640 1 -- 172.21.15.26:0/1794612378 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff124003530 con 0x7ff13806f6c0 2023-12-19T18:17:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff1367fc640 1 -- 172.21.15.26:0/1794612378 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016972226 0 0) 0x7ff13806fe40 con 0x7ff138159c00 2023-12-19T18:17:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff1367fc640 1 -- 172.21.15.26:0/1794612378 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff13806fb60 con 0x7ff138159c00 2023-12-19T18:17:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff136ffd640 1 -- 172.21.15.26:0/1794612378 >> v1:172.21.15.160:6789/0 conn(0x7ff138159c00 legacy=0x7ff138070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff1367fc640 1 -- 172.21.15.26:0/1794612378 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3270855105 0 0) 0x7ff124003530 con 0x7ff13806f6c0 2023-12-19T18:17:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff136ffd640 1 -- 172.21.15.26:0/1794612378 >> v1:172.21.15.26:6789/0 conn(0x7ff13806f6c0 legacy=0x7ff1380c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.073+0000 7ff136ffd640 1 -- 172.21.15.26:0/1794612378 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff138070120 con 0x7ff138159820 2023-12-19T18:17:13.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.077+0000 7ff136ffd640 1 --2- 172.21.15.26:0/1794612378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff138159820 0x7ff13815df30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff1280097f0 tx=0x7ff12801e5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9306c407f64724ab server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:13.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.077+0000 7ff1367fc640 1 -- 172.21.15.26:0/1794612378 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120001680 con 0x7ff138159820 2023-12-19T18:17:13.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.077+0000 7ff1367fc640 1 -- 172.21.15.26:0/1794612378 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1280ab410 con 0x7ff138159820 2023-12-19T18:17:13.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.077+0000 7ff1367fc640 1 -- 172.21.15.26:0/1794612378 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1280ab7a0 con 0x7ff138159820 2023-12-19T18:17:13.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.077+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1794612378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff138159820 msgr2=0x7ff13815df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.077+0000 7ff13d5e9640 1 --2- 172.21.15.26:0/1794612378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff138159820 0x7ff13815df30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff1280097f0 tx=0x7ff12801e5c0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.081 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.077+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1794612378 shutdown_connections 2023-12-19T18:17:13.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.077+0000 7ff13d5e9640 1 --2- 172.21.15.26:0/1794612378 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff138159820 0x7ff13815df30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.077+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1794612378 >> 172.21.15.26:0/1794612378 conn(0x7ff1380ce730 msgr2=0x7ff1380ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:13.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.077+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1794612378 shutdown_connections 2023-12-19T18:17:13.082 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.077+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1794612378 wait complete. 2023-12-19T18:17:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff13d5e9640 1 Processor -- start 2023-12-19T18:17:13.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff13d5e9640 1 -- start start 2023-12-19T18:17:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff13d5e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff13806f6c0 0x7ff13814c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff13d5e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff138159820 0x7ff13814c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff13d5e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff138159c00 0x7ff1380ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff13d5e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1381480b0 con 0x7ff138159c00 2023-12-19T18:17:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff13d5e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1380b19a0 con 0x7ff13806f6c0 2023-12-19T18:17:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff13d5e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1380af080 con 0x7ff138159820 2023-12-19T18:17:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff1377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff138159c00 0x7ff1380ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff136ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff13806f6c0 0x7ff13814c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff1377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff138159c00 0x7ff1380ae430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49020/0 (socket says 172.21.15.26:49020) 2023-12-19T18:17:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff1377fe640 1 -- 172.21.15.26:0/1679818021 learned_addr learned my addr 172.21.15.26:0/1679818021 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff12ffff640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff138159820 0x7ff13814c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff1377fe640 1 -- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff138159820 msgr2=0x7ff13814c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff1377fe640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff138159820 0x7ff13814c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff1377fe640 1 -- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff13806f6c0 msgr2=0x7ff13814c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff1377fe640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff13806f6c0 0x7ff13814c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff1377fe640 1 -- 172.21.15.26:0/1679818021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff128009420 con 0x7ff138159c00 2023-12-19T18:17:13.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff136ffd640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff13806f6c0 0x7ff13814c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:13.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff1377fe640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff138159c00 0x7ff1380ae430 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7ff13000c730 tx=0x7ff13000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:13.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff12f7fe640 1 -- 172.21.15.26:0/1679818021 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130003b80 con 0x7ff138159c00 2023-12-19T18:17:13.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff12f7fe640 1 -- 172.21.15.26:0/1679818021 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff130003ce0 con 0x7ff138159c00 2023-12-19T18:17:13.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff12f7fe640 1 -- 172.21.15.26:0/1679818021 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13000aa90 con 0x7ff138159c00 2023-12-19T18:17:13.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1679818021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13806b880 con 0x7ff138159c00 2023-12-19T18:17:13.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1679818021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1380b1350 con 0x7ff138159c00 2023-12-19T18:17:13.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.081+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1679818021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13806b680 con 0x7ff138159c00 2023-12-19T18:17:13.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.085+0000 7ff12f7fe640 1 -- 172.21.15.26:0/1679818021 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ff13000abf0 con 0x7ff138159c00 2023-12-19T18:17:13.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.085+0000 7ff12f7fe640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff118065cd0 0x7ff118068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.085+0000 7ff12f7fe640 1 -- 172.21.15.26:0/1679818021 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7298+0+0 (secure 0 0 0) 0x7ff1300838f0 con 0x7ff138159c00 2023-12-19T18:17:13.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.085+0000 7ff136ffd640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff118065cd0 0x7ff118068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.085+0000 7ff136ffd640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff118065cd0 0x7ff118068190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff1280b0000 tx=0x7ff1280a9000 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:13.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.089+0000 7ff12f7fe640 1 -- 172.21.15.26:0/1679818021 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff1300509c0 con 0x7ff138159c00 2023-12-19T18:17:13.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.249+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1679818021 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff138074270 con 0x7ff138159c00 2023-12-19T18:17:13.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.249+0000 7ff12f7fe640 1 -- 172.21.15.26:0/1679818021 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v221) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff130053fe0 con 0x7ff138159c00 2023-12-19T18:17:13.255 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:17:13.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.253+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff118065cd0 msgr2=0x7ff118068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.253+0000 7ff13d5e9640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff118065cd0 0x7ff118068190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff1280b0000 tx=0x7ff1280a9000 comp rx=0 tx=0).stop 2023-12-19T18:17:13.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.253+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1679818021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff138159c00 msgr2=0x7ff1380ae430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.253+0000 7ff13d5e9640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff138159c00 0x7ff1380ae430 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7ff13000c730 tx=0x7ff13000cbf0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.253+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1679818021 shutdown_connections 2023-12-19T18:17:13.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.253+0000 7ff13d5e9640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff118065cd0 0x7ff118068190 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.253+0000 7ff13d5e9640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff138159c00 0x7ff1380ae430 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.253+0000 7ff13d5e9640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff138159820 0x7ff13814c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.253+0000 7ff13d5e9640 1 --2- 172.21.15.26:0/1679818021 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff13806f6c0 0x7ff13814c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.253+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1679818021 >> 172.21.15.26:0/1679818021 conn(0x7ff1380ce730 msgr2=0x7ff13806ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:13.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.253+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1679818021 shutdown_connections 2023-12-19T18:17:13.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.253+0000 7ff13d5e9640 1 -- 172.21.15.26:0/1679818021 wait complete. 2023-12-19T18:17:13.276 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-19T18:17:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.417+0000 7f93a522d640 1 Processor -- start 2023-12-19T18:17:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.417+0000 7f93a522d640 1 -- start start 2023-12-19T18:17:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f93a522d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f93a006f6c0 0x7f93a015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f93a522d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a0159c00 0x7f93a0070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f93a522d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a006fb60 con 0x7f93a0159820 2023-12-19T18:17:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f93a522d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a015e470 con 0x7f93a006f6c0 2023-12-19T18:17:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f93a522d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a006fe40 con 0x7f93a0159c00 2023-12-19T18:17:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f939e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f93a006f6c0 0x7f93a015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f939ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f93a0159820 0x7f93a00c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58244/0 (socket says 172.21.15.26:58244) 2023-12-19T18:17:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f939ed9d640 1 -- 172.21.15.26:0/4097505540 learned_addr learned my addr 172.21.15.26:0/4097505540 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f939f59e640 1 --2- 172.21.15.26:0/4097505540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a0159c00 0x7f93a0070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f939e59c640 1 -- 172.21.15.26:0/4097505540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a0159c00 msgr2=0x7f93a0070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f939e59c640 1 --2- 172.21.15.26:0/4097505540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a0159c00 0x7f93a0070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f939e59c640 1 -- 172.21.15.26:0/4097505540 >> v1:172.21.15.26:6789/0 conn(0x7f93a0159820 legacy=0x7f93a00c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f939e59c640 1 -- 172.21.15.26:0/4097505540 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a0070120 con 0x7f93a006f6c0 2023-12-19T18:17:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f939f59e640 1 --2- 172.21.15.26:0/4097505540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a0159c00 0x7f93a0070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.421+0000 7f939dd9b640 1 -- 172.21.15.26:0/4097505540 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476500554 0 0) 0x7f93a006fb60 con 0x7f93a0159820 2023-12-19T18:17:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f939e59c640 1 --2- 172.21.15.26:0/4097505540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a006f6c0 0x7f93a015df30 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f93940097c0 tx=0x7f939401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8459648e007964f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f939dd9b640 1 -- 172.21.15.26:0/4097505540 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940ad660 con 0x7f93a006f6c0 2023-12-19T18:17:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f939dd9b640 1 -- 172.21.15.26:0/4097505540 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93940ad7c0 con 0x7f93a006f6c0 2023-12-19T18:17:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f939dd9b640 1 -- 172.21.15.26:0/4097505540 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940adb50 con 0x7f93a006f6c0 2023-12-19T18:17:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f93a522d640 1 -- 172.21.15.26:0/4097505540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a006f6c0 msgr2=0x7f93a015df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f93a522d640 1 --2- 172.21.15.26:0/4097505540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a006f6c0 0x7f93a015df30 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f93940097c0 tx=0x7f939401c450 comp rx=0 tx=0).stop 2023-12-19T18:17:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f93a522d640 1 -- 172.21.15.26:0/4097505540 shutdown_connections 2023-12-19T18:17:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f93a522d640 1 --2- 172.21.15.26:0/4097505540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a0159c00 0x7f93a0070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f93a522d640 1 --2- 172.21.15.26:0/4097505540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a006f6c0 0x7f93a015df30 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f93a522d640 1 -- 172.21.15.26:0/4097505540 >> 172.21.15.26:0/4097505540 conn(0x7f93a00ce730 msgr2=0x7f93a00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f93a522d640 1 -- 172.21.15.26:0/4097505540 shutdown_connections 2023-12-19T18:17:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f93a522d640 1 -- 172.21.15.26:0/4097505540 wait complete. 2023-12-19T18:17:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f93a522d640 1 Processor -- start 2023-12-19T18:17:13.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.425+0000 7f93a522d640 1 -- start start 2023-12-19T18:17:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f93a522d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93a006f6c0 0x7f93a014db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f93a522d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0159820 0x7f93a015b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f93a522d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a0159c00 0x7f93a015b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f93a522d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a015bf10 con 0x7f93a0159820 2023-12-19T18:17:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f93a522d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a015c1f0 con 0x7f93a006f6c0 2023-12-19T18:17:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f93a522d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a015c4d0 con 0x7f93a0159c00 2023-12-19T18:17:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f939e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0159820 0x7f93a015b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f939e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0159820 0x7f93a015b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49040/0 (socket says 172.21.15.26:49040) 2023-12-19T18:17:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f939e59c640 1 -- 172.21.15.26:0/3641680650 learned_addr learned my addr 172.21.15.26:0/3641680650 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f939ed9d640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93a006f6c0 0x7f93a014db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f939f59e640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a0159c00 0x7f93a015b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f939e59c640 1 -- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a0159c00 msgr2=0x7f93a015b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f939e59c640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a0159c00 0x7f93a015b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f939e59c640 1 -- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93a006f6c0 msgr2=0x7f93a014db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f939e59c640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93a006f6c0 0x7f93a014db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f939e59c640 1 -- 172.21.15.26:0/3641680650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9394009420 con 0x7f93a0159820 2023-12-19T18:17:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f939e59c640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0159820 0x7f93a015b390 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f939401ab00 tx=0x7f93940191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f93877fe640 1 -- 172.21.15.26:0/3641680650 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394004230 con 0x7f93a0159820 2023-12-19T18:17:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f93877fe640 1 -- 172.21.15.26:0/3641680650 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9394004390 con 0x7f93a0159820 2023-12-19T18:17:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f93a522d640 1 -- 172.21.15.26:0/3641680650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a006b880 con 0x7f93a0159820 2023-12-19T18:17:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f93877fe640 1 -- 172.21.15.26:0/3641680650 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394003820 con 0x7f93a0159820 2023-12-19T18:17:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f93a522d640 1 -- 172.21.15.26:0/3641680650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a015ca90 con 0x7f93a0159820 2023-12-19T18:17:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f93a522d640 1 -- 172.21.15.26:0/3641680650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a0074270 con 0x7f93a0159820 2023-12-19T18:17:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.429+0000 7f93877fe640 1 -- 172.21.15.26:0/3641680650 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f93940b4040 con 0x7f93a0159820 2023-12-19T18:17:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.433+0000 7f93877fe640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f938c065ca0 0x7f938c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.433+0000 7f93877fe640 1 -- 172.21.15.26:0/3641680650 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f939411d3a0 con 0x7f93a0159820 2023-12-19T18:17:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.433+0000 7f939ed9d640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f938c065ca0 0x7f938c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.433+0000 7f939ed9d640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f938c065ca0 0x7f938c068160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9390007fd0 tx=0x7f9390007d60 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.437+0000 7f93877fe640 1 -- 172.21.15.26:0/3641680650 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f93940ea230 con 0x7f93a0159820 2023-12-19T18:17:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.593+0000 7f93a522d640 1 -- 172.21.15.26:0/3641680650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f93a015d640 con 0x7f93a0159820 2023-12-19T18:17:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.593+0000 7f93877fe640 1 -- 172.21.15.26:0/3641680650 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v221) v1 ==== 103+0+11 (secure 0 0 0) 0x7f93940b20a0 con 0x7f93a0159820 2023-12-19T18:17:13.597 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:17:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.593+0000 7f93a522d640 1 -- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f938c065ca0 msgr2=0x7f938c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.593+0000 7f93a522d640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f938c065ca0 0x7f938c068160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9390007fd0 tx=0x7f9390007d60 comp rx=0 tx=0).stop 2023-12-19T18:17:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.597+0000 7f93a522d640 1 -- 172.21.15.26:0/3641680650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0159820 msgr2=0x7f93a015b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.597+0000 7f93a522d640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0159820 0x7f93a015b390 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f939401ab00 tx=0x7f93940191f0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.597+0000 7f93a522d640 1 -- 172.21.15.26:0/3641680650 shutdown_connections 2023-12-19T18:17:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.597+0000 7f93a522d640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f938c065ca0 0x7f938c068160 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.597+0000 7f93a522d640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f93a0159c00 0x7f93a015b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.597+0000 7f93a522d640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93a0159820 0x7f93a015b390 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.597+0000 7f93a522d640 1 --2- 172.21.15.26:0/3641680650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f93a006f6c0 0x7f93a014db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.597+0000 7f93a522d640 1 -- 172.21.15.26:0/3641680650 >> 172.21.15.26:0/3641680650 conn(0x7f93a00ce730 msgr2=0x7f93a0151c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.597+0000 7f93a522d640 1 -- 172.21.15.26:0/3641680650 shutdown_connections 2023-12-19T18:17:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.597+0000 7f93a522d640 1 -- 172.21.15.26:0/3641680650 wait complete. 2023-12-19T18:17:13.622 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:13.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.761+0000 7ffb0bc8c640 1 Processor -- start 2023-12-19T18:17:13.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.761+0000 7ffb0bc8c640 1 -- start start 2023-12-19T18:17:13.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.761+0000 7ffb0bc8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb04155030 0x7ffb0415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.761+0000 7ffb0bc8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb040ce740 0x7ffb040cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.761+0000 7ffb0bc8c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb0406c970 con 0x7ffb0415f370 2023-12-19T18:17:13.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.761+0000 7ffb0bc8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb0406c790 con 0x7ffb040ce740 2023-12-19T18:17:13.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.761+0000 7ffb0bc8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040cf130 con 0x7ffb04155030 2023-12-19T18:17:13.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.761+0000 7ffb0a229640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb040ce740 0x7ffb040cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.761+0000 7ffb0a229640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb040ce740 0x7ffb040cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49056/0 (socket says 172.21.15.26:49056) 2023-12-19T18:17:13.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.761+0000 7ffb0a229640 1 -- 172.21.15.26:0/2284472160 learned_addr learned my addr 172.21.15.26:0/2284472160 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.765+0000 7ffb0a229640 1 -- 172.21.15.26:0/2284472160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb04155030 msgr2=0x7ffb0415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.765+0000 7ffb09a28640 1 --2- 172.21.15.26:0/2284472160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb04155030 0x7ffb0415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.765+0000 7ffb0a229640 1 --2- 172.21.15.26:0/2284472160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb04155030 0x7ffb0415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.765+0000 7ffb0a229640 1 -- 172.21.15.26:0/2284472160 >> v1:172.21.15.160:6789/0 conn(0x7ffb0415f370 legacy=0x7ffb0406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.765+0000 7ffb0a229640 1 -- 172.21.15.26:0/2284472160 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0415b1f0 con 0x7ffb040ce740 2023-12-19T18:17:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.765+0000 7ffb09a28640 1 --2- 172.21.15.26:0/2284472160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb04155030 0x7ffb0415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:13.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.765+0000 7ffb0a229640 1 --2- 172.21.15.26:0/2284472160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb040ce740 0x7ffb040cebf0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009e50 tx=0x7ffafc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203d52b1330b1e88 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:13.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.765+0000 7ffb08a26640 1 -- 172.21.15.26:0/2284472160 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0a5820 con 0x7ffb040ce740 2023-12-19T18:17:13.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.765+0000 7ffb08a26640 1 -- 172.21.15.26:0/2284472160 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc0a5980 con 0x7ffb040ce740 2023-12-19T18:17:13.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.765+0000 7ffb08a26640 1 -- 172.21.15.26:0/2284472160 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0a5d10 con 0x7ffb040ce740 2023-12-19T18:17:13.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2284472160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb040ce740 msgr2=0x7ffb040cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 --2- 172.21.15.26:0/2284472160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb040ce740 0x7ffb040cebf0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009e50 tx=0x7ffafc01c9a0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2284472160 shutdown_connections 2023-12-19T18:17:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 --2- 172.21.15.26:0/2284472160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb040ce740 0x7ffb040cebf0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 --2- 172.21.15.26:0/2284472160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb04155030 0x7ffb0415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2284472160 >> 172.21.15.26:0/2284472160 conn(0x7ffb040658e0 msgr2=0x7ffb04065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2284472160 shutdown_connections 2023-12-19T18:17:13.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2284472160 wait complete. 2023-12-19T18:17:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 Processor -- start 2023-12-19T18:17:13.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 -- start start 2023-12-19T18:17:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb040ce740 0x7ffb040adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb04155030 0x7ffb041431b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb0415f370 0x7ffb0414cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040af590 con 0x7ffb0415f370 2023-12-19T18:17:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040af7c0 con 0x7ffb04155030 2023-12-19T18:17:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0bc8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040afaa0 con 0x7ffb040ce740 2023-12-19T18:17:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0a229640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb0415f370 0x7ffb0414cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0a229640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb0415f370 0x7ffb0414cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49064/0 (socket says 172.21.15.26:49064) 2023-12-19T18:17:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0a229640 1 -- 172.21.15.26:0/2508913030 learned_addr learned my addr 172.21.15.26:0/2508913030 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0a229640 1 -- 172.21.15.26:0/2508913030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb040ce740 msgr2=0x7ffb040adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0a229640 1 --2- 172.21.15.26:0/2508913030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb040ce740 0x7ffb040adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0a229640 1 -- 172.21.15.26:0/2508913030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb04155030 msgr2=0x7ffb041431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0a229640 1 --2- 172.21.15.26:0/2508913030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb04155030 0x7ffb041431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.769+0000 7ffb0a229640 1 -- 172.21.15.26:0/2508913030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc009b30 con 0x7ffb0415f370 2023-12-19T18:17:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.773+0000 7ffb0a229640 1 --2- 172.21.15.26:0/2508913030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb0415f370 0x7ffb0414cc30 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7ffafc0af040 tx=0x7ffafc019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.773+0000 7ffafa7fc640 1 -- 172.21.15.26:0/2508913030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0b8880 con 0x7ffb0415f370 2023-12-19T18:17:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.773+0000 7ffafa7fc640 1 -- 172.21.15.26:0/2508913030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc0b8a20 con 0x7ffb0415f370 2023-12-19T18:17:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.773+0000 7ffafa7fc640 1 -- 172.21.15.26:0/2508913030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0b8dd0 con 0x7ffb0415f370 2023-12-19T18:17:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.773+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2508913030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb040afd80 con 0x7ffb0415f370 2023-12-19T18:17:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.773+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2508913030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb040b0340 con 0x7ffb0415f370 2023-12-19T18:17:13.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.773+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2508913030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb040caf40 con 0x7ffb0415f370 2023-12-19T18:17:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.773+0000 7ffafa7fc640 1 -- 172.21.15.26:0/2508913030 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ffafc0c3460 con 0x7ffb0415f370 2023-12-19T18:17:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.773+0000 7ffafa7fc640 1 --2- 172.21.15.26:0/2508913030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffae8065ca0 0x7ffae8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.773+0000 7ffafa7fc640 1 -- 172.21.15.26:0/2508913030 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7298+0+0 (secure 0 0 0) 0x7ffafc11f460 con 0x7ffb0415f370 2023-12-19T18:17:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.773+0000 7ffb09a28640 1 --2- 172.21.15.26:0/2508913030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffae8065ca0 0x7ffae8068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.777+0000 7ffb09a28640 1 --2- 172.21.15.26:0/2508913030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffae8065ca0 0x7ffae8068160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000a8d0 tx=0x7ffb00008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:13.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.777+0000 7ffafa7fc640 1 -- 172.21.15.26:0/2508913030 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffafc0ec3e0 con 0x7ffb0415f370 2023-12-19T18:17:13.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2508913030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb040caf40 con 0x7ffb0415f370 2023-12-19T18:17:13.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffafa7fc640 1 -- 172.21.15.26:0/2508913030 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4731 (secure 0 0 0) 0x7ffafc0efa00 con 0x7ffb0415f370 2023-12-19T18:17:13.972 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:13.972 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":104,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3349262044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3349262044},{"type":"v1","addr":"172.21.15.160:6836","nonce":3349262044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3865658274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3865658274},{"type":"v1","addr":"172.21.15.26:6833","nonce":3865658274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":11262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1792196808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1792196808},{"type":"v1","addr":"172.21.15.26:6835","nonce":1792196808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:40.537606+0000","modified":"2023-12-19T18:16:43.550176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10605":{"gid":10605,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.160:6837/1294752100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1294752100},{"type":"v1","addr":"172.21.15.160:6837","nonce":1294752100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-19T18:17:13.972 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 104 2023-12-19T18:17:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2508913030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffae8065ca0 msgr2=0x7ffae8068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffb0bc8c640 1 --2- 172.21.15.26:0/2508913030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffae8065ca0 0x7ffae8068160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000a8d0 tx=0x7ffb00008040 comp rx=0 tx=0).stop 2023-12-19T18:17:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2508913030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb0415f370 msgr2=0x7ffb0414cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffb0bc8c640 1 --2- 172.21.15.26:0/2508913030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb0415f370 0x7ffb0414cc30 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7ffafc0af040 tx=0x7ffafc019740 comp rx=0 tx=0).stop 2023-12-19T18:17:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2508913030 shutdown_connections 2023-12-19T18:17:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffb0bc8c640 1 --2- 172.21.15.26:0/2508913030 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffae8065ca0 0x7ffae8068160 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffb0bc8c640 1 --2- 172.21.15.26:0/2508913030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb0415f370 0x7ffb0414cc30 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffb0bc8c640 1 --2- 172.21.15.26:0/2508913030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb04155030 0x7ffb041431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffb0bc8c640 1 --2- 172.21.15.26:0/2508913030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb040ce740 0x7ffb040adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2508913030 >> 172.21.15.26:0/2508913030 conn(0x7ffb040658e0 msgr2=0x7ffb04150280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:13.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2508913030 shutdown_connections 2023-12-19T18:17:13.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:13.969+0000 7ffb0bc8c640 1 -- 172.21.15.26:0/2508913030 wait complete. 2023-12-19T18:17:13.993 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:14.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.133+0000 7fe669c95640 1 Processor -- start 2023-12-19T18:17:14.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.133+0000 7fe669c95640 1 -- start start 2023-12-19T18:17:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.133+0000 7fe669c95640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe664154220 0x7fe66415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.133+0000 7fe669c95640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe66406b3d0 0x7fe6640ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe669c95640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66406bc30 con 0x7fe6640cec40 2023-12-19T18:17:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe669c95640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6640cf0d0 con 0x7fe664154220 2023-12-19T18:17:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe669c95640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6640cc6e0 con 0x7fe66406b3d0 2023-12-19T18:17:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe6637fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe664154220 0x7fe66415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe6637fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe664154220 0x7fe66415aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49078/0 (socket says 172.21.15.26:49078) 2023-12-19T18:17:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe6637fe640 1 -- 172.21.15.26:0/1907824217 learned_addr learned my addr 172.21.15.26:0/1907824217 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe6637fe640 1 -- 172.21.15.26:0/1907824217 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe66406b3d0 msgr2=0x7fe6640ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe6637fe640 1 --2- 172.21.15.26:0/1907824217 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe66406b3d0 0x7fe6640ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe6637fe640 1 -- 172.21.15.26:0/1907824217 >> v1:172.21.15.160:6789/0 conn(0x7fe6640cec40 legacy=0x7fe66415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:14.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe6637fe640 1 -- 172.21.15.26:0/1907824217 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66415afc0 con 0x7fe664154220 2023-12-19T18:17:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe6637fe640 1 --2- 172.21.15.26:0/1907824217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe664154220 0x7fe66415aa80 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fe654009870 tx=0x7fe65401e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff2b80d905be4aa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe6627fc640 1 -- 172.21.15.26:0/1907824217 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6540b2600 con 0x7fe664154220 2023-12-19T18:17:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe6627fc640 1 -- 172.21.15.26:0/1907824217 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6540b2760 con 0x7fe664154220 2023-12-19T18:17:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.137+0000 7fe6627fc640 1 -- 172.21.15.26:0/1907824217 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6540b2af0 con 0x7fe664154220 2023-12-19T18:17:14.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 -- 172.21.15.26:0/1907824217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe664154220 msgr2=0x7fe66415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 --2- 172.21.15.26:0/1907824217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe664154220 0x7fe66415aa80 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fe654009870 tx=0x7fe65401e6e0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 -- 172.21.15.26:0/1907824217 shutdown_connections 2023-12-19T18:17:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 --2- 172.21.15.26:0/1907824217 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe66406b3d0 0x7fe6640ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 --2- 172.21.15.26:0/1907824217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe664154220 0x7fe66415aa80 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 -- 172.21.15.26:0/1907824217 >> 172.21.15.26:0/1907824217 conn(0x7fe6640658e0 msgr2=0x7fe664065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 -- 172.21.15.26:0/1907824217 shutdown_connections 2023-12-19T18:17:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 -- 172.21.15.26:0/1907824217 wait complete. 2023-12-19T18:17:14.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 Processor -- start 2023-12-19T18:17:14.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 -- start start 2023-12-19T18:17:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66406b3d0 0x7fe66414b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6640cec40 0x7fe6640b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe664154220 0x7fe66415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6641f1ed0 con 0x7fe66406b3d0 2023-12-19T18:17:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66414b7e0 con 0x7fe6640cec40 2023-12-19T18:17:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe669c95640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6641f1250 con 0x7fe664154220 2023-12-19T18:17:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe6637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66406b3d0 0x7fe66414b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe6637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66406b3d0 0x7fe66414b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49094/0 (socket says 172.21.15.26:49094) 2023-12-19T18:17:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe6637fe640 1 -- 172.21.15.26:0/4007448119 learned_addr learned my addr 172.21.15.26:0/4007448119 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe662ffd640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6640cec40 0x7fe6640b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe6637fe640 1 -- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe664154220 msgr2=0x7fe66415c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.141+0000 7fe663fff640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe664154220 0x7fe66415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe6637fe640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe664154220 0x7fe66415c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe6637fe640 1 -- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6640cec40 msgr2=0x7fe6640b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe6637fe640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6640cec40 0x7fe6640b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe6637fe640 1 -- 172.21.15.26:0/4007448119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6540094d0 con 0x7fe66406b3d0 2023-12-19T18:17:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe662ffd640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6640cec40 0x7fe6640b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe663fff640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe664154220 0x7fe66415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe6637fe640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66406b3d0 0x7fe66414b2a0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fe65401cd90 tx=0x7fe65401b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe64bfff640 1 -- 172.21.15.26:0/4007448119 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6540aa800 con 0x7fe66406b3d0 2023-12-19T18:17:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe64bfff640 1 -- 172.21.15.26:0/4007448119 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6540aa960 con 0x7fe66406b3d0 2023-12-19T18:17:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe669c95640 1 -- 172.21.15.26:0/4007448119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6641f1530 con 0x7fe66406b3d0 2023-12-19T18:17:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe64bfff640 1 -- 172.21.15.26:0/4007448119 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6540aad10 con 0x7fe66406b3d0 2023-12-19T18:17:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe669c95640 1 -- 172.21.15.26:0/4007448119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6641f1af0 con 0x7fe66406b3d0 2023-12-19T18:17:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe669c95640 1 -- 172.21.15.26:0/4007448119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6641f2010 con 0x7fe66406b3d0 2023-12-19T18:17:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe64bfff640 1 -- 172.21.15.26:0/4007448119 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fe6540c6450 con 0x7fe66406b3d0 2023-12-19T18:17:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe64bfff640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe650065ca0 0x7fe650068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.145+0000 7fe64bfff640 1 -- 172.21.15.26:0/4007448119 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7298+0+0 (secure 0 0 0) 0x7fe6541208f0 con 0x7fe66406b3d0 2023-12-19T18:17:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.149+0000 7fe662ffd640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe650065ca0 0x7fe650068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.149+0000 7fe662ffd640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe650065ca0 0x7fe650068160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe6640cbba0 tx=0x7fe658008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.153+0000 7fe64bfff640 1 -- 172.21.15.26:0/4007448119 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe6641f2010 con 0x7fe66406b3d0 2023-12-19T18:17:14.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.309+0000 7fe669c95640 1 -- 172.21.15.26:0/4007448119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6641f2010 con 0x7fe66406b3d0 2023-12-19T18:17:14.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.309+0000 7fe64bfff640 1 -- 172.21.15.26:0/4007448119 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v221) v1 ==== 74+0+18472 (secure 0 0 0) 0x7fe6641f2010 con 0x7fe66406b3d0 2023-12-19T18:17:14.314 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:14.314 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":221,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:12.136783+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:16:38.226152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:16:39.540217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1665633474":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3561084215":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1351823889":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3136172808":"2023-12-20T18:17:04.905418+0000","172.21.15.26:6835/3981286992":"2023-12-20T18:17:08.808978+0000","172.21.15.26:6834/3981286992":"2023-12-20T18:17:08.808978+0000","172.21.15.160:0/3161716183":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/2592629251":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1089029505":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1544351773":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1134319862":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/4291270549":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3401801452":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/765362777":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/1696607504":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6833/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6832/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/2355423136":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/3498637577":"2023-12-20T18:16:55.864876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.313+0000 7fe669c95640 1 -- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe650065ca0 msgr2=0x7fe650068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.313+0000 7fe669c95640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe650065ca0 0x7fe650068160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe6640cbba0 tx=0x7fe658008040 comp rx=0 tx=0).stop 2023-12-19T18:17:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.313+0000 7fe669c95640 1 -- 172.21.15.26:0/4007448119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66406b3d0 msgr2=0x7fe66414b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.313+0000 7fe669c95640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66406b3d0 0x7fe66414b2a0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fe65401cd90 tx=0x7fe65401b480 comp rx=0 tx=0).stop 2023-12-19T18:17:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.313+0000 7fe669c95640 1 -- 172.21.15.26:0/4007448119 shutdown_connections 2023-12-19T18:17:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.313+0000 7fe669c95640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe650065ca0 0x7fe650068160 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.313+0000 7fe669c95640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe664154220 0x7fe66415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.313+0000 7fe669c95640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6640cec40 0x7fe6640b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.313+0000 7fe669c95640 1 --2- 172.21.15.26:0/4007448119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe66406b3d0 0x7fe66414b2a0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.313+0000 7fe669c95640 1 -- 172.21.15.26:0/4007448119 >> 172.21.15.26:0/4007448119 conn(0x7fe6640658e0 msgr2=0x7fe664067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.313+0000 7fe669c95640 1 -- 172.21.15.26:0/4007448119 shutdown_connections 2023-12-19T18:17:14.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.313+0000 7fe669c95640 1 -- 172.21.15.26:0/4007448119 wait complete. 2023-12-19T18:17:14.336 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-12-19T18:17:14.336 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:14.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f093db640 1 Processor -- start 2023-12-19T18:17:14.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f093db640 1 -- start start 2023-12-19T18:17:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f093db640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3f040cda40 0x7f3f0406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f093db640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0406b510 con 0x7f3f040b25d0 2023-12-19T18:17:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f093db640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0406b790 con 0x7f3f040d1290 2023-12-19T18:17:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f093db640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0406abb0 con 0x7f3f040cda40 2023-12-19T18:17:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3efbfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3f040b25d0 0x7f3f040cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58254/0 (socket says 172.21.15.26:58254) 2023-12-19T18:17:14.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3efbfff640 1 -- 172.21.15.26:0/2289842751 learned_addr learned my addr 172.21.15.26:0/2289842751 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f037fe640 1 --2- 172.21.15.26:0/2289842751 >> v2:172.21.15.160:3300/0 conn(0x7f3f040cda40 0x7f3f0406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720741794 0 0) 0x7f3f0406b510 con 0x7f3f040b25d0 2023-12-19T18:17:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef0003530 con 0x7f3f040b25d0 2023-12-19T18:17:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3979018444 0 0) 0x7f3ef0003530 con 0x7f3f040b25d0 2023-12-19T18:17:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 >> v2:172.21.15.160:3300/0 conn(0x7f3f040cda40 msgr2=0x7f3f0406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:14.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 --2- 172.21.15.26:0/2289842751 >> v2:172.21.15.160:3300/0 conn(0x7f3f040cda40 0x7f3f0406afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 >> v1:172.21.15.160:6789/0 conn(0x7f3f040d1290 legacy=0x7f3f040b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0406bce0 con 0x7f3f040b25d0 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f3eec0016a0 con 0x7f3f040b25d0 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 >> v1:172.21.15.26:6789/0 conn(0x7f3f040b25d0 legacy=0x7f3f040cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 --2- 172.21.15.26:0/2289842751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ef0003980 0x7f3ef0003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 --2- 172.21.15.26:0/2289842751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ef0004380 0x7f3ef0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 --2- 172.21.15.26:0/2289842751 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ef00050f0 0x7f3ef000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0406abb0 con 0x7f3ef0004380 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef000c030 con 0x7f3ef0003980 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef000c1f0 con 0x7f3ef00050f0 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3efbfff640 1 --2- 172.21.15.26:0/2289842751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ef0004380 0x7f3ef0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f02ffd640 1 --2- 172.21.15.26:0/2289842751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ef0003980 0x7f3ef0003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f037fe640 1 --2- 172.21.15.26:0/2289842751 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ef00050f0 0x7f3ef000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3efbfff640 1 -- 172.21.15.26:0/2289842751 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ef00050f0 msgr2=0x7f3ef000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3efbfff640 1 --2- 172.21.15.26:0/2289842751 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3ef00050f0 0x7f3ef000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3efbfff640 1 -- 172.21.15.26:0/2289842751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ef0003980 msgr2=0x7f3ef0003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3efbfff640 1 --2- 172.21.15.26:0/2289842751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ef0003980 0x7f3ef0003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3efbfff640 1 -- 172.21.15.26:0/2289842751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef0003530 con 0x7f3ef0004380 2023-12-19T18:17:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f02ffd640 1 --2- 172.21.15.26:0/2289842751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ef0003980 0x7f3ef0003d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:14.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.477+0000 7f3f037fe640 1 -- 172.21.15.26:0/2289842751 reap_dead start 2023-12-19T18:17:14.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.481+0000 7f3efbfff640 1 --2- 172.21.15.26:0/2289842751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ef0004380 0x7f3ef0004850 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f3f0406b6a0 tx=0x7f3eec01cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882052246362c33a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.481+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec005ae0 con 0x7f3ef0004380 2023-12-19T18:17:14.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.481+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec005c60 con 0x7f3ef0004380 2023-12-19T18:17:14.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.481+0000 7f3f027fc640 1 -- 172.21.15.26:0/2289842751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0a5780 con 0x7f3ef0004380 2023-12-19T18:17:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.481+0000 7f3f093db640 1 -- 172.21.15.26:0/2289842751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ef0004380 msgr2=0x7f3ef0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.481+0000 7f3f093db640 1 --2- 172.21.15.26:0/2289842751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ef0004380 0x7f3ef0004850 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f3f0406b6a0 tx=0x7f3eec01cb00 comp rx=0 tx=0).stop 2023-12-19T18:17:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.481+0000 7f3f093db640 1 -- 172.21.15.26:0/2289842751 shutdown_connections 2023-12-19T18:17:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.481+0000 7f3f093db640 1 --2- 172.21.15.26:0/2289842751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3ef0004380 0x7f3ef0004850 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.481+0000 7f3f093db640 1 -- 172.21.15.26:0/2289842751 >> 172.21.15.26:0/2289842751 conn(0x7f3f0406d570 msgr2=0x7f3f0406d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.481+0000 7f3f093db640 1 -- 172.21.15.26:0/2289842751 shutdown_connections 2023-12-19T18:17:14.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.481+0000 7f3f093db640 1 -- 172.21.15.26:0/2289842751 wait complete. 2023-12-19T18:17:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f093db640 1 Processor -- start 2023-12-19T18:17:14.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f093db640 1 -- start start 2023-12-19T18:17:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f093db640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ef0004380 0x7f3f040b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f093db640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f040b5050 0x7f3f04150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f093db640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f040b1630 0x7f3f040b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f093db640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f040c4420 con 0x7f3f040b1630 2023-12-19T18:17:14.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f093db640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f040b1ff0 con 0x7f3ef0004380 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f093db640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f040b5930 con 0x7f3f040b5050 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f040b1630 0x7f3f040b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f037fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f040b1630 0x7f3f040b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49106/0 (socket says 172.21.15.26:49106) 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f037fe640 1 -- 172.21.15.26:0/668375117 learned_addr learned my addr 172.21.15.26:0/668375117 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3efbfff640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f040b5050 0x7f3f04150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f02ffd640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ef0004380 0x7f3f040b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f037fe640 1 -- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f040b5050 msgr2=0x7f3f04150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f037fe640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f040b5050 0x7f3f04150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f037fe640 1 -- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ef0004380 msgr2=0x7f3f040b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f037fe640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ef0004380 0x7f3f040b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f037fe640 1 -- 172.21.15.26:0/668375117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f040b5c10 con 0x7f3f040b1630 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f02ffd640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ef0004380 0x7f3f040b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3efbfff640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f040b5050 0x7f3f04150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f037fe640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f040b1630 0x7f3f040b1ab0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f3f0406ce30 tx=0x7f3efc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3efb7fe640 1 -- 172.21.15.26:0/668375117 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc00e070 con 0x7f3f040b1630 2023-12-19T18:17:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3efb7fe640 1 -- 172.21.15.26:0/668375117 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc011e10 con 0x7f3f040b1630 2023-12-19T18:17:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f093db640 1 -- 172.21.15.26:0/668375117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f040b5ef0 con 0x7f3f040b1630 2023-12-19T18:17:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3efb7fe640 1 -- 172.21.15.26:0/668375117 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc0039a0 con 0x7f3f040b1630 2023-12-19T18:17:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.485+0000 7f3f093db640 1 -- 172.21.15.26:0/668375117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f041510d0 con 0x7f3f040b1630 2023-12-19T18:17:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.489+0000 7f3f093db640 1 -- 172.21.15.26:0/668375117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f040b63a0 con 0x7f3f040b1630 2023-12-19T18:17:14.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.489+0000 7f3efb7fe640 1 -- 172.21.15.26:0/668375117 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f3efc003b00 con 0x7f3f040b1630 2023-12-19T18:17:14.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.489+0000 7f3efb7fe640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3ee4065c70 0x7f3ee4068130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.489+0000 7f3efb7fe640 1 -- 172.21.15.26:0/668375117 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f3efc084520 con 0x7f3f040b1630 2023-12-19T18:17:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.489+0000 7f3f02ffd640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3ee4065c70 0x7f3ee4068130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.489+0000 7f3f02ffd640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3ee4065c70 0x7f3ee4068130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4007fd0 tx=0x7f3ef4009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.493+0000 7f3efb7fe640 1 -- 172.21.15.26:0/668375117 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3efc051390 con 0x7f3f040b1630 2023-12-19T18:17:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.681+0000 7f3f093db640 1 -- 172.21.15.26:0/668375117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f04074270 con 0x7f3f040b1630 2023-12-19T18:17:14.686 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.681+0000 7f3efb7fe640 1 -- 172.21.15.26:0/668375117 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4731 (secure 0 0 0) 0x7f3efc0549b0 con 0x7f3f040b1630 2023-12-19T18:17:14.687 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:14.687 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":104,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3349262044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3349262044},{"type":"v1","addr":"172.21.15.160:6836","nonce":3349262044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3865658274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3865658274},{"type":"v1","addr":"172.21.15.26:6833","nonce":3865658274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":11262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1792196808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1792196808},{"type":"v1","addr":"172.21.15.26:6835","nonce":1792196808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:40.537606+0000","modified":"2023-12-19T18:16:43.550176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10605":{"gid":10605,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.160:6837/1294752100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1294752100},{"type":"v1","addr":"172.21.15.160:6837","nonce":1294752100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-19T18:17:14.687 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 104 2023-12-19T18:17:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.685+0000 7f3f093db640 1 -- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3ee4065c70 msgr2=0x7f3ee4068130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.685+0000 7f3f093db640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3ee4065c70 0x7f3ee4068130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4007fd0 tx=0x7f3ef4009040 comp rx=0 tx=0).stop 2023-12-19T18:17:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.685+0000 7f3f093db640 1 -- 172.21.15.26:0/668375117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f040b1630 msgr2=0x7f3f040b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.685+0000 7f3f093db640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f040b1630 0x7f3f040b1ab0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f3f0406ce30 tx=0x7f3efc0115a0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.685+0000 7f3f093db640 1 -- 172.21.15.26:0/668375117 shutdown_connections 2023-12-19T18:17:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.685+0000 7f3f093db640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3ee4065c70 0x7f3ee4068130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.685+0000 7f3f093db640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f040b1630 0x7f3f040b1ab0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.685+0000 7f3f093db640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f040b5050 0x7f3f04150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.685+0000 7f3f093db640 1 --2- 172.21.15.26:0/668375117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ef0004380 0x7f3f040b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.685+0000 7f3f093db640 1 -- 172.21.15.26:0/668375117 >> 172.21.15.26:0/668375117 conn(0x7f3f0406d570 msgr2=0x7f3f0415f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.685+0000 7f3f093db640 1 -- 172.21.15.26:0/668375117 shutdown_connections 2023-12-19T18:17:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.685+0000 7f3f093db640 1 -- 172.21.15.26:0/668375117 wait complete. 2023-12-19T18:17:14.708 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.845+0000 7f686ac8c640 1 Processor -- start 2023-12-19T18:17:14.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f686ac8c640 1 -- start start 2023-12-19T18:17:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f686ac8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f686406f9d0 0x7f68640ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f686ac8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f68640c4070 0x7f68640cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f686ac8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68640c5b40 con 0x7f686415a7c0 2023-12-19T18:17:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f686ac8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68640c5e20 con 0x7f68640c4070 2023-12-19T18:17:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f686ac8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68640ce770 con 0x7f686406f9d0 2023-12-19T18:17:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f6869229640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f68640c4070 0x7f68640cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f6869229640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f68640c4070 0x7f68640cf530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49116/0 (socket says 172.21.15.26:49116) 2023-12-19T18:17:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f6868a28640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f686415a7c0 0x7f686415ddb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58260/0 (socket says 172.21.15.26:58260) 2023-12-19T18:17:14.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f6869229640 1 -- 172.21.15.26:0/2898620263 learned_addr learned my addr 172.21.15.26:0/2898620263 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f6863fff640 1 --2- 172.21.15.26:0/2898620263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f686406f9d0 0x7f68640ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f6869229640 1 -- 172.21.15.26:0/2898620263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f686406f9d0 msgr2=0x7f68640ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f6869229640 1 --2- 172.21.15.26:0/2898620263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f686406f9d0 0x7f68640ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f6869229640 1 -- 172.21.15.26:0/2898620263 >> v1:172.21.15.26:6789/0 conn(0x7f686415a7c0 legacy=0x7f686415ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f6869229640 1 -- 172.21.15.26:0/2898620263 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68640cea50 con 0x7f68640c4070 2023-12-19T18:17:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f6863fff640 1 --2- 172.21.15.26:0/2898620263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f686406f9d0 0x7f68640ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:14.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.849+0000 7f68637fe640 1 -- 172.21.15.26:0/2898620263 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524042720 0 0) 0x7f68640c5b40 con 0x7f686415a7c0 2023-12-19T18:17:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f6869229640 1 --2- 172.21.15.26:0/2898620263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68640c4070 0x7f68640cf530 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f685c009e50 tx=0x7f685c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72691b9bdb848cb2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f68637fe640 1 -- 172.21.15.26:0/2898620263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c0a5830 con 0x7f68640c4070 2023-12-19T18:17:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f68637fe640 1 -- 172.21.15.26:0/2898620263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685c0a5990 con 0x7f68640c4070 2023-12-19T18:17:14.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f68637fe640 1 -- 172.21.15.26:0/2898620263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c0a5d20 con 0x7f68640c4070 2023-12-19T18:17:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f686ac8c640 1 -- 172.21.15.26:0/2898620263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68640c4070 msgr2=0x7f68640cf530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f686ac8c640 1 --2- 172.21.15.26:0/2898620263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68640c4070 0x7f68640cf530 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f685c009e50 tx=0x7f685c01c950 comp rx=0 tx=0).stop 2023-12-19T18:17:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f686ac8c640 1 -- 172.21.15.26:0/2898620263 shutdown_connections 2023-12-19T18:17:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f686ac8c640 1 --2- 172.21.15.26:0/2898620263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68640c4070 0x7f68640cf530 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f686ac8c640 1 --2- 172.21.15.26:0/2898620263 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f686406f9d0 0x7f68640ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f686ac8c640 1 -- 172.21.15.26:0/2898620263 >> 172.21.15.26:0/2898620263 conn(0x7f686406d5d0 msgr2=0x7f6864069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f686ac8c640 1 -- 172.21.15.26:0/2898620263 shutdown_connections 2023-12-19T18:17:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f686ac8c640 1 -- 172.21.15.26:0/2898620263 wait complete. 2023-12-19T18:17:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f686ac8c640 1 Processor -- start 2023-12-19T18:17:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.853+0000 7f686ac8c640 1 -- start start 2023-12-19T18:17:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f686ac8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f686406f9d0 0x7f68640cf250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f686ac8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68640c4070 0x7f6864156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f686ac8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f686415a7c0 0x7f6864151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f686ac8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68640b4920 con 0x7f68640c4070 2023-12-19T18:17:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f686ac8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68640b3e00 con 0x7f686415a7c0 2023-12-19T18:17:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f686ac8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68640b4080 con 0x7f686406f9d0 2023-12-19T18:17:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f6863fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68640c4070 0x7f6864156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f6863fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68640c4070 0x7f6864156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49122/0 (socket says 172.21.15.26:49122) 2023-12-19T18:17:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f6863fff640 1 -- 172.21.15.26:0/3030286524 learned_addr learned my addr 172.21.15.26:0/3030286524 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f6868a28640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f686406f9d0 0x7f68640cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f6869229640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f686415a7c0 0x7f6864151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f6863fff640 1 -- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f686406f9d0 msgr2=0x7f68640cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f6863fff640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f686406f9d0 0x7f68640cf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f6863fff640 1 -- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f686415a7c0 msgr2=0x7f6864151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f6863fff640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f686415a7c0 0x7f6864151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f6863fff640 1 -- 172.21.15.26:0/3030286524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c009b30 con 0x7f68640c4070 2023-12-19T18:17:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f6868a28640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f686406f9d0 0x7f68640cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f6863fff640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68640c4070 0x7f6864156790 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f685000c820 tx=0x7f685000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f68617fa640 1 -- 172.21.15.26:0/3030286524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6850007a30 con 0x7f68640c4070 2023-12-19T18:17:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f68617fa640 1 -- 172.21.15.26:0/3030286524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6850007b90 con 0x7f68640c4070 2023-12-19T18:17:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f686ac8c640 1 -- 172.21.15.26:0/3030286524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68640b4360 con 0x7f68640c4070 2023-12-19T18:17:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f68617fa640 1 -- 172.21.15.26:0/3030286524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68500113f0 con 0x7f68640c4070 2023-12-19T18:17:14.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f686ac8c640 1 -- 172.21.15.26:0/3030286524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6864152230 con 0x7f68640c4070 2023-12-19T18:17:14.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.857+0000 7f686ac8c640 1 -- 172.21.15.26:0/3030286524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68640b1e00 con 0x7f68640c4070 2023-12-19T18:17:14.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.861+0000 7f68617fa640 1 -- 172.21.15.26:0/3030286524 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f6850011550 con 0x7f68640c4070 2023-12-19T18:17:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.861+0000 7f68617fa640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6848065cd0 0x7f6848068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.861+0000 7f68617fa640 1 -- 172.21.15.26:0/3030286524 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f6850083ee0 con 0x7f68640c4070 2023-12-19T18:17:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.861+0000 7f6868a28640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6848065cd0 0x7f6848068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.861+0000 7f6868a28640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6848065cd0 0x7f6848068190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6858001720 tx=0x7f6858009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:14.865+0000 7f68617fa640 1 -- 172.21.15.26:0/3030286524 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f68640b1e00 con 0x7f68640c4070 2023-12-19T18:17:15.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.053+0000 7f686ac8c640 1 -- 172.21.15.26:0/3030286524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68640b1e00 con 0x7f68640c4070 2023-12-19T18:17:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.053+0000 7f68617fa640 1 -- 172.21.15.26:0/3030286524 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4731 (secure 0 0 0) 0x7f68640b1e00 con 0x7f68640c4070 2023-12-19T18:17:15.058 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:15.058 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":104,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3349262044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3349262044},{"type":"v1","addr":"172.21.15.160:6836","nonce":3349262044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3865658274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3865658274},{"type":"v1","addr":"172.21.15.26:6833","nonce":3865658274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":11262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1792196808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1792196808},{"type":"v1","addr":"172.21.15.26:6835","nonce":1792196808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:40.537606+0000","modified":"2023-12-19T18:16:43.550176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10605},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10605":{"gid":10605,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.160:6837/1294752100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1294752100},{"type":"v1","addr":"172.21.15.160:6837","nonce":1294752100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-19T18:17:15.058 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 104 2023-12-19T18:17:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.057+0000 7f686ac8c640 1 -- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6848065cd0 msgr2=0x7f6848068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.057+0000 7f686ac8c640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6848065cd0 0x7f6848068190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6858001720 tx=0x7f6858009040 comp rx=0 tx=0).stop 2023-12-19T18:17:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.057+0000 7f686ac8c640 1 -- 172.21.15.26:0/3030286524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68640c4070 msgr2=0x7f6864156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.057+0000 7f686ac8c640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68640c4070 0x7f6864156790 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f685000c820 tx=0x7f685000cce0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.057+0000 7f686ac8c640 1 -- 172.21.15.26:0/3030286524 shutdown_connections 2023-12-19T18:17:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.057+0000 7f686ac8c640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6848065cd0 0x7f6848068190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.057+0000 7f686ac8c640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f686415a7c0 0x7f6864151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.057+0000 7f686ac8c640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68640c4070 0x7f6864156790 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.057+0000 7f686ac8c640 1 --2- 172.21.15.26:0/3030286524 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f686406f9d0 0x7f68640cf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.057+0000 7f686ac8c640 1 -- 172.21.15.26:0/3030286524 >> 172.21.15.26:0/3030286524 conn(0x7f686406d5d0 msgr2=0x7f686415ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:15.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.057+0000 7f686ac8c640 1 -- 172.21.15.26:0/3030286524 shutdown_connections 2023-12-19T18:17:15.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.057+0000 7f686ac8c640 1 -- 172.21.15.26:0/3030286524 wait complete. 2023-12-19T18:17:15.080 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:15.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f573c7f3640 1 Processor -- start 2023-12-19T18:17:15.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f573c7f3640 1 -- start start 2023-12-19T18:17:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f573c7f3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f57340658e0 0x7f573406e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f573c7f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f573415d260 0x7f5734065cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f573c7f3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57340b3060 con 0x7f57340703a0 2023-12-19T18:17:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f573c7f3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5734066200 con 0x7f57340658e0 2023-12-19T18:17:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f573c7f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573415d820 con 0x7f573415d260 2023-12-19T18:17:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f5739d8e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f573415d260 0x7f5734065cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.226 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f5739d8e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f573415d260 0x7f5734065cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:45132/0 (socket says 172.21.15.26:45132) 2023-12-19T18:17:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f5739d8e640 1 -- 172.21.15.26:0/2342527574 learned_addr learned my addr 172.21.15.26:0/2342527574 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f573a58f640 1 --2- 172.21.15.26:0/2342527574 >> v2:172.21.15.160:3300/0 conn(0x7f57340658e0 0x7f573406e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f5739d8e640 1 -- 172.21.15.26:0/2342527574 >> v2:172.21.15.160:3300/0 conn(0x7f57340658e0 msgr2=0x7f573406e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f5739d8e640 1 --2- 172.21.15.26:0/2342527574 >> v2:172.21.15.160:3300/0 conn(0x7f57340658e0 0x7f573406e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f5739d8e640 1 -- 172.21.15.26:0/2342527574 >> v1:172.21.15.160:6789/0 conn(0x7f57340703a0 legacy=0x7f5734070810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f5739d8e640 1 -- 172.21.15.26:0/2342527574 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573415db00 con 0x7f573415d260 2023-12-19T18:17:15.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.221+0000 7f573a58f640 1 --2- 172.21.15.26:0/2342527574 >> v2:172.21.15.160:3300/0 conn(0x7f57340658e0 0x7f573406e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:15.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.225+0000 7f5739d8e640 1 --2- 172.21.15.26:0/2342527574 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f573415d260 0x7f5734065cc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5724009870 tx=0x7f572401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0f441f6045fabfe server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.225+0000 7f573958d640 1 -- 172.21.15.26:0/2342527574 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57240aa670 con 0x7f573415d260 2023-12-19T18:17:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.225+0000 7f573958d640 1 -- 172.21.15.26:0/2342527574 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57240aa7d0 con 0x7f573415d260 2023-12-19T18:17:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.225+0000 7f573958d640 1 -- 172.21.15.26:0/2342527574 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57240aab60 con 0x7f573415d260 2023-12-19T18:17:15.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.225+0000 7f573c7f3640 1 -- 172.21.15.26:0/2342527574 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f573415d260 msgr2=0x7f5734065cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.225+0000 7f573c7f3640 1 --2- 172.21.15.26:0/2342527574 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f573415d260 0x7f5734065cc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5724009870 tx=0x7f572401c510 comp rx=0 tx=0).stop 2023-12-19T18:17:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.225+0000 7f573c7f3640 1 -- 172.21.15.26:0/2342527574 shutdown_connections 2023-12-19T18:17:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.225+0000 7f573c7f3640 1 --2- 172.21.15.26:0/2342527574 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f573415d260 0x7f5734065cc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.225+0000 7f573c7f3640 1 --2- 172.21.15.26:0/2342527574 >> v2:172.21.15.160:3300/0 conn(0x7f57340658e0 0x7f573406e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.225+0000 7f573c7f3640 1 -- 172.21.15.26:0/2342527574 >> 172.21.15.26:0/2342527574 conn(0x7f573406a320 msgr2=0x7f57340b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.225+0000 7f573c7f3640 1 -- 172.21.15.26:0/2342527574 shutdown_connections 2023-12-19T18:17:15.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.225+0000 7f573c7f3640 1 -- 172.21.15.26:0/2342527574 wait complete. 2023-12-19T18:17:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573c7f3640 1 Processor -- start 2023-12-19T18:17:15.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573c7f3640 1 -- start start 2023-12-19T18:17:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573c7f3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57340658e0 0x7f573413a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573c7f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57340703a0 0x7f5734139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573c7f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f573415d260 0x7f573413cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573c7f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57341f5620 con 0x7f573415d260 2023-12-19T18:17:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573c7f3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57341f90f0 con 0x7f57340658e0 2023-12-19T18:17:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573c7f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57341f93d0 con 0x7f57340703a0 2023-12-19T18:17:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573ad90640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f573415d260 0x7f573413cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573ad90640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f573415d260 0x7f573413cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49136/0 (socket says 172.21.15.26:49136) 2023-12-19T18:17:15.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573ad90640 1 -- 172.21.15.26:0/602580171 learned_addr learned my addr 172.21.15.26:0/602580171 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:15.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573a58f640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57340658e0 0x7f573413a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f5739d8e640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57340703a0 0x7f5734139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573ad90640 1 -- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57340703a0 msgr2=0x7f5734139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573ad90640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57340703a0 0x7f5734139940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573ad90640 1 -- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57340658e0 msgr2=0x7f573413a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573ad90640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57340658e0 0x7f573413a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573ad90640 1 -- 172.21.15.26:0/602580171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5730009420 con 0x7f573415d260 2023-12-19T18:17:15.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573ad90640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f573415d260 0x7f573413cde0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f572c00c770 tx=0x7f572c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:15.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f572affd640 1 -- 172.21.15.26:0/602580171 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c003b80 con 0x7f573415d260 2023-12-19T18:17:15.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f572affd640 1 -- 172.21.15.26:0/602580171 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c003ce0 con 0x7f573415d260 2023-12-19T18:17:15.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f572affd640 1 -- 172.21.15.26:0/602580171 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00aa90 con 0x7f573415d260 2023-12-19T18:17:15.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.229+0000 7f573c7f3640 1 -- 172.21.15.26:0/602580171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57240094d0 con 0x7f573415d260 2023-12-19T18:17:15.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.233+0000 7f573c7f3640 1 -- 172.21.15.26:0/602580171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57341f9990 con 0x7f573415d260 2023-12-19T18:17:15.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.233+0000 7f573c7f3640 1 -- 172.21.15.26:0/602580171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57341f57c0 con 0x7f573415d260 2023-12-19T18:17:15.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.233+0000 7f572affd640 1 -- 172.21.15.26:0/602580171 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f572c00abf0 con 0x7f573415d260 2023-12-19T18:17:15.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.233+0000 7f572affd640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5718065cd0 0x7f5718068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.233+0000 7f572affd640 1 -- 172.21.15.26:0/602580171 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f572c083950 con 0x7f573415d260 2023-12-19T18:17:15.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.233+0000 7f573a58f640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5718065cd0 0x7f5718068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.233+0000 7f573a58f640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5718065cd0 0x7f5718068190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5730009c50 tx=0x7f5730009190 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:15.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.237+0000 7f572affd640 1 -- 172.21.15.26:0/602580171 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f572c050980 con 0x7f573415d260 2023-12-19T18:17:15.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.397+0000 7f573c7f3640 1 -- 172.21.15.26:0/602580171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5734074270 con 0x7f573415d260 2023-12-19T18:17:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.397+0000 7f572affd640 1 -- 172.21.15.26:0/602580171 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v221) v1 ==== 74+0+18472 (secure 0 0 0) 0x7f572c053fa0 con 0x7f573415d260 2023-12-19T18:17:15.403 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:15.403 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":221,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:12.136783+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:16:38.226152+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"215","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:16:39.540217+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"216","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1665633474":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3561084215":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1351823889":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3136172808":"2023-12-20T18:17:04.905418+0000","172.21.15.26:6835/3981286992":"2023-12-20T18:17:08.808978+0000","172.21.15.26:6834/3981286992":"2023-12-20T18:17:08.808978+0000","172.21.15.160:0/3161716183":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/2592629251":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1089029505":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1544351773":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/1134319862":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/4291270549":"2023-12-20T18:17:04.905418+0000","172.21.15.160:0/3401801452":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/765362777":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/1696607504":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6833/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:6832/48394":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/2355423136":"2023-12-20T18:16:55.864876+0000","172.21.15.160:0/3498637577":"2023-12-20T18:16:55.864876+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.401+0000 7f573c7f3640 1 -- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5718065cd0 msgr2=0x7f5718068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.401+0000 7f573c7f3640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5718065cd0 0x7f5718068190 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5730009c50 tx=0x7f5730009190 comp rx=0 tx=0).stop 2023-12-19T18:17:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.401+0000 7f573c7f3640 1 -- 172.21.15.26:0/602580171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f573415d260 msgr2=0x7f573413cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.401+0000 7f573c7f3640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f573415d260 0x7f573413cde0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f572c00c770 tx=0x7f572c00cc30 comp rx=0 tx=0).stop 2023-12-19T18:17:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.401+0000 7f573c7f3640 1 -- 172.21.15.26:0/602580171 shutdown_connections 2023-12-19T18:17:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.401+0000 7f573c7f3640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5718065cd0 0x7f5718068190 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.401+0000 7f573c7f3640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f573415d260 0x7f573413cde0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.401+0000 7f573c7f3640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f57340703a0 0x7f5734139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.401+0000 7f573c7f3640 1 --2- 172.21.15.26:0/602580171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57340658e0 0x7f573413a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.401+0000 7f573c7f3640 1 -- 172.21.15.26:0/602580171 >> 172.21.15.26:0/602580171 conn(0x7f573406a320 msgr2=0x7f57340adea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:15.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.401+0000 7f573c7f3640 1 -- 172.21.15.26:0/602580171 shutdown_connections 2023-12-19T18:17:15.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.401+0000 7f573c7f3640 1 -- 172.21.15.26:0/602580171 wait complete. 2023-12-19T18:17:15.424 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-12-19T18:17:15.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fb2f8f640 1 Processor -- start 2023-12-19T18:17:15.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fb2f8f640 1 -- start start 2023-12-19T18:17:15.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fb2f8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fac15cd40 0x7f2fac0ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fb2f8f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2fac0d3300 0x7f2fac160d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fb2f8f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fac0d29d0 con 0x7f2fac160900 2023-12-19T18:17:15.570 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fb2f8f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fac161280 con 0x7f2fac0d3300 2023-12-19T18:17:15.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fb2f8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac15d680 con 0x7f2fac15cd40 2023-12-19T18:17:15.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fabfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2fac160900 0x7f2fac0d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58272/0 (socket says 172.21.15.26:58272) 2023-12-19T18:17:15.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fabfff640 1 -- 172.21.15.26:0/311233292 learned_addr learned my addr 172.21.15.26:0/311233292 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:15.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fb0d2b640 1 --2- 172.21.15.26:0/311233292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fac15cd40 0x7f2fac0ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fb152c640 1 --2- 172.21.15.26:0/311233292 >> v2:172.21.15.160:3300/0 conn(0x7f2fac0d3300 0x7f2fac160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fab7fe640 1 -- 172.21.15.26:0/311233292 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708864116 0 0) 0x7f2f98001120 con 0x7f2fac160900 2023-12-19T18:17:15.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fab7fe640 1 -- 172.21.15.26:0/311233292 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f9c003530 con 0x7f2fac160900 2023-12-19T18:17:15.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fb0d2b640 1 -- 172.21.15.26:0/311233292 >> v2:172.21.15.160:3300/0 conn(0x7f2fac0d3300 msgr2=0x7f2fac160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:15.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fb0d2b640 1 --2- 172.21.15.26:0/311233292 >> v2:172.21.15.160:3300/0 conn(0x7f2fac0d3300 0x7f2fac160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fb0d2b640 1 -- 172.21.15.26:0/311233292 >> v1:172.21.15.26:6789/0 conn(0x7f2fac160900 legacy=0x7f2fac0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.565+0000 7f2fb0d2b640 1 -- 172.21.15.26:0/311233292 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac161750 con 0x7f2fac15cd40 2023-12-19T18:17:15.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.569+0000 7f2fb0d2b640 1 --2- 172.21.15.26:0/311233292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fac15cd40 0x7f2fac0ce040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00097a0 tx=0x7f2fa001c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b187fd01e7e6a48 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:15.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.569+0000 7f2fab7fe640 1 -- 172.21.15.26:0/311233292 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0004230 con 0x7f2fac15cd40 2023-12-19T18:17:15.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.569+0000 7f2fab7fe640 1 -- 172.21.15.26:0/311233292 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa00a7050 con 0x7f2fac15cd40 2023-12-19T18:17:15.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.569+0000 7f2fab7fe640 1 -- 172.21.15.26:0/311233292 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0003620 con 0x7f2fac15cd40 2023-12-19T18:17:15.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.569+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/311233292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fac15cd40 msgr2=0x7f2fac0ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.569+0000 7f2fb2f8f640 1 --2- 172.21.15.26:0/311233292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fac15cd40 0x7f2fac0ce040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00097a0 tx=0x7f2fa001c460 comp rx=0 tx=0).stop 2023-12-19T18:17:15.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.569+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/311233292 shutdown_connections 2023-12-19T18:17:15.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.569+0000 7f2fb2f8f640 1 --2- 172.21.15.26:0/311233292 >> v2:172.21.15.160:3300/0 conn(0x7f2fac0d3300 0x7f2fac160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.569+0000 7f2fb2f8f640 1 --2- 172.21.15.26:0/311233292 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fac15cd40 0x7f2fac0ce040 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.569+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/311233292 >> 172.21.15.26:0/311233292 conn(0x7f2fac06a120 msgr2=0x7f2fac06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:15.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.569+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/311233292 shutdown_connections 2023-12-19T18:17:15.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.569+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/311233292 wait complete. 2023-12-19T18:17:15.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb2f8f640 1 Processor -- start 2023-12-19T18:17:15.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb2f8f640 1 -- start start 2023-12-19T18:17:15.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb2f8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fac0d3300 0x7f2fac15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb2f8f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fac15cd40 0x7f2fac152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb2f8f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2fac160900 0x7f2fac155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb2f8f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac15c210 con 0x7f2fac0d3300 2023-12-19T18:17:15.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb2f8f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac1531f0 con 0x7f2fac160900 2023-12-19T18:17:15.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb2f8f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac15b590 con 0x7f2fac15cd40 2023-12-19T18:17:15.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb0d2b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fac0d3300 0x7f2fac15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb0d2b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fac0d3300 0x7f2fac15afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49138/0 (socket says 172.21.15.26:49138) 2023-12-19T18:17:15.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb0d2b640 1 -- 172.21.15.26:0/217871336 learned_addr learned my addr 172.21.15.26:0/217871336 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:15.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb152c640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2fac160900 0x7f2fac155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fabfff640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fac15cd40 0x7f2fac152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb0d2b640 1 -- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fac15cd40 msgr2=0x7f2fac152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb0d2b640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fac15cd40 0x7f2fac152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb0d2b640 1 -- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2fac160900 msgr2=0x7f2fac155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb0d2b640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2fac160900 0x7f2fac155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb0d2b640 1 -- 172.21.15.26:0/217871336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa00093b0 con 0x7f2fac0d3300 2023-12-19T18:17:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fabfff640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fac15cd40 0x7f2fac152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:15.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb0d2b640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fac0d3300 0x7f2fac15afb0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f2fa001a090 tx=0x7f2fa0019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:15.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fa97fa640 1 -- 172.21.15.26:0/217871336 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0005ba0 con 0x7f2fac0d3300 2023-12-19T18:17:15.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fa97fa640 1 -- 172.21.15.26:0/217871336 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa0005d00 con 0x7f2fac0d3300 2023-12-19T18:17:15.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fa97fa640 1 -- 172.21.15.26:0/217871336 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0003b10 con 0x7f2fac0d3300 2023-12-19T18:17:15.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/217871336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac15b870 con 0x7f2fac0d3300 2023-12-19T18:17:15.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.573+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/217871336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac15be30 con 0x7f2fac0d3300 2023-12-19T18:17:15.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.577+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/217871336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fac074270 con 0x7f2fac0d3300 2023-12-19T18:17:15.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.577+0000 7f2fa97fa640 1 -- 172.21.15.26:0/217871336 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f2fa00b2050 con 0x7f2fac0d3300 2023-12-19T18:17:15.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.577+0000 7f2fa97fa640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f90065ca0 0x7f2f90068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.577+0000 7f2fa97fa640 1 -- 172.21.15.26:0/217871336 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 7298+0+0 (secure 0 0 0) 0x7f2fa011d230 con 0x7f2fac0d3300 2023-12-19T18:17:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.577+0000 7f2fabfff640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f90065ca0 0x7f2f90068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.577+0000 7f2fabfff640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f90065ca0 0x7f2f90068160 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0d26d0 tx=0x7f2f98009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:15.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.581+0000 7f2fa97fa640 1 -- 172.21.15.26:0/217871336 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2fa00ea0e0 con 0x7f2fac0d3300 2023-12-19T18:17:15.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.773+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/217871336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7f2fac0d2420 con 0x7f2fac0d3300 2023-12-19T18:17:15.778 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T18:17:15.773+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:17:15.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.785+0000 7f2fa97fa640 1 -- 172.21.15.26:0/217871336 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v105) v1 ==== 167+0+0 (secure 0 0 0) 0x7f2fa00b0090 con 0x7f2fac0d3300 2023-12-19T18:17:15.788 INFO:teuthology.orchestra.run.smithi026.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:17:15.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.785+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f90065ca0 msgr2=0x7f2f90068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.785+0000 7f2fb2f8f640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f90065ca0 0x7f2f90068160 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2fac0d26d0 tx=0x7f2f98009040 comp rx=0 tx=0).stop 2023-12-19T18:17:15.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.785+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/217871336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fac0d3300 msgr2=0x7f2fac15afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.785+0000 7f2fb2f8f640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fac0d3300 0x7f2fac15afb0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f2fa001a090 tx=0x7f2fa0019200 comp rx=0 tx=0).stop 2023-12-19T18:17:15.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.785+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/217871336 shutdown_connections 2023-12-19T18:17:15.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.785+0000 7f2fb2f8f640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2f90065ca0 0x7f2f90068160 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.785+0000 7f2fb2f8f640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2fac160900 0x7f2fac155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.785+0000 7f2fb2f8f640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fac15cd40 0x7f2fac152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.785+0000 7f2fb2f8f640 1 --2- 172.21.15.26:0/217871336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fac0d3300 0x7f2fac15afb0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.785+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/217871336 >> 172.21.15.26:0/217871336 conn(0x7f2fac06a120 msgr2=0x7f2fac0c5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:15.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.785+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/217871336 shutdown_connections 2023-12-19T18:17:15.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.785+0000 7f2fb2f8f640 1 -- 172.21.15.26:0/217871336 wait complete. 2023-12-19T18:17:15.794 INFO:tasks.ceph.mds.d.smithi160.stderr: -1873> 2023-12-19T18:16:42.857+0000 7f2d85203640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:17:15.809 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-12-19T18:17:15.843 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T18:17:15.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f182996d640 1 Processor -- start 2023-12-19T18:17:15.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f182996d640 1 -- start start 2023-12-19T18:17:15.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f182996d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f182415cd10 0x7f18240cb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f182996d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f182406d3e0 0x7f182406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f182996d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182406c130 0x7f1824160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f182996d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182406c5c0 con 0x7f182406d3e0 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f182996d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182415d0f0 con 0x7f182415cd10 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f182996d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18240c98b0 con 0x7f182406c130 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f181bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f182406d3e0 0x7f182406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f181bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f182406d3e0 0x7f182406bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50900/0 (socket says 172.21.15.26:50900) 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f181bfff640 1 -- 172.21.15.26:0/3023036813 learned_addr learned my addr 172.21.15.26:0/3023036813 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f18237fe640 1 --2- 172.21.15.26:0/3023036813 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182406c130 0x7f1824160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f1822ffd640 1 --2- 172.21.15.26:0/3023036813 >> v2:172.21.15.160:3300/0 conn(0x7f182415cd10 0x7f18240cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f181bfff640 1 -- 172.21.15.26:0/3023036813 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182406c130 msgr2=0x7f1824160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f181bfff640 1 --2- 172.21.15.26:0/3023036813 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182406c130 0x7f1824160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f181bfff640 1 -- 172.21.15.26:0/3023036813 >> v2:172.21.15.160:3300/0 conn(0x7f182415cd10 msgr2=0x7f18240cb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f181bfff640 1 --2- 172.21.15.26:0/3023036813 >> v2:172.21.15.160:3300/0 conn(0x7f182415cd10 0x7f18240cb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.953+0000 7f181bfff640 1 -- 172.21.15.26:0/3023036813 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18240cd0e0 con 0x7f182406d3e0 2023-12-19T18:17:15.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.957+0000 7f181bfff640 1 --2- 172.21.15.26:0/3023036813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182406d3e0 0x7f182406bbf0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f18240cff10 tx=0x7f180c01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c86f4af4e38e825 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.957+0000 7f18227fc640 1 -- 172.21.15.26:0/3023036813 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c0a89b0 con 0x7f182406d3e0 2023-12-19T18:17:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.957+0000 7f18227fc640 1 -- 172.21.15.26:0/3023036813 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c0a8b10 con 0x7f182406d3e0 2023-12-19T18:17:15.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.957+0000 7f18227fc640 1 -- 172.21.15.26:0/3023036813 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c0a8ea0 con 0x7f182406d3e0 2023-12-19T18:17:15.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.957+0000 7f182996d640 1 -- 172.21.15.26:0/3023036813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182406d3e0 msgr2=0x7f182406bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.957+0000 7f182996d640 1 --2- 172.21.15.26:0/3023036813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182406d3e0 0x7f182406bbf0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f18240cff10 tx=0x7f180c01ecf0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.957+0000 7f182996d640 1 -- 172.21.15.26:0/3023036813 shutdown_connections 2023-12-19T18:17:15.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.957+0000 7f182996d640 1 --2- 172.21.15.26:0/3023036813 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182406c130 0x7f1824160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.957+0000 7f182996d640 1 --2- 172.21.15.26:0/3023036813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182406d3e0 0x7f182406bbf0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.957+0000 7f182996d640 1 --2- 172.21.15.26:0/3023036813 >> v2:172.21.15.160:3300/0 conn(0x7f182415cd10 0x7f18240cb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.957+0000 7f182996d640 1 -- 172.21.15.26:0/3023036813 >> 172.21.15.26:0/3023036813 conn(0x7f18240cf260 msgr2=0x7f18240cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:15.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.957+0000 7f182996d640 1 -- 172.21.15.26:0/3023036813 shutdown_connections 2023-12-19T18:17:15.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.957+0000 7f182996d640 1 -- 172.21.15.26:0/3023036813 wait complete. 2023-12-19T18:17:15.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f182996d640 1 Processor -- start 2023-12-19T18:17:15.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f182996d640 1 -- start start 2023-12-19T18:17:15.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f182996d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f182406c130 0x7f1824155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f182996d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182406d3e0 0x7f18240b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f182996d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182415cd10 0x7f18240c3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f182996d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1824155840 con 0x7f182415cd10 2023-12-19T18:17:15.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f182996d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18240c5790 con 0x7f182406c130 2023-12-19T18:17:15.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f182996d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18240c5a70 con 0x7f182406d3e0 2023-12-19T18:17:15.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f18237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182415cd10 0x7f18240c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f18237fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182415cd10 0x7f18240c3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50910/0 (socket says 172.21.15.26:50910) 2023-12-19T18:17:15.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f18237fe640 1 -- 172.21.15.26:0/2848056084 learned_addr learned my addr 172.21.15.26:0/2848056084 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:15.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f181bfff640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182406d3e0 0x7f18240b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f1822ffd640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f182406c130 0x7f1824155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f18237fe640 1 -- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182406d3e0 msgr2=0x7f18240b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f18237fe640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182406d3e0 0x7f18240b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f18237fe640 1 -- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f182406c130 msgr2=0x7f1824155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f18237fe640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f182406c130 0x7f1824155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f18237fe640 1 -- 172.21.15.26:0/2848056084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0094d0 con 0x7f182415cd10 2023-12-19T18:17:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f1822ffd640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f182406c130 0x7f1824155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:15.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f18237fe640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182415cd10 0x7f18240c3830 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f181c007e60 tx=0x7f181c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f1820ff9640 1 -- 172.21.15.26:0/2848056084 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c014620 con 0x7f182415cd10 2023-12-19T18:17:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f1820ff9640 1 -- 172.21.15.26:0/2848056084 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c00ee90 con 0x7f182415cd10 2023-12-19T18:17:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f182996d640 1 -- 172.21.15.26:0/2848056084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18240c2d50 con 0x7f182415cd10 2023-12-19T18:17:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f1820ff9640 1 -- 172.21.15.26:0/2848056084 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c0149e0 con 0x7f182415cd10 2023-12-19T18:17:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f182996d640 1 -- 172.21.15.26:0/2848056084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18240c3310 con 0x7f182415cd10 2023-12-19T18:17:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.961+0000 7f182996d640 1 -- 172.21.15.26:0/2848056084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18240c3500 con 0x7f182415cd10 2023-12-19T18:17:15.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.965+0000 7f1820ff9640 1 -- 172.21.15.26:0/2848056084 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f181c014b40 con 0x7f182415cd10 2023-12-19T18:17:15.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.965+0000 7f1820ff9640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1804065ca0 0x7f1804068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.965+0000 7f1820ff9640 1 -- 172.21.15.26:0/2848056084 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 7384+0+0 (secure 0 0 0) 0x7f181c0844a0 con 0x7f182415cd10 2023-12-19T18:17:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.965+0000 7f1822ffd640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1804065ca0 0x7f1804068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.965+0000 7f1822ffd640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1804065ca0 0x7f1804068160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f18140097a0 tx=0x7f1814008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:15.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:15.969+0000 7f1820ff9640 1 -- 172.21.15.26:0/2848056084 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f181c051530 con 0x7f182415cd10 2023-12-19T18:17:16.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.161+0000 7f182996d640 1 -- 172.21.15.26:0/2848056084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1824074270 con 0x7f182415cd10 2023-12-19T18:17:16.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.785+0000 7f1820ff9640 1 -- 172.21.15.26:0/2848056084 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v106) v1 ==== 107+0+0 (secure 0 0 0) 0x7f181c054b50 con 0x7f182415cd10 2023-12-19T18:17:16.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.789+0000 7f182996d640 1 -- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1804065ca0 msgr2=0x7f1804068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:16.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.789+0000 7f182996d640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1804065ca0 0x7f1804068160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f18140097a0 tx=0x7f1814008040 comp rx=0 tx=0).stop 2023-12-19T18:17:16.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.789+0000 7f182996d640 1 -- 172.21.15.26:0/2848056084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182415cd10 msgr2=0x7f18240c3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:16.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.789+0000 7f182996d640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182415cd10 0x7f18240c3830 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f181c007e60 tx=0x7f181c00e4b0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.789+0000 7f182996d640 1 -- 172.21.15.26:0/2848056084 shutdown_connections 2023-12-19T18:17:16.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.789+0000 7f182996d640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1804065ca0 0x7f1804068160 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.789+0000 7f182996d640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f182415cd10 0x7f18240c3830 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.789+0000 7f182996d640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f182406d3e0 0x7f18240b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.789+0000 7f182996d640 1 --2- 172.21.15.26:0/2848056084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f182406c130 0x7f1824155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.789+0000 7f182996d640 1 -- 172.21.15.26:0/2848056084 >> 172.21.15.26:0/2848056084 conn(0x7f18240cf260 msgr2=0x7f182406fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:16.794 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.789+0000 7f182996d640 1 -- 172.21.15.26:0/2848056084 shutdown_connections 2023-12-19T18:17:16.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.789+0000 7f182996d640 1 -- 172.21.15.26:0/2848056084 wait complete. 2023-12-19T18:17:16.817 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-12-19T18:17:16.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.965+0000 7faf7aab6640 1 Processor -- start 2023-12-19T18:17:16.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.965+0000 7faf7aab6640 1 -- start start 2023-12-19T18:17:16.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.965+0000 7faf7aab6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf740c9310 0x7faf740cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:16.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.965+0000 7faf7aab6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7415e3d0 con 0x7faf740cb110 2023-12-19T18:17:16.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.965+0000 7faf7aab6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7415de10 con 0x7faf74159830 2023-12-19T18:17:16.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.965+0000 7faf7aab6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf7406f7c0 con 0x7faf740c9310 2023-12-19T18:17:16.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.965+0000 7faf73fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf740c9310 0x7faf740cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:16.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.965+0000 7faf73fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf740c9310 0x7faf740cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50926/0 (socket says 172.21.15.26:50926) 2023-12-19T18:17:16.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.965+0000 7faf73fff640 1 -- 172.21.15.26:0/1499402813 learned_addr learned my addr 172.21.15.26:0/1499402813 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:16.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.965+0000 7faf79053640 1 --1- 172.21.15.26:0/1499402813 >> v1:172.21.15.26:6789/0 conn(0x7faf740cb110 0x7faf740701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44208/0 (socket says 172.21.15.26:44208) 2023-12-19T18:17:16.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf73fff640 1 -- 172.21.15.26:0/1499402813 >> v1:172.21.15.160:6789/0 conn(0x7faf74159830 legacy=0x7faf7415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:16.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf73fff640 1 -- 172.21.15.26:0/1499402813 >> v1:172.21.15.26:6789/0 conn(0x7faf740cb110 legacy=0x7faf740701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:16.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf73fff640 1 -- 172.21.15.26:0/1499402813 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7415e0f0 con 0x7faf740c9310 2023-12-19T18:17:16.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf737fe640 1 -- 172.21.15.26:0/1499402813 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627632184 0 0) 0x7faf7415e3d0 con 0x7faf740cb110 2023-12-19T18:17:16.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf73fff640 1 --2- 172.21.15.26:0/1499402813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf740c9310 0x7faf740cc430 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7faf60009870 tx=0x7faf6001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3c96c19eabfc78 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:16.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf737fe640 1 -- 172.21.15.26:0/1499402813 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf600ab600 con 0x7faf740c9310 2023-12-19T18:17:16.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf737fe640 1 -- 172.21.15.26:0/1499402813 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf600a7050 con 0x7faf740c9310 2023-12-19T18:17:16.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf737fe640 1 -- 172.21.15.26:0/1499402813 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf600ab980 con 0x7faf740c9310 2023-12-19T18:17:16.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf7aab6640 1 -- 172.21.15.26:0/1499402813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf740c9310 msgr2=0x7faf740cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:16.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf7aab6640 1 --2- 172.21.15.26:0/1499402813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf740c9310 0x7faf740cc430 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7faf60009870 tx=0x7faf6001c470 comp rx=0 tx=0).stop 2023-12-19T18:17:16.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf7aab6640 1 -- 172.21.15.26:0/1499402813 shutdown_connections 2023-12-19T18:17:16.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf7aab6640 1 --2- 172.21.15.26:0/1499402813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf740c9310 0x7faf740cc430 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf7aab6640 1 -- 172.21.15.26:0/1499402813 >> 172.21.15.26:0/1499402813 conn(0x7faf740b0270 msgr2=0x7faf740b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:16.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf7aab6640 1 -- 172.21.15.26:0/1499402813 shutdown_connections 2023-12-19T18:17:16.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.969+0000 7faf7aab6640 1 -- 172.21.15.26:0/1499402813 wait complete. 2023-12-19T18:17:16.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf7aab6640 1 Processor -- start 2023-12-19T18:17:16.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf7aab6640 1 -- start start 2023-12-19T18:17:16.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf7aab6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf740c9310 0x7faf740c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:16.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf7aab6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf740cb110 0x7faf740b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:16.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf7aab6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf74159830 0x7faf740b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:16.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf7aab6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740c6590 con 0x7faf74159830 2023-12-19T18:17:16.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf7aab6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740c6870 con 0x7faf740cb110 2023-12-19T18:17:16.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf7aab6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf740c6b50 con 0x7faf740c9310 2023-12-19T18:17:16.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf79053640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf74159830 0x7faf740b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:16.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf79053640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf74159830 0x7faf740b3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50932/0 (socket says 172.21.15.26:50932) 2023-12-19T18:17:16.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf79053640 1 -- 172.21.15.26:0/3949174211 learned_addr learned my addr 172.21.15.26:0/3949174211 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:16.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf78852640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf740c9310 0x7faf740c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:16.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf73fff640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf740cb110 0x7faf740b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:16.979 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf79053640 1 -- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf740c9310 msgr2=0x7faf740c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:16.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf79053640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf740c9310 0x7faf740c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf79053640 1 -- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf740cb110 msgr2=0x7faf740b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:16.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf79053640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf740cb110 0x7faf740b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.973+0000 7faf79053640 1 -- 172.21.15.26:0/3949174211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf600094d0 con 0x7faf74159830 2023-12-19T18:17:16.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.977+0000 7faf79053640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf74159830 0x7faf740b3eb0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00c7a0 tx=0x7faf6c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:16.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.977+0000 7faf717fa640 1 -- 172.21.15.26:0/3949174211 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c00a640 con 0x7faf74159830 2023-12-19T18:17:16.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.977+0000 7faf717fa640 1 -- 172.21.15.26:0/3949174211 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6c00a7a0 con 0x7faf74159830 2023-12-19T18:17:16.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.977+0000 7faf717fa640 1 -- 172.21.15.26:0/3949174211 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c00abd0 con 0x7faf74159830 2023-12-19T18:17:16.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.977+0000 7faf7aab6640 1 -- 172.21.15.26:0/3949174211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf740cbab0 con 0x7faf74159830 2023-12-19T18:17:16.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.977+0000 7faf7aab6640 1 -- 172.21.15.26:0/3949174211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf740c7110 con 0x7faf74159830 2023-12-19T18:17:16.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.977+0000 7faf7aab6640 1 -- 172.21.15.26:0/3949174211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf740c7300 con 0x7faf74159830 2023-12-19T18:17:16.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.977+0000 7faf717fa640 1 -- 172.21.15.26:0/3949174211 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7faf6c00ad30 con 0x7faf74159830 2023-12-19T18:17:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.977+0000 7faf717fa640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf58065ca0 0x7faf58068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.977+0000 7faf717fa640 1 -- 172.21.15.26:0/3949174211 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(222..222 src has 1..222) v4 ==== 7384+0+0 (secure 0 0 0) 0x7faf6c083120 con 0x7faf74159830 2023-12-19T18:17:16.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.977+0000 7faf78852640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf58065ca0 0x7faf58068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:16.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.981+0000 7faf78852640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf58065ca0 0x7faf58068160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf680098e0 tx=0x7faf68009060 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:16.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:16.981+0000 7faf717fa640 1 -- 172.21.15.26:0/3949174211 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faf6c050190 con 0x7faf74159830 2023-12-19T18:17:17.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.141+0000 7faf7aab6640 1 -- 172.21.15.26:0/3949174211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faf74074270 con 0x7faf74159830 2023-12-19T18:17:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.793+0000 7faf717fa640 1 -- 172.21.15.26:0/3949174211 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v223) v1 ==== 190+0+0 (secure 0 0 0) 0x7faf6c018020 con 0x7faf74159830 2023-12-19T18:17:17.799 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'backup_fs_metadata' removed 2023-12-19T18:17:17.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.797+0000 7faf7aab6640 1 -- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf58065ca0 msgr2=0x7faf58068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:17.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.797+0000 7faf7aab6640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf58065ca0 0x7faf58068160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf680098e0 tx=0x7faf68009060 comp rx=0 tx=0).stop 2023-12-19T18:17:17.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.797+0000 7faf7aab6640 1 -- 172.21.15.26:0/3949174211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf74159830 msgr2=0x7faf740b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:17.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.797+0000 7faf7aab6640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf74159830 0x7faf740b3eb0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7faf6c00c7a0 tx=0x7faf6c00cc60 comp rx=0 tx=0).stop 2023-12-19T18:17:17.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.797+0000 7faf7aab6640 1 -- 172.21.15.26:0/3949174211 shutdown_connections 2023-12-19T18:17:17.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.797+0000 7faf7aab6640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf58065ca0 0x7faf58068160 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.797+0000 7faf7aab6640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf74159830 0x7faf740b3eb0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.797+0000 7faf7aab6640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf740cb110 0x7faf740b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.797+0000 7faf7aab6640 1 --2- 172.21.15.26:0/3949174211 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf740c9310 0x7faf740c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.797+0000 7faf7aab6640 1 -- 172.21.15.26:0/3949174211 >> 172.21.15.26:0/3949174211 conn(0x7faf740b0270 msgr2=0x7faf74158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.797+0000 7faf7aab6640 1 -- 172.21.15.26:0/3949174211 shutdown_connections 2023-12-19T18:17:17.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.797+0000 7faf7aab6640 1 -- 172.21.15.26:0/3949174211 wait complete. 2023-12-19T18:17:17.826 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-12-19T18:17:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb3a141f640 1 Processor -- start 2023-12-19T18:17:17.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb3a141f640 1 -- start start 2023-12-19T18:17:17.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb3a141f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb39c0c9310 0x7fb39c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:17.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb3a141f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb39c0cb110 0x7fb39c0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:17.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb3a141f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39c15e3d0 con 0x7fb39c159830 2023-12-19T18:17:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb3a141f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39c06f7c0 con 0x7fb39c0c9310 2023-12-19T18:17:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb3a141f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39c0ca2b0 con 0x7fb39c0cb110 2023-12-19T18:17:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb393fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb39c0c9310 0x7fb39c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb39affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb39c159830 0x7fb39c15d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44222/0 (socket says 172.21.15.26:44222) 2023-12-19T18:17:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb39affd640 1 -- 172.21.15.26:0/2041248060 learned_addr learned my addr 172.21.15.26:0/2041248060 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb39b7fe640 1 --2- 172.21.15.26:0/2041248060 >> v2:172.21.15.160:3300/0 conn(0x7fb39c0cb110 0x7fb39c0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb393fff640 1 -- 172.21.15.26:0/2041248060 >> v2:172.21.15.160:3300/0 conn(0x7fb39c0cb110 msgr2=0x7fb39c0cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb393fff640 1 --2- 172.21.15.26:0/2041248060 >> v2:172.21.15.160:3300/0 conn(0x7fb39c0cb110 0x7fb39c0cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb393fff640 1 -- 172.21.15.26:0/2041248060 >> v1:172.21.15.26:6789/0 conn(0x7fb39c159830 legacy=0x7fb39c15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb393fff640 1 -- 172.21.15.26:0/2041248060 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39c0cba80 con 0x7fb39c0c9310 2023-12-19T18:17:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb39a7fc640 1 -- 172.21.15.26:0/2041248060 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277509756 0 0) 0x7fb39c15e3d0 con 0x7fb39c159830 2023-12-19T18:17:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.977+0000 7fb39b7fe640 1 --2- 172.21.15.26:0/2041248060 >> v2:172.21.15.160:3300/0 conn(0x7fb39c0cb110 0x7fb39c0cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:17.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.981+0000 7fb393fff640 1 --2- 172.21.15.26:0/2041248060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39c0c9310 0x7fb39c0cc430 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0ae070 tx=0x7fb38401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6070dbe121255d35 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:17.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.981+0000 7fb39a7fc640 1 -- 172.21.15.26:0/2041248060 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3840ab600 con 0x7fb39c0c9310 2023-12-19T18:17:17.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.981+0000 7fb39a7fc640 1 -- 172.21.15.26:0/2041248060 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3840a7050 con 0x7fb39c0c9310 2023-12-19T18:17:17.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.981+0000 7fb39a7fc640 1 -- 172.21.15.26:0/2041248060 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3840ab980 con 0x7fb39c0c9310 2023-12-19T18:17:17.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.981+0000 7fb3a141f640 1 -- 172.21.15.26:0/2041248060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39c0c9310 msgr2=0x7fb39c0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:17.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.981+0000 7fb3a141f640 1 --2- 172.21.15.26:0/2041248060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39c0c9310 0x7fb39c0cc430 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0ae070 tx=0x7fb38401c3d0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.981+0000 7fb3a141f640 1 -- 172.21.15.26:0/2041248060 shutdown_connections 2023-12-19T18:17:17.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.981+0000 7fb3a141f640 1 --2- 172.21.15.26:0/2041248060 >> v2:172.21.15.160:3300/0 conn(0x7fb39c0cb110 0x7fb39c0cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.981+0000 7fb3a141f640 1 --2- 172.21.15.26:0/2041248060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39c0c9310 0x7fb39c0cc430 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.981+0000 7fb3a141f640 1 -- 172.21.15.26:0/2041248060 >> 172.21.15.26:0/2041248060 conn(0x7fb39c0b0270 msgr2=0x7fb39c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:17.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.981+0000 7fb3a141f640 1 -- 172.21.15.26:0/2041248060 shutdown_connections 2023-12-19T18:17:17.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.981+0000 7fb3a141f640 1 -- 172.21.15.26:0/2041248060 wait complete. 2023-12-19T18:17:17.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb3a141f640 1 Processor -- start 2023-12-19T18:17:17.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb3a141f640 1 -- start start 2023-12-19T18:17:17.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb3a141f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb39c0c9310 0x7fb39c152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:17.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb3a141f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39c0cb110 0x7fb39c0c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:17.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb3a141f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb39c159830 0x7fb39c0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:17.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb3a141f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c0715e0 con 0x7fb39c0cb110 2023-12-19T18:17:17.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb3a141f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c0b4310 con 0x7fb39c159830 2023-12-19T18:17:17.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb3a141f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39c1531e0 con 0x7fb39c0c9310 2023-12-19T18:17:17.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb393fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39c0cb110 0x7fb39c0c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb393fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39c0cb110 0x7fb39c0c2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50940/0 (socket says 172.21.15.26:50940) 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb393fff640 1 -- 172.21.15.26:0/3173577123 learned_addr learned my addr 172.21.15.26:0/3173577123 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb39b7fe640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb39c159830 0x7fb39c0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb39affd640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb39c0c9310 0x7fb39c152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb393fff640 1 -- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb39c0c9310 msgr2=0x7fb39c152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb393fff640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb39c0c9310 0x7fb39c152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb393fff640 1 -- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb39c159830 msgr2=0x7fb39c0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb393fff640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb39c159830 0x7fb39c0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb393fff640 1 -- 172.21.15.26:0/3173577123 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb384009420 con 0x7fb39c0cb110 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb39b7fe640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb39c159830 0x7fb39c0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb393fff640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39c0cb110 0x7fb39c0c2250 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fb38401a000 tx=0x7fb384019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb3937fe640 1 -- 172.21.15.26:0/3173577123 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3840b4430 con 0x7fb39c0cb110 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb3937fe640 1 -- 172.21.15.26:0/3173577123 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3840b45b0 con 0x7fb39c0cb110 2023-12-19T18:17:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb3a141f640 1 -- 172.21.15.26:0/3173577123 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39c0c2790 con 0x7fb39c0cb110 2023-12-19T18:17:17.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb3937fe640 1 -- 172.21.15.26:0/3173577123 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3840b4960 con 0x7fb39c0cb110 2023-12-19T18:17:17.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.985+0000 7fb3a141f640 1 -- 172.21.15.26:0/3173577123 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39c0b3a70 con 0x7fb39c0cb110 2023-12-19T18:17:17.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.989+0000 7fb3a141f640 1 -- 172.21.15.26:0/3173577123 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39c074270 con 0x7fb39c0cb110 2023-12-19T18:17:17.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.989+0000 7fb3937fe640 1 -- 172.21.15.26:0/3173577123 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb3840b2040 con 0x7fb39c0cb110 2023-12-19T18:17:17.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.989+0000 7fb3937fe640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb37c065ca0 0x7fb37c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:17.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.989+0000 7fb3937fe640 1 -- 172.21.15.26:0/3173577123 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(223..223 src has 1..223) v4 ==== 6940+0+0 (secure 0 0 0) 0x7fb38411db50 con 0x7fb39c0cb110 2023-12-19T18:17:17.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.989+0000 7fb39affd640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb37c065ca0 0x7fb37c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:17.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.993+0000 7fb39affd640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb37c065ca0 0x7fb37c068160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb38c000f70 tx=0x7fb38c007c10 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:17.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:17.993+0000 7fb3937fe640 1 -- 172.21.15.26:0/3173577123 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb3840eac00 con 0x7fb39c0cb110 2023-12-19T18:17:18.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.153+0000 7fb3a141f640 1 -- 172.21.15.26:0/3173577123 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb39c06fa30 con 0x7fb39c0cb110 2023-12-19T18:17:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.797+0000 7fb3937fe640 1 -- 172.21.15.26:0/3173577123 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v224) v1 ==== 178+0+0 (secure 0 0 0) 0x7fb3840ee220 con 0x7fb39c0cb110 2023-12-19T18:17:18.802 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'backup_fs_data' removed 2023-12-19T18:17:18.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.801+0000 7fb3a141f640 1 -- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb37c065ca0 msgr2=0x7fb37c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:18.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.801+0000 7fb3a141f640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb37c065ca0 0x7fb37c068160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb38c000f70 tx=0x7fb38c007c10 comp rx=0 tx=0).stop 2023-12-19T18:17:18.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.801+0000 7fb3a141f640 1 -- 172.21.15.26:0/3173577123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39c0cb110 msgr2=0x7fb39c0c2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:18.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.801+0000 7fb3a141f640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39c0cb110 0x7fb39c0c2250 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fb38401a000 tx=0x7fb384019170 comp rx=0 tx=0).stop 2023-12-19T18:17:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.801+0000 7fb3a141f640 1 -- 172.21.15.26:0/3173577123 shutdown_connections 2023-12-19T18:17:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.801+0000 7fb3a141f640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb37c065ca0 0x7fb37c068160 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.801+0000 7fb3a141f640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb39c159830 0x7fb39c0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.801+0000 7fb3a141f640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39c0cb110 0x7fb39c0c2250 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.801+0000 7fb3a141f640 1 --2- 172.21.15.26:0/3173577123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb39c0c9310 0x7fb39c152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.801+0000 7fb3a141f640 1 -- 172.21.15.26:0/3173577123 >> 172.21.15.26:0/3173577123 conn(0x7fb39c0b0270 msgr2=0x7fb39c0b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.805+0000 7fb3a141f640 1 -- 172.21.15.26:0/3173577123 shutdown_connections 2023-12-19T18:17:18.807 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.805+0000 7fb3a141f640 1 -- 172.21.15.26:0/3173577123 wait complete. 2023-12-19T18:17:18.832 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-19T18:17:18.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.993+0000 7f1798d96640 1 Processor -- start 2023-12-19T18:17:18.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.993+0000 7f1798d96640 1 -- start start 2023-12-19T18:17:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.997+0000 7f1798d96640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f17940cb860 0x7f179406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.997+0000 7f1798d96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f179415d530 0x7f179415d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.997+0000 7f1798d96640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f179415e240 con 0x7f17941599e0 2023-12-19T18:17:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.997+0000 7f1798d96640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17940c9520 con 0x7f17940cb860 2023-12-19T18:17:19.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.997+0000 7f1798d96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17940c9800 con 0x7f179415d530 2023-12-19T18:17:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.997+0000 7f1792d9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f179415d530 0x7f179415d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.997+0000 7f179259c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f17941599e0 0x7f17940cb150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:51004/0 (socket says 172.21.15.26:51004) 2023-12-19T18:17:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.997+0000 7f1791d9b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f17940cb860 0x7f179406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.997+0000 7f179259c640 1 -- 172.21.15.26:0/1343048502 learned_addr learned my addr 172.21.15.26:0/1343048502 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.997+0000 7f1792d9d640 1 -- 172.21.15.26:0/1343048502 >> v2:172.21.15.160:3300/0 conn(0x7f17940cb860 msgr2=0x7f179406f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.997+0000 7f1792d9d640 1 --2- 172.21.15.26:0/1343048502 >> v2:172.21.15.160:3300/0 conn(0x7f17940cb860 0x7f179406f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.997+0000 7f1792d9d640 1 -- 172.21.15.26:0/1343048502 >> v1:172.21.15.160:6789/0 conn(0x7f17941599e0 legacy=0x7f17940cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:18.997+0000 7f1792d9d640 1 -- 172.21.15.26:0/1343048502 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17940c9ae0 con 0x7f179415d530 2023-12-19T18:17:19.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f1792d9d640 1 --2- 172.21.15.26:0/1343048502 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f179415d530 0x7f179415d980 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f178c009d90 tx=0x7f178c01c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e2985c9378ac78 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f179159a640 1 -- 172.21.15.26:0/1343048502 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c0a5a30 con 0x7f179415d530 2023-12-19T18:17:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f179159a640 1 -- 172.21.15.26:0/1343048502 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178c0a5b90 con 0x7f179415d530 2023-12-19T18:17:19.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f179159a640 1 -- 172.21.15.26:0/1343048502 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c0ad440 con 0x7f179415d530 2023-12-19T18:17:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f1798d96640 1 -- 172.21.15.26:0/1343048502 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f179415d530 msgr2=0x7f179415d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f1798d96640 1 --2- 172.21.15.26:0/1343048502 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f179415d530 0x7f179415d980 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f178c009d90 tx=0x7f178c01c950 comp rx=0 tx=0).stop 2023-12-19T18:17:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f1798d96640 1 -- 172.21.15.26:0/1343048502 shutdown_connections 2023-12-19T18:17:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f1798d96640 1 --2- 172.21.15.26:0/1343048502 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f179415d530 0x7f179415d980 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f1798d96640 1 --2- 172.21.15.26:0/1343048502 >> v2:172.21.15.160:3300/0 conn(0x7f17940cb860 0x7f179406f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f1798d96640 1 -- 172.21.15.26:0/1343048502 >> 172.21.15.26:0/1343048502 conn(0x7f1794066160 msgr2=0x7f17940b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f1798d96640 1 -- 172.21.15.26:0/1343048502 shutdown_connections 2023-12-19T18:17:19.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f1798d96640 1 -- 172.21.15.26:0/1343048502 wait complete. 2023-12-19T18:17:19.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f1798d96640 1 Processor -- start 2023-12-19T18:17:19.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.001+0000 7f1798d96640 1 -- start start 2023-12-19T18:17:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1798d96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17940cb860 0x7f179414d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1798d96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17941599e0 0x7f179415c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1798d96640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f179415d530 0x7f17940ce740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1798d96640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179415ce60 con 0x7f17940cb860 2023-12-19T18:17:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1798d96640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17940ae4f0 con 0x7f179415d530 2023-12-19T18:17:19.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1798d96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17940ae7d0 con 0x7f17941599e0 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f179259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17940cb860 0x7f179414d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1792d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f179415d530 0x7f17940ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f179259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17940cb860 0x7f179414d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50944/0 (socket says 172.21.15.26:50944) 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f179259c640 1 -- 172.21.15.26:0/3266781129 learned_addr learned my addr 172.21.15.26:0/3266781129 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1791d9b640 1 --2- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17941599e0 0x7f179415c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1792d9d640 1 -- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17941599e0 msgr2=0x7f179415c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1792d9d640 1 --2- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17941599e0 0x7f179415c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1792d9d640 1 -- 172.21.15.26:0/3266781129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17940cb860 msgr2=0x7f179414d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1792d9d640 1 --2- 172.21.15.26:0/3266781129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17940cb860 0x7f179414d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1792d9d640 1 -- 172.21.15.26:0/3266781129 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c009a70 con 0x7f179415d530 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1792d9d640 1 --2- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f179415d530 0x7f17940ce740 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f178c0ac000 tx=0x7f17940cb010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1782ffd640 1 -- 172.21.15.26:0/3266781129 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c0be070 con 0x7f179415d530 2023-12-19T18:17:19.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1782ffd640 1 -- 172.21.15.26:0/3266781129 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178c009450 con 0x7f179415d530 2023-12-19T18:17:19.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1798d96640 1 -- 172.21.15.26:0/3266781129 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17940aeab0 con 0x7f179415d530 2023-12-19T18:17:19.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1782ffd640 1 -- 172.21.15.26:0/3266781129 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178c0ade10 con 0x7f179415d530 2023-12-19T18:17:19.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1798d96640 1 -- 172.21.15.26:0/3266781129 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17940af070 con 0x7f179415d530 2023-12-19T18:17:19.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.005+0000 7f1798d96640 1 -- 172.21.15.26:0/3266781129 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17940c3690 con 0x7f179415d530 2023-12-19T18:17:19.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.009+0000 7f1782ffd640 1 -- 172.21.15.26:0/3266781129 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f178c0b4040 con 0x7f179415d530 2023-12-19T18:17:19.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.009+0000 7f1782ffd640 1 --2- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1774065ca0 0x7f1774068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.009+0000 7f1782ffd640 1 -- 172.21.15.26:0/3266781129 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6520+0+0 (secure 0 0 0) 0x7f178c11e0c0 con 0x7f179415d530 2023-12-19T18:17:19.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.009+0000 7f179259c640 1 --2- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1774065ca0 0x7f1774068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.009+0000 7f179259c640 1 --2- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1774065ca0 0x7f1774068160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1784000f70 tx=0x7f1784009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.013+0000 7f1782ffd640 1 -- 172.21.15.26:0/3266781129 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f178c0eb5e0 con 0x7f179415d530 2023-12-19T18:17:19.176 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.173+0000 7f1798d96640 1 -- 172.21.15.26:0/3266781129 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f1794074270 con 0x7f179415d530 2023-12-19T18:17:19.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.177+0000 7f1782ffd640 1 -- 172.21.15.26:0/3266781129 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v73) v1 ==== 120+0+0 (secure 0 0 0) 0x7f178c0eec00 con 0x7f179415d530 2023-12-19T18:17:19.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.181+0000 7f1798d96640 1 -- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1774065ca0 msgr2=0x7f1774068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.181+0000 7f1798d96640 1 --2- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1774065ca0 0x7f1774068160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1784000f70 tx=0x7f1784009040 comp rx=0 tx=0).stop 2023-12-19T18:17:19.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.181+0000 7f1798d96640 1 -- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f179415d530 msgr2=0x7f17940ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.181+0000 7f1798d96640 1 --2- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f179415d530 0x7f17940ce740 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f178c0ac000 tx=0x7f17940cb010 comp rx=0 tx=0).stop 2023-12-19T18:17:19.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.181+0000 7f1798d96640 1 -- 172.21.15.26:0/3266781129 shutdown_connections 2023-12-19T18:17:19.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.181+0000 7f1798d96640 1 --2- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1774065ca0 0x7f1774068160 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.181+0000 7f1798d96640 1 --2- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f179415d530 0x7f17940ce740 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.181+0000 7f1798d96640 1 --2- 172.21.15.26:0/3266781129 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f17941599e0 0x7f179415c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.181+0000 7f1798d96640 1 --2- 172.21.15.26:0/3266781129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f17940cb860 0x7f179414d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.181+0000 7f1798d96640 1 -- 172.21.15.26:0/3266781129 >> 172.21.15.26:0/3266781129 conn(0x7f1794066160 msgr2=0x7f17940cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:19.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.181+0000 7f1798d96640 1 -- 172.21.15.26:0/3266781129 shutdown_connections 2023-12-19T18:17:19.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.181+0000 7f1798d96640 1 -- 172.21.15.26:0/3266781129 wait complete. 2023-12-19T18:17:19.204 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:17:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.345+0000 7f0d0b59e640 1 Processor -- start 2023-12-19T18:17:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.345+0000 7f0d0b59e640 1 -- start start 2023-12-19T18:17:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0b59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d0c071d40 0x7f0d0c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0b59e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0d0c06baa0 0x7f0d0c15c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0b59e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d0c0d0490 con 0x7f0d0c072120 2023-12-19T18:17:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0b59e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d0c0cfb60 con 0x7f0d0c06baa0 2023-12-19T18:17:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0b59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c0cfe40 con 0x7f0d0c071d40 2023-12-19T18:17:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0ad9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0d0c06baa0 0x7f0d0c15c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0a59c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d0c071d40 0x7f0d0c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d09d9b640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f0d0c072120 0x7f0d0c06b390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:51010/0 (socket says 172.21.15.26:51010) 2023-12-19T18:17:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0a59c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d0c071d40 0x7f0d0c071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:60130/0 (socket says 172.21.15.26:60130) 2023-12-19T18:17:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d09d9b640 1 -- 172.21.15.26:0/1062244457 learned_addr learned my addr 172.21.15.26:0/1062244457 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0ad9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0d0c06baa0 0x7f0d0c15c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:47676/0 (socket says 172.21.15.26:47676) 2023-12-19T18:17:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0ad9d640 1 -- 172.21.15.26:0/1062244457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d0c071d40 msgr2=0x7f0d0c071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0ad9d640 1 --2- 172.21.15.26:0/1062244457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d0c071d40 0x7f0d0c071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0ad9d640 1 -- 172.21.15.26:0/1062244457 >> v1:172.21.15.160:6789/0 conn(0x7f0d0c072120 legacy=0x7f0d0c06b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0ad9d640 1 -- 172.21.15.26:0/1062244457 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c0d0120 con 0x7f0d0c06baa0 2023-12-19T18:17:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.349+0000 7f0d0959a640 1 -- 172.21.15.26:0/1062244457 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847822651 0 0) 0x7f0d0c0d0490 con 0x7f0d0c072120 2023-12-19T18:17:19.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0ad9d640 1 --2- 172.21.15.26:0/1062244457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d0c06baa0 0x7f0d0c15c400 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009d90 tx=0x7f0d0401c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37cc00158504884 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0959a640 1 -- 172.21.15.26:0/1062244457 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0cd080 con 0x7f0d0c06baa0 2023-12-19T18:17:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0959a640 1 -- 172.21.15.26:0/1062244457 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d040a59a0 con 0x7f0d0c06baa0 2023-12-19T18:17:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0959a640 1 -- 172.21.15.26:0/1062244457 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040a5d30 con 0x7f0d0c06baa0 2023-12-19T18:17:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0b59e640 1 -- 172.21.15.26:0/1062244457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d0c06baa0 msgr2=0x7f0d0c15c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0b59e640 1 --2- 172.21.15.26:0/1062244457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d0c06baa0 0x7f0d0c15c400 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009d90 tx=0x7f0d0401c8c0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0b59e640 1 -- 172.21.15.26:0/1062244457 shutdown_connections 2023-12-19T18:17:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0b59e640 1 --2- 172.21.15.26:0/1062244457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d0c06baa0 0x7f0d0c15c400 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0b59e640 1 --2- 172.21.15.26:0/1062244457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d0c071d40 0x7f0d0c071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0b59e640 1 -- 172.21.15.26:0/1062244457 >> 172.21.15.26:0/1062244457 conn(0x7f0d0c0cd210 msgr2=0x7f0d0c0cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0b59e640 1 -- 172.21.15.26:0/1062244457 shutdown_connections 2023-12-19T18:17:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0b59e640 1 -- 172.21.15.26:0/1062244457 wait complete. 2023-12-19T18:17:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0b59e640 1 Processor -- start 2023-12-19T18:17:19.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.353+0000 7f0d0b59e640 1 -- start start 2023-12-19T18:17:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d0b59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d0c06baa0 0x7f0d0c0ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d0b59e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d0c071d40 0x7f0d0c155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d0b59e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d0c072120 0x7f0d0c0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d0b59e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c0adc30 con 0x7f0d0c071d40 2023-12-19T18:17:19.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d0b59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c1563d0 con 0x7f0d0c06baa0 2023-12-19T18:17:19.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d0b59e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c0acda0 con 0x7f0d0c072120 2023-12-19T18:17:19.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d09d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d0c071d40 0x7f0d0c155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d0a59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d0c06baa0 0x7f0d0c0ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d09d9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d0c071d40 0x7f0d0c155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50946/0 (socket says 172.21.15.26:50946) 2023-12-19T18:17:19.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d09d9b640 1 -- 172.21.15.26:0/3442198554 learned_addr learned my addr 172.21.15.26:0/3442198554 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:19.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d0ad9d640 1 --2- 172.21.15.26:0/3442198554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d0c072120 0x7f0d0c0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d09d9b640 1 -- 172.21.15.26:0/3442198554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d0c072120 msgr2=0x7f0d0c0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d09d9b640 1 --2- 172.21.15.26:0/3442198554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d0c072120 0x7f0d0c0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d09d9b640 1 -- 172.21.15.26:0/3442198554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d0c06baa0 msgr2=0x7f0d0c0ae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d09d9b640 1 --2- 172.21.15.26:0/3442198554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d0c06baa0 0x7f0d0c0ae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d09d9b640 1 -- 172.21.15.26:0/3442198554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d04009a70 con 0x7f0d0c071d40 2023-12-19T18:17:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d09d9b640 1 --2- 172.21.15.26:0/3442198554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d0c071d40 0x7f0d0c155e90 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c480 tx=0x7f0d0000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0cfaffd640 1 -- 172.21.15.26:0/3442198554 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d000074e0 con 0x7f0d0c071d40 2023-12-19T18:17:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0cfaffd640 1 -- 172.21.15.26:0/3442198554 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d00007640 con 0x7f0d0c071d40 2023-12-19T18:17:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0cfaffd640 1 -- 172.21.15.26:0/3442198554 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d000049f0 con 0x7f0d0c071d40 2023-12-19T18:17:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d0b59e640 1 -- 172.21.15.26:0/3442198554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d0c0acfb0 con 0x7f0d0c071d40 2023-12-19T18:17:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d0b59e640 1 -- 172.21.15.26:0/3442198554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0c0ad570 con 0x7f0d0c071d40 2023-12-19T18:17:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0d0b59e640 1 -- 172.21.15.26:0/3442198554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d0c0add70 con 0x7f0d0c071d40 2023-12-19T18:17:19.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.357+0000 7f0cfaffd640 1 -- 172.21.15.26:0/3442198554 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f0d00004b50 con 0x7f0d0c071d40 2023-12-19T18:17:19.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.361+0000 7f0cfaffd640 1 --2- 172.21.15.26:0/3442198554 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0cec065ca0 0x7f0cec068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.361+0000 7f0cfaffd640 1 -- 172.21.15.26:0/3442198554 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6520+0+0 (secure 0 0 0) 0x7f0d00082a90 con 0x7f0d0c071d40 2023-12-19T18:17:19.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.361+0000 7f0d0a59c640 1 --2- 172.21.15.26:0/3442198554 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0cec065ca0 0x7f0cec068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.361+0000 7f0d0a59c640 1 --2- 172.21.15.26:0/3442198554 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0cec065ca0 0x7f0cec068160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c168890 tx=0x7f0cfc009190 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.367 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.365+0000 7f0cfaffd640 1 -- 172.21.15.26:0/3442198554 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0d0c0add70 con 0x7f0d0c071d40 2023-12-19T18:17:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.521+0000 7f0d0b59e640 1 -- 172.21.15.26:0/3442198554 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0d0c0add70 con 0x7f0d0c071d40 2023-12-19T18:17:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.525+0000 7f0cfaffd640 1 -- 172.21.15.26:0/3442198554 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v74) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0d0c0add70 con 0x7f0d0c071d40 2023-12-19T18:17:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.529+0000 7f0d0b59e640 1 -- 172.21.15.26:0/3442198554 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0cec065ca0 msgr2=0x7f0cec068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.529+0000 7f0d0b59e640 1 --2- 172.21.15.26:0/3442198554 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0cec065ca0 0x7f0cec068160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c168890 tx=0x7f0cfc009190 comp rx=0 tx=0).stop 2023-12-19T18:17:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.529+0000 7f0d0b59e640 1 -- 172.21.15.26:0/3442198554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d0c071d40 msgr2=0x7f0d0c155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.529+0000 7f0d0b59e640 1 --2- 172.21.15.26:0/3442198554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d0c071d40 0x7f0d0c155e90 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000c480 tx=0x7f0d0000c940 comp rx=0 tx=0).stop 2023-12-19T18:17:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.529+0000 7f0d0b59e640 1 -- 172.21.15.26:0/3442198554 shutdown_connections 2023-12-19T18:17:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.529+0000 7f0d0b59e640 1 --2- 172.21.15.26:0/3442198554 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0cec065ca0 0x7f0cec068160 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.529+0000 7f0d0b59e640 1 --2- 172.21.15.26:0/3442198554 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0d0c072120 0x7f0d0c0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.529+0000 7f0d0b59e640 1 --2- 172.21.15.26:0/3442198554 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d0c071d40 0x7f0d0c155e90 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.529+0000 7f0d0b59e640 1 --2- 172.21.15.26:0/3442198554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0d0c06baa0 0x7f0d0c0ae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.529+0000 7f0d0b59e640 1 -- 172.21.15.26:0/3442198554 >> 172.21.15.26:0/3442198554 conn(0x7f0d0c0cd210 msgr2=0x7f0d0c0b4720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:19.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.529+0000 7f0d0b59e640 1 -- 172.21.15.26:0/3442198554 shutdown_connections 2023-12-19T18:17:19.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.529+0000 7f0d0b59e640 1 -- 172.21.15.26:0/3442198554 wait complete. 2023-12-19T18:17:19.552 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-19T18:17:19.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.693+0000 7fee61c8c640 1 Processor -- start 2023-12-19T18:17:19.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.693+0000 7fee61c8c640 1 -- start start 2023-12-19T18:17:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.693+0000 7fee61c8c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fee5c0ca820 0x7fee5c06b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.693+0000 7fee61c8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee5c06b850 0x7fee5c0ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.693+0000 7fee61c8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5c0cc310 con 0x7fee5c159090 2023-12-19T18:17:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.693+0000 7fee61c8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee5c06bce0 con 0x7fee5c06b850 2023-12-19T18:17:19.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.693+0000 7fee61c8c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee5c0c9eb0 con 0x7fee5c0ca820 2023-12-19T18:17:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.697+0000 7fee5bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee5c06b850 0x7fee5c0ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.697+0000 7fee5b7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fee5c159090 0x7fee5c0c97a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44228/0 (socket says 172.21.15.26:44228) 2023-12-19T18:17:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.697+0000 7fee5b7fe640 1 -- 172.21.15.26:0/1846075086 learned_addr learned my addr 172.21.15.26:0/1846075086 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.697+0000 7fee5affd640 1 --2- 172.21.15.26:0/1846075086 >> v2:172.21.15.160:3300/0 conn(0x7fee5c0ca820 0x7fee5c06b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.697+0000 7fee5bfff640 1 -- 172.21.15.26:0/1846075086 >> v2:172.21.15.160:3300/0 conn(0x7fee5c0ca820 msgr2=0x7fee5c06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.697+0000 7fee5bfff640 1 --2- 172.21.15.26:0/1846075086 >> v2:172.21.15.160:3300/0 conn(0x7fee5c0ca820 0x7fee5c06b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.697+0000 7fee5bfff640 1 -- 172.21.15.26:0/1846075086 >> v1:172.21.15.26:6789/0 conn(0x7fee5c159090 legacy=0x7fee5c0c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.697+0000 7fee5bfff640 1 -- 172.21.15.26:0/1846075086 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c0ca110 con 0x7fee5c06b850 2023-12-19T18:17:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.697+0000 7fee5bfff640 1 --2- 172.21.15.26:0/1846075086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c06b850 0x7fee5c0ccd60 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0b0f00 tx=0x7fee5401eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762384add32b25b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.697+0000 7fee5a7fc640 1 -- 172.21.15.26:0/1846075086 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee540a7b50 con 0x7fee5c06b850 2023-12-19T18:17:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.697+0000 7fee5a7fc640 1 -- 172.21.15.26:0/1846075086 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee540a7cb0 con 0x7fee5c06b850 2023-12-19T18:17:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.697+0000 7fee5a7fc640 1 -- 172.21.15.26:0/1846075086 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee540af660 con 0x7fee5c06b850 2023-12-19T18:17:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 -- 172.21.15.26:0/1846075086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c06b850 msgr2=0x7fee5c0ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 --2- 172.21.15.26:0/1846075086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c06b850 0x7fee5c0ccd60 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fee5c0b0f00 tx=0x7fee5401eb00 comp rx=0 tx=0).stop 2023-12-19T18:17:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 -- 172.21.15.26:0/1846075086 shutdown_connections 2023-12-19T18:17:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 --2- 172.21.15.26:0/1846075086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c06b850 0x7fee5c0ccd60 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 --2- 172.21.15.26:0/1846075086 >> v2:172.21.15.160:3300/0 conn(0x7fee5c0ca820 0x7fee5c06b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 -- 172.21.15.26:0/1846075086 >> 172.21.15.26:0/1846075086 conn(0x7fee5c066160 msgr2=0x7fee5c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 -- 172.21.15.26:0/1846075086 shutdown_connections 2023-12-19T18:17:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 -- 172.21.15.26:0/1846075086 wait complete. 2023-12-19T18:17:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 Processor -- start 2023-12-19T18:17:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 -- start start 2023-12-19T18:17:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee5c06b850 0x7fee5c1631e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee5c0ca820 0x7fee5c0c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c159090 0x7fee5c070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0c6da0 con 0x7fee5c159090 2023-12-19T18:17:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0c6fd0 con 0x7fee5c06b850 2023-12-19T18:17:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee61c8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0c72b0 con 0x7fee5c0ca820 2023-12-19T18:17:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee5bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c159090 0x7fee5c070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee5bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c159090 0x7fee5c070360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50964/0 (socket says 172.21.15.26:50964) 2023-12-19T18:17:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee5bfff640 1 -- 172.21.15.26:0/1498253441 learned_addr learned my addr 172.21.15.26:0/1498253441 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee5affd640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee5c0ca820 0x7fee5c0c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee5b7fe640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee5c06b850 0x7fee5c1631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee5bfff640 1 -- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee5c0ca820 msgr2=0x7fee5c0c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee5bfff640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee5c0ca820 0x7fee5c0c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee5bfff640 1 -- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee5c06b850 msgr2=0x7fee5c1631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee5bfff640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee5c06b850 0x7fee5c1631e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.701+0000 7fee5bfff640 1 -- 172.21.15.26:0/1498253441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee54009a70 con 0x7fee5c159090 2023-12-19T18:17:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.705+0000 7fee5bfff640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c159090 0x7fee5c070360 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fee5401b710 tx=0x7fee5401b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.705+0000 7fee43fff640 1 -- 172.21.15.26:0/1498253441 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5400f250 con 0x7fee5c159090 2023-12-19T18:17:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.705+0000 7fee43fff640 1 -- 172.21.15.26:0/1498253441 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee54009450 con 0x7fee5c159090 2023-12-19T18:17:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.705+0000 7fee43fff640 1 -- 172.21.15.26:0/1498253441 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee540afc20 con 0x7fee5c159090 2023-12-19T18:17:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.705+0000 7fee61c8c640 1 -- 172.21.15.26:0/1498253441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c0c7590 con 0x7fee5c159090 2023-12-19T18:17:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.705+0000 7fee61c8c640 1 -- 172.21.15.26:0/1498253441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c1f5db0 con 0x7fee5c159090 2023-12-19T18:17:19.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.705+0000 7fee61c8c640 1 -- 172.21.15.26:0/1498253441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5c074270 con 0x7fee5c159090 2023-12-19T18:17:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.705+0000 7fee43fff640 1 -- 172.21.15.26:0/1498253441 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fee540b6070 con 0x7fee5c159090 2023-12-19T18:17:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.705+0000 7fee43fff640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee48065cd0 0x7fee48068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.705+0000 7fee43fff640 1 -- 172.21.15.26:0/1498253441 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6520+0+0 (secure 0 0 0) 0x7fee5411edf0 con 0x7fee5c159090 2023-12-19T18:17:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.705+0000 7fee5b7fe640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee48065cd0 0x7fee48068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.709+0000 7fee5b7fe640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee48065cd0 0x7fee48068190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fee4c005890 tx=0x7fee4c009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.714 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.709+0000 7fee43fff640 1 -- 172.21.15.26:0/1498253441 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fee540ec080 con 0x7fee5c159090 2023-12-19T18:17:19.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.869+0000 7fee61c8c640 1 -- 172.21.15.26:0/1498253441 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fee5c0cbe80 con 0x7fee5c159090 2023-12-19T18:17:19.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.873+0000 7fee43fff640 1 -- 172.21.15.26:0/1498253441 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v75) v1 ==== 115+0+0 (secure 0 0 0) 0x7fee540b40b0 con 0x7fee5c159090 2023-12-19T18:17:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.873+0000 7fee61c8c640 1 -- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee48065cd0 msgr2=0x7fee48068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.873+0000 7fee61c8c640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee48065cd0 0x7fee48068190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fee4c005890 tx=0x7fee4c009040 comp rx=0 tx=0).stop 2023-12-19T18:17:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.877+0000 7fee61c8c640 1 -- 172.21.15.26:0/1498253441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c159090 msgr2=0x7fee5c070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.877+0000 7fee61c8c640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c159090 0x7fee5c070360 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fee5401b710 tx=0x7fee5401b8a0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.877+0000 7fee61c8c640 1 -- 172.21.15.26:0/1498253441 shutdown_connections 2023-12-19T18:17:19.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.877+0000 7fee61c8c640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee48065cd0 0x7fee48068190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.877+0000 7fee61c8c640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee5c159090 0x7fee5c070360 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.877+0000 7fee61c8c640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee5c0ca820 0x7fee5c0c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.877+0000 7fee61c8c640 1 --2- 172.21.15.26:0/1498253441 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee5c06b850 0x7fee5c1631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.877+0000 7fee61c8c640 1 -- 172.21.15.26:0/1498253441 >> 172.21.15.26:0/1498253441 conn(0x7fee5c066160 msgr2=0x7fee5c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.877+0000 7fee61c8c640 1 -- 172.21.15.26:0/1498253441 shutdown_connections 2023-12-19T18:17:19.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:19.877+0000 7fee61c8c640 1 -- 172.21.15.26:0/1498253441 wait complete. 2023-12-19T18:17:19.899 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-12-19T18:17:20.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c51e2d640 1 Processor -- start 2023-12-19T18:17:20.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c51e2d640 1 -- start start 2023-12-19T18:17:20.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c51e2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2c4c154220 0x7f2c4c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c51e2d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2c4c06b3d0 0x7f2c4c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c51e2d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2c4c0cec40 0x7f2c4c15d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c51e2d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4c0cf0d0 con 0x7f2c4c0cec40 2023-12-19T18:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c51e2d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4c06bc30 con 0x7f2c4c06b3d0 2023-12-19T18:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c51e2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c0cc6e0 con 0x7f2c4c154220 2023-12-19T18:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c4bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2c4c0cec40 0x7f2c4c15d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c4bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2c4c0cec40 0x7f2c4c15d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50974/0 (socket says 172.21.15.26:50974) 2023-12-19T18:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c4bfff640 1 -- 172.21.15.26:0/540871662 learned_addr learned my addr 172.21.15.26:0/540871662 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c4affd640 1 --2- 172.21.15.26:0/540871662 >> v2:172.21.15.160:3300/0 conn(0x7f2c4c06b3d0 0x7f2c4c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c4b7fe640 1 --2- 172.21.15.26:0/540871662 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2c4c154220 0x7f2c4c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c4bfff640 1 -- 172.21.15.26:0/540871662 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2c4c154220 msgr2=0x7f2c4c15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c4bfff640 1 --2- 172.21.15.26:0/540871662 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2c4c154220 0x7f2c4c15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c4bfff640 1 -- 172.21.15.26:0/540871662 >> v2:172.21.15.160:3300/0 conn(0x7f2c4c06b3d0 msgr2=0x7f2c4c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c4bfff640 1 --2- 172.21.15.26:0/540871662 >> v2:172.21.15.160:3300/0 conn(0x7f2c4c06b3d0 0x7f2c4c0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.041+0000 7f2c4bfff640 1 -- 172.21.15.26:0/540871662 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c15afc0 con 0x7f2c4c0cec40 2023-12-19T18:17:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c4bfff640 1 --2- 172.21.15.26:0/540871662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c0cec40 0x7f2c4c15d5f0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0cc430 tx=0x7f2c4401c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c65905e0e6e66b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c4a7fc640 1 -- 172.21.15.26:0/540871662 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c440a5560 con 0x7f2c4c0cec40 2023-12-19T18:17:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c4a7fc640 1 -- 172.21.15.26:0/540871662 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c440a56c0 con 0x7f2c4c0cec40 2023-12-19T18:17:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c4a7fc640 1 -- 172.21.15.26:0/540871662 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c440a5a50 con 0x7f2c4c0cec40 2023-12-19T18:17:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c51e2d640 1 -- 172.21.15.26:0/540871662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c0cec40 msgr2=0x7f2c4c15d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c51e2d640 1 --2- 172.21.15.26:0/540871662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c0cec40 0x7f2c4c15d5f0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0cc430 tx=0x7f2c4401c6e0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c51e2d640 1 -- 172.21.15.26:0/540871662 shutdown_connections 2023-12-19T18:17:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c51e2d640 1 --2- 172.21.15.26:0/540871662 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c0cec40 0x7f2c4c15d5f0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c51e2d640 1 --2- 172.21.15.26:0/540871662 >> v2:172.21.15.160:3300/0 conn(0x7f2c4c06b3d0 0x7f2c4c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c51e2d640 1 --2- 172.21.15.26:0/540871662 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2c4c154220 0x7f2c4c15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c51e2d640 1 -- 172.21.15.26:0/540871662 >> 172.21.15.26:0/540871662 conn(0x7f2c4c0658e0 msgr2=0x7f2c4c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c51e2d640 1 -- 172.21.15.26:0/540871662 shutdown_connections 2023-12-19T18:17:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c51e2d640 1 -- 172.21.15.26:0/540871662 wait complete. 2023-12-19T18:17:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c51e2d640 1 Processor -- start 2023-12-19T18:17:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.045+0000 7f2c51e2d640 1 -- start start 2023-12-19T18:17:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c51e2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2c4c06b3d0 0x7f2c4c155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c51e2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c0cec40 0x7f2c4c14dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c51e2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c4c154220 0x7f2c4c15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c51e2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c0b2470 con 0x7f2c4c0cec40 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c51e2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c0b18f0 con 0x7f2c4c154220 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c51e2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c0b1bd0 con 0x7f2c4c06b3d0 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c4affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c0cec40 0x7f2c4c14dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c4affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c0cec40 0x7f2c4c14dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50984/0 (socket says 172.21.15.26:50984) 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c4affd640 1 -- 172.21.15.26:0/2357063620 learned_addr learned my addr 172.21.15.26:0/2357063620 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c4b7fe640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2c4c06b3d0 0x7f2c4c155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c4bfff640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c4c154220 0x7f2c4c15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c4affd640 1 -- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2c4c06b3d0 msgr2=0x7f2c4c155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c4affd640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2c4c06b3d0 0x7f2c4c155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c4affd640 1 -- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c4c154220 msgr2=0x7f2c4c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c4affd640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c4c154220 0x7f2c4c15cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c4affd640 1 -- 172.21.15.26:0/2357063620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c44009b30 con 0x7f2c4c0cec40 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c4b7fe640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2c4c06b3d0 0x7f2c4c155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c4affd640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c0cec40 0x7f2c4c14dec0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000e870 tx=0x7f2c4000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c33fff640 1 -- 172.21.15.26:0/2357063620 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4000ee60 con 0x7f2c4c0cec40 2023-12-19T18:17:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c33fff640 1 -- 172.21.15.26:0/2357063620 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4000c940 con 0x7f2c4c0cec40 2023-12-19T18:17:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c51e2d640 1 -- 172.21.15.26:0/2357063620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4c0b1eb0 con 0x7f2c4c0cec40 2023-12-19T18:17:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c33fff640 1 -- 172.21.15.26:0/2357063620 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4000ccf0 con 0x7f2c4c0cec40 2023-12-19T18:17:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c51e2d640 1 -- 172.21.15.26:0/2357063620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c14e400 con 0x7f2c4c0cec40 2023-12-19T18:17:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.049+0000 7f2c51e2d640 1 -- 172.21.15.26:0/2357063620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4c074270 con 0x7f2c4c0cec40 2023-12-19T18:17:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.053+0000 7f2c33fff640 1 -- 172.21.15.26:0/2357063620 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f2c40013470 con 0x7f2c4c0cec40 2023-12-19T18:17:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.053+0000 7f2c33fff640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2c38065cd0 0x7f2c38068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.053+0000 7f2c33fff640 1 -- 172.21.15.26:0/2357063620 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6520+0+0 (secure 0 0 0) 0x7f2c40083880 con 0x7f2c4c0cec40 2023-12-19T18:17:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.053+0000 7f2c4b7fe640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2c38065cd0 0x7f2c38068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.053+0000 7f2c4b7fe640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2c38065cd0 0x7f2c38068190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009f50 tx=0x7f2c3c009190 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.057+0000 7f2c33fff640 1 -- 172.21.15.26:0/2357063620 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2c40050c60 con 0x7f2c4c0cec40 2023-12-19T18:17:20.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.269+0000 7f2c51e2d640 1 -- 172.21.15.26:0/2357063620 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7f2c4c06be30 con 0x7f2c4c0cec40 2023-12-19T18:17:20.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.805+0000 7f2c33fff640 1 -- 172.21.15.26:0/2357063620 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3085) v1 ==== 186+0+0 (secure 0 0 0) 0x7f2c40054280 con 0x7f2c4c0cec40 2023-12-19T18:17:20.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.805+0000 7f2c51e2d640 1 -- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2c38065cd0 msgr2=0x7f2c38068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.805+0000 7f2c51e2d640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2c38065cd0 0x7f2c38068190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009f50 tx=0x7f2c3c009190 comp rx=0 tx=0).stop 2023-12-19T18:17:20.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.805+0000 7f2c51e2d640 1 -- 172.21.15.26:0/2357063620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c0cec40 msgr2=0x7f2c4c14dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.805+0000 7f2c51e2d640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c0cec40 0x7f2c4c14dec0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000e870 tx=0x7f2c4000ed30 comp rx=0 tx=0).stop 2023-12-19T18:17:20.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.805+0000 7f2c51e2d640 1 -- 172.21.15.26:0/2357063620 shutdown_connections 2023-12-19T18:17:20.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.805+0000 7f2c51e2d640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2c38065cd0 0x7f2c38068190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.805+0000 7f2c51e2d640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c4c154220 0x7f2c4c15cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.805+0000 7f2c51e2d640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c4c0cec40 0x7f2c4c14dec0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.805+0000 7f2c51e2d640 1 --2- 172.21.15.26:0/2357063620 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2c4c06b3d0 0x7f2c4c155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.805+0000 7f2c51e2d640 1 -- 172.21.15.26:0/2357063620 >> 172.21.15.26:0/2357063620 conn(0x7f2c4c0658e0 msgr2=0x7f2c4c065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.809+0000 7f2c51e2d640 1 -- 172.21.15.26:0/2357063620 shutdown_connections 2023-12-19T18:17:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.809+0000 7f2c51e2d640 1 -- 172.21.15.26:0/2357063620 wait complete. 2023-12-19T18:17:20.834 INFO:tasks.cephfs_test_runner:Test command succes with --fs argument for non-default fs ... ok 2023-12-19T18:17:20.834 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command fails without --fs argument in presence of multiple fs 2023-12-19T18:17:20.834 INFO:tasks.cephfs_test_runner:test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-12-19T18:17:20.835 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-12-19T18:17:20.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.977+0000 7f23249a6640 1 Processor -- start 2023-12-19T18:17:20.981 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.977+0000 7f23249a6640 1 -- start start 2023-12-19T18:17:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.977+0000 7f23249a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2320155030 0x7f232015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.977+0000 7f23249a6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f232006c970 con 0x7f232015f370 2023-12-19T18:17:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.977+0000 7f23249a6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f232015b1f0 con 0x7f23200ce740 2023-12-19T18:17:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.977+0000 7f23249a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232006c790 con 0x7f2320155030 2023-12-19T18:17:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.977+0000 7f2317fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f232015f370 0x7f232006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44234/0 (socket says 172.21.15.26:44234) 2023-12-19T18:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.977+0000 7f2317fff640 1 -- 172.21.15.26:0/1310304097 learned_addr learned my addr 172.21.15.26:0/1310304097 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.977+0000 7f231e59c640 1 --2- 172.21.15.26:0/1310304097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2320155030 0x7f232015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.981+0000 7f231dd9b640 1 -- 172.21.15.26:0/1310304097 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294135737 0 0) 0x7f232006c970 con 0x7f232015f370 2023-12-19T18:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.981+0000 7f231dd9b640 1 -- 172.21.15.26:0/1310304097 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f230c003530 con 0x7f232015f370 2023-12-19T18:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.981+0000 7f231dd9b640 1 -- 172.21.15.26:0/1310304097 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3834059664 0 0) 0x7f232015b1f0 con 0x7f23200ce740 2023-12-19T18:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.981+0000 7f231dd9b640 1 -- 172.21.15.26:0/1310304097 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f232006c970 con 0x7f23200ce740 2023-12-19T18:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.981+0000 7f231dd9b640 1 -- 172.21.15.26:0/1310304097 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3178711064 0 0) 0x7f230c003530 con 0x7f232015f370 2023-12-19T18:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.981+0000 7f231e59c640 1 -- 172.21.15.26:0/1310304097 >> v1:172.21.15.160:6789/0 conn(0x7f23200ce740 legacy=0x7f23200ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.981+0000 7f231e59c640 1 -- 172.21.15.26:0/1310304097 >> v1:172.21.15.26:6789/0 conn(0x7f232015f370 legacy=0x7f232006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.981+0000 7f231e59c640 1 -- 172.21.15.26:0/1310304097 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232015b4d0 con 0x7f2320155030 2023-12-19T18:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.981+0000 7f231e59c640 1 --2- 172.21.15.26:0/1310304097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2320155030 0x7f232015bcd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f23100097f0 tx=0x7f231001c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0d3f92a35f6794 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.981+0000 7f231dd9b640 1 -- 172.21.15.26:0/1310304097 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308001680 con 0x7f2320155030 2023-12-19T18:17:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.981+0000 7f231dd9b640 1 -- 172.21.15.26:0/1310304097 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23100a7050 con 0x7f2320155030 2023-12-19T18:17:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.981+0000 7f231dd9b640 1 -- 172.21.15.26:0/1310304097 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310004250 con 0x7f2320155030 2023-12-19T18:17:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 -- 172.21.15.26:0/1310304097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2320155030 msgr2=0x7f232015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 --2- 172.21.15.26:0/1310304097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2320155030 0x7f232015bcd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f23100097f0 tx=0x7f231001c3b0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 -- 172.21.15.26:0/1310304097 shutdown_connections 2023-12-19T18:17:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 --2- 172.21.15.26:0/1310304097 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2320155030 0x7f232015bcd0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 -- 172.21.15.26:0/1310304097 >> 172.21.15.26:0/1310304097 conn(0x7f23200658e0 msgr2=0x7f2320065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 -- 172.21.15.26:0/1310304097 shutdown_connections 2023-12-19T18:17:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 -- 172.21.15.26:0/1310304097 wait complete. 2023-12-19T18:17:20.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 Processor -- start 2023-12-19T18:17:20.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 -- start start 2023-12-19T18:17:20.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23200ce740 0x7f2320149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2320155030 0x7f2320149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f232015f370 0x7f23200b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23200ae880 con 0x7f23200ce740 2023-12-19T18:17:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232014d530 con 0x7f232015f370 2023-12-19T18:17:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f23249a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232014d810 con 0x7f2320155030 2023-12-19T18:17:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f231e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23200ce740 0x7f2320149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f231e59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23200ce740 0x7f2320149e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50996/0 (socket says 172.21.15.26:50996) 2023-12-19T18:17:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f231e59c640 1 -- 172.21.15.26:0/2357764852 learned_addr learned my addr 172.21.15.26:0/2357764852 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f231ed9d640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f232015f370 0x7f23200b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f2317fff640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2320155030 0x7f2320149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f231e59c640 1 -- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2320155030 msgr2=0x7f2320149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f231e59c640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2320155030 0x7f2320149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f231e59c640 1 -- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f232015f370 msgr2=0x7f23200b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f231e59c640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f232015f370 0x7f23200b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f231e59c640 1 -- 172.21.15.26:0/2357764852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2310009420 con 0x7f23200ce740 2023-12-19T18:17:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.985+0000 7f231ed9d640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f232015f370 0x7f23200b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.989+0000 7f231e59c640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23200ce740 0x7f2320149e20 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f23100ae040 tx=0x7f2310019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.989+0000 7f2316ffd640 1 -- 172.21.15.26:0/2357764852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310003af0 con 0x7f23200ce740 2023-12-19T18:17:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.989+0000 7f2316ffd640 1 -- 172.21.15.26:0/2357764852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2310003c50 con 0x7f23200ce740 2023-12-19T18:17:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.989+0000 7f2316ffd640 1 -- 172.21.15.26:0/2357764852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23100ab440 con 0x7f23200ce740 2023-12-19T18:17:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.989+0000 7f23249a6640 1 -- 172.21.15.26:0/2357764852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23200ae050 con 0x7f23200ce740 2023-12-19T18:17:20.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.989+0000 7f23249a6640 1 -- 172.21.15.26:0/2357764852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23200ae5a0 con 0x7f23200ce740 2023-12-19T18:17:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.989+0000 7f23249a6640 1 -- 172.21.15.26:0/2357764852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2320074270 con 0x7f23200ce740 2023-12-19T18:17:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.989+0000 7f2316ffd640 1 -- 172.21.15.26:0/2357764852 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f23100b2040 con 0x7f23200ce740 2023-12-19T18:17:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.989+0000 7f2316ffd640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2300065c70 0x7f2300068130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.989+0000 7f2316ffd640 1 -- 172.21.15.26:0/2357764852 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6520+0+0 (secure 0 0 0) 0x7f231011d860 con 0x7f23200ce740 2023-12-19T18:17:20.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.993+0000 7f2317fff640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2300065c70 0x7f2300068130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.993+0000 7f2317fff640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2300065c70 0x7f2300068130 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2320167a50 tx=0x7f2308009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:20.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:20.993+0000 7f2316ffd640 1 -- 172.21.15.26:0/2357764852 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f23100eab40 con 0x7f23200ce740 2023-12-19T18:17:21.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.205+0000 7f23249a6640 1 -- 172.21.15.26:0/2357764852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7f23200ccc30 con 0x7f23200ce740 2023-12-19T18:17:21.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.809+0000 7f2316ffd640 1 -- 172.21.15.26:0/2357764852 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3086) v1 ==== 190+0+0 (secure 0 0 0) 0x7f23100b0100 con 0x7f23200ce740 2023-12-19T18:17:21.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.809+0000 7f23249a6640 1 -- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2300065c70 msgr2=0x7f2300068130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.809+0000 7f23249a6640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2300065c70 0x7f2300068130 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2320167a50 tx=0x7f2308009040 comp rx=0 tx=0).stop 2023-12-19T18:17:21.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.809+0000 7f23249a6640 1 -- 172.21.15.26:0/2357764852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23200ce740 msgr2=0x7f2320149e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.809+0000 7f23249a6640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23200ce740 0x7f2320149e20 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f23100ae040 tx=0x7f2310019150 comp rx=0 tx=0).stop 2023-12-19T18:17:21.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.809+0000 7f23249a6640 1 -- 172.21.15.26:0/2357764852 shutdown_connections 2023-12-19T18:17:21.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.809+0000 7f23249a6640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2300065c70 0x7f2300068130 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.809+0000 7f23249a6640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f232015f370 0x7f23200b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.809+0000 7f23249a6640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2320155030 0x7f2320149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.809+0000 7f23249a6640 1 --2- 172.21.15.26:0/2357764852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23200ce740 0x7f2320149e20 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.809+0000 7f23249a6640 1 -- 172.21.15.26:0/2357764852 >> 172.21.15.26:0/2357764852 conn(0x7f23200658e0 msgr2=0x7f23200b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:21.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.809+0000 7f23249a6640 1 -- 172.21.15.26:0/2357764852 shutdown_connections 2023-12-19T18:17:21.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.809+0000 7f23249a6640 1 -- 172.21.15.26:0/2357764852 wait complete. 2023-12-19T18:17:21.837 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:17:21.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f223e5e9640 1 Processor -- start 2023-12-19T18:17:21.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f223e5e9640 1 -- start start 2023-12-19T18:17:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f223e5e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f223806f6c0 0x7f223815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f223e5e9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223806fb60 con 0x7f2238159c00 2023-12-19T18:17:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f223e5e9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f223806fe40 con 0x7f2238159820 2023-12-19T18:17:21.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f223e5e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223815e470 con 0x7f223806f6c0 2023-12-19T18:17:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f223cb86640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2238159c00 0x7f2238070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44250/0 (socket says 172.21.15.26:44250) 2023-12-19T18:17:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f223cb86640 1 -- 172.21.15.26:0/3146531916 learned_addr learned my addr 172.21.15.26:0/3146531916 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f22377fe640 1 --2- 172.21.15.26:0/3146531916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f223806f6c0 0x7f223815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f2236ffd640 1 -- 172.21.15.26:0/3146531916 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939810522 0 0) 0x7f223806fb60 con 0x7f2238159c00 2023-12-19T18:17:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f2236ffd640 1 -- 172.21.15.26:0/3146531916 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2220003530 con 0x7f2238159c00 2023-12-19T18:17:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f2236ffd640 1 -- 172.21.15.26:0/3146531916 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834501063 0 0) 0x7f223806fe40 con 0x7f2238159820 2023-12-19T18:17:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f2236ffd640 1 -- 172.21.15.26:0/3146531916 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f223806fb60 con 0x7f2238159820 2023-12-19T18:17:21.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f2236ffd640 1 -- 172.21.15.26:0/3146531916 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3261390481 0 0) 0x7f2220003530 con 0x7f2238159c00 2023-12-19T18:17:21.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f22377fe640 1 -- 172.21.15.26:0/3146531916 >> v1:172.21.15.160:6789/0 conn(0x7f2238159820 legacy=0x7f22380c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f22377fe640 1 -- 172.21.15.26:0/3146531916 >> v1:172.21.15.26:6789/0 conn(0x7f2238159c00 legacy=0x7f2238070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.985+0000 7f22377fe640 1 -- 172.21.15.26:0/3146531916 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2238070120 con 0x7f223806f6c0 2023-12-19T18:17:21.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.989+0000 7f22377fe640 1 --2- 172.21.15.26:0/3146531916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f223806f6c0 0x7f223815df30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f222c0097f0 tx=0x7f222c01c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f819de5ef448b516 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:21.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.989+0000 7f2236ffd640 1 -- 172.21.15.26:0/3146531916 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230001a00 con 0x7f223806f6c0 2023-12-19T18:17:21.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.989+0000 7f2236ffd640 1 -- 172.21.15.26:0/3146531916 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c0a7050 con 0x7f223806f6c0 2023-12-19T18:17:21.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.989+0000 7f2236ffd640 1 -- 172.21.15.26:0/3146531916 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c0ab620 con 0x7f223806f6c0 2023-12-19T18:17:21.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.989+0000 7f223e5e9640 1 -- 172.21.15.26:0/3146531916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f223806f6c0 msgr2=0x7f223815df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.989+0000 7f223e5e9640 1 --2- 172.21.15.26:0/3146531916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f223806f6c0 0x7f223815df30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f222c0097f0 tx=0x7f222c01c3b0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.989+0000 7f223e5e9640 1 -- 172.21.15.26:0/3146531916 shutdown_connections 2023-12-19T18:17:21.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.989+0000 7f223e5e9640 1 --2- 172.21.15.26:0/3146531916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f223806f6c0 0x7f223815df30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.989+0000 7f223e5e9640 1 -- 172.21.15.26:0/3146531916 >> 172.21.15.26:0/3146531916 conn(0x7f22380ce730 msgr2=0x7f22380ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:21.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.989+0000 7f223e5e9640 1 -- 172.21.15.26:0/3146531916 shutdown_connections 2023-12-19T18:17:21.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.989+0000 7f223e5e9640 1 -- 172.21.15.26:0/3146531916 wait complete. 2023-12-19T18:17:21.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f223e5e9640 1 Processor -- start 2023-12-19T18:17:21.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f223e5e9640 1 -- start start 2023-12-19T18:17:21.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f223e5e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f223806f6c0 0x7f22380b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f223e5e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2238159820 0x7f22380cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f223e5e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2238159c00 0x7f2238152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f223e5e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223814cb40 con 0x7f223806f6c0 2023-12-19T18:17:21.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f223e5e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223814f3f0 con 0x7f2238159820 2023-12-19T18:17:21.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f223e5e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223814f690 con 0x7f2238159c00 2023-12-19T18:17:21.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f2237fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f223806f6c0 0x7f22380b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f2237fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f223806f6c0 0x7f22380b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50998/0 (socket says 172.21.15.26:50998) 2023-12-19T18:17:21.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f2237fff640 1 -- 172.21.15.26:0/1829775026 learned_addr learned my addr 172.21.15.26:0/1829775026 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:21.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f22377fe640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2238159820 0x7f22380cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f223cb86640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2238159c00 0x7f2238152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f2237fff640 1 -- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2238159c00 msgr2=0x7f2238152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:22.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f2237fff640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2238159c00 0x7f2238152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f2237fff640 1 -- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2238159820 msgr2=0x7f22380cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:22.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f2237fff640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2238159820 0x7f22380cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f2237fff640 1 -- 172.21.15.26:0/1829775026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c009420 con 0x7f223806f6c0 2023-12-19T18:17:22.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f2237fff640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f223806f6c0 0x7f22380b1320 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f222800c490 tx=0x7f222800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:22.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f2234ff9640 1 -- 172.21.15.26:0/1829775026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22280052d0 con 0x7f223806f6c0 2023-12-19T18:17:22.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f2234ff9640 1 -- 172.21.15.26:0/1829775026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2228005430 con 0x7f223806f6c0 2023-12-19T18:17:22.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f223e5e9640 1 -- 172.21.15.26:0/1829775026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223806b880 con 0x7f223806f6c0 2023-12-19T18:17:22.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f2234ff9640 1 -- 172.21.15.26:0/1829775026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2228012870 con 0x7f223806f6c0 2023-12-19T18:17:22.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.993+0000 7f223e5e9640 1 -- 172.21.15.26:0/1829775026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223814fc50 con 0x7f223806f6c0 2023-12-19T18:17:22.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.997+0000 7f223e5e9640 1 -- 172.21.15.26:0/1829775026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2238074270 con 0x7f223806f6c0 2023-12-19T18:17:22.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.997+0000 7f2234ff9640 1 -- 172.21.15.26:0/1829775026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f22280129d0 con 0x7f223806f6c0 2023-12-19T18:17:22.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.997+0000 7f2234ff9640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2224065ca0 0x7f2224068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:22.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.997+0000 7f2234ff9640 1 -- 172.21.15.26:0/1829775026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6520+0+0 (secure 0 0 0) 0x7f2228083390 con 0x7f223806f6c0 2023-12-19T18:17:22.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.997+0000 7f22377fe640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2224065ca0 0x7f2224068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:22.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:21.997+0000 7f22377fe640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2224065ca0 0x7f2224068160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f222c0ad000 tx=0x7f222c008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:22.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.001+0000 7f2234ff9640 1 -- 172.21.15.26:0/1829775026 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2228050750 con 0x7f223806f6c0 2023-12-19T18:17:22.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.161+0000 7f223e5e9640 1 -- 172.21.15.26:0/1829775026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f22380b46b0 con 0x7f223806f6c0 2023-12-19T18:17:22.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.165+0000 7f2234ff9640 1 -- 172.21.15.26:0/1829775026 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v76)=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2228053d70 con 0x7f223806f6c0 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.165+0000 7f223e5e9640 1 -- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2224065ca0 msgr2=0x7f2224068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.165+0000 7f223e5e9640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2224065ca0 0x7f2224068160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f222c0ad000 tx=0x7f222c008040 comp rx=0 tx=0).stop 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.165+0000 7f223e5e9640 1 -- 172.21.15.26:0/1829775026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f223806f6c0 msgr2=0x7f22380b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.165+0000 7f223e5e9640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f223806f6c0 0x7f22380b1320 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f222800c490 tx=0x7f222800c950 comp rx=0 tx=0).stop 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.165+0000 7f223e5e9640 1 -- 172.21.15.26:0/1829775026 shutdown_connections 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.165+0000 7f223e5e9640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2224065ca0 0x7f2224068160 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.165+0000 7f223e5e9640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2238159c00 0x7f2238152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.165+0000 7f223e5e9640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2238159820 0x7f22380cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.165+0000 7f223e5e9640 1 --2- 172.21.15.26:0/1829775026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f223806f6c0 0x7f22380b1320 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.165+0000 7f223e5e9640 1 -- 172.21.15.26:0/1829775026 >> 172.21.15.26:0/1829775026 conn(0x7f22380ce730 msgr2=0x7f22381536d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:22.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.169+0000 7f223e5e9640 1 -- 172.21.15.26:0/1829775026 shutdown_connections 2023-12-19T18:17:22.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.169+0000 7f223e5e9640 1 -- 172.21.15.26:0/1829775026 wait complete. 2023-12-19T18:17:22.190 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:17:22.191 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:17:22.277 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:17:22.354 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:17:22.354 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:17:22.354 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:22.364 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:17:22.364 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:17:22.364 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:22.375 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:17:22.375 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:17:22.375 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:22.419 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:17:22.419 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:17:22.419 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:22.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:17:22.429 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:17:22.429 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:22.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8ee895640 1 Processor -- start 2023-12-19T18:17:22.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8ee895640 1 -- start start 2023-12-19T18:17:22.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8ee895640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff8e80d3300 0x7ff8e80d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:22.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8ee895640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e80d29d0 con 0x7ff8e815cd40 2023-12-19T18:17:22.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8ee895640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e815d680 con 0x7ff8e80ce040 2023-12-19T18:17:22.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8ee895640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e80ce4b0 con 0x7ff8e80d3300 2023-12-19T18:17:22.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff8e815cd40 0x7ff8e8160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44262/0 (socket says 172.21.15.26:44262) 2023-12-19T18:17:22.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e7fff640 1 -- 172.21.15.26:0/1234779404 learned_addr learned my addr 172.21.15.26:0/1234779404 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:22.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8ece32640 1 --2- 172.21.15.26:0/1234779404 >> v2:172.21.15.160:3300/0 conn(0x7ff8e80d3300 0x7ff8e80d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:22.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427821540 0 0) 0x7ff8d80012a0 con 0x7ff8e815cd40 2023-12-19T18:17:22.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8d0003530 con 0x7ff8e815cd40 2023-12-19T18:17:22.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1009077529 0 0) 0x7ff8d0003530 con 0x7ff8e815cd40 2023-12-19T18:17:22.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 >> v2:172.21.15.160:3300/0 conn(0x7ff8e80d3300 msgr2=0x7ff8e80d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:22.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 --2- 172.21.15.26:0/1234779404 >> v2:172.21.15.160:3300/0 conn(0x7ff8e80d3300 0x7ff8e80d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 >> v1:172.21.15.160:6789/0 conn(0x7ff8e80ce040 legacy=0x7ff8e80d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:22.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e8161750 con 0x7ff8e815cd40 2023-12-19T18:17:22.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7ff8d8001990 con 0x7ff8e815cd40 2023-12-19T18:17:22.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 >> v1:172.21.15.26:6789/0 conn(0x7ff8e815cd40 legacy=0x7ff8e8160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:22.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 --2- 172.21.15.26:0/1234779404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d0003980 0x7ff8d0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:22.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 --2- 172.21.15.26:0/1234779404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8d00047d0 0x7ff8d0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:22.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 --2- 172.21.15.26:0/1234779404 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8d0005190 0x7ff8d000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:22.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e80ce4b0 con 0x7ff8d0003980 2023-12-19T18:17:22.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d000c030 con 0x7ff8d00047d0 2023-12-19T18:17:22.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.601+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d000c1f0 con 0x7ff8d0005190 2023-12-19T18:17:22.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.605+0000 7ff8e7fff640 1 --2- 172.21.15.26:0/1234779404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d0003980 0x7ff8d0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:22.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.605+0000 7ff8e77fe640 1 --2- 172.21.15.26:0/1234779404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8d00047d0 0x7ff8d0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:22.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.605+0000 7ff8ece32640 1 --2- 172.21.15.26:0/1234779404 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8d0005190 0x7ff8d000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:22.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.605+0000 7ff8e7fff640 1 -- 172.21.15.26:0/1234779404 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8d0005190 msgr2=0x7ff8d000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:22.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.605+0000 7ff8e7fff640 1 --2- 172.21.15.26:0/1234779404 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8d0005190 0x7ff8d000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.605+0000 7ff8e7fff640 1 -- 172.21.15.26:0/1234779404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8d00047d0 msgr2=0x7ff8d0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:22.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.605+0000 7ff8e7fff640 1 --2- 172.21.15.26:0/1234779404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8d00047d0 0x7ff8d0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.605+0000 7ff8e7fff640 1 -- 172.21.15.26:0/1234779404 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d0003530 con 0x7ff8d0003980 2023-12-19T18:17:22.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.605+0000 7ff8ece32640 1 -- 172.21.15.26:0/1234779404 reap_dead start 2023-12-19T18:17:22.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.605+0000 7ff8e7fff640 1 --2- 172.21.15.26:0/1234779404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d0003980 0x7ff8d0003da0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c190 tx=0x7ff8d801cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80b322c0dec6267 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:22.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.605+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8007270 con 0x7ff8d0003980 2023-12-19T18:17:22.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.605+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d80073f0 con 0x7ff8d0003980 2023-12-19T18:17:22.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.605+0000 7ff8e6ffd640 1 -- 172.21.15.26:0/1234779404 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d80a5a30 con 0x7ff8d0003980 2023-12-19T18:17:22.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 -- 172.21.15.26:0/1234779404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d0003980 msgr2=0x7ff8d0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:22.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 --2- 172.21.15.26:0/1234779404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d0003980 0x7ff8d0003da0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7ff8d800c190 tx=0x7ff8d801cb70 comp rx=0 tx=0).stop 2023-12-19T18:17:22.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 -- 172.21.15.26:0/1234779404 shutdown_connections 2023-12-19T18:17:22.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 --2- 172.21.15.26:0/1234779404 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8d0003980 0x7ff8d0003da0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 -- 172.21.15.26:0/1234779404 >> 172.21.15.26:0/1234779404 conn(0x7ff8e806a120 msgr2=0x7ff8e806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:22.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 -- 172.21.15.26:0/1234779404 shutdown_connections 2023-12-19T18:17:22.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 -- 172.21.15.26:0/1234779404 wait complete. 2023-12-19T18:17:22.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 Processor -- start 2023-12-19T18:17:22.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 -- start start 2023-12-19T18:17:22.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8d0003980 0x7ff8e81f5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:22.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8e80caa80 0x7ff8e8158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:22.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e815ea90 0x7ff8e815ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:22.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8157ab0 con 0x7ff8e80caa80 2023-12-19T18:17:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e815f450 con 0x7ff8e815ea90 2023-12-19T18:17:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ee895640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e8157ed0 con 0x7ff8d0003980 2023-12-19T18:17:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8e77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8e80caa80 0x7ff8e8158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8e77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8e80caa80 0x7ff8e8158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51016/0 (socket says 172.21.15.26:51016) 2023-12-19T18:17:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8e77fe640 1 -- 172.21.15.26:0/188017696 learned_addr learned my addr 172.21.15.26:0/188017696 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8e7fff640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8d0003980 0x7ff8e81f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8ece32640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e815ea90 0x7ff8e815ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8e77fe640 1 -- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8d0003980 msgr2=0x7ff8e81f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8e77fe640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8d0003980 0x7ff8e81f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8e77fe640 1 -- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e815ea90 msgr2=0x7ff8e815ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8e77fe640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e815ea90 0x7ff8e815ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8e77fe640 1 -- 172.21.15.26:0/188017696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e8156e30 con 0x7ff8e80caa80 2023-12-19T18:17:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.609+0000 7ff8e7fff640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8d0003980 0x7ff8e81f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.613+0000 7ff8e77fe640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8e80caa80 0x7ff8e8158640 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00e4b0 tx=0x7ff8dc00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.613+0000 7ff8e4ff9640 1 -- 172.21.15.26:0/188017696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0052e0 con 0x7ff8e80caa80 2023-12-19T18:17:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.613+0000 7ff8e4ff9640 1 -- 172.21.15.26:0/188017696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc005440 con 0x7ff8e80caa80 2023-12-19T18:17:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.613+0000 7ff8ee895640 1 -- 172.21.15.26:0/188017696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e8157110 con 0x7ff8e80caa80 2023-12-19T18:17:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.613+0000 7ff8e4ff9640 1 -- 172.21.15.26:0/188017696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc003840 con 0x7ff8e80caa80 2023-12-19T18:17:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.613+0000 7ff8ee895640 1 -- 172.21.15.26:0/188017696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e81576d0 con 0x7ff8e80caa80 2023-12-19T18:17:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.613+0000 7ff8ee895640 1 -- 172.21.15.26:0/188017696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e8157890 con 0x7ff8e80caa80 2023-12-19T18:17:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.613+0000 7ff8e4ff9640 1 -- 172.21.15.26:0/188017696 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ff8dc0039a0 con 0x7ff8e80caa80 2023-12-19T18:17:22.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.613+0000 7ff8e4ff9640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff8d4065ca0 0x7ff8d4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:22.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.613+0000 7ff8e4ff9640 1 -- 172.21.15.26:0/188017696 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(224..224 src has 1..224) v4 ==== 6520+0+0 (secure 0 0 0) 0x7ff8dc0836a0 con 0x7ff8e80caa80 2023-12-19T18:17:22.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.617+0000 7ff8e7fff640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff8d4065ca0 0x7ff8d4068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:22.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.617+0000 7ff8e7fff640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff8d4065ca0 0x7ff8d4068160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80ab000 tx=0x7ff8d80a7070 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:22.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.617+0000 7ff8e4ff9640 1 -- 172.21.15.26:0/188017696 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff8dc0509c0 con 0x7ff8e80caa80 2023-12-19T18:17:22.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8ee895640 1 -- 172.21.15.26:0/188017696 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8e8074270 con 0x7ff8e80caa80 2023-12-19T18:17:22.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8e4ff9640 1 -- 172.21.15.26:0/188017696 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+3542 (secure 0 0 0) 0x7ff8dc053fe0 con 0x7ff8e80caa80 2023-12-19T18:17:22.813 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:22.813 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":106,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/3349262044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":3349262044},{"type":"v1","addr":"172.21.15.160:6836","nonce":3349262044}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10590,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3865658274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3865658274},{"type":"v1","addr":"172.21.15.26:6833","nonce":3865658274}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10681,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3645961317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3645961317},{"type":"v1","addr":"172.21.15.160:6837","nonce":3645961317}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":11262,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1792196808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1792196808},{"type":"v1","addr":"172.21.15.26:6835","nonce":1792196808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[]} 2023-12-19T18:17:22.813 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 106 2023-12-19T18:17:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8ee895640 1 -- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff8d4065ca0 msgr2=0x7ff8d4068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8ee895640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff8d4065ca0 0x7ff8d4068160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80ab000 tx=0x7ff8d80a7070 comp rx=0 tx=0).stop 2023-12-19T18:17:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8ee895640 1 -- 172.21.15.26:0/188017696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8e80caa80 msgr2=0x7ff8e8158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8ee895640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8e80caa80 0x7ff8e8158640 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00e4b0 tx=0x7ff8dc00e970 comp rx=0 tx=0).stop 2023-12-19T18:17:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8ee895640 1 -- 172.21.15.26:0/188017696 shutdown_connections 2023-12-19T18:17:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8ee895640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff8d4065ca0 0x7ff8d4068160 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8ee895640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8e815ea90 0x7ff8e815ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8ee895640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8e80caa80 0x7ff8e8158640 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8ee895640 1 --2- 172.21.15.26:0/188017696 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8d0003980 0x7ff8e81f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8ee895640 1 -- 172.21.15.26:0/188017696 >> 172.21.15.26:0/188017696 conn(0x7ff8e806a120 msgr2=0x7ff8e8066030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:22.815 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8ee895640 1 -- 172.21.15.26:0/188017696 shutdown_connections 2023-12-19T18:17:22.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:22.809+0000 7ff8ee895640 1 -- 172.21.15.26:0/188017696 wait complete. 2023-12-19T18:17:22.834 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:17:22.834 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:17:22.835 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:17:22.835 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:17:22.835 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:17:22.835 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:17:22.835 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:17:22.835 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:17:22.835 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:17:22.836 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:17:22.836 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:17:22.836 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:17:22.836 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:17:22.836 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:17:22.836 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:17:22.836 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:17:22.849 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:17:22.850 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:17:22.850 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:17:22.851 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:17:22.852 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:17:22.852 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:17:22.853 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:17:22.853 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:17:22.898 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:17:22.898 DEBUG:teuthology.parallel:result is None 2023-12-19T18:17:22.900 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:17:22.900 DEBUG:teuthology.parallel:result is None 2023-12-19T18:17:22.902 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:17:22.902 DEBUG:teuthology.parallel:result is None 2023-12-19T18:17:22.904 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:17:22.904 DEBUG:teuthology.parallel:result is None 2023-12-19T18:17:22.905 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:17:22.988 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T18:17:22.988 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T18:17:22.996 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T18:17:22.997 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T18:17:23.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.053+0000 7f73e01bc640 1 Processor -- start 2023-12-19T18:17:23.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.053+0000 7f73e01bc640 1 -- start start 2023-12-19T18:17:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73e01bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73d815ab30 0x7f73d80cfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73e01bc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d80cf4d0 con 0x7f73d815af10 2023-12-19T18:17:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73e01bc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d80b32f0 con 0x7f73d80afd30 2023-12-19T18:17:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73e01bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80b6460 con 0x7f73d815ab30 2023-12-19T18:17:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73dd757640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f73d815af10 0x7f73d80b24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44282/0 (socket says 172.21.15.26:44282) 2023-12-19T18:17:23.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73dd757640 1 -- 172.21.15.26:0/674031016 learned_addr learned my addr 172.21.15.26:0/674031016 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73ddf58640 1 --2- 172.21.15.26:0/674031016 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73d815ab30 0x7f73d80cfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73dcf56640 1 -- 172.21.15.26:0/674031016 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075710559 0 0) 0x7f73d80cf4d0 con 0x7f73d815af10 2023-12-19T18:17:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73dcf56640 1 -- 172.21.15.26:0/674031016 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73c4003530 con 0x7f73d815af10 2023-12-19T18:17:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73dcf56640 1 -- 172.21.15.26:0/674031016 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416346959 0 0) 0x7f73d80b32f0 con 0x7f73d80afd30 2023-12-19T18:17:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73dcf56640 1 -- 172.21.15.26:0/674031016 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73d80cf4d0 con 0x7f73d80afd30 2023-12-19T18:17:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73ddf58640 1 -- 172.21.15.26:0/674031016 >> v1:172.21.15.160:6789/0 conn(0x7f73d80afd30 legacy=0x7f73d80b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73ddf58640 1 -- 172.21.15.26:0/674031016 >> v1:172.21.15.26:6789/0 conn(0x7f73d815af10 legacy=0x7f73d80b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.057+0000 7f73ddf58640 1 -- 172.21.15.26:0/674031016 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d80b69e0 con 0x7f73d815ab30 2023-12-19T18:17:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.061+0000 7f73ddf58640 1 --2- 172.21.15.26:0/674031016 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73d815ab30 0x7f73d80cfdd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f73d40097f0 tx=0x7f73d401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d370d2ac5a6e94bd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.061+0000 7f73dcf56640 1 -- 172.21.15.26:0/674031016 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d40ab620 con 0x7f73d815ab30 2023-12-19T18:17:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.061+0000 7f73dcf56640 1 -- 172.21.15.26:0/674031016 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d40ab780 con 0x7f73d815ab30 2023-12-19T18:17:23.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.061+0000 7f73dcf56640 1 -- 172.21.15.26:0/674031016 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d40abb10 con 0x7f73d815ab30 2023-12-19T18:17:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.061+0000 7f73e01bc640 1 -- 172.21.15.26:0/674031016 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73d815ab30 msgr2=0x7f73d80cfdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.061+0000 7f73e01bc640 1 --2- 172.21.15.26:0/674031016 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73d815ab30 0x7f73d80cfdd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f73d40097f0 tx=0x7f73d401c400 comp rx=0 tx=0).stop 2023-12-19T18:17:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.061+0000 7f73e01bc640 1 -- 172.21.15.26:0/674031016 shutdown_connections 2023-12-19T18:17:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.061+0000 7f73e01bc640 1 --2- 172.21.15.26:0/674031016 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73d815ab30 0x7f73d80cfdd0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.061+0000 7f73e01bc640 1 -- 172.21.15.26:0/674031016 >> 172.21.15.26:0/674031016 conn(0x7f73d806d510 msgr2=0x7f73d806d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.061+0000 7f73e01bc640 1 -- 172.21.15.26:0/674031016 shutdown_connections 2023-12-19T18:17:23.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.061+0000 7f73e01bc640 1 -- 172.21.15.26:0/674031016 wait complete. 2023-12-19T18:17:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.061+0000 7f73e01bc640 1 Processor -- start 2023-12-19T18:17:23.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.061+0000 7f73e01bc640 1 -- start start 2023-12-19T18:17:23.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73e01bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d80afd30 0x7f73d80ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73e01bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d815ab30 0x7f73d8152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73e01bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73d815af10 0x7f73d80b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73e01bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80aeb30 con 0x7f73d815ab30 2023-12-19T18:17:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73e01bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d806b670 con 0x7f73d80afd30 2023-12-19T18:17:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73e01bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d8155970 con 0x7f73d815af10 2023-12-19T18:17:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73dd757640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d815ab30 0x7f73d8152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73dd757640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d815ab30 0x7f73d8152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51048/0 (socket says 172.21.15.26:51048) 2023-12-19T18:17:23.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73dd757640 1 -- 172.21.15.26:0/3179335946 learned_addr learned my addr 172.21.15.26:0/3179335946 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73ddf58640 1 --2- 172.21.15.26:0/3179335946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d80afd30 0x7f73d80ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73dd757640 1 -- 172.21.15.26:0/3179335946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73d815af10 msgr2=0x7f73d80b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73dd757640 1 --2- 172.21.15.26:0/3179335946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73d815af10 0x7f73d80b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73dd757640 1 -- 172.21.15.26:0/3179335946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d80afd30 msgr2=0x7f73d80ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73dd757640 1 --2- 172.21.15.26:0/3179335946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d80afd30 0x7f73d80ae5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73dd757640 1 -- 172.21.15.26:0/3179335946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d4009420 con 0x7f73d815ab30 2023-12-19T18:17:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73dd757640 1 --2- 172.21.15.26:0/3179335946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d815ab30 0x7f73d8152cc0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f73c800c4d0 tx=0x7f73c800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73ce7fc640 1 -- 172.21.15.26:0/3179335946 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8005cf0 con 0x7f73d815ab30 2023-12-19T18:17:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73ce7fc640 1 -- 172.21.15.26:0/3179335946 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c8005e50 con 0x7f73d815ab30 2023-12-19T18:17:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73e01bc640 1 -- 172.21.15.26:0/3179335946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d806b910 con 0x7f73d815ab30 2023-12-19T18:17:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73ce7fc640 1 -- 172.21.15.26:0/3179335946 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8007290 con 0x7f73d815ab30 2023-12-19T18:17:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73e01bc640 1 -- 172.21.15.26:0/3179335946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d8155c30 con 0x7f73d815ab30 2023-12-19T18:17:23.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.065+0000 7f73e01bc640 1 -- 172.21.15.26:0/3179335946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d8074270 con 0x7f73d815ab30 2023-12-19T18:17:23.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.069+0000 7f73ce7fc640 1 -- 172.21.15.26:0/3179335946 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f73c80073f0 con 0x7f73d815ab30 2023-12-19T18:17:23.071 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.069+0000 7f73ce7fc640 1 --2- 172.21.15.26:0/3179335946 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f73bc065ca0 0x7f73bc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.069+0000 7f73ce7fc640 1 -- 172.21.15.26:0/3179335946 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 6520+0+0 (secure 0 0 0) 0x7f73c8082e50 con 0x7f73d815ab30 2023-12-19T18:17:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.069+0000 7f73ddf58640 1 --2- 172.21.15.26:0/3179335946 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f73bc065ca0 0x7f73bc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:23.072 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.069+0000 7f73ddf58640 1 --2- 172.21.15.26:0/3179335946 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f73bc065ca0 0x7f73bc068160 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f73d40af000 tx=0x7f73d4008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:23.075 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.073+0000 7f73ce7fc640 1 -- 172.21.15.26:0/3179335946 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f73c8050210 con 0x7f73d815ab30 2023-12-19T18:17:23.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:23.241+0000 7f73e01bc640 1 -- 172.21.15.26:0/3179335946 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f73d80b3bd0 con 0x7f73d815ab30 2023-12-19T18:17:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.025+0000 7f73ce7fc640 1 -- 172.21.15.26:0/3179335946 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v229) v1 ==== 97+0+0 (secure 0 0 0) 0x7f73c8017080 con 0x7f73d815ab30 2023-12-19T18:17:24.031 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2023-12-19T18:17:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.029+0000 7f73e01bc640 1 -- 172.21.15.26:0/3179335946 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f73bc065ca0 msgr2=0x7f73bc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.029+0000 7f73e01bc640 1 --2- 172.21.15.26:0/3179335946 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f73bc065ca0 0x7f73bc068160 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f73d40af000 tx=0x7f73d4008040 comp rx=0 tx=0).stop 2023-12-19T18:17:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.029+0000 7f73e01bc640 1 -- 172.21.15.26:0/3179335946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d815ab30 msgr2=0x7f73d8152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.029+0000 7f73e01bc640 1 --2- 172.21.15.26:0/3179335946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d815ab30 0x7f73d8152cc0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f73c800c4d0 tx=0x7f73c800c990 comp rx=0 tx=0).stop 2023-12-19T18:17:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.029+0000 7f73e01bc640 1 -- 172.21.15.26:0/3179335946 shutdown_connections 2023-12-19T18:17:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.029+0000 7f73e01bc640 1 --2- 172.21.15.26:0/3179335946 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f73bc065ca0 0x7f73bc068160 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.029+0000 7f73e01bc640 1 --2- 172.21.15.26:0/3179335946 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73d815af10 0x7f73d80b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.029+0000 7f73e01bc640 1 --2- 172.21.15.26:0/3179335946 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73d815ab30 0x7f73d8152cc0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.029+0000 7f73e01bc640 1 --2- 172.21.15.26:0/3179335946 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73d80afd30 0x7f73d80ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.029+0000 7f73e01bc640 1 -- 172.21.15.26:0/3179335946 >> 172.21.15.26:0/3179335946 conn(0x7f73d806d510 msgr2=0x7f73d8154e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:24.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.029+0000 7f73e01bc640 1 -- 172.21.15.26:0/3179335946 shutdown_connections 2023-12-19T18:17:24.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.029+0000 7f73e01bc640 1 -- 172.21.15.26:0/3179335946 wait complete. 2023-12-19T18:17:24.062 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:17:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.205+0000 7f61171ce640 1 Processor -- start 2023-12-19T18:17:24.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.205+0000 7f61171ce640 1 -- start start 2023-12-19T18:17:24.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.209+0000 7f61171ce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f611015c7f0 0x7f611006c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.209+0000 7f61171ce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f61100adb80 0x7f61100cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.209+0000 7f61171ce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611006ced0 con 0x7f61100ccc20 2023-12-19T18:17:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.209+0000 7f61171ce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611006d770 con 0x7f611015c7f0 2023-12-19T18:17:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.209+0000 7f61171ce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611006d1b0 con 0x7f61100adb80 2023-12-19T18:17:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.209+0000 7f6114f6a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f611015c7f0 0x7f611006c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.209+0000 7f6114f6a640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f611015c7f0 0x7f611006c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51064/0 (socket says 172.21.15.26:51064) 2023-12-19T18:17:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.209+0000 7f6114f6a640 1 -- 172.21.15.26:0/564586983 learned_addr learned my addr 172.21.15.26:0/564586983 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.209+0000 7f6114f6a640 1 -- 172.21.15.26:0/564586983 >> v2:172.21.15.160:3300/0 conn(0x7f61100adb80 msgr2=0x7f61100cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.209+0000 7f6114f6a640 1 --2- 172.21.15.26:0/564586983 >> v2:172.21.15.160:3300/0 conn(0x7f61100adb80 0x7f61100cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.209+0000 7f6114f6a640 1 -- 172.21.15.26:0/564586983 >> v1:172.21.15.160:6789/0 conn(0x7f61100ccc20 legacy=0x7f61100b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:24.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.209+0000 7f6114f6a640 1 -- 172.21.15.26:0/564586983 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611006d490 con 0x7f611015c7f0 2023-12-19T18:17:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f6114f6a640 1 --2- 172.21.15.26:0/564586983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f611015c7f0 0x7f611006c7f0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f61100670d0 tx=0x7f610401d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b82e6474fbdde3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f610f7fe640 1 -- 172.21.15.26:0/564586983 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61100ae200 con 0x7f611015c7f0 2023-12-19T18:17:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f610f7fe640 1 -- 172.21.15.26:0/564586983 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61040ac670 con 0x7f611015c7f0 2023-12-19T18:17:24.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f610f7fe640 1 -- 172.21.15.26:0/564586983 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61040aca00 con 0x7f611015c7f0 2023-12-19T18:17:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f61171ce640 1 -- 172.21.15.26:0/564586983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f611015c7f0 msgr2=0x7f611006c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f61171ce640 1 --2- 172.21.15.26:0/564586983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f611015c7f0 0x7f611006c7f0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f61100670d0 tx=0x7f610401d5d0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f61171ce640 1 -- 172.21.15.26:0/564586983 shutdown_connections 2023-12-19T18:17:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f61171ce640 1 --2- 172.21.15.26:0/564586983 >> v2:172.21.15.160:3300/0 conn(0x7f61100adb80 0x7f61100cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f61171ce640 1 --2- 172.21.15.26:0/564586983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f611015c7f0 0x7f611006c7f0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f61171ce640 1 -- 172.21.15.26:0/564586983 >> 172.21.15.26:0/564586983 conn(0x7f61100658e0 msgr2=0x7f6110065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f61171ce640 1 -- 172.21.15.26:0/564586983 shutdown_connections 2023-12-19T18:17:24.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f61171ce640 1 -- 172.21.15.26:0/564586983 wait complete. 2023-12-19T18:17:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f61171ce640 1 Processor -- start 2023-12-19T18:17:24.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.213+0000 7f61171ce640 1 -- start start 2023-12-19T18:17:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f61171ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61100adb80 0x7f61100afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f61171ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61100ccc20 0x7f61100ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f61171ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f611015c7f0 0x7f611014d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f61171ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61100b00c0 con 0x7f61100ccc20 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f61171ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61100af6f0 con 0x7f611015c7f0 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f61171ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61100aeae0 con 0x7f61100adb80 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f610ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61100ccc20 0x7f61100ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f610ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61100ccc20 0x7f61100ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51078/0 (socket says 172.21.15.26:51078) 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f610ffff640 1 -- 172.21.15.26:0/3095020927 learned_addr learned my addr 172.21.15.26:0/3095020927 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f6114f6a640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61100adb80 0x7f61100afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f611576b640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f611015c7f0 0x7f611014d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f610ffff640 1 -- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61100adb80 msgr2=0x7f61100afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f610ffff640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61100adb80 0x7f61100afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f610ffff640 1 -- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f611015c7f0 msgr2=0x7f611014d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f610ffff640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f611015c7f0 0x7f611014d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f610ffff640 1 -- 172.21.15.26:0/3095020927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61040094d0 con 0x7f61100ccc20 2023-12-19T18:17:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f610ffff640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61100ccc20 0x7f61100ae200 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00b550 tx=0x7f60fc00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f610d7fa640 1 -- 172.21.15.26:0/3095020927 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc010870 con 0x7f61100ccc20 2023-12-19T18:17:24.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f610d7fa640 1 -- 172.21.15.26:0/3095020927 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60fc0109f0 con 0x7f61100ccc20 2023-12-19T18:17:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f610d7fa640 1 -- 172.21.15.26:0/3095020927 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc010da0 con 0x7f61100ccc20 2023-12-19T18:17:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f61171ce640 1 -- 172.21.15.26:0/3095020927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61100aedc0 con 0x7f61100ccc20 2023-12-19T18:17:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f61171ce640 1 -- 172.21.15.26:0/3095020927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61100af380 con 0x7f61100ccc20 2023-12-19T18:17:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.217+0000 7f61171ce640 1 -- 172.21.15.26:0/3095020927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6110074270 con 0x7f61100ccc20 2023-12-19T18:17:24.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.221+0000 7f610d7fa640 1 -- 172.21.15.26:0/3095020927 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f60fc00f3c0 con 0x7f61100ccc20 2023-12-19T18:17:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.221+0000 7f610d7fa640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f60f4065ca0 0x7f60f4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.221+0000 7f610d7fa640 1 -- 172.21.15.26:0/3095020927 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f60fc083550 con 0x7f61100ccc20 2023-12-19T18:17:24.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.221+0000 7f6114f6a640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f60f4065ca0 0x7f60f4068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.225 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.221+0000 7f6114f6a640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f60f4065ca0 0x7f60f4068160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61040b0000 tx=0x7f6104008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:24.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.225+0000 7f610d7fa640 1 -- 172.21.15.26:0/3095020927 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f60fc050ca0 con 0x7f61100ccc20 2023-12-19T18:17:24.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f61171ce640 1 -- 172.21.15.26:0/3095020927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f61100af830 con 0x7f61100ccc20 2023-12-19T18:17:24.441 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f610d7fa640 1 -- 172.21.15.26:0/3095020927 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v52) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f60fc0542c0 con 0x7f61100ccc20 2023-12-19T18:17:24.441 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:24.441 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:17:24.441 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2023-12-19T18:17:24.441 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.441 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2023-12-19T18:17:24.441 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg==", 2023-12-19T18:17:24.441 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg==", 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:17:24.442 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg==", 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.443 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw==", 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig==", 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:24.444 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:17:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2023-12-19T18:17:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA==", 2023-12-19T18:17:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:17:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.445 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ==", 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ==", 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:24.446 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A==", 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.447 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ==", 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg==", 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:17:24.448 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2023-12-19T18:17:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg==", 2023-12-19T18:17:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:17:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.449 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ==", 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ==", 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T18:17:24.450 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ==", 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.451 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2023-12-19T18:17:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ==", 2023-12-19T18:17:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:17:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T18:17:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T18:17:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T18:17:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.452 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2023-12-19T18:17:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlT5ynDxAA7Sjbdwrc1RyAXJboysB54Q==", 2023-12-19T18:17:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T18:17:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:17:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:17:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2023-12-19T18:17:24.453 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA==", 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg==", 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:17:24.454 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.455 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:24.455 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-12-19T18:17:24.455 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:17:24.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f61171ce640 1 -- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f60f4065ca0 msgr2=0x7f60f4068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f61171ce640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f60f4065ca0 0x7f60f4068160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61040b0000 tx=0x7f6104008040 comp rx=0 tx=0).stop 2023-12-19T18:17:24.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f61171ce640 1 -- 172.21.15.26:0/3095020927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61100ccc20 msgr2=0x7f61100ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f61171ce640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61100ccc20 0x7f61100ae200 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f60fc00b550 tx=0x7f60fc00ba10 comp rx=0 tx=0).stop 2023-12-19T18:17:24.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f61171ce640 1 -- 172.21.15.26:0/3095020927 shutdown_connections 2023-12-19T18:17:24.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f61171ce640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f60f4065ca0 0x7f60f4068160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f61171ce640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f611015c7f0 0x7f611014d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f61171ce640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61100ccc20 0x7f61100ae200 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f61171ce640 1 --2- 172.21.15.26:0/3095020927 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f61100adb80 0x7f61100afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f61171ce640 1 -- 172.21.15.26:0/3095020927 >> 172.21.15.26:0/3095020927 conn(0x7f61100658e0 msgr2=0x7f61100ac850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f61171ce640 1 -- 172.21.15.26:0/3095020927 shutdown_connections 2023-12-19T18:17:24.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.437+0000 7f61171ce640 1 -- 172.21.15.26:0/3095020927 wait complete. 2023-12-19T18:17:24.463 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:24.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.601+0000 7f03b0c66640 1 Processor -- start 2023-12-19T18:17:24.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.601+0000 7f03b0c66640 1 -- start start 2023-12-19T18:17:24.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03b0c66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f03ac154220 0x7f03ac15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03b0c66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03ac06b3d0 0x7f03ac0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03b0c66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03ac0cec40 0x7f03ac15d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03b0c66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ac0cf0d0 con 0x7f03ac06b3d0 2023-12-19T18:17:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03b0c66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ac06bc30 con 0x7f03ac154220 2023-12-19T18:17:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03b0c66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0cc6e0 con 0x7f03ac0cec40 2023-12-19T18:17:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03a3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03ac06b3d0 0x7f03ac0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03a3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03ac06b3d0 0x7f03ac0ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51092/0 (socket says 172.21.15.26:51092) 2023-12-19T18:17:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03a3fff640 1 -- 172.21.15.26:0/3356460710 learned_addr learned my addr 172.21.15.26:0/3356460710 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03aa59c640 1 --2- 172.21.15.26:0/3356460710 >> v2:172.21.15.160:3300/0 conn(0x7f03ac154220 0x7f03ac15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03aad9d640 1 --2- 172.21.15.26:0/3356460710 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03ac0cec40 0x7f03ac15d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03a3fff640 1 -- 172.21.15.26:0/3356460710 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03ac0cec40 msgr2=0x7f03ac15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03a3fff640 1 --2- 172.21.15.26:0/3356460710 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03ac0cec40 0x7f03ac15d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03a3fff640 1 -- 172.21.15.26:0/3356460710 >> v2:172.21.15.160:3300/0 conn(0x7f03ac154220 msgr2=0x7f03ac15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:24.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03a3fff640 1 --2- 172.21.15.26:0/3356460710 >> v2:172.21.15.160:3300/0 conn(0x7f03ac154220 0x7f03ac15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03a3fff640 1 -- 172.21.15.26:0/3356460710 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac15afc0 con 0x7f03ac06b3d0 2023-12-19T18:17:24.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.605+0000 7f03aad9d640 1 --2- 172.21.15.26:0/3356460710 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03ac0cec40 0x7f03ac15d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:24.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03a3fff640 1 --2- 172.21.15.26:0/3356460710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ac06b3d0 0x7f03ac0ce700 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0667b0 tx=0x7f039401c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f7fa115ea6acc0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:24.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03a9d9b640 1 -- 172.21.15.26:0/3356460710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03940ab600 con 0x7f03ac06b3d0 2023-12-19T18:17:24.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03a9d9b640 1 -- 172.21.15.26:0/3356460710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03940a7050 con 0x7f03ac06b3d0 2023-12-19T18:17:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03a9d9b640 1 -- 172.21.15.26:0/3356460710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03940ab980 con 0x7f03ac06b3d0 2023-12-19T18:17:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03b0c66640 1 -- 172.21.15.26:0/3356460710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ac06b3d0 msgr2=0x7f03ac0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03b0c66640 1 --2- 172.21.15.26:0/3356460710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ac06b3d0 0x7f03ac0ce700 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0667b0 tx=0x7f039401c210 comp rx=0 tx=0).stop 2023-12-19T18:17:24.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03b0c66640 1 -- 172.21.15.26:0/3356460710 shutdown_connections 2023-12-19T18:17:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03b0c66640 1 --2- 172.21.15.26:0/3356460710 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03ac0cec40 0x7f03ac15d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03b0c66640 1 --2- 172.21.15.26:0/3356460710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ac06b3d0 0x7f03ac0ce700 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03b0c66640 1 --2- 172.21.15.26:0/3356460710 >> v2:172.21.15.160:3300/0 conn(0x7f03ac154220 0x7f03ac15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03b0c66640 1 -- 172.21.15.26:0/3356460710 >> 172.21.15.26:0/3356460710 conn(0x7f03ac0658e0 msgr2=0x7f03ac065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:24.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03b0c66640 1 -- 172.21.15.26:0/3356460710 shutdown_connections 2023-12-19T18:17:24.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03b0c66640 1 -- 172.21.15.26:0/3356460710 wait complete. 2023-12-19T18:17:24.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03b0c66640 1 Processor -- start 2023-12-19T18:17:24.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.609+0000 7f03b0c66640 1 -- start start 2023-12-19T18:17:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03b0c66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03ac06b3d0 0x7f03ac14b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03b0c66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03ac0cec40 0x7f03ac0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03b0c66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ac154220 0x7f03ac0aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03b0c66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac150470 con 0x7f03ac154220 2023-12-19T18:17:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03b0c66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac14dea0 con 0x7f03ac06b3d0 2023-12-19T18:17:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03b0c66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac150680 con 0x7f03ac0cec40 2023-12-19T18:17:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ac154220 0x7f03ac0aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03aad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ac154220 0x7f03ac0aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51098/0 (socket says 172.21.15.26:51098) 2023-12-19T18:17:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03aad9d640 1 -- 172.21.15.26:0/2463713754 learned_addr learned my addr 172.21.15.26:0/2463713754 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03a3fff640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03ac0cec40 0x7f03ac0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03aa59c640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03ac06b3d0 0x7f03ac14b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03aad9d640 1 -- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03ac0cec40 msgr2=0x7f03ac0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03aad9d640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03ac0cec40 0x7f03ac0ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03aad9d640 1 -- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03ac06b3d0 msgr2=0x7f03ac14b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03aad9d640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03ac06b3d0 0x7f03ac14b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03aad9d640 1 -- 172.21.15.26:0/2463713754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03940094d0 con 0x7f03ac154220 2023-12-19T18:17:24.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03aa59c640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03ac06b3d0 0x7f03ac14b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:24.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03aad9d640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ac154220 0x7f03ac0aed00 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009e10 tx=0x7f03a400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:24.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03a2ffd640 1 -- 172.21.15.26:0/2463713754 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4003ba0 con 0x7f03ac154220 2023-12-19T18:17:24.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03a2ffd640 1 -- 172.21.15.26:0/2463713754 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a4011040 con 0x7f03ac154220 2023-12-19T18:17:24.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03a2ffd640 1 -- 172.21.15.26:0/2463713754 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a400fe80 con 0x7f03ac154220 2023-12-19T18:17:24.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03b0c66640 1 -- 172.21.15.26:0/2463713754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac0af330 con 0x7f03ac154220 2023-12-19T18:17:24.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03b0c66640 1 -- 172.21.15.26:0/2463713754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac0af8f0 con 0x7f03ac154220 2023-12-19T18:17:24.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03b0c66640 1 -- 172.21.15.26:0/2463713754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac150840 con 0x7f03ac154220 2023-12-19T18:17:24.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.613+0000 7f03a2ffd640 1 -- 172.21.15.26:0/2463713754 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f03a4004030 con 0x7f03ac154220 2023-12-19T18:17:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.617+0000 7f03a2ffd640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f038c065ca0 0x7f038c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.617+0000 7f03a2ffd640 1 -- 172.21.15.26:0/2463713754 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f03a4085950 con 0x7f03ac154220 2023-12-19T18:17:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.617+0000 7f03aa59c640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f038c065ca0 0x7f038c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.617+0000 7f03aa59c640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f038c065ca0 0x7f038c068160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f039c009bb0 tx=0x7f039c008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:24.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.621+0000 7f03a2ffd640 1 -- 172.21.15.26:0/2463713754 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f03a4052f10 con 0x7f03ac154220 2023-12-19T18:17:24.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.777+0000 7f03b0c66640 1 -- 172.21.15.26:0/2463713754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03ac074270 con 0x7f03ac154220 2023-12-19T18:17:24.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.777+0000 7f03a2ffd640 1 -- 172.21.15.26:0/2463713754 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v229) v1 ==== 74+0+13549 (secure 0 0 0) 0x7f03a401c090 con 0x7f03ac154220 2023-12-19T18:17:24.783 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:24.783 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":229,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:24.026431+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":214,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.781+0000 7f03b0c66640 1 -- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f038c065ca0 msgr2=0x7f038c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.781+0000 7f03b0c66640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f038c065ca0 0x7f038c068160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f039c009bb0 tx=0x7f039c008040 comp rx=0 tx=0).stop 2023-12-19T18:17:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.781+0000 7f03b0c66640 1 -- 172.21.15.26:0/2463713754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ac154220 msgr2=0x7f03ac0aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.781+0000 7f03b0c66640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ac154220 0x7f03ac0aed00 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009e10 tx=0x7f03a400f4a0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.781+0000 7f03b0c66640 1 -- 172.21.15.26:0/2463713754 shutdown_connections 2023-12-19T18:17:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.781+0000 7f03b0c66640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f038c065ca0 0x7f038c068160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.781+0000 7f03b0c66640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03ac154220 0x7f03ac0aed00 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.781+0000 7f03b0c66640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03ac0cec40 0x7f03ac0ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.781+0000 7f03b0c66640 1 --2- 172.21.15.26:0/2463713754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03ac06b3d0 0x7f03ac14b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.781+0000 7f03b0c66640 1 -- 172.21.15.26:0/2463713754 >> 172.21.15.26:0/2463713754 conn(0x7f03ac0658e0 msgr2=0x7f03ac15a750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:24.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.781+0000 7f03b0c66640 1 -- 172.21.15.26:0/2463713754 shutdown_connections 2023-12-19T18:17:24.786 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.781+0000 7f03b0c66640 1 -- 172.21.15.26:0/2463713754 wait complete. 2023-12-19T18:17:24.804 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:17:24.804 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:17:24.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.949+0000 7fca3de66640 1 Processor -- start 2023-12-19T18:17:24.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.949+0000 7fca3de66640 1 -- start start 2023-12-19T18:17:24.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.949+0000 7fca3de66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fca3815a9c0 0x7fca380706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.949+0000 7fca3de66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fca3815ada0 0x7fca380b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.949+0000 7fca3de66640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca380b46c0 con 0x7fca3806ac80 2023-12-19T18:17:24.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.949+0000 7fca3de66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca380cc270 con 0x7fca3815ada0 2023-12-19T18:17:24.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.949+0000 7fca3de66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca38070d90 con 0x7fca3815a9c0 2023-12-19T18:17:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.949+0000 7fca377fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fca3815a9c0 0x7fca380706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.949+0000 7fca377fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fca3815a9c0 0x7fca380706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:60362/0 (socket says 172.21.15.26:60362) 2023-12-19T18:17:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.949+0000 7fca377fe640 1 -- 172.21.15.26:0/1958928231 learned_addr learned my addr 172.21.15.26:0/1958928231 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.949+0000 7fca36ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fca3815ada0 0x7fca380b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.949+0000 7fca37fff640 1 --1- 172.21.15.26:0/1958928231 >> v1:172.21.15.160:6789/0 conn(0x7fca3806ac80 0x7fca380b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:51082/0 (socket says 172.21.15.26:51082) 2023-12-19T18:17:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.953+0000 7fca377fe640 1 -- 172.21.15.26:0/1958928231 >> v2:172.21.15.160:3300/0 conn(0x7fca3815ada0 msgr2=0x7fca380b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.953+0000 7fca377fe640 1 --2- 172.21.15.26:0/1958928231 >> v2:172.21.15.160:3300/0 conn(0x7fca3815ada0 0x7fca380b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.953+0000 7fca377fe640 1 -- 172.21.15.26:0/1958928231 >> v1:172.21.15.160:6789/0 conn(0x7fca3806ac80 legacy=0x7fca380b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.953+0000 7fca377fe640 1 -- 172.21.15.26:0/1958928231 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca38071070 con 0x7fca3815a9c0 2023-12-19T18:17:24.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.953+0000 7fca36ffd640 1 --2- 172.21.15.26:0/1958928231 >> v2:172.21.15.160:3300/0 conn(0x7fca3815ada0 0x7fca380b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:24.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.953+0000 7fca377fe640 1 --2- 172.21.15.26:0/1958928231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fca3815a9c0 0x7fca380706d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fca28009870 tx=0x7fca2801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cac0ff635e9daf2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:24.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.953+0000 7fca367fc640 1 -- 172.21.15.26:0/1958928231 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca280ab680 con 0x7fca3815a9c0 2023-12-19T18:17:24.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca367fc640 1 -- 172.21.15.26:0/1958928231 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca280ab7e0 con 0x7fca3815a9c0 2023-12-19T18:17:24.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca367fc640 1 -- 172.21.15.26:0/1958928231 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca280abb70 con 0x7fca3815a9c0 2023-12-19T18:17:24.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 -- 172.21.15.26:0/1958928231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fca3815a9c0 msgr2=0x7fca380706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 --2- 172.21.15.26:0/1958928231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fca3815a9c0 0x7fca380706d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fca28009870 tx=0x7fca2801c480 comp rx=0 tx=0).stop 2023-12-19T18:17:24.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 -- 172.21.15.26:0/1958928231 shutdown_connections 2023-12-19T18:17:24.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 --2- 172.21.15.26:0/1958928231 >> v2:172.21.15.160:3300/0 conn(0x7fca3815ada0 0x7fca380b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 --2- 172.21.15.26:0/1958928231 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fca3815a9c0 0x7fca380706d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 -- 172.21.15.26:0/1958928231 >> 172.21.15.26:0/1958928231 conn(0x7fca380ce730 msgr2=0x7fca380ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:24.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 -- 172.21.15.26:0/1958928231 shutdown_connections 2023-12-19T18:17:24.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 -- 172.21.15.26:0/1958928231 wait complete. 2023-12-19T18:17:24.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 Processor -- start 2023-12-19T18:17:24.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 -- start start 2023-12-19T18:17:24.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3806ac80 0x7fca380b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca3815a9c0 0x7fca380af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fca3815ada0 0x7fca3814f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca380c64a0 con 0x7fca3806ac80 2023-12-19T18:17:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca380c5890 con 0x7fca3815a9c0 2023-12-19T18:17:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca3de66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca380c5b70 con 0x7fca3815ada0 2023-12-19T18:17:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3806ac80 0x7fca380b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca377fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3806ac80 0x7fca380b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51100/0 (socket says 172.21.15.26:51100) 2023-12-19T18:17:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca377fe640 1 -- 172.21.15.26:0/124850537 learned_addr learned my addr 172.21.15.26:0/124850537 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca36ffd640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca3815a9c0 0x7fca380af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.957+0000 7fca37fff640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fca3815ada0 0x7fca3814f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca377fe640 1 -- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fca3815ada0 msgr2=0x7fca3814f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca377fe640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fca3815ada0 0x7fca3814f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca377fe640 1 -- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca3815a9c0 msgr2=0x7fca380af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca377fe640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca3815a9c0 0x7fca380af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca377fe640 1 -- 172.21.15.26:0/124850537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c009420 con 0x7fca3806ac80 2023-12-19T18:17:24.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca377fe640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3806ac80 0x7fca380b31c0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fca2801a0b0 tx=0x7fca2801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:24.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca1ffff640 1 -- 172.21.15.26:0/124850537 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca280ab680 con 0x7fca3806ac80 2023-12-19T18:17:24.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca1ffff640 1 -- 172.21.15.26:0/124850537 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca280b4800 con 0x7fca3806ac80 2023-12-19T18:17:24.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca1ffff640 1 -- 172.21.15.26:0/124850537 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca280b4bb0 con 0x7fca3806ac80 2023-12-19T18:17:24.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca3de66640 1 -- 172.21.15.26:0/124850537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca280094d0 con 0x7fca3806ac80 2023-12-19T18:17:24.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca3de66640 1 -- 172.21.15.26:0/124850537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca380c6130 con 0x7fca3806ac80 2023-12-19T18:17:24.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca3de66640 1 -- 172.21.15.26:0/124850537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca38074270 con 0x7fca3806ac80 2023-12-19T18:17:24.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca1ffff640 1 -- 172.21.15.26:0/124850537 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fca280b4d10 con 0x7fca3806ac80 2023-12-19T18:17:24.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca1ffff640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fca24065ca0 0x7fca24068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.961+0000 7fca1ffff640 1 -- 172.21.15.26:0/124850537 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5617+0+0 (secure 0 0 0) 0x7fca2811dff0 con 0x7fca3806ac80 2023-12-19T18:17:24.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.965+0000 7fca36ffd640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fca24065ca0 0x7fca24068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.965+0000 7fca36ffd640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fca24065ca0 0x7fca24068160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca38156500 tx=0x7fca2c009190 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:24.970 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:24.965+0000 7fca1ffff640 1 -- 172.21.15.26:0/124850537 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fca280eb600 con 0x7fca3806ac80 2023-12-19T18:17:25.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.125+0000 7fca3de66640 1 -- 172.21.15.26:0/124850537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca380cd7b0 con 0x7fca3806ac80 2023-12-19T18:17:25.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.125+0000 7fca1ffff640 1 -- 172.21.15.26:0/124850537 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v229) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca280eec20 con 0x7fca3806ac80 2023-12-19T18:17:25.129 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:17:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.125+0000 7fca3de66640 1 -- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fca24065ca0 msgr2=0x7fca24068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.125+0000 7fca3de66640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fca24065ca0 0x7fca24068160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca38156500 tx=0x7fca2c009190 comp rx=0 tx=0).stop 2023-12-19T18:17:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.125+0000 7fca3de66640 1 -- 172.21.15.26:0/124850537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3806ac80 msgr2=0x7fca380b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.125+0000 7fca3de66640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3806ac80 0x7fca380b31c0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fca2801a0b0 tx=0x7fca2801acc0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.125+0000 7fca3de66640 1 -- 172.21.15.26:0/124850537 shutdown_connections 2023-12-19T18:17:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.125+0000 7fca3de66640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fca24065ca0 0x7fca24068160 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.125+0000 7fca3de66640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fca3815ada0 0x7fca3814f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.125+0000 7fca3de66640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fca3815a9c0 0x7fca380af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.125+0000 7fca3de66640 1 --2- 172.21.15.26:0/124850537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fca3806ac80 0x7fca380b31c0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.125+0000 7fca3de66640 1 -- 172.21.15.26:0/124850537 >> 172.21.15.26:0/124850537 conn(0x7fca380ce730 msgr2=0x7fca3806c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:25.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.129+0000 7fca3de66640 1 -- 172.21.15.26:0/124850537 shutdown_connections 2023-12-19T18:17:25.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.129+0000 7fca3de66640 1 -- 172.21.15.26:0/124850537 wait complete. 2023-12-19T18:17:25.151 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:17:25.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.289+0000 7fc68552c640 1 Processor -- start 2023-12-19T18:17:25.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.289+0000 7fc68552c640 1 -- start start 2023-12-19T18:17:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.293+0000 7fc68552c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc68015a7c0 0x7fc68015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.293+0000 7fc68552c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6800cf700 0x7fc6800cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.293+0000 7fc68552c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc68015e2f0 con 0x7fc68006f9d0 2023-12-19T18:17:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.293+0000 7fc68552c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6800c5b40 con 0x7fc68015a7c0 2023-12-19T18:17:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.293+0000 7fc68552c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6800c5e20 con 0x7fc6800cf700 2023-12-19T18:17:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.293+0000 7fc67f7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6800cf700 0x7fc6800cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.293+0000 7fc67f7fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6800cf700 0x7fc6800cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:60378/0 (socket says 172.21.15.26:60378) 2023-12-19T18:17:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.293+0000 7fc67f7fe640 1 -- 172.21.15.26:0/3852389412 learned_addr learned my addr 172.21.15.26:0/3852389412 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.293+0000 7fc67effd640 1 --2- 172.21.15.26:0/3852389412 >> v2:172.21.15.160:3300/0 conn(0x7fc68015a7c0 0x7fc68015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.293+0000 7fc67f7fe640 1 -- 172.21.15.26:0/3852389412 >> v2:172.21.15.160:3300/0 conn(0x7fc68015a7c0 msgr2=0x7fc68015ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.293+0000 7fc67f7fe640 1 --2- 172.21.15.26:0/3852389412 >> v2:172.21.15.160:3300/0 conn(0x7fc68015a7c0 0x7fc68015ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.293+0000 7fc67f7fe640 1 -- 172.21.15.26:0/3852389412 >> v1:172.21.15.160:6789/0 conn(0x7fc68006f9d0 legacy=0x7fc6800ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.293+0000 7fc67f7fe640 1 -- 172.21.15.26:0/3852389412 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6800ce770 con 0x7fc6800cf700 2023-12-19T18:17:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.297+0000 7fc67f7fe640 1 --2- 172.21.15.26:0/3852389412 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6800cf700 0x7fc6800cd240 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc678009e50 tx=0x7fc67801c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b5c6ba6d404a09 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.297+0000 7fc67e7fc640 1 -- 172.21.15.26:0/3852389412 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6780a56c0 con 0x7fc6800cf700 2023-12-19T18:17:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.297+0000 7fc67e7fc640 1 -- 172.21.15.26:0/3852389412 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6780a5820 con 0x7fc6800cf700 2023-12-19T18:17:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.297+0000 7fc67e7fc640 1 -- 172.21.15.26:0/3852389412 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6780a5bb0 con 0x7fc6800cf700 2023-12-19T18:17:25.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.297+0000 7fc68552c640 1 -- 172.21.15.26:0/3852389412 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6800cf700 msgr2=0x7fc6800cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.297+0000 7fc68552c640 1 --2- 172.21.15.26:0/3852389412 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6800cf700 0x7fc6800cd240 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc678009e50 tx=0x7fc67801c8a0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.297+0000 7fc68552c640 1 -- 172.21.15.26:0/3852389412 shutdown_connections 2023-12-19T18:17:25.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.297+0000 7fc68552c640 1 --2- 172.21.15.26:0/3852389412 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6800cf700 0x7fc6800cd240 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.297+0000 7fc68552c640 1 --2- 172.21.15.26:0/3852389412 >> v2:172.21.15.160:3300/0 conn(0x7fc68015a7c0 0x7fc68015ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.297+0000 7fc68552c640 1 -- 172.21.15.26:0/3852389412 >> 172.21.15.26:0/3852389412 conn(0x7fc68006d5d0 msgr2=0x7fc680069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:25.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.297+0000 7fc68552c640 1 -- 172.21.15.26:0/3852389412 shutdown_connections 2023-12-19T18:17:25.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.297+0000 7fc68552c640 1 -- 172.21.15.26:0/3852389412 wait complete. 2023-12-19T18:17:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.297+0000 7fc68552c640 1 Processor -- start 2023-12-19T18:17:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc68552c640 1 -- start start 2023-12-19T18:17:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc68552c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc68006f9d0 0x7fc6801526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc68552c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6800cf700 0x7fc68014e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc68552c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc68015a7c0 0x7fc6801f3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc68552c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc680150420 con 0x7fc6800cf700 2023-12-19T18:17:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc68552c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6801f3db0 con 0x7fc68006f9d0 2023-12-19T18:17:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc68552c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68014eff0 con 0x7fc68015a7c0 2023-12-19T18:17:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc677fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6800cf700 0x7fc68014e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc677fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6800cf700 0x7fc68014e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51112/0 (socket says 172.21.15.26:51112) 2023-12-19T18:17:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc677fff640 1 -- 172.21.15.26:0/1161605631 learned_addr learned my addr 172.21.15.26:0/1161605631 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc67effd640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc68006f9d0 0x7fc6801526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc67f7fe640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc68015a7c0 0x7fc6801f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc677fff640 1 -- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc68015a7c0 msgr2=0x7fc6801f3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc677fff640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc68015a7c0 0x7fc6801f3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc677fff640 1 -- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc68006f9d0 msgr2=0x7fc6801526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc677fff640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc68006f9d0 0x7fc6801526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc677fff640 1 -- 172.21.15.26:0/1161605631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc678009b30 con 0x7fc6800cf700 2023-12-19T18:17:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc67f7fe640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc68015a7c0 0x7fc6801f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc67effd640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc68006f9d0 0x7fc6801526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc677fff640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6800cf700 0x7fc68014e700 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fc668009e00 tx=0x7fc66800c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc6777fe640 1 -- 172.21.15.26:0/1161605631 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66800ccf0 con 0x7fc6800cf700 2023-12-19T18:17:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc6777fe640 1 -- 172.21.15.26:0/1161605631 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66800ce70 con 0x7fc6800cf700 2023-12-19T18:17:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc68552c640 1 -- 172.21.15.26:0/1161605631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc680152cd0 con 0x7fc6800cf700 2023-12-19T18:17:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc6777fe640 1 -- 172.21.15.26:0/1161605631 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668004240 con 0x7fc6800cf700 2023-12-19T18:17:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc68552c640 1 -- 172.21.15.26:0/1161605631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68014fd60 con 0x7fc6800cf700 2023-12-19T18:17:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.301+0000 7fc68552c640 1 -- 172.21.15.26:0/1161605631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc680150560 con 0x7fc6800cf700 2023-12-19T18:17:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.305+0000 7fc6777fe640 1 -- 172.21.15.26:0/1161605631 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fc6680087f0 con 0x7fc6800cf700 2023-12-19T18:17:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.305+0000 7fc6777fe640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc660065c70 0x7fc660068130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.305+0000 7fc6777fe640 1 -- 172.21.15.26:0/1161605631 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5617+0+0 (secure 0 0 0) 0x7fc668085850 con 0x7fc6800cf700 2023-12-19T18:17:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.305+0000 7fc67effd640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc660065c70 0x7fc660068130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:25.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.305+0000 7fc67effd640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc660065c70 0x7fc660068130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc6800cc260 tx=0x7fc670008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:25.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.309+0000 7fc6777fe640 1 -- 172.21.15.26:0/1161605631 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc680150560 con 0x7fc6800cf700 2023-12-19T18:17:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc68552c640 1 -- 172.21.15.26:0/1161605631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc680150560 con 0x7fc6800cf700 2023-12-19T18:17:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc6777fe640 1 -- 172.21.15.26:0/1161605631 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v229) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc680150560 con 0x7fc6800cf700 2023-12-19T18:17:25.472 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:25.472 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T18:17:25.472 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:25.472 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T18:17:25.472 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T18:17:25.472 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:25.472 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T18:17:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc68552c640 1 -- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc660065c70 msgr2=0x7fc660068130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc68552c640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc660065c70 0x7fc660068130 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc6800cc260 tx=0x7fc670008040 comp rx=0 tx=0).stop 2023-12-19T18:17:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc68552c640 1 -- 172.21.15.26:0/1161605631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6800cf700 msgr2=0x7fc68014e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc68552c640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6800cf700 0x7fc68014e700 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fc668009e00 tx=0x7fc66800c430 comp rx=0 tx=0).stop 2023-12-19T18:17:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc68552c640 1 -- 172.21.15.26:0/1161605631 shutdown_connections 2023-12-19T18:17:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc68552c640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc660065c70 0x7fc660068130 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc68552c640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc68015a7c0 0x7fc6801f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc68552c640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6800cf700 0x7fc68014e700 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc68552c640 1 --2- 172.21.15.26:0/1161605631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc68006f9d0 0x7fc6801526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc68552c640 1 -- 172.21.15.26:0/1161605631 >> 172.21.15.26:0/1161605631 conn(0x7fc68006d5d0 msgr2=0x7fc6801589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc68552c640 1 -- 172.21.15.26:0/1161605631 shutdown_connections 2023-12-19T18:17:25.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.469+0000 7fc68552c640 1 -- 172.21.15.26:0/1161605631 wait complete. 2023-12-19T18:17:25.495 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:17:25.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:17:25.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.637+0000 7f7110f39640 1 Processor -- start 2023-12-19T18:17:25.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.637+0000 7f7110f39640 1 -- start start 2023-12-19T18:17:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f7110f39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0b1590 0x7f710c06c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f7110f39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f710c0b3f90 0x7f710c0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f7110f39640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710c06cbf0 con 0x7f710c15aad0 2023-12-19T18:17:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f7110f39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f710c0ca8b0 con 0x7f710c0b3f90 2023-12-19T18:17:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f7110f39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710c06b6f0 con 0x7f710c0b1590 2023-12-19T18:17:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f710a59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f710c15aad0 0x7f710c070870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44292/0 (socket says 172.21.15.26:44292) 2023-12-19T18:17:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f710a59c640 1 -- 172.21.15.26:0/2286198049 learned_addr learned my addr 172.21.15.26:0/2286198049 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f710ad9d640 1 --2- 172.21.15.26:0/2286198049 >> v2:172.21.15.26:3300/0 conn(0x7f710c0b3f90 0x7f710c0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f7109d9b640 1 --2- 172.21.15.26:0/2286198049 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0b1590 0x7f710c06c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:25.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f710959a640 1 -- 172.21.15.26:0/2286198049 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419717698 0 0) 0x7f70fc001120 con 0x7f710c15aad0 2023-12-19T18:17:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f710ad9d640 1 -- 172.21.15.26:0/2286198049 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0b1590 msgr2=0x7f710c06c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f710ad9d640 1 --2- 172.21.15.26:0/2286198049 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0b1590 0x7f710c06c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f710ad9d640 1 -- 172.21.15.26:0/2286198049 >> v1:172.21.15.26:6789/0 conn(0x7f710c15aad0 legacy=0x7f710c070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.641+0000 7f710ad9d640 1 -- 172.21.15.26:0/2286198049 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710c06b9d0 con 0x7f710c0b3f90 2023-12-19T18:17:25.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f710ad9d640 1 --2- 172.21.15.26:0/2286198049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0b3f90 0x7f710c0cad40 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f7104009d90 tx=0x7f710401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16edf984677946f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f710959a640 1 -- 172.21.15.26:0/2286198049 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040a58f0 con 0x7f710c0b3f90 2023-12-19T18:17:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f710959a640 1 -- 172.21.15.26:0/2286198049 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71040a5a50 con 0x7f710c0b3f90 2023-12-19T18:17:25.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f710959a640 1 -- 172.21.15.26:0/2286198049 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040a5de0 con 0x7f710c0b3f90 2023-12-19T18:17:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f7110f39640 1 -- 172.21.15.26:0/2286198049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0b3f90 msgr2=0x7f710c0cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f7110f39640 1 --2- 172.21.15.26:0/2286198049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0b3f90 0x7f710c0cad40 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f7104009d90 tx=0x7f710401c920 comp rx=0 tx=0).stop 2023-12-19T18:17:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f7110f39640 1 -- 172.21.15.26:0/2286198049 shutdown_connections 2023-12-19T18:17:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f7110f39640 1 --2- 172.21.15.26:0/2286198049 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0b3f90 0x7f710c0cad40 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f7110f39640 1 --2- 172.21.15.26:0/2286198049 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0b1590 0x7f710c06c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f7110f39640 1 -- 172.21.15.26:0/2286198049 >> 172.21.15.26:0/2286198049 conn(0x7f710c0b0270 msgr2=0x7f710c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f7110f39640 1 -- 172.21.15.26:0/2286198049 shutdown_connections 2023-12-19T18:17:25.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f7110f39640 1 -- 172.21.15.26:0/2286198049 wait complete. 2023-12-19T18:17:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f7110f39640 1 Processor -- start 2023-12-19T18:17:25.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.645+0000 7f7110f39640 1 -- start start 2023-12-19T18:17:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f7110f39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0b1590 0x7f710c144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f7110f39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0b3f90 0x7f710c144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f7110f39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c15aad0 0x7f710c147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f7110f39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c1453e0 con 0x7f710c0b1590 2023-12-19T18:17:25.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f7110f39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c15bbc0 con 0x7f710c15aad0 2023-12-19T18:17:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f7110f39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710c15bea0 con 0x7f710c0b3f90 2023-12-19T18:17:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f710a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0b1590 0x7f710c144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f710a59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0b1590 0x7f710c144960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51126/0 (socket says 172.21.15.26:51126) 2023-12-19T18:17:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f710a59c640 1 -- 172.21.15.26:0/2867031955 learned_addr learned my addr 172.21.15.26:0/2867031955 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f7109d9b640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0b3f90 0x7f710c144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f710ad9d640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c15aad0 0x7f710c147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f710a59c640 1 -- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0b3f90 msgr2=0x7f710c144ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f710a59c640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0b3f90 0x7f710c144ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f710a59c640 1 -- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c15aad0 msgr2=0x7f710c147910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f710a59c640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c15aad0 0x7f710c147910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f710a59c640 1 -- 172.21.15.26:0/2867031955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7104009a70 con 0x7f710c0b1590 2023-12-19T18:17:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f7109d9b640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0b3f90 0x7f710c144ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f710a59c640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0b1590 0x7f710c144960 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00c780 tx=0x7f70fc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f70faffd640 1 -- 172.21.15.26:0/2867031955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0072d0 con 0x7f710c0b1590 2023-12-19T18:17:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f70faffd640 1 -- 172.21.15.26:0/2867031955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc007450 con 0x7f710c0b1590 2023-12-19T18:17:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f70faffd640 1 -- 172.21.15.26:0/2867031955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0047f0 con 0x7f710c0b1590 2023-12-19T18:17:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f7110f39640 1 -- 172.21.15.26:0/2867031955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710c15c180 con 0x7f710c0b1590 2023-12-19T18:17:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f7110f39640 1 -- 172.21.15.26:0/2867031955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710c15c740 con 0x7f710c0b1590 2023-12-19T18:17:25.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.649+0000 7f7110f39640 1 -- 172.21.15.26:0/2867031955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710c145620 con 0x7f710c0b1590 2023-12-19T18:17:25.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.653+0000 7f70faffd640 1 -- 172.21.15.26:0/2867031955 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f70fc004950 con 0x7f710c0b1590 2023-12-19T18:17:25.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.653+0000 7f70faffd640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f70ec065ca0 0x7f70ec068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:25.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.653+0000 7f70faffd640 1 -- 172.21.15.26:0/2867031955 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f70fc082760 con 0x7f710c0b1590 2023-12-19T18:17:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.653+0000 7f7109d9b640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f70ec065ca0 0x7f70ec068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.653+0000 7f7109d9b640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f70ec065ca0 0x7f70ec068160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f710c0b34f0 tx=0x7f7100008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.657+0000 7f70faffd640 1 -- 172.21.15.26:0/2867031955 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f70fc04fd70 con 0x7f710c0b1590 2023-12-19T18:17:25.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.813+0000 7f7110f39640 1 -- 172.21.15.26:0/2867031955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f710c06cbf0 con 0x7f710c0b1590 2023-12-19T18:17:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.917+0000 7f70faffd640 1 -- 172.21.15.26:0/2867031955 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v230) v1 ==== 138+0+0 (secure 0 0 0) 0x7f710c06cbf0 con 0x7f710c0b1590 2023-12-19T18:17:25.921 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T18:17:25.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.921+0000 7f7110f39640 1 -- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f70ec065ca0 msgr2=0x7f70ec068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.921+0000 7f7110f39640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f70ec065ca0 0x7f70ec068160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f710c0b34f0 tx=0x7f7100008040 comp rx=0 tx=0).stop 2023-12-19T18:17:25.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.921+0000 7f7110f39640 1 -- 172.21.15.26:0/2867031955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0b1590 msgr2=0x7f710c144960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.921+0000 7f7110f39640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0b1590 0x7f710c144960 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00c780 tx=0x7f70fc00cc40 comp rx=0 tx=0).stop 2023-12-19T18:17:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.921+0000 7f7110f39640 1 -- 172.21.15.26:0/2867031955 shutdown_connections 2023-12-19T18:17:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.921+0000 7f7110f39640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f70ec065ca0 0x7f70ec068160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.921+0000 7f7110f39640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f710c15aad0 0x7f710c147910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.921+0000 7f7110f39640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f710c0b3f90 0x7f710c144ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.921+0000 7f7110f39640 1 --2- 172.21.15.26:0/2867031955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f710c0b1590 0x7f710c144960 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.921+0000 7f7110f39640 1 -- 172.21.15.26:0/2867031955 >> 172.21.15.26:0/2867031955 conn(0x7f710c0b0270 msgr2=0x7f710c0b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.921+0000 7f7110f39640 1 -- 172.21.15.26:0/2867031955 shutdown_connections 2023-12-19T18:17:25.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:25.921+0000 7f7110f39640 1 -- 172.21.15.26:0/2867031955 wait complete. 2023-12-19T18:17:25.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:17:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.093+0000 7f1f98041640 1 Processor -- start 2023-12-19T18:17:26.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.093+0000 7f1f98041640 1 -- start start 2023-12-19T18:17:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f98041640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9006c740 0x7f1f9006f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f98041640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1f9006cb20 0x7f1f900ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f98041640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9006f790 con 0x7f1f900cad40 2023-12-19T18:17:26.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f98041640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9015d5a0 con 0x7f1f9006cb20 2023-12-19T18:17:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f98041640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9015d880 con 0x7f1f9006c740 2023-12-19T18:17:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f965de640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1f900cad40 0x7f1f900c9080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47620/0 (socket says 172.21.15.26:47620) 2023-12-19T18:17:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f965de640 1 -- 172.21.15.26:0/1924138711 learned_addr learned my addr 172.21.15.26:0/1924138711 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f95ddd640 1 --2- 172.21.15.26:0/1924138711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9006c740 0x7f1f9006f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f955dc640 1 --2- 172.21.15.26:0/1924138711 >> v2:172.21.15.160:3300/0 conn(0x7f1f9006cb20 0x7f1f900ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079102798 0 0) 0x7f1f9006f790 con 0x7f1f900cad40 2023-12-19T18:17:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f7c003530 con 0x7f1f900cad40 2023-12-19T18:17:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4226280175 0 0) 0x7f1f7c003530 con 0x7f1f900cad40 2023-12-19T18:17:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9006c740 msgr2=0x7f1f9006f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 --2- 172.21.15.26:0/1924138711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9006c740 0x7f1f9006f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 >> v2:172.21.15.160:3300/0 conn(0x7f1f9006cb20 msgr2=0x7f1f900ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 --2- 172.21.15.26:0/1924138711 >> v2:172.21.15.160:3300/0 conn(0x7f1f9006cb20 0x7f1f900ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9015db60 con 0x7f1f900cad40 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f1f88001a00 con 0x7f1f900cad40 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f955dc640 1 --2- 172.21.15.26:0/1924138711 >> v2:172.21.15.160:3300/0 conn(0x7f1f9006cb20 0x7f1f900ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 >> v1:172.21.15.26:6789/0 conn(0x7f1f900cad40 legacy=0x7f1f900c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 --2- 172.21.15.26:0/1924138711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7c003980 0x7f1f7c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 --2- 172.21.15.26:0/1924138711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f7c0047d0 0x7f1f7c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 --2- 172.21.15.26:0/1924138711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9006c740 0x7f1f7c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9015d5a0 con 0x7f1f7c003980 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9015d880 con 0x7f1f7c0047d0 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c005190 con 0x7f1f9006c740 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f95ddd640 1 --2- 172.21.15.26:0/1924138711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7c003980 0x7f1f7c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f955dc640 1 --2- 172.21.15.26:0/1924138711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f7c0047d0 0x7f1f7c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f965de640 1 --2- 172.21.15.26:0/1924138711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9006c740 0x7f1f7c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f95ddd640 1 -- 172.21.15.26:0/1924138711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9006c740 msgr2=0x7f1f7c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f95ddd640 1 --2- 172.21.15.26:0/1924138711 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9006c740 0x7f1f7c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f95ddd640 1 -- 172.21.15.26:0/1924138711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f7c0047d0 msgr2=0x7f1f7c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f95ddd640 1 --2- 172.21.15.26:0/1924138711 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f7c0047d0 0x7f1f7c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.097+0000 7f1f95ddd640 1 -- 172.21.15.26:0/1924138711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c003530 con 0x7f1f7c003980 2023-12-19T18:17:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.101+0000 7f1f95ddd640 1 --2- 172.21.15.26:0/1924138711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7c003980 0x7f1f7c003da0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009ae0 tx=0x7f1f8c01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912fcff9ef416cbc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.101+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0a6510 con 0x7f1f7c003980 2023-12-19T18:17:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.101+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c0a6670 con 0x7f1f7c003980 2023-12-19T18:17:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.101+0000 7f1f94ddb640 1 -- 172.21.15.26:0/1924138711 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0a6a20 con 0x7f1f7c003980 2023-12-19T18:17:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.101+0000 7f1f98041640 1 -- 172.21.15.26:0/1924138711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7c003980 msgr2=0x7f1f7c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.101+0000 7f1f98041640 1 --2- 172.21.15.26:0/1924138711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7c003980 0x7f1f7c003da0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009ae0 tx=0x7f1f8c01ca20 comp rx=0 tx=0).stop 2023-12-19T18:17:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.101+0000 7f1f965de640 1 -- 172.21.15.26:0/1924138711 reap_dead start 2023-12-19T18:17:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.101+0000 7f1f98041640 1 -- 172.21.15.26:0/1924138711 shutdown_connections 2023-12-19T18:17:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.101+0000 7f1f98041640 1 -- 172.21.15.26:0/1924138711 >> 172.21.15.26:0/1924138711 conn(0x7f1f900660d0 msgr2=0x7f1f900ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.101+0000 7f1f98041640 1 -- 172.21.15.26:0/1924138711 shutdown_connections 2023-12-19T18:17:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.101+0000 7f1f98041640 1 -- 172.21.15.26:0/1924138711 wait complete. 2023-12-19T18:17:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f98041640 1 Processor -- start 2023-12-19T18:17:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f98041640 1 -- start start 2023-12-19T18:17:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f98041640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f7c003980 0x7f1f90157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f98041640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f90151e20 0x7f1f90151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f98041640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9015eb00 0x7f1f9015efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f98041640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9015f4f0 con 0x7f1f9015eb00 2023-12-19T18:17:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f98041640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900c50a0 con 0x7f1f7c003980 2023-12-19T18:17:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f98041640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f901579c0 con 0x7f1f90151e20 2023-12-19T18:17:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f965de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9015eb00 0x7f1f9015efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f965de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9015eb00 0x7f1f9015efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47508/0 (socket says 172.21.15.26:47508) 2023-12-19T18:17:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f965de640 1 -- 172.21.15.26:0/1985380902 learned_addr learned my addr 172.21.15.26:0/1985380902 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f95ddd640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f7c003980 0x7f1f90157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f955dc640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f90151e20 0x7f1f90151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f965de640 1 -- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f90151e20 msgr2=0x7f1f90151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f965de640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f90151e20 0x7f1f90151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f965de640 1 -- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f7c003980 msgr2=0x7f1f90157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f965de640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f7c003980 0x7f1f90157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f965de640 1 -- 172.21.15.26:0/1985380902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f90151ad0 con 0x7f1f9015eb00 2023-12-19T18:17:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f955dc640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f90151e20 0x7f1f90151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f965de640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9015eb00 0x7f1f9015efb0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f1f900aec70 tx=0x7f1f88007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f867fc640 1 -- 172.21.15.26:0/1985380902 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8800abc0 con 0x7f1f9015eb00 2023-12-19T18:17:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f867fc640 1 -- 172.21.15.26:0/1985380902 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8800ad20 con 0x7f1f9015eb00 2023-12-19T18:17:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f98041640 1 -- 172.21.15.26:0/1985380902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f900c6d20 con 0x7f1f9015eb00 2023-12-19T18:17:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f867fc640 1 -- 172.21.15.26:0/1985380902 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88013630 con 0x7f1f9015eb00 2023-12-19T18:17:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f98041640 1 -- 172.21.15.26:0/1985380902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f900c72e0 con 0x7f1f9015eb00 2023-12-19T18:17:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.105+0000 7f1f98041640 1 -- 172.21.15.26:0/1985380902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f90074270 con 0x7f1f9015eb00 2023-12-19T18:17:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.109+0000 7f1f867fc640 1 -- 172.21.15.26:0/1985380902 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f1f88013790 con 0x7f1f9015eb00 2023-12-19T18:17:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.109+0000 7f1f867fc640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f74065cd0 0x7f1f74068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.109+0000 7f1f867fc640 1 -- 172.21.15.26:0/1985380902 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5987+0+0 (secure 0 0 0) 0x7f1f880844e0 con 0x7f1f9015eb00 2023-12-19T18:17:26.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.109+0000 7f1f95ddd640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f74065cd0 0x7f1f74068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.113+0000 7f1f95ddd640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f74065cd0 0x7f1f74068190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0af000 tx=0x7f1f8c0046d0 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.113+0000 7f1f867fc640 1 -- 172.21.15.26:0/1985380902 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1f88051a90 con 0x7f1f9015eb00 2023-12-19T18:17:26.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.273+0000 7f1f98041640 1 -- 172.21.15.26:0/1985380902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1f9015d310 con 0x7f1f9015eb00 2023-12-19T18:17:26.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.921+0000 7f1f867fc640 1 -- 172.21.15.26:0/1985380902 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v231) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1f880170d0 con 0x7f1f9015eb00 2023-12-19T18:17:26.925 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T18:17:26.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.929+0000 7f1f98041640 1 -- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f74065cd0 msgr2=0x7f1f74068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:26.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.929+0000 7f1f98041640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f74065cd0 0x7f1f74068190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0af000 tx=0x7f1f8c0046d0 comp rx=0 tx=0).stop 2023-12-19T18:17:26.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.929+0000 7f1f98041640 1 -- 172.21.15.26:0/1985380902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9015eb00 msgr2=0x7f1f9015efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:26.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.929+0000 7f1f98041640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9015eb00 0x7f1f9015efb0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f1f900aec70 tx=0x7f1f88007b00 comp rx=0 tx=0).stop 2023-12-19T18:17:26.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.933+0000 7f1f98041640 1 -- 172.21.15.26:0/1985380902 shutdown_connections 2023-12-19T18:17:26.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.933+0000 7f1f98041640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f74065cd0 0x7f1f74068190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:26.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.933+0000 7f1f98041640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f9015eb00 0x7f1f9015efb0 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:26.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.933+0000 7f1f98041640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f90151e20 0x7f1f90151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:26.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.933+0000 7f1f98041640 1 --2- 172.21.15.26:0/1985380902 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f7c003980 0x7f1f90157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:26.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.933+0000 7f1f98041640 1 -- 172.21.15.26:0/1985380902 >> 172.21.15.26:0/1985380902 conn(0x7f1f900660d0 msgr2=0x7f1f9006bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:26.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.933+0000 7f1f98041640 1 -- 172.21.15.26:0/1985380902 shutdown_connections 2023-12-19T18:17:26.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:26.937+0000 7f1f98041640 1 -- 172.21.15.26:0/1985380902 wait complete. 2023-12-19T18:17:26.968 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:17:27.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.109+0000 7fb200d7b640 1 Processor -- start 2023-12-19T18:17:27.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb200d7b640 1 -- start start 2023-12-19T18:17:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb200d7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1fc0706d0 0x7fb1fc0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb200d7b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1fc070b20 con 0x7fb1fc15ada0 2023-12-19T18:17:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb200d7b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1fc070d90 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb200d7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0cc270 con 0x7fb1fc0706d0 2023-12-19T18:17:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb1f3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb1fc15ada0 0x7fb1fc06ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47622/0 (socket says 172.21.15.26:47622) 2023-12-19T18:17:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb1f3fff640 1 -- 172.21.15.26:0/1419126210 learned_addr learned my addr 172.21.15.26:0/1419126210 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb1fad9d640 1 --2- 172.21.15.26:0/1419126210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1fc0706d0 0x7fb1fc0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb1f9d9b640 1 -- 172.21.15.26:0/1419126210 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411662534 0 0) 0x7fb1fc070b20 con 0x7fb1fc15ada0 2023-12-19T18:17:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb1f9d9b640 1 -- 172.21.15.26:0/1419126210 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1e8003530 con 0x7fb1fc15ada0 2023-12-19T18:17:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb1f9d9b640 1 -- 172.21.15.26:0/1419126210 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981306975 0 0) 0x7fb1fc070d90 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb1f9d9b640 1 -- 172.21.15.26:0/1419126210 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1fc070b20 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb1fad9d640 1 -- 172.21.15.26:0/1419126210 >> v1:172.21.15.160:6789/0 conn(0x7fb1fc15a9c0 legacy=0x7fb1fc0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb1fad9d640 1 -- 172.21.15.26:0/1419126210 >> v1:172.21.15.26:6789/0 conn(0x7fb1fc15ada0 legacy=0x7fb1fc06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.113+0000 7fb1fad9d640 1 -- 172.21.15.26:0/1419126210 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc071070 con 0x7fb1fc0706d0 2023-12-19T18:17:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.117+0000 7fb1fad9d640 1 --2- 172.21.15.26:0/1419126210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1fc0706d0 0x7fb1fc0b01b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009dc0 tx=0x7fb1f401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9b28c11cb4d22ff server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.117+0000 7fb1f9d9b640 1 -- 172.21.15.26:0/1419126210 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40a5880 con 0x7fb1fc0706d0 2023-12-19T18:17:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.117+0000 7fb1f9d9b640 1 -- 172.21.15.26:0/1419126210 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f40a59e0 con 0x7fb1fc0706d0 2023-12-19T18:17:27.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.117+0000 7fb1f9d9b640 1 -- 172.21.15.26:0/1419126210 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40a5d70 con 0x7fb1fc0706d0 2023-12-19T18:17:27.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.117+0000 7fb200d7b640 1 -- 172.21.15.26:0/1419126210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1fc0706d0 msgr2=0x7fb1fc0b01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.117+0000 7fb200d7b640 1 --2- 172.21.15.26:0/1419126210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1fc0706d0 0x7fb1fc0b01b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009dc0 tx=0x7fb1f401c8b0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.117+0000 7fb200d7b640 1 -- 172.21.15.26:0/1419126210 shutdown_connections 2023-12-19T18:17:27.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.117+0000 7fb200d7b640 1 --2- 172.21.15.26:0/1419126210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1fc0706d0 0x7fb1fc0b01b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.117+0000 7fb200d7b640 1 -- 172.21.15.26:0/1419126210 >> 172.21.15.26:0/1419126210 conn(0x7fb1fc0ce730 msgr2=0x7fb1fc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:27.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.117+0000 7fb200d7b640 1 -- 172.21.15.26:0/1419126210 shutdown_connections 2023-12-19T18:17:27.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.117+0000 7fb200d7b640 1 -- 172.21.15.26:0/1419126210 wait complete. 2023-12-19T18:17:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.117+0000 7fb200d7b640 1 Processor -- start 2023-12-19T18:17:27.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.117+0000 7fb200d7b640 1 -- start start 2023-12-19T18:17:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb200d7b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0706d0 0x7fb1fc0c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb200d7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1fc15a9c0 0x7fb1fc0b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb200d7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1fc15ada0 0x7fb1fc0af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb200d7b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0b3770 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb200d7b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0b31e0 con 0x7fb1fc0706d0 2023-12-19T18:17:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb200d7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0b3490 con 0x7fb1fc15ada0 2023-12-19T18:17:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1fc15a9c0 0x7fb1fc0b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1f3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1fc15a9c0 0x7fb1fc0b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47524/0 (socket says 172.21.15.26:47524) 2023-12-19T18:17:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1f3fff640 1 -- 172.21.15.26:0/1880444963 learned_addr learned my addr 172.21.15.26:0/1880444963 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1fad9d640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1fc15ada0 0x7fb1fc0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1fa59c640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0706d0 0x7fb1fc0c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1f3fff640 1 -- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1fc15ada0 msgr2=0x7fb1fc0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1f3fff640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1fc15ada0 0x7fb1fc0af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1f3fff640 1 -- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0706d0 msgr2=0x7fb1fc0c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1f3fff640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0706d0 0x7fb1fc0c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1f3fff640 1 -- 172.21.15.26:0/1880444963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f4009a70 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1f3fff640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1fc15a9c0 0x7fb1fc0b2ca0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400c4d0 tx=0x7fb1e400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1f2ffd640 1 -- 172.21.15.26:0/1880444963 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4005cf0 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1f2ffd640 1 -- 172.21.15.26:0/1880444963 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e4005e50 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb1f2ffd640 1 -- 172.21.15.26:0/1880444963 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40072b0 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb200d7b640 1 -- 172.21.15.26:0/1880444963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1fc153b40 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb200d7b640 1 -- 172.21.15.26:0/1880444963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1fc154100 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.121+0000 7fb200d7b640 1 -- 172.21.15.26:0/1880444963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1fc154320 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.125+0000 7fb1f2ffd640 1 -- 172.21.15.26:0/1880444963 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb1e4007410 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.125+0000 7fb1f2ffd640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb1dc065ca0 0x7fb1dc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.125+0000 7fb1f2ffd640 1 -- 172.21.15.26:0/1880444963 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 6369+0+0 (secure 0 0 0) 0x7fb1e4083280 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.125+0000 7fb1fa59c640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb1dc065ca0 0x7fb1dc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.125+0000 7fb1fa59c640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb1dc065ca0 0x7fb1dc068160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0011f0 tx=0x7fb1ec009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:27.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.129+0000 7fb1f2ffd640 1 -- 172.21.15.26:0/1880444963 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb1e40506d0 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.317+0000 7fb200d7b640 1 -- 172.21.15.26:0/1880444963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb1fc074270 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.329+0000 7fb1f2ffd640 1 -- 172.21.15.26:0/1880444963 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 28 and data pool 29 v108) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb1e4053cf0 con 0x7fb1fc15a9c0 2023-12-19T18:17:27.331 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 28 and data pool 29 2023-12-19T18:17:27.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.333+0000 7fb200d7b640 1 -- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb1dc065ca0 msgr2=0x7fb1dc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.333+0000 7fb200d7b640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb1dc065ca0 0x7fb1dc068160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0011f0 tx=0x7fb1ec009040 comp rx=0 tx=0).stop 2023-12-19T18:17:27.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.333+0000 7fb200d7b640 1 -- 172.21.15.26:0/1880444963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1fc15a9c0 msgr2=0x7fb1fc0b2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.333+0000 7fb200d7b640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1fc15a9c0 0x7fb1fc0b2ca0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400c4d0 tx=0x7fb1e400c990 comp rx=0 tx=0).stop 2023-12-19T18:17:27.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.333+0000 7fb200d7b640 1 -- 172.21.15.26:0/1880444963 shutdown_connections 2023-12-19T18:17:27.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.333+0000 7fb200d7b640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb1dc065ca0 0x7fb1dc068160 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec0011f0 tx=0x7fb1ec009040 comp rx=0 tx=0).stop 2023-12-19T18:17:27.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.333+0000 7fb200d7b640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb1fc15ada0 0x7fb1fc0af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.333+0000 7fb200d7b640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1fc15a9c0 0x7fb1fc0b2ca0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.333+0000 7fb200d7b640 1 --2- 172.21.15.26:0/1880444963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0706d0 0x7fb1fc0c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.333+0000 7fb200d7b640 1 -- 172.21.15.26:0/1880444963 >> 172.21.15.26:0/1880444963 conn(0x7fb1fc0ce730 msgr2=0x7fb1fc0ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.333+0000 7fb200d7b640 1 -- 172.21.15.26:0/1880444963 shutdown_connections 2023-12-19T18:17:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.333+0000 7fb200d7b640 1 -- 172.21.15.26:0/1880444963 wait complete. 2023-12-19T18:17:27.375 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:27.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.513+0000 7fc489752640 1 Processor -- start 2023-12-19T18:17:27.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.513+0000 7fc489752640 1 -- start start 2023-12-19T18:17:27.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc489752640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc484155030 0x7fc48415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc489752640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48406c970 con 0x7fc4840ce740 2023-12-19T18:17:27.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc489752640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48415b1f0 con 0x7fc48415f370 2023-12-19T18:17:27.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc489752640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48406c790 con 0x7fc484155030 2023-12-19T18:17:27.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4837fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc4840ce740 0x7fc4840ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47632/0 (socket says 172.21.15.26:47632) 2023-12-19T18:17:27.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4837fe640 1 -- 172.21.15.26:0/224782408 learned_addr learned my addr 172.21.15.26:0/224782408 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc482ffd640 1 --2- 172.21.15.26:0/224782408 >> v2:172.21.15.160:3300/0 conn(0x7fc484155030 0x7fc48415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946569066 0 0) 0x7fc48406c970 con 0x7fc4840ce740 2023-12-19T18:17:27.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc470003530 con 0x7fc4840ce740 2023-12-19T18:17:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387216556 0 0) 0x7fc48415b1f0 con 0x7fc48415f370 2023-12-19T18:17:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc48406c970 con 0x7fc48415f370 2023-12-19T18:17:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3071020813 0 0) 0x7fc470003530 con 0x7fc4840ce740 2023-12-19T18:17:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 >> v2:172.21.15.160:3300/0 conn(0x7fc484155030 msgr2=0x7fc48415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 --2- 172.21.15.26:0/224782408 >> v2:172.21.15.160:3300/0 conn(0x7fc484155030 0x7fc48415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 >> v1:172.21.15.160:6789/0 conn(0x7fc48415f370 legacy=0x7fc48406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48415b4d0 con 0x7fc4840ce740 2023-12-19T18:17:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fc47c001a00 con 0x7fc4840ce740 2023-12-19T18:17:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc482ffd640 1 --2- 172.21.15.26:0/224782408 >> v2:172.21.15.160:3300/0 conn(0x7fc484155030 0x7fc48415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 >> v1:172.21.15.26:6789/0 conn(0x7fc4840ce740 legacy=0x7fc4840ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 --2- 172.21.15.26:0/224782408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc470003b10 0x7fc470003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 --2- 172.21.15.26:0/224782408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4700047d0 0x7fc470004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 --2- 172.21.15.26:0/224782408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc470005190 0x7fc47000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48406c790 con 0x7fc4700047d0 2023-12-19T18:17:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47000c010 con 0x7fc470005190 2023-12-19T18:17:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47000c1d0 con 0x7fc470003b10 2023-12-19T18:17:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc47bfff640 1 --2- 172.21.15.26:0/224782408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4700047d0 0x7fc470004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc482ffd640 1 --2- 172.21.15.26:0/224782408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc470003b10 0x7fc470003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4837fe640 1 --2- 172.21.15.26:0/224782408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc470005190 0x7fc47000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc482ffd640 1 -- 172.21.15.26:0/224782408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc470005190 msgr2=0x7fc47000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc482ffd640 1 --2- 172.21.15.26:0/224782408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc470005190 0x7fc47000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc482ffd640 1 -- 172.21.15.26:0/224782408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4700047d0 msgr2=0x7fc470004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc482ffd640 1 --2- 172.21.15.26:0/224782408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc4700047d0 0x7fc470004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc482ffd640 1 -- 172.21.15.26:0/224782408 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc470003530 con 0x7fc470003b10 2023-12-19T18:17:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.517+0000 7fc4837fe640 1 -- 172.21.15.26:0/224782408 reap_dead start 2023-12-19T18:17:27.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.521+0000 7fc482ffd640 1 --2- 172.21.15.26:0/224782408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc470003b10 0x7fc470003f10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc474009cb0 tx=0x7fc47401cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b68f5d4a22309dd7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:27.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.521+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4740a58b0 con 0x7fc470003b10 2023-12-19T18:17:27.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.521+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4740a5a10 con 0x7fc470003b10 2023-12-19T18:17:27.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.521+0000 7fc4827fc640 1 -- 172.21.15.26:0/224782408 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4740a5e00 con 0x7fc470003b10 2023-12-19T18:17:27.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.521+0000 7fc489752640 1 -- 172.21.15.26:0/224782408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc470003b10 msgr2=0x7fc470003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.521+0000 7fc489752640 1 --2- 172.21.15.26:0/224782408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc470003b10 0x7fc470003f10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc474009cb0 tx=0x7fc47401cbf0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.521+0000 7fc489752640 1 -- 172.21.15.26:0/224782408 shutdown_connections 2023-12-19T18:17:27.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.521+0000 7fc489752640 1 --2- 172.21.15.26:0/224782408 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc470003b10 0x7fc470003f10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.521+0000 7fc489752640 1 -- 172.21.15.26:0/224782408 >> 172.21.15.26:0/224782408 conn(0x7fc4840658e0 msgr2=0x7fc484065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:27.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.521+0000 7fc489752640 1 -- 172.21.15.26:0/224782408 shutdown_connections 2023-12-19T18:17:27.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.521+0000 7fc489752640 1 -- 172.21.15.26:0/224782408 wait complete. 2023-12-19T18:17:27.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.521+0000 7fc489752640 1 Processor -- start 2023-12-19T18:17:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.521+0000 7fc489752640 1 -- start start 2023-12-19T18:17:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc489752640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc470003b10 0x7fc484149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc489752640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc48414b3e0 0x7fc484152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc489752640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc484152e80 0x7fc4840c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc489752640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc484153300 con 0x7fc484152e80 2023-12-19T18:17:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc489752640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48414a3d0 con 0x7fc470003b10 2023-12-19T18:17:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc489752640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48414a620 con 0x7fc48414b3e0 2023-12-19T18:17:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc4837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc484152e80 0x7fc4840c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc4837fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc484152e80 0x7fc4840c36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47532/0 (socket says 172.21.15.26:47532) 2023-12-19T18:17:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc4837fe640 1 -- 172.21.15.26:0/2298395413 learned_addr learned my addr 172.21.15.26:0/2298395413 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc482ffd640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc470003b10 0x7fc484149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc47bfff640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc48414b3e0 0x7fc484152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc4837fe640 1 -- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc48414b3e0 msgr2=0x7fc484152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc4837fe640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc48414b3e0 0x7fc484152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc4837fe640 1 -- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc470003b10 msgr2=0x7fc484149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc4837fe640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc470003b10 0x7fc484149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc4837fe640 1 -- 172.21.15.26:0/2298395413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48414a900 con 0x7fc484152e80 2023-12-19T18:17:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc4837fe640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc484152e80 0x7fc4840c36d0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fc484071800 tx=0x7fc47c007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc47b7fe640 1 -- 172.21.15.26:0/2298395413 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c00aaf0 con 0x7fc484152e80 2023-12-19T18:17:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc47b7fe640 1 -- 172.21.15.26:0/2298395413 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47c00ac70 con 0x7fc484152e80 2023-12-19T18:17:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc489752640 1 -- 172.21.15.26:0/2298395413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48414abe0 con 0x7fc484152e80 2023-12-19T18:17:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc47b7fe640 1 -- 172.21.15.26:0/2298395413 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0133f0 con 0x7fc484152e80 2023-12-19T18:17:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc489752640 1 -- 172.21.15.26:0/2298395413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48414a130 con 0x7fc484152e80 2023-12-19T18:17:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.525+0000 7fc489752640 1 -- 172.21.15.26:0/2298395413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc484074270 con 0x7fc484152e80 2023-12-19T18:17:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.529+0000 7fc47b7fe640 1 -- 172.21.15.26:0/2298395413 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fc47c013570 con 0x7fc484152e80 2023-12-19T18:17:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.529+0000 7fc47b7fe640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc464065ca0 0x7fc464068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.529+0000 7fc47b7fe640 1 -- 172.21.15.26:0/2298395413 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fc47c0841b0 con 0x7fc484152e80 2023-12-19T18:17:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.529+0000 7fc482ffd640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc464065ca0 0x7fc464068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.529+0000 7fc482ffd640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc464065ca0 0x7fc464068160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4740af000 tx=0x7fc4740046d0 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:27.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.533+0000 7fc47b7fe640 1 -- 172.21.15.26:0/2298395413 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc47c051500 con 0x7fc484152e80 2023-12-19T18:17:27.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.689+0000 7fc489752640 1 -- 172.21.15.26:0/2298395413 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4840ccc30 con 0x7fc484152e80 2023-12-19T18:17:27.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.689+0000 7fc47b7fe640 1 -- 172.21.15.26:0/2298395413 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v232) v1 ==== 74+0+17274 (secure 0 0 0) 0x7fc47c054b20 con 0x7fc484152e80 2023-12-19T18:17:27.695 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:27.695 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":232,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:27.324937+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:25.867829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:26.330514+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.693+0000 7fc489752640 1 -- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc464065ca0 msgr2=0x7fc464068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.693+0000 7fc489752640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc464065ca0 0x7fc464068160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4740af000 tx=0x7fc4740046d0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.693+0000 7fc489752640 1 -- 172.21.15.26:0/2298395413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc484152e80 msgr2=0x7fc4840c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.693+0000 7fc489752640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc484152e80 0x7fc4840c36d0 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fc484071800 tx=0x7fc47c007b10 comp rx=0 tx=0).stop 2023-12-19T18:17:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.693+0000 7fc489752640 1 -- 172.21.15.26:0/2298395413 shutdown_connections 2023-12-19T18:17:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.693+0000 7fc489752640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc464065ca0 0x7fc464068160 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.693+0000 7fc489752640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc484152e80 0x7fc4840c36d0 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.693+0000 7fc489752640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc48414b3e0 0x7fc484152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.693+0000 7fc489752640 1 --2- 172.21.15.26:0/2298395413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc470003b10 0x7fc484149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.693+0000 7fc489752640 1 -- 172.21.15.26:0/2298395413 >> 172.21.15.26:0/2298395413 conn(0x7fc4840658e0 msgr2=0x7fc48415a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:27.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.693+0000 7fc489752640 1 -- 172.21.15.26:0/2298395413 shutdown_connections 2023-12-19T18:17:27.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.693+0000 7fc489752640 1 -- 172.21.15.26:0/2298395413 wait complete. 2023-12-19T18:17:27.716 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75c3a6640 1 Processor -- start 2023-12-19T18:17:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75c3a6640 1 -- start start 2023-12-19T18:17:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75c3a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540ccc20 0x7fb754161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75c3a6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb7540cd050 0x7fb7540cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75c3a6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb754160890 con 0x7fb7540cfd70 2023-12-19T18:17:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75c3a6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7540cdaf0 con 0x7fb7540cd050 2023-12-19T18:17:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75c3a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb754160b40 con 0x7fb7540ccc20 2023-12-19T18:17:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75a142640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540ccc20 0x7fb754161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb759941640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb7540cd050 0x7fb7540cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75a943640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fb7540cfd70 0x7fb7540d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:60830/0 (socket says 172.21.15.26:60830) 2023-12-19T18:17:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75a943640 1 -- 172.21.15.26:0/2305349920 learned_addr learned my addr 172.21.15.26:0/2305349920 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75a142640 1 -- 172.21.15.26:0/2305349920 >> v2:172.21.15.160:3300/0 conn(0x7fb7540cd050 msgr2=0x7fb7540cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75a142640 1 --2- 172.21.15.26:0/2305349920 >> v2:172.21.15.160:3300/0 conn(0x7fb7540cd050 0x7fb7540cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75a142640 1 -- 172.21.15.26:0/2305349920 >> v1:172.21.15.160:6789/0 conn(0x7fb7540cfd70 legacy=0x7fb7540d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.857+0000 7fb75a142640 1 -- 172.21.15.26:0/2305349920 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb754160e20 con 0x7fb7540ccc20 2023-12-19T18:17:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.861+0000 7fb75a142640 1 --2- 172.21.15.26:0/2305349920 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540ccc20 0x7fb754161620 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb7500097c0 tx=0x7fb75001c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b36e90f3724139d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.861+0000 7fb759140640 1 -- 172.21.15.26:0/2305349920 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7500ac410 con 0x7fb7540ccc20 2023-12-19T18:17:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.861+0000 7fb759140640 1 -- 172.21.15.26:0/2305349920 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7500ac570 con 0x7fb7540ccc20 2023-12-19T18:17:27.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.861+0000 7fb759140640 1 -- 172.21.15.26:0/2305349920 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7500ac900 con 0x7fb7540ccc20 2023-12-19T18:17:27.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.861+0000 7fb75c3a6640 1 -- 172.21.15.26:0/2305349920 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540ccc20 msgr2=0x7fb754161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.861+0000 7fb75c3a6640 1 --2- 172.21.15.26:0/2305349920 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540ccc20 0x7fb754161620 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb7500097c0 tx=0x7fb75001c3f0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.861+0000 7fb75c3a6640 1 -- 172.21.15.26:0/2305349920 shutdown_connections 2023-12-19T18:17:27.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.861+0000 7fb75c3a6640 1 --2- 172.21.15.26:0/2305349920 >> v2:172.21.15.160:3300/0 conn(0x7fb7540cd050 0x7fb7540cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.861+0000 7fb75c3a6640 1 --2- 172.21.15.26:0/2305349920 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540ccc20 0x7fb754161620 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.861+0000 7fb75c3a6640 1 -- 172.21.15.26:0/2305349920 >> 172.21.15.26:0/2305349920 conn(0x7fb75406d510 msgr2=0x7fb75406d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.861+0000 7fb75c3a6640 1 -- 172.21.15.26:0/2305349920 shutdown_connections 2023-12-19T18:17:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.861+0000 7fb75c3a6640 1 -- 172.21.15.26:0/2305349920 wait complete. 2023-12-19T18:17:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75c3a6640 1 Processor -- start 2023-12-19T18:17:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75c3a6640 1 -- start start 2023-12-19T18:17:27.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75c3a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7540ccc20 0x7fb754149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75c3a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540cd050 0x7fb75415eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75c3a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cfd70 0x7fb75415f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75c3a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75414a3b0 con 0x7fb7540cfd70 2023-12-19T18:17:27.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75c3a6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75415b0e0 con 0x7fb7540ccc20 2023-12-19T18:17:27.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75c3a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75415b3c0 con 0x7fb7540cd050 2023-12-19T18:17:27.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75a943640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cfd70 0x7fb75415f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75a943640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cfd70 0x7fb75415f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47548/0 (socket says 172.21.15.26:47548) 2023-12-19T18:17:27.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75a943640 1 -- 172.21.15.26:0/3832521338 learned_addr learned my addr 172.21.15.26:0/3832521338 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:27.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb759941640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540cd050 0x7fb75415eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75a142640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7540ccc20 0x7fb754149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75a943640 1 -- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540cd050 msgr2=0x7fb75415eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75a943640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540cd050 0x7fb75415eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75a943640 1 -- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7540ccc20 msgr2=0x7fb754149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:27.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75a943640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7540ccc20 0x7fb754149e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:27.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75a943640 1 -- 172.21.15.26:0/3832521338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb750009420 con 0x7fb7540cfd70 2023-12-19T18:17:27.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75a943640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cfd70 0x7fb75415f420 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00ca80 tx=0x7fb74c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:27.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb74affd640 1 -- 172.21.15.26:0/3832521338 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c00ac30 con 0x7fb7540cfd70 2023-12-19T18:17:27.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb74affd640 1 -- 172.21.15.26:0/3832521338 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74c00ad90 con 0x7fb7540cfd70 2023-12-19T18:17:27.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb74affd640 1 -- 172.21.15.26:0/3832521338 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c012600 con 0x7fb7540cfd70 2023-12-19T18:17:27.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75c3a6640 1 -- 172.21.15.26:0/3832521338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75415b6a0 con 0x7fb7540cfd70 2023-12-19T18:17:27.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.865+0000 7fb75c3a6640 1 -- 172.21.15.26:0/3832521338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75415bc60 con 0x7fb7540cfd70 2023-12-19T18:17:27.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.869+0000 7fb75c3a6640 1 -- 172.21.15.26:0/3832521338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75415bea0 con 0x7fb7540cfd70 2023-12-19T18:17:27.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.869+0000 7fb74affd640 1 -- 172.21.15.26:0/3832521338 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb74c007070 con 0x7fb7540cfd70 2023-12-19T18:17:27.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.869+0000 7fb74affd640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb738065ca0 0x7fb738068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:27.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.869+0000 7fb74affd640 1 -- 172.21.15.26:0/3832521338 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fb74c083500 con 0x7fb7540cfd70 2023-12-19T18:17:27.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.869+0000 7fb75a142640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb738065ca0 0x7fb738068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:27.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.869+0000 7fb75a142640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb738065ca0 0x7fb738068160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb7500ae000 tx=0x7fb750008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:27.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:27.873+0000 7fb74affd640 1 -- 172.21.15.26:0/3832521338 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb75415bea0 con 0x7fb7540cfd70 2023-12-19T18:17:28.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.033+0000 7fb75c3a6640 1 -- 172.21.15.26:0/3832521338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb754160890 con 0x7fb7540cfd70 2023-12-19T18:17:28.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.033+0000 7fb74affd640 1 -- 172.21.15.26:0/3832521338 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v232) v1 ==== 74+0+17274 (secure 0 0 0) 0x7fb754160890 con 0x7fb7540cfd70 2023-12-19T18:17:28.038 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:28.038 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":232,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:27.324937+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:25.867829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:26.330514+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.037+0000 7fb75c3a6640 1 -- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb738065ca0 msgr2=0x7fb738068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.037+0000 7fb75c3a6640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb738065ca0 0x7fb738068160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb7500ae000 tx=0x7fb750008040 comp rx=0 tx=0).stop 2023-12-19T18:17:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.037+0000 7fb75c3a6640 1 -- 172.21.15.26:0/3832521338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cfd70 msgr2=0x7fb75415f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.037+0000 7fb75c3a6640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cfd70 0x7fb75415f420 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00ca80 tx=0x7fb74c00cf40 comp rx=0 tx=0).stop 2023-12-19T18:17:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.037+0000 7fb75c3a6640 1 -- 172.21.15.26:0/3832521338 shutdown_connections 2023-12-19T18:17:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.037+0000 7fb75c3a6640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb738065ca0 0x7fb738068160 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.037+0000 7fb75c3a6640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb7540cfd70 0x7fb75415f420 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.037+0000 7fb75c3a6640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb7540cd050 0x7fb75415eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.037+0000 7fb75c3a6640 1 --2- 172.21.15.26:0/3832521338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb7540ccc20 0x7fb754149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.037+0000 7fb75c3a6640 1 -- 172.21.15.26:0/3832521338 >> 172.21.15.26:0/3832521338 conn(0x7fb75406d510 msgr2=0x7fb75406f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:28.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.037+0000 7fb75c3a6640 1 -- 172.21.15.26:0/3832521338 shutdown_connections 2023-12-19T18:17:28.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.037+0000 7fb75c3a6640 1 -- 172.21.15.26:0/3832521338 wait complete. 2023-12-19T18:17:28.060 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:17:28.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.201+0000 7f0e6a32c640 1 Processor -- start 2023-12-19T18:17:28.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.201+0000 7f0e6a32c640 1 -- start start 2023-12-19T18:17:28.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.201+0000 7f0e6a32c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0e64159c00 0x7f0e6415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:28.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.201+0000 7f0e6a32c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6406fb60 con 0x7f0e64159820 2023-12-19T18:17:28.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.201+0000 7f0e6a32c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6406fe40 con 0x7f0e6406f6c0 2023-12-19T18:17:28.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.201+0000 7f0e6a32c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e6415e470 con 0x7f0e64159c00 2023-12-19T18:17:28.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.201+0000 7f0e63fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0e64159820 0x7f0e640c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47634/0 (socket says 172.21.15.26:47634) 2023-12-19T18:17:28.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.201+0000 7f0e63fff640 1 -- 172.21.15.26:0/2226345623 learned_addr learned my addr 172.21.15.26:0/2226345623 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:28.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.201+0000 7f0e688c9640 1 --2- 172.21.15.26:0/2226345623 >> v2:172.21.15.160:3300/0 conn(0x7f0e64159c00 0x7f0e6415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.205+0000 7f0e62ffd640 1 -- 172.21.15.26:0/2226345623 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688941024 0 0) 0x7f0e6406fb60 con 0x7f0e64159820 2023-12-19T18:17:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.205+0000 7f0e62ffd640 1 -- 172.21.15.26:0/2226345623 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e4c005740 con 0x7f0e64159820 2023-12-19T18:17:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.205+0000 7f0e62ffd640 1 -- 172.21.15.26:0/2226345623 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613093303 0 0) 0x7f0e6406fe40 con 0x7f0e6406f6c0 2023-12-19T18:17:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.205+0000 7f0e62ffd640 1 -- 172.21.15.26:0/2226345623 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e6406fb60 con 0x7f0e6406f6c0 2023-12-19T18:17:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.205+0000 7f0e688c9640 1 -- 172.21.15.26:0/2226345623 >> v1:172.21.15.160:6789/0 conn(0x7f0e6406f6c0 legacy=0x7f0e64070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.205+0000 7f0e688c9640 1 -- 172.21.15.26:0/2226345623 >> v1:172.21.15.26:6789/0 conn(0x7f0e64159820 legacy=0x7f0e640c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.205+0000 7f0e688c9640 1 -- 172.21.15.26:0/2226345623 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e64070120 con 0x7f0e64159c00 2023-12-19T18:17:28.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.205+0000 7f0e688c9640 1 --2- 172.21.15.26:0/2226345623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e64159c00 0x7f0e6415df30 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009d90 tx=0x7f0e5c01c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa4398fd6ed7c49f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.205+0000 7f0e62ffd640 1 -- 172.21.15.26:0/2226345623 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0a59e0 con 0x7f0e64159c00 2023-12-19T18:17:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.205+0000 7f0e62ffd640 1 -- 172.21.15.26:0/2226345623 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5c0a5b40 con 0x7f0e64159c00 2023-12-19T18:17:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.205+0000 7f0e62ffd640 1 -- 172.21.15.26:0/2226345623 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0ad440 con 0x7f0e64159c00 2023-12-19T18:17:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2226345623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e64159c00 msgr2=0x7f0e6415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 --2- 172.21.15.26:0/2226345623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e64159c00 0x7f0e6415df30 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c009d90 tx=0x7f0e5c01c920 comp rx=0 tx=0).stop 2023-12-19T18:17:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2226345623 shutdown_connections 2023-12-19T18:17:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 --2- 172.21.15.26:0/2226345623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e64159c00 0x7f0e6415df30 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2226345623 >> 172.21.15.26:0/2226345623 conn(0x7f0e640ce730 msgr2=0x7f0e640ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2226345623 shutdown_connections 2023-12-19T18:17:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2226345623 wait complete. 2023-12-19T18:17:28.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 Processor -- start 2023-12-19T18:17:28.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 -- start start 2023-12-19T18:17:28.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e6406f6c0 0x7f0e641507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:28.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e64159820 0x7f0e64149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:28.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e64159c00 0x7f0e640ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:28.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640ca0d0 con 0x7f0e64159820 2023-12-19T18:17:28.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e641538e0 con 0x7f0e64159c00 2023-12-19T18:17:28.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e6a32c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640c21d0 con 0x7f0e6406f6c0 2023-12-19T18:17:28.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e64159820 0x7f0e64149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:28.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e637fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e64159820 0x7f0e64149d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47562/0 (socket says 172.21.15.26:47562) 2023-12-19T18:17:28.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e637fe640 1 -- 172.21.15.26:0/2686040439 learned_addr learned my addr 172.21.15.26:0/2686040439 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:28.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e63fff640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e6406f6c0 0x7f0e641507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:28.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.209+0000 7f0e688c9640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e64159c00 0x7f0e640ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:28.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e637fe640 1 -- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e6406f6c0 msgr2=0x7f0e641507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e637fe640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e6406f6c0 0x7f0e641507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e637fe640 1 -- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e64159c00 msgr2=0x7f0e640ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e637fe640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e64159c00 0x7f0e640ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e637fe640 1 -- 172.21.15.26:0/2686040439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5c009a70 con 0x7f0e64159820 2023-12-19T18:17:28.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e637fe640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e64159820 0x7f0e64149d00 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800c3c0 tx=0x7f0e5800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e60ff9640 1 -- 172.21.15.26:0/2686040439 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e580052d0 con 0x7f0e64159820 2023-12-19T18:17:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e60ff9640 1 -- 172.21.15.26:0/2686040439 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e58005430 con 0x7f0e64159820 2023-12-19T18:17:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2686040439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6406b880 con 0x7f0e64159820 2023-12-19T18:17:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e60ff9640 1 -- 172.21.15.26:0/2686040439 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58003840 con 0x7f0e64159820 2023-12-19T18:17:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2686040439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e640c9b10 con 0x7f0e64159820 2023-12-19T18:17:28.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2686040439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e64074270 con 0x7f0e64159820 2023-12-19T18:17:28.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e60ff9640 1 -- 172.21.15.26:0/2686040439 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f0e580039a0 con 0x7f0e64159820 2023-12-19T18:17:28.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e60ff9640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0e50065cd0 0x7f0e50068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:28.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.213+0000 7f0e60ff9640 1 -- 172.21.15.26:0/2686040439 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f0e58082cc0 con 0x7f0e64159820 2023-12-19T18:17:28.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.217+0000 7f0e63fff640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0e50065cd0 0x7f0e50068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:28.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.217+0000 7f0e63fff640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0e50065cd0 0x7f0e50068190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0e540011f0 tx=0x7f0e54007d40 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:28.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.217+0000 7f0e60ff9640 1 -- 172.21.15.26:0/2686040439 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0e58089080 con 0x7f0e64159820 2023-12-19T18:17:28.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:28.413+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2686040439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0e640b46b0 con 0x7f0e64159820 2023-12-19T18:17:28.826 INFO:tasks.ceph.mds.c.smithi026.stderr:2023-12-19T18:17:28.825+0000 7f71716d6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:17:29.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.337+0000 7f0e60ff9640 1 -- 172.21.15.26:0/2686040439 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v111) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0e580170b0 con 0x7f0e64159820 2023-12-19T18:17:29.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.337+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0e50065cd0 msgr2=0x7f0e50068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:29.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.337+0000 7f0e6a32c640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0e50065cd0 0x7f0e50068190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0e540011f0 tx=0x7f0e54007d40 comp rx=0 tx=0).stop 2023-12-19T18:17:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.337+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2686040439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e64159820 msgr2=0x7f0e64149d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.337+0000 7f0e6a32c640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e64159820 0x7f0e64149d00 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800c3c0 tx=0x7f0e5800c880 comp rx=0 tx=0).stop 2023-12-19T18:17:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.337+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2686040439 shutdown_connections 2023-12-19T18:17:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.337+0000 7f0e6a32c640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0e50065cd0 0x7f0e50068190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.337+0000 7f0e6a32c640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0e64159c00 0x7f0e640ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.337+0000 7f0e6a32c640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e64159820 0x7f0e64149d00 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.337+0000 7f0e6a32c640 1 --2- 172.21.15.26:0/2686040439 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0e6406f6c0 0x7f0e641507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.337+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2686040439 >> 172.21.15.26:0/2686040439 conn(0x7f0e640ce730 msgr2=0x7f0e640ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.337+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2686040439 shutdown_connections 2023-12-19T18:17:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.337+0000 7f0e6a32c640 1 -- 172.21.15.26:0/2686040439 wait complete. 2023-12-19T18:17:29.366 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:17:29.367 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:17:29.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.513+0000 7f2ef0df3640 1 Processor -- start 2023-12-19T18:17:29.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.513+0000 7f2ef0df3640 1 -- start start 2023-12-19T18:17:29.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.513+0000 7f2ef0df3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2eec06f6c0 0x7f2eec15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:29.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2ef0df3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2eec159c00 0x7f2eec070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:29.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2ef0df3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eec06fb60 con 0x7f2eec159820 2023-12-19T18:17:29.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2ef0df3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eec15e470 con 0x7f2eec159c00 2023-12-19T18:17:29.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2ef0df3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eec06fe40 con 0x7f2eec06f6c0 2023-12-19T18:17:29.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2eea59c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2eec159820 0x7f2eec0c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47646/0 (socket says 172.21.15.26:47646) 2023-12-19T18:17:29.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2eea59c640 1 -- 172.21.15.26:0/3489107036 learned_addr learned my addr 172.21.15.26:0/3489107036 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:29.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2eead9d640 1 --2- 172.21.15.26:0/3489107036 >> v2:172.21.15.26:3300/0 conn(0x7f2eec159c00 0x7f2eec070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2ee9d9b640 1 -- 172.21.15.26:0/3489107036 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142297952 0 0) 0x7f2eec06fb60 con 0x7f2eec159820 2023-12-19T18:17:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2ee9d9b640 1 -- 172.21.15.26:0/3489107036 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ed8003530 con 0x7f2eec159820 2023-12-19T18:17:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2ee3fff640 1 --2- 172.21.15.26:0/3489107036 >> v2:172.21.15.160:3300/0 conn(0x7f2eec06f6c0 0x7f2eec15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2eead9d640 1 -- 172.21.15.26:0/3489107036 >> v2:172.21.15.160:3300/0 conn(0x7f2eec06f6c0 msgr2=0x7f2eec15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2eead9d640 1 --2- 172.21.15.26:0/3489107036 >> v2:172.21.15.160:3300/0 conn(0x7f2eec06f6c0 0x7f2eec15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2eead9d640 1 -- 172.21.15.26:0/3489107036 >> v1:172.21.15.26:6789/0 conn(0x7f2eec159820 legacy=0x7f2eec0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2eead9d640 1 -- 172.21.15.26:0/3489107036 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eec070120 con 0x7f2eec159c00 2023-12-19T18:17:29.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2ee3fff640 1 --2- 172.21.15.26:0/3489107036 >> v2:172.21.15.160:3300/0 conn(0x7f2eec06f6c0 0x7f2eec15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:29.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2eead9d640 1 --2- 172.21.15.26:0/3489107036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eec159c00 0x7f2eec070c00 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0caf30 tx=0x7f2ee401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e06a481f41b6a36 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:29.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2ee9d9b640 1 -- 172.21.15.26:0/3489107036 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee40a58b0 con 0x7f2eec159c00 2023-12-19T18:17:29.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2ee9d9b640 1 -- 172.21.15.26:0/3489107036 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee40a5a10 con 0x7f2eec159c00 2023-12-19T18:17:29.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.517+0000 7f2ee9d9b640 1 -- 172.21.15.26:0/3489107036 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee40a5da0 con 0x7f2eec159c00 2023-12-19T18:17:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 -- 172.21.15.26:0/3489107036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eec159c00 msgr2=0x7f2eec070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 --2- 172.21.15.26:0/3489107036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eec159c00 0x7f2eec070c00 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0caf30 tx=0x7f2ee401c8e0 comp rx=0 tx=0).stop 2023-12-19T18:17:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 -- 172.21.15.26:0/3489107036 shutdown_connections 2023-12-19T18:17:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 --2- 172.21.15.26:0/3489107036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eec159c00 0x7f2eec070c00 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 --2- 172.21.15.26:0/3489107036 >> v2:172.21.15.160:3300/0 conn(0x7f2eec06f6c0 0x7f2eec15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 -- 172.21.15.26:0/3489107036 >> 172.21.15.26:0/3489107036 conn(0x7f2eec0ce730 msgr2=0x7f2eec0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 -- 172.21.15.26:0/3489107036 shutdown_connections 2023-12-19T18:17:29.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 -- 172.21.15.26:0/3489107036 wait complete. 2023-12-19T18:17:29.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 Processor -- start 2023-12-19T18:17:29.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 -- start start 2023-12-19T18:17:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2eec06f6c0 0x7f2eec14da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2eec159820 0x7f2eec14e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eec159c00 0x7f2eec15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec0c21d0 con 0x7f2eec159c00 2023-12-19T18:17:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec150ba0 con 0x7f2eec06f6c0 2023-12-19T18:17:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.521+0000 7f2ef0df3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eec0c2420 con 0x7f2eec159820 2023-12-19T18:17:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2eead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eec159c00 0x7f2eec15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2eead9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eec159c00 0x7f2eec15b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47574/0 (socket says 172.21.15.26:47574) 2023-12-19T18:17:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2eead9d640 1 -- 172.21.15.26:0/2334993565 learned_addr learned my addr 172.21.15.26:0/2334993565 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2eea59c640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2eec06f6c0 0x7f2eec14da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:29.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2ee3fff640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2eec159820 0x7f2eec14e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:29.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2eead9d640 1 -- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2eec159820 msgr2=0x7f2eec14e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2eead9d640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2eec159820 0x7f2eec14e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2eead9d640 1 -- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2eec06f6c0 msgr2=0x7f2eec14da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2eead9d640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2eec06f6c0 0x7f2eec14da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2eead9d640 1 -- 172.21.15.26:0/2334993565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee4009a70 con 0x7f2eec159c00 2023-12-19T18:17:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2eead9d640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eec159c00 0x7f2eec15b6d0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40194f0 tx=0x7f2ee4019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2ee2ffd640 1 -- 172.21.15.26:0/2334993565 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee40b9650 con 0x7f2eec159c00 2023-12-19T18:17:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2ee2ffd640 1 -- 172.21.15.26:0/2334993565 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee40b97b0 con 0x7f2eec159c00 2023-12-19T18:17:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2ef0df3640 1 -- 172.21.15.26:0/2334993565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eec06b880 con 0x7f2eec159c00 2023-12-19T18:17:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2ee2ffd640 1 -- 172.21.15.26:0/2334993565 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee40b9b60 con 0x7f2eec159c00 2023-12-19T18:17:29.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2ef0df3640 1 -- 172.21.15.26:0/2334993565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eec0c29e0 con 0x7f2eec159c00 2023-12-19T18:17:29.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2ef0df3640 1 -- 172.21.15.26:0/2334993565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eec074270 con 0x7f2eec159c00 2023-12-19T18:17:29.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.525+0000 7f2ee2ffd640 1 -- 172.21.15.26:0/2334993565 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f2ee40b9cc0 con 0x7f2eec159c00 2023-12-19T18:17:29.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.529+0000 7f2ee2ffd640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2ecc065ca0 0x7f2ecc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:29.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.529+0000 7f2ee2ffd640 1 -- 172.21.15.26:0/2334993565 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f2ee411ecd0 con 0x7f2eec159c00 2023-12-19T18:17:29.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.529+0000 7f2eea59c640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2ecc065ca0 0x7f2ecc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:29.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.529+0000 7f2eea59c640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2ecc065ca0 0x7f2ecc068160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0011f0 tx=0x7f2edc007d40 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:29.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.533+0000 7f2ee2ffd640 1 -- 172.21.15.26:0/2334993565 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2ee40ec170 con 0x7f2eec159c00 2023-12-19T18:17:29.725 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:29.721+0000 7f2ef0df3640 1 -- 172.21.15.26:0/2334993565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2eec0b46b0 con 0x7f2eec159c00 2023-12-19T18:17:30.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.337+0000 7f2ee2ffd640 1 -- 172.21.15.26:0/2334993565 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v112) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2ee40b20c0 con 0x7f2eec159c00 2023-12-19T18:17:30.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.337+0000 7f2ef0df3640 1 -- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2ecc065ca0 msgr2=0x7f2ecc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:30.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.337+0000 7f2ef0df3640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2ecc065ca0 0x7f2ecc068160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2edc0011f0 tx=0x7f2edc007d40 comp rx=0 tx=0).stop 2023-12-19T18:17:30.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.337+0000 7f2ef0df3640 1 -- 172.21.15.26:0/2334993565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eec159c00 msgr2=0x7f2eec15b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:30.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.337+0000 7f2ef0df3640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eec159c00 0x7f2eec15b6d0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40194f0 tx=0x7f2ee4019680 comp rx=0 tx=0).stop 2023-12-19T18:17:30.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.341+0000 7f2ef0df3640 1 -- 172.21.15.26:0/2334993565 shutdown_connections 2023-12-19T18:17:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.341+0000 7f2ef0df3640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2ecc065ca0 0x7f2ecc068160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.341+0000 7f2ef0df3640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2eec159c00 0x7f2eec15b6d0 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.341+0000 7f2ef0df3640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2eec159820 0x7f2eec14e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.341+0000 7f2ef0df3640 1 --2- 172.21.15.26:0/2334993565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2eec06f6c0 0x7f2eec14da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.341+0000 7f2ef0df3640 1 -- 172.21.15.26:0/2334993565 >> 172.21.15.26:0/2334993565 conn(0x7f2eec0ce730 msgr2=0x7f2eec154c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.341+0000 7f2ef0df3640 1 -- 172.21.15.26:0/2334993565 shutdown_connections 2023-12-19T18:17:30.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.341+0000 7f2ef0df3640 1 -- 172.21.15.26:0/2334993565 wait complete. 2023-12-19T18:17:30.365 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:17:30.365 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:30.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.505+0000 7fbdacda6640 1 Processor -- start 2023-12-19T18:17:30.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.505+0000 7fbdacda6640 1 -- start start 2023-12-19T18:17:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.509+0000 7fbdacda6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda8154220 0x7fbda815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.509+0000 7fbdacda6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbda80cee10 0x7fbda815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:30.512 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.509+0000 7fbdacda6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda806bc30 con 0x7fbda806b3d0 2023-12-19T18:17:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.509+0000 7fbdacda6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda80cc6e0 con 0x7fbda80cee10 2023-12-19T18:17:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.509+0000 7fbdacda6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda815afc0 con 0x7fbda8154220 2023-12-19T18:17:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.509+0000 7fbda6d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbda80cee10 0x7fbda815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.509+0000 7fbda6d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbda80cee10 0x7fbda815b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47578/0 (socket says 172.21.15.26:47578) 2023-12-19T18:17:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.509+0000 7fbda6d9d640 1 -- 172.21.15.26:0/2778212001 learned_addr learned my addr 172.21.15.26:0/2778212001 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.509+0000 7fbda659c640 1 --2- 172.21.15.26:0/2778212001 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda8154220 0x7fbda815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.509+0000 7fbda6d9d640 1 -- 172.21.15.26:0/2778212001 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda8154220 msgr2=0x7fbda815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.509+0000 7fbda6d9d640 1 --2- 172.21.15.26:0/2778212001 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda8154220 0x7fbda815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.509+0000 7fbda6d9d640 1 -- 172.21.15.26:0/2778212001 >> v1:172.21.15.160:6789/0 conn(0x7fbda806b3d0 legacy=0x7fbda80ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:30.513 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.509+0000 7fbda6d9d640 1 -- 172.21.15.26:0/2778212001 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda806b950 con 0x7fbda80cee10 2023-12-19T18:17:30.515 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbda6d9d640 1 --2- 172.21.15.26:0/2778212001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda80cee10 0x7fbda815b1c0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009e50 tx=0x7fbda001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b870ea45988812f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbda5d9b640 1 -- 172.21.15.26:0/2778212001 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda00a58a0 con 0x7fbda80cee10 2023-12-19T18:17:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbda5d9b640 1 -- 172.21.15.26:0/2778212001 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda00a5a00 con 0x7fbda80cee10 2023-12-19T18:17:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbda5d9b640 1 -- 172.21.15.26:0/2778212001 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda00a5d90 con 0x7fbda80cee10 2023-12-19T18:17:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbdacda6640 1 -- 172.21.15.26:0/2778212001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda80cee10 msgr2=0x7fbda815b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbdacda6640 1 --2- 172.21.15.26:0/2778212001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda80cee10 0x7fbda815b1c0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fbda0009e50 tx=0x7fbda001c9e0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbdacda6640 1 -- 172.21.15.26:0/2778212001 shutdown_connections 2023-12-19T18:17:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbdacda6640 1 --2- 172.21.15.26:0/2778212001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda80cee10 0x7fbda815b1c0 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbdacda6640 1 --2- 172.21.15.26:0/2778212001 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda8154220 0x7fbda815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbdacda6640 1 -- 172.21.15.26:0/2778212001 >> 172.21.15.26:0/2778212001 conn(0x7fbda80658e0 msgr2=0x7fbda8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbdacda6640 1 -- 172.21.15.26:0/2778212001 shutdown_connections 2023-12-19T18:17:30.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbdacda6640 1 -- 172.21.15.26:0/2778212001 wait complete. 2023-12-19T18:17:30.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbdacda6640 1 Processor -- start 2023-12-19T18:17:30.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.513+0000 7fbdacda6640 1 -- start start 2023-12-19T18:17:30.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbdacda6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda806b3d0 0x7fbda80b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbdacda6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda80cee10 0x7fbda8146ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbdacda6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda8154220 0x7fbda815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbdacda6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda81554b0 con 0x7fbda80cee10 2023-12-19T18:17:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbdacda6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80b2f30 con 0x7fbda806b3d0 2023-12-19T18:17:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbdacda6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80b31d0 con 0x7fbda8154220 2023-12-19T18:17:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbd9ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda80cee10 0x7fbda8146ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbd9ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda80cee10 0x7fbda8146ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47586/0 (socket says 172.21.15.26:47586) 2023-12-19T18:17:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbd9ffff640 1 -- 172.21.15.26:0/3930011977 learned_addr learned my addr 172.21.15.26:0/3930011977 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbda659c640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda806b3d0 0x7fbda80b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbda6d9d640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda8154220 0x7fbda815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbd9ffff640 1 -- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda8154220 msgr2=0x7fbda815c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbd9ffff640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda8154220 0x7fbda815c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbd9ffff640 1 -- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda806b3d0 msgr2=0x7fbda80b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbd9ffff640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda806b3d0 0x7fbda80b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbd9ffff640 1 -- 172.21.15.26:0/3930011977 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda0009b30 con 0x7fbda80cee10 2023-12-19T18:17:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbda659c640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda806b3d0 0x7fbda80b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbd9ffff640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda80cee10 0x7fbda8146ea0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000c3c0 tx=0x7fbd9000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbd9effd640 1 -- 172.21.15.26:0/3930011977 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd900052d0 con 0x7fbda80cee10 2023-12-19T18:17:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbd9effd640 1 -- 172.21.15.26:0/3930011977 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd90005430 con 0x7fbda80cee10 2023-12-19T18:17:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbd9effd640 1 -- 172.21.15.26:0/3930011977 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90003840 con 0x7fbda80cee10 2023-12-19T18:17:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbdacda6640 1 -- 172.21.15.26:0/3930011977 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda81556e0 con 0x7fbda80cee10 2023-12-19T18:17:30.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbdacda6640 1 -- 172.21.15.26:0/3930011977 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda8155ca0 con 0x7fbda80cee10 2023-12-19T18:17:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbdacda6640 1 -- 172.21.15.26:0/3930011977 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda8074270 con 0x7fbda80cee10 2023-12-19T18:17:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.517+0000 7fbd9effd640 1 -- 172.21.15.26:0/3930011977 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fbd900039a0 con 0x7fbda80cee10 2023-12-19T18:17:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.521+0000 7fbd9effd640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbd88065ca0 0x7fbd88068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:30.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.521+0000 7fbd9effd640 1 -- 172.21.15.26:0/3930011977 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fbd90082d50 con 0x7fbda80cee10 2023-12-19T18:17:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.521+0000 7fbda659c640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbd88065ca0 0x7fbd88068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:30.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.521+0000 7fbda659c640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbd88065ca0 0x7fbd88068160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd98007a70 tx=0x7fbd98008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:30.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.525+0000 7fbd9effd640 1 -- 172.21.15.26:0/3930011977 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbd90086030 con 0x7fbda80cee10 2023-12-19T18:17:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.713+0000 7fbdacda6640 1 -- 172.21.15.26:0/3930011977 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbda80cdf60 con 0x7fbda80cee10 2023-12-19T18:17:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.713+0000 7fbd9effd640 1 -- 172.21.15.26:0/3930011977 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4730 (secure 0 0 0) 0x7fbd90017090 con 0x7fbda80cee10 2023-12-19T18:17:30.717 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:30.718 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":112,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2727392302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2727392302},{"type":"v1","addr":"172.21.15.160:6836","nonce":2727392302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1029020739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1029020739},{"type":"v1","addr":"172.21.15.160:6837","nonce":1029020739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":11424,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3251191056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3251191056},{"type":"v1","addr":"172.21.15.26:6834","nonce":3251191056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:27.325271+0000","modified":"2023-12-19T18:17:30.337993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11427":{"gid":11427,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2285370499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2285370499},{"type":"v1","addr":"172.21.15.26:6835","nonce":2285370499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-12-19T18:17:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 112 2023-12-19T18:17:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.713+0000 7fbdacda6640 1 -- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbd88065ca0 msgr2=0x7fbd88068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.713+0000 7fbdacda6640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbd88065ca0 0x7fbd88068160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd98007a70 tx=0x7fbd98008040 comp rx=0 tx=0).stop 2023-12-19T18:17:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.713+0000 7fbdacda6640 1 -- 172.21.15.26:0/3930011977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda80cee10 msgr2=0x7fbda8146ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.713+0000 7fbdacda6640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda80cee10 0x7fbda8146ea0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000c3c0 tx=0x7fbd9000c880 comp rx=0 tx=0).stop 2023-12-19T18:17:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.717+0000 7fbdacda6640 1 -- 172.21.15.26:0/3930011977 shutdown_connections 2023-12-19T18:17:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.717+0000 7fbdacda6640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbd88065ca0 0x7fbd88068160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.717+0000 7fbdacda6640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbda8154220 0x7fbda815c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.717+0000 7fbdacda6640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbda80cee10 0x7fbda8146ea0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.717+0000 7fbdacda6640 1 --2- 172.21.15.26:0/3930011977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbda806b3d0 0x7fbda80b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.717+0000 7fbdacda6640 1 -- 172.21.15.26:0/3930011977 >> 172.21.15.26:0/3930011977 conn(0x7fbda80658e0 msgr2=0x7fbda80ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.717+0000 7fbdacda6640 1 -- 172.21.15.26:0/3930011977 shutdown_connections 2023-12-19T18:17:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.717+0000 7fbdacda6640 1 -- 172.21.15.26:0/3930011977 wait complete. 2023-12-19T18:17:30.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:30.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.877+0000 7f49ad2fc640 1 Processor -- start 2023-12-19T18:17:30.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.877+0000 7f49ad2fc640 1 -- start start 2023-12-19T18:17:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f49ad2fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f49a815d260 0x7f49a806e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f49ad2fc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a80b3060 con 0x7f49a80b34f0 2023-12-19T18:17:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f49ad2fc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a815d820 con 0x7f49a80658e0 2023-12-19T18:17:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f49ad2fc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a815db00 con 0x7f49a815d260 2023-12-19T18:17:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f49a77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f49a80b34f0 0x7f49a80b3920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47648/0 (socket says 172.21.15.26:47648) 2023-12-19T18:17:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f49a77fe640 1 -- 172.21.15.26:0/2591327473 learned_addr learned my addr 172.21.15.26:0/2591327473 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:30.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f499e7fc640 1 --2- 172.21.15.26:0/2591327473 >> v2:172.21.15.160:3300/0 conn(0x7f49a815d260 0x7f49a806e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f49a67fc640 1 -- 172.21.15.26:0/2591327473 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826394868 0 0) 0x7f49a80b3060 con 0x7f49a80b34f0 2023-12-19T18:17:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f49a67fc640 1 -- 172.21.15.26:0/2591327473 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4994003530 con 0x7f49a80b34f0 2023-12-19T18:17:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f49a67fc640 1 -- 172.21.15.26:0/2591327473 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677456140 0 0) 0x7f49a815d820 con 0x7f49a80658e0 2023-12-19T18:17:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f49a67fc640 1 -- 172.21.15.26:0/2591327473 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a80b3060 con 0x7f49a80658e0 2023-12-19T18:17:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f499e7fc640 1 -- 172.21.15.26:0/2591327473 >> v1:172.21.15.160:6789/0 conn(0x7f49a80658e0 legacy=0x7f49a8065cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f499e7fc640 1 -- 172.21.15.26:0/2591327473 >> v1:172.21.15.26:6789/0 conn(0x7f49a80b34f0 legacy=0x7f49a80b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:30.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.881+0000 7f499e7fc640 1 -- 172.21.15.26:0/2591327473 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a815dde0 con 0x7f49a815d260 2023-12-19T18:17:30.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.885+0000 7f499e7fc640 1 --2- 172.21.15.26:0/2591327473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49a815d260 0x7f49a806e5a0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f4990009d90 tx=0x7f499001c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6c920e8a1fe3b92 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:30.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.885+0000 7f49a67fc640 1 -- 172.21.15.26:0/2591327473 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49900a5860 con 0x7f49a815d260 2023-12-19T18:17:30.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.885+0000 7f49a67fc640 1 -- 172.21.15.26:0/2591327473 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49900a59c0 con 0x7f49a815d260 2023-12-19T18:17:30.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.885+0000 7f49a67fc640 1 -- 172.21.15.26:0/2591327473 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49900a5d50 con 0x7f49a815d260 2023-12-19T18:17:30.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.885+0000 7f49ad2fc640 1 -- 172.21.15.26:0/2591327473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49a815d260 msgr2=0x7f49a806e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.885+0000 7f49ad2fc640 1 --2- 172.21.15.26:0/2591327473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49a815d260 0x7f49a806e5a0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f4990009d90 tx=0x7f499001c890 comp rx=0 tx=0).stop 2023-12-19T18:17:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.885+0000 7f49ad2fc640 1 -- 172.21.15.26:0/2591327473 shutdown_connections 2023-12-19T18:17:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.885+0000 7f49ad2fc640 1 --2- 172.21.15.26:0/2591327473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49a815d260 0x7f49a806e5a0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.885+0000 7f49ad2fc640 1 -- 172.21.15.26:0/2591327473 >> 172.21.15.26:0/2591327473 conn(0x7f49a806a320 msgr2=0x7f49a80b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.885+0000 7f49ad2fc640 1 -- 172.21.15.26:0/2591327473 shutdown_connections 2023-12-19T18:17:30.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.885+0000 7f49ad2fc640 1 -- 172.21.15.26:0/2591327473 wait complete. 2023-12-19T18:17:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.885+0000 7f49ad2fc640 1 Processor -- start 2023-12-19T18:17:30.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.885+0000 7f49ad2fc640 1 -- start start 2023-12-19T18:17:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49ad2fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49a80658e0 0x7f49a80cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49ad2fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49a80b34f0 0x7f49a813e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49ad2fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a815d260 0x7f49a8137080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49ad2fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a8135fe0 con 0x7f49a815d260 2023-12-19T18:17:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49ad2fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a81362c0 con 0x7f49a80658e0 2023-12-19T18:17:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49ad2fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a81365a0 con 0x7f49a80b34f0 2023-12-19T18:17:30.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a815d260 0x7f49a8137080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49a77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a815d260 0x7f49a8137080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47596/0 (socket says 172.21.15.26:47596) 2023-12-19T18:17:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49a77fe640 1 -- 172.21.15.26:0/1846339897 learned_addr learned my addr 172.21.15.26:0/1846339897 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49a6ffd640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49a80658e0 0x7f49a80cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:30.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f499e7fc640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49a80b34f0 0x7f49a813e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49a77fe640 1 -- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49a80b34f0 msgr2=0x7f49a813e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49a77fe640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49a80b34f0 0x7f49a813e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49a77fe640 1 -- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49a80658e0 msgr2=0x7f49a80cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49a77fe640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49a80658e0 0x7f49a80cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49a77fe640 1 -- 172.21.15.26:0/1846339897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4990009a70 con 0x7f49a815d260 2023-12-19T18:17:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49a77fe640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a815d260 0x7f49a8137080 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f49a000c450 tx=0x7f49a000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f499ffff640 1 -- 172.21.15.26:0/1846339897 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a0005db0 con 0x7f49a815d260 2023-12-19T18:17:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f499ffff640 1 -- 172.21.15.26:0/1846339897 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a0007070 con 0x7f49a815d260 2023-12-19T18:17:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f499ffff640 1 -- 172.21.15.26:0/1846339897 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a0007400 con 0x7f49a815d260 2023-12-19T18:17:30.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49ad2fc640 1 -- 172.21.15.26:0/1846339897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a8136880 con 0x7f49a815d260 2023-12-19T18:17:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49ad2fc640 1 -- 172.21.15.26:0/1846339897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a8135c40 con 0x7f49a815d260 2023-12-19T18:17:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.889+0000 7f49ad2fc640 1 -- 172.21.15.26:0/1846339897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a8074270 con 0x7f49a815d260 2023-12-19T18:17:30.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.893+0000 7f499ffff640 1 -- 172.21.15.26:0/1846339897 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f49a0007560 con 0x7f49a815d260 2023-12-19T18:17:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.893+0000 7f499ffff640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4988065ca0 0x7f4988068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.893+0000 7f499ffff640 1 -- 172.21.15.26:0/1846339897 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f49a0082e30 con 0x7f49a815d260 2023-12-19T18:17:30.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.893+0000 7f49a6ffd640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4988065ca0 0x7f4988068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:30.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.893+0000 7f49a6ffd640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4988065ca0 0x7f4988068160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f49980011f0 tx=0x7f4998009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:30.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:30.897+0000 7f499ffff640 1 -- 172.21.15.26:0/1846339897 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f49a00501a0 con 0x7f49a815d260 2023-12-19T18:17:31.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.057+0000 7f49ad2fc640 1 -- 172.21.15.26:0/1846339897 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49a8136d30 con 0x7f49a815d260 2023-12-19T18:17:31.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.057+0000 7f499ffff640 1 -- 172.21.15.26:0/1846339897 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v233) v1 ==== 74+0+17261 (secure 0 0 0) 0x7f49a0017020 con 0x7f49a815d260 2023-12-19T18:17:31.061 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:31.062 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":233,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:28.328548+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:25.867829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:26.330514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:31.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.061+0000 7f49ad2fc640 1 -- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4988065ca0 msgr2=0x7f4988068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.061+0000 7f49ad2fc640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4988065ca0 0x7f4988068160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f49980011f0 tx=0x7f4998009040 comp rx=0 tx=0).stop 2023-12-19T18:17:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.061+0000 7f49ad2fc640 1 -- 172.21.15.26:0/1846339897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a815d260 msgr2=0x7f49a8137080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.061+0000 7f49ad2fc640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a815d260 0x7f49a8137080 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f49a000c450 tx=0x7f49a000c910 comp rx=0 tx=0).stop 2023-12-19T18:17:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.061+0000 7f49ad2fc640 1 -- 172.21.15.26:0/1846339897 shutdown_connections 2023-12-19T18:17:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.061+0000 7f49ad2fc640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4988065ca0 0x7f4988068160 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.061+0000 7f49ad2fc640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49a815d260 0x7f49a8137080 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.061+0000 7f49ad2fc640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f49a80b34f0 0x7f49a813e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.061+0000 7f49ad2fc640 1 --2- 172.21.15.26:0/1846339897 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49a80658e0 0x7f49a80cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.061+0000 7f49ad2fc640 1 -- 172.21.15.26:0/1846339897 >> 172.21.15.26:0/1846339897 conn(0x7f49a806a320 msgr2=0x7f49a8070c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.061+0000 7f49ad2fc640 1 -- 172.21.15.26:0/1846339897 shutdown_connections 2023-12-19T18:17:31.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.061+0000 7f49ad2fc640 1 -- 172.21.15.26:0/1846339897 wait complete. 2023-12-19T18:17:31.083 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:17:31.083 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:17:31.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bd95bc640 1 Processor -- start 2023-12-19T18:17:31.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bd95bc640 1 -- start start 2023-12-19T18:17:31.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bd95bc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4bd406b3d0 0x7f4bd40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:31.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bd95bc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd406bc30 con 0x7f4bd40cec40 2023-12-19T18:17:31.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bd95bc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd40cc6e0 con 0x7f4bd4154220 2023-12-19T18:17:31.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bd95bc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd40cf0d0 con 0x7f4bd406b3d0 2023-12-19T18:17:31.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bd37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4bd40cec40 0x7f4bd415b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47658/0 (socket says 172.21.15.26:47658) 2023-12-19T18:17:31.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bd37fe640 1 -- 172.21.15.26:0/654663548 learned_addr learned my addr 172.21.15.26:0/654663548 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:31.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bcbfff640 1 --2- 172.21.15.26:0/654663548 >> v2:172.21.15.160:3300/0 conn(0x7f4bd406b3d0 0x7f4bd40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bd27fc640 1 -- 172.21.15.26:0/654663548 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071492300 0 0) 0x7f4bd406bc30 con 0x7f4bd40cec40 2023-12-19T18:17:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bd27fc640 1 -- 172.21.15.26:0/654663548 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc0003530 con 0x7f4bd40cec40 2023-12-19T18:17:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bd27fc640 1 -- 172.21.15.26:0/654663548 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895484141 0 0) 0x7f4bd40cc6e0 con 0x7f4bd4154220 2023-12-19T18:17:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bd27fc640 1 -- 172.21.15.26:0/654663548 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bd406bc30 con 0x7f4bd4154220 2023-12-19T18:17:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bcbfff640 1 -- 172.21.15.26:0/654663548 >> v1:172.21.15.160:6789/0 conn(0x7f4bd4154220 legacy=0x7f4bd415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bd27fc640 1 -- 172.21.15.26:0/654663548 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2377415411 0 0) 0x7f4bc0003530 con 0x7f4bd40cec40 2023-12-19T18:17:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bcbfff640 1 -- 172.21.15.26:0/654663548 >> v1:172.21.15.26:6789/0 conn(0x7f4bd40cec40 legacy=0x7f4bd415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.225+0000 7f4bcbfff640 1 -- 172.21.15.26:0/654663548 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd406b950 con 0x7f4bd406b3d0 2023-12-19T18:17:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.229+0000 7f4bcbfff640 1 --2- 172.21.15.26:0/654663548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 0x7f4bd40ce700 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0097c0 tx=0x7f4bbc01c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b619498d95a8a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:31.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.229+0000 7f4bd27fc640 1 -- 172.21.15.26:0/654663548 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc001a00 con 0x7f4bd406b3d0 2023-12-19T18:17:31.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.229+0000 7f4bd27fc640 1 -- 172.21.15.26:0/654663548 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc0a7050 con 0x7f4bd406b3d0 2023-12-19T18:17:31.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.229+0000 7f4bd27fc640 1 -- 172.21.15.26:0/654663548 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0ab620 con 0x7f4bd406b3d0 2023-12-19T18:17:31.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.229+0000 7f4bd95bc640 1 -- 172.21.15.26:0/654663548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 msgr2=0x7f4bd40ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.229+0000 7f4bd95bc640 1 --2- 172.21.15.26:0/654663548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 0x7f4bd40ce700 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0097c0 tx=0x7f4bbc01c370 comp rx=0 tx=0).stop 2023-12-19T18:17:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.229+0000 7f4bd95bc640 1 -- 172.21.15.26:0/654663548 shutdown_connections 2023-12-19T18:17:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.229+0000 7f4bd95bc640 1 --2- 172.21.15.26:0/654663548 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 0x7f4bd40ce700 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.229+0000 7f4bd95bc640 1 -- 172.21.15.26:0/654663548 >> 172.21.15.26:0/654663548 conn(0x7f4bd40658e0 msgr2=0x7f4bd4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.229+0000 7f4bd95bc640 1 -- 172.21.15.26:0/654663548 shutdown_connections 2023-12-19T18:17:31.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.229+0000 7f4bd95bc640 1 -- 172.21.15.26:0/654663548 wait complete. 2023-12-19T18:17:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bd95bc640 1 Processor -- start 2023-12-19T18:17:31.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bd95bc640 1 -- start start 2023-12-19T18:17:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bd95bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 0x7f4bd4165380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bd95bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd40cec40 0x7f4bd41f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bd95bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd4154220 0x7f4bd40afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bd95bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40b0070 con 0x7f4bd40cec40 2023-12-19T18:17:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bd95bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd4154690 con 0x7f4bd406b3d0 2023-12-19T18:17:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bd95bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd4154970 con 0x7f4bd4154220 2023-12-19T18:17:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bcbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd40cec40 0x7f4bd41f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:31.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bcbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd40cec40 0x7f4bd41f5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47608/0 (socket says 172.21.15.26:47608) 2023-12-19T18:17:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bcbfff640 1 -- 172.21.15.26:0/2921175360 learned_addr learned my addr 172.21.15.26:0/2921175360 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bd2ffd640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 0x7f4bd4165380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bd37fe640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd4154220 0x7f4bd40afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bcbfff640 1 -- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd4154220 msgr2=0x7f4bd40afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bcbfff640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd4154220 0x7f4bd40afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bcbfff640 1 -- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 msgr2=0x7f4bd4165380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bcbfff640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 0x7f4bd4165380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bcbfff640 1 -- 172.21.15.26:0/2921175360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc009420 con 0x7f4bd40cec40 2023-12-19T18:17:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bd2ffd640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 0x7f4bd4165380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bcbfff640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd40cec40 0x7f4bd41f5860 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0097c0 tx=0x7f4bbc019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bcb7fe640 1 -- 172.21.15.26:0/2921175360 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0abe70 con 0x7f4bd40cec40 2023-12-19T18:17:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bcb7fe640 1 -- 172.21.15.26:0/2921175360 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bbc004020 con 0x7f4bd40cec40 2023-12-19T18:17:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bcb7fe640 1 -- 172.21.15.26:0/2921175360 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0043b0 con 0x7f4bd40cec40 2023-12-19T18:17:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bd95bc640 1 -- 172.21.15.26:0/2921175360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd4154c50 con 0x7f4bd40cec40 2023-12-19T18:17:31.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.233+0000 7f4bd95bc640 1 -- 172.21.15.26:0/2921175360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd4155210 con 0x7f4bd40cec40 2023-12-19T18:17:31.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.237+0000 7f4bd95bc640 1 -- 172.21.15.26:0/2921175360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd4155400 con 0x7f4bd40cec40 2023-12-19T18:17:31.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.237+0000 7f4bcb7fe640 1 -- 172.21.15.26:0/2921175360 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f4bbc0b2040 con 0x7f4bd40cec40 2023-12-19T18:17:31.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.237+0000 7f4bcb7fe640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4bb4065ca0 0x7f4bb4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:31.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.237+0000 7f4bcb7fe640 1 -- 172.21.15.26:0/2921175360 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f4bbc11cc10 con 0x7f4bd40cec40 2023-12-19T18:17:31.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.237+0000 7f4bd2ffd640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4bb4065ca0 0x7f4bb4068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:31.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.237+0000 7f4bd2ffd640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4bb4065ca0 0x7f4bb4068160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40011f0 tx=0x7f4bc4009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:31.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.241+0000 7f4bcb7fe640 1 -- 172.21.15.26:0/2921175360 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4bbc0e9e90 con 0x7f4bd40cec40 2023-12-19T18:17:31.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.397+0000 7f4bd95bc640 1 -- 172.21.15.26:0/2921175360 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4bd40ccef0 con 0x7f4bb4065ca0 2023-12-19T18:17:31.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.405+0000 7f4bcb7fe640 1 -- 172.21.15.26:0/2921175360 <== mgr.10429 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289520 (secure 0 0 0) 0x7f4bc4001cf0 con 0x7f4bb4065ca0 2023-12-19T18:17:31.413 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:31.415 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:17:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.409+0000 7f4bd95bc640 1 -- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4bb4065ca0 msgr2=0x7f4bb4068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.409+0000 7f4bd95bc640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4bb4065ca0 0x7f4bb4068160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40011f0 tx=0x7f4bc4009040 comp rx=0 tx=0).stop 2023-12-19T18:17:31.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.409+0000 7f4bd95bc640 1 -- 172.21.15.26:0/2921175360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd40cec40 msgr2=0x7f4bd41f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.409+0000 7f4bd95bc640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd40cec40 0x7f4bd41f5860 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0097c0 tx=0x7f4bbc019110 comp rx=0 tx=0).stop 2023-12-19T18:17:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.409+0000 7f4bd95bc640 1 -- 172.21.15.26:0/2921175360 shutdown_connections 2023-12-19T18:17:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.409+0000 7f4bd95bc640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4bb4065ca0 0x7f4bb4068160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.409+0000 7f4bd95bc640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4bd4154220 0x7f4bd40afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.409+0000 7f4bd95bc640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bd40cec40 0x7f4bd41f5860 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.409+0000 7f4bd95bc640 1 --2- 172.21.15.26:0/2921175360 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd406b3d0 0x7f4bd4165380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.409+0000 7f4bd95bc640 1 -- 172.21.15.26:0/2921175360 >> 172.21.15.26:0/2921175360 conn(0x7f4bd40658e0 msgr2=0x7f4bd4157c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.409+0000 7f4bd95bc640 1 -- 172.21.15.26:0/2921175360 shutdown_connections 2023-12-19T18:17:31.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.409+0000 7f4bd95bc640 1 -- 172.21.15.26:0/2921175360 wait complete. 2023-12-19T18:17:31.434 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-12-19T18:17:30.935404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2306,"num_read_kb":4823,"num_write":659,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":406,"ondisk_log_size":406,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":408616,"kb_used_data":24040,"kb_used_omap":0,"kb_used_meta":384512,"kb_avail":754566104,"statfs":{"total":773094113280,"available":772675690496,"internally_reserved":0,"allocated":24616960,"data_stored":13638172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":393740288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.013143"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486632+0000","last_change":"2023-12-19T18:17:27.353410+0000","last_active":"2023-12-19T18:17:28.486632+0000","last_peered":"2023-12-19T18:17:28.486632+0000","last_clean":"2023-12-19T18:17:28.486632+0000","last_became_active":"2023-12-19T18:17:27.352446+0000","last_became_peered":"2023-12-19T18:17:27.352446+0000","last_unstale":"2023-12-19T18:17:28.486632+0000","last_undegraded":"2023-12-19T18:17:28.486632+0000","last_fullsized":"2023-12-19T18:17:28.486632+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:55.042619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.340490+0000","last_change":"2023-12-19T18:17:27.358427+0000","last_active":"2023-12-19T18:17:28.340490+0000","last_peered":"2023-12-19T18:17:28.340490+0000","last_clean":"2023-12-19T18:17:28.340490+0000","last_became_active":"2023-12-19T18:17:27.358141+0000","last_became_peered":"2023-12-19T18:17:27.358141+0000","last_unstale":"2023-12-19T18:17:28.340490+0000","last_undegraded":"2023-12-19T18:17:28.340490+0000","last_fullsized":"2023-12-19T18:17:28.340490+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:14.654686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486619+0000","last_change":"2023-12-19T18:17:27.356031+0000","last_active":"2023-12-19T18:17:28.486619+0000","last_peered":"2023-12-19T18:17:28.486619+0000","last_clean":"2023-12-19T18:17:28.486619+0000","last_became_active":"2023-12-19T18:17:27.354195+0000","last_became_peered":"2023-12-19T18:17:27.354195+0000","last_unstale":"2023-12-19T18:17:28.486619+0000","last_undegraded":"2023-12-19T18:17:28.486619+0000","last_fullsized":"2023-12-19T18:17:28.486619+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:16.644439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485397+0000","last_change":"2023-12-19T18:17:27.348408+0000","last_active":"2023-12-19T18:17:28.485397+0000","last_peered":"2023-12-19T18:17:28.485397+0000","last_clean":"2023-12-19T18:17:28.485397+0000","last_became_active":"2023-12-19T18:17:27.348138+0000","last_became_peered":"2023-12-19T18:17:27.348138+0000","last_unstale":"2023-12-19T18:17:28.485397+0000","last_undegraded":"2023-12-19T18:17:28.485397+0000","last_fullsized":"2023-12-19T18:17:28.485397+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:19.968471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485212+0000","last_change":"2023-12-19T18:17:27.351573+0000","last_active":"2023-12-19T18:17:28.485212+0000","last_peered":"2023-12-19T18:17:28.485212+0000","last_clean":"2023-12-19T18:17:28.485212+0000","last_became_active":"2023-12-19T18:17:27.351222+0000","last_became_peered":"2023-12-19T18:17:27.351222+0000","last_unstale":"2023-12-19T18:17:28.485212+0000","last_undegraded":"2023-12-19T18:17:28.485212+0000","last_fullsized":"2023-12-19T18:17:28.485212+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:17:31.537417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485126+0000","last_change":"2023-12-19T18:17:27.364283+0000","last_active":"2023-12-19T18:17:28.485126+0000","last_peered":"2023-12-19T18:17:28.485126+0000","last_clean":"2023-12-19T18:17:28.485126+0000","last_became_active":"2023-12-19T18:17:27.363962+0000","last_became_peered":"2023-12-19T18:17:27.363962+0000","last_unstale":"2023-12-19T18:17:28.485126+0000","last_undegraded":"2023-12-19T18:17:28.485126+0000","last_fullsized":"2023-12-19T18:17:28.485126+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:25.159077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485978+0000","last_change":"2023-12-19T18:17:27.357178+0000","last_active":"2023-12-19T18:17:28.485978+0000","last_peered":"2023-12-19T18:17:28.485978+0000","last_clean":"2023-12-19T18:17:28.485978+0000","last_became_active":"2023-12-19T18:17:27.356893+0000","last_became_peered":"2023-12-19T18:17:27.356893+0000","last_unstale":"2023-12-19T18:17:28.485978+0000","last_undegraded":"2023-12-19T18:17:28.485978+0000","last_fullsized":"2023-12-19T18:17:28.485978+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:31.153454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.337759+0000","last_change":"2023-12-19T18:17:27.344049+0000","last_active":"2023-12-19T18:17:28.337759+0000","last_peered":"2023-12-19T18:17:28.337759+0000","last_clean":"2023-12-19T18:17:28.337759+0000","last_became_active":"2023-12-19T18:17:27.343738+0000","last_became_peered":"2023-12-19T18:17:27.343738+0000","last_unstale":"2023-12-19T18:17:28.337759+0000","last_undegraded":"2023-12-19T18:17:28.337759+0000","last_fullsized":"2023-12-19T18:17:28.337759+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:31.445521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486859+0000","last_change":"2023-12-19T18:17:27.363691+0000","last_active":"2023-12-19T18:17:28.486859+0000","last_peered":"2023-12-19T18:17:28.486859+0000","last_clean":"2023-12-19T18:17:28.486859+0000","last_became_active":"2023-12-19T18:17:27.363412+0000","last_became_peered":"2023-12-19T18:17:27.363412+0000","last_unstale":"2023-12-19T18:17:28.486859+0000","last_undegraded":"2023-12-19T18:17:28.486859+0000","last_fullsized":"2023-12-19T18:17:28.486859+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:47.826883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485068+0000","last_change":"2023-12-19T18:17:27.343713+0000","last_active":"2023-12-19T18:17:28.485068+0000","last_peered":"2023-12-19T18:17:28.485068+0000","last_clean":"2023-12-19T18:17:28.485068+0000","last_became_active":"2023-12-19T18:17:27.343409+0000","last_became_peered":"2023-12-19T18:17:27.343409+0000","last_unstale":"2023-12-19T18:17:28.485068+0000","last_undegraded":"2023-12-19T18:17:28.485068+0000","last_fullsized":"2023-12-19T18:17:28.485068+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:40.489404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486121+0000","last_change":"2023-12-19T18:17:27.353890+0000","last_active":"2023-12-19T18:17:28.486121+0000","last_peered":"2023-12-19T18:17:28.486121+0000","last_clean":"2023-12-19T18:17:28.486121+0000","last_became_active":"2023-12-19T18:17:27.352505+0000","last_became_peered":"2023-12-19T18:17:27.352505+0000","last_unstale":"2023-12-19T18:17:28.486121+0000","last_undegraded":"2023-12-19T18:17:28.486121+0000","last_fullsized":"2023-12-19T18:17:28.486121+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:49:56.496681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.340424+0000","last_change":"2023-12-19T18:17:27.353636+0000","last_active":"2023-12-19T18:17:28.340424+0000","last_peered":"2023-12-19T18:17:28.340424+0000","last_clean":"2023-12-19T18:17:28.340424+0000","last_became_active":"2023-12-19T18:17:27.353234+0000","last_became_peered":"2023-12-19T18:17:27.353234+0000","last_unstale":"2023-12-19T18:17:28.340424+0000","last_undegraded":"2023-12-19T18:17:28.340424+0000","last_fullsized":"2023-12-19T18:17:28.340424+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:50.421353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.340116+0000","last_change":"2023-12-19T18:17:27.368093+0000","last_active":"2023-12-19T18:17:28.340116+0000","last_peered":"2023-12-19T18:17:28.340116+0000","last_clean":"2023-12-19T18:17:28.340116+0000","last_became_active":"2023-12-19T18:17:27.367300+0000","last_became_peered":"2023-12-19T18:17:27.367300+0000","last_unstale":"2023-12-19T18:17:28.340116+0000","last_undegraded":"2023-12-19T18:17:28.340116+0000","last_fullsized":"2023-12-19T18:17:28.340116+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:16.644439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.340089+0000","last_change":"2023-12-19T18:17:27.362240+0000","last_active":"2023-12-19T18:17:28.340089+0000","last_peered":"2023-12-19T18:17:28.340089+0000","last_clean":"2023-12-19T18:17:28.340089+0000","last_became_active":"2023-12-19T18:17:27.361917+0000","last_became_peered":"2023-12-19T18:17:27.361917+0000","last_unstale":"2023-12-19T18:17:28.340089+0000","last_undegraded":"2023-12-19T18:17:28.340089+0000","last_fullsized":"2023-12-19T18:17:28.340089+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:44:22.931173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.484629+0000","last_change":"2023-12-19T18:17:27.356217+0000","last_active":"2023-12-19T18:17:28.484629+0000","last_peered":"2023-12-19T18:17:28.484629+0000","last_clean":"2023-12-19T18:17:28.484629+0000","last_became_active":"2023-12-19T18:17:27.355721+0000","last_became_peered":"2023-12-19T18:17:27.355721+0000","last_unstale":"2023-12-19T18:17:28.484629+0000","last_undegraded":"2023-12-19T18:17:28.484629+0000","last_fullsized":"2023-12-19T18:17:28.484629+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:20:57.806100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339493+0000","last_change":"2023-12-19T18:17:27.350328+0000","last_active":"2023-12-19T18:17:28.339493+0000","last_peered":"2023-12-19T18:17:28.339493+0000","last_clean":"2023-12-19T18:17:28.339493+0000","last_became_active":"2023-12-19T18:17:27.350006+0000","last_became_peered":"2023-12-19T18:17:27.350006+0000","last_unstale":"2023-12-19T18:17:28.339493+0000","last_undegraded":"2023-12-19T18:17:28.339493+0000","last_fullsized":"2023-12-19T18:17:28.339493+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:00.178479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339872+0000","last_change":"2023-12-19T18:17:27.367615+0000","last_active":"2023-12-19T18:17:28.339872+0000","last_peered":"2023-12-19T18:17:28.339872+0000","last_clean":"2023-12-19T18:17:28.339872+0000","last_became_active":"2023-12-19T18:17:27.366800+0000","last_became_peered":"2023-12-19T18:17:27.366800+0000","last_unstale":"2023-12-19T18:17:28.339872+0000","last_undegraded":"2023-12-19T18:17:28.339872+0000","last_fullsized":"2023-12-19T18:17:28.339872+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:19:50.020207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485266+0000","last_change":"2023-12-19T18:17:27.357129+0000","last_active":"2023-12-19T18:17:28.485266+0000","last_peered":"2023-12-19T18:17:28.485266+0000","last_clean":"2023-12-19T18:17:28.485266+0000","last_became_active":"2023-12-19T18:17:27.356614+0000","last_became_peered":"2023-12-19T18:17:27.356614+0000","last_unstale":"2023-12-19T18:17:28.485266+0000","last_undegraded":"2023-12-19T18:17:28.485266+0000","last_fullsized":"2023-12-19T18:17:28.485266+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:22.773760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485580+0000","last_change":"2023-12-19T18:17:27.353111+0000","last_active":"2023-12-19T18:17:28.485580+0000","last_peered":"2023-12-19T18:17:28.485580+0000","last_clean":"2023-12-19T18:17:28.485580+0000","last_became_active":"2023-12-19T18:17:27.352840+0000","last_became_peered":"2023-12-19T18:17:27.352840+0000","last_unstale":"2023-12-19T18:17:28.485580+0000","last_undegraded":"2023-12-19T18:17:28.485580+0000","last_fullsized":"2023-12-19T18:17:28.485580+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:28.649300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485120+0000","last_change":"2023-12-19T18:17:27.340972+0000","last_active":"2023-12-19T18:17:28.485120+0000","last_peered":"2023-12-19T18:17:28.485120+0000","last_clean":"2023-12-19T18:17:28.485120+0000","last_became_active":"2023-12-19T18:17:27.340523+0000","last_became_peered":"2023-12-19T18:17:27.340523+0000","last_unstale":"2023-12-19T18:17:28.485120+0000","last_undegraded":"2023-12-19T18:17:28.485120+0000","last_fullsized":"2023-12-19T18:17:28.485120+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:50.621127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485313+0000","last_change":"2023-12-19T18:17:27.355317+0000","last_active":"2023-12-19T18:17:28.485313+0000","last_peered":"2023-12-19T18:17:28.485313+0000","last_clean":"2023-12-19T18:17:28.485313+0000","last_became_active":"2023-12-19T18:17:27.355002+0000","last_became_peered":"2023-12-19T18:17:27.355002+0000","last_unstale":"2023-12-19T18:17:28.485313+0000","last_undegraded":"2023-12-19T18:17:28.485313+0000","last_fullsized":"2023-12-19T18:17:28.485313+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:50.621127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485589+0000","last_change":"2023-12-19T18:17:27.347478+0000","last_active":"2023-12-19T18:17:28.485589+0000","last_peered":"2023-12-19T18:17:28.485589+0000","last_clean":"2023-12-19T18:17:28.485589+0000","last_became_active":"2023-12-19T18:17:27.346998+0000","last_became_peered":"2023-12-19T18:17:27.346998+0000","last_unstale":"2023-12-19T18:17:28.485589+0000","last_undegraded":"2023-12-19T18:17:28.485589+0000","last_fullsized":"2023-12-19T18:17:28.485589+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:22.470263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485549+0000","last_change":"2023-12-19T18:17:27.339790+0000","last_active":"2023-12-19T18:17:28.485549+0000","last_peered":"2023-12-19T18:17:28.485549+0000","last_clean":"2023-12-19T18:17:28.485549+0000","last_became_active":"2023-12-19T18:17:27.339477+0000","last_became_peered":"2023-12-19T18:17:27.339477+0000","last_unstale":"2023-12-19T18:17:28.485549+0000","last_undegraded":"2023-12-19T18:17:28.485549+0000","last_fullsized":"2023-12-19T18:17:28.485549+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:23:29.725587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.484833+0000","last_change":"2023-12-19T18:17:27.348543+0000","last_active":"2023-12-19T18:17:28.484833+0000","last_peered":"2023-12-19T18:17:28.484833+0000","last_clean":"2023-12-19T18:17:28.484833+0000","last_became_active":"2023-12-19T18:17:27.348257+0000","last_became_peered":"2023-12-19T18:17:27.348257+0000","last_unstale":"2023-12-19T18:17:28.484833+0000","last_undegraded":"2023-12-19T18:17:28.484833+0000","last_fullsized":"2023-12-19T18:17:28.484833+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:44:09.238317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486788+0000","last_change":"2023-12-19T18:17:27.353080+0000","last_active":"2023-12-19T18:17:28.486788+0000","last_peered":"2023-12-19T18:17:28.486788+0000","last_clean":"2023-12-19T18:17:28.486788+0000","last_became_active":"2023-12-19T18:17:27.352664+0000","last_became_peered":"2023-12-19T18:17:27.352664+0000","last_unstale":"2023-12-19T18:17:28.486788+0000","last_undegraded":"2023-12-19T18:17:28.486788+0000","last_fullsized":"2023-12-19T18:17:28.486788+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:22:10.869020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.338292+0000","last_change":"2023-12-19T18:17:27.367346+0000","last_active":"2023-12-19T18:17:28.338292+0000","last_peered":"2023-12-19T18:17:28.338292+0000","last_clean":"2023-12-19T18:17:28.338292+0000","last_became_active":"2023-12-19T18:17:27.367053+0000","last_became_peered":"2023-12-19T18:17:27.367053+0000","last_unstale":"2023-12-19T18:17:28.338292+0000","last_undegraded":"2023-12-19T18:17:28.338292+0000","last_fullsized":"2023-12-19T18:17:28.338292+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:44.734608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486154+0000","last_change":"2023-12-19T18:17:27.355178+0000","last_active":"2023-12-19T18:17:28.486154+0000","last_peered":"2023-12-19T18:17:28.486154+0000","last_clean":"2023-12-19T18:17:28.486154+0000","last_became_active":"2023-12-19T18:17:27.354771+0000","last_became_peered":"2023-12-19T18:17:27.354771+0000","last_unstale":"2023-12-19T18:17:28.486154+0000","last_undegraded":"2023-12-19T18:17:28.486154+0000","last_fullsized":"2023-12-19T18:17:28.486154+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:06.555153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485498+0000","last_change":"2023-12-19T18:17:27.349197+0000","last_active":"2023-12-19T18:17:28.485498+0000","last_peered":"2023-12-19T18:17:28.485498+0000","last_clean":"2023-12-19T18:17:28.485498+0000","last_became_active":"2023-12-19T18:17:27.348940+0000","last_became_peered":"2023-12-19T18:17:27.348940+0000","last_unstale":"2023-12-19T18:17:28.485498+0000","last_undegraded":"2023-12-19T18:17:28.485498+0000","last_fullsized":"2023-12-19T18:17:28.485498+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:19.968471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339651+0000","last_change":"2023-12-19T18:17:27.369700+0000","last_active":"2023-12-19T18:17:28.339651+0000","last_peered":"2023-12-19T18:17:28.339651+0000","last_clean":"2023-12-19T18:17:28.339651+0000","last_became_active":"2023-12-19T18:17:27.369398+0000","last_became_peered":"2023-12-19T18:17:27.369398+0000","last_unstale":"2023-12-19T18:17:28.339651+0000","last_undegraded":"2023-12-19T18:17:28.339651+0000","last_fullsized":"2023-12-19T18:17:28.339651+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:35:43.593726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485714+0000","last_change":"2023-12-19T18:17:27.360694+0000","last_active":"2023-12-19T18:17:28.485714+0000","last_peered":"2023-12-19T18:17:28.485714+0000","last_clean":"2023-12-19T18:17:28.485714+0000","last_became_active":"2023-12-19T18:17:27.357706+0000","last_became_peered":"2023-12-19T18:17:27.357706+0000","last_unstale":"2023-12-19T18:17:28.485714+0000","last_undegraded":"2023-12-19T18:17:28.485714+0000","last_fullsized":"2023-12-19T18:17:28.485714+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:22:01.458247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.484920+0000","last_change":"2023-12-19T18:17:27.361414+0000","last_active":"2023-12-19T18:17:28.484920+0000","last_peered":"2023-12-19T18:17:28.484920+0000","last_clean":"2023-12-19T18:17:28.484920+0000","last_became_active":"2023-12-19T18:17:27.361054+0000","last_became_peered":"2023-12-19T18:17:27.361054+0000","last_unstale":"2023-12-19T18:17:28.484920+0000","last_undegraded":"2023-12-19T18:17:28.484920+0000","last_fullsized":"2023-12-19T18:17:28.484920+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:04:33.062599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.484686+0000","last_change":"2023-12-19T18:17:27.359512+0000","last_active":"2023-12-19T18:17:28.484686+0000","last_peered":"2023-12-19T18:17:28.484686+0000","last_clean":"2023-12-19T18:17:28.484686+0000","last_became_active":"2023-12-19T18:17:27.359173+0000","last_became_peered":"2023-12-19T18:17:27.359173+0000","last_unstale":"2023-12-19T18:17:28.484686+0000","last_undegraded":"2023-12-19T18:17:28.484686+0000","last_fullsized":"2023-12-19T18:17:28.484686+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:42.446500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486554+0000","last_change":"2023-12-19T18:17:27.355393+0000","last_active":"2023-12-19T18:17:28.486554+0000","last_peered":"2023-12-19T18:17:28.486554+0000","last_clean":"2023-12-19T18:17:28.486554+0000","last_became_active":"2023-12-19T18:17:27.355013+0000","last_became_peered":"2023-12-19T18:17:27.355013+0000","last_unstale":"2023-12-19T18:17:28.486554+0000","last_undegraded":"2023-12-19T18:17:28.486554+0000","last_fullsized":"2023-12-19T18:17:28.486554+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:49.759094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"232'4","reported_seq":15,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.361758+0000","last_change":"2023-12-19T18:17:26.951474+0000","last_active":"2023-12-19T18:17:27.361758+0000","last_peered":"2023-12-19T18:17:27.361758+0000","last_clean":"2023-12-19T18:17:27.361758+0000","last_became_active":"2023-12-19T18:17:26.950990+0000","last_became_peered":"2023-12-19T18:17:26.950990+0000","last_unstale":"2023-12-19T18:17:27.361758+0000","last_undegraded":"2023-12-19T18:17:27.361758+0000","last_fullsized":"2023-12-19T18:17:27.361758+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:37:49.322125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.338181+0000","last_change":"2023-12-19T18:17:27.346026+0000","last_active":"2023-12-19T18:17:28.338181+0000","last_peered":"2023-12-19T18:17:28.338181+0000","last_clean":"2023-12-19T18:17:28.338181+0000","last_became_active":"2023-12-19T18:17:27.345752+0000","last_became_peered":"2023-12-19T18:17:27.345752+0000","last_unstale":"2023-12-19T18:17:28.338181+0000","last_undegraded":"2023-12-19T18:17:28.338181+0000","last_fullsized":"2023-12-19T18:17:28.338181+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:18.690231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.361889+0000","last_change":"2023-12-19T18:17:26.946867+0000","last_active":"2023-12-19T18:17:27.361889+0000","last_peered":"2023-12-19T18:17:27.361889+0000","last_clean":"2023-12-19T18:17:27.361889+0000","last_became_active":"2023-12-19T18:17:26.946549+0000","last_became_peered":"2023-12-19T18:17:26.946549+0000","last_unstale":"2023-12-19T18:17:27.361889+0000","last_undegraded":"2023-12-19T18:17:27.361889+0000","last_fullsized":"2023-12-19T18:17:27.361889+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:36.912904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339926+0000","last_change":"2023-12-19T18:17:27.369534+0000","last_active":"2023-12-19T18:17:28.339926+0000","last_peered":"2023-12-19T18:17:28.339926+0000","last_clean":"2023-12-19T18:17:28.339926+0000","last_became_active":"2023-12-19T18:17:27.369101+0000","last_became_peered":"2023-12-19T18:17:27.369101+0000","last_unstale":"2023-12-19T18:17:28.339926+0000","last_undegraded":"2023-12-19T18:17:28.339926+0000","last_fullsized":"2023-12-19T18:17:28.339926+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:38:39.157091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.358793+0000","last_change":"2023-12-19T18:17:26.947910+0000","last_active":"2023-12-19T18:17:27.358793+0000","last_peered":"2023-12-19T18:17:27.358793+0000","last_clean":"2023-12-19T18:17:27.358793+0000","last_became_active":"2023-12-19T18:17:26.946840+0000","last_became_peered":"2023-12-19T18:17:26.946840+0000","last_unstale":"2023-12-19T18:17:27.358793+0000","last_undegraded":"2023-12-19T18:17:27.358793+0000","last_fullsized":"2023-12-19T18:17:27.358793+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:41.372298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485942+0000","last_change":"2023-12-19T18:17:27.361418+0000","last_active":"2023-12-19T18:17:28.485942+0000","last_peered":"2023-12-19T18:17:28.485942+0000","last_clean":"2023-12-19T18:17:28.485942+0000","last_became_active":"2023-12-19T18:17:27.361131+0000","last_became_peered":"2023-12-19T18:17:27.361131+0000","last_unstale":"2023-12-19T18:17:28.485942+0000","last_undegraded":"2023-12-19T18:17:28.485942+0000","last_fullsized":"2023-12-19T18:17:28.485942+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:44:06.399365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"232'1","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.337991+0000","last_change":"2023-12-19T18:17:26.942982+0000","last_active":"2023-12-19T18:17:28.337991+0000","last_peered":"2023-12-19T18:17:28.337991+0000","last_clean":"2023-12-19T18:17:28.337991+0000","last_became_active":"2023-12-19T18:17:26.942719+0000","last_became_peered":"2023-12-19T18:17:26.942719+0000","last_unstale":"2023-12-19T18:17:28.337991+0000","last_undegraded":"2023-12-19T18:17:28.337991+0000","last_fullsized":"2023-12-19T18:17:28.337991+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:10.474482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485408+0000","last_change":"2023-12-19T18:17:27.361092+0000","last_active":"2023-12-19T18:17:28.485408+0000","last_peered":"2023-12-19T18:17:28.485408+0000","last_clean":"2023-12-19T18:17:28.485408+0000","last_became_active":"2023-12-19T18:17:27.360778+0000","last_became_peered":"2023-12-19T18:17:27.360778+0000","last_unstale":"2023-12-19T18:17:28.485408+0000","last_undegraded":"2023-12-19T18:17:28.485408+0000","last_fullsized":"2023-12-19T18:17:28.485408+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:40.489404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"232'1","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486223+0000","last_change":"2023-12-19T18:17:26.940960+0000","last_active":"2023-12-19T18:17:28.486223+0000","last_peered":"2023-12-19T18:17:28.486223+0000","last_clean":"2023-12-19T18:17:28.486223+0000","last_became_active":"2023-12-19T18:17:26.939915+0000","last_became_peered":"2023-12-19T18:17:26.939915+0000","last_unstale":"2023-12-19T18:17:28.486223+0000","last_undegraded":"2023-12-19T18:17:28.486223+0000","last_fullsized":"2023-12-19T18:17:28.486223+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:52:17.872321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.484852+0000","last_change":"2023-12-19T18:17:27.347577+0000","last_active":"2023-12-19T18:17:28.484852+0000","last_peered":"2023-12-19T18:17:28.484852+0000","last_clean":"2023-12-19T18:17:28.484852+0000","last_became_active":"2023-12-19T18:17:27.346549+0000","last_became_peered":"2023-12-19T18:17:27.346549+0000","last_unstale":"2023-12-19T18:17:28.484852+0000","last_undegraded":"2023-12-19T18:17:28.484852+0000","last_fullsized":"2023-12-19T18:17:28.484852+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:28.649300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.355042+0000","last_change":"2023-12-19T18:17:26.950047+0000","last_active":"2023-12-19T18:17:27.355042+0000","last_peered":"2023-12-19T18:17:27.355042+0000","last_clean":"2023-12-19T18:17:27.355042+0000","last_became_active":"2023-12-19T18:17:26.949678+0000","last_became_peered":"2023-12-19T18:17:26.949678+0000","last_unstale":"2023-12-19T18:17:27.355042+0000","last_undegraded":"2023-12-19T18:17:27.355042+0000","last_fullsized":"2023-12-19T18:17:27.355042+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:49:16.046354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339905+0000","last_change":"2023-12-19T18:17:27.363788+0000","last_active":"2023-12-19T18:17:28.339905+0000","last_peered":"2023-12-19T18:17:28.339905+0000","last_clean":"2023-12-19T18:17:28.339905+0000","last_became_active":"2023-12-19T18:17:27.363513+0000","last_became_peered":"2023-12-19T18:17:27.363513+0000","last_unstale":"2023-12-19T18:17:28.339905+0000","last_undegraded":"2023-12-19T18:17:28.339905+0000","last_fullsized":"2023-12-19T18:17:28.339905+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:23.278389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.330747+0000","last_change":"2023-12-19T18:17:26.944745+0000","last_active":"2023-12-19T18:17:27.330747+0000","last_peered":"2023-12-19T18:17:27.330747+0000","last_clean":"2023-12-19T18:17:27.330747+0000","last_became_active":"2023-12-19T18:17:26.944481+0000","last_became_peered":"2023-12-19T18:17:26.944481+0000","last_unstale":"2023-12-19T18:17:27.330747+0000","last_undegraded":"2023-12-19T18:17:27.330747+0000","last_fullsized":"2023-12-19T18:17:27.330747+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:30.604242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.340556+0000","last_change":"2023-12-19T18:17:27.356985+0000","last_active":"2023-12-19T18:17:28.340556+0000","last_peered":"2023-12-19T18:17:28.340556+0000","last_clean":"2023-12-19T18:17:28.340556+0000","last_became_active":"2023-12-19T18:17:27.356695+0000","last_became_peered":"2023-12-19T18:17:27.356695+0000","last_unstale":"2023-12-19T18:17:28.340556+0000","last_undegraded":"2023-12-19T18:17:28.340556+0000","last_fullsized":"2023-12-19T18:17:28.340556+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:31.153454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.335875+0000","last_change":"2023-12-19T18:17:26.944872+0000","last_active":"2023-12-19T18:17:27.335875+0000","last_peered":"2023-12-19T18:17:27.335875+0000","last_clean":"2023-12-19T18:17:27.335875+0000","last_became_active":"2023-12-19T18:17:26.944558+0000","last_became_peered":"2023-12-19T18:17:26.944558+0000","last_unstale":"2023-12-19T18:17:27.335875+0000","last_undegraded":"2023-12-19T18:17:27.335875+0000","last_fullsized":"2023-12-19T18:17:27.335875+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:30.151215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486857+0000","last_change":"2023-12-19T18:17:27.342915+0000","last_active":"2023-12-19T18:17:28.486857+0000","last_peered":"2023-12-19T18:17:28.486857+0000","last_clean":"2023-12-19T18:17:28.486857+0000","last_became_active":"2023-12-19T18:17:27.342549+0000","last_became_peered":"2023-12-19T18:17:27.342549+0000","last_unstale":"2023-12-19T18:17:28.486857+0000","last_undegraded":"2023-12-19T18:17:28.486857+0000","last_fullsized":"2023-12-19T18:17:28.486857+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:03.786746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.338131+0000","last_change":"2023-12-19T18:17:26.943010+0000","last_active":"2023-12-19T18:17:27.338131+0000","last_peered":"2023-12-19T18:17:27.338131+0000","last_clean":"2023-12-19T18:17:27.338131+0000","last_became_active":"2023-12-19T18:17:26.942721+0000","last_became_peered":"2023-12-19T18:17:26.942721+0000","last_unstale":"2023-12-19T18:17:27.338131+0000","last_undegraded":"2023-12-19T18:17:27.338131+0000","last_fullsized":"2023-12-19T18:17:27.338131+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:09:16.907014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339728+0000","last_change":"2023-12-19T18:17:27.368101+0000","last_active":"2023-12-19T18:17:28.339728+0000","last_peered":"2023-12-19T18:17:28.339728+0000","last_clean":"2023-12-19T18:17:28.339728+0000","last_became_active":"2023-12-19T18:17:27.367822+0000","last_became_peered":"2023-12-19T18:17:27.367822+0000","last_unstale":"2023-12-19T18:17:28.339728+0000","last_undegraded":"2023-12-19T18:17:28.339728+0000","last_fullsized":"2023-12-19T18:17:28.339728+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:11:20.286347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"232'3","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485669+0000","last_change":"2023-12-19T18:17:26.938268+0000","last_active":"2023-12-19T18:17:28.485669+0000","last_peered":"2023-12-19T18:17:28.485669+0000","last_clean":"2023-12-19T18:17:28.485669+0000","last_became_active":"2023-12-19T18:17:26.937724+0000","last_became_peered":"2023-12-19T18:17:26.937724+0000","last_unstale":"2023-12-19T18:17:28.485669+0000","last_undegraded":"2023-12-19T18:17:28.485669+0000","last_fullsized":"2023-12-19T18:17:28.485669+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:05.552913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486053+0000","last_change":"2023-12-19T18:17:27.344496+0000","last_active":"2023-12-19T18:17:28.486053+0000","last_peered":"2023-12-19T18:17:28.486053+0000","last_clean":"2023-12-19T18:17:28.486053+0000","last_became_active":"2023-12-19T18:17:27.343611+0000","last_became_peered":"2023-12-19T18:17:27.343611+0000","last_unstale":"2023-12-19T18:17:28.486053+0000","last_undegraded":"2023-12-19T18:17:28.486053+0000","last_fullsized":"2023-12-19T18:17:28.486053+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:26:28.887778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"232'6","reported_seq":17,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.365295+0000","last_change":"2023-12-19T18:17:26.941650+0000","last_active":"2023-12-19T18:17:27.365295+0000","last_peered":"2023-12-19T18:17:27.365295+0000","last_clean":"2023-12-19T18:17:27.365295+0000","last_became_active":"2023-12-19T18:17:26.941388+0000","last_became_peered":"2023-12-19T18:17:26.941388+0000","last_unstale":"2023-12-19T18:17:27.365295+0000","last_undegraded":"2023-12-19T18:17:27.365295+0000","last_fullsized":"2023-12-19T18:17:27.365295+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:35:14.834857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485891+0000","last_change":"2023-12-19T18:17:27.350307+0000","last_active":"2023-12-19T18:17:28.485891+0000","last_peered":"2023-12-19T18:17:28.485891+0000","last_clean":"2023-12-19T18:17:28.485891+0000","last_became_active":"2023-12-19T18:17:27.350036+0000","last_became_peered":"2023-12-19T18:17:27.350036+0000","last_unstale":"2023-12-19T18:17:28.485891+0000","last_undegraded":"2023-12-19T18:17:28.485891+0000","last_fullsized":"2023-12-19T18:17:28.485891+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:24:50.816850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"232'2","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.338048+0000","last_change":"2023-12-19T18:17:26.941112+0000","last_active":"2023-12-19T18:17:28.338048+0000","last_peered":"2023-12-19T18:17:28.338048+0000","last_clean":"2023-12-19T18:17:28.338048+0000","last_became_active":"2023-12-19T18:17:26.940838+0000","last_became_peered":"2023-12-19T18:17:26.940838+0000","last_unstale":"2023-12-19T18:17:28.338048+0000","last_undegraded":"2023-12-19T18:17:28.338048+0000","last_fullsized":"2023-12-19T18:17:28.338048+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:47:48.785673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486373+0000","last_change":"2023-12-19T18:17:27.362157+0000","last_active":"2023-12-19T18:17:28.486373+0000","last_peered":"2023-12-19T18:17:28.486373+0000","last_clean":"2023-12-19T18:17:28.486373+0000","last_became_active":"2023-12-19T18:17:27.361879+0000","last_became_peered":"2023-12-19T18:17:27.361879+0000","last_unstale":"2023-12-19T18:17:28.486373+0000","last_undegraded":"2023-12-19T18:17:28.486373+0000","last_fullsized":"2023-12-19T18:17:28.486373+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:00.178479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.338035+0000","last_change":"2023-12-19T18:17:26.948613+0000","last_active":"2023-12-19T18:17:27.338035+0000","last_peered":"2023-12-19T18:17:27.338035+0000","last_clean":"2023-12-19T18:17:27.338035+0000","last_became_active":"2023-12-19T18:17:26.948318+0000","last_became_peered":"2023-12-19T18:17:26.948318+0000","last_unstale":"2023-12-19T18:17:27.338035+0000","last_undegraded":"2023-12-19T18:17:27.338035+0000","last_fullsized":"2023-12-19T18:17:27.338035+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:13:19.247070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485463+0000","last_change":"2023-12-19T18:17:27.361736+0000","last_active":"2023-12-19T18:17:28.485463+0000","last_peered":"2023-12-19T18:17:28.485463+0000","last_clean":"2023-12-19T18:17:28.485463+0000","last_became_active":"2023-12-19T18:17:27.361355+0000","last_became_peered":"2023-12-19T18:17:27.361355+0000","last_unstale":"2023-12-19T18:17:28.485463+0000","last_undegraded":"2023-12-19T18:17:28.485463+0000","last_fullsized":"2023-12-19T18:17:28.485463+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:02.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":11,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485475+0000","last_change":"2023-12-19T18:17:26.944351+0000","last_active":"2023-12-19T18:17:28.485475+0000","last_peered":"2023-12-19T18:17:28.485475+0000","last_clean":"2023-12-19T18:17:28.485475+0000","last_became_active":"2023-12-19T18:17:26.943692+0000","last_became_peered":"2023-12-19T18:17:26.943692+0000","last_unstale":"2023-12-19T18:17:28.485475+0000","last_undegraded":"2023-12-19T18:17:28.485475+0000","last_fullsized":"2023-12-19T18:17:28.485475+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:54.040380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339242+0000","last_change":"2023-12-19T18:17:27.366044+0000","last_active":"2023-12-19T18:17:28.339242+0000","last_peered":"2023-12-19T18:17:28.339242+0000","last_clean":"2023-12-19T18:17:28.339242+0000","last_became_active":"2023-12-19T18:17:27.365761+0000","last_became_peered":"2023-12-19T18:17:27.365761+0000","last_unstale":"2023-12-19T18:17:28.339242+0000","last_undegraded":"2023-12-19T18:17:28.339242+0000","last_fullsized":"2023-12-19T18:17:28.339242+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:05:07.652398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"232'8","reported_seq":19,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.365469+0000","last_change":"2023-12-19T18:17:26.948258+0000","last_active":"2023-12-19T18:17:27.365469+0000","last_peered":"2023-12-19T18:17:27.365469+0000","last_clean":"2023-12-19T18:17:27.365469+0000","last_became_active":"2023-12-19T18:17:26.947974+0000","last_became_peered":"2023-12-19T18:17:26.947974+0000","last_unstale":"2023-12-19T18:17:27.365469+0000","last_undegraded":"2023-12-19T18:17:27.365469+0000","last_fullsized":"2023-12-19T18:17:27.365469+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:49.244280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.338382+0000","last_change":"2023-12-19T18:17:27.349954+0000","last_active":"2023-12-19T18:17:28.338382+0000","last_peered":"2023-12-19T18:17:28.338382+0000","last_clean":"2023-12-19T18:17:28.338382+0000","last_became_active":"2023-12-19T18:17:27.349694+0000","last_became_peered":"2023-12-19T18:17:27.349694+0000","last_unstale":"2023-12-19T18:17:28.338382+0000","last_undegraded":"2023-12-19T18:17:28.338382+0000","last_fullsized":"2023-12-19T18:17:28.338382+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:36:07.296035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"232'1","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486953+0000","last_change":"2023-12-19T18:17:26.944980+0000","last_active":"2023-12-19T18:17:28.486953+0000","last_peered":"2023-12-19T18:17:28.486953+0000","last_clean":"2023-12-19T18:17:28.486953+0000","last_became_active":"2023-12-19T18:17:26.943863+0000","last_became_peered":"2023-12-19T18:17:26.943863+0000","last_unstale":"2023-12-19T18:17:28.486953+0000","last_undegraded":"2023-12-19T18:17:28.486953+0000","last_fullsized":"2023-12-19T18:17:28.486953+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:38:38.154851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485305+0000","last_change":"2023-12-19T18:17:27.364634+0000","last_active":"2023-12-19T18:17:28.485305+0000","last_peered":"2023-12-19T18:17:28.485305+0000","last_clean":"2023-12-19T18:17:28.485305+0000","last_became_active":"2023-12-19T18:17:27.364357+0000","last_became_peered":"2023-12-19T18:17:27.364357+0000","last_unstale":"2023-12-19T18:17:28.485305+0000","last_undegraded":"2023-12-19T18:17:28.485305+0000","last_fullsized":"2023-12-19T18:17:28.485305+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:05:35.230888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.354463+0000","last_change":"2023-12-19T18:17:26.938178+0000","last_active":"2023-12-19T18:17:27.354463+0000","last_peered":"2023-12-19T18:17:27.354463+0000","last_clean":"2023-12-19T18:17:27.354463+0000","last_became_active":"2023-12-19T18:17:26.937910+0000","last_became_peered":"2023-12-19T18:17:26.937910+0000","last_unstale":"2023-12-19T18:17:27.354463+0000","last_undegraded":"2023-12-19T18:17:27.354463+0000","last_fullsized":"2023-12-19T18:17:27.354463+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:19:13.215706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486340+0000","last_change":"2023-12-19T18:17:27.360665+0000","last_active":"2023-12-19T18:17:28.486340+0000","last_peered":"2023-12-19T18:17:28.486340+0000","last_clean":"2023-12-19T18:17:28.486340+0000","last_became_active":"2023-12-19T18:17:27.360397+0000","last_became_peered":"2023-12-19T18:17:27.360397+0000","last_unstale":"2023-12-19T18:17:28.486340+0000","last_undegraded":"2023-12-19T18:17:28.486340+0000","last_fullsized":"2023-12-19T18:17:28.486340+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:33.393492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.361613+0000","last_change":"2023-12-19T18:17:26.946790+0000","last_active":"2023-12-19T18:17:27.361613+0000","last_peered":"2023-12-19T18:17:27.361613+0000","last_clean":"2023-12-19T18:17:27.361613+0000","last_became_active":"2023-12-19T18:17:26.946487+0000","last_became_peered":"2023-12-19T18:17:26.946487+0000","last_unstale":"2023-12-19T18:17:27.361613+0000","last_undegraded":"2023-12-19T18:17:27.361613+0000","last_fullsized":"2023-12-19T18:17:27.361613+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:49.419114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486712+0000","last_change":"2023-12-19T18:17:27.365652+0000","last_active":"2023-12-19T18:17:28.486712+0000","last_peered":"2023-12-19T18:17:28.486712+0000","last_clean":"2023-12-19T18:17:28.486712+0000","last_became_active":"2023-12-19T18:17:27.365336+0000","last_became_peered":"2023-12-19T18:17:27.365336+0000","last_unstale":"2023-12-19T18:17:28.486712+0000","last_undegraded":"2023-12-19T18:17:28.486712+0000","last_fullsized":"2023-12-19T18:17:28.486712+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:54.566933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.331525+0000","last_change":"2023-12-19T18:17:26.942091+0000","last_active":"2023-12-19T18:17:27.331525+0000","last_peered":"2023-12-19T18:17:27.331525+0000","last_clean":"2023-12-19T18:17:27.331525+0000","last_became_active":"2023-12-19T18:17:26.941806+0000","last_became_peered":"2023-12-19T18:17:26.941806+0000","last_unstale":"2023-12-19T18:17:27.331525+0000","last_undegraded":"2023-12-19T18:17:27.331525+0000","last_fullsized":"2023-12-19T18:17:27.331525+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:27:28.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485134+0000","last_change":"2023-12-19T18:17:27.367095+0000","last_active":"2023-12-19T18:17:28.485134+0000","last_peered":"2023-12-19T18:17:28.485134+0000","last_clean":"2023-12-19T18:17:28.485134+0000","last_became_active":"2023-12-19T18:17:27.366441+0000","last_became_peered":"2023-12-19T18:17:27.366441+0000","last_unstale":"2023-12-19T18:17:28.485134+0000","last_undegraded":"2023-12-19T18:17:28.485134+0000","last_fullsized":"2023-12-19T18:17:28.485134+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:10:47.532188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"232'2","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486748+0000","last_change":"2023-12-19T18:17:26.948576+0000","last_active":"2023-12-19T18:17:28.486748+0000","last_peered":"2023-12-19T18:17:28.486748+0000","last_clean":"2023-12-19T18:17:28.486748+0000","last_became_active":"2023-12-19T18:17:26.948310+0000","last_became_peered":"2023-12-19T18:17:26.948310+0000","last_unstale":"2023-12-19T18:17:28.486748+0000","last_undegraded":"2023-12-19T18:17:28.486748+0000","last_fullsized":"2023-12-19T18:17:28.486748+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:58.733231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486393+0000","last_change":"2023-12-19T18:17:27.358552+0000","last_active":"2023-12-19T18:17:28.486393+0000","last_peered":"2023-12-19T18:17:28.486393+0000","last_clean":"2023-12-19T18:17:28.486393+0000","last_became_active":"2023-12-19T18:17:27.358174+0000","last_became_peered":"2023-12-19T18:17:27.358174+0000","last_unstale":"2023-12-19T18:17:28.486393+0000","last_undegraded":"2023-12-19T18:17:28.486393+0000","last_fullsized":"2023-12-19T18:17:28.486393+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:14.654686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.354913+0000","last_change":"2023-12-19T18:17:26.947007+0000","last_active":"2023-12-19T18:17:27.354913+0000","last_peered":"2023-12-19T18:17:27.354913+0000","last_clean":"2023-12-19T18:17:27.354913+0000","last_became_active":"2023-12-19T18:17:26.946741+0000","last_became_peered":"2023-12-19T18:17:26.946741+0000","last_unstale":"2023-12-19T18:17:27.354913+0000","last_undegraded":"2023-12-19T18:17:27.354913+0000","last_fullsized":"2023-12-19T18:17:27.354913+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:20.105057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.337760+0000","last_change":"2023-12-19T18:17:27.348563+0000","last_active":"2023-12-19T18:17:28.337760+0000","last_peered":"2023-12-19T18:17:28.337760+0000","last_clean":"2023-12-19T18:17:28.337760+0000","last_became_active":"2023-12-19T18:17:27.348275+0000","last_became_peered":"2023-12-19T18:17:27.348275+0000","last_unstale":"2023-12-19T18:17:28.337760+0000","last_undegraded":"2023-12-19T18:17:28.337760+0000","last_fullsized":"2023-12-19T18:17:28.337760+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:45:15.369206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.355147+0000","last_change":"2023-12-19T18:17:26.944588+0000","last_active":"2023-12-19T18:17:27.355147+0000","last_peered":"2023-12-19T18:17:27.355147+0000","last_clean":"2023-12-19T18:17:27.355147+0000","last_became_active":"2023-12-19T18:17:26.944323+0000","last_became_peered":"2023-12-19T18:17:26.944323+0000","last_unstale":"2023-12-19T18:17:27.355147+0000","last_undegraded":"2023-12-19T18:17:27.355147+0000","last_fullsized":"2023-12-19T18:17:27.355147+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:25:45.208073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.484920+0000","last_change":"2023-12-19T18:17:27.342936+0000","last_active":"2023-12-19T18:17:28.484920+0000","last_peered":"2023-12-19T18:17:28.484920+0000","last_clean":"2023-12-19T18:17:28.484920+0000","last_became_active":"2023-12-19T18:17:27.342636+0000","last_became_peered":"2023-12-19T18:17:27.342636+0000","last_unstale":"2023-12-19T18:17:28.484920+0000","last_undegraded":"2023-12-19T18:17:28.484920+0000","last_fullsized":"2023-12-19T18:17:28.484920+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:43.945499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":11,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485577+0000","last_change":"2023-12-19T18:17:26.950102+0000","last_active":"2023-12-19T18:17:28.485577+0000","last_peered":"2023-12-19T18:17:28.485577+0000","last_clean":"2023-12-19T18:17:28.485577+0000","last_became_active":"2023-12-19T18:17:26.949810+0000","last_became_peered":"2023-12-19T18:17:26.949810+0000","last_unstale":"2023-12-19T18:17:28.485577+0000","last_undegraded":"2023-12-19T18:17:28.485577+0000","last_fullsized":"2023-12-19T18:17:28.485577+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:04:02.387088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485712+0000","last_change":"2023-12-19T18:17:27.350933+0000","last_active":"2023-12-19T18:17:28.485712+0000","last_peered":"2023-12-19T18:17:28.485712+0000","last_clean":"2023-12-19T18:17:28.485712+0000","last_became_active":"2023-12-19T18:17:27.350661+0000","last_became_peered":"2023-12-19T18:17:27.350661+0000","last_unstale":"2023-12-19T18:17:28.485712+0000","last_undegraded":"2023-12-19T18:17:28.485712+0000","last_fullsized":"2023-12-19T18:17:28.485712+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:25.159077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"232'2","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485792+0000","last_change":"2023-12-19T18:17:26.944158+0000","last_active":"2023-12-19T18:17:28.485792+0000","last_peered":"2023-12-19T18:17:28.485792+0000","last_clean":"2023-12-19T18:17:28.485792+0000","last_became_active":"2023-12-19T18:17:26.943845+0000","last_became_peered":"2023-12-19T18:17:26.943845+0000","last_unstale":"2023-12-19T18:17:28.485792+0000","last_undegraded":"2023-12-19T18:17:28.485792+0000","last_fullsized":"2023-12-19T18:17:28.485792+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:10.097246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486908+0000","last_change":"2023-12-19T18:17:27.361893+0000","last_active":"2023-12-19T18:17:28.486908+0000","last_peered":"2023-12-19T18:17:28.486908+0000","last_clean":"2023-12-19T18:17:28.486908+0000","last_became_active":"2023-12-19T18:17:27.361618+0000","last_became_peered":"2023-12-19T18:17:27.361618+0000","last_unstale":"2023-12-19T18:17:28.486908+0000","last_undegraded":"2023-12-19T18:17:28.486908+0000","last_fullsized":"2023-12-19T18:17:28.486908+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:50.421353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.334821+0000","last_change":"2023-12-19T18:17:26.944398+0000","last_active":"2023-12-19T18:17:27.334821+0000","last_peered":"2023-12-19T18:17:27.334821+0000","last_clean":"2023-12-19T18:17:27.334821+0000","last_became_active":"2023-12-19T18:17:26.942848+0000","last_became_peered":"2023-12-19T18:17:26.942848+0000","last_unstale":"2023-12-19T18:17:27.334821+0000","last_undegraded":"2023-12-19T18:17:27.334821+0000","last_fullsized":"2023-12-19T18:17:27.334821+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:01.415653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486648+0000","last_change":"2023-12-19T18:17:27.359609+0000","last_active":"2023-12-19T18:17:28.486648+0000","last_peered":"2023-12-19T18:17:28.486648+0000","last_clean":"2023-12-19T18:17:28.486648+0000","last_became_active":"2023-12-19T18:17:27.358162+0000","last_became_peered":"2023-12-19T18:17:27.358162+0000","last_unstale":"2023-12-19T18:17:28.486648+0000","last_undegraded":"2023-12-19T18:17:28.486648+0000","last_fullsized":"2023-12-19T18:17:28.486648+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:31:06.407037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.335792+0000","last_change":"2023-12-19T18:17:26.941195+0000","last_active":"2023-12-19T18:17:27.335792+0000","last_peered":"2023-12-19T18:17:27.335792+0000","last_clean":"2023-12-19T18:17:27.335792+0000","last_became_active":"2023-12-19T18:17:26.940845+0000","last_became_peered":"2023-12-19T18:17:26.940845+0000","last_unstale":"2023-12-19T18:17:27.335792+0000","last_undegraded":"2023-12-19T18:17:27.335792+0000","last_fullsized":"2023-12-19T18:17:27.335792+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:22.276149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485458+0000","last_change":"2023-12-19T18:17:27.351030+0000","last_active":"2023-12-19T18:17:28.485458+0000","last_peered":"2023-12-19T18:17:28.485458+0000","last_clean":"2023-12-19T18:17:28.485458+0000","last_became_active":"2023-12-19T18:17:27.350692+0000","last_became_peered":"2023-12-19T18:17:27.350692+0000","last_unstale":"2023-12-19T18:17:28.485458+0000","last_undegraded":"2023-12-19T18:17:28.485458+0000","last_fullsized":"2023-12-19T18:17:28.485458+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:15:54.445257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":11,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486754+0000","last_change":"2023-12-19T18:17:26.943936+0000","last_active":"2023-12-19T18:17:28.486754+0000","last_peered":"2023-12-19T18:17:28.486754+0000","last_clean":"2023-12-19T18:17:28.486754+0000","last_became_active":"2023-12-19T18:17:26.943609+0000","last_became_peered":"2023-12-19T18:17:26.943609+0000","last_unstale":"2023-12-19T18:17:28.486754+0000","last_undegraded":"2023-12-19T18:17:28.486754+0000","last_fullsized":"2023-12-19T18:17:28.486754+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:12.547671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339426+0000","last_change":"2023-12-19T18:17:27.342408+0000","last_active":"2023-12-19T18:17:28.339426+0000","last_peered":"2023-12-19T18:17:28.339426+0000","last_clean":"2023-12-19T18:17:28.339426+0000","last_became_active":"2023-12-19T18:17:27.342114+0000","last_became_peered":"2023-12-19T18:17:27.342114+0000","last_unstale":"2023-12-19T18:17:28.339426+0000","last_undegraded":"2023-12-19T18:17:28.339426+0000","last_fullsized":"2023-12-19T18:17:28.339426+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:02.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.354734+0000","last_change":"2023-12-19T18:17:26.945192+0000","last_active":"2023-12-19T18:17:27.354734+0000","last_peered":"2023-12-19T18:17:27.354734+0000","last_clean":"2023-12-19T18:17:27.354734+0000","last_became_active":"2023-12-19T18:17:26.943303+0000","last_became_peered":"2023-12-19T18:17:26.943303+0000","last_unstale":"2023-12-19T18:17:27.354734+0000","last_undegraded":"2023-12-19T18:17:27.354734+0000","last_fullsized":"2023-12-19T18:17:27.354734+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486521+0000","last_change":"2023-12-19T18:17:27.362825+0000","last_active":"2023-12-19T18:17:28.486521+0000","last_peered":"2023-12-19T18:17:28.486521+0000","last_clean":"2023-12-19T18:17:28.486521+0000","last_became_active":"2023-12-19T18:17:27.362414+0000","last_became_peered":"2023-12-19T18:17:27.362414+0000","last_unstale":"2023-12-19T18:17:28.486521+0000","last_undegraded":"2023-12-19T18:17:28.486521+0000","last_fullsized":"2023-12-19T18:17:28.486521+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:21:44.939055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.338128+0000","last_change":"2023-12-19T18:17:26.940233+0000","last_active":"2023-12-19T18:17:27.338128+0000","last_peered":"2023-12-19T18:17:27.338128+0000","last_clean":"2023-12-19T18:17:27.338128+0000","last_became_active":"2023-12-19T18:17:26.939941+0000","last_became_peered":"2023-12-19T18:17:26.939941+0000","last_unstale":"2023-12-19T18:17:27.338128+0000","last_undegraded":"2023-12-19T18:17:27.338128+0000","last_fullsized":"2023-12-19T18:17:27.338128+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:15:20.805126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485559+0000","last_change":"2023-12-19T18:17:27.356126+0000","last_active":"2023-12-19T18:17:28.485559+0000","last_peered":"2023-12-19T18:17:28.485559+0000","last_clean":"2023-12-19T18:17:28.485559+0000","last_became_active":"2023-12-19T18:17:27.355794+0000","last_became_peered":"2023-12-19T18:17:27.355794+0000","last_unstale":"2023-12-19T18:17:28.485559+0000","last_undegraded":"2023-12-19T18:17:28.485559+0000","last_fullsized":"2023-12-19T18:17:28.485559+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:23.278389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":11,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485103+0000","last_change":"2023-12-19T18:17:26.944096+0000","last_active":"2023-12-19T18:17:28.485103+0000","last_peered":"2023-12-19T18:17:28.485103+0000","last_clean":"2023-12-19T18:17:28.485103+0000","last_became_active":"2023-12-19T18:17:26.941219+0000","last_became_peered":"2023-12-19T18:17:26.941219+0000","last_unstale":"2023-12-19T18:17:28.485103+0000","last_undegraded":"2023-12-19T18:17:28.485103+0000","last_fullsized":"2023-12-19T18:17:28.485103+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:22:09.866781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485551+0000","last_change":"2023-12-19T18:17:27.343300+0000","last_active":"2023-12-19T18:17:28.485551+0000","last_peered":"2023-12-19T18:17:28.485551+0000","last_clean":"2023-12-19T18:17:28.485551+0000","last_became_active":"2023-12-19T18:17:27.343020+0000","last_became_peered":"2023-12-19T18:17:27.343020+0000","last_unstale":"2023-12-19T18:17:28.485551+0000","last_undegraded":"2023-12-19T18:17:28.485551+0000","last_fullsized":"2023-12-19T18:17:28.485551+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:11.099486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":11,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485651+0000","last_change":"2023-12-19T18:17:26.950561+0000","last_active":"2023-12-19T18:17:28.485651+0000","last_peered":"2023-12-19T18:17:28.485651+0000","last_clean":"2023-12-19T18:17:28.485651+0000","last_became_active":"2023-12-19T18:17:26.949052+0000","last_became_peered":"2023-12-19T18:17:26.949052+0000","last_unstale":"2023-12-19T18:17:28.485651+0000","last_undegraded":"2023-12-19T18:17:28.485651+0000","last_fullsized":"2023-12-19T18:17:28.485651+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:09:01.022466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486079+0000","last_change":"2023-12-19T18:17:27.349332+0000","last_active":"2023-12-19T18:17:28.486079+0000","last_peered":"2023-12-19T18:17:28.486079+0000","last_clean":"2023-12-19T18:17:28.486079+0000","last_became_active":"2023-12-19T18:17:27.349050+0000","last_became_peered":"2023-12-19T18:17:27.349050+0000","last_unstale":"2023-12-19T18:17:28.486079+0000","last_undegraded":"2023-12-19T18:17:28.486079+0000","last_fullsized":"2023-12-19T18:17:28.486079+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:33.501865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.369367+0000","last_change":"2023-12-19T18:17:26.947941+0000","last_active":"2023-12-19T18:17:27.369367+0000","last_peered":"2023-12-19T18:17:27.369367+0000","last_clean":"2023-12-19T18:17:27.369367+0000","last_became_active":"2023-12-19T18:17:26.945013+0000","last_became_peered":"2023-12-19T18:17:26.945013+0000","last_unstale":"2023-12-19T18:17:27.369367+0000","last_undegraded":"2023-12-19T18:17:27.369367+0000","last_fullsized":"2023-12-19T18:17:27.369367+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:47:39.421442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"218'368","reported_seq":1280,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.333681+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:17:27.333681+0000","last_peered":"2023-12-19T18:17:27.333681+0000","last_clean":"2023-12-19T18:17:27.333681+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:17:27.333681+0000","last_undegraded":"2023-12-19T18:17:27.333681+0000","last_fullsized":"2023-12-19T18:17:27.333681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2306,"num_read_kb":4823,"num_write":615,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2306,"num_read_kb":4823,"num_write":615,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":368,"ondisk_log_size":368,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510577,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48436,"kb_used_data":3436,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323404,"statfs":{"total":96636764160,"available":96587165696,"internally_reserved":0,"allocated":3518464,"data_stored":2147027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68500000000000005,"15min":0.71499999999999997},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79000000000000004,"5min":1.048,"15min":1.1180000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71299999999999997,"15min":0.73199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90900000000000003,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":1.0389999999999999,"15min":1.0580000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.74199999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.69099999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.73099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.0669999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.996,"5min":1.0820000000000001,"15min":1.236},"last":0.73399999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88100000000000001,"5min":1.002,"15min":1.079},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.49399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85899999999999999,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.79200000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.441,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.302},"last":0.755},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.622,"15min":0.629},"min":{"1min":0.50800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.0209999999999999},"last":0.71299999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.90600000000000003,"5min":1.097,"15min":1.1299999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.55800000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95699999999999996,"5min":1.026,"15min":1.0800000000000001},"last":0.624}]},{"osd":6,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.61699999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.0469999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.093},"last":0.80800000000000005}]}]},{"osd":6,"up_from":15,"seq":64424510577,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47860,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323980,"statfs":{"total":96636764160,"available":96587755520,"internally_reserved":0,"allocated":2928640,"data_stored":1556645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72299999999999998,"15min":0.73799999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.042},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.42799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97599999999999998,"5min":1.224,"15min":1.224},"last":0.86099999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68899999999999995,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86699999999999999,"5min":1.103,"15min":1.103},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.745,"15min":0.752},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":0.93600000000000005,"5min":0.97999999999999998,"15min":1.1259999999999999},"last":0.96099999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.752,"15min":0.76000000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98399999999999999,"5min":1.0549999999999999,"15min":1.0609999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70899999999999996,"15min":0.745},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98999999999999999,"5min":1.0089999999999999,"15min":1.111},"last":0.78600000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.754,"15min":0.77800000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.1519999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71599999999999997,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.036,"5min":1.268,"15min":1.268},"last":0.876}]},{"osd":4,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.373,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0469999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63200000000000001,"15min":0.61599999999999999},"min":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.55300000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.47599999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.997,"5min":1.194,"15min":1.194},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68999999999999995,"15min":0.67100000000000004},"min":{"1min":0.46800000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1339999999999999,"5min":1.141,"15min":1.141},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56599999999999995,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.54600000000000004,"5min":0.377,"15min":0.377},"max":{"1min":1.1140000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.82999999999999996}]}]},{"osd":5,"up_from":15,"seq":64424510577,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47852,"kb_used_data":2852,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323988,"statfs":{"total":96636764160,"available":96587763712,"internally_reserved":0,"allocated":2920448,"data_stored":1556589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.69599999999999995,"15min":0.72499999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.94999999999999996,"5min":1.002,"15min":1.0149999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.93100000000000005,"5min":0.96099999999999997,"15min":1.0229999999999999},"last":0.628}]},{"osd":1,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.497},"max":{"1min":0.84599999999999997,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.498,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97599999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59699999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.58599999999999997,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":1.0069999999999999,"5min":1.0509999999999999,"15min":1.2250000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.73199999999999998},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88600000000000001,"5min":1.004,"15min":1.4279999999999999},"last":0.76100000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.748,"15min":0.76200000000000001},"min":{"1min":0.56799999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.90400000000000003,"5min":1.0129999999999999,"15min":1.05},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93400000000000005,"5min":1.022,"15min":1.0329999999999999},"last":0.65300000000000002}]},{"osd":4,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.81699999999999995,"5min":0.94499999999999995,"15min":1.0800000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94299999999999995,"5min":1.208,"15min":1.208},"last":0.79800000000000004}]},{"osd":6,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65800000000000003,"15min":0.65700000000000003},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.85799999999999998,"5min":1.008,"15min":1.0449999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.501,"5min":0.439,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":1.169,"15min":1.169},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51100000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.079,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81899999999999995}]}]},{"osd":4,"up_from":15,"seq":64424510577,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47856,"kb_used_data":2856,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323984,"statfs":{"total":96636764160,"available":96587759616,"internally_reserved":0,"allocated":2924544,"data_stored":1557171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.51600000000000001,"5min":0.45800000000000002,"15min":0.45000000000000001},"max":{"1min":1.016,"5min":1.036,"15min":1.109},"last":0.621},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.502,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.91800000000000004,"5min":0.997,"15min":1.01},"last":0.64000000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.59199999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94699999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.997,"15min":1.1020000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79100000000000004,"15min":0.77700000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.47699999999999998},"max":{"1min":0.998,"5min":1.0289999999999999,"15min":1.071},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.78100000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76700000000000002,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93899999999999995,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.55500000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.59599999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65300000000000002,"15min":0.63100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67200000000000004,"15min":0.64600000000000002},"min":{"1min":0.495,"5min":0.42899999999999999,"15min":0.39600000000000002},"max":{"1min":0.88900000000000001,"5min":1.042,"15min":1.042},"last":0.80200000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.66100000000000003},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":1.024,"15min":1.0669999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0760000000000001},"last":0.83299999999999996}]},{"osd":7,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.41899999999999998},"max":{"1min":0.92400000000000004,"5min":1.087,"15min":1.087},"last":0.748},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77600000000000002,"15min":0.76200000000000001},"min":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.46999999999999997},"max":{"1min":0.98599999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.81999999999999995}]}]},{"osd":3,"up_from":15,"seq":64424510577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47868,"kb_used_data":2868,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323972,"statfs":{"total":96636764160,"available":96587747328,"internally_reserved":0,"allocated":2936832,"data_stored":1558223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.63200000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.502,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74099999999999999,"5min":0.78300000000000003,"15min":0.80000000000000004},"last":0.40999999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.498},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71499999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.77000000000000002,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81599999999999995,"15min":0.82799999999999996},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.012,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.81999999999999995,"15min":0.82299999999999995},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.56599999999999995},"max":{"1min":0.86499999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.81399999999999995}]},{"osd":5,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82799999999999996,"15min":0.84199999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99099999999999999,"5min":1.1739999999999999,"15min":1.397},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.59399999999999997,"5min":0.55900000000000005,"15min":0.55100000000000005},"max":{"1min":0.91000000000000003,"5min":1.2230000000000001,"15min":1.262},"last":0.72199999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.83899999999999997,"15min":0.84799999999999998},"min":{"1min":0.60199999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.96099999999999997,"5min":1.234,"15min":1.274},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83799999999999997,"15min":0.84499999999999997},"min":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94599999999999995,"5min":1.165,"15min":1.302},"last":0.748}]},{"osd":7,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0640000000000001,"5min":1.3540000000000001,"15min":1.486},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89500000000000002},"min":{"1min":0.68899999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.93000000000000005,"5min":1.421,"15min":1.421},"last":0.89600000000000002}]}]},{"osd":2,"up_from":15,"seq":64424510576,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47872,"kb_used_data":2872,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323968,"statfs":{"total":96636764160,"available":96587743232,"internally_reserved":0,"allocated":2940928,"data_stored":1558209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.441,"15min":0.45000000000000001},"min":{"1min":0.26300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.70499999999999996,"5min":0.754,"15min":0.754},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48899999999999999,"15min":0.504},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68000000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.45200000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59899999999999998,"15min":0.61599999999999999},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"max":{"1min":0.77800000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.73899999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41399999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58599999999999997,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76800000000000002,"5min":0.80800000000000005,"15min":0.95999999999999996},"last":0.5},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.36899999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.875},"last":0.54500000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0580000000000001,"5min":1.0880000000000001,"15min":1.306},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77500000000000002,"15min":0.79200000000000004},"min":{"1min":0.48299999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0609999999999999,"5min":1.2070000000000001,"15min":1.466},"last":0.70299999999999996}]},{"osd":5,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.77200000000000002,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.3049999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.80000000000000004,"15min":0.81899999999999995},"min":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.04,"5min":1.1519999999999999,"15min":1.429},"last":0.752}]},{"osd":6,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.81299999999999994,"15min":0.83799999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0329999999999999,"5min":1.216,"15min":1.4119999999999999},"last":0.88},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77600000000000002,"15min":0.80000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.008,"5min":1.117,"15min":1.2470000000000001},"last":0.90200000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.82199999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.56000000000000005},"max":{"1min":1.0269999999999999,"5min":1.159,"15min":1.45},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.83899999999999997,"15min":0.85599999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.093,"5min":1.276,"15min":1.276},"last":0.84699999999999998}]}]},{"osd":1,"up_from":15,"seq":64424510577,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60144,"kb_used_data":2856,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311696,"statfs":{"total":96636764160,"available":96575176704,"internally_reserved":0,"allocated":2924544,"data_stored":1556679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44800000000000001,"15min":0.45800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56399999999999995,"5min":0.71999999999999997,"15min":0.89400000000000002},"last":0.371},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79600000000000004,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.495}]},{"osd":2,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66000000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.66600000000000004,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.41499999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.622,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.747,"5min":0.94199999999999995,"15min":0.94499999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.628,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.78100000000000003,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92700000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77700000000000002,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.081,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.76000000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76900000000000002,"15min":0.80000000000000004},"min":{"1min":0.58699999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0189999999999999,"5min":1.236,"15min":1.2609999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81799999999999995},"min":{"1min":0.58199999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0669999999999999,"5min":1.353,"15min":1.353},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.505},"max":{"1min":1.0009999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.82399999999999995,"15min":0.84099999999999997},"min":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.034,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.84499999999999997}]},{"osd":7,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84599999999999997,"15min":0.85399999999999998},"min":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.56499999999999995},"max":{"1min":1.147,"5min":1.399,"15min":1.399},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.82799999999999996,"15min":0.85199999999999998},"min":{"1min":0.53000000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.181,"5min":1.365,"15min":1.365},"last":0.89500000000000002}]}]},{"osd":0,"up_from":14,"seq":60129543282,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60728,"kb_used_data":3440,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311112,"statfs":{"total":96636764160,"available":96574578688,"internally_reserved":0,"allocated":3522560,"data_stored":2147629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41899999999999998,"15min":0.45300000000000001},"min":{"1min":0.29699999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55300000000000005,"5min":0.69299999999999995,"15min":1.216},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.49399999999999999,"15min":0.51900000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67600000000000005,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.48999999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.48799999999999999,"15min":0.52600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.77000000000000002,"15min":1.1699999999999999},"last":0.441},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52300000000000002,"15min":0.54700000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.96799999999999997},"last":0.54400000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.51400000000000001,"15min":0.55000000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.59999999999999998,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59899999999999998,"15min":0.64400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.032},"last":0.47299999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.80600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.496},"max":{"1min":0.93899999999999995,"5min":1.1200000000000001,"15min":1.381},"last":0.873},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.78600000000000003,"15min":0.81100000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.47799999999999998},"max":{"1min":0.84799999999999998,"5min":1.137,"15min":1.2849999999999999},"last":0.84799999999999998}]},{"osd":5,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.93799999999999994,"5min":1.1180000000000001,"15min":1.3420000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.79100000000000004,"15min":0.81299999999999994},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.052,"5min":1.1259999999999999,"15min":1.288},"last":0.83699999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81200000000000006,"15min":0.83899999999999997},"min":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.51300000000000001},"max":{"1min":0.97899999999999998,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.80000000000000004,"15min":0.82699999999999996},"min":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97999999999999998,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.85599999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0149999999999999,"5min":1.1899999999999999,"15min":1.415},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.82599999999999996,"15min":0.85099999999999998},"min":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96899999999999997,"5min":1.0980000000000001,"15min":1.395},"last":0.89000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:17:31.441 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:17:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.613+0000 7fdfb2de9640 1 Processor -- start 2023-12-19T18:17:31.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.613+0000 7fdfb2de9640 1 -- start start 2023-12-19T18:17:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.613+0000 7fdfb2de9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfac154220 0x7fdfac15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.613+0000 7fdfb2de9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdfac06b3d0 0x7fdfac0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.613+0000 7fdfb2de9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfac06bc30 con 0x7fdfac0cec40 2023-12-19T18:17:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.613+0000 7fdfb2de9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfac0cf0d0 con 0x7fdfac06b3d0 2023-12-19T18:17:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.613+0000 7fdfb2de9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac0cc6e0 con 0x7fdfac154220 2023-12-19T18:17:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.613+0000 7fdfabfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdfac06b3d0 0x7fdfac0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.613+0000 7fdfb1386640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdfac0cec40 0x7fdfac15d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47662/0 (socket says 172.21.15.26:47662) 2023-12-19T18:17:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.613+0000 7fdfb1386640 1 -- 172.21.15.26:0/1513562628 learned_addr learned my addr 172.21.15.26:0/1513562628 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.613+0000 7fdfb0b85640 1 --2- 172.21.15.26:0/1513562628 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfac154220 0x7fdfac15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:31.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.613+0000 7fdfab7fe640 1 -- 172.21.15.26:0/1513562628 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734991338 0 0) 0x7fdfac06bc30 con 0x7fdfac0cec40 2023-12-19T18:17:31.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfab7fe640 1 -- 172.21.15.26:0/1513562628 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf9c003530 con 0x7fdfac0cec40 2023-12-19T18:17:31.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfabfff640 1 -- 172.21.15.26:0/1513562628 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfac154220 msgr2=0x7fdfac15aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:31.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfabfff640 1 --2- 172.21.15.26:0/1513562628 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfac154220 0x7fdfac15aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfabfff640 1 -- 172.21.15.26:0/1513562628 >> v1:172.21.15.26:6789/0 conn(0x7fdfac0cec40 legacy=0x7fdfac15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfabfff640 1 -- 172.21.15.26:0/1513562628 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac15afc0 con 0x7fdfac06b3d0 2023-12-19T18:17:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfabfff640 1 --2- 172.21.15.26:0/1513562628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfac06b3d0 0x7fdfac0ce700 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fdf980097c0 tx=0x7fdf9801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa5d795b14280c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfab7fe640 1 -- 172.21.15.26:0/1513562628 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf980ab600 con 0x7fdfac06b3d0 2023-12-19T18:17:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfab7fe640 1 -- 172.21.15.26:0/1513562628 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf980a7050 con 0x7fdfac06b3d0 2023-12-19T18:17:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfab7fe640 1 -- 172.21.15.26:0/1513562628 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf980ab960 con 0x7fdfac06b3d0 2023-12-19T18:17:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1513562628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfac06b3d0 msgr2=0x7fdfac0ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfb2de9640 1 --2- 172.21.15.26:0/1513562628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfac06b3d0 0x7fdfac0ce700 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fdf980097c0 tx=0x7fdf9801c450 comp rx=0 tx=0).stop 2023-12-19T18:17:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1513562628 shutdown_connections 2023-12-19T18:17:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfb2de9640 1 --2- 172.21.15.26:0/1513562628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfac06b3d0 0x7fdfac0ce700 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfb2de9640 1 --2- 172.21.15.26:0/1513562628 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfac154220 0x7fdfac15aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.617+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1513562628 >> 172.21.15.26:0/1513562628 conn(0x7fdfac0658e0 msgr2=0x7fdfac065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:31.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1513562628 shutdown_connections 2023-12-19T18:17:31.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1513562628 wait complete. 2023-12-19T18:17:31.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb2de9640 1 Processor -- start 2023-12-19T18:17:31.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb2de9640 1 -- start start 2023-12-19T18:17:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb2de9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfac06b3d0 0x7fdfac1f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb2de9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfac0cec40 0x7fdfac0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb2de9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfac154220 0x7fdfac1584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb2de9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac0b27c0 con 0x7fdfac154220 2023-12-19T18:17:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb2de9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac1f5570 con 0x7fdfac0cec40 2023-12-19T18:17:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb2de9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac1f5850 con 0x7fdfac06b3d0 2023-12-19T18:17:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb1386640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfac154220 0x7fdfac1584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb1386640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfac154220 0x7fdfac1584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47618/0 (socket says 172.21.15.26:47618) 2023-12-19T18:17:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb1386640 1 -- 172.21.15.26:0/1318182835 learned_addr learned my addr 172.21.15.26:0/1318182835 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb1386640 1 -- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfac06b3d0 msgr2=0x7fdfac1f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb0b85640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfac06b3d0 0x7fdfac1f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb1386640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfac06b3d0 0x7fdfac1f4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb1386640 1 -- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfac0cec40 msgr2=0x7fdfac0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfabfff640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfac0cec40 0x7fdfac0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb1386640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfac0cec40 0x7fdfac0b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb1386640 1 -- 172.21.15.26:0/1318182835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf98009420 con 0x7fdfac154220 2023-12-19T18:17:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb0b85640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfac06b3d0 0x7fdfac1f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfabfff640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfac0cec40 0x7fdfac0b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.621+0000 7fdfb1386640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfac154220 0x7fdfac1584d0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400caa0 tx=0x7fdfa400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.625+0000 7fdfa97fa640 1 -- 172.21.15.26:0/1318182835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400ac00 con 0x7fdfac154220 2023-12-19T18:17:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.625+0000 7fdfa97fa640 1 -- 172.21.15.26:0/1318182835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa400ad60 con 0x7fdfac154220 2023-12-19T18:17:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.625+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1318182835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfac1f5b30 con 0x7fdfac154220 2023-12-19T18:17:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.625+0000 7fdfa97fa640 1 -- 172.21.15.26:0/1318182835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4012620 con 0x7fdfac154220 2023-12-19T18:17:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.625+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1318182835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfac1f5e10 con 0x7fdfac154220 2023-12-19T18:17:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.625+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1318182835 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfac1f8810 con 0x7fdfac154220 2023-12-19T18:17:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.625+0000 7fdfa97fa640 1 -- 172.21.15.26:0/1318182835 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fdfa4007070 con 0x7fdfac154220 2023-12-19T18:17:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.625+0000 7fdfa97fa640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdf90065ca0 0x7fdf90068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.625+0000 7fdfa97fa640 1 -- 172.21.15.26:0/1318182835 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fdfa40838a0 con 0x7fdfac154220 2023-12-19T18:17:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.625+0000 7fdfb0b85640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdf90065ca0 0x7fdf90068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.629+0000 7fdfb0b85640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdf90065ca0 0x7fdf90068160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009ae0 tx=0x7fdfa0008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:31.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.629+0000 7fdfa97fa640 1 -- 172.21.15.26:0/1318182835 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdfac1f8810 con 0x7fdfac154220 2023-12-19T18:17:31.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.785+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1318182835 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdfac0ccef0 con 0x7fdf90065ca0 2023-12-19T18:17:31.796 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.793+0000 7fdfa97fa640 1 -- 172.21.15.26:0/1318182835 <== mgr.10429 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289520 (secure 0 0 0) 0x7fdfa0010930 con 0x7fdf90065ca0 2023-12-19T18:17:31.799 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:31.800 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:17:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.797+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdf90065ca0 msgr2=0x7fdf90068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.797+0000 7fdfb2de9640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdf90065ca0 0x7fdf90068160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009ae0 tx=0x7fdfa0008040 comp rx=0 tx=0).stop 2023-12-19T18:17:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.797+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1318182835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfac154220 msgr2=0x7fdfac1584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.797+0000 7fdfb2de9640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfac154220 0x7fdfac1584d0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fdfa400caa0 tx=0x7fdfa400cf60 comp rx=0 tx=0).stop 2023-12-19T18:17:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.797+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1318182835 shutdown_connections 2023-12-19T18:17:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.797+0000 7fdfb2de9640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdf90065ca0 0x7fdf90068160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.797+0000 7fdfb2de9640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfac154220 0x7fdfac1584d0 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.797+0000 7fdfb2de9640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdfac0cec40 0x7fdfac0b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.797+0000 7fdfb2de9640 1 --2- 172.21.15.26:0/1318182835 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdfac06b3d0 0x7fdfac1f4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.797+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1318182835 >> 172.21.15.26:0/1318182835 conn(0x7fdfac0658e0 msgr2=0x7fdfac067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.797+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1318182835 shutdown_connections 2023-12-19T18:17:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:31.797+0000 7fdfb2de9640 1 -- 172.21.15.26:0/1318182835 wait complete. 2023-12-19T18:17:31.820 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-12-19T18:17:30.935404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2306,"num_read_kb":4823,"num_write":659,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":406,"ondisk_log_size":406,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":408616,"kb_used_data":24040,"kb_used_omap":0,"kb_used_meta":384512,"kb_avail":754566104,"statfs":{"total":773094113280,"available":772675690496,"internally_reserved":0,"allocated":24616960,"data_stored":13638172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":393740288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.013143"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486632+0000","last_change":"2023-12-19T18:17:27.353410+0000","last_active":"2023-12-19T18:17:28.486632+0000","last_peered":"2023-12-19T18:17:28.486632+0000","last_clean":"2023-12-19T18:17:28.486632+0000","last_became_active":"2023-12-19T18:17:27.352446+0000","last_became_peered":"2023-12-19T18:17:27.352446+0000","last_unstale":"2023-12-19T18:17:28.486632+0000","last_undegraded":"2023-12-19T18:17:28.486632+0000","last_fullsized":"2023-12-19T18:17:28.486632+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:55.042619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.340490+0000","last_change":"2023-12-19T18:17:27.358427+0000","last_active":"2023-12-19T18:17:28.340490+0000","last_peered":"2023-12-19T18:17:28.340490+0000","last_clean":"2023-12-19T18:17:28.340490+0000","last_became_active":"2023-12-19T18:17:27.358141+0000","last_became_peered":"2023-12-19T18:17:27.358141+0000","last_unstale":"2023-12-19T18:17:28.340490+0000","last_undegraded":"2023-12-19T18:17:28.340490+0000","last_fullsized":"2023-12-19T18:17:28.340490+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:14.654686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486619+0000","last_change":"2023-12-19T18:17:27.356031+0000","last_active":"2023-12-19T18:17:28.486619+0000","last_peered":"2023-12-19T18:17:28.486619+0000","last_clean":"2023-12-19T18:17:28.486619+0000","last_became_active":"2023-12-19T18:17:27.354195+0000","last_became_peered":"2023-12-19T18:17:27.354195+0000","last_unstale":"2023-12-19T18:17:28.486619+0000","last_undegraded":"2023-12-19T18:17:28.486619+0000","last_fullsized":"2023-12-19T18:17:28.486619+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:16.644439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485397+0000","last_change":"2023-12-19T18:17:27.348408+0000","last_active":"2023-12-19T18:17:28.485397+0000","last_peered":"2023-12-19T18:17:28.485397+0000","last_clean":"2023-12-19T18:17:28.485397+0000","last_became_active":"2023-12-19T18:17:27.348138+0000","last_became_peered":"2023-12-19T18:17:27.348138+0000","last_unstale":"2023-12-19T18:17:28.485397+0000","last_undegraded":"2023-12-19T18:17:28.485397+0000","last_fullsized":"2023-12-19T18:17:28.485397+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:19.968471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485212+0000","last_change":"2023-12-19T18:17:27.351573+0000","last_active":"2023-12-19T18:17:28.485212+0000","last_peered":"2023-12-19T18:17:28.485212+0000","last_clean":"2023-12-19T18:17:28.485212+0000","last_became_active":"2023-12-19T18:17:27.351222+0000","last_became_peered":"2023-12-19T18:17:27.351222+0000","last_unstale":"2023-12-19T18:17:28.485212+0000","last_undegraded":"2023-12-19T18:17:28.485212+0000","last_fullsized":"2023-12-19T18:17:28.485212+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:17:31.537417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485126+0000","last_change":"2023-12-19T18:17:27.364283+0000","last_active":"2023-12-19T18:17:28.485126+0000","last_peered":"2023-12-19T18:17:28.485126+0000","last_clean":"2023-12-19T18:17:28.485126+0000","last_became_active":"2023-12-19T18:17:27.363962+0000","last_became_peered":"2023-12-19T18:17:27.363962+0000","last_unstale":"2023-12-19T18:17:28.485126+0000","last_undegraded":"2023-12-19T18:17:28.485126+0000","last_fullsized":"2023-12-19T18:17:28.485126+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:25.159077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485978+0000","last_change":"2023-12-19T18:17:27.357178+0000","last_active":"2023-12-19T18:17:28.485978+0000","last_peered":"2023-12-19T18:17:28.485978+0000","last_clean":"2023-12-19T18:17:28.485978+0000","last_became_active":"2023-12-19T18:17:27.356893+0000","last_became_peered":"2023-12-19T18:17:27.356893+0000","last_unstale":"2023-12-19T18:17:28.485978+0000","last_undegraded":"2023-12-19T18:17:28.485978+0000","last_fullsized":"2023-12-19T18:17:28.485978+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:31.153454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.337759+0000","last_change":"2023-12-19T18:17:27.344049+0000","last_active":"2023-12-19T18:17:28.337759+0000","last_peered":"2023-12-19T18:17:28.337759+0000","last_clean":"2023-12-19T18:17:28.337759+0000","last_became_active":"2023-12-19T18:17:27.343738+0000","last_became_peered":"2023-12-19T18:17:27.343738+0000","last_unstale":"2023-12-19T18:17:28.337759+0000","last_undegraded":"2023-12-19T18:17:28.337759+0000","last_fullsized":"2023-12-19T18:17:28.337759+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:31.445521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486859+0000","last_change":"2023-12-19T18:17:27.363691+0000","last_active":"2023-12-19T18:17:28.486859+0000","last_peered":"2023-12-19T18:17:28.486859+0000","last_clean":"2023-12-19T18:17:28.486859+0000","last_became_active":"2023-12-19T18:17:27.363412+0000","last_became_peered":"2023-12-19T18:17:27.363412+0000","last_unstale":"2023-12-19T18:17:28.486859+0000","last_undegraded":"2023-12-19T18:17:28.486859+0000","last_fullsized":"2023-12-19T18:17:28.486859+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:47.826883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485068+0000","last_change":"2023-12-19T18:17:27.343713+0000","last_active":"2023-12-19T18:17:28.485068+0000","last_peered":"2023-12-19T18:17:28.485068+0000","last_clean":"2023-12-19T18:17:28.485068+0000","last_became_active":"2023-12-19T18:17:27.343409+0000","last_became_peered":"2023-12-19T18:17:27.343409+0000","last_unstale":"2023-12-19T18:17:28.485068+0000","last_undegraded":"2023-12-19T18:17:28.485068+0000","last_fullsized":"2023-12-19T18:17:28.485068+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:40.489404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486121+0000","last_change":"2023-12-19T18:17:27.353890+0000","last_active":"2023-12-19T18:17:28.486121+0000","last_peered":"2023-12-19T18:17:28.486121+0000","last_clean":"2023-12-19T18:17:28.486121+0000","last_became_active":"2023-12-19T18:17:27.352505+0000","last_became_peered":"2023-12-19T18:17:27.352505+0000","last_unstale":"2023-12-19T18:17:28.486121+0000","last_undegraded":"2023-12-19T18:17:28.486121+0000","last_fullsized":"2023-12-19T18:17:28.486121+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:49:56.496681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.340424+0000","last_change":"2023-12-19T18:17:27.353636+0000","last_active":"2023-12-19T18:17:28.340424+0000","last_peered":"2023-12-19T18:17:28.340424+0000","last_clean":"2023-12-19T18:17:28.340424+0000","last_became_active":"2023-12-19T18:17:27.353234+0000","last_became_peered":"2023-12-19T18:17:27.353234+0000","last_unstale":"2023-12-19T18:17:28.340424+0000","last_undegraded":"2023-12-19T18:17:28.340424+0000","last_fullsized":"2023-12-19T18:17:28.340424+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:50.421353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.340116+0000","last_change":"2023-12-19T18:17:27.368093+0000","last_active":"2023-12-19T18:17:28.340116+0000","last_peered":"2023-12-19T18:17:28.340116+0000","last_clean":"2023-12-19T18:17:28.340116+0000","last_became_active":"2023-12-19T18:17:27.367300+0000","last_became_peered":"2023-12-19T18:17:27.367300+0000","last_unstale":"2023-12-19T18:17:28.340116+0000","last_undegraded":"2023-12-19T18:17:28.340116+0000","last_fullsized":"2023-12-19T18:17:28.340116+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:16.644439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.340089+0000","last_change":"2023-12-19T18:17:27.362240+0000","last_active":"2023-12-19T18:17:28.340089+0000","last_peered":"2023-12-19T18:17:28.340089+0000","last_clean":"2023-12-19T18:17:28.340089+0000","last_became_active":"2023-12-19T18:17:27.361917+0000","last_became_peered":"2023-12-19T18:17:27.361917+0000","last_unstale":"2023-12-19T18:17:28.340089+0000","last_undegraded":"2023-12-19T18:17:28.340089+0000","last_fullsized":"2023-12-19T18:17:28.340089+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:44:22.931173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.484629+0000","last_change":"2023-12-19T18:17:27.356217+0000","last_active":"2023-12-19T18:17:28.484629+0000","last_peered":"2023-12-19T18:17:28.484629+0000","last_clean":"2023-12-19T18:17:28.484629+0000","last_became_active":"2023-12-19T18:17:27.355721+0000","last_became_peered":"2023-12-19T18:17:27.355721+0000","last_unstale":"2023-12-19T18:17:28.484629+0000","last_undegraded":"2023-12-19T18:17:28.484629+0000","last_fullsized":"2023-12-19T18:17:28.484629+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:20:57.806100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339493+0000","last_change":"2023-12-19T18:17:27.350328+0000","last_active":"2023-12-19T18:17:28.339493+0000","last_peered":"2023-12-19T18:17:28.339493+0000","last_clean":"2023-12-19T18:17:28.339493+0000","last_became_active":"2023-12-19T18:17:27.350006+0000","last_became_peered":"2023-12-19T18:17:27.350006+0000","last_unstale":"2023-12-19T18:17:28.339493+0000","last_undegraded":"2023-12-19T18:17:28.339493+0000","last_fullsized":"2023-12-19T18:17:28.339493+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:00.178479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339872+0000","last_change":"2023-12-19T18:17:27.367615+0000","last_active":"2023-12-19T18:17:28.339872+0000","last_peered":"2023-12-19T18:17:28.339872+0000","last_clean":"2023-12-19T18:17:28.339872+0000","last_became_active":"2023-12-19T18:17:27.366800+0000","last_became_peered":"2023-12-19T18:17:27.366800+0000","last_unstale":"2023-12-19T18:17:28.339872+0000","last_undegraded":"2023-12-19T18:17:28.339872+0000","last_fullsized":"2023-12-19T18:17:28.339872+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:19:50.020207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485266+0000","last_change":"2023-12-19T18:17:27.357129+0000","last_active":"2023-12-19T18:17:28.485266+0000","last_peered":"2023-12-19T18:17:28.485266+0000","last_clean":"2023-12-19T18:17:28.485266+0000","last_became_active":"2023-12-19T18:17:27.356614+0000","last_became_peered":"2023-12-19T18:17:27.356614+0000","last_unstale":"2023-12-19T18:17:28.485266+0000","last_undegraded":"2023-12-19T18:17:28.485266+0000","last_fullsized":"2023-12-19T18:17:28.485266+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:22.773760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485580+0000","last_change":"2023-12-19T18:17:27.353111+0000","last_active":"2023-12-19T18:17:28.485580+0000","last_peered":"2023-12-19T18:17:28.485580+0000","last_clean":"2023-12-19T18:17:28.485580+0000","last_became_active":"2023-12-19T18:17:27.352840+0000","last_became_peered":"2023-12-19T18:17:27.352840+0000","last_unstale":"2023-12-19T18:17:28.485580+0000","last_undegraded":"2023-12-19T18:17:28.485580+0000","last_fullsized":"2023-12-19T18:17:28.485580+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:28.649300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485120+0000","last_change":"2023-12-19T18:17:27.340972+0000","last_active":"2023-12-19T18:17:28.485120+0000","last_peered":"2023-12-19T18:17:28.485120+0000","last_clean":"2023-12-19T18:17:28.485120+0000","last_became_active":"2023-12-19T18:17:27.340523+0000","last_became_peered":"2023-12-19T18:17:27.340523+0000","last_unstale":"2023-12-19T18:17:28.485120+0000","last_undegraded":"2023-12-19T18:17:28.485120+0000","last_fullsized":"2023-12-19T18:17:28.485120+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:50.621127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485313+0000","last_change":"2023-12-19T18:17:27.355317+0000","last_active":"2023-12-19T18:17:28.485313+0000","last_peered":"2023-12-19T18:17:28.485313+0000","last_clean":"2023-12-19T18:17:28.485313+0000","last_became_active":"2023-12-19T18:17:27.355002+0000","last_became_peered":"2023-12-19T18:17:27.355002+0000","last_unstale":"2023-12-19T18:17:28.485313+0000","last_undegraded":"2023-12-19T18:17:28.485313+0000","last_fullsized":"2023-12-19T18:17:28.485313+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:50.621127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485589+0000","last_change":"2023-12-19T18:17:27.347478+0000","last_active":"2023-12-19T18:17:28.485589+0000","last_peered":"2023-12-19T18:17:28.485589+0000","last_clean":"2023-12-19T18:17:28.485589+0000","last_became_active":"2023-12-19T18:17:27.346998+0000","last_became_peered":"2023-12-19T18:17:27.346998+0000","last_unstale":"2023-12-19T18:17:28.485589+0000","last_undegraded":"2023-12-19T18:17:28.485589+0000","last_fullsized":"2023-12-19T18:17:28.485589+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:22.470263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485549+0000","last_change":"2023-12-19T18:17:27.339790+0000","last_active":"2023-12-19T18:17:28.485549+0000","last_peered":"2023-12-19T18:17:28.485549+0000","last_clean":"2023-12-19T18:17:28.485549+0000","last_became_active":"2023-12-19T18:17:27.339477+0000","last_became_peered":"2023-12-19T18:17:27.339477+0000","last_unstale":"2023-12-19T18:17:28.485549+0000","last_undegraded":"2023-12-19T18:17:28.485549+0000","last_fullsized":"2023-12-19T18:17:28.485549+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:23:29.725587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.484833+0000","last_change":"2023-12-19T18:17:27.348543+0000","last_active":"2023-12-19T18:17:28.484833+0000","last_peered":"2023-12-19T18:17:28.484833+0000","last_clean":"2023-12-19T18:17:28.484833+0000","last_became_active":"2023-12-19T18:17:27.348257+0000","last_became_peered":"2023-12-19T18:17:27.348257+0000","last_unstale":"2023-12-19T18:17:28.484833+0000","last_undegraded":"2023-12-19T18:17:28.484833+0000","last_fullsized":"2023-12-19T18:17:28.484833+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:44:09.238317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486788+0000","last_change":"2023-12-19T18:17:27.353080+0000","last_active":"2023-12-19T18:17:28.486788+0000","last_peered":"2023-12-19T18:17:28.486788+0000","last_clean":"2023-12-19T18:17:28.486788+0000","last_became_active":"2023-12-19T18:17:27.352664+0000","last_became_peered":"2023-12-19T18:17:27.352664+0000","last_unstale":"2023-12-19T18:17:28.486788+0000","last_undegraded":"2023-12-19T18:17:28.486788+0000","last_fullsized":"2023-12-19T18:17:28.486788+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:22:10.869020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.338292+0000","last_change":"2023-12-19T18:17:27.367346+0000","last_active":"2023-12-19T18:17:28.338292+0000","last_peered":"2023-12-19T18:17:28.338292+0000","last_clean":"2023-12-19T18:17:28.338292+0000","last_became_active":"2023-12-19T18:17:27.367053+0000","last_became_peered":"2023-12-19T18:17:27.367053+0000","last_unstale":"2023-12-19T18:17:28.338292+0000","last_undegraded":"2023-12-19T18:17:28.338292+0000","last_fullsized":"2023-12-19T18:17:28.338292+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:44.734608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486154+0000","last_change":"2023-12-19T18:17:27.355178+0000","last_active":"2023-12-19T18:17:28.486154+0000","last_peered":"2023-12-19T18:17:28.486154+0000","last_clean":"2023-12-19T18:17:28.486154+0000","last_became_active":"2023-12-19T18:17:27.354771+0000","last_became_peered":"2023-12-19T18:17:27.354771+0000","last_unstale":"2023-12-19T18:17:28.486154+0000","last_undegraded":"2023-12-19T18:17:28.486154+0000","last_fullsized":"2023-12-19T18:17:28.486154+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:06.555153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485498+0000","last_change":"2023-12-19T18:17:27.349197+0000","last_active":"2023-12-19T18:17:28.485498+0000","last_peered":"2023-12-19T18:17:28.485498+0000","last_clean":"2023-12-19T18:17:28.485498+0000","last_became_active":"2023-12-19T18:17:27.348940+0000","last_became_peered":"2023-12-19T18:17:27.348940+0000","last_unstale":"2023-12-19T18:17:28.485498+0000","last_undegraded":"2023-12-19T18:17:28.485498+0000","last_fullsized":"2023-12-19T18:17:28.485498+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:19.968471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339651+0000","last_change":"2023-12-19T18:17:27.369700+0000","last_active":"2023-12-19T18:17:28.339651+0000","last_peered":"2023-12-19T18:17:28.339651+0000","last_clean":"2023-12-19T18:17:28.339651+0000","last_became_active":"2023-12-19T18:17:27.369398+0000","last_became_peered":"2023-12-19T18:17:27.369398+0000","last_unstale":"2023-12-19T18:17:28.339651+0000","last_undegraded":"2023-12-19T18:17:28.339651+0000","last_fullsized":"2023-12-19T18:17:28.339651+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:35:43.593726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485714+0000","last_change":"2023-12-19T18:17:27.360694+0000","last_active":"2023-12-19T18:17:28.485714+0000","last_peered":"2023-12-19T18:17:28.485714+0000","last_clean":"2023-12-19T18:17:28.485714+0000","last_became_active":"2023-12-19T18:17:27.357706+0000","last_became_peered":"2023-12-19T18:17:27.357706+0000","last_unstale":"2023-12-19T18:17:28.485714+0000","last_undegraded":"2023-12-19T18:17:28.485714+0000","last_fullsized":"2023-12-19T18:17:28.485714+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:22:01.458247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.484920+0000","last_change":"2023-12-19T18:17:27.361414+0000","last_active":"2023-12-19T18:17:28.484920+0000","last_peered":"2023-12-19T18:17:28.484920+0000","last_clean":"2023-12-19T18:17:28.484920+0000","last_became_active":"2023-12-19T18:17:27.361054+0000","last_became_peered":"2023-12-19T18:17:27.361054+0000","last_unstale":"2023-12-19T18:17:28.484920+0000","last_undegraded":"2023-12-19T18:17:28.484920+0000","last_fullsized":"2023-12-19T18:17:28.484920+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:04:33.062599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.484686+0000","last_change":"2023-12-19T18:17:27.359512+0000","last_active":"2023-12-19T18:17:28.484686+0000","last_peered":"2023-12-19T18:17:28.484686+0000","last_clean":"2023-12-19T18:17:28.484686+0000","last_became_active":"2023-12-19T18:17:27.359173+0000","last_became_peered":"2023-12-19T18:17:27.359173+0000","last_unstale":"2023-12-19T18:17:28.484686+0000","last_undegraded":"2023-12-19T18:17:28.484686+0000","last_fullsized":"2023-12-19T18:17:28.484686+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:42.446500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486554+0000","last_change":"2023-12-19T18:17:27.355393+0000","last_active":"2023-12-19T18:17:28.486554+0000","last_peered":"2023-12-19T18:17:28.486554+0000","last_clean":"2023-12-19T18:17:28.486554+0000","last_became_active":"2023-12-19T18:17:27.355013+0000","last_became_peered":"2023-12-19T18:17:27.355013+0000","last_unstale":"2023-12-19T18:17:28.486554+0000","last_undegraded":"2023-12-19T18:17:28.486554+0000","last_fullsized":"2023-12-19T18:17:28.486554+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:49.759094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"232'4","reported_seq":15,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.361758+0000","last_change":"2023-12-19T18:17:26.951474+0000","last_active":"2023-12-19T18:17:27.361758+0000","last_peered":"2023-12-19T18:17:27.361758+0000","last_clean":"2023-12-19T18:17:27.361758+0000","last_became_active":"2023-12-19T18:17:26.950990+0000","last_became_peered":"2023-12-19T18:17:26.950990+0000","last_unstale":"2023-12-19T18:17:27.361758+0000","last_undegraded":"2023-12-19T18:17:27.361758+0000","last_fullsized":"2023-12-19T18:17:27.361758+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:37:49.322125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.338181+0000","last_change":"2023-12-19T18:17:27.346026+0000","last_active":"2023-12-19T18:17:28.338181+0000","last_peered":"2023-12-19T18:17:28.338181+0000","last_clean":"2023-12-19T18:17:28.338181+0000","last_became_active":"2023-12-19T18:17:27.345752+0000","last_became_peered":"2023-12-19T18:17:27.345752+0000","last_unstale":"2023-12-19T18:17:28.338181+0000","last_undegraded":"2023-12-19T18:17:28.338181+0000","last_fullsized":"2023-12-19T18:17:28.338181+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:18.690231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.361889+0000","last_change":"2023-12-19T18:17:26.946867+0000","last_active":"2023-12-19T18:17:27.361889+0000","last_peered":"2023-12-19T18:17:27.361889+0000","last_clean":"2023-12-19T18:17:27.361889+0000","last_became_active":"2023-12-19T18:17:26.946549+0000","last_became_peered":"2023-12-19T18:17:26.946549+0000","last_unstale":"2023-12-19T18:17:27.361889+0000","last_undegraded":"2023-12-19T18:17:27.361889+0000","last_fullsized":"2023-12-19T18:17:27.361889+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:36.912904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339926+0000","last_change":"2023-12-19T18:17:27.369534+0000","last_active":"2023-12-19T18:17:28.339926+0000","last_peered":"2023-12-19T18:17:28.339926+0000","last_clean":"2023-12-19T18:17:28.339926+0000","last_became_active":"2023-12-19T18:17:27.369101+0000","last_became_peered":"2023-12-19T18:17:27.369101+0000","last_unstale":"2023-12-19T18:17:28.339926+0000","last_undegraded":"2023-12-19T18:17:28.339926+0000","last_fullsized":"2023-12-19T18:17:28.339926+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:38:39.157091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.358793+0000","last_change":"2023-12-19T18:17:26.947910+0000","last_active":"2023-12-19T18:17:27.358793+0000","last_peered":"2023-12-19T18:17:27.358793+0000","last_clean":"2023-12-19T18:17:27.358793+0000","last_became_active":"2023-12-19T18:17:26.946840+0000","last_became_peered":"2023-12-19T18:17:26.946840+0000","last_unstale":"2023-12-19T18:17:27.358793+0000","last_undegraded":"2023-12-19T18:17:27.358793+0000","last_fullsized":"2023-12-19T18:17:27.358793+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:41.372298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485942+0000","last_change":"2023-12-19T18:17:27.361418+0000","last_active":"2023-12-19T18:17:28.485942+0000","last_peered":"2023-12-19T18:17:28.485942+0000","last_clean":"2023-12-19T18:17:28.485942+0000","last_became_active":"2023-12-19T18:17:27.361131+0000","last_became_peered":"2023-12-19T18:17:27.361131+0000","last_unstale":"2023-12-19T18:17:28.485942+0000","last_undegraded":"2023-12-19T18:17:28.485942+0000","last_fullsized":"2023-12-19T18:17:28.485942+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:44:06.399365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"232'1","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.337991+0000","last_change":"2023-12-19T18:17:26.942982+0000","last_active":"2023-12-19T18:17:28.337991+0000","last_peered":"2023-12-19T18:17:28.337991+0000","last_clean":"2023-12-19T18:17:28.337991+0000","last_became_active":"2023-12-19T18:17:26.942719+0000","last_became_peered":"2023-12-19T18:17:26.942719+0000","last_unstale":"2023-12-19T18:17:28.337991+0000","last_undegraded":"2023-12-19T18:17:28.337991+0000","last_fullsized":"2023-12-19T18:17:28.337991+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:10.474482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485408+0000","last_change":"2023-12-19T18:17:27.361092+0000","last_active":"2023-12-19T18:17:28.485408+0000","last_peered":"2023-12-19T18:17:28.485408+0000","last_clean":"2023-12-19T18:17:28.485408+0000","last_became_active":"2023-12-19T18:17:27.360778+0000","last_became_peered":"2023-12-19T18:17:27.360778+0000","last_unstale":"2023-12-19T18:17:28.485408+0000","last_undegraded":"2023-12-19T18:17:28.485408+0000","last_fullsized":"2023-12-19T18:17:28.485408+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:40.489404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"232'1","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486223+0000","last_change":"2023-12-19T18:17:26.940960+0000","last_active":"2023-12-19T18:17:28.486223+0000","last_peered":"2023-12-19T18:17:28.486223+0000","last_clean":"2023-12-19T18:17:28.486223+0000","last_became_active":"2023-12-19T18:17:26.939915+0000","last_became_peered":"2023-12-19T18:17:26.939915+0000","last_unstale":"2023-12-19T18:17:28.486223+0000","last_undegraded":"2023-12-19T18:17:28.486223+0000","last_fullsized":"2023-12-19T18:17:28.486223+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:52:17.872321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.484852+0000","last_change":"2023-12-19T18:17:27.347577+0000","last_active":"2023-12-19T18:17:28.484852+0000","last_peered":"2023-12-19T18:17:28.484852+0000","last_clean":"2023-12-19T18:17:28.484852+0000","last_became_active":"2023-12-19T18:17:27.346549+0000","last_became_peered":"2023-12-19T18:17:27.346549+0000","last_unstale":"2023-12-19T18:17:28.484852+0000","last_undegraded":"2023-12-19T18:17:28.484852+0000","last_fullsized":"2023-12-19T18:17:28.484852+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:28.649300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.355042+0000","last_change":"2023-12-19T18:17:26.950047+0000","last_active":"2023-12-19T18:17:27.355042+0000","last_peered":"2023-12-19T18:17:27.355042+0000","last_clean":"2023-12-19T18:17:27.355042+0000","last_became_active":"2023-12-19T18:17:26.949678+0000","last_became_peered":"2023-12-19T18:17:26.949678+0000","last_unstale":"2023-12-19T18:17:27.355042+0000","last_undegraded":"2023-12-19T18:17:27.355042+0000","last_fullsized":"2023-12-19T18:17:27.355042+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:49:16.046354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339905+0000","last_change":"2023-12-19T18:17:27.363788+0000","last_active":"2023-12-19T18:17:28.339905+0000","last_peered":"2023-12-19T18:17:28.339905+0000","last_clean":"2023-12-19T18:17:28.339905+0000","last_became_active":"2023-12-19T18:17:27.363513+0000","last_became_peered":"2023-12-19T18:17:27.363513+0000","last_unstale":"2023-12-19T18:17:28.339905+0000","last_undegraded":"2023-12-19T18:17:28.339905+0000","last_fullsized":"2023-12-19T18:17:28.339905+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:23.278389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.330747+0000","last_change":"2023-12-19T18:17:26.944745+0000","last_active":"2023-12-19T18:17:27.330747+0000","last_peered":"2023-12-19T18:17:27.330747+0000","last_clean":"2023-12-19T18:17:27.330747+0000","last_became_active":"2023-12-19T18:17:26.944481+0000","last_became_peered":"2023-12-19T18:17:26.944481+0000","last_unstale":"2023-12-19T18:17:27.330747+0000","last_undegraded":"2023-12-19T18:17:27.330747+0000","last_fullsized":"2023-12-19T18:17:27.330747+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:30.604242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.340556+0000","last_change":"2023-12-19T18:17:27.356985+0000","last_active":"2023-12-19T18:17:28.340556+0000","last_peered":"2023-12-19T18:17:28.340556+0000","last_clean":"2023-12-19T18:17:28.340556+0000","last_became_active":"2023-12-19T18:17:27.356695+0000","last_became_peered":"2023-12-19T18:17:27.356695+0000","last_unstale":"2023-12-19T18:17:28.340556+0000","last_undegraded":"2023-12-19T18:17:28.340556+0000","last_fullsized":"2023-12-19T18:17:28.340556+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:31.153454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.335875+0000","last_change":"2023-12-19T18:17:26.944872+0000","last_active":"2023-12-19T18:17:27.335875+0000","last_peered":"2023-12-19T18:17:27.335875+0000","last_clean":"2023-12-19T18:17:27.335875+0000","last_became_active":"2023-12-19T18:17:26.944558+0000","last_became_peered":"2023-12-19T18:17:26.944558+0000","last_unstale":"2023-12-19T18:17:27.335875+0000","last_undegraded":"2023-12-19T18:17:27.335875+0000","last_fullsized":"2023-12-19T18:17:27.335875+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:30.151215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486857+0000","last_change":"2023-12-19T18:17:27.342915+0000","last_active":"2023-12-19T18:17:28.486857+0000","last_peered":"2023-12-19T18:17:28.486857+0000","last_clean":"2023-12-19T18:17:28.486857+0000","last_became_active":"2023-12-19T18:17:27.342549+0000","last_became_peered":"2023-12-19T18:17:27.342549+0000","last_unstale":"2023-12-19T18:17:28.486857+0000","last_undegraded":"2023-12-19T18:17:28.486857+0000","last_fullsized":"2023-12-19T18:17:28.486857+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:03.786746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.338131+0000","last_change":"2023-12-19T18:17:26.943010+0000","last_active":"2023-12-19T18:17:27.338131+0000","last_peered":"2023-12-19T18:17:27.338131+0000","last_clean":"2023-12-19T18:17:27.338131+0000","last_became_active":"2023-12-19T18:17:26.942721+0000","last_became_peered":"2023-12-19T18:17:26.942721+0000","last_unstale":"2023-12-19T18:17:27.338131+0000","last_undegraded":"2023-12-19T18:17:27.338131+0000","last_fullsized":"2023-12-19T18:17:27.338131+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:09:16.907014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339728+0000","last_change":"2023-12-19T18:17:27.368101+0000","last_active":"2023-12-19T18:17:28.339728+0000","last_peered":"2023-12-19T18:17:28.339728+0000","last_clean":"2023-12-19T18:17:28.339728+0000","last_became_active":"2023-12-19T18:17:27.367822+0000","last_became_peered":"2023-12-19T18:17:27.367822+0000","last_unstale":"2023-12-19T18:17:28.339728+0000","last_undegraded":"2023-12-19T18:17:28.339728+0000","last_fullsized":"2023-12-19T18:17:28.339728+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:11:20.286347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"232'3","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485669+0000","last_change":"2023-12-19T18:17:26.938268+0000","last_active":"2023-12-19T18:17:28.485669+0000","last_peered":"2023-12-19T18:17:28.485669+0000","last_clean":"2023-12-19T18:17:28.485669+0000","last_became_active":"2023-12-19T18:17:26.937724+0000","last_became_peered":"2023-12-19T18:17:26.937724+0000","last_unstale":"2023-12-19T18:17:28.485669+0000","last_undegraded":"2023-12-19T18:17:28.485669+0000","last_fullsized":"2023-12-19T18:17:28.485669+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:05.552913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486053+0000","last_change":"2023-12-19T18:17:27.344496+0000","last_active":"2023-12-19T18:17:28.486053+0000","last_peered":"2023-12-19T18:17:28.486053+0000","last_clean":"2023-12-19T18:17:28.486053+0000","last_became_active":"2023-12-19T18:17:27.343611+0000","last_became_peered":"2023-12-19T18:17:27.343611+0000","last_unstale":"2023-12-19T18:17:28.486053+0000","last_undegraded":"2023-12-19T18:17:28.486053+0000","last_fullsized":"2023-12-19T18:17:28.486053+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:26:28.887778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"232'6","reported_seq":17,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.365295+0000","last_change":"2023-12-19T18:17:26.941650+0000","last_active":"2023-12-19T18:17:27.365295+0000","last_peered":"2023-12-19T18:17:27.365295+0000","last_clean":"2023-12-19T18:17:27.365295+0000","last_became_active":"2023-12-19T18:17:26.941388+0000","last_became_peered":"2023-12-19T18:17:26.941388+0000","last_unstale":"2023-12-19T18:17:27.365295+0000","last_undegraded":"2023-12-19T18:17:27.365295+0000","last_fullsized":"2023-12-19T18:17:27.365295+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:35:14.834857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485891+0000","last_change":"2023-12-19T18:17:27.350307+0000","last_active":"2023-12-19T18:17:28.485891+0000","last_peered":"2023-12-19T18:17:28.485891+0000","last_clean":"2023-12-19T18:17:28.485891+0000","last_became_active":"2023-12-19T18:17:27.350036+0000","last_became_peered":"2023-12-19T18:17:27.350036+0000","last_unstale":"2023-12-19T18:17:28.485891+0000","last_undegraded":"2023-12-19T18:17:28.485891+0000","last_fullsized":"2023-12-19T18:17:28.485891+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:24:50.816850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"232'2","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.338048+0000","last_change":"2023-12-19T18:17:26.941112+0000","last_active":"2023-12-19T18:17:28.338048+0000","last_peered":"2023-12-19T18:17:28.338048+0000","last_clean":"2023-12-19T18:17:28.338048+0000","last_became_active":"2023-12-19T18:17:26.940838+0000","last_became_peered":"2023-12-19T18:17:26.940838+0000","last_unstale":"2023-12-19T18:17:28.338048+0000","last_undegraded":"2023-12-19T18:17:28.338048+0000","last_fullsized":"2023-12-19T18:17:28.338048+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:47:48.785673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486373+0000","last_change":"2023-12-19T18:17:27.362157+0000","last_active":"2023-12-19T18:17:28.486373+0000","last_peered":"2023-12-19T18:17:28.486373+0000","last_clean":"2023-12-19T18:17:28.486373+0000","last_became_active":"2023-12-19T18:17:27.361879+0000","last_became_peered":"2023-12-19T18:17:27.361879+0000","last_unstale":"2023-12-19T18:17:28.486373+0000","last_undegraded":"2023-12-19T18:17:28.486373+0000","last_fullsized":"2023-12-19T18:17:28.486373+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:00.178479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.338035+0000","last_change":"2023-12-19T18:17:26.948613+0000","last_active":"2023-12-19T18:17:27.338035+0000","last_peered":"2023-12-19T18:17:27.338035+0000","last_clean":"2023-12-19T18:17:27.338035+0000","last_became_active":"2023-12-19T18:17:26.948318+0000","last_became_peered":"2023-12-19T18:17:26.948318+0000","last_unstale":"2023-12-19T18:17:27.338035+0000","last_undegraded":"2023-12-19T18:17:27.338035+0000","last_fullsized":"2023-12-19T18:17:27.338035+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:13:19.247070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485463+0000","last_change":"2023-12-19T18:17:27.361736+0000","last_active":"2023-12-19T18:17:28.485463+0000","last_peered":"2023-12-19T18:17:28.485463+0000","last_clean":"2023-12-19T18:17:28.485463+0000","last_became_active":"2023-12-19T18:17:27.361355+0000","last_became_peered":"2023-12-19T18:17:27.361355+0000","last_unstale":"2023-12-19T18:17:28.485463+0000","last_undegraded":"2023-12-19T18:17:28.485463+0000","last_fullsized":"2023-12-19T18:17:28.485463+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:02.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":11,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485475+0000","last_change":"2023-12-19T18:17:26.944351+0000","last_active":"2023-12-19T18:17:28.485475+0000","last_peered":"2023-12-19T18:17:28.485475+0000","last_clean":"2023-12-19T18:17:28.485475+0000","last_became_active":"2023-12-19T18:17:26.943692+0000","last_became_peered":"2023-12-19T18:17:26.943692+0000","last_unstale":"2023-12-19T18:17:28.485475+0000","last_undegraded":"2023-12-19T18:17:28.485475+0000","last_fullsized":"2023-12-19T18:17:28.485475+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:54.040380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339242+0000","last_change":"2023-12-19T18:17:27.366044+0000","last_active":"2023-12-19T18:17:28.339242+0000","last_peered":"2023-12-19T18:17:28.339242+0000","last_clean":"2023-12-19T18:17:28.339242+0000","last_became_active":"2023-12-19T18:17:27.365761+0000","last_became_peered":"2023-12-19T18:17:27.365761+0000","last_unstale":"2023-12-19T18:17:28.339242+0000","last_undegraded":"2023-12-19T18:17:28.339242+0000","last_fullsized":"2023-12-19T18:17:28.339242+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:05:07.652398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"232'8","reported_seq":19,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.365469+0000","last_change":"2023-12-19T18:17:26.948258+0000","last_active":"2023-12-19T18:17:27.365469+0000","last_peered":"2023-12-19T18:17:27.365469+0000","last_clean":"2023-12-19T18:17:27.365469+0000","last_became_active":"2023-12-19T18:17:26.947974+0000","last_became_peered":"2023-12-19T18:17:26.947974+0000","last_unstale":"2023-12-19T18:17:27.365469+0000","last_undegraded":"2023-12-19T18:17:27.365469+0000","last_fullsized":"2023-12-19T18:17:27.365469+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:49.244280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.338382+0000","last_change":"2023-12-19T18:17:27.349954+0000","last_active":"2023-12-19T18:17:28.338382+0000","last_peered":"2023-12-19T18:17:28.338382+0000","last_clean":"2023-12-19T18:17:28.338382+0000","last_became_active":"2023-12-19T18:17:27.349694+0000","last_became_peered":"2023-12-19T18:17:27.349694+0000","last_unstale":"2023-12-19T18:17:28.338382+0000","last_undegraded":"2023-12-19T18:17:28.338382+0000","last_fullsized":"2023-12-19T18:17:28.338382+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:36:07.296035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"232'1","reported_seq":12,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486953+0000","last_change":"2023-12-19T18:17:26.944980+0000","last_active":"2023-12-19T18:17:28.486953+0000","last_peered":"2023-12-19T18:17:28.486953+0000","last_clean":"2023-12-19T18:17:28.486953+0000","last_became_active":"2023-12-19T18:17:26.943863+0000","last_became_peered":"2023-12-19T18:17:26.943863+0000","last_unstale":"2023-12-19T18:17:28.486953+0000","last_undegraded":"2023-12-19T18:17:28.486953+0000","last_fullsized":"2023-12-19T18:17:28.486953+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:38:38.154851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485305+0000","last_change":"2023-12-19T18:17:27.364634+0000","last_active":"2023-12-19T18:17:28.485305+0000","last_peered":"2023-12-19T18:17:28.485305+0000","last_clean":"2023-12-19T18:17:28.485305+0000","last_became_active":"2023-12-19T18:17:27.364357+0000","last_became_peered":"2023-12-19T18:17:27.364357+0000","last_unstale":"2023-12-19T18:17:28.485305+0000","last_undegraded":"2023-12-19T18:17:28.485305+0000","last_fullsized":"2023-12-19T18:17:28.485305+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:05:35.230888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.354463+0000","last_change":"2023-12-19T18:17:26.938178+0000","last_active":"2023-12-19T18:17:27.354463+0000","last_peered":"2023-12-19T18:17:27.354463+0000","last_clean":"2023-12-19T18:17:27.354463+0000","last_became_active":"2023-12-19T18:17:26.937910+0000","last_became_peered":"2023-12-19T18:17:26.937910+0000","last_unstale":"2023-12-19T18:17:27.354463+0000","last_undegraded":"2023-12-19T18:17:27.354463+0000","last_fullsized":"2023-12-19T18:17:27.354463+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:19:13.215706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486340+0000","last_change":"2023-12-19T18:17:27.360665+0000","last_active":"2023-12-19T18:17:28.486340+0000","last_peered":"2023-12-19T18:17:28.486340+0000","last_clean":"2023-12-19T18:17:28.486340+0000","last_became_active":"2023-12-19T18:17:27.360397+0000","last_became_peered":"2023-12-19T18:17:27.360397+0000","last_unstale":"2023-12-19T18:17:28.486340+0000","last_undegraded":"2023-12-19T18:17:28.486340+0000","last_fullsized":"2023-12-19T18:17:28.486340+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:33.393492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"232'1","reported_seq":12,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.361613+0000","last_change":"2023-12-19T18:17:26.946790+0000","last_active":"2023-12-19T18:17:27.361613+0000","last_peered":"2023-12-19T18:17:27.361613+0000","last_clean":"2023-12-19T18:17:27.361613+0000","last_became_active":"2023-12-19T18:17:26.946487+0000","last_became_peered":"2023-12-19T18:17:26.946487+0000","last_unstale":"2023-12-19T18:17:27.361613+0000","last_undegraded":"2023-12-19T18:17:27.361613+0000","last_fullsized":"2023-12-19T18:17:27.361613+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:49.419114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486712+0000","last_change":"2023-12-19T18:17:27.365652+0000","last_active":"2023-12-19T18:17:28.486712+0000","last_peered":"2023-12-19T18:17:28.486712+0000","last_clean":"2023-12-19T18:17:28.486712+0000","last_became_active":"2023-12-19T18:17:27.365336+0000","last_became_peered":"2023-12-19T18:17:27.365336+0000","last_unstale":"2023-12-19T18:17:28.486712+0000","last_undegraded":"2023-12-19T18:17:28.486712+0000","last_fullsized":"2023-12-19T18:17:28.486712+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:54.566933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.331525+0000","last_change":"2023-12-19T18:17:26.942091+0000","last_active":"2023-12-19T18:17:27.331525+0000","last_peered":"2023-12-19T18:17:27.331525+0000","last_clean":"2023-12-19T18:17:27.331525+0000","last_became_active":"2023-12-19T18:17:26.941806+0000","last_became_peered":"2023-12-19T18:17:26.941806+0000","last_unstale":"2023-12-19T18:17:27.331525+0000","last_undegraded":"2023-12-19T18:17:27.331525+0000","last_fullsized":"2023-12-19T18:17:27.331525+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:27:28.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485134+0000","last_change":"2023-12-19T18:17:27.367095+0000","last_active":"2023-12-19T18:17:28.485134+0000","last_peered":"2023-12-19T18:17:28.485134+0000","last_clean":"2023-12-19T18:17:28.485134+0000","last_became_active":"2023-12-19T18:17:27.366441+0000","last_became_peered":"2023-12-19T18:17:27.366441+0000","last_unstale":"2023-12-19T18:17:28.485134+0000","last_undegraded":"2023-12-19T18:17:28.485134+0000","last_fullsized":"2023-12-19T18:17:28.485134+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:10:47.532188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"232'2","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486748+0000","last_change":"2023-12-19T18:17:26.948576+0000","last_active":"2023-12-19T18:17:28.486748+0000","last_peered":"2023-12-19T18:17:28.486748+0000","last_clean":"2023-12-19T18:17:28.486748+0000","last_became_active":"2023-12-19T18:17:26.948310+0000","last_became_peered":"2023-12-19T18:17:26.948310+0000","last_unstale":"2023-12-19T18:17:28.486748+0000","last_undegraded":"2023-12-19T18:17:28.486748+0000","last_fullsized":"2023-12-19T18:17:28.486748+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:58.733231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486393+0000","last_change":"2023-12-19T18:17:27.358552+0000","last_active":"2023-12-19T18:17:28.486393+0000","last_peered":"2023-12-19T18:17:28.486393+0000","last_clean":"2023-12-19T18:17:28.486393+0000","last_became_active":"2023-12-19T18:17:27.358174+0000","last_became_peered":"2023-12-19T18:17:27.358174+0000","last_unstale":"2023-12-19T18:17:28.486393+0000","last_undegraded":"2023-12-19T18:17:28.486393+0000","last_fullsized":"2023-12-19T18:17:28.486393+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:14.654686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.354913+0000","last_change":"2023-12-19T18:17:26.947007+0000","last_active":"2023-12-19T18:17:27.354913+0000","last_peered":"2023-12-19T18:17:27.354913+0000","last_clean":"2023-12-19T18:17:27.354913+0000","last_became_active":"2023-12-19T18:17:26.946741+0000","last_became_peered":"2023-12-19T18:17:26.946741+0000","last_unstale":"2023-12-19T18:17:27.354913+0000","last_undegraded":"2023-12-19T18:17:27.354913+0000","last_fullsized":"2023-12-19T18:17:27.354913+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:20.105057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.337760+0000","last_change":"2023-12-19T18:17:27.348563+0000","last_active":"2023-12-19T18:17:28.337760+0000","last_peered":"2023-12-19T18:17:28.337760+0000","last_clean":"2023-12-19T18:17:28.337760+0000","last_became_active":"2023-12-19T18:17:27.348275+0000","last_became_peered":"2023-12-19T18:17:27.348275+0000","last_unstale":"2023-12-19T18:17:28.337760+0000","last_undegraded":"2023-12-19T18:17:28.337760+0000","last_fullsized":"2023-12-19T18:17:28.337760+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:45:15.369206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.355147+0000","last_change":"2023-12-19T18:17:26.944588+0000","last_active":"2023-12-19T18:17:27.355147+0000","last_peered":"2023-12-19T18:17:27.355147+0000","last_clean":"2023-12-19T18:17:27.355147+0000","last_became_active":"2023-12-19T18:17:26.944323+0000","last_became_peered":"2023-12-19T18:17:26.944323+0000","last_unstale":"2023-12-19T18:17:27.355147+0000","last_undegraded":"2023-12-19T18:17:27.355147+0000","last_fullsized":"2023-12-19T18:17:27.355147+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:25:45.208073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.484920+0000","last_change":"2023-12-19T18:17:27.342936+0000","last_active":"2023-12-19T18:17:28.484920+0000","last_peered":"2023-12-19T18:17:28.484920+0000","last_clean":"2023-12-19T18:17:28.484920+0000","last_became_active":"2023-12-19T18:17:27.342636+0000","last_became_peered":"2023-12-19T18:17:27.342636+0000","last_unstale":"2023-12-19T18:17:28.484920+0000","last_undegraded":"2023-12-19T18:17:28.484920+0000","last_fullsized":"2023-12-19T18:17:28.484920+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:43.945499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":11,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485577+0000","last_change":"2023-12-19T18:17:26.950102+0000","last_active":"2023-12-19T18:17:28.485577+0000","last_peered":"2023-12-19T18:17:28.485577+0000","last_clean":"2023-12-19T18:17:28.485577+0000","last_became_active":"2023-12-19T18:17:26.949810+0000","last_became_peered":"2023-12-19T18:17:26.949810+0000","last_unstale":"2023-12-19T18:17:28.485577+0000","last_undegraded":"2023-12-19T18:17:28.485577+0000","last_fullsized":"2023-12-19T18:17:28.485577+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:04:02.387088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485712+0000","last_change":"2023-12-19T18:17:27.350933+0000","last_active":"2023-12-19T18:17:28.485712+0000","last_peered":"2023-12-19T18:17:28.485712+0000","last_clean":"2023-12-19T18:17:28.485712+0000","last_became_active":"2023-12-19T18:17:27.350661+0000","last_became_peered":"2023-12-19T18:17:27.350661+0000","last_unstale":"2023-12-19T18:17:28.485712+0000","last_undegraded":"2023-12-19T18:17:28.485712+0000","last_fullsized":"2023-12-19T18:17:28.485712+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:25.159077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"232'2","reported_seq":13,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485792+0000","last_change":"2023-12-19T18:17:26.944158+0000","last_active":"2023-12-19T18:17:28.485792+0000","last_peered":"2023-12-19T18:17:28.485792+0000","last_clean":"2023-12-19T18:17:28.485792+0000","last_became_active":"2023-12-19T18:17:26.943845+0000","last_became_peered":"2023-12-19T18:17:26.943845+0000","last_unstale":"2023-12-19T18:17:28.485792+0000","last_undegraded":"2023-12-19T18:17:28.485792+0000","last_fullsized":"2023-12-19T18:17:28.485792+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:10.097246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486908+0000","last_change":"2023-12-19T18:17:27.361893+0000","last_active":"2023-12-19T18:17:28.486908+0000","last_peered":"2023-12-19T18:17:28.486908+0000","last_clean":"2023-12-19T18:17:28.486908+0000","last_became_active":"2023-12-19T18:17:27.361618+0000","last_became_peered":"2023-12-19T18:17:27.361618+0000","last_unstale":"2023-12-19T18:17:28.486908+0000","last_undegraded":"2023-12-19T18:17:28.486908+0000","last_fullsized":"2023-12-19T18:17:28.486908+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:50.421353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.334821+0000","last_change":"2023-12-19T18:17:26.944398+0000","last_active":"2023-12-19T18:17:27.334821+0000","last_peered":"2023-12-19T18:17:27.334821+0000","last_clean":"2023-12-19T18:17:27.334821+0000","last_became_active":"2023-12-19T18:17:26.942848+0000","last_became_peered":"2023-12-19T18:17:26.942848+0000","last_unstale":"2023-12-19T18:17:27.334821+0000","last_undegraded":"2023-12-19T18:17:27.334821+0000","last_fullsized":"2023-12-19T18:17:27.334821+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:01.415653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486648+0000","last_change":"2023-12-19T18:17:27.359609+0000","last_active":"2023-12-19T18:17:28.486648+0000","last_peered":"2023-12-19T18:17:28.486648+0000","last_clean":"2023-12-19T18:17:28.486648+0000","last_became_active":"2023-12-19T18:17:27.358162+0000","last_became_peered":"2023-12-19T18:17:27.358162+0000","last_unstale":"2023-12-19T18:17:28.486648+0000","last_undegraded":"2023-12-19T18:17:28.486648+0000","last_fullsized":"2023-12-19T18:17:28.486648+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:31:06.407037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.335792+0000","last_change":"2023-12-19T18:17:26.941195+0000","last_active":"2023-12-19T18:17:27.335792+0000","last_peered":"2023-12-19T18:17:27.335792+0000","last_clean":"2023-12-19T18:17:27.335792+0000","last_became_active":"2023-12-19T18:17:26.940845+0000","last_became_peered":"2023-12-19T18:17:26.940845+0000","last_unstale":"2023-12-19T18:17:27.335792+0000","last_undegraded":"2023-12-19T18:17:27.335792+0000","last_fullsized":"2023-12-19T18:17:27.335792+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:22.276149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485458+0000","last_change":"2023-12-19T18:17:27.351030+0000","last_active":"2023-12-19T18:17:28.485458+0000","last_peered":"2023-12-19T18:17:28.485458+0000","last_clean":"2023-12-19T18:17:28.485458+0000","last_became_active":"2023-12-19T18:17:27.350692+0000","last_became_peered":"2023-12-19T18:17:27.350692+0000","last_unstale":"2023-12-19T18:17:28.485458+0000","last_undegraded":"2023-12-19T18:17:28.485458+0000","last_fullsized":"2023-12-19T18:17:28.485458+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:15:54.445257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":11,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486754+0000","last_change":"2023-12-19T18:17:26.943936+0000","last_active":"2023-12-19T18:17:28.486754+0000","last_peered":"2023-12-19T18:17:28.486754+0000","last_clean":"2023-12-19T18:17:28.486754+0000","last_became_active":"2023-12-19T18:17:26.943609+0000","last_became_peered":"2023-12-19T18:17:26.943609+0000","last_unstale":"2023-12-19T18:17:28.486754+0000","last_undegraded":"2023-12-19T18:17:28.486754+0000","last_fullsized":"2023-12-19T18:17:28.486754+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:12.547671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.339426+0000","last_change":"2023-12-19T18:17:27.342408+0000","last_active":"2023-12-19T18:17:28.339426+0000","last_peered":"2023-12-19T18:17:28.339426+0000","last_clean":"2023-12-19T18:17:28.339426+0000","last_became_active":"2023-12-19T18:17:27.342114+0000","last_became_peered":"2023-12-19T18:17:27.342114+0000","last_unstale":"2023-12-19T18:17:28.339426+0000","last_undegraded":"2023-12-19T18:17:28.339426+0000","last_fullsized":"2023-12-19T18:17:28.339426+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:02.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.354734+0000","last_change":"2023-12-19T18:17:26.945192+0000","last_active":"2023-12-19T18:17:27.354734+0000","last_peered":"2023-12-19T18:17:27.354734+0000","last_clean":"2023-12-19T18:17:27.354734+0000","last_became_active":"2023-12-19T18:17:26.943303+0000","last_became_peered":"2023-12-19T18:17:26.943303+0000","last_unstale":"2023-12-19T18:17:27.354734+0000","last_undegraded":"2023-12-19T18:17:27.354734+0000","last_fullsized":"2023-12-19T18:17:27.354734+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486521+0000","last_change":"2023-12-19T18:17:27.362825+0000","last_active":"2023-12-19T18:17:28.486521+0000","last_peered":"2023-12-19T18:17:28.486521+0000","last_clean":"2023-12-19T18:17:28.486521+0000","last_became_active":"2023-12-19T18:17:27.362414+0000","last_became_peered":"2023-12-19T18:17:27.362414+0000","last_unstale":"2023-12-19T18:17:28.486521+0000","last_undegraded":"2023-12-19T18:17:28.486521+0000","last_fullsized":"2023-12-19T18:17:28.486521+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:21:44.939055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":11,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.338128+0000","last_change":"2023-12-19T18:17:26.940233+0000","last_active":"2023-12-19T18:17:27.338128+0000","last_peered":"2023-12-19T18:17:27.338128+0000","last_clean":"2023-12-19T18:17:27.338128+0000","last_became_active":"2023-12-19T18:17:26.939941+0000","last_became_peered":"2023-12-19T18:17:26.939941+0000","last_unstale":"2023-12-19T18:17:27.338128+0000","last_undegraded":"2023-12-19T18:17:27.338128+0000","last_fullsized":"2023-12-19T18:17:27.338128+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:15:20.805126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485559+0000","last_change":"2023-12-19T18:17:27.356126+0000","last_active":"2023-12-19T18:17:28.485559+0000","last_peered":"2023-12-19T18:17:28.485559+0000","last_clean":"2023-12-19T18:17:28.485559+0000","last_became_active":"2023-12-19T18:17:27.355794+0000","last_became_peered":"2023-12-19T18:17:27.355794+0000","last_unstale":"2023-12-19T18:17:28.485559+0000","last_undegraded":"2023-12-19T18:17:28.485559+0000","last_fullsized":"2023-12-19T18:17:28.485559+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:23.278389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":11,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485103+0000","last_change":"2023-12-19T18:17:26.944096+0000","last_active":"2023-12-19T18:17:28.485103+0000","last_peered":"2023-12-19T18:17:28.485103+0000","last_clean":"2023-12-19T18:17:28.485103+0000","last_became_active":"2023-12-19T18:17:26.941219+0000","last_became_peered":"2023-12-19T18:17:26.941219+0000","last_unstale":"2023-12-19T18:17:28.485103+0000","last_undegraded":"2023-12-19T18:17:28.485103+0000","last_fullsized":"2023-12-19T18:17:28.485103+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:22:09.866781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485551+0000","last_change":"2023-12-19T18:17:27.343300+0000","last_active":"2023-12-19T18:17:28.485551+0000","last_peered":"2023-12-19T18:17:28.485551+0000","last_clean":"2023-12-19T18:17:28.485551+0000","last_became_active":"2023-12-19T18:17:27.343020+0000","last_became_peered":"2023-12-19T18:17:27.343020+0000","last_unstale":"2023-12-19T18:17:28.485551+0000","last_undegraded":"2023-12-19T18:17:28.485551+0000","last_fullsized":"2023-12-19T18:17:28.485551+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:11.099486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":11,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.485651+0000","last_change":"2023-12-19T18:17:26.950561+0000","last_active":"2023-12-19T18:17:28.485651+0000","last_peered":"2023-12-19T18:17:28.485651+0000","last_clean":"2023-12-19T18:17:28.485651+0000","last_became_active":"2023-12-19T18:17:26.949052+0000","last_became_peered":"2023-12-19T18:17:26.949052+0000","last_unstale":"2023-12-19T18:17:28.485651+0000","last_undegraded":"2023-12-19T18:17:28.485651+0000","last_fullsized":"2023-12-19T18:17:28.485651+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:09:01.022466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":10,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-19T18:17:28.486079+0000","last_change":"2023-12-19T18:17:27.349332+0000","last_active":"2023-12-19T18:17:28.486079+0000","last_peered":"2023-12-19T18:17:28.486079+0000","last_clean":"2023-12-19T18:17:28.486079+0000","last_became_active":"2023-12-19T18:17:27.349050+0000","last_became_peered":"2023-12-19T18:17:27.349050+0000","last_unstale":"2023-12-19T18:17:28.486079+0000","last_undegraded":"2023-12-19T18:17:28.486079+0000","last_fullsized":"2023-12-19T18:17:28.486079+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:33.501865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"232'2","reported_seq":13,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.369367+0000","last_change":"2023-12-19T18:17:26.947941+0000","last_active":"2023-12-19T18:17:27.369367+0000","last_peered":"2023-12-19T18:17:27.369367+0000","last_clean":"2023-12-19T18:17:27.369367+0000","last_became_active":"2023-12-19T18:17:26.945013+0000","last_became_peered":"2023-12-19T18:17:26.945013+0000","last_unstale":"2023-12-19T18:17:27.369367+0000","last_undegraded":"2023-12-19T18:17:27.369367+0000","last_fullsized":"2023-12-19T18:17:27.369367+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:47:39.421442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"218'368","reported_seq":1280,"reported_epoch":232,"state":"active+clean","last_fresh":"2023-12-19T18:17:27.333681+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:17:27.333681+0000","last_peered":"2023-12-19T18:17:27.333681+0000","last_clean":"2023-12-19T18:17:27.333681+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:17:27.333681+0000","last_undegraded":"2023-12-19T18:17:27.333681+0000","last_fullsized":"2023-12-19T18:17:27.333681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2306,"num_read_kb":4823,"num_write":615,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2306,"num_read_kb":4823,"num_write":615,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":368,"ondisk_log_size":368,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510577,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48436,"kb_used_data":3436,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323404,"statfs":{"total":96636764160,"available":96587165696,"internally_reserved":0,"allocated":3518464,"data_stored":2147027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68500000000000005,"15min":0.71499999999999997},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79000000000000004,"5min":1.048,"15min":1.1180000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71299999999999997,"15min":0.73199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90900000000000003,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":1.0389999999999999,"15min":1.0580000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.74199999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.69099999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.73099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.0669999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.996,"5min":1.0820000000000001,"15min":1.236},"last":0.73399999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88100000000000001,"5min":1.002,"15min":1.079},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.49399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85899999999999999,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.79200000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.441,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.302},"last":0.755},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.622,"15min":0.629},"min":{"1min":0.50800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.0209999999999999},"last":0.71299999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.90600000000000003,"5min":1.097,"15min":1.1299999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.55800000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95699999999999996,"5min":1.026,"15min":1.0800000000000001},"last":0.624}]},{"osd":6,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.61699999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.0469999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.093},"last":0.80800000000000005}]}]},{"osd":6,"up_from":15,"seq":64424510577,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47860,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323980,"statfs":{"total":96636764160,"available":96587755520,"internally_reserved":0,"allocated":2928640,"data_stored":1556645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72299999999999998,"15min":0.73799999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.042},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.42799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97599999999999998,"5min":1.224,"15min":1.224},"last":0.86099999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68899999999999995,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86699999999999999,"5min":1.103,"15min":1.103},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.745,"15min":0.752},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":0.93600000000000005,"5min":0.97999999999999998,"15min":1.1259999999999999},"last":0.96099999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.752,"15min":0.76000000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98399999999999999,"5min":1.0549999999999999,"15min":1.0609999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70899999999999996,"15min":0.745},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98999999999999999,"5min":1.0089999999999999,"15min":1.111},"last":0.78600000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.754,"15min":0.77800000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.1519999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71599999999999997,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.036,"5min":1.268,"15min":1.268},"last":0.876}]},{"osd":4,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.373,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0469999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63200000000000001,"15min":0.61599999999999999},"min":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.55300000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.47599999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.997,"5min":1.194,"15min":1.194},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68999999999999995,"15min":0.67100000000000004},"min":{"1min":0.46800000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1339999999999999,"5min":1.141,"15min":1.141},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56599999999999995,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.54600000000000004,"5min":0.377,"15min":0.377},"max":{"1min":1.1140000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.82999999999999996}]}]},{"osd":5,"up_from":15,"seq":64424510577,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47852,"kb_used_data":2852,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323988,"statfs":{"total":96636764160,"available":96587763712,"internally_reserved":0,"allocated":2920448,"data_stored":1556589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.69599999999999995,"15min":0.72499999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.94999999999999996,"5min":1.002,"15min":1.0149999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.93100000000000005,"5min":0.96099999999999997,"15min":1.0229999999999999},"last":0.628}]},{"osd":1,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.497},"max":{"1min":0.84599999999999997,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.498,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97599999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59699999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.58599999999999997,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":1.0069999999999999,"5min":1.0509999999999999,"15min":1.2250000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.73199999999999998},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88600000000000001,"5min":1.004,"15min":1.4279999999999999},"last":0.76100000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.748,"15min":0.76200000000000001},"min":{"1min":0.56799999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.90400000000000003,"5min":1.0129999999999999,"15min":1.05},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93400000000000005,"5min":1.022,"15min":1.0329999999999999},"last":0.65300000000000002}]},{"osd":4,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.81699999999999995,"5min":0.94499999999999995,"15min":1.0800000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94299999999999995,"5min":1.208,"15min":1.208},"last":0.79800000000000004}]},{"osd":6,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65800000000000003,"15min":0.65700000000000003},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.85799999999999998,"5min":1.008,"15min":1.0449999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.501,"5min":0.439,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":1.169,"15min":1.169},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51100000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.079,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81899999999999995}]}]},{"osd":4,"up_from":15,"seq":64424510577,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47856,"kb_used_data":2856,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323984,"statfs":{"total":96636764160,"available":96587759616,"internally_reserved":0,"allocated":2924544,"data_stored":1557171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.51600000000000001,"5min":0.45800000000000002,"15min":0.45000000000000001},"max":{"1min":1.016,"5min":1.036,"15min":1.109},"last":0.621},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75600000000000001,"15min":0.752},"min":{"1min":0.502,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.91800000000000004,"5min":0.997,"15min":1.01},"last":0.64000000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.59199999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.94699999999999995,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76800000000000002,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.97899999999999998,"5min":0.997,"15min":1.1020000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79100000000000004,"15min":0.77700000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.47699999999999998},"max":{"1min":0.998,"5min":1.0289999999999999,"15min":1.071},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.79000000000000004,"15min":0.77700000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0109999999999999,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.78100000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76700000000000002,"15min":0.75800000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93899999999999995,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.55500000000000005},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.59599999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65300000000000002,"15min":0.63100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67200000000000004,"15min":0.64600000000000002},"min":{"1min":0.495,"5min":0.42899999999999999,"15min":0.39600000000000002},"max":{"1min":0.88900000000000001,"5min":1.042,"15min":1.042},"last":0.80200000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.66100000000000003},"min":{"1min":0.51800000000000002,"5min":0.47099999999999997,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":1.024,"15min":1.0669999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0760000000000001},"last":0.83299999999999996}]},{"osd":7,"last update":"Tue Dec 19 18:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.53300000000000003,"5min":0.50800000000000001,"15min":0.41899999999999998},"max":{"1min":0.92400000000000004,"5min":1.087,"15min":1.087},"last":0.748},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77600000000000002,"15min":0.76200000000000001},"min":{"1min":0.56799999999999995,"5min":0.56100000000000005,"15min":0.46999999999999997},"max":{"1min":0.98599999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.81999999999999995}]}]},{"osd":3,"up_from":15,"seq":64424510577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47868,"kb_used_data":2868,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323972,"statfs":{"total":96636764160,"available":96587747328,"internally_reserved":0,"allocated":2936832,"data_stored":1558223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.34599999999999997,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.63200000000000001,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.502,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.74099999999999999,"5min":0.78300000000000003,"15min":0.80000000000000004},"last":0.40999999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.499,"15min":0.498},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.71499999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.71799999999999997,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.84199999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.77000000000000002,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.55500000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.81599999999999995,"15min":0.82799999999999996},"min":{"1min":0.57299999999999995,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.012,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.81999999999999995,"15min":0.82299999999999995},"min":{"1min":0.59799999999999998,"5min":0.57499999999999996,"15min":0.56599999999999995},"max":{"1min":0.86499999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.81399999999999995}]},{"osd":5,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82799999999999996,"15min":0.84199999999999997},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99099999999999999,"5min":1.1739999999999999,"15min":1.397},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.59399999999999997,"5min":0.55900000000000005,"15min":0.55100000000000005},"max":{"1min":0.91000000000000003,"5min":1.2230000000000001,"15min":1.262},"last":0.72199999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.83899999999999997,"15min":0.84799999999999998},"min":{"1min":0.60199999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.96099999999999997,"5min":1.234,"15min":1.274},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.83799999999999997,"15min":0.84499999999999997},"min":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.94599999999999995,"5min":1.165,"15min":1.302},"last":0.748}]},{"osd":7,"last update":"Tue Dec 19 18:16:34 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.90400000000000003,"15min":0.90500000000000003},"min":{"1min":0.66000000000000003,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0640000000000001,"5min":1.3540000000000001,"15min":1.486},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89500000000000002},"min":{"1min":0.68899999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":0.93000000000000005,"5min":1.421,"15min":1.421},"last":0.89600000000000002}]}]},{"osd":2,"up_from":15,"seq":64424510576,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47872,"kb_used_data":2872,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323968,"statfs":{"total":96636764160,"available":96587743232,"internally_reserved":0,"allocated":2940928,"data_stored":1558209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.441,"15min":0.45000000000000001},"min":{"1min":0.26300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.70499999999999996,"5min":0.754,"15min":0.754},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48899999999999999,"15min":0.504},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68000000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.45200000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59899999999999998,"15min":0.61599999999999999},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"max":{"1min":0.77800000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.73899999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41399999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58599999999999997,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76800000000000002,"5min":0.80800000000000005,"15min":0.95999999999999996},"last":0.5},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.36899999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.875},"last":0.54500000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0580000000000001,"5min":1.0880000000000001,"15min":1.306},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77500000000000002,"15min":0.79200000000000004},"min":{"1min":0.48299999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0609999999999999,"5min":1.2070000000000001,"15min":1.466},"last":0.70299999999999996}]},{"osd":5,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.77200000000000002,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.3049999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.80000000000000004,"15min":0.81899999999999995},"min":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.04,"5min":1.1519999999999999,"15min":1.429},"last":0.752}]},{"osd":6,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.81299999999999994,"15min":0.83799999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0329999999999999,"5min":1.216,"15min":1.4119999999999999},"last":0.88},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77600000000000002,"15min":0.80000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.008,"5min":1.117,"15min":1.2470000000000001},"last":0.90200000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.82199999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.56000000000000005},"max":{"1min":1.0269999999999999,"5min":1.159,"15min":1.45},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.83899999999999997,"15min":0.85599999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.093,"5min":1.276,"15min":1.276},"last":0.84699999999999998}]}]},{"osd":1,"up_from":15,"seq":64424510577,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60144,"kb_used_data":2856,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311696,"statfs":{"total":96636764160,"available":96575176704,"internally_reserved":0,"allocated":2924544,"data_stored":1556679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44800000000000001,"15min":0.45800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56399999999999995,"5min":0.71999999999999997,"15min":0.89400000000000002},"last":0.371},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79600000000000004,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.495}]},{"osd":2,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66000000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.66600000000000004,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.41499999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.622,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.747,"5min":0.94199999999999995,"15min":0.94499999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.628,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.78100000000000003,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92700000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77700000000000002,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.081,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.76000000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76900000000000002,"15min":0.80000000000000004},"min":{"1min":0.58699999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0189999999999999,"5min":1.236,"15min":1.2609999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81799999999999995},"min":{"1min":0.58199999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0669999999999999,"5min":1.353,"15min":1.353},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.505},"max":{"1min":1.0009999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.82399999999999995,"15min":0.84099999999999997},"min":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.034,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.84499999999999997}]},{"osd":7,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84599999999999997,"15min":0.85399999999999998},"min":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.56499999999999995},"max":{"1min":1.147,"5min":1.399,"15min":1.399},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.82799999999999996,"15min":0.85199999999999998},"min":{"1min":0.53000000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.181,"5min":1.365,"15min":1.365},"last":0.89500000000000002}]}]},{"osd":0,"up_from":14,"seq":60129543282,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60728,"kb_used_data":3440,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311112,"statfs":{"total":96636764160,"available":96574578688,"internally_reserved":0,"allocated":3522560,"data_stored":2147629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41899999999999998,"15min":0.45300000000000001},"min":{"1min":0.29699999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55300000000000005,"5min":0.69299999999999995,"15min":1.216},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.49399999999999999,"15min":0.51900000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67600000000000005,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.48999999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.48799999999999999,"15min":0.52600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.77000000000000002,"15min":1.1699999999999999},"last":0.441},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52300000000000002,"15min":0.54700000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.96799999999999997},"last":0.54400000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.51400000000000001,"15min":0.55000000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.59999999999999998,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59899999999999998,"15min":0.64400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.032},"last":0.47299999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.80600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.496},"max":{"1min":0.93899999999999995,"5min":1.1200000000000001,"15min":1.381},"last":0.873},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.78600000000000003,"15min":0.81100000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.47799999999999998},"max":{"1min":0.84799999999999998,"5min":1.137,"15min":1.2849999999999999},"last":0.84799999999999998}]},{"osd":5,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.93799999999999994,"5min":1.1180000000000001,"15min":1.3420000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.79100000000000004,"15min":0.81299999999999994},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.052,"5min":1.1259999999999999,"15min":1.288},"last":0.83699999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81200000000000006,"15min":0.83899999999999997},"min":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.51300000000000001},"max":{"1min":0.97899999999999998,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.80000000000000004,"15min":0.82699999999999996},"min":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97999999999999998,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.85599999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0149999999999999,"5min":1.1899999999999999,"15min":1.415},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.82599999999999996,"15min":0.85099999999999998},"min":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96899999999999997,"5min":1.0980000000000001,"15min":1.395},"last":0.89000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:17:31.828 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:17:31.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:17:32.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f3498ec3640 1 Processor -- start 2023-12-19T18:17:32.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f3498ec3640 1 -- start start 2023-12-19T18:17:32.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f3498ec3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34940b0c30 0x7f34940ada90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:32.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f3498ec3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3494070db0 0x7f349406b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:32.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f3498ec3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349406b500 con 0x7f349415ada0 2023-12-19T18:17:32.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f3498ec3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34940715b0 con 0x7f34940b0c30 2023-12-19T18:17:32.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f3498ec3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34940adfd0 con 0x7f3494070db0 2023-12-19T18:17:32.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f348bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34940b0c30 0x7f34940ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:32.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f348bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34940b0c30 0x7f34940ada90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47620/0 (socket says 172.21.15.26:47620) 2023-12-19T18:17:32.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f348bfff640 1 -- 172.21.15.26:0/458353236 learned_addr learned my addr 172.21.15.26:0/458353236 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:32.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f3492d9d640 1 --2- 172.21.15.26:0/458353236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3494070db0 0x7f349406b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:32.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f348bfff640 1 -- 172.21.15.26:0/458353236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3494070db0 msgr2=0x7f349406b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:32.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f348bfff640 1 --2- 172.21.15.26:0/458353236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3494070db0 0x7f349406b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f348bfff640 1 -- 172.21.15.26:0/458353236 >> v1:172.21.15.160:6789/0 conn(0x7f349415ada0 legacy=0x7f34940b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:32.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.013+0000 7f348bfff640 1 -- 172.21.15.26:0/458353236 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3494070b60 con 0x7f34940b0c30 2023-12-19T18:17:32.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.017+0000 7f348bfff640 1 --2- 172.21.15.26:0/458353236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34940b0c30 0x7f34940ada90 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f347c0098a0 tx=0x7f347c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e94375c2f2cb742 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:32.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.017+0000 7f3491d9b640 1 -- 172.21.15.26:0/458353236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c0aa670 con 0x7f34940b0c30 2023-12-19T18:17:32.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.017+0000 7f3491d9b640 1 -- 172.21.15.26:0/458353236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c0aa7d0 con 0x7f34940b0c30 2023-12-19T18:17:32.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.017+0000 7f3491d9b640 1 -- 172.21.15.26:0/458353236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c0aab60 con 0x7f34940b0c30 2023-12-19T18:17:32.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.017+0000 7f3498ec3640 1 -- 172.21.15.26:0/458353236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34940b0c30 msgr2=0x7f34940ada90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:32.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.017+0000 7f3498ec3640 1 --2- 172.21.15.26:0/458353236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34940b0c30 0x7f34940ada90 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f347c0098a0 tx=0x7f347c01c4f0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.017+0000 7f3498ec3640 1 -- 172.21.15.26:0/458353236 shutdown_connections 2023-12-19T18:17:32.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.017+0000 7f3498ec3640 1 --2- 172.21.15.26:0/458353236 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3494070db0 0x7f349406b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.017+0000 7f3498ec3640 1 --2- 172.21.15.26:0/458353236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34940b0c30 0x7f34940ada90 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.017+0000 7f3498ec3640 1 -- 172.21.15.26:0/458353236 >> 172.21.15.26:0/458353236 conn(0x7f34940660d0 msgr2=0x7f34940ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:32.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.017+0000 7f3498ec3640 1 -- 172.21.15.26:0/458353236 shutdown_connections 2023-12-19T18:17:32.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.017+0000 7f3498ec3640 1 -- 172.21.15.26:0/458353236 wait complete. 2023-12-19T18:17:32.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3498ec3640 1 Processor -- start 2023-12-19T18:17:32.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3498ec3640 1 -- start start 2023-12-19T18:17:32.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3498ec3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3494070db0 0x7f3494154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:32.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3498ec3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34940b0c30 0x7f34940c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:32.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3498ec3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f349415ada0 0x7f34940c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:32.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3498ec3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349415e120 con 0x7f349415ada0 2023-12-19T18:17:32.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3498ec3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34940c7780 con 0x7f3494070db0 2023-12-19T18:17:32.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3498ec3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34940c6bb0 con 0x7f34940b0c30 2023-12-19T18:17:32.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3492d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f349415ada0 0x7f34940c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:32.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3492d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f349415ada0 0x7f34940c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47622/0 (socket says 172.21.15.26:47622) 2023-12-19T18:17:32.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3492d9d640 1 -- 172.21.15.26:0/3519321631 learned_addr learned my addr 172.21.15.26:0/3519321631 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:32.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f349259c640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3494070db0 0x7f3494154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:32.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f348bfff640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34940b0c30 0x7f34940c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:32.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3492d9d640 1 -- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34940b0c30 msgr2=0x7f34940c7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:32.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3492d9d640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34940b0c30 0x7f34940c7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3492d9d640 1 -- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3494070db0 msgr2=0x7f3494154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:32.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3492d9d640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3494070db0 0x7f3494154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3492d9d640 1 -- 172.21.15.26:0/3519321631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c0094d0 con 0x7f349415ada0 2023-12-19T18:17:32.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3492d9d640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f349415ada0 0x7f34940c8150 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f348c00efc0 tx=0x7f348c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:32.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f348affd640 1 -- 172.21.15.26:0/3519321631 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c014480 con 0x7f349415ada0 2023-12-19T18:17:32.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f348affd640 1 -- 172.21.15.26:0/3519321631 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c00f040 con 0x7f349415ada0 2023-12-19T18:17:32.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3498ec3640 1 -- 172.21.15.26:0/3519321631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34940c6e50 con 0x7f349415ada0 2023-12-19T18:17:32.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f348affd640 1 -- 172.21.15.26:0/3519321631 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c014820 con 0x7f349415ada0 2023-12-19T18:17:32.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.021+0000 7f3498ec3640 1 -- 172.21.15.26:0/3519321631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34940c7410 con 0x7f349415ada0 2023-12-19T18:17:32.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.025+0000 7f3498ec3640 1 -- 172.21.15.26:0/3519321631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34940c7910 con 0x7f349415ada0 2023-12-19T18:17:32.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.025+0000 7f348affd640 1 -- 172.21.15.26:0/3519321631 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f348c014980 con 0x7f349415ada0 2023-12-19T18:17:32.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.025+0000 7f348affd640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3474065cd0 0x7f3474068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:32.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.025+0000 7f348affd640 1 -- 172.21.15.26:0/3519321631 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f348c0844b0 con 0x7f349415ada0 2023-12-19T18:17:32.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.025+0000 7f349259c640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3474065cd0 0x7f3474068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:32.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.025+0000 7f349259c640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3474065cd0 0x7f3474068190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3484007fd0 tx=0x7f3484009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:32.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.029+0000 7f348affd640 1 -- 172.21.15.26:0/3519321631 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f348c051710 con 0x7f349415ada0 2023-12-19T18:17:32.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.241+0000 7f3498ec3640 1 -- 172.21.15.26:0/3519321631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3494074270 con 0x7f349415ada0 2023-12-19T18:17:32.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.245+0000 7f348affd640 1 -- 172.21.15.26:0/3519321631 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v53) v1 ==== 190+0+0 (secure 0 0 0) 0x7f348c054d30 con 0x7f349415ada0 2023-12-19T18:17:32.250 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2023-12-19T18:17:32.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.249+0000 7f3498ec3640 1 -- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3474065cd0 msgr2=0x7f3474068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:32.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.249+0000 7f3498ec3640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3474065cd0 0x7f3474068190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3484007fd0 tx=0x7f3484009040 comp rx=0 tx=0).stop 2023-12-19T18:17:32.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.249+0000 7f3498ec3640 1 -- 172.21.15.26:0/3519321631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f349415ada0 msgr2=0x7f34940c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:32.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.249+0000 7f3498ec3640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f349415ada0 0x7f34940c8150 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f348c00efc0 tx=0x7f348c00c490 comp rx=0 tx=0).stop 2023-12-19T18:17:32.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.249+0000 7f3498ec3640 1 -- 172.21.15.26:0/3519321631 shutdown_connections 2023-12-19T18:17:32.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.249+0000 7f3498ec3640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3474065cd0 0x7f3474068190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.249+0000 7f3498ec3640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f349415ada0 0x7f34940c8150 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.249+0000 7f3498ec3640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f34940b0c30 0x7f34940c7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.249+0000 7f3498ec3640 1 --2- 172.21.15.26:0/3519321631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3494070db0 0x7f3494154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.249+0000 7f3498ec3640 1 -- 172.21.15.26:0/3519321631 >> 172.21.15.26:0/3519321631 conn(0x7f34940660d0 msgr2=0x7f34941535f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:32.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.249+0000 7f3498ec3640 1 -- 172.21.15.26:0/3519321631 shutdown_connections 2023-12-19T18:17:32.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.249+0000 7f3498ec3640 1 -- 172.21.15.26:0/3519321631 wait complete. 2023-12-19T18:17:32.272 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa2b2f2640 1 Processor -- start 2023-12-19T18:17:32.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa2b2f2640 1 -- start start 2023-12-19T18:17:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa2b2f2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faa240b88e0 0x7faa240b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa2b2f2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa240d2960 con 0x7faa240b4b40 2023-12-19T18:17:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa2b2f2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa240b4f40 con 0x7faa2415cf60 2023-12-19T18:17:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa2b2f2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa240d15f0 con 0x7faa240b88e0 2023-12-19T18:17:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa2988f640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faa240b4b40 0x7faa240b5cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47664/0 (socket says 172.21.15.26:47664) 2023-12-19T18:17:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa2988f640 1 -- 172.21.15.26:0/3644750084 learned_addr learned my addr 172.21.15.26:0/3644750084 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa2888d640 1 --2- 172.21.15.26:0/3644750084 >> v2:172.21.15.160:3300/0 conn(0x7faa240b88e0 0x7faa240b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:32.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa1bfff640 1 -- 172.21.15.26:0/3644750084 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980128449 0 0) 0x7faa240d2960 con 0x7faa240b4b40 2023-12-19T18:17:32.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa1bfff640 1 -- 172.21.15.26:0/3644750084 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa10003530 con 0x7faa240b4b40 2023-12-19T18:17:32.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa1bfff640 1 -- 172.21.15.26:0/3644750084 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337253139 0 0) 0x7faa240b4f40 con 0x7faa2415cf60 2023-12-19T18:17:32.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa1bfff640 1 -- 172.21.15.26:0/3644750084 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa240d2960 con 0x7faa2415cf60 2023-12-19T18:17:32.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa2888d640 1 -- 172.21.15.26:0/3644750084 >> v1:172.21.15.160:6789/0 conn(0x7faa2415cf60 legacy=0x7faa240d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:32.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa2888d640 1 -- 172.21.15.26:0/3644750084 >> v1:172.21.15.26:6789/0 conn(0x7faa240b4b40 legacy=0x7faa240b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:32.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.413+0000 7faa2888d640 1 -- 172.21.15.26:0/3644750084 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa240d18c0 con 0x7faa240b88e0 2023-12-19T18:17:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.417+0000 7faa2888d640 1 --2- 172.21.15.26:0/3644750084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa240b88e0 0x7faa240b4600 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7faa140097c0 tx=0x7faa1401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee500ae658bae0b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.417+0000 7faa1bfff640 1 -- 172.21.15.26:0/3644750084 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140a8970 con 0x7faa240b88e0 2023-12-19T18:17:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.417+0000 7faa1bfff640 1 -- 172.21.15.26:0/3644750084 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa140a8ad0 con 0x7faa240b88e0 2023-12-19T18:17:32.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.417+0000 7faa1bfff640 1 -- 172.21.15.26:0/3644750084 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140a8e60 con 0x7faa240b88e0 2023-12-19T18:17:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.417+0000 7faa2b2f2640 1 -- 172.21.15.26:0/3644750084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa240b88e0 msgr2=0x7faa240b4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.417+0000 7faa2b2f2640 1 --2- 172.21.15.26:0/3644750084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa240b88e0 0x7faa240b4600 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7faa140097c0 tx=0x7faa1401ec70 comp rx=0 tx=0).stop 2023-12-19T18:17:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.417+0000 7faa2b2f2640 1 -- 172.21.15.26:0/3644750084 shutdown_connections 2023-12-19T18:17:32.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.417+0000 7faa2b2f2640 1 --2- 172.21.15.26:0/3644750084 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa240b88e0 0x7faa240b4600 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.417+0000 7faa2b2f2640 1 -- 172.21.15.26:0/3644750084 >> 172.21.15.26:0/3644750084 conn(0x7faa24065ee0 msgr2=0x7faa240cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.417+0000 7faa2b2f2640 1 -- 172.21.15.26:0/3644750084 shutdown_connections 2023-12-19T18:17:32.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.417+0000 7faa2b2f2640 1 -- 172.21.15.26:0/3644750084 wait complete. 2023-12-19T18:17:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2b2f2640 1 Processor -- start 2023-12-19T18:17:32.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2b2f2640 1 -- start start 2023-12-19T18:17:32.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2b2f2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa240b4b40 0x7faa2415f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:32.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2b2f2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa240b88e0 0x7faa240b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:32.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2b2f2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa2415cf60 0x7faa240b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2b2f2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2415a880 con 0x7faa240b88e0 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2b2f2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa240aa2f0 con 0x7faa240b4b40 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2b2f2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa240aa5d0 con 0x7faa2415cf60 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2888d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa240b88e0 0x7faa240b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2888d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa240b88e0 0x7faa240b77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47632/0 (socket says 172.21.15.26:47632) 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2888d640 1 -- 172.21.15.26:0/492924541 learned_addr learned my addr 172.21.15.26:0/492924541 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2908e640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa240b4b40 0x7faa2415f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2988f640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa2415cf60 0x7faa240b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2888d640 1 -- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa2415cf60 msgr2=0x7faa240b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2888d640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa2415cf60 0x7faa240b7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2888d640 1 -- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa240b4b40 msgr2=0x7faa2415f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2888d640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa240b4b40 0x7faa2415f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2888d640 1 -- 172.21.15.26:0/492924541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa14009420 con 0x7faa240b88e0 2023-12-19T18:17:32.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2888d640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa240b88e0 0x7faa240b77a0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7faa140b3040 tx=0x7faa1401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:32.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa19ffb640 1 -- 172.21.15.26:0/492924541 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140b0d60 con 0x7faa240b88e0 2023-12-19T18:17:32.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa19ffb640 1 -- 172.21.15.26:0/492924541 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa140b9430 con 0x7faa240b88e0 2023-12-19T18:17:32.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa19ffb640 1 -- 172.21.15.26:0/492924541 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140b97e0 con 0x7faa240b88e0 2023-12-19T18:17:32.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2b2f2640 1 -- 172.21.15.26:0/492924541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa240aa8b0 con 0x7faa240b88e0 2023-12-19T18:17:32.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.421+0000 7faa2b2f2640 1 -- 172.21.15.26:0/492924541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa240aae70 con 0x7faa240b88e0 2023-12-19T18:17:32.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.425+0000 7faa2b2f2640 1 -- 172.21.15.26:0/492924541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa240ab060 con 0x7faa240b88e0 2023-12-19T18:17:32.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.425+0000 7faa19ffb640 1 -- 172.21.15.26:0/492924541 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7faa140b7040 con 0x7faa240b88e0 2023-12-19T18:17:32.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.425+0000 7faa19ffb640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faa08065ca0 0x7faa08068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:32.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.425+0000 7faa19ffb640 1 -- 172.21.15.26:0/492924541 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7faa14121400 con 0x7faa240b88e0 2023-12-19T18:17:32.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.425+0000 7faa2908e640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faa08065ca0 0x7faa08068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:32.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.425+0000 7faa2908e640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faa08065ca0 0x7faa08068160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa200011f0 tx=0x7faa20009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:32.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.429+0000 7faa19ffb640 1 -- 172.21.15.26:0/492924541 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faa140ee6d0 con 0x7faa240b88e0 2023-12-19T18:17:32.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.617+0000 7faa2b2f2640 1 -- 172.21.15.26:0/492924541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa24074270 con 0x7faa240b88e0 2023-12-19T18:17:32.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.621+0000 7faa19ffb640 1 -- 172.21.15.26:0/492924541 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4730 (secure 0 0 0) 0x7faa140f1cf0 con 0x7faa240b88e0 2023-12-19T18:17:32.623 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:32.623 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":112,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2727392302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2727392302},{"type":"v1","addr":"172.21.15.160:6836","nonce":2727392302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1029020739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1029020739},{"type":"v1","addr":"172.21.15.160:6837","nonce":1029020739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":11424,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3251191056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3251191056},{"type":"v1","addr":"172.21.15.26:6834","nonce":3251191056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:27.325271+0000","modified":"2023-12-19T18:17:30.337993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11427":{"gid":11427,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2285370499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2285370499},{"type":"v1","addr":"172.21.15.26:6835","nonce":2285370499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-12-19T18:17:32.624 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 112 2023-12-19T18:17:32.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.621+0000 7faa2b2f2640 1 -- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faa08065ca0 msgr2=0x7faa08068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.621+0000 7faa2b2f2640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faa08065ca0 0x7faa08068160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faa200011f0 tx=0x7faa20009040 comp rx=0 tx=0).stop 2023-12-19T18:17:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.621+0000 7faa2b2f2640 1 -- 172.21.15.26:0/492924541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa240b88e0 msgr2=0x7faa240b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.621+0000 7faa2b2f2640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa240b88e0 0x7faa240b77a0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7faa140b3040 tx=0x7faa1401ba10 comp rx=0 tx=0).stop 2023-12-19T18:17:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.621+0000 7faa2b2f2640 1 -- 172.21.15.26:0/492924541 shutdown_connections 2023-12-19T18:17:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.621+0000 7faa2b2f2640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faa08065ca0 0x7faa08068160 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.621+0000 7faa2b2f2640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faa2415cf60 0x7faa240b7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.621+0000 7faa2b2f2640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa240b88e0 0x7faa240b77a0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.621+0000 7faa2b2f2640 1 --2- 172.21.15.26:0/492924541 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faa240b4b40 0x7faa2415f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.621+0000 7faa2b2f2640 1 -- 172.21.15.26:0/492924541 >> 172.21.15.26:0/492924541 conn(0x7faa24065ee0 msgr2=0x7faa24069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.621+0000 7faa2b2f2640 1 -- 172.21.15.26:0/492924541 shutdown_connections 2023-12-19T18:17:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:32.621+0000 7faa2b2f2640 1 -- 172.21.15.26:0/492924541 wait complete. 2023-12-19T18:17:33.048 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:17:27.325271+0000', 'modified': '2023-12-19T18:17:30.337993+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11427': {'gid': 11427, 'name': 'c', 'rank': 0, 'incarnation': 108, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/2285370499', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6832', 'nonce': 2285370499}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 2285370499}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:17:33.048 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:17:33.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-19T18:17:33.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf67812640 1 Processor -- start 2023-12-19T18:17:33.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf67812640 1 -- start start 2023-12-19T18:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf67812640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcf60155030 0x7fcf6015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf67812640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcf6015f370 0x7fcf6006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf67812640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6006c970 con 0x7fcf600ce740 2023-12-19T18:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf67812640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf6015b1f0 con 0x7fcf6015f370 2023-12-19T18:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf67812640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf6015b4d0 con 0x7fcf60155030 2023-12-19T18:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf64dad640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcf6015f370 0x7fcf6006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf64dad640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcf6015f370 0x7fcf6006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47642/0 (socket says 172.21.15.26:47642) 2023-12-19T18:17:33.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf64dad640 1 -- 172.21.15.26:0/498345777 learned_addr learned my addr 172.21.15.26:0/498345777 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf65daf640 1 --1- 172.21.15.26:0/498345777 >> v1:172.21.15.26:6789/0 conn(0x7fcf600ce740 0x7fcf600ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47676/0 (socket says 172.21.15.26:47676) 2023-12-19T18:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf655ae640 1 --2- 172.21.15.26:0/498345777 >> v2:172.21.15.160:3300/0 conn(0x7fcf60155030 0x7fcf6015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf64dad640 1 -- 172.21.15.26:0/498345777 >> v2:172.21.15.160:3300/0 conn(0x7fcf60155030 msgr2=0x7fcf6015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf64dad640 1 --2- 172.21.15.26:0/498345777 >> v2:172.21.15.160:3300/0 conn(0x7fcf60155030 0x7fcf6015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf64dad640 1 -- 172.21.15.26:0/498345777 >> v1:172.21.15.26:6789/0 conn(0x7fcf600ce740 legacy=0x7fcf600ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf64dad640 1 -- 172.21.15.26:0/498345777 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6015b7b0 con 0x7fcf6015f370 2023-12-19T18:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf57fff640 1 -- 172.21.15.26:0/498345777 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592638378 0 0) 0x7fcf6006c970 con 0x7fcf600ce740 2023-12-19T18:17:33.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.193+0000 7fcf655ae640 1 --2- 172.21.15.26:0/498345777 >> v2:172.21.15.160:3300/0 conn(0x7fcf60155030 0x7fcf6015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:33.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.197+0000 7fcf64dad640 1 --2- 172.21.15.26:0/498345777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf6015f370 0x7fcf6006c080 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fcf600cf4a0 tx=0x7fcf5001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0a8396ba6164ad server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.197+0000 7fcf57fff640 1 -- 172.21.15.26:0/498345777 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf500ab620 con 0x7fcf6015f370 2023-12-19T18:17:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.197+0000 7fcf57fff640 1 -- 172.21.15.26:0/498345777 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf500ab780 con 0x7fcf6015f370 2023-12-19T18:17:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.197+0000 7fcf57fff640 1 -- 172.21.15.26:0/498345777 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf500abb10 con 0x7fcf6015f370 2023-12-19T18:17:33.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.197+0000 7fcf67812640 1 -- 172.21.15.26:0/498345777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf6015f370 msgr2=0x7fcf6006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.197+0000 7fcf67812640 1 --2- 172.21.15.26:0/498345777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf6015f370 0x7fcf6006c080 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fcf600cf4a0 tx=0x7fcf5001c500 comp rx=0 tx=0).stop 2023-12-19T18:17:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.197+0000 7fcf67812640 1 -- 172.21.15.26:0/498345777 shutdown_connections 2023-12-19T18:17:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.197+0000 7fcf67812640 1 --2- 172.21.15.26:0/498345777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf6015f370 0x7fcf6006c080 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.197+0000 7fcf67812640 1 --2- 172.21.15.26:0/498345777 >> v2:172.21.15.160:3300/0 conn(0x7fcf60155030 0x7fcf6015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.197+0000 7fcf67812640 1 -- 172.21.15.26:0/498345777 >> 172.21.15.26:0/498345777 conn(0x7fcf600658e0 msgr2=0x7fcf60065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:33.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.197+0000 7fcf67812640 1 -- 172.21.15.26:0/498345777 shutdown_connections 2023-12-19T18:17:33.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.197+0000 7fcf67812640 1 -- 172.21.15.26:0/498345777 wait complete. 2023-12-19T18:17:33.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf67812640 1 Processor -- start 2023-12-19T18:17:33.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf67812640 1 -- start start 2023-12-19T18:17:33.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf67812640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf600ce740 0x7fcf60138100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf67812640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf60155030 0x7fcf601537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf67812640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf6015f370 0x7fcf6014f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf67812640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf601519e0 con 0x7fcf60155030 2023-12-19T18:17:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf67812640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf60151cc0 con 0x7fcf600ce740 2023-12-19T18:17:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf67812640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf60151fa0 con 0x7fcf6015f370 2023-12-19T18:17:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf64dad640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf60155030 0x7fcf601537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf64dad640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf60155030 0x7fcf601537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47658/0 (socket says 172.21.15.26:47658) 2023-12-19T18:17:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf64dad640 1 -- 172.21.15.26:0/226305967 learned_addr learned my addr 172.21.15.26:0/226305967 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf64dad640 1 -- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf6015f370 msgr2=0x7fcf6014f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf64dad640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf6015f370 0x7fcf6014f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf64dad640 1 -- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf600ce740 msgr2=0x7fcf60138100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf655ae640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf600ce740 0x7fcf60138100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf64dad640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf600ce740 0x7fcf60138100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf64dad640 1 -- 172.21.15.26:0/226305967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf500094d0 con 0x7fcf60155030 2023-12-19T18:17:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf655ae640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf600ce740 0x7fcf60138100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf64dad640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf60155030 0x7fcf601537c0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7fcf5001a130 tx=0x7fcf500192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf55ffb640 1 -- 172.21.15.26:0/226305967 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf500abdc0 con 0x7fcf60155030 2023-12-19T18:17:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf55ffb640 1 -- 172.21.15.26:0/226305967 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf500b9070 con 0x7fcf60155030 2023-12-19T18:17:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf55ffb640 1 -- 172.21.15.26:0/226305967 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50004250 con 0x7fcf60155030 2023-12-19T18:17:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.201+0000 7fcf67812640 1 -- 172.21.15.26:0/226305967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf600cf0e0 con 0x7fcf60155030 2023-12-19T18:17:33.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.205+0000 7fcf67812640 1 -- 172.21.15.26:0/226305967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf60152560 con 0x7fcf60155030 2023-12-19T18:17:33.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.205+0000 7fcf55ffb640 1 -- 172.21.15.26:0/226305967 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fcf500043b0 con 0x7fcf60155030 2023-12-19T18:17:33.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.205+0000 7fcf55ffb640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcf44065ca0 0x7fcf44068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.205+0000 7fcf55ffb640 1 -- 172.21.15.26:0/226305967 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fcf500bb070 con 0x7fcf60155030 2023-12-19T18:17:33.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.205+0000 7fcf655ae640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcf44065ca0 0x7fcf44068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.210 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.205+0000 7fcf6ab89000 1 Processor -- start 2023-12-19T18:17:33.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.205+0000 7fcf6ab89000 1 -- start start 2023-12-19T18:17:33.211 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf655ae640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcf44065ca0 0x7fcf44068160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009980 tx=0x7fcf5c008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bad21070 0x55f6bad21450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21990 0x55f6bacd6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 0x55f6bacd75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6baafac90 con 0x55f6bacd71c0 2023-12-19T18:17:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bab2c220 con 0x55f6bad21990 2023-12-19T18:17:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6bab297f0 con 0x55f6bad21070 2023-12-19T18:17:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf65daf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 0x55f6bacd75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf65daf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 0x55f6bacd75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47670/0 (socket says 172.21.15.26:47670) 2023-12-19T18:17:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf65daf640 1 -- 172.21.15.26:0/428798173 learned_addr learned my addr 172.21.15.26:0/428798173 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:33.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf64dad640 1 --2- 172.21.15.26:0/428798173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21990 0x55f6bacd6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf655ae640 1 --2- 172.21.15.26:0/428798173 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bad21070 0x55f6bad21450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf65daf640 1 -- 172.21.15.26:0/428798173 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bad21070 msgr2=0x55f6bad21450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf65daf640 1 --2- 172.21.15.26:0/428798173 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bad21070 0x55f6bad21450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf65daf640 1 -- 172.21.15.26:0/428798173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21990 msgr2=0x55f6bacd6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf65daf640 1 --2- 172.21.15.26:0/428798173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21990 0x55f6bacd6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf65daf640 1 -- 172.21.15.26:0/428798173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6ba821890 con 0x55f6bacd71c0 2023-12-19T18:17:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf64dad640 1 --2- 172.21.15.26:0/428798173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21990 0x55f6bacd6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf65daf640 1 --2- 172.21.15.26:0/428798173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 0x55f6bacd75a0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fcf58001d90 tx=0x7fcf5800c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9b04a7023285f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf42ffd640 1 -- 172.21.15.26:0/428798173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800c910 con 0x55f6bacd71c0 2023-12-19T18:17:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf42ffd640 1 -- 172.21.15.26:0/428798173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5800ca70 con 0x55f6bacd71c0 2023-12-19T18:17:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf42ffd640 1 -- 172.21.15.26:0/428798173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800ce00 con 0x55f6bacd71c0 2023-12-19T18:17:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 -- 172.21.15.26:0/428798173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 msgr2=0x55f6bacd75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/428798173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 0x55f6bacd75a0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7fcf58001d90 tx=0x7fcf5800c0e0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 -- 172.21.15.26:0/428798173 shutdown_connections 2023-12-19T18:17:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/428798173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 0x55f6bacd75a0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/428798173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21990 0x55f6bacd6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/428798173 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bad21070 0x55f6bad21450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 -- 172.21.15.26:0/428798173 >> 172.21.15.26:0/428798173 conn(0x55f6bacdd950 msgr2=0x55f6bacd5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 -- 172.21.15.26:0/428798173 shutdown_connections 2023-12-19T18:17:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 -- 172.21.15.26:0/428798173 wait complete. 2023-12-19T18:17:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 -- start start 2023-12-19T18:17:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.209+0000 7fcf6ab89000 1 Processor -- start 2023-12-19T18:17:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bacd71c0 0x55f6bacdc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 0x55f6bace7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bad21990 0x55f6bace79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bab250e0 con 0x55f6bad21990 2023-12-19T18:17:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6baac4310 con 0x55f6bad21070 2023-12-19T18:17:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6baae2770 con 0x55f6bacd71c0 2023-12-19T18:17:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 10 client.0 _ll_register_callbacks cb 0x55f6bacc2b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:17:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf65daf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bad21990 0x55f6bace79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf65daf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bad21990 0x55f6bace79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47672/0 (socket says 172.21.15.26:47672) 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf65daf640 1 -- 172.21.15.26:0/4055050399 learned_addr learned my addr 172.21.15.26:0/4055050399 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf655ae640 1 --2- 172.21.15.26:0/4055050399 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bacd71c0 0x55f6bacdc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf64dad640 1 --2- 172.21.15.26:0/4055050399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 0x55f6bace7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf65daf640 1 -- 172.21.15.26:0/4055050399 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bacd71c0 msgr2=0x55f6bacdc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf65daf640 1 --2- 172.21.15.26:0/4055050399 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bacd71c0 0x55f6bacdc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf65daf640 1 -- 172.21.15.26:0/4055050399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 msgr2=0x55f6bace7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf65daf640 1 --2- 172.21.15.26:0/4055050399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 0x55f6bace7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf65daf640 1 -- 172.21.15.26:0/4055050399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf580019c0 con 0x55f6bad21990 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf65daf640 1 --2- 172.21.15.26:0/4055050399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bad21990 0x55f6bace79d0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800c120 tx=0x7fcf58001410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10cc307957fe90e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf3bfff640 1 -- 172.21.15.26:0/4055050399 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58014630 con 0x55f6bad21990 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf3bfff640 1 -- 172.21.15.26:0/4055050399 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fcf58020dd0 con 0x55f6bad21990 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 1 -- 172.21.15.26:0/4055050399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6baaf4c10 con 0x55f6bad21990 2023-12-19T18:17:33.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf3bfff640 1 -- 172.21.15.26:0/4055050399 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf58014790 con 0x55f6bad21990 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 10 client.11556 fetch_fsmap learned FSMap version 112 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 1 -- 172.21.15.26:0/4055050399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55f6baaea260 con 0x55f6bad21990 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf3bfff640 1 -- 172.21.15.26:0/4055050399 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f6baaf4c10 con 0x55f6bad21990 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf3bfff640 1 -- 172.21.15.26:0/4055050399 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 112) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcf580148f0 con 0x55f6bad21990 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 10 client.11556 fetch_fsmap finished waiting for FSMap version 112 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 10 client.11556 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 20 client.11556 populate_metadata read hostname 'smithi026' 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/4055050399 >> [v2:172.21.15.26:6832/2285370499,v1:172.21.15.26:6835/2285370499] conn(0x55f6bad2cb20 0x55f6bad2cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 4 client.11556 mds_command: new command op to 11427 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf6ab89000 1 -- 172.21.15.26:0/4055050399 --> [v2:172.21.15.26:6832/2285370499,v1:172.21.15.26:6835/2285370499] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6bab23a30 con 0x55f6bad2cb20 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf655ae640 1 --2- 172.21.15.26:0/4055050399 >> [v2:172.21.15.26:6832/2285370499,v1:172.21.15.26:6835/2285370499] conn(0x55f6bad2cb20 0x55f6bad2cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf655ae640 1 --2- 172.21.15.26:0/4055050399 >> [v2:172.21.15.26:6832/2285370499,v1:172.21.15.26:6835/2285370499] conn(0x55f6bad2cb20 0x55f6bad2cf00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b60fd28eb97835d server_cookie=b2a397e0da04ce78 in_seq=0 out_seq=0 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.213+0000 7fcf3bfff640 10 client.11556 ms_handle_connect on v2:172.21.15.26:6832/2285370499 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf3bfff640 1 -- 172.21.15.26:0/4055050399 <== mds.0 v2:172.21.15.26:6832/2285370499 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f6bab23a30 con 0x55f6bad2cb20 2023-12-19T18:17:33.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf3bfff640 10 client.11556 handle_command_reply: tid=0 2023-12-19T18:17:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf6ab89000 1 client.11556 shutdown 2023-12-19T18:17:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf6ab89000 1 -- 172.21.15.26:0/4055050399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bad21990 msgr2=0x55f6bace79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/4055050399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bad21990 0x55f6bace79d0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fcf5800c120 tx=0x7fcf58001410 comp rx=0 tx=0).stop 2023-12-19T18:17:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf6ab89000 1 -- 172.21.15.26:0/4055050399 shutdown_connections 2023-12-19T18:17:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/4055050399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bad21990 0x55f6bace79d0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/4055050399 >> [v2:172.21.15.26:6832/2285370499,v1:172.21.15.26:6835/2285370499] conn(0x55f6bad2cb20 0x55f6bad2cf00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/4055050399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 0x55f6bace7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/4055050399 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bacd71c0 0x55f6bacdc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf3bfff640 0 client.11556 ms_handle_reset on v2:172.21.15.26:6832/2285370499 2023-12-19T18:17:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf6ab89000 1 -- 172.21.15.26:0/4055050399 >> 172.21.15.26:0/4055050399 conn(0x55f6bacdd950 msgr2=0x55f6bacd5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf6ab89000 1 -- 172.21.15.26:0/4055050399 shutdown_connections 2023-12-19T18:17:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf6ab89000 1 -- 172.21.15.26:0/4055050399 wait complete. 2023-12-19T18:17:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.217+0000 7fcf6ab89000 20 client.11556 trim_cache size 0 max 16384 2023-12-19T18:17:33.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.229+0000 7fcf6ab89000 1 Processor -- start 2023-12-19T18:17:33.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 -- start start 2023-12-19T18:17:33.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 0x55f6bad7d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 0x55f6bad7dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bad21990 0x55f6bad7e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bad2c510 con 0x55f6bacd71c0 2023-12-19T18:17:33.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6ba7e2940 con 0x55f6bad21070 2023-12-19T18:17:33.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6bab68ce0 con 0x55f6bad21990 2023-12-19T18:17:33.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf655ae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 0x55f6bad7d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf655ae640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 0x55f6bad7d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47676/0 (socket says 172.21.15.26:47676) 2023-12-19T18:17:33.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf655ae640 1 -- 172.21.15.26:0/2070723540 learned_addr learned my addr 172.21.15.26:0/2070723540 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:33.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf64dad640 1 --2- 172.21.15.26:0/2070723540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 0x55f6bad7dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf65daf640 1 --2- 172.21.15.26:0/2070723540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bad21990 0x55f6bad7e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf655ae640 1 -- 172.21.15.26:0/2070723540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bad21990 msgr2=0x55f6bad7e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:33.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf655ae640 1 --2- 172.21.15.26:0/2070723540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bad21990 0x55f6bad7e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf655ae640 1 -- 172.21.15.26:0/2070723540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 msgr2=0x55f6bad7dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:33.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf655ae640 1 --2- 172.21.15.26:0/2070723540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 0x55f6bad7dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf655ae640 1 -- 172.21.15.26:0/2070723540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6baad03b0 con 0x55f6bacd71c0 2023-12-19T18:17:33.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf64dad640 1 --2- 172.21.15.26:0/2070723540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 0x55f6bad7dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:33.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf655ae640 1 --2- 172.21.15.26:0/2070723540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 0x55f6bad7d750 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00fb80 tx=0x7fcf5c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ed5de5b7ad53d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:33.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf437fe640 1 -- 172.21.15.26:0/2070723540 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c028070 con 0x55f6bacd71c0 2023-12-19T18:17:33.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf437fe640 1 -- 172.21.15.26:0/2070723540 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5c029250 con 0x55f6bacd71c0 2023-12-19T18:17:33.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf437fe640 1 -- 172.21.15.26:0/2070723540 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c0293c0 con 0x55f6bacd71c0 2023-12-19T18:17:33.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 -- 172.21.15.26:0/2070723540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 msgr2=0x55f6bad7d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/2070723540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 0x55f6bad7d750 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00fb80 tx=0x7fcf5c015000 comp rx=0 tx=0).stop 2023-12-19T18:17:33.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 -- 172.21.15.26:0/2070723540 shutdown_connections 2023-12-19T18:17:33.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/2070723540 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bad21990 0x55f6bad7e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/2070723540 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 0x55f6bad7dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/2070723540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bacd71c0 0x55f6bad7d750 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 -- 172.21.15.26:0/2070723540 >> 172.21.15.26:0/2070723540 conn(0x55f6bacdd950 msgr2=0x55f6bace28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:33.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 -- 172.21.15.26:0/2070723540 shutdown_connections 2023-12-19T18:17:33.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 -- 172.21.15.26:0/2070723540 wait complete. 2023-12-19T18:17:33.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 -- start start 2023-12-19T18:17:33.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.233+0000 7fcf6ab89000 1 Processor -- start 2023-12-19T18:17:33.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bacd71c0 0x55f6bad82da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 0x55f6bad832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bad21990 0x55f6bad380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bab1a1f0 con 0x55f6bad21990 2023-12-19T18:17:33.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bacd9d70 con 0x55f6bad21070 2023-12-19T18:17:33.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6baaca770 con 0x55f6bacd71c0 2023-12-19T18:17:33.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 10 client.0 _ll_register_callbacks cb 0x55f6baad3640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:17:33.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf65daf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bad21990 0x55f6bad380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf655ae640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bacd71c0 0x55f6bad82da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf64dad640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 0x55f6bad832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf655ae640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bacd71c0 0x55f6bad82da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:52306/0 (socket says 172.21.15.26:52306) 2023-12-19T18:17:33.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf655ae640 1 -- 172.21.15.26:0/755321127 learned_addr learned my addr 172.21.15.26:0/755321127 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:33.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf65daf640 1 -- 172.21.15.26:0/755321127 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bacd71c0 msgr2=0x55f6bad82da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf65daf640 1 --2- 172.21.15.26:0/755321127 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bacd71c0 0x55f6bad82da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf65daf640 1 -- 172.21.15.26:0/755321127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 msgr2=0x55f6bad832e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf65daf640 1 --2- 172.21.15.26:0/755321127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 0x55f6bad832e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf65daf640 1 -- 172.21.15.26:0/755321127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c00e040 con 0x55f6bad21990 2023-12-19T18:17:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf65daf640 1 --2- 172.21.15.26:0/755321127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bad21990 0x55f6bad380a0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fcf580012f0 tx=0x7fcf580016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79896800721ba1a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T18:17:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf3bfff640 1 -- 172.21.15.26:0/755321127 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800c690 con 0x55f6bad21990 2023-12-19T18:17:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf3bfff640 1 -- 172.21.15.26:0/755321127 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fcf580125e0 con 0x55f6bad21990 2023-12-19T18:17:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 1 -- 172.21.15.26:0/755321127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6baaeae00 con 0x55f6bad21990 2023-12-19T18:17:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf3bfff640 1 -- 172.21.15.26:0/755321127 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf5800cc20 con 0x55f6bad21990 2023-12-19T18:17:33.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 10 client.11562 fetch_fsmap learned FSMap version 112 2023-12-19T18:17:33.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 1 -- 172.21.15.26:0/755321127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55f6ba821cc0 con 0x55f6bad21990 2023-12-19T18:17:33.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf3bfff640 1 -- 172.21.15.26:0/755321127 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f6baaeae00 con 0x55f6bad21990 2023-12-19T18:17:33.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf3bfff640 1 -- 172.21.15.26:0/755321127 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 112) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcf58014630 con 0x55f6bad21990 2023-12-19T18:17:33.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 10 client.11562 fetch_fsmap finished waiting for FSMap version 112 2023-12-19T18:17:33.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 10 client.11562 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:17:33.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 20 client.11562 populate_metadata read hostname 'smithi026' 2023-12-19T18:17:33.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/755321127 >> [v2:172.21.15.26:6832/2285370499,v1:172.21.15.26:6835/2285370499] conn(0x55f6bad385e0 0x55f6bad389c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 4 client.11562 mds_command: new command op to 11427 tid=0[{"prefix": "status"}] 2023-12-19T18:17:33.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf6ab89000 1 -- 172.21.15.26:0/755321127 --> [v2:172.21.15.26:6832/2285370499,v1:172.21.15.26:6835/2285370499] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f6bab1e160 con 0x55f6bad385e0 2023-12-19T18:17:33.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf655ae640 1 --2- 172.21.15.26:0/755321127 >> [v2:172.21.15.26:6832/2285370499,v1:172.21.15.26:6835/2285370499] conn(0x55f6bad385e0 0x55f6bad389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf655ae640 1 --2- 172.21.15.26:0/755321127 >> [v2:172.21.15.26:6832/2285370499,v1:172.21.15.26:6835/2285370499] conn(0x55f6bad385e0 0x55f6bad389c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89cc23f1cb695132 server_cookie=9267a622dbfc50d6 in_seq=0 out_seq=0 2023-12-19T18:17:33.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.237+0000 7fcf3bfff640 10 client.11562 ms_handle_connect on v2:172.21.15.26:6832/2285370499 2023-12-19T18:17:33.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf3bfff640 1 -- 172.21.15.26:0/755321127 <== mds.0 v2:172.21.15.26:6832/2285370499 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x55f6bab1e160 con 0x55f6bad385e0 2023-12-19T18:17:33.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf3bfff640 10 client.11562 handle_command_reply: tid=0 2023-12-19T18:17:33.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf6ab89000 1 client.11562 shutdown 2023-12-19T18:17:33.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf6ab89000 1 -- 172.21.15.26:0/755321127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bad21990 msgr2=0x55f6bad380a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/755321127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bad21990 0x55f6bad380a0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fcf580012f0 tx=0x7fcf580016f0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf6ab89000 1 -- 172.21.15.26:0/755321127 shutdown_connections 2023-12-19T18:17:33.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/755321127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f6bad21990 0x55f6bad380a0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/755321127 >> [v2:172.21.15.26:6832/2285370499,v1:172.21.15.26:6835/2285370499] conn(0x55f6bad385e0 0x55f6bad389c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/755321127 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f6bad21070 0x55f6bad832e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf6ab89000 1 --2- 172.21.15.26:0/755321127 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x55f6bacd71c0 0x55f6bad82da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf3bfff640 0 client.11562 ms_handle_reset on v2:172.21.15.26:6832/2285370499 2023-12-19T18:17:33.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf6ab89000 1 -- 172.21.15.26:0/755321127 >> 172.21.15.26:0/755321127 conn(0x55f6bacdd950 msgr2=0x55f6bad1e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:33.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf6ab89000 1 -- 172.21.15.26:0/755321127 shutdown_connections 2023-12-19T18:17:33.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf6ab89000 1 -- 172.21.15.26:0/755321127 wait complete. 2023-12-19T18:17:33.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf6ab89000 20 client.11562 trim_cache size 0 max 16384 2023-12-19T18:17:33.253 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:17:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T18:17:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T18:17:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 11427, 2023-12-19T18:17:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T18:17:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T18:17:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T18:17:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.9164207510000004, 2023-12-19T18:17:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 112, 2023-12-19T18:17:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 233, 2023-12-19T18:17:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 233, 2023-12-19T18:17:33.254 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 10.250791778 2023-12-19T18:17:33.255 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:17:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf67812640 1 -- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcf44065ca0 msgr2=0x7fcf44068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf67812640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcf44065ca0 0x7fcf44068160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009980 tx=0x7fcf5c008040 comp rx=0 tx=0).stop 2023-12-19T18:17:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf67812640 1 -- 172.21.15.26:0/226305967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf60155030 msgr2=0x7fcf601537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf67812640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf60155030 0x7fcf601537c0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7fcf5001a130 tx=0x7fcf500192a0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf67812640 1 -- 172.21.15.26:0/226305967 shutdown_connections 2023-12-19T18:17:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf67812640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcf44065ca0 0x7fcf44068160 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf67812640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcf6015f370 0x7fcf6014f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf67812640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf60155030 0x7fcf601537c0 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf67812640 1 --2- 172.21.15.26:0/226305967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf600ce740 0x7fcf60138100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf67812640 1 -- 172.21.15.26:0/226305967 >> 172.21.15.26:0/226305967 conn(0x7fcf600658e0 msgr2=0x7fcf60150560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:33.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf67812640 1 -- 172.21.15.26:0/226305967 shutdown_connections 2023-12-19T18:17:33.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.241+0000 7fcf67812640 1 -- 172.21.15.26:0/226305967 wait complete. 2023-12-19T18:17:33.264 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:17:33.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.405+0000 7f2b2b718640 1 Processor -- start 2023-12-19T18:17:33.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.405+0000 7f2b2b718640 1 -- start start 2023-12-19T18:17:33.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.405+0000 7f2b2b718640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b2415f370 0x7f2b2415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.405+0000 7f2b2b718640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b240ce740 0x7f2b240cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.405+0000 7f2b2b718640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2406c970 con 0x7f2b24155030 2023-12-19T18:17:33.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.405+0000 7f2b2b718640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2406c790 con 0x7f2b2415f370 2023-12-19T18:17:33.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.405+0000 7f2b2b718640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b240cf0f0 con 0x7f2b240ce740 2023-12-19T18:17:33.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.405+0000 7f2b28cb3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b2415f370 0x7f2b2415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.405+0000 7f2b28cb3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b2415f370 0x7f2b2415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47696/0 (socket says 172.21.15.26:47696) 2023-12-19T18:17:33.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.405+0000 7f2b294b4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2b24155030 0x7f2b2406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47684/0 (socket says 172.21.15.26:47684) 2023-12-19T18:17:33.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.405+0000 7f2b28cb3640 1 -- 172.21.15.26:0/3262620712 learned_addr learned my addr 172.21.15.26:0/3262620712 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:33.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.409+0000 7f2b29cb5640 1 --2- 172.21.15.26:0/3262620712 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b240ce740 0x7f2b240cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.409+0000 7f2b28cb3640 1 -- 172.21.15.26:0/3262620712 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b240ce740 msgr2=0x7f2b240cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:33.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.409+0000 7f2b28cb3640 1 --2- 172.21.15.26:0/3262620712 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b240ce740 0x7f2b240cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.409+0000 7f2b28cb3640 1 -- 172.21.15.26:0/3262620712 >> v1:172.21.15.26:6789/0 conn(0x7f2b24155030 legacy=0x7f2b2406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.409+0000 7f2b28cb3640 1 -- 172.21.15.26:0/3262620712 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2415b1f0 con 0x7f2b2415f370 2023-12-19T18:17:33.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.409+0000 7f2b1bfff640 1 -- 172.21.15.26:0/3262620712 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262508093 0 0) 0x7f2b2406c970 con 0x7f2b24155030 2023-12-19T18:17:33.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.409+0000 7f2b28cb3640 1 --2- 172.21.15.26:0/3262620712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2415f370 0x7f2b2415bcd0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009870 tx=0x7f2b1401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f17bca9426b0cf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:33.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.409+0000 7f2b1bfff640 1 -- 172.21.15.26:0/3262620712 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b140aa670 con 0x7f2b2415f370 2023-12-19T18:17:33.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.409+0000 7f2b1bfff640 1 -- 172.21.15.26:0/3262620712 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b140aa7d0 con 0x7f2b2415f370 2023-12-19T18:17:33.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.409+0000 7f2b1bfff640 1 -- 172.21.15.26:0/3262620712 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b140aab60 con 0x7f2b2415f370 2023-12-19T18:17:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.409+0000 7f2b2b718640 1 -- 172.21.15.26:0/3262620712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2415f370 msgr2=0x7f2b2415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.409+0000 7f2b2b718640 1 --2- 172.21.15.26:0/3262620712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2415f370 0x7f2b2415bcd0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009870 tx=0x7f2b1401c510 comp rx=0 tx=0).stop 2023-12-19T18:17:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 -- 172.21.15.26:0/3262620712 shutdown_connections 2023-12-19T18:17:33.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 --2- 172.21.15.26:0/3262620712 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b240ce740 0x7f2b240cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 --2- 172.21.15.26:0/3262620712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2415f370 0x7f2b2415bcd0 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 -- 172.21.15.26:0/3262620712 >> 172.21.15.26:0/3262620712 conn(0x7f2b240658e0 msgr2=0x7f2b24065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 -- 172.21.15.26:0/3262620712 shutdown_connections 2023-12-19T18:17:33.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 -- 172.21.15.26:0/3262620712 wait complete. 2023-12-19T18:17:33.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 Processor -- start 2023-12-19T18:17:33.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 -- start start 2023-12-19T18:17:33.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b240ce740 0x7f2b2413bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b24155030 0x7f2b240b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2415f370 0x7f2b24153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.418 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2413c2c0 con 0x7f2b2415f370 2023-12-19T18:17:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2413ea20 con 0x7f2b240ce740 2023-12-19T18:17:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b2b718640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2413ed00 con 0x7f2b24155030 2023-12-19T18:17:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b29cb5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2415f370 0x7f2b24153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b29cb5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2415f370 0x7f2b24153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47712/0 (socket says 172.21.15.26:47712) 2023-12-19T18:17:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b29cb5640 1 -- 172.21.15.26:0/3675335 learned_addr learned my addr 172.21.15.26:0/3675335 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b28cb3640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b24155030 0x7f2b240b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b294b4640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b240ce740 0x7f2b2413bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b29cb5640 1 -- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b24155030 msgr2=0x7f2b240b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b29cb5640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b24155030 0x7f2b240b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b29cb5640 1 -- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b240ce740 msgr2=0x7f2b2413bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b29cb5640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b240ce740 0x7f2b2413bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b29cb5640 1 -- 172.21.15.26:0/3675335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b140094d0 con 0x7f2b2415f370 2023-12-19T18:17:33.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.413+0000 7f2b294b4640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b240ce740 0x7f2b2413bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.417+0000 7f2b29cb5640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2415f370 0x7f2b24153f70 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c007e60 tx=0x7f2b1c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.417+0000 7f2b19ffb640 1 -- 172.21.15.26:0/3675335 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c014420 con 0x7f2b2415f370 2023-12-19T18:17:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.417+0000 7f2b19ffb640 1 -- 172.21.15.26:0/3675335 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c0145a0 con 0x7f2b2415f370 2023-12-19T18:17:33.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.417+0000 7f2b2b718640 1 -- 172.21.15.26:0/3675335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2413efe0 con 0x7f2b2415f370 2023-12-19T18:17:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.417+0000 7f2b19ffb640 1 -- 172.21.15.26:0/3675335 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c014950 con 0x7f2b2415f370 2023-12-19T18:17:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.417+0000 7f2b2b718640 1 -- 172.21.15.26:0/3675335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b240b0a90 con 0x7f2b2415f370 2023-12-19T18:17:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.417+0000 7f2b2b718640 1 -- 172.21.15.26:0/3675335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b24074270 con 0x7f2b2415f370 2023-12-19T18:17:33.421 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.417+0000 7f2b19ffb640 1 -- 172.21.15.26:0/3675335 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f2b1c014ab0 con 0x7f2b2415f370 2023-12-19T18:17:33.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.417+0000 7f2b19ffb640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2b08065cd0 0x7f2b08068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:33.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.417+0000 7f2b19ffb640 1 -- 172.21.15.26:0/3675335 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f2b1c084020 con 0x7f2b2415f370 2023-12-19T18:17:33.422 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.417+0000 7f2b294b4640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2b08065cd0 0x7f2b08068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:33.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.421+0000 7f2b294b4640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2b08065cd0 0x7f2b08068190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b20007fd0 tx=0x7f2b20007c20 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:33.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.421+0000 7f2b19ffb640 1 -- 172.21.15.26:0/3675335 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2b1c051460 con 0x7f2b2415f370 2023-12-19T18:17:33.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.613+0000 7f2b2b718640 1 -- 172.21.15.26:0/3675335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2b240ccc30 con 0x7f2b2415f370 2023-12-19T18:17:33.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.613+0000 7f2b19ffb640 1 -- 172.21.15.26:0/3675335 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2b1c054a80 con 0x7f2b2415f370 2023-12-19T18:17:33.618 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:17:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.613+0000 7f2b2b718640 1 -- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2b08065cd0 msgr2=0x7f2b08068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.613+0000 7f2b2b718640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2b08065cd0 0x7f2b08068190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b20007fd0 tx=0x7f2b20007c20 comp rx=0 tx=0).stop 2023-12-19T18:17:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.613+0000 7f2b2b718640 1 -- 172.21.15.26:0/3675335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2415f370 msgr2=0x7f2b24153f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.613+0000 7f2b2b718640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2415f370 0x7f2b24153f70 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c007e60 tx=0x7f2b1c00e4b0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.613+0000 7f2b2b718640 1 -- 172.21.15.26:0/3675335 shutdown_connections 2023-12-19T18:17:33.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.617+0000 7f2b2b718640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2b08065cd0 0x7f2b08068190 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.617+0000 7f2b2b718640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b2415f370 0x7f2b24153f70 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.617+0000 7f2b2b718640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2b24155030 0x7f2b240b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.617+0000 7f2b2b718640 1 --2- 172.21.15.26:0/3675335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b240ce740 0x7f2b2413bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.617+0000 7f2b2b718640 1 -- 172.21.15.26:0/3675335 >> 172.21.15.26:0/3675335 conn(0x7f2b240658e0 msgr2=0x7f2b2415bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.617+0000 7f2b2b718640 1 -- 172.21.15.26:0/3675335 shutdown_connections 2023-12-19T18:17:33.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:33.617+0000 7f2b2b718640 1 -- 172.21.15.26:0/3675335 wait complete. 2023-12-19T18:17:33.639 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-19T18:17:33.639 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:17:33.639 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T18:17:33.639 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:17:33.639 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:17:33.639 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-19T18:17:33.639 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-19T18:17:33.640 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:17:33.640 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:17:33.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:33.640 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T18:17:33.651 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:17:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:17:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:17:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T18:17:33.651 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:33.651 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 36687sec preferred_lft 36687sec 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T18:17:33.652 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:17:33.653 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-19T18:17:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:33.654 INFO:teuthology.orchestra.run.smithi026.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:17:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:17:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-19T18:17:33.654 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:33.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:33.654 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:17:33.720 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T18:17:33.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:33.722 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T18:17:33.776 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T18:17:33.776 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:17:33.777 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T18:17:33.777 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T18:17:33.777 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T18:17:33.777 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:17:33.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:33.777 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T18:17:33.777 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T18:17:33.777 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:17:33.777 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:17:33.777 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:17:33.778 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T18:17:33.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:33.871 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T18:17:33.881 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:17:33.881 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:17:33.881 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:17:33.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:33.882 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T18:17:33.948 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:17:33.948 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:17:34.003 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:17:34.004 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:17:34.004 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:17:34.055 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T18:17:34.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:34.055 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T18:17:34.151 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T18:17:34.151 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T18:17:34.163 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T18:17:34.163 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T18:17:34.163 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T18:17:34.163 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T18:17:34.163 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T18:17:34.166 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T18:17:34.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:34.167 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:17:34.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:34.189 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T18:17:34.248 INFO:teuthology.orchestra.run.smithi026.stdout:client11574 2023-12-19T18:17:34.248 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:17:34.248 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:34.300 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-12-19T18:17:34.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.485+0000 7f2681e95640 1 Processor -- start 2023-12-19T18:17:34.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.485+0000 7f2681e95640 1 -- start start 2023-12-19T18:17:34.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.485+0000 7f2681e95640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f267c159830 0x7f267c15d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:34.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.485+0000 7f2681e95640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f267c0cb110 0x7f267c0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:34.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.485+0000 7f2681e95640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f267c15e3d0 con 0x7f267c0c9310 2023-12-19T18:17:34.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.485+0000 7f2681e95640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f267c06f7c0 con 0x7f267c0cb110 2023-12-19T18:17:34.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.485+0000 7f2681e95640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267c0ca2b0 con 0x7f267c159830 2023-12-19T18:17:34.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f267c0c9310 0x7f267c0cc430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47692/0 (socket says 172.21.15.26:47692) 2023-12-19T18:17:34.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267affd640 1 -- 172.21.15.26:0/1680640159 learned_addr learned my addr 172.21.15.26:0/1680640159 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:34.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867971430 0 0) 0x7f267c15e3d0 con 0x7f267c0c9310 2023-12-19T18:17:34.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267bfff640 1 --2- 172.21.15.26:0/1680640159 >> v2:172.21.15.160:3300/0 conn(0x7f267c0cb110 0x7f267c0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2664005740 con 0x7f267c0c9310 2023-12-19T18:17:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267b7fe640 1 --2- 172.21.15.26:0/1680640159 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f267c159830 0x7f267c15d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 640790903 0 0) 0x7f2664005740 con 0x7f267c0c9310 2023-12-19T18:17:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f267c159830 msgr2=0x7f267c15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 --2- 172.21.15.26:0/1680640159 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f267c159830 0x7f267c15d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 >> v2:172.21.15.160:3300/0 conn(0x7f267c0cb110 msgr2=0x7f267c0cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 --2- 172.21.15.26:0/1680640159 >> v2:172.21.15.160:3300/0 conn(0x7f267c0cb110 0x7f267c0cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c0cba80 con 0x7f267c0c9310 2023-12-19T18:17:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f2670001680 con 0x7f267c0c9310 2023-12-19T18:17:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 >> v1:172.21.15.26:6789/0 conn(0x7f267c0c9310 legacy=0x7f267c0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 --2- 172.21.15.26:0/1680640159 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f267c159830 0x7f2664005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 --2- 172.21.15.26:0/1680640159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2664006110 0x7f26640069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 --2- 172.21.15.26:0/1680640159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664006f20 0x7f26640073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:34.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c06f7c0 con 0x7f2664006110 2023-12-19T18:17:34.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c0ca2b0 con 0x7f2664006f20 2023-12-19T18:17:34.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266400dbf0 con 0x7f267c159830 2023-12-19T18:17:34.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267affd640 1 --2- 172.21.15.26:0/1680640159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2664006110 0x7f26640069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:34.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267bfff640 1 --2- 172.21.15.26:0/1680640159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664006f20 0x7f26640073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:34.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267b7fe640 1 --2- 172.21.15.26:0/1680640159 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f267c159830 0x7f2664005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:34.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267affd640 1 -- 172.21.15.26:0/1680640159 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f267c159830 msgr2=0x7f2664005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:34.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267affd640 1 --2- 172.21.15.26:0/1680640159 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f267c159830 0x7f2664005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:34.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267affd640 1 -- 172.21.15.26:0/1680640159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664006f20 msgr2=0x7f26640073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:34.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267affd640 1 --2- 172.21.15.26:0/1680640159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2664006f20 0x7f26640073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:34.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.489+0000 7f267affd640 1 -- 172.21.15.26:0/1680640159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2664005740 con 0x7f2664006110 2023-12-19T18:17:34.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.493+0000 7f267affd640 1 --2- 172.21.15.26:0/1680640159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2664006110 0x7f26640069e0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f267000c290 tx=0x7f267001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf8968cb4e92f52 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.493+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26700058e0 con 0x7f2664006110 2023-12-19T18:17:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.493+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2670005a40 con 0x7f2664006110 2023-12-19T18:17:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.493+0000 7f267a7fc640 1 -- 172.21.15.26:0/1680640159 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670005df0 con 0x7f2664006110 2023-12-19T18:17:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.493+0000 7f2681e95640 1 -- 172.21.15.26:0/1680640159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2664006110 msgr2=0x7f26640069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.493+0000 7f2681e95640 1 --2- 172.21.15.26:0/1680640159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2664006110 0x7f26640069e0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f267000c290 tx=0x7f267001cca0 comp rx=0 tx=0).stop 2023-12-19T18:17:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.493+0000 7f267bfff640 1 -- 172.21.15.26:0/1680640159 reap_dead start 2023-12-19T18:17:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.493+0000 7f2681e95640 1 -- 172.21.15.26:0/1680640159 shutdown_connections 2023-12-19T18:17:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.493+0000 7f2681e95640 1 -- 172.21.15.26:0/1680640159 >> 172.21.15.26:0/1680640159 conn(0x7f267c0b0270 msgr2=0x7f267c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.493+0000 7f2681e95640 1 -- 172.21.15.26:0/1680640159 shutdown_connections 2023-12-19T18:17:34.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.493+0000 7f2681e95640 1 -- 172.21.15.26:0/1680640159 wait complete. 2023-12-19T18:17:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.493+0000 7f2681e95640 1 Processor -- start 2023-12-19T18:17:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.493+0000 7f2681e95640 1 -- start start 2023-12-19T18:17:34.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f2681e95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2664006110 0x7f267c14fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:34.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f2681e95640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f267c15cce0 0x7f267c0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:34.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f2681e95640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f267c159ff0 0x7f267c070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f2681e95640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c14ff60 con 0x7f2664006110 2023-12-19T18:17:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f2681e95640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c15a540 con 0x7f267c15cce0 2023-12-19T18:17:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f2681e95640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267c15a820 con 0x7f267c159ff0 2023-12-19T18:17:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2664006110 0x7f267c14fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f267b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2664006110 0x7f267c14fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47718/0 (socket says 172.21.15.26:47718) 2023-12-19T18:17:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f267b7fe640 1 -- 172.21.15.26:0/2746746801 learned_addr learned my addr 172.21.15.26:0/2746746801 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f267affd640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f267c15cce0 0x7f267c0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f267bfff640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f267c159ff0 0x7f267c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f267b7fe640 1 -- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f267c159ff0 msgr2=0x7f267c070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f267b7fe640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f267c159ff0 0x7f267c070710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f267b7fe640 1 -- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f267c15cce0 msgr2=0x7f267c0701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f267b7fe640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f267c15cce0 0x7f267c0701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f267b7fe640 1 -- 172.21.15.26:0/2746746801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267c15ab00 con 0x7f2664006110 2023-12-19T18:17:34.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f267b7fe640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2664006110 0x7f267c14fa20 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f266c0115e0 tx=0x7f266c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:34.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f2663fff640 1 -- 172.21.15.26:0/2746746801 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c00f8f0 con 0x7f2664006110 2023-12-19T18:17:34.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f2663fff640 1 -- 172.21.15.26:0/2746746801 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266c00fa50 con 0x7f2664006110 2023-12-19T18:17:34.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f2681e95640 1 -- 172.21.15.26:0/2746746801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c15ade0 con 0x7f2664006110 2023-12-19T18:17:34.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f2663fff640 1 -- 172.21.15.26:0/2746746801 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c00fdc0 con 0x7f2664006110 2023-12-19T18:17:34.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f2681e95640 1 -- 172.21.15.26:0/2746746801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c15bd40 con 0x7f2664006110 2023-12-19T18:17:34.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f2681e95640 1 -- 172.21.15.26:0/2746746801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267c15c020 con 0x7f2664006110 2023-12-19T18:17:34.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.497+0000 7f2663fff640 1 -- 172.21.15.26:0/2746746801 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f266c015460 con 0x7f2664006110 2023-12-19T18:17:34.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.501+0000 7f2663fff640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2668065ca0 0x7f2668068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:34.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.501+0000 7f2663fff640 1 -- 172.21.15.26:0/2746746801 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f266c083ca0 con 0x7f2664006110 2023-12-19T18:17:34.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.501+0000 7f267affd640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2668065ca0 0x7f2668068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:34.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.501+0000 7f267affd640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2668065ca0 0x7f2668068160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26700aa000 tx=0x7f2670001a10 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:34.508 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.505+0000 7f2663fff640 1 -- 172.21.15.26:0/2746746801 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f266c050f60 con 0x7f2664006110 2023-12-19T18:17:34.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:34.697+0000 7f2681e95640 1 -- 172.21.15.26:0/2746746801 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f267c074270 con 0x7f2664006110 2023-12-19T18:17:35.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.361+0000 7f2663fff640 1 -- 172.21.15.26:0/2746746801 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v113) v1 ==== 136+0+0 (secure 0 0 0) 0x7f266c054580 con 0x7f2664006110 2023-12-19T18:17:35.369 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.365+0000 7f2681e95640 1 -- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2668065ca0 msgr2=0x7f2668068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:35.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.365+0000 7f2681e95640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2668065ca0 0x7f2668068160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26700aa000 tx=0x7f2670001a10 comp rx=0 tx=0).stop 2023-12-19T18:17:35.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.365+0000 7f2681e95640 1 -- 172.21.15.26:0/2746746801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2664006110 msgr2=0x7f267c14fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:35.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.365+0000 7f2681e95640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2664006110 0x7f267c14fa20 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f266c0115e0 tx=0x7f266c011aa0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.365+0000 7f2681e95640 1 -- 172.21.15.26:0/2746746801 shutdown_connections 2023-12-19T18:17:35.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.365+0000 7f2681e95640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2668065ca0 0x7f2668068160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.365+0000 7f2681e95640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f267c159ff0 0x7f267c070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.365+0000 7f2681e95640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f267c15cce0 0x7f267c0701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.365+0000 7f2681e95640 1 --2- 172.21.15.26:0/2746746801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2664006110 0x7f267c14fa20 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.365+0000 7f2681e95640 1 -- 172.21.15.26:0/2746746801 >> 172.21.15.26:0/2746746801 conn(0x7f267c0b0270 msgr2=0x7f267c0cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:35.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.365+0000 7f2681e95640 1 -- 172.21.15.26:0/2746746801 shutdown_connections 2023-12-19T18:17:35.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.365+0000 7f2681e95640 1 -- 172.21.15.26:0/2746746801 wait complete. 2023-12-19T18:17:35.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:35.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.537+0000 7ff87b96d640 1 Processor -- start 2023-12-19T18:17:35.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.537+0000 7ff87b96d640 1 -- start start 2023-12-19T18:17:35.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.537+0000 7ff87b96d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb1d0 0x7ff8740cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:35.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.537+0000 7ff87b96d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff87406bee0 0x7ff87406c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.537+0000 7ff87b96d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87406d4b0 con 0x7ff874164c90 2023-12-19T18:17:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.537+0000 7ff87b96d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8740cc030 con 0x7ff87406bee0 2023-12-19T18:17:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.537+0000 7ff87b96d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740cbaf0 con 0x7ff8740cb1d0 2023-12-19T18:17:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.537+0000 7ff878f08640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff87406bee0 0x7ff87406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.537+0000 7ff878f08640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff87406bee0 0x7ff87406c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47724/0 (socket says 172.21.15.26:47724) 2023-12-19T18:17:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.537+0000 7ff879f0a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff874164c90 0x7ff874165120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47708/0 (socket says 172.21.15.26:47708) 2023-12-19T18:17:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.537+0000 7ff878f08640 1 -- 172.21.15.26:0/1658005854 learned_addr learned my addr 172.21.15.26:0/1658005854 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:35.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.537+0000 7ff879709640 1 --2- 172.21.15.26:0/1658005854 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb1d0 0x7ff8740cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff878f08640 1 -- 172.21.15.26:0/1658005854 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb1d0 msgr2=0x7ff8740cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff878f08640 1 --2- 172.21.15.26:0/1658005854 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb1d0 0x7ff8740cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff878f08640 1 -- 172.21.15.26:0/1658005854 >> v1:172.21.15.26:6789/0 conn(0x7ff874164c90 legacy=0x7ff874165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff878f08640 1 -- 172.21.15.26:0/1658005854 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87406cc10 con 0x7ff87406bee0 2023-12-19T18:17:35.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff86bfff640 1 -- 172.21.15.26:0/1658005854 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799083274 0 0) 0x7ff87406d4b0 con 0x7ff874164c90 2023-12-19T18:17:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff878f08640 1 --2- 172.21.15.26:0/1658005854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87406bee0 0x7ff87406c340 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7ff864009870 tx=0x7ff86401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374bf1a9d89e9529 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff86bfff640 1 -- 172.21.15.26:0/1658005854 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8640ab680 con 0x7ff87406bee0 2023-12-19T18:17:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff86bfff640 1 -- 172.21.15.26:0/1658005854 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8640ab7e0 con 0x7ff87406bee0 2023-12-19T18:17:35.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff86bfff640 1 -- 172.21.15.26:0/1658005854 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8640abb70 con 0x7ff87406bee0 2023-12-19T18:17:35.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff87b96d640 1 -- 172.21.15.26:0/1658005854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87406bee0 msgr2=0x7ff87406c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:35.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff87b96d640 1 --2- 172.21.15.26:0/1658005854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87406bee0 0x7ff87406c340 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7ff864009870 tx=0x7ff86401c480 comp rx=0 tx=0).stop 2023-12-19T18:17:35.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff87b96d640 1 -- 172.21.15.26:0/1658005854 shutdown_connections 2023-12-19T18:17:35.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff87b96d640 1 --2- 172.21.15.26:0/1658005854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87406bee0 0x7ff87406c340 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff87b96d640 1 --2- 172.21.15.26:0/1658005854 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb1d0 0x7ff8740cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.541+0000 7ff87b96d640 1 -- 172.21.15.26:0/1658005854 >> 172.21.15.26:0/1658005854 conn(0x7ff8740658e0 msgr2=0x7ff874065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff87b96d640 1 -- 172.21.15.26:0/1658005854 shutdown_connections 2023-12-19T18:17:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff87b96d640 1 -- 172.21.15.26:0/1658005854 wait complete. 2023-12-19T18:17:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff87b96d640 1 Processor -- start 2023-12-19T18:17:35.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff87b96d640 1 -- start start 2023-12-19T18:17:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff87b96d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87406bee0 0x7ff8740b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff87b96d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb1d0 0x7ff87414e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff87b96d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff874164c90 0x7ff874151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff87b96d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8740cd580 con 0x7ff87406bee0 2023-12-19T18:17:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff87b96d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff874160310 con 0x7ff874164c90 2023-12-19T18:17:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff87b96d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff874160ae0 con 0x7ff8740cb1d0 2023-12-19T18:17:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff879709640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87406bee0 0x7ff8740b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff879709640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87406bee0 0x7ff8740b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47738/0 (socket says 172.21.15.26:47738) 2023-12-19T18:17:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff879709640 1 -- 172.21.15.26:0/2689038133 learned_addr learned my addr 172.21.15.26:0/2689038133 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff878f08640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb1d0 0x7ff87414e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff879f0a640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff874164c90 0x7ff874151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:35.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff879709640 1 -- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb1d0 msgr2=0x7ff87414e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff879709640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb1d0 0x7ff87414e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff879709640 1 -- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff874164c90 msgr2=0x7ff874151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff879709640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff874164c90 0x7ff874151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff879709640 1 -- 172.21.15.26:0/2689038133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8640094d0 con 0x7ff87406bee0 2023-12-19T18:17:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff879709640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87406bee0 0x7ff8740b26b0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7ff87000e930 tx=0x7ff87000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff869ffb640 1 -- 172.21.15.26:0/2689038133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870009690 con 0x7ff87406bee0 2023-12-19T18:17:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff869ffb640 1 -- 172.21.15.26:0/2689038133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87000ca00 con 0x7ff87406bee0 2023-12-19T18:17:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff869ffb640 1 -- 172.21.15.26:0/2689038133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87000cd70 con 0x7ff87406bee0 2023-12-19T18:17:35.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.545+0000 7ff87b96d640 1 -- 172.21.15.26:0/2689038133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87414e790 con 0x7ff87406bee0 2023-12-19T18:17:35.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.549+0000 7ff87b96d640 1 -- 172.21.15.26:0/2689038133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff874160800 con 0x7ff87406bee0 2023-12-19T18:17:35.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.549+0000 7ff87b96d640 1 -- 172.21.15.26:0/2689038133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff874074270 con 0x7ff87406bee0 2023-12-19T18:17:35.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.549+0000 7ff869ffb640 1 -- 172.21.15.26:0/2689038133 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ff870013470 con 0x7ff87406bee0 2023-12-19T18:17:35.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.549+0000 7ff869ffb640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff858065ca0 0x7ff858068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:35.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.549+0000 7ff869ffb640 1 -- 172.21.15.26:0/2689038133 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7ff870083b10 con 0x7ff87406bee0 2023-12-19T18:17:35.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.549+0000 7ff878f08640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff858065ca0 0x7ff858068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:35.555 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.553+0000 7ff878f08640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff858065ca0 0x7ff858068160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff8640af000 tx=0x7ff8640a7000 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:35.557 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.553+0000 7ff869ffb640 1 -- 172.21.15.26:0/2689038133 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff870050fd0 con 0x7ff87406bee0 2023-12-19T18:17:35.715 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.713+0000 7ff87b96d640 1 -- 172.21.15.26:0/2689038133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8740b2490 con 0x7ff87406bee0 2023-12-19T18:17:35.716 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.713+0000 7ff869ffb640 1 -- 172.21.15.26:0/2689038133 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v233) v1 ==== 74+0+17261 (secure 0 0 0) 0x7ff870017090 con 0x7ff87406bee0 2023-12-19T18:17:35.717 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:35.717 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":233,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:28.328548+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:25.867829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:26.330514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":231,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.713+0000 7ff87b96d640 1 -- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff858065ca0 msgr2=0x7ff858068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.713+0000 7ff87b96d640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff858065ca0 0x7ff858068160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff8640af000 tx=0x7ff8640a7000 comp rx=0 tx=0).stop 2023-12-19T18:17:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.713+0000 7ff87b96d640 1 -- 172.21.15.26:0/2689038133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87406bee0 msgr2=0x7ff8740b26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.713+0000 7ff87b96d640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87406bee0 0x7ff8740b26b0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7ff87000e930 tx=0x7ff87000edf0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.713+0000 7ff87b96d640 1 -- 172.21.15.26:0/2689038133 shutdown_connections 2023-12-19T18:17:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.713+0000 7ff87b96d640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff858065ca0 0x7ff858068160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.713+0000 7ff87b96d640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff874164c90 0x7ff874151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.719 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.713+0000 7ff87b96d640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8740cb1d0 0x7ff87414e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.713+0000 7ff87b96d640 1 --2- 172.21.15.26:0/2689038133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87406bee0 0x7ff8740b26b0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.713+0000 7ff87b96d640 1 -- 172.21.15.26:0/2689038133 >> 172.21.15.26:0/2689038133 conn(0x7ff8740658e0 msgr2=0x7ff8740b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:35.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.717+0000 7ff87b96d640 1 -- 172.21.15.26:0/2689038133 shutdown_connections 2023-12-19T18:17:35.720 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.717+0000 7ff87b96d640 1 -- 172.21.15.26:0/2689038133 wait complete. 2023-12-19T18:17:35.738 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:17:25.867829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '232', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:17:26.330514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '233', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:17:35.740 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:17:35.886 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.881+0000 7f30d8d5e640 1 Processor -- start 2023-12-19T18:17:35.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.881+0000 7f30d8d5e640 1 -- start start 2023-12-19T18:17:35.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d8d5e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30d415a770 0x7f30d40cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:35.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d8d5e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30d40cfae0 0x7f30d4160a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:35.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d8d5e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d40cff00 con 0x7f30d4160f80 2023-12-19T18:17:35.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d8d5e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d40cd490 con 0x7f30d40cfae0 2023-12-19T18:17:35.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d8d5e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40cd770 con 0x7f30d415a770 2023-12-19T18:17:35.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30d40cfae0 0x7f30d4160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:35.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d3fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f30d4160f80 0x7f30d40b4610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47714/0 (socket says 172.21.15.26:47714) 2023-12-19T18:17:35.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d3fff640 1 -- 172.21.15.26:0/653121290 learned_addr learned my addr 172.21.15.26:0/653121290 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:35.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d37fe640 1 --2- 172.21.15.26:0/653121290 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30d415a770 0x7f30d40cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:35.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d2ffd640 1 -- 172.21.15.26:0/653121290 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30d415a770 msgr2=0x7f30d40cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:35.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d2ffd640 1 --2- 172.21.15.26:0/653121290 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30d415a770 0x7f30d40cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d2ffd640 1 -- 172.21.15.26:0/653121290 >> v1:172.21.15.26:6789/0 conn(0x7f30d4160f80 legacy=0x7f30d40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:35.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d2ffd640 1 -- 172.21.15.26:0/653121290 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d40cda50 con 0x7f30d40cfae0 2023-12-19T18:17:35.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.885+0000 7f30d27fc640 1 -- 172.21.15.26:0/653121290 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338562304 0 0) 0x7f30d40cff00 con 0x7f30d4160f80 2023-12-19T18:17:35.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d2ffd640 1 --2- 172.21.15.26:0/653121290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d40cfae0 0x7f30d4160a40 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f30c80097c0 tx=0x7f30c801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb16f21552895ef server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:35.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d27fc640 1 -- 172.21.15.26:0/653121290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c80ac640 con 0x7f30d40cfae0 2023-12-19T18:17:35.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d27fc640 1 -- 172.21.15.26:0/653121290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c80ac7a0 con 0x7f30d40cfae0 2023-12-19T18:17:35.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d27fc640 1 -- 172.21.15.26:0/653121290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c80acb30 con 0x7f30d40cfae0 2023-12-19T18:17:35.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d8d5e640 1 -- 172.21.15.26:0/653121290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d40cfae0 msgr2=0x7f30d4160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:35.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d8d5e640 1 --2- 172.21.15.26:0/653121290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d40cfae0 0x7f30d4160a40 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f30c80097c0 tx=0x7f30c801c450 comp rx=0 tx=0).stop 2023-12-19T18:17:35.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d8d5e640 1 -- 172.21.15.26:0/653121290 shutdown_connections 2023-12-19T18:17:35.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d8d5e640 1 --2- 172.21.15.26:0/653121290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d40cfae0 0x7f30d4160a40 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d8d5e640 1 --2- 172.21.15.26:0/653121290 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30d415a770 0x7f30d40cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d8d5e640 1 -- 172.21.15.26:0/653121290 >> 172.21.15.26:0/653121290 conn(0x7f30d406a150 msgr2=0x7f30d406a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:35.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d8d5e640 1 -- 172.21.15.26:0/653121290 shutdown_connections 2023-12-19T18:17:35.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d8d5e640 1 -- 172.21.15.26:0/653121290 wait complete. 2023-12-19T18:17:35.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d8d5e640 1 Processor -- start 2023-12-19T18:17:35.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d8d5e640 1 -- start start 2023-12-19T18:17:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.889+0000 7f30d8d5e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30d40cfae0 0x7f30d414f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d8d5e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d415a770 0x7f30d4166b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d8d5e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d4160f80 0x7f30d4167090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d8d5e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d41589e0 con 0x7f30d415a770 2023-12-19T18:17:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d8d5e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4157e10 con 0x7f30d4160f80 2023-12-19T18:17:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d8d5e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d414f6b0 con 0x7f30d40cfae0 2023-12-19T18:17:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d415a770 0x7f30d4166b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d415a770 0x7f30d4166b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36512/0 (socket says 172.21.15.26:36512) 2023-12-19T18:17:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d2ffd640 1 -- 172.21.15.26:0/2018670160 learned_addr learned my addr 172.21.15.26:0/2018670160 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d2ffd640 1 -- 172.21.15.26:0/2018670160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30d40cfae0 msgr2=0x7f30d414f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d2ffd640 1 --2- 172.21.15.26:0/2018670160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30d40cfae0 0x7f30d414f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d2ffd640 1 -- 172.21.15.26:0/2018670160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d4160f80 msgr2=0x7f30d4167090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:35.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d2ffd640 1 --2- 172.21.15.26:0/2018670160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d4160f80 0x7f30d4167090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:35.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d2ffd640 1 -- 172.21.15.26:0/2018670160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c8009420 con 0x7f30d415a770 2023-12-19T18:17:35.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d2ffd640 1 --2- 172.21.15.26:0/2018670160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d415a770 0x7f30d4166b50 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f30c801a080 tx=0x7f30c80191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:35.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30bbfff640 1 -- 172.21.15.26:0/2018670160 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c80b5610 con 0x7f30d415a770 2023-12-19T18:17:35.898 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30bbfff640 1 -- 172.21.15.26:0/2018670160 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c80b5770 con 0x7f30d415a770 2023-12-19T18:17:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30bbfff640 1 -- 172.21.15.26:0/2018670160 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c80b5b20 con 0x7f30d415a770 2023-12-19T18:17:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d8d5e640 1 -- 172.21.15.26:0/2018670160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d41580b0 con 0x7f30d415a770 2023-12-19T18:17:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d8d5e640 1 -- 172.21.15.26:0/2018670160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d4158670 con 0x7f30d415a770 2023-12-19T18:17:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30d8d5e640 1 -- 172.21.15.26:0/2018670160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d4061930 con 0x7f30d415a770 2023-12-19T18:17:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.893+0000 7f30bbfff640 1 -- 172.21.15.26:0/2018670160 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f30c80b3040 con 0x7f30d415a770 2023-12-19T18:17:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.897+0000 7f30bbfff640 1 --2- 172.21.15.26:0/2018670160 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f30c0065ca0 0x7f30c0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:35.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.897+0000 7f30bbfff640 1 -- 172.21.15.26:0/2018670160 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f30c811dbf0 con 0x7f30d415a770 2023-12-19T18:17:35.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.897+0000 7f30d37fe640 1 --2- 172.21.15.26:0/2018670160 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f30c0065ca0 0x7f30c0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:35.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.897+0000 7f30d37fe640 1 --2- 172.21.15.26:0/2018670160 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f30c0065ca0 0x7f30c0068160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f30c4006aa0 tx=0x7f30c4006a30 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:35.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:35.901+0000 7f30bbfff640 1 -- 172.21.15.26:0/2018670160 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f30c80eb060 con 0x7f30d415a770 2023-12-19T18:17:36.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.053+0000 7f30d8d5e640 1 -- 172.21.15.26:0/2018670160 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30d4158b20 con 0x7f30d415a770 2023-12-19T18:17:36.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.057+0000 7f30bbfff640 1 -- 172.21.15.26:0/2018670160 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v233) v1 ==== 93+0+10 (secure 0 0 0) 0x7f30c80b1020 con 0x7f30d415a770 2023-12-19T18:17:36.060 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:17:36.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.057+0000 7f30d8d5e640 1 -- 172.21.15.26:0/2018670160 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f30c0065ca0 msgr2=0x7f30c0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.057+0000 7f30d8d5e640 1 --2- 172.21.15.26:0/2018670160 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f30c0065ca0 0x7f30c0068160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f30c4006aa0 tx=0x7f30c4006a30 comp rx=0 tx=0).stop 2023-12-19T18:17:36.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.057+0000 7f30d8d5e640 1 -- 172.21.15.26:0/2018670160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d415a770 msgr2=0x7f30d4166b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.057+0000 7f30d8d5e640 1 --2- 172.21.15.26:0/2018670160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d415a770 0x7f30d4166b50 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f30c801a080 tx=0x7f30c80191f0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.057+0000 7f30d8d5e640 1 -- 172.21.15.26:0/2018670160 shutdown_connections 2023-12-19T18:17:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.057+0000 7f30d8d5e640 1 --2- 172.21.15.26:0/2018670160 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f30c0065ca0 0x7f30c0068160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.057+0000 7f30d8d5e640 1 --2- 172.21.15.26:0/2018670160 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30d4160f80 0x7f30d4167090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.057+0000 7f30d8d5e640 1 --2- 172.21.15.26:0/2018670160 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30d415a770 0x7f30d4166b50 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.057+0000 7f30d8d5e640 1 --2- 172.21.15.26:0/2018670160 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30d40cfae0 0x7f30d414f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.057+0000 7f30d8d5e640 1 -- 172.21.15.26:0/2018670160 >> 172.21.15.26:0/2018670160 conn(0x7f30d406a150 msgr2=0x7f30d4160340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.057+0000 7f30d8d5e640 1 -- 172.21.15.26:0/2018670160 shutdown_connections 2023-12-19T18:17:36.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.057+0000 7f30d8d5e640 1 -- 172.21.15.26:0/2018670160 wait complete. 2023-12-19T18:17:36.081 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:17:36.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc43ac8c640 1 Processor -- start 2023-12-19T18:17:36.229 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc43ac8c640 1 -- start start 2023-12-19T18:17:36.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc43ac8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc434159820 0x7fc43415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc43ac8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc434159c00 0x7fc434070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc43ac8c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43406fb60 con 0x7fc43406f6c0 2023-12-19T18:17:36.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc43ac8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc43415e470 con 0x7fc434159820 2023-12-19T18:17:36.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc43ac8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43406fe40 con 0x7fc434159c00 2023-12-19T18:17:36.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc438a28640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc434159820 0x7fc43415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc438a28640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc434159820 0x7fc43415df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36514/0 (socket says 172.21.15.26:36514) 2023-12-19T18:17:36.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc438a28640 1 -- 172.21.15.26:0/54689457 learned_addr learned my addr 172.21.15.26:0/54689457 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:36.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc439229640 1 --2- 172.21.15.26:0/54689457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc434159c00 0x7fc434070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc438a28640 1 -- 172.21.15.26:0/54689457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc434159c00 msgr2=0x7fc434070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:36.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc438a28640 1 --2- 172.21.15.26:0/54689457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc434159c00 0x7fc434070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc438a28640 1 -- 172.21.15.26:0/54689457 >> v1:172.21.15.160:6789/0 conn(0x7fc43406f6c0 legacy=0x7fc4340c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:36.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.225+0000 7fc438a28640 1 -- 172.21.15.26:0/54689457 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc434070120 con 0x7fc434159820 2023-12-19T18:17:36.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.229+0000 7fc438a28640 1 --2- 172.21.15.26:0/54689457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434159820 0x7fc43415df30 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fc428009870 tx=0x7fc42801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2577c29d1d72cdc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:36.233 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.229+0000 7fc4337fe640 1 -- 172.21.15.26:0/54689457 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280ac600 con 0x7fc434159820 2023-12-19T18:17:36.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.229+0000 7fc4337fe640 1 -- 172.21.15.26:0/54689457 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4280a8050 con 0x7fc434159820 2023-12-19T18:17:36.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.229+0000 7fc4337fe640 1 -- 172.21.15.26:0/54689457 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4280ac980 con 0x7fc434159820 2023-12-19T18:17:36.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.229+0000 7fc43ac8c640 1 -- 172.21.15.26:0/54689457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434159820 msgr2=0x7fc43415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.229+0000 7fc43ac8c640 1 --2- 172.21.15.26:0/54689457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434159820 0x7fc43415df30 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fc428009870 tx=0x7fc42801c4d0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.229+0000 7fc43ac8c640 1 -- 172.21.15.26:0/54689457 shutdown_connections 2023-12-19T18:17:36.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.229+0000 7fc43ac8c640 1 --2- 172.21.15.26:0/54689457 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc434159c00 0x7fc434070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.229+0000 7fc43ac8c640 1 --2- 172.21.15.26:0/54689457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434159820 0x7fc43415df30 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.229+0000 7fc43ac8c640 1 -- 172.21.15.26:0/54689457 >> 172.21.15.26:0/54689457 conn(0x7fc4340ce730 msgr2=0x7fc4340ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:36.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.229+0000 7fc43ac8c640 1 -- 172.21.15.26:0/54689457 shutdown_connections 2023-12-19T18:17:36.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.229+0000 7fc43ac8c640 1 -- 172.21.15.26:0/54689457 wait complete. 2023-12-19T18:17:36.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc43ac8c640 1 Processor -- start 2023-12-19T18:17:36.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc43ac8c640 1 -- start start 2023-12-19T18:17:36.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc43ac8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc43406f6c0 0x7fc4340c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc43ac8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc434159820 0x7fc4340b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc43ac8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434159c00 0x7fc43414ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc43ac8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43414f4b0 con 0x7fc434159c00 2023-12-19T18:17:36.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc43ac8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43415a670 con 0x7fc434159820 2023-12-19T18:17:36.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc43ac8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4340b37b0 con 0x7fc43406f6c0 2023-12-19T18:17:36.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc439229640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434159c00 0x7fc43414ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc439229640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434159c00 0x7fc43414ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36528/0 (socket says 172.21.15.26:36528) 2023-12-19T18:17:36.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc439229640 1 -- 172.21.15.26:0/2137793210 learned_addr learned my addr 172.21.15.26:0/2137793210 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:36.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc433fff640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc434159820 0x7fc4340b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc438a28640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc43406f6c0 0x7fc4340c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.239 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc439229640 1 -- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc43406f6c0 msgr2=0x7fc4340c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc439229640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc43406f6c0 0x7fc4340c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc439229640 1 -- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc434159820 msgr2=0x7fc4340b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc439229640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc434159820 0x7fc4340b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc439229640 1 -- 172.21.15.26:0/2137793210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4280094d0 con 0x7fc434159c00 2023-12-19T18:17:36.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc433fff640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc434159820 0x7fc4340b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:36.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc438a28640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc43406f6c0 0x7fc4340c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:36.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc439229640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434159c00 0x7fc43414ef70 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fc42c007e90 tx=0x7fc42c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:36.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc4317fa640 1 -- 172.21.15.26:0/2137793210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c014420 con 0x7fc434159c00 2023-12-19T18:17:36.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc4317fa640 1 -- 172.21.15.26:0/2137793210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42c014580 con 0x7fc434159c00 2023-12-19T18:17:36.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc43ac8c640 1 -- 172.21.15.26:0/2137793210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43406b880 con 0x7fc434159c00 2023-12-19T18:17:36.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc4317fa640 1 -- 172.21.15.26:0/2137793210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42c014930 con 0x7fc434159c00 2023-12-19T18:17:36.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.233+0000 7fc43ac8c640 1 -- 172.21.15.26:0/2137793210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43415a020 con 0x7fc434159c00 2023-12-19T18:17:36.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.237+0000 7fc43ac8c640 1 -- 172.21.15.26:0/2137793210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43415a800 con 0x7fc434159c00 2023-12-19T18:17:36.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.237+0000 7fc4317fa640 1 -- 172.21.15.26:0/2137793210 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fc42c014a90 con 0x7fc434159c00 2023-12-19T18:17:36.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.237+0000 7fc4317fa640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc418065ca0 0x7fc418068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.237+0000 7fc4317fa640 1 -- 172.21.15.26:0/2137793210 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fc42c084190 con 0x7fc434159c00 2023-12-19T18:17:36.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.237+0000 7fc438a28640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc418065ca0 0x7fc418068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.241+0000 7fc438a28640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc418065ca0 0x7fc418068160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4280ae000 tx=0x7fc4280a7000 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:36.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.241+0000 7fc4317fa640 1 -- 172.21.15.26:0/2137793210 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc42c051520 con 0x7fc434159c00 2023-12-19T18:17:36.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc43ac8c640 1 -- 172.21.15.26:0/2137793210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc434074270 con 0x7fc434159c00 2023-12-19T18:17:36.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc4317fa640 1 -- 172.21.15.26:0/2137793210 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v233) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc42c054b40 con 0x7fc434159c00 2023-12-19T18:17:36.404 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:17:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc43ac8c640 1 -- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc418065ca0 msgr2=0x7fc418068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc43ac8c640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc418065ca0 0x7fc418068160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4280ae000 tx=0x7fc4280a7000 comp rx=0 tx=0).stop 2023-12-19T18:17:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc43ac8c640 1 -- 172.21.15.26:0/2137793210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434159c00 msgr2=0x7fc43414ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc43ac8c640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434159c00 0x7fc43414ef70 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fc42c007e90 tx=0x7fc42c00e4b0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc43ac8c640 1 -- 172.21.15.26:0/2137793210 shutdown_connections 2023-12-19T18:17:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc43ac8c640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc418065ca0 0x7fc418068160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc43ac8c640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc434159c00 0x7fc43414ef70 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc43ac8c640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc434159820 0x7fc4340b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc43ac8c640 1 --2- 172.21.15.26:0/2137793210 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc43406f6c0 0x7fc4340c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc43ac8c640 1 -- 172.21.15.26:0/2137793210 >> 172.21.15.26:0/2137793210 conn(0x7fc4340ce730 msgr2=0x7fc43406ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc43ac8c640 1 -- 172.21.15.26:0/2137793210 shutdown_connections 2023-12-19T18:17:36.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.401+0000 7fc43ac8c640 1 -- 172.21.15.26:0/2137793210 wait complete. 2023-12-19T18:17:36.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:17:36.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.565+0000 7f26f4af2640 1 Processor -- start 2023-12-19T18:17:36.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.565+0000 7f26f4af2640 1 -- start start 2023-12-19T18:17:36.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.569+0000 7f26f4af2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26f015ada0 0x7f26f00706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.569+0000 7f26f4af2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26f006ac80 0x7f26f00b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.569+0000 7f26f4af2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f0070d90 con 0x7f26f015a9c0 2023-12-19T18:17:36.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.569+0000 7f26f4af2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f00cc270 con 0x7f26f015ada0 2023-12-19T18:17:36.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.569+0000 7f26f4af2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f0071070 con 0x7f26f006ac80 2023-12-19T18:17:36.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.569+0000 7f26e7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26f015ada0 0x7f26f00706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.569+0000 7f26e7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26f015ada0 0x7f26f00706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36538/0 (socket says 172.21.15.26:36538) 2023-12-19T18:17:36.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.569+0000 7f26e7fff640 1 -- 172.21.15.26:0/2225047780 learned_addr learned my addr 172.21.15.26:0/2225047780 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:36.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.569+0000 7f26eed9d640 1 --2- 172.21.15.26:0/2225047780 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26f006ac80 0x7f26f00b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.569+0000 7f26e7fff640 1 -- 172.21.15.26:0/2225047780 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26f006ac80 msgr2=0x7f26f00b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:36.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.569+0000 7f26e7fff640 1 --2- 172.21.15.26:0/2225047780 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26f006ac80 0x7f26f00b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.569+0000 7f26e7fff640 1 -- 172.21.15.26:0/2225047780 >> v1:172.21.15.160:6789/0 conn(0x7f26f015a9c0 legacy=0x7f26f00b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:36.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.569+0000 7f26e7fff640 1 -- 172.21.15.26:0/2225047780 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f0071350 con 0x7f26f015ada0 2023-12-19T18:17:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26e7fff640 1 --2- 172.21.15.26:0/2225047780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26f015ada0 0x7f26f00706d0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009870 tx=0x7f26d801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb666ca0f5358d23 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26edd9b640 1 -- 172.21.15.26:0/2225047780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d80ad640 con 0x7f26f015ada0 2023-12-19T18:17:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26edd9b640 1 -- 172.21.15.26:0/2225047780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d80ad7a0 con 0x7f26f015ada0 2023-12-19T18:17:36.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26edd9b640 1 -- 172.21.15.26:0/2225047780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d80adb30 con 0x7f26f015ada0 2023-12-19T18:17:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26f4af2640 1 -- 172.21.15.26:0/2225047780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26f015ada0 msgr2=0x7f26f00706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26f4af2640 1 --2- 172.21.15.26:0/2225047780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26f015ada0 0x7f26f00706d0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009870 tx=0x7f26d801c480 comp rx=0 tx=0).stop 2023-12-19T18:17:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26f4af2640 1 -- 172.21.15.26:0/2225047780 shutdown_connections 2023-12-19T18:17:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26f4af2640 1 --2- 172.21.15.26:0/2225047780 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26f006ac80 0x7f26f00b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26f4af2640 1 --2- 172.21.15.26:0/2225047780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26f015ada0 0x7f26f00706d0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26f4af2640 1 -- 172.21.15.26:0/2225047780 >> 172.21.15.26:0/2225047780 conn(0x7f26f00ce730 msgr2=0x7f26f00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26f4af2640 1 -- 172.21.15.26:0/2225047780 shutdown_connections 2023-12-19T18:17:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26f4af2640 1 -- 172.21.15.26:0/2225047780 wait complete. 2023-12-19T18:17:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26f4af2640 1 Processor -- start 2023-12-19T18:17:36.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.573+0000 7f26f4af2640 1 -- start start 2023-12-19T18:17:36.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26f4af2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26f006ac80 0x7f26f015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26f4af2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26f015a9c0 0x7f26f00c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26f4af2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26f015ada0 0x7f26f0159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26f4af2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f00c4a60 con 0x7f26f015a9c0 2023-12-19T18:17:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26f4af2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f00c4d40 con 0x7f26f006ac80 2023-12-19T18:17:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26f4af2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f00c5020 con 0x7f26f015ada0 2023-12-19T18:17:36.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26e7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26f015a9c0 0x7f26f00c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26e7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26f015a9c0 0x7f26f00c1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36546/0 (socket says 172.21.15.26:36546) 2023-12-19T18:17:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26e7fff640 1 -- 172.21.15.26:0/274381630 learned_addr learned my addr 172.21.15.26:0/274381630 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26ee59c640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26f006ac80 0x7f26f015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26eed9d640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26f015ada0 0x7f26f0159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26e7fff640 1 -- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26f015ada0 msgr2=0x7f26f0159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26e7fff640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26f015ada0 0x7f26f0159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26e7fff640 1 -- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26f006ac80 msgr2=0x7f26f015c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26e7fff640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26f006ac80 0x7f26f015c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26e7fff640 1 -- 172.21.15.26:0/274381630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d80094d0 con 0x7f26f015a9c0 2023-12-19T18:17:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26eed9d640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26f015ada0 0x7f26f0159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26e7fff640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26f015a9c0 0x7f26f00c1dc0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f26d80b0040 tx=0x7f26d8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26e6ffd640 1 -- 172.21.15.26:0/274381630 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d80bc070 con 0x7f26f015a9c0 2023-12-19T18:17:36.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26e6ffd640 1 -- 172.21.15.26:0/274381630 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d80a6040 con 0x7f26f015a9c0 2023-12-19T18:17:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26f4af2640 1 -- 172.21.15.26:0/274381630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f00c5300 con 0x7f26f015a9c0 2023-12-19T18:17:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26e6ffd640 1 -- 172.21.15.26:0/274381630 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d80b7850 con 0x7f26f015a9c0 2023-12-19T18:17:36.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26f4af2640 1 -- 172.21.15.26:0/274381630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f00c4710 con 0x7f26f015a9c0 2023-12-19T18:17:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.577+0000 7f26f4af2640 1 -- 172.21.15.26:0/274381630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f0074270 con 0x7f26f015a9c0 2023-12-19T18:17:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.581+0000 7f26e6ffd640 1 -- 172.21.15.26:0/274381630 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f26d80b79b0 con 0x7f26f015a9c0 2023-12-19T18:17:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.581+0000 7f26e6ffd640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f26d0065ca0 0x7f26d0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.581+0000 7f26e6ffd640 1 -- 172.21.15.26:0/274381630 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f26d811e7a0 con 0x7f26f015a9c0 2023-12-19T18:17:36.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.581+0000 7f26ee59c640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f26d0065ca0 0x7f26d0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.581+0000 7f26ee59c640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f26d0065ca0 0x7f26d0068160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f26e0007fd0 tx=0x7f26e0009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:36.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.585+0000 7f26e6ffd640 1 -- 172.21.15.26:0/274381630 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f26d80ebc90 con 0x7f26f015a9c0 2023-12-19T18:17:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26f4af2640 1 -- 172.21.15.26:0/274381630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f26f00cd7b0 con 0x7f26f015a9c0 2023-12-19T18:17:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26e6ffd640 1 -- 172.21.15.26:0/274381630 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v233) v1 ==== 100+0+11 (secure 0 0 0) 0x7f26d80ef2b0 con 0x7f26f015a9c0 2023-12-19T18:17:36.744 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:17:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26f4af2640 1 -- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f26d0065ca0 msgr2=0x7f26d0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26f4af2640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f26d0065ca0 0x7f26d0068160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f26e0007fd0 tx=0x7f26e0009040 comp rx=0 tx=0).stop 2023-12-19T18:17:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26f4af2640 1 -- 172.21.15.26:0/274381630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26f015a9c0 msgr2=0x7f26f00c1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26f4af2640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26f015a9c0 0x7f26f00c1dc0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f26d80b0040 tx=0x7f26d8019220 comp rx=0 tx=0).stop 2023-12-19T18:17:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26f4af2640 1 -- 172.21.15.26:0/274381630 shutdown_connections 2023-12-19T18:17:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26f4af2640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f26d0065ca0 0x7f26d0068160 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26f4af2640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f26f015ada0 0x7f26f0159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26f4af2640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26f015a9c0 0x7f26f00c1dc0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26f4af2640 1 --2- 172.21.15.26:0/274381630 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26f006ac80 0x7f26f015c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26f4af2640 1 -- 172.21.15.26:0/274381630 >> 172.21.15.26:0/274381630 conn(0x7f26f00ce730 msgr2=0x7f26f00b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26f4af2640 1 -- 172.21.15.26:0/274381630 shutdown_connections 2023-12-19T18:17:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.741+0000 7f26f4af2640 1 -- 172.21.15.26:0/274381630 wait complete. 2023-12-19T18:17:36.766 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:17:36.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.905+0000 7fd3f738e640 1 Processor -- start 2023-12-19T18:17:36.910 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.905+0000 7fd3f738e640 1 -- start start 2023-12-19T18:17:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.905+0000 7fd3f738e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd3f00affe0 0x7fd3f0071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.905+0000 7fd3f738e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3f00b3960 0x7fd3f00b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.905+0000 7fd3f738e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f015e7c0 con 0x7fd3f00b2030 2023-12-19T18:17:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.905+0000 7fd3f738e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f00c97a0 con 0x7fd3f00b3960 2023-12-19T18:17:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.905+0000 7fd3f738e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f0072360 con 0x7fd3f00affe0 2023-12-19T18:17:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.909+0000 7fd3f6b8d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3f00b3960 0x7fd3f00b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.909+0000 7fd3f6b8d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3f00b3960 0x7fd3f00b3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36558/0 (socket says 172.21.15.26:36558) 2023-12-19T18:17:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.909+0000 7fd3f6b8d640 1 -- 172.21.15.26:0/3590390466 learned_addr learned my addr 172.21.15.26:0/3590390466 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.909+0000 7fd3f5b8b640 1 --2- 172.21.15.26:0/3590390466 >> v2:172.21.15.160:3300/0 conn(0x7fd3f00affe0 0x7fd3f0071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.909+0000 7fd3f6b8d640 1 -- 172.21.15.26:0/3590390466 >> v2:172.21.15.160:3300/0 conn(0x7fd3f00affe0 msgr2=0x7fd3f0071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.909+0000 7fd3f6b8d640 1 --2- 172.21.15.26:0/3590390466 >> v2:172.21.15.160:3300/0 conn(0x7fd3f00affe0 0x7fd3f0071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.909+0000 7fd3f6b8d640 1 -- 172.21.15.26:0/3590390466 >> v1:172.21.15.26:6789/0 conn(0x7fd3f00b2030 legacy=0x7fd3f00c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.909+0000 7fd3f6b8d640 1 -- 172.21.15.26:0/3590390466 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f00b04c0 con 0x7fd3f00b3960 2023-12-19T18:17:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.909+0000 7fd3f5b8b640 1 --2- 172.21.15.26:0/3590390466 >> v2:172.21.15.160:3300/0 conn(0x7fd3f00affe0 0x7fd3f0071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:36.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.909+0000 7fd3f6b8d640 1 --2- 172.21.15.26:0/3590390466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f00b3960 0x7fd3f00b3e10 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00cbd80 tx=0x7fd3e801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f31a807cd5908bd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.909+0000 7fd3f538a640 1 -- 172.21.15.26:0/3590390466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80a5860 con 0x7fd3f00b3960 2023-12-19T18:17:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.909+0000 7fd3f538a640 1 -- 172.21.15.26:0/3590390466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e80a59c0 con 0x7fd3f00b3960 2023-12-19T18:17:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.909+0000 7fd3f538a640 1 -- 172.21.15.26:0/3590390466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80a5d50 con 0x7fd3f00b3960 2023-12-19T18:17:36.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 -- 172.21.15.26:0/3590390466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f00b3960 msgr2=0x7fd3f00b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 --2- 172.21.15.26:0/3590390466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f00b3960 0x7fd3f00b3e10 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00cbd80 tx=0x7fd3e801c9a0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 -- 172.21.15.26:0/3590390466 shutdown_connections 2023-12-19T18:17:36.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 --2- 172.21.15.26:0/3590390466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f00b3960 0x7fd3f00b3e10 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 --2- 172.21.15.26:0/3590390466 >> v2:172.21.15.160:3300/0 conn(0x7fd3f00affe0 0x7fd3f0071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 -- 172.21.15.26:0/3590390466 >> 172.21.15.26:0/3590390466 conn(0x7fd3f006b0f0 msgr2=0x7fd3f006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:36.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 -- 172.21.15.26:0/3590390466 shutdown_connections 2023-12-19T18:17:36.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 -- 172.21.15.26:0/3590390466 wait complete. 2023-12-19T18:17:36.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 Processor -- start 2023-12-19T18:17:36.917 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 -- start start 2023-12-19T18:17:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3f00affe0 0x7fd3f00c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f00b2030 0x7fd3f014a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd3f00b3960 0x7fd3f0157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00c7400 con 0x7fd3f00b2030 2023-12-19T18:17:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00c76e0 con 0x7fd3f00affe0 2023-12-19T18:17:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f738e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f00c79c0 con 0x7fd3f00b3960 2023-12-19T18:17:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f5b8b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f00b2030 0x7fd3f014a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f5b8b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f00b2030 0x7fd3f014a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36570/0 (socket says 172.21.15.26:36570) 2023-12-19T18:17:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.913+0000 7fd3f5b8b640 1 -- 172.21.15.26:0/1290829939 learned_addr learned my addr 172.21.15.26:0/1290829939 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3f6b8d640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd3f00b3960 0x7fd3f0157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3f638c640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3f00affe0 0x7fd3f00c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3f5b8b640 1 -- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd3f00b3960 msgr2=0x7fd3f0157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3f5b8b640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd3f00b3960 0x7fd3f0157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3f5b8b640 1 -- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3f00affe0 msgr2=0x7fd3f00c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3f5b8b640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3f00affe0 0x7fd3f00c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3f5b8b640 1 -- 172.21.15.26:0/1290829939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e8009b30 con 0x7fd3f00b2030 2023-12-19T18:17:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3f6b8d640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd3f00b3960 0x7fd3f0157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3f638c640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3f00affe0 0x7fd3f00c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3f5b8b640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f00b2030 0x7fd3f014a5e0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000d800 tx=0x7fd3e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3e6ffd640 1 -- 172.21.15.26:0/1290829939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e000b6c0 con 0x7fd3f00b2030 2023-12-19T18:17:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3e6ffd640 1 -- 172.21.15.26:0/1290829939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e000b820 con 0x7fd3f00b2030 2023-12-19T18:17:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3e6ffd640 1 -- 172.21.15.26:0/1290829939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e000bbd0 con 0x7fd3f00b2030 2023-12-19T18:17:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3f738e640 1 -- 172.21.15.26:0/1290829939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f00c7ca0 con 0x7fd3f00b2030 2023-12-19T18:17:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3f738e640 1 -- 172.21.15.26:0/1290829939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f00c5ae0 con 0x7fd3f00b2030 2023-12-19T18:17:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3f738e640 1 -- 172.21.15.26:0/1290829939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f0067400 con 0x7fd3f00b2030 2023-12-19T18:17:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3e6ffd640 1 -- 172.21.15.26:0/1290829939 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fd3e000bd30 con 0x7fd3f00b2030 2023-12-19T18:17:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3e6ffd640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd3d4065cd0 0x7fd3d4068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.917+0000 7fd3e6ffd640 1 -- 172.21.15.26:0/1290829939 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fd3e0083e20 con 0x7fd3f00b2030 2023-12-19T18:17:36.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.921+0000 7fd3f638c640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd3d4065cd0 0x7fd3d4068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:36.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.921+0000 7fd3f638c640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd3d4065cd0 0x7fd3d4068190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec000f70 tx=0x7fd3ec007b50 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:36.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:36.921+0000 7fd3e6ffd640 1 -- 172.21.15.26:0/1290829939 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd3e00510e0 con 0x7fd3f00b2030 2023-12-19T18:17:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.081+0000 7fd3f738e640 1 -- 172.21.15.26:0/1290829939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd3f00742c0 con 0x7fd3f00b2030 2023-12-19T18:17:37.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.081+0000 7fd3e6ffd640 1 -- 172.21.15.26:0/1290829939 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v233) v1 ==== 84+0+209 (secure 0 0 0) 0x7fd3e0054700 con 0x7fd3f00b2030 2023-12-19T18:17:37.087 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:37.087 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T18:17:37.087 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:37.087 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T18:17:37.087 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T18:17:37.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:37.087 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:37.087 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 28, 2023-12-19T18:17:37.087 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": "cephfs_metadata" 2023-12-19T18:17:37.087 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:37.087 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:37.087 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 29, 2023-12-19T18:17:37.087 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": "cephfs_data" 2023-12-19T18:17:37.088 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:37.088 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T18:17:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.085+0000 7fd3f738e640 1 -- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd3d4065cd0 msgr2=0x7fd3d4068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.085+0000 7fd3f738e640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd3d4065cd0 0x7fd3d4068190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec000f70 tx=0x7fd3ec007b50 comp rx=0 tx=0).stop 2023-12-19T18:17:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.085+0000 7fd3f738e640 1 -- 172.21.15.26:0/1290829939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f00b2030 msgr2=0x7fd3f014a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.085+0000 7fd3f738e640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f00b2030 0x7fd3f014a5e0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000d800 tx=0x7fd3e000dcc0 comp rx=0 tx=0).stop 2023-12-19T18:17:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.085+0000 7fd3f738e640 1 -- 172.21.15.26:0/1290829939 shutdown_connections 2023-12-19T18:17:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.085+0000 7fd3f738e640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd3d4065cd0 0x7fd3d4068190 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.085+0000 7fd3f738e640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd3f00b3960 0x7fd3f0157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.085+0000 7fd3f738e640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3f00b2030 0x7fd3f014a5e0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:37.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.085+0000 7fd3f738e640 1 --2- 172.21.15.26:0/1290829939 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd3f00affe0 0x7fd3f00c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.085+0000 7fd3f738e640 1 -- 172.21.15.26:0/1290829939 >> 172.21.15.26:0/1290829939 conn(0x7fd3f006b0f0 msgr2=0x7fd3f00b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.085+0000 7fd3f738e640 1 -- 172.21.15.26:0/1290829939 shutdown_connections 2023-12-19T18:17:37.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.085+0000 7fd3f738e640 1 -- 172.21.15.26:0/1290829939 wait complete. 2023-12-19T18:17:37.108 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-12-19T18:17:37.108 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-12-19T18:17:37.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51eefce640 1 Processor -- start 2023-12-19T18:17:37.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51eefce640 1 -- start start 2023-12-19T18:17:37.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51eefce640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51e80aee50 0x7f51e8070850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:37.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51eefce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e80ceef0 con 0x7f51e80cd3c0 2023-12-19T18:17:37.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51eefce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e80cd8a0 con 0x7f51e815cdd0 2023-12-19T18:17:37.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51eefce640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e80ce130 con 0x7f51e80aee50 2023-12-19T18:17:37.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51e7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51e80aee50 0x7f51e8070850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:37.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51ed56b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f51e80cd3c0 0x7f51e8160950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59474/0 (socket says 172.21.15.26:59474) 2023-12-19T18:17:37.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51e7fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f51e80aee50 0x7f51e8070850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36582/0 (socket says 172.21.15.26:36582) 2023-12-19T18:17:37.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51ed56b640 1 -- 172.21.15.26:0/1685000902 learned_addr learned my addr 172.21.15.26:0/1685000902 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:37.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51e7fff640 1 -- 172.21.15.26:0/1685000902 >> v1:172.21.15.160:6789/0 conn(0x7f51e815cdd0 legacy=0x7f51e80ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:37.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51e7fff640 1 -- 172.21.15.26:0/1685000902 >> v1:172.21.15.26:6789/0 conn(0x7f51e80cd3c0 legacy=0x7f51e8160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:37.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51e7fff640 1 -- 172.21.15.26:0/1685000902 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e80ce410 con 0x7f51e80aee50 2023-12-19T18:17:37.259 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.253+0000 7f51e77fe640 1 -- 172.21.15.26:0/1685000902 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256556518 0 0) 0x7f51e80ceef0 con 0x7f51e80cd3c0 2023-12-19T18:17:37.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.257+0000 7f51e7fff640 1 --2- 172.21.15.26:0/1685000902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e80aee50 0x7f51e8070850 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f51d40097c0 tx=0x7f51d401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e98466a540cfdc5c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:37.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.257+0000 7f51e77fe640 1 -- 172.21.15.26:0/1685000902 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d40a8970 con 0x7f51e80aee50 2023-12-19T18:17:37.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.257+0000 7f51e77fe640 1 -- 172.21.15.26:0/1685000902 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d40a8ad0 con 0x7f51e80aee50 2023-12-19T18:17:37.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.257+0000 7f51e77fe640 1 -- 172.21.15.26:0/1685000902 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d40a8e60 con 0x7f51e80aee50 2023-12-19T18:17:37.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.257+0000 7f51eefce640 1 -- 172.21.15.26:0/1685000902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e80aee50 msgr2=0x7f51e8070850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.257+0000 7f51eefce640 1 --2- 172.21.15.26:0/1685000902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e80aee50 0x7f51e8070850 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f51d40097c0 tx=0x7f51d401ec70 comp rx=0 tx=0).stop 2023-12-19T18:17:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.257+0000 7f51eefce640 1 -- 172.21.15.26:0/1685000902 shutdown_connections 2023-12-19T18:17:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.257+0000 7f51eefce640 1 --2- 172.21.15.26:0/1685000902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e80aee50 0x7f51e8070850 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.257+0000 7f51eefce640 1 -- 172.21.15.26:0/1685000902 >> 172.21.15.26:0/1685000902 conn(0x7f51e80cfa50 msgr2=0x7f51e80cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.257+0000 7f51eefce640 1 -- 172.21.15.26:0/1685000902 shutdown_connections 2023-12-19T18:17:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.257+0000 7f51eefce640 1 -- 172.21.15.26:0/1685000902 wait complete. 2023-12-19T18:17:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51eefce640 1 Processor -- start 2023-12-19T18:17:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51eefce640 1 -- start start 2023-12-19T18:17:37.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51eefce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51e80aee50 0x7f51e81f1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:37.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51eefce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e80cd3c0 0x7f51e815f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:37.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51eefce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e815cdd0 0x7f51e8159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:37.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51eefce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e81f2160 con 0x7f51e80cd3c0 2023-12-19T18:17:37.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51eefce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e81f5d10 con 0x7f51e80aee50 2023-12-19T18:17:37.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51eefce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e81f5ff0 con 0x7f51e815cdd0 2023-12-19T18:17:37.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e80cd3c0 0x7f51e815f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51e7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e80cd3c0 0x7f51e815f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36596/0 (socket says 172.21.15.26:36596) 2023-12-19T18:17:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51e7fff640 1 -- 172.21.15.26:0/2454515970 learned_addr learned my addr 172.21.15.26:0/2454515970 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51ecd6a640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51e80aee50 0x7f51e81f1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51ed56b640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e815cdd0 0x7f51e8159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51e7fff640 1 -- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e815cdd0 msgr2=0x7f51e8159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51e7fff640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e815cdd0 0x7f51e8159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51e7fff640 1 -- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51e80aee50 msgr2=0x7f51e81f1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51e7fff640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51e80aee50 0x7f51e81f1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51e7fff640 1 -- 172.21.15.26:0/2454515970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d4009420 con 0x7f51e80cd3c0 2023-12-19T18:17:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51ecd6a640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51e80aee50 0x7f51e81f1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51e7fff640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e80cd3c0 0x7f51e815f800 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f51d401b880 tx=0x7f51d401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51e57fa640 1 -- 172.21.15.26:0/2454515970 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d40b0ea0 con 0x7f51e80cd3c0 2023-12-19T18:17:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51e57fa640 1 -- 172.21.15.26:0/2454515970 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d40b9410 con 0x7f51e80cd3c0 2023-12-19T18:17:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51e57fa640 1 -- 172.21.15.26:0/2454515970 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d40b97a0 con 0x7f51e80cd3c0 2023-12-19T18:17:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51eefce640 1 -- 172.21.15.26:0/2454515970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e81f62d0 con 0x7f51e80cd3c0 2023-12-19T18:17:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.261+0000 7f51eefce640 1 -- 172.21.15.26:0/2454515970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e815fd40 con 0x7f51e80cd3c0 2023-12-19T18:17:37.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.265+0000 7f51eefce640 1 -- 172.21.15.26:0/2454515970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e8074270 con 0x7f51e80cd3c0 2023-12-19T18:17:37.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.265+0000 7f51e57fa640 1 -- 172.21.15.26:0/2454515970 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f51d40b7040 con 0x7f51e80cd3c0 2023-12-19T18:17:37.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.265+0000 7f51e57fa640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f51cc065ca0 0x7f51cc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:37.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.265+0000 7f51e57fa640 1 -- 172.21.15.26:0/2454515970 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f51d4121610 con 0x7f51e80cd3c0 2023-12-19T18:17:37.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.265+0000 7f51ecd6a640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f51cc065ca0 0x7f51cc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.265+0000 7f51ecd6a640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f51cc065ca0 0x7f51cc068160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f51dc000f70 tx=0x7f51dc009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.269+0000 7f51e57fa640 1 -- 172.21.15.26:0/2454515970 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f51d40ee8d0 con 0x7f51e80cd3c0 2023-12-19T18:17:37.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:37.429+0000 7f51eefce640 1 -- 172.21.15.26:0/2454515970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f51e80c9a40 con 0x7f51e80cd3c0 2023-12-19T18:17:38.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.381+0000 7f51e57fa640 1 -- 172.21.15.26:0/2454515970 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v234) v1 ==== 144+0+0 (secure 0 0 0) 0x7f51d40f1ef0 con 0x7f51e80cd3c0 2023-12-19T18:17:38.386 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'backup_fs_metadata' created 2023-12-19T18:17:38.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.385+0000 7f51eefce640 1 -- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f51cc065ca0 msgr2=0x7f51cc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:38.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.385+0000 7f51eefce640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f51cc065ca0 0x7f51cc068160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f51dc000f70 tx=0x7f51dc009040 comp rx=0 tx=0).stop 2023-12-19T18:17:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.385+0000 7f51eefce640 1 -- 172.21.15.26:0/2454515970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e80cd3c0 msgr2=0x7f51e815f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.385+0000 7f51eefce640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e80cd3c0 0x7f51e815f800 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f51d401b880 tx=0x7f51d401ba10 comp rx=0 tx=0).stop 2023-12-19T18:17:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.385+0000 7f51eefce640 1 -- 172.21.15.26:0/2454515970 shutdown_connections 2023-12-19T18:17:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.389+0000 7f51eefce640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f51cc065ca0 0x7f51cc068160 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.389+0000 7f51eefce640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51e815cdd0 0x7f51e8159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.389+0000 7f51eefce640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51e80cd3c0 0x7f51e815f800 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.389+0000 7f51eefce640 1 --2- 172.21.15.26:0/2454515970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51e80aee50 0x7f51e81f1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.389+0000 7f51eefce640 1 -- 172.21.15.26:0/2454515970 >> 172.21.15.26:0/2454515970 conn(0x7f51e80cfa50 msgr2=0x7f51e815b470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:38.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.389+0000 7f51eefce640 1 -- 172.21.15.26:0/2454515970 shutdown_connections 2023-12-19T18:17:38.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.389+0000 7f51eefce640 1 -- 172.21.15.26:0/2454515970 wait complete. 2023-12-19T18:17:38.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:17:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.569+0000 7f673377b640 1 Processor -- start 2023-12-19T18:17:38.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.569+0000 7f673377b640 1 -- start start 2023-12-19T18:17:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.569+0000 7f673377b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f672c15cfa0 0x7f672c06d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.569+0000 7f673377b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f672c0cc310 0x7f672c0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.569+0000 7f673377b640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672c0c6710 con 0x7f672c15cbc0 2023-12-19T18:17:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.569+0000 7f673377b640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672c0ccd00 con 0x7f672c15cfa0 2023-12-19T18:17:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.569+0000 7f673377b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c0c9ec0 con 0x7f672c0cc310 2023-12-19T18:17:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.569+0000 7f6730d16640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f672c15cfa0 0x7f672c06d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.569+0000 7f6730d16640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f672c15cfa0 0x7f672c06d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36604/0 (socket says 172.21.15.26:36604) 2023-12-19T18:17:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.569+0000 7f6730d16640 1 -- 172.21.15.26:0/2486934030 learned_addr learned my addr 172.21.15.26:0/2486934030 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:38.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.569+0000 7f6731d18640 1 --2- 172.21.15.26:0/2486934030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f672c0cc310 0x7f672c0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f6730d16640 1 -- 172.21.15.26:0/2486934030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f672c0cc310 msgr2=0x7f672c0cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f6730d16640 1 --2- 172.21.15.26:0/2486934030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f672c0cc310 0x7f672c0cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f6723fff640 1 -- 172.21.15.26:0/2486934030 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2630457017 0 0) 0x7f672c0c6710 con 0x7f672c15cbc0 2023-12-19T18:17:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f6730d16640 1 -- 172.21.15.26:0/2486934030 >> v1:172.21.15.26:6789/0 conn(0x7f672c15cbc0 legacy=0x7f672c0c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:38.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f6730d16640 1 -- 172.21.15.26:0/2486934030 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c070ce0 con 0x7f672c15cfa0 2023-12-19T18:17:38.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f6730d16640 1 --2- 172.21.15.26:0/2486934030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f672c15cfa0 0x7f672c06d7a0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f671c009870 tx=0x7f671c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953d026a5be196e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f6723fff640 1 -- 172.21.15.26:0/2486934030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0ac600 con 0x7f672c15cfa0 2023-12-19T18:17:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f6723fff640 1 -- 172.21.15.26:0/2486934030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c0a8050 con 0x7f672c15cfa0 2023-12-19T18:17:38.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f6723fff640 1 -- 172.21.15.26:0/2486934030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0ac980 con 0x7f672c15cfa0 2023-12-19T18:17:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f673377b640 1 -- 172.21.15.26:0/2486934030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f672c15cfa0 msgr2=0x7f672c06d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f673377b640 1 --2- 172.21.15.26:0/2486934030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f672c15cfa0 0x7f672c06d7a0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f671c009870 tx=0x7f671c01c4b0 comp rx=0 tx=0).stop 2023-12-19T18:17:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f673377b640 1 -- 172.21.15.26:0/2486934030 shutdown_connections 2023-12-19T18:17:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f673377b640 1 --2- 172.21.15.26:0/2486934030 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f672c0cc310 0x7f672c0cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f673377b640 1 --2- 172.21.15.26:0/2486934030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f672c15cfa0 0x7f672c06d7a0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.573+0000 7f673377b640 1 -- 172.21.15.26:0/2486934030 >> 172.21.15.26:0/2486934030 conn(0x7f672c069d20 msgr2=0x7f672c06a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f673377b640 1 -- 172.21.15.26:0/2486934030 shutdown_connections 2023-12-19T18:17:38.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f673377b640 1 -- 172.21.15.26:0/2486934030 wait complete. 2023-12-19T18:17:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f673377b640 1 Processor -- start 2023-12-19T18:17:38.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f673377b640 1 -- start start 2023-12-19T18:17:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f673377b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f672c0cc310 0x7f672c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f673377b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f672c15cbc0 0x7f672c14fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f673377b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f672c15cfa0 0x7f672c1492e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:38.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f673377b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c0ceec0 con 0x7f672c15cbc0 2023-12-19T18:17:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f673377b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c15e890 con 0x7f672c15cfa0 2023-12-19T18:17:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f673377b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c0cf180 con 0x7f672c0cc310 2023-12-19T18:17:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6730d16640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f672c15cbc0 0x7f672c14fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6730d16640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f672c15cbc0 0x7f672c14fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36620/0 (socket says 172.21.15.26:36620) 2023-12-19T18:17:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6730d16640 1 -- 172.21.15.26:0/3611772430 learned_addr learned my addr 172.21.15.26:0/3611772430 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6731517640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f672c0cc310 0x7f672c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6731d18640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f672c15cfa0 0x7f672c1492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6730d16640 1 -- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f672c0cc310 msgr2=0x7f672c0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6730d16640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f672c0cc310 0x7f672c0cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6730d16640 1 -- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f672c15cfa0 msgr2=0x7f672c1492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6730d16640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f672c15cfa0 0x7f672c1492e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:38.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6730d16640 1 -- 172.21.15.26:0/3611772430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c0094d0 con 0x7f672c15cbc0 2023-12-19T18:17:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6731d18640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f672c15cfa0 0x7f672c1492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6730d16640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f672c15cbc0 0x7f672c14fc40 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f671c01a0e0 tx=0x7f671c019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:38.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6721ffb640 1 -- 172.21.15.26:0/3611772430 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0b7620 con 0x7f672c15cbc0 2023-12-19T18:17:38.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6721ffb640 1 -- 172.21.15.26:0/3611772430 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c0b7780 con 0x7f672c15cbc0 2023-12-19T18:17:38.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.577+0000 7f6721ffb640 1 -- 172.21.15.26:0/3611772430 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c0b7b30 con 0x7f672c15cbc0 2023-12-19T18:17:38.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.581+0000 7f673377b640 1 -- 172.21.15.26:0/3611772430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672c0cf460 con 0x7f672c15cbc0 2023-12-19T18:17:38.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.581+0000 7f673377b640 1 -- 172.21.15.26:0/3611772430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c150350 con 0x7f672c15cbc0 2023-12-19T18:17:38.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.581+0000 7f673377b640 1 -- 172.21.15.26:0/3611772430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672c074270 con 0x7f672c15cbc0 2023-12-19T18:17:38.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.581+0000 7f6721ffb640 1 -- 172.21.15.26:0/3611772430 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f671c0b3040 con 0x7f672c15cbc0 2023-12-19T18:17:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.581+0000 7f6721ffb640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6710065ca0 0x7f6710068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.581+0000 7f6721ffb640 1 -- 172.21.15.26:0/3611772430 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6842+0+0 (secure 0 0 0) 0x7f671c11e140 con 0x7f672c15cbc0 2023-12-19T18:17:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.581+0000 7f6731517640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6710065ca0 0x7f6710068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:38.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.585+0000 7f6731517640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6710065ca0 0x7f6710068160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6728000f70 tx=0x7f6728009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:38.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.585+0000 7f6721ffb640 1 -- 172.21.15.26:0/3611772430 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f671c0eb530 con 0x7f672c15cbc0 2023-12-19T18:17:38.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:38.745+0000 7f673377b640 1 -- 172.21.15.26:0/3611772430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f672c0ae170 con 0x7f672c15cbc0 2023-12-19T18:17:39.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.393+0000 7f6721ffb640 1 -- 172.21.15.26:0/3611772430 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v235) v1 ==== 176+0+0 (secure 0 0 0) 0x7f671c0eeb50 con 0x7f672c15cbc0 2023-12-19T18:17:39.399 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'backup_fs_data' created 2023-12-19T18:17:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.401+0000 7f673377b640 1 -- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6710065ca0 msgr2=0x7f6710068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.401+0000 7f673377b640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6710065ca0 0x7f6710068160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6728000f70 tx=0x7f6728009040 comp rx=0 tx=0).stop 2023-12-19T18:17:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.401+0000 7f673377b640 1 -- 172.21.15.26:0/3611772430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f672c15cbc0 msgr2=0x7f672c14fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:39.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.401+0000 7f673377b640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f672c15cbc0 0x7f672c14fc40 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f671c01a0e0 tx=0x7f671c019250 comp rx=0 tx=0).stop 2023-12-19T18:17:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.405+0000 7f673377b640 1 -- 172.21.15.26:0/3611772430 shutdown_connections 2023-12-19T18:17:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.405+0000 7f673377b640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6710065ca0 0x7f6710068160 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.405+0000 7f673377b640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f672c15cfa0 0x7f672c1492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.405+0000 7f673377b640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f672c15cbc0 0x7f672c14fc40 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.405+0000 7f673377b640 1 --2- 172.21.15.26:0/3611772430 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f672c0cc310 0x7f672c0cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.405+0000 7f673377b640 1 -- 172.21.15.26:0/3611772430 >> 172.21.15.26:0/3611772430 conn(0x7f672c069d20 msgr2=0x7f672c151cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:39.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.405+0000 7f673377b640 1 -- 172.21.15.26:0/3611772430 shutdown_connections 2023-12-19T18:17:39.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.405+0000 7f673377b640 1 -- 172.21.15.26:0/3611772430 wait complete. 2023-12-19T18:17:39.442 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-12-19T18:17:39.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.585+0000 7f18f5a04640 1 Processor -- start 2023-12-19T18:17:39.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.585+0000 7f18f5a04640 1 -- start start 2023-12-19T18:17:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18f5a04640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f18f0160330 0x7f18f00d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18f5a04640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f18f00b8460 0x7f18f00b4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18f5a04640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18f00b4a90 0x7f18f0069d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18f5a04640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f00b4f20 con 0x7f18f0160330 2023-12-19T18:17:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18f5a04640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f00d2370 con 0x7f18f00b8460 2023-12-19T18:17:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18f5a04640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f00d2800 con 0x7f18f00b4a90 2023-12-19T18:17:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18eeffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f18f0160330 0x7f18f00d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:39.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18eeffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f18f0160330 0x7f18f00d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36630/0 (socket says 172.21.15.26:36630) 2023-12-19T18:17:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18eeffd640 1 -- 172.21.15.26:0/1996244583 learned_addr learned my addr 172.21.15.26:0/1996244583 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18ef7fe640 1 --2- 172.21.15.26:0/1996244583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18f00b4a90 0x7f18f0069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18e7fff640 1 --2- 172.21.15.26:0/1996244583 >> v2:172.21.15.160:3300/0 conn(0x7f18f00b8460 0x7f18f00b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18eeffd640 1 -- 172.21.15.26:0/1996244583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18f00b4a90 msgr2=0x7f18f0069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18eeffd640 1 --2- 172.21.15.26:0/1996244583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18f00b4a90 0x7f18f0069d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18eeffd640 1 -- 172.21.15.26:0/1996244583 >> v2:172.21.15.160:3300/0 conn(0x7f18f00b8460 msgr2=0x7f18f00b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18eeffd640 1 --2- 172.21.15.26:0/1996244583 >> v2:172.21.15.160:3300/0 conn(0x7f18f00b8460 0x7f18f00b4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18eeffd640 1 -- 172.21.15.26:0/1996244583 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f00d1760 con 0x7f18f0160330 2023-12-19T18:17:39.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.589+0000 7f18e7fff640 1 --2- 172.21.15.26:0/1996244583 >> v2:172.21.15.160:3300/0 conn(0x7f18f00b8460 0x7f18f00b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:39.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18eeffd640 1 --2- 172.21.15.26:0/1996244583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18f0160330 0x7f18f00d0f10 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f18f00c6420 tx=0x7f18e001c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68eb3af6332ca0f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:39.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18ee7fc640 1 -- 172.21.15.26:0/1996244583 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e00ab600 con 0x7f18f0160330 2023-12-19T18:17:39.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18ee7fc640 1 -- 172.21.15.26:0/1996244583 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e00a7050 con 0x7f18f0160330 2023-12-19T18:17:39.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18ee7fc640 1 -- 172.21.15.26:0/1996244583 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e00ab980 con 0x7f18f0160330 2023-12-19T18:17:39.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18f5a04640 1 -- 172.21.15.26:0/1996244583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18f0160330 msgr2=0x7f18f00d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:39.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18f5a04640 1 --2- 172.21.15.26:0/1996244583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18f0160330 0x7f18f00d0f10 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f18f00c6420 tx=0x7f18e001c210 comp rx=0 tx=0).stop 2023-12-19T18:17:39.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18f5a04640 1 -- 172.21.15.26:0/1996244583 shutdown_connections 2023-12-19T18:17:39.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18f5a04640 1 --2- 172.21.15.26:0/1996244583 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18f00b4a90 0x7f18f0069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18f5a04640 1 --2- 172.21.15.26:0/1996244583 >> v2:172.21.15.160:3300/0 conn(0x7f18f00b8460 0x7f18f00b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18f5a04640 1 --2- 172.21.15.26:0/1996244583 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18f0160330 0x7f18f00d0f10 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18f5a04640 1 -- 172.21.15.26:0/1996244583 >> 172.21.15.26:0/1996244583 conn(0x7f18f0065ee0 msgr2=0x7f18f00cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:39.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18f5a04640 1 -- 172.21.15.26:0/1996244583 shutdown_connections 2023-12-19T18:17:39.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18f5a04640 1 -- 172.21.15.26:0/1996244583 wait complete. 2023-12-19T18:17:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18f5a04640 1 Processor -- start 2023-12-19T18:17:39.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18f5a04640 1 -- start start 2023-12-19T18:17:39.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.593+0000 7f18f5a04640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18f00b4a90 0x7f18f014c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18f5a04640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18f00b8460 0x7f18f014fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18f5a04640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18f0160330 0x7f18f00c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18f5a04640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f014c5d0 con 0x7f18f0160330 2023-12-19T18:17:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18f5a04640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f014cf30 con 0x7f18f00b8460 2023-12-19T18:17:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18f5a04640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f014bc30 con 0x7f18f00b4a90 2023-12-19T18:17:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18ef7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18f0160330 0x7f18f00c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18eeffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18f00b4a90 0x7f18f014c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18ef7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18f0160330 0x7f18f00c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36634/0 (socket says 172.21.15.26:36634) 2023-12-19T18:17:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18ef7fe640 1 -- 172.21.15.26:0/2160682525 learned_addr learned my addr 172.21.15.26:0/2160682525 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18eeffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18f00b4a90 0x7f18f014c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:55154/0 (socket says 172.21.15.26:55154) 2023-12-19T18:17:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18e7fff640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18f00b8460 0x7f18f014fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:39.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18eeffd640 1 -- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18f00b8460 msgr2=0x7f18f014fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:39.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18eeffd640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18f00b8460 0x7f18f014fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18eeffd640 1 -- 172.21.15.26:0/2160682525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18f0160330 msgr2=0x7f18f00c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:39.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18eeffd640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18f0160330 0x7f18f00c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18eeffd640 1 -- 172.21.15.26:0/2160682525 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e00094d0 con 0x7f18f00b4a90 2023-12-19T18:17:39.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18ef7fe640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18f0160330 0x7f18f00c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:39.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18eeffd640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18f00b4a90 0x7f18f014c9f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f18e00ab000 tx=0x7f18e0018fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:39.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18ecff9640 1 -- 172.21.15.26:0/2160682525 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e00b4440 con 0x7f18f00b4a90 2023-12-19T18:17:39.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18ecff9640 1 -- 172.21.15.26:0/2160682525 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e00b45a0 con 0x7f18f00b4a90 2023-12-19T18:17:39.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18f5a04640 1 -- 172.21.15.26:0/2160682525 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f014bf10 con 0x7f18f00b4a90 2023-12-19T18:17:39.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18ecff9640 1 -- 172.21.15.26:0/2160682525 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e00b4910 con 0x7f18f00b4a90 2023-12-19T18:17:39.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18f5a04640 1 -- 172.21.15.26:0/2160682525 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f01502d0 con 0x7f18f00b4a90 2023-12-19T18:17:39.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18f5a04640 1 -- 172.21.15.26:0/2160682525 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f014c710 con 0x7f18f00b4a90 2023-12-19T18:17:39.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.597+0000 7f18ecff9640 1 -- 172.21.15.26:0/2160682525 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f18e00b2070 con 0x7f18f00b4a90 2023-12-19T18:17:39.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.601+0000 7f18ecff9640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f18d0065ca0 0x7f18d0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:39.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.601+0000 7f18ecff9640 1 -- 172.21.15.26:0/2160682525 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 7227+0+0 (secure 0 0 0) 0x7f18e011de30 con 0x7f18f00b4a90 2023-12-19T18:17:39.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.601+0000 7f18e7fff640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f18d0065ca0 0x7f18d0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:39.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.601+0000 7f18e7fff640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f18d0065ca0 0x7f18d0068160 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18f015fbc0 tx=0x7f18d8008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:39.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.605+0000 7f18ecff9640 1 -- 172.21.15.26:0/2160682525 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f18e00eaf80 con 0x7f18f00b4a90 2023-12-19T18:17:39.797 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.793+0000 7f18f5a04640 1 -- 172.21.15.26:0/2160682525 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7f18f004e850 con 0x7f18f00b4a90 2023-12-19T18:17:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.809+0000 7f18ecff9640 1 -- 172.21.15.26:0/2160682525 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 30 and data pool 31 v114) v1 ==== 183+0+0 (secure 0 0 0) 0x7f18e00b8070 con 0x7f18f00b4a90 2023-12-19T18:17:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 30 and data pool 31 2023-12-19T18:17:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.817+0000 7f18f5a04640 1 -- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f18d0065ca0 msgr2=0x7f18d0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.817+0000 7f18f5a04640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f18d0065ca0 0x7f18d0068160 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18f015fbc0 tx=0x7f18d8008040 comp rx=0 tx=0).stop 2023-12-19T18:17:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.817+0000 7f18f5a04640 1 -- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18f00b4a90 msgr2=0x7f18f014c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:39.820 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.817+0000 7f18f5a04640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18f00b4a90 0x7f18f014c9f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f18e00ab000 tx=0x7f18e0018fb0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.817+0000 7f18f5a04640 1 -- 172.21.15.26:0/2160682525 shutdown_connections 2023-12-19T18:17:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.817+0000 7f18f5a04640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f18d0065ca0 0x7f18d0068160 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f18f015fbc0 tx=0x7f18d8008040 comp rx=0 tx=0).stop 2023-12-19T18:17:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.817+0000 7f18f5a04640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18f0160330 0x7f18f00c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.817+0000 7f18f5a04640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f18f00b8460 0x7f18f014fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.817+0000 7f18f5a04640 1 --2- 172.21.15.26:0/2160682525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f18f00b4a90 0x7f18f014c9f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.817+0000 7f18f5a04640 1 -- 172.21.15.26:0/2160682525 >> 172.21.15.26:0/2160682525 conn(0x7f18f0065ee0 msgr2=0x7f18f00d2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.821+0000 7f18f5a04640 1 -- 172.21.15.26:0/2160682525 shutdown_connections 2023-12-19T18:17:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:39.821+0000 7f18f5a04640 1 -- 172.21.15.26:0/2160682525 wait complete. 2023-12-19T18:17:39.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf21109640 1 Processor -- start 2023-12-19T18:17:40.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf21109640 1 -- start start 2023-12-19T18:17:40.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf21109640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf1c0ce040 0x7faf1c0d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf21109640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf1c0d29d0 con 0x7faf1c15cd40 2023-12-19T18:17:40.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf21109640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf1c0d3570 con 0x7faf1c0d3130 2023-12-19T18:17:40.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf21109640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c15d680 con 0x7faf1c0ce040 2023-12-19T18:17:40.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf1ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faf1c15cd40 0x7faf1c160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59482/0 (socket says 172.21.15.26:59482) 2023-12-19T18:17:40.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf1ad9d640 1 -- 172.21.15.26:0/730300556 learned_addr learned my addr 172.21.15.26:0/730300556 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:40.006 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf1259c640 1 --2- 172.21.15.26:0/730300556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf1c0ce040 0x7faf1c0d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf1a59c640 1 -- 172.21.15.26:0/730300556 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311913889 0 0) 0x7faf0c0012a0 con 0x7faf1c15cd40 2023-12-19T18:17:40.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf1a59c640 1 -- 172.21.15.26:0/730300556 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf08003530 con 0x7faf1c15cd40 2023-12-19T18:17:40.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf1a59c640 1 -- 172.21.15.26:0/730300556 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050291814 0 0) 0x7faf1c0d3570 con 0x7faf1c0d3130 2023-12-19T18:17:40.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf1a59c640 1 -- 172.21.15.26:0/730300556 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf0c0012a0 con 0x7faf1c0d3130 2023-12-19T18:17:40.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf1259c640 1 -- 172.21.15.26:0/730300556 >> v1:172.21.15.160:6789/0 conn(0x7faf1c0d3130 legacy=0x7faf1c0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf1259c640 1 -- 172.21.15.26:0/730300556 >> v1:172.21.15.26:6789/0 conn(0x7faf1c15cd40 legacy=0x7faf1c160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.001+0000 7faf1259c640 1 -- 172.21.15.26:0/730300556 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1c161750 con 0x7faf1c0ce040 2023-12-19T18:17:40.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.005+0000 7faf1259c640 1 --2- 172.21.15.26:0/730300556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf1c0ce040 0x7faf1c0d2bf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7faf04009dc0 tx=0x7faf0401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4941254302e7601b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:40.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.005+0000 7faf1a59c640 1 -- 172.21.15.26:0/730300556 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf040a5a10 con 0x7faf1c0ce040 2023-12-19T18:17:40.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.005+0000 7faf1a59c640 1 -- 172.21.15.26:0/730300556 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf040a5b70 con 0x7faf1c0ce040 2023-12-19T18:17:40.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.005+0000 7faf1a59c640 1 -- 172.21.15.26:0/730300556 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf040ad440 con 0x7faf1c0ce040 2023-12-19T18:17:40.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.005+0000 7faf21109640 1 -- 172.21.15.26:0/730300556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf1c0ce040 msgr2=0x7faf1c0d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.005+0000 7faf21109640 1 --2- 172.21.15.26:0/730300556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf1c0ce040 0x7faf1c0d2bf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7faf04009dc0 tx=0x7faf0401c950 comp rx=0 tx=0).stop 2023-12-19T18:17:40.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.005+0000 7faf21109640 1 -- 172.21.15.26:0/730300556 shutdown_connections 2023-12-19T18:17:40.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.005+0000 7faf21109640 1 --2- 172.21.15.26:0/730300556 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf1c0ce040 0x7faf1c0d2bf0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.005+0000 7faf21109640 1 -- 172.21.15.26:0/730300556 >> 172.21.15.26:0/730300556 conn(0x7faf1c06a120 msgr2=0x7faf1c06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:40.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.005+0000 7faf21109640 1 -- 172.21.15.26:0/730300556 shutdown_connections 2023-12-19T18:17:40.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.005+0000 7faf21109640 1 -- 172.21.15.26:0/730300556 wait complete. 2023-12-19T18:17:40.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf21109640 1 Processor -- start 2023-12-19T18:17:40.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf21109640 1 -- start start 2023-12-19T18:17:40.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf21109640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c0ce040 0x7faf1c0b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf21109640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf1c0d3130 0x7faf1c150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf21109640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf1c15cd40 0x7faf1c150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf21109640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c0c7d30 con 0x7faf1c0ce040 2023-12-19T18:17:40.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf21109640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c0cb5e0 con 0x7faf1c15cd40 2023-12-19T18:17:40.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf21109640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c0cad40 con 0x7faf1c0d3130 2023-12-19T18:17:40.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf1ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c0ce040 0x7faf1c0b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf1ad9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c0ce040 0x7faf1c0b5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36636/0 (socket says 172.21.15.26:36636) 2023-12-19T18:17:40.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf1ad9d640 1 -- 172.21.15.26:0/2919587106 learned_addr learned my addr 172.21.15.26:0/2919587106 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:40.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf1259c640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf1c0d3130 0x7faf1c150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf1b59e640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf1c15cd40 0x7faf1c150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf1ad9d640 1 -- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf1c0d3130 msgr2=0x7faf1c150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf1ad9d640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf1c0d3130 0x7faf1c150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf1ad9d640 1 -- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf1c15cd40 msgr2=0x7faf1c150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf1ad9d640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf1c15cd40 0x7faf1c150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf1ad9d640 1 -- 172.21.15.26:0/2919587106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf04009a70 con 0x7faf1c0ce040 2023-12-19T18:17:40.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf1ad9d640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c0ce040 0x7faf1c0b5210 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00c680 tx=0x7faf0c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:40.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf13fff640 1 -- 172.21.15.26:0/2919587106 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c007270 con 0x7faf1c0ce040 2023-12-19T18:17:40.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf13fff640 1 -- 172.21.15.26:0/2919587106 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c0073d0 con 0x7faf1c0ce040 2023-12-19T18:17:40.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf13fff640 1 -- 172.21.15.26:0/2919587106 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0057b0 con 0x7faf1c0ce040 2023-12-19T18:17:40.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf21109640 1 -- 172.21.15.26:0/2919587106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1c0cb020 con 0x7faf1c0ce040 2023-12-19T18:17:40.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.009+0000 7faf21109640 1 -- 172.21.15.26:0/2919587106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1c0ca960 con 0x7faf1c0ce040 2023-12-19T18:17:40.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.013+0000 7faf21109640 1 -- 172.21.15.26:0/2919587106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf1c0c7b20 con 0x7faf1c0ce040 2023-12-19T18:17:40.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.013+0000 7faf13fff640 1 -- 172.21.15.26:0/2919587106 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7faf0c005910 con 0x7faf1c0ce040 2023-12-19T18:17:40.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.013+0000 7faf13fff640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faefc065ca0 0x7faefc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.013+0000 7faf13fff640 1 -- 172.21.15.26:0/2919587106 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7333+0+0 (secure 0 0 0) 0x7faf0c083060 con 0x7faf1c0ce040 2023-12-19T18:17:40.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.013+0000 7faf1259c640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faefc065ca0 0x7faefc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.013+0000 7faf1259c640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faefc065ca0 0x7faefc068160 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf040af000 tx=0x7faf040a7070 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:40.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.017+0000 7faf13fff640 1 -- 172.21.15.26:0/2919587106 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faf0c050110 con 0x7faf1c0ce040 2023-12-19T18:17:40.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.177+0000 7faf21109640 1 -- 172.21.15.26:0/2919587106 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf1c0702a0 con 0x7faf1c0ce040 2023-12-19T18:17:40.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.181+0000 7faf13fff640 1 -- 172.21.15.26:0/2919587106 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v236) v1 ==== 74+0+20938 (secure 0 0 0) 0x7faf0c017020 con 0x7faf1c0ce040 2023-12-19T18:17:40.184 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:40.184 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":236,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:39.801144+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:25.867829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:26.330514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:17:37.484211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:17:38.799276+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:40.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.181+0000 7faf21109640 1 -- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faefc065ca0 msgr2=0x7faefc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.181+0000 7faf21109640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faefc065ca0 0x7faefc068160 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faf040af000 tx=0x7faf040a7070 comp rx=0 tx=0).stop 2023-12-19T18:17:40.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.181+0000 7faf21109640 1 -- 172.21.15.26:0/2919587106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c0ce040 msgr2=0x7faf1c0b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.181+0000 7faf21109640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c0ce040 0x7faf1c0b5210 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00c680 tx=0x7faf0c00cb40 comp rx=0 tx=0).stop 2023-12-19T18:17:40.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.181+0000 7faf21109640 1 -- 172.21.15.26:0/2919587106 shutdown_connections 2023-12-19T18:17:40.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.181+0000 7faf21109640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faefc065ca0 0x7faefc068160 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.181+0000 7faf21109640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf1c15cd40 0x7faf1c150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.181+0000 7faf21109640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf1c0d3130 0x7faf1c150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.181+0000 7faf21109640 1 --2- 172.21.15.26:0/2919587106 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf1c0ce040 0x7faf1c0b5210 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.181+0000 7faf21109640 1 -- 172.21.15.26:0/2919587106 >> 172.21.15.26:0/2919587106 conn(0x7faf1c06a120 msgr2=0x7faf1c160ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:40.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.181+0000 7faf21109640 1 -- 172.21.15.26:0/2919587106 shutdown_connections 2023-12-19T18:17:40.186 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.181+0000 7faf21109640 1 -- 172.21.15.26:0/2919587106 wait complete. 2023-12-19T18:17:40.206 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:40.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.345+0000 7f4051339640 1 Processor -- start 2023-12-19T18:17:40.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.345+0000 7f4051339640 1 -- start start 2023-12-19T18:17:40.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.345+0000 7f4051339640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f404c155030 0x7f404c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.345+0000 7f4051339640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f404c15f370 0x7f404c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.345+0000 7f4051339640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c06c970 con 0x7f404c0ce740 2023-12-19T18:17:40.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.345+0000 7f4051339640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404c15b1f0 con 0x7f404c155030 2023-12-19T18:17:40.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.345+0000 7f4051339640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404c15b4d0 con 0x7f404c15f370 2023-12-19T18:17:40.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.345+0000 7f404affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f404c155030 0x7f404c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.345+0000 7f404affd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f404c155030 0x7f404c15bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36648/0 (socket says 172.21.15.26:36648) 2023-12-19T18:17:40.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.345+0000 7f404affd640 1 -- 172.21.15.26:0/2114870109 learned_addr learned my addr 172.21.15.26:0/2114870109 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:40.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.345+0000 7f404b7fe640 1 --1- 172.21.15.26:0/2114870109 >> v1:172.21.15.26:6789/0 conn(0x7f404c0ce740 0x7f404c0ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59494/0 (socket says 172.21.15.26:59494) 2023-12-19T18:17:40.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.349+0000 7f4043fff640 1 --2- 172.21.15.26:0/2114870109 >> v2:172.21.15.160:3300/0 conn(0x7f404c15f370 0x7f404c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.349+0000 7f404affd640 1 -- 172.21.15.26:0/2114870109 >> v2:172.21.15.160:3300/0 conn(0x7f404c15f370 msgr2=0x7f404c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:40.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.349+0000 7f404affd640 1 --2- 172.21.15.26:0/2114870109 >> v2:172.21.15.160:3300/0 conn(0x7f404c15f370 0x7f404c06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.349+0000 7f404affd640 1 -- 172.21.15.26:0/2114870109 >> v1:172.21.15.26:6789/0 conn(0x7f404c0ce740 legacy=0x7f404c0ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.349+0000 7f404affd640 1 -- 172.21.15.26:0/2114870109 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c15b7b0 con 0x7f404c155030 2023-12-19T18:17:40.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.349+0000 7f404affd640 1 --2- 172.21.15.26:0/2114870109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c155030 0x7f404c15bcd0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f404c0b1780 tx=0x7f403c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce31509bd6848516 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:40.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.349+0000 7f404a7fc640 1 -- 172.21.15.26:0/2114870109 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c0ab620 con 0x7f404c155030 2023-12-19T18:17:40.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.349+0000 7f404a7fc640 1 -- 172.21.15.26:0/2114870109 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c0ab780 con 0x7f404c155030 2023-12-19T18:17:40.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.349+0000 7f404a7fc640 1 -- 172.21.15.26:0/2114870109 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c0abb10 con 0x7f404c155030 2023-12-19T18:17:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.349+0000 7f4051339640 1 -- 172.21.15.26:0/2114870109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c155030 msgr2=0x7f404c15bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.349+0000 7f4051339640 1 --2- 172.21.15.26:0/2114870109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c155030 0x7f404c15bcd0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f404c0b1780 tx=0x7f403c01c500 comp rx=0 tx=0).stop 2023-12-19T18:17:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 -- 172.21.15.26:0/2114870109 shutdown_connections 2023-12-19T18:17:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 --2- 172.21.15.26:0/2114870109 >> v2:172.21.15.160:3300/0 conn(0x7f404c15f370 0x7f404c06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 --2- 172.21.15.26:0/2114870109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c155030 0x7f404c15bcd0 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 -- 172.21.15.26:0/2114870109 >> 172.21.15.26:0/2114870109 conn(0x7f404c0658e0 msgr2=0x7f404c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:40.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 -- 172.21.15.26:0/2114870109 shutdown_connections 2023-12-19T18:17:40.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 -- 172.21.15.26:0/2114870109 wait complete. 2023-12-19T18:17:40.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 Processor -- start 2023-12-19T18:17:40.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 -- start start 2023-12-19T18:17:40.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c0ce740 0x7f404c15d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f404c155030 0x7f404c1479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f404c15f370 0x7f404c147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c0ad930 con 0x7f404c0ce740 2023-12-19T18:17:40.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c14c340 con 0x7f404c15f370 2023-12-19T18:17:40.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4051339640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404c14c620 con 0x7f404c155030 2023-12-19T18:17:40.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f404affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c0ce740 0x7f404c15d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f404affd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c0ce740 0x7f404c15d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36662/0 (socket says 172.21.15.26:36662) 2023-12-19T18:17:40.363 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f404affd640 1 -- 172.21.15.26:0/1814138515 learned_addr learned my addr 172.21.15.26:0/1814138515 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4043fff640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f404c155030 0x7f404c1479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f404b7fe640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f404c15f370 0x7f404c147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f404affd640 1 -- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f404c155030 msgr2=0x7f404c1479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f404affd640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f404c155030 0x7f404c1479b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f404affd640 1 -- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f404c15f370 msgr2=0x7f404c147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f404affd640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f404c15f370 0x7f404c147ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f404affd640 1 -- 172.21.15.26:0/1814138515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0094d0 con 0x7f404c0ce740 2023-12-19T18:17:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f404b7fe640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f404c15f370 0x7f404c147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f4043fff640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f404c155030 0x7f404c1479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.353+0000 7f404affd640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c0ce740 0x7f404c15d330 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f403c0ad040 tx=0x7f403c01ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.357+0000 7f40437fe640 1 -- 172.21.15.26:0/1814138515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c0be3f0 con 0x7f404c0ce740 2023-12-19T18:17:40.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.357+0000 7f40437fe640 1 -- 172.21.15.26:0/1814138515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c0be550 con 0x7f404c0ce740 2023-12-19T18:17:40.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.357+0000 7f40437fe640 1 -- 172.21.15.26:0/1814138515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c0be8e0 con 0x7f404c0ce740 2023-12-19T18:17:40.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.357+0000 7f4051339640 1 -- 172.21.15.26:0/1814138515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404c0cf0e0 con 0x7f404c0ce740 2023-12-19T18:17:40.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.357+0000 7f4051339640 1 -- 172.21.15.26:0/1814138515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c14cbe0 con 0x7f404c0ce740 2023-12-19T18:17:40.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.357+0000 7f4051339640 1 -- 172.21.15.26:0/1814138515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404c074270 con 0x7f404c0ce740 2023-12-19T18:17:40.366 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.357+0000 7f40437fe640 1 -- 172.21.15.26:0/1814138515 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f403c0b43f0 con 0x7f404c0ce740 2023-12-19T18:17:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.357+0000 7f40437fe640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f402c065cd0 0x7f402c068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.357+0000 7f40437fe640 1 -- 172.21.15.26:0/1814138515 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f403c11dfb0 con 0x7f404c0ce740 2023-12-19T18:17:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.357+0000 7f4043fff640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f402c065cd0 0x7f402c068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.361+0000 7f4043fff640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f402c065cd0 0x7f402c068190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f404c0acc30 tx=0x7f4034008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:40.368 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.361+0000 7f40437fe640 1 -- 172.21.15.26:0/1814138515 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f403c0eaf80 con 0x7f404c0ce740 2023-12-19T18:17:40.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f4051339640 1 -- 172.21.15.26:0/1814138515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f404c0ccc30 con 0x7f404c0ce740 2023-12-19T18:17:40.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f40437fe640 1 -- 172.21.15.26:0/1814138515 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v236) v1 ==== 74+0+20938 (secure 0 0 0) 0x7f403c0c4020 con 0x7f404c0ce740 2023-12-19T18:17:40.525 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:40.525 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":236,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:39.801144+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:25.867829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:26.330514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:17:37.484211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:17:38.799276+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f4051339640 1 -- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f402c065cd0 msgr2=0x7f402c068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f4051339640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f402c065cd0 0x7f402c068190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f404c0acc30 tx=0x7f4034008040 comp rx=0 tx=0).stop 2023-12-19T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f4051339640 1 -- 172.21.15.26:0/1814138515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c0ce740 msgr2=0x7f404c15d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f4051339640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c0ce740 0x7f404c15d330 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f403c0ad040 tx=0x7f403c01ad40 comp rx=0 tx=0).stop 2023-12-19T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f4051339640 1 -- 172.21.15.26:0/1814138515 shutdown_connections 2023-12-19T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f4051339640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f402c065cd0 0x7f402c068190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f4051339640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f404c15f370 0x7f404c147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f4051339640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f404c155030 0x7f404c1479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f4051339640 1 --2- 172.21.15.26:0/1814138515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f404c0ce740 0x7f404c15d330 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f4051339640 1 -- 172.21.15.26:0/1814138515 >> 172.21.15.26:0/1814138515 conn(0x7f404c0658e0 msgr2=0x7f404c15bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:40.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f4051339640 1 -- 172.21.15.26:0/1814138515 shutdown_connections 2023-12-19T18:17:40.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.521+0000 7f4051339640 1 -- 172.21.15.26:0/1814138515 wait complete. 2023-12-19T18:17:40.546 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-12-19T18:17:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.689+0000 7fbab89ce640 1 Processor -- start 2023-12-19T18:17:40.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.689+0000 7fbab89ce640 1 -- start start 2023-12-19T18:17:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.689+0000 7fbab89ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbab415ada0 0x7fbab40706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.689+0000 7fbab89ce640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab4070d90 con 0x7fbab415a9c0 2023-12-19T18:17:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.689+0000 7fbab89ce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab4071070 con 0x7fbab406ac80 2023-12-19T18:17:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.689+0000 7fbab89ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40cc270 con 0x7fbab415ada0 2023-12-19T18:17:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.689+0000 7fbab259c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbab415a9c0 0x7fbab40b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59500/0 (socket says 172.21.15.26:59500) 2023-12-19T18:17:40.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.689+0000 7fbab259c640 1 -- 172.21.15.26:0/2130378987 learned_addr learned my addr 172.21.15.26:0/2130378987 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.689+0000 7fbaabfff640 1 --2- 172.21.15.26:0/2130378987 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbab415ada0 0x7fbab40706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.689+0000 7fbab1d9b640 1 -- 172.21.15.26:0/2130378987 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890927723 0 0) 0x7fbab4070d90 con 0x7fbab415a9c0 2023-12-19T18:17:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.689+0000 7fbab1d9b640 1 -- 172.21.15.26:0/2130378987 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbaa0003530 con 0x7fbab415a9c0 2023-12-19T18:17:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.693+0000 7fbab1d9b640 1 -- 172.21.15.26:0/2130378987 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813182973 0 0) 0x7fbab4071070 con 0x7fbab406ac80 2023-12-19T18:17:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.693+0000 7fbab1d9b640 1 -- 172.21.15.26:0/2130378987 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbab4070d90 con 0x7fbab406ac80 2023-12-19T18:17:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.693+0000 7fbab1d9b640 1 -- 172.21.15.26:0/2130378987 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1615369113 0 0) 0x7fbaa0003530 con 0x7fbab415a9c0 2023-12-19T18:17:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.693+0000 7fbaabfff640 1 -- 172.21.15.26:0/2130378987 >> v1:172.21.15.160:6789/0 conn(0x7fbab406ac80 legacy=0x7fbab40b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.693+0000 7fbaabfff640 1 -- 172.21.15.26:0/2130378987 >> v1:172.21.15.26:6789/0 conn(0x7fbab415a9c0 legacy=0x7fbab40b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.693+0000 7fbaabfff640 1 -- 172.21.15.26:0/2130378987 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab4071350 con 0x7fbab415ada0 2023-12-19T18:17:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.693+0000 7fbaabfff640 1 --2- 172.21.15.26:0/2130378987 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbab415ada0 0x7fbab40706d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0097f0 tx=0x7fba9c01c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b34e4c52e5cf837 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.693+0000 7fbab1d9b640 1 -- 172.21.15.26:0/2130378987 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4001680 con 0x7fbab415ada0 2023-12-19T18:17:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.693+0000 7fbab1d9b640 1 -- 172.21.15.26:0/2130378987 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c0a7050 con 0x7fbab415ada0 2023-12-19T18:17:40.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.693+0000 7fbab1d9b640 1 -- 172.21.15.26:0/2130378987 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0ab620 con 0x7fbab415ada0 2023-12-19T18:17:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 -- 172.21.15.26:0/2130378987 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbab415ada0 msgr2=0x7fbab40706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 --2- 172.21.15.26:0/2130378987 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbab415ada0 0x7fbab40706d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0097f0 tx=0x7fba9c01c320 comp rx=0 tx=0).stop 2023-12-19T18:17:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 -- 172.21.15.26:0/2130378987 shutdown_connections 2023-12-19T18:17:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 --2- 172.21.15.26:0/2130378987 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbab415ada0 0x7fbab40706d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 -- 172.21.15.26:0/2130378987 >> 172.21.15.26:0/2130378987 conn(0x7fbab40ce730 msgr2=0x7fbab40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 -- 172.21.15.26:0/2130378987 shutdown_connections 2023-12-19T18:17:40.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 -- 172.21.15.26:0/2130378987 wait complete. 2023-12-19T18:17:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 Processor -- start 2023-12-19T18:17:40.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 -- start start 2023-12-19T18:17:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbab406ac80 0x7fbab40ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab415a9c0 0x7fbab40c1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbab415ada0 0x7fbab4148010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40c9950 con 0x7fbab415a9c0 2023-12-19T18:17:40.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40c8cf0 con 0x7fbab415ada0 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab89ce640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab40c9d70 con 0x7fbab406ac80 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab415a9c0 0x7fbab40c1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbaabfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab415a9c0 0x7fbab40c1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36678/0 (socket says 172.21.15.26:36678) 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbaabfff640 1 -- 172.21.15.26:0/2869105963 learned_addr learned my addr 172.21.15.26:0/2869105963 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab259c640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbab406ac80 0x7fbab40ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbab2d9d640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbab415ada0 0x7fbab4148010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbaabfff640 1 -- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbab406ac80 msgr2=0x7fbab40ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbaabfff640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbab406ac80 0x7fbab40ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbaabfff640 1 -- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbab415ada0 msgr2=0x7fbab4148010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbaabfff640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbab415ada0 0x7fbab4148010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbaabfff640 1 -- 172.21.15.26:0/2869105963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9c009420 con 0x7fbab415a9c0 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.697+0000 7fbaabfff640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab415a9c0 0x7fbab40c1630 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0097f0 tx=0x7fba9c0190c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.701+0000 7fbaaaffd640 1 -- 172.21.15.26:0/2869105963 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0abce0 con 0x7fbab415a9c0 2023-12-19T18:17:40.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.701+0000 7fbaaaffd640 1 -- 172.21.15.26:0/2869105963 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9c0abe40 con 0x7fbab415a9c0 2023-12-19T18:17:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.701+0000 7fbab89ce640 1 -- 172.21.15.26:0/2869105963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab40c8fb0 con 0x7fbab415a9c0 2023-12-19T18:17:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.701+0000 7fbaaaffd640 1 -- 172.21.15.26:0/2869105963 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c004220 con 0x7fbab415a9c0 2023-12-19T18:17:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.701+0000 7fbab89ce640 1 -- 172.21.15.26:0/2869105963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab40c9570 con 0x7fbab415a9c0 2023-12-19T18:17:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.701+0000 7fbab89ce640 1 -- 172.21.15.26:0/2869105963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab40c9a90 con 0x7fbab415a9c0 2023-12-19T18:17:40.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.701+0000 7fbaaaffd640 1 -- 172.21.15.26:0/2869105963 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fba9c0b2040 con 0x7fbab415a9c0 2023-12-19T18:17:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.701+0000 7fbaaaffd640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fba94065ca0 0x7fba94068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.701+0000 7fbaaaffd640 1 -- 172.21.15.26:0/2869105963 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fba9c11daa0 con 0x7fbab415a9c0 2023-12-19T18:17:40.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.701+0000 7fbab259c640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fba94065ca0 0x7fba94068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:40.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.705+0000 7fbab259c640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fba94065ca0 0x7fba94068160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000f70 tx=0x7fbaa4009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:40.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.705+0000 7fbaaaffd640 1 -- 172.21.15.26:0/2869105963 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fba9c0eab80 con 0x7fbab415a9c0 2023-12-19T18:17:40.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:40.897+0000 7fbab89ce640 1 -- 172.21.15.26:0/2869105963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbab40c9a90 con 0x7fbab415a9c0 2023-12-19T18:17:41.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:41.829+0000 7fbaaaffd640 1 -- 172.21.15.26:0/2869105963 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v117) v1 ==== 121+0+0 (secure 0 0 0) 0x7fbab40c9a90 con 0x7fbab415a9c0 2023-12-19T18:17:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:41.833+0000 7fbab89ce640 1 -- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fba94065ca0 msgr2=0x7fba94068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:41.833+0000 7fbab89ce640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fba94065ca0 0x7fba94068160 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4000f70 tx=0x7fbaa4009040 comp rx=0 tx=0).stop 2023-12-19T18:17:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:41.833+0000 7fbab89ce640 1 -- 172.21.15.26:0/2869105963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab415a9c0 msgr2=0x7fbab40c1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:41.833+0000 7fbab89ce640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab415a9c0 0x7fbab40c1630 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0097f0 tx=0x7fba9c0190c0 comp rx=0 tx=0).stop 2023-12-19T18:17:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:41.833+0000 7fbab89ce640 1 -- 172.21.15.26:0/2869105963 shutdown_connections 2023-12-19T18:17:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:41.833+0000 7fbab89ce640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fba94065ca0 0x7fba94068160 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:41.833+0000 7fbab89ce640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbab415ada0 0x7fbab4148010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:41.833+0000 7fbab89ce640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbab415a9c0 0x7fbab40c1630 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:41.833+0000 7fbab89ce640 1 --2- 172.21.15.26:0/2869105963 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbab406ac80 0x7fbab40ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:41.833+0000 7fbab89ce640 1 -- 172.21.15.26:0/2869105963 >> 172.21.15.26:0/2869105963 conn(0x7fbab40ce730 msgr2=0x7fbab40ae160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:41.833+0000 7fbab89ce640 1 -- 172.21.15.26:0/2869105963 shutdown_connections 2023-12-19T18:17:41.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:41.833+0000 7fbab89ce640 1 -- 172.21.15.26:0/2869105963 wait complete. 2023-12-19T18:17:41.859 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:17:41.860 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-12-19T18:17:42.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7787bbc640 1 Processor -- start 2023-12-19T18:17:42.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7787bbc640 1 -- start start 2023-12-19T18:17:42.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7787bbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7780159c00 0x7f778015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:42.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7787bbc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778006fb60 con 0x7f778006f6c0 2023-12-19T18:17:42.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7787bbc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778006fe40 con 0x7f7780159820 2023-12-19T18:17:42.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7787bbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778015e470 con 0x7f7780159c00 2023-12-19T18:17:42.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7785157640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f778006f6c0 0x7f7780070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59508/0 (socket says 172.21.15.26:59508) 2023-12-19T18:17:42.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7785157640 1 -- 172.21.15.26:0/2453543098 learned_addr learned my addr 172.21.15.26:0/2453543098 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:42.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7786159640 1 --2- 172.21.15.26:0/2453543098 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7780159c00 0x7f778015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:42.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7784956640 1 -- 172.21.15.26:0/2453543098 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209504225 0 0) 0x7f778006fb60 con 0x7f778006f6c0 2023-12-19T18:17:42.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7784956640 1 -- 172.21.15.26:0/2453543098 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f776c005740 con 0x7f778006f6c0 2023-12-19T18:17:42.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7784956640 1 -- 172.21.15.26:0/2453543098 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882483637 0 0) 0x7f778006fe40 con 0x7f7780159820 2023-12-19T18:17:42.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7784956640 1 -- 172.21.15.26:0/2453543098 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f778006fb60 con 0x7f7780159820 2023-12-19T18:17:42.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7786159640 1 -- 172.21.15.26:0/2453543098 >> v1:172.21.15.160:6789/0 conn(0x7f7780159820 legacy=0x7f77800c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:42.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7786159640 1 -- 172.21.15.26:0/2453543098 >> v1:172.21.15.26:6789/0 conn(0x7f778006f6c0 legacy=0x7f7780070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:42.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.009+0000 7f7786159640 1 -- 172.21.15.26:0/2453543098 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7780070120 con 0x7f7780159c00 2023-12-19T18:17:42.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.013+0000 7f7786159640 1 --2- 172.21.15.26:0/2453543098 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7780159c00 0x7f778015df30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7778009dc0 tx=0x7f777801c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=279ea9a1a4ae2b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:42.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.013+0000 7f7784956640 1 -- 172.21.15.26:0/2453543098 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77780a5a00 con 0x7f7780159c00 2023-12-19T18:17:42.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.013+0000 7f7784956640 1 -- 172.21.15.26:0/2453543098 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77780a5b60 con 0x7f7780159c00 2023-12-19T18:17:42.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.013+0000 7f7784956640 1 -- 172.21.15.26:0/2453543098 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77780ad440 con 0x7f7780159c00 2023-12-19T18:17:42.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.013+0000 7f7787bbc640 1 -- 172.21.15.26:0/2453543098 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7780159c00 msgr2=0x7f778015df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:42.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.013+0000 7f7787bbc640 1 --2- 172.21.15.26:0/2453543098 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7780159c00 0x7f778015df30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7778009dc0 tx=0x7f777801c940 comp rx=0 tx=0).stop 2023-12-19T18:17:42.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.013+0000 7f7787bbc640 1 -- 172.21.15.26:0/2453543098 shutdown_connections 2023-12-19T18:17:42.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.013+0000 7f7787bbc640 1 --2- 172.21.15.26:0/2453543098 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7780159c00 0x7f778015df30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:42.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.013+0000 7f7787bbc640 1 -- 172.21.15.26:0/2453543098 >> 172.21.15.26:0/2453543098 conn(0x7f77800ce730 msgr2=0x7f77800ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:42.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.013+0000 7f7787bbc640 1 -- 172.21.15.26:0/2453543098 shutdown_connections 2023-12-19T18:17:42.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.013+0000 7f7787bbc640 1 -- 172.21.15.26:0/2453543098 wait complete. 2023-12-19T18:17:42.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7787bbc640 1 Processor -- start 2023-12-19T18:17:42.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7787bbc640 1 -- start start 2023-12-19T18:17:42.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7787bbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f778006f6c0 0x7f77800b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:42.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7787bbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7780159820 0x7f7780153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7787bbc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7780159c00 0x7f77800b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7787bbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77800b1b30 con 0x7f7780159820 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7787bbc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77800b1e10 con 0x7f7780159c00 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7787bbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77800b20f0 con 0x7f778006f6c0 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7785157640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7780159820 0x7f7780153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7785157640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7780159820 0x7f7780153bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36686/0 (socket says 172.21.15.26:36686) 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7785157640 1 -- 172.21.15.26:0/1277438080 learned_addr learned my addr 172.21.15.26:0/1277438080 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7786159640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7780159c00 0x7f77800b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7785958640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f778006f6c0 0x7f77800b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7785157640 1 -- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f778006f6c0 msgr2=0x7f77800b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7785157640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f778006f6c0 0x7f77800b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7785157640 1 -- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7780159c00 msgr2=0x7f77800b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7785157640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7780159c00 0x7f77800b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:42.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7785157640 1 -- 172.21.15.26:0/1277438080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7778009a70 con 0x7f7780159820 2023-12-19T18:17:42.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7785157640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7780159820 0x7f7780153bf0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f7770005890 tx=0x7f777000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:42.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f77767fc640 1 -- 172.21.15.26:0/1277438080 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770005cf0 con 0x7f7780159820 2023-12-19T18:17:42.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f77767fc640 1 -- 172.21.15.26:0/1277438080 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7770005e50 con 0x7f7780159820 2023-12-19T18:17:42.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f77767fc640 1 -- 172.21.15.26:0/1277438080 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770007290 con 0x7f7780159820 2023-12-19T18:17:42.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7787bbc640 1 -- 172.21.15.26:0/1277438080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778006b880 con 0x7f7780159820 2023-12-19T18:17:42.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.017+0000 7f7787bbc640 1 -- 172.21.15.26:0/1277438080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77800b26b0 con 0x7f7780159820 2023-12-19T18:17:42.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.021+0000 7f7787bbc640 1 -- 172.21.15.26:0/1277438080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7780074270 con 0x7f7780159820 2023-12-19T18:17:42.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.021+0000 7f77767fc640 1 -- 172.21.15.26:0/1277438080 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f77700073f0 con 0x7f7780159820 2023-12-19T18:17:42.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.021+0000 7f77767fc640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7764065ca0 0x7f7764068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:42.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.021+0000 7f77767fc640 1 -- 172.21.15.26:0/1277438080 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f77700832e0 con 0x7f7780159820 2023-12-19T18:17:42.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.021+0000 7f7785958640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7764065ca0 0x7f7764068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:42.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.021+0000 7f7785958640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7764065ca0 0x7f7764068160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f777c000f70 tx=0x7f777c009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:42.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.025+0000 7f77767fc640 1 -- 172.21.15.26:0/1277438080 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7770050240 con 0x7f7780159820 2023-12-19T18:17:42.217 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.213+0000 7f7787bbc640 1 -- 172.21.15.26:0/1277438080 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f77800b46b0 con 0x7f7780159820 2023-12-19T18:17:42.540 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:17:42.545 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:17:42.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.833+0000 7f77767fc640 1 -- 172.21.15.26:0/1277438080 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v118) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7770053860 con 0x7f7780159820 2023-12-19T18:17:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.833+0000 7f7787bbc640 1 -- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7764065ca0 msgr2=0x7f7764068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.833+0000 7f7787bbc640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7764065ca0 0x7f7764068160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f777c000f70 tx=0x7f777c009040 comp rx=0 tx=0).stop 2023-12-19T18:17:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.833+0000 7f7787bbc640 1 -- 172.21.15.26:0/1277438080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7780159820 msgr2=0x7f7780153bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.833+0000 7f7787bbc640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7780159820 0x7f7780153bf0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f7770005890 tx=0x7f777000c980 comp rx=0 tx=0).stop 2023-12-19T18:17:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.833+0000 7f7787bbc640 1 -- 172.21.15.26:0/1277438080 shutdown_connections 2023-12-19T18:17:42.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.833+0000 7f7787bbc640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7764065ca0 0x7f7764068160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.833+0000 7f7787bbc640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7780159c00 0x7f77800b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.833+0000 7f7787bbc640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7780159820 0x7f7780153bf0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.833+0000 7f7787bbc640 1 --2- 172.21.15.26:0/1277438080 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f778006f6c0 0x7f77800b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.833+0000 7f7787bbc640 1 -- 172.21.15.26:0/1277438080 >> 172.21.15.26:0/1277438080 conn(0x7f77800ce730 msgr2=0x7f77801544b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.833+0000 7f7787bbc640 1 -- 172.21.15.26:0/1277438080 shutdown_connections 2023-12-19T18:17:42.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:42.833+0000 7f7787bbc640 1 -- 172.21.15.26:0/1277438080 wait complete. 2023-12-19T18:17:42.861 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-12-19T18:17:42.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:43.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.005+0000 7fe895f18640 1 Processor -- start 2023-12-19T18:17:43.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.005+0000 7fe895f18640 1 -- start start 2023-12-19T18:17:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe895f18640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe89015f370 0x7fe89015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe895f18640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe8900ce740 0x7fe8900cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe895f18640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89006c970 con 0x7fe890155030 2023-12-19T18:17:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe895f18640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89006c790 con 0x7fe8900ce740 2023-12-19T18:17:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe895f18640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8900cf130 con 0x7fe89015f370 2023-12-19T18:17:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe88f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe890155030 0x7fe89006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59518/0 (socket says 172.21.15.26:59518) 2023-12-19T18:17:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe88f7fe640 1 -- 172.21.15.26:0/486846009 learned_addr learned my addr 172.21.15.26:0/486846009 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe88ffff640 1 --2- 172.21.15.26:0/486846009 >> v2:172.21.15.26:3300/0 conn(0x7fe8900ce740 0x7fe8900cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe88effd640 1 --2- 172.21.15.26:0/486846009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe89015f370 0x7fe89015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe88e7fc640 1 -- 172.21.15.26:0/486846009 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304901774 0 0) 0x7fe89006c970 con 0x7fe890155030 2023-12-19T18:17:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe88e7fc640 1 -- 172.21.15.26:0/486846009 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe878003530 con 0x7fe890155030 2023-12-19T18:17:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe88ffff640 1 -- 172.21.15.26:0/486846009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe89015f370 msgr2=0x7fe89015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe88ffff640 1 --2- 172.21.15.26:0/486846009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe89015f370 0x7fe89015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe88ffff640 1 -- 172.21.15.26:0/486846009 >> v1:172.21.15.26:6789/0 conn(0x7fe890155030 legacy=0x7fe89006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe88ffff640 1 -- 172.21.15.26:0/486846009 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89015b1f0 con 0x7fe8900ce740 2023-12-19T18:17:43.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.009+0000 7fe88ffff640 1 --2- 172.21.15.26:0/486846009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8900ce740 0x7fe8900cebf0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fe888009d90 tx=0x7fe88801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9904c649acdb97e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:43.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe88e7fc640 1 -- 172.21.15.26:0/486846009 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8880a59e0 con 0x7fe8900ce740 2023-12-19T18:17:43.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe88e7fc640 1 -- 172.21.15.26:0/486846009 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8880a5b40 con 0x7fe8900ce740 2023-12-19T18:17:43.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe88e7fc640 1 -- 172.21.15.26:0/486846009 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8880ad440 con 0x7fe8900ce740 2023-12-19T18:17:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 -- 172.21.15.26:0/486846009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8900ce740 msgr2=0x7fe8900cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 --2- 172.21.15.26:0/486846009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8900ce740 0x7fe8900cebf0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fe888009d90 tx=0x7fe88801c920 comp rx=0 tx=0).stop 2023-12-19T18:17:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 -- 172.21.15.26:0/486846009 shutdown_connections 2023-12-19T18:17:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 --2- 172.21.15.26:0/486846009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe8900ce740 0x7fe8900cebf0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 --2- 172.21.15.26:0/486846009 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe89015f370 0x7fe89015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 -- 172.21.15.26:0/486846009 >> 172.21.15.26:0/486846009 conn(0x7fe8900658e0 msgr2=0x7fe890065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:43.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 -- 172.21.15.26:0/486846009 shutdown_connections 2023-12-19T18:17:43.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 -- 172.21.15.26:0/486846009 wait complete. 2023-12-19T18:17:43.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 Processor -- start 2023-12-19T18:17:43.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 -- start start 2023-12-19T18:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8900ce740 0x7fe8901f8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe890155030 0x7fe89014fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89015f370 0x7fe8900b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8901563e0 con 0x7fe89015f370 2023-12-19T18:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe890157170 con 0x7fe8900ce740 2023-12-19T18:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.013+0000 7fe895f18640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8901565f0 con 0x7fe890155030 2023-12-19T18:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe88ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89015f370 0x7fe8900b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe88ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89015f370 0x7fe8900b01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36696/0 (socket says 172.21.15.26:36696) 2023-12-19T18:17:43.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe88ffff640 1 -- 172.21.15.26:0/4152713312 learned_addr learned my addr 172.21.15.26:0/4152713312 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe88effd640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe890155030 0x7fe89014fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe88f7fe640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8900ce740 0x7fe8901f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe88ffff640 1 -- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe890155030 msgr2=0x7fe89014fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe88ffff640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe890155030 0x7fe89014fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe88ffff640 1 -- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8900ce740 msgr2=0x7fe8901f8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe88ffff640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8900ce740 0x7fe8901f8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe88ffff640 1 -- 172.21.15.26:0/4152713312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe888009a70 con 0x7fe89015f370 2023-12-19T18:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe88ffff640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89015f370 0x7fe8900b01d0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fe88801a550 tx=0x7fe8880196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe877fff640 1 -- 172.21.15.26:0/4152713312 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888004040 con 0x7fe89015f370 2023-12-19T18:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe877fff640 1 -- 172.21.15.26:0/4152713312 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8880041a0 con 0x7fe89015f370 2023-12-19T18:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe895f18640 1 -- 172.21.15.26:0/4152713312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8901568d0 con 0x7fe89015f370 2023-12-19T18:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe877fff640 1 -- 172.21.15.26:0/4152713312 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888004530 con 0x7fe89015f370 2023-12-19T18:17:43.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe895f18640 1 -- 172.21.15.26:0/4152713312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe890156e90 con 0x7fe89015f370 2023-12-19T18:17:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe895f18640 1 -- 172.21.15.26:0/4152713312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe890074270 con 0x7fe89015f370 2023-12-19T18:17:43.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe877fff640 1 -- 172.21.15.26:0/4152713312 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fe8880b4070 con 0x7fe89015f370 2023-12-19T18:17:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.017+0000 7fe877fff640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe87c065ca0 0x7fe87c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.021+0000 7fe877fff640 1 -- 172.21.15.26:0/4152713312 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fe88811eff0 con 0x7fe89015f370 2023-12-19T18:17:43.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.021+0000 7fe88f7fe640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe87c065ca0 0x7fe87c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.021+0000 7fe88f7fe640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe87c065ca0 0x7fe87c068160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe8800011f0 tx=0x7fe880009040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:43.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.025+0000 7fe877fff640 1 -- 172.21.15.26:0/4152713312 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe8880ec090 con 0x7fe89015f370 2023-12-19T18:17:43.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.213+0000 7fe895f18640 1 -- 172.21.15.26:0/4152713312 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8900ccc30 con 0x7fe89015f370 2023-12-19T18:17:43.219 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.217+0000 7fe877fff640 1 -- 172.21.15.26:0/4152713312 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5921 (secure 0 0 0) 0x7fe8880ef6b0 con 0x7fe89015f370 2023-12-19T18:17:43.220 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:43.220 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2727392302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2727392302},{"type":"v1","addr":"172.21.15.160:6836","nonce":2727392302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1029020739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1029020739},{"type":"v1","addr":"172.21.15.160:6837","nonce":1029020739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:27.325271+0000","modified":"2023-12-19T18:17:30.337993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11427":{"gid":11427,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2285370499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2285370499},{"type":"v1","addr":"172.21.15.26:6835","nonce":2285370499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:39.801489+0000","modified":"2023-12-19T18:17:42.831660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11424":{"gid":11424,"name":"a","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6834/3251191056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3251191056},{"type":"v1","addr":"172.21.15.26:6834","nonce":3251191056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-19T18:17:43.220 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 118 2023-12-19T18:17:43.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.217+0000 7fe895f18640 1 -- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe87c065ca0 msgr2=0x7fe87c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.217+0000 7fe895f18640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe87c065ca0 0x7fe87c068160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe8800011f0 tx=0x7fe880009040 comp rx=0 tx=0).stop 2023-12-19T18:17:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.217+0000 7fe895f18640 1 -- 172.21.15.26:0/4152713312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89015f370 msgr2=0x7fe8900b01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.217+0000 7fe895f18640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89015f370 0x7fe8900b01d0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fe88801a550 tx=0x7fe8880196c0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.217+0000 7fe895f18640 1 -- 172.21.15.26:0/4152713312 shutdown_connections 2023-12-19T18:17:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.217+0000 7fe895f18640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe87c065ca0 0x7fe87c068160 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.217+0000 7fe895f18640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe89015f370 0x7fe8900b01d0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.217+0000 7fe895f18640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe890155030 0x7fe89014fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.217+0000 7fe895f18640 1 --2- 172.21.15.26:0/4152713312 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8900ce740 0x7fe8901f8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.217+0000 7fe895f18640 1 -- 172.21.15.26:0/4152713312 >> 172.21.15.26:0/4152713312 conn(0x7fe8900658e0 msgr2=0x7fe890154470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:43.222 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.217+0000 7fe895f18640 1 -- 172.21.15.26:0/4152713312 shutdown_connections 2023-12-19T18:17:43.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.217+0000 7fe895f18640 1 -- 172.21.15.26:0/4152713312 wait complete. 2023-12-19T18:17:43.241 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:43.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab359e640 1 Processor -- start 2023-12-19T18:17:43.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab359e640 1 -- start start 2023-12-19T18:17:43.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab359e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcab40cc0a0 0x7fcab40cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab359e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcab40cdee0 0x7fcab40b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab359e640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab40adde0 con 0x7fcab4159d20 2023-12-19T18:17:43.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab359e640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab40c9c20 con 0x7fcab40cc0a0 2023-12-19T18:17:43.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab359e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab40b28d0 con 0x7fcab40cdee0 2023-12-19T18:17:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcab40cc0a0 0x7fcab40cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab1d9b640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcab4159d20 0x7fcab40cc9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59522/0 (socket says 172.21.15.26:59522) 2023-12-19T18:17:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab1d9b640 1 -- 172.21.15.26:0/4281197675 learned_addr learned my addr 172.21.15.26:0/4281197675 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab2d9d640 1 --2- 172.21.15.26:0/4281197675 >> v2:172.21.15.160:3300/0 conn(0x7fcab40cdee0 0x7fcab40b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab259c640 1 -- 172.21.15.26:0/4281197675 >> v2:172.21.15.160:3300/0 conn(0x7fcab40cdee0 msgr2=0x7fcab40b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab259c640 1 --2- 172.21.15.26:0/4281197675 >> v2:172.21.15.160:3300/0 conn(0x7fcab40cdee0 0x7fcab40b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab159a640 1 -- 172.21.15.26:0/4281197675 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068166402 0 0) 0x7fcab40adde0 con 0x7fcab4159d20 2023-12-19T18:17:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab259c640 1 -- 172.21.15.26:0/4281197675 >> v1:172.21.15.26:6789/0 conn(0x7fcab4159d20 legacy=0x7fcab40cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.385+0000 7fcab259c640 1 -- 172.21.15.26:0/4281197675 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab40adb00 con 0x7fcab40cc0a0 2023-12-19T18:17:43.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.389+0000 7fcab259c640 1 --2- 172.21.15.26:0/4281197675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab40cc0a0 0x7fcab40cc480 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fcab4071fe0 tx=0x7fcaa401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33962d5dc2a0ef63 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:43.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.389+0000 7fcab159a640 1 -- 172.21.15.26:0/4281197675 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40a5930 con 0x7fcab40cc0a0 2023-12-19T18:17:43.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.389+0000 7fcab159a640 1 -- 172.21.15.26:0/4281197675 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa40a5a90 con 0x7fcab40cc0a0 2023-12-19T18:17:43.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.389+0000 7fcab159a640 1 -- 172.21.15.26:0/4281197675 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40a5e20 con 0x7fcab40cc0a0 2023-12-19T18:17:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.389+0000 7fcab359e640 1 -- 172.21.15.26:0/4281197675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab40cc0a0 msgr2=0x7fcab40cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.389+0000 7fcab359e640 1 --2- 172.21.15.26:0/4281197675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab40cc0a0 0x7fcab40cc480 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fcab4071fe0 tx=0x7fcaa401c850 comp rx=0 tx=0).stop 2023-12-19T18:17:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.389+0000 7fcab359e640 1 -- 172.21.15.26:0/4281197675 shutdown_connections 2023-12-19T18:17:43.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.389+0000 7fcab359e640 1 --2- 172.21.15.26:0/4281197675 >> v2:172.21.15.160:3300/0 conn(0x7fcab40cdee0 0x7fcab40b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.389+0000 7fcab359e640 1 --2- 172.21.15.26:0/4281197675 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab40cc0a0 0x7fcab40cc480 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.389+0000 7fcab359e640 1 -- 172.21.15.26:0/4281197675 >> 172.21.15.26:0/4281197675 conn(0x7fcab406b0f0 msgr2=0x7fcab406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:43.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.389+0000 7fcab359e640 1 -- 172.21.15.26:0/4281197675 shutdown_connections 2023-12-19T18:17:43.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.389+0000 7fcab359e640 1 -- 172.21.15.26:0/4281197675 wait complete. 2023-12-19T18:17:43.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab359e640 1 Processor -- start 2023-12-19T18:17:43.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab359e640 1 -- start start 2023-12-19T18:17:43.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab40cc0a0 0x7fcab40c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab359e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcab40cdee0 0x7fcab4151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab359e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4159d20 0x7fcab40c2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab359e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40c3f40 con 0x7fcab40cc0a0 2023-12-19T18:17:43.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab359e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40c3a30 con 0x7fcab4159d20 2023-12-19T18:17:43.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab359e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40c56d0 con 0x7fcab40cdee0 2023-12-19T18:17:43.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab40cc0a0 0x7fcab40c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab40cc0a0 0x7fcab40c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36710/0 (socket says 172.21.15.26:36710) 2023-12-19T18:17:43.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab259c640 1 -- 172.21.15.26:0/2764549345 learned_addr learned my addr 172.21.15.26:0/2764549345 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:43.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab1d9b640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcab40cdee0 0x7fcab4151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab2d9d640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4159d20 0x7fcab40c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab259c640 1 -- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcab40cdee0 msgr2=0x7fcab4151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab259c640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcab40cdee0 0x7fcab4151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab259c640 1 -- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4159d20 msgr2=0x7fcab40c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab259c640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4159d20 0x7fcab40c2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab259c640 1 -- 172.21.15.26:0/2764549345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa4009a70 con 0x7fcab40cc0a0 2023-12-19T18:17:43.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab259c640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab40cc0a0 0x7fcab40c4af0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4019460 tx=0x7fcaa40195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:43.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcaa2ffd640 1 -- 172.21.15.26:0/2764549345 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40ade10 con 0x7fcab40cc0a0 2023-12-19T18:17:43.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcaa2ffd640 1 -- 172.21.15.26:0/2764549345 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa4009450 con 0x7fcab40cc0a0 2023-12-19T18:17:43.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcaa2ffd640 1 -- 172.21.15.26:0/2764549345 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40b7650 con 0x7fcab40cc0a0 2023-12-19T18:17:43.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab359e640 1 -- 172.21.15.26:0/2764549345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab40c3190 con 0x7fcab40cc0a0 2023-12-19T18:17:43.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab359e640 1 -- 172.21.15.26:0/2764549345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab40c3750 con 0x7fcab40cc0a0 2023-12-19T18:17:43.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.393+0000 7fcab359e640 1 -- 172.21.15.26:0/2764549345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab40c5890 con 0x7fcab40cc0a0 2023-12-19T18:17:43.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.397+0000 7fcaa2ffd640 1 -- 172.21.15.26:0/2764549345 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fcaa40b4050 con 0x7fcab40cc0a0 2023-12-19T18:17:43.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.397+0000 7fcaa2ffd640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fca94065ca0 0x7fca94068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.397+0000 7fcaa2ffd640 1 -- 172.21.15.26:0/2764549345 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fcaa411f1d0 con 0x7fcab40cc0a0 2023-12-19T18:17:43.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.397+0000 7fcab1d9b640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fca94065ca0 0x7fca94068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.397+0000 7fcab1d9b640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fca94065ca0 0x7fca94068160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcab4156a60 tx=0x7fcaa8007df0 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:43.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.401+0000 7fcaa2ffd640 1 -- 172.21.15.26:0/2764549345 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcaa40ec270 con 0x7fcab40cc0a0 2023-12-19T18:17:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcab359e640 1 -- 172.21.15.26:0/2764549345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcab4067400 con 0x7fcab40cc0a0 2023-12-19T18:17:43.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcaa2ffd640 1 -- 172.21.15.26:0/2764549345 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v237) v1 ==== 74+0+20925 (secure 0 0 0) 0x7fcaa40ef890 con 0x7fcab40cc0a0 2023-12-19T18:17:43.565 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:43.565 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":237,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:40.804351+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:25.867829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:26.330514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:17:37.484211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:17:38.799276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcab359e640 1 -- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fca94065ca0 msgr2=0x7fca94068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcab359e640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fca94065ca0 0x7fca94068160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcab4156a60 tx=0x7fcaa8007df0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcab359e640 1 -- 172.21.15.26:0/2764549345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab40cc0a0 msgr2=0x7fcab40c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcab359e640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab40cc0a0 0x7fcab40c4af0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4019460 tx=0x7fcaa40195f0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcab359e640 1 -- 172.21.15.26:0/2764549345 shutdown_connections 2023-12-19T18:17:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcab359e640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fca94065ca0 0x7fca94068160 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcab359e640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcab4159d20 0x7fcab40c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcab359e640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcab40cdee0 0x7fcab4151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcab359e640 1 --2- 172.21.15.26:0/2764549345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcab40cc0a0 0x7fcab40c4af0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcab359e640 1 -- 172.21.15.26:0/2764549345 >> 172.21.15.26:0/2764549345 conn(0x7fcab406b0f0 msgr2=0x7fcab40cac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:43.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcab359e640 1 -- 172.21.15.26:0/2764549345 shutdown_connections 2023-12-19T18:17:43.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.561+0000 7fcab359e640 1 -- 172.21.15.26:0/2764549345 wait complete. 2023-12-19T18:17:43.586 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:17:43.587 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:17:43.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.725+0000 7fd546f18640 1 Processor -- start 2023-12-19T18:17:43.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.725+0000 7fd546f18640 1 -- start start 2023-12-19T18:17:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd546f18640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd540155030 0x7fd54015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd546f18640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd5400ce740 0x7fd5400ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd546f18640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5400cf0b0 con 0x7fd54015f370 2023-12-19T18:17:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd546f18640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54006c790 con 0x7fd540155030 2023-12-19T18:17:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd546f18640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54006c970 con 0x7fd5400ce740 2023-12-19T18:17:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd544cb4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd540155030 0x7fd54015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd53ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd54015f370 0x7fd54006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59538/0 (socket says 172.21.15.26:59538) 2023-12-19T18:17:43.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd53ffff640 1 -- 172.21.15.26:0/2445797047 learned_addr learned my addr 172.21.15.26:0/2445797047 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd544cb4640 1 -- 172.21.15.26:0/2445797047 >> v2:172.21.15.160:3300/0 conn(0x7fd5400ce740 msgr2=0x7fd5400ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd5454b5640 1 --2- 172.21.15.26:0/2445797047 >> v2:172.21.15.160:3300/0 conn(0x7fd5400ce740 0x7fd5400ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd544cb4640 1 --2- 172.21.15.26:0/2445797047 >> v2:172.21.15.160:3300/0 conn(0x7fd5400ce740 0x7fd5400ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd53f7fe640 1 -- 172.21.15.26:0/2445797047 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040290886 0 0) 0x7fd5400cf0b0 con 0x7fd54015f370 2023-12-19T18:17:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd544cb4640 1 -- 172.21.15.26:0/2445797047 >> v1:172.21.15.26:6789/0 conn(0x7fd54015f370 legacy=0x7fd54006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd544cb4640 1 -- 172.21.15.26:0/2445797047 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54015b1f0 con 0x7fd540155030 2023-12-19T18:17:43.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.729+0000 7fd5454b5640 1 --2- 172.21.15.26:0/2445797047 >> v2:172.21.15.160:3300/0 conn(0x7fd5400ce740 0x7fd5400ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:43.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd544cb4640 1 --2- 172.21.15.26:0/2445797047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd540155030 0x7fd54015bcd0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fd5400cc390 tx=0x7fd53401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6588c64256fec578 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:43.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd53f7fe640 1 -- 172.21.15.26:0/2445797047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5340a7070 con 0x7fd540155030 2023-12-19T18:17:43.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd53f7fe640 1 -- 172.21.15.26:0/2445797047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5340ab430 con 0x7fd540155030 2023-12-19T18:17:43.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd53f7fe640 1 -- 172.21.15.26:0/2445797047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5340ab7c0 con 0x7fd540155030 2023-12-19T18:17:43.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd546f18640 1 -- 172.21.15.26:0/2445797047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd540155030 msgr2=0x7fd54015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd546f18640 1 --2- 172.21.15.26:0/2445797047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd540155030 0x7fd54015bcd0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7fd5400cc390 tx=0x7fd53401c430 comp rx=0 tx=0).stop 2023-12-19T18:17:43.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd546f18640 1 -- 172.21.15.26:0/2445797047 shutdown_connections 2023-12-19T18:17:43.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd546f18640 1 --2- 172.21.15.26:0/2445797047 >> v2:172.21.15.160:3300/0 conn(0x7fd5400ce740 0x7fd5400ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd546f18640 1 --2- 172.21.15.26:0/2445797047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd540155030 0x7fd54015bcd0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd546f18640 1 -- 172.21.15.26:0/2445797047 >> 172.21.15.26:0/2445797047 conn(0x7fd5400658e0 msgr2=0x7fd540065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:43.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd546f18640 1 -- 172.21.15.26:0/2445797047 shutdown_connections 2023-12-19T18:17:43.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd546f18640 1 -- 172.21.15.26:0/2445797047 wait complete. 2023-12-19T18:17:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd546f18640 1 Processor -- start 2023-12-19T18:17:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.733+0000 7fd546f18640 1 -- start start 2023-12-19T18:17:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd546f18640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5400ce740 0x7fd5400af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd546f18640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd540155030 0x7fd5400b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd546f18640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54015f370 0x7fd5400b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd546f18640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd540142910 con 0x7fd54015f370 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd546f18640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd540150bc0 con 0x7fd540155030 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd546f18640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd540150ea0 con 0x7fd5400ce740 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd5454b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54015f370 0x7fd5400b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd5454b5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54015f370 0x7fd5400b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36716/0 (socket says 172.21.15.26:36716) 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd5454b5640 1 -- 172.21.15.26:0/404723135 learned_addr learned my addr 172.21.15.26:0/404723135 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd544cb4640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5400ce740 0x7fd5400af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd53ffff640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd540155030 0x7fd5400b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd5454b5640 1 -- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5400ce740 msgr2=0x7fd5400af640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd5454b5640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5400ce740 0x7fd5400af640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd5454b5640 1 -- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd540155030 msgr2=0x7fd5400b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd5454b5640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd540155030 0x7fd5400b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd5454b5640 1 -- 172.21.15.26:0/404723135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd534009420 con 0x7fd54015f370 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd53ffff640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd540155030 0x7fd5400b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd5454b5640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54015f370 0x7fd5400b1d30 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fd53800cef0 tx=0x7fd53800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd53d7fa640 1 -- 172.21.15.26:0/404723135 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53800d840 con 0x7fd54015f370 2023-12-19T18:17:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd53d7fa640 1 -- 172.21.15.26:0/404723135 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53800d9c0 con 0x7fd54015f370 2023-12-19T18:17:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd53d7fa640 1 -- 172.21.15.26:0/404723135 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53800dd70 con 0x7fd54015f370 2023-12-19T18:17:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd546f18640 1 -- 172.21.15.26:0/404723135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd540151180 con 0x7fd54015f370 2023-12-19T18:17:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd546f18640 1 -- 172.21.15.26:0/404723135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5401f82a0 con 0x7fd54015f370 2023-12-19T18:17:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.737+0000 7fd546f18640 1 -- 172.21.15.26:0/404723135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5401f8520 con 0x7fd54015f370 2023-12-19T18:17:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.741+0000 7fd53d7fa640 1 -- 172.21.15.26:0/404723135 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fd53801e3b0 con 0x7fd54015f370 2023-12-19T18:17:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.741+0000 7fd53d7fa640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd524065cd0 0x7fd524068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.741+0000 7fd53d7fa640 1 -- 172.21.15.26:0/404723135 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fd538084740 con 0x7fd54015f370 2023-12-19T18:17:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.741+0000 7fd544cb4640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd524065cd0 0x7fd524068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.741+0000 7fd544cb4640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd524065cd0 0x7fd524068190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd5340af000 tx=0x7fd5340a7000 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.745+0000 7fd53d7fa640 1 -- 172.21.15.26:0/404723135 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd538051760 con 0x7fd54015f370 2023-12-19T18:17:43.828 INFO:tasks.ceph.mds.a.smithi026.stderr:2023-12-19T18:17:43.825+0000 7f9d5c382640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:17:43.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.901+0000 7fd546f18640 1 -- 172.21.15.26:0/404723135 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd54006cc50 con 0x7fd524065cd0 2023-12-19T18:17:43.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.913+0000 7fd53d7fa640 1 -- 172.21.15.26:0/404723135 <== mgr.10429 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528453 (secure 0 0 0) 0x7fd5340aba20 con 0x7fd524065cd0 2023-12-19T18:17:43.923 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:43.928 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:17:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.917+0000 7fd546f18640 1 -- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd524065cd0 msgr2=0x7fd524068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.917+0000 7fd546f18640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd524065cd0 0x7fd524068190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd5340af000 tx=0x7fd5340a7000 comp rx=0 tx=0).stop 2023-12-19T18:17:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.917+0000 7fd546f18640 1 -- 172.21.15.26:0/404723135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54015f370 msgr2=0x7fd5400b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.917+0000 7fd546f18640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54015f370 0x7fd5400b1d30 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fd53800cef0 tx=0x7fd53800a710 comp rx=0 tx=0).stop 2023-12-19T18:17:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.917+0000 7fd546f18640 1 -- 172.21.15.26:0/404723135 shutdown_connections 2023-12-19T18:17:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.917+0000 7fd546f18640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd524065cd0 0x7fd524068190 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.917+0000 7fd546f18640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54015f370 0x7fd5400b1d30 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.917+0000 7fd546f18640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd540155030 0x7fd5400b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.917+0000 7fd546f18640 1 --2- 172.21.15.26:0/404723135 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd5400ce740 0x7fd5400af640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:43.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.921+0000 7fd546f18640 1 -- 172.21.15.26:0/404723135 >> 172.21.15.26:0/404723135 conn(0x7fd5400658e0 msgr2=0x7fd540151b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:43.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.921+0000 7fd546f18640 1 -- 172.21.15.26:0/404723135 shutdown_connections 2023-12-19T18:17:43.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:43.921+0000 7fd546f18640 1 -- 172.21.15.26:0/404723135 wait complete. 2023-12-19T18:17:43.943 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-12-19T18:17:42.938817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596333,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2306,"num_read_kb":4823,"num_write":706,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":447,"ondisk_log_size":447,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":280,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":408968,"kb_used_data":24392,"kb_used_omap":0,"kb_used_meta":384512,"kb_avail":754565752,"statfs":{"total":773094113280,"available":772675330048,"internally_reserved":0,"allocated":24977408,"data_stored":13835266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":393740288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3349,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.552648"},"pg_stats":[{"pgid":"31.3d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.837659+0000","last_change":"2023-12-19T18:17:39.833552+0000","last_active":"2023-12-19T18:17:39.837659+0000","last_peered":"2023-12-19T18:17:39.837659+0000","last_clean":"2023-12-19T18:17:39.837659+0000","last_became_active":"2023-12-19T18:17:39.832016+0000","last_became_peered":"2023-12-19T18:17:39.832016+0000","last_unstale":"2023-12-19T18:17:39.837659+0000","last_undegraded":"2023-12-19T18:17:39.837659+0000","last_fullsized":"2023-12-19T18:17:39.837659+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:42:22.136168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413591+0000","last_change":"2023-12-19T18:17:27.353410+0000","last_active":"2023-12-19T18:17:39.413591+0000","last_peered":"2023-12-19T18:17:39.413591+0000","last_clean":"2023-12-19T18:17:39.413591+0000","last_became_active":"2023-12-19T18:17:27.352446+0000","last_became_peered":"2023-12-19T18:17:27.352446+0000","last_unstale":"2023-12-19T18:17:39.413591+0000","last_undegraded":"2023-12-19T18:17:39.413591+0000","last_fullsized":"2023-12-19T18:17:39.413591+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:55.042619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.839318+0000","last_change":"2023-12-19T18:17:39.833095+0000","last_active":"2023-12-19T18:17:39.839318+0000","last_peered":"2023-12-19T18:17:39.839318+0000","last_clean":"2023-12-19T18:17:39.839318+0000","last_became_active":"2023-12-19T18:17:39.832795+0000","last_became_peered":"2023-12-19T18:17:39.832795+0000","last_unstale":"2023-12-19T18:17:39.839318+0000","last_undegraded":"2023-12-19T18:17:39.839318+0000","last_fullsized":"2023-12-19T18:17:39.839318+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:09.670408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.409997+0000","last_change":"2023-12-19T18:17:27.358427+0000","last_active":"2023-12-19T18:17:39.409997+0000","last_peered":"2023-12-19T18:17:39.409997+0000","last_clean":"2023-12-19T18:17:39.409997+0000","last_became_active":"2023-12-19T18:17:27.358141+0000","last_became_peered":"2023-12-19T18:17:27.358141+0000","last_unstale":"2023-12-19T18:17:39.409997+0000","last_undegraded":"2023-12-19T18:17:39.409997+0000","last_fullsized":"2023-12-19T18:17:39.409997+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:14.654686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.3f","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.821083+0000","last_change":"2023-12-19T18:17:39.815673+0000","last_active":"2023-12-19T18:17:39.821083+0000","last_peered":"2023-12-19T18:17:39.821083+0000","last_clean":"2023-12-19T18:17:39.821083+0000","last_became_active":"2023-12-19T18:17:39.815424+0000","last_became_peered":"2023-12-19T18:17:39.815424+0000","last_unstale":"2023-12-19T18:17:39.821083+0000","last_undegraded":"2023-12-19T18:17:39.821083+0000","last_fullsized":"2023-12-19T18:17:39.821083+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:11.577769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820465+0000","last_change":"2023-12-19T18:17:27.356031+0000","last_active":"2023-12-19T18:17:39.820465+0000","last_peered":"2023-12-19T18:17:39.820465+0000","last_clean":"2023-12-19T18:17:39.820465+0000","last_became_active":"2023-12-19T18:17:27.354195+0000","last_became_peered":"2023-12-19T18:17:27.354195+0000","last_unstale":"2023-12-19T18:17:39.820465+0000","last_undegraded":"2023-12-19T18:17:39.820465+0000","last_fullsized":"2023-12-19T18:17:39.820465+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:16.644439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.826135+0000","last_change":"2023-12-19T18:17:39.818843+0000","last_active":"2023-12-19T18:17:39.826135+0000","last_peered":"2023-12-19T18:17:39.826135+0000","last_clean":"2023-12-19T18:17:39.826135+0000","last_became_active":"2023-12-19T18:17:39.818569+0000","last_became_peered":"2023-12-19T18:17:39.818569+0000","last_unstale":"2023-12-19T18:17:39.826135+0000","last_undegraded":"2023-12-19T18:17:39.826135+0000","last_fullsized":"2023-12-19T18:17:39.826135+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:20:14.678399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413904+0000","last_change":"2023-12-19T18:17:27.348408+0000","last_active":"2023-12-19T18:17:39.413904+0000","last_peered":"2023-12-19T18:17:39.413904+0000","last_clean":"2023-12-19T18:17:39.413904+0000","last_became_active":"2023-12-19T18:17:27.348138+0000","last_became_peered":"2023-12-19T18:17:27.348138+0000","last_unstale":"2023-12-19T18:17:39.413904+0000","last_undegraded":"2023-12-19T18:17:39.413904+0000","last_fullsized":"2023-12-19T18:17:39.413904+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:19.968471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812300+0000","last_change":"2023-12-19T18:17:39.839350+0000","last_active":"2023-12-19T18:17:40.812300+0000","last_peered":"2023-12-19T18:17:40.812300+0000","last_clean":"2023-12-19T18:17:40.812300+0000","last_became_active":"2023-12-19T18:17:39.839054+0000","last_became_peered":"2023-12-19T18:17:39.839054+0000","last_unstale":"2023-12-19T18:17:40.812300+0000","last_undegraded":"2023-12-19T18:17:40.812300+0000","last_fullsized":"2023-12-19T18:17:40.812300+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:14:03.086377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413526+0000","last_change":"2023-12-19T18:17:27.351573+0000","last_active":"2023-12-19T18:17:39.413526+0000","last_peered":"2023-12-19T18:17:39.413526+0000","last_clean":"2023-12-19T18:17:39.413526+0000","last_became_active":"2023-12-19T18:17:27.351222+0000","last_became_peered":"2023-12-19T18:17:27.351222+0000","last_unstale":"2023-12-19T18:17:39.413526+0000","last_undegraded":"2023-12-19T18:17:39.413526+0000","last_fullsized":"2023-12-19T18:17:39.413526+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:17:31.537417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.829015+0000","last_change":"2023-12-19T18:17:39.825005+0000","last_active":"2023-12-19T18:17:39.829015+0000","last_peered":"2023-12-19T18:17:39.829015+0000","last_clean":"2023-12-19T18:17:39.829015+0000","last_became_active":"2023-12-19T18:17:39.824659+0000","last_became_peered":"2023-12-19T18:17:39.824659+0000","last_unstale":"2023-12-19T18:17:39.829015+0000","last_undegraded":"2023-12-19T18:17:39.829015+0000","last_fullsized":"2023-12-19T18:17:39.829015+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:15:55.948961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.405823+0000","last_change":"2023-12-19T18:17:27.364283+0000","last_active":"2023-12-19T18:17:39.405823+0000","last_peered":"2023-12-19T18:17:39.405823+0000","last_clean":"2023-12-19T18:17:39.405823+0000","last_became_active":"2023-12-19T18:17:27.363962+0000","last_became_peered":"2023-12-19T18:17:27.363962+0000","last_unstale":"2023-12-19T18:17:39.405823+0000","last_undegraded":"2023-12-19T18:17:39.405823+0000","last_fullsized":"2023-12-19T18:17:39.405823+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:25.159077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.811721+0000","last_change":"2023-12-19T18:17:39.823676+0000","last_active":"2023-12-19T18:17:40.811721+0000","last_peered":"2023-12-19T18:17:40.811721+0000","last_clean":"2023-12-19T18:17:40.811721+0000","last_became_active":"2023-12-19T18:17:39.823405+0000","last_became_peered":"2023-12-19T18:17:39.823405+0000","last_unstale":"2023-12-19T18:17:40.811721+0000","last_undegraded":"2023-12-19T18:17:40.811721+0000","last_fullsized":"2023-12-19T18:17:40.811721+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:31:18.872287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820413+0000","last_change":"2023-12-19T18:17:27.357178+0000","last_active":"2023-12-19T18:17:39.820413+0000","last_peered":"2023-12-19T18:17:39.820413+0000","last_clean":"2023-12-19T18:17:39.820413+0000","last_became_active":"2023-12-19T18:17:27.356893+0000","last_became_peered":"2023-12-19T18:17:27.356893+0000","last_unstale":"2023-12-19T18:17:39.820413+0000","last_undegraded":"2023-12-19T18:17:39.820413+0000","last_fullsized":"2023-12-19T18:17:39.820413+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:31.153454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.840809+0000","last_change":"2023-12-19T18:17:39.838729+0000","last_active":"2023-12-19T18:17:39.840809+0000","last_peered":"2023-12-19T18:17:39.840809+0000","last_clean":"2023-12-19T18:17:39.840809+0000","last_became_active":"2023-12-19T18:17:39.838043+0000","last_became_peered":"2023-12-19T18:17:39.838043+0000","last_unstale":"2023-12-19T18:17:39.840809+0000","last_undegraded":"2023-12-19T18:17:39.840809+0000","last_fullsized":"2023-12-19T18:17:39.840809+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:02.711769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408878+0000","last_change":"2023-12-19T18:17:27.344049+0000","last_active":"2023-12-19T18:17:39.408878+0000","last_peered":"2023-12-19T18:17:39.408878+0000","last_clean":"2023-12-19T18:17:39.408878+0000","last_became_active":"2023-12-19T18:17:27.343738+0000","last_became_peered":"2023-12-19T18:17:27.343738+0000","last_unstale":"2023-12-19T18:17:39.408878+0000","last_undegraded":"2023-12-19T18:17:39.408878+0000","last_fullsized":"2023-12-19T18:17:39.408878+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:31.445521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809457+0000","last_change":"2023-12-19T18:17:39.844452+0000","last_active":"2023-12-19T18:17:40.809457+0000","last_peered":"2023-12-19T18:17:40.809457+0000","last_clean":"2023-12-19T18:17:40.809457+0000","last_became_active":"2023-12-19T18:17:39.844014+0000","last_became_peered":"2023-12-19T18:17:39.844014+0000","last_unstale":"2023-12-19T18:17:40.809457+0000","last_undegraded":"2023-12-19T18:17:40.809457+0000","last_fullsized":"2023-12-19T18:17:40.809457+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:52:35.124201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.822570+0000","last_change":"2023-12-19T18:17:27.363691+0000","last_active":"2023-12-19T18:17:39.822570+0000","last_peered":"2023-12-19T18:17:39.822570+0000","last_clean":"2023-12-19T18:17:39.822570+0000","last_became_active":"2023-12-19T18:17:27.363412+0000","last_became_peered":"2023-12-19T18:17:27.363412+0000","last_unstale":"2023-12-19T18:17:39.822570+0000","last_undegraded":"2023-12-19T18:17:39.822570+0000","last_fullsized":"2023-12-19T18:17:39.822570+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:47.826883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.826556+0000","last_change":"2023-12-19T18:17:39.822612+0000","last_active":"2023-12-19T18:17:39.826556+0000","last_peered":"2023-12-19T18:17:39.826556+0000","last_clean":"2023-12-19T18:17:39.826556+0000","last_became_active":"2023-12-19T18:17:39.822251+0000","last_became_peered":"2023-12-19T18:17:39.822251+0000","last_unstale":"2023-12-19T18:17:39.826556+0000","last_undegraded":"2023-12-19T18:17:39.826556+0000","last_fullsized":"2023-12-19T18:17:39.826556+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:14:03.086377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.417741+0000","last_change":"2023-12-19T18:17:27.343713+0000","last_active":"2023-12-19T18:17:39.417741+0000","last_peered":"2023-12-19T18:17:39.417741+0000","last_clean":"2023-12-19T18:17:39.417741+0000","last_became_active":"2023-12-19T18:17:27.343409+0000","last_became_peered":"2023-12-19T18:17:27.343409+0000","last_unstale":"2023-12-19T18:17:39.417741+0000","last_undegraded":"2023-12-19T18:17:39.417741+0000","last_fullsized":"2023-12-19T18:17:39.417741+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:40.489404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.843459+0000","last_change":"2023-12-19T18:17:39.842790+0000","last_active":"2023-12-19T18:17:39.843459+0000","last_peered":"2023-12-19T18:17:39.843459+0000","last_clean":"2023-12-19T18:17:39.843459+0000","last_became_active":"2023-12-19T18:17:39.842464+0000","last_became_peered":"2023-12-19T18:17:39.842464+0000","last_unstale":"2023-12-19T18:17:39.843459+0000","last_undegraded":"2023-12-19T18:17:39.843459+0000","last_fullsized":"2023-12-19T18:17:39.843459+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:05.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416082+0000","last_change":"2023-12-19T18:17:27.353890+0000","last_active":"2023-12-19T18:17:39.416082+0000","last_peered":"2023-12-19T18:17:39.416082+0000","last_clean":"2023-12-19T18:17:39.416082+0000","last_became_active":"2023-12-19T18:17:27.352505+0000","last_became_peered":"2023-12-19T18:17:27.352505+0000","last_unstale":"2023-12-19T18:17:39.416082+0000","last_undegraded":"2023-12-19T18:17:39.416082+0000","last_fullsized":"2023-12-19T18:17:39.416082+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:49:56.496681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.813410+0000","last_change":"2023-12-19T18:17:39.830133+0000","last_active":"2023-12-19T18:17:40.813410+0000","last_peered":"2023-12-19T18:17:40.813410+0000","last_clean":"2023-12-19T18:17:40.813410+0000","last_became_active":"2023-12-19T18:17:39.829808+0000","last_became_peered":"2023-12-19T18:17:39.829808+0000","last_unstale":"2023-12-19T18:17:40.813410+0000","last_undegraded":"2023-12-19T18:17:40.813410+0000","last_fullsized":"2023-12-19T18:17:40.813410+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:44.450250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408825+0000","last_change":"2023-12-19T18:17:27.353636+0000","last_active":"2023-12-19T18:17:39.408825+0000","last_peered":"2023-12-19T18:17:39.408825+0000","last_clean":"2023-12-19T18:17:39.408825+0000","last_became_active":"2023-12-19T18:17:27.353234+0000","last_became_peered":"2023-12-19T18:17:27.353234+0000","last_unstale":"2023-12-19T18:17:39.408825+0000","last_undegraded":"2023-12-19T18:17:39.408825+0000","last_fullsized":"2023-12-19T18:17:39.408825+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:50.421353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.843881+0000","last_change":"2023-12-19T18:17:39.843212+0000","last_active":"2023-12-19T18:17:39.843881+0000","last_peered":"2023-12-19T18:17:39.843881+0000","last_clean":"2023-12-19T18:17:39.843881+0000","last_became_active":"2023-12-19T18:17:39.842963+0000","last_became_peered":"2023-12-19T18:17:39.842963+0000","last_unstale":"2023-12-19T18:17:39.843881+0000","last_undegraded":"2023-12-19T18:17:39.843881+0000","last_fullsized":"2023-12-19T18:17:39.843881+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:34:52.684868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408357+0000","last_change":"2023-12-19T18:17:27.368093+0000","last_active":"2023-12-19T18:17:39.408357+0000","last_peered":"2023-12-19T18:17:39.408357+0000","last_clean":"2023-12-19T18:17:39.408357+0000","last_became_active":"2023-12-19T18:17:27.367300+0000","last_became_peered":"2023-12-19T18:17:27.367300+0000","last_unstale":"2023-12-19T18:17:39.408357+0000","last_undegraded":"2023-12-19T18:17:39.408357+0000","last_fullsized":"2023-12-19T18:17:39.408357+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:16.644439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.838226+0000","last_change":"2023-12-19T18:17:39.833455+0000","last_active":"2023-12-19T18:17:39.838226+0000","last_peered":"2023-12-19T18:17:39.838226+0000","last_clean":"2023-12-19T18:17:39.838226+0000","last_became_active":"2023-12-19T18:17:39.832753+0000","last_became_peered":"2023-12-19T18:17:39.832753+0000","last_unstale":"2023-12-19T18:17:39.838226+0000","last_undegraded":"2023-12-19T18:17:39.838226+0000","last_fullsized":"2023-12-19T18:17:39.838226+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:58:27.250517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810496+0000","last_change":"2023-12-19T18:17:27.362240+0000","last_active":"2023-12-19T18:17:39.810496+0000","last_peered":"2023-12-19T18:17:39.810496+0000","last_clean":"2023-12-19T18:17:39.810496+0000","last_became_active":"2023-12-19T18:17:27.361917+0000","last_became_peered":"2023-12-19T18:17:27.361917+0000","last_unstale":"2023-12-19T18:17:39.810496+0000","last_undegraded":"2023-12-19T18:17:39.810496+0000","last_fullsized":"2023-12-19T18:17:39.810496+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:44:22.931173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812849+0000","last_change":"2023-12-19T18:17:39.839497+0000","last_active":"2023-12-19T18:17:40.812849+0000","last_peered":"2023-12-19T18:17:40.812849+0000","last_clean":"2023-12-19T18:17:40.812849+0000","last_became_active":"2023-12-19T18:17:39.839216+0000","last_became_peered":"2023-12-19T18:17:39.839216+0000","last_unstale":"2023-12-19T18:17:40.812849+0000","last_undegraded":"2023-12-19T18:17:40.812849+0000","last_fullsized":"2023-12-19T18:17:40.812849+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:34:52.684868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830166+0000","last_change":"2023-12-19T18:17:27.356217+0000","last_active":"2023-12-19T18:17:39.830166+0000","last_peered":"2023-12-19T18:17:39.830166+0000","last_clean":"2023-12-19T18:17:39.830166+0000","last_became_active":"2023-12-19T18:17:27.355721+0000","last_became_peered":"2023-12-19T18:17:27.355721+0000","last_unstale":"2023-12-19T18:17:39.830166+0000","last_undegraded":"2023-12-19T18:17:39.830166+0000","last_fullsized":"2023-12-19T18:17:39.830166+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:20:57.806100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.823931+0000","last_change":"2023-12-19T18:17:39.819672+0000","last_active":"2023-12-19T18:17:39.823931+0000","last_peered":"2023-12-19T18:17:39.823931+0000","last_clean":"2023-12-19T18:17:39.823931+0000","last_became_active":"2023-12-19T18:17:39.819371+0000","last_became_peered":"2023-12-19T18:17:39.819371+0000","last_unstale":"2023-12-19T18:17:39.823931+0000","last_undegraded":"2023-12-19T18:17:39.823931+0000","last_fullsized":"2023-12-19T18:17:39.823931+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:34:51.758688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408201+0000","last_change":"2023-12-19T18:17:27.350328+0000","last_active":"2023-12-19T18:17:39.408201+0000","last_peered":"2023-12-19T18:17:39.408201+0000","last_clean":"2023-12-19T18:17:39.408201+0000","last_became_active":"2023-12-19T18:17:27.350006+0000","last_became_peered":"2023-12-19T18:17:27.350006+0000","last_unstale":"2023-12-19T18:17:39.408201+0000","last_undegraded":"2023-12-19T18:17:39.408201+0000","last_fullsized":"2023-12-19T18:17:39.408201+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:00.178479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.846852+0000","last_change":"2023-12-19T18:17:39.845313+0000","last_active":"2023-12-19T18:17:39.846852+0000","last_peered":"2023-12-19T18:17:39.846852+0000","last_clean":"2023-12-19T18:17:39.846852+0000","last_became_active":"2023-12-19T18:17:39.845016+0000","last_became_peered":"2023-12-19T18:17:39.845016+0000","last_unstale":"2023-12-19T18:17:39.846852+0000","last_undegraded":"2023-12-19T18:17:39.846852+0000","last_fullsized":"2023-12-19T18:17:39.846852+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:17:44.002667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810896+0000","last_change":"2023-12-19T18:17:27.367615+0000","last_active":"2023-12-19T18:17:39.810896+0000","last_peered":"2023-12-19T18:17:39.810896+0000","last_clean":"2023-12-19T18:17:39.810896+0000","last_became_active":"2023-12-19T18:17:27.366800+0000","last_became_peered":"2023-12-19T18:17:27.366800+0000","last_unstale":"2023-12-19T18:17:39.810896+0000","last_undegraded":"2023-12-19T18:17:39.810896+0000","last_fullsized":"2023-12-19T18:17:39.810896+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:19:50.020207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.825903+0000","last_change":"2023-12-19T18:17:39.818585+0000","last_active":"2023-12-19T18:17:39.825903+0000","last_peered":"2023-12-19T18:17:39.825903+0000","last_clean":"2023-12-19T18:17:39.825903+0000","last_became_active":"2023-12-19T18:17:39.818218+0000","last_became_peered":"2023-12-19T18:17:39.818218+0000","last_unstale":"2023-12-19T18:17:39.825903+0000","last_undegraded":"2023-12-19T18:17:39.825903+0000","last_fullsized":"2023-12-19T18:17:39.825903+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:45.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.417825+0000","last_change":"2023-12-19T18:17:27.357129+0000","last_active":"2023-12-19T18:17:39.417825+0000","last_peered":"2023-12-19T18:17:39.417825+0000","last_clean":"2023-12-19T18:17:39.417825+0000","last_became_active":"2023-12-19T18:17:27.356614+0000","last_became_peered":"2023-12-19T18:17:27.356614+0000","last_unstale":"2023-12-19T18:17:39.417825+0000","last_undegraded":"2023-12-19T18:17:39.417825+0000","last_fullsized":"2023-12-19T18:17:39.417825+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:22.773760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.842026+0000","last_change":"2023-12-19T18:17:39.841034+0000","last_active":"2023-12-19T18:17:39.842026+0000","last_peered":"2023-12-19T18:17:39.842026+0000","last_clean":"2023-12-19T18:17:39.842026+0000","last_became_active":"2023-12-19T18:17:39.840678+0000","last_became_peered":"2023-12-19T18:17:39.840678+0000","last_unstale":"2023-12-19T18:17:39.842026+0000","last_undegraded":"2023-12-19T18:17:39.842026+0000","last_fullsized":"2023-12-19T18:17:39.842026+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:44:21.703567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413550+0000","last_change":"2023-12-19T18:17:27.353111+0000","last_active":"2023-12-19T18:17:39.413550+0000","last_peered":"2023-12-19T18:17:39.413550+0000","last_clean":"2023-12-19T18:17:39.413550+0000","last_became_active":"2023-12-19T18:17:27.352840+0000","last_became_peered":"2023-12-19T18:17:27.352840+0000","last_unstale":"2023-12-19T18:17:39.413550+0000","last_undegraded":"2023-12-19T18:17:39.413550+0000","last_fullsized":"2023-12-19T18:17:39.413550+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:28.649300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830572+0000","last_change":"2023-12-19T18:17:39.825874+0000","last_active":"2023-12-19T18:17:39.830572+0000","last_peered":"2023-12-19T18:17:39.830572+0000","last_clean":"2023-12-19T18:17:39.830572+0000","last_became_active":"2023-12-19T18:17:39.823789+0000","last_became_peered":"2023-12-19T18:17:39.823789+0000","last_unstale":"2023-12-19T18:17:39.830572+0000","last_undegraded":"2023-12-19T18:17:39.830572+0000","last_fullsized":"2023-12-19T18:17:39.830572+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:04:11.535860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416217+0000","last_change":"2023-12-19T18:17:27.340972+0000","last_active":"2023-12-19T18:17:39.416217+0000","last_peered":"2023-12-19T18:17:39.416217+0000","last_clean":"2023-12-19T18:17:39.416217+0000","last_became_active":"2023-12-19T18:17:27.340523+0000","last_became_peered":"2023-12-19T18:17:27.340523+0000","last_unstale":"2023-12-19T18:17:39.416217+0000","last_undegraded":"2023-12-19T18:17:39.416217+0000","last_fullsized":"2023-12-19T18:17:39.416217+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:50.621127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.828320+0000","last_change":"2023-12-19T18:17:39.823149+0000","last_active":"2023-12-19T18:17:39.828320+0000","last_peered":"2023-12-19T18:17:39.828320+0000","last_clean":"2023-12-19T18:17:39.828320+0000","last_became_active":"2023-12-19T18:17:39.822891+0000","last_became_peered":"2023-12-19T18:17:39.822891+0000","last_unstale":"2023-12-19T18:17:39.828320+0000","last_undegraded":"2023-12-19T18:17:39.828320+0000","last_fullsized":"2023-12-19T18:17:39.828320+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:04:45.527849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413613+0000","last_change":"2023-12-19T18:17:27.355317+0000","last_active":"2023-12-19T18:17:39.413613+0000","last_peered":"2023-12-19T18:17:39.413613+0000","last_clean":"2023-12-19T18:17:39.413613+0000","last_became_active":"2023-12-19T18:17:27.355002+0000","last_became_peered":"2023-12-19T18:17:27.355002+0000","last_unstale":"2023-12-19T18:17:39.413613+0000","last_undegraded":"2023-12-19T18:17:39.413613+0000","last_fullsized":"2023-12-19T18:17:39.413613+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:50.621127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.842164+0000","last_change":"2023-12-19T18:17:39.838759+0000","last_active":"2023-12-19T18:17:39.842164+0000","last_peered":"2023-12-19T18:17:39.842164+0000","last_clean":"2023-12-19T18:17:39.842164+0000","last_became_active":"2023-12-19T18:17:39.838479+0000","last_became_peered":"2023-12-19T18:17:39.838479+0000","last_unstale":"2023-12-19T18:17:39.842164+0000","last_undegraded":"2023-12-19T18:17:39.842164+0000","last_fullsized":"2023-12-19T18:17:39.842164+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:38:23.125499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834118+0000","last_change":"2023-12-19T18:17:27.347478+0000","last_active":"2023-12-19T18:17:39.834118+0000","last_peered":"2023-12-19T18:17:39.834118+0000","last_clean":"2023-12-19T18:17:39.834118+0000","last_became_active":"2023-12-19T18:17:27.346998+0000","last_became_peered":"2023-12-19T18:17:27.346998+0000","last_unstale":"2023-12-19T18:17:39.834118+0000","last_undegraded":"2023-12-19T18:17:39.834118+0000","last_fullsized":"2023-12-19T18:17:39.834118+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:22.470263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834392+0000","last_change":"2023-12-19T18:17:39.832098+0000","last_active":"2023-12-19T18:17:39.834392+0000","last_peered":"2023-12-19T18:17:39.834392+0000","last_clean":"2023-12-19T18:17:39.834392+0000","last_became_active":"2023-12-19T18:17:39.831637+0000","last_became_peered":"2023-12-19T18:17:39.831637+0000","last_unstale":"2023-12-19T18:17:39.834392+0000","last_undegraded":"2023-12-19T18:17:39.834392+0000","last_fullsized":"2023-12-19T18:17:39.834392+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:32.433721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416878+0000","last_change":"2023-12-19T18:17:27.339790+0000","last_active":"2023-12-19T18:17:39.416878+0000","last_peered":"2023-12-19T18:17:39.416878+0000","last_clean":"2023-12-19T18:17:39.416878+0000","last_became_active":"2023-12-19T18:17:27.339477+0000","last_became_peered":"2023-12-19T18:17:27.339477+0000","last_unstale":"2023-12-19T18:17:39.416878+0000","last_undegraded":"2023-12-19T18:17:39.416878+0000","last_fullsized":"2023-12-19T18:17:39.416878+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:23:29.725587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812196+0000","last_change":"2023-12-19T18:17:39.831883+0000","last_active":"2023-12-19T18:17:40.812196+0000","last_peered":"2023-12-19T18:17:40.812196+0000","last_clean":"2023-12-19T18:17:40.812196+0000","last_became_active":"2023-12-19T18:17:39.831600+0000","last_became_peered":"2023-12-19T18:17:39.831600+0000","last_unstale":"2023-12-19T18:17:40.812196+0000","last_undegraded":"2023-12-19T18:17:40.812196+0000","last_fullsized":"2023-12-19T18:17:40.812196+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:55:24.562567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408836+0000","last_change":"2023-12-19T18:17:27.348543+0000","last_active":"2023-12-19T18:17:39.408836+0000","last_peered":"2023-12-19T18:17:39.408836+0000","last_clean":"2023-12-19T18:17:39.408836+0000","last_became_active":"2023-12-19T18:17:27.348257+0000","last_became_peered":"2023-12-19T18:17:27.348257+0000","last_unstale":"2023-12-19T18:17:39.408836+0000","last_undegraded":"2023-12-19T18:17:39.408836+0000","last_fullsized":"2023-12-19T18:17:39.408836+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:44:09.238317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809544+0000","last_change":"2023-12-19T18:17:39.844300+0000","last_active":"2023-12-19T18:17:40.809544+0000","last_peered":"2023-12-19T18:17:40.809544+0000","last_clean":"2023-12-19T18:17:40.809544+0000","last_became_active":"2023-12-19T18:17:39.843747+0000","last_became_peered":"2023-12-19T18:17:39.843747+0000","last_unstale":"2023-12-19T18:17:40.809544+0000","last_undegraded":"2023-12-19T18:17:40.809544+0000","last_fullsized":"2023-12-19T18:17:40.809544+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:58:44.245127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413680+0000","last_change":"2023-12-19T18:17:27.353080+0000","last_active":"2023-12-19T18:17:39.413680+0000","last_peered":"2023-12-19T18:17:39.413680+0000","last_clean":"2023-12-19T18:17:39.413680+0000","last_became_active":"2023-12-19T18:17:27.352664+0000","last_became_peered":"2023-12-19T18:17:27.352664+0000","last_unstale":"2023-12-19T18:17:39.413680+0000","last_undegraded":"2023-12-19T18:17:39.413680+0000","last_fullsized":"2023-12-19T18:17:39.413680+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:22:10.869020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.825949+0000","last_change":"2023-12-19T18:17:39.817852+0000","last_active":"2023-12-19T18:17:39.825949+0000","last_peered":"2023-12-19T18:17:39.825949+0000","last_clean":"2023-12-19T18:17:39.825949+0000","last_became_active":"2023-12-19T18:17:39.817606+0000","last_became_peered":"2023-12-19T18:17:39.817606+0000","last_unstale":"2023-12-19T18:17:39.825949+0000","last_undegraded":"2023-12-19T18:17:39.825949+0000","last_fullsized":"2023-12-19T18:17:39.825949+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:39:15.230291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.412507+0000","last_change":"2023-12-19T18:17:27.367346+0000","last_active":"2023-12-19T18:17:39.412507+0000","last_peered":"2023-12-19T18:17:39.412507+0000","last_clean":"2023-12-19T18:17:39.412507+0000","last_became_active":"2023-12-19T18:17:27.367053+0000","last_became_peered":"2023-12-19T18:17:27.367053+0000","last_unstale":"2023-12-19T18:17:39.412507+0000","last_undegraded":"2023-12-19T18:17:39.412507+0000","last_fullsized":"2023-12-19T18:17:39.412507+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:44.734608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.840547+0000","last_change":"2023-12-19T18:17:39.837983+0000","last_active":"2023-12-19T18:17:39.840547+0000","last_peered":"2023-12-19T18:17:39.840547+0000","last_clean":"2023-12-19T18:17:39.840547+0000","last_became_active":"2023-12-19T18:17:39.837721+0000","last_became_peered":"2023-12-19T18:17:39.837721+0000","last_unstale":"2023-12-19T18:17:39.840547+0000","last_undegraded":"2023-12-19T18:17:39.840547+0000","last_fullsized":"2023-12-19T18:17:39.840547+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:01:49.026786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.415841+0000","last_change":"2023-12-19T18:17:27.355178+0000","last_active":"2023-12-19T18:17:39.415841+0000","last_peered":"2023-12-19T18:17:39.415841+0000","last_clean":"2023-12-19T18:17:39.415841+0000","last_became_active":"2023-12-19T18:17:27.354771+0000","last_became_peered":"2023-12-19T18:17:27.354771+0000","last_unstale":"2023-12-19T18:17:39.415841+0000","last_undegraded":"2023-12-19T18:17:39.415841+0000","last_fullsized":"2023-12-19T18:17:39.415841+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:06.555153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.813367+0000","last_change":"2023-12-19T18:17:39.846098+0000","last_active":"2023-12-19T18:17:40.813367+0000","last_peered":"2023-12-19T18:17:40.813367+0000","last_clean":"2023-12-19T18:17:40.813367+0000","last_became_active":"2023-12-19T18:17:39.845679+0000","last_became_peered":"2023-12-19T18:17:39.845679+0000","last_unstale":"2023-12-19T18:17:40.813367+0000","last_undegraded":"2023-12-19T18:17:40.813367+0000","last_fullsized":"2023-12-19T18:17:40.813367+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:05.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416296+0000","last_change":"2023-12-19T18:17:27.349197+0000","last_active":"2023-12-19T18:17:39.416296+0000","last_peered":"2023-12-19T18:17:39.416296+0000","last_clean":"2023-12-19T18:17:39.416296+0000","last_became_active":"2023-12-19T18:17:27.348940+0000","last_became_peered":"2023-12-19T18:17:27.348940+0000","last_unstale":"2023-12-19T18:17:39.416296+0000","last_undegraded":"2023-12-19T18:17:39.416296+0000","last_fullsized":"2023-12-19T18:17:39.416296+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:19.968471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.831810+0000","last_change":"2023-12-19T18:17:39.823044+0000","last_active":"2023-12-19T18:17:39.831810+0000","last_peered":"2023-12-19T18:17:39.831810+0000","last_clean":"2023-12-19T18:17:39.831810+0000","last_became_active":"2023-12-19T18:17:39.822772+0000","last_became_peered":"2023-12-19T18:17:39.822772+0000","last_unstale":"2023-12-19T18:17:39.831810+0000","last_undegraded":"2023-12-19T18:17:39.831810+0000","last_fullsized":"2023-12-19T18:17:39.831810+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:50.380394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810228+0000","last_change":"2023-12-19T18:17:27.369700+0000","last_active":"2023-12-19T18:17:39.810228+0000","last_peered":"2023-12-19T18:17:39.810228+0000","last_clean":"2023-12-19T18:17:39.810228+0000","last_became_active":"2023-12-19T18:17:27.369398+0000","last_became_peered":"2023-12-19T18:17:27.369398+0000","last_unstale":"2023-12-19T18:17:39.810228+0000","last_undegraded":"2023-12-19T18:17:39.810228+0000","last_fullsized":"2023-12-19T18:17:39.810228+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:35:43.593726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.833651+0000","last_change":"2023-12-19T18:17:39.829518+0000","last_active":"2023-12-19T18:17:39.833651+0000","last_peered":"2023-12-19T18:17:39.833651+0000","last_clean":"2023-12-19T18:17:39.833651+0000","last_became_active":"2023-12-19T18:17:39.829226+0000","last_became_peered":"2023-12-19T18:17:39.829226+0000","last_unstale":"2023-12-19T18:17:39.833651+0000","last_undegraded":"2023-12-19T18:17:39.833651+0000","last_fullsized":"2023-12-19T18:17:39.833651+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:09:23.641063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.414643+0000","last_change":"2023-12-19T18:17:27.360694+0000","last_active":"2023-12-19T18:17:39.414643+0000","last_peered":"2023-12-19T18:17:39.414643+0000","last_clean":"2023-12-19T18:17:39.414643+0000","last_became_active":"2023-12-19T18:17:27.357706+0000","last_became_peered":"2023-12-19T18:17:27.357706+0000","last_unstale":"2023-12-19T18:17:39.414643+0000","last_undegraded":"2023-12-19T18:17:39.414643+0000","last_fullsized":"2023-12-19T18:17:39.414643+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:22:01.458247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809006+0000","last_change":"2023-12-19T18:17:39.838929+0000","last_active":"2023-12-19T18:17:40.809006+0000","last_peered":"2023-12-19T18:17:40.809006+0000","last_clean":"2023-12-19T18:17:40.809006+0000","last_became_active":"2023-12-19T18:17:39.838626+0000","last_became_peered":"2023-12-19T18:17:39.838626+0000","last_unstale":"2023-12-19T18:17:40.809006+0000","last_undegraded":"2023-12-19T18:17:40.809006+0000","last_fullsized":"2023-12-19T18:17:40.809006+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:51:43.486308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413594+0000","last_change":"2023-12-19T18:17:27.361414+0000","last_active":"2023-12-19T18:17:39.413594+0000","last_peered":"2023-12-19T18:17:39.413594+0000","last_clean":"2023-12-19T18:17:39.413594+0000","last_became_active":"2023-12-19T18:17:27.361054+0000","last_became_peered":"2023-12-19T18:17:27.361054+0000","last_unstale":"2023-12-19T18:17:39.413594+0000","last_undegraded":"2023-12-19T18:17:39.413594+0000","last_fullsized":"2023-12-19T18:17:39.413594+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:04:33.062599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.811709+0000","last_change":"2023-12-19T18:17:39.832398+0000","last_active":"2023-12-19T18:17:40.811709+0000","last_peered":"2023-12-19T18:17:40.811709+0000","last_clean":"2023-12-19T18:17:40.811709+0000","last_became_active":"2023-12-19T18:17:39.832061+0000","last_became_peered":"2023-12-19T18:17:39.832061+0000","last_unstale":"2023-12-19T18:17:40.811709+0000","last_undegraded":"2023-12-19T18:17:40.811709+0000","last_fullsized":"2023-12-19T18:17:40.811709+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:10:02.886603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.831200+0000","last_change":"2023-12-19T18:17:27.359512+0000","last_active":"2023-12-19T18:17:39.831200+0000","last_peered":"2023-12-19T18:17:39.831200+0000","last_clean":"2023-12-19T18:17:39.831200+0000","last_became_active":"2023-12-19T18:17:27.359173+0000","last_became_peered":"2023-12-19T18:17:27.359173+0000","last_unstale":"2023-12-19T18:17:39.831200+0000","last_undegraded":"2023-12-19T18:17:39.831200+0000","last_fullsized":"2023-12-19T18:17:39.831200+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:42.446500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809011+0000","last_change":"2023-12-19T18:17:39.839100+0000","last_active":"2023-12-19T18:17:40.809011+0000","last_peered":"2023-12-19T18:17:40.809011+0000","last_clean":"2023-12-19T18:17:40.809011+0000","last_became_active":"2023-12-19T18:17:39.838793+0000","last_became_peered":"2023-12-19T18:17:39.838793+0000","last_unstale":"2023-12-19T18:17:40.809011+0000","last_undegraded":"2023-12-19T18:17:40.809011+0000","last_fullsized":"2023-12-19T18:17:40.809011+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:40:14.968536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"236'2","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.835110+0000","last_change":"2023-12-19T18:17:39.422159+0000","last_active":"2023-12-19T18:17:39.835110+0000","last_peered":"2023-12-19T18:17:39.835110+0000","last_clean":"2023-12-19T18:17:39.835110+0000","last_became_active":"2023-12-19T18:17:39.421685+0000","last_became_peered":"2023-12-19T18:17:39.421685+0000","last_unstale":"2023-12-19T18:17:39.835110+0000","last_undegraded":"2023-12-19T18:17:39.835110+0000","last_fullsized":"2023-12-19T18:17:39.835110+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:09.331836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.824721+0000","last_change":"2023-12-19T18:17:27.355393+0000","last_active":"2023-12-19T18:17:39.824721+0000","last_peered":"2023-12-19T18:17:39.824721+0000","last_clean":"2023-12-19T18:17:39.824721+0000","last_became_active":"2023-12-19T18:17:27.355013+0000","last_became_peered":"2023-12-19T18:17:27.355013+0000","last_unstale":"2023-12-19T18:17:39.824721+0000","last_undegraded":"2023-12-19T18:17:39.824721+0000","last_fullsized":"2023-12-19T18:17:39.824721+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:49.759094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"232'4","reported_seq":19,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.833813+0000","last_change":"2023-12-19T18:17:26.951474+0000","last_active":"2023-12-19T18:17:39.833813+0000","last_peered":"2023-12-19T18:17:39.833813+0000","last_clean":"2023-12-19T18:17:39.833813+0000","last_became_active":"2023-12-19T18:17:26.950990+0000","last_became_peered":"2023-12-19T18:17:26.950990+0000","last_unstale":"2023-12-19T18:17:39.833813+0000","last_undegraded":"2023-12-19T18:17:39.833813+0000","last_fullsized":"2023-12-19T18:17:39.833813+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:37:49.322125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.841995+0000","last_change":"2023-12-19T18:17:39.841012+0000","last_active":"2023-12-19T18:17:39.841995+0000","last_peered":"2023-12-19T18:17:39.841995+0000","last_clean":"2023-12-19T18:17:39.841995+0000","last_became_active":"2023-12-19T18:17:39.840709+0000","last_became_peered":"2023-12-19T18:17:39.840709+0000","last_unstale":"2023-12-19T18:17:39.841995+0000","last_undegraded":"2023-12-19T18:17:39.841995+0000","last_fullsized":"2023-12-19T18:17:39.841995+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:56.410749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"236'2","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.831379+0000","last_change":"2023-12-19T18:17:39.422455+0000","last_active":"2023-12-19T18:17:39.831379+0000","last_peered":"2023-12-19T18:17:39.831379+0000","last_clean":"2023-12-19T18:17:39.831379+0000","last_became_active":"2023-12-19T18:17:39.422159+0000","last_became_peered":"2023-12-19T18:17:39.422159+0000","last_unstale":"2023-12-19T18:17:39.831379+0000","last_undegraded":"2023-12-19T18:17:39.831379+0000","last_fullsized":"2023-12-19T18:17:39.831379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:37.101616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.411522+0000","last_change":"2023-12-19T18:17:27.346026+0000","last_active":"2023-12-19T18:17:39.411522+0000","last_peered":"2023-12-19T18:17:39.411522+0000","last_clean":"2023-12-19T18:17:39.411522+0000","last_became_active":"2023-12-19T18:17:27.345752+0000","last_became_peered":"2023-12-19T18:17:27.345752+0000","last_unstale":"2023-12-19T18:17:39.411522+0000","last_undegraded":"2023-12-19T18:17:39.411522+0000","last_fullsized":"2023-12-19T18:17:39.411522+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:18.690231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"232'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.410197+0000","last_change":"2023-12-19T18:17:26.946867+0000","last_active":"2023-12-19T18:17:39.410197+0000","last_peered":"2023-12-19T18:17:39.410197+0000","last_clean":"2023-12-19T18:17:39.410197+0000","last_became_active":"2023-12-19T18:17:26.946549+0000","last_became_peered":"2023-12-19T18:17:26.946549+0000","last_unstale":"2023-12-19T18:17:39.410197+0000","last_undegraded":"2023-12-19T18:17:39.410197+0000","last_fullsized":"2023-12-19T18:17:39.410197+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:36.912904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830232+0000","last_change":"2023-12-19T18:17:39.826375+0000","last_active":"2023-12-19T18:17:39.830232+0000","last_peered":"2023-12-19T18:17:39.830232+0000","last_clean":"2023-12-19T18:17:39.830232+0000","last_became_active":"2023-12-19T18:17:39.822261+0000","last_became_peered":"2023-12-19T18:17:39.822261+0000","last_unstale":"2023-12-19T18:17:39.830232+0000","last_undegraded":"2023-12-19T18:17:39.830232+0000","last_fullsized":"2023-12-19T18:17:39.830232+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:46:57.405646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"236'1","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.813473+0000","last_change":"2023-12-19T18:17:39.422538+0000","last_active":"2023-12-19T18:17:40.813473+0000","last_peered":"2023-12-19T18:17:40.813473+0000","last_clean":"2023-12-19T18:17:40.813473+0000","last_became_active":"2023-12-19T18:17:39.422010+0000","last_became_peered":"2023-12-19T18:17:39.422010+0000","last_unstale":"2023-12-19T18:17:40.813473+0000","last_undegraded":"2023-12-19T18:17:40.813473+0000","last_fullsized":"2023-12-19T18:17:40.813473+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:43.061826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810743+0000","last_change":"2023-12-19T18:17:27.369534+0000","last_active":"2023-12-19T18:17:39.810743+0000","last_peered":"2023-12-19T18:17:39.810743+0000","last_clean":"2023-12-19T18:17:39.810743+0000","last_became_active":"2023-12-19T18:17:27.369101+0000","last_became_peered":"2023-12-19T18:17:27.369101+0000","last_unstale":"2023-12-19T18:17:39.810743+0000","last_undegraded":"2023-12-19T18:17:39.810743+0000","last_fullsized":"2023-12-19T18:17:39.810743+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:38:39.157091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"232'2","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408926+0000","last_change":"2023-12-19T18:17:26.947910+0000","last_active":"2023-12-19T18:17:39.408926+0000","last_peered":"2023-12-19T18:17:39.408926+0000","last_clean":"2023-12-19T18:17:39.408926+0000","last_became_active":"2023-12-19T18:17:26.946840+0000","last_became_peered":"2023-12-19T18:17:26.946840+0000","last_unstale":"2023-12-19T18:17:39.408926+0000","last_undegraded":"2023-12-19T18:17:39.408926+0000","last_fullsized":"2023-12-19T18:17:39.408926+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:41.372298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.838243+0000","last_change":"2023-12-19T18:17:39.833636+0000","last_active":"2023-12-19T18:17:39.838243+0000","last_peered":"2023-12-19T18:17:39.838243+0000","last_clean":"2023-12-19T18:17:39.838243+0000","last_became_active":"2023-12-19T18:17:39.833142+0000","last_became_peered":"2023-12-19T18:17:39.833142+0000","last_unstale":"2023-12-19T18:17:39.838243+0000","last_undegraded":"2023-12-19T18:17:39.838243+0000","last_fullsized":"2023-12-19T18:17:39.838243+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:23.184958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812458+0000","last_change":"2023-12-19T18:17:39.421070+0000","last_active":"2023-12-19T18:17:40.812458+0000","last_peered":"2023-12-19T18:17:40.812458+0000","last_clean":"2023-12-19T18:17:40.812458+0000","last_became_active":"2023-12-19T18:17:39.419992+0000","last_became_peered":"2023-12-19T18:17:39.419992+0000","last_unstale":"2023-12-19T18:17:40.812458+0000","last_undegraded":"2023-12-19T18:17:40.812458+0000","last_fullsized":"2023-12-19T18:17:40.812458+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:23:37.568299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.415498+0000","last_change":"2023-12-19T18:17:27.361418+0000","last_active":"2023-12-19T18:17:39.415498+0000","last_peered":"2023-12-19T18:17:39.415498+0000","last_clean":"2023-12-19T18:17:39.415498+0000","last_became_active":"2023-12-19T18:17:27.361131+0000","last_became_peered":"2023-12-19T18:17:27.361131+0000","last_unstale":"2023-12-19T18:17:39.415498+0000","last_undegraded":"2023-12-19T18:17:39.415498+0000","last_fullsized":"2023-12-19T18:17:39.415498+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:44:06.399365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"232'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.410498+0000","last_change":"2023-12-19T18:17:26.942982+0000","last_active":"2023-12-19T18:17:39.410498+0000","last_peered":"2023-12-19T18:17:39.410498+0000","last_clean":"2023-12-19T18:17:39.410498+0000","last_became_active":"2023-12-19T18:17:26.942719+0000","last_became_peered":"2023-12-19T18:17:26.942719+0000","last_unstale":"2023-12-19T18:17:39.410498+0000","last_undegraded":"2023-12-19T18:17:39.410498+0000","last_fullsized":"2023-12-19T18:17:39.410498+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:10.474482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.835751+0000","last_change":"2023-12-19T18:17:39.833354+0000","last_active":"2023-12-19T18:17:39.835751+0000","last_peered":"2023-12-19T18:17:39.835751+0000","last_clean":"2023-12-19T18:17:39.835751+0000","last_became_active":"2023-12-19T18:17:39.833103+0000","last_became_peered":"2023-12-19T18:17:39.833103+0000","last_unstale":"2023-12-19T18:17:39.835751+0000","last_undegraded":"2023-12-19T18:17:39.835751+0000","last_fullsized":"2023-12-19T18:17:39.835751+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:40:18.669674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.813201+0000","last_change":"2023-12-19T18:17:39.423176+0000","last_active":"2023-12-19T18:17:39.813201+0000","last_peered":"2023-12-19T18:17:39.813201+0000","last_clean":"2023-12-19T18:17:39.813201+0000","last_became_active":"2023-12-19T18:17:39.422890+0000","last_became_peered":"2023-12-19T18:17:39.422890+0000","last_unstale":"2023-12-19T18:17:39.813201+0000","last_undegraded":"2023-12-19T18:17:39.813201+0000","last_fullsized":"2023-12-19T18:17:39.813201+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:14:24.154817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413688+0000","last_change":"2023-12-19T18:17:27.361092+0000","last_active":"2023-12-19T18:17:39.413688+0000","last_peered":"2023-12-19T18:17:39.413688+0000","last_clean":"2023-12-19T18:17:39.413688+0000","last_became_active":"2023-12-19T18:17:27.360778+0000","last_became_peered":"2023-12-19T18:17:27.360778+0000","last_unstale":"2023-12-19T18:17:39.413688+0000","last_undegraded":"2023-12-19T18:17:39.413688+0000","last_fullsized":"2023-12-19T18:17:39.413688+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:40.489404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"232'1","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.819128+0000","last_change":"2023-12-19T18:17:26.940960+0000","last_active":"2023-12-19T18:17:39.819128+0000","last_peered":"2023-12-19T18:17:39.819128+0000","last_clean":"2023-12-19T18:17:39.819128+0000","last_became_active":"2023-12-19T18:17:26.939915+0000","last_became_peered":"2023-12-19T18:17:26.939915+0000","last_unstale":"2023-12-19T18:17:39.819128+0000","last_undegraded":"2023-12-19T18:17:39.819128+0000","last_fullsized":"2023-12-19T18:17:39.819128+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:52:17.872321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.808930+0000","last_change":"2023-12-19T18:17:39.830173+0000","last_active":"2023-12-19T18:17:40.808930+0000","last_peered":"2023-12-19T18:17:40.808930+0000","last_clean":"2023-12-19T18:17:40.808930+0000","last_became_active":"2023-12-19T18:17:39.829840+0000","last_became_peered":"2023-12-19T18:17:39.829840+0000","last_unstale":"2023-12-19T18:17:40.808930+0000","last_undegraded":"2023-12-19T18:17:40.808930+0000","last_fullsized":"2023-12-19T18:17:40.808930+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:14:56.855127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"236'1","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.844024+0000","last_change":"2023-12-19T18:17:39.422364+0000","last_active":"2023-12-19T18:17:39.844024+0000","last_peered":"2023-12-19T18:17:39.844024+0000","last_clean":"2023-12-19T18:17:39.844024+0000","last_became_active":"2023-12-19T18:17:39.422059+0000","last_became_peered":"2023-12-19T18:17:39.422059+0000","last_unstale":"2023-12-19T18:17:39.844024+0000","last_undegraded":"2023-12-19T18:17:39.844024+0000","last_fullsized":"2023-12-19T18:17:39.844024+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:09:13.480050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.404579+0000","last_change":"2023-12-19T18:17:27.347577+0000","last_active":"2023-12-19T18:17:39.404579+0000","last_peered":"2023-12-19T18:17:39.404579+0000","last_clean":"2023-12-19T18:17:39.404579+0000","last_became_active":"2023-12-19T18:17:27.346549+0000","last_became_peered":"2023-12-19T18:17:27.346549+0000","last_unstale":"2023-12-19T18:17:39.404579+0000","last_undegraded":"2023-12-19T18:17:39.404579+0000","last_fullsized":"2023-12-19T18:17:39.404579+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:28.649300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.811009+0000","last_change":"2023-12-19T18:17:26.950047+0000","last_active":"2023-12-19T18:17:39.811009+0000","last_peered":"2023-12-19T18:17:39.811009+0000","last_clean":"2023-12-19T18:17:39.811009+0000","last_became_active":"2023-12-19T18:17:26.949678+0000","last_became_peered":"2023-12-19T18:17:26.949678+0000","last_unstale":"2023-12-19T18:17:39.811009+0000","last_undegraded":"2023-12-19T18:17:39.811009+0000","last_fullsized":"2023-12-19T18:17:39.811009+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:49:16.046354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.846622+0000","last_change":"2023-12-19T18:17:39.845533+0000","last_active":"2023-12-19T18:17:39.846622+0000","last_peered":"2023-12-19T18:17:39.846622+0000","last_clean":"2023-12-19T18:17:39.846622+0000","last_became_active":"2023-12-19T18:17:39.845197+0000","last_became_peered":"2023-12-19T18:17:39.845197+0000","last_unstale":"2023-12-19T18:17:39.846622+0000","last_undegraded":"2023-12-19T18:17:39.846622+0000","last_fullsized":"2023-12-19T18:17:39.846622+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:27.280365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"236'1","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834552+0000","last_change":"2023-12-19T18:17:39.423872+0000","last_active":"2023-12-19T18:17:39.834552+0000","last_peered":"2023-12-19T18:17:39.834552+0000","last_clean":"2023-12-19T18:17:39.834552+0000","last_became_active":"2023-12-19T18:17:39.423556+0000","last_became_peered":"2023-12-19T18:17:39.423556+0000","last_unstale":"2023-12-19T18:17:39.834552+0000","last_undegraded":"2023-12-19T18:17:39.834552+0000","last_fullsized":"2023-12-19T18:17:39.834552+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:55:50.947760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.407857+0000","last_change":"2023-12-19T18:17:27.363788+0000","last_active":"2023-12-19T18:17:39.407857+0000","last_peered":"2023-12-19T18:17:39.407857+0000","last_clean":"2023-12-19T18:17:39.407857+0000","last_became_active":"2023-12-19T18:17:27.363513+0000","last_became_peered":"2023-12-19T18:17:27.363513+0000","last_unstale":"2023-12-19T18:17:39.407857+0000","last_undegraded":"2023-12-19T18:17:39.407857+0000","last_fullsized":"2023-12-19T18:17:39.407857+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:23.278389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.404674+0000","last_change":"2023-12-19T18:17:26.944745+0000","last_active":"2023-12-19T18:17:39.404674+0000","last_peered":"2023-12-19T18:17:39.404674+0000","last_clean":"2023-12-19T18:17:39.404674+0000","last_became_active":"2023-12-19T18:17:26.944481+0000","last_became_peered":"2023-12-19T18:17:26.944481+0000","last_unstale":"2023-12-19T18:17:39.404674+0000","last_undegraded":"2023-12-19T18:17:39.404674+0000","last_fullsized":"2023-12-19T18:17:39.404674+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:30.604242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.843343+0000","last_change":"2023-12-19T18:17:39.841724+0000","last_active":"2023-12-19T18:17:39.843343+0000","last_peered":"2023-12-19T18:17:39.843343+0000","last_clean":"2023-12-19T18:17:39.843343+0000","last_became_active":"2023-12-19T18:17:39.841465+0000","last_became_peered":"2023-12-19T18:17:39.841465+0000","last_unstale":"2023-12-19T18:17:39.843343+0000","last_undegraded":"2023-12-19T18:17:39.843343+0000","last_fullsized":"2023-12-19T18:17:39.843343+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:04:11.535860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"236'3","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.840966+0000","last_change":"2023-12-19T18:17:39.417418+0000","last_active":"2023-12-19T18:17:39.840966+0000","last_peered":"2023-12-19T18:17:39.840966+0000","last_clean":"2023-12-19T18:17:39.840966+0000","last_became_active":"2023-12-19T18:17:39.416743+0000","last_became_peered":"2023-12-19T18:17:39.416743+0000","last_unstale":"2023-12-19T18:17:39.840966+0000","last_undegraded":"2023-12-19T18:17:39.840966+0000","last_fullsized":"2023-12-19T18:17:39.840966+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:10.544888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.410105+0000","last_change":"2023-12-19T18:17:27.356985+0000","last_active":"2023-12-19T18:17:39.410105+0000","last_peered":"2023-12-19T18:17:39.410105+0000","last_clean":"2023-12-19T18:17:39.410105+0000","last_became_active":"2023-12-19T18:17:27.356695+0000","last_became_peered":"2023-12-19T18:17:27.356695+0000","last_unstale":"2023-12-19T18:17:39.410105+0000","last_undegraded":"2023-12-19T18:17:39.410105+0000","last_fullsized":"2023-12-19T18:17:39.410105+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:31.153454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413880+0000","last_change":"2023-12-19T18:17:26.944872+0000","last_active":"2023-12-19T18:17:39.413880+0000","last_peered":"2023-12-19T18:17:39.413880+0000","last_clean":"2023-12-19T18:17:39.413880+0000","last_became_active":"2023-12-19T18:17:26.944558+0000","last_became_peered":"2023-12-19T18:17:26.944558+0000","last_unstale":"2023-12-19T18:17:39.413880+0000","last_undegraded":"2023-12-19T18:17:39.413880+0000","last_fullsized":"2023-12-19T18:17:39.413880+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:30.151215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.813139+0000","last_change":"2023-12-19T18:17:39.833936+0000","last_active":"2023-12-19T18:17:40.813139+0000","last_peered":"2023-12-19T18:17:40.813139+0000","last_clean":"2023-12-19T18:17:40.813139+0000","last_became_active":"2023-12-19T18:17:39.833640+0000","last_became_peered":"2023-12-19T18:17:39.833640+0000","last_unstale":"2023-12-19T18:17:40.813139+0000","last_undegraded":"2023-12-19T18:17:40.813139+0000","last_fullsized":"2023-12-19T18:17:40.813139+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:37.624327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"236'6","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.847428+0000","last_change":"2023-12-19T18:17:39.423807+0000","last_active":"2023-12-19T18:17:39.847428+0000","last_peered":"2023-12-19T18:17:39.847428+0000","last_clean":"2023-12-19T18:17:39.847428+0000","last_became_active":"2023-12-19T18:17:39.423499+0000","last_became_peered":"2023-12-19T18:17:39.423499+0000","last_unstale":"2023-12-19T18:17:39.847428+0000","last_undegraded":"2023-12-19T18:17:39.847428+0000","last_fullsized":"2023-12-19T18:17:39.847428+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:50:37.326872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413790+0000","last_change":"2023-12-19T18:17:27.342915+0000","last_active":"2023-12-19T18:17:39.413790+0000","last_peered":"2023-12-19T18:17:39.413790+0000","last_clean":"2023-12-19T18:17:39.413790+0000","last_became_active":"2023-12-19T18:17:27.342549+0000","last_became_peered":"2023-12-19T18:17:27.342549+0000","last_unstale":"2023-12-19T18:17:39.413790+0000","last_undegraded":"2023-12-19T18:17:39.413790+0000","last_fullsized":"2023-12-19T18:17:39.413790+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:03.786746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830971+0000","last_change":"2023-12-19T18:17:26.943010+0000","last_active":"2023-12-19T18:17:39.830971+0000","last_peered":"2023-12-19T18:17:39.830971+0000","last_clean":"2023-12-19T18:17:39.830971+0000","last_became_active":"2023-12-19T18:17:26.942721+0000","last_became_peered":"2023-12-19T18:17:26.942721+0000","last_unstale":"2023-12-19T18:17:39.830971+0000","last_undegraded":"2023-12-19T18:17:39.830971+0000","last_fullsized":"2023-12-19T18:17:39.830971+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:09:16.907014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.826155+0000","last_change":"2023-12-19T18:17:39.818981+0000","last_active":"2023-12-19T18:17:39.826155+0000","last_peered":"2023-12-19T18:17:39.826155+0000","last_clean":"2023-12-19T18:17:39.826155+0000","last_became_active":"2023-12-19T18:17:39.818715+0000","last_became_peered":"2023-12-19T18:17:39.818715+0000","last_unstale":"2023-12-19T18:17:39.826155+0000","last_undegraded":"2023-12-19T18:17:39.826155+0000","last_fullsized":"2023-12-19T18:17:39.826155+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:45.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.813268+0000","last_change":"2023-12-19T18:17:39.418260+0000","last_active":"2023-12-19T18:17:39.813268+0000","last_peered":"2023-12-19T18:17:39.813268+0000","last_clean":"2023-12-19T18:17:39.813268+0000","last_became_active":"2023-12-19T18:17:39.417966+0000","last_became_peered":"2023-12-19T18:17:39.417966+0000","last_unstale":"2023-12-19T18:17:39.813268+0000","last_undegraded":"2023-12-19T18:17:39.813268+0000","last_fullsized":"2023-12-19T18:17:39.813268+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:18.010498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810034+0000","last_change":"2023-12-19T18:17:27.368101+0000","last_active":"2023-12-19T18:17:39.810034+0000","last_peered":"2023-12-19T18:17:39.810034+0000","last_clean":"2023-12-19T18:17:39.810034+0000","last_became_active":"2023-12-19T18:17:27.367822+0000","last_became_peered":"2023-12-19T18:17:27.367822+0000","last_unstale":"2023-12-19T18:17:39.810034+0000","last_undegraded":"2023-12-19T18:17:39.810034+0000","last_fullsized":"2023-12-19T18:17:39.810034+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:11:20.286347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"232'3","reported_seq":17,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413647+0000","last_change":"2023-12-19T18:17:26.938268+0000","last_active":"2023-12-19T18:17:39.413647+0000","last_peered":"2023-12-19T18:17:39.413647+0000","last_clean":"2023-12-19T18:17:39.413647+0000","last_became_active":"2023-12-19T18:17:26.937724+0000","last_became_peered":"2023-12-19T18:17:26.937724+0000","last_unstale":"2023-12-19T18:17:39.413647+0000","last_undegraded":"2023-12-19T18:17:39.413647+0000","last_fullsized":"2023-12-19T18:17:39.413647+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:05.552913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809400+0000","last_change":"2023-12-19T18:17:39.842511+0000","last_active":"2023-12-19T18:17:40.809400+0000","last_peered":"2023-12-19T18:17:40.809400+0000","last_clean":"2023-12-19T18:17:40.809400+0000","last_became_active":"2023-12-19T18:17:39.839597+0000","last_became_peered":"2023-12-19T18:17:39.839597+0000","last_unstale":"2023-12-19T18:17:40.809400+0000","last_undegraded":"2023-12-19T18:17:40.809400+0000","last_fullsized":"2023-12-19T18:17:40.809400+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:17:06.505163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.816182+0000","last_change":"2023-12-19T18:17:39.426212+0000","last_active":"2023-12-19T18:17:39.816182+0000","last_peered":"2023-12-19T18:17:39.816182+0000","last_clean":"2023-12-19T18:17:39.816182+0000","last_became_active":"2023-12-19T18:17:39.425383+0000","last_became_peered":"2023-12-19T18:17:39.425383+0000","last_unstale":"2023-12-19T18:17:39.816182+0000","last_undegraded":"2023-12-19T18:17:39.816182+0000","last_fullsized":"2023-12-19T18:17:39.816182+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:33:57.603913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.415753+0000","last_change":"2023-12-19T18:17:27.344496+0000","last_active":"2023-12-19T18:17:39.415753+0000","last_peered":"2023-12-19T18:17:39.415753+0000","last_clean":"2023-12-19T18:17:39.415753+0000","last_became_active":"2023-12-19T18:17:27.343611+0000","last_became_peered":"2023-12-19T18:17:27.343611+0000","last_unstale":"2023-12-19T18:17:39.415753+0000","last_undegraded":"2023-12-19T18:17:39.415753+0000","last_fullsized":"2023-12-19T18:17:39.415753+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:26:28.887778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"233'8","reported_seq":22,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416379+0000","last_change":"2023-12-19T18:17:26.941650+0000","last_active":"2023-12-19T18:17:39.416379+0000","last_peered":"2023-12-19T18:17:39.416379+0000","last_clean":"2023-12-19T18:17:39.416379+0000","last_became_active":"2023-12-19T18:17:26.941388+0000","last_became_peered":"2023-12-19T18:17:26.941388+0000","last_unstale":"2023-12-19T18:17:39.416379+0000","last_undegraded":"2023-12-19T18:17:39.416379+0000","last_fullsized":"2023-12-19T18:17:39.416379+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:35:14.834857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830961+0000","last_change":"2023-12-19T18:17:39.823276+0000","last_active":"2023-12-19T18:17:39.830961+0000","last_peered":"2023-12-19T18:17:39.830961+0000","last_clean":"2023-12-19T18:17:39.830961+0000","last_became_active":"2023-12-19T18:17:39.823022+0000","last_became_peered":"2023-12-19T18:17:39.823022+0000","last_unstale":"2023-12-19T18:17:39.830961+0000","last_undegraded":"2023-12-19T18:17:39.830961+0000","last_fullsized":"2023-12-19T18:17:39.830961+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:03:47.613528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812535+0000","last_change":"2023-12-19T18:17:39.425705+0000","last_active":"2023-12-19T18:17:40.812535+0000","last_peered":"2023-12-19T18:17:40.812535+0000","last_clean":"2023-12-19T18:17:40.812535+0000","last_became_active":"2023-12-19T18:17:39.425418+0000","last_became_peered":"2023-12-19T18:17:39.425418+0000","last_unstale":"2023-12-19T18:17:40.812535+0000","last_undegraded":"2023-12-19T18:17:40.812535+0000","last_fullsized":"2023-12-19T18:17:40.812535+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:36.472195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834202+0000","last_change":"2023-12-19T18:17:27.350307+0000","last_active":"2023-12-19T18:17:39.834202+0000","last_peered":"2023-12-19T18:17:39.834202+0000","last_clean":"2023-12-19T18:17:39.834202+0000","last_became_active":"2023-12-19T18:17:27.350036+0000","last_became_peered":"2023-12-19T18:17:27.350036+0000","last_unstale":"2023-12-19T18:17:39.834202+0000","last_undegraded":"2023-12-19T18:17:39.834202+0000","last_fullsized":"2023-12-19T18:17:39.834202+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:24:50.816850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"232'2","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.410595+0000","last_change":"2023-12-19T18:17:26.941112+0000","last_active":"2023-12-19T18:17:39.410595+0000","last_peered":"2023-12-19T18:17:39.410595+0000","last_clean":"2023-12-19T18:17:39.410595+0000","last_became_active":"2023-12-19T18:17:26.940838+0000","last_became_peered":"2023-12-19T18:17:26.940838+0000","last_unstale":"2023-12-19T18:17:39.410595+0000","last_undegraded":"2023-12-19T18:17:39.410595+0000","last_fullsized":"2023-12-19T18:17:39.410595+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:47:48.785673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812383+0000","last_change":"2023-12-19T18:17:39.832557+0000","last_active":"2023-12-19T18:17:40.812383+0000","last_peered":"2023-12-19T18:17:40.812383+0000","last_clean":"2023-12-19T18:17:40.812383+0000","last_became_active":"2023-12-19T18:17:39.832288+0000","last_became_peered":"2023-12-19T18:17:39.832288+0000","last_unstale":"2023-12-19T18:17:40.812383+0000","last_undegraded":"2023-12-19T18:17:40.812383+0000","last_fullsized":"2023-12-19T18:17:40.812383+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:13:14.873886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"236'8","reported_seq":19,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.808845+0000","last_change":"2023-12-19T18:17:39.415764+0000","last_active":"2023-12-19T18:17:40.808845+0000","last_peered":"2023-12-19T18:17:40.808845+0000","last_clean":"2023-12-19T18:17:40.808845+0000","last_became_active":"2023-12-19T18:17:39.414955+0000","last_became_peered":"2023-12-19T18:17:39.414955+0000","last_unstale":"2023-12-19T18:17:40.808845+0000","last_undegraded":"2023-12-19T18:17:40.808845+0000","last_fullsized":"2023-12-19T18:17:40.808845+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:08.661657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820226+0000","last_change":"2023-12-19T18:17:27.362157+0000","last_active":"2023-12-19T18:17:39.820226+0000","last_peered":"2023-12-19T18:17:39.820226+0000","last_clean":"2023-12-19T18:17:39.820226+0000","last_became_active":"2023-12-19T18:17:27.361879+0000","last_became_peered":"2023-12-19T18:17:27.361879+0000","last_unstale":"2023-12-19T18:17:39.820226+0000","last_undegraded":"2023-12-19T18:17:39.820226+0000","last_fullsized":"2023-12-19T18:17:39.820226+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:00.178479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834282+0000","last_change":"2023-12-19T18:17:26.948613+0000","last_active":"2023-12-19T18:17:39.834282+0000","last_peered":"2023-12-19T18:17:39.834282+0000","last_clean":"2023-12-19T18:17:39.834282+0000","last_became_active":"2023-12-19T18:17:26.948318+0000","last_became_peered":"2023-12-19T18:17:26.948318+0000","last_unstale":"2023-12-19T18:17:39.834282+0000","last_undegraded":"2023-12-19T18:17:39.834282+0000","last_fullsized":"2023-12-19T18:17:39.834282+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:13:19.247070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812745+0000","last_change":"2023-12-19T18:17:39.821325+0000","last_active":"2023-12-19T18:17:40.812745+0000","last_peered":"2023-12-19T18:17:40.812745+0000","last_clean":"2023-12-19T18:17:40.812745+0000","last_became_active":"2023-12-19T18:17:39.821042+0000","last_became_peered":"2023-12-19T18:17:39.821042+0000","last_unstale":"2023-12-19T18:17:40.812745+0000","last_undegraded":"2023-12-19T18:17:40.812745+0000","last_fullsized":"2023-12-19T18:17:40.812745+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:07:21.803515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"236'2","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.845920+0000","last_change":"2023-12-19T18:17:39.415614+0000","last_active":"2023-12-19T18:17:39.845920+0000","last_peered":"2023-12-19T18:17:39.845920+0000","last_clean":"2023-12-19T18:17:39.845920+0000","last_became_active":"2023-12-19T18:17:39.414534+0000","last_became_peered":"2023-12-19T18:17:39.414534+0000","last_unstale":"2023-12-19T18:17:39.845920+0000","last_undegraded":"2023-12-19T18:17:39.845920+0000","last_fullsized":"2023-12-19T18:17:39.845920+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:25:07.972942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.819746+0000","last_change":"2023-12-19T18:17:27.361736+0000","last_active":"2023-12-19T18:17:39.819746+0000","last_peered":"2023-12-19T18:17:39.819746+0000","last_clean":"2023-12-19T18:17:39.819746+0000","last_became_active":"2023-12-19T18:17:27.361355+0000","last_became_peered":"2023-12-19T18:17:27.361355+0000","last_unstale":"2023-12-19T18:17:39.819746+0000","last_undegraded":"2023-12-19T18:17:39.819746+0000","last_fullsized":"2023-12-19T18:17:39.819746+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:02.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413769+0000","last_change":"2023-12-19T18:17:26.944351+0000","last_active":"2023-12-19T18:17:39.413769+0000","last_peered":"2023-12-19T18:17:39.413769+0000","last_clean":"2023-12-19T18:17:39.413769+0000","last_became_active":"2023-12-19T18:17:26.943692+0000","last_became_peered":"2023-12-19T18:17:26.943692+0000","last_unstale":"2023-12-19T18:17:39.413769+0000","last_undegraded":"2023-12-19T18:17:39.413769+0000","last_fullsized":"2023-12-19T18:17:39.413769+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:54.040380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.835209+0000","last_change":"2023-12-19T18:17:39.832355+0000","last_active":"2023-12-19T18:17:39.835209+0000","last_peered":"2023-12-19T18:17:39.835209+0000","last_clean":"2023-12-19T18:17:39.835209+0000","last_became_active":"2023-12-19T18:17:39.832058+0000","last_became_peered":"2023-12-19T18:17:39.832058+0000","last_unstale":"2023-12-19T18:17:39.835209+0000","last_undegraded":"2023-12-19T18:17:39.835209+0000","last_fullsized":"2023-12-19T18:17:39.835209+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:21:07.719679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.814321+0000","last_change":"2023-12-19T18:17:39.420031+0000","last_active":"2023-12-19T18:17:39.814321+0000","last_peered":"2023-12-19T18:17:39.814321+0000","last_clean":"2023-12-19T18:17:39.814321+0000","last_became_active":"2023-12-19T18:17:39.419757+0000","last_became_peered":"2023-12-19T18:17:39.419757+0000","last_unstale":"2023-12-19T18:17:39.814321+0000","last_undegraded":"2023-12-19T18:17:39.814321+0000","last_fullsized":"2023-12-19T18:17:39.814321+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:45.886255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.811442+0000","last_change":"2023-12-19T18:17:27.366044+0000","last_active":"2023-12-19T18:17:39.811442+0000","last_peered":"2023-12-19T18:17:39.811442+0000","last_clean":"2023-12-19T18:17:39.811442+0000","last_became_active":"2023-12-19T18:17:27.365761+0000","last_became_peered":"2023-12-19T18:17:27.365761+0000","last_unstale":"2023-12-19T18:17:39.811442+0000","last_undegraded":"2023-12-19T18:17:39.811442+0000","last_fullsized":"2023-12-19T18:17:39.811442+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:05:07.652398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"232'8","reported_seq":22,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408317+0000","last_change":"2023-12-19T18:17:26.948258+0000","last_active":"2023-12-19T18:17:39.408317+0000","last_peered":"2023-12-19T18:17:39.408317+0000","last_clean":"2023-12-19T18:17:39.408317+0000","last_became_active":"2023-12-19T18:17:26.947974+0000","last_became_peered":"2023-12-19T18:17:26.947974+0000","last_unstale":"2023-12-19T18:17:39.408317+0000","last_undegraded":"2023-12-19T18:17:39.408317+0000","last_fullsized":"2023-12-19T18:17:39.408317+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:49.244280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.835313+0000","last_change":"2023-12-19T18:17:39.832119+0000","last_active":"2023-12-19T18:17:39.835313+0000","last_peered":"2023-12-19T18:17:39.835313+0000","last_clean":"2023-12-19T18:17:39.835313+0000","last_became_active":"2023-12-19T18:17:39.831851+0000","last_became_peered":"2023-12-19T18:17:39.831851+0000","last_unstale":"2023-12-19T18:17:39.835313+0000","last_undegraded":"2023-12-19T18:17:39.835313+0000","last_fullsized":"2023-12-19T18:17:39.835313+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:44.450250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"236'1","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834104+0000","last_change":"2023-12-19T18:17:39.425937+0000","last_active":"2023-12-19T18:17:39.834104+0000","last_peered":"2023-12-19T18:17:39.834104+0000","last_clean":"2023-12-19T18:17:39.834104+0000","last_became_active":"2023-12-19T18:17:39.423851+0000","last_became_peered":"2023-12-19T18:17:39.423851+0000","last_unstale":"2023-12-19T18:17:39.834104+0000","last_undegraded":"2023-12-19T18:17:39.834104+0000","last_fullsized":"2023-12-19T18:17:39.834104+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:14:02.076472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.412219+0000","last_change":"2023-12-19T18:17:27.349954+0000","last_active":"2023-12-19T18:17:39.412219+0000","last_peered":"2023-12-19T18:17:39.412219+0000","last_clean":"2023-12-19T18:17:39.412219+0000","last_became_active":"2023-12-19T18:17:27.349694+0000","last_became_peered":"2023-12-19T18:17:27.349694+0000","last_unstale":"2023-12-19T18:17:39.412219+0000","last_undegraded":"2023-12-19T18:17:39.412219+0000","last_fullsized":"2023-12-19T18:17:39.412219+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:36:07.296035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"232'1","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.822729+0000","last_change":"2023-12-19T18:17:26.944980+0000","last_active":"2023-12-19T18:17:39.822729+0000","last_peered":"2023-12-19T18:17:39.822729+0000","last_clean":"2023-12-19T18:17:39.822729+0000","last_became_active":"2023-12-19T18:17:26.943863+0000","last_became_peered":"2023-12-19T18:17:26.943863+0000","last_unstale":"2023-12-19T18:17:39.822729+0000","last_undegraded":"2023-12-19T18:17:39.822729+0000","last_fullsized":"2023-12-19T18:17:39.822729+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:38:38.154851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.841408+0000","last_change":"2023-12-19T18:17:39.840268+0000","last_active":"2023-12-19T18:17:39.841408+0000","last_peered":"2023-12-19T18:17:39.841408+0000","last_clean":"2023-12-19T18:17:39.841408+0000","last_became_active":"2023-12-19T18:17:39.839905+0000","last_became_peered":"2023-12-19T18:17:39.839905+0000","last_unstale":"2023-12-19T18:17:39.841408+0000","last_undegraded":"2023-12-19T18:17:39.841408+0000","last_fullsized":"2023-12-19T18:17:39.841408+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:10:59.997438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809204+0000","last_change":"2023-12-19T18:17:39.420566+0000","last_active":"2023-12-19T18:17:40.809204+0000","last_peered":"2023-12-19T18:17:40.809204+0000","last_clean":"2023-12-19T18:17:40.809204+0000","last_became_active":"2023-12-19T18:17:39.420248+0000","last_became_peered":"2023-12-19T18:17:39.420248+0000","last_unstale":"2023-12-19T18:17:40.809204+0000","last_undegraded":"2023-12-19T18:17:40.809204+0000","last_fullsized":"2023-12-19T18:17:40.809204+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:11.094591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.417573+0000","last_change":"2023-12-19T18:17:27.364634+0000","last_active":"2023-12-19T18:17:39.417573+0000","last_peered":"2023-12-19T18:17:39.417573+0000","last_clean":"2023-12-19T18:17:39.417573+0000","last_became_active":"2023-12-19T18:17:27.364357+0000","last_became_peered":"2023-12-19T18:17:27.364357+0000","last_unstale":"2023-12-19T18:17:39.417573+0000","last_undegraded":"2023-12-19T18:17:39.417573+0000","last_fullsized":"2023-12-19T18:17:39.417573+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:05:35.230888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"232'2","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.831082+0000","last_change":"2023-12-19T18:17:26.938178+0000","last_active":"2023-12-19T18:17:39.831082+0000","last_peered":"2023-12-19T18:17:39.831082+0000","last_clean":"2023-12-19T18:17:39.831082+0000","last_became_active":"2023-12-19T18:17:26.937910+0000","last_became_peered":"2023-12-19T18:17:26.937910+0000","last_unstale":"2023-12-19T18:17:39.831082+0000","last_undegraded":"2023-12-19T18:17:39.831082+0000","last_fullsized":"2023-12-19T18:17:39.831082+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:19:13.215706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.839653+0000","last_change":"2023-12-19T18:17:39.833695+0000","last_active":"2023-12-19T18:17:39.839653+0000","last_peered":"2023-12-19T18:17:39.839653+0000","last_clean":"2023-12-19T18:17:39.839653+0000","last_became_active":"2023-12-19T18:17:39.833458+0000","last_became_peered":"2023-12-19T18:17:39.833458+0000","last_unstale":"2023-12-19T18:17:39.839653+0000","last_undegraded":"2023-12-19T18:17:39.839653+0000","last_fullsized":"2023-12-19T18:17:39.839653+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:27:16.215134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"236'1","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.813485+0000","last_change":"2023-12-19T18:17:39.420721+0000","last_active":"2023-12-19T18:17:40.813485+0000","last_peered":"2023-12-19T18:17:40.813485+0000","last_clean":"2023-12-19T18:17:40.813485+0000","last_became_active":"2023-12-19T18:17:39.420084+0000","last_became_peered":"2023-12-19T18:17:39.420084+0000","last_unstale":"2023-12-19T18:17:40.813485+0000","last_undegraded":"2023-12-19T18:17:40.813485+0000","last_fullsized":"2023-12-19T18:17:40.813485+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:35:27.292441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820686+0000","last_change":"2023-12-19T18:17:27.360665+0000","last_active":"2023-12-19T18:17:39.820686+0000","last_peered":"2023-12-19T18:17:39.820686+0000","last_clean":"2023-12-19T18:17:39.820686+0000","last_became_active":"2023-12-19T18:17:27.360397+0000","last_became_peered":"2023-12-19T18:17:27.360397+0000","last_unstale":"2023-12-19T18:17:39.820686+0000","last_undegraded":"2023-12-19T18:17:39.820686+0000","last_fullsized":"2023-12-19T18:17:39.820686+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:33.393492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"232'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.415927+0000","last_change":"2023-12-19T18:17:26.946790+0000","last_active":"2023-12-19T18:17:39.415927+0000","last_peered":"2023-12-19T18:17:39.415927+0000","last_clean":"2023-12-19T18:17:39.415927+0000","last_became_active":"2023-12-19T18:17:26.946487+0000","last_became_peered":"2023-12-19T18:17:26.946487+0000","last_unstale":"2023-12-19T18:17:39.415927+0000","last_undegraded":"2023-12-19T18:17:39.415927+0000","last_fullsized":"2023-12-19T18:17:39.415927+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:49.419114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.829656+0000","last_change":"2023-12-19T18:17:39.823685+0000","last_active":"2023-12-19T18:17:39.829656+0000","last_peered":"2023-12-19T18:17:39.829656+0000","last_clean":"2023-12-19T18:17:39.829656+0000","last_became_active":"2023-12-19T18:17:39.822618+0000","last_became_peered":"2023-12-19T18:17:39.822618+0000","last_unstale":"2023-12-19T18:17:39.829656+0000","last_undegraded":"2023-12-19T18:17:39.829656+0000","last_fullsized":"2023-12-19T18:17:39.829656+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:20:14.678399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"236'2","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834119+0000","last_change":"2023-12-19T18:17:39.422146+0000","last_active":"2023-12-19T18:17:39.834119+0000","last_peered":"2023-12-19T18:17:39.834119+0000","last_clean":"2023-12-19T18:17:39.834119+0000","last_became_active":"2023-12-19T18:17:39.421744+0000","last_became_peered":"2023-12-19T18:17:39.421744+0000","last_unstale":"2023-12-19T18:17:39.834119+0000","last_undegraded":"2023-12-19T18:17:39.834119+0000","last_fullsized":"2023-12-19T18:17:39.834119+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:41:19.519584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820550+0000","last_change":"2023-12-19T18:17:27.365652+0000","last_active":"2023-12-19T18:17:39.820550+0000","last_peered":"2023-12-19T18:17:39.820550+0000","last_clean":"2023-12-19T18:17:39.820550+0000","last_became_active":"2023-12-19T18:17:27.365336+0000","last_became_peered":"2023-12-19T18:17:27.365336+0000","last_unstale":"2023-12-19T18:17:39.820550+0000","last_undegraded":"2023-12-19T18:17:39.820550+0000","last_fullsized":"2023-12-19T18:17:39.820550+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:54.566933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416460+0000","last_change":"2023-12-19T18:17:26.942091+0000","last_active":"2023-12-19T18:17:39.416460+0000","last_peered":"2023-12-19T18:17:39.416460+0000","last_clean":"2023-12-19T18:17:39.416460+0000","last_became_active":"2023-12-19T18:17:26.941806+0000","last_became_peered":"2023-12-19T18:17:26.941806+0000","last_unstale":"2023-12-19T18:17:39.416460+0000","last_undegraded":"2023-12-19T18:17:39.416460+0000","last_fullsized":"2023-12-19T18:17:39.416460+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:27:28.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.808651+0000","last_change":"2023-12-19T18:17:39.839169+0000","last_active":"2023-12-19T18:17:40.808651+0000","last_peered":"2023-12-19T18:17:40.808651+0000","last_clean":"2023-12-19T18:17:40.808651+0000","last_became_active":"2023-12-19T18:17:39.838252+0000","last_became_peered":"2023-12-19T18:17:39.838252+0000","last_unstale":"2023-12-19T18:17:40.808651+0000","last_undegraded":"2023-12-19T18:17:40.808651+0000","last_fullsized":"2023-12-19T18:17:40.808651+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:46:35.106086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.821679+0000","last_change":"2023-12-19T18:17:39.426473+0000","last_active":"2023-12-19T18:17:39.821679+0000","last_peered":"2023-12-19T18:17:39.821679+0000","last_clean":"2023-12-19T18:17:39.821679+0000","last_became_active":"2023-12-19T18:17:39.426209+0000","last_became_peered":"2023-12-19T18:17:39.426209+0000","last_unstale":"2023-12-19T18:17:39.821679+0000","last_undegraded":"2023-12-19T18:17:39.821679+0000","last_fullsized":"2023-12-19T18:17:39.821679+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:40.104645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.412727+0000","last_change":"2023-12-19T18:17:27.367095+0000","last_active":"2023-12-19T18:17:39.412727+0000","last_peered":"2023-12-19T18:17:39.412727+0000","last_clean":"2023-12-19T18:17:39.412727+0000","last_became_active":"2023-12-19T18:17:27.366441+0000","last_became_peered":"2023-12-19T18:17:27.366441+0000","last_unstale":"2023-12-19T18:17:39.412727+0000","last_undegraded":"2023-12-19T18:17:39.412727+0000","last_fullsized":"2023-12-19T18:17:39.412727+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:10:47.532188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"232'2","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820898+0000","last_change":"2023-12-19T18:17:26.948576+0000","last_active":"2023-12-19T18:17:39.820898+0000","last_peered":"2023-12-19T18:17:39.820898+0000","last_clean":"2023-12-19T18:17:39.820898+0000","last_became_active":"2023-12-19T18:17:26.948310+0000","last_became_peered":"2023-12-19T18:17:26.948310+0000","last_unstale":"2023-12-19T18:17:39.820898+0000","last_undegraded":"2023-12-19T18:17:39.820898+0000","last_fullsized":"2023-12-19T18:17:39.820898+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:58.733231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830331+0000","last_change":"2023-12-19T18:17:39.826366+0000","last_active":"2023-12-19T18:17:39.830331+0000","last_peered":"2023-12-19T18:17:39.830331+0000","last_clean":"2023-12-19T18:17:39.830331+0000","last_became_active":"2023-12-19T18:17:39.826114+0000","last_became_peered":"2023-12-19T18:17:39.826114+0000","last_unstale":"2023-12-19T18:17:39.830331+0000","last_undegraded":"2023-12-19T18:17:39.830331+0000","last_fullsized":"2023-12-19T18:17:39.830331+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:44.071731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.811944+0000","last_change":"2023-12-19T18:17:39.424437+0000","last_active":"2023-12-19T18:17:40.811944+0000","last_peered":"2023-12-19T18:17:40.811944+0000","last_clean":"2023-12-19T18:17:40.811944+0000","last_became_active":"2023-12-19T18:17:39.422298+0000","last_became_peered":"2023-12-19T18:17:39.422298+0000","last_unstale":"2023-12-19T18:17:40.811944+0000","last_undegraded":"2023-12-19T18:17:40.811944+0000","last_fullsized":"2023-12-19T18:17:40.811944+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:20:55.327417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820972+0000","last_change":"2023-12-19T18:17:27.358552+0000","last_active":"2023-12-19T18:17:39.820972+0000","last_peered":"2023-12-19T18:17:39.820972+0000","last_clean":"2023-12-19T18:17:39.820972+0000","last_became_active":"2023-12-19T18:17:27.358174+0000","last_became_peered":"2023-12-19T18:17:27.358174+0000","last_unstale":"2023-12-19T18:17:39.820972+0000","last_undegraded":"2023-12-19T18:17:39.820972+0000","last_fullsized":"2023-12-19T18:17:39.820972+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:14.654686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810586+0000","last_change":"2023-12-19T18:17:26.947007+0000","last_active":"2023-12-19T18:17:39.810586+0000","last_peered":"2023-12-19T18:17:39.810586+0000","last_clean":"2023-12-19T18:17:39.810586+0000","last_became_active":"2023-12-19T18:17:26.946741+0000","last_became_peered":"2023-12-19T18:17:26.946741+0000","last_unstale":"2023-12-19T18:17:39.810586+0000","last_undegraded":"2023-12-19T18:17:39.810586+0000","last_fullsized":"2023-12-19T18:17:39.810586+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:20.105057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.826429+0000","last_change":"2023-12-19T18:17:39.822781+0000","last_active":"2023-12-19T18:17:39.826429+0000","last_peered":"2023-12-19T18:17:39.826429+0000","last_clean":"2023-12-19T18:17:39.826429+0000","last_became_active":"2023-12-19T18:17:39.822463+0000","last_became_peered":"2023-12-19T18:17:39.822463+0000","last_unstale":"2023-12-19T18:17:39.826429+0000","last_undegraded":"2023-12-19T18:17:39.826429+0000","last_fullsized":"2023-12-19T18:17:39.826429+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:41.114550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"236'2","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.831810+0000","last_change":"2023-12-19T18:17:39.425769+0000","last_active":"2023-12-19T18:17:39.831810+0000","last_peered":"2023-12-19T18:17:39.831810+0000","last_clean":"2023-12-19T18:17:39.831810+0000","last_became_active":"2023-12-19T18:17:39.425446+0000","last_became_peered":"2023-12-19T18:17:39.425446+0000","last_unstale":"2023-12-19T18:17:39.831810+0000","last_undegraded":"2023-12-19T18:17:39.831810+0000","last_fullsized":"2023-12-19T18:17:39.831810+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:26.262151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.410034+0000","last_change":"2023-12-19T18:17:27.348563+0000","last_active":"2023-12-19T18:17:39.410034+0000","last_peered":"2023-12-19T18:17:39.410034+0000","last_clean":"2023-12-19T18:17:39.410034+0000","last_became_active":"2023-12-19T18:17:27.348275+0000","last_became_peered":"2023-12-19T18:17:27.348275+0000","last_unstale":"2023-12-19T18:17:39.410034+0000","last_undegraded":"2023-12-19T18:17:39.410034+0000","last_fullsized":"2023-12-19T18:17:39.410034+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:45:15.369206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.811163+0000","last_change":"2023-12-19T18:17:26.944588+0000","last_active":"2023-12-19T18:17:39.811163+0000","last_peered":"2023-12-19T18:17:39.811163+0000","last_clean":"2023-12-19T18:17:39.811163+0000","last_became_active":"2023-12-19T18:17:26.944323+0000","last_became_peered":"2023-12-19T18:17:26.944323+0000","last_unstale":"2023-12-19T18:17:39.811163+0000","last_undegraded":"2023-12-19T18:17:39.811163+0000","last_fullsized":"2023-12-19T18:17:39.811163+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:25:45.208073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.833736+0000","last_change":"2023-12-19T18:17:39.829879+0000","last_active":"2023-12-19T18:17:39.833736+0000","last_peered":"2023-12-19T18:17:39.833736+0000","last_clean":"2023-12-19T18:17:39.833736+0000","last_became_active":"2023-12-19T18:17:39.829196+0000","last_became_peered":"2023-12-19T18:17:39.829196+0000","last_unstale":"2023-12-19T18:17:39.833736+0000","last_undegraded":"2023-12-19T18:17:39.833736+0000","last_fullsized":"2023-12-19T18:17:39.833736+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:57.527231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.808828+0000","last_change":"2023-12-19T18:17:39.420394+0000","last_active":"2023-12-19T18:17:40.808828+0000","last_peered":"2023-12-19T18:17:40.808828+0000","last_clean":"2023-12-19T18:17:40.808828+0000","last_became_active":"2023-12-19T18:17:39.420074+0000","last_became_peered":"2023-12-19T18:17:39.420074+0000","last_unstale":"2023-12-19T18:17:40.808828+0000","last_undegraded":"2023-12-19T18:17:40.808828+0000","last_fullsized":"2023-12-19T18:17:40.808828+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:40:28.357211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413682+0000","last_change":"2023-12-19T18:17:27.342936+0000","last_active":"2023-12-19T18:17:39.413682+0000","last_peered":"2023-12-19T18:17:39.413682+0000","last_clean":"2023-12-19T18:17:39.413682+0000","last_became_active":"2023-12-19T18:17:27.342636+0000","last_became_peered":"2023-12-19T18:17:27.342636+0000","last_unstale":"2023-12-19T18:17:39.413682+0000","last_undegraded":"2023-12-19T18:17:39.413682+0000","last_fullsized":"2023-12-19T18:17:39.413682+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:43.945499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.819918+0000","last_change":"2023-12-19T18:17:26.950102+0000","last_active":"2023-12-19T18:17:39.819918+0000","last_peered":"2023-12-19T18:17:39.819918+0000","last_clean":"2023-12-19T18:17:39.819918+0000","last_became_active":"2023-12-19T18:17:26.949810+0000","last_became_peered":"2023-12-19T18:17:26.949810+0000","last_unstale":"2023-12-19T18:17:39.819918+0000","last_undegraded":"2023-12-19T18:17:39.819918+0000","last_fullsized":"2023-12-19T18:17:39.819918+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:04:02.387088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809122+0000","last_change":"2023-12-19T18:17:39.838380+0000","last_active":"2023-12-19T18:17:40.809122+0000","last_peered":"2023-12-19T18:17:40.809122+0000","last_clean":"2023-12-19T18:17:40.809122+0000","last_became_active":"2023-12-19T18:17:39.837519+0000","last_became_peered":"2023-12-19T18:17:39.837519+0000","last_unstale":"2023-12-19T18:17:40.809122+0000","last_undegraded":"2023-12-19T18:17:40.809122+0000","last_fullsized":"2023-12-19T18:17:40.809122+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:53:27.350034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.825910+0000","last_change":"2023-12-19T18:17:39.423805+0000","last_active":"2023-12-19T18:17:39.825910+0000","last_peered":"2023-12-19T18:17:39.825910+0000","last_clean":"2023-12-19T18:17:39.825910+0000","last_became_active":"2023-12-19T18:17:39.423461+0000","last_became_peered":"2023-12-19T18:17:39.423461+0000","last_unstale":"2023-12-19T18:17:39.825910+0000","last_undegraded":"2023-12-19T18:17:39.825910+0000","last_fullsized":"2023-12-19T18:17:39.825910+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:31.423816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.412786+0000","last_change":"2023-12-19T18:17:27.350933+0000","last_active":"2023-12-19T18:17:39.412786+0000","last_peered":"2023-12-19T18:17:39.412786+0000","last_clean":"2023-12-19T18:17:39.412786+0000","last_became_active":"2023-12-19T18:17:27.350661+0000","last_became_peered":"2023-12-19T18:17:27.350661+0000","last_unstale":"2023-12-19T18:17:39.412786+0000","last_undegraded":"2023-12-19T18:17:39.412786+0000","last_fullsized":"2023-12-19T18:17:39.412786+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:25.159077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"232'2","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413198+0000","last_change":"2023-12-19T18:17:26.944158+0000","last_active":"2023-12-19T18:17:39.413198+0000","last_peered":"2023-12-19T18:17:39.413198+0000","last_clean":"2023-12-19T18:17:39.413198+0000","last_became_active":"2023-12-19T18:17:26.943845+0000","last_became_peered":"2023-12-19T18:17:26.943845+0000","last_unstale":"2023-12-19T18:17:39.413198+0000","last_undegraded":"2023-12-19T18:17:39.413198+0000","last_fullsized":"2023-12-19T18:17:39.413198+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:10.097246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809631+0000","last_change":"2023-12-19T18:17:39.844260+0000","last_active":"2023-12-19T18:17:40.809631+0000","last_peered":"2023-12-19T18:17:40.809631+0000","last_clean":"2023-12-19T18:17:40.809631+0000","last_became_active":"2023-12-19T18:17:39.843498+0000","last_became_peered":"2023-12-19T18:17:39.843498+0000","last_unstale":"2023-12-19T18:17:40.809631+0000","last_undegraded":"2023-12-19T18:17:40.809631+0000","last_fullsized":"2023-12-19T18:17:40.809631+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:45:48.663670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809286+0000","last_change":"2023-12-19T18:17:39.410366+0000","last_active":"2023-12-19T18:17:40.809286+0000","last_peered":"2023-12-19T18:17:40.809286+0000","last_clean":"2023-12-19T18:17:40.809286+0000","last_became_active":"2023-12-19T18:17:39.409876+0000","last_became_peered":"2023-12-19T18:17:39.409876+0000","last_unstale":"2023-12-19T18:17:40.809286+0000","last_undegraded":"2023-12-19T18:17:40.809286+0000","last_fullsized":"2023-12-19T18:17:40.809286+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:28.705197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.822646+0000","last_change":"2023-12-19T18:17:27.361893+0000","last_active":"2023-12-19T18:17:39.822646+0000","last_peered":"2023-12-19T18:17:39.822646+0000","last_clean":"2023-12-19T18:17:39.822646+0000","last_became_active":"2023-12-19T18:17:27.361618+0000","last_became_peered":"2023-12-19T18:17:27.361618+0000","last_unstale":"2023-12-19T18:17:39.822646+0000","last_undegraded":"2023-12-19T18:17:39.822646+0000","last_fullsized":"2023-12-19T18:17:39.822646+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:50.421353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"233'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.417658+0000","last_change":"2023-12-19T18:17:26.944398+0000","last_active":"2023-12-19T18:17:39.417658+0000","last_peered":"2023-12-19T18:17:39.417658+0000","last_clean":"2023-12-19T18:17:39.417658+0000","last_became_active":"2023-12-19T18:17:26.942848+0000","last_became_peered":"2023-12-19T18:17:26.942848+0000","last_unstale":"2023-12-19T18:17:39.417658+0000","last_undegraded":"2023-12-19T18:17:39.417658+0000","last_fullsized":"2023-12-19T18:17:39.417658+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:01.415653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.813706+0000","last_change":"2023-12-19T18:17:39.823115+0000","last_active":"2023-12-19T18:17:40.813706+0000","last_peered":"2023-12-19T18:17:40.813706+0000","last_clean":"2023-12-19T18:17:40.813706+0000","last_became_active":"2023-12-19T18:17:39.820305+0000","last_became_peered":"2023-12-19T18:17:39.820305+0000","last_unstale":"2023-12-19T18:17:40.813706+0000","last_undegraded":"2023-12-19T18:17:40.813706+0000","last_fullsized":"2023-12-19T18:17:40.813706+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:05:47.696138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"236'2","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.849209+0000","last_change":"2023-12-19T18:17:39.415322+0000","last_active":"2023-12-19T18:17:39.849209+0000","last_peered":"2023-12-19T18:17:39.849209+0000","last_clean":"2023-12-19T18:17:39.849209+0000","last_became_active":"2023-12-19T18:17:39.415021+0000","last_became_peered":"2023-12-19T18:17:39.415021+0000","last_unstale":"2023-12-19T18:17:39.849209+0000","last_undegraded":"2023-12-19T18:17:39.849209+0000","last_fullsized":"2023-12-19T18:17:39.849209+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:04:14.844672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820780+0000","last_change":"2023-12-19T18:17:27.359609+0000","last_active":"2023-12-19T18:17:39.820780+0000","last_peered":"2023-12-19T18:17:39.820780+0000","last_clean":"2023-12-19T18:17:39.820780+0000","last_became_active":"2023-12-19T18:17:27.358162+0000","last_became_peered":"2023-12-19T18:17:27.358162+0000","last_unstale":"2023-12-19T18:17:39.820780+0000","last_undegraded":"2023-12-19T18:17:39.820780+0000","last_fullsized":"2023-12-19T18:17:39.820780+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:31:06.407037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416004+0000","last_change":"2023-12-19T18:17:26.941195+0000","last_active":"2023-12-19T18:17:39.416004+0000","last_peered":"2023-12-19T18:17:39.416004+0000","last_clean":"2023-12-19T18:17:39.416004+0000","last_became_active":"2023-12-19T18:17:26.940845+0000","last_became_peered":"2023-12-19T18:17:26.940845+0000","last_unstale":"2023-12-19T18:17:39.416004+0000","last_undegraded":"2023-12-19T18:17:39.416004+0000","last_fullsized":"2023-12-19T18:17:39.416004+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:22.276149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.844143+0000","last_change":"2023-12-19T18:17:39.843161+0000","last_active":"2023-12-19T18:17:39.844143+0000","last_peered":"2023-12-19T18:17:39.844143+0000","last_clean":"2023-12-19T18:17:39.844143+0000","last_became_active":"2023-12-19T18:17:39.842884+0000","last_became_peered":"2023-12-19T18:17:39.842884+0000","last_unstale":"2023-12-19T18:17:39.844143+0000","last_undegraded":"2023-12-19T18:17:39.844143+0000","last_fullsized":"2023-12-19T18:17:39.844143+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:05:25.018602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812281+0000","last_change":"2023-12-19T18:17:39.419763+0000","last_active":"2023-12-19T18:17:40.812281+0000","last_peered":"2023-12-19T18:17:40.812281+0000","last_clean":"2023-12-19T18:17:40.812281+0000","last_became_active":"2023-12-19T18:17:39.419484+0000","last_became_peered":"2023-12-19T18:17:39.419484+0000","last_unstale":"2023-12-19T18:17:40.812281+0000","last_undegraded":"2023-12-19T18:17:40.812281+0000","last_fullsized":"2023-12-19T18:17:40.812281+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:33:38.145177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.414098+0000","last_change":"2023-12-19T18:17:27.351030+0000","last_active":"2023-12-19T18:17:39.414098+0000","last_peered":"2023-12-19T18:17:39.414098+0000","last_clean":"2023-12-19T18:17:39.414098+0000","last_became_active":"2023-12-19T18:17:27.350692+0000","last_became_peered":"2023-12-19T18:17:27.350692+0000","last_unstale":"2023-12-19T18:17:39.414098+0000","last_undegraded":"2023-12-19T18:17:39.414098+0000","last_fullsized":"2023-12-19T18:17:39.414098+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:15:54.445257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820875+0000","last_change":"2023-12-19T18:17:26.943936+0000","last_active":"2023-12-19T18:17:39.820875+0000","last_peered":"2023-12-19T18:17:39.820875+0000","last_clean":"2023-12-19T18:17:39.820875+0000","last_became_active":"2023-12-19T18:17:26.943609+0000","last_became_peered":"2023-12-19T18:17:26.943609+0000","last_unstale":"2023-12-19T18:17:39.820875+0000","last_undegraded":"2023-12-19T18:17:39.820875+0000","last_fullsized":"2023-12-19T18:17:39.820875+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:12.547671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.831297+0000","last_change":"2023-12-19T18:17:39.828877+0000","last_active":"2023-12-19T18:17:39.831297+0000","last_peered":"2023-12-19T18:17:39.831297+0000","last_clean":"2023-12-19T18:17:39.831297+0000","last_became_active":"2023-12-19T18:17:39.825032+0000","last_became_peered":"2023-12-19T18:17:39.825032+0000","last_unstale":"2023-12-19T18:17:39.831297+0000","last_undegraded":"2023-12-19T18:17:39.831297+0000","last_fullsized":"2023-12-19T18:17:39.831297+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:16:06.910507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.813466+0000","last_change":"2023-12-19T18:17:39.423064+0000","last_active":"2023-12-19T18:17:39.813466+0000","last_peered":"2023-12-19T18:17:39.813466+0000","last_clean":"2023-12-19T18:17:39.813466+0000","last_became_active":"2023-12-19T18:17:39.422717+0000","last_became_peered":"2023-12-19T18:17:39.422717+0000","last_unstale":"2023-12-19T18:17:39.813466+0000","last_undegraded":"2023-12-19T18:17:39.813466+0000","last_fullsized":"2023-12-19T18:17:39.813466+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:38:22.115594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408249+0000","last_change":"2023-12-19T18:17:27.342408+0000","last_active":"2023-12-19T18:17:39.408249+0000","last_peered":"2023-12-19T18:17:39.408249+0000","last_clean":"2023-12-19T18:17:39.408249+0000","last_became_active":"2023-12-19T18:17:27.342114+0000","last_became_peered":"2023-12-19T18:17:27.342114+0000","last_unstale":"2023-12-19T18:17:39.408249+0000","last_undegraded":"2023-12-19T18:17:39.408249+0000","last_fullsized":"2023-12-19T18:17:39.408249+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:02.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810372+0000","last_change":"2023-12-19T18:17:26.945192+0000","last_active":"2023-12-19T18:17:39.810372+0000","last_peered":"2023-12-19T18:17:39.810372+0000","last_clean":"2023-12-19T18:17:39.810372+0000","last_became_active":"2023-12-19T18:17:26.943303+0000","last_became_peered":"2023-12-19T18:17:26.943303+0000","last_unstale":"2023-12-19T18:17:39.810372+0000","last_undegraded":"2023-12-19T18:17:39.810372+0000","last_fullsized":"2023-12-19T18:17:39.810372+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.821104+0000","last_change":"2023-12-19T18:17:39.816900+0000","last_active":"2023-12-19T18:17:39.821104+0000","last_peered":"2023-12-19T18:17:39.821104+0000","last_clean":"2023-12-19T18:17:39.821104+0000","last_became_active":"2023-12-19T18:17:39.816650+0000","last_became_peered":"2023-12-19T18:17:39.816650+0000","last_unstale":"2023-12-19T18:17:39.821104+0000","last_undegraded":"2023-12-19T18:17:39.821104+0000","last_fullsized":"2023-12-19T18:17:39.821104+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:03.678500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.816630+0000","last_change":"2023-12-19T18:17:39.426984+0000","last_active":"2023-12-19T18:17:39.816630+0000","last_peered":"2023-12-19T18:17:39.816630+0000","last_clean":"2023-12-19T18:17:39.816630+0000","last_became_active":"2023-12-19T18:17:39.426585+0000","last_became_peered":"2023-12-19T18:17:39.426585+0000","last_unstale":"2023-12-19T18:17:39.816630+0000","last_undegraded":"2023-12-19T18:17:39.816630+0000","last_fullsized":"2023-12-19T18:17:39.816630+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:38.862837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820660+0000","last_change":"2023-12-19T18:17:27.362825+0000","last_active":"2023-12-19T18:17:39.820660+0000","last_peered":"2023-12-19T18:17:39.820660+0000","last_clean":"2023-12-19T18:17:39.820660+0000","last_became_active":"2023-12-19T18:17:27.362414+0000","last_became_peered":"2023-12-19T18:17:27.362414+0000","last_unstale":"2023-12-19T18:17:39.820660+0000","last_undegraded":"2023-12-19T18:17:39.820660+0000","last_fullsized":"2023-12-19T18:17:39.820660+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:21:44.939055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834359+0000","last_change":"2023-12-19T18:17:26.940233+0000","last_active":"2023-12-19T18:17:39.834359+0000","last_peered":"2023-12-19T18:17:39.834359+0000","last_clean":"2023-12-19T18:17:39.834359+0000","last_became_active":"2023-12-19T18:17:26.939941+0000","last_became_peered":"2023-12-19T18:17:26.939941+0000","last_unstale":"2023-12-19T18:17:39.834359+0000","last_undegraded":"2023-12-19T18:17:39.834359+0000","last_fullsized":"2023-12-19T18:17:39.834359+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:15:20.805126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812840+0000","last_change":"2023-12-19T18:17:39.820665+0000","last_active":"2023-12-19T18:17:40.812840+0000","last_peered":"2023-12-19T18:17:40.812840+0000","last_clean":"2023-12-19T18:17:40.812840+0000","last_became_active":"2023-12-19T18:17:39.820253+0000","last_became_peered":"2023-12-19T18:17:39.820253+0000","last_unstale":"2023-12-19T18:17:40.812840+0000","last_undegraded":"2023-12-19T18:17:40.812840+0000","last_fullsized":"2023-12-19T18:17:40.812840+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:35.239010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812940+0000","last_change":"2023-12-19T18:17:39.417596+0000","last_active":"2023-12-19T18:17:40.812940+0000","last_peered":"2023-12-19T18:17:40.812940+0000","last_clean":"2023-12-19T18:17:40.812940+0000","last_became_active":"2023-12-19T18:17:39.417251+0000","last_became_peered":"2023-12-19T18:17:39.417251+0000","last_unstale":"2023-12-19T18:17:40.812940+0000","last_undegraded":"2023-12-19T18:17:40.812940+0000","last_fullsized":"2023-12-19T18:17:40.812940+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:27:40.705352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.819831+0000","last_change":"2023-12-19T18:17:27.356126+0000","last_active":"2023-12-19T18:17:39.819831+0000","last_peered":"2023-12-19T18:17:39.819831+0000","last_clean":"2023-12-19T18:17:39.819831+0000","last_became_active":"2023-12-19T18:17:27.355794+0000","last_became_peered":"2023-12-19T18:17:27.355794+0000","last_unstale":"2023-12-19T18:17:39.819831+0000","last_undegraded":"2023-12-19T18:17:39.819831+0000","last_fullsized":"2023-12-19T18:17:39.819831+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:23.278389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413871+0000","last_change":"2023-12-19T18:17:26.944096+0000","last_active":"2023-12-19T18:17:39.413871+0000","last_peered":"2023-12-19T18:17:39.413871+0000","last_clean":"2023-12-19T18:17:39.413871+0000","last_became_active":"2023-12-19T18:17:26.941219+0000","last_became_peered":"2023-12-19T18:17:26.941219+0000","last_unstale":"2023-12-19T18:17:39.413871+0000","last_undegraded":"2023-12-19T18:17:39.413871+0000","last_fullsized":"2023-12-19T18:17:39.413871+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:22:09.866781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830456+0000","last_change":"2023-12-19T18:17:39.825795+0000","last_active":"2023-12-19T18:17:39.830456+0000","last_peered":"2023-12-19T18:17:39.830456+0000","last_clean":"2023-12-19T18:17:39.830456+0000","last_became_active":"2023-12-19T18:17:39.825535+0000","last_became_peered":"2023-12-19T18:17:39.825535+0000","last_unstale":"2023-12-19T18:17:39.830456+0000","last_undegraded":"2023-12-19T18:17:39.830456+0000","last_fullsized":"2023-12-19T18:17:39.830456+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:52.954654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.812019+0000","last_change":"2023-12-19T18:17:39.415538+0000","last_active":"2023-12-19T18:17:39.812019+0000","last_peered":"2023-12-19T18:17:39.812019+0000","last_clean":"2023-12-19T18:17:39.812019+0000","last_became_active":"2023-12-19T18:17:39.415278+0000","last_became_peered":"2023-12-19T18:17:39.415278+0000","last_unstale":"2023-12-19T18:17:39.812019+0000","last_undegraded":"2023-12-19T18:17:39.812019+0000","last_fullsized":"2023-12-19T18:17:39.812019+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:46:16.298136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416970+0000","last_change":"2023-12-19T18:17:27.343300+0000","last_active":"2023-12-19T18:17:39.416970+0000","last_peered":"2023-12-19T18:17:39.416970+0000","last_clean":"2023-12-19T18:17:39.416970+0000","last_became_active":"2023-12-19T18:17:27.343020+0000","last_became_peered":"2023-12-19T18:17:27.343020+0000","last_unstale":"2023-12-19T18:17:39.416970+0000","last_undegraded":"2023-12-19T18:17:39.416970+0000","last_fullsized":"2023-12-19T18:17:39.416970+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:11.099486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.819996+0000","last_change":"2023-12-19T18:17:26.950561+0000","last_active":"2023-12-19T18:17:39.819996+0000","last_peered":"2023-12-19T18:17:39.819996+0000","last_clean":"2023-12-19T18:17:39.819996+0000","last_became_active":"2023-12-19T18:17:26.949052+0000","last_became_peered":"2023-12-19T18:17:26.949052+0000","last_unstale":"2023-12-19T18:17:39.819996+0000","last_undegraded":"2023-12-19T18:17:39.819996+0000","last_fullsized":"2023-12-19T18:17:39.819996+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:09:01.022466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812203+0000","last_change":"2023-12-19T18:17:39.846801+0000","last_active":"2023-12-19T18:17:40.812203+0000","last_peered":"2023-12-19T18:17:40.812203+0000","last_clean":"2023-12-19T18:17:40.812203+0000","last_became_active":"2023-12-19T18:17:39.846413+0000","last_became_peered":"2023-12-19T18:17:39.846413+0000","last_unstale":"2023-12-19T18:17:40.812203+0000","last_undegraded":"2023-12-19T18:17:40.812203+0000","last_fullsized":"2023-12-19T18:17:40.812203+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:12.643729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"236'4","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.836841+0000","last_change":"2023-12-19T18:17:39.423184+0000","last_active":"2023-12-19T18:17:39.836841+0000","last_peered":"2023-12-19T18:17:39.836841+0000","last_clean":"2023-12-19T18:17:39.836841+0000","last_became_active":"2023-12-19T18:17:39.422852+0000","last_became_peered":"2023-12-19T18:17:39.422852+0000","last_unstale":"2023-12-19T18:17:39.836841+0000","last_undegraded":"2023-12-19T18:17:39.836841+0000","last_fullsized":"2023-12-19T18:17:39.836841+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:26:15.844246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.414351+0000","last_change":"2023-12-19T18:17:27.349332+0000","last_active":"2023-12-19T18:17:39.414351+0000","last_peered":"2023-12-19T18:17:39.414351+0000","last_clean":"2023-12-19T18:17:39.414351+0000","last_became_active":"2023-12-19T18:17:27.349050+0000","last_became_peered":"2023-12-19T18:17:27.349050+0000","last_unstale":"2023-12-19T18:17:39.414351+0000","last_undegraded":"2023-12-19T18:17:39.414351+0000","last_fullsized":"2023-12-19T18:17:39.414351+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:33.501865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"232'2","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810749+0000","last_change":"2023-12-19T18:17:26.947941+0000","last_active":"2023-12-19T18:17:39.810749+0000","last_peered":"2023-12-19T18:17:39.810749+0000","last_clean":"2023-12-19T18:17:39.810749+0000","last_became_active":"2023-12-19T18:17:26.945013+0000","last_became_peered":"2023-12-19T18:17:26.945013+0000","last_unstale":"2023-12-19T18:17:39.810749+0000","last_undegraded":"2023-12-19T18:17:39.810749+0000","last_fullsized":"2023-12-19T18:17:39.810749+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:47:39.421442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"218'368","reported_seq":1283,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.414380+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:17:39.414380+0000","last_peered":"2023-12-19T18:17:39.414380+0000","last_clean":"2023-12-19T18:17:39.414380+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:17:39.414380+0000","last_undegraded":"2023-12-19T18:17:39.414380+0000","last_fullsized":"2023-12-19T18:17:39.414380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2306,"num_read_kb":4823,"num_write":615,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2306,"num_read_kb":4823,"num_write":615,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":368,"ondisk_log_size":368,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510579,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48476,"kb_used_data":3476,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323364,"statfs":{"total":96636764160,"available":96587124736,"internally_reserved":0,"allocated":3559424,"data_stored":2170833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68500000000000005,"15min":0.71499999999999997},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79000000000000004,"5min":1.048,"15min":1.1180000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71299999999999997,"15min":0.73199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90900000000000003,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63200000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":1.0389999999999999,"15min":1.0580000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.74199999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.73099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.0669999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.996,"5min":1.0820000000000001,"15min":1.236},"last":0.91300000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88100000000000001,"5min":1.002,"15min":1.079},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.49399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85899999999999999,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.78800000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.441,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.302},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.622,"15min":0.629},"min":{"1min":0.50800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.0209999999999999},"last":0.76100000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.90600000000000003,"5min":1.097,"15min":1.1299999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.55800000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95699999999999996,"5min":1.026,"15min":1.0800000000000001},"last":0.746}]},{"osd":6,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.61699999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.0469999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.093},"last":0.876}]}]},{"osd":6,"up_from":15,"seq":64424510579,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47904,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323936,"statfs":{"total":96636764160,"available":96587710464,"internally_reserved":0,"allocated":2973696,"data_stored":1580985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72299999999999998,"15min":0.73799999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.042},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.42799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97599999999999998,"5min":1.224,"15min":1.224},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68899999999999995,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86699999999999999,"5min":1.103,"15min":1.103},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.745,"15min":0.752},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":0.93600000000000005,"5min":0.97999999999999998,"15min":1.1259999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.752,"15min":0.76000000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98399999999999999,"5min":1.0549999999999999,"15min":1.0609999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70899999999999996,"15min":0.745},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98999999999999999,"5min":1.0089999999999999,"15min":1.111},"last":0.53700000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.754,"15min":0.77800000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.1519999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71599999999999997,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.036,"5min":1.268,"15min":1.268},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.373,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0469999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63200000000000001,"15min":0.61599999999999999},"min":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.64700000000000002}]},{"osd":5,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.47599999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.997,"5min":1.194,"15min":1.194},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68999999999999995,"15min":0.67100000000000004},"min":{"1min":0.46800000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1339999999999999,"5min":1.141,"15min":1.141},"last":0.62}]},{"osd":7,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56599999999999995,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.54600000000000004,"5min":0.377,"15min":0.377},"max":{"1min":1.1140000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.63400000000000001}]}]},{"osd":5,"up_from":15,"seq":64424510579,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47888,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323952,"statfs":{"total":96636764160,"available":96587726848,"internally_reserved":0,"allocated":2957312,"data_stored":1580325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.69599999999999995,"15min":0.72499999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.94999999999999996,"5min":1.002,"15min":1.0149999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.93100000000000005,"5min":0.96099999999999997,"15min":1.0229999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.497},"max":{"1min":0.84599999999999997,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.498,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97599999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.56599999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.58599999999999997,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":1.0069999999999999,"5min":1.0509999999999999,"15min":1.2250000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.73199999999999998},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88600000000000001,"5min":1.004,"15min":1.4279999999999999},"last":0.57299999999999995}]},{"osd":3,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.748,"15min":0.76200000000000001},"min":{"1min":0.56799999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.90400000000000003,"5min":1.0129999999999999,"15min":1.05},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93400000000000005,"5min":1.022,"15min":1.0329999999999999},"last":0.68200000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.81699999999999995,"5min":0.94499999999999995,"15min":1.0800000000000001},"last":0.442},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94299999999999995,"5min":1.208,"15min":1.208},"last":0.55100000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65800000000000003,"15min":0.65700000000000003},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.85799999999999998,"5min":1.008,"15min":1.0449999999999999},"last":0.53700000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.501,"5min":0.439,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":1.169,"15min":1.169},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51100000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.079,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.627}]}]},{"osd":4,"up_from":15,"seq":64424510579,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47912,"kb_used_data":2912,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323928,"statfs":{"total":96636764160,"available":96587702272,"internally_reserved":0,"allocated":2981888,"data_stored":1582563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.72599999999999998,"15min":0.73899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86699999999999999,"5min":1.036,"15min":1.109},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.88100000000000001,"5min":0.997,"15min":1.01},"last":0.67400000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.752,"15min":0.753},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90500000000000003,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.753,"15min":0.76200000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86699999999999999,"5min":0.997,"15min":1.1020000000000001},"last":0.77100000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0289999999999999,"15min":1.071},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.77400000000000002,"15min":0.77300000000000002},"min":{"1min":0.51800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91800000000000004,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.751,"15min":0.751},"min":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84699999999999998,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.77000000000000002,"15min":0.78600000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.85999999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.56799999999999995}]},{"osd":5,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65400000000000003,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"max":{"1min":0.91000000000000003,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":0.89800000000000002,"5min":1.042,"15min":1.042},"last":0.71199999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":1.024,"15min":1.024},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73699999999999999,"15min":0.71499999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90900000000000003,"5min":1.0680000000000001,"15min":1.0760000000000001},"last":0.85199999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.41899999999999998},"max":{"1min":0.93500000000000005,"5min":1.087,"15min":1.087},"last":0.753},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77500000000000002,"15min":0.76000000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.46999999999999997},"max":{"1min":0.98499999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.88500000000000001}]}]},{"osd":3,"up_from":15,"seq":64424510579,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47908,"kb_used_data":2908,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323932,"statfs":{"total":96636764160,"available":96587706368,"internally_reserved":0,"allocated":2977792,"data_stored":1583782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.498}]},{"osd":1,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48999999999999999,"15min":0.495},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72599999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56100000000000005,"15min":0.57299999999999995},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67700000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.36199999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73099999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.58199999999999996,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80700000000000005,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.55900000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.54900000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0109999999999999,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80500000000000005,"15min":0.81699999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0329999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.67600000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.80600000000000005,"15min":0.83399999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.99199999999999999,"5min":1.1739999999999999,"15min":1.397},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.80100000000000005,"15min":0.82599999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.042,"5min":1.2230000000000001,"15min":1.262},"last":0.40999999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82799999999999996,"15min":0.84499999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0329999999999999,"5min":1.234,"15min":1.274},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80700000000000005,"15min":0.83899999999999997},"min":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.93799999999999994,"5min":1.165,"15min":1.302},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.872,"15min":0.89800000000000002},"min":{"1min":0.65100000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.153,"5min":1.3540000000000001,"15min":1.486},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.86599999999999999,"15min":0.88600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1279999999999999,"5min":1.421,"15min":1.421},"last":0.76000000000000001}]}]},{"osd":2,"up_from":15,"seq":64424510578,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47912,"kb_used_data":2912,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323928,"statfs":{"total":96636764160,"available":96587702272,"internally_reserved":0,"allocated":2981888,"data_stored":1583276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.441,"15min":0.45000000000000001},"min":{"1min":0.26300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.70499999999999996,"5min":0.754,"15min":0.754},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48899999999999999,"15min":0.504},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68000000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44700000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59899999999999998,"15min":0.61599999999999999},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"max":{"1min":0.77800000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.73899999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.40000000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58599999999999997,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76800000000000002,"5min":0.80800000000000005,"15min":0.95999999999999996},"last":0.499},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.36899999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.875},"last":0.56399999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0580000000000001,"5min":1.0880000000000001,"15min":1.306},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77500000000000002,"15min":0.79200000000000004},"min":{"1min":0.48299999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0609999999999999,"5min":1.2070000000000001,"15min":1.466},"last":0.71299999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.77200000000000002,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.3049999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.80000000000000004,"15min":0.81899999999999995},"min":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.04,"5min":1.1519999999999999,"15min":1.429},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.81299999999999994,"15min":0.83799999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0329999999999999,"5min":1.216,"15min":1.4119999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77600000000000002,"15min":0.80000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.008,"5min":1.117,"15min":1.2470000000000001},"last":0.68600000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.82199999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.56000000000000005},"max":{"1min":1.0269999999999999,"5min":1.159,"15min":1.45},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.83899999999999997,"15min":0.85599999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.093,"5min":1.276,"15min":1.276},"last":0.77700000000000002}]}]},{"osd":1,"up_from":15,"seq":64424510579,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60200,"kb_used_data":2912,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311640,"statfs":{"total":96636764160,"available":96575119360,"internally_reserved":0,"allocated":2981888,"data_stored":1582103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44800000000000001,"15min":0.45800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56399999999999995,"5min":0.71999999999999997,"15min":0.89400000000000002},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79600000000000004,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.40200000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66000000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.66600000000000004,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.46800000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.622,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.747,"5min":0.94199999999999995,"15min":0.94499999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.628,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.78100000000000003,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92700000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77700000000000002,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.081,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76900000000000002,"15min":0.80000000000000004},"min":{"1min":0.58699999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0189999999999999,"5min":1.236,"15min":1.2609999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81799999999999995},"min":{"1min":0.58199999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0669999999999999,"5min":1.353,"15min":1.353},"last":0.75}]},{"osd":6,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.505},"max":{"1min":1.0009999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.82399999999999995,"15min":0.84099999999999997},"min":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.034,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.77400000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84599999999999997,"15min":0.85399999999999998},"min":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.56499999999999995},"max":{"1min":1.147,"5min":1.399,"15min":1.399},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.82799999999999996,"15min":0.85199999999999998},"min":{"1min":0.53000000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.181,"5min":1.365,"15min":1.365},"last":0.81499999999999995}]}]},{"osd":0,"up_from":14,"seq":60129543284,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60768,"kb_used_data":3480,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311072,"statfs":{"total":96636764160,"available":96574537728,"internally_reserved":0,"allocated":3563520,"data_stored":2171399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41899999999999998,"15min":0.45300000000000001},"min":{"1min":0.29699999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55300000000000005,"5min":0.69299999999999995,"15min":1.216},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.49399999999999999,"15min":0.51900000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67600000000000005,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.51600000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.48799999999999999,"15min":0.52600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.77000000000000002,"15min":1.1699999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52300000000000002,"15min":0.54700000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.96799999999999997},"last":0.47299999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.51400000000000001,"15min":0.55000000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.59999999999999998,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59899999999999998,"15min":0.64400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.032},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.80600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.496},"max":{"1min":0.93899999999999995,"5min":1.1200000000000001,"15min":1.381},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.78600000000000003,"15min":0.81100000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.47799999999999998},"max":{"1min":0.84799999999999998,"5min":1.137,"15min":1.2849999999999999},"last":0.78400000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.93799999999999994,"5min":1.1180000000000001,"15min":1.3420000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.79100000000000004,"15min":0.81299999999999994},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.052,"5min":1.1259999999999999,"15min":1.288},"last":0.76800000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81200000000000006,"15min":0.83899999999999997},"min":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.51300000000000001},"max":{"1min":0.97899999999999998,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.80000000000000004,"15min":0.82699999999999996},"min":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97999999999999998,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.69499999999999995}]},{"osd":7,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0149999999999999,"5min":1.1899999999999999,"15min":1.415},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.82599999999999996,"15min":0.85099999999999998},"min":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96899999999999997,"5min":1.0980000000000001,"15min":1.395},"last":0.81200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:17:43.953 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:17:44.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.125+0000 7fe57db6d640 1 Processor -- start 2023-12-19T18:17:44.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.125+0000 7fe57db6d640 1 -- start start 2023-12-19T18:17:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe57db6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe578154220 0x7fe57815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe57db6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe57806b3d0 0x7fe5780ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe57db6d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe5780cec40 0x7fe57815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe57db6d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5780cf0d0 con 0x7fe578154220 2023-12-19T18:17:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe57db6d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57806bc30 con 0x7fe5780cec40 2023-12-19T18:17:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe57db6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5780cc6e0 con 0x7fe57806b3d0 2023-12-19T18:17:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe5777fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe578154220 0x7fe57815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe5777fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe578154220 0x7fe57815aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36728/0 (socket says 172.21.15.26:36728) 2023-12-19T18:17:44.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe5777fe640 1 -- 172.21.15.26:0/1712142512 learned_addr learned my addr 172.21.15.26:0/1712142512 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe5777fe640 1 -- 172.21.15.26:0/1712142512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe57806b3d0 msgr2=0x7fe5780ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe5777fe640 1 --2- 172.21.15.26:0/1712142512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe57806b3d0 0x7fe5780ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe5777fe640 1 -- 172.21.15.26:0/1712142512 >> v2:172.21.15.160:3300/0 conn(0x7fe5780cec40 msgr2=0x7fe57815d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe5777fe640 1 --2- 172.21.15.26:0/1712142512 >> v2:172.21.15.160:3300/0 conn(0x7fe5780cec40 0x7fe57815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.129+0000 7fe5777fe640 1 -- 172.21.15.26:0/1712142512 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57815afc0 con 0x7fe578154220 2023-12-19T18:17:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe5777fe640 1 --2- 172.21.15.26:0/1712142512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe578154220 0x7fe57815aa80 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fe5780ae310 tx=0x7fe56801e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f65c6ad481e18c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe5767fc640 1 -- 172.21.15.26:0/1712142512 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5680a7970 con 0x7fe578154220 2023-12-19T18:17:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe5767fc640 1 -- 172.21.15.26:0/1712142512 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5680a7ad0 con 0x7fe578154220 2023-12-19T18:17:44.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe5767fc640 1 -- 172.21.15.26:0/1712142512 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5680a7e60 con 0x7fe578154220 2023-12-19T18:17:44.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe57db6d640 1 -- 172.21.15.26:0/1712142512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe578154220 msgr2=0x7fe57815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:44.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe57db6d640 1 --2- 172.21.15.26:0/1712142512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe578154220 0x7fe57815aa80 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fe5780ae310 tx=0x7fe56801e8e0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe57db6d640 1 -- 172.21.15.26:0/1712142512 shutdown_connections 2023-12-19T18:17:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe57db6d640 1 --2- 172.21.15.26:0/1712142512 >> v2:172.21.15.160:3300/0 conn(0x7fe5780cec40 0x7fe57815d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe57db6d640 1 --2- 172.21.15.26:0/1712142512 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe57806b3d0 0x7fe5780ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe57db6d640 1 --2- 172.21.15.26:0/1712142512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe578154220 0x7fe57815aa80 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe57db6d640 1 -- 172.21.15.26:0/1712142512 >> 172.21.15.26:0/1712142512 conn(0x7fe5780658e0 msgr2=0x7fe578065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe57db6d640 1 -- 172.21.15.26:0/1712142512 shutdown_connections 2023-12-19T18:17:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe57db6d640 1 -- 172.21.15.26:0/1712142512 wait complete. 2023-12-19T18:17:44.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe57db6d640 1 Processor -- start 2023-12-19T18:17:44.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.133+0000 7fe57db6d640 1 -- start start 2023-12-19T18:17:44.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe57db6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe57806b3d0 0x7fe5780aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe57db6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5780cec40 0x7fe5780cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe57db6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe578154220 0x7fe57814b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe57db6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe578154ea0 con 0x7fe578154220 2023-12-19T18:17:44.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe57db6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe578155a90 con 0x7fe5780cec40 2023-12-19T18:17:44.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe57db6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5781550f0 con 0x7fe57806b3d0 2023-12-19T18:17:44.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe577fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe578154220 0x7fe57814b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:44.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe577fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe578154220 0x7fe57814b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36730/0 (socket says 172.21.15.26:36730) 2023-12-19T18:17:44.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe5777fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe57806b3d0 0x7fe5780aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:44.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe577fff640 1 -- 172.21.15.26:0/2339669929 learned_addr learned my addr 172.21.15.26:0/2339669929 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:44.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe576ffd640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5780cec40 0x7fe5780cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:44.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe577fff640 1 -- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe57806b3d0 msgr2=0x7fe5780aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:44.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe577fff640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe57806b3d0 0x7fe5780aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe577fff640 1 -- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5780cec40 msgr2=0x7fe5780cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:44.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe577fff640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5780cec40 0x7fe5780cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe577fff640 1 -- 172.21.15.26:0/2339669929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56800b770 con 0x7fe578154220 2023-12-19T18:17:44.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe576ffd640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5780cec40 0x7fe5780cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:44.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe577fff640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe578154220 0x7fe57814b9f0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fe57000b970 tx=0x7fe57000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:44.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe55ffff640 1 -- 172.21.15.26:0/2339669929 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57000cb40 con 0x7fe578154220 2023-12-19T18:17:44.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe55ffff640 1 -- 172.21.15.26:0/2339669929 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57000ccc0 con 0x7fe578154220 2023-12-19T18:17:44.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe55ffff640 1 -- 172.21.15.26:0/2339669929 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570014680 con 0x7fe578154220 2023-12-19T18:17:44.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe57db6d640 1 -- 172.21.15.26:0/2339669929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5781553d0 con 0x7fe578154220 2023-12-19T18:17:44.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe57db6d640 1 -- 172.21.15.26:0/2339669929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57814e930 con 0x7fe578154220 2023-12-19T18:17:44.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe57db6d640 1 -- 172.21.15.26:0/2339669929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57814c640 con 0x7fe578154220 2023-12-19T18:17:44.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.137+0000 7fe55ffff640 1 -- 172.21.15.26:0/2339669929 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fe57001d3b0 con 0x7fe578154220 2023-12-19T18:17:44.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.141+0000 7fe55ffff640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe564065ca0 0x7fe564068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.141+0000 7fe55ffff640 1 -- 172.21.15.26:0/2339669929 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fe570083af0 con 0x7fe578154220 2023-12-19T18:17:44.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.141+0000 7fe5777fe640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe564065ca0 0x7fe564068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:44.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.141+0000 7fe5777fe640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe564065ca0 0x7fe564068160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe5680b1000 tx=0x7fe5680a9070 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:44.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.145+0000 7fe55ffff640 1 -- 172.21.15.26:0/2339669929 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe57814c640 con 0x7fe578154220 2023-12-19T18:17:44.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.297+0000 7fe57db6d640 1 -- 172.21.15.26:0/2339669929 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5780ccef0 con 0x7fe564065ca0 2023-12-19T18:17:44.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.313+0000 7fe55ffff640 1 -- 172.21.15.26:0/2339669929 <== mgr.10429 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528453 (secure 0 0 0) 0x7fe5680be090 con 0x7fe564065ca0 2023-12-19T18:17:44.320 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:44.324 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:17:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.317+0000 7fe57db6d640 1 -- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe564065ca0 msgr2=0x7fe564068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.317+0000 7fe57db6d640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe564065ca0 0x7fe564068160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe5680b1000 tx=0x7fe5680a9070 comp rx=0 tx=0).stop 2023-12-19T18:17:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.317+0000 7fe57db6d640 1 -- 172.21.15.26:0/2339669929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe578154220 msgr2=0x7fe57814b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.317+0000 7fe57db6d640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe578154220 0x7fe57814b9f0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fe57000b970 tx=0x7fe57000be30 comp rx=0 tx=0).stop 2023-12-19T18:17:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.317+0000 7fe57db6d640 1 -- 172.21.15.26:0/2339669929 shutdown_connections 2023-12-19T18:17:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.317+0000 7fe57db6d640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe564065ca0 0x7fe564068160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.317+0000 7fe57db6d640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe578154220 0x7fe57814b9f0 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.317+0000 7fe57db6d640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5780cec40 0x7fe5780cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.317+0000 7fe57db6d640 1 --2- 172.21.15.26:0/2339669929 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe57806b3d0 0x7fe5780aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.317+0000 7fe57db6d640 1 -- 172.21.15.26:0/2339669929 >> 172.21.15.26:0/2339669929 conn(0x7fe5780658e0 msgr2=0x7fe578065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:44.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.317+0000 7fe57db6d640 1 -- 172.21.15.26:0/2339669929 shutdown_connections 2023-12-19T18:17:44.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.317+0000 7fe57db6d640 1 -- 172.21.15.26:0/2339669929 wait complete. 2023-12-19T18:17:44.340 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-12-19T18:17:42.938817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596333,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2306,"num_read_kb":4823,"num_write":706,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":447,"ondisk_log_size":447,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":280,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":408968,"kb_used_data":24392,"kb_used_omap":0,"kb_used_meta":384512,"kb_avail":754565752,"statfs":{"total":773094113280,"available":772675330048,"internally_reserved":0,"allocated":24977408,"data_stored":13835266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":393740288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3349,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.552648"},"pg_stats":[{"pgid":"31.3d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.837659+0000","last_change":"2023-12-19T18:17:39.833552+0000","last_active":"2023-12-19T18:17:39.837659+0000","last_peered":"2023-12-19T18:17:39.837659+0000","last_clean":"2023-12-19T18:17:39.837659+0000","last_became_active":"2023-12-19T18:17:39.832016+0000","last_became_peered":"2023-12-19T18:17:39.832016+0000","last_unstale":"2023-12-19T18:17:39.837659+0000","last_undegraded":"2023-12-19T18:17:39.837659+0000","last_fullsized":"2023-12-19T18:17:39.837659+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:42:22.136168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413591+0000","last_change":"2023-12-19T18:17:27.353410+0000","last_active":"2023-12-19T18:17:39.413591+0000","last_peered":"2023-12-19T18:17:39.413591+0000","last_clean":"2023-12-19T18:17:39.413591+0000","last_became_active":"2023-12-19T18:17:27.352446+0000","last_became_peered":"2023-12-19T18:17:27.352446+0000","last_unstale":"2023-12-19T18:17:39.413591+0000","last_undegraded":"2023-12-19T18:17:39.413591+0000","last_fullsized":"2023-12-19T18:17:39.413591+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:55.042619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.839318+0000","last_change":"2023-12-19T18:17:39.833095+0000","last_active":"2023-12-19T18:17:39.839318+0000","last_peered":"2023-12-19T18:17:39.839318+0000","last_clean":"2023-12-19T18:17:39.839318+0000","last_became_active":"2023-12-19T18:17:39.832795+0000","last_became_peered":"2023-12-19T18:17:39.832795+0000","last_unstale":"2023-12-19T18:17:39.839318+0000","last_undegraded":"2023-12-19T18:17:39.839318+0000","last_fullsized":"2023-12-19T18:17:39.839318+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:09.670408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.409997+0000","last_change":"2023-12-19T18:17:27.358427+0000","last_active":"2023-12-19T18:17:39.409997+0000","last_peered":"2023-12-19T18:17:39.409997+0000","last_clean":"2023-12-19T18:17:39.409997+0000","last_became_active":"2023-12-19T18:17:27.358141+0000","last_became_peered":"2023-12-19T18:17:27.358141+0000","last_unstale":"2023-12-19T18:17:39.409997+0000","last_undegraded":"2023-12-19T18:17:39.409997+0000","last_fullsized":"2023-12-19T18:17:39.409997+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:14.654686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.3f","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.821083+0000","last_change":"2023-12-19T18:17:39.815673+0000","last_active":"2023-12-19T18:17:39.821083+0000","last_peered":"2023-12-19T18:17:39.821083+0000","last_clean":"2023-12-19T18:17:39.821083+0000","last_became_active":"2023-12-19T18:17:39.815424+0000","last_became_peered":"2023-12-19T18:17:39.815424+0000","last_unstale":"2023-12-19T18:17:39.821083+0000","last_undegraded":"2023-12-19T18:17:39.821083+0000","last_fullsized":"2023-12-19T18:17:39.821083+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:18:11.577769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820465+0000","last_change":"2023-12-19T18:17:27.356031+0000","last_active":"2023-12-19T18:17:39.820465+0000","last_peered":"2023-12-19T18:17:39.820465+0000","last_clean":"2023-12-19T18:17:39.820465+0000","last_became_active":"2023-12-19T18:17:27.354195+0000","last_became_peered":"2023-12-19T18:17:27.354195+0000","last_unstale":"2023-12-19T18:17:39.820465+0000","last_undegraded":"2023-12-19T18:17:39.820465+0000","last_fullsized":"2023-12-19T18:17:39.820465+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:16.644439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.826135+0000","last_change":"2023-12-19T18:17:39.818843+0000","last_active":"2023-12-19T18:17:39.826135+0000","last_peered":"2023-12-19T18:17:39.826135+0000","last_clean":"2023-12-19T18:17:39.826135+0000","last_became_active":"2023-12-19T18:17:39.818569+0000","last_became_peered":"2023-12-19T18:17:39.818569+0000","last_unstale":"2023-12-19T18:17:39.826135+0000","last_undegraded":"2023-12-19T18:17:39.826135+0000","last_fullsized":"2023-12-19T18:17:39.826135+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:20:14.678399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413904+0000","last_change":"2023-12-19T18:17:27.348408+0000","last_active":"2023-12-19T18:17:39.413904+0000","last_peered":"2023-12-19T18:17:39.413904+0000","last_clean":"2023-12-19T18:17:39.413904+0000","last_became_active":"2023-12-19T18:17:27.348138+0000","last_became_peered":"2023-12-19T18:17:27.348138+0000","last_unstale":"2023-12-19T18:17:39.413904+0000","last_undegraded":"2023-12-19T18:17:39.413904+0000","last_fullsized":"2023-12-19T18:17:39.413904+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:19.968471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812300+0000","last_change":"2023-12-19T18:17:39.839350+0000","last_active":"2023-12-19T18:17:40.812300+0000","last_peered":"2023-12-19T18:17:40.812300+0000","last_clean":"2023-12-19T18:17:40.812300+0000","last_became_active":"2023-12-19T18:17:39.839054+0000","last_became_peered":"2023-12-19T18:17:39.839054+0000","last_unstale":"2023-12-19T18:17:40.812300+0000","last_undegraded":"2023-12-19T18:17:40.812300+0000","last_fullsized":"2023-12-19T18:17:40.812300+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:14:03.086377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413526+0000","last_change":"2023-12-19T18:17:27.351573+0000","last_active":"2023-12-19T18:17:39.413526+0000","last_peered":"2023-12-19T18:17:39.413526+0000","last_clean":"2023-12-19T18:17:39.413526+0000","last_became_active":"2023-12-19T18:17:27.351222+0000","last_became_peered":"2023-12-19T18:17:27.351222+0000","last_unstale":"2023-12-19T18:17:39.413526+0000","last_undegraded":"2023-12-19T18:17:39.413526+0000","last_fullsized":"2023-12-19T18:17:39.413526+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:17:31.537417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.829015+0000","last_change":"2023-12-19T18:17:39.825005+0000","last_active":"2023-12-19T18:17:39.829015+0000","last_peered":"2023-12-19T18:17:39.829015+0000","last_clean":"2023-12-19T18:17:39.829015+0000","last_became_active":"2023-12-19T18:17:39.824659+0000","last_became_peered":"2023-12-19T18:17:39.824659+0000","last_unstale":"2023-12-19T18:17:39.829015+0000","last_undegraded":"2023-12-19T18:17:39.829015+0000","last_fullsized":"2023-12-19T18:17:39.829015+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:15:55.948961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.405823+0000","last_change":"2023-12-19T18:17:27.364283+0000","last_active":"2023-12-19T18:17:39.405823+0000","last_peered":"2023-12-19T18:17:39.405823+0000","last_clean":"2023-12-19T18:17:39.405823+0000","last_became_active":"2023-12-19T18:17:27.363962+0000","last_became_peered":"2023-12-19T18:17:27.363962+0000","last_unstale":"2023-12-19T18:17:39.405823+0000","last_undegraded":"2023-12-19T18:17:39.405823+0000","last_fullsized":"2023-12-19T18:17:39.405823+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:25.159077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.811721+0000","last_change":"2023-12-19T18:17:39.823676+0000","last_active":"2023-12-19T18:17:40.811721+0000","last_peered":"2023-12-19T18:17:40.811721+0000","last_clean":"2023-12-19T18:17:40.811721+0000","last_became_active":"2023-12-19T18:17:39.823405+0000","last_became_peered":"2023-12-19T18:17:39.823405+0000","last_unstale":"2023-12-19T18:17:40.811721+0000","last_undegraded":"2023-12-19T18:17:40.811721+0000","last_fullsized":"2023-12-19T18:17:40.811721+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:31:18.872287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820413+0000","last_change":"2023-12-19T18:17:27.357178+0000","last_active":"2023-12-19T18:17:39.820413+0000","last_peered":"2023-12-19T18:17:39.820413+0000","last_clean":"2023-12-19T18:17:39.820413+0000","last_became_active":"2023-12-19T18:17:27.356893+0000","last_became_peered":"2023-12-19T18:17:27.356893+0000","last_unstale":"2023-12-19T18:17:39.820413+0000","last_undegraded":"2023-12-19T18:17:39.820413+0000","last_fullsized":"2023-12-19T18:17:39.820413+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:31.153454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.840809+0000","last_change":"2023-12-19T18:17:39.838729+0000","last_active":"2023-12-19T18:17:39.840809+0000","last_peered":"2023-12-19T18:17:39.840809+0000","last_clean":"2023-12-19T18:17:39.840809+0000","last_became_active":"2023-12-19T18:17:39.838043+0000","last_became_peered":"2023-12-19T18:17:39.838043+0000","last_unstale":"2023-12-19T18:17:39.840809+0000","last_undegraded":"2023-12-19T18:17:39.840809+0000","last_fullsized":"2023-12-19T18:17:39.840809+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:02.711769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408878+0000","last_change":"2023-12-19T18:17:27.344049+0000","last_active":"2023-12-19T18:17:39.408878+0000","last_peered":"2023-12-19T18:17:39.408878+0000","last_clean":"2023-12-19T18:17:39.408878+0000","last_became_active":"2023-12-19T18:17:27.343738+0000","last_became_peered":"2023-12-19T18:17:27.343738+0000","last_unstale":"2023-12-19T18:17:39.408878+0000","last_undegraded":"2023-12-19T18:17:39.408878+0000","last_fullsized":"2023-12-19T18:17:39.408878+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:31.445521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809457+0000","last_change":"2023-12-19T18:17:39.844452+0000","last_active":"2023-12-19T18:17:40.809457+0000","last_peered":"2023-12-19T18:17:40.809457+0000","last_clean":"2023-12-19T18:17:40.809457+0000","last_became_active":"2023-12-19T18:17:39.844014+0000","last_became_peered":"2023-12-19T18:17:39.844014+0000","last_unstale":"2023-12-19T18:17:40.809457+0000","last_undegraded":"2023-12-19T18:17:40.809457+0000","last_fullsized":"2023-12-19T18:17:40.809457+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:52:35.124201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.822570+0000","last_change":"2023-12-19T18:17:27.363691+0000","last_active":"2023-12-19T18:17:39.822570+0000","last_peered":"2023-12-19T18:17:39.822570+0000","last_clean":"2023-12-19T18:17:39.822570+0000","last_became_active":"2023-12-19T18:17:27.363412+0000","last_became_peered":"2023-12-19T18:17:27.363412+0000","last_unstale":"2023-12-19T18:17:39.822570+0000","last_undegraded":"2023-12-19T18:17:39.822570+0000","last_fullsized":"2023-12-19T18:17:39.822570+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:47.826883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.826556+0000","last_change":"2023-12-19T18:17:39.822612+0000","last_active":"2023-12-19T18:17:39.826556+0000","last_peered":"2023-12-19T18:17:39.826556+0000","last_clean":"2023-12-19T18:17:39.826556+0000","last_became_active":"2023-12-19T18:17:39.822251+0000","last_became_peered":"2023-12-19T18:17:39.822251+0000","last_unstale":"2023-12-19T18:17:39.826556+0000","last_undegraded":"2023-12-19T18:17:39.826556+0000","last_fullsized":"2023-12-19T18:17:39.826556+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:14:03.086377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.417741+0000","last_change":"2023-12-19T18:17:27.343713+0000","last_active":"2023-12-19T18:17:39.417741+0000","last_peered":"2023-12-19T18:17:39.417741+0000","last_clean":"2023-12-19T18:17:39.417741+0000","last_became_active":"2023-12-19T18:17:27.343409+0000","last_became_peered":"2023-12-19T18:17:27.343409+0000","last_unstale":"2023-12-19T18:17:39.417741+0000","last_undegraded":"2023-12-19T18:17:39.417741+0000","last_fullsized":"2023-12-19T18:17:39.417741+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:40.489404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.843459+0000","last_change":"2023-12-19T18:17:39.842790+0000","last_active":"2023-12-19T18:17:39.843459+0000","last_peered":"2023-12-19T18:17:39.843459+0000","last_clean":"2023-12-19T18:17:39.843459+0000","last_became_active":"2023-12-19T18:17:39.842464+0000","last_became_peered":"2023-12-19T18:17:39.842464+0000","last_unstale":"2023-12-19T18:17:39.843459+0000","last_undegraded":"2023-12-19T18:17:39.843459+0000","last_fullsized":"2023-12-19T18:17:39.843459+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:05.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416082+0000","last_change":"2023-12-19T18:17:27.353890+0000","last_active":"2023-12-19T18:17:39.416082+0000","last_peered":"2023-12-19T18:17:39.416082+0000","last_clean":"2023-12-19T18:17:39.416082+0000","last_became_active":"2023-12-19T18:17:27.352505+0000","last_became_peered":"2023-12-19T18:17:27.352505+0000","last_unstale":"2023-12-19T18:17:39.416082+0000","last_undegraded":"2023-12-19T18:17:39.416082+0000","last_fullsized":"2023-12-19T18:17:39.416082+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:49:56.496681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.813410+0000","last_change":"2023-12-19T18:17:39.830133+0000","last_active":"2023-12-19T18:17:40.813410+0000","last_peered":"2023-12-19T18:17:40.813410+0000","last_clean":"2023-12-19T18:17:40.813410+0000","last_became_active":"2023-12-19T18:17:39.829808+0000","last_became_peered":"2023-12-19T18:17:39.829808+0000","last_unstale":"2023-12-19T18:17:40.813410+0000","last_undegraded":"2023-12-19T18:17:40.813410+0000","last_fullsized":"2023-12-19T18:17:40.813410+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:44.450250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408825+0000","last_change":"2023-12-19T18:17:27.353636+0000","last_active":"2023-12-19T18:17:39.408825+0000","last_peered":"2023-12-19T18:17:39.408825+0000","last_clean":"2023-12-19T18:17:39.408825+0000","last_became_active":"2023-12-19T18:17:27.353234+0000","last_became_peered":"2023-12-19T18:17:27.353234+0000","last_unstale":"2023-12-19T18:17:39.408825+0000","last_undegraded":"2023-12-19T18:17:39.408825+0000","last_fullsized":"2023-12-19T18:17:39.408825+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:50.421353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.843881+0000","last_change":"2023-12-19T18:17:39.843212+0000","last_active":"2023-12-19T18:17:39.843881+0000","last_peered":"2023-12-19T18:17:39.843881+0000","last_clean":"2023-12-19T18:17:39.843881+0000","last_became_active":"2023-12-19T18:17:39.842963+0000","last_became_peered":"2023-12-19T18:17:39.842963+0000","last_unstale":"2023-12-19T18:17:39.843881+0000","last_undegraded":"2023-12-19T18:17:39.843881+0000","last_fullsized":"2023-12-19T18:17:39.843881+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:34:52.684868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408357+0000","last_change":"2023-12-19T18:17:27.368093+0000","last_active":"2023-12-19T18:17:39.408357+0000","last_peered":"2023-12-19T18:17:39.408357+0000","last_clean":"2023-12-19T18:17:39.408357+0000","last_became_active":"2023-12-19T18:17:27.367300+0000","last_became_peered":"2023-12-19T18:17:27.367300+0000","last_unstale":"2023-12-19T18:17:39.408357+0000","last_undegraded":"2023-12-19T18:17:39.408357+0000","last_fullsized":"2023-12-19T18:17:39.408357+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:16.644439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.838226+0000","last_change":"2023-12-19T18:17:39.833455+0000","last_active":"2023-12-19T18:17:39.838226+0000","last_peered":"2023-12-19T18:17:39.838226+0000","last_clean":"2023-12-19T18:17:39.838226+0000","last_became_active":"2023-12-19T18:17:39.832753+0000","last_became_peered":"2023-12-19T18:17:39.832753+0000","last_unstale":"2023-12-19T18:17:39.838226+0000","last_undegraded":"2023-12-19T18:17:39.838226+0000","last_fullsized":"2023-12-19T18:17:39.838226+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:58:27.250517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810496+0000","last_change":"2023-12-19T18:17:27.362240+0000","last_active":"2023-12-19T18:17:39.810496+0000","last_peered":"2023-12-19T18:17:39.810496+0000","last_clean":"2023-12-19T18:17:39.810496+0000","last_became_active":"2023-12-19T18:17:27.361917+0000","last_became_peered":"2023-12-19T18:17:27.361917+0000","last_unstale":"2023-12-19T18:17:39.810496+0000","last_undegraded":"2023-12-19T18:17:39.810496+0000","last_fullsized":"2023-12-19T18:17:39.810496+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:44:22.931173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812849+0000","last_change":"2023-12-19T18:17:39.839497+0000","last_active":"2023-12-19T18:17:40.812849+0000","last_peered":"2023-12-19T18:17:40.812849+0000","last_clean":"2023-12-19T18:17:40.812849+0000","last_became_active":"2023-12-19T18:17:39.839216+0000","last_became_peered":"2023-12-19T18:17:39.839216+0000","last_unstale":"2023-12-19T18:17:40.812849+0000","last_undegraded":"2023-12-19T18:17:40.812849+0000","last_fullsized":"2023-12-19T18:17:40.812849+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:34:52.684868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830166+0000","last_change":"2023-12-19T18:17:27.356217+0000","last_active":"2023-12-19T18:17:39.830166+0000","last_peered":"2023-12-19T18:17:39.830166+0000","last_clean":"2023-12-19T18:17:39.830166+0000","last_became_active":"2023-12-19T18:17:27.355721+0000","last_became_peered":"2023-12-19T18:17:27.355721+0000","last_unstale":"2023-12-19T18:17:39.830166+0000","last_undegraded":"2023-12-19T18:17:39.830166+0000","last_fullsized":"2023-12-19T18:17:39.830166+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:20:57.806100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.823931+0000","last_change":"2023-12-19T18:17:39.819672+0000","last_active":"2023-12-19T18:17:39.823931+0000","last_peered":"2023-12-19T18:17:39.823931+0000","last_clean":"2023-12-19T18:17:39.823931+0000","last_became_active":"2023-12-19T18:17:39.819371+0000","last_became_peered":"2023-12-19T18:17:39.819371+0000","last_unstale":"2023-12-19T18:17:39.823931+0000","last_undegraded":"2023-12-19T18:17:39.823931+0000","last_fullsized":"2023-12-19T18:17:39.823931+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:34:51.758688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408201+0000","last_change":"2023-12-19T18:17:27.350328+0000","last_active":"2023-12-19T18:17:39.408201+0000","last_peered":"2023-12-19T18:17:39.408201+0000","last_clean":"2023-12-19T18:17:39.408201+0000","last_became_active":"2023-12-19T18:17:27.350006+0000","last_became_peered":"2023-12-19T18:17:27.350006+0000","last_unstale":"2023-12-19T18:17:39.408201+0000","last_undegraded":"2023-12-19T18:17:39.408201+0000","last_fullsized":"2023-12-19T18:17:39.408201+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:00.178479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.846852+0000","last_change":"2023-12-19T18:17:39.845313+0000","last_active":"2023-12-19T18:17:39.846852+0000","last_peered":"2023-12-19T18:17:39.846852+0000","last_clean":"2023-12-19T18:17:39.846852+0000","last_became_active":"2023-12-19T18:17:39.845016+0000","last_became_peered":"2023-12-19T18:17:39.845016+0000","last_unstale":"2023-12-19T18:17:39.846852+0000","last_undegraded":"2023-12-19T18:17:39.846852+0000","last_fullsized":"2023-12-19T18:17:39.846852+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:17:44.002667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810896+0000","last_change":"2023-12-19T18:17:27.367615+0000","last_active":"2023-12-19T18:17:39.810896+0000","last_peered":"2023-12-19T18:17:39.810896+0000","last_clean":"2023-12-19T18:17:39.810896+0000","last_became_active":"2023-12-19T18:17:27.366800+0000","last_became_peered":"2023-12-19T18:17:27.366800+0000","last_unstale":"2023-12-19T18:17:39.810896+0000","last_undegraded":"2023-12-19T18:17:39.810896+0000","last_fullsized":"2023-12-19T18:17:39.810896+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:19:50.020207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.825903+0000","last_change":"2023-12-19T18:17:39.818585+0000","last_active":"2023-12-19T18:17:39.825903+0000","last_peered":"2023-12-19T18:17:39.825903+0000","last_clean":"2023-12-19T18:17:39.825903+0000","last_became_active":"2023-12-19T18:17:39.818218+0000","last_became_peered":"2023-12-19T18:17:39.818218+0000","last_unstale":"2023-12-19T18:17:39.825903+0000","last_undegraded":"2023-12-19T18:17:39.825903+0000","last_fullsized":"2023-12-19T18:17:39.825903+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:45.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.417825+0000","last_change":"2023-12-19T18:17:27.357129+0000","last_active":"2023-12-19T18:17:39.417825+0000","last_peered":"2023-12-19T18:17:39.417825+0000","last_clean":"2023-12-19T18:17:39.417825+0000","last_became_active":"2023-12-19T18:17:27.356614+0000","last_became_peered":"2023-12-19T18:17:27.356614+0000","last_unstale":"2023-12-19T18:17:39.417825+0000","last_undegraded":"2023-12-19T18:17:39.417825+0000","last_fullsized":"2023-12-19T18:17:39.417825+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:22.773760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.842026+0000","last_change":"2023-12-19T18:17:39.841034+0000","last_active":"2023-12-19T18:17:39.842026+0000","last_peered":"2023-12-19T18:17:39.842026+0000","last_clean":"2023-12-19T18:17:39.842026+0000","last_became_active":"2023-12-19T18:17:39.840678+0000","last_became_peered":"2023-12-19T18:17:39.840678+0000","last_unstale":"2023-12-19T18:17:39.842026+0000","last_undegraded":"2023-12-19T18:17:39.842026+0000","last_fullsized":"2023-12-19T18:17:39.842026+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:44:21.703567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413550+0000","last_change":"2023-12-19T18:17:27.353111+0000","last_active":"2023-12-19T18:17:39.413550+0000","last_peered":"2023-12-19T18:17:39.413550+0000","last_clean":"2023-12-19T18:17:39.413550+0000","last_became_active":"2023-12-19T18:17:27.352840+0000","last_became_peered":"2023-12-19T18:17:27.352840+0000","last_unstale":"2023-12-19T18:17:39.413550+0000","last_undegraded":"2023-12-19T18:17:39.413550+0000","last_fullsized":"2023-12-19T18:17:39.413550+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:28.649300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830572+0000","last_change":"2023-12-19T18:17:39.825874+0000","last_active":"2023-12-19T18:17:39.830572+0000","last_peered":"2023-12-19T18:17:39.830572+0000","last_clean":"2023-12-19T18:17:39.830572+0000","last_became_active":"2023-12-19T18:17:39.823789+0000","last_became_peered":"2023-12-19T18:17:39.823789+0000","last_unstale":"2023-12-19T18:17:39.830572+0000","last_undegraded":"2023-12-19T18:17:39.830572+0000","last_fullsized":"2023-12-19T18:17:39.830572+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:04:11.535860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416217+0000","last_change":"2023-12-19T18:17:27.340972+0000","last_active":"2023-12-19T18:17:39.416217+0000","last_peered":"2023-12-19T18:17:39.416217+0000","last_clean":"2023-12-19T18:17:39.416217+0000","last_became_active":"2023-12-19T18:17:27.340523+0000","last_became_peered":"2023-12-19T18:17:27.340523+0000","last_unstale":"2023-12-19T18:17:39.416217+0000","last_undegraded":"2023-12-19T18:17:39.416217+0000","last_fullsized":"2023-12-19T18:17:39.416217+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:50.621127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.828320+0000","last_change":"2023-12-19T18:17:39.823149+0000","last_active":"2023-12-19T18:17:39.828320+0000","last_peered":"2023-12-19T18:17:39.828320+0000","last_clean":"2023-12-19T18:17:39.828320+0000","last_became_active":"2023-12-19T18:17:39.822891+0000","last_became_peered":"2023-12-19T18:17:39.822891+0000","last_unstale":"2023-12-19T18:17:39.828320+0000","last_undegraded":"2023-12-19T18:17:39.828320+0000","last_fullsized":"2023-12-19T18:17:39.828320+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:04:45.527849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413613+0000","last_change":"2023-12-19T18:17:27.355317+0000","last_active":"2023-12-19T18:17:39.413613+0000","last_peered":"2023-12-19T18:17:39.413613+0000","last_clean":"2023-12-19T18:17:39.413613+0000","last_became_active":"2023-12-19T18:17:27.355002+0000","last_became_peered":"2023-12-19T18:17:27.355002+0000","last_unstale":"2023-12-19T18:17:39.413613+0000","last_undegraded":"2023-12-19T18:17:39.413613+0000","last_fullsized":"2023-12-19T18:17:39.413613+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:50.621127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.842164+0000","last_change":"2023-12-19T18:17:39.838759+0000","last_active":"2023-12-19T18:17:39.842164+0000","last_peered":"2023-12-19T18:17:39.842164+0000","last_clean":"2023-12-19T18:17:39.842164+0000","last_became_active":"2023-12-19T18:17:39.838479+0000","last_became_peered":"2023-12-19T18:17:39.838479+0000","last_unstale":"2023-12-19T18:17:39.842164+0000","last_undegraded":"2023-12-19T18:17:39.842164+0000","last_fullsized":"2023-12-19T18:17:39.842164+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:38:23.125499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834118+0000","last_change":"2023-12-19T18:17:27.347478+0000","last_active":"2023-12-19T18:17:39.834118+0000","last_peered":"2023-12-19T18:17:39.834118+0000","last_clean":"2023-12-19T18:17:39.834118+0000","last_became_active":"2023-12-19T18:17:27.346998+0000","last_became_peered":"2023-12-19T18:17:27.346998+0000","last_unstale":"2023-12-19T18:17:39.834118+0000","last_undegraded":"2023-12-19T18:17:39.834118+0000","last_fullsized":"2023-12-19T18:17:39.834118+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:07:22.470263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834392+0000","last_change":"2023-12-19T18:17:39.832098+0000","last_active":"2023-12-19T18:17:39.834392+0000","last_peered":"2023-12-19T18:17:39.834392+0000","last_clean":"2023-12-19T18:17:39.834392+0000","last_became_active":"2023-12-19T18:17:39.831637+0000","last_became_peered":"2023-12-19T18:17:39.831637+0000","last_unstale":"2023-12-19T18:17:39.834392+0000","last_undegraded":"2023-12-19T18:17:39.834392+0000","last_fullsized":"2023-12-19T18:17:39.834392+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:32.433721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416878+0000","last_change":"2023-12-19T18:17:27.339790+0000","last_active":"2023-12-19T18:17:39.416878+0000","last_peered":"2023-12-19T18:17:39.416878+0000","last_clean":"2023-12-19T18:17:39.416878+0000","last_became_active":"2023-12-19T18:17:27.339477+0000","last_became_peered":"2023-12-19T18:17:27.339477+0000","last_unstale":"2023-12-19T18:17:39.416878+0000","last_undegraded":"2023-12-19T18:17:39.416878+0000","last_fullsized":"2023-12-19T18:17:39.416878+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:23:29.725587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812196+0000","last_change":"2023-12-19T18:17:39.831883+0000","last_active":"2023-12-19T18:17:40.812196+0000","last_peered":"2023-12-19T18:17:40.812196+0000","last_clean":"2023-12-19T18:17:40.812196+0000","last_became_active":"2023-12-19T18:17:39.831600+0000","last_became_peered":"2023-12-19T18:17:39.831600+0000","last_unstale":"2023-12-19T18:17:40.812196+0000","last_undegraded":"2023-12-19T18:17:40.812196+0000","last_fullsized":"2023-12-19T18:17:40.812196+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:55:24.562567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408836+0000","last_change":"2023-12-19T18:17:27.348543+0000","last_active":"2023-12-19T18:17:39.408836+0000","last_peered":"2023-12-19T18:17:39.408836+0000","last_clean":"2023-12-19T18:17:39.408836+0000","last_became_active":"2023-12-19T18:17:27.348257+0000","last_became_peered":"2023-12-19T18:17:27.348257+0000","last_unstale":"2023-12-19T18:17:39.408836+0000","last_undegraded":"2023-12-19T18:17:39.408836+0000","last_fullsized":"2023-12-19T18:17:39.408836+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:44:09.238317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809544+0000","last_change":"2023-12-19T18:17:39.844300+0000","last_active":"2023-12-19T18:17:40.809544+0000","last_peered":"2023-12-19T18:17:40.809544+0000","last_clean":"2023-12-19T18:17:40.809544+0000","last_became_active":"2023-12-19T18:17:39.843747+0000","last_became_peered":"2023-12-19T18:17:39.843747+0000","last_unstale":"2023-12-19T18:17:40.809544+0000","last_undegraded":"2023-12-19T18:17:40.809544+0000","last_fullsized":"2023-12-19T18:17:40.809544+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:58:44.245127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413680+0000","last_change":"2023-12-19T18:17:27.353080+0000","last_active":"2023-12-19T18:17:39.413680+0000","last_peered":"2023-12-19T18:17:39.413680+0000","last_clean":"2023-12-19T18:17:39.413680+0000","last_became_active":"2023-12-19T18:17:27.352664+0000","last_became_peered":"2023-12-19T18:17:27.352664+0000","last_unstale":"2023-12-19T18:17:39.413680+0000","last_undegraded":"2023-12-19T18:17:39.413680+0000","last_fullsized":"2023-12-19T18:17:39.413680+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:22:10.869020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.825949+0000","last_change":"2023-12-19T18:17:39.817852+0000","last_active":"2023-12-19T18:17:39.825949+0000","last_peered":"2023-12-19T18:17:39.825949+0000","last_clean":"2023-12-19T18:17:39.825949+0000","last_became_active":"2023-12-19T18:17:39.817606+0000","last_became_peered":"2023-12-19T18:17:39.817606+0000","last_unstale":"2023-12-19T18:17:39.825949+0000","last_undegraded":"2023-12-19T18:17:39.825949+0000","last_fullsized":"2023-12-19T18:17:39.825949+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:39:15.230291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.412507+0000","last_change":"2023-12-19T18:17:27.367346+0000","last_active":"2023-12-19T18:17:39.412507+0000","last_peered":"2023-12-19T18:17:39.412507+0000","last_clean":"2023-12-19T18:17:39.412507+0000","last_became_active":"2023-12-19T18:17:27.367053+0000","last_became_peered":"2023-12-19T18:17:27.367053+0000","last_unstale":"2023-12-19T18:17:39.412507+0000","last_undegraded":"2023-12-19T18:17:39.412507+0000","last_fullsized":"2023-12-19T18:17:39.412507+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:44.734608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.840547+0000","last_change":"2023-12-19T18:17:39.837983+0000","last_active":"2023-12-19T18:17:39.840547+0000","last_peered":"2023-12-19T18:17:39.840547+0000","last_clean":"2023-12-19T18:17:39.840547+0000","last_became_active":"2023-12-19T18:17:39.837721+0000","last_became_peered":"2023-12-19T18:17:39.837721+0000","last_unstale":"2023-12-19T18:17:39.840547+0000","last_undegraded":"2023-12-19T18:17:39.840547+0000","last_fullsized":"2023-12-19T18:17:39.840547+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:01:49.026786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.415841+0000","last_change":"2023-12-19T18:17:27.355178+0000","last_active":"2023-12-19T18:17:39.415841+0000","last_peered":"2023-12-19T18:17:39.415841+0000","last_clean":"2023-12-19T18:17:39.415841+0000","last_became_active":"2023-12-19T18:17:27.354771+0000","last_became_peered":"2023-12-19T18:17:27.354771+0000","last_unstale":"2023-12-19T18:17:39.415841+0000","last_undegraded":"2023-12-19T18:17:39.415841+0000","last_fullsized":"2023-12-19T18:17:39.415841+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:06.555153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.813367+0000","last_change":"2023-12-19T18:17:39.846098+0000","last_active":"2023-12-19T18:17:40.813367+0000","last_peered":"2023-12-19T18:17:40.813367+0000","last_clean":"2023-12-19T18:17:40.813367+0000","last_became_active":"2023-12-19T18:17:39.845679+0000","last_became_peered":"2023-12-19T18:17:39.845679+0000","last_unstale":"2023-12-19T18:17:40.813367+0000","last_undegraded":"2023-12-19T18:17:40.813367+0000","last_fullsized":"2023-12-19T18:17:40.813367+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:05.752218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416296+0000","last_change":"2023-12-19T18:17:27.349197+0000","last_active":"2023-12-19T18:17:39.416296+0000","last_peered":"2023-12-19T18:17:39.416296+0000","last_clean":"2023-12-19T18:17:39.416296+0000","last_became_active":"2023-12-19T18:17:27.348940+0000","last_became_peered":"2023-12-19T18:17:27.348940+0000","last_unstale":"2023-12-19T18:17:39.416296+0000","last_undegraded":"2023-12-19T18:17:39.416296+0000","last_fullsized":"2023-12-19T18:17:39.416296+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:19.968471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.831810+0000","last_change":"2023-12-19T18:17:39.823044+0000","last_active":"2023-12-19T18:17:39.831810+0000","last_peered":"2023-12-19T18:17:39.831810+0000","last_clean":"2023-12-19T18:17:39.831810+0000","last_became_active":"2023-12-19T18:17:39.822772+0000","last_became_peered":"2023-12-19T18:17:39.822772+0000","last_unstale":"2023-12-19T18:17:39.831810+0000","last_undegraded":"2023-12-19T18:17:39.831810+0000","last_fullsized":"2023-12-19T18:17:39.831810+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:50.380394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810228+0000","last_change":"2023-12-19T18:17:27.369700+0000","last_active":"2023-12-19T18:17:39.810228+0000","last_peered":"2023-12-19T18:17:39.810228+0000","last_clean":"2023-12-19T18:17:39.810228+0000","last_became_active":"2023-12-19T18:17:27.369398+0000","last_became_peered":"2023-12-19T18:17:27.369398+0000","last_unstale":"2023-12-19T18:17:39.810228+0000","last_undegraded":"2023-12-19T18:17:39.810228+0000","last_fullsized":"2023-12-19T18:17:39.810228+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:35:43.593726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.833651+0000","last_change":"2023-12-19T18:17:39.829518+0000","last_active":"2023-12-19T18:17:39.833651+0000","last_peered":"2023-12-19T18:17:39.833651+0000","last_clean":"2023-12-19T18:17:39.833651+0000","last_became_active":"2023-12-19T18:17:39.829226+0000","last_became_peered":"2023-12-19T18:17:39.829226+0000","last_unstale":"2023-12-19T18:17:39.833651+0000","last_undegraded":"2023-12-19T18:17:39.833651+0000","last_fullsized":"2023-12-19T18:17:39.833651+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:09:23.641063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.414643+0000","last_change":"2023-12-19T18:17:27.360694+0000","last_active":"2023-12-19T18:17:39.414643+0000","last_peered":"2023-12-19T18:17:39.414643+0000","last_clean":"2023-12-19T18:17:39.414643+0000","last_became_active":"2023-12-19T18:17:27.357706+0000","last_became_peered":"2023-12-19T18:17:27.357706+0000","last_unstale":"2023-12-19T18:17:39.414643+0000","last_undegraded":"2023-12-19T18:17:39.414643+0000","last_fullsized":"2023-12-19T18:17:39.414643+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:22:01.458247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809006+0000","last_change":"2023-12-19T18:17:39.838929+0000","last_active":"2023-12-19T18:17:40.809006+0000","last_peered":"2023-12-19T18:17:40.809006+0000","last_clean":"2023-12-19T18:17:40.809006+0000","last_became_active":"2023-12-19T18:17:39.838626+0000","last_became_peered":"2023-12-19T18:17:39.838626+0000","last_unstale":"2023-12-19T18:17:40.809006+0000","last_undegraded":"2023-12-19T18:17:40.809006+0000","last_fullsized":"2023-12-19T18:17:40.809006+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:51:43.486308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413594+0000","last_change":"2023-12-19T18:17:27.361414+0000","last_active":"2023-12-19T18:17:39.413594+0000","last_peered":"2023-12-19T18:17:39.413594+0000","last_clean":"2023-12-19T18:17:39.413594+0000","last_became_active":"2023-12-19T18:17:27.361054+0000","last_became_peered":"2023-12-19T18:17:27.361054+0000","last_unstale":"2023-12-19T18:17:39.413594+0000","last_undegraded":"2023-12-19T18:17:39.413594+0000","last_fullsized":"2023-12-19T18:17:39.413594+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:04:33.062599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.811709+0000","last_change":"2023-12-19T18:17:39.832398+0000","last_active":"2023-12-19T18:17:40.811709+0000","last_peered":"2023-12-19T18:17:40.811709+0000","last_clean":"2023-12-19T18:17:40.811709+0000","last_became_active":"2023-12-19T18:17:39.832061+0000","last_became_peered":"2023-12-19T18:17:39.832061+0000","last_unstale":"2023-12-19T18:17:40.811709+0000","last_undegraded":"2023-12-19T18:17:40.811709+0000","last_fullsized":"2023-12-19T18:17:40.811709+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:10:02.886603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.831200+0000","last_change":"2023-12-19T18:17:27.359512+0000","last_active":"2023-12-19T18:17:39.831200+0000","last_peered":"2023-12-19T18:17:39.831200+0000","last_clean":"2023-12-19T18:17:39.831200+0000","last_became_active":"2023-12-19T18:17:27.359173+0000","last_became_peered":"2023-12-19T18:17:27.359173+0000","last_unstale":"2023-12-19T18:17:39.831200+0000","last_undegraded":"2023-12-19T18:17:39.831200+0000","last_fullsized":"2023-12-19T18:17:39.831200+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:42.446500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809011+0000","last_change":"2023-12-19T18:17:39.839100+0000","last_active":"2023-12-19T18:17:40.809011+0000","last_peered":"2023-12-19T18:17:40.809011+0000","last_clean":"2023-12-19T18:17:40.809011+0000","last_became_active":"2023-12-19T18:17:39.838793+0000","last_became_peered":"2023-12-19T18:17:39.838793+0000","last_unstale":"2023-12-19T18:17:40.809011+0000","last_undegraded":"2023-12-19T18:17:40.809011+0000","last_fullsized":"2023-12-19T18:17:40.809011+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:40:14.968536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"236'2","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.835110+0000","last_change":"2023-12-19T18:17:39.422159+0000","last_active":"2023-12-19T18:17:39.835110+0000","last_peered":"2023-12-19T18:17:39.835110+0000","last_clean":"2023-12-19T18:17:39.835110+0000","last_became_active":"2023-12-19T18:17:39.421685+0000","last_became_peered":"2023-12-19T18:17:39.421685+0000","last_unstale":"2023-12-19T18:17:39.835110+0000","last_undegraded":"2023-12-19T18:17:39.835110+0000","last_fullsized":"2023-12-19T18:17:39.835110+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:09.331836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.824721+0000","last_change":"2023-12-19T18:17:27.355393+0000","last_active":"2023-12-19T18:17:39.824721+0000","last_peered":"2023-12-19T18:17:39.824721+0000","last_clean":"2023-12-19T18:17:39.824721+0000","last_became_active":"2023-12-19T18:17:27.355013+0000","last_became_peered":"2023-12-19T18:17:27.355013+0000","last_unstale":"2023-12-19T18:17:39.824721+0000","last_undegraded":"2023-12-19T18:17:39.824721+0000","last_fullsized":"2023-12-19T18:17:39.824721+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:49.759094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"232'4","reported_seq":19,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.833813+0000","last_change":"2023-12-19T18:17:26.951474+0000","last_active":"2023-12-19T18:17:39.833813+0000","last_peered":"2023-12-19T18:17:39.833813+0000","last_clean":"2023-12-19T18:17:39.833813+0000","last_became_active":"2023-12-19T18:17:26.950990+0000","last_became_peered":"2023-12-19T18:17:26.950990+0000","last_unstale":"2023-12-19T18:17:39.833813+0000","last_undegraded":"2023-12-19T18:17:39.833813+0000","last_fullsized":"2023-12-19T18:17:39.833813+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:37:49.322125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.841995+0000","last_change":"2023-12-19T18:17:39.841012+0000","last_active":"2023-12-19T18:17:39.841995+0000","last_peered":"2023-12-19T18:17:39.841995+0000","last_clean":"2023-12-19T18:17:39.841995+0000","last_became_active":"2023-12-19T18:17:39.840709+0000","last_became_peered":"2023-12-19T18:17:39.840709+0000","last_unstale":"2023-12-19T18:17:39.841995+0000","last_undegraded":"2023-12-19T18:17:39.841995+0000","last_fullsized":"2023-12-19T18:17:39.841995+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:56.410749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"236'2","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.831379+0000","last_change":"2023-12-19T18:17:39.422455+0000","last_active":"2023-12-19T18:17:39.831379+0000","last_peered":"2023-12-19T18:17:39.831379+0000","last_clean":"2023-12-19T18:17:39.831379+0000","last_became_active":"2023-12-19T18:17:39.422159+0000","last_became_peered":"2023-12-19T18:17:39.422159+0000","last_unstale":"2023-12-19T18:17:39.831379+0000","last_undegraded":"2023-12-19T18:17:39.831379+0000","last_fullsized":"2023-12-19T18:17:39.831379+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:37.101616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.411522+0000","last_change":"2023-12-19T18:17:27.346026+0000","last_active":"2023-12-19T18:17:39.411522+0000","last_peered":"2023-12-19T18:17:39.411522+0000","last_clean":"2023-12-19T18:17:39.411522+0000","last_became_active":"2023-12-19T18:17:27.345752+0000","last_became_peered":"2023-12-19T18:17:27.345752+0000","last_unstale":"2023-12-19T18:17:39.411522+0000","last_undegraded":"2023-12-19T18:17:39.411522+0000","last_fullsized":"2023-12-19T18:17:39.411522+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:18.690231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"232'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.410197+0000","last_change":"2023-12-19T18:17:26.946867+0000","last_active":"2023-12-19T18:17:39.410197+0000","last_peered":"2023-12-19T18:17:39.410197+0000","last_clean":"2023-12-19T18:17:39.410197+0000","last_became_active":"2023-12-19T18:17:26.946549+0000","last_became_peered":"2023-12-19T18:17:26.946549+0000","last_unstale":"2023-12-19T18:17:39.410197+0000","last_undegraded":"2023-12-19T18:17:39.410197+0000","last_fullsized":"2023-12-19T18:17:39.410197+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:36.912904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830232+0000","last_change":"2023-12-19T18:17:39.826375+0000","last_active":"2023-12-19T18:17:39.830232+0000","last_peered":"2023-12-19T18:17:39.830232+0000","last_clean":"2023-12-19T18:17:39.830232+0000","last_became_active":"2023-12-19T18:17:39.822261+0000","last_became_peered":"2023-12-19T18:17:39.822261+0000","last_unstale":"2023-12-19T18:17:39.830232+0000","last_undegraded":"2023-12-19T18:17:39.830232+0000","last_fullsized":"2023-12-19T18:17:39.830232+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:46:57.405646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"236'1","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.813473+0000","last_change":"2023-12-19T18:17:39.422538+0000","last_active":"2023-12-19T18:17:40.813473+0000","last_peered":"2023-12-19T18:17:40.813473+0000","last_clean":"2023-12-19T18:17:40.813473+0000","last_became_active":"2023-12-19T18:17:39.422010+0000","last_became_peered":"2023-12-19T18:17:39.422010+0000","last_unstale":"2023-12-19T18:17:40.813473+0000","last_undegraded":"2023-12-19T18:17:40.813473+0000","last_fullsized":"2023-12-19T18:17:40.813473+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:43.061826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810743+0000","last_change":"2023-12-19T18:17:27.369534+0000","last_active":"2023-12-19T18:17:39.810743+0000","last_peered":"2023-12-19T18:17:39.810743+0000","last_clean":"2023-12-19T18:17:39.810743+0000","last_became_active":"2023-12-19T18:17:27.369101+0000","last_became_peered":"2023-12-19T18:17:27.369101+0000","last_unstale":"2023-12-19T18:17:39.810743+0000","last_undegraded":"2023-12-19T18:17:39.810743+0000","last_fullsized":"2023-12-19T18:17:39.810743+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:38:39.157091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"232'2","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408926+0000","last_change":"2023-12-19T18:17:26.947910+0000","last_active":"2023-12-19T18:17:39.408926+0000","last_peered":"2023-12-19T18:17:39.408926+0000","last_clean":"2023-12-19T18:17:39.408926+0000","last_became_active":"2023-12-19T18:17:26.946840+0000","last_became_peered":"2023-12-19T18:17:26.946840+0000","last_unstale":"2023-12-19T18:17:39.408926+0000","last_undegraded":"2023-12-19T18:17:39.408926+0000","last_fullsized":"2023-12-19T18:17:39.408926+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:41.372298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.838243+0000","last_change":"2023-12-19T18:17:39.833636+0000","last_active":"2023-12-19T18:17:39.838243+0000","last_peered":"2023-12-19T18:17:39.838243+0000","last_clean":"2023-12-19T18:17:39.838243+0000","last_became_active":"2023-12-19T18:17:39.833142+0000","last_became_peered":"2023-12-19T18:17:39.833142+0000","last_unstale":"2023-12-19T18:17:39.838243+0000","last_undegraded":"2023-12-19T18:17:39.838243+0000","last_fullsized":"2023-12-19T18:17:39.838243+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:23.184958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812458+0000","last_change":"2023-12-19T18:17:39.421070+0000","last_active":"2023-12-19T18:17:40.812458+0000","last_peered":"2023-12-19T18:17:40.812458+0000","last_clean":"2023-12-19T18:17:40.812458+0000","last_became_active":"2023-12-19T18:17:39.419992+0000","last_became_peered":"2023-12-19T18:17:39.419992+0000","last_unstale":"2023-12-19T18:17:40.812458+0000","last_undegraded":"2023-12-19T18:17:40.812458+0000","last_fullsized":"2023-12-19T18:17:40.812458+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:23:37.568299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.415498+0000","last_change":"2023-12-19T18:17:27.361418+0000","last_active":"2023-12-19T18:17:39.415498+0000","last_peered":"2023-12-19T18:17:39.415498+0000","last_clean":"2023-12-19T18:17:39.415498+0000","last_became_active":"2023-12-19T18:17:27.361131+0000","last_became_peered":"2023-12-19T18:17:27.361131+0000","last_unstale":"2023-12-19T18:17:39.415498+0000","last_undegraded":"2023-12-19T18:17:39.415498+0000","last_fullsized":"2023-12-19T18:17:39.415498+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:44:06.399365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"232'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.410498+0000","last_change":"2023-12-19T18:17:26.942982+0000","last_active":"2023-12-19T18:17:39.410498+0000","last_peered":"2023-12-19T18:17:39.410498+0000","last_clean":"2023-12-19T18:17:39.410498+0000","last_became_active":"2023-12-19T18:17:26.942719+0000","last_became_peered":"2023-12-19T18:17:26.942719+0000","last_unstale":"2023-12-19T18:17:39.410498+0000","last_undegraded":"2023-12-19T18:17:39.410498+0000","last_fullsized":"2023-12-19T18:17:39.410498+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:10.474482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.835751+0000","last_change":"2023-12-19T18:17:39.833354+0000","last_active":"2023-12-19T18:17:39.835751+0000","last_peered":"2023-12-19T18:17:39.835751+0000","last_clean":"2023-12-19T18:17:39.835751+0000","last_became_active":"2023-12-19T18:17:39.833103+0000","last_became_peered":"2023-12-19T18:17:39.833103+0000","last_unstale":"2023-12-19T18:17:39.835751+0000","last_undegraded":"2023-12-19T18:17:39.835751+0000","last_fullsized":"2023-12-19T18:17:39.835751+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:40:18.669674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.813201+0000","last_change":"2023-12-19T18:17:39.423176+0000","last_active":"2023-12-19T18:17:39.813201+0000","last_peered":"2023-12-19T18:17:39.813201+0000","last_clean":"2023-12-19T18:17:39.813201+0000","last_became_active":"2023-12-19T18:17:39.422890+0000","last_became_peered":"2023-12-19T18:17:39.422890+0000","last_unstale":"2023-12-19T18:17:39.813201+0000","last_undegraded":"2023-12-19T18:17:39.813201+0000","last_fullsized":"2023-12-19T18:17:39.813201+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:14:24.154817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413688+0000","last_change":"2023-12-19T18:17:27.361092+0000","last_active":"2023-12-19T18:17:39.413688+0000","last_peered":"2023-12-19T18:17:39.413688+0000","last_clean":"2023-12-19T18:17:39.413688+0000","last_became_active":"2023-12-19T18:17:27.360778+0000","last_became_peered":"2023-12-19T18:17:27.360778+0000","last_unstale":"2023-12-19T18:17:39.413688+0000","last_undegraded":"2023-12-19T18:17:39.413688+0000","last_fullsized":"2023-12-19T18:17:39.413688+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:40.489404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"232'1","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.819128+0000","last_change":"2023-12-19T18:17:26.940960+0000","last_active":"2023-12-19T18:17:39.819128+0000","last_peered":"2023-12-19T18:17:39.819128+0000","last_clean":"2023-12-19T18:17:39.819128+0000","last_became_active":"2023-12-19T18:17:26.939915+0000","last_became_peered":"2023-12-19T18:17:26.939915+0000","last_unstale":"2023-12-19T18:17:39.819128+0000","last_undegraded":"2023-12-19T18:17:39.819128+0000","last_fullsized":"2023-12-19T18:17:39.819128+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:52:17.872321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.808930+0000","last_change":"2023-12-19T18:17:39.830173+0000","last_active":"2023-12-19T18:17:40.808930+0000","last_peered":"2023-12-19T18:17:40.808930+0000","last_clean":"2023-12-19T18:17:40.808930+0000","last_became_active":"2023-12-19T18:17:39.829840+0000","last_became_peered":"2023-12-19T18:17:39.829840+0000","last_unstale":"2023-12-19T18:17:40.808930+0000","last_undegraded":"2023-12-19T18:17:40.808930+0000","last_fullsized":"2023-12-19T18:17:40.808930+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:14:56.855127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"236'1","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.844024+0000","last_change":"2023-12-19T18:17:39.422364+0000","last_active":"2023-12-19T18:17:39.844024+0000","last_peered":"2023-12-19T18:17:39.844024+0000","last_clean":"2023-12-19T18:17:39.844024+0000","last_became_active":"2023-12-19T18:17:39.422059+0000","last_became_peered":"2023-12-19T18:17:39.422059+0000","last_unstale":"2023-12-19T18:17:39.844024+0000","last_undegraded":"2023-12-19T18:17:39.844024+0000","last_fullsized":"2023-12-19T18:17:39.844024+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:09:13.480050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.404579+0000","last_change":"2023-12-19T18:17:27.347577+0000","last_active":"2023-12-19T18:17:39.404579+0000","last_peered":"2023-12-19T18:17:39.404579+0000","last_clean":"2023-12-19T18:17:39.404579+0000","last_became_active":"2023-12-19T18:17:27.346549+0000","last_became_peered":"2023-12-19T18:17:27.346549+0000","last_unstale":"2023-12-19T18:17:39.404579+0000","last_undegraded":"2023-12-19T18:17:39.404579+0000","last_fullsized":"2023-12-19T18:17:39.404579+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:28.649300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.811009+0000","last_change":"2023-12-19T18:17:26.950047+0000","last_active":"2023-12-19T18:17:39.811009+0000","last_peered":"2023-12-19T18:17:39.811009+0000","last_clean":"2023-12-19T18:17:39.811009+0000","last_became_active":"2023-12-19T18:17:26.949678+0000","last_became_peered":"2023-12-19T18:17:26.949678+0000","last_unstale":"2023-12-19T18:17:39.811009+0000","last_undegraded":"2023-12-19T18:17:39.811009+0000","last_fullsized":"2023-12-19T18:17:39.811009+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:49:16.046354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.846622+0000","last_change":"2023-12-19T18:17:39.845533+0000","last_active":"2023-12-19T18:17:39.846622+0000","last_peered":"2023-12-19T18:17:39.846622+0000","last_clean":"2023-12-19T18:17:39.846622+0000","last_became_active":"2023-12-19T18:17:39.845197+0000","last_became_peered":"2023-12-19T18:17:39.845197+0000","last_unstale":"2023-12-19T18:17:39.846622+0000","last_undegraded":"2023-12-19T18:17:39.846622+0000","last_fullsized":"2023-12-19T18:17:39.846622+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:27.280365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"236'1","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834552+0000","last_change":"2023-12-19T18:17:39.423872+0000","last_active":"2023-12-19T18:17:39.834552+0000","last_peered":"2023-12-19T18:17:39.834552+0000","last_clean":"2023-12-19T18:17:39.834552+0000","last_became_active":"2023-12-19T18:17:39.423556+0000","last_became_peered":"2023-12-19T18:17:39.423556+0000","last_unstale":"2023-12-19T18:17:39.834552+0000","last_undegraded":"2023-12-19T18:17:39.834552+0000","last_fullsized":"2023-12-19T18:17:39.834552+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:55:50.947760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.407857+0000","last_change":"2023-12-19T18:17:27.363788+0000","last_active":"2023-12-19T18:17:39.407857+0000","last_peered":"2023-12-19T18:17:39.407857+0000","last_clean":"2023-12-19T18:17:39.407857+0000","last_became_active":"2023-12-19T18:17:27.363513+0000","last_became_peered":"2023-12-19T18:17:27.363513+0000","last_unstale":"2023-12-19T18:17:39.407857+0000","last_undegraded":"2023-12-19T18:17:39.407857+0000","last_fullsized":"2023-12-19T18:17:39.407857+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:23.278389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.404674+0000","last_change":"2023-12-19T18:17:26.944745+0000","last_active":"2023-12-19T18:17:39.404674+0000","last_peered":"2023-12-19T18:17:39.404674+0000","last_clean":"2023-12-19T18:17:39.404674+0000","last_became_active":"2023-12-19T18:17:26.944481+0000","last_became_peered":"2023-12-19T18:17:26.944481+0000","last_unstale":"2023-12-19T18:17:39.404674+0000","last_undegraded":"2023-12-19T18:17:39.404674+0000","last_fullsized":"2023-12-19T18:17:39.404674+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:30.604242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.843343+0000","last_change":"2023-12-19T18:17:39.841724+0000","last_active":"2023-12-19T18:17:39.843343+0000","last_peered":"2023-12-19T18:17:39.843343+0000","last_clean":"2023-12-19T18:17:39.843343+0000","last_became_active":"2023-12-19T18:17:39.841465+0000","last_became_peered":"2023-12-19T18:17:39.841465+0000","last_unstale":"2023-12-19T18:17:39.843343+0000","last_undegraded":"2023-12-19T18:17:39.843343+0000","last_fullsized":"2023-12-19T18:17:39.843343+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:04:11.535860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"236'3","reported_seq":13,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.840966+0000","last_change":"2023-12-19T18:17:39.417418+0000","last_active":"2023-12-19T18:17:39.840966+0000","last_peered":"2023-12-19T18:17:39.840966+0000","last_clean":"2023-12-19T18:17:39.840966+0000","last_became_active":"2023-12-19T18:17:39.416743+0000","last_became_peered":"2023-12-19T18:17:39.416743+0000","last_unstale":"2023-12-19T18:17:39.840966+0000","last_undegraded":"2023-12-19T18:17:39.840966+0000","last_fullsized":"2023-12-19T18:17:39.840966+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:10.544888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.410105+0000","last_change":"2023-12-19T18:17:27.356985+0000","last_active":"2023-12-19T18:17:39.410105+0000","last_peered":"2023-12-19T18:17:39.410105+0000","last_clean":"2023-12-19T18:17:39.410105+0000","last_became_active":"2023-12-19T18:17:27.356695+0000","last_became_peered":"2023-12-19T18:17:27.356695+0000","last_unstale":"2023-12-19T18:17:39.410105+0000","last_undegraded":"2023-12-19T18:17:39.410105+0000","last_fullsized":"2023-12-19T18:17:39.410105+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:31.153454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413880+0000","last_change":"2023-12-19T18:17:26.944872+0000","last_active":"2023-12-19T18:17:39.413880+0000","last_peered":"2023-12-19T18:17:39.413880+0000","last_clean":"2023-12-19T18:17:39.413880+0000","last_became_active":"2023-12-19T18:17:26.944558+0000","last_became_peered":"2023-12-19T18:17:26.944558+0000","last_unstale":"2023-12-19T18:17:39.413880+0000","last_undegraded":"2023-12-19T18:17:39.413880+0000","last_fullsized":"2023-12-19T18:17:39.413880+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:30.151215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.813139+0000","last_change":"2023-12-19T18:17:39.833936+0000","last_active":"2023-12-19T18:17:40.813139+0000","last_peered":"2023-12-19T18:17:40.813139+0000","last_clean":"2023-12-19T18:17:40.813139+0000","last_became_active":"2023-12-19T18:17:39.833640+0000","last_became_peered":"2023-12-19T18:17:39.833640+0000","last_unstale":"2023-12-19T18:17:40.813139+0000","last_undegraded":"2023-12-19T18:17:40.813139+0000","last_fullsized":"2023-12-19T18:17:40.813139+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:37.624327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"236'6","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.847428+0000","last_change":"2023-12-19T18:17:39.423807+0000","last_active":"2023-12-19T18:17:39.847428+0000","last_peered":"2023-12-19T18:17:39.847428+0000","last_clean":"2023-12-19T18:17:39.847428+0000","last_became_active":"2023-12-19T18:17:39.423499+0000","last_became_peered":"2023-12-19T18:17:39.423499+0000","last_unstale":"2023-12-19T18:17:39.847428+0000","last_undegraded":"2023-12-19T18:17:39.847428+0000","last_fullsized":"2023-12-19T18:17:39.847428+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:50:37.326872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413790+0000","last_change":"2023-12-19T18:17:27.342915+0000","last_active":"2023-12-19T18:17:39.413790+0000","last_peered":"2023-12-19T18:17:39.413790+0000","last_clean":"2023-12-19T18:17:39.413790+0000","last_became_active":"2023-12-19T18:17:27.342549+0000","last_became_peered":"2023-12-19T18:17:27.342549+0000","last_unstale":"2023-12-19T18:17:39.413790+0000","last_undegraded":"2023-12-19T18:17:39.413790+0000","last_fullsized":"2023-12-19T18:17:39.413790+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:03.786746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830971+0000","last_change":"2023-12-19T18:17:26.943010+0000","last_active":"2023-12-19T18:17:39.830971+0000","last_peered":"2023-12-19T18:17:39.830971+0000","last_clean":"2023-12-19T18:17:39.830971+0000","last_became_active":"2023-12-19T18:17:26.942721+0000","last_became_peered":"2023-12-19T18:17:26.942721+0000","last_unstale":"2023-12-19T18:17:39.830971+0000","last_undegraded":"2023-12-19T18:17:39.830971+0000","last_fullsized":"2023-12-19T18:17:39.830971+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:09:16.907014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.826155+0000","last_change":"2023-12-19T18:17:39.818981+0000","last_active":"2023-12-19T18:17:39.826155+0000","last_peered":"2023-12-19T18:17:39.826155+0000","last_clean":"2023-12-19T18:17:39.826155+0000","last_became_active":"2023-12-19T18:17:39.818715+0000","last_became_peered":"2023-12-19T18:17:39.818715+0000","last_unstale":"2023-12-19T18:17:39.826155+0000","last_undegraded":"2023-12-19T18:17:39.826155+0000","last_fullsized":"2023-12-19T18:17:39.826155+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:45.848291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.813268+0000","last_change":"2023-12-19T18:17:39.418260+0000","last_active":"2023-12-19T18:17:39.813268+0000","last_peered":"2023-12-19T18:17:39.813268+0000","last_clean":"2023-12-19T18:17:39.813268+0000","last_became_active":"2023-12-19T18:17:39.417966+0000","last_became_peered":"2023-12-19T18:17:39.417966+0000","last_unstale":"2023-12-19T18:17:39.813268+0000","last_undegraded":"2023-12-19T18:17:39.813268+0000","last_fullsized":"2023-12-19T18:17:39.813268+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:18.010498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810034+0000","last_change":"2023-12-19T18:17:27.368101+0000","last_active":"2023-12-19T18:17:39.810034+0000","last_peered":"2023-12-19T18:17:39.810034+0000","last_clean":"2023-12-19T18:17:39.810034+0000","last_became_active":"2023-12-19T18:17:27.367822+0000","last_became_peered":"2023-12-19T18:17:27.367822+0000","last_unstale":"2023-12-19T18:17:39.810034+0000","last_undegraded":"2023-12-19T18:17:39.810034+0000","last_fullsized":"2023-12-19T18:17:39.810034+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:11:20.286347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"232'3","reported_seq":17,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413647+0000","last_change":"2023-12-19T18:17:26.938268+0000","last_active":"2023-12-19T18:17:39.413647+0000","last_peered":"2023-12-19T18:17:39.413647+0000","last_clean":"2023-12-19T18:17:39.413647+0000","last_became_active":"2023-12-19T18:17:26.937724+0000","last_became_peered":"2023-12-19T18:17:26.937724+0000","last_unstale":"2023-12-19T18:17:39.413647+0000","last_undegraded":"2023-12-19T18:17:39.413647+0000","last_fullsized":"2023-12-19T18:17:39.413647+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:05.552913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809400+0000","last_change":"2023-12-19T18:17:39.842511+0000","last_active":"2023-12-19T18:17:40.809400+0000","last_peered":"2023-12-19T18:17:40.809400+0000","last_clean":"2023-12-19T18:17:40.809400+0000","last_became_active":"2023-12-19T18:17:39.839597+0000","last_became_peered":"2023-12-19T18:17:39.839597+0000","last_unstale":"2023-12-19T18:17:40.809400+0000","last_undegraded":"2023-12-19T18:17:40.809400+0000","last_fullsized":"2023-12-19T18:17:40.809400+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:17:06.505163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.816182+0000","last_change":"2023-12-19T18:17:39.426212+0000","last_active":"2023-12-19T18:17:39.816182+0000","last_peered":"2023-12-19T18:17:39.816182+0000","last_clean":"2023-12-19T18:17:39.816182+0000","last_became_active":"2023-12-19T18:17:39.425383+0000","last_became_peered":"2023-12-19T18:17:39.425383+0000","last_unstale":"2023-12-19T18:17:39.816182+0000","last_undegraded":"2023-12-19T18:17:39.816182+0000","last_fullsized":"2023-12-19T18:17:39.816182+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:33:57.603913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.415753+0000","last_change":"2023-12-19T18:17:27.344496+0000","last_active":"2023-12-19T18:17:39.415753+0000","last_peered":"2023-12-19T18:17:39.415753+0000","last_clean":"2023-12-19T18:17:39.415753+0000","last_became_active":"2023-12-19T18:17:27.343611+0000","last_became_peered":"2023-12-19T18:17:27.343611+0000","last_unstale":"2023-12-19T18:17:39.415753+0000","last_undegraded":"2023-12-19T18:17:39.415753+0000","last_fullsized":"2023-12-19T18:17:39.415753+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:26:28.887778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"233'8","reported_seq":22,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416379+0000","last_change":"2023-12-19T18:17:26.941650+0000","last_active":"2023-12-19T18:17:39.416379+0000","last_peered":"2023-12-19T18:17:39.416379+0000","last_clean":"2023-12-19T18:17:39.416379+0000","last_became_active":"2023-12-19T18:17:26.941388+0000","last_became_peered":"2023-12-19T18:17:26.941388+0000","last_unstale":"2023-12-19T18:17:39.416379+0000","last_undegraded":"2023-12-19T18:17:39.416379+0000","last_fullsized":"2023-12-19T18:17:39.416379+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:35:14.834857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830961+0000","last_change":"2023-12-19T18:17:39.823276+0000","last_active":"2023-12-19T18:17:39.830961+0000","last_peered":"2023-12-19T18:17:39.830961+0000","last_clean":"2023-12-19T18:17:39.830961+0000","last_became_active":"2023-12-19T18:17:39.823022+0000","last_became_peered":"2023-12-19T18:17:39.823022+0000","last_unstale":"2023-12-19T18:17:39.830961+0000","last_undegraded":"2023-12-19T18:17:39.830961+0000","last_fullsized":"2023-12-19T18:17:39.830961+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:03:47.613528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812535+0000","last_change":"2023-12-19T18:17:39.425705+0000","last_active":"2023-12-19T18:17:40.812535+0000","last_peered":"2023-12-19T18:17:40.812535+0000","last_clean":"2023-12-19T18:17:40.812535+0000","last_became_active":"2023-12-19T18:17:39.425418+0000","last_became_peered":"2023-12-19T18:17:39.425418+0000","last_unstale":"2023-12-19T18:17:40.812535+0000","last_undegraded":"2023-12-19T18:17:40.812535+0000","last_fullsized":"2023-12-19T18:17:40.812535+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:47:36.472195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834202+0000","last_change":"2023-12-19T18:17:27.350307+0000","last_active":"2023-12-19T18:17:39.834202+0000","last_peered":"2023-12-19T18:17:39.834202+0000","last_clean":"2023-12-19T18:17:39.834202+0000","last_became_active":"2023-12-19T18:17:27.350036+0000","last_became_peered":"2023-12-19T18:17:27.350036+0000","last_unstale":"2023-12-19T18:17:39.834202+0000","last_undegraded":"2023-12-19T18:17:39.834202+0000","last_fullsized":"2023-12-19T18:17:39.834202+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:24:50.816850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"232'2","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.410595+0000","last_change":"2023-12-19T18:17:26.941112+0000","last_active":"2023-12-19T18:17:39.410595+0000","last_peered":"2023-12-19T18:17:39.410595+0000","last_clean":"2023-12-19T18:17:39.410595+0000","last_became_active":"2023-12-19T18:17:26.940838+0000","last_became_peered":"2023-12-19T18:17:26.940838+0000","last_unstale":"2023-12-19T18:17:39.410595+0000","last_undegraded":"2023-12-19T18:17:39.410595+0000","last_fullsized":"2023-12-19T18:17:39.410595+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:47:48.785673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812383+0000","last_change":"2023-12-19T18:17:39.832557+0000","last_active":"2023-12-19T18:17:40.812383+0000","last_peered":"2023-12-19T18:17:40.812383+0000","last_clean":"2023-12-19T18:17:40.812383+0000","last_became_active":"2023-12-19T18:17:39.832288+0000","last_became_peered":"2023-12-19T18:17:39.832288+0000","last_unstale":"2023-12-19T18:17:40.812383+0000","last_undegraded":"2023-12-19T18:17:40.812383+0000","last_fullsized":"2023-12-19T18:17:40.812383+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:13:14.873886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"236'8","reported_seq":19,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.808845+0000","last_change":"2023-12-19T18:17:39.415764+0000","last_active":"2023-12-19T18:17:40.808845+0000","last_peered":"2023-12-19T18:17:40.808845+0000","last_clean":"2023-12-19T18:17:40.808845+0000","last_became_active":"2023-12-19T18:17:39.414955+0000","last_became_peered":"2023-12-19T18:17:39.414955+0000","last_unstale":"2023-12-19T18:17:40.808845+0000","last_undegraded":"2023-12-19T18:17:40.808845+0000","last_fullsized":"2023-12-19T18:17:40.808845+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:08.661657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820226+0000","last_change":"2023-12-19T18:17:27.362157+0000","last_active":"2023-12-19T18:17:39.820226+0000","last_peered":"2023-12-19T18:17:39.820226+0000","last_clean":"2023-12-19T18:17:39.820226+0000","last_became_active":"2023-12-19T18:17:27.361879+0000","last_became_peered":"2023-12-19T18:17:27.361879+0000","last_unstale":"2023-12-19T18:17:39.820226+0000","last_undegraded":"2023-12-19T18:17:39.820226+0000","last_fullsized":"2023-12-19T18:17:39.820226+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:00.178479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834282+0000","last_change":"2023-12-19T18:17:26.948613+0000","last_active":"2023-12-19T18:17:39.834282+0000","last_peered":"2023-12-19T18:17:39.834282+0000","last_clean":"2023-12-19T18:17:39.834282+0000","last_became_active":"2023-12-19T18:17:26.948318+0000","last_became_peered":"2023-12-19T18:17:26.948318+0000","last_unstale":"2023-12-19T18:17:39.834282+0000","last_undegraded":"2023-12-19T18:17:39.834282+0000","last_fullsized":"2023-12-19T18:17:39.834282+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:13:19.247070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812745+0000","last_change":"2023-12-19T18:17:39.821325+0000","last_active":"2023-12-19T18:17:40.812745+0000","last_peered":"2023-12-19T18:17:40.812745+0000","last_clean":"2023-12-19T18:17:40.812745+0000","last_became_active":"2023-12-19T18:17:39.821042+0000","last_became_peered":"2023-12-19T18:17:39.821042+0000","last_unstale":"2023-12-19T18:17:40.812745+0000","last_undegraded":"2023-12-19T18:17:40.812745+0000","last_fullsized":"2023-12-19T18:17:40.812745+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:07:21.803515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"236'2","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.845920+0000","last_change":"2023-12-19T18:17:39.415614+0000","last_active":"2023-12-19T18:17:39.845920+0000","last_peered":"2023-12-19T18:17:39.845920+0000","last_clean":"2023-12-19T18:17:39.845920+0000","last_became_active":"2023-12-19T18:17:39.414534+0000","last_became_peered":"2023-12-19T18:17:39.414534+0000","last_unstale":"2023-12-19T18:17:39.845920+0000","last_undegraded":"2023-12-19T18:17:39.845920+0000","last_fullsized":"2023-12-19T18:17:39.845920+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:25:07.972942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.819746+0000","last_change":"2023-12-19T18:17:27.361736+0000","last_active":"2023-12-19T18:17:39.819746+0000","last_peered":"2023-12-19T18:17:39.819746+0000","last_clean":"2023-12-19T18:17:39.819746+0000","last_became_active":"2023-12-19T18:17:27.361355+0000","last_became_peered":"2023-12-19T18:17:27.361355+0000","last_unstale":"2023-12-19T18:17:39.819746+0000","last_undegraded":"2023-12-19T18:17:39.819746+0000","last_fullsized":"2023-12-19T18:17:39.819746+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:02.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413769+0000","last_change":"2023-12-19T18:17:26.944351+0000","last_active":"2023-12-19T18:17:39.413769+0000","last_peered":"2023-12-19T18:17:39.413769+0000","last_clean":"2023-12-19T18:17:39.413769+0000","last_became_active":"2023-12-19T18:17:26.943692+0000","last_became_peered":"2023-12-19T18:17:26.943692+0000","last_unstale":"2023-12-19T18:17:39.413769+0000","last_undegraded":"2023-12-19T18:17:39.413769+0000","last_fullsized":"2023-12-19T18:17:39.413769+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:54.040380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.835209+0000","last_change":"2023-12-19T18:17:39.832355+0000","last_active":"2023-12-19T18:17:39.835209+0000","last_peered":"2023-12-19T18:17:39.835209+0000","last_clean":"2023-12-19T18:17:39.835209+0000","last_became_active":"2023-12-19T18:17:39.832058+0000","last_became_peered":"2023-12-19T18:17:39.832058+0000","last_unstale":"2023-12-19T18:17:39.835209+0000","last_undegraded":"2023-12-19T18:17:39.835209+0000","last_fullsized":"2023-12-19T18:17:39.835209+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:21:07.719679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.814321+0000","last_change":"2023-12-19T18:17:39.420031+0000","last_active":"2023-12-19T18:17:39.814321+0000","last_peered":"2023-12-19T18:17:39.814321+0000","last_clean":"2023-12-19T18:17:39.814321+0000","last_became_active":"2023-12-19T18:17:39.419757+0000","last_became_peered":"2023-12-19T18:17:39.419757+0000","last_unstale":"2023-12-19T18:17:39.814321+0000","last_undegraded":"2023-12-19T18:17:39.814321+0000","last_fullsized":"2023-12-19T18:17:39.814321+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:45.886255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.811442+0000","last_change":"2023-12-19T18:17:27.366044+0000","last_active":"2023-12-19T18:17:39.811442+0000","last_peered":"2023-12-19T18:17:39.811442+0000","last_clean":"2023-12-19T18:17:39.811442+0000","last_became_active":"2023-12-19T18:17:27.365761+0000","last_became_peered":"2023-12-19T18:17:27.365761+0000","last_unstale":"2023-12-19T18:17:39.811442+0000","last_undegraded":"2023-12-19T18:17:39.811442+0000","last_fullsized":"2023-12-19T18:17:39.811442+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:05:07.652398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"232'8","reported_seq":22,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408317+0000","last_change":"2023-12-19T18:17:26.948258+0000","last_active":"2023-12-19T18:17:39.408317+0000","last_peered":"2023-12-19T18:17:39.408317+0000","last_clean":"2023-12-19T18:17:39.408317+0000","last_became_active":"2023-12-19T18:17:26.947974+0000","last_became_peered":"2023-12-19T18:17:26.947974+0000","last_unstale":"2023-12-19T18:17:39.408317+0000","last_undegraded":"2023-12-19T18:17:39.408317+0000","last_fullsized":"2023-12-19T18:17:39.408317+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:56:49.244280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.835313+0000","last_change":"2023-12-19T18:17:39.832119+0000","last_active":"2023-12-19T18:17:39.835313+0000","last_peered":"2023-12-19T18:17:39.835313+0000","last_clean":"2023-12-19T18:17:39.835313+0000","last_became_active":"2023-12-19T18:17:39.831851+0000","last_became_peered":"2023-12-19T18:17:39.831851+0000","last_unstale":"2023-12-19T18:17:39.835313+0000","last_undegraded":"2023-12-19T18:17:39.835313+0000","last_fullsized":"2023-12-19T18:17:39.835313+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:44.450250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"236'1","reported_seq":11,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834104+0000","last_change":"2023-12-19T18:17:39.425937+0000","last_active":"2023-12-19T18:17:39.834104+0000","last_peered":"2023-12-19T18:17:39.834104+0000","last_clean":"2023-12-19T18:17:39.834104+0000","last_became_active":"2023-12-19T18:17:39.423851+0000","last_became_peered":"2023-12-19T18:17:39.423851+0000","last_unstale":"2023-12-19T18:17:39.834104+0000","last_undegraded":"2023-12-19T18:17:39.834104+0000","last_fullsized":"2023-12-19T18:17:39.834104+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:14:02.076472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.412219+0000","last_change":"2023-12-19T18:17:27.349954+0000","last_active":"2023-12-19T18:17:39.412219+0000","last_peered":"2023-12-19T18:17:39.412219+0000","last_clean":"2023-12-19T18:17:39.412219+0000","last_became_active":"2023-12-19T18:17:27.349694+0000","last_became_peered":"2023-12-19T18:17:27.349694+0000","last_unstale":"2023-12-19T18:17:39.412219+0000","last_undegraded":"2023-12-19T18:17:39.412219+0000","last_fullsized":"2023-12-19T18:17:39.412219+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:36:07.296035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"232'1","reported_seq":16,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.822729+0000","last_change":"2023-12-19T18:17:26.944980+0000","last_active":"2023-12-19T18:17:39.822729+0000","last_peered":"2023-12-19T18:17:39.822729+0000","last_clean":"2023-12-19T18:17:39.822729+0000","last_became_active":"2023-12-19T18:17:26.943863+0000","last_became_peered":"2023-12-19T18:17:26.943863+0000","last_unstale":"2023-12-19T18:17:39.822729+0000","last_undegraded":"2023-12-19T18:17:39.822729+0000","last_fullsized":"2023-12-19T18:17:39.822729+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:38:38.154851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.841408+0000","last_change":"2023-12-19T18:17:39.840268+0000","last_active":"2023-12-19T18:17:39.841408+0000","last_peered":"2023-12-19T18:17:39.841408+0000","last_clean":"2023-12-19T18:17:39.841408+0000","last_became_active":"2023-12-19T18:17:39.839905+0000","last_became_peered":"2023-12-19T18:17:39.839905+0000","last_unstale":"2023-12-19T18:17:39.841408+0000","last_undegraded":"2023-12-19T18:17:39.841408+0000","last_fullsized":"2023-12-19T18:17:39.841408+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:10:59.997438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809204+0000","last_change":"2023-12-19T18:17:39.420566+0000","last_active":"2023-12-19T18:17:40.809204+0000","last_peered":"2023-12-19T18:17:40.809204+0000","last_clean":"2023-12-19T18:17:40.809204+0000","last_became_active":"2023-12-19T18:17:39.420248+0000","last_became_peered":"2023-12-19T18:17:39.420248+0000","last_unstale":"2023-12-19T18:17:40.809204+0000","last_undegraded":"2023-12-19T18:17:40.809204+0000","last_fullsized":"2023-12-19T18:17:40.809204+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:11.094591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.417573+0000","last_change":"2023-12-19T18:17:27.364634+0000","last_active":"2023-12-19T18:17:39.417573+0000","last_peered":"2023-12-19T18:17:39.417573+0000","last_clean":"2023-12-19T18:17:39.417573+0000","last_became_active":"2023-12-19T18:17:27.364357+0000","last_became_peered":"2023-12-19T18:17:27.364357+0000","last_unstale":"2023-12-19T18:17:39.417573+0000","last_undegraded":"2023-12-19T18:17:39.417573+0000","last_fullsized":"2023-12-19T18:17:39.417573+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:05:35.230888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"232'2","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.831082+0000","last_change":"2023-12-19T18:17:26.938178+0000","last_active":"2023-12-19T18:17:39.831082+0000","last_peered":"2023-12-19T18:17:39.831082+0000","last_clean":"2023-12-19T18:17:39.831082+0000","last_became_active":"2023-12-19T18:17:26.937910+0000","last_became_peered":"2023-12-19T18:17:26.937910+0000","last_unstale":"2023-12-19T18:17:39.831082+0000","last_undegraded":"2023-12-19T18:17:39.831082+0000","last_fullsized":"2023-12-19T18:17:39.831082+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:19:13.215706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.839653+0000","last_change":"2023-12-19T18:17:39.833695+0000","last_active":"2023-12-19T18:17:39.839653+0000","last_peered":"2023-12-19T18:17:39.839653+0000","last_clean":"2023-12-19T18:17:39.839653+0000","last_became_active":"2023-12-19T18:17:39.833458+0000","last_became_peered":"2023-12-19T18:17:39.833458+0000","last_unstale":"2023-12-19T18:17:39.839653+0000","last_undegraded":"2023-12-19T18:17:39.839653+0000","last_fullsized":"2023-12-19T18:17:39.839653+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:27:16.215134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"236'1","reported_seq":12,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.813485+0000","last_change":"2023-12-19T18:17:39.420721+0000","last_active":"2023-12-19T18:17:40.813485+0000","last_peered":"2023-12-19T18:17:40.813485+0000","last_clean":"2023-12-19T18:17:40.813485+0000","last_became_active":"2023-12-19T18:17:39.420084+0000","last_became_peered":"2023-12-19T18:17:39.420084+0000","last_unstale":"2023-12-19T18:17:40.813485+0000","last_undegraded":"2023-12-19T18:17:40.813485+0000","last_fullsized":"2023-12-19T18:17:40.813485+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:35:27.292441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820686+0000","last_change":"2023-12-19T18:17:27.360665+0000","last_active":"2023-12-19T18:17:39.820686+0000","last_peered":"2023-12-19T18:17:39.820686+0000","last_clean":"2023-12-19T18:17:39.820686+0000","last_became_active":"2023-12-19T18:17:27.360397+0000","last_became_peered":"2023-12-19T18:17:27.360397+0000","last_unstale":"2023-12-19T18:17:39.820686+0000","last_undegraded":"2023-12-19T18:17:39.820686+0000","last_fullsized":"2023-12-19T18:17:39.820686+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:33.393492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"232'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.415927+0000","last_change":"2023-12-19T18:17:26.946790+0000","last_active":"2023-12-19T18:17:39.415927+0000","last_peered":"2023-12-19T18:17:39.415927+0000","last_clean":"2023-12-19T18:17:39.415927+0000","last_became_active":"2023-12-19T18:17:26.946487+0000","last_became_peered":"2023-12-19T18:17:26.946487+0000","last_unstale":"2023-12-19T18:17:39.415927+0000","last_undegraded":"2023-12-19T18:17:39.415927+0000","last_fullsized":"2023-12-19T18:17:39.415927+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:49.419114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.829656+0000","last_change":"2023-12-19T18:17:39.823685+0000","last_active":"2023-12-19T18:17:39.829656+0000","last_peered":"2023-12-19T18:17:39.829656+0000","last_clean":"2023-12-19T18:17:39.829656+0000","last_became_active":"2023-12-19T18:17:39.822618+0000","last_became_peered":"2023-12-19T18:17:39.822618+0000","last_unstale":"2023-12-19T18:17:39.829656+0000","last_undegraded":"2023-12-19T18:17:39.829656+0000","last_fullsized":"2023-12-19T18:17:39.829656+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:20:14.678399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"236'2","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834119+0000","last_change":"2023-12-19T18:17:39.422146+0000","last_active":"2023-12-19T18:17:39.834119+0000","last_peered":"2023-12-19T18:17:39.834119+0000","last_clean":"2023-12-19T18:17:39.834119+0000","last_became_active":"2023-12-19T18:17:39.421744+0000","last_became_peered":"2023-12-19T18:17:39.421744+0000","last_unstale":"2023-12-19T18:17:39.834119+0000","last_undegraded":"2023-12-19T18:17:39.834119+0000","last_fullsized":"2023-12-19T18:17:39.834119+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:41:19.519584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820550+0000","last_change":"2023-12-19T18:17:27.365652+0000","last_active":"2023-12-19T18:17:39.820550+0000","last_peered":"2023-12-19T18:17:39.820550+0000","last_clean":"2023-12-19T18:17:39.820550+0000","last_became_active":"2023-12-19T18:17:27.365336+0000","last_became_peered":"2023-12-19T18:17:27.365336+0000","last_unstale":"2023-12-19T18:17:39.820550+0000","last_undegraded":"2023-12-19T18:17:39.820550+0000","last_fullsized":"2023-12-19T18:17:39.820550+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:54.566933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416460+0000","last_change":"2023-12-19T18:17:26.942091+0000","last_active":"2023-12-19T18:17:39.416460+0000","last_peered":"2023-12-19T18:17:39.416460+0000","last_clean":"2023-12-19T18:17:39.416460+0000","last_became_active":"2023-12-19T18:17:26.941806+0000","last_became_peered":"2023-12-19T18:17:26.941806+0000","last_unstale":"2023-12-19T18:17:39.416460+0000","last_undegraded":"2023-12-19T18:17:39.416460+0000","last_fullsized":"2023-12-19T18:17:39.416460+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:27:28.247767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.808651+0000","last_change":"2023-12-19T18:17:39.839169+0000","last_active":"2023-12-19T18:17:40.808651+0000","last_peered":"2023-12-19T18:17:40.808651+0000","last_clean":"2023-12-19T18:17:40.808651+0000","last_became_active":"2023-12-19T18:17:39.838252+0000","last_became_peered":"2023-12-19T18:17:39.838252+0000","last_unstale":"2023-12-19T18:17:40.808651+0000","last_undegraded":"2023-12-19T18:17:40.808651+0000","last_fullsized":"2023-12-19T18:17:40.808651+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:46:35.106086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.821679+0000","last_change":"2023-12-19T18:17:39.426473+0000","last_active":"2023-12-19T18:17:39.821679+0000","last_peered":"2023-12-19T18:17:39.821679+0000","last_clean":"2023-12-19T18:17:39.821679+0000","last_became_active":"2023-12-19T18:17:39.426209+0000","last_became_peered":"2023-12-19T18:17:39.426209+0000","last_unstale":"2023-12-19T18:17:39.821679+0000","last_undegraded":"2023-12-19T18:17:39.821679+0000","last_fullsized":"2023-12-19T18:17:39.821679+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:40.104645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.412727+0000","last_change":"2023-12-19T18:17:27.367095+0000","last_active":"2023-12-19T18:17:39.412727+0000","last_peered":"2023-12-19T18:17:39.412727+0000","last_clean":"2023-12-19T18:17:39.412727+0000","last_became_active":"2023-12-19T18:17:27.366441+0000","last_became_peered":"2023-12-19T18:17:27.366441+0000","last_unstale":"2023-12-19T18:17:39.412727+0000","last_undegraded":"2023-12-19T18:17:39.412727+0000","last_fullsized":"2023-12-19T18:17:39.412727+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:10:47.532188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"232'2","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820898+0000","last_change":"2023-12-19T18:17:26.948576+0000","last_active":"2023-12-19T18:17:39.820898+0000","last_peered":"2023-12-19T18:17:39.820898+0000","last_clean":"2023-12-19T18:17:39.820898+0000","last_became_active":"2023-12-19T18:17:26.948310+0000","last_became_peered":"2023-12-19T18:17:26.948310+0000","last_unstale":"2023-12-19T18:17:39.820898+0000","last_undegraded":"2023-12-19T18:17:39.820898+0000","last_fullsized":"2023-12-19T18:17:39.820898+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:07:58.733231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830331+0000","last_change":"2023-12-19T18:17:39.826366+0000","last_active":"2023-12-19T18:17:39.830331+0000","last_peered":"2023-12-19T18:17:39.830331+0000","last_clean":"2023-12-19T18:17:39.830331+0000","last_became_active":"2023-12-19T18:17:39.826114+0000","last_became_peered":"2023-12-19T18:17:39.826114+0000","last_unstale":"2023-12-19T18:17:39.830331+0000","last_undegraded":"2023-12-19T18:17:39.830331+0000","last_fullsized":"2023-12-19T18:17:39.830331+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:22:44.071731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.811944+0000","last_change":"2023-12-19T18:17:39.424437+0000","last_active":"2023-12-19T18:17:40.811944+0000","last_peered":"2023-12-19T18:17:40.811944+0000","last_clean":"2023-12-19T18:17:40.811944+0000","last_became_active":"2023-12-19T18:17:39.422298+0000","last_became_peered":"2023-12-19T18:17:39.422298+0000","last_unstale":"2023-12-19T18:17:40.811944+0000","last_undegraded":"2023-12-19T18:17:40.811944+0000","last_fullsized":"2023-12-19T18:17:40.811944+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:20:55.327417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820972+0000","last_change":"2023-12-19T18:17:27.358552+0000","last_active":"2023-12-19T18:17:39.820972+0000","last_peered":"2023-12-19T18:17:39.820972+0000","last_clean":"2023-12-19T18:17:39.820972+0000","last_became_active":"2023-12-19T18:17:27.358174+0000","last_became_peered":"2023-12-19T18:17:27.358174+0000","last_unstale":"2023-12-19T18:17:39.820972+0000","last_undegraded":"2023-12-19T18:17:39.820972+0000","last_fullsized":"2023-12-19T18:17:39.820972+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:14.654686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810586+0000","last_change":"2023-12-19T18:17:26.947007+0000","last_active":"2023-12-19T18:17:39.810586+0000","last_peered":"2023-12-19T18:17:39.810586+0000","last_clean":"2023-12-19T18:17:39.810586+0000","last_became_active":"2023-12-19T18:17:26.946741+0000","last_became_peered":"2023-12-19T18:17:26.946741+0000","last_unstale":"2023-12-19T18:17:39.810586+0000","last_undegraded":"2023-12-19T18:17:39.810586+0000","last_fullsized":"2023-12-19T18:17:39.810586+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:20.105057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.826429+0000","last_change":"2023-12-19T18:17:39.822781+0000","last_active":"2023-12-19T18:17:39.826429+0000","last_peered":"2023-12-19T18:17:39.826429+0000","last_clean":"2023-12-19T18:17:39.826429+0000","last_became_active":"2023-12-19T18:17:39.822463+0000","last_became_peered":"2023-12-19T18:17:39.822463+0000","last_unstale":"2023-12-19T18:17:39.826429+0000","last_undegraded":"2023-12-19T18:17:39.826429+0000","last_fullsized":"2023-12-19T18:17:39.826429+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:41.114550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"236'2","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.831810+0000","last_change":"2023-12-19T18:17:39.425769+0000","last_active":"2023-12-19T18:17:39.831810+0000","last_peered":"2023-12-19T18:17:39.831810+0000","last_clean":"2023-12-19T18:17:39.831810+0000","last_became_active":"2023-12-19T18:17:39.425446+0000","last_became_peered":"2023-12-19T18:17:39.425446+0000","last_unstale":"2023-12-19T18:17:39.831810+0000","last_undegraded":"2023-12-19T18:17:39.831810+0000","last_fullsized":"2023-12-19T18:17:39.831810+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:26.262151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.410034+0000","last_change":"2023-12-19T18:17:27.348563+0000","last_active":"2023-12-19T18:17:39.410034+0000","last_peered":"2023-12-19T18:17:39.410034+0000","last_clean":"2023-12-19T18:17:39.410034+0000","last_became_active":"2023-12-19T18:17:27.348275+0000","last_became_peered":"2023-12-19T18:17:27.348275+0000","last_unstale":"2023-12-19T18:17:39.410034+0000","last_undegraded":"2023-12-19T18:17:39.410034+0000","last_fullsized":"2023-12-19T18:17:39.410034+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:45:15.369206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.811163+0000","last_change":"2023-12-19T18:17:26.944588+0000","last_active":"2023-12-19T18:17:39.811163+0000","last_peered":"2023-12-19T18:17:39.811163+0000","last_clean":"2023-12-19T18:17:39.811163+0000","last_became_active":"2023-12-19T18:17:26.944323+0000","last_became_peered":"2023-12-19T18:17:26.944323+0000","last_unstale":"2023-12-19T18:17:39.811163+0000","last_undegraded":"2023-12-19T18:17:39.811163+0000","last_fullsized":"2023-12-19T18:17:39.811163+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:25:45.208073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.833736+0000","last_change":"2023-12-19T18:17:39.829879+0000","last_active":"2023-12-19T18:17:39.833736+0000","last_peered":"2023-12-19T18:17:39.833736+0000","last_clean":"2023-12-19T18:17:39.833736+0000","last_became_active":"2023-12-19T18:17:39.829196+0000","last_became_peered":"2023-12-19T18:17:39.829196+0000","last_unstale":"2023-12-19T18:17:39.833736+0000","last_undegraded":"2023-12-19T18:17:39.833736+0000","last_fullsized":"2023-12-19T18:17:39.833736+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:57.527231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.808828+0000","last_change":"2023-12-19T18:17:39.420394+0000","last_active":"2023-12-19T18:17:40.808828+0000","last_peered":"2023-12-19T18:17:40.808828+0000","last_clean":"2023-12-19T18:17:40.808828+0000","last_became_active":"2023-12-19T18:17:39.420074+0000","last_became_peered":"2023-12-19T18:17:39.420074+0000","last_unstale":"2023-12-19T18:17:40.808828+0000","last_undegraded":"2023-12-19T18:17:40.808828+0000","last_fullsized":"2023-12-19T18:17:40.808828+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:40:28.357211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413682+0000","last_change":"2023-12-19T18:17:27.342936+0000","last_active":"2023-12-19T18:17:39.413682+0000","last_peered":"2023-12-19T18:17:39.413682+0000","last_clean":"2023-12-19T18:17:39.413682+0000","last_became_active":"2023-12-19T18:17:27.342636+0000","last_became_peered":"2023-12-19T18:17:27.342636+0000","last_unstale":"2023-12-19T18:17:39.413682+0000","last_undegraded":"2023-12-19T18:17:39.413682+0000","last_fullsized":"2023-12-19T18:17:39.413682+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:43.945499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.819918+0000","last_change":"2023-12-19T18:17:26.950102+0000","last_active":"2023-12-19T18:17:39.819918+0000","last_peered":"2023-12-19T18:17:39.819918+0000","last_clean":"2023-12-19T18:17:39.819918+0000","last_became_active":"2023-12-19T18:17:26.949810+0000","last_became_peered":"2023-12-19T18:17:26.949810+0000","last_unstale":"2023-12-19T18:17:39.819918+0000","last_undegraded":"2023-12-19T18:17:39.819918+0000","last_fullsized":"2023-12-19T18:17:39.819918+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:04:02.387088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809122+0000","last_change":"2023-12-19T18:17:39.838380+0000","last_active":"2023-12-19T18:17:40.809122+0000","last_peered":"2023-12-19T18:17:40.809122+0000","last_clean":"2023-12-19T18:17:40.809122+0000","last_became_active":"2023-12-19T18:17:39.837519+0000","last_became_peered":"2023-12-19T18:17:39.837519+0000","last_unstale":"2023-12-19T18:17:40.809122+0000","last_undegraded":"2023-12-19T18:17:40.809122+0000","last_fullsized":"2023-12-19T18:17:40.809122+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:53:27.350034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.825910+0000","last_change":"2023-12-19T18:17:39.423805+0000","last_active":"2023-12-19T18:17:39.825910+0000","last_peered":"2023-12-19T18:17:39.825910+0000","last_clean":"2023-12-19T18:17:39.825910+0000","last_became_active":"2023-12-19T18:17:39.423461+0000","last_became_peered":"2023-12-19T18:17:39.423461+0000","last_unstale":"2023-12-19T18:17:39.825910+0000","last_undegraded":"2023-12-19T18:17:39.825910+0000","last_fullsized":"2023-12-19T18:17:39.825910+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:31.423816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.412786+0000","last_change":"2023-12-19T18:17:27.350933+0000","last_active":"2023-12-19T18:17:39.412786+0000","last_peered":"2023-12-19T18:17:39.412786+0000","last_clean":"2023-12-19T18:17:39.412786+0000","last_became_active":"2023-12-19T18:17:27.350661+0000","last_became_peered":"2023-12-19T18:17:27.350661+0000","last_unstale":"2023-12-19T18:17:39.412786+0000","last_undegraded":"2023-12-19T18:17:39.412786+0000","last_fullsized":"2023-12-19T18:17:39.412786+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:18:25.159077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"232'2","reported_seq":16,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413198+0000","last_change":"2023-12-19T18:17:26.944158+0000","last_active":"2023-12-19T18:17:39.413198+0000","last_peered":"2023-12-19T18:17:39.413198+0000","last_clean":"2023-12-19T18:17:39.413198+0000","last_became_active":"2023-12-19T18:17:26.943845+0000","last_became_peered":"2023-12-19T18:17:26.943845+0000","last_unstale":"2023-12-19T18:17:39.413198+0000","last_undegraded":"2023-12-19T18:17:39.413198+0000","last_fullsized":"2023-12-19T18:17:39.413198+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:10.097246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809631+0000","last_change":"2023-12-19T18:17:39.844260+0000","last_active":"2023-12-19T18:17:40.809631+0000","last_peered":"2023-12-19T18:17:40.809631+0000","last_clean":"2023-12-19T18:17:40.809631+0000","last_became_active":"2023-12-19T18:17:39.843498+0000","last_became_peered":"2023-12-19T18:17:39.843498+0000","last_unstale":"2023-12-19T18:17:40.809631+0000","last_undegraded":"2023-12-19T18:17:40.809631+0000","last_fullsized":"2023-12-19T18:17:40.809631+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:45:48.663670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.809286+0000","last_change":"2023-12-19T18:17:39.410366+0000","last_active":"2023-12-19T18:17:40.809286+0000","last_peered":"2023-12-19T18:17:40.809286+0000","last_clean":"2023-12-19T18:17:40.809286+0000","last_became_active":"2023-12-19T18:17:39.409876+0000","last_became_peered":"2023-12-19T18:17:39.409876+0000","last_unstale":"2023-12-19T18:17:40.809286+0000","last_undegraded":"2023-12-19T18:17:40.809286+0000","last_fullsized":"2023-12-19T18:17:40.809286+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:52:28.705197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.822646+0000","last_change":"2023-12-19T18:17:27.361893+0000","last_active":"2023-12-19T18:17:39.822646+0000","last_peered":"2023-12-19T18:17:39.822646+0000","last_clean":"2023-12-19T18:17:39.822646+0000","last_became_active":"2023-12-19T18:17:27.361618+0000","last_became_peered":"2023-12-19T18:17:27.361618+0000","last_unstale":"2023-12-19T18:17:39.822646+0000","last_undegraded":"2023-12-19T18:17:39.822646+0000","last_fullsized":"2023-12-19T18:17:39.822646+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:50.421353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"233'1","reported_seq":15,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.417658+0000","last_change":"2023-12-19T18:17:26.944398+0000","last_active":"2023-12-19T18:17:39.417658+0000","last_peered":"2023-12-19T18:17:39.417658+0000","last_clean":"2023-12-19T18:17:39.417658+0000","last_became_active":"2023-12-19T18:17:26.942848+0000","last_became_peered":"2023-12-19T18:17:26.942848+0000","last_unstale":"2023-12-19T18:17:39.417658+0000","last_undegraded":"2023-12-19T18:17:39.417658+0000","last_fullsized":"2023-12-19T18:17:39.417658+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:01.415653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.813706+0000","last_change":"2023-12-19T18:17:39.823115+0000","last_active":"2023-12-19T18:17:40.813706+0000","last_peered":"2023-12-19T18:17:40.813706+0000","last_clean":"2023-12-19T18:17:40.813706+0000","last_became_active":"2023-12-19T18:17:39.820305+0000","last_became_peered":"2023-12-19T18:17:39.820305+0000","last_unstale":"2023-12-19T18:17:40.813706+0000","last_undegraded":"2023-12-19T18:17:40.813706+0000","last_fullsized":"2023-12-19T18:17:40.813706+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:05:47.696138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"236'2","reported_seq":12,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.849209+0000","last_change":"2023-12-19T18:17:39.415322+0000","last_active":"2023-12-19T18:17:39.849209+0000","last_peered":"2023-12-19T18:17:39.849209+0000","last_clean":"2023-12-19T18:17:39.849209+0000","last_became_active":"2023-12-19T18:17:39.415021+0000","last_became_peered":"2023-12-19T18:17:39.415021+0000","last_unstale":"2023-12-19T18:17:39.849209+0000","last_undegraded":"2023-12-19T18:17:39.849209+0000","last_fullsized":"2023-12-19T18:17:39.849209+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:04:14.844672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820780+0000","last_change":"2023-12-19T18:17:27.359609+0000","last_active":"2023-12-19T18:17:39.820780+0000","last_peered":"2023-12-19T18:17:39.820780+0000","last_clean":"2023-12-19T18:17:39.820780+0000","last_became_active":"2023-12-19T18:17:27.358162+0000","last_became_peered":"2023-12-19T18:17:27.358162+0000","last_unstale":"2023-12-19T18:17:39.820780+0000","last_undegraded":"2023-12-19T18:17:39.820780+0000","last_fullsized":"2023-12-19T18:17:39.820780+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:31:06.407037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416004+0000","last_change":"2023-12-19T18:17:26.941195+0000","last_active":"2023-12-19T18:17:39.416004+0000","last_peered":"2023-12-19T18:17:39.416004+0000","last_clean":"2023-12-19T18:17:39.416004+0000","last_became_active":"2023-12-19T18:17:26.940845+0000","last_became_peered":"2023-12-19T18:17:26.940845+0000","last_unstale":"2023-12-19T18:17:39.416004+0000","last_undegraded":"2023-12-19T18:17:39.416004+0000","last_fullsized":"2023-12-19T18:17:39.416004+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:22.276149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.844143+0000","last_change":"2023-12-19T18:17:39.843161+0000","last_active":"2023-12-19T18:17:39.844143+0000","last_peered":"2023-12-19T18:17:39.844143+0000","last_clean":"2023-12-19T18:17:39.844143+0000","last_became_active":"2023-12-19T18:17:39.842884+0000","last_became_peered":"2023-12-19T18:17:39.842884+0000","last_unstale":"2023-12-19T18:17:39.844143+0000","last_undegraded":"2023-12-19T18:17:39.844143+0000","last_fullsized":"2023-12-19T18:17:39.844143+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:05:25.018602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812281+0000","last_change":"2023-12-19T18:17:39.419763+0000","last_active":"2023-12-19T18:17:40.812281+0000","last_peered":"2023-12-19T18:17:40.812281+0000","last_clean":"2023-12-19T18:17:40.812281+0000","last_became_active":"2023-12-19T18:17:39.419484+0000","last_became_peered":"2023-12-19T18:17:39.419484+0000","last_unstale":"2023-12-19T18:17:40.812281+0000","last_undegraded":"2023-12-19T18:17:40.812281+0000","last_fullsized":"2023-12-19T18:17:40.812281+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:33:38.145177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.414098+0000","last_change":"2023-12-19T18:17:27.351030+0000","last_active":"2023-12-19T18:17:39.414098+0000","last_peered":"2023-12-19T18:17:39.414098+0000","last_clean":"2023-12-19T18:17:39.414098+0000","last_became_active":"2023-12-19T18:17:27.350692+0000","last_became_peered":"2023-12-19T18:17:27.350692+0000","last_unstale":"2023-12-19T18:17:39.414098+0000","last_undegraded":"2023-12-19T18:17:39.414098+0000","last_fullsized":"2023-12-19T18:17:39.414098+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:15:54.445257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820875+0000","last_change":"2023-12-19T18:17:26.943936+0000","last_active":"2023-12-19T18:17:39.820875+0000","last_peered":"2023-12-19T18:17:39.820875+0000","last_clean":"2023-12-19T18:17:39.820875+0000","last_became_active":"2023-12-19T18:17:26.943609+0000","last_became_peered":"2023-12-19T18:17:26.943609+0000","last_unstale":"2023-12-19T18:17:39.820875+0000","last_undegraded":"2023-12-19T18:17:39.820875+0000","last_fullsized":"2023-12-19T18:17:39.820875+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:12.547671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.831297+0000","last_change":"2023-12-19T18:17:39.828877+0000","last_active":"2023-12-19T18:17:39.831297+0000","last_peered":"2023-12-19T18:17:39.831297+0000","last_clean":"2023-12-19T18:17:39.831297+0000","last_became_active":"2023-12-19T18:17:39.825032+0000","last_became_peered":"2023-12-19T18:17:39.825032+0000","last_unstale":"2023-12-19T18:17:39.831297+0000","last_undegraded":"2023-12-19T18:17:39.831297+0000","last_fullsized":"2023-12-19T18:17:39.831297+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:16:06.910507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.813466+0000","last_change":"2023-12-19T18:17:39.423064+0000","last_active":"2023-12-19T18:17:39.813466+0000","last_peered":"2023-12-19T18:17:39.813466+0000","last_clean":"2023-12-19T18:17:39.813466+0000","last_became_active":"2023-12-19T18:17:39.422717+0000","last_became_peered":"2023-12-19T18:17:39.422717+0000","last_unstale":"2023-12-19T18:17:39.813466+0000","last_undegraded":"2023-12-19T18:17:39.813466+0000","last_fullsized":"2023-12-19T18:17:39.813466+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:38:22.115594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.408249+0000","last_change":"2023-12-19T18:17:27.342408+0000","last_active":"2023-12-19T18:17:39.408249+0000","last_peered":"2023-12-19T18:17:39.408249+0000","last_clean":"2023-12-19T18:17:39.408249+0000","last_became_active":"2023-12-19T18:17:27.342114+0000","last_became_peered":"2023-12-19T18:17:27.342114+0000","last_unstale":"2023-12-19T18:17:39.408249+0000","last_undegraded":"2023-12-19T18:17:39.408249+0000","last_fullsized":"2023-12-19T18:17:39.408249+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:02.417893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810372+0000","last_change":"2023-12-19T18:17:26.945192+0000","last_active":"2023-12-19T18:17:39.810372+0000","last_peered":"2023-12-19T18:17:39.810372+0000","last_clean":"2023-12-19T18:17:39.810372+0000","last_became_active":"2023-12-19T18:17:26.943303+0000","last_became_peered":"2023-12-19T18:17:26.943303+0000","last_unstale":"2023-12-19T18:17:39.810372+0000","last_undegraded":"2023-12-19T18:17:39.810372+0000","last_fullsized":"2023-12-19T18:17:39.810372+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.821104+0000","last_change":"2023-12-19T18:17:39.816900+0000","last_active":"2023-12-19T18:17:39.821104+0000","last_peered":"2023-12-19T18:17:39.821104+0000","last_clean":"2023-12-19T18:17:39.821104+0000","last_became_active":"2023-12-19T18:17:39.816650+0000","last_became_peered":"2023-12-19T18:17:39.816650+0000","last_unstale":"2023-12-19T18:17:39.821104+0000","last_undegraded":"2023-12-19T18:17:39.821104+0000","last_fullsized":"2023-12-19T18:17:39.821104+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:03.678500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.816630+0000","last_change":"2023-12-19T18:17:39.426984+0000","last_active":"2023-12-19T18:17:39.816630+0000","last_peered":"2023-12-19T18:17:39.816630+0000","last_clean":"2023-12-19T18:17:39.816630+0000","last_became_active":"2023-12-19T18:17:39.426585+0000","last_became_peered":"2023-12-19T18:17:39.426585+0000","last_unstale":"2023-12-19T18:17:39.816630+0000","last_undegraded":"2023-12-19T18:17:39.816630+0000","last_fullsized":"2023-12-19T18:17:39.816630+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:38.862837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.820660+0000","last_change":"2023-12-19T18:17:27.362825+0000","last_active":"2023-12-19T18:17:39.820660+0000","last_peered":"2023-12-19T18:17:39.820660+0000","last_clean":"2023-12-19T18:17:39.820660+0000","last_became_active":"2023-12-19T18:17:27.362414+0000","last_became_peered":"2023-12-19T18:17:27.362414+0000","last_unstale":"2023-12-19T18:17:39.820660+0000","last_undegraded":"2023-12-19T18:17:39.820660+0000","last_fullsized":"2023-12-19T18:17:39.820660+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:21:44.939055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.834359+0000","last_change":"2023-12-19T18:17:26.940233+0000","last_active":"2023-12-19T18:17:39.834359+0000","last_peered":"2023-12-19T18:17:39.834359+0000","last_clean":"2023-12-19T18:17:39.834359+0000","last_became_active":"2023-12-19T18:17:26.939941+0000","last_became_peered":"2023-12-19T18:17:26.939941+0000","last_unstale":"2023-12-19T18:17:39.834359+0000","last_undegraded":"2023-12-19T18:17:39.834359+0000","last_fullsized":"2023-12-19T18:17:39.834359+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:15:20.805126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812840+0000","last_change":"2023-12-19T18:17:39.820665+0000","last_active":"2023-12-19T18:17:40.812840+0000","last_peered":"2023-12-19T18:17:40.812840+0000","last_clean":"2023-12-19T18:17:40.812840+0000","last_became_active":"2023-12-19T18:17:39.820253+0000","last_became_peered":"2023-12-19T18:17:39.820253+0000","last_unstale":"2023-12-19T18:17:40.812840+0000","last_undegraded":"2023-12-19T18:17:40.812840+0000","last_fullsized":"2023-12-19T18:17:40.812840+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:35.239010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":11,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812940+0000","last_change":"2023-12-19T18:17:39.417596+0000","last_active":"2023-12-19T18:17:40.812940+0000","last_peered":"2023-12-19T18:17:40.812940+0000","last_clean":"2023-12-19T18:17:40.812940+0000","last_became_active":"2023-12-19T18:17:39.417251+0000","last_became_peered":"2023-12-19T18:17:39.417251+0000","last_unstale":"2023-12-19T18:17:40.812940+0000","last_undegraded":"2023-12-19T18:17:40.812940+0000","last_fullsized":"2023-12-19T18:17:40.812940+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:27:40.705352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.819831+0000","last_change":"2023-12-19T18:17:27.356126+0000","last_active":"2023-12-19T18:17:39.819831+0000","last_peered":"2023-12-19T18:17:39.819831+0000","last_clean":"2023-12-19T18:17:39.819831+0000","last_became_active":"2023-12-19T18:17:27.355794+0000","last_became_peered":"2023-12-19T18:17:27.355794+0000","last_unstale":"2023-12-19T18:17:39.819831+0000","last_undegraded":"2023-12-19T18:17:39.819831+0000","last_fullsized":"2023-12-19T18:17:39.819831+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:35:23.278389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":14,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.413871+0000","last_change":"2023-12-19T18:17:26.944096+0000","last_active":"2023-12-19T18:17:39.413871+0000","last_peered":"2023-12-19T18:17:39.413871+0000","last_clean":"2023-12-19T18:17:39.413871+0000","last_became_active":"2023-12-19T18:17:26.941219+0000","last_became_peered":"2023-12-19T18:17:26.941219+0000","last_unstale":"2023-12-19T18:17:39.413871+0000","last_undegraded":"2023-12-19T18:17:39.413871+0000","last_fullsized":"2023-12-19T18:17:39.413871+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:22:09.866781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":9,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.830456+0000","last_change":"2023-12-19T18:17:39.825795+0000","last_active":"2023-12-19T18:17:39.830456+0000","last_peered":"2023-12-19T18:17:39.830456+0000","last_clean":"2023-12-19T18:17:39.830456+0000","last_became_active":"2023-12-19T18:17:39.825535+0000","last_became_peered":"2023-12-19T18:17:39.825535+0000","last_unstale":"2023-12-19T18:17:39.830456+0000","last_undegraded":"2023-12-19T18:17:39.830456+0000","last_fullsized":"2023-12-19T18:17:39.830456+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:52.954654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":10,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.812019+0000","last_change":"2023-12-19T18:17:39.415538+0000","last_active":"2023-12-19T18:17:39.812019+0000","last_peered":"2023-12-19T18:17:39.812019+0000","last_clean":"2023-12-19T18:17:39.812019+0000","last_became_active":"2023-12-19T18:17:39.415278+0000","last_became_peered":"2023-12-19T18:17:39.415278+0000","last_unstale":"2023-12-19T18:17:39.812019+0000","last_undegraded":"2023-12-19T18:17:39.812019+0000","last_fullsized":"2023-12-19T18:17:39.812019+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:46:16.298136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.416970+0000","last_change":"2023-12-19T18:17:27.343300+0000","last_active":"2023-12-19T18:17:39.416970+0000","last_peered":"2023-12-19T18:17:39.416970+0000","last_clean":"2023-12-19T18:17:39.416970+0000","last_became_active":"2023-12-19T18:17:27.343020+0000","last_became_peered":"2023-12-19T18:17:27.343020+0000","last_unstale":"2023-12-19T18:17:39.416970+0000","last_undegraded":"2023-12-19T18:17:39.416970+0000","last_fullsized":"2023-12-19T18:17:39.416970+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:11.099486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":15,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.819996+0000","last_change":"2023-12-19T18:17:26.950561+0000","last_active":"2023-12-19T18:17:39.819996+0000","last_peered":"2023-12-19T18:17:39.819996+0000","last_clean":"2023-12-19T18:17:39.819996+0000","last_became_active":"2023-12-19T18:17:26.949052+0000","last_became_peered":"2023-12-19T18:17:26.949052+0000","last_unstale":"2023-12-19T18:17:39.819996+0000","last_undegraded":"2023-12-19T18:17:39.819996+0000","last_fullsized":"2023-12-19T18:17:39.819996+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:09:01.022466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":10,"reported_epoch":237,"state":"active+clean","last_fresh":"2023-12-19T18:17:40.812203+0000","last_change":"2023-12-19T18:17:39.846801+0000","last_active":"2023-12-19T18:17:40.812203+0000","last_peered":"2023-12-19T18:17:40.812203+0000","last_clean":"2023-12-19T18:17:40.812203+0000","last_became_active":"2023-12-19T18:17:39.846413+0000","last_became_peered":"2023-12-19T18:17:39.846413+0000","last_unstale":"2023-12-19T18:17:40.812203+0000","last_undegraded":"2023-12-19T18:17:40.812203+0000","last_fullsized":"2023-12-19T18:17:40.812203+0000","mapping_epoch":235,"log_start":"0'0","ondisk_log_start":"0'0","created":235,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:39.385547+0000","last_clean_scrub_stamp":"2023-12-19T18:17:39.385547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:12.643729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"236'4","reported_seq":14,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.836841+0000","last_change":"2023-12-19T18:17:39.423184+0000","last_active":"2023-12-19T18:17:39.836841+0000","last_peered":"2023-12-19T18:17:39.836841+0000","last_clean":"2023-12-19T18:17:39.836841+0000","last_became_active":"2023-12-19T18:17:39.422852+0000","last_became_peered":"2023-12-19T18:17:39.422852+0000","last_unstale":"2023-12-19T18:17:39.836841+0000","last_undegraded":"2023-12-19T18:17:39.836841+0000","last_fullsized":"2023-12-19T18:17:39.836841+0000","mapping_epoch":234,"log_start":"0'0","ondisk_log_start":"0'0","created":234,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:38.375642+0000","last_clean_scrub_stamp":"2023-12-19T18:17:38.375642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:26:15.844246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":13,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.414351+0000","last_change":"2023-12-19T18:17:27.349332+0000","last_active":"2023-12-19T18:17:39.414351+0000","last_peered":"2023-12-19T18:17:39.414351+0000","last_clean":"2023-12-19T18:17:39.414351+0000","last_became_active":"2023-12-19T18:17:27.349050+0000","last_became_peered":"2023-12-19T18:17:27.349050+0000","last_unstale":"2023-12-19T18:17:39.414351+0000","last_undegraded":"2023-12-19T18:17:39.414351+0000","last_fullsized":"2023-12-19T18:17:39.414351+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:26.920297+0000","last_clean_scrub_stamp":"2023-12-19T18:17:26.920297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:33.501865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"232'2","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.810749+0000","last_change":"2023-12-19T18:17:26.947941+0000","last_active":"2023-12-19T18:17:39.810749+0000","last_peered":"2023-12-19T18:17:39.810749+0000","last_clean":"2023-12-19T18:17:39.810749+0000","last_became_active":"2023-12-19T18:17:26.945013+0000","last_became_peered":"2023-12-19T18:17:26.945013+0000","last_unstale":"2023-12-19T18:17:39.810749+0000","last_undegraded":"2023-12-19T18:17:39.810749+0000","last_fullsized":"2023-12-19T18:17:39.810749+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":230,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:25.918058+0000","last_clean_scrub_stamp":"2023-12-19T18:17:25.918058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:47:39.421442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"218'368","reported_seq":1283,"reported_epoch":235,"state":"active+clean","last_fresh":"2023-12-19T18:17:39.414380+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:17:39.414380+0000","last_peered":"2023-12-19T18:17:39.414380+0000","last_clean":"2023-12-19T18:17:39.414380+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:17:39.414380+0000","last_undegraded":"2023-12-19T18:17:39.414380+0000","last_fullsized":"2023-12-19T18:17:39.414380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2306,"num_read_kb":4823,"num_write":615,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2306,"num_read_kb":4823,"num_write":615,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":368,"ondisk_log_size":368,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510579,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48476,"kb_used_data":3476,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323364,"statfs":{"total":96636764160,"available":96587124736,"internally_reserved":0,"allocated":3559424,"data_stored":2170833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68500000000000005,"15min":0.71499999999999997},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79000000000000004,"5min":1.048,"15min":1.1180000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71299999999999997,"15min":0.73199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90900000000000003,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63200000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":1.0389999999999999,"15min":1.0580000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.74199999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.73099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.0669999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.996,"5min":1.0820000000000001,"15min":1.236},"last":0.91300000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88100000000000001,"5min":1.002,"15min":1.079},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.49399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85899999999999999,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.78800000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.441,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.302},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.622,"15min":0.629},"min":{"1min":0.50800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.0209999999999999},"last":0.76100000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.90600000000000003,"5min":1.097,"15min":1.1299999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.55800000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95699999999999996,"5min":1.026,"15min":1.0800000000000001},"last":0.746}]},{"osd":6,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.61699999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.0469999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.093},"last":0.876}]}]},{"osd":6,"up_from":15,"seq":64424510579,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47904,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323936,"statfs":{"total":96636764160,"available":96587710464,"internally_reserved":0,"allocated":2973696,"data_stored":1580985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72299999999999998,"15min":0.73799999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.042},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.42799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.97599999999999998,"5min":1.224,"15min":1.224},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68899999999999995,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86699999999999999,"5min":1.103,"15min":1.103},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.745,"15min":0.752},"min":{"1min":0.53500000000000003,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":0.93600000000000005,"5min":0.97999999999999998,"15min":1.1259999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.752,"15min":0.76000000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98399999999999999,"5min":1.0549999999999999,"15min":1.0609999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.70899999999999996,"15min":0.745},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.98999999999999999,"5min":1.0089999999999999,"15min":1.111},"last":0.53700000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.754,"15min":0.77800000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.046,"5min":1.046,"15min":1.1519999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71599999999999997,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":1.036,"5min":1.268,"15min":1.268},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.624,"15min":0.59899999999999998},"min":{"1min":0.373,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0469999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63200000000000001,"15min":0.61599999999999999},"min":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.64700000000000002}]},{"osd":5,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.628,"15min":0.61399999999999999},"min":{"1min":0.47599999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.997,"5min":1.194,"15min":1.194},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.68999999999999995,"15min":0.67100000000000004},"min":{"1min":0.46800000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1339999999999999,"5min":1.141,"15min":1.141},"last":0.62}]},{"osd":7,"last update":"Tue Dec 19 18:17:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56599999999999995,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.54600000000000004,"5min":0.377,"15min":0.377},"max":{"1min":1.1140000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.63400000000000001}]}]},{"osd":5,"up_from":15,"seq":64424510579,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47888,"kb_used_data":2888,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323952,"statfs":{"total":96636764160,"available":96587726848,"internally_reserved":0,"allocated":2957312,"data_stored":1580325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.69599999999999995,"15min":0.72499999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.94999999999999996,"5min":1.002,"15min":1.0149999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.93100000000000005,"5min":0.96099999999999997,"15min":1.0229999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.497},"max":{"1min":0.84599999999999997,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.498,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97599999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.56599999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.58599999999999997,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":1.0069999999999999,"5min":1.0509999999999999,"15min":1.2250000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.73199999999999998},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88600000000000001,"5min":1.004,"15min":1.4279999999999999},"last":0.57299999999999995}]},{"osd":3,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.748,"15min":0.76200000000000001},"min":{"1min":0.56799999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.90400000000000003,"5min":1.0129999999999999,"15min":1.05},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93400000000000005,"5min":1.022,"15min":1.0329999999999999},"last":0.68200000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.81699999999999995,"5min":0.94499999999999995,"15min":1.0800000000000001},"last":0.442},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94299999999999995,"5min":1.208,"15min":1.208},"last":0.55100000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65800000000000003,"15min":0.65700000000000003},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.85799999999999998,"5min":1.008,"15min":1.0449999999999999},"last":0.53700000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.501,"5min":0.439,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":1.169,"15min":1.169},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51100000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.079,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.627}]}]},{"osd":4,"up_from":15,"seq":64424510579,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47912,"kb_used_data":2912,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323928,"statfs":{"total":96636764160,"available":96587702272,"internally_reserved":0,"allocated":2981888,"data_stored":1582563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.72599999999999998,"15min":0.73899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86699999999999999,"5min":1.036,"15min":1.109},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.88100000000000001,"5min":0.997,"15min":1.01},"last":0.67400000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.752,"15min":0.753},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90500000000000003,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.753,"15min":0.76200000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86699999999999999,"5min":0.997,"15min":1.1020000000000001},"last":0.77100000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0289999999999999,"15min":1.071},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.77400000000000002,"15min":0.77300000000000002},"min":{"1min":0.51800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91800000000000004,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.751,"15min":0.751},"min":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84699999999999998,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.77000000000000002,"15min":0.78600000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.85999999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.56799999999999995}]},{"osd":5,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65400000000000003,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"max":{"1min":0.91000000000000003,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":0.89800000000000002,"5min":1.042,"15min":1.042},"last":0.71199999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":1.024,"15min":1.024},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73699999999999999,"15min":0.71499999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90900000000000003,"5min":1.0680000000000001,"15min":1.0760000000000001},"last":0.85199999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.41899999999999998},"max":{"1min":0.93500000000000005,"5min":1.087,"15min":1.087},"last":0.753},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77500000000000002,"15min":0.76000000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.46999999999999997},"max":{"1min":0.98499999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.88500000000000001}]}]},{"osd":3,"up_from":15,"seq":64424510579,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47908,"kb_used_data":2908,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323932,"statfs":{"total":96636764160,"available":96587706368,"internally_reserved":0,"allocated":2977792,"data_stored":1583782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.498}]},{"osd":1,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48999999999999999,"15min":0.495},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72599999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56100000000000005,"15min":0.57299999999999995},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67700000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.36199999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73099999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.58199999999999996,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80700000000000005,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.55900000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.54900000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0109999999999999,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80500000000000005,"15min":0.81699999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0329999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.67600000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.80600000000000005,"15min":0.83399999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.99199999999999999,"5min":1.1739999999999999,"15min":1.397},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.80100000000000005,"15min":0.82599999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.042,"5min":1.2230000000000001,"15min":1.262},"last":0.40999999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82799999999999996,"15min":0.84499999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0329999999999999,"5min":1.234,"15min":1.274},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80700000000000005,"15min":0.83899999999999997},"min":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.93799999999999994,"5min":1.165,"15min":1.302},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.872,"15min":0.89800000000000002},"min":{"1min":0.65100000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.153,"5min":1.3540000000000001,"15min":1.486},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.86599999999999999,"15min":0.88600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1279999999999999,"5min":1.421,"15min":1.421},"last":0.76000000000000001}]}]},{"osd":2,"up_from":15,"seq":64424510578,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47912,"kb_used_data":2912,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323928,"statfs":{"total":96636764160,"available":96587702272,"internally_reserved":0,"allocated":2981888,"data_stored":1583276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.441,"15min":0.45000000000000001},"min":{"1min":0.26300000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.70499999999999996,"5min":0.754,"15min":0.754},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48899999999999999,"15min":0.504},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68000000000000005,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44700000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59899999999999998,"15min":0.61599999999999999},"min":{"1min":0.44400000000000001,"5min":0.39500000000000002,"15min":0.38500000000000001},"max":{"1min":0.77800000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51000000000000001,"15min":0.52600000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.73899999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.40000000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.58599999999999997,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.76800000000000002,"5min":0.80800000000000005,"15min":0.95999999999999996},"last":0.499},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.36899999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.875},"last":0.56399999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.77800000000000002,"15min":0.80000000000000004},"min":{"1min":0.55300000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0580000000000001,"5min":1.0880000000000001,"15min":1.306},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77500000000000002,"15min":0.79200000000000004},"min":{"1min":0.48299999999999998,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0609999999999999,"5min":1.2070000000000001,"15min":1.466},"last":0.71299999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.77200000000000002,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.3049999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.80000000000000004,"15min":0.81899999999999995},"min":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.04,"5min":1.1519999999999999,"15min":1.429},"last":0.74399999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.81299999999999994,"15min":0.83799999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0329999999999999,"5min":1.216,"15min":1.4119999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.77600000000000002,"15min":0.80000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.008,"5min":1.117,"15min":1.2470000000000001},"last":0.68600000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:17:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.78500000000000003,"15min":0.82199999999999995},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.56000000000000005},"max":{"1min":1.0269999999999999,"5min":1.159,"15min":1.45},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.83899999999999997,"15min":0.85599999999999998},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.093,"5min":1.276,"15min":1.276},"last":0.77700000000000002}]}]},{"osd":1,"up_from":15,"seq":64424510579,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60200,"kb_used_data":2912,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311640,"statfs":{"total":96636764160,"available":96575119360,"internally_reserved":0,"allocated":2981888,"data_stored":1582103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44800000000000001,"15min":0.45800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56399999999999995,"5min":0.71999999999999997,"15min":0.89400000000000002},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79600000000000004,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.40200000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66000000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.66600000000000004,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.46800000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.622,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.747,"5min":0.94199999999999995,"15min":0.94499999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.628,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.78100000000000003,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92700000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77700000000000002,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.081,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76900000000000002,"15min":0.80000000000000004},"min":{"1min":0.58699999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0189999999999999,"5min":1.236,"15min":1.2609999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81799999999999995},"min":{"1min":0.58199999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0669999999999999,"5min":1.353,"15min":1.353},"last":0.75}]},{"osd":6,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.505},"max":{"1min":1.0009999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.82399999999999995,"15min":0.84099999999999997},"min":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.034,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.77400000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84599999999999997,"15min":0.85399999999999998},"min":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.56499999999999995},"max":{"1min":1.147,"5min":1.399,"15min":1.399},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.82799999999999996,"15min":0.85199999999999998},"min":{"1min":0.53000000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.181,"5min":1.365,"15min":1.365},"last":0.81499999999999995}]}]},{"osd":0,"up_from":14,"seq":60129543284,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60768,"kb_used_data":3480,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311072,"statfs":{"total":96636764160,"available":96574537728,"internally_reserved":0,"allocated":3563520,"data_stored":2171399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41899999999999998,"15min":0.45300000000000001},"min":{"1min":0.29699999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55300000000000005,"5min":0.69299999999999995,"15min":1.216},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.49399999999999999,"15min":0.51900000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67600000000000005,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.51600000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.48799999999999999,"15min":0.52600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.77000000000000002,"15min":1.1699999999999999},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52300000000000002,"15min":0.54700000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.96799999999999997},"last":0.47299999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.51400000000000001,"15min":0.55000000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.59999999999999998,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59899999999999998,"15min":0.64400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.032},"last":0.49299999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.80600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.496},"max":{"1min":0.93899999999999995,"5min":1.1200000000000001,"15min":1.381},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.78600000000000003,"15min":0.81100000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.47799999999999998},"max":{"1min":0.84799999999999998,"5min":1.137,"15min":1.2849999999999999},"last":0.78400000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.93799999999999994,"5min":1.1180000000000001,"15min":1.3420000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.79100000000000004,"15min":0.81299999999999994},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.052,"5min":1.1259999999999999,"15min":1.288},"last":0.76800000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81200000000000006,"15min":0.83899999999999997},"min":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.51300000000000001},"max":{"1min":0.97899999999999998,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.80000000000000004,"15min":0.82699999999999996},"min":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97999999999999998,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.69499999999999995}]},{"osd":7,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0149999999999999,"5min":1.1899999999999999,"15min":1.415},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.82599999999999996,"15min":0.85099999999999998},"min":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96899999999999997,"5min":1.0980000000000001,"15min":1.395},"last":0.81200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:17:44.349 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:17:44.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:44.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69d536d640 1 Processor -- start 2023-12-19T18:17:44.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69d536d640 1 -- start start 2023-12-19T18:17:44.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69d536d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69d0155030 0x7f69d015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69d536d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f69d015f370 0x7f69d006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69d536d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69d00ce740 0x7f69d00cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69d536d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d00cf130 con 0x7f69d00ce740 2023-12-19T18:17:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69d536d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d006c970 con 0x7f69d015f370 2023-12-19T18:17:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69d536d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d015b1f0 con 0x7f69d0155030 2023-12-19T18:17:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69cf7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69d00ce740 0x7f69d00cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69cf7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69d00ce740 0x7f69d00cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36742/0 (socket says 172.21.15.26:36742) 2023-12-19T18:17:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69cf7fe640 1 -- 172.21.15.26:0/2205905602 learned_addr learned my addr 172.21.15.26:0/2205905602 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69ceffd640 1 --2- 172.21.15.26:0/2205905602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69d0155030 0x7f69d015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69c7fff640 1 --2- 172.21.15.26:0/2205905602 >> v2:172.21.15.160:3300/0 conn(0x7f69d015f370 0x7f69d006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69cf7fe640 1 -- 172.21.15.26:0/2205905602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69d0155030 msgr2=0x7f69d015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69cf7fe640 1 --2- 172.21.15.26:0/2205905602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69d0155030 0x7f69d015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69cf7fe640 1 -- 172.21.15.26:0/2205905602 >> v2:172.21.15.160:3300/0 conn(0x7f69d015f370 msgr2=0x7f69d006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:44.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69cf7fe640 1 --2- 172.21.15.26:0/2205905602 >> v2:172.21.15.160:3300/0 conn(0x7f69d015f370 0x7f69d006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69cf7fe640 1 -- 172.21.15.26:0/2205905602 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d015b4d0 con 0x7f69d00ce740 2023-12-19T18:17:44.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.521+0000 7f69c7fff640 1 --2- 172.21.15.26:0/2205905602 >> v2:172.21.15.160:3300/0 conn(0x7f69d015f370 0x7f69d006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:44.529 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.525+0000 7f69cf7fe640 1 --2- 172.21.15.26:0/2205905602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d00ce740 0x7f69d00cebf0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f69d00b3ac0 tx=0x7f69c801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26adccee1e1bb1c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:44.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.525+0000 7f69ce7fc640 1 -- 172.21.15.26:0/2205905602 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c80a5840 con 0x7f69d00ce740 2023-12-19T18:17:44.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.525+0000 7f69ce7fc640 1 -- 172.21.15.26:0/2205905602 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c80a59a0 con 0x7f69d00ce740 2023-12-19T18:17:44.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.525+0000 7f69ce7fc640 1 -- 172.21.15.26:0/2205905602 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c80a5d30 con 0x7f69d00ce740 2023-12-19T18:17:44.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.525+0000 7f69d536d640 1 -- 172.21.15.26:0/2205905602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d00ce740 msgr2=0x7f69d00cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:44.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.525+0000 7f69d536d640 1 --2- 172.21.15.26:0/2205905602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d00ce740 0x7f69d00cebf0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f69d00b3ac0 tx=0x7f69c801c9a0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.525+0000 7f69d536d640 1 -- 172.21.15.26:0/2205905602 shutdown_connections 2023-12-19T18:17:44.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.525+0000 7f69d536d640 1 --2- 172.21.15.26:0/2205905602 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d00ce740 0x7f69d00cebf0 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.525+0000 7f69d536d640 1 --2- 172.21.15.26:0/2205905602 >> v2:172.21.15.160:3300/0 conn(0x7f69d015f370 0x7f69d006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.525+0000 7f69d536d640 1 --2- 172.21.15.26:0/2205905602 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69d0155030 0x7f69d015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.525+0000 7f69d536d640 1 -- 172.21.15.26:0/2205905602 >> 172.21.15.26:0/2205905602 conn(0x7f69d00658e0 msgr2=0x7f69d0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:44.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.525+0000 7f69d536d640 1 -- 172.21.15.26:0/2205905602 shutdown_connections 2023-12-19T18:17:44.531 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.525+0000 7f69d536d640 1 -- 172.21.15.26:0/2205905602 wait complete. 2023-12-19T18:17:44.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69d536d640 1 Processor -- start 2023-12-19T18:17:44.532 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69d536d640 1 -- start start 2023-12-19T18:17:44.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69d536d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d00ce740 0x7f69d00b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69d536d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69d0155030 0x7f69d0159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69d536d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69d015f370 0x7f69d0150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69d536d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d0155bf0 con 0x7f69d00ce740 2023-12-19T18:17:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69d536d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d0159fa0 con 0x7f69d0155030 2023-12-19T18:17:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69d536d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d015a280 con 0x7f69d015f370 2023-12-19T18:17:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69ceffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d00ce740 0x7f69d00b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69ceffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d00ce740 0x7f69d00b1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36752/0 (socket says 172.21.15.26:36752) 2023-12-19T18:17:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69ceffd640 1 -- 172.21.15.26:0/2948419131 learned_addr learned my addr 172.21.15.26:0/2948419131 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69cf7fe640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69d015f370 0x7f69d0150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:44.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69c7fff640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69d0155030 0x7f69d0159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69ceffd640 1 -- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69d015f370 msgr2=0x7f69d0150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69ceffd640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69d015f370 0x7f69d0150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69ceffd640 1 -- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69d0155030 msgr2=0x7f69d0159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69ceffd640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69d0155030 0x7f69d0159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69ceffd640 1 -- 172.21.15.26:0/2948419131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c8009b30 con 0x7f69d00ce740 2023-12-19T18:17:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69cf7fe640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69d015f370 0x7f69d0150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69c7fff640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69d0155030 0x7f69d0159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69ceffd640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d00ce740 0x7f69d00b1a50 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f69c000e870 tx=0x7f69c000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69c77fe640 1 -- 172.21.15.26:0/2948419131 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c0009510 con 0x7f69d00ce740 2023-12-19T18:17:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69c77fe640 1 -- 172.21.15.26:0/2948419131 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c000eec0 con 0x7f69d00ce740 2023-12-19T18:17:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69d536d640 1 -- 172.21.15.26:0/2948419131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d006c740 con 0x7f69d00ce740 2023-12-19T18:17:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69c77fe640 1 -- 172.21.15.26:0/2948419131 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c000cb40 con 0x7f69d00ce740 2023-12-19T18:17:44.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.529+0000 7f69d536d640 1 -- 172.21.15.26:0/2948419131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d00b3570 con 0x7f69d00ce740 2023-12-19T18:17:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.533+0000 7f69d536d640 1 -- 172.21.15.26:0/2948419131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d0074270 con 0x7f69d00ce740 2023-12-19T18:17:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.533+0000 7f69c77fe640 1 -- 172.21.15.26:0/2948419131 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f69c000cca0 con 0x7f69d00ce740 2023-12-19T18:17:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.533+0000 7f69c77fe640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f69b0065ca0 0x7f69b0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:44.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.533+0000 7f69c77fe640 1 -- 172.21.15.26:0/2948419131 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f69c0083d30 con 0x7f69d00ce740 2023-12-19T18:17:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.533+0000 7f69c7fff640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f69b0065ca0 0x7f69b0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:44.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.533+0000 7f69c7fff640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f69b0065ca0 0x7f69b0068160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f69d00cabb0 tx=0x7f69b8008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:44.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.537+0000 7f69c77fe640 1 -- 172.21.15.26:0/2948419131 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f69c0050d30 con 0x7f69d00ce740 2023-12-19T18:17:44.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.725+0000 7f69d536d640 1 -- 172.21.15.26:0/2948419131 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69d00ccc30 con 0x7f69d00ce740 2023-12-19T18:17:44.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.725+0000 7f69c77fe640 1 -- 172.21.15.26:0/2948419131 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5921 (secure 0 0 0) 0x7f69c0017090 con 0x7f69d00ce740 2023-12-19T18:17:44.730 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:44.730 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2727392302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2727392302},{"type":"v1","addr":"172.21.15.160:6836","nonce":2727392302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1029020739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1029020739},{"type":"v1","addr":"172.21.15.160:6837","nonce":1029020739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:27.325271+0000","modified":"2023-12-19T18:17:30.337993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11427":{"gid":11427,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2285370499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2285370499},{"type":"v1","addr":"172.21.15.26:6835","nonce":2285370499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:39.801489+0000","modified":"2023-12-19T18:17:42.831660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11424":{"gid":11424,"name":"a","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6834/3251191056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3251191056},{"type":"v1","addr":"172.21.15.26:6834","nonce":3251191056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-19T18:17:44.731 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 118 2023-12-19T18:17:44.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.729+0000 7f69d536d640 1 -- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f69b0065ca0 msgr2=0x7f69b0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:44.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.729+0000 7f69d536d640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f69b0065ca0 0x7f69b0068160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f69d00cabb0 tx=0x7f69b8008040 comp rx=0 tx=0).stop 2023-12-19T18:17:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.729+0000 7f69d536d640 1 -- 172.21.15.26:0/2948419131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d00ce740 msgr2=0x7f69d00b1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.729+0000 7f69d536d640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d00ce740 0x7f69d00b1a50 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f69c000e870 tx=0x7f69c000ec70 comp rx=0 tx=0).stop 2023-12-19T18:17:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.729+0000 7f69d536d640 1 -- 172.21.15.26:0/2948419131 shutdown_connections 2023-12-19T18:17:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.729+0000 7f69d536d640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f69b0065ca0 0x7f69b0068160 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.729+0000 7f69d536d640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69d015f370 0x7f69d0150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.729+0000 7f69d536d640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69d0155030 0x7f69d0159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.729+0000 7f69d536d640 1 --2- 172.21.15.26:0/2948419131 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d00ce740 0x7f69d00b1a50 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.729+0000 7f69d536d640 1 -- 172.21.15.26:0/2948419131 >> 172.21.15.26:0/2948419131 conn(0x7f69d00658e0 msgr2=0x7f69d0151f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.729+0000 7f69d536d640 1 -- 172.21.15.26:0/2948419131 shutdown_connections 2023-12-19T18:17:44.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:44.729+0000 7f69d536d640 1 -- 172.21.15.26:0/2948419131 wait complete. 2023-12-19T18:17:45.154 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 118, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:17:39.801489+0000', 'modified': '2023-12-19T18:17:42.831660+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11424}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11424': {'gid': 11424, 'name': 'a', 'rank': 0, 'incarnation': 114, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.26:6834/3251191056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 3251191056}, {'type': 'v1', 'addr': '172.21.15.26:6834', 'nonce': 3251191056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [31], 'metadata_pool': 30, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:17:45.155 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:17:45.155 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-19T18:17:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.297+0000 7fbee4939640 1 Processor -- start 2023-12-19T18:17:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.297+0000 7fbee4939640 1 -- start start 2023-12-19T18:17:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.301+0000 7fbee4939640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbee006b3d0 0x7fbee00ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.301+0000 7fbee4939640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbee00cec40 0x7fbee015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.301+0000 7fbee4939640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee006bc30 con 0x7fbee0154220 2023-12-19T18:17:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.301+0000 7fbee4939640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee00cf0d0 con 0x7fbee00cec40 2023-12-19T18:17:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.301+0000 7fbee4939640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee00cc6e0 con 0x7fbee006b3d0 2023-12-19T18:17:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.301+0000 7fbeded9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbee00cec40 0x7fbee015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.301+0000 7fbeded9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbee00cec40 0x7fbee015b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36762/0 (socket says 172.21.15.26:36762) 2023-12-19T18:17:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.301+0000 7fbeded9d640 1 -- 172.21.15.26:0/2648504113 learned_addr learned my addr 172.21.15.26:0/2648504113 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.301+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/2648504113 >> v2:172.21.15.160:3300/0 conn(0x7fbee006b3d0 0x7fbee00ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.301+0000 7fbeded9d640 1 -- 172.21.15.26:0/2648504113 >> v2:172.21.15.160:3300/0 conn(0x7fbee006b3d0 msgr2=0x7fbee00ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.301+0000 7fbeded9d640 1 --2- 172.21.15.26:0/2648504113 >> v2:172.21.15.160:3300/0 conn(0x7fbee006b3d0 0x7fbee00ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.301+0000 7fbeded9d640 1 -- 172.21.15.26:0/2648504113 >> v1:172.21.15.160:6789/0 conn(0x7fbee0154220 legacy=0x7fbee015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.301+0000 7fbeded9d640 1 -- 172.21.15.26:0/2648504113 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee006b950 con 0x7fbee00cec40 2023-12-19T18:17:45.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.305+0000 7fbeded9d640 1 --2- 172.21.15.26:0/2648504113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee00cec40 0x7fbee015b1c0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fbee00cf880 tx=0x7fbed801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0ba9983f2be234 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.305+0000 7fbedd59a640 1 -- 172.21.15.26:0/2648504113 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed80a5860 con 0x7fbee00cec40 2023-12-19T18:17:45.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.305+0000 7fbedd59a640 1 -- 172.21.15.26:0/2648504113 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed80a59c0 con 0x7fbee00cec40 2023-12-19T18:17:45.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.305+0000 7fbedd59a640 1 -- 172.21.15.26:0/2648504113 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed80a5d50 con 0x7fbee00cec40 2023-12-19T18:17:45.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.305+0000 7fbee4939640 1 -- 172.21.15.26:0/2648504113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee00cec40 msgr2=0x7fbee015b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.305+0000 7fbee4939640 1 --2- 172.21.15.26:0/2648504113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee00cec40 0x7fbee015b1c0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fbee00cf880 tx=0x7fbed801c9a0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.305+0000 7fbee4939640 1 -- 172.21.15.26:0/2648504113 shutdown_connections 2023-12-19T18:17:45.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.305+0000 7fbee4939640 1 --2- 172.21.15.26:0/2648504113 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee00cec40 0x7fbee015b1c0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.305+0000 7fbee4939640 1 --2- 172.21.15.26:0/2648504113 >> v2:172.21.15.160:3300/0 conn(0x7fbee006b3d0 0x7fbee00ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.305+0000 7fbee4939640 1 -- 172.21.15.26:0/2648504113 >> 172.21.15.26:0/2648504113 conn(0x7fbee00658e0 msgr2=0x7fbee0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:45.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.305+0000 7fbee4939640 1 -- 172.21.15.26:0/2648504113 shutdown_connections 2023-12-19T18:17:45.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.305+0000 7fbee4939640 1 -- 172.21.15.26:0/2648504113 wait complete. 2023-12-19T18:17:45.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.305+0000 7fbee4939640 1 Processor -- start 2023-12-19T18:17:45.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbee4939640 1 -- start start 2023-12-19T18:17:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbee4939640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee006b3d0 0x7fbee0070f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbee4939640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbee00cec40 0x7fbee014e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbee4939640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbee0154220 0x7fbee014c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbee4939640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee014ed40 con 0x7fbee006b3d0 2023-12-19T18:17:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbee4939640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00ad420 con 0x7fbee0154220 2023-12-19T18:17:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbee4939640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee01508d0 con 0x7fbee00cec40 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbede59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee006b3d0 0x7fbee0070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbede59c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee006b3d0 0x7fbee0070f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36776/0 (socket says 172.21.15.26:36776) 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbede59c640 1 -- 172.21.15.26:0/2800653384 learned_addr learned my addr 172.21.15.26:0/2800653384 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbee00cec40 0x7fbee014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbeded9d640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbee0154220 0x7fbee014c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbede59c640 1 -- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbee00cec40 msgr2=0x7fbee014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbede59c640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbee00cec40 0x7fbee014e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbede59c640 1 -- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbee0154220 msgr2=0x7fbee014c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbede59c640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbee0154220 0x7fbee014c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbede59c640 1 -- 172.21.15.26:0/2800653384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed8009b30 con 0x7fbee006b3d0 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbede59c640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee006b3d0 0x7fbee0070f90 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fbed000c500 tx=0x7fbed000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbeceffd640 1 -- 172.21.15.26:0/2800653384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0005220 con 0x7fbee006b3d0 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbeceffd640 1 -- 172.21.15.26:0/2800653384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed0005380 con 0x7fbee006b3d0 2023-12-19T18:17:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbeceffd640 1 -- 172.21.15.26:0/2800653384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0007270 con 0x7fbee006b3d0 2023-12-19T18:17:45.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbee4939640 1 -- 172.21.15.26:0/2800653384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee00ad6c0 con 0x7fbee006b3d0 2023-12-19T18:17:45.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.309+0000 7fbee4939640 1 -- 172.21.15.26:0/2800653384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee014c9a0 con 0x7fbee006b3d0 2023-12-19T18:17:45.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbeceffd640 1 -- 172.21.15.26:0/2800653384 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fbed00073d0 con 0x7fbee006b3d0 2023-12-19T18:17:45.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbeceffd640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbec0065cf0 0x7fbec00681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbeceffd640 1 -- 172.21.15.26:0/2800653384 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fbed0082e60 con 0x7fbee006b3d0 2023-12-19T18:17:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbee7c9b000 1 Processor -- start 2023-12-19T18:17:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbec0065cf0 0x7fbec00681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbee7c9b000 1 -- start start 2023-12-19T18:17:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbec0065cf0 0x7fbec00681b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbee015e590 tx=0x7fbed4008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbee7c9b000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf34c070 0x563fdf34c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbee7c9b000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c990 0x563fdf301c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbee7c9b000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf3021c0 0x563fdf3025a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbee7c9b000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdf125c90 con 0x563fdf34c990 2023-12-19T18:17:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbee7c9b000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdf157220 con 0x563fdf34c070 2023-12-19T18:17:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbee7c9b000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fdf1547f0 con 0x563fdf3021c0 2023-12-19T18:17:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbeddd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c990 0x563fdf301c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbeddd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c990 0x563fdf301c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36792/0 (socket says 172.21.15.26:36792) 2023-12-19T18:17:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbeddd9b640 1 -- 172.21.15.26:0/3576863656 learned_addr learned my addr 172.21.15.26:0/3576863656 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbeded9d640 1 --2- 172.21.15.26:0/3576863656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf3021c0 0x563fdf3025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.313+0000 7fbede59c640 1 --2- 172.21.15.26:0/3576863656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf34c070 0x563fdf34c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbede59c640 1 -- 172.21.15.26:0/3576863656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf3021c0 msgr2=0x563fdf3025a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbede59c640 1 --2- 172.21.15.26:0/3576863656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf3021c0 0x563fdf3025a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbede59c640 1 -- 172.21.15.26:0/3576863656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c990 msgr2=0x563fdf301c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbede59c640 1 --2- 172.21.15.26:0/3576863656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c990 0x563fdf301c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbede59c640 1 -- 172.21.15.26:0/3576863656 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fdee4c890 con 0x563fdf34c070 2023-12-19T18:17:45.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/3576863656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c990 0x563fdf301c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:45.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbede59c640 1 --2- 172.21.15.26:0/3576863656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf34c070 0x563fdf34c450 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fbed0080ac0 tx=0x7fbed0053870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c34cf78a56ee84b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbebffff640 1 -- 172.21.15.26:0/3576863656 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed006b070 con 0x563fdf34c070 2023-12-19T18:17:45.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbebffff640 1 -- 172.21.15.26:0/3576863656 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed006b1d0 con 0x563fdf34c070 2023-12-19T18:17:45.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbebffff640 1 -- 172.21.15.26:0/3576863656 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0053c60 con 0x563fdf34c070 2023-12-19T18:17:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 -- 172.21.15.26:0/3576863656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf34c070 msgr2=0x563fdf34c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/3576863656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf34c070 0x563fdf34c450 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fbed0080ac0 tx=0x7fbed0053870 comp rx=0 tx=0).stop 2023-12-19T18:17:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 -- 172.21.15.26:0/3576863656 shutdown_connections 2023-12-19T18:17:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/3576863656 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf3021c0 0x563fdf3025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/3576863656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c990 0x563fdf301c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/3576863656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf34c070 0x563fdf34c450 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 -- 172.21.15.26:0/3576863656 >> 172.21.15.26:0/3576863656 conn(0x563fdf308950 msgr2=0x563fdf300560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 -- 172.21.15.26:0/3576863656 shutdown_connections 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 -- 172.21.15.26:0/3576863656 wait complete. 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 -- start start 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 Processor -- start 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf3021c0 0x563fdf307740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf30e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf34c990 0x563fdf30e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdf1500e0 con 0x563fdf34c070 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdf0ef310 con 0x563fdf3021c0 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fdf10d770 con 0x563fdf34c990 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbee7c9b000 10 client.0 _ll_register_callbacks cb 0x563fdf2edb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:17:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbeddd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf30e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbeddd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf30e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36802/0 (socket says 172.21.15.26:36802) 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.317+0000 7fbeddd9b640 1 -- 172.21.15.26:0/459356912 learned_addr learned my addr 172.21.15.26:0/459356912 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbede59c640 1 --2- 172.21.15.26:0/459356912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf3021c0 0x563fdf307740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbeded9d640 1 --2- 172.21.15.26:0/459356912 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf34c990 0x563fdf30e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbeddd9b640 1 -- 172.21.15.26:0/459356912 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf34c990 msgr2=0x563fdf30e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/459356912 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf34c990 0x563fdf30e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbeddd9b640 1 -- 172.21.15.26:0/459356912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf3021c0 msgr2=0x563fdf307740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/459356912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf3021c0 0x563fdf307740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbeddd9b640 1 -- 172.21.15.26:0/459356912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed400e040 con 0x563fdf34c070 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/459356912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf30e080 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fbed40074c0 tx=0x7fbed40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4aa23e1298d3822 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbebd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbebd7fa640 1 -- 172.21.15.26:0/459356912 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed40147c0 con 0x563fdf34c070 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbebd7fa640 1 -- 172.21.15.26:0/459356912 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fbed4014920 con 0x563fdf34c070 2023-12-19T18:17:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 -- 172.21.15.26:0/459356912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fdf11fc10 con 0x563fdf34c070 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbebd7fa640 1 -- 172.21.15.26:0/459356912 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbed4014ae0 con 0x563fdf34c070 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 10 client.11721 fetch_fsmap learned FSMap version 118 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 -- 172.21.15.26:0/459356912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) v3 -- 0x7fbee014c9a0 con 0x563fdf34c070 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbebd7fa640 1 -- 172.21.15.26:0/459356912 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563fdf11fc10 con 0x563fdf34c070 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbebd7fa640 1 -- 172.21.15.26:0/459356912 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 118) v1 ==== 3681+0+0 (secure 0 0 0) 0x7fbed4014c40 con 0x563fdf34c070 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 10 client.11721 fetch_fsmap finished waiting for FSMap version 118 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 10 client.11721 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 20 client.11721 populate_metadata read hostname 'smithi026' 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/459356912 >> [v2:172.21.15.26:6833/3251191056,v1:172.21.15.26:6834/3251191056] conn(0x563fdf3538d0 0x563fdf353cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 4 client.11721 mds_command: new command op to 11424 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 -- 172.21.15.26:0/459356912 --> [v2:172.21.15.26:6833/3251191056,v1:172.21.15.26:6834/3251191056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fdf14ea30 con 0x563fdf3538d0 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbede59c640 1 --2- 172.21.15.26:0/459356912 >> [v2:172.21.15.26:6833/3251191056,v1:172.21.15.26:6834/3251191056] conn(0x563fdf3538d0 0x563fdf353cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbede59c640 1 --2- 172.21.15.26:0/459356912 >> [v2:172.21.15.26:6833/3251191056,v1:172.21.15.26:6834/3251191056] conn(0x563fdf3538d0 0x563fdf353cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=392c8156cd0da618 server_cookie=67bc711d6a5c7dae in_seq=0 out_seq=0 2023-12-19T18:17:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbebd7fa640 10 client.11721 ms_handle_connect on v2:172.21.15.26:6833/3251191056 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbebd7fa640 1 -- 172.21.15.26:0/459356912 <== mds.0 v2:172.21.15.26:6833/3251191056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563fdf14ea30 con 0x563fdf3538d0 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbebd7fa640 10 client.11721 handle_command_reply: tid=0 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 client.11721 shutdown 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 -- 172.21.15.26:0/459356912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 msgr2=0x563fdf30e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/459356912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf30e080 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7fbed40074c0 tx=0x7fbed40046d0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 -- 172.21.15.26:0/459356912 shutdown_connections 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/459356912 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf34c990 0x563fdf30e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/459356912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf30e080 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/459356912 >> [v2:172.21.15.26:6833/3251191056,v1:172.21.15.26:6834/3251191056] conn(0x563fdf3538d0 0x563fdf353cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/459356912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf3021c0 0x563fdf307740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 -- 172.21.15.26:0/459356912 >> 172.21.15.26:0/459356912 conn(0x563fdf308950 msgr2=0x563fdf300c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbebd7fa640 0 client.11721 ms_handle_reset on v2:172.21.15.26:6833/3251191056 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 -- 172.21.15.26:0/459356912 shutdown_connections 2023-12-19T18:17:45.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.321+0000 7fbee7c9b000 1 -- 172.21.15.26:0/459356912 wait complete. 2023-12-19T18:17:45.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.325+0000 7fbee7c9b000 20 client.11721 trim_cache size 0 max 16384 2023-12-19T18:17:45.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.337+0000 7fbee7c9b000 1 Processor -- start 2023-12-19T18:17:45.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.337+0000 7fbee7c9b000 1 -- start start 2023-12-19T18:17:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf3021c0 0x563fdf30dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf30e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf34c990 0x563fdf34a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdee0d940 con 0x563fdf34c070 2023-12-19T18:17:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdf304fa0 con 0x563fdf34c990 2023-12-19T18:17:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fdf0f8860 con 0x563fdf3021c0 2023-12-19T18:17:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbeddd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf30e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbeddd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf30e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36810/0 (socket says 172.21.15.26:36810) 2023-12-19T18:17:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbeddd9b640 1 -- 172.21.15.26:0/4210450054 learned_addr learned my addr 172.21.15.26:0/4210450054 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbeddd9b640 1 -- 172.21.15.26:0/4210450054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf3021c0 msgr2=0x563fdf30dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/4210450054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf3021c0 0x563fdf30dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbeddd9b640 1 -- 172.21.15.26:0/4210450054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf34c990 msgr2=0x563fdf34a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/4210450054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf34c990 0x563fdf34a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbeddd9b640 1 -- 172.21.15.26:0/4210450054 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fdf3050e0 con 0x563fdf34c070 2023-12-19T18:17:45.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/4210450054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf30e510 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fbed4008010 tx=0x7fbed4023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b49f8cb3deb175 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbeccff9640 1 -- 172.21.15.26:0/4210450054 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4006de0 con 0x563fdf34c070 2023-12-19T18:17:45.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbeccff9640 1 -- 172.21.15.26:0/4210450054 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed4037040 con 0x563fdf34c070 2023-12-19T18:17:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbeccff9640 1 -- 172.21.15.26:0/4210450054 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4016ea0 con 0x563fdf34c070 2023-12-19T18:17:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 -- 172.21.15.26:0/4210450054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 msgr2=0x563fdf30e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/4210450054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf30e510 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fbed4008010 tx=0x7fbed4023000 comp rx=0 tx=0).stop 2023-12-19T18:17:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 -- 172.21.15.26:0/4210450054 shutdown_connections 2023-12-19T18:17:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/4210450054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf34c990 0x563fdf34a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/4210450054 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf30e510 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/4210450054 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf3021c0 0x563fdf30dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 -- 172.21.15.26:0/4210450054 >> 172.21.15.26:0/4210450054 conn(0x563fdf308950 msgr2=0x563fdf30d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 -- 172.21.15.26:0/4210450054 shutdown_connections 2023-12-19T18:17:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 -- 172.21.15.26:0/4210450054 wait complete. 2023-12-19T18:17:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 -- start start 2023-12-19T18:17:45.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.341+0000 7fbee7c9b000 1 Processor -- start 2023-12-19T18:17:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf3021c0 0x563fdf38bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf38c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf34c990 0x563fdf38ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdf151670 con 0x563fdf34c070 2023-12-19T18:17:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdf1584b0 con 0x563fdf3021c0 2023-12-19T18:17:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fdf0f5770 con 0x563fdf34c990 2023-12-19T18:17:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 10 client.0 _ll_register_callbacks cb 0x563fdf0fe640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:17:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbeddd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf38c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbeddd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf38c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36812/0 (socket says 172.21.15.26:36812) 2023-12-19T18:17:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbeddd9b640 1 -- 172.21.15.26:0/798754577 learned_addr learned my addr 172.21.15.26:0/798754577 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:45.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbede59c640 1 --2- 172.21.15.26:0/798754577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf3021c0 0x563fdf38bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbeded9d640 1 --2- 172.21.15.26:0/798754577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf34c990 0x563fdf38ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbeddd9b640 1 -- 172.21.15.26:0/798754577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf34c990 msgr2=0x563fdf38ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/798754577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf34c990 0x563fdf38ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbeddd9b640 1 -- 172.21.15.26:0/798754577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf3021c0 msgr2=0x563fdf38bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/798754577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf3021c0 0x563fdf38bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbeddd9b640 1 -- 172.21.15.26:0/798754577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee014c9a0 con 0x563fdf34c070 2023-12-19T18:17:45.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbeddd9b640 1 --2- 172.21.15.26:0/798754577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf38c940 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fbed40074c0 tx=0x7fbed4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8035e3e2a508e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbebd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T18:17:45.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbebd7fa640 1 -- 172.21.15.26:0/798754577 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed4016ea0 con 0x563fdf34c070 2023-12-19T18:17:45.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbebd7fa640 1 -- 172.21.15.26:0/798754577 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fbed402b580 con 0x563fdf34c070 2023-12-19T18:17:45.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 -- 172.21.15.26:0/798754577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563fdf115e00 con 0x563fdf34c070 2023-12-19T18:17:45.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbebd7fa640 1 -- 172.21.15.26:0/798754577 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbed4006de0 con 0x563fdf34c070 2023-12-19T18:17:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 10 client.11727 fetch_fsmap learned FSMap version 118 2023-12-19T18:17:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 -- 172.21.15.26:0/798754577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) v3 -- 0x563fdf115260 con 0x563fdf34c070 2023-12-19T18:17:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbebd7fa640 1 -- 172.21.15.26:0/798754577 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563fdf115e00 con 0x563fdf34c070 2023-12-19T18:17:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbebd7fa640 1 -- 172.21.15.26:0/798754577 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 118) v1 ==== 3681+0+0 (secure 0 0 0) 0x7fbed4042070 con 0x563fdf34c070 2023-12-19T18:17:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 10 client.11727 fetch_fsmap finished waiting for FSMap version 118 2023-12-19T18:17:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 10 client.11727 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T18:17:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 20 client.11727 populate_metadata read hostname 'smithi026' 2023-12-19T18:17:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/798754577 >> [v2:172.21.15.26:6833/3251191056,v1:172.21.15.26:6834/3251191056] conn(0x563fdf38d3c0 0x563fdf38d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 4 client.11727 mds_command: new command op to 11424 tid=0[{"prefix": "status"}] 2023-12-19T18:17:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 -- 172.21.15.26:0/798754577 --> [v2:172.21.15.26:6833/3251191056,v1:172.21.15.26:6834/3251191056] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563fdf149160 con 0x563fdf38d3c0 2023-12-19T18:17:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbede59c640 1 --2- 172.21.15.26:0/798754577 >> [v2:172.21.15.26:6833/3251191056,v1:172.21.15.26:6834/3251191056] conn(0x563fdf38d3c0 0x563fdf38d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbede59c640 1 --2- 172.21.15.26:0/798754577 >> [v2:172.21.15.26:6833/3251191056,v1:172.21.15.26:6834/3251191056] conn(0x563fdf38d3c0 0x563fdf38d7a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=457ec8d7f76de452 server_cookie=f3b934fe39eaf3 in_seq=0 out_seq=0 2023-12-19T18:17:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbebd7fa640 10 client.11727 ms_handle_connect on v2:172.21.15.26:6833/3251191056 2023-12-19T18:17:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbebd7fa640 1 -- 172.21.15.26:0/798754577 <== mds.0 v2:172.21.15.26:6833/3251191056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563fdf149160 con 0x563fdf38d3c0 2023-12-19T18:17:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbebd7fa640 10 client.11727 handle_command_reply: tid=0 2023-12-19T18:17:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 client.11727 shutdown 2023-12-19T18:17:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 -- 172.21.15.26:0/798754577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 msgr2=0x563fdf38c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/798754577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf38c940 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7fbed40074c0 tx=0x7fbed4017000 comp rx=0 tx=0).stop 2023-12-19T18:17:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 -- 172.21.15.26:0/798754577 shutdown_connections 2023-12-19T18:17:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/798754577 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x563fdf34c990 0x563fdf38ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/798754577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563fdf34c070 0x563fdf38c940 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/798754577 >> [v2:172.21.15.26:6833/3251191056,v1:172.21.15.26:6834/3251191056] conn(0x563fdf38d3c0 0x563fdf38d7a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 --2- 172.21.15.26:0/798754577 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563fdf3021c0 0x563fdf38bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 -- 172.21.15.26:0/798754577 >> 172.21.15.26:0/798754577 conn(0x563fdf308950 msgr2=0x563fdf349030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbebd7fa640 0 client.11727 ms_handle_reset on v2:172.21.15.26:6833/3251191056 2023-12-19T18:17:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 -- 172.21.15.26:0/798754577 shutdown_connections 2023-12-19T18:17:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 1 -- 172.21.15.26:0/798754577 wait complete. 2023-12-19T18:17:45.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.345+0000 7fbee7c9b000 20 client.11727 trim_cache size 0 max 16384 2023-12-19T18:17:45.356 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:17:45.356 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T18:17:45.356 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T18:17:45.356 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 11424, 2023-12-19T18:17:45.357 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T18:17:45.357 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T18:17:45.357 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "backup_fs", 2023-12-19T18:17:45.357 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.5467162419999996, 2023-12-19T18:17:45.357 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 118, 2023-12-19T18:17:45.357 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 237, 2023-12-19T18:17:45.357 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 237, 2023-12-19T18:17:45.357 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 22.358160544 2023-12-19T18:17:45.357 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:17:45.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.349+0000 7fbee4939640 1 -- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbec0065cf0 msgr2=0x7fbec00681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.349+0000 7fbee4939640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbec0065cf0 0x7fbec00681b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbee015e590 tx=0x7fbed4008040 comp rx=0 tx=0).stop 2023-12-19T18:17:45.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.349+0000 7fbee4939640 1 -- 172.21.15.26:0/2800653384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee006b3d0 msgr2=0x7fbee0070f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.349+0000 7fbee4939640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee006b3d0 0x7fbee0070f90 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fbed000c500 tx=0x7fbed000c9c0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.349+0000 7fbee4939640 1 -- 172.21.15.26:0/2800653384 shutdown_connections 2023-12-19T18:17:45.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.349+0000 7fbee4939640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbec0065cf0 0x7fbec00681b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.349+0000 7fbee4939640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbee0154220 0x7fbee014c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.349+0000 7fbee4939640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbee00cec40 0x7fbee014e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.349+0000 7fbee4939640 1 --2- 172.21.15.26:0/2800653384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee006b3d0 0x7fbee0070f90 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.349+0000 7fbee4939640 1 -- 172.21.15.26:0/2800653384 >> 172.21.15.26:0/2800653384 conn(0x7fbee00658e0 msgr2=0x7fbee015c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:45.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.349+0000 7fbee4939640 1 -- 172.21.15.26:0/2800653384 shutdown_connections 2023-12-19T18:17:45.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.349+0000 7fbee4939640 1 -- 172.21.15.26:0/2800653384 wait complete. 2023-12-19T18:17:45.370 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-19T18:17:45.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.509+0000 7f261f5bc640 1 Processor -- start 2023-12-19T18:17:45.514 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.509+0000 7f261f5bc640 1 -- start start 2023-12-19T18:17:45.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261f5bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2618155030 0x7f261815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261f5bc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f261815f370 0x7f261806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261f5bc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26180ce740 0x7f26180cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261f5bc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26180cf130 con 0x7f26180ce740 2023-12-19T18:17:45.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261f5bc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261806c970 con 0x7f261815f370 2023-12-19T18:17:45.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261f5bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261815b1f0 con 0x7f2618155030 2023-12-19T18:17:45.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261db59640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26180ce740 0x7f26180cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261db59640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f26180ce740 0x7f26180cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36814/0 (socket says 172.21.15.26:36814) 2023-12-19T18:17:45.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261db59640 1 -- 172.21.15.26:0/3541143766 learned_addr learned my addr 172.21.15.26:0/3541143766 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:45.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261d358640 1 --2- 172.21.15.26:0/3541143766 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2618155030 0x7f261815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261cb57640 1 --2- 172.21.15.26:0/3541143766 >> v2:172.21.15.160:3300/0 conn(0x7f261815f370 0x7f261806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261db59640 1 -- 172.21.15.26:0/3541143766 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2618155030 msgr2=0x7f261815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261db59640 1 --2- 172.21.15.26:0/3541143766 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2618155030 0x7f261815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261db59640 1 -- 172.21.15.26:0/3541143766 >> v2:172.21.15.160:3300/0 conn(0x7f261815f370 msgr2=0x7f261806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261db59640 1 --2- 172.21.15.26:0/3541143766 >> v2:172.21.15.160:3300/0 conn(0x7f261815f370 0x7f261806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.513+0000 7f261db59640 1 -- 172.21.15.26:0/3541143766 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261815b4d0 con 0x7f26180ce740 2023-12-19T18:17:45.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261db59640 1 --2- 172.21.15.26:0/3541143766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26180ce740 0x7f26180cebf0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f26180b3ac0 tx=0x7f261001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3c4a0b1953628b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f260ffff640 1 -- 172.21.15.26:0/3541143766 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26100a5840 con 0x7f26180ce740 2023-12-19T18:17:45.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f260ffff640 1 -- 172.21.15.26:0/3541143766 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26100a59a0 con 0x7f26180ce740 2023-12-19T18:17:45.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f260ffff640 1 -- 172.21.15.26:0/3541143766 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26100a5d30 con 0x7f26180ce740 2023-12-19T18:17:45.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 -- 172.21.15.26:0/3541143766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26180ce740 msgr2=0x7f26180cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 --2- 172.21.15.26:0/3541143766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26180ce740 0x7f26180cebf0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f26180b3ac0 tx=0x7f261001c9a0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 -- 172.21.15.26:0/3541143766 shutdown_connections 2023-12-19T18:17:45.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 --2- 172.21.15.26:0/3541143766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26180ce740 0x7f26180cebf0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 --2- 172.21.15.26:0/3541143766 >> v2:172.21.15.160:3300/0 conn(0x7f261815f370 0x7f261806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 --2- 172.21.15.26:0/3541143766 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2618155030 0x7f261815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 -- 172.21.15.26:0/3541143766 >> 172.21.15.26:0/3541143766 conn(0x7f26180658e0 msgr2=0x7f2618065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:45.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 -- 172.21.15.26:0/3541143766 shutdown_connections 2023-12-19T18:17:45.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 -- 172.21.15.26:0/3541143766 wait complete. 2023-12-19T18:17:45.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 Processor -- start 2023-12-19T18:17:45.522 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 -- start start 2023-12-19T18:17:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26180ce740 0x7f261814bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2618155030 0x7f261815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f261815f370 0x7f261815caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26181522a0 con 0x7f26180ce740 2023-12-19T18:17:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261813ccc0 con 0x7f261815f370 2023-12-19T18:17:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.517+0000 7f261f5bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261814c280 con 0x7f2618155030 2023-12-19T18:17:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261d358640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26180ce740 0x7f261814bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261d358640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26180ce740 0x7f261814bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36818/0 (socket says 172.21.15.26:36818) 2023-12-19T18:17:45.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261d358640 1 -- 172.21.15.26:0/4253193122 learned_addr learned my addr 172.21.15.26:0/4253193122 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261cb57640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2618155030 0x7f261815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261db59640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f261815f370 0x7f261815caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261d358640 1 -- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2618155030 msgr2=0x7f261815c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261d358640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2618155030 0x7f261815c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261d358640 1 -- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f261815f370 msgr2=0x7f261815caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261d358640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f261815f370 0x7f261815caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261d358640 1 -- 172.21.15.26:0/4253193122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2610009b30 con 0x7f26180ce740 2023-12-19T18:17:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261cb57640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2618155030 0x7f261815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261db59640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f261815f370 0x7f261815caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261d358640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26180ce740 0x7f261814bd40 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f261400d5f0 tx=0x7f261400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f260dffb640 1 -- 172.21.15.26:0/4253193122 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26140125f0 con 0x7f26180ce740 2023-12-19T18:17:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f260dffb640 1 -- 172.21.15.26:0/4253193122 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2614012750 con 0x7f26180ce740 2023-12-19T18:17:45.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f260dffb640 1 -- 172.21.15.26:0/4253193122 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614012ae0 con 0x7f26180ce740 2023-12-19T18:17:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261f5bc640 1 -- 172.21.15.26:0/4253193122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261806c740 con 0x7f26180ce740 2023-12-19T18:17:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261f5bc640 1 -- 172.21.15.26:0/4253193122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26180adca0 con 0x7f26180ce740 2023-12-19T18:17:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f261f5bc640 1 -- 172.21.15.26:0/4253193122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2618074270 con 0x7f26180ce740 2023-12-19T18:17:45.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f260dffb640 1 -- 172.21.15.26:0/4253193122 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f2614012c40 con 0x7f26180ce740 2023-12-19T18:17:45.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f260dffb640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f25fc06a0b0 0x7f25fc06c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.521+0000 7f260dffb640 1 -- 172.21.15.26:0/4253193122 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f2614084140 con 0x7f26180ce740 2023-12-19T18:17:45.527 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.525+0000 7f261cb57640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f25fc06a0b0 0x7f25fc06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.528 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.525+0000 7f261cb57640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f25fc06a0b0 0x7f25fc06c570 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f261814b4e0 tx=0x7f2608008040 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.530 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.525+0000 7f260dffb640 1 -- 172.21.15.26:0/4253193122 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f26140510a0 con 0x7f26180ce740 2023-12-19T18:17:45.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.705+0000 7f261f5bc640 1 -- 172.21.15.26:0/4253193122 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f261806c5c0 con 0x7f25fc06a0b0 2023-12-19T18:17:45.709 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.705+0000 7f260dffb640 1 -- 172.21.15.26:0/4253193122 <== mgr.10429 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f2608007680 con 0x7f25fc06a0b0 2023-12-19T18:17:45.709 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T18:17:45.709 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:45.710 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2023-12-19T18:17:45.710 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:17:45.710 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:17:45.710 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "backup_fs" 2023-12-19T18:17:45.710 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:17:45.710 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T18:17:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.709+0000 7f261f5bc640 1 -- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f25fc06a0b0 msgr2=0x7f25fc06c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.709+0000 7f261f5bc640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f25fc06a0b0 0x7f25fc06c570 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f261814b4e0 tx=0x7f2608008040 comp rx=0 tx=0).stop 2023-12-19T18:17:45.711 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.709+0000 7f261f5bc640 1 -- 172.21.15.26:0/4253193122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26180ce740 msgr2=0x7f261814bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.709+0000 7f261f5bc640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26180ce740 0x7f261814bd40 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f261400d5f0 tx=0x7f261400dab0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.709+0000 7f261f5bc640 1 -- 172.21.15.26:0/4253193122 shutdown_connections 2023-12-19T18:17:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.709+0000 7f261f5bc640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f25fc06a0b0 0x7f25fc06c570 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.709+0000 7f261f5bc640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f261815f370 0x7f261815caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.709+0000 7f261f5bc640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2618155030 0x7f261815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.709+0000 7f261f5bc640 1 --2- 172.21.15.26:0/4253193122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26180ce740 0x7f261814bd40 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.709+0000 7f261f5bc640 1 -- 172.21.15.26:0/4253193122 >> 172.21.15.26:0/4253193122 conn(0x7f26180658e0 msgr2=0x7f261815d150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.709+0000 7f261f5bc640 1 -- 172.21.15.26:0/4253193122 shutdown_connections 2023-12-19T18:17:45.712 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.709+0000 7f261f5bc640 1 -- 172.21.15.26:0/4253193122 wait complete. 2023-12-19T18:17:45.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-19T18:17:45.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb64612c640 1 Processor -- start 2023-12-19T18:17:45.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb64612c640 1 -- start start 2023-12-19T18:17:45.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb64612c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb6400cb150 0x7fb6400cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb64612c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6400ce3f0 con 0x7fb6401585f0 2023-12-19T18:17:45.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb64612c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb64015b1b0 con 0x7fb64015e690 2023-12-19T18:17:45.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb64612c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6400cbb40 con 0x7fb6400cb150 2023-12-19T18:17:45.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb6401585f0 0x7fb6400cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59542/0 (socket says 172.21.15.26:59542) 2023-12-19T18:17:45.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63f7fe640 1 -- 172.21.15.26:0/1414672489 learned_addr learned my addr 172.21.15.26:0/1414672489 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:45.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63effd640 1 --2- 172.21.15.26:0/1414672489 >> v2:172.21.15.160:3300/0 conn(0x7fb6400cb150 0x7fb6400cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347719070 0 0) 0x7fb6400ce3f0 con 0x7fb6401585f0 2023-12-19T18:17:45.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb628003530 con 0x7fb6401585f0 2023-12-19T18:17:45.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3177886095 0 0) 0x7fb628003530 con 0x7fb6401585f0 2023-12-19T18:17:45.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 >> v2:172.21.15.160:3300/0 conn(0x7fb6400cb150 msgr2=0x7fb6400cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 --2- 172.21.15.26:0/1414672489 >> v2:172.21.15.160:3300/0 conn(0x7fb6400cb150 0x7fb6400cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 >> v1:172.21.15.160:6789/0 conn(0x7fb64015e690 legacy=0x7fb64015eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64015b490 con 0x7fb6401585f0 2023-12-19T18:17:45.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fb630001680 con 0x7fb6401585f0 2023-12-19T18:17:45.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 >> v1:172.21.15.26:6789/0 conn(0x7fb6401585f0 legacy=0x7fb6400cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 --2- 172.21.15.26:0/1414672489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb628003980 0x7fb628003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 --2- 172.21.15.26:0/1414672489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb628004380 0x7fb628004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 --2- 172.21.15.26:0/1414672489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6280050f0 0x7fb62800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.880 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400cbb40 con 0x7fb628003980 2023-12-19T18:17:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62800c030 con 0x7fb628004380 2023-12-19T18:17:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62800c1f0 con 0x7fb6280050f0 2023-12-19T18:17:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.873+0000 7fb63f7fe640 1 --2- 172.21.15.26:0/1414672489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb628003980 0x7fb628003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb63effd640 1 --2- 172.21.15.26:0/1414672489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb628004380 0x7fb628004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb63ffff640 1 --2- 172.21.15.26:0/1414672489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6280050f0 0x7fb62800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb63f7fe640 1 -- 172.21.15.26:0/1414672489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6280050f0 msgr2=0x7fb62800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb63f7fe640 1 --2- 172.21.15.26:0/1414672489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6280050f0 0x7fb62800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb63f7fe640 1 -- 172.21.15.26:0/1414672489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb628004380 msgr2=0x7fb628004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb63f7fe640 1 --2- 172.21.15.26:0/1414672489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb628004380 0x7fb628004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb63f7fe640 1 -- 172.21.15.26:0/1414672489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb628003530 con 0x7fb628003980 2023-12-19T18:17:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb63ffff640 1 --2- 172.21.15.26:0/1414672489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb6280050f0 0x7fb62800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb63ffff640 1 -- 172.21.15.26:0/1414672489 reap_dead start 2023-12-19T18:17:45.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb63f7fe640 1 --2- 172.21.15.26:0/1414672489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb628003980 0x7fb628003d60 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c210 tx=0x7fb63001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ac653053a0fc7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630005ae0 con 0x7fb628003980 2023-12-19T18:17:45.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb630005c60 con 0x7fb628003980 2023-12-19T18:17:45.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb63e7fc640 1 -- 172.21.15.26:0/1414672489 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6300072b0 con 0x7fb628003980 2023-12-19T18:17:45.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb64612c640 1 -- 172.21.15.26:0/1414672489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb628003980 msgr2=0x7fb628003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb64612c640 1 --2- 172.21.15.26:0/1414672489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb628003980 0x7fb628003d60 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c210 tx=0x7fb63001cc60 comp rx=0 tx=0).stop 2023-12-19T18:17:45.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb64612c640 1 -- 172.21.15.26:0/1414672489 shutdown_connections 2023-12-19T18:17:45.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb64612c640 1 --2- 172.21.15.26:0/1414672489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb628003980 0x7fb628003d60 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.877+0000 7fb64612c640 1 -- 172.21.15.26:0/1414672489 >> 172.21.15.26:0/1414672489 conn(0x7fb640066160 msgr2=0x7fb6400b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:45.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb64612c640 1 -- 172.21.15.26:0/1414672489 shutdown_connections 2023-12-19T18:17:45.883 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb64612c640 1 -- 172.21.15.26:0/1414672489 wait complete. 2023-12-19T18:17:45.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb64612c640 1 Processor -- start 2023-12-19T18:17:45.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb64612c640 1 -- start start 2023-12-19T18:17:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb64612c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb628003980 0x7fb6400c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb64612c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6400c5ad0 0x7fb6400b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb64612c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb640157b30 0x7fb64006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb64612c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400ae420 con 0x7fb628003980 2023-12-19T18:17:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb64612c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400ae0d0 con 0x7fb6400c5ad0 2023-12-19T18:17:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb64612c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400c7ed0 con 0x7fb640157b30 2023-12-19T18:17:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb63f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb628003980 0x7fb6400c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb63f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb628003980 0x7fb6400c2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36844/0 (socket says 172.21.15.26:36844) 2023-12-19T18:17:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb63f7fe640 1 -- 172.21.15.26:0/2733360331 learned_addr learned my addr 172.21.15.26:0/2733360331 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb63effd640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6400c5ad0 0x7fb6400b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb63ffff640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb640157b30 0x7fb64006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb63f7fe640 1 -- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb640157b30 msgr2=0x7fb64006c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb63f7fe640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb640157b30 0x7fb64006c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb63f7fe640 1 -- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6400c5ad0 msgr2=0x7fb6400b01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb63f7fe640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6400c5ad0 0x7fb6400b01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb63f7fe640 1 -- 172.21.15.26:0/2733360331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6400c81b0 con 0x7fb628003980 2023-12-19T18:17:45.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.881+0000 7fb63f7fe640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb628003980 0x7fb6400c2270 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c210 tx=0x7fb628005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.885+0000 7fb627fff640 1 -- 172.21.15.26:0/2733360331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6300b4070 con 0x7fb628003980 2023-12-19T18:17:45.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.885+0000 7fb627fff640 1 -- 172.21.15.26:0/2733360331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb630001440 con 0x7fb628003980 2023-12-19T18:17:45.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.885+0000 7fb627fff640 1 -- 172.21.15.26:0/2733360331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630001810 con 0x7fb628003980 2023-12-19T18:17:45.888 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.885+0000 7fb64612c640 1 -- 172.21.15.26:0/2733360331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6400c8490 con 0x7fb628003980 2023-12-19T18:17:45.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.885+0000 7fb64612c640 1 -- 172.21.15.26:0/2733360331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb640157fb0 con 0x7fb628003980 2023-12-19T18:17:45.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.885+0000 7fb64612c640 1 -- 172.21.15.26:0/2733360331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6401581d0 con 0x7fb628003980 2023-12-19T18:17:45.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.885+0000 7fb627fff640 1 -- 172.21.15.26:0/2733360331 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 128) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb630007070 con 0x7fb628003980 2023-12-19T18:17:45.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.885+0000 7fb627fff640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb62c065ca0 0x7fb62c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.885+0000 7fb627fff640 1 -- 172.21.15.26:0/2733360331 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fb63011d530 con 0x7fb628003980 2023-12-19T18:17:45.890 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.885+0000 7fb63effd640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb62c065ca0 0x7fb62c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.891 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.889+0000 7fb63effd640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb62c065ca0 0x7fb62c068160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb6400b1140 tx=0x7fb634009190 comp rx=0 tx=0).ready entity=mgr.10429 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:45.889+0000 7fb627fff640 1 -- 172.21.15.26:0/2733360331 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb6300ea520 con 0x7fb628003980 2023-12-19T18:17:46.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.085+0000 7fb64612c640 1 -- 172.21.15.26:0/2733360331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fb640074270 con 0x7fb628003980 2023-12-19T18:17:46.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.105+0000 7fb627fff640 1 -- 172.21.15.26:0/2733360331 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 129) v1 ==== 85449+0+0 (secure 0 0 0) 0x7fb6300f2d50 con 0x7fb628003980 2023-12-19T18:17:46.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.109+0000 7fb627fff640 1 -- 172.21.15.26:0/2733360331 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v129) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb6300edfb0 con 0x7fb628003980 2023-12-19T18:17:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.109+0000 7fb64612c640 1 -- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb62c065ca0 msgr2=0x7fb62c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.109+0000 7fb64612c640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb62c065ca0 0x7fb62c068160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb6400b1140 tx=0x7fb634009190 comp rx=0 tx=0).stop 2023-12-19T18:17:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.109+0000 7fb64612c640 1 -- 172.21.15.26:0/2733360331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb628003980 msgr2=0x7fb6400c2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.109+0000 7fb64612c640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb628003980 0x7fb6400c2270 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c210 tx=0x7fb628005800 comp rx=0 tx=0).stop 2023-12-19T18:17:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.109+0000 7fb64612c640 1 -- 172.21.15.26:0/2733360331 shutdown_connections 2023-12-19T18:17:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.109+0000 7fb64612c640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb62c065ca0 0x7fb62c068160 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.109+0000 7fb64612c640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb640157b30 0x7fb64006c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.109+0000 7fb64612c640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6400c5ad0 0x7fb6400b01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.109+0000 7fb64612c640 1 --2- 172.21.15.26:0/2733360331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb628003980 0x7fb6400c2270 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.109+0000 7fb64612c640 1 -- 172.21.15.26:0/2733360331 >> 172.21.15.26:0/2733360331 conn(0x7fb640066160 msgr2=0x7fb64015a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:46.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.109+0000 7fb64612c640 1 -- 172.21.15.26:0/2733360331 shutdown_connections 2023-12-19T18:17:46.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.113+0000 7fb64612c640 1 -- 172.21.15.26:0/2733360331 wait complete. 2023-12-19T18:17:46.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-19T18:17:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.281+0000 7fe1fea95640 1 Processor -- start 2023-12-19T18:17:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.281+0000 7fe1fea95640 1 -- start start 2023-12-19T18:17:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1fea95640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1f815d0d0 0x7fe1f80b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1fea95640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe1f80b4600 0x7fe1f80b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1fea95640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f80b4f90 con 0x7fe1f815ccf0 2023-12-19T18:17:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1fea95640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f815e200 con 0x7fe1f80b4600 2023-12-19T18:17:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1fea95640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f80d0aa0 con 0x7fe1f815d0d0 2023-12-19T18:17:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1fc831640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe1f815ccf0 0x7fe1f80d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46290/0 (socket says 172.21.15.26:46290) 2023-12-19T18:17:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1fc831640 1 -- 172.21.15.26:0/778145730 learned_addr learned my addr 172.21.15.26:0/778145730 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1f7fff640 1 --2- 172.21.15.26:0/778145730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1f815d0d0 0x7fe1f80b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1fd032640 1 --2- 172.21.15.26:0/778145730 >> v2:172.21.15.160:3300/0 conn(0x7fe1f80b4600 0x7fe1f80b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1f77fe640 1 -- 172.21.15.26:0/778145730 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981005822 0 0) 0x7fe1ec001120 con 0x7fe1f815ccf0 2023-12-19T18:17:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1f77fe640 1 -- 172.21.15.26:0/778145730 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e8003530 con 0x7fe1f815ccf0 2023-12-19T18:17:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1f7fff640 1 -- 172.21.15.26:0/778145730 >> v2:172.21.15.160:3300/0 conn(0x7fe1f80b4600 msgr2=0x7fe1f80b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1f7fff640 1 --2- 172.21.15.26:0/778145730 >> v2:172.21.15.160:3300/0 conn(0x7fe1f80b4600 0x7fe1f80b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1f7fff640 1 -- 172.21.15.26:0/778145730 >> v1:172.21.15.160:6789/0 conn(0x7fe1f815ccf0 legacy=0x7fe1f80d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.285+0000 7fe1f7fff640 1 -- 172.21.15.26:0/778145730 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f815d680 con 0x7fe1f815d0d0 2023-12-19T18:17:46.289 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:46.285+0000 7f6745a76180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:17:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1f7fff640 1 --2- 172.21.15.26:0/778145730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1f815d0d0 0x7fe1f80b8400 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40097c0 tx=0x7fe1e401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66200fd6ed614a26 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1f77fe640 1 -- 172.21.15.26:0/778145730 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40ab650 con 0x7fe1f815d0d0 2023-12-19T18:17:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1f77fe640 1 -- 172.21.15.26:0/778145730 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e40ab7b0 con 0x7fe1f815d0d0 2023-12-19T18:17:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1f77fe640 1 -- 172.21.15.26:0/778145730 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40abb20 con 0x7fe1f815d0d0 2023-12-19T18:17:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1fea95640 1 -- 172.21.15.26:0/778145730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1f815d0d0 msgr2=0x7fe1f80b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1fea95640 1 --2- 172.21.15.26:0/778145730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1f815d0d0 0x7fe1f80b8400 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40097c0 tx=0x7fe1e401c450 comp rx=0 tx=0).stop 2023-12-19T18:17:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1fea95640 1 -- 172.21.15.26:0/778145730 shutdown_connections 2023-12-19T18:17:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1fea95640 1 --2- 172.21.15.26:0/778145730 >> v2:172.21.15.160:3300/0 conn(0x7fe1f80b4600 0x7fe1f80b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1fea95640 1 --2- 172.21.15.26:0/778145730 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1f815d0d0 0x7fe1f80b8400 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1fea95640 1 -- 172.21.15.26:0/778145730 >> 172.21.15.26:0/778145730 conn(0x7fe1f8065ee0 msgr2=0x7fe1f80cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1fea95640 1 -- 172.21.15.26:0/778145730 shutdown_connections 2023-12-19T18:17:46.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1fea95640 1 -- 172.21.15.26:0/778145730 wait complete. 2023-12-19T18:17:46.293 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:46.292+0000 7f5950ba0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:17:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1fea95640 1 Processor -- start 2023-12-19T18:17:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.289+0000 7fe1fea95640 1 -- start start 2023-12-19T18:17:46.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1fea95640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80b4600 0x7fe1f80b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1fea95640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1f815ccf0 0x7fe1f8154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1fea95640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1f815d0d0 0x7fe1f8153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1fea95640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f80b6a20 con 0x7fe1f815ccf0 2023-12-19T18:17:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1fea95640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f81567e0 con 0x7fe1f80b4600 2023-12-19T18:17:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1fea95640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f8156ac0 con 0x7fe1f815d0d0 2023-12-19T18:17:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1f7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1f815ccf0 0x7fe1f8154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1f7fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1f815ccf0 0x7fe1f8154870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34172/0 (socket says 172.21.15.26:34172) 2023-12-19T18:17:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1fc831640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80b4600 0x7fe1f80b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1f7fff640 1 -- 172.21.15.26:0/1020669165 learned_addr learned my addr 172.21.15.26:0/1020669165 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:46.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1fd032640 1 --2- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1f815d0d0 0x7fe1f8153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1f7fff640 1 -- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1f815d0d0 msgr2=0x7fe1f8153c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1f7fff640 1 --2- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1f815d0d0 0x7fe1f8153c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1f7fff640 1 -- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80b4600 msgr2=0x7fe1f80b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1f7fff640 1 --2- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80b4600 0x7fe1f80b5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1f7fff640 1 -- 172.21.15.26:0/1020669165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e4009420 con 0x7fe1f815ccf0 2023-12-19T18:17:46.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1fc831640 1 --2- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80b4600 0x7fe1f80b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:46.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1f7fff640 1 --2- 172.21.15.26:0/1020669165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1f815ccf0 0x7fe1f8154870 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fe1e401ccf0 tx=0x7fe1e40191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:46.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1f57fa640 1 -- 172.21.15.26:0/1020669165 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40abb20 con 0x7fe1f815ccf0 2023-12-19T18:17:46.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1f57fa640 1 -- 172.21.15.26:0/1020669165 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e40b9070 con 0x7fe1f815ccf0 2023-12-19T18:17:46.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1fea95640 1 -- 172.21.15.26:0/1020669165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f8156da0 con 0x7fe1f815ccf0 2023-12-19T18:17:46.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1f57fa640 1 -- 172.21.15.26:0/1020669165 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40abe80 con 0x7fe1f815ccf0 2023-12-19T18:17:46.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1fea95640 1 -- 172.21.15.26:0/1020669165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f8157360 con 0x7fe1f815ccf0 2023-12-19T18:17:46.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.293+0000 7fe1fea95640 1 -- 172.21.15.26:0/1020669165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f8074270 con 0x7fe1f815ccf0 2023-12-19T18:17:46.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.297+0000 7fe1f57fa640 1 -- 172.21.15.26:0/1020669165 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 129) v1 ==== 85449+0+0 (secure 0 0 0) 0x7fe1e4003620 con 0x7fe1f815ccf0 2023-12-19T18:17:46.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.297+0000 7fe1f57fa640 1 --2- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe1dc065cf0 0x7fe1dc0681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.297+0000 7fe1f57fa640 1 -- 172.21.15.26:0/1020669165 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7fe1e40bb070 con 0x7fe1f815ccf0 2023-12-19T18:17:46.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.297+0000 7fe1fc831640 1 -- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe1dc065cf0 msgr2=0x7fe1dc0681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:46.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.297+0000 7fe1fc831640 1 --2- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe1dc065cf0 0x7fe1dc0681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:46.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.305+0000 7fe1f57fa640 1 -- 172.21.15.26:0/1020669165 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe1e40ec040 con 0x7fe1f815ccf0 2023-12-19T18:17:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.461+0000 7fe1fea95640 1 -- 172.21.15.26:0/1020669165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fe1f80b3040 con 0x7fe1f815ccf0 2023-12-19T18:17:46.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.469+0000 7fe1f57fa640 1 -- 172.21.15.26:0/1020669165 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v77)=0 v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe1e40c5020 con 0x7fe1f815ccf0 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.469+0000 7fe1fea95640 1 -- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe1dc065cf0 msgr2=0x7fe1dc0681b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.469+0000 7fe1fea95640 1 --2- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe1dc065cf0 0x7fe1dc0681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.469+0000 7fe1fea95640 1 -- 172.21.15.26:0/1020669165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1f815ccf0 msgr2=0x7fe1f8154870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.469+0000 7fe1fea95640 1 --2- 172.21.15.26:0/1020669165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1f815ccf0 0x7fe1f8154870 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fe1e401ccf0 tx=0x7fe1e40191f0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.469+0000 7fe1fea95640 1 -- 172.21.15.26:0/1020669165 shutdown_connections 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.469+0000 7fe1fea95640 1 --2- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe1dc065cf0 0x7fe1dc0681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.469+0000 7fe1fea95640 1 --2- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe1f815d0d0 0x7fe1f8153c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.469+0000 7fe1fea95640 1 --2- 172.21.15.26:0/1020669165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1f815ccf0 0x7fe1f8154870 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.469+0000 7fe1fea95640 1 --2- 172.21.15.26:0/1020669165 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe1f80b4600 0x7fe1f80b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.469+0000 7fe1fea95640 1 -- 172.21.15.26:0/1020669165 >> 172.21.15.26:0/1020669165 conn(0x7fe1f8065ee0 msgr2=0x7fe1f80b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.469+0000 7fe1fea95640 1 -- 172.21.15.26:0/1020669165 shutdown_connections 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.469+0000 7fe1fea95640 1 -- 172.21.15.26:0/1020669165 wait complete. 2023-12-19T18:17:46.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-19T18:17:46.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f0801811640 1 Processor -- start 2023-12-19T18:17:46.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f0801811640 1 -- start start 2023-12-19T18:17:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f0801811640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07fc15cc90 0x7f07fc0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f0801811640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07fc160a20 0x7f07fc0d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f0801811640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07fc0cdb20 con 0x7f07fc15d070 2023-12-19T18:17:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f0801811640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07fc0cde00 con 0x7f07fc160a20 2023-12-19T18:17:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f0801811640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc0ce0e0 con 0x7f07fc15cc90 2023-12-19T18:17:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f07fbfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f07fc15d070 0x7f07fc0cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34136/0 (socket says 172.21.15.26:34136) 2023-12-19T18:17:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f07fbfff640 1 -- 172.21.15.26:0/3422611283 learned_addr learned my addr 172.21.15.26:0/3422611283 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f0801010640 1 --2- 172.21.15.26:0/3422611283 >> v2:172.21.15.26:3300/0 conn(0x7f07fc160a20 0x7f07fc0d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f080080f640 1 --2- 172.21.15.26:0/3422611283 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07fc15cc90 0x7f07fc0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f0801010640 1 -- 172.21.15.26:0/3422611283 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07fc15cc90 msgr2=0x7f07fc0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f0801010640 1 --2- 172.21.15.26:0/3422611283 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07fc15cc90 0x7f07fc0cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f0801010640 1 -- 172.21.15.26:0/3422611283 >> v1:172.21.15.26:6789/0 conn(0x7f07fc15d070 legacy=0x7f07fc0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f0801010640 1 -- 172.21.15.26:0/3422611283 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07fc0ce3c0 con 0x7f07fc160a20 2023-12-19T18:17:46.646 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.641+0000 7f07fb7fe640 1 -- 172.21.15.26:0/3422611283 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57915333 0 0) 0x7f07fc0cdb20 con 0x7f07fc15d070 2023-12-19T18:17:46.648 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.645+0000 7f0801010640 1 --2- 172.21.15.26:0/3422611283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc160a20 0x7f07fc0d1760 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009d90 tx=0x7f07f401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d855b23ae2fd2473 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.645+0000 7f07fb7fe640 1 -- 172.21.15.26:0/3422611283 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f40a59c0 con 0x7f07fc160a20 2023-12-19T18:17:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.645+0000 7f07fb7fe640 1 -- 172.21.15.26:0/3422611283 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f40a5b20 con 0x7f07fc160a20 2023-12-19T18:17:46.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.645+0000 7f07fb7fe640 1 -- 172.21.15.26:0/3422611283 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f40ad440 con 0x7f07fc160a20 2023-12-19T18:17:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.645+0000 7f0801811640 1 -- 172.21.15.26:0/3422611283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc160a20 msgr2=0x7f07fc0d1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.645+0000 7f0801811640 1 --2- 172.21.15.26:0/3422611283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc160a20 0x7f07fc0d1760 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009d90 tx=0x7f07f401c920 comp rx=0 tx=0).stop 2023-12-19T18:17:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.645+0000 7f0801811640 1 -- 172.21.15.26:0/3422611283 shutdown_connections 2023-12-19T18:17:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.645+0000 7f0801811640 1 --2- 172.21.15.26:0/3422611283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc160a20 0x7f07fc0d1760 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.645+0000 7f0801811640 1 --2- 172.21.15.26:0/3422611283 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07fc15cc90 0x7f07fc0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.645+0000 7f0801811640 1 -- 172.21.15.26:0/3422611283 >> 172.21.15.26:0/3422611283 conn(0x7f07fc06a320 msgr2=0x7f07fc0658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.645+0000 7f0801811640 1 -- 172.21.15.26:0/3422611283 shutdown_connections 2023-12-19T18:17:46.650 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.645+0000 7f0801811640 1 -- 172.21.15.26:0/3422611283 wait complete. 2023-12-19T18:17:46.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f0801811640 1 Processor -- start 2023-12-19T18:17:46.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f0801811640 1 -- start start 2023-12-19T18:17:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f0801811640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07fc15cc90 0x7f07fc1f9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f0801811640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc15d070 0x7f07fc0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f0801811640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07fc160a20 0x7f07fc15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f0801811640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc0b23f0 con 0x7f07fc15d070 2023-12-19T18:17:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f0801811640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc1f9560 con 0x7f07fc160a20 2023-12-19T18:17:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f0801811640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc0b4b20 con 0x7f07fc15cc90 2023-12-19T18:17:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f07fbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc15d070 0x7f07fc0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f07fbfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc15d070 0x7f07fc0b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34202/0 (socket says 172.21.15.26:34202) 2023-12-19T18:17:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f07fbfff640 1 -- 172.21.15.26:0/2121974840 learned_addr learned my addr 172.21.15.26:0/2121974840 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f080080f640 1 --2- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07fc15cc90 0x7f07fc1f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f0801010640 1 --2- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07fc160a20 0x7f07fc15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f07fbfff640 1 -- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07fc15cc90 msgr2=0x7f07fc1f9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f07fbfff640 1 --2- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07fc15cc90 0x7f07fc1f9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f07fbfff640 1 -- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07fc160a20 msgr2=0x7f07fc15bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f07fbfff640 1 --2- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07fc160a20 0x7f07fc15bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f07fbfff640 1 -- 172.21.15.26:0/2121974840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f4009a70 con 0x7f07fc15d070 2023-12-19T18:17:46.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f07fbfff640 1 --2- 172.21.15.26:0/2121974840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc15d070 0x7f07fc0b4060 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f07e80011f0 tx=0x7f07e800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:46.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f07f97fa640 1 -- 172.21.15.26:0/2121974840 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e80074e0 con 0x7f07fc15d070 2023-12-19T18:17:46.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f07f97fa640 1 -- 172.21.15.26:0/2121974840 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e8007640 con 0x7f07fc15d070 2023-12-19T18:17:46.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f07f97fa640 1 -- 172.21.15.26:0/2121974840 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e8004a10 con 0x7f07fc15d070 2023-12-19T18:17:46.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f0801811640 1 -- 172.21.15.26:0/2121974840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07fc0b4e00 con 0x7f07fc15d070 2023-12-19T18:17:46.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f0801811640 1 -- 172.21.15.26:0/2121974840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc0b53c0 con 0x7f07fc15d070 2023-12-19T18:17:46.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.649+0000 7f0801811640 1 -- 172.21.15.26:0/2121974840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07fc061930 con 0x7f07fc15d070 2023-12-19T18:17:46.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.653+0000 7f07f97fa640 1 -- 172.21.15.26:0/2121974840 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 129) v1 ==== 85449+0+0 (secure 0 0 0) 0x7f07e8004b70 con 0x7f07fc15d070 2023-12-19T18:17:46.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.653+0000 7f07f97fa640 1 --2- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f07e0065cf0 0x7f07e00681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.653+0000 7f07f97fa640 1 -- 172.21.15.26:0/2121974840 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f07e8082f60 con 0x7f07fc15d070 2023-12-19T18:17:46.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.653+0000 7f080080f640 1 -- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f07e0065cf0 msgr2=0x7f07e00681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:46.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.653+0000 7f080080f640 1 --2- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f07e0065cf0 0x7f07e00681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:46.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.657+0000 7f07f97fa640 1 -- 172.21.15.26:0/2121974840 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f07e804ff70 con 0x7f07fc15d070 2023-12-19T18:17:46.705 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:46.704+0000 7f5950ba0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:17:46.739 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:46.737+0000 7f6745a76180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:17:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.821+0000 7f0801811640 1 -- 172.21.15.26:0/2121974840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f07fc0b1550 con 0x7f07fc15d070 2023-12-19T18:17:46.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.825+0000 7f07f97fa640 1 -- 172.21.15.26:0/2121974840 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v78)=0 v78) v1 ==== 133+0+0 (secure 0 0 0) 0x7f07e8018070 con 0x7f07fc15d070 2023-12-19T18:17:46.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.829+0000 7f0801811640 1 -- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f07e0065cf0 msgr2=0x7f07e00681b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:46.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.829+0000 7f0801811640 1 --2- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f07e0065cf0 0x7f07e00681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.831 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.829+0000 7f0801811640 1 -- 172.21.15.26:0/2121974840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc15d070 msgr2=0x7f07fc0b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.829+0000 7f0801811640 1 --2- 172.21.15.26:0/2121974840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc15d070 0x7f07fc0b4060 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f07e80011f0 tx=0x7f07e800c9b0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.829+0000 7f0801811640 1 -- 172.21.15.26:0/2121974840 shutdown_connections 2023-12-19T18:17:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.829+0000 7f0801811640 1 --2- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f07e0065cf0 0x7f07e00681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.829+0000 7f0801811640 1 --2- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07fc160a20 0x7f07fc15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.829+0000 7f0801811640 1 --2- 172.21.15.26:0/2121974840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07fc15d070 0x7f07fc0b4060 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.829+0000 7f0801811640 1 --2- 172.21.15.26:0/2121974840 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f07fc15cc90 0x7f07fc1f9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.829+0000 7f0801811640 1 -- 172.21.15.26:0/2121974840 >> 172.21.15.26:0/2121974840 conn(0x7f07fc06a320 msgr2=0x7f07fc1593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.829+0000 7f0801811640 1 -- 172.21.15.26:0/2121974840 shutdown_connections 2023-12-19T18:17:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:46.829+0000 7f0801811640 1 -- 172.21.15.26:0/2121974840 wait complete. 2023-12-19T18:17:46.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:46.852 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-12-19T18:17:46.857 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:46.856+0000 7f5950ba0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:17:46.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir 1M 2023-12-19T18:17:46.902 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:46.897+0000 7f6745a76180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:17:47.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.045+0000 7f6d4556d640 1 Processor -- start 2023-12-19T18:17:47.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d4556d640 1 -- start start 2023-12-19T18:17:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d4556d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d4015a9c0 0x7f6d400706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d4556d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d40070d90 con 0x7f6d4006ac80 2023-12-19T18:17:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d4556d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d40071070 con 0x7f6d4015ada0 2023-12-19T18:17:47.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d4556d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400cc270 con 0x7f6d4015a9c0 2023-12-19T18:17:47.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d3f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6d4006ac80 0x7f6d400b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34142/0 (socket says 172.21.15.26:34142) 2023-12-19T18:17:47.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d3f7fe640 1 -- 172.21.15.26:0/1546637744 learned_addr learned my addr 172.21.15.26:0/1546637744 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:47.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d3effd640 1 --2- 172.21.15.26:0/1546637744 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d4015a9c0 0x7f6d400706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:47.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d3e7fc640 1 -- 172.21.15.26:0/1546637744 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858760242 0 0) 0x7f6d40070d90 con 0x7f6d4006ac80 2023-12-19T18:17:47.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d3e7fc640 1 -- 172.21.15.26:0/1546637744 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d2c003530 con 0x7f6d4006ac80 2023-12-19T18:17:47.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d3e7fc640 1 -- 172.21.15.26:0/1546637744 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3738800004 0 0) 0x7f6d40071070 con 0x7f6d4015ada0 2023-12-19T18:17:47.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d3e7fc640 1 -- 172.21.15.26:0/1546637744 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d40070d90 con 0x7f6d4015ada0 2023-12-19T18:17:47.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d3effd640 1 -- 172.21.15.26:0/1546637744 >> v1:172.21.15.160:6789/0 conn(0x7f6d4015ada0 legacy=0x7f6d400b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:47.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d3effd640 1 -- 172.21.15.26:0/1546637744 >> v1:172.21.15.26:6789/0 conn(0x7f6d4006ac80 legacy=0x7f6d400b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:47.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.049+0000 7f6d3effd640 1 -- 172.21.15.26:0/1546637744 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d40071350 con 0x7f6d4015a9c0 2023-12-19T18:17:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.053+0000 7f6d3effd640 1 --2- 172.21.15.26:0/1546637744 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d4015a9c0 0x7f6d400706d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6d300097f0 tx=0x7f6d3001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82249c31da84cbd8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.053+0000 7f6d3e7fc640 1 -- 172.21.15.26:0/1546637744 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d300ab620 con 0x7f6d4015a9c0 2023-12-19T18:17:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.053+0000 7f6d3e7fc640 1 -- 172.21.15.26:0/1546637744 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d300ab780 con 0x7f6d4015a9c0 2023-12-19T18:17:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.053+0000 7f6d3e7fc640 1 -- 172.21.15.26:0/1546637744 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d300abb10 con 0x7f6d4015a9c0 2023-12-19T18:17:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.053+0000 7f6d4556d640 1 -- 172.21.15.26:0/1546637744 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d4015a9c0 msgr2=0x7f6d400706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.053+0000 7f6d4556d640 1 --2- 172.21.15.26:0/1546637744 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d4015a9c0 0x7f6d400706d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6d300097f0 tx=0x7f6d3001c400 comp rx=0 tx=0).stop 2023-12-19T18:17:47.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.053+0000 7f6d4556d640 1 -- 172.21.15.26:0/1546637744 shutdown_connections 2023-12-19T18:17:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.053+0000 7f6d4556d640 1 --2- 172.21.15.26:0/1546637744 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d4015a9c0 0x7f6d400706d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.053+0000 7f6d4556d640 1 -- 172.21.15.26:0/1546637744 >> 172.21.15.26:0/1546637744 conn(0x7f6d400ce730 msgr2=0x7f6d400ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.053+0000 7f6d4556d640 1 -- 172.21.15.26:0/1546637744 shutdown_connections 2023-12-19T18:17:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.053+0000 7f6d4556d640 1 -- 172.21.15.26:0/1546637744 wait complete. 2023-12-19T18:17:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.053+0000 7f6d4556d640 1 Processor -- start 2023-12-19T18:17:47.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d4556d640 1 -- start start 2023-12-19T18:17:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d4556d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d4006ac80 0x7f6d40154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d4556d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d4015a9c0 0x7f6d40148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d4556d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d4015ada0 0x7f6d40150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d4556d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400c4330 con 0x7f6d4015a9c0 2023-12-19T18:17:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d4556d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400c3a90 con 0x7f6d4015ada0 2023-12-19T18:17:47.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d4556d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400c3d70 con 0x7f6d4006ac80 2023-12-19T18:17:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d4015a9c0 0x7f6d40148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d4015a9c0 0x7f6d40148270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34204/0 (socket says 172.21.15.26:34204) 2023-12-19T18:17:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d37fff640 1 -- 172.21.15.26:0/4191171633 learned_addr learned my addr 172.21.15.26:0/4191171633 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d3effd640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d4006ac80 0x7f6d40154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d3f7fe640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d4015ada0 0x7f6d40150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:47.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d37fff640 1 -- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d4006ac80 msgr2=0x7f6d40154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d37fff640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6d4006ac80 0x7f6d40154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d37fff640 1 -- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d4015ada0 msgr2=0x7f6d40150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d37fff640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d4015ada0 0x7f6d40150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d37fff640 1 -- 172.21.15.26:0/4191171633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d30009420 con 0x7f6d4015a9c0 2023-12-19T18:17:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d37fff640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6d4015a9c0 0x7f6d40148270 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800c490 tx=0x7f6d2800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:47.067 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d377fe640 1 -- 172.21.15.26:0/4191171633 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d280052d0 con 0x7f6d4015a9c0 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d377fe640 1 -- 172.21.15.26:0/4191171633 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d28005430 con 0x7f6d4015a9c0 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d377fe640 1 -- 172.21.15.26:0/4191171633 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28012870 con 0x7f6d4015a9c0 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d4556d640 1 -- 172.21.15.26:0/4191171633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d400c4050 con 0x7f6d4015a9c0 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d4556d640 1 -- 172.21.15.26:0/4191171633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d401551c0 con 0x7f6d4015a9c0 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.057+0000 7f6d4556d640 1 -- 172.21.15.26:0/4191171633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d40146860 con 0x7f6d4015a9c0 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.061+0000 7f6d377fe640 1 -- 172.21.15.26:0/4191171633 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 129) v1 ==== 85449+0+0 (secure 0 0 0) 0x7f6d280129d0 con 0x7f6d4015a9c0 2023-12-19T18:17:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.061+0000 7f6d377fe640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 0x7f6d200681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.061+0000 7f6d377fe640 1 -- 172.21.15.26:0/4191171633 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7333+0+0 (secure 0 0 0) 0x7f6d28083720 con 0x7f6d4015a9c0 2023-12-19T18:17:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.061+0000 7f6d3effd640 1 -- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 msgr2=0x7f6d200681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:47.069 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.061+0000 7f6d3effd640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 0x7f6d200681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:47.070 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.065+0000 7f6d377fe640 1 -- 172.21.15.26:0/4191171633 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6d40146860 con 0x7f6d4015a9c0 2023-12-19T18:17:47.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.257+0000 7f6d4556d640 1 -- 172.21.15.26:0/4191171633 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d40168ad0 con 0x7f6d20065cf0 2023-12-19T18:17:47.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.265+0000 7f6d3effd640 1 -- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 msgr2=0x7f6d200681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:47.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.265+0000 7f6d3effd640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 0x7f6d200681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T18:17:47.338 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:47.336+0000 7f5950ba0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:17:47.395 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:47.392+0000 7f5950ba0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:17:47.436 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:47.433+0000 7f6745a76180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:17:47.453 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:47.452+0000 7f5950ba0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:17:47.498 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:47.493+0000 7f6745a76180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:17:47.538 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:47.536+0000 7f5950ba0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:17:47.556 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:47.553+0000 7f6745a76180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:17:47.594 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:47.592+0000 7f5950ba0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:17:47.649 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:47.645+0000 7f6745a76180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:17:47.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.669+0000 7f6d3effd640 1 -- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 msgr2=0x7f6d200681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:47.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:47.669+0000 7f6d3effd640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 0x7f6d200681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-19T18:17:47.678 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:47.676+0000 7f5950ba0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:17:47.707 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:47.701+0000 7f6745a76180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:17:47.734 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:47.732+0000 7f5950ba0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:17:47.795 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:47.793+0000 7f6745a76180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:17:47.857 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:47.853+0000 7f6745a76180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:17:47.881 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:47.880+0000 7f5950ba0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:17:47.939 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:47.936+0000 7f5950ba0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:17:48.009 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:48.005+0000 7f6745a76180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:17:48.070 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:48.065+0000 7f6745a76180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:17:48.178 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:17:48.179 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:17:48.179 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:17:48.187 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:48.184+0000 7f5950ba0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:17:48.321 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:17:48.321 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:17:48.321 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:17:48.322 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:48.320+0000 7f5950ba0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:17:48.329 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:48.325+0000 7f6745a76180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:17:48.445 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:48.444+0000 7f5950ba0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:17:48.471 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:48.469+0000 7f6745a76180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:17:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:48.473+0000 7f6d3effd640 1 -- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 msgr2=0x7f6d200681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:48.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:48.473+0000 7f6d3effd640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 0x7f6d200681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-19T18:17:48.515 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:48.512+0000 7f5950ba0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:17:48.603 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:48.601+0000 7f6745a76180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:17:48.650 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:48.648+0000 7f5950ba0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:17:48.679 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:48.673+0000 7f6745a76180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:17:48.822 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:48.817+0000 7f6745a76180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:17:49.189 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:49.188+0000 7f5950ba0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:17:49.272 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:49.272+0000 7f5950ba0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:17:49.394 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:49.389+0000 7f6745a76180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:17:49.420 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:49.420+0000 7f5950ba0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:17:49.482 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:49.477+0000 7f6745a76180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:17:49.502 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:49.500+0000 7f5950ba0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:17:49.639 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:49.633+0000 7f6745a76180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:17:49.727 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:49.725+0000 7f6745a76180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:17:49.816 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:49.816+0000 7f5950ba0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:17:49.876 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:49.876+0000 7f5950ba0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:17:49.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:49.885+0000 7f6d377fe640 1 -- 172.21.15.26:0/4191171633 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 130) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f6d28050b90 con 0x7f6d4015a9c0 2023-12-19T18:17:49.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:49.885+0000 7f6d377fe640 1 -- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 msgr2=0x7f6d200681b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:49.887 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:49.885+0000 7f6d377fe640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 0x7f6d200681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:50.044 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.040+0000 7f58d6312640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.044 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.040+0000 7f58d6312640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.044 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.040+0000 7f58d6312640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.044 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.040+0000 7f58d6312640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.044 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.040+0000 7f58d6312640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.044 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.044+0000 7f58d330c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.044 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.044+0000 7f58d330c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.044 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.044+0000 7f58d330c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.045 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.044+0000 7f58d330c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.045 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.044+0000 7f58d330c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.057 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.056+0000 7f58d6312640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.057 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.056+0000 7f58d6312640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.057 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.056+0000 7f58d6312640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.057 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.056+0000 7f58d6312640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.058 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.056+0000 7f58d6312640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.058 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.056+0000 7f58d330c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.058 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.056+0000 7f58d330c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.058 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.056+0000 7f58d330c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.058 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.056+0000 7f58d330c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.058 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.056+0000 7f58d330c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:50.083 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:50.077+0000 7f6745a76180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:17:50.148 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:50.145+0000 7f6745a76180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:17:50.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:50.889+0000 7f6d377fe640 1 -- 172.21.15.26:0/4191171633 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 131) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f6d28012c80 con 0x7f6d4015a9c0 2023-12-19T18:17:50.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:50.889+0000 7f6d377fe640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 0x7f6d200681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:50.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:50.889+0000 7f6d377fe640 1 -- 172.21.15.26:0/4191171633 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d2006c6c0 con 0x7f6d20065cf0 2023-12-19T18:17:50.895 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:50.893+0000 7f6d3effd640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 0x7f6d200681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:50.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:50.893+0000 7f6d3effd640 1 --2- 172.21.15.26:0/4191171633 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6d20065cf0 0x7f6d200681b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6d300af000 tx=0x7f6d300a7000 comp rx=0 tx=0).ready entity=mgr.11011 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:50.900 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:50.900+0000 7f58dd320640 -1 mgr.server reply reply (22) Invalid argument filesystem argument is required when there is more than one file system 2023-12-19T18:17:50.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:50.897+0000 7f6d377fe640 1 -- 172.21.15.26:0/4191171633 <== mgr.11011 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: -22 filesystem argument is required when there is more than one file system) v1 ==== 79+0+0 (secure 0 0 0) 0x7f6d2006c6c0 con 0x7f6d20065cf0 2023-12-19T18:17:50.901 INFO:teuthology.orchestra.run.smithi026.stderr:Error EINVAL: filesystem argument is required when there is more than one file system 2023-12-19T18:17:50.911 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-12-19T18:17:50.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:50.912 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-12-19T18:17:50.925 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-19T18:17:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.117+0000 7f568ed52640 1 Processor -- start 2023-12-19T18:17:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.117+0000 7f568ed52640 1 -- start start 2023-12-19T18:17:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f568ed52640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f56880ca820 0x7f568806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f568ed52640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f568806b850 0x7f56880ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f568ed52640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56880cc310 con 0x7f5688159090 2023-12-19T18:17:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f568ed52640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568806bce0 con 0x7f568806b850 2023-12-19T18:17:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f568ed52640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56880c9eb0 con 0x7f56880ca820 2023-12-19T18:17:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f568caee640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5688159090 0x7f56880c97a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34152/0 (socket says 172.21.15.26:34152) 2023-12-19T18:17:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f568caee640 1 -- 172.21.15.26:0/2136004782 learned_addr learned my addr 172.21.15.26:0/2136004782 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f568d2ef640 1 --2- 172.21.15.26:0/2136004782 >> v2:172.21.15.26:3300/0 conn(0x7f568806b850 0x7f56880ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f5687fff640 1 --2- 172.21.15.26:0/2136004782 >> v2:172.21.15.160:3300/0 conn(0x7f56880ca820 0x7f568806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f56877fe640 1 -- 172.21.15.26:0/2136004782 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802462310 0 0) 0x7f56880cc310 con 0x7f5688159090 2023-12-19T18:17:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f56877fe640 1 -- 172.21.15.26:0/2136004782 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5678003530 con 0x7f5688159090 2023-12-19T18:17:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f568d2ef640 1 -- 172.21.15.26:0/2136004782 >> v2:172.21.15.160:3300/0 conn(0x7f56880ca820 msgr2=0x7f568806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f568d2ef640 1 --2- 172.21.15.26:0/2136004782 >> v2:172.21.15.160:3300/0 conn(0x7f56880ca820 0x7f568806b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f568d2ef640 1 -- 172.21.15.26:0/2136004782 >> v1:172.21.15.26:6789/0 conn(0x7f5688159090 legacy=0x7f56880c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:51.126 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.121+0000 7f568d2ef640 1 -- 172.21.15.26:0/2136004782 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880ca110 con 0x7f568806b850 2023-12-19T18:17:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f568d2ef640 1 --2- 172.21.15.26:0/2136004782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568806b850 0x7f56880ccd60 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f56880b0f00 tx=0x7f568001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfc98e12340fbaa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f56877fe640 1 -- 172.21.15.26:0/2136004782 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56800a5920 con 0x7f568806b850 2023-12-19T18:17:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f56877fe640 1 -- 172.21.15.26:0/2136004782 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56800a5a80 con 0x7f568806b850 2023-12-19T18:17:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f56877fe640 1 -- 172.21.15.26:0/2136004782 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56800a5e10 con 0x7f568806b850 2023-12-19T18:17:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f568ed52640 1 -- 172.21.15.26:0/2136004782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568806b850 msgr2=0x7f56880ccd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f568ed52640 1 --2- 172.21.15.26:0/2136004782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568806b850 0x7f56880ccd60 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f56880b0f00 tx=0x7f568001c8e0 comp rx=0 tx=0).stop 2023-12-19T18:17:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f568ed52640 1 -- 172.21.15.26:0/2136004782 shutdown_connections 2023-12-19T18:17:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f568ed52640 1 --2- 172.21.15.26:0/2136004782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568806b850 0x7f56880ccd60 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f568ed52640 1 --2- 172.21.15.26:0/2136004782 >> v2:172.21.15.160:3300/0 conn(0x7f56880ca820 0x7f568806b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f568ed52640 1 -- 172.21.15.26:0/2136004782 >> 172.21.15.26:0/2136004782 conn(0x7f5688066160 msgr2=0x7f56880b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f568ed52640 1 -- 172.21.15.26:0/2136004782 shutdown_connections 2023-12-19T18:17:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f568ed52640 1 -- 172.21.15.26:0/2136004782 wait complete. 2023-12-19T18:17:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f568ed52640 1 Processor -- start 2023-12-19T18:17:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.125+0000 7f568ed52640 1 -- start start 2023-12-19T18:17:51.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568ed52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568806b850 0x7f5688154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:51.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568ed52640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56880ca820 0x7f56881540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:51.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568ed52640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5688159090 0x7f56880a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:51.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568ed52640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880c1e90 con 0x7f568806b850 2023-12-19T18:17:51.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568ed52640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880a9330 con 0x7f5688159090 2023-12-19T18:17:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568ed52640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56880ab930 con 0x7f56880ca820 2023-12-19T18:17:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568caee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568806b850 0x7f5688154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568caee640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568806b850 0x7f5688154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34236/0 (socket says 172.21.15.26:34236) 2023-12-19T18:17:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568caee640 1 -- 172.21.15.26:0/2532230858 learned_addr learned my addr 172.21.15.26:0/2532230858 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568d2ef640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5688159090 0x7f56880a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f5687fff640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56880ca820 0x7f56881540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568caee640 1 -- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56880ca820 msgr2=0x7f56881540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568caee640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56880ca820 0x7f56881540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568caee640 1 -- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5688159090 msgr2=0x7f56880a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568caee640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5688159090 0x7f56880a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568caee640 1 -- 172.21.15.26:0/2532230858 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5680009a70 con 0x7f568806b850 2023-12-19T18:17:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f5687fff640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56880ca820 0x7f56881540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568caee640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568806b850 0x7f5688154c80 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f567c00c3c0 tx=0x7f567c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f56857fa640 1 -- 172.21.15.26:0/2532230858 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c005cf0 con 0x7f568806b850 2023-12-19T18:17:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f56857fa640 1 -- 172.21.15.26:0/2532230858 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c005e50 con 0x7f568806b850 2023-12-19T18:17:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f56857fa640 1 -- 172.21.15.26:0/2532230858 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0072f0 con 0x7f568806b850 2023-12-19T18:17:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568ed52640 1 -- 172.21.15.26:0/2532230858 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5688158040 con 0x7f568806b850 2023-12-19T18:17:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568ed52640 1 -- 172.21.15.26:0/2532230858 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5688158600 con 0x7f568806b850 2023-12-19T18:17:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f568ed52640 1 -- 172.21.15.26:0/2532230858 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5688074270 con 0x7f568806b850 2023-12-19T18:17:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.129+0000 7f56857fa640 1 -- 172.21.15.26:0/2532230858 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 131) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f567c007450 con 0x7f568806b850 2023-12-19T18:17:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.133+0000 7f56857fa640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f566c065930 0x7f566c067df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.133+0000 7f56857fa640 1 -- 172.21.15.26:0/2532230858 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7763+0+0 (secure 0 0 0) 0x7f567c082bb0 con 0x7f568806b850 2023-12-19T18:17:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.133+0000 7f5687fff640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f566c065930 0x7f566c067df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:51.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.133+0000 7f5687fff640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f566c065930 0x7f566c067df0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f56880ae480 tx=0x7f5674008040 comp rx=0 tx=0).ready entity=mgr.11011 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:51.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.137+0000 7f56857fa640 1 -- 172.21.15.26:0/2532230858 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f567c0500a0 con 0x7f568806b850 2023-12-19T18:17:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:51.329+0000 7f568ed52640 1 -- 172.21.15.26:0/2532230858 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f56880cbe80 con 0x7f568806b850 2023-12-19T18:17:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.105+0000 7f56857fa640 1 -- 172.21.15.26:0/2532230858 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 132) v1 ==== 85433+0+0 (secure 0 0 0) 0x7f567c050470 con 0x7f568806b850 2023-12-19T18:17:52.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.105+0000 7f56857fa640 1 -- 172.21.15.26:0/2532230858 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v132) v1 ==== 93+0+0 (secure 0 0 0) 0x7f567c0500a0 con 0x7f568806b850 2023-12-19T18:17:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.109+0000 7f568ed52640 1 -- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f566c065930 msgr2=0x7f566c067df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.109+0000 7f568ed52640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f566c065930 0x7f566c067df0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f56880ae480 tx=0x7f5674008040 comp rx=0 tx=0).stop 2023-12-19T18:17:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.109+0000 7f568ed52640 1 -- 172.21.15.26:0/2532230858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568806b850 msgr2=0x7f5688154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.109+0000 7f568ed52640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568806b850 0x7f5688154c80 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f567c00c3c0 tx=0x7f567c00c880 comp rx=0 tx=0).stop 2023-12-19T18:17:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.109+0000 7f568ed52640 1 -- 172.21.15.26:0/2532230858 shutdown_connections 2023-12-19T18:17:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.109+0000 7f568ed52640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f566c065930 0x7f566c067df0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.109+0000 7f568ed52640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5688159090 0x7f56880a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.109+0000 7f568ed52640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f56880ca820 0x7f56881540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.109+0000 7f568ed52640 1 --2- 172.21.15.26:0/2532230858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f568806b850 0x7f5688154c80 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.109+0000 7f568ed52640 1 -- 172.21.15.26:0/2532230858 >> 172.21.15.26:0/2532230858 conn(0x7f5688066160 msgr2=0x7f5688155680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.109+0000 7f568ed52640 1 -- 172.21.15.26:0/2532230858 shutdown_connections 2023-12-19T18:17:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.109+0000 7f568ed52640 1 -- 172.21.15.26:0/2532230858 wait complete. 2023-12-19T18:17:52.136 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:17:52.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:17:52.213 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:17:52.270 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:17:52.271 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:17:52.271 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:52.281 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:17:52.282 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:52.288 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:52.288+0000 7f20f7e9b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:17:52.289 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:52.285+0000 7fdafdadc180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:17:52.298 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63f76d640 1 Processor -- start 2023-12-19T18:17:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63f76d640 1 -- start start 2023-12-19T18:17:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63f76d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb63815cd40 0x7fb6380ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63f76d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb638160900 0x7fb638160d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63f76d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6380d29d0 con 0x7fb6380d2bf0 2023-12-19T18:17:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63f76d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63815d680 con 0x7fb63815cd40 2023-12-19T18:17:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63f76d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb638161750 con 0x7fb638160900 2023-12-19T18:17:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63d509640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb63815cd40 0x7fb6380ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63dd0a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb6380d2bf0 0x7fb6380d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34168/0 (socket says 172.21.15.26:34168) 2023-12-19T18:17:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63dd0a640 1 -- 172.21.15.26:0/415675099 learned_addr learned my addr 172.21.15.26:0/415675099 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63d509640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb63815cd40 0x7fb6380ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34284/0 (socket says 172.21.15.26:34284) 2023-12-19T18:17:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63cd08640 1 --2- 172.21.15.26:0/415675099 >> v2:172.21.15.160:3300/0 conn(0x7fb638160900 0x7fb638160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63d509640 1 -- 172.21.15.26:0/415675099 >> v2:172.21.15.160:3300/0 conn(0x7fb638160900 msgr2=0x7fb638160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63d509640 1 --2- 172.21.15.26:0/415675099 >> v2:172.21.15.160:3300/0 conn(0x7fb638160900 0x7fb638160d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63d509640 1 -- 172.21.15.26:0/415675099 >> v1:172.21.15.26:6789/0 conn(0x7fb6380d2bf0 legacy=0x7fb6380d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63d509640 1 -- 172.21.15.26:0/415675099 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb638161470 con 0x7fb63815cd40 2023-12-19T18:17:52.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.481+0000 7fb63cd08640 1 --2- 172.21.15.26:0/415675099 >> v2:172.21.15.160:3300/0 conn(0x7fb638160900 0x7fb638160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.485+0000 7fb63d509640 1 --2- 172.21.15.26:0/415675099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63815cd40 0x7fb6380ce040 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fb6380d0bf0 tx=0x7fb63401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07d047ddd733d2e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.485+0000 7fb62ffff640 1 -- 172.21.15.26:0/415675099 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634004230 con 0x7fb63815cd40 2023-12-19T18:17:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.485+0000 7fb62ffff640 1 -- 172.21.15.26:0/415675099 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6340a7050 con 0x7fb63815cd40 2023-12-19T18:17:52.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.485+0000 7fb62ffff640 1 -- 172.21.15.26:0/415675099 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634003620 con 0x7fb63815cd40 2023-12-19T18:17:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.485+0000 7fb63f76d640 1 -- 172.21.15.26:0/415675099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63815cd40 msgr2=0x7fb6380ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.485+0000 7fb63f76d640 1 --2- 172.21.15.26:0/415675099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63815cd40 0x7fb6380ce040 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fb6380d0bf0 tx=0x7fb63401c440 comp rx=0 tx=0).stop 2023-12-19T18:17:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.485+0000 7fb63f76d640 1 -- 172.21.15.26:0/415675099 shutdown_connections 2023-12-19T18:17:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.485+0000 7fb63f76d640 1 --2- 172.21.15.26:0/415675099 >> v2:172.21.15.160:3300/0 conn(0x7fb638160900 0x7fb638160d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.485+0000 7fb63f76d640 1 --2- 172.21.15.26:0/415675099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63815cd40 0x7fb6380ce040 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.485+0000 7fb63f76d640 1 -- 172.21.15.26:0/415675099 >> 172.21.15.26:0/415675099 conn(0x7fb63806a120 msgr2=0x7fb63806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.485+0000 7fb63f76d640 1 -- 172.21.15.26:0/415675099 shutdown_connections 2023-12-19T18:17:52.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.485+0000 7fb63f76d640 1 -- 172.21.15.26:0/415675099 wait complete. 2023-12-19T18:17:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63f76d640 1 Processor -- start 2023-12-19T18:17:52.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63f76d640 1 -- start start 2023-12-19T18:17:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63f76d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6380d2bf0 0x7fb6380c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63f76d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63815cd40 0x7fb63815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63f76d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb638160900 0x7fb63815c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63f76d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb638162f10 con 0x7fb6380d2bf0 2023-12-19T18:17:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63f76d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb638143a30 con 0x7fb63815cd40 2023-12-19T18:17:52.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63f76d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb638142e20 con 0x7fb638160900 2023-12-19T18:17:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63d509640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6380d2bf0 0x7fb6380c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63d509640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6380d2bf0 0x7fb6380c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34288/0 (socket says 172.21.15.26:34288) 2023-12-19T18:17:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63d509640 1 -- 172.21.15.26:0/3429824925 learned_addr learned my addr 172.21.15.26:0/3429824925 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63cd08640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63815cd40 0x7fb63815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63dd0a640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb638160900 0x7fb63815c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63d509640 1 -- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb638160900 msgr2=0x7fb63815c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63d509640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb638160900 0x7fb63815c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63d509640 1 -- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63815cd40 msgr2=0x7fb63815b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63d509640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63815cd40 0x7fb63815b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63d509640 1 -- 172.21.15.26:0/3429824925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6340093b0 con 0x7fb6380d2bf0 2023-12-19T18:17:52.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63dd0a640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb638160900 0x7fb63815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63d509640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6380d2bf0 0x7fb6380c9d30 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fb634019050 tx=0x7fb6340191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb62dffb640 1 -- 172.21.15.26:0/3429824925 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634003c70 con 0x7fb6380d2bf0 2023-12-19T18:17:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb62dffb640 1 -- 172.21.15.26:0/3429824925 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb634003dd0 con 0x7fb6380d2bf0 2023-12-19T18:17:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb62dffb640 1 -- 172.21.15.26:0/3429824925 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634005bc0 con 0x7fb6380d2bf0 2023-12-19T18:17:52.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.489+0000 7fb63f76d640 1 -- 172.21.15.26:0/3429824925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb638143100 con 0x7fb6380d2bf0 2023-12-19T18:17:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.493+0000 7fb63f76d640 1 -- 172.21.15.26:0/3429824925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6381436c0 con 0x7fb6380d2bf0 2023-12-19T18:17:52.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.493+0000 7fb63f76d640 1 -- 172.21.15.26:0/3429824925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb638074270 con 0x7fb6380d2bf0 2023-12-19T18:17:52.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.493+0000 7fb62dffb640 1 -- 172.21.15.26:0/3429824925 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 132) v1 ==== 85433+0+0 (secure 0 0 0) 0x7fb6340b2040 con 0x7fb6380d2bf0 2023-12-19T18:17:52.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.497+0000 7fb62dffb640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065ca0 0x7fb61c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:52.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.497+0000 7fb62dffb640 1 -- 172.21.15.26:0/3429824925 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7763+0+0 (secure 0 0 0) 0x7fb63411dbd0 con 0x7fb6380d2bf0 2023-12-19T18:17:52.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.497+0000 7fb63cd08640 1 -- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065ca0 msgr2=0x7fb61c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:52.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.497+0000 7fb63cd08640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065ca0 0x7fb61c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:52.503 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.501+0000 7fb62dffb640 1 -- 172.21.15.26:0/3429824925 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb6340eaa80 con 0x7fb6380d2bf0 2023-12-19T18:17:52.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.661+0000 7fb63f76d640 1 -- 172.21.15.26:0/3429824925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb63815e880 con 0x7fb6380d2bf0 2023-12-19T18:17:52.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.661+0000 7fb62dffb640 1 -- 172.21.15.26:0/3429824925 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v238) v1 ==== 74+0+21548 (secure 0 0 0) 0x7fb6340c2070 con 0x7fb6380d2bf0 2023-12-19T18:17:52.667 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:52.667 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":238,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:49.880106+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:25.867829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:26.330514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:17:37.484211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:17:38.799276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1431434235":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/4035377594":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3900732845":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1388670559":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/332404059":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1786797637":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3396499768":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6833/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6832/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/359874050":"2023-12-20T18:17:49.880070+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:52.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.665+0000 7fb63f76d640 1 -- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065ca0 msgr2=0x7fb61c068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:52.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.665+0000 7fb63f76d640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065ca0 0x7fb61c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.665+0000 7fb63f76d640 1 -- 172.21.15.26:0/3429824925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6380d2bf0 msgr2=0x7fb6380c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:52.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.665+0000 7fb63f76d640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6380d2bf0 0x7fb6380c9d30 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7fb634019050 tx=0x7fb6340191e0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.665+0000 7fb63f76d640 1 -- 172.21.15.26:0/3429824925 shutdown_connections 2023-12-19T18:17:52.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.665+0000 7fb63f76d640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065ca0 0x7fb61c068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.665+0000 7fb63f76d640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb638160900 0x7fb63815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.665+0000 7fb63f76d640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63815cd40 0x7fb63815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.665+0000 7fb63f76d640 1 --2- 172.21.15.26:0/3429824925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6380d2bf0 0x7fb6380c9d30 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.665+0000 7fb63f76d640 1 -- 172.21.15.26:0/3429824925 >> 172.21.15.26:0/3429824925 conn(0x7fb63806a120 msgr2=0x7fb63815d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:52.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.665+0000 7fb63f76d640 1 -- 172.21.15.26:0/3429824925 shutdown_connections 2023-12-19T18:17:52.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.665+0000 7fb63f76d640 1 -- 172.21.15.26:0/3429824925 wait complete. 2023-12-19T18:17:52.690 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:17:25.867829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '232', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:17:26.330514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '233', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-19T18:17:37.484211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '236', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-19T18:17:38.799276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '237', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:17:52.691 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:17:52.704 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:52.704+0000 7f20f7e9b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:17:52.735 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:52.733+0000 7fdafdadc180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:17:52.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f201af6d640 1 Processor -- start 2023-12-19T18:17:52.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f201af6d640 1 -- start start 2023-12-19T18:17:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f201af6d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f201415cd20 0x7f20140b8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f201af6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20140b89c0 0x7f20140b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f201af6d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20140b5c00 con 0x7f20140b4c00 2023-12-19T18:17:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f201af6d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20140b50b0 con 0x7f201415cd20 2023-12-19T18:17:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f201af6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201415d440 con 0x7f20140b89c0 2023-12-19T18:17:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f2018d09640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f201415cd20 0x7f20140b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f2013fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20140b89c0 0x7f20140b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f201950a640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f20140b4c00 0x7f20140b1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46302/0 (socket says 172.21.15.26:46302) 2023-12-19T18:17:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f201950a640 1 -- 172.21.15.26:0/796385948 learned_addr learned my addr 172.21.15.26:0/796385948 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f20137fe640 1 -- 172.21.15.26:0/796385948 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124685182 0 0) 0x7f20140b5c00 con 0x7f20140b4c00 2023-12-19T18:17:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f20137fe640 1 -- 172.21.15.26:0/796385948 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2004003530 con 0x7f20140b4c00 2023-12-19T18:17:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f2018d09640 1 -- 172.21.15.26:0/796385948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20140b89c0 msgr2=0x7f20140b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:52.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f2018d09640 1 --2- 172.21.15.26:0/796385948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20140b89c0 0x7f20140b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f2018d09640 1 -- 172.21.15.26:0/796385948 >> v1:172.21.15.160:6789/0 conn(0x7f20140b4c00 legacy=0x7f20140b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:52.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f2018d09640 1 -- 172.21.15.26:0/796385948 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201415d6e0 con 0x7f201415cd20 2023-12-19T18:17:52.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.841+0000 7f2013fff640 1 --2- 172.21.15.26:0/796385948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20140b89c0 0x7f20140b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:52.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.845+0000 7f2018d09640 1 --2- 172.21.15.26:0/796385948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f201415cd20 0x7f20140b8480 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f20080097c0 tx=0x7f200801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2041f1ab11ee4a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:52.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.845+0000 7f20137fe640 1 -- 172.21.15.26:0/796385948 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080a8970 con 0x7f201415cd20 2023-12-19T18:17:52.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.845+0000 7f20137fe640 1 -- 172.21.15.26:0/796385948 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20080a8ad0 con 0x7f201415cd20 2023-12-19T18:17:52.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.845+0000 7f20137fe640 1 -- 172.21.15.26:0/796385948 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080a8e60 con 0x7f201415cd20 2023-12-19T18:17:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.845+0000 7f201af6d640 1 -- 172.21.15.26:0/796385948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f201415cd20 msgr2=0x7f20140b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.845+0000 7f201af6d640 1 --2- 172.21.15.26:0/796385948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f201415cd20 0x7f20140b8480 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f20080097c0 tx=0x7f200801ec70 comp rx=0 tx=0).stop 2023-12-19T18:17:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.845+0000 7f201af6d640 1 -- 172.21.15.26:0/796385948 shutdown_connections 2023-12-19T18:17:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.845+0000 7f201af6d640 1 --2- 172.21.15.26:0/796385948 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f20140b89c0 0x7f20140b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.845+0000 7f201af6d640 1 --2- 172.21.15.26:0/796385948 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f201415cd20 0x7f20140b8480 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.845+0000 7f201af6d640 1 -- 172.21.15.26:0/796385948 >> 172.21.15.26:0/796385948 conn(0x7f201406a120 msgr2=0x7f201406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.845+0000 7f201af6d640 1 -- 172.21.15.26:0/796385948 shutdown_connections 2023-12-19T18:17:52.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.845+0000 7f201af6d640 1 -- 172.21.15.26:0/796385948 wait complete. 2023-12-19T18:17:52.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f201af6d640 1 Processor -- start 2023-12-19T18:17:52.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f201af6d640 1 -- start start 2023-12-19T18:17:52.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f201af6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20140b4c00 0x7f201414b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:52.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f201af6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20140b89c0 0x7f20140c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:52.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f201af6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f201415cd20 0x7f20140cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:52.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f201af6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20140c8ca0 con 0x7f20140b4c00 2023-12-19T18:17:52.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f201af6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20140c8090 con 0x7f20140b89c0 2023-12-19T18:17:52.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f201af6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20140c8370 con 0x7f201415cd20 2023-12-19T18:17:52.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f2018d09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20140b4c00 0x7f201414b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:52.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f2018d09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20140b4c00 0x7f201414b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34298/0 (socket says 172.21.15.26:34298) 2023-12-19T18:17:52.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f2018d09640 1 -- 172.21.15.26:0/2784223384 learned_addr learned my addr 172.21.15.26:0/2784223384 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:52.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f201950a640 1 --2- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f201415cd20 0x7f20140cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:52.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f2013fff640 1 --2- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20140b89c0 0x7f20140c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:52.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f2018d09640 1 -- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f201415cd20 msgr2=0x7f20140cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:52.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f2018d09640 1 --2- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f201415cd20 0x7f20140cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f2018d09640 1 -- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20140b89c0 msgr2=0x7f20140c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:52.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f2018d09640 1 --2- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20140b89c0 0x7f20140c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:52.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f2018d09640 1 -- 172.21.15.26:0/2784223384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2000009420 con 0x7f20140b4c00 2023-12-19T18:17:52.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f2018d09640 1 --2- 172.21.15.26:0/2784223384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20140b4c00 0x7f201414b1f0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f200801b880 tx=0x7f200801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:52.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f20117fa640 1 -- 172.21.15.26:0/2784223384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080b1d10 con 0x7f20140b4c00 2023-12-19T18:17:52.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f20117fa640 1 -- 172.21.15.26:0/2784223384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20080b1e70 con 0x7f20140b4c00 2023-12-19T18:17:52.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f20117fa640 1 -- 172.21.15.26:0/2784223384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20080b9630 con 0x7f20140b4c00 2023-12-19T18:17:52.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.849+0000 7f201af6d640 1 -- 172.21.15.26:0/2784223384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2008009420 con 0x7f20140b4c00 2023-12-19T18:17:52.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.853+0000 7f201af6d640 1 -- 172.21.15.26:0/2784223384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20140c8930 con 0x7f20140b4c00 2023-12-19T18:17:52.857 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:52.856+0000 7f20f7e9b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:17:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.853+0000 7f201af6d640 1 -- 172.21.15.26:0/2784223384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20140c8e10 con 0x7f20140b4c00 2023-12-19T18:17:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.853+0000 7f20117fa640 1 -- 172.21.15.26:0/2784223384 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 132) v1 ==== 85433+0+0 (secure 0 0 0) 0x7f20080b7040 con 0x7f20140b4c00 2023-12-19T18:17:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.853+0000 7f20117fa640 1 --2- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ff8065ca0 0x7f1ff8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.853+0000 7f20117fa640 1 -- 172.21.15.26:0/2784223384 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7763+0+0 (secure 0 0 0) 0x7f2008121520 con 0x7f20140b4c00 2023-12-19T18:17:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.853+0000 7f2013fff640 1 -- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ff8065ca0 msgr2=0x7f1ff8068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:52.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.853+0000 7f2013fff640 1 --2- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ff8065ca0 0x7f1ff8068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:52.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:52.857+0000 7f20117fa640 1 -- 172.21.15.26:0/2784223384 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f20080ee380 con 0x7f20140b4c00 2023-12-19T18:17:52.899 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:52.893+0000 7fdafdadc180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:17:53.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.017+0000 7f201af6d640 1 -- 172.21.15.26:0/2784223384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2014074270 con 0x7f20140b4c00 2023-12-19T18:17:53.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.017+0000 7f20117fa640 1 -- 172.21.15.26:0/2784223384 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v238) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20080b5070 con 0x7f20140b4c00 2023-12-19T18:17:53.021 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:17:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.017+0000 7f201af6d640 1 -- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ff8065ca0 msgr2=0x7f1ff8068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.017+0000 7f201af6d640 1 --2- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ff8065ca0 0x7f1ff8068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.017+0000 7f201af6d640 1 -- 172.21.15.26:0/2784223384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20140b4c00 msgr2=0x7f201414b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.017+0000 7f201af6d640 1 --2- 172.21.15.26:0/2784223384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20140b4c00 0x7f201414b1f0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f200801b880 tx=0x7f200801ba10 comp rx=0 tx=0).stop 2023-12-19T18:17:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.021+0000 7f201af6d640 1 -- 172.21.15.26:0/2784223384 shutdown_connections 2023-12-19T18:17:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.021+0000 7f201af6d640 1 --2- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ff8065ca0 0x7f1ff8068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.021+0000 7f201af6d640 1 --2- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f201415cd20 0x7f20140cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.021+0000 7f201af6d640 1 --2- 172.21.15.26:0/2784223384 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20140b89c0 0x7f20140c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.021+0000 7f201af6d640 1 --2- 172.21.15.26:0/2784223384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20140b4c00 0x7f201414b1f0 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.021+0000 7f201af6d640 1 -- 172.21.15.26:0/2784223384 >> 172.21.15.26:0/2784223384 conn(0x7f201406a120 msgr2=0x7f20140d06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:53.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.021+0000 7f201af6d640 1 -- 172.21.15.26:0/2784223384 shutdown_connections 2023-12-19T18:17:53.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.021+0000 7f201af6d640 1 -- 172.21.15.26:0/2784223384 wait complete. 2023-12-19T18:17:53.043 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:17:53.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.229+0000 7fb569f39640 1 Processor -- start 2023-12-19T18:17:53.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.229+0000 7fb569f39640 1 -- start start 2023-12-19T18:17:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb569f39640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb5640ccf10 0x7fb5640ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb569f39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb5640b08c0 0x7fb56415d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.236 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb569f39640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5640ac860 con 0x7fb56415d250 2023-12-19T18:17:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb569f39640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb564157730 con 0x7fb5640ccf10 2023-12-19T18:17:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb569f39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5640ad7f0 con 0x7fb5640b08c0 2023-12-19T18:17:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb5637fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb5640ccf10 0x7fb5640ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb562ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb56415d250 0x7fb5640b01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34180/0 (socket says 172.21.15.26:34180) 2023-12-19T18:17:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb562ffd640 1 -- 172.21.15.26:0/2385234187 learned_addr learned my addr 172.21.15.26:0/2385234187 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb563fff640 1 --2- 172.21.15.26:0/2385234187 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb5640b08c0 0x7fb56415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb5637fe640 1 -- 172.21.15.26:0/2385234187 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb5640b08c0 msgr2=0x7fb56415d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:53.237 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb5637fe640 1 --2- 172.21.15.26:0/2385234187 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb5640b08c0 0x7fb56415d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb5637fe640 1 -- 172.21.15.26:0/2385234187 >> v1:172.21.15.26:6789/0 conn(0x7fb56415d250 legacy=0x7fb5640b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb5637fe640 1 -- 172.21.15.26:0/2385234187 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56415dd10 con 0x7fb5640ccf10 2023-12-19T18:17:53.238 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.233+0000 7fb5627fc640 1 -- 172.21.15.26:0/2385234187 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404566399 0 0) 0x7fb5640ac860 con 0x7fb56415d250 2023-12-19T18:17:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.237+0000 7fb5637fe640 1 --2- 172.21.15.26:0/2385234187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ccf10 0x7fb5640ad2b0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fb5540097c0 tx=0x7fb55401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc074914b8133 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.237+0000 7fb5627fc640 1 -- 172.21.15.26:0/2385234187 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5540a8970 con 0x7fb5640ccf10 2023-12-19T18:17:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.237+0000 7fb5627fc640 1 -- 172.21.15.26:0/2385234187 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5540a8ad0 con 0x7fb5640ccf10 2023-12-19T18:17:53.240 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.237+0000 7fb5627fc640 1 -- 172.21.15.26:0/2385234187 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5540a8e60 con 0x7fb5640ccf10 2023-12-19T18:17:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.237+0000 7fb569f39640 1 -- 172.21.15.26:0/2385234187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ccf10 msgr2=0x7fb5640ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.237+0000 7fb569f39640 1 --2- 172.21.15.26:0/2385234187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ccf10 0x7fb5640ad2b0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7fb5540097c0 tx=0x7fb55401ec70 comp rx=0 tx=0).stop 2023-12-19T18:17:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.237+0000 7fb569f39640 1 -- 172.21.15.26:0/2385234187 shutdown_connections 2023-12-19T18:17:53.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.237+0000 7fb569f39640 1 --2- 172.21.15.26:0/2385234187 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb5640b08c0 0x7fb56415d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.237+0000 7fb569f39640 1 --2- 172.21.15.26:0/2385234187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ccf10 0x7fb5640ad2b0 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.237+0000 7fb569f39640 1 -- 172.21.15.26:0/2385234187 >> 172.21.15.26:0/2385234187 conn(0x7fb5640ce730 msgr2=0x7fb5640ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.237+0000 7fb569f39640 1 -- 172.21.15.26:0/2385234187 shutdown_connections 2023-12-19T18:17:53.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.237+0000 7fb569f39640 1 -- 172.21.15.26:0/2385234187 wait complete. 2023-12-19T18:17:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb569f39640 1 Processor -- start 2023-12-19T18:17:53.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb569f39640 1 -- start start 2023-12-19T18:17:53.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb569f39640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5640b08c0 0x7fb5640c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb569f39640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ccf10 0x7fb564160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb569f39640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb56415d250 0x7fb564160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb569f39640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5640c5810 con 0x7fb5640ccf10 2023-12-19T18:17:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb569f39640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5640c5af0 con 0x7fb5640b08c0 2023-12-19T18:17:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb569f39640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5640c5dd0 con 0x7fb56415d250 2023-12-19T18:17:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb562ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ccf10 0x7fb564160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb562ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ccf10 0x7fb564160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34322/0 (socket says 172.21.15.26:34322) 2023-12-19T18:17:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb562ffd640 1 -- 172.21.15.26:0/3914196786 learned_addr learned my addr 172.21.15.26:0/3914196786 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb5637fe640 1 --2- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5640b08c0 0x7fb5640c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb563fff640 1 --2- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb56415d250 0x7fb564160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb562ffd640 1 -- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb56415d250 msgr2=0x7fb564160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb562ffd640 1 --2- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb56415d250 0x7fb564160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb562ffd640 1 -- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5640b08c0 msgr2=0x7fb5640c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb562ffd640 1 --2- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5640b08c0 0x7fb5640c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb562ffd640 1 -- 172.21.15.26:0/3914196786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb554009420 con 0x7fb5640ccf10 2023-12-19T18:17:53.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb562ffd640 1 --2- 172.21.15.26:0/3914196786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ccf10 0x7fb564160550 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7fb55800c480 tx=0x7fb55800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:53.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb54bfff640 1 -- 172.21.15.26:0/3914196786 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5580074e0 con 0x7fb5640ccf10 2023-12-19T18:17:53.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb54bfff640 1 -- 172.21.15.26:0/3914196786 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb558007660 con 0x7fb5640ccf10 2023-12-19T18:17:53.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb54bfff640 1 -- 172.21.15.26:0/3914196786 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb558004a30 con 0x7fb5640ccf10 2023-12-19T18:17:53.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb569f39640 1 -- 172.21.15.26:0/3914196786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5640c60b0 con 0x7fb5640ccf10 2023-12-19T18:17:53.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.241+0000 7fb569f39640 1 -- 172.21.15.26:0/3914196786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb564156770 con 0x7fb5640ccf10 2023-12-19T18:17:53.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.245+0000 7fb569f39640 1 -- 172.21.15.26:0/3914196786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb564074270 con 0x7fb5640ccf10 2023-12-19T18:17:53.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.245+0000 7fb54bfff640 1 -- 172.21.15.26:0/3914196786 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 132) v1 ==== 85433+0+0 (secure 0 0 0) 0x7fb558004b90 con 0x7fb5640ccf10 2023-12-19T18:17:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.245+0000 7fb54bfff640 1 --2- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb550065cd0 0x7fb550068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.245+0000 7fb54bfff640 1 -- 172.21.15.26:0/3914196786 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7763+0+0 (secure 0 0 0) 0x7fb558082e10 con 0x7fb5640ccf10 2023-12-19T18:17:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.245+0000 7fb5637fe640 1 -- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb550065cd0 msgr2=0x7fb550068190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:53.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.245+0000 7fb5637fe640 1 --2- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb550065cd0 0x7fb550068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:53.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.253+0000 7fb54bfff640 1 -- 172.21.15.26:0/3914196786 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb55804fda0 con 0x7fb5640ccf10 2023-12-19T18:17:53.340 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:53.340+0000 7f20f7e9b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:17:53.402 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:53.396+0000 7f20f7e9b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:17:53.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.413+0000 7fb569f39640 1 -- 172.21.15.26:0/3914196786 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5640c5950 con 0x7fb5640ccf10 2023-12-19T18:17:53.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.413+0000 7fb54bfff640 1 -- 172.21.15.26:0/3914196786 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v238) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb558018020 con 0x7fb5640ccf10 2023-12-19T18:17:53.417 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:17:53.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.417+0000 7fb569f39640 1 -- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb550065cd0 msgr2=0x7fb550068190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:53.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.417+0000 7fb569f39640 1 --2- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb550065cd0 0x7fb550068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.417+0000 7fb569f39640 1 -- 172.21.15.26:0/3914196786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ccf10 msgr2=0x7fb564160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.417+0000 7fb569f39640 1 --2- 172.21.15.26:0/3914196786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ccf10 0x7fb564160550 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7fb55800c480 tx=0x7fb55800c940 comp rx=0 tx=0).stop 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.417+0000 7fb569f39640 1 -- 172.21.15.26:0/3914196786 shutdown_connections 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.417+0000 7fb569f39640 1 --2- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb550065cd0 0x7fb550068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.417+0000 7fb569f39640 1 --2- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb56415d250 0x7fb564160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.417+0000 7fb569f39640 1 --2- 172.21.15.26:0/3914196786 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5640ccf10 0x7fb564160550 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.417+0000 7fb569f39640 1 --2- 172.21.15.26:0/3914196786 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5640b08c0 0x7fb5640c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.417+0000 7fb569f39640 1 -- 172.21.15.26:0/3914196786 >> 172.21.15.26:0/3914196786 conn(0x7fb5640ce730 msgr2=0x7fb5640ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.417+0000 7fb569f39640 1 -- 172.21.15.26:0/3914196786 shutdown_connections 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.417+0000 7fb569f39640 1 -- 172.21.15.26:0/3914196786 wait complete. 2023-12-19T18:17:53.421 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:53.417+0000 7fdafdadc180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:17:53.440 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:17:53.454 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:53.452+0000 7f20f7e9b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:17:53.484 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:53.481+0000 7fdafdadc180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:17:53.540 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:53.540+0000 7f20f7e9b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:17:53.544 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:53.541+0000 7fdafdadc180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:17:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.585+0000 7f122ae41640 1 Processor -- start 2023-12-19T18:17:53.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f122ae41640 1 -- start start 2023-12-19T18:17:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f122ae41640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1224159140 0x7f12240ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f122ae41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f122406b390 0x7f122406f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f122ae41640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122406f490 con 0x7f12240cd190 2023-12-19T18:17:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f122ae41640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1224159520 con 0x7f1224159140 2023-12-19T18:17:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f122ae41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12240cd690 con 0x7f122406b390 2023-12-19T18:17:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f1228bdd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1224159140 0x7f12240ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f12293de640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f122406b390 0x7f122406f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f1223fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f12240cd190 0x7f122406ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46316/0 (socket says 172.21.15.26:46316) 2023-12-19T18:17:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f1223fff640 1 -- 172.21.15.26:0/3081322416 learned_addr learned my addr 172.21.15.26:0/3081322416 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f12237fe640 1 -- 172.21.15.26:0/3081322416 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320044021 0 0) 0x7f122406f490 con 0x7f12240cd190 2023-12-19T18:17:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f12293de640 1 -- 172.21.15.26:0/3081322416 >> v2:172.21.15.160:3300/0 conn(0x7f1224159140 msgr2=0x7f12240ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f12293de640 1 --2- 172.21.15.26:0/3081322416 >> v2:172.21.15.160:3300/0 conn(0x7f1224159140 0x7f12240ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f12293de640 1 -- 172.21.15.26:0/3081322416 >> v1:172.21.15.160:6789/0 conn(0x7f12240cd190 legacy=0x7f122406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f12293de640 1 -- 172.21.15.26:0/3081322416 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12240cd970 con 0x7f122406b390 2023-12-19T18:17:53.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.589+0000 7f1228bdd640 1 --2- 172.21.15.26:0/3081322416 >> v2:172.21.15.160:3300/0 conn(0x7f1224159140 0x7f12240ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f12293de640 1 --2- 172.21.15.26:0/3081322416 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f122406b390 0x7f122406f920 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f121c009d90 tx=0x7f121c01c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3995276cf84aeadf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f12237fe640 1 -- 172.21.15.26:0/3081322416 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c0a5960 con 0x7f122406b390 2023-12-19T18:17:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f12237fe640 1 -- 172.21.15.26:0/3081322416 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121c0a5ac0 con 0x7f122406b390 2023-12-19T18:17:53.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f12237fe640 1 -- 172.21.15.26:0/3081322416 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c0a5e50 con 0x7f122406b390 2023-12-19T18:17:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f122ae41640 1 -- 172.21.15.26:0/3081322416 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f122406b390 msgr2=0x7f122406f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f122ae41640 1 --2- 172.21.15.26:0/3081322416 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f122406b390 0x7f122406f920 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f121c009d90 tx=0x7f121c01c920 comp rx=0 tx=0).stop 2023-12-19T18:17:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f122ae41640 1 -- 172.21.15.26:0/3081322416 shutdown_connections 2023-12-19T18:17:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f122ae41640 1 --2- 172.21.15.26:0/3081322416 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f122406b390 0x7f122406f920 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f122ae41640 1 --2- 172.21.15.26:0/3081322416 >> v2:172.21.15.160:3300/0 conn(0x7f1224159140 0x7f12240ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f122ae41640 1 -- 172.21.15.26:0/3081322416 >> 172.21.15.26:0/3081322416 conn(0x7f12240ce730 msgr2=0x7f12240ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f122ae41640 1 -- 172.21.15.26:0/3081322416 shutdown_connections 2023-12-19T18:17:53.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f122ae41640 1 -- 172.21.15.26:0/3081322416 wait complete. 2023-12-19T18:17:53.601 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:53.596+0000 7f20f7e9b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:17:53.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f122ae41640 1 Processor -- start 2023-12-19T18:17:53.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.593+0000 7f122ae41640 1 -- start start 2023-12-19T18:17:53.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f122ae41640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f122406b390 0x7f122414d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f122ae41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12240cd190 0x7f12240b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f122ae41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1224159140 0x7f1224157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f122ae41640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12240b1890 con 0x7f1224159140 2023-12-19T18:17:53.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f122ae41640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122414ecf0 con 0x7f122406b390 2023-12-19T18:17:53.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f122ae41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122414efd0 con 0x7f12240cd190 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f12293de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1224159140 0x7f1224157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f12293de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1224159140 0x7f1224157620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34328/0 (socket says 172.21.15.26:34328) 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f12293de640 1 -- 172.21.15.26:0/2899067921 learned_addr learned my addr 172.21.15.26:0/2899067921 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f1228bdd640 1 --2- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f122406b390 0x7f122414d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f1223fff640 1 --2- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12240cd190 0x7f12240b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f1228bdd640 1 -- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12240cd190 msgr2=0x7f12240b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f1228bdd640 1 --2- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12240cd190 0x7f12240b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f1228bdd640 1 -- 172.21.15.26:0/2899067921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1224159140 msgr2=0x7f1224157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f1228bdd640 1 --2- 172.21.15.26:0/2899067921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1224159140 0x7f1224157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f1228bdd640 1 -- 172.21.15.26:0/2899067921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121800a5c0 con 0x7f122406b390 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f1228bdd640 1 --2- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f122406b390 0x7f122414d180 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f121800e8e0 tx=0x7f121800eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f12217fa640 1 -- 172.21.15.26:0/2899067921 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121800cc40 con 0x7f122406b390 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f12217fa640 1 -- 172.21.15.26:0/2899067921 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121800cda0 con 0x7f122406b390 2023-12-19T18:17:53.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f122ae41640 1 -- 172.21.15.26:0/2899067921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121c009a70 con 0x7f122406b390 2023-12-19T18:17:53.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f12217fa640 1 -- 172.21.15.26:0/2899067921 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218004230 con 0x7f122406b390 2023-12-19T18:17:53.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f122ae41640 1 -- 172.21.15.26:0/2899067921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12241554b0 con 0x7f122406b390 2023-12-19T18:17:53.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.597+0000 7f122ae41640 1 -- 172.21.15.26:0/2899067921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122414f4f0 con 0x7f122406b390 2023-12-19T18:17:53.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.601+0000 7f12217fa640 1 -- 172.21.15.26:0/2899067921 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 132) v1 ==== 85433+0+0 (secure 0 0 0) 0x7f1218004390 con 0x7f122406b390 2023-12-19T18:17:53.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.601+0000 7f12217fa640 1 --2- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1208065ca0 0x7f1208068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.601+0000 7f12217fa640 1 -- 172.21.15.26:0/2899067921 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7763+0+0 (secure 0 0 0) 0x7f1218083b50 con 0x7f122406b390 2023-12-19T18:17:53.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.601+0000 7f1223fff640 1 -- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1208065ca0 msgr2=0x7f1208068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:53.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.601+0000 7f1223fff640 1 --2- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1208065ca0 0x7f1208068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:53.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.605+0000 7f12217fa640 1 -- 172.21.15.26:0/2899067921 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f122414f4f0 con 0x7f122406b390 2023-12-19T18:17:53.639 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:53.637+0000 7fdafdadc180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:17:53.685 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:53.684+0000 7f20f7e9b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:17:53.699 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:53.697+0000 7fdafdadc180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:17:53.742 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:53.740+0000 7f20f7e9b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:17:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.761+0000 7f122ae41640 1 -- 172.21.15.26:0/2899067921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f122414f4f0 con 0x7f122406b390 2023-12-19T18:17:53.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.765+0000 7f12217fa640 1 -- 172.21.15.26:0/2899067921 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v238) v1 ==== 100+0+11 (secure 0 0 0) 0x7f122414f4f0 con 0x7f122406b390 2023-12-19T18:17:53.767 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:17:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.765+0000 7f122ae41640 1 -- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1208065ca0 msgr2=0x7f1208068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.765+0000 7f122ae41640 1 --2- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1208065ca0 0x7f1208068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.765+0000 7f122ae41640 1 -- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f122406b390 msgr2=0x7f122414d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.765+0000 7f122ae41640 1 --2- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f122406b390 0x7f122414d180 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f121800e8e0 tx=0x7f121800eda0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.765+0000 7f122ae41640 1 -- 172.21.15.26:0/2899067921 shutdown_connections 2023-12-19T18:17:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.765+0000 7f122ae41640 1 --2- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1208065ca0 0x7f1208068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.765+0000 7f122ae41640 1 --2- 172.21.15.26:0/2899067921 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1224159140 0x7f1224157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.765+0000 7f122ae41640 1 --2- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12240cd190 0x7f12240b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.765+0000 7f122ae41640 1 --2- 172.21.15.26:0/2899067921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f122406b390 0x7f122414d180 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.765+0000 7f122ae41640 1 -- 172.21.15.26:0/2899067921 >> 172.21.15.26:0/2899067921 conn(0x7f12240ce730 msgr2=0x7f12240cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:53.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.765+0000 7f122ae41640 1 -- 172.21.15.26:0/2899067921 shutdown_connections 2023-12-19T18:17:53.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.765+0000 7f122ae41640 1 -- 172.21.15.26:0/2899067921 wait complete. 2023-12-19T18:17:53.800 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-19T18:17:53.800 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:53.797+0000 7fdafdadc180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:17:53.859 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:53.857+0000 7fdafdadc180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:17:53.887 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:53.884+0000 7f20f7e9b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:17:53.946 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:53.944+0000 7f20f7e9b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:17:53.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.985+0000 7fc6d8b18640 1 Processor -- start 2023-12-19T18:17:53.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.985+0000 7fc6d8b18640 1 -- start start 2023-12-19T18:17:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.985+0000 7fc6d8b18640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6d415d0d0 0x7fc6d40b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.985+0000 7fc6d8b18640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d415e200 con 0x7fc6d40b4600 2023-12-19T18:17:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.985+0000 7fc6d8b18640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d40d0aa0 con 0x7fc6d415ccf0 2023-12-19T18:17:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.985+0000 7fc6d8b18640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d415d680 con 0x7fc6d415d0d0 2023-12-19T18:17:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.989+0000 7fc6d2d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc6d40b4600 0x7fc6d4160c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34184/0 (socket says 172.21.15.26:34184) 2023-12-19T18:17:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.989+0000 7fc6d2d9d640 1 -- 172.21.15.26:0/1318113593 learned_addr learned my addr 172.21.15.26:0/1318113593 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:53.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.989+0000 7fc6d1d9b640 1 --2- 172.21.15.26:0/1318113593 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6d415d0d0 0x7fc6d40b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.989+0000 7fc6d159a640 1 -- 172.21.15.26:0/1318113593 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479642528 0 0) 0x7fc6d415e200 con 0x7fc6d40b4600 2023-12-19T18:17:53.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.989+0000 7fc6d159a640 1 -- 172.21.15.26:0/1318113593 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6bc003530 con 0x7fc6d40b4600 2023-12-19T18:17:53.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.989+0000 7fc6d159a640 1 -- 172.21.15.26:0/1318113593 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29402389 0 0) 0x7fc6d40d0aa0 con 0x7fc6d415ccf0 2023-12-19T18:17:53.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.989+0000 7fc6d1d9b640 1 -- 172.21.15.26:0/1318113593 >> v1:172.21.15.160:6789/0 conn(0x7fc6d415ccf0 legacy=0x7fc6d40d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.989+0000 7fc6d1d9b640 1 -- 172.21.15.26:0/1318113593 >> v1:172.21.15.26:6789/0 conn(0x7fc6d40b4600 legacy=0x7fc6d4160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.989+0000 7fc6d1d9b640 1 -- 172.21.15.26:0/1318113593 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d415d960 con 0x7fc6d415d0d0 2023-12-19T18:17:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.989+0000 7fc6d1d9b640 1 --2- 172.21.15.26:0/1318113593 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6d415d0d0 0x7fc6d40b8400 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80097f0 tx=0x7fc6c801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4afb05c0a0827ea7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.989+0000 7fc6d159a640 1 -- 172.21.15.26:0/1318113593 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c80aa5f0 con 0x7fc6d415d0d0 2023-12-19T18:17:53.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.989+0000 7fc6d159a640 1 -- 172.21.15.26:0/1318113593 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c80aa750 con 0x7fc6d415d0d0 2023-12-19T18:17:53.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.989+0000 7fc6d159a640 1 -- 172.21.15.26:0/1318113593 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c80aaae0 con 0x7fc6d415d0d0 2023-12-19T18:17:53.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 -- 172.21.15.26:0/1318113593 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6d415d0d0 msgr2=0x7fc6d40b8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 --2- 172.21.15.26:0/1318113593 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6d415d0d0 0x7fc6d40b8400 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc6c80097f0 tx=0x7fc6c801c490 comp rx=0 tx=0).stop 2023-12-19T18:17:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 -- 172.21.15.26:0/1318113593 shutdown_connections 2023-12-19T18:17:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 --2- 172.21.15.26:0/1318113593 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6d415d0d0 0x7fc6d40b8400 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 -- 172.21.15.26:0/1318113593 >> 172.21.15.26:0/1318113593 conn(0x7fc6d4065ee0 msgr2=0x7fc6d40cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 -- 172.21.15.26:0/1318113593 shutdown_connections 2023-12-19T18:17:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 -- 172.21.15.26:0/1318113593 wait complete. 2023-12-19T18:17:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 Processor -- start 2023-12-19T18:17:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 -- start start 2023-12-19T18:17:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6d40b4600 0x7fc6d41568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6d415ccf0 0x7fc6d415a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6d415d0d0 0x7fc6d40b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d4157860 con 0x7fc6d40b4600 2023-12-19T18:17:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d41572a0 con 0x7fc6d415d0d0 2023-12-19T18:17:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d8b18640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d4157580 con 0x7fc6d415ccf0 2023-12-19T18:17:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.993+0000 7fc6d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6d40b4600 0x7fc6d41568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:54.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6d40b4600 0x7fc6d41568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34330/0 (socket says 172.21.15.26:34330) 2023-12-19T18:17:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d259c640 1 -- 172.21.15.26:0/451757601 learned_addr learned my addr 172.21.15.26:0/451757601 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d1d9b640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6d415ccf0 0x7fc6d415a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d2d9d640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6d415d0d0 0x7fc6d40b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d259c640 1 -- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6d415ccf0 msgr2=0x7fc6d415a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d259c640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6d415ccf0 0x7fc6d415a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d259c640 1 -- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6d415d0d0 msgr2=0x7fc6d40b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d259c640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6d415d0d0 0x7fc6d40b6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d259c640 1 -- 172.21.15.26:0/451757601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c8009420 con 0x7fc6d40b4600 2023-12-19T18:17:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d1d9b640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6d415ccf0 0x7fc6d415a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d2d9d640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6d415d0d0 0x7fc6d40b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d259c640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6d40b4600 0x7fc6d41568e0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4001ec0 tx=0x7fc6c400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6c2ffd640 1 -- 172.21.15.26:0/451757601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c40054e0 con 0x7fc6d40b4600 2023-12-19T18:17:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6c2ffd640 1 -- 172.21.15.26:0/451757601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c4003600 con 0x7fc6d40b4600 2023-12-19T18:17:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d8b18640 1 -- 172.21.15.26:0/451757601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d4159f90 con 0x7fc6d40b4600 2023-12-19T18:17:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6c2ffd640 1 -- 172.21.15.26:0/451757601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c40039b0 con 0x7fc6d40b4600 2023-12-19T18:17:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d8b18640 1 -- 172.21.15.26:0/451757601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d415b330 con 0x7fc6d40b4600 2023-12-19T18:17:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6d8b18640 1 -- 172.21.15.26:0/451757601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d4074270 con 0x7fc6d40b4600 2023-12-19T18:17:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:53.997+0000 7fc6c2ffd640 1 -- 172.21.15.26:0/451757601 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 132) v1 ==== 85433+0+0 (secure 0 0 0) 0x7fc6c4003b10 con 0x7fc6d40b4600 2023-12-19T18:17:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.001+0000 7fc6c2ffd640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc6b4065ca0 0x7fc6b4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.001+0000 7fc6c2ffd640 1 -- 172.21.15.26:0/451757601 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7763+0+0 (secure 0 0 0) 0x7fc6c4083270 con 0x7fc6d40b4600 2023-12-19T18:17:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.001+0000 7fc6d1d9b640 1 -- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc6b4065ca0 msgr2=0x7fc6b4068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.001+0000 7fc6d1d9b640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc6b4065ca0 0x7fc6b4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:54.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.005+0000 7fc6c2ffd640 1 -- 172.21.15.26:0/451757601 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc6c4050200 con 0x7fc6d40b4600 2023-12-19T18:17:54.014 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:54.009+0000 7fdafdadc180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:17:54.076 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:54.073+0000 7fdafdadc180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:17:54.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.161+0000 7fc6d8b18640 1 -- 172.21.15.26:0/451757601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6d415a440 con 0x7fc6d40b4600 2023-12-19T18:17:54.167 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.161+0000 7fc6c2ffd640 1 -- 172.21.15.26:0/451757601 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v238) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc6c4017070 con 0x7fc6d40b4600 2023-12-19T18:17:54.167 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:17:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.165+0000 7fc6d8b18640 1 -- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc6b4065ca0 msgr2=0x7fc6b4068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.165+0000 7fc6d8b18640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc6b4065ca0 0x7fc6b4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.165+0000 7fc6d8b18640 1 -- 172.21.15.26:0/451757601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6d40b4600 msgr2=0x7fc6d41568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.165+0000 7fc6d8b18640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6d40b4600 0x7fc6d41568e0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4001ec0 tx=0x7fc6c400c8e0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.165+0000 7fc6d8b18640 1 -- 172.21.15.26:0/451757601 shutdown_connections 2023-12-19T18:17:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.165+0000 7fc6d8b18640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fc6b4065ca0 0x7fc6b4068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.165+0000 7fc6d8b18640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc6d415d0d0 0x7fc6d40b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.165+0000 7fc6d8b18640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fc6d415ccf0 0x7fc6d415a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.165+0000 7fc6d8b18640 1 --2- 172.21.15.26:0/451757601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6d40b4600 0x7fc6d41568e0 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.165+0000 7fc6d8b18640 1 -- 172.21.15.26:0/451757601 >> 172.21.15.26:0/451757601 conn(0x7fc6d4065ee0 msgr2=0x7fc6d40d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.165+0000 7fc6d8b18640 1 -- 172.21.15.26:0/451757601 shutdown_connections 2023-12-19T18:17:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.165+0000 7fc6d8b18640 1 -- 172.21.15.26:0/451757601 wait complete. 2023-12-19T18:17:54.189 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-19T18:17:54.189 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:17:54.190 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:17:54.190 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:17:54.197 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:54.196+0000 7f20f7e9b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:17:54.330 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:17:54.330 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:17:54.330 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:17:54.332 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:54.332+0000 7f20f7e9b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:17:54.340 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:54.337+0000 7fdafdadc180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:17:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.421+0000 7fe7cc2df640 1 Processor -- start 2023-12-19T18:17:54.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.421+0000 7fe7cc2df640 1 -- start start 2023-12-19T18:17:54.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.421+0000 7fe7cc2df640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe7c415ccf0 0x7fe7c40d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.421+0000 7fe7cc2df640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c415e200 con 0x7fe7c40b4600 2023-12-19T18:17:54.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.421+0000 7fe7cc2df640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c40d0aa0 con 0x7fe7c415d0d0 2023-12-19T18:17:54.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.421+0000 7fe7cc2df640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c40d1380 con 0x7fe7c415ccf0 2023-12-19T18:17:54.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.421+0000 7fe7ca87c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe7c40b4600 0x7fe7c40b4a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34186/0 (socket says 172.21.15.26:34186) 2023-12-19T18:17:54.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.421+0000 7fe7ca87c640 1 -- 172.21.15.26:0/1685531763 learned_addr learned my addr 172.21.15.26:0/1685531763 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:54.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.421+0000 7fe7ca07b640 1 --2- 172.21.15.26:0/1685531763 >> v2:172.21.15.160:3300/0 conn(0x7fe7c415ccf0 0x7fe7c40d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:54.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.425+0000 7fe7c9079640 1 -- 172.21.15.26:0/1685531763 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251209319 0 0) 0x7fe7c415e200 con 0x7fe7c40b4600 2023-12-19T18:17:54.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.425+0000 7fe7c9079640 1 -- 172.21.15.26:0/1685531763 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7b0003530 con 0x7fe7c40b4600 2023-12-19T18:17:54.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.425+0000 7fe7ca07b640 1 -- 172.21.15.26:0/1685531763 >> v1:172.21.15.160:6789/0 conn(0x7fe7c415d0d0 legacy=0x7fe7c40b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.425+0000 7fe7c9079640 1 -- 172.21.15.26:0/1685531763 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400457376 0 0) 0x7fe7c40d0aa0 con 0x7fe7c415d0d0 2023-12-19T18:17:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.425+0000 7fe7ca07b640 1 -- 172.21.15.26:0/1685531763 >> v1:172.21.15.26:6789/0 conn(0x7fe7c40b4600 legacy=0x7fe7c40b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.425+0000 7fe7ca07b640 1 -- 172.21.15.26:0/1685531763 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c415d680 con 0x7fe7c415ccf0 2023-12-19T18:17:54.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.425+0000 7fe7ca07b640 1 --2- 172.21.15.26:0/1685531763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c415ccf0 0x7fe7c40d0e40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00097c0 tx=0x7fe7c001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d90369597a05267 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:54.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.425+0000 7fe7c9079640 1 -- 172.21.15.26:0/1685531763 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00ab650 con 0x7fe7c415ccf0 2023-12-19T18:17:54.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.425+0000 7fe7c9079640 1 -- 172.21.15.26:0/1685531763 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c00ab7b0 con 0x7fe7c415ccf0 2023-12-19T18:17:54.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.425+0000 7fe7c9079640 1 -- 172.21.15.26:0/1685531763 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00abb40 con 0x7fe7c415ccf0 2023-12-19T18:17:54.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 -- 172.21.15.26:0/1685531763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c415ccf0 msgr2=0x7fe7c40d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 --2- 172.21.15.26:0/1685531763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c415ccf0 0x7fe7c40d0e40 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fe7c00097c0 tx=0x7fe7c001c450 comp rx=0 tx=0).stop 2023-12-19T18:17:54.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 -- 172.21.15.26:0/1685531763 shutdown_connections 2023-12-19T18:17:54.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 --2- 172.21.15.26:0/1685531763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c415ccf0 0x7fe7c40d0e40 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 -- 172.21.15.26:0/1685531763 >> 172.21.15.26:0/1685531763 conn(0x7fe7c4065ee0 msgr2=0x7fe7c40cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:54.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 -- 172.21.15.26:0/1685531763 shutdown_connections 2023-12-19T18:17:54.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 -- 172.21.15.26:0/1685531763 wait complete. 2023-12-19T18:17:54.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 Processor -- start 2023-12-19T18:17:54.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 -- start start 2023-12-19T18:17:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7c40b4600 0x7fe7c40c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c415ccf0 0x7fe7c4155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c415d0d0 0x7fe7c415ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c4157130 con 0x7fe7c415ccf0 2023-12-19T18:17:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c40ca450 con 0x7fe7c415d0d0 2023-12-19T18:17:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7cc2df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c41573d0 con 0x7fe7c40b4600 2023-12-19T18:17:54.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7c987a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c415ccf0 0x7fe7c4155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7c987a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c415ccf0 0x7fe7c4155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34344/0 (socket says 172.21.15.26:34344) 2023-12-19T18:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7c987a640 1 -- 172.21.15.26:0/2782116961 learned_addr learned my addr 172.21.15.26:0/2782116961 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7ca07b640 1 --2- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7c40b4600 0x7fe7c40c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.429+0000 7fe7ca87c640 1 --2- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c415d0d0 0x7fe7c415ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.433+0000 7fe7c987a640 1 -- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7c40b4600 msgr2=0x7fe7c40c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.433+0000 7fe7c987a640 1 --2- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7c40b4600 0x7fe7c40c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.433+0000 7fe7c987a640 1 -- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c415d0d0 msgr2=0x7fe7c415ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.433+0000 7fe7c987a640 1 --2- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c415d0d0 0x7fe7c415ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.433+0000 7fe7c987a640 1 -- 172.21.15.26:0/2782116961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c0009420 con 0x7fe7c415ccf0 2023-12-19T18:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.433+0000 7fe7c987a640 1 --2- 172.21.15.26:0/2782116961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c415ccf0 0x7fe7c4155980 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400c4d0 tx=0x7fe7b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.433+0000 7fe7baffd640 1 -- 172.21.15.26:0/2782116961 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4005480 con 0x7fe7c415ccf0 2023-12-19T18:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.433+0000 7fe7baffd640 1 -- 172.21.15.26:0/2782116961 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b4003600 con 0x7fe7c415ccf0 2023-12-19T18:17:54.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.433+0000 7fe7cc2df640 1 -- 172.21.15.26:0/2782116961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c41576b0 con 0x7fe7c415ccf0 2023-12-19T18:17:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.433+0000 7fe7baffd640 1 -- 172.21.15.26:0/2782116961 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40039b0 con 0x7fe7c415ccf0 2023-12-19T18:17:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.433+0000 7fe7cc2df640 1 -- 172.21.15.26:0/2782116961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c4157c70 con 0x7fe7c415ccf0 2023-12-19T18:17:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.433+0000 7fe7cc2df640 1 -- 172.21.15.26:0/2782116961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c4157e60 con 0x7fe7c415ccf0 2023-12-19T18:17:54.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.433+0000 7fe7baffd640 1 -- 172.21.15.26:0/2782116961 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 132) v1 ==== 85433+0+0 (secure 0 0 0) 0x7fe7b4003b10 con 0x7fe7c415ccf0 2023-12-19T18:17:54.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.437+0000 7fe7baffd640 1 --2- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe7a8065cd0 0x7fe7a8068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.437+0000 7fe7ca07b640 1 -- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe7a8065cd0 msgr2=0x7fe7a8068190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:54.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.437+0000 7fe7baffd640 1 -- 172.21.15.26:0/2782116961 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7763+0+0 (secure 0 0 0) 0x7fe7b4083290 con 0x7fe7c415ccf0 2023-12-19T18:17:54.439 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.437+0000 7fe7ca07b640 1 --2- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe7a8065cd0 0x7fe7a8068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:54.442 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.437+0000 7fe7baffd640 1 -- 172.21.15.26:0/2782116961 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe7b40502e0 con 0x7fe7c415ccf0 2023-12-19T18:17:54.459 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:54.456+0000 7f20f7e9b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:17:54.487 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:54.481+0000 7fdafdadc180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:17:54.530 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:54.528+0000 7f20f7e9b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:17:54.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.597+0000 7fe7cc2df640 1 -- 172.21.15.26:0/2782116961 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7c4074270 con 0x7fe7c415ccf0 2023-12-19T18:17:54.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.597+0000 7fe7baffd640 1 -- 172.21.15.26:0/2782116961 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v238) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe7b4017070 con 0x7fe7c415ccf0 2023-12-19T18:17:54.602 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:17:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.601+0000 7fe7cc2df640 1 -- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe7a8065cd0 msgr2=0x7fe7a8068190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.601+0000 7fe7cc2df640 1 --2- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe7a8065cd0 0x7fe7a8068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.601+0000 7fe7cc2df640 1 -- 172.21.15.26:0/2782116961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c415ccf0 msgr2=0x7fe7c4155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.601+0000 7fe7cc2df640 1 --2- 172.21.15.26:0/2782116961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c415ccf0 0x7fe7c4155980 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400c4d0 tx=0x7fe7b400c990 comp rx=0 tx=0).stop 2023-12-19T18:17:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.601+0000 7fe7cc2df640 1 -- 172.21.15.26:0/2782116961 shutdown_connections 2023-12-19T18:17:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.601+0000 7fe7cc2df640 1 --2- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe7a8065cd0 0x7fe7a8068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.601+0000 7fe7cc2df640 1 --2- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe7c415d0d0 0x7fe7c415ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.601+0000 7fe7cc2df640 1 --2- 172.21.15.26:0/2782116961 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7c415ccf0 0x7fe7c4155980 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.601+0000 7fe7cc2df640 1 --2- 172.21.15.26:0/2782116961 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe7c40b4600 0x7fe7c40c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.601+0000 7fe7cc2df640 1 -- 172.21.15.26:0/2782116961 >> 172.21.15.26:0/2782116961 conn(0x7fe7c4065ee0 msgr2=0x7fe7c40b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:54.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.601+0000 7fe7cc2df640 1 -- 172.21.15.26:0/2782116961 shutdown_connections 2023-12-19T18:17:54.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.601+0000 7fe7cc2df640 1 -- 172.21.15.26:0/2782116961 wait complete. 2023-12-19T18:17:54.619 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:54.617+0000 7fdafdadc180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:17:54.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:17:54.665 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:54.664+0000 7f20f7e9b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:17:54.695 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:54.693+0000 7fdafdadc180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:17:54.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f3935da2640 1 Processor -- start 2023-12-19T18:17:54.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f3935da2640 1 -- start start 2023-12-19T18:17:54.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f3935da2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f393015cca0 0x7f39300b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f3935da2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 0x7f39300cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f3935da2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39300b6fb0 con 0x7f39300b6b20 2023-12-19T18:17:54.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f3935da2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39300b7c30 con 0x7f393015cca0 2023-12-19T18:17:54.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f3935da2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39300b7290 con 0x7f39300b8760 2023-12-19T18:17:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f392ffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 0x7f39300cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f3934da0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f393015cca0 0x7f39300b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f392ffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 0x7f39300cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:55456/0 (socket says 172.21.15.26:55456) 2023-12-19T18:17:54.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f392ffff640 1 -- 172.21.15.26:0/3060343258 learned_addr learned my addr 172.21.15.26:0/3060343258 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f39355a1640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f39300b6b20 0x7f39300b4420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46340/0 (socket says 172.21.15.26:46340) 2023-12-19T18:17:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f392f7fe640 1 -- 172.21.15.26:0/3060343258 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307406738 0 0) 0x7f39300b6fb0 con 0x7f39300b6b20 2023-12-19T18:17:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f392f7fe640 1 -- 172.21.15.26:0/3060343258 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3920003530 con 0x7f39300b6b20 2023-12-19T18:17:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f392ffff640 1 -- 172.21.15.26:0/3060343258 >> v2:172.21.15.160:3300/0 conn(0x7f393015cca0 msgr2=0x7f39300b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f392ffff640 1 --2- 172.21.15.26:0/3060343258 >> v2:172.21.15.160:3300/0 conn(0x7f393015cca0 0x7f39300b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f392ffff640 1 -- 172.21.15.26:0/3060343258 >> v1:172.21.15.160:6789/0 conn(0x7f39300b6b20 legacy=0x7f39300b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f392ffff640 1 -- 172.21.15.26:0/3060343258 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39300b7570 con 0x7f39300b8760 2023-12-19T18:17:54.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.769+0000 7f3934da0640 1 --2- 172.21.15.26:0/3060343258 >> v2:172.21.15.160:3300/0 conn(0x7f393015cca0 0x7f39300b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.773+0000 7f392ffff640 1 --2- 172.21.15.26:0/3060343258 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 0x7f39300cd670 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f391c009870 tx=0x7f391c01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4174ba6fa7263c78 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.773+0000 7f392f7fe640 1 -- 172.21.15.26:0/3060343258 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c0a95f0 con 0x7f39300b8760 2023-12-19T18:17:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.773+0000 7f392f7fe640 1 -- 172.21.15.26:0/3060343258 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391c0a9750 con 0x7f39300b8760 2023-12-19T18:17:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.773+0000 7f392f7fe640 1 -- 172.21.15.26:0/3060343258 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c0a9ae0 con 0x7f39300b8760 2023-12-19T18:17:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.773+0000 7f3935da2640 1 -- 172.21.15.26:0/3060343258 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 msgr2=0x7f39300cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.773+0000 7f3935da2640 1 --2- 172.21.15.26:0/3060343258 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 0x7f39300cd670 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f391c009870 tx=0x7f391c01c510 comp rx=0 tx=0).stop 2023-12-19T18:17:54.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.773+0000 7f3935da2640 1 -- 172.21.15.26:0/3060343258 shutdown_connections 2023-12-19T18:17:54.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.773+0000 7f3935da2640 1 --2- 172.21.15.26:0/3060343258 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 0x7f39300cd670 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.773+0000 7f3935da2640 1 --2- 172.21.15.26:0/3060343258 >> v2:172.21.15.160:3300/0 conn(0x7f393015cca0 0x7f39300b65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.773+0000 7f3935da2640 1 -- 172.21.15.26:0/3060343258 >> 172.21.15.26:0/3060343258 conn(0x7f393006a120 msgr2=0x7f393006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:54.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.773+0000 7f3935da2640 1 -- 172.21.15.26:0/3060343258 shutdown_connections 2023-12-19T18:17:54.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.773+0000 7f3935da2640 1 -- 172.21.15.26:0/3060343258 wait complete. 2023-12-19T18:17:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3935da2640 1 Processor -- start 2023-12-19T18:17:54.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3935da2640 1 -- start start 2023-12-19T18:17:54.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3935da2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 0x7f3930155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3935da2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f393015cca0 0x7f393014b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3935da2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f393014d120 0x7f393014b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3935da2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39300cab90 con 0x7f393014d120 2023-12-19T18:17:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3935da2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39300cae30 con 0x7f393015cca0 2023-12-19T18:17:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3935da2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39300cb110 con 0x7f39300b8760 2023-12-19T18:17:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f39355a1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f393014d120 0x7f393014b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3934da0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 0x7f3930155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3934da0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 0x7f3930155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:55472/0 (socket says 172.21.15.26:55472) 2023-12-19T18:17:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f392ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f393015cca0 0x7f393014b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:54.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3934da0640 1 -- 172.21.15.26:0/2527221374 learned_addr learned my addr 172.21.15.26:0/2527221374 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:54.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3934da0640 1 -- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f393015cca0 msgr2=0x7f393014b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3934da0640 1 --2- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f393015cca0 0x7f393014b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3934da0640 1 -- 172.21.15.26:0/2527221374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f393014d120 msgr2=0x7f393014b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3934da0640 1 --2- 172.21.15.26:0/2527221374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f393014d120 0x7f393014b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3934da0640 1 -- 172.21.15.26:0/2527221374 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3924009420 con 0x7f39300b8760 2023-12-19T18:17:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f392ffff640 1 --2- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f393015cca0 0x7f393014b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3934da0640 1 --2- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 0x7f3930155db0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f392400e980 tx=0x7f392400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f392d7fa640 1 -- 172.21.15.26:0/2527221374 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392400ca00 con 0x7f39300b8760 2023-12-19T18:17:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f392d7fa640 1 -- 172.21.15.26:0/2527221374 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392400cb60 con 0x7f39300b8760 2023-12-19T18:17:54.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3935da2640 1 -- 172.21.15.26:0/2527221374 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391c0094d0 con 0x7f39300b8760 2023-12-19T18:17:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f392d7fa640 1 -- 172.21.15.26:0/2527221374 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924013470 con 0x7f39300b8760 2023-12-19T18:17:54.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.777+0000 7f3935da2640 1 -- 172.21.15.26:0/2527221374 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39300cb6d0 con 0x7f39300b8760 2023-12-19T18:17:54.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.781+0000 7f3935da2640 1 -- 172.21.15.26:0/2527221374 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3930067400 con 0x7f39300b8760 2023-12-19T18:17:54.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.781+0000 7f392d7fa640 1 -- 172.21.15.26:0/2527221374 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 132) v1 ==== 85433+0+0 (secure 0 0 0) 0x7f39240135d0 con 0x7f39300b8760 2023-12-19T18:17:54.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.781+0000 7f392d7fa640 1 --2- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3914065ca0 0x7f3914068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:54.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.781+0000 7f392d7fa640 1 -- 172.21.15.26:0/2527221374 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7763+0+0 (secure 0 0 0) 0x7f39240840f0 con 0x7f39300b8760 2023-12-19T18:17:54.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.781+0000 7f392ffff640 1 -- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3914065ca0 msgr2=0x7f3914068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:54.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.781+0000 7f392ffff640 1 --2- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3914065ca0 0x7f3914068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:54.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.785+0000 7f392d7fa640 1 -- 172.21.15.26:0/2527221374 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3924050ec0 con 0x7f39300b8760 2023-12-19T18:17:54.845 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:54.837+0000 7fdafdadc180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:17:54.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.957+0000 7f3935da2640 1 -- 172.21.15.26:0/2527221374 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f39300ceee0 con 0x7f39300b8760 2023-12-19T18:17:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.957+0000 7f392d7fa640 1 -- 172.21.15.26:0/2527221374 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v238) v1 ==== 81+0+584 (secure 0 0 0) 0x7f3924021260 con 0x7f39300b8760 2023-12-19T18:17:54.961 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1431434235 2023-12-20T18:17:49.880070+0000 2023-12-19T18:17:54.961 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/4035377594 2023-12-20T18:17:49.880070+0000 2023-12-19T18:17:54.961 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3900732845 2023-12-20T18:17:49.880070+0000 2023-12-19T18:17:54.961 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1388670559 2023-12-20T18:17:49.880070+0000 2023-12-19T18:17:54.962 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/332404059 2023-12-20T18:17:49.880070+0000 2023-12-19T18:17:54.962 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1786797637 2023-12-20T18:17:49.880070+0000 2023-12-19T18:17:54.962 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3396499768 2023-12-20T18:17:49.880070+0000 2023-12-19T18:17:54.962 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6833/48394 2023-12-20T18:17:49.880070+0000 2023-12-19T18:17:54.962 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6832/48394 2023-12-20T18:17:49.880070+0000 2023-12-19T18:17:54.962 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/359874050 2023-12-20T18:17:49.880070+0000 2023-12-19T18:17:54.962 INFO:teuthology.orchestra.run.smithi026.stderr:listed 10 entries 2023-12-19T18:17:54.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.961+0000 7f3935da2640 1 -- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3914065ca0 msgr2=0x7f3914068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.961+0000 7f3935da2640 1 --2- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3914065ca0 0x7f3914068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.961+0000 7f3935da2640 1 -- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 msgr2=0x7f3930155db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.961+0000 7f3935da2640 1 --2- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 0x7f3930155db0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f392400e980 tx=0x7f392400ee40 comp rx=0 tx=0).stop 2023-12-19T18:17:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.961+0000 7f3935da2640 1 -- 172.21.15.26:0/2527221374 shutdown_connections 2023-12-19T18:17:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.961+0000 7f3935da2640 1 --2- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3914065ca0 0x7f3914068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.961+0000 7f3935da2640 1 --2- 172.21.15.26:0/2527221374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f393014d120 0x7f393014b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.961+0000 7f3935da2640 1 --2- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f393015cca0 0x7f393014b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.961+0000 7f3935da2640 1 --2- 172.21.15.26:0/2527221374 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39300b8760 0x7f3930155db0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.961+0000 7f3935da2640 1 -- 172.21.15.26:0/2527221374 >> 172.21.15.26:0/2527221374 conn(0x7f393006a120 msgr2=0x7f39300c8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.961+0000 7f3935da2640 1 -- 172.21.15.26:0/2527221374 shutdown_connections 2023-12-19T18:17:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:54.961+0000 7f3935da2640 1 -- 172.21.15.26:0/2527221374 wait complete. 2023-12-19T18:17:54.983 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T18:17:54.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:54.984 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-19T18:17:55.011 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T18:17:55.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:55.011 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:17:55.064 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:17:55.064 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:55.074 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:17:55.074 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:17:55.074 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:55.116 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:17:55.116 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:17:55.116 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:55.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:17:55.127 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:55.212 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:55.212+0000 7f20f7e9b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:17:55.297 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:55.296+0000 7f20f7e9b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:17:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.301+0000 7ff6393e9640 1 Processor -- start 2023-12-19T18:17:55.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.301+0000 7ff6393e9640 1 -- start start 2023-12-19T18:17:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6393e9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff63415c7f0 0x7ff63406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:55.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6393e9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff6340adb80 0x7ff6340cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6393e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6340ccc20 0x7ff6340b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6393e9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63406d770 con 0x7ff6340adb80 2023-12-19T18:17:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6393e9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63406ced0 con 0x7ff63415c7f0 2023-12-19T18:17:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6393e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63406d1b0 con 0x7ff6340ccc20 2023-12-19T18:17:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6327fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff6340adb80 0x7ff6340cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6327fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff6340adb80 0x7ff6340cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34362/0 (socket says 172.21.15.26:34362) 2023-12-19T18:17:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6327fc640 1 -- 172.21.15.26:0/2943986925 learned_addr learned my addr 172.21.15.26:0/2943986925 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6337fe640 1 --2- 172.21.15.26:0/2943986925 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6340ccc20 0x7ff6340b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff632ffd640 1 --2- 172.21.15.26:0/2943986925 >> v2:172.21.15.160:3300/0 conn(0x7ff63415c7f0 0x7ff63406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6327fc640 1 -- 172.21.15.26:0/2943986925 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6340ccc20 msgr2=0x7ff6340b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6327fc640 1 --2- 172.21.15.26:0/2943986925 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6340ccc20 0x7ff6340b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6327fc640 1 -- 172.21.15.26:0/2943986925 >> v2:172.21.15.160:3300/0 conn(0x7ff63415c7f0 msgr2=0x7ff63406c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:55.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6327fc640 1 --2- 172.21.15.26:0/2943986925 >> v2:172.21.15.160:3300/0 conn(0x7ff63415c7f0 0x7ff63406c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.305+0000 7ff6327fc640 1 -- 172.21.15.26:0/2943986925 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63406d490 con 0x7ff6340adb80 2023-12-19T18:17:55.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff6327fc640 1 --2- 172.21.15.26:0/2943986925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6340adb80 0x7ff6340cc6e0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7ff6340670d0 tx=0x7ff62801d610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74c135349e4ce61 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:55.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff631ffb640 1 -- 172.21.15.26:0/2943986925 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280ae660 con 0x7ff6340adb80 2023-12-19T18:17:55.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff631ffb640 1 -- 172.21.15.26:0/2943986925 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6280ae7c0 con 0x7ff6340adb80 2023-12-19T18:17:55.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff631ffb640 1 -- 172.21.15.26:0/2943986925 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280aeb50 con 0x7ff6340adb80 2023-12-19T18:17:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff6393e9640 1 -- 172.21.15.26:0/2943986925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6340adb80 msgr2=0x7ff6340cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff6393e9640 1 --2- 172.21.15.26:0/2943986925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6340adb80 0x7ff6340cc6e0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7ff6340670d0 tx=0x7ff62801d610 comp rx=0 tx=0).stop 2023-12-19T18:17:55.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff6393e9640 1 -- 172.21.15.26:0/2943986925 shutdown_connections 2023-12-19T18:17:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff6393e9640 1 --2- 172.21.15.26:0/2943986925 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6340ccc20 0x7ff6340b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff6393e9640 1 --2- 172.21.15.26:0/2943986925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6340adb80 0x7ff6340cc6e0 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff6393e9640 1 --2- 172.21.15.26:0/2943986925 >> v2:172.21.15.160:3300/0 conn(0x7ff63415c7f0 0x7ff63406c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff6393e9640 1 -- 172.21.15.26:0/2943986925 >> 172.21.15.26:0/2943986925 conn(0x7ff6340658e0 msgr2=0x7ff634065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff6393e9640 1 -- 172.21.15.26:0/2943986925 shutdown_connections 2023-12-19T18:17:55.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff6393e9640 1 -- 172.21.15.26:0/2943986925 wait complete. 2023-12-19T18:17:55.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff6393e9640 1 Processor -- start 2023-12-19T18:17:55.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.309+0000 7ff6393e9640 1 -- start start 2023-12-19T18:17:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6393e9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6340adb80 0x7ff6340ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6393e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6340ccc20 0x7ff634157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6393e9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff63415c7f0 0x7ff6340afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6393e9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6341582e0 con 0x7ff63415c7f0 2023-12-19T18:17:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6393e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6340b0170 con 0x7ff6340ccc20 2023-12-19T18:17:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6393e9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6340ae870 con 0x7ff6340adb80 2023-12-19T18:17:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6340adb80 0x7ff6340ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6337fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff63415c7f0 0x7ff6340afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:55.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff632ffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6340adb80 0x7ff6340ae220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:55488/0 (socket says 172.21.15.26:55488) 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff632ffd640 1 -- 172.21.15.26:0/3583098565 learned_addr learned my addr 172.21.15.26:0/3583098565 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6327fc640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6340ccc20 0x7ff634157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6327fc640 1 -- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6340adb80 msgr2=0x7ff6340ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6327fc640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6340adb80 0x7ff6340ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6327fc640 1 -- 172.21.15.26:0/3583098565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff63415c7f0 msgr2=0x7ff6340afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6327fc640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff63415c7f0 0x7ff6340afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6327fc640 1 -- 172.21.15.26:0/3583098565 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280094d0 con 0x7ff6340ccc20 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff632ffd640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6340adb80 0x7ff6340ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6337fe640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff63415c7f0 0x7ff6340afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6327fc640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6340ccc20 0x7ff634157da0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff6280ad000 tx=0x7ff62801a3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff61b7fe640 1 -- 172.21.15.26:0/3583098565 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280b9680 con 0x7ff6340ccc20 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff61b7fe640 1 -- 172.21.15.26:0/3583098565 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6280b9800 con 0x7ff6340ccc20 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff61b7fe640 1 -- 172.21.15.26:0/3583098565 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280b9bb0 con 0x7ff6340ccc20 2023-12-19T18:17:55.318 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6393e9640 1 -- 172.21.15.26:0/3583098565 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6340aeb50 con 0x7ff6340ccc20 2023-12-19T18:17:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6393e9640 1 -- 172.21.15.26:0/3583098565 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6340af110 con 0x7ff6340ccc20 2023-12-19T18:17:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.313+0000 7ff6393e9640 1 -- 172.21.15.26:0/3583098565 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff634074270 con 0x7ff6340ccc20 2023-12-19T18:17:55.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.317+0000 7ff61b7fe640 1 -- 172.21.15.26:0/3583098565 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 132) v1 ==== 85433+0+0 (secure 0 0 0) 0x7ff6280b9d10 con 0x7ff6340ccc20 2023-12-19T18:17:55.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.317+0000 7ff61b7fe640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff620065ca0 0x7ff620068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:55.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.317+0000 7ff61b7fe640 1 -- 172.21.15.26:0/3583098565 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7763+0+0 (secure 0 0 0) 0x7ff62811f0f0 con 0x7ff6340ccc20 2023-12-19T18:17:55.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.317+0000 7ff632ffd640 1 -- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff620065ca0 msgr2=0x7ff620068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:55.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.317+0000 7ff632ffd640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff620065ca0 0x7ff620068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:55.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.317+0000 7ff61b7fe640 1 -- 172.21.15.26:0/3583098565 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff6280ec0f0 con 0x7ff6340ccc20 2023-12-19T18:17:55.431 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:55.429+0000 7fdafdadc180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:17:55.451 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:55.448+0000 7f20f7e9b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:17:55.516 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.513+0000 7ff6393e9640 1 -- 172.21.15.26:0/3583098565 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff634070fd0 con 0x7ff6340ccc20 2023-12-19T18:17:55.517 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.513+0000 7ff61b7fe640 1 -- 172.21.15.26:0/3583098565 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5921 (secure 0 0 0) 0x7ff6280b3020 con 0x7ff6340ccc20 2023-12-19T18:17:55.517 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:55.517 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2727392302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2727392302},{"type":"v1","addr":"172.21.15.160:6836","nonce":2727392302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1029020739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1029020739},{"type":"v1","addr":"172.21.15.160:6837","nonce":1029020739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:27.325271+0000","modified":"2023-12-19T18:17:30.337993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11427":{"gid":11427,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2285370499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2285370499},{"type":"v1","addr":"172.21.15.26:6835","nonce":2285370499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:39.801489+0000","modified":"2023-12-19T18:17:42.831660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11424":{"gid":11424,"name":"a","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6834/3251191056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3251191056},{"type":"v1","addr":"172.21.15.26:6834","nonce":3251191056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-19T18:17:55.517 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 118 2023-12-19T18:17:55.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.517+0000 7ff6393e9640 1 -- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff620065ca0 msgr2=0x7ff620068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:55.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.517+0000 7ff6393e9640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff620065ca0 0x7ff620068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.517+0000 7ff6393e9640 1 -- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6340ccc20 msgr2=0x7ff634157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:55.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.517+0000 7ff6393e9640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6340ccc20 0x7ff634157da0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff6280ad000 tx=0x7ff62801a3b0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.517+0000 7ff6393e9640 1 -- 172.21.15.26:0/3583098565 shutdown_connections 2023-12-19T18:17:55.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.517+0000 7ff6393e9640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff620065ca0 0x7ff620068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.517+0000 7ff6393e9640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff63415c7f0 0x7ff6340afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.517+0000 7ff6393e9640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff6340ccc20 0x7ff634157da0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.517+0000 7ff6393e9640 1 --2- 172.21.15.26:0/3583098565 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff6340adb80 0x7ff6340ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.517+0000 7ff6393e9640 1 -- 172.21.15.26:0/3583098565 >> 172.21.15.26:0/3583098565 conn(0x7ff6340658e0 msgr2=0x7ff634065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:55.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.517+0000 7ff6393e9640 1 -- 172.21.15.26:0/3583098565 shutdown_connections 2023-12-19T18:17:55.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.517+0000 7ff6393e9640 1 -- 172.21.15.26:0/3583098565 wait complete. 2023-12-19T18:17:55.521 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:55.517+0000 7fdafdadc180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:17:55.535 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:55.532+0000 7f20f7e9b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:17:55.539 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:55.681 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:55.677+0000 7fdafdadc180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:17:55.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.681+0000 7fad774f9640 1 Processor -- start 2023-12-19T18:17:55.685 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.681+0000 7fad774f9640 1 -- start start 2023-12-19T18:17:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.681+0000 7fad774f9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fad7015cd40 0x7fad700ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.681+0000 7fad774f9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad70160900 0x7fad70160d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.681+0000 7fad774f9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad700d29d0 con 0x7fad700d2bf0 2023-12-19T18:17:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.681+0000 7fad774f9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad7015d680 con 0x7fad70160900 2023-12-19T18:17:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.681+0000 7fad774f9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad70161750 con 0x7fad7015cd40 2023-12-19T18:17:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.681+0000 7fad74a94640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad70160900 0x7fad70160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:55.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.681+0000 7fad75a96640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fad700d2bf0 0x7fad700d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34190/0 (socket says 172.21.15.26:34190) 2023-12-19T18:17:55.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.681+0000 7fad75a96640 1 -- 172.21.15.26:0/1575382205 learned_addr learned my addr 172.21.15.26:0/1575382205 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:55.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.681+0000 7fad74a94640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad70160900 0x7fad70160d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34376/0 (socket says 172.21.15.26:34376) 2023-12-19T18:17:55.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.681+0000 7fad75295640 1 --2- 172.21.15.26:0/1575382205 >> v2:172.21.15.160:3300/0 conn(0x7fad7015cd40 0x7fad700ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:55.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.685+0000 7fad74a94640 1 -- 172.21.15.26:0/1575382205 >> v2:172.21.15.160:3300/0 conn(0x7fad7015cd40 msgr2=0x7fad700ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:55.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.685+0000 7fad74a94640 1 --2- 172.21.15.26:0/1575382205 >> v2:172.21.15.160:3300/0 conn(0x7fad7015cd40 0x7fad700ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.685+0000 7fad74a94640 1 -- 172.21.15.26:0/1575382205 >> v1:172.21.15.26:6789/0 conn(0x7fad700d2bf0 legacy=0x7fad700d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:55.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.685+0000 7fad74a94640 1 -- 172.21.15.26:0/1575382205 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad70161470 con 0x7fad70160900 2023-12-19T18:17:55.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.685+0000 7fad74a94640 1 --2- 172.21.15.26:0/1575382205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70160900 0x7fad70160d70 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fad700d3940 tx=0x7fad6001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64dbe831af976c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:55.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.685+0000 7fad67fff640 1 -- 172.21.15.26:0/1575382205 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600ab600 con 0x7fad70160900 2023-12-19T18:17:55.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.685+0000 7fad67fff640 1 -- 172.21.15.26:0/1575382205 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad600a7050 con 0x7fad70160900 2023-12-19T18:17:55.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.685+0000 7fad67fff640 1 -- 172.21.15.26:0/1575382205 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600ab980 con 0x7fad70160900 2023-12-19T18:17:55.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 -- 172.21.15.26:0/1575382205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70160900 msgr2=0x7fad70160d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:55.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 --2- 172.21.15.26:0/1575382205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70160900 0x7fad70160d70 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7fad700d3940 tx=0x7fad6001c460 comp rx=0 tx=0).stop 2023-12-19T18:17:55.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 -- 172.21.15.26:0/1575382205 shutdown_connections 2023-12-19T18:17:55.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 --2- 172.21.15.26:0/1575382205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70160900 0x7fad70160d70 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 --2- 172.21.15.26:0/1575382205 >> v2:172.21.15.160:3300/0 conn(0x7fad7015cd40 0x7fad700ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 -- 172.21.15.26:0/1575382205 >> 172.21.15.26:0/1575382205 conn(0x7fad7006a120 msgr2=0x7fad7006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:55.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 -- 172.21.15.26:0/1575382205 shutdown_connections 2023-12-19T18:17:55.692 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 -- 172.21.15.26:0/1575382205 wait complete. 2023-12-19T18:17:55.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 Processor -- start 2023-12-19T18:17:55.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 -- start start 2023-12-19T18:17:55.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad700d2bf0 0x7fad700cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:55.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad7015cd40 0x7fad70157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:55.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70160900 0x7fad700c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:55.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad700cc920 con 0x7fad70160900 2023-12-19T18:17:55.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad70155f00 con 0x7fad7015cd40 2023-12-19T18:17:55.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad774f9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad700ccb70 con 0x7fad700d2bf0 2023-12-19T18:17:55.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad75a96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70160900 0x7fad700c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:55.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad74a94640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad7015cd40 0x7fad70157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:55.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad75295640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad700d2bf0 0x7fad700cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:55.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad74a94640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad7015cd40 0x7fad70157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:35410/0 (socket says 172.21.15.26:35410) 2023-12-19T18:17:55.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad75a96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70160900 0x7fad700c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34382/0 (socket says 172.21.15.26:34382) 2023-12-19T18:17:55.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.689+0000 7fad74a94640 1 -- 172.21.15.26:0/2840113536 learned_addr learned my addr 172.21.15.26:0/2840113536 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:55.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.693+0000 7fad74a94640 1 -- 172.21.15.26:0/2840113536 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad700d2bf0 msgr2=0x7fad700cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:55.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.693+0000 7fad74a94640 1 --2- 172.21.15.26:0/2840113536 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad700d2bf0 0x7fad700cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.693+0000 7fad74a94640 1 -- 172.21.15.26:0/2840113536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70160900 msgr2=0x7fad700c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:55.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.693+0000 7fad74a94640 1 --2- 172.21.15.26:0/2840113536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70160900 0x7fad700c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.693+0000 7fad74a94640 1 -- 172.21.15.26:0/2840113536 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad60009420 con 0x7fad7015cd40 2023-12-19T18:17:55.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.693+0000 7fad74a94640 1 --2- 172.21.15.26:0/2840113536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad7015cd40 0x7fad70157c50 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fad600ae040 tx=0x7fad60019200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:55.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.693+0000 7fad65ffb640 1 -- 172.21.15.26:0/2840113536 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600b4430 con 0x7fad7015cd40 2023-12-19T18:17:55.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.693+0000 7fad65ffb640 1 -- 172.21.15.26:0/2840113536 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad600b45b0 con 0x7fad7015cd40 2023-12-19T18:17:55.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.693+0000 7fad65ffb640 1 -- 172.21.15.26:0/2840113536 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad600b4960 con 0x7fad7015cd40 2023-12-19T18:17:55.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.693+0000 7fad774f9640 1 -- 172.21.15.26:0/2840113536 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad700c6830 con 0x7fad7015cd40 2023-12-19T18:17:55.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.693+0000 7fad774f9640 1 -- 172.21.15.26:0/2840113536 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad700c6df0 con 0x7fad7015cd40 2023-12-19T18:17:55.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.693+0000 7fad774f9640 1 -- 172.21.15.26:0/2840113536 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad70074270 con 0x7fad7015cd40 2023-12-19T18:17:55.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.693+0000 7fad65ffb640 1 -- 172.21.15.26:0/2840113536 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 132) v1 ==== 85433+0+0 (secure 0 0 0) 0x7fad600b2040 con 0x7fad7015cd40 2023-12-19T18:17:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.697+0000 7fad65ffb640 1 --2- 172.21.15.26:0/2840113536 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fad54065ca0 0x7fad54068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.697+0000 7fad65ffb640 1 -- 172.21.15.26:0/2840113536 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7763+0+0 (secure 0 0 0) 0x7fad6011da90 con 0x7fad7015cd40 2023-12-19T18:17:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.697+0000 7fad75295640 1 -- 172.21.15.26:0/2840113536 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fad54065ca0 msgr2=0x7fad54068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:17:55.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.697+0000 7fad75295640 1 --2- 172.21.15.26:0/2840113536 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fad54065ca0 0x7fad54068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:17:55.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.697+0000 7fad65ffb640 1 -- 172.21.15.26:0/2840113536 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fad600ea8b0 con 0x7fad7015cd40 2023-12-19T18:17:55.772 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:55.769+0000 7fdafdadc180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:17:55.859 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:55.856+0000 7f20f7e9b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:17:55.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.865+0000 7fad774f9640 1 -- 172.21.15.26:0/2840113536 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad700b85f0 con 0x7fad7015cd40 2023-12-19T18:17:55.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.865+0000 7fad65ffb640 1 -- 172.21.15.26:0/2840113536 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v238) v1 ==== 74+0+21548 (secure 0 0 0) 0x7fad600eded0 con 0x7fad7015cd40 2023-12-19T18:17:55.871 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:55.871 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":238,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:49.880106+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:25.867829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:26.330514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:17:37.484211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:17:38.799276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1431434235":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/4035377594":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3900732845":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1388670559":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/332404059":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1786797637":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3396499768":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6833/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6832/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/359874050":"2023-12-20T18:17:49.880070+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.869+0000 7fad774f9640 1 -- 172.21.15.26:0/2840113536 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fad54065ca0 msgr2=0x7fad54068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:17:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.869+0000 7fad774f9640 1 --2- 172.21.15.26:0/2840113536 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fad54065ca0 0x7fad54068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.869+0000 7fad774f9640 1 -- 172.21.15.26:0/2840113536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad7015cd40 msgr2=0x7fad70157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.869+0000 7fad774f9640 1 --2- 172.21.15.26:0/2840113536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad7015cd40 0x7fad70157c50 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fad600ae040 tx=0x7fad60019200 comp rx=0 tx=0).stop 2023-12-19T18:17:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.869+0000 7fad774f9640 1 -- 172.21.15.26:0/2840113536 shutdown_connections 2023-12-19T18:17:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.869+0000 7fad774f9640 1 --2- 172.21.15.26:0/2840113536 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fad54065ca0 0x7fad54068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.869+0000 7fad774f9640 1 --2- 172.21.15.26:0/2840113536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad70160900 0x7fad700c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.869+0000 7fad774f9640 1 --2- 172.21.15.26:0/2840113536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fad7015cd40 0x7fad70157c50 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.869+0000 7fad774f9640 1 --2- 172.21.15.26:0/2840113536 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fad700d2bf0 0x7fad700cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:55.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.869+0000 7fad774f9640 1 -- 172.21.15.26:0/2840113536 >> 172.21.15.26:0/2840113536 conn(0x7fad7006a120 msgr2=0x7fad7015d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:55.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.869+0000 7fad774f9640 1 -- 172.21.15.26:0/2840113536 shutdown_connections 2023-12-19T18:17:55.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:55.869+0000 7fad774f9640 1 -- 172.21.15.26:0/2840113536 wait complete. 2023-12-19T18:17:55.893 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:17:25.867829+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '232', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:17:26.330514+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '233', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-19T18:17:37.484211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '236', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-19T18:17:38.799276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '237', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:17:55.894 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:17:55.920 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:55.920+0000 7f20f7e9b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:17:56.001 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.000+0000 7f207fd20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.001 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.000+0000 7f207fd20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.002 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.000+0000 7f207fd20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.002 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.000+0000 7f207fd20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.002 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.000+0000 7f207fd20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.002 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.000+0000 7f2083527640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.002 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.000+0000 7f2083527640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.002 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.000+0000 7f2083527640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.002 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.000+0000 7f2083527640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.002 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.000+0000 7f2083527640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.013 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.012+0000 7f2083527640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.013 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.012+0000 7f2083527640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.013 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.012+0000 7f2083527640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.013 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.012+0000 7f2083527640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.013 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.012+0000 7f2083527640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.014 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.012+0000 7f207fd20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.014 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.012+0000 7f207fd20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.014 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.012+0000 7f207fd20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.014 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.012+0000 7f207fd20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.014 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:17:56.012+0000 7f207fd20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:17:56.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa181e66640 1 Processor -- start 2023-12-19T18:17:56.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa181e66640 1 -- start start 2023-12-19T18:17:56.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa181e66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa17c15ccf0 0x7fa17c0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa181e66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa17c0b4600 0x7fa17c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa181e66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17c0b4f50 con 0x7fa17c15d0d0 2023-12-19T18:17:56.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa181e66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c15e200 con 0x7fa17c15ccf0 2023-12-19T18:17:56.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa181e66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c0d0aa0 con 0x7fa17c0b4600 2023-12-19T18:17:56.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa17b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa17c15ccf0 0x7fa17c0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa17b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa17c15ccf0 0x7fa17c0d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41852/0 (socket says 172.21.15.26:41852) 2023-12-19T18:17:56.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa17b7fe640 1 -- 172.21.15.26:0/2485641792 learned_addr learned my addr 172.21.15.26:0/2485641792 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:56.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa17bfff640 1 --2- 172.21.15.26:0/2485641792 >> v2:172.21.15.160:3300/0 conn(0x7fa17c0b4600 0x7fa17c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa17b7fe640 1 -- 172.21.15.26:0/2485641792 >> v2:172.21.15.160:3300/0 conn(0x7fa17c0b4600 msgr2=0x7fa17c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:56.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa17b7fe640 1 --2- 172.21.15.26:0/2485641792 >> v2:172.21.15.160:3300/0 conn(0x7fa17c0b4600 0x7fa17c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa17b7fe640 1 -- 172.21.15.26:0/2485641792 >> v1:172.21.15.26:6789/0 conn(0x7fa17c15d0d0 legacy=0x7fa17c0b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.049+0000 7fa17b7fe640 1 -- 172.21.15.26:0/2485641792 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c0d1380 con 0x7fa17c15ccf0 2023-12-19T18:17:56.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.053+0000 7fa17b7fe640 1 --2- 172.21.15.26:0/2485641792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa17c15ccf0 0x7fa17c0d0e40 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0d1570 tx=0x7fa16c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3790d8f7aee65bfc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:56.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.053+0000 7fa17a7fc640 1 -- 172.21.15.26:0/2485641792 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0aa430 con 0x7fa17c15ccf0 2023-12-19T18:17:56.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.053+0000 7fa17a7fc640 1 -- 172.21.15.26:0/2485641792 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16c0aa590 con 0x7fa17c15ccf0 2023-12-19T18:17:56.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.053+0000 7fa17a7fc640 1 -- 172.21.15.26:0/2485641792 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0aa920 con 0x7fa17c15ccf0 2023-12-19T18:17:56.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.053+0000 7fa181e66640 1 -- 172.21.15.26:0/2485641792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa17c15ccf0 msgr2=0x7fa17c0d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.053+0000 7fa181e66640 1 --2- 172.21.15.26:0/2485641792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa17c15ccf0 0x7fa17c0d0e40 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0d1570 tx=0x7fa16c01c4d0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.053+0000 7fa181e66640 1 -- 172.21.15.26:0/2485641792 shutdown_connections 2023-12-19T18:17:56.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.053+0000 7fa181e66640 1 --2- 172.21.15.26:0/2485641792 >> v2:172.21.15.160:3300/0 conn(0x7fa17c0b4600 0x7fa17c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.053+0000 7fa181e66640 1 --2- 172.21.15.26:0/2485641792 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa17c15ccf0 0x7fa17c0d0e40 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.053+0000 7fa181e66640 1 -- 172.21.15.26:0/2485641792 >> 172.21.15.26:0/2485641792 conn(0x7fa17c065ee0 msgr2=0x7fa17c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:56.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.053+0000 7fa181e66640 1 -- 172.21.15.26:0/2485641792 shutdown_connections 2023-12-19T18:17:56.058 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.053+0000 7fa181e66640 1 -- 172.21.15.26:0/2485641792 wait complete. 2023-12-19T18:17:56.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa181e66640 1 Processor -- start 2023-12-19T18:17:56.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa181e66640 1 -- start start 2023-12-19T18:17:56.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa181e66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa17c0b4600 0x7fa17c14d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa181e66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa17c15ccf0 0x7fa17c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa181e66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa17c15d0d0 0x7fa17c1584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa181e66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c156310 con 0x7fa17c15d0d0 2023-12-19T18:17:56.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa181e66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c1565f0 con 0x7fa17c15ccf0 2023-12-19T18:17:56.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa181e66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c1568d0 con 0x7fa17c0b4600 2023-12-19T18:17:56.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa17bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa17c15d0d0 0x7fa17c1584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa17affd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa17c15ccf0 0x7fa17c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa17bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa17c15d0d0 0x7fa17c1584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41866/0 (socket says 172.21.15.26:41866) 2023-12-19T18:17:56.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa17bfff640 1 -- 172.21.15.26:0/1276440922 learned_addr learned my addr 172.21.15.26:0/1276440922 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:56.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa17b7fe640 1 --2- 172.21.15.26:0/1276440922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa17c0b4600 0x7fa17c14d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa17affd640 1 -- 172.21.15.26:0/1276440922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa17c0b4600 msgr2=0x7fa17c14d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa17affd640 1 --2- 172.21.15.26:0/1276440922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa17c0b4600 0x7fa17c14d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa17affd640 1 -- 172.21.15.26:0/1276440922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa17c15d0d0 msgr2=0x7fa17c1584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa17affd640 1 --2- 172.21.15.26:0/1276440922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa17c15d0d0 0x7fa17c1584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa17affd640 1 -- 172.21.15.26:0/1276440922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c0094d0 con 0x7fa17c15ccf0 2023-12-19T18:17:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa17bfff640 1 --2- 172.21.15.26:0/1276440922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa17c15d0d0 0x7fa17c1584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa17affd640 1 --2- 172.21.15.26:0/1276440922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa17c15ccf0 0x7fa17c0c9c50 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa170000f70 tx=0x7fa17000c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa163fff640 1 -- 172.21.15.26:0/1276440922 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1700052d0 con 0x7fa17c15ccf0 2023-12-19T18:17:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa163fff640 1 -- 172.21.15.26:0/1276440922 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa170005430 con 0x7fa17c15ccf0 2023-12-19T18:17:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa163fff640 1 -- 172.21.15.26:0/1276440922 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170003840 con 0x7fa17c15ccf0 2023-12-19T18:17:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa181e66640 1 -- 172.21.15.26:0/1276440922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c157ce0 con 0x7fa17c15ccf0 2023-12-19T18:17:56.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.057+0000 7fa181e66640 1 -- 172.21.15.26:0/1276440922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c0b69a0 con 0x7fa17c15ccf0 2023-12-19T18:17:56.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.061+0000 7fa181e66640 1 -- 172.21.15.26:0/1276440922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17c074270 con 0x7fa17c15ccf0 2023-12-19T18:17:56.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.061+0000 7fa163fff640 1 -- 172.21.15.26:0/1276440922 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 133) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fa1700039a0 con 0x7fa17c15ccf0 2023-12-19T18:17:56.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.061+0000 7fa163fff640 1 -- 172.21.15.26:0/1276440922 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 8107+0+0 (secure 0 0 0) 0x7fa170082b30 con 0x7fa17c15ccf0 2023-12-19T18:17:56.068 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.065+0000 7fa163fff640 1 -- 172.21.15.26:0/1276440922 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa17004ffa0 con 0x7fa17c15ccf0 2023-12-19T18:17:56.134 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:56.129+0000 7fdafdadc180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:17:56.198 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:17:56.193+0000 7fdafdadc180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:17:56.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.229+0000 7fa181e66640 1 -- 172.21.15.26:0/1276440922 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa17c0b7f90 con 0x7fa17c15ccf0 2023-12-19T18:17:56.232 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.229+0000 7fa163fff640 1 -- 172.21.15.26:0/1276440922 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v239) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa170017020 con 0x7fa17c15ccf0 2023-12-19T18:17:56.232 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:17:56.234 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.229+0000 7fa181e66640 1 -- 172.21.15.26:0/1276440922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa17c15ccf0 msgr2=0x7fa17c0c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.229+0000 7fa181e66640 1 --2- 172.21.15.26:0/1276440922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa17c15ccf0 0x7fa17c0c9c50 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa170000f70 tx=0x7fa17000c820 comp rx=0 tx=0).stop 2023-12-19T18:17:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.229+0000 7fa181e66640 1 -- 172.21.15.26:0/1276440922 shutdown_connections 2023-12-19T18:17:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.229+0000 7fa181e66640 1 --2- 172.21.15.26:0/1276440922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa17c15d0d0 0x7fa17c1584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.229+0000 7fa181e66640 1 --2- 172.21.15.26:0/1276440922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa17c15ccf0 0x7fa17c0c9c50 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.229+0000 7fa181e66640 1 --2- 172.21.15.26:0/1276440922 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa17c0b4600 0x7fa17c14d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.229+0000 7fa181e66640 1 -- 172.21.15.26:0/1276440922 >> 172.21.15.26:0/1276440922 conn(0x7fa17c065ee0 msgr2=0x7fa17c0d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.229+0000 7fa181e66640 1 -- 172.21.15.26:0/1276440922 shutdown_connections 2023-12-19T18:17:56.235 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.229+0000 7fa181e66640 1 -- 172.21.15.26:0/1276440922 wait complete. 2023-12-19T18:17:56.254 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:17:56.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f086d352640 1 Processor -- start 2023-12-19T18:17:56.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f086d352640 1 -- start start 2023-12-19T18:17:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f086d352640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08680b4600 0x7f08680b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f086d352640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08680b4f50 con 0x7f086815d0d0 2023-12-19T18:17:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f086d352640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086815e200 con 0x7f086815ccf0 2023-12-19T18:17:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f086d352640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680d0aa0 con 0x7f08680b4600 2023-12-19T18:17:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f085e7fc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f086815d0d0 0x7f08680b8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34768/0 (socket says 172.21.15.26:34768) 2023-12-19T18:17:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f085e7fc640 1 -- 172.21.15.26:0/2983543823 learned_addr learned my addr 172.21.15.26:0/2983543823 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:56.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f08677fe640 1 --2- 172.21.15.26:0/2983543823 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08680b4600 0x7f08680b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f08667fc640 1 -- 172.21.15.26:0/2983543823 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569985564 0 0) 0x7f08680b4f50 con 0x7f086815d0d0 2023-12-19T18:17:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f08667fc640 1 -- 172.21.15.26:0/2983543823 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0854003530 con 0x7f086815d0d0 2023-12-19T18:17:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f08667fc640 1 -- 172.21.15.26:0/2983543823 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493219729 0 0) 0x7f086815e200 con 0x7f086815ccf0 2023-12-19T18:17:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f08667fc640 1 -- 172.21.15.26:0/2983543823 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08680b4f50 con 0x7f086815ccf0 2023-12-19T18:17:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f08677fe640 1 -- 172.21.15.26:0/2983543823 >> v1:172.21.15.160:6789/0 conn(0x7f086815ccf0 legacy=0x7f08680d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f08677fe640 1 -- 172.21.15.26:0/2983543823 >> v1:172.21.15.26:6789/0 conn(0x7f086815d0d0 legacy=0x7f08680b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.397+0000 7f08677fe640 1 -- 172.21.15.26:0/2983543823 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086815d680 con 0x7f08680b4600 2023-12-19T18:17:56.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.401+0000 7f08677fe640 1 --2- 172.21.15.26:0/2983543823 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08680b4600 0x7f08680b4a10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f08600097f0 tx=0x7f086001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4dcc18bbedd9ee server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.401+0000 7f08667fc640 1 -- 172.21.15.26:0/2983543823 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08600aa5f0 con 0x7f08680b4600 2023-12-19T18:17:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.401+0000 7f08667fc640 1 -- 172.21.15.26:0/2983543823 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08600aa750 con 0x7f08680b4600 2023-12-19T18:17:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.401+0000 7f08667fc640 1 -- 172.21.15.26:0/2983543823 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08600aaae0 con 0x7f08680b4600 2023-12-19T18:17:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.401+0000 7f086d352640 1 -- 172.21.15.26:0/2983543823 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08680b4600 msgr2=0x7f08680b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.401+0000 7f086d352640 1 --2- 172.21.15.26:0/2983543823 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08680b4600 0x7f08680b4a10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f08600097f0 tx=0x7f086001c490 comp rx=0 tx=0).stop 2023-12-19T18:17:56.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.401+0000 7f086d352640 1 -- 172.21.15.26:0/2983543823 shutdown_connections 2023-12-19T18:17:56.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.401+0000 7f086d352640 1 --2- 172.21.15.26:0/2983543823 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08680b4600 0x7f08680b4a10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.401+0000 7f086d352640 1 -- 172.21.15.26:0/2983543823 >> 172.21.15.26:0/2983543823 conn(0x7f0868065ee0 msgr2=0x7f08680cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:56.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.401+0000 7f086d352640 1 -- 172.21.15.26:0/2983543823 shutdown_connections 2023-12-19T18:17:56.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.401+0000 7f086d352640 1 -- 172.21.15.26:0/2983543823 wait complete. 2023-12-19T18:17:56.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f086d352640 1 Processor -- start 2023-12-19T18:17:56.408 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f086d352640 1 -- start start 2023-12-19T18:17:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f086d352640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08680b4600 0x7f0868154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f086d352640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f086815ccf0 0x7f0868152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f086d352640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f086815d0d0 0x7f086815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f086d352640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08681546c0 con 0x7f086815d0d0 2023-12-19T18:17:56.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f086d352640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0868153ab0 con 0x7f086815ccf0 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f086d352640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0868153d90 con 0x7f08680b4600 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f08677fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f086815d0d0 0x7f086815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f08677fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f086815d0d0 0x7f086815a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41878/0 (socket says 172.21.15.26:41878) 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f08677fe640 1 -- 172.21.15.26:0/3929611668 learned_addr learned my addr 172.21.15.26:0/3929611668 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f085e7fc640 1 --2- 172.21.15.26:0/3929611668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f086815ccf0 0x7f0868152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f0866ffd640 1 --2- 172.21.15.26:0/3929611668 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08680b4600 0x7f0868154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f08677fe640 1 -- 172.21.15.26:0/3929611668 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08680b4600 msgr2=0x7f0868154b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f08677fe640 1 --2- 172.21.15.26:0/3929611668 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08680b4600 0x7f0868154b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f08677fe640 1 -- 172.21.15.26:0/3929611668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f086815ccf0 msgr2=0x7f0868152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f08677fe640 1 --2- 172.21.15.26:0/3929611668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f086815ccf0 0x7f0868152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f08677fe640 1 -- 172.21.15.26:0/3929611668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0860009420 con 0x7f086815d0d0 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f08677fe640 1 --2- 172.21.15.26:0/3929611668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f086815d0d0 0x7f086815a0b0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f08600097f0 tx=0x7f086001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.405+0000 7f085ffff640 1 -- 172.21.15.26:0/3929611668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08600be610 con 0x7f086815d0d0 2023-12-19T18:17:56.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.409+0000 7f085ffff640 1 -- 172.21.15.26:0/3929611668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08600be770 con 0x7f086815d0d0 2023-12-19T18:17:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.409+0000 7f086d352640 1 -- 172.21.15.26:0/3929611668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0868154070 con 0x7f086815d0d0 2023-12-19T18:17:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.409+0000 7f085ffff640 1 -- 172.21.15.26:0/3929611668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08600beb00 con 0x7f086815d0d0 2023-12-19T18:17:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.409+0000 7f086d352640 1 -- 172.21.15.26:0/3929611668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08681528d0 con 0x7f086815d0d0 2023-12-19T18:17:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.409+0000 7f086d352640 1 -- 172.21.15.26:0/3929611668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0868154830 con 0x7f086815d0d0 2023-12-19T18:17:56.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.409+0000 7f085ffff640 1 -- 172.21.15.26:0/3929611668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 133) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f08600bec60 con 0x7f086815d0d0 2023-12-19T18:17:56.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.409+0000 7f085ffff640 1 -- 172.21.15.26:0/3929611668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 8107+0+0 (secure 0 0 0) 0x7f08600a7070 con 0x7f086815d0d0 2023-12-19T18:17:56.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.413+0000 7f085ffff640 1 -- 172.21.15.26:0/3929611668 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f08600eb500 con 0x7f086815d0d0 2023-12-19T18:17:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.569+0000 7f086d352640 1 -- 172.21.15.26:0/3929611668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0868074270 con 0x7f086815d0d0 2023-12-19T18:17:56.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.569+0000 7f085ffff640 1 -- 172.21.15.26:0/3929611668 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v239) v1 ==== 104+0+11 (secure 0 0 0) 0x7f08600eeb20 con 0x7f086815d0d0 2023-12-19T18:17:56.574 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:17:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.573+0000 7f086d352640 1 -- 172.21.15.26:0/3929611668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f086815d0d0 msgr2=0x7f086815a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.573+0000 7f086d352640 1 --2- 172.21.15.26:0/3929611668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f086815d0d0 0x7f086815a0b0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f08600097f0 tx=0x7f086001acd0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.573+0000 7f086d352640 1 -- 172.21.15.26:0/3929611668 shutdown_connections 2023-12-19T18:17:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.573+0000 7f086d352640 1 --2- 172.21.15.26:0/3929611668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f086815d0d0 0x7f086815a0b0 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.573+0000 7f086d352640 1 --2- 172.21.15.26:0/3929611668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f086815ccf0 0x7f0868152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.573+0000 7f086d352640 1 --2- 172.21.15.26:0/3929611668 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f08680b4600 0x7f0868154b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.573+0000 7f086d352640 1 -- 172.21.15.26:0/3929611668 >> 172.21.15.26:0/3929611668 conn(0x7f0868065ee0 msgr2=0x7f08680d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.573+0000 7f086d352640 1 -- 172.21.15.26:0/3929611668 shutdown_connections 2023-12-19T18:17:56.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.573+0000 7f086d352640 1 -- 172.21.15.26:0/3929611668 wait complete. 2023-12-19T18:17:56.595 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:17:56.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.737+0000 7f1d043a6640 1 Processor -- start 2023-12-19T18:17:56.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.737+0000 7f1d043a6640 1 -- start start 2023-12-19T18:17:56.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.737+0000 7f1d043a6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cfc06f6c0 0x7f1cfc15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.737+0000 7f1d043a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cfc159c00 0x7f1cfc070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.737+0000 7f1d043a6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cfc06fb60 con 0x7f1cfc159820 2023-12-19T18:17:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.737+0000 7f1d043a6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc15e470 con 0x7f1cfc06f6c0 2023-12-19T18:17:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.737+0000 7f1d043a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc06fe40 con 0x7f1cfc159c00 2023-12-19T18:17:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.737+0000 7f1d01941640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cfc06f6c0 0x7f1cfc15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.737+0000 7f1d01941640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1cfc06f6c0 0x7f1cfc15df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41880/0 (socket says 172.21.15.26:41880) 2023-12-19T18:17:56.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.737+0000 7f1d01941640 1 -- 172.21.15.26:0/1004344499 learned_addr learned my addr 172.21.15.26:0/1004344499 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:56.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.737+0000 7f1d01941640 1 -- 172.21.15.26:0/1004344499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cfc159c00 msgr2=0x7f1cfc070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:56.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.737+0000 7f1d02943640 1 --2- 172.21.15.26:0/1004344499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cfc159c00 0x7f1cfc070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d01941640 1 --2- 172.21.15.26:0/1004344499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cfc159c00 0x7f1cfc070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d01941640 1 -- 172.21.15.26:0/1004344499 >> v1:172.21.15.160:6789/0 conn(0x7f1cfc159820 legacy=0x7f1cfc0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:56.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d01941640 1 -- 172.21.15.26:0/1004344499 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc070120 con 0x7f1cfc06f6c0 2023-12-19T18:17:56.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d02943640 1 --2- 172.21.15.26:0/1004344499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cfc159c00 0x7f1cfc070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:56.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d01941640 1 --2- 172.21.15.26:0/1004344499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc06f6c0 0x7f1cfc15df30 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009870 tx=0x7f1cec01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f038eed8931c4b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:56.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d01140640 1 -- 172.21.15.26:0/1004344499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0ac600 con 0x7f1cfc06f6c0 2023-12-19T18:17:56.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d01140640 1 -- 172.21.15.26:0/1004344499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec0a8050 con 0x7f1cfc06f6c0 2023-12-19T18:17:56.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d01140640 1 -- 172.21.15.26:0/1004344499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0ac980 con 0x7f1cfc06f6c0 2023-12-19T18:17:56.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d043a6640 1 -- 172.21.15.26:0/1004344499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc06f6c0 msgr2=0x7f1cfc15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d043a6640 1 --2- 172.21.15.26:0/1004344499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc06f6c0 0x7f1cfc15df30 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009870 tx=0x7f1cec01c4d0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d043a6640 1 -- 172.21.15.26:0/1004344499 shutdown_connections 2023-12-19T18:17:56.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d043a6640 1 --2- 172.21.15.26:0/1004344499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cfc159c00 0x7f1cfc070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d043a6640 1 --2- 172.21.15.26:0/1004344499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc06f6c0 0x7f1cfc15df30 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d043a6640 1 -- 172.21.15.26:0/1004344499 >> 172.21.15.26:0/1004344499 conn(0x7f1cfc0ce730 msgr2=0x7f1cfc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:56.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.741+0000 7f1d043a6640 1 -- 172.21.15.26:0/1004344499 shutdown_connections 2023-12-19T18:17:56.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d043a6640 1 -- 172.21.15.26:0/1004344499 wait complete. 2023-12-19T18:17:56.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d043a6640 1 Processor -- start 2023-12-19T18:17:56.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d043a6640 1 -- start start 2023-12-19T18:17:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d043a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cfc06f6c0 0x7f1cfc14fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d043a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc159820 0x7f1cfc0c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d043a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cfc159c00 0x7f1cfc158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:56.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d043a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc153b10 con 0x7f1cfc159820 2023-12-19T18:17:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d043a6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc154970 con 0x7f1cfc06f6c0 2023-12-19T18:17:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d043a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc153d60 con 0x7f1cfc159c00 2023-12-19T18:17:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d01941640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc159820 0x7f1cfc0c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d01941640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc159820 0x7f1cfc0c21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41890/0 (socket says 172.21.15.26:41890) 2023-12-19T18:17:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d01941640 1 -- 172.21.15.26:0/918316396 learned_addr learned my addr 172.21.15.26:0/918316396 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d02142640 1 --2- 172.21.15.26:0/918316396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cfc06f6c0 0x7f1cfc14fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d02943640 1 --2- 172.21.15.26:0/918316396 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cfc159c00 0x7f1cfc158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d01941640 1 -- 172.21.15.26:0/918316396 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cfc159c00 msgr2=0x7f1cfc158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d01941640 1 --2- 172.21.15.26:0/918316396 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cfc159c00 0x7f1cfc158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d01941640 1 -- 172.21.15.26:0/918316396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cfc06f6c0 msgr2=0x7f1cfc14fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d01941640 1 --2- 172.21.15.26:0/918316396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cfc06f6c0 0x7f1cfc14fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d01941640 1 -- 172.21.15.26:0/918316396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec0094d0 con 0x7f1cfc159820 2023-12-19T18:17:56.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.745+0000 7f1d01941640 1 --2- 172.21.15.26:0/918316396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc159820 0x7f1cfc0c21d0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f1cec01cd70 tx=0x7f1cec01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.749+0000 7f1cf2ffd640 1 -- 172.21.15.26:0/918316396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0b7620 con 0x7f1cfc159820 2023-12-19T18:17:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.749+0000 7f1cf2ffd640 1 -- 172.21.15.26:0/918316396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec0b77a0 con 0x7f1cfc159820 2023-12-19T18:17:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.749+0000 7f1cf2ffd640 1 -- 172.21.15.26:0/918316396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0b7b50 con 0x7f1cfc159820 2023-12-19T18:17:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.749+0000 7f1d043a6640 1 -- 172.21.15.26:0/918316396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cfc06b880 con 0x7f1cfc159820 2023-12-19T18:17:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.749+0000 7f1d043a6640 1 -- 172.21.15.26:0/918316396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc154320 con 0x7f1cfc159820 2023-12-19T18:17:56.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.749+0000 7f1d043a6640 1 -- 172.21.15.26:0/918316396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cfc0cd0d0 con 0x7f1cfc159820 2023-12-19T18:17:56.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.749+0000 7f1cf2ffd640 1 -- 172.21.15.26:0/918316396 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 133) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f1cec0b3040 con 0x7f1cfc159820 2023-12-19T18:17:56.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.749+0000 7f1cf2ffd640 1 -- 172.21.15.26:0/918316396 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 8107+0+0 (secure 0 0 0) 0x7f1cec11d820 con 0x7f1cfc159820 2023-12-19T18:17:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.753+0000 7f1cf2ffd640 1 -- 172.21.15.26:0/918316396 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1cec0ead80 con 0x7f1cfc159820 2023-12-19T18:17:56.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.909+0000 7f1d043a6640 1 -- 172.21.15.26:0/918316396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1cfc154ae0 con 0x7f1cfc159820 2023-12-19T18:17:56.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.909+0000 7f1cf2ffd640 1 -- 172.21.15.26:0/918316396 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v239) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1cec0ee3a0 con 0x7f1cfc159820 2023-12-19T18:17:56.914 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:17:56.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.913+0000 7f1d043a6640 1 -- 172.21.15.26:0/918316396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc159820 msgr2=0x7f1cfc0c21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:56.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.913+0000 7f1d043a6640 1 --2- 172.21.15.26:0/918316396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc159820 0x7f1cfc0c21d0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f1cec01cd70 tx=0x7f1cec01ad10 comp rx=0 tx=0).stop 2023-12-19T18:17:56.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.913+0000 7f1d043a6640 1 -- 172.21.15.26:0/918316396 shutdown_connections 2023-12-19T18:17:56.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.913+0000 7f1d043a6640 1 --2- 172.21.15.26:0/918316396 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1cfc159c00 0x7f1cfc158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.913+0000 7f1d043a6640 1 --2- 172.21.15.26:0/918316396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1cfc159820 0x7f1cfc0c21d0 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.913+0000 7f1d043a6640 1 --2- 172.21.15.26:0/918316396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cfc06f6c0 0x7f1cfc14fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:56.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.913+0000 7f1d043a6640 1 -- 172.21.15.26:0/918316396 >> 172.21.15.26:0/918316396 conn(0x7f1cfc0ce730 msgr2=0x7f1cfc06ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:56.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.913+0000 7f1d043a6640 1 -- 172.21.15.26:0/918316396 shutdown_connections 2023-12-19T18:17:56.916 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:56.913+0000 7f1d043a6640 1 -- 172.21.15.26:0/918316396 wait complete. 2023-12-19T18:17:56.936 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-19T18:17:57.083 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e8012c640 1 Processor -- start 2023-12-19T18:17:57.084 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e8012c640 1 -- start start 2023-12-19T18:17:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e8012c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7e78159c00 0x7f7e7815df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e8012c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7806fb60 con 0x7f7e78159820 2023-12-19T18:17:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e8012c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7806fe40 con 0x7f7e7806f6c0 2023-12-19T18:17:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e8012c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7815e470 con 0x7f7e78159c00 2023-12-19T18:17:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e7dec8640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7e78159820 0x7f7e780c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34778/0 (socket says 172.21.15.26:34778) 2023-12-19T18:17:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e7dec8640 1 -- 172.21.15.26:0/2595042282 learned_addr learned my addr 172.21.15.26:0/2595042282 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:57.085 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e7e6c9640 1 --2- 172.21.15.26:0/2595042282 >> v2:172.21.15.160:3300/0 conn(0x7f7e78159c00 0x7f7e7815df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e7cec6640 1 -- 172.21.15.26:0/2595042282 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041227119 0 0) 0x7f7e7806fb60 con 0x7f7e78159820 2023-12-19T18:17:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e7cec6640 1 -- 172.21.15.26:0/2595042282 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e64003530 con 0x7f7e78159820 2023-12-19T18:17:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e7cec6640 1 -- 172.21.15.26:0/2595042282 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095915787 0 0) 0x7f7e7806fe40 con 0x7f7e7806f6c0 2023-12-19T18:17:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e7cec6640 1 -- 172.21.15.26:0/2595042282 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e7806fb60 con 0x7f7e7806f6c0 2023-12-19T18:17:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e7e6c9640 1 -- 172.21.15.26:0/2595042282 >> v1:172.21.15.160:6789/0 conn(0x7f7e7806f6c0 legacy=0x7f7e78070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e7e6c9640 1 -- 172.21.15.26:0/2595042282 >> v1:172.21.15.26:6789/0 conn(0x7f7e78159820 legacy=0x7f7e780c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.081+0000 7f7e7e6c9640 1 -- 172.21.15.26:0/2595042282 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e78070120 con 0x7f7e78159c00 2023-12-19T18:17:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.085+0000 7f7e7e6c9640 1 --2- 172.21.15.26:0/2595042282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e78159c00 0x7f7e7815df30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009d90 tx=0x7f7e7001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fb8664f67aaa072 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.085+0000 7f7e7cec6640 1 -- 172.21.15.26:0/2595042282 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e700a59e0 con 0x7f7e78159c00 2023-12-19T18:17:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.085+0000 7f7e7cec6640 1 -- 172.21.15.26:0/2595042282 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e700a5b40 con 0x7f7e78159c00 2023-12-19T18:17:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.085+0000 7f7e7cec6640 1 -- 172.21.15.26:0/2595042282 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e700ad440 con 0x7f7e78159c00 2023-12-19T18:17:57.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.085+0000 7f7e8012c640 1 -- 172.21.15.26:0/2595042282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e78159c00 msgr2=0x7f7e7815df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.085+0000 7f7e8012c640 1 --2- 172.21.15.26:0/2595042282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e78159c00 0x7f7e7815df30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009d90 tx=0x7f7e7001c920 comp rx=0 tx=0).stop 2023-12-19T18:17:57.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.085+0000 7f7e8012c640 1 -- 172.21.15.26:0/2595042282 shutdown_connections 2023-12-19T18:17:57.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.085+0000 7f7e8012c640 1 --2- 172.21.15.26:0/2595042282 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e78159c00 0x7f7e7815df30 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.085+0000 7f7e8012c640 1 -- 172.21.15.26:0/2595042282 >> 172.21.15.26:0/2595042282 conn(0x7f7e780ce730 msgr2=0x7f7e780ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:57.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.085+0000 7f7e8012c640 1 -- 172.21.15.26:0/2595042282 shutdown_connections 2023-12-19T18:17:57.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.085+0000 7f7e8012c640 1 -- 172.21.15.26:0/2595042282 wait complete. 2023-12-19T18:17:57.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e8012c640 1 Processor -- start 2023-12-19T18:17:57.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e8012c640 1 -- start start 2023-12-19T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e8012c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e7806f6c0 0x7f7e7814a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e8012c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e78159820 0x7f7e7814ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e8012c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e78159c00 0x7f7e78152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e8012c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7814f0a0 con 0x7f7e78159c00 2023-12-19T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e8012c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7814aac0 con 0x7f7e7806f6c0 2023-12-19T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e8012c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781540b0 con 0x7f7e78159820 2023-12-19T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e7e6c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e78159c00 0x7f7e78152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e7e6c9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e78159c00 0x7f7e78152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41906/0 (socket says 172.21.15.26:41906) 2023-12-19T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e7e6c9640 1 -- 172.21.15.26:0/3625965545 learned_addr learned my addr 172.21.15.26:0/3625965545 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e7d6c7640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e78159820 0x7f7e7814ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e7dec8640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e7806f6c0 0x7f7e7814a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e7e6c9640 1 -- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e78159820 msgr2=0x7f7e7814ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e7e6c9640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e78159820 0x7f7e7814ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e7e6c9640 1 -- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e7806f6c0 msgr2=0x7f7e7814a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e7e6c9640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e7806f6c0 0x7f7e7814a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e7e6c9640 1 -- 172.21.15.26:0/3625965545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e70009a70 con 0x7f7e78159c00 2023-12-19T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e7d6c7640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e78159820 0x7f7e7814ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e7dec8640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e7806f6c0 0x7f7e7814a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e7e6c9640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e78159c00 0x7f7e78152900 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f7e700b0040 tx=0x7f7e700196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e6e7fc640 1 -- 172.21.15.26:0/3625965545 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e700adce0 con 0x7f7e78159c00 2023-12-19T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e6e7fc640 1 -- 172.21.15.26:0/3625965545 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e700ade40 con 0x7f7e78159c00 2023-12-19T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e6e7fc640 1 -- 172.21.15.26:0/3625965545 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e700b9690 con 0x7f7e78159c00 2023-12-19T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e8012c640 1 -- 172.21.15.26:0/3625965545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7806b880 con 0x7f7e78159c00 2023-12-19T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.089+0000 7f7e8012c640 1 -- 172.21.15.26:0/3625965545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e78151ae0 con 0x7f7e78159c00 2023-12-19T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.093+0000 7f7e8012c640 1 -- 172.21.15.26:0/3625965545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e78151cd0 con 0x7f7e78159c00 2023-12-19T18:17:57.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.093+0000 7f7e6e7fc640 1 -- 172.21.15.26:0/3625965545 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 134) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f7e700b97f0 con 0x7f7e78159c00 2023-12-19T18:17:57.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.093+0000 7f7e6e7fc640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7e5c0658b0 0x7f7e5c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.093+0000 7f7e6e7fc640 1 -- 172.21.15.26:0/3625965545 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 8107+0+0 (secure 0 0 0) 0x7f7e7011efd0 con 0x7f7e78159c00 2023-12-19T18:17:57.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.093+0000 7f7e7dec8640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7e5c0658b0 0x7f7e5c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.093+0000 7f7e7dec8640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7e5c0658b0 0x7f7e5c067d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7e74000f70 tx=0x7f7e74009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:57.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.097+0000 7f7e6e7fc640 1 -- 172.21.15.26:0/3625965545 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7e700ec220 con 0x7f7e78159c00 2023-12-19T18:17:57.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.253+0000 7f7e8012c640 1 -- 172.21.15.26:0/3625965545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e78074270 con 0x7f7e78159c00 2023-12-19T18:17:57.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.253+0000 7f7e6e7fc640 1 -- 172.21.15.26:0/3625965545 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v239) v1 ==== 107+0+11 (secure 0 0 0) 0x7f7e700b20b0 con 0x7f7e78159c00 2023-12-19T18:17:57.258 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:17:57.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.257+0000 7f7e8012c640 1 -- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7e5c0658b0 msgr2=0x7f7e5c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.257+0000 7f7e8012c640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7e5c0658b0 0x7f7e5c067d70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7e74000f70 tx=0x7f7e74009040 comp rx=0 tx=0).stop 2023-12-19T18:17:57.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.257+0000 7f7e8012c640 1 -- 172.21.15.26:0/3625965545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e78159c00 msgr2=0x7f7e78152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.257+0000 7f7e8012c640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e78159c00 0x7f7e78152900 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f7e700b0040 tx=0x7f7e700196c0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.257+0000 7f7e8012c640 1 -- 172.21.15.26:0/3625965545 shutdown_connections 2023-12-19T18:17:57.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.257+0000 7f7e8012c640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7e5c0658b0 0x7f7e5c067d70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.257+0000 7f7e8012c640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e78159c00 0x7f7e78152900 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.257+0000 7f7e8012c640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7e78159820 0x7f7e7814ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.260 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.257+0000 7f7e8012c640 1 --2- 172.21.15.26:0/3625965545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e7806f6c0 0x7f7e7814a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.257+0000 7f7e8012c640 1 -- 172.21.15.26:0/3625965545 >> 172.21.15.26:0/3625965545 conn(0x7f7e780ce730 msgr2=0x7f7e7815db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:57.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.257+0000 7f7e8012c640 1 -- 172.21.15.26:0/3625965545 shutdown_connections 2023-12-19T18:17:57.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.257+0000 7f7e8012c640 1 -- 172.21.15.26:0/3625965545 wait complete. 2023-12-19T18:17:57.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-19T18:17:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.421+0000 7f0b02fbe640 1 Processor -- start 2023-12-19T18:17:57.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.421+0000 7f0b02fbe640 1 -- start start 2023-12-19T18:17:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.421+0000 7f0b02fbe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0afc0706d0 0x7f0afc0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.421+0000 7f0b02fbe640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0afc070ae0 con 0x7f0afc15a9c0 2023-12-19T18:17:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.421+0000 7f0b02fbe640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0afc070d90 con 0x7f0afc15ada0 2023-12-19T18:17:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.421+0000 7f0b02fbe640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0afc0cc270 con 0x7f0afc0706d0 2023-12-19T18:17:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.421+0000 7f0b00d5a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0afc15a9c0 0x7f0afc0b4180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34782/0 (socket says 172.21.15.26:34782) 2023-12-19T18:17:57.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.421+0000 7f0b00d5a640 1 -- 172.21.15.26:0/664997738 learned_addr learned my addr 172.21.15.26:0/664997738 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.421+0000 7f0b0155b640 1 --2- 172.21.15.26:0/664997738 >> v2:172.21.15.160:3300/0 conn(0x7f0afc0706d0 0x7f0afc0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1694203934 0 0) 0x7f0afc070ae0 con 0x7f0afc15a9c0 2023-12-19T18:17:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aec003530 con 0x7f0afc15a9c0 2023-12-19T18:17:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853059698 0 0) 0x7f0afc070d90 con 0x7f0afc15ada0 2023-12-19T18:17:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0afc070ae0 con 0x7f0afc15ada0 2023-12-19T18:17:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1018024927 0 0) 0x7f0aec003530 con 0x7f0afc15a9c0 2023-12-19T18:17:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 >> v2:172.21.15.160:3300/0 conn(0x7f0afc0706d0 msgr2=0x7f0afc0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 --2- 172.21.15.26:0/664997738 >> v2:172.21.15.160:3300/0 conn(0x7f0afc0706d0 0x7f0afc0b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 >> v1:172.21.15.160:6789/0 conn(0x7f0afc15ada0 legacy=0x7f0afc06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0afc071070 con 0x7f0afc15a9c0 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0b0155b640 1 --2- 172.21.15.26:0/664997738 >> v2:172.21.15.160:3300/0 conn(0x7f0afc0706d0 0x7f0afc0b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f0af0001680 con 0x7f0afc15a9c0 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 >> v1:172.21.15.26:6789/0 conn(0x7f0afc15a9c0 legacy=0x7f0afc0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 --2- 172.21.15.26:0/664997738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec003b10 0x7f0aec003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 --2- 172.21.15.26:0/664997738 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0aec0047d0 0x7f0aec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 --2- 172.21.15.26:0/664997738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aec005190 0x7f0aec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc0cc270 con 0x7f0aec003b10 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec00c0c0 con 0x7f0aec005190 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec00c280 con 0x7f0aec0047d0 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0b00d5a640 1 --2- 172.21.15.26:0/664997738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec003b10 0x7f0aec003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0b0155b640 1 --2- 172.21.15.26:0/664997738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aec005190 0x7f0aec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0afbfff640 1 --2- 172.21.15.26:0/664997738 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0aec0047d0 0x7f0aec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0b00d5a640 1 -- 172.21.15.26:0/664997738 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0aec0047d0 msgr2=0x7f0aec004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0b00d5a640 1 --2- 172.21.15.26:0/664997738 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0aec0047d0 0x7f0aec004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0b00d5a640 1 -- 172.21.15.26:0/664997738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aec005190 msgr2=0x7f0aec00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0b00d5a640 1 --2- 172.21.15.26:0/664997738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aec005190 0x7f0aec00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0b00d5a640 1 -- 172.21.15.26:0/664997738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec003530 con 0x7f0aec003b10 2023-12-19T18:17:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0b0155b640 1 --2- 172.21.15.26:0/664997738 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aec005190 0x7f0aec00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:57.431 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.425+0000 7f0b0155b640 1 -- 172.21.15.26:0/664997738 reap_dead start 2023-12-19T18:17:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b00d5a640 1 --2- 172.21.15.26:0/664997738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec003b10 0x7f0aec003f10 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f0af000c160 tx=0x7f0af001cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60065db583cca1b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0afc06b490 con 0x7f0aec003b10 2023-12-19T18:17:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af00058e0 con 0x7f0aec003b10 2023-12-19T18:17:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0afb7fe640 1 -- 172.21.15.26:0/664997738 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0005cf0 con 0x7f0aec003b10 2023-12-19T18:17:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 -- 172.21.15.26:0/664997738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec003b10 msgr2=0x7f0aec003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 --2- 172.21.15.26:0/664997738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec003b10 0x7f0aec003f10 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7f0af000c160 tx=0x7f0af001cb90 comp rx=0 tx=0).stop 2023-12-19T18:17:57.432 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 -- 172.21.15.26:0/664997738 shutdown_connections 2023-12-19T18:17:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 --2- 172.21.15.26:0/664997738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0aec003b10 0x7f0aec003f10 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 -- 172.21.15.26:0/664997738 >> 172.21.15.26:0/664997738 conn(0x7f0afc0ce730 msgr2=0x7f0afc0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 -- 172.21.15.26:0/664997738 shutdown_connections 2023-12-19T18:17:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 -- 172.21.15.26:0/664997738 wait complete. 2023-12-19T18:17:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 Processor -- start 2023-12-19T18:17:57.433 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 -- start start 2023-12-19T18:17:57.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aec003b10 0x7f0afc0c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.435 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0afc141560 0x7f0afc1419e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1e67a0 0x7f0afc1e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc1e8150 con 0x7f0afc1e67a0 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc141f20 con 0x7f0aec003b10 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.429+0000 7f0b02fbe640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0afc1e74d0 con 0x7f0afc141560 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0b0155b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1e67a0 0x7f0afc1e6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0b0155b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1e67a0 0x7f0afc1e6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41918/0 (socket says 172.21.15.26:41918) 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0b0155b640 1 -- 172.21.15.26:0/553168386 learned_addr learned my addr 172.21.15.26:0/553168386 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0b00d5a640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aec003b10 0x7f0afc0c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0afbfff640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0afc141560 0x7f0afc1419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0b0155b640 1 -- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0afc141560 msgr2=0x7f0afc1419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0b0155b640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0afc141560 0x7f0afc1419e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0b0155b640 1 -- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aec003b10 msgr2=0x7f0afc0c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0b0155b640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aec003b10 0x7f0afc0c59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.436 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0b0155b640 1 -- 172.21.15.26:0/553168386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0afc1e77b0 con 0x7f0afc1e67a0 2023-12-19T18:17:57.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0b0155b640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1e67a0 0x7f0afc1e6c20 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f0afc06d600 tx=0x7f0af4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:57.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0af97fa640 1 -- 172.21.15.26:0/553168386 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af400e070 con 0x7f0afc1e67a0 2023-12-19T18:17:57.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0af97fa640 1 -- 172.21.15.26:0/553168386 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af4011e60 con 0x7f0afc1e67a0 2023-12-19T18:17:57.437 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0af97fa640 1 -- 172.21.15.26:0/553168386 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af40039a0 con 0x7f0afc1e67a0 2023-12-19T18:17:57.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0b02fbe640 1 -- 172.21.15.26:0/553168386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0afc1e7a90 con 0x7f0afc1e67a0 2023-12-19T18:17:57.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0b02fbe640 1 -- 172.21.15.26:0/553168386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0afc1e7160 con 0x7f0afc1e67a0 2023-12-19T18:17:57.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0b02fbe640 1 -- 172.21.15.26:0/553168386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0afc0c2190 con 0x7f0afc1e67a0 2023-12-19T18:17:57.438 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.433+0000 7f0af97fa640 1 -- 172.21.15.26:0/553168386 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 134) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f0af4003b00 con 0x7f0afc1e67a0 2023-12-19T18:17:57.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.437+0000 7f0af97fa640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0ae00658b0 0x7f0ae0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.437+0000 7f0af97fa640 1 -- 172.21.15.26:0/553168386 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 8107+0+0 (secure 0 0 0) 0x7f0af4083f10 con 0x7f0afc1e67a0 2023-12-19T18:17:57.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.437+0000 7f0b00d5a640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0ae00658b0 0x7f0ae0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.440 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.437+0000 7f0b00d5a640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0ae00658b0 0x7f0ae0067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0af00aa000 tx=0x7f0af0001a00 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:57.443 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.441+0000 7f0af97fa640 1 -- 172.21.15.26:0/553168386 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0afc0c2190 con 0x7f0afc1e67a0 2023-12-19T18:17:57.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.597+0000 7f0b02fbe640 1 -- 172.21.15.26:0/553168386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f0afc0c2190 con 0x7f0afc1e67a0 2023-12-19T18:17:57.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.597+0000 7f0af97fa640 1 -- 172.21.15.26:0/553168386 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v239) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0afc0c2190 con 0x7f0afc1e67a0 2023-12-19T18:17:57.602 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:17:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.601+0000 7f0b02fbe640 1 -- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0ae00658b0 msgr2=0x7f0ae0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.601+0000 7f0b02fbe640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0ae00658b0 0x7f0ae0067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0af00aa000 tx=0x7f0af0001a00 comp rx=0 tx=0).stop 2023-12-19T18:17:57.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.601+0000 7f0b02fbe640 1 -- 172.21.15.26:0/553168386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1e67a0 msgr2=0x7f0afc1e6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.601+0000 7f0b02fbe640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1e67a0 0x7f0afc1e6c20 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f0afc06d600 tx=0x7f0af4009f90 comp rx=0 tx=0).stop 2023-12-19T18:17:57.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.601+0000 7f0b02fbe640 1 -- 172.21.15.26:0/553168386 shutdown_connections 2023-12-19T18:17:57.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.601+0000 7f0b02fbe640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0ae00658b0 0x7f0ae0067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.601+0000 7f0b02fbe640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0afc1e67a0 0x7f0afc1e6c20 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.601+0000 7f0b02fbe640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0afc141560 0x7f0afc1419e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.601+0000 7f0b02fbe640 1 --2- 172.21.15.26:0/553168386 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aec003b10 0x7f0afc0c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.601+0000 7f0b02fbe640 1 -- 172.21.15.26:0/553168386 >> 172.21.15.26:0/553168386 conn(0x7f0afc0ce730 msgr2=0x7f0afc14f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:57.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.601+0000 7f0b02fbe640 1 -- 172.21.15.26:0/553168386 shutdown_connections 2023-12-19T18:17:57.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.601+0000 7f0b02fbe640 1 -- 172.21.15.26:0/553168386 wait complete. 2023-12-19T18:17:57.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f905dd91640 1 Processor -- start 2023-12-19T18:17:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f905dd91640 1 -- start start 2023-12-19T18:17:57.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f905dd91640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9058160040 0x7f90580cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f905dd91640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90580b1ab0 0x7f90580cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f905dd91640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f90580658e0 0x7f9058065d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f905dd91640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9058066290 con 0x7f90580b1ab0 2023-12-19T18:17:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f905dd91640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90580ce570 con 0x7f90580658e0 2023-12-19T18:17:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f905dd91640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90580cdae0 con 0x7f9058160040 2023-12-19T18:17:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f9057fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90580b1ab0 0x7f90580cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f9057fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90580b1ab0 0x7f90580cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41928/0 (socket says 172.21.15.26:41928) 2023-12-19T18:17:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f9057fff640 1 -- 172.21.15.26:0/3432937475 learned_addr learned my addr 172.21.15.26:0/3432937475 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f905cd8f640 1 --2- 172.21.15.26:0/3432937475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9058160040 0x7f90580cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f905d590640 1 --2- 172.21.15.26:0/3432937475 >> v2:172.21.15.160:3300/0 conn(0x7f90580658e0 0x7f9058065d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f9057fff640 1 -- 172.21.15.26:0/3432937475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9058160040 msgr2=0x7f90580cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f9057fff640 1 --2- 172.21.15.26:0/3432937475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9058160040 0x7f90580cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f9057fff640 1 -- 172.21.15.26:0/3432937475 >> v2:172.21.15.160:3300/0 conn(0x7f90580658e0 msgr2=0x7f9058065d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f9057fff640 1 --2- 172.21.15.26:0/3432937475 >> v2:172.21.15.160:3300/0 conn(0x7f90580658e0 0x7f9058065d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f9057fff640 1 -- 172.21.15.26:0/3432937475 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90580cb660 con 0x7f90580b1ab0 2023-12-19T18:17:57.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.765+0000 7f905d590640 1 --2- 172.21.15.26:0/3432937475 >> v2:172.21.15.160:3300/0 conn(0x7f90580658e0 0x7f9058065d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:57.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f9057fff640 1 --2- 172.21.15.26:0/3432937475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90580b1ab0 0x7f90580cdd90 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f905806ff80 tx=0x7f904401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66315aa84ba9dda1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:57.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f90577fe640 1 -- 172.21.15.26:0/3432937475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90440ab600 con 0x7f90580b1ab0 2023-12-19T18:17:57.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f90577fe640 1 -- 172.21.15.26:0/3432937475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90440a7050 con 0x7f90580b1ab0 2023-12-19T18:17:57.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f90577fe640 1 -- 172.21.15.26:0/3432937475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90440ab980 con 0x7f90580b1ab0 2023-12-19T18:17:57.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f905dd91640 1 -- 172.21.15.26:0/3432937475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90580b1ab0 msgr2=0x7f90580cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f905dd91640 1 --2- 172.21.15.26:0/3432937475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90580b1ab0 0x7f90580cdd90 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f905806ff80 tx=0x7f904401c500 comp rx=0 tx=0).stop 2023-12-19T18:17:57.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f905dd91640 1 -- 172.21.15.26:0/3432937475 shutdown_connections 2023-12-19T18:17:57.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f905dd91640 1 --2- 172.21.15.26:0/3432937475 >> v2:172.21.15.160:3300/0 conn(0x7f90580658e0 0x7f9058065d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f905dd91640 1 --2- 172.21.15.26:0/3432937475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90580b1ab0 0x7f90580cdd90 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f905dd91640 1 --2- 172.21.15.26:0/3432937475 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9058160040 0x7f90580cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f905dd91640 1 -- 172.21.15.26:0/3432937475 >> 172.21.15.26:0/3432937475 conn(0x7f90580cf7b0 msgr2=0x7f90580cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:57.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f905dd91640 1 -- 172.21.15.26:0/3432937475 shutdown_connections 2023-12-19T18:17:57.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f905dd91640 1 -- 172.21.15.26:0/3432937475 wait complete. 2023-12-19T18:17:57.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.769+0000 7f905dd91640 1 Processor -- start 2023-12-19T18:17:57.775 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905dd91640 1 -- start start 2023-12-19T18:17:57.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905dd91640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90580658e0 0x7f9058149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905dd91640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90580b1ab0 0x7f905815f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905dd91640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9058160040 0x7f90580b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905dd91640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580b4f60 con 0x7f90580658e0 2023-12-19T18:17:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905dd91640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9058070150 con 0x7f90580b1ab0 2023-12-19T18:17:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905dd91640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90580b51b0 con 0x7f9058160040 2023-12-19T18:17:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905cd8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90580658e0 0x7f9058149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905cd8f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90580658e0 0x7f9058149d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41936/0 (socket says 172.21.15.26:41936) 2023-12-19T18:17:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905cd8f640 1 -- 172.21.15.26:0/1204996637 learned_addr learned my addr 172.21.15.26:0/1204996637 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905cd8f640 1 -- 172.21.15.26:0/1204996637 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9058160040 msgr2=0x7f90580b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905cd8f640 1 --2- 172.21.15.26:0/1204996637 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9058160040 0x7f90580b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905cd8f640 1 -- 172.21.15.26:0/1204996637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90580b1ab0 msgr2=0x7f905815f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905cd8f640 1 --2- 172.21.15.26:0/1204996637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90580b1ab0 0x7f905815f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.777 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905cd8f640 1 -- 172.21.15.26:0/1204996637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90440094d0 con 0x7f90580658e0 2023-12-19T18:17:57.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905cd8f640 1 --2- 172.21.15.26:0/1204996637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90580658e0 0x7f9058149d00 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f904c00d620 tx=0x7f904c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:57.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f90557fa640 1 -- 172.21.15.26:0/1204996637 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c00b620 con 0x7f90580658e0 2023-12-19T18:17:57.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f90557fa640 1 -- 172.21.15.26:0/1204996637 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c00b780 con 0x7f90580658e0 2023-12-19T18:17:57.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f90557fa640 1 -- 172.21.15.26:0/1204996637 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904c00bb10 con 0x7f90580658e0 2023-12-19T18:17:57.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905dd91640 1 -- 172.21.15.26:0/1204996637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90580703a0 con 0x7f90580658e0 2023-12-19T18:17:57.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905dd91640 1 -- 172.21.15.26:0/1204996637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9058070960 con 0x7f90580658e0 2023-12-19T18:17:57.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.773+0000 7f905dd91640 1 -- 172.21.15.26:0/1204996637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9058067400 con 0x7f90580658e0 2023-12-19T18:17:57.779 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.777+0000 7f90557fa640 1 -- 172.21.15.26:0/1204996637 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 134) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f904c00bc70 con 0x7f90580658e0 2023-12-19T18:17:57.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.777+0000 7f90557fa640 1 --2- 172.21.15.26:0/1204996637 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f903c0658e0 0x7f903c067da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:57.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.777+0000 7f90557fa640 1 -- 172.21.15.26:0/1204996637 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 8107+0+0 (secure 0 0 0) 0x7f904c083b30 con 0x7f90580658e0 2023-12-19T18:17:57.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.777+0000 7f9057fff640 1 --2- 172.21.15.26:0/1204996637 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f903c0658e0 0x7f903c067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:57.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.777+0000 7f9057fff640 1 --2- 172.21.15.26:0/1204996637 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f903c0658e0 0x7f903c067da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f90440ad000 tx=0x7f9044004740 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.781+0000 7f90557fa640 1 -- 172.21.15.26:0/1204996637 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f904c050ed0 con 0x7f90580658e0 2023-12-19T18:17:57.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.969+0000 7f905dd91640 1 -- 172.21.15.26:0/1204996637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f905815d5c0 con 0x7f90580658e0 2023-12-19T18:17:57.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.969+0000 7f90557fa640 1 -- 172.21.15.26:0/1204996637 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5921 (secure 0 0 0) 0x7f904c0544f0 con 0x7f90580658e0 2023-12-19T18:17:57.975 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:57.975 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2727392302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2727392302},{"type":"v1","addr":"172.21.15.160:6836","nonce":2727392302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1029020739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1029020739},{"type":"v1","addr":"172.21.15.160:6837","nonce":1029020739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:27.325271+0000","modified":"2023-12-19T18:17:30.337993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11427":{"gid":11427,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2285370499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2285370499},{"type":"v1","addr":"172.21.15.26:6835","nonce":2285370499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:39.801489+0000","modified":"2023-12-19T18:17:42.831660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11424":{"gid":11424,"name":"a","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6834/3251191056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3251191056},{"type":"v1","addr":"172.21.15.26:6834","nonce":3251191056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-19T18:17:57.975 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 118 2023-12-19T18:17:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.973+0000 7f905dd91640 1 -- 172.21.15.26:0/1204996637 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f903c0658e0 msgr2=0x7f903c067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.973+0000 7f905dd91640 1 --2- 172.21.15.26:0/1204996637 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f903c0658e0 0x7f903c067da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f90440ad000 tx=0x7f9044004740 comp rx=0 tx=0).stop 2023-12-19T18:17:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.973+0000 7f905dd91640 1 -- 172.21.15.26:0/1204996637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90580658e0 msgr2=0x7f9058149d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.973+0000 7f905dd91640 1 --2- 172.21.15.26:0/1204996637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90580658e0 0x7f9058149d00 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f904c00d620 tx=0x7f904c00dae0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.973+0000 7f905dd91640 1 -- 172.21.15.26:0/1204996637 shutdown_connections 2023-12-19T18:17:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.973+0000 7f905dd91640 1 --2- 172.21.15.26:0/1204996637 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f903c0658e0 0x7f903c067da0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.973+0000 7f905dd91640 1 --2- 172.21.15.26:0/1204996637 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9058160040 0x7f90580b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.973+0000 7f905dd91640 1 --2- 172.21.15.26:0/1204996637 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90580b1ab0 0x7f905815f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.973+0000 7f905dd91640 1 --2- 172.21.15.26:0/1204996637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90580658e0 0x7f9058149d00 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.973+0000 7f905dd91640 1 -- 172.21.15.26:0/1204996637 >> 172.21.15.26:0/1204996637 conn(0x7f90580cf7b0 msgr2=0x7f90580cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.973+0000 7f905dd91640 1 -- 172.21.15.26:0/1204996637 shutdown_connections 2023-12-19T18:17:57.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:57.973+0000 7f905dd91640 1 -- 172.21.15.26:0/1204996637 wait complete. 2023-12-19T18:17:57.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff9309cb640 1 Processor -- start 2023-12-19T18:17:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff9309cb640 1 -- start start 2023-12-19T18:17:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff9309cb640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff92c15cd50 0x7ff92c0b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff9309cb640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff92c0d0050 0x7ff92c071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff9309cb640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff92c161740 con 0x7ff92c0b2750 2023-12-19T18:17:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff9309cb640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92c0d0600 con 0x7ff92c15cd50 2023-12-19T18:17:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff9309cb640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92c15d130 con 0x7ff92c0d0050 2023-12-19T18:17:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff92b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff92c15cd50 0x7ff92c0b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff92affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff92c0b2750 0x7ff92c0cf940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34790/0 (socket says 172.21.15.26:34790) 2023-12-19T18:17:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff92b7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff92c15cd50 0x7ff92c0b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41938/0 (socket says 172.21.15.26:41938) 2023-12-19T18:17:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff92affd640 1 -- 172.21.15.26:0/2751121521 learned_addr learned my addr 172.21.15.26:0/2751121521 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff92bfff640 1 --2- 172.21.15.26:0/2751121521 >> v2:172.21.15.160:3300/0 conn(0x7ff92c0d0050 0x7ff92c071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff92b7fe640 1 -- 172.21.15.26:0/2751121521 >> v2:172.21.15.160:3300/0 conn(0x7ff92c0d0050 msgr2=0x7ff92c071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff92b7fe640 1 --2- 172.21.15.26:0/2751121521 >> v2:172.21.15.160:3300/0 conn(0x7ff92c0d0050 0x7ff92c071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff92b7fe640 1 -- 172.21.15.26:0/2751121521 >> v1:172.21.15.26:6789/0 conn(0x7ff92c0b2750 legacy=0x7ff92c0cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.141+0000 7ff92b7fe640 1 -- 172.21.15.26:0/2751121521 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c0b2b90 con 0x7ff92c15cd50 2023-12-19T18:17:58.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.145+0000 7ff92b7fe640 1 --2- 172.21.15.26:0/2751121521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92c15cd50 0x7ff92c0b2210 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0cad70 tx=0x7ff91c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747873dc4e1d12f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:58.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.145+0000 7ff92a7fc640 1 -- 172.21.15.26:0/2751121521 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c004230 con 0x7ff92c15cd50 2023-12-19T18:17:58.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.145+0000 7ff92a7fc640 1 -- 172.21.15.26:0/2751121521 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c0a7050 con 0x7ff92c15cd50 2023-12-19T18:17:58.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.145+0000 7ff92a7fc640 1 -- 172.21.15.26:0/2751121521 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c003620 con 0x7ff92c15cd50 2023-12-19T18:17:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 -- 172.21.15.26:0/2751121521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92c15cd50 msgr2=0x7ff92c0b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 --2- 172.21.15.26:0/2751121521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92c15cd50 0x7ff92c0b2210 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0cad70 tx=0x7ff91c01c460 comp rx=0 tx=0).stop 2023-12-19T18:17:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 -- 172.21.15.26:0/2751121521 shutdown_connections 2023-12-19T18:17:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 --2- 172.21.15.26:0/2751121521 >> v2:172.21.15.160:3300/0 conn(0x7ff92c0d0050 0x7ff92c071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 --2- 172.21.15.26:0/2751121521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92c15cd50 0x7ff92c0b2210 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 -- 172.21.15.26:0/2751121521 >> 172.21.15.26:0/2751121521 conn(0x7ff92c0cd210 msgr2=0x7ff92c0cd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 -- 172.21.15.26:0/2751121521 shutdown_connections 2023-12-19T18:17:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 -- 172.21.15.26:0/2751121521 wait complete. 2023-12-19T18:17:58.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 Processor -- start 2023-12-19T18:17:58.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 -- start start 2023-12-19T18:17:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff92c0b2750 0x7ff92c158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff92c0d0050 0x7ff92c0ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92c15cd50 0x7ff92c0aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0b35e0 con 0x7ff92c15cd50 2023-12-19T18:17:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0acbe0 con 0x7ff92c0b2750 2023-12-19T18:17:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff9309cb640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0ac900 con 0x7ff92c0d0050 2023-12-19T18:17:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff92bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92c15cd50 0x7ff92c0aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff92bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92c15cd50 0x7ff92c0aec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41950/0 (socket says 172.21.15.26:41950) 2023-12-19T18:17:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.149+0000 7ff92bfff640 1 -- 172.21.15.26:0/430919911 learned_addr learned my addr 172.21.15.26:0/430919911 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff92affd640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff92c0d0050 0x7ff92c0ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff92b7fe640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff92c0b2750 0x7ff92c158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff92bfff640 1 -- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff92c0d0050 msgr2=0x7ff92c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.156 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff92bfff640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff92c0d0050 0x7ff92c0ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff92bfff640 1 -- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff92c0b2750 msgr2=0x7ff92c158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff92bfff640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff92c0b2750 0x7ff92c158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff92bfff640 1 -- 172.21.15.26:0/430919911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c009420 con 0x7ff92c15cd50 2023-12-19T18:17:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff92affd640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff92c0d0050 0x7ff92c0ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff92bfff640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92c15cd50 0x7ff92c0aec20 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7ff924007e40 tx=0x7ff92400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff913fff640 1 -- 172.21.15.26:0/430919911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924014620 con 0x7ff92c15cd50 2023-12-19T18:17:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff913fff640 1 -- 172.21.15.26:0/430919911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92400ee90 con 0x7ff92c15cd50 2023-12-19T18:17:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff913fff640 1 -- 172.21.15.26:0/430919911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9240149e0 con 0x7ff92c15cd50 2023-12-19T18:17:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff9309cb640 1 -- 172.21.15.26:0/430919911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92c0c6530 con 0x7ff92c15cd50 2023-12-19T18:17:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff9309cb640 1 -- 172.21.15.26:0/430919911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c0ae150 con 0x7ff92c15cd50 2023-12-19T18:17:58.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff9309cb640 1 -- 172.21.15.26:0/430919911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c067400 con 0x7ff92c15cd50 2023-12-19T18:17:58.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.153+0000 7ff913fff640 1 -- 172.21.15.26:0/430919911 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 135) v1 ==== 85376+0+0 (secure 0 0 0) 0x7ff924014b40 con 0x7ff92c15cd50 2023-12-19T18:17:58.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.157+0000 7ff913fff640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff918065c10 0x7ff9180680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.157+0000 7ff913fff640 1 -- 172.21.15.26:0/430919911 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 8107+0+0 (secure 0 0 0) 0x7ff924084820 con 0x7ff92c15cd50 2023-12-19T18:17:58.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.157+0000 7ff92b7fe640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff918065c10 0x7ff9180680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.157+0000 7ff92b7fe640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff918065c10 0x7ff9180680d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0ad000 tx=0x7ff91c008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:58.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.157+0000 7ff913fff640 1 -- 172.21.15.26:0/430919911 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff924051560 con 0x7ff92c15cd50 2023-12-19T18:17:58.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.317+0000 7ff9309cb640 1 -- 172.21.15.26:0/430919911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff92c0b6710 con 0x7ff92c15cd50 2023-12-19T18:17:58.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.321+0000 7ff913fff640 1 -- 172.21.15.26:0/430919911 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+22049 (secure 0 0 0) 0x7ff924054b80 con 0x7ff92c15cd50 2023-12-19T18:17:58.323 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:58.323 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":239,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:55.924334+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:25.867829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:26.330514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:17:37.484211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:17:38.799276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/598924254":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1489104438":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/621398302":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1548583339":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1246495635":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1431434235":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/4035377594":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3900732845":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1388670559":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3872447087":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/332404059":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1786797637":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/563864573":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/2354390435":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/3396499768":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6833/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6832/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/359874050":"2023-12-20T18:17:49.880070+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:58.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.321+0000 7ff9309cb640 1 -- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff918065c10 msgr2=0x7ff9180680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.321+0000 7ff9309cb640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff918065c10 0x7ff9180680d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0ad000 tx=0x7ff91c008040 comp rx=0 tx=0).stop 2023-12-19T18:17:58.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.321+0000 7ff9309cb640 1 -- 172.21.15.26:0/430919911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92c15cd50 msgr2=0x7ff92c0aec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.321+0000 7ff9309cb640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92c15cd50 0x7ff92c0aec20 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7ff924007e40 tx=0x7ff92400e4b0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.321+0000 7ff9309cb640 1 -- 172.21.15.26:0/430919911 shutdown_connections 2023-12-19T18:17:58.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.321+0000 7ff9309cb640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff918065c10 0x7ff9180680d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.321+0000 7ff9309cb640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92c15cd50 0x7ff92c0aec20 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.321+0000 7ff9309cb640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff92c0d0050 0x7ff92c0ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.321+0000 7ff9309cb640 1 --2- 172.21.15.26:0/430919911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff92c0b2750 0x7ff92c158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.321+0000 7ff9309cb640 1 -- 172.21.15.26:0/430919911 >> 172.21.15.26:0/430919911 conn(0x7ff92c0cd210 msgr2=0x7ff92c15bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:58.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.321+0000 7ff9309cb640 1 -- 172.21.15.26:0/430919911 shutdown_connections 2023-12-19T18:17:58.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.321+0000 7ff9309cb640 1 -- 172.21.15.26:0/430919911 wait complete. 2023-12-19T18:17:58.345 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:17:58.345 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:58.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f5245c66640 1 Processor -- start 2023-12-19T18:17:58.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f5245c66640 1 -- start start 2023-12-19T18:17:58.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f5245c66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5240155030 0x7f524015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f5245c66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f52400ce740 0x7f52400ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f5245c66640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52400cf0b0 con 0x7f524015f370 2023-12-19T18:17:58.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f5245c66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524006c790 con 0x7f5240155030 2023-12-19T18:17:58.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f5245c66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524006c970 con 0x7f52400ce740 2023-12-19T18:17:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f523f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5240155030 0x7f524015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f523f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5240155030 0x7f524015bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41964/0 (socket says 172.21.15.26:41964) 2023-12-19T18:17:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f523f7fe640 1 -- 172.21.15.26:0/492846177 learned_addr learned my addr 172.21.15.26:0/492846177 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f523ffff640 1 --2- 172.21.15.26:0/492846177 >> v2:172.21.15.160:3300/0 conn(0x7f52400ce740 0x7f52400ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f523f7fe640 1 -- 172.21.15.26:0/492846177 >> v2:172.21.15.160:3300/0 conn(0x7f52400ce740 msgr2=0x7f52400ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f523f7fe640 1 --2- 172.21.15.26:0/492846177 >> v2:172.21.15.160:3300/0 conn(0x7f52400ce740 0x7f52400ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f523f7fe640 1 -- 172.21.15.26:0/492846177 >> v1:172.21.15.160:6789/0 conn(0x7f524015f370 legacy=0x7f524006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.485+0000 7f523f7fe640 1 -- 172.21.15.26:0/492846177 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524015b1f0 con 0x7f5240155030 2023-12-19T18:17:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.489+0000 7f523f7fe640 1 --2- 172.21.15.26:0/492846177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5240155030 0x7f524015bcd0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f52400cc390 tx=0x7f523001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7ca1bfe0ce70a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.489+0000 7f523effd640 1 -- 172.21.15.26:0/492846177 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52300aa430 con 0x7f5240155030 2023-12-19T18:17:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.489+0000 7f523effd640 1 -- 172.21.15.26:0/492846177 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52300aa590 con 0x7f5240155030 2023-12-19T18:17:58.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.489+0000 7f523effd640 1 -- 172.21.15.26:0/492846177 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52300aa920 con 0x7f5240155030 2023-12-19T18:17:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.489+0000 7f5245c66640 1 -- 172.21.15.26:0/492846177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5240155030 msgr2=0x7f524015bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.489+0000 7f5245c66640 1 --2- 172.21.15.26:0/492846177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5240155030 0x7f524015bcd0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f52400cc390 tx=0x7f523001c4d0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.489+0000 7f5245c66640 1 -- 172.21.15.26:0/492846177 shutdown_connections 2023-12-19T18:17:58.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.489+0000 7f5245c66640 1 --2- 172.21.15.26:0/492846177 >> v2:172.21.15.160:3300/0 conn(0x7f52400ce740 0x7f52400ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.489+0000 7f5245c66640 1 --2- 172.21.15.26:0/492846177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5240155030 0x7f524015bcd0 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.489+0000 7f5245c66640 1 -- 172.21.15.26:0/492846177 >> 172.21.15.26:0/492846177 conn(0x7f52400658e0 msgr2=0x7f5240065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:58.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.489+0000 7f5245c66640 1 -- 172.21.15.26:0/492846177 shutdown_connections 2023-12-19T18:17:58.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.489+0000 7f5245c66640 1 -- 172.21.15.26:0/492846177 wait complete. 2023-12-19T18:17:58.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f5245c66640 1 Processor -- start 2023-12-19T18:17:58.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f5245c66640 1 -- start start 2023-12-19T18:17:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f5245c66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52400ce740 0x7f5240141680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f5245c66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5240155030 0x7f52400b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f5245c66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524015f370 0x7f52400b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f5245c66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52401404a0 con 0x7f524015f370 2023-12-19T18:17:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f5245c66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5240141bc0 con 0x7f5240155030 2023-12-19T18:17:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f5245c66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52400b1bd0 con 0x7f52400ce740 2023-12-19T18:17:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f523ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524015f370 0x7f52400b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f523ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524015f370 0x7f52400b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41978/0 (socket says 172.21.15.26:41978) 2023-12-19T18:17:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f523ffff640 1 -- 172.21.15.26:0/1211669286 learned_addr learned my addr 172.21.15.26:0/1211669286 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:58.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f523f7fe640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52400ce740 0x7f5240141680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f5237fff640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5240155030 0x7f52400b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f523ffff640 1 -- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52400ce740 msgr2=0x7f5240141680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f523ffff640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52400ce740 0x7f5240141680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f523ffff640 1 -- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5240155030 msgr2=0x7f52400b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f523ffff640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5240155030 0x7f52400b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f523ffff640 1 -- 172.21.15.26:0/1211669286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52300094d0 con 0x7f524015f370 2023-12-19T18:17:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f523ffff640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524015f370 0x7f52400b2ed0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f5238007e40 tx=0x7f523800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f523cff9640 1 -- 172.21.15.26:0/1211669286 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238014620 con 0x7f524015f370 2023-12-19T18:17:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f523cff9640 1 -- 172.21.15.26:0/1211669286 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523800ee90 con 0x7f524015f370 2023-12-19T18:17:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f5245c66640 1 -- 172.21.15.26:0/1211669286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52400b1eb0 con 0x7f524015f370 2023-12-19T18:17:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f523cff9640 1 -- 172.21.15.26:0/1211669286 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52380149e0 con 0x7f524015f370 2023-12-19T18:17:58.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.493+0000 7f5245c66640 1 -- 172.21.15.26:0/1211669286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52400b2470 con 0x7f524015f370 2023-12-19T18:17:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.497+0000 7f5245c66640 1 -- 172.21.15.26:0/1211669286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5240074270 con 0x7f524015f370 2023-12-19T18:17:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.497+0000 7f523cff9640 1 -- 172.21.15.26:0/1211669286 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 135) v1 ==== 85376+0+0 (secure 0 0 0) 0x7f5238014b40 con 0x7f524015f370 2023-12-19T18:17:58.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.497+0000 7f523cff9640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5220065c10 0x7f52200680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.497+0000 7f523cff9640 1 -- 172.21.15.26:0/1211669286 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 8107+0+0 (secure 0 0 0) 0x7f52380846f0 con 0x7f524015f370 2023-12-19T18:17:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.497+0000 7f523f7fe640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5220065c10 0x7f52200680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.497+0000 7f523f7fe640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5220065c10 0x7f52200680d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f52300ae000 tx=0x7f52300b5040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:58.505 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.501+0000 7f523cff9640 1 -- 172.21.15.26:0/1211669286 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f52380514e0 con 0x7f524015f370 2023-12-19T18:17:58.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.693+0000 7f5245c66640 1 -- 172.21.15.26:0/1211669286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52400ccc30 con 0x7f524015f370 2023-12-19T18:17:58.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.693+0000 7f523cff9640 1 -- 172.21.15.26:0/1211669286 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5921 (secure 0 0 0) 0x7f5238054b00 con 0x7f524015f370 2023-12-19T18:17:58.697 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:58.698 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2727392302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2727392302},{"type":"v1","addr":"172.21.15.160:6836","nonce":2727392302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1029020739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1029020739},{"type":"v1","addr":"172.21.15.160:6837","nonce":1029020739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:27.325271+0000","modified":"2023-12-19T18:17:30.337993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11427":{"gid":11427,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2285370499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2285370499},{"type":"v1","addr":"172.21.15.26:6835","nonce":2285370499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:39.801489+0000","modified":"2023-12-19T18:17:42.831660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11424":{"gid":11424,"name":"a","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6834/3251191056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3251191056},{"type":"v1","addr":"172.21.15.26:6834","nonce":3251191056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-19T18:17:58.698 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 118 2023-12-19T18:17:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.701+0000 7f5245c66640 1 -- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5220065c10 msgr2=0x7f52200680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.701+0000 7f5245c66640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5220065c10 0x7f52200680d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f52300ae000 tx=0x7f52300b5040 comp rx=0 tx=0).stop 2023-12-19T18:17:58.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.701+0000 7f5245c66640 1 -- 172.21.15.26:0/1211669286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524015f370 msgr2=0x7f52400b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.701+0000 7f5245c66640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524015f370 0x7f52400b2ed0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f5238007e40 tx=0x7f523800e4b0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.701+0000 7f5245c66640 1 -- 172.21.15.26:0/1211669286 shutdown_connections 2023-12-19T18:17:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.701+0000 7f5245c66640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5220065c10 0x7f52200680d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.701+0000 7f5245c66640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f524015f370 0x7f52400b2ed0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.701+0000 7f5245c66640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5240155030 0x7f52400b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.701+0000 7f5245c66640 1 --2- 172.21.15.26:0/1211669286 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52400ce740 0x7f5240141680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.701+0000 7f5245c66640 1 -- 172.21.15.26:0/1211669286 >> 172.21.15.26:0/1211669286 conn(0x7f52400658e0 msgr2=0x7f5240071090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.701+0000 7f5245c66640 1 -- 172.21.15.26:0/1211669286 shutdown_connections 2023-12-19T18:17:58.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.701+0000 7f5245c66640 1 -- 172.21.15.26:0/1211669286 wait complete. 2023-12-19T18:17:58.719 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.861+0000 7f4e98803640 1 Processor -- start 2023-12-19T18:17:58.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.861+0000 7f4e98803640 1 -- start start 2023-12-19T18:17:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e98803640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4e940cc0a0 0x7f4e940cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e98803640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4e94159d20 0x7f4e940cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e98803640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e940adde0 con 0x7f4e940cdee0 2023-12-19T18:17:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e98803640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e940c9c20 con 0x7f4e940cc0a0 2023-12-19T18:17:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e98803640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940b28d0 con 0x7f4e94159d20 2023-12-19T18:17:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e9259c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4e94159d20 0x7f4e940cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e9359e640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f4e940cdee0 0x7f4e9415d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:58324/0 (socket says 172.21.15.26:58324) 2023-12-19T18:17:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e9359e640 1 -- 172.21.15.26:0/1308707659 learned_addr learned my addr 172.21.15.26:0/1308707659 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:58.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e92d9d640 1 --2- 172.21.15.26:0/1308707659 >> v2:172.21.15.160:3300/0 conn(0x7f4e940cc0a0 0x7f4e940cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e91d9b640 1 -- 172.21.15.26:0/1308707659 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257673306 0 0) 0x7f4e940adde0 con 0x7f4e940cdee0 2023-12-19T18:17:58.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e9259c640 1 -- 172.21.15.26:0/1308707659 >> v2:172.21.15.160:3300/0 conn(0x7f4e940cc0a0 msgr2=0x7f4e940cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:58.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e9259c640 1 --2- 172.21.15.26:0/1308707659 >> v2:172.21.15.160:3300/0 conn(0x7f4e940cc0a0 0x7f4e940cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e9259c640 1 -- 172.21.15.26:0/1308707659 >> v1:172.21.15.160:6789/0 conn(0x7f4e940cdee0 legacy=0x7f4e9415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e9259c640 1 -- 172.21.15.26:0/1308707659 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e940ccf00 con 0x7f4e94159d20 2023-12-19T18:17:58.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e9259c640 1 --2- 172.21.15.26:0/1308707659 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4e94159d20 0x7f4e940cc9c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4e880097c0 tx=0x7f4e8801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f01fab7e1decd4dd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:58.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e91d9b640 1 -- 172.21.15.26:0/1308707659 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e880ab600 con 0x7f4e94159d20 2023-12-19T18:17:58.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e91d9b640 1 -- 172.21.15.26:0/1308707659 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e880a7050 con 0x7f4e94159d20 2023-12-19T18:17:58.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e91d9b640 1 -- 172.21.15.26:0/1308707659 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e880ab980 con 0x7f4e94159d20 2023-12-19T18:17:58.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e98803640 1 -- 172.21.15.26:0/1308707659 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4e94159d20 msgr2=0x7f4e940cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e98803640 1 --2- 172.21.15.26:0/1308707659 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4e94159d20 0x7f4e940cc9c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4e880097c0 tx=0x7f4e8801c3c0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e98803640 1 -- 172.21.15.26:0/1308707659 shutdown_connections 2023-12-19T18:17:58.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e98803640 1 --2- 172.21.15.26:0/1308707659 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4e94159d20 0x7f4e940cc9c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e98803640 1 --2- 172.21.15.26:0/1308707659 >> v2:172.21.15.160:3300/0 conn(0x7f4e940cc0a0 0x7f4e940cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e98803640 1 -- 172.21.15.26:0/1308707659 >> 172.21.15.26:0/1308707659 conn(0x7f4e9406b0f0 msgr2=0x7f4e9406b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:58.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e98803640 1 -- 172.21.15.26:0/1308707659 shutdown_connections 2023-12-19T18:17:58.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.865+0000 7f4e98803640 1 -- 172.21.15.26:0/1308707659 wait complete. 2023-12-19T18:17:58.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e98803640 1 Processor -- start 2023-12-19T18:17:58.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e98803640 1 -- start start 2023-12-19T18:17:58.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e98803640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e940cc0a0 0x7f4e94151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e98803640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940cdee0 0x7f4e9414d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e98803640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4e94159d20 0x7f4e94153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e98803640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940ae7f0 con 0x7f4e940cdee0 2023-12-19T18:17:58.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e98803640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e94152a00 con 0x7f4e940cc0a0 2023-12-19T18:17:58.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e98803640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e94152110 con 0x7f4e94159d20 2023-12-19T18:17:58.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e9259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940cdee0 0x7f4e9414d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e9259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940cdee0 0x7f4e9414d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41986/0 (socket says 172.21.15.26:41986) 2023-12-19T18:17:58.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e9259c640 1 -- 172.21.15.26:0/2299926456 learned_addr learned my addr 172.21.15.26:0/2299926456 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:58.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e9359e640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4e94159d20 0x7f4e94153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e92d9d640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e940cc0a0 0x7f4e94151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e9259c640 1 -- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4e94159d20 msgr2=0x7f4e94153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e9259c640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4e94159d20 0x7f4e94153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.874 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e9259c640 1 -- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e940cc0a0 msgr2=0x7f4e94151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e9259c640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e940cc0a0 0x7f4e94151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e9259c640 1 -- 172.21.15.26:0/2299926456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e88009420 con 0x7f4e940cdee0 2023-12-19T18:17:58.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e9359e640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4e94159d20 0x7f4e94153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:58.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e9259c640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940cdee0 0x7f4e9414d540 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f4e880ae040 tx=0x7f4e8801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:58.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e7b7fe640 1 -- 172.21.15.26:0/2299926456 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e880b7450 con 0x7f4e940cdee0 2023-12-19T18:17:58.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e7b7fe640 1 -- 172.21.15.26:0/2299926456 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e880b75b0 con 0x7f4e940cdee0 2023-12-19T18:17:58.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e98803640 1 -- 172.21.15.26:0/2299926456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e94152720 con 0x7f4e940cdee0 2023-12-19T18:17:58.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e7b7fe640 1 -- 172.21.15.26:0/2299926456 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e880b7960 con 0x7f4e940cdee0 2023-12-19T18:17:58.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.869+0000 7f4e98803640 1 -- 172.21.15.26:0/2299926456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e940ae410 con 0x7f4e940cdee0 2023-12-19T18:17:58.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.873+0000 7f4e98803640 1 -- 172.21.15.26:0/2299926456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e940adf90 con 0x7f4e940cdee0 2023-12-19T18:17:58.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.873+0000 7f4e7b7fe640 1 -- 172.21.15.26:0/2299926456 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 135) v1 ==== 85376+0+0 (secure 0 0 0) 0x7f4e880b7ac0 con 0x7f4e940cdee0 2023-12-19T18:17:58.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.873+0000 7f4e7b7fe640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4e80065c10 0x7f4e800680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.873+0000 7f4e7b7fe640 1 -- 172.21.15.26:0/2299926456 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 8107+0+0 (secure 0 0 0) 0x7f4e8811e6d0 con 0x7f4e940cdee0 2023-12-19T18:17:58.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.873+0000 7f4e92d9d640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4e80065c10 0x7f4e800680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.873+0000 7f4e92d9d640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4e80065c10 0x7f4e800680d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400b240 tx=0x7f4e8400b3a0 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:58.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:58.877+0000 7f4e7b7fe640 1 -- 172.21.15.26:0/2299926456 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4e940adf90 con 0x7f4e940cdee0 2023-12-19T18:17:59.076 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.073+0000 7f4e98803640 1 -- 172.21.15.26:0/2299926456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e940ae930 con 0x7f4e940cdee0 2023-12-19T18:17:59.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.073+0000 7f4e7b7fe640 1 -- 172.21.15.26:0/2299926456 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5921 (secure 0 0 0) 0x7f4e940ae930 con 0x7f4e940cdee0 2023-12-19T18:17:59.077 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:59.077 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2727392302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2727392302},{"type":"v1","addr":"172.21.15.160:6836","nonce":2727392302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1029020739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1029020739},{"type":"v1","addr":"172.21.15.160:6837","nonce":1029020739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:27.325271+0000","modified":"2023-12-19T18:17:30.337993+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11427":{"gid":11427,"name":"c","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/2285370499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2285370499},{"type":"v1","addr":"172.21.15.26:6835","nonce":2285370499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:39.801489+0000","modified":"2023-12-19T18:17:42.831660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11424":{"gid":11424,"name":"a","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6834/3251191056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3251191056},{"type":"v1","addr":"172.21.15.26:6834","nonce":3251191056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-19T18:17:59.078 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 118 2023-12-19T18:17:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.073+0000 7f4e98803640 1 -- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4e80065c10 msgr2=0x7f4e800680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.073+0000 7f4e98803640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4e80065c10 0x7f4e800680d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4e8400b240 tx=0x7f4e8400b3a0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.073+0000 7f4e98803640 1 -- 172.21.15.26:0/2299926456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940cdee0 msgr2=0x7f4e9414d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.073+0000 7f4e98803640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940cdee0 0x7f4e9414d540 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f4e880ae040 tx=0x7f4e8801ac00 comp rx=0 tx=0).stop 2023-12-19T18:17:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.077+0000 7f4e98803640 1 -- 172.21.15.26:0/2299926456 shutdown_connections 2023-12-19T18:17:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.077+0000 7f4e98803640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4e80065c10 0x7f4e800680d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.077+0000 7f4e98803640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f4e94159d20 0x7f4e94153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.077+0000 7f4e98803640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940cdee0 0x7f4e9414d540 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.077+0000 7f4e98803640 1 --2- 172.21.15.26:0/2299926456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4e940cc0a0 0x7f4e94151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.077+0000 7f4e98803640 1 -- 172.21.15.26:0/2299926456 >> 172.21.15.26:0/2299926456 conn(0x7f4e9406b0f0 msgr2=0x7f4e940ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.077+0000 7f4e98803640 1 -- 172.21.15.26:0/2299926456 shutdown_connections 2023-12-19T18:17:59.080 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.077+0000 7f4e98803640 1 -- 172.21.15.26:0/2299926456 wait complete. 2023-12-19T18:17:59.099 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 Processor -- start 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 -- start start 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f24280ce040 0x7f24280d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24280d3130 0x7f24280d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280d29d0 con 0x7f242815cd40 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24280d3570 con 0x7f24280ce040 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242815d680 con 0x7f24280d3130 2023-12-19T18:17:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f24277fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24280d3130 0x7f24280d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f2426ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f242815cd40 0x7f2428160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:58332/0 (socket says 172.21.15.26:58332) 2023-12-19T18:17:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f241ffff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f24280ce040 0x7f24280d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f2426ffd640 1 -- 172.21.15.26:0/2699024350 learned_addr learned my addr 172.21.15.26:0/2699024350 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f24277fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24280d3130 0x7f24280d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:51832/0 (socket says 172.21.15.26:51832) 2023-12-19T18:17:59.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f24267fc640 1 -- 172.21.15.26:0/2699024350 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645190135 0 0) 0x7f2418001170 con 0x7f242815cd40 2023-12-19T18:17:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f24267fc640 1 -- 172.21.15.26:0/2699024350 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2414003530 con 0x7f242815cd40 2023-12-19T18:17:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f241ffff640 1 -- 172.21.15.26:0/2699024350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24280d3130 msgr2=0x7f24280d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f241ffff640 1 --2- 172.21.15.26:0/2699024350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24280d3130 0x7f24280d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f241ffff640 1 -- 172.21.15.26:0/2699024350 >> v1:172.21.15.160:6789/0 conn(0x7f242815cd40 legacy=0x7f2428160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f241ffff640 1 -- 172.21.15.26:0/2699024350 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280ce4b0 con 0x7f24280ce040 2023-12-19T18:17:59.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f24277fe640 1 --2- 172.21.15.26:0/2699024350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24280d3130 0x7f24280d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:59.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f241ffff640 1 --2- 172.21.15.26:0/2699024350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ce040 0x7f24280d2bf0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f24100097c0 tx=0x7f241001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa78d7e6bd6d70d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:59.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f24267fc640 1 -- 172.21.15.26:0/2699024350 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24100ab680 con 0x7f24280ce040 2023-12-19T18:17:59.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f24267fc640 1 -- 172.21.15.26:0/2699024350 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24100ab7e0 con 0x7f24280ce040 2023-12-19T18:17:59.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f24267fc640 1 -- 172.21.15.26:0/2699024350 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24100abb70 con 0x7f24280ce040 2023-12-19T18:17:59.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 -- 172.21.15.26:0/2699024350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ce040 msgr2=0x7f24280d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 --2- 172.21.15.26:0/2699024350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ce040 0x7f24280d2bf0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f24100097c0 tx=0x7f241001c480 comp rx=0 tx=0).stop 2023-12-19T18:17:59.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 -- 172.21.15.26:0/2699024350 shutdown_connections 2023-12-19T18:17:59.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 --2- 172.21.15.26:0/2699024350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24280d3130 0x7f24280d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 --2- 172.21.15.26:0/2699024350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24280ce040 0x7f24280d2bf0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 -- 172.21.15.26:0/2699024350 >> 172.21.15.26:0/2699024350 conn(0x7f242806a120 msgr2=0x7f242806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:59.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 -- 172.21.15.26:0/2699024350 shutdown_connections 2023-12-19T18:17:59.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.245+0000 7f242d3f3640 1 -- 172.21.15.26:0/2699024350 wait complete. 2023-12-19T18:17:59.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f242d3f3640 1 Processor -- start 2023-12-19T18:17:59.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f242d3f3640 1 -- start start 2023-12-19T18:17:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f242d3f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24280ce040 0x7f242814cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f242d3f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24280d3130 0x7f24281566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f242d3f3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f242815cd40 0x7f242814d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f242d3f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242814cff0 con 0x7f24280ce040 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f242d3f3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428156c00 con 0x7f242815cd40 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f242d3f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242815c6d0 con 0x7f24280d3130 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f2426ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24280ce040 0x7f242814cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f2426ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24280ce040 0x7f242814cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41998/0 (socket says 172.21.15.26:41998) 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f2426ffd640 1 -- 172.21.15.26:0/558153120 learned_addr learned my addr 172.21.15.26:0/558153120 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f24277fe640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f242815cd40 0x7f242814d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f241ffff640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24280d3130 0x7f24281566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f2426ffd640 1 -- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24280d3130 msgr2=0x7f24281566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f2426ffd640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24280d3130 0x7f24281566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f2426ffd640 1 -- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f242815cd40 msgr2=0x7f242814d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f2426ffd640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f242815cd40 0x7f242814d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f2426ffd640 1 -- 172.21.15.26:0/558153120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2420009a70 con 0x7f24280ce040 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f2426ffd640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24280ce040 0x7f242814cab0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f241800c430 tx=0x7f241800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f241f7fe640 1 -- 172.21.15.26:0/558153120 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418005d40 con 0x7f24280ce040 2023-12-19T18:17:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f241f7fe640 1 -- 172.21.15.26:0/558153120 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24180070d0 con 0x7f24280ce040 2023-12-19T18:17:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f242d3f3640 1 -- 172.21.15.26:0/558153120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2410009420 con 0x7f24280ce040 2023-12-19T18:17:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f241f7fe640 1 -- 172.21.15.26:0/558153120 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418005ea0 con 0x7f24280ce040 2023-12-19T18:17:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f242d3f3640 1 -- 172.21.15.26:0/558153120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2428155900 con 0x7f24280ce040 2023-12-19T18:17:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f242d3f3640 1 -- 172.21.15.26:0/558153120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2428074270 con 0x7f24280ce040 2023-12-19T18:17:59.256 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f241f7fe640 1 -- 172.21.15.26:0/558153120 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 135) v1 ==== 85376+0+0 (secure 0 0 0) 0x7f2418007440 con 0x7f24280ce040 2023-12-19T18:17:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f241f7fe640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2408065c40 0x7f2408068100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f241f7fe640 1 -- 172.21.15.26:0/558153120 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 8107+0+0 (secure 0 0 0) 0x7f2418082fc0 con 0x7f24280ce040 2023-12-19T18:17:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f241ffff640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2408065c40 0x7f2408068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.253+0000 7f241ffff640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2408065c40 0x7f2408068100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f24100af000 tx=0x7f24100a7000 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:59.261 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.257+0000 7f241f7fe640 1 -- 172.21.15.26:0/558153120 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f241804fef0 con 0x7f24280ce040 2023-12-19T18:17:59.424 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.425+0000 7f242d3f3640 1 -- 172.21.15.26:0/558153120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f242814dbf0 con 0x7f24280ce040 2023-12-19T18:17:59.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.425+0000 7f241f7fe640 1 -- 172.21.15.26:0/558153120 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+22049 (secure 0 0 0) 0x7f2418017020 con 0x7f24280ce040 2023-12-19T18:17:59.425 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:17:59.426 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":239,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:17:55.924334+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:25.867829+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"232","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:26.330514+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:17:37.484211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:17:38.799276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/598924254":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1489104438":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/621398302":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1548583339":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1246495635":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1431434235":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/4035377594":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3900732845":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1388670559":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3872447087":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/332404059":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1786797637":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/563864573":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/2354390435":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/3396499768":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6833/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6832/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/359874050":"2023-12-20T18:17:49.880070+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:59.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.429+0000 7f242d3f3640 1 -- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2408065c40 msgr2=0x7f2408068100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.429+0000 7f242d3f3640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2408065c40 0x7f2408068100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f24100af000 tx=0x7f24100a7000 comp rx=0 tx=0).stop 2023-12-19T18:17:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.429+0000 7f242d3f3640 1 -- 172.21.15.26:0/558153120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24280ce040 msgr2=0x7f242814cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.429+0000 7f242d3f3640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24280ce040 0x7f242814cab0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f241800c430 tx=0x7f241800c830 comp rx=0 tx=0).stop 2023-12-19T18:17:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.429+0000 7f242d3f3640 1 -- 172.21.15.26:0/558153120 shutdown_connections 2023-12-19T18:17:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.429+0000 7f242d3f3640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2408065c40 0x7f2408068100 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.429+0000 7f242d3f3640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f242815cd40 0x7f242814d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.429+0000 7f242d3f3640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f24280d3130 0x7f24281566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.429+0000 7f242d3f3640 1 --2- 172.21.15.26:0/558153120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24280ce040 0x7f242814cab0 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.429+0000 7f242d3f3640 1 -- 172.21.15.26:0/558153120 >> 172.21.15.26:0/558153120 conn(0x7f242806a120 msgr2=0x7f2428160890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.429+0000 7f242d3f3640 1 -- 172.21.15.26:0/558153120 shutdown_connections 2023-12-19T18:17:59.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.429+0000 7f242d3f3640 1 -- 172.21.15.26:0/558153120 wait complete. 2023-12-19T18:17:59.447 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:17:59.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f6284da2640 1 Processor -- start 2023-12-19T18:17:59.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f6284da2640 1 -- start start 2023-12-19T18:17:59.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f6284da2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62800cd2c0 0x7f62800cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f6284da2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6280161100 0x7f6280161550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f6284da2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62800cdbe0 con 0x7f62800affe0 2023-12-19T18:17:59.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f6284da2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62801609d0 con 0x7f6280161100 2023-12-19T18:17:59.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f6284da2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628015dc10 con 0x7f62800cd2c0 2023-12-19T18:17:59.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f627ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6280161100 0x7f6280161550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f627ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6280161100 0x7f6280161550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42010/0 (socket says 172.21.15.26:42010) 2023-12-19T18:17:59.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f627ffff640 1 -- 172.21.15.26:0/4083934784 learned_addr learned my addr 172.21.15.26:0/4083934784 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:59.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f627f7fe640 1 --2- 172.21.15.26:0/4083934784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62800cd2c0 0x7f62800cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f627ffff640 1 -- 172.21.15.26:0/4083934784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62800cd2c0 msgr2=0x7f62800cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:59.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f627ffff640 1 --2- 172.21.15.26:0/4083934784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62800cd2c0 0x7f62800cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f627ffff640 1 -- 172.21.15.26:0/4083934784 >> v1:172.21.15.160:6789/0 conn(0x7f62800affe0 legacy=0x7f628015d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.594 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.593+0000 7f627ffff640 1 -- 172.21.15.26:0/4083934784 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628015def0 con 0x7f6280161100 2023-12-19T18:17:59.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.597+0000 7f627ffff640 1 --2- 172.21.15.26:0/4083934784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6280161100 0x7f6280161550 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f6278009e50 tx=0x7f627801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6929fcb0e4b515 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:59.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.597+0000 7f627e7fc640 1 -- 172.21.15.26:0/4083934784 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780a5810 con 0x7f6280161100 2023-12-19T18:17:59.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.597+0000 7f627e7fc640 1 -- 172.21.15.26:0/4083934784 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62780a5970 con 0x7f6280161100 2023-12-19T18:17:59.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.597+0000 7f627e7fc640 1 -- 172.21.15.26:0/4083934784 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62780a5d00 con 0x7f6280161100 2023-12-19T18:17:59.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.597+0000 7f6284da2640 1 -- 172.21.15.26:0/4083934784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6280161100 msgr2=0x7f6280161550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.597+0000 7f6284da2640 1 --2- 172.21.15.26:0/4083934784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6280161100 0x7f6280161550 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f6278009e50 tx=0x7f627801c950 comp rx=0 tx=0).stop 2023-12-19T18:17:59.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.597+0000 7f6284da2640 1 -- 172.21.15.26:0/4083934784 shutdown_connections 2023-12-19T18:17:59.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.597+0000 7f6284da2640 1 --2- 172.21.15.26:0/4083934784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6280161100 0x7f6280161550 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.597+0000 7f6284da2640 1 --2- 172.21.15.26:0/4083934784 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f62800cd2c0 0x7f62800cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.597+0000 7f6284da2640 1 -- 172.21.15.26:0/4083934784 >> 172.21.15.26:0/4083934784 conn(0x7f628006b0f0 msgr2=0x7f628006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:59.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.597+0000 7f6284da2640 1 -- 172.21.15.26:0/4083934784 shutdown_connections 2023-12-19T18:17:59.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.597+0000 7f6284da2640 1 -- 172.21.15.26:0/4083934784 wait complete. 2023-12-19T18:17:59.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f6284da2640 1 Processor -- start 2023-12-19T18:17:59.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f6284da2640 1 -- start start 2023-12-19T18:17:59.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f6284da2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62800affe0 0x7f6280071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f6284da2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62800cd2c0 0x7f62800c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f6284da2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6280161100 0x7f628015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f6284da2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6280070a40 con 0x7f62800affe0 2023-12-19T18:17:59.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f6284da2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628015b7f0 con 0x7f62800cd2c0 2023-12-19T18:17:59.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f6284da2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62800c8420 con 0x7f6280161100 2023-12-19T18:17:59.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f627f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62800affe0 0x7f6280071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f627f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62800affe0 0x7f6280071a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42026/0 (socket says 172.21.15.26:42026) 2023-12-19T18:17:59.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f627f7fe640 1 -- 172.21.15.26:0/2669281061 learned_addr learned my addr 172.21.15.26:0/2669281061 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:59.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f627effd640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62800cd2c0 0x7f62800c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f627ffff640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6280161100 0x7f628015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f627f7fe640 1 -- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6280161100 msgr2=0x7f628015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f627f7fe640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6280161100 0x7f628015b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f627f7fe640 1 -- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62800cd2c0 msgr2=0x7f62800c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f627f7fe640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62800cd2c0 0x7f62800c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f627f7fe640 1 -- 172.21.15.26:0/2669281061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6278009b30 con 0x7f62800affe0 2023-12-19T18:17:59.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f627effd640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62800cd2c0 0x7f62800c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:59.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f627ffff640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6280161100 0x7f628015b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:59.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f627f7fe640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62800affe0 0x7f6280071a10 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f627000e960 tx=0x7f627000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:59.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f6267fff640 1 -- 172.21.15.26:0/2669281061 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6270009690 con 0x7f62800affe0 2023-12-19T18:17:59.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f6267fff640 1 -- 172.21.15.26:0/2669281061 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627000c9a0 con 0x7f62800affe0 2023-12-19T18:17:59.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f6267fff640 1 -- 172.21.15.26:0/2669281061 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627000cd10 con 0x7f62800affe0 2023-12-19T18:17:59.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f6284da2640 1 -- 172.21.15.26:0/2669281061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6280070c50 con 0x7f62800affe0 2023-12-19T18:17:59.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.601+0000 7f6284da2640 1 -- 172.21.15.26:0/2669281061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6280071210 con 0x7f62800affe0 2023-12-19T18:17:59.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.605+0000 7f6284da2640 1 -- 172.21.15.26:0/2669281061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6280067400 con 0x7f62800affe0 2023-12-19T18:17:59.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.605+0000 7f6267fff640 1 -- 172.21.15.26:0/2669281061 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 135) v1 ==== 85376+0+0 (secure 0 0 0) 0x7f6270013470 con 0x7f62800affe0 2023-12-19T18:17:59.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.605+0000 7f6267fff640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f626c065c10 0x7f626c0680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.604 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.605+0000 7f6267fff640 1 -- 172.21.15.26:0/2669281061 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 8107+0+0 (secure 0 0 0) 0x7f62700840c0 con 0x7f62800affe0 2023-12-19T18:17:59.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.605+0000 7f627effd640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f626c065c10 0x7f626c0680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.605 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.605+0000 7f627effd640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f626c065c10 0x7f626c0680d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f62800ad130 tx=0x7f6274009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:59.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.609+0000 7f6267fff640 1 -- 172.21.15.26:0/2669281061 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6270050e00 con 0x7f62800affe0 2023-12-19T18:17:59.799 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.801+0000 7f6284da2640 1 -- 172.21.15.26:0/2669281061 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f62800c1ca0 con 0x7f62800affe0 2023-12-19T18:17:59.800 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T18:17:59.801+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:17:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.813+0000 7f6267fff640 1 -- 172.21.15.26:0/2669281061 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v119) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6270017090 con 0x7f62800affe0 2023-12-19T18:17:59.814 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:17:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.817+0000 7f6284da2640 1 -- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f626c065c10 msgr2=0x7f626c0680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.817+0000 7f6284da2640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f626c065c10 0x7f626c0680d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f62800ad130 tx=0x7f6274009040 comp rx=0 tx=0).stop 2023-12-19T18:17:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.817+0000 7f6284da2640 1 -- 172.21.15.26:0/2669281061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62800affe0 msgr2=0x7f6280071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.817+0000 7f6284da2640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62800affe0 0x7f6280071a10 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f627000e960 tx=0x7f627000ee20 comp rx=0 tx=0).stop 2023-12-19T18:17:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.817+0000 7f6284da2640 1 -- 172.21.15.26:0/2669281061 shutdown_connections 2023-12-19T18:17:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.817+0000 7f6284da2640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f626c065c10 0x7f626c0680d0 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f62800ad130 tx=0x7f6274009040 comp rx=0 tx=0).stop 2023-12-19T18:17:59.816 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.817+0000 7f6284da2640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6280161100 0x7f628015b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.817+0000 7f6284da2640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62800cd2c0 0x7f62800c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.817+0000 7f6284da2640 1 --2- 172.21.15.26:0/2669281061 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f62800affe0 0x7f6280071a10 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.817+0000 7f6284da2640 1 -- 172.21.15.26:0/2669281061 >> 172.21.15.26:0/2669281061 conn(0x7f628006b0f0 msgr2=0x7f628015d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.817+0000 7f6284da2640 1 -- 172.21.15.26:0/2669281061 shutdown_connections 2023-12-19T18:17:59.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.817+0000 7f6284da2640 1 -- 172.21.15.26:0/2669281061 wait complete. 2023-12-19T18:17:59.821 INFO:tasks.ceph.mds.c.smithi026.stderr: -4170> 2023-12-19T18:17:28.825+0000 7f71716d6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:17:59.839 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:17:59.878 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T18:17:59.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10ac3bc640 1 Processor -- start 2023-12-19T18:17:59.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10ac3bc640 1 -- start start 2023-12-19T18:17:59.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10ac3bc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f10a41599e0 0x7f10a406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10ac3bc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a415e240 con 0x7f10a415dc40 2023-12-19T18:17:59.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10ac3bc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a40c9520 con 0x7f10a40cb150 2023-12-19T18:17:59.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10ac3bc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a415e050 con 0x7f10a41599e0 2023-12-19T18:17:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10aa959640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f10a415dc40 0x7f10a40ca000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34798/0 (socket says 172.21.15.26:34798) 2023-12-19T18:17:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10aa959640 1 -- 172.21.15.26:0/2011650387 learned_addr learned my addr 172.21.15.26:0/2011650387 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:17:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10aa158640 1 --2- 172.21.15.26:0/2011650387 >> v2:172.21.15.160:3300/0 conn(0x7f10a41599e0 0x7f10a406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10a9156640 1 -- 172.21.15.26:0/2011650387 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168133972 0 0) 0x7f10a415e240 con 0x7f10a415dc40 2023-12-19T18:17:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10a9156640 1 -- 172.21.15.26:0/2011650387 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1090003530 con 0x7f10a415dc40 2023-12-19T18:17:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10a9156640 1 -- 172.21.15.26:0/2011650387 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191914117 0 0) 0x7f10a40c9520 con 0x7f10a40cb150 2023-12-19T18:17:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10a9156640 1 -- 172.21.15.26:0/2011650387 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10a415e240 con 0x7f10a40cb150 2023-12-19T18:17:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10aa158640 1 -- 172.21.15.26:0/2011650387 >> v1:172.21.15.160:6789/0 conn(0x7f10a40cb150 legacy=0x7f10a415d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10aa158640 1 -- 172.21.15.26:0/2011650387 >> v1:172.21.15.26:6789/0 conn(0x7f10a415dc40 legacy=0x7f10a40ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.989+0000 7f10aa158640 1 -- 172.21.15.26:0/2011650387 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a40c9800 con 0x7f10a41599e0 2023-12-19T18:17:59.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.993+0000 7f10aa158640 1 --2- 172.21.15.26:0/2011650387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a41599e0 0x7f10a406f6d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f10a00097c0 tx=0x7f10a001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cb58f6ff4e38e8a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:59.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.993+0000 7f10a9156640 1 -- 172.21.15.26:0/2011650387 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a40caf20 con 0x7f10a41599e0 2023-12-19T18:17:59.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.993+0000 7f10a9156640 1 -- 172.21.15.26:0/2011650387 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a00a7070 con 0x7f10a41599e0 2023-12-19T18:17:59.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.993+0000 7f10a9156640 1 -- 172.21.15.26:0/2011650387 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a00ab840 con 0x7f10a41599e0 2023-12-19T18:17:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.993+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2011650387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a41599e0 msgr2=0x7f10a406f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.993+0000 7f10ac3bc640 1 --2- 172.21.15.26:0/2011650387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a41599e0 0x7f10a406f6d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f10a00097c0 tx=0x7f10a001c480 comp rx=0 tx=0).stop 2023-12-19T18:17:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.993+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2011650387 shutdown_connections 2023-12-19T18:17:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.993+0000 7f10ac3bc640 1 --2- 172.21.15.26:0/2011650387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a41599e0 0x7f10a406f6d0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.993+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2011650387 >> 172.21.15.26:0/2011650387 conn(0x7f10a4066160 msgr2=0x7f10a40b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.993+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2011650387 shutdown_connections 2023-12-19T18:17:59.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.993+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2011650387 wait complete. 2023-12-19T18:17:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10ac3bc640 1 Processor -- start 2023-12-19T18:17:59.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10ac3bc640 1 -- start start 2023-12-19T18:17:59.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10ac3bc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a40cb150 0x7f10a415ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10ac3bc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10a41599e0 0x7f10a40c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10ac3bc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10a415dc40 0x7f10a414f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10ac3bc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40c4050 con 0x7f10a41599e0 2023-12-19T18:17:59.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10ac3bc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40c31c0 con 0x7f10a40cb150 2023-12-19T18:17:59.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10ac3bc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40c3440 con 0x7f10a415dc40 2023-12-19T18:17:59.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10a9957640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10a41599e0 0x7f10a40c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10a9957640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10a41599e0 0x7f10a40c2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42046/0 (socket says 172.21.15.26:42046) 2023-12-19T18:17:59.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10a9957640 1 -- 172.21.15.26:0/2795352923 learned_addr learned my addr 172.21.15.26:0/2795352923 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:17:59.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10aa158640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a40cb150 0x7f10a415ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10aa959640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10a415dc40 0x7f10a414f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10a9957640 1 -- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10a415dc40 msgr2=0x7f10a414f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10a9957640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10a415dc40 0x7f10a414f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10a9957640 1 -- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a40cb150 msgr2=0x7f10a415ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10a9957640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a40cb150 0x7f10a415ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10a9957640 1 -- 172.21.15.26:0/2795352923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a0009420 con 0x7f10a41599e0 2023-12-19T18:17:59.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10aa959640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10a415dc40 0x7f10a414f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:59.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10a9957640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10a41599e0 0x7f10a40c2c80 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f109400c530 tx=0x7f109400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:59.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f109affd640 1 -- 172.21.15.26:0/2795352923 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10940052d0 con 0x7f10a41599e0 2023-12-19T18:18:00.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f109affd640 1 -- 172.21.15.26:0/2795352923 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1094005450 con 0x7f10a41599e0 2023-12-19T18:18:00.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2795352923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a40cbaf0 con 0x7f10a41599e0 2023-12-19T18:18:00.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f109affd640 1 -- 172.21.15.26:0/2795352923 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094012870 con 0x7f10a41599e0 2023-12-19T18:18:00.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:17:59.997+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2795352923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a40c3a00 con 0x7f10a41599e0 2023-12-19T18:18:00.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.001+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2795352923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a40c41c0 con 0x7f10a41599e0 2023-12-19T18:18:00.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.001+0000 7f109affd640 1 -- 172.21.15.26:0/2795352923 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 135) v1 ==== 85376+0+0 (secure 0 0 0) 0x7f10940129d0 con 0x7f10a41599e0 2023-12-19T18:18:00.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.001+0000 7f109affd640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1088065c10 0x7f10880680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:00.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.001+0000 7f109affd640 1 -- 172.21.15.26:0/2795352923 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 8193+0+0 (secure 0 0 0) 0x7f10940839c0 con 0x7f10a41599e0 2023-12-19T18:18:00.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.001+0000 7f10aa158640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1088065c10 0x7f10880680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:00.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.001+0000 7f10aa158640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1088065c10 0x7f10880680d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f10a00af000 tx=0x7f10a00a7000 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:00.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.005+0000 7f109affd640 1 -- 172.21.15.26:0/2795352923 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f10940506a0 con 0x7f10a41599e0 2023-12-19T18:18:00.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.197+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2795352923 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f10a4074270 con 0x7f10a41599e0 2023-12-19T18:18:00.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.817+0000 7f109affd640 1 -- 172.21.15.26:0/2795352923 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v120) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1094053cc0 con 0x7f10a41599e0 2023-12-19T18:18:00.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.821+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1088065c10 msgr2=0x7f10880680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:00.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.821+0000 7f10ac3bc640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1088065c10 0x7f10880680d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f10a00af000 tx=0x7f10a00a7000 comp rx=0 tx=0).stop 2023-12-19T18:18:00.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.821+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2795352923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10a41599e0 msgr2=0x7f10a40c2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:00.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.821+0000 7f10ac3bc640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10a41599e0 0x7f10a40c2c80 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f109400c530 tx=0x7f109400c9f0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.821+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2795352923 shutdown_connections 2023-12-19T18:18:00.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.821+0000 7f10ac3bc640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1088065c10 0x7f10880680d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.821+0000 7f10ac3bc640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f10a415dc40 0x7f10a414f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.821+0000 7f10ac3bc640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10a41599e0 0x7f10a40c2c80 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.821+0000 7f10ac3bc640 1 --2- 172.21.15.26:0/2795352923 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f10a40cb150 0x7f10a415ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.821+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2795352923 >> 172.21.15.26:0/2795352923 conn(0x7f10a4066160 msgr2=0x7f10a40ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:00.821 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.821+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2795352923 shutdown_connections 2023-12-19T18:18:00.822 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:00.821+0000 7f10ac3bc640 1 -- 172.21.15.26:0/2795352923 wait complete. 2023-12-19T18:18:00.848 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:18:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff4843b3640 1 Processor -- start 2023-12-19T18:18:01.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff4843b3640 1 -- start start 2023-12-19T18:18:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff4843b3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff47c159830 0x7ff47c15d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff4843b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff47c0cb110 0x7ff47c0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff4843b3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47c15e3d0 con 0x7ff47c0c9310 2023-12-19T18:18:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff4843b3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47c06f7c0 con 0x7ff47c159830 2023-12-19T18:18:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff4843b3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47c0ca2b0 con 0x7ff47c0cb110 2023-12-19T18:18:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff48214f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff47c159830 0x7ff47c15d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff48214f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff47c159830 0x7ff47c15d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42062/0 (socket says 172.21.15.26:42062) 2023-12-19T18:18:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff48214f640 1 -- 172.21.15.26:0/329953559 learned_addr learned my addr 172.21.15.26:0/329953559 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff482950640 1 --2- 172.21.15.26:0/329953559 >> v2:172.21.15.160:3300/0 conn(0x7ff47c0cb110 0x7ff47c0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff48214f640 1 -- 172.21.15.26:0/329953559 >> v2:172.21.15.160:3300/0 conn(0x7ff47c0cb110 msgr2=0x7ff47c0cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff48214f640 1 --2- 172.21.15.26:0/329953559 >> v2:172.21.15.160:3300/0 conn(0x7ff47c0cb110 0x7ff47c0cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:01.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff48214f640 1 -- 172.21.15.26:0/329953559 >> v1:172.21.15.160:6789/0 conn(0x7ff47c0c9310 legacy=0x7ff47c0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:01.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.045+0000 7ff48214f640 1 -- 172.21.15.26:0/329953559 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c0cba80 con 0x7ff47c159830 2023-12-19T18:18:01.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.049+0000 7ff48214f640 1 --2- 172.21.15.26:0/329953559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47c159830 0x7ff47c15d4e0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0ae070 tx=0x7ff47801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da099b49f1413188 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.049+0000 7ff48114d640 1 -- 172.21.15.26:0/329953559 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4780ab600 con 0x7ff47c159830 2023-12-19T18:18:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.049+0000 7ff48114d640 1 -- 172.21.15.26:0/329953559 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4780a7050 con 0x7ff47c159830 2023-12-19T18:18:01.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.049+0000 7ff48114d640 1 -- 172.21.15.26:0/329953559 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4780ab980 con 0x7ff47c159830 2023-12-19T18:18:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.049+0000 7ff4843b3640 1 -- 172.21.15.26:0/329953559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47c159830 msgr2=0x7ff47c15d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.049+0000 7ff4843b3640 1 --2- 172.21.15.26:0/329953559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47c159830 0x7ff47c15d4e0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7ff47c0ae070 tx=0x7ff47801c430 comp rx=0 tx=0).stop 2023-12-19T18:18:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.049+0000 7ff4843b3640 1 -- 172.21.15.26:0/329953559 shutdown_connections 2023-12-19T18:18:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.049+0000 7ff4843b3640 1 --2- 172.21.15.26:0/329953559 >> v2:172.21.15.160:3300/0 conn(0x7ff47c0cb110 0x7ff47c0cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.049+0000 7ff4843b3640 1 --2- 172.21.15.26:0/329953559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47c159830 0x7ff47c15d4e0 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.049+0000 7ff4843b3640 1 -- 172.21.15.26:0/329953559 >> 172.21.15.26:0/329953559 conn(0x7ff47c0b0270 msgr2=0x7ff47c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:01.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.049+0000 7ff4843b3640 1 -- 172.21.15.26:0/329953559 shutdown_connections 2023-12-19T18:18:01.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.049+0000 7ff4843b3640 1 -- 172.21.15.26:0/329953559 wait complete. 2023-12-19T18:18:01.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff4843b3640 1 Processor -- start 2023-12-19T18:18:01.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff4843b3640 1 -- start start 2023-12-19T18:18:01.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff4843b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff47c0c9310 0x7ff47c15c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff4843b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff47c0cb110 0x7ff47c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff4843b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47c159830 0x7ff47c15ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff4843b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0c3e30 con 0x7ff47c159830 2023-12-19T18:18:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff4843b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0c3220 con 0x7ff47c0cb110 2023-12-19T18:18:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff4843b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0c3500 con 0x7ff47c0c9310 2023-12-19T18:18:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff482950640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47c159830 0x7ff47c15ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff482950640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47c159830 0x7ff47c15ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42078/0 (socket says 172.21.15.26:42078) 2023-12-19T18:18:01.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff482950640 1 -- 172.21.15.26:0/3702178012 learned_addr learned my addr 172.21.15.26:0/3702178012 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff48214f640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff47c0c9310 0x7ff47c15c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff48194e640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff47c0cb110 0x7ff47c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:01.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff482950640 1 -- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff47c0c9310 msgr2=0x7ff47c15c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff482950640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff47c0c9310 0x7ff47c15c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff482950640 1 -- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff47c0cb110 msgr2=0x7ff47c0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff482950640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff47c0cb110 0x7ff47c0aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff482950640 1 -- 172.21.15.26:0/3702178012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4780094d0 con 0x7ff47c159830 2023-12-19T18:18:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff482950640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47c159830 0x7ff47c15ac60 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7ff474007e40 tx=0x7ff47400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff472ffd640 1 -- 172.21.15.26:0/3702178012 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474014620 con 0x7ff47c159830 2023-12-19T18:18:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff472ffd640 1 -- 172.21.15.26:0/3702178012 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff47400ee90 con 0x7ff47c159830 2023-12-19T18:18:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff472ffd640 1 -- 172.21.15.26:0/3702178012 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4740149e0 con 0x7ff47c159830 2023-12-19T18:18:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff4843b3640 1 -- 172.21.15.26:0/3702178012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47c0c37e0 con 0x7ff47c159830 2023-12-19T18:18:01.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff4843b3640 1 -- 172.21.15.26:0/3702178012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47c15c9d0 con 0x7ff47c159830 2023-12-19T18:18:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.053+0000 7ff4843b3640 1 -- 172.21.15.26:0/3702178012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47c0c3fa0 con 0x7ff47c159830 2023-12-19T18:18:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.057+0000 7ff472ffd640 1 -- 172.21.15.26:0/3702178012 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ff474014b40 con 0x7ff47c159830 2023-12-19T18:18:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.057+0000 7ff472ffd640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff460065ca0 0x7ff460068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:01.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.057+0000 7ff472ffd640 1 -- 172.21.15.26:0/3702178012 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 8193+0+0 (secure 0 0 0) 0x7ff474084850 con 0x7ff47c159830 2023-12-19T18:18:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.057+0000 7ff48214f640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff460065ca0 0x7ff460068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:01.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.057+0000 7ff48214f640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff460065ca0 0x7ff460068160 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff4780ad000 tx=0x7ff478008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:01.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.061+0000 7ff472ffd640 1 -- 172.21.15.26:0/3702178012 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff4740515a0 con 0x7ff47c159830 2023-12-19T18:18:01.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:01.221+0000 7ff4843b3640 1 -- 172.21.15.26:0/3702178012 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff47c06fa30 con 0x7ff47c159830 2023-12-19T18:18:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.156+0000 7ff472ffd640 1 -- 172.21.15.26:0/3702178012 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v241) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff474054bc0 con 0x7ff47c159830 2023-12-19T18:18:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:18:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.160+0000 7ff4843b3640 1 -- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff460065ca0 msgr2=0x7ff460068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.160+0000 7ff4843b3640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff460065ca0 0x7ff460068160 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff4780ad000 tx=0x7ff478008040 comp rx=0 tx=0).stop 2023-12-19T18:18:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.160+0000 7ff4843b3640 1 -- 172.21.15.26:0/3702178012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47c159830 msgr2=0x7ff47c15ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.160+0000 7ff4843b3640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47c159830 0x7ff47c15ac60 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7ff474007e40 tx=0x7ff47400e4b0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.164+0000 7ff4843b3640 1 -- 172.21.15.26:0/3702178012 shutdown_connections 2023-12-19T18:18:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.164+0000 7ff4843b3640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff460065ca0 0x7ff460068160 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.164+0000 7ff4843b3640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff47c159830 0x7ff47c15ac60 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.164+0000 7ff4843b3640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff47c0cb110 0x7ff47c0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.164+0000 7ff4843b3640 1 --2- 172.21.15.26:0/3702178012 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff47c0c9310 0x7ff47c15c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.164+0000 7ff4843b3640 1 -- 172.21.15.26:0/3702178012 >> 172.21.15.26:0/3702178012 conn(0x7ff47c0b0270 msgr2=0x7ff47c0afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.164+0000 7ff4843b3640 1 -- 172.21.15.26:0/3702178012 shutdown_connections 2023-12-19T18:18:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.164+0000 7ff4843b3640 1 -- 172.21.15.26:0/3702178012 wait complete. 2023-12-19T18:18:02.189 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:18:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.340+0000 7f50360b6640 1 Processor -- start 2023-12-19T18:18:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.340+0000 7f50360b6640 1 -- start start 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.340+0000 7f50360b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503015cbb0 0x7f50301612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.340+0000 7f50360b6640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f503015cf90 0x7f50300cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.340+0000 7f50360b6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50300cdcc0 0x7f50300cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.340+0000 7f50360b6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50300cd810 con 0x7f50300cdcc0 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.340+0000 7f50360b6640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5030161810 con 0x7f503015cf90 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.340+0000 7f50360b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50300ce180 con 0x7f503015cbb0 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50300cdcc0 0x7f50300cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50300cdcc0 0x7f50300cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42084/0 (socket says 172.21.15.26:42084) 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502ffff640 1 -- 172.21.15.26:0/97886104 learned_addr learned my addr 172.21.15.26:0/97886104 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502effd640 1 --2- 172.21.15.26:0/97886104 >> v2:172.21.15.160:3300/0 conn(0x7f503015cf90 0x7f50300cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502f7fe640 1 --2- 172.21.15.26:0/97886104 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503015cbb0 0x7f50301612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502ffff640 1 -- 172.21.15.26:0/97886104 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503015cbb0 msgr2=0x7f50301612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502ffff640 1 --2- 172.21.15.26:0/97886104 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503015cbb0 0x7f50301612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502ffff640 1 -- 172.21.15.26:0/97886104 >> v2:172.21.15.160:3300/0 conn(0x7f503015cf90 msgr2=0x7f50300cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502ffff640 1 --2- 172.21.15.26:0/97886104 >> v2:172.21.15.160:3300/0 conn(0x7f503015cf90 0x7f50300cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502ffff640 1 -- 172.21.15.26:0/97886104 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50300d0ef0 con 0x7f50300cdcc0 2023-12-19T18:18:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502ffff640 1 --2- 172.21.15.26:0/97886104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50300cdcc0 0x7f50300cfac0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f50300b57b0 tx=0x7f502801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817e1b7076808ebe server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:02.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502e7fc640 1 -- 172.21.15.26:0/97886104 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50280a5790 con 0x7f50300cdcc0 2023-12-19T18:18:02.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502e7fc640 1 -- 172.21.15.26:0/97886104 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50280a58f0 con 0x7f50300cdcc0 2023-12-19T18:18:02.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.344+0000 7f502e7fc640 1 -- 172.21.15.26:0/97886104 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50280a5c80 con 0x7f50300cdcc0 2023-12-19T18:18:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 -- 172.21.15.26:0/97886104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50300cdcc0 msgr2=0x7f50300cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 --2- 172.21.15.26:0/97886104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50300cdcc0 0x7f50300cfac0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f50300b57b0 tx=0x7f502801c910 comp rx=0 tx=0).stop 2023-12-19T18:18:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 -- 172.21.15.26:0/97886104 shutdown_connections 2023-12-19T18:18:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 --2- 172.21.15.26:0/97886104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50300cdcc0 0x7f50300cfac0 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 --2- 172.21.15.26:0/97886104 >> v2:172.21.15.160:3300/0 conn(0x7f503015cf90 0x7f50300cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 --2- 172.21.15.26:0/97886104 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503015cbb0 0x7f50301612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 -- 172.21.15.26:0/97886104 >> 172.21.15.26:0/97886104 conn(0x7f50300658e0 msgr2=0x7f5030065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 -- 172.21.15.26:0/97886104 shutdown_connections 2023-12-19T18:18:02.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 -- 172.21.15.26:0/97886104 wait complete. 2023-12-19T18:18:02.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 Processor -- start 2023-12-19T18:18:02.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 -- start start 2023-12-19T18:18:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50300cdcc0 0x7f50300c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503015cbb0 0x7f50300b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503015cf90 0x7f50300c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503014d4e0 con 0x7f503015cbb0 2023-12-19T18:18:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50300c50f0 con 0x7f50300cdcc0 2023-12-19T18:18:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f50360b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5030152560 con 0x7f503015cf90 2023-12-19T18:18:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f502effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503015cbb0 0x7f50300b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f502effd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503015cbb0 0x7f50300b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42090/0 (socket says 172.21.15.26:42090) 2023-12-19T18:18:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.348+0000 7f502effd640 1 -- 172.21.15.26:0/3473994173 learned_addr learned my addr 172.21.15.26:0/3473994173 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f502ffff640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503015cf90 0x7f50300c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f502f7fe640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50300cdcc0 0x7f50300c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f502effd640 1 -- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503015cf90 msgr2=0x7f50300c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f502effd640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503015cf90 0x7f50300c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f502effd640 1 -- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50300cdcc0 msgr2=0x7f50300c5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f502effd640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50300cdcc0 0x7f50300c5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f502effd640 1 -- 172.21.15.26:0/3473994173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5028009b30 con 0x7f503015cbb0 2023-12-19T18:18:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f502effd640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503015cbb0 0x7f50300b1d40 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f502400d5f0 tx=0x7f502400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:02.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f5017fff640 1 -- 172.21.15.26:0/3473994173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50240125f0 con 0x7f503015cbb0 2023-12-19T18:18:02.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f5017fff640 1 -- 172.21.15.26:0/3473994173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5024012750 con 0x7f503015cbb0 2023-12-19T18:18:02.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f5017fff640 1 -- 172.21.15.26:0/3473994173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024012ae0 con 0x7f503015cbb0 2023-12-19T18:18:02.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f50360b6640 1 -- 172.21.15.26:0/3473994173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5030152840 con 0x7f503015cbb0 2023-12-19T18:18:02.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f50360b6640 1 -- 172.21.15.26:0/3473994173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503014d6f0 con 0x7f503015cbb0 2023-12-19T18:18:02.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f50360b6640 1 -- 172.21.15.26:0/3473994173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5030074270 con 0x7f503015cbb0 2023-12-19T18:18:02.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f5017fff640 1 -- 172.21.15.26:0/3473994173 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f5024012c40 con 0x7f503015cbb0 2023-12-19T18:18:02.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f5017fff640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f501c065ca0 0x7f501c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.352+0000 7f5017fff640 1 -- 172.21.15.26:0/3473994173 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 7755+0+0 (secure 0 0 0) 0x7f50240842d0 con 0x7f503015cbb0 2023-12-19T18:18:02.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.356+0000 7f502f7fe640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f501c065ca0 0x7f501c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.356+0000 7f502f7fe640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f501c065ca0 0x7f501c068160 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5030070310 tx=0x7f5020008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:02.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.356+0000 7f5017fff640 1 -- 172.21.15.26:0/3473994173 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5024051120 con 0x7f503015cbb0 2023-12-19T18:18:02.518 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:02.516+0000 7f50360b6640 1 -- 172.21.15.26:0/3473994173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f50300cd090 con 0x7f503015cbb0 2023-12-19T18:18:03.162 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.160+0000 7f5017fff640 1 -- 172.21.15.26:0/3473994173 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v242) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5024054740 con 0x7f503015cbb0 2023-12-19T18:18:03.162 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T18:18:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.164+0000 7f50360b6640 1 -- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f501c065ca0 msgr2=0x7f501c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.164+0000 7f50360b6640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f501c065ca0 0x7f501c068160 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5030070310 tx=0x7f5020008040 comp rx=0 tx=0).stop 2023-12-19T18:18:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.164+0000 7f50360b6640 1 -- 172.21.15.26:0/3473994173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503015cbb0 msgr2=0x7f50300b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.164+0000 7f50360b6640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503015cbb0 0x7f50300b1d40 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f502400d5f0 tx=0x7f502400dab0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.164+0000 7f50360b6640 1 -- 172.21.15.26:0/3473994173 shutdown_connections 2023-12-19T18:18:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.164+0000 7f50360b6640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f501c065ca0 0x7f501c068160 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.164+0000 7f50360b6640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f503015cf90 0x7f50300c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.165 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.164+0000 7f50360b6640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f503015cbb0 0x7f50300b1d40 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.164+0000 7f50360b6640 1 --2- 172.21.15.26:0/3473994173 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50300cdcc0 0x7f50300c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.164+0000 7f50360b6640 1 -- 172.21.15.26:0/3473994173 >> 172.21.15.26:0/3473994173 conn(0x7f50300658e0 msgr2=0x7f50300d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:03.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.164+0000 7f50360b6640 1 -- 172.21.15.26:0/3473994173 shutdown_connections 2023-12-19T18:18:03.166 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.164+0000 7f50360b6640 1 -- 172.21.15.26:0/3473994173 wait complete. 2023-12-19T18:18:03.188 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:03.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.336+0000 7fd036a55640 1 Processor -- start 2023-12-19T18:18:03.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd036a55640 1 -- start start 2023-12-19T18:18:03.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd036a55640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd0300cb1d0 0x7fd0300cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd036a55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03006bee0 0x7fd03006c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd036a55640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd030164c90 0x7fd030165120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd036a55640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0300cc030 con 0x7fd030164c90 2023-12-19T18:18:03.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd036a55640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03006d4b0 con 0x7fd0300cb1d0 2023-12-19T18:18:03.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd036a55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0300cbaf0 con 0x7fd03006bee0 2023-12-19T18:18:03.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd034ff2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd030164c90 0x7fd030165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd034ff2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd030164c90 0x7fd030165120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42098/0 (socket says 172.21.15.26:42098) 2023-12-19T18:18:03.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd034ff2640 1 -- 172.21.15.26:0/88202266 learned_addr learned my addr 172.21.15.26:0/88202266 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:03.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd02ffff640 1 --2- 172.21.15.26:0/88202266 >> v2:172.21.15.160:3300/0 conn(0x7fd0300cb1d0 0x7fd0300cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd027fff640 1 --2- 172.21.15.26:0/88202266 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03006bee0 0x7fd03006c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd034ff2640 1 -- 172.21.15.26:0/88202266 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03006bee0 msgr2=0x7fd03006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:03.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd034ff2640 1 --2- 172.21.15.26:0/88202266 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03006bee0 0x7fd03006c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd034ff2640 1 -- 172.21.15.26:0/88202266 >> v2:172.21.15.160:3300/0 conn(0x7fd0300cb1d0 msgr2=0x7fd0300cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:03.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd034ff2640 1 --2- 172.21.15.26:0/88202266 >> v2:172.21.15.160:3300/0 conn(0x7fd0300cb1d0 0x7fd0300cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.340+0000 7fd034ff2640 1 -- 172.21.15.26:0/88202266 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03006cc10 con 0x7fd030164c90 2023-12-19T18:18:03.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.344+0000 7fd034ff2640 1 --2- 172.21.15.26:0/88202266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd030164c90 0x7fd030165120 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fd0300ccb20 tx=0x7fd02801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccbc9943b10b89f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:03.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.344+0000 7fd02f7fe640 1 -- 172.21.15.26:0/88202266 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0280a5790 con 0x7fd030164c90 2023-12-19T18:18:03.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.344+0000 7fd02f7fe640 1 -- 172.21.15.26:0/88202266 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0280a58f0 con 0x7fd030164c90 2023-12-19T18:18:03.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.344+0000 7fd02f7fe640 1 -- 172.21.15.26:0/88202266 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0280a5c80 con 0x7fd030164c90 2023-12-19T18:18:03.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.344+0000 7fd036a55640 1 -- 172.21.15.26:0/88202266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd030164c90 msgr2=0x7fd030165120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.344+0000 7fd036a55640 1 --2- 172.21.15.26:0/88202266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd030164c90 0x7fd030165120 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fd0300ccb20 tx=0x7fd02801c910 comp rx=0 tx=0).stop 2023-12-19T18:18:03.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.344+0000 7fd036a55640 1 -- 172.21.15.26:0/88202266 shutdown_connections 2023-12-19T18:18:03.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.344+0000 7fd036a55640 1 --2- 172.21.15.26:0/88202266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd030164c90 0x7fd030165120 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.344+0000 7fd036a55640 1 --2- 172.21.15.26:0/88202266 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03006bee0 0x7fd03006c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.344+0000 7fd036a55640 1 --2- 172.21.15.26:0/88202266 >> v2:172.21.15.160:3300/0 conn(0x7fd0300cb1d0 0x7fd0300cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.344+0000 7fd036a55640 1 -- 172.21.15.26:0/88202266 >> 172.21.15.26:0/88202266 conn(0x7fd0300658e0 msgr2=0x7fd030065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:03.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.344+0000 7fd036a55640 1 -- 172.21.15.26:0/88202266 shutdown_connections 2023-12-19T18:18:03.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.344+0000 7fd036a55640 1 -- 172.21.15.26:0/88202266 wait complete. 2023-12-19T18:18:03.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd036a55640 1 Processor -- start 2023-12-19T18:18:03.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd036a55640 1 -- start start 2023-12-19T18:18:03.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd036a55640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03006bee0 0x7fd0300cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd036a55640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0300cb1d0 0x7fd03015b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd036a55640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd030164c90 0x7fd03015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd036a55640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03015bed0 con 0x7fd030164c90 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd036a55640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03014c760 con 0x7fd0300cb1d0 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd036a55640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03015d910 con 0x7fd03006bee0 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd034ff2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd030164c90 0x7fd03015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03006bee0 0x7fd0300cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03006bee0 0x7fd0300cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:51956/0 (socket says 172.21.15.26:51956) 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd02ffff640 1 -- 172.21.15.26:0/1806539304 learned_addr learned my addr 172.21.15.26:0/1806539304 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd027fff640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0300cb1d0 0x7fd03015b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd02ffff640 1 -- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0300cb1d0 msgr2=0x7fd03015b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd02ffff640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0300cb1d0 0x7fd03015b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd02ffff640 1 -- 172.21.15.26:0/1806539304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd030164c90 msgr2=0x7fd03015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd02ffff640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd030164c90 0x7fd03015d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd02ffff640 1 -- 172.21.15.26:0/1806539304 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd028009b30 con 0x7fd03006bee0 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd034ff2640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd030164c90 0x7fd03015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:03.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd02ffff640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03006bee0 0x7fd0300cda20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd02000e870 tx=0x7fd02000ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:03.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd02d7fa640 1 -- 172.21.15.26:0/1806539304 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd020009510 con 0x7fd03006bee0 2023-12-19T18:18:03.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd02d7fa640 1 -- 172.21.15.26:0/1806539304 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02000eec0 con 0x7fd03006bee0 2023-12-19T18:18:03.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd036a55640 1 -- 172.21.15.26:0/1806539304 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0300ce810 con 0x7fd03006bee0 2023-12-19T18:18:03.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd02d7fa640 1 -- 172.21.15.26:0/1806539304 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02000cb40 con 0x7fd03006bee0 2023-12-19T18:18:03.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.348+0000 7fd036a55640 1 -- 172.21.15.26:0/1806539304 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0300ced90 con 0x7fd03006bee0 2023-12-19T18:18:03.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.352+0000 7fd036a55640 1 -- 172.21.15.26:0/1806539304 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd030074270 con 0x7fd03006bee0 2023-12-19T18:18:03.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.352+0000 7fd02d7fa640 1 -- 172.21.15.26:0/1806539304 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fd02000cca0 con 0x7fd03006bee0 2023-12-19T18:18:03.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.352+0000 7fd02d7fa640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd010065ca0 0x7fd010068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.352+0000 7fd02d7fa640 1 -- 172.21.15.26:0/1806539304 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7341+0+0 (secure 0 0 0) 0x7fd020083fe0 con 0x7fd03006bee0 2023-12-19T18:18:03.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.352+0000 7fd027fff640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd010065ca0 0x7fd010068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.352+0000 7fd027fff640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd010065ca0 0x7fd010068160 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd0301fcfc0 tx=0x7fd018009190 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:03.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.356+0000 7fd02d7fa640 1 -- 172.21.15.26:0/1806539304 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd0200510a0 con 0x7fd03006bee0 2023-12-19T18:18:03.519 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.516+0000 7fd036a55640 1 -- 172.21.15.26:0/1806539304 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0300ccdf0 con 0x7fd03006bee0 2023-12-19T18:18:03.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.520+0000 7fd02d7fa640 1 -- 172.21.15.26:0/1806539304 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v242) v1 ==== 74+0+18467 (secure 0 0 0) 0x7fd020017090 con 0x7fd03006bee0 2023-12-19T18:18:03.521 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:03.521 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":242,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:18:03.156388+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:17:37.484211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:17:38.799276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/2285370499":"2023-12-20T18:17:59.801991+0000","172.21.15.26:6832/2285370499":"2023-12-20T18:17:59.801991+0000","172.21.15.160:0/598924254":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1489104438":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/621398302":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1548583339":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1246495635":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1431434235":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/4035377594":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3900732845":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1388670559":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3872447087":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/332404059":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1786797637":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/563864573":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/2354390435":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/3396499768":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6833/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6832/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/359874050":"2023-12-20T18:17:49.880070+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.520+0000 7fd036a55640 1 -- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd010065ca0 msgr2=0x7fd010068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.520+0000 7fd036a55640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd010065ca0 0x7fd010068160 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd0301fcfc0 tx=0x7fd018009190 comp rx=0 tx=0).stop 2023-12-19T18:18:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.520+0000 7fd036a55640 1 -- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03006bee0 msgr2=0x7fd0300cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.520+0000 7fd036a55640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03006bee0 0x7fd0300cda20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd02000e870 tx=0x7fd02000ec70 comp rx=0 tx=0).stop 2023-12-19T18:18:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.520+0000 7fd036a55640 1 -- 172.21.15.26:0/1806539304 shutdown_connections 2023-12-19T18:18:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.520+0000 7fd036a55640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd010065ca0 0x7fd010068160 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.520+0000 7fd036a55640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd030164c90 0x7fd03015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.520+0000 7fd036a55640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0300cb1d0 0x7fd03015b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.520+0000 7fd036a55640 1 --2- 172.21.15.26:0/1806539304 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd03006bee0 0x7fd0300cda20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.520+0000 7fd036a55640 1 -- 172.21.15.26:0/1806539304 >> 172.21.15.26:0/1806539304 conn(0x7fd0300658e0 msgr2=0x7fd0300b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:03.523 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.520+0000 7fd036a55640 1 -- 172.21.15.26:0/1806539304 shutdown_connections 2023-12-19T18:18:03.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.520+0000 7fd036a55640 1 -- 172.21.15.26:0/1806539304 wait complete. 2023-12-19T18:18:03.542 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-19T18:17:37.484211+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '236', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-19T18:17:38.799276+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '237', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:18:03.542 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:18:03.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd394909640 1 Processor -- start 2023-12-19T18:18:03.724 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd394909640 1 -- start start 2023-12-19T18:18:03.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd394909640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd390159820 0x7fd39015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd394909640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd390159c00 0x7fd390070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd394909640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd39006fb60 con 0x7fd39006f6c0 2023-12-19T18:18:03.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd394909640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd39015e470 con 0x7fd390159c00 2023-12-19T18:18:03.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd394909640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39006fe40 con 0x7fd390159820 2023-12-19T18:18:03.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd387fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd39006f6c0 0x7fd3900c8e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34804/0 (socket says 172.21.15.26:34804) 2023-12-19T18:18:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd387fff640 1 -- 172.21.15.26:0/1932401958 learned_addr learned my addr 172.21.15.26:0/1932401958 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38e59c640 1 --2- 172.21.15.26:0/1932401958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd390159820 0x7fd39015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38ed9d640 1 --2- 172.21.15.26:0/1932401958 >> v2:172.21.15.160:3300/0 conn(0x7fd390159c00 0x7fd390070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050296394 0 0) 0x7fd39006fb60 con 0x7fd39006f6c0 2023-12-19T18:18:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd37c005740 con 0x7fd39006f6c0 2023-12-19T18:18:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445386594 0 0) 0x7fd37c005740 con 0x7fd39006f6c0 2023-12-19T18:18:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd390159820 msgr2=0x7fd39015df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 --2- 172.21.15.26:0/1932401958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd390159820 0x7fd39015df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 >> v2:172.21.15.160:3300/0 conn(0x7fd390159c00 msgr2=0x7fd390070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 --2- 172.21.15.26:0/1932401958 >> v2:172.21.15.160:3300/0 conn(0x7fd390159c00 0x7fd390070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd390070120 con 0x7fd39006f6c0 2023-12-19T18:18:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fd378001680 con 0x7fd39006f6c0 2023-12-19T18:18:03.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 >> v1:172.21.15.26:6789/0 conn(0x7fd39006f6c0 legacy=0x7fd3900c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 --2- 172.21.15.26:0/1932401958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd37c005b90 0x7fd37c005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 --2- 172.21.15.26:0/1932401958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd37c0069e0 0x7fd37c006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 --2- 172.21.15.26:0/1932401958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd37c0073a0 0x7fd37c00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39015e470 con 0x7fd37c005b90 2023-12-19T18:18:03.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39006fe40 con 0x7fd37c0069e0 2023-12-19T18:18:03.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c00e0e0 con 0x7fd37c0073a0 2023-12-19T18:18:03.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38ed9d640 1 --2- 172.21.15.26:0/1932401958 >> v2:172.21.15.160:3300/0 conn(0x7fd390159c00 0x7fd390070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:03.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38e59c640 1 --2- 172.21.15.26:0/1932401958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd390159820 0x7fd39015df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:03.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38e59c640 1 --2- 172.21.15.26:0/1932401958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd37c005b90 0x7fd37c005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd387fff640 1 --2- 172.21.15.26:0/1932401958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd37c0069e0 0x7fd37c006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38ed9d640 1 --2- 172.21.15.26:0/1932401958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd37c0073a0 0x7fd37c00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38e59c640 1 -- 172.21.15.26:0/1932401958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd37c0073a0 msgr2=0x7fd37c00dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38e59c640 1 --2- 172.21.15.26:0/1932401958 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd37c0073a0 0x7fd37c00dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38e59c640 1 -- 172.21.15.26:0/1932401958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd37c0069e0 msgr2=0x7fd37c006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38e59c640 1 --2- 172.21.15.26:0/1932401958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd37c0069e0 0x7fd37c006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.692+0000 7fd38e59c640 1 -- 172.21.15.26:0/1932401958 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c005740 con 0x7fd37c005b90 2023-12-19T18:18:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.696+0000 7fd38e59c640 1 --2- 172.21.15.26:0/1932401958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd37c005b90 0x7fd37c005fb0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fd380009b20 tx=0x7fd38001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e926e6eacd6c8093 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.696+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380004030 con 0x7fd37c005b90 2023-12-19T18:18:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.696+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd380004190 con 0x7fd37c005b90 2023-12-19T18:18:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.696+0000 7fd38dd9b640 1 -- 172.21.15.26:0/1932401958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380004520 con 0x7fd37c005b90 2023-12-19T18:18:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.696+0000 7fd394909640 1 -- 172.21.15.26:0/1932401958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd37c005b90 msgr2=0x7fd37c005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.696+0000 7fd394909640 1 --2- 172.21.15.26:0/1932401958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd37c005b90 0x7fd37c005fb0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fd380009b20 tx=0x7fd38001ca60 comp rx=0 tx=0).stop 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.696+0000 7fd38ed9d640 1 -- 172.21.15.26:0/1932401958 reap_dead start 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.696+0000 7fd394909640 1 -- 172.21.15.26:0/1932401958 shutdown_connections 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.696+0000 7fd394909640 1 -- 172.21.15.26:0/1932401958 >> 172.21.15.26:0/1932401958 conn(0x7fd3900ce730 msgr2=0x7fd3900ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.696+0000 7fd394909640 1 -- 172.21.15.26:0/1932401958 shutdown_connections 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.696+0000 7fd394909640 1 -- 172.21.15.26:0/1932401958 wait complete. 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd394909640 1 Processor -- start 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd394909640 1 -- start start 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd394909640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd37c005b90 0x7fd390151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd394909640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd3900c61b0 0x7fd3900c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd394909640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3900c41d0 0x7fd39015c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd394909640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3900c4650 con 0x7fd3900c41d0 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd394909640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd390152110 con 0x7fd37c005b90 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd394909640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39015ce00 con 0x7fd3900c61b0 2023-12-19T18:18:03.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd38ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3900c41d0 0x7fd39015c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd38ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3900c41d0 0x7fd39015c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42122/0 (socket says 172.21.15.26:42122) 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd38ed9d640 1 -- 172.21.15.26:0/1708679327 learned_addr learned my addr 172.21.15.26:0/1708679327 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd38e59c640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd37c005b90 0x7fd390151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd387fff640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd3900c61b0 0x7fd3900c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd38ed9d640 1 -- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd3900c61b0 msgr2=0x7fd3900c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd38ed9d640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd3900c61b0 0x7fd3900c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd38ed9d640 1 -- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd37c005b90 msgr2=0x7fd390151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd38ed9d640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd37c005b90 0x7fd390151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd38ed9d640 1 -- 172.21.15.26:0/1708679327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd39006b880 con 0x7fd3900c41d0 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd38ed9d640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3900c41d0 0x7fd39015c110 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fd388009f50 tx=0x7fd38800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd386ffd640 1 -- 172.21.15.26:0/1708679327 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38800e040 con 0x7fd3900c41d0 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd386ffd640 1 -- 172.21.15.26:0/1708679327 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38800f040 con 0x7fd3900c41d0 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd386ffd640 1 -- 172.21.15.26:0/1708679327 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388003de0 con 0x7fd3900c41d0 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd394909640 1 -- 172.21.15.26:0/1708679327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39015a0f0 con 0x7fd3900c41d0 2023-12-19T18:18:03.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.700+0000 7fd394909640 1 -- 172.21.15.26:0/1708679327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39015a6b0 con 0x7fd3900c41d0 2023-12-19T18:18:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.704+0000 7fd394909640 1 -- 172.21.15.26:0/1708679327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd390074270 con 0x7fd3900c41d0 2023-12-19T18:18:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.704+0000 7fd386ffd640 1 -- 172.21.15.26:0/1708679327 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fd388003f40 con 0x7fd3900c41d0 2023-12-19T18:18:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.704+0000 7fd386ffd640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd370065ca0 0x7fd370068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.704+0000 7fd386ffd640 1 -- 172.21.15.26:0/1708679327 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7341+0+0 (secure 0 0 0) 0x7fd388084630 con 0x7fd3900c41d0 2023-12-19T18:18:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.704+0000 7fd38e59c640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd370065ca0 0x7fd370068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.704+0000 7fd38e59c640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd370065ca0 0x7fd370068160 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd3800ad000 tx=0x7fd3800046d0 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:03.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.708+0000 7fd386ffd640 1 -- 172.21.15.26:0/1708679327 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd3880515c0 con 0x7fd3900c41d0 2023-12-19T18:18:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.868+0000 7fd394909640 1 -- 172.21.15.26:0/1708679327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd3900b46b0 con 0x7fd3900c41d0 2023-12-19T18:18:03.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.868+0000 7fd386ffd640 1 -- 172.21.15.26:0/1708679327 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v242) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd388054be0 con 0x7fd3900c41d0 2023-12-19T18:18:03.870 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:18:03.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.872+0000 7fd394909640 1 -- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd370065ca0 msgr2=0x7fd370068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.872+0000 7fd394909640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd370065ca0 0x7fd370068160 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd3800ad000 tx=0x7fd3800046d0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.872+0000 7fd394909640 1 -- 172.21.15.26:0/1708679327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3900c41d0 msgr2=0x7fd39015c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.872+0000 7fd394909640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3900c41d0 0x7fd39015c110 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fd388009f50 tx=0x7fd38800f580 comp rx=0 tx=0).stop 2023-12-19T18:18:03.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.872+0000 7fd394909640 1 -- 172.21.15.26:0/1708679327 shutdown_connections 2023-12-19T18:18:03.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.872+0000 7fd394909640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd370065ca0 0x7fd370068160 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.872+0000 7fd394909640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3900c41d0 0x7fd39015c110 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.872+0000 7fd394909640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd3900c61b0 0x7fd3900c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.872+0000 7fd394909640 1 --2- 172.21.15.26:0/1708679327 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd37c005b90 0x7fd390151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.872+0000 7fd394909640 1 -- 172.21.15.26:0/1708679327 >> 172.21.15.26:0/1708679327 conn(0x7fd3900ce730 msgr2=0x7fd3900cd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:03.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.872+0000 7fd394909640 1 -- 172.21.15.26:0/1708679327 shutdown_connections 2023-12-19T18:18:03.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:03.872+0000 7fd394909640 1 -- 172.21.15.26:0/1708679327 wait complete. 2023-12-19T18:18:03.892 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-19T18:18:04.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.036+0000 7f2e9d139640 1 Processor -- start 2023-12-19T18:18:04.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.036+0000 7f2e9d139640 1 -- start start 2023-12-19T18:18:04.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.040+0000 7f2e9d139640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2e9815ccf0 0x7f2e980d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.040+0000 7f2e9d139640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e980b4600 0x7f2e980b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.040+0000 7f2e9d139640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e980b4f50 con 0x7f2e9815d0d0 2023-12-19T18:18:04.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.040+0000 7f2e9d139640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9815e200 con 0x7f2e980b4600 2023-12-19T18:18:04.041 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.040+0000 7f2e9d139640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e980d0aa0 con 0x7f2e9815ccf0 2023-12-19T18:18:04.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.040+0000 7f2e9759e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e980b4600 0x7f2e980b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.040+0000 7f2e9759e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2e980b4600 0x7f2e980b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42134/0 (socket says 172.21.15.26:42134) 2023-12-19T18:18:04.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.040+0000 7f2e9759e640 1 -- 172.21.15.26:0/3106834080 learned_addr learned my addr 172.21.15.26:0/3106834080 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:04.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.040+0000 7f2e96d9d640 1 --2- 172.21.15.26:0/3106834080 >> v2:172.21.15.160:3300/0 conn(0x7f2e9815ccf0 0x7f2e980d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.040+0000 7f2e9759e640 1 -- 172.21.15.26:0/3106834080 >> v2:172.21.15.160:3300/0 conn(0x7f2e9815ccf0 msgr2=0x7f2e980d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:04.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.040+0000 7f2e9759e640 1 --2- 172.21.15.26:0/3106834080 >> v2:172.21.15.160:3300/0 conn(0x7f2e9815ccf0 0x7f2e980d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.040+0000 7f2e9759e640 1 -- 172.21.15.26:0/3106834080 >> v1:172.21.15.160:6789/0 conn(0x7f2e9815d0d0 legacy=0x7f2e980b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.040+0000 7f2e9759e640 1 -- 172.21.15.26:0/3106834080 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e980d1380 con 0x7f2e980b4600 2023-12-19T18:18:04.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9759e640 1 --2- 172.21.15.26:0/3106834080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e980b4600 0x7f2e980b4a10 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f2e980d1570 tx=0x7f2e9001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2daedc59af77fd4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9659c640 1 -- 172.21.15.26:0/3106834080 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e900a5860 con 0x7f2e980b4600 2023-12-19T18:18:04.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9659c640 1 -- 172.21.15.26:0/3106834080 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e900a59c0 con 0x7f2e980b4600 2023-12-19T18:18:04.044 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9659c640 1 -- 172.21.15.26:0/3106834080 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e900a5d50 con 0x7f2e980b4600 2023-12-19T18:18:04.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9d139640 1 -- 172.21.15.26:0/3106834080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e980b4600 msgr2=0x7f2e980b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9d139640 1 --2- 172.21.15.26:0/3106834080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e980b4600 0x7f2e980b4a10 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f2e980d1570 tx=0x7f2e9001c9a0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9d139640 1 -- 172.21.15.26:0/3106834080 shutdown_connections 2023-12-19T18:18:04.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9d139640 1 --2- 172.21.15.26:0/3106834080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e980b4600 0x7f2e980b4a10 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9d139640 1 --2- 172.21.15.26:0/3106834080 >> v2:172.21.15.160:3300/0 conn(0x7f2e9815ccf0 0x7f2e980d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9d139640 1 -- 172.21.15.26:0/3106834080 >> 172.21.15.26:0/3106834080 conn(0x7f2e98065ee0 msgr2=0x7f2e980cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:04.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9d139640 1 -- 172.21.15.26:0/3106834080 shutdown_connections 2023-12-19T18:18:04.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9d139640 1 -- 172.21.15.26:0/3106834080 wait complete. 2023-12-19T18:18:04.046 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9d139640 1 Processor -- start 2023-12-19T18:18:04.047 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.044+0000 7f2e9d139640 1 -- start start 2023-12-19T18:18:04.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e9d139640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e980b4600 0x7f2e98155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e9d139640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e9815ccf0 0x7f2e980cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e9d139640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e9815d0d0 0x7f2e981564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e9d139640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980ccc80 con 0x7f2e980b4600 2023-12-19T18:18:04.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e9d139640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e98155d70 con 0x7f2e9815d0d0 2023-12-19T18:18:04.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e9d139640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e980cc070 con 0x7f2e9815ccf0 2023-12-19T18:18:04.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e96d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e980b4600 0x7f2e98155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e96d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e980b4600 0x7f2e98155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42136/0 (socket says 172.21.15.26:42136) 2023-12-19T18:18:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e96d9d640 1 -- 172.21.15.26:0/2152827348 learned_addr learned my addr 172.21.15.26:0/2152827348 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e9759e640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e9815d0d0 0x7f2e981564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e8ffff640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e9815ccf0 0x7f2e980cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e96d9d640 1 -- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e9815ccf0 msgr2=0x7f2e980cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e96d9d640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e9815ccf0 0x7f2e980cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e96d9d640 1 -- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e9815d0d0 msgr2=0x7f2e981564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e96d9d640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e9815d0d0 0x7f2e981564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e96d9d640 1 -- 172.21.15.26:0/2152827348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e90009b30 con 0x7f2e980b4600 2023-12-19T18:18:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e8ffff640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e9815ccf0 0x7f2e980cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e96d9d640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e980b4600 0x7f2e98155830 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800d800 tx=0x7f2e8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e8f7fe640 1 -- 172.21.15.26:0/2152827348 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800b6c0 con 0x7f2e980b4600 2023-12-19T18:18:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e8f7fe640 1 -- 172.21.15.26:0/2152827348 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8800b820 con 0x7f2e980b4600 2023-12-19T18:18:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e8f7fe640 1 -- 172.21.15.26:0/2152827348 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800bb90 con 0x7f2e980b4600 2023-12-19T18:18:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e9d139640 1 -- 172.21.15.26:0/2152827348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e980cc350 con 0x7f2e980b4600 2023-12-19T18:18:04.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e9d139640 1 -- 172.21.15.26:0/2152827348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e980cc910 con 0x7f2e980b4600 2023-12-19T18:18:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e9d139640 1 -- 172.21.15.26:0/2152827348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e98074270 con 0x7f2e980b4600 2023-12-19T18:18:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.048+0000 7f2e8f7fe640 1 -- 172.21.15.26:0/2152827348 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f2e8800bcf0 con 0x7f2e980b4600 2023-12-19T18:18:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.052+0000 7f2e8f7fe640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2e78065cd0 0x7f2e78068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.052+0000 7f2e8f7fe640 1 -- 172.21.15.26:0/2152827348 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7341+0+0 (secure 0 0 0) 0x7f2e88084110 con 0x7f2e980b4600 2023-12-19T18:18:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.052+0000 7f2e8ffff640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2e78065cd0 0x7f2e78068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.052+0000 7f2e8ffff640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2e78065cd0 0x7f2e78068190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e981f8dc0 tx=0x7f2e80009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.056+0000 7f2e8f7fe640 1 -- 172.21.15.26:0/2152827348 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2e88051070 con 0x7f2e980b4600 2023-12-19T18:18:04.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.216+0000 7f2e9d139640 1 -- 172.21.15.26:0/2152827348 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2e9815dc10 con 0x7f2e980b4600 2023-12-19T18:18:04.218 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.216+0000 7f2e8f7fe640 1 -- 172.21.15.26:0/2152827348 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v242) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2e88054690 con 0x7f2e980b4600 2023-12-19T18:18:04.218 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:18:04.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.220+0000 7f2e9d139640 1 -- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2e78065cd0 msgr2=0x7f2e78068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.220 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.220+0000 7f2e9d139640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2e78065cd0 0x7f2e78068190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e981f8dc0 tx=0x7f2e80009040 comp rx=0 tx=0).stop 2023-12-19T18:18:04.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.220+0000 7f2e9d139640 1 -- 172.21.15.26:0/2152827348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e980b4600 msgr2=0x7f2e98155830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.220+0000 7f2e9d139640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e980b4600 0x7f2e98155830 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800d800 tx=0x7f2e8800dcc0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.220+0000 7f2e9d139640 1 -- 172.21.15.26:0/2152827348 shutdown_connections 2023-12-19T18:18:04.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.220+0000 7f2e9d139640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2e78065cd0 0x7f2e78068190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.220+0000 7f2e9d139640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e9815d0d0 0x7f2e981564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.220+0000 7f2e9d139640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2e9815ccf0 0x7f2e980cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.220+0000 7f2e9d139640 1 --2- 172.21.15.26:0/2152827348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e980b4600 0x7f2e98155830 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.220+0000 7f2e9d139640 1 -- 172.21.15.26:0/2152827348 >> 172.21.15.26:0/2152827348 conn(0x7f2e98065ee0 msgr2=0x7f2e980b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:04.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.220+0000 7f2e9d139640 1 -- 172.21.15.26:0/2152827348 shutdown_connections 2023-12-19T18:18:04.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.220+0000 7f2e9d139640 1 -- 172.21.15.26:0/2152827348 wait complete. 2023-12-19T18:18:04.240 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-19T18:18:04.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.384+0000 7f0930dbc640 1 Processor -- start 2023-12-19T18:18:04.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.384+0000 7f0930dbc640 1 -- start start 2023-12-19T18:18:04.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f0930dbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092c159820 0x7f092c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f0930dbc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f092c06f6c0 0x7f092c0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f0930dbc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092c06fb60 con 0x7f092c159c00 2023-12-19T18:18:04.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f0930dbc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f092c15e470 con 0x7f092c06f6c0 2023-12-19T18:18:04.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f0930dbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c06fe40 con 0x7f092c159820 2023-12-19T18:18:04.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f0923fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f092c06f6c0 0x7f092c0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f092ad9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f092c159c00 0x7f092c070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34810/0 (socket says 172.21.15.26:34810) 2023-12-19T18:18:04.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f092ad9d640 1 -- 172.21.15.26:0/1416511562 learned_addr learned my addr 172.21.15.26:0/1416511562 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:04.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f092a59c640 1 --2- 172.21.15.26:0/1416511562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092c159820 0x7f092c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f0923fff640 1 -- 172.21.15.26:0/1416511562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092c159820 msgr2=0x7f092c15df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:04.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f0923fff640 1 --2- 172.21.15.26:0/1416511562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092c159820 0x7f092c15df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f0923fff640 1 -- 172.21.15.26:0/1416511562 >> v1:172.21.15.26:6789/0 conn(0x7f092c159c00 legacy=0x7f092c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f0923fff640 1 -- 172.21.15.26:0/1416511562 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c070120 con 0x7f092c06f6c0 2023-12-19T18:18:04.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f092a59c640 1 --2- 172.21.15.26:0/1416511562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092c159820 0x7f092c15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:04.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.388+0000 7f0929d9b640 1 -- 172.21.15.26:0/1416511562 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000853629 0 0) 0x7f092c06fb60 con 0x7f092c159c00 2023-12-19T18:18:04.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0923fff640 1 --2- 172.21.15.26:0/1416511562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c06f6c0 0x7f092c0c8e80 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f09140097c0 tx=0x7f091401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958b31697487a9ef server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0929d9b640 1 -- 172.21.15.26:0/1416511562 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140ab650 con 0x7f092c06f6c0 2023-12-19T18:18:04.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0929d9b640 1 -- 172.21.15.26:0/1416511562 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09140ab7b0 con 0x7f092c06f6c0 2023-12-19T18:18:04.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0929d9b640 1 -- 172.21.15.26:0/1416511562 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140abb40 con 0x7f092c06f6c0 2023-12-19T18:18:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0930dbc640 1 -- 172.21.15.26:0/1416511562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c06f6c0 msgr2=0x7f092c0c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0930dbc640 1 --2- 172.21.15.26:0/1416511562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c06f6c0 0x7f092c0c8e80 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f09140097c0 tx=0x7f091401c450 comp rx=0 tx=0).stop 2023-12-19T18:18:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0930dbc640 1 -- 172.21.15.26:0/1416511562 shutdown_connections 2023-12-19T18:18:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0930dbc640 1 --2- 172.21.15.26:0/1416511562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c06f6c0 0x7f092c0c8e80 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0930dbc640 1 --2- 172.21.15.26:0/1416511562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092c159820 0x7f092c15df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0930dbc640 1 -- 172.21.15.26:0/1416511562 >> 172.21.15.26:0/1416511562 conn(0x7f092c0ce730 msgr2=0x7f092c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0930dbc640 1 -- 172.21.15.26:0/1416511562 shutdown_connections 2023-12-19T18:18:04.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0930dbc640 1 -- 172.21.15.26:0/1416511562 wait complete. 2023-12-19T18:18:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0930dbc640 1 Processor -- start 2023-12-19T18:18:04.394 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.392+0000 7f0930dbc640 1 -- start start 2023-12-19T18:18:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0930dbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092c06f6c0 0x7f092c1504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0930dbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c159820 0x7f092c13d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0930dbc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f092c159c00 0x7f092c151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0930dbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0b4190 con 0x7f092c159820 2023-12-19T18:18:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0930dbc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0c37b0 con 0x7f092c159c00 2023-12-19T18:18:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0930dbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c0ca5b0 con 0x7f092c06f6c0 2023-12-19T18:18:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0923fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c159820 0x7f092c13d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0923fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c159820 0x7f092c13d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42160/0 (socket says 172.21.15.26:42160) 2023-12-19T18:18:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0923fff640 1 -- 172.21.15.26:0/1378569762 learned_addr learned my addr 172.21.15.26:0/1378569762 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f092a59c640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092c06f6c0 0x7f092c1504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f092ad9d640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f092c159c00 0x7f092c151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0923fff640 1 -- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092c06f6c0 msgr2=0x7f092c1504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0923fff640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092c06f6c0 0x7f092c1504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0923fff640 1 -- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f092c159c00 msgr2=0x7f092c151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0923fff640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f092c159c00 0x7f092c151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0923fff640 1 -- 172.21.15.26:0/1378569762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914009420 con 0x7f092c159820 2023-12-19T18:18:04.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f092a59c640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092c06f6c0 0x7f092c1504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:04.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f092ad9d640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f092c159c00 0x7f092c151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:04.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0923fff640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c159820 0x7f092c13d620 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f09140ad040 tx=0x7f091401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0922ffd640 1 -- 172.21.15.26:0/1378569762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140b5810 con 0x7f092c159820 2023-12-19T18:18:04.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0922ffd640 1 -- 172.21.15.26:0/1378569762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09140b5970 con 0x7f092c159820 2023-12-19T18:18:04.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0922ffd640 1 -- 172.21.15.26:0/1378569762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140b5d20 con 0x7f092c159820 2023-12-19T18:18:04.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0930dbc640 1 -- 172.21.15.26:0/1378569762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f092c06b880 con 0x7f092c159820 2023-12-19T18:18:04.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0930dbc640 1 -- 172.21.15.26:0/1378569762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f092c0c3d30 con 0x7f092c159820 2023-12-19T18:18:04.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.396+0000 7f0930dbc640 1 -- 172.21.15.26:0/1378569762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f092c074270 con 0x7f092c159820 2023-12-19T18:18:04.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.400+0000 7f0922ffd640 1 -- 172.21.15.26:0/1378569762 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f09140bf410 con 0x7f092c159820 2023-12-19T18:18:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.400+0000 7f0922ffd640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f090c065ca0 0x7f090c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.400+0000 7f0922ffd640 1 -- 172.21.15.26:0/1378569762 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7341+0+0 (secure 0 0 0) 0x7f091411eac0 con 0x7f092c159820 2023-12-19T18:18:04.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.400+0000 7f092a59c640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f090c065ca0 0x7f090c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.400+0000 7f092a59c640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f090c065ca0 0x7f090c068160 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f091c009980 tx=0x7f091c008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.404 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.404+0000 7f0922ffd640 1 -- 172.21.15.26:0/1378569762 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f09140ebc70 con 0x7f092c159820 2023-12-19T18:18:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0930dbc640 1 -- 172.21.15.26:0/1378569762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f092c068030 con 0x7f092c159820 2023-12-19T18:18:04.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0922ffd640 1 -- 172.21.15.26:0/1378569762 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v242) v1 ==== 103+0+11 (secure 0 0 0) 0x7f091411ee60 con 0x7f092c159820 2023-12-19T18:18:04.564 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:18:04.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0930dbc640 1 -- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f090c065ca0 msgr2=0x7f090c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0930dbc640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f090c065ca0 0x7f090c068160 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f091c009980 tx=0x7f091c008040 comp rx=0 tx=0).stop 2023-12-19T18:18:04.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0930dbc640 1 -- 172.21.15.26:0/1378569762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c159820 msgr2=0x7f092c13d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0930dbc640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c159820 0x7f092c13d620 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f09140ad040 tx=0x7f091401ac90 comp rx=0 tx=0).stop 2023-12-19T18:18:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0930dbc640 1 -- 172.21.15.26:0/1378569762 shutdown_connections 2023-12-19T18:18:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0930dbc640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f090c065ca0 0x7f090c068160 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0930dbc640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f092c159c00 0x7f092c151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0930dbc640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f092c159820 0x7f092c13d620 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0930dbc640 1 --2- 172.21.15.26:0/1378569762 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f092c06f6c0 0x7f092c1504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0930dbc640 1 -- 172.21.15.26:0/1378569762 >> 172.21.15.26:0/1378569762 conn(0x7f092c0ce730 msgr2=0x7f092c0af950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0930dbc640 1 -- 172.21.15.26:0/1378569762 shutdown_connections 2023-12-19T18:18:04.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.564+0000 7f0930dbc640 1 -- 172.21.15.26:0/1378569762 wait complete. 2023-12-19T18:18:04.584 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:04.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3cde7df640 1 Processor -- start 2023-12-19T18:18:04.727 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3cde7df640 1 -- start start 2023-12-19T18:18:04.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3cde7df640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3cd815cca0 0x7f3cd80b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3cde7df640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd80b6fb0 con 0x7f3cd80b8760 2023-12-19T18:18:04.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3cde7df640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd80b7290 con 0x7f3cd80b6b20 2023-12-19T18:18:04.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3cde7df640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd80b7c30 con 0x7f3cd815cca0 2023-12-19T18:18:04.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3cdcfdc640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3cd80b8760 0x7f3cd80cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34816/0 (socket says 172.21.15.26:34816) 2023-12-19T18:18:04.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3cdcfdc640 1 -- 172.21.15.26:0/1368508781 learned_addr learned my addr 172.21.15.26:0/1368508781 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:04.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3cdd7dd640 1 --2- 172.21.15.26:0/1368508781 >> v2:172.21.15.160:3300/0 conn(0x7f3cd815cca0 0x7f3cd80b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3ccffff640 1 -- 172.21.15.26:0/1368508781 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986545228 0 0) 0x7f3cd80b6fb0 con 0x7f3cd80b8760 2023-12-19T18:18:04.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3ccffff640 1 -- 172.21.15.26:0/1368508781 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cc4003530 con 0x7f3cd80b8760 2023-12-19T18:18:04.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3ccffff640 1 -- 172.21.15.26:0/1368508781 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570267359 0 0) 0x7f3cd80b7290 con 0x7f3cd80b6b20 2023-12-19T18:18:04.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3ccffff640 1 -- 172.21.15.26:0/1368508781 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cd80b6fb0 con 0x7f3cd80b6b20 2023-12-19T18:18:04.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3cdd7dd640 1 -- 172.21.15.26:0/1368508781 >> v1:172.21.15.160:6789/0 conn(0x7f3cd80b6b20 legacy=0x7f3cd80d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3cdd7dd640 1 -- 172.21.15.26:0/1368508781 >> v1:172.21.15.26:6789/0 conn(0x7f3cd80b8760 legacy=0x7f3cd80cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.728+0000 7f3cdd7dd640 1 -- 172.21.15.26:0/1368508781 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd80b7570 con 0x7f3cd815cca0 2023-12-19T18:18:04.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.732+0000 7f3cdd7dd640 1 --2- 172.21.15.26:0/1368508781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd815cca0 0x7f3cd80b65e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40097c0 tx=0x7f3cd401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=790e702be194b523 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.732+0000 7f3ccffff640 1 -- 172.21.15.26:0/1368508781 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd40ab670 con 0x7f3cd815cca0 2023-12-19T18:18:04.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.732+0000 7f3ccffff640 1 -- 172.21.15.26:0/1368508781 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd40ab7d0 con 0x7f3cd815cca0 2023-12-19T18:18:04.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.732+0000 7f3ccffff640 1 -- 172.21.15.26:0/1368508781 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd40abb40 con 0x7f3cd815cca0 2023-12-19T18:18:04.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.732+0000 7f3cde7df640 1 -- 172.21.15.26:0/1368508781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd815cca0 msgr2=0x7f3cd80b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.732+0000 7f3cde7df640 1 --2- 172.21.15.26:0/1368508781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd815cca0 0x7f3cd80b65e0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40097c0 tx=0x7f3cd401c450 comp rx=0 tx=0).stop 2023-12-19T18:18:04.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.732+0000 7f3cde7df640 1 -- 172.21.15.26:0/1368508781 shutdown_connections 2023-12-19T18:18:04.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.732+0000 7f3cde7df640 1 --2- 172.21.15.26:0/1368508781 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd815cca0 0x7f3cd80b65e0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.732+0000 7f3cde7df640 1 -- 172.21.15.26:0/1368508781 >> 172.21.15.26:0/1368508781 conn(0x7f3cd806a120 msgr2=0x7f3cd806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:04.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.732+0000 7f3cde7df640 1 -- 172.21.15.26:0/1368508781 shutdown_connections 2023-12-19T18:18:04.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.732+0000 7f3cde7df640 1 -- 172.21.15.26:0/1368508781 wait complete. 2023-12-19T18:18:04.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cde7df640 1 Processor -- start 2023-12-19T18:18:04.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cde7df640 1 -- start start 2023-12-19T18:18:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cde7df640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd80b8760 0x7f3cd80ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cde7df640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd815cca0 0x7f3cd8153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cde7df640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd80cc4b0 0x7f3cd814bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cde7df640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd80cb130 con 0x7f3cd815cca0 2023-12-19T18:18:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cde7df640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd80cb410 con 0x7f3cd80cc4b0 2023-12-19T18:18:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cde7df640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd80cb6f0 con 0x7f3cd80b8760 2023-12-19T18:18:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cdcfdc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd815cca0 0x7f3cd8153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cdcfdc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd815cca0 0x7f3cd8153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42170/0 (socket says 172.21.15.26:42170) 2023-12-19T18:18:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cdcfdc640 1 -- 172.21.15.26:0/3513619844 learned_addr learned my addr 172.21.15.26:0/3513619844 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cddfde640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd80cc4b0 0x7f3cd814bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cdd7dd640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd80b8760 0x7f3cd80ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cdcfdc640 1 -- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd80b8760 msgr2=0x7f3cd80ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cdcfdc640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd80b8760 0x7f3cd80ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cdcfdc640 1 -- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd80cc4b0 msgr2=0x7f3cd814bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cdcfdc640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd80cc4b0 0x7f3cd814bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cdcfdc640 1 -- 172.21.15.26:0/3513619844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd4009420 con 0x7f3cd815cca0 2023-12-19T18:18:04.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cdcfdc640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd815cca0 0x7f3cd8153230 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800c4d0 tx=0x7f3cc800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3ccdffb640 1 -- 172.21.15.26:0/3513619844 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8005cf0 con 0x7f3cd815cca0 2023-12-19T18:18:04.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3ccdffb640 1 -- 172.21.15.26:0/3513619844 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc8005e50 con 0x7f3cd815cca0 2023-12-19T18:18:04.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cde7df640 1 -- 172.21.15.26:0/3513619844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd80cb9d0 con 0x7f3cd815cca0 2023-12-19T18:18:04.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3ccdffb640 1 -- 172.21.15.26:0/3513619844 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80072b0 con 0x7f3cd815cca0 2023-12-19T18:18:04.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.736+0000 7f3cde7df640 1 -- 172.21.15.26:0/3513619844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd80cbf90 con 0x7f3cd815cca0 2023-12-19T18:18:04.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.740+0000 7f3cde7df640 1 -- 172.21.15.26:0/3513619844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd80cc1b0 con 0x7f3cd815cca0 2023-12-19T18:18:04.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.740+0000 7f3ccdffb640 1 -- 172.21.15.26:0/3513619844 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f3cc8007410 con 0x7f3cd815cca0 2023-12-19T18:18:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.740+0000 7f3ccdffb640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3cbc065ca0 0x7f3cbc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.740+0000 7f3ccdffb640 1 -- 172.21.15.26:0/3513619844 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7341+0+0 (secure 0 0 0) 0x7f3cc80832b0 con 0x7f3cd815cca0 2023-12-19T18:18:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.740+0000 7f3cdd7dd640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3cbc065ca0 0x7f3cbc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.740+0000 7f3cdd7dd640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3cbc065ca0 0x7f3cbc068160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40af000 tx=0x7f3cd40a7000 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.744+0000 7f3ccdffb640 1 -- 172.21.15.26:0/3513619844 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3cc8050210 con 0x7f3cd815cca0 2023-12-19T18:18:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.936+0000 7f3cde7df640 1 -- 172.21.15.26:0/3513619844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3cd8067400 con 0x7f3cd815cca0 2023-12-19T18:18:04.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.936+0000 7f3ccdffb640 1 -- 172.21.15.26:0/3513619844 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4733 (secure 0 0 0) 0x7f3cc8053830 con 0x7f3cd815cca0 2023-12-19T18:18:04.938 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:04.938 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":120,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2727392302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2727392302},{"type":"v1","addr":"172.21.15.160:6836","nonce":2727392302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1029020739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1029020739},{"type":"v1","addr":"172.21.15.160:6837","nonce":1029020739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":12003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3746262698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3746262698},{"type":"v1","addr":"172.21.15.26:6835","nonce":3746262698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:39.801489+0000","modified":"2023-12-19T18:17:42.831660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11424":{"gid":11424,"name":"a","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6834/3251191056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3251191056},{"type":"v1","addr":"172.21.15.26:6834","nonce":3251191056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-19T18:18:04.938 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 120 2023-12-19T18:18:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.940+0000 7f3cde7df640 1 -- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3cbc065ca0 msgr2=0x7f3cbc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.940+0000 7f3cde7df640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3cbc065ca0 0x7f3cbc068160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40af000 tx=0x7f3cd40a7000 comp rx=0 tx=0).stop 2023-12-19T18:18:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.940+0000 7f3cde7df640 1 -- 172.21.15.26:0/3513619844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd815cca0 msgr2=0x7f3cd8153230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.940+0000 7f3cde7df640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd815cca0 0x7f3cd8153230 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800c4d0 tx=0x7f3cc800c990 comp rx=0 tx=0).stop 2023-12-19T18:18:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.940+0000 7f3cde7df640 1 -- 172.21.15.26:0/3513619844 shutdown_connections 2023-12-19T18:18:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.940+0000 7f3cde7df640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3cbc065ca0 0x7f3cbc068160 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.940+0000 7f3cde7df640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3cd80cc4b0 0x7f3cd814bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.940+0000 7f3cde7df640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd815cca0 0x7f3cd8153230 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.940+0000 7f3cde7df640 1 --2- 172.21.15.26:0/3513619844 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3cd80b8760 0x7f3cd80ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.940+0000 7f3cde7df640 1 -- 172.21.15.26:0/3513619844 >> 172.21.15.26:0/3513619844 conn(0x7f3cd806a120 msgr2=0x7f3cd8157e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.940+0000 7f3cde7df640 1 -- 172.21.15.26:0/3513619844 shutdown_connections 2023-12-19T18:18:04.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:04.940+0000 7f3cde7df640 1 -- 172.21.15.26:0/3513619844 wait complete. 2023-12-19T18:18:04.959 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:05.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd972151640 1 Processor -- start 2023-12-19T18:18:05.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd972151640 1 -- start start 2023-12-19T18:18:05.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd972151640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd96c15cd10 0x7fd96c0d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd972151640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd96c0d1060 0x7fd96c0cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd972151640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd96c15d0f0 con 0x7fd96c0cf6f0 2023-12-19T18:18:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd972151640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96c0d1550 con 0x7fd96c0d1060 2023-12-19T18:18:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd972151640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96c0d1830 con 0x7fd96c15cd10 2023-12-19T18:18:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd97094e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd96c0d1060 0x7fd96c0cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd97094e640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd96c0d1060 0x7fd96c0cf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42182/0 (socket says 172.21.15.26:42182) 2023-12-19T18:18:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd97094e640 1 -- 172.21.15.26:0/2865106711 learned_addr learned my addr 172.21.15.26:0/2865106711 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd97114f640 1 --2- 172.21.15.26:0/2865106711 >> v2:172.21.15.160:3300/0 conn(0x7fd96c15cd10 0x7fd96c0d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd97094e640 1 -- 172.21.15.26:0/2865106711 >> v2:172.21.15.160:3300/0 conn(0x7fd96c15cd10 msgr2=0x7fd96c0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd97094e640 1 --2- 172.21.15.26:0/2865106711 >> v2:172.21.15.160:3300/0 conn(0x7fd96c15cd10 0x7fd96c0d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd97094e640 1 -- 172.21.15.26:0/2865106711 >> v1:172.21.15.160:6789/0 conn(0x7fd96c0cf6f0 legacy=0x7fd96c0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd97094e640 1 -- 172.21.15.26:0/2865106711 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c0d1b10 con 0x7fd96c0d1060 2023-12-19T18:18:05.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.104+0000 7fd97114f640 1 --2- 172.21.15.26:0/2865106711 >> v2:172.21.15.160:3300/0 conn(0x7fd96c15cd10 0x7fd96c0d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:05.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.108+0000 7fd97094e640 1 --2- 172.21.15.26:0/2865106711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd96c0d1060 0x7fd96c0cf1b0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0d0340 tx=0x7fd95c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77314be62e18a6a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.108+0000 7fd963fff640 1 -- 172.21.15.26:0/2865106711 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0ac410 con 0x7fd96c0d1060 2023-12-19T18:18:05.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.108+0000 7fd963fff640 1 -- 172.21.15.26:0/2865106711 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c0ac570 con 0x7fd96c0d1060 2023-12-19T18:18:05.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.108+0000 7fd963fff640 1 -- 172.21.15.26:0/2865106711 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0ac900 con 0x7fd96c0d1060 2023-12-19T18:18:05.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.108+0000 7fd972151640 1 -- 172.21.15.26:0/2865106711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd96c0d1060 msgr2=0x7fd96c0cf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.108+0000 7fd972151640 1 --2- 172.21.15.26:0/2865106711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd96c0d1060 0x7fd96c0cf1b0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0d0340 tx=0x7fd95c01c490 comp rx=0 tx=0).stop 2023-12-19T18:18:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.108+0000 7fd972151640 1 -- 172.21.15.26:0/2865106711 shutdown_connections 2023-12-19T18:18:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.108+0000 7fd972151640 1 --2- 172.21.15.26:0/2865106711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd96c0d1060 0x7fd96c0cf1b0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.108+0000 7fd972151640 1 --2- 172.21.15.26:0/2865106711 >> v2:172.21.15.160:3300/0 conn(0x7fd96c15cd10 0x7fd96c0d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.108+0000 7fd972151640 1 -- 172.21.15.26:0/2865106711 >> 172.21.15.26:0/2865106711 conn(0x7fd96c0658e0 msgr2=0x7fd96c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.108+0000 7fd972151640 1 -- 172.21.15.26:0/2865106711 shutdown_connections 2023-12-19T18:18:05.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.108+0000 7fd972151640 1 -- 172.21.15.26:0/2865106711 wait complete. 2023-12-19T18:18:05.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd972151640 1 Processor -- start 2023-12-19T18:18:05.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd972151640 1 -- start start 2023-12-19T18:18:05.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd972151640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd96c0cf6f0 0x7fd96c0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd972151640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd96c0d1060 0x7fd96c0b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd972151640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd96c15cd10 0x7fd96c152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd972151640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c0cb600 con 0x7fd96c0d1060 2023-12-19T18:18:05.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd972151640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c0ca9f0 con 0x7fd96c0cf6f0 2023-12-19T18:18:05.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd972151640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c0cacd0 con 0x7fd96c15cd10 2023-12-19T18:18:05.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd97094e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd96c0d1060 0x7fd96c0b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.112 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd97094e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd96c0d1060 0x7fd96c0b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42190/0 (socket says 172.21.15.26:42190) 2023-12-19T18:18:05.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd97094e640 1 -- 172.21.15.26:0/3114058955 learned_addr learned my addr 172.21.15.26:0/3114058955 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:05.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd97114f640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd96c0cf6f0 0x7fd96c0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd971950640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd96c15cd10 0x7fd96c152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd97094e640 1 -- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd96c15cd10 msgr2=0x7fd96c152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd97094e640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd96c15cd10 0x7fd96c152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd97094e640 1 -- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd96c0cf6f0 msgr2=0x7fd96c0c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd97094e640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd96c0cf6f0 0x7fd96c0c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd97094e640 1 -- 172.21.15.26:0/3114058955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c0094d0 con 0x7fd96c0d1060 2023-12-19T18:18:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd97094e640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd96c0d1060 0x7fd96c0b3620 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fd95c01a0c0 tx=0x7fd95c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd961ffb640 1 -- 172.21.15.26:0/3114058955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0ac410 con 0x7fd96c0d1060 2023-12-19T18:18:05.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd961ffb640 1 -- 172.21.15.26:0/3114058955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c0b65f0 con 0x7fd96c0d1060 2023-12-19T18:18:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd961ffb640 1 -- 172.21.15.26:0/3114058955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0b69c0 con 0x7fd96c0d1060 2023-12-19T18:18:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd972151640 1 -- 172.21.15.26:0/3114058955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96c0cafb0 con 0x7fd96c0d1060 2023-12-19T18:18:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd972151640 1 -- 172.21.15.26:0/3114058955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96c0c83d0 con 0x7fd96c0d1060 2023-12-19T18:18:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.112+0000 7fd972151640 1 -- 172.21.15.26:0/3114058955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96c0cb770 con 0x7fd96c0d1060 2023-12-19T18:18:05.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.116+0000 7fd961ffb640 1 -- 172.21.15.26:0/3114058955 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fd95c0b6b20 con 0x7fd96c0d1060 2023-12-19T18:18:05.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.116+0000 7fd961ffb640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd950065ca0 0x7fd950068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.116+0000 7fd961ffb640 1 -- 172.21.15.26:0/3114058955 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7341+0+0 (secure 0 0 0) 0x7fd95c11e4b0 con 0x7fd96c0d1060 2023-12-19T18:18:05.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.116+0000 7fd97114f640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd950065ca0 0x7fd950068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.116+0000 7fd97114f640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd950065ca0 0x7fd950068160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0c7d40 tx=0x7fd968008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.120+0000 7fd961ffb640 1 -- 172.21.15.26:0/3114058955 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd95c0eb550 con 0x7fd96c0d1060 2023-12-19T18:18:05.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.280+0000 7fd972151640 1 -- 172.21.15.26:0/3114058955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd96c067400 con 0x7fd96c0d1060 2023-12-19T18:18:05.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.280+0000 7fd961ffb640 1 -- 172.21.15.26:0/3114058955 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v242) v1 ==== 74+0+18467 (secure 0 0 0) 0x7fd95c0eeb70 con 0x7fd96c0d1060 2023-12-19T18:18:05.281 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:05.281 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":242,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:18:03.156388+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:17:37.484211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:17:38.799276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/2285370499":"2023-12-20T18:17:59.801991+0000","172.21.15.26:6832/2285370499":"2023-12-20T18:17:59.801991+0000","172.21.15.160:0/598924254":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1489104438":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/621398302":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1548583339":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1246495635":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1431434235":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/4035377594":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3900732845":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1388670559":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3872447087":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/332404059":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1786797637":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/563864573":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/2354390435":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/3396499768":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6833/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6832/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/359874050":"2023-12-20T18:17:49.880070+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:05.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.284+0000 7fd972151640 1 -- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd950065ca0 msgr2=0x7fd950068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.284+0000 7fd972151640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd950065ca0 0x7fd950068160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0c7d40 tx=0x7fd968008040 comp rx=0 tx=0).stop 2023-12-19T18:18:05.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.284+0000 7fd972151640 1 -- 172.21.15.26:0/3114058955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd96c0d1060 msgr2=0x7fd96c0b3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.284+0000 7fd972151640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd96c0d1060 0x7fd96c0b3620 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7fd95c01a0c0 tx=0x7fd95c019230 comp rx=0 tx=0).stop 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.284+0000 7fd972151640 1 -- 172.21.15.26:0/3114058955 shutdown_connections 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.284+0000 7fd972151640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd950065ca0 0x7fd950068160 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.284+0000 7fd972151640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd96c15cd10 0x7fd96c152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.284+0000 7fd972151640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd96c0d1060 0x7fd96c0b3620 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.284+0000 7fd972151640 1 --2- 172.21.15.26:0/3114058955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd96c0cf6f0 0x7fd96c0c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.284+0000 7fd972151640 1 -- 172.21.15.26:0/3114058955 >> 172.21.15.26:0/3114058955 conn(0x7fd96c0658e0 msgr2=0x7fd96c15bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.284+0000 7fd972151640 1 -- 172.21.15.26:0/3114058955 shutdown_connections 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.284+0000 7fd972151640 1 -- 172.21.15.26:0/3114058955 wait complete. 2023-12-19T18:18:05.303 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-12-19T18:18:05.303 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:05.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf51f25640 1 Processor -- start 2023-12-19T18:18:05.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf51f25640 1 -- start start 2023-12-19T18:18:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf51f25640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf4c154220 0x7faf4c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf51f25640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c06bc30 con 0x7faf4c06b3d0 2023-12-19T18:18:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf51f25640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c0cc6e0 con 0x7faf4c0cee10 2023-12-19T18:18:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf51f25640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c15afc0 con 0x7faf4c154220 2023-12-19T18:18:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf4affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faf4c06b3d0 0x7faf4c0ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34832/0 (socket says 172.21.15.26:34832) 2023-12-19T18:18:05.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf4affd640 1 -- 172.21.15.26:0/259562601 learned_addr learned my addr 172.21.15.26:0/259562601 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf4b7fe640 1 --2- 172.21.15.26:0/259562601 >> v2:172.21.15.160:3300/0 conn(0x7faf4c154220 0x7faf4c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf4a7fc640 1 -- 172.21.15.26:0/259562601 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629355157 0 0) 0x7faf4c06bc30 con 0x7faf4c06b3d0 2023-12-19T18:18:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf4a7fc640 1 -- 172.21.15.26:0/259562601 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf34003530 con 0x7faf4c06b3d0 2023-12-19T18:18:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf4a7fc640 1 -- 172.21.15.26:0/259562601 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983932917 0 0) 0x7faf4c0cc6e0 con 0x7faf4c0cee10 2023-12-19T18:18:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf4a7fc640 1 -- 172.21.15.26:0/259562601 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf4c06bc30 con 0x7faf4c0cee10 2023-12-19T18:18:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf4b7fe640 1 -- 172.21.15.26:0/259562601 >> v1:172.21.15.160:6789/0 conn(0x7faf4c0cee10 legacy=0x7faf4c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf4b7fe640 1 -- 172.21.15.26:0/259562601 >> v1:172.21.15.26:6789/0 conn(0x7faf4c06b3d0 legacy=0x7faf4c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.448+0000 7faf4b7fe640 1 -- 172.21.15.26:0/259562601 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c06b950 con 0x7faf4c154220 2023-12-19T18:18:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.452+0000 7faf4b7fe640 1 --2- 172.21.15.26:0/259562601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c154220 0x7faf4c15aa80 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0097c0 tx=0x7faf3c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c53bc161173de3d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.452+0000 7faf4a7fc640 1 -- 172.21.15.26:0/259562601 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0ab650 con 0x7faf4c154220 2023-12-19T18:18:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.452+0000 7faf4a7fc640 1 -- 172.21.15.26:0/259562601 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c0ab7b0 con 0x7faf4c154220 2023-12-19T18:18:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.452+0000 7faf4a7fc640 1 -- 172.21.15.26:0/259562601 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0abb20 con 0x7faf4c154220 2023-12-19T18:18:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.452+0000 7faf51f25640 1 -- 172.21.15.26:0/259562601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c154220 msgr2=0x7faf4c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.452+0000 7faf51f25640 1 --2- 172.21.15.26:0/259562601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c154220 0x7faf4c15aa80 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0097c0 tx=0x7faf3c01c450 comp rx=0 tx=0).stop 2023-12-19T18:18:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.452+0000 7faf51f25640 1 -- 172.21.15.26:0/259562601 shutdown_connections 2023-12-19T18:18:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.452+0000 7faf51f25640 1 --2- 172.21.15.26:0/259562601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c154220 0x7faf4c15aa80 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.452+0000 7faf51f25640 1 -- 172.21.15.26:0/259562601 >> 172.21.15.26:0/259562601 conn(0x7faf4c0658e0 msgr2=0x7faf4c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.452+0000 7faf51f25640 1 -- 172.21.15.26:0/259562601 shutdown_connections 2023-12-19T18:18:05.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.452+0000 7faf51f25640 1 -- 172.21.15.26:0/259562601 wait complete. 2023-12-19T18:18:05.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf51f25640 1 Processor -- start 2023-12-19T18:18:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf51f25640 1 -- start start 2023-12-19T18:18:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf51f25640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf4c06b3d0 0x7faf4c155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf51f25640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c0cee10 0x7faf4c14dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf51f25640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf4c154220 0x7faf4c14e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf51f25640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c155c70 con 0x7faf4c06b3d0 2023-12-19T18:18:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf51f25640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c1552a0 con 0x7faf4c0cee10 2023-12-19T18:18:05.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf51f25640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c154690 con 0x7faf4c154220 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf4b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf4c06b3d0 0x7faf4c155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf4b7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf4c06b3d0 0x7faf4c155730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42196/0 (socket says 172.21.15.26:42196) 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf4b7fe640 1 -- 172.21.15.26:0/386356681 learned_addr learned my addr 172.21.15.26:0/386356681 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf4b7fe640 1 -- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf4c154220 msgr2=0x7faf4c14e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf4b7fe640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf4c154220 0x7faf4c14e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf4b7fe640 1 -- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c0cee10 msgr2=0x7faf4c14dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf4affd640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c0cee10 0x7faf4c14dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf4b7fe640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c0cee10 0x7faf4c14dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf4b7fe640 1 -- 172.21.15.26:0/386356681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c009420 con 0x7faf4c06b3d0 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf4affd640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c0cee10 0x7faf4c14dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf4b7fe640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf4c06b3d0 0x7faf4c155730 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7faf3c000c10 tx=0x7faf3c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf33fff640 1 -- 172.21.15.26:0/386356681 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0b5600 con 0x7faf4c06b3d0 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf33fff640 1 -- 172.21.15.26:0/386356681 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c0b5760 con 0x7faf4c06b3d0 2023-12-19T18:18:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf33fff640 1 -- 172.21.15.26:0/386356681 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0b5ad0 con 0x7faf4c06b3d0 2023-12-19T18:18:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf51f25640 1 -- 172.21.15.26:0/386356681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c154970 con 0x7faf4c06b3d0 2023-12-19T18:18:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.456+0000 7faf51f25640 1 -- 172.21.15.26:0/386356681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c154f30 con 0x7faf4c06b3d0 2023-12-19T18:18:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.460+0000 7faf51f25640 1 -- 172.21.15.26:0/386356681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c074270 con 0x7faf4c06b3d0 2023-12-19T18:18:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.460+0000 7faf33fff640 1 -- 172.21.15.26:0/386356681 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7faf3c0b5c30 con 0x7faf4c06b3d0 2023-12-19T18:18:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.460+0000 7faf33fff640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf38065ca0 0x7faf38068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.460+0000 7faf33fff640 1 -- 172.21.15.26:0/386356681 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7341+0+0 (secure 0 0 0) 0x7faf3c11e7f0 con 0x7faf4c06b3d0 2023-12-19T18:18:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.460+0000 7faf4affd640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf38065ca0 0x7faf38068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.460+0000 7faf4affd640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf38065ca0 0x7faf38068160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0b33c0 tx=0x7faf40009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.464+0000 7faf33fff640 1 -- 172.21.15.26:0/386356681 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faf3c0eb920 con 0x7faf4c06b3d0 2023-12-19T18:18:05.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.656+0000 7faf51f25640 1 -- 172.21.15.26:0/386356681 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf4c0b3430 con 0x7faf4c06b3d0 2023-12-19T18:18:05.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.656+0000 7faf33fff640 1 -- 172.21.15.26:0/386356681 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4733 (secure 0 0 0) 0x7faf3c0c4090 con 0x7faf4c06b3d0 2023-12-19T18:18:05.659 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:05.659 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":120,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2727392302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2727392302},{"type":"v1","addr":"172.21.15.160:6836","nonce":2727392302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1029020739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1029020739},{"type":"v1","addr":"172.21.15.160:6837","nonce":1029020739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":12003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3746262698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3746262698},{"type":"v1","addr":"172.21.15.26:6835","nonce":3746262698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:39.801489+0000","modified":"2023-12-19T18:17:42.831660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11424":{"gid":11424,"name":"a","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6834/3251191056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3251191056},{"type":"v1","addr":"172.21.15.26:6834","nonce":3251191056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-19T18:18:05.659 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 120 2023-12-19T18:18:05.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.660+0000 7faf51f25640 1 -- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf38065ca0 msgr2=0x7faf38068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.660+0000 7faf51f25640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf38065ca0 0x7faf38068160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0b33c0 tx=0x7faf40009040 comp rx=0 tx=0).stop 2023-12-19T18:18:05.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.660+0000 7faf51f25640 1 -- 172.21.15.26:0/386356681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf4c06b3d0 msgr2=0x7faf4c155730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.660+0000 7faf51f25640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf4c06b3d0 0x7faf4c155730 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7faf3c000c10 tx=0x7faf3c01ac90 comp rx=0 tx=0).stop 2023-12-19T18:18:05.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.660+0000 7faf51f25640 1 -- 172.21.15.26:0/386356681 shutdown_connections 2023-12-19T18:18:05.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.660+0000 7faf51f25640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf38065ca0 0x7faf38068160 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.660+0000 7faf51f25640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf4c154220 0x7faf4c14e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.660+0000 7faf51f25640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf4c0cee10 0x7faf4c14dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.660+0000 7faf51f25640 1 --2- 172.21.15.26:0/386356681 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf4c06b3d0 0x7faf4c155730 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.660+0000 7faf51f25640 1 -- 172.21.15.26:0/386356681 >> 172.21.15.26:0/386356681 conn(0x7faf4c0658e0 msgr2=0x7faf4c0ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:05.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.660+0000 7faf51f25640 1 -- 172.21.15.26:0/386356681 shutdown_connections 2023-12-19T18:18:05.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.660+0000 7faf51f25640 1 -- 172.21.15.26:0/386356681 wait complete. 2023-12-19T18:18:05.680 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.820+0000 7f58993f3640 1 Processor -- start 2023-12-19T18:18:05.823 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f58993f3640 1 -- start start 2023-12-19T18:18:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f58993f3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f589415f370 0x7f589415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f58993f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58940ce740 0x7f58940cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f58993f3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f589406c970 con 0x7f5894155030 2023-12-19T18:18:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f58993f3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f589406c790 con 0x7f589415f370 2023-12-19T18:18:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f58993f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58940cf0f0 con 0x7f58940ce740 2023-12-19T18:18:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f588bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f589415f370 0x7f589415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f588bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f589415f370 0x7f589415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42200/0 (socket says 172.21.15.26:42200) 2023-12-19T18:18:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f588bfff640 1 -- 172.21.15.26:0/2429704499 learned_addr learned my addr 172.21.15.26:0/2429704499 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f58937fe640 1 --2- 172.21.15.26:0/2429704499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58940ce740 0x7f58940cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f588bfff640 1 -- 172.21.15.26:0/2429704499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58940ce740 msgr2=0x7f58940cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:05.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f588bfff640 1 --2- 172.21.15.26:0/2429704499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58940ce740 0x7f58940cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f588bfff640 1 -- 172.21.15.26:0/2429704499 >> v1:172.21.15.26:6789/0 conn(0x7f5894155030 legacy=0x7f589406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f588bfff640 1 -- 172.21.15.26:0/2429704499 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589415b1f0 con 0x7f589415f370 2023-12-19T18:18:05.826 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.824+0000 7f58927fc640 1 -- 172.21.15.26:0/2429704499 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215139147 0 0) 0x7f589406c970 con 0x7f5894155030 2023-12-19T18:18:05.827 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f588bfff640 1 --2- 172.21.15.26:0/2429704499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f589415f370 0x7f589415bcd0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f587c009870 tx=0x7f587c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4130386484cd0404 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f58927fc640 1 -- 172.21.15.26:0/2429704499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c0aa670 con 0x7f589415f370 2023-12-19T18:18:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f58927fc640 1 -- 172.21.15.26:0/2429704499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c0aa7d0 con 0x7f589415f370 2023-12-19T18:18:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f58927fc640 1 -- 172.21.15.26:0/2429704499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c0aab60 con 0x7f589415f370 2023-12-19T18:18:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f58993f3640 1 -- 172.21.15.26:0/2429704499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f589415f370 msgr2=0x7f589415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f58993f3640 1 --2- 172.21.15.26:0/2429704499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f589415f370 0x7f589415bcd0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f587c009870 tx=0x7f587c01c510 comp rx=0 tx=0).stop 2023-12-19T18:18:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f58993f3640 1 -- 172.21.15.26:0/2429704499 shutdown_connections 2023-12-19T18:18:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f58993f3640 1 --2- 172.21.15.26:0/2429704499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58940ce740 0x7f58940cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f58993f3640 1 --2- 172.21.15.26:0/2429704499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f589415f370 0x7f589415bcd0 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.829 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f58993f3640 1 -- 172.21.15.26:0/2429704499 >> 172.21.15.26:0/2429704499 conn(0x7f58940658e0 msgr2=0x7f5894065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:05.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f58993f3640 1 -- 172.21.15.26:0/2429704499 shutdown_connections 2023-12-19T18:18:05.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f58993f3640 1 -- 172.21.15.26:0/2429704499 wait complete. 2023-12-19T18:18:05.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f58993f3640 1 Processor -- start 2023-12-19T18:18:05.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.828+0000 7f58993f3640 1 -- start start 2023-12-19T18:18:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f58993f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58940ce740 0x7f589414e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f58993f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5894155030 0x7f589415c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f58993f3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589415f370 0x7f589415c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f58993f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589413a420 con 0x7f5894155030 2023-12-19T18:18:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f58993f3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589414eea0 con 0x7f589415f370 2023-12-19T18:18:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f58993f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589414f180 con 0x7f58940ce740 2023-12-19T18:18:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f588bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5894155030 0x7f589415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f588bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5894155030 0x7f589415c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42210/0 (socket says 172.21.15.26:42210) 2023-12-19T18:18:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f588bfff640 1 -- 172.21.15.26:0/3256400252 learned_addr learned my addr 172.21.15.26:0/3256400252 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f58937fe640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589415f370 0x7f589415c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f5892ffd640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58940ce740 0x7f589414e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f588bfff640 1 -- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58940ce740 msgr2=0x7f589414e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f588bfff640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58940ce740 0x7f589414e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f588bfff640 1 -- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589415f370 msgr2=0x7f589415c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f588bfff640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589415f370 0x7f589415c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f588bfff640 1 -- 172.21.15.26:0/3256400252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c0094d0 con 0x7f5894155030 2023-12-19T18:18:05.834 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f588bfff640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5894155030 0x7f589415c460 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f587c0ac040 tx=0x7f587c0192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f588b7fe640 1 -- 172.21.15.26:0/3256400252 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c0aa670 con 0x7f5894155030 2023-12-19T18:18:05.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f588b7fe640 1 -- 172.21.15.26:0/3256400252 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c0b4830 con 0x7f5894155030 2023-12-19T18:18:05.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f58993f3640 1 -- 172.21.15.26:0/3256400252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f589414f460 con 0x7f5894155030 2023-12-19T18:18:05.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f588b7fe640 1 -- 172.21.15.26:0/3256400252 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c0b4be0 con 0x7f5894155030 2023-12-19T18:18:05.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f58993f3640 1 -- 172.21.15.26:0/3256400252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f589414ea70 con 0x7f5894155030 2023-12-19T18:18:05.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.832+0000 7f58993f3640 1 -- 172.21.15.26:0/3256400252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f589414ec60 con 0x7f5894155030 2023-12-19T18:18:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.836+0000 7f588b7fe640 1 -- 172.21.15.26:0/3256400252 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f587c0b4d40 con 0x7f5894155030 2023-12-19T18:18:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.836+0000 7f588b7fe640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5874065ca0 0x7f5874068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.836+0000 7f588b7fe640 1 -- 172.21.15.26:0/3256400252 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7341+0+0 (secure 0 0 0) 0x7f587c11ebc0 con 0x7f5894155030 2023-12-19T18:18:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.836+0000 7f5892ffd640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5874065ca0 0x7f5874068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.836+0000 7f5892ffd640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5874065ca0 0x7f5874068160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5884000f70 tx=0x7f5884009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:05.840+0000 7f588b7fe640 1 -- 172.21.15.26:0/3256400252 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f589414ec60 con 0x7f5894155030 2023-12-19T18:18:06.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.028+0000 7f58993f3640 1 -- 172.21.15.26:0/3256400252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f589414ec60 con 0x7f5894155030 2023-12-19T18:18:06.030 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.028+0000 7f588b7fe640 1 -- 172.21.15.26:0/3256400252 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4733 (secure 0 0 0) 0x7f589414ec60 con 0x7f5894155030 2023-12-19T18:18:06.030 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:06.030 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":120,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2727392302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2727392302},{"type":"v1","addr":"172.21.15.160:6836","nonce":2727392302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1029020739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1029020739},{"type":"v1","addr":"172.21.15.160:6837","nonce":1029020739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":12003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3746262698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3746262698},{"type":"v1","addr":"172.21.15.26:6835","nonce":3746262698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:39.801489+0000","modified":"2023-12-19T18:17:42.831660+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11424},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11424":{"gid":11424,"name":"a","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.26:6834/3251191056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":3251191056},{"type":"v1","addr":"172.21.15.26:6834","nonce":3251191056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-19T18:18:06.031 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 120 2023-12-19T18:18:06.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.032+0000 7f58993f3640 1 -- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5874065ca0 msgr2=0x7f5874068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.032+0000 7f58993f3640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5874065ca0 0x7f5874068160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5884000f70 tx=0x7f5884009040 comp rx=0 tx=0).stop 2023-12-19T18:18:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.032+0000 7f58993f3640 1 -- 172.21.15.26:0/3256400252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5894155030 msgr2=0x7f589415c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.032+0000 7f58993f3640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5894155030 0x7f589415c460 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f587c0ac040 tx=0x7f587c0192b0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.032+0000 7f58993f3640 1 -- 172.21.15.26:0/3256400252 shutdown_connections 2023-12-19T18:18:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.032+0000 7f58993f3640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5874065ca0 0x7f5874068160 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.032+0000 7f58993f3640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f589415f370 0x7f589415c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.032+0000 7f58993f3640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5894155030 0x7f589415c460 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.032+0000 7f58993f3640 1 --2- 172.21.15.26:0/3256400252 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f58940ce740 0x7f589414e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.032+0000 7f58993f3640 1 -- 172.21.15.26:0/3256400252 >> 172.21.15.26:0/3256400252 conn(0x7f58940658e0 msgr2=0x7f589415dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.032+0000 7f58993f3640 1 -- 172.21.15.26:0/3256400252 shutdown_connections 2023-12-19T18:18:06.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.032+0000 7f58993f3640 1 -- 172.21.15.26:0/3256400252 wait complete. 2023-12-19T18:18:06.051 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:06.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.192+0000 7f8c148f9640 1 Processor -- start 2023-12-19T18:18:06.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.192+0000 7f8c148f9640 1 -- start start 2023-12-19T18:18:06.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c148f9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8c1006b3d0 0x7f8c100ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c148f9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1006bc30 con 0x7f8c100cec40 2023-12-19T18:18:06.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c148f9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c100cc6e0 con 0x7f8c10154220 2023-12-19T18:18:06.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c148f9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c100cf0d0 con 0x7f8c1006b3d0 2023-12-19T18:18:06.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c0ed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8c100cec40 0x7f8c1015b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35270/0 (socket says 172.21.15.26:35270) 2023-12-19T18:18:06.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c0ed9d640 1 -- 172.21.15.26:0/343314851 learned_addr learned my addr 172.21.15.26:0/343314851 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:06.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c07fff640 1 --2- 172.21.15.26:0/343314851 >> v2:172.21.15.160:3300/0 conn(0x7f8c1006b3d0 0x7f8c100ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c0dd9b640 1 -- 172.21.15.26:0/343314851 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085643679 0 0) 0x7f8c1006bc30 con 0x7f8c100cec40 2023-12-19T18:18:06.197 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c0dd9b640 1 -- 172.21.15.26:0/343314851 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bfc003530 con 0x7f8c100cec40 2023-12-19T18:18:06.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c0dd9b640 1 -- 172.21.15.26:0/343314851 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002371825 0 0) 0x7f8c100cc6e0 con 0x7f8c10154220 2023-12-19T18:18:06.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c0dd9b640 1 -- 172.21.15.26:0/343314851 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c1006bc30 con 0x7f8c10154220 2023-12-19T18:18:06.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c07fff640 1 -- 172.21.15.26:0/343314851 >> v1:172.21.15.160:6789/0 conn(0x7f8c10154220 legacy=0x7f8c1015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c07fff640 1 -- 172.21.15.26:0/343314851 >> v1:172.21.15.26:6789/0 conn(0x7f8c100cec40 legacy=0x7f8c1015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.196+0000 7f8c07fff640 1 -- 172.21.15.26:0/343314851 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1006b950 con 0x7f8c1006b3d0 2023-12-19T18:18:06.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.200+0000 7f8c07fff640 1 --2- 172.21.15.26:0/343314851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1006b3d0 0x7f8c100ce700 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80097c0 tx=0x7f8bf801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0279fadd22d463e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.200+0000 7f8c0dd9b640 1 -- 172.21.15.26:0/343314851 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf80ab650 con 0x7f8c1006b3d0 2023-12-19T18:18:06.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.200+0000 7f8c0dd9b640 1 -- 172.21.15.26:0/343314851 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf80ab7b0 con 0x7f8c1006b3d0 2023-12-19T18:18:06.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.200+0000 7f8c0dd9b640 1 -- 172.21.15.26:0/343314851 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf80abb40 con 0x7f8c1006b3d0 2023-12-19T18:18:06.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.200+0000 7f8c148f9640 1 -- 172.21.15.26:0/343314851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1006b3d0 msgr2=0x7f8c100ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.200+0000 7f8c148f9640 1 --2- 172.21.15.26:0/343314851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1006b3d0 0x7f8c100ce700 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80097c0 tx=0x7f8bf801c450 comp rx=0 tx=0).stop 2023-12-19T18:18:06.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.200+0000 7f8c148f9640 1 -- 172.21.15.26:0/343314851 shutdown_connections 2023-12-19T18:18:06.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.200+0000 7f8c148f9640 1 --2- 172.21.15.26:0/343314851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1006b3d0 0x7f8c100ce700 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.200+0000 7f8c148f9640 1 -- 172.21.15.26:0/343314851 >> 172.21.15.26:0/343314851 conn(0x7f8c100658e0 msgr2=0x7f8c10065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:06.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.200+0000 7f8c148f9640 1 -- 172.21.15.26:0/343314851 shutdown_connections 2023-12-19T18:18:06.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.200+0000 7f8c148f9640 1 -- 172.21.15.26:0/343314851 wait complete. 2023-12-19T18:18:06.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c148f9640 1 Processor -- start 2023-12-19T18:18:06.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c148f9640 1 -- start start 2023-12-19T18:18:06.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c148f9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1006b3d0 0x7f8c100b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c148f9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c100cec40 0x7f8c1014a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c148f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c10154220 0x7f8c1014aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c148f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100b1460 con 0x7f8c10154220 2023-12-19T18:18:06.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c148f9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c10158f10 con 0x7f8c1006b3d0 2023-12-19T18:18:06.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c148f9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c10167b50 con 0x7f8c100cec40 2023-12-19T18:18:06.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c0ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c10154220 0x7f8c1014aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c0ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c10154220 0x7f8c1014aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54252/0 (socket says 172.21.15.26:54252) 2023-12-19T18:18:06.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c0ed9d640 1 -- 172.21.15.26:0/3969815743 learned_addr learned my addr 172.21.15.26:0/3969815743 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:06.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c0e59c640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1006b3d0 0x7f8c100b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c07fff640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c100cec40 0x7f8c1014a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c0ed9d640 1 -- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c100cec40 msgr2=0x7f8c1014a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c0ed9d640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c100cec40 0x7f8c1014a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c0ed9d640 1 -- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1006b3d0 msgr2=0x7f8c100b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c0ed9d640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1006b3d0 0x7f8c100b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c0ed9d640 1 -- 172.21.15.26:0/3969815743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf8009420 con 0x7f8c10154220 2023-12-19T18:18:06.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c0ed9d640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c10154220 0x7f8c1014aa20 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800cb40 tx=0x7f8c0800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c06ffd640 1 -- 172.21.15.26:0/3969815743 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800aca0 con 0x7f8c10154220 2023-12-19T18:18:06.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c06ffd640 1 -- 172.21.15.26:0/3969815743 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0800ae00 con 0x7f8c10154220 2023-12-19T18:18:06.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c148f9640 1 -- 172.21.15.26:0/3969815743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c10158c30 con 0x7f8c10154220 2023-12-19T18:18:06.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c06ffd640 1 -- 172.21.15.26:0/3969815743 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08012620 con 0x7f8c10154220 2023-12-19T18:18:06.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c148f9640 1 -- 172.21.15.26:0/3969815743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c101680d0 con 0x7f8c10154220 2023-12-19T18:18:06.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.204+0000 7f8c148f9640 1 -- 172.21.15.26:0/3969815743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c101679b0 con 0x7f8c10154220 2023-12-19T18:18:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.208+0000 7f8c06ffd640 1 -- 172.21.15.26:0/3969815743 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f8c08007070 con 0x7f8c10154220 2023-12-19T18:18:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.208+0000 7f8c06ffd640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8bf0065ca0 0x7f8bf0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.208+0000 7f8c06ffd640 1 -- 172.21.15.26:0/3969815743 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7341+0+0 (secure 0 0 0) 0x7f8c08083b90 con 0x7f8c10154220 2023-12-19T18:18:06.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.208+0000 7f8c0e59c640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8bf0065ca0 0x7f8bf0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.208+0000 7f8c0e59c640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8bf0065ca0 0x7f8bf0068160 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8c000011f0 tx=0x7f8c00009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.212+0000 7f8c06ffd640 1 -- 172.21.15.26:0/3969815743 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8c101679b0 con 0x7f8c10154220 2023-12-19T18:18:06.370 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.368+0000 7f8c148f9640 1 -- 172.21.15.26:0/3969815743 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c101679b0 con 0x7f8c10154220 2023-12-19T18:18:06.371 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.372+0000 7f8c06ffd640 1 -- 172.21.15.26:0/3969815743 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v242) v1 ==== 74+0+18467 (secure 0 0 0) 0x7f8c101679b0 con 0x7f8c10154220 2023-12-19T18:18:06.371 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:06.372 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":242,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:18:03.156388+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-19T18:17:37.484211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"236","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-19T18:17:38.799276+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"237","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/2285370499":"2023-12-20T18:17:59.801991+0000","172.21.15.26:6832/2285370499":"2023-12-20T18:17:59.801991+0000","172.21.15.160:0/598924254":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1489104438":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/621398302":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1548583339":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1246495635":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/1431434235":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/4035377594":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3900732845":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1388670559":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/3872447087":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/332404059":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/1786797637":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/563864573":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/2354390435":"2023-12-20T18:17:55.924288+0000","172.21.15.160:0/3396499768":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6833/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:6832/48394":"2023-12-20T18:17:49.880070+0000","172.21.15.160:0/359874050":"2023-12-20T18:17:49.880070+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:06.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.372+0000 7f8c148f9640 1 -- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8bf0065ca0 msgr2=0x7f8bf0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.373 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.372+0000 7f8c148f9640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8bf0065ca0 0x7f8bf0068160 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8c000011f0 tx=0x7f8c00009040 comp rx=0 tx=0).stop 2023-12-19T18:18:06.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.372+0000 7f8c148f9640 1 -- 172.21.15.26:0/3969815743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c10154220 msgr2=0x7f8c1014aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.372+0000 7f8c148f9640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c10154220 0x7f8c1014aa20 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800cb40 tx=0x7f8c0800a060 comp rx=0 tx=0).stop 2023-12-19T18:18:06.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.372+0000 7f8c148f9640 1 -- 172.21.15.26:0/3969815743 shutdown_connections 2023-12-19T18:18:06.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.372+0000 7f8c148f9640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f8bf0065ca0 0x7f8bf0068160 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.372+0000 7f8c148f9640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8c10154220 0x7f8c1014aa20 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.372+0000 7f8c148f9640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f8c100cec40 0x7f8c1014a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.372+0000 7f8c148f9640 1 --2- 172.21.15.26:0/3969815743 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c1006b3d0 0x7f8c100b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.372+0000 7f8c148f9640 1 -- 172.21.15.26:0/3969815743 >> 172.21.15.26:0/3969815743 conn(0x7f8c100658e0 msgr2=0x7f8c100cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:06.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.372+0000 7f8c148f9640 1 -- 172.21.15.26:0/3969815743 shutdown_connections 2023-12-19T18:18:06.374 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.372+0000 7f8c148f9640 1 -- 172.21.15.26:0/3969815743 wait complete. 2023-12-19T18:18:06.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-12-19T18:18:06.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.536+0000 7fed5c9f3640 1 Processor -- start 2023-12-19T18:18:06.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.536+0000 7fed5c9f3640 1 -- start start 2023-12-19T18:18:06.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.536+0000 7fed5c9f3640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed5815f370 0x7fed5815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.536+0000 7fed5c9f3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fed580ce740 0x7fed580ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.536+0000 7fed5c9f3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed580cf0b0 con 0x7fed58155030 2023-12-19T18:18:06.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.536+0000 7fed5c9f3640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5806c790 con 0x7fed5815f370 2023-12-19T18:18:06.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.536+0000 7fed5c9f3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5806c970 con 0x7fed580ce740 2023-12-19T18:18:06.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.536+0000 7fed55d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed5815f370 0x7fed5815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.540+0000 7fed55d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed5815f370 0x7fed5815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54258/0 (socket says 172.21.15.26:54258) 2023-12-19T18:18:06.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.540+0000 7fed55d9b640 1 -- 172.21.15.26:0/3976775411 learned_addr learned my addr 172.21.15.26:0/3976775411 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:06.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.540+0000 7fed56d9d640 1 --2- 172.21.15.26:0/3976775411 >> v2:172.21.15.160:3300/0 conn(0x7fed580ce740 0x7fed580ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.540+0000 7fed55d9b640 1 -- 172.21.15.26:0/3976775411 >> v2:172.21.15.160:3300/0 conn(0x7fed580ce740 msgr2=0x7fed580ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:06.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.540+0000 7fed55d9b640 1 --2- 172.21.15.26:0/3976775411 >> v2:172.21.15.160:3300/0 conn(0x7fed580ce740 0x7fed580ceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.540+0000 7fed55d9b640 1 -- 172.21.15.26:0/3976775411 >> v1:172.21.15.160:6789/0 conn(0x7fed58155030 legacy=0x7fed5806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.540+0000 7fed55d9b640 1 -- 172.21.15.26:0/3976775411 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5815b1f0 con 0x7fed5815f370 2023-12-19T18:18:06.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.540+0000 7fed55d9b640 1 --2- 172.21.15.26:0/3976775411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5815f370 0x7fed5815bcd0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fed580cc390 tx=0x7fed4c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97780d56378c994 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.540+0000 7fed5559a640 1 -- 172.21.15.26:0/3976775411 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0aa430 con 0x7fed5815f370 2023-12-19T18:18:06.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.540+0000 7fed5559a640 1 -- 172.21.15.26:0/3976775411 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c0aa590 con 0x7fed5815f370 2023-12-19T18:18:06.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.540+0000 7fed5559a640 1 -- 172.21.15.26:0/3976775411 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0aa920 con 0x7fed5815f370 2023-12-19T18:18:06.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 -- 172.21.15.26:0/3976775411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5815f370 msgr2=0x7fed5815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 --2- 172.21.15.26:0/3976775411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5815f370 0x7fed5815bcd0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fed580cc390 tx=0x7fed4c01c4d0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 -- 172.21.15.26:0/3976775411 shutdown_connections 2023-12-19T18:18:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 --2- 172.21.15.26:0/3976775411 >> v2:172.21.15.160:3300/0 conn(0x7fed580ce740 0x7fed580ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 --2- 172.21.15.26:0/3976775411 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5815f370 0x7fed5815bcd0 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 -- 172.21.15.26:0/3976775411 >> 172.21.15.26:0/3976775411 conn(0x7fed580658e0 msgr2=0x7fed58065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 -- 172.21.15.26:0/3976775411 shutdown_connections 2023-12-19T18:18:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 -- 172.21.15.26:0/3976775411 wait complete. 2023-12-19T18:18:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 Processor -- start 2023-12-19T18:18:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 -- start start 2023-12-19T18:18:06.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed580ce740 0x7fed58141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed58155030 0x7fed58145aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5815f370 0x7fed580ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed58142130 con 0x7fed5815f370 2023-12-19T18:18:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5814d010 con 0x7fed58155030 2023-12-19T18:18:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5c9f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed58146310 con 0x7fed580ce740 2023-12-19T18:18:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed56d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5815f370 0x7fed580ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed56d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5815f370 0x7fed580ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54266/0 (socket says 172.21.15.26:54266) 2023-12-19T18:18:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed56d9d640 1 -- 172.21.15.26:0/985962830 learned_addr learned my addr 172.21.15.26:0/985962830 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.544+0000 7fed5659c640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed580ce740 0x7fed58141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed56d9d640 1 -- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed580ce740 msgr2=0x7fed58141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed56d9d640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed580ce740 0x7fed58141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed56d9d640 1 -- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed58155030 msgr2=0x7fed58145aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed56d9d640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed58155030 0x7fed58145aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed56d9d640 1 -- 172.21.15.26:0/985962830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c0094d0 con 0x7fed5815f370 2023-12-19T18:18:06.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed5659c640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed580ce740 0x7fed58141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:06.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed56d9d640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5815f370 0x7fed580ac850 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fed50007e40 tx=0x7fed5000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed46ffd640 1 -- 172.21.15.26:0/985962830 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed50014620 con 0x7fed5815f370 2023-12-19T18:18:06.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed46ffd640 1 -- 172.21.15.26:0/985962830 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5000ee90 con 0x7fed5815f370 2023-12-19T18:18:06.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed46ffd640 1 -- 172.21.15.26:0/985962830 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed500149e0 con 0x7fed5815f370 2023-12-19T18:18:06.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed5c9f3640 1 -- 172.21.15.26:0/985962830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed581465f0 con 0x7fed5815f370 2023-12-19T18:18:06.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed5c9f3640 1 -- 172.21.15.26:0/985962830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed58146bb0 con 0x7fed5815f370 2023-12-19T18:18:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed5c9f3640 1 -- 172.21.15.26:0/985962830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed581422a0 con 0x7fed5815f370 2023-12-19T18:18:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed46ffd640 1 -- 172.21.15.26:0/985962830 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fed50014b40 con 0x7fed5815f370 2023-12-19T18:18:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed46ffd640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fed38065cd0 0x7fed38068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed46ffd640 1 -- 172.21.15.26:0/985962830 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 7341+0+0 (secure 0 0 0) 0x7fed500845a0 con 0x7fed5815f370 2023-12-19T18:18:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.548+0000 7fed5659c640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fed38065cd0 0x7fed38068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.552+0000 7fed5659c640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fed38065cd0 0x7fed38068190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fed581439b0 tx=0x7fed48009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.552+0000 7fed46ffd640 1 -- 172.21.15.26:0/985962830 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fed581422a0 con 0x7fed5815f370 2023-12-19T18:18:06.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.744+0000 7fed5c9f3640 1 -- 172.21.15.26:0/985962830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7fed58142270 con 0x7fed5815f370 2023-12-19T18:18:06.744 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T18:18:06.744+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:18:06.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.752+0000 7fed46ffd640 1 -- 172.21.15.26:0/985962830 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v121) v1 ==== 167+0+0 (secure 0 0 0) 0x7fed500515d0 con 0x7fed5815f370 2023-12-19T18:18:06.755 INFO:teuthology.orchestra.run.smithi026.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:18:06.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.756+0000 7fed5c9f3640 1 -- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fed38065cd0 msgr2=0x7fed38068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.756+0000 7fed5c9f3640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fed38065cd0 0x7fed38068190 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fed581439b0 tx=0x7fed48009040 comp rx=0 tx=0).stop 2023-12-19T18:18:06.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.756+0000 7fed5c9f3640 1 -- 172.21.15.26:0/985962830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5815f370 msgr2=0x7fed580ac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.756+0000 7fed5c9f3640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5815f370 0x7fed580ac850 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fed50007e40 tx=0x7fed5000e4b0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.756+0000 7fed5c9f3640 1 -- 172.21.15.26:0/985962830 shutdown_connections 2023-12-19T18:18:06.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.756+0000 7fed5c9f3640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fed38065cd0 0x7fed38068190 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.756+0000 7fed5c9f3640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5815f370 0x7fed580ac850 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.756+0000 7fed5c9f3640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fed58155030 0x7fed58145aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.756+0000 7fed5c9f3640 1 --2- 172.21.15.26:0/985962830 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fed580ce740 0x7fed58141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.756+0000 7fed5c9f3640 1 -- 172.21.15.26:0/985962830 >> 172.21.15.26:0/985962830 conn(0x7fed580658e0 msgr2=0x7fed58065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:06.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.756+0000 7fed5c9f3640 1 -- 172.21.15.26:0/985962830 shutdown_connections 2023-12-19T18:18:06.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.756+0000 7fed5c9f3640 1 -- 172.21.15.26:0/985962830 wait complete. 2023-12-19T18:18:06.760 INFO:tasks.ceph.mds.a.smithi026.stderr: -1534> 2023-12-19T18:17:43.825+0000 7f9d5c382640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:18:06.777 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-12-19T18:18:06.804 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T18:18:06.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03bbf6d640 1 Processor -- start 2023-12-19T18:18:06.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03bbf6d640 1 -- start start 2023-12-19T18:18:06.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03bbf6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03b41585f0 0x7f03b40cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03bbf6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03b40cb150 0x7f03b40cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03bbf6d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f03b40cc310 0x7f03b40cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03bbf6d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b40cbb10 con 0x7f03b41585f0 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03bbf6d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b40b47a0 con 0x7f03b40cc310 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03bbf6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b415b1b0 con 0x7f03b40cb150 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03b9d09640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03b41585f0 0x7f03b40cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03b9d09640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f03b41585f0 0x7f03b40cdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54300/0 (socket says 172.21.15.26:54300) 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03b9d09640 1 -- 172.21.15.26:0/3962133942 learned_addr learned my addr 172.21.15.26:0/3962133942 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03b9508640 1 --2- 172.21.15.26:0/3962133942 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03b40cb150 0x7f03b40cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03ba50a640 1 --2- 172.21.15.26:0/3962133942 >> v2:172.21.15.160:3300/0 conn(0x7f03b40cc310 0x7f03b40cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03b9d09640 1 -- 172.21.15.26:0/3962133942 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03b40cb150 msgr2=0x7f03b40cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03b9d09640 1 --2- 172.21.15.26:0/3962133942 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03b40cb150 0x7f03b40cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03b9d09640 1 -- 172.21.15.26:0/3962133942 >> v2:172.21.15.160:3300/0 conn(0x7f03b40cc310 msgr2=0x7f03b40cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:06.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03b9d09640 1 --2- 172.21.15.26:0/3962133942 >> v2:172.21.15.160:3300/0 conn(0x7f03b40cc310 0x7f03b40cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03b9d09640 1 -- 172.21.15.26:0/3962133942 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b415b490 con 0x7f03b41585f0 2023-12-19T18:18:06.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03b9508640 1 --2- 172.21.15.26:0/3962133942 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03b40cb150 0x7f03b40cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:06.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.924+0000 7f03ba50a640 1 --2- 172.21.15.26:0/3962133942 >> v2:172.21.15.160:3300/0 conn(0x7f03b40cc310 0x7f03b40cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:06.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.928+0000 7f03b9d09640 1 --2- 172.21.15.26:0/3962133942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b41585f0 0x7f03b40cdce0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f03b406d740 tx=0x7f03b001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233f57d11a3ffcde server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.928+0000 7f03b8d07640 1 -- 172.21.15.26:0/3962133942 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b00ab600 con 0x7f03b41585f0 2023-12-19T18:18:06.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.928+0000 7f03b8d07640 1 -- 172.21.15.26:0/3962133942 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b00ab760 con 0x7f03b41585f0 2023-12-19T18:18:06.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.928+0000 7f03b8d07640 1 -- 172.21.15.26:0/3962133942 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b00abaf0 con 0x7f03b41585f0 2023-12-19T18:18:06.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.928+0000 7f03bbf6d640 1 -- 172.21.15.26:0/3962133942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b41585f0 msgr2=0x7f03b40cdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.928+0000 7f03bbf6d640 1 --2- 172.21.15.26:0/3962133942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b41585f0 0x7f03b40cdce0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7f03b406d740 tx=0x7f03b001c440 comp rx=0 tx=0).stop 2023-12-19T18:18:06.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.928+0000 7f03bbf6d640 1 -- 172.21.15.26:0/3962133942 shutdown_connections 2023-12-19T18:18:06.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.928+0000 7f03bbf6d640 1 --2- 172.21.15.26:0/3962133942 >> v2:172.21.15.160:3300/0 conn(0x7f03b40cc310 0x7f03b40cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.928+0000 7f03bbf6d640 1 --2- 172.21.15.26:0/3962133942 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03b40cb150 0x7f03b40cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.928+0000 7f03bbf6d640 1 --2- 172.21.15.26:0/3962133942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b41585f0 0x7f03b40cdce0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.928+0000 7f03bbf6d640 1 -- 172.21.15.26:0/3962133942 >> 172.21.15.26:0/3962133942 conn(0x7f03b4066160 msgr2=0x7f03b40b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:06.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.928+0000 7f03bbf6d640 1 -- 172.21.15.26:0/3962133942 shutdown_connections 2023-12-19T18:18:06.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.928+0000 7f03bbf6d640 1 -- 172.21.15.26:0/3962133942 wait complete. 2023-12-19T18:18:06.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03bbf6d640 1 Processor -- start 2023-12-19T18:18:06.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03bbf6d640 1 -- start start 2023-12-19T18:18:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03bbf6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b40cb150 0x7f03b4141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03bbf6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03b40cc310 0x7f03b4143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03bbf6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03b41585f0 0x7f03b40af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03bbf6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b4155930 con 0x7f03b40cb150 2023-12-19T18:18:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03bbf6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b4141d60 con 0x7f03b40cc310 2023-12-19T18:18:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03bbf6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b4144090 con 0x7f03b41585f0 2023-12-19T18:18:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03b9d09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b40cb150 0x7f03b4141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03b9d09640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b40cb150 0x7f03b4141820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54308/0 (socket says 172.21.15.26:54308) 2023-12-19T18:18:06.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03b9d09640 1 -- 172.21.15.26:0/1169990650 learned_addr learned my addr 172.21.15.26:0/1169990650 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03ba50a640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03b41585f0 0x7f03b40af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03b9508640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03b40cc310 0x7f03b4143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03b9d09640 1 -- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03b41585f0 msgr2=0x7f03b40af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03b9d09640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03b41585f0 0x7f03b40af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03b9d09640 1 -- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03b40cc310 msgr2=0x7f03b4143870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03b9d09640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03b40cc310 0x7f03b4143870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03b9d09640 1 -- 172.21.15.26:0/1169990650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b00094d0 con 0x7f03b40cb150 2023-12-19T18:18:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03b9508640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03b40cc310 0x7f03b4143870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03b9d09640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b40cb150 0x7f03b4141820 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f03b00ad040 tx=0x7f03b001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03aa7fc640 1 -- 172.21.15.26:0/1169990650 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b00b4800 con 0x7f03b40cb150 2023-12-19T18:18:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03aa7fc640 1 -- 172.21.15.26:0/1169990650 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b00b4980 con 0x7f03b40cb150 2023-12-19T18:18:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03aa7fc640 1 -- 172.21.15.26:0/1169990650 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b00b4d30 con 0x7f03b40cb150 2023-12-19T18:18:06.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03bbf6d640 1 -- 172.21.15.26:0/1169990650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b4144370 con 0x7f03b40cb150 2023-12-19T18:18:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03bbf6d640 1 -- 172.21.15.26:0/1169990650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b4155b40 con 0x7f03b40cb150 2023-12-19T18:18:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.932+0000 7f03bbf6d640 1 -- 172.21.15.26:0/1169990650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b4144820 con 0x7f03b40cb150 2023-12-19T18:18:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.936+0000 7f03aa7fc640 1 -- 172.21.15.26:0/1169990650 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f03b00c7440 con 0x7f03b40cb150 2023-12-19T18:18:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.936+0000 7f03aa7fc640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0398065ca0 0x7f0398068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.936+0000 7f03aa7fc640 1 -- 172.21.15.26:0/1169990650 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7427+0+0 (secure 0 0 0) 0x7f03b011e7e0 con 0x7f03b40cb150 2023-12-19T18:18:06.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.936+0000 7f03b9508640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0398065ca0 0x7f0398068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.936+0000 7f03b9508640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0398065ca0 0x7f0398068160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03b415a950 tx=0x7f03a4008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:06.940+0000 7f03aa7fc640 1 -- 172.21.15.26:0/1169990650 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f03b00eb710 con 0x7f03b40cb150 2023-12-19T18:18:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.136+0000 7f03bbf6d640 1 -- 172.21.15.26:0/1169990650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f03b4074270 con 0x7f03b40cb150 2023-12-19T18:18:07.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.756+0000 7f03aa7fc640 1 -- 172.21.15.26:0/1169990650 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v122) v1 ==== 107+0+0 (secure 0 0 0) 0x7f03b00eed30 con 0x7f03b40cb150 2023-12-19T18:18:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.760+0000 7f03bbf6d640 1 -- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0398065ca0 msgr2=0x7f0398068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:07.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.760+0000 7f03bbf6d640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0398065ca0 0x7f0398068160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03b415a950 tx=0x7f03a4008040 comp rx=0 tx=0).stop 2023-12-19T18:18:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.760+0000 7f03bbf6d640 1 -- 172.21.15.26:0/1169990650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b40cb150 msgr2=0x7f03b4141820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.760+0000 7f03bbf6d640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b40cb150 0x7f03b4141820 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f03b00ad040 tx=0x7f03b001ac80 comp rx=0 tx=0).stop 2023-12-19T18:18:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.760+0000 7f03bbf6d640 1 -- 172.21.15.26:0/1169990650 shutdown_connections 2023-12-19T18:18:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.760+0000 7f03bbf6d640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0398065ca0 0x7f0398068160 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.760+0000 7f03bbf6d640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f03b41585f0 0x7f03b40af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.760+0000 7f03bbf6d640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03b40cc310 0x7f03b4143870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.760+0000 7f03bbf6d640 1 --2- 172.21.15.26:0/1169990650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b40cb150 0x7f03b4141820 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.760+0000 7f03bbf6d640 1 -- 172.21.15.26:0/1169990650 >> 172.21.15.26:0/1169990650 conn(0x7f03b4066160 msgr2=0x7f03b41593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.760+0000 7f03bbf6d640 1 -- 172.21.15.26:0/1169990650 shutdown_connections 2023-12-19T18:18:07.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.760+0000 7f03bbf6d640 1 -- 172.21.15.26:0/1169990650 wait complete. 2023-12-19T18:18:07.785 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-12-19T18:18:07.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.984+0000 7f83fe066640 1 Processor -- start 2023-12-19T18:18:07.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.984+0000 7f83fe066640 1 -- start start 2023-12-19T18:18:07.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.984+0000 7f83fe066640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815ccf0 0x7f83f80ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:07.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.984+0000 7f83fe066640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f80cd100 con 0x7f83f80ce7f0 2023-12-19T18:18:07.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.984+0000 7f83fe066640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f806da40 con 0x7f83f806c0f0 2023-12-19T18:18:07.986 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.984+0000 7f83fe066640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f806e000 con 0x7f83f815ccf0 2023-12-19T18:18:07.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.984+0000 7f83f6ffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f83f80ce7f0 0x7f83f806b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35278/0 (socket says 172.21.15.26:35278) 2023-12-19T18:18:07.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.984+0000 7f83f6ffd640 1 -- 172.21.15.26:0/4271502393 learned_addr learned my addr 172.21.15.26:0/4271502393 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f77fe640 1 --2- 172.21.15.26:0/4271502393 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815ccf0 0x7f83f80ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032419715 0 0) 0x7f83f80cd100 con 0x7f83f80ce7f0 2023-12-19T18:18:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e0003530 con 0x7f83f80ce7f0 2023-12-19T18:18:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 543516833 0 0) 0x7f83e0003530 con 0x7f83f80ce7f0 2023-12-19T18:18:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815ccf0 msgr2=0x7f83f80ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 --2- 172.21.15.26:0/4271502393 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815ccf0 0x7f83f80ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 >> v1:172.21.15.160:6789/0 conn(0x7f83f806c0f0 legacy=0x7f83f80ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f806dd20 con 0x7f83f80ce7f0 2023-12-19T18:18:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f83ec001680 con 0x7f83f80ce7f0 2023-12-19T18:18:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 >> v1:172.21.15.26:6789/0 conn(0x7f83f80ce7f0 legacy=0x7f83f806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 --2- 172.21.15.26:0/4271502393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e00039b0 0x7f83e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 --2- 172.21.15.26:0/4271502393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83e0004360 0x7f83e0004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 --2- 172.21.15.26:0/4271502393 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815ccf0 0x7f83e0005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f806e000 con 0x7f83e00039b0 2023-12-19T18:18:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e000ba40 con 0x7f83e0004360 2023-12-19T18:18:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e000bc00 con 0x7f83f815ccf0 2023-12-19T18:18:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f77fe640 1 --2- 172.21.15.26:0/4271502393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e00039b0 0x7f83e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f6ffd640 1 --2- 172.21.15.26:0/4271502393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83e0004360 0x7f83e0004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f7fff640 1 --2- 172.21.15.26:0/4271502393 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815ccf0 0x7f83e0005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f77fe640 1 -- 172.21.15.26:0/4271502393 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815ccf0 msgr2=0x7f83e0005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f77fe640 1 --2- 172.21.15.26:0/4271502393 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815ccf0 0x7f83e0005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f77fe640 1 -- 172.21.15.26:0/4271502393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83e0004360 msgr2=0x7f83e0004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f77fe640 1 --2- 172.21.15.26:0/4271502393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83e0004360 0x7f83e0004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.988+0000 7f83f77fe640 1 -- 172.21.15.26:0/4271502393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e0003530 con 0x7f83e00039b0 2023-12-19T18:18:07.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.992+0000 7f83f77fe640 1 --2- 172.21.15.26:0/4271502393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e00039b0 0x7f83e0003d90 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009ac0 tx=0x7f83e801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6e2e87e3ad43f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:07.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.992+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80059a0 con 0x7f83e00039b0 2023-12-19T18:18:07.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.992+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e8005b00 con 0x7f83e00039b0 2023-12-19T18:18:07.991 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.992+0000 7f83f67fc640 1 -- 172.21.15.26:0/4271502393 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8006030 con 0x7f83e00039b0 2023-12-19T18:18:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.992+0000 7f83fe066640 1 -- 172.21.15.26:0/4271502393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e00039b0 msgr2=0x7f83e0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.992+0000 7f83fe066640 1 --2- 172.21.15.26:0/4271502393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e00039b0 0x7f83e0003d90 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009ac0 tx=0x7f83e801ca30 comp rx=0 tx=0).stop 2023-12-19T18:18:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.992+0000 7f83f7fff640 1 -- 172.21.15.26:0/4271502393 reap_dead start 2023-12-19T18:18:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.992+0000 7f83fe066640 1 -- 172.21.15.26:0/4271502393 shutdown_connections 2023-12-19T18:18:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.992+0000 7f83fe066640 1 -- 172.21.15.26:0/4271502393 >> 172.21.15.26:0/4271502393 conn(0x7f83f80cf020 msgr2=0x7f83f80cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.992+0000 7f83fe066640 1 -- 172.21.15.26:0/4271502393 shutdown_connections 2023-12-19T18:18:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.992+0000 7f83fe066640 1 -- 172.21.15.26:0/4271502393 wait complete. 2023-12-19T18:18:07.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.992+0000 7f83fe066640 1 Processor -- start 2023-12-19T18:18:07.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.992+0000 7f83fe066640 1 -- start start 2023-12-19T18:18:07.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83fe066640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e00039b0 0x7f83f815eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:07.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83fe066640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83f8156d20 0x7f83f815f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:07.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83fe066640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815f730 0x7f83f815d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:07.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83fe066640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f8159aa0 con 0x7f83e00039b0 2023-12-19T18:18:07.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83fe066640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80c4ab0 con 0x7f83f8156d20 2023-12-19T18:18:07.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83fe066640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80c4d90 con 0x7f83f815f730 2023-12-19T18:18:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83f77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e00039b0 0x7f83f815eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83f77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e00039b0 0x7f83f815eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54328/0 (socket says 172.21.15.26:54328) 2023-12-19T18:18:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83f77fe640 1 -- 172.21.15.26:0/3977905471 learned_addr learned my addr 172.21.15.26:0/3977905471 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83f6ffd640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83f8156d20 0x7f83f815f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83f7fff640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815f730 0x7f83f815d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83f77fe640 1 -- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815f730 msgr2=0x7f83f815d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83f77fe640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815f730 0x7f83f815d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83f77fe640 1 -- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83f8156d20 msgr2=0x7f83f815f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83f77fe640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83f8156d20 0x7f83f815f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83f77fe640 1 -- 172.21.15.26:0/3977905471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f80c5070 con 0x7f83e00039b0 2023-12-19T18:18:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83f6ffd640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83f8156d20 0x7f83f815f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83f7fff640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815f730 0x7f83f815d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:07.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83f77fe640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e00039b0 0x7f83f815eae0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f83e801c860 tx=0x7f83e0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:07.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83dffff640 1 -- 172.21.15.26:0/3977905471 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8006750 con 0x7f83e00039b0 2023-12-19T18:18:07.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83dffff640 1 -- 172.21.15.26:0/3977905471 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e8005c70 con 0x7f83e00039b0 2023-12-19T18:18:07.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83dffff640 1 -- 172.21.15.26:0/3977905471 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80068f0 con 0x7f83e00039b0 2023-12-19T18:18:07.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83fe066640 1 -- 172.21.15.26:0/3977905471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f80c5350 con 0x7f83e00039b0 2023-12-19T18:18:07.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83fe066640 1 -- 172.21.15.26:0/3977905471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f80c4730 con 0x7f83e00039b0 2023-12-19T18:18:07.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83fe066640 1 -- 172.21.15.26:0/3977905471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f8074270 con 0x7f83e00039b0 2023-12-19T18:18:07.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:07.996+0000 7f83dffff640 1 -- 172.21.15.26:0/3977905471 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f83e8004020 con 0x7f83e00039b0 2023-12-19T18:18:07.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.000+0000 7f83dffff640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f83e4065cd0 0x7f83e4068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:07.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.000+0000 7f83dffff640 1 -- 172.21.15.26:0/3977905471 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(243..243 src has 1..243) v4 ==== 7427+0+0 (secure 0 0 0) 0x7f83e811e1d0 con 0x7f83e00039b0 2023-12-19T18:18:07.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.000+0000 7f83f6ffd640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f83e4065cd0 0x7f83e4068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.000 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.000+0000 7f83f6ffd640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f83e4065cd0 0x7f83e4068190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f83f80aff30 tx=0x7f83ec009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.004+0000 7f83dffff640 1 -- 172.21.15.26:0/3977905471 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f83e80eb160 con 0x7f83e00039b0 2023-12-19T18:18:08.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.160+0000 7f83fe066640 1 -- 172.21.15.26:0/3977905471 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f83f8067c70 con 0x7f83e00039b0 2023-12-19T18:18:08.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.764+0000 7f83dffff640 1 -- 172.21.15.26:0/3977905471 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v244) v1 ==== 190+0+0 (secure 0 0 0) 0x7f83e80ee780 con 0x7f83e00039b0 2023-12-19T18:18:08.766 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'backup_fs_metadata' removed 2023-12-19T18:18:08.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.768+0000 7f83fe066640 1 -- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f83e4065cd0 msgr2=0x7f83e4068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.768+0000 7f83fe066640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f83e4065cd0 0x7f83e4068190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f83f80aff30 tx=0x7f83ec009040 comp rx=0 tx=0).stop 2023-12-19T18:18:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.768+0000 7f83fe066640 1 -- 172.21.15.26:0/3977905471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e00039b0 msgr2=0x7f83f815eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.768+0000 7f83fe066640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e00039b0 0x7f83f815eae0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f83e801c860 tx=0x7f83e0005800 comp rx=0 tx=0).stop 2023-12-19T18:18:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.768+0000 7f83fe066640 1 -- 172.21.15.26:0/3977905471 shutdown_connections 2023-12-19T18:18:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.768+0000 7f83fe066640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f83e4065cd0 0x7f83e4068190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.768+0000 7f83fe066640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f83f815f730 0x7f83f815d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.768+0000 7f83fe066640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83f8156d20 0x7f83f815f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.768+0000 7f83fe066640 1 --2- 172.21.15.26:0/3977905471 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83e00039b0 0x7f83f815eae0 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.768+0000 7f83fe066640 1 -- 172.21.15.26:0/3977905471 >> 172.21.15.26:0/3977905471 conn(0x7f83f80cf020 msgr2=0x7f83f81607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.768+0000 7f83fe066640 1 -- 172.21.15.26:0/3977905471 shutdown_connections 2023-12-19T18:18:08.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.768+0000 7f83fe066640 1 -- 172.21.15.26:0/3977905471 wait complete. 2023-12-19T18:18:08.792 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-12-19T18:18:08.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.940+0000 7f0f0cfc0640 1 Processor -- start 2023-12-19T18:18:08.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.940+0000 7f0f0cfc0640 1 -- start start 2023-12-19T18:18:08.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0f0cfc0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0f0815cf10 0x7f0f080ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0f0cfc0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f080ce7f0 0x7f0f0806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0f0cfc0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f080ccbf0 con 0x7f0f0806bf20 2023-12-19T18:18:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0f0cfc0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0806c330 con 0x7f0f080ce7f0 2023-12-19T18:18:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0f0cfc0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0806dae0 con 0x7f0f0815cf10 2023-12-19T18:18:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0efffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f080ce7f0 0x7f0f0806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0efffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0f080ce7f0 0x7f0f0806b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54332/0 (socket says 172.21.15.26:54332) 2023-12-19T18:18:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0efffff640 1 -- 172.21.15.26:0/3797884645 learned_addr learned my addr 172.21.15.26:0/3797884645 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0f06d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0f0806bf20 0x7f0f08160860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35288/0 (socket says 172.21.15.26:35288) 2023-12-19T18:18:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0f0659c640 1 --2- 172.21.15.26:0/3797884645 >> v2:172.21.15.160:3300/0 conn(0x7f0f0815cf10 0x7f0f080ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0efffff640 1 -- 172.21.15.26:0/3797884645 >> v2:172.21.15.160:3300/0 conn(0x7f0f0815cf10 msgr2=0x7f0f080ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0efffff640 1 --2- 172.21.15.26:0/3797884645 >> v2:172.21.15.160:3300/0 conn(0x7f0f0815cf10 0x7f0f080ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0efffff640 1 -- 172.21.15.26:0/3797884645 >> v1:172.21.15.26:6789/0 conn(0x7f0f0806bf20 legacy=0x7f0f08160860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0efffff640 1 -- 172.21.15.26:0/3797884645 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0806ddc0 con 0x7f0f080ce7f0 2023-12-19T18:18:08.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.944+0000 7f0f05d9b640 1 -- 172.21.15.26:0/3797884645 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649606825 0 0) 0x7f0f080ccbf0 con 0x7f0f0806bf20 2023-12-19T18:18:08.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0efffff640 1 --2- 172.21.15.26:0/3797884645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f080ce7f0 0x7f0f0806b9e0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f0f0806e3d0 tx=0x7f0ef001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3451bcecf3b637 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:08.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0f05d9b640 1 -- 172.21.15.26:0/3797884645 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00ab640 con 0x7f0f080ce7f0 2023-12-19T18:18:08.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0f05d9b640 1 -- 172.21.15.26:0/3797884645 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ef00a7070 con 0x7f0f080ce7f0 2023-12-19T18:18:08.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0f05d9b640 1 -- 172.21.15.26:0/3797884645 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ef00ab9c0 con 0x7f0f080ce7f0 2023-12-19T18:18:08.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3797884645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f080ce7f0 msgr2=0x7f0f0806b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0f0cfc0640 1 --2- 172.21.15.26:0/3797884645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f080ce7f0 0x7f0f0806b9e0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f0f0806e3d0 tx=0x7f0ef001c440 comp rx=0 tx=0).stop 2023-12-19T18:18:08.950 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3797884645 shutdown_connections 2023-12-19T18:18:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0f0cfc0640 1 --2- 172.21.15.26:0/3797884645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f080ce7f0 0x7f0f0806b9e0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0f0cfc0640 1 --2- 172.21.15.26:0/3797884645 >> v2:172.21.15.160:3300/0 conn(0x7f0f0815cf10 0x7f0f080ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3797884645 >> 172.21.15.26:0/3797884645 conn(0x7f0f080cf020 msgr2=0x7f0f080cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3797884645 shutdown_connections 2023-12-19T18:18:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3797884645 wait complete. 2023-12-19T18:18:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0f0cfc0640 1 Processor -- start 2023-12-19T18:18:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.948+0000 7f0f0cfc0640 1 -- start start 2023-12-19T18:18:08.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f0cfc0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f0806bf20 0x7f0f0815fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f0cfc0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f080ce7f0 0x7f0f080c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f0cfc0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0815cf10 0x7f0f080ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f0cfc0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0815d2f0 con 0x7f0f0815cf10 2023-12-19T18:18:08.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f0cfc0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f080c8d00 con 0x7f0f080ce7f0 2023-12-19T18:18:08.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f0cfc0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f081552f0 con 0x7f0f0806bf20 2023-12-19T18:18:08.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f06d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0815cf10 0x7f0f080ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f06d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0815cf10 0x7f0f080ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54348/0 (socket says 172.21.15.26:54348) 2023-12-19T18:18:08.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f06d9d640 1 -- 172.21.15.26:0/3340533513 learned_addr learned my addr 172.21.15.26:0/3340533513 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:08.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0efffff640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f080ce7f0 0x7f0f080c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f0659c640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f0806bf20 0x7f0f0815fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f06d9d640 1 -- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f0806bf20 msgr2=0x7f0f0815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f06d9d640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f0806bf20 0x7f0f0815fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f06d9d640 1 -- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f080ce7f0 msgr2=0x7f0f080c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f06d9d640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f080ce7f0 0x7f0f080c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f06d9d640 1 -- 172.21.15.26:0/3340533513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef00094d0 con 0x7f0f0815cf10 2023-12-19T18:18:08.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f06d9d640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0815cf10 0x7f0f080ca9a0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c7d0 tx=0x7f0f0000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:08.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0efeffd640 1 -- 172.21.15.26:0/3340533513 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000a640 con 0x7f0f0815cf10 2023-12-19T18:18:08.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0efeffd640 1 -- 172.21.15.26:0/3340533513 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0000a7a0 con 0x7f0f0815cf10 2023-12-19T18:18:08.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3340533513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f080caee0 con 0x7f0f0815cf10 2023-12-19T18:18:08.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0efeffd640 1 -- 172.21.15.26:0/3340533513 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0000ab90 con 0x7f0f0815cf10 2023-12-19T18:18:08.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3340533513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0815d820 con 0x7f0f0815cf10 2023-12-19T18:18:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3340533513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f081554b0 con 0x7f0f0815cf10 2023-12-19T18:18:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.952+0000 7f0efeffd640 1 -- 172.21.15.26:0/3340533513 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f0f0000acf0 con 0x7f0f0815cf10 2023-12-19T18:18:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.956+0000 7f0efeffd640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0ee8065ca0 0x7f0ee8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.956+0000 7f0efeffd640 1 -- 172.21.15.26:0/3340533513 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(244..244 src has 1..244) v4 ==== 6983+0+0 (secure 0 0 0) 0x7f0f00083260 con 0x7f0f0815cf10 2023-12-19T18:18:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.956+0000 7f0f0659c640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0ee8065ca0 0x7f0ee8068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.956+0000 7f0f0659c640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0ee8065ca0 0x7f0ee8068160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0f08070fc0 tx=0x7f0ef8008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:08.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:08.960+0000 7f0efeffd640 1 -- 172.21.15.26:0/3340533513 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0f00050460 con 0x7f0f0815cf10 2023-12-19T18:18:09.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:09.116+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3340533513 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0f08074270 con 0x7f0f0815cf10 2023-12-19T18:18:10.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.112+0000 7f0efeffd640 1 -- 172.21.15.26:0/3340533513 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v245) v1 ==== 178+0+0 (secure 0 0 0) 0x7f0f00053a80 con 0x7f0f0815cf10 2023-12-19T18:18:10.115 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'backup_fs_data' removed 2023-12-19T18:18:10.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.116+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0ee8065ca0 msgr2=0x7f0ee8068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.116+0000 7f0f0cfc0640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0ee8065ca0 0x7f0ee8068160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0f08070fc0 tx=0x7f0ef8008040 comp rx=0 tx=0).stop 2023-12-19T18:18:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.116+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3340533513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0815cf10 msgr2=0x7f0f080ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.116+0000 7f0f0cfc0640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0815cf10 0x7f0f080ca9a0 secure :-1 s=READY pgs=2291 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000c7d0 tx=0x7f0f0000cc90 comp rx=0 tx=0).stop 2023-12-19T18:18:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.116+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3340533513 shutdown_connections 2023-12-19T18:18:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.116+0000 7f0f0cfc0640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0ee8065ca0 0x7f0ee8068160 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.116+0000 7f0f0cfc0640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f0815cf10 0x7f0f080ca9a0 unknown :-1 s=CLOSED pgs=2291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.116+0000 7f0f0cfc0640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0f080ce7f0 0x7f0f080c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.116+0000 7f0f0cfc0640 1 --2- 172.21.15.26:0/3340533513 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0f0806bf20 0x7f0f0815fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.116+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3340533513 >> 172.21.15.26:0/3340533513 conn(0x7f0f080cf020 msgr2=0x7f0f08070580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:10.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.116+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3340533513 shutdown_connections 2023-12-19T18:18:10.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.116+0000 7f0f0cfc0640 1 -- 172.21.15.26:0/3340533513 wait complete. 2023-12-19T18:18:10.144 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-19T18:18:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.296+0000 7fee8c266640 1 Processor -- start 2023-12-19T18:18:10.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.296+0000 7fee8c266640 1 -- start start 2023-12-19T18:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee8c266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee840cb600 0x7fee8415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee8c266640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee840ce3f0 con 0x7fee841585f0 2023-12-19T18:18:10.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee8c266640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee840cba50 con 0x7fee840cb150 2023-12-19T18:18:10.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee8c266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee840b47a0 con 0x7fee840cb600 2023-12-19T18:18:10.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee8a002640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fee841585f0 0x7fee840cdce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35300/0 (socket says 172.21.15.26:35300) 2023-12-19T18:18:10.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee8a002640 1 -- 172.21.15.26:0/1440749580 learned_addr learned my addr 172.21.15.26:0/1440749580 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:10.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee89801640 1 --1- 172.21.15.26:0/1440749580 >> v1:172.21.15.160:6789/0 conn(0x7fee840cb150 0x7fee8415e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:49444/0 (socket says 172.21.15.26:49444) 2023-12-19T18:18:10.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee8a803640 1 --2- 172.21.15.26:0/1440749580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee840cb600 0x7fee8415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee89000640 1 -- 172.21.15.26:0/1440749580 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172620287 0 0) 0x7fee840ce3f0 con 0x7fee841585f0 2023-12-19T18:18:10.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee89000640 1 -- 172.21.15.26:0/1440749580 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee70003530 con 0x7fee841585f0 2023-12-19T18:18:10.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee89000640 1 -- 172.21.15.26:0/1440749580 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1499212194 0 0) 0x7fee840cba50 con 0x7fee840cb150 2023-12-19T18:18:10.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee89000640 1 -- 172.21.15.26:0/1440749580 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee840ce3f0 con 0x7fee840cb150 2023-12-19T18:18:10.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee8a803640 1 -- 172.21.15.26:0/1440749580 >> v1:172.21.15.160:6789/0 conn(0x7fee840cb150 legacy=0x7fee8415e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee8a803640 1 -- 172.21.15.26:0/1440749580 >> v1:172.21.15.26:6789/0 conn(0x7fee841585f0 legacy=0x7fee840cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.300+0000 7fee8a803640 1 -- 172.21.15.26:0/1440749580 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8415b1b0 con 0x7fee840cb600 2023-12-19T18:18:10.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee8a803640 1 --2- 172.21.15.26:0/1440749580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee840cb600 0x7fee8415eda0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009dc0 tx=0x7fee7c01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945bae46ccb1193f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:10.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee89000640 1 -- 172.21.15.26:0/1440749580 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0a5970 con 0x7fee840cb600 2023-12-19T18:18:10.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee89000640 1 -- 172.21.15.26:0/1440749580 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7c0a5ad0 con 0x7fee840cb600 2023-12-19T18:18:10.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee89000640 1 -- 172.21.15.26:0/1440749580 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c0a5e40 con 0x7fee840cb600 2023-12-19T18:18:10.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee8c266640 1 -- 172.21.15.26:0/1440749580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee840cb600 msgr2=0x7fee8415eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee8c266640 1 --2- 172.21.15.26:0/1440749580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee840cb600 0x7fee8415eda0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009dc0 tx=0x7fee7c01c8b0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee8c266640 1 -- 172.21.15.26:0/1440749580 shutdown_connections 2023-12-19T18:18:10.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee8c266640 1 --2- 172.21.15.26:0/1440749580 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee840cb600 0x7fee8415eda0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee8c266640 1 -- 172.21.15.26:0/1440749580 >> 172.21.15.26:0/1440749580 conn(0x7fee84066160 msgr2=0x7fee840b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:10.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee8c266640 1 -- 172.21.15.26:0/1440749580 shutdown_connections 2023-12-19T18:18:10.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee8c266640 1 -- 172.21.15.26:0/1440749580 wait complete. 2023-12-19T18:18:10.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee8c266640 1 Processor -- start 2023-12-19T18:18:10.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee8c266640 1 -- start start 2023-12-19T18:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.304+0000 7fee8c266640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee840cb150 0x7fee84151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8c266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee840cb600 0x7fee8414b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8c266640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee841585f0 0x7fee8406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8c266640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee84152ea0 con 0x7fee841585f0 2023-12-19T18:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8c266640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee84151e00 con 0x7fee840cb150 2023-12-19T18:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8c266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee841520e0 con 0x7fee840cb600 2023-12-19T18:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8a803640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee841585f0 0x7fee8406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8a803640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee841585f0 0x7fee8406c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54356/0 (socket says 172.21.15.26:54356) 2023-12-19T18:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8a803640 1 -- 172.21.15.26:0/3957068226 learned_addr learned my addr 172.21.15.26:0/3957068226 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee89801640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee840cb600 0x7fee8414b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8a002640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee840cb150 0x7fee84151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8a803640 1 -- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee840cb600 msgr2=0x7fee8414b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8a803640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee840cb600 0x7fee8414b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8a803640 1 -- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee840cb150 msgr2=0x7fee84151500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8a803640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee840cb150 0x7fee84151500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8a803640 1 -- 172.21.15.26:0/3957068226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c009a70 con 0x7fee841585f0 2023-12-19T18:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8a002640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee840cb150 0x7fee84151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:10.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8a803640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee841585f0 0x7fee8406c360 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fee7c01a4e0 tx=0x7fee7c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee7a7fc640 1 -- 172.21.15.26:0/3957068226 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c003ba0 con 0x7fee841585f0 2023-12-19T18:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee7a7fc640 1 -- 172.21.15.26:0/3957068226 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7c003d00 con 0x7fee841585f0 2023-12-19T18:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee7a7fc640 1 -- 172.21.15.26:0/3957068226 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c004260 con 0x7fee841585f0 2023-12-19T18:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8c266640 1 -- 172.21.15.26:0/3957068226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee841523c0 con 0x7fee841585f0 2023-12-19T18:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8c266640 1 -- 172.21.15.26:0/3957068226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee84152980 con 0x7fee841585f0 2023-12-19T18:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee8c266640 1 -- 172.21.15.26:0/3957068226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee84074270 con 0x7fee841585f0 2023-12-19T18:18:10.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.308+0000 7fee7a7fc640 1 -- 172.21.15.26:0/3957068226 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fee7c0b4070 con 0x7fee841585f0 2023-12-19T18:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.312+0000 7fee7a7fc640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee68065ca0 0x7fee68068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.312+0000 7fee7a7fc640 1 -- 172.21.15.26:0/3957068226 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6563+0+0 (secure 0 0 0) 0x7fee7c11e090 con 0x7fee841585f0 2023-12-19T18:18:10.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.312+0000 7fee8a002640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee68065ca0 0x7fee68068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.312+0000 7fee8a002640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee68065ca0 0x7fee68068160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee80000f70 tx=0x7fee80009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:10.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.316+0000 7fee7a7fc640 1 -- 172.21.15.26:0/3957068226 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fee7c0eb4c0 con 0x7fee841585f0 2023-12-19T18:18:10.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.484+0000 7fee8c266640 1 -- 172.21.15.26:0/3957068226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fee840c2050 con 0x7fee841585f0 2023-12-19T18:18:10.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.488+0000 7fee7a7fc640 1 -- 172.21.15.26:0/3957068226 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v79) v1 ==== 120+0+0 (secure 0 0 0) 0x7fee7c0eeae0 con 0x7fee841585f0 2023-12-19T18:18:10.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.488+0000 7fee8c266640 1 -- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee68065ca0 msgr2=0x7fee68068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.488+0000 7fee8c266640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee68065ca0 0x7fee68068160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fee80000f70 tx=0x7fee80009040 comp rx=0 tx=0).stop 2023-12-19T18:18:10.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.488+0000 7fee8c266640 1 -- 172.21.15.26:0/3957068226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee841585f0 msgr2=0x7fee8406c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.488+0000 7fee8c266640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee841585f0 0x7fee8406c360 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fee7c01a4e0 tx=0x7fee7c019650 comp rx=0 tx=0).stop 2023-12-19T18:18:10.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.488+0000 7fee8c266640 1 -- 172.21.15.26:0/3957068226 shutdown_connections 2023-12-19T18:18:10.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.488+0000 7fee8c266640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fee68065ca0 0x7fee68068160 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.488+0000 7fee8c266640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee841585f0 0x7fee8406c360 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.488+0000 7fee8c266640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fee840cb600 0x7fee8414b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.488+0000 7fee8c266640 1 --2- 172.21.15.26:0/3957068226 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee840cb150 0x7fee84151500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.488+0000 7fee8c266640 1 -- 172.21.15.26:0/3957068226 >> 172.21.15.26:0/3957068226 conn(0x7fee84066160 msgr2=0x7fee841534b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:10.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.488+0000 7fee8c266640 1 -- 172.21.15.26:0/3957068226 shutdown_connections 2023-12-19T18:18:10.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.488+0000 7fee8c266640 1 -- 172.21.15.26:0/3957068226 wait complete. 2023-12-19T18:18:10.510 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:18:10.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f3ae5640 1 Processor -- start 2023-12-19T18:18:10.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f3ae5640 1 -- start start 2023-12-19T18:18:10.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f3ae5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1ec0adc60 0x7ff1ec0ca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f3ae5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff1ec06d1e0 0x7ff1ec06f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f3ae5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec071590 con 0x7ff1ec071100 2023-12-19T18:18:10.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f3ae5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec1615b0 con 0x7ff1ec0adc60 2023-12-19T18:18:10.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f3ae5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec0c8d90 con 0x7ff1ec06d1e0 2023-12-19T18:18:10.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f1881640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1ec0adc60 0x7ff1ec0ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f1881640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff1ec0adc60 0x7ff1ec0ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54372/0 (socket says 172.21.15.26:54372) 2023-12-19T18:18:10.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f1881640 1 -- 172.21.15.26:0/2016541589 learned_addr learned my addr 172.21.15.26:0/2016541589 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:10.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f1881640 1 -- 172.21.15.26:0/2016541589 >> v2:172.21.15.160:3300/0 conn(0x7ff1ec06d1e0 msgr2=0x7ff1ec06f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:10.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f1881640 1 --2- 172.21.15.26:0/2016541589 >> v2:172.21.15.160:3300/0 conn(0x7ff1ec06d1e0 0x7ff1ec06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f1881640 1 -- 172.21.15.26:0/2016541589 >> v1:172.21.15.160:6789/0 conn(0x7ff1ec071100 legacy=0x7ff1ec06cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:10.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.656+0000 7ff1f1881640 1 -- 172.21.15.26:0/2016541589 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec0c9070 con 0x7ff1ec0adc60 2023-12-19T18:18:10.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.660+0000 7ff1f1881640 1 --2- 172.21.15.26:0/2016541589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ec0adc60 0x7ff1ec0ca640 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0ca3f0 tx=0x7ff1e801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd11d5df479c003c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:10.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.660+0000 7ff1f087f640 1 -- 172.21.15.26:0/2016541589 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80aa5f0 con 0x7ff1ec0adc60 2023-12-19T18:18:10.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.660+0000 7ff1f087f640 1 -- 172.21.15.26:0/2016541589 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e80aa750 con 0x7ff1ec0adc60 2023-12-19T18:18:10.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.660+0000 7ff1f087f640 1 -- 172.21.15.26:0/2016541589 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e80aaae0 con 0x7ff1ec0adc60 2023-12-19T18:18:10.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.660+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/2016541589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ec0adc60 msgr2=0x7ff1ec0ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.660+0000 7ff1f3ae5640 1 --2- 172.21.15.26:0/2016541589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ec0adc60 0x7ff1ec0ca640 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0ca3f0 tx=0x7ff1e801c490 comp rx=0 tx=0).stop 2023-12-19T18:18:10.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.660+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/2016541589 shutdown_connections 2023-12-19T18:18:10.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.660+0000 7ff1f3ae5640 1 --2- 172.21.15.26:0/2016541589 >> v2:172.21.15.160:3300/0 conn(0x7ff1ec06d1e0 0x7ff1ec06f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.660+0000 7ff1f3ae5640 1 --2- 172.21.15.26:0/2016541589 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ec0adc60 0x7ff1ec0ca640 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.660+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/2016541589 >> 172.21.15.26:0/2016541589 conn(0x7ff1ec066160 msgr2=0x7ff1ec0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:10.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.660+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/2016541589 shutdown_connections 2023-12-19T18:18:10.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.660+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/2016541589 wait complete. 2023-12-19T18:18:10.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f3ae5640 1 Processor -- start 2023-12-19T18:18:10.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f3ae5640 1 -- start start 2023-12-19T18:18:10.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f3ae5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1ec06d1e0 0x7ff1ec160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f3ae5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ec071100 0x7ff1ec1fcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f3ae5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1ec0adc60 0x7ff1ec152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f3ae5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec1fda90 con 0x7ff1ec071100 2023-12-19T18:18:10.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f3ae5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec15eb70 con 0x7ff1ec06d1e0 2023-12-19T18:18:10.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f3ae5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec1fcfe0 con 0x7ff1ec0adc60 2023-12-19T18:18:10.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f1080640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ec071100 0x7ff1ec1fcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f1080640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ec071100 0x7ff1ec1fcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54378/0 (socket says 172.21.15.26:54378) 2023-12-19T18:18:10.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f1080640 1 -- 172.21.15.26:0/3312273076 learned_addr learned my addr 172.21.15.26:0/3312273076 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:10.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f1881640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1ec06d1e0 0x7ff1ec160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f1080640 1 -- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1ec0adc60 msgr2=0x7ff1ec152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f1080640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1ec0adc60 0x7ff1ec152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f1080640 1 -- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1ec06d1e0 msgr2=0x7ff1ec160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f1080640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1ec06d1e0 0x7ff1ec160d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f1080640 1 -- 172.21.15.26:0/3312273076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e80094d0 con 0x7ff1ec071100 2023-12-19T18:18:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f1881640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1ec06d1e0 0x7ff1ec160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f1080640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ec071100 0x7ff1ec1fcaa0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c490 tx=0x7ff1dc001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1e27fc640 1 -- 172.21.15.26:0/3312273076 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc003600 con 0x7ff1ec071100 2023-12-19T18:18:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1e27fc640 1 -- 172.21.15.26:0/3312273076 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc003780 con 0x7ff1ec071100 2023-12-19T18:18:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/3312273076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec1fd1f0 con 0x7ff1ec071100 2023-12-19T18:18:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1e27fc640 1 -- 172.21.15.26:0/3312273076 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc003b70 con 0x7ff1ec071100 2023-12-19T18:18:10.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/3312273076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec1fd7b0 con 0x7ff1ec071100 2023-12-19T18:18:10.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.664+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/3312273076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec15e8d0 con 0x7ff1ec071100 2023-12-19T18:18:10.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.668+0000 7ff1e27fc640 1 -- 172.21.15.26:0/3312273076 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ff1dc003cd0 con 0x7ff1ec071100 2023-12-19T18:18:10.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.668+0000 7ff1e27fc640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff1d0065ca0 0x7ff1d0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.668+0000 7ff1e27fc640 1 -- 172.21.15.26:0/3312273076 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6563+0+0 (secure 0 0 0) 0x7ff1dc082b80 con 0x7ff1ec071100 2023-12-19T18:18:10.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.668+0000 7ff1f1881640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff1d0065ca0 0x7ff1d0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.668+0000 7ff1f1881640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff1d0065ca0 0x7ff1d0068160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80ae000 tx=0x7ff1e8008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:10.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.672+0000 7ff1e27fc640 1 -- 172.21.15.26:0/3312273076 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff1ec15e8d0 con 0x7ff1ec071100 2023-12-19T18:18:10.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.828+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/3312273076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff1ec15ecb0 con 0x7ff1ec071100 2023-12-19T18:18:10.835 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.836+0000 7ff1e27fc640 1 -- 172.21.15.26:0/3312273076 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v80) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff1ec15ecb0 con 0x7ff1ec071100 2023-12-19T18:18:10.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.836+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff1d0065ca0 msgr2=0x7ff1d0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.836+0000 7ff1f3ae5640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff1d0065ca0 0x7ff1d0068160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff1e80ae000 tx=0x7ff1e8008040 comp rx=0 tx=0).stop 2023-12-19T18:18:10.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.836+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/3312273076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ec071100 msgr2=0x7ff1ec1fcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.836+0000 7ff1f3ae5640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ec071100 0x7ff1ec1fcaa0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c490 tx=0x7ff1dc001e80 comp rx=0 tx=0).stop 2023-12-19T18:18:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.836+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/3312273076 shutdown_connections 2023-12-19T18:18:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.836+0000 7ff1f3ae5640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff1d0065ca0 0x7ff1d0068160 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.836+0000 7ff1f3ae5640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff1ec0adc60 0x7ff1ec152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.836+0000 7ff1f3ae5640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff1ec071100 0x7ff1ec1fcaa0 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.836+0000 7ff1f3ae5640 1 --2- 172.21.15.26:0/3312273076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1ec06d1e0 0x7ff1ec160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.836+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/3312273076 >> 172.21.15.26:0/3312273076 conn(0x7ff1ec066160 msgr2=0x7ff1ec15d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.836+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/3312273076 shutdown_connections 2023-12-19T18:18:10.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:10.836+0000 7ff1f3ae5640 1 -- 172.21.15.26:0/3312273076 wait complete. 2023-12-19T18:18:10.857 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-19T18:18:11.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.000+0000 7ffa4af6d640 1 Processor -- start 2023-12-19T18:18:11.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.000+0000 7ffa4af6d640 1 -- start start 2023-12-19T18:18:11.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa4af6d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffa441585f0 0x7ffa440cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa4af6d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffa440cb150 0x7ffa440cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa4af6d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4415b1b0 con 0x7ffa440cc310 2023-12-19T18:18:11.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa4af6d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa440cbb40 con 0x7ffa440cb150 2023-12-19T18:18:11.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa4af6d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa440b47a0 con 0x7ffa441585f0 2023-12-19T18:18:11.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa43fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffa440cb150 0x7ffa440cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa43fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffa440cb150 0x7ffa440cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54384/0 (socket says 172.21.15.26:54384) 2023-12-19T18:18:11.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa43fff640 1 -- 172.21.15.26:0/3230336994 learned_addr learned my addr 172.21.15.26:0/3230336994 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:11.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa4950a640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ffa440cc310 0x7ffa440cc760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35306/0 (socket says 172.21.15.26:35306) 2023-12-19T18:18:11.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa48d09640 1 --2- 172.21.15.26:0/3230336994 >> v2:172.21.15.160:3300/0 conn(0x7ffa441585f0 0x7ffa440cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa43fff640 1 -- 172.21.15.26:0/3230336994 >> v2:172.21.15.160:3300/0 conn(0x7ffa441585f0 msgr2=0x7ffa440cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:11.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa43fff640 1 --2- 172.21.15.26:0/3230336994 >> v2:172.21.15.160:3300/0 conn(0x7ffa441585f0 0x7ffa440cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa43fff640 1 -- 172.21.15.26:0/3230336994 >> v1:172.21.15.26:6789/0 conn(0x7ffa440cc310 legacy=0x7ffa440cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa437fe640 1 -- 172.21.15.26:0/3230336994 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3784379161 0 0) 0x7ffa4415b1b0 con 0x7ffa440cc310 2023-12-19T18:18:11.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.004+0000 7ffa43fff640 1 -- 172.21.15.26:0/3230336994 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4415b490 con 0x7ffa440cb150 2023-12-19T18:18:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa43fff640 1 --2- 172.21.15.26:0/3230336994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa440cb150 0x7ffa440cb600 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7ffa440cc2c0 tx=0x7ffa3001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491dd98968f38fd8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa437fe640 1 -- 172.21.15.26:0/3230336994 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa300ab640 con 0x7ffa440cb150 2023-12-19T18:18:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa437fe640 1 -- 172.21.15.26:0/3230336994 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa300a7070 con 0x7ffa440cb150 2023-12-19T18:18:11.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa437fe640 1 -- 172.21.15.26:0/3230336994 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa300ab9c0 con 0x7ffa440cb150 2023-12-19T18:18:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3230336994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa440cb150 msgr2=0x7ffa440cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa4af6d640 1 --2- 172.21.15.26:0/3230336994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa440cb150 0x7ffa440cb600 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7ffa440cc2c0 tx=0x7ffa3001c440 comp rx=0 tx=0).stop 2023-12-19T18:18:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3230336994 shutdown_connections 2023-12-19T18:18:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa4af6d640 1 --2- 172.21.15.26:0/3230336994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa440cb150 0x7ffa440cb600 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa4af6d640 1 --2- 172.21.15.26:0/3230336994 >> v2:172.21.15.160:3300/0 conn(0x7ffa441585f0 0x7ffa440cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3230336994 >> 172.21.15.26:0/3230336994 conn(0x7ffa44066160 msgr2=0x7ffa440b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3230336994 shutdown_connections 2023-12-19T18:18:11.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3230336994 wait complete. 2023-12-19T18:18:11.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa4af6d640 1 Processor -- start 2023-12-19T18:18:11.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa4af6d640 1 -- start start 2023-12-19T18:18:11.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa4af6d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa440cb150 0x7ffa4415dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.008+0000 7ffa4af6d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa440cc310 0x7ffa4406c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa4af6d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa441585f0 0x7ffa4406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa4af6d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4415e530 con 0x7ffa440cc310 2023-12-19T18:18:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa4af6d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4415e7d0 con 0x7ffa441585f0 2023-12-19T18:18:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa4af6d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4415eab0 con 0x7ffa440cb150 2023-12-19T18:18:11.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa43fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa440cc310 0x7ffa4406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa43fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa440cc310 0x7ffa4406c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54396/0 (socket says 172.21.15.26:54396) 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa43fff640 1 -- 172.21.15.26:0/3995753821 learned_addr learned my addr 172.21.15.26:0/3995753821 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa48d09640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa440cb150 0x7ffa4415dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa4950a640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa441585f0 0x7ffa4406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa43fff640 1 -- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa440cb150 msgr2=0x7ffa4415dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa43fff640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa440cb150 0x7ffa4415dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa43fff640 1 -- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa441585f0 msgr2=0x7ffa4406c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa43fff640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa441585f0 0x7ffa4406c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa43fff640 1 -- 172.21.15.26:0/3995753821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa300094d0 con 0x7ffa440cc310 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa43fff640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa440cc310 0x7ffa4406c120 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7ffa300ad040 tx=0x7ffa3001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa417fa640 1 -- 172.21.15.26:0/3995753821 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa300abde0 con 0x7ffa440cc310 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa417fa640 1 -- 172.21.15.26:0/3995753821 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa300be3f0 con 0x7ffa440cc310 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa417fa640 1 -- 172.21.15.26:0/3995753821 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa300be7a0 con 0x7ffa440cc310 2023-12-19T18:18:11.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3995753821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4415c3e0 con 0x7ffa440cc310 2023-12-19T18:18:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3995753821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4415f070 con 0x7ffa440cc310 2023-12-19T18:18:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3995753821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa440c57f0 con 0x7ffa440cc310 2023-12-19T18:18:11.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.012+0000 7ffa417fa640 1 -- 172.21.15.26:0/3995753821 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ffa300c7440 con 0x7ffa440cc310 2023-12-19T18:18:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.016+0000 7ffa417fa640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffa28065cd0 0x7ffa28068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.016+0000 7ffa417fa640 1 -- 172.21.15.26:0/3995753821 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6563+0+0 (secure 0 0 0) 0x7ffa3011e240 con 0x7ffa440cc310 2023-12-19T18:18:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.016+0000 7ffa48d09640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffa28065cd0 0x7ffa28068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.016+0000 7ffa48d09640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffa28065cd0 0x7ffa28068190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009b60 tx=0x7ffa38009190 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:11.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.020+0000 7ffa417fa640 1 -- 172.21.15.26:0/3995753821 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffa300eb690 con 0x7ffa440cc310 2023-12-19T18:18:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.176+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3995753821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7ffa44074270 con 0x7ffa440cc310 2023-12-19T18:18:11.182 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.180+0000 7ffa417fa640 1 -- 172.21.15.26:0/3995753821 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v81) v1 ==== 115+0+0 (secure 0 0 0) 0x7ffa300eecb0 con 0x7ffa440cc310 2023-12-19T18:18:11.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.184+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffa28065cd0 msgr2=0x7ffa28068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.184+0000 7ffa4af6d640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffa28065cd0 0x7ffa28068190 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009b60 tx=0x7ffa38009190 comp rx=0 tx=0).stop 2023-12-19T18:18:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.184+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3995753821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa440cc310 msgr2=0x7ffa4406c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.184+0000 7ffa4af6d640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa440cc310 0x7ffa4406c120 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7ffa300ad040 tx=0x7ffa3001ac80 comp rx=0 tx=0).stop 2023-12-19T18:18:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.184+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3995753821 shutdown_connections 2023-12-19T18:18:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.184+0000 7ffa4af6d640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffa28065cd0 0x7ffa28068190 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.184+0000 7ffa4af6d640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa441585f0 0x7ffa4406c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.184+0000 7ffa4af6d640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa440cc310 0x7ffa4406c120 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.184+0000 7ffa4af6d640 1 --2- 172.21.15.26:0/3995753821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffa440cb150 0x7ffa4415dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.184+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3995753821 >> 172.21.15.26:0/3995753821 conn(0x7ffa44066160 msgr2=0x7ffa44158b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.184+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3995753821 shutdown_connections 2023-12-19T18:18:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.184+0000 7ffa4af6d640 1 -- 172.21.15.26:0/3995753821 wait complete. 2023-12-19T18:18:11.204 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-12-19T18:18:11.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0458063640 1 Processor -- start 2023-12-19T18:18:11.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0458063640 1 -- start start 2023-12-19T18:18:11.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0458063640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f045006b3d0 0x7f04500ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0458063640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f04500cec40 0x7f045015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0458063640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f045006bc30 con 0x7f0450154220 2023-12-19T18:18:11.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0458063640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04500cf0d0 con 0x7f04500cec40 2023-12-19T18:18:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0458063640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04500cc6e0 con 0x7f045006b3d0 2023-12-19T18:18:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0456600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f04500cec40 0x7f045015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0456600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f04500cec40 0x7f045015b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54412/0 (socket says 172.21.15.26:54412) 2023-12-19T18:18:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0456600640 1 -- 172.21.15.26:0/4022112350 learned_addr learned my addr 172.21.15.26:0/4022112350 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f04555fe640 1 --2- 172.21.15.26:0/4022112350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f045006b3d0 0x7f04500ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0456600640 1 -- 172.21.15.26:0/4022112350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f045006b3d0 msgr2=0x7f04500ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0456600640 1 --2- 172.21.15.26:0/4022112350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f045006b3d0 0x7f04500ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0456600640 1 -- 172.21.15.26:0/4022112350 >> v1:172.21.15.160:6789/0 conn(0x7f0450154220 legacy=0x7f045015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:11.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.348+0000 7f0456600640 1 -- 172.21.15.26:0/4022112350 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045006b950 con 0x7f04500cec40 2023-12-19T18:18:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.352+0000 7f0456600640 1 --2- 172.21.15.26:0/4022112350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04500cec40 0x7f045015b1c0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f0448009e50 tx=0x7f044801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4d125e7cb019c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.352+0000 7f0454dfd640 1 -- 172.21.15.26:0/4022112350 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04480a58a0 con 0x7f04500cec40 2023-12-19T18:18:11.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.352+0000 7f0454dfd640 1 -- 172.21.15.26:0/4022112350 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04480a5a00 con 0x7f04500cec40 2023-12-19T18:18:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.352+0000 7f0454dfd640 1 -- 172.21.15.26:0/4022112350 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04480a5d90 con 0x7f04500cec40 2023-12-19T18:18:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.352+0000 7f0458063640 1 -- 172.21.15.26:0/4022112350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04500cec40 msgr2=0x7f045015b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.352+0000 7f0458063640 1 --2- 172.21.15.26:0/4022112350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04500cec40 0x7f045015b1c0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7f0448009e50 tx=0x7f044801c9e0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.352+0000 7f0458063640 1 -- 172.21.15.26:0/4022112350 shutdown_connections 2023-12-19T18:18:11.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.352+0000 7f0458063640 1 --2- 172.21.15.26:0/4022112350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04500cec40 0x7f045015b1c0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.352+0000 7f0458063640 1 --2- 172.21.15.26:0/4022112350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f045006b3d0 0x7f04500ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.352+0000 7f0458063640 1 -- 172.21.15.26:0/4022112350 >> 172.21.15.26:0/4022112350 conn(0x7f04500658e0 msgr2=0x7f0450065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:11.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.352+0000 7f0458063640 1 -- 172.21.15.26:0/4022112350 shutdown_connections 2023-12-19T18:18:11.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.352+0000 7f0458063640 1 -- 172.21.15.26:0/4022112350 wait complete. 2023-12-19T18:18:11.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f0458063640 1 Processor -- start 2023-12-19T18:18:11.355 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f0458063640 1 -- start start 2023-12-19T18:18:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f0458063640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f045006b3d0 0x7f045014b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f0458063640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04500cec40 0x7f045014c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f0458063640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0450154220 0x7f04500ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f0458063640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0450150560 con 0x7f04500cec40 2023-12-19T18:18:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f0458063640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045006f6f0 con 0x7f0450154220 2023-12-19T18:18:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f0458063640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045006f9d0 con 0x7f045006b3d0 2023-12-19T18:18:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f04555fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04500cec40 0x7f045014c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f04555fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04500cec40 0x7f045014c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54414/0 (socket says 172.21.15.26:54414) 2023-12-19T18:18:11.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f04555fe640 1 -- 172.21.15.26:0/1749584377 learned_addr learned my addr 172.21.15.26:0/1749584377 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f0456600640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0450154220 0x7f04500ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f0455dff640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f045006b3d0 0x7f045014b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f04555fe640 1 -- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f045006b3d0 msgr2=0x7f045014b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f04555fe640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f045006b3d0 0x7f045014b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f04555fe640 1 -- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0450154220 msgr2=0x7f04500ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f04555fe640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0450154220 0x7f04500ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f04555fe640 1 -- 172.21.15.26:0/1749584377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0448009b30 con 0x7f04500cec40 2023-12-19T18:18:11.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f04555fe640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04500cec40 0x7f045014c2b0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f044000d5f0 tx=0x7f044000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f04467fc640 1 -- 172.21.15.26:0/1749584377 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04400125f0 con 0x7f04500cec40 2023-12-19T18:18:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f04467fc640 1 -- 172.21.15.26:0/1749584377 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0440012750 con 0x7f04500cec40 2023-12-19T18:18:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f04467fc640 1 -- 172.21.15.26:0/1749584377 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0440012ae0 con 0x7f04500cec40 2023-12-19T18:18:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f0458063640 1 -- 172.21.15.26:0/1749584377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045006fcb0 con 0x7f04500cec40 2023-12-19T18:18:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f0458063640 1 -- 172.21.15.26:0/1749584377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045014b770 con 0x7f04500cec40 2023-12-19T18:18:11.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.356+0000 7f0458063640 1 -- 172.21.15.26:0/1749584377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0450074270 con 0x7f04500cec40 2023-12-19T18:18:11.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.360+0000 7f04467fc640 1 -- 172.21.15.26:0/1749584377 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f0440012c40 con 0x7f04500cec40 2023-12-19T18:18:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.360+0000 7f04467fc640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0434065ca0 0x7f0434068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.360+0000 7f04467fc640 1 -- 172.21.15.26:0/1749584377 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6563+0+0 (secure 0 0 0) 0x7f0440083de0 con 0x7f04500cec40 2023-12-19T18:18:11.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.360+0000 7f0455dff640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0434065ca0 0x7f0434068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.362 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.360+0000 7f0455dff640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0434065ca0 0x7f0434068160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0450151440 tx=0x7f044c009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:11.365 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.364+0000 7f04467fc640 1 -- 172.21.15.26:0/1749584377 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0440051040 con 0x7f04500cec40 2023-12-19T18:18:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:11.576+0000 7f0458063640 1 -- 172.21.15.26:0/1749584377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7f04500cdf60 con 0x7f04500cec40 2023-12-19T18:18:12.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.116+0000 7f04467fc640 1 -- 172.21.15.26:0/1749584377 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3132) v1 ==== 187+0+0 (secure 0 0 0) 0x7f0440054660 con 0x7f04500cec40 2023-12-19T18:18:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.120+0000 7f0458063640 1 -- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0434065ca0 msgr2=0x7f0434068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.120+0000 7f0458063640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0434065ca0 0x7f0434068160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0450151440 tx=0x7f044c009040 comp rx=0 tx=0).stop 2023-12-19T18:18:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.120+0000 7f0458063640 1 -- 172.21.15.26:0/1749584377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04500cec40 msgr2=0x7f045014c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.120+0000 7f0458063640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04500cec40 0x7f045014c2b0 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f044000d5f0 tx=0x7f044000dab0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.120+0000 7f0458063640 1 -- 172.21.15.26:0/1749584377 shutdown_connections 2023-12-19T18:18:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.120+0000 7f0458063640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0434065ca0 0x7f0434068160 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.120 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.120+0000 7f0458063640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0450154220 0x7f04500ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.120+0000 7f0458063640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04500cec40 0x7f045014c2b0 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.120+0000 7f0458063640 1 --2- 172.21.15.26:0/1749584377 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f045006b3d0 0x7f045014b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.120+0000 7f0458063640 1 -- 172.21.15.26:0/1749584377 >> 172.21.15.26:0/1749584377 conn(0x7f04500658e0 msgr2=0x7f045015cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:12.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.120+0000 7f0458063640 1 -- 172.21.15.26:0/1749584377 shutdown_connections 2023-12-19T18:18:12.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.120+0000 7f0458063640 1 -- 172.21.15.26:0/1749584377 wait complete. 2023-12-19T18:18:12.143 INFO:tasks.cephfs_test_runner:Test command fails without --fs argument in presence of multiple fs ... ok 2023-12-19T18:18:12.144 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-12-19T18:18:12.144 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-12-19T18:18:12.144 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-12-19T18:18:12.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.292+0000 7fd73318f640 1 Processor -- start 2023-12-19T18:18:12.292 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.292+0000 7fd73318f640 1 -- start start 2023-12-19T18:18:12.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.292+0000 7fd73318f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd72c15a7c0 0x7fd72c15ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.292+0000 7fd73318f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72c15e2f0 con 0x7fd72c0cf700 2023-12-19T18:18:12.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.292+0000 7fd73318f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72c0c5b40 con 0x7fd72c06f9d0 2023-12-19T18:18:12.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.292+0000 7fd73318f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd72c0c5e20 con 0x7fd72c15a7c0 2023-12-19T18:18:12.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.292+0000 7fd730f2b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd72c15a7c0 0x7fd72c15ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.292+0000 7fd73172c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd72c0cf700 0x7fd72c0cd240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35310/0 (socket says 172.21.15.26:35310) 2023-12-19T18:18:12.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.292+0000 7fd73172c640 1 -- 172.21.15.26:0/684718821 learned_addr learned my addr 172.21.15.26:0/684718821 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:12.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.292+0000 7fd730f2b640 1 -- 172.21.15.26:0/684718821 >> v1:172.21.15.160:6789/0 conn(0x7fd72c06f9d0 legacy=0x7fd72c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:12.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.292+0000 7fd730f2b640 1 -- 172.21.15.26:0/684718821 >> v1:172.21.15.26:6789/0 conn(0x7fd72c0cf700 legacy=0x7fd72c0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.292+0000 7fd730f2b640 1 -- 172.21.15.26:0/684718821 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c0ce770 con 0x7fd72c15a7c0 2023-12-19T18:18:12.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.292+0000 7fd72b7fe640 1 -- 172.21.15.26:0/684718821 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3863823412 0 0) 0x7fd72c15e2f0 con 0x7fd72c0cf700 2023-12-19T18:18:12.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.296+0000 7fd730f2b640 1 --2- 172.21.15.26:0/684718821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd72c15a7c0 0x7fd72c15ddb0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fd7200097a0 tx=0x7fd72001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d21961c5bb9823 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:12.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.296+0000 7fd72b7fe640 1 -- 172.21.15.26:0/684718821 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7200ab5f0 con 0x7fd72c15a7c0 2023-12-19T18:18:12.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.296+0000 7fd72b7fe640 1 -- 172.21.15.26:0/684718821 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7200ab750 con 0x7fd72c15a7c0 2023-12-19T18:18:12.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.296+0000 7fd72b7fe640 1 -- 172.21.15.26:0/684718821 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7200abae0 con 0x7fd72c15a7c0 2023-12-19T18:18:12.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.296+0000 7fd73318f640 1 -- 172.21.15.26:0/684718821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd72c15a7c0 msgr2=0x7fd72c15ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.296+0000 7fd73318f640 1 --2- 172.21.15.26:0/684718821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd72c15a7c0 0x7fd72c15ddb0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fd7200097a0 tx=0x7fd72001c3f0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.296+0000 7fd73318f640 1 -- 172.21.15.26:0/684718821 shutdown_connections 2023-12-19T18:18:12.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.296+0000 7fd73318f640 1 --2- 172.21.15.26:0/684718821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd72c15a7c0 0x7fd72c15ddb0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.296+0000 7fd73318f640 1 -- 172.21.15.26:0/684718821 >> 172.21.15.26:0/684718821 conn(0x7fd72c06d5d0 msgr2=0x7fd72c069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:12.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.296+0000 7fd73318f640 1 -- 172.21.15.26:0/684718821 shutdown_connections 2023-12-19T18:18:12.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.296+0000 7fd73318f640 1 -- 172.21.15.26:0/684718821 wait complete. 2023-12-19T18:18:12.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd73318f640 1 Processor -- start 2023-12-19T18:18:12.299 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd73318f640 1 -- start start 2023-12-19T18:18:12.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd73318f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd72c06f9d0 0x7fd72c14c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd73318f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd72c0cf700 0x7fd72c1f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd73318f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72c15a7c0 0x7fd72c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd73318f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c0b20e0 con 0x7fd72c06f9d0 2023-12-19T18:18:12.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd73318f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c1f6650 con 0x7fd72c15a7c0 2023-12-19T18:18:12.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd73318f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c1f6930 con 0x7fd72c0cf700 2023-12-19T18:18:12.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd730f2b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd72c06f9d0 0x7fd72c14c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd730f2b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd72c06f9d0 0x7fd72c14c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54436/0 (socket says 172.21.15.26:54436) 2023-12-19T18:18:12.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd730f2b640 1 -- 172.21.15.26:0/291027763 learned_addr learned my addr 172.21.15.26:0/291027763 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:12.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd72bfff640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd72c0cf700 0x7fd72c1f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd73172c640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72c15a7c0 0x7fd72c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd730f2b640 1 -- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd72c0cf700 msgr2=0x7fd72c1f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd730f2b640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd72c0cf700 0x7fd72c1f5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd730f2b640 1 -- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72c15a7c0 msgr2=0x7fd72c15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd730f2b640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72c15a7c0 0x7fd72c15e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd730f2b640 1 -- 172.21.15.26:0/291027763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7200093b0 con 0x7fd72c06f9d0 2023-12-19T18:18:12.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd730f2b640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd72c06f9d0 0x7fd72c14c120 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fd720019000 tx=0x7fd72001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:12.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd7297fa640 1 -- 172.21.15.26:0/291027763 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7200bf610 con 0x7fd72c06f9d0 2023-12-19T18:18:12.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd7297fa640 1 -- 172.21.15.26:0/291027763 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7200bf770 con 0x7fd72c06f9d0 2023-12-19T18:18:12.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd73318f640 1 -- 172.21.15.26:0/291027763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72c1f6c10 con 0x7fd72c06f9d0 2023-12-19T18:18:12.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd7297fa640 1 -- 172.21.15.26:0/291027763 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7200bfb20 con 0x7fd72c06f9d0 2023-12-19T18:18:12.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.300+0000 7fd73318f640 1 -- 172.21.15.26:0/291027763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72c14ee60 con 0x7fd72c06f9d0 2023-12-19T18:18:12.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.304+0000 7fd73318f640 1 -- 172.21.15.26:0/291027763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72c074270 con 0x7fd72c06f9d0 2023-12-19T18:18:12.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.304+0000 7fd7297fa640 1 -- 172.21.15.26:0/291027763 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fd7200b5400 con 0x7fd72c06f9d0 2023-12-19T18:18:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.304+0000 7fd7297fa640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd710065ca0 0x7fd710068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.304+0000 7fd7297fa640 1 -- 172.21.15.26:0/291027763 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6563+0+0 (secure 0 0 0) 0x7fd7200a8050 con 0x7fd72c06f9d0 2023-12-19T18:18:12.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.304+0000 7fd72bfff640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd710065ca0 0x7fd710068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.304+0000 7fd72bfff640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd710065ca0 0x7fd710068160 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd72c15e5f0 tx=0x7fd718009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:12.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.308+0000 7fd7297fa640 1 -- 172.21.15.26:0/291027763 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd7200eb920 con 0x7fd72c06f9d0 2023-12-19T18:18:12.520 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:12.520+0000 7fd73318f640 1 -- 172.21.15.26:0/291027763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7fd72c0b4f30 con 0x7fd72c06f9d0 2023-12-19T18:18:12.604 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:18:12.609 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:18:13.119 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.120+0000 7fd7297fa640 1 -- 172.21.15.26:0/291027763 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3133) v1 ==== 158+0+0 (secure 0 0 0) 0x7fd7200eef40 con 0x7fd72c06f9d0 2023-12-19T18:18:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.120+0000 7fd73318f640 1 -- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd710065ca0 msgr2=0x7fd710068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.120+0000 7fd73318f640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd710065ca0 0x7fd710068160 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd72c15e5f0 tx=0x7fd718009040 comp rx=0 tx=0).stop 2023-12-19T18:18:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.120+0000 7fd73318f640 1 -- 172.21.15.26:0/291027763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd72c06f9d0 msgr2=0x7fd72c14c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.120+0000 7fd73318f640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd72c06f9d0 0x7fd72c14c120 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fd720019000 tx=0x7fd72001ac30 comp rx=0 tx=0).stop 2023-12-19T18:18:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.120+0000 7fd73318f640 1 -- 172.21.15.26:0/291027763 shutdown_connections 2023-12-19T18:18:13.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.120+0000 7fd73318f640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fd710065ca0 0x7fd710068160 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.120+0000 7fd73318f640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd72c15a7c0 0x7fd72c15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.120+0000 7fd73318f640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fd72c0cf700 0x7fd72c1f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.120+0000 7fd73318f640 1 --2- 172.21.15.26:0/291027763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd72c06f9d0 0x7fd72c14c120 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.120+0000 7fd73318f640 1 -- 172.21.15.26:0/291027763 >> 172.21.15.26:0/291027763 conn(0x7fd72c06d5d0 msgr2=0x7fd72c069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.120+0000 7fd73318f640 1 -- 172.21.15.26:0/291027763 shutdown_connections 2023-12-19T18:18:13.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.120+0000 7fd73318f640 1 -- 172.21.15.26:0/291027763 wait complete. 2023-12-19T18:18:13.145 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:18:13.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a3afc640 1 Processor -- start 2023-12-19T18:18:13.296 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a3afc640 1 -- start start 2023-12-19T18:18:13.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a3afc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa99c06f6c0 0x7fa99c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a3afc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99c06fb60 con 0x7fa99c159c00 2023-12-19T18:18:13.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a3afc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99c06fe40 con 0x7fa99c159820 2023-12-19T18:18:13.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a3afc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa99c15e470 con 0x7fa99c06f6c0 2023-12-19T18:18:13.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a1097640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa99c06f6c0 0x7fa99c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.297 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a2099640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa99c159c00 0x7fa99c070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35322/0 (socket says 172.21.15.26:35322) 2023-12-19T18:18:13.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a2099640 1 -- 172.21.15.26:0/3729894318 learned_addr learned my addr 172.21.15.26:0/3729894318 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:13.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a0896640 1 -- 172.21.15.26:0/3729894318 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586476927 0 0) 0x7fa99c06fb60 con 0x7fa99c159c00 2023-12-19T18:18:13.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a0896640 1 -- 172.21.15.26:0/3729894318 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa988003530 con 0x7fa99c159c00 2023-12-19T18:18:13.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a1097640 1 -- 172.21.15.26:0/3729894318 >> v1:172.21.15.160:6789/0 conn(0x7fa99c159820 legacy=0x7fa99c0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a1097640 1 -- 172.21.15.26:0/3729894318 >> v1:172.21.15.26:6789/0 conn(0x7fa99c159c00 legacy=0x7fa99c070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.298 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.296+0000 7fa9a1097640 1 -- 172.21.15.26:0/3729894318 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c070120 con 0x7fa99c06f6c0 2023-12-19T18:18:13.300 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.300+0000 7fa9a1097640 1 --2- 172.21.15.26:0/3729894318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99c06f6c0 0x7fa99c15df30 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0097c0 tx=0x7fa98c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443729687223be47 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:13.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.300+0000 7fa9a0896640 1 -- 172.21.15.26:0/3729894318 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0ab650 con 0x7fa99c06f6c0 2023-12-19T18:18:13.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.300+0000 7fa9a0896640 1 -- 172.21.15.26:0/3729894318 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c0ab7b0 con 0x7fa99c06f6c0 2023-12-19T18:18:13.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.300+0000 7fa9a0896640 1 -- 172.21.15.26:0/3729894318 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0abb40 con 0x7fa99c06f6c0 2023-12-19T18:18:13.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.300+0000 7fa9a3afc640 1 -- 172.21.15.26:0/3729894318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99c06f6c0 msgr2=0x7fa99c15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.300+0000 7fa9a3afc640 1 --2- 172.21.15.26:0/3729894318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99c06f6c0 0x7fa99c15df30 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0097c0 tx=0x7fa98c01c450 comp rx=0 tx=0).stop 2023-12-19T18:18:13.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.300+0000 7fa9a3afc640 1 -- 172.21.15.26:0/3729894318 shutdown_connections 2023-12-19T18:18:13.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.300+0000 7fa9a3afc640 1 --2- 172.21.15.26:0/3729894318 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99c06f6c0 0x7fa99c15df30 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.300+0000 7fa9a3afc640 1 -- 172.21.15.26:0/3729894318 >> 172.21.15.26:0/3729894318 conn(0x7fa99c0ce730 msgr2=0x7fa99c0ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:13.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.300+0000 7fa9a3afc640 1 -- 172.21.15.26:0/3729894318 shutdown_connections 2023-12-19T18:18:13.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.300+0000 7fa9a3afc640 1 -- 172.21.15.26:0/3729894318 wait complete. 2023-12-19T18:18:13.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a3afc640 1 Processor -- start 2023-12-19T18:18:13.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a3afc640 1 -- start start 2023-12-19T18:18:13.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a3afc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa99c06f6c0 0x7fa99c15a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a3afc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99c159820 0x7fa99c0c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a3afc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa99c159c00 0x7fa99c14e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a3afc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c0c85f0 con 0x7fa99c159820 2023-12-19T18:18:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a3afc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c14eac0 con 0x7fa99c06f6c0 2023-12-19T18:18:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a3afc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c0c7df0 con 0x7fa99c159c00 2023-12-19T18:18:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a1097640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99c159820 0x7fa99c0c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a1097640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99c159820 0x7fa99c0c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54458/0 (socket says 172.21.15.26:54458) 2023-12-19T18:18:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a1097640 1 -- 172.21.15.26:0/1227789187 learned_addr learned my addr 172.21.15.26:0/1227789187 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a1898640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa99c06f6c0 0x7fa99c15a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a2099640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa99c159c00 0x7fa99c14e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a1097640 1 -- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa99c159c00 msgr2=0x7fa99c14e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a1097640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa99c159c00 0x7fa99c14e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a1097640 1 -- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa99c06f6c0 msgr2=0x7fa99c15a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a1097640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa99c06f6c0 0x7fa99c15a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a1097640 1 -- 172.21.15.26:0/1227789187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c009420 con 0x7fa99c159820 2023-12-19T18:18:13.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a1097640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99c159820 0x7fa99c0c80b0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0ad040 tx=0x7fa98c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:13.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9927fc640 1 -- 172.21.15.26:0/1227789187 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0bf610 con 0x7fa99c159820 2023-12-19T18:18:13.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9927fc640 1 -- 172.21.15.26:0/1227789187 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c0bf790 con 0x7fa99c159820 2023-12-19T18:18:13.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9927fc640 1 -- 172.21.15.26:0/1227789187 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0bfb40 con 0x7fa99c159820 2023-12-19T18:18:13.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a3afc640 1 -- 172.21.15.26:0/1227789187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99c06b880 con 0x7fa99c159820 2023-12-19T18:18:13.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a3afc640 1 -- 172.21.15.26:0/1227789187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99c14f450 con 0x7fa99c159820 2023-12-19T18:18:13.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.304+0000 7fa9a3afc640 1 -- 172.21.15.26:0/1227789187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99c074270 con 0x7fa99c159820 2023-12-19T18:18:13.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.308+0000 7fa9927fc640 1 -- 172.21.15.26:0/1227789187 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fa98c0b5400 con 0x7fa99c159820 2023-12-19T18:18:13.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.308+0000 7fa9927fc640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa980065cd0 0x7fa980068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.308+0000 7fa9927fc640 1 -- 172.21.15.26:0/1227789187 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6563+0+0 (secure 0 0 0) 0x7fa98c11e3f0 con 0x7fa99c159820 2023-12-19T18:18:13.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.308+0000 7fa9a1898640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa980065cd0 0x7fa980068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.308+0000 7fa9a1898640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa980065cd0 0x7fa980068190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa99c151b80 tx=0x7fa998009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:13.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.312+0000 7fa9927fc640 1 -- 172.21.15.26:0/1227789187 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa98c0eb780 con 0x7fa99c159820 2023-12-19T18:18:13.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.468+0000 7fa9a3afc640 1 -- 172.21.15.26:0/1227789187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa99c0b46b0 con 0x7fa99c159820 2023-12-19T18:18:13.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.472+0000 7fa9927fc640 1 -- 172.21.15.26:0/1227789187 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v82)=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa98c0eeda0 con 0x7fa99c159820 2023-12-19T18:18:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.472+0000 7fa9a3afc640 1 -- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa980065cd0 msgr2=0x7fa980068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.472+0000 7fa9a3afc640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa980065cd0 0x7fa980068190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa99c151b80 tx=0x7fa998009040 comp rx=0 tx=0).stop 2023-12-19T18:18:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.472+0000 7fa9a3afc640 1 -- 172.21.15.26:0/1227789187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99c159820 msgr2=0x7fa99c0c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.472+0000 7fa9a3afc640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99c159820 0x7fa99c0c80b0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0ad040 tx=0x7fa98c01ac90 comp rx=0 tx=0).stop 2023-12-19T18:18:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.472+0000 7fa9a3afc640 1 -- 172.21.15.26:0/1227789187 shutdown_connections 2023-12-19T18:18:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.472+0000 7fa9a3afc640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa980065cd0 0x7fa980068190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.472+0000 7fa9a3afc640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa99c159c00 0x7fa99c14e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.472+0000 7fa9a3afc640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa99c159820 0x7fa99c0c80b0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.472+0000 7fa9a3afc640 1 --2- 172.21.15.26:0/1227789187 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa99c06f6c0 0x7fa99c15a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.472+0000 7fa9a3afc640 1 -- 172.21.15.26:0/1227789187 >> 172.21.15.26:0/1227789187 conn(0x7fa99c0ce730 msgr2=0x7fa99c0c41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:13.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.472+0000 7fa9a3afc640 1 -- 172.21.15.26:0/1227789187 shutdown_connections 2023-12-19T18:18:13.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.476+0000 7fa9a3afc640 1 -- 172.21.15.26:0/1227789187 wait complete. 2023-12-19T18:18:13.494 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:18:13.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:18:13.578 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:18:13.639 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:18:13.639 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:18:13.639 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:13.648 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:18:13.648 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:18:13.649 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:13.659 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:18:13.659 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:18:13.659 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:13.704 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:18:13.704 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:18:13.704 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:13.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:18:13.713 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:18:13.714 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:13.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.888+0000 7f23744d1640 1 Processor -- start 2023-12-19T18:18:13.889 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.888+0000 7f23744d1640 1 -- start start 2023-12-19T18:18:13.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.888+0000 7f23744d1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f236c154220 0x7f236c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.888+0000 7f23744d1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f236c0cee10 0x7f236c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.888+0000 7f23744d1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c06bc30 con 0x7f236c06b3d0 2023-12-19T18:18:13.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.888+0000 7f23744d1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0cc6e0 con 0x7f236c154220 2023-12-19T18:18:13.892 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.888+0000 7f23744d1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c15afc0 con 0x7f236c0cee10 2023-12-19T18:18:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f2372a6e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f236c0cee10 0x7f236c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f237226d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f236c154220 0x7f236c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f2371a6c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f236c06b3d0 0x7f236c0ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:49474/0 (socket says 172.21.15.26:49474) 2023-12-19T18:18:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f2372a6e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f236c0cee10 0x7f236c15b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:40306/0 (socket says 172.21.15.26:40306) 2023-12-19T18:18:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f2371a6c640 1 -- 172.21.15.26:0/2152673399 learned_addr learned my addr 172.21.15.26:0/2152673399 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f237226d640 1 -- 172.21.15.26:0/2152673399 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f236c0cee10 msgr2=0x7f236c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:13.893 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f237226d640 1 --2- 172.21.15.26:0/2152673399 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f236c0cee10 0x7f236c15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f237226d640 1 -- 172.21.15.26:0/2152673399 >> v1:172.21.15.160:6789/0 conn(0x7f236c06b3d0 legacy=0x7f236c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f237226d640 1 -- 172.21.15.26:0/2152673399 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c06b950 con 0x7f236c154220 2023-12-19T18:18:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f2372a6e640 1 --2- 172.21.15.26:0/2152673399 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f236c0cee10 0x7f236c15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:13.894 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f237126b640 1 -- 172.21.15.26:0/2152673399 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654616402 0 0) 0x7f236c06bc30 con 0x7f236c06b3d0 2023-12-19T18:18:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f237226d640 1 --2- 172.21.15.26:0/2152673399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f236c154220 0x7f236c15aa80 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f23680097c0 tx=0x7f236801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54808a913411dfd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f237126b640 1 -- 172.21.15.26:0/2152673399 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23680ab650 con 0x7f236c154220 2023-12-19T18:18:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f237126b640 1 -- 172.21.15.26:0/2152673399 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23680ab7b0 con 0x7f236c154220 2023-12-19T18:18:13.896 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.892+0000 7f237126b640 1 -- 172.21.15.26:0/2152673399 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23680abb40 con 0x7f236c154220 2023-12-19T18:18:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 -- 172.21.15.26:0/2152673399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f236c154220 msgr2=0x7f236c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 --2- 172.21.15.26:0/2152673399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f236c154220 0x7f236c15aa80 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f23680097c0 tx=0x7f236801c450 comp rx=0 tx=0).stop 2023-12-19T18:18:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 -- 172.21.15.26:0/2152673399 shutdown_connections 2023-12-19T18:18:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 --2- 172.21.15.26:0/2152673399 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f236c0cee10 0x7f236c15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 --2- 172.21.15.26:0/2152673399 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f236c154220 0x7f236c15aa80 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 -- 172.21.15.26:0/2152673399 >> 172.21.15.26:0/2152673399 conn(0x7f236c0658e0 msgr2=0x7f236c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 -- 172.21.15.26:0/2152673399 shutdown_connections 2023-12-19T18:18:13.897 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 -- 172.21.15.26:0/2152673399 wait complete. 2023-12-19T18:18:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 Processor -- start 2023-12-19T18:18:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 -- start start 2023-12-19T18:18:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f236c06b3d0 0x7f236c1434d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f236c0cee10 0x7f236c151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c154220 0x7f236c151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c070ec0 con 0x7f236c154220 2023-12-19T18:18:13.899 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c0b2db0 con 0x7f236c0cee10 2023-12-19T18:18:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f23744d1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c143a10 con 0x7f236c06b3d0 2023-12-19T18:18:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f2372a6e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c154220 0x7f236c151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f2372a6e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c154220 0x7f236c151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54470/0 (socket says 172.21.15.26:54470) 2023-12-19T18:18:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f2372a6e640 1 -- 172.21.15.26:0/3551074257 learned_addr learned my addr 172.21.15.26:0/3551074257 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f2371a6c640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f236c0cee10 0x7f236c151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.896+0000 7f237226d640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f236c06b3d0 0x7f236c1434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.900+0000 7f2372a6e640 1 -- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f236c06b3d0 msgr2=0x7f236c1434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.900+0000 7f2372a6e640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f236c06b3d0 0x7f236c1434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.900+0000 7f2372a6e640 1 -- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f236c0cee10 msgr2=0x7f236c151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.900+0000 7f2372a6e640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f236c0cee10 0x7f236c151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.900+0000 7f2372a6e640 1 -- 172.21.15.26:0/3551074257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2364009a70 con 0x7f236c154220 2023-12-19T18:18:13.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.900+0000 7f2372a6e640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c154220 0x7f236c151b50 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f2364009f00 tx=0x7f236400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.900+0000 7f2362ffd640 1 -- 172.21.15.26:0/3551074257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364009d70 con 0x7f236c154220 2023-12-19T18:18:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.900+0000 7f2362ffd640 1 -- 172.21.15.26:0/3551074257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236400f040 con 0x7f236c154220 2023-12-19T18:18:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.900+0000 7f23744d1640 1 -- 172.21.15.26:0/3551074257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2368009420 con 0x7f236c154220 2023-12-19T18:18:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.900+0000 7f2362ffd640 1 -- 172.21.15.26:0/3551074257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364003bc0 con 0x7f236c154220 2023-12-19T18:18:13.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.900+0000 7f23744d1640 1 -- 172.21.15.26:0/3551074257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c0b3000 con 0x7f236c154220 2023-12-19T18:18:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.900+0000 7f23744d1640 1 -- 172.21.15.26:0/3551074257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c157cc0 con 0x7f236c154220 2023-12-19T18:18:13.902 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.900+0000 7f2362ffd640 1 -- 172.21.15.26:0/3551074257 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f2364014480 con 0x7f236c154220 2023-12-19T18:18:13.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.904+0000 7f2362ffd640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2350065cd0 0x7f2350068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.904+0000 7f2362ffd640 1 -- 172.21.15.26:0/3551074257 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(245..245 src has 1..245) v4 ==== 6563+0+0 (secure 0 0 0) 0x7f2364084550 con 0x7f236c154220 2023-12-19T18:18:13.903 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.904+0000 7f237226d640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2350065cd0 0x7f2350068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.904 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.904+0000 7f237226d640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2350065cd0 0x7f2350068190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f23680af000 tx=0x7f23680a7000 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:13.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:13.908+0000 7f2362ffd640 1 -- 172.21.15.26:0/3551074257 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f23640517f0 con 0x7f236c154220 2023-12-19T18:18:14.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f23744d1640 1 -- 172.21.15.26:0/3551074257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f236c074270 con 0x7f236c154220 2023-12-19T18:18:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f2362ffd640 1 -- 172.21.15.26:0/3551074257 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+3543 (secure 0 0 0) 0x7f2364018df0 con 0x7f236c154220 2023-12-19T18:18:14.096 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:14.096 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":122,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/2727392302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2727392302},{"type":"v1","addr":"172.21.15.160:6836","nonce":2727392302}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10768,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/1029020739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":1029020739},{"type":"v1","addr":"172.21.15.160:6837","nonce":1029020739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":12003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3746262698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3746262698},{"type":"v1","addr":"172.21.15.26:6835","nonce":3746262698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":12093,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/4173418691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4173418691},{"type":"v1","addr":"172.21.15.26:6834","nonce":4173418691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":122}],"filesystems":[]} 2023-12-19T18:18:14.097 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 122 2023-12-19T18:18:14.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f23744d1640 1 -- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2350065cd0 msgr2=0x7f2350068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:14.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f23744d1640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2350065cd0 0x7f2350068190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f23680af000 tx=0x7f23680a7000 comp rx=0 tx=0).stop 2023-12-19T18:18:14.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f23744d1640 1 -- 172.21.15.26:0/3551074257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c154220 msgr2=0x7f236c151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:14.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f23744d1640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c154220 0x7f236c151b50 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f2364009f00 tx=0x7f236400c490 comp rx=0 tx=0).stop 2023-12-19T18:18:14.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f23744d1640 1 -- 172.21.15.26:0/3551074257 shutdown_connections 2023-12-19T18:18:14.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f23744d1640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2350065cd0 0x7f2350068190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f23744d1640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c154220 0x7f236c151b50 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f23744d1640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f236c0cee10 0x7f236c151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f23744d1640 1 --2- 172.21.15.26:0/3551074257 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f236c06b3d0 0x7f236c1434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f23744d1640 1 -- 172.21.15.26:0/3551074257 >> 172.21.15.26:0/3551074257 conn(0x7f236c0658e0 msgr2=0x7f236c0cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:14.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f23744d1640 1 -- 172.21.15.26:0/3551074257 shutdown_connections 2023-12-19T18:18:14.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.096+0000 7f23744d1640 1 -- 172.21.15.26:0/3551074257 wait complete. 2023-12-19T18:18:14.119 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:18:14.119 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:18:14.119 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:18:14.119 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:18:14.119 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:18:14.119 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:18:14.120 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:18:14.120 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:18:14.120 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:18:14.120 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:18:14.120 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:18:14.120 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:18:14.120 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:18:14.121 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:18:14.121 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:18:14.121 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:18:14.134 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:18:14.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:18:14.136 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:18:14.136 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:18:14.138 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:18:14.138 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:18:14.139 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:18:14.139 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:18:14.178 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:18:14.179 DEBUG:teuthology.parallel:result is None 2023-12-19T18:18:14.181 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:18:14.181 DEBUG:teuthology.parallel:result is None 2023-12-19T18:18:14.183 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:18:14.183 DEBUG:teuthology.parallel:result is None 2023-12-19T18:18:14.185 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:18:14.185 DEBUG:teuthology.parallel:result is None 2023-12-19T18:18:14.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:18:14.267 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2023-12-19T18:18:14.267 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T18:18:14.275 INFO:tasks.ceph.mds.d.smithi160.stdout:starting mds.d at 2023-12-19T18:18:14.276 INFO:tasks.ceph.mds.b.smithi160.stdout:starting mds.b at 2023-12-19T18:18:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42e0525640 1 Processor -- start 2023-12-19T18:18:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42e0525640 1 -- start start 2023-12-19T18:18:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42e0525640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f42d8155030 0x7f42d815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42e0525640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f42d815f370 0x7f42d806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42e0525640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d806c970 con 0x7f42d80ce740 2023-12-19T18:18:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42e0525640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d815b1f0 con 0x7f42d8155030 2023-12-19T18:18:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42e0525640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d815b4d0 con 0x7f42d815f370 2023-12-19T18:18:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42de2c1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f42d8155030 0x7f42d815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42de2c1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f42d8155030 0x7f42d815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54510/0 (socket says 172.21.15.26:54510) 2023-12-19T18:18:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42de2c1640 1 -- 172.21.15.26:0/842085202 learned_addr learned my addr 172.21.15.26:0/842085202 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42ddac0640 1 --2- 172.21.15.26:0/842085202 >> v2:172.21.15.160:3300/0 conn(0x7f42d815f370 0x7f42d806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42de2c1640 1 -- 172.21.15.26:0/842085202 >> v2:172.21.15.160:3300/0 conn(0x7f42d815f370 msgr2=0x7f42d806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42de2c1640 1 --2- 172.21.15.26:0/842085202 >> v2:172.21.15.160:3300/0 conn(0x7f42d815f370 0x7f42d806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42de2c1640 1 -- 172.21.15.26:0/842085202 >> v1:172.21.15.26:6789/0 conn(0x7f42d80ce740 legacy=0x7f42d80ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42de2c1640 1 -- 172.21.15.26:0/842085202 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d815b7b0 con 0x7f42d8155030 2023-12-19T18:18:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.340+0000 7f42dd2bf640 1 -- 172.21.15.26:0/842085202 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909271486 0 0) 0x7f42d806c970 con 0x7f42d80ce740 2023-12-19T18:18:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.344+0000 7f42de2c1640 1 --2- 172.21.15.26:0/842085202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d8155030 0x7f42d815bcd0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f42d80b1780 tx=0x7f42d401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26385d86e42786e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.344+0000 7f42dd2bf640 1 -- 172.21.15.26:0/842085202 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d40ab620 con 0x7f42d8155030 2023-12-19T18:18:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.344+0000 7f42dd2bf640 1 -- 172.21.15.26:0/842085202 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d40ab780 con 0x7f42d8155030 2023-12-19T18:18:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.344+0000 7f42dd2bf640 1 -- 172.21.15.26:0/842085202 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d40abb10 con 0x7f42d8155030 2023-12-19T18:18:14.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.344+0000 7f42e0525640 1 -- 172.21.15.26:0/842085202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d8155030 msgr2=0x7f42d815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.344+0000 7f42e0525640 1 --2- 172.21.15.26:0/842085202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d8155030 0x7f42d815bcd0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f42d80b1780 tx=0x7f42d401c500 comp rx=0 tx=0).stop 2023-12-19T18:18:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.344+0000 7f42e0525640 1 -- 172.21.15.26:0/842085202 shutdown_connections 2023-12-19T18:18:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.344+0000 7f42e0525640 1 --2- 172.21.15.26:0/842085202 >> v2:172.21.15.160:3300/0 conn(0x7f42d815f370 0x7f42d806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.344+0000 7f42e0525640 1 --2- 172.21.15.26:0/842085202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d8155030 0x7f42d815bcd0 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.344+0000 7f42e0525640 1 -- 172.21.15.26:0/842085202 >> 172.21.15.26:0/842085202 conn(0x7f42d80658e0 msgr2=0x7f42d8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.344+0000 7f42e0525640 1 -- 172.21.15.26:0/842085202 shutdown_connections 2023-12-19T18:18:14.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.344+0000 7f42e0525640 1 -- 172.21.15.26:0/842085202 wait complete. 2023-12-19T18:18:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.344+0000 7f42e0525640 1 Processor -- start 2023-12-19T18:18:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42e0525640 1 -- start start 2023-12-19T18:18:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42e0525640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42d80ce740 0x7f42d8146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42e0525640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d8155030 0x7f42d81498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42e0525640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42d815f370 0x7f42d814ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42e0525640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d814eb10 con 0x7f42d8155030 2023-12-19T18:18:14.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42e0525640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8146890 con 0x7f42d80ce740 2023-12-19T18:18:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42e0525640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8146b70 con 0x7f42d815f370 2023-12-19T18:18:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42ddac0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d8155030 0x7f42d81498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42ddac0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d8155030 0x7f42d81498f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54524/0 (socket says 172.21.15.26:54524) 2023-12-19T18:18:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42ddac0640 1 -- 172.21.15.26:0/3022660768 learned_addr learned my addr 172.21.15.26:0/3022660768 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42de2c1640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42d80ce740 0x7f42d8146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42deac2640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42d815f370 0x7f42d814ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42ddac0640 1 -- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42d815f370 msgr2=0x7f42d814ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42ddac0640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42d815f370 0x7f42d814ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42ddac0640 1 -- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42d80ce740 msgr2=0x7f42d8146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:14.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42ddac0640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42d80ce740 0x7f42d8146190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42ddac0640 1 -- 172.21.15.26:0/3022660768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d40094d0 con 0x7f42d8155030 2023-12-19T18:18:14.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42deac2640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42d815f370 0x7f42d814ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:14.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42ddac0640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d8155030 0x7f42d81498f0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f42c800e870 tx=0x7f42c800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:14.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42ceffd640 1 -- 172.21.15.26:0/3022660768 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8009510 con 0x7f42d8155030 2023-12-19T18:18:14.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42ceffd640 1 -- 172.21.15.26:0/3022660768 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c800eec0 con 0x7f42d8155030 2023-12-19T18:18:14.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42ceffd640 1 -- 172.21.15.26:0/3022660768 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c800cb40 con 0x7f42d8155030 2023-12-19T18:18:14.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42e0525640 1 -- 172.21.15.26:0/3022660768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d80cf0e0 con 0x7f42d8155030 2023-12-19T18:18:14.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42e0525640 1 -- 172.21.15.26:0/3022660768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d8147130 con 0x7f42d8155030 2023-12-19T18:18:14.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.348+0000 7f42e0525640 1 -- 172.21.15.26:0/3022660768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d8074270 con 0x7f42d8155030 2023-12-19T18:18:14.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.352+0000 7f42ceffd640 1 -- 172.21.15.26:0/3022660768 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f42c800cca0 con 0x7f42d8155030 2023-12-19T18:18:14.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.352+0000 7f42ceffd640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42bc065ca0 0x7f42bc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:14.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.352+0000 7f42ceffd640 1 -- 172.21.15.26:0/3022660768 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 6563+0+0 (secure 0 0 0) 0x7f42c8083a00 con 0x7f42d8155030 2023-12-19T18:18:14.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.352+0000 7f42de2c1640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42bc065ca0 0x7f42bc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:14.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.352+0000 7f42de2c1640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42bc065ca0 0x7f42bc068160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42d40af000 tx=0x7f42d40a7000 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:14.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.356+0000 7f42ceffd640 1 -- 172.21.15.26:0/3022660768 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f42c8050ce0 con 0x7f42d8155030 2023-12-19T18:18:14.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:14.524+0000 7f42e0525640 1 -- 172.21.15.26:0/3022660768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f42d815aeb0 con 0x7f42d8155030 2023-12-19T18:18:15.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.476+0000 7f42ceffd640 1 -- 172.21.15.26:0/3022660768 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v250) v1 ==== 97+0+0 (secure 0 0 0) 0x7f42c8054300 con 0x7f42d8155030 2023-12-19T18:18:15.478 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2023-12-19T18:18:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.480+0000 7f42e0525640 1 -- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42bc065ca0 msgr2=0x7f42bc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.480+0000 7f42e0525640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42bc065ca0 0x7f42bc068160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42d40af000 tx=0x7f42d40a7000 comp rx=0 tx=0).stop 2023-12-19T18:18:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.480+0000 7f42e0525640 1 -- 172.21.15.26:0/3022660768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d8155030 msgr2=0x7f42d81498f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.480+0000 7f42e0525640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d8155030 0x7f42d81498f0 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f42c800e870 tx=0x7f42c800ec70 comp rx=0 tx=0).stop 2023-12-19T18:18:15.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.480+0000 7f42e0525640 1 -- 172.21.15.26:0/3022660768 shutdown_connections 2023-12-19T18:18:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.480+0000 7f42e0525640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f42bc065ca0 0x7f42bc068160 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.480+0000 7f42e0525640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f42d815f370 0x7f42d814ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.480+0000 7f42e0525640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42d8155030 0x7f42d81498f0 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.480+0000 7f42e0525640 1 --2- 172.21.15.26:0/3022660768 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42d80ce740 0x7f42d8146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.480+0000 7f42e0525640 1 -- 172.21.15.26:0/3022660768 >> 172.21.15.26:0/3022660768 conn(0x7f42d80658e0 msgr2=0x7f42d815d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.480+0000 7f42e0525640 1 -- 172.21.15.26:0/3022660768 shutdown_connections 2023-12-19T18:18:15.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.480+0000 7f42e0525640 1 -- 172.21.15.26:0/3022660768 wait complete. 2023-12-19T18:18:15.504 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:18:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.652+0000 7fea9fbbc640 1 Processor -- start 2023-12-19T18:18:15.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.652+0000 7fea9fbbc640 1 -- start start 2023-12-19T18:18:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9fbbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea980cb1d0 0x7fea980cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9fbbc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fea980b1d40 0x7fea980b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:15.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9fbbc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea9806d4b0 con 0x7fea9806bee0 2023-12-19T18:18:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9fbbc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea980cc030 con 0x7fea980b1d40 2023-12-19T18:18:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9fbbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea980cbaf0 con 0x7fea980cb1d0 2023-12-19T18:18:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9e159640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fea980b1d40 0x7fea980b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9d958640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea980cb1d0 0x7fea980cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9d958640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea980cb1d0 0x7fea980cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:40350/0 (socket says 172.21.15.26:40350) 2023-12-19T18:18:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9d958640 1 -- 172.21.15.26:0/939386330 learned_addr learned my addr 172.21.15.26:0/939386330 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9e159640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fea980b1d40 0x7fea980b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:58990/0 (socket says 172.21.15.26:58990) 2023-12-19T18:18:15.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9d958640 1 -- 172.21.15.26:0/939386330 >> v2:172.21.15.160:3300/0 conn(0x7fea980b1d40 msgr2=0x7fea980b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9d958640 1 --2- 172.21.15.26:0/939386330 >> v2:172.21.15.160:3300/0 conn(0x7fea980b1d40 0x7fea980b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9d958640 1 -- 172.21.15.26:0/939386330 >> v1:172.21.15.160:6789/0 conn(0x7fea9806bee0 legacy=0x7fea9806c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9d958640 1 -- 172.21.15.26:0/939386330 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9806cc10 con 0x7fea980cb1d0 2023-12-19T18:18:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9e159640 1 --2- 172.21.15.26:0/939386330 >> v2:172.21.15.160:3300/0 conn(0x7fea980b1d40 0x7fea980b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:15.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.656+0000 7fea9c956640 1 -- 172.21.15.26:0/939386330 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907710223 0 0) 0x7fea9806d4b0 con 0x7fea9806bee0 2023-12-19T18:18:15.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9d958640 1 --2- 172.21.15.26:0/939386330 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea980cb1d0 0x7fea980cb5b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fea94009870 tx=0x7fea9401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7dfa1b7f668a093 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:15.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9c956640 1 -- 172.21.15.26:0/939386330 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea940aa610 con 0x7fea980cb1d0 2023-12-19T18:18:15.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9c956640 1 -- 172.21.15.26:0/939386330 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea940a6040 con 0x7fea980cb1d0 2023-12-19T18:18:15.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9c956640 1 -- 172.21.15.26:0/939386330 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea940aa990 con 0x7fea980cb1d0 2023-12-19T18:18:15.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 -- 172.21.15.26:0/939386330 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea980cb1d0 msgr2=0x7fea980cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:15.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 --2- 172.21.15.26:0/939386330 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea980cb1d0 0x7fea980cb5b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fea94009870 tx=0x7fea9401c4b0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 -- 172.21.15.26:0/939386330 shutdown_connections 2023-12-19T18:18:15.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 --2- 172.21.15.26:0/939386330 >> v2:172.21.15.160:3300/0 conn(0x7fea980b1d40 0x7fea980b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 --2- 172.21.15.26:0/939386330 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea980cb1d0 0x7fea980cb5b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 -- 172.21.15.26:0/939386330 >> 172.21.15.26:0/939386330 conn(0x7fea980658e0 msgr2=0x7fea98065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:15.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 -- 172.21.15.26:0/939386330 shutdown_connections 2023-12-19T18:18:15.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 -- 172.21.15.26:0/939386330 wait complete. 2023-12-19T18:18:15.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 Processor -- start 2023-12-19T18:18:15.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 -- start start 2023-12-19T18:18:15.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea9806bee0 0x7fea980acc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:15.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea980b1d40 0x7fea980af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:15.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea980cb1d0 0x7fea980cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:15.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea980ad150 con 0x7fea980cb1d0 2023-12-19T18:18:15.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9814f3b0 con 0x7fea9806bee0 2023-12-19T18:18:15.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.660+0000 7fea9fbbc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea980ae300 con 0x7fea980b1d40 2023-12-19T18:18:15.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9e159640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea980cb1d0 0x7fea980cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:15.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9d958640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea9806bee0 0x7fea980acc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:15.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9e159640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea980cb1d0 0x7fea980cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54532/0 (socket says 172.21.15.26:54532) 2023-12-19T18:18:15.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9e159640 1 -- 172.21.15.26:0/3746955041 learned_addr learned my addr 172.21.15.26:0/3746955041 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:15.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9d157640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea980b1d40 0x7fea980af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9d958640 1 -- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea980b1d40 msgr2=0x7fea980af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9d958640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea980b1d40 0x7fea980af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9d958640 1 -- 172.21.15.26:0/3746955041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea980cb1d0 msgr2=0x7fea980cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9d958640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea980cb1d0 0x7fea980cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9d958640 1 -- 172.21.15.26:0/3746955041 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea90009a70 con 0x7fea9806bee0 2023-12-19T18:18:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9e159640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea980cb1d0 0x7fea980cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9d958640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea9806bee0 0x7fea980acc10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fea940ab000 tx=0x7fea9401acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea8e7fc640 1 -- 172.21.15.26:0/3746955041 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea940bd5f0 con 0x7fea9806bee0 2023-12-19T18:18:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea8e7fc640 1 -- 172.21.15.26:0/3746955041 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea940bd750 con 0x7fea9806bee0 2023-12-19T18:18:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea8e7fc640 1 -- 172.21.15.26:0/3746955041 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea940bdb00 con 0x7fea9806bee0 2023-12-19T18:18:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9fbbc640 1 -- 172.21.15.26:0/3746955041 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea940094d0 con 0x7fea9806bee0 2023-12-19T18:18:15.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9fbbc640 1 -- 172.21.15.26:0/3746955041 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea980ae8c0 con 0x7fea9806bee0 2023-12-19T18:18:15.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea9fbbc640 1 -- 172.21.15.26:0/3746955041 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea98074270 con 0x7fea9806bee0 2023-12-19T18:18:15.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.664+0000 7fea8e7fc640 1 -- 172.21.15.26:0/3746955041 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fea940059a0 con 0x7fea9806bee0 2023-12-19T18:18:15.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.668+0000 7fea8e7fc640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fea7c065ca0 0x7fea7c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:15.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.668+0000 7fea8e7fc640 1 -- 172.21.15.26:0/3746955041 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5617+0+0 (secure 0 0 0) 0x7fea9411ea00 con 0x7fea9806bee0 2023-12-19T18:18:15.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.668+0000 7fea9d157640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fea7c065ca0 0x7fea7c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:15.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.668+0000 7fea9d157640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fea7c065ca0 0x7fea7c068160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fea98070ba0 tx=0x7fea88009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:15.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.672+0000 7fea8e7fc640 1 -- 172.21.15.26:0/3746955041 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fea940ec150 con 0x7fea9806bee0 2023-12-19T18:18:15.884 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea9fbbc640 1 -- 172.21.15.26:0/3746955041 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fea980ccdf0 con 0x7fea9806bee0 2023-12-19T18:18:15.885 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea8e7fc640 1 -- 172.21.15.26:0/3746955041 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v54) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fea940ef770 con 0x7fea9806bee0 2023-12-19T18:18:15.885 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:15.885 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:18:15.885 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2023-12-19T18:18:15.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.885 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2023-12-19T18:18:15.885 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlkAVGLBAAW6iw06HzN+PiqSVdBcR3Sg==", 2023-12-19T18:18:15.885 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.886 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:18:15.886 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:18:15.886 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:18:15.886 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.886 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.886 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2023-12-19T18:18:15.886 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4Flsg/aOBAAp9Qp0KuDWK6HILR+Z27lFg==", 2023-12-19T18:18:15.886 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.886 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:18:15.886 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllGfjMxAA3fkMuJUtB0JTHjwJGyFntg==", 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.888 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2023-12-19T18:18:15.888 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1viGBBAAwUqy2c/l+LiItxDqTgd9iw==", 2023-12-19T18:18:15.888 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.888 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:18:15.888 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:18:15.888 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:18:15.888 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.888 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.888 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.888 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2023-12-19T18:18:15.889 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC7x4FlsI6QJRAAEe017CFrqN+fNl09G/56ig==", 2023-12-19T18:18:15.889 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.889 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:15.889 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:18:15.889 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.889 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.889 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.889 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.889 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2023-12-19T18:18:15.889 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC+x4FlBSzTLBAAXsS0u8wB2tR+SPeX525lTA==", 2023-12-19T18:18:15.889 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.889 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDBx4Flp3JbMxAATQ/ji9eLEhqgPi1pOfZaZQ==", 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.890 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.891 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2023-12-19T18:18:15.891 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDEx4FlVmt2ORAAkKuvh1P4p3kxglKY5/c1HQ==", 2023-12-19T18:18:15.891 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.891 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:15.891 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:18:15.891 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.891 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.891 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.891 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.891 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2023-12-19T18:18:15.891 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDLx4Fl9k7+BBAArIS+eJ3/PxRcHPVQi47R4A==", 2023-12-19T18:18:15.891 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.892 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:15.892 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:18:15.892 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.892 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.892 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.892 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.892 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2023-12-19T18:18:15.892 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDOx4FlaZG6DBAAh6ripytivaIUBlT0Nl/HmQ==", 2023-12-19T18:18:15.892 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.892 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:15.892 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:18:15.892 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.892 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.893 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2023-12-19T18:18:15.893 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDRx4FlKnc0FBAA8/QpJ2Y2p06cpocbkpvhFg==", 2023-12-19T18:18:15.893 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.893 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:15.893 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:18:15.893 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.893 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.893 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.893 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.893 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQDUx4Flnb9lGxAAYxiwkewRPyHm/BQDVuk+rg==", 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4Fl1vmxCBAAKv5443D5uEcsTeaacaB8XQ==", 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2023-12-19T18:18:15.894 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T18:18:15.895 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.895 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.895 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2023-12-19T18:18:15.895 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlrrQLEBAA8Kik/tnJhdJG2mvqfd4IxQ==", 2023-12-19T18:18:15.895 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.895 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:18:15.895 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T18:18:15.895 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T18:18:15.895 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T18:18:15.895 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.895 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.896 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2023-12-19T18:18:15.896 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlzTZrDBAARwiXfuZPVS5xsvAZHmYOVQ==", 2023-12-19T18:18:15.896 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.896 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:18:15.896 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T18:18:15.896 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T18:18:15.896 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T18:18:15.896 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.896 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.896 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.896 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2023-12-19T18:18:15.897 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC6x4FlRYQ7FhAAn4Pg/Xx9lqTlGbFpMZkekQ==", 2023-12-19T18:18:15.897 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.897 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2023-12-19T18:18:15.897 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2023-12-19T18:18:15.897 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2023-12-19T18:18:15.897 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2023-12-19T18:18:15.897 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.897 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.897 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.897 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2023-12-19T18:18:15.897 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlT5ynDxAA7Sjbdwrc1RyAXJboysB54Q==", 2023-12-19T18:18:15.897 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.897 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T18:18:15.898 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2023-12-19T18:18:15.898 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2023-12-19T18:18:15.898 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.898 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.898 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.898 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.898 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2023-12-19T18:18:15.898 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FllEcVKBAA8G3XOLTiMvHMQMiFq2LSQA==", 2023-12-19T18:18:15.898 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.898 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T18:18:15.898 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T18:18:15.898 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.898 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.899 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2023-12-19T18:18:15.899 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:15.899 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2023-12-19T18:18:15.899 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQC5x4FlOpYxJBAA7apqT0xfUGNYzBruTDckYg==", 2023-12-19T18:18:15.899 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2023-12-19T18:18:15.899 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2023-12-19T18:18:15.899 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2023-12-19T18:18:15.899 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2023-12-19T18:18:15.899 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.899 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:15.899 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2023-12-19T18:18:15.899 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:18:15.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea9fbbc640 1 -- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fea7c065ca0 msgr2=0x7fea7c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:15.900 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea9fbbc640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fea7c065ca0 0x7fea7c068160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fea98070ba0 tx=0x7fea88009040 comp rx=0 tx=0).stop 2023-12-19T18:18:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea9fbbc640 1 -- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea9806bee0 msgr2=0x7fea980acc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea9fbbc640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea9806bee0 0x7fea980acc10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fea940ab000 tx=0x7fea9401acf0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea9fbbc640 1 -- 172.21.15.26:0/3746955041 shutdown_connections 2023-12-19T18:18:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea9fbbc640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fea7c065ca0 0x7fea7c068160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea9fbbc640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea980cb1d0 0x7fea980cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea9fbbc640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea980b1d40 0x7fea980af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea9fbbc640 1 --2- 172.21.15.26:0/3746955041 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea9806bee0 0x7fea980acc10 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea9fbbc640 1 -- 172.21.15.26:0/3746955041 >> 172.21.15.26:0/3746955041 conn(0x7fea980658e0 msgr2=0x7fea98065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea9fbbc640 1 -- 172.21.15.26:0/3746955041 shutdown_connections 2023-12-19T18:18:15.901 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:15.884+0000 7fea9fbbc640 1 -- 172.21.15.26:0/3746955041 wait complete. 2023-12-19T18:18:15.906 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:16.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.048+0000 7f71ea3f0640 1 Processor -- start 2023-12-19T18:18:16.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.048+0000 7f71ea3f0640 1 -- start start 2023-12-19T18:18:16.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.048+0000 7f71ea3f0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f71e4154220 0x7f71e415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.048+0000 7f71ea3f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71e406b3d0 0x7f71e40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.048+0000 7f71ea3f0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e406bc30 con 0x7f71e40cec40 2023-12-19T18:18:16.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.048+0000 7f71ea3f0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e40cf0d0 con 0x7f71e4154220 2023-12-19T18:18:16.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.048+0000 7f71ea3f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e40cc6e0 con 0x7f71e406b3d0 2023-12-19T18:18:16.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e3fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f71e4154220 0x7f71e415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e898d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f71e40cec40 0x7f71e415d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36138/0 (socket says 172.21.15.26:36138) 2023-12-19T18:18:16.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e898d640 1 -- 172.21.15.26:0/4049376074 learned_addr learned my addr 172.21.15.26:0/4049376074 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:16.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e37fe640 1 --2- 172.21.15.26:0/4049376074 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71e406b3d0 0x7f71e40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e3fff640 1 -- 172.21.15.26:0/4049376074 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71e406b3d0 msgr2=0x7f71e40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:16.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e3fff640 1 --2- 172.21.15.26:0/4049376074 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71e406b3d0 0x7f71e40ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e3fff640 1 -- 172.21.15.26:0/4049376074 >> v1:172.21.15.26:6789/0 conn(0x7f71e40cec40 legacy=0x7f71e415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e3fff640 1 -- 172.21.15.26:0/4049376074 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e415afc0 con 0x7f71e4154220 2023-12-19T18:18:16.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e37fe640 1 --2- 172.21.15.26:0/4049376074 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71e406b3d0 0x7f71e40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:16.053 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e2ffd640 1 -- 172.21.15.26:0/4049376074 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969418154 0 0) 0x7f71e406bc30 con 0x7f71e40cec40 2023-12-19T18:18:16.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e3fff640 1 --2- 172.21.15.26:0/4049376074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e4154220 0x7f71e415aa80 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f71d40097c0 tx=0x7f71d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4636bb024d856332 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e2ffd640 1 -- 172.21.15.26:0/4049376074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d40ab600 con 0x7f71e4154220 2023-12-19T18:18:16.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e2ffd640 1 -- 172.21.15.26:0/4049376074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d40a7050 con 0x7f71e4154220 2023-12-19T18:18:16.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.052+0000 7f71e2ffd640 1 -- 172.21.15.26:0/4049376074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d40ab980 con 0x7f71e4154220 2023-12-19T18:18:16.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 -- 172.21.15.26:0/4049376074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e4154220 msgr2=0x7f71e415aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 --2- 172.21.15.26:0/4049376074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e4154220 0x7f71e415aa80 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f71d40097c0 tx=0x7f71d401c450 comp rx=0 tx=0).stop 2023-12-19T18:18:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 -- 172.21.15.26:0/4049376074 shutdown_connections 2023-12-19T18:18:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 --2- 172.21.15.26:0/4049376074 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71e406b3d0 0x7f71e40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 --2- 172.21.15.26:0/4049376074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e4154220 0x7f71e415aa80 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 -- 172.21.15.26:0/4049376074 >> 172.21.15.26:0/4049376074 conn(0x7f71e40658e0 msgr2=0x7f71e4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 -- 172.21.15.26:0/4049376074 shutdown_connections 2023-12-19T18:18:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 -- 172.21.15.26:0/4049376074 wait complete. 2023-12-19T18:18:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 Processor -- start 2023-12-19T18:18:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 -- start start 2023-12-19T18:18:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71e406b3d0 0x7f71e4158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71e40cec40 0x7f71e413d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e4154220 0x7f71e40b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e4155c20 con 0x7f71e4154220 2023-12-19T18:18:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e4155010 con 0x7f71e406b3d0 2023-12-19T18:18:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71ea3f0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e41552f0 con 0x7f71e40cec40 2023-12-19T18:18:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71e898d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e4154220 0x7f71e40b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71e898d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e4154220 0x7f71e40b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38138/0 (socket says 172.21.15.26:38138) 2023-12-19T18:18:16.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71e898d640 1 -- 172.21.15.26:0/1754092024 learned_addr learned my addr 172.21.15.26:0/1754092024 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:16.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71e3fff640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71e406b3d0 0x7f71e4158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.056+0000 7f71e37fe640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71e40cec40 0x7f71e413d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71e898d640 1 -- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71e40cec40 msgr2=0x7f71e413d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71e898d640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71e40cec40 0x7f71e413d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71e898d640 1 -- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71e406b3d0 msgr2=0x7f71e4158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71e898d640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71e406b3d0 0x7f71e4158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71e898d640 1 -- 172.21.15.26:0/1754092024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d4009420 con 0x7f71e4154220 2023-12-19T18:18:16.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71e898d640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e4154220 0x7f71e40b03d0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00c780 tx=0x7f71dc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71e0ff9640 1 -- 172.21.15.26:0/1754092024 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc00ce90 con 0x7f71e4154220 2023-12-19T18:18:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71e0ff9640 1 -- 172.21.15.26:0/1754092024 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71dc00a640 con 0x7f71e4154220 2023-12-19T18:18:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71e0ff9640 1 -- 172.21.15.26:0/1754092024 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc00a9d0 con 0x7f71e4154220 2023-12-19T18:18:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71ea3f0640 1 -- 172.21.15.26:0/1754092024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e41555d0 con 0x7f71e4154220 2023-12-19T18:18:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71ea3f0640 1 -- 172.21.15.26:0/1754092024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e413d5f0 con 0x7f71e4154220 2023-12-19T18:18:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71ea3f0640 1 -- 172.21.15.26:0/1754092024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e4155d90 con 0x7f71e4154220 2023-12-19T18:18:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71e0ff9640 1 -- 172.21.15.26:0/1754092024 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f71dc00ab30 con 0x7f71e4154220 2023-12-19T18:18:16.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71e0ff9640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71d0065ca0 0x7f71d0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.060+0000 7f71e0ff9640 1 -- 172.21.15.26:0/1754092024 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f71dc082ab0 con 0x7f71e4154220 2023-12-19T18:18:16.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.064+0000 7f71e3fff640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71d0065ca0 0x7f71d0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.064+0000 7f71e3fff640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71d0065ca0 0x7f71d0068160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71d40ad000 tx=0x7f71d40a6000 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.064+0000 7f71e0ff9640 1 -- 172.21.15.26:0/1754092024 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f71dc050170 con 0x7f71e4154220 2023-12-19T18:18:16.227 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71ea3f0640 1 -- 172.21.15.26:0/1754092024 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71e4074270 con 0x7f71e4154220 2023-12-19T18:18:16.228 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71e0ff9640 1 -- 172.21.15.26:0/1754092024 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v250) v1 ==== 74+0+13549 (secure 0 0 0) 0x7f71dc018070 con 0x7f71e4154220 2023-12-19T18:18:16.228 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:16.228 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":250,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:18:15.473951+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":235,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71ea3f0640 1 -- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71d0065ca0 msgr2=0x7f71d0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71ea3f0640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71d0065ca0 0x7f71d0068160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71d40ad000 tx=0x7f71d40a6000 comp rx=0 tx=0).stop 2023-12-19T18:18:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71ea3f0640 1 -- 172.21.15.26:0/1754092024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e4154220 msgr2=0x7f71e40b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71ea3f0640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e4154220 0x7f71e40b03d0 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00c780 tx=0x7f71dc00cc40 comp rx=0 tx=0).stop 2023-12-19T18:18:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71ea3f0640 1 -- 172.21.15.26:0/1754092024 shutdown_connections 2023-12-19T18:18:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71ea3f0640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71d0065ca0 0x7f71d0068160 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71ea3f0640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71e4154220 0x7f71e40b03d0 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71ea3f0640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71e40cec40 0x7f71e413d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71ea3f0640 1 --2- 172.21.15.26:0/1754092024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71e406b3d0 0x7f71e4158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71ea3f0640 1 -- 172.21.15.26:0/1754092024 >> 172.21.15.26:0/1754092024 conn(0x7f71e40658e0 msgr2=0x7f71e4065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71ea3f0640 1 -- 172.21.15.26:0/1754092024 shutdown_connections 2023-12-19T18:18:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.228+0000 7f71ea3f0640 1 -- 172.21.15.26:0/1754092024 wait complete. 2023-12-19T18:18:16.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:18:16.251 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:18:16.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.396+0000 7f90e8325640 1 Processor -- start 2023-12-19T18:18:16.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.396+0000 7f90e8325640 1 -- start start 2023-12-19T18:18:16.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.396+0000 7f90e8325640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90e0159820 0x7f90e015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.396+0000 7f90e8325640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90e006f6c0 0x7f90e00c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.396+0000 7f90e8325640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e006fb60 con 0x7f90e0159c00 2023-12-19T18:18:16.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.396+0000 7f90e8325640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e015e470 con 0x7f90e006f6c0 2023-12-19T18:18:16.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.396+0000 7f90e8325640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e006fe40 con 0x7f90e0159820 2023-12-19T18:18:16.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.396+0000 7f90e58c0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90e006f6c0 0x7f90e00c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.396+0000 7f90e58c0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90e006f6c0 0x7f90e00c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38140/0 (socket says 172.21.15.26:38140) 2023-12-19T18:18:16.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.396+0000 7f90e58c0640 1 -- 172.21.15.26:0/407266361 learned_addr learned my addr 172.21.15.26:0/407266361 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:16.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.396+0000 7f90e60c1640 1 --2- 172.21.15.26:0/407266361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90e0159820 0x7f90e015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e58c0640 1 -- 172.21.15.26:0/407266361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90e0159820 msgr2=0x7f90e015df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:16.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e58c0640 1 --2- 172.21.15.26:0/407266361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90e0159820 0x7f90e015df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e58c0640 1 -- 172.21.15.26:0/407266361 >> v1:172.21.15.160:6789/0 conn(0x7f90e0159c00 legacy=0x7f90e0070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e58c0640 1 -- 172.21.15.26:0/407266361 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e0070120 con 0x7f90e006f6c0 2023-12-19T18:18:16.401 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e58c0640 1 --2- 172.21.15.26:0/407266361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e006f6c0 0x7f90e00c8e80 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009870 tx=0x7f90d001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf4309d937abd47 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e50bf640 1 -- 172.21.15.26:0/407266361 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d00aa670 con 0x7f90e006f6c0 2023-12-19T18:18:16.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e50bf640 1 -- 172.21.15.26:0/407266361 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d00aa7d0 con 0x7f90e006f6c0 2023-12-19T18:18:16.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e50bf640 1 -- 172.21.15.26:0/407266361 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d00aab60 con 0x7f90e006f6c0 2023-12-19T18:18:16.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e8325640 1 -- 172.21.15.26:0/407266361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e006f6c0 msgr2=0x7f90e00c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e8325640 1 --2- 172.21.15.26:0/407266361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e006f6c0 0x7f90e00c8e80 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7f90d0009870 tx=0x7f90d001c510 comp rx=0 tx=0).stop 2023-12-19T18:18:16.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e8325640 1 -- 172.21.15.26:0/407266361 shutdown_connections 2023-12-19T18:18:16.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e8325640 1 --2- 172.21.15.26:0/407266361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e006f6c0 0x7f90e00c8e80 unknown :-1 s=CLOSED pgs=2321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e8325640 1 --2- 172.21.15.26:0/407266361 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90e0159820 0x7f90e015df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.400+0000 7f90e8325640 1 -- 172.21.15.26:0/407266361 >> 172.21.15.26:0/407266361 conn(0x7f90e00ce730 msgr2=0x7f90e00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:16.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e8325640 1 -- 172.21.15.26:0/407266361 shutdown_connections 2023-12-19T18:18:16.403 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e8325640 1 -- 172.21.15.26:0/407266361 wait complete. 2023-12-19T18:18:16.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e8325640 1 Processor -- start 2023-12-19T18:18:16.405 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e8325640 1 -- start start 2023-12-19T18:18:16.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e8325640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90e006f6c0 0x7f90e0149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e8325640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90e0159820 0x7f90e00cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e8325640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e0159c00 0x7f90e00ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e8325640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e014afd0 con 0x7f90e0159c00 2023-12-19T18:18:16.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e8325640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00cd760 con 0x7f90e006f6c0 2023-12-19T18:18:16.406 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e8325640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e014b220 con 0x7f90e0159820 2023-12-19T18:18:16.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e68c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e0159c00 0x7f90e00ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e60c1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90e006f6c0 0x7f90e0149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e68c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e0159c00 0x7f90e00ccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38152/0 (socket says 172.21.15.26:38152) 2023-12-19T18:18:16.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e68c2640 1 -- 172.21.15.26:0/2409811269 learned_addr learned my addr 172.21.15.26:0/2409811269 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:16.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e58c0640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90e0159820 0x7f90e00cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e60c1640 1 -- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90e0159820 msgr2=0x7f90e00cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e60c1640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90e0159820 0x7f90e00cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e60c1640 1 -- 172.21.15.26:0/2409811269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e0159c00 msgr2=0x7f90e00ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e60c1640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e0159c00 0x7f90e00ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e60c1640 1 -- 172.21.15.26:0/2409811269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d00094d0 con 0x7f90e006f6c0 2023-12-19T18:18:16.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e68c2640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e0159c00 0x7f90e00ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:16.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.404+0000 7f90e60c1640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90e006f6c0 0x7f90e0149d00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00e870 tx=0x7f90dc00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.408+0000 7f90d6ffd640 1 -- 172.21.15.26:0/2409811269 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0095d0 con 0x7f90e006f6c0 2023-12-19T18:18:16.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.408+0000 7f90d6ffd640 1 -- 172.21.15.26:0/2409811269 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90dc00c940 con 0x7f90e006f6c0 2023-12-19T18:18:16.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.408+0000 7f90d6ffd640 1 -- 172.21.15.26:0/2409811269 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90dc00ccb0 con 0x7f90e006f6c0 2023-12-19T18:18:16.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.408+0000 7f90e8325640 1 -- 172.21.15.26:0/2409811269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e006b880 con 0x7f90e006f6c0 2023-12-19T18:18:16.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.408+0000 7f90e8325640 1 -- 172.21.15.26:0/2409811269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e014b7e0 con 0x7f90e006f6c0 2023-12-19T18:18:16.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.408+0000 7f90e8325640 1 -- 172.21.15.26:0/2409811269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e0074270 con 0x7f90e006f6c0 2023-12-19T18:18:16.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.408+0000 7f90d6ffd640 1 -- 172.21.15.26:0/2409811269 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f90dc013470 con 0x7f90e006f6c0 2023-12-19T18:18:16.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.408+0000 7f90d6ffd640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f90c4065ca0 0x7f90c4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.408+0000 7f90d6ffd640 1 -- 172.21.15.26:0/2409811269 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f90dc083670 con 0x7f90e006f6c0 2023-12-19T18:18:16.411 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.408+0000 7f90e58c0640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f90c4065ca0 0x7f90c4068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.412 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.412+0000 7f90e58c0640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f90c4065ca0 0x7f90c4068160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f90d00ae000 tx=0x7f90d00b5040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.415 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.416+0000 7f90d6ffd640 1 -- 172.21.15.26:0/2409811269 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f90dc050dc0 con 0x7f90e006f6c0 2023-12-19T18:18:16.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.572+0000 7f90e8325640 1 -- 172.21.15.26:0/2409811269 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f90e00b46b0 con 0x7f90e006f6c0 2023-12-19T18:18:16.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.576+0000 7f90d6ffd640 1 -- 172.21.15.26:0/2409811269 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v250) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90dc0543e0 con 0x7f90e006f6c0 2023-12-19T18:18:16.576 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:18:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.576+0000 7f90e8325640 1 -- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f90c4065ca0 msgr2=0x7f90c4068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.576+0000 7f90e8325640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f90c4065ca0 0x7f90c4068160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f90d00ae000 tx=0x7f90d00b5040 comp rx=0 tx=0).stop 2023-12-19T18:18:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.576+0000 7f90e8325640 1 -- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90e006f6c0 msgr2=0x7f90e0149d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.576+0000 7f90e8325640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90e006f6c0 0x7f90e0149d00 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00e870 tx=0x7f90dc00ed30 comp rx=0 tx=0).stop 2023-12-19T18:18:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.576+0000 7f90e8325640 1 -- 172.21.15.26:0/2409811269 shutdown_connections 2023-12-19T18:18:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.576+0000 7f90e8325640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f90c4065ca0 0x7f90c4068160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.576+0000 7f90e8325640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90e0159c00 0x7f90e00ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.576+0000 7f90e8325640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f90e0159820 0x7f90e00cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.576+0000 7f90e8325640 1 --2- 172.21.15.26:0/2409811269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f90e006f6c0 0x7f90e0149d00 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.576+0000 7f90e8325640 1 -- 172.21.15.26:0/2409811269 >> 172.21.15.26:0/2409811269 conn(0x7f90e00ce730 msgr2=0x7f90e0150f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:16.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.576+0000 7f90e8325640 1 -- 172.21.15.26:0/2409811269 shutdown_connections 2023-12-19T18:18:16.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.576+0000 7f90e8325640 1 -- 172.21.15.26:0/2409811269 wait complete. 2023-12-19T18:18:16.597 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:18:16.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.740+0000 7f5184255640 1 Processor -- start 2023-12-19T18:18:16.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.740+0000 7f5184255640 1 -- start start 2023-12-19T18:18:16.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.740+0000 7f5184255640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f517c15c7f0 0x7f517c06c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.740+0000 7f5184255640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517c06ced0 con 0x7f517c0adb80 2023-12-19T18:18:16.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.740+0000 7f5184255640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517c06d1b0 con 0x7f517c0b30d0 2023-12-19T18:18:16.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.740+0000 7f5184255640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517c06d770 con 0x7f517c15c7f0 2023-12-19T18:18:16.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f51817f0640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f517c0adb80 0x7f517c0cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36150/0 (socket says 172.21.15.26:36150) 2023-12-19T18:18:16.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f51817f0640 1 -- 172.21.15.26:0/1169941821 learned_addr learned my addr 172.21.15.26:0/1169941821 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:16.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5181ff1640 1 --2- 172.21.15.26:0/1169941821 >> v2:172.21.15.160:3300/0 conn(0x7f517c15c7f0 0x7f517c06c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926018753 0 0) 0x7f517c06ced0 con 0x7f517c0adb80 2023-12-19T18:18:16.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5168003530 con 0x7f517c0adb80 2023-12-19T18:18:16.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1445593344 0 0) 0x7f5168003530 con 0x7f517c0adb80 2023-12-19T18:18:16.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 >> v2:172.21.15.160:3300/0 conn(0x7f517c15c7f0 msgr2=0x7f517c06c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:16.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 --2- 172.21.15.26:0/1169941821 >> v2:172.21.15.160:3300/0 conn(0x7f517c15c7f0 0x7f517c06c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 >> v1:172.21.15.160:6789/0 conn(0x7f517c0b30d0 legacy=0x7f517c0b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c06d490 con 0x7f517c0adb80 2023-12-19T18:18:16.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f516c001680 con 0x7f517c0adb80 2023-12-19T18:18:16.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 >> v1:172.21.15.26:6789/0 conn(0x7f517c0adb80 legacy=0x7f517c0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 --2- 172.21.15.26:0/1169941821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5168003980 0x7f5168003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 --2- 172.21.15.26:0/1169941821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51680047d0 0x7f5168004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 --2- 172.21.15.26:0/1169941821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5168005190 0x7f516800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c06d770 con 0x7f51680047d0 2023-12-19T18:18:16.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516800c030 con 0x7f5168003980 2023-12-19T18:18:16.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516800c1f0 con 0x7f5168005190 2023-12-19T18:18:16.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f51817f0640 1 --2- 172.21.15.26:0/1169941821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51680047d0 0x7f5168004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f5181ff1640 1 --2- 172.21.15.26:0/1169941821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5168003980 0x7f5168003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f51827f2640 1 --2- 172.21.15.26:0/1169941821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5168005190 0x7f516800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f51817f0640 1 -- 172.21.15.26:0/1169941821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5168005190 msgr2=0x7f516800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:16.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f51817f0640 1 --2- 172.21.15.26:0/1169941821 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f5168005190 0x7f516800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f51817f0640 1 -- 172.21.15.26:0/1169941821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5168003980 msgr2=0x7f5168003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:16.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f51817f0640 1 --2- 172.21.15.26:0/1169941821 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5168003980 0x7f5168003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f51817f0640 1 -- 172.21.15.26:0/1169941821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5168003530 con 0x7f51680047d0 2023-12-19T18:18:16.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.744+0000 7f51827f2640 1 -- 172.21.15.26:0/1169941821 reap_dead start 2023-12-19T18:18:16.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.748+0000 7f51817f0640 1 --2- 172.21.15.26:0/1169941821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51680047d0 0x7f5168004c50 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f516c00c240 tx=0x7f516c01f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfc5b97e3d29781 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.748+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c005ae0 con 0x7f51680047d0 2023-12-19T18:18:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.748+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c0ab070 con 0x7f51680047d0 2023-12-19T18:18:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.748+0000 7f5180fef640 1 -- 172.21.15.26:0/1169941821 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c007070 con 0x7f51680047d0 2023-12-19T18:18:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.748+0000 7f5184255640 1 -- 172.21.15.26:0/1169941821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51680047d0 msgr2=0x7f5168004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.748+0000 7f5184255640 1 --2- 172.21.15.26:0/1169941821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51680047d0 0x7f5168004c50 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f516c00c240 tx=0x7f516c01f3b0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.748+0000 7f5184255640 1 -- 172.21.15.26:0/1169941821 shutdown_connections 2023-12-19T18:18:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.748+0000 7f5184255640 1 --2- 172.21.15.26:0/1169941821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f51680047d0 0x7f5168004c50 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.748+0000 7f5184255640 1 -- 172.21.15.26:0/1169941821 >> 172.21.15.26:0/1169941821 conn(0x7f517c0658e0 msgr2=0x7f517c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.748+0000 7f5184255640 1 -- 172.21.15.26:0/1169941821 shutdown_connections 2023-12-19T18:18:16.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.748+0000 7f5184255640 1 -- 172.21.15.26:0/1169941821 wait complete. 2023-12-19T18:18:16.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.748+0000 7f5184255640 1 Processor -- start 2023-12-19T18:18:16.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.748+0000 7f5184255640 1 -- start start 2023-12-19T18:18:16.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f5184255640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51680047d0 0x7f517c1e2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f5184255640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f517c1e20c0 0x7f517c150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f5184255640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f517c0b1790 0x7f517c0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f5184255640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c1e33a0 con 0x7f517c1e20c0 2023-12-19T18:18:16.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f5184255640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0b2150 con 0x7f517c0b1790 2023-12-19T18:18:16.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f5184255640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517c15a830 con 0x7f51680047d0 2023-12-19T18:18:16.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51817f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f517c1e20c0 0x7f517c150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51817f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f517c1e20c0 0x7f517c150460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38182/0 (socket says 172.21.15.26:38182) 2023-12-19T18:18:16.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51817f0640 1 -- 172.21.15.26:0/59430285 learned_addr learned my addr 172.21.15.26:0/59430285 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:16.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51827f2640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f517c0b1790 0x7f517c0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f5181ff1640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51680047d0 0x7f517c1e2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51817f0640 1 -- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51680047d0 msgr2=0x7f517c1e2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51817f0640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51680047d0 0x7f517c1e2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51817f0640 1 -- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f517c0b1790 msgr2=0x7f517c0b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51817f0640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f517c0b1790 0x7f517c0b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51817f0640 1 -- 172.21.15.26:0/59430285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c150dd0 con 0x7f517c1e20c0 2023-12-19T18:18:16.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51827f2640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f517c0b1790 0x7f517c0b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:16.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51817f0640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f517c1e20c0 0x7f517c150460 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f516c001940 tx=0x7f5168005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51727fc640 1 -- 172.21.15.26:0/59430285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c001640 con 0x7f517c1e20c0 2023-12-19T18:18:16.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51727fc640 1 -- 172.21.15.26:0/59430285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c007070 con 0x7f517c1e20c0 2023-12-19T18:18:16.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51727fc640 1 -- 172.21.15.26:0/59430285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c007400 con 0x7f517c1e20c0 2023-12-19T18:18:16.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f5184255640 1 -- 172.21.15.26:0/59430285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517c1510b0 con 0x7f517c1e20c0 2023-12-19T18:18:16.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f5184255640 1 -- 172.21.15.26:0/59430285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517c151670 con 0x7f517c1e20c0 2023-12-19T18:18:16.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f5184255640 1 -- 172.21.15.26:0/59430285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517c074270 con 0x7f517c1e20c0 2023-12-19T18:18:16.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.752+0000 7f51727fc640 1 -- 172.21.15.26:0/59430285 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f516c0b9030 con 0x7f517c1e20c0 2023-12-19T18:18:16.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.756+0000 7f51727fc640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5160065cd0 0x7f5160068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.756+0000 7f51727fc640 1 -- 172.21.15.26:0/59430285 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f516c0b7070 con 0x7f517c1e20c0 2023-12-19T18:18:16.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.756+0000 7f5181ff1640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5160065cd0 0x7f5160068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.756+0000 7f5181ff1640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5160065cd0 0x7f5160068190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5178009ba0 tx=0x7f5178008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.760+0000 7f51727fc640 1 -- 172.21.15.26:0/59430285 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f516c0eda40 con 0x7f517c1e20c0 2023-12-19T18:18:16.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.916+0000 7f5184255640 1 -- 172.21.15.26:0/59430285 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f517c15a970 con 0x7f517c1e20c0 2023-12-19T18:18:16.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.916+0000 7f51727fc640 1 -- 172.21.15.26:0/59430285 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v250) v1 ==== 84+0+65 (secure 0 0 0) 0x7f516c0c1030 con 0x7f517c1e20c0 2023-12-19T18:18:16.918 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:16.918 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T18:18:16.919 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:16.919 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2023-12-19T18:18:16.919 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2023-12-19T18:18:16.919 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:16.919 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T18:18:16.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.920+0000 7f5184255640 1 -- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5160065cd0 msgr2=0x7f5160068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.920+0000 7f5184255640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5160065cd0 0x7f5160068190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5178009ba0 tx=0x7f5178008040 comp rx=0 tx=0).stop 2023-12-19T18:18:16.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.920+0000 7f5184255640 1 -- 172.21.15.26:0/59430285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f517c1e20c0 msgr2=0x7f517c150460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.920+0000 7f5184255640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f517c1e20c0 0x7f517c150460 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f516c001940 tx=0x7f5168005800 comp rx=0 tx=0).stop 2023-12-19T18:18:16.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.920+0000 7f5184255640 1 -- 172.21.15.26:0/59430285 shutdown_connections 2023-12-19T18:18:16.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.920+0000 7f5184255640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f5160065cd0 0x7f5160068190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.920+0000 7f5184255640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f517c0b1790 0x7f517c0b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.920+0000 7f5184255640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f517c1e20c0 0x7f517c150460 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.920+0000 7f5184255640 1 --2- 172.21.15.26:0/59430285 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f51680047d0 0x7f517c1e2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.920+0000 7f5184255640 1 -- 172.21.15.26:0/59430285 >> 172.21.15.26:0/59430285 conn(0x7f517c0658e0 msgr2=0x7f517c0705f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:16.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.920+0000 7f5184255640 1 -- 172.21.15.26:0/59430285 shutdown_connections 2023-12-19T18:18:16.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:16.920+0000 7f5184255640 1 -- 172.21.15.26:0/59430285 wait complete. 2023-12-19T18:18:16.939 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:18:16.940 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:18:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.084+0000 7f40bdedf640 1 Processor -- start 2023-12-19T18:18:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40bdedf640 1 -- start start 2023-12-19T18:18:17.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40bdedf640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f40b80c9310 0x7f40b80cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:17.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40bdedf640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40b80cb110 0x7f40b80cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:17.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40bdedf640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b80cbaa0 con 0x7f40b8159830 2023-12-19T18:18:17.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40bdedf640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b806f7c0 con 0x7f40b80c9310 2023-12-19T18:18:17.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40bdedf640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b80ca2b0 con 0x7f40b80cb110 2023-12-19T18:18:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b77fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f40b8159830 0x7f40b815d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36156/0 (socket says 172.21.15.26:36156) 2023-12-19T18:18:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b77fe640 1 -- 172.21.15.26:0/2028257446 learned_addr learned my addr 172.21.15.26:0/2028257446 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2136037852 0 0) 0x7f40a8001120 con 0x7f40b8159830 2023-12-19T18:18:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b7fff640 1 --2- 172.21.15.26:0/2028257446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40b80cb110 0x7f40b80cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b6ffd640 1 --2- 172.21.15.26:0/2028257446 >> v2:172.21.15.160:3300/0 conn(0x7f40b80c9310 0x7f40b80cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40a0005740 con 0x7f40b8159830 2023-12-19T18:18:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 165265184 0 0) 0x7f40a0005740 con 0x7f40b8159830 2023-12-19T18:18:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40b80cb110 msgr2=0x7f40b80cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 --2- 172.21.15.26:0/2028257446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40b80cb110 0x7f40b80cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 >> v2:172.21.15.160:3300/0 conn(0x7f40b80c9310 msgr2=0x7f40b80cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 --2- 172.21.15.26:0/2028257446 >> v2:172.21.15.160:3300/0 conn(0x7f40b80c9310 0x7f40b80cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b815e3d0 con 0x7f40b8159830 2023-12-19T18:18:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f40a8001810 con 0x7f40b8159830 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 >> v1:172.21.15.26:6789/0 conn(0x7f40b8159830 legacy=0x7f40b815d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 --2- 172.21.15.26:0/2028257446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40b80cb110 0x7f40a0005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 --2- 172.21.15.26:0/2028257446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40a0006110 0x7f40a00069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 --2- 172.21.15.26:0/2028257446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40a0006f20 0x7f40a00073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b806f7c0 con 0x7f40a0006f20 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b80ca2b0 con 0x7f40a0006110 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a000dbf0 con 0x7f40b80cb110 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b7fff640 1 --2- 172.21.15.26:0/2028257446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40a0006f20 0x7f40a00073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b6ffd640 1 --2- 172.21.15.26:0/2028257446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40a0006110 0x7f40a00069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b77fe640 1 --2- 172.21.15.26:0/2028257446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40b80cb110 0x7f40a0005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b7fff640 1 -- 172.21.15.26:0/2028257446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40b80cb110 msgr2=0x7f40a0005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b7fff640 1 --2- 172.21.15.26:0/2028257446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40b80cb110 0x7f40a0005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b7fff640 1 -- 172.21.15.26:0/2028257446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40a0006110 msgr2=0x7f40a00069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b7fff640 1 --2- 172.21.15.26:0/2028257446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40a0006110 0x7f40a00069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.088+0000 7f40b7fff640 1 -- 172.21.15.26:0/2028257446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a0005740 con 0x7f40a0006f20 2023-12-19T18:18:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.092+0000 7f40b7fff640 1 --2- 172.21.15.26:0/2028257446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40a0006f20 0x7f40a00073a0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f40b000e170 tx=0x7f40b001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676544833556892b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.092+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b00a5b40 con 0x7f40a0006f20 2023-12-19T18:18:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.092+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b000f040 con 0x7f40a0006f20 2023-12-19T18:18:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.092+0000 7f40b67fc640 1 -- 172.21.15.26:0/2028257446 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b00ad440 con 0x7f40a0006f20 2023-12-19T18:18:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.092+0000 7f40bdedf640 1 -- 172.21.15.26:0/2028257446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40a0006f20 msgr2=0x7f40a00073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.092+0000 7f40bdedf640 1 --2- 172.21.15.26:0/2028257446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40a0006f20 0x7f40a00073a0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f40b000e170 tx=0x7f40b001cc40 comp rx=0 tx=0).stop 2023-12-19T18:18:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.092+0000 7f40b7fff640 1 -- 172.21.15.26:0/2028257446 reap_dead start 2023-12-19T18:18:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.092+0000 7f40bdedf640 1 -- 172.21.15.26:0/2028257446 shutdown_connections 2023-12-19T18:18:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.092+0000 7f40bdedf640 1 -- 172.21.15.26:0/2028257446 >> 172.21.15.26:0/2028257446 conn(0x7f40b80b0270 msgr2=0x7f40b80b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.092+0000 7f40bdedf640 1 -- 172.21.15.26:0/2028257446 shutdown_connections 2023-12-19T18:18:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.092+0000 7f40bdedf640 1 -- 172.21.15.26:0/2028257446 wait complete. 2023-12-19T18:18:17.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40bdedf640 1 Processor -- start 2023-12-19T18:18:17.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40bdedf640 1 -- start start 2023-12-19T18:18:17.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40bdedf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40a0006f20 0x7f40b8151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:17.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40bdedf640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40b81535e0 0x7f40b8153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40bdedf640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b8154480 0x7f40b8157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40bdedf640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b81523d0 con 0x7f40a0006f20 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40bdedf640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b8152860 con 0x7f40b8154480 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40bdedf640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b8154990 con 0x7f40b81535e0 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40a0006f20 0x7f40b8151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40b77fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40a0006f20 0x7f40b8151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38200/0 (socket says 172.21.15.26:38200) 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40b77fe640 1 -- 172.21.15.26:0/3104559725 learned_addr learned my addr 172.21.15.26:0/3104559725 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40b6ffd640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40b81535e0 0x7f40b8153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40b7fff640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b8154480 0x7f40b8157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40b77fe640 1 -- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40b81535e0 msgr2=0x7f40b8153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40b77fe640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40b81535e0 0x7f40b8153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40b77fe640 1 -- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b8154480 msgr2=0x7f40b8157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40b77fe640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b8154480 0x7f40b8157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40b77fe640 1 -- 172.21.15.26:0/3104559725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b8152b00 con 0x7f40a0006f20 2023-12-19T18:18:17.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40b77fe640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40a0006f20 0x7f40b8151e90 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f40a8007fd0 tx=0x7f40a8012920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:17.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f409ffff640 1 -- 172.21.15.26:0/3104559725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a80148d0 con 0x7f40a0006f20 2023-12-19T18:18:17.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f409ffff640 1 -- 172.21.15.26:0/3104559725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a8014a30 con 0x7f40a0006f20 2023-12-19T18:18:17.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f409ffff640 1 -- 172.21.15.26:0/3104559725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8014dc0 con 0x7f40a0006f20 2023-12-19T18:18:17.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40bdedf640 1 -- 172.21.15.26:0/3104559725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b8152de0 con 0x7f40a0006f20 2023-12-19T18:18:17.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.096+0000 7f40bdedf640 1 -- 172.21.15.26:0/3104559725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b80c4a70 con 0x7f40a0006f20 2023-12-19T18:18:17.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.100+0000 7f40bdedf640 1 -- 172.21.15.26:0/3104559725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b8074270 con 0x7f40a0006f20 2023-12-19T18:18:17.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.100+0000 7f409ffff640 1 -- 172.21.15.26:0/3104559725 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f40a80153e0 con 0x7f40a0006f20 2023-12-19T18:18:17.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.100+0000 7f409ffff640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f40a4065d20 0x7f40a40681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:17.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.100+0000 7f409ffff640 1 -- 172.21.15.26:0/3104559725 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5617+0+0 (secure 0 0 0) 0x7f40a8086110 con 0x7f40a0006f20 2023-12-19T18:18:17.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.100+0000 7f40b6ffd640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f40a4065d20 0x7f40a40681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:17.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.100+0000 7f40b6ffd640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f40a4065d20 0x7f40a40681e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40b81588d0 tx=0x7f40ac009190 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:17.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.104+0000 7f409ffff640 1 -- 172.21.15.26:0/3104559725 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f40a8053850 con 0x7f40a0006f20 2023-12-19T18:18:17.262 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:17.260+0000 7f40bdedf640 1 -- 172.21.15.26:0/3104559725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f40b80cbaa0 con 0x7f40a0006f20 2023-12-19T18:18:18.143 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.144+0000 7f409ffff640 1 -- 172.21.15.26:0/3104559725 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v251) v1 ==== 138+0+0 (secure 0 0 0) 0x7f40b80cbaa0 con 0x7f40a0006f20 2023-12-19T18:18:18.143 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2023-12-19T18:18:18.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.144+0000 7f40bdedf640 1 -- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f40a4065d20 msgr2=0x7f40a40681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:18.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.148+0000 7f40bdedf640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f40a4065d20 0x7f40a40681e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40b81588d0 tx=0x7f40ac009190 comp rx=0 tx=0).stop 2023-12-19T18:18:18.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.148+0000 7f40bdedf640 1 -- 172.21.15.26:0/3104559725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40a0006f20 msgr2=0x7f40b8151e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:18.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.148+0000 7f40bdedf640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40a0006f20 0x7f40b8151e90 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f40a8007fd0 tx=0x7f40a8012920 comp rx=0 tx=0).stop 2023-12-19T18:18:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.148+0000 7f40bdedf640 1 -- 172.21.15.26:0/3104559725 shutdown_connections 2023-12-19T18:18:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.148+0000 7f40bdedf640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f40a4065d20 0x7f40a40681e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.148+0000 7f40bdedf640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f40b8154480 0x7f40b8157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.148+0000 7f40bdedf640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f40b81535e0 0x7f40b8153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.148+0000 7f40bdedf640 1 --2- 172.21.15.26:0/3104559725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40a0006f20 0x7f40b8151e90 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.148+0000 7f40bdedf640 1 -- 172.21.15.26:0/3104559725 >> 172.21.15.26:0/3104559725 conn(0x7f40b80b0270 msgr2=0x7f40b8070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.148+0000 7f40bdedf640 1 -- 172.21.15.26:0/3104559725 shutdown_connections 2023-12-19T18:18:18.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.148+0000 7f40bdedf640 1 -- 172.21.15.26:0/3104559725 wait complete. 2023-12-19T18:18:18.175 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:18:18.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.328+0000 7fec12004640 1 Processor -- start 2023-12-19T18:18:18.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.328+0000 7fec12004640 1 -- start start 2023-12-19T18:18:18.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec12004640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec0c06e3a0 0x7fec0c071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:18.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec12004640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec0c0b22f0 0x7fec0c0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:18.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec12004640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec0c0cc9f0 con 0x7fec0c0b5e50 2023-12-19T18:18:18.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec12004640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec0c0b2cc0 con 0x7fec0c0b22f0 2023-12-19T18:18:18.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec12004640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c0b6230 con 0x7fec0c06e3a0 2023-12-19T18:18:18.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec11002640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fec0c0b5e50 0x7fec0c0cc2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:44162/0 (socket says 172.21.15.26:44162) 2023-12-19T18:18:18.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec11002640 1 -- 172.21.15.26:0/4244044058 learned_addr learned my addr 172.21.15.26:0/4244044058 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:18.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec10801640 1 --2- 172.21.15.26:0/4244044058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec0c06e3a0 0x7fec0c071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:18.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec11803640 1 --2- 172.21.15.26:0/4244044058 >> v2:172.21.15.26:3300/0 conn(0x7fec0c0b22f0 0x7fec0c0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:18.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec03fff640 1 -- 172.21.15.26:0/4244044058 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400072605 0 0) 0x7fec08001120 con 0x7fec0c0b5e50 2023-12-19T18:18:18.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec03fff640 1 -- 172.21.15.26:0/4244044058 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febf8003530 con 0x7fec0c0b5e50 2023-12-19T18:18:18.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec10801640 1 -- 172.21.15.26:0/4244044058 >> v2:172.21.15.26:3300/0 conn(0x7fec0c0b22f0 msgr2=0x7fec0c0b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:18.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec10801640 1 --2- 172.21.15.26:0/4244044058 >> v2:172.21.15.26:3300/0 conn(0x7fec0c0b22f0 0x7fec0c0b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec10801640 1 -- 172.21.15.26:0/4244044058 >> v1:172.21.15.160:6789/0 conn(0x7fec0c0b5e50 legacy=0x7fec0c0cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:18.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.332+0000 7fec10801640 1 -- 172.21.15.26:0/4244044058 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c0ccc30 con 0x7fec0c06e3a0 2023-12-19T18:18:18.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec10801640 1 --2- 172.21.15.26:0/4244044058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec0c06e3a0 0x7fec0c071d90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7febfc0097c0 tx=0x7febfc01ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f93ea117988d6917 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:18.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec03fff640 1 -- 172.21.15.26:0/4244044058 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc0a8970 con 0x7fec0c06e3a0 2023-12-19T18:18:18.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec03fff640 1 -- 172.21.15.26:0/4244044058 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc0a8ad0 con 0x7fec0c06e3a0 2023-12-19T18:18:18.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec03fff640 1 -- 172.21.15.26:0/4244044058 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc0a8e60 con 0x7fec0c06e3a0 2023-12-19T18:18:18.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec12004640 1 -- 172.21.15.26:0/4244044058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec0c06e3a0 msgr2=0x7fec0c071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:18.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec12004640 1 --2- 172.21.15.26:0/4244044058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec0c06e3a0 0x7fec0c071d90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7febfc0097c0 tx=0x7febfc01ec70 comp rx=0 tx=0).stop 2023-12-19T18:18:18.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec12004640 1 -- 172.21.15.26:0/4244044058 shutdown_connections 2023-12-19T18:18:18.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec12004640 1 --2- 172.21.15.26:0/4244044058 >> v2:172.21.15.26:3300/0 conn(0x7fec0c0b22f0 0x7fec0c0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec12004640 1 --2- 172.21.15.26:0/4244044058 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec0c06e3a0 0x7fec0c071d90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec12004640 1 -- 172.21.15.26:0/4244044058 >> 172.21.15.26:0/4244044058 conn(0x7fec0c0cd2f0 msgr2=0x7fec0c0cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:18.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec12004640 1 -- 172.21.15.26:0/4244044058 shutdown_connections 2023-12-19T18:18:18.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec12004640 1 -- 172.21.15.26:0/4244044058 wait complete. 2023-12-19T18:18:18.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec12004640 1 Processor -- start 2023-12-19T18:18:18.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.336+0000 7fec12004640 1 -- start start 2023-12-19T18:18:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec12004640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec0c06e3a0 0x7fec0c0c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec12004640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0c0b22f0 0x7fec0c06b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec12004640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec0c0b5e50 0x7fec0c0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec12004640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c06a320 con 0x7fec0c0b22f0 2023-12-19T18:18:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec12004640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c0c7d20 con 0x7fec0c0b5e50 2023-12-19T18:18:18.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec12004640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c0c8000 con 0x7fec0c06e3a0 2023-12-19T18:18:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec10801640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0c0b22f0 0x7fec0c06b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec10801640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0c0b22f0 0x7fec0c06b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38214/0 (socket says 172.21.15.26:38214) 2023-12-19T18:18:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec10801640 1 -- 172.21.15.26:0/1643514613 learned_addr learned my addr 172.21.15.26:0/1643514613 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec11002640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec0c06e3a0 0x7fec0c0c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec11803640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec0c0b5e50 0x7fec0c0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec10801640 1 -- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec0c06e3a0 msgr2=0x7fec0c0c43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec10801640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec0c06e3a0 0x7fec0c0c43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec10801640 1 -- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec0c0b5e50 msgr2=0x7fec0c0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec10801640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec0c0b5e50 0x7fec0c0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec10801640 1 -- 172.21.15.26:0/1643514613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc009420 con 0x7fec0c0b22f0 2023-12-19T18:18:18.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec11002640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec0c06e3a0 0x7fec0c0c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec10801640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0c0b22f0 0x7fec0c06b5e0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7febfc0b3040 tx=0x7febfc01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec01ffb640 1 -- 172.21.15.26:0/1643514613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc0b0d40 con 0x7fec0c0b22f0 2023-12-19T18:18:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec01ffb640 1 -- 172.21.15.26:0/1643514613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc0b9410 con 0x7fec0c0b22f0 2023-12-19T18:18:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec01ffb640 1 -- 172.21.15.26:0/1643514613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc0b0ea0 con 0x7fec0c0b22f0 2023-12-19T18:18:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec12004640 1 -- 172.21.15.26:0/1643514613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c0c82e0 con 0x7fec0c0b22f0 2023-12-19T18:18:18.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec12004640 1 -- 172.21.15.26:0/1643514613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0c0c48e0 con 0x7fec0c0b22f0 2023-12-19T18:18:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.340+0000 7fec12004640 1 -- 172.21.15.26:0/1643514613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0c06a490 con 0x7fec0c0b22f0 2023-12-19T18:18:18.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.344+0000 7fec01ffb640 1 -- 172.21.15.26:0/1643514613 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7febfc0b7040 con 0x7fec0c0b22f0 2023-12-19T18:18:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.344+0000 7fec01ffb640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7febf0065cd0 0x7febf0068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.344+0000 7fec01ffb640 1 -- 172.21.15.26:0/1643514613 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5987+0+0 (secure 0 0 0) 0x7febfc121ac0 con 0x7fec0c0b22f0 2023-12-19T18:18:18.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.344+0000 7fec11002640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7febf0065cd0 0x7febf0068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:18.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.344+0000 7fec11002640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7febf0065cd0 0x7febf0068190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec08001280 tx=0x7fec08009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:18.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.348+0000 7fec01ffb640 1 -- 172.21.15.26:0/1643514613 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7febfc0eefe0 con 0x7fec0c0b22f0 2023-12-19T18:18:18.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:18.504+0000 7fec12004640 1 -- 172.21.15.26:0/1643514613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fec0c06a460 con 0x7fec0c0b22f0 2023-12-19T18:18:19.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.144+0000 7fec01ffb640 1 -- 172.21.15.26:0/1643514613 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v252) v1 ==== 170+0+0 (secure 0 0 0) 0x7febfc0b5020 con 0x7fec0c0b22f0 2023-12-19T18:18:19.147 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2023-12-19T18:18:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.152+0000 7fec12004640 1 -- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7febf0065cd0 msgr2=0x7febf0068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.152+0000 7fec12004640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7febf0065cd0 0x7febf0068190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec08001280 tx=0x7fec08009040 comp rx=0 tx=0).stop 2023-12-19T18:18:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.152+0000 7fec12004640 1 -- 172.21.15.26:0/1643514613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0c0b22f0 msgr2=0x7fec0c06b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.152+0000 7fec12004640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0c0b22f0 0x7fec0c06b5e0 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7febfc0b3040 tx=0x7febfc01ba10 comp rx=0 tx=0).stop 2023-12-19T18:18:19.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.152+0000 7fec12004640 1 -- 172.21.15.26:0/1643514613 shutdown_connections 2023-12-19T18:18:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.152+0000 7fec12004640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7febf0065cd0 0x7febf0068190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.152+0000 7fec12004640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fec0c0b5e50 0x7fec0c0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.152+0000 7fec12004640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec0c0b22f0 0x7fec0c06b5e0 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.152+0000 7fec12004640 1 --2- 172.21.15.26:0/1643514613 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fec0c06e3a0 0x7fec0c0c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.152+0000 7fec12004640 1 -- 172.21.15.26:0/1643514613 >> 172.21.15.26:0/1643514613 conn(0x7fec0c0cd2f0 msgr2=0x7fec0c153ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.156+0000 7fec12004640 1 -- 172.21.15.26:0/1643514613 shutdown_connections 2023-12-19T18:18:19.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.156+0000 7fec12004640 1 -- 172.21.15.26:0/1643514613 wait complete. 2023-12-19T18:18:19.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:18:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.344+0000 7fce38e5f640 1 Processor -- start 2023-12-19T18:18:19.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.344+0000 7fce38e5f640 1 -- start start 2023-12-19T18:18:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.344+0000 7fce38e5f640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce34159820 0x7fce3415df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.344+0000 7fce38e5f640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3406fb60 con 0x7fce34159c00 2023-12-19T18:18:19.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.344+0000 7fce38e5f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3406fe40 con 0x7fce3406f6c0 2023-12-19T18:18:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.344+0000 7fce38e5f640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3415e470 con 0x7fce34159820 2023-12-19T18:18:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.344+0000 7fce3259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce34159820 0x7fce3415df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.344+0000 7fce3259c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce34159820 0x7fce3415df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38222/0 (socket says 172.21.15.26:38222) 2023-12-19T18:18:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.344+0000 7fce3259c640 1 -- 172.21.15.26:0/4252122447 learned_addr learned my addr 172.21.15.26:0/4252122447 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.344+0000 7fce3259c640 1 -- 172.21.15.26:0/4252122447 >> v1:172.21.15.160:6789/0 conn(0x7fce3406f6c0 legacy=0x7fce340c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.344+0000 7fce3259c640 1 -- 172.21.15.26:0/4252122447 >> v1:172.21.15.26:6789/0 conn(0x7fce34159c00 legacy=0x7fce34070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.344+0000 7fce3259c640 1 -- 172.21.15.26:0/4252122447 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34070120 con 0x7fce34159820 2023-12-19T18:18:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.348+0000 7fce3259c640 1 --2- 172.21.15.26:0/4252122447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce34159820 0x7fce3415df30 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fce24009870 tx=0x7fce2401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5675787d1acac94 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.348+0000 7fce31d9b640 1 -- 172.21.15.26:0/4252122447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce240aa670 con 0x7fce34159820 2023-12-19T18:18:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.348+0000 7fce31d9b640 1 -- 172.21.15.26:0/4252122447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce240aa7d0 con 0x7fce34159820 2023-12-19T18:18:19.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.348+0000 7fce31d9b640 1 -- 172.21.15.26:0/4252122447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce240aab60 con 0x7fce34159820 2023-12-19T18:18:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.348+0000 7fce38e5f640 1 -- 172.21.15.26:0/4252122447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce34159820 msgr2=0x7fce3415df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.348+0000 7fce38e5f640 1 --2- 172.21.15.26:0/4252122447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce34159820 0x7fce3415df30 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7fce24009870 tx=0x7fce2401c510 comp rx=0 tx=0).stop 2023-12-19T18:18:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.348+0000 7fce38e5f640 1 -- 172.21.15.26:0/4252122447 shutdown_connections 2023-12-19T18:18:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.348+0000 7fce38e5f640 1 --2- 172.21.15.26:0/4252122447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce34159820 0x7fce3415df30 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.348+0000 7fce38e5f640 1 -- 172.21.15.26:0/4252122447 >> 172.21.15.26:0/4252122447 conn(0x7fce340ce730 msgr2=0x7fce340ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.348+0000 7fce38e5f640 1 -- 172.21.15.26:0/4252122447 shutdown_connections 2023-12-19T18:18:19.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.348+0000 7fce38e5f640 1 -- 172.21.15.26:0/4252122447 wait complete. 2023-12-19T18:18:19.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.348+0000 7fce38e5f640 1 Processor -- start 2023-12-19T18:18:19.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.348+0000 7fce38e5f640 1 -- start start 2023-12-19T18:18:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce38e5f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce3406f6c0 0x7fce340b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce38e5f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce34159820 0x7fce34150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce38e5f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce34159c00 0x7fce340c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce38e5f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce340cb410 con 0x7fce34159c00 2023-12-19T18:18:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce38e5f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce34150660 con 0x7fce3406f6c0 2023-12-19T18:18:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce38e5f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce34158860 con 0x7fce34159820 2023-12-19T18:18:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce32d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce34159c00 0x7fce340c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce3259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce3406f6c0 0x7fce340b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce3259c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce3406f6c0 0x7fce340b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:53348/0 (socket says 172.21.15.26:53348) 2023-12-19T18:18:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce3259c640 1 -- 172.21.15.26:0/572324909 learned_addr learned my addr 172.21.15.26:0/572324909 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:19.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce32d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce34159c00 0x7fce340c26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38228/0 (socket says 172.21.15.26:38228) 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce2bfff640 1 --2- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce34159820 0x7fce34150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce3259c640 1 -- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce34159820 msgr2=0x7fce34150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce3259c640 1 --2- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce34159820 0x7fce34150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce3259c640 1 -- 172.21.15.26:0/572324909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce34159c00 msgr2=0x7fce340c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce3259c640 1 --2- 172.21.15.26:0/572324909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce34159c00 0x7fce340c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce3259c640 1 -- 172.21.15.26:0/572324909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce240094d0 con 0x7fce3406f6c0 2023-12-19T18:18:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce3259c640 1 --2- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce3406f6c0 0x7fce340b1320 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fce24019120 tx=0x7fce2401ad50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce2affd640 1 -- 172.21.15.26:0/572324909 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce240b38c0 con 0x7fce3406f6c0 2023-12-19T18:18:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce2affd640 1 -- 172.21.15.26:0/572324909 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce240b3a20 con 0x7fce3406f6c0 2023-12-19T18:18:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce2affd640 1 -- 172.21.15.26:0/572324909 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce240b3dd0 con 0x7fce3406f6c0 2023-12-19T18:18:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce38e5f640 1 -- 172.21.15.26:0/572324909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3406b880 con 0x7fce3406f6c0 2023-12-19T18:18:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce38e5f640 1 -- 172.21.15.26:0/572324909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce34158ab0 con 0x7fce3406f6c0 2023-12-19T18:18:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.352+0000 7fce38e5f640 1 -- 172.21.15.26:0/572324909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce34074270 con 0x7fce3406f6c0 2023-12-19T18:18:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.356+0000 7fce2affd640 1 -- 172.21.15.26:0/572324909 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fce240b2380 con 0x7fce3406f6c0 2023-12-19T18:18:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.356+0000 7fce2affd640 1 --2- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fce14065ca0 0x7fce14068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.356 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.356+0000 7fce2affd640 1 -- 172.21.15.26:0/572324909 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6369+0+0 (secure 0 0 0) 0x7fce2411e2b0 con 0x7fce3406f6c0 2023-12-19T18:18:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.356+0000 7fce2bfff640 1 --2- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fce14065ca0 0x7fce14068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.357 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.356+0000 7fce2bfff640 1 --2- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fce14065ca0 0x7fce14068160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fce341f8e10 tx=0x7fce1c009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:19.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.360+0000 7fce2affd640 1 -- 172.21.15.26:0/572324909 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fce240eb570 con 0x7fce3406f6c0 2023-12-19T18:18:19.552 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.552+0000 7fce38e5f640 1 -- 172.21.15.26:0/572324909 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fce340b46b0 con 0x7fce3406f6c0 2023-12-19T18:18:19.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.564+0000 7fce2affd640 1 -- 172.21.15.26:0/572324909 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 32 and data pool 33 v124) v1 ==== 174+0+0 (secure 0 0 0) 0x7fce240eeb90 con 0x7fce3406f6c0 2023-12-19T18:18:19.570 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 32 and data pool 33 2023-12-19T18:18:19.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.572+0000 7fce38e5f640 1 -- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fce14065ca0 msgr2=0x7fce14068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.572+0000 7fce38e5f640 1 --2- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fce14065ca0 0x7fce14068160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fce341f8e10 tx=0x7fce1c009040 comp rx=0 tx=0).stop 2023-12-19T18:18:19.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.572+0000 7fce38e5f640 1 -- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce3406f6c0 msgr2=0x7fce340b1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.572+0000 7fce38e5f640 1 --2- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce3406f6c0 0x7fce340b1320 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fce24019120 tx=0x7fce2401ad50 comp rx=0 tx=0).stop 2023-12-19T18:18:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.572+0000 7fce38e5f640 1 -- 172.21.15.26:0/572324909 shutdown_connections 2023-12-19T18:18:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.572+0000 7fce38e5f640 1 --2- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fce14065ca0 0x7fce14068160 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fce341f8e10 tx=0x7fce1c009040 comp rx=0 tx=0).stop 2023-12-19T18:18:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.572+0000 7fce38e5f640 1 --2- 172.21.15.26:0/572324909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce34159c00 0x7fce340c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.575 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.572+0000 7fce38e5f640 1 --2- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fce34159820 0x7fce34150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.572+0000 7fce38e5f640 1 --2- 172.21.15.26:0/572324909 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce3406f6c0 0x7fce340b1320 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.572+0000 7fce38e5f640 1 -- 172.21.15.26:0/572324909 >> 172.21.15.26:0/572324909 conn(0x7fce340ce730 msgr2=0x7fce34067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:19.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.576+0000 7fce38e5f640 1 -- 172.21.15.26:0/572324909 shutdown_connections 2023-12-19T18:18:19.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.576+0000 7fce38e5f640 1 -- 172.21.15.26:0/572324909 wait complete. 2023-12-19T18:18:19.613 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:19.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.756+0000 7f71d1266640 1 Processor -- start 2023-12-19T18:18:19.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.756+0000 7f71d1266640 1 -- start start 2023-12-19T18:18:19.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.756+0000 7f71d1266640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f71cc155030 0x7f71cc15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.756+0000 7f71d1266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71cc15f370 0x7f71cc06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.756+0000 7f71d1266640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f71cc0ce740 0x7f71cc0cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.756+0000 7f71d1266640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71cc0cf130 con 0x7f71cc0ce740 2023-12-19T18:18:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.756+0000 7f71d1266640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71cc06c970 con 0x7f71cc155030 2023-12-19T18:18:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.756+0000 7f71d1266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc15b1f0 con 0x7f71cc15f370 2023-12-19T18:18:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.756+0000 7f71cb7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f71cc0ce740 0x7f71cc0cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.756+0000 7f71cb7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f71cc0ce740 0x7f71cc0cebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38244/0 (socket says 172.21.15.26:38244) 2023-12-19T18:18:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.756+0000 7f71cb7fe640 1 -- 172.21.15.26:0/3711159394 learned_addr learned my addr 172.21.15.26:0/3711159394 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:19.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.756+0000 7f71caffd640 1 --2- 172.21.15.26:0/3711159394 >> v2:172.21.15.160:3300/0 conn(0x7f71cc155030 0x7f71cc15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.756+0000 7f71c3fff640 1 --2- 172.21.15.26:0/3711159394 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71cc15f370 0x7f71cc06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.760+0000 7f71cb7fe640 1 -- 172.21.15.26:0/3711159394 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71cc15f370 msgr2=0x7f71cc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.760+0000 7f71cb7fe640 1 --2- 172.21.15.26:0/3711159394 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71cc15f370 0x7f71cc06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.760+0000 7f71cb7fe640 1 -- 172.21.15.26:0/3711159394 >> v2:172.21.15.160:3300/0 conn(0x7f71cc155030 msgr2=0x7f71cc15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.760+0000 7f71cb7fe640 1 --2- 172.21.15.26:0/3711159394 >> v2:172.21.15.160:3300/0 conn(0x7f71cc155030 0x7f71cc15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.760+0000 7f71cb7fe640 1 -- 172.21.15.26:0/3711159394 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc15b4d0 con 0x7f71cc0ce740 2023-12-19T18:18:19.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.760+0000 7f71cb7fe640 1 --2- 172.21.15.26:0/3711159394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71cc0ce740 0x7f71cc0cebf0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0cf4a0 tx=0x7f71c401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9404fc0ad66cdf8e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:19.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.760+0000 7f71ca7fc640 1 -- 172.21.15.26:0/3711159394 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c40a5840 con 0x7f71cc0ce740 2023-12-19T18:18:19.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.760+0000 7f71ca7fc640 1 -- 172.21.15.26:0/3711159394 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c40a59a0 con 0x7f71cc0ce740 2023-12-19T18:18:19.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.760+0000 7f71ca7fc640 1 -- 172.21.15.26:0/3711159394 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c40a5d30 con 0x7f71cc0ce740 2023-12-19T18:18:19.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 -- 172.21.15.26:0/3711159394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71cc0ce740 msgr2=0x7f71cc0cebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 --2- 172.21.15.26:0/3711159394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71cc0ce740 0x7f71cc0cebf0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0cf4a0 tx=0x7f71c401c9a0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 -- 172.21.15.26:0/3711159394 shutdown_connections 2023-12-19T18:18:19.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 --2- 172.21.15.26:0/3711159394 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71cc0ce740 0x7f71cc0cebf0 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 --2- 172.21.15.26:0/3711159394 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71cc15f370 0x7f71cc06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 --2- 172.21.15.26:0/3711159394 >> v2:172.21.15.160:3300/0 conn(0x7f71cc155030 0x7f71cc15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 -- 172.21.15.26:0/3711159394 >> 172.21.15.26:0/3711159394 conn(0x7f71cc0658e0 msgr2=0x7f71cc065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 -- 172.21.15.26:0/3711159394 shutdown_connections 2023-12-19T18:18:19.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 -- 172.21.15.26:0/3711159394 wait complete. 2023-12-19T18:18:19.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 Processor -- start 2023-12-19T18:18:19.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 -- start start 2023-12-19T18:18:19.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71cc0ce740 0x7f71cc1463e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71cc155030 0x7f71cc15c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71cc15f370 0x7f71cc15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc071420 con 0x7f71cc15f370 2023-12-19T18:18:19.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc15d050 con 0x7f71cc0ce740 2023-12-19T18:18:19.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71d1266640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc1f5190 con 0x7f71cc155030 2023-12-19T18:18:19.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71cc15f370 0x7f71cc15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71cc15f370 0x7f71cc15c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38256/0 (socket says 172.21.15.26:38256) 2023-12-19T18:18:19.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71cb7fe640 1 -- 172.21.15.26:0/2762553955 learned_addr learned my addr 172.21.15.26:0/2762553955 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:19.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71caffd640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71cc0ce740 0x7f71cc1463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71c3fff640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71cc155030 0x7f71cc15c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71cb7fe640 1 -- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71cc155030 msgr2=0x7f71cc15c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71cb7fe640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71cc155030 0x7f71cc15c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71cb7fe640 1 -- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71cc0ce740 msgr2=0x7f71cc1463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71cb7fe640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71cc0ce740 0x7f71cc1463e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71cb7fe640 1 -- 172.21.15.26:0/2762553955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c4009b30 con 0x7f71cc15f370 2023-12-19T18:18:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.764+0000 7f71cb7fe640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71cc15f370 0x7f71cc15c9d0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f71c401b050 tx=0x7f71c4019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:19.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.768+0000 7f71c37fe640 1 -- 172.21.15.26:0/2762553955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c40b8670 con 0x7f71cc15f370 2023-12-19T18:18:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.768+0000 7f71c37fe640 1 -- 172.21.15.26:0/2762553955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c40b87d0 con 0x7f71cc15f370 2023-12-19T18:18:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.768+0000 7f71d1266640 1 -- 172.21.15.26:0/2762553955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71cc06c740 con 0x7f71cc15f370 2023-12-19T18:18:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.768+0000 7f71c37fe640 1 -- 172.21.15.26:0/2762553955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c40b8b80 con 0x7f71cc15f370 2023-12-19T18:18:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.768+0000 7f71d1266640 1 -- 172.21.15.26:0/2762553955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71cc1f5750 con 0x7f71cc15f370 2023-12-19T18:18:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.768+0000 7f71d1266640 1 -- 172.21.15.26:0/2762553955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71cc074270 con 0x7f71cc15f370 2023-12-19T18:18:19.772 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.768+0000 7f71c37fe640 1 -- 172.21.15.26:0/2762553955 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f71c40b8ce0 con 0x7f71cc15f370 2023-12-19T18:18:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.768+0000 7f71c37fe640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71ac065ca0 0x7f71ac068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.768+0000 7f71c37fe640 1 -- 172.21.15.26:0/2762553955 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f71c411f580 con 0x7f71cc15f370 2023-12-19T18:18:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.768+0000 7f71caffd640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71ac065ca0 0x7f71ac068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.772+0000 7f71caffd640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71ac065ca0 0x7f71ac068160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009ac0 tx=0x7f71bc009190 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.772+0000 7f71c37fe640 1 -- 172.21.15.26:0/2762553955 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f71c40ecb40 con 0x7f71cc15f370 2023-12-19T18:18:19.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.932+0000 7f71d1266640 1 -- 172.21.15.26:0/2762553955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71cc0ccc30 con 0x7f71cc15f370 2023-12-19T18:18:19.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.936+0000 7f71c37fe640 1 -- 172.21.15.26:0/2762553955 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v253) v1 ==== 74+0+17218 (secure 0 0 0) 0x7f71c4009510 con 0x7f71cc15f370 2023-12-19T18:18:19.935 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:19.936 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":253,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:18:19.555012+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:17.314988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:18.558040+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:19.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.936+0000 7f71d1266640 1 -- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71ac065ca0 msgr2=0x7f71ac068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.936+0000 7f71d1266640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71ac065ca0 0x7f71ac068160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009ac0 tx=0x7f71bc009190 comp rx=0 tx=0).stop 2023-12-19T18:18:19.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.936+0000 7f71d1266640 1 -- 172.21.15.26:0/2762553955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71cc15f370 msgr2=0x7f71cc15c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.936+0000 7f71d1266640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71cc15f370 0x7f71cc15c9d0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7f71c401b050 tx=0x7f71c4019740 comp rx=0 tx=0).stop 2023-12-19T18:18:19.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.936+0000 7f71d1266640 1 -- 172.21.15.26:0/2762553955 shutdown_connections 2023-12-19T18:18:19.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.936+0000 7f71d1266640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f71ac065ca0 0x7f71ac068160 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.936+0000 7f71d1266640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71cc15f370 0x7f71cc15c9d0 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.936+0000 7f71d1266640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f71cc155030 0x7f71cc15c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.936+0000 7f71d1266640 1 --2- 172.21.15.26:0/2762553955 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71cc0ce740 0x7f71cc1463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.936+0000 7f71d1266640 1 -- 172.21.15.26:0/2762553955 >> 172.21.15.26:0/2762553955 conn(0x7f71cc0658e0 msgr2=0x7f71cc14ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:19.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.936+0000 7f71d1266640 1 -- 172.21.15.26:0/2762553955 shutdown_connections 2023-12-19T18:18:19.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:19.936+0000 7f71d1266640 1 -- 172.21.15.26:0/2762553955 wait complete. 2023-12-19T18:18:19.958 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:20.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.100+0000 7f1edf455640 1 Processor -- start 2023-12-19T18:18:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.100+0000 7f1edf455640 1 -- start start 2023-12-19T18:18:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.100+0000 7f1edf455640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1ed806b3d0 0x7f1ed80ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.100+0000 7f1edf455640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ed80cec40 0x7f1ed815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.100+0000 7f1edf455640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed806bc30 con 0x7f1ed8154220 2023-12-19T18:18:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.100+0000 7f1edf455640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed80cf0d0 con 0x7f1ed80cec40 2023-12-19T18:18:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.100+0000 7f1edf455640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed80cc6e0 con 0x7f1ed806b3d0 2023-12-19T18:18:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.104+0000 7f1edd9f2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ed80cec40 0x7f1ed815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.104+0000 7f1edd9f2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ed80cec40 0x7f1ed815b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38270/0 (socket says 172.21.15.26:38270) 2023-12-19T18:18:20.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.104+0000 7f1edd9f2640 1 -- 172.21.15.26:0/3246601938 learned_addr learned my addr 172.21.15.26:0/3246601938 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:20.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.104+0000 7f1edc9f0640 1 --2- 172.21.15.26:0/3246601938 >> v2:172.21.15.160:3300/0 conn(0x7f1ed806b3d0 0x7f1ed80ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.104+0000 7f1edd9f2640 1 -- 172.21.15.26:0/3246601938 >> v2:172.21.15.160:3300/0 conn(0x7f1ed806b3d0 msgr2=0x7f1ed80ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:20.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.104+0000 7f1edd9f2640 1 --2- 172.21.15.26:0/3246601938 >> v2:172.21.15.160:3300/0 conn(0x7f1ed806b3d0 0x7f1ed80ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.104+0000 7f1edd9f2640 1 -- 172.21.15.26:0/3246601938 >> v1:172.21.15.26:6789/0 conn(0x7f1ed8154220 legacy=0x7f1ed815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.104+0000 7f1edd9f2640 1 -- 172.21.15.26:0/3246601938 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed806b950 con 0x7f1ed80cec40 2023-12-19T18:18:20.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.104+0000 7f1ecffff640 1 -- 172.21.15.26:0/3246601938 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993334176 0 0) 0x7f1ed806bc30 con 0x7f1ed8154220 2023-12-19T18:18:20.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.104+0000 7f1edd9f2640 1 --2- 172.21.15.26:0/3246601938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed80cec40 0x7f1ed815b1c0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80cf880 tx=0x7f1ed001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ceb5d58ace9046 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:20.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.104+0000 7f1ecffff640 1 -- 172.21.15.26:0/3246601938 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00a5860 con 0x7f1ed80cec40 2023-12-19T18:18:20.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.104+0000 7f1ecffff640 1 -- 172.21.15.26:0/3246601938 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed00a59c0 con 0x7f1ed80cec40 2023-12-19T18:18:20.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.104+0000 7f1ecffff640 1 -- 172.21.15.26:0/3246601938 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00a5d50 con 0x7f1ed80cec40 2023-12-19T18:18:20.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 -- 172.21.15.26:0/3246601938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed80cec40 msgr2=0x7f1ed815b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 --2- 172.21.15.26:0/3246601938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed80cec40 0x7f1ed815b1c0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80cf880 tx=0x7f1ed001c9a0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 -- 172.21.15.26:0/3246601938 shutdown_connections 2023-12-19T18:18:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 --2- 172.21.15.26:0/3246601938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed80cec40 0x7f1ed815b1c0 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 --2- 172.21.15.26:0/3246601938 >> v2:172.21.15.160:3300/0 conn(0x7f1ed806b3d0 0x7f1ed80ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 -- 172.21.15.26:0/3246601938 >> 172.21.15.26:0/3246601938 conn(0x7f1ed80658e0 msgr2=0x7f1ed8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 -- 172.21.15.26:0/3246601938 shutdown_connections 2023-12-19T18:18:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 -- 172.21.15.26:0/3246601938 wait complete. 2023-12-19T18:18:20.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 Processor -- start 2023-12-19T18:18:20.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 -- start start 2023-12-19T18:18:20.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed806b3d0 0x7f1ed8158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed80cec40 0x7f1ed814b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed8154220 0x7f1ed814beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed81f5260 con 0x7f1ed80cec40 2023-12-19T18:18:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80b19a0 con 0x7f1ed8154220 2023-12-19T18:18:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edf455640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80b1670 con 0x7f1ed806b3d0 2023-12-19T18:18:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edc9f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed80cec40 0x7f1ed814b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edc9f0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed80cec40 0x7f1ed814b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38274/0 (socket says 172.21.15.26:38274) 2023-12-19T18:18:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edc9f0640 1 -- 172.21.15.26:0/144803660 learned_addr learned my addr 172.21.15.26:0/144803660 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edd1f1640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed806b3d0 0x7f1ed8158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.108+0000 7f1edd9f2640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed8154220 0x7f1ed814beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1edc9f0640 1 -- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed806b3d0 msgr2=0x7f1ed8158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1edc9f0640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed806b3d0 0x7f1ed8158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1edc9f0640 1 -- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed8154220 msgr2=0x7f1ed814beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1edc9f0640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed8154220 0x7f1ed814beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1edc9f0640 1 -- 172.21.15.26:0/144803660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed0009b30 con 0x7f1ed80cec40 2023-12-19T18:18:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1edc9f0640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed80cec40 0x7f1ed814b970 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800e870 tx=0x7f1ec800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1ecdffb640 1 -- 172.21.15.26:0/144803660 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8009510 con 0x7f1ed80cec40 2023-12-19T18:18:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1ecdffb640 1 -- 172.21.15.26:0/144803660 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec800eec0 con 0x7f1ed80cec40 2023-12-19T18:18:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1edf455640 1 -- 172.21.15.26:0/144803660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed8158e80 con 0x7f1ed80cec40 2023-12-19T18:18:20.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1ecdffb640 1 -- 172.21.15.26:0/144803660 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800cae0 con 0x7f1ed80cec40 2023-12-19T18:18:20.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1edf455640 1 -- 172.21.15.26:0/144803660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed8159440 con 0x7f1ed80cec40 2023-12-19T18:18:20.114 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1edf455640 1 -- 172.21.15.26:0/144803660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed8074270 con 0x7f1ed80cec40 2023-12-19T18:18:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1ecdffb640 1 -- 172.21.15.26:0/144803660 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f1ec800cc40 con 0x7f1ed80cec40 2023-12-19T18:18:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1ecdffb640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ebc065ca0 0x7f1ebc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.112+0000 7f1ecdffb640 1 -- 172.21.15.26:0/144803660 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f1ec80839a0 con 0x7f1ed80cec40 2023-12-19T18:18:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.116+0000 7f1edd1f1640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ebc065ca0 0x7f1ebc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.116+0000 7f1edd1f1640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ebc065ca0 0x7f1ebc068160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4000f70 tx=0x7f1ed4009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:20.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.116+0000 7f1ecdffb640 1 -- 172.21.15.26:0/144803660 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1ec8050d10 con 0x7f1ed80cec40 2023-12-19T18:18:20.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.276+0000 7f1edf455640 1 -- 172.21.15.26:0/144803660 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ed80b17b0 con 0x7f1ed80cec40 2023-12-19T18:18:20.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.276+0000 7f1ecdffb640 1 -- 172.21.15.26:0/144803660 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v253) v1 ==== 74+0+17218 (secure 0 0 0) 0x7f1ec8054330 con 0x7f1ed80cec40 2023-12-19T18:18:20.279 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:20.279 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":253,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:18:19.555012+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:17.314988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:18.558040+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.280+0000 7f1edf455640 1 -- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ebc065ca0 msgr2=0x7f1ebc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.280+0000 7f1edf455640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ebc065ca0 0x7f1ebc068160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4000f70 tx=0x7f1ed4009040 comp rx=0 tx=0).stop 2023-12-19T18:18:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.280+0000 7f1edf455640 1 -- 172.21.15.26:0/144803660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed80cec40 msgr2=0x7f1ed814b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.280+0000 7f1edf455640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed80cec40 0x7f1ed814b970 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800e870 tx=0x7f1ec800ec70 comp rx=0 tx=0).stop 2023-12-19T18:18:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.280+0000 7f1edf455640 1 -- 172.21.15.26:0/144803660 shutdown_connections 2023-12-19T18:18:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.280+0000 7f1edf455640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1ebc065ca0 0x7f1ebc068160 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.280+0000 7f1edf455640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed8154220 0x7f1ed814beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.280+0000 7f1edf455640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ed80cec40 0x7f1ed814b970 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.280+0000 7f1edf455640 1 --2- 172.21.15.26:0/144803660 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1ed806b3d0 0x7f1ed8158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.280+0000 7f1edf455640 1 -- 172.21.15.26:0/144803660 >> 172.21.15.26:0/144803660 conn(0x7f1ed80658e0 msgr2=0x7f1ed81534e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:20.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.280+0000 7f1edf455640 1 -- 172.21.15.26:0/144803660 shutdown_connections 2023-12-19T18:18:20.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.280+0000 7f1edf455640 1 -- 172.21.15.26:0/144803660 wait complete. 2023-12-19T18:18:20.301 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:18:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7acd073640 1 Processor -- start 2023-12-19T18:18:20.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7acd073640 1 -- start start 2023-12-19T18:18:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7acd073640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ac8145ed0 0x7f7ac80ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7acd073640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7ac80ad980 0x7f7ac80ca250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7acd073640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac80cd560 con 0x7f7ac80cd100 2023-12-19T18:18:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7acd073640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac80ae260 con 0x7f7ac80ad980 2023-12-19T18:18:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7acd073640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80adf80 con 0x7f7ac8145ed0 2023-12-19T18:18:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7abffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7ac80cd100 0x7f7ac80ad270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36186/0 (socket says 172.21.15.26:36186) 2023-12-19T18:18:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7abffff640 1 -- 172.21.15.26:0/4094371243 learned_addr learned my addr 172.21.15.26:0/4094371243 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac759e640 1 --2- 172.21.15.26:0/4094371243 >> v2:172.21.15.160:3300/0 conn(0x7f7ac80ad980 0x7f7ac80ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac6d9d640 1 --2- 172.21.15.26:0/4094371243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ac8145ed0 0x7f7ac80ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632578608 0 0) 0x7f7ac80cd560 con 0x7f7ac80cd100 2023-12-19T18:18:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab4003530 con 0x7f7ac80cd100 2023-12-19T18:18:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2119493467 0 0) 0x7f7ab4003530 con 0x7f7ac80cd100 2023-12-19T18:18:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ac8145ed0 msgr2=0x7f7ac80ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 --2- 172.21.15.26:0/4094371243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ac8145ed0 0x7f7ac80ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 >> v2:172.21.15.160:3300/0 conn(0x7f7ac80ad980 msgr2=0x7f7ac80ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 --2- 172.21.15.26:0/4094371243 >> v2:172.21.15.160:3300/0 conn(0x7f7ac80ad980 0x7f7ac80ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80c9770 con 0x7f7ac80cd100 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f7ab0001680 con 0x7f7ac80cd100 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 >> v1:172.21.15.26:6789/0 conn(0x7f7ac80cd100 legacy=0x7f7ac80ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 --2- 172.21.15.26:0/4094371243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003980 0x7f7ab4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 --2- 172.21.15.26:0/4094371243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab40047d0 0x7f7ab4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 --2- 172.21.15.26:0/4094371243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ac8145ed0 0x7f7ab4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80ae260 con 0x7f7ab4003980 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80adf80 con 0x7f7ab40047d0 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.448+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab400ba10 con 0x7f7ac8145ed0 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.452+0000 7f7ac6d9d640 1 --2- 172.21.15.26:0/4094371243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003980 0x7f7ab4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.452+0000 7f7ac759e640 1 --2- 172.21.15.26:0/4094371243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ac8145ed0 0x7f7ab4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.452+0000 7f7abffff640 1 --2- 172.21.15.26:0/4094371243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab40047d0 0x7f7ab4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.452+0000 7f7ac6d9d640 1 -- 172.21.15.26:0/4094371243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ac8145ed0 msgr2=0x7f7ab4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.452+0000 7f7ac6d9d640 1 --2- 172.21.15.26:0/4094371243 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ac8145ed0 0x7f7ab4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.452+0000 7f7ac6d9d640 1 -- 172.21.15.26:0/4094371243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab40047d0 msgr2=0x7f7ab4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.452+0000 7f7ac6d9d640 1 --2- 172.21.15.26:0/4094371243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab40047d0 0x7f7ab4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.452+0000 7f7ac6d9d640 1 -- 172.21.15.26:0/4094371243 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab4003530 con 0x7f7ab4003980 2023-12-19T18:18:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.452+0000 7f7abffff640 1 --2- 172.21.15.26:0/4094371243 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab40047d0 0x7f7ab4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.452+0000 7f7ac6d9d640 1 --2- 172.21.15.26:0/4094371243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003980 0x7f7ab4003da0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009ae0 tx=0x7f7ab801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c207f32c27d42d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.452+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80a6510 con 0x7f7ab4003980 2023-12-19T18:18:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.452+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab80a6670 con 0x7f7ab4003980 2023-12-19T18:18:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.452+0000 7f7ac659c640 1 -- 172.21.15.26:0/4094371243 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80a6a20 con 0x7f7ab4003980 2023-12-19T18:18:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 -- 172.21.15.26:0/4094371243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003980 msgr2=0x7f7ab4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 --2- 172.21.15.26:0/4094371243 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ab4003980 0x7f7ab4003da0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8009ae0 tx=0x7f7ab801ca20 comp rx=0 tx=0).stop 2023-12-19T18:18:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7ac759e640 1 -- 172.21.15.26:0/4094371243 reap_dead start 2023-12-19T18:18:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 -- 172.21.15.26:0/4094371243 shutdown_connections 2023-12-19T18:18:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 -- 172.21.15.26:0/4094371243 >> 172.21.15.26:0/4094371243 conn(0x7f7ac80b0b60 msgr2=0x7f7ac80b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 -- 172.21.15.26:0/4094371243 shutdown_connections 2023-12-19T18:18:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 -- 172.21.15.26:0/4094371243 wait complete. 2023-12-19T18:18:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 Processor -- start 2023-12-19T18:18:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 -- start start 2023-12-19T18:18:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab4003980 0x7f7ac80c0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ac815fde0 0x7f7ac815dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ac815aca0 0x7f7ac815b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80c5510 con 0x7f7ac815fde0 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac815e3e0 con 0x7f7ab4003980 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7acd073640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac815b660 con 0x7f7ac815aca0 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7abffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ac815fde0 0x7f7ac815dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7abffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ac815fde0 0x7f7ac815dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38296/0 (socket says 172.21.15.26:38296) 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7abffff640 1 -- 172.21.15.26:0/4217349574 learned_addr learned my addr 172.21.15.26:0/4217349574 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7ac6d9d640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab4003980 0x7f7ac80c0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.456+0000 7f7ac759e640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ac815aca0 0x7f7ac815b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7abffff640 1 -- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ac815aca0 msgr2=0x7f7ac815b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7abffff640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ac815aca0 0x7f7ac815b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7abffff640 1 -- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab4003980 msgr2=0x7f7ac80c0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7abffff640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab4003980 0x7f7ac80c0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7abffff640 1 -- 172.21.15.26:0/4217349574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac8160260 con 0x7f7ac815fde0 2023-12-19T18:18:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7abffff640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ac815fde0 0x7f7ac815dea0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0000fc0 tx=0x7f7ab000eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7abf7fe640 1 -- 172.21.15.26:0/4217349574 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab000c6c0 con 0x7f7ac815fde0 2023-12-19T18:18:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7abf7fe640 1 -- 172.21.15.26:0/4217349574 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab000c820 con 0x7f7ac815fde0 2023-12-19T18:18:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7acd073640 1 -- 172.21.15.26:0/4217349574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac815d400 con 0x7f7ac815fde0 2023-12-19T18:18:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7abf7fe640 1 -- 172.21.15.26:0/4217349574 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab000cbb0 con 0x7f7ac815fde0 2023-12-19T18:18:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7acd073640 1 -- 172.21.15.26:0/4217349574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac815d6a0 con 0x7f7ac815fde0 2023-12-19T18:18:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7acd073640 1 -- 172.21.15.26:0/4217349574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac80b0990 con 0x7f7ac815fde0 2023-12-19T18:18:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7abf7fe640 1 -- 172.21.15.26:0/4217349574 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f7ab000cd10 con 0x7f7ac815fde0 2023-12-19T18:18:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.460+0000 7f7abf7fe640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7aa8065ca0 0x7f7aa8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.464+0000 7f7abf7fe640 1 -- 172.21.15.26:0/4217349574 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f7ab00835a0 con 0x7f7ac815fde0 2023-12-19T18:18:20.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.464+0000 7f7ac6d9d640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7aa8065ca0 0x7f7aa8068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.464+0000 7f7ac6d9d640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7aa8065ca0 0x7f7aa8068160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80af000 tx=0x7f7ab80046d0 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:20.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.464+0000 7f7abf7fe640 1 -- 172.21.15.26:0/4217349574 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7ab0050b10 con 0x7f7ac815fde0 2023-12-19T18:18:20.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:20.660+0000 7f7acd073640 1 -- 172.21.15.26:0/4217349574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7ac8074270 con 0x7f7ac815fde0 2023-12-19T18:18:21.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.568+0000 7f7abf7fe640 1 -- 172.21.15.26:0/4217349574 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v127) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7ab0054130 con 0x7f7ac815fde0 2023-12-19T18:18:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.572+0000 7f7acd073640 1 -- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7aa8065ca0 msgr2=0x7f7aa8068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.572+0000 7f7acd073640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7aa8065ca0 0x7f7aa8068160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80af000 tx=0x7f7ab80046d0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.572+0000 7f7acd073640 1 -- 172.21.15.26:0/4217349574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ac815fde0 msgr2=0x7f7ac815dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.572+0000 7f7acd073640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ac815fde0 0x7f7ac815dea0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0000fc0 tx=0x7f7ab000eb60 comp rx=0 tx=0).stop 2023-12-19T18:18:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.572+0000 7f7acd073640 1 -- 172.21.15.26:0/4217349574 shutdown_connections 2023-12-19T18:18:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.572+0000 7f7acd073640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7aa8065ca0 0x7f7aa8068160 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.572+0000 7f7acd073640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7ac815aca0 0x7f7ac815b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.572+0000 7f7acd073640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7ac815fde0 0x7f7ac815dea0 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.572+0000 7f7acd073640 1 --2- 172.21.15.26:0/4217349574 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ab4003980 0x7f7ac80c0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.572+0000 7f7acd073640 1 -- 172.21.15.26:0/4217349574 >> 172.21.15.26:0/4217349574 conn(0x7f7ac80b0b60 msgr2=0x7f7ac806c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:21.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.572+0000 7f7acd073640 1 -- 172.21.15.26:0/4217349574 shutdown_connections 2023-12-19T18:18:21.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.572+0000 7f7acd073640 1 -- 172.21.15.26:0/4217349574 wait complete. 2023-12-19T18:18:21.595 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:18:21.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:18:21.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b7752640 1 Processor -- start 2023-12-19T18:18:21.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b7752640 1 -- start start 2023-12-19T18:18:21.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b7752640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69b0161230 0x7f69b00ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:21.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b7752640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69b00c8df0 0x7f69b00c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:21.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b7752640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b00b01b0 con 0x7f69b006c3f0 2023-12-19T18:18:21.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b7752640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b00adfb0 con 0x7f69b0161230 2023-12-19T18:18:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b7752640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b00ca4e0 con 0x7f69b00c8df0 2023-12-19T18:18:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b54ee640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69b0161230 0x7f69b00ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b54ee640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69b0161230 0x7f69b00ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38304/0 (socket says 172.21.15.26:38304) 2023-12-19T18:18:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b54ee640 1 -- 172.21.15.26:0/656422760 learned_addr learned my addr 172.21.15.26:0/656422760 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b5cef640 1 --2- 172.21.15.26:0/656422760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69b00c8df0 0x7f69b00c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b54ee640 1 -- 172.21.15.26:0/656422760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69b00c8df0 msgr2=0x7f69b00c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b54ee640 1 --2- 172.21.15.26:0/656422760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69b00c8df0 0x7f69b00c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b54ee640 1 -- 172.21.15.26:0/656422760 >> v1:172.21.15.26:6789/0 conn(0x7f69b006c3f0 legacy=0x7f69b006c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69b54ee640 1 -- 172.21.15.26:0/656422760 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b00b0400 con 0x7f69b0161230 2023-12-19T18:18:21.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.748+0000 7f69a7fff640 1 -- 172.21.15.26:0/656422760 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919802682 0 0) 0x7f69b00b01b0 con 0x7f69b006c3f0 2023-12-19T18:18:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.752+0000 7f69b54ee640 1 --2- 172.21.15.26:0/656422760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b0161230 0x7f69b00ada70 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009870 tx=0x7f69ac01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1694d8b8b48bbd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:21.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.752+0000 7f69a7fff640 1 -- 172.21.15.26:0/656422760 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac0ab680 con 0x7f69b0161230 2023-12-19T18:18:21.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.752+0000 7f69a7fff640 1 -- 172.21.15.26:0/656422760 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ac0ab7e0 con 0x7f69b0161230 2023-12-19T18:18:21.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.752+0000 7f69a7fff640 1 -- 172.21.15.26:0/656422760 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac0abb70 con 0x7f69b0161230 2023-12-19T18:18:21.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.752+0000 7f69b7752640 1 -- 172.21.15.26:0/656422760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b0161230 msgr2=0x7f69b00ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:21.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.752+0000 7f69b7752640 1 --2- 172.21.15.26:0/656422760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b0161230 0x7f69b00ada70 secure :-1 s=READY pgs=2340 cs=0 l=1 rev1=1 crypto rx=0x7f69ac009870 tx=0x7f69ac01c480 comp rx=0 tx=0).stop 2023-12-19T18:18:21.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.752+0000 7f69b7752640 1 -- 172.21.15.26:0/656422760 shutdown_connections 2023-12-19T18:18:21.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.752+0000 7f69b7752640 1 --2- 172.21.15.26:0/656422760 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69b00c8df0 0x7f69b00c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.752+0000 7f69b7752640 1 --2- 172.21.15.26:0/656422760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b0161230 0x7f69b00ada70 unknown :-1 s=CLOSED pgs=2340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.752+0000 7f69b7752640 1 -- 172.21.15.26:0/656422760 >> 172.21.15.26:0/656422760 conn(0x7f69b00ce730 msgr2=0x7f69b00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:21.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.752+0000 7f69b7752640 1 -- 172.21.15.26:0/656422760 shutdown_connections 2023-12-19T18:18:21.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.752+0000 7f69b7752640 1 -- 172.21.15.26:0/656422760 wait complete. 2023-12-19T18:18:21.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b7752640 1 Processor -- start 2023-12-19T18:18:21.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b7752640 1 -- start start 2023-12-19T18:18:21.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b7752640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69b006c3f0 0x7f69b0150a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:21.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b7752640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b00c8df0 0x7f69b014fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:21.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b7752640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69b0161230 0x7f69b0158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:21.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b7752640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b0162140 con 0x7f69b00c8df0 2023-12-19T18:18:21.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b7752640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b00c5ac0 con 0x7f69b0161230 2023-12-19T18:18:21.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b7752640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b0150fd0 con 0x7f69b006c3f0 2023-12-19T18:18:21.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b4ced640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b00c8df0 0x7f69b014fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:21.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b4ced640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b00c8df0 0x7f69b014fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38308/0 (socket says 172.21.15.26:38308) 2023-12-19T18:18:21.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b4ced640 1 -- 172.21.15.26:0/1095001371 learned_addr learned my addr 172.21.15.26:0/1095001371 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:21.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b54ee640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69b006c3f0 0x7f69b0150a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:21.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b4ced640 1 -- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69b006c3f0 msgr2=0x7f69b0150a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:21.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b4ced640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69b006c3f0 0x7f69b0150a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b5cef640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69b0161230 0x7f69b0158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:21.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b4ced640 1 -- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69b0161230 msgr2=0x7f69b0158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:21.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b4ced640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69b0161230 0x7f69b0158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b4ced640 1 -- 172.21.15.26:0/1095001371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac0094d0 con 0x7f69b00c8df0 2023-12-19T18:18:21.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b54ee640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69b006c3f0 0x7f69b0150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:21.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b5cef640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69b0161230 0x7f69b0158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:21.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b4ced640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b00c8df0 0x7f69b014fff0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f69a000c450 tx=0x7f69a000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:21.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69a5ffb640 1 -- 172.21.15.26:0/1095001371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a0005480 con 0x7f69b00c8df0 2023-12-19T18:18:21.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69a5ffb640 1 -- 172.21.15.26:0/1095001371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a0003600 con 0x7f69b00c8df0 2023-12-19T18:18:21.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b7752640 1 -- 172.21.15.26:0/1095001371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b01601b0 con 0x7f69b00c8df0 2023-12-19T18:18:21.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69a5ffb640 1 -- 172.21.15.26:0/1095001371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a00039b0 con 0x7f69b00c8df0 2023-12-19T18:18:21.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b7752640 1 -- 172.21.15.26:0/1095001371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b0161dd0 con 0x7f69b00c8df0 2023-12-19T18:18:21.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.756+0000 7f69b7752640 1 -- 172.21.15.26:0/1095001371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b0161950 con 0x7f69b00c8df0 2023-12-19T18:18:21.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.760+0000 7f69a5ffb640 1 -- 172.21.15.26:0/1095001371 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f69a0003b10 con 0x7f69b00c8df0 2023-12-19T18:18:21.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.760+0000 7f69a5ffb640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6994065ca0 0x7f6994068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:21.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.760+0000 7f69a5ffb640 1 -- 172.21.15.26:0/1095001371 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f69a0082ec0 con 0x7f69b00c8df0 2023-12-19T18:18:21.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.760+0000 7f69b54ee640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6994065ca0 0x7f6994068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:21.761 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.760+0000 7f69b54ee640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6994065ca0 0x7f6994068160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0af000 tx=0x7f69ac008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:21.764 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.764+0000 7f69a5ffb640 1 -- 172.21.15.26:0/1095001371 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f69b0161950 con 0x7f69b00c8df0 2023-12-19T18:18:21.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:21.960+0000 7f69b7752640 1 -- 172.21.15.26:0/1095001371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f69b01622f0 con 0x7f69b00c8df0 2023-12-19T18:18:22.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.576+0000 7f69a5ffb640 1 -- 172.21.15.26:0/1095001371 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v128) v1 ==== 122+0+0 (secure 0 0 0) 0x7f69b01622f0 con 0x7f69b00c8df0 2023-12-19T18:18:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.580+0000 7f69b7752640 1 -- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6994065ca0 msgr2=0x7f6994068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.580+0000 7f69b7752640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6994065ca0 0x7f6994068160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f69ac0af000 tx=0x7f69ac008040 comp rx=0 tx=0).stop 2023-12-19T18:18:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.580+0000 7f69b7752640 1 -- 172.21.15.26:0/1095001371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b00c8df0 msgr2=0x7f69b014fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.580+0000 7f69b7752640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b00c8df0 0x7f69b014fff0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7f69a000c450 tx=0x7f69a000c910 comp rx=0 tx=0).stop 2023-12-19T18:18:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.580+0000 7f69b7752640 1 -- 172.21.15.26:0/1095001371 shutdown_connections 2023-12-19T18:18:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.580+0000 7f69b7752640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6994065ca0 0x7f6994068160 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.580+0000 7f69b7752640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69b0161230 0x7f69b0158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.580+0000 7f69b7752640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69b00c8df0 0x7f69b014fff0 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.580+0000 7f69b7752640 1 --2- 172.21.15.26:0/1095001371 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f69b006c3f0 0x7f69b0150a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.580+0000 7f69b7752640 1 -- 172.21.15.26:0/1095001371 >> 172.21.15.26:0/1095001371 conn(0x7f69b00ce730 msgr2=0x7f69b00afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.580+0000 7f69b7752640 1 -- 172.21.15.26:0/1095001371 shutdown_connections 2023-12-19T18:18:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.580+0000 7f69b7752640 1 -- 172.21.15.26:0/1095001371 wait complete. 2023-12-19T18:18:22.604 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:18:22.604 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f96e66640 1 Processor -- start 2023-12-19T18:18:22.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f96e66640 1 -- start start 2023-12-19T18:18:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f96e66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1f9015cda0 0x7f1f90160920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f96e66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f900b85e0 0x7f1f900cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f96e66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f900d0d40 con 0x7f1f900cde60 2023-12-19T18:18:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f96e66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900ce2d0 con 0x7f1f900b85e0 2023-12-19T18:18:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f96e66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f90161150 con 0x7f1f9015cda0 2023-12-19T18:18:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f8ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f900b85e0 0x7f1f900cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f8ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f900b85e0 0x7f1f900cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38320/0 (socket says 172.21.15.26:38320) 2023-12-19T18:18:22.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f8ffff640 1 -- 172.21.15.26:0/224684501 learned_addr learned my addr 172.21.15.26:0/224684501 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:22.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f94c02640 1 --2- 172.21.15.26:0/224684501 >> v2:172.21.15.160:3300/0 conn(0x7f1f9015cda0 0x7f1f90160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f8ffff640 1 -- 172.21.15.26:0/224684501 >> v2:172.21.15.160:3300/0 conn(0x7f1f9015cda0 msgr2=0x7f1f90160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:22.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f8ffff640 1 --2- 172.21.15.26:0/224684501 >> v2:172.21.15.160:3300/0 conn(0x7f1f9015cda0 0x7f1f90160920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f8ffff640 1 -- 172.21.15.26:0/224684501 >> v1:172.21.15.26:6789/0 conn(0x7f1f900cde60 legacy=0x7f1f900b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.752+0000 7f1f8ffff640 1 -- 172.21.15.26:0/224684501 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f90161430 con 0x7f1f900b85e0 2023-12-19T18:18:22.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.756+0000 7f1f8ffff640 1 --2- 172.21.15.26:0/224684501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f900b85e0 0x7f1f900cd920 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f1f900d3c80 tx=0x7f1f7c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70aeb5c779875dff server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:22.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.756+0000 7f1f8f7fe640 1 -- 172.21.15.26:0/224684501 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0aa430 con 0x7f1f900b85e0 2023-12-19T18:18:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.756+0000 7f1f8f7fe640 1 -- 172.21.15.26:0/224684501 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c0aa590 con 0x7f1f900b85e0 2023-12-19T18:18:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.756+0000 7f1f8f7fe640 1 -- 172.21.15.26:0/224684501 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0aa920 con 0x7f1f900b85e0 2023-12-19T18:18:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.756+0000 7f1f96e66640 1 -- 172.21.15.26:0/224684501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f900b85e0 msgr2=0x7f1f900cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.756+0000 7f1f96e66640 1 --2- 172.21.15.26:0/224684501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f900b85e0 0x7f1f900cd920 secure :-1 s=READY pgs=2343 cs=0 l=1 rev1=1 crypto rx=0x7f1f900d3c80 tx=0x7f1f7c01c4d0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.756+0000 7f1f96e66640 1 -- 172.21.15.26:0/224684501 shutdown_connections 2023-12-19T18:18:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.756+0000 7f1f96e66640 1 --2- 172.21.15.26:0/224684501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f900b85e0 0x7f1f900cd920 unknown :-1 s=CLOSED pgs=2343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.756+0000 7f1f96e66640 1 --2- 172.21.15.26:0/224684501 >> v2:172.21.15.160:3300/0 conn(0x7f1f9015cda0 0x7f1f90160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.756+0000 7f1f96e66640 1 -- 172.21.15.26:0/224684501 >> 172.21.15.26:0/224684501 conn(0x7f1f900658e0 msgr2=0x7f1f90065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:22.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.756+0000 7f1f96e66640 1 -- 172.21.15.26:0/224684501 shutdown_connections 2023-12-19T18:18:22.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.756+0000 7f1f96e66640 1 -- 172.21.15.26:0/224684501 wait complete. 2023-12-19T18:18:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f96e66640 1 Processor -- start 2023-12-19T18:18:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f96e66640 1 -- start start 2023-12-19T18:18:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f96e66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900b85e0 0x7f1f90159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f96e66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f900cde60 0x7f1f90152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f96e66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9015cda0 0x7f1f9015d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f96e66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9015a210 con 0x7f1f900cde60 2023-12-19T18:18:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f96e66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900c7190 con 0x7f1f900b85e0 2023-12-19T18:18:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f96e66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900c7750 con 0x7f1f9015cda0 2023-12-19T18:18:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f8ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f900cde60 0x7f1f90152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f8ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f900cde60 0x7f1f90152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38334/0 (socket says 172.21.15.26:38334) 2023-12-19T18:18:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f8ffff640 1 -- 172.21.15.26:0/4230300907 learned_addr learned my addr 172.21.15.26:0/4230300907 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f94c02640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900b85e0 0x7f1f90159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f8ffff640 1 -- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9015cda0 msgr2=0x7f1f9015d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f95403640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9015cda0 0x7f1f9015d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f8ffff640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9015cda0 0x7f1f9015d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f8ffff640 1 -- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900b85e0 msgr2=0x7f1f90159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f8ffff640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900b85e0 0x7f1f90159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f8ffff640 1 -- 172.21.15.26:0/4230300907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c009480 con 0x7f1f900cde60 2023-12-19T18:18:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f94c02640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900b85e0 0x7f1f90159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f95403640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9015cda0 0x7f1f9015d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f8ffff640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f900cde60 0x7f1f90152240 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c01a100 tx=0x7f1f7c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f8d7fa640 1 -- 172.21.15.26:0/4230300907 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0bb070 con 0x7f1f900cde60 2023-12-19T18:18:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f8d7fa640 1 -- 172.21.15.26:0/4230300907 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c0aa9d0 con 0x7f1f900cde60 2023-12-19T18:18:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f8d7fa640 1 -- 172.21.15.26:0/4230300907 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0aad40 con 0x7f1f900cde60 2023-12-19T18:18:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f96e66640 1 -- 172.21.15.26:0/4230300907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f900cc6f0 con 0x7f1f900cde60 2023-12-19T18:18:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.760+0000 7f1f96e66640 1 -- 172.21.15.26:0/4230300907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f900c73e0 con 0x7f1f900cde60 2023-12-19T18:18:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.764+0000 7f1f96e66640 1 -- 172.21.15.26:0/4230300907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f900c78e0 con 0x7f1f900cde60 2023-12-19T18:18:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.764+0000 7f1f8d7fa640 1 -- 172.21.15.26:0/4230300907 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f1f7c003620 con 0x7f1f900cde60 2023-12-19T18:18:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.764+0000 7f1f8d7fa640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f74065ca0 0x7f1f74068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.764+0000 7f1f8d7fa640 1 -- 172.21.15.26:0/4230300907 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f1f7c11dad0 con 0x7f1f900cde60 2023-12-19T18:18:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.764+0000 7f1f94c02640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f74065ca0 0x7f1f74068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.764+0000 7f1f94c02640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f74065ca0 0x7f1f74068160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009840 tx=0x7f1f84008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.768+0000 7f1f8d7fa640 1 -- 172.21.15.26:0/4230300907 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1f7c0ead00 con 0x7f1f900cde60 2023-12-19T18:18:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f96e66640 1 -- 172.21.15.26:0/4230300907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f90074270 con 0x7f1f900cde60 2023-12-19T18:18:22.960 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f8d7fa640 1 -- 172.21.15.26:0/4230300907 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4728 (secure 0 0 0) 0x7f1f7c0ee320 con 0x7f1f900cde60 2023-12-19T18:18:22.960 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:22.961 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":128,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/282680544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":282680544},{"type":"v1","addr":"172.21.15.160:6836","nonce":282680544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3302634175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3302634175},{"type":"v1","addr":"172.21.15.160:6837","nonce":3302634175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3434994961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3434994961},{"type":"v1","addr":"172.21.15.26:6834","nonce":3434994961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:18:19.555514+0000","modified":"2023-12-19T18:18:22.569161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1019381645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1019381645},{"type":"v1","addr":"172.21.15.26:6835","nonce":1019381645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-19T18:18:22.961 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 128 2023-12-19T18:18:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f96e66640 1 -- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f74065ca0 msgr2=0x7f1f74068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f96e66640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f74065ca0 0x7f1f74068160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009840 tx=0x7f1f84008040 comp rx=0 tx=0).stop 2023-12-19T18:18:22.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f96e66640 1 -- 172.21.15.26:0/4230300907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f900cde60 msgr2=0x7f1f90152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f96e66640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f900cde60 0x7f1f90152240 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c01a100 tx=0x7f1f7c019270 comp rx=0 tx=0).stop 2023-12-19T18:18:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f96e66640 1 -- 172.21.15.26:0/4230300907 shutdown_connections 2023-12-19T18:18:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f96e66640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f74065ca0 0x7f1f74068160 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f96e66640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f9015cda0 0x7f1f9015d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f96e66640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f900cde60 0x7f1f90152240 unknown :-1 s=CLOSED pgs=2344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f96e66640 1 --2- 172.21.15.26:0/4230300907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f900b85e0 0x7f1f90159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f96e66640 1 -- 172.21.15.26:0/4230300907 >> 172.21.15.26:0/4230300907 conn(0x7f1f900658e0 msgr2=0x7f1f9015b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f96e66640 1 -- 172.21.15.26:0/4230300907 shutdown_connections 2023-12-19T18:18:22.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:22.960+0000 7f1f96e66640 1 -- 172.21.15.26:0/4230300907 wait complete. 2023-12-19T18:18:22.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:23.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6bb0b51640 1 Processor -- start 2023-12-19T18:18:23.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6bb0b51640 1 -- start start 2023-12-19T18:18:23.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6bb0b51640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac15cca0 0x7f6bac0b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6bb0b51640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac0b6fb0 con 0x7f6bac0b8760 2023-12-19T18:18:23.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6bb0b51640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac0b7290 con 0x7f6bac0b6b20 2023-12-19T18:18:23.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6bb0b51640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0b7c30 con 0x7f6bac15cca0 2023-12-19T18:18:23.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6baaffd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6bac0b8760 0x7f6bac0cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36218/0 (socket says 172.21.15.26:36218) 2023-12-19T18:18:23.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6baaffd640 1 -- 172.21.15.26:0/4005272728 learned_addr learned my addr 172.21.15.26:0/4005272728 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6bab7fe640 1 --2- 172.21.15.26:0/4005272728 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac15cca0 0x7f6bac0b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6baa7fc640 1 -- 172.21.15.26:0/4005272728 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324997925 0 0) 0x7f6bac0b6fb0 con 0x7f6bac0b8760 2023-12-19T18:18:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6baa7fc640 1 -- 172.21.15.26:0/4005272728 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b94003530 con 0x7f6bac0b8760 2023-12-19T18:18:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6baa7fc640 1 -- 172.21.15.26:0/4005272728 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889264243 0 0) 0x7f6bac0b7290 con 0x7f6bac0b6b20 2023-12-19T18:18:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6baa7fc640 1 -- 172.21.15.26:0/4005272728 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bac0b6fb0 con 0x7f6bac0b6b20 2023-12-19T18:18:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6bab7fe640 1 -- 172.21.15.26:0/4005272728 >> v1:172.21.15.160:6789/0 conn(0x7f6bac0b6b20 legacy=0x7f6bac0d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6bab7fe640 1 -- 172.21.15.26:0/4005272728 >> v1:172.21.15.26:6789/0 conn(0x7f6bac0b8760 legacy=0x7f6bac0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6baa7fc640 1 -- 172.21.15.26:0/4005272728 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3176528693 0 0) 0x7f6b94003530 con 0x7f6bac0b8760 2023-12-19T18:18:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.128+0000 7f6bab7fe640 1 -- 172.21.15.26:0/4005272728 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac0b7570 con 0x7f6bac15cca0 2023-12-19T18:18:23.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.132+0000 7f6bab7fe640 1 --2- 172.21.15.26:0/4005272728 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac15cca0 0x7f6bac0b65e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0097f0 tx=0x7f6b9c01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99a407cf596bd99f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.132+0000 7f6baa7fc640 1 -- 172.21.15.26:0/4005272728 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0aa610 con 0x7f6bac15cca0 2023-12-19T18:18:23.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.132+0000 7f6baa7fc640 1 -- 172.21.15.26:0/4005272728 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c0a6040 con 0x7f6bac15cca0 2023-12-19T18:18:23.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.132+0000 7f6baa7fc640 1 -- 172.21.15.26:0/4005272728 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0aa990 con 0x7f6bac15cca0 2023-12-19T18:18:23.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.132+0000 7f6bb0b51640 1 -- 172.21.15.26:0/4005272728 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac15cca0 msgr2=0x7f6bac0b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.132+0000 7f6bb0b51640 1 --2- 172.21.15.26:0/4005272728 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac15cca0 0x7f6bac0b65e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0097f0 tx=0x7f6b9c01c490 comp rx=0 tx=0).stop 2023-12-19T18:18:23.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.132+0000 7f6bb0b51640 1 -- 172.21.15.26:0/4005272728 shutdown_connections 2023-12-19T18:18:23.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.132+0000 7f6bb0b51640 1 --2- 172.21.15.26:0/4005272728 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac15cca0 0x7f6bac0b65e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.132+0000 7f6bb0b51640 1 -- 172.21.15.26:0/4005272728 >> 172.21.15.26:0/4005272728 conn(0x7f6bac06a120 msgr2=0x7f6bac06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:23.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.132+0000 7f6bb0b51640 1 -- 172.21.15.26:0/4005272728 shutdown_connections 2023-12-19T18:18:23.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.132+0000 7f6bb0b51640 1 -- 172.21.15.26:0/4005272728 wait complete. 2023-12-19T18:18:23.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6bb0b51640 1 Processor -- start 2023-12-19T18:18:23.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6bb0b51640 1 -- start start 2023-12-19T18:18:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6bb0b51640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac0b8760 0x7f6bac151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6bb0b51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15cca0 0x7f6bac0cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6bb0b51640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac0c7350 0x7f6bac0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6bb0b51640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0c8570 con 0x7f6bac15cca0 2023-12-19T18:18:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6bb0b51640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0ccd00 con 0x7f6bac0b8760 2023-12-19T18:18:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6bb0b51640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac152360 con 0x7f6bac0c7350 2023-12-19T18:18:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6baaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15cca0 0x7f6bac0cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6baaffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15cca0 0x7f6bac0cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38350/0 (socket says 172.21.15.26:38350) 2023-12-19T18:18:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6baaffd640 1 -- 172.21.15.26:0/3833988134 learned_addr learned my addr 172.21.15.26:0/3833988134 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6bab7fe640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac0b8760 0x7f6bac151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6babfff640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac0c7350 0x7f6bac0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6baaffd640 1 -- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac0c7350 msgr2=0x7f6bac0cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6baaffd640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac0c7350 0x7f6bac0cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6baaffd640 1 -- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac0b8760 msgr2=0x7f6bac151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6baaffd640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac0b8760 0x7f6bac151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6baaffd640 1 -- 172.21.15.26:0/3833988134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c009420 con 0x7f6bac15cca0 2023-12-19T18:18:23.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6bab7fe640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac0b8760 0x7f6bac151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:23.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6babfff640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac0c7350 0x7f6bac0cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:23.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6baaffd640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15cca0 0x7f6bac0cc280 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c430 tx=0x7f6ba000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6b93fff640 1 -- 172.21.15.26:0/3833988134 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0005cf0 con 0x7f6bac15cca0 2023-12-19T18:18:23.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6b93fff640 1 -- 172.21.15.26:0/3833988134 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba0005e50 con 0x7f6bac15cca0 2023-12-19T18:18:23.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6bb0b51640 1 -- 172.21.15.26:0/3833988134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac0c78f0 con 0x7f6bac15cca0 2023-12-19T18:18:23.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6b93fff640 1 -- 172.21.15.26:0/3833988134 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba00072d0 con 0x7f6bac15cca0 2023-12-19T18:18:23.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6bb0b51640 1 -- 172.21.15.26:0/3833988134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac0c7eb0 con 0x7f6bac15cca0 2023-12-19T18:18:23.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.136+0000 7f6bb0b51640 1 -- 172.21.15.26:0/3833988134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac0c86b0 con 0x7f6bac15cca0 2023-12-19T18:18:23.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.140+0000 7f6b93fff640 1 -- 172.21.15.26:0/3833988134 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f6ba0007430 con 0x7f6bac15cca0 2023-12-19T18:18:23.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.140+0000 7f6b93fff640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b98065ca0 0x7f6b98068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.140 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.140+0000 7f6b93fff640 1 -- 172.21.15.26:0/3833988134 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f6ba0082e50 con 0x7f6bac15cca0 2023-12-19T18:18:23.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.140+0000 7f6bab7fe640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b98065ca0 0x7f6b98068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.140+0000 7f6bab7fe640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b98065ca0 0x7f6b98068160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0ae000 tx=0x7f6b9c008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.144 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.144+0000 7f6b93fff640 1 -- 172.21.15.26:0/3833988134 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6bac0c86b0 con 0x7f6bac15cca0 2023-12-19T18:18:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.308+0000 7f6bb0b51640 1 -- 172.21.15.26:0/3833988134 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bac0c86b0 con 0x7f6bac15cca0 2023-12-19T18:18:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.308+0000 7f6b93fff640 1 -- 172.21.15.26:0/3833988134 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v254) v1 ==== 74+0+17205 (secure 0 0 0) 0x7f6ba007ae60 con 0x7f6bac15cca0 2023-12-19T18:18:23.310 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:23.310 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":254,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:18:20.558643+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:17.314988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:18.558040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.312+0000 7f6bb0b51640 1 -- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b98065ca0 msgr2=0x7f6b98068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.312+0000 7f6bb0b51640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b98065ca0 0x7f6b98068160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0ae000 tx=0x7f6b9c008040 comp rx=0 tx=0).stop 2023-12-19T18:18:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.312+0000 7f6bb0b51640 1 -- 172.21.15.26:0/3833988134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15cca0 msgr2=0x7f6bac0cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.312+0000 7f6bb0b51640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15cca0 0x7f6bac0cc280 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f6ba000c430 tx=0x7f6ba000c8f0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.312+0000 7f6bb0b51640 1 -- 172.21.15.26:0/3833988134 shutdown_connections 2023-12-19T18:18:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.312+0000 7f6bb0b51640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b98065ca0 0x7f6b98068160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.312+0000 7f6bb0b51640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6bac0c7350 0x7f6bac0cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.312+0000 7f6bb0b51640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac15cca0 0x7f6bac0cc280 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.312+0000 7f6bb0b51640 1 --2- 172.21.15.26:0/3833988134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6bac0b8760 0x7f6bac151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.312+0000 7f6bb0b51640 1 -- 172.21.15.26:0/3833988134 >> 172.21.15.26:0/3833988134 conn(0x7f6bac06a120 msgr2=0x7f6bac0b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.312+0000 7f6bb0b51640 1 -- 172.21.15.26:0/3833988134 shutdown_connections 2023-12-19T18:18:23.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.312+0000 7f6bb0b51640 1 -- 172.21.15.26:0/3833988134 wait complete. 2023-12-19T18:18:23.330 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:18:23.331 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:18:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.472+0000 7f9334118640 1 Processor -- start 2023-12-19T18:18:23.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.472+0000 7f9334118640 1 -- start start 2023-12-19T18:18:23.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.472+0000 7f9334118640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f932c06bee0 0x7f932c06c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.472+0000 7f9334118640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932c0b1d40 0x7f932c0b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.472+0000 7f9334118640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932c06d4b0 con 0x7f932c0cb1d0 2023-12-19T18:18:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.472+0000 7f9334118640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932c0cc030 con 0x7f932c06bee0 2023-12-19T18:18:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.472+0000 7f9334118640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c06cc10 con 0x7f932c0b1d40 2023-12-19T18:18:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.472+0000 7f9331eb4640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f932c0cb1d0 0x7f932c0cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36232/0 (socket says 172.21.15.26:36232) 2023-12-19T18:18:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.472+0000 7f9331eb4640 1 -- 172.21.15.26:0/3033314703 learned_addr learned my addr 172.21.15.26:0/3033314703 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.472+0000 7f93326b5640 1 --2- 172.21.15.26:0/3033314703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932c0b1d40 0x7f932c0b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.475 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.472+0000 7f93316b3640 1 --2- 172.21.15.26:0/3033314703 >> v2:172.21.15.160:3300/0 conn(0x7f932c06bee0 0x7f932c06c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.476+0000 7f9330eb2640 1 -- 172.21.15.26:0/3033314703 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066023732 0 0) 0x7f932c06d4b0 con 0x7f932c0cb1d0 2023-12-19T18:18:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.476+0000 7f9330eb2640 1 -- 172.21.15.26:0/3033314703 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9318003530 con 0x7f932c0cb1d0 2023-12-19T18:18:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.476+0000 7f93326b5640 1 -- 172.21.15.26:0/3033314703 >> v2:172.21.15.160:3300/0 conn(0x7f932c06bee0 msgr2=0x7f932c06c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.476+0000 7f93326b5640 1 --2- 172.21.15.26:0/3033314703 >> v2:172.21.15.160:3300/0 conn(0x7f932c06bee0 0x7f932c06c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.476+0000 7f93326b5640 1 -- 172.21.15.26:0/3033314703 >> v1:172.21.15.26:6789/0 conn(0x7f932c0cb1d0 legacy=0x7f932c0cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.476+0000 7f93326b5640 1 -- 172.21.15.26:0/3033314703 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c06cef0 con 0x7f932c0b1d40 2023-12-19T18:18:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.476+0000 7f93326b5640 1 --2- 172.21.15.26:0/3033314703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932c0b1d40 0x7f932c0b21d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9324009d90 tx=0x7f932401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=899222760be2b0c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.476+0000 7f9330eb2640 1 -- 172.21.15.26:0/3033314703 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240a5950 con 0x7f932c0b1d40 2023-12-19T18:18:23.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.476+0000 7f9330eb2640 1 -- 172.21.15.26:0/3033314703 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93240a5ab0 con 0x7f932c0b1d40 2023-12-19T18:18:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.476+0000 7f9330eb2640 1 -- 172.21.15.26:0/3033314703 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93240a5e40 con 0x7f932c0b1d40 2023-12-19T18:18:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 -- 172.21.15.26:0/3033314703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932c0b1d40 msgr2=0x7f932c0b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 --2- 172.21.15.26:0/3033314703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932c0b1d40 0x7f932c0b21d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9324009d90 tx=0x7f932401c890 comp rx=0 tx=0).stop 2023-12-19T18:18:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 -- 172.21.15.26:0/3033314703 shutdown_connections 2023-12-19T18:18:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 --2- 172.21.15.26:0/3033314703 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932c0b1d40 0x7f932c0b21d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 --2- 172.21.15.26:0/3033314703 >> v2:172.21.15.160:3300/0 conn(0x7f932c06bee0 0x7f932c06c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 -- 172.21.15.26:0/3033314703 >> 172.21.15.26:0/3033314703 conn(0x7f932c0658e0 msgr2=0x7f932c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 -- 172.21.15.26:0/3033314703 shutdown_connections 2023-12-19T18:18:23.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 -- 172.21.15.26:0/3033314703 wait complete. 2023-12-19T18:18:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 Processor -- start 2023-12-19T18:18:23.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 -- start start 2023-12-19T18:18:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c06bee0 0x7f932c14dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0b1d40 0x7f932c1497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932c0cb1d0 0x7f932c149d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c1605a0 con 0x7f932c0b1d40 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c14f7e0 con 0x7f932c06bee0 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9334118640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c160840 con 0x7f932c0cb1d0 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f93316b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0b1d40 0x7f932c1497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f93316b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0b1d40 0x7f932c1497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38366/0 (socket says 172.21.15.26:38366) 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f93316b3640 1 -- 172.21.15.26:0/408041805 learned_addr learned my addr 172.21.15.26:0/408041805 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f9331eb4640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c06bee0 0x7f932c14dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f93326b5640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932c0cb1d0 0x7f932c149d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f93316b3640 1 -- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932c0cb1d0 msgr2=0x7f932c149d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f93316b3640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932c0cb1d0 0x7f932c149d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f93316b3640 1 -- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c06bee0 msgr2=0x7f932c14dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f93316b3640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c06bee0 0x7f932c14dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.480+0000 7f93316b3640 1 -- 172.21.15.26:0/408041805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9324009a70 con 0x7f932c0b1d40 2023-12-19T18:18:23.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.484+0000 7f93316b3640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0b1d40 0x7f932c1497e0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f931c00e930 tx=0x7f931c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.484+0000 7f93227fc640 1 -- 172.21.15.26:0/408041805 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c009690 con 0x7f932c0b1d40 2023-12-19T18:18:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.484+0000 7f93227fc640 1 -- 172.21.15.26:0/408041805 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c00ca00 con 0x7f932c0b1d40 2023-12-19T18:18:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.484+0000 7f93227fc640 1 -- 172.21.15.26:0/408041805 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c00cd70 con 0x7f932c0b1d40 2023-12-19T18:18:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.484+0000 7f9334118640 1 -- 172.21.15.26:0/408041805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932c14fa80 con 0x7f932c0b1d40 2023-12-19T18:18:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.484+0000 7f9334118640 1 -- 172.21.15.26:0/408041805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932c14e100 con 0x7f932c0b1d40 2023-12-19T18:18:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.484+0000 7f9334118640 1 -- 172.21.15.26:0/408041805 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932c14f640 con 0x7f932c0b1d40 2023-12-19T18:18:23.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.484+0000 7f93227fc640 1 -- 172.21.15.26:0/408041805 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f931c013470 con 0x7f932c0b1d40 2023-12-19T18:18:23.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.484+0000 7f93227fc640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9310065ca0 0x7f9310068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.484+0000 7f93227fc640 1 -- 172.21.15.26:0/408041805 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f931c083bc0 con 0x7f932c0b1d40 2023-12-19T18:18:23.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.484+0000 7f9331eb4640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9310065ca0 0x7f9310068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.488+0000 7f9331eb4640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9310065ca0 0x7f9310068160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9328005890 tx=0x7f9328009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.488+0000 7f93227fc640 1 -- 172.21.15.26:0/408041805 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f932c14f640 con 0x7f932c0b1d40 2023-12-19T18:18:23.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.644+0000 7f9334118640 1 -- 172.21.15.26:0/408041805 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f932c1609b0 con 0x7f9310065ca0 2023-12-19T18:18:23.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.656+0000 7f93227fc640 1 -- 172.21.15.26:0/408041805 <== mgr.9680 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289367 (secure 0 0 0) 0x7f932c1609b0 con 0x7f9310065ca0 2023-12-19T18:18:23.661 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:23.663 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:18:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.660+0000 7f9334118640 1 -- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9310065ca0 msgr2=0x7f9310068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.660+0000 7f9334118640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9310065ca0 0x7f9310068160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9328005890 tx=0x7f9328009040 comp rx=0 tx=0).stop 2023-12-19T18:18:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.660+0000 7f9334118640 1 -- 172.21.15.26:0/408041805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0b1d40 msgr2=0x7f932c1497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.660+0000 7f9334118640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0b1d40 0x7f932c1497e0 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f931c00e930 tx=0x7f931c00edf0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.660+0000 7f9334118640 1 -- 172.21.15.26:0/408041805 shutdown_connections 2023-12-19T18:18:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.660+0000 7f9334118640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9310065ca0 0x7f9310068160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.660+0000 7f9334118640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f932c0cb1d0 0x7f932c149d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.660+0000 7f9334118640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0b1d40 0x7f932c1497e0 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.660+0000 7f9334118640 1 --2- 172.21.15.26:0/408041805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f932c06bee0 0x7f932c14dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.660+0000 7f9334118640 1 -- 172.21.15.26:0/408041805 >> 172.21.15.26:0/408041805 conn(0x7f932c0658e0 msgr2=0x7f932c06fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.660+0000 7f9334118640 1 -- 172.21.15.26:0/408041805 shutdown_connections 2023-12-19T18:18:23.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.660+0000 7f9334118640 1 -- 172.21.15.26:0/408041805 wait complete. 2023-12-19T18:18:23.682 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-12-19T18:18:21.953096+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2498,"num_read_kb":5235,"num_write":699,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":428,"ondisk_log_size":428,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":427144,"kb_used_data":26184,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754547576,"statfs":{"total":773094113280,"available":772656717824,"internally_reserved":0,"allocated":26812416,"data_stored":14902340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000898"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.571329+0000","last_change":"2023-12-19T18:18:19.586634+0000","last_active":"2023-12-19T18:18:20.571329+0000","last_peered":"2023-12-19T18:18:20.571329+0000","last_clean":"2023-12-19T18:18:20.571329+0000","last_became_active":"2023-12-19T18:18:19.586328+0000","last_became_peered":"2023-12-19T18:18:19.586328+0000","last_unstale":"2023-12-19T18:18:20.571329+0000","last_undegraded":"2023-12-19T18:18:20.571329+0000","last_fullsized":"2023-12-19T18:18:20.571329+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:06.547846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856382+0000","last_change":"2023-12-19T18:18:19.587556+0000","last_active":"2023-12-19T18:18:20.856382+0000","last_peered":"2023-12-19T18:18:20.856382+0000","last_clean":"2023-12-19T18:18:20.856382+0000","last_became_active":"2023-12-19T18:18:19.586818+0000","last_became_peered":"2023-12-19T18:18:19.586818+0000","last_unstale":"2023-12-19T18:18:20.856382+0000","last_undegraded":"2023-12-19T18:18:20.856382+0000","last_fullsized":"2023-12-19T18:18:20.856382+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:11:09.830159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570743+0000","last_change":"2023-12-19T18:18:19.579130+0000","last_active":"2023-12-19T18:18:20.570743+0000","last_peered":"2023-12-19T18:18:20.570743+0000","last_clean":"2023-12-19T18:18:20.570743+0000","last_became_active":"2023-12-19T18:18:19.578721+0000","last_became_peered":"2023-12-19T18:18:19.578721+0000","last_unstale":"2023-12-19T18:18:20.570743+0000","last_undegraded":"2023-12-19T18:18:20.570743+0000","last_fullsized":"2023-12-19T18:18:20.570743+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:54:55.928387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570512+0000","last_change":"2023-12-19T18:18:19.590935+0000","last_active":"2023-12-19T18:18:20.570512+0000","last_peered":"2023-12-19T18:18:20.570512+0000","last_clean":"2023-12-19T18:18:20.570512+0000","last_became_active":"2023-12-19T18:18:19.589372+0000","last_became_peered":"2023-12-19T18:18:19.589372+0000","last_unstale":"2023-12-19T18:18:20.570512+0000","last_undegraded":"2023-12-19T18:18:20.570512+0000","last_fullsized":"2023-12-19T18:18:20.570512+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:25:48.739322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570507+0000","last_change":"2023-12-19T18:18:19.583837+0000","last_active":"2023-12-19T18:18:20.570507+0000","last_peered":"2023-12-19T18:18:20.570507+0000","last_clean":"2023-12-19T18:18:20.570507+0000","last_became_active":"2023-12-19T18:18:19.583573+0000","last_became_peered":"2023-12-19T18:18:19.583573+0000","last_unstale":"2023-12-19T18:18:20.570507+0000","last_undegraded":"2023-12-19T18:18:20.570507+0000","last_fullsized":"2023-12-19T18:18:20.570507+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:18.093251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855424+0000","last_change":"2023-12-19T18:18:19.580670+0000","last_active":"2023-12-19T18:18:20.855424+0000","last_peered":"2023-12-19T18:18:20.855424+0000","last_clean":"2023-12-19T18:18:20.855424+0000","last_became_active":"2023-12-19T18:18:19.580313+0000","last_became_peered":"2023-12-19T18:18:19.580313+0000","last_unstale":"2023-12-19T18:18:20.855424+0000","last_undegraded":"2023-12-19T18:18:20.855424+0000","last_fullsized":"2023-12-19T18:18:20.855424+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:20:01.112918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856662+0000","last_change":"2023-12-19T18:18:19.584213+0000","last_active":"2023-12-19T18:18:20.856662+0000","last_peered":"2023-12-19T18:18:20.856662+0000","last_clean":"2023-12-19T18:18:20.856662+0000","last_became_active":"2023-12-19T18:18:19.583886+0000","last_became_peered":"2023-12-19T18:18:19.583886+0000","last_unstale":"2023-12-19T18:18:20.856662+0000","last_undegraded":"2023-12-19T18:18:20.856662+0000","last_fullsized":"2023-12-19T18:18:20.856662+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:56:58.406195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570237+0000","last_change":"2023-12-19T18:18:19.589110+0000","last_active":"2023-12-19T18:18:20.570237+0000","last_peered":"2023-12-19T18:18:20.570237+0000","last_clean":"2023-12-19T18:18:20.570237+0000","last_became_active":"2023-12-19T18:18:19.588826+0000","last_became_peered":"2023-12-19T18:18:19.588826+0000","last_unstale":"2023-12-19T18:18:20.570237+0000","last_undegraded":"2023-12-19T18:18:20.570237+0000","last_fullsized":"2023-12-19T18:18:20.570237+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:50.803341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569386+0000","last_change":"2023-12-19T18:18:19.580018+0000","last_active":"2023-12-19T18:18:20.569386+0000","last_peered":"2023-12-19T18:18:20.569386+0000","last_clean":"2023-12-19T18:18:20.569386+0000","last_became_active":"2023-12-19T18:18:19.579761+0000","last_became_peered":"2023-12-19T18:18:19.579761+0000","last_unstale":"2023-12-19T18:18:20.569386+0000","last_undegraded":"2023-12-19T18:18:20.569386+0000","last_fullsized":"2023-12-19T18:18:20.569386+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:19.227642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.857138+0000","last_change":"2023-12-19T18:18:19.589673+0000","last_active":"2023-12-19T18:18:20.857138+0000","last_peered":"2023-12-19T18:18:20.857138+0000","last_clean":"2023-12-19T18:18:20.857138+0000","last_became_active":"2023-12-19T18:18:19.589360+0000","last_became_peered":"2023-12-19T18:18:19.589360+0000","last_unstale":"2023-12-19T18:18:20.857138+0000","last_undegraded":"2023-12-19T18:18:20.857138+0000","last_fullsized":"2023-12-19T18:18:20.857138+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:21:09.116614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856918+0000","last_change":"2023-12-19T18:18:19.582645+0000","last_active":"2023-12-19T18:18:20.856918+0000","last_peered":"2023-12-19T18:18:20.856918+0000","last_clean":"2023-12-19T18:18:20.856918+0000","last_became_active":"2023-12-19T18:18:19.582301+0000","last_became_peered":"2023-12-19T18:18:19.582301+0000","last_unstale":"2023-12-19T18:18:20.856918+0000","last_undegraded":"2023-12-19T18:18:20.856918+0000","last_fullsized":"2023-12-19T18:18:20.856918+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:28:17.264359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.571031+0000","last_change":"2023-12-19T18:18:19.568458+0000","last_active":"2023-12-19T18:18:20.571031+0000","last_peered":"2023-12-19T18:18:20.571031+0000","last_clean":"2023-12-19T18:18:20.571031+0000","last_became_active":"2023-12-19T18:18:19.568155+0000","last_became_peered":"2023-12-19T18:18:19.568155+0000","last_unstale":"2023-12-19T18:18:20.571031+0000","last_undegraded":"2023-12-19T18:18:20.571031+0000","last_fullsized":"2023-12-19T18:18:20.571031+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:28.854918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791679+0000","last_change":"2023-12-19T18:18:19.587652+0000","last_active":"2023-12-19T18:18:20.791679+0000","last_peered":"2023-12-19T18:18:20.791679+0000","last_clean":"2023-12-19T18:18:20.791679+0000","last_became_active":"2023-12-19T18:18:19.583977+0000","last_became_peered":"2023-12-19T18:18:19.583977+0000","last_unstale":"2023-12-19T18:18:20.791679+0000","last_undegraded":"2023-12-19T18:18:20.791679+0000","last_fullsized":"2023-12-19T18:18:20.791679+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:17.867996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791661+0000","last_change":"2023-12-19T18:18:19.570641+0000","last_active":"2023-12-19T18:18:20.791661+0000","last_peered":"2023-12-19T18:18:20.791661+0000","last_clean":"2023-12-19T18:18:20.791661+0000","last_became_active":"2023-12-19T18:18:19.570230+0000","last_became_peered":"2023-12-19T18:18:19.570230+0000","last_unstale":"2023-12-19T18:18:20.791661+0000","last_undegraded":"2023-12-19T18:18:20.791661+0000","last_fullsized":"2023-12-19T18:18:20.791661+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:51.311268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791719+0000","last_change":"2023-12-19T18:18:19.585897+0000","last_active":"2023-12-19T18:18:20.791719+0000","last_peered":"2023-12-19T18:18:20.791719+0000","last_clean":"2023-12-19T18:18:20.791719+0000","last_became_active":"2023-12-19T18:18:19.578143+0000","last_became_peered":"2023-12-19T18:18:19.578143+0000","last_unstale":"2023-12-19T18:18:20.791719+0000","last_undegraded":"2023-12-19T18:18:20.791719+0000","last_fullsized":"2023-12-19T18:18:20.791719+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:54:55.928387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855246+0000","last_change":"2023-12-19T18:18:19.593987+0000","last_active":"2023-12-19T18:18:20.855246+0000","last_peered":"2023-12-19T18:18:20.855246+0000","last_clean":"2023-12-19T18:18:20.855246+0000","last_became_active":"2023-12-19T18:18:19.591467+0000","last_became_peered":"2023-12-19T18:18:19.591467+0000","last_unstale":"2023-12-19T18:18:20.855246+0000","last_undegraded":"2023-12-19T18:18:20.855246+0000","last_fullsized":"2023-12-19T18:18:20.855246+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:48:24.682318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855894+0000","last_change":"2023-12-19T18:18:19.585647+0000","last_active":"2023-12-19T18:18:20.855894+0000","last_peered":"2023-12-19T18:18:20.855894+0000","last_clean":"2023-12-19T18:18:20.855894+0000","last_became_active":"2023-12-19T18:18:19.585355+0000","last_became_peered":"2023-12-19T18:18:19.585355+0000","last_unstale":"2023-12-19T18:18:20.855894+0000","last_undegraded":"2023-12-19T18:18:20.855894+0000","last_fullsized":"2023-12-19T18:18:20.855894+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:49.021053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.857259+0000","last_change":"2023-12-19T18:18:19.585633+0000","last_active":"2023-12-19T18:18:20.857259+0000","last_peered":"2023-12-19T18:18:20.857259+0000","last_clean":"2023-12-19T18:18:20.857259+0000","last_became_active":"2023-12-19T18:18:19.584124+0000","last_became_peered":"2023-12-19T18:18:19.584124+0000","last_unstale":"2023-12-19T18:18:20.857259+0000","last_undegraded":"2023-12-19T18:18:20.857259+0000","last_fullsized":"2023-12-19T18:18:20.857259+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:45.244289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856913+0000","last_change":"2023-12-19T18:18:19.591381+0000","last_active":"2023-12-19T18:18:20.856913+0000","last_peered":"2023-12-19T18:18:20.856913+0000","last_clean":"2023-12-19T18:18:20.856913+0000","last_became_active":"2023-12-19T18:18:19.591088+0000","last_became_peered":"2023-12-19T18:18:19.591088+0000","last_unstale":"2023-12-19T18:18:20.856913+0000","last_undegraded":"2023-12-19T18:18:20.856913+0000","last_fullsized":"2023-12-19T18:18:20.856913+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:50.206100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856081+0000","last_change":"2023-12-19T18:18:19.590556+0000","last_active":"2023-12-19T18:18:20.856081+0000","last_peered":"2023-12-19T18:18:20.856081+0000","last_clean":"2023-12-19T18:18:20.856081+0000","last_became_active":"2023-12-19T18:18:19.590315+0000","last_became_peered":"2023-12-19T18:18:19.590315+0000","last_unstale":"2023-12-19T18:18:20.856081+0000","last_undegraded":"2023-12-19T18:18:20.856081+0000","last_fullsized":"2023-12-19T18:18:20.856081+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:24.266208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569308+0000","last_change":"2023-12-19T18:18:19.594931+0000","last_active":"2023-12-19T18:18:20.569308+0000","last_peered":"2023-12-19T18:18:20.569308+0000","last_clean":"2023-12-19T18:18:20.569308+0000","last_became_active":"2023-12-19T18:18:19.594676+0000","last_became_peered":"2023-12-19T18:18:19.594676+0000","last_unstale":"2023-12-19T18:18:20.569308+0000","last_undegraded":"2023-12-19T18:18:20.569308+0000","last_fullsized":"2023-12-19T18:18:20.569308+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:14:28.152600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569066+0000","last_change":"2023-12-19T18:18:19.579261+0000","last_active":"2023-12-19T18:18:20.569066+0000","last_peered":"2023-12-19T18:18:20.569066+0000","last_clean":"2023-12-19T18:18:20.569066+0000","last_became_active":"2023-12-19T18:18:19.578977+0000","last_became_peered":"2023-12-19T18:18:19.578977+0000","last_unstale":"2023-12-19T18:18:20.569066+0000","last_undegraded":"2023-12-19T18:18:20.569066+0000","last_fullsized":"2023-12-19T18:18:20.569066+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:51:24.364308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856613+0000","last_change":"2023-12-19T18:18:19.583658+0000","last_active":"2023-12-19T18:18:20.856613+0000","last_peered":"2023-12-19T18:18:20.856613+0000","last_clean":"2023-12-19T18:18:20.856613+0000","last_became_active":"2023-12-19T18:18:19.583370+0000","last_became_peered":"2023-12-19T18:18:19.583370+0000","last_unstale":"2023-12-19T18:18:20.856613+0000","last_undegraded":"2023-12-19T18:18:20.856613+0000","last_fullsized":"2023-12-19T18:18:20.856613+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:05.578139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855456+0000","last_change":"2023-12-19T18:18:19.585514+0000","last_active":"2023-12-19T18:18:20.855456+0000","last_peered":"2023-12-19T18:18:20.855456+0000","last_clean":"2023-12-19T18:18:20.855456+0000","last_became_active":"2023-12-19T18:18:19.584938+0000","last_became_peered":"2023-12-19T18:18:19.584938+0000","last_unstale":"2023-12-19T18:18:20.855456+0000","last_undegraded":"2023-12-19T18:18:20.855456+0000","last_fullsized":"2023-12-19T18:18:20.855456+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:48:24.682318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791965+0000","last_change":"2023-12-19T18:18:19.583858+0000","last_active":"2023-12-19T18:18:20.791965+0000","last_peered":"2023-12-19T18:18:20.791965+0000","last_clean":"2023-12-19T18:18:20.791965+0000","last_became_active":"2023-12-19T18:18:19.583559+0000","last_became_peered":"2023-12-19T18:18:19.583559+0000","last_unstale":"2023-12-19T18:18:20.791965+0000","last_undegraded":"2023-12-19T18:18:20.791965+0000","last_fullsized":"2023-12-19T18:18:20.791965+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:18.093251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570937+0000","last_change":"2023-12-19T18:18:19.588236+0000","last_active":"2023-12-19T18:18:20.570937+0000","last_peered":"2023-12-19T18:18:20.570937+0000","last_clean":"2023-12-19T18:18:20.570937+0000","last_became_active":"2023-12-19T18:18:19.587925+0000","last_became_peered":"2023-12-19T18:18:19.587925+0000","last_unstale":"2023-12-19T18:18:20.570937+0000","last_undegraded":"2023-12-19T18:18:20.570937+0000","last_fullsized":"2023-12-19T18:18:20.570937+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:18:00.411251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.857067+0000","last_change":"2023-12-19T18:18:19.582852+0000","last_active":"2023-12-19T18:18:20.857067+0000","last_peered":"2023-12-19T18:18:20.857067+0000","last_clean":"2023-12-19T18:18:20.857067+0000","last_became_active":"2023-12-19T18:18:19.582239+0000","last_became_peered":"2023-12-19T18:18:19.582239+0000","last_unstale":"2023-12-19T18:18:20.857067+0000","last_undegraded":"2023-12-19T18:18:20.857067+0000","last_fullsized":"2023-12-19T18:18:20.857067+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:38.446159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791965+0000","last_change":"2023-12-19T18:18:19.581125+0000","last_active":"2023-12-19T18:18:20.791965+0000","last_peered":"2023-12-19T18:18:20.791965+0000","last_clean":"2023-12-19T18:18:20.791965+0000","last_became_active":"2023-12-19T18:18:19.580831+0000","last_became_peered":"2023-12-19T18:18:19.580831+0000","last_unstale":"2023-12-19T18:18:20.791965+0000","last_undegraded":"2023-12-19T18:18:20.791965+0000","last_fullsized":"2023-12-19T18:18:20.791965+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:01:30.699377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855858+0000","last_change":"2023-12-19T18:18:19.581884+0000","last_active":"2023-12-19T18:18:20.855858+0000","last_peered":"2023-12-19T18:18:20.855858+0000","last_clean":"2023-12-19T18:18:20.855858+0000","last_became_active":"2023-12-19T18:18:19.580389+0000","last_became_peered":"2023-12-19T18:18:19.580389+0000","last_unstale":"2023-12-19T18:18:20.855858+0000","last_undegraded":"2023-12-19T18:18:20.855858+0000","last_fullsized":"2023-12-19T18:18:20.855858+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:54.364546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791653+0000","last_change":"2023-12-19T18:18:19.570466+0000","last_active":"2023-12-19T18:18:20.791653+0000","last_peered":"2023-12-19T18:18:20.791653+0000","last_clean":"2023-12-19T18:18:20.791653+0000","last_became_active":"2023-12-19T18:18:19.569899+0000","last_became_peered":"2023-12-19T18:18:19.569899+0000","last_unstale":"2023-12-19T18:18:20.791653+0000","last_undegraded":"2023-12-19T18:18:20.791653+0000","last_fullsized":"2023-12-19T18:18:20.791653+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:31.714140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570297+0000","last_change":"2023-12-19T18:18:19.574972+0000","last_active":"2023-12-19T18:18:20.570297+0000","last_peered":"2023-12-19T18:18:20.570297+0000","last_clean":"2023-12-19T18:18:20.570297+0000","last_became_active":"2023-12-19T18:18:19.574500+0000","last_became_peered":"2023-12-19T18:18:19.574500+0000","last_unstale":"2023-12-19T18:18:20.570297+0000","last_undegraded":"2023-12-19T18:18:20.570297+0000","last_fullsized":"2023-12-19T18:18:20.570297+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:31:26.474823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.857160+0000","last_change":"2023-12-19T18:18:19.584815+0000","last_active":"2023-12-19T18:18:20.857160+0000","last_peered":"2023-12-19T18:18:20.857160+0000","last_clean":"2023-12-19T18:18:20.857160+0000","last_became_active":"2023-12-19T18:18:19.583340+0000","last_became_peered":"2023-12-19T18:18:19.583340+0000","last_unstale":"2023-12-19T18:18:20.857160+0000","last_undegraded":"2023-12-19T18:18:20.857160+0000","last_fullsized":"2023-12-19T18:18:20.857160+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:17:39.917070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"253'4","reported_seq":15,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856094+0000","last_change":"2023-12-19T18:18:19.164806+0000","last_active":"2023-12-19T18:18:20.856094+0000","last_peered":"2023-12-19T18:18:20.856094+0000","last_clean":"2023-12-19T18:18:20.856094+0000","last_became_active":"2023-12-19T18:18:19.164560+0000","last_became_peered":"2023-12-19T18:18:19.164560+0000","last_unstale":"2023-12-19T18:18:20.856094+0000","last_undegraded":"2023-12-19T18:18:20.856094+0000","last_fullsized":"2023-12-19T18:18:20.856094+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:44.401553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791999+0000","last_change":"2023-12-19T18:18:19.571559+0000","last_active":"2023-12-19T18:18:20.791999+0000","last_peered":"2023-12-19T18:18:20.791999+0000","last_clean":"2023-12-19T18:18:20.791999+0000","last_became_active":"2023-12-19T18:18:19.571303+0000","last_became_peered":"2023-12-19T18:18:19.571303+0000","last_unstale":"2023-12-19T18:18:20.791999+0000","last_undegraded":"2023-12-19T18:18:20.791999+0000","last_fullsized":"2023-12-19T18:18:20.791999+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:51:19.629217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"253'1","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.792034+0000","last_change":"2023-12-19T18:18:19.172686+0000","last_active":"2023-12-19T18:18:20.792034+0000","last_peered":"2023-12-19T18:18:20.792034+0000","last_clean":"2023-12-19T18:18:20.792034+0000","last_became_active":"2023-12-19T18:18:19.172357+0000","last_became_peered":"2023-12-19T18:18:19.172357+0000","last_unstale":"2023-12-19T18:18:20.792034+0000","last_undegraded":"2023-12-19T18:18:20.792034+0000","last_fullsized":"2023-12-19T18:18:20.792034+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:58:13.987113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856876+0000","last_change":"2023-12-19T18:18:19.584695+0000","last_active":"2023-12-19T18:18:20.856876+0000","last_peered":"2023-12-19T18:18:20.856876+0000","last_clean":"2023-12-19T18:18:20.856876+0000","last_became_active":"2023-12-19T18:18:19.580748+0000","last_became_peered":"2023-12-19T18:18:19.580748+0000","last_unstale":"2023-12-19T18:18:20.856876+0000","last_undegraded":"2023-12-19T18:18:20.856876+0000","last_fullsized":"2023-12-19T18:18:20.856876+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:27.782019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"253'2","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856355+0000","last_change":"2023-12-19T18:18:19.168516+0000","last_active":"2023-12-19T18:18:20.856355+0000","last_peered":"2023-12-19T18:18:20.856355+0000","last_clean":"2023-12-19T18:18:20.856355+0000","last_became_active":"2023-12-19T18:18:19.168109+0000","last_became_peered":"2023-12-19T18:18:19.168109+0000","last_unstale":"2023-12-19T18:18:20.856355+0000","last_undegraded":"2023-12-19T18:18:20.856355+0000","last_fullsized":"2023-12-19T18:18:20.856355+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:25.760030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570751+0000","last_change":"2023-12-19T18:18:19.578243+0000","last_active":"2023-12-19T18:18:20.570751+0000","last_peered":"2023-12-19T18:18:20.570751+0000","last_clean":"2023-12-19T18:18:20.570751+0000","last_became_active":"2023-12-19T18:18:19.577973+0000","last_became_peered":"2023-12-19T18:18:19.577973+0000","last_unstale":"2023-12-19T18:18:20.570751+0000","last_undegraded":"2023-12-19T18:18:20.570751+0000","last_fullsized":"2023-12-19T18:18:20.570751+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:49:33.985436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"253'2","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856166+0000","last_change":"2023-12-19T18:18:19.167573+0000","last_active":"2023-12-19T18:18:20.856166+0000","last_peered":"2023-12-19T18:18:20.856166+0000","last_clean":"2023-12-19T18:18:20.856166+0000","last_became_active":"2023-12-19T18:18:19.167302+0000","last_became_peered":"2023-12-19T18:18:19.167302+0000","last_unstale":"2023-12-19T18:18:20.856166+0000","last_undegraded":"2023-12-19T18:18:20.856166+0000","last_fullsized":"2023-12-19T18:18:20.856166+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:58:37.805961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569096+0000","last_change":"2023-12-19T18:18:19.584233+0000","last_active":"2023-12-19T18:18:20.569096+0000","last_peered":"2023-12-19T18:18:20.569096+0000","last_clean":"2023-12-19T18:18:20.569096+0000","last_became_active":"2023-12-19T18:18:19.583963+0000","last_became_peered":"2023-12-19T18:18:19.583963+0000","last_unstale":"2023-12-19T18:18:20.569096+0000","last_undegraded":"2023-12-19T18:18:20.569096+0000","last_fullsized":"2023-12-19T18:18:20.569096+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:03:08.883628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"253'1","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856776+0000","last_change":"2023-12-19T18:18:19.171282+0000","last_active":"2023-12-19T18:18:20.856776+0000","last_peered":"2023-12-19T18:18:20.856776+0000","last_clean":"2023-12-19T18:18:20.856776+0000","last_became_active":"2023-12-19T18:18:19.170854+0000","last_became_peered":"2023-12-19T18:18:19.170854+0000","last_unstale":"2023-12-19T18:18:20.856776+0000","last_undegraded":"2023-12-19T18:18:20.856776+0000","last_fullsized":"2023-12-19T18:18:20.856776+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:51:08.618256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855842+0000","last_change":"2023-12-19T18:18:19.572802+0000","last_active":"2023-12-19T18:18:20.855842+0000","last_peered":"2023-12-19T18:18:20.855842+0000","last_clean":"2023-12-19T18:18:20.855842+0000","last_became_active":"2023-12-19T18:18:19.572003+0000","last_became_peered":"2023-12-19T18:18:19.572003+0000","last_unstale":"2023-12-19T18:18:20.855842+0000","last_undegraded":"2023-12-19T18:18:20.855842+0000","last_fullsized":"2023-12-19T18:18:20.855842+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:33:54.942965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"253'1","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855835+0000","last_change":"2023-12-19T18:18:19.168565+0000","last_active":"2023-12-19T18:18:20.855835+0000","last_peered":"2023-12-19T18:18:20.855835+0000","last_clean":"2023-12-19T18:18:20.855835+0000","last_became_active":"2023-12-19T18:18:19.168222+0000","last_became_peered":"2023-12-19T18:18:19.168222+0000","last_unstale":"2023-12-19T18:18:20.855835+0000","last_undegraded":"2023-12-19T18:18:20.855835+0000","last_fullsized":"2023-12-19T18:18:20.855835+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:25.760030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855257+0000","last_change":"2023-12-19T18:18:19.585278+0000","last_active":"2023-12-19T18:18:20.855257+0000","last_peered":"2023-12-19T18:18:20.855257+0000","last_clean":"2023-12-19T18:18:20.855257+0000","last_became_active":"2023-12-19T18:18:19.584932+0000","last_became_peered":"2023-12-19T18:18:19.584932+0000","last_unstale":"2023-12-19T18:18:20.855257+0000","last_undegraded":"2023-12-19T18:18:20.855257+0000","last_fullsized":"2023-12-19T18:18:20.855257+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:42:38.379799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570255+0000","last_change":"2023-12-19T18:18:19.165444+0000","last_active":"2023-12-19T18:18:20.570255+0000","last_peered":"2023-12-19T18:18:20.570255+0000","last_clean":"2023-12-19T18:18:20.570255+0000","last_became_active":"2023-12-19T18:18:19.165170+0000","last_became_peered":"2023-12-19T18:18:19.165170+0000","last_unstale":"2023-12-19T18:18:20.570255+0000","last_undegraded":"2023-12-19T18:18:20.570255+0000","last_fullsized":"2023-12-19T18:18:20.570255+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:35:31.432971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856859+0000","last_change":"2023-12-19T18:18:19.584245+0000","last_active":"2023-12-19T18:18:20.856859+0000","last_peered":"2023-12-19T18:18:20.856859+0000","last_clean":"2023-12-19T18:18:20.856859+0000","last_became_active":"2023-12-19T18:18:19.582819+0000","last_became_peered":"2023-12-19T18:18:19.582819+0000","last_unstale":"2023-12-19T18:18:20.856859+0000","last_undegraded":"2023-12-19T18:18:20.856859+0000","last_fullsized":"2023-12-19T18:18:20.856859+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:51:55.767484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856658+0000","last_change":"2023-12-19T18:18:19.171413+0000","last_active":"2023-12-19T18:18:20.856658+0000","last_peered":"2023-12-19T18:18:20.856658+0000","last_clean":"2023-12-19T18:18:20.856658+0000","last_became_active":"2023-12-19T18:18:19.170374+0000","last_became_peered":"2023-12-19T18:18:19.170374+0000","last_unstale":"2023-12-19T18:18:20.856658+0000","last_undegraded":"2023-12-19T18:18:20.856658+0000","last_fullsized":"2023-12-19T18:18:20.856658+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:31:25.466452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.857226+0000","last_change":"2023-12-19T18:18:19.585005+0000","last_active":"2023-12-19T18:18:20.857226+0000","last_peered":"2023-12-19T18:18:20.857226+0000","last_clean":"2023-12-19T18:18:20.857226+0000","last_became_active":"2023-12-19T18:18:19.583722+0000","last_became_peered":"2023-12-19T18:18:19.583722+0000","last_unstale":"2023-12-19T18:18:20.857226+0000","last_undegraded":"2023-12-19T18:18:20.857226+0000","last_fullsized":"2023-12-19T18:18:20.857226+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:34:22.743194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856122+0000","last_change":"2023-12-19T18:18:19.172613+0000","last_active":"2023-12-19T18:18:20.856122+0000","last_peered":"2023-12-19T18:18:20.856122+0000","last_clean":"2023-12-19T18:18:20.856122+0000","last_became_active":"2023-12-19T18:18:19.171540+0000","last_became_peered":"2023-12-19T18:18:19.171540+0000","last_unstale":"2023-12-19T18:18:20.856122+0000","last_undegraded":"2023-12-19T18:18:20.856122+0000","last_fullsized":"2023-12-19T18:18:20.856122+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:46:18.299306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856006+0000","last_change":"2023-12-19T18:18:19.577767+0000","last_active":"2023-12-19T18:18:20.856006+0000","last_peered":"2023-12-19T18:18:20.856006+0000","last_clean":"2023-12-19T18:18:20.856006+0000","last_became_active":"2023-12-19T18:18:19.577494+0000","last_became_peered":"2023-12-19T18:18:19.577494+0000","last_unstale":"2023-12-19T18:18:20.856006+0000","last_undegraded":"2023-12-19T18:18:20.856006+0000","last_fullsized":"2023-12-19T18:18:20.856006+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:40.993599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856154+0000","last_change":"2023-12-19T18:18:19.172871+0000","last_active":"2023-12-19T18:18:20.856154+0000","last_peered":"2023-12-19T18:18:20.856154+0000","last_clean":"2023-12-19T18:18:20.856154+0000","last_became_active":"2023-12-19T18:18:19.172561+0000","last_became_peered":"2023-12-19T18:18:19.172561+0000","last_unstale":"2023-12-19T18:18:20.856154+0000","last_undegraded":"2023-12-19T18:18:20.856154+0000","last_fullsized":"2023-12-19T18:18:20.856154+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:06:09.659288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791845+0000","last_change":"2023-12-19T18:18:19.575004+0000","last_active":"2023-12-19T18:18:20.791845+0000","last_peered":"2023-12-19T18:18:20.791845+0000","last_clean":"2023-12-19T18:18:20.791845+0000","last_became_active":"2023-12-19T18:18:19.574524+0000","last_became_peered":"2023-12-19T18:18:19.574524+0000","last_unstale":"2023-12-19T18:18:20.791845+0000","last_undegraded":"2023-12-19T18:18:20.791845+0000","last_fullsized":"2023-12-19T18:18:20.791845+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:18:00.411251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"253'3","reported_seq":14,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856389+0000","last_change":"2023-12-19T18:18:19.175923+0000","last_active":"2023-12-19T18:18:20.856389+0000","last_peered":"2023-12-19T18:18:20.856389+0000","last_clean":"2023-12-19T18:18:20.856389+0000","last_became_active":"2023-12-19T18:18:19.175660+0000","last_became_peered":"2023-12-19T18:18:19.175660+0000","last_unstale":"2023-12-19T18:18:20.856389+0000","last_undegraded":"2023-12-19T18:18:20.856389+0000","last_fullsized":"2023-12-19T18:18:20.856389+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:54:53.266347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855782+0000","last_change":"2023-12-19T18:18:19.580468+0000","last_active":"2023-12-19T18:18:20.855782+0000","last_peered":"2023-12-19T18:18:20.855782+0000","last_clean":"2023-12-19T18:18:20.855782+0000","last_became_active":"2023-12-19T18:18:19.580200+0000","last_became_peered":"2023-12-19T18:18:19.580200+0000","last_unstale":"2023-12-19T18:18:20.855782+0000","last_undegraded":"2023-12-19T18:18:20.855782+0000","last_fullsized":"2023-12-19T18:18:20.855782+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:31:49.737327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"253'6","reported_seq":17,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855704+0000","last_change":"2023-12-19T18:18:19.168003+0000","last_active":"2023-12-19T18:18:20.855704+0000","last_peered":"2023-12-19T18:18:20.855704+0000","last_clean":"2023-12-19T18:18:20.855704+0000","last_became_active":"2023-12-19T18:18:19.167688+0000","last_became_peered":"2023-12-19T18:18:19.167688+0000","last_unstale":"2023-12-19T18:18:20.855704+0000","last_undegraded":"2023-12-19T18:18:20.855704+0000","last_fullsized":"2023-12-19T18:18:20.855704+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:00.867540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855721+0000","last_change":"2023-12-19T18:18:19.588507+0000","last_active":"2023-12-19T18:18:20.855721+0000","last_peered":"2023-12-19T18:18:20.855721+0000","last_clean":"2023-12-19T18:18:20.855721+0000","last_became_active":"2023-12-19T18:18:19.588220+0000","last_became_peered":"2023-12-19T18:18:19.588220+0000","last_unstale":"2023-12-19T18:18:20.855721+0000","last_undegraded":"2023-12-19T18:18:20.855721+0000","last_fullsized":"2023-12-19T18:18:20.855721+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:46:07.926163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"253'2","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570856+0000","last_change":"2023-12-19T18:18:19.164280+0000","last_active":"2023-12-19T18:18:20.570856+0000","last_peered":"2023-12-19T18:18:20.570856+0000","last_clean":"2023-12-19T18:18:20.570856+0000","last_became_active":"2023-12-19T18:18:19.163443+0000","last_became_peered":"2023-12-19T18:18:19.163443+0000","last_unstale":"2023-12-19T18:18:20.570856+0000","last_undegraded":"2023-12-19T18:18:20.570856+0000","last_fullsized":"2023-12-19T18:18:20.570856+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:00:45.109361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856037+0000","last_change":"2023-12-19T18:18:19.587328+0000","last_active":"2023-12-19T18:18:20.856037+0000","last_peered":"2023-12-19T18:18:20.856037+0000","last_clean":"2023-12-19T18:18:20.856037+0000","last_became_active":"2023-12-19T18:18:19.586921+0000","last_became_peered":"2023-12-19T18:18:19.586921+0000","last_unstale":"2023-12-19T18:18:20.856037+0000","last_undegraded":"2023-12-19T18:18:20.856037+0000","last_fullsized":"2023-12-19T18:18:20.856037+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:04:19.617805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569148+0000","last_change":"2023-12-19T18:18:19.166958+0000","last_active":"2023-12-19T18:18:20.569148+0000","last_peered":"2023-12-19T18:18:20.569148+0000","last_clean":"2023-12-19T18:18:20.569148+0000","last_became_active":"2023-12-19T18:18:19.166685+0000","last_became_peered":"2023-12-19T18:18:19.166685+0000","last_unstale":"2023-12-19T18:18:20.569148+0000","last_undegraded":"2023-12-19T18:18:20.569148+0000","last_fullsized":"2023-12-19T18:18:20.569148+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:31:28.614678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855457+0000","last_change":"2023-12-19T18:18:19.597100+0000","last_active":"2023-12-19T18:18:20.855457+0000","last_peered":"2023-12-19T18:18:20.855457+0000","last_clean":"2023-12-19T18:18:20.855457+0000","last_became_active":"2023-12-19T18:18:19.596783+0000","last_became_peered":"2023-12-19T18:18:19.596783+0000","last_unstale":"2023-12-19T18:18:20.855457+0000","last_undegraded":"2023-12-19T18:18:20.855457+0000","last_fullsized":"2023-12-19T18:18:20.855457+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:06:03.752660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856110+0000","last_change":"2023-12-19T18:18:19.171189+0000","last_active":"2023-12-19T18:18:20.856110+0000","last_peered":"2023-12-19T18:18:20.856110+0000","last_clean":"2023-12-19T18:18:20.856110+0000","last_became_active":"2023-12-19T18:18:19.170861+0000","last_became_peered":"2023-12-19T18:18:19.170861+0000","last_unstale":"2023-12-19T18:18:20.856110+0000","last_undegraded":"2023-12-19T18:18:20.856110+0000","last_fullsized":"2023-12-19T18:18:20.856110+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:27.846546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855410+0000","last_change":"2023-12-19T18:18:19.580433+0000","last_active":"2023-12-19T18:18:20.855410+0000","last_peered":"2023-12-19T18:18:20.855410+0000","last_clean":"2023-12-19T18:18:20.855410+0000","last_became_active":"2023-12-19T18:18:19.580162+0000","last_became_peered":"2023-12-19T18:18:19.580162+0000","last_unstale":"2023-12-19T18:18:20.855410+0000","last_undegraded":"2023-12-19T18:18:20.855410+0000","last_fullsized":"2023-12-19T18:18:20.855410+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:33:54.942965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"253'8","reported_seq":19,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855373+0000","last_change":"2023-12-19T18:18:19.159958+0000","last_active":"2023-12-19T18:18:20.855373+0000","last_peered":"2023-12-19T18:18:20.855373+0000","last_clean":"2023-12-19T18:18:20.855373+0000","last_became_active":"2023-12-19T18:18:19.159675+0000","last_became_peered":"2023-12-19T18:18:19.159675+0000","last_unstale":"2023-12-19T18:18:20.855373+0000","last_undegraded":"2023-12-19T18:18:20.855373+0000","last_fullsized":"2023-12-19T18:18:20.855373+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:46:18.299306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791907+0000","last_change":"2023-12-19T18:18:19.575961+0000","last_active":"2023-12-19T18:18:20.791907+0000","last_peered":"2023-12-19T18:18:20.791907+0000","last_clean":"2023-12-19T18:18:20.791907+0000","last_became_active":"2023-12-19T18:18:19.575350+0000","last_became_peered":"2023-12-19T18:18:19.575350+0000","last_unstale":"2023-12-19T18:18:20.791907+0000","last_undegraded":"2023-12-19T18:18:20.791907+0000","last_fullsized":"2023-12-19T18:18:20.791907+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:07.028531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"253'1","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.571376+0000","last_change":"2023-12-19T18:18:19.170499+0000","last_active":"2023-12-19T18:18:20.571376+0000","last_peered":"2023-12-19T18:18:20.571376+0000","last_clean":"2023-12-19T18:18:20.571376+0000","last_became_active":"2023-12-19T18:18:19.170208+0000","last_became_peered":"2023-12-19T18:18:19.170208+0000","last_unstale":"2023-12-19T18:18:20.571376+0000","last_undegraded":"2023-12-19T18:18:20.571376+0000","last_fullsized":"2023-12-19T18:18:20.571376+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:42.153499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856372+0000","last_change":"2023-12-19T18:18:19.592701+0000","last_active":"2023-12-19T18:18:20.856372+0000","last_peered":"2023-12-19T18:18:20.856372+0000","last_clean":"2023-12-19T18:18:20.856372+0000","last_became_active":"2023-12-19T18:18:19.592373+0000","last_became_peered":"2023-12-19T18:18:19.592373+0000","last_unstale":"2023-12-19T18:18:20.856372+0000","last_undegraded":"2023-12-19T18:18:20.856372+0000","last_fullsized":"2023-12-19T18:18:20.856372+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:22.953633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"253'2","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856302+0000","last_change":"2023-12-19T18:18:19.159393+0000","last_active":"2023-12-19T18:18:20.856302+0000","last_peered":"2023-12-19T18:18:20.856302+0000","last_clean":"2023-12-19T18:18:20.856302+0000","last_became_active":"2023-12-19T18:18:19.159111+0000","last_became_peered":"2023-12-19T18:18:19.159111+0000","last_unstale":"2023-12-19T18:18:20.856302+0000","last_undegraded":"2023-12-19T18:18:20.856302+0000","last_fullsized":"2023-12-19T18:18:20.856302+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:24.242078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791964+0000","last_change":"2023-12-19T18:18:19.576960+0000","last_active":"2023-12-19T18:18:20.791964+0000","last_peered":"2023-12-19T18:18:20.791964+0000","last_clean":"2023-12-19T18:18:20.791964+0000","last_became_active":"2023-12-19T18:18:19.576683+0000","last_became_peered":"2023-12-19T18:18:19.576683+0000","last_unstale":"2023-12-19T18:18:20.791964+0000","last_undegraded":"2023-12-19T18:18:20.791964+0000","last_fullsized":"2023-12-19T18:18:20.791964+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:25:13.980357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"253'1","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856063+0000","last_change":"2023-12-19T18:18:19.172950+0000","last_active":"2023-12-19T18:18:20.856063+0000","last_peered":"2023-12-19T18:18:20.856063+0000","last_clean":"2023-12-19T18:18:20.856063+0000","last_became_active":"2023-12-19T18:18:19.172632+0000","last_became_peered":"2023-12-19T18:18:19.172632+0000","last_unstale":"2023-12-19T18:18:20.856063+0000","last_undegraded":"2023-12-19T18:18:20.856063+0000","last_fullsized":"2023-12-19T18:18:20.856063+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:05.539474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570842+0000","last_change":"2023-12-19T18:18:19.587143+0000","last_active":"2023-12-19T18:18:20.570842+0000","last_peered":"2023-12-19T18:18:20.570842+0000","last_clean":"2023-12-19T18:18:20.570842+0000","last_became_active":"2023-12-19T18:18:19.585269+0000","last_became_peered":"2023-12-19T18:18:19.585269+0000","last_unstale":"2023-12-19T18:18:20.570842+0000","last_undegraded":"2023-12-19T18:18:20.570842+0000","last_fullsized":"2023-12-19T18:18:20.570842+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:01:30.699377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.792036+0000","last_change":"2023-12-19T18:18:19.174353+0000","last_active":"2023-12-19T18:18:20.792036+0000","last_peered":"2023-12-19T18:18:20.792036+0000","last_clean":"2023-12-19T18:18:20.792036+0000","last_became_active":"2023-12-19T18:18:19.174051+0000","last_became_peered":"2023-12-19T18:18:19.174051+0000","last_unstale":"2023-12-19T18:18:20.792036+0000","last_undegraded":"2023-12-19T18:18:20.792036+0000","last_fullsized":"2023-12-19T18:18:20.792036+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:11.181824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855559+0000","last_change":"2023-12-19T18:18:19.577766+0000","last_active":"2023-12-19T18:18:20.855559+0000","last_peered":"2023-12-19T18:18:20.855559+0000","last_clean":"2023-12-19T18:18:20.855559+0000","last_became_active":"2023-12-19T18:18:19.577484+0000","last_became_peered":"2023-12-19T18:18:19.577484+0000","last_unstale":"2023-12-19T18:18:20.855559+0000","last_undegraded":"2023-12-19T18:18:20.855559+0000","last_fullsized":"2023-12-19T18:18:20.855559+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:36.541142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"253'2","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570985+0000","last_change":"2023-12-19T18:18:19.166552+0000","last_active":"2023-12-19T18:18:20.570985+0000","last_peered":"2023-12-19T18:18:20.570985+0000","last_clean":"2023-12-19T18:18:20.570985+0000","last_became_active":"2023-12-19T18:18:19.166288+0000","last_became_peered":"2023-12-19T18:18:19.166288+0000","last_unstale":"2023-12-19T18:18:20.570985+0000","last_undegraded":"2023-12-19T18:18:20.570985+0000","last_fullsized":"2023-12-19T18:18:20.570985+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:44.500321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856066+0000","last_change":"2023-12-19T18:18:19.591037+0000","last_active":"2023-12-19T18:18:20.856066+0000","last_peered":"2023-12-19T18:18:20.856066+0000","last_clean":"2023-12-19T18:18:20.856066+0000","last_became_active":"2023-12-19T18:18:19.590782+0000","last_became_peered":"2023-12-19T18:18:19.590782+0000","last_unstale":"2023-12-19T18:18:20.856066+0000","last_undegraded":"2023-12-19T18:18:20.856066+0000","last_fullsized":"2023-12-19T18:18:20.856066+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:47:57.808912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570613+0000","last_change":"2023-12-19T18:18:19.168858+0000","last_active":"2023-12-19T18:18:20.570613+0000","last_peered":"2023-12-19T18:18:20.570613+0000","last_clean":"2023-12-19T18:18:20.570613+0000","last_became_active":"2023-12-19T18:18:19.168561+0000","last_became_peered":"2023-12-19T18:18:19.168561+0000","last_unstale":"2023-12-19T18:18:20.570613+0000","last_undegraded":"2023-12-19T18:18:20.570613+0000","last_fullsized":"2023-12-19T18:18:20.570613+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:01:11.335521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855904+0000","last_change":"2023-12-19T18:18:19.585324+0000","last_active":"2023-12-19T18:18:20.855904+0000","last_peered":"2023-12-19T18:18:20.855904+0000","last_clean":"2023-12-19T18:18:20.855904+0000","last_became_active":"2023-12-19T18:18:19.584615+0000","last_became_peered":"2023-12-19T18:18:19.584615+0000","last_unstale":"2023-12-19T18:18:20.855904+0000","last_undegraded":"2023-12-19T18:18:20.855904+0000","last_fullsized":"2023-12-19T18:18:20.855904+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:02:20.434330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856216+0000","last_change":"2023-12-19T18:18:19.174032+0000","last_active":"2023-12-19T18:18:20.856216+0000","last_peered":"2023-12-19T18:18:20.856216+0000","last_clean":"2023-12-19T18:18:20.856216+0000","last_became_active":"2023-12-19T18:18:19.173764+0000","last_became_peered":"2023-12-19T18:18:19.173764+0000","last_unstale":"2023-12-19T18:18:20.856216+0000","last_undegraded":"2023-12-19T18:18:20.856216+0000","last_fullsized":"2023-12-19T18:18:20.856216+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:31:25.466452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.568938+0000","last_change":"2023-12-19T18:18:19.598180+0000","last_active":"2023-12-19T18:18:20.568938+0000","last_peered":"2023-12-19T18:18:20.568938+0000","last_clean":"2023-12-19T18:18:20.568938+0000","last_became_active":"2023-12-19T18:18:19.595299+0000","last_became_peered":"2023-12-19T18:18:19.595299+0000","last_unstale":"2023-12-19T18:18:20.568938+0000","last_undegraded":"2023-12-19T18:18:20.568938+0000","last_fullsized":"2023-12-19T18:18:20.568938+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:06.670616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570329+0000","last_change":"2023-12-19T18:18:19.164923+0000","last_active":"2023-12-19T18:18:20.570329+0000","last_peered":"2023-12-19T18:18:20.570329+0000","last_clean":"2023-12-19T18:18:20.570329+0000","last_became_active":"2023-12-19T18:18:19.164660+0000","last_became_peered":"2023-12-19T18:18:19.164660+0000","last_unstale":"2023-12-19T18:18:20.570329+0000","last_undegraded":"2023-12-19T18:18:20.570329+0000","last_fullsized":"2023-12-19T18:18:20.570329+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:35.595940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791932+0000","last_change":"2023-12-19T18:18:19.576132+0000","last_active":"2023-12-19T18:18:20.791932+0000","last_peered":"2023-12-19T18:18:20.791932+0000","last_clean":"2023-12-19T18:18:20.791932+0000","last_became_active":"2023-12-19T18:18:19.575711+0000","last_became_peered":"2023-12-19T18:18:19.575711+0000","last_unstale":"2023-12-19T18:18:20.791932+0000","last_undegraded":"2023-12-19T18:18:20.791932+0000","last_fullsized":"2023-12-19T18:18:20.791932+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:26:56.610626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"253'2","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.792073+0000","last_change":"2023-12-19T18:18:19.170075+0000","last_active":"2023-12-19T18:18:20.792073+0000","last_peered":"2023-12-19T18:18:20.792073+0000","last_clean":"2023-12-19T18:18:20.792073+0000","last_became_active":"2023-12-19T18:18:19.169795+0000","last_became_peered":"2023-12-19T18:18:19.169795+0000","last_unstale":"2023-12-19T18:18:20.792073+0000","last_undegraded":"2023-12-19T18:18:20.792073+0000","last_fullsized":"2023-12-19T18:18:20.792073+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:06:26.444241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856035+0000","last_change":"2023-12-19T18:18:19.581374+0000","last_active":"2023-12-19T18:18:20.856035+0000","last_peered":"2023-12-19T18:18:20.856035+0000","last_clean":"2023-12-19T18:18:20.856035+0000","last_became_active":"2023-12-19T18:18:19.581091+0000","last_became_peered":"2023-12-19T18:18:19.581091+0000","last_unstale":"2023-12-19T18:18:20.856035+0000","last_undegraded":"2023-12-19T18:18:20.856035+0000","last_fullsized":"2023-12-19T18:18:20.856035+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:47:57.808912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570854+0000","last_change":"2023-12-19T18:18:19.171868+0000","last_active":"2023-12-19T18:18:20.570854+0000","last_peered":"2023-12-19T18:18:20.570854+0000","last_clean":"2023-12-19T18:18:20.570854+0000","last_became_active":"2023-12-19T18:18:19.171566+0000","last_became_peered":"2023-12-19T18:18:19.171566+0000","last_unstale":"2023-12-19T18:18:20.570854+0000","last_undegraded":"2023-12-19T18:18:20.570854+0000","last_fullsized":"2023-12-19T18:18:20.570854+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:23.198353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.571060+0000","last_change":"2023-12-19T18:18:19.589854+0000","last_active":"2023-12-19T18:18:20.571060+0000","last_peered":"2023-12-19T18:18:20.571060+0000","last_clean":"2023-12-19T18:18:20.571060+0000","last_became_active":"2023-12-19T18:18:19.586190+0000","last_became_peered":"2023-12-19T18:18:19.586190+0000","last_unstale":"2023-12-19T18:18:20.571060+0000","last_undegraded":"2023-12-19T18:18:20.571060+0000","last_fullsized":"2023-12-19T18:18:20.571060+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:52:50.568291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.857091+0000","last_change":"2023-12-19T18:18:19.171097+0000","last_active":"2023-12-19T18:18:20.857091+0000","last_peered":"2023-12-19T18:18:20.857091+0000","last_clean":"2023-12-19T18:18:20.857091+0000","last_became_active":"2023-12-19T18:18:19.170690+0000","last_became_peered":"2023-12-19T18:18:19.170690+0000","last_unstale":"2023-12-19T18:18:20.857091+0000","last_undegraded":"2023-12-19T18:18:20.857091+0000","last_fullsized":"2023-12-19T18:18:20.857091+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:06:34.583280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570937+0000","last_change":"2023-12-19T18:18:19.580259+0000","last_active":"2023-12-19T18:18:20.570937+0000","last_peered":"2023-12-19T18:18:20.570937+0000","last_clean":"2023-12-19T18:18:20.570937+0000","last_became_active":"2023-12-19T18:18:19.579980+0000","last_became_peered":"2023-12-19T18:18:19.579980+0000","last_unstale":"2023-12-19T18:18:20.570937+0000","last_undegraded":"2023-12-19T18:18:20.570937+0000","last_fullsized":"2023-12-19T18:18:20.570937+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:51.311268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855697+0000","last_change":"2023-12-19T18:18:19.171492+0000","last_active":"2023-12-19T18:18:20.855697+0000","last_peered":"2023-12-19T18:18:20.855697+0000","last_clean":"2023-12-19T18:18:20.855697+0000","last_became_active":"2023-12-19T18:18:19.170962+0000","last_became_peered":"2023-12-19T18:18:19.170962+0000","last_unstale":"2023-12-19T18:18:20.855697+0000","last_undegraded":"2023-12-19T18:18:20.855697+0000","last_fullsized":"2023-12-19T18:18:20.855697+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:58:41.004354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791754+0000","last_change":"2023-12-19T18:18:19.581835+0000","last_active":"2023-12-19T18:18:20.791754+0000","last_peered":"2023-12-19T18:18:20.791754+0000","last_clean":"2023-12-19T18:18:20.791754+0000","last_became_active":"2023-12-19T18:18:19.581577+0000","last_became_peered":"2023-12-19T18:18:19.581577+0000","last_unstale":"2023-12-19T18:18:20.791754+0000","last_undegraded":"2023-12-19T18:18:20.791754+0000","last_fullsized":"2023-12-19T18:18:20.791754+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:15:04.921197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569148+0000","last_change":"2023-12-19T18:18:19.163272+0000","last_active":"2023-12-19T18:18:20.569148+0000","last_peered":"2023-12-19T18:18:20.569148+0000","last_clean":"2023-12-19T18:18:20.569148+0000","last_became_active":"2023-12-19T18:18:19.162992+0000","last_became_peered":"2023-12-19T18:18:19.162992+0000","last_unstale":"2023-12-19T18:18:20.569148+0000","last_undegraded":"2023-12-19T18:18:20.569148+0000","last_fullsized":"2023-12-19T18:18:20.569148+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:38:27.027379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791806+0000","last_change":"2023-12-19T18:18:19.571044+0000","last_active":"2023-12-19T18:18:20.791806+0000","last_peered":"2023-12-19T18:18:20.791806+0000","last_clean":"2023-12-19T18:18:20.791806+0000","last_became_active":"2023-12-19T18:18:19.570779+0000","last_became_peered":"2023-12-19T18:18:19.570779+0000","last_unstale":"2023-12-19T18:18:20.791806+0000","last_undegraded":"2023-12-19T18:18:20.791806+0000","last_fullsized":"2023-12-19T18:18:20.791806+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:54.881623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569182+0000","last_change":"2023-12-19T18:18:19.170818+0000","last_active":"2023-12-19T18:18:20.569182+0000","last_peered":"2023-12-19T18:18:20.569182+0000","last_clean":"2023-12-19T18:18:20.569182+0000","last_became_active":"2023-12-19T18:18:19.170523+0000","last_became_peered":"2023-12-19T18:18:19.170523+0000","last_unstale":"2023-12-19T18:18:20.569182+0000","last_undegraded":"2023-12-19T18:18:20.569182+0000","last_fullsized":"2023-12-19T18:18:20.569182+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:52:55.832348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791748+0000","last_change":"2023-12-19T18:18:19.589779+0000","last_active":"2023-12-19T18:18:20.791748+0000","last_peered":"2023-12-19T18:18:20.791748+0000","last_clean":"2023-12-19T18:18:20.791748+0000","last_became_active":"2023-12-19T18:18:19.587841+0000","last_became_peered":"2023-12-19T18:18:19.587841+0000","last_unstale":"2023-12-19T18:18:20.791748+0000","last_undegraded":"2023-12-19T18:18:20.791748+0000","last_fullsized":"2023-12-19T18:18:20.791748+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:25:48.739322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791795+0000","last_change":"2023-12-19T18:18:19.168943+0000","last_active":"2023-12-19T18:18:20.791795+0000","last_peered":"2023-12-19T18:18:20.791795+0000","last_clean":"2023-12-19T18:18:20.791795+0000","last_became_active":"2023-12-19T18:18:19.168650+0000","last_became_peered":"2023-12-19T18:18:19.168650+0000","last_unstale":"2023-12-19T18:18:20.791795+0000","last_undegraded":"2023-12-19T18:18:20.791795+0000","last_fullsized":"2023-12-19T18:18:20.791795+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:16.372429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570171+0000","last_change":"2023-12-19T18:18:19.596288+0000","last_active":"2023-12-19T18:18:20.570171+0000","last_peered":"2023-12-19T18:18:20.570171+0000","last_clean":"2023-12-19T18:18:20.570171+0000","last_became_active":"2023-12-19T18:18:19.595896+0000","last_became_peered":"2023-12-19T18:18:19.595896+0000","last_unstale":"2023-12-19T18:18:20.570171+0000","last_undegraded":"2023-12-19T18:18:20.570171+0000","last_fullsized":"2023-12-19T18:18:20.570171+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:31.714140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855666+0000","last_change":"2023-12-19T18:18:19.171495+0000","last_active":"2023-12-19T18:18:20.855666+0000","last_peered":"2023-12-19T18:18:20.855666+0000","last_clean":"2023-12-19T18:18:20.855666+0000","last_became_active":"2023-12-19T18:18:19.171133+0000","last_became_peered":"2023-12-19T18:18:19.171133+0000","last_unstale":"2023-12-19T18:18:20.855666+0000","last_undegraded":"2023-12-19T18:18:20.855666+0000","last_fullsized":"2023-12-19T18:18:20.855666+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:24.242078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856535+0000","last_change":"2023-12-19T18:18:19.595513+0000","last_active":"2023-12-19T18:18:20.856535+0000","last_peered":"2023-12-19T18:18:20.856535+0000","last_clean":"2023-12-19T18:18:20.856535+0000","last_became_active":"2023-12-19T18:18:19.595314+0000","last_became_peered":"2023-12-19T18:18:19.595314+0000","last_unstale":"2023-12-19T18:18:20.856535+0000","last_undegraded":"2023-12-19T18:18:20.856535+0000","last_fullsized":"2023-12-19T18:18:20.856535+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:16:47.558060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"239'390","reported_seq":1377,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-12-19T18:18:19.566574+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:18:19.566574+0000","last_peered":"2023-12-19T18:18:19.566574+0000","last_clean":"2023-12-19T18:18:19.566574+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:18:19.566574+0000","last_undegraded":"2023-12-19T18:18:19.566574+0000","last_fullsized":"2023-12-19T18:18:19.566574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2498,"num_read_kb":5235,"num_write":655,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2498,"num_read_kb":5235,"num_write":655,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":390,"ondisk_log_size":390,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510587,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48704,"kb_used_data":3704,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323136,"statfs":{"total":96636764160,"available":96586891264,"internally_reserved":0,"allocated":3792896,"data_stored":2305048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68500000000000005,"15min":0.71499999999999997},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79000000000000004,"5min":1.048,"15min":1.1180000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71299999999999997,"15min":0.73199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90900000000000003,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.79500000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":1.0389999999999999,"15min":1.0580000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.74199999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.70999999999999996}]},{"osd":2,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.73099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.0669999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.996,"5min":1.0820000000000001,"15min":1.236},"last":0.80900000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88100000000000001,"5min":1.002,"15min":1.079},"last":0.755},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.49399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85899999999999999,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.78000000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.441,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.302},"last":0.622},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.622,"15min":0.629},"min":{"1min":0.50800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.0209999999999999},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.90600000000000003,"5min":1.097,"15min":1.1299999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.55800000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95699999999999996,"5min":1.026,"15min":1.0800000000000001},"last":0.51400000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.61699999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.0469999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.093},"last":0.59799999999999998}]}]},{"osd":6,"up_from":15,"seq":64424510587,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48128,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323712,"statfs":{"total":96636764160,"available":96587481088,"internally_reserved":0,"allocated":3203072,"data_stored":1714722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90600000000000003,"5min":1.0289999999999999,"15min":1.042},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69299999999999995,"15min":0.72399999999999998},"min":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95999999999999996,"5min":1.224,"15min":1.224},"last":0.59799999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94799999999999995,"5min":1.103,"15min":1.103},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.746,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.58199999999999996}]},{"osd":2,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.73699999999999999},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.879,"5min":0.98999999999999999,"15min":1.111},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.748,"15min":0.77400000000000002},"min":{"1min":0.57799999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94799999999999995,"5min":1.046,"15min":1.1519999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.90300000000000002,"5min":1.268,"15min":1.268},"last":0.57599999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.49199999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.97499999999999998,"5min":1.194,"15min":1.194},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.67700000000000005},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.53500000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.92200000000000004,"5min":1.218,"15min":1.218},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.60999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.1200000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.67000000000000004}]}]},{"osd":5,"up_from":15,"seq":64424510587,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52216,"kb_used_data":3120,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319624,"statfs":{"total":96636764160,"available":96583294976,"internally_reserved":0,"allocated":3194880,"data_stored":1714610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.69599999999999995,"15min":0.72499999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.94999999999999996,"5min":1.002,"15min":1.0149999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.93100000000000005,"5min":0.96099999999999997,"15min":1.0229999999999999},"last":0.629}]},{"osd":1,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.497},"max":{"1min":0.84599999999999997,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.498,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97599999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59999999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.58599999999999997,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":1.0069999999999999,"5min":1.0509999999999999,"15min":1.2250000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.73199999999999998},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88600000000000001,"5min":1.004,"15min":1.4279999999999999},"last":0.61299999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.748,"15min":0.76200000000000001},"min":{"1min":0.56799999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.90400000000000003,"5min":1.0129999999999999,"15min":1.05},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93400000000000005,"5min":1.022,"15min":1.0329999999999999},"last":0.65700000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.81699999999999995,"5min":0.94499999999999995,"15min":1.0800000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94299999999999995,"5min":1.208,"15min":1.208},"last":0.441}]},{"osd":6,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65800000000000003,"15min":0.65700000000000003},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.85799999999999998,"5min":1.008,"15min":1.0449999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.501,"5min":0.439,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":1.169,"15min":1.169},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51100000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.079,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.68100000000000005}]}]},{"osd":4,"up_from":15,"seq":64424510587,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48140,"kb_used_data":3140,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323700,"statfs":{"total":96636764160,"available":96587468800,"internally_reserved":0,"allocated":3215360,"data_stored":1716230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.72599999999999998,"15min":0.73899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86699999999999999,"5min":1.036,"15min":1.109},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.88100000000000001,"5min":0.997,"15min":1.01},"last":0.71099999999999997}]},{"osd":1,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.752,"15min":0.753},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90500000000000003,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.753,"15min":0.76200000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86699999999999999,"5min":0.997,"15min":1.1020000000000001},"last":0.63400000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0289999999999999,"15min":1.071},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.77400000000000002,"15min":0.77300000000000002},"min":{"1min":0.51800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91800000000000004,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.69899999999999995}]},{"osd":3,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.751,"15min":0.751},"min":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84699999999999998,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.77000000000000002,"15min":0.78600000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.85999999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.59999999999999998}]},{"osd":5,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65400000000000003,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"max":{"1min":0.91000000000000003,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":0.89800000000000002,"5min":1.042,"15min":1.042},"last":0.60999999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":1.024,"15min":1.024},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73699999999999999,"15min":0.71499999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90900000000000003,"5min":1.0680000000000001,"15min":1.0760000000000001},"last":0.55400000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.41899999999999998},"max":{"1min":0.93500000000000005,"5min":1.087,"15min":1.087},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77500000000000002,"15min":0.76000000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.46999999999999997},"max":{"1min":0.98499999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65100000000000002}]}]},{"osd":3,"up_from":15,"seq":64424510587,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52232,"kb_used_data":3136,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319608,"statfs":{"total":96636764160,"available":96583278592,"internally_reserved":0,"allocated":3211264,"data_stored":1716756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.372},{"interface":"front","average":{"1min":0.5,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.505}]},{"osd":1,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48999999999999999,"15min":0.495},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72599999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56100000000000005,"15min":0.57299999999999995},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67700000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73099999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.58199999999999996,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80700000000000005,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.54100000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.54900000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0109999999999999,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80500000000000005,"15min":0.81699999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0329999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.76300000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.80600000000000005,"15min":0.83399999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.99199999999999999,"5min":1.1739999999999999,"15min":1.397},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.80100000000000005,"15min":0.82599999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.042,"5min":1.2230000000000001,"15min":1.262},"last":0.77400000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82799999999999996,"15min":0.84499999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0329999999999999,"5min":1.234,"15min":1.274},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80700000000000005,"15min":0.83899999999999997},"min":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.93799999999999994,"5min":1.165,"15min":1.302},"last":0.79200000000000004}]},{"osd":7,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.872,"15min":0.89800000000000002},"min":{"1min":0.65100000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.153,"5min":1.3540000000000001,"15min":1.486},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.86599999999999999,"15min":0.88600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1279999999999999,"5min":1.421,"15min":1.421},"last":0.92400000000000004}]}]},{"osd":2,"up_from":15,"seq":64424510586,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52216,"kb_used_data":3120,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319624,"statfs":{"total":96636764160,"available":96583294976,"internally_reserved":0,"allocated":3194880,"data_stored":1714610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42499999999999999,"15min":0.44700000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.622,"5min":0.754,"15min":0.754},"last":0.435},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48099999999999998,"15min":0.5},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66000000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.51500000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.57299999999999995,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.64800000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.36199999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.66200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.441,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.86199999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.36899999999999999},"max":{"1min":0.68400000000000005,"5min":0.871,"15min":0.875},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.94599999999999995,"5min":1.0880000000000001,"15min":1.306},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76500000000000001,"15min":0.78300000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81799999999999995,"5min":1.2070000000000001,"15min":1.3580000000000001},"last":0.89900000000000002}]},{"osd":5,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76800000000000002,"15min":0.79700000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":1.1080000000000001,"15min":1.2050000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89400000000000002,"5min":1.1519999999999999,"15min":1.429},"last":0.90900000000000003}]},{"osd":6,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.83199999999999996},"min":{"1min":0.67500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.91700000000000004,"5min":1.216,"15min":1.365},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76900000000000002,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.117,"15min":1.2},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77800000000000002,"15min":0.81399999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96299999999999997,"5min":1.159,"15min":1.45},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.85099999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0229999999999999,"5min":1.276,"15min":1.276},"last":0.87}]}]},{"osd":1,"up_from":15,"seq":64424510587,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60424,"kb_used_data":3136,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311416,"statfs":{"total":96636764160,"available":96574889984,"internally_reserved":0,"allocated":3211264,"data_stored":1715386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44800000000000001,"15min":0.45800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56399999999999995,"5min":0.71999999999999997,"15min":0.89400000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79600000000000004,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.36399999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66000000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.66600000000000004,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.41199999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.622,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.747,"5min":0.94199999999999995,"15min":0.94499999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.628,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.434}]},{"osd":4,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.78100000000000003,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92700000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77700000000000002,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.081,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.875}]},{"osd":5,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76900000000000002,"15min":0.80000000000000004},"min":{"1min":0.58699999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0189999999999999,"5min":1.236,"15min":1.2609999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81799999999999995},"min":{"1min":0.58199999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0669999999999999,"5min":1.353,"15min":1.353},"last":0.91700000000000004}]},{"osd":6,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.505},"max":{"1min":1.0009999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.82399999999999995,"15min":0.84099999999999997},"min":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.034,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.73899999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84599999999999997,"15min":0.85399999999999998},"min":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.56499999999999995},"max":{"1min":1.147,"5min":1.399,"15min":1.399},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.82799999999999996,"15min":0.85199999999999998},"min":{"1min":0.53000000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.181,"5min":1.365,"15min":1.365},"last":0.79500000000000004}]}]},{"osd":0,"up_from":14,"seq":60129543292,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65084,"kb_used_data":3700,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94306756,"statfs":{"total":96636764160,"available":96570118144,"internally_reserved":0,"allocated":3788800,"data_stored":2304978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41899999999999998,"15min":0.45300000000000001},"min":{"1min":0.29699999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55300000000000005,"5min":0.69299999999999995,"15min":1.216},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.49399999999999999,"15min":0.51900000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67600000000000005,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.48799999999999999,"15min":0.52600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.77000000000000002,"15min":1.1699999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52300000000000002,"15min":0.54700000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.96799999999999997},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.51400000000000001,"15min":0.55000000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.59999999999999998,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59899999999999998,"15min":0.64400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.032},"last":0.441}]},{"osd":4,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.80600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.496},"max":{"1min":0.93899999999999995,"5min":1.1200000000000001,"15min":1.381},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.78600000000000003,"15min":0.81100000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.47799999999999998},"max":{"1min":0.84799999999999998,"5min":1.137,"15min":1.2849999999999999},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.93799999999999994,"5min":1.1180000000000001,"15min":1.3420000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.79100000000000004,"15min":0.81299999999999994},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.052,"5min":1.1259999999999999,"15min":1.288},"last":0.749}]},{"osd":6,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81200000000000006,"15min":0.83899999999999997},"min":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.51300000000000001},"max":{"1min":0.97899999999999998,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.80000000000000004,"15min":0.82699999999999996},"min":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97999999999999998,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0149999999999999,"5min":1.1899999999999999,"15min":1.415},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.82599999999999996,"15min":0.85099999999999998},"min":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96899999999999997,"5min":1.0980000000000001,"15min":1.395},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:18:23.687 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:18:23.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.864+0000 7fdda35c0640 1 Processor -- start 2023-12-19T18:18:23.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.864+0000 7fdda35c0640 1 -- start start 2023-12-19T18:18:23.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.864+0000 7fdda35c0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdd9c06bee0 0x7fdd9c06c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.864+0000 7fdda35c0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd9c06d4b0 con 0x7fdd9c0cb1d0 2023-12-19T18:18:23.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.864+0000 7fdda35c0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd9c06cc10 con 0x7fdd9c0b1d40 2023-12-19T18:18:23.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.864+0000 7fdda35c0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd9c0cc030 con 0x7fdd9c06bee0 2023-12-19T18:18:23.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.864+0000 7fdda135c640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdd9c0cb1d0 0x7fdd9c0cb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36240/0 (socket says 172.21.15.26:36240) 2023-12-19T18:18:23.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.864+0000 7fdda135c640 1 -- 172.21.15.26:0/2291359329 learned_addr learned my addr 172.21.15.26:0/2291359329 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:23.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.864+0000 7fdda0b5b640 1 --2- 172.21.15.26:0/2291359329 >> v2:172.21.15.26:3300/0 conn(0x7fdd9c06bee0 0x7fdd9c06c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.868+0000 7fdd93fff640 1 -- 172.21.15.26:0/2291359329 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111502394 0 0) 0x7fdd9c06d4b0 con 0x7fdd9c0cb1d0 2023-12-19T18:18:23.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.868+0000 7fdd93fff640 1 -- 172.21.15.26:0/2291359329 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd88003530 con 0x7fdd9c0cb1d0 2023-12-19T18:18:23.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.868+0000 7fdda0b5b640 1 -- 172.21.15.26:0/2291359329 >> v1:172.21.15.160:6789/0 conn(0x7fdd9c0b1d40 legacy=0x7fdd9c0b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:23.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.868+0000 7fdda0b5b640 1 -- 172.21.15.26:0/2291359329 >> v1:172.21.15.26:6789/0 conn(0x7fdd9c0cb1d0 legacy=0x7fdd9c0cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.868+0000 7fdda0b5b640 1 -- 172.21.15.26:0/2291359329 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c06cef0 con 0x7fdd9c06bee0 2023-12-19T18:18:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.868+0000 7fdda0b5b640 1 --2- 172.21.15.26:0/2291359329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c06bee0 0x7fdd9c06c340 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0097c0 tx=0x7fdd8c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eadbe0906788ed7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.868+0000 7fdd93fff640 1 -- 172.21.15.26:0/2291359329 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c0ad680 con 0x7fdd9c06bee0 2023-12-19T18:18:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.868+0000 7fdd93fff640 1 -- 172.21.15.26:0/2291359329 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8c0ad7e0 con 0x7fdd9c06bee0 2023-12-19T18:18:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.868+0000 7fdd93fff640 1 -- 172.21.15.26:0/2291359329 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c0adb70 con 0x7fdd9c06bee0 2023-12-19T18:18:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 -- 172.21.15.26:0/2291359329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c06bee0 msgr2=0x7fdd9c06c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 --2- 172.21.15.26:0/2291359329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c06bee0 0x7fdd9c06c340 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0097c0 tx=0x7fdd8c01c380 comp rx=0 tx=0).stop 2023-12-19T18:18:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 -- 172.21.15.26:0/2291359329 shutdown_connections 2023-12-19T18:18:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 --2- 172.21.15.26:0/2291359329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c06bee0 0x7fdd9c06c340 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 -- 172.21.15.26:0/2291359329 >> 172.21.15.26:0/2291359329 conn(0x7fdd9c0658e0 msgr2=0x7fdd9c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 -- 172.21.15.26:0/2291359329 shutdown_connections 2023-12-19T18:18:23.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 -- 172.21.15.26:0/2291359329 wait complete. 2023-12-19T18:18:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 Processor -- start 2023-12-19T18:18:23.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 -- start start 2023-12-19T18:18:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd9c06bee0 0x7fdd9c14d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd9c0b1d40 0x7fdd9c156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c0cb1d0 0x7fdd9c0afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c14eaa0 con 0x7fdd9c0cb1d0 2023-12-19T18:18:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c14dc00 con 0x7fdd9c06bee0 2023-12-19T18:18:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda35c0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9c156ac0 con 0x7fdd9c0b1d40 2023-12-19T18:18:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda1b5d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c0cb1d0 0x7fdd9c0afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda1b5d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c0cb1d0 0x7fdd9c0afad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38398/0 (socket says 172.21.15.26:38398) 2023-12-19T18:18:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda1b5d640 1 -- 172.21.15.26:0/3740877000 learned_addr learned my addr 172.21.15.26:0/3740877000 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.872+0000 7fdda1b5d640 1 -- 172.21.15.26:0/3740877000 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd9c0b1d40 msgr2=0x7fdd9c156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdda1b5d640 1 --2- 172.21.15.26:0/3740877000 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd9c0b1d40 0x7fdd9c156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdda1b5d640 1 -- 172.21.15.26:0/3740877000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd9c06bee0 msgr2=0x7fdd9c14d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdda1b5d640 1 --2- 172.21.15.26:0/3740877000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd9c06bee0 0x7fdd9c14d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdda1b5d640 1 -- 172.21.15.26:0/3740877000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c009420 con 0x7fdd9c0cb1d0 2023-12-19T18:18:23.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdda1b5d640 1 --2- 172.21.15.26:0/3740877000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c0cb1d0 0x7fdd9c0afad0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400c7b0 tx=0x7fdd9400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdd91ffb640 1 -- 172.21.15.26:0/3740877000 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd94004b70 con 0x7fdd9c0cb1d0 2023-12-19T18:18:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdd91ffb640 1 -- 172.21.15.26:0/3740877000 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd94004cd0 con 0x7fdd9c0cb1d0 2023-12-19T18:18:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdda35c0640 1 -- 172.21.15.26:0/3740877000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9c14de20 con 0x7fdd9c0cb1d0 2023-12-19T18:18:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdd91ffb640 1 -- 172.21.15.26:0/3740877000 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400aa90 con 0x7fdd9c0cb1d0 2023-12-19T18:18:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdda35c0640 1 -- 172.21.15.26:0/3740877000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9c14e3e0 con 0x7fdd9c0cb1d0 2023-12-19T18:18:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdda35c0640 1 -- 172.21.15.26:0/3740877000 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd9c074270 con 0x7fdd9c0cb1d0 2023-12-19T18:18:23.877 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdd91ffb640 1 -- 172.21.15.26:0/3740877000 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fdd9400abf0 con 0x7fdd9c0cb1d0 2023-12-19T18:18:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdd91ffb640 1 --2- 172.21.15.26:0/3740877000 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdd80065cd0 0x7fdd80068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.878 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdd91ffb640 1 -- 172.21.15.26:0/3740877000 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fdd94083a80 con 0x7fdd9c0cb1d0 2023-12-19T18:18:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.876+0000 7fdda135c640 1 --2- 172.21.15.26:0/3740877000 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdd80065cd0 0x7fdd80068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.879 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.880+0000 7fdda135c640 1 --2- 172.21.15.26:0/3740877000 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdd80065cd0 0x7fdd80068190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd98007fd0 tx=0x7fdd98007940 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.882 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:23.880+0000 7fdd91ffb640 1 -- 172.21.15.26:0/3740877000 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdd94050d40 con 0x7fdd9c0cb1d0 2023-12-19T18:18:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.036+0000 7fdda35c0640 1 -- 172.21.15.26:0/3740877000 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd9c06c880 con 0x7fdd80065cd0 2023-12-19T18:18:24.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.044+0000 7fdd91ffb640 1 -- 172.21.15.26:0/3740877000 <== mgr.9680 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289367 (secure 0 0 0) 0x7fdd980058c0 con 0x7fdd80065cd0 2023-12-19T18:18:24.048 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:24.050 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:18:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.048+0000 7fdda35c0640 1 -- 172.21.15.26:0/3740877000 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdd80065cd0 msgr2=0x7fdd80068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.048+0000 7fdda35c0640 1 --2- 172.21.15.26:0/3740877000 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdd80065cd0 0x7fdd80068190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdd98007fd0 tx=0x7fdd98007940 comp rx=0 tx=0).stop 2023-12-19T18:18:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.048+0000 7fdda35c0640 1 -- 172.21.15.26:0/3740877000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c0cb1d0 msgr2=0x7fdd9c0afad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.048+0000 7fdda35c0640 1 --2- 172.21.15.26:0/3740877000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c0cb1d0 0x7fdd9c0afad0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400c7b0 tx=0x7fdd9400cc70 comp rx=0 tx=0).stop 2023-12-19T18:18:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.048+0000 7fdda35c0640 1 -- 172.21.15.26:0/3740877000 shutdown_connections 2023-12-19T18:18:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.048+0000 7fdda35c0640 1 --2- 172.21.15.26:0/3740877000 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdd80065cd0 0x7fdd80068190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.048+0000 7fdda35c0640 1 --2- 172.21.15.26:0/3740877000 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd9c0cb1d0 0x7fdd9c0afad0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.048+0000 7fdda35c0640 1 --2- 172.21.15.26:0/3740877000 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdd9c0b1d40 0x7fdd9c156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.048+0000 7fdda35c0640 1 --2- 172.21.15.26:0/3740877000 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdd9c06bee0 0x7fdd9c14d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.048+0000 7fdda35c0640 1 -- 172.21.15.26:0/3740877000 >> 172.21.15.26:0/3740877000 conn(0x7fdd9c0658e0 msgr2=0x7fdd9c065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.048+0000 7fdda35c0640 1 -- 172.21.15.26:0/3740877000 shutdown_connections 2023-12-19T18:18:24.052 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.048+0000 7fdda35c0640 1 -- 172.21.15.26:0/3740877000 wait complete. 2023-12-19T18:18:24.069 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-12-19T18:18:23.953600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2498,"num_read_kb":5235,"num_write":699,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":428,"ondisk_log_size":428,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":427144,"kb_used_data":26184,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754547576,"statfs":{"total":773094113280,"available":772656717824,"internally_reserved":0,"allocated":26812416,"data_stored":14902340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.810199"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.571329+0000","last_change":"2023-12-19T18:18:19.586634+0000","last_active":"2023-12-19T18:18:20.571329+0000","last_peered":"2023-12-19T18:18:20.571329+0000","last_clean":"2023-12-19T18:18:20.571329+0000","last_became_active":"2023-12-19T18:18:19.586328+0000","last_became_peered":"2023-12-19T18:18:19.586328+0000","last_unstale":"2023-12-19T18:18:20.571329+0000","last_undegraded":"2023-12-19T18:18:20.571329+0000","last_fullsized":"2023-12-19T18:18:20.571329+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:06.547846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856382+0000","last_change":"2023-12-19T18:18:19.587556+0000","last_active":"2023-12-19T18:18:20.856382+0000","last_peered":"2023-12-19T18:18:20.856382+0000","last_clean":"2023-12-19T18:18:20.856382+0000","last_became_active":"2023-12-19T18:18:19.586818+0000","last_became_peered":"2023-12-19T18:18:19.586818+0000","last_unstale":"2023-12-19T18:18:20.856382+0000","last_undegraded":"2023-12-19T18:18:20.856382+0000","last_fullsized":"2023-12-19T18:18:20.856382+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:11:09.830159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570743+0000","last_change":"2023-12-19T18:18:19.579130+0000","last_active":"2023-12-19T18:18:20.570743+0000","last_peered":"2023-12-19T18:18:20.570743+0000","last_clean":"2023-12-19T18:18:20.570743+0000","last_became_active":"2023-12-19T18:18:19.578721+0000","last_became_peered":"2023-12-19T18:18:19.578721+0000","last_unstale":"2023-12-19T18:18:20.570743+0000","last_undegraded":"2023-12-19T18:18:20.570743+0000","last_fullsized":"2023-12-19T18:18:20.570743+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:54:55.928387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570512+0000","last_change":"2023-12-19T18:18:19.590935+0000","last_active":"2023-12-19T18:18:20.570512+0000","last_peered":"2023-12-19T18:18:20.570512+0000","last_clean":"2023-12-19T18:18:20.570512+0000","last_became_active":"2023-12-19T18:18:19.589372+0000","last_became_peered":"2023-12-19T18:18:19.589372+0000","last_unstale":"2023-12-19T18:18:20.570512+0000","last_undegraded":"2023-12-19T18:18:20.570512+0000","last_fullsized":"2023-12-19T18:18:20.570512+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:25:48.739322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570507+0000","last_change":"2023-12-19T18:18:19.583837+0000","last_active":"2023-12-19T18:18:20.570507+0000","last_peered":"2023-12-19T18:18:20.570507+0000","last_clean":"2023-12-19T18:18:20.570507+0000","last_became_active":"2023-12-19T18:18:19.583573+0000","last_became_peered":"2023-12-19T18:18:19.583573+0000","last_unstale":"2023-12-19T18:18:20.570507+0000","last_undegraded":"2023-12-19T18:18:20.570507+0000","last_fullsized":"2023-12-19T18:18:20.570507+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:18.093251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855424+0000","last_change":"2023-12-19T18:18:19.580670+0000","last_active":"2023-12-19T18:18:20.855424+0000","last_peered":"2023-12-19T18:18:20.855424+0000","last_clean":"2023-12-19T18:18:20.855424+0000","last_became_active":"2023-12-19T18:18:19.580313+0000","last_became_peered":"2023-12-19T18:18:19.580313+0000","last_unstale":"2023-12-19T18:18:20.855424+0000","last_undegraded":"2023-12-19T18:18:20.855424+0000","last_fullsized":"2023-12-19T18:18:20.855424+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:20:01.112918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856662+0000","last_change":"2023-12-19T18:18:19.584213+0000","last_active":"2023-12-19T18:18:20.856662+0000","last_peered":"2023-12-19T18:18:20.856662+0000","last_clean":"2023-12-19T18:18:20.856662+0000","last_became_active":"2023-12-19T18:18:19.583886+0000","last_became_peered":"2023-12-19T18:18:19.583886+0000","last_unstale":"2023-12-19T18:18:20.856662+0000","last_undegraded":"2023-12-19T18:18:20.856662+0000","last_fullsized":"2023-12-19T18:18:20.856662+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:56:58.406195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570237+0000","last_change":"2023-12-19T18:18:19.589110+0000","last_active":"2023-12-19T18:18:20.570237+0000","last_peered":"2023-12-19T18:18:20.570237+0000","last_clean":"2023-12-19T18:18:20.570237+0000","last_became_active":"2023-12-19T18:18:19.588826+0000","last_became_peered":"2023-12-19T18:18:19.588826+0000","last_unstale":"2023-12-19T18:18:20.570237+0000","last_undegraded":"2023-12-19T18:18:20.570237+0000","last_fullsized":"2023-12-19T18:18:20.570237+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:50.803341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569386+0000","last_change":"2023-12-19T18:18:19.580018+0000","last_active":"2023-12-19T18:18:20.569386+0000","last_peered":"2023-12-19T18:18:20.569386+0000","last_clean":"2023-12-19T18:18:20.569386+0000","last_became_active":"2023-12-19T18:18:19.579761+0000","last_became_peered":"2023-12-19T18:18:19.579761+0000","last_unstale":"2023-12-19T18:18:20.569386+0000","last_undegraded":"2023-12-19T18:18:20.569386+0000","last_fullsized":"2023-12-19T18:18:20.569386+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:19.227642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.857138+0000","last_change":"2023-12-19T18:18:19.589673+0000","last_active":"2023-12-19T18:18:20.857138+0000","last_peered":"2023-12-19T18:18:20.857138+0000","last_clean":"2023-12-19T18:18:20.857138+0000","last_became_active":"2023-12-19T18:18:19.589360+0000","last_became_peered":"2023-12-19T18:18:19.589360+0000","last_unstale":"2023-12-19T18:18:20.857138+0000","last_undegraded":"2023-12-19T18:18:20.857138+0000","last_fullsized":"2023-12-19T18:18:20.857138+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:21:09.116614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856918+0000","last_change":"2023-12-19T18:18:19.582645+0000","last_active":"2023-12-19T18:18:20.856918+0000","last_peered":"2023-12-19T18:18:20.856918+0000","last_clean":"2023-12-19T18:18:20.856918+0000","last_became_active":"2023-12-19T18:18:19.582301+0000","last_became_peered":"2023-12-19T18:18:19.582301+0000","last_unstale":"2023-12-19T18:18:20.856918+0000","last_undegraded":"2023-12-19T18:18:20.856918+0000","last_fullsized":"2023-12-19T18:18:20.856918+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:28:17.264359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.571031+0000","last_change":"2023-12-19T18:18:19.568458+0000","last_active":"2023-12-19T18:18:20.571031+0000","last_peered":"2023-12-19T18:18:20.571031+0000","last_clean":"2023-12-19T18:18:20.571031+0000","last_became_active":"2023-12-19T18:18:19.568155+0000","last_became_peered":"2023-12-19T18:18:19.568155+0000","last_unstale":"2023-12-19T18:18:20.571031+0000","last_undegraded":"2023-12-19T18:18:20.571031+0000","last_fullsized":"2023-12-19T18:18:20.571031+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:28.854918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791679+0000","last_change":"2023-12-19T18:18:19.587652+0000","last_active":"2023-12-19T18:18:20.791679+0000","last_peered":"2023-12-19T18:18:20.791679+0000","last_clean":"2023-12-19T18:18:20.791679+0000","last_became_active":"2023-12-19T18:18:19.583977+0000","last_became_peered":"2023-12-19T18:18:19.583977+0000","last_unstale":"2023-12-19T18:18:20.791679+0000","last_undegraded":"2023-12-19T18:18:20.791679+0000","last_fullsized":"2023-12-19T18:18:20.791679+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:17.867996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791661+0000","last_change":"2023-12-19T18:18:19.570641+0000","last_active":"2023-12-19T18:18:20.791661+0000","last_peered":"2023-12-19T18:18:20.791661+0000","last_clean":"2023-12-19T18:18:20.791661+0000","last_became_active":"2023-12-19T18:18:19.570230+0000","last_became_peered":"2023-12-19T18:18:19.570230+0000","last_unstale":"2023-12-19T18:18:20.791661+0000","last_undegraded":"2023-12-19T18:18:20.791661+0000","last_fullsized":"2023-12-19T18:18:20.791661+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:51.311268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791719+0000","last_change":"2023-12-19T18:18:19.585897+0000","last_active":"2023-12-19T18:18:20.791719+0000","last_peered":"2023-12-19T18:18:20.791719+0000","last_clean":"2023-12-19T18:18:20.791719+0000","last_became_active":"2023-12-19T18:18:19.578143+0000","last_became_peered":"2023-12-19T18:18:19.578143+0000","last_unstale":"2023-12-19T18:18:20.791719+0000","last_undegraded":"2023-12-19T18:18:20.791719+0000","last_fullsized":"2023-12-19T18:18:20.791719+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:54:55.928387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855246+0000","last_change":"2023-12-19T18:18:19.593987+0000","last_active":"2023-12-19T18:18:20.855246+0000","last_peered":"2023-12-19T18:18:20.855246+0000","last_clean":"2023-12-19T18:18:20.855246+0000","last_became_active":"2023-12-19T18:18:19.591467+0000","last_became_peered":"2023-12-19T18:18:19.591467+0000","last_unstale":"2023-12-19T18:18:20.855246+0000","last_undegraded":"2023-12-19T18:18:20.855246+0000","last_fullsized":"2023-12-19T18:18:20.855246+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:48:24.682318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855894+0000","last_change":"2023-12-19T18:18:19.585647+0000","last_active":"2023-12-19T18:18:20.855894+0000","last_peered":"2023-12-19T18:18:20.855894+0000","last_clean":"2023-12-19T18:18:20.855894+0000","last_became_active":"2023-12-19T18:18:19.585355+0000","last_became_peered":"2023-12-19T18:18:19.585355+0000","last_unstale":"2023-12-19T18:18:20.855894+0000","last_undegraded":"2023-12-19T18:18:20.855894+0000","last_fullsized":"2023-12-19T18:18:20.855894+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:49.021053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.857259+0000","last_change":"2023-12-19T18:18:19.585633+0000","last_active":"2023-12-19T18:18:20.857259+0000","last_peered":"2023-12-19T18:18:20.857259+0000","last_clean":"2023-12-19T18:18:20.857259+0000","last_became_active":"2023-12-19T18:18:19.584124+0000","last_became_peered":"2023-12-19T18:18:19.584124+0000","last_unstale":"2023-12-19T18:18:20.857259+0000","last_undegraded":"2023-12-19T18:18:20.857259+0000","last_fullsized":"2023-12-19T18:18:20.857259+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:45.244289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856913+0000","last_change":"2023-12-19T18:18:19.591381+0000","last_active":"2023-12-19T18:18:20.856913+0000","last_peered":"2023-12-19T18:18:20.856913+0000","last_clean":"2023-12-19T18:18:20.856913+0000","last_became_active":"2023-12-19T18:18:19.591088+0000","last_became_peered":"2023-12-19T18:18:19.591088+0000","last_unstale":"2023-12-19T18:18:20.856913+0000","last_undegraded":"2023-12-19T18:18:20.856913+0000","last_fullsized":"2023-12-19T18:18:20.856913+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:50.206100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856081+0000","last_change":"2023-12-19T18:18:19.590556+0000","last_active":"2023-12-19T18:18:20.856081+0000","last_peered":"2023-12-19T18:18:20.856081+0000","last_clean":"2023-12-19T18:18:20.856081+0000","last_became_active":"2023-12-19T18:18:19.590315+0000","last_became_peered":"2023-12-19T18:18:19.590315+0000","last_unstale":"2023-12-19T18:18:20.856081+0000","last_undegraded":"2023-12-19T18:18:20.856081+0000","last_fullsized":"2023-12-19T18:18:20.856081+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:24.266208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569308+0000","last_change":"2023-12-19T18:18:19.594931+0000","last_active":"2023-12-19T18:18:20.569308+0000","last_peered":"2023-12-19T18:18:20.569308+0000","last_clean":"2023-12-19T18:18:20.569308+0000","last_became_active":"2023-12-19T18:18:19.594676+0000","last_became_peered":"2023-12-19T18:18:19.594676+0000","last_unstale":"2023-12-19T18:18:20.569308+0000","last_undegraded":"2023-12-19T18:18:20.569308+0000","last_fullsized":"2023-12-19T18:18:20.569308+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:14:28.152600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569066+0000","last_change":"2023-12-19T18:18:19.579261+0000","last_active":"2023-12-19T18:18:20.569066+0000","last_peered":"2023-12-19T18:18:20.569066+0000","last_clean":"2023-12-19T18:18:20.569066+0000","last_became_active":"2023-12-19T18:18:19.578977+0000","last_became_peered":"2023-12-19T18:18:19.578977+0000","last_unstale":"2023-12-19T18:18:20.569066+0000","last_undegraded":"2023-12-19T18:18:20.569066+0000","last_fullsized":"2023-12-19T18:18:20.569066+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:51:24.364308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856613+0000","last_change":"2023-12-19T18:18:19.583658+0000","last_active":"2023-12-19T18:18:20.856613+0000","last_peered":"2023-12-19T18:18:20.856613+0000","last_clean":"2023-12-19T18:18:20.856613+0000","last_became_active":"2023-12-19T18:18:19.583370+0000","last_became_peered":"2023-12-19T18:18:19.583370+0000","last_unstale":"2023-12-19T18:18:20.856613+0000","last_undegraded":"2023-12-19T18:18:20.856613+0000","last_fullsized":"2023-12-19T18:18:20.856613+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:05.578139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855456+0000","last_change":"2023-12-19T18:18:19.585514+0000","last_active":"2023-12-19T18:18:20.855456+0000","last_peered":"2023-12-19T18:18:20.855456+0000","last_clean":"2023-12-19T18:18:20.855456+0000","last_became_active":"2023-12-19T18:18:19.584938+0000","last_became_peered":"2023-12-19T18:18:19.584938+0000","last_unstale":"2023-12-19T18:18:20.855456+0000","last_undegraded":"2023-12-19T18:18:20.855456+0000","last_fullsized":"2023-12-19T18:18:20.855456+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:48:24.682318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791965+0000","last_change":"2023-12-19T18:18:19.583858+0000","last_active":"2023-12-19T18:18:20.791965+0000","last_peered":"2023-12-19T18:18:20.791965+0000","last_clean":"2023-12-19T18:18:20.791965+0000","last_became_active":"2023-12-19T18:18:19.583559+0000","last_became_peered":"2023-12-19T18:18:19.583559+0000","last_unstale":"2023-12-19T18:18:20.791965+0000","last_undegraded":"2023-12-19T18:18:20.791965+0000","last_fullsized":"2023-12-19T18:18:20.791965+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:18.093251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570937+0000","last_change":"2023-12-19T18:18:19.588236+0000","last_active":"2023-12-19T18:18:20.570937+0000","last_peered":"2023-12-19T18:18:20.570937+0000","last_clean":"2023-12-19T18:18:20.570937+0000","last_became_active":"2023-12-19T18:18:19.587925+0000","last_became_peered":"2023-12-19T18:18:19.587925+0000","last_unstale":"2023-12-19T18:18:20.570937+0000","last_undegraded":"2023-12-19T18:18:20.570937+0000","last_fullsized":"2023-12-19T18:18:20.570937+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:18:00.411251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.857067+0000","last_change":"2023-12-19T18:18:19.582852+0000","last_active":"2023-12-19T18:18:20.857067+0000","last_peered":"2023-12-19T18:18:20.857067+0000","last_clean":"2023-12-19T18:18:20.857067+0000","last_became_active":"2023-12-19T18:18:19.582239+0000","last_became_peered":"2023-12-19T18:18:19.582239+0000","last_unstale":"2023-12-19T18:18:20.857067+0000","last_undegraded":"2023-12-19T18:18:20.857067+0000","last_fullsized":"2023-12-19T18:18:20.857067+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:29:38.446159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791965+0000","last_change":"2023-12-19T18:18:19.581125+0000","last_active":"2023-12-19T18:18:20.791965+0000","last_peered":"2023-12-19T18:18:20.791965+0000","last_clean":"2023-12-19T18:18:20.791965+0000","last_became_active":"2023-12-19T18:18:19.580831+0000","last_became_peered":"2023-12-19T18:18:19.580831+0000","last_unstale":"2023-12-19T18:18:20.791965+0000","last_undegraded":"2023-12-19T18:18:20.791965+0000","last_fullsized":"2023-12-19T18:18:20.791965+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:01:30.699377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855858+0000","last_change":"2023-12-19T18:18:19.581884+0000","last_active":"2023-12-19T18:18:20.855858+0000","last_peered":"2023-12-19T18:18:20.855858+0000","last_clean":"2023-12-19T18:18:20.855858+0000","last_became_active":"2023-12-19T18:18:19.580389+0000","last_became_peered":"2023-12-19T18:18:19.580389+0000","last_unstale":"2023-12-19T18:18:20.855858+0000","last_undegraded":"2023-12-19T18:18:20.855858+0000","last_fullsized":"2023-12-19T18:18:20.855858+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:54.364546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791653+0000","last_change":"2023-12-19T18:18:19.570466+0000","last_active":"2023-12-19T18:18:20.791653+0000","last_peered":"2023-12-19T18:18:20.791653+0000","last_clean":"2023-12-19T18:18:20.791653+0000","last_became_active":"2023-12-19T18:18:19.569899+0000","last_became_peered":"2023-12-19T18:18:19.569899+0000","last_unstale":"2023-12-19T18:18:20.791653+0000","last_undegraded":"2023-12-19T18:18:20.791653+0000","last_fullsized":"2023-12-19T18:18:20.791653+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:31.714140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570297+0000","last_change":"2023-12-19T18:18:19.574972+0000","last_active":"2023-12-19T18:18:20.570297+0000","last_peered":"2023-12-19T18:18:20.570297+0000","last_clean":"2023-12-19T18:18:20.570297+0000","last_became_active":"2023-12-19T18:18:19.574500+0000","last_became_peered":"2023-12-19T18:18:19.574500+0000","last_unstale":"2023-12-19T18:18:20.570297+0000","last_undegraded":"2023-12-19T18:18:20.570297+0000","last_fullsized":"2023-12-19T18:18:20.570297+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:31:26.474823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.857160+0000","last_change":"2023-12-19T18:18:19.584815+0000","last_active":"2023-12-19T18:18:20.857160+0000","last_peered":"2023-12-19T18:18:20.857160+0000","last_clean":"2023-12-19T18:18:20.857160+0000","last_became_active":"2023-12-19T18:18:19.583340+0000","last_became_peered":"2023-12-19T18:18:19.583340+0000","last_unstale":"2023-12-19T18:18:20.857160+0000","last_undegraded":"2023-12-19T18:18:20.857160+0000","last_fullsized":"2023-12-19T18:18:20.857160+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:17:39.917070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"253'4","reported_seq":15,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856094+0000","last_change":"2023-12-19T18:18:19.164806+0000","last_active":"2023-12-19T18:18:20.856094+0000","last_peered":"2023-12-19T18:18:20.856094+0000","last_clean":"2023-12-19T18:18:20.856094+0000","last_became_active":"2023-12-19T18:18:19.164560+0000","last_became_peered":"2023-12-19T18:18:19.164560+0000","last_unstale":"2023-12-19T18:18:20.856094+0000","last_undegraded":"2023-12-19T18:18:20.856094+0000","last_fullsized":"2023-12-19T18:18:20.856094+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:44.401553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791999+0000","last_change":"2023-12-19T18:18:19.571559+0000","last_active":"2023-12-19T18:18:20.791999+0000","last_peered":"2023-12-19T18:18:20.791999+0000","last_clean":"2023-12-19T18:18:20.791999+0000","last_became_active":"2023-12-19T18:18:19.571303+0000","last_became_peered":"2023-12-19T18:18:19.571303+0000","last_unstale":"2023-12-19T18:18:20.791999+0000","last_undegraded":"2023-12-19T18:18:20.791999+0000","last_fullsized":"2023-12-19T18:18:20.791999+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:51:19.629217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"253'1","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.792034+0000","last_change":"2023-12-19T18:18:19.172686+0000","last_active":"2023-12-19T18:18:20.792034+0000","last_peered":"2023-12-19T18:18:20.792034+0000","last_clean":"2023-12-19T18:18:20.792034+0000","last_became_active":"2023-12-19T18:18:19.172357+0000","last_became_peered":"2023-12-19T18:18:19.172357+0000","last_unstale":"2023-12-19T18:18:20.792034+0000","last_undegraded":"2023-12-19T18:18:20.792034+0000","last_fullsized":"2023-12-19T18:18:20.792034+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:58:13.987113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856876+0000","last_change":"2023-12-19T18:18:19.584695+0000","last_active":"2023-12-19T18:18:20.856876+0000","last_peered":"2023-12-19T18:18:20.856876+0000","last_clean":"2023-12-19T18:18:20.856876+0000","last_became_active":"2023-12-19T18:18:19.580748+0000","last_became_peered":"2023-12-19T18:18:19.580748+0000","last_unstale":"2023-12-19T18:18:20.856876+0000","last_undegraded":"2023-12-19T18:18:20.856876+0000","last_fullsized":"2023-12-19T18:18:20.856876+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:27.782019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"253'2","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856355+0000","last_change":"2023-12-19T18:18:19.168516+0000","last_active":"2023-12-19T18:18:20.856355+0000","last_peered":"2023-12-19T18:18:20.856355+0000","last_clean":"2023-12-19T18:18:20.856355+0000","last_became_active":"2023-12-19T18:18:19.168109+0000","last_became_peered":"2023-12-19T18:18:19.168109+0000","last_unstale":"2023-12-19T18:18:20.856355+0000","last_undegraded":"2023-12-19T18:18:20.856355+0000","last_fullsized":"2023-12-19T18:18:20.856355+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:25.760030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570751+0000","last_change":"2023-12-19T18:18:19.578243+0000","last_active":"2023-12-19T18:18:20.570751+0000","last_peered":"2023-12-19T18:18:20.570751+0000","last_clean":"2023-12-19T18:18:20.570751+0000","last_became_active":"2023-12-19T18:18:19.577973+0000","last_became_peered":"2023-12-19T18:18:19.577973+0000","last_unstale":"2023-12-19T18:18:20.570751+0000","last_undegraded":"2023-12-19T18:18:20.570751+0000","last_fullsized":"2023-12-19T18:18:20.570751+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:49:33.985436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"253'2","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856166+0000","last_change":"2023-12-19T18:18:19.167573+0000","last_active":"2023-12-19T18:18:20.856166+0000","last_peered":"2023-12-19T18:18:20.856166+0000","last_clean":"2023-12-19T18:18:20.856166+0000","last_became_active":"2023-12-19T18:18:19.167302+0000","last_became_peered":"2023-12-19T18:18:19.167302+0000","last_unstale":"2023-12-19T18:18:20.856166+0000","last_undegraded":"2023-12-19T18:18:20.856166+0000","last_fullsized":"2023-12-19T18:18:20.856166+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:58:37.805961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569096+0000","last_change":"2023-12-19T18:18:19.584233+0000","last_active":"2023-12-19T18:18:20.569096+0000","last_peered":"2023-12-19T18:18:20.569096+0000","last_clean":"2023-12-19T18:18:20.569096+0000","last_became_active":"2023-12-19T18:18:19.583963+0000","last_became_peered":"2023-12-19T18:18:19.583963+0000","last_unstale":"2023-12-19T18:18:20.569096+0000","last_undegraded":"2023-12-19T18:18:20.569096+0000","last_fullsized":"2023-12-19T18:18:20.569096+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:03:08.883628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"253'1","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856776+0000","last_change":"2023-12-19T18:18:19.171282+0000","last_active":"2023-12-19T18:18:20.856776+0000","last_peered":"2023-12-19T18:18:20.856776+0000","last_clean":"2023-12-19T18:18:20.856776+0000","last_became_active":"2023-12-19T18:18:19.170854+0000","last_became_peered":"2023-12-19T18:18:19.170854+0000","last_unstale":"2023-12-19T18:18:20.856776+0000","last_undegraded":"2023-12-19T18:18:20.856776+0000","last_fullsized":"2023-12-19T18:18:20.856776+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:51:08.618256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855842+0000","last_change":"2023-12-19T18:18:19.572802+0000","last_active":"2023-12-19T18:18:20.855842+0000","last_peered":"2023-12-19T18:18:20.855842+0000","last_clean":"2023-12-19T18:18:20.855842+0000","last_became_active":"2023-12-19T18:18:19.572003+0000","last_became_peered":"2023-12-19T18:18:19.572003+0000","last_unstale":"2023-12-19T18:18:20.855842+0000","last_undegraded":"2023-12-19T18:18:20.855842+0000","last_fullsized":"2023-12-19T18:18:20.855842+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:33:54.942965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"253'1","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855835+0000","last_change":"2023-12-19T18:18:19.168565+0000","last_active":"2023-12-19T18:18:20.855835+0000","last_peered":"2023-12-19T18:18:20.855835+0000","last_clean":"2023-12-19T18:18:20.855835+0000","last_became_active":"2023-12-19T18:18:19.168222+0000","last_became_peered":"2023-12-19T18:18:19.168222+0000","last_unstale":"2023-12-19T18:18:20.855835+0000","last_undegraded":"2023-12-19T18:18:20.855835+0000","last_fullsized":"2023-12-19T18:18:20.855835+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:25.760030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855257+0000","last_change":"2023-12-19T18:18:19.585278+0000","last_active":"2023-12-19T18:18:20.855257+0000","last_peered":"2023-12-19T18:18:20.855257+0000","last_clean":"2023-12-19T18:18:20.855257+0000","last_became_active":"2023-12-19T18:18:19.584932+0000","last_became_peered":"2023-12-19T18:18:19.584932+0000","last_unstale":"2023-12-19T18:18:20.855257+0000","last_undegraded":"2023-12-19T18:18:20.855257+0000","last_fullsized":"2023-12-19T18:18:20.855257+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:42:38.379799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570255+0000","last_change":"2023-12-19T18:18:19.165444+0000","last_active":"2023-12-19T18:18:20.570255+0000","last_peered":"2023-12-19T18:18:20.570255+0000","last_clean":"2023-12-19T18:18:20.570255+0000","last_became_active":"2023-12-19T18:18:19.165170+0000","last_became_peered":"2023-12-19T18:18:19.165170+0000","last_unstale":"2023-12-19T18:18:20.570255+0000","last_undegraded":"2023-12-19T18:18:20.570255+0000","last_fullsized":"2023-12-19T18:18:20.570255+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:35:31.432971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856859+0000","last_change":"2023-12-19T18:18:19.584245+0000","last_active":"2023-12-19T18:18:20.856859+0000","last_peered":"2023-12-19T18:18:20.856859+0000","last_clean":"2023-12-19T18:18:20.856859+0000","last_became_active":"2023-12-19T18:18:19.582819+0000","last_became_peered":"2023-12-19T18:18:19.582819+0000","last_unstale":"2023-12-19T18:18:20.856859+0000","last_undegraded":"2023-12-19T18:18:20.856859+0000","last_fullsized":"2023-12-19T18:18:20.856859+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:51:55.767484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856658+0000","last_change":"2023-12-19T18:18:19.171413+0000","last_active":"2023-12-19T18:18:20.856658+0000","last_peered":"2023-12-19T18:18:20.856658+0000","last_clean":"2023-12-19T18:18:20.856658+0000","last_became_active":"2023-12-19T18:18:19.170374+0000","last_became_peered":"2023-12-19T18:18:19.170374+0000","last_unstale":"2023-12-19T18:18:20.856658+0000","last_undegraded":"2023-12-19T18:18:20.856658+0000","last_fullsized":"2023-12-19T18:18:20.856658+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:31:25.466452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.857226+0000","last_change":"2023-12-19T18:18:19.585005+0000","last_active":"2023-12-19T18:18:20.857226+0000","last_peered":"2023-12-19T18:18:20.857226+0000","last_clean":"2023-12-19T18:18:20.857226+0000","last_became_active":"2023-12-19T18:18:19.583722+0000","last_became_peered":"2023-12-19T18:18:19.583722+0000","last_unstale":"2023-12-19T18:18:20.857226+0000","last_undegraded":"2023-12-19T18:18:20.857226+0000","last_fullsized":"2023-12-19T18:18:20.857226+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:34:22.743194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856122+0000","last_change":"2023-12-19T18:18:19.172613+0000","last_active":"2023-12-19T18:18:20.856122+0000","last_peered":"2023-12-19T18:18:20.856122+0000","last_clean":"2023-12-19T18:18:20.856122+0000","last_became_active":"2023-12-19T18:18:19.171540+0000","last_became_peered":"2023-12-19T18:18:19.171540+0000","last_unstale":"2023-12-19T18:18:20.856122+0000","last_undegraded":"2023-12-19T18:18:20.856122+0000","last_fullsized":"2023-12-19T18:18:20.856122+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:46:18.299306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856006+0000","last_change":"2023-12-19T18:18:19.577767+0000","last_active":"2023-12-19T18:18:20.856006+0000","last_peered":"2023-12-19T18:18:20.856006+0000","last_clean":"2023-12-19T18:18:20.856006+0000","last_became_active":"2023-12-19T18:18:19.577494+0000","last_became_peered":"2023-12-19T18:18:19.577494+0000","last_unstale":"2023-12-19T18:18:20.856006+0000","last_undegraded":"2023-12-19T18:18:20.856006+0000","last_fullsized":"2023-12-19T18:18:20.856006+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:40.993599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856154+0000","last_change":"2023-12-19T18:18:19.172871+0000","last_active":"2023-12-19T18:18:20.856154+0000","last_peered":"2023-12-19T18:18:20.856154+0000","last_clean":"2023-12-19T18:18:20.856154+0000","last_became_active":"2023-12-19T18:18:19.172561+0000","last_became_peered":"2023-12-19T18:18:19.172561+0000","last_unstale":"2023-12-19T18:18:20.856154+0000","last_undegraded":"2023-12-19T18:18:20.856154+0000","last_fullsized":"2023-12-19T18:18:20.856154+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:06:09.659288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791845+0000","last_change":"2023-12-19T18:18:19.575004+0000","last_active":"2023-12-19T18:18:20.791845+0000","last_peered":"2023-12-19T18:18:20.791845+0000","last_clean":"2023-12-19T18:18:20.791845+0000","last_became_active":"2023-12-19T18:18:19.574524+0000","last_became_peered":"2023-12-19T18:18:19.574524+0000","last_unstale":"2023-12-19T18:18:20.791845+0000","last_undegraded":"2023-12-19T18:18:20.791845+0000","last_fullsized":"2023-12-19T18:18:20.791845+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:18:00.411251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"253'3","reported_seq":14,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856389+0000","last_change":"2023-12-19T18:18:19.175923+0000","last_active":"2023-12-19T18:18:20.856389+0000","last_peered":"2023-12-19T18:18:20.856389+0000","last_clean":"2023-12-19T18:18:20.856389+0000","last_became_active":"2023-12-19T18:18:19.175660+0000","last_became_peered":"2023-12-19T18:18:19.175660+0000","last_unstale":"2023-12-19T18:18:20.856389+0000","last_undegraded":"2023-12-19T18:18:20.856389+0000","last_fullsized":"2023-12-19T18:18:20.856389+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:54:53.266347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855782+0000","last_change":"2023-12-19T18:18:19.580468+0000","last_active":"2023-12-19T18:18:20.855782+0000","last_peered":"2023-12-19T18:18:20.855782+0000","last_clean":"2023-12-19T18:18:20.855782+0000","last_became_active":"2023-12-19T18:18:19.580200+0000","last_became_peered":"2023-12-19T18:18:19.580200+0000","last_unstale":"2023-12-19T18:18:20.855782+0000","last_undegraded":"2023-12-19T18:18:20.855782+0000","last_fullsized":"2023-12-19T18:18:20.855782+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:31:49.737327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"253'6","reported_seq":17,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855704+0000","last_change":"2023-12-19T18:18:19.168003+0000","last_active":"2023-12-19T18:18:20.855704+0000","last_peered":"2023-12-19T18:18:20.855704+0000","last_clean":"2023-12-19T18:18:20.855704+0000","last_became_active":"2023-12-19T18:18:19.167688+0000","last_became_peered":"2023-12-19T18:18:19.167688+0000","last_unstale":"2023-12-19T18:18:20.855704+0000","last_undegraded":"2023-12-19T18:18:20.855704+0000","last_fullsized":"2023-12-19T18:18:20.855704+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:00.867540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855721+0000","last_change":"2023-12-19T18:18:19.588507+0000","last_active":"2023-12-19T18:18:20.855721+0000","last_peered":"2023-12-19T18:18:20.855721+0000","last_clean":"2023-12-19T18:18:20.855721+0000","last_became_active":"2023-12-19T18:18:19.588220+0000","last_became_peered":"2023-12-19T18:18:19.588220+0000","last_unstale":"2023-12-19T18:18:20.855721+0000","last_undegraded":"2023-12-19T18:18:20.855721+0000","last_fullsized":"2023-12-19T18:18:20.855721+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:46:07.926163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"253'2","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570856+0000","last_change":"2023-12-19T18:18:19.164280+0000","last_active":"2023-12-19T18:18:20.570856+0000","last_peered":"2023-12-19T18:18:20.570856+0000","last_clean":"2023-12-19T18:18:20.570856+0000","last_became_active":"2023-12-19T18:18:19.163443+0000","last_became_peered":"2023-12-19T18:18:19.163443+0000","last_unstale":"2023-12-19T18:18:20.570856+0000","last_undegraded":"2023-12-19T18:18:20.570856+0000","last_fullsized":"2023-12-19T18:18:20.570856+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:00:45.109361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856037+0000","last_change":"2023-12-19T18:18:19.587328+0000","last_active":"2023-12-19T18:18:20.856037+0000","last_peered":"2023-12-19T18:18:20.856037+0000","last_clean":"2023-12-19T18:18:20.856037+0000","last_became_active":"2023-12-19T18:18:19.586921+0000","last_became_peered":"2023-12-19T18:18:19.586921+0000","last_unstale":"2023-12-19T18:18:20.856037+0000","last_undegraded":"2023-12-19T18:18:20.856037+0000","last_fullsized":"2023-12-19T18:18:20.856037+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:04:19.617805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569148+0000","last_change":"2023-12-19T18:18:19.166958+0000","last_active":"2023-12-19T18:18:20.569148+0000","last_peered":"2023-12-19T18:18:20.569148+0000","last_clean":"2023-12-19T18:18:20.569148+0000","last_became_active":"2023-12-19T18:18:19.166685+0000","last_became_peered":"2023-12-19T18:18:19.166685+0000","last_unstale":"2023-12-19T18:18:20.569148+0000","last_undegraded":"2023-12-19T18:18:20.569148+0000","last_fullsized":"2023-12-19T18:18:20.569148+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:31:28.614678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855457+0000","last_change":"2023-12-19T18:18:19.597100+0000","last_active":"2023-12-19T18:18:20.855457+0000","last_peered":"2023-12-19T18:18:20.855457+0000","last_clean":"2023-12-19T18:18:20.855457+0000","last_became_active":"2023-12-19T18:18:19.596783+0000","last_became_peered":"2023-12-19T18:18:19.596783+0000","last_unstale":"2023-12-19T18:18:20.855457+0000","last_undegraded":"2023-12-19T18:18:20.855457+0000","last_fullsized":"2023-12-19T18:18:20.855457+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:06:03.752660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856110+0000","last_change":"2023-12-19T18:18:19.171189+0000","last_active":"2023-12-19T18:18:20.856110+0000","last_peered":"2023-12-19T18:18:20.856110+0000","last_clean":"2023-12-19T18:18:20.856110+0000","last_became_active":"2023-12-19T18:18:19.170861+0000","last_became_peered":"2023-12-19T18:18:19.170861+0000","last_unstale":"2023-12-19T18:18:20.856110+0000","last_undegraded":"2023-12-19T18:18:20.856110+0000","last_fullsized":"2023-12-19T18:18:20.856110+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:07:27.846546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855410+0000","last_change":"2023-12-19T18:18:19.580433+0000","last_active":"2023-12-19T18:18:20.855410+0000","last_peered":"2023-12-19T18:18:20.855410+0000","last_clean":"2023-12-19T18:18:20.855410+0000","last_became_active":"2023-12-19T18:18:19.580162+0000","last_became_peered":"2023-12-19T18:18:19.580162+0000","last_unstale":"2023-12-19T18:18:20.855410+0000","last_undegraded":"2023-12-19T18:18:20.855410+0000","last_fullsized":"2023-12-19T18:18:20.855410+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:33:54.942965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"253'8","reported_seq":19,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855373+0000","last_change":"2023-12-19T18:18:19.159958+0000","last_active":"2023-12-19T18:18:20.855373+0000","last_peered":"2023-12-19T18:18:20.855373+0000","last_clean":"2023-12-19T18:18:20.855373+0000","last_became_active":"2023-12-19T18:18:19.159675+0000","last_became_peered":"2023-12-19T18:18:19.159675+0000","last_unstale":"2023-12-19T18:18:20.855373+0000","last_undegraded":"2023-12-19T18:18:20.855373+0000","last_fullsized":"2023-12-19T18:18:20.855373+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:46:18.299306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791907+0000","last_change":"2023-12-19T18:18:19.575961+0000","last_active":"2023-12-19T18:18:20.791907+0000","last_peered":"2023-12-19T18:18:20.791907+0000","last_clean":"2023-12-19T18:18:20.791907+0000","last_became_active":"2023-12-19T18:18:19.575350+0000","last_became_peered":"2023-12-19T18:18:19.575350+0000","last_unstale":"2023-12-19T18:18:20.791907+0000","last_undegraded":"2023-12-19T18:18:20.791907+0000","last_fullsized":"2023-12-19T18:18:20.791907+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:07.028531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"253'1","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.571376+0000","last_change":"2023-12-19T18:18:19.170499+0000","last_active":"2023-12-19T18:18:20.571376+0000","last_peered":"2023-12-19T18:18:20.571376+0000","last_clean":"2023-12-19T18:18:20.571376+0000","last_became_active":"2023-12-19T18:18:19.170208+0000","last_became_peered":"2023-12-19T18:18:19.170208+0000","last_unstale":"2023-12-19T18:18:20.571376+0000","last_undegraded":"2023-12-19T18:18:20.571376+0000","last_fullsized":"2023-12-19T18:18:20.571376+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:42.153499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856372+0000","last_change":"2023-12-19T18:18:19.592701+0000","last_active":"2023-12-19T18:18:20.856372+0000","last_peered":"2023-12-19T18:18:20.856372+0000","last_clean":"2023-12-19T18:18:20.856372+0000","last_became_active":"2023-12-19T18:18:19.592373+0000","last_became_peered":"2023-12-19T18:18:19.592373+0000","last_unstale":"2023-12-19T18:18:20.856372+0000","last_undegraded":"2023-12-19T18:18:20.856372+0000","last_fullsized":"2023-12-19T18:18:20.856372+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:22.953633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"253'2","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856302+0000","last_change":"2023-12-19T18:18:19.159393+0000","last_active":"2023-12-19T18:18:20.856302+0000","last_peered":"2023-12-19T18:18:20.856302+0000","last_clean":"2023-12-19T18:18:20.856302+0000","last_became_active":"2023-12-19T18:18:19.159111+0000","last_became_peered":"2023-12-19T18:18:19.159111+0000","last_unstale":"2023-12-19T18:18:20.856302+0000","last_undegraded":"2023-12-19T18:18:20.856302+0000","last_fullsized":"2023-12-19T18:18:20.856302+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:24.242078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791964+0000","last_change":"2023-12-19T18:18:19.576960+0000","last_active":"2023-12-19T18:18:20.791964+0000","last_peered":"2023-12-19T18:18:20.791964+0000","last_clean":"2023-12-19T18:18:20.791964+0000","last_became_active":"2023-12-19T18:18:19.576683+0000","last_became_peered":"2023-12-19T18:18:19.576683+0000","last_unstale":"2023-12-19T18:18:20.791964+0000","last_undegraded":"2023-12-19T18:18:20.791964+0000","last_fullsized":"2023-12-19T18:18:20.791964+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:25:13.980357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"253'1","reported_seq":12,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856063+0000","last_change":"2023-12-19T18:18:19.172950+0000","last_active":"2023-12-19T18:18:20.856063+0000","last_peered":"2023-12-19T18:18:20.856063+0000","last_clean":"2023-12-19T18:18:20.856063+0000","last_became_active":"2023-12-19T18:18:19.172632+0000","last_became_peered":"2023-12-19T18:18:19.172632+0000","last_unstale":"2023-12-19T18:18:20.856063+0000","last_undegraded":"2023-12-19T18:18:20.856063+0000","last_fullsized":"2023-12-19T18:18:20.856063+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:05.539474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570842+0000","last_change":"2023-12-19T18:18:19.587143+0000","last_active":"2023-12-19T18:18:20.570842+0000","last_peered":"2023-12-19T18:18:20.570842+0000","last_clean":"2023-12-19T18:18:20.570842+0000","last_became_active":"2023-12-19T18:18:19.585269+0000","last_became_peered":"2023-12-19T18:18:19.585269+0000","last_unstale":"2023-12-19T18:18:20.570842+0000","last_undegraded":"2023-12-19T18:18:20.570842+0000","last_fullsized":"2023-12-19T18:18:20.570842+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:01:30.699377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.792036+0000","last_change":"2023-12-19T18:18:19.174353+0000","last_active":"2023-12-19T18:18:20.792036+0000","last_peered":"2023-12-19T18:18:20.792036+0000","last_clean":"2023-12-19T18:18:20.792036+0000","last_became_active":"2023-12-19T18:18:19.174051+0000","last_became_peered":"2023-12-19T18:18:19.174051+0000","last_unstale":"2023-12-19T18:18:20.792036+0000","last_undegraded":"2023-12-19T18:18:20.792036+0000","last_fullsized":"2023-12-19T18:18:20.792036+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:11.181824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855559+0000","last_change":"2023-12-19T18:18:19.577766+0000","last_active":"2023-12-19T18:18:20.855559+0000","last_peered":"2023-12-19T18:18:20.855559+0000","last_clean":"2023-12-19T18:18:20.855559+0000","last_became_active":"2023-12-19T18:18:19.577484+0000","last_became_peered":"2023-12-19T18:18:19.577484+0000","last_unstale":"2023-12-19T18:18:20.855559+0000","last_undegraded":"2023-12-19T18:18:20.855559+0000","last_fullsized":"2023-12-19T18:18:20.855559+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:36.541142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"253'2","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570985+0000","last_change":"2023-12-19T18:18:19.166552+0000","last_active":"2023-12-19T18:18:20.570985+0000","last_peered":"2023-12-19T18:18:20.570985+0000","last_clean":"2023-12-19T18:18:20.570985+0000","last_became_active":"2023-12-19T18:18:19.166288+0000","last_became_peered":"2023-12-19T18:18:19.166288+0000","last_unstale":"2023-12-19T18:18:20.570985+0000","last_undegraded":"2023-12-19T18:18:20.570985+0000","last_fullsized":"2023-12-19T18:18:20.570985+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:44.500321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856066+0000","last_change":"2023-12-19T18:18:19.591037+0000","last_active":"2023-12-19T18:18:20.856066+0000","last_peered":"2023-12-19T18:18:20.856066+0000","last_clean":"2023-12-19T18:18:20.856066+0000","last_became_active":"2023-12-19T18:18:19.590782+0000","last_became_peered":"2023-12-19T18:18:19.590782+0000","last_unstale":"2023-12-19T18:18:20.856066+0000","last_undegraded":"2023-12-19T18:18:20.856066+0000","last_fullsized":"2023-12-19T18:18:20.856066+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:47:57.808912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570613+0000","last_change":"2023-12-19T18:18:19.168858+0000","last_active":"2023-12-19T18:18:20.570613+0000","last_peered":"2023-12-19T18:18:20.570613+0000","last_clean":"2023-12-19T18:18:20.570613+0000","last_became_active":"2023-12-19T18:18:19.168561+0000","last_became_peered":"2023-12-19T18:18:19.168561+0000","last_unstale":"2023-12-19T18:18:20.570613+0000","last_undegraded":"2023-12-19T18:18:20.570613+0000","last_fullsized":"2023-12-19T18:18:20.570613+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:01:11.335521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855904+0000","last_change":"2023-12-19T18:18:19.585324+0000","last_active":"2023-12-19T18:18:20.855904+0000","last_peered":"2023-12-19T18:18:20.855904+0000","last_clean":"2023-12-19T18:18:20.855904+0000","last_became_active":"2023-12-19T18:18:19.584615+0000","last_became_peered":"2023-12-19T18:18:19.584615+0000","last_unstale":"2023-12-19T18:18:20.855904+0000","last_undegraded":"2023-12-19T18:18:20.855904+0000","last_fullsized":"2023-12-19T18:18:20.855904+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:02:20.434330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856216+0000","last_change":"2023-12-19T18:18:19.174032+0000","last_active":"2023-12-19T18:18:20.856216+0000","last_peered":"2023-12-19T18:18:20.856216+0000","last_clean":"2023-12-19T18:18:20.856216+0000","last_became_active":"2023-12-19T18:18:19.173764+0000","last_became_peered":"2023-12-19T18:18:19.173764+0000","last_unstale":"2023-12-19T18:18:20.856216+0000","last_undegraded":"2023-12-19T18:18:20.856216+0000","last_fullsized":"2023-12-19T18:18:20.856216+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:31:25.466452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.568938+0000","last_change":"2023-12-19T18:18:19.598180+0000","last_active":"2023-12-19T18:18:20.568938+0000","last_peered":"2023-12-19T18:18:20.568938+0000","last_clean":"2023-12-19T18:18:20.568938+0000","last_became_active":"2023-12-19T18:18:19.595299+0000","last_became_peered":"2023-12-19T18:18:19.595299+0000","last_unstale":"2023-12-19T18:18:20.568938+0000","last_undegraded":"2023-12-19T18:18:20.568938+0000","last_fullsized":"2023-12-19T18:18:20.568938+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:06.670616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570329+0000","last_change":"2023-12-19T18:18:19.164923+0000","last_active":"2023-12-19T18:18:20.570329+0000","last_peered":"2023-12-19T18:18:20.570329+0000","last_clean":"2023-12-19T18:18:20.570329+0000","last_became_active":"2023-12-19T18:18:19.164660+0000","last_became_peered":"2023-12-19T18:18:19.164660+0000","last_unstale":"2023-12-19T18:18:20.570329+0000","last_undegraded":"2023-12-19T18:18:20.570329+0000","last_fullsized":"2023-12-19T18:18:20.570329+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:35.595940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791932+0000","last_change":"2023-12-19T18:18:19.576132+0000","last_active":"2023-12-19T18:18:20.791932+0000","last_peered":"2023-12-19T18:18:20.791932+0000","last_clean":"2023-12-19T18:18:20.791932+0000","last_became_active":"2023-12-19T18:18:19.575711+0000","last_became_peered":"2023-12-19T18:18:19.575711+0000","last_unstale":"2023-12-19T18:18:20.791932+0000","last_undegraded":"2023-12-19T18:18:20.791932+0000","last_fullsized":"2023-12-19T18:18:20.791932+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:26:56.610626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"253'2","reported_seq":13,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.792073+0000","last_change":"2023-12-19T18:18:19.170075+0000","last_active":"2023-12-19T18:18:20.792073+0000","last_peered":"2023-12-19T18:18:20.792073+0000","last_clean":"2023-12-19T18:18:20.792073+0000","last_became_active":"2023-12-19T18:18:19.169795+0000","last_became_peered":"2023-12-19T18:18:19.169795+0000","last_unstale":"2023-12-19T18:18:20.792073+0000","last_undegraded":"2023-12-19T18:18:20.792073+0000","last_fullsized":"2023-12-19T18:18:20.792073+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:06:26.444241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856035+0000","last_change":"2023-12-19T18:18:19.581374+0000","last_active":"2023-12-19T18:18:20.856035+0000","last_peered":"2023-12-19T18:18:20.856035+0000","last_clean":"2023-12-19T18:18:20.856035+0000","last_became_active":"2023-12-19T18:18:19.581091+0000","last_became_peered":"2023-12-19T18:18:19.581091+0000","last_unstale":"2023-12-19T18:18:20.856035+0000","last_undegraded":"2023-12-19T18:18:20.856035+0000","last_fullsized":"2023-12-19T18:18:20.856035+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:47:57.808912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570854+0000","last_change":"2023-12-19T18:18:19.171868+0000","last_active":"2023-12-19T18:18:20.570854+0000","last_peered":"2023-12-19T18:18:20.570854+0000","last_clean":"2023-12-19T18:18:20.570854+0000","last_became_active":"2023-12-19T18:18:19.171566+0000","last_became_peered":"2023-12-19T18:18:19.171566+0000","last_unstale":"2023-12-19T18:18:20.570854+0000","last_undegraded":"2023-12-19T18:18:20.570854+0000","last_fullsized":"2023-12-19T18:18:20.570854+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:23.198353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.571060+0000","last_change":"2023-12-19T18:18:19.589854+0000","last_active":"2023-12-19T18:18:20.571060+0000","last_peered":"2023-12-19T18:18:20.571060+0000","last_clean":"2023-12-19T18:18:20.571060+0000","last_became_active":"2023-12-19T18:18:19.586190+0000","last_became_peered":"2023-12-19T18:18:19.586190+0000","last_unstale":"2023-12-19T18:18:20.571060+0000","last_undegraded":"2023-12-19T18:18:20.571060+0000","last_fullsized":"2023-12-19T18:18:20.571060+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:52:50.568291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.857091+0000","last_change":"2023-12-19T18:18:19.171097+0000","last_active":"2023-12-19T18:18:20.857091+0000","last_peered":"2023-12-19T18:18:20.857091+0000","last_clean":"2023-12-19T18:18:20.857091+0000","last_became_active":"2023-12-19T18:18:19.170690+0000","last_became_peered":"2023-12-19T18:18:19.170690+0000","last_unstale":"2023-12-19T18:18:20.857091+0000","last_undegraded":"2023-12-19T18:18:20.857091+0000","last_fullsized":"2023-12-19T18:18:20.857091+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:06:34.583280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570937+0000","last_change":"2023-12-19T18:18:19.580259+0000","last_active":"2023-12-19T18:18:20.570937+0000","last_peered":"2023-12-19T18:18:20.570937+0000","last_clean":"2023-12-19T18:18:20.570937+0000","last_became_active":"2023-12-19T18:18:19.579980+0000","last_became_peered":"2023-12-19T18:18:19.579980+0000","last_unstale":"2023-12-19T18:18:20.570937+0000","last_undegraded":"2023-12-19T18:18:20.570937+0000","last_fullsized":"2023-12-19T18:18:20.570937+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:54:51.311268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855697+0000","last_change":"2023-12-19T18:18:19.171492+0000","last_active":"2023-12-19T18:18:20.855697+0000","last_peered":"2023-12-19T18:18:20.855697+0000","last_clean":"2023-12-19T18:18:20.855697+0000","last_became_active":"2023-12-19T18:18:19.170962+0000","last_became_peered":"2023-12-19T18:18:19.170962+0000","last_unstale":"2023-12-19T18:18:20.855697+0000","last_undegraded":"2023-12-19T18:18:20.855697+0000","last_fullsized":"2023-12-19T18:18:20.855697+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:58:41.004354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791754+0000","last_change":"2023-12-19T18:18:19.581835+0000","last_active":"2023-12-19T18:18:20.791754+0000","last_peered":"2023-12-19T18:18:20.791754+0000","last_clean":"2023-12-19T18:18:20.791754+0000","last_became_active":"2023-12-19T18:18:19.581577+0000","last_became_peered":"2023-12-19T18:18:19.581577+0000","last_unstale":"2023-12-19T18:18:20.791754+0000","last_undegraded":"2023-12-19T18:18:20.791754+0000","last_fullsized":"2023-12-19T18:18:20.791754+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:15:04.921197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569148+0000","last_change":"2023-12-19T18:18:19.163272+0000","last_active":"2023-12-19T18:18:20.569148+0000","last_peered":"2023-12-19T18:18:20.569148+0000","last_clean":"2023-12-19T18:18:20.569148+0000","last_became_active":"2023-12-19T18:18:19.162992+0000","last_became_peered":"2023-12-19T18:18:19.162992+0000","last_unstale":"2023-12-19T18:18:20.569148+0000","last_undegraded":"2023-12-19T18:18:20.569148+0000","last_fullsized":"2023-12-19T18:18:20.569148+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:38:27.027379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791806+0000","last_change":"2023-12-19T18:18:19.571044+0000","last_active":"2023-12-19T18:18:20.791806+0000","last_peered":"2023-12-19T18:18:20.791806+0000","last_clean":"2023-12-19T18:18:20.791806+0000","last_became_active":"2023-12-19T18:18:19.570779+0000","last_became_peered":"2023-12-19T18:18:19.570779+0000","last_unstale":"2023-12-19T18:18:20.791806+0000","last_undegraded":"2023-12-19T18:18:20.791806+0000","last_fullsized":"2023-12-19T18:18:20.791806+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:54.881623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.569182+0000","last_change":"2023-12-19T18:18:19.170818+0000","last_active":"2023-12-19T18:18:20.569182+0000","last_peered":"2023-12-19T18:18:20.569182+0000","last_clean":"2023-12-19T18:18:20.569182+0000","last_became_active":"2023-12-19T18:18:19.170523+0000","last_became_peered":"2023-12-19T18:18:19.170523+0000","last_unstale":"2023-12-19T18:18:20.569182+0000","last_undegraded":"2023-12-19T18:18:20.569182+0000","last_fullsized":"2023-12-19T18:18:20.569182+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:52:55.832348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791748+0000","last_change":"2023-12-19T18:18:19.589779+0000","last_active":"2023-12-19T18:18:20.791748+0000","last_peered":"2023-12-19T18:18:20.791748+0000","last_clean":"2023-12-19T18:18:20.791748+0000","last_became_active":"2023-12-19T18:18:19.587841+0000","last_became_peered":"2023-12-19T18:18:19.587841+0000","last_unstale":"2023-12-19T18:18:20.791748+0000","last_undegraded":"2023-12-19T18:18:20.791748+0000","last_fullsized":"2023-12-19T18:18:20.791748+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:25:48.739322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.791795+0000","last_change":"2023-12-19T18:18:19.168943+0000","last_active":"2023-12-19T18:18:20.791795+0000","last_peered":"2023-12-19T18:18:20.791795+0000","last_clean":"2023-12-19T18:18:20.791795+0000","last_became_active":"2023-12-19T18:18:19.168650+0000","last_became_peered":"2023-12-19T18:18:19.168650+0000","last_unstale":"2023-12-19T18:18:20.791795+0000","last_undegraded":"2023-12-19T18:18:20.791795+0000","last_fullsized":"2023-12-19T18:18:20.791795+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:16.372429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.570171+0000","last_change":"2023-12-19T18:18:19.596288+0000","last_active":"2023-12-19T18:18:20.570171+0000","last_peered":"2023-12-19T18:18:20.570171+0000","last_clean":"2023-12-19T18:18:20.570171+0000","last_became_active":"2023-12-19T18:18:19.595896+0000","last_became_peered":"2023-12-19T18:18:19.595896+0000","last_unstale":"2023-12-19T18:18:20.570171+0000","last_undegraded":"2023-12-19T18:18:20.570171+0000","last_fullsized":"2023-12-19T18:18:20.570171+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:31.714140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":11,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.855666+0000","last_change":"2023-12-19T18:18:19.171495+0000","last_active":"2023-12-19T18:18:20.855666+0000","last_peered":"2023-12-19T18:18:20.855666+0000","last_clean":"2023-12-19T18:18:20.855666+0000","last_became_active":"2023-12-19T18:18:19.171133+0000","last_became_peered":"2023-12-19T18:18:19.171133+0000","last_unstale":"2023-12-19T18:18:20.855666+0000","last_undegraded":"2023-12-19T18:18:20.855666+0000","last_fullsized":"2023-12-19T18:18:20.855666+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":251,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:18.133650+0000","last_clean_scrub_stamp":"2023-12-19T18:18:18.133650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:26:24.242078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":10,"reported_epoch":254,"state":"active+clean","last_fresh":"2023-12-19T18:18:20.856535+0000","last_change":"2023-12-19T18:18:19.595513+0000","last_active":"2023-12-19T18:18:20.856535+0000","last_peered":"2023-12-19T18:18:20.856535+0000","last_clean":"2023-12-19T18:18:20.856535+0000","last_became_active":"2023-12-19T18:18:19.595314+0000","last_became_peered":"2023-12-19T18:18:19.595314+0000","last_unstale":"2023-12-19T18:18:20.856535+0000","last_undegraded":"2023-12-19T18:18:20.856535+0000","last_fullsized":"2023-12-19T18:18:20.856535+0000","mapping_epoch":252,"log_start":"0'0","ondisk_log_start":"0'0","created":252,"last_epoch_clean":253,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:19.142022+0000","last_clean_scrub_stamp":"2023-12-19T18:18:19.142022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:16:47.558060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"239'390","reported_seq":1377,"reported_epoch":253,"state":"active+clean","last_fresh":"2023-12-19T18:18:19.566574+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:18:19.566574+0000","last_peered":"2023-12-19T18:18:19.566574+0000","last_clean":"2023-12-19T18:18:19.566574+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:18:19.566574+0000","last_undegraded":"2023-12-19T18:18:19.566574+0000","last_fullsized":"2023-12-19T18:18:19.566574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2498,"num_read_kb":5235,"num_write":655,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2498,"num_read_kb":5235,"num_write":655,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":390,"ondisk_log_size":390,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510587,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48704,"kb_used_data":3704,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323136,"statfs":{"total":96636764160,"available":96586891264,"internally_reserved":0,"allocated":3792896,"data_stored":2305048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68500000000000005,"15min":0.71499999999999997},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79000000000000004,"5min":1.048,"15min":1.1180000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.71299999999999997,"15min":0.73199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.90900000000000003,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.79500000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.47699999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.93200000000000005,"5min":1.0389999999999999,"15min":1.0580000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.74199999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.0620000000000001,"15min":1.0660000000000001},"last":0.70999999999999996}]},{"osd":2,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.73099999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.83399999999999996,"5min":1.026,"15min":1.0669999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.751,"15min":0.77700000000000002},"min":{"1min":0.51600000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.996,"5min":1.0820000000000001,"15min":1.236},"last":0.80900000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.52300000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.88100000000000001,"5min":1.002,"15min":1.079},"last":0.755},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.49399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.85899999999999999,"5min":1.1120000000000001,"15min":1.1870000000000001},"last":0.78000000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60699999999999998,"15min":0.59899999999999998},"min":{"1min":0.441,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.302},"last":0.622},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.622,"15min":0.629},"min":{"1min":0.50800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.0209999999999999},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.63300000000000001,"15min":0.63400000000000001},"min":{"1min":0.41899999999999998,"5min":0.40899999999999997,"15min":0.38800000000000001},"max":{"1min":0.90600000000000003,"5min":1.097,"15min":1.1299999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.55800000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95699999999999996,"5min":1.026,"15min":1.0800000000000001},"last":0.51400000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:17:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.71799999999999997,"15min":0.71699999999999997},"min":{"1min":0.61699999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.0469999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.093},"last":0.59799999999999998}]}]},{"osd":6,"up_from":15,"seq":64424510587,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48128,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323712,"statfs":{"total":96636764160,"available":96587481088,"internally_reserved":0,"allocated":3203072,"data_stored":1714722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90600000000000003,"5min":1.0289999999999999,"15min":1.042},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69299999999999995,"15min":0.72399999999999998},"min":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95999999999999996,"5min":1.224,"15min":1.224},"last":0.59799999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94799999999999995,"5min":1.103,"15min":1.103},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.746,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.58199999999999996}]},{"osd":2,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.73699999999999999},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.879,"5min":0.98999999999999999,"15min":1.111},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.748,"15min":0.77400000000000002},"min":{"1min":0.57799999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94799999999999995,"5min":1.046,"15min":1.1519999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.90300000000000002,"5min":1.268,"15min":1.268},"last":0.57599999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.49199999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.97499999999999998,"5min":1.194,"15min":1.194},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.67700000000000005},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.53500000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.92200000000000004,"5min":1.218,"15min":1.218},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.60999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.1200000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.67000000000000004}]}]},{"osd":5,"up_from":15,"seq":64424510587,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52216,"kb_used_data":3120,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319624,"statfs":{"total":96636764160,"available":96583294976,"internally_reserved":0,"allocated":3194880,"data_stored":1714610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.69599999999999995,"15min":0.72499999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.94999999999999996,"5min":1.002,"15min":1.0149999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.93100000000000005,"5min":0.96099999999999997,"15min":1.0229999999999999},"last":0.629}]},{"osd":1,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.54500000000000004,"5min":0.52100000000000002,"15min":0.497},"max":{"1min":0.84599999999999997,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.498,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97599999999999998,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59999999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.58599999999999997,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":1.0069999999999999,"5min":1.0509999999999999,"15min":1.2250000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70799999999999996,"15min":0.73199999999999998},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88600000000000001,"5min":1.004,"15min":1.4279999999999999},"last":0.61299999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.748,"15min":0.76200000000000001},"min":{"1min":0.56799999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.90400000000000003,"5min":1.0129999999999999,"15min":1.05},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93400000000000005,"5min":1.022,"15min":1.0329999999999999},"last":0.65700000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.39100000000000001,"5min":0.36099999999999999,"15min":0.35399999999999998},"max":{"1min":0.81699999999999995,"5min":0.94499999999999995,"15min":1.0800000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.94299999999999995,"5min":1.208,"15min":1.208},"last":0.441}]},{"osd":6,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.65800000000000003,"15min":0.65700000000000003},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.85799999999999998,"5min":1.008,"15min":1.0449999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Tue Dec 19 18:17:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69399999999999995,"15min":0.68899999999999995},"min":{"1min":0.501,"5min":0.439,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":1.169,"15min":1.169},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51100000000000001,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.079,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.68100000000000005}]}]},{"osd":4,"up_from":15,"seq":64424510587,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48140,"kb_used_data":3140,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323700,"statfs":{"total":96636764160,"available":96587468800,"internally_reserved":0,"allocated":3215360,"data_stored":1716230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.72599999999999998,"15min":0.73899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86699999999999999,"5min":1.036,"15min":1.109},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.88100000000000001,"5min":0.997,"15min":1.01},"last":0.71099999999999997}]},{"osd":1,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.752,"15min":0.753},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.90500000000000003,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.753,"15min":0.76200000000000001},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86699999999999999,"5min":0.997,"15min":1.1020000000000001},"last":0.63400000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90500000000000003,"5min":1.0289999999999999,"15min":1.071},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.77400000000000002,"15min":0.77300000000000002},"min":{"1min":0.51800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.91800000000000004,"5min":1.0389999999999999,"15min":1.0760000000000001},"last":0.69899999999999995}]},{"osd":3,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.751,"15min":0.751},"min":{"1min":0.49099999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.84699999999999998,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.77000000000000002,"15min":0.78600000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.85999999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.59999999999999998}]},{"osd":5,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65400000000000003,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.40200000000000002,"15min":0.40100000000000002},"max":{"1min":0.91000000000000003,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":0.89800000000000002,"5min":1.042,"15min":1.042},"last":0.60999999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67300000000000004,"15min":0.65700000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.94899999999999995,"5min":1.024,"15min":1.024},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73699999999999999,"15min":0.71499999999999997},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.90900000000000003,"5min":1.0680000000000001,"15min":1.0760000000000001},"last":0.55400000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.41899999999999998},"max":{"1min":0.93500000000000005,"5min":1.087,"15min":1.087},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77500000000000002,"15min":0.76000000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.46999999999999997},"max":{"1min":0.98499999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65100000000000002}]}]},{"osd":3,"up_from":15,"seq":64424510587,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52232,"kb_used_data":3136,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319608,"statfs":{"total":96636764160,"available":96583278592,"internally_reserved":0,"allocated":3211264,"data_stored":1716756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.49099999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.307,"15min":0.30299999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.372},{"interface":"front","average":{"1min":0.5,"5min":0.52100000000000002,"15min":0.52700000000000002},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.505}]},{"osd":1,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.48999999999999999,"15min":0.495},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.72599999999999998,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56100000000000005,"15min":0.57299999999999995},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67700000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56399999999999995,"15min":0.57199999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73099999999999998,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.58199999999999996,"15min":0.59899999999999998},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.80700000000000005,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.54100000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.79700000000000004,"15min":0.81999999999999995},"min":{"1min":0.54900000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0109999999999999,"5min":1.2769999999999999,"15min":1.4410000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.80500000000000005,"15min":0.81699999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0329999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.76300000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.80600000000000005,"15min":0.83399999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.99199999999999999,"5min":1.1739999999999999,"15min":1.397},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.80100000000000005,"15min":0.82599999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.042,"5min":1.2230000000000001,"15min":1.262},"last":0.77400000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82799999999999996,"15min":0.84499999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0329999999999999,"5min":1.234,"15min":1.274},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80700000000000005,"15min":0.83899999999999997},"min":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.93799999999999994,"5min":1.165,"15min":1.302},"last":0.79200000000000004}]},{"osd":7,"last update":"Tue Dec 19 18:17:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.872,"15min":0.89800000000000002},"min":{"1min":0.65100000000000002,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.153,"5min":1.3540000000000001,"15min":1.486},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.86599999999999999,"15min":0.88600000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1279999999999999,"5min":1.421,"15min":1.421},"last":0.92400000000000004}]}]},{"osd":2,"up_from":15,"seq":64424510586,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52216,"kb_used_data":3120,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319624,"statfs":{"total":96636764160,"available":96583294976,"internally_reserved":0,"allocated":3194880,"data_stored":1714610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42499999999999999,"15min":0.44700000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.622,"5min":0.754,"15min":0.754},"last":0.435},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48099999999999998,"15min":0.5},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66000000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.51500000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.57299999999999995,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.64800000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.36199999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.66200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.441,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.86199999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.36899999999999999},"max":{"1min":0.68400000000000005,"5min":0.871,"15min":0.875},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.94599999999999995,"5min":1.0880000000000001,"15min":1.306},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76500000000000001,"15min":0.78300000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81799999999999995,"5min":1.2070000000000001,"15min":1.3580000000000001},"last":0.89900000000000002}]},{"osd":5,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76800000000000002,"15min":0.79700000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":1.1080000000000001,"15min":1.2050000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89400000000000002,"5min":1.1519999999999999,"15min":1.429},"last":0.90900000000000003}]},{"osd":6,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.83199999999999996},"min":{"1min":0.67500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.91700000000000004,"5min":1.216,"15min":1.365},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76900000000000002,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.117,"15min":1.2},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77800000000000002,"15min":0.81399999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96299999999999997,"5min":1.159,"15min":1.45},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.85099999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0229999999999999,"5min":1.276,"15min":1.276},"last":0.87}]}]},{"osd":1,"up_from":15,"seq":64424510587,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60424,"kb_used_data":3136,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311416,"statfs":{"total":96636764160,"available":96574889984,"internally_reserved":0,"allocated":3211264,"data_stored":1715386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.44800000000000001,"15min":0.45800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.56399999999999995,"5min":0.71999999999999997,"15min":0.89400000000000002},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51400000000000001,"15min":0.52900000000000003},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.79600000000000004,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.36399999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51900000000000002,"15min":0.53000000000000003},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.66000000000000003,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.66600000000000004,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.41199999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.622,"15min":0.629},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.747,"5min":0.94199999999999995,"15min":0.94499999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.35399999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.628,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.434}]},{"osd":4,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.78100000000000003,"15min":0.79300000000000004},"min":{"1min":0.54500000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.92700000000000005,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.77700000000000002,"15min":0.80600000000000005},"min":{"1min":0.52600000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.081,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.875}]},{"osd":5,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76900000000000002,"15min":0.80000000000000004},"min":{"1min":0.58699999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0189999999999999,"5min":1.236,"15min":1.2609999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.81100000000000005,"15min":0.81799999999999995},"min":{"1min":0.58199999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0669999999999999,"5min":1.353,"15min":1.353},"last":0.91700000000000004}]},{"osd":6,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.505},"max":{"1min":1.0009999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.82399999999999995,"15min":0.84099999999999997},"min":{"1min":0.57199999999999995,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.034,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.73899999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84599999999999997,"15min":0.85399999999999998},"min":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.56499999999999995},"max":{"1min":1.147,"5min":1.399,"15min":1.399},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.82799999999999996,"15min":0.85199999999999998},"min":{"1min":0.53000000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.181,"5min":1.365,"15min":1.365},"last":0.79500000000000004}]}]},{"osd":0,"up_from":14,"seq":60129543292,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65084,"kb_used_data":3700,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94306756,"statfs":{"total":96636764160,"available":96570118144,"internally_reserved":0,"allocated":3788800,"data_stored":2304978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.41899999999999998,"15min":0.45300000000000001},"min":{"1min":0.29699999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.55300000000000005,"5min":0.69299999999999995,"15min":1.216},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.49399999999999999,"15min":0.51900000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67600000000000005,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.48799999999999999,"15min":0.52600000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.73899999999999999,"5min":0.77000000000000002,"15min":1.1699999999999999},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.52300000000000002,"15min":0.54700000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.96799999999999997},"last":0.53500000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.51400000000000001,"15min":0.55000000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.59999999999999998,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59899999999999998,"15min":0.64400000000000002},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":1.032},"last":0.441}]},{"osd":4,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.78000000000000003,"15min":0.80600000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.496},"max":{"1min":0.93899999999999995,"5min":1.1200000000000001,"15min":1.381},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.78600000000000003,"15min":0.81100000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.47799999999999998},"max":{"1min":0.84799999999999998,"5min":1.137,"15min":1.2849999999999999},"last":0.63900000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.78800000000000003,"15min":0.81499999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.93799999999999994,"5min":1.1180000000000001,"15min":1.3420000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.79100000000000004,"15min":0.81299999999999994},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.052,"5min":1.1259999999999999,"15min":1.288},"last":0.749}]},{"osd":6,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81200000000000006,"15min":0.83899999999999997},"min":{"1min":0.60899999999999999,"5min":0.58299999999999996,"15min":0.51300000000000001},"max":{"1min":0.97899999999999998,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.80000000000000004,"15min":0.82699999999999996},"min":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97999999999999998,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:17:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.82299999999999995,"15min":0.84299999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.0149999999999999,"5min":1.1899999999999999,"15min":1.415},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.82599999999999996,"15min":0.85099999999999998},"min":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.96899999999999997,"5min":1.0980000000000001,"15min":1.395},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:18:24.076 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:18:24.076 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:18:24.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.264+0000 7fb2968f9640 1 Processor -- start 2023-12-19T18:18:24.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.264+0000 7fb2968f9640 1 -- start start 2023-12-19T18:18:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.264+0000 7fb2968f9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb29006cb20 0x7fb29006f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.264+0000 7fb2968f9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb29015a3c0 0x7fb29015a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.264+0000 7fb2968f9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29006f790 con 0x7fb29006c740 2023-12-19T18:18:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.264+0000 7fb2968f9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29015d5a0 con 0x7fb29006cb20 2023-12-19T18:18:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.264+0000 7fb2968f9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29015d880 con 0x7fb29015a3c0 2023-12-19T18:18:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.264+0000 7fb28ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb29006c740 0x7fb2900ca800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36252/0 (socket says 172.21.15.26:36252) 2023-12-19T18:18:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.264+0000 7fb28ffff640 1 -- 172.21.15.26:0/1741521902 learned_addr learned my addr 172.21.15.26:0/1741521902 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.264+0000 7fb28f7fe640 1 --2- 172.21.15.26:0/1741521902 >> v2:172.21.15.26:3300/0 conn(0x7fb29006cb20 0x7fb29006f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.264+0000 7fb294e96640 1 --2- 172.21.15.26:0/1741521902 >> v2:172.21.15.160:3300/0 conn(0x7fb29015a3c0 0x7fb29015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.264+0000 7fb28effd640 1 -- 172.21.15.26:0/1741521902 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3315383753 0 0) 0x7fb29006f790 con 0x7fb29006c740 2023-12-19T18:18:24.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb28effd640 1 -- 172.21.15.26:0/1741521902 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb278003530 con 0x7fb29006c740 2023-12-19T18:18:24.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb28f7fe640 1 -- 172.21.15.26:0/1741521902 >> v2:172.21.15.160:3300/0 conn(0x7fb29015a3c0 msgr2=0x7fb29015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:24.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb28f7fe640 1 --2- 172.21.15.26:0/1741521902 >> v2:172.21.15.160:3300/0 conn(0x7fb29015a3c0 0x7fb29015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb28f7fe640 1 -- 172.21.15.26:0/1741521902 >> v1:172.21.15.26:6789/0 conn(0x7fb29006c740 legacy=0x7fb2900ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb28f7fe640 1 -- 172.21.15.26:0/1741521902 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29015db60 con 0x7fb29006cb20 2023-12-19T18:18:24.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb28f7fe640 1 --2- 172.21.15.26:0/1741521902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29006cb20 0x7fb29006f250 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fb2900aec40 tx=0x7fb28401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c21d41f5155cae4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:24.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb28effd640 1 -- 172.21.15.26:0/1741521902 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2840a7070 con 0x7fb29006cb20 2023-12-19T18:18:24.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb28effd640 1 -- 172.21.15.26:0/1741521902 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2840ab430 con 0x7fb29006cb20 2023-12-19T18:18:24.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb28effd640 1 -- 172.21.15.26:0/1741521902 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2840ab7c0 con 0x7fb29006cb20 2023-12-19T18:18:24.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb2968f9640 1 -- 172.21.15.26:0/1741521902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29006cb20 msgr2=0x7fb29006f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb2968f9640 1 --2- 172.21.15.26:0/1741521902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29006cb20 0x7fb29006f250 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fb2900aec40 tx=0x7fb28401c430 comp rx=0 tx=0).stop 2023-12-19T18:18:24.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb2968f9640 1 -- 172.21.15.26:0/1741521902 shutdown_connections 2023-12-19T18:18:24.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb2968f9640 1 --2- 172.21.15.26:0/1741521902 >> v2:172.21.15.160:3300/0 conn(0x7fb29015a3c0 0x7fb29015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb2968f9640 1 --2- 172.21.15.26:0/1741521902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29006cb20 0x7fb29006f250 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb2968f9640 1 -- 172.21.15.26:0/1741521902 >> 172.21.15.26:0/1741521902 conn(0x7fb2900660d0 msgr2=0x7fb2900ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:24.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb2968f9640 1 -- 172.21.15.26:0/1741521902 shutdown_connections 2023-12-19T18:18:24.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.268+0000 7fb2968f9640 1 -- 172.21.15.26:0/1741521902 wait complete. 2023-12-19T18:18:24.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb2968f9640 1 Processor -- start 2023-12-19T18:18:24.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb2968f9640 1 -- start start 2023-12-19T18:18:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb2968f9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb29006c740 0x7fb2900c2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb2968f9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb29006cb20 0x7fb2900c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb2968f9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29015a3c0 0x7fb29014edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb2968f9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29014e930 con 0x7fb29015a3c0 2023-12-19T18:18:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb2968f9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29014dd20 con 0x7fb29006c740 2023-12-19T18:18:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb2968f9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29014e000 con 0x7fb29006cb20 2023-12-19T18:18:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb294e96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29015a3c0 0x7fb29014edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb294e96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29015a3c0 0x7fb29014edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38408/0 (socket says 172.21.15.26:38408) 2023-12-19T18:18:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb294e96640 1 -- 172.21.15.26:0/3299259128 learned_addr learned my addr 172.21.15.26:0/3299259128 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb28f7fe640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb29006cb20 0x7fb2900c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb28ffff640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb29006c740 0x7fb2900c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb294e96640 1 -- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb29006cb20 msgr2=0x7fb2900c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb294e96640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb29006cb20 0x7fb2900c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb294e96640 1 -- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb29006c740 msgr2=0x7fb2900c2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb294e96640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb29006c740 0x7fb2900c2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb294e96640 1 -- 172.21.15.26:0/3299259128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb284009420 con 0x7fb29015a3c0 2023-12-19T18:18:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb294e96640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29015a3c0 0x7fb29014edc0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fb28800efc0 tx=0x7fb28800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb28cff9640 1 -- 172.21.15.26:0/3299259128 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb288009ba0 con 0x7fb29015a3c0 2023-12-19T18:18:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb28cff9640 1 -- 172.21.15.26:0/3299259128 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28800f040 con 0x7fb29015a3c0 2023-12-19T18:18:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.272+0000 7fb28cff9640 1 -- 172.21.15.26:0/3299259128 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2880146c0 con 0x7fb29015a3c0 2023-12-19T18:18:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.276+0000 7fb2968f9640 1 -- 172.21.15.26:0/3299259128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29014e2e0 con 0x7fb29015a3c0 2023-12-19T18:18:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.276+0000 7fb2968f9640 1 -- 172.21.15.26:0/3299259128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2900c1a00 con 0x7fb29015a3c0 2023-12-19T18:18:24.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.276+0000 7fb2968f9640 1 -- 172.21.15.26:0/3299259128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb290074270 con 0x7fb29015a3c0 2023-12-19T18:18:24.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.276+0000 7fb28cff9640 1 -- 172.21.15.26:0/3299259128 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb288014820 con 0x7fb29015a3c0 2023-12-19T18:18:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.276+0000 7fb28cff9640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb27c065cd0 0x7fb27c068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.276+0000 7fb28cff9640 1 -- 172.21.15.26:0/3299259128 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fb288084530 con 0x7fb29015a3c0 2023-12-19T18:18:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.276+0000 7fb28ffff640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb27c065cd0 0x7fb27c068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.280+0000 7fb28ffff640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb27c065cd0 0x7fb27c068190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb2800011f0 tx=0x7fb280009040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:24.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.280+0000 7fb28cff9640 1 -- 172.21.15.26:0/3299259128 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb2880516e0 con 0x7fb29015a3c0 2023-12-19T18:18:24.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.492+0000 7fb2968f9640 1 -- 172.21.15.26:0/3299259128 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb29015d310 con 0x7fb29015a3c0 2023-12-19T18:18:24.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.500+0000 7fb28cff9640 1 -- 172.21.15.26:0/3299259128 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v55) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb288054d00 con 0x7fb29015a3c0 2023-12-19T18:18:24.499 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2023-12-19T18:18:24.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.500+0000 7fb2968f9640 1 -- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb27c065cd0 msgr2=0x7fb27c068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.500+0000 7fb2968f9640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb27c065cd0 0x7fb27c068190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb2800011f0 tx=0x7fb280009040 comp rx=0 tx=0).stop 2023-12-19T18:18:24.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.500+0000 7fb2968f9640 1 -- 172.21.15.26:0/3299259128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29015a3c0 msgr2=0x7fb29014edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.500+0000 7fb2968f9640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29015a3c0 0x7fb29014edc0 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7fb28800efc0 tx=0x7fb28800c490 comp rx=0 tx=0).stop 2023-12-19T18:18:24.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.500+0000 7fb2968f9640 1 -- 172.21.15.26:0/3299259128 shutdown_connections 2023-12-19T18:18:24.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.500+0000 7fb2968f9640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb27c065cd0 0x7fb27c068190 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.500+0000 7fb2968f9640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb29015a3c0 0x7fb29014edc0 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.500+0000 7fb2968f9640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb29006cb20 0x7fb2900c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.500+0000 7fb2968f9640 1 --2- 172.21.15.26:0/3299259128 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb29006c740 0x7fb2900c2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.500+0000 7fb2968f9640 1 -- 172.21.15.26:0/3299259128 >> 172.21.15.26:0/3299259128 conn(0x7fb2900660d0 msgr2=0x7fb29006f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:24.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.500+0000 7fb2968f9640 1 -- 172.21.15.26:0/3299259128 shutdown_connections 2023-12-19T18:18:24.502 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.500+0000 7fb2968f9640 1 -- 172.21.15.26:0/3299259128 wait complete. 2023-12-19T18:18:24.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:24.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.664+0000 7f4141fa2640 1 Processor -- start 2023-12-19T18:18:24.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.664+0000 7f4141fa2640 1 -- start start 2023-12-19T18:18:24.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.664+0000 7f4141fa2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f413c0b2030 0x7f413c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.664+0000 7f4141fa2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f413c0affe0 0x7f413c0c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.664+0000 7f4141fa2640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413c15e7c0 con 0x7f413c0b3960 2023-12-19T18:18:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.664+0000 7f4141fa2640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f413c072360 con 0x7f413c0b2030 2023-12-19T18:18:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.664+0000 7f4141fa2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413c0b04c0 con 0x7f413c0affe0 2023-12-19T18:18:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.664+0000 7f4140fa0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f413c0b2030 0x7f413c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.664+0000 7f41417a1640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f413c0b3960 0x7f413c0b3e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36258/0 (socket says 172.21.15.26:36258) 2023-12-19T18:18:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.664+0000 7f41417a1640 1 -- 172.21.15.26:0/1983101280 learned_addr learned my addr 172.21.15.26:0/1983101280 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.664+0000 7f4140fa0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f413c0b2030 0x7f413c071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38412/0 (socket says 172.21.15.26:38412) 2023-12-19T18:18:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.664+0000 7f413bfff640 1 --2- 172.21.15.26:0/1983101280 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f413c0affe0 0x7f413c0c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f4140fa0640 1 -- 172.21.15.26:0/1983101280 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f413c0affe0 msgr2=0x7f413c0c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f4140fa0640 1 --2- 172.21.15.26:0/1983101280 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f413c0affe0 0x7f413c0c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f4140fa0640 1 -- 172.21.15.26:0/1983101280 >> v1:172.21.15.26:6789/0 conn(0x7f413c0b3960 legacy=0x7f413c0b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f413b7fe640 1 -- 172.21.15.26:0/1983101280 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972546470 0 0) 0x7f413c15e7c0 con 0x7f413c0b3960 2023-12-19T18:18:24.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f4140fa0640 1 -- 172.21.15.26:0/1983101280 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413c0b07a0 con 0x7f413c0b2030 2023-12-19T18:18:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f4140fa0640 1 --2- 172.21.15.26:0/1983101280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f413c0b2030 0x7f413c071e20 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f41300097c0 tx=0x7f413001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cdba913df276bb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f413b7fe640 1 -- 172.21.15.26:0/1983101280 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300ab650 con 0x7f413c0b2030 2023-12-19T18:18:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f413b7fe640 1 -- 172.21.15.26:0/1983101280 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41300ab7b0 con 0x7f413c0b2030 2023-12-19T18:18:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f413b7fe640 1 -- 172.21.15.26:0/1983101280 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300abb40 con 0x7f413c0b2030 2023-12-19T18:18:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f4141fa2640 1 -- 172.21.15.26:0/1983101280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f413c0b2030 msgr2=0x7f413c071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f4141fa2640 1 --2- 172.21.15.26:0/1983101280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f413c0b2030 0x7f413c071e20 secure :-1 s=READY pgs=2356 cs=0 l=1 rev1=1 crypto rx=0x7f41300097c0 tx=0x7f413001c450 comp rx=0 tx=0).stop 2023-12-19T18:18:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f4141fa2640 1 -- 172.21.15.26:0/1983101280 shutdown_connections 2023-12-19T18:18:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f4141fa2640 1 --2- 172.21.15.26:0/1983101280 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f413c0affe0 0x7f413c0c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f4141fa2640 1 --2- 172.21.15.26:0/1983101280 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f413c0b2030 0x7f413c071e20 unknown :-1 s=CLOSED pgs=2356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.668+0000 7f4141fa2640 1 -- 172.21.15.26:0/1983101280 >> 172.21.15.26:0/1983101280 conn(0x7f413c06b0f0 msgr2=0x7f413c06b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4141fa2640 1 -- 172.21.15.26:0/1983101280 shutdown_connections 2023-12-19T18:18:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4141fa2640 1 -- 172.21.15.26:0/1983101280 wait complete. 2023-12-19T18:18:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4141fa2640 1 Processor -- start 2023-12-19T18:18:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4141fa2640 1 -- start start 2023-12-19T18:18:24.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4141fa2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f413c0affe0 0x7f413c0c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4141fa2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f413c0b2030 0x7f413c153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4141fa2640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f413c0b3960 0x7f413c159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4141fa2640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c158450 con 0x7f413c0affe0 2023-12-19T18:18:24.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4141fa2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413c0c4730 con 0x7f413c0b2030 2023-12-19T18:18:24.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4141fa2640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413c1541e0 con 0x7f413c0b3960 2023-12-19T18:18:24.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4140fa0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f413c0affe0 0x7f413c0c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4140fa0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f413c0affe0 0x7f413c0c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38414/0 (socket says 172.21.15.26:38414) 2023-12-19T18:18:24.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4140fa0640 1 -- 172.21.15.26:0/136858233 learned_addr learned my addr 172.21.15.26:0/136858233 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:24.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f41417a1640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f413c0b3960 0x7f413c159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f413bfff640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f413c0b2030 0x7f413c153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4140fa0640 1 -- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f413c0b3960 msgr2=0x7f413c159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4140fa0640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f413c0b3960 0x7f413c159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4140fa0640 1 -- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f413c0b2030 msgr2=0x7f413c153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4140fa0640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f413c0b2030 0x7f413c153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4140fa0640 1 -- 172.21.15.26:0/136858233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4130009420 con 0x7f413c0affe0 2023-12-19T18:18:24.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f413bfff640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f413c0b2030 0x7f413c153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:24.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.672+0000 7f4140fa0640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f413c0affe0 0x7f413c0c41f0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f41300ad040 tx=0x7f413001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:24.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.676+0000 7f41397fa640 1 -- 172.21.15.26:0/136858233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300b5810 con 0x7f413c0affe0 2023-12-19T18:18:24.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.676+0000 7f41397fa640 1 -- 172.21.15.26:0/136858233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41300b5970 con 0x7f413c0affe0 2023-12-19T18:18:24.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.676+0000 7f4141fa2640 1 -- 172.21.15.26:0/136858233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f413c157ab0 con 0x7f413c0affe0 2023-12-19T18:18:24.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.676+0000 7f41397fa640 1 -- 172.21.15.26:0/136858233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41300b5d20 con 0x7f413c0affe0 2023-12-19T18:18:24.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.676+0000 7f4141fa2640 1 -- 172.21.15.26:0/136858233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413c158070 con 0x7f413c0affe0 2023-12-19T18:18:24.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.676+0000 7f4141fa2640 1 -- 172.21.15.26:0/136858233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f413c158590 con 0x7f413c0affe0 2023-12-19T18:18:24.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.676+0000 7f41397fa640 1 -- 172.21.15.26:0/136858233 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f41300bf410 con 0x7f413c0affe0 2023-12-19T18:18:24.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.676+0000 7f41397fa640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4120065ca0 0x7f4120068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.676+0000 7f41397fa640 1 -- 172.21.15.26:0/136858233 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f413011e740 con 0x7f413c0affe0 2023-12-19T18:18:24.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.676+0000 7f413bfff640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4120065ca0 0x7f4120068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.680+0000 7f413bfff640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4120065ca0 0x7f4120068160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f413c1f85b0 tx=0x7f4128008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:24.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.680+0000 7f41397fa640 1 -- 172.21.15.26:0/136858233 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f41300ebbe0 con 0x7f413c0affe0 2023-12-19T18:18:24.872 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.872+0000 7f4141fa2640 1 -- 172.21.15.26:0/136858233 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f413c067400 con 0x7f413c0affe0 2023-12-19T18:18:24.873 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.872+0000 7f41397fa640 1 -- 172.21.15.26:0/136858233 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4728 (secure 0 0 0) 0x7f41300ab610 con 0x7f413c0affe0 2023-12-19T18:18:24.873 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:24.873 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":128,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/282680544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":282680544},{"type":"v1","addr":"172.21.15.160:6836","nonce":282680544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3302634175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3302634175},{"type":"v1","addr":"172.21.15.160:6837","nonce":3302634175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3434994961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3434994961},{"type":"v1","addr":"172.21.15.26:6834","nonce":3434994961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:18:19.555514+0000","modified":"2023-12-19T18:18:22.569161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1019381645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1019381645},{"type":"v1","addr":"172.21.15.26:6835","nonce":1019381645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-19T18:18:24.873 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 128 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.872+0000 7f4141fa2640 1 -- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4120065ca0 msgr2=0x7f4120068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.872+0000 7f4141fa2640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4120065ca0 0x7f4120068160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f413c1f85b0 tx=0x7f4128008040 comp rx=0 tx=0).stop 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.872+0000 7f4141fa2640 1 -- 172.21.15.26:0/136858233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f413c0affe0 msgr2=0x7f413c0c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.872+0000 7f4141fa2640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f413c0affe0 0x7f413c0c41f0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f41300ad040 tx=0x7f413001ac90 comp rx=0 tx=0).stop 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.872+0000 7f4141fa2640 1 -- 172.21.15.26:0/136858233 shutdown_connections 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.872+0000 7f4141fa2640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4120065ca0 0x7f4120068160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.872+0000 7f4141fa2640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f413c0b3960 0x7f413c159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.872+0000 7f4141fa2640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f413c0b2030 0x7f413c153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.872+0000 7f4141fa2640 1 --2- 172.21.15.26:0/136858233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f413c0affe0 0x7f413c0c41f0 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.872+0000 7f4141fa2640 1 -- 172.21.15.26:0/136858233 >> 172.21.15.26:0/136858233 conn(0x7f413c06b0f0 msgr2=0x7f413c0ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.876+0000 7f4141fa2640 1 -- 172.21.15.26:0/136858233 shutdown_connections 2023-12-19T18:18:24.876 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:24.876+0000 7f4141fa2640 1 -- 172.21.15.26:0/136858233 wait complete. 2023-12-19T18:18:25.130 INFO:tasks.ceph.mds.c.smithi026.stderr:2023-12-19T18:18:25.128+0000 7f9c0794f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:18:25.298 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 128, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:18:19.555514+0000', 'modified': '2023-12-19T18:18:22.569161+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12183}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12183': {'gid': 12183, 'name': 'c', 'rank': 0, 'incarnation': 124, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/1019381645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1019381645}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1019381645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [33], 'metadata_pool': 32, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:18:25.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:18:25.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-19T18:18:25.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f3174b96640 1 Processor -- start 2023-12-19T18:18:25.445 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f3174b96640 1 -- start start 2023-12-19T18:18:25.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f3174b96640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3170154220 0x7f317015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f3174b96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f317006b3d0 0x7f31700ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.447 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f3174b96640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f31700cec40 0x7f317015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f3174b96640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31700cf0d0 con 0x7f3170154220 2023-12-19T18:18:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f3174b96640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317006bc30 con 0x7f31700cec40 2023-12-19T18:18:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f3174b96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31700cc6e0 con 0x7f317006b3d0 2023-12-19T18:18:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f316e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3170154220 0x7f317015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f316e59c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3170154220 0x7f317015aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38420/0 (socket says 172.21.15.26:38420) 2023-12-19T18:18:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f316e59c640 1 -- 172.21.15.26:0/2025528345 learned_addr learned my addr 172.21.15.26:0/2025528345 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f316e59c640 1 -- 172.21.15.26:0/2025528345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f317006b3d0 msgr2=0x7f31700ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f316e59c640 1 --2- 172.21.15.26:0/2025528345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f317006b3d0 0x7f31700ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f316e59c640 1 -- 172.21.15.26:0/2025528345 >> v2:172.21.15.160:3300/0 conn(0x7f31700cec40 msgr2=0x7f317015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:25.448 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f316ed9d640 1 --2- 172.21.15.26:0/2025528345 >> v2:172.21.15.160:3300/0 conn(0x7f31700cec40 0x7f317015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f316e59c640 1 --2- 172.21.15.26:0/2025528345 >> v2:172.21.15.160:3300/0 conn(0x7f31700cec40 0x7f317015d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f316e59c640 1 -- 172.21.15.26:0/2025528345 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317015afc0 con 0x7f3170154220 2023-12-19T18:18:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.444+0000 7f316ed9d640 1 --2- 172.21.15.26:0/2025528345 >> v2:172.21.15.160:3300/0 conn(0x7f31700cec40 0x7f317015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.448+0000 7f316e59c640 1 --2- 172.21.15.26:0/2025528345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3170154220 0x7f317015aa80 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f31700ae310 tx=0x7f316001c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346fb529d1ff953a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:25.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.448+0000 7f316d59a640 1 -- 172.21.15.26:0/2025528345 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31600ab600 con 0x7f3170154220 2023-12-19T18:18:25.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.448+0000 7f316d59a640 1 -- 172.21.15.26:0/2025528345 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31600a7050 con 0x7f3170154220 2023-12-19T18:18:25.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.448+0000 7f316d59a640 1 -- 172.21.15.26:0/2025528345 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31600ab980 con 0x7f3170154220 2023-12-19T18:18:25.450 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.448+0000 7f3174b96640 1 -- 172.21.15.26:0/2025528345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3170154220 msgr2=0x7f317015aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.448+0000 7f3174b96640 1 --2- 172.21.15.26:0/2025528345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3170154220 0x7f317015aa80 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f31700ae310 tx=0x7f316001c210 comp rx=0 tx=0).stop 2023-12-19T18:18:25.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.448+0000 7f3174b96640 1 -- 172.21.15.26:0/2025528345 shutdown_connections 2023-12-19T18:18:25.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.448+0000 7f3174b96640 1 --2- 172.21.15.26:0/2025528345 >> v2:172.21.15.160:3300/0 conn(0x7f31700cec40 0x7f317015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.451 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.448+0000 7f3174b96640 1 --2- 172.21.15.26:0/2025528345 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f317006b3d0 0x7f31700ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.448+0000 7f3174b96640 1 --2- 172.21.15.26:0/2025528345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3170154220 0x7f317015aa80 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.448+0000 7f3174b96640 1 -- 172.21.15.26:0/2025528345 >> 172.21.15.26:0/2025528345 conn(0x7f31700658e0 msgr2=0x7f3170065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.448+0000 7f3174b96640 1 -- 172.21.15.26:0/2025528345 shutdown_connections 2023-12-19T18:18:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.448+0000 7f3174b96640 1 -- 172.21.15.26:0/2025528345 wait complete. 2023-12-19T18:18:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f3174b96640 1 Processor -- start 2023-12-19T18:18:25.452 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f3174b96640 1 -- start start 2023-12-19T18:18:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f3174b96640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f317006b3d0 0x7f3170151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f3174b96640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31700cec40 0x7f31700ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f3174b96640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3170154220 0x7f3170070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f3174b96640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3170152e20 con 0x7f3170154220 2023-12-19T18:18:25.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f3174b96640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3170070c40 con 0x7f317006b3d0 2023-12-19T18:18:25.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f3174b96640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3170152210 con 0x7f31700cec40 2023-12-19T18:18:25.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f316ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3170154220 0x7f3170070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f316ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3170154220 0x7f3170070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38434/0 (socket says 172.21.15.26:38434) 2023-12-19T18:18:25.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f316ed9d640 1 -- 172.21.15.26:0/917488500 learned_addr learned my addr 172.21.15.26:0/917488500 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:25.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f316e59c640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f317006b3d0 0x7f3170151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f316dd9b640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31700cec40 0x7f31700ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f316ed9d640 1 -- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31700cec40 msgr2=0x7f31700ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f316ed9d640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31700cec40 0x7f31700ad920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f316ed9d640 1 -- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f317006b3d0 msgr2=0x7f3170151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f316ed9d640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f317006b3d0 0x7f3170151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f316ed9d640 1 -- 172.21.15.26:0/917488500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31600094d0 con 0x7f3170154220 2023-12-19T18:18:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f316dd9b640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31700cec40 0x7f31700ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f316ed9d640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3170154220 0x7f3170070700 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f316800cef0 tx=0x7f316800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f315effd640 1 -- 172.21.15.26:0/917488500 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316800d840 con 0x7f3170154220 2023-12-19T18:18:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f315effd640 1 -- 172.21.15.26:0/917488500 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316800d9c0 con 0x7f3170154220 2023-12-19T18:18:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f315effd640 1 -- 172.21.15.26:0/917488500 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316800dd70 con 0x7f3170154220 2023-12-19T18:18:25.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.452+0000 7f3174b96640 1 -- 172.21.15.26:0/917488500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31701524f0 con 0x7f3170154220 2023-12-19T18:18:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.456+0000 7f3174b96640 1 -- 172.21.15.26:0/917488500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3170152ab0 con 0x7f3170154220 2023-12-19T18:18:25.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.456+0000 7f315effd640 1 -- 172.21.15.26:0/917488500 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f316801e3b0 con 0x7f3170154220 2023-12-19T18:18:25.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.456+0000 7f315effd640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3150065cd0 0x7f3150068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.456+0000 7f315effd640 1 -- 172.21.15.26:0/917488500 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f3168084330 con 0x7f3170154220 2023-12-19T18:18:25.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.456+0000 7f3177eea000 1 Processor -- start 2023-12-19T18:18:25.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.456+0000 7f316e59c640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3150065cd0 0x7f3150068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.456+0000 7f3177eea000 1 -- start start 2023-12-19T18:18:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f316e59c640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3150065cd0 0x7f3150068190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31600ad000 tx=0x7f31600a6000 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a9582c0 0x56367a9586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 0x56367a90e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a90e5d0 0x56367a90e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367a72dc90 con 0x56367a958be0 2023-12-19T18:18:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367a75f220 con 0x56367a90e5d0 2023-12-19T18:18:25.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x56367a75c7f0 con 0x56367a9582c0 2023-12-19T18:18:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f316dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 0x56367a90e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f316dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 0x56367a90e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38442/0 (socket says 172.21.15.26:38442) 2023-12-19T18:18:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f316dd9b640 1 -- 172.21.15.26:0/3540780225 learned_addr learned my addr 172.21.15.26:0/3540780225 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f316ed9d640 1 --2- 172.21.15.26:0/3540780225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a90e5d0 0x56367a90e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f316e59c640 1 --2- 172.21.15.26:0/3540780225 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a9582c0 0x56367a9586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f316dd9b640 1 -- 172.21.15.26:0/3540780225 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a9582c0 msgr2=0x56367a9586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f316dd9b640 1 --2- 172.21.15.26:0/3540780225 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a9582c0 0x56367a9586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f316dd9b640 1 -- 172.21.15.26:0/3540780225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a90e5d0 msgr2=0x56367a90e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f316dd9b640 1 --2- 172.21.15.26:0/3540780225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a90e5d0 0x56367a90e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f316dd9b640 1 -- 172.21.15.26:0/3540780225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56367a454890 con 0x56367a958be0 2023-12-19T18:18:25.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f316e59c640 1 --2- 172.21.15.26:0/3540780225 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a9582c0 0x56367a9586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:25.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f316dd9b640 1 --2- 172.21.15.26:0/3540780225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 0x56367a90e090 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f31700b3600 tx=0x7f316400bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2073bd989bb535b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:25.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f314ffff640 1 -- 172.21.15.26:0/3540780225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164018070 con 0x56367a958be0 2023-12-19T18:18:25.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f314ffff640 1 -- 172.21.15.26:0/3540780225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316400be30 con 0x56367a958be0 2023-12-19T18:18:25.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f314ffff640 1 -- 172.21.15.26:0/3540780225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316400d950 con 0x56367a958be0 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 -- 172.21.15.26:0/3540780225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 msgr2=0x56367a90e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 --2- 172.21.15.26:0/3540780225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 0x56367a90e090 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f31700b3600 tx=0x7f316400bbe0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 -- 172.21.15.26:0/3540780225 shutdown_connections 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 --2- 172.21.15.26:0/3540780225 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a90e5d0 0x56367a90e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 --2- 172.21.15.26:0/3540780225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 0x56367a90e090 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 --2- 172.21.15.26:0/3540780225 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a9582c0 0x56367a9586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 -- 172.21.15.26:0/3540780225 >> 172.21.15.26:0/3540780225 conn(0x56367a910950 msgr2=0x56367a908560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 -- 172.21.15.26:0/3540780225 shutdown_connections 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 -- 172.21.15.26:0/3540780225 wait complete. 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.460+0000 7f3177eea000 1 -- start start 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 1 Processor -- start 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 0x56367a945b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 0x56367a946090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a958be0 0x56367a9465d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367a7580e0 con 0x56367a9582c0 2023-12-19T18:18:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367a6f7310 con 0x56367a958be0 2023-12-19T18:18:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x56367a715770 con 0x56367a90e5d0 2023-12-19T18:18:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 10 client.0 _ll_register_callbacks cb 0x56367a8f5b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:18:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f316dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 0x56367a946090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f316dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 0x56367a946090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38450/0 (socket says 172.21.15.26:38450) 2023-12-19T18:18:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f316dd9b640 1 -- 172.21.15.26:0/1381351876 learned_addr learned my addr 172.21.15.26:0/1381351876 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f316e59c640 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 0x56367a945b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f316ed9d640 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a958be0 0x56367a9465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f316dd9b640 1 -- 172.21.15.26:0/1381351876 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 msgr2=0x56367a945b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f316dd9b640 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 0x56367a945b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f316dd9b640 1 -- 172.21.15.26:0/1381351876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a958be0 msgr2=0x56367a9465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:25.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f316dd9b640 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a958be0 0x56367a9465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f316dd9b640 1 -- 172.21.15.26:0/1381351876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316400e040 con 0x56367a9582c0 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f316ed9d640 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a958be0 0x56367a9465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f316dd9b640 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 0x56367a946090 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x56367a808540 tx=0x7f31640046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5a5bf7e5314b0e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f314d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f314d7fa640 1 -- 172.21.15.26:0/1381351876 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316400a040 con 0x56367a9582c0 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f314d7fa640 1 -- 172.21.15.26:0/1381351876 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f316401d460 con 0x56367a9582c0 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 1 -- 172.21.15.26:0/1381351876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56367a727c10 con 0x56367a9582c0 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f314d7fa640 1 -- 172.21.15.26:0/1381351876 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3164018040 con 0x56367a9582c0 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 10 client.12333 fetch_fsmap learned FSMap version 128 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 1 -- 172.21.15.26:0/1381351876 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x56367a71d260 con 0x56367a9582c0 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f314d7fa640 1 -- 172.21.15.26:0/1381351876 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56367a727c10 con 0x56367a9582c0 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f314d7fa640 1 -- 172.21.15.26:0/1381351876 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 128) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f316401d620 con 0x56367a9582c0 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 10 client.12333 fetch_fsmap finished waiting for FSMap version 128 2023-12-19T18:18:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 10 client.12333 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:18:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 20 client.12333 populate_metadata read hostname 'smithi026' 2023-12-19T18:18:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.26:6833/1019381645,v1:172.21.15.26:6835/1019381645] conn(0x56367a946e70 0x56367a969830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 4 client.12333 mds_command: new command op to 12183 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:18:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f3177eea000 1 -- 172.21.15.26:0/1381351876 --> [v2:172.21.15.26:6833/1019381645,v1:172.21.15.26:6835/1019381645] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56367a756a30 con 0x56367a946e70 2023-12-19T18:18:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.464+0000 7f316e59c640 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.26:6833/1019381645,v1:172.21.15.26:6835/1019381645] conn(0x56367a946e70 0x56367a969830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f314d7fa640 10 client.12333 ms_handle_connect on v2:172.21.15.26:6833/1019381645 2023-12-19T18:18:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f316e59c640 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.26:6833/1019381645,v1:172.21.15.26:6835/1019381645] conn(0x56367a946e70 0x56367a969830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d609660f2df4db05 server_cookie=88bcdb092aaf4367 in_seq=0 out_seq=0 2023-12-19T18:18:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f314d7fa640 1 -- 172.21.15.26:0/1381351876 <== mds.0 v2:172.21.15.26:6833/1019381645 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56367a756a30 con 0x56367a946e70 2023-12-19T18:18:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f314d7fa640 10 client.12333 handle_command_reply: tid=0 2023-12-19T18:18:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f3177eea000 1 client.12333 shutdown 2023-12-19T18:18:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f3177eea000 1 -- 172.21.15.26:0/1381351876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 msgr2=0x56367a946090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f3177eea000 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 0x56367a946090 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x56367a808540 tx=0x7f31640046d0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f3177eea000 1 -- 172.21.15.26:0/1381351876 shutdown_connections 2023-12-19T18:18:25.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f3177eea000 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.26:6833/1019381645,v1:172.21.15.26:6835/1019381645] conn(0x56367a946e70 0x56367a969830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f3177eea000 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a958be0 0x56367a9465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f3177eea000 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 0x56367a946090 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f3177eea000 1 --2- 172.21.15.26:0/1381351876 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 0x56367a945b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f314d7fa640 0 client.12333 ms_handle_reset on v2:172.21.15.26:6833/1019381645 2023-12-19T18:18:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f3177eea000 1 -- 172.21.15.26:0/1381351876 >> 172.21.15.26:0/1381351876 conn(0x56367a910950 msgr2=0x56367a907cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f3177eea000 1 -- 172.21.15.26:0/1381351876 shutdown_connections 2023-12-19T18:18:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f3177eea000 1 -- 172.21.15.26:0/1381351876 wait complete. 2023-12-19T18:18:25.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.468+0000 7f3177eea000 20 client.12333 trim_cache size 0 max 16384 2023-12-19T18:18:25.483 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 Processor -- start 2023-12-19T18:18:25.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 -- start start 2023-12-19T18:18:25.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 0x56367a9afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 0x56367a9b0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a958be0 0x56367a9b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367a415940 con 0x56367a9582c0 2023-12-19T18:18:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367a7558f0 con 0x56367a958be0 2023-12-19T18:18:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x56367a700860 con 0x56367a90e5d0 2023-12-19T18:18:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f316dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 0x56367a9b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f316dd9b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 0x56367a9b0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38456/0 (socket says 172.21.15.26:38456) 2023-12-19T18:18:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f316dd9b640 1 -- 172.21.15.26:0/3284030115 learned_addr learned my addr 172.21.15.26:0/3284030115 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f316e59c640 1 --2- 172.21.15.26:0/3284030115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 0x56367a9afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f316ed9d640 1 --2- 172.21.15.26:0/3284030115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a958be0 0x56367a9b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f316dd9b640 1 -- 172.21.15.26:0/3284030115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 msgr2=0x56367a9afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f316dd9b640 1 --2- 172.21.15.26:0/3284030115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 0x56367a9afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f316dd9b640 1 -- 172.21.15.26:0/3284030115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a958be0 msgr2=0x56367a9b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f316dd9b640 1 --2- 172.21.15.26:0/3284030115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a958be0 0x56367a9b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f316dd9b640 1 -- 172.21.15.26:0/3284030115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56367a7033b0 con 0x56367a9582c0 2023-12-19T18:18:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f316dd9b640 1 --2- 172.21.15.26:0/3284030115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 0x56367a9b0200 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f316400bf70 tx=0x7f316401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1da416f1896814d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f315cff9640 1 -- 172.21.15.26:0/3284030115 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164018040 con 0x56367a9582c0 2023-12-19T18:18:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f315cff9640 1 -- 172.21.15.26:0/3284030115 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316400db60 con 0x56367a9582c0 2023-12-19T18:18:25.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f315cff9640 1 -- 172.21.15.26:0/3284030115 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316401d6a0 con 0x56367a9582c0 2023-12-19T18:18:25.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 -- 172.21.15.26:0/3284030115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 msgr2=0x56367a9b0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 --2- 172.21.15.26:0/3284030115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 0x56367a9b0200 secure :-1 s=READY pgs=2362 cs=0 l=1 rev1=1 crypto rx=0x7f316400bf70 tx=0x7f316401b000 comp rx=0 tx=0).stop 2023-12-19T18:18:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 -- 172.21.15.26:0/3284030115 shutdown_connections 2023-12-19T18:18:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 --2- 172.21.15.26:0/3284030115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a958be0 0x56367a9b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 --2- 172.21.15.26:0/3284030115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a9582c0 0x56367a9b0200 unknown :-1 s=CLOSED pgs=2362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 --2- 172.21.15.26:0/3284030115 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 0x56367a9afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 -- 172.21.15.26:0/3284030115 >> 172.21.15.26:0/3284030115 conn(0x56367a910950 msgr2=0x56367a90e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 -- 172.21.15.26:0/3284030115 shutdown_connections 2023-12-19T18:18:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 -- 172.21.15.26:0/3284030115 wait complete. 2023-12-19T18:18:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.484+0000 7f3177eea000 1 -- start start 2023-12-19T18:18:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 1 Processor -- start 2023-12-19T18:18:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 0x56367a944af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a9582c0 0x56367a968870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 0x56367a968db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367a819eb0 con 0x56367a958be0 2023-12-19T18:18:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367a711170 con 0x56367a9582c0 2023-12-19T18:18:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x56367a7e0760 con 0x56367a90e5d0 2023-12-19T18:18:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 10 client.0 _ll_register_callbacks cb 0x56367a706640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:18:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 0x56367a968db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316ed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 0x56367a968db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38466/0 (socket says 172.21.15.26:38466) 2023-12-19T18:18:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316ed9d640 1 -- 172.21.15.26:0/1784791873 learned_addr learned my addr 172.21.15.26:0/1784791873 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316e59c640 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 0x56367a944af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316dd9b640 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a9582c0 0x56367a968870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316ed9d640 1 -- 172.21.15.26:0/1784791873 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 msgr2=0x56367a944af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316ed9d640 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 0x56367a944af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316ed9d640 1 -- 172.21.15.26:0/1784791873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a9582c0 msgr2=0x56367a968870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316ed9d640 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a9582c0 0x56367a968870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316ed9d640 1 -- 172.21.15.26:0/1784791873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56367a71d260 con 0x56367a958be0 2023-12-19T18:18:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316e59c640 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 0x56367a944af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316ed9d640 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 0x56367a968db0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f3168051920 tx=0x7f316800adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e0cce9afd9c372 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f314d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2023-12-19T18:18:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f314d7fa640 1 -- 172.21.15.26:0/1784791873 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3168054b10 con 0x56367a958be0 2023-12-19T18:18:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f314d7fa640 1 -- 172.21.15.26:0/1784791873 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f3168053880 con 0x56367a958be0 2023-12-19T18:18:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 1 -- 172.21.15.26:0/1784791873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56367a71de00 con 0x56367a958be0 2023-12-19T18:18:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f314d7fa640 1 -- 172.21.15.26:0/1784791873 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7f316805b2c0 con 0x56367a958be0 2023-12-19T18:18:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 10 client.12339 fetch_fsmap learned FSMap version 128 2023-12-19T18:18:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 1 -- 172.21.15.26:0/1784791873 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x56367a454cc0 con 0x56367a958be0 2023-12-19T18:18:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f314d7fa640 1 -- 172.21.15.26:0/1784791873 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56367a71de00 con 0x56367a958be0 2023-12-19T18:18:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f314d7fa640 1 -- 172.21.15.26:0/1784791873 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 128) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f3168005d20 con 0x56367a958be0 2023-12-19T18:18:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 10 client.12339 fetch_fsmap finished waiting for FSMap version 128 2023-12-19T18:18:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 10 client.12339 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:18:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 20 client.12339 populate_metadata read hostname 'smithi026' 2023-12-19T18:18:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.26:6833/1019381645,v1:172.21.15.26:6835/1019381645] conn(0x56367a9b4160 0x56367a9b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 4 client.12339 mds_command: new command op to 12183 tid=0[{"prefix": "status"}] 2023-12-19T18:18:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 1 -- 172.21.15.26:0/1784791873 --> [v2:172.21.15.26:6833/1019381645,v1:172.21.15.26:6835/1019381645] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56367a751160 con 0x56367a9b4160 2023-12-19T18:18:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316e59c640 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.26:6833/1019381645,v1:172.21.15.26:6835/1019381645] conn(0x56367a9b4160 0x56367a9b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f314d7fa640 10 client.12339 ms_handle_connect on v2:172.21.15.26:6833/1019381645 2023-12-19T18:18:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f316e59c640 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.26:6833/1019381645,v1:172.21.15.26:6835/1019381645] conn(0x56367a9b4160 0x56367a9b4540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48eefc65e03e78e5 server_cookie=39870bc7bc117e1f in_seq=0 out_seq=0 2023-12-19T18:18:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f314d7fa640 1 -- 172.21.15.26:0/1784791873 <== mds.0 v2:172.21.15.26:6833/1019381645 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x56367a751160 con 0x56367a9b4160 2023-12-19T18:18:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f314d7fa640 10 client.12339 handle_command_reply: tid=0 2023-12-19T18:18:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.488+0000 7f3177eea000 1 client.12339 shutdown 2023-12-19T18:18:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3177eea000 1 -- 172.21.15.26:0/1784791873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 msgr2=0x56367a968db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3177eea000 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 0x56367a968db0 secure :-1 s=READY pgs=2363 cs=0 l=1 rev1=1 crypto rx=0x7f3168051920 tx=0x7f316800adc0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3177eea000 1 -- 172.21.15.26:0/1784791873 shutdown_connections 2023-12-19T18:18:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3177eea000 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.26:6833/1019381645,v1:172.21.15.26:6835/1019381645] conn(0x56367a9b4160 0x56367a9b4540 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3177eea000 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x56367a958be0 0x56367a968db0 unknown :-1 s=CLOSED pgs=2363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3177eea000 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56367a9582c0 0x56367a968870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f314d7fa640 0 client.12339 ms_handle_reset on v2:172.21.15.26:6833/1019381645 2023-12-19T18:18:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3177eea000 1 --2- 172.21.15.26:0/1784791873 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x56367a90e5d0 0x56367a944af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3177eea000 1 -- 172.21.15.26:0/1784791873 >> 172.21.15.26:0/1784791873 conn(0x56367a910950 msgr2=0x56367a90f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3177eea000 1 -- 172.21.15.26:0/1784791873 shutdown_connections 2023-12-19T18:18:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3177eea000 1 -- 172.21.15.26:0/1784791873 wait complete. 2023-12-19T18:18:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3177eea000 20 client.12339 trim_cache size 0 max 16384 2023-12-19T18:18:25.498 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:18:25.499 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "2264810b-2623-438e-8790-ac9d6fbbbfd5", 2023-12-19T18:18:25.499 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2023-12-19T18:18:25.499 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 12183, 2023-12-19T18:18:25.499 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2023-12-19T18:18:25.499 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2023-12-19T18:18:25.499 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2023-12-19T18:18:25.499 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 5.933897118, 2023-12-19T18:18:25.499 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 128, 2023-12-19T18:18:25.499 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 254, 2023-12-19T18:18:25.499 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 254, 2023-12-19T18:18:25.499 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 11.220056852000001 2023-12-19T18:18:25.499 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:18:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3174b96640 1 -- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3150065cd0 msgr2=0x7f3150068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3174b96640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3150065cd0 0x7f3150068190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f31600ad000 tx=0x7f31600a6000 comp rx=0 tx=0).stop 2023-12-19T18:18:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3174b96640 1 -- 172.21.15.26:0/917488500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3170154220 msgr2=0x7f3170070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3174b96640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3170154220 0x7f3170070700 secure :-1 s=READY pgs=2359 cs=0 l=1 rev1=1 crypto rx=0x7f316800cef0 tx=0x7f316800a710 comp rx=0 tx=0).stop 2023-12-19T18:18:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3174b96640 1 -- 172.21.15.26:0/917488500 shutdown_connections 2023-12-19T18:18:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3174b96640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3150065cd0 0x7f3150068190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3174b96640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3170154220 0x7f3170070700 unknown :-1 s=CLOSED pgs=2359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3174b96640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31700cec40 0x7f31700ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3174b96640 1 --2- 172.21.15.26:0/917488500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f317006b3d0 0x7f3170151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3174b96640 1 -- 172.21.15.26:0/917488500 >> 172.21.15.26:0/917488500 conn(0x7f31700658e0 msgr2=0x7f31701500d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3174b96640 1 -- 172.21.15.26:0/917488500 shutdown_connections 2023-12-19T18:18:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.492+0000 7f3174b96640 1 -- 172.21.15.26:0/917488500 wait complete. 2023-12-19T18:18:25.511 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:18:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.652+0000 7fe9a436d640 1 Processor -- start 2023-12-19T18:18:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.652+0000 7fe9a436d640 1 -- start start 2023-12-19T18:18:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a436d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99c154220 0x7fe99c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a436d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe99c06b3d0 0x7fe99c0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a436d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99c06bc30 con 0x7fe99c0cec40 2023-12-19T18:18:25.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a436d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99c0cf0d0 con 0x7fe99c06b3d0 2023-12-19T18:18:25.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a436d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c0cc6e0 con 0x7fe99c154220 2023-12-19T18:18:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a1908640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe99c06b3d0 0x7fe99c0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a2109640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99c154220 0x7fe99c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a1908640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe99c06b3d0 0x7fe99c0ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:53528/0 (socket says 172.21.15.26:53528) 2023-12-19T18:18:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a1908640 1 -- 172.21.15.26:0/252843177 learned_addr learned my addr 172.21.15.26:0/252843177 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a2109640 1 -- 172.21.15.26:0/252843177 >> v2:172.21.15.160:3300/0 conn(0x7fe99c06b3d0 msgr2=0x7fe99c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a2109640 1 --2- 172.21.15.26:0/252843177 >> v2:172.21.15.160:3300/0 conn(0x7fe99c06b3d0 0x7fe99c0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a2109640 1 -- 172.21.15.26:0/252843177 >> v1:172.21.15.160:6789/0 conn(0x7fe99c0cec40 legacy=0x7fe99c15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a2109640 1 -- 172.21.15.26:0/252843177 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c15afc0 con 0x7fe99c154220 2023-12-19T18:18:25.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.656+0000 7fe9a1908640 1 --2- 172.21.15.26:0/252843177 >> v2:172.21.15.160:3300/0 conn(0x7fe99c06b3d0 0x7fe99c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:25.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a2109640 1 --2- 172.21.15.26:0/252843177 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99c154220 0x7fe99c15aa80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe9980097c0 tx=0x7fe99801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cc7026927c51b5b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a1107640 1 -- 172.21.15.26:0/252843177 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9980ab600 con 0x7fe99c154220 2023-12-19T18:18:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a1107640 1 -- 172.21.15.26:0/252843177 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9980a7050 con 0x7fe99c154220 2023-12-19T18:18:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a1107640 1 -- 172.21.15.26:0/252843177 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9980ab980 con 0x7fe99c154220 2023-12-19T18:18:25.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a436d640 1 -- 172.21.15.26:0/252843177 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99c154220 msgr2=0x7fe99c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a436d640 1 --2- 172.21.15.26:0/252843177 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99c154220 0x7fe99c15aa80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe9980097c0 tx=0x7fe99801c450 comp rx=0 tx=0).stop 2023-12-19T18:18:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a436d640 1 -- 172.21.15.26:0/252843177 shutdown_connections 2023-12-19T18:18:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a436d640 1 --2- 172.21.15.26:0/252843177 >> v2:172.21.15.160:3300/0 conn(0x7fe99c06b3d0 0x7fe99c0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a436d640 1 --2- 172.21.15.26:0/252843177 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99c154220 0x7fe99c15aa80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a436d640 1 -- 172.21.15.26:0/252843177 >> 172.21.15.26:0/252843177 conn(0x7fe99c0658e0 msgr2=0x7fe99c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a436d640 1 -- 172.21.15.26:0/252843177 shutdown_connections 2023-12-19T18:18:25.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a436d640 1 -- 172.21.15.26:0/252843177 wait complete. 2023-12-19T18:18:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a436d640 1 Processor -- start 2023-12-19T18:18:25.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.660+0000 7fe9a436d640 1 -- start start 2023-12-19T18:18:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a436d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99c06b3d0 0x7fe99c14cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a436d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe99c0cec40 0x7fe99c14b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a436d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99c154220 0x7fe99c0b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a436d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c14b900 con 0x7fe99c154220 2023-12-19T18:18:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a436d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c14dd80 con 0x7fe99c0cec40 2023-12-19T18:18:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a436d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c0b3a30 con 0x7fe99c06b3d0 2023-12-19T18:18:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a290a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99c154220 0x7fe99c0b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a290a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99c154220 0x7fe99c0b34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38474/0 (socket says 172.21.15.26:38474) 2023-12-19T18:18:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a290a640 1 -- 172.21.15.26:0/3148845604 learned_addr learned my addr 172.21.15.26:0/3148845604 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a1908640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe99c0cec40 0x7fe99c14b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a2109640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99c06b3d0 0x7fe99c14cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a290a640 1 -- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99c06b3d0 msgr2=0x7fe99c14cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a290a640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99c06b3d0 0x7fe99c14cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a290a640 1 -- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe99c0cec40 msgr2=0x7fe99c14b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a290a640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe99c0cec40 0x7fe99c14b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a290a640 1 -- 172.21.15.26:0/3148845604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c00b770 con 0x7fe99c154220 2023-12-19T18:18:25.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a290a640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99c154220 0x7fe99c0b34f0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fe99400c760 tx=0x7fe99400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:25.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe992ffd640 1 -- 172.21.15.26:0/3148845604 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe994003b80 con 0x7fe99c154220 2023-12-19T18:18:25.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe992ffd640 1 -- 172.21.15.26:0/3148845604 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe994003ce0 con 0x7fe99c154220 2023-12-19T18:18:25.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe992ffd640 1 -- 172.21.15.26:0/3148845604 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99400aa30 con 0x7fe99c154220 2023-12-19T18:18:25.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a436d640 1 -- 172.21.15.26:0/3148845604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe998009420 con 0x7fe99c154220 2023-12-19T18:18:25.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a436d640 1 -- 172.21.15.26:0/3148845604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99c14d450 con 0x7fe99c154220 2023-12-19T18:18:25.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.664+0000 7fe9a436d640 1 -- 172.21.15.26:0/3148845604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99c074270 con 0x7fe99c154220 2023-12-19T18:18:25.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.668+0000 7fe992ffd640 1 -- 172.21.15.26:0/3148845604 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fe99400ab90 con 0x7fe99c154220 2023-12-19T18:18:25.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.668+0000 7fe992ffd640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe980065ca0 0x7fe980068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.668+0000 7fe992ffd640 1 -- 172.21.15.26:0/3148845604 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fe9940834a0 con 0x7fe99c154220 2023-12-19T18:18:25.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.668+0000 7fe9a2109640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe980065ca0 0x7fe980068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.668+0000 7fe9a2109640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe980065ca0 0x7fe980068160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe9980ad000 tx=0x7fe998008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:25.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.672+0000 7fe992ffd640 1 -- 172.21.15.26:0/3148845604 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe994050940 con 0x7fe99c154220 2023-12-19T18:18:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.864+0000 7fe9a436d640 1 -- 172.21.15.26:0/3148845604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe99c14dec0 con 0x7fe99c154220 2023-12-19T18:18:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.864+0000 7fe992ffd640 1 -- 172.21.15.26:0/3148845604 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v128) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe994053f60 con 0x7fe99c154220 2023-12-19T18:18:25.865 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:18:25.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.868+0000 7fe9a436d640 1 -- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe980065ca0 msgr2=0x7fe980068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.868+0000 7fe9a436d640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe980065ca0 0x7fe980068160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe9980ad000 tx=0x7fe998008040 comp rx=0 tx=0).stop 2023-12-19T18:18:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.868+0000 7fe9a436d640 1 -- 172.21.15.26:0/3148845604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99c154220 msgr2=0x7fe99c0b34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.868+0000 7fe9a436d640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99c154220 0x7fe99c0b34f0 secure :-1 s=READY pgs=2364 cs=0 l=1 rev1=1 crypto rx=0x7fe99400c760 tx=0x7fe99400cc20 comp rx=0 tx=0).stop 2023-12-19T18:18:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.868+0000 7fe9a436d640 1 -- 172.21.15.26:0/3148845604 shutdown_connections 2023-12-19T18:18:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.868+0000 7fe9a436d640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fe980065ca0 0x7fe980068160 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.868+0000 7fe9a436d640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe99c154220 0x7fe99c0b34f0 unknown :-1 s=CLOSED pgs=2364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.868+0000 7fe9a436d640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe99c0cec40 0x7fe99c14b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.868+0000 7fe9a436d640 1 --2- 172.21.15.26:0/3148845604 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fe99c06b3d0 0x7fe99c14cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.868+0000 7fe9a436d640 1 -- 172.21.15.26:0/3148845604 >> 172.21.15.26:0/3148845604 conn(0x7fe99c0658e0 msgr2=0x7fe99c0cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.868+0000 7fe9a436d640 1 -- 172.21.15.26:0/3148845604 shutdown_connections 2023-12-19T18:18:25.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:25.868+0000 7fe9a436d640 1 -- 172.21.15.26:0/3148845604 wait complete. 2023-12-19T18:18:25.886 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-19T18:18:25.887 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:18:25.887 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2023-12-19T18:18:25.887 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:18:25.887 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:18:25.887 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-19T18:18:25.887 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-19T18:18:25.887 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:18:25.887 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:18:25.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:25.887 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 36635sec preferred_lft 36635sec 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-19T18:18:25.900 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:25.900 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2023-12-19T18:18:25.900 INFO:teuthology.orchestra.run.smithi026.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-19T18:18:25.900 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 02:42:1e:d5:36:ea brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:25.900 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-19T18:18:25.900 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:25.900 INFO:teuthology.orchestra.run.smithi026.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:18:25.900 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:25.900 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:25.902 INFO:teuthology.orchestra.run.smithi026.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:18:25.902 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:18:25.902 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-19T18:18:25.902 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:25.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:25.902 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:18:25.958 INFO:teuthology.orchestra.run.smithi026.stdout:1 2023-12-19T18:18:25.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:25.960 DEBUG:teuthology.orchestra.run.smithi026:> route 2023-12-19T18:18:26.016 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2023-12-19T18:18:26.017 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:18:26.017 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-19T18:18:26.017 INFO:teuthology.orchestra.run.smithi026.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-19T18:18:26.017 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-19T18:18:26.017 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:18:26.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:26.017 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2023-12-19T18:18:26.017 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2023-12-19T18:18:26.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:18:26.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:18:26.018 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:18:26.018 DEBUG:teuthology.orchestra.run.smithi026:> ') 2023-12-19T18:18:26.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:26.113 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T18:18:26.122 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:18:26.122 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:18:26.123 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:18:26.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:26.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2023-12-19T18:18:26.192 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:18:26.192 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:18:26.243 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:18:26.243 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:18:26.244 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:18:26.295 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-19T18:18:26.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:26.296 DEBUG:teuthology.orchestra.run.smithi026:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T18:18:26.390 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T18:18:26.390 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T18:18:26.404 INFO:teuthology.orchestra.run.smithi026.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-19T18:18:26.404 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-19T18:18:26.404 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: resolved to: "172.21.15.26:3300,172.21.15.160:3300,172.21.15.160:3301" 2023-12-19T18:18:26.404 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: trying mount with new device syntax: 0@2264810b-2623-438e-8790-ac9d6fbbbfd5.cephfs=/ 2023-12-19T18:18:26.404 INFO:teuthology.orchestra.run.smithi026.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.26:3300/172.21.15.160:3300/172.21.15.160:3301" will pass to kernel 2023-12-19T18:18:26.408 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-19T18:18:26.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:26.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:18:26.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:26.430 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-19T18:18:26.496 INFO:teuthology.orchestra.run.smithi026.stdout:client11563 2023-12-19T18:18:26.496 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:18:26.496 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:26.548 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-19T18:18:26.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.732+0000 7ff85b4fc640 1 Processor -- start 2023-12-19T18:18:26.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.732+0000 7ff85b4fc640 1 -- start start 2023-12-19T18:18:26.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff85b4fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8540cbc50 0x7ff8540b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff85b4fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff8540b3310 0x7ff8540ad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff85b4fc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8540cc0d0 con 0x7ff85406cb30 2023-12-19T18:18:26.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff85b4fc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85415ddc0 con 0x7ff8540b3310 2023-12-19T18:18:26.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff85b4fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85415e0a0 con 0x7ff8540cbc50 2023-12-19T18:18:26.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff859298640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff85406cb30 0x7ff85415d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52724/0 (socket says 172.21.15.26:52724) 2023-12-19T18:18:26.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff859298640 1 -- 172.21.15.26:0/938249522 learned_addr learned my addr 172.21.15.26:0/938249522 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:26.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff859a99640 1 --2- 172.21.15.26:0/938249522 >> v2:172.21.15.160:3300/0 conn(0x7ff8540b3310 0x7ff8540ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff858a97640 1 --2- 172.21.15.26:0/938249522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8540cbc50 0x7ff8540b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867786691 0 0) 0x7ff850001120 con 0x7ff85406cb30 2023-12-19T18:18:26.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff840003530 con 0x7ff85406cb30 2023-12-19T18:18:26.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 649666832 0 0) 0x7ff840003530 con 0x7ff85406cb30 2023-12-19T18:18:26.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8540cbc50 msgr2=0x7ff8540b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:26.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 --2- 172.21.15.26:0/938249522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8540cbc50 0x7ff8540b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.738 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 >> v2:172.21.15.160:3300/0 conn(0x7ff8540b3310 msgr2=0x7ff8540ad180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 --2- 172.21.15.26:0/938249522 >> v2:172.21.15.160:3300/0 conn(0x7ff8540b3310 0x7ff8540ad180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8540cb450 con 0x7ff85406cb30 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7ff850001810 con 0x7ff85406cb30 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 >> v1:172.21.15.26:6789/0 conn(0x7ff85406cb30 legacy=0x7ff85415d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 --2- 172.21.15.26:0/938249522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff840003980 0x7ff840003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 --2- 172.21.15.26:0/938249522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8400047d0 0x7ff840004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 --2- 172.21.15.26:0/938249522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8540cbc50 0x7ff840005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85415ddc0 con 0x7ff840003980 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85415e0a0 con 0x7ff8400047d0 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84000b990 con 0x7ff8540cbc50 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff859298640 1 --2- 172.21.15.26:0/938249522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff840003980 0x7ff840003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff858a97640 1 --2- 172.21.15.26:0/938249522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8400047d0 0x7ff840004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff859a99640 1 --2- 172.21.15.26:0/938249522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8540cbc50 0x7ff840005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff859298640 1 -- 172.21.15.26:0/938249522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8540cbc50 msgr2=0x7ff840005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff859298640 1 --2- 172.21.15.26:0/938249522 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8540cbc50 0x7ff840005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff859298640 1 -- 172.21.15.26:0/938249522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8400047d0 msgr2=0x7ff840004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff859298640 1 --2- 172.21.15.26:0/938249522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff8400047d0 0x7ff840004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.736+0000 7ff859298640 1 -- 172.21.15.26:0/938249522 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff840003530 con 0x7ff840003980 2023-12-19T18:18:26.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.740+0000 7ff859298640 1 --2- 172.21.15.26:0/938249522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff840003980 0x7ff840003da0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7ff85000c210 tx=0x7ff85001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ce17b01bb34362 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:26.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.740+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8540aded0 con 0x7ff840003980 2023-12-19T18:18:26.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.740+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8500058e0 con 0x7ff840003980 2023-12-19T18:18:26.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.740+0000 7ff84bfff640 1 -- 172.21.15.26:0/938249522 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850005cb0 con 0x7ff840003980 2023-12-19T18:18:26.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.740+0000 7ff85b4fc640 1 -- 172.21.15.26:0/938249522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff840003980 msgr2=0x7ff840003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.740+0000 7ff85b4fc640 1 --2- 172.21.15.26:0/938249522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff840003980 0x7ff840003da0 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7ff85000c210 tx=0x7ff85001cc30 comp rx=0 tx=0).stop 2023-12-19T18:18:26.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.740+0000 7ff859a99640 1 -- 172.21.15.26:0/938249522 reap_dead start 2023-12-19T18:18:26.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.740+0000 7ff85b4fc640 1 -- 172.21.15.26:0/938249522 shutdown_connections 2023-12-19T18:18:26.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.740+0000 7ff85b4fc640 1 -- 172.21.15.26:0/938249522 >> 172.21.15.26:0/938249522 conn(0x7ff8540658e0 msgr2=0x7ff854065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:26.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.740+0000 7ff85b4fc640 1 -- 172.21.15.26:0/938249522 shutdown_connections 2023-12-19T18:18:26.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.740+0000 7ff85b4fc640 1 -- 172.21.15.26:0/938249522 wait complete. 2023-12-19T18:18:26.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.740+0000 7ff85b4fc640 1 Processor -- start 2023-12-19T18:18:26.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.740+0000 7ff85b4fc640 1 -- start start 2023-12-19T18:18:26.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff85b4fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff840003980 0x7ff854142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff85b4fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8541640d0 0x7ff8541f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff85b4fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff854164550 0x7ff8541f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff85b4fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8541636c0 con 0x7ff854164550 2023-12-19T18:18:26.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff85b4fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8541f6340 con 0x7ff840003980 2023-12-19T18:18:26.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff85b4fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8541f6620 con 0x7ff8541640d0 2023-12-19T18:18:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff859a99640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff854164550 0x7ff8541f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff859a99640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff854164550 0x7ff8541f5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37862/0 (socket says 172.21.15.26:37862) 2023-12-19T18:18:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff859a99640 1 -- 172.21.15.26:0/3943883758 learned_addr learned my addr 172.21.15.26:0/3943883758 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff859298640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff840003980 0x7ff854142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff858a97640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8541640d0 0x7ff8541f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff859a99640 1 -- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8541640d0 msgr2=0x7ff8541f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff859a99640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8541640d0 0x7ff8541f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff859a99640 1 -- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff840003980 msgr2=0x7ff854142450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff859a99640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff840003980 0x7ff854142450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff859a99640 1 -- 172.21.15.26:0/3943883758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8541638d0 con 0x7ff854164550 2023-12-19T18:18:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff859a99640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff854164550 0x7ff8541f5d50 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7ff85415e3b0 tx=0x7ff84c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff849ffb640 1 -- 172.21.15.26:0/3943883758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c00e070 con 0x7ff854164550 2023-12-19T18:18:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff849ffb640 1 -- 172.21.15.26:0/3943883758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c011ed0 con 0x7ff854164550 2023-12-19T18:18:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff849ffb640 1 -- 172.21.15.26:0/3943883758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c0039a0 con 0x7ff854164550 2023-12-19T18:18:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff85b4fc640 1 -- 172.21.15.26:0/3943883758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff854163bb0 con 0x7ff854164550 2023-12-19T18:18:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff85b4fc640 1 -- 172.21.15.26:0/3943883758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff854144b70 con 0x7ff854164550 2023-12-19T18:18:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.744+0000 7ff85b4fc640 1 -- 172.21.15.26:0/3943883758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8541f67e0 con 0x7ff854164550 2023-12-19T18:18:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.748+0000 7ff849ffb640 1 -- 172.21.15.26:0/3943883758 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ff84c003b00 con 0x7ff854164550 2023-12-19T18:18:26.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.748+0000 7ff849ffb640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff838065ca0 0x7ff838068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.748+0000 7ff849ffb640 1 -- 172.21.15.26:0/3943883758 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7ff84c083f50 con 0x7ff854164550 2023-12-19T18:18:26.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.748+0000 7ff859298640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff838065ca0 0x7ff838068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.748+0000 7ff859298640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff838065ca0 0x7ff838068160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff8500aa000 tx=0x7ff850001c10 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:26.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.752+0000 7ff849ffb640 1 -- 172.21.15.26:0/3943883758 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff84c051100 con 0x7ff854164550 2023-12-19T18:18:26.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.932+0000 7ff85b4fc640 1 -- 172.21.15.26:0/3943883758 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8540b3720 con 0x7ff838065ca0 2023-12-19T18:18:26.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.936+0000 7ff849ffb640 1 -- 172.21.15.26:0/3943883758 <== mgr.9680 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff8500015d0 con 0x7ff838065ca0 2023-12-19T18:18:26.937 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2023-12-19T18:18:26.938 INFO:teuthology.orchestra.run.smithi026.stdout: { 2023-12-19T18:18:26.938 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2023-12-19T18:18:26.938 INFO:teuthology.orchestra.run.smithi026.stdout: } 2023-12-19T18:18:26.938 INFO:teuthology.orchestra.run.smithi026.stdout:] 2023-12-19T18:18:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.936+0000 7ff85b4fc640 1 -- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff838065ca0 msgr2=0x7ff838068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.936+0000 7ff85b4fc640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff838065ca0 0x7ff838068160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff8500aa000 tx=0x7ff850001c10 comp rx=0 tx=0).stop 2023-12-19T18:18:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.936+0000 7ff85b4fc640 1 -- 172.21.15.26:0/3943883758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff854164550 msgr2=0x7ff8541f5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.936+0000 7ff85b4fc640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff854164550 0x7ff8541f5d50 secure :-1 s=READY pgs=2367 cs=0 l=1 rev1=1 crypto rx=0x7ff85415e3b0 tx=0x7ff84c0115a0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.940+0000 7ff85b4fc640 1 -- 172.21.15.26:0/3943883758 shutdown_connections 2023-12-19T18:18:26.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.940+0000 7ff85b4fc640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff838065ca0 0x7ff838068160 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.940+0000 7ff85b4fc640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff854164550 0x7ff8541f5d50 unknown :-1 s=CLOSED pgs=2367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.940+0000 7ff85b4fc640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff8541640d0 0x7ff8541f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.940+0000 7ff85b4fc640 1 --2- 172.21.15.26:0/3943883758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff840003980 0x7ff854142450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.940+0000 7ff85b4fc640 1 -- 172.21.15.26:0/3943883758 >> 172.21.15.26:0/3943883758 conn(0x7ff8540658e0 msgr2=0x7ff85406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:26.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.940+0000 7ff85b4fc640 1 -- 172.21.15.26:0/3943883758 shutdown_connections 2023-12-19T18:18:26.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:26.940+0000 7ff85b4fc640 1 -- 172.21.15.26:0/3943883758 wait complete. 2023-12-19T18:18:26.958 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-19T18:18:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f390e196640 1 Processor -- start 2023-12-19T18:18:27.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f390e196640 1 -- start start 2023-12-19T18:18:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f390e196640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39080cb860 0x7f390806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f390e196640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f390815d530 0x7f390815d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f390e196640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390815e240 con 0x7f39081599e0 2023-12-19T18:18:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f390e196640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39080c9520 con 0x7f390815d530 2023-12-19T18:18:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f390e196640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39080c9800 con 0x7f39080cb860 2023-12-19T18:18:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f3907fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f390815d530 0x7f390815d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f3907fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f390815d530 0x7f390815d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37868/0 (socket says 172.21.15.26:37868) 2023-12-19T18:18:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f3907fff640 1 -- 172.21.15.26:0/2692589896 learned_addr learned my addr 172.21.15.26:0/2692589896 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f3906ffd640 1 --2- 172.21.15.26:0/2692589896 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39080cb860 0x7f390806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f3907fff640 1 -- 172.21.15.26:0/2692589896 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39080cb860 msgr2=0x7f390806f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:27.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f3907fff640 1 --2- 172.21.15.26:0/2692589896 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39080cb860 0x7f390806f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:27.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f3907fff640 1 -- 172.21.15.26:0/2692589896 >> v1:172.21.15.160:6789/0 conn(0x7f39081599e0 legacy=0x7f39080cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:27.107 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.104+0000 7f3907fff640 1 -- 172.21.15.26:0/2692589896 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39080c9ae0 con 0x7f390815d530 2023-12-19T18:18:27.108 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.108+0000 7f3907fff640 1 --2- 172.21.15.26:0/2692589896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f390815d530 0x7f390815d980 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f3900009e50 tx=0x7f390001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fada1be2217472 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:27.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.108+0000 7f39067fc640 1 -- 172.21.15.26:0/2692589896 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39000a5900 con 0x7f390815d530 2023-12-19T18:18:27.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.108+0000 7f39067fc640 1 -- 172.21.15.26:0/2692589896 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39000a5a60 con 0x7f390815d530 2023-12-19T18:18:27.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.108+0000 7f39067fc640 1 -- 172.21.15.26:0/2692589896 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39000a5df0 con 0x7f390815d530 2023-12-19T18:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.108+0000 7f390e196640 1 -- 172.21.15.26:0/2692589896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f390815d530 msgr2=0x7f390815d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.108+0000 7f390e196640 1 --2- 172.21.15.26:0/2692589896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f390815d530 0x7f390815d980 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f3900009e50 tx=0x7f390001ca10 comp rx=0 tx=0).stop 2023-12-19T18:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.108+0000 7f390e196640 1 -- 172.21.15.26:0/2692589896 shutdown_connections 2023-12-19T18:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.108+0000 7f390e196640 1 --2- 172.21.15.26:0/2692589896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f390815d530 0x7f390815d980 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.108+0000 7f390e196640 1 --2- 172.21.15.26:0/2692589896 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f39080cb860 0x7f390806f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.108+0000 7f390e196640 1 -- 172.21.15.26:0/2692589896 >> 172.21.15.26:0/2692589896 conn(0x7f3908066160 msgr2=0x7f39080b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.108+0000 7f390e196640 1 -- 172.21.15.26:0/2692589896 shutdown_connections 2023-12-19T18:18:27.110 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.108+0000 7f390e196640 1 -- 172.21.15.26:0/2692589896 wait complete. 2023-12-19T18:18:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f390e196640 1 Processor -- start 2023-12-19T18:18:27.111 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f390e196640 1 -- start start 2023-12-19T18:18:27.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f390e196640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39080cb860 0x7f3908150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:27.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f390e196640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39081599e0 0x7f390814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:27.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f390e196640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f390815d530 0x7f3908153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:27.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f390e196640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390815af90 con 0x7f39080cb860 2023-12-19T18:18:27.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f390e196640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39081507b0 con 0x7f39081599e0 2023-12-19T18:18:27.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f390e196640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390815a030 con 0x7f390815d530 2023-12-19T18:18:27.115 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f39077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39080cb860 0x7f3908150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f39077fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39080cb860 0x7f3908150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37870/0 (socket says 172.21.15.26:37870) 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f39077fe640 1 -- 172.21.15.26:0/298906672 learned_addr learned my addr 172.21.15.26:0/298906672 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f3906ffd640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39081599e0 0x7f390814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f3907fff640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f390815d530 0x7f3908153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f39077fe640 1 -- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f390815d530 msgr2=0x7f3908153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f39077fe640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f390815d530 0x7f3908153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f39077fe640 1 -- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39081599e0 msgr2=0x7f390814fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f39077fe640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39081599e0 0x7f390814fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f39077fe640 1 -- 172.21.15.26:0/298906672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3900009b30 con 0x7f39080cb860 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f3906ffd640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39081599e0 0x7f390814fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f39077fe640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39080cb860 0x7f3908150270 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f38f800c530 tx=0x7f38f800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f38effff640 1 -- 172.21.15.26:0/298906672 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8005220 con 0x7f39080cb860 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f38effff640 1 -- 172.21.15.26:0/298906672 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f8005380 con 0x7f39080cb860 2023-12-19T18:18:27.116 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f390e196640 1 -- 172.21.15.26:0/298906672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390815a310 con 0x7f39080cb860 2023-12-19T18:18:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f38effff640 1 -- 172.21.15.26:0/298906672 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8007270 con 0x7f39080cb860 2023-12-19T18:18:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.112+0000 7f390e196640 1 -- 172.21.15.26:0/298906672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390815a8d0 con 0x7f39080cb860 2023-12-19T18:18:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.116+0000 7f390e196640 1 -- 172.21.15.26:0/298906672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3908074270 con 0x7f39080cb860 2023-12-19T18:18:27.117 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.116+0000 7f38effff640 1 -- 172.21.15.26:0/298906672 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 136) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f38f80073d0 con 0x7f39080cb860 2023-12-19T18:18:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.116+0000 7f38effff640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f38f4065cd0 0x7f38f4068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.116+0000 7f38effff640 1 -- 172.21.15.26:0/298906672 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f38f8083000 con 0x7f39080cb860 2023-12-19T18:18:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.116+0000 7f3906ffd640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f38f4065cd0 0x7f38f4068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:27.118 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.116+0000 7f3906ffd640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f38f4065cd0 0x7f38f4068190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3908154f50 tx=0x7f38fc008040 comp rx=0 tx=0).ready entity=mgr.9680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:27.121 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.120+0000 7f38effff640 1 -- 172.21.15.26:0/298906672 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f38f80503f0 con 0x7f39080cb860 2023-12-19T18:18:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:27.320+0000 7f390e196640 1 -- 172.21.15.26:0/298906672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f39080c9180 con 0x7f39080cb860 2023-12-19T18:18:28.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.136+0000 7f38effff640 1 -- 172.21.15.26:0/298906672 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 137) v1 ==== 85449+0+0 (secure 0 0 0) 0x7f38f80507c0 con 0x7f39080cb860 2023-12-19T18:18:28.138 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.136+0000 7f38effff640 1 -- 172.21.15.26:0/298906672 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v137) v1 ==== 92+0+0 (secure 0 0 0) 0x7f38f8080860 con 0x7f39080cb860 2023-12-19T18:18:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.140+0000 7f390e196640 1 -- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f38f4065cd0 msgr2=0x7f38f4068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.140+0000 7f390e196640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f38f4065cd0 0x7f38f4068190 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3908154f50 tx=0x7f38fc008040 comp rx=0 tx=0).stop 2023-12-19T18:18:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.140+0000 7f390e196640 1 -- 172.21.15.26:0/298906672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39080cb860 msgr2=0x7f3908150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.140+0000 7f390e196640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39080cb860 0x7f3908150270 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f38f800c530 tx=0x7f38f800c9f0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.140+0000 7f390e196640 1 -- 172.21.15.26:0/298906672 shutdown_connections 2023-12-19T18:18:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.140+0000 7f390e196640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f38f4065cd0 0x7f38f4068190 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.140+0000 7f390e196640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f390815d530 0x7f3908153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.140+0000 7f390e196640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f39081599e0 0x7f390814fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.140+0000 7f390e196640 1 --2- 172.21.15.26:0/298906672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f39080cb860 0x7f3908150270 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.140+0000 7f390e196640 1 -- 172.21.15.26:0/298906672 >> 172.21.15.26:0/298906672 conn(0x7f3908066160 msgr2=0x7f39080cb130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.140+0000 7f390e196640 1 -- 172.21.15.26:0/298906672 shutdown_connections 2023-12-19T18:18:28.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.140+0000 7f390e196640 1 -- 172.21.15.26:0/298906672 wait complete. 2023-12-19T18:18:28.165 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-19T18:18:28.317 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:28.316+0000 7f35ba999180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:18:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.316+0000 7f9568e2d640 1 Processor -- start 2023-12-19T18:18:28.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9568e2d640 1 -- start start 2023-12-19T18:18:28.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9568e2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f956415cde0 0x7f95640b8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9568e2d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f956415d1c0 0x7f95640b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9568e2d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f956415d620 0x7f95640b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9568e2d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956415e380 con 0x7f956415d620 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9568e2d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95640b8b40 con 0x7f956415d1c0 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9568e2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95640b5bd0 con 0x7f956415cde0 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9562d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f956415d620 0x7f95640b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9562d9d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f956415d620 0x7f95640b4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37890/0 (socket says 172.21.15.26:37890) 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9561d9b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f956415d1c0 0x7f95640b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9562d9d640 1 -- 172.21.15.26:0/577739171 learned_addr learned my addr 172.21.15.26:0/577739171 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f956259c640 1 --2- 172.21.15.26:0/577739171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f956415cde0 0x7f95640b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9562d9d640 1 -- 172.21.15.26:0/577739171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f956415cde0 msgr2=0x7f95640b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9562d9d640 1 --2- 172.21.15.26:0/577739171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f956415cde0 0x7f95640b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9562d9d640 1 -- 172.21.15.26:0/577739171 >> v2:172.21.15.160:3300/0 conn(0x7f956415d1c0 msgr2=0x7f95640b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9562d9d640 1 --2- 172.21.15.26:0/577739171 >> v2:172.21.15.160:3300/0 conn(0x7f956415d1c0 0x7f95640b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9562d9d640 1 -- 172.21.15.26:0/577739171 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95640d0ee0 con 0x7f956415d620 2023-12-19T18:18:28.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f9561d9b640 1 --2- 172.21.15.26:0/577739171 >> v2:172.21.15.160:3300/0 conn(0x7f956415d1c0 0x7f95640b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:28.323 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:28.320+0000 7f3618857180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:18:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.324+0000 7f9562d9d640 1 --2- 172.21.15.26:0/577739171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956415d620 0x7f95640b4b30 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f956406adf0 tx=0x7f955c01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6b5034c112892 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.324+0000 7f956159a640 1 -- 172.21.15.26:0/577739171 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c0af400 con 0x7f956415d620 2023-12-19T18:18:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.324+0000 7f956159a640 1 -- 172.21.15.26:0/577739171 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c0ab050 con 0x7f956415d620 2023-12-19T18:18:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.324+0000 7f956159a640 1 -- 172.21.15.26:0/577739171 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c0af780 con 0x7f956415d620 2023-12-19T18:18:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 -- 172.21.15.26:0/577739171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956415d620 msgr2=0x7f95640b4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 --2- 172.21.15.26:0/577739171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956415d620 0x7f95640b4b30 secure :-1 s=READY pgs=2373 cs=0 l=1 rev1=1 crypto rx=0x7f956406adf0 tx=0x7f955c01f1c0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 -- 172.21.15.26:0/577739171 shutdown_connections 2023-12-19T18:18:28.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 --2- 172.21.15.26:0/577739171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956415d620 0x7f95640b4b30 unknown :-1 s=CLOSED pgs=2373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 --2- 172.21.15.26:0/577739171 >> v2:172.21.15.160:3300/0 conn(0x7f956415d1c0 0x7f95640b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 --2- 172.21.15.26:0/577739171 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f956415cde0 0x7f95640b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 -- 172.21.15.26:0/577739171 >> 172.21.15.26:0/577739171 conn(0x7f9564065ee0 msgr2=0x7f95640d2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:28.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 -- 172.21.15.26:0/577739171 shutdown_connections 2023-12-19T18:18:28.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 -- 172.21.15.26:0/577739171 wait complete. 2023-12-19T18:18:28.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 Processor -- start 2023-12-19T18:18:28.329 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 -- start start 2023-12-19T18:18:28.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956415cde0 0x7f95640b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f956415d1c0 0x7f95640c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f956415d620 0x7f95640c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95640b1290 con 0x7f956415cde0 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95640b80b0 con 0x7f956415d1c0 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9568e2d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95640ca420 con 0x7f956415d620 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f956259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956415cde0 0x7f95640b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f956259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956415cde0 0x7f95640b0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37892/0 (socket says 172.21.15.26:37892) 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f956259c640 1 -- 172.21.15.26:0/1031146750 learned_addr learned my addr 172.21.15.26:0/1031146750 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9561d9b640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f956415d1c0 0x7f95640c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.328+0000 7f9562d9d640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f956415d620 0x7f95640c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f956259c640 1 -- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f956415d620 msgr2=0x7f95640c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f956259c640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f956415d620 0x7f95640c6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f956259c640 1 -- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f956415d1c0 msgr2=0x7f95640c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f956259c640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f956415d1c0 0x7f95640c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f956259c640 1 -- 172.21.15.26:0/1031146750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9558009420 con 0x7f956415cde0 2023-12-19T18:18:28.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f9562d9d640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f956415d620 0x7f95640c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:28.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f956259c640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956415cde0 0x7f95640b0d50 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f955400e930 tx=0x7f955400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:28.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f9552ffd640 1 -- 172.21.15.26:0/1031146750 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554009690 con 0x7f956415cde0 2023-12-19T18:18:28.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f9552ffd640 1 -- 172.21.15.26:0/1031146750 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955400ca00 con 0x7f956415cde0 2023-12-19T18:18:28.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f9552ffd640 1 -- 172.21.15.26:0/1031146750 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955400cd70 con 0x7f956415cde0 2023-12-19T18:18:28.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f9568e2d640 1 -- 172.21.15.26:0/1031146750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955c009b30 con 0x7f956415cde0 2023-12-19T18:18:28.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f9568e2d640 1 -- 172.21.15.26:0/1031146750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95640ca9e0 con 0x7f956415cde0 2023-12-19T18:18:28.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f9568e2d640 1 -- 172.21.15.26:0/1031146750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9564074270 con 0x7f956415cde0 2023-12-19T18:18:28.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f9552ffd640 1 -- 172.21.15.26:0/1031146750 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 137) v1 ==== 85449+0+0 (secure 0 0 0) 0x7f9554013470 con 0x7f956415cde0 2023-12-19T18:18:28.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.332+0000 7f9552ffd640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9544065cf0 0x7f95440681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.336+0000 7f9552ffd640 1 -- 172.21.15.26:0/1031146750 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7f9554083b90 con 0x7f956415cde0 2023-12-19T18:18:28.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.336+0000 7f9561d9b640 1 -- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9544065cf0 msgr2=0x7f95440681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:28.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.336+0000 7f9561d9b640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9544065cf0 0x7f95440681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:18:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.336+0000 7f9552ffd640 1 -- 172.21.15.26:0/1031146750 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9554050fc0 con 0x7f956415cde0 2023-12-19T18:18:28.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.500+0000 7f9568e2d640 1 -- 172.21.15.26:0/1031146750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f9564070970 con 0x7f956415cde0 2023-12-19T18:18:28.506 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.504+0000 7f9552ffd640 1 -- 172.21.15.26:0/1031146750 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v83)=0 v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f95540170d0 con 0x7f956415cde0 2023-12-19T18:18:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.508+0000 7f9568e2d640 1 -- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9544065cf0 msgr2=0x7f95440681b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:18:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.508+0000 7f9568e2d640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9544065cf0 0x7f95440681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.508+0000 7f9568e2d640 1 -- 172.21.15.26:0/1031146750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956415cde0 msgr2=0x7f95640b0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.508+0000 7f9568e2d640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956415cde0 0x7f95640b0d50 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f955400e930 tx=0x7f955400edf0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.508+0000 7f9568e2d640 1 -- 172.21.15.26:0/1031146750 shutdown_connections 2023-12-19T18:18:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.508+0000 7f9568e2d640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9544065cf0 0x7f95440681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.508+0000 7f9568e2d640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f956415d620 0x7f95640c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.508+0000 7f9568e2d640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f956415d1c0 0x7f95640c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.508+0000 7f9568e2d640 1 --2- 172.21.15.26:0/1031146750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f956415cde0 0x7f95640b0d50 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.508+0000 7f9568e2d640 1 -- 172.21.15.26:0/1031146750 >> 172.21.15.26:0/1031146750 conn(0x7f9564065ee0 msgr2=0x7f9564069db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.508+0000 7f9568e2d640 1 -- 172.21.15.26:0/1031146750 shutdown_connections 2023-12-19T18:18:28.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.508+0000 7f9568e2d640 1 -- 172.21.15.26:0/1031146750 wait complete. 2023-12-19T18:18:28.528 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-19T18:18:28.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.676+0000 7fb4df612640 1 Processor -- start 2023-12-19T18:18:28.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.676+0000 7fb4df612640 1 -- start start 2023-12-19T18:18:28.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.676+0000 7fb4df612640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb4d815ccf0 0x7fb4d80d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.676+0000 7fb4df612640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d815e200 con 0x7fb4d815d0d0 2023-12-19T18:18:28.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.676+0000 7fb4df612640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d80d0aa0 con 0x7fb4d80b4600 2023-12-19T18:18:28.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.676+0000 7fb4df612640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d80d1380 con 0x7fb4d815ccf0 2023-12-19T18:18:28.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.676+0000 7fb4dcbad640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb4d815d0d0 0x7fb4d80b8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52746/0 (socket says 172.21.15.26:52746) 2023-12-19T18:18:28.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.676+0000 7fb4dcbad640 1 -- 172.21.15.26:0/1293325688 learned_addr learned my addr 172.21.15.26:0/1293325688 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:28.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.676+0000 7fb4dd3ae640 1 --2- 172.21.15.26:0/1293325688 >> v2:172.21.15.26:3300/0 conn(0x7fb4d815ccf0 0x7fb4d80d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.676+0000 7fb4dd3ae640 1 -- 172.21.15.26:0/1293325688 >> v1:172.21.15.160:6789/0 conn(0x7fb4d80b4600 legacy=0x7fb4d80b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.676+0000 7fb4dd3ae640 1 -- 172.21.15.26:0/1293325688 >> v1:172.21.15.26:6789/0 conn(0x7fb4d815d0d0 legacy=0x7fb4d80b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.676+0000 7fb4cffff640 1 -- 172.21.15.26:0/1293325688 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480431445 0 0) 0x7fb4d815e200 con 0x7fb4d815d0d0 2023-12-19T18:18:28.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.676+0000 7fb4dd3ae640 1 -- 172.21.15.26:0/1293325688 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d815d680 con 0x7fb4d815ccf0 2023-12-19T18:18:28.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.680+0000 7fb4dd3ae640 1 --2- 172.21.15.26:0/1293325688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d815ccf0 0x7fb4d80d0e40 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40097c0 tx=0x7fb4d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbfdbc25cec69a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:28.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.680+0000 7fb4cffff640 1 -- 172.21.15.26:0/1293325688 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d40ab650 con 0x7fb4d815ccf0 2023-12-19T18:18:28.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.680+0000 7fb4cffff640 1 -- 172.21.15.26:0/1293325688 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d40ab7b0 con 0x7fb4d815ccf0 2023-12-19T18:18:28.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.680+0000 7fb4cffff640 1 -- 172.21.15.26:0/1293325688 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d40abb40 con 0x7fb4d815ccf0 2023-12-19T18:18:28.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.680+0000 7fb4df612640 1 -- 172.21.15.26:0/1293325688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d815ccf0 msgr2=0x7fb4d80d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.680+0000 7fb4df612640 1 --2- 172.21.15.26:0/1293325688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d815ccf0 0x7fb4d80d0e40 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40097c0 tx=0x7fb4d401c450 comp rx=0 tx=0).stop 2023-12-19T18:18:28.682 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.680+0000 7fb4df612640 1 -- 172.21.15.26:0/1293325688 shutdown_connections 2023-12-19T18:18:28.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.680+0000 7fb4df612640 1 --2- 172.21.15.26:0/1293325688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d815ccf0 0x7fb4d80d0e40 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.680+0000 7fb4df612640 1 -- 172.21.15.26:0/1293325688 >> 172.21.15.26:0/1293325688 conn(0x7fb4d8065ee0 msgr2=0x7fb4d80cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:28.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4df612640 1 -- 172.21.15.26:0/1293325688 shutdown_connections 2023-12-19T18:18:28.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4df612640 1 -- 172.21.15.26:0/1293325688 wait complete. 2023-12-19T18:18:28.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4df612640 1 Processor -- start 2023-12-19T18:18:28.684 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4df612640 1 -- start start 2023-12-19T18:18:28.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4df612640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4d80b4600 0x7fb4d815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4df612640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d815ccf0 0x7fb4d8154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4df612640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d815d0d0 0x7fb4d8154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4df612640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d8152f40 con 0x7fb4d815d0d0 2023-12-19T18:18:28.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4df612640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d815af50 con 0x7fb4d815ccf0 2023-12-19T18:18:28.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4df612640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d815b210 con 0x7fb4d80b4600 2023-12-19T18:18:28.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4ddbaf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d815d0d0 0x7fb4d8154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4dcbad640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d815ccf0 0x7fb4d8154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4dd3ae640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4d80b4600 0x7fb4d815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4ddbaf640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d815d0d0 0x7fb4d8154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37910/0 (socket says 172.21.15.26:37910) 2023-12-19T18:18:28.687 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4ddbaf640 1 -- 172.21.15.26:0/724101120 learned_addr learned my addr 172.21.15.26:0/724101120 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:28.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4ddbaf640 1 -- 172.21.15.26:0/724101120 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4d80b4600 msgr2=0x7fb4d815c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4ddbaf640 1 --2- 172.21.15.26:0/724101120 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4d80b4600 0x7fb4d815c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4ddbaf640 1 -- 172.21.15.26:0/724101120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d815ccf0 msgr2=0x7fb4d8154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4ddbaf640 1 --2- 172.21.15.26:0/724101120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d815ccf0 0x7fb4d8154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4ddbaf640 1 -- 172.21.15.26:0/724101120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d4009420 con 0x7fb4d815d0d0 2023-12-19T18:18:28.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.684+0000 7fb4ddbaf640 1 --2- 172.21.15.26:0/724101120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d815d0d0 0x7fb4d8154ff0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000c730 tx=0x7fb4d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:28.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.688+0000 7fb4cdffb640 1 -- 172.21.15.26:0/724101120 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0003b80 con 0x7fb4d815d0d0 2023-12-19T18:18:28.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.688+0000 7fb4cdffb640 1 -- 172.21.15.26:0/724101120 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d0003ce0 con 0x7fb4d815d0d0 2023-12-19T18:18:28.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.688+0000 7fb4cdffb640 1 -- 172.21.15.26:0/724101120 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d000aa30 con 0x7fb4d815d0d0 2023-12-19T18:18:28.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.688+0000 7fb4df612640 1 -- 172.21.15.26:0/724101120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d815b4f0 con 0x7fb4d815d0d0 2023-12-19T18:18:28.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.688+0000 7fb4df612640 1 -- 172.21.15.26:0/724101120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d815bab0 con 0x7fb4d815d0d0 2023-12-19T18:18:28.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.688+0000 7fb4df612640 1 -- 172.21.15.26:0/724101120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d8074270 con 0x7fb4d815d0d0 2023-12-19T18:18:28.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.688+0000 7fb4cdffb640 1 -- 172.21.15.26:0/724101120 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 137) v1 ==== 85449+0+0 (secure 0 0 0) 0x7fb4d000ab90 con 0x7fb4d815d0d0 2023-12-19T18:18:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.688+0000 7fb4cdffb640 1 --2- 172.21.15.26:0/724101120 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb4bc065cf0 0x7fb4bc0681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.688+0000 7fb4cdffb640 1 -- 172.21.15.26:0/724101120 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fb4d0083630 con 0x7fb4d815d0d0 2023-12-19T18:18:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.688+0000 7fb4dd3ae640 1 -- 172.21.15.26:0/724101120 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb4bc065cf0 msgr2=0x7fb4bc0681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.688+0000 7fb4dd3ae640 1 --2- 172.21.15.26:0/724101120 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb4bc065cf0 0x7fb4bc0681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:18:28.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.696+0000 7fb4cdffb640 1 -- 172.21.15.26:0/724101120 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb4d00509a0 con 0x7fb4d815d0d0 2023-12-19T18:18:28.734 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:28.732+0000 7f35ba999180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:18:28.777 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:28.776+0000 7f3618857180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:18:28.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.860+0000 7fb4df612640 1 -- 172.21.15.26:0/724101120 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fb4d8153690 con 0x7fb4d815d0d0 2023-12-19T18:18:28.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.864+0000 7fb4cdffb640 1 -- 172.21.15.26:0/724101120 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v84)=0 v84) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb4d0053fc0 con 0x7fb4d815d0d0 2023-12-19T18:18:28.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.864+0000 7fb4df612640 1 -- 172.21.15.26:0/724101120 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb4bc065cf0 msgr2=0x7fb4bc0681b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:18:28.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.864+0000 7fb4df612640 1 --2- 172.21.15.26:0/724101120 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb4bc065cf0 0x7fb4bc0681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.864+0000 7fb4df612640 1 -- 172.21.15.26:0/724101120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d815d0d0 msgr2=0x7fb4d8154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.864+0000 7fb4df612640 1 --2- 172.21.15.26:0/724101120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d815d0d0 0x7fb4d8154ff0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000c730 tx=0x7fb4d000cbf0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.864+0000 7fb4df612640 1 -- 172.21.15.26:0/724101120 shutdown_connections 2023-12-19T18:18:28.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.864+0000 7fb4df612640 1 --2- 172.21.15.26:0/724101120 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb4bc065cf0 0x7fb4bc0681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.864+0000 7fb4df612640 1 --2- 172.21.15.26:0/724101120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4d815d0d0 0x7fb4d8154ff0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.864+0000 7fb4df612640 1 --2- 172.21.15.26:0/724101120 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4d815ccf0 0x7fb4d8154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.864+0000 7fb4df612640 1 --2- 172.21.15.26:0/724101120 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb4d80b4600 0x7fb4d815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.864+0000 7fb4df612640 1 -- 172.21.15.26:0/724101120 >> 172.21.15.26:0/724101120 conn(0x7fb4d8065ee0 msgr2=0x7fb4d80b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:28.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.868+0000 7fb4df612640 1 -- 172.21.15.26:0/724101120 shutdown_connections 2023-12-19T18:18:28.868 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:28.868+0000 7fb4df612640 1 -- 172.21.15.26:0/724101120 wait complete. 2023-12-19T18:18:28.884 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:28.884+0000 7f35ba999180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:18:28.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:28.887 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-19T18:18:28.903 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1M 2023-12-19T18:18:28.939 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:28.936+0000 7f3618857180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:18:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.084+0000 7fb63ea97640 1 Processor -- start 2023-12-19T18:18:29.086 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.084+0000 7fb63ea97640 1 -- start start 2023-12-19T18:18:29.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb63ea97640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb63806b110 0x7fb638070030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:29.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb63ea97640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63806f490 con 0x7fb638159140 2023-12-19T18:18:29.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb63ea97640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb638159520 con 0x7fb63806ac80 2023-12-19T18:18:29.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb63ea97640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6380cd690 con 0x7fb63806b110 2023-12-19T18:18:29.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb63c833640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb638159140 0x7fb6380ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52756/0 (socket says 172.21.15.26:52756) 2023-12-19T18:18:29.087 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb63c833640 1 -- 172.21.15.26:0/1497193872 learned_addr learned my addr 172.21.15.26:0/1497193872 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb63d034640 1 --2- 172.21.15.26:0/1497193872 >> v2:172.21.15.160:3300/0 conn(0x7fb63806b110 0x7fb638070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb6377fe640 1 -- 172.21.15.26:0/1497193872 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140391256 0 0) 0x7fb63806f490 con 0x7fb638159140 2023-12-19T18:18:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb6377fe640 1 -- 172.21.15.26:0/1497193872 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb628004640 con 0x7fb638159140 2023-12-19T18:18:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb63d034640 1 -- 172.21.15.26:0/1497193872 >> v1:172.21.15.160:6789/0 conn(0x7fb63806ac80 legacy=0x7fb63806f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb6377fe640 1 -- 172.21.15.26:0/1497193872 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679099657 0 0) 0x7fb638159520 con 0x7fb63806ac80 2023-12-19T18:18:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb63d034640 1 -- 172.21.15.26:0/1497193872 >> v1:172.21.15.26:6789/0 conn(0x7fb638159140 legacy=0x7fb6380ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb63d034640 1 -- 172.21.15.26:0/1497193872 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6380cd970 con 0x7fb63806b110 2023-12-19T18:18:29.090 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.088+0000 7fb63d034640 1 --2- 172.21.15.26:0/1497193872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63806b110 0x7fb638070030 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb630009d90 tx=0x7fb63001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2de9500ca9683b81 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:29.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb6377fe640 1 -- 172.21.15.26:0/1497193872 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6300a59e0 con 0x7fb63806b110 2023-12-19T18:18:29.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb6377fe640 1 -- 172.21.15.26:0/1497193872 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6300a5b40 con 0x7fb63806b110 2023-12-19T18:18:29.091 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb6377fe640 1 -- 172.21.15.26:0/1497193872 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6300ad440 con 0x7fb63806b110 2023-12-19T18:18:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 -- 172.21.15.26:0/1497193872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63806b110 msgr2=0x7fb638070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 --2- 172.21.15.26:0/1497193872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63806b110 0x7fb638070030 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb630009d90 tx=0x7fb63001c920 comp rx=0 tx=0).stop 2023-12-19T18:18:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 -- 172.21.15.26:0/1497193872 shutdown_connections 2023-12-19T18:18:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 --2- 172.21.15.26:0/1497193872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb63806b110 0x7fb638070030 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 -- 172.21.15.26:0/1497193872 >> 172.21.15.26:0/1497193872 conn(0x7fb6380ce730 msgr2=0x7fb6380ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 -- 172.21.15.26:0/1497193872 shutdown_connections 2023-12-19T18:18:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 -- 172.21.15.26:0/1497193872 wait complete. 2023-12-19T18:18:29.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 Processor -- start 2023-12-19T18:18:29.093 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 -- start start 2023-12-19T18:18:29.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb63806ac80 0x7fb6380c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:29.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63806b110 0x7fb6380c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:29.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb638159140 0x7fb63814a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:29.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6380c3680 con 0x7fb63806b110 2023-12-19T18:18:29.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6380c2a00 con 0x7fb638159140 2023-12-19T18:18:29.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.092+0000 7fb63ea97640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6380c2ce0 con 0x7fb63806ac80 2023-12-19T18:18:29.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb637fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63806b110 0x7fb6380c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:29.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb63c833640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb63806ac80 0x7fb6380c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:29.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb637fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63806b110 0x7fb6380c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37912/0 (socket says 172.21.15.26:37912) 2023-12-19T18:18:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb63c833640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb63806ac80 0x7fb6380c21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:48398/0 (socket says 172.21.15.26:48398) 2023-12-19T18:18:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb637fff640 1 -- 172.21.15.26:0/982618053 learned_addr learned my addr 172.21.15.26:0/982618053 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb63d034640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb638159140 0x7fb63814a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb637fff640 1 -- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb63806ac80 msgr2=0x7fb6380c21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb637fff640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb63806ac80 0x7fb6380c21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb637fff640 1 -- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb638159140 msgr2=0x7fb63814a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb637fff640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb638159140 0x7fb63814a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb637fff640 1 -- 172.21.15.26:0/982618053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb630009a70 con 0x7fb63806b110 2023-12-19T18:18:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb637fff640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63806b110 0x7fb6380c94d0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c410 tx=0x7fb62400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb6357fa640 1 -- 172.21.15.26:0/982618053 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6240054e0 con 0x7fb63806b110 2023-12-19T18:18:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb6357fa640 1 -- 172.21.15.26:0/982618053 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb624003600 con 0x7fb63806b110 2023-12-19T18:18:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb6357fa640 1 -- 172.21.15.26:0/982618053 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6240039b0 con 0x7fb63806b110 2023-12-19T18:18:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb63ea97640 1 -- 172.21.15.26:0/982618053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6380c2fc0 con 0x7fb63806b110 2023-12-19T18:18:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb63ea97640 1 -- 172.21.15.26:0/982618053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6380c2710 con 0x7fb63806b110 2023-12-19T18:18:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb63ea97640 1 -- 172.21.15.26:0/982618053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6380c37c0 con 0x7fb63806b110 2023-12-19T18:18:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.096+0000 7fb6357fa640 1 -- 172.21.15.26:0/982618053 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 137) v1 ==== 85449+0+0 (secure 0 0 0) 0x7fb624003b10 con 0x7fb63806b110 2023-12-19T18:18:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.100+0000 7fb6357fa640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 0x7fb61c0681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.100+0000 7fb6357fa640 1 -- 172.21.15.26:0/982618053 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fb624082f10 con 0x7fb63806b110 2023-12-19T18:18:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.100+0000 7fb63c833640 1 -- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 msgr2=0x7fb61c0681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.100+0000 7fb63c833640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 0x7fb61c0681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:18:29.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.104+0000 7fb6357fa640 1 -- 172.21.15.26:0/982618053 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb6380c37c0 con 0x7fb63806b110 2023-12-19T18:18:29.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.284+0000 7fb63ea97640 1 -- 172.21.15.26:0/982618053 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6380cd360 con 0x7fb61c065cf0 2023-12-19T18:18:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.304+0000 7fb63c833640 1 -- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 msgr2=0x7fb61c0681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.304+0000 7fb63c833640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 0x7fb61c0681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T18:18:29.365 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:29.364+0000 7f35ba999180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:18:29.423 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:29.420+0000 7f35ba999180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:18:29.455 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:29.452+0000 7f3618857180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:18:29.478 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:29.476+0000 7f35ba999180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:18:29.516 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:29.516+0000 7f3618857180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:18:29.563 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:29.560+0000 7f35ba999180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:18:29.574 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:29.572+0000 7f3618857180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:18:29.620 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:29.620+0000 7f35ba999180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:18:29.665 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:29.664+0000 7f3618857180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:18:29.704 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:29.704+0000 7f35ba999180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:18:29.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.708+0000 7fb63c833640 1 -- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 msgr2=0x7fb61c0681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:29.707 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:29.708+0000 7fb63c833640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 0x7fb61c0681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-19T18:18:29.724 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:29.724+0000 7f3618857180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:18:29.760 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:29.760+0000 7f35ba999180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:18:29.812 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:29.812+0000 7f3618857180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:18:29.872 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:29.872+0000 7f3618857180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:18:29.903 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:29.900+0000 7f35ba999180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:18:29.961 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:29.960+0000 7f35ba999180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:18:30.025 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:30.024+0000 7f3618857180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:18:30.086 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:30.084+0000 7f3618857180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:18:30.200 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:18:30.201 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:18:30.201 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:18:30.209 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:30.208+0000 7f35ba999180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:18:30.337 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:18:30.337 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:18:30.337 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:18:30.343 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:30.340+0000 7f35ba999180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:18:30.345 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:30.344+0000 7f3618857180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:18:30.467 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:30.464+0000 7f35ba999180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:18:30.487 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:30.488+0000 7f3618857180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:18:30.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:30.512+0000 7fb63c833640 1 -- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 msgr2=0x7fb61c0681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:30.511 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:30.512+0000 7fb63c833640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 0x7fb61c0681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-19T18:18:30.537 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:30.536+0000 7f35ba999180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:18:30.619 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:30.620+0000 7f3618857180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:18:30.672 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:30.671+0000 7f35ba999180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:18:30.694 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:30.692+0000 7f3618857180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:18:30.837 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:30.836+0000 7f3618857180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:18:31.206 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:31.203+0000 7f35ba999180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:18:31.290 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:31.287+0000 7f35ba999180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:18:31.407 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:31.404+0000 7f3618857180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:18:31.437 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:31.435+0000 7f35ba999180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:18:31.495 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:31.496+0000 7f3618857180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:18:31.520 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:31.519+0000 7f35ba999180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:18:31.652 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:31.652+0000 7f3618857180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:18:31.740 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:31.740+0000 7f3618857180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:18:31.835 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:31.831+0000 7f35ba999180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:18:31.895 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:31.891+0000 7f35ba999180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:18:31.906 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:31.904+0000 7fb6357fa640 1 -- 172.21.15.26:0/982618053 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 138) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fb624050650 con 0x7fb63806b110 2023-12-19T18:18:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:31.904+0000 7fb6357fa640 1 -- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 msgr2=0x7fb61c0681b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:18:31.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:31.904+0000 7fb6357fa640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 0x7fb61c0681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.020 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:32.019+0000 7f353d90d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:32.020 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:32.019+0000 7f353d90d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:32.020 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:32.019+0000 7f353d90d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:32.020 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:32.019+0000 7f353d90d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:32.020 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:32.019+0000 7f353d90d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:32.021 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:32.019+0000 7f3540913640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:32.021 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:32.019+0000 7f3540913640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:32.021 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:32.019+0000 7f3540913640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:32.021 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:32.019+0000 7f3540913640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:32.021 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:32.019+0000 7f3540913640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:32.085 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:32.084+0000 7f3618857180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:18:32.148 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:32.148+0000 7f3618857180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:18:32.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.912+0000 7fb6357fa640 1 -- 172.21.15.26:0/982618053 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 139) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fb624080870 con 0x7fb63806b110 2023-12-19T18:18:32.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.912+0000 7fb6357fa640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 0x7fb61c068130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:32.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.912+0000 7fb6357fa640 1 -- 172.21.15.26:0/982618053 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb61c06bcf0 con 0x7fb61c065cf0 2023-12-19T18:18:32.913 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.912+0000 7fb63c833640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 0x7fb61c068130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:32.915 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.916+0000 7fb63c833640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 0x7fb61c068130 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb62c007fd0 tx=0x7fb62c000bd0 comp rx=0 tx=0).ready entity=mgr.11587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:32.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.972+0000 7fb6357fa640 1 -- 172.21.15.26:0/982618053 <== mgr.11587 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb61c06bcf0 con 0x7fb61c065cf0 2023-12-19T18:18:32.974 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule set for path snap_test_dir1 2023-12-19T18:18:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.976+0000 7fb63ea97640 1 -- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 msgr2=0x7fb61c068130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.976+0000 7fb63ea97640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 0x7fb61c068130 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fb62c007fd0 tx=0x7fb62c000bd0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.976+0000 7fb63ea97640 1 -- 172.21.15.26:0/982618053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63806b110 msgr2=0x7fb6380c94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.976+0000 7fb63ea97640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63806b110 0x7fb6380c94d0 secure :-1 s=READY pgs=2379 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c410 tx=0x7fb62400c8d0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.976+0000 7fb63ea97640 1 -- 172.21.15.26:0/982618053 shutdown_connections 2023-12-19T18:18:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.976+0000 7fb63ea97640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb61c065cf0 0x7fb61c068130 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.976+0000 7fb63ea97640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb638159140 0x7fb63814a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.976+0000 7fb63ea97640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb63806b110 0x7fb6380c94d0 unknown :-1 s=CLOSED pgs=2379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.976+0000 7fb63ea97640 1 --2- 172.21.15.26:0/982618053 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb63806ac80 0x7fb6380c21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.976+0000 7fb63ea97640 1 -- 172.21.15.26:0/982618053 >> 172.21.15.26:0/982618053 conn(0x7fb6380ce730 msgr2=0x7fb63806f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:32.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.976+0000 7fb63ea97640 1 -- 172.21.15.26:0/982618053 shutdown_connections 2023-12-19T18:18:32.977 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:32.976+0000 7fb63ea97640 1 -- 172.21.15.26:0/982618053 wait complete. 2023-12-19T18:18:32.997 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-19T18:18:32.997 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1M 2023-12-19T18:18:33.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.144+0000 7f98d18fc640 1 Processor -- start 2023-12-19T18:18:33.146 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.144+0000 7f98d18fc640 1 -- start start 2023-12-19T18:18:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.144+0000 7f98d18fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f98cc1597d0 0x7f98cc070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.144+0000 7f98d18fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98cc06ab40 0x7f98cc0cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.144+0000 7f98d18fc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc06c4c0 con 0x7f98cc06af80 2023-12-19T18:18:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.144+0000 7f98d18fc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98cc0703a0 con 0x7f98cc1597d0 2023-12-19T18:18:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.144+0000 7f98d18fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc159530 con 0x7f98cc06ab40 2023-12-19T18:18:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.148+0000 7f98caffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f98cc1597d0 0x7f98cc070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.148+0000 7f98caffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f98cc1597d0 0x7f98cc070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37914/0 (socket says 172.21.15.26:37914) 2023-12-19T18:18:33.147 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.148+0000 7f98caffd640 1 -- 172.21.15.26:0/4140823750 learned_addr learned my addr 172.21.15.26:0/4140823750 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:33.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.148+0000 7f98ca7fc640 1 --2- 172.21.15.26:0/4140823750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98cc06ab40 0x7f98cc0cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.148+0000 7f98caffd640 1 -- 172.21.15.26:0/4140823750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98cc06ab40 msgr2=0x7f98cc0cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.148+0000 7f98caffd640 1 --2- 172.21.15.26:0/4140823750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98cc06ab40 0x7f98cc0cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.148+0000 7f98caffd640 1 -- 172.21.15.26:0/4140823750 >> v1:172.21.15.160:6789/0 conn(0x7f98cc06af80 legacy=0x7f98cc0c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.148+0000 7f98caffd640 1 -- 172.21.15.26:0/4140823750 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc06b420 con 0x7f98cc1597d0 2023-12-19T18:18:33.148 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.148+0000 7f98c9ffb640 1 -- 172.21.15.26:0/4140823750 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504404342 0 0) 0x7f98cc06c4c0 con 0x7f98cc06af80 2023-12-19T18:18:33.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.148+0000 7f98caffd640 1 --2- 172.21.15.26:0/4140823750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98cc1597d0 0x7f98cc070ff0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00bb40 tx=0x7f98bc01eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3ea4ba5667d195 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98c9ffb640 1 -- 172.21.15.26:0/4140823750 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0a7b00 con 0x7f98cc1597d0 2023-12-19T18:18:33.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98c9ffb640 1 -- 172.21.15.26:0/4140823750 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc0a7c60 con 0x7f98cc1597d0 2023-12-19T18:18:33.151 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98c9ffb640 1 -- 172.21.15.26:0/4140823750 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc0af440 con 0x7f98cc1597d0 2023-12-19T18:18:33.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 -- 172.21.15.26:0/4140823750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98cc1597d0 msgr2=0x7f98cc070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 --2- 172.21.15.26:0/4140823750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98cc1597d0 0x7f98cc070ff0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00bb40 tx=0x7f98bc01eb30 comp rx=0 tx=0).stop 2023-12-19T18:18:33.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 -- 172.21.15.26:0/4140823750 shutdown_connections 2023-12-19T18:18:33.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 --2- 172.21.15.26:0/4140823750 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98cc06ab40 0x7f98cc0cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 --2- 172.21.15.26:0/4140823750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98cc1597d0 0x7f98cc070ff0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.152 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 -- 172.21.15.26:0/4140823750 >> 172.21.15.26:0/4140823750 conn(0x7f98cc0b0270 msgr2=0x7f98cc0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:33.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 -- 172.21.15.26:0/4140823750 shutdown_connections 2023-12-19T18:18:33.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 -- 172.21.15.26:0/4140823750 wait complete. 2023-12-19T18:18:33.153 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 Processor -- start 2023-12-19T18:18:33.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 -- start start 2023-12-19T18:18:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98cc06ab40 0x7f98cc14f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98cc06af80 0x7f98cc150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc1597d0 0x7f98cc0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc14f970 con 0x7f98cc06ab40 2023-12-19T18:18:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc14fc50 con 0x7f98cc1597d0 2023-12-19T18:18:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.152+0000 7f98d18fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc14ff30 con 0x7f98cc06af80 2023-12-19T18:18:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98caffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98cc06ab40 0x7f98cc14f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98ca7fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98cc06af80 0x7f98cc150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98cb7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc1597d0 0x7f98cc0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98ca7fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98cc06af80 0x7f98cc150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:48418/0 (socket says 172.21.15.26:48418) 2023-12-19T18:18:33.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98ca7fc640 1 -- 172.21.15.26:0/1560690489 learned_addr learned my addr 172.21.15.26:0/1560690489 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98ca7fc640 1 -- 172.21.15.26:0/1560690489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc1597d0 msgr2=0x7f98cc0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98ca7fc640 1 --2- 172.21.15.26:0/1560690489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc1597d0 0x7f98cc0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98ca7fc640 1 -- 172.21.15.26:0/1560690489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98cc06ab40 msgr2=0x7f98cc14f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98ca7fc640 1 --2- 172.21.15.26:0/1560690489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98cc06ab40 0x7f98cc14f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98ca7fc640 1 -- 172.21.15.26:0/1560690489 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc00b770 con 0x7f98cc06af80 2023-12-19T18:18:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98caffd640 1 --2- 172.21.15.26:0/1560690489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98cc06ab40 0x7f98cc14f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98ca7fc640 1 --2- 172.21.15.26:0/1560690489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98cc06af80 0x7f98cc150950 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f98c000e930 tx=0x7f98c000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98b3fff640 1 -- 172.21.15.26:0/1560690489 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0009690 con 0x7f98cc06af80 2023-12-19T18:18:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98b3fff640 1 -- 172.21.15.26:0/1560690489 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c000ca00 con 0x7f98cc06af80 2023-12-19T18:18:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98b3fff640 1 -- 172.21.15.26:0/1560690489 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c000cd70 con 0x7f98cc06af80 2023-12-19T18:18:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98d18fc640 1 -- 172.21.15.26:0/1560690489 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc150210 con 0x7f98cc06af80 2023-12-19T18:18:33.157 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98d18fc640 1 -- 172.21.15.26:0/1560690489 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc150e90 con 0x7f98cc06af80 2023-12-19T18:18:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98d18fc640 1 -- 172.21.15.26:0/1560690489 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc074270 con 0x7f98cc06af80 2023-12-19T18:18:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98b3fff640 1 -- 172.21.15.26:0/1560690489 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 139) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f98c0013470 con 0x7f98cc06af80 2023-12-19T18:18:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.156+0000 7f98b3fff640 1 --2- 172.21.15.26:0/1560690489 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f98b8065900 0x7f98b8067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.160+0000 7f98b3fff640 1 -- 172.21.15.26:0/1560690489 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6899+0+0 (secure 0 0 0) 0x7f98c0083800 con 0x7f98cc06af80 2023-12-19T18:18:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.160+0000 7f98caffd640 1 --2- 172.21.15.26:0/1560690489 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f98b8065900 0x7f98b8067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.160+0000 7f98caffd640 1 --2- 172.21.15.26:0/1560690489 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f98b8065900 0x7f98b8067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0b1000 tx=0x7f98bc008040 comp rx=0 tx=0).ready entity=mgr.11587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.164+0000 7f98b3fff640 1 -- 172.21.15.26:0/1560690489 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f98c00511c0 con 0x7f98cc06af80 2023-12-19T18:18:33.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.340+0000 7f98d18fc640 1 -- 172.21.15.26:0/1560690489 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f98cc06bfa0 con 0x7f98b8065900 2023-12-19T18:18:33.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.388+0000 7f98b3fff640 1 -- 172.21.15.26:0/1560690489 <== mgr.11587 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f98bc0bf0d0 con 0x7f98b8065900 2023-12-19T18:18:33.387 INFO:teuthology.orchestra.run.smithi026.stdout:Retention added to path snap_test_dir1 2023-12-19T18:18:33.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.388+0000 7f98d18fc640 1 -- 172.21.15.26:0/1560690489 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f98b8065900 msgr2=0x7f98b8067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.388+0000 7f98d18fc640 1 --2- 172.21.15.26:0/1560690489 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f98b8065900 0x7f98b8067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0b1000 tx=0x7f98bc008040 comp rx=0 tx=0).stop 2023-12-19T18:18:33.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.388+0000 7f98d18fc640 1 -- 172.21.15.26:0/1560690489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98cc06af80 msgr2=0x7f98cc150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.388+0000 7f98d18fc640 1 --2- 172.21.15.26:0/1560690489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98cc06af80 0x7f98cc150950 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f98c000e930 tx=0x7f98c000edf0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.388+0000 7f98d18fc640 1 -- 172.21.15.26:0/1560690489 shutdown_connections 2023-12-19T18:18:33.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.388+0000 7f98d18fc640 1 --2- 172.21.15.26:0/1560690489 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f98b8065900 0x7f98b8067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.388+0000 7f98d18fc640 1 --2- 172.21.15.26:0/1560690489 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f98cc1597d0 0x7f98cc0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.388+0000 7f98d18fc640 1 --2- 172.21.15.26:0/1560690489 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f98cc06af80 0x7f98cc150950 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.388+0000 7f98d18fc640 1 --2- 172.21.15.26:0/1560690489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f98cc06ab40 0x7f98cc14f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.388+0000 7f98d18fc640 1 -- 172.21.15.26:0/1560690489 >> 172.21.15.26:0/1560690489 conn(0x7f98cc0b0270 msgr2=0x7f98cc0af370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:33.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.388+0000 7f98d18fc640 1 -- 172.21.15.26:0/1560690489 shutdown_connections 2023-12-19T18:18:33.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.388+0000 7f98d18fc640 1 -- 172.21.15.26:0/1560690489 wait complete. 2023-12-19T18:18:33.409 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-12-19T18:18:33.415 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-19-18_19_00 in ~27s... 2023-12-19T18:18:33.415 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-12-19T18:18:33.415 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-12-19T18:18:33.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.560+0000 7fb236604640 1 Processor -- start 2023-12-19T18:18:33.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.560+0000 7fb236604640 1 -- start start 2023-12-19T18:18:33.563 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.560+0000 7fb236604640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2300ada30 0x7fb2300c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.560+0000 7fb236604640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb230164f00 0x7fb2300cae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.560+0000 7fb236604640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2300cb810 con 0x7fb2300cb3a0 2023-12-19T18:18:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.560+0000 7fb236604640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2300b16e0 con 0x7fb230164f00 2023-12-19T18:18:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.560+0000 7fb236604640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300ade10 con 0x7fb2300ada30 2023-12-19T18:18:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.564+0000 7fb234e01640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb230164f00 0x7fb2300cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.564+0000 7fb234e01640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb230164f00 0x7fb2300cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37926/0 (socket says 172.21.15.26:37926) 2023-12-19T18:18:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.564+0000 7fb234e01640 1 -- 172.21.15.26:0/2246758082 learned_addr learned my addr 172.21.15.26:0/2246758082 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.564+0000 7fb235602640 1 --2- 172.21.15.26:0/2246758082 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2300ada30 0x7fb2300c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.564+0000 7fb234e01640 1 -- 172.21.15.26:0/2246758082 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2300ada30 msgr2=0x7fb2300c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.564+0000 7fb234e01640 1 --2- 172.21.15.26:0/2246758082 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2300ada30 0x7fb2300c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.564+0000 7fb234e01640 1 -- 172.21.15.26:0/2246758082 >> v1:172.21.15.160:6789/0 conn(0x7fb2300cb3a0 legacy=0x7fb2300affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.565 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.564+0000 7fb234e01640 1 -- 172.21.15.26:0/2246758082 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2300cc9d0 con 0x7fb230164f00 2023-12-19T18:18:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.564+0000 7fb234e01640 1 --2- 172.21.15.26:0/2246758082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb230164f00 0x7fb2300cae60 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fb220009870 tx=0x7fb22001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a2a79012c57338 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.564+0000 7fb227fff640 1 -- 172.21.15.26:0/2246758082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2200ab600 con 0x7fb230164f00 2023-12-19T18:18:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.564+0000 7fb227fff640 1 -- 172.21.15.26:0/2246758082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2200a7050 con 0x7fb230164f00 2023-12-19T18:18:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.564+0000 7fb227fff640 1 -- 172.21.15.26:0/2246758082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2200ab980 con 0x7fb230164f00 2023-12-19T18:18:33.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 -- 172.21.15.26:0/2246758082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb230164f00 msgr2=0x7fb2300cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 --2- 172.21.15.26:0/2246758082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb230164f00 0x7fb2300cae60 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fb220009870 tx=0x7fb22001c540 comp rx=0 tx=0).stop 2023-12-19T18:18:33.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 -- 172.21.15.26:0/2246758082 shutdown_connections 2023-12-19T18:18:33.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 --2- 172.21.15.26:0/2246758082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb230164f00 0x7fb2300cae60 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 --2- 172.21.15.26:0/2246758082 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2300ada30 0x7fb2300c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 -- 172.21.15.26:0/2246758082 >> 172.21.15.26:0/2246758082 conn(0x7fb23006b0f0 msgr2=0x7fb23006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:33.568 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 -- 172.21.15.26:0/2246758082 shutdown_connections 2023-12-19T18:18:33.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 -- 172.21.15.26:0/2246758082 wait complete. 2023-12-19T18:18:33.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 Processor -- start 2023-12-19T18:18:33.569 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 -- start start 2023-12-19T18:18:33.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2300ada30 0x7fb230154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2300cb3a0 0x7fb23006ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb230164f00 0x7fb23006f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb230154a70 con 0x7fb230164f00 2023-12-19T18:18:33.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23015b3d0 con 0x7fb2300cb3a0 2023-12-19T18:18:33.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb236604640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb230155f50 con 0x7fb2300ada30 2023-12-19T18:18:33.571 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb235602640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2300ada30 0x7fb230154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb235e03640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb230164f00 0x7fb23006f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb234e01640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2300cb3a0 0x7fb23006ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb235e03640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb230164f00 0x7fb23006f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37932/0 (socket says 172.21.15.26:37932) 2023-12-19T18:18:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.568+0000 7fb235e03640 1 -- 172.21.15.26:0/283996860 learned_addr learned my addr 172.21.15.26:0/283996860 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.572+0000 7fb235e03640 1 -- 172.21.15.26:0/283996860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2300ada30 msgr2=0x7fb230154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.572+0000 7fb235e03640 1 --2- 172.21.15.26:0/283996860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2300ada30 0x7fb230154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.572+0000 7fb235e03640 1 -- 172.21.15.26:0/283996860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2300cb3a0 msgr2=0x7fb23006ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.572+0000 7fb235e03640 1 --2- 172.21.15.26:0/283996860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2300cb3a0 0x7fb23006ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.572+0000 7fb235e03640 1 -- 172.21.15.26:0/283996860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2200094d0 con 0x7fb230164f00 2023-12-19T18:18:33.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.572+0000 7fb235e03640 1 --2- 172.21.15.26:0/283996860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb230164f00 0x7fb23006f460 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fb22800c7a0 tx=0x7fb22800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.573 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.572+0000 7fb225ffb640 1 -- 172.21.15.26:0/283996860 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228003b80 con 0x7fb230164f00 2023-12-19T18:18:33.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.572+0000 7fb225ffb640 1 -- 172.21.15.26:0/283996860 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb228003ce0 con 0x7fb230164f00 2023-12-19T18:18:33.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.572+0000 7fb225ffb640 1 -- 172.21.15.26:0/283996860 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800aa90 con 0x7fb230164f00 2023-12-19T18:18:33.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.572+0000 7fb236604640 1 -- 172.21.15.26:0/283996860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2300c3680 con 0x7fb230164f00 2023-12-19T18:18:33.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.572+0000 7fb236604640 1 -- 172.21.15.26:0/283996860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2300c3070 con 0x7fb230164f00 2023-12-19T18:18:33.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.572+0000 7fb236604640 1 -- 172.21.15.26:0/283996860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2300c3810 con 0x7fb230164f00 2023-12-19T18:18:33.574 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.572+0000 7fb225ffb640 1 -- 172.21.15.26:0/283996860 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 139) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fb22800abf0 con 0x7fb230164f00 2023-12-19T18:18:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.576+0000 7fb225ffb640 1 --2- 172.21.15.26:0/283996860 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb214065900 0x7fb214067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.576+0000 7fb225ffb640 1 -- 172.21.15.26:0/283996860 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6899+0+0 (secure 0 0 0) 0x7fb228082ec0 con 0x7fb230164f00 2023-12-19T18:18:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.576+0000 7fb235602640 1 --2- 172.21.15.26:0/283996860 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb214065900 0x7fb214067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.577 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.576+0000 7fb235602640 1 --2- 172.21.15.26:0/283996860 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb214065900 0x7fb214067dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb22c006aa0 tx=0x7fb22c008040 comp rx=0 tx=0).ready entity=mgr.11587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.580+0000 7fb225ffb640 1 -- 172.21.15.26:0/283996860 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb2280506f0 con 0x7fb230164f00 2023-12-19T18:18:33.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.756+0000 7fb236604640 1 -- 172.21.15.26:0/283996860 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb2301659a0 con 0x7fb214065900 2023-12-19T18:18:33.763 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.764+0000 7fb225ffb640 1 -- 172.21.15.26:0/283996860 <== mgr.11587 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7fb22c0076d0 con 0x7fb214065900 2023-12-19T18:18:33.763 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:33.764 INFO:teuthology.orchestra.run.smithi026.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-19T18:18:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.764+0000 7fb236604640 1 -- 172.21.15.26:0/283996860 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb214065900 msgr2=0x7fb214067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.764+0000 7fb236604640 1 --2- 172.21.15.26:0/283996860 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb214065900 0x7fb214067dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb22c006aa0 tx=0x7fb22c008040 comp rx=0 tx=0).stop 2023-12-19T18:18:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.764+0000 7fb236604640 1 -- 172.21.15.26:0/283996860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb230164f00 msgr2=0x7fb23006f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.764+0000 7fb236604640 1 --2- 172.21.15.26:0/283996860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb230164f00 0x7fb23006f460 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7fb22800c7a0 tx=0x7fb22800cc60 comp rx=0 tx=0).stop 2023-12-19T18:18:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.764+0000 7fb236604640 1 -- 172.21.15.26:0/283996860 shutdown_connections 2023-12-19T18:18:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.764+0000 7fb236604640 1 --2- 172.21.15.26:0/283996860 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb214065900 0x7fb214067dc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.764+0000 7fb236604640 1 --2- 172.21.15.26:0/283996860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb230164f00 0x7fb23006f460 unknown :-1 s=CLOSED pgs=2383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.764+0000 7fb236604640 1 --2- 172.21.15.26:0/283996860 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2300cb3a0 0x7fb23006ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.764+0000 7fb236604640 1 --2- 172.21.15.26:0/283996860 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb2300ada30 0x7fb230154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.766 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.764+0000 7fb236604640 1 -- 172.21.15.26:0/283996860 >> 172.21.15.26:0/283996860 conn(0x7fb23006b0f0 msgr2=0x7fb230070610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:33.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.764+0000 7fb236604640 1 -- 172.21.15.26:0/283996860 shutdown_connections 2023-12-19T18:18:33.767 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.764+0000 7fb236604640 1 -- 172.21.15.26:0/283996860 wait complete. 2023-12-19T18:18:33.786 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-19T18:18:33.786 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-12-19T18:18:33.786 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-19T18:18:33.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc54d51640 1 Processor -- start 2023-12-19T18:18:33.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc54d51640 1 -- start start 2023-12-19T18:18:33.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc54d51640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdc500affe0 0x7fdc50071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc54d51640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc500b3960 0x7fdc500b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc54d51640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc5015e7c0 con 0x7fdc500b2030 2023-12-19T18:18:33.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc54d51640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc500c97a0 con 0x7fdc500affe0 2023-12-19T18:18:33.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc54d51640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50072360 con 0x7fdc500b3960 2023-12-19T18:18:33.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc4f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdc500b2030 0x7fdc500c9090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52764/0 (socket says 172.21.15.26:52764) 2023-12-19T18:18:33.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc4effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdc500affe0 0x7fdc50071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc4f7fe640 1 -- 172.21.15.26:0/3678204337 learned_addr learned my addr 172.21.15.26:0/3678204337 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:33.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc4ffff640 1 --2- 172.21.15.26:0/3678204337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc500b3960 0x7fdc500b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc4effd640 1 -- 172.21.15.26:0/3678204337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc500b3960 msgr2=0x7fdc500b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc4effd640 1 --2- 172.21.15.26:0/3678204337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc500b3960 0x7fdc500b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc4effd640 1 -- 172.21.15.26:0/3678204337 >> v1:172.21.15.26:6789/0 conn(0x7fdc500b2030 legacy=0x7fdc500c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc4effd640 1 -- 172.21.15.26:0/3678204337 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc500b04c0 con 0x7fdc500affe0 2023-12-19T18:18:33.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.932+0000 7fdc4e7fc640 1 -- 172.21.15.26:0/3678204337 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473502175 0 0) 0x7fdc5015e7c0 con 0x7fdc500b2030 2023-12-19T18:18:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.936+0000 7fdc4effd640 1 --2- 172.21.15.26:0/3678204337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc500affe0 0x7fdc50071e20 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fdc440097f0 tx=0x7fdc4401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788088dc1b31e771 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.936+0000 7fdc4e7fc640 1 -- 172.21.15.26:0/3678204337 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc440ab650 con 0x7fdc500affe0 2023-12-19T18:18:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.936+0000 7fdc4e7fc640 1 -- 172.21.15.26:0/3678204337 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc440ab7b0 con 0x7fdc500affe0 2023-12-19T18:18:33.937 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.936+0000 7fdc4e7fc640 1 -- 172.21.15.26:0/3678204337 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc440abb40 con 0x7fdc500affe0 2023-12-19T18:18:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.936+0000 7fdc54d51640 1 -- 172.21.15.26:0/3678204337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc500affe0 msgr2=0x7fdc50071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.936+0000 7fdc54d51640 1 --2- 172.21.15.26:0/3678204337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc500affe0 0x7fdc50071e20 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fdc440097f0 tx=0x7fdc4401c450 comp rx=0 tx=0).stop 2023-12-19T18:18:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.936+0000 7fdc54d51640 1 -- 172.21.15.26:0/3678204337 shutdown_connections 2023-12-19T18:18:33.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.936+0000 7fdc54d51640 1 --2- 172.21.15.26:0/3678204337 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc500b3960 0x7fdc500b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.936+0000 7fdc54d51640 1 --2- 172.21.15.26:0/3678204337 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc500affe0 0x7fdc50071e20 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.936+0000 7fdc54d51640 1 -- 172.21.15.26:0/3678204337 >> 172.21.15.26:0/3678204337 conn(0x7fdc5006b0f0 msgr2=0x7fdc5006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:33.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.936+0000 7fdc54d51640 1 -- 172.21.15.26:0/3678204337 shutdown_connections 2023-12-19T18:18:33.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.936+0000 7fdc54d51640 1 -- 172.21.15.26:0/3678204337 wait complete. 2023-12-19T18:18:33.939 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc54d51640 1 Processor -- start 2023-12-19T18:18:33.940 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc54d51640 1 -- start start 2023-12-19T18:18:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc54d51640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc500affe0 0x7fdc50153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc54d51640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc500b2030 0x7fdc501581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.942 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc54d51640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc500b3960 0x7fdc500705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc54d51640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc501f9700 con 0x7fdc500affe0 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc54d51640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50070150 con 0x7fdc500b2030 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc54d51640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5006f860 con 0x7fdc500b3960 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc4f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc500affe0 0x7fdc50153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc4f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc500affe0 0x7fdc50153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37948/0 (socket says 172.21.15.26:37948) 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc4f7fe640 1 -- 172.21.15.26:0/2390602429 learned_addr learned my addr 172.21.15.26:0/2390602429 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc4ffff640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc500b3960 0x7fdc500705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc4effd640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc500b2030 0x7fdc501581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc4f7fe640 1 -- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc500b3960 msgr2=0x7fdc500705e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc4f7fe640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc500b3960 0x7fdc500705e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc4f7fe640 1 -- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc500b2030 msgr2=0x7fdc501581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc4f7fe640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc500b2030 0x7fdc501581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc4f7fe640 1 -- 172.21.15.26:0/2390602429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc44009420 con 0x7fdc500affe0 2023-12-19T18:18:33.943 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc4f7fe640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc500affe0 0x7fdc50153ee0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000c480 tx=0x7fdc4000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc37fff640 1 -- 172.21.15.26:0/2390602429 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc400074e0 con 0x7fdc500affe0 2023-12-19T18:18:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc37fff640 1 -- 172.21.15.26:0/2390602429 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc40007660 con 0x7fdc500affe0 2023-12-19T18:18:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc37fff640 1 -- 172.21.15.26:0/2390602429 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40004a30 con 0x7fdc500affe0 2023-12-19T18:18:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc54d51640 1 -- 172.21.15.26:0/2390602429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5006fb00 con 0x7fdc500affe0 2023-12-19T18:18:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.940+0000 7fdc54d51640 1 -- 172.21.15.26:0/2390602429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc50154420 con 0x7fdc500affe0 2023-12-19T18:18:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.944+0000 7fdc54d51640 1 -- 172.21.15.26:0/2390602429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc5004e850 con 0x7fdc500affe0 2023-12-19T18:18:33.944 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.944+0000 7fdc37fff640 1 -- 172.21.15.26:0/2390602429 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 139) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fdc40004b90 con 0x7fdc500affe0 2023-12-19T18:18:33.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.944+0000 7fdc37fff640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdc3c065900 0x7fdc3c067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.945 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.944+0000 7fdc37fff640 1 -- 172.21.15.26:0/2390602429 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6899+0+0 (secure 0 0 0) 0x7fdc400824a0 con 0x7fdc500affe0 2023-12-19T18:18:33.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.944+0000 7fdc4effd640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdc3c065900 0x7fdc3c067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.946 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.944+0000 7fdc4effd640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdc3c065900 0x7fdc3c067dc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdc440af000 tx=0x7fdc440a7000 comp rx=0 tx=0).ready entity=mgr.11587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.949 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.948+0000 7fdc37fff640 1 -- 172.21.15.26:0/2390602429 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdc4004fcd0 con 0x7fdc500affe0 2023-12-19T18:18:33.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:33.972+0000 7fdc37fff640 1 -- 172.21.15.26:0/2390602429 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 140) v1 ==== 85348+0+0 (secure 0 0 0) 0x7fdc400500a0 con 0x7fdc500affe0 2023-12-19T18:18:34.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.132+0000 7fdc54d51640 1 -- 172.21.15.26:0/2390602429 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc500b4350 con 0x7fdc3c065900 2023-12-19T18:18:34.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.212+0000 7fdc37fff640 1 -- 172.21.15.26:0/2390602429 <== mgr.11587 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fdc440ab920 con 0x7fdc3c065900 2023-12-19T18:18:34.212 INFO:teuthology.orchestra.run.smithi026.stdout:Schedule removed for path snap_test_dir1 2023-12-19T18:18:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.212+0000 7fdc54d51640 1 -- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdc3c065900 msgr2=0x7fdc3c067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.212+0000 7fdc54d51640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdc3c065900 0x7fdc3c067dc0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdc440af000 tx=0x7fdc440a7000 comp rx=0 tx=0).stop 2023-12-19T18:18:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.212+0000 7fdc54d51640 1 -- 172.21.15.26:0/2390602429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc500affe0 msgr2=0x7fdc50153ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.212+0000 7fdc54d51640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc500affe0 0x7fdc50153ee0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000c480 tx=0x7fdc4000c940 comp rx=0 tx=0).stop 2023-12-19T18:18:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.212+0000 7fdc54d51640 1 -- 172.21.15.26:0/2390602429 shutdown_connections 2023-12-19T18:18:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.212+0000 7fdc54d51640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fdc3c065900 0x7fdc3c067dc0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.212+0000 7fdc54d51640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fdc500b3960 0x7fdc500705e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.212+0000 7fdc54d51640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc500b2030 0x7fdc501581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.212+0000 7fdc54d51640 1 --2- 172.21.15.26:0/2390602429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc500affe0 0x7fdc50153ee0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:34.214 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.212+0000 7fdc54d51640 1 -- 172.21.15.26:0/2390602429 >> 172.21.15.26:0/2390602429 conn(0x7fdc5006b0f0 msgr2=0x7fdc500c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.212+0000 7fdc54d51640 1 -- 172.21.15.26:0/2390602429 shutdown_connections 2023-12-19T18:18:34.215 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.212+0000 7fdc54d51640 1 -- 172.21.15.26:0/2390602429 wait complete. 2023-12-19T18:18:34.233 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-19T18:18:34.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:34.233 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-19T18:18:34.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:34.244 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-19T18:18:34.302 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-19T18:18:34.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d8811640 1 Processor -- start 2023-12-19T18:18:34.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d8811640 1 -- start start 2023-12-19T18:18:34.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d8811640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95d40cb860 0x7f95d406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:34.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d8811640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95d415d530 0x7f95d415d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:34.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d8811640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d415e240 con 0x7f95d41599e0 2023-12-19T18:18:34.489 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d8811640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d40c9520 con 0x7f95d40cb860 2023-12-19T18:18:34.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d8811640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d40c9800 con 0x7f95d415d530 2023-12-19T18:18:34.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d1d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95d40cb860 0x7f95d406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:34.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d1d9b640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95d40cb860 0x7f95d406f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37950/0 (socket says 172.21.15.26:37950) 2023-12-19T18:18:34.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d1d9b640 1 -- 172.21.15.26:0/3500609785 learned_addr learned my addr 172.21.15.26:0/3500609785 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:34.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d2d9d640 1 --2- 172.21.15.26:0/3500609785 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95d415d530 0x7f95d415d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:34.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d1d9b640 1 -- 172.21.15.26:0/3500609785 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95d415d530 msgr2=0x7f95d415d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:34.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d1d9b640 1 --2- 172.21.15.26:0/3500609785 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95d415d530 0x7f95d415d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:34.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d1d9b640 1 -- 172.21.15.26:0/3500609785 >> v1:172.21.15.160:6789/0 conn(0x7f95d41599e0 legacy=0x7f95d40cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:34.490 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.488+0000 7f95d1d9b640 1 -- 172.21.15.26:0/3500609785 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40c9ae0 con 0x7f95d40cb860 2023-12-19T18:18:34.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.492+0000 7f95d1d9b640 1 --2- 172.21.15.26:0/3500609785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d40cb860 0x7f95d406f6d0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009870 tx=0x7f95c801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea90db38ee114c38 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:34.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.492+0000 7f95d159a640 1 -- 172.21.15.26:0/3500609785 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c80a9620 con 0x7f95d40cb860 2023-12-19T18:18:34.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.492+0000 7f95d159a640 1 -- 172.21.15.26:0/3500609785 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c80a9780 con 0x7f95d40cb860 2023-12-19T18:18:34.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.492+0000 7f95d159a640 1 -- 172.21.15.26:0/3500609785 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c80a9b10 con 0x7f95d40cb860 2023-12-19T18:18:34.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.492+0000 7f95d8811640 1 -- 172.21.15.26:0/3500609785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d40cb860 msgr2=0x7f95d406f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:34.493 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.492+0000 7f95d8811640 1 --2- 172.21.15.26:0/3500609785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d40cb860 0x7f95d406f6d0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009870 tx=0x7f95c801c540 comp rx=0 tx=0).stop 2023-12-19T18:18:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.492+0000 7f95d8811640 1 -- 172.21.15.26:0/3500609785 shutdown_connections 2023-12-19T18:18:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.492+0000 7f95d8811640 1 --2- 172.21.15.26:0/3500609785 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95d415d530 0x7f95d415d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.492+0000 7f95d8811640 1 --2- 172.21.15.26:0/3500609785 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d40cb860 0x7f95d406f6d0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.492+0000 7f95d8811640 1 -- 172.21.15.26:0/3500609785 >> 172.21.15.26:0/3500609785 conn(0x7f95d4066160 msgr2=0x7f95d40b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.492+0000 7f95d8811640 1 -- 172.21.15.26:0/3500609785 shutdown_connections 2023-12-19T18:18:34.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.492+0000 7f95d8811640 1 -- 172.21.15.26:0/3500609785 wait complete. 2023-12-19T18:18:34.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d8811640 1 Processor -- start 2023-12-19T18:18:34.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d8811640 1 -- start start 2023-12-19T18:18:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d8811640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d40cb860 0x7f95d415b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d8811640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95d41599e0 0x7f95d40b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d8811640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95d415d530 0x7f95d4148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d8811640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d415b1b0 con 0x7f95d40cb860 2023-12-19T18:18:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d8811640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d40b3970 con 0x7f95d415d530 2023-12-19T18:18:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d8811640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d415a530 con 0x7f95d41599e0 2023-12-19T18:18:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d40cb860 0x7f95d415b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d259c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d40cb860 0x7f95d415b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37954/0 (socket says 172.21.15.26:37954) 2023-12-19T18:18:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d259c640 1 -- 172.21.15.26:0/396798595 learned_addr learned my addr 172.21.15.26:0/396798595 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d2d9d640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95d415d530 0x7f95d4148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d1d9b640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95d41599e0 0x7f95d40b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d259c640 1 -- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95d41599e0 msgr2=0x7f95d40b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:34.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d259c640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95d41599e0 0x7f95d40b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:34.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d259c640 1 -- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95d415d530 msgr2=0x7f95d4148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:34.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d259c640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95d415d530 0x7f95d4148b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:34.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d259c640 1 -- 172.21.15.26:0/396798595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c80094d0 con 0x7f95d40cb860 2023-12-19T18:18:34.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d259c640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d40cb860 0x7f95d415b5d0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c450 tx=0x7f95c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:34.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95c2ffd640 1 -- 172.21.15.26:0/396798595 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c40052d0 con 0x7f95d40cb860 2023-12-19T18:18:34.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95c2ffd640 1 -- 172.21.15.26:0/396798595 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c4005450 con 0x7f95d40cb860 2023-12-19T18:18:34.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d8811640 1 -- 172.21.15.26:0/396798595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d415a810 con 0x7f95d40cb860 2023-12-19T18:18:34.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95c2ffd640 1 -- 172.21.15.26:0/396798595 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4012870 con 0x7f95d40cb860 2023-12-19T18:18:34.499 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d8811640 1 -- 172.21.15.26:0/396798595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d415add0 con 0x7f95d40cb860 2023-12-19T18:18:34.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.496+0000 7f95d8811640 1 -- 172.21.15.26:0/396798595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d4074270 con 0x7f95d40cb860 2023-12-19T18:18:34.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.500+0000 7f95c2ffd640 1 -- 172.21.15.26:0/396798595 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 140) v1 ==== 85348+0+0 (secure 0 0 0) 0x7f95c40129d0 con 0x7f95d40cb860 2023-12-19T18:18:34.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.500+0000 7f95c2ffd640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f95b4065c00 0x7f95b40680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:34.500 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.500+0000 7f95c2ffd640 1 -- 172.21.15.26:0/396798595 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6899+0+0 (secure 0 0 0) 0x7f95c4083250 con 0x7f95d40cb860 2023-12-19T18:18:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.500+0000 7f95d1d9b640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f95b4065c00 0x7f95b40680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:34.501 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.500+0000 7f95d1d9b640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f95b4065c00 0x7f95b40680c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f95c80ae000 tx=0x7f95c80b3040 comp rx=0 tx=0).ready entity=mgr.11587 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:34.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.504+0000 7f95c2ffd640 1 -- 172.21.15.26:0/396798595 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f95c4050660 con 0x7f95d40cb860 2023-12-19T18:18:34.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:34.700+0000 7f95d8811640 1 -- 172.21.15.26:0/396798595 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f95d4158a10 con 0x7f95d40cb860 2023-12-19T18:18:35.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.120+0000 7f95c2ffd640 1 -- 172.21.15.26:0/396798595 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 141) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f95c405a080 con 0x7f95d40cb860 2023-12-19T18:18:35.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.124+0000 7f95c2ffd640 1 -- 172.21.15.26:0/396798595 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v141) v1 ==== 93+0+0 (secure 0 0 0) 0x7f95c407fc00 con 0x7f95d40cb860 2023-12-19T18:18:35.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.124+0000 7f95d8811640 1 -- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f95b4065c00 msgr2=0x7f95b40680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:35.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.124+0000 7f95d8811640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f95b4065c00 0x7f95b40680c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f95c80ae000 tx=0x7f95c80b3040 comp rx=0 tx=0).stop 2023-12-19T18:18:35.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.124+0000 7f95d8811640 1 -- 172.21.15.26:0/396798595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d40cb860 msgr2=0x7f95d415b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:35.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.124+0000 7f95d8811640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d40cb860 0x7f95d415b5d0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c450 tx=0x7f95c400c910 comp rx=0 tx=0).stop 2023-12-19T18:18:35.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.124+0000 7f95d8811640 1 -- 172.21.15.26:0/396798595 shutdown_connections 2023-12-19T18:18:35.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.124+0000 7f95d8811640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f95b4065c00 0x7f95b40680c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.124+0000 7f95d8811640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f95d415d530 0x7f95d4148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.124+0000 7f95d8811640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f95d41599e0 0x7f95d40b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.124+0000 7f95d8811640 1 --2- 172.21.15.26:0/396798595 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95d40cb860 0x7f95d415b5d0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.124+0000 7f95d8811640 1 -- 172.21.15.26:0/396798595 >> 172.21.15.26:0/396798595 conn(0x7f95d4066160 msgr2=0x7f95d40b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:35.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.124+0000 7f95d8811640 1 -- 172.21.15.26:0/396798595 shutdown_connections 2023-12-19T18:18:35.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.124+0000 7f95d8811640 1 -- 172.21.15.26:0/396798595 wait complete. 2023-12-19T18:18:35.148 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:18:35.148 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:18:35.274 DEBUG:teuthology.orchestra.run.smithi160:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:18:35.304 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:35.304+0000 7f707bc9b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:18:35.307 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:35.303+0000 7f0b7011a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:18:35.354 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:18:35.354 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:18:35.354 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:35.364 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:18:35.364 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:35.416 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:35.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.604+0000 7f61692a7640 1 Processor -- start 2023-12-19T18:18:35.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.604+0000 7f61692a7640 1 -- start start 2023-12-19T18:18:35.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f61692a7640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f616415cca0 0x7f61640b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:35.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f61692a7640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f61640b6b20 0x7f61640d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:35.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f61692a7640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61640b6fb0 con 0x7f61640b8760 2023-12-19T18:18:35.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f61692a7640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61640b7c30 con 0x7f616415cca0 2023-12-19T18:18:35.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f61692a7640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61640b7290 con 0x7f61640b6b20 2023-12-19T18:18:35.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f6163fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f616415cca0 0x7f61640b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:35.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f6163fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f616415cca0 0x7f61640b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37992/0 (socket says 172.21.15.26:37992) 2023-12-19T18:18:35.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f6163fff640 1 -- 172.21.15.26:0/1652812822 learned_addr learned my addr 172.21.15.26:0/1652812822 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:35.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f6168aa6640 1 --2- 172.21.15.26:0/1652812822 >> v2:172.21.15.160:3300/0 conn(0x7f61640b6b20 0x7f61640d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:35.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f6163fff640 1 -- 172.21.15.26:0/1652812822 >> v2:172.21.15.160:3300/0 conn(0x7f61640b6b20 msgr2=0x7f61640d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:35.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f6163fff640 1 --2- 172.21.15.26:0/1652812822 >> v2:172.21.15.160:3300/0 conn(0x7f61640b6b20 0x7f61640d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f6163fff640 1 -- 172.21.15.26:0/1652812822 >> v1:172.21.15.160:6789/0 conn(0x7f61640b8760 legacy=0x7f61640cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:35.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f6163fff640 1 -- 172.21.15.26:0/1652812822 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640b7570 con 0x7f616415cca0 2023-12-19T18:18:35.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.608+0000 7f6168aa6640 1 --2- 172.21.15.26:0/1652812822 >> v2:172.21.15.160:3300/0 conn(0x7f61640b6b20 0x7f61640d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f6163fff640 1 --2- 172.21.15.26:0/1652812822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f616415cca0 0x7f61640b65e0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f61640d3510 tx=0x7f615401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b3e265deda09c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:35.611 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f6162ffd640 1 -- 172.21.15.26:0/1652812822 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61540aa610 con 0x7f616415cca0 2023-12-19T18:18:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f6162ffd640 1 -- 172.21.15.26:0/1652812822 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61540a6040 con 0x7f616415cca0 2023-12-19T18:18:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f6162ffd640 1 -- 172.21.15.26:0/1652812822 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61540aa990 con 0x7f616415cca0 2023-12-19T18:18:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f61692a7640 1 -- 172.21.15.26:0/1652812822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f616415cca0 msgr2=0x7f61640b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:35.612 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f61692a7640 1 --2- 172.21.15.26:0/1652812822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f616415cca0 0x7f61640b65e0 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f61640d3510 tx=0x7f615401c490 comp rx=0 tx=0).stop 2023-12-19T18:18:35.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f61692a7640 1 -- 172.21.15.26:0/1652812822 shutdown_connections 2023-12-19T18:18:35.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f61692a7640 1 --2- 172.21.15.26:0/1652812822 >> v2:172.21.15.160:3300/0 conn(0x7f61640b6b20 0x7f61640d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f61692a7640 1 --2- 172.21.15.26:0/1652812822 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f616415cca0 0x7f61640b65e0 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f61692a7640 1 -- 172.21.15.26:0/1652812822 >> 172.21.15.26:0/1652812822 conn(0x7f616406a120 msgr2=0x7f616406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:35.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f61692a7640 1 -- 172.21.15.26:0/1652812822 shutdown_connections 2023-12-19T18:18:35.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f61692a7640 1 -- 172.21.15.26:0/1652812822 wait complete. 2023-12-19T18:18:35.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f61692a7640 1 Processor -- start 2023-12-19T18:18:35.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.612+0000 7f61692a7640 1 -- start start 2023-12-19T18:18:35.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f61692a7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640b8760 0x7f6164156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:35.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f61692a7640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f616415cca0 0x7f616414f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:35.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f61692a7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6164153200 0x7f6164150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:35.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f61692a7640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164157680 con 0x7f61640b8760 2023-12-19T18:18:35.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f61692a7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164157960 con 0x7f6164153200 2023-12-19T18:18:35.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f61692a7640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6164157c40 con 0x7f616415cca0 2023-12-19T18:18:35.616 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f6163fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640b8760 0x7f6164156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:35.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f6163fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640b8760 0x7f6164156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37998/0 (socket says 172.21.15.26:37998) 2023-12-19T18:18:35.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f6163fff640 1 -- 172.21.15.26:0/2876208350 learned_addr learned my addr 172.21.15.26:0/2876208350 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:35.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f6168aa6640 1 --2- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6164153200 0x7f6164150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:35.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f61637fe640 1 --2- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f616415cca0 0x7f616414f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:35.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f6163fff640 1 -- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f616415cca0 msgr2=0x7f616414f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:35.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f6163fff640 1 --2- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f616415cca0 0x7f616414f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f6163fff640 1 -- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6164153200 msgr2=0x7f6164150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:35.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f6163fff640 1 --2- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6164153200 0x7f6164150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f6163fff640 1 -- 172.21.15.26:0/2876208350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61540094d0 con 0x7f61640b8760 2023-12-19T18:18:35.617 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f6163fff640 1 --2- 172.21.15.26:0/2876208350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640b8760 0x7f6164156ff0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f615401a0c0 tx=0x7f615401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:35.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f6160ff9640 1 -- 172.21.15.26:0/2876208350 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61540bd3f0 con 0x7f61640b8760 2023-12-19T18:18:35.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f6160ff9640 1 -- 172.21.15.26:0/2876208350 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61540bd550 con 0x7f61640b8760 2023-12-19T18:18:35.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f6160ff9640 1 -- 172.21.15.26:0/2876208350 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61540bd900 con 0x7f61640b8760 2023-12-19T18:18:35.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f61692a7640 1 -- 172.21.15.26:0/2876208350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6164157f20 con 0x7f61640b8760 2023-12-19T18:18:35.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f61692a7640 1 -- 172.21.15.26:0/2876208350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61641501e0 con 0x7f61640b8760 2023-12-19T18:18:35.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.616+0000 7f61692a7640 1 -- 172.21.15.26:0/2876208350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6164067400 con 0x7f61640b8760 2023-12-19T18:18:35.620 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.620+0000 7f6160ff9640 1 -- 172.21.15.26:0/2876208350 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 141) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f61540b33f0 con 0x7f61640b8760 2023-12-19T18:18:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.620+0000 7f6160ff9640 1 --2- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6150065bb0 0x7f6150068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.620+0000 7f6160ff9640 1 -- 172.21.15.26:0/2876208350 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6899+0+0 (secure 0 0 0) 0x7f615411dac0 con 0x7f61640b8760 2023-12-19T18:18:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.620+0000 7f61637fe640 1 -- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6150065bb0 msgr2=0x7f6150068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:35.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.620+0000 7f61637fe640 1 --2- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6150065bb0 0x7f6150068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:18:35.625 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.624+0000 7f6160ff9640 1 -- 172.21.15.26:0/2876208350 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f61540eae60 con 0x7f61640b8760 2023-12-19T18:18:35.723 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:35.719+0000 7f0b7011a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:18:35.753 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:35.752+0000 7f707bc9b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:18:35.787 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.784+0000 7f61692a7640 1 -- 172.21.15.26:0/2876208350 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61640b2c30 con 0x7f61640b8760 2023-12-19T18:18:35.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.788+0000 7f6160ff9640 1 -- 172.21.15.26:0/2876208350 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v255) v1 ==== 74+0+17830 (secure 0 0 0) 0x7f61540ee480 con 0x7f61640b8760 2023-12-19T18:18:35.789 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:35.789 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":255,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:18:31.899356+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:17.314988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:18.558040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1122697388":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/1748568728":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/2225649811":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/2614640962":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/1309045557":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3586017330":"2023-12-20T18:18:31.899321+0000","172.21.15.160:6833/48394":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3350159660":"2023-12-20T18:18:31.899321+0000","172.21.15.160:6832/48394":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3229222145":"2023-12-20T18:18:31.899321+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:35.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.792+0000 7f61692a7640 1 -- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6150065bb0 msgr2=0x7f6150068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:18:35.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.792+0000 7f61692a7640 1 --2- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6150065bb0 0x7f6150068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.792+0000 7f61692a7640 1 -- 172.21.15.26:0/2876208350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640b8760 msgr2=0x7f6164156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:35.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.792+0000 7f61692a7640 1 --2- 172.21.15.26:0/2876208350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640b8760 0x7f6164156ff0 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f615401a0c0 tx=0x7f615401acd0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.792+0000 7f61692a7640 1 -- 172.21.15.26:0/2876208350 shutdown_connections 2023-12-19T18:18:35.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.792+0000 7f61692a7640 1 --2- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6150065bb0 0x7f6150068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.792+0000 7f61692a7640 1 --2- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6164153200 0x7f6164150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.792+0000 7f61692a7640 1 --2- 172.21.15.26:0/2876208350 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f616415cca0 0x7f616414f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.792+0000 7f61692a7640 1 --2- 172.21.15.26:0/2876208350 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f61640b8760 0x7f6164156ff0 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.792+0000 7f61692a7640 1 -- 172.21.15.26:0/2876208350 >> 172.21.15.26:0/2876208350 conn(0x7f616406a120 msgr2=0x7f61640658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:35.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.792+0000 7f61692a7640 1 -- 172.21.15.26:0/2876208350 shutdown_connections 2023-12-19T18:18:35.793 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.792+0000 7f61692a7640 1 -- 172.21.15.26:0/2876208350 wait complete. 2023-12-19T18:18:35.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:18:17.314988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '253', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:18:18.558040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '254', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:18:35.813 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:18:35.872 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:35.871+0000 7f0b7011a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:18:35.915 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:35.912+0000 7f707bc9b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:18:35.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.960+0000 7f2940655640 1 Processor -- start 2023-12-19T18:18:35.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.960+0000 7f2940655640 1 -- start start 2023-12-19T18:18:35.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.960+0000 7f2940655640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f293815a9c0 0x7f29380706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:35.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.960+0000 7f2940655640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f293815ada0 0x7f29380b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:35.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.960+0000 7f2940655640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29380b46c0 con 0x7f293806ac80 2023-12-19T18:18:35.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.960+0000 7f2940655640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29380cc270 con 0x7f293815a9c0 2023-12-19T18:18:35.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.960+0000 7f2940655640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2938070d90 con 0x7f293815ada0 2023-12-19T18:18:35.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.960+0000 7f293e3f1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f293815a9c0 0x7f29380706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:35.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.960+0000 7f293e3f1640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f293815a9c0 0x7f29380706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36004/0 (socket says 172.21.15.26:36004) 2023-12-19T18:18:35.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.960+0000 7f293e3f1640 1 -- 172.21.15.26:0/1481330759 learned_addr learned my addr 172.21.15.26:0/1481330759 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:35.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.964+0000 7f293dbf0640 1 --2- 172.21.15.26:0/1481330759 >> v2:172.21.15.160:3300/0 conn(0x7f293815ada0 0x7f29380b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:35.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.964+0000 7f293e3f1640 1 -- 172.21.15.26:0/1481330759 >> v2:172.21.15.160:3300/0 conn(0x7f293815ada0 msgr2=0x7f29380b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:35.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.964+0000 7f293e3f1640 1 --2- 172.21.15.26:0/1481330759 >> v2:172.21.15.160:3300/0 conn(0x7f293815ada0 0x7f29380b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.964+0000 7f293e3f1640 1 -- 172.21.15.26:0/1481330759 >> v1:172.21.15.160:6789/0 conn(0x7f293806ac80 legacy=0x7f29380b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:35.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.964+0000 7f293e3f1640 1 -- 172.21.15.26:0/1481330759 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2938071070 con 0x7f293815a9c0 2023-12-19T18:18:35.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.964+0000 7f293e3f1640 1 --2- 172.21.15.26:0/1481330759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f293815a9c0 0x7f29380706d0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f2938070160 tx=0x7f293401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce2fa931fd75f6d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:35.966 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.964+0000 7f293d3ef640 1 -- 172.21.15.26:0/1481330759 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29340ab640 con 0x7f293815a9c0 2023-12-19T18:18:35.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.964+0000 7f293d3ef640 1 -- 172.21.15.26:0/1481330759 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29340a7070 con 0x7f293815a9c0 2023-12-19T18:18:35.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.964+0000 7f293d3ef640 1 -- 172.21.15.26:0/1481330759 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29340ab9c0 con 0x7f293815a9c0 2023-12-19T18:18:35.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 -- 172.21.15.26:0/1481330759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f293815a9c0 msgr2=0x7f29380706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:35.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 --2- 172.21.15.26:0/1481330759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f293815a9c0 0x7f29380706d0 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f2938070160 tx=0x7f293401c440 comp rx=0 tx=0).stop 2023-12-19T18:18:35.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 -- 172.21.15.26:0/1481330759 shutdown_connections 2023-12-19T18:18:35.967 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 --2- 172.21.15.26:0/1481330759 >> v2:172.21.15.160:3300/0 conn(0x7f293815ada0 0x7f29380b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 --2- 172.21.15.26:0/1481330759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f293815a9c0 0x7f29380706d0 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 -- 172.21.15.26:0/1481330759 >> 172.21.15.26:0/1481330759 conn(0x7f29380ce730 msgr2=0x7f29380ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 -- 172.21.15.26:0/1481330759 shutdown_connections 2023-12-19T18:18:35.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 -- 172.21.15.26:0/1481330759 wait complete. 2023-12-19T18:18:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 Processor -- start 2023-12-19T18:18:35.969 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 -- start start 2023-12-19T18:18:35.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f293806ac80 0x7f29381514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:35.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f293815a9c0 0x7f293814cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:35.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f293815ada0 0x7f29381585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:35.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29380b1fc0 con 0x7f293806ac80 2023-12-19T18:18:35.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29380b1dc0 con 0x7f293815a9c0 2023-12-19T18:18:35.971 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f2940655640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2938151ab0 con 0x7f293815ada0 2023-12-19T18:18:35.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f293e3f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f293806ac80 0x7f29381514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:35.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f293e3f1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f293806ac80 0x7f29381514c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36008/0 (socket says 172.21.15.26:36008) 2023-12-19T18:18:35.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f293e3f1640 1 -- 172.21.15.26:0/1355406246 learned_addr learned my addr 172.21.15.26:0/1355406246 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:35.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f293dbf0640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f293815a9c0 0x7f293814cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:35.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f293ebf2640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f293815ada0 0x7f29381585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:35.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f293e3f1640 1 -- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f293815ada0 msgr2=0x7f29381585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:35.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f293e3f1640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f293815ada0 0x7f29381585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f293e3f1640 1 -- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f293815a9c0 msgr2=0x7f293814cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:35.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f293e3f1640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f293815a9c0 0x7f293814cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:35.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f293e3f1640 1 -- 172.21.15.26:0/1355406246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29340094d0 con 0x7f293806ac80 2023-12-19T18:18:35.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.968+0000 7f293ebf2640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f293815ada0 0x7f29381585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:35.972 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.972+0000 7f293e3f1640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f293806ac80 0x7f29381514c0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f2934019050 tx=0x7f293401ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:35.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.972+0000 7f292effd640 1 -- 172.21.15.26:0/1355406246 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29340abe80 con 0x7f293806ac80 2023-12-19T18:18:35.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.972+0000 7f292effd640 1 -- 172.21.15.26:0/1355406246 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29340be3f0 con 0x7f293806ac80 2023-12-19T18:18:35.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.972+0000 7f2940655640 1 -- 172.21.15.26:0/1355406246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2938151d90 con 0x7f293806ac80 2023-12-19T18:18:35.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.972+0000 7f292effd640 1 -- 172.21.15.26:0/1355406246 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29340be780 con 0x7f293806ac80 2023-12-19T18:18:35.973 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.972+0000 7f2940655640 1 -- 172.21.15.26:0/1355406246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2938152350 con 0x7f293806ac80 2023-12-19T18:18:35.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.972+0000 7f2940655640 1 -- 172.21.15.26:0/1355406246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29380b0840 con 0x7f293806ac80 2023-12-19T18:18:35.974 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.972+0000 7f292effd640 1 -- 172.21.15.26:0/1355406246 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 141) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f29340b43f0 con 0x7f293806ac80 2023-12-19T18:18:35.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.972+0000 7f292effd640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f291c065bb0 0x7f291c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:35.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.972+0000 7f292effd640 1 -- 172.21.15.26:0/1355406246 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6899+0+0 (secure 0 0 0) 0x7f293411d520 con 0x7f293806ac80 2023-12-19T18:18:35.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.972+0000 7f293dbf0640 1 -- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f291c065bb0 msgr2=0x7f291c068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:35.975 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.972+0000 7f293dbf0640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f291c065bb0 0x7f291c068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:18:35.978 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:35.976+0000 7f292effd640 1 -- 172.21.15.26:0/1355406246 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f29340ea830 con 0x7f293806ac80 2023-12-19T18:18:36.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.136+0000 7f2940655640 1 -- 172.21.15.26:0/1355406246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2938074270 con 0x7f293806ac80 2023-12-19T18:18:36.139 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.140+0000 7f292effd640 1 -- 172.21.15.26:0/1355406246 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v255) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29340ede50 con 0x7f293806ac80 2023-12-19T18:18:36.139 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:18:36.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.140+0000 7f2940655640 1 -- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f291c065bb0 msgr2=0x7f291c068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:18:36.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.140+0000 7f2940655640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f291c065bb0 0x7f291c068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.141 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.140+0000 7f2940655640 1 -- 172.21.15.26:0/1355406246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f293806ac80 msgr2=0x7f29381514c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:36.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.140+0000 7f2940655640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f293806ac80 0x7f29381514c0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f2934019050 tx=0x7f293401ac80 comp rx=0 tx=0).stop 2023-12-19T18:18:36.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.140+0000 7f2940655640 1 -- 172.21.15.26:0/1355406246 shutdown_connections 2023-12-19T18:18:36.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.140+0000 7f2940655640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f291c065bb0 0x7f291c068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.140+0000 7f2940655640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f293815ada0 0x7f29381585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.140+0000 7f2940655640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f293815a9c0 0x7f293814cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.140+0000 7f2940655640 1 --2- 172.21.15.26:0/1355406246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f293806ac80 0x7f29381514c0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.140+0000 7f2940655640 1 -- 172.21.15.26:0/1355406246 >> 172.21.15.26:0/1355406246 conn(0x7f29380ce730 msgr2=0x7f29381536c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:36.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.140+0000 7f2940655640 1 -- 172.21.15.26:0/1355406246 shutdown_connections 2023-12-19T18:18:36.142 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.140+0000 7f2940655640 1 -- 172.21.15.26:0/1355406246 wait complete. 2023-12-19T18:18:36.162 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:18:36.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.308+0000 7f7faf3b3640 1 Processor -- start 2023-12-19T18:18:36.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.308+0000 7f7faf3b3640 1 -- start start 2023-12-19T18:18:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.308+0000 7f7faf3b3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7fa815cf60 0x7f7fa80d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.308+0000 7f7faf3b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fa80b88c0 0x7f7fa80d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.308+0000 7f7faf3b3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa80d1570 con 0x7f7fa80b4600 2023-12-19T18:18:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.308+0000 7f7faf3b3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa80d1850 con 0x7f7fa815cf60 2023-12-19T18:18:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.308+0000 7f7faf3b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80d1b30 con 0x7f7fa80b88c0 2023-12-19T18:18:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.308+0000 7f7fad950640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7fa80b4600 0x7f7fa8069d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34012/0 (socket says 172.21.15.26:34012) 2023-12-19T18:18:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.308+0000 7f7fad950640 1 -- 172.21.15.26:0/2021813916 learned_addr learned my addr 172.21.15.26:0/2021813916 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.308+0000 7f7fad14f640 1 --2- 172.21.15.26:0/2021813916 >> v2:172.21.15.160:3300/0 conn(0x7f7fa815cf60 0x7f7fa80d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.312+0000 7f7fac94e640 1 --2- 172.21.15.26:0/2021813916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fa80b88c0 0x7f7fa80d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.312+0000 7f7f9ffff640 1 -- 172.21.15.26:0/2021813916 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975549751 0 0) 0x7f7fa80d1570 con 0x7f7fa80b4600 2023-12-19T18:18:36.311 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.312+0000 7f7f9ffff640 1 -- 172.21.15.26:0/2021813916 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f94003530 con 0x7f7fa80b4600 2023-12-19T18:18:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.312+0000 7f7fad14f640 1 -- 172.21.15.26:0/2021813916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fa80b88c0 msgr2=0x7f7fa80d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.312+0000 7f7fad14f640 1 --2- 172.21.15.26:0/2021813916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fa80b88c0 0x7f7fa80d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.312+0000 7f7fad14f640 1 -- 172.21.15.26:0/2021813916 >> v1:172.21.15.26:6789/0 conn(0x7f7fa80b4600 legacy=0x7f7fa8069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:36.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.312+0000 7f7fad14f640 1 -- 172.21.15.26:0/2021813916 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa80d1e10 con 0x7f7fa815cf60 2023-12-19T18:18:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.312+0000 7f7fad14f640 1 --2- 172.21.15.26:0/2021813916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa815cf60 0x7f7fa80d2610 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40097c0 tx=0x7f7fa401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc35e3fbfb40a917 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:36.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.312+0000 7f7f9ffff640 1 -- 172.21.15.26:0/2021813916 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40ac640 con 0x7f7fa815cf60 2023-12-19T18:18:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.312+0000 7f7f9ffff640 1 -- 172.21.15.26:0/2021813916 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa40ac7a0 con 0x7f7fa815cf60 2023-12-19T18:18:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.312+0000 7f7f9ffff640 1 -- 172.21.15.26:0/2021813916 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40acb30 con 0x7f7fa815cf60 2023-12-19T18:18:36.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 -- 172.21.15.26:0/2021813916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa815cf60 msgr2=0x7f7fa80d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 --2- 172.21.15.26:0/2021813916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa815cf60 0x7f7fa80d2610 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40097c0 tx=0x7f7fa401c3c0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 -- 172.21.15.26:0/2021813916 shutdown_connections 2023-12-19T18:18:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 --2- 172.21.15.26:0/2021813916 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fa80b88c0 0x7f7fa80d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 --2- 172.21.15.26:0/2021813916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa815cf60 0x7f7fa80d2610 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 -- 172.21.15.26:0/2021813916 >> 172.21.15.26:0/2021813916 conn(0x7f7fa8065ee0 msgr2=0x7f7fa80cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 -- 172.21.15.26:0/2021813916 shutdown_connections 2023-12-19T18:18:36.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 -- 172.21.15.26:0/2021813916 wait complete. 2023-12-19T18:18:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 Processor -- start 2023-12-19T18:18:36.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 -- start start 2023-12-19T18:18:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa80b4600 0x7f7fa815eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fa80b88c0 0x7f7fa80c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa815cf60 0x7f7fa80c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa815e780 con 0x7f7fa80b4600 2023-12-19T18:18:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80b83a0 con 0x7f7fa815cf60 2023-12-19T18:18:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7faf3b3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa815dde0 con 0x7f7fa80b88c0 2023-12-19T18:18:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7fad14f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa80b4600 0x7f7fa815eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7fad14f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa80b4600 0x7f7fa815eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36024/0 (socket says 172.21.15.26:36024) 2023-12-19T18:18:36.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7fad14f640 1 -- 172.21.15.26:0/4093900989 learned_addr learned my addr 172.21.15.26:0/4093900989 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7fad950640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa815cf60 0x7f7fa80c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.316+0000 7f7fac94e640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fa80b88c0 0x7f7fa80c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7fad14f640 1 -- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fa80b88c0 msgr2=0x7f7fa80c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7fad14f640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fa80b88c0 0x7f7fa80c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7fad14f640 1 -- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa815cf60 msgr2=0x7f7fa80c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7fad14f640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa815cf60 0x7f7fa80c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7fad14f640 1 -- 172.21.15.26:0/4093900989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa4009420 con 0x7f7fa80b4600 2023-12-19T18:18:36.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7fac94e640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fa80b88c0 0x7f7fa80c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7fad14f640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa80b4600 0x7f7fa815eb40 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4019ff0 tx=0x7f7fa401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7f9dffb640 1 -- 172.21.15.26:0/4093900989 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40b7620 con 0x7f7fa80b4600 2023-12-19T18:18:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7f9dffb640 1 -- 172.21.15.26:0/4093900989 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa40b7780 con 0x7f7fa80b4600 2023-12-19T18:18:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7faf3b3640 1 -- 172.21.15.26:0/4093900989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa815e0c0 con 0x7f7fa80b4600 2023-12-19T18:18:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7f9dffb640 1 -- 172.21.15.26:0/4093900989 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40b7b30 con 0x7f7fa80b4600 2023-12-19T18:18:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7faf3b3640 1 -- 172.21.15.26:0/4093900989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa80b8050 con 0x7f7fa80b4600 2023-12-19T18:18:36.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7faf3b3640 1 -- 172.21.15.26:0/4093900989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa80c70c0 con 0x7f7fa80b4600 2023-12-19T18:18:36.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.320+0000 7f7f9dffb640 1 -- 172.21.15.26:0/4093900989 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 141) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f7fa40b3040 con 0x7f7fa80b4600 2023-12-19T18:18:36.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.324+0000 7f7f9dffb640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f8c065bb0 0x7f7f8c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:36.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.324+0000 7f7f9dffb640 1 -- 172.21.15.26:0/4093900989 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6899+0+0 (secure 0 0 0) 0x7f7fa411dda0 con 0x7f7fa80b4600 2023-12-19T18:18:36.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.324+0000 7f7fac94e640 1 -- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f8c065bb0 msgr2=0x7f7f8c068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:36.323 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.324+0000 7f7fac94e640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f8c065bb0 0x7f7f8c068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:18:36.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.324+0000 7f7f9dffb640 1 -- 172.21.15.26:0/4093900989 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7fa40eb2e0 con 0x7f7fa80b4600 2023-12-19T18:18:36.350 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:36.347+0000 7f0b7011a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:18:36.409 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:36.407+0000 7f0b7011a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:18:36.432 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:36.432+0000 7f707bc9b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:18:36.463 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:36.459+0000 7f0b7011a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:18:36.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.484+0000 7f7faf3b3640 1 -- 172.21.15.26:0/4093900989 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7fa815df20 con 0x7f7fa80b4600 2023-12-19T18:18:36.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.484+0000 7f7f9dffb640 1 -- 172.21.15.26:0/4093900989 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v255) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7fa815df20 con 0x7f7fa80b4600 2023-12-19T18:18:36.485 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:18:36.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.488+0000 7f7faf3b3640 1 -- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f8c065bb0 msgr2=0x7f7f8c068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:18:36.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.488+0000 7f7faf3b3640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f8c065bb0 0x7f7f8c068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.488+0000 7f7faf3b3640 1 -- 172.21.15.26:0/4093900989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa80b4600 msgr2=0x7f7fa815eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:36.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.488+0000 7f7faf3b3640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa80b4600 0x7f7fa815eb40 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4019ff0 tx=0x7f7fa401ac00 comp rx=0 tx=0).stop 2023-12-19T18:18:36.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.488+0000 7f7faf3b3640 1 -- 172.21.15.26:0/4093900989 shutdown_connections 2023-12-19T18:18:36.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.488+0000 7f7faf3b3640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7f8c065bb0 0x7f7f8c068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.488+0000 7f7faf3b3640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7fa815cf60 0x7f7fa80c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.488+0000 7f7faf3b3640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7fa80b88c0 0x7f7fa80c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.488+0000 7f7faf3b3640 1 --2- 172.21.15.26:0/4093900989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa80b4600 0x7f7fa815eb40 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.488+0000 7f7faf3b3640 1 -- 172.21.15.26:0/4093900989 >> 172.21.15.26:0/4093900989 conn(0x7f7fa8065ee0 msgr2=0x7f7fa80cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:36.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.488+0000 7f7faf3b3640 1 -- 172.21.15.26:0/4093900989 shutdown_connections 2023-12-19T18:18:36.488 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.488+0000 7f7faf3b3640 1 -- 172.21.15.26:0/4093900989 wait complete. 2023-12-19T18:18:36.495 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:36.496+0000 7f707bc9b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:18:36.507 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:18:36.547 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:36.543+0000 7f0b7011a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:18:36.556 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:36.556+0000 7f707bc9b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:18:36.604 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:36.603+0000 7f0b7011a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:18:36.649 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:36.648+0000 7f707bc9b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:18:36.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.656+0000 7fa4def25640 1 Processor -- start 2023-12-19T18:18:36.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.656+0000 7fa4def25640 1 -- start start 2023-12-19T18:18:36.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.656+0000 7fa4def25640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa4d815cee0 0x7fa4d81608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:36.658 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.656+0000 7fa4def25640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa4d80b4d00 0x7fa4d80d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:36.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.656+0000 7fa4def25640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d81617f0 con 0x7fa4d80b85e0 2023-12-19T18:18:36.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.656+0000 7fa4def25640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d8160f50 con 0x7fa4d815cee0 2023-12-19T18:18:36.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.656+0000 7fa4def25640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d8161230 con 0x7fa4d80b4d00 2023-12-19T18:18:36.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.656+0000 7fa4d7fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa4d80b85e0 0x7fa4d80b45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34014/0 (socket says 172.21.15.26:34014) 2023-12-19T18:18:36.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.656+0000 7fa4d7fff640 1 -- 172.21.15.26:0/2297186612 learned_addr learned my addr 172.21.15.26:0/2297186612 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:36.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.656+0000 7fa4dccc1640 1 --2- 172.21.15.26:0/2297186612 >> v2:172.21.15.26:3300/0 conn(0x7fa4d815cee0 0x7fa4d81608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:36.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.656+0000 7fa4dd4c2640 1 --2- 172.21.15.26:0/2297186612 >> v2:172.21.15.160:3300/0 conn(0x7fa4d80b4d00 0x7fa4d80d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:36.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.660+0000 7fa4d77fe640 1 -- 172.21.15.26:0/2297186612 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891088503 0 0) 0x7fa4d81617f0 con 0x7fa4d80b85e0 2023-12-19T18:18:36.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.660+0000 7fa4d77fe640 1 -- 172.21.15.26:0/2297186612 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4c8003530 con 0x7fa4d80b85e0 2023-12-19T18:18:36.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.660+0000 7fa4dccc1640 1 -- 172.21.15.26:0/2297186612 >> v2:172.21.15.160:3300/0 conn(0x7fa4d80b4d00 msgr2=0x7fa4d80d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:36.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.660+0000 7fa4dccc1640 1 --2- 172.21.15.26:0/2297186612 >> v2:172.21.15.160:3300/0 conn(0x7fa4d80b4d00 0x7fa4d80d14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.660+0000 7fa4dccc1640 1 -- 172.21.15.26:0/2297186612 >> v1:172.21.15.26:6789/0 conn(0x7fa4d80b85e0 legacy=0x7fa4d80b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:36.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.660+0000 7fa4dccc1640 1 -- 172.21.15.26:0/2297186612 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d8161510 con 0x7fa4d815cee0 2023-12-19T18:18:36.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.660+0000 7fa4dd4c2640 1 --2- 172.21.15.26:0/2297186612 >> v2:172.21.15.160:3300/0 conn(0x7fa4d80b4d00 0x7fa4d80d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:36.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.660+0000 7fa4dccc1640 1 --2- 172.21.15.26:0/2297186612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d815cee0 0x7fa4d81608e0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fa4d808bc30 tx=0x7fa4cc01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974d065f26891cd9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:36.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.660+0000 7fa4d77fe640 1 -- 172.21.15.26:0/2297186612 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc0ab600 con 0x7fa4d815cee0 2023-12-19T18:18:36.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.660+0000 7fa4d77fe640 1 -- 172.21.15.26:0/2297186612 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4cc0a7050 con 0x7fa4d815cee0 2023-12-19T18:18:36.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.660+0000 7fa4d77fe640 1 -- 172.21.15.26:0/2297186612 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc0ab960 con 0x7fa4d815cee0 2023-12-19T18:18:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.660+0000 7fa4def25640 1 -- 172.21.15.26:0/2297186612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d815cee0 msgr2=0x7fa4d81608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.660+0000 7fa4def25640 1 --2- 172.21.15.26:0/2297186612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d815cee0 0x7fa4d81608e0 secure :-1 s=READY pgs=2398 cs=0 l=1 rev1=1 crypto rx=0x7fa4d808bc30 tx=0x7fa4cc01c340 comp rx=0 tx=0).stop 2023-12-19T18:18:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 -- 172.21.15.26:0/2297186612 shutdown_connections 2023-12-19T18:18:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 --2- 172.21.15.26:0/2297186612 >> v2:172.21.15.160:3300/0 conn(0x7fa4d80b4d00 0x7fa4d80d14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 --2- 172.21.15.26:0/2297186612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d815cee0 0x7fa4d81608e0 unknown :-1 s=CLOSED pgs=2398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 -- 172.21.15.26:0/2297186612 >> 172.21.15.26:0/2297186612 conn(0x7fa4d80cf5f0 msgr2=0x7fa4d80cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:36.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 -- 172.21.15.26:0/2297186612 shutdown_connections 2023-12-19T18:18:36.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 -- 172.21.15.26:0/2297186612 wait complete. 2023-12-19T18:18:36.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 Processor -- start 2023-12-19T18:18:36.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 -- start start 2023-12-19T18:18:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4d80b4d00 0x7fa4d80b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4d80b85e0 0x7fa4d80b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d815cee0 0x7fa4d815aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:36.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d80b7fa0 con 0x7fa4d815cee0 2023-12-19T18:18:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d815f070 con 0x7fa4d80b85e0 2023-12-19T18:18:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4def25640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d814eda0 con 0x7fa4d80b4d00 2023-12-19T18:18:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4dd4c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d815cee0 0x7fa4d815aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4dd4c2640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d815cee0 0x7fa4d815aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36044/0 (socket says 172.21.15.26:36044) 2023-12-19T18:18:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4dd4c2640 1 -- 172.21.15.26:0/1649905149 learned_addr learned my addr 172.21.15.26:0/1649905149 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4d7fff640 1 --2- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4d80b85e0 0x7fa4d80b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4dccc1640 1 --2- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4d80b4d00 0x7fa4d80b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4dd4c2640 1 -- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4d80b4d00 msgr2=0x7fa4d80b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4dd4c2640 1 --2- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4d80b4d00 0x7fa4d80b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4dd4c2640 1 -- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4d80b85e0 msgr2=0x7fa4d80b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4dd4c2640 1 --2- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4d80b85e0 0x7fa4d80b7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.664+0000 7fa4dd4c2640 1 -- 172.21.15.26:0/1649905149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d0009a70 con 0x7fa4d815cee0 2023-12-19T18:18:36.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.668+0000 7fa4dd4c2640 1 --2- 172.21.15.26:0/1649905149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d815cee0 0x7fa4d815aec0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000efc0 tx=0x7fa4d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:36.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.668+0000 7fa4d57fa640 1 -- 172.21.15.26:0/1649905149 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d000cdf0 con 0x7fa4d815cee0 2023-12-19T18:18:36.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.668+0000 7fa4d57fa640 1 -- 172.21.15.26:0/1649905149 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d000f040 con 0x7fa4d815cee0 2023-12-19T18:18:36.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.668+0000 7fa4d57fa640 1 -- 172.21.15.26:0/1649905149 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0003be0 con 0x7fa4d815cee0 2023-12-19T18:18:36.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.668+0000 7fa4def25640 1 -- 172.21.15.26:0/1649905149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4cc009420 con 0x7fa4d815cee0 2023-12-19T18:18:36.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.668+0000 7fa4def25640 1 -- 172.21.15.26:0/1649905149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d80cc460 con 0x7fa4d815cee0 2023-12-19T18:18:36.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.668+0000 7fa4def25640 1 -- 172.21.15.26:0/1649905149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d815f200 con 0x7fa4d815cee0 2023-12-19T18:18:36.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.668+0000 7fa4d57fa640 1 -- 172.21.15.26:0/1649905149 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 141) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fa4d0014480 con 0x7fa4d815cee0 2023-12-19T18:18:36.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.668+0000 7fa4d57fa640 1 --2- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa4bc065bb0 0x7fa4bc068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:36.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.668+0000 7fa4d57fa640 1 -- 172.21.15.26:0/1649905149 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6899+0+0 (secure 0 0 0) 0x7fa4d00842b0 con 0x7fa4d815cee0 2023-12-19T18:18:36.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.668+0000 7fa4dccc1640 1 -- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa4bc065bb0 msgr2=0x7fa4bc068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:36.670 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.668+0000 7fa4dccc1640 1 --2- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa4bc065bb0 0x7fa4bc068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:18:36.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.676+0000 7fa4d57fa640 1 -- 172.21.15.26:0/1649905149 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa4d00516e0 con 0x7fa4d815cee0 2023-12-19T18:18:36.689 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:36.687+0000 7f0b7011a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:18:36.714 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:36.712+0000 7f707bc9b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:18:36.745 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:36.743+0000 7f0b7011a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:18:36.804 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:36.804+0000 7f707bc9b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:18:36.837 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.836+0000 7fa4def25640 1 -- 172.21.15.26:0/1649905149 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa4d8074270 con 0x7fa4d815cee0 2023-12-19T18:18:36.838 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.836+0000 7fa4d57fa640 1 -- 172.21.15.26:0/1649905149 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v255) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4d0059e60 con 0x7fa4d815cee0 2023-12-19T18:18:36.838 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:18:36.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.840+0000 7fa4def25640 1 -- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa4bc065bb0 msgr2=0x7fa4bc068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:18:36.840 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.840+0000 7fa4def25640 1 --2- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa4bc065bb0 0x7fa4bc068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.840+0000 7fa4def25640 1 -- 172.21.15.26:0/1649905149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d815cee0 msgr2=0x7fa4d815aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:36.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.840+0000 7fa4def25640 1 --2- 172.21.15.26:0/1649905149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d815cee0 0x7fa4d815aec0 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000efc0 tx=0x7fa4d000c490 comp rx=0 tx=0).stop 2023-12-19T18:18:36.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.840+0000 7fa4def25640 1 -- 172.21.15.26:0/1649905149 shutdown_connections 2023-12-19T18:18:36.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.840+0000 7fa4def25640 1 --2- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa4bc065bb0 0x7fa4bc068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.840+0000 7fa4def25640 1 --2- 172.21.15.26:0/1649905149 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4d815cee0 0x7fa4d815aec0 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.840+0000 7fa4def25640 1 --2- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4d80b85e0 0x7fa4d80b7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.840+0000 7fa4def25640 1 --2- 172.21.15.26:0/1649905149 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa4d80b4d00 0x7fa4d80b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:36.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.840+0000 7fa4def25640 1 -- 172.21.15.26:0/1649905149 >> 172.21.15.26:0/1649905149 conn(0x7fa4d80cf5f0 msgr2=0x7fa4d80b2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:36.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.840+0000 7fa4def25640 1 -- 172.21.15.26:0/1649905149 shutdown_connections 2023-12-19T18:18:36.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:36.840+0000 7fa4def25640 1 -- 172.21.15.26:0/1649905149 wait complete. 2023-12-19T18:18:36.861 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:18:36.865 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:36.864+0000 7f707bc9b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:18:36.887 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:36.883+0000 7f0b7011a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:18:36.945 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:36.943+0000 7f0b7011a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:18:37.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7433525640 1 Processor -- start 2023-12-19T18:18:37.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7433525640 1 -- start start 2023-12-19T18:18:37.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7433525640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f742c0cee50 0x7f742c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7433525640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742c06c970 con 0x7f742c155030 2023-12-19T18:18:37.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7433525640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742c15b1f0 con 0x7f742c15f370 2023-12-19T18:18:37.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7433525640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742c06c790 con 0x7f742c0cee50 2023-12-19T18:18:37.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f74312c1640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f742c155030 0x7f742c06c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34022/0 (socket says 172.21.15.26:34022) 2023-12-19T18:18:37.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f74312c1640 1 -- 172.21.15.26:0/3613315894 learned_addr learned my addr 172.21.15.26:0/3613315894 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:37.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7431ac2640 1 --2- 172.21.15.26:0/3613315894 >> v2:172.21.15.26:3300/0 conn(0x7f742c0cee50 0x7f742c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7423fff640 1 -- 172.21.15.26:0/3613315894 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704104167 0 0) 0x7f742c06c970 con 0x7f742c155030 2023-12-19T18:18:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7423fff640 1 -- 172.21.15.26:0/3613315894 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7418003530 con 0x7f742c155030 2023-12-19T18:18:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7423fff640 1 -- 172.21.15.26:0/3613315894 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059274019 0 0) 0x7f742c15b1f0 con 0x7f742c15f370 2023-12-19T18:18:37.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7423fff640 1 -- 172.21.15.26:0/3613315894 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f742c06c970 con 0x7f742c15f370 2023-12-19T18:18:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7431ac2640 1 -- 172.21.15.26:0/3613315894 >> v1:172.21.15.160:6789/0 conn(0x7f742c15f370 legacy=0x7f742c0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7431ac2640 1 -- 172.21.15.26:0/3613315894 >> v1:172.21.15.26:6789/0 conn(0x7f742c155030 legacy=0x7f742c06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.011 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.008+0000 7f7431ac2640 1 -- 172.21.15.26:0/3613315894 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c15b4d0 con 0x7f742c0cee50 2023-12-19T18:18:37.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.012+0000 7f7431ac2640 1 --2- 172.21.15.26:0/3613315894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f742c0cee50 0x7f742c15bcd0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f7424009d90 tx=0x7f742401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee03771c933241e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.012+0000 7f7423fff640 1 -- 172.21.15.26:0/3613315894 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240a5a00 con 0x7f742c0cee50 2023-12-19T18:18:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.012+0000 7f7423fff640 1 -- 172.21.15.26:0/3613315894 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74240a5b60 con 0x7f742c0cee50 2023-12-19T18:18:37.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.012+0000 7f7423fff640 1 -- 172.21.15.26:0/3613315894 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240ad440 con 0x7f742c0cee50 2023-12-19T18:18:37.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.012+0000 7f7433525640 1 -- 172.21.15.26:0/3613315894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f742c0cee50 msgr2=0x7f742c15bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.012+0000 7f7433525640 1 --2- 172.21.15.26:0/3613315894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f742c0cee50 0x7f742c15bcd0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f7424009d90 tx=0x7f742401c920 comp rx=0 tx=0).stop 2023-12-19T18:18:37.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.012+0000 7f7433525640 1 -- 172.21.15.26:0/3613315894 shutdown_connections 2023-12-19T18:18:37.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.012+0000 7f7433525640 1 --2- 172.21.15.26:0/3613315894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f742c0cee50 0x7f742c15bcd0 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.012+0000 7f7433525640 1 -- 172.21.15.26:0/3613315894 >> 172.21.15.26:0/3613315894 conn(0x7f742c0658e0 msgr2=0x7f742c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:37.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.012+0000 7f7433525640 1 -- 172.21.15.26:0/3613315894 shutdown_connections 2023-12-19T18:18:37.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.012+0000 7f7433525640 1 -- 172.21.15.26:0/3613315894 wait complete. 2023-12-19T18:18:37.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7433525640 1 Processor -- start 2023-12-19T18:18:37.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7433525640 1 -- start start 2023-12-19T18:18:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7433525640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f742c0cee50 0x7f742c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7433525640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f742c155030 0x7f742c1483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7433525640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f742c15f370 0x7f742c1488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7433525640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c145b20 con 0x7f742c155030 2023-12-19T18:18:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7433525640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c0cbf70 con 0x7f742c15f370 2023-12-19T18:18:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7433525640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c0cb1f0 con 0x7f742c0cee50 2023-12-19T18:18:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7430ac0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f742c155030 0x7f742c1483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7430ac0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f742c155030 0x7f742c1483a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36076/0 (socket says 172.21.15.26:36076) 2023-12-19T18:18:37.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7430ac0640 1 -- 172.21.15.26:0/2172155151 learned_addr learned my addr 172.21.15.26:0/2172155151 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f74312c1640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f742c0cee50 0x7f742c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7431ac2640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f742c15f370 0x7f742c1488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7430ac0640 1 -- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f742c0cee50 msgr2=0x7f742c0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7430ac0640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f742c0cee50 0x7f742c0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7430ac0640 1 -- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f742c15f370 msgr2=0x7f742c1488e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7430ac0640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f742c15f370 0x7f742c1488e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7430ac0640 1 -- 172.21.15.26:0/2172155151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7424009a70 con 0x7f742c155030 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f74312c1640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f742c0cee50 0x7f742c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7431ac2640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f742c15f370 0x7f742c1488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7430ac0640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f742c155030 0x7f742c1483a0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f741c00c430 tx=0x7f741c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7421ffb640 1 -- 172.21.15.26:0/2172155151 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c0052d0 con 0x7f742c155030 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7421ffb640 1 -- 172.21.15.26:0/2172155151 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c005450 con 0x7f742c155030 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7433525640 1 -- 172.21.15.26:0/2172155151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742c0b3800 con 0x7f742c155030 2023-12-19T18:18:37.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7421ffb640 1 -- 172.21.15.26:0/2172155151 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c003840 con 0x7f742c155030 2023-12-19T18:18:37.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.016+0000 7f7433525640 1 -- 172.21.15.26:0/2172155151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742c0cb490 con 0x7f742c155030 2023-12-19T18:18:37.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.020+0000 7f7433525640 1 -- 172.21.15.26:0/2172155151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742c074270 con 0x7f742c155030 2023-12-19T18:18:37.021 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:37.020+0000 7f707bc9b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:18:37.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.020+0000 7f7421ffb640 1 -- 172.21.15.26:0/2172155151 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 141) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f741c0039a0 con 0x7f742c155030 2023-12-19T18:18:37.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.020+0000 7f7421ffb640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7410065b80 0x7f7410068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.020+0000 7f7421ffb640 1 -- 172.21.15.26:0/2172155151 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6899+0+0 (secure 0 0 0) 0x7f741c082e80 con 0x7f742c155030 2023-12-19T18:18:37.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.020+0000 7f74312c1640 1 -- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7410065b80 msgr2=0x7f7410068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:37.022 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.020+0000 7f74312c1640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7410065b80 0x7f7410068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:18:37.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.028+0000 7f7421ffb640 1 -- 172.21.15.26:0/2172155151 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f741c0501e0 con 0x7f742c155030 2023-12-19T18:18:37.085 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:37.084+0000 7f707bc9b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:18:37.184 INFO:tasks.ceph.mgr.x.smithi160.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:18:37.185 INFO:tasks.ceph.mgr.x.smithi160.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:18:37.185 INFO:tasks.ceph.mgr.x.smithi160.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:18:37.192 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:37.191+0000 7f0b7011a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:18:37.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.196+0000 7f7433525640 1 -- 172.21.15.26:0/2172155151 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f742c0ccc30 con 0x7f742c155030 2023-12-19T18:18:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.200+0000 7f7421ffb640 1 -- 172.21.15.26:0/2172155151 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v255) v1 ==== 81+0+586 (secure 0 0 0) 0x7f741c017080 con 0x7f742c155030 2023-12-19T18:18:37.199 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1122697388 2023-12-20T18:18:31.899321+0000 2023-12-19T18:18:37.199 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1748568728 2023-12-20T18:18:31.899321+0000 2023-12-19T18:18:37.199 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2225649811 2023-12-20T18:18:31.899321+0000 2023-12-19T18:18:37.200 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/2614640962 2023-12-20T18:18:31.899321+0000 2023-12-19T18:18:37.200 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/1309045557 2023-12-20T18:18:31.899321+0000 2023-12-19T18:18:37.200 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3586017330 2023-12-20T18:18:31.899321+0000 2023-12-19T18:18:37.200 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6833/48394 2023-12-20T18:18:31.899321+0000 2023-12-19T18:18:37.200 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3350159660 2023-12-20T18:18:31.899321+0000 2023-12-19T18:18:37.200 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:6832/48394 2023-12-20T18:18:31.899321+0000 2023-12-19T18:18:37.200 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.160:0/3229222145 2023-12-20T18:18:31.899321+0000 2023-12-19T18:18:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:listed 10 entries 2023-12-19T18:18:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.200+0000 7f7433525640 1 -- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7410065b80 msgr2=0x7f7410068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:18:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.200+0000 7f7433525640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7410065b80 0x7f7410068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.200+0000 7f7433525640 1 -- 172.21.15.26:0/2172155151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f742c155030 msgr2=0x7f742c1483a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.200+0000 7f7433525640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f742c155030 0x7f742c1483a0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f741c00c430 tx=0x7f741c00c8f0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.200+0000 7f7433525640 1 -- 172.21.15.26:0/2172155151 shutdown_connections 2023-12-19T18:18:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.200+0000 7f7433525640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7410065b80 0x7f7410068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.200+0000 7f7433525640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f742c15f370 0x7f742c1488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.200+0000 7f7433525640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f742c155030 0x7f742c1483a0 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.200+0000 7f7433525640 1 --2- 172.21.15.26:0/2172155151 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f742c0cee50 0x7f742c0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.200+0000 7f7433525640 1 -- 172.21.15.26:0/2172155151 >> 172.21.15.26:0/2172155151 conn(0x7f742c0658e0 msgr2=0x7f742c070e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.200+0000 7f7433525640 1 -- 172.21.15.26:0/2172155151 shutdown_connections 2023-12-19T18:18:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.200+0000 7f7433525640 1 -- 172.21.15.26:0/2172155151 wait complete. 2023-12-19T18:18:37.221 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-19T18:18:37.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:37.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-19T18:18:37.277 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T18:18:37.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:37.277 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:18:37.295 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:18:37.295 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:37.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:18:37.305 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:18:37.305 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:37.328 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:37.327+0000 7f0b7011a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:18:37.339 INFO:tasks.ceph.mgr.y.smithi026.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-19T18:18:37.339 INFO:tasks.ceph.mgr.y.smithi026.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-19T18:18:37.339 INFO:tasks.ceph.mgr.y.smithi026.stderr: from numpy import show_config as show_numpy_config 2023-12-19T18:18:37.348 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:18:37.348 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:18:37.348 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:37.349 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:37.348+0000 7f707bc9b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:18:37.358 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:18:37.358 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:37.453 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:37.451+0000 7f0b7011a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:18:37.491 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:37.492+0000 7f707bc9b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:18:37.522 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:37.519+0000 7f0b7011a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:18:37.534 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.532+0000 7f32da9f3640 1 Processor -- start 2023-12-19T18:18:37.535 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.532+0000 7f32da9f3640 1 -- start start 2023-12-19T18:18:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32da9f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d4155030 0x7f32d415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32da9f3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d406c970 con 0x7f32d415f370 2023-12-19T18:18:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32da9f3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d415b1f0 con 0x7f32d40ce740 2023-12-19T18:18:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32da9f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d406c790 con 0x7f32d4155030 2023-12-19T18:18:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32d37fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f32d415f370 0x7f32d406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34028/0 (socket says 172.21.15.26:34028) 2023-12-19T18:18:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32d37fe640 1 -- 172.21.15.26:0/2748744499 learned_addr learned my addr 172.21.15.26:0/2748744499 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32d3fff640 1 --2- 172.21.15.26:0/2748744499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d4155030 0x7f32d415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32d2ffd640 1 -- 172.21.15.26:0/2748744499 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873036033 0 0) 0x7f32d406c970 con 0x7f32d415f370 2023-12-19T18:18:37.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32d2ffd640 1 -- 172.21.15.26:0/2748744499 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32bc003530 con 0x7f32d415f370 2023-12-19T18:18:37.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32d2ffd640 1 -- 172.21.15.26:0/2748744499 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57651573 0 0) 0x7f32d415b1f0 con 0x7f32d40ce740 2023-12-19T18:18:37.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32d2ffd640 1 -- 172.21.15.26:0/2748744499 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32d406c970 con 0x7f32d40ce740 2023-12-19T18:18:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32d3fff640 1 -- 172.21.15.26:0/2748744499 >> v1:172.21.15.160:6789/0 conn(0x7f32d40ce740 legacy=0x7f32d40ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32d3fff640 1 -- 172.21.15.26:0/2748744499 >> v1:172.21.15.26:6789/0 conn(0x7f32d415f370 legacy=0x7f32d406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.538 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.536+0000 7f32d3fff640 1 -- 172.21.15.26:0/2748744499 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d415b4d0 con 0x7f32d4155030 2023-12-19T18:18:37.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.540+0000 7f32d3fff640 1 --2- 172.21.15.26:0/2748744499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d4155030 0x7f32d415bcd0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f32c40097f0 tx=0x7f32c401c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362964d1568f13ea server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:37.540 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.540+0000 7f32d2ffd640 1 -- 172.21.15.26:0/2748744499 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c40aa610 con 0x7f32d4155030 2023-12-19T18:18:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.540+0000 7f32d2ffd640 1 -- 172.21.15.26:0/2748744499 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c40a6040 con 0x7f32d4155030 2023-12-19T18:18:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.540+0000 7f32d2ffd640 1 -- 172.21.15.26:0/2748744499 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c40aa990 con 0x7f32d4155030 2023-12-19T18:18:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.540+0000 7f32da9f3640 1 -- 172.21.15.26:0/2748744499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d4155030 msgr2=0x7f32d415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.540+0000 7f32da9f3640 1 --2- 172.21.15.26:0/2748744499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d4155030 0x7f32d415bcd0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f32c40097f0 tx=0x7f32c401c490 comp rx=0 tx=0).stop 2023-12-19T18:18:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.540+0000 7f32da9f3640 1 -- 172.21.15.26:0/2748744499 shutdown_connections 2023-12-19T18:18:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.540+0000 7f32da9f3640 1 --2- 172.21.15.26:0/2748744499 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d4155030 0x7f32d415bcd0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.540+0000 7f32da9f3640 1 -- 172.21.15.26:0/2748744499 >> 172.21.15.26:0/2748744499 conn(0x7f32d40658e0 msgr2=0x7f32d4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.540+0000 7f32da9f3640 1 -- 172.21.15.26:0/2748744499 shutdown_connections 2023-12-19T18:18:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.540+0000 7f32da9f3640 1 -- 172.21.15.26:0/2748744499 wait complete. 2023-12-19T18:18:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32da9f3640 1 Processor -- start 2023-12-19T18:18:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32da9f3640 1 -- start start 2023-12-19T18:18:37.544 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32da9f3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d40ce740 0x7f32d40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32da9f3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d4155030 0x7f32d4152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32da9f3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d415f370 0x7f32d41469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32da9f3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d40afae0 con 0x7f32d415f370 2023-12-19T18:18:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32da9f3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d4153300 con 0x7f32d4155030 2023-12-19T18:18:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32da9f3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d4147580 con 0x7f32d40ce740 2023-12-19T18:18:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d8f90640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d415f370 0x7f32d41469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d3fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d40ce740 0x7f32d40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d8f90640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d415f370 0x7f32d41469e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36078/0 (socket says 172.21.15.26:36078) 2023-12-19T18:18:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d8f90640 1 -- 172.21.15.26:0/589314799 learned_addr learned my addr 172.21.15.26:0/589314799 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d3fff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d40ce740 0x7f32d40aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:50890/0 (socket says 172.21.15.26:50890) 2023-12-19T18:18:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d37fe640 1 --2- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d4155030 0x7f32d4152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d8f90640 1 -- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d40ce740 msgr2=0x7f32d40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d8f90640 1 --2- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d40ce740 0x7f32d40aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.546 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d8f90640 1 -- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d4155030 msgr2=0x7f32d4152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d8f90640 1 --2- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d4155030 0x7f32d4152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d8f90640 1 -- 172.21.15.26:0/589314799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c4009420 con 0x7f32d415f370 2023-12-19T18:18:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d8f90640 1 --2- 172.21.15.26:0/589314799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d415f370 0x7f32d41469e0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00c730 tx=0x7f32cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d0ff9640 1 -- 172.21.15.26:0/589314799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc003b80 con 0x7f32d415f370 2023-12-19T18:18:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d0ff9640 1 -- 172.21.15.26:0/589314799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc003ce0 con 0x7f32d415f370 2023-12-19T18:18:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32da9f3640 1 -- 172.21.15.26:0/589314799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d40aeed0 con 0x7f32d415f370 2023-12-19T18:18:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32d0ff9640 1 -- 172.21.15.26:0/589314799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc00aa90 con 0x7f32d415f370 2023-12-19T18:18:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.544+0000 7f32da9f3640 1 -- 172.21.15.26:0/589314799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d40af490 con 0x7f32d415f370 2023-12-19T18:18:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.548+0000 7f32da9f3640 1 -- 172.21.15.26:0/589314799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d4074270 con 0x7f32d415f370 2023-12-19T18:18:37.548 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.548+0000 7f32d0ff9640 1 -- 172.21.15.26:0/589314799 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 141) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f32cc00abf0 con 0x7f32d415f370 2023-12-19T18:18:37.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.548+0000 7f32d0ff9640 1 --2- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 0x7f32c0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.548+0000 7f32d3fff640 1 -- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 msgr2=0x7f32c0068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:37.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.548+0000 7f32d0ff9640 1 -- 172.21.15.26:0/589314799 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6899+0+0 (secure 0 0 0) 0x7f32cc083520 con 0x7f32d415f370 2023-12-19T18:18:37.549 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.548+0000 7f32d3fff640 1 --2- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 0x7f32c0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:18:37.553 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.552+0000 7f32d0ff9640 1 -- 172.21.15.26:0/589314799 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f32cc050930 con 0x7f32d415f370 2023-12-19T18:18:37.631 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:37.628+0000 7f707bc9b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:18:37.661 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:37.659+0000 7f0b7011a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:18:37.706 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:37.704+0000 7f707bc9b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:18:37.750 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.748+0000 7f32da9f3640 1 -- 172.21.15.26:0/589314799 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32d40ccc30 con 0x7f32d415f370 2023-12-19T18:18:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32d0ff9640 1 -- 172.21.15.26:0/589314799 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4728 (secure 0 0 0) 0x7f32cc053f50 con 0x7f32d415f370 2023-12-19T18:18:37.751 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:37.751 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":128,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/282680544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":282680544},{"type":"v1","addr":"172.21.15.160:6836","nonce":282680544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3302634175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3302634175},{"type":"v1","addr":"172.21.15.160:6837","nonce":3302634175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3434994961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3434994961},{"type":"v1","addr":"172.21.15.26:6834","nonce":3434994961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:18:19.555514+0000","modified":"2023-12-19T18:18:22.569161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1019381645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1019381645},{"type":"v1","addr":"172.21.15.26:6835","nonce":1019381645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-19T18:18:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 128 2023-12-19T18:18:37.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32d3fff640 1 -- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 msgr2=0x7f32c0068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32d3fff640 1 --2- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 0x7f32c0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-19T18:18:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32da9f3640 1 -- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 msgr2=0x7f32c0068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:18:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32da9f3640 1 --2- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 0x7f32c0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32da9f3640 1 -- 172.21.15.26:0/589314799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d415f370 msgr2=0x7f32d41469e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32da9f3640 1 --2- 172.21.15.26:0/589314799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d415f370 0x7f32d41469e0 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00c730 tx=0x7f32cc00cbf0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32da9f3640 1 -- 172.21.15.26:0/589314799 shutdown_connections 2023-12-19T18:18:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32da9f3640 1 --2- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f32c0065b80 0x7f32c0068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32da9f3640 1 --2- 172.21.15.26:0/589314799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32d415f370 0x7f32d41469e0 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32da9f3640 1 --2- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32d4155030 0x7f32d4152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32da9f3640 1 --2- 172.21.15.26:0/589314799 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f32d40ce740 0x7f32d40aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32da9f3640 1 -- 172.21.15.26:0/589314799 >> 172.21.15.26:0/589314799 conn(0x7f32d40658e0 msgr2=0x7f32d406b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32da9f3640 1 -- 172.21.15.26:0/589314799 shutdown_connections 2023-12-19T18:18:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.752+0000 7f32da9f3640 1 -- 172.21.15.26:0/589314799 wait complete. 2023-12-19T18:18:37.773 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:37.851 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:37.852+0000 7f707bc9b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:18:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.916+0000 7f6d0162d640 1 Processor -- start 2023-12-19T18:18:37.918 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.916+0000 7f6d0162d640 1 -- start start 2023-12-19T18:18:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6d0162d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cfc0d1290 0x7f6cfc0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.919 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6d0162d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6cfc0cd330 0x7f6cfc0cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6d0162d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cfc06b790 con 0x7f6cfc06afd0 2023-12-19T18:18:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6d0162d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cfc06abb0 con 0x7f6cfc0cd330 2023-12-19T18:18:37.920 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6d0162d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc0cdd00 con 0x7f6cfc0d1290 2023-12-19T18:18:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6cfaffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cfc0d1290 0x7f6cfc0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6cf3fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f6cfc06afd0 0x7f6cfc0b2dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46500/0 (socket says 172.21.15.26:46500) 2023-12-19T18:18:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6cfb7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6cfc0cd330 0x7f6cfc0cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6cf3fff640 1 -- 172.21.15.26:0/817849978 learned_addr learned my addr 172.21.15.26:0/817849978 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6cfa7fc640 1 -- 172.21.15.26:0/817849978 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459048883 0 0) 0x7f6cfc06b790 con 0x7f6cfc06afd0 2023-12-19T18:18:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6cfa7fc640 1 -- 172.21.15.26:0/817849978 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ce8003530 con 0x7f6cfc06afd0 2023-12-19T18:18:37.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6cfaffd640 1 -- 172.21.15.26:0/817849978 >> v2:172.21.15.160:3300/0 conn(0x7f6cfc0cd330 msgr2=0x7f6cfc0cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6cfaffd640 1 --2- 172.21.15.26:0/817849978 >> v2:172.21.15.160:3300/0 conn(0x7f6cfc0cd330 0x7f6cfc0cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6cfaffd640 1 -- 172.21.15.26:0/817849978 >> v1:172.21.15.160:6789/0 conn(0x7f6cfc06afd0 legacy=0x7f6cfc0b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6cfaffd640 1 -- 172.21.15.26:0/817849978 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc06bce0 con 0x7f6cfc0d1290 2023-12-19T18:18:37.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.920+0000 7f6cfb7fe640 1 --2- 172.21.15.26:0/817849978 >> v2:172.21.15.160:3300/0 conn(0x7f6cfc0cd330 0x7f6cfc0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:37.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6cfaffd640 1 --2- 172.21.15.26:0/817849978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cfc0d1290 0x7f6cfc0b25d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0097c0 tx=0x7f6cec01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc57d0ce013f27a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6cfa7fc640 1 -- 172.21.15.26:0/817849978 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec0ac620 con 0x7f6cfc0d1290 2023-12-19T18:18:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6cfa7fc640 1 -- 172.21.15.26:0/817849978 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec0ac780 con 0x7f6cfc0d1290 2023-12-19T18:18:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6cfa7fc640 1 -- 172.21.15.26:0/817849978 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec0acb10 con 0x7f6cfc0d1290 2023-12-19T18:18:37.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6d0162d640 1 -- 172.21.15.26:0/817849978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cfc0d1290 msgr2=0x7f6cfc0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6d0162d640 1 --2- 172.21.15.26:0/817849978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cfc0d1290 0x7f6cfc0b25d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0097c0 tx=0x7f6cec01c3c0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6d0162d640 1 -- 172.21.15.26:0/817849978 shutdown_connections 2023-12-19T18:18:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6d0162d640 1 --2- 172.21.15.26:0/817849978 >> v2:172.21.15.160:3300/0 conn(0x7f6cfc0cd330 0x7f6cfc0cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6d0162d640 1 --2- 172.21.15.26:0/817849978 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cfc0d1290 0x7f6cfc0b25d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6d0162d640 1 -- 172.21.15.26:0/817849978 >> 172.21.15.26:0/817849978 conn(0x7f6cfc06d570 msgr2=0x7f6cfc06d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6d0162d640 1 -- 172.21.15.26:0/817849978 shutdown_connections 2023-12-19T18:18:37.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6d0162d640 1 -- 172.21.15.26:0/817849978 wait complete. 2023-12-19T18:18:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6d0162d640 1 Processor -- start 2023-12-19T18:18:37.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.924+0000 7f6d0162d640 1 -- start start 2023-12-19T18:18:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6d0162d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cfc06afd0 0x7f6cfc1478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6d0162d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cfc0cd330 0x7f6cfc0c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6d0162d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cfc0d1290 0x7f6cfc0c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6d0162d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc0af580 con 0x7f6cfc0cd330 2023-12-19T18:18:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6d0162d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc147e20 con 0x7f6cfc0d1290 2023-12-19T18:18:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6d0162d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc0ae970 con 0x7f6cfc06afd0 2023-12-19T18:18:37.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cf3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cfc0cd330 0x7f6cfc0c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cfaffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cfc06afd0 0x7f6cfc1478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cf3fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cfc0cd330 0x7f6cfc0c6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36092/0 (socket says 172.21.15.26:36092) 2023-12-19T18:18:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cf3fff640 1 -- 172.21.15.26:0/3436529837 learned_addr learned my addr 172.21.15.26:0/3436529837 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cfaffd640 1 --2- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cfc06afd0 0x7f6cfc1478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:50912/0 (socket says 172.21.15.26:50912) 2023-12-19T18:18:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cfb7fe640 1 --2- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cfc0d1290 0x7f6cfc0c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cf3fff640 1 -- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cfc06afd0 msgr2=0x7f6cfc1478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cf3fff640 1 --2- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cfc06afd0 0x7f6cfc1478e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cf3fff640 1 -- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cfc0d1290 msgr2=0x7f6cfc0c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cf3fff640 1 --2- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cfc0d1290 0x7f6cfc0c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cf3fff640 1 -- 172.21.15.26:0/3436529837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf4009a70 con 0x7f6cfc0cd330 2023-12-19T18:18:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cf3fff640 1 --2- 172.21.15.26:0/3436529837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cfc0cd330 0x7f6cfc0c6bd0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c420 tx=0x7f6ce400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:37.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cf37fe640 1 -- 172.21.15.26:0/3436529837 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4005cf0 con 0x7f6cfc0cd330 2023-12-19T18:18:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cf37fe640 1 -- 172.21.15.26:0/3436529837 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce4005e50 con 0x7f6cfc0cd330 2023-12-19T18:18:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6cf37fe640 1 -- 172.21.15.26:0/3436529837 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40072d0 con 0x7f6cfc0cd330 2023-12-19T18:18:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6d0162d640 1 -- 172.21.15.26:0/3436529837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec009420 con 0x7f6cfc0cd330 2023-12-19T18:18:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6d0162d640 1 -- 172.21.15.26:0/3436529837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cfc0aef30 con 0x7f6cfc0cd330 2023-12-19T18:18:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.928+0000 7f6d0162d640 1 -- 172.21.15.26:0/3436529837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cfc074270 con 0x7f6cfc0cd330 2023-12-19T18:18:37.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.932+0000 7f6cf37fe640 1 -- 172.21.15.26:0/3436529837 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 141) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f6ce4007430 con 0x7f6cfc0cd330 2023-12-19T18:18:37.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.932+0000 7f6cf37fe640 1 --2- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6cdc065b80 0x7f6cdc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.932+0000 7f6cf37fe640 1 -- 172.21.15.26:0/3436529837 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6899+0+0 (secure 0 0 0) 0x7f6ce4082c70 con 0x7f6cfc0cd330 2023-12-19T18:18:37.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.932+0000 7f6cfaffd640 1 -- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6cdc065b80 msgr2=0x7f6cdc068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:37.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.932+0000 7f6cfaffd640 1 --2- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6cdc065b80 0x7f6cdc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:18:37.936 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:37.936+0000 7f6cf37fe640 1 -- 172.21.15.26:0/3436529837 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6ce4050090 con 0x7f6cfc0cd330 2023-12-19T18:18:38.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.096+0000 7f6d0162d640 1 -- 172.21.15.26:0/3436529837 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cfc069e30 con 0x7f6cfc0cd330 2023-12-19T18:18:38.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.096+0000 7f6cf37fe640 1 -- 172.21.15.26:0/3436529837 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v255) v1 ==== 74+0+17830 (secure 0 0 0) 0x7f6ce4017020 con 0x7f6cfc0cd330 2023-12-19T18:18:38.098 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:38.098 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":255,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:18:31.899356+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:17.314988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:18.558040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1122697388":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/1748568728":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/2225649811":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/2614640962":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/1309045557":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3586017330":"2023-12-20T18:18:31.899321+0000","172.21.15.160:6833/48394":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3350159660":"2023-12-20T18:18:31.899321+0000","172.21.15.160:6832/48394":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3229222145":"2023-12-20T18:18:31.899321+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:38.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.100+0000 7f6d0162d640 1 -- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6cdc065b80 msgr2=0x7f6cdc068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:18:38.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.100+0000 7f6d0162d640 1 --2- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6cdc065b80 0x7f6cdc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.100+0000 7f6d0162d640 1 -- 172.21.15.26:0/3436529837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cfc0cd330 msgr2=0x7f6cfc0c6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.100+0000 7f6d0162d640 1 --2- 172.21.15.26:0/3436529837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cfc0cd330 0x7f6cfc0c6bd0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c420 tx=0x7f6ce400c820 comp rx=0 tx=0).stop 2023-12-19T18:18:38.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.100+0000 7f6d0162d640 1 -- 172.21.15.26:0/3436529837 shutdown_connections 2023-12-19T18:18:38.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.100+0000 7f6d0162d640 1 --2- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6cdc065b80 0x7f6cdc068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.100+0000 7f6d0162d640 1 --2- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cfc0d1290 0x7f6cfc0c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.100+0000 7f6d0162d640 1 --2- 172.21.15.26:0/3436529837 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cfc0cd330 0x7f6cfc0c6bd0 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.100+0000 7f6d0162d640 1 --2- 172.21.15.26:0/3436529837 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6cfc06afd0 0x7f6cfc1478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.100+0000 7f6d0162d640 1 -- 172.21.15.26:0/3436529837 >> 172.21.15.26:0/3436529837 conn(0x7f6cfc06d570 msgr2=0x7f6cfc06d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:38.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.100+0000 7f6d0162d640 1 -- 172.21.15.26:0/3436529837 shutdown_connections 2023-12-19T18:18:38.101 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.100+0000 7f6d0162d640 1 -- 172.21.15.26:0/3436529837 wait complete. 2023-12-19T18:18:38.121 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T16:42:15.699170+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:18:17.314988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '253', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:18:18.558040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '254', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:18:38.121 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:18:38.198 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.195+0000 7f0b7011a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:18:38.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.268+0000 7f4456539640 1 Processor -- start 2023-12-19T18:18:38.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.268+0000 7f4456539640 1 -- start start 2023-12-19T18:18:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f4456539640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f445015a9c0 0x7f44500706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f4456539640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f445015ada0 0x7f44500b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f4456539640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f445006ac80 0x7f445006b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f4456539640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44500cc270 con 0x7f445015ada0 2023-12-19T18:18:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f4456539640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44500b46c0 con 0x7f445006ac80 2023-12-19T18:18:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f4456539640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4450070d90 con 0x7f445015a9c0 2023-12-19T18:18:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f444f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f445015ada0 0x7f44500b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f444f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f445015ada0 0x7f44500b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36100/0 (socket says 172.21.15.26:36100) 2023-12-19T18:18:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f444f7fe640 1 -- 172.21.15.26:0/2684766011 learned_addr learned my addr 172.21.15.26:0/2684766011 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f4454ad6640 1 --2- 172.21.15.26:0/2684766011 >> v2:172.21.15.160:3300/0 conn(0x7f445006ac80 0x7f445006b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f444ffff640 1 --2- 172.21.15.26:0/2684766011 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f445015a9c0 0x7f44500706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f444f7fe640 1 -- 172.21.15.26:0/2684766011 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f445015a9c0 msgr2=0x7f44500706d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f444f7fe640 1 --2- 172.21.15.26:0/2684766011 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f445015a9c0 0x7f44500706d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f444f7fe640 1 -- 172.21.15.26:0/2684766011 >> v2:172.21.15.160:3300/0 conn(0x7f445006ac80 msgr2=0x7f445006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f444f7fe640 1 --2- 172.21.15.26:0/2684766011 >> v2:172.21.15.160:3300/0 conn(0x7f445006ac80 0x7f445006b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.272+0000 7f444f7fe640 1 -- 172.21.15.26:0/2684766011 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4450071070 con 0x7f445015ada0 2023-12-19T18:18:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f444f7fe640 1 --2- 172.21.15.26:0/2684766011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f445015ada0 0x7f44500b4180 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f44500c5700 tx=0x7f444401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d9e45f68dba5ba server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f444effd640 1 -- 172.21.15.26:0/2684766011 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44440ab600 con 0x7f445015ada0 2023-12-19T18:18:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f444effd640 1 -- 172.21.15.26:0/2684766011 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44440ab760 con 0x7f445015ada0 2023-12-19T18:18:38.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f444effd640 1 -- 172.21.15.26:0/2684766011 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44440abaf0 con 0x7f445015ada0 2023-12-19T18:18:38.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f4456539640 1 -- 172.21.15.26:0/2684766011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f445015ada0 msgr2=0x7f44500b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f4456539640 1 --2- 172.21.15.26:0/2684766011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f445015ada0 0x7f44500b4180 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f44500c5700 tx=0x7f444401c440 comp rx=0 tx=0).stop 2023-12-19T18:18:38.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f4456539640 1 -- 172.21.15.26:0/2684766011 shutdown_connections 2023-12-19T18:18:38.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f4456539640 1 --2- 172.21.15.26:0/2684766011 >> v2:172.21.15.160:3300/0 conn(0x7f445006ac80 0x7f445006b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f4456539640 1 --2- 172.21.15.26:0/2684766011 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f445015ada0 0x7f44500b4180 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f4456539640 1 --2- 172.21.15.26:0/2684766011 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f445015a9c0 0x7f44500706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f4456539640 1 -- 172.21.15.26:0/2684766011 >> 172.21.15.26:0/2684766011 conn(0x7f44500ce730 msgr2=0x7f44500ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:38.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f4456539640 1 -- 172.21.15.26:0/2684766011 shutdown_connections 2023-12-19T18:18:38.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f4456539640 1 -- 172.21.15.26:0/2684766011 wait complete. 2023-12-19T18:18:38.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f4456539640 1 Processor -- start 2023-12-19T18:18:38.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.276+0000 7f4456539640 1 -- start start 2023-12-19T18:18:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f4456539640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f445006ac80 0x7f445014b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f4456539640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f445015a9c0 0x7f44501548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f4456539640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f445015ada0 0x7f445006b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f4456539640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445014e110 con 0x7f445006ac80 2023-12-19T18:18:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f4456539640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44500c6ef0 con 0x7f445015a9c0 2023-12-19T18:18:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f4456539640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44500c6930 con 0x7f445015ada0 2023-12-19T18:18:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f445006ac80 0x7f445014b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f445006ac80 0x7f445014b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36114/0 (socket says 172.21.15.26:36114) 2023-12-19T18:18:38.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444ffff640 1 -- 172.21.15.26:0/534118872 learned_addr learned my addr 172.21.15.26:0/534118872 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:38.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444f7fe640 1 --2- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f445015a9c0 0x7f44501548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f4454ad6640 1 --2- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f445015ada0 0x7f445006b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444ffff640 1 -- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f445015ada0 msgr2=0x7f445006b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444ffff640 1 --2- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f445015ada0 0x7f445006b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444ffff640 1 -- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f445015a9c0 msgr2=0x7f44501548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444ffff640 1 --2- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f445015a9c0 0x7f44501548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.281 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444ffff640 1 -- 172.21.15.26:0/534118872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44440094d0 con 0x7f445006ac80 2023-12-19T18:18:38.282 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444ffff640 1 --2- 172.21.15.26:0/534118872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f445006ac80 0x7f445014b700 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f444000e930 tx=0x7f444000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:38.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444cff9640 1 -- 172.21.15.26:0/534118872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440009690 con 0x7f445006ac80 2023-12-19T18:18:38.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444cff9640 1 -- 172.21.15.26:0/534118872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444000ca00 con 0x7f445006ac80 2023-12-19T18:18:38.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444cff9640 1 -- 172.21.15.26:0/534118872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444000cd70 con 0x7f445006ac80 2023-12-19T18:18:38.283 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f4456539640 1 -- 172.21.15.26:0/534118872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44500c6c10 con 0x7f445006ac80 2023-12-19T18:18:38.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f4456539640 1 -- 172.21.15.26:0/534118872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445014bc40 con 0x7f445006ac80 2023-12-19T18:18:38.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f4456539640 1 -- 172.21.15.26:0/534118872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445014dbe0 con 0x7f445006ac80 2023-12-19T18:18:38.284 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.280+0000 7f444cff9640 1 -- 172.21.15.26:0/534118872 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 141) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f4440013470 con 0x7f445006ac80 2023-12-19T18:18:38.284 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.279+0000 7f0b7011a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:18:38.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.284+0000 7f444cff9640 1 --2- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f443c065b80 0x7f443c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.284+0000 7f444cff9640 1 -- 172.21.15.26:0/534118872 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6899+0+0 (secure 0 0 0) 0x7f44400839d0 con 0x7f445006ac80 2023-12-19T18:18:38.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.284+0000 7f444f7fe640 1 -- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f443c065b80 msgr2=0x7f443c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:38.285 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.284+0000 7f444f7fe640 1 --2- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f443c065b80 0x7f443c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:18:38.287 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.288+0000 7f444cff9640 1 -- 172.21.15.26:0/534118872 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4440050ea0 con 0x7f445006ac80 2023-12-19T18:18:38.431 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.427+0000 7f0b7011a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:18:38.442 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:38.440+0000 7f707bc9b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:18:38.455 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f4456539640 1 -- 172.21.15.26:0/534118872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4450074270 con 0x7f445006ac80 2023-12-19T18:18:38.456 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f444cff9640 1 -- 172.21.15.26:0/534118872 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v255) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44400544c0 con 0x7f445006ac80 2023-12-19T18:18:38.456 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2023-12-19T18:18:38.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f4456539640 1 -- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f443c065b80 msgr2=0x7f443c068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:18:38.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f4456539640 1 --2- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f443c065b80 0x7f443c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f4456539640 1 -- 172.21.15.26:0/534118872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f445006ac80 msgr2=0x7f445014b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f4456539640 1 --2- 172.21.15.26:0/534118872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f445006ac80 0x7f445014b700 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f444000e930 tx=0x7f444000edf0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f4456539640 1 -- 172.21.15.26:0/534118872 shutdown_connections 2023-12-19T18:18:38.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f4456539640 1 --2- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f443c065b80 0x7f443c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f4456539640 1 --2- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f445015ada0 0x7f445006b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f4456539640 1 --2- 172.21.15.26:0/534118872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f445015a9c0 0x7f44501548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f4456539640 1 --2- 172.21.15.26:0/534118872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f445006ac80 0x7f445014b700 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f4456539640 1 -- 172.21.15.26:0/534118872 >> 172.21.15.26:0/534118872 conn(0x7f44500ce730 msgr2=0x7f44500ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:38.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f4456539640 1 -- 172.21.15.26:0/534118872 shutdown_connections 2023-12-19T18:18:38.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.456+0000 7f4456539640 1 -- 172.21.15.26:0/534118872 wait complete. 2023-12-19T18:18:38.479 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:18:38.514 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.511+0000 7f0b7011a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:18:38.534 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:38.532+0000 7f707bc9b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:18:38.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.624+0000 7effc122a640 1 Processor -- start 2023-12-19T18:18:38.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.624+0000 7effc122a640 1 -- start start 2023-12-19T18:18:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effc122a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effbc15cc90 0x7effbc0cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effc122a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7effbc15d070 0x7effbc0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effc122a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effbc0cdb20 con 0x7effbc160a20 2023-12-19T18:18:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effc122a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effbc0cde00 con 0x7effbc15d070 2023-12-19T18:18:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effc122a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc0ce0e0 con 0x7effbc15cc90 2023-12-19T18:18:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effc0a29640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7effbc160a20 0x7effbc0b45a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34036/0 (socket says 172.21.15.26:34036) 2023-12-19T18:18:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effc0a29640 1 -- 172.21.15.26:0/3261725485 learned_addr learned my addr 172.21.15.26:0/3261725485 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:38.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effbb7fe640 1 --2- 172.21.15.26:0/3261725485 >> v2:172.21.15.160:3300/0 conn(0x7effbc15d070 0x7effbc0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effbbfff640 1 --2- 172.21.15.26:0/3261725485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effbc15cc90 0x7effbc0cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effbaffd640 1 -- 172.21.15.26:0/3261725485 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779107993 0 0) 0x7effbc0cdb20 con 0x7effbc160a20 2023-12-19T18:18:38.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effbaffd640 1 -- 172.21.15.26:0/3261725485 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effa4003530 con 0x7effbc160a20 2023-12-19T18:18:38.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effbb7fe640 1 -- 172.21.15.26:0/3261725485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effbc15cc90 msgr2=0x7effbc0cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:38.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effbb7fe640 1 --2- 172.21.15.26:0/3261725485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effbc15cc90 0x7effbc0cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effbb7fe640 1 -- 172.21.15.26:0/3261725485 >> v1:172.21.15.26:6789/0 conn(0x7effbc160a20 legacy=0x7effbc0b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effbb7fe640 1 -- 172.21.15.26:0/3261725485 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc0ce3c0 con 0x7effbc15d070 2023-12-19T18:18:38.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.628+0000 7effbbfff640 1 --2- 172.21.15.26:0/3261725485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effbc15cc90 0x7effbc0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.632+0000 7effbb7fe640 1 --2- 172.21.15.26:0/3261725485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effbc15d070 0x7effbc0cf3c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7effb00097c0 tx=0x7effb001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbd31b92122561d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:38.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.632+0000 7effbaffd640 1 -- 172.21.15.26:0/3261725485 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb00a7070 con 0x7effbc15d070 2023-12-19T18:18:38.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.632+0000 7effbaffd640 1 -- 172.21.15.26:0/3261725485 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb00ab430 con 0x7effbc15d070 2023-12-19T18:18:38.634 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.632+0000 7effbaffd640 1 -- 172.21.15.26:0/3261725485 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb00ab7c0 con 0x7effbc15d070 2023-12-19T18:18:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.632+0000 7effc122a640 1 -- 172.21.15.26:0/3261725485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effbc15d070 msgr2=0x7effbc0cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.632+0000 7effc122a640 1 --2- 172.21.15.26:0/3261725485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effbc15d070 0x7effbc0cf3c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7effb00097c0 tx=0x7effb001c450 comp rx=0 tx=0).stop 2023-12-19T18:18:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.632+0000 7effc122a640 1 -- 172.21.15.26:0/3261725485 shutdown_connections 2023-12-19T18:18:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.632+0000 7effc122a640 1 --2- 172.21.15.26:0/3261725485 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effbc15d070 0x7effbc0cf3c0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.632+0000 7effc122a640 1 --2- 172.21.15.26:0/3261725485 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effbc15cc90 0x7effbc0cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.632+0000 7effc122a640 1 -- 172.21.15.26:0/3261725485 >> 172.21.15.26:0/3261725485 conn(0x7effbc06a320 msgr2=0x7effbc0658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.632+0000 7effc122a640 1 -- 172.21.15.26:0/3261725485 shutdown_connections 2023-12-19T18:18:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effc122a640 1 -- 172.21.15.26:0/3261725485 wait complete. 2023-12-19T18:18:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effc122a640 1 Processor -- start 2023-12-19T18:18:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effc122a640 1 -- start start 2023-12-19T18:18:38.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effc122a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effbc15cc90 0x7effbc152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.638 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effc122a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effbc15d070 0x7effbc151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effc122a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effbc160a20 0x7effbc157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effc122a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc15c590 con 0x7effbc15d070 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effc122a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc151280 con 0x7effbc15cc90 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effc122a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effbc15c2b0 con 0x7effbc160a20 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effbc15d070 0x7effbc151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effbbfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effbc15cc90 0x7effbc152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effbb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effbc15d070 0x7effbc151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36126/0 (socket says 172.21.15.26:36126) 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effbb7fe640 1 -- 172.21.15.26:0/1585616138 learned_addr learned my addr 172.21.15.26:0/1585616138 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effc0a29640 1 --2- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effbc160a20 0x7effbc157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effbbfff640 1 -- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effbc160a20 msgr2=0x7effbc157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effbbfff640 1 --2- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effbc160a20 0x7effbc157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effbbfff640 1 -- 172.21.15.26:0/1585616138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effbc15d070 msgr2=0x7effbc151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effbbfff640 1 --2- 172.21.15.26:0/1585616138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effbc15d070 0x7effbc151710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effbbfff640 1 -- 172.21.15.26:0/1585616138 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effac009420 con 0x7effbc15cc90 2023-12-19T18:18:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effbb7fe640 1 --2- 172.21.15.26:0/1585616138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effbc15d070 0x7effbc151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.636+0000 7effbbfff640 1 --2- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effbc15cc90 0x7effbc152a10 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7effac00e9b0 tx=0x7effac00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.640+0000 7effb8ff9640 1 -- 172.21.15.26:0/1585616138 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac00c9a0 con 0x7effbc15cc90 2023-12-19T18:18:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.640+0000 7effb8ff9640 1 -- 172.21.15.26:0/1585616138 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effac00cb00 con 0x7effbc15cc90 2023-12-19T18:18:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.640+0000 7effc122a640 1 -- 172.21.15.26:0/1585616138 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb0009420 con 0x7effbc15cc90 2023-12-19T18:18:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.640+0000 7effb8ff9640 1 -- 172.21.15.26:0/1585616138 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effac013470 con 0x7effbc15cc90 2023-12-19T18:18:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.640+0000 7effc122a640 1 -- 172.21.15.26:0/1585616138 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc150f10 con 0x7effbc15cc90 2023-12-19T18:18:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.640+0000 7effc122a640 1 -- 172.21.15.26:0/1585616138 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effbc061930 con 0x7effbc15cc90 2023-12-19T18:18:38.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.640+0000 7effb8ff9640 1 -- 172.21.15.26:0/1585616138 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 141) v1 ==== 85331+0+0 (secure 0 0 0) 0x7effac0059a0 con 0x7effbc15cc90 2023-12-19T18:18:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.640+0000 7effb8ff9640 1 --2- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7effa8065bb0 0x7effa8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.640+0000 7effb8ff9640 1 -- 172.21.15.26:0/1585616138 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6899+0+0 (secure 0 0 0) 0x7effac083a70 con 0x7effbc15cc90 2023-12-19T18:18:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.640+0000 7effbb7fe640 1 -- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7effa8065bb0 msgr2=0x7effa8068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.160:6832/48394 2023-12-19T18:18:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.640+0000 7effbb7fe640 1 --2- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7effa8065bb0 0x7effa8068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:18:38.645 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.644+0000 7effb8ff9640 1 -- 172.21.15.26:0/1585616138 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7effac050f90 con 0x7effbc15cc90 2023-12-19T18:18:38.699 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:38.696+0000 7f707bc9b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:18:38.788 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:38.788+0000 7f707bc9b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:18:38.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.804+0000 7effc122a640 1 -- 172.21.15.26:0/1585616138 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7effbc0d0970 con 0x7effbc15cc90 2023-12-19T18:18:38.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.804+0000 7effb8ff9640 1 -- 172.21.15.26:0/1585616138 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v255) v1 ==== 104+0+11 (secure 0 0 0) 0x7effac0170a0 con 0x7effbc15cc90 2023-12-19T18:18:38.805 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2023-12-19T18:18:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.808+0000 7effc122a640 1 -- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7effa8065bb0 msgr2=0x7effa8068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:18:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.808+0000 7effc122a640 1 --2- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7effa8065bb0 0x7effa8068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.808+0000 7effc122a640 1 -- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effbc15cc90 msgr2=0x7effbc152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.808+0000 7effc122a640 1 --2- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effbc15cc90 0x7effbc152a10 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7effac00e9b0 tx=0x7effac00ee70 comp rx=0 tx=0).stop 2023-12-19T18:18:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.808+0000 7effc122a640 1 -- 172.21.15.26:0/1585616138 shutdown_connections 2023-12-19T18:18:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.808+0000 7effc122a640 1 --2- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7effa8065bb0 0x7effa8068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.808+0000 7effc122a640 1 --2- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7effbc160a20 0x7effbc157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.808+0000 7effc122a640 1 --2- 172.21.15.26:0/1585616138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7effbc15d070 0x7effbc151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.808+0000 7effc122a640 1 --2- 172.21.15.26:0/1585616138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7effbc15cc90 0x7effbc152a10 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.808+0000 7effc122a640 1 -- 172.21.15.26:0/1585616138 >> 172.21.15.26:0/1585616138 conn(0x7effbc06a320 msgr2=0x7effbc158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.808+0000 7effc122a640 1 -- 172.21.15.26:0/1585616138 shutdown_connections 2023-12-19T18:18:38.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.808+0000 7effc122a640 1 -- 172.21.15.26:0/1585616138 wait complete. 2023-12-19T18:18:38.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:18:38.830 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.827+0000 7f0b7011a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:18:38.890 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.887+0000 7f0b7011a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:18:38.968 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.967+0000 7f0af8789640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:38.968 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.967+0000 7f0af8789640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:38.968 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.967+0000 7f0af8789640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:38.968 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.967+0000 7f0af8789640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:38.969 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.967+0000 7f0af8789640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:38.969 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.967+0000 7f0afa78d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:38.969 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.967+0000 7f0afa78d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:38.969 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.967+0000 7f0afa78d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:38.969 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.967+0000 7f0afa78d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:38.969 INFO:tasks.ceph.mgr.x.smithi160.stderr:2023-12-19T18:18:38.967+0000 7f0afa78d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-19T18:18:38.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.980+0000 7f911a391640 1 Processor -- start 2023-12-19T18:18:38.982 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.980+0000 7f911a391640 1 -- start start 2023-12-19T18:18:38.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.984+0000 7f911a391640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9114160a20 0x7f9114167d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.984+0000 7f911a391640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91140cdb20 con 0x7f911415d070 2023-12-19T18:18:38.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.984+0000 7f911a391640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91140cde00 con 0x7f911415cc90 2023-12-19T18:18:38.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.984+0000 7f911a391640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91140ce0e0 con 0x7f9114160a20 2023-12-19T18:18:38.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.984+0000 7f9118b8e640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f911415d070 0x7f91140cf3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34046/0 (socket says 172.21.15.26:34046) 2023-12-19T18:18:38.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.984+0000 7f9118b8e640 1 -- 172.21.15.26:0/3239223004 learned_addr learned my addr 172.21.15.26:0/3239223004 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:38.984 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.984+0000 7f9119b90640 1 --2- 172.21.15.26:0/3239223004 >> v2:172.21.15.26:3300/0 conn(0x7f9114160a20 0x7f9114167d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.984+0000 7f9119b90640 1 -- 172.21.15.26:0/3239223004 >> v1:172.21.15.160:6789/0 conn(0x7f911415cc90 legacy=0x7f91140cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.984+0000 7f9119b90640 1 -- 172.21.15.26:0/3239223004 >> v1:172.21.15.26:6789/0 conn(0x7f911415d070 legacy=0x7f91140cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.985 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.984+0000 7f9119b90640 1 -- 172.21.15.26:0/3239223004 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91140ce3c0 con 0x7f9114160a20 2023-12-19T18:18:38.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.988+0000 7f9119b90640 1 --2- 172.21.15.26:0/3239223004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9114160a20 0x7f9114167d40 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f910c009d90 tx=0x7f910c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5874d8495f6085 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:38.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.988+0000 7f910bfff640 1 -- 172.21.15.26:0/3239223004 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c0a5910 con 0x7f9114160a20 2023-12-19T18:18:38.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.988+0000 7f910bfff640 1 -- 172.21.15.26:0/3239223004 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910c0a5a70 con 0x7f9114160a20 2023-12-19T18:18:38.987 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.988+0000 7f910bfff640 1 -- 172.21.15.26:0/3239223004 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c0a5e00 con 0x7f9114160a20 2023-12-19T18:18:38.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.988+0000 7f911a391640 1 -- 172.21.15.26:0/3239223004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9114160a20 msgr2=0x7f9114167d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.988+0000 7f911a391640 1 --2- 172.21.15.26:0/3239223004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9114160a20 0x7f9114167d40 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f910c009d90 tx=0x7f910c01c920 comp rx=0 tx=0).stop 2023-12-19T18:18:38.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.988+0000 7f911a391640 1 -- 172.21.15.26:0/3239223004 shutdown_connections 2023-12-19T18:18:38.988 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.988+0000 7f911a391640 1 --2- 172.21.15.26:0/3239223004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9114160a20 0x7f9114167d40 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.988+0000 7f911a391640 1 -- 172.21.15.26:0/3239223004 >> 172.21.15.26:0/3239223004 conn(0x7f911406a320 msgr2=0x7f91140658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:38.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.988+0000 7f911a391640 1 -- 172.21.15.26:0/3239223004 shutdown_connections 2023-12-19T18:18:38.989 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.988+0000 7f911a391640 1 -- 172.21.15.26:0/3239223004 wait complete. 2023-12-19T18:18:38.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.988+0000 7f911a391640 1 Processor -- start 2023-12-19T18:18:38.990 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.988+0000 7f911a391640 1 -- start start 2023-12-19T18:18:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f911a391640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f911415cc90 0x7f91141502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f911a391640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f911415d070 0x7f91140b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f911a391640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9114160a20 0x7f9114155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f911a391640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91140b3e20 con 0x7f9114160a20 2023-12-19T18:18:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f911a391640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91140b34f0 con 0x7f911415cc90 2023-12-19T18:18:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f911a391640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91140b37d0 con 0x7f911415d070 2023-12-19T18:18:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9119b90640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9114160a20 0x7f9114155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f911938f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f911415cc90 0x7f91141502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9118b8e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f911415d070 0x7f91140b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:38.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f911938f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f911415cc90 0x7f91141502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:59510/0 (socket says 172.21.15.26:59510) 2023-12-19T18:18:38.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9119b90640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9114160a20 0x7f9114155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36150/0 (socket says 172.21.15.26:36150) 2023-12-19T18:18:38.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f911938f640 1 -- 172.21.15.26:0/2335974220 learned_addr learned my addr 172.21.15.26:0/2335974220 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:38.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9118b8e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f911415d070 0x7f91140b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:50984/0 (socket says 172.21.15.26:50984) 2023-12-19T18:18:38.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9118b8e640 1 -- 172.21.15.26:0/2335974220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f911415cc90 msgr2=0x7f91141502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9118b8e640 1 --2- 172.21.15.26:0/2335974220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f911415cc90 0x7f91141502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9118b8e640 1 -- 172.21.15.26:0/2335974220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9114160a20 msgr2=0x7f9114155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:38.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9118b8e640 1 --2- 172.21.15.26:0/2335974220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9114160a20 0x7f9114155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:38.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9118b8e640 1 -- 172.21.15.26:0/2335974220 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c009a70 con 0x7f911415d070 2023-12-19T18:18:38.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f911938f640 1 --2- 172.21.15.26:0/2335974220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f911415cc90 0x7f91141502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:38.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9119b90640 1 --2- 172.21.15.26:0/2335974220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9114160a20 0x7f9114155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:38.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9118b8e640 1 --2- 172.21.15.26:0/2335974220 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f911415d070 0x7f91140b42b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f910400c430 tx=0x7f910400c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:38.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9109ffb640 1 -- 172.21.15.26:0/2335974220 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104005cf0 con 0x7f911415d070 2023-12-19T18:18:38.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9109ffb640 1 -- 172.21.15.26:0/2335974220 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9104005e50 con 0x7f911415d070 2023-12-19T18:18:38.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f911a391640 1 -- 172.21.15.26:0/2335974220 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91140b3ab0 con 0x7f911415d070 2023-12-19T18:18:38.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9109ffb640 1 -- 172.21.15.26:0/2335974220 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91040072d0 con 0x7f911415d070 2023-12-19T18:18:38.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f911a391640 1 -- 172.21.15.26:0/2335974220 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91141531b0 con 0x7f911415d070 2023-12-19T18:18:38.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f911a391640 1 -- 172.21.15.26:0/2335974220 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91140b3f90 con 0x7f911415d070 2023-12-19T18:18:38.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.992+0000 7f9109ffb640 1 -- 172.21.15.26:0/2335974220 <== mon.2 v2:172.21.15.160:3301/0 4 ==== mgrmap(e 142) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f9104007430 con 0x7f911415d070 2023-12-19T18:18:38.995 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:38.996+0000 7f9109ffb640 1 -- 172.21.15.26:0/2335974220 <== mon.2 v2:172.21.15.160:3301/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 7157+0+0 (secure 0 0 0) 0x7f9104082760 con 0x7f911415d070 2023-12-19T18:18:38.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.000+0000 7f9109ffb640 1 -- 172.21.15.26:0/2335974220 <== mon.2 v2:172.21.15.160:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f910404ff80 con 0x7f911415d070 2023-12-19T18:18:39.144 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:39.144+0000 7f707bc9b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:18:39.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.160+0000 7f911a391640 1 -- 172.21.15.26:0/2335974220 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9114061930 con 0x7f911415d070 2023-12-19T18:18:39.160 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.160+0000 7f9109ffb640 1 -- 172.21.15.26:0/2335974220 <== mon.2 v2:172.21.15.160:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v256) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9104017020 con 0x7f911415d070 2023-12-19T18:18:39.160 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2023-12-19T18:18:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.160+0000 7f911a391640 1 -- 172.21.15.26:0/2335974220 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f911415d070 msgr2=0x7f91140b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.160+0000 7f911a391640 1 --2- 172.21.15.26:0/2335974220 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f911415d070 0x7f91140b42b0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f910400c430 tx=0x7f910400c8f0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.160+0000 7f911a391640 1 -- 172.21.15.26:0/2335974220 shutdown_connections 2023-12-19T18:18:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.160+0000 7f911a391640 1 --2- 172.21.15.26:0/2335974220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9114160a20 0x7f9114155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.160+0000 7f911a391640 1 --2- 172.21.15.26:0/2335974220 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f911415d070 0x7f91140b42b0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.160+0000 7f911a391640 1 --2- 172.21.15.26:0/2335974220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f911415cc90 0x7f91141502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.160+0000 7f911a391640 1 -- 172.21.15.26:0/2335974220 >> 172.21.15.26:0/2335974220 conn(0x7f911406a320 msgr2=0x7f91140c5920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.160+0000 7f911a391640 1 -- 172.21.15.26:0/2335974220 shutdown_connections 2023-12-19T18:18:39.163 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.160+0000 7f911a391640 1 -- 172.21.15.26:0/2335974220 wait complete. 2023-12-19T18:18:39.183 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:39.209 INFO:tasks.ceph.mgr.y.smithi026.stderr:2023-12-19T18:18:39.208+0000 7f707bc9b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:18:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.328+0000 7efeb801f640 1 Processor -- start 2023-12-19T18:18:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.328+0000 7efeb801f640 1 -- start start 2023-12-19T18:18:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb801f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb00cb1d0 0x7efeb00cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb801f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efeb00b1d40 0x7efeb00b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb801f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb006d4b0 con 0x7efeb006bee0 2023-12-19T18:18:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb801f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb00cc030 con 0x7efeb00b1d40 2023-12-19T18:18:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb801f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00cbaf0 con 0x7efeb00cb1d0 2023-12-19T18:18:39.332 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb65bc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efeb00b1d40 0x7efeb00b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb5dbb640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb00cb1d0 0x7efeb00cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb65bc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efeb00b1d40 0x7efeb00b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.26:59522/0 (socket says 172.21.15.26:59522) 2023-12-19T18:18:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb55ba640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7efeb006bee0 0x7efeb006c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46518/0 (socket says 172.21.15.26:46518) 2023-12-19T18:18:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb65bc640 1 -- 172.21.15.26:0/2800068915 learned_addr learned my addr 172.21.15.26:0/2800068915 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb65bc640 1 -- 172.21.15.26:0/2800068915 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb00cb1d0 msgr2=0x7efeb00cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb65bc640 1 --2- 172.21.15.26:0/2800068915 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb00cb1d0 0x7efeb00cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb65bc640 1 -- 172.21.15.26:0/2800068915 >> v1:172.21.15.160:6789/0 conn(0x7efeb006bee0 legacy=0x7efeb006c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb65bc640 1 -- 172.21.15.26:0/2800068915 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb006cc10 con 0x7efeb00b1d40 2023-12-19T18:18:39.333 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.332+0000 7efeb4db9640 1 -- 172.21.15.26:0/2800068915 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438431193 0 0) 0x7efeb006d4b0 con 0x7efeb006bee0 2023-12-19T18:18:39.335 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb65bc640 1 --2- 172.21.15.26:0/2800068915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00b1d40 0x7efeb00b21d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7efea8009e50 tx=0x7efea801c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=381524ca87f0fedc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:39.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb4db9640 1 -- 172.21.15.26:0/2800068915 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea80a5840 con 0x7efeb00b1d40 2023-12-19T18:18:39.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb4db9640 1 -- 172.21.15.26:0/2800068915 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea80a59a0 con 0x7efeb00b1d40 2023-12-19T18:18:39.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb4db9640 1 -- 172.21.15.26:0/2800068915 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea80a5d30 con 0x7efeb00b1d40 2023-12-19T18:18:39.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb801f640 1 -- 172.21.15.26:0/2800068915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00b1d40 msgr2=0x7efeb00b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:39.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb801f640 1 --2- 172.21.15.26:0/2800068915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00b1d40 0x7efeb00b21d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7efea8009e50 tx=0x7efea801c980 comp rx=0 tx=0).stop 2023-12-19T18:18:39.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb801f640 1 -- 172.21.15.26:0/2800068915 shutdown_connections 2023-12-19T18:18:39.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb801f640 1 --2- 172.21.15.26:0/2800068915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00b1d40 0x7efeb00b21d0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb801f640 1 --2- 172.21.15.26:0/2800068915 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb00cb1d0 0x7efeb00cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb801f640 1 -- 172.21.15.26:0/2800068915 >> 172.21.15.26:0/2800068915 conn(0x7efeb00658e0 msgr2=0x7efeb0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:39.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb801f640 1 -- 172.21.15.26:0/2800068915 shutdown_connections 2023-12-19T18:18:39.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb801f640 1 -- 172.21.15.26:0/2800068915 wait complete. 2023-12-19T18:18:39.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb801f640 1 Processor -- start 2023-12-19T18:18:39.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb801f640 1 -- start start 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb801f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb006bee0 0x7efeb0155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb801f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb00b1d40 0x7efeb00ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.336+0000 7efeb801f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00cb1d0 0x7efeb0153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb801f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0156bb0 con 0x7efeb00b1d40 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb801f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00ae7a0 con 0x7efeb00cb1d0 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb801f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb01572e0 con 0x7efeb006bee0 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb55ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb00b1d40 0x7efeb00ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb55ba640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb00b1d40 0x7efeb00ae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36158/0 (socket says 172.21.15.26:36158) 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb55ba640 1 -- 172.21.15.26:0/3001397995 learned_addr learned my addr 172.21.15.26:0/3001397995 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb65bc640 1 --2- 172.21.15.26:0/3001397995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00cb1d0 0x7efeb0153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb5dbb640 1 --2- 172.21.15.26:0/3001397995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb006bee0 0x7efeb0155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb55ba640 1 -- 172.21.15.26:0/3001397995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb006bee0 msgr2=0x7efeb0155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb55ba640 1 --2- 172.21.15.26:0/3001397995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb006bee0 0x7efeb0155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb55ba640 1 -- 172.21.15.26:0/3001397995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00cb1d0 msgr2=0x7efeb0153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb55ba640 1 --2- 172.21.15.26:0/3001397995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00cb1d0 0x7efeb0153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb55ba640 1 -- 172.21.15.26:0/3001397995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea8009b30 con 0x7efeb00b1d40 2023-12-19T18:18:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb55ba640 1 --2- 172.21.15.26:0/3001397995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb00b1d40 0x7efeb00ae260 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7efea000c480 tx=0x7efea000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:39.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efea67fc640 1 -- 172.21.15.26:0/3001397995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea00074e0 con 0x7efeb00b1d40 2023-12-19T18:18:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efea67fc640 1 -- 172.21.15.26:0/3001397995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea0007660 con 0x7efeb00b1d40 2023-12-19T18:18:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efea67fc640 1 -- 172.21.15.26:0/3001397995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0004a30 con 0x7efeb00b1d40 2023-12-19T18:18:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb801f640 1 -- 172.21.15.26:0/3001397995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb015ef60 con 0x7efeb00b1d40 2023-12-19T18:18:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb801f640 1 -- 172.21.15.26:0/3001397995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb00cd390 con 0x7efeb00b1d40 2023-12-19T18:18:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efeb801f640 1 -- 172.21.15.26:0/3001397995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb00cd580 con 0x7efeb00b1d40 2023-12-19T18:18:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.340+0000 7efea67fc640 1 -- 172.21.15.26:0/3001397995 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 142) v1 ==== 84939+0+0 (secure 0 0 0) 0x7efea0004b90 con 0x7efeb00b1d40 2023-12-19T18:18:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.344+0000 7efea67fc640 1 -- 172.21.15.26:0/3001397995 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 7157+0+0 (secure 0 0 0) 0x7efea0082340 con 0x7efeb00b1d40 2023-12-19T18:18:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.348+0000 7efea67fc640 1 -- 172.21.15.26:0/3001397995 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efeb00cd580 con 0x7efeb00b1d40 2023-12-19T18:18:39.536 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.536+0000 7efeb801f640 1 -- 172.21.15.26:0/3001397995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efeb00cd580 con 0x7efeb00b1d40 2023-12-19T18:18:39.537 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.536+0000 7efea67fc640 1 -- 172.21.15.26:0/3001397995 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4728 (secure 0 0 0) 0x7efeb00cd580 con 0x7efeb00b1d40 2023-12-19T18:18:39.537 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:39.537 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":128,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/282680544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":282680544},{"type":"v1","addr":"172.21.15.160:6836","nonce":282680544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3302634175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3302634175},{"type":"v1","addr":"172.21.15.160:6837","nonce":3302634175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3434994961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3434994961},{"type":"v1","addr":"172.21.15.26:6834","nonce":3434994961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:18:19.555514+0000","modified":"2023-12-19T18:18:22.569161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1019381645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1019381645},{"type":"v1","addr":"172.21.15.26:6835","nonce":1019381645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-19T18:18:39.537 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 128 2023-12-19T18:18:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.536+0000 7efeb801f640 1 -- 172.21.15.26:0/3001397995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb00b1d40 msgr2=0x7efeb00ae260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.536+0000 7efeb801f640 1 --2- 172.21.15.26:0/3001397995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb00b1d40 0x7efeb00ae260 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7efea000c480 tx=0x7efea000c940 comp rx=0 tx=0).stop 2023-12-19T18:18:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.536+0000 7efeb801f640 1 -- 172.21.15.26:0/3001397995 shutdown_connections 2023-12-19T18:18:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.536+0000 7efeb801f640 1 --2- 172.21.15.26:0/3001397995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00cb1d0 0x7efeb0153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.536+0000 7efeb801f640 1 --2- 172.21.15.26:0/3001397995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efeb00b1d40 0x7efeb00ae260 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.536+0000 7efeb801f640 1 --2- 172.21.15.26:0/3001397995 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7efeb006bee0 0x7efeb0155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.536+0000 7efeb801f640 1 -- 172.21.15.26:0/3001397995 >> 172.21.15.26:0/3001397995 conn(0x7efeb00658e0 msgr2=0x7efeb015fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.536+0000 7efeb801f640 1 -- 172.21.15.26:0/3001397995 shutdown_connections 2023-12-19T18:18:39.539 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.536+0000 7efeb801f640 1 -- 172.21.15.26:0/3001397995 wait complete. 2023-12-19T18:18:39.558 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:39.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.744+0000 7f52fbda5640 1 Processor -- start 2023-12-19T18:18:39.746 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.744+0000 7f52fbda5640 1 -- start start 2023-12-19T18:18:39.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.744+0000 7f52fbda5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52f40cb1d0 0x7f52f40cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:39.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.744+0000 7f52fbda5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f52f406bee0 0x7f52f406c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:39.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.744+0000 7f52fbda5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52f4164c90 0x7f52f4165120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:39.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.744+0000 7f52fbda5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f40cc030 con 0x7f52f40cb1d0 2023-12-19T18:18:39.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52fbda5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f406d4b0 con 0x7f52f406bee0 2023-12-19T18:18:39.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52fbda5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40cbaf0 con 0x7f52f4164c90 2023-12-19T18:18:39.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f9b41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52f40cb1d0 0x7f52f40cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:39.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f9b41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52f40cb1d0 0x7f52f40cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36166/0 (socket says 172.21.15.26:36166) 2023-12-19T18:18:39.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f9b41640 1 -- 172.21.15.26:0/1796344871 learned_addr learned my addr 172.21.15.26:0/1796344871 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:39.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f9b41640 1 -- 172.21.15.26:0/1796344871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52f4164c90 msgr2=0x7f52f4165120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:39.748 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f9340640 1 --2- 172.21.15.26:0/1796344871 >> v2:172.21.15.160:3300/0 conn(0x7f52f406bee0 0x7f52f406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:39.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52fa342640 1 --2- 172.21.15.26:0/1796344871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52f4164c90 0x7f52f4165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:39.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f9b41640 1 --2- 172.21.15.26:0/1796344871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52f4164c90 0x7f52f4165120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f9b41640 1 -- 172.21.15.26:0/1796344871 >> v2:172.21.15.160:3300/0 conn(0x7f52f406bee0 msgr2=0x7f52f406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:39.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f9b41640 1 --2- 172.21.15.26:0/1796344871 >> v2:172.21.15.160:3300/0 conn(0x7f52f406bee0 0x7f52f406c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f9b41640 1 -- 172.21.15.26:0/1796344871 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f406cc10 con 0x7f52f40cb1d0 2023-12-19T18:18:39.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f9340640 1 --2- 172.21.15.26:0/1796344871 >> v2:172.21.15.160:3300/0 conn(0x7f52f406bee0 0x7f52f406c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:39.749 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52fa342640 1 --2- 172.21.15.26:0/1796344871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52f4164c90 0x7f52f4165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:39.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f9b41640 1 --2- 172.21.15.26:0/1796344871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f40cb1d0 0x7f52f40cb5b0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f52f40667b0 tx=0x7f52f001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4ef53dec53543d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:39.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f8b3f640 1 -- 172.21.15.26:0/1796344871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f00ab600 con 0x7f52f40cb1d0 2023-12-19T18:18:39.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f8b3f640 1 -- 172.21.15.26:0/1796344871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f00ab760 con 0x7f52f40cb1d0 2023-12-19T18:18:39.751 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.748+0000 7f52f8b3f640 1 -- 172.21.15.26:0/1796344871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f00abaf0 con 0x7f52f40cb1d0 2023-12-19T18:18:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 -- 172.21.15.26:0/1796344871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f40cb1d0 msgr2=0x7f52f40cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 --2- 172.21.15.26:0/1796344871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f40cb1d0 0x7f52f40cb5b0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f52f40667b0 tx=0x7f52f001c440 comp rx=0 tx=0).stop 2023-12-19T18:18:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 -- 172.21.15.26:0/1796344871 shutdown_connections 2023-12-19T18:18:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 --2- 172.21.15.26:0/1796344871 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52f4164c90 0x7f52f4165120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 --2- 172.21.15.26:0/1796344871 >> v2:172.21.15.160:3300/0 conn(0x7f52f406bee0 0x7f52f406c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 --2- 172.21.15.26:0/1796344871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f40cb1d0 0x7f52f40cb5b0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 -- 172.21.15.26:0/1796344871 >> 172.21.15.26:0/1796344871 conn(0x7f52f40658e0 msgr2=0x7f52f4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:39.752 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 -- 172.21.15.26:0/1796344871 shutdown_connections 2023-12-19T18:18:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 -- 172.21.15.26:0/1796344871 wait complete. 2023-12-19T18:18:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 Processor -- start 2023-12-19T18:18:39.753 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 -- start start 2023-12-19T18:18:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f406bee0 0x7f52f40ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52f40cb1d0 0x7f52f4161d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52f4164c90 0x7f52f4150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f4157dd0 con 0x7f52f406bee0 2023-12-19T18:18:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f4158950 con 0x7f52f4164c90 2023-12-19T18:18:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fbda5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f4158020 con 0x7f52f40cb1d0 2023-12-19T18:18:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52f9b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f406bee0 0x7f52f40ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52f9b41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f406bee0 0x7f52f40ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36170/0 (socket says 172.21.15.26:36170) 2023-12-19T18:18:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52f9b41640 1 -- 172.21.15.26:0/1532641591 learned_addr learned my addr 172.21.15.26:0/1532641591 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52f9340640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52f40cb1d0 0x7f52f4161d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:39.755 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.752+0000 7f52fa342640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52f4164c90 0x7f52f4150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:39.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52f9b41640 1 -- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52f40cb1d0 msgr2=0x7f52f4161d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:39.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52f9b41640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52f40cb1d0 0x7f52f4161d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52f9b41640 1 -- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52f4164c90 msgr2=0x7f52f4150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:39.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52f9b41640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52f4164c90 0x7f52f4150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.756 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52f9b41640 1 -- 172.21.15.26:0/1532641591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f00094d0 con 0x7f52f406bee0 2023-12-19T18:18:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52f9b41640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f406bee0 0x7f52f40ce0d0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f52f00ad040 tx=0x7f52f001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52ea7fc640 1 -- 172.21.15.26:0/1532641591 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f00b4800 con 0x7f52f406bee0 2023-12-19T18:18:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52ea7fc640 1 -- 172.21.15.26:0/1532641591 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f00b4980 con 0x7f52f406bee0 2023-12-19T18:18:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52ea7fc640 1 -- 172.21.15.26:0/1532641591 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f00b4d30 con 0x7f52f406bee0 2023-12-19T18:18:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52fbda5640 1 -- 172.21.15.26:0/1532641591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f4158300 con 0x7f52f406bee0 2023-12-19T18:18:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52fbda5640 1 -- 172.21.15.26:0/1532641591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f40ce610 con 0x7f52f406bee0 2023-12-19T18:18:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52fbda5640 1 -- 172.21.15.26:0/1532641591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f4074270 con 0x7f52f406bee0 2023-12-19T18:18:39.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52ea7fc640 1 -- 172.21.15.26:0/1532641591 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 142) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f52f00c7440 con 0x7f52f406bee0 2023-12-19T18:18:39.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.756+0000 7f52ea7fc640 1 -- 172.21.15.26:0/1532641591 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 7157+0+0 (secure 0 0 0) 0x7f52f011dda0 con 0x7f52f406bee0 2023-12-19T18:18:39.762 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.760+0000 7f52ea7fc640 1 -- 172.21.15.26:0/1532641591 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f52f00eb5c0 con 0x7f52f406bee0 2023-12-19T18:18:39.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.908+0000 7f52ea7fc640 1 -- 172.21.15.26:0/1532641591 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 143) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f52f00eb070 con 0x7f52f406bee0 2023-12-19T18:18:39.907 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.908+0000 7f52ea7fc640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52d801c9b0 0x7f52d806db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:39.909 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.908+0000 7f52f9340640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52d801c9b0 0x7f52d806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:39.914 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.912+0000 7f52f9340640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52d801c9b0 0x7f52d806db20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f52f41612a0 tx=0x7f52e4008040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:39.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.924+0000 7f52fbda5640 1 -- 172.21.15.26:0/1532641591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52f406b5a0 con 0x7f52f406bee0 2023-12-19T18:18:39.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.924+0000 7f52ea7fc640 1 -- 172.21.15.26:0/1532641591 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v256) v1 ==== 74+0+18208 (secure 0 0 0) 0x7f52f00f1d00 con 0x7f52f406bee0 2023-12-19T18:18:39.926 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:39.927 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":256,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:18:38.894683+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:17.314988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:18.558040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1406825903":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/2675273338":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/1263112506":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/3295238442":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/2546173214":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/1122697388":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/1748568728":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/2326417443":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/2225649811":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/2614640962":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/1309045557":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3586017330":"2023-12-20T18:18:31.899321+0000","172.21.15.160:6833/48394":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3350159660":"2023-12-20T18:18:31.899321+0000","172.21.15.160:6832/48394":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3229222145":"2023-12-20T18:18:31.899321+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:39.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.928+0000 7f52fbda5640 1 -- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52d801c9b0 msgr2=0x7f52d806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:39.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.928+0000 7f52fbda5640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52d801c9b0 0x7f52d806db20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f52f41612a0 tx=0x7f52e4008040 comp rx=0 tx=0).stop 2023-12-19T18:18:39.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.928+0000 7f52fbda5640 1 -- 172.21.15.26:0/1532641591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f406bee0 msgr2=0x7f52f40ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:39.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.928+0000 7f52fbda5640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f406bee0 0x7f52f40ce0d0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f52f00ad040 tx=0x7f52f001ac80 comp rx=0 tx=0).stop 2023-12-19T18:18:39.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.928+0000 7f52fbda5640 1 -- 172.21.15.26:0/1532641591 shutdown_connections 2023-12-19T18:18:39.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.928+0000 7f52fbda5640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f52d801c9b0 0x7f52d806db20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.928+0000 7f52fbda5640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52f4164c90 0x7f52f4150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.928+0000 7f52fbda5640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f52f40cb1d0 0x7f52f4161d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.928+0000 7f52fbda5640 1 --2- 172.21.15.26:0/1532641591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52f406bee0 0x7f52f40ce0d0 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:39.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.928+0000 7f52fbda5640 1 -- 172.21.15.26:0/1532641591 >> 172.21.15.26:0/1532641591 conn(0x7f52f40658e0 msgr2=0x7f52f4162d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:39.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.928+0000 7f52fbda5640 1 -- 172.21.15.26:0/1532641591 shutdown_connections 2023-12-19T18:18:39.930 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:39.928+0000 7f52fbda5640 1 -- 172.21.15.26:0/1532641591 wait complete. 2023-12-19T18:18:39.948 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:18:39.948 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:40.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd45a41640 1 Processor -- start 2023-12-19T18:18:40.092 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd45a41640 1 -- start start 2023-12-19T18:18:40.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd45a41640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd4015d260 0x7fcd4006e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd45a41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd400b34f0 0x7fcd400b3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd45a41640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd400b3060 con 0x7fcd400658e0 2023-12-19T18:18:40.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd45a41640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4015d820 con 0x7fcd4015d260 2023-12-19T18:18:40.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd45a41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4015db00 con 0x7fcd400b34f0 2023-12-19T18:18:40.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd37fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd4015d260 0x7fcd4006e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd37fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd4015d260 0x7fcd4006e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36178/0 (socket says 172.21.15.26:36178) 2023-12-19T18:18:40.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd37fff640 1 -- 172.21.15.26:0/1157291768 learned_addr learned my addr 172.21.15.26:0/1157291768 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:40.094 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd3f7fe640 1 --2- 172.21.15.26:0/1157291768 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd400b34f0 0x7fcd400b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd37fff640 1 -- 172.21.15.26:0/1157291768 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd400b34f0 msgr2=0x7fcd400b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:40.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd37fff640 1 --2- 172.21.15.26:0/1157291768 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd400b34f0 0x7fcd400b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd37fff640 1 -- 172.21.15.26:0/1157291768 >> v1:172.21.15.160:6789/0 conn(0x7fcd400658e0 legacy=0x7fcd40065cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.095 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.092+0000 7fcd37fff640 1 -- 172.21.15.26:0/1157291768 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4015dde0 con 0x7fcd4015d260 2023-12-19T18:18:40.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.096+0000 7fcd37fff640 1 --2- 172.21.15.26:0/1157291768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd4015d260 0x7fcd4006e5a0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009870 tx=0x7fcd2801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5839b65308fc8e16 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:40.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.096+0000 7fcd3e7fc640 1 -- 172.21.15.26:0/1157291768 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280ad640 con 0x7fcd4015d260 2023-12-19T18:18:40.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.096+0000 7fcd3e7fc640 1 -- 172.21.15.26:0/1157291768 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd280ad7a0 con 0x7fcd4015d260 2023-12-19T18:18:40.097 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.096+0000 7fcd3e7fc640 1 -- 172.21.15.26:0/1157291768 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280adb30 con 0x7fcd4015d260 2023-12-19T18:18:40.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.096+0000 7fcd45a41640 1 -- 172.21.15.26:0/1157291768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd4015d260 msgr2=0x7fcd4006e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.096+0000 7fcd45a41640 1 --2- 172.21.15.26:0/1157291768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd4015d260 0x7fcd4006e5a0 secure :-1 s=READY pgs=2416 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009870 tx=0x7fcd2801c480 comp rx=0 tx=0).stop 2023-12-19T18:18:40.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.096+0000 7fcd45a41640 1 -- 172.21.15.26:0/1157291768 shutdown_connections 2023-12-19T18:18:40.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.096+0000 7fcd45a41640 1 --2- 172.21.15.26:0/1157291768 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd400b34f0 0x7fcd400b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.096+0000 7fcd45a41640 1 --2- 172.21.15.26:0/1157291768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd4015d260 0x7fcd4006e5a0 unknown :-1 s=CLOSED pgs=2416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.096+0000 7fcd45a41640 1 -- 172.21.15.26:0/1157291768 >> 172.21.15.26:0/1157291768 conn(0x7fcd4006a320 msgr2=0x7fcd400b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:40.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.096+0000 7fcd45a41640 1 -- 172.21.15.26:0/1157291768 shutdown_connections 2023-12-19T18:18:40.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.096+0000 7fcd45a41640 1 -- 172.21.15.26:0/1157291768 wait complete. 2023-12-19T18:18:40.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd45a41640 1 Processor -- start 2023-12-19T18:18:40.100 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd45a41640 1 -- start start 2023-12-19T18:18:40.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd45a41640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd400658e0 0x7fcd4013d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd45a41640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd400b34f0 0x7fcd4013f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd45a41640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd4015d260 0x7fcd4015c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd45a41640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd401395a0 con 0x7fcd400b34f0 2023-12-19T18:18:40.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd45a41640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd401389f0 con 0x7fcd4015d260 2023-12-19T18:18:40.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd45a41640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4013db60 con 0x7fcd400658e0 2023-12-19T18:18:40.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd400b34f0 0x7fcd4013f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd37fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd400b34f0 0x7fcd4013f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36194/0 (socket says 172.21.15.26:36194) 2023-12-19T18:18:40.102 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd37fff640 1 -- 172.21.15.26:0/1976976972 learned_addr learned my addr 172.21.15.26:0/1976976972 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:40.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd3f7fe640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd4015d260 0x7fcd4015c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd3effd640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd400658e0 0x7fcd4013d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd37fff640 1 -- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd400658e0 msgr2=0x7fcd4013d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd37fff640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd400658e0 0x7fcd4013d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd37fff640 1 -- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd4015d260 msgr2=0x7fcd4015c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd37fff640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd4015d260 0x7fcd4015c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd37fff640 1 -- 172.21.15.26:0/1976976972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd280094d0 con 0x7fcd400b34f0 2023-12-19T18:18:40.103 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd37fff640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd400b34f0 0x7fcd4013f7e0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fcd280b0040 tx=0x7fcd28019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:40.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd3cff9640 1 -- 172.21.15.26:0/1976976972 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280b8860 con 0x7fcd400b34f0 2023-12-19T18:18:40.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd3cff9640 1 -- 172.21.15.26:0/1976976972 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd280a6040 con 0x7fcd400b34f0 2023-12-19T18:18:40.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd45a41640 1 -- 172.21.15.26:0/1976976972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd40138c00 con 0x7fcd400b34f0 2023-12-19T18:18:40.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd3cff9640 1 -- 172.21.15.26:0/1976976972 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd280b8c40 con 0x7fcd400b34f0 2023-12-19T18:18:40.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.100+0000 7fcd45a41640 1 -- 172.21.15.26:0/1976976972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd401391c0 con 0x7fcd400b34f0 2023-12-19T18:18:40.104 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.104+0000 7fcd45a41640 1 -- 172.21.15.26:0/1976976972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd401396e0 con 0x7fcd400b34f0 2023-12-19T18:18:40.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.104+0000 7fcd3cff9640 1 -- 172.21.15.26:0/1976976972 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 143) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fcd280b4070 con 0x7fcd400b34f0 2023-12-19T18:18:40.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.104+0000 7fcd3cff9640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcd200658b0 0x7fcd20067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.105 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.104+0000 7fcd3cff9640 1 -- 172.21.15.26:0/1976976972 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 7157+0+0 (secure 0 0 0) 0x7fcd2811db90 con 0x7fcd400b34f0 2023-12-19T18:18:40.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.104+0000 7fcd3effd640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcd200658b0 0x7fcd20067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.106 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.104+0000 7fcd3effd640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcd200658b0 0x7fcd20067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcd400d0420 tx=0x7fcd30009040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:40.109 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.108+0000 7fcd3cff9640 1 -- 172.21.15.26:0/1976976972 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcd401396e0 con 0x7fcd400b34f0 2023-12-19T18:18:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.300+0000 7fcd45a41640 1 -- 172.21.15.26:0/1976976972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd401396e0 con 0x7fcd400b34f0 2023-12-19T18:18:40.301 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.300+0000 7fcd3cff9640 1 -- 172.21.15.26:0/1976976972 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4728 (secure 0 0 0) 0x7fcd401396e0 con 0x7fcd400b34f0 2023-12-19T18:18:40.301 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:40.302 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":128,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/282680544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":282680544},{"type":"v1","addr":"172.21.15.160:6836","nonce":282680544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3302634175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3302634175},{"type":"v1","addr":"172.21.15.160:6837","nonce":3302634175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3434994961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3434994961},{"type":"v1","addr":"172.21.15.26:6834","nonce":3434994961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:18:19.555514+0000","modified":"2023-12-19T18:18:22.569161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1019381645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1019381645},{"type":"v1","addr":"172.21.15.26:6835","nonce":1019381645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-19T18:18:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 128 2023-12-19T18:18:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.304+0000 7fcd45a41640 1 -- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcd200658b0 msgr2=0x7fcd20067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.304+0000 7fcd45a41640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcd200658b0 0x7fcd20067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcd400d0420 tx=0x7fcd30009040 comp rx=0 tx=0).stop 2023-12-19T18:18:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.304+0000 7fcd45a41640 1 -- 172.21.15.26:0/1976976972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd400b34f0 msgr2=0x7fcd4013f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.304+0000 7fcd45a41640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd400b34f0 0x7fcd4013f7e0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7fcd280b0040 tx=0x7fcd28019220 comp rx=0 tx=0).stop 2023-12-19T18:18:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.304+0000 7fcd45a41640 1 -- 172.21.15.26:0/1976976972 shutdown_connections 2023-12-19T18:18:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.304+0000 7fcd45a41640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fcd200658b0 0x7fcd20067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.304+0000 7fcd45a41640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd4015d260 0x7fcd4015c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.304+0000 7fcd45a41640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd400b34f0 0x7fcd4013f7e0 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.304+0000 7fcd45a41640 1 --2- 172.21.15.26:0/1976976972 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fcd400658e0 0x7fcd4013d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.304+0000 7fcd45a41640 1 -- 172.21.15.26:0/1976976972 >> 172.21.15.26:0/1976976972 conn(0x7fcd4006a320 msgr2=0x7fcd400add50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.304+0000 7fcd45a41640 1 -- 172.21.15.26:0/1976976972 shutdown_connections 2023-12-19T18:18:40.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.304+0000 7fcd45a41640 1 -- 172.21.15.26:0/1976976972 wait complete. 2023-12-19T18:18:40.323 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:40.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.464+0000 7ffb3359e640 1 Processor -- start 2023-12-19T18:18:40.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb3359e640 1 -- start start 2023-12-19T18:18:40.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb3359e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffb3415cb80 0x7ffb340cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb3359e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb340d0030 0x7ffb340b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb3359e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb340d3320 con 0x7ffb340d1fc0 2023-12-19T18:18:40.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb3359e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb340d2710 con 0x7ffb3415cb80 2023-12-19T18:18:40.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb3359e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340d29f0 con 0x7ffb340d0030 2023-12-19T18:18:40.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb3259c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffb3415cb80 0x7ffb340cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb32d9d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb340d0030 0x7ffb340b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb31d9b640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7ffb340d1fc0 0x7ffb340cf920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46532/0 (socket says 172.21.15.26:46532) 2023-12-19T18:18:40.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb31d9b640 1 -- 172.21.15.26:0/4058769739 learned_addr learned my addr 172.21.15.26:0/4058769739 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:40.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb3259c640 1 -- 172.21.15.26:0/4058769739 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb340d0030 msgr2=0x7ffb340b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:40.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb3259c640 1 --2- 172.21.15.26:0/4058769739 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb340d0030 0x7ffb340b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb3259c640 1 -- 172.21.15.26:0/4058769739 >> v1:172.21.15.160:6789/0 conn(0x7ffb340d1fc0 legacy=0x7ffb340cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb3259c640 1 -- 172.21.15.26:0/4058769739 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb340d2cd0 con 0x7ffb3415cb80 2023-12-19T18:18:40.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.468+0000 7ffb32d9d640 1 --2- 172.21.15.26:0/4058769739 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb340d0030 0x7ffb340b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:40.472 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.472+0000 7ffb3259c640 1 --2- 172.21.15.26:0/4058769739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb3415cb80 0x7ffb340cdc80 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ffb240097c0 tx=0x7ffb2401c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=724aeaa4dfbc002d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:40.473 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.472+0000 7ffb3159a640 1 -- 172.21.15.26:0/4058769739 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb240ab600 con 0x7ffb3415cb80 2023-12-19T18:18:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.472+0000 7ffb3159a640 1 -- 172.21.15.26:0/4058769739 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb240a7050 con 0x7ffb3415cb80 2023-12-19T18:18:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.472+0000 7ffb3159a640 1 -- 172.21.15.26:0/4058769739 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb240ab980 con 0x7ffb3415cb80 2023-12-19T18:18:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.472+0000 7ffb3359e640 1 -- 172.21.15.26:0/4058769739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb3415cb80 msgr2=0x7ffb340cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.472+0000 7ffb3359e640 1 --2- 172.21.15.26:0/4058769739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb3415cb80 0x7ffb340cdc80 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ffb240097c0 tx=0x7ffb2401c3a0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.472+0000 7ffb3359e640 1 -- 172.21.15.26:0/4058769739 shutdown_connections 2023-12-19T18:18:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.472+0000 7ffb3359e640 1 --2- 172.21.15.26:0/4058769739 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb340d0030 0x7ffb340b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.472+0000 7ffb3359e640 1 --2- 172.21.15.26:0/4058769739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb3415cb80 0x7ffb340cdc80 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.472+0000 7ffb3359e640 1 -- 172.21.15.26:0/4058769739 >> 172.21.15.26:0/4058769739 conn(0x7ffb3406a120 msgr2=0x7ffb3406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.472+0000 7ffb3359e640 1 -- 172.21.15.26:0/4058769739 shutdown_connections 2023-12-19T18:18:40.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.472+0000 7ffb3359e640 1 -- 172.21.15.26:0/4058769739 wait complete. 2023-12-19T18:18:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb3359e640 1 Processor -- start 2023-12-19T18:18:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb3359e640 1 -- start start 2023-12-19T18:18:40.476 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb3359e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb340d0030 0x7ffb341f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb3359e640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb340d1fc0 0x7ffb340b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb3359e640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb3415cb80 0x7ffb340b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb3359e640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340b7770 con 0x7ffb3415cb80 2023-12-19T18:18:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb3359e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340b1a90 con 0x7ffb340d0030 2023-12-19T18:18:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb3359e640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340b8030 con 0x7ffb340d1fc0 2023-12-19T18:18:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb32d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb3415cb80 0x7ffb340b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb32d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb3415cb80 0x7ffb340b7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36210/0 (socket says 172.21.15.26:36210) 2023-12-19T18:18:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb32d9d640 1 -- 172.21.15.26:0/2283437980 learned_addr learned my addr 172.21.15.26:0/2283437980 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb3259c640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb340d0030 0x7ffb341f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.477 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb31d9b640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb340d1fc0 0x7ffb340b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.478 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb32d9d640 1 -- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb340d1fc0 msgr2=0x7ffb340b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb32d9d640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb340d1fc0 0x7ffb340b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb32d9d640 1 -- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb340d0030 msgr2=0x7ffb341f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb32d9d640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb340d0030 0x7ffb341f24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb32d9d640 1 -- 172.21.15.26:0/2283437980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c009a70 con 0x7ffb3415cb80 2023-12-19T18:18:40.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb32d9d640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb3415cb80 0x7ffb340b7af0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009f00 tx=0x7ffb2c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:40.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb22ffd640 1 -- 172.21.15.26:0/2283437980 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c009d70 con 0x7ffb3415cb80 2023-12-19T18:18:40.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb22ffd640 1 -- 172.21.15.26:0/2283437980 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c00f040 con 0x7ffb3415cb80 2023-12-19T18:18:40.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb3359e640 1 -- 172.21.15.26:0/2283437980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb24009420 con 0x7ffb3415cb80 2023-12-19T18:18:40.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb22ffd640 1 -- 172.21.15.26:0/2283437980 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c003bc0 con 0x7ffb3415cb80 2023-12-19T18:18:40.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb3359e640 1 -- 172.21.15.26:0/2283437980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb341f2b70 con 0x7ffb3415cb80 2023-12-19T18:18:40.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.476+0000 7ffb3359e640 1 -- 172.21.15.26:0/2283437980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb340d3790 con 0x7ffb3415cb80 2023-12-19T18:18:40.480 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.480+0000 7ffb22ffd640 1 -- 172.21.15.26:0/2283437980 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 143) v1 ==== 85066+0+0 (secure 0 0 0) 0x7ffb2c014480 con 0x7ffb3415cb80 2023-12-19T18:18:40.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.480+0000 7ffb22ffd640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffb140658b0 0x7ffb14067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.480+0000 7ffb22ffd640 1 -- 172.21.15.26:0/2283437980 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 7157+0+0 (secure 0 0 0) 0x7ffb2c083f30 con 0x7ffb3415cb80 2023-12-19T18:18:40.481 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.480+0000 7ffb3259c640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffb140658b0 0x7ffb14067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.482 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.480+0000 7ffb3259c640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffb140658b0 0x7ffb14067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffb240ad000 tx=0x7ffb24008040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:40.485 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.484+0000 7ffb22ffd640 1 -- 172.21.15.26:0/2283437980 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffb340d3790 con 0x7ffb3415cb80 2023-12-19T18:18:40.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb3359e640 1 -- 172.21.15.26:0/2283437980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb340d3790 con 0x7ffb3415cb80 2023-12-19T18:18:40.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb22ffd640 1 -- 172.21.15.26:0/2283437980 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4728 (secure 0 0 0) 0x7ffb340d3790 con 0x7ffb3415cb80 2023-12-19T18:18:40.676 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:40.676 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":128,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/282680544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":282680544},{"type":"v1","addr":"172.21.15.160:6836","nonce":282680544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3302634175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3302634175},{"type":"v1","addr":"172.21.15.160:6837","nonce":3302634175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3434994961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3434994961},{"type":"v1","addr":"172.21.15.26:6834","nonce":3434994961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:18:19.555514+0000","modified":"2023-12-19T18:18:22.569161+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12183},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12183":{"gid":12183,"name":"c","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1019381645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1019381645},{"type":"v1","addr":"172.21.15.26:6835","nonce":1019381645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-19T18:18:40.676 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 128 2023-12-19T18:18:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb3359e640 1 -- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffb140658b0 msgr2=0x7ffb14067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb3359e640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffb140658b0 0x7ffb14067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffb240ad000 tx=0x7ffb24008040 comp rx=0 tx=0).stop 2023-12-19T18:18:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb3359e640 1 -- 172.21.15.26:0/2283437980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb3415cb80 msgr2=0x7ffb340b7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb3359e640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb3415cb80 0x7ffb340b7af0 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009f00 tx=0x7ffb2c00c490 comp rx=0 tx=0).stop 2023-12-19T18:18:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb3359e640 1 -- 172.21.15.26:0/2283437980 shutdown_connections 2023-12-19T18:18:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb3359e640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ffb140658b0 0x7ffb14067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb3359e640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb3415cb80 0x7ffb340b7af0 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb3359e640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ffb340d1fc0 0x7ffb340b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb3359e640 1 --2- 172.21.15.26:0/2283437980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffb340d0030 0x7ffb341f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb3359e640 1 -- 172.21.15.26:0/2283437980 >> 172.21.15.26:0/2283437980 conn(0x7ffb3406a120 msgr2=0x7ffb3406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb3359e640 1 -- 172.21.15.26:0/2283437980 shutdown_connections 2023-12-19T18:18:40.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.676+0000 7ffb3359e640 1 -- 172.21.15.26:0/2283437980 wait complete. 2023-12-19T18:18:40.698 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:40.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.840+0000 7f0ce98e5640 1 Processor -- start 2023-12-19T18:18:40.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.840+0000 7f0ce98e5640 1 -- start start 2023-12-19T18:18:40.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.840+0000 7f0ce98e5640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ce4155030 0x7f0ce415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.840+0000 7f0ce98e5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0ce415f370 0x7f0ce406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.840+0000 7f0ce98e5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce406c970 con 0x7f0ce40ce740 2023-12-19T18:18:40.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.840+0000 7f0ce98e5640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce415b1f0 con 0x7f0ce4155030 2023-12-19T18:18:40.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.840+0000 7f0ce98e5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce415b4d0 con 0x7f0ce415f370 2023-12-19T18:18:40.843 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.840+0000 7f0ce2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ce4155030 0x7f0ce415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.844+0000 7f0ce2ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ce4155030 0x7f0ce415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36212/0 (socket says 172.21.15.26:36212) 2023-12-19T18:18:40.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.844+0000 7f0ce2ffd640 1 -- 172.21.15.26:0/3885109348 learned_addr learned my addr 172.21.15.26:0/3885109348 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:40.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.844+0000 7f0cdbfff640 1 --2- 172.21.15.26:0/3885109348 >> v2:172.21.15.160:3300/0 conn(0x7f0ce415f370 0x7f0ce406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.844+0000 7f0ce2ffd640 1 -- 172.21.15.26:0/3885109348 >> v2:172.21.15.160:3300/0 conn(0x7f0ce415f370 msgr2=0x7f0ce406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:40.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.844+0000 7f0ce2ffd640 1 --2- 172.21.15.26:0/3885109348 >> v2:172.21.15.160:3300/0 conn(0x7f0ce415f370 0x7f0ce406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.844+0000 7f0ce2ffd640 1 -- 172.21.15.26:0/3885109348 >> v1:172.21.15.160:6789/0 conn(0x7f0ce40ce740 legacy=0x7f0ce40ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.844 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.844+0000 7f0ce2ffd640 1 -- 172.21.15.26:0/3885109348 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce415b7b0 con 0x7f0ce4155030 2023-12-19T18:18:40.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.844+0000 7f0ce2ffd640 1 --2- 172.21.15.26:0/3885109348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce4155030 0x7f0ce415bcd0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40b1780 tx=0x7f0cd401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df48424284df4f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:40.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.844+0000 7f0ce27fc640 1 -- 172.21.15.26:0/3885109348 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40ab620 con 0x7f0ce4155030 2023-12-19T18:18:40.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.844+0000 7f0ce27fc640 1 -- 172.21.15.26:0/3885109348 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd40ab780 con 0x7f0ce4155030 2023-12-19T18:18:40.846 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.844+0000 7f0ce27fc640 1 -- 172.21.15.26:0/3885109348 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40abb10 con 0x7f0ce4155030 2023-12-19T18:18:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3885109348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce4155030 msgr2=0x7f0ce415bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 --2- 172.21.15.26:0/3885109348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce4155030 0x7f0ce415bcd0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40b1780 tx=0x7f0cd401c500 comp rx=0 tx=0).stop 2023-12-19T18:18:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3885109348 shutdown_connections 2023-12-19T18:18:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 --2- 172.21.15.26:0/3885109348 >> v2:172.21.15.160:3300/0 conn(0x7f0ce415f370 0x7f0ce406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 --2- 172.21.15.26:0/3885109348 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce4155030 0x7f0ce415bcd0 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.847 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3885109348 >> 172.21.15.26:0/3885109348 conn(0x7f0ce40658e0 msgr2=0x7f0ce4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:40.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3885109348 shutdown_connections 2023-12-19T18:18:40.848 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3885109348 wait complete. 2023-12-19T18:18:40.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 Processor -- start 2023-12-19T18:18:40.849 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 -- start start 2023-12-19T18:18:40.850 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce40ce740 0x7f0ce414be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ce4155030 0x7f0ce414b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ce415f370 0x7f0ce4153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40b0b00 con 0x7f0ce40ce740 2023-12-19T18:18:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce414c6e0 con 0x7f0ce4155030 2023-12-19T18:18:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce98e5640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce40afe80 con 0x7f0ce415f370 2023-12-19T18:18:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce40ce740 0x7f0ce414be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce2ffd640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce40ce740 0x7f0ce414be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36214/0 (socket says 172.21.15.26:36214) 2023-12-19T18:18:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce2ffd640 1 -- 172.21.15.26:0/3368951562 learned_addr learned my addr 172.21.15.26:0/3368951562 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0cdbfff640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ce4155030 0x7f0ce414b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce37fe640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ce415f370 0x7f0ce4153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce2ffd640 1 -- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ce415f370 msgr2=0x7f0ce4153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce2ffd640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ce415f370 0x7f0ce4153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce2ffd640 1 -- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ce4155030 msgr2=0x7f0ce414b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce2ffd640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ce4155030 0x7f0ce414b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce2ffd640 1 -- 172.21.15.26:0/3368951562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40094d0 con 0x7f0ce40ce740 2023-12-19T18:18:40.852 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.848+0000 7f0ce37fe640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ce415f370 0x7f0ce4153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:40.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.852+0000 7f0ce2ffd640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce40ce740 0x7f0ce414be80 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40ad040 tx=0x7f0cd401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:40.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.852+0000 7f0ce0ff9640 1 -- 172.21.15.26:0/3368951562 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40be3f0 con 0x7f0ce40ce740 2023-12-19T18:18:40.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.852+0000 7f0ce0ff9640 1 -- 172.21.15.26:0/3368951562 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd40be550 con 0x7f0ce40ce740 2023-12-19T18:18:40.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.852+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3368951562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce40cf0e0 con 0x7f0ce40ce740 2023-12-19T18:18:40.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.852+0000 7f0ce0ff9640 1 -- 172.21.15.26:0/3368951562 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40be8e0 con 0x7f0ce40ce740 2023-12-19T18:18:40.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.852+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3368951562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce40b0440 con 0x7f0ce40ce740 2023-12-19T18:18:40.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.852+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3368951562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce4074270 con 0x7f0ce40ce740 2023-12-19T18:18:40.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.852+0000 7f0ce0ff9640 1 -- 172.21.15.26:0/3368951562 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 143) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f0cd40b43f0 con 0x7f0ce40ce740 2023-12-19T18:18:40.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.852+0000 7f0ce0ff9640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0cc40658e0 0x7f0cc4067da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:40.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.852+0000 7f0ce0ff9640 1 -- 172.21.15.26:0/3368951562 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 7157+0+0 (secure 0 0 0) 0x7f0cd411dbf0 con 0x7f0ce40ce740 2023-12-19T18:18:40.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.852+0000 7f0cdbfff640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0cc40658e0 0x7f0cc4067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:40.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.856+0000 7f0cdbfff640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0cc40658e0 0x7f0cc4067da0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0ce41f8ff0 tx=0x7f0ccc008040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:40.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:40.856+0000 7f0ce0ff9640 1 -- 172.21.15.26:0/3368951562 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0cd40eb320 con 0x7f0ce40ce740 2023-12-19T18:18:41.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3368951562 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ce40ccc30 con 0x7f0ce40ce740 2023-12-19T18:18:41.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce0ff9640 1 -- 172.21.15.26:0/3368951562 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v256) v1 ==== 74+0+18208 (secure 0 0 0) 0x7f0cd40ee940 con 0x7f0ce40ce740 2023-12-19T18:18:41.021 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:41.021 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":256,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:18:38.894683+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:17.314988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"253","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:18.558040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"254","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.160:0/1406825903":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/2675273338":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/1263112506":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/3295238442":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/2546173214":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/1122697388":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/1748568728":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/2326417443":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/2225649811":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/2614640962":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/1309045557":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3586017330":"2023-12-20T18:18:31.899321+0000","172.21.15.160:6833/48394":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3350159660":"2023-12-20T18:18:31.899321+0000","172.21.15.160:6832/48394":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3229222145":"2023-12-20T18:18:31.899321+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0cc40658e0 msgr2=0x7f0cc4067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce98e5640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0cc40658e0 0x7f0cc4067da0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0ce41f8ff0 tx=0x7f0ccc008040 comp rx=0 tx=0).stop 2023-12-19T18:18:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3368951562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce40ce740 msgr2=0x7f0ce414be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce98e5640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce40ce740 0x7f0ce414be80 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40ad040 tx=0x7f0cd401ad40 comp rx=0 tx=0).stop 2023-12-19T18:18:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3368951562 shutdown_connections 2023-12-19T18:18:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce98e5640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f0cc40658e0 0x7f0cc4067da0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce98e5640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f0ce415f370 0x7f0ce4153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce98e5640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0ce4155030 0x7f0ce414b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce98e5640 1 --2- 172.21.15.26:0/3368951562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ce40ce740 0x7f0ce414be80 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3368951562 >> 172.21.15.26:0/3368951562 conn(0x7f0ce40658e0 msgr2=0x7f0ce40b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3368951562 shutdown_connections 2023-12-19T18:18:41.023 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.020+0000 7f0ce98e5640 1 -- 172.21.15.26:0/3368951562 wait complete. 2023-12-19T18:18:41.042 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:18:41.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.188+0000 7fb92ea04640 1 Processor -- start 2023-12-19T18:18:41.188 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.188+0000 7fb92ea04640 1 -- start start 2023-12-19T18:18:41.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.188+0000 7fb92ea04640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb9280b8760 0x7fb9280cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.188+0000 7fb92ea04640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9280cdff0 con 0x7fb92815cca0 2023-12-19T18:18:41.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.188+0000 7fb92ea04640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9280b6cf0 con 0x7fb9280cdbb0 2023-12-19T18:18:41.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.188+0000 7fb92ea04640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9280b7c30 con 0x7fb9280b8760 2023-12-19T18:18:41.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.188+0000 7fb92da02640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb92815cca0 0x7fb9280b65e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34062/0 (socket says 172.21.15.26:34062) 2023-12-19T18:18:41.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.188+0000 7fb92d201640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb9280b8760 0x7fb9280cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.190 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.188+0000 7fb92da02640 1 -- 172.21.15.26:0/2439277060 learned_addr learned my addr 172.21.15.26:0/2439277060 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:41.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.188+0000 7fb92d201640 1 -- 172.21.15.26:0/2439277060 >> v1:172.21.15.160:6789/0 conn(0x7fb9280cdbb0 legacy=0x7fb9280d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.188+0000 7fb92d201640 1 -- 172.21.15.26:0/2439277060 >> v1:172.21.15.26:6789/0 conn(0x7fb92815cca0 legacy=0x7fb9280b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.188+0000 7fb92ca00640 1 -- 172.21.15.26:0/2439277060 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329665035 0 0) 0x7fb9280b6cf0 con 0x7fb9280cdbb0 2023-12-19T18:18:41.191 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.188+0000 7fb92d201640 1 -- 172.21.15.26:0/2439277060 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9280b6fb0 con 0x7fb9280b8760 2023-12-19T18:18:41.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.192+0000 7fb92d201640 1 --2- 172.21.15.26:0/2439277060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280b8760 0x7fb9280cd670 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fb9180097c0 tx=0x7fb91801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c369495d5d65c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:41.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.192+0000 7fb92ca00640 1 -- 172.21.15.26:0/2439277060 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180ab650 con 0x7fb9280b8760 2023-12-19T18:18:41.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.192+0000 7fb92ca00640 1 -- 172.21.15.26:0/2439277060 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9180ab7b0 con 0x7fb9280b8760 2023-12-19T18:18:41.193 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.192+0000 7fb92ca00640 1 -- 172.21.15.26:0/2439277060 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180abb40 con 0x7fb9280b8760 2023-12-19T18:18:41.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.192+0000 7fb92ea04640 1 -- 172.21.15.26:0/2439277060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280b8760 msgr2=0x7fb9280cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.192+0000 7fb92ea04640 1 --2- 172.21.15.26:0/2439277060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280b8760 0x7fb9280cd670 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7fb9180097c0 tx=0x7fb91801c450 comp rx=0 tx=0).stop 2023-12-19T18:18:41.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.192+0000 7fb92ea04640 1 -- 172.21.15.26:0/2439277060 shutdown_connections 2023-12-19T18:18:41.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.192+0000 7fb92ea04640 1 --2- 172.21.15.26:0/2439277060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280b8760 0x7fb9280cd670 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.192+0000 7fb92ea04640 1 -- 172.21.15.26:0/2439277060 >> 172.21.15.26:0/2439277060 conn(0x7fb92806a120 msgr2=0x7fb92806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:41.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.192+0000 7fb92ea04640 1 -- 172.21.15.26:0/2439277060 shutdown_connections 2023-12-19T18:18:41.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.192+0000 7fb92ea04640 1 -- 172.21.15.26:0/2439277060 wait complete. 2023-12-19T18:18:41.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92ea04640 1 Processor -- start 2023-12-19T18:18:41.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92ea04640 1 -- start start 2023-12-19T18:18:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92ea04640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280b8760 0x7fb92815fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92ea04640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9280cdbb0 0x7fb92814e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92ea04640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb92815cca0 0x7fb92815def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92ea04640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92814e5b0 con 0x7fb9280b8760 2023-12-19T18:18:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92ea04640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92815ec20 con 0x7fb92815cca0 2023-12-19T18:18:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92ea04640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92815ef00 con 0x7fb9280cdbb0 2023-12-19T18:18:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92da02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280b8760 0x7fb92815fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92da02640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280b8760 0x7fb92815fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36226/0 (socket says 172.21.15.26:36226) 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92da02640 1 -- 172.21.15.26:0/3291222395 learned_addr learned my addr 172.21.15.26:0/3291222395 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92e203640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb92815cca0 0x7fb92815def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92d201640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9280cdbb0 0x7fb92814e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92da02640 1 -- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9280cdbb0 msgr2=0x7fb92814e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92da02640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9280cdbb0 0x7fb92814e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92da02640 1 -- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb92815cca0 msgr2=0x7fb92815def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92da02640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb92815cca0 0x7fb92815def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92da02640 1 -- 172.21.15.26:0/3291222395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918009420 con 0x7fb9280b8760 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92da02640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280b8760 0x7fb92815fcc0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7fb92400c370 tx=0x7fb92400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb91e7fc640 1 -- 172.21.15.26:0/3291222395 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924005cf0 con 0x7fb9280b8760 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb91e7fc640 1 -- 172.21.15.26:0/3291222395 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb924005e50 con 0x7fb9280b8760 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb91e7fc640 1 -- 172.21.15.26:0/3291222395 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9240072f0 con 0x7fb9280b8760 2023-12-19T18:18:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92ea04640 1 -- 172.21.15.26:0/3291222395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92815f1e0 con 0x7fb9280b8760 2023-12-19T18:18:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.196+0000 7fb92ea04640 1 -- 172.21.15.26:0/3291222395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92815f7a0 con 0x7fb9280b8760 2023-12-19T18:18:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.200+0000 7fb92ea04640 1 -- 172.21.15.26:0/3291222395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb92815f9c0 con 0x7fb9280b8760 2023-12-19T18:18:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.200+0000 7fb91e7fc640 1 -- 172.21.15.26:0/3291222395 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 144) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fb924007450 con 0x7fb9280b8760 2023-12-19T18:18:41.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.200+0000 7fb91e7fc640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb90c065af0 0x7fb90c067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.200+0000 7fb91e7fc640 1 -- 172.21.15.26:0/3291222395 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 7157+0+0 (secure 0 0 0) 0x7fb924082dd0 con 0x7fb9280b8760 2023-12-19T18:18:41.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.200+0000 7fb92d201640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb90c065af0 0x7fb90c067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.200+0000 7fb92d201640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb90c065af0 0x7fb90c067fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb9180af000 tx=0x7fb9180a7000 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:41.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.204+0000 7fb91e7fc640 1 -- 172.21.15.26:0/3291222395 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb9240500f0 con 0x7fb9280b8760 2023-12-19T18:18:41.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.396+0000 7fb92ea04640 1 -- 172.21.15.26:0/3291222395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb928067400 con 0x7fb9280b8760 2023-12-19T18:18:41.397 INFO:tasks.ceph.mon.a.smithi026.stderr:2023-12-19T18:18:41.396+0000 7f05113f6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:18:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.404+0000 7fb91e7fc640 1 -- 172.21.15.26:0/3291222395 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v129) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb924017070 con 0x7fb9280b8760 2023-12-19T18:18:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:18:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.408+0000 7fb92ea04640 1 -- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb90c065af0 msgr2=0x7fb90c067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.408+0000 7fb92ea04640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb90c065af0 0x7fb90c067fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb9180af000 tx=0x7fb9180a7000 comp rx=0 tx=0).stop 2023-12-19T18:18:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.408+0000 7fb92ea04640 1 -- 172.21.15.26:0/3291222395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280b8760 msgr2=0x7fb92815fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.408+0000 7fb92ea04640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280b8760 0x7fb92815fcc0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7fb92400c370 tx=0x7fb92400c830 comp rx=0 tx=0).stop 2023-12-19T18:18:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.408+0000 7fb92ea04640 1 -- 172.21.15.26:0/3291222395 shutdown_connections 2023-12-19T18:18:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.408+0000 7fb92ea04640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fb90c065af0 0x7fb90c067fb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.408+0000 7fb92ea04640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb92815cca0 0x7fb92815def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.408+0000 7fb92ea04640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fb9280cdbb0 0x7fb92814e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.408+0000 7fb92ea04640 1 --2- 172.21.15.26:0/3291222395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9280b8760 0x7fb92815fcc0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.408+0000 7fb92ea04640 1 -- 172.21.15.26:0/3291222395 >> 172.21.15.26:0/3291222395 conn(0x7fb92806a120 msgr2=0x7fb928154190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.408+0000 7fb92ea04640 1 -- 172.21.15.26:0/3291222395 shutdown_connections 2023-12-19T18:18:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.408+0000 7fb92ea04640 1 -- 172.21.15.26:0/3291222395 wait complete. 2023-12-19T18:18:41.415 INFO:tasks.ceph.mds.c.smithi026.stderr: -3903> 2023-12-19T18:18:25.128+0000 7f9c0794f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:18:41.429 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:18:41.464 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2023-12-19T18:18:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.576+0000 7f70f6f7b640 1 Processor -- start 2023-12-19T18:18:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.576+0000 7f70f6f7b640 1 -- start start 2023-12-19T18:18:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.580+0000 7f70f6f7b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f70f015cf50 0x7f70f00b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.580+0000 7f70f6f7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70f00b85e0 0x7f70f00cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.580+0000 7f70f6f7b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f015d330 con 0x7f70f00b1a00 2023-12-19T18:18:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.580+0000 7f70f6f7b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f00d10c0 con 0x7f70f015cf50 2023-12-19T18:18:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.580+0000 7f70f6f7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00d13a0 con 0x7f70f00b85e0 2023-12-19T18:18:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.580+0000 7f70effff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70f00b85e0 0x7f70f00cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.580+0000 7f70effff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70f00b85e0 0x7f70f00cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:51072/0 (socket says 172.21.15.26:51072) 2023-12-19T18:18:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.580+0000 7f70effff640 1 -- 172.21.15.26:0/2931389275 learned_addr learned my addr 172.21.15.26:0/2931389275 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.580+0000 7f70f4d17640 1 --2- 172.21.15.26:0/2931389275 >> v2:172.21.15.160:3300/0 conn(0x7f70f015cf50 0x7f70f00b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.580+0000 7f70effff640 1 -- 172.21.15.26:0/2931389275 >> v2:172.21.15.160:3300/0 conn(0x7f70f015cf50 msgr2=0x7f70f00b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.580+0000 7f70effff640 1 --2- 172.21.15.26:0/2931389275 >> v2:172.21.15.160:3300/0 conn(0x7f70f015cf50 0x7f70f00b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.580+0000 7f70effff640 1 -- 172.21.15.26:0/2931389275 >> v1:172.21.15.160:6789/0 conn(0x7f70f00b1a00 legacy=0x7f70f00d1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.580+0000 7f70effff640 1 -- 172.21.15.26:0/2931389275 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f00d1680 con 0x7f70f00b85e0 2023-12-19T18:18:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70effff640 1 --2- 172.21.15.26:0/2931389275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70f00b85e0 0x7f70f00cda20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f70dc006fd0 tx=0x7f70dc01c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40d756bb6fa20c29 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70ef7fe640 1 -- 172.21.15.26:0/2931389275 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0aa670 con 0x7f70f00b85e0 2023-12-19T18:18:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70ef7fe640 1 -- 172.21.15.26:0/2931389275 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc0aa7d0 con 0x7f70f00b85e0 2023-12-19T18:18:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70ef7fe640 1 -- 172.21.15.26:0/2931389275 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0aab60 con 0x7f70f00b85e0 2023-12-19T18:18:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2931389275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70f00b85e0 msgr2=0x7f70f00cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70f6f7b640 1 --2- 172.21.15.26:0/2931389275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70f00b85e0 0x7f70f00cda20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f70dc006fd0 tx=0x7f70dc01c4f0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2931389275 shutdown_connections 2023-12-19T18:18:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70f6f7b640 1 --2- 172.21.15.26:0/2931389275 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70f00b85e0 0x7f70f00cda20 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70f6f7b640 1 --2- 172.21.15.26:0/2931389275 >> v2:172.21.15.160:3300/0 conn(0x7f70f015cf50 0x7f70f00b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2931389275 >> 172.21.15.26:0/2931389275 conn(0x7f70f00658e0 msgr2=0x7f70f0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2931389275 shutdown_connections 2023-12-19T18:18:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2931389275 wait complete. 2023-12-19T18:18:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70f6f7b640 1 Processor -- start 2023-12-19T18:18:41.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.584+0000 7f70f6f7b640 1 -- start start 2023-12-19T18:18:41.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70f6f7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70f00b1a00 0x7f70f00b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70f6f7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70f00b85e0 0x7f70f00c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70f6f7b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70f015cf50 0x7f70f0154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70f6f7b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00b49a0 con 0x7f70f00b85e0 2023-12-19T18:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70f6f7b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00b5490 con 0x7f70f015cf50 2023-12-19T18:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70f6f7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f00c9ea0 con 0x7f70f00b1a00 2023-12-19T18:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70f00b85e0 0x7f70f00c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70effff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70f00b85e0 0x7f70f00c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36238/0 (socket says 172.21.15.26:36238) 2023-12-19T18:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70effff640 1 -- 172.21.15.26:0/2275015492 learned_addr learned my addr 172.21.15.26:0/2275015492 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70f4d17640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70f00b1a00 0x7f70f00b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70f5518640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70f015cf50 0x7f70f0154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70effff640 1 -- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70f00b1a00 msgr2=0x7f70f00b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70effff640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70f00b1a00 0x7f70f00b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70effff640 1 -- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70f015cf50 msgr2=0x7f70f0154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70effff640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70f015cf50 0x7f70f0154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70effff640 1 -- 172.21.15.26:0/2275015492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc009480 con 0x7f70f00b85e0 2023-12-19T18:18:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70effff640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70f00b85e0 0x7f70f00c8000 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f70dc01a120 tx=0x7f70dc019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70ed7fa640 1 -- 172.21.15.26:0/2275015492 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc0aab40 con 0x7f70f00b85e0 2023-12-19T18:18:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70ed7fa640 1 -- 172.21.15.26:0/2275015492 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70dc0aaca0 con 0x7f70f00b85e0 2023-12-19T18:18:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70ed7fa640 1 -- 172.21.15.26:0/2275015492 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70dc003860 con 0x7f70f00b85e0 2023-12-19T18:18:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2275015492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f00ca180 con 0x7f70f00b85e0 2023-12-19T18:18:41.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2275015492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f00ca740 con 0x7f70f00b85e0 2023-12-19T18:18:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2275015492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f0066710 con 0x7f70f00b85e0 2023-12-19T18:18:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.588+0000 7f70ed7fa640 1 -- 172.21.15.26:0/2275015492 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 144) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f70dc003bd0 con 0x7f70f00b85e0 2023-12-19T18:18:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.592+0000 7f70ed7fa640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f70d4065af0 0x7f70d4067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.592+0000 7f70ed7fa640 1 -- 172.21.15.26:0/2275015492 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 7243+0+0 (secure 0 0 0) 0x7f70dc11e5c0 con 0x7f70f00b85e0 2023-12-19T18:18:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.592+0000 7f70f4d17640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f70d4065af0 0x7f70d4067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.592+0000 7f70f4d17640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f70d4065af0 0x7f70d4067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f70e4011340 tx=0x7f70e400b040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.596+0000 7f70ed7fa640 1 -- 172.21.15.26:0/2275015492 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f70dc0eb8b0 con 0x7f70f00b85e0 2023-12-19T18:18:41.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:41.784+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2275015492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f70f00ca930 con 0x7f70f00b85e0 2023-12-19T18:18:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.408+0000 7f70ed7fa640 1 -- 172.21.15.26:0/2275015492 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v130) v1 ==== 104+0+0 (secure 0 0 0) 0x7f70dc0c3020 con 0x7f70f00b85e0 2023-12-19T18:18:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.412+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f70d4065af0 msgr2=0x7f70d4067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.412+0000 7f70f6f7b640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f70d4065af0 0x7f70d4067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f70e4011340 tx=0x7f70e400b040 comp rx=0 tx=0).stop 2023-12-19T18:18:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.412+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2275015492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70f00b85e0 msgr2=0x7f70f00c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.412+0000 7f70f6f7b640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70f00b85e0 0x7f70f00c8000 secure :-1 s=READY pgs=2428 cs=0 l=1 rev1=1 crypto rx=0x7f70dc01a120 tx=0x7f70dc019290 comp rx=0 tx=0).stop 2023-12-19T18:18:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.412+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2275015492 shutdown_connections 2023-12-19T18:18:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.412+0000 7f70f6f7b640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f70d4065af0 0x7f70d4067fb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.412+0000 7f70f6f7b640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70f015cf50 0x7f70f0154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.412+0000 7f70f6f7b640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70f00b85e0 0x7f70f00c8000 unknown :-1 s=CLOSED pgs=2428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.412+0000 7f70f6f7b640 1 --2- 172.21.15.26:0/2275015492 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f70f00b1a00 0x7f70f00b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.412+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2275015492 >> 172.21.15.26:0/2275015492 conn(0x7f70f00658e0 msgr2=0x7f70f0067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.412+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2275015492 shutdown_connections 2023-12-19T18:18:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.412+0000 7f70f6f7b640 1 -- 172.21.15.26:0/2275015492 wait complete. 2023-12-19T18:18:42.435 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:18:42.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.628+0000 7f80f2855640 1 Processor -- start 2023-12-19T18:18:42.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.628+0000 7f80f2855640 1 -- start start 2023-12-19T18:18:42.631 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.632+0000 7f80f2855640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80ec0c8d90 0x7f80ec0c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:42.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.632+0000 7f80f2855640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ec0c9720 con 0x7f80ec1597d0 2023-12-19T18:18:42.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.632+0000 7f80f2855640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ec06c4c0 con 0x7f80ec070ff0 2023-12-19T18:18:42.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.632+0000 7f80f2855640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80ec0cb8d0 con 0x7f80ec0c8d90 2023-12-19T18:18:42.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.632+0000 7f80f0df2640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f80ec0c8d90 0x7f80ec0c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:42.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.632+0000 7f80ebfff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f80ec1597d0 0x7f80ec06ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34082/0 (socket says 172.21.15.26:34082) 2023-12-19T18:18:42.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.632+0000 7f80ebfff640 1 -- 172.21.15.26:0/2419765882 learned_addr learned my addr 172.21.15.26:0/2419765882 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:42.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.632+0000 7f80f0df2640 1 -- 172.21.15.26:0/2419765882 >> v1:172.21.15.160:6789/0 conn(0x7f80ec070ff0 legacy=0x7f80ec0cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:42.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.632+0000 7f80f0df2640 1 -- 172.21.15.26:0/2419765882 >> v1:172.21.15.26:6789/0 conn(0x7f80ec1597d0 legacy=0x7f80ec06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:42.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.632+0000 7f80f0df2640 1 -- 172.21.15.26:0/2419765882 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ec0703a0 con 0x7f80ec0c8d90 2023-12-19T18:18:42.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.632+0000 7f80eaffd640 1 -- 172.21.15.26:0/2419765882 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2030754003 0 0) 0x7f80dc001120 con 0x7f80ec1597d0 2023-12-19T18:18:42.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f0df2640 1 --2- 172.21.15.26:0/2419765882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80ec0c8d90 0x7f80ec0c91e0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009d90 tx=0x7f80e401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5493cc7114584904 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:42.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80eaffd640 1 -- 172.21.15.26:0/2419765882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40a5830 con 0x7f80ec0c8d90 2023-12-19T18:18:42.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80eaffd640 1 -- 172.21.15.26:0/2419765882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e40a5990 con 0x7f80ec0c8d90 2023-12-19T18:18:42.635 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80eaffd640 1 -- 172.21.15.26:0/2419765882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40a5d20 con 0x7f80ec0c8d90 2023-12-19T18:18:42.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 -- 172.21.15.26:0/2419765882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80ec0c8d90 msgr2=0x7f80ec0c91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:42.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 --2- 172.21.15.26:0/2419765882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80ec0c8d90 0x7f80ec0c91e0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f80e4009d90 tx=0x7f80e401c880 comp rx=0 tx=0).stop 2023-12-19T18:18:42.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 -- 172.21.15.26:0/2419765882 shutdown_connections 2023-12-19T18:18:42.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 --2- 172.21.15.26:0/2419765882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80ec0c8d90 0x7f80ec0c91e0 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:42.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 -- 172.21.15.26:0/2419765882 >> 172.21.15.26:0/2419765882 conn(0x7f80ec0b0270 msgr2=0x7f80ec0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:42.636 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 -- 172.21.15.26:0/2419765882 shutdown_connections 2023-12-19T18:18:42.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 -- 172.21.15.26:0/2419765882 wait complete. 2023-12-19T18:18:42.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 Processor -- start 2023-12-19T18:18:42.637 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 -- start start 2023-12-19T18:18:42.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80ec070ff0 0x7f80ec0c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:42.639 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80ec0c8d90 0x7f80ec0ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80ec1597d0 0x7f80ec151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec0c7a50 con 0x7f80ec0c8d90 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec0c6e40 con 0x7f80ec1597d0 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80f2855640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec0c7120 con 0x7f80ec070ff0 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.636+0000 7f80eb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80ec0c8d90 0x7f80ec0ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80eb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80ec0c8d90 0x7f80ec0ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36256/0 (socket says 172.21.15.26:36256) 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80eb7fe640 1 -- 172.21.15.26:0/845997124 learned_addr learned my addr 172.21.15.26:0/845997124 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80f0df2640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80ec1597d0 0x7f80ec151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80ebfff640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80ec070ff0 0x7f80ec0c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80eb7fe640 1 -- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80ec070ff0 msgr2=0x7f80ec0c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80eb7fe640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80ec070ff0 0x7f80ec0c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80eb7fe640 1 -- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80ec1597d0 msgr2=0x7f80ec151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80eb7fe640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80ec1597d0 0x7f80ec151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:42.640 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80eb7fe640 1 -- 172.21.15.26:0/845997124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e4009a70 con 0x7f80ec0c8d90 2023-12-19T18:18:42.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80eb7fe640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80ec0c8d90 0x7f80ec0ae030 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b410 tx=0x7f80e000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:42.641 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80e8ff9640 1 -- 172.21.15.26:0/845997124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0004020 con 0x7f80ec0c8d90 2023-12-19T18:18:42.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80e8ff9640 1 -- 172.21.15.26:0/845997124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e0004180 con 0x7f80ec0c8d90 2023-12-19T18:18:42.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80e8ff9640 1 -- 172.21.15.26:0/845997124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e0011630 con 0x7f80ec0c8d90 2023-12-19T18:18:42.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80f2855640 1 -- 172.21.15.26:0/845997124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ec0c7400 con 0x7f80ec0c8d90 2023-12-19T18:18:42.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80f2855640 1 -- 172.21.15.26:0/845997124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ec0ae570 con 0x7f80ec0c8d90 2023-12-19T18:18:42.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80f2855640 1 -- 172.21.15.26:0/845997124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ec0c7c00 con 0x7f80ec0c8d90 2023-12-19T18:18:42.642 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.640+0000 7f80e8ff9640 1 -- 172.21.15.26:0/845997124 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 144) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f80e0011790 con 0x7f80ec0c8d90 2023-12-19T18:18:42.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.644+0000 7f80e8ff9640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f80d8065b20 0x7f80d8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:42.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.644+0000 7f80e8ff9640 1 -- 172.21.15.26:0/845997124 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(257..257 src has 1..257) v4 ==== 7243+0+0 (secure 0 0 0) 0x7f80e00833c0 con 0x7f80ec0c8d90 2023-12-19T18:18:42.643 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.644+0000 7f80ebfff640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f80d8065b20 0x7f80d8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:42.644 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.644+0000 7f80ebfff640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f80d8065b20 0x7f80d8067fe0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f80dc000f70 tx=0x7f80dc007ba0 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:42.647 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.648+0000 7f80e8ff9640 1 -- 172.21.15.26:0/845997124 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f80ec0c7c00 con 0x7f80ec0c8d90 2023-12-19T18:18:42.660 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:18:42.666 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:18:42.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:42.808+0000 7f80f2855640 1 -- 172.21.15.26:0/845997124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f80ec074270 con 0x7f80ec0c8d90 2023-12-19T18:18:43.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.424+0000 7f80e8ff9640 1 -- 172.21.15.26:0/845997124 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f80e0050ba0 con 0x7f80ec0c8d90 2023-12-19T18:18:43.425 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.424+0000 7f80e8ff9640 1 -- 172.21.15.26:0/845997124 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v258) v1 ==== 181+0+0 (secure 0 0 0) 0x7f80e00802e0 con 0x7f80ec0c8d90 2023-12-19T18:18:43.425 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:18:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.428+0000 7f80f2855640 1 -- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f80d8065b20 msgr2=0x7f80d8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:43.428 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.428+0000 7f80f2855640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f80d8065b20 0x7f80d8067fe0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f80dc000f70 tx=0x7f80dc007ba0 comp rx=0 tx=0).stop 2023-12-19T18:18:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.428+0000 7f80f2855640 1 -- 172.21.15.26:0/845997124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80ec0c8d90 msgr2=0x7f80ec0ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.428+0000 7f80f2855640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80ec0c8d90 0x7f80ec0ae030 secure :-1 s=READY pgs=2431 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b410 tx=0x7f80e000b8d0 comp rx=0 tx=0).stop 2023-12-19T18:18:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.428+0000 7f80f2855640 1 -- 172.21.15.26:0/845997124 shutdown_connections 2023-12-19T18:18:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.428+0000 7f80f2855640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f80d8065b20 0x7f80d8067fe0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.428+0000 7f80f2855640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f80ec1597d0 0x7f80ec151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.428+0000 7f80f2855640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80ec0c8d90 0x7f80ec0ae030 unknown :-1 s=CLOSED pgs=2431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.428+0000 7f80f2855640 1 --2- 172.21.15.26:0/845997124 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f80ec070ff0 0x7f80ec0c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.428+0000 7f80f2855640 1 -- 172.21.15.26:0/845997124 >> 172.21.15.26:0/845997124 conn(0x7f80ec0b0270 msgr2=0x7f80ec0cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.428+0000 7f80f2855640 1 -- 172.21.15.26:0/845997124 shutdown_connections 2023-12-19T18:18:43.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.428+0000 7f80f2855640 1 -- 172.21.15.26:0/845997124 wait complete. 2023-12-19T18:18:43.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:18:43.614 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.612+0000 7faf561a6640 1 Processor -- start 2023-12-19T18:18:43.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.612+0000 7faf561a6640 1 -- start start 2023-12-19T18:18:43.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf561a6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf5015cc00 0x7faf500cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:43.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf561a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf500cc850 0x7faf500ac860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:43.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf561a6640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf500ad210 con 0x7faf500acda0 2023-12-19T18:18:43.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf561a6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf500ccce0 con 0x7faf5015cc00 2023-12-19T18:18:43.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf561a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf500ad440 con 0x7faf500cc850 2023-12-19T18:18:43.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf4f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf5015cc00 0x7faf500cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:43.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf4ffff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faf500acda0 0x7faf5006b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34098/0 (socket says 172.21.15.26:34098) 2023-12-19T18:18:43.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf4f7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf5015cc00 0x7faf500cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36258/0 (socket says 172.21.15.26:36258) 2023-12-19T18:18:43.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf4ffff640 1 -- 172.21.15.26:0/2100089136 learned_addr learned my addr 172.21.15.26:0/2100089136 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:43.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf4effd640 1 --2- 172.21.15.26:0/2100089136 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf500cc850 0x7faf500ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:43.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf4e7fc640 1 -- 172.21.15.26:0/2100089136 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865158304 0 0) 0x7faf500ad210 con 0x7faf500acda0 2023-12-19T18:18:43.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf4f7fe640 1 -- 172.21.15.26:0/2100089136 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf500cc850 msgr2=0x7faf500ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:43.618 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf4f7fe640 1 --2- 172.21.15.26:0/2100089136 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf500cc850 0x7faf500ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:43.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf4f7fe640 1 -- 172.21.15.26:0/2100089136 >> v1:172.21.15.26:6789/0 conn(0x7faf500acda0 legacy=0x7faf5006b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:43.619 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.616+0000 7faf4f7fe640 1 -- 172.21.15.26:0/2100089136 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500ad720 con 0x7faf5015cc00 2023-12-19T18:18:43.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.620+0000 7faf4f7fe640 1 --2- 172.21.15.26:0/2100089136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf5015cc00 0x7faf500cc310 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7faf400097c0 tx=0x7faf4001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9be5a0119189b3f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:43.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.620+0000 7faf4e7fc640 1 -- 172.21.15.26:0/2100089136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400ab680 con 0x7faf5015cc00 2023-12-19T18:18:43.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.620+0000 7faf4e7fc640 1 -- 172.21.15.26:0/2100089136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf400ab7e0 con 0x7faf5015cc00 2023-12-19T18:18:43.621 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.620+0000 7faf4e7fc640 1 -- 172.21.15.26:0/2100089136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400abb70 con 0x7faf5015cc00 2023-12-19T18:18:43.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.620+0000 7faf561a6640 1 -- 172.21.15.26:0/2100089136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf5015cc00 msgr2=0x7faf500cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:43.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.620+0000 7faf561a6640 1 --2- 172.21.15.26:0/2100089136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf5015cc00 0x7faf500cc310 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7faf400097c0 tx=0x7faf4001c480 comp rx=0 tx=0).stop 2023-12-19T18:18:43.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.620+0000 7faf561a6640 1 -- 172.21.15.26:0/2100089136 shutdown_connections 2023-12-19T18:18:43.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.620+0000 7faf561a6640 1 --2- 172.21.15.26:0/2100089136 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf500cc850 0x7faf500ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:43.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.620+0000 7faf561a6640 1 --2- 172.21.15.26:0/2100089136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf5015cc00 0x7faf500cc310 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:43.622 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.620+0000 7faf561a6640 1 -- 172.21.15.26:0/2100089136 >> 172.21.15.26:0/2100089136 conn(0x7faf500b0270 msgr2=0x7faf500b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:43.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.620+0000 7faf561a6640 1 -- 172.21.15.26:0/2100089136 shutdown_connections 2023-12-19T18:18:43.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.620+0000 7faf561a6640 1 -- 172.21.15.26:0/2100089136 wait complete. 2023-12-19T18:18:43.623 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf561a6640 1 Processor -- start 2023-12-19T18:18:43.624 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf561a6640 1 -- start start 2023-12-19T18:18:43.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf561a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf500acda0 0x7faf501581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:43.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf561a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf500cc850 0x7faf50157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:43.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf561a6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf5015cc00 0x7faf500c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:43.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf561a6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf501575c0 con 0x7faf5015cc00 2023-12-19T18:18:43.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf561a6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf500c5240 con 0x7faf500acda0 2023-12-19T18:18:43.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf561a6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf50157840 con 0x7faf500cc850 2023-12-19T18:18:43.626 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf4ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf5015cc00 0x7faf500c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:43.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf4ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf5015cc00 0x7faf500c4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36270/0 (socket says 172.21.15.26:36270) 2023-12-19T18:18:43.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf4ffff640 1 -- 172.21.15.26:0/516458787 learned_addr learned my addr 172.21.15.26:0/516458787 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:43.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf4effd640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf500cc850 0x7faf50157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:43.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf4f7fe640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf500acda0 0x7faf501581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:43.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf4ffff640 1 -- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf500cc850 msgr2=0x7faf50157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:43.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf4ffff640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf500cc850 0x7faf50157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:43.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf4ffff640 1 -- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf500acda0 msgr2=0x7faf501581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:43.627 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf4ffff640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf500acda0 0x7faf501581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:43.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf4ffff640 1 -- 172.21.15.26:0/516458787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf40009420 con 0x7faf5015cc00 2023-12-19T18:18:43.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf4f7fe640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf500acda0 0x7faf501581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:43.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf4effd640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf500cc850 0x7faf50157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:43.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf4ffff640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf5015cc00 0x7faf500c4d00 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7faf4800c700 tx=0x7faf4800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:43.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf37fff640 1 -- 172.21.15.26:0/516458787 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4800ce10 con 0x7faf5015cc00 2023-12-19T18:18:43.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf37fff640 1 -- 172.21.15.26:0/516458787 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4800a640 con 0x7faf5015cc00 2023-12-19T18:18:43.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf37fff640 1 -- 172.21.15.26:0/516458787 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4800aa70 con 0x7faf5015cc00 2023-12-19T18:18:43.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf561a6640 1 -- 172.21.15.26:0/516458787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf50157b20 con 0x7faf5015cc00 2023-12-19T18:18:43.628 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.624+0000 7faf561a6640 1 -- 172.21.15.26:0/516458787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf50153570 con 0x7faf5015cc00 2023-12-19T18:18:43.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.628+0000 7faf561a6640 1 -- 172.21.15.26:0/516458787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf501530f0 con 0x7faf5015cc00 2023-12-19T18:18:43.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.628+0000 7faf37fff640 1 -- 172.21.15.26:0/516458787 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faf4800abd0 con 0x7faf5015cc00 2023-12-19T18:18:43.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.628+0000 7faf37fff640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf3c065b80 0x7faf3c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:43.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.628+0000 7faf37fff640 1 -- 172.21.15.26:0/516458787 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(258..258 src has 1..258) v4 ==== 6805+0+0 (secure 0 0 0) 0x7faf48082cb0 con 0x7faf5015cc00 2023-12-19T18:18:43.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.628+0000 7faf4f7fe640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf3c065b80 0x7faf3c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:43.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.628+0000 7faf4f7fe640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf3c065b80 0x7faf3c068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faf400af000 tx=0x7faf400a7000 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:43.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.632+0000 7faf37fff640 1 -- 172.21.15.26:0/516458787 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faf501530f0 con 0x7faf5015cc00 2023-12-19T18:18:43.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:43.788+0000 7faf561a6640 1 -- 172.21.15.26:0/516458787 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faf501530f0 con 0x7faf5015cc00 2023-12-19T18:18:44.423 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.420+0000 7faf37fff640 1 -- 172.21.15.26:0/516458787 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v259) v1 ==== 169+0+0 (secure 0 0 0) 0x7faf501530f0 con 0x7faf5015cc00 2023-12-19T18:18:44.423 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2023-12-19T18:18:44.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.424+0000 7faf561a6640 1 -- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf3c065b80 msgr2=0x7faf3c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:44.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.424+0000 7faf561a6640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf3c065b80 0x7faf3c068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faf400af000 tx=0x7faf400a7000 comp rx=0 tx=0).stop 2023-12-19T18:18:44.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.424+0000 7faf561a6640 1 -- 172.21.15.26:0/516458787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf5015cc00 msgr2=0x7faf500c4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:44.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.424+0000 7faf561a6640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf5015cc00 0x7faf500c4d00 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7faf4800c700 tx=0x7faf4800cbc0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.424+0000 7faf561a6640 1 -- 172.21.15.26:0/516458787 shutdown_connections 2023-12-19T18:18:44.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.424+0000 7faf561a6640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf3c065b80 0x7faf3c068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.424+0000 7faf561a6640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf5015cc00 0x7faf500c4d00 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.424+0000 7faf561a6640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf500cc850 0x7faf50157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.424+0000 7faf561a6640 1 --2- 172.21.15.26:0/516458787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf500acda0 0x7faf501581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.424+0000 7faf561a6640 1 -- 172.21.15.26:0/516458787 >> 172.21.15.26:0/516458787 conn(0x7faf500b0270 msgr2=0x7faf5015d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:44.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.424+0000 7faf561a6640 1 -- 172.21.15.26:0/516458787 shutdown_connections 2023-12-19T18:18:44.427 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.424+0000 7faf561a6640 1 -- 172.21.15.26:0/516458787 wait complete. 2023-12-19T18:18:44.450 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-19T18:18:44.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff632441640 1 Processor -- start 2023-12-19T18:18:44.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff632441640 1 -- start start 2023-12-19T18:18:44.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff632441640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff62c0cb150 0x7ff62c0cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:44.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff632441640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff62c15e690 0x7ff62c15eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:44.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff632441640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62c0ce3f0 con 0x7ff62c1585f0 2023-12-19T18:18:44.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff632441640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62c0cbb40 con 0x7ff62c15e690 2023-12-19T18:18:44.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff632441640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62c0b47a0 con 0x7ff62c0cb150 2023-12-19T18:18:44.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff6309de640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff62c15e690 0x7ff62c15eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:44.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff6309de640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff62c15e690 0x7ff62c15eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36276/0 (socket says 172.21.15.26:36276) 2023-12-19T18:18:44.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff6309de640 1 -- 172.21.15.26:0/2681581368 learned_addr learned my addr 172.21.15.26:0/2681581368 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:44.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff62b7fe640 1 --2- 172.21.15.26:0/2681581368 >> v2:172.21.15.160:3300/0 conn(0x7ff62c0cb150 0x7ff62c0cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:44.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff6309de640 1 -- 172.21.15.26:0/2681581368 >> v2:172.21.15.160:3300/0 conn(0x7ff62c0cb150 msgr2=0x7ff62c0cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:44.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff6309de640 1 --2- 172.21.15.26:0/2681581368 >> v2:172.21.15.160:3300/0 conn(0x7ff62c0cb150 0x7ff62c0cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff6309de640 1 -- 172.21.15.26:0/2681581368 >> v1:172.21.15.160:6789/0 conn(0x7ff62c1585f0 legacy=0x7ff62c0cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:44.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.596+0000 7ff6309de640 1 -- 172.21.15.26:0/2681581368 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c15b1b0 con 0x7ff62c15e690 2023-12-19T18:18:44.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.600+0000 7ff6309de640 1 --2- 172.21.15.26:0/2681581368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff62c15e690 0x7ff62c15eae0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7ff62c15b0a0 tx=0x7ff62401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df677bca8866f26 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:44.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.600+0000 7ff62affd640 1 -- 172.21.15.26:0/2681581368 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240a57d0 con 0x7ff62c15e690 2023-12-19T18:18:44.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.600+0000 7ff62affd640 1 -- 172.21.15.26:0/2681581368 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6240a5930 con 0x7ff62c15e690 2023-12-19T18:18:44.601 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.600+0000 7ff62affd640 1 -- 172.21.15.26:0/2681581368 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240a5cc0 con 0x7ff62c15e690 2023-12-19T18:18:44.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.600+0000 7ff632441640 1 -- 172.21.15.26:0/2681581368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff62c15e690 msgr2=0x7ff62c15eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:44.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.600+0000 7ff632441640 1 --2- 172.21.15.26:0/2681581368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff62c15e690 0x7ff62c15eae0 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7ff62c15b0a0 tx=0x7ff62401c910 comp rx=0 tx=0).stop 2023-12-19T18:18:44.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.600+0000 7ff632441640 1 -- 172.21.15.26:0/2681581368 shutdown_connections 2023-12-19T18:18:44.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.600+0000 7ff632441640 1 --2- 172.21.15.26:0/2681581368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff62c15e690 0x7ff62c15eae0 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.600+0000 7ff632441640 1 --2- 172.21.15.26:0/2681581368 >> v2:172.21.15.160:3300/0 conn(0x7ff62c0cb150 0x7ff62c0cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.600+0000 7ff632441640 1 -- 172.21.15.26:0/2681581368 >> 172.21.15.26:0/2681581368 conn(0x7ff62c066160 msgr2=0x7ff62c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:44.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.600+0000 7ff632441640 1 -- 172.21.15.26:0/2681581368 shutdown_connections 2023-12-19T18:18:44.602 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.600+0000 7ff632441640 1 -- 172.21.15.26:0/2681581368 wait complete. 2023-12-19T18:18:44.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff632441640 1 Processor -- start 2023-12-19T18:18:44.603 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff632441640 1 -- start start 2023-12-19T18:18:44.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff632441640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff62c0cb150 0x7ff62c1598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:44.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff632441640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff62c1585f0 0x7ff62c151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:44.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff632441640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff62c15e690 0x7ff62c0c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:44.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff632441640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c06cc60 con 0x7ff62c15e690 2023-12-19T18:18:44.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff632441640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c06c3c0 con 0x7ff62c1585f0 2023-12-19T18:18:44.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff632441640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c06c6a0 con 0x7ff62c0cb150 2023-12-19T18:18:44.606 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff6309de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff62c15e690 0x7ff62c0c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff6309de640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff62c15e690 0x7ff62c0c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36286/0 (socket says 172.21.15.26:36286) 2023-12-19T18:18:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff6309de640 1 -- 172.21.15.26:0/2728594246 learned_addr learned my addr 172.21.15.26:0/2728594246 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff62b7fe640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff62c1585f0 0x7ff62c151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff62bfff640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff62c0cb150 0x7ff62c1598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff6309de640 1 -- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff62c0cb150 msgr2=0x7ff62c1598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff6309de640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff62c0cb150 0x7ff62c1598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff6309de640 1 -- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff62c1585f0 msgr2=0x7ff62c151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff6309de640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff62c1585f0 0x7ff62c151250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff6309de640 1 -- 172.21.15.26:0/2728594246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff624009b30 con 0x7ff62c15e690 2023-12-19T18:18:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff6309de640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff62c15e690 0x7ff62c0c7b20 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7ff624019520 tx=0x7ff6240196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff628ff9640 1 -- 172.21.15.26:0/2728594246 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240b8670 con 0x7ff62c15e690 2023-12-19T18:18:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff628ff9640 1 -- 172.21.15.26:0/2728594246 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6240b87f0 con 0x7ff62c15e690 2023-12-19T18:18:44.607 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff628ff9640 1 -- 172.21.15.26:0/2728594246 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240b8ba0 con 0x7ff62c15e690 2023-12-19T18:18:44.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff632441640 1 -- 172.21.15.26:0/2728594246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62c06c980 con 0x7ff62c15e690 2023-12-19T18:18:44.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.604+0000 7ff632441640 1 -- 172.21.15.26:0/2728594246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62c0c8060 con 0x7ff62c15e690 2023-12-19T18:18:44.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.608+0000 7ff632441640 1 -- 172.21.15.26:0/2728594246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62c15a090 con 0x7ff62c15e690 2023-12-19T18:18:44.608 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.608+0000 7ff628ff9640 1 -- 172.21.15.26:0/2728594246 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff6240b8d00 con 0x7ff62c15e690 2023-12-19T18:18:44.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.608+0000 7ff628ff9640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff618065b80 0x7ff618068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:44.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.608+0000 7ff628ff9640 1 -- 172.21.15.26:0/2728594246 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7ff62411eb20 con 0x7ff62c15e690 2023-12-19T18:18:44.609 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.608+0000 7ff62bfff640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff618065b80 0x7ff618068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:44.610 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.608+0000 7ff62bfff640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff618065b80 0x7ff618068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff62c06cfa0 tx=0x7ff61c009040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:44.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.612+0000 7ff628ff9640 1 -- 172.21.15.26:0/2728594246 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff6240ec250 con 0x7ff62c15e690 2023-12-19T18:18:44.776 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.776+0000 7ff632441640 1 -- 172.21.15.26:0/2728594246 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7ff62c074270 con 0x7ff62c15e690 2023-12-19T18:18:44.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.780+0000 7ff628ff9640 1 -- 172.21.15.26:0/2728594246 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v85) v1 ==== 120+0+0 (secure 0 0 0) 0x7ff6240ef870 con 0x7ff62c15e690 2023-12-19T18:18:44.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.784+0000 7ff632441640 1 -- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff618065b80 msgr2=0x7ff618068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:44.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.784+0000 7ff632441640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff618065b80 0x7ff618068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff62c06cfa0 tx=0x7ff61c009040 comp rx=0 tx=0).stop 2023-12-19T18:18:44.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.784+0000 7ff632441640 1 -- 172.21.15.26:0/2728594246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff62c15e690 msgr2=0x7ff62c0c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:44.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.784+0000 7ff632441640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff62c15e690 0x7ff62c0c7b20 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7ff624019520 tx=0x7ff6240196b0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.784+0000 7ff632441640 1 -- 172.21.15.26:0/2728594246 shutdown_connections 2023-12-19T18:18:44.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.784+0000 7ff632441640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7ff618065b80 0x7ff618068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.784+0000 7ff632441640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff62c15e690 0x7ff62c0c7b20 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.784+0000 7ff632441640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff62c1585f0 0x7ff62c151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.784+0000 7ff632441640 1 --2- 172.21.15.26:0/2728594246 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7ff62c0cb150 0x7ff62c1598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.784+0000 7ff632441640 1 -- 172.21.15.26:0/2728594246 >> 172.21.15.26:0/2728594246 conn(0x7ff62c066160 msgr2=0x7ff62c15da00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:44.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.784+0000 7ff632441640 1 -- 172.21.15.26:0/2728594246 shutdown_connections 2023-12-19T18:18:44.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.784+0000 7ff632441640 1 -- 172.21.15.26:0/2728594246 wait complete. 2023-12-19T18:18:44.806 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:18:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f3005d52640 1 Processor -- start 2023-12-19T18:18:44.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f3005d52640 1 -- start start 2023-12-19T18:18:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f3005d52640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30001599e0 0x7f300006f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:44.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f3005d52640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f30000cb150 0x7f30000cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f3005d52640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f300015d530 0x7f300015d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f3005d52640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30000cbb20 con 0x7f30001599e0 2023-12-19T18:18:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f3005d52640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300015e240 con 0x7f30000cb150 2023-12-19T18:18:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f3005d52640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30000c9520 con 0x7f300015d530 2023-12-19T18:18:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f2fff7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30001599e0 0x7f300006f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f2fff7fe640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30001599e0 0x7f300006f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36292/0 (socket says 172.21.15.26:36292) 2023-12-19T18:18:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f2fff7fe640 1 -- 172.21.15.26:0/2600439915 learned_addr learned my addr 172.21.15.26:0/2600439915 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f2fff7fe640 1 -- 172.21.15.26:0/2600439915 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f300015d530 msgr2=0x7f300015d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f2fff7fe640 1 --2- 172.21.15.26:0/2600439915 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f300015d530 0x7f300015d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f2fff7fe640 1 -- 172.21.15.26:0/2600439915 >> v2:172.21.15.160:3300/0 conn(0x7f30000cb150 msgr2=0x7f30000cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f2fff7fe640 1 --2- 172.21.15.26:0/2600439915 >> v2:172.21.15.160:3300/0 conn(0x7f30000cb150 0x7f30000cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.954 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.952+0000 7f2fff7fe640 1 -- 172.21.15.26:0/2600439915 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30000c9800 con 0x7f30001599e0 2023-12-19T18:18:44.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.956+0000 7f2fff7fe640 1 --2- 172.21.15.26:0/2600439915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30001599e0 0x7f300006f6d0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f30000cc220 tx=0x7f2ff001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f7b7cb26ab0c32 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.956+0000 7f2ffe7fc640 1 -- 172.21.15.26:0/2600439915 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00ac630 con 0x7f30001599e0 2023-12-19T18:18:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.956+0000 7f2ffe7fc640 1 -- 172.21.15.26:0/2600439915 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff00ac790 con 0x7f30001599e0 2023-12-19T18:18:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.956+0000 7f2ffe7fc640 1 -- 172.21.15.26:0/2600439915 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff00acb20 con 0x7f30001599e0 2023-12-19T18:18:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.956+0000 7f3005d52640 1 -- 172.21.15.26:0/2600439915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30001599e0 msgr2=0x7f300006f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:44.957 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.956+0000 7f3005d52640 1 --2- 172.21.15.26:0/2600439915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30001599e0 0x7f300006f6d0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7f30000cc220 tx=0x7f2ff001c500 comp rx=0 tx=0).stop 2023-12-19T18:18:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.956+0000 7f3005d52640 1 -- 172.21.15.26:0/2600439915 shutdown_connections 2023-12-19T18:18:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.956+0000 7f3005d52640 1 --2- 172.21.15.26:0/2600439915 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f300015d530 0x7f300015d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.956+0000 7f3005d52640 1 --2- 172.21.15.26:0/2600439915 >> v2:172.21.15.160:3300/0 conn(0x7f30000cb150 0x7f30000cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.956+0000 7f3005d52640 1 --2- 172.21.15.26:0/2600439915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30001599e0 0x7f300006f6d0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.956+0000 7f3005d52640 1 -- 172.21.15.26:0/2600439915 >> 172.21.15.26:0/2600439915 conn(0x7f3000066160 msgr2=0x7f30000b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.956+0000 7f3005d52640 1 -- 172.21.15.26:0/2600439915 shutdown_connections 2023-12-19T18:18:44.958 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.956+0000 7f3005d52640 1 -- 172.21.15.26:0/2600439915 wait complete. 2023-12-19T18:18:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f3005d52640 1 Processor -- start 2023-12-19T18:18:44.959 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f3005d52640 1 -- start start 2023-12-19T18:18:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f3005d52640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30000cb150 0x7f30000c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f3005d52640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30001599e0 0x7f30000c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f3005d52640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300015d530 0x7f30000c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f3005d52640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30000c5e80 con 0x7f300015d530 2023-12-19T18:18:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f3005d52640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30000c25e0 con 0x7f30000cb150 2023-12-19T18:18:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f3005d52640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30000c28c0 con 0x7f30001599e0 2023-12-19T18:18:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300015d530 0x7f30000c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2ffffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300015d530 0x7f30000c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36296/0 (socket says 172.21.15.26:36296) 2023-12-19T18:18:44.961 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2ffffff640 1 -- 172.21.15.26:0/1669791462 learned_addr learned my addr 172.21.15.26:0/1669791462 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2fff7fe640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30000cb150 0x7f30000c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2ffeffd640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30001599e0 0x7f30000c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2ffffff640 1 -- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30001599e0 msgr2=0x7f30000c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2ffffff640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30001599e0 0x7f30000c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2ffffff640 1 -- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30000cb150 msgr2=0x7f30000c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2ffffff640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30000cb150 0x7f30000c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:44.962 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2ffffff640 1 -- 172.21.15.26:0/1669791462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff00094d0 con 0x7f300015d530 2023-12-19T18:18:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2ffffff640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300015d530 0x7f30000c84c0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800b970 tx=0x7f2ff800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2fe7fff640 1 -- 172.21.15.26:0/1669791462 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff800cb40 con 0x7f300015d530 2023-12-19T18:18:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2fe7fff640 1 -- 172.21.15.26:0/1669791462 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff800ccc0 con 0x7f300015d530 2023-12-19T18:18:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f3005d52640 1 -- 172.21.15.26:0/1669791462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30000c7fa0 con 0x7f300015d530 2023-12-19T18:18:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f2fe7fff640 1 -- 172.21.15.26:0/1669791462 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8014680 con 0x7f300015d530 2023-12-19T18:18:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f3005d52640 1 -- 172.21.15.26:0/1669791462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30000c47e0 con 0x7f300015d530 2023-12-19T18:18:44.963 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.960+0000 7f3005d52640 1 -- 172.21.15.26:0/1669791462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30000c81e0 con 0x7f300015d530 2023-12-19T18:18:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.964+0000 7f2fe7fff640 1 -- 172.21.15.26:0/1669791462 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2ff801d3b0 con 0x7f300015d530 2023-12-19T18:18:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.964+0000 7f2fe7fff640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2fec065bb0 0x7f2fec068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:44.964 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.964+0000 7f2fe7fff640 1 -- 172.21.15.26:0/1669791462 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f2ff8083450 con 0x7f300015d530 2023-12-19T18:18:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.964+0000 7f2fff7fe640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2fec065bb0 0x7f2fec068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:44.965 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.964+0000 7f2fff7fe640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2fec065bb0 0x7f2fec068070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00ae000 tx=0x7f2ff0004740 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:44.968 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:44.968+0000 7f2fe7fff640 1 -- 172.21.15.26:0/1669791462 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f30000c81e0 con 0x7f300015d530 2023-12-19T18:18:45.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.124+0000 7f3005d52640 1 -- 172.21.15.26:0/1669791462 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f30000c81e0 con 0x7f300015d530 2023-12-19T18:18:45.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.128+0000 7f2fe7fff640 1 -- 172.21.15.26:0/1669791462 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v86) v1 ==== 104+0+0 (secure 0 0 0) 0x7f30000c81e0 con 0x7f300015d530 2023-12-19T18:18:45.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.132+0000 7f3005d52640 1 -- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2fec065bb0 msgr2=0x7f2fec068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:45.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.132+0000 7f3005d52640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2fec065bb0 0x7f2fec068070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2ff00ae000 tx=0x7f2ff0004740 comp rx=0 tx=0).stop 2023-12-19T18:18:45.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.132+0000 7f3005d52640 1 -- 172.21.15.26:0/1669791462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300015d530 msgr2=0x7f30000c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:45.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.132+0000 7f3005d52640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300015d530 0x7f30000c84c0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800b970 tx=0x7f2ff800be30 comp rx=0 tx=0).stop 2023-12-19T18:18:45.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.132+0000 7f3005d52640 1 -- 172.21.15.26:0/1669791462 shutdown_connections 2023-12-19T18:18:45.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.132+0000 7f3005d52640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2fec065bb0 0x7f2fec068070 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.132+0000 7f3005d52640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f300015d530 0x7f30000c84c0 unknown :-1 s=CLOSED pgs=2438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.132+0000 7f3005d52640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30001599e0 0x7f30000c4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.132+0000 7f3005d52640 1 --2- 172.21.15.26:0/1669791462 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30000cb150 0x7f30000c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.132+0000 7f3005d52640 1 -- 172.21.15.26:0/1669791462 >> 172.21.15.26:0/1669791462 conn(0x7f3000066160 msgr2=0x7f30000cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:45.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.132+0000 7f3005d52640 1 -- 172.21.15.26:0/1669791462 shutdown_connections 2023-12-19T18:18:45.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.132+0000 7f3005d52640 1 -- 172.21.15.26:0/1669791462 wait complete. 2023-12-19T18:18:45.157 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-19T18:18:45.302 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.300+0000 7f753b16d640 1 Processor -- start 2023-12-19T18:18:45.303 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.300+0000 7f753b16d640 1 -- start start 2023-12-19T18:18:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.304+0000 7f753b16d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f75341585f0 0x7f75340cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.304+0000 7f753b16d640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f75340cb150 0x7f75340cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.304+0000 7f753b16d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753415b1b0 con 0x7f75340cc310 2023-12-19T18:18:45.304 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.304+0000 7f753b16d640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75340cbb10 con 0x7f75340cb150 2023-12-19T18:18:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.304+0000 7f753b16d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75340b47a0 con 0x7f75341585f0 2023-12-19T18:18:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.304+0000 7f7533fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f75340cb150 0x7f75340cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.304+0000 7f7533fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f75340cb150 0x7f75340cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36308/0 (socket says 172.21.15.26:36308) 2023-12-19T18:18:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.304+0000 7f7533fff640 1 -- 172.21.15.26:0/215783680 learned_addr learned my addr 172.21.15.26:0/215783680 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.304+0000 7f7538f09640 1 --2- 172.21.15.26:0/215783680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f75341585f0 0x7f75340cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.304+0000 7f7533fff640 1 -- 172.21.15.26:0/215783680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f75341585f0 msgr2=0x7f75340cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.304+0000 7f7533fff640 1 --2- 172.21.15.26:0/215783680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f75341585f0 0x7f75340cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.304+0000 7f7533fff640 1 -- 172.21.15.26:0/215783680 >> v1:172.21.15.160:6789/0 conn(0x7f75340cc310 legacy=0x7f75340cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:45.305 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.304+0000 7f7533fff640 1 -- 172.21.15.26:0/215783680 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753415b490 con 0x7f75340cb150 2023-12-19T18:18:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f7533fff640 1 --2- 172.21.15.26:0/215783680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75340cb150 0x7f75340cb5d0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f7520009870 tx=0x7f752001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f8f0b831250d9c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f75337fe640 1 -- 172.21.15.26:0/215783680 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75200ab600 con 0x7f75340cb150 2023-12-19T18:18:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f75337fe640 1 -- 172.21.15.26:0/215783680 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75200ab760 con 0x7f75340cb150 2023-12-19T18:18:45.307 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f75337fe640 1 -- 172.21.15.26:0/215783680 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75200abaf0 con 0x7f75340cb150 2023-12-19T18:18:45.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f753b16d640 1 -- 172.21.15.26:0/215783680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75340cb150 msgr2=0x7f75340cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:45.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f753b16d640 1 --2- 172.21.15.26:0/215783680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75340cb150 0x7f75340cb5d0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f7520009870 tx=0x7f752001c440 comp rx=0 tx=0).stop 2023-12-19T18:18:45.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f753b16d640 1 -- 172.21.15.26:0/215783680 shutdown_connections 2023-12-19T18:18:45.308 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f753b16d640 1 --2- 172.21.15.26:0/215783680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75340cb150 0x7f75340cb5d0 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f753b16d640 1 --2- 172.21.15.26:0/215783680 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f75341585f0 0x7f75340cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f753b16d640 1 -- 172.21.15.26:0/215783680 >> 172.21.15.26:0/215783680 conn(0x7f7534066160 msgr2=0x7f75340b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:45.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f753b16d640 1 -- 172.21.15.26:0/215783680 shutdown_connections 2023-12-19T18:18:45.309 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f753b16d640 1 -- 172.21.15.26:0/215783680 wait complete. 2023-12-19T18:18:45.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f753b16d640 1 Processor -- start 2023-12-19T18:18:45.310 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.308+0000 7f753b16d640 1 -- start start 2023-12-19T18:18:45.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f753b16d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f75340cb150 0x7f753415c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:45.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f753b16d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75340cc310 0x7f753414f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:45.312 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f753b16d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75341585f0 0x7f75340c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f753b16d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753414fcd0 con 0x7f75340cc310 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f753b16d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753406b310 con 0x7f75341585f0 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f753b16d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753415f4e0 con 0x7f75340cb150 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f7533fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75340cc310 0x7f753414f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f7533fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75340cc310 0x7f753414f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36322/0 (socket says 172.21.15.26:36322) 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f7533fff640 1 -- 172.21.15.26:0/3306307288 learned_addr learned my addr 172.21.15.26:0/3306307288 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f753970a640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75341585f0 0x7f75340c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f7538f09640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f75340cb150 0x7f753415c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f7533fff640 1 -- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f75340cb150 msgr2=0x7f753415c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f7533fff640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f75340cb150 0x7f753415c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f7533fff640 1 -- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75341585f0 msgr2=0x7f75340c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f7533fff640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75341585f0 0x7f75340c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f7533fff640 1 -- 172.21.15.26:0/3306307288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75200094d0 con 0x7f75340cc310 2023-12-19T18:18:45.313 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f753970a640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75341585f0 0x7f75340c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f7533fff640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75340cc310 0x7f753414f790 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f752001aaf0 tx=0x7f752001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f75317fa640 1 -- 172.21.15.26:0/3306307288 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75200be5f0 con 0x7f75340cc310 2023-12-19T18:18:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f75317fa640 1 -- 172.21.15.26:0/3306307288 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75200be770 con 0x7f75340cc310 2023-12-19T18:18:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f75317fa640 1 -- 172.21.15.26:0/3306307288 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75200beb20 con 0x7f75340cc310 2023-12-19T18:18:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f753b16d640 1 -- 172.21.15.26:0/3306307288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753415c3e0 con 0x7f75340cc310 2023-12-19T18:18:45.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f753b16d640 1 -- 172.21.15.26:0/3306307288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753415ce30 con 0x7f75340cc310 2023-12-19T18:18:45.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f753b16d640 1 -- 172.21.15.26:0/3306307288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7534074270 con 0x7f75340cc310 2023-12-19T18:18:45.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.312+0000 7f75317fa640 1 -- 172.21.15.26:0/3306307288 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f75200b4600 con 0x7f75340cc310 2023-12-19T18:18:45.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.316+0000 7f75317fa640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7518065b80 0x7f7518068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:45.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.316+0000 7f75317fa640 1 -- 172.21.15.26:0/3306307288 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f752011e570 con 0x7f75340cc310 2023-12-19T18:18:45.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.316+0000 7f7538f09640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7518065b80 0x7f7518068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:45.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.316+0000 7f7538f09640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7518065b80 0x7f7518068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f752800b930 tx=0x7f7528008040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:45.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.320+0000 7f75317fa640 1 -- 172.21.15.26:0/3306307288 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f75200eba70 con 0x7f75340cc310 2023-12-19T18:18:45.479 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.476+0000 7f753b16d640 1 -- 172.21.15.26:0/3306307288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f75340cd890 con 0x7f75340cc310 2023-12-19T18:18:45.484 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.484+0000 7f75317fa640 1 -- 172.21.15.26:0/3306307288 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v87) v1 ==== 115+0+0 (secure 0 0 0) 0x7f75200ef090 con 0x7f75340cc310 2023-12-19T18:18:45.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.484+0000 7f753b16d640 1 -- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7518065b80 msgr2=0x7f7518068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:45.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.484+0000 7f753b16d640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7518065b80 0x7f7518068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f752800b930 tx=0x7f7528008040 comp rx=0 tx=0).stop 2023-12-19T18:18:45.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.484+0000 7f753b16d640 1 -- 172.21.15.26:0/3306307288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75340cc310 msgr2=0x7f753414f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:45.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.484+0000 7f753b16d640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75340cc310 0x7f753414f790 secure :-1 s=READY pgs=2440 cs=0 l=1 rev1=1 crypto rx=0x7f752001aaf0 tx=0x7f752001ac80 comp rx=0 tx=0).stop 2023-12-19T18:18:45.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.484+0000 7f753b16d640 1 -- 172.21.15.26:0/3306307288 shutdown_connections 2023-12-19T18:18:45.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.484+0000 7f753b16d640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7518065b80 0x7f7518068040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.486 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.484+0000 7f753b16d640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75341585f0 0x7f75340c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.484+0000 7f753b16d640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75340cc310 0x7f753414f790 unknown :-1 s=CLOSED pgs=2440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.484+0000 7f753b16d640 1 --2- 172.21.15.26:0/3306307288 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f75340cb150 0x7f753415c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.484+0000 7f753b16d640 1 -- 172.21.15.26:0/3306307288 >> 172.21.15.26:0/3306307288 conn(0x7f7534066160 msgr2=0x7f753415a010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:45.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.484+0000 7f753b16d640 1 -- 172.21.15.26:0/3306307288 shutdown_connections 2023-12-19T18:18:45.487 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.484+0000 7f753b16d640 1 -- 172.21.15.26:0/3306307288 wait complete. 2023-12-19T18:18:45.506 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-12-19T18:18:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.648+0000 7fa31bd60640 1 Processor -- start 2023-12-19T18:18:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.648+0000 7fa31bd60640 1 -- start start 2023-12-19T18:18:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.648+0000 7fa31bd60640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3140cb1d0 0x7fa3140cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.648+0000 7fa31bd60640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa3140b1d40 0x7fa3140b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.648+0000 7fa31bd60640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31406d4b0 con 0x7fa31406bee0 2023-12-19T18:18:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.648+0000 7fa31bd60640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3140cc030 con 0x7fa3140b1d40 2023-12-19T18:18:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.648+0000 7fa31bd60640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3140cbaf0 con 0x7fa3140cb1d0 2023-12-19T18:18:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.652+0000 7fa319afc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3140cb1d0 0x7fa3140cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.652+0000 7fa319afc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3140cb1d0 0x7fa3140cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:51178/0 (socket says 172.21.15.26:51178) 2023-12-19T18:18:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.652+0000 7fa31a2fd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa3140b1d40 0x7fa3140b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.652+0000 7fa319afc640 1 -- 172.21.15.26:0/1613303249 learned_addr learned my addr 172.21.15.26:0/1613303249 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.652+0000 7fa3192fb640 1 --1- 172.21.15.26:0/1613303249 >> v1:172.21.15.160:6789/0 conn(0x7fa31406bee0 0x7fa31406c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:46570/0 (socket says 172.21.15.26:46570) 2023-12-19T18:18:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.652+0000 7fa318afa640 1 -- 172.21.15.26:0/1613303249 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92468910 0 0) 0x7fa31406d4b0 con 0x7fa31406bee0 2023-12-19T18:18:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.652+0000 7fa318afa640 1 -- 172.21.15.26:0/1613303249 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa300003530 con 0x7fa31406bee0 2023-12-19T18:18:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.652+0000 7fa31a2fd640 1 -- 172.21.15.26:0/1613303249 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3140cb1d0 msgr2=0x7fa3140cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.652+0000 7fa31a2fd640 1 --2- 172.21.15.26:0/1613303249 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3140cb1d0 0x7fa3140cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.652+0000 7fa31a2fd640 1 -- 172.21.15.26:0/1613303249 >> v1:172.21.15.160:6789/0 conn(0x7fa31406bee0 legacy=0x7fa31406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.652+0000 7fa31a2fd640 1 -- 172.21.15.26:0/1613303249 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31406cc10 con 0x7fa3140b1d40 2023-12-19T18:18:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.652+0000 7fa319afc640 1 --2- 172.21.15.26:0/1613303249 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3140cb1d0 0x7fa3140cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.652+0000 7fa31a2fd640 1 --2- 172.21.15.26:0/1613303249 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3140b1d40 0x7fa3140b21d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009d90 tx=0x7fa30c01c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f45c3b2454f092d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa318afa640 1 -- 172.21.15.26:0/1613303249 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0a5980 con 0x7fa3140b1d40 2023-12-19T18:18:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa318afa640 1 -- 172.21.15.26:0/1613303249 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c0a5ae0 con 0x7fa3140b1d40 2023-12-19T18:18:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa318afa640 1 -- 172.21.15.26:0/1613303249 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0a5e70 con 0x7fa3140b1d40 2023-12-19T18:18:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 -- 172.21.15.26:0/1613303249 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3140b1d40 msgr2=0x7fa3140b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 --2- 172.21.15.26:0/1613303249 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3140b1d40 0x7fa3140b21d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009d90 tx=0x7fa30c01c8c0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 -- 172.21.15.26:0/1613303249 shutdown_connections 2023-12-19T18:18:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 --2- 172.21.15.26:0/1613303249 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3140b1d40 0x7fa3140b21d0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 --2- 172.21.15.26:0/1613303249 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3140cb1d0 0x7fa3140cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 -- 172.21.15.26:0/1613303249 >> 172.21.15.26:0/1613303249 conn(0x7fa3140658e0 msgr2=0x7fa314065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 -- 172.21.15.26:0/1613303249 shutdown_connections 2023-12-19T18:18:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 -- 172.21.15.26:0/1613303249 wait complete. 2023-12-19T18:18:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 Processor -- start 2023-12-19T18:18:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 -- start start 2023-12-19T18:18:45.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa31406bee0 0x7fa31415e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:45.659 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3140b1d40 0x7fa3140cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3140cb1d0 0x7fa3141501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3141549f0 con 0x7fa31406bee0 2023-12-19T18:18:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3140cdfd0 con 0x7fa3140b1d40 2023-12-19T18:18:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa31bd60640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa314154c60 con 0x7fa3140cb1d0 2023-12-19T18:18:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.656+0000 7fa319afc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa31406bee0 0x7fa31415e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa319afc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa31406bee0 0x7fa31415e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36330/0 (socket says 172.21.15.26:36330) 2023-12-19T18:18:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa319afc640 1 -- 172.21.15.26:0/888995459 learned_addr learned my addr 172.21.15.26:0/888995459 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa3192fb640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3140b1d40 0x7fa3140cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa31a2fd640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3140cb1d0 0x7fa3141501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:45.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa319afc640 1 -- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3140cb1d0 msgr2=0x7fa3141501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:45.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa319afc640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3140cb1d0 0x7fa3141501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa319afc640 1 -- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3140b1d40 msgr2=0x7fa3140cda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:45.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa319afc640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3140b1d40 0x7fa3140cda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:45.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa319afc640 1 -- 172.21.15.26:0/888995459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3100094d0 con 0x7fa31406bee0 2023-12-19T18:18:45.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa319afc640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa31406bee0 0x7fa31415e800 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7fa31000ea00 tx=0x7fa31000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa30a7fc640 1 -- 172.21.15.26:0/888995459 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31000ca60 con 0x7fa31406bee0 2023-12-19T18:18:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa30a7fc640 1 -- 172.21.15.26:0/888995459 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31000cbc0 con 0x7fa31406bee0 2023-12-19T18:18:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa30a7fc640 1 -- 172.21.15.26:0/888995459 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310013470 con 0x7fa31406bee0 2023-12-19T18:18:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa31bd60640 1 -- 172.21.15.26:0/888995459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30c009a70 con 0x7fa31406bee0 2023-12-19T18:18:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa31bd60640 1 -- 172.21.15.26:0/888995459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa314155220 con 0x7fa31406bee0 2023-12-19T18:18:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa31bd60640 1 -- 172.21.15.26:0/888995459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa314074270 con 0x7fa31406bee0 2023-12-19T18:18:45.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa30a7fc640 1 -- 172.21.15.26:0/888995459 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa3100135d0 con 0x7fa31406bee0 2023-12-19T18:18:45.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa30a7fc640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa2f8065b80 0x7fa2f8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:45.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.660+0000 7fa30a7fc640 1 -- 172.21.15.26:0/888995459 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7fa310083880 con 0x7fa31406bee0 2023-12-19T18:18:45.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.664+0000 7fa3192fb640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa2f8065b80 0x7fa2f8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:45.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.664+0000 7fa3192fb640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa2f8065b80 0x7fa2f8068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa3141fd960 tx=0x7fa304007d40 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:45.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.664+0000 7fa30a7fc640 1 -- 172.21.15.26:0/888995459 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa310050e90 con 0x7fa31406bee0 2023-12-19T18:18:45.881 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:45.880+0000 7fa31bd60640 1 -- 172.21.15.26:0/888995459 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7fa3140ccdf0 con 0x7fa31406bee0 2023-12-19T18:18:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:46.424+0000 7fa30a7fc640 1 -- 172.21.15.26:0/888995459 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3163) v1 ==== 155+0+0 (secure 0 0 0) 0x7fa3100544b0 con 0x7fa31406bee0 2023-12-19T18:18:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:46.428+0000 7fa31bd60640 1 -- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa2f8065b80 msgr2=0x7fa2f8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:46.428+0000 7fa31bd60640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa2f8065b80 0x7fa2f8068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa3141fd960 tx=0x7fa304007d40 comp rx=0 tx=0).stop 2023-12-19T18:18:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:46.428+0000 7fa31bd60640 1 -- 172.21.15.26:0/888995459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa31406bee0 msgr2=0x7fa31415e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:46.428+0000 7fa31bd60640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa31406bee0 0x7fa31415e800 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7fa31000ea00 tx=0x7fa31000eec0 comp rx=0 tx=0).stop 2023-12-19T18:18:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:46.428+0000 7fa31bd60640 1 -- 172.21.15.26:0/888995459 shutdown_connections 2023-12-19T18:18:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:46.428+0000 7fa31bd60640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fa2f8065b80 0x7fa2f8068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:46.428+0000 7fa31bd60640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fa3140cb1d0 0x7fa3141501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:46.428+0000 7fa31bd60640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa3140b1d40 0x7fa3140cda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:46.428+0000 7fa31bd60640 1 --2- 172.21.15.26:0/888995459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa31406bee0 0x7fa31415e800 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:46.428+0000 7fa31bd60640 1 -- 172.21.15.26:0/888995459 >> 172.21.15.26:0/888995459 conn(0x7fa3140658e0 msgr2=0x7fa314070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:46.428+0000 7fa31bd60640 1 -- 172.21.15.26:0/888995459 shutdown_connections 2023-12-19T18:18:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:46.428+0000 7fa31bd60640 1 -- 172.21.15.26:0/888995459 wait complete. 2023-12-19T18:18:46.451 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-12-19T18:18:46.451 INFO:tasks.cephfs_test_runner: 2023-12-19T18:18:46.451 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-19T18:18:46.451 INFO:tasks.cephfs_test_runner:Ran 13 tests in 5749.843s 2023-12-19T18:18:46.452 INFO:tasks.cephfs_test_runner: 2023-12-19T18:18:46.452 INFO:tasks.cephfs_test_runner:OK 2023-12-19T18:18:46.452 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-12-19T18:18:46.465 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-12-19T18:18:46.479 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-19T18:18:46.479 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:18:46.479 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:46.489 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:18:46.489 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2023-12-19T18:18:46.489 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:46.539 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=925 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:18:46.539 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:18:46.539 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:46.550 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:18:46.550 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-19T18:18:46.550 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:46.606 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354680k,nr_inodes=4088670,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3483 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:18:46.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:46.607 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T18:18:46.617 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:18:46.617 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:18:46.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:46.618 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:18:46.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:46.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:18:46.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:46.774 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2023-12-19T18:18:46.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:46.986 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2023-12-19T18:18:46.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:46.996 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2023-12-19T18:18:47.063 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2023-12-19T18:18:47.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:18:47.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:47.066 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2023-12-19T18:18:47.076 INFO:teuthology.orchestra.run.smithi160.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T18:18:47.076 INFO:teuthology.orchestra.run.smithi160.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-19T18:18:47.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:47.077 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:18:47.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:47.142 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T18:18:47.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:47.236 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link delete ceph-brx 2023-12-19T18:18:47.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:47.427 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2023-12-19T18:18:47.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:47.438 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link delete ceph-brx 2023-12-19T18:18:47.509 INFO:teuthology.orchestra.run.smithi160.stderr:Cannot find device "ceph-brx" 2023-12-19T18:18:47.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:18:47.511 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-12-19T18:18:47.521 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-19T18:18:47.521 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:18:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.664+0000 7f73c02fc640 1 Processor -- start 2023-12-19T18:18:47.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.664+0000 7f73c02fc640 1 -- start start 2023-12-19T18:18:47.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73c02fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f73b815c7f0 0x7f73b806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:47.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73c02fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73b80b30d0 0x7f73b80b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:47.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73c02fc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73b80b3a60 con 0x7f73b80adb80 2023-12-19T18:18:47.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73c02fc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b806d770 con 0x7f73b815c7f0 2023-12-19T18:18:47.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73c02fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b806ced0 con 0x7f73b80b30d0 2023-12-19T18:18:47.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73bd897640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f73b80adb80 0x7f73b80cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42968/0 (socket says 172.21.15.26:42968) 2023-12-19T18:18:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73bd897640 1 -- 172.21.15.26:0/1275690631 learned_addr learned my addr 172.21.15.26:0/1275690631 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73be098640 1 --2- 172.21.15.26:0/1275690631 >> v2:172.21.15.160:3300/0 conn(0x7f73b815c7f0 0x7f73b806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73be899640 1 --2- 172.21.15.26:0/1275690631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73b80b30d0 0x7f73b80b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73bd096640 1 -- 172.21.15.26:0/1275690631 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566737117 0 0) 0x7f73b80b3a60 con 0x7f73b80adb80 2023-12-19T18:18:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73bd096640 1 -- 172.21.15.26:0/1275690631 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73a4003450 con 0x7f73b80adb80 2023-12-19T18:18:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73be098640 1 -- 172.21.15.26:0/1275690631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73b80b30d0 msgr2=0x7f73b80b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73be098640 1 --2- 172.21.15.26:0/1275690631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73b80b30d0 0x7f73b80b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:47.669 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73be098640 1 -- 172.21.15.26:0/1275690631 >> v1:172.21.15.26:6789/0 conn(0x7f73b80adb80 legacy=0x7f73b80cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:47.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.668+0000 7f73be098640 1 -- 172.21.15.26:0/1275690631 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b806d1b0 con 0x7f73b815c7f0 2023-12-19T18:18:47.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73be098640 1 --2- 172.21.15.26:0/1275690631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73b815c7f0 0x7f73b806c7f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f73b40097c0 tx=0x7f73b401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bf551b6ddbc0b4b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:47.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73bd096640 1 -- 172.21.15.26:0/1275690631 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b40ab600 con 0x7f73b815c7f0 2023-12-19T18:18:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73bd096640 1 -- 172.21.15.26:0/1275690631 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b40a7050 con 0x7f73b815c7f0 2023-12-19T18:18:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73bd096640 1 -- 172.21.15.26:0/1275690631 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b40ab980 con 0x7f73b815c7f0 2023-12-19T18:18:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73c02fc640 1 -- 172.21.15.26:0/1275690631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73b815c7f0 msgr2=0x7f73b806c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73c02fc640 1 --2- 172.21.15.26:0/1275690631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73b815c7f0 0x7f73b806c7f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f73b40097c0 tx=0x7f73b401c480 comp rx=0 tx=0).stop 2023-12-19T18:18:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73c02fc640 1 -- 172.21.15.26:0/1275690631 shutdown_connections 2023-12-19T18:18:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73c02fc640 1 --2- 172.21.15.26:0/1275690631 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73b80b30d0 0x7f73b80b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73c02fc640 1 --2- 172.21.15.26:0/1275690631 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73b815c7f0 0x7f73b806c7f0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73c02fc640 1 -- 172.21.15.26:0/1275690631 >> 172.21.15.26:0/1275690631 conn(0x7f73b80658e0 msgr2=0x7f73b8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73c02fc640 1 -- 172.21.15.26:0/1275690631 shutdown_connections 2023-12-19T18:18:47.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73c02fc640 1 -- 172.21.15.26:0/1275690631 wait complete. 2023-12-19T18:18:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73c02fc640 1 Processor -- start 2023-12-19T18:18:47.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.672+0000 7f73c02fc640 1 -- start start 2023-12-19T18:18:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73c02fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73b80adb80 0x7f73b8158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73c02fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73b80b30d0 0x7f73b813f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:47.677 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73c02fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73b815c7f0 0x7f73b806fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73c02fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b8159720 con 0x7f73b815c7f0 2023-12-19T18:18:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73c02fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b8159a00 con 0x7f73b80b30d0 2023-12-19T18:18:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73c02fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b8159ce0 con 0x7f73b80adb80 2023-12-19T18:18:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73be899640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73b815c7f0 0x7f73b806fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73be899640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73b815c7f0 0x7f73b806fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53738/0 (socket says 172.21.15.26:53738) 2023-12-19T18:18:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73be899640 1 -- 172.21.15.26:0/1078756037 learned_addr learned my addr 172.21.15.26:0/1078756037 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73be098640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73b80adb80 0x7f73b8158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:47.678 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73bd897640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73b80b30d0 0x7f73b813f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73be899640 1 -- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73b80adb80 msgr2=0x7f73b8158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73be899640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73b80adb80 0x7f73b8158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73be899640 1 -- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73b80b30d0 msgr2=0x7f73b813f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73be899640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73b80b30d0 0x7f73b813f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73be899640 1 -- 172.21.15.26:0/1078756037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b4009420 con 0x7f73b815c7f0 2023-12-19T18:18:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73be899640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73b815c7f0 0x7f73b806fc10 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f73b000efc0 tx=0x7f73b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73aeffd640 1 -- 172.21.15.26:0/1078756037 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0014480 con 0x7f73b815c7f0 2023-12-19T18:18:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73aeffd640 1 -- 172.21.15.26:0/1078756037 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b000f040 con 0x7f73b815c7f0 2023-12-19T18:18:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73c02fc640 1 -- 172.21.15.26:0/1078756037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b8159fc0 con 0x7f73b815c7f0 2023-12-19T18:18:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73aeffd640 1 -- 172.21.15.26:0/1078756037 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b0014820 con 0x7f73b815c7f0 2023-12-19T18:18:47.679 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73c02fc640 1 -- 172.21.15.26:0/1078756037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b8159410 con 0x7f73b815c7f0 2023-12-19T18:18:47.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.676+0000 7f73c02fc640 1 -- 172.21.15.26:0/1078756037 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b815a4e0 con 0x7f73b815c7f0 2023-12-19T18:18:47.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.680+0000 7f73aeffd640 1 -- 172.21.15.26:0/1078756037 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f73b0014980 con 0x7f73b815c7f0 2023-12-19T18:18:47.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.680+0000 7f73aeffd640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f739c065b80 0x7f739c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:47.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.680+0000 7f73aeffd640 1 -- 172.21.15.26:0/1078756037 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f73b0084580 con 0x7f73b815c7f0 2023-12-19T18:18:47.680 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.680+0000 7f73be098640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f739c065b80 0x7f739c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:47.681 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.680+0000 7f73be098640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f739c065b80 0x7f739c068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f73b40ad000 tx=0x7f73b40a6000 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.684+0000 7f73aeffd640 1 -- 172.21.15.26:0/1078756037 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f73b815a4e0 con 0x7f73b815c7f0 2023-12-19T18:18:47.836 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.836+0000 7f73c02fc640 1 -- 172.21.15.26:0/1078756037 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73b814e5b0 con 0x7f739c065b80 2023-12-19T18:18:47.839 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.840+0000 7f73aeffd640 1 -- 172.21.15.26:0/1078756037 <== mgr.11683 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50484 (secure 0 0 0) 0x7f73b814e5b0 con 0x7f739c065b80 2023-12-19T18:18:47.840 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:47.841 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:18:47.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.840+0000 7f73c02fc640 1 -- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f739c065b80 msgr2=0x7f739c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:47.841 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.840+0000 7f73c02fc640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f739c065b80 0x7f739c068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f73b40ad000 tx=0x7f73b40a6000 comp rx=0 tx=0).stop 2023-12-19T18:18:47.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.840+0000 7f73c02fc640 1 -- 172.21.15.26:0/1078756037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73b815c7f0 msgr2=0x7f73b806fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:47.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.840+0000 7f73c02fc640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73b815c7f0 0x7f73b806fc10 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f73b000efc0 tx=0x7f73b000c490 comp rx=0 tx=0).stop 2023-12-19T18:18:47.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.840+0000 7f73c02fc640 1 -- 172.21.15.26:0/1078756037 shutdown_connections 2023-12-19T18:18:47.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.840+0000 7f73c02fc640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f739c065b80 0x7f739c068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:47.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.840+0000 7f73c02fc640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73b815c7f0 0x7f73b806fc10 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:47.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.840+0000 7f73c02fc640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73b80b30d0 0x7f73b813f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:47.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.840+0000 7f73c02fc640 1 --2- 172.21.15.26:0/1078756037 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f73b80adb80 0x7f73b8158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:47.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.840+0000 7f73c02fc640 1 -- 172.21.15.26:0/1078756037 >> 172.21.15.26:0/1078756037 conn(0x7f73b80658e0 msgr2=0x7f73b80afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:47.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.840+0000 7f73c02fc640 1 -- 172.21.15.26:0/1078756037 shutdown_connections 2023-12-19T18:18:47.842 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:47.840+0000 7f73c02fc640 1 -- 172.21.15.26:0/1078756037 wait complete. 2023-12-19T18:18:47.860 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-12-19T18:18:46.921395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":412,"ondisk_log_size":412,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":427496,"kb_used_data":26536,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754547224,"statfs":{"total":773094113280,"available":772656357376,"internally_reserved":0,"allocated":27172864,"data_stored":15139296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668220,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-262,"num_write_kb":-2054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001206"},"pg_stats":[{"pgid":"1.0","version":"256'412","reported_seq":1459,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-12-19T18:18:42.279400+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:18:42.279400+0000","last_peered":"2023-12-19T18:18:42.279400+0000","last_clean":"2023-12-19T18:18:42.279400+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:18:42.279400+0000","last_undegraded":"2023-12-19T18:18:42.279400+0000","last_fullsized":"2023-12-19T18:18:42.279400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":412,"ondisk_log_size":412,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510591,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48752,"kb_used_data":3752,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323088,"statfs":{"total":96636764160,"available":96586842112,"internally_reserved":0,"allocated":3842048,"data_stored":2335188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.70899999999999996},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84099999999999997,"5min":0.96399999999999997,"15min":1.1180000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68999999999999995,"15min":0.72399999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82599999999999996,"5min":1.022,"15min":1.0369999999999999},"last":0.67700000000000005}]},{"osd":1,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.70499999999999996,"15min":0.73699999999999999},"min":{"1min":0.46600000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.0580000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69099999999999995,"15min":0.73499999999999999},"min":{"1min":0.53500000000000003,"5min":0.45700000000000002,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":0.92000000000000004,"15min":1.0660000000000001},"last":0.61499999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.72599999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.86099999999999999,"5min":0.97899999999999998,"15min":1.0669999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74299999999999999,"15min":0.77200000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.50800000000000001},"max":{"1min":0.91300000000000003,"5min":1.0109999999999999,"15min":1.236},"last":0.81000000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.749},"min":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96299999999999997,"5min":1.002,"15min":1.079},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.755,"15min":0.77400000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.46700000000000003},"max":{"1min":0.88300000000000001,"5min":1.05,"15min":1.1870000000000001},"last":0.84999999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61099999999999999,"15min":0.59999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85999999999999999,"5min":1.0149999999999999,"15min":1.302},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":0.92200000000000004,"15min":1.0209999999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65300000000000002,"15min":0.63700000000000001},"min":{"1min":0.46600000000000003,"5min":0.41899999999999998,"15min":0.38800000000000001},"max":{"1min":0.88200000000000001,"5min":1.097,"15min":1.1299999999999999},"last":0.874},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68200000000000005,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.0800000000000001},"last":0.71899999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.57999999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.0620000000000001,"5min":1.083,"15min":1.1000000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.54300000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94599999999999995,"5min":0.97799999999999998,"15min":1.093},"last":0.82899999999999996}]}]},{"osd":6,"up_from":15,"seq":64424510591,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48172,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323668,"statfs":{"total":96636764160,"available":96587436032,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90600000000000003,"5min":1.0289999999999999,"15min":1.042},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69299999999999995,"15min":0.72399999999999998},"min":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95999999999999996,"5min":1.224,"15min":1.224},"last":0.80700000000000005}]},{"osd":1,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94799999999999995,"5min":1.103,"15min":1.103},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.746,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.73699999999999999},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.879,"5min":0.98999999999999999,"15min":1.111},"last":0.68500000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.748,"15min":0.77400000000000002},"min":{"1min":0.57799999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94799999999999995,"5min":1.046,"15min":1.1519999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.90300000000000002,"5min":1.268,"15min":1.268},"last":0.67400000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.755}]},{"osd":5,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.97499999999999998,"5min":1.194,"15min":1.194},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.67700000000000005},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.57099999999999995}]},{"osd":7,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.92200000000000004,"5min":1.218,"15min":1.218},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.60999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.1200000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.748}]}]},{"osd":5,"up_from":15,"seq":64424510591,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52268,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319572,"statfs":{"total":96636764160,"available":96583241728,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67400000000000004,"15min":0.71899999999999997},"min":{"1min":0.434,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":1.002,"15min":1.0149999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70099999999999996,"15min":0.73199999999999998},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.72099999999999997,"15min":0.751},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.751},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68400000000000005,"15min":0.72399999999999998},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81999999999999995,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71899999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73499999999999999,"15min":0.77000000000000002},"min":{"1min":0.51300000000000001,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":0.93500000000000005,"5min":1.0289999999999999,"15min":1.2250000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68200000000000005,"15min":0.72499999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77500000000000002,"5min":1.004,"15min":1.4279999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85899999999999999,"5min":1.0129999999999999,"15min":1.05},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.73399999999999999,"15min":0.76200000000000001},"min":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.78900000000000003,"5min":1.022,"15min":1.0329999999999999},"last":0.79900000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.442,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.89100000000000001,"5min":0.90200000000000002,"15min":1.0800000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.94999999999999996,"5min":0.98499999999999999,"15min":1.208},"last":0.70799999999999996}]},{"osd":6,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89900000000000002,"5min":1.036,"15min":1.036},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.47299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96499999999999997,"5min":1,"15min":1.0449999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.439,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.169},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0620000000000001,"5min":1.079,"15min":1.0900000000000001},"last":0.85399999999999998}]}]},{"osd":4,"up_from":15,"seq":64424510591,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48172,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323668,"statfs":{"total":96636764160,"available":96587436032,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.70799999999999996,"15min":0.73399999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.109},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88700000000000001,"5min":0.997,"15min":1.01},"last":0.64400000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.73199999999999998,"15min":0.748},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.0680000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.72899999999999998,"15min":0.752},"min":{"1min":0.47199999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.876,"5min":0.97899999999999998,"15min":1.1020000000000001},"last":0.71599999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.48099999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90100000000000002,"5min":1.0289999999999999,"15min":1.071},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.76900000000000002},"min":{"1min":0.48399999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.0760000000000001},"last":1.012}]},{"osd":3,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96299999999999997,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.75700000000000001,"15min":0.78300000000000003},"min":{"1min":0.57399999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98799999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.98799999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.63600000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84299999999999997,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.40899999999999997,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":0.80800000000000005,"5min":1.042,"15min":1.042},"last":0.80800000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46300000000000002,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.97299999999999998,"5min":1.024,"15min":1.024},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.133}]},{"osd":7,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.56000000000000005,"5min":0.496,"15min":0.41899999999999998},"max":{"1min":0.93899999999999995,"5min":1.087,"15min":1.087},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.77200000000000002,"15min":0.75900000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.038,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":1.038}]}]},{"osd":3,"up_from":15,"seq":64424510591,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52268,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319572,"statfs":{"total":96636764160,"available":96583241728,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84999999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51300000000000001,"15min":0.52400000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95399999999999996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.46500000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47699999999999998,"15min":0.49099999999999999},"min":{"1min":0.308,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.377},{"interface":"front","average":{"1min":0.501,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92900000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":0.56599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79200000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56200000000000006,"15min":0.59099999999999997},"min":{"1min":0.38,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73299999999999998,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76500000000000001,"15min":0.80800000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88900000000000001,"5min":1.0609999999999999,"15min":1.4410000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.80900000000000005},"min":{"1min":0.53600000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89700000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.77600000000000002,"15min":0.82399999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93600000000000005,"5min":1.1120000000000001,"15min":1.397},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78100000000000003,"15min":0.81799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96499999999999997,"5min":1.2230000000000001,"15min":1.262},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80300000000000005,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95599999999999996,"5min":1.1859999999999999,"15min":1.274},"last":0.754},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.83099999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0549999999999999,"5min":1.165,"15min":1.302},"last":0.73799999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.84099999999999997,"15min":0.89200000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0529999999999999,"5min":1.2330000000000001,"15min":1.486},"last":0.621},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83899999999999997,"15min":0.877},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97099999999999997,"5min":1.421,"15min":1.421},"last":0.71599999999999997}]}]},{"osd":2,"up_from":15,"seq":64424510590,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52268,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319572,"statfs":{"total":96636764160,"available":96583241728,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42499999999999999,"15min":0.44700000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.622,"5min":0.754,"15min":0.754},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48099999999999998,"15min":0.5},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66000000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42099999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.57299999999999995,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.64800000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.499},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.36199999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.66200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.441,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.86199999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.36899999999999999},"max":{"1min":0.68400000000000005,"5min":0.871,"15min":0.875},"last":0.44700000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.94599999999999995,"5min":1.0880000000000001,"15min":1.306},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76500000000000001,"15min":0.78300000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81799999999999995,"5min":1.2070000000000001,"15min":1.3580000000000001},"last":0.74199999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76800000000000002,"15min":0.79700000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":1.1080000000000001,"15min":1.2050000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89400000000000002,"5min":1.1519999999999999,"15min":1.429},"last":0.97899999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.83199999999999996},"min":{"1min":0.67500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.91700000000000004,"5min":1.216,"15min":1.365},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76900000000000002,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.117,"15min":1.2},"last":0.83599999999999997}]},{"osd":7,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77800000000000002,"15min":0.81399999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96299999999999997,"5min":1.159,"15min":1.45},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.85099999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0229999999999999,"5min":1.276,"15min":1.276},"last":0.80700000000000005}]}]},{"osd":1,"up_from":15,"seq":64424510591,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.432,"15min":0.45800000000000002},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.89400000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.443,"5min":0.497,"15min":0.52200000000000002},"min":{"1min":0.33200000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70199999999999996,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.52800000000000002},"min":{"1min":0.39100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69299999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.54600000000000004,"15min":0.56399999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63700000000000001,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.497}]},{"osd":3,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.58999999999999997,"15min":0.61899999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81499999999999995,"5min":0.86699999999999999,"15min":0.94499999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.52800000000000002,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84299999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.754,"15min":0.78600000000000003},"min":{"1min":0.50700000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95299999999999996,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.755,"15min":0.79900000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95399999999999996,"5min":1.2549999999999999,"15min":1.2569999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.752,"15min":0.79300000000000004},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":1.236,"15min":1.2609999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78900000000000003,"15min":0.81100000000000005},"min":{"1min":0.55600000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.93400000000000005,"5min":1.353,"15min":1.353},"last":0.59299999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77400000000000002,"15min":0.80400000000000005},"min":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.505},"max":{"1min":0.997,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.79800000000000004,"15min":0.83199999999999996},"min":{"1min":0.57499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.94499999999999995,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.56499999999999995},"max":{"1min":0.94099999999999995,"5min":1.399,"15min":1.399},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.80400000000000005,"15min":0.84399999999999997},"min":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0449999999999999,"5min":1.365,"15min":1.365},"last":0.69899999999999995}]}]},{"osd":0,"up_from":14,"seq":60129543296,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65136,"kb_used_data":3752,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94306704,"statfs":{"total":96636764160,"available":96570064896,"internally_reserved":0,"allocated":3842048,"data_stored":2335188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41599999999999998,"15min":0.44900000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.55100000000000005,"5min":0.69299999999999995,"15min":1.216},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.47599999999999998,"15min":0.51100000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.629,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.48099999999999998,"15min":0.52100000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66500000000000004,"5min":0.77000000000000002,"15min":1.1699999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.54300000000000004},"min":{"1min":0.372,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69199999999999995,"5min":0.85699999999999998,"15min":0.96799999999999997},"last":0.497}]},{"osd":3,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.50600000000000001,"15min":0.54100000000000004},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70099999999999996,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.58199999999999996,"15min":0.63200000000000001},"min":{"1min":0.441,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71699999999999997,"5min":0.89700000000000002,"15min":1.032},"last":0.53000000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76100000000000001,"15min":0.80000000000000004},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.496},"max":{"1min":1.052,"5min":1.1200000000000001,"15min":1.381},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.77100000000000002,"15min":0.80600000000000005},"min":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.47799999999999998},"max":{"1min":1.0860000000000001,"5min":1.137,"15min":1.2849999999999999},"last":0.97599999999999998}]},{"osd":5,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76900000000000002,"15min":0.80900000000000005},"min":{"1min":0.58399999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.117,"5min":1.1180000000000001,"15min":1.3420000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78100000000000003,"15min":0.81000000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.996,"5min":1.1259999999999999,"15min":1.288},"last":0.85499999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79600000000000004,"15min":0.82999999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.51300000000000001},"max":{"1min":1.1499999999999999,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0009999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.78200000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.169,"5min":1.1899999999999999,"15min":1.415},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.80200000000000005,"15min":0.84299999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.395},"last":0.88600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:18:47.862 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:18:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.004+0000 7fbebce12640 1 Processor -- start 2023-12-19T18:18:48.007 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.004+0000 7fbebce12640 1 -- start start 2023-12-19T18:18:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbebce12640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbeb8155030 0x7fbeb815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbebce12640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbeb815f370 0x7fbeb806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbebce12640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb806c970 con 0x7fbeb80ce740 2023-12-19T18:18:48.008 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbebce12640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb815b1f0 con 0x7fbeb815f370 2023-12-19T18:18:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbebce12640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb815b4d0 con 0x7fbeb8155030 2023-12-19T18:18:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbeb6d9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbeb80ce740 0x7fbeb80ad490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42972/0 (socket says 172.21.15.26:42972) 2023-12-19T18:18:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbeb6d9d640 1 -- 172.21.15.26:0/344926817 learned_addr learned my addr 172.21.15.26:0/344926817 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbeb659c640 1 --2- 172.21.15.26:0/344926817 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbeb8155030 0x7fbeb815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbeaffff640 1 --2- 172.21.15.26:0/344926817 >> v2:172.21.15.160:3300/0 conn(0x7fbeb815f370 0x7fbeb806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbeb5d9b640 1 -- 172.21.15.26:0/344926817 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055758340 0 0) 0x7fbeb806c970 con 0x7fbeb80ce740 2023-12-19T18:18:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbeb5d9b640 1 -- 172.21.15.26:0/344926817 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbea4003530 con 0x7fbeb80ce740 2023-12-19T18:18:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbeaffff640 1 -- 172.21.15.26:0/344926817 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbeb8155030 msgr2=0x7fbeb815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbeaffff640 1 --2- 172.21.15.26:0/344926817 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbeb8155030 0x7fbeb815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbeaffff640 1 -- 172.21.15.26:0/344926817 >> v1:172.21.15.26:6789/0 conn(0x7fbeb80ce740 legacy=0x7fbeb80ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.010 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.008+0000 7fbeaffff640 1 -- 172.21.15.26:0/344926817 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb815b7b0 con 0x7fbeb815f370 2023-12-19T18:18:48.012 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.012+0000 7fbeaffff640 1 --2- 172.21.15.26:0/344926817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb815f370 0x7fbeb806c080 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fbea00097c0 tx=0x7fbea001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3848fe751f506b80 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.012+0000 7fbeb5d9b640 1 -- 172.21.15.26:0/344926817 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea00a6040 con 0x7fbeb815f370 2023-12-19T18:18:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.012+0000 7fbeb5d9b640 1 -- 172.21.15.26:0/344926817 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea00aa600 con 0x7fbeb815f370 2023-12-19T18:18:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.012+0000 7fbeb5d9b640 1 -- 172.21.15.26:0/344926817 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea00aa990 con 0x7fbeb815f370 2023-12-19T18:18:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.012+0000 7fbebce12640 1 -- 172.21.15.26:0/344926817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb815f370 msgr2=0x7fbeb806c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.012+0000 7fbebce12640 1 --2- 172.21.15.26:0/344926817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb815f370 0x7fbeb806c080 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fbea00097c0 tx=0x7fbea001c480 comp rx=0 tx=0).stop 2023-12-19T18:18:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.012+0000 7fbebce12640 1 -- 172.21.15.26:0/344926817 shutdown_connections 2023-12-19T18:18:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.012+0000 7fbebce12640 1 --2- 172.21.15.26:0/344926817 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb815f370 0x7fbeb806c080 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.012+0000 7fbebce12640 1 --2- 172.21.15.26:0/344926817 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbeb8155030 0x7fbeb815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.012+0000 7fbebce12640 1 -- 172.21.15.26:0/344926817 >> 172.21.15.26:0/344926817 conn(0x7fbeb80658e0 msgr2=0x7fbeb8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.012+0000 7fbebce12640 1 -- 172.21.15.26:0/344926817 shutdown_connections 2023-12-19T18:18:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.012+0000 7fbebce12640 1 -- 172.21.15.26:0/344926817 wait complete. 2023-12-19T18:18:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.012+0000 7fbebce12640 1 Processor -- start 2023-12-19T18:18:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbebce12640 1 -- start start 2023-12-19T18:18:48.016 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbebce12640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbeb80ce740 0x7fbeb814bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbebce12640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbeb8155030 0x7fbeb80adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbebce12640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb815f370 0x7fbeb80b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbebce12640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb814e280 con 0x7fbeb80ce740 2023-12-19T18:18:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbebce12640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb814ea90 con 0x7fbeb815f370 2023-12-19T18:18:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbebce12640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb814dd40 con 0x7fbeb8155030 2023-12-19T18:18:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeb659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbeb80ce740 0x7fbeb814bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeb659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbeb80ce740 0x7fbeb814bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53746/0 (socket says 172.21.15.26:53746) 2023-12-19T18:18:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeb659c640 1 -- 172.21.15.26:0/245925016 learned_addr learned my addr 172.21.15.26:0/245925016 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeb6d9d640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb815f370 0x7fbeb80b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeaffff640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbeb8155030 0x7fbeb80adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeb659c640 1 -- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbeb8155030 msgr2=0x7fbeb80adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeb659c640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbeb8155030 0x7fbeb80adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.017 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeb659c640 1 -- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb815f370 msgr2=0x7fbeb80b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeb659c640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb815f370 0x7fbeb80b06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeb659c640 1 -- 172.21.15.26:0/245925016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea0009420 con 0x7fbeb80ce740 2023-12-19T18:18:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeb6d9d640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb815f370 0x7fbeb80b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeb659c640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbeb80ce740 0x7fbeb814bd80 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7fbea800e930 tx=0x7fbea800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeaeffd640 1 -- 172.21.15.26:0/245925016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8009690 con 0x7fbeb80ce740 2023-12-19T18:18:48.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeaeffd640 1 -- 172.21.15.26:0/245925016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea800ca00 con 0x7fbeb80ce740 2023-12-19T18:18:48.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbeaeffd640 1 -- 172.21.15.26:0/245925016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800cd70 con 0x7fbeb80ce740 2023-12-19T18:18:48.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbebce12640 1 -- 172.21.15.26:0/245925016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb806c740 con 0x7fbeb80ce740 2023-12-19T18:18:48.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbebce12640 1 -- 172.21.15.26:0/245925016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb814e7b0 con 0x7fbeb80ce740 2023-12-19T18:18:48.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.016+0000 7fbebce12640 1 -- 172.21.15.26:0/245925016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb80cb060 con 0x7fbeb80ce740 2023-12-19T18:18:48.019 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.020+0000 7fbeaeffd640 1 -- 172.21.15.26:0/245925016 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbea8013470 con 0x7fbeb80ce740 2023-12-19T18:18:48.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.020+0000 7fbeaeffd640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbe98065b80 0x7fbe98068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.020+0000 7fbeaeffd640 1 -- 172.21.15.26:0/245925016 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7fbea8083850 con 0x7fbeb80ce740 2023-12-19T18:18:48.020 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.020+0000 7fbeaffff640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbe98065b80 0x7fbe98068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.021 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.020+0000 7fbeaffff640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbe98065b80 0x7fbe98068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbea00ae000 tx=0x7fbea00b5040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.024+0000 7fbeaeffd640 1 -- 172.21.15.26:0/245925016 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbea8050f00 con 0x7fbeb80ce740 2023-12-19T18:18:48.178 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.176+0000 7fbebce12640 1 -- 172.21.15.26:0/245925016 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbeb806c5c0 con 0x7fbe98065b80 2023-12-19T18:18:48.180 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.180+0000 7fbeaeffd640 1 -- 172.21.15.26:0/245925016 <== mgr.11683 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50484 (secure 0 0 0) 0x7fbea00aa7f0 con 0x7fbe98065b80 2023-12-19T18:18:48.181 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:48.182 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:18:48.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.180+0000 7fbebce12640 1 -- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbe98065b80 msgr2=0x7fbe98068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.180+0000 7fbebce12640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbe98065b80 0x7fbe98068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbea00ae000 tx=0x7fbea00b5040 comp rx=0 tx=0).stop 2023-12-19T18:18:48.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.180+0000 7fbebce12640 1 -- 172.21.15.26:0/245925016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbeb80ce740 msgr2=0x7fbeb814bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.180+0000 7fbebce12640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbeb80ce740 0x7fbeb814bd80 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7fbea800e930 tx=0x7fbea800edf0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.183 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.180+0000 7fbebce12640 1 -- 172.21.15.26:0/245925016 shutdown_connections 2023-12-19T18:18:48.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.180+0000 7fbebce12640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbe98065b80 0x7fbe98068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.180+0000 7fbebce12640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbeb815f370 0x7fbeb80b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.180+0000 7fbebce12640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbeb8155030 0x7fbeb80adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.180+0000 7fbebce12640 1 --2- 172.21.15.26:0/245925016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbeb80ce740 0x7fbeb814bd80 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.180+0000 7fbebce12640 1 -- 172.21.15.26:0/245925016 >> 172.21.15.26:0/245925016 conn(0x7fbeb80658e0 msgr2=0x7fbeb80b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:48.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.184+0000 7fbebce12640 1 -- 172.21.15.26:0/245925016 shutdown_connections 2023-12-19T18:18:48.184 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.184+0000 7fbebce12640 1 -- 172.21.15.26:0/245925016 wait complete. 2023-12-19T18:18:48.202 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-12-19T18:18:46.921395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":412,"ondisk_log_size":412,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":427496,"kb_used_data":26536,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754547224,"statfs":{"total":773094113280,"available":772656357376,"internally_reserved":0,"allocated":27172864,"data_stored":15139296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668220,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-262,"num_write_kb":-2054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001206"},"pg_stats":[{"pgid":"1.0","version":"256'412","reported_seq":1459,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-12-19T18:18:42.279400+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:18:42.279400+0000","last_peered":"2023-12-19T18:18:42.279400+0000","last_clean":"2023-12-19T18:18:42.279400+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:18:42.279400+0000","last_undegraded":"2023-12-19T18:18:42.279400+0000","last_fullsized":"2023-12-19T18:18:42.279400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":412,"ondisk_log_size":412,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510591,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48752,"kb_used_data":3752,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323088,"statfs":{"total":96636764160,"available":96586842112,"internally_reserved":0,"allocated":3842048,"data_stored":2335188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.70899999999999996},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84099999999999997,"5min":0.96399999999999997,"15min":1.1180000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68999999999999995,"15min":0.72399999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82599999999999996,"5min":1.022,"15min":1.0369999999999999},"last":0.67700000000000005}]},{"osd":1,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.70499999999999996,"15min":0.73699999999999999},"min":{"1min":0.46600000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.0580000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69099999999999995,"15min":0.73499999999999999},"min":{"1min":0.53500000000000003,"5min":0.45700000000000002,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":0.92000000000000004,"15min":1.0660000000000001},"last":0.61499999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.72599999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.86099999999999999,"5min":0.97899999999999998,"15min":1.0669999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74299999999999999,"15min":0.77200000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.50800000000000001},"max":{"1min":0.91300000000000003,"5min":1.0109999999999999,"15min":1.236},"last":0.81000000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.749},"min":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96299999999999997,"5min":1.002,"15min":1.079},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.755,"15min":0.77400000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.46700000000000003},"max":{"1min":0.88300000000000001,"5min":1.05,"15min":1.1870000000000001},"last":0.84999999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61099999999999999,"15min":0.59999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85999999999999999,"5min":1.0149999999999999,"15min":1.302},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":0.92200000000000004,"15min":1.0209999999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65300000000000002,"15min":0.63700000000000001},"min":{"1min":0.46600000000000003,"5min":0.41899999999999998,"15min":0.38800000000000001},"max":{"1min":0.88200000000000001,"5min":1.097,"15min":1.1299999999999999},"last":0.874},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68200000000000005,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.0800000000000001},"last":0.71899999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.57999999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.0620000000000001,"5min":1.083,"15min":1.1000000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.54300000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94599999999999995,"5min":0.97799999999999998,"15min":1.093},"last":0.82899999999999996}]}]},{"osd":6,"up_from":15,"seq":64424510591,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48172,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323668,"statfs":{"total":96636764160,"available":96587436032,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90600000000000003,"5min":1.0289999999999999,"15min":1.042},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69299999999999995,"15min":0.72399999999999998},"min":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95999999999999996,"5min":1.224,"15min":1.224},"last":0.80700000000000005}]},{"osd":1,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94799999999999995,"5min":1.103,"15min":1.103},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.746,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.73699999999999999},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.879,"5min":0.98999999999999999,"15min":1.111},"last":0.68500000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.748,"15min":0.77400000000000002},"min":{"1min":0.57799999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94799999999999995,"5min":1.046,"15min":1.1519999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.90300000000000002,"5min":1.268,"15min":1.268},"last":0.67400000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.755}]},{"osd":5,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.97499999999999998,"5min":1.194,"15min":1.194},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.67700000000000005},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.57099999999999995}]},{"osd":7,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.92200000000000004,"5min":1.218,"15min":1.218},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.60999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.1200000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.748}]}]},{"osd":5,"up_from":15,"seq":64424510591,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52268,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319572,"statfs":{"total":96636764160,"available":96583241728,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67400000000000004,"15min":0.71899999999999997},"min":{"1min":0.434,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":1.002,"15min":1.0149999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70099999999999996,"15min":0.73199999999999998},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.72099999999999997,"15min":0.751},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.751},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68400000000000005,"15min":0.72399999999999998},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81999999999999995,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71899999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73499999999999999,"15min":0.77000000000000002},"min":{"1min":0.51300000000000001,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":0.93500000000000005,"5min":1.0289999999999999,"15min":1.2250000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68200000000000005,"15min":0.72499999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77500000000000002,"5min":1.004,"15min":1.4279999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85899999999999999,"5min":1.0129999999999999,"15min":1.05},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.73399999999999999,"15min":0.76200000000000001},"min":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.78900000000000003,"5min":1.022,"15min":1.0329999999999999},"last":0.79900000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.442,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.89100000000000001,"5min":0.90200000000000002,"15min":1.0800000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.94999999999999996,"5min":0.98499999999999999,"15min":1.208},"last":0.70799999999999996}]},{"osd":6,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89900000000000002,"5min":1.036,"15min":1.036},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.47299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96499999999999997,"5min":1,"15min":1.0449999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.439,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.169},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0620000000000001,"5min":1.079,"15min":1.0900000000000001},"last":0.85399999999999998}]}]},{"osd":4,"up_from":15,"seq":64424510591,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48172,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323668,"statfs":{"total":96636764160,"available":96587436032,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.70799999999999996,"15min":0.73399999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.109},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88700000000000001,"5min":0.997,"15min":1.01},"last":0.64400000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.73199999999999998,"15min":0.748},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.0680000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.72899999999999998,"15min":0.752},"min":{"1min":0.47199999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.876,"5min":0.97899999999999998,"15min":1.1020000000000001},"last":0.71599999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.48099999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90100000000000002,"5min":1.0289999999999999,"15min":1.071},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.76900000000000002},"min":{"1min":0.48399999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.0760000000000001},"last":1.012}]},{"osd":3,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96299999999999997,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.75700000000000001,"15min":0.78300000000000003},"min":{"1min":0.57399999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98799999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.98799999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.63600000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84299999999999997,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.40899999999999997,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":0.80800000000000005,"5min":1.042,"15min":1.042},"last":0.80800000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46300000000000002,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.97299999999999998,"5min":1.024,"15min":1.024},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.133}]},{"osd":7,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.56000000000000005,"5min":0.496,"15min":0.41899999999999998},"max":{"1min":0.93899999999999995,"5min":1.087,"15min":1.087},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.77200000000000002,"15min":0.75900000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.038,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":1.038}]}]},{"osd":3,"up_from":15,"seq":64424510591,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52268,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319572,"statfs":{"total":96636764160,"available":96583241728,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84999999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51300000000000001,"15min":0.52400000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95399999999999996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.46500000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47699999999999998,"15min":0.49099999999999999},"min":{"1min":0.308,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.377},{"interface":"front","average":{"1min":0.501,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92900000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":0.56599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79200000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56200000000000006,"15min":0.59099999999999997},"min":{"1min":0.38,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73299999999999998,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76500000000000001,"15min":0.80800000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88900000000000001,"5min":1.0609999999999999,"15min":1.4410000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.80900000000000005},"min":{"1min":0.53600000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89700000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.77600000000000002,"15min":0.82399999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93600000000000005,"5min":1.1120000000000001,"15min":1.397},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78100000000000003,"15min":0.81799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96499999999999997,"5min":1.2230000000000001,"15min":1.262},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80300000000000005,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95599999999999996,"5min":1.1859999999999999,"15min":1.274},"last":0.754},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.83099999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0549999999999999,"5min":1.165,"15min":1.302},"last":0.73799999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.84099999999999997,"15min":0.89200000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0529999999999999,"5min":1.2330000000000001,"15min":1.486},"last":0.621},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83899999999999997,"15min":0.877},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97099999999999997,"5min":1.421,"15min":1.421},"last":0.71599999999999997}]}]},{"osd":2,"up_from":15,"seq":64424510590,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52268,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319572,"statfs":{"total":96636764160,"available":96583241728,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42499999999999999,"15min":0.44700000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.622,"5min":0.754,"15min":0.754},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48099999999999998,"15min":0.5},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66000000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42099999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.57299999999999995,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.64800000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.499},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.36199999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.66200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.441,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.86199999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.36899999999999999},"max":{"1min":0.68400000000000005,"5min":0.871,"15min":0.875},"last":0.44700000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.94599999999999995,"5min":1.0880000000000001,"15min":1.306},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76500000000000001,"15min":0.78300000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81799999999999995,"5min":1.2070000000000001,"15min":1.3580000000000001},"last":0.74199999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76800000000000002,"15min":0.79700000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":1.1080000000000001,"15min":1.2050000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89400000000000002,"5min":1.1519999999999999,"15min":1.429},"last":0.97899999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.83199999999999996},"min":{"1min":0.67500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.91700000000000004,"5min":1.216,"15min":1.365},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76900000000000002,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.117,"15min":1.2},"last":0.83599999999999997}]},{"osd":7,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77800000000000002,"15min":0.81399999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96299999999999997,"5min":1.159,"15min":1.45},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.85099999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0229999999999999,"5min":1.276,"15min":1.276},"last":0.80700000000000005}]}]},{"osd":1,"up_from":15,"seq":64424510591,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.432,"15min":0.45800000000000002},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.89400000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.443,"5min":0.497,"15min":0.52200000000000002},"min":{"1min":0.33200000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70199999999999996,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.52800000000000002},"min":{"1min":0.39100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69299999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.54600000000000004,"15min":0.56399999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63700000000000001,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.497}]},{"osd":3,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.58999999999999997,"15min":0.61899999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81499999999999995,"5min":0.86699999999999999,"15min":0.94499999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.52800000000000002,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84299999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.754,"15min":0.78600000000000003},"min":{"1min":0.50700000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95299999999999996,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.755,"15min":0.79900000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95399999999999996,"5min":1.2549999999999999,"15min":1.2569999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.752,"15min":0.79300000000000004},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":1.236,"15min":1.2609999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78900000000000003,"15min":0.81100000000000005},"min":{"1min":0.55600000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.93400000000000005,"5min":1.353,"15min":1.353},"last":0.59299999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77400000000000002,"15min":0.80400000000000005},"min":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.505},"max":{"1min":0.997,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.79800000000000004,"15min":0.83199999999999996},"min":{"1min":0.57499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.94499999999999995,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.56499999999999995},"max":{"1min":0.94099999999999995,"5min":1.399,"15min":1.399},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.80400000000000005,"15min":0.84399999999999997},"min":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0449999999999999,"5min":1.365,"15min":1.365},"last":0.69899999999999995}]}]},{"osd":0,"up_from":14,"seq":60129543296,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65136,"kb_used_data":3752,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94306704,"statfs":{"total":96636764160,"available":96570064896,"internally_reserved":0,"allocated":3842048,"data_stored":2335188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41599999999999998,"15min":0.44900000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.55100000000000005,"5min":0.69299999999999995,"15min":1.216},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.47599999999999998,"15min":0.51100000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.629,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.48099999999999998,"15min":0.52100000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66500000000000004,"5min":0.77000000000000002,"15min":1.1699999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.54300000000000004},"min":{"1min":0.372,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69199999999999995,"5min":0.85699999999999998,"15min":0.96799999999999997},"last":0.497}]},{"osd":3,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.50600000000000001,"15min":0.54100000000000004},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70099999999999996,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.58199999999999996,"15min":0.63200000000000001},"min":{"1min":0.441,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71699999999999997,"5min":0.89700000000000002,"15min":1.032},"last":0.53000000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76100000000000001,"15min":0.80000000000000004},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.496},"max":{"1min":1.052,"5min":1.1200000000000001,"15min":1.381},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.77100000000000002,"15min":0.80600000000000005},"min":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.47799999999999998},"max":{"1min":1.0860000000000001,"5min":1.137,"15min":1.2849999999999999},"last":0.97599999999999998}]},{"osd":5,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76900000000000002,"15min":0.80900000000000005},"min":{"1min":0.58399999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.117,"5min":1.1180000000000001,"15min":1.3420000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78100000000000003,"15min":0.81000000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.996,"5min":1.1259999999999999,"15min":1.288},"last":0.85499999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79600000000000004,"15min":0.82999999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.51300000000000001},"max":{"1min":1.1499999999999999,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0009999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.78200000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.169,"5min":1.1899999999999999,"15min":1.415},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.80200000000000005,"15min":0.84299999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.395},"last":0.88600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:18:48.204 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-19T18:18:48.204 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:18:48.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.344+0000 7f4240b7b640 1 Processor -- start 2023-12-19T18:18:48.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.344+0000 7f4240b7b640 1 -- start start 2023-12-19T18:18:48.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f4240b7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423c154220 0x7f423c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f4240b7b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f423c0cee10 0x7f423c15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f4240b7b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f423c06bc30 con 0x7f423c06b3d0 2023-12-19T18:18:48.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f4240b7b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423c0cc6e0 con 0x7f423c0cee10 2023-12-19T18:18:48.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f4240b7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c15afc0 con 0x7f423c154220 2023-12-19T18:18:48.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f4233fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f423c06b3d0 0x7f423c0ce700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:54074/0 (socket says 172.21.15.26:54074) 2023-12-19T18:18:48.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f4233fff640 1 -- 172.21.15.26:0/3667372265 learned_addr learned my addr 172.21.15.26:0/3667372265 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:48.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f423a59c640 1 --2- 172.21.15.26:0/3667372265 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423c154220 0x7f423c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f423ad9d640 1 --2- 172.21.15.26:0/3667372265 >> v2:172.21.15.160:3300/0 conn(0x7f423c0cee10 0x7f423c15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f4239d9b640 1 -- 172.21.15.26:0/3667372265 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122351152 0 0) 0x7f423c06bc30 con 0x7f423c06b3d0 2023-12-19T18:18:48.349 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f4239d9b640 1 -- 172.21.15.26:0/3667372265 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4228005740 con 0x7f423c06b3d0 2023-12-19T18:18:48.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f423a59c640 1 -- 172.21.15.26:0/3667372265 >> v2:172.21.15.160:3300/0 conn(0x7f423c0cee10 msgr2=0x7f423c15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:48.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f423a59c640 1 --2- 172.21.15.26:0/3667372265 >> v2:172.21.15.160:3300/0 conn(0x7f423c0cee10 0x7f423c15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f423a59c640 1 -- 172.21.15.26:0/3667372265 >> v1:172.21.15.160:6789/0 conn(0x7f423c06b3d0 legacy=0x7f423c0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.350 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.348+0000 7f423a59c640 1 -- 172.21.15.26:0/3667372265 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c06b950 con 0x7f423c154220 2023-12-19T18:18:48.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.352+0000 7f423a59c640 1 --2- 172.21.15.26:0/3667372265 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423c154220 0x7f423c15aa80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f422c0097c0 tx=0x7f422c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c583fb31a1d0c2e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:48.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.352+0000 7f4239d9b640 1 -- 172.21.15.26:0/3667372265 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c0ab650 con 0x7f423c154220 2023-12-19T18:18:48.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.352+0000 7f4239d9b640 1 -- 172.21.15.26:0/3667372265 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422c0ab7b0 con 0x7f423c154220 2023-12-19T18:18:48.352 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.352+0000 7f4239d9b640 1 -- 172.21.15.26:0/3667372265 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422c0abb40 con 0x7f423c154220 2023-12-19T18:18:48.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.352+0000 7f4240b7b640 1 -- 172.21.15.26:0/3667372265 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423c154220 msgr2=0x7f423c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.352+0000 7f4240b7b640 1 --2- 172.21.15.26:0/3667372265 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423c154220 0x7f423c15aa80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f422c0097c0 tx=0x7f422c01c450 comp rx=0 tx=0).stop 2023-12-19T18:18:48.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.352+0000 7f4240b7b640 1 -- 172.21.15.26:0/3667372265 shutdown_connections 2023-12-19T18:18:48.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.352+0000 7f4240b7b640 1 --2- 172.21.15.26:0/3667372265 >> v2:172.21.15.160:3300/0 conn(0x7f423c0cee10 0x7f423c15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.352+0000 7f4240b7b640 1 --2- 172.21.15.26:0/3667372265 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423c154220 0x7f423c15aa80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.352+0000 7f4240b7b640 1 -- 172.21.15.26:0/3667372265 >> 172.21.15.26:0/3667372265 conn(0x7f423c0658e0 msgr2=0x7f423c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:48.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.352+0000 7f4240b7b640 1 -- 172.21.15.26:0/3667372265 shutdown_connections 2023-12-19T18:18:48.353 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.352+0000 7f4240b7b640 1 -- 172.21.15.26:0/3667372265 wait complete. 2023-12-19T18:18:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.352+0000 7f4240b7b640 1 Processor -- start 2023-12-19T18:18:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4240b7b640 1 -- start start 2023-12-19T18:18:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4240b7b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f423c06b3d0 0x7f423c0b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4240b7b640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0cee10 0x7f423c0b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4240b7b640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423c154220 0x7f423c150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4240b7b640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c1408f0 con 0x7f423c0cee10 2023-12-19T18:18:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4240b7b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c1f86b0 con 0x7f423c06b3d0 2023-12-19T18:18:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4240b7b640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423c1f8990 con 0x7f423c154220 2023-12-19T18:18:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4233fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0cee10 0x7f423c0b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4233fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0cee10 0x7f423c0b29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53760/0 (socket says 172.21.15.26:53760) 2023-12-19T18:18:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4233fff640 1 -- 172.21.15.26:0/1369316783 learned_addr learned my addr 172.21.15.26:0/1369316783 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:48.358 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f423a59c640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f423c06b3d0 0x7f423c0b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f423ad9d640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423c154220 0x7f423c150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4233fff640 1 -- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423c154220 msgr2=0x7f423c150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4233fff640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423c154220 0x7f423c150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4233fff640 1 -- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f423c06b3d0 msgr2=0x7f423c0b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4233fff640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f423c06b3d0 0x7f423c0b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4233fff640 1 -- 172.21.15.26:0/1369316783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c009420 con 0x7f423c0cee10 2023-12-19T18:18:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f423a59c640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f423c06b3d0 0x7f423c0b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4233fff640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0cee10 0x7f423c0b29f0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f422400c4d0 tx=0x7f422400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4232ffd640 1 -- 172.21.15.26:0/1369316783 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224005cf0 con 0x7f423c0cee10 2023-12-19T18:18:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4232ffd640 1 -- 172.21.15.26:0/1369316783 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4224005e50 con 0x7f423c0cee10 2023-12-19T18:18:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4240b7b640 1 -- 172.21.15.26:0/1369316783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c1f8c70 con 0x7f423c0cee10 2023-12-19T18:18:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4232ffd640 1 -- 172.21.15.26:0/1369316783 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4224007270 con 0x7f423c0cee10 2023-12-19T18:18:48.359 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4240b7b640 1 -- 172.21.15.26:0/1369316783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c1f9470 con 0x7f423c0cee10 2023-12-19T18:18:48.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.356+0000 7f4240b7b640 1 -- 172.21.15.26:0/1369316783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c074270 con 0x7f423c0cee10 2023-12-19T18:18:48.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.360+0000 7f4232ffd640 1 -- 172.21.15.26:0/1369316783 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f42240073d0 con 0x7f423c0cee10 2023-12-19T18:18:48.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.360+0000 7f4232ffd640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f421c065bb0 0x7f421c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.360+0000 7f4232ffd640 1 -- 172.21.15.26:0/1369316783 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f4224082a50 con 0x7f423c0cee10 2023-12-19T18:18:48.360 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.360+0000 7f423a59c640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f421c065bb0 0x7f421c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.361 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.360+0000 7f423a59c640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f421c065bb0 0x7f421c068070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f422c0af000 tx=0x7f422c0a7000 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:48.364 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.364+0000 7f4232ffd640 1 -- 172.21.15.26:0/1369316783 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f42240500f0 con 0x7f423c0cee10 2023-12-19T18:18:48.521 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.520+0000 7f4240b7b640 1 -- 172.21.15.26:0/1369316783 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f423c1f8820 con 0x7f421c065bb0 2023-12-19T18:18:48.524 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.524+0000 7f4232ffd640 1 -- 172.21.15.26:0/1369316783 <== mgr.11683 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50484 (secure 0 0 0) 0x7f423c1f8820 con 0x7f421c065bb0 2023-12-19T18:18:48.524 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:18:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.524+0000 7f4240b7b640 1 -- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f421c065bb0 msgr2=0x7f421c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.524+0000 7f4240b7b640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f421c065bb0 0x7f421c068070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f422c0af000 tx=0x7f422c0a7000 comp rx=0 tx=0).stop 2023-12-19T18:18:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.524+0000 7f4240b7b640 1 -- 172.21.15.26:0/1369316783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0cee10 msgr2=0x7f423c0b29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.524+0000 7f4240b7b640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0cee10 0x7f423c0b29f0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f422400c4d0 tx=0x7f422400c990 comp rx=0 tx=0).stop 2023-12-19T18:18:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.524+0000 7f4240b7b640 1 -- 172.21.15.26:0/1369316783 shutdown_connections 2023-12-19T18:18:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.524+0000 7f4240b7b640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f421c065bb0 0x7f421c068070 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.524+0000 7f4240b7b640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f423c154220 0x7f423c150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.524+0000 7f4240b7b640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f423c0cee10 0x7f423c0b29f0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.524+0000 7f4240b7b640 1 --2- 172.21.15.26:0/1369316783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f423c06b3d0 0x7f423c0b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.524+0000 7f4240b7b640 1 -- 172.21.15.26:0/1369316783 >> 172.21.15.26:0/1369316783 conn(0x7f423c0658e0 msgr2=0x7f423c0af440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.524+0000 7f4240b7b640 1 -- 172.21.15.26:0/1369316783 shutdown_connections 2023-12-19T18:18:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.524+0000 7f4240b7b640 1 -- 172.21.15.26:0/1369316783 wait complete. 2023-12-19T18:18:48.545 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-12-19T18:18:46.921395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":412,"ondisk_log_size":412,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":165,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":427496,"kb_used_data":26536,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754547224,"statfs":{"total":773094113280,"available":772656357376,"internally_reserved":0,"allocated":27172864,"data_stored":15139296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668220,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-262,"num_write_kb":-2054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001206"},"pg_stats":[{"pgid":"1.0","version":"256'412","reported_seq":1459,"reported_epoch":257,"state":"active+clean","last_fresh":"2023-12-19T18:18:42.279400+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:18:42.279400+0000","last_peered":"2023-12-19T18:18:42.279400+0000","last_clean":"2023-12-19T18:18:42.279400+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:18:42.279400+0000","last_undegraded":"2023-12-19T18:18:42.279400+0000","last_fullsized":"2023-12-19T18:18:42.279400+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":412,"ondisk_log_size":412,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510591,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48752,"kb_used_data":3752,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323088,"statfs":{"total":96636764160,"available":96586842112,"internally_reserved":0,"allocated":3842048,"data_stored":2335188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.70899999999999996},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84099999999999997,"5min":0.96399999999999997,"15min":1.1180000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68999999999999995,"15min":0.72399999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82599999999999996,"5min":1.022,"15min":1.0369999999999999},"last":0.67700000000000005}]},{"osd":1,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.70499999999999996,"15min":0.73699999999999999},"min":{"1min":0.46600000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.0580000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69099999999999995,"15min":0.73499999999999999},"min":{"1min":0.53500000000000003,"5min":0.45700000000000002,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":0.92000000000000004,"15min":1.0660000000000001},"last":0.61499999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.72599999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.86099999999999999,"5min":0.97899999999999998,"15min":1.0669999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74299999999999999,"15min":0.77200000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.50800000000000001},"max":{"1min":0.91300000000000003,"5min":1.0109999999999999,"15min":1.236},"last":0.81000000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.749},"min":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96299999999999997,"5min":1.002,"15min":1.079},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.755,"15min":0.77400000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.46700000000000003},"max":{"1min":0.88300000000000001,"5min":1.05,"15min":1.1870000000000001},"last":0.84999999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61099999999999999,"15min":0.59999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85999999999999999,"5min":1.0149999999999999,"15min":1.302},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":0.92200000000000004,"15min":1.0209999999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65300000000000002,"15min":0.63700000000000001},"min":{"1min":0.46600000000000003,"5min":0.41899999999999998,"15min":0.38800000000000001},"max":{"1min":0.88200000000000001,"5min":1.097,"15min":1.1299999999999999},"last":0.874},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68200000000000005,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.0800000000000001},"last":0.71899999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.57999999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.0620000000000001,"5min":1.083,"15min":1.1000000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.54300000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94599999999999995,"5min":0.97799999999999998,"15min":1.093},"last":0.82899999999999996}]}]},{"osd":6,"up_from":15,"seq":64424510591,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48172,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323668,"statfs":{"total":96636764160,"available":96587436032,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90600000000000003,"5min":1.0289999999999999,"15min":1.042},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69299999999999995,"15min":0.72399999999999998},"min":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95999999999999996,"5min":1.224,"15min":1.224},"last":0.80700000000000005}]},{"osd":1,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94799999999999995,"5min":1.103,"15min":1.103},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.746,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.73699999999999999},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.879,"5min":0.98999999999999999,"15min":1.111},"last":0.68500000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.748,"15min":0.77400000000000002},"min":{"1min":0.57799999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94799999999999995,"5min":1.046,"15min":1.1519999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.90300000000000002,"5min":1.268,"15min":1.268},"last":0.67400000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.755}]},{"osd":5,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.97499999999999998,"5min":1.194,"15min":1.194},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.67700000000000005},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.57099999999999995}]},{"osd":7,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.92200000000000004,"5min":1.218,"15min":1.218},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.60999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.1200000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.748}]}]},{"osd":5,"up_from":15,"seq":64424510591,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52268,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319572,"statfs":{"total":96636764160,"available":96583241728,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67400000000000004,"15min":0.71899999999999997},"min":{"1min":0.434,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":1.002,"15min":1.0149999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70099999999999996,"15min":0.73199999999999998},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.47899999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.72099999999999997,"15min":0.751},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.751},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68400000000000005,"15min":0.72399999999999998},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81999999999999995,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71899999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73499999999999999,"15min":0.77000000000000002},"min":{"1min":0.51300000000000001,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":0.93500000000000005,"5min":1.0289999999999999,"15min":1.2250000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68200000000000005,"15min":0.72499999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77500000000000002,"5min":1.004,"15min":1.4279999999999999},"last":0.78700000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85899999999999999,"5min":1.0129999999999999,"15min":1.05},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.73399999999999999,"15min":0.76200000000000001},"min":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.78900000000000003,"5min":1.022,"15min":1.0329999999999999},"last":0.79900000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.442,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.89100000000000001,"5min":0.90200000000000002,"15min":1.0800000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.94999999999999996,"5min":0.98499999999999999,"15min":1.208},"last":0.70799999999999996}]},{"osd":6,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89900000000000002,"5min":1.036,"15min":1.036},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.47299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96499999999999997,"5min":1,"15min":1.0449999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.439,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.169},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0620000000000001,"5min":1.079,"15min":1.0900000000000001},"last":0.85399999999999998}]}]},{"osd":4,"up_from":15,"seq":64424510591,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48172,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323668,"statfs":{"total":96636764160,"available":96587436032,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.70799999999999996,"15min":0.73399999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.109},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88700000000000001,"5min":0.997,"15min":1.01},"last":0.64400000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.73199999999999998,"15min":0.748},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.0680000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.72899999999999998,"15min":0.752},"min":{"1min":0.47199999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.876,"5min":0.97899999999999998,"15min":1.1020000000000001},"last":0.71599999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.48099999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90100000000000002,"5min":1.0289999999999999,"15min":1.071},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.76900000000000002},"min":{"1min":0.48399999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.0760000000000001},"last":1.012}]},{"osd":3,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96299999999999997,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.75700000000000001,"15min":0.78300000000000003},"min":{"1min":0.57399999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98799999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.98799999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.63600000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84299999999999997,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.40899999999999997,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":0.80800000000000005,"5min":1.042,"15min":1.042},"last":0.80800000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46300000000000002,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.97299999999999998,"5min":1.024,"15min":1.024},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":1.133}]},{"osd":7,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.56000000000000005,"5min":0.496,"15min":0.41899999999999998},"max":{"1min":0.93899999999999995,"5min":1.087,"15min":1.087},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.77200000000000002,"15min":0.75900000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.038,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":1.038}]}]},{"osd":3,"up_from":15,"seq":64424510591,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52268,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319572,"statfs":{"total":96636764160,"available":96583241728,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84999999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51300000000000001,"15min":0.52400000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95399999999999996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.46500000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47699999999999998,"15min":0.49099999999999999},"min":{"1min":0.308,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.377},{"interface":"front","average":{"1min":0.501,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92900000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.44600000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":0.56599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79200000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56200000000000006,"15min":0.59099999999999997},"min":{"1min":0.38,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73299999999999998,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76500000000000001,"15min":0.80800000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88900000000000001,"5min":1.0609999999999999,"15min":1.4410000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.80900000000000005},"min":{"1min":0.53600000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89700000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.63800000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.77600000000000002,"15min":0.82399999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93600000000000005,"5min":1.1120000000000001,"15min":1.397},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78100000000000003,"15min":0.81799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96499999999999997,"5min":1.2230000000000001,"15min":1.262},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80300000000000005,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95599999999999996,"5min":1.1859999999999999,"15min":1.274},"last":0.754},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.83099999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0549999999999999,"5min":1.165,"15min":1.302},"last":0.73799999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.84099999999999997,"15min":0.89200000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0529999999999999,"5min":1.2330000000000001,"15min":1.486},"last":0.621},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83899999999999997,"15min":0.877},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97099999999999997,"5min":1.421,"15min":1.421},"last":0.71599999999999997}]}]},{"osd":2,"up_from":15,"seq":64424510590,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52268,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319572,"statfs":{"total":96636764160,"available":96583241728,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42499999999999999,"15min":0.44700000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.622,"5min":0.754,"15min":0.754},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48099999999999998,"15min":0.5},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66000000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42099999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.57299999999999995,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.64800000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.499},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.36199999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.66200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.441,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.86199999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.36899999999999999},"max":{"1min":0.68400000000000005,"5min":0.871,"15min":0.875},"last":0.44700000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.94599999999999995,"5min":1.0880000000000001,"15min":1.306},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76500000000000001,"15min":0.78300000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81799999999999995,"5min":1.2070000000000001,"15min":1.3580000000000001},"last":0.74199999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76800000000000002,"15min":0.79700000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":1.1080000000000001,"15min":1.2050000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89400000000000002,"5min":1.1519999999999999,"15min":1.429},"last":0.97899999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.83199999999999996},"min":{"1min":0.67500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.91700000000000004,"5min":1.216,"15min":1.365},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76900000000000002,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.117,"15min":1.2},"last":0.83599999999999997}]},{"osd":7,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77800000000000002,"15min":0.81399999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96299999999999997,"5min":1.159,"15min":1.45},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.85099999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0229999999999999,"5min":1.276,"15min":1.276},"last":0.80700000000000005}]}]},{"osd":1,"up_from":15,"seq":64424510591,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60460,"kb_used_data":3172,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311380,"statfs":{"total":96636764160,"available":96574853120,"internally_reserved":0,"allocated":3248128,"data_stored":1744820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.432,"15min":0.45800000000000002},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.89400000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.443,"5min":0.497,"15min":0.52200000000000002},"min":{"1min":0.33200000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70199999999999996,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.46600000000000003}]},{"osd":2,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.52800000000000002},"min":{"1min":0.39100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69299999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.54600000000000004,"15min":0.56399999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63700000000000001,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.497}]},{"osd":3,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.58999999999999997,"15min":0.61899999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81499999999999995,"5min":0.86699999999999999,"15min":0.94499999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.52800000000000002,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84299999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45100000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.754,"15min":0.78600000000000003},"min":{"1min":0.50700000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95299999999999996,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.755,"15min":0.79900000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95399999999999996,"5min":1.2549999999999999,"15min":1.2569999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.752,"15min":0.79300000000000004},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":1.236,"15min":1.2609999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78900000000000003,"15min":0.81100000000000005},"min":{"1min":0.55600000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.93400000000000005,"5min":1.353,"15min":1.353},"last":0.59299999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77400000000000002,"15min":0.80400000000000005},"min":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.505},"max":{"1min":0.997,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.79800000000000004,"15min":0.83199999999999996},"min":{"1min":0.57499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.94499999999999995,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.56499999999999995},"max":{"1min":0.94099999999999995,"5min":1.399,"15min":1.399},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.80400000000000005,"15min":0.84399999999999997},"min":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0449999999999999,"5min":1.365,"15min":1.365},"last":0.69899999999999995}]}]},{"osd":0,"up_from":14,"seq":60129543296,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65136,"kb_used_data":3752,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94306704,"statfs":{"total":96636764160,"available":96570064896,"internally_reserved":0,"allocated":3842048,"data_stored":2335188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41599999999999998,"15min":0.44900000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.55100000000000005,"5min":0.69299999999999995,"15min":1.216},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.47599999999999998,"15min":0.51100000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.629,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.41199999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.48099999999999998,"15min":0.52100000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66500000000000004,"5min":0.77000000000000002,"15min":1.1699999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.54300000000000004},"min":{"1min":0.372,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69199999999999995,"5min":0.85699999999999998,"15min":0.96799999999999997},"last":0.497}]},{"osd":3,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.50600000000000001,"15min":0.54100000000000004},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70099999999999996,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.58199999999999996,"15min":0.63200000000000001},"min":{"1min":0.441,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71699999999999997,"5min":0.89700000000000002,"15min":1.032},"last":0.53000000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76100000000000001,"15min":0.80000000000000004},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.496},"max":{"1min":1.052,"5min":1.1200000000000001,"15min":1.381},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.77100000000000002,"15min":0.80600000000000005},"min":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.47799999999999998},"max":{"1min":1.0860000000000001,"5min":1.137,"15min":1.2849999999999999},"last":0.97599999999999998}]},{"osd":5,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76900000000000002,"15min":0.80900000000000005},"min":{"1min":0.58399999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.117,"5min":1.1180000000000001,"15min":1.3420000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78100000000000003,"15min":0.81000000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.996,"5min":1.1259999999999999,"15min":1.288},"last":0.85499999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79600000000000004,"15min":0.82999999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.51300000000000001},"max":{"1min":1.1499999999999999,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0009999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.78200000000000003}]},{"osd":7,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.169,"5min":1.1899999999999999,"15min":1.415},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.80200000000000005,"15min":0.84299999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.395},"last":0.88600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:18:48.546 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:48.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.688+0000 7f23d24b6640 1 Processor -- start 2023-12-19T18:18:48.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.688+0000 7f23d24b6640 1 -- start start 2023-12-19T18:18:48.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.688+0000 7f23d24b6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f23cc154220 0x7f23cc15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.688+0000 7f23d24b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23cc06b3d0 0x7f23cc0ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.688+0000 7f23d24b6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23cc06bc30 con 0x7f23cc0cec40 2023-12-19T18:18:48.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.688+0000 7f23d24b6640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23cc0cf0d0 con 0x7f23cc154220 2023-12-19T18:18:48.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.688+0000 7f23d24b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0cc6e0 con 0x7f23cc06b3d0 2023-12-19T18:18:48.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.688+0000 7f23cbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f23cc154220 0x7f23cc15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.692+0000 7f23cbfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f23cc154220 0x7f23cc15aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53772/0 (socket says 172.21.15.26:53772) 2023-12-19T18:18:48.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.692+0000 7f23cbfff640 1 -- 172.21.15.26:0/2209839908 learned_addr learned my addr 172.21.15.26:0/2209839908 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:48.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.692+0000 7f23cb7fe640 1 --2- 172.21.15.26:0/2209839908 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23cc06b3d0 0x7f23cc0ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.692+0000 7f23cbfff640 1 -- 172.21.15.26:0/2209839908 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23cc06b3d0 msgr2=0x7f23cc0ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:48.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.692+0000 7f23cbfff640 1 --2- 172.21.15.26:0/2209839908 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23cc06b3d0 0x7f23cc0ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.692+0000 7f23cbfff640 1 -- 172.21.15.26:0/2209839908 >> v1:172.21.15.160:6789/0 conn(0x7f23cc0cec40 legacy=0x7f23cc15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.692+0000 7f23cbfff640 1 -- 172.21.15.26:0/2209839908 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc15afc0 con 0x7f23cc154220 2023-12-19T18:18:48.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.692+0000 7f23cbfff640 1 --2- 172.21.15.26:0/2209839908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc154220 0x7f23cc15aa80 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009870 tx=0x7f23bc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8403fc7533592bc7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:48.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.692+0000 7f23caffd640 1 -- 172.21.15.26:0/2209839908 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0a7070 con 0x7f23cc154220 2023-12-19T18:18:48.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.692+0000 7f23caffd640 1 -- 172.21.15.26:0/2209839908 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc0ab430 con 0x7f23cc154220 2023-12-19T18:18:48.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.692+0000 7f23caffd640 1 -- 172.21.15.26:0/2209839908 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0ab7c0 con 0x7f23cc154220 2023-12-19T18:18:48.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 -- 172.21.15.26:0/2209839908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc154220 msgr2=0x7f23cc15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 --2- 172.21.15.26:0/2209839908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc154220 0x7f23cc15aa80 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009870 tx=0x7f23bc01c510 comp rx=0 tx=0).stop 2023-12-19T18:18:48.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 -- 172.21.15.26:0/2209839908 shutdown_connections 2023-12-19T18:18:48.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 --2- 172.21.15.26:0/2209839908 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23cc06b3d0 0x7f23cc0ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 --2- 172.21.15.26:0/2209839908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc154220 0x7f23cc15aa80 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 -- 172.21.15.26:0/2209839908 >> 172.21.15.26:0/2209839908 conn(0x7f23cc0658e0 msgr2=0x7f23cc065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:48.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 -- 172.21.15.26:0/2209839908 shutdown_connections 2023-12-19T18:18:48.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 -- 172.21.15.26:0/2209839908 wait complete. 2023-12-19T18:18:48.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 Processor -- start 2023-12-19T18:18:48.697 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 -- start start 2023-12-19T18:18:48.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23cc06b3d0 0x7f23cc1508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc0cec40 0x7f23cc1f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23cc154220 0x7f23cc159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc14edc0 con 0x7f23cc0cec40 2023-12-19T18:18:48.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc150e30 con 0x7f23cc06b3d0 2023-12-19T18:18:48.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d24b6640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc1579b0 con 0x7f23cc154220 2023-12-19T18:18:48.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc0cec40 0x7f23cc1f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23cb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc0cec40 0x7f23cc1f4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53788/0 (socket says 172.21.15.26:53788) 2023-12-19T18:18:48.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23cb7fe640 1 -- 172.21.15.26:0/847279653 learned_addr learned my addr 172.21.15.26:0/847279653 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:48.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23cbfff640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23cc06b3d0 0x7f23cc1508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.696+0000 7f23d0a53640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23cc154220 0x7f23cc159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23cb7fe640 1 -- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23cc154220 msgr2=0x7f23cc159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23cb7fe640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23cc154220 0x7f23cc159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23cb7fe640 1 -- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23cc06b3d0 msgr2=0x7f23cc1508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23cb7fe640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23cc06b3d0 0x7f23cc1508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23cb7fe640 1 -- 172.21.15.26:0/847279653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc0094d0 con 0x7f23cc0cec40 2023-12-19T18:18:48.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23cb7fe640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc0cec40 0x7f23cc1f4cc0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f23c000e8a0 tx=0x7f23c000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:48.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23c8ff9640 1 -- 172.21.15.26:0/847279653 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c0009510 con 0x7f23cc0cec40 2023-12-19T18:18:48.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23c8ff9640 1 -- 172.21.15.26:0/847279653 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c000c940 con 0x7f23cc0cec40 2023-12-19T18:18:48.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23c8ff9640 1 -- 172.21.15.26:0/847279653 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c000ccb0 con 0x7f23cc0cec40 2023-12-19T18:18:48.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23d24b6640 1 -- 172.21.15.26:0/847279653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc1525c0 con 0x7f23cc0cec40 2023-12-19T18:18:48.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23d24b6640 1 -- 172.21.15.26:0/847279653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc152b80 con 0x7f23cc0cec40 2023-12-19T18:18:48.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23d24b6640 1 -- 172.21.15.26:0/847279653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc14ef70 con 0x7f23cc0cec40 2023-12-19T18:18:48.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23c8ff9640 1 -- 172.21.15.26:0/847279653 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f23c0013470 con 0x7f23cc0cec40 2023-12-19T18:18:48.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23c8ff9640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f23b8065b80 0x7f23b8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:48.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.700+0000 7f23c8ff9640 1 -- 172.21.15.26:0/847279653 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f23c0083620 con 0x7f23cc0cec40 2023-12-19T18:18:48.703 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.704+0000 7f23cbfff640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f23b8065b80 0x7f23b8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:48.704 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.704+0000 7f23cbfff640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f23b8065b80 0x7f23b8068040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0af000 tx=0x7f23bc008040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:48.706 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.704+0000 7f23c8ff9640 1 -- 172.21.15.26:0/847279653 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f23cc14ef70 con 0x7f23cc0cec40 2023-12-19T18:18:48.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.864+0000 7f23d24b6640 1 -- 172.21.15.26:0/847279653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23cc14ef70 con 0x7f23cc0cec40 2023-12-19T18:18:48.866 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.864+0000 7f23c8ff9640 1 -- 172.21.15.26:0/847279653 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v259) v1 ==== 74+0+14682 (secure 0 0 0) 0x7f23cc14ef70 con 0x7f23cc0cec40 2023-12-19T18:18:48.867 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:48.867 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":259,"fsid":"2264810b-2623-438e-8790-ac9d6fbbbfd5","created":"2023-12-19T16:42:02.486727+0000","modified":"2023-12-19T18:18:44.418382+0000","last_up_change":"2023-12-19T16:42:14.188224+0000","last_in_change":"2023-12-19T16:42:05.061321+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T16:42:15.699170+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17f9dbf7-85f6-4164-aa85-d87685bda626","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":38117},{"type":"v1","addr":"172.21.15.26:6801","nonce":38117}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":38117},{"type":"v1","addr":"172.21.15.26:6803","nonce":38117}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":38117},{"type":"v1","addr":"172.21.15.26:6807","nonce":38117}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":38117},{"type":"v1","addr":"172.21.15.26:6805","nonce":38117}]},"public_addr":"172.21.15.26:6801/38117","cluster_addr":"172.21.15.26:6803/38117","heartbeat_back_addr":"172.21.15.26:6807/38117","heartbeat_front_addr":"172.21.15.26:6805/38117","state":["exists","up"]},{"osd":1,"uuid":"2b12025e-9cb7-43d2-a3a0-5563968aec5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":38116},{"type":"v1","addr":"172.21.15.26:6810","nonce":38116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6811","nonce":38116},{"type":"v1","addr":"172.21.15.26:6813","nonce":38116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":38116},{"type":"v1","addr":"172.21.15.26:6819","nonce":38116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":38116},{"type":"v1","addr":"172.21.15.26:6816","nonce":38116}]},"public_addr":"172.21.15.26:6810/38116","cluster_addr":"172.21.15.26:6813/38116","heartbeat_back_addr":"172.21.15.26:6819/38116","heartbeat_front_addr":"172.21.15.26:6816/38116","state":["exists","up"]},{"osd":2,"uuid":"a7110e7e-9060-4ad4-9ea8-93108111a6c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6809","nonce":38119},{"type":"v1","addr":"172.21.15.26:6812","nonce":38119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6815","nonce":38119},{"type":"v1","addr":"172.21.15.26:6817","nonce":38119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":38119},{"type":"v1","addr":"172.21.15.26:6823","nonce":38119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":38119},{"type":"v1","addr":"172.21.15.26:6821","nonce":38119}]},"public_addr":"172.21.15.26:6812/38119","cluster_addr":"172.21.15.26:6817/38119","heartbeat_back_addr":"172.21.15.26:6823/38119","heartbeat_front_addr":"172.21.15.26:6821/38119","state":["exists","up"]},{"osd":3,"uuid":"cfa611d8-cd56-4ce2-b72b-7e3a70ed1886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":38120},{"type":"v1","addr":"172.21.15.26:6825","nonce":38120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":38120},{"type":"v1","addr":"172.21.15.26:6827","nonce":38120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":38120},{"type":"v1","addr":"172.21.15.26:6831","nonce":38120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":38120},{"type":"v1","addr":"172.21.15.26:6829","nonce":38120}]},"public_addr":"172.21.15.26:6825/38120","cluster_addr":"172.21.15.26:6827/38120","heartbeat_back_addr":"172.21.15.26:6831/38120","heartbeat_front_addr":"172.21.15.26:6829/38120","state":["exists","up"]},{"osd":4,"uuid":"5e877a40-25b0-4de4-8f3f-7a8b8c3e4992","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":38418},{"type":"v1","addr":"172.21.15.160:6801","nonce":38418}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":38418},{"type":"v1","addr":"172.21.15.160:6807","nonce":38418}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6815","nonce":38418},{"type":"v1","addr":"172.21.15.160:6819","nonce":38418}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":38418},{"type":"v1","addr":"172.21.15.160:6812","nonce":38418}]},"public_addr":"172.21.15.160:6801/38418","cluster_addr":"172.21.15.160:6807/38418","heartbeat_back_addr":"172.21.15.160:6819/38418","heartbeat_front_addr":"172.21.15.160:6812/38418","state":["exists","up"]},{"osd":5,"uuid":"32858744-828a-4628-be89-edb9f3a19ee9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":38419},{"type":"v1","addr":"172.21.15.160:6805","nonce":38419}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6809","nonce":38419},{"type":"v1","addr":"172.21.15.160:6813","nonce":38419}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":38419},{"type":"v1","addr":"172.21.15.160:6823","nonce":38419}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6817","nonce":38419},{"type":"v1","addr":"172.21.15.160:6820","nonce":38419}]},"public_addr":"172.21.15.160:6805/38419","cluster_addr":"172.21.15.160:6813/38419","heartbeat_back_addr":"172.21.15.160:6823/38419","heartbeat_front_addr":"172.21.15.160:6820/38419","state":["exists","up"]},{"osd":6,"uuid":"896666f0-45ff-4a17-aa09-eb6947ad2a1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":38421},{"type":"v1","addr":"172.21.15.160:6825","nonce":38421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":38421},{"type":"v1","addr":"172.21.15.160:6827","nonce":38421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":38421},{"type":"v1","addr":"172.21.15.160:6831","nonce":38421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":38421},{"type":"v1","addr":"172.21.15.160:6829","nonce":38421}]},"public_addr":"172.21.15.160:6825/38421","cluster_addr":"172.21.15.160:6827/38421","heartbeat_back_addr":"172.21.15.160:6831/38421","heartbeat_front_addr":"172.21.15.160:6829/38421","state":["exists","up"]},{"osd":7,"uuid":"6de47e3b-3a62-43f4-bce9-8a18c07b306f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":252,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6803","nonce":38420},{"type":"v1","addr":"172.21.15.160:6806","nonce":38420}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":38420},{"type":"v1","addr":"172.21.15.160:6811","nonce":38420}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":38420},{"type":"v1","addr":"172.21.15.160:6821","nonce":38420}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":38420},{"type":"v1","addr":"172.21.15.160:6816","nonce":38420}]},"public_addr":"172.21.15.160:6806/38420","cluster_addr":"172.21.15.160:6811/38420","heartbeat_back_addr":"172.21.15.160:6821/38420","heartbeat_front_addr":"172.21.15.160:6816/38420","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.602430+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.691038+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.740745+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.689963+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.706419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.881513+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.765146+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T16:42:10.957031+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/1019381645":"2023-12-20T18:18:41.398658+0000","172.21.15.26:6833/1019381645":"2023-12-20T18:18:41.398658+0000","172.21.15.160:0/1406825903":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/2675273338":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/1263112506":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/3295238442":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/2546173214":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/1122697388":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/1748568728":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/2326417443":"2023-12-20T18:18:38.894628+0000","172.21.15.160:0/2225649811":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/2614640962":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/1309045557":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3586017330":"2023-12-20T18:18:31.899321+0000","172.21.15.160:6833/48394":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3350159660":"2023-12-20T18:18:31.899321+0000","172.21.15.160:6832/48394":"2023-12-20T18:18:31.899321+0000","172.21.15.160:0/3229222145":"2023-12-20T18:18:31.899321+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":9,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":19,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":30,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":41,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":51,"length":1},{"begin":53,"length":1},{"begin":55,"length":1},{"begin":59,"length":1},{"begin":61,"length":1},{"begin":63,"length":1},{"begin":65,"length":1},{"begin":69,"length":1},{"begin":71,"length":1},{"begin":73,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":83,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":1},{"begin":93,"length":1},{"begin":95,"length":1},{"begin":97,"length":1},{"begin":101,"length":1},{"begin":103,"length":1},{"begin":105,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":177,"length":45}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:48.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.868+0000 7f23d24b6640 1 -- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f23b8065b80 msgr2=0x7f23b8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.869 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.868+0000 7f23d24b6640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f23b8065b80 0x7f23b8068040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0af000 tx=0x7f23bc008040 comp rx=0 tx=0).stop 2023-12-19T18:18:48.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.868+0000 7f23d24b6640 1 -- 172.21.15.26:0/847279653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc0cec40 msgr2=0x7f23cc1f4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:48.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.868+0000 7f23d24b6640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc0cec40 0x7f23cc1f4cc0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f23c000e8a0 tx=0x7f23c000eca0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.868+0000 7f23d24b6640 1 -- 172.21.15.26:0/847279653 shutdown_connections 2023-12-19T18:18:48.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.868+0000 7f23d24b6640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f23b8065b80 0x7f23b8068040 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.868+0000 7f23d24b6640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f23cc154220 0x7f23cc159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.868+0000 7f23d24b6640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23cc0cec40 0x7f23cc1f4cc0 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.870 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.868+0000 7f23d24b6640 1 --2- 172.21.15.26:0/847279653 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23cc06b3d0 0x7f23cc1508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:48.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.868+0000 7f23d24b6640 1 -- 172.21.15.26:0/847279653 >> 172.21.15.26:0/847279653 conn(0x7f23cc0658e0 msgr2=0x7f23cc067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:48.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.868+0000 7f23d24b6640 1 -- 172.21.15.26:0/847279653 shutdown_connections 2023-12-19T18:18:48.871 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:48.868+0000 7f23d24b6640 1 -- 172.21.15.26:0/847279653 wait complete. 2023-12-19T18:18:49.889 INFO:tasks.ceph:Scrubbing osd.0 2023-12-19T18:18:49.889 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:18:50.048 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b6682d640 1 Processor -- start 2023-12-19T18:18:50.049 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b6682d640 1 -- start start 2023-12-19T18:18:50.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b6682d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6015aa50 0x7f6b600b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b6682d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b600b2d90 con 0x7f6b600ccb60 2023-12-19T18:18:50.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b6682d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b600cdd40 con 0x7f6b600b2930 2023-12-19T18:18:50.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b6682d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b600b3250 con 0x7f6b6015aa50 2023-12-19T18:18:50.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b64dca640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6b600ccb60 0x7f6b600af4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42984/0 (socket says 172.21.15.26:42984) 2023-12-19T18:18:50.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b64dca640 1 -- 172.21.15.26:0/2972882208 learned_addr learned my addr 172.21.15.26:0/2972882208 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:50.050 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b5ffff640 1 --2- 172.21.15.26:0/2972882208 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6015aa50 0x7f6b600b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b5effd640 1 -- 172.21.15.26:0/2972882208 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729637685 0 0) 0x7f6b600b2d90 con 0x7f6b600ccb60 2023-12-19T18:18:50.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b5effd640 1 -- 172.21.15.26:0/2972882208 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b48003530 con 0x7f6b600ccb60 2023-12-19T18:18:50.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b5effd640 1 -- 172.21.15.26:0/2972882208 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 641904621 0 0) 0x7f6b600cdd40 con 0x7f6b600b2930 2023-12-19T18:18:50.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b5ffff640 1 -- 172.21.15.26:0/2972882208 >> v1:172.21.15.160:6789/0 conn(0x7f6b600b2930 legacy=0x7f6b600cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b5ffff640 1 -- 172.21.15.26:0/2972882208 >> v1:172.21.15.26:6789/0 conn(0x7f6b600ccb60 legacy=0x7f6b600af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.051 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.048+0000 7f6b5ffff640 1 -- 172.21.15.26:0/2972882208 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b600cd130 con 0x7f6b6015aa50 2023-12-19T18:18:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.052+0000 7f6b5ffff640 1 --2- 172.21.15.26:0/2972882208 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6015aa50 0x7f6b600b23f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6b500097f0 tx=0x7f6b5001c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac91e5be8aea761 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.052+0000 7f6b5effd640 1 -- 172.21.15.26:0/2972882208 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b500aa610 con 0x7f6b6015aa50 2023-12-19T18:18:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.052+0000 7f6b5effd640 1 -- 172.21.15.26:0/2972882208 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b500a6040 con 0x7f6b6015aa50 2023-12-19T18:18:50.054 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.052+0000 7f6b5effd640 1 -- 172.21.15.26:0/2972882208 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b500aa970 con 0x7f6b6015aa50 2023-12-19T18:18:50.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.052+0000 7f6b6682d640 1 -- 172.21.15.26:0/2972882208 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6015aa50 msgr2=0x7f6b600b23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.052+0000 7f6b6682d640 1 --2- 172.21.15.26:0/2972882208 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6015aa50 0x7f6b600b23f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6b500097f0 tx=0x7f6b5001c490 comp rx=0 tx=0).stop 2023-12-19T18:18:50.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.052+0000 7f6b6682d640 1 -- 172.21.15.26:0/2972882208 shutdown_connections 2023-12-19T18:18:50.055 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.052+0000 7f6b6682d640 1 --2- 172.21.15.26:0/2972882208 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6015aa50 0x7f6b600b23f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.052+0000 7f6b6682d640 1 -- 172.21.15.26:0/2972882208 >> 172.21.15.26:0/2972882208 conn(0x7f6b60071460 msgr2=0x7f6b60071870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:50.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.052+0000 7f6b6682d640 1 -- 172.21.15.26:0/2972882208 shutdown_connections 2023-12-19T18:18:50.056 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.052+0000 7f6b6682d640 1 -- 172.21.15.26:0/2972882208 wait complete. 2023-12-19T18:18:50.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b6682d640 1 Processor -- start 2023-12-19T18:18:50.057 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b6682d640 1 -- start start 2023-12-19T18:18:50.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b6682d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b600b2930 0x7f6b60149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.060 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b6682d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b600ccb60 0x7f6b60144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b6682d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6015aa50 0x7f6b60144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b6682d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6014e990 con 0x7f6b600b2930 2023-12-19T18:18:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b6682d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b600c9d10 con 0x7f6b600ccb60 2023-12-19T18:18:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b6682d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6014e6b0 con 0x7f6b6015aa50 2023-12-19T18:18:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b5ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b600b2930 0x7f6b60149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b5ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b600b2930 0x7f6b60149d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53802/0 (socket says 172.21.15.26:53802) 2023-12-19T18:18:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b5ffff640 1 -- 172.21.15.26:0/193466502 learned_addr learned my addr 172.21.15.26:0/193466502 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:50.061 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b64dca640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6015aa50 0x7f6b60144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b5f7fe640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b600ccb60 0x7f6b60144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b5ffff640 1 -- 172.21.15.26:0/193466502 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6015aa50 msgr2=0x7f6b60144ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b5ffff640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b6015aa50 0x7f6b60144ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.062 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b5ffff640 1 -- 172.21.15.26:0/193466502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b600ccb60 msgr2=0x7f6b60144660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b5ffff640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b600ccb60 0x7f6b60144660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b5ffff640 1 -- 172.21.15.26:0/193466502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b50009420 con 0x7f6b600b2930 2023-12-19T18:18:50.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.056+0000 7f6b5ffff640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b600b2930 0x7f6b60149d80 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f6b50000c10 tx=0x7f6b50019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b5cff9640 1 -- 172.21.15.26:0/193466502 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b500aabd0 con 0x7f6b600b2930 2023-12-19T18:18:50.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b5cff9640 1 -- 172.21.15.26:0/193466502 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b500aad30 con 0x7f6b600b2930 2023-12-19T18:18:50.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b5cff9640 1 -- 172.21.15.26:0/193466502 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b500aaea0 con 0x7f6b600b2930 2023-12-19T18:18:50.063 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b6682d640 1 -- 172.21.15.26:0/193466502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b600c96c0 con 0x7f6b600b2930 2023-12-19T18:18:50.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b6682d640 1 -- 172.21.15.26:0/193466502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6014e2f0 con 0x7f6b600b2930 2023-12-19T18:18:50.064 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b6682d640 1 -- 172.21.15.26:0/193466502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6b600cd9d0 con 0x7f6b600b2930 2023-12-19T18:18:50.065 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b5cff9640 1 -- 172.21.15.26:0/193466502 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6b500be670 con 0x7f6b600b2930 2023-12-19T18:18:50.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b5cff9640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b4c065bb0 0x7f6b4c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b5cff9640 1 -- 172.21.15.26:0/193466502 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f6b5011dae0 con 0x7f6b600b2930 2023-12-19T18:18:50.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b5cff9640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] conn(0x7f6b4c06c790 0x7f6b4c06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b5cff9640 1 -- 172.21.15.26:0/193466502 --> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b4c06f1b0 con 0x7f6b4c06c790 2023-12-19T18:18:50.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b5cff9640 1 -- 172.21.15.26:0/193466502 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=259) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b501222a0 con 0x7f6b600b2930 2023-12-19T18:18:50.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b5f7fe640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b4c065bb0 0x7f6b4c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.060+0000 7f6b64dca640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] conn(0x7f6b4c06c790 0x7f6b4c06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.064+0000 7f6b5f7fe640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b4c065bb0 0x7f6b4c068070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6b6014f5a0 tx=0x7f6b54009040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.064+0000 7f6b64dca640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] conn(0x7f6b4c06c790 0x7f6b4c06eb80 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.066 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.064+0000 7f6b5cff9640 1 -- 172.21.15.26:0/193466502 <== osd.0 v2:172.21.15.26:6800/38117 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6b4c06f1b0 con 0x7f6b4c06c790 2023-12-19T18:18:50.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.076+0000 7f6b6682d640 1 -- 172.21.15.26:0/193466502 --> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6b600c9e50 con 0x7f6b4c06c790 2023-12-19T18:18:50.077 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.076+0000 7f6b5cff9640 1 -- 172.21.15.26:0/193466502 <== osd.0 v2:172.21.15.26:6800/38117 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6b58001f60 con 0x7f6b4c06c790 2023-12-19T18:18:50.077 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:18:50.077 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-19T18:18:50.077 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:18:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.076+0000 7f6b6682d640 1 -- 172.21.15.26:0/193466502 >> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] conn(0x7f6b4c06c790 msgr2=0x7f6b4c06eb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.076+0000 7f6b6682d640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.26:6800/38117,v1:172.21.15.26:6801/38117] conn(0x7f6b4c06c790 0x7f6b4c06eb80 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.076+0000 7f6b6682d640 1 -- 172.21.15.26:0/193466502 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b4c065bb0 msgr2=0x7f6b4c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.076+0000 7f6b6682d640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6b4c065bb0 0x7f6b4c068070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6b6014f5a0 tx=0x7f6b54009040 comp rx=0 tx=0).stop 2023-12-19T18:18:50.078 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.076+0000 7f6b6682d640 1 -- 172.21.15.26:0/193466502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b600b2930 msgr2=0x7f6b60149d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.076+0000 7f6b6682d640 1 --2- 172.21.15.26:0/193466502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b600b2930 0x7f6b60149d80 secure :-1 s=READY pgs=2450 cs=0 l=1 rev1=1 crypto rx=0x7f6b50000c10 tx=0x7f6b50019230 comp rx=0 tx=0).stop 2023-12-19T18:18:50.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.076+0000 7f6b64dca640 1 -- 172.21.15.26:0/193466502 reap_dead start 2023-12-19T18:18:50.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.076+0000 7f6b6682d640 1 -- 172.21.15.26:0/193466502 shutdown_connections 2023-12-19T18:18:50.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.076+0000 7f6b6682d640 1 -- 172.21.15.26:0/193466502 >> 172.21.15.26:0/193466502 conn(0x7f6b60071460 msgr2=0x7f6b6006f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:50.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.076+0000 7f6b6682d640 1 -- 172.21.15.26:0/193466502 shutdown_connections 2023-12-19T18:18:50.079 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.076+0000 7f6b6682d640 1 -- 172.21.15.26:0/193466502 wait complete. 2023-12-19T18:18:50.097 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-12-19T18:18:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f389a70a640 1 Processor -- start 2023-12-19T18:18:50.241 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f389a70a640 1 -- start start 2023-12-19T18:18:50.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f389a70a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3894154220 0x7f389415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f389a70a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f389406b3d0 0x7f38940ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f389a70a640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f389406bc30 con 0x7f38940cec40 2023-12-19T18:18:50.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f389a70a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38940cf0d0 con 0x7f3894154220 2023-12-19T18:18:50.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f389a70a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38940cc6e0 con 0x7f389406b3d0 2023-12-19T18:18:50.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f3898ca7640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f38940cec40 0x7f389415d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42988/0 (socket says 172.21.15.26:42988) 2023-12-19T18:18:50.242 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f3898ca7640 1 -- 172.21.15.26:0/3334469815 learned_addr learned my addr 172.21.15.26:0/3334469815 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:50.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f38937fe640 1 --2- 172.21.15.26:0/3334469815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f389406b3d0 0x7f38940ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f3893fff640 1 --2- 172.21.15.26:0/3334469815 >> v2:172.21.15.160:3300/0 conn(0x7f3894154220 0x7f389415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f3892ffd640 1 -- 172.21.15.26:0/3334469815 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922988514 0 0) 0x7f389406bc30 con 0x7f38940cec40 2023-12-19T18:18:50.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f3892ffd640 1 -- 172.21.15.26:0/3334469815 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f387c003530 con 0x7f38940cec40 2023-12-19T18:18:50.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f38937fe640 1 -- 172.21.15.26:0/3334469815 >> v2:172.21.15.160:3300/0 conn(0x7f3894154220 msgr2=0x7f389415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:50.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f38937fe640 1 --2- 172.21.15.26:0/3334469815 >> v2:172.21.15.160:3300/0 conn(0x7f3894154220 0x7f389415aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f38937fe640 1 -- 172.21.15.26:0/3334469815 >> v1:172.21.15.26:6789/0 conn(0x7f38940cec40 legacy=0x7f389415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.243 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.240+0000 7f38937fe640 1 -- 172.21.15.26:0/3334469815 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389415afc0 con 0x7f389406b3d0 2023-12-19T18:18:50.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.244+0000 7f38937fe640 1 --2- 172.21.15.26:0/3334469815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f389406b3d0 0x7f38940ce700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f38880097c0 tx=0x7f388801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d814bfea9437e76b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.244+0000 7f3892ffd640 1 -- 172.21.15.26:0/3334469815 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38880ab600 con 0x7f389406b3d0 2023-12-19T18:18:50.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.244+0000 7f3892ffd640 1 -- 172.21.15.26:0/3334469815 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38880a7050 con 0x7f389406b3d0 2023-12-19T18:18:50.246 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.244+0000 7f3892ffd640 1 -- 172.21.15.26:0/3334469815 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38880ab980 con 0x7f389406b3d0 2023-12-19T18:18:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.244+0000 7f389a70a640 1 -- 172.21.15.26:0/3334469815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f389406b3d0 msgr2=0x7f38940ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.244+0000 7f389a70a640 1 --2- 172.21.15.26:0/3334469815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f389406b3d0 0x7f38940ce700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f38880097c0 tx=0x7f388801c450 comp rx=0 tx=0).stop 2023-12-19T18:18:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.244+0000 7f389a70a640 1 -- 172.21.15.26:0/3334469815 shutdown_connections 2023-12-19T18:18:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.244+0000 7f389a70a640 1 --2- 172.21.15.26:0/3334469815 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f389406b3d0 0x7f38940ce700 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.244+0000 7f389a70a640 1 --2- 172.21.15.26:0/3334469815 >> v2:172.21.15.160:3300/0 conn(0x7f3894154220 0x7f389415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.244+0000 7f389a70a640 1 -- 172.21.15.26:0/3334469815 >> 172.21.15.26:0/3334469815 conn(0x7f38940658e0 msgr2=0x7f3894065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.244+0000 7f389a70a640 1 -- 172.21.15.26:0/3334469815 shutdown_connections 2023-12-19T18:18:50.247 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.244+0000 7f389a70a640 1 -- 172.21.15.26:0/3334469815 wait complete. 2023-12-19T18:18:50.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f389a70a640 1 Processor -- start 2023-12-19T18:18:50.248 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f389a70a640 1 -- start start 2023-12-19T18:18:50.249 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f389a70a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389406b3d0 0x7f3894148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f389a70a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38940cec40 0x7f38940ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f389a70a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3894154220 0x7f38940aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f389a70a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3894149410 con 0x7f389406b3d0 2023-12-19T18:18:50.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f389a70a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38941496f0 con 0x7f3894154220 2023-12-19T18:18:50.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f389a70a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38941499d0 con 0x7f38940cec40 2023-12-19T18:18:50.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3893fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389406b3d0 0x7f3894148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3898ca7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3894154220 0x7f38940aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3893fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389406b3d0 0x7f3894148090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53814/0 (socket says 172.21.15.26:53814) 2023-12-19T18:18:50.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3893fff640 1 -- 172.21.15.26:0/1338849943 learned_addr learned my addr 172.21.15.26:0/1338849943 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:50.250 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f38937fe640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38940cec40 0x7f38940ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3898ca7640 1 -- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38940cec40 msgr2=0x7f38940ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3898ca7640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38940cec40 0x7f38940ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3898ca7640 1 -- 172.21.15.26:0/1338849943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389406b3d0 msgr2=0x7f3894148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.251 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3898ca7640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389406b3d0 0x7f3894148090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3898ca7640 1 -- 172.21.15.26:0/1338849943 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3888009420 con 0x7f3894154220 2023-12-19T18:18:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3893fff640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389406b3d0 0x7f3894148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3898ca7640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3894154220 0x7f38940aebd0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f388c00ca80 tx=0x7f388c00cf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3890ff9640 1 -- 172.21.15.26:0/1338849943 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c00ac30 con 0x7f3894154220 2023-12-19T18:18:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3890ff9640 1 -- 172.21.15.26:0/1338849943 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c00ad90 con 0x7f3894154220 2023-12-19T18:18:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f3890ff9640 1 -- 172.21.15.26:0/1338849943 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c012600 con 0x7f3894154220 2023-12-19T18:18:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.248+0000 7f389a70a640 1 -- 172.21.15.26:0/1338849943 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3894149cb0 con 0x7f3894154220 2023-12-19T18:18:50.252 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.252+0000 7f389a70a640 1 -- 172.21.15.26:0/1338849943 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38941490c0 con 0x7f3894154220 2023-12-19T18:18:50.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.252+0000 7f389a70a640 1 -- 172.21.15.26:0/1338849943 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3894074270 con 0x7f3894154220 2023-12-19T18:18:50.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.252+0000 7f3890ff9640 1 -- 172.21.15.26:0/1338849943 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f388c007070 con 0x7f3894154220 2023-12-19T18:18:50.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.252+0000 7f3890ff9640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3880065bb0 0x7f3880068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.253 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.252+0000 7f3890ff9640 1 -- 172.21.15.26:0/1338849943 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f388c0833a0 con 0x7f3894154220 2023-12-19T18:18:50.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.252+0000 7f3893fff640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3880065bb0 0x7f3880068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.254 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.252+0000 7f3893fff640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3880065bb0 0x7f3880068070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f38940717a0 tx=0x7f3884009190 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.258 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.256+0000 7f3890ff9640 1 -- 172.21.15.26:0/1338849943 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f388c050a40 con 0x7f3894154220 2023-12-19T18:18:50.416 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.416+0000 7f389a70a640 1 -- 172.21.15.26:0/1338849943 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f3894149b40 con 0x7f3880065bb0 2023-12-19T18:18:50.417 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.416+0000 7f3890ff9640 1 -- 172.21.15.26:0/1338849943 <== mgr.11683 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3894149b40 con 0x7f3880065bb0 2023-12-19T18:18:50.417 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 0 to deep-scrub 2023-12-19T18:18:50.419 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.420+0000 7f389a70a640 1 -- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3880065bb0 msgr2=0x7f3880068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.420+0000 7f389a70a640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3880065bb0 0x7f3880068070 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f38940717a0 tx=0x7f3884009190 comp rx=0 tx=0).stop 2023-12-19T18:18:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.420+0000 7f389a70a640 1 -- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3894154220 msgr2=0x7f38940aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.420+0000 7f389a70a640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3894154220 0x7f38940aebd0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f388c00ca80 tx=0x7f388c00cf40 comp rx=0 tx=0).stop 2023-12-19T18:18:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.420+0000 7f389a70a640 1 -- 172.21.15.26:0/1338849943 shutdown_connections 2023-12-19T18:18:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.420+0000 7f389a70a640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3880065bb0 0x7f3880068070 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.420+0000 7f389a70a640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3894154220 0x7f38940aebd0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.420+0000 7f389a70a640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f38940cec40 0x7f38940ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.420+0000 7f389a70a640 1 --2- 172.21.15.26:0/1338849943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f389406b3d0 0x7f3894148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.420+0000 7f389a70a640 1 -- 172.21.15.26:0/1338849943 >> 172.21.15.26:0/1338849943 conn(0x7f38940658e0 msgr2=0x7f3894153280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.420+0000 7f389a70a640 1 -- 172.21.15.26:0/1338849943 shutdown_connections 2023-12-19T18:18:50.420 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.420+0000 7f389a70a640 1 -- 172.21.15.26:0/1338849943 wait complete. 2023-12-19T18:18:50.439 INFO:tasks.ceph:Scrubbing osd.1 2023-12-19T18:18:50.439 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:18:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.584+0000 7faf5c98f640 1 Processor -- start 2023-12-19T18:18:50.586 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.584+0000 7faf5c98f640 1 -- start start 2023-12-19T18:18:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.584+0000 7faf5c98f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf580c9310 0x7faf580cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.584+0000 7faf5c98f640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faf580cb110 0x7faf580cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.584+0000 7faf5c98f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5815e3d0 con 0x7faf58159830 2023-12-19T18:18:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.584+0000 7faf5c98f640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5806f7c0 con 0x7faf580cb110 2023-12-19T18:18:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.584+0000 7faf5c98f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf580ca2b0 con 0x7faf580c9310 2023-12-19T18:18:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.588+0000 7faf4ffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf580c9310 0x7faf580cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.588+0000 7faf4ffff640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf580c9310 0x7faf580cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:44240/0 (socket says 172.21.15.26:44240) 2023-12-19T18:18:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.588+0000 7faf4ffff640 1 -- 172.21.15.26:0/2176845306 learned_addr learned my addr 172.21.15.26:0/2176845306 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.588+0000 7faf5659c640 1 --1- 172.21.15.26:0/2176845306 >> v1:172.21.15.160:6789/0 conn(0x7faf58159830 0x7faf5815d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.26:54108/0 (socket says 172.21.15.26:54108) 2023-12-19T18:18:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.588+0000 7faf56d9d640 1 --2- 172.21.15.26:0/2176845306 >> v2:172.21.15.160:3300/0 conn(0x7faf580cb110 0x7faf580cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.588+0000 7faf4ffff640 1 -- 172.21.15.26:0/2176845306 >> v2:172.21.15.160:3300/0 conn(0x7faf580cb110 msgr2=0x7faf580cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.588+0000 7faf4ffff640 1 --2- 172.21.15.26:0/2176845306 >> v2:172.21.15.160:3300/0 conn(0x7faf580cb110 0x7faf580cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.588+0000 7faf4ffff640 1 -- 172.21.15.26:0/2176845306 >> v1:172.21.15.160:6789/0 conn(0x7faf58159830 legacy=0x7faf5815d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.588+0000 7faf4ffff640 1 -- 172.21.15.26:0/2176845306 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580cba80 con 0x7faf580c9310 2023-12-19T18:18:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.588+0000 7faf55d9b640 1 -- 172.21.15.26:0/2176845306 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22597934 0 0) 0x7faf5815e3d0 con 0x7faf58159830 2023-12-19T18:18:50.590 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.588+0000 7faf4ffff640 1 --2- 172.21.15.26:0/2176845306 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf580c9310 0x7faf580cc430 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7faf4000bb10 tx=0x7faf4001eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65f5a7d2ade4eb2d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf55d9b640 1 -- 172.21.15.26:0/2176845306 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400a7b40 con 0x7faf580c9310 2023-12-19T18:18:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf55d9b640 1 -- 172.21.15.26:0/2176845306 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf400a7ca0 con 0x7faf580c9310 2023-12-19T18:18:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf55d9b640 1 -- 172.21.15.26:0/2176845306 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf400af660 con 0x7faf580c9310 2023-12-19T18:18:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 -- 172.21.15.26:0/2176845306 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf580c9310 msgr2=0x7faf580cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 --2- 172.21.15.26:0/2176845306 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf580c9310 0x7faf580cc430 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7faf4000bb10 tx=0x7faf4001eb80 comp rx=0 tx=0).stop 2023-12-19T18:18:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 -- 172.21.15.26:0/2176845306 shutdown_connections 2023-12-19T18:18:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 --2- 172.21.15.26:0/2176845306 >> v2:172.21.15.160:3300/0 conn(0x7faf580cb110 0x7faf580cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 --2- 172.21.15.26:0/2176845306 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf580c9310 0x7faf580cc430 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 -- 172.21.15.26:0/2176845306 >> 172.21.15.26:0/2176845306 conn(0x7faf580b0270 msgr2=0x7faf580b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 -- 172.21.15.26:0/2176845306 shutdown_connections 2023-12-19T18:18:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 -- 172.21.15.26:0/2176845306 wait complete. 2023-12-19T18:18:50.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 Processor -- start 2023-12-19T18:18:50.593 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 -- start start 2023-12-19T18:18:50.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf580c9310 0x7faf5814ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf580cb110 0x7faf5814e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf58159830 0x7faf5814eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5814f520 con 0x7faf58159830 2023-12-19T18:18:50.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf581f5820 con 0x7faf580cb110 2023-12-19T18:18:50.595 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.592+0000 7faf5c98f640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5814f770 con 0x7faf580c9310 2023-12-19T18:18:50.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf56d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf58159830 0x7faf5814eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf56d9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf58159830 0x7faf5814eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53822/0 (socket says 172.21.15.26:53822) 2023-12-19T18:18:50.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf56d9d640 1 -- 172.21.15.26:0/1496383606 learned_addr learned my addr 172.21.15.26:0/1496383606 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:50.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf4ffff640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf580cb110 0x7faf5814e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf5659c640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf580c9310 0x7faf5814ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf56d9d640 1 -- 172.21.15.26:0/1496383606 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf580c9310 msgr2=0x7faf5814ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf56d9d640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7faf580c9310 0x7faf5814ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf56d9d640 1 -- 172.21.15.26:0/1496383606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf580cb110 msgr2=0x7faf5814e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf56d9d640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faf580cb110 0x7faf5814e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.596 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf56d9d640 1 -- 172.21.15.26:0/1496383606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4000b770 con 0x7faf58159830 2023-12-19T18:18:50.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf56d9d640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf58159830 0x7faf5814eed0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7faf5000efc0 tx=0x7faf5000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf4effd640 1 -- 172.21.15.26:0/1496383606 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50014480 con 0x7faf58159830 2023-12-19T18:18:50.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf4effd640 1 -- 172.21.15.26:0/1496383606 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5000f040 con 0x7faf58159830 2023-12-19T18:18:50.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf5c98f640 1 -- 172.21.15.26:0/1496383606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5814fa50 con 0x7faf58159830 2023-12-19T18:18:50.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf4effd640 1 -- 172.21.15.26:0/1496383606 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50014820 con 0x7faf58159830 2023-12-19T18:18:50.597 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf5c98f640 1 -- 172.21.15.26:0/1496383606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5814b260 con 0x7faf58159830 2023-12-19T18:18:50.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf5c98f640 1 -- 172.21.15.26:0/1496383606 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf5806fcb0 con 0x7faf58159830 2023-12-19T18:18:50.598 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf4effd640 1 -- 172.21.15.26:0/1496383606 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faf50014980 con 0x7faf58159830 2023-12-19T18:18:50.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.596+0000 7faf4effd640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf38065bb0 0x7faf38068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.600+0000 7faf4effd640 1 -- 172.21.15.26:0/1496383606 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7faf500840e0 con 0x7faf58159830 2023-12-19T18:18:50.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.600+0000 7faf4effd640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] conn(0x7faf3806c790 0x7faf3806eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.600+0000 7faf4effd640 1 -- 172.21.15.26:0/1496383606 --> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf3806f1b0 con 0x7faf3806c790 2023-12-19T18:18:50.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.600+0000 7faf4effd640 1 -- 172.21.15.26:0/1496383606 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=259) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf500866c0 con 0x7faf58159830 2023-12-19T18:18:50.599 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.600+0000 7faf5659c640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf38065bb0 0x7faf38068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.600+0000 7faf4ffff640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] conn(0x7faf3806c790 0x7faf3806eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.600+0000 7faf5659c640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf38065bb0 0x7faf38068070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faf48000f70 tx=0x7faf48007df0 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.600+0000 7faf4ffff640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] conn(0x7faf3806c790 0x7faf3806eb80 crc :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.600 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.600+0000 7faf4effd640 1 -- 172.21.15.26:0/1496383606 <== osd.1 v2:172.21.15.26:6808/38116 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7faf3806f1b0 con 0x7faf3806c790 2023-12-19T18:18:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.612+0000 7faf5c98f640 1 -- 172.21.15.26:0/1496383606 --> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7faf5806fa50 con 0x7faf3806c790 2023-12-19T18:18:50.613 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.612+0000 7faf4effd640 1 -- 172.21.15.26:0/1496383606 <== osd.1 v2:172.21.15.26:6808/38116 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7faf400b9440 con 0x7faf3806c790 2023-12-19T18:18:50.614 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:18:50.614 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-19T18:18:50.614 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:18:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.612+0000 7faf5c98f640 1 -- 172.21.15.26:0/1496383606 >> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] conn(0x7faf3806c790 msgr2=0x7faf3806eb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.612+0000 7faf5c98f640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.26:6808/38116,v1:172.21.15.26:6810/38116] conn(0x7faf3806c790 0x7faf3806eb80 crc :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.612+0000 7faf5c98f640 1 -- 172.21.15.26:0/1496383606 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf38065bb0 msgr2=0x7faf38068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.612+0000 7faf5c98f640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7faf38065bb0 0x7faf38068070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faf48000f70 tx=0x7faf48007df0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.612+0000 7faf5c98f640 1 -- 172.21.15.26:0/1496383606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf58159830 msgr2=0x7faf5814eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.612+0000 7faf5c98f640 1 --2- 172.21.15.26:0/1496383606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf58159830 0x7faf5814eed0 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7faf5000efc0 tx=0x7faf5000c490 comp rx=0 tx=0).stop 2023-12-19T18:18:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.612+0000 7faf56d9d640 1 -- 172.21.15.26:0/1496383606 reap_dead start 2023-12-19T18:18:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.612+0000 7faf5c98f640 1 -- 172.21.15.26:0/1496383606 shutdown_connections 2023-12-19T18:18:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.612+0000 7faf5c98f640 1 -- 172.21.15.26:0/1496383606 >> 172.21.15.26:0/1496383606 conn(0x7faf580b0270 msgr2=0x7faf58070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.612+0000 7faf5c98f640 1 -- 172.21.15.26:0/1496383606 shutdown_connections 2023-12-19T18:18:50.615 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.612+0000 7faf5c98f640 1 -- 172.21.15.26:0/1496383606 wait complete. 2023-12-19T18:18:50.633 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-12-19T18:18:50.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.776+0000 7f7d1ecc4640 1 Processor -- start 2023-12-19T18:18:50.778 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.776+0000 7f7d1ecc4640 1 -- start start 2023-12-19T18:18:50.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1ecc4640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7d1815c7f0 0x7f7d1806c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1ecc4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d180adb80 0x7f7d180cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1ecc4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d180ccc20 0x7f7d180b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1ecc4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1806d770 con 0x7f7d180ccc20 2023-12-19T18:18:50.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1ecc4640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1806ced0 con 0x7f7d1815c7f0 2023-12-19T18:18:50.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1ecc4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1806d1b0 con 0x7f7d180adb80 2023-12-19T18:18:50.780 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1d261640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d180ccc20 0x7f7d180b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1d261640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d180ccc20 0x7f7d180b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53832/0 (socket says 172.21.15.26:53832) 2023-12-19T18:18:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1d261640 1 -- 172.21.15.26:0/1285209307 learned_addr learned my addr 172.21.15.26:0/1285209307 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1ca60640 1 --2- 172.21.15.26:0/1285209307 >> v2:172.21.15.160:3300/0 conn(0x7f7d1815c7f0 0x7f7d1806c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d17fff640 1 --2- 172.21.15.26:0/1285209307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d180adb80 0x7f7d180cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1d261640 1 -- 172.21.15.26:0/1285209307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d180adb80 msgr2=0x7f7d180cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1d261640 1 --2- 172.21.15.26:0/1285209307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d180adb80 0x7f7d180cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1d261640 1 -- 172.21.15.26:0/1285209307 >> v2:172.21.15.160:3300/0 conn(0x7f7d1815c7f0 msgr2=0x7f7d1806c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1d261640 1 --2- 172.21.15.26:0/1285209307 >> v2:172.21.15.160:3300/0 conn(0x7f7d1815c7f0 0x7f7d1806c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1d261640 1 -- 172.21.15.26:0/1285209307 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1806d490 con 0x7f7d180ccc20 2023-12-19T18:18:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1ca60640 1 --2- 172.21.15.26:0/1285209307 >> v2:172.21.15.160:3300/0 conn(0x7f7d1815c7f0 0x7f7d1806c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:50.782 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d1d261640 1 --2- 172.21.15.26:0/1285209307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d180ccc20 0x7f7d180b30d0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f7d180670d0 tx=0x7f7d1001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d5b74f8a9713cb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.780+0000 7f7d177fe640 1 -- 172.21.15.26:0/1285209307 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d100a55d0 con 0x7f7d180ccc20 2023-12-19T18:18:50.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d177fe640 1 -- 172.21.15.26:0/1285209307 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d100a5730 con 0x7f7d180ccc20 2023-12-19T18:18:50.783 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d177fe640 1 -- 172.21.15.26:0/1285209307 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d100a5ac0 con 0x7f7d180ccc20 2023-12-19T18:18:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/1285209307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d180ccc20 msgr2=0x7f7d180b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 --2- 172.21.15.26:0/1285209307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d180ccc20 0x7f7d180b30d0 secure :-1 s=READY pgs=2453 cs=0 l=1 rev1=1 crypto rx=0x7f7d180670d0 tx=0x7f7d1001c9d0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/1285209307 shutdown_connections 2023-12-19T18:18:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 --2- 172.21.15.26:0/1285209307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d180ccc20 0x7f7d180b30d0 unknown :-1 s=CLOSED pgs=2453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 --2- 172.21.15.26:0/1285209307 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d180adb80 0x7f7d180cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 --2- 172.21.15.26:0/1285209307 >> v2:172.21.15.160:3300/0 conn(0x7f7d1815c7f0 0x7f7d1806c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/1285209307 >> 172.21.15.26:0/1285209307 conn(0x7f7d180658e0 msgr2=0x7f7d18065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/1285209307 shutdown_connections 2023-12-19T18:18:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/1285209307 wait complete. 2023-12-19T18:18:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 Processor -- start 2023-12-19T18:18:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 -- start start 2023-12-19T18:18:50.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d180adb80 0x7f7d18155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d180ccc20 0x7f7d18157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d1815c7f0 0x7f7d181f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18155610 con 0x7f7d180ccc20 2023-12-19T18:18:50.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18155f70 con 0x7f7d180adb80 2023-12-19T18:18:50.788 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.784+0000 7f7d1ecc4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d181577a0 con 0x7f7d1815c7f0 2023-12-19T18:18:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d180ccc20 0x7f7d18157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d180ccc20 0x7f7d18157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53840/0 (socket says 172.21.15.26:53840) 2023-12-19T18:18:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d17fff640 1 -- 172.21.15.26:0/3281477751 learned_addr learned my addr 172.21.15.26:0/3281477751 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d1ca60640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d180adb80 0x7f7d18155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d1d261640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d1815c7f0 0x7f7d181f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d17fff640 1 -- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d1815c7f0 msgr2=0x7f7d181f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d17fff640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d1815c7f0 0x7f7d181f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d17fff640 1 -- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d180adb80 msgr2=0x7f7d18155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d17fff640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d180adb80 0x7f7d18155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d17fff640 1 -- 172.21.15.26:0/3281477751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d10009b30 con 0x7f7d180ccc20 2023-12-19T18:18:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d1ca60640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d180adb80 0x7f7d18155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d17fff640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d180ccc20 0x7f7d18157260 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400d930 tx=0x7f7d0400ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d157fa640 1 -- 172.21.15.26:0/3281477751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400aac0 con 0x7f7d180ccc20 2023-12-19T18:18:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d157fa640 1 -- 172.21.15.26:0/3281477751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0400ac20 con 0x7f7d180ccc20 2023-12-19T18:18:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d157fa640 1 -- 172.21.15.26:0/3281477751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04011460 con 0x7f7d180ccc20 2023-12-19T18:18:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/3281477751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d181579d0 con 0x7f7d180ccc20 2023-12-19T18:18:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/3281477751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d18154f50 con 0x7f7d180ccc20 2023-12-19T18:18:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/3281477751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d18074270 con 0x7f7d180ccc20 2023-12-19T18:18:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d157fa640 1 -- 172.21.15.26:0/3281477751 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7d040115c0 con 0x7f7d180ccc20 2023-12-19T18:18:50.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.788+0000 7f7d157fa640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7cfc065b80 0x7f7cfc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:50.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.792+0000 7f7d157fa640 1 -- 172.21.15.26:0/3281477751 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f7d04083990 con 0x7f7d180ccc20 2023-12-19T18:18:50.791 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.792+0000 7f7d1ca60640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7cfc065b80 0x7f7cfc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:50.792 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.792+0000 7f7d1ca60640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7cfc065b80 0x7f7cfc068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7d180663d0 tx=0x7f7d0c008040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:50.795 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.796+0000 7f7d157fa640 1 -- 172.21.15.26:0/3281477751 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7d04050f30 con 0x7f7d180ccc20 2023-12-19T18:18:50.952 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.952+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/3281477751 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d180ae170 con 0x7f7cfc065b80 2023-12-19T18:18:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.952+0000 7f7d157fa640 1 -- 172.21.15.26:0/3281477751 <== mgr.11683 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7d0c00b680 con 0x7f7cfc065b80 2023-12-19T18:18:50.953 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 1 to deep-scrub 2023-12-19T18:18:50.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.952+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7cfc065b80 msgr2=0x7f7cfc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.952+0000 7f7d1ecc4640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7cfc065b80 0x7f7cfc068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7d180663d0 tx=0x7f7d0c008040 comp rx=0 tx=0).stop 2023-12-19T18:18:50.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.952+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/3281477751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d180ccc20 msgr2=0x7f7d18157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:50.955 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.952+0000 7f7d1ecc4640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d180ccc20 0x7f7d18157260 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400d930 tx=0x7f7d0400ddf0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.956+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/3281477751 shutdown_connections 2023-12-19T18:18:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.956+0000 7f7d1ecc4640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f7cfc065b80 0x7f7cfc068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.956+0000 7f7d1ecc4640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f7d1815c7f0 0x7f7d181f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.956+0000 7f7d1ecc4640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d180ccc20 0x7f7d18157260 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.956+0000 7f7d1ecc4640 1 --2- 172.21.15.26:0/3281477751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7d180adb80 0x7f7d18155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.956+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/3281477751 >> 172.21.15.26:0/3281477751 conn(0x7f7d180658e0 msgr2=0x7f7d1815ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.956+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/3281477751 shutdown_connections 2023-12-19T18:18:50.956 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:50.956+0000 7f7d1ecc4640 1 -- 172.21.15.26:0/3281477751 wait complete. 2023-12-19T18:18:50.975 INFO:tasks.ceph:Scrubbing osd.2 2023-12-19T18:18:50.975 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:18:51.122 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.120+0000 7f1289673640 1 Processor -- start 2023-12-19T18:18:51.123 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.120+0000 7f1289673640 1 -- start start 2023-12-19T18:18:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1289673640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f128415aad0 0x7f128406c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1289673640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12840b1590 0x7f1284070870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1289673640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f12840b3f90 0x7f1284070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1289673640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f128406cbf0 con 0x7f128415aad0 2023-12-19T18:18:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1289673640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12840ca8b0 con 0x7f12840b3f90 2023-12-19T18:18:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1289673640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12840712f0 con 0x7f12840b1590 2023-12-19T18:18:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1282ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f128415aad0 0x7f128406c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1282ffd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f128415aad0 0x7f128406c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53846/0 (socket says 172.21.15.26:53846) 2023-12-19T18:18:51.124 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1282ffd640 1 -- 172.21.15.26:0/3225301423 learned_addr learned my addr 172.21.15.26:0/3225301423 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1282ffd640 1 -- 172.21.15.26:0/3225301423 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12840b1590 msgr2=0x7f1284070870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1282ffd640 1 --2- 172.21.15.26:0/3225301423 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12840b1590 0x7f1284070870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1282ffd640 1 -- 172.21.15.26:0/3225301423 >> v2:172.21.15.160:3300/0 conn(0x7f12840b3f90 msgr2=0x7f1284070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1282ffd640 1 --2- 172.21.15.26:0/3225301423 >> v2:172.21.15.160:3300/0 conn(0x7f12840b3f90 0x7f1284070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.125 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.124+0000 7f1282ffd640 1 -- 172.21.15.26:0/3225301423 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128406b6f0 con 0x7f128415aad0 2023-12-19T18:18:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1282ffd640 1 --2- 172.21.15.26:0/3225301423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f128415aad0 0x7f128406c6b0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f128406c5d0 tx=0x7f127401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbca5731734bcad server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1281ffb640 1 -- 172.21.15.26:0/3225301423 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12740aa5f0 con 0x7f128415aad0 2023-12-19T18:18:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1281ffb640 1 -- 172.21.15.26:0/3225301423 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12740aa750 con 0x7f128415aad0 2023-12-19T18:18:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1281ffb640 1 -- 172.21.15.26:0/3225301423 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12740aaae0 con 0x7f128415aad0 2023-12-19T18:18:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 -- 172.21.15.26:0/3225301423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f128415aad0 msgr2=0x7f128406c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 --2- 172.21.15.26:0/3225301423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f128415aad0 0x7f128406c6b0 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f128406c5d0 tx=0x7f127401c4d0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 -- 172.21.15.26:0/3225301423 shutdown_connections 2023-12-19T18:18:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 --2- 172.21.15.26:0/3225301423 >> v2:172.21.15.160:3300/0 conn(0x7f12840b3f90 0x7f1284070db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 --2- 172.21.15.26:0/3225301423 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f12840b1590 0x7f1284070870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 --2- 172.21.15.26:0/3225301423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f128415aad0 0x7f128406c6b0 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 -- 172.21.15.26:0/3225301423 >> 172.21.15.26:0/3225301423 conn(0x7f12840b0270 msgr2=0x7f12840b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 -- 172.21.15.26:0/3225301423 shutdown_connections 2023-12-19T18:18:51.129 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 -- 172.21.15.26:0/3225301423 wait complete. 2023-12-19T18:18:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 Processor -- start 2023-12-19T18:18:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 -- start start 2023-12-19T18:18:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12840b1590 0x7f12840c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12840b3f90 0x7f1284154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128415aad0 0x7f12841549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12840c5ea0 con 0x7f12840b3f90 2023-12-19T18:18:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12840c4570 con 0x7f12840b1590 2023-12-19T18:18:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.128+0000 7f1289673640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12840c4850 con 0x7f128415aad0 2023-12-19T18:18:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f12827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12840b3f90 0x7f1284154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f12837fe640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128415aad0 0x7f12841549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f12827fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12840b3f90 0x7f1284154100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53854/0 (socket says 172.21.15.26:53854) 2023-12-19T18:18:51.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f12827fc640 1 -- 172.21.15.26:0/3415262661 learned_addr learned my addr 172.21.15.26:0/3415262661 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:51.132 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f1282ffd640 1 --2- 172.21.15.26:0/3415262661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12840b1590 0x7f12840c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f12827fc640 1 -- 172.21.15.26:0/3415262661 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128415aad0 msgr2=0x7f12841549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f12827fc640 1 --2- 172.21.15.26:0/3415262661 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f128415aad0 0x7f12841549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f12827fc640 1 -- 172.21.15.26:0/3415262661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12840b1590 msgr2=0x7f12840c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f12827fc640 1 --2- 172.21.15.26:0/3415262661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f12840b1590 0x7f12840c8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f12827fc640 1 -- 172.21.15.26:0/3415262661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12740094d0 con 0x7f12840b3f90 2023-12-19T18:18:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f12827fc640 1 --2- 172.21.15.26:0/3415262661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12840b3f90 0x7f1284154100 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f127800b480 tx=0x7f127800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f126b7fe640 1 -- 172.21.15.26:0/3415262661 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278010870 con 0x7f12840b3f90 2023-12-19T18:18:51.133 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f126b7fe640 1 -- 172.21.15.26:0/3415262661 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12780109d0 con 0x7f12840b3f90 2023-12-19T18:18:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f126b7fe640 1 -- 172.21.15.26:0/3415262661 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278010d80 con 0x7f12840b3f90 2023-12-19T18:18:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f1289673640 1 -- 172.21.15.26:0/3415262661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12840c4b30 con 0x7f12840b3f90 2023-12-19T18:18:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f1289673640 1 -- 172.21.15.26:0/3415262661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12840c8880 con 0x7f12840b3f90 2023-12-19T18:18:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f1289673640 1 -- 172.21.15.26:0/3415262661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f12840ca540 con 0x7f12840b3f90 2023-12-19T18:18:51.134 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.132+0000 7f126b7fe640 1 -- 172.21.15.26:0/3415262661 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f127800f3c0 con 0x7f12840b3f90 2023-12-19T18:18:51.135 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.136+0000 7f126b7fe640 1 --2- 172.21.15.26:0/3415262661 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1270065b80 0x7f1270068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.136+0000 7f126b7fe640 1 -- 172.21.15.26:0/3415262661 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f1278083540 con 0x7f12840b3f90 2023-12-19T18:18:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.136+0000 7f126b7fe640 1 --2- 172.21.15.26:0/3415262661 >> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] conn(0x7f127006c760 0x7f127006eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.136+0000 7f126b7fe640 1 -- 172.21.15.26:0/3415262661 --> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f127006f180 con 0x7f127006c760 2023-12-19T18:18:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.136+0000 7f126b7fe640 1 -- 172.21.15.26:0/3415262661 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=259) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1278085b20 con 0x7f12840b3f90 2023-12-19T18:18:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.136+0000 7f1282ffd640 1 --2- 172.21.15.26:0/3415262661 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1270065b80 0x7f1270068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.136+0000 7f12837fe640 1 --2- 172.21.15.26:0/3415262661 >> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] conn(0x7f127006c760 0x7f127006eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.136 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.136+0000 7f12837fe640 1 --2- 172.21.15.26:0/3415262661 >> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] conn(0x7f127006c760 0x7f127006eb50 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.136+0000 7f1282ffd640 1 --2- 172.21.15.26:0/3415262661 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1270065b80 0x7f1270068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f12740ae000 tx=0x7f12740b5040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.137 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.136+0000 7f126b7fe640 1 -- 172.21.15.26:0/3415262661 <== osd.2 v2:172.21.15.26:6809/38119 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f127006f180 con 0x7f127006c760 2023-12-19T18:18:51.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.148+0000 7f1289673640 1 -- 172.21.15.26:0/3415262661 --> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1284074270 con 0x7f127006c760 2023-12-19T18:18:51.150 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.148+0000 7f126b7fe640 1 -- 172.21.15.26:0/3415262661 <== osd.2 v2:172.21.15.26:6809/38119 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1284074270 con 0x7f127006c760 2023-12-19T18:18:51.153 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:18:51.153 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-19T18:18:51.153 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:18:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.148+0000 7f1289673640 1 -- 172.21.15.26:0/3415262661 >> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] conn(0x7f127006c760 msgr2=0x7f127006eb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.148+0000 7f1289673640 1 --2- 172.21.15.26:0/3415262661 >> [v2:172.21.15.26:6809/38119,v1:172.21.15.26:6812/38119] conn(0x7f127006c760 0x7f127006eb50 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.154 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.148+0000 7f1289673640 1 -- 172.21.15.26:0/3415262661 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1270065b80 msgr2=0x7f1270068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.148+0000 7f1289673640 1 --2- 172.21.15.26:0/3415262661 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1270065b80 0x7f1270068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f12740ae000 tx=0x7f12740b5040 comp rx=0 tx=0).stop 2023-12-19T18:18:51.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.148+0000 7f1289673640 1 -- 172.21.15.26:0/3415262661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12840b3f90 msgr2=0x7f1284154100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.148+0000 7f1289673640 1 --2- 172.21.15.26:0/3415262661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12840b3f90 0x7f1284154100 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f127800b480 tx=0x7f127800b940 comp rx=0 tx=0).stop 2023-12-19T18:18:51.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.148+0000 7f12837fe640 1 -- 172.21.15.26:0/3415262661 reap_dead start 2023-12-19T18:18:51.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.148+0000 7f1289673640 1 -- 172.21.15.26:0/3415262661 shutdown_connections 2023-12-19T18:18:51.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.148+0000 7f1289673640 1 -- 172.21.15.26:0/3415262661 >> 172.21.15.26:0/3415262661 conn(0x7f12840b0270 msgr2=0x7f1284159850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:51.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.148+0000 7f1289673640 1 -- 172.21.15.26:0/3415262661 shutdown_connections 2023-12-19T18:18:51.155 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.148+0000 7f1289673640 1 -- 172.21.15.26:0/3415262661 wait complete. 2023-12-19T18:18:51.169 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-12-19T18:18:51.314 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.312+0000 7f2feb739640 1 Processor -- start 2023-12-19T18:18:51.315 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.312+0000 7f2feb739640 1 -- start start 2023-12-19T18:18:51.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.316+0000 7f2feb739640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2fe40b34e0 0x7f2fe40ad180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.316+0000 7f2feb739640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe40cc0d0 con 0x7f2fe406cb30 2023-12-19T18:18:51.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.316+0000 7f2feb739640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe415ddc0 con 0x7f2fe40cbc50 2023-12-19T18:18:51.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.316+0000 7f2feb739640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe415e0a0 con 0x7f2fe40b34e0 2023-12-19T18:18:51.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.316+0000 7f2fe9cd6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2fe40b34e0 0x7f2fe40ad180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.316 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.316+0000 7f2fe9cd6640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2fe40b34e0 0x7f2fe40ad180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53858/0 (socket says 172.21.15.26:53858) 2023-12-19T18:18:51.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.316+0000 7f2fe9cd6640 1 -- 172.21.15.26:0/487070216 learned_addr learned my addr 172.21.15.26:0/487070216 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:51.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.316+0000 7f2fe94d5640 1 --1- 172.21.15.26:0/487070216 >> v1:172.21.15.26:6789/0 conn(0x7f2fe406cb30 0x7f2fe415d2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42998/0 (socket says 172.21.15.26:42998) 2023-12-19T18:18:51.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.316+0000 7f2fe9cd6640 1 -- 172.21.15.26:0/487070216 >> v1:172.21.15.160:6789/0 conn(0x7f2fe40cbc50 legacy=0x7f2fe40b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.316+0000 7f2fe9cd6640 1 -- 172.21.15.26:0/487070216 >> v1:172.21.15.26:6789/0 conn(0x7f2fe406cb30 legacy=0x7f2fe415d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.316+0000 7f2fe9cd6640 1 -- 172.21.15.26:0/487070216 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe40cb450 con 0x7f2fe40b34e0 2023-12-19T18:18:51.317 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.316+0000 7f2fdbfff640 1 -- 172.21.15.26:0/487070216 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2430743749 0 0) 0x7f2fe0001120 con 0x7f2fe406cb30 2023-12-19T18:18:51.319 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.320+0000 7f2fe9cd6640 1 --2- 172.21.15.26:0/487070216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fe40b34e0 0x7f2fe40ad180 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009e50 tx=0x7f2fdc01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c3a38f514125fd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.320+0000 7f2fdbfff640 1 -- 172.21.15.26:0/487070216 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe40aded0 con 0x7f2fe40b34e0 2023-12-19T18:18:51.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.320+0000 7f2fdbfff640 1 -- 172.21.15.26:0/487070216 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc0a58d0 con 0x7f2fe40b34e0 2023-12-19T18:18:51.320 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.320+0000 7f2fdbfff640 1 -- 172.21.15.26:0/487070216 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc0a5c60 con 0x7f2fe40b34e0 2023-12-19T18:18:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.320+0000 7f2feb739640 1 -- 172.21.15.26:0/487070216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fe40b34e0 msgr2=0x7f2fe40ad180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.320+0000 7f2feb739640 1 --2- 172.21.15.26:0/487070216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fe40b34e0 0x7f2fe40ad180 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009e50 tx=0x7f2fdc01c9e0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.320+0000 7f2feb739640 1 -- 172.21.15.26:0/487070216 shutdown_connections 2023-12-19T18:18:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.320+0000 7f2feb739640 1 --2- 172.21.15.26:0/487070216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fe40b34e0 0x7f2fe40ad180 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.320+0000 7f2feb739640 1 -- 172.21.15.26:0/487070216 >> 172.21.15.26:0/487070216 conn(0x7f2fe40658e0 msgr2=0x7f2fe4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.320+0000 7f2feb739640 1 -- 172.21.15.26:0/487070216 shutdown_connections 2023-12-19T18:18:51.321 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.320+0000 7f2feb739640 1 -- 172.21.15.26:0/487070216 wait complete. 2023-12-19T18:18:51.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.320+0000 7f2feb739640 1 Processor -- start 2023-12-19T18:18:51.322 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.320+0000 7f2feb739640 1 -- start start 2023-12-19T18:18:51.324 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2feb739640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fe406cb30 0x7f2fe40709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2feb739640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2fe40b34e0 0x7f2fe412e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2feb739640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fe40cbc50 0x7f2fe4141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2feb739640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe412d230 con 0x7f2fe406cb30 2023-12-19T18:18:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2feb739640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe412d510 con 0x7f2fe40b34e0 2023-12-19T18:18:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2feb739640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe412d7f0 con 0x7f2fe40cbc50 2023-12-19T18:18:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fe94d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fe406cb30 0x7f2fe40709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fe94d5640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fe406cb30 0x7f2fe40709f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53868/0 (socket says 172.21.15.26:53868) 2023-12-19T18:18:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fe94d5640 1 -- 172.21.15.26:0/4206174211 learned_addr learned my addr 172.21.15.26:0/4206174211 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fe9cd6640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fe40cbc50 0x7f2fe4141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fe8cd4640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2fe40b34e0 0x7f2fe412e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fe94d5640 1 -- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fe40cbc50 msgr2=0x7f2fe4141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fe94d5640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fe40cbc50 0x7f2fe4141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.325 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fe94d5640 1 -- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2fe40b34e0 msgr2=0x7f2fe412e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fe94d5640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2fe40b34e0 0x7f2fe412e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fe94d5640 1 -- 172.21.15.26:0/4206174211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc009b30 con 0x7f2fe406cb30 2023-12-19T18:18:51.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fe8cd4640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2fe40b34e0 0x7f2fe412e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:51.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fe94d5640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fe406cb30 0x7f2fe40709f0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c780 tx=0x7f2fe000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fd9ffb640 1 -- 172.21.15.26:0/4206174211 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00072d0 con 0x7f2fe406cb30 2023-12-19T18:18:51.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fd9ffb640 1 -- 172.21.15.26:0/4206174211 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe0007430 con 0x7f2fe406cb30 2023-12-19T18:18:51.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fd9ffb640 1 -- 172.21.15.26:0/4206174211 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0001d20 con 0x7f2fe406cb30 2023-12-19T18:18:51.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2feb739640 1 -- 172.21.15.26:0/4206174211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe412dad0 con 0x7f2fe406cb30 2023-12-19T18:18:51.326 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2feb739640 1 -- 172.21.15.26:0/4206174211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe4145eb0 con 0x7f2fe406cb30 2023-12-19T18:18:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2feb739640 1 -- 172.21.15.26:0/4206174211 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe4074270 con 0x7f2fe406cb30 2023-12-19T18:18:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.324+0000 7f2fd9ffb640 1 -- 172.21.15.26:0/4206174211 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2fe00047f0 con 0x7f2fe406cb30 2023-12-19T18:18:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.328+0000 7f2fd9ffb640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2fc8065b80 0x7f2fc8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.327 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.328+0000 7f2fd9ffb640 1 -- 172.21.15.26:0/4206174211 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f2fe0082700 con 0x7f2fe406cb30 2023-12-19T18:18:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.328+0000 7f2fe8cd4640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2fc8065b80 0x7f2fc8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.328 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.328+0000 7f2fe8cd4640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2fc8065b80 0x7f2fc8068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4140680 tx=0x7f2fd4009040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.331 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.332+0000 7f2fd9ffb640 1 -- 172.21.15.26:0/4206174211 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2fe004fdb0 con 0x7f2fe406cb30 2023-12-19T18:18:51.491 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2feb739640 1 -- 172.21.15.26:0/4206174211 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fe4160ae0 con 0x7f2fc8065b80 2023-12-19T18:18:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2fd9ffb640 1 -- 172.21.15.26:0/4206174211 <== mgr.11683 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2fd4001ca0 con 0x7f2fc8065b80 2023-12-19T18:18:51.492 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 2 to deep-scrub 2023-12-19T18:18:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2feb739640 1 -- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2fc8065b80 msgr2=0x7f2fc8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2feb739640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2fc8065b80 0x7f2fc8068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4140680 tx=0x7f2fd4009040 comp rx=0 tx=0).stop 2023-12-19T18:18:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2feb739640 1 -- 172.21.15.26:0/4206174211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fe406cb30 msgr2=0x7f2fe40709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2feb739640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fe406cb30 0x7f2fe40709f0 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c780 tx=0x7f2fe000cc40 comp rx=0 tx=0).stop 2023-12-19T18:18:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2feb739640 1 -- 172.21.15.26:0/4206174211 shutdown_connections 2023-12-19T18:18:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2feb739640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2fc8065b80 0x7f2fc8068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2feb739640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2fe40cbc50 0x7f2fe4141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2feb739640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2fe40b34e0 0x7f2fe412e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.494 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2feb739640 1 --2- 172.21.15.26:0/4206174211 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fe406cb30 0x7f2fe40709f0 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2feb739640 1 -- 172.21.15.26:0/4206174211 >> 172.21.15.26:0/4206174211 conn(0x7f2fe40658e0 msgr2=0x7f2fe406f370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2feb739640 1 -- 172.21.15.26:0/4206174211 shutdown_connections 2023-12-19T18:18:51.495 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.492+0000 7f2feb739640 1 -- 172.21.15.26:0/4206174211 wait complete. 2023-12-19T18:18:51.513 INFO:tasks.ceph:Scrubbing osd.3 2023-12-19T18:18:51.513 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:18:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a842d1640 1 Processor -- start 2023-12-19T18:18:51.661 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a842d1640 1 -- start start 2023-12-19T18:18:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a842d1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9a7c0c9310 0x7f9a7c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a842d1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a7c0cb110 0x7f9a7c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a842d1640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a7c0cbaa0 con 0x7f9a7c159830 2023-12-19T18:18:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a842d1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a7c06f7c0 con 0x7f9a7c0c9310 2023-12-19T18:18:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a842d1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0ca2b0 con 0x7f9a7c0cb110 2023-12-19T18:18:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a8206d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9a7c159830 0x7f9a7c15d4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43010/0 (socket says 172.21.15.26:43010) 2023-12-19T18:18:51.662 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a8206d640 1 -- 172.21.15.26:0/1680149004 learned_addr learned my addr 172.21.15.26:0/1680149004 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a8286e640 1 --2- 172.21.15.26:0/1680149004 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a7c0cb110 0x7f9a7c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a8186c640 1 --2- 172.21.15.26:0/1680149004 >> v2:172.21.15.160:3300/0 conn(0x7f9a7c0c9310 0x7f9a7c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a8106b640 1 -- 172.21.15.26:0/1680149004 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654590881 0 0) 0x7f9a78001120 con 0x7f9a7c159830 2023-12-19T18:18:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a8106b640 1 -- 172.21.15.26:0/1680149004 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a68003530 con 0x7f9a7c159830 2023-12-19T18:18:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a8186c640 1 -- 172.21.15.26:0/1680149004 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a7c0cb110 msgr2=0x7f9a7c0cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a8186c640 1 --2- 172.21.15.26:0/1680149004 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a7c0cb110 0x7f9a7c0cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a8186c640 1 -- 172.21.15.26:0/1680149004 >> v1:172.21.15.26:6789/0 conn(0x7f9a7c159830 legacy=0x7f9a7c15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.660+0000 7f9a8186c640 1 -- 172.21.15.26:0/1680149004 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c15e3d0 con 0x7f9a7c0c9310 2023-12-19T18:18:51.664 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.664+0000 7f9a8286e640 1 --2- 172.21.15.26:0/1680149004 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a7c0cb110 0x7f9a7c0cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:51.665 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.664+0000 7f9a8186c640 1 --2- 172.21.15.26:0/1680149004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a7c0c9310 0x7f9a7c0cc430 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0097c0 tx=0x7f9a6c01e600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84cdd88bb9524e97 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.664+0000 7f9a8106b640 1 -- 172.21.15.26:0/1680149004 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0ac660 con 0x7f9a7c0c9310 2023-12-19T18:18:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.664+0000 7f9a8106b640 1 -- 172.21.15.26:0/1680149004 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c0ac7c0 con 0x7f9a7c0c9310 2023-12-19T18:18:51.666 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.664+0000 7f9a8106b640 1 -- 172.21.15.26:0/1680149004 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0acb50 con 0x7f9a7c0c9310 2023-12-19T18:18:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.664+0000 7f9a842d1640 1 -- 172.21.15.26:0/1680149004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a7c0c9310 msgr2=0x7f9a7c0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.664+0000 7f9a842d1640 1 --2- 172.21.15.26:0/1680149004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a7c0c9310 0x7f9a7c0cc430 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0097c0 tx=0x7f9a6c01e600 comp rx=0 tx=0).stop 2023-12-19T18:18:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 -- 172.21.15.26:0/1680149004 shutdown_connections 2023-12-19T18:18:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 --2- 172.21.15.26:0/1680149004 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a7c0cb110 0x7f9a7c0cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 --2- 172.21.15.26:0/1680149004 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a7c0c9310 0x7f9a7c0cc430 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 -- 172.21.15.26:0/1680149004 >> 172.21.15.26:0/1680149004 conn(0x7f9a7c0b0270 msgr2=0x7f9a7c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 -- 172.21.15.26:0/1680149004 shutdown_connections 2023-12-19T18:18:51.667 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 -- 172.21.15.26:0/1680149004 wait complete. 2023-12-19T18:18:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 Processor -- start 2023-12-19T18:18:51.668 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 -- start start 2023-12-19T18:18:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a7c0c9310 0x7f9a7c0c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a7c0cb110 0x7f9a7c1538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a7c159830 0x7f9a7c153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0c4540 con 0x7f9a7c0c9310 2023-12-19T18:18:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0c8720 con 0x7f9a7c159830 2023-12-19T18:18:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a842d1640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7c0c8160 con 0x7f9a7c0cb110 2023-12-19T18:18:51.671 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a8206d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a7c0c9310 0x7f9a7c0c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a8206d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a7c0c9310 0x7f9a7c0c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53870/0 (socket says 172.21.15.26:53870) 2023-12-19T18:18:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a8206d640 1 -- 172.21.15.26:0/3704532907 learned_addr learned my addr 172.21.15.26:0/3704532907 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a8286e640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a7c159830 0x7f9a7c153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a8186c640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a7c0cb110 0x7f9a7c1538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a8206d640 1 -- 172.21.15.26:0/3704532907 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a7c0cb110 msgr2=0x7f9a7c1538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a8206d640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9a7c0cb110 0x7f9a7c1538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a8206d640 1 -- 172.21.15.26:0/3704532907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a7c159830 msgr2=0x7f9a7c153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a8206d640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a7c159830 0x7f9a7c153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a8206d640 1 -- 172.21.15.26:0/3704532907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a74009a70 con 0x7f9a7c0c9310 2023-12-19T18:18:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a8206d640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a7c0c9310 0x7f9a7c0c49d0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800c420 tx=0x7f9a7800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.672 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.668+0000 7f9a72ffd640 1 -- 172.21.15.26:0/3704532907 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78005cf0 con 0x7f9a7c0c9310 2023-12-19T18:18:51.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.672+0000 7f9a72ffd640 1 -- 172.21.15.26:0/3704532907 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a78005e50 con 0x7f9a7c0c9310 2023-12-19T18:18:51.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.672+0000 7f9a72ffd640 1 -- 172.21.15.26:0/3704532907 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a780072f0 con 0x7f9a7c0c9310 2023-12-19T18:18:51.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.672+0000 7f9a842d1640 1 -- 172.21.15.26:0/3704532907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6c009420 con 0x7f9a7c0c9310 2023-12-19T18:18:51.673 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.672+0000 7f9a842d1640 1 -- 172.21.15.26:0/3704532907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7c06fa80 con 0x7f9a7c0c9310 2023-12-19T18:18:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.672+0000 7f9a842d1640 1 -- 172.21.15.26:0/3704532907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a7c06fcb0 con 0x7f9a7c0c9310 2023-12-19T18:18:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.672+0000 7f9a72ffd640 1 -- 172.21.15.26:0/3704532907 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9a78007450 con 0x7f9a7c0c9310 2023-12-19T18:18:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.672+0000 7f9a72ffd640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9a60065b80 0x7f9a60068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.674 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.672+0000 7f9a72ffd640 1 -- 172.21.15.26:0/3704532907 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f9a78082a00 con 0x7f9a7c0c9310 2023-12-19T18:18:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.672+0000 7f9a72ffd640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] conn(0x7f9a6006c760 0x7f9a6006eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.672+0000 7f9a72ffd640 1 -- 172.21.15.26:0/3704532907 --> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a6006f180 con 0x7f9a6006c760 2023-12-19T18:18:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.672+0000 7f9a72ffd640 1 -- 172.21.15.26:0/3704532907 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=259) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a780872a0 con 0x7f9a7c0c9310 2023-12-19T18:18:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.672+0000 7f9a8186c640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9a60065b80 0x7f9a60068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.672+0000 7f9a8286e640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] conn(0x7f9a6006c760 0x7f9a6006eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.676+0000 7f9a8286e640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] conn(0x7f9a6006c760 0x7f9a6006eb50 crc :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.675 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.676+0000 7f9a8186c640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9a60065b80 0x7f9a60068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0b0000 tx=0x7f9a6c0a9000 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.676 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.676+0000 7f9a72ffd640 1 -- 172.21.15.26:0/3704532907 <== osd.3 v2:172.21.15.26:6824/38120 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9a6006f180 con 0x7f9a6006c760 2023-12-19T18:18:51.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.688+0000 7f9a842d1640 1 -- 172.21.15.26:0/3704532907 --> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9a7c0c4680 con 0x7f9a6006c760 2023-12-19T18:18:51.689 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.688+0000 7f9a72ffd640 1 -- 172.21.15.26:0/3704532907 <== osd.3 v2:172.21.15.26:6824/38120 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9a7400f040 con 0x7f9a6006c760 2023-12-19T18:18:51.689 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:18:51.689 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-19T18:18:51.689 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:18:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.688+0000 7f9a842d1640 1 -- 172.21.15.26:0/3704532907 >> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] conn(0x7f9a6006c760 msgr2=0x7f9a6006eb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.688+0000 7f9a842d1640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.26:6824/38120,v1:172.21.15.26:6825/38120] conn(0x7f9a6006c760 0x7f9a6006eb50 crc :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.688+0000 7f9a842d1640 1 -- 172.21.15.26:0/3704532907 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9a60065b80 msgr2=0x7f9a60068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.688+0000 7f9a842d1640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9a60065b80 0x7f9a60068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0b0000 tx=0x7f9a6c0a9000 comp rx=0 tx=0).stop 2023-12-19T18:18:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.688+0000 7f9a842d1640 1 -- 172.21.15.26:0/3704532907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a7c0c9310 msgr2=0x7f9a7c0c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.688+0000 7f9a842d1640 1 --2- 172.21.15.26:0/3704532907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a7c0c9310 0x7f9a7c0c49d0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800c420 tx=0x7f9a7800c820 comp rx=0 tx=0).stop 2023-12-19T18:18:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.688+0000 7f9a8286e640 1 -- 172.21.15.26:0/3704532907 reap_dead start 2023-12-19T18:18:51.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.688+0000 7f9a842d1640 1 -- 172.21.15.26:0/3704532907 shutdown_connections 2023-12-19T18:18:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.688+0000 7f9a842d1640 1 -- 172.21.15.26:0/3704532907 >> 172.21.15.26:0/3704532907 conn(0x7f9a7c0b0270 msgr2=0x7f9a7c0af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.688+0000 7f9a842d1640 1 -- 172.21.15.26:0/3704532907 shutdown_connections 2023-12-19T18:18:51.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.688+0000 7f9a842d1640 1 -- 172.21.15.26:0/3704532907 wait complete. 2023-12-19T18:18:51.707 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-12-19T18:18:51.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.848+0000 7f3f8eaf9640 1 Processor -- start 2023-12-19T18:18:51.851 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.848+0000 7f3f8eaf9640 1 -- start start 2023-12-19T18:18:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f8eaf9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3f8806b3d0 0x7f3f880ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f8eaf9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f880cec40 0x7f3f8815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f8eaf9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8806bc30 con 0x7f3f88154220 2023-12-19T18:18:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f8eaf9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f880cf0d0 con 0x7f3f8806b3d0 2023-12-19T18:18:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f8eaf9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880cc6e0 con 0x7f3f880cec40 2023-12-19T18:18:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f87fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3f8806b3d0 0x7f3f880ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f87fff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3f8806b3d0 0x7f3f880ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53884/0 (socket says 172.21.15.26:53884) 2023-12-19T18:18:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f87fff640 1 -- 172.21.15.26:0/510141939 learned_addr learned my addr 172.21.15.26:0/510141939 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f8d096640 1 --2- 172.21.15.26:0/510141939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f880cec40 0x7f3f8815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f87fff640 1 -- 172.21.15.26:0/510141939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f880cec40 msgr2=0x7f3f8815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:51.853 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f87fff640 1 --2- 172.21.15.26:0/510141939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f880cec40 0x7f3f8815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f87fff640 1 -- 172.21.15.26:0/510141939 >> v1:172.21.15.160:6789/0 conn(0x7f3f88154220 legacy=0x7f3f8815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f87fff640 1 -- 172.21.15.26:0/510141939 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8806b950 con 0x7f3f8806b3d0 2023-12-19T18:18:51.854 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.852+0000 7f3f8d096640 1 --2- 172.21.15.26:0/510141939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f880cec40 0x7f3f8815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:51.855 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f87fff640 1 --2- 172.21.15.26:0/510141939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f8806b3d0 0x7f3f880ce700 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009870 tx=0x7f3f7401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5baf64e1417d3bc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f877fe640 1 -- 172.21.15.26:0/510141939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f740aa5f0 con 0x7f3f8806b3d0 2023-12-19T18:18:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f877fe640 1 -- 172.21.15.26:0/510141939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f740aa750 con 0x7f3f8806b3d0 2023-12-19T18:18:51.856 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f877fe640 1 -- 172.21.15.26:0/510141939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f740aaae0 con 0x7f3f8806b3d0 2023-12-19T18:18:51.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/510141939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f8806b3d0 msgr2=0x7f3f880ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f8eaf9640 1 --2- 172.21.15.26:0/510141939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f8806b3d0 0x7f3f880ce700 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009870 tx=0x7f3f7401c4d0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/510141939 shutdown_connections 2023-12-19T18:18:51.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f8eaf9640 1 --2- 172.21.15.26:0/510141939 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f880cec40 0x7f3f8815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f8eaf9640 1 --2- 172.21.15.26:0/510141939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f8806b3d0 0x7f3f880ce700 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.857 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/510141939 >> 172.21.15.26:0/510141939 conn(0x7f3f880658e0 msgr2=0x7f3f88065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/510141939 shutdown_connections 2023-12-19T18:18:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/510141939 wait complete. 2023-12-19T18:18:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f8eaf9640 1 Processor -- start 2023-12-19T18:18:51.858 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.856+0000 7f3f8eaf9640 1 -- start start 2023-12-19T18:18:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8eaf9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f8806b3d0 0x7f3f880b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8eaf9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f880cec40 0x7f3f88158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8eaf9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f88154220 0x7f3f881592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8eaf9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f881fdcc0 con 0x7f3f8806b3d0 2023-12-19T18:18:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8eaf9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f881fd0b0 con 0x7f3f88154220 2023-12-19T18:18:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8eaf9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f881fd390 con 0x7f3f880cec40 2023-12-19T18:18:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8c895640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f8806b3d0 0x7f3f880b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8c895640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f8806b3d0 0x7f3f880b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53890/0 (socket says 172.21.15.26:53890) 2023-12-19T18:18:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8c895640 1 -- 172.21.15.26:0/4019306520 learned_addr learned my addr 172.21.15.26:0/4019306520 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f87fff640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f880cec40 0x7f3f88158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8d096640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f88154220 0x7f3f881592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8c895640 1 -- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f880cec40 msgr2=0x7f3f88158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8c895640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f880cec40 0x7f3f88158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.861 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8c895640 1 -- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f88154220 msgr2=0x7f3f881592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8c895640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f88154220 0x7f3f881592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8c895640 1 -- 172.21.15.26:0/4019306520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740094d0 con 0x7f3f8806b3d0 2023-12-19T18:18:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8d096640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f88154220 0x7f3f881592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8c895640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f8806b3d0 0x7f3f880b17c0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00b410 tx=0x7f3f7c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f857fa640 1 -- 172.21.15.26:0/4019306520 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c004020 con 0x7f3f8806b3d0 2023-12-19T18:18:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f857fa640 1 -- 172.21.15.26:0/4019306520 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c004180 con 0x7f3f8806b3d0 2023-12-19T18:18:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f857fa640 1 -- 172.21.15.26:0/4019306520 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c011630 con 0x7f3f8806b3d0 2023-12-19T18:18:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/4019306520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f881fd670 con 0x7f3f8806b3d0 2023-12-19T18:18:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/4019306520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f88145de0 con 0x7f3f8806b3d0 2023-12-19T18:18:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/4019306520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8806d450 con 0x7f3f8806b3d0 2023-12-19T18:18:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.860+0000 7f3f857fa640 1 -- 172.21.15.26:0/4019306520 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3f7c011790 con 0x7f3f8806b3d0 2023-12-19T18:18:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.864+0000 7f3f857fa640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3f6c065b80 0x7f3f6c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.864+0000 7f3f857fa640 1 -- 172.21.15.26:0/4019306520 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f3f7c0832e0 con 0x7f3f8806b3d0 2023-12-19T18:18:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.864+0000 7f3f87fff640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3f6c065b80 0x7f3f6c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:51.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.864+0000 7f3f87fff640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3f6c065b80 0x7f3f6c068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f740ae000 tx=0x7f3f740b5040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:51.868+0000 7f3f857fa640 1 -- 172.21.15.26:0/4019306520 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3f7c0508e0 con 0x7f3f8806b3d0 2023-12-19T18:18:52.024 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.024+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/4019306520 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f880ccef0 con 0x7f3f6c065b80 2023-12-19T18:18:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.024+0000 7f3f857fa640 1 -- 172.21.15.26:0/4019306520 <== mgr.11683 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3f740aa940 con 0x7f3f6c065b80 2023-12-19T18:18:52.025 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 3 to deep-scrub 2023-12-19T18:18:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.024+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3f6c065b80 msgr2=0x7f3f6c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.024+0000 7f3f8eaf9640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3f6c065b80 0x7f3f6c068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f740ae000 tx=0x7f3f740b5040 comp rx=0 tx=0).stop 2023-12-19T18:18:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.024+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/4019306520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f8806b3d0 msgr2=0x7f3f880b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.024+0000 7f3f8eaf9640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f8806b3d0 0x7f3f880b17c0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00b410 tx=0x7f3f7c00b8d0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.024+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/4019306520 shutdown_connections 2023-12-19T18:18:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.024+0000 7f3f8eaf9640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3f6c065b80 0x7f3f6c068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.024+0000 7f3f8eaf9640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f88154220 0x7f3f881592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.024+0000 7f3f8eaf9640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f3f880cec40 0x7f3f88158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.024+0000 7f3f8eaf9640 1 --2- 172.21.15.26:0/4019306520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f8806b3d0 0x7f3f880b17c0 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.024+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/4019306520 >> 172.21.15.26:0/4019306520 conn(0x7f3f880658e0 msgr2=0x7f3f8815c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.024+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/4019306520 shutdown_connections 2023-12-19T18:18:52.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.028+0000 7f3f8eaf9640 1 -- 172.21.15.26:0/4019306520 wait complete. 2023-12-19T18:18:52.046 INFO:tasks.ceph:Scrubbing osd.4 2023-12-19T18:18:52.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:18:52.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.192+0000 7f6eccff0640 1 Processor -- start 2023-12-19T18:18:52.194 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.192+0000 7f6eccff0640 1 -- start start 2023-12-19T18:18:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.192+0000 7f6eccff0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ec80cc310 0x7f6ec80cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.195 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.192+0000 7f6eccff0640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6ec815dcc0 0x7f6ec806b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.192+0000 7f6eccff0640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec806f290 con 0x7f6ec815ced0 2023-12-19T18:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.192+0000 7f6eccff0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec815e230 con 0x7f6ec80cc310 2023-12-19T18:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.192+0000 7f6eccff0640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec80ccc30 con 0x7f6ec815dcc0 2023-12-19T18:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.196+0000 7f6ec659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ec80cc310 0x7f6ec80cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.196+0000 7f6ec659c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ec80cc310 0x7f6ec80cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53898/0 (socket says 172.21.15.26:53898) 2023-12-19T18:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.196+0000 7f6ec659c640 1 -- 172.21.15.26:0/4074849170 learned_addr learned my addr 172.21.15.26:0/4074849170 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.196+0000 7f6ec6d9d640 1 --2- 172.21.15.26:0/4074849170 >> v2:172.21.15.160:3300/0 conn(0x7f6ec815dcc0 0x7f6ec806b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.196+0000 7f6ec659c640 1 -- 172.21.15.26:0/4074849170 >> v2:172.21.15.160:3300/0 conn(0x7f6ec815dcc0 msgr2=0x7f6ec806b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.196+0000 7f6ec659c640 1 --2- 172.21.15.26:0/4074849170 >> v2:172.21.15.160:3300/0 conn(0x7f6ec815dcc0 0x7f6ec806b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.196+0000 7f6ec659c640 1 -- 172.21.15.26:0/4074849170 >> v1:172.21.15.160:6789/0 conn(0x7f6ec815ced0 legacy=0x7f6ec806b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.196 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.196+0000 7f6ec659c640 1 -- 172.21.15.26:0/4074849170 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec8161720 con 0x7f6ec80cc310 2023-12-19T18:18:52.198 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.196+0000 7f6ec659c640 1 --2- 172.21.15.26:0/4074849170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ec80cc310 0x7f6ec80cc6f0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80cc230 tx=0x7f6eb801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6f0ab58a955e60 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.196+0000 7f6ec5d9b640 1 -- 172.21.15.26:0/4074849170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80ac410 con 0x7f6ec80cc310 2023-12-19T18:18:52.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.196+0000 7f6ec5d9b640 1 -- 172.21.15.26:0/4074849170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb80ac570 con 0x7f6ec80cc310 2023-12-19T18:18:52.199 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.196+0000 7f6ec5d9b640 1 -- 172.21.15.26:0/4074849170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80ac900 con 0x7f6ec80cc310 2023-12-19T18:18:52.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 -- 172.21.15.26:0/4074849170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ec80cc310 msgr2=0x7f6ec80cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 --2- 172.21.15.26:0/4074849170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ec80cc310 0x7f6ec80cc6f0 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80cc230 tx=0x7f6eb801c3b0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 -- 172.21.15.26:0/4074849170 shutdown_connections 2023-12-19T18:18:52.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 --2- 172.21.15.26:0/4074849170 >> v2:172.21.15.160:3300/0 conn(0x7f6ec815dcc0 0x7f6ec806b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 --2- 172.21.15.26:0/4074849170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ec80cc310 0x7f6ec80cc6f0 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 -- 172.21.15.26:0/4074849170 >> 172.21.15.26:0/4074849170 conn(0x7f6ec80b0270 msgr2=0x7f6ec80b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:52.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 -- 172.21.15.26:0/4074849170 shutdown_connections 2023-12-19T18:18:52.200 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 -- 172.21.15.26:0/4074849170 wait complete. 2023-12-19T18:18:52.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 Processor -- start 2023-12-19T18:18:52.201 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 -- start start 2023-12-19T18:18:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ec80cc310 0x7f6ec8154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ec815ced0 0x7f6ec8152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ec815dcc0 0x7f6ec80b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.202 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec81546b0 con 0x7f6ec80cc310 2023-12-19T18:18:52.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80cbce0 con 0x7f6ec815dcc0 2023-12-19T18:18:52.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6eccff0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec80caa10 con 0x7f6ec815ced0 2023-12-19T18:18:52.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6ec659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ec80cc310 0x7f6ec8154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6ec659c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ec80cc310 0x7f6ec8154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53912/0 (socket says 172.21.15.26:53912) 2023-12-19T18:18:52.203 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6ec659c640 1 -- 172.21.15.26:0/1561645295 learned_addr learned my addr 172.21.15.26:0/1561645295 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:52.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6ec659c640 1 -- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ec815ced0 msgr2=0x7f6ec8152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6ec659c640 1 --2- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6ec815ced0 0x7f6ec8152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6ec659c640 1 -- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ec815dcc0 msgr2=0x7f6ec80b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6ec659c640 1 --2- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6ec815dcc0 0x7f6ec80b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.200+0000 7f6ec659c640 1 -- 172.21.15.26:0/1561645295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb80094d0 con 0x7f6ec80cc310 2023-12-19T18:18:52.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6ec659c640 1 --2- 172.21.15.26:0/1561645295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ec80cc310 0x7f6ec8154a40 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80af040 tx=0x7f6eb8019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6ebeffd640 1 -- 172.21.15.26:0/1561645295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80ac410 con 0x7f6ec80cc310 2023-12-19T18:18:52.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6ebeffd640 1 -- 172.21.15.26:0/1561645295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb80b7480 con 0x7f6ec80cc310 2023-12-19T18:18:52.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6ebeffd640 1 -- 172.21.15.26:0/1561645295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb80b7830 con 0x7f6ec80cc310 2023-12-19T18:18:52.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6eccff0640 1 -- 172.21.15.26:0/1561645295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec80cac60 con 0x7f6ec80cc310 2023-12-19T18:18:52.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6eccff0640 1 -- 172.21.15.26:0/1561645295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec81542d0 con 0x7f6ec80cc310 2023-12-19T18:18:52.205 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6eccff0640 1 -- 172.21.15.26:0/1561645295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6ec8161200 con 0x7f6ec80cc310 2023-12-19T18:18:52.206 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6ebeffd640 1 -- 172.21.15.26:0/1561645295 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6eb80b7990 con 0x7f6ec80cc310 2023-12-19T18:18:52.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6ebeffd640 1 --2- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6ea8065b80 0x7f6ea8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6ebeffd640 1 -- 172.21.15.26:0/1561645295 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f6eb811de40 con 0x7f6ec80cc310 2023-12-19T18:18:52.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6ebeffd640 1 --2- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] conn(0x7f6ea806c760 0x7f6ea806eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6ebeffd640 1 -- 172.21.15.26:0/1561645295 --> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ea806f180 con 0x7f6ea806c760 2023-12-19T18:18:52.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6ebeffd640 1 -- 172.21.15.26:0/1561645295 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=259) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eb8120420 con 0x7f6ec80cc310 2023-12-19T18:18:52.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6ebffff640 1 --2- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6ea8065b80 0x7f6ea8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.207 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.204+0000 7f6ec6d9d640 1 --2- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] conn(0x7f6ea806c760 0x7f6ea806eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.208+0000 7f6ebffff640 1 --2- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6ea8065b80 0x7f6ea8068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0000f70 tx=0x7f6eb00079c0 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.208+0000 7f6ec6d9d640 1 --2- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] conn(0x7f6ea806c760 0x7f6ea806eb50 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.208 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.208+0000 7f6ebeffd640 1 -- 172.21.15.26:0/1561645295 <== osd.4 v2:172.21.15.160:6800/38418 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6ea806f180 con 0x7f6ea806c760 2023-12-19T18:18:52.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.220+0000 7f6eccff0640 1 -- 172.21.15.26:0/1561645295 --> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6ec8160f20 con 0x7f6ea806c760 2023-12-19T18:18:52.221 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.220+0000 7f6ebeffd640 1 -- 172.21.15.26:0/1561645295 <== osd.4 v2:172.21.15.160:6800/38418 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6ec000f040 con 0x7f6ea806c760 2023-12-19T18:18:52.222 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:18:52.222 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-19T18:18:52.222 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:18:52.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.220+0000 7f6eccff0640 1 -- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] conn(0x7f6ea806c760 msgr2=0x7f6ea806eb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.220+0000 7f6eccff0640 1 --2- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:6800/38418,v1:172.21.15.160:6801/38418] conn(0x7f6ea806c760 0x7f6ea806eb50 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.220+0000 7f6eccff0640 1 -- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6ea8065b80 msgr2=0x7f6ea8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.220+0000 7f6eccff0640 1 --2- 172.21.15.26:0/1561645295 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6ea8065b80 0x7f6ea8068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0000f70 tx=0x7f6eb00079c0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.220+0000 7f6eccff0640 1 -- 172.21.15.26:0/1561645295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ec80cc310 msgr2=0x7f6ec8154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.220+0000 7f6eccff0640 1 --2- 172.21.15.26:0/1561645295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ec80cc310 0x7f6ec8154a40 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80af040 tx=0x7f6eb8019150 comp rx=0 tx=0).stop 2023-12-19T18:18:52.223 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.220+0000 7f6ec6d9d640 1 -- 172.21.15.26:0/1561645295 reap_dead start 2023-12-19T18:18:52.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.220+0000 7f6eccff0640 1 -- 172.21.15.26:0/1561645295 shutdown_connections 2023-12-19T18:18:52.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.220+0000 7f6eccff0640 1 -- 172.21.15.26:0/1561645295 >> 172.21.15.26:0/1561645295 conn(0x7f6ec80b0270 msgr2=0x7f6ec80aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:52.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.220+0000 7f6eccff0640 1 -- 172.21.15.26:0/1561645295 shutdown_connections 2023-12-19T18:18:52.224 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.220+0000 7f6eccff0640 1 -- 172.21.15.26:0/1561645295 wait complete. 2023-12-19T18:18:52.241 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-12-19T18:18:52.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.384+0000 7f1f24fb0640 1 Processor -- start 2023-12-19T18:18:52.386 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.384+0000 7f1f24fb0640 1 -- start start 2023-12-19T18:18:52.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f24fb0640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f20159d20 0x7f1f200ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f24fb0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f200cdee0 0x7f1f200b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f24fb0640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f200adde0 con 0x7f1f200cc0a0 2023-12-19T18:18:52.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f24fb0640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f200c9c20 con 0x7f1f20159d20 2023-12-19T18:18:52.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f24fb0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200b28d0 con 0x7f1f200cdee0 2023-12-19T18:18:52.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f1effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f20159d20 0x7f1f200ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f1f7fe640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1f200cc0a0 0x7f1f200cc480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43014/0 (socket says 172.21.15.26:43014) 2023-12-19T18:18:52.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f1effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f20159d20 0x7f1f200ccb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53928/0 (socket says 172.21.15.26:53928) 2023-12-19T18:18:52.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f1f7fe640 1 -- 172.21.15.26:0/1989855866 learned_addr learned my addr 172.21.15.26:0/1989855866 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:52.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f1ffff640 1 --2- 172.21.15.26:0/1989855866 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f200cdee0 0x7f1f200b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f1effd640 1 -- 172.21.15.26:0/1989855866 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f200cdee0 msgr2=0x7f1f200b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:52.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f1effd640 1 --2- 172.21.15.26:0/1989855866 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f200cdee0 0x7f1f200b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f1effd640 1 -- 172.21.15.26:0/1989855866 >> v1:172.21.15.26:6789/0 conn(0x7f1f200cc0a0 legacy=0x7f1f200cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f1effd640 1 -- 172.21.15.26:0/1989855866 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200adb00 con 0x7f1f20159d20 2023-12-19T18:18:52.389 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f1e7fc640 1 -- 172.21.15.26:0/1989855866 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889948021 0 0) 0x7f1f200adde0 con 0x7f1f200cc0a0 2023-12-19T18:18:52.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.388+0000 7f1f1effd640 1 --2- 172.21.15.26:0/1989855866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f20159d20 0x7f1f200ccb90 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400ba60 tx=0x7f1f1401ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fefe378bcbf70dc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f1e7fc640 1 -- 172.21.15.26:0/1989855866 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140a7b80 con 0x7f1f20159d20 2023-12-19T18:18:52.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f1e7fc640 1 -- 172.21.15.26:0/1989855866 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f140a7ce0 con 0x7f1f20159d20 2023-12-19T18:18:52.391 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f1e7fc640 1 -- 172.21.15.26:0/1989855866 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140af660 con 0x7f1f20159d20 2023-12-19T18:18:52.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 -- 172.21.15.26:0/1989855866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f20159d20 msgr2=0x7f1f200ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 --2- 172.21.15.26:0/1989855866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f20159d20 0x7f1f200ccb90 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f1f1400ba60 tx=0x7f1f1401ea90 comp rx=0 tx=0).stop 2023-12-19T18:18:52.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 -- 172.21.15.26:0/1989855866 shutdown_connections 2023-12-19T18:18:52.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 --2- 172.21.15.26:0/1989855866 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f200cdee0 0x7f1f200b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 --2- 172.21.15.26:0/1989855866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f20159d20 0x7f1f200ccb90 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 -- 172.21.15.26:0/1989855866 >> 172.21.15.26:0/1989855866 conn(0x7f1f2006b0f0 msgr2=0x7f1f2006b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:52.392 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 -- 172.21.15.26:0/1989855866 shutdown_connections 2023-12-19T18:18:52.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 -- 172.21.15.26:0/1989855866 wait complete. 2023-12-19T18:18:52.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 Processor -- start 2023-12-19T18:18:52.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 -- start start 2023-12-19T18:18:52.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200cc0a0 0x7f1f20155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f200cdee0 0x7f1f2015a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f20159d20 0x7f1f2015a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2015d190 con 0x7f1f200cc0a0 2023-12-19T18:18:52.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200c6030 con 0x7f1f20159d20 2023-12-19T18:18:52.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f24fb0640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20155d60 con 0x7f1f200cdee0 2023-12-19T18:18:52.395 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f1f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200cc0a0 0x7f1f20155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f1f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200cc0a0 0x7f1f20155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53934/0 (socket says 172.21.15.26:53934) 2023-12-19T18:18:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.392+0000 7f1f1f7fe640 1 -- 172.21.15.26:0/2661583119 learned_addr learned my addr 172.21.15.26:0/2661583119 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f1effd640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f200cdee0 0x7f1f2015a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f1ffff640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f20159d20 0x7f1f2015a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f1f7fe640 1 -- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f200cdee0 msgr2=0x7f1f2015a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f1f7fe640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f200cdee0 0x7f1f2015a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f1f7fe640 1 -- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f20159d20 msgr2=0x7f1f2015a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f1f7fe640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f20159d20 0x7f1f2015a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f1f7fe640 1 -- 172.21.15.26:0/2661583119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1400b6c0 con 0x7f1f200cc0a0 2023-12-19T18:18:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f1ffff640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f20159d20 0x7f1f2015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:52.396 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f1f7fe640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200cc0a0 0x7f1f20155820 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f1f100011f0 tx=0x7f1f1000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f07fff640 1 -- 172.21.15.26:0/2661583119 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f100074e0 con 0x7f1f200cc0a0 2023-12-19T18:18:52.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f07fff640 1 -- 172.21.15.26:0/2661583119 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f10007640 con 0x7f1f200cc0a0 2023-12-19T18:18:52.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f07fff640 1 -- 172.21.15.26:0/2661583119 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10004a10 con 0x7f1f200cc0a0 2023-12-19T18:18:52.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f24fb0640 1 -- 172.21.15.26:0/2661583119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f200c62a0 con 0x7f1f200cc0a0 2023-12-19T18:18:52.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f24fb0640 1 -- 172.21.15.26:0/2661583119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f200c6860 con 0x7f1f200cc0a0 2023-12-19T18:18:52.397 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f24fb0640 1 -- 172.21.15.26:0/2661583119 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f200cdc40 con 0x7f1f200cc0a0 2023-12-19T18:18:52.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f07fff640 1 -- 172.21.15.26:0/2661583119 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1f10004b70 con 0x7f1f200cc0a0 2023-12-19T18:18:52.398 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f07fff640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f0c065b80 0x7f1f0c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.396+0000 7f1f07fff640 1 -- 172.21.15.26:0/2661583119 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f1f10082840 con 0x7f1f200cc0a0 2023-12-19T18:18:52.399 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.400+0000 7f1f1effd640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f0c065b80 0x7f1f0c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.400 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.400+0000 7f1f1effd640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f0c065b80 0x7f1f0c068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1f140b1000 tx=0x7f1f140a9070 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.402 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.400+0000 7f1f07fff640 1 -- 172.21.15.26:0/2661583119 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1f1004fed0 con 0x7f1f200cc0a0 2023-12-19T18:18:52.558 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.556+0000 7f1f24fb0640 1 -- 172.21.15.26:0/2661583119 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f20067400 con 0x7f1f0c065b80 2023-12-19T18:18:52.559 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.556+0000 7f1f07fff640 1 -- 172.21.15.26:0/2661583119 <== mgr.11683 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1f20067400 con 0x7f1f0c065b80 2023-12-19T18:18:52.559 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 4 to deep-scrub 2023-12-19T18:18:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.560+0000 7f1f24fb0640 1 -- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f0c065b80 msgr2=0x7f1f0c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.560+0000 7f1f24fb0640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f0c065b80 0x7f1f0c068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1f140b1000 tx=0x7f1f140a9070 comp rx=0 tx=0).stop 2023-12-19T18:18:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.560+0000 7f1f24fb0640 1 -- 172.21.15.26:0/2661583119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200cc0a0 msgr2=0x7f1f20155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.560+0000 7f1f24fb0640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200cc0a0 0x7f1f20155820 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f1f100011f0 tx=0x7f1f1000c9b0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.560+0000 7f1f24fb0640 1 -- 172.21.15.26:0/2661583119 shutdown_connections 2023-12-19T18:18:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.560+0000 7f1f24fb0640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f1f0c065b80 0x7f1f0c068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.560+0000 7f1f24fb0640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f20159d20 0x7f1f2015a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.560+0000 7f1f24fb0640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f1f200cdee0 0x7f1f2015a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.560+0000 7f1f24fb0640 1 --2- 172.21.15.26:0/2661583119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f200cc0a0 0x7f1f20155820 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.560+0000 7f1f24fb0640 1 -- 172.21.15.26:0/2661583119 >> 172.21.15.26:0/2661583119 conn(0x7f1f2006b0f0 msgr2=0x7f1f2006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:52.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.560+0000 7f1f24fb0640 1 -- 172.21.15.26:0/2661583119 shutdown_connections 2023-12-19T18:18:52.562 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.560+0000 7f1f24fb0640 1 -- 172.21.15.26:0/2661583119 wait complete. 2023-12-19T18:18:52.582 INFO:tasks.ceph:Scrubbing osd.5 2023-12-19T18:18:52.582 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:18:52.729 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.728+0000 7f30eab0a640 1 Processor -- start 2023-12-19T18:18:52.730 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.728+0000 7f30eab0a640 1 -- start start 2023-12-19T18:18:52.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.728+0000 7f30eab0a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30e4159830 0x7f30e415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.728+0000 7f30eab0a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f30e40c9310 0x7f30e40cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.728+0000 7f30eab0a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e415e3d0 con 0x7f30e40cb110 2023-12-19T18:18:52.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.728+0000 7f30eab0a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e406f7c0 con 0x7f30e40c9310 2023-12-19T18:18:52.731 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.728+0000 7f30eab0a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40ca2b0 con 0x7f30e4159830 2023-12-19T18:18:52.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e88a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30e4159830 0x7f30e415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e88a6640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30e4159830 0x7f30e415d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:44366/0 (socket says 172.21.15.26:44366) 2023-12-19T18:18:52.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e88a6640 1 -- 172.21.15.26:0/3694750525 learned_addr learned my addr 172.21.15.26:0/3694750525 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:52.732 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e3fff640 1 --2- 172.21.15.26:0/3694750525 >> v2:172.21.15.160:3300/0 conn(0x7f30e40c9310 0x7f30e40cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e37fe640 1 -- 172.21.15.26:0/3694750525 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289500905 0 0) 0x7f30e415e3d0 con 0x7f30e40cb110 2023-12-19T18:18:52.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e37fe640 1 -- 172.21.15.26:0/3694750525 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30d4003530 con 0x7f30e40cb110 2023-12-19T18:18:52.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e88a6640 1 -- 172.21.15.26:0/3694750525 >> v2:172.21.15.160:3300/0 conn(0x7f30e40c9310 msgr2=0x7f30e40cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:52.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e88a6640 1 --2- 172.21.15.26:0/3694750525 >> v2:172.21.15.160:3300/0 conn(0x7f30e40c9310 0x7f30e40cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e88a6640 1 -- 172.21.15.26:0/3694750525 >> v1:172.21.15.160:6789/0 conn(0x7f30e40cb110 legacy=0x7f30e40701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.733 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e88a6640 1 -- 172.21.15.26:0/3694750525 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e415de10 con 0x7f30e4159830 2023-12-19T18:18:52.734 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e88a6640 1 --2- 172.21.15.26:0/3694750525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30e4159830 0x7f30e415d4e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009870 tx=0x7f30d801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b37d2a2b6ec53860 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e37fe640 1 -- 172.21.15.26:0/3694750525 <== mon.2 v2:172.21.15.160:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d80ab680 con 0x7f30e4159830 2023-12-19T18:18:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e37fe640 1 -- 172.21.15.26:0/3694750525 <== mon.2 v2:172.21.15.160:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d80ab7e0 con 0x7f30e4159830 2023-12-19T18:18:52.735 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.732+0000 7f30e37fe640 1 -- 172.21.15.26:0/3694750525 <== mon.2 v2:172.21.15.160:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d80abb70 con 0x7f30e4159830 2023-12-19T18:18:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 -- 172.21.15.26:0/3694750525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30e4159830 msgr2=0x7f30e415d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 --2- 172.21.15.26:0/3694750525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30e4159830 0x7f30e415d4e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009870 tx=0x7f30d801c480 comp rx=0 tx=0).stop 2023-12-19T18:18:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 -- 172.21.15.26:0/3694750525 shutdown_connections 2023-12-19T18:18:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 --2- 172.21.15.26:0/3694750525 >> v2:172.21.15.160:3300/0 conn(0x7f30e40c9310 0x7f30e40cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 --2- 172.21.15.26:0/3694750525 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30e4159830 0x7f30e415d4e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 -- 172.21.15.26:0/3694750525 >> 172.21.15.26:0/3694750525 conn(0x7f30e40b0270 msgr2=0x7f30e40b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 -- 172.21.15.26:0/3694750525 shutdown_connections 2023-12-19T18:18:52.736 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 -- 172.21.15.26:0/3694750525 wait complete. 2023-12-19T18:18:52.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 Processor -- start 2023-12-19T18:18:52.737 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 -- start start 2023-12-19T18:18:52.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e40c9310 0x7f30e40c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30e40cb110 0x7f30e40c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4159830 0x7f30e40c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e41f5470 con 0x7f30e4159830 2023-12-19T18:18:52.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40c6900 con 0x7f30e40c9310 2023-12-19T18:18:52.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30eab0a640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40c4ff0 con 0x7f30e40cb110 2023-12-19T18:18:52.739 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30e90a7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4159830 0x7f30e40c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.736+0000 7f30e90a7640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4159830 0x7f30e40c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53938/0 (socket says 172.21.15.26:53938) 2023-12-19T18:18:52.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e90a7640 1 -- 172.21.15.26:0/3400875190 learned_addr learned my addr 172.21.15.26:0/3400875190 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:52.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e88a6640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e40c9310 0x7f30e40c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.740 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e3fff640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30e40cb110 0x7f30e40c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e90a7640 1 -- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30e40cb110 msgr2=0x7f30e40c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e90a7640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f30e40cb110 0x7f30e40c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e90a7640 1 -- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e40c9310 msgr2=0x7f30e40c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e90a7640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e40c9310 0x7f30e40c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.741 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e90a7640 1 -- 172.21.15.26:0/3400875190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d80094d0 con 0x7f30e4159830 2023-12-19T18:18:52.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e88a6640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e40c9310 0x7f30e40c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:52.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e90a7640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4159830 0x7f30e40c4ab0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c7c0 tx=0x7f30dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e17fa640 1 -- 172.21.15.26:0/3400875190 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc003b80 con 0x7f30e4159830 2023-12-19T18:18:52.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e17fa640 1 -- 172.21.15.26:0/3400875190 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc003ce0 con 0x7f30e4159830 2023-12-19T18:18:52.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e17fa640 1 -- 172.21.15.26:0/3400875190 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00aa50 con 0x7f30e4159830 2023-12-19T18:18:52.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30eab0a640 1 -- 172.21.15.26:0/3400875190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e40cbab0 con 0x7f30e4159830 2023-12-19T18:18:52.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30eab0a640 1 -- 172.21.15.26:0/3400875190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e41f5920 con 0x7f30e4159830 2023-12-19T18:18:52.742 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30eab0a640 1 -- 172.21.15.26:0/3400875190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f30e40cb930 con 0x7f30e4159830 2023-12-19T18:18:52.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e17fa640 1 -- 172.21.15.26:0/3400875190 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f30dc00abb0 con 0x7f30e4159830 2023-12-19T18:18:52.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e17fa640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f30c8065b80 0x7f30c8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.740+0000 7f30e17fa640 1 -- 172.21.15.26:0/3400875190 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f30dc083480 con 0x7f30e4159830 2023-12-19T18:18:52.743 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.744+0000 7f30e17fa640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] conn(0x7f30c806c760 0x7f30c806eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.744+0000 7f30e17fa640 1 -- 172.21.15.26:0/3400875190 --> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30c806f180 con 0x7f30c806c760 2023-12-19T18:18:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.744+0000 7f30e17fa640 1 -- 172.21.15.26:0/3400875190 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=259) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30dc085a60 con 0x7f30e4159830 2023-12-19T18:18:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.744+0000 7f30e88a6640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f30c8065b80 0x7f30c8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.744+0000 7f30e3fff640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] conn(0x7f30c806c760 0x7f30c806eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.744+0000 7f30e88a6640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f30c8065b80 0x7f30c8068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f30d80af000 tx=0x7f30d80a7000 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.744 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.744+0000 7f30e3fff640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] conn(0x7f30c806c760 0x7f30c806eb50 crc :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.745 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.744+0000 7f30e17fa640 1 -- 172.21.15.26:0/3400875190 <== osd.5 v2:172.21.15.160:6802/38419 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f30c806f180 con 0x7f30c806c760 2023-12-19T18:18:52.757 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.756+0000 7f30eab0a640 1 -- 172.21.15.26:0/3400875190 --> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f30e406fcb0 con 0x7f30c806c760 2023-12-19T18:18:52.758 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.756+0000 7f30e17fa640 1 -- 172.21.15.26:0/3400875190 <== osd.5 v2:172.21.15.160:6802/38419 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f30d0009d70 con 0x7f30c806c760 2023-12-19T18:18:52.758 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:18:52.758 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-19T18:18:52.759 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:18:52.759 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.756+0000 7f30eab0a640 1 -- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] conn(0x7f30c806c760 msgr2=0x7f30c806eb50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.756+0000 7f30eab0a640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:6802/38419,v1:172.21.15.160:6805/38419] conn(0x7f30c806c760 0x7f30c806eb50 crc :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.756+0000 7f30eab0a640 1 -- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f30c8065b80 msgr2=0x7f30c8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.756+0000 7f30eab0a640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f30c8065b80 0x7f30c8068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f30d80af000 tx=0x7f30d80a7000 comp rx=0 tx=0).stop 2023-12-19T18:18:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.756+0000 7f30eab0a640 1 -- 172.21.15.26:0/3400875190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4159830 msgr2=0x7f30e40c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.756+0000 7f30eab0a640 1 --2- 172.21.15.26:0/3400875190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e4159830 0x7f30e40c4ab0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00c7c0 tx=0x7f30dc00cc80 comp rx=0 tx=0).stop 2023-12-19T18:18:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.756+0000 7f30e90a7640 1 -- 172.21.15.26:0/3400875190 reap_dead start 2023-12-19T18:18:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.756+0000 7f30eab0a640 1 -- 172.21.15.26:0/3400875190 shutdown_connections 2023-12-19T18:18:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.756+0000 7f30eab0a640 1 -- 172.21.15.26:0/3400875190 >> 172.21.15.26:0/3400875190 conn(0x7f30e40b0270 msgr2=0x7f30e4158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.756+0000 7f30eab0a640 1 -- 172.21.15.26:0/3400875190 shutdown_connections 2023-12-19T18:18:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.756+0000 7f30eab0a640 1 -- 172.21.15.26:0/3400875190 wait complete. 2023-12-19T18:18:52.777 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-12-19T18:18:52.921 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.920+0000 7f994d466640 1 Processor -- start 2023-12-19T18:18:52.922 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.920+0000 7f994d466640 1 -- start start 2023-12-19T18:18:52.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.920+0000 7f994d466640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9948153d40 0x7f99480702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.923 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.920+0000 7f994d466640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9948154120 0x7f99480b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.920+0000 7f994d466640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99480b02c0 0x7f99480b0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.920+0000 7f994d466640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99480ae150 con 0x7f9948154120 2023-12-19T18:18:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.920+0000 7f994d466640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99480b0c90 con 0x7f9948153d40 2023-12-19T18:18:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.920+0000 7f994d466640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994815d7a0 con 0x7f99480b02c0 2023-12-19T18:18:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f993ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9948154120 0x7f99480b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f993ffff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9948154120 0x7f99480b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53946/0 (socket says 172.21.15.26:53946) 2023-12-19T18:18:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f993ffff640 1 -- 172.21.15.26:0/3354010335 learned_addr learned my addr 172.21.15.26:0/3354010335 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f99477fe640 1 --2- 172.21.15.26:0/3354010335 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99480b02c0 0x7f99480b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f9946ffd640 1 --2- 172.21.15.26:0/3354010335 >> v2:172.21.15.160:3300/0 conn(0x7f9948153d40 0x7f99480702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f993ffff640 1 -- 172.21.15.26:0/3354010335 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99480b02c0 msgr2=0x7f99480b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f993ffff640 1 --2- 172.21.15.26:0/3354010335 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99480b02c0 0x7f99480b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.924 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f993ffff640 1 -- 172.21.15.26:0/3354010335 >> v2:172.21.15.160:3300/0 conn(0x7f9948153d40 msgr2=0x7f99480702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f993ffff640 1 --2- 172.21.15.26:0/3354010335 >> v2:172.21.15.160:3300/0 conn(0x7f9948153d40 0x7f99480702e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f993ffff640 1 -- 172.21.15.26:0/3354010335 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994815da80 con 0x7f9948154120 2023-12-19T18:18:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f9946ffd640 1 --2- 172.21.15.26:0/3354010335 >> v2:172.21.15.160:3300/0 conn(0x7f9948153d40 0x7f99480702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f993ffff640 1 --2- 172.21.15.26:0/3354010335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9948154120 0x7f99480b3370 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f9948066410 tx=0x7f993001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9c5d8a3f14bdc3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.926 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f99467fc640 1 -- 172.21.15.26:0/3354010335 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99300ac600 con 0x7f9948154120 2023-12-19T18:18:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f99467fc640 1 -- 172.21.15.26:0/3354010335 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99300a8050 con 0x7f9948154120 2023-12-19T18:18:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.924+0000 7f99467fc640 1 -- 172.21.15.26:0/3354010335 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99300ac980 con 0x7f9948154120 2023-12-19T18:18:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 -- 172.21.15.26:0/3354010335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9948154120 msgr2=0x7f99480b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 --2- 172.21.15.26:0/3354010335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9948154120 0x7f99480b3370 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f9948066410 tx=0x7f993001c3e0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 -- 172.21.15.26:0/3354010335 shutdown_connections 2023-12-19T18:18:52.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 --2- 172.21.15.26:0/3354010335 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99480b02c0 0x7f99480b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 --2- 172.21.15.26:0/3354010335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9948154120 0x7f99480b3370 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 --2- 172.21.15.26:0/3354010335 >> v2:172.21.15.160:3300/0 conn(0x7f9948153d40 0x7f99480702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 -- 172.21.15.26:0/3354010335 >> 172.21.15.26:0/3354010335 conn(0x7f9948073360 msgr2=0x7f99480658e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:52.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 -- 172.21.15.26:0/3354010335 shutdown_connections 2023-12-19T18:18:52.928 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 -- 172.21.15.26:0/3354010335 wait complete. 2023-12-19T18:18:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 Processor -- start 2023-12-19T18:18:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 -- start start 2023-12-19T18:18:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99480b02c0 0x7f99480cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9948153d40 0x7f994813cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9948154120 0x7f99480cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99480cc2e0 con 0x7f9948153d40 2023-12-19T18:18:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99480cc5c0 con 0x7f9948154120 2023-12-19T18:18:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f994d466640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99480cc8a0 con 0x7f99480b02c0 2023-12-19T18:18:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f993ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9948153d40 0x7f994813cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f993ffff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9948153d40 0x7f994813cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53962/0 (socket says 172.21.15.26:53962) 2023-12-19T18:18:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f993ffff640 1 -- 172.21.15.26:0/621727446 learned_addr learned my addr 172.21.15.26:0/621727446 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f99477fe640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9948154120 0x7f99480cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f9946ffd640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99480b02c0 0x7f99480cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f993ffff640 1 -- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99480b02c0 msgr2=0x7f99480cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f993ffff640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99480b02c0 0x7f99480cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f993ffff640 1 -- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9948154120 msgr2=0x7f99480cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f993ffff640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9948154120 0x7f99480cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.928+0000 7f993ffff640 1 -- 172.21.15.26:0/621727446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99300094d0 con 0x7f9948153d40 2023-12-19T18:18:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.932+0000 7f993ffff640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9948153d40 0x7f994813cfb0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f993001cc80 tx=0x7f993001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.932+0000 7f993f7fe640 1 -- 172.21.15.26:0/621727446 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99300b7650 con 0x7f9948153d40 2023-12-19T18:18:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.932+0000 7f993f7fe640 1 -- 172.21.15.26:0/621727446 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99300b77d0 con 0x7f9948153d40 2023-12-19T18:18:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.932+0000 7f993f7fe640 1 -- 172.21.15.26:0/621727446 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99300b7b80 con 0x7f9948153d40 2023-12-19T18:18:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.932+0000 7f994d466640 1 -- 172.21.15.26:0/621727446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99480ccb80 con 0x7f9948153d40 2023-12-19T18:18:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.932+0000 7f994d466640 1 -- 172.21.15.26:0/621727446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994813d4f0 con 0x7f9948153d40 2023-12-19T18:18:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.932+0000 7f994d466640 1 -- 172.21.15.26:0/621727446 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994804e850 con 0x7f9948153d40 2023-12-19T18:18:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.932+0000 7f993f7fe640 1 -- 172.21.15.26:0/621727446 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f99300b3070 con 0x7f9948153d40 2023-12-19T18:18:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.932+0000 7f993f7fe640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9928065b80 0x7f9928068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.932+0000 7f993f7fe640 1 -- 172.21.15.26:0/621727446 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f993011d9b0 con 0x7f9948153d40 2023-12-19T18:18:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.932+0000 7f9946ffd640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9928065b80 0x7f9928068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:52.935 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.936+0000 7f9946ffd640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9928065b80 0x7f9928068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9938009830 tx=0x7f9938008040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:52.936+0000 7f993f7fe640 1 -- 172.21.15.26:0/621727446 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f99300eaf20 con 0x7f9948153d40 2023-12-19T18:18:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f994d466640 1 -- 172.21.15.26:0/621727446 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f994815d430 con 0x7f9928065b80 2023-12-19T18:18:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f993f7fe640 1 -- 172.21.15.26:0/621727446 <== mgr.11683 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9938009e10 con 0x7f9928065b80 2023-12-19T18:18:53.096 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 5 to deep-scrub 2023-12-19T18:18:53.098 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f994d466640 1 -- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9928065b80 msgr2=0x7f9928068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f994d466640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9928065b80 0x7f9928068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9938009830 tx=0x7f9938008040 comp rx=0 tx=0).stop 2023-12-19T18:18:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f994d466640 1 -- 172.21.15.26:0/621727446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9948153d40 msgr2=0x7f994813cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f994d466640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9948153d40 0x7f994813cfb0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f993001cc80 tx=0x7f993001ac20 comp rx=0 tx=0).stop 2023-12-19T18:18:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f994d466640 1 -- 172.21.15.26:0/621727446 shutdown_connections 2023-12-19T18:18:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f994d466640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9928065b80 0x7f9928068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f994d466640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9948154120 0x7f99480cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f994d466640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9948153d40 0x7f994813cfb0 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f994d466640 1 --2- 172.21.15.26:0/621727446 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f99480b02c0 0x7f99480cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f994d466640 1 -- 172.21.15.26:0/621727446 >> 172.21.15.26:0/621727446 conn(0x7f9948073360 msgr2=0x7f99480ae2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f994d466640 1 -- 172.21.15.26:0/621727446 shutdown_connections 2023-12-19T18:18:53.099 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.096+0000 7f994d466640 1 -- 172.21.15.26:0/621727446 wait complete. 2023-12-19T18:18:53.117 INFO:tasks.ceph:Scrubbing osd.6 2023-12-19T18:18:53.118 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:18:53.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.264+0000 7f4491de9640 1 Processor -- start 2023-12-19T18:18:53.265 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.264+0000 7f4491de9640 1 -- start start 2023-12-19T18:18:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.264+0000 7f4491de9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f448c070ff0 0x7f448c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.264+0000 7f4491de9640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f448c0c94a0 0x7f448c06f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.264+0000 7f4491de9640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448c06c4c0 con 0x7f448c1597d0 2023-12-19T18:18:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.264+0000 7f4491de9640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448c0713d0 con 0x7f448c0c94a0 2023-12-19T18:18:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.264+0000 7f4491de9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c159530 con 0x7f448c070ff0 2023-12-19T18:18:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.264+0000 7f448bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f448c0c94a0 0x7f448c06f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.264+0000 7f448affd640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f448c1597d0 0x7f448c0c8d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43020/0 (socket says 172.21.15.26:43020) 2023-12-19T18:18:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.264+0000 7f448bfff640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f448c0c94a0 0x7f448c06f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53972/0 (socket says 172.21.15.26:53972) 2023-12-19T18:18:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.264+0000 7f448affd640 1 -- 172.21.15.26:0/1417025266 learned_addr learned my addr 172.21.15.26:0/1417025266 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:53.267 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.264+0000 7f448b7fe640 1 --2- 172.21.15.26:0/1417025266 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f448c070ff0 0x7f448c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f448bfff640 1 -- 172.21.15.26:0/1417025266 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f448c070ff0 msgr2=0x7f448c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f448bfff640 1 --2- 172.21.15.26:0/1417025266 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f448c070ff0 0x7f448c06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f448bfff640 1 -- 172.21.15.26:0/1417025266 >> v1:172.21.15.26:6789/0 conn(0x7f448c1597d0 legacy=0x7f448c0c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f448bfff640 1 -- 172.21.15.26:0/1417025266 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c06b420 con 0x7f448c0c94a0 2023-12-19T18:18:53.268 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f448a7fc640 1 -- 172.21.15.26:0/1417025266 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435773037 0 0) 0x7f448c06c4c0 con 0x7f448c1597d0 2023-12-19T18:18:53.270 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f448bfff640 1 --2- 172.21.15.26:0/1417025266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f448c0c94a0 0x7f448c06f290 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f4484009d90 tx=0x7f448401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22476939737d3bef server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f448a7fc640 1 -- 172.21.15.26:0/1417025266 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44840a5960 con 0x7f448c0c94a0 2023-12-19T18:18:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f448a7fc640 1 -- 172.21.15.26:0/1417025266 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44840a5ac0 con 0x7f448c0c94a0 2023-12-19T18:18:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f448a7fc640 1 -- 172.21.15.26:0/1417025266 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44840a5e50 con 0x7f448c0c94a0 2023-12-19T18:18:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f4491de9640 1 -- 172.21.15.26:0/1417025266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f448c0c94a0 msgr2=0x7f448c06f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f4491de9640 1 --2- 172.21.15.26:0/1417025266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f448c0c94a0 0x7f448c06f290 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7f4484009d90 tx=0x7f448401c920 comp rx=0 tx=0).stop 2023-12-19T18:18:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f4491de9640 1 -- 172.21.15.26:0/1417025266 shutdown_connections 2023-12-19T18:18:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f4491de9640 1 --2- 172.21.15.26:0/1417025266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f448c0c94a0 0x7f448c06f290 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f4491de9640 1 --2- 172.21.15.26:0/1417025266 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f448c070ff0 0x7f448c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.268+0000 7f4491de9640 1 -- 172.21.15.26:0/1417025266 >> 172.21.15.26:0/1417025266 conn(0x7f448c0b0270 msgr2=0x7f448c0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f4491de9640 1 -- 172.21.15.26:0/1417025266 shutdown_connections 2023-12-19T18:18:53.271 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f4491de9640 1 -- 172.21.15.26:0/1417025266 wait complete. 2023-12-19T18:18:53.272 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f4491de9640 1 Processor -- start 2023-12-19T18:18:53.273 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f4491de9640 1 -- start start 2023-12-19T18:18:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f4491de9640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f448c070ff0 0x7f448c14f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.274 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f4491de9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f448c0c94a0 0x7f448c156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f4491de9640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f448c1597d0 0x7f448c0b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f4491de9640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c0b2bc0 con 0x7f448c1597d0 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f4491de9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448c0c6270 con 0x7f448c0c94a0 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f4491de9640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448c14f6d0 con 0x7f448c070ff0 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f448bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f448c1597d0 0x7f448c0b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f448bfff640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f448c1597d0 0x7f448c0b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53984/0 (socket says 172.21.15.26:53984) 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f448bfff640 1 -- 172.21.15.26:0/350516626 learned_addr learned my addr 172.21.15.26:0/350516626 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f448affd640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f448c0c94a0 0x7f448c156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f448b7fe640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f448c070ff0 0x7f448c14f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f448bfff640 1 -- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f448c070ff0 msgr2=0x7f448c14f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f448bfff640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f448c070ff0 0x7f448c14f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f448bfff640 1 -- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f448c0c94a0 msgr2=0x7f448c156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f448bfff640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f448c0c94a0 0x7f448c156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.275 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f448bfff640 1 -- 172.21.15.26:0/350516626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4484009a70 con 0x7f448c1597d0 2023-12-19T18:18:53.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.272+0000 7f448bfff640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f448c1597d0 0x7f448c0b2dc0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f448401a550 tx=0x7f44840196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:53.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f4473fff640 1 -- 172.21.15.26:0/350516626 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44840aee00 con 0x7f448c1597d0 2023-12-19T18:18:53.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f4473fff640 1 -- 172.21.15.26:0/350516626 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4484009450 con 0x7f448c1597d0 2023-12-19T18:18:53.276 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f4473fff640 1 -- 172.21.15.26:0/350516626 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484003ba0 con 0x7f448c1597d0 2023-12-19T18:18:53.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f4491de9640 1 -- 172.21.15.26:0/350516626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448c0c5940 con 0x7f448c1597d0 2023-12-19T18:18:53.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f4491de9640 1 -- 172.21.15.26:0/350516626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448c0c5f00 con 0x7f448c1597d0 2023-12-19T18:18:53.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f4491de9640 1 -- 172.21.15.26:0/350516626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f448c06bfa0 con 0x7f448c1597d0 2023-12-19T18:18:53.277 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f4473fff640 1 -- 172.21.15.26:0/350516626 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f44840b4050 con 0x7f448c1597d0 2023-12-19T18:18:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f4473fff640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4478065bb0 0x7f4478068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f4473fff640 1 -- 172.21.15.26:0/350516626 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f448411ea50 con 0x7f448c1597d0 2023-12-19T18:18:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f4473fff640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] conn(0x7f447806c790 0x7f447806eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f4473fff640 1 -- 172.21.15.26:0/350516626 --> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f447806f1b0 con 0x7f447806c790 2023-12-19T18:18:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f4473fff640 1 -- 172.21.15.26:0/350516626 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=259) v2 ==== 24+0+0 (secure 0 0 0) 0x7f448411edc0 con 0x7f448c1597d0 2023-12-19T18:18:53.278 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f448b7fe640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4478065bb0 0x7f4478068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.276+0000 7f448affd640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] conn(0x7f447806c790 0x7f447806eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.280+0000 7f448b7fe640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4478065bb0 0x7f4478068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f447c009b60 tx=0x7f447c009190 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:53.279 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.280+0000 7f448affd640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] conn(0x7f447806c790 0x7f447806eb80 crc :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:53.280 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.280+0000 7f4473fff640 1 -- 172.21.15.26:0/350516626 <== osd.6 v2:172.21.15.160:6824/38421 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f447806f1b0 con 0x7f447806c790 2023-12-19T18:18:53.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.292+0000 7f4491de9640 1 -- 172.21.15.26:0/350516626 --> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f448c06bcc0 con 0x7f447806c790 2023-12-19T18:18:53.293 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.292+0000 7f4473fff640 1 -- 172.21.15.26:0/350516626 <== osd.6 v2:172.21.15.160:6824/38421 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f44800014f0 con 0x7f447806c790 2023-12-19T18:18:53.293 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:18:53.294 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-19T18:18:53.294 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:18:53.294 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.292+0000 7f4491de9640 1 -- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] conn(0x7f447806c790 msgr2=0x7f447806eb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.292+0000 7f4491de9640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:6824/38421,v1:172.21.15.160:6825/38421] conn(0x7f447806c790 0x7f447806eb80 crc :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.292+0000 7f4491de9640 1 -- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4478065bb0 msgr2=0x7f4478068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.292+0000 7f4491de9640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f4478065bb0 0x7f4478068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f447c009b60 tx=0x7f447c009190 comp rx=0 tx=0).stop 2023-12-19T18:18:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.292+0000 7f4491de9640 1 -- 172.21.15.26:0/350516626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f448c1597d0 msgr2=0x7f448c0b2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.292+0000 7f4491de9640 1 --2- 172.21.15.26:0/350516626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f448c1597d0 0x7f448c0b2dc0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f448401a550 tx=0x7f44840196c0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.292+0000 7f448bfff640 1 -- 172.21.15.26:0/350516626 reap_dead start 2023-12-19T18:18:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.292+0000 7f4491de9640 1 -- 172.21.15.26:0/350516626 shutdown_connections 2023-12-19T18:18:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.292+0000 7f4491de9640 1 -- 172.21.15.26:0/350516626 >> 172.21.15.26:0/350516626 conn(0x7f448c0b0270 msgr2=0x7f448c1553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.292+0000 7f4491de9640 1 -- 172.21.15.26:0/350516626 shutdown_connections 2023-12-19T18:18:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.292+0000 7f4491de9640 1 -- 172.21.15.26:0/350516626 wait complete. 2023-12-19T18:18:53.312 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-12-19T18:18:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f912ba66640 1 Processor -- start 2023-12-19T18:18:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f912ba66640 1 -- start start 2023-12-19T18:18:53.457 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f912ba66640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91240b85e0 0x7f91240cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f912ba66640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f91240cde60 0x7f91240d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f912ba66640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91240d0d40 con 0x7f912415cda0 2023-12-19T18:18:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f912ba66640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91240ce2d0 con 0x7f91240b85e0 2023-12-19T18:18:53.458 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f912ba66640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9124161150 con 0x7f91240cde60 2023-12-19T18:18:53.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f9129802640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f912415cda0 0x7f9124160920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43036/0 (socket says 172.21.15.26:43036) 2023-12-19T18:18:53.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f9129001640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91240b85e0 0x7f91240cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f9129802640 1 -- 172.21.15.26:0/1476688521 learned_addr learned my addr 172.21.15.26:0/1476688521 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:18:53.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f912a003640 1 --2- 172.21.15.26:0/1476688521 >> v2:172.21.15.160:3300/0 conn(0x7f91240cde60 0x7f91240d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f9129001640 1 -- 172.21.15.26:0/1476688521 >> v2:172.21.15.160:3300/0 conn(0x7f91240cde60 msgr2=0x7f91240d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:53.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f9129001640 1 --2- 172.21.15.26:0/1476688521 >> v2:172.21.15.160:3300/0 conn(0x7f91240cde60 0x7f91240d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.459 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f9129001640 1 -- 172.21.15.26:0/1476688521 >> v1:172.21.15.26:6789/0 conn(0x7f912415cda0 legacy=0x7f9124160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f9128800640 1 -- 172.21.15.26:0/1476688521 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 477458386 0 0) 0x7f91240d0d40 con 0x7f912415cda0 2023-12-19T18:18:53.460 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.456+0000 7f9129001640 1 -- 172.21.15.26:0/1476688521 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9124161430 con 0x7f91240b85e0 2023-12-19T18:18:53.461 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.460+0000 7f9129001640 1 --2- 172.21.15.26:0/1476688521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91240b85e0 0x7f91240cd920 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f912408bc40 tx=0x7f911401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ab0a89d3d66544 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.460+0000 7f9128800640 1 -- 172.21.15.26:0/1476688521 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91140ab610 con 0x7f91240b85e0 2023-12-19T18:18:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.460+0000 7f9128800640 1 -- 172.21.15.26:0/1476688521 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91140ab770 con 0x7f91240b85e0 2023-12-19T18:18:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.460+0000 7f9128800640 1 -- 172.21.15.26:0/1476688521 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91140abb00 con 0x7f91240b85e0 2023-12-19T18:18:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.460+0000 7f912ba66640 1 -- 172.21.15.26:0/1476688521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91240b85e0 msgr2=0x7f91240cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.460+0000 7f912ba66640 1 --2- 172.21.15.26:0/1476688521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91240b85e0 0x7f91240cd920 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f912408bc40 tx=0x7f911401c410 comp rx=0 tx=0).stop 2023-12-19T18:18:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.460+0000 7f912ba66640 1 -- 172.21.15.26:0/1476688521 shutdown_connections 2023-12-19T18:18:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.460+0000 7f912ba66640 1 --2- 172.21.15.26:0/1476688521 >> v2:172.21.15.160:3300/0 conn(0x7f91240cde60 0x7f91240d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.460+0000 7f912ba66640 1 --2- 172.21.15.26:0/1476688521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91240b85e0 0x7f91240cd920 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.460+0000 7f912ba66640 1 -- 172.21.15.26:0/1476688521 >> 172.21.15.26:0/1476688521 conn(0x7f91240658e0 msgr2=0x7f9124065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.460+0000 7f912ba66640 1 -- 172.21.15.26:0/1476688521 shutdown_connections 2023-12-19T18:18:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.460+0000 7f912ba66640 1 -- 172.21.15.26:0/1476688521 wait complete. 2023-12-19T18:18:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f912ba66640 1 Processor -- start 2023-12-19T18:18:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f912ba66640 1 -- start start 2023-12-19T18:18:53.466 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f912ba66640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91240b85e0 0x7f912414e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f912ba66640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91240cde60 0x7f91240c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f912ba66640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f912415cda0 0x7f91240c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f912ba66640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91240c8520 con 0x7f91240b85e0 2023-12-19T18:18:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f912ba66640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91240c9ec0 con 0x7f91240cde60 2023-12-19T18:18:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f912ba66640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91240ca1a0 con 0x7f912415cda0 2023-12-19T18:18:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f9129802640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91240b85e0 0x7f912414e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f9129802640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91240b85e0 0x7f912414e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54000/0 (socket says 172.21.15.26:54000) 2023-12-19T18:18:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f9129802640 1 -- 172.21.15.26:0/1687702158 learned_addr learned my addr 172.21.15.26:0/1687702158 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f9129001640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91240cde60 0x7f91240c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f912a003640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f912415cda0 0x7f91240c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f9129802640 1 -- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f912415cda0 msgr2=0x7f91240c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.467 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f9129802640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f912415cda0 0x7f91240c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f9129802640 1 -- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91240cde60 msgr2=0x7f91240c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f9129802640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91240cde60 0x7f91240c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f9129802640 1 -- 172.21.15.26:0/1687702158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91140093c0 con 0x7f91240b85e0 2023-12-19T18:18:53.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f9129001640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91240cde60 0x7f91240c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:53.468 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.464+0000 7f9129802640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91240b85e0 0x7f912414e220 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f912000c480 tx=0x7f912000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.468+0000 7f911a7fc640 1 -- 172.21.15.26:0/1687702158 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91200074e0 con 0x7f91240b85e0 2023-12-19T18:18:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.468+0000 7f911a7fc640 1 -- 172.21.15.26:0/1687702158 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9120007660 con 0x7f91240b85e0 2023-12-19T18:18:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.468+0000 7f912ba66640 1 -- 172.21.15.26:0/1687702158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91240ca480 con 0x7f91240b85e0 2023-12-19T18:18:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.468+0000 7f911a7fc640 1 -- 172.21.15.26:0/1687702158 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120004a30 con 0x7f91240b85e0 2023-12-19T18:18:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.468+0000 7f912ba66640 1 -- 172.21.15.26:0/1687702158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912414e760 con 0x7f91240b85e0 2023-12-19T18:18:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.468+0000 7f912ba66640 1 -- 172.21.15.26:0/1687702158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9124074270 con 0x7f91240b85e0 2023-12-19T18:18:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.468+0000 7f911a7fc640 1 -- 172.21.15.26:0/1687702158 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9120004b90 con 0x7f91240b85e0 2023-12-19T18:18:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.468+0000 7f911a7fc640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9108065b80 0x7f9108068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.468+0000 7f911a7fc640 1 -- 172.21.15.26:0/1687702158 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f91200829b0 con 0x7f91240b85e0 2023-12-19T18:18:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.468+0000 7f9129001640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9108065b80 0x7f9108068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.472+0000 7f9129001640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9108065b80 0x7f9108068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f91140af000 tx=0x7f91140a7000 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.472+0000 7f911a7fc640 1 -- 172.21.15.26:0/1687702158 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f91200500f0 con 0x7f91240b85e0 2023-12-19T18:18:53.629 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.628+0000 7f912ba66640 1 -- 172.21.15.26:0/1687702158 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f912415d180 con 0x7f9108065b80 2023-12-19T18:18:53.630 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.628+0000 7f911a7fc640 1 -- 172.21.15.26:0/1687702158 <== mgr.11683 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f91140abd10 con 0x7f9108065b80 2023-12-19T18:18:53.630 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 6 to deep-scrub 2023-12-19T18:18:53.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.632+0000 7f912ba66640 1 -- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9108065b80 msgr2=0x7f9108068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.632 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.632+0000 7f912ba66640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9108065b80 0x7f9108068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f91140af000 tx=0x7f91140a7000 comp rx=0 tx=0).stop 2023-12-19T18:18:53.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.632+0000 7f912ba66640 1 -- 172.21.15.26:0/1687702158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91240b85e0 msgr2=0x7f912414e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.632+0000 7f912ba66640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91240b85e0 0x7f912414e220 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f912000c480 tx=0x7f912000c940 comp rx=0 tx=0).stop 2023-12-19T18:18:53.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.632+0000 7f912ba66640 1 -- 172.21.15.26:0/1687702158 shutdown_connections 2023-12-19T18:18:53.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.632+0000 7f912ba66640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f9108065b80 0x7f9108068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.632+0000 7f912ba66640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f912415cda0 0x7f91240c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.632+0000 7f912ba66640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91240cde60 0x7f91240c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.632+0000 7f912ba66640 1 --2- 172.21.15.26:0/1687702158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91240b85e0 0x7f912414e220 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.632+0000 7f912ba66640 1 -- 172.21.15.26:0/1687702158 >> 172.21.15.26:0/1687702158 conn(0x7f91240658e0 msgr2=0x7f91240ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:53.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.632+0000 7f912ba66640 1 -- 172.21.15.26:0/1687702158 shutdown_connections 2023-12-19T18:18:53.633 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.632+0000 7f912ba66640 1 -- 172.21.15.26:0/1687702158 wait complete. 2023-12-19T18:18:53.652 INFO:tasks.ceph:Scrubbing osd.7 2023-12-19T18:18:53.652 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:18:53.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ed1641640 1 Processor -- start 2023-12-19T18:18:53.800 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ed1641640 1 -- start start 2023-12-19T18:18:53.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ed1641640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ecc15cec0 0x7f2ecc0cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ed1641640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2ecc161560 0x7f2ecc0d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ed1641640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ecc0d0610 con 0x7f2ecc06f230 2023-12-19T18:18:53.801 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ed1641640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ecc15d2a0 con 0x7f2ecc161560 2023-12-19T18:18:53.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ed1641640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc06f9d0 con 0x7f2ecc15cec0 2023-12-19T18:18:53.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ecaffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ecc15cec0 0x7f2ecc0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ecaffd640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ecc15cec0 0x7f2ecc0cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3301/0 says I am v2:172.21.15.26:44440/0 (socket says 172.21.15.26:44440) 2023-12-19T18:18:53.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ecb7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2ecc161560 0x7f2ecc0d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ecaffd640 1 -- 172.21.15.26:0/1240839123 learned_addr learned my addr 172.21.15.26:0/1240839123 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:53.802 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ecb7fe640 1 -- 172.21.15.26:0/1240839123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ecc15cec0 msgr2=0x7f2ecc0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:53.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ecb7fe640 1 --2- 172.21.15.26:0/1240839123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ecc15cec0 0x7f2ecc0cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ecb7fe640 1 -- 172.21.15.26:0/1240839123 >> v1:172.21.15.160:6789/0 conn(0x7f2ecc06f230 legacy=0x7f2ecc0b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ecb7fe640 1 -- 172.21.15.26:0/1240839123 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc06f6f0 con 0x7f2ecc161560 2023-12-19T18:18:53.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2eca7fc640 1 -- 172.21.15.26:0/1240839123 <== mon.1 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853389729 0 0) 0x7f2ecc0d0610 con 0x7f2ecc06f230 2023-12-19T18:18:53.803 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.800+0000 7f2ecaffd640 1 --2- 172.21.15.26:0/1240839123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ecc15cec0 0x7f2ecc0cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:53.804 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.804+0000 7f2ecb7fe640 1 --2- 172.21.15.26:0/1240839123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ecc161560 0x7f2ecc0d00d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009d90 tx=0x7f2ec401c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b35ca475c0f2b148 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:53.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.804+0000 7f2eca7fc640 1 -- 172.21.15.26:0/1240839123 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40a5980 con 0x7f2ecc161560 2023-12-19T18:18:53.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.804+0000 7f2eca7fc640 1 -- 172.21.15.26:0/1240839123 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec40a5ae0 con 0x7f2ecc161560 2023-12-19T18:18:53.805 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.804+0000 7f2eca7fc640 1 -- 172.21.15.26:0/1240839123 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40a5e70 con 0x7f2ecc161560 2023-12-19T18:18:53.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.804+0000 7f2ed1641640 1 -- 172.21.15.26:0/1240839123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ecc161560 msgr2=0x7f2ecc0d00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.804+0000 7f2ed1641640 1 --2- 172.21.15.26:0/1240839123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ecc161560 0x7f2ecc0d00d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009d90 tx=0x7f2ec401c8c0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.804+0000 7f2ed1641640 1 -- 172.21.15.26:0/1240839123 shutdown_connections 2023-12-19T18:18:53.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.804+0000 7f2ed1641640 1 --2- 172.21.15.26:0/1240839123 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ecc161560 0x7f2ecc0d00d0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.804+0000 7f2ed1641640 1 --2- 172.21.15.26:0/1240839123 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ecc15cec0 0x7f2ecc0cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.804+0000 7f2ed1641640 1 -- 172.21.15.26:0/1240839123 >> 172.21.15.26:0/1240839123 conn(0x7f2ecc071460 msgr2=0x7f2ecc071870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:53.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.804+0000 7f2ed1641640 1 -- 172.21.15.26:0/1240839123 shutdown_connections 2023-12-19T18:18:53.806 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.804+0000 7f2ed1641640 1 -- 172.21.15.26:0/1240839123 wait complete. 2023-12-19T18:18:53.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ed1641640 1 Processor -- start 2023-12-19T18:18:53.808 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ed1641640 1 -- start start 2023-12-19T18:18:53.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ed1641640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ecc06f230 0x7f2ecc0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ed1641640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ecc15cec0 0x7f2ecc15edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.809 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ed1641640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ecc161560 0x7f2ecc0b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ed1641640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc15ddf0 con 0x7f2ecc161560 2023-12-19T18:18:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ed1641640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc15f300 con 0x7f2ecc06f230 2023-12-19T18:18:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ed1641640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc15e000 con 0x7f2ecc15cec0 2023-12-19T18:18:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ecb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ecc161560 0x7f2ecc0b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ecb7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ecc161560 0x7f2ecc0b5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54010/0 (socket says 172.21.15.26:54010) 2023-12-19T18:18:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ecb7fe640 1 -- 172.21.15.26:0/1654774117 learned_addr learned my addr 172.21.15.26:0/1654774117 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ec3fff640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ecc15cec0 0x7f2ecc15edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ecaffd640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ecc06f230 0x7f2ecc0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ecb7fe640 1 -- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ecc15cec0 msgr2=0x7f2ecc15edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ecb7fe640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ecc15cec0 0x7f2ecc15edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ecb7fe640 1 -- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ecc06f230 msgr2=0x7f2ecc0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ecb7fe640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ecc06f230 0x7f2ecc0b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.810 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ecb7fe640 1 -- 172.21.15.26:0/1654774117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0094d0 con 0x7f2ecc161560 2023-12-19T18:18:53.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ec3fff640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f2ecc15cec0 0x7f2ecc15edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:53.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ecaffd640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ecc06f230 0x7f2ecc0b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:53.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ecb7fe640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ecc161560 0x7f2ecc0b5470 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009d90 tx=0x7f2ec4019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:53.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ec37fe640 1 -- 172.21.15.26:0/1654774117 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40b9860 con 0x7f2ecc161560 2023-12-19T18:18:53.811 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ec37fe640 1 -- 172.21.15.26:0/1654774117 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec40a5970 con 0x7f2ecc161560 2023-12-19T18:18:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ed1641640 1 -- 172.21.15.26:0/1654774117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec4009a70 con 0x7f2ecc161560 2023-12-19T18:18:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ec37fe640 1 -- 172.21.15.26:0/1654774117 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40b9bc0 con 0x7f2ecc161560 2023-12-19T18:18:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ed1641640 1 -- 172.21.15.26:0/1654774117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc15e5c0 con 0x7f2ecc161560 2023-12-19T18:18:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.808+0000 7f2ed1641640 1 -- 172.21.15.26:0/1654774117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ecc161040 con 0x7f2ecc161560 2023-12-19T18:18:53.812 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.812+0000 7f2ec37fe640 1 -- 172.21.15.26:0/1654774117 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2ec40b9d20 con 0x7f2ecc161560 2023-12-19T18:18:53.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.812+0000 7f2ec37fe640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2eac065bb0 0x7f2eac068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.812+0000 7f2ec37fe640 1 -- 172.21.15.26:0/1654774117 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f2ec411ec50 con 0x7f2ecc161560 2023-12-19T18:18:53.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.812+0000 7f2ec37fe640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] conn(0x7f2eac06c790 0x7f2eac06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.812+0000 7f2ec37fe640 1 -- 172.21.15.26:0/1654774117 --> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2eac06f1b0 con 0x7f2eac06c790 2023-12-19T18:18:53.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.812+0000 7f2ec37fe640 1 -- 172.21.15.26:0/1654774117 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=259) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ec411efc0 con 0x7f2ecc161560 2023-12-19T18:18:53.813 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.812+0000 7f2ecaffd640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2eac065bb0 0x7f2eac068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.814 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.812+0000 7f2ec3fff640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] conn(0x7f2eac06c790 0x7f2eac06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.812+0000 7f2ecaffd640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2eac065bb0 0x7f2eac068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009d30 tx=0x7f2ebc008040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:53.817 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.812+0000 7f2ec3fff640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] conn(0x7f2eac06c790 0x7f2eac06eb80 crc :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:53.818 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.816+0000 7f2ec37fe640 1 -- 172.21.15.26:0/1654774117 <== osd.7 v2:172.21.15.160:6803/38420 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2eac06f1b0 con 0x7f2eac06c790 2023-12-19T18:18:53.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.828+0000 7f2ed1641640 1 -- 172.21.15.26:0/1654774117 --> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2ecc06bf30 con 0x7f2eac06c790 2023-12-19T18:18:53.828 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.828+0000 7f2ec37fe640 1 -- 172.21.15.26:0/1654774117 <== osd.7 v2:172.21.15.160:6803/38420 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2ecc06bf30 con 0x7f2eac06c790 2023-12-19T18:18:53.829 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2023-12-19T18:18:53.829 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-19T18:18:53.829 INFO:teuthology.orchestra.run.smithi026.stdout:} 2023-12-19T18:18:53.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.828+0000 7f2ed1641640 1 -- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] conn(0x7f2eac06c790 msgr2=0x7f2eac06eb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.828+0000 7f2ed1641640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:6803/38420,v1:172.21.15.160:6806/38420] conn(0x7f2eac06c790 0x7f2eac06eb80 crc :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.828+0000 7f2ed1641640 1 -- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2eac065bb0 msgr2=0x7f2eac068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.828+0000 7f2ed1641640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f2eac065bb0 0x7f2eac068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009d30 tx=0x7f2ebc008040 comp rx=0 tx=0).stop 2023-12-19T18:18:53.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.828+0000 7f2ed1641640 1 -- 172.21.15.26:0/1654774117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ecc161560 msgr2=0x7f2ecc0b5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.828+0000 7f2ed1641640 1 --2- 172.21.15.26:0/1654774117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ecc161560 0x7f2ecc0b5470 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009d90 tx=0x7f2ec4019660 comp rx=0 tx=0).stop 2023-12-19T18:18:53.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.828+0000 7f2ecb7fe640 1 -- 172.21.15.26:0/1654774117 reap_dead start 2023-12-19T18:18:53.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.828+0000 7f2ed1641640 1 -- 172.21.15.26:0/1654774117 shutdown_connections 2023-12-19T18:18:53.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.828+0000 7f2ed1641640 1 -- 172.21.15.26:0/1654774117 >> 172.21.15.26:0/1654774117 conn(0x7f2ecc071460 msgr2=0x7f2ecc0b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:53.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.828+0000 7f2ed1641640 1 -- 172.21.15.26:0/1654774117 shutdown_connections 2023-12-19T18:18:53.830 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.828+0000 7f2ed1641640 1 -- 172.21.15.26:0/1654774117 wait complete. 2023-12-19T18:18:53.848 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-12-19T18:18:53.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea852c4640 1 Processor -- start 2023-12-19T18:18:53.992 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea852c4640 1 -- start start 2023-12-19T18:18:53.993 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea852c4640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea8015cc60 0x7fea800cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea852c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea800b5a80 0x7fea800b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea852c4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea8015d040 con 0x7fea800d2fb0 2023-12-19T18:18:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea852c4640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea800d05c0 con 0x7fea8015cc60 2023-12-19T18:18:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea852c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea800b8270 con 0x7fea800b5a80 2023-12-19T18:18:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea7effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea8015cc60 0x7fea800cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea7effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea8015cc60 0x7fea800cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54026/0 (socket says 172.21.15.26:54026) 2023-12-19T18:18:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea7effd640 1 -- 172.21.15.26:0/3908279145 learned_addr learned my addr 172.21.15.26:0/3908279145 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea767fc640 1 --2- 172.21.15.26:0/3908279145 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea800b5a80 0x7fea800b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea7effd640 1 -- 172.21.15.26:0/3908279145 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea800b5a80 msgr2=0x7fea800b5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea7effd640 1 --2- 172.21.15.26:0/3908279145 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea800b5a80 0x7fea800b5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea7effd640 1 -- 172.21.15.26:0/3908279145 >> v1:172.21.15.160:6789/0 conn(0x7fea800d2fb0 legacy=0x7fea800d0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.994 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.992+0000 7fea7effd640 1 -- 172.21.15.26:0/3908279145 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea800b8550 con 0x7fea8015cc60 2023-12-19T18:18:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.996+0000 7fea7effd640 1 --2- 172.21.15.26:0/3908279145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea8015cc60 0x7fea800cda50 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fea70009e50 tx=0x7fea7001f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76cfdffc37b6b69b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:53.996 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.996+0000 7fea7e7fc640 1 -- 172.21.15.26:0/3908279145 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea700af400 con 0x7fea8015cc60 2023-12-19T18:18:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.996+0000 7fea7e7fc640 1 -- 172.21.15.26:0/3908279145 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea700ab050 con 0x7fea8015cc60 2023-12-19T18:18:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.996+0000 7fea7e7fc640 1 -- 172.21.15.26:0/3908279145 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea700af780 con 0x7fea8015cc60 2023-12-19T18:18:53.997 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.996+0000 7fea852c4640 1 -- 172.21.15.26:0/3908279145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea8015cc60 msgr2=0x7fea800cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.996+0000 7fea852c4640 1 --2- 172.21.15.26:0/3908279145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea8015cc60 0x7fea800cda50 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fea70009e50 tx=0x7fea7001f1f0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.996+0000 7fea852c4640 1 -- 172.21.15.26:0/3908279145 shutdown_connections 2023-12-19T18:18:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.996+0000 7fea852c4640 1 --2- 172.21.15.26:0/3908279145 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea800b5a80 0x7fea800b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.996+0000 7fea852c4640 1 --2- 172.21.15.26:0/3908279145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea8015cc60 0x7fea800cda50 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.996+0000 7fea852c4640 1 -- 172.21.15.26:0/3908279145 >> 172.21.15.26:0/3908279145 conn(0x7fea8006a120 msgr2=0x7fea8006a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.996+0000 7fea852c4640 1 -- 172.21.15.26:0/3908279145 shutdown_connections 2023-12-19T18:18:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.996+0000 7fea852c4640 1 -- 172.21.15.26:0/3908279145 wait complete. 2023-12-19T18:18:53.998 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:53.996+0000 7fea852c4640 1 Processor -- start 2023-12-19T18:18:53.999 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea852c4640 1 -- start start 2023-12-19T18:18:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea852c4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea800b5a80 0x7fea80152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea852c4640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea800d2fb0 0x7fea801f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea852c4640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea8015cc60 0x7fea801593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea852c4640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea800c6e80 con 0x7fea800d2fb0 2023-12-19T18:18:54.001 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea852c4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea801f5c90 con 0x7fea800b5a80 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea852c4640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea80153000 con 0x7fea8015cc60 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea767fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea800d2fb0 0x7fea801f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea767fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea800d2fb0 0x7fea801f6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54030/0 (socket says 172.21.15.26:54030) 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea767fc640 1 -- 172.21.15.26:0/1273391101 learned_addr learned my addr 172.21.15.26:0/1273391101 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea7effd640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea800b5a80 0x7fea80152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea7f7fe640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea8015cc60 0x7fea801593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea767fc640 1 -- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea8015cc60 msgr2=0x7fea801593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea767fc640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea8015cc60 0x7fea801593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea767fc640 1 -- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea800b5a80 msgr2=0x7fea80152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea767fc640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea800b5a80 0x7fea80152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea767fc640 1 -- 172.21.15.26:0/1273391101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea70009b30 con 0x7fea800d2fb0 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea7f7fe640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea8015cc60 0x7fea801593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea767fc640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea800d2fb0 0x7fea801f6120 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fea6800e960 tx=0x7fea6800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:54.002 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea77fff640 1 -- 172.21.15.26:0/1273391101 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68009690 con 0x7fea800d2fb0 2023-12-19T18:18:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea77fff640 1 -- 172.21.15.26:0/1273391101 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6800ca00 con 0x7fea800d2fb0 2023-12-19T18:18:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea77fff640 1 -- 172.21.15.26:0/1273391101 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6800cd70 con 0x7fea800d2fb0 2023-12-19T18:18:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea852c4640 1 -- 172.21.15.26:0/1273391101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea801f5960 con 0x7fea800d2fb0 2023-12-19T18:18:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea852c4640 1 -- 172.21.15.26:0/1273391101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea800c70d0 con 0x7fea800d2fb0 2023-12-19T18:18:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.000+0000 7fea852c4640 1 -- 172.21.15.26:0/1273391101 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea8004e850 con 0x7fea800d2fb0 2023-12-19T18:18:54.003 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.004+0000 7fea77fff640 1 -- 172.21.15.26:0/1273391101 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fea68013470 con 0x7fea800d2fb0 2023-12-19T18:18:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.004+0000 7fea77fff640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fea60065b80 0x7fea60068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:54.004 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.004+0000 7fea77fff640 1 -- 172.21.15.26:0/1273391101 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7fea68083ba0 con 0x7fea800d2fb0 2023-12-19T18:18:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.004+0000 7fea7effd640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fea60065b80 0x7fea60068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:54.005 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.004+0000 7fea7effd640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fea60065b80 0x7fea60068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea700b1000 tx=0x7fea700aa000 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:54.009 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.008+0000 7fea77fff640 1 -- 172.21.15.26:0/1273391101 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fea68051190 con 0x7fea800d2fb0 2023-12-19T18:18:54.169 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.168+0000 7fea852c4640 1 -- 172.21.15.26:0/1273391101 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fea800d07a0 con 0x7fea60065b80 2023-12-19T18:18:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.168+0000 7fea77fff640 1 -- 172.21.15.26:0/1273391101 <== mgr.11683 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fea700af400 con 0x7fea60065b80 2023-12-19T18:18:54.170 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 7 to deep-scrub 2023-12-19T18:18:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.172+0000 7fea852c4640 1 -- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fea60065b80 msgr2=0x7fea60068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.172+0000 7fea852c4640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fea60065b80 0x7fea60068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea700b1000 tx=0x7fea700aa000 comp rx=0 tx=0).stop 2023-12-19T18:18:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.172+0000 7fea852c4640 1 -- 172.21.15.26:0/1273391101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea800d2fb0 msgr2=0x7fea801f6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:54.172 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.172+0000 7fea852c4640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea800d2fb0 0x7fea801f6120 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fea6800e960 tx=0x7fea6800ee20 comp rx=0 tx=0).stop 2023-12-19T18:18:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.172+0000 7fea852c4640 1 -- 172.21.15.26:0/1273391101 shutdown_connections 2023-12-19T18:18:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.172+0000 7fea852c4640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fea60065b80 0x7fea60068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.172+0000 7fea852c4640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fea8015cc60 0x7fea801593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.172+0000 7fea852c4640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea800d2fb0 0x7fea801f6120 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.172+0000 7fea852c4640 1 --2- 172.21.15.26:0/1273391101 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fea800b5a80 0x7fea80152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.172+0000 7fea852c4640 1 -- 172.21.15.26:0/1273391101 >> 172.21.15.26:0/1273391101 conn(0x7fea8006a120 msgr2=0x7fea800b74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.172+0000 7fea852c4640 1 -- 172.21.15.26:0/1273391101 shutdown_connections 2023-12-19T18:18:54.173 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.172+0000 7fea852c4640 1 -- 172.21.15.26:0/1273391101 wait complete. 2023-12-19T18:18:54.192 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:18:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.332+0000 7f946f8fc640 1 Processor -- start 2023-12-19T18:18:54.334 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.332+0000 7f946f8fc640 1 -- start start 2023-12-19T18:18:54.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946f8fc640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f946806b3d0 0x7f94680ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:54.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946f8fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f94680cec40 0x7f946815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:54.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946f8fc640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946806bc30 con 0x7f9468154220 2023-12-19T18:18:54.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946f8fc640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94680cf0d0 con 0x7f946806b3d0 2023-12-19T18:18:54.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946f8fc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94680cc6e0 con 0x7f94680cec40 2023-12-19T18:18:54.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946ce97640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f946806b3d0 0x7f94680ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:54.336 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946ce97640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f946806b3d0 0x7f94680ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54046/0 (socket says 172.21.15.26:54046) 2023-12-19T18:18:54.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946ce97640 1 -- 172.21.15.26:0/179447309 learned_addr learned my addr 172.21.15.26:0/179447309 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:54.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946de99640 1 --2- 172.21.15.26:0/179447309 >> v2:172.21.15.160:3300/0 conn(0x7f94680cec40 0x7f946815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:54.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946ce97640 1 -- 172.21.15.26:0/179447309 >> v2:172.21.15.160:3300/0 conn(0x7f94680cec40 msgr2=0x7f946815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:54.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946ce97640 1 --2- 172.21.15.26:0/179447309 >> v2:172.21.15.160:3300/0 conn(0x7f94680cec40 0x7f946815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946ce97640 1 -- 172.21.15.26:0/179447309 >> v1:172.21.15.26:6789/0 conn(0x7f9468154220 legacy=0x7f946815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:54.337 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946ce97640 1 -- 172.21.15.26:0/179447309 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946806b950 con 0x7f946806b3d0 2023-12-19T18:18:54.338 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.336+0000 7f946de99640 1 --2- 172.21.15.26:0/179447309 >> v2:172.21.15.160:3300/0 conn(0x7f94680cec40 0x7f946815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:54.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f946ce97640 1 --2- 172.21.15.26:0/179447309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946806b3d0 0x7f94680ce700 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f9468157bd0 tx=0x7f945801e6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd426ae0f1e77b7c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:54.339 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f945ffff640 1 -- 172.21.15.26:0/179447309 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94580a86b0 con 0x7f946806b3d0 2023-12-19T18:18:54.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f945ffff640 1 -- 172.21.15.26:0/179447309 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94580a8810 con 0x7f946806b3d0 2023-12-19T18:18:54.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f945ffff640 1 -- 172.21.15.26:0/179447309 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94580a8ba0 con 0x7f946806b3d0 2023-12-19T18:18:54.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f946f8fc640 1 -- 172.21.15.26:0/179447309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946806b3d0 msgr2=0x7f94680ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:54.340 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f946f8fc640 1 --2- 172.21.15.26:0/179447309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946806b3d0 0x7f94680ce700 secure :-1 s=READY pgs=2482 cs=0 l=1 rev1=1 crypto rx=0x7f9468157bd0 tx=0x7f945801e6e0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f946f8fc640 1 -- 172.21.15.26:0/179447309 shutdown_connections 2023-12-19T18:18:54.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f946f8fc640 1 --2- 172.21.15.26:0/179447309 >> v2:172.21.15.160:3300/0 conn(0x7f94680cec40 0x7f946815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f946f8fc640 1 --2- 172.21.15.26:0/179447309 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f946806b3d0 0x7f94680ce700 unknown :-1 s=CLOSED pgs=2482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f946f8fc640 1 -- 172.21.15.26:0/179447309 >> 172.21.15.26:0/179447309 conn(0x7f94680658e0 msgr2=0x7f9468065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:54.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f946f8fc640 1 -- 172.21.15.26:0/179447309 shutdown_connections 2023-12-19T18:18:54.341 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f946f8fc640 1 -- 172.21.15.26:0/179447309 wait complete. 2023-12-19T18:18:54.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f946f8fc640 1 Processor -- start 2023-12-19T18:18:54.342 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.340+0000 7f946f8fc640 1 -- start start 2023-12-19T18:18:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946f8fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f946806b3d0 0x7f9468145820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:54.344 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946f8fc640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94680cec40 0x7f946814ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946f8fc640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9468154220 0x7f9468145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946f8fc640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946814e540 con 0x7f94680cec40 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946f8fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946814f190 con 0x7f946806b3d0 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946f8fc640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946814f660 con 0x7f9468154220 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946ce97640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94680cec40 0x7f946814ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946ce97640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94680cec40 0x7f946814ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54054/0 (socket says 172.21.15.26:54054) 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946ce97640 1 -- 172.21.15.26:0/478386436 learned_addr learned my addr 172.21.15.26:0/478386436 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946d698640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f946806b3d0 0x7f9468145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946de99640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9468154220 0x7f9468145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946ce97640 1 -- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9468154220 msgr2=0x7f9468145e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946ce97640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9468154220 0x7f9468145e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946ce97640 1 -- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f946806b3d0 msgr2=0x7f9468145820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946ce97640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f946806b3d0 0x7f9468145820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.345 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946ce97640 1 -- 172.21.15.26:0/478386436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94580094d0 con 0x7f94680cec40 2023-12-19T18:18:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946ce97640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94680cec40 0x7f946814ec50 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f9458024cd0 tx=0x7f945801cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f945dffb640 1 -- 172.21.15.26:0/478386436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94580ba430 con 0x7f94680cec40 2023-12-19T18:18:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f945dffb640 1 -- 172.21.15.26:0/478386436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94580ba590 con 0x7f94680cec40 2023-12-19T18:18:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f945dffb640 1 -- 172.21.15.26:0/478386436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94580ba940 con 0x7f94680cec40 2023-12-19T18:18:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946f8fc640 1 -- 172.21.15.26:0/478386436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946814f940 con 0x7f94680cec40 2023-12-19T18:18:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946f8fc640 1 -- 172.21.15.26:0/478386436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94680b2150 con 0x7f94680cec40 2023-12-19T18:18:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f946f8fc640 1 -- 172.21.15.26:0/478386436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9468074270 con 0x7f94680cec40 2023-12-19T18:18:54.346 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.344+0000 7f945dffb640 1 -- 172.21.15.26:0/478386436 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f94580baaa0 con 0x7f94680cec40 2023-12-19T18:18:54.347 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.348+0000 7f945dffb640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f944c065b80 0x7f944c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:54.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.348+0000 7f945dffb640 1 -- 172.21.15.26:0/478386436 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f94580ac070 con 0x7f94680cec40 2023-12-19T18:18:54.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.348+0000 7f946d698640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f944c065b80 0x7f944c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:54.348 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.348+0000 7f946d698640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f944c065b80 0x7f944c068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f94680b3400 tx=0x7f9464009040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:54.351 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.352+0000 7f945dffb640 1 -- 172.21.15.26:0/478386436 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f94580eb560 con 0x7f94680cec40 2023-12-19T18:18:54.504 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.504+0000 7f946f8fc640 1 -- 172.21.15.26:0/478386436 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94680ccef0 con 0x7f944c065b80 2023-12-19T18:18:54.507 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.508+0000 7f945dffb640 1 -- 172.21.15.26:0/478386436 <== mgr.11683 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50429 (secure 0 0 0) 0x7f9464001c80 con 0x7f944c065b80 2023-12-19T18:18:54.508 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:18:54.508 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:18:54.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.508+0000 7f946f8fc640 1 -- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f944c065b80 msgr2=0x7f944c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:54.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.508+0000 7f946f8fc640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f944c065b80 0x7f944c068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f94680b3400 tx=0x7f9464009040 comp rx=0 tx=0).stop 2023-12-19T18:18:54.509 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.508+0000 7f946f8fc640 1 -- 172.21.15.26:0/478386436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94680cec40 msgr2=0x7f946814ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:54.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.508+0000 7f946f8fc640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94680cec40 0x7f946814ec50 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f9458024cd0 tx=0x7f945801cf20 comp rx=0 tx=0).stop 2023-12-19T18:18:54.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.508+0000 7f946f8fc640 1 -- 172.21.15.26:0/478386436 shutdown_connections 2023-12-19T18:18:54.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.508+0000 7f946f8fc640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f944c065b80 0x7f944c068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.508+0000 7f946f8fc640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f9468154220 0x7f9468145e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.508+0000 7f946f8fc640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94680cec40 0x7f946814ec50 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.508+0000 7f946f8fc640 1 --2- 172.21.15.26:0/478386436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f946806b3d0 0x7f9468145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:54.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.508+0000 7f946f8fc640 1 -- 172.21.15.26:0/478386436 >> 172.21.15.26:0/478386436 conn(0x7f94680658e0 msgr2=0x7f946815c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:54.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.508+0000 7f946f8fc640 1 -- 172.21.15.26:0/478386436 shutdown_connections 2023-12-19T18:18:54.510 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:18:54.508+0000 7f946f8fc640 1 -- 172.21.15.26:0/478386436 wait complete. 2023-12-19T18:18:54.529 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-12-19T18:18:52.922564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":412,"ondisk_log_size":412,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":427624,"kb_used_data":26664,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754547096,"statfs":{"total":773094113280,"available":772656226304,"internally_reserved":0,"allocated":27303936,"data_stored":15195400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.501690"},"pg_stats":[{"pgid":"1.0","version":"256'412","reported_seq":1460,"reported_epoch":258,"state":"active+clean","last_fresh":"2023-12-19T18:18:43.601120+0000","last_change":"2023-12-19T16:42:17.242567+0000","last_active":"2023-12-19T18:18:43.601120+0000","last_peered":"2023-12-19T18:18:43.601120+0000","last_clean":"2023-12-19T18:18:43.601120+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:18:43.601120+0000","last_undegraded":"2023-12-19T18:18:43.601120+0000","last_fullsized":"2023-12-19T18:18:43.601120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T16:42:16.193274+0000","last_clean_scrub_stamp":"2023-12-19T16:42:16.193274+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:50.203744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":412,"ondisk_log_size":412,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510592,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48768,"kb_used_data":3768,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323072,"statfs":{"total":96636764160,"available":96586825728,"internally_reserved":0,"allocated":3858432,"data_stored":2342201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.70899999999999996},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84099999999999997,"5min":0.96399999999999997,"15min":1.1180000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68999999999999995,"15min":0.72399999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82599999999999996,"5min":1.022,"15min":1.0369999999999999},"last":0.72399999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.70499999999999996,"15min":0.73699999999999999},"min":{"1min":0.46600000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.0580000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69099999999999995,"15min":0.73499999999999999},"min":{"1min":0.53500000000000003,"5min":0.45700000000000002,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":0.92000000000000004,"15min":1.0660000000000001},"last":0.78500000000000003}]},{"osd":2,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.72599999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.86099999999999999,"5min":0.97899999999999998,"15min":1.0669999999999999},"last":1.081},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74299999999999999,"15min":0.77200000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.50800000000000001},"max":{"1min":0.91300000000000003,"5min":1.0109999999999999,"15min":1.236},"last":0.86599999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.749},"min":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96299999999999997,"5min":1.002,"15min":1.079},"last":1.21},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.755,"15min":0.77400000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.46700000000000003},"max":{"1min":0.88300000000000001,"5min":1.05,"15min":1.1870000000000001},"last":0.80800000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61099999999999999,"15min":0.59999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85999999999999999,"5min":1.0149999999999999,"15min":1.302},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":0.92200000000000004,"15min":1.0209999999999999},"last":1.2729999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65300000000000002,"15min":0.63700000000000001},"min":{"1min":0.46600000000000003,"5min":0.41899999999999998,"15min":0.38800000000000001},"max":{"1min":0.88200000000000001,"5min":1.097,"15min":1.1299999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68200000000000005,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.0800000000000001},"last":1.0389999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.57999999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.0620000000000001,"5min":1.083,"15min":1.1000000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.54300000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94599999999999995,"5min":0.97799999999999998,"15min":1.093},"last":1.321}]}]},{"osd":6,"up_from":15,"seq":64424510592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48188,"kb_used_data":3188,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323652,"statfs":{"total":96636764160,"available":96587419648,"internally_reserved":0,"allocated":3264512,"data_stored":1751833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90600000000000003,"5min":1.0289999999999999,"15min":1.042},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69299999999999995,"15min":0.72399999999999998},"min":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95999999999999996,"5min":1.224,"15min":1.224},"last":0.78600000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94799999999999995,"5min":1.103,"15min":1.103},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.746,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.69399999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.73699999999999999},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.879,"5min":0.98999999999999999,"15min":1.111},"last":0.65200000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.748,"15min":0.77400000000000002},"min":{"1min":0.57799999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94799999999999995,"5min":1.046,"15min":1.1519999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.90300000000000002,"5min":1.268,"15min":1.268},"last":0.61899999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.83599999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.97499999999999998,"5min":1.194,"15min":1.194},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.67700000000000005},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.86699999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.92200000000000004,"5min":1.218,"15min":1.218},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.60999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.1200000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.749}]}]},{"osd":5,"up_from":15,"seq":64424510592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52284,"kb_used_data":3188,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319556,"statfs":{"total":96636764160,"available":96583225344,"internally_reserved":0,"allocated":3264512,"data_stored":1751833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67400000000000004,"15min":0.71899999999999997},"min":{"1min":0.434,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":1.002,"15min":1.0149999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70099999999999996,"15min":0.73199999999999998},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.74199999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.72099999999999997,"15min":0.751},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68400000000000005,"15min":0.72399999999999998},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81999999999999995,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.72999999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73499999999999999,"15min":0.77000000000000002},"min":{"1min":0.51300000000000001,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":0.93500000000000005,"5min":1.0289999999999999,"15min":1.2250000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68200000000000005,"15min":0.72499999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77500000000000002,"5min":1.004,"15min":1.4279999999999999},"last":0.65500000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85899999999999999,"5min":1.0129999999999999,"15min":1.05},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.73399999999999999,"15min":0.76200000000000001},"min":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.78900000000000003,"5min":1.022,"15min":1.0329999999999999},"last":0.78000000000000003}]},{"osd":4,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.442,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.89100000000000001,"5min":0.90200000000000002,"15min":1.0800000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.94999999999999996,"5min":0.98499999999999999,"15min":1.208},"last":0.56599999999999995}]},{"osd":6,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89900000000000002,"5min":1.036,"15min":1.036},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.47299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96499999999999997,"5min":1,"15min":1.0449999999999999},"last":0.54400000000000004}]},{"osd":7,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.439,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.169},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0620000000000001,"5min":1.079,"15min":1.0900000000000001},"last":0.58799999999999997}]}]},{"osd":4,"up_from":15,"seq":64424510592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48188,"kb_used_data":3188,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323652,"statfs":{"total":96636764160,"available":96587419648,"internally_reserved":0,"allocated":3264512,"data_stored":1751833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.70799999999999996,"15min":0.73399999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.109},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88700000000000001,"5min":0.997,"15min":1.01},"last":0.70999999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.73199999999999998,"15min":0.748},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.0680000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.72899999999999998,"15min":0.752},"min":{"1min":0.47199999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.876,"5min":0.97899999999999998,"15min":1.1020000000000001},"last":0.84899999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.48099999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90100000000000002,"5min":1.0289999999999999,"15min":1.071},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.76900000000000002},"min":{"1min":0.48399999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.0760000000000001},"last":0.80200000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96299999999999997,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.75700000000000001,"15min":0.78300000000000003},"min":{"1min":0.57399999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98799999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.78000000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.63600000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84299999999999997,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.40899999999999997,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":0.80800000000000005,"5min":1.042,"15min":1.042},"last":0.625}]},{"osd":6,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46300000000000002,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.97299999999999998,"5min":1.024,"15min":1.024},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.74099999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.56000000000000005,"5min":0.496,"15min":0.41899999999999998},"max":{"1min":0.93899999999999995,"5min":1.087,"15min":1.087},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.77200000000000002,"15min":0.75900000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.038,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.68500000000000005}]}]},{"osd":3,"up_from":15,"seq":64424510592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52284,"kb_used_data":3188,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319556,"statfs":{"total":96636764160,"available":96583225344,"internally_reserved":0,"allocated":3264512,"data_stored":1751833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84999999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51300000000000001,"15min":0.52400000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95399999999999996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.495}]},{"osd":1,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47699999999999998,"15min":0.49099999999999999},"min":{"1min":0.308,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92900000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.379}]},{"osd":2,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":0.56599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79200000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56200000000000006,"15min":0.59099999999999997},"min":{"1min":0.38,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73299999999999998,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.51800000000000002}]},{"osd":4,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76500000000000001,"15min":0.80800000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88900000000000001,"5min":1.0609999999999999,"15min":1.4410000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.80900000000000005},"min":{"1min":0.53600000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89700000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.68300000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.77600000000000002,"15min":0.82399999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93600000000000005,"5min":1.1120000000000001,"15min":1.397},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78100000000000003,"15min":0.81799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96499999999999997,"5min":1.2230000000000001,"15min":1.262},"last":0.85499999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80300000000000005,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95599999999999996,"5min":1.1859999999999999,"15min":1.274},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.83099999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0549999999999999,"5min":1.165,"15min":1.302},"last":0.73499999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.84099999999999997,"15min":0.89200000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0529999999999999,"5min":1.2330000000000001,"15min":1.486},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83899999999999997,"15min":0.877},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97099999999999997,"5min":1.421,"15min":1.421},"last":0.84399999999999997}]}]},{"osd":2,"up_from":15,"seq":64424510591,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52284,"kb_used_data":3188,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319556,"statfs":{"total":96636764160,"available":96583225344,"internally_reserved":0,"allocated":3264512,"data_stored":1751833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42499999999999999,"15min":0.44700000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.622,"5min":0.754,"15min":0.754},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48099999999999998,"15min":0.5},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66000000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42099999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.57299999999999995,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.64800000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.499},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.36199999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.66200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.48999999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.441,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.86199999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.36899999999999999},"max":{"1min":0.68400000000000005,"5min":0.871,"15min":0.875},"last":0.44700000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.94599999999999995,"5min":1.0880000000000001,"15min":1.306},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76500000000000001,"15min":0.78300000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81799999999999995,"5min":1.2070000000000001,"15min":1.3580000000000001},"last":0.74199999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76800000000000002,"15min":0.79700000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":1.1080000000000001,"15min":1.2050000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89400000000000002,"5min":1.1519999999999999,"15min":1.429},"last":0.97899999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.83199999999999996},"min":{"1min":0.67500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.91700000000000004,"5min":1.216,"15min":1.365},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76900000000000002,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.117,"15min":1.2},"last":0.83599999999999997}]},{"osd":7,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77800000000000002,"15min":0.81399999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96299999999999997,"5min":1.159,"15min":1.45},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.85099999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0229999999999999,"5min":1.276,"15min":1.276},"last":0.80700000000000005}]}]},{"osd":1,"up_from":15,"seq":64424510592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60476,"kb_used_data":3188,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311364,"statfs":{"total":96636764160,"available":96574836736,"internally_reserved":0,"allocated":3264512,"data_stored":1751833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.432,"15min":0.45800000000000002},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.89400000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.497,"15min":0.52200000000000002},"min":{"1min":0.33200000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70199999999999996,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.26600000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.52800000000000002},"min":{"1min":0.39100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69299999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.54600000000000004,"15min":0.56399999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63700000000000001,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.374}]},{"osd":3,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.58999999999999997,"15min":0.61899999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81499999999999995,"5min":0.86699999999999999,"15min":0.94499999999999995},"last":0.44},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.52800000000000002,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84299999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.33400000000000002}]},{"osd":4,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.754,"15min":0.78600000000000003},"min":{"1min":0.50700000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95299999999999996,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.755,"15min":0.79900000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95399999999999996,"5min":1.2549999999999999,"15min":1.2569999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.752,"15min":0.79300000000000004},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":1.236,"15min":1.2609999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78900000000000003,"15min":0.81100000000000005},"min":{"1min":0.55600000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.93400000000000005,"5min":1.353,"15min":1.353},"last":0.63400000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77400000000000002,"15min":0.80400000000000005},"min":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.505},"max":{"1min":0.997,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.79800000000000004,"15min":0.83199999999999996},"min":{"1min":0.57499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.94499999999999995,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.56499999999999995},"max":{"1min":0.94099999999999995,"5min":1.399,"15min":1.399},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.80400000000000005,"15min":0.84399999999999997},"min":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0449999999999999,"5min":1.365,"15min":1.365},"last":0.70699999999999996}]}]},{"osd":0,"up_from":14,"seq":60129543297,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65152,"kb_used_data":3768,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94306688,"statfs":{"total":96636764160,"available":96570048512,"internally_reserved":0,"allocated":3858432,"data_stored":2342201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41599999999999998,"15min":0.44900000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.55100000000000005,"5min":0.69299999999999995,"15min":1.216},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.47599999999999998,"15min":0.51100000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.629,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.41699999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.48099999999999998,"15min":0.52100000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66500000000000004,"5min":0.77000000000000002,"15min":1.1699999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.54300000000000004},"min":{"1min":0.372,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69199999999999995,"5min":0.85699999999999998,"15min":0.96799999999999997},"last":0.58599999999999997}]},{"osd":3,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.50600000000000001,"15min":0.54100000000000004},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70099999999999996,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.58199999999999996,"15min":0.63200000000000001},"min":{"1min":0.441,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71699999999999997,"5min":0.89700000000000002,"15min":1.032},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76100000000000001,"15min":0.80000000000000004},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.496},"max":{"1min":1.052,"5min":1.1200000000000001,"15min":1.381},"last":0.875},{"interface":"front","average":{"1min":0.754,"5min":0.77100000000000002,"15min":0.80600000000000005},"min":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.47799999999999998},"max":{"1min":1.0860000000000001,"5min":1.137,"15min":1.2849999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76900000000000002,"15min":0.80900000000000005},"min":{"1min":0.58399999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.117,"5min":1.1180000000000001,"15min":1.3420000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78100000000000003,"15min":0.81000000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.996,"5min":1.1259999999999999,"15min":1.288},"last":0.92300000000000004}]},{"osd":6,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79600000000000004,"15min":0.82999999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.51300000000000001},"max":{"1min":1.1499999999999999,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0009999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":0.80700000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.169,"5min":1.1899999999999999,"15min":1.415},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.80200000000000005,"15min":0.84299999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.395},"last":0.90000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:18:54.533 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-12-19T16:42:16.193274+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=19, tm_hour=16, tm_min=42, tm_sec=16, tm_wday=1, tm_yday=353, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=19, tm_hour=18, tm_min=18, tm_sec=48, tm_wday=1, tm_yday=353, tm_isdst=0) 2023-12-19T18:18:54.533 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-12-19T18:19:12.807 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:19:12.811 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:19:14.534 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:19:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b15a8c640 1 Processor -- start 2023-12-19T18:19:14.688 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b15a8c640 1 -- start start 2023-12-19T18:19:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b15a8c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6b100ccc20 0x7f6b10161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b15a8c640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b100cd050 0x7f6b100cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:14.690 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b15a8c640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b10160890 con 0x7f6b100cfd70 2023-12-19T18:19:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b15a8c640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b100cdaf0 con 0x7f6b100cd050 2023-12-19T18:19:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b15a8c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b10160b40 con 0x7f6b100ccc20 2023-12-19T18:19:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b0effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b100cd050 0x7f6b100cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b0effd640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6b100cd050 0x7f6b100cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46286/0 (socket says 172.21.15.26:46286) 2023-12-19T18:19:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b0effd640 1 -- 172.21.15.26:0/2700680851 learned_addr learned my addr 172.21.15.26:0/2700680851 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:19:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b0ffff640 1 --1- 172.21.15.26:0/2700680851 >> v1:172.21.15.26:6789/0 conn(0x7f6b100cfd70 0x7f6b100d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49628/0 (socket says 172.21.15.26:49628) 2023-12-19T18:19:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b0f7fe640 1 --2- 172.21.15.26:0/2700680851 >> v2:172.21.15.160:3300/0 conn(0x7f6b100ccc20 0x7f6b10161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b0effd640 1 -- 172.21.15.26:0/2700680851 >> v2:172.21.15.160:3300/0 conn(0x7f6b100ccc20 msgr2=0x7f6b10161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b0effd640 1 --2- 172.21.15.26:0/2700680851 >> v2:172.21.15.160:3300/0 conn(0x7f6b100ccc20 0x7f6b10161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b0effd640 1 -- 172.21.15.26:0/2700680851 >> v1:172.21.15.26:6789/0 conn(0x7f6b100cfd70 legacy=0x7f6b100d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:14.691 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.687+0000 7f6b0effd640 1 -- 172.21.15.26:0/2700680851 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b10160e20 con 0x7f6b100cd050 2023-12-19T18:19:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.691+0000 7f6b0effd640 1 --2- 172.21.15.26:0/2700680851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100cd050 0x7f6b100cd490 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f6b1006c8b0 tx=0x7f6b0401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505376046d85c3cb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.691+0000 7f6b0e7fc640 1 -- 172.21.15.26:0/2700680851 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b040a7070 con 0x7f6b100cd050 2023-12-19T18:19:14.693 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.691+0000 7f6b0e7fc640 1 -- 172.21.15.26:0/2700680851 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b040ab430 con 0x7f6b100cd050 2023-12-19T18:19:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.691+0000 7f6b0e7fc640 1 -- 172.21.15.26:0/2700680851 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b040ab7c0 con 0x7f6b100cd050 2023-12-19T18:19:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.691+0000 7f6b15a8c640 1 -- 172.21.15.26:0/2700680851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100cd050 msgr2=0x7f6b100cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.691+0000 7f6b15a8c640 1 --2- 172.21.15.26:0/2700680851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100cd050 0x7f6b100cd490 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f6b1006c8b0 tx=0x7f6b0401c470 comp rx=0 tx=0).stop 2023-12-19T18:19:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.691+0000 7f6b15a8c640 1 -- 172.21.15.26:0/2700680851 shutdown_connections 2023-12-19T18:19:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.691+0000 7f6b15a8c640 1 --2- 172.21.15.26:0/2700680851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100cd050 0x7f6b100cd490 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:14.694 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.691+0000 7f6b15a8c640 1 --2- 172.21.15.26:0/2700680851 >> v2:172.21.15.160:3300/0 conn(0x7f6b100ccc20 0x7f6b10161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:14.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.691+0000 7f6b15a8c640 1 -- 172.21.15.26:0/2700680851 >> 172.21.15.26:0/2700680851 conn(0x7f6b1006d510 msgr2=0x7f6b1006d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:14.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.691+0000 7f6b15a8c640 1 -- 172.21.15.26:0/2700680851 shutdown_connections 2023-12-19T18:19:14.695 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.691+0000 7f6b15a8c640 1 -- 172.21.15.26:0/2700680851 wait complete. 2023-12-19T18:19:14.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b15a8c640 1 Processor -- start 2023-12-19T18:19:14.696 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b15a8c640 1 -- start start 2023-12-19T18:19:14.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b15a8c640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100ccc20 0x7f6b10157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:14.698 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b15a8c640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b100cd050 0x7f6b1015eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b15a8c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b100cfd70 0x7f6b1015f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b15a8c640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100ad780 con 0x7f6b100ccc20 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b15a8c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100aca00 con 0x7f6b100cfd70 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b15a8c640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b100aeee0 con 0x7f6b100cd050 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b0f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100ccc20 0x7f6b10157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b0f7fe640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100ccc20 0x7f6b10157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46292/0 (socket says 172.21.15.26:46292) 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b0f7fe640 1 -- 172.21.15.26:0/593938721 learned_addr learned my addr 172.21.15.26:0/593938721 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b0effd640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b100cd050 0x7f6b1015eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b0f7fe640 1 -- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b100cd050 msgr2=0x7f6b1015eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b0f7fe640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b100cd050 0x7f6b1015eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b0f7fe640 1 -- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b100cfd70 msgr2=0x7f6b1015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b0f7fe640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b100cfd70 0x7f6b1015f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b0f7fe640 1 -- 172.21.15.26:0/593938721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b040094d0 con 0x7f6b100ccc20 2023-12-19T18:19:14.699 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b0effd640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b100cd050 0x7f6b1015eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:14.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b0f7fe640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100ccc20 0x7f6b10157260 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000e930 tx=0x7f6b0000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:14.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6af7fff640 1 -- 172.21.15.26:0/593938721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b00009690 con 0x7f6b100ccc20 2023-12-19T18:19:14.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6af7fff640 1 -- 172.21.15.26:0/593938721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0000ca00 con 0x7f6b100ccc20 2023-12-19T18:19:14.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b15a8c640 1 -- 172.21.15.26:0/593938721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b100acca0 con 0x7f6b100ccc20 2023-12-19T18:19:14.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6af7fff640 1 -- 172.21.15.26:0/593938721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0000cd70 con 0x7f6b100ccc20 2023-12-19T18:19:14.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.695+0000 7f6b15a8c640 1 -- 172.21.15.26:0/593938721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b100ad260 con 0x7f6b100ccc20 2023-12-19T18:19:14.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.699+0000 7f6b15a8c640 1 -- 172.21.15.26:0/593938721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b10074270 con 0x7f6b100ccc20 2023-12-19T18:19:14.700 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.699+0000 7f6af7fff640 1 -- 172.21.15.26:0/593938721 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6b00013470 con 0x7f6b100ccc20 2023-12-19T18:19:14.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.699+0000 7f6af7fff640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6afc065b80 0x7f6afc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:14.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.699+0000 7f6af7fff640 1 -- 172.21.15.26:0/593938721 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f6b00083810 con 0x7f6b100ccc20 2023-12-19T18:19:14.701 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.699+0000 7f6b0effd640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6afc065b80 0x7f6afc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:14.702 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.699+0000 7f6b0effd640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6afc065b80 0x7f6afc068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6b040af000 tx=0x7f6b04008040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:14.705 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.703+0000 7f6af7fff640 1 -- 172.21.15.26:0/593938721 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6b00050ec0 con 0x7f6b100ccc20 2023-12-19T18:19:14.859 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.859+0000 7f6b15a8c640 1 -- 172.21.15.26:0/593938721 --> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b1006a530 con 0x7f6afc065b80 2023-12-19T18:19:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.859+0000 7f6af7fff640 1 -- 172.21.15.26:0/593938721 <== mgr.11683 v2:172.21.15.160:6832/48394 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50325 (secure 0 0 0) 0x7f6b040bb090 con 0x7f6afc065b80 2023-12-19T18:19:14.863 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:19:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2023-12-19T18:19:14.864 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.863+0000 7f6b15a8c640 1 -- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6afc065b80 msgr2=0x7f6afc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.863+0000 7f6b15a8c640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6afc065b80 0x7f6afc068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6b040af000 tx=0x7f6b04008040 comp rx=0 tx=0).stop 2023-12-19T18:19:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.863+0000 7f6b15a8c640 1 -- 172.21.15.26:0/593938721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100ccc20 msgr2=0x7f6b10157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.863+0000 7f6b15a8c640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100ccc20 0x7f6b10157260 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000e930 tx=0x7f6b0000edf0 comp rx=0 tx=0).stop 2023-12-19T18:19:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.863+0000 7f6b15a8c640 1 -- 172.21.15.26:0/593938721 shutdown_connections 2023-12-19T18:19:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.863+0000 7f6b15a8c640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f6afc065b80 0x7f6afc068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.863+0000 7f6b15a8c640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b100cfd70 0x7f6b1015f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.863+0000 7f6b15a8c640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f6b100cd050 0x7f6b1015eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.863+0000 7f6b15a8c640 1 --2- 172.21.15.26:0/593938721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b100ccc20 0x7f6b10157260 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.863+0000 7f6b15a8c640 1 -- 172.21.15.26:0/593938721 >> 172.21.15.26:0/593938721 conn(0x7f6b1006d510 msgr2=0x7f6b1006d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.863+0000 7f6b15a8c640 1 -- 172.21.15.26:0/593938721 shutdown_connections 2023-12-19T18:19:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:14.863+0000 7f6b15a8c640 1 -- 172.21.15.26:0/593938721 wait complete. 2023-12-19T18:19:14.883 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-19T18:19:12.926620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":413,"ondisk_log_size":413,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":427624,"kb_used_data":26664,"kb_used_omap":0,"kb_used_meta":400896,"kb_avail":754547096,"statfs":{"total":773094113280,"available":772656226304,"internally_reserved":0,"allocated":27303936,"data_stored":15195400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":410517504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002582"},"pg_stats":[{"pgid":"1.0","version":"259'413","reported_seq":1474,"reported_epoch":259,"state":"active+clean","last_fresh":"2023-12-19T18:18:54.978492+0000","last_change":"2023-12-19T18:18:54.978492+0000","last_active":"2023-12-19T18:18:54.978492+0000","last_peered":"2023-12-19T18:18:54.978492+0000","last_clean":"2023-12-19T18:18:54.978492+0000","last_became_active":"2023-12-19T16:42:17.241275+0000","last_became_peered":"2023-12-19T16:42:17.241275+0000","last_unstale":"2023-12-19T18:18:54.978492+0000","last_undegraded":"2023-12-19T18:18:54.978492+0000","last_fullsized":"2023-12-19T18:18:54.978492+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'413","last_scrub_stamp":"2023-12-19T18:18:54.978330+0000","last_deep_scrub":"259'413","last_deep_scrub_stamp":"2023-12-19T18:18:54.978330+0000","last_clean_scrub_stamp":"2023-12-19T18:18:54.978330+0000","objects_scrubbed":2,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:30.524983+0000","scrub_duration":0.0077241840000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2690,"num_read_kb":5647,"num_write":695,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":413,"ondisk_log_size":413,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424510596,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48768,"kb_used_data":3768,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323072,"statfs":{"total":96636764160,"available":96586825728,"internally_reserved":0,"allocated":3858432,"data_stored":2342201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67300000000000004,"15min":0.70899999999999996},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84099999999999997,"5min":0.96399999999999997,"15min":1.1180000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68999999999999995,"15min":0.72399999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82599999999999996,"5min":1.022,"15min":1.0369999999999999},"last":0.78500000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.70499999999999996,"15min":0.73699999999999999},"min":{"1min":0.46600000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.0580000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.69099999999999995,"15min":0.73499999999999999},"min":{"1min":0.53500000000000003,"5min":0.45700000000000002,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":0.92000000000000004,"15min":1.0660000000000001},"last":0.75600000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69099999999999995,"15min":0.72599999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.38900000000000001},"max":{"1min":0.86099999999999999,"5min":0.97899999999999998,"15min":1.0669999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.74299999999999999,"15min":0.77200000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.50800000000000001},"max":{"1min":0.91300000000000003,"5min":1.0109999999999999,"15min":1.236},"last":0.83499999999999996}]},{"osd":3,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.749},"min":{"1min":0.51400000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.96299999999999997,"5min":1.002,"15min":1.079},"last":0.748},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.755,"15min":0.77400000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.46700000000000003},"max":{"1min":0.88300000000000001,"5min":1.05,"15min":1.1870000000000001},"last":0.79900000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61099999999999999,"15min":0.59999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85999999999999999,"5min":1.0149999999999999,"15min":1.302},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63300000000000001},"min":{"1min":0.433,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":0.92200000000000004,"15min":1.0209999999999999},"last":0.58599999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.65300000000000002,"15min":0.63700000000000001},"min":{"1min":0.46600000000000003,"5min":0.41899999999999998,"15min":0.38800000000000001},"max":{"1min":0.88200000000000001,"5min":1.097,"15min":1.1299999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68200000000000005,"15min":0.66900000000000004},"min":{"1min":0.48699999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.0800000000000001},"last":0.71099999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:18:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.57999999999999996,"5min":0.443,"15min":0.443},"max":{"1min":1.0620000000000001,"5min":1.083,"15min":1.1000000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.70799999999999996,"15min":0.70699999999999996},"min":{"1min":0.54300000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94599999999999995,"5min":0.97799999999999998,"15min":1.093},"last":0.63800000000000001}]}]},{"osd":6,"up_from":15,"seq":64424510596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48188,"kb_used_data":3188,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323652,"statfs":{"total":96636764160,"available":96587419648,"internally_reserved":0,"allocated":3264512,"data_stored":1751833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.90600000000000003,"5min":1.0289999999999999,"15min":1.042},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69299999999999995,"15min":0.72399999999999998},"min":{"1min":0.48299999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95999999999999996,"5min":1.224,"15min":1.224},"last":0.78000000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67900000000000005,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94799999999999995,"5min":1.103,"15min":1.103},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.746,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.46800000000000003,"15min":0.434},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.83099999999999996}]},{"osd":2,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.748,"15min":0.75700000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":0.754},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.73699999999999999},"min":{"1min":0.51900000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.879,"5min":0.98999999999999999,"15min":1.111},"last":0.80500000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.748,"15min":0.77400000000000002},"min":{"1min":0.57799999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.94799999999999995,"5min":1.046,"15min":1.1519999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.69999999999999996,"15min":0.72199999999999998},"min":{"1min":0.48999999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"max":{"1min":0.90300000000000002,"5min":1.268,"15min":1.268},"last":0.70499999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.64100000000000001,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.63700000000000001,"15min":0.61899999999999999},"min":{"1min":0.46400000000000002,"5min":0.39500000000000002,"15min":0.375},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":0.45900000000000002}]},{"osd":5,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.63800000000000001,"15min":0.61599999999999999},"min":{"1min":0.47999999999999998,"5min":0.41999999999999998,"15min":0.374},"max":{"1min":0.97499999999999998,"5min":1.194,"15min":1.194},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70299999999999996,"15min":0.67700000000000005},"min":{"1min":0.48499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":2.48,"5min":2.48,"15min":2.48},"last":0.502}]},{"osd":7,"last update":"Tue Dec 19 18:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.73599999999999999,"15min":0.72299999999999998},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.46000000000000002},"max":{"1min":0.92200000000000004,"5min":1.218,"15min":1.218},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.748,"15min":0.73799999999999999},"min":{"1min":0.60999999999999999,"5min":0.377,"15min":0.377},"max":{"1min":1.1200000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.57899999999999996}]}]},{"osd":5,"up_from":15,"seq":64424510596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52284,"kb_used_data":3188,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319556,"statfs":{"total":96636764160,"available":96583225344,"internally_reserved":0,"allocated":3264512,"data_stored":1751833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67400000000000004,"15min":0.71899999999999997},"min":{"1min":0.434,"5min":0.39000000000000001,"15min":0.34399999999999997},"max":{"1min":0.82799999999999996,"5min":1.002,"15min":1.0149999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70099999999999996,"15min":0.73199999999999998},"min":{"1min":0.45100000000000001,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.75600000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.72099999999999997,"15min":0.751},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88900000000000001,"5min":1.0429999999999999,"15min":1.0489999999999999},"last":0.623},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.68400000000000005,"15min":0.72399999999999998},"min":{"1min":0.443,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81999999999999995,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.81000000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73499999999999999,"15min":0.77000000000000002},"min":{"1min":0.51300000000000001,"5min":0.5,"15min":0.47999999999999998},"max":{"1min":0.93500000000000005,"5min":1.0289999999999999,"15min":1.2250000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.68200000000000005,"15min":0.72499999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77500000000000002,"5min":1.004,"15min":1.4279999999999999},"last":0.73199999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.72499999999999998,"15min":0.75800000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.85899999999999999,"5min":1.0129999999999999,"15min":1.05},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.73399999999999999,"15min":0.76200000000000001},"min":{"1min":0.48999999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.78900000000000003,"5min":1.022,"15min":1.0329999999999999},"last":0.74399999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.442,"5min":0.36099999999999999,"15min":0.35899999999999999},"max":{"1min":0.89100000000000001,"5min":0.90200000000000002,"15min":1.0800000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.94999999999999996,"5min":0.98499999999999999,"15min":1.208},"last":0.83999999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.40600000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89900000000000002,"5min":1.036,"15min":1.036},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.47299999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.96499999999999997,"5min":1,"15min":1.0449999999999999},"last":0.68300000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:18:29 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.44900000000000001,"5min":0.439,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.169},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69899999999999995,"15min":0.70999999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0620000000000001,"5min":1.079,"15min":1.0900000000000001},"last":0.70099999999999996}]}]},{"osd":4,"up_from":15,"seq":64424510596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":48188,"kb_used_data":3188,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323652,"statfs":{"total":96636764160,"available":96587419648,"internally_reserved":0,"allocated":3264512,"data_stored":1751833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.70799999999999996,"15min":0.73399999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.109},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88700000000000001,"5min":0.997,"15min":1.01},"last":0.56599999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.73199999999999998,"15min":0.748},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.97999999999999998,"5min":1.0049999999999999,"15min":1.0680000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.72899999999999998,"15min":0.752},"min":{"1min":0.47199999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.876,"5min":0.97899999999999998,"15min":1.1020000000000001},"last":0.60899999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75800000000000001,"15min":0.76600000000000001},"min":{"1min":0.48099999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90100000000000002,"5min":1.0289999999999999,"15min":1.071},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.75800000000000001,"15min":0.76900000000000002},"min":{"1min":0.48399999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.0760000000000001},"last":0.66400000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72899999999999998,"15min":0.74299999999999999},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96299999999999997,"5min":0.96799999999999997,"15min":1.0409999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.75700000000000001,"15min":0.78300000000000003},"min":{"1min":0.57399999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.98799999999999999,"5min":1.0449999999999999,"15min":1.056},"last":0.624}]},{"osd":5,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65500000000000003,"15min":0.63600000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84299999999999997,"5min":1.0369999999999999,"15min":1.1020000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.40899999999999997,"5min":0.39700000000000002,"15min":0.39600000000000002},"max":{"1min":0.80800000000000005,"5min":1.042,"15min":1.042},"last":0.443}]},{"osd":6,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67900000000000005,"15min":0.66000000000000003},"min":{"1min":0.46300000000000002,"5min":0.41699999999999998,"15min":0.40200000000000002},"max":{"1min":0.97299999999999998,"5min":1.024,"15min":1.024},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73199999999999998,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.73799999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:18:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.71599999999999997},"min":{"1min":0.56000000000000005,"5min":0.496,"15min":0.41899999999999998},"max":{"1min":0.93899999999999995,"5min":1.087,"15min":1.087},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.77200000000000002,"15min":0.75900000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.038,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.72399999999999998}]}]},{"osd":3,"up_from":15,"seq":64424510596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52284,"kb_used_data":3188,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319556,"statfs":{"total":96636764160,"available":96583225344,"internally_reserved":0,"allocated":3264512,"data_stored":1751833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84999999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51300000000000001,"15min":0.52400000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.95399999999999996,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.70799999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.47699999999999998,"15min":0.49099999999999999},"min":{"1min":0.308,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68600000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.92900000000000005,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.66400000000000003}]},{"osd":2,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55400000000000005,"15min":0.56599999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79200000000000004,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.56200000000000006,"15min":0.59099999999999997},"min":{"1min":0.38,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73299999999999998,"5min":0.89100000000000001,"15min":0.91300000000000003},"last":0.80000000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76500000000000001,"15min":0.80800000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88900000000000001,"5min":1.0609999999999999,"15min":1.4410000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.77600000000000002,"15min":0.80900000000000005},"min":{"1min":0.53600000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.89700000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":1.204}]},{"osd":5,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.77600000000000002,"15min":0.82399999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.93600000000000005,"5min":1.1120000000000001,"15min":1.397},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.78100000000000003,"15min":0.81799999999999995},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.96499999999999997,"5min":1.2230000000000001,"15min":1.262},"last":1.179}]},{"osd":6,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80300000000000005,"15min":0.83699999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95599999999999996,"5min":1.1859999999999999,"15min":1.274},"last":1.131},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.83099999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0549999999999999,"5min":1.165,"15min":1.302},"last":1.159}]},{"osd":7,"last update":"Tue Dec 19 18:18:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.84099999999999997,"15min":0.89200000000000002},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0529999999999999,"5min":1.2330000000000001,"15min":1.486},"last":1.2809999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.83899999999999997,"15min":0.877},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97099999999999997,"5min":1.421,"15min":1.421},"last":1.2549999999999999}]}]},{"osd":2,"up_from":15,"seq":64424510595,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52284,"kb_used_data":3188,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319556,"statfs":{"total":96636764160,"available":96583225344,"internally_reserved":0,"allocated":3264512,"data_stored":1751833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.42499999999999999,"15min":0.44700000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.622,"5min":0.754,"15min":0.754},"last":0.505},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.48099999999999998,"15min":0.5},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.66000000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.60999999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.57299999999999995,"15min":0.60999999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.64800000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.50700000000000001,"15min":0.52200000000000002},"min":{"1min":0.36199999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.66200000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.79700000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56699999999999995,"15min":0.59899999999999998},"min":{"1min":0.441,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.86199999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54200000000000004,"15min":0.56699999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.36899999999999999},"max":{"1min":0.68400000000000005,"5min":0.871,"15min":0.875},"last":0.748}]},{"osd":4,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77300000000000002,"15min":0.79300000000000004},"min":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.94599999999999995,"5min":1.0880000000000001,"15min":1.306},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.76500000000000001,"15min":0.78300000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81799999999999995,"5min":1.2070000000000001,"15min":1.3580000000000001},"last":1.0900000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.76800000000000002,"15min":0.79700000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.441},"max":{"1min":0.93799999999999994,"5min":1.1080000000000001,"15min":1.2050000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.89400000000000002,"5min":1.1519999999999999,"15min":1.429},"last":0.95199999999999996}]},{"osd":6,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.83199999999999996},"min":{"1min":0.67500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.91700000000000004,"5min":1.216,"15min":1.365},"last":1.0449999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76900000000000002,"15min":0.79200000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0029999999999999,"5min":1.117,"15min":1.2},"last":0.97199999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:18:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77800000000000002,"15min":0.81399999999999995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96299999999999997,"5min":1.159,"15min":1.45},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82499999999999996,"15min":0.85099999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0229999999999999,"5min":1.276,"15min":1.276},"last":1.0089999999999999}]}]},{"osd":1,"up_from":15,"seq":64424510596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60476,"kb_used_data":3188,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311364,"statfs":{"total":96636764160,"available":96574836736,"internally_reserved":0,"allocated":3264512,"data_stored":1751833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.432,"15min":0.45800000000000002},"min":{"1min":0.29899999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.60099999999999998,"5min":0.65900000000000003,"15min":0.89400000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.497,"15min":0.52200000000000002},"min":{"1min":0.33200000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70199999999999996,"5min":0.82199999999999995,"15min":0.89200000000000002},"last":0.45700000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51600000000000001,"15min":0.52800000000000002},"min":{"1min":0.39100000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.69299999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.435},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.54600000000000004,"15min":0.56399999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63700000000000001,"5min":0.83699999999999997,"15min":0.86199999999999999},"last":0.65900000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.58999999999999997,"15min":0.61899999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81499999999999995,"5min":0.86699999999999999,"15min":0.94499999999999995},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.52800000000000002,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.84299999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.59299999999999997}]},{"osd":4,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.754,"15min":0.78600000000000003},"min":{"1min":0.50700000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.95299999999999996,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.751},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.755,"15min":0.79900000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95399999999999996,"5min":1.2549999999999999,"15min":1.2569999999999999},"last":0.86599999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.752,"15min":0.79300000000000004},"min":{"1min":0.46200000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93300000000000005,"5min":1.236,"15min":1.2609999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78900000000000003,"15min":0.81100000000000005},"min":{"1min":0.55600000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.93400000000000005,"5min":1.353,"15min":1.353},"last":0.82899999999999996}]},{"osd":6,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.77400000000000002,"15min":0.80400000000000005},"min":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.505},"max":{"1min":0.997,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.79800000000000004,"15min":0.83199999999999996},"min":{"1min":0.57499999999999996,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.94499999999999995,"5min":1.3240000000000001,"15min":1.3500000000000001},"last":0.76500000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.83099999999999996,"15min":0.84599999999999997},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.56499999999999995},"max":{"1min":0.94099999999999995,"5min":1.399,"15min":1.399},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.80400000000000005,"15min":0.84399999999999997},"min":{"1min":0.52600000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0449999999999999,"5min":1.365,"15min":1.365},"last":0.81399999999999995}]}]},{"osd":0,"up_from":14,"seq":60129543301,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65152,"kb_used_data":3768,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94306688,"statfs":{"total":96636764160,"available":96570048512,"internally_reserved":0,"allocated":3858432,"data_stored":2342201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41599999999999998,"15min":0.44900000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.55100000000000005,"5min":0.69299999999999995,"15min":1.216},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.47599999999999998,"15min":0.51100000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.629,"5min":0.77000000000000002,"15min":1.4590000000000001},"last":0.59899999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.48099999999999998,"15min":0.52100000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.66500000000000004,"5min":0.77000000000000002,"15min":1.1699999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51500000000000001,"15min":0.54300000000000004},"min":{"1min":0.372,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69199999999999995,"5min":0.85699999999999998,"15min":0.96799999999999997},"last":0.63700000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.50600000000000001,"15min":0.54100000000000004},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70099999999999996,"5min":0.81599999999999995,"15min":1.2270000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.58199999999999996,"15min":0.63200000000000001},"min":{"1min":0.441,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.71699999999999997,"5min":0.89700000000000002,"15min":1.032},"last":0.73099999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76100000000000001,"15min":0.80000000000000004},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.496},"max":{"1min":1.052,"5min":1.1200000000000001,"15min":1.381},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.77100000000000002,"15min":0.80600000000000005},"min":{"1min":0.61399999999999999,"5min":0.56399999999999995,"15min":0.47799999999999998},"max":{"1min":1.0860000000000001,"5min":1.137,"15min":1.2849999999999999},"last":0.95199999999999996}]},{"osd":5,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76900000000000002,"15min":0.80900000000000005},"min":{"1min":0.58399999999999996,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.117,"5min":1.1180000000000001,"15min":1.3420000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78100000000000003,"15min":0.81000000000000005},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.996,"5min":1.1259999999999999,"15min":1.288},"last":1.0049999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.79600000000000004,"15min":0.82999999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.51300000000000001},"max":{"1min":1.1499999999999999,"5min":1.1519999999999999,"15min":1.4019999999999999},"last":1.0960000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.77600000000000002,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.0009999999999999,"5min":1.1719999999999999,"15min":1.2210000000000001},"last":1.028}]},{"osd":7,"last update":"Tue Dec 19 18:18:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.80700000000000005,"15min":0.83699999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.169,"5min":1.1899999999999999,"15min":1.415},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.80200000000000005,"15min":0.84299999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.115,"5min":1.115,"15min":1.395},"last":1.1479999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-19T18:19:14.885 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-12-19T18:19:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcf482d640 1 Processor -- start 2023-12-19T18:19:15.027 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcf482d640 1 -- start start 2023-12-19T18:19:15.028 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcf482d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbcf0159820 0x7fbcf015df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcf482d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcf006f6c0 0x7fbcf00c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcf482d640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf006fb60 con 0x7fbcf0159c00 2023-12-19T18:19:15.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcf482d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf015e470 con 0x7fbcf0159820 2023-12-19T18:19:15.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcf482d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf006fe40 con 0x7fbcf006f6c0 2023-12-19T18:19:15.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbceed9d640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbcf0159c00 0x7fbcf0070c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49634/0 (socket says 172.21.15.26:49634) 2023-12-19T18:19:15.029 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbceed9d640 1 -- 172.21.15.26:0/2216675473 learned_addr learned my addr 172.21.15.26:0/2216675473 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:19:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcedd9b640 1 --2- 172.21.15.26:0/2216675473 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcf006f6c0 0x7fbcf00c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcee59c640 1 --2- 172.21.15.26:0/2216675473 >> v2:172.21.15.160:3300/0 conn(0x7fbcf0159820 0x7fbcf015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133095203 0 0) 0x7fbcf006fb60 con 0x7fbcf0159c00 2023-12-19T18:19:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcd8003530 con 0x7fbcf0159c00 2023-12-19T18:19:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2194394377 0 0) 0x7fbcd8003530 con 0x7fbcf0159c00 2023-12-19T18:19:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcf006f6c0 msgr2=0x7fbcf00c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 --2- 172.21.15.26:0/2216675473 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcf006f6c0 0x7fbcf00c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 >> v2:172.21.15.160:3300/0 conn(0x7fbcf0159820 msgr2=0x7fbcf015df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 --2- 172.21.15.26:0/2216675473 >> v2:172.21.15.160:3300/0 conn(0x7fbcf0159820 0x7fbcf015df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf0070120 con 0x7fbcf0159c00 2023-12-19T18:19:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7fbce8001a00 con 0x7fbcf0159c00 2023-12-19T18:19:15.031 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 >> v1:172.21.15.26:6789/0 conn(0x7fbcf0159c00 legacy=0x7fbcf0070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 --2- 172.21.15.26:0/2216675473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcd8003980 0x7fbcd8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 --2- 172.21.15.26:0/2216675473 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcf006f6c0 0x7fbcd80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 --2- 172.21.15.26:0/2216675473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcd8004d10 0x7fbcd8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf015e470 con 0x7fbcd8003980 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf006fe40 con 0x7fbcd8004d10 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd800b990 con 0x7fbcf006f6c0 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcee59c640 1 --2- 172.21.15.26:0/2216675473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcd8003980 0x7fbcd8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcedd9b640 1 --2- 172.21.15.26:0/2216675473 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcf006f6c0 0x7fbcd80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbceed9d640 1 --2- 172.21.15.26:0/2216675473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcd8004d10 0x7fbcd8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcee59c640 1 -- 172.21.15.26:0/2216675473 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcf006f6c0 msgr2=0x7fbcd80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcee59c640 1 --2- 172.21.15.26:0/2216675473 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcf006f6c0 0x7fbcd80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcee59c640 1 -- 172.21.15.26:0/2216675473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcd8004d10 msgr2=0x7fbcd8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcee59c640 1 --2- 172.21.15.26:0/2216675473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcd8004d10 0x7fbcd8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbcee59c640 1 -- 172.21.15.26:0/2216675473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd8003530 con 0x7fbcd8003980 2023-12-19T18:19:15.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.027+0000 7fbceed9d640 1 --2- 172.21.15.26:0/2216675473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcd8004d10 0x7fbcd8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:19:15.033 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.031+0000 7fbcee59c640 1 --2- 172.21.15.26:0/2216675473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcd8003980 0x7fbcd8003da0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009ae0 tx=0x7fbce001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d24da2448481b99 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.031+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce00a5630 con 0x7fbcd8003980 2023-12-19T18:19:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.031+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce00a5790 con 0x7fbcd8003980 2023-12-19T18:19:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.031+0000 7fbced59a640 1 -- 172.21.15.26:0/2216675473 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce00a5b40 con 0x7fbcd8003980 2023-12-19T18:19:15.034 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.031+0000 7fbcf482d640 1 -- 172.21.15.26:0/2216675473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcd8003980 msgr2=0x7fbcd8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.031+0000 7fbcf482d640 1 --2- 172.21.15.26:0/2216675473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcd8003980 0x7fbcd8003da0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009ae0 tx=0x7fbce001ca60 comp rx=0 tx=0).stop 2023-12-19T18:19:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.031+0000 7fbceed9d640 1 -- 172.21.15.26:0/2216675473 reap_dead start 2023-12-19T18:19:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.031+0000 7fbcf482d640 1 -- 172.21.15.26:0/2216675473 shutdown_connections 2023-12-19T18:19:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.031+0000 7fbcf482d640 1 -- 172.21.15.26:0/2216675473 >> 172.21.15.26:0/2216675473 conn(0x7fbcf00ce730 msgr2=0x7fbcf00ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.031+0000 7fbcf482d640 1 -- 172.21.15.26:0/2216675473 shutdown_connections 2023-12-19T18:19:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.031+0000 7fbcf482d640 1 -- 172.21.15.26:0/2216675473 wait complete. 2023-12-19T18:19:15.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcf482d640 1 Processor -- start 2023-12-19T18:19:15.036 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcf482d640 1 -- start start 2023-12-19T18:19:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcf482d640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcd8003980 0x7fbcf01551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcf482d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcf01648d0 0x7fbcf01525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcf482d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf014a8b0 0x7fbcf014ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcf482d640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf014b270 con 0x7fbcf014a8b0 2023-12-19T18:19:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcf482d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf014c0d0 con 0x7fbcf01648d0 2023-12-19T18:19:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcf482d640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf014cc80 con 0x7fbcd8003980 2023-12-19T18:19:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbceed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf014a8b0 0x7fbcf014ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbceed9d640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf014a8b0 0x7fbcf014ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46312/0 (socket says 172.21.15.26:46312) 2023-12-19T18:19:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbceed9d640 1 -- 172.21.15.26:0/954257445 learned_addr learned my addr 172.21.15.26:0/954257445 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:19:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcedd9b640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcf01648d0 0x7fbcf01525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcee59c640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcd8003980 0x7fbcf01551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbceed9d640 1 -- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcd8003980 msgr2=0x7fbcf01551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbceed9d640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcd8003980 0x7fbcf01551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbceed9d640 1 -- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcf01648d0 msgr2=0x7fbcf01525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbceed9d640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcf01648d0 0x7fbcf01525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbceed9d640 1 -- 172.21.15.26:0/954257445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf006b880 con 0x7fbcf014a8b0 2023-12-19T18:19:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcedd9b640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcf01648d0 0x7fbcf01525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:19:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcee59c640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcd8003980 0x7fbcf01551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:19:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbceed9d640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf014a8b0 0x7fbcf014ad30 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7fbcf006f250 tx=0x7fbce800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcdeffd640 1 -- 172.21.15.26:0/954257445 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce800eb90 con 0x7fbcf014a8b0 2023-12-19T18:19:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcf482d640 1 -- 172.21.15.26:0/954257445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf014b4c0 con 0x7fbcf014a8b0 2023-12-19T18:19:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcdeffd640 1 -- 172.21.15.26:0/954257445 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce800ed10 con 0x7fbcf014a8b0 2023-12-19T18:19:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcdeffd640 1 -- 172.21.15.26:0/954257445 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce8013660 con 0x7fbcf014a8b0 2023-12-19T18:19:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.035+0000 7fbcf482d640 1 -- 172.21.15.26:0/954257445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf014ba80 con 0x7fbcf014a8b0 2023-12-19T18:19:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.039+0000 7fbcf482d640 1 -- 172.21.15.26:0/954257445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf0164cb0 con 0x7fbcf014a8b0 2023-12-19T18:19:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.039+0000 7fbcdeffd640 1 -- 172.21.15.26:0/954257445 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbce80137c0 con 0x7fbcf014a8b0 2023-12-19T18:19:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.039+0000 7fbcdeffd640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbcd0065b80 0x7fbcd0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.039+0000 7fbcdeffd640 1 -- 172.21.15.26:0/954257445 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7fbce8083b10 con 0x7fbcf014a8b0 2023-12-19T18:19:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.039+0000 7fbcee59c640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbcd0065b80 0x7fbcd0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.039+0000 7fbcee59c640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbcd0065b80 0x7fbcd0068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbce00af000 tx=0x7fbce00046d0 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:15.045 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.043+0000 7fbcdeffd640 1 -- 172.21.15.26:0/954257445 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbce8050ff0 con 0x7fbcf014a8b0 2023-12-19T18:19:15.204 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.203+0000 7fbcf482d640 1 -- 172.21.15.26:0/954257445 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fbcf0164cb0 con 0x7fbcf014a8b0 2023-12-19T18:19:15.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.207+0000 7fbcdeffd640 1 -- 172.21.15.26:0/954257445 <== mon.0 v2:172.21.15.26:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbce8050ff0 con 0x7fbcf014a8b0 2023-12-19T18:19:15.209 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.207+0000 7fbcdeffd640 1 -- 172.21.15.26:0/954257445 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v88)=0 v88) v1 ==== 123+0+0 (secure 0 0 0) 0x7fbce8054610 con 0x7fbcf014a8b0 2023-12-19T18:19:15.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.211+0000 7fbcf482d640 1 -- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbcd0065b80 msgr2=0x7fbcd0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:15.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.211+0000 7fbcf482d640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbcd0065b80 0x7fbcd0068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbce00af000 tx=0x7fbce00046d0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.211+0000 7fbcf482d640 1 -- 172.21.15.26:0/954257445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf014a8b0 msgr2=0x7fbcf014ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:15.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.211+0000 7fbcf482d640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf014a8b0 0x7fbcf014ad30 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7fbcf006f250 tx=0x7fbce800e530 comp rx=0 tx=0).stop 2023-12-19T18:19:15.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.211+0000 7fbcf482d640 1 -- 172.21.15.26:0/954257445 shutdown_connections 2023-12-19T18:19:15.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.211+0000 7fbcf482d640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7fbcd0065b80 0x7fbcd0068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.212 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.211+0000 7fbcf482d640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf014a8b0 0x7fbcf014ad30 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.211+0000 7fbcf482d640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcf01648d0 0x7fbcf01525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.211+0000 7fbcf482d640 1 --2- 172.21.15.26:0/954257445 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7fbcd8003980 0x7fbcf01551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.211+0000 7fbcf482d640 1 -- 172.21.15.26:0/954257445 >> 172.21.15.26:0/954257445 conn(0x7fbcf00ce730 msgr2=0x7fbcf01557d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:15.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.211+0000 7fbcf482d640 1 -- 172.21.15.26:0/954257445 shutdown_connections 2023-12-19T18:19:15.213 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.211+0000 7fbcf482d640 1 -- 172.21.15.26:0/954257445 wait complete. 2023-12-19T18:19:15.231 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:19:15.232 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:19:15.375 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d8c3640 1 Processor -- start 2023-12-19T18:19:15.376 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d8c3640 1 -- start start 2023-12-19T18:19:15.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d8c3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f317815cca0 0x7f31780b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d8c3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31780b6b20 0x7f31780d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d8c3640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31780b6fb0 con 0x7f31780b8760 2023-12-19T18:19:15.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d8c3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31780b7c30 con 0x7f317815cca0 2023-12-19T18:19:15.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d8c3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31780b7290 con 0x7f31780b6b20 2023-12-19T18:19:15.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f3177fff640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f31780b8760 0x7f31780cd670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49650/0 (socket says 172.21.15.26:49650) 2023-12-19T18:19:15.377 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f3177fff640 1 -- 172.21.15.26:0/2505165324 learned_addr learned my addr 172.21.15.26:0/2505165324 (peer_addr_for_me v1:172.21.15.26:0/0) 2023-12-19T18:19:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d0c2640 1 --2- 172.21.15.26:0/2505165324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31780b6b20 0x7f31780d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317c8c1640 1 --2- 172.21.15.26:0/2505165324 >> v2:172.21.15.160:3300/0 conn(0x7f317815cca0 0x7f31780b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181369836 0 0) 0x7f31780b6fb0 con 0x7f31780b8760 2023-12-19T18:19:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3160003530 con 0x7f31780b8760 2023-12-19T18:19:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 318922587 0 0) 0x7f3160003530 con 0x7f31780b8760 2023-12-19T18:19:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31780b6b20 msgr2=0x7f31780d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 --2- 172.21.15.26:0/2505165324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31780b6b20 0x7f31780d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 >> v2:172.21.15.160:3300/0 conn(0x7f317815cca0 msgr2=0x7f31780b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 --2- 172.21.15.26:0/2505165324 >> v2:172.21.15.160:3300/0 conn(0x7f317815cca0 0x7f31780b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31780b7570 con 0x7f31780b8760 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3184199896 0 0) 0x7f3168001680 con 0x7f31780b8760 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 >> v1:172.21.15.26:6789/0 conn(0x7f31780b8760 legacy=0x7f31780cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 --2- 172.21.15.26:0/2505165324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31600039f0 0x7f3160003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 --2- 172.21.15.26:0/2505165324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31600047d0 0x7f3160004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 --2- 172.21.15.26:0/2505165324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3160005150 0x7f316000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31780b7c30 con 0x7f3160005150 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31780b7290 con 0x7f31600039f0 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316000bed0 con 0x7f31600047d0 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d0c2640 1 --2- 172.21.15.26:0/2505165324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3160005150 0x7f316000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317c8c1640 1 --2- 172.21.15.26:0/2505165324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31600039f0 0x7f3160003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f3177fff640 1 --2- 172.21.15.26:0/2505165324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31600047d0 0x7f3160004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d0c2640 1 -- 172.21.15.26:0/2505165324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31600047d0 msgr2=0x7f3160004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d0c2640 1 --2- 172.21.15.26:0/2505165324 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31600047d0 0x7f3160004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d0c2640 1 -- 172.21.15.26:0/2505165324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31600039f0 msgr2=0x7f3160003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:15.380 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d0c2640 1 --2- 172.21.15.26:0/2505165324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31600039f0 0x7f3160003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317d0c2640 1 -- 172.21.15.26:0/2505165324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3160003530 con 0x7f3160005150 2023-12-19T18:19:15.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.375+0000 7f317c8c1640 1 --2- 172.21.15.26:0/2505165324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31600039f0 0x7f3160003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:19:15.381 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.379+0000 7f317d0c2640 1 --2- 172.21.15.26:0/2505165324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3160005150 0x7f316000b990 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f3170009fc0 tx=0x7f317001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93bd32d005b38336 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.379+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31700a5c70 con 0x7f3160005150 2023-12-19T18:19:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.379+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f317000f040 con 0x7f3160005150 2023-12-19T18:19:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.379+0000 7f31777fe640 1 -- 172.21.15.26:0/2505165324 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31700ad440 con 0x7f3160005150 2023-12-19T18:19:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.379+0000 7f317d8c3640 1 -- 172.21.15.26:0/2505165324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3160005150 msgr2=0x7f316000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.379+0000 7f317d8c3640 1 --2- 172.21.15.26:0/2505165324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3160005150 0x7f316000b990 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f3170009fc0 tx=0x7f317001cc60 comp rx=0 tx=0).stop 2023-12-19T18:19:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.379+0000 7f317d0c2640 1 -- 172.21.15.26:0/2505165324 reap_dead start 2023-12-19T18:19:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.379+0000 7f317d8c3640 1 -- 172.21.15.26:0/2505165324 shutdown_connections 2023-12-19T18:19:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.379+0000 7f317d8c3640 1 -- 172.21.15.26:0/2505165324 >> 172.21.15.26:0/2505165324 conn(0x7f317806a120 msgr2=0x7f317806a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.379+0000 7f317d8c3640 1 -- 172.21.15.26:0/2505165324 shutdown_connections 2023-12-19T18:19:15.383 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.379+0000 7f317d8c3640 1 -- 172.21.15.26:0/2505165324 wait complete. 2023-12-19T18:19:15.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317d8c3640 1 Processor -- start 2023-12-19T18:19:15.384 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317d8c3640 1 -- start start 2023-12-19T18:19:15.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317d8c3640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3160005150 0x7f3178159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317d8c3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31781e6590 0x7f31781e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317d8c3640 1 --2- >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31781f5e60 0x7f31781f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317d8c3640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31781f6850 con 0x7f3160005150 2023-12-19T18:19:15.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317d8c3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31781e6f30 con 0x7f31781e6590 2023-12-19T18:19:15.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317d8c3640 1 -- --> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3178159d10 con 0x7f31781f5e60 2023-12-19T18:19:15.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317c8c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3160005150 0x7f3178159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.387 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317c8c1640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3160005150 0x7f3178159430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46328/0 (socket says 172.21.15.26:46328) 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317c8c1640 1 -- 172.21.15.26:0/1081488985 learned_addr learned my addr 172.21.15.26:0/1081488985 (peer_addr_for_me v2:172.21.15.26:0/0) 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317d0c2640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31781f5e60 0x7f31781f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f3177fff640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31781e6590 0x7f31781e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317c8c1640 1 -- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31781f5e60 msgr2=0x7f31781f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317c8c1640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31781f5e60 0x7f31781f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317c8c1640 1 -- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31781e6590 msgr2=0x7f31781e69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317c8c1640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31781e6590 0x7f31781e69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317c8c1640 1 -- 172.21.15.26:0/1081488985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3178159ff0 con 0x7f3160005150 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317c8c1640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3160005150 0x7f3178159430 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f316c009fd0 tx=0x7f316c011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f31757fa640 1 -- 172.21.15.26:0/1081488985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c004220 con 0x7f3160005150 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f31757fa640 1 -- 172.21.15.26:0/1081488985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f316c004380 con 0x7f3160005150 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f31757fa640 1 -- 172.21.15.26:0/1081488985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c00fa00 con 0x7f3160005150 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317d8c3640 1 -- 172.21.15.26:0/1081488985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317815a2d0 con 0x7f3160005150 2023-12-19T18:19:15.388 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.383+0000 7f317d8c3640 1 -- 172.21.15.26:0/1081488985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317815a5b0 con 0x7f3160005150 2023-12-19T18:19:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.387+0000 7f317d8c3640 1 -- 172.21.15.26:0/1081488985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3178067400 con 0x7f3160005150 2023-12-19T18:19:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.387+0000 7f31757fa640 1 -- 172.21.15.26:0/1081488985 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 145) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f316c00fb60 con 0x7f3160005150 2023-12-19T18:19:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.387+0000 7f31757fa640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3164065b80 0x7f3164068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.387+0000 7f31757fa640 1 -- 172.21.15.26:0/1081488985 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(259..259 src has 1..259) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f316c083bf0 con 0x7f3160005150 2023-12-19T18:19:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.387+0000 7f3177fff640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3164065b80 0x7f3164068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:15.390 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.387+0000 7f3177fff640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3164065b80 0x7f3164068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3168001ec0 tx=0x7f3168009040 comp rx=0 tx=0).ready entity=mgr.11683 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:15.393 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.391+0000 7f31757fa640 1 -- 172.21.15.26:0/1081488985 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f316c0511e0 con 0x7f3160005150 2023-12-19T18:19:15.584 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f317d8c3640 1 -- 172.21.15.26:0/1081488985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31780b7820 con 0x7f3160005150 2023-12-19T18:19:15.585 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f31757fa640 1 -- 172.21.15.26:0/1081488985 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+3541 (secure 0 0 0) 0x7f316c022020 con 0x7f3160005150 2023-12-19T18:19:15.585 INFO:teuthology.orchestra.run.smithi026.stdout: 2023-12-19T18:19:15.585 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":130,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/282680544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":282680544},{"type":"v1","addr":"172.21.15.160:6836","nonce":282680544}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/3302634175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":3302634175},{"type":"v1","addr":"172.21.15.160:6837","nonce":3302634175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3434994961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3434994961},{"type":"v1","addr":"172.21.15.26:6834","nonce":3434994961}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12609,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4289797511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4289797511},{"type":"v1","addr":"172.21.15.26:6835","nonce":4289797511}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[]} 2023-12-19T18:19:15.585 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 130 2023-12-19T18:19:15.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f317d8c3640 1 -- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3164065b80 msgr2=0x7f3164068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:15.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f317d8c3640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3164065b80 0x7f3164068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3168001ec0 tx=0x7f3168009040 comp rx=0 tx=0).stop 2023-12-19T18:19:15.587 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f317d8c3640 1 -- 172.21.15.26:0/1081488985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3160005150 msgr2=0x7f3178159430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:15.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f317d8c3640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3160005150 0x7f3178159430 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7f316c009fd0 tx=0x7f316c011b30 comp rx=0 tx=0).stop 2023-12-19T18:19:15.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f317d8c3640 1 -- 172.21.15.26:0/1081488985 shutdown_connections 2023-12-19T18:19:15.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f317d8c3640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:6832/48394,v1:172.21.15.160:6833/48394] conn(0x7f3164065b80 0x7f3164068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f317d8c3640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:3301/0,v1:172.21.15.160:6790/0] conn(0x7f31781f5e60 0x7f31781f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.588 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f317d8c3640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f31781e6590 0x7f31781e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f317d8c3640 1 --2- 172.21.15.26:0/1081488985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3160005150 0x7f3178159430 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:15.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f317d8c3640 1 -- 172.21.15.26:0/1081488985 >> 172.21.15.26:0/1081488985 conn(0x7f317806a120 msgr2=0x7f317815bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:15.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f317d8c3640 1 -- 172.21.15.26:0/1081488985 shutdown_connections 2023-12-19T18:19:15.589 INFO:teuthology.orchestra.run.smithi026.stderr:2023-12-19T18:19:15.583+0000 7f317d8c3640 1 -- 172.21.15.26:0/1081488985 wait complete. 2023-12-19T18:19:15.606 DEBUG:tasks.cephfs.filesystem:already dead... 2023-12-19T18:19:15.607 INFO:teuthology.misc:Shutting down mds daemons... 2023-12-19T18:19:15.607 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:19:15.607 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:15.618 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:19:15.618 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:19:15.619 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:15.670 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:19:15.671 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:19:15.671 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:15.686 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:19:15.686 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:19:15.686 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:15.742 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:19:15.743 INFO:teuthology.misc:Shutting down osd daemons... 2023-12-19T18:19:15.743 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-12-19T18:19:15.743 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:15.822 INFO:tasks.ceph.osd.0:Stopped 2023-12-19T18:19:15.822 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-12-19T18:19:15.822 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:19.640 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-12-19T18:19:21.924 INFO:tasks.ceph.osd.1:Stopped 2023-12-19T18:19:21.925 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-12-19T18:19:21.925 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:21.987 INFO:tasks.ceph.osd.2:Stopped 2023-12-19T18:19:21.988 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-12-19T18:19:21.988 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:25.548 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-12-19T18:19:25.548 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-12-19T18:19:28.089 INFO:tasks.ceph.osd.3:Stopped 2023-12-19T18:19:28.090 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-12-19T18:19:28.090 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:28.181 INFO:tasks.ceph.osd.4:Stopped 2023-12-19T18:19:28.181 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-12-19T18:19:28.181 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:28.263 INFO:tasks.ceph.osd.5:Stopped 2023-12-19T18:19:28.263 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-12-19T18:19:28.263 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:31.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-12-19T18:19:31.154 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-12-19T18:19:34.365 INFO:tasks.ceph.osd.6:Stopped 2023-12-19T18:19:34.365 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-12-19T18:19:34.365 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:34.447 INFO:tasks.ceph.osd.7:Stopped 2023-12-19T18:19:34.447 INFO:teuthology.misc:Shutting down mgr daemons... 2023-12-19T18:19:34.447 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-12-19T18:19:34.447 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:34.508 INFO:tasks.ceph.mgr.y:Stopped 2023-12-19T18:19:34.508 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-12-19T18:19:34.508 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:34.571 INFO:tasks.ceph.mgr.x:Stopped 2023-12-19T18:19:34.571 INFO:teuthology.misc:Shutting down mon daemons... 2023-12-19T18:19:34.571 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-12-19T18:19:34.571 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:34.604 INFO:tasks.ceph.mon.a:Stopped 2023-12-19T18:19:34.604 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-19T18:19:34.604 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:34.627 INFO:tasks.ceph.mon.b:Stopped 2023-12-19T18:19:34.627 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-19T18:19:34.627 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:34.694 INFO:tasks.ceph.mon.c:Stopped 2023-12-19T18:19:34.695 INFO:tasks.ceph:Checking cluster log for badness... 2023-12-19T18:19:34.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-19T18:19:34.727 INFO:teuthology.orchestra.run.smithi026.stdout:2023-12-19T16:56:14.639107+0000 mon.a (mon.0) 499 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-12-19T18:19:34.728 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-12-19T18:19:34.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-19T18:19:34.806 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-19T18:19:34.882 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-19T18:19:34.961 INFO:teuthology.orchestra.run.smithi026.stdout:2023-12-19T16:56:14.639107+0000 mon.a (mon.0) 499 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-12-19T18:19:34.962 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T18:19:34.962 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-12-19T18:19:35.356 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T18:19:35.356 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-12-19T18:19:35.579 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T18:19:35.579 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-12-19T18:19:35.761 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T18:19:35.761 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-12-19T18:19:35.961 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi160.front.sepia.ceph.com 2023-12-19T18:19:35.961 DEBUG:teuthology.orchestra.run.smithi160:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-12-19T18:19:36.155 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-12-19T18:19:36.221 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi160.front.sepia.ceph.com 2023-12-19T18:19:36.222 DEBUG:teuthology.orchestra.run.smithi160:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-12-19T18:19:36.381 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi160.front.sepia.ceph.com 2023-12-19T18:19:36.381 DEBUG:teuthology.orchestra.run.smithi160:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-12-19T18:19:36.523 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi160.front.sepia.ceph.com 2023-12-19T18:19:36.524 DEBUG:teuthology.orchestra.run.smithi160:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-12-19T18:19:36.673 INFO:tasks.ceph:Archiving mon data... 2023-12-19T18:19:36.674 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112/data/mon.a.tgz 2023-12-19T18:19:36.674 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2023-12-19T18:19:36.684 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.InjEY2uOE2 2023-12-19T18:19:36.685 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.InjEY2uOE2 2023-12-19T18:19:37.313 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.InjEY2uOE2 2023-12-19T18:19:37.388 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.InjEY2uOE2 is 3MB 2023-12-19T18:19:37.508 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.InjEY2uOE2 2023-12-19T18:19:37.518 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112/data/mon.b.tgz 2023-12-19T18:19:37.518 DEBUG:teuthology.orchestra.run.smithi160:> mktemp 2023-12-19T18:19:37.528 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.oqXsGMD68L 2023-12-19T18:19:37.528 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.oqXsGMD68L 2023-12-19T18:19:38.128 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.oqXsGMD68L 2023-12-19T18:19:38.203 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.oqXsGMD68L is 4MB 2023-12-19T18:19:38.328 DEBUG:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.oqXsGMD68L 2023-12-19T18:19:38.338 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112/data/mon.c.tgz 2023-12-19T18:19:38.338 DEBUG:teuthology.orchestra.run.smithi160:> mktemp 2023-12-19T18:19:38.391 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.vSgAToR0gE 2023-12-19T18:19:38.392 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.vSgAToR0gE 2023-12-19T18:19:38.993 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.vSgAToR0gE 2023-12-19T18:19:39.068 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.vSgAToR0gE is 4MB 2023-12-19T18:19:39.181 DEBUG:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.vSgAToR0gE 2023-12-19T18:19:39.192 INFO:tasks.ceph:Cleaning ceph cluster... 2023-12-19T18:19:39.192 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-19T18:19:39.196 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-19T18:19:39.259 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-12-19T18:19:39.259 DEBUG:teuthology.orchestra.run.smithi026:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-12-19T18:19:39.266 DEBUG:teuthology.orchestra.run.smithi160:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-12-19T18:19:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-12-19T18:19:39.340 INFO:teuthology.orchestra.run.smithi160.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-12-19T18:19:39.347 INFO:tasks.ceph:Archiving crash dumps... 2023-12-19T18:19:39.349 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112/remote/smithi026/crash 2023-12-19T18:19:39.350 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-19T18:19:39.373 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112/remote/smithi160/crash 2023-12-19T18:19:39.373 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-19T18:19:39.410 INFO:tasks.ceph:Shutting down logrotate 2023-12-19T18:19:39.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:19:39.424 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:19:39.501 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-19T18:19:39.524 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-19T18:19:39.544 INFO:tasks.ceph:Compressing logs... 2023-12-19T18:19:39.545 DEBUG:teuthology.orchestra.run.smithi026:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-19T18:19:39.570 DEBUG:teuthology.orchestra.run.smithi160:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-19T18:19:59.825 INFO:tasks.ceph:Archiving logs... 2023-12-19T18:19:59.826 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112/remote/smithi026/log 2023-12-19T18:19:59.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-19T18:20:02.406 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112/remote/smithi160/log 2023-12-19T18:20:02.407 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-19T18:20:04.048 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-19T18:20:04.058 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-19T18:20:04.058 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-19T18:20:04.062 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-19T18:20:04.184 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-19T18:20:04.194 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-19T18:20:04.194 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-19T18:20:04.198 DEBUG:teuthology.orchestra.run.smithi160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-19T18:20:04.214 INFO:teuthology.orchestra.run.smithi026.stdout: remote refid st t when poll reach delay offset jitter 2023-12-19T18:20:04.215 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================== 2023-12-19T18:20:04.215 INFO:teuthology.orchestra.run.smithi026.stdout:*hv01.front.sepi 44.4.53.2 2 u 119 128 377 0.166 -0.530 0.135 2023-12-19T18:20:04.215 INFO:teuthology.orchestra.run.smithi026.stdout:+hv02.front.sepi 96.245.170.99 3 u 52 128 377 0.179 -0.778 0.083 2023-12-19T18:20:04.215 INFO:teuthology.orchestra.run.smithi026.stdout:+hv03.front.sepi 74.6.168.72 3 u 32 128 377 0.157 -0.814 0.205 2023-12-19T18:20:04.215 INFO:teuthology.orchestra.run.smithi026.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2023-12-19T18:20:04.218 INFO:teuthology.orchestra.run.smithi160.stdout: remote refid st t when poll reach delay offset jitter 2023-12-19T18:20:04.218 INFO:teuthology.orchestra.run.smithi160.stdout:============================================================================== 2023-12-19T18:20:04.218 INFO:teuthology.orchestra.run.smithi160.stdout:*hv01.front.sepi 44.4.53.2 2 u 93 128 377 0.170 -0.314 0.124 2023-12-19T18:20:04.218 INFO:teuthology.orchestra.run.smithi160.stdout:+hv02.front.sepi 96.245.170.99 3 u 105 128 377 0.192 -0.773 0.126 2023-12-19T18:20:04.218 INFO:teuthology.orchestra.run.smithi160.stdout:+hv03.front.sepi 74.6.168.72 3 u 100 128 377 0.168 -0.773 0.246 2023-12-19T18:20:04.218 INFO:teuthology.orchestra.run.smithi160.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2023-12-19T18:20:04.219 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-19T18:20:04.228 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-19T18:20:04.228 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-19T18:20:04.237 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-19T18:20:04.246 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-19T18:20:04.254 INFO:teuthology.task.internal:Duration was 6620.143943 seconds 2023-12-19T18:20:04.254 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-19T18:20:04.263 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-19T18:20:04.263 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-19T18:20:04.267 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-19T18:20:04.365 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-19T18:20:04.366 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2023-12-19T18:20:04.366 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-19T18:20:04.389 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi160.front.sepia.ceph.com 2023-12-19T18:20:04.389 DEBUG:teuthology.orchestra.run.smithi160:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-19T18:20:04.412 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-19T18:20:04.413 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-19T18:20:04.438 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-19T18:20:04.495 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-19T18:20:04.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-19T18:20:04.522 DEBUG:teuthology.orchestra.run.smithi160:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-19T18:20:04.617 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-19T18:20:04.628 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-19T18:20:04.628 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-19T18:20:04.654 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-19T18:20:04.675 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-19T18:20:04.685 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-19T18:20:04.697 DEBUG:teuthology.orchestra.run.smithi160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-19T18:20:04.713 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2023-12-19T18:20:04.737 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = core 2023-12-19T18:20:04.756 DEBUG:teuthology.orchestra.run.smithi026:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-19T18:20:04.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:20:04.786 DEBUG:teuthology.orchestra.run.smithi160:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-19T18:20:04.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:20:04.807 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-19T18:20:04.818 INFO:teuthology.task.internal:Transferring archived files... 2023-12-19T18:20:04.819 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112/remote/smithi026 2023-12-19T18:20:04.819 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-19T18:20:04.857 DEBUG:teuthology.misc:Transferring archived files from smithi160:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112/remote/smithi160 2023-12-19T18:20:04.858 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-19T18:20:04.883 INFO:teuthology.task.internal:Removing archive directory... 2023-12-19T18:20:04.883 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-19T18:20:04.898 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-19T18:20:04.931 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-19T18:20:04.943 INFO:teuthology.task.internal:Not uploading archives. 2023-12-19T18:20:04.943 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-19T18:20:04.953 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-19T18:20:04.953 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-19T18:20:04.957 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-19T18:20:04.962 INFO:teuthology.orchestra.run.smithi026.stdout: 658058 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 19 18:20 /home/ubuntu/cephtest 2023-12-19T18:20:04.983 INFO:teuthology.orchestra.run.smithi160.stdout: 658057 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 19 18:20 /home/ubuntu/cephtest 2023-12-19T18:20:04.985 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-19T18:20:04.996 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-19T18:20:05.020 INFO:teuthology.nuke:Checking targets against current locks 2023-12-19T18:20:05.054 DEBUG:teuthology.nuke:shortname: smithi026 2023-12-19T18:20:05.055 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-19T18:20:05.097 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 16:19:32.316466', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEThGRU3mLKvXf/qg5JsL9om6klrL7d2pnzrm5Fi5uTyVqXIF4g8S5zX5tYXJppael5Gt/C+TDpmOvRWe3aZ47c='} 2023-12-19T18:20:05.110 DEBUG:teuthology.nuke:shortname: smithi160 2023-12-19T18:20:05.110 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-19T18:20:05.156 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497112', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 16:19:32.315056', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF897hQ5wuG30hBord1CoC4OmMeyPjDjjNM2QCswEqM6Iw6w8vkW9m7Db/dfhBMAlZZSpd+l6Ah3H5U7ILP4eCY='} 2023-12-19T18:20:05.209 INFO:teuthology.orchestra.console.smithi026:Power off 2023-12-19T18:20:05.209 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T18:20:05.250 INFO:teuthology.orchestra.console.smithi160:Power off 2023-12-19T18:20:05.250 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T18:20:05.261 DEBUG:teuthology.orchestra.console.smithi026:power off output: Chassis Power Control: Down/Off 2023-12-19T18:20:05.261 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:20:05.303 DEBUG:teuthology.orchestra.console.smithi160:power off output: Chassis Power Control: Down/Off 2023-12-19T18:20:05.304 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:20:05.317 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2023-12-19T18:20:05.359 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on 2023-12-19T18:20:09.318 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:20:09.360 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:20:09.430 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2023-12-19T18:20:09.472 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on 2023-12-19T18:20:13.433 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:20:13.472 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:20:13.545 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is off 2023-12-19T18:20:13.583 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off 2023-12-19T18:20:13.646 INFO:teuthology.orchestra.console.smithi026:Power off completed 2023-12-19T18:20:13.684 INFO:teuthology.orchestra.console.smithi160:Power off completed 2023-12-19T18:20:13.789 INFO:teuthology.run:Summary data: client.0-kernel-sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c client.1-kernel-sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6620.143942832947 failure_reason: '"2023-12-19T16:56:14.639107+0000 mon.a (mon.0) 499 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2023-12-19T18:20:13.790 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T18:20:13.900 INFO:teuthology.run:FAIL